2023-10-06T04:55:53.396 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T04:55:53.396 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318 --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/orig.config.yaml 2023-10-06T04:55:53.442 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:55:53.524 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} email: yweinste@redhat.com first_in_suite: false job_id: '7412318' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - but it is still running - slow request - MDS_CLIENT_LATE_RELEASE - t responding to mclientcaps sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsN2KlXCfhDPmYLfoVfF8ye4qHC8UXV+jzCMG27elNHskyYRXcyMSrNycVha+feEaRJ54JvvraDvWOgcUF7W2I= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGh2SIXiMe51UgQvKWo7LPq52jMpsA7kU3YeiWFADY9lDVWX3mJJIxTnL9dtfZo12gEyWHIUJoC+Wr3mferqT6U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_recovery teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T04:55:53.524 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T04:55:53.525 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T04:55:53.525 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T04:55:53.527 INFO:teuthology.task.internal:Checking packages... 2023-10-06T04:55:53.549 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T04:55:53.549 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T04:55:53.549 INFO:teuthology.packaging:ref: None 2023-10-06T04:55:53.549 INFO:teuthology.packaging:tag: None 2023-10-06T04:55:53.549 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:55:53.550 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:55:53.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:55:53.692 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T04:55:53.692 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T04:55:53.699 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T04:55:53.700 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T04:55:53.736 INFO:teuthology.task.internal:Saving configuration 2023-10-06T04:55:53.752 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T04:55:53.768 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:55:53.795 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:50:30.495883', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsN2KlXCfhDPmYLfoVfF8ye4qHC8UXV+jzCMG27elNHskyYRXcyMSrNycVha+feEaRJ54JvvraDvWOgcUF7W2I='} 2023-10-06T04:55:53.817 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:50:30.494548', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGh2SIXiMe51UgQvKWo7LPq52jMpsA7kU3YeiWFADY9lDVWX3mJJIxTnL9dtfZo12gEyWHIUJoC+Wr3mferqT6U='} 2023-10-06T04:55:53.817 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T04:55:53.823 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T04:55:53.823 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T04:55:53.824 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T04:55:53.904 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9f73cb7b80>, signals=[15]) 2023-10-06T04:55:53.904 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T04:55:53.911 INFO:teuthology.task.internal:Opening connections... 2023-10-06T04:55:53.911 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T04:55:53.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:55:53.982 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T04:55:53.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:55:54.050 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T04:55:54.056 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2023-10-06T04:55:54.075 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2023-10-06T04:55:54.076 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2023-10-06T04:55:54.132 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T04:55:54.133 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T04:55:54.133 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel" 2023-10-06T04:55:54.133 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora" 2023-10-06T04:55:54.133 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.4" 2023-10-06T04:55:54.133 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:55:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T04:55:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2023-10-06T04:55:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T04:55:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T04:55:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T04:55:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:55:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T04:55:54.135 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:55:54.135 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T04:55:54.135 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T04:55:54.135 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T04:55:54.136 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-10-06T04:55:54.155 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-10-06T04:55:54.172 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-10-06T04:55:54.172 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.4" 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:55:54.228 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T04:55:54.229 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T04:55:54.230 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-10-06T04:55:54.255 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T04:55:54.264 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T04:55:54.296 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T04:55:54.296 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:55:54.299 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:55:54.315 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T04:55:54.321 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T04:55:54.321 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:55:54.357 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:55:54.372 INFO:teuthology.orchestra.run.smithi033.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T04:55:54.373 INFO:teuthology.orchestra.run.smithi167.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T04:55:54.373 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T04:55:54.427 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T04:55:54.442 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T04:55:54.443 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-06T04:55:54.443 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-06T04:55:54.443 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:55:54.444 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:55:54.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:55:54.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T04:55:54.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:55:54.464 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T04:55:54.630 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:55:54.630 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T04:55:54.630 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-10-06T04:55:54.632 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:55:54.632 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T04:55:54.633 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2023-10-06T04:55:54.649 INFO:teuthology.orchestra.run.smithi167.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T04:55:54.649 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:55:54.649 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T04:55:54.650 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum upgrade -y linux-firmware 2023-10-06T04:55:54.651 INFO:teuthology.orchestra.run.smithi033.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T04:55:54.651 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:55:54.651 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T04:55:54.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum upgrade -y linux-firmware 2023-10-06T04:55:56.178 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T04:55:56.179 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2023-10-06T04:55:56.228 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2023-10-06T04:55:56.229 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2023-10-06T04:55:57.577 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-06T04:55:57.712 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-10-06T04:55:58.262 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-06T04:55:59.200 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-10-06T04:56:01.561 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 3.7 MB/s | 24 kB 00:00 2023-10-06T04:56:02.446 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-06T04:56:02.447 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-06T04:56:02.448 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-06T04:56:02.571 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T04:56:02.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:56:02.718 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T04:56:02.718 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T04:56:02.720 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi167.front.sepia.ceph.com to complete... 2023-10-06T04:56:02.756 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-06T04:56:03.712 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2023-10-06T04:56:03.713 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2023-10-06T04:56:03.714 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2023-10-06T04:56:03.822 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T04:56:03.822 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:56:04.037 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T04:56:04.038 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T04:56:04.040 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi033.front.sepia.ceph.com to complete... 2023-10-06T04:56:18.355 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06 04:56:18 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T04:56:18.355 INFO:teuthology.orchestra.run.smithi167.stderr:FINISHED --2023-10-06 04:56:18-- 2023-10-06T04:56:18.356 INFO:teuthology.orchestra.run.smithi167.stderr:Total wall clock time: 16s 2023-10-06T04:56:18.356 INFO:teuthology.orchestra.run.smithi167.stderr:Downloaded: 1 files, 363M in 15s (23.6 MB/s) 2023-10-06T04:56:18.357 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T04:56:18.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T04:56:20.102 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying... ######################################## 2023-10-06T04:56:20.531 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing... ######################################## 2023-10-06T04:56:20.657 INFO:teuthology.orchestra.run.smithi167.stdout:Updating / installing... 2023-10-06T04:56:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06 04:56:30 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T04:56:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:FINISHED --2023-10-06 04:56:30-- 2023-10-06T04:56:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:Total wall clock time: 27s 2023-10-06T04:56:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:Downloaded: 1 files, 363M in 26s (13.8 MB/s) 2023-10-06T04:56:30.797 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T04:56:30.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T04:56:32.613 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying... ######################################## 2023-10-06T04:56:32.990 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing... ######################################## 2023-10-06T04:56:33.121 INFO:teuthology.orchestra.run.smithi033.stdout:Updating / installing... 2023-10-06T04:56:58.589 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T04:57:13.724 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T04:57:23.154 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T04:57:23.154 DEBUG:teuthology.orchestra.run.smithi167:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T04:57:23.210 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.210 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.210 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T04:57:23.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T04:57:23.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T04:57:23.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T04:57:23.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T04:57:23.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T04:57:23.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T04:57:23.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T04:57:23.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T04:57:23.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T04:57:23.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T04:57:23.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T04:57:23.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T04:57:23.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T04:57:23.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T04:57:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T04:57:23.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T04:57:23.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T04:57:23.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T04:57:23.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T04:57:23.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T04:57:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T04:57:23.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T04:57:23.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T04:57:23.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T04:57:23.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T04:57:23.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T04:57:23.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T04:57:23.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T04:57:23.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T04:57:23.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T04:57:23.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T04:57:23.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T04:57:23.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T04:57:23.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T04:57:23.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T04:57:23.246 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T04:57:23.247 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T04:57:23.248 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T04:57:23.249 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T04:57:23.250 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T04:57:23.251 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T04:57:23.252 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T04:57:23.253 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T04:57:23.254 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T04:57:23.255 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T04:57:23.256 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T04:57:23.257 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T04:57:23.258 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T04:57:23.259 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T04:57:23.260 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T04:57:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T04:57:23.262 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T04:57:23.263 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T04:57:23.264 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T04:57:23.265 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T04:57:23.266 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T04:57:23.267 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T04:57:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T04:57:23.269 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T04:57:23.270 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T04:57:23.271 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T04:57:23.272 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T04:57:23.273 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T04:57:23.274 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T04:57:23.275 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T04:57:23.276 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T04:57:23.277 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T04:57:23.278 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T04:57:23.279 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T04:57:23.280 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T04:57:23.281 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T04:57:23.282 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T04:57:23.283 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T04:57:23.284 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T04:57:23.285 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T04:57:23.286 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T04:57:23.287 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T04:57:23.288 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T04:57:23.289 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T04:57:23.290 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T04:57:23.291 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T04:57:23.292 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T04:57:23.293 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T04:57:23.294 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T04:57:23.295 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T04:57:23.296 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T04:57:23.297 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T04:57:23.298 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T04:57:23.299 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T04:57:23.300 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T04:57:23.301 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T04:57:23.302 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T04:57:23.303 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T04:57:23.304 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T04:57:23.305 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T04:57:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T04:57:23.307 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T04:57:23.308 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T04:57:23.309 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T04:57:23.310 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T04:57:23.311 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T04:57:23.312 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T04:57:23.313 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T04:57:23.314 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T04:57:23.315 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T04:57:23.316 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T04:57:23.317 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T04:57:23.318 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T04:57:23.319 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T04:57:23.320 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T04:57:23.321 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T04:57:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T04:57:23.324 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T04:57:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T04:57:23.326 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T04:57:23.327 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T04:57:23.328 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T04:57:23.329 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T04:57:23.330 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T04:57:23.331 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T04:57:23.332 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T04:57:23.333 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T04:57:23.334 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T04:57:23.335 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T04:57:23.336 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T04:57:23.337 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T04:57:23.338 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T04:57:23.339 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T04:57:23.340 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T04:57:23.341 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T04:57:23.342 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T04:57:23.343 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T04:57:23.344 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T04:57:23.345 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T04:57:23.346 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T04:57:23.347 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T04:57:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T04:57:23.349 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T04:57:23.350 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T04:57:23.351 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T04:57:23.353 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T04:57:23.353 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T04:57:23.353 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T04:57:23.353 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T04:57:23.354 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T04:57:23.355 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T04:57:23.356 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T04:57:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T04:57:23.358 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T04:57:23.359 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T04:57:23.360 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T04:57:23.361 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T04:57:23.362 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T04:57:23.363 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T04:57:23.364 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T04:57:23.365 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T04:57:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T04:57:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T04:57:23.368 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T04:57:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T04:57:23.370 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T04:57:23.371 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T04:57:23.371 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T04:57:23.371 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T04:57:23.371 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T04:57:23.371 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T04:57:23.371 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T04:57:23.372 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T04:57:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T04:57:23.374 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T04:57:23.375 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T04:57:23.376 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T04:57:23.377 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T04:57:23.378 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T04:57:23.379 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T04:57:23.380 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T04:57:23.380 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T04:57:23.380 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T04:57:23.380 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.380 DEBUG:teuthology.orchestra.run.smithi167:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2023-10-06T04:57:23.405 INFO:teuthology.orchestra.run.smithi167.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.406 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2023-10-06T04:57:23.407 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2023-10-06T04:57:23.480 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2023-10-06T04:57:23.480 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2023-10-06T04:57:23.480 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.02 2023-10-06T04:57:23.480 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 99.el8_4.1 2023-10-06T04:57:23.480 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2023-10-06T04:57:23.480 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Group : System Environment/Base 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 9264631 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Relocations : (not relocatable) 2023-10-06T04:57:23.481 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : Red Hat, Inc. 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : Red Hat, Inc. 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T04:57:23.482 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T04:57:23.483 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T04:57:23.484 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:23.484 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T04:57:26.856 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2023-10-06T04:57:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:done 2023-10-06T04:57:28.698 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T04:57:28.712 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.YCJUWzZWaD 2023-10-06T04:57:28.713 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YCJUWzZWaD 2023-10-06T04:57:28.782 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.YCJUWzZWaD 2023-10-06T04:57:28.938 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.YCJUWzZWaD is 6KB 2023-10-06T04:57:28.988 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.YCJUWzZWaD 2023-10-06T04:57:29.001 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T04:57:29.062 INFO:teuthology.orchestra.run.smithi167.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T04:57:29.063 INFO:teuthology.orchestra.run.smithi167.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T04:57:29.063 INFO:teuthology.orchestra.run.smithi167.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T04:57:29.063 INFO:teuthology.orchestra.run.smithi167.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T04:57:29.064 INFO:teuthology.orchestra.run.smithi167.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T04:57:29.066 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:29.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2023-10-06T04:57:36.306 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T04:57:36.306 DEBUG:teuthology.orchestra.run.smithi033:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T04:57:36.359 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T04:57:36.360 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T04:57:36.361 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T04:57:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T04:57:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T04:57:36.364 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T04:57:36.365 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T04:57:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T04:57:36.367 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T04:57:36.368 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T04:57:36.369 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T04:57:36.370 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T04:57:36.371 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T04:57:36.372 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T04:57:36.373 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T04:57:36.374 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T04:57:36.375 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T04:57:36.376 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T04:57:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T04:57:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T04:57:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T04:57:36.380 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T04:57:36.381 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T04:57:36.382 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T04:57:36.383 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T04:57:36.384 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T04:57:36.385 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T04:57:36.386 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T04:57:36.387 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T04:57:36.388 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T04:57:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T04:57:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T04:57:36.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T04:57:36.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T04:57:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T04:57:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T04:57:36.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T04:57:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T04:57:36.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T04:57:36.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T04:57:36.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T04:57:36.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T04:57:36.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T04:57:36.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T04:57:36.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T04:57:36.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T04:57:36.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T04:57:36.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T04:57:36.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T04:57:36.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T04:57:36.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T04:57:36.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T04:57:36.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T04:57:36.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T04:57:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T04:57:36.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T04:57:36.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T04:57:36.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T04:57:36.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T04:57:36.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T04:57:36.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T04:57:36.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T04:57:36.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T04:57:36.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T04:57:36.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T04:57:36.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T04:57:36.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T04:57:36.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T04:57:36.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T04:57:36.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T04:57:36.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T04:57:36.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T04:57:36.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T04:57:36.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T04:57:36.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T04:57:36.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T04:57:36.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T04:57:36.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T04:57:36.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T04:57:36.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T04:57:36.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T04:57:36.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T04:57:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T04:57:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T04:57:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T04:57:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T04:57:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T04:57:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T04:57:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T04:57:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T04:57:36.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T04:57:36.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T04:57:36.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T04:57:36.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T04:57:36.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T04:57:36.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T04:57:36.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T04:57:36.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T04:57:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T04:57:36.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T04:57:36.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T04:57:36.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T04:57:36.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T04:57:36.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T04:57:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T04:57:36.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T04:57:36.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T04:57:36.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T04:57:36.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T04:57:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T04:57:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T04:57:36.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T04:57:36.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T04:57:36.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T04:57:36.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T04:57:36.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T04:57:36.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T04:57:36.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T04:57:36.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T04:57:36.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T04:57:36.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T04:57:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T04:57:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T04:57:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T04:57:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T04:57:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T04:57:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T04:57:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T04:57:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T04:57:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T04:57:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T04:57:36.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T04:57:36.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T04:57:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T04:57:36.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T04:57:36.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T04:57:36.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T04:57:36.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T04:57:36.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T04:57:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T04:57:36.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T04:57:36.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T04:57:36.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T04:57:36.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T04:57:36.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T04:57:36.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T04:57:36.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T04:57:36.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T04:57:36.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T04:57:36.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T04:57:36.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T04:57:36.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T04:57:36.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T04:57:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T04:57:36.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T04:57:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T04:57:36.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T04:57:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T04:57:36.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T04:57:36.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T04:57:36.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T04:57:36.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T04:57:36.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T04:57:36.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T04:57:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T04:57:36.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T04:57:36.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T04:57:36.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T04:57:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T04:57:36.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T04:57:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T04:57:36.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T04:57:36.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T04:57:36.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T04:57:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T04:57:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T04:57:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T04:57:36.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T04:57:36.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T04:57:36.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T04:57:36.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T04:57:36.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T04:57:36.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T04:57:36.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T04:57:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T04:57:36.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T04:57:36.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T04:57:36.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T04:57:36.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T04:57:36.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T04:57:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T04:57:36.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T04:57:36.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T04:57:36.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T04:57:36.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T04:57:36.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T04:57:36.528 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T04:57:36.529 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T04:57:36.530 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T04:57:36.531 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T04:57:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T04:57:36.533 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T04:57:36.534 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T04:57:36.534 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.535 DEBUG:teuthology.orchestra.run.smithi033:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:36.559 INFO:teuthology.orchestra.run.smithi033.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.560 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2023-10-06T04:57:36.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.02 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 99.el8_4.1 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T04:57:36.634 INFO:teuthology.orchestra.run.smithi033.stdout:Group : System Environment/Base 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 9264631 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Relocations : (not relocatable) 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : Red Hat, Inc. 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : Red Hat, Inc. 2023-10-06T04:57:36.635 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T04:57:36.636 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T04:57:36.637 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T04:57:36.637 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:36.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T04:57:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2023-10-06T04:57:44.589 INFO:teuthology.orchestra.run.smithi033.stderr:done 2023-10-06T04:57:44.590 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2023-10-06T04:57:44.618 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.S7epbcR3U9 2023-10-06T04:57:44.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.S7epbcR3U9 2023-10-06T04:57:44.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.S7epbcR3U9 2023-10-06T04:57:44.845 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.S7epbcR3U9 is 6KB 2023-10-06T04:57:44.932 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.S7epbcR3U9 2023-10-06T04:57:44.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T04:57:45.017 INFO:teuthology.orchestra.run.smithi033.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T04:57:45.017 INFO:teuthology.orchestra.run.smithi033.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T04:57:45.017 INFO:teuthology.orchestra.run.smithi033.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T04:57:45.018 INFO:teuthology.orchestra.run.smithi033.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T04:57:45.018 INFO:teuthology.orchestra.run.smithi033.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T04:57:45.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-06T04:57:45.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2023-10-06T04:57:59.266 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T04:57:59.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T04:57:59.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:58:15.234 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T04:58:15.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T04:58:15.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:58:33.722 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-06T04:58:42.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T04:58:42.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:59:01.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-06T04:59:13.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T04:59:13.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:59:15.238 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T04:59:16.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-06T04:59:24.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T04:59:24.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:59:24.648 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-10-06T04:59:25.037 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T04:59:25.037 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T04:59:25.038 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2023-10-06T04:59:25.073 INFO:teuthology.orchestra.run.smithi033.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T04:59:25.074 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:59:25.074 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T04:59:25.074 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T04:59:26.075 DEBUG:teuthology.parallel:result is None 2023-10-06T04:59:31.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T04:59:31.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:59:31.592 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-06T04:59:31.998 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T04:59:31.999 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T04:59:31.999 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-10-06T04:59:32.034 INFO:teuthology.orchestra.run.smithi167.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T04:59:32.034 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T04:59:32.034 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T04:59:32.035 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T04:59:33.035 DEBUG:teuthology.parallel:result is None 2023-10-06T04:59:33.036 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T04:59:33.042 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T04:59:33.043 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T04:59:33.048 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T04:59:33.088 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T04:59:33.095 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T04:59:33.124 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T04:59:33.124 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T04:59:33.130 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T04:59:33.184 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T04:59:33.191 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T04:59:33.191 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T04:59:33.196 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T04:59:33.325 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:59:33.326 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:59:33.362 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:59:33.363 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:59:33.366 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T04:59:33.373 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T04:59:33.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T04:59:33.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T04:59:33.439 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T04:59:33.448 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T04:59:33.448 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T04:59:33.453 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T04:59:33.488 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:59:33.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:59:33.692 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:59:33.760 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:59:33.833 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T04:59:33.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T04:59:33.902 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:59:33.970 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:59:34.066 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:59:34.127 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:59:34.206 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T04:59:34.206 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T04:59:34.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2023-10-06T04:59:34.273 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2023-10-06T04:59:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:59:34.409 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:59:34.847 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T04:59:34.855 INFO:teuthology.task.internal:Starting timer... 2023-10-06T04:59:34.855 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T04:59:34.901 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T04:59:34.933 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T04:59:34.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2023-10-06T04:59:35.046 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: [ OK ] 2023-10-06T04:59:35.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2023-10-06T04:59:35.165 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: [ OK ] 2023-10-06T04:59:35.168 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T04:59:35.168 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2023-10-06T04:59:35.229 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2023-10-06T04:59:35.229 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2023-10-06T04:59:35.270 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2023-10-06T04:59:35.271 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2023-10-06T04:59:35.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:59:35.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:59:35.336 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:59:35.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:59:35.398 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T04:59:35.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2023-10-06T04:59:35.461 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2023-10-06T04:59:35.523 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T04:59:35.532 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T04:59:35.532 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T04:59:35.548 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T04:59:35.549 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T05:10:53.034 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2023-10-06T05:10:53.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T05:10:53.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:10:53.128 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-10-06T05:10:53.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T05:10:53.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T05:10:53.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:10:53.334 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-06T05:10:53.442 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T05:10:53.442 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T05:10:53.455 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T05:10:53.455 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T05:10:53.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:10:53.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T05:10:53.460 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:10:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T05:10:53.548 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T05:10:53.560 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T05:10:53.582 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T05:10:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2023-10-06T05:10:53.643 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2023-10-06T05:10:53.650 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2023-10-06T05:10:53.670 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2023-10-06T05:10:53.684 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T05:10:53.703 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T05:10:53.719 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T05:10:53.737 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T05:10:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2023-10-06T05:10:53.810 INFO:teuthology.orchestra.run.smithi033.stdout:210 Number of sources = 4 2023-10-06T05:10:53.810 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:10:53.811 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2023-10-06T05:10:53.811 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.811 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.811 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.811 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.818 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-10-06T05:10:53.823 INFO:teuthology.orchestra.run.smithi167.stdout:210 Number of sources = 4 2023-10-06T05:10:53.823 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:10:53.823 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-10-06T05:10:53.823 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.823 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.824 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.824 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:10:53.825 INFO:teuthology.run_tasks:Running task install... 2023-10-06T05:10:53.850 DEBUG:teuthology.task.install:project ceph 2023-10-06T05:10:53.850 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T05:10:53.850 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T05:10:53.850 INFO:teuthology.task.install:Using flavor: default 2023-10-06T05:10:53.856 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T05:10:53.856 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T05:10:53.856 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T05:10:53.857 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:10:53.858 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T05:10:53.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:10:54.002 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T05:10:54.002 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T05:10:54.070 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T05:10:54.070 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T05:10:54.174 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T05:10:54.174 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:10:54.174 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T05:10:54.236 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T05:10:54.237 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T05:10:54.247 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T05:10:54.248 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:10:54.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T05:10:54.312 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T05:10:54.312 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T05:10:54.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T05:10:54.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T05:10:54.427 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T05:10:54.487 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2023-10-06T05:10:54.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2023-10-06T05:10:54.506 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T05:10:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2023-10-06T05:10:54.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2023-10-06T05:10:54.948 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T05:10:55.066 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2023-10-06T05:10:55.472 INFO:teuthology.orchestra.run.smithi167.stdout:44 files removed 2023-10-06T05:10:55.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T05:10:55.707 INFO:teuthology.orchestra.run.smithi033.stdout:44 files removed 2023-10-06T05:10:55.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T05:10:55.990 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T05:10:56.269 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2023-10-06T05:10:56.953 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 249 kB/s | 78 kB 00:00 2023-10-06T05:10:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 209 kB/s | 78 kB 00:00 2023-10-06T05:10:57.300 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-06T05:10:57.514 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 48 kB/s | 9.2 kB 00:00 2023-10-06T05:10:57.638 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 4.7 kB/s | 1.4 kB 00:00 2023-10-06T05:10:57.735 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 7.7 kB/s | 1.4 kB 00:00 2023-10-06T05:10:57.748 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-06T05:10:57.847 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-06T05:10:58.961 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-10-06T05:10:59.466 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-06T05:11:02.480 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-06T05:11:02.982 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 79 MB/s | 33 MB 00:00 2023-10-06T05:11:03.162 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-06T05:11:03.747 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 77 MB/s | 38 MB 00:00 2023-10-06T05:11:08.539 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 MB/s | 38 MB 00:00 2023-10-06T05:11:09.903 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 75 MB/s | 33 MB 00:00 2023-10-06T05:11:17.225 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:17.225 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:17.235 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T05:11:17.236 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T05:11:17.237 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T05:11:17.238 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T05:11:17.240 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T05:11:17.242 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T05:11:17.243 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T05:11:17.744 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-06T05:11:17.755 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================================== 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================================== 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T05:11:17.756 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T05:11:17.757 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T05:11:17.758 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T05:11:17.759 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T05:11:17.760 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T05:11:17.761 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T05:11:17.762 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T05:11:17.763 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T05:11:17.764 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T05:11:17.765 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T05:11:17.766 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T05:11:17.767 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:11:17.768 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-06T05:11:17.768 INFO:teuthology.orchestra.run.smithi167.stdout:====================================================================================================================== 2023-10-06T05:11:17.768 INFO:teuthology.orchestra.run.smithi167.stdout:Install 118 Packages 2023-10-06T05:11:17.768 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2023-10-06T05:11:17.768 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:11:17.768 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 201 M 2023-10-06T05:11:17.769 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-06T05:11:18.016 INFO:teuthology.orchestra.run.smithi167.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 70 kB/s | 6.4 kB 00:00 2023-10-06T05:11:18.274 INFO:teuthology.orchestra.run.smithi167.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 3.3 MB/s | 877 kB 00:00 2023-10-06T05:11:18.350 INFO:teuthology.orchestra.run.smithi167.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 2.1 MB/s | 162 kB 00:00 2023-10-06T05:11:18.753 INFO:teuthology.orchestra.run.smithi167.stdout:(4/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 6.2 MB/s | 5.2 MB 00:00 2023-10-06T05:11:18.774 INFO:teuthology.orchestra.run.smithi167.stdout:(5/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 5.4 MB/s | 2.3 MB 00:00 2023-10-06T05:11:18.914 INFO:teuthology.orchestra.run.smithi167.stdout:(6/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 9.8 MB/s | 1.6 MB 00:00 2023-10-06T05:11:18.932 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:18.932 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:18.944 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T05:11:18.945 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T05:11:18.946 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T05:11:18.947 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T05:11:18.949 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T05:11:18.951 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T05:11:18.952 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T05:11:19.477 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2023-10-06T05:11:19.489 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================================================== 2023-10-06T05:11:19.489 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2023-10-06T05:11:19.489 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================================================== 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T05:11:19.490 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T05:11:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T05:11:19.492 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T05:11:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T05:11:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T05:11:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T05:11:19.496 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T05:11:19.497 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T05:11:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T05:11:19.499 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T05:11:19.500 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T05:11:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T05:11:19.502 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:11:19.503 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2023-10-06T05:11:19.504 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================================================== 2023-10-06T05:11:19.504 INFO:teuthology.orchestra.run.smithi033.stdout:Install 118 Packages 2023-10-06T05:11:19.504 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 2 Packages 2023-10-06T05:11:19.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:11:19.505 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 201 M 2023-10-06T05:11:19.505 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2023-10-06T05:11:19.681 INFO:teuthology.orchestra.run.smithi167.stdout:(7/120): ceph-common-17.2.6-1274.g68e53a4a.el8. 13 MB/s | 23 MB 00:01 2023-10-06T05:11:19.707 INFO:teuthology.orchestra.run.smithi033.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 45 kB/s | 6.4 kB 00:00 2023-10-06T05:11:20.019 INFO:teuthology.orchestra.run.smithi167.stdout:(8/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 3.2 MB/s | 3.9 MB 00:01 2023-10-06T05:11:20.084 INFO:teuthology.orchestra.run.smithi167.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 377 kB/s | 24 kB 00:00 2023-10-06T05:11:20.236 INFO:teuthology.orchestra.run.smithi167.stdout:(10/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 13 MB/s | 17 MB 00:01 2023-10-06T05:11:20.284 INFO:teuthology.orchestra.run.smithi167.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 5.3 MB/s | 255 kB 00:00 2023-10-06T05:11:20.322 INFO:teuthology.orchestra.run.smithi167.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 656 kB/s | 25 kB 00:00 2023-10-06T05:11:20.393 INFO:teuthology.orchestra.run.smithi167.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 11 MB/s | 797 kB 00:00 2023-10-06T05:11:20.439 INFO:teuthology.orchestra.run.smithi167.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 3.7 MB/s | 175 kB 00:00 2023-10-06T05:11:20.484 INFO:teuthology.orchestra.run.smithi167.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 2.9 MB/s | 130 kB 00:00 2023-10-06T05:11:20.542 INFO:teuthology.orchestra.run.smithi167.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 8.6 MB/s | 506 kB 00:00 2023-10-06T05:11:21.010 INFO:teuthology.orchestra.run.smithi167.stdout:(17/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 13 MB/s | 17 MB 00:01 2023-10-06T05:11:21.032 INFO:teuthology.orchestra.run.smithi167.stdout:(18/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 11 MB/s | 5.6 MB 00:00 2023-10-06T05:11:21.033 INFO:teuthology.orchestra.run.smithi167.stdout:(19/120): python3-ceph-argparse-17.2.6-1274.g68 1.9 MB/s | 45 kB 00:00 2023-10-06T05:11:21.056 INFO:teuthology.orchestra.run.smithi167.stdout:(20/120): python3-ceph-common-17.2.6-1274.g68e5 4.9 MB/s | 115 kB 00:00 2023-10-06T05:11:21.061 INFO:teuthology.orchestra.run.smithi167.stdout:(21/120): python3-cephfs-17.2.6-1274.g68e53a4a. 6.9 MB/s | 190 kB 00:00 2023-10-06T05:11:21.087 INFO:teuthology.orchestra.run.smithi167.stdout:(22/120): python3-rados-17.2.6-1274.g68e53a4a.e 12 MB/s | 371 kB 00:00 2023-10-06T05:11:21.101 INFO:teuthology.orchestra.run.smithi167.stdout:(23/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 8.9 MB/s | 358 kB 00:00 2023-10-06T05:11:21.118 INFO:teuthology.orchestra.run.smithi167.stdout:(24/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 3.4 MB/s | 107 kB 00:00 2023-10-06T05:11:21.124 INFO:teuthology.orchestra.run.smithi167.stdout:(25/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 3.8 MB/s | 91 kB 00:00 2023-10-06T05:11:21.179 INFO:teuthology.orchestra.run.smithi167.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 3.1 MB/s | 173 kB 00:00 2023-10-06T05:11:21.230 INFO:teuthology.orchestra.run.smithi167.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 459 kB/s | 23 kB 00:00 2023-10-06T05:11:21.230 INFO:teuthology.orchestra.run.smithi033.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 576 kB/s | 877 kB 00:01 2023-10-06T05:11:21.286 INFO:teuthology.orchestra.run.smithi167.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 2.1 MB/s | 121 kB 00:00 2023-10-06T05:11:21.320 INFO:teuthology.orchestra.run.smithi167.stdout:(29/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 17 MB/s | 3.5 MB 00:00 2023-10-06T05:11:21.465 INFO:teuthology.orchestra.run.smithi033.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 691 kB/s | 162 kB 00:00 2023-10-06T05:11:21.489 INFO:teuthology.orchestra.run.smithi167.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 8.4 MB/s | 1.7 MB 00:00 2023-10-06T05:11:21.546 INFO:teuthology.orchestra.run.smithi167.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 4.1 MB/s | 238 kB 00:00 2023-10-06T05:11:21.599 INFO:teuthology.orchestra.run.smithi167.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 910 kB/s | 47 kB 00:00 2023-10-06T05:11:21.649 INFO:teuthology.orchestra.run.smithi167.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 287 kB/s | 14 kB 00:00 2023-10-06T05:11:21.709 INFO:teuthology.orchestra.run.smithi167.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 4.4 MB/s | 267 kB 00:00 2023-10-06T05:11:21.782 INFO:teuthology.orchestra.run.smithi167.stdout:(35/120): ceph-mgr-diskprediction-local-17.2.6- 16 MB/s | 7.4 MB 00:00 2023-10-06T05:11:21.783 INFO:teuthology.orchestra.run.smithi167.stdout:(36/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.1 MB/s | 80 kB 00:00 2023-10-06T05:11:21.804 INFO:teuthology.orchestra.run.smithi167.stdout:(37/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 832 kB/s | 18 kB 00:00 2023-10-06T05:11:22.163 INFO:teuthology.orchestra.run.smithi167.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 164 kB/s | 59 kB 00:00 2023-10-06T05:11:22.327 INFO:teuthology.orchestra.run.smithi167.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 460 kB/s | 75 kB 00:00 2023-10-06T05:11:22.355 INFO:teuthology.orchestra.run.smithi167.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 536 kB/s | 306 kB 00:00 2023-10-06T05:11:22.444 INFO:teuthology.orchestra.run.smithi167.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 492 kB/s | 44 kB 00:00 2023-10-06T05:11:22.487 INFO:teuthology.orchestra.run.smithi033.stdout:(4/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-10-06T05:11:22.549 INFO:teuthology.orchestra.run.smithi033.stdout:(5/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 1.7 MB/s | 5.2 MB 00:02 2023-10-06T05:11:22.560 INFO:teuthology.orchestra.run.smithi167.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-10-06T05:11:22.603 INFO:teuthology.orchestra.run.smithi167.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-10-06T05:11:22.647 INFO:teuthology.orchestra.run.smithi167.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 379 kB/s | 33 kB 00:00 2023-10-06T05:11:22.686 INFO:teuthology.orchestra.run.smithi167.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 197 kB/s | 16 kB 00:00 2023-10-06T05:11:22.761 INFO:teuthology.orchestra.run.smithi167.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.5 MB/s | 173 kB 00:00 2023-10-06T05:11:22.809 INFO:teuthology.orchestra.run.smithi167.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.1 MB/s | 384 kB 00:00 2023-10-06T05:11:22.858 INFO:teuthology.orchestra.run.smithi167.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 839 kB/s | 81 kB 00:00 2023-10-06T05:11:22.891 INFO:teuthology.orchestra.run.smithi167.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 135 kB/s | 11 kB 00:00 2023-10-06T05:11:22.923 INFO:teuthology.orchestra.run.smithi033.stdout:(6/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 3.6 MB/s | 1.6 MB 00:00 2023-10-06T05:11:22.943 INFO:teuthology.orchestra.run.smithi167.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 209 kB/s | 18 kB 00:00 2023-10-06T05:11:23.033 INFO:teuthology.orchestra.run.smithi167.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 537 kB/s | 49 kB 00:00 2023-10-06T05:11:23.102 INFO:teuthology.orchestra.run.smithi167.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 5.5 MB/s | 1.2 MB 00:00 2023-10-06T05:11:23.126 INFO:teuthology.orchestra.run.smithi167.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 637 kB/s | 59 kB 00:00 2023-10-06T05:11:23.186 INFO:teuthology.orchestra.run.smithi167.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 722 kB/s | 60 kB 00:00 2023-10-06T05:11:23.261 INFO:teuthology.orchestra.run.smithi167.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.0 MB/s | 283 kB 00:00 2023-10-06T05:11:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 190 kB/s | 16 kB 00:00 2023-10-06T05:11:23.323 INFO:teuthology.orchestra.run.smithi033.stdout:(7/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 5.1 MB/s | 3.9 MB 00:00 2023-10-06T05:11:23.348 INFO:teuthology.orchestra.run.smithi167.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 388 kB/s | 34 kB 00:00 2023-10-06T05:11:23.357 INFO:teuthology.orchestra.run.smithi167.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 2.2 MB/s | 196 kB 00:00 2023-10-06T05:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 702 kB/s | 64 kB 00:00 2023-10-06T05:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 226 kB/s | 19 kB 00:00 2023-10-06T05:11:23.523 INFO:teuthology.orchestra.run.smithi167.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 357 kB/s | 29 kB 00:00 2023-10-06T05:11:23.525 INFO:teuthology.orchestra.run.smithi167.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 282 kB/s | 24 kB 00:00 2023-10-06T05:11:23.605 INFO:teuthology.orchestra.run.smithi167.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 335 kB/s | 27 kB 00:00 2023-10-06T05:11:23.650 INFO:teuthology.orchestra.run.smithi167.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-10-06T05:11:23.696 INFO:teuthology.orchestra.run.smithi167.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-10-06T05:11:23.742 INFO:teuthology.orchestra.run.smithi167.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 667 kB/s | 61 kB 00:00 2023-10-06T05:11:23.781 INFO:teuthology.orchestra.run.smithi167.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-06T05:11:23.827 INFO:teuthology.orchestra.run.smithi167.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 271 kB/s | 23 kB 00:00 2023-10-06T05:11:23.832 INFO:teuthology.orchestra.run.smithi167.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-06T05:11:23.933 INFO:teuthology.orchestra.run.smithi167.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-10-06T05:11:23.974 INFO:teuthology.orchestra.run.smithi167.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-10-06T05:11:23.995 INFO:teuthology.orchestra.run.smithi167.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.3 MB/s | 27 kB 00:00 2023-10-06T05:11:24.017 INFO:teuthology.orchestra.run.smithi167.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 7.2 MB/s | 161 kB 00:00 2023-10-06T05:11:24.043 INFO:teuthology.orchestra.run.smithi167.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 12 MB/s | 320 kB 00:00 2023-10-06T05:11:24.087 INFO:teuthology.orchestra.run.smithi167.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 10 MB/s | 457 kB 00:00 2023-10-06T05:11:24.158 INFO:teuthology.orchestra.run.smithi167.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 45 MB/s | 15 MB 00:00 2023-10-06T05:11:24.179 INFO:teuthology.orchestra.run.smithi167.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 1.8 MB/s | 39 kB 00:00 2023-10-06T05:11:24.217 INFO:teuthology.orchestra.run.smithi167.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 18 MB/s | 688 kB 00:00 2023-10-06T05:11:24.240 INFO:teuthology.orchestra.run.smithi167.stdout:(79/120): python3-babel-2.5.1-5.el8.noarch.rpm 31 MB/s | 4.8 MB 00:00 2023-10-06T05:11:24.256 INFO:teuthology.orchestra.run.smithi167.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.5 MB/s | 259 kB 00:00 2023-10-06T05:11:24.276 INFO:teuthology.orchestra.run.smithi167.stdout:(81/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 6.2 MB/s | 126 kB 00:00 2023-10-06T05:11:24.294 INFO:teuthology.orchestra.run.smithi167.stdout:(82/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 4.1 MB/s | 225 kB 00:00 2023-10-06T05:11:24.321 INFO:teuthology.orchestra.run.smithi167.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 6.8 MB/s | 187 kB 00:00 2023-10-06T05:11:24.344 INFO:teuthology.orchestra.run.smithi167.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 23 MB/s | 538 kB 00:00 2023-10-06T05:11:24.370 INFO:teuthology.orchestra.run.smithi167.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 40 MB/s | 3.7 MB 00:00 2023-10-06T05:11:24.376 INFO:teuthology.orchestra.run.smithi167.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 9.4 MB/s | 302 kB 00:00 2023-10-06T05:11:24.404 INFO:teuthology.orchestra.run.smithi167.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-10-06T05:11:24.406 INFO:teuthology.orchestra.run.smithi167.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 3.6 MB/s | 110 kB 00:00 2023-10-06T05:11:24.432 INFO:teuthology.orchestra.run.smithi167.stdout:(89/120): python3-jsonpointer-1.10-11.el8.noarc 794 kB/s | 20 kB 00:00 2023-10-06T05:11:24.444 INFO:teuthology.orchestra.run.smithi167.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 8.7 MB/s | 353 kB 00:00 2023-10-06T05:11:24.463 INFO:teuthology.orchestra.run.smithi167.stdout:(91/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.8 MB/s | 54 kB 00:00 2023-10-06T05:11:24.477 INFO:teuthology.orchestra.run.smithi167.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 15 MB/s | 709 kB 00:00 2023-10-06T05:11:24.581 INFO:teuthology.orchestra.run.smithi167.stdout:(93/120): openblas-threads-0.3.12-1.el8.x86_64. 40 MB/s | 4.7 MB 00:00 2023-10-06T05:11:24.614 INFO:teuthology.orchestra.run.smithi167.stdout:(94/120): openblas-0.3.12-1.el8.x86_64.rpm 34 MB/s | 4.6 MB 00:00 2023-10-06T05:11:24.624 INFO:teuthology.orchestra.run.smithi167.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 16 MB/s | 713 kB 00:00 2023-10-06T05:11:24.640 INFO:teuthology.orchestra.run.smithi167.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 707 kB/s | 19 kB 00:00 2023-10-06T05:11:24.645 INFO:teuthology.orchestra.run.smithi167.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.9 MB/s | 43 kB 00:00 2023-10-06T05:11:24.663 INFO:teuthology.orchestra.run.smithi167.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 6.7 MB/s | 155 kB 00:00 2023-10-06T05:11:24.680 INFO:teuthology.orchestra.run.smithi167.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.2 MB/s | 220 kB 00:00 2023-10-06T05:11:24.695 INFO:teuthology.orchestra.run.smithi167.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 1.5 MB/s | 48 kB 00:00 2023-10-06T05:11:24.708 INFO:teuthology.orchestra.run.smithi167.stdout:(101/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 8.2 MB/s | 229 kB 00:00 2023-10-06T05:11:24.731 INFO:teuthology.orchestra.run.smithi167.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 1.2 MB/s | 43 kB 00:00 2023-10-06T05:11:24.753 INFO:teuthology.orchestra.run.smithi167.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 808 kB/s | 18 kB 00:00 2023-10-06T05:11:24.781 INFO:teuthology.orchestra.run.smithi167.stdout:(104/120): openssl-devel-1.1.1g-15.el8_3.x86_64 32 MB/s | 2.3 MB 00:00 2023-10-06T05:11:24.790 INFO:teuthology.orchestra.run.smithi167.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 16 MB/s | 605 kB 00:00 2023-10-06T05:11:24.803 INFO:teuthology.orchestra.run.smithi167.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 4.0 MB/s | 83 kB 00:00 2023-10-06T05:11:24.818 INFO:teuthology.orchestra.run.smithi167.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 2.1 MB/s | 59 kB 00:00 2023-10-06T05:11:24.836 INFO:teuthology.orchestra.run.smithi167.stdout:(108/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.0 MB/s | 38 kB 00:00 2023-10-06T05:11:24.856 INFO:teuthology.orchestra.run.smithi167.stdout:(109/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.3 MB/s | 86 kB 00:00 2023-10-06T05:11:24.875 INFO:teuthology.orchestra.run.smithi167.stdout:(110/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.0 MB/s | 58 kB 00:00 2023-10-06T05:11:24.899 INFO:teuthology.orchestra.run.smithi167.stdout:(111/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.9 MB/s | 47 kB 00:00 2023-10-06T05:11:24.916 INFO:teuthology.orchestra.run.smithi167.stdout:(112/120): libquadmath-8.4.1-1.el8.x86_64.rpm 1.4 MB/s | 169 kB 00:00 2023-10-06T05:11:24.929 INFO:teuthology.orchestra.run.smithi167.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 18 MB/s | 559 kB 00:00 2023-10-06T05:11:24.952 INFO:teuthology.orchestra.run.smithi167.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 17 MB/s | 639 kB 00:00 2023-10-06T05:11:24.955 INFO:teuthology.orchestra.run.smithi167.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 2.7 MB/s | 70 kB 00:00 2023-10-06T05:11:24.985 INFO:teuthology.orchestra.run.smithi167.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 6.0 MB/s | 200 kB 00:00 2023-10-06T05:11:24.988 INFO:teuthology.orchestra.run.smithi167.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 2.9 MB/s | 97 kB 00:00 2023-10-06T05:11:25.522 INFO:teuthology.orchestra.run.smithi167.stdout:(118/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 6.7 MB/s | 3.5 MB 00:00 2023-10-06T05:11:25.545 INFO:teuthology.orchestra.run.smithi167.stdout:(119/120): librados2-17.2.6-1274.g68e53a4a.el8. 6.1 MB/s | 3.4 MB 00:00 2023-10-06T05:11:25.878 INFO:teuthology.orchestra.run.smithi033.stdout:(8/120): ceph-common-17.2.6-1274.g68e53a4a.el8. 3.7 MB/s | 23 MB 00:06 2023-10-06T05:11:25.918 INFO:teuthology.orchestra.run.smithi033.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 623 kB/s | 24 kB 00:00 2023-10-06T05:11:26.089 INFO:teuthology.orchestra.run.smithi033.stdout:(10/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 6.1 MB/s | 17 MB 00:02 2023-10-06T05:11:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:(11/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 5.1 MB/s | 17 MB 00:03 2023-10-06T05:11:26.185 INFO:teuthology.orchestra.run.smithi033.stdout:(12/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 2.6 MB/s | 255 kB 00:00 2023-10-06T05:11:26.205 INFO:teuthology.orchestra.run.smithi033.stdout:(13/120): libcephfs-devel-17.2.6-1274.g68e53a4a 1.1 MB/s | 25 kB 00:00 2023-10-06T05:11:26.228 INFO:teuthology.orchestra.run.smithi033.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 7.5 MB/s | 175 kB 00:00 2023-10-06T05:11:26.256 INFO:teuthology.orchestra.run.smithi033.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 4.5 MB/s | 130 kB 00:00 2023-10-06T05:11:26.294 INFO:teuthology.orchestra.run.smithi033.stdout:(16/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 7.2 MB/s | 797 kB 00:00 2023-10-06T05:11:26.337 INFO:teuthology.orchestra.run.smithi033.stdout:(17/120): libradosstriper1-17.2.6-1274.g68e53a4 6.1 MB/s | 506 kB 00:00 2023-10-06T05:11:26.363 INFO:teuthology.orchestra.run.smithi033.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 1.7 MB/s | 45 kB 00:00 2023-10-06T05:11:26.387 INFO:teuthology.orchestra.run.smithi033.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 4.7 MB/s | 115 kB 00:00 2023-10-06T05:11:26.438 INFO:teuthology.orchestra.run.smithi033.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 3.7 MB/s | 190 kB 00:00 2023-10-06T05:11:26.515 INFO:teuthology.orchestra.run.smithi033.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 4.7 MB/s | 371 kB 00:00 2023-10-06T05:11:26.574 INFO:teuthology.orchestra.run.smithi033.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 6.0 MB/s | 358 kB 00:00 2023-10-06T05:11:26.600 INFO:teuthology.orchestra.run.smithi033.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 4.2 MB/s | 107 kB 00:00 2023-10-06T05:11:26.622 INFO:teuthology.orchestra.run.smithi033.stdout:(24/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 4.0 MB/s | 91 kB 00:00 2023-10-06T05:11:26.777 INFO:teuthology.orchestra.run.smithi167.stdout:(120/120): ceph-test-17.2.6-1274.g68e53a4a.el8. 7.7 MB/s | 52 MB 00:06 2023-10-06T05:11:26.780 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:11:26.781 INFO:teuthology.orchestra.run.smithi167.stdout:Total 22 MB/s | 201 MB 00:09 2023-10-06T05:11:26.852 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-06T05:11:26.940 INFO:teuthology.orchestra.run.smithi033.stdout:(25/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 8.6 MB/s | 5.6 MB 00:00 2023-10-06T05:11:26.969 INFO:teuthology.orchestra.run.smithi033.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 6.0 MB/s | 173 kB 00:00 2023-10-06T05:11:26.977 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-06T05:11:26.977 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-06T05:11:26.990 INFO:teuthology.orchestra.run.smithi033.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 1.1 MB/s | 23 kB 00:00 2023-10-06T05:11:27.012 INFO:teuthology.orchestra.run.smithi033.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 5.4 MB/s | 121 kB 00:00 2023-10-06T05:11:27.036 INFO:teuthology.orchestra.run.smithi033.stdout:(29/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 8.3 MB/s | 3.5 MB 00:00 2023-10-06T05:11:27.189 INFO:teuthology.orchestra.run.smithi033.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 9.6 MB/s | 1.7 MB 00:00 2023-10-06T05:11:27.232 INFO:teuthology.orchestra.run.smithi033.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 5.5 MB/s | 238 kB 00:00 2023-10-06T05:11:27.253 INFO:teuthology.orchestra.run.smithi033.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 2.2 MB/s | 47 kB 00:00 2023-10-06T05:11:27.275 INFO:teuthology.orchestra.run.smithi033.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 695 kB/s | 14 kB 00:00 2023-10-06T05:11:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 6.2 MB/s | 267 kB 00:00 2023-10-06T05:11:27.339 INFO:teuthology.orchestra.run.smithi033.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 3.6 MB/s | 80 kB 00:00 2023-10-06T05:11:27.361 INFO:teuthology.orchestra.run.smithi033.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 844 kB/s | 18 kB 00:00 2023-10-06T05:11:27.607 INFO:teuthology.orchestra.run.smithi033.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-10-06T05:11:27.646 INFO:teuthology.orchestra.run.smithi033.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.5 MB/s | 59 kB 00:00 2023-10-06T05:11:27.685 INFO:teuthology.orchestra.run.smithi033.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-10-06T05:11:27.758 INFO:teuthology.orchestra.run.smithi033.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 6.4 MB/s | 476 kB 00:00 2023-10-06T05:11:27.793 INFO:teuthology.orchestra.run.smithi033.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.2 MB/s | 44 kB 00:00 2023-10-06T05:11:27.829 INFO:teuthology.orchestra.run.smithi033.stdout:(42/120): ceph-mgr-diskprediction-local-17.2.6- 9.3 MB/s | 7.4 MB 00:00 2023-10-06T05:11:27.832 INFO:teuthology.orchestra.run.smithi033.stdout:(43/120): python3-beautifulsoup4-4.6.3-2.el8.1. 4.7 MB/s | 185 kB 00:00 2023-10-06T05:11:27.868 INFO:teuthology.orchestra.run.smithi033.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 456 kB/s | 16 kB 00:00 2023-10-06T05:11:27.907 INFO:teuthology.orchestra.run.smithi033.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.4 MB/s | 173 kB 00:00 2023-10-06T05:11:27.949 INFO:teuthology.orchestra.run.smithi033.stdout:(46/120): python3-cachetools-3.1.1-4.el8.noarch 276 kB/s | 33 kB 00:00 2023-10-06T05:11:27.952 INFO:teuthology.orchestra.run.smithi033.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.3 MB/s | 384 kB 00:00 2023-10-06T05:11:27.987 INFO:teuthology.orchestra.run.smithi033.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 322 kB/s | 11 kB 00:00 2023-10-06T05:11:28.023 INFO:teuthology.orchestra.run.smithi033.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 502 kB/s | 18 kB 00:00 2023-10-06T05:11:28.049 INFO:teuthology.orchestra.run.smithi033.stdout:(50/120): python3-google-auth-1.1.1-10.el8.noar 815 kB/s | 81 kB 00:00 2023-10-06T05:11:28.093 INFO:teuthology.orchestra.run.smithi033.stdout:(51/120): python3-kubernetes-11.0.0-6.el8.noarc 17 MB/s | 1.2 MB 00:00 2023-10-06T05:11:28.097 INFO:teuthology.orchestra.run.smithi033.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 1.0 MB/s | 49 kB 00:00 2023-10-06T05:11:28.128 INFO:teuthology.orchestra.run.smithi033.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.6 MB/s | 59 kB 00:00 2023-10-06T05:11:28.149 INFO:teuthology.orchestra.run.smithi033.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.1 MB/s | 60 kB 00:00 2023-10-06T05:11:28.168 INFO:teuthology.orchestra.run.smithi033.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 7.0 MB/s | 283 kB 00:00 2023-10-06T05:11:28.187 INFO:teuthology.orchestra.run.smithi033.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 408 kB/s | 16 kB 00:00 2023-10-06T05:11:28.203 INFO:teuthology.orchestra.run.smithi033.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 974 kB/s | 34 kB 00:00 2023-10-06T05:11:28.238 INFO:teuthology.orchestra.run.smithi033.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 1.8 MB/s | 64 kB 00:00 2023-10-06T05:11:28.272 INFO:teuthology.orchestra.run.smithi033.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 546 kB/s | 19 kB 00:00 2023-10-06T05:11:28.276 INFO:teuthology.orchestra.run.smithi033.stdout:(60/120): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-10-06T05:11:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 694 kB/s | 24 kB 00:00 2023-10-06T05:11:28.315 INFO:teuthology.orchestra.run.smithi033.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 776 kB/s | 29 kB 00:00 2023-10-06T05:11:28.342 INFO:teuthology.orchestra.run.smithi033.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 794 kB/s | 27 kB 00:00 2023-10-06T05:11:28.375 INFO:teuthology.orchestra.run.smithi033.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 4.2 MB/s | 255 kB 00:00 2023-10-06T05:11:28.381 INFO:teuthology.orchestra.run.smithi033.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 6.3 MB/s | 251 kB 00:00 2023-10-06T05:11:28.414 INFO:teuthology.orchestra.run.smithi033.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.5 MB/s | 61 kB 00:00 2023-10-06T05:11:28.417 INFO:teuthology.orchestra.run.smithi033.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 2.4 MB/s | 85 kB 00:00 2023-10-06T05:11:28.450 INFO:teuthology.orchestra.run.smithi033.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 649 kB/s | 23 kB 00:00 2023-10-06T05:11:28.454 INFO:teuthology.orchestra.run.smithi033.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-10-06T05:11:28.491 INFO:teuthology.orchestra.run.smithi033.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 24 MB/s | 1.7 MB 00:00 2023-10-06T05:11:28.496 INFO:teuthology.orchestra.run.smithi033.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 450 kB/s | 19 kB 00:00 2023-10-06T05:11:28.514 INFO:teuthology.orchestra.run.smithi033.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.6 MB/s | 155 kB 00:00 2023-10-06T05:11:28.525 INFO:teuthology.orchestra.run.smithi033.stdout:(73/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-10-06T05:11:28.534 INFO:teuthology.orchestra.run.smithi033.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 11 MB/s | 220 kB 00:00 2023-10-06T05:11:28.540 INFO:teuthology.orchestra.run.smithi033.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.2 MB/s | 48 kB 00:00 2023-10-06T05:11:28.554 INFO:teuthology.orchestra.run.smithi033.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 11 MB/s | 229 kB 00:00 2023-10-06T05:11:28.556 INFO:teuthology.orchestra.run.smithi033.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.7 MB/s | 43 kB 00:00 2023-10-06T05:11:28.572 INFO:teuthology.orchestra.run.smithi033.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.1 MB/s | 18 kB 00:00 2023-10-06T05:11:28.604 INFO:teuthology.orchestra.run.smithi033.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 19 MB/s | 605 kB 00:00 2023-10-06T05:11:28.618 INFO:teuthology.orchestra.run.smithi033.stdout:(80/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 36 MB/s | 2.3 MB 00:00 2023-10-06T05:11:28.623 INFO:teuthology.orchestra.run.smithi033.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 4.2 MB/s | 83 kB 00:00 2023-10-06T05:11:28.637 INFO:teuthology.orchestra.run.smithi033.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.1 MB/s | 59 kB 00:00 2023-10-06T05:11:28.641 INFO:teuthology.orchestra.run.smithi033.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.4 MB/s | 169 kB 00:00 2023-10-06T05:11:28.656 INFO:teuthology.orchestra.run.smithi033.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.0 MB/s | 38 kB 00:00 2023-10-06T05:11:28.658 INFO:teuthology.orchestra.run.smithi033.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.0 MB/s | 86 kB 00:00 2023-10-06T05:11:28.675 INFO:teuthology.orchestra.run.smithi033.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.0 MB/s | 58 kB 00:00 2023-10-06T05:11:28.676 INFO:teuthology.orchestra.run.smithi033.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.6 MB/s | 47 kB 00:00 2023-10-06T05:11:28.707 INFO:teuthology.orchestra.run.smithi033.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 17 MB/s | 559 kB 00:00 2023-10-06T05:11:28.713 INFO:teuthology.orchestra.run.smithi033.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 17 MB/s | 639 kB 00:00 2023-10-06T05:11:28.728 INFO:teuthology.orchestra.run.smithi033.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 3.4 MB/s | 70 kB 00:00 2023-10-06T05:11:28.733 INFO:teuthology.orchestra.run.smithi033.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 9.6 MB/s | 200 kB 00:00 2023-10-06T05:11:28.748 INFO:teuthology.orchestra.run.smithi033.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 4.8 MB/s | 97 kB 00:00 2023-10-06T05:11:28.754 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-06T05:11:28.759 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-06T05:11:28.766 INFO:teuthology.orchestra.run.smithi033.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 2.3 MB/s | 44 kB 00:00 2023-10-06T05:11:28.784 INFO:teuthology.orchestra.run.smithi033.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.5 MB/s | 27 kB 00:00 2023-10-06T05:11:28.808 INFO:teuthology.orchestra.run.smithi033.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 7.0 MB/s | 161 kB 00:00 2023-10-06T05:11:28.832 INFO:teuthology.orchestra.run.smithi033.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 13 MB/s | 320 kB 00:00 2023-10-06T05:11:28.858 INFO:teuthology.orchestra.run.smithi033.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 17 MB/s | 457 kB 00:00 2023-10-06T05:11:28.990 INFO:teuthology.orchestra.run.smithi033.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 36 MB/s | 4.8 MB 00:00 2023-10-06T05:11:29.005 INFO:teuthology.orchestra.run.smithi033.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.8 MB/s | 39 kB 00:00 2023-10-06T05:11:29.065 INFO:teuthology.orchestra.run.smithi033.stdout:(100/120): python3-scipy-1.0.0-20.module+el8.1. 44 MB/s | 15 MB 00:00 2023-10-06T05:11:29.078 INFO:teuthology.orchestra.run.smithi033.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 9.2 MB/s | 688 kB 00:00 2023-10-06T05:11:29.083 INFO:teuthology.orchestra.run.smithi033.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 14 MB/s | 259 kB 00:00 2023-10-06T05:11:29.097 INFO:teuthology.orchestra.run.smithi033.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 12 MB/s | 225 kB 00:00 2023-10-06T05:11:29.101 INFO:teuthology.orchestra.run.smithi033.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 6.8 MB/s | 126 kB 00:00 2023-10-06T05:11:29.127 INFO:teuthology.orchestra.run.smithi033.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 7.2 MB/s | 187 kB 00:00 2023-10-06T05:11:29.152 INFO:teuthology.orchestra.run.smithi033.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 22 MB/s | 538 kB 00:00 2023-10-06T05:11:29.174 INFO:teuthology.orchestra.run.smithi033.stdout:(107/120): socat-1.7.3.3-2.el8.x86_64.rpm 13 MB/s | 302 kB 00:00 2023-10-06T05:11:29.193 INFO:teuthology.orchestra.run.smithi033.stdout:(108/120): python3-numpy-1.14.3-9.el8.x86_64.rp 39 MB/s | 3.7 MB 00:00 2023-10-06T05:11:29.196 INFO:teuthology.orchestra.run.smithi033.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 7.1 MB/s | 157 kB 00:00 2023-10-06T05:11:29.208 INFO:teuthology.orchestra.run.smithi033.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.2 MB/s | 110 kB 00:00 2023-10-06T05:11:29.217 INFO:teuthology.orchestra.run.smithi033.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 17 MB/s | 353 kB 00:00 2023-10-06T05:11:29.223 INFO:teuthology.orchestra.run.smithi033.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.4 MB/s | 20 kB 00:00 2023-10-06T05:11:29.237 INFO:teuthology.orchestra.run.smithi033.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.9 MB/s | 54 kB 00:00 2023-10-06T05:11:29.245 INFO:teuthology.orchestra.run.smithi033.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 25 MB/s | 709 kB 00:00 2023-10-06T05:11:29.359 INFO:teuthology.orchestra.run.smithi033.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 39 MB/s | 4.7 MB 00:00 2023-10-06T05:11:29.384 INFO:teuthology.orchestra.run.smithi033.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 33 MB/s | 4.6 MB 00:00 2023-10-06T05:11:29.396 INFO:teuthology.orchestra.run.smithi033.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2023-10-06T05:11:29.799 INFO:teuthology.orchestra.run.smithi033.stdout:(118/120): librados2-17.2.6-1274.g68e53a4a.el8. 8.2 MB/s | 3.4 MB 00:00 2023-10-06T05:11:29.813 INFO:teuthology.orchestra.run.smithi033.stdout:(119/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 8.5 MB/s | 3.5 MB 00:00 2023-10-06T05:11:33.177 INFO:teuthology.orchestra.run.smithi033.stdout:(120/120): ceph-test-17.2.6-1274.g68e53a4a.el8. 7.1 MB/s | 52 MB 00:07 2023-10-06T05:11:33.181 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:11:33.181 INFO:teuthology.orchestra.run.smithi033.stdout:Total 15 MB/s | 201 MB 00:13 2023-10-06T05:11:33.255 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2023-10-06T05:11:33.288 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-06T05:11:33.391 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2023-10-06T05:11:33.391 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2023-10-06T05:11:33.485 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T05:11:33.708 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T05:11:33.903 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T05:11:34.119 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T05:11:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T05:11:34.780 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T05:11:34.994 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T05:11:35.247 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2023-10-06T05:11:35.252 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2023-10-06T05:11:35.435 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T05:11:35.670 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T05:11:35.827 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T05:11:35.994 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T05:11:36.111 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T05:11:36.513 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T05:11:37.216 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T05:11:37.369 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T05:11:37.512 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T05:11:37.792 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T05:11:38.013 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T05:11:38.194 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T05:11:38.320 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T05:11:38.510 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T05:11:38.603 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T05:11:38.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T05:11:38.944 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T05:11:39.102 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T05:11:39.952 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T05:11:40.006 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2023-10-06T05:11:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T05:11:40.555 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T05:11:40.556 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T05:11:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T05:11:40.718 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T05:11:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T05:11:41.323 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T05:11:41.384 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T05:11:41.485 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T05:11:41.605 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T05:11:42.115 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T05:11:42.313 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T05:11:42.341 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T05:11:42.461 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T05:11:42.644 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T05:11:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T05:11:42.768 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T05:11:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T05:11:42.978 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T05:11:43.001 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T05:11:43.259 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T05:11:43.340 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T05:11:43.626 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T05:11:43.883 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T05:11:44.092 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T05:11:44.100 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T05:11:44.239 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T05:11:44.313 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T05:11:44.464 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T05:11:44.507 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T05:11:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T05:11:45.237 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T05:11:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T05:11:45.588 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T05:11:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T05:11:45.856 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T05:11:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T05:11:46.050 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T05:11:46.206 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T05:11:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T05:11:46.476 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T05:11:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T05:11:46.739 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T05:11:46.803 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T05:11:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T05:11:47.050 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T05:11:47.148 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T05:11:47.292 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T05:11:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T05:11:47.992 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T05:11:48.179 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T05:11:48.850 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T05:11:49.104 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T05:11:49.717 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T05:11:50.035 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T05:11:50.045 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T05:11:50.291 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T05:11:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T05:11:50.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T05:11:50.615 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T05:11:50.696 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T05:11:50.875 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T05:11:50.905 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T05:11:51.066 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T05:11:51.070 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T05:11:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T05:11:51.282 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T05:11:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T05:11:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T05:11:51.662 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T05:11:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T05:11:51.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T05:11:52.015 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T05:11:52.116 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T05:11:52.205 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T05:11:52.364 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T05:11:52.395 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T05:11:52.726 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T05:11:53.051 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T05:11:53.345 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T05:11:53.733 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T05:11:53.992 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T05:11:54.168 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T05:11:54.370 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T05:11:54.556 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T05:11:54.725 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T05:11:54.832 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T05:11:54.933 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T05:11:55.077 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T05:11:55.277 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T05:11:55.338 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T05:11:55.486 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T05:11:55.677 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T05:11:55.950 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T05:11:56.227 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T05:11:56.417 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T05:11:56.522 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T05:11:56.564 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T05:11:56.722 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T05:11:56.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T05:11:56.950 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T05:11:57.160 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T05:11:57.174 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T05:11:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T05:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T05:11:57.604 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T05:11:57.733 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T05:11:57.801 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T05:11:57.985 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T05:11:58.049 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T05:11:58.212 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T05:11:58.333 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T05:11:58.488 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T05:11:58.574 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T05:11:58.762 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T05:11:58.910 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T05:11:58.954 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T05:11:59.185 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T05:11:59.201 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T05:11:59.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T05:11:59.427 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T05:11:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T05:11:59.684 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T05:11:59.696 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T05:11:59.897 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T05:12:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T05:12:00.284 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T05:12:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T05:12:00.667 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T05:12:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T05:12:01.031 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T05:12:01.134 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T05:12:01.254 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T05:12:01.474 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T05:12:01.485 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T05:12:01.801 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T05:12:01.855 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T05:12:02.137 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T05:12:02.191 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T05:12:02.403 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T05:12:02.494 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T05:12:02.803 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T05:12:02.810 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T05:12:02.980 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T05:12:03.040 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T05:12:03.223 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T05:12:03.271 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T05:12:03.482 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T05:12:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T05:12:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T05:12:03.944 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T05:12:04.117 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T05:12:04.275 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T05:12:04.524 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T05:12:04.798 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T05:12:05.015 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T05:12:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T05:12:05.439 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T05:12:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T05:12:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T05:12:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T05:12:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T05:12:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T05:12:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T05:12:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T05:12:08.969 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T05:12:09.277 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T05:12:09.511 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T05:12:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T05:12:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T05:12:10.881 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T05:12:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T05:12:11.198 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T05:12:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T05:12:11.473 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T05:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T05:12:12.146 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T05:12:12.434 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T05:12:12.574 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T05:12:12.747 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T05:12:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T05:12:19.408 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T05:12:20.122 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T05:12:20.447 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T05:12:21.261 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T05:12:21.398 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T05:12:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2023-10-06T05:12:32.224 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-10-06T05:12:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:12:32.653 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T05:12:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T05:12:34.464 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T05:12:34.736 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T05:12:34.894 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T05:12:35.356 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T05:12:35.528 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T05:12:35.568 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T05:12:40.008 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T05:12:40.186 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T05:12:40.402 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T05:12:40.581 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T05:12:40.760 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T05:12:41.028 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T05:12:41.254 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T05:12:41.581 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T05:12:41.582 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2023-10-06T05:12:41.582 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2023-10-06T05:12:41.582 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2023-10-06T05:12:41.582 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /run 2023-10-06T05:12:41.582 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2023-10-06T05:12:41.582 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2023-10-06T05:12:41.583 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2023-10-06T05:12:41.583 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2023-10-06T05:12:41.583 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2023-10-06T05:12:41.583 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:12:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T05:12:42.372 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T05:12:42.576 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T05:12:42.718 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T05:12:43.136 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T05:12:43.338 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T05:12:43.407 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T05:12:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T05:12:43.578 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T05:12:43.772 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T05:12:43.968 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T05:12:44.254 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T05:12:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T05:12:44.925 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T05:12:45.091 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T05:12:45.133 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T05:12:45.292 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T05:12:45.555 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T05:12:45.678 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T05:12:45.834 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T05:12:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T05:12:46.117 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T05:12:46.530 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T05:12:46.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T05:12:49.823 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T05:12:50.021 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T05:12:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T05:12:50.449 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T05:12:50.652 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T05:12:50.932 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T05:12:51.108 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T05:12:52.077 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T05:12:52.295 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T05:12:52.401 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T05:12:52.477 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T05:12:52.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T05:12:52.624 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T05:12:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T05:12:53.308 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T05:12:53.379 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T05:12:53.551 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T05:12:53.581 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T05:12:53.645 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T05:12:54.077 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T05:12:54.279 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 111/122 2023-10-06T05:12:54.454 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T05:12:54.630 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T05:12:54.798 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T05:12:54.968 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T05:12:55.210 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T05:12:55.343 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T05:12:55.458 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T05:12:55.566 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T05:12:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T05:12:55.757 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T05:12:55.837 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T05:12:55.874 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T05:12:56.035 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T05:12:56.049 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T05:12:56.073 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T05:12:56.121 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T05:12:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T05:12:56.285 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T05:12:56.297 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T05:12:56.297 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T05:12:56.676 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T05:12:56.676 INFO:teuthology.orchestra.run.smithi167.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T05:12:56.676 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:12:56.720 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T05:12:56.909 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T05:12:57.559 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T05:12:57.559 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T05:12:57.560 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T05:12:57.561 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T05:12:57.562 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T05:12:57.570 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T05:12:57.571 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T05:12:57.572 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T05:12:57.573 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T05:12:57.574 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T05:12:57.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T05:12:57.576 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T05:12:57.577 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T05:12:57.578 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T05:12:59.942 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T05:12:59.942 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:51 ago on Fri 06 Oct 2023 05:11:08 AM UTC. 2023-10-06T05:13:02.256 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-10-06T05:13:02.388 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:02.388 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2023-10-06T05:13:02.388 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.388 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.388 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.389 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.390 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.5-12.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T05:13:02.391 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T05:13:02.392 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T05:13:02.393 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.394 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T05:13:02.395 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T05:13:02.396 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T05:13:02.397 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:02.398 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T05:13:02.399 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T05:13:02.399 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T05:13:02.399 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T05:13:02.399 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T05:13:02.399 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:02.399 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-06T05:13:02.667 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T05:13:02.715 DEBUG:teuthology.parallel:result is None 2023-10-06T05:13:04.075 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T05:13:04.493 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T05:13:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 110/122 2023-10-06T05:13:05.000 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T05:13:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T05:13:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T05:13:05.548 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T05:13:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T05:13:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T05:13:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T05:13:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T05:13:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T05:13:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T05:13:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T05:13:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T05:13:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T05:13:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T05:13:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T05:13:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T05:13:07.656 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T05:13:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T05:13:07.656 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:08.544 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T05:13:08.588 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T05:13:08.588 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T05:13:08.588 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T05:13:08.588 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T05:13:08.588 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T05:13:08.589 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T05:13:08.589 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T05:13:08.589 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T05:13:08.591 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T05:13:08.591 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T05:13:08.592 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T05:13:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T05:13:08.601 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T05:13:08.601 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T05:13:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T05:13:08.603 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T05:13:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T05:13:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T05:13:08.606 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T05:13:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T05:13:08.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T05:13:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T05:13:11.055 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T05:13:11.055 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:02 ago on Fri 06 Oct 2023 05:11:09 AM UTC. 2023-10-06T05:13:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:Installed products updated. 2023-10-06T05:13:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T05:13:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.5-12.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T05:13:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T05:13:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T05:13:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T05:13:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T05:13:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T05:13:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T05:13:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T05:13:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T05:13:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:13.333 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2023-10-06T05:13:13.658 DEBUG:teuthology.parallel:result is None 2023-10-06T05:13:13.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:13:13.798 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:13:13.846 INFO:teuthology.orchestra.run.smithi033.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T05:13:13.846 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T05:13:13.846 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T05:13:13.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:13:14.198 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:13:14.251 INFO:teuthology.orchestra.run.smithi167.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T05:13:14.251 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T05:13:14.251 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T05:13:14.252 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T05:13:14.252 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:14.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T05:13:14.337 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:14.337 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T05:13:14.422 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T05:13:14.423 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:14.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T05:13:14.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T05:13:14.597 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:14.597 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T05:13:14.658 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T05:13:14.758 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T05:13:14.759 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:14.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T05:13:14.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T05:13:14.924 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:14.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T05:13:14.986 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T05:13:15.085 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T05:13:15.086 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:15.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T05:13:15.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T05:13:15.249 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:15.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T05:13:15.310 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T05:13:15.410 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T05:13:15.470 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T05:13:15.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2023-10-06T05:13:15.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2023-10-06T05:13:15.536 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T05:13:15.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T05:13:15.540 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T05:13:15.600 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T05:13:15.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T05:13:15.605 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T05:13:15.672 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T05:13:15.673 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T05:13:15.673 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T05:13:15.673 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:15.673 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T05:13:15.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T05:13:16.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T05:13:16.189 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:16.189 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T05:13:16.226 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T05:13:16.773 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T05:13:16.842 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T05:13:16.842 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T05:13:16.842 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412318', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsN2KlXCfhDPmYLfoVfF8ye4qHC8UXV+jzCMG27elNHskyYRXcyMSrNycVha+feEaRJ54JvvraDvWOgcUF7W2I=', 'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGh2SIXiMe51UgQvKWo7LPq52jMpsA7kU3YeiWFADY9lDVWX3mJJIxTnL9dtfZo12gEyWHIUJoC+Wr3mferqT6U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_recovery']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T05:13:16.843 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T05:13:16.847 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T05:13:16.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T05:13:16.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T05:13:16.986 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:16.986 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T05:13:17.023 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T05:13:17.023 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2023-10-06T05:13:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T05:13:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:17.101 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T05:13:17.101 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:17.101 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:17.101 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 05:13:07.974179043 +0000 2023-10-06T05:13:17.102 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 05:10:12.928029990 +0000 2023-10-06T05:13:17.102 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 05:10:12.928029990 +0000 2023-10-06T05:13:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T05:13:17.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T05:13:17.200 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T05:13:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T05:13:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000554528 s, 923 kB/s 2023-10-06T05:13:17.203 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T05:13:17.242 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2023-10-06T05:13:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T05:13:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:17.320 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T05:13:17.320 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:17.320 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:17.320 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 05:13:07.974179043 +0000 2023-10-06T05:13:17.321 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 05:10:13.748017366 +0000 2023-10-06T05:13:17.321 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 05:10:13.748017366 +0000 2023-10-06T05:13:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T05:13:17.321 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T05:13:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T05:13:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T05:13:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000298113 s, 1.7 MB/s 2023-10-06T05:13:17.425 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T05:13:17.464 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2023-10-06T05:13:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T05:13:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 05:13:07.974179043 +0000 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 05:10:14.550005019 +0000 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 05:10:14.550005019 +0000 2023-10-06T05:13:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T05:13:17.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T05:13:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T05:13:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T05:13:17.640 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000301026 s, 1.7 MB/s 2023-10-06T05:13:17.642 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T05:13:17.682 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 05:13:07.974179043 +0000 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 05:10:15.352992657 +0000 2023-10-06T05:13:17.760 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 05:10:15.352992657 +0000 2023-10-06T05:13:17.761 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T05:13:17.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T05:13:17.859 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T05:13:17.859 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T05:13:17.859 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000305744 s, 1.7 MB/s 2023-10-06T05:13:17.862 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T05:13:17.900 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:17.900 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:17.900 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T05:13:17.942 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T05:13:17.942 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:12:56.977128970 +0000 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:10:00.923966024 +0000 2023-10-06T05:13:18.022 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:10:00.923966024 +0000 2023-10-06T05:13:18.023 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:13:18.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T05:13:18.124 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T05:13:18.124 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T05:13:18.124 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000386582 s, 1.3 MB/s 2023-10-06T05:13:18.126 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T05:13:18.166 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2023-10-06T05:13:18.243 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T05:13:18.243 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:18.243 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-06T05:13:18.243 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:18.243 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:18.244 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:12:56.978128953 +0000 2023-10-06T05:13:18.244 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:10:01.707952692 +0000 2023-10-06T05:13:18.244 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:10:01.707952692 +0000 2023-10-06T05:13:18.244 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:13:18.244 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T05:13:18.341 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T05:13:18.341 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T05:13:18.341 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000301912 s, 1.7 MB/s 2023-10-06T05:13:18.343 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T05:13:18.381 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2023-10-06T05:13:18.458 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T05:13:18.458 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:18.458 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-06T05:13:18.458 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:18.458 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:18.458 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:12:56.978128953 +0000 2023-10-06T05:13:18.459 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:10:02.465939803 +0000 2023-10-06T05:13:18.459 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:10:02.465939803 +0000 2023-10-06T05:13:18.459 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:13:18.459 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T05:13:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T05:13:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T05:13:18.556 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000302453 s, 1.7 MB/s 2023-10-06T05:13:18.559 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T05:13:18.597 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:12:56.978128953 +0000 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:10:03.244926556 +0000 2023-10-06T05:13:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:10:03.244926556 +0000 2023-10-06T05:13:18.675 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:13:18.675 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T05:13:18.771 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T05:13:18.772 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T05:13:18.772 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000344603 s, 1.5 MB/s 2023-10-06T05:13:18.775 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T05:13:18.813 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:18.813 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T05:13:18.813 INFO:tasks.ceph:Generating config... 2023-10-06T05:13:18.814 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T05:13:18.814 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T05:13:18.814 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T05:13:18.815 INFO:tasks.ceph:[client] pid file = 2023-10-06T05:13:18.815 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T05:13:18.815 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T05:13:18.815 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T05:13:18.815 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T05:13:18.816 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T05:13:18.816 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T05:13:18.816 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T05:13:18.816 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T05:13:18.816 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T05:13:18.817 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T05:13:18.818 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T05:13:18.818 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T05:13:18.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T05:13:18.919 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T05:13:18.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T05:13:19.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:13:19.072 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.167'), ('mon.c', '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]')] 2023-10-06T05:13:19.073 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T05:13:19.074 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:19.074 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T05:13:19.110 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.167 --addv c '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid abad2d63-0b71-4f3e-871a-ad3209a2fd6e 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = octopus 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:fsid abad2d63-0b71-4f3e-871a-ad3209a2fd6e 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2023-10-06T05:13:19.222623+0000 2023-10-06T05:13:19.223 INFO:teuthology.orchestra.run.smithi033.stdout:created 2023-10-06T05:13:19.222623+0000 2023-10-06T05:13:19.224 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 15 (octopus) 2023-10-06T05:13:19.224 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2023-10-06T05:13:19.224 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2023-10-06T05:13:19.224 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2023-10-06T05:13:19.224 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2023-10-06T05:13:19.224 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T05:13:19.225 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T05:13:19.261 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID abad2d63-0b71-4f3e-871a-ad3209a2fd6e... 2023-10-06T05:13:19.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:13:19.306 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:13:19.414 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2023-10-06T05:13:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:13:19.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:13:19.417 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2023-10-06T05:13:19.418 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on skipped message = False 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = abad2d63-0b71-4f3e-871a-ad3209a2fd6e 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2023-10-06T05:13:19.419 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:13:19.420 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: bdev async discard = True 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: bdev enable discard = True 2023-10-06T05:13:19.421 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore allocator = bitmap 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 1/20 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 1/20 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 4/10 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2023-10-06T05:13:19.422 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:13:19.423 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2023-10-06T05:13:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:13:19.426 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = "" 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2023-10-06T05:13:19.427 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2023-10-06T05:13:19.428 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.429 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2023-10-06T05:13:19.430 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:13:19.431 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2023-10-06T05:13:19.432 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.33,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on skipped message = False 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = abad2d63-0b71-4f3e-871a-ad3209a2fd6e 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.433 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2023-10-06T05:13:19.434 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.435 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: bdev async discard = True 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: bdev enable discard = True 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore allocator = bitmap 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 1/20 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 1/20 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2023-10-06T05:13:19.436 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 4/10 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2023-10-06T05:13:19.437 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-10-06T05:13:19.438 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:13:19.439 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2023-10-06T05:13:19.440 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2023-10-06T05:13:19.441 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2023-10-06T05:13:19.442 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2023-10-06T05:13:19.443 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2023-10-06T05:13:19.443 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2023-10-06T05:13:19.443 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2023-10-06T05:13:19.448 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T05:13:19.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T05:13:19.548 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T05:13:19.548 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:19.549 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T05:13:19.586 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:19.586 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T05:13:19.661 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:13:19.662 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:19.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T05:13:19.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:13:19.789 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:19.790 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:19.865 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:13:19.865 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:19.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T05:13:19.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:13:19.953 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:19.953 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:20.029 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T05:13:20.029 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T05:13:20.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T05:13:20.141 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T05:13:20.147 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T05:13:20.261 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T05:13:20.267 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T05:13:20.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T05:13:20.379 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T05:13:20.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T05:13:20.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T05:13:20.558 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T05:13:20.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T05:13:20.624 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T05:13:20.735 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T05:13:20.740 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T05:13:20.802 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T05:13:20.910 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T05:13:20.915 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T05:13:20.976 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T05:13:20.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T05:13:21.062 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T05:13:21.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T05:13:21.178 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T05:13:21.209 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T05:13:21.292 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T05:13:21.323 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T05:13:21.405 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T05:13:21.436 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T05:13:21.436 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T05:13:21.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T05:13:21.499 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:21.499 INFO:tasks.ceph:role: osd.0 2023-10-06T05:13:21.499 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:13:21.499 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T05:13:21.568 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:21.568 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:21.568 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:21.568 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T05:13:21.568 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:21.569 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:21.569 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:21.569 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:21.569 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:21.569 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:21.996 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-10-06T05:13:22.000 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T05:13:22.000 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T05:13:22.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T05:13:22.265 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T05:13:22.327 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:22.327 INFO:tasks.ceph:role: osd.1 2023-10-06T05:13:22.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:13:22.327 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T05:13:22.396 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:22.396 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:22.396 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:22.397 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:22.812 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-10-06T05:13:23.675 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T05:13:23.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T05:13:23.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T05:13:23.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T05:13:23.996 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:23.996 INFO:tasks.ceph:role: osd.2 2023-10-06T05:13:23.996 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:13:23.996 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T05:13:24.065 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T05:13:24.250 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:24.251 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:24.251 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:24.251 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:24.500 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-10-06T05:13:24.504 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T05:13:24.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T05:13:24.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T05:13:24.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T05:13:24.771 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:24.771 INFO:tasks.ceph:role: osd.3 2023-10-06T05:13:24.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:13:24.771 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T05:13:24.841 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T05:13:24.842 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:24.842 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:24.842 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:24.842 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:25.280 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2023-10-06T05:13:25.284 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T05:13:25.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T05:13:25.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T05:13:25.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:26.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:26.573+0000 7fc1ba68c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T05:13:26.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:26.574+0000 7fc1ba68c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T05:13:26.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:26.574+0000 7fc1ba68c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:13:26.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:26.574+0000 7fc1ba68c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:13:26.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:26.574+0000 7fc1ba68c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:13:26.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:26.576+0000 7fc1ba68c540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T05:13:27.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:27.120+0000 7fc1ba68c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:27.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:27.881+0000 7fc1ba68c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:28.645+0000 7fc1ba68c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:29.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T05:13:29.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:29.628+0000 7f308f98d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T05:13:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:29.628+0000 7f308f98d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T05:13:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:29.628+0000 7f308f98d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:13:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:29.628+0000 7f308f98d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:13:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:29.628+0000 7f308f98d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:13:29.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:29.630+0000 7f308f98d540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T05:13:30.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:30.170+0000 7f308f98d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:30.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:30.936+0000 7f308f98d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:31.701+0000 7f308f98d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:32.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T05:13:32.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:32.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:32.691+0000 7ff2978b2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T05:13:32.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:32.691+0000 7ff2978b2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T05:13:32.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:32.691+0000 7ff2978b2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:13:32.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:32.691+0000 7ff2978b2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:13:32.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:32.691+0000 7ff2978b2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:13:32.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:32.693+0000 7ff2978b2540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T05:13:33.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:33.231+0000 7ff2978b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:34.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:34.006+0000 7ff2978b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:34.767+0000 7ff2978b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:35.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T05:13:35.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:35.741+0000 7fca78f93540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T05:13:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:35.742+0000 7fca78f93540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T05:13:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:35.742+0000 7fca78f93540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:13:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:35.742+0000 7fca78f93540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:13:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:35.742+0000 7fca78f93540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:13:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:35.743+0000 7fca78f93540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T05:13:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:36.282+0000 7fca78f93540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:37.044+0000 7fca78f93540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:37.813+0000 7fca78f93540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:38.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T05:13:38.693 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T05:13:38.762 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:38.762 INFO:tasks.ceph:role: osd.4 2023-10-06T05:13:38.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:13:38.762 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T05:13:38.832 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:38.832 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:38.833 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:39.269 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-06T05:13:39.275 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T05:13:39.275 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T05:13:39.470 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T05:13:39.542 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T05:13:39.604 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:39.604 INFO:tasks.ceph:role: osd.5 2023-10-06T05:13:39.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:13:39.604 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T05:13:39.671 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:39.672 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:40.080 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-06T05:13:40.084 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T05:13:40.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T05:13:40.229 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T05:13:40.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T05:13:40.357 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:40.357 INFO:tasks.ceph:role: osd.6 2023-10-06T05:13:40.357 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:13:40.357 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T05:13:40.425 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:40.425 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:40.425 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:40.425 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T05:13:40.426 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:40.426 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:40.426 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:40.426 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:40.426 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:40.426 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:40.867 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-06T05:13:40.871 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T05:13:40.871 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T05:13:41.058 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T05:13:41.127 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T05:13:41.189 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:13:41.189 INFO:tasks.ceph:role: osd.7 2023-10-06T05:13:41.189 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:13:41.189 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T05:13:41.257 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:13:41.257 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:13:41.257 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:13:41.258 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T05:13:41.258 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:13:41.258 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:13:41.258 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:13:41.258 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:13:41.258 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:13:41.259 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:13:41.689 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-06T05:13:41.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T05:13:41.693 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T05:13:41.865 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T05:13:41.932 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:42.044 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.042+0000 7f90517a4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T05:13:42.044 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.042+0000 7f90517a4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T05:13:42.045 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.042+0000 7f90517a4540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:13:42.045 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.042+0000 7f90517a4540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:13:42.045 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.042+0000 7f90517a4540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:13:42.046 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.044+0000 7f90517a4540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T05:13:42.582 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:42.580+0000 7f90517a4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:43.344 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:43.343+0000 7f90517a4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:44.108 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:44.107+0000 7f90517a4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:44.923 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T05:13:44.989 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:45.083 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.080+0000 7f6df5d87540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T05:13:45.083 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.081+0000 7f6df5d87540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T05:13:45.083 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.081+0000 7f6df5d87540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:13:45.083 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.081+0000 7f6df5d87540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:13:45.083 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.081+0000 7f6df5d87540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:13:45.084 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.082+0000 7f6df5d87540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T05:13:45.625 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:45.624+0000 7f6df5d87540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:46.384 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:46.383+0000 7f6df5d87540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:46.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:13:46.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:13:47.149 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:47.147+0000 7f6df5d87540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:47.960 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T05:13:48.021 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.110+0000 7f5ee06ac540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T05:13:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.110+0000 7f5ee06ac540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T05:13:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.110+0000 7f5ee06ac540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:13:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.110+0000 7f5ee06ac540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:13:48.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.110+0000 7f5ee06ac540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:13:48.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.112+0000 7f5ee06ac540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T05:13:48.650 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:48.649+0000 7f5ee06ac540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:49.415 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:49.413+0000 7f5ee06ac540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:50.177 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:50.175+0000 7f5ee06ac540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:50.993 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T05:13:51.056 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:51.148 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.146+0000 7ff511892540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T05:13:51.148 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.146+0000 7ff511892540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T05:13:51.148 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.146+0000 7ff511892540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:13:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.146+0000 7ff511892540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:13:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.146+0000 7ff511892540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:13:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.148+0000 7ff511892540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T05:13:51.685 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:51.684+0000 7ff511892540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:52.447 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:52.445+0000 7ff511892540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:53.210 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:53.208+0000 7ff511892540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:13:54.024 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T05:13:54.086 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T05:13:54.086 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T05:13:54.155 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T05:13:54.259 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T05:13:54.361 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T05:13:54.463 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T05:13:54.564 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T05:13:54.665 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:54.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T05:13:54.767 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:54.767 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T05:13:54.830 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:54.831 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T05:13:54.932 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:54.932 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T05:13:55.033 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:55.034 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T05:13:55.134 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:55.134 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T05:13:55.233 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:55.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T05:13:55.333 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:55.334 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T05:13:55.434 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:55.434 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T05:13:55.472 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:55.472 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T05:13:55.550 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:55.550 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T05:13:55.585 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:55.585 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T05:13:55.661 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T05:13:55.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T05:13:55.666 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T05:13:55.722 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2023-10-06T05:13:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg== 2023-10-06T05:13:55.722 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2023-10-06T05:13:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg== 2023-10-06T05:13:55.722 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2023-10-06T05:13:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg== 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ== 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw== 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ== 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ== 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ== 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2023-10-06T05:13:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA== 2023-10-06T05:13:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:[client.2] 2023-10-06T05:13:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A== 2023-10-06T05:13:55.725 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2023-10-06T05:13:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA== 2023-10-06T05:13:55.725 INFO:teuthology.orchestra.run.smithi033.stdout:[client.3] 2023-10-06T05:13:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA== 2023-10-06T05:13:55.757 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2023-10-06T05:13:55.757 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg== 2023-10-06T05:13:55.757 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2023-10-06T05:13:55.757 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg== 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg== 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ== 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw== 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ== 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ== 2023-10-06T05:13:55.758 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ== 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA== 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ== 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg== 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw== 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg== 2023-10-06T05:13:55.759 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw== 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA== 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A== 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA== 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2023-10-06T05:13:55.760 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA== 2023-10-06T05:13:55.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:13:55.770 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:13:55.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:55.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:55.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:56.002 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:56.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.105 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.227 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.322 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.326 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.424 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:13:56.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:13:56.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:56.624 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:56.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:56.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:13:56.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:56.926 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:57.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:57.026 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:57.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:57.153 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:13:57.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.252 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.354 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.457 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:13:57.656 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T05:13:57.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T05:13:57.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:13:57.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T05:13:57.890 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T05:13:57.952 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:13:58.061 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T05:13:58.123 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T05:13:58.184 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:13:58.294 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T05:13:58.355 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:58.360 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:13:58.396 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T05:13:58.396 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T05:13:58.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T05:13:58.439 INFO:tasks.ceph.mon.a:Started 2023-10-06T05:13:58.440 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T05:13:58.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T05:13:58.444 INFO:tasks.ceph.mon.b:Started 2023-10-06T05:13:58.444 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T05:13:58.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T05:13:58.449 INFO:tasks.ceph.mon.c:Started 2023-10-06T05:13:58.449 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T05:13:58.449 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T05:13:58.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T05:13:58.453 INFO:tasks.ceph.mgr.y:Started 2023-10-06T05:13:58.453 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T05:13:58.454 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T05:13:58.458 INFO:tasks.ceph.mgr.x:Started 2023-10-06T05:13:58.458 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T05:13:58.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T05:13:58.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.645+0000 7fced5d9d700 1 Processor -- start 2023-10-06T05:13:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.645+0000 7fced5d9d700 1 -- start start 2023-10-06T05:13:58.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.646+0000 7fced5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fced00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.646+0000 7fced5d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 0x7fced015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.646+0000 7fced5d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced015d660 con 0x7fced015a650 2023-10-06T05:13:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.646+0000 7fced5d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced00ce890 con 0x7fced00cd5e0 2023-10-06T05:13:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.646+0000 7fced5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced015aac0 con 0x7fced00af540 2023-10-06T05:13:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 0x7fced015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 0x7fced015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:56748/0 (socket says 172.21.15.33:56748) 2023-10-06T05:13:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 learned_addr learned my addr 172.21.15.33:0/3591214783 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced4d9b700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fced00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fcecffff700 1 --1- 172.21.15.33:0/3591214783 >> v1:172.21.15.33:6789/0 conn(0x7fced015a650 0x7fced00cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49846/0 (socket says 172.21.15.33:49846) 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 msgr2=0x7fced015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 msgr2=0x7fced015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 --2- 172.21.15.33:0/3591214783 >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 0x7fced015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced559c700 1 --2- 172.21.15.33:0/3591214783 >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 0x7fced015d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced4d9b700 1 -- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 msgr2=0x7fced00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced4d9b700 1 -- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 msgr2=0x7fced00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced4d9b700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fced00af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:13:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.647+0000 7fced4d9b700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fced00af910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:13:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656409129 0 0) 0x7fced015d660 con 0x7fced015a650 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec0003570 con 0x7fced015a650 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2616019430 0 0) 0x7fcec0003570 con 0x7fced015a650 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 msgr2=0x7fced00af910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fced00af910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 msgr2=0x7fced015d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 --2- 172.21.15.33:0/3591214783 >> v2:172.21.15.167:3300/0 conn(0x7fced00cd5e0 0x7fced015d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.841+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced015d890 con 0x7fced015a650 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fcebc001470 con 0x7fced015a650 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 >> v1:172.21.15.33:6789/0 conn(0x7fced015a650 legacy=0x7fced00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fcec0003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004160 0x7fcec0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0004d50 0x7fcec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00ce890 con 0x7fcec0004d50 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced015aac0 con 0x7fcec0004160 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec000bd20 con 0x7fced00af540 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced559c700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0004d50 0x7fcec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fcecffff700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004160 0x7fcec0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced4d9b700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fcec0003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 msgr2=0x7fcec0003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced559c700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fcec0003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004160 msgr2=0x7fcec0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced559c700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004160 0x7fcec0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0003570 con 0x7fcec0004d50 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.842+0000 7fced4d9b700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced00af540 0x7fcec0003ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.843+0000 7fced559c700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0004d50 0x7fcec00051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcec8009410 tx=0x7fcec80091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e3205e99385eed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.843+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80097c0 con 0x7fcec0004d50 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.843+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec800f040 con 0x7fcec0004d50 2023-10-06T05:13:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.843+0000 7fcecf7fe700 1 -- 172.21.15.33:0/3591214783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8007d70 con 0x7fcec0004d50 2023-10-06T05:13:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced5d9d700 1 -- 172.21.15.33:0/3591214783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0004d50 msgr2=0x7fcec00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced5d9d700 1 --2- 172.21.15.33:0/3591214783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0004d50 0x7fcec00051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcec8009410 tx=0x7fcec80091c0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced559c700 1 -- 172.21.15.33:0/3591214783 reap_dead start 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced5d9d700 1 -- 172.21.15.33:0/3591214783 shutdown_connections 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced5d9d700 1 -- 172.21.15.33:0/3591214783 >> 172.21.15.33:0/3591214783 conn(0x7fced006d9b0 msgr2=0x7fced006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced5d9d700 1 -- 172.21.15.33:0/3591214783 shutdown_connections 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.844+0000 7fced5d9d700 1 -- 172.21.15.33:0/3591214783 wait complete. 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.845+0000 7fced5d9d700 1 Processor -- start 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.845+0000 7fced5d9d700 1 -- start start 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.846+0000 7fced5d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004d50 0x7fced00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.846+0000 7fced5d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 0x7fced00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.846+0000 7fced5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced01570e0 0x7fced006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.846+0000 7fced5d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00b4890 con 0x7fced00c8ef0 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.846+0000 7fced5d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00c9360 con 0x7fcec0004d50 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.846+0000 7fced5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced00c95f0 con 0x7fced01570e0 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 0x7fced00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 0x7fced00b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50430/0 (socket says 172.21.15.33:50430) 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 -- 172.21.15.33:0/290852526 learned_addr learned my addr 172.21.15.33:0/290852526 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fced4d9b700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004d50 0x7fced00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 -- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced01570e0 msgr2=0x7fced006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced01570e0 0x7fced006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 -- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004d50 msgr2=0x7fced00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004d50 0x7fced00b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 -- 172.21.15.33:0/290852526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced00c98d0 con 0x7fced00c8ef0 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fcecffff700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 0x7fced00b5530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e8d0 tx=0x7fced00ce800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00a520 con 0x7fced00c8ef0 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc010040 con 0x7fced00c8ef0 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00a8d0 con 0x7fced00c8ef0 2023-10-06T05:13:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.847+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced00b4240 con 0x7fced00c8ef0 2023-10-06T05:13:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.848+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced0157560 con 0x7fced00c8ef0 2023-10-06T05:13:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.848+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fcebc00ac40 con 0x7fced00c8ef0 2023-10-06T05:13:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.848+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcebc01ee10 con 0x7fced00c8ef0 2023-10-06T05:13:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.849+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced004c380 con 0x7fced00c8ef0 2023-10-06T05:13:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.851+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fcebc019070 con 0x7fced00c8ef0 2023-10-06T05:13:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.931+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fced015db70 con 0x7fced00c8ef0 2023-10-06T05:13:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.938+0000 7fceccff9700 1 -- 172.21.15.33:0/290852526 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcebc01e5d0 con 0x7fced00c8ef0 2023-10-06T05:13:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 msgr2=0x7fced00b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 0x7fced00b5530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e8d0 tx=0x7fced00ce800 comp rx=0 tx=0).stop 2023-10-06T05:13:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 shutdown_connections 2023-10-06T05:13:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced01570e0 0x7fced006c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcec0004d50 0x7fced00b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 --2- 172.21.15.33:0/290852526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced00c8ef0 0x7fced00b5530 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.939+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 >> 172.21.15.33:0/290852526 conn(0x7fced006d9b0 msgr2=0x7fced0158030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:13:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.940+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 shutdown_connections 2023-10-06T05:13:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:13:58.940+0000 7fced5d9d700 1 -- 172.21.15.33:0/290852526 wait complete. 2023-10-06T05:13:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2023-10-06T05:13:58.957 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T05:13:58.958 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:58.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T05:13:59.019 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:59.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T05:13:59.120 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:59.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T05:13:59.224 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T05:13:59.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T05:13:59.327 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:59.327 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T05:13:59.389 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:59.389 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T05:13:59.490 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:59.491 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T05:13:59.593 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T05:13:59.593 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T05:13:59.698 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new c4fc2102-e639-40b8-914d-00d54ee9b747 0 2023-10-06T05:13:59.918 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.916+0000 7fd67759e700 1 Processor -- start 2023-10-06T05:13:59.918 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.917+0000 7fd67759e700 1 -- start start 2023-10-06T05:13:59.922 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6780af540 0x7fd6780af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67815a650 0x7fd67815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6780cced0 0x7fd6780cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780cd890 con 0x7fd6780cced0 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67815d660 con 0x7fd6780af540 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780ce890 con 0x7fd67815a650 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67659c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6780af540 0x7fd6780af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67815a650 0x7fd67815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67659c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6780af540 0x7fd6780af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49398/0 (socket says 172.21.15.167:49398) 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67815a650 0x7fd67815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49664/0 (socket says 172.21.15.167:49664) 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd67659c700 1 -- 172.21.15.167:0/1143118813 learned_addr learned my addr 172.21.15.167:0/1143118813 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:13:59.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.918+0000 7fd676d9d700 1 --2- 172.21.15.167:0/1143118813 >> v2:172.21.15.33:3300/0 conn(0x7fd6780cced0 0x7fd6780cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67659c700 1 -- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67815a650 msgr2=0x7fd67815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67659c700 1 --2- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67815a650 0x7fd67815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67659c700 1 -- 172.21.15.167:0/1143118813 >> v2:172.21.15.33:3300/0 conn(0x7fd6780cced0 msgr2=0x7fd6780cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67659c700 1 --2- 172.21.15.167:0/1143118813 >> v2:172.21.15.33:3300/0 conn(0x7fd6780cced0 0x7fd6780cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67659c700 1 -- 172.21.15.167:0/1143118813 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67815aac0 con 0x7fd6780af540 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67659c700 1 --2- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780af540 0x7fd6780af910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6780c7530 tx=0x7fd668009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e1e301b676af29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67559a700 1 -- 172.21.15.167:0/1143118813 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668009e70 con 0x7fd6780af540 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67559a700 1 -- 172.21.15.167:0/1143118813 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800b860 con 0x7fd6780af540 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.919+0000 7fd67559a700 1 -- 172.21.15.167:0/1143118813 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800bc00 con 0x7fd6780af540 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 -- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780af540 msgr2=0x7fd6780af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:59.924 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 --2- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780af540 0x7fd6780af910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6780c7530 tx=0x7fd668009250 comp rx=0 tx=0).stop 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 -- 172.21.15.167:0/1143118813 shutdown_connections 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 --2- 172.21.15.167:0/1143118813 >> v2:172.21.15.33:3300/0 conn(0x7fd6780cced0 0x7fd6780cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 --2- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780af540 0x7fd6780af910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 --2- 172.21.15.167:0/1143118813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67815a650 0x7fd67815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 -- 172.21.15.167:0/1143118813 >> 172.21.15.167:0/1143118813 conn(0x7fd67806d9b0 msgr2=0x7fd67806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 -- 172.21.15.167:0/1143118813 shutdown_connections 2023-10-06T05:13:59.925 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.920+0000 7fd67759e700 1 -- 172.21.15.167:0/1143118813 wait complete. 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.921+0000 7fd67759e700 1 Processor -- start 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 -- start start 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6780af540 0x7fd6780c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 0x7fd678157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67815a650 0x7fd6780d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780b4370 con 0x7fd67815a650 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780c7db0 con 0x7fd6780cced0 2023-10-06T05:13:59.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.922+0000 7fd67759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780c8090 con 0x7fd6780af540 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 0x7fd678157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd67659c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6780af540 0x7fd6780c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 0x7fd678157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49406/0 (socket says 172.21.15.167:49406) 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 -- 172.21.15.167:0/389123499 learned_addr learned my addr 172.21.15.167:0/389123499 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd676d9d700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67815a650 0x7fd6780d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 -- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6780af540 msgr2=0x7fd6780c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6780af540 0x7fd6780c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 -- 172.21.15.167:0/389123499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67815a650 msgr2=0x7fd6780d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67815a650 0x7fd6780d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 -- 172.21.15.167:0/389123499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680094f0 con 0x7fd6780cced0 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd676d9d700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67815a650 0x7fd6780d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:13:59.929 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd675d9b700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 0x7fd678157630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00ec80 tx=0x7fd66c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.923+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00c850 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.924+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c00c9d0 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.924+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6780c8370 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.924+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00cd90 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.924+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6780c7ac0 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.924+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd66c005c80 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.924+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd66c006370 con 0x7fd6780cced0 2023-10-06T05:13:59.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.925+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67804c380 con 0x7fd6780cced0 2023-10-06T05:13:59.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:13:59.927+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd66c021060 con 0x7fd6780cced0 2023-10-06T05:14:00.014 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.012+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c4fc2102-e639-40b8-914d-00d54ee9b747", "id": 0} v 0) v1 -- 0x7fd67815ce90 con 0x7fd6780cced0 2023-10-06T05:14:00.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.019+0000 7fd6667fc700 1 -- 172.21.15.167:0/389123499 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c4fc2102-e639-40b8-914d-00d54ee9b747", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd66c017070 con 0x7fd6780cced0 2023-10-06T05:14:00.021 INFO:teuthology.orchestra.run.smithi167.stdout:0 2023-10-06T05:14:00.022 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.020+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 msgr2=0x7fd678157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.020+0000 7fd67759e700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 0x7fd678157630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00ec80 tx=0x7fd66c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 shutdown_connections 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67815a650 0x7fd6780d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6780af540 0x7fd6780c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 --2- 172.21.15.167:0/389123499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6780cced0 0x7fd678157630 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 >> 172.21.15.167:0/389123499 conn(0x7fd67806d9b0 msgr2=0x7fd6780cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 shutdown_connections 2023-10-06T05:14:00.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.021+0000 7fd67759e700 1 -- 172.21.15.167:0/389123499 wait complete. 2023-10-06T05:14:00.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new a4410d06-3707-4e1d-9194-02f96c4861c8 1 2023-10-06T05:14:00.217 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.215+0000 7f683759e700 1 Processor -- start 2023-10-06T05:14:00.217 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.215+0000 7f683759e700 1 -- start start 2023-10-06T05:14:00.219 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68380af540 0x7f68380af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68380cd5e0 0x7f683815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683815d660 con 0x7f683815a650 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380ce890 con 0x7f68380cd5e0 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683815aac0 con 0x7f68380af540 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68380af540 0x7f68380af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68380af540 0x7f68380af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49410/0 (socket says 172.21.15.167:49410) 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 -- 172.21.15.167:0/279274504 learned_addr learned my addr 172.21.15.167:0/279274504 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f6836d9d700 1 --2- 172.21.15.167:0/279274504 >> v2:172.21.15.33:3300/0 conn(0x7f68380cd5e0 0x7f683815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 -- 172.21.15.167:0/279274504 >> v2:172.21.15.33:3300/0 conn(0x7f68380cd5e0 msgr2=0x7f683815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 --2- 172.21.15.167:0/279274504 >> v2:172.21.15.33:3300/0 conn(0x7f68380cd5e0 0x7f683815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 -- 172.21.15.167:0/279274504 >> v1:172.21.15.33:6789/0 conn(0x7f683815a650 legacy=0x7f68380cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683659c700 1 -- 172.21.15.167:0/279274504 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683815d890 con 0x7f68380af540 2023-10-06T05:14:00.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.217+0000 7f683559a700 1 -- 172.21.15.167:0/279274504 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196740502 0 0) 0x7f683815d660 con 0x7f683815a650 2023-10-06T05:14:00.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.218+0000 7f683659c700 1 --2- 172.21.15.167:0/279274504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380af540 0x7f68380af910 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f68380c7530 tx=0x7f6828009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e6f6ff632a8cbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:00.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.218+0000 7f683559a700 1 -- 172.21.15.167:0/279274504 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828009e70 con 0x7f68380af540 2023-10-06T05:14:00.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.218+0000 7f683559a700 1 -- 172.21.15.167:0/279274504 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800b860 con 0x7f68380af540 2023-10-06T05:14:00.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.218+0000 7f683559a700 1 -- 172.21.15.167:0/279274504 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800bc00 con 0x7f68380af540 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 -- 172.21.15.167:0/279274504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380af540 msgr2=0x7f68380af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 --2- 172.21.15.167:0/279274504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380af540 0x7f68380af910 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f68380c7530 tx=0x7f6828009250 comp rx=0 tx=0).stop 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 -- 172.21.15.167:0/279274504 shutdown_connections 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 --2- 172.21.15.167:0/279274504 >> v2:172.21.15.33:3300/0 conn(0x7f68380cd5e0 0x7f683815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 --2- 172.21.15.167:0/279274504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380af540 0x7f68380af910 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 -- 172.21.15.167:0/279274504 >> 172.21.15.167:0/279274504 conn(0x7f683806d9b0 msgr2=0x7f683806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 -- 172.21.15.167:0/279274504 shutdown_connections 2023-10-06T05:14:00.222 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.219+0000 7f683759e700 1 -- 172.21.15.167:0/279274504 wait complete. 2023-10-06T05:14:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.221+0000 7f683759e700 1 Processor -- start 2023-10-06T05:14:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.221+0000 7f683759e700 1 -- start start 2023-10-06T05:14:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 0x7f683816a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380cd5e0 0x7f68380c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.224 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683815a650 0x7f6838157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c8f60 con 0x7f683815a650 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c7c60 con 0x7f68380cd5e0 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c7f40 con 0x7f68380af540 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f6835d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380cd5e0 0x7f68380c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 0x7f683816a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 0x7f683816a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49678/0 (socket says 172.21.15.167:49678) 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 -- 172.21.15.167:0/42358858 learned_addr learned my addr 172.21.15.167:0/42358858 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f6836d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683815a650 0x7f6838157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 -- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380cd5e0 msgr2=0x7f68380c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380cd5e0 0x7f68380c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 -- 172.21.15.167:0/42358858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683815a650 msgr2=0x7f6838157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683815a650 0x7f6838157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.225 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.222+0000 7f683659c700 1 -- 172.21.15.167:0/42358858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280094f0 con 0x7f68380af540 2023-10-06T05:14:00.226 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f683659c700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 0x7f683816a040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68280094c0 tx=0x7f6828017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:00.226 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828000c90 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800b9d0 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828025420 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380c8220 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6838157b90 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6828023070 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.223+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f682802f800 con 0x7f68380af540 2023-10-06T05:14:00.227 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.224+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683804c380 con 0x7f68380af540 2023-10-06T05:14:00.229 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.227+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f682802a070 con 0x7f68380af540 2023-10-06T05:14:00.288 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:00.286+0000 7fa0b8e7e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:14:00.315 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.313+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4410d06-3707-4e1d-9194-02f96c4861c8", "id": 1} v 0) v1 -- 0x7f683815ce90 con 0x7f68380af540 2023-10-06T05:14:00.321 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.319+0000 7f68267fc700 1 -- 172.21.15.167:0/42358858 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4410d06-3707-4e1d-9194-02f96c4861c8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6828015020 con 0x7f68380af540 2023-10-06T05:14:00.321 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 msgr2=0x7f683816a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 0x7f683816a040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68280094c0 tx=0x7f6828017000 comp rx=0 tx=0).stop 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 shutdown_connections 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683815a650 0x7f6838157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68380af540 0x7f683816a040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 --2- 172.21.15.167:0/42358858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68380cd5e0 0x7f68380c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.322 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 >> 172.21.15.167:0/42358858 conn(0x7f683806d9b0 msgr2=0x7f68380c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.323 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 shutdown_connections 2023-10-06T05:14:00.323 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.320+0000 7f683759e700 1 -- 172.21.15.167:0/42358858 wait complete. 2023-10-06T05:14:00.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new c3b0c4a3-8a0b-4774-8a01-d5abfe37776c 2 2023-10-06T05:14:00.463 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:00.461+0000 7fa0b8e7e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.516+0000 7f040759e700 1 Processor -- start 2023-10-06T05:14:00.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.516+0000 7f040759e700 1 -- start start 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f040759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04080af540 0x7f04080af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f040759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f040815a650 0x7f040815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f040759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040815d660 con 0x7f04080cced0 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f040759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04080ce890 con 0x7f04080af540 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f040759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040815aac0 con 0x7f040815a650 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f0405d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f040815a650 0x7f040815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f0405d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f040815a650 0x7f040815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49430/0 (socket says 172.21.15.167:49430) 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f0405d9b700 1 -- 172.21.15.167:0/841031412 learned_addr learned my addr 172.21.15.167:0/841031412 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f0406d9d700 1 --1- 172.21.15.167:0/841031412 >> v1:172.21.15.167:6789/0 conn(0x7f04080cced0 0x7f040806c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52712/0 (socket says 172.21.15.167:52712) 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.517+0000 7f040659c700 1 --2- 172.21.15.167:0/841031412 >> v2:172.21.15.33:3300/0 conn(0x7f04080af540 0x7f04080af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f0405d9b700 1 -- 172.21.15.167:0/841031412 >> v2:172.21.15.33:3300/0 conn(0x7f04080af540 msgr2=0x7f04080af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f0405d9b700 1 --2- 172.21.15.167:0/841031412 >> v2:172.21.15.33:3300/0 conn(0x7f04080af540 0x7f04080af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f0405d9b700 1 -- 172.21.15.167:0/841031412 >> v1:172.21.15.167:6789/0 conn(0x7f04080cced0 legacy=0x7f040806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f0405d9b700 1 -- 172.21.15.167:0/841031412 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040815d890 con 0x7f040815a650 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f0405d9b700 1 --2- 172.21.15.167:0/841031412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815d120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f04080893f0 tx=0x7f03fc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469495b0ef8e2729 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f040559a700 1 -- 172.21.15.167:0/841031412 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00b860 con 0x7f040815a650 2023-10-06T05:14:00.521 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f040559a700 1 -- 172.21.15.167:0/841031412 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc00b9c0 con 0x7f040815a650 2023-10-06T05:14:00.522 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.518+0000 7f040559a700 1 -- 172.21.15.167:0/841031412 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00bd60 con 0x7f040815a650 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 -- 172.21.15.167:0/841031412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 msgr2=0x7f040815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 --2- 172.21.15.167:0/841031412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815d120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f04080893f0 tx=0x7f03fc009250 comp rx=0 tx=0).stop 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 -- 172.21.15.167:0/841031412 shutdown_connections 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 --2- 172.21.15.167:0/841031412 >> v2:172.21.15.33:3300/0 conn(0x7f04080af540 0x7f04080af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 --2- 172.21.15.167:0/841031412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815d120 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 -- 172.21.15.167:0/841031412 >> 172.21.15.167:0/841031412 conn(0x7f040806d9b0 msgr2=0x7f040806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 -- 172.21.15.167:0/841031412 shutdown_connections 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.519+0000 7f040759e700 1 -- 172.21.15.167:0/841031412 wait complete. 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.520+0000 7f040759e700 1 Processor -- start 2023-10-06T05:14:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.521+0000 7f040759e700 1 -- start start 2023-10-06T05:14:00.525 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.521+0000 7f040759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04080af540 0x7f04081561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.525 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.521+0000 7f040759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 0x7f04080c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.521+0000 7f040759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f040759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04080cad70 con 0x7f04080af540 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f040759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04080c8680 con 0x7f040815a650 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f040759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04080aec90 con 0x7f04080cced0 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 0x7f04080c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 0x7f04080c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49694/0 (socket says 172.21.15.167:49694) 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 -- 172.21.15.167:0/3187618463 learned_addr learned my addr 172.21.15.167:0/3187618463 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f040659c700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04080af540 0x7f04081561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0406d9d700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.526 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 -- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 msgr2=0x7f040815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 -- 172.21.15.167:0/3187618463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04080af540 msgr2=0x7f04081561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04080af540 0x7f04081561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 -- 172.21.15.167:0/3187618463 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0094f0 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.522+0000 7f0405d9b700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 0x7f04080c7580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f040815ac00 tx=0x7f03fc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc01a450 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc01a5b0 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc01a970 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04080caa90 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04080c8310 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f03fc01ad20 con 0x7f04080cced0 2023-10-06T05:14:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.523+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f03fc025c30 con 0x7f04080cced0 2023-10-06T05:14:00.528 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.524+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040815a420 con 0x7f04080cced0 2023-10-06T05:14:00.528 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.526+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f03fc0456d0 con 0x7f04080cced0 2023-10-06T05:14:00.564 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:00.563+0000 7fa0b8e7e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:14:00.613 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.612+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c3b0c4a3-8a0b-4774-8a01-d5abfe37776c", "id": 2} v 0) v1 -- 0x7f04080c87c0 con 0x7f04080cced0 2023-10-06T05:14:00.617 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.616+0000 7f03f67fc700 1 -- 172.21.15.167:0/3187618463 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3b0c4a3-8a0b-4774-8a01-d5abfe37776c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f03fc034030 con 0x7f04080cced0 2023-10-06T05:14:00.618 INFO:teuthology.orchestra.run.smithi167.stdout:2 2023-10-06T05:14:00.618 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:00.617+0000 7fcf74e8b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 msgr2=0x7f04080c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 0x7f04080c7580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f040815ac00 tx=0x7f03fc017000 comp rx=0 tx=0).stop 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 shutdown_connections 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f040815a650 0x7f040815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04080af540 0x7f04081561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 --2- 172.21.15.167:0/3187618463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04080cced0 0x7f04080c7580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.619 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 >> 172.21.15.167:0/3187618463 conn(0x7f040806d9b0 msgr2=0x7f04080b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 shutdown_connections 2023-10-06T05:14:00.620 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.617+0000 7f040759e700 1 -- 172.21.15.167:0/3187618463 wait complete. 2023-10-06T05:14:00.636 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 3d9cc09d-af9f-4ae7-bfb6-b06490f028a6 3 2023-10-06T05:14:00.788 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:00.786+0000 7fcf74e8b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:00.806 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:00.804+0000 7fa0b8e7e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:00.820 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.818+0000 7f3c5f5ff700 1 Processor -- start 2023-10-06T05:14:00.820 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.818+0000 7f3c5f5ff700 1 -- start start 2023-10-06T05:14:00.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.819+0000 7f3c5f5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c58159b10 0x7f3c5806f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5f5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c5806d140 0x7f3c5806f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5f5ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c580d1120 con 0x7f3c580cabf0 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5f5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580d1400 con 0x7f3c5806d140 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5f5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5806e920 con 0x7f3c58159b10 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5cb9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c58159b10 0x7f3c5806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5d39b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3c580cabf0 0x7f3c580c7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52714/0 (socket says 172.21.15.167:52714) 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5d39b700 1 -- 172.21.15.167:0/2265129467 learned_addr learned my addr 172.21.15.167:0/2265129467 (peer_addr_for_me v1:172.21.15.167:0/0) 2023-10-06T05:14:00.822 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5db9c700 1 --2- 172.21.15.167:0/2265129467 >> v2:172.21.15.33:3300/0 conn(0x7f3c5806d140 0x7f3c5806f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.824 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c4ffff700 1 -- 172.21.15.167:0/2265129467 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392312697 0 0) 0x7f3c580d1120 con 0x7f3c580cabf0 2023-10-06T05:14:00.824 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c4ffff700 1 -- 172.21.15.167:0/2265129467 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c44003570 con 0x7f3c580cabf0 2023-10-06T05:14:00.824 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5cb9a700 1 -- 172.21.15.167:0/2265129467 >> v2:172.21.15.33:3300/0 conn(0x7f3c5806d140 msgr2=0x7f3c5806f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:00.824 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5cb9a700 1 --2- 172.21.15.167:0/2265129467 >> v2:172.21.15.33:3300/0 conn(0x7f3c5806d140 0x7f3c5806f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5cb9a700 1 -- 172.21.15.167:0/2265129467 >> v1:172.21.15.167:6789/0 conn(0x7f3c580cabf0 legacy=0x7f3c580c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.820+0000 7f3c5cb9a700 1 -- 172.21.15.167:0/2265129467 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5806ec00 con 0x7f3c58159b10 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.821+0000 7f3c5cb9a700 1 --2- 172.21.15.167:0/2265129467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c58159b10 0x7f3c5806f120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3c480098e0 tx=0x7f3c480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c406a2700f0f27e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.821+0000 7f3c4ffff700 1 -- 172.21.15.167:0/2265129467 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48011630 con 0x7f3c58159b10 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.821+0000 7f3c4ffff700 1 -- 172.21.15.167:0/2265129467 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c48011790 con 0x7f3c58159b10 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.821+0000 7f3c4ffff700 1 -- 172.21.15.167:0/2265129467 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48011b30 con 0x7f3c58159b10 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.821+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/2265129467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c58159b10 msgr2=0x7f3c5806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.821+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/2265129467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c58159b10 0x7f3c5806f120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3c480098e0 tx=0x7f3c480091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.822+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/2265129467 shutdown_connections 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.822+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/2265129467 >> v2:172.21.15.33:3300/0 conn(0x7f3c5806d140 0x7f3c5806f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.825 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.822+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/2265129467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c58159b10 0x7f3c5806f120 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.822+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/2265129467 >> 172.21.15.167:0/2265129467 conn(0x7f3c5806cd40 msgr2=0x7f3c580d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.822+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/2265129467 shutdown_connections 2023-10-06T05:14:00.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.822+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/2265129467 wait complete. 2023-10-06T05:14:00.828 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.823+0000 7f3c5f5ff700 1 Processor -- start 2023-10-06T05:14:00.828 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.823+0000 7f3c5f5ff700 1 -- start start 2023-10-06T05:14:00.828 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5f5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 0x7f3c5815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.828 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5f5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580cabf0 0x7f3c58156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5f5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58159b10 0x7f3c580c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5f5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58156cd0 con 0x7f3c580cabf0 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5f5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580c6900 con 0x7f3c58159b10 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5f5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580c5aa0 con 0x7f3c5806d140 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.824+0000 7f3c5d39b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 0x7f3c5815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 0x7f3c5815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49708/0 (socket says 172.21.15.167:49708) 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 -- 172.21.15.167:0/682654777 learned_addr learned my addr 172.21.15.167:0/682654777 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5db9c700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58159b10 0x7f3c580c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.829 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5cb9a700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580cabf0 0x7f3c58156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 -- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58159b10 msgr2=0x7f3c580c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58159b10 0x7f3c580c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 -- 172.21.15.167:0/682654777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580cabf0 msgr2=0x7f3c58156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580cabf0 0x7f3c58156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 -- 172.21.15.167:0/682654777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48009420 con 0x7f3c5806d140 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c5d39b700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 0x7f3c5815c0e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c790 tx=0x7f3c54007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540072e0 con 0x7f3c5806d140 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54007440 con 0x7f3c5806d140 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.825+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540057f0 con 0x7f3c5806d140 2023-10-06T05:14:00.830 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.826+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c580c5cf0 con 0x7f3c5806d140 2023-10-06T05:14:00.831 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.826+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580c62b0 con 0x7f3c5806d140 2023-10-06T05:14:00.831 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.826+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3c5401c440 con 0x7f3c5806d140 2023-10-06T05:14:00.831 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.826+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f3c54012400 con 0x7f3c5806d140 2023-10-06T05:14:00.831 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.827+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5804c380 con 0x7f3c5806d140 2023-10-06T05:14:00.831 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.829+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3c54021060 con 0x7f3c5806d140 2023-10-06T05:14:00.887 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:00.885+0000 7fcf74e8b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:14:00.917 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.915+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3d9cc09d-af9f-4ae7-bfb6-b06490f028a6", "id": 3} v 0) v1 -- 0x7f3c5815c7d0 con 0x7f3c5806d140 2023-10-06T05:14:00.921 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.919+0000 7f3c4d7fa700 1 -- 172.21.15.167:0/682654777 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3d9cc09d-af9f-4ae7-bfb6-b06490f028a6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3c54017070 con 0x7f3c5806d140 2023-10-06T05:14:00.921 INFO:teuthology.orchestra.run.smithi167.stdout:3 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 msgr2=0x7f3c5815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 0x7f3c5815c0e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c790 tx=0x7f3c54007c80 comp rx=0 tx=0).stop 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 shutdown_connections 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58159b10 0x7f3c580c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5806d140 0x7f3c5815c0e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 --2- 172.21.15.167:0/682654777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580cabf0 0x7f3c58156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 >> 172.21.15.167:0/682654777 conn(0x7f3c5806cd40 msgr2=0x7f3c580ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 shutdown_connections 2023-10-06T05:14:00.923 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:00.921+0000 7f3c5f5ff700 1 -- 172.21.15.167:0/682654777 wait complete. 2023-10-06T05:14:00.939 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new db1d7cf0-6c8d-47b7-a0bb-b45726447197 4 2023-10-06T05:14:01.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:01.033+0000 7fa0b8e7e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:14:01.115 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:01.113+0000 7fcf74e8b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.115+0000 7f6c3b39d700 1 Processor -- start 2023-10-06T05:14:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.115+0000 7f6c3b39d700 1 -- start start 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3b39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c340af540 0x7f6c340af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c340ce890 con 0x7f6c3415a650 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3b39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3415aac0 con 0x7f6c340cd5e0 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3b39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3415d890 con 0x7f6c340af540 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3a39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c340af540 0x7f6c340af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3ab9c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6c340cd5e0 0x7f6c340ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52728/0 (socket says 172.21.15.167:52728) 2023-10-06T05:14:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c3ab9c700 1 -- 172.21.15.167:0/223086392 learned_addr learned my addr 172.21.15.167:0/223086392 (peer_addr_for_me v1:172.21.15.167:0/0) 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286612650 0 0) 0x7f6c340ce890 con 0x7f6c3415a650 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.117+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c20003570 con 0x7f6c3415a650 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681583180 0 0) 0x7f6c3415aac0 con 0x7f6c340cd5e0 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c340ce890 con 0x7f6c340cd5e0 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c3a39b700 1 -- 172.21.15.167:0/223086392 >> v1:172.21.15.167:6789/0 conn(0x7f6c340cd5e0 legacy=0x7f6c340ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c3a39b700 1 -- 172.21.15.167:0/223086392 >> v1:172.21.15.33:6789/0 conn(0x7f6c3415a650 legacy=0x7f6c340cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c3a39b700 1 -- 172.21.15.167:0/223086392 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3415db70 con 0x7f6c340af540 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c3a39b700 1 --2- 172.21.15.167:0/223086392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c340af910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c300098e0 tx=0x7f6c300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6991da522a1827 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30011630 con 0x7f6c340af540 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30011790 con 0x7f6c340af540 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.118+0000 7f6c39399700 1 -- 172.21.15.167:0/223086392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30011b30 con 0x7f6c340af540 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 -- 172.21.15.167:0/223086392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 msgr2=0x7f6c340af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 --2- 172.21.15.167:0/223086392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c340af910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c300098e0 tx=0x7f6c300091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 -- 172.21.15.167:0/223086392 shutdown_connections 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 --2- 172.21.15.167:0/223086392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c340af910 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 -- 172.21.15.167:0/223086392 >> 172.21.15.167:0/223086392 conn(0x7f6c3406d9b0 msgr2=0x7f6c3406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 -- 172.21.15.167:0/223086392 shutdown_connections 2023-10-06T05:14:01.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.119+0000 7f6c3b39d700 1 -- 172.21.15.167:0/223086392 wait complete. 2023-10-06T05:14:01.124 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.120+0000 7f6c3b39d700 1 Processor -- start 2023-10-06T05:14:01.124 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.120+0000 7f6c3b39d700 1 -- start start 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c3b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c34169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c3b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 0x7f6c340b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c3b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3415a650 0x7f6c340c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c3b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340b4370 con 0x7f6c340af540 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c3b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340b6300 con 0x7f6c340cd5e0 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c3b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340c7ac0 con 0x7f6c3415a650 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.121+0000 7f6c39b9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 0x7f6c340b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c3a39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c34169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 0x7f6c340b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49462/0 (socket says 172.21.15.167:49462) 2023-10-06T05:14:01.125 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 -- 172.21.15.167:0/3097564513 learned_addr learned my addr 172.21.15.167:0/3097564513 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c3ab9c700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3415a650 0x7f6c340c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 -- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3415a650 msgr2=0x7f6c340c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3415a650 0x7f6c340c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 -- 172.21.15.167:0/3097564513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 msgr2=0x7f6c34169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c34169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 -- 172.21.15.167:0/3097564513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009420 con 0x7f6c340cd5e0 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c39b9a700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 0x7f6c340b5dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c790 tx=0x7f6c24007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.122+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24010510 con 0x7f6c340cd5e0 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.123+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24010670 con 0x7f6c340cd5e0 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.123+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240109f0 con 0x7f6c340cd5e0 2023-10-06T05:14:01.126 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.123+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3416a270 con 0x7f6c340cd5e0 2023-10-06T05:14:01.127 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.123+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3416a830 con 0x7f6c340cd5e0 2023-10-06T05:14:01.127 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.123+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6c240126c0 con 0x7f6c340cd5e0 2023-10-06T05:14:01.127 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.123+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f6c2401b3b0 con 0x7f6c340cd5e0 2023-10-06T05:14:01.127 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.124+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c340b4520 con 0x7f6c340cd5e0 2023-10-06T05:14:01.128 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.126+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6c340b4520 con 0x7f6c340cd5e0 2023-10-06T05:14:01.136 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:01.134+0000 7fa0b8e7e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:14:01.213 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.212+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "db1d7cf0-6c8d-47b7-a0bb-b45726447197", "id": 4} v 0) v1 -- 0x7f6c3404c380 con 0x7f6c340cd5e0 2023-10-06T05:14:01.219 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.217+0000 7f6c2a7fc700 1 -- 172.21.15.167:0/3097564513 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db1d7cf0-6c8d-47b7-a0bb-b45726447197", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6c24021060 con 0x7f6c340cd5e0 2023-10-06T05:14:01.219 INFO:teuthology.orchestra.run.smithi167.stdout:4 2023-10-06T05:14:01.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.218+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 msgr2=0x7f6c340b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.218+0000 7f6c3b39d700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 0x7f6c340b5dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c790 tx=0x7f6c24007e30 comp rx=0 tx=0).stop 2023-10-06T05:14:01.220 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.218+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 shutdown_connections 2023-10-06T05:14:01.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.218+0000 7f6c3b39d700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3415a650 0x7f6c340c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.219+0000 7f6c3b39d700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c340af540 0x7f6c34169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.219+0000 7f6c3b39d700 1 --2- 172.21.15.167:0/3097564513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c340cd5e0 0x7f6c340b5dc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.219+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 >> 172.21.15.167:0/3097564513 conn(0x7f6c3406d9b0 msgr2=0x7f6c340cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:01.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.219+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 shutdown_connections 2023-10-06T05:14:01.221 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.219+0000 7f6c3b39d700 1 -- 172.21.15.167:0/3097564513 wait complete. 2023-10-06T05:14:01.237 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new ca568a38-27d1-437e-8a93-19c9a04d9cd8 5 2023-10-06T05:14:01.308 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:01.306+0000 7fa0b8e7e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:14:01.338 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:01.336+0000 7fcf74e8b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:14:01.413 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.411+0000 7f15bb39d700 1 Processor -- start 2023-10-06T05:14:01.413 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.411+0000 7f15bb39d700 1 -- start start 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bb39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bb39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15b40cd5e0 0x7f15b415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bb39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b415d660 con 0x7f15b415a650 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bb39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40ce890 con 0x7f15b40cd5e0 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bb39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b415aac0 con 0x7f15b40af540 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15ba39b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bab9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15b40cd5e0 0x7f15b415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15ba39b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49730/0 (socket says 172.21.15.167:49730) 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15bab9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15b40cd5e0 0x7f15b415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49476/0 (socket says 172.21.15.167:49476) 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.413+0000 7f15ba39b700 1 -- 172.21.15.167:0/3796233477 learned_addr learned my addr 172.21.15.167:0/3796233477 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15bab9c700 1 -- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 msgr2=0x7f15b40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15bab9c700 1 --2- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15bab9c700 1 -- 172.21.15.167:0/3796233477 >> v1:172.21.15.167:6789/0 conn(0x7f15b415a650 legacy=0x7f15b40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15bab9c700 1 -- 172.21.15.167:0/3796233477 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b415d890 con 0x7f15b40cd5e0 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15ba39b700 1 --2- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15bab9c700 1 --2- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b415d120 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009e60 tx=0x7f15ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f14f92b3791aa9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15b9399700 1 -- 172.21.15.167:0/3796233477 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00e070 con 0x7f15b40cd5e0 2023-10-06T05:14:01.419 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15b9399700 1 -- 172.21.15.167:0/3796233477 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac00bb60 con 0x7f15b40cd5e0 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.414+0000 7f15b9399700 1 -- 172.21.15.167:0/3796233477 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac012430 con 0x7f15b40cd5e0 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 -- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 msgr2=0x7f15b415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 --2- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b415d120 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009e60 tx=0x7f15ac009780 comp rx=0 tx=0).stop 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 -- 172.21.15.167:0/3796233477 shutdown_connections 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 --2- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b415d120 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 --2- 172.21.15.167:0/3796233477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 -- 172.21.15.167:0/3796233477 >> 172.21.15.167:0/3796233477 conn(0x7f15b406d9b0 msgr2=0x7f15b406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 -- 172.21.15.167:0/3796233477 shutdown_connections 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.415+0000 7f15bb39d700 1 -- 172.21.15.167:0/3796233477 wait complete. 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.416+0000 7f15bb39d700 1 Processor -- start 2023-10-06T05:14:01.420 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.416+0000 7f15bb39d700 1 -- start start 2023-10-06T05:14:01.422 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.417+0000 7f15bb39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.417+0000 7f15bb39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b40c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.417+0000 7f15bb39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b415a650 0x7f15b41444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.417+0000 7f15bb39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40c9fd0 con 0x7f15b415a650 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.417+0000 7f15bb39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40c93c0 con 0x7f15b40cd5e0 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.417+0000 7f15bb39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40c96a0 con 0x7f15b40af540 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15b9b9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b40c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49742/0 (socket says 172.21.15.167:49742) 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 -- 172.21.15.167:0/2061805585 learned_addr learned my addr 172.21.15.167:0/2061805585 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:01.423 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15bab9c700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b415a650 0x7f15b41444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 -- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 msgr2=0x7f15b40c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b40c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 -- 172.21.15.167:0/2061805585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b415a650 msgr2=0x7f15b41444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b415a650 0x7f15b41444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 -- 172.21.15.167:0/2061805585 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00094f0 con 0x7f15b40af540 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15bab9c700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b415a650 0x7f15b41444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15b9b9a700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b40c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15ba39b700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40ca460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15b000ec50 tx=0x7f15b0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.418+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000c850 con 0x7f15b40af540 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.419+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b000c9b0 con 0x7f15b40af540 2023-10-06T05:14:01.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.419+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac009a20 con 0x7f15b40af540 2023-10-06T05:14:01.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.419+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000cd30 con 0x7f15b40af540 2023-10-06T05:14:01.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.419+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b40c9c60 con 0x7f15b40af540 2023-10-06T05:14:01.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.419+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f15b0005c80 con 0x7f15b40af540 2023-10-06T05:14:01.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.419+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f15b0013c70 con 0x7f15b40af540 2023-10-06T05:14:01.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.420+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b404c380 con 0x7f15b40af540 2023-10-06T05:14:01.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.422+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f15b003b6d0 con 0x7f15b40af540 2023-10-06T05:14:01.432 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:01.430+0000 7fa0b8e7e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:01.435 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:01.434+0000 7fcf74e8b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:14:01.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.506+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ca568a38-27d1-437e-8a93-19c9a04d9cd8", "id": 5} v 0) v1 -- 0x7f15b415db70 con 0x7f15b40af540 2023-10-06T05:14:01.516 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.514+0000 7f15aa7fc700 1 -- 172.21.15.167:0/2061805585 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ca568a38-27d1-437e-8a93-19c9a04d9cd8", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f15b002a020 con 0x7f15b40af540 2023-10-06T05:14:01.516 INFO:teuthology.orchestra.run.smithi167.stdout:5 2023-10-06T05:14:01.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 msgr2=0x7f15b40ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40ca460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15b000ec50 tx=0x7f15b0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 shutdown_connections 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b415a650 0x7f15b41444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15b40af540 0x7f15b40ca460 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 --2- 172.21.15.167:0/2061805585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15b40cd5e0 0x7f15b40c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 >> 172.21.15.167:0/2061805585 conn(0x7f15b406d9b0 msgr2=0x7f15b4156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 shutdown_connections 2023-10-06T05:14:01.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.516+0000 7f15bb39d700 1 -- 172.21.15.167:0/2061805585 wait complete. 2023-10-06T05:14:01.534 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 895c30ab-410a-411b-84ad-e7f759abf82a 6 2023-10-06T05:14:01.601 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:01.599+0000 7fcf74e8b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:14:01.711 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.709+0000 7f45e359e700 1 Processor -- start 2023-10-06T05:14:01.711 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.709+0000 7f45e359e700 1 -- start start 2023-10-06T05:14:01.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f45e40af910 0x7f45e415a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40ce890 con 0x7f45e40af540 2023-10-06T05:14:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e415d890 con 0x7f45e40cd5e0 2023-10-06T05:14:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e40afd50 con 0x7f45e40af910 2023-10-06T05:14:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e1d9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f45e40cd5e0 0x7f45e40ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52758/0 (socket says 172.21.15.167:52758) 2023-10-06T05:14:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e1d9b700 1 -- 172.21.15.167:0/2963664580 learned_addr learned my addr 172.21.15.167:0/2963664580 (peer_addr_for_me v1:172.21.15.167:0/0) 2023-10-06T05:14:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e2d9d700 1 --2- 172.21.15.167:0/2963664580 >> v2:172.21.15.167:3300/0 conn(0x7f45e40af910 0x7f45e415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e159a700 1 -- 172.21.15.167:0/2963664580 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488450365 0 0) 0x7f45e415d890 con 0x7f45e40cd5e0 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e159a700 1 -- 172.21.15.167:0/2963664580 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45cc003570 con 0x7f45e40cd5e0 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e159a700 1 -- 172.21.15.167:0/2963664580 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378712546 0 0) 0x7f45e40ce890 con 0x7f45e40af540 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e2d9d700 1 -- 172.21.15.167:0/2963664580 >> v1:172.21.15.167:6789/0 conn(0x7f45e40cd5e0 legacy=0x7f45e40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e2d9d700 1 -- 172.21.15.167:0/2963664580 >> v1:172.21.15.33:6789/0 conn(0x7f45e40af540 legacy=0x7f45e40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.711+0000 7f45e2d9d700 1 -- 172.21.15.167:0/2963664580 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e415db70 con 0x7f45e40af910 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e2d9d700 1 --2- 172.21.15.167:0/2963664580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af910 0x7f45e415a650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009e60 tx=0x7f45dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b38d5f673ec60ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e159a700 1 -- 172.21.15.167:0/2963664580 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00e070 con 0x7f45e40af910 2023-10-06T05:14:01.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e159a700 1 -- 172.21.15.167:0/2963664580 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc00bb60 con 0x7f45e40af910 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e159a700 1 -- 172.21.15.167:0/2963664580 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc012430 con 0x7f45e40af910 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e359e700 1 -- 172.21.15.167:0/2963664580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af910 msgr2=0x7f45e415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e359e700 1 --2- 172.21.15.167:0/2963664580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af910 0x7f45e415a650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009e60 tx=0x7f45dc009780 comp rx=0 tx=0).stop 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e359e700 1 -- 172.21.15.167:0/2963664580 shutdown_connections 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e359e700 1 --2- 172.21.15.167:0/2963664580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af910 0x7f45e415a650 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e359e700 1 -- 172.21.15.167:0/2963664580 >> 172.21.15.167:0/2963664580 conn(0x7f45e406d9b0 msgr2=0x7f45e406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.712+0000 7f45e359e700 1 -- 172.21.15.167:0/2963664580 shutdown_connections 2023-10-06T05:14:01.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.713+0000 7f45e359e700 1 -- 172.21.15.167:0/2963664580 wait complete. 2023-10-06T05:14:01.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.714+0000 7f45e359e700 1 Processor -- start 2023-10-06T05:14:01.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.714+0000 7f45e359e700 1 -- start start 2023-10-06T05:14:01.718 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 0x7f45e40c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45e40af910 0x7f45e4144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45e40cd5e0 0x7f45e40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40b09d0 con 0x7f45e40af910 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40ca730 con 0x7f45e40af540 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40caa10 con 0x7f45e40cd5e0 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 0x7f45e40c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 0x7f45e40c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49512/0 (socket says 172.21.15.167:49512) 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e2d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45e40cd5e0 0x7f45e40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e259c700 1 -- 172.21.15.167:0/1153734060 learned_addr learned my addr 172.21.15.167:0/1153734060 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e1d9b700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45e40af910 0x7f45e4144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:01.719 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e259c700 1 -- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45e40cd5e0 msgr2=0x7f45e40c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.715+0000 7f45e259c700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45e40cd5e0 0x7f45e40c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45e259c700 1 -- 172.21.15.167:0/1153734060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45e40af910 msgr2=0x7f45e4144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45e259c700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45e40af910 0x7f45e4144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45e259c700 1 -- 172.21.15.167:0/1153734060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc009a20 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45e259c700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 0x7f45e40c7b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45d400c530 tx=0x7f45d400c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4011590 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4011710 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4011ad0 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e40b0380 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e40ca2f0 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f45d401c6c0 con 0x7f45e40af540 2023-10-06T05:14:01.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.716+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f45d4012a20 con 0x7f45e40af540 2023-10-06T05:14:01.721 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.717+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e404c380 con 0x7f45e40af540 2023-10-06T05:14:01.721 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.719+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f45d4017070 con 0x7f45e40af540 2023-10-06T05:14:01.723 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:01.721+0000 7fcf74e8b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:01.806 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.805+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "895c30ab-410a-411b-84ad-e7f759abf82a", "id": 6} v 0) v1 -- 0x7f45e415de50 con 0x7f45e40af540 2023-10-06T05:14:01.815 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.813+0000 7f45d27fc700 1 -- 172.21.15.167:0/1153734060 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "895c30ab-410a-411b-84ad-e7f759abf82a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f45d401c970 con 0x7f45e40af540 2023-10-06T05:14:01.815 INFO:teuthology.orchestra.run.smithi167.stdout:6 2023-10-06T05:14:01.817 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 msgr2=0x7f45e40c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:01.817 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 0x7f45e40c7b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45d400c530 tx=0x7f45d400c840 comp rx=0 tx=0).stop 2023-10-06T05:14:01.817 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 shutdown_connections 2023-10-06T05:14:01.817 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45e40cd5e0 0x7f45e40c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.817 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45e40af540 0x7f45e40c7b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.817 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 --2- 172.21.15.167:0/1153734060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45e40af910 0x7f45e4144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:01.818 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 >> 172.21.15.167:0/1153734060 conn(0x7f45e406d9b0 msgr2=0x7f45e40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:01.818 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 shutdown_connections 2023-10-06T05:14:01.818 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:01.815+0000 7f45e359e700 1 -- 172.21.15.167:0/1153734060 wait complete. 2023-10-06T05:14:01.834 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 8e18fadc-363e-4401-a5ee-07229d17b12b 7 2023-10-06T05:14:02.016 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.014+0000 7f81bbfff700 1 Processor -- start 2023-10-06T05:14:02.016 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.014+0000 7f81bbfff700 1 -- start start 2023-10-06T05:14:02.017 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.015+0000 7f81bbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81bc0af540 0x7f81bc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.017 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.015+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc15a650 0x7f81bc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.017 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.015+0000 7f81bbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc15d660 con 0x7f81bc0cced0 2023-10-06T05:14:02.018 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.015+0000 7f81bbfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0ce890 con 0x7f81bc0af540 2023-10-06T05:14:02.018 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.015+0000 7f81bbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc15aac0 con 0x7f81bc15a650 2023-10-06T05:14:02.018 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81ba7fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc15a650 0x7f81bc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.018 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81bb7fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f81bc0cced0 0x7f81bc06c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:52770/0 (socket says 172.21.15.167:52770) 2023-10-06T05:14:02.018 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81bb7fe700 1 -- 172.21.15.167:0/354549366 learned_addr learned my addr 172.21.15.167:0/354549366 (peer_addr_for_me v1:172.21.15.167:0/0) 2023-10-06T05:14:02.018 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81baffd700 1 --2- 172.21.15.167:0/354549366 >> v2:172.21.15.33:3300/0 conn(0x7f81bc0af540 0x7f81bc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81ba7fc700 1 -- 172.21.15.167:0/354549366 >> v2:172.21.15.33:3300/0 conn(0x7f81bc0af540 msgr2=0x7f81bc0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81ba7fc700 1 --2- 172.21.15.167:0/354549366 >> v2:172.21.15.33:3300/0 conn(0x7f81bc0af540 0x7f81bc0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81ba7fc700 1 -- 172.21.15.167:0/354549366 >> v1:172.21.15.167:6789/0 conn(0x7f81bc0cced0 legacy=0x7f81bc06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81ba7fc700 1 -- 172.21.15.167:0/354549366 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc15d890 con 0x7f81bc15a650 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81b9ffb700 1 -- 172.21.15.167:0/354549366 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364364348 0 0) 0x7f81bc15d660 con 0x7f81bc0cced0 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.016+0000 7f81ba7fc700 1 --2- 172.21.15.167:0/354549366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc15a650 0x7f81bc15d120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098e0 tx=0x7f81b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108dedad69a6cf49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81b9ffb700 1 -- 172.21.15.167:0/354549366 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000b640 con 0x7f81bc15a650 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81b9ffb700 1 -- 172.21.15.167:0/354549366 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000b7a0 con 0x7f81bc15a650 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81b9ffb700 1 -- 172.21.15.167:0/354549366 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000bb40 con 0x7f81bc15a650 2023-10-06T05:14:02.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 -- 172.21.15.167:0/354549366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc15a650 msgr2=0x7f81bc15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 --2- 172.21.15.167:0/354549366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc15a650 0x7f81bc15d120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098e0 tx=0x7f81b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 -- 172.21.15.167:0/354549366 shutdown_connections 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 --2- 172.21.15.167:0/354549366 >> v2:172.21.15.33:3300/0 conn(0x7f81bc0af540 0x7f81bc0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 --2- 172.21.15.167:0/354549366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc15a650 0x7f81bc15d120 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 -- 172.21.15.167:0/354549366 >> 172.21.15.167:0/354549366 conn(0x7f81bc06d9b0 msgr2=0x7f81bc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 -- 172.21.15.167:0/354549366 shutdown_connections 2023-10-06T05:14:02.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.017+0000 7f81bbfff700 1 -- 172.21.15.167:0/354549366 wait complete. 2023-10-06T05:14:02.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.019+0000 7f81bbfff700 1 Processor -- start 2023-10-06T05:14:02.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.019+0000 7f81bbfff700 1 -- start start 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 0x7f81bc169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81bc0cced0 0x7f81bc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81bc15a650 0x7f81bc157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c8680 con 0x7f81bc15a650 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0cf550 con 0x7f81bc0cced0 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc16a3f0 con 0x7f81bc0af540 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 0x7f81bc169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 0x7f81bc169da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:49762/0 (socket says 172.21.15.167:49762) 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 -- 172.21.15.167:0/327379915 learned_addr learned my addr 172.21.15.167:0/327379915 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81ba7fc700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81bc0cced0 0x7f81bc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bb7fe700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81bc15a650 0x7f81bc157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 -- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81bc0cced0 msgr2=0x7f81bc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81bc0cced0 0x7f81bc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 -- 172.21.15.167:0/327379915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81bc15a650 msgr2=0x7f81bc157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81bc15a650 0x7f81bc157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81baffd700 1 -- 172.21.15.167:0/327379915 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009420 con 0x7f81bc0af540 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.020+0000 7f81bb7fe700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81bc15a650 0x7f81bc157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81baffd700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 0x7f81bc169da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00e930 tx=0x7f81ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00eed0 con 0x7f81bc0af540 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac00c850 con 0x7f81bc0af540 2023-10-06T05:14:02.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc16a6d0 con 0x7f81bc0af540 2023-10-06T05:14:02.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00cbf0 con 0x7f81bc0af540 2023-10-06T05:14:02.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc16ac90 con 0x7f81bc0af540 2023-10-06T05:14:02.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f81ac0136f0 con 0x7f81bc0af540 2023-10-06T05:14:02.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.021+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f81ac01cc90 con 0x7f81bc0af540 2023-10-06T05:14:02.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.022+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc04c380 con 0x7f81bc0af540 2023-10-06T05:14:02.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.024+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f81ac021060 con 0x7f81bc0af540 2023-10-06T05:14:02.081 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:02.080+0000 7fa0b8e7e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:14:02.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.111+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8e18fadc-363e-4401-a5ee-07229d17b12b", "id": 7} v 0) v1 -- 0x7f81bc15db70 con 0x7f81bc0af540 2023-10-06T05:14:02.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.117+0000 7f81a2ffd700 1 -- 172.21.15.167:0/327379915 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e18fadc-363e-4401-a5ee-07229d17b12b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f81ac017070 con 0x7f81bc0af540 2023-10-06T05:14:02.119 INFO:teuthology.orchestra.run.smithi167.stdout:7 2023-10-06T05:14:02.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 msgr2=0x7f81bc169da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 0x7f81bc169da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00e930 tx=0x7f81ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 shutdown_connections 2023-10-06T05:14:02.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81bc15a650 0x7f81bc157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81bc0af540 0x7f81bc169da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 --2- 172.21.15.167:0/327379915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81bc0cced0 0x7f81bc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 >> 172.21.15.167:0/327379915 conn(0x7f81bc06d9b0 msgr2=0x7f81bc158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:02.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 shutdown_connections 2023-10-06T05:14:02.122 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T05:14:02.119+0000 7f81bbfff700 1 -- 172.21.15.167:0/327379915 wait complete. 2023-10-06T05:14:02.144 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T05:14:02.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T05:14:02.149 INFO:tasks.ceph.osd.0:Started 2023-10-06T05:14:02.149 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T05:14:02.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T05:14:02.154 INFO:tasks.ceph.osd.1:Started 2023-10-06T05:14:02.154 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T05:14:02.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T05:14:02.159 INFO:tasks.ceph.osd.2:Started 2023-10-06T05:14:02.159 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T05:14:02.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T05:14:02.166 INFO:tasks.ceph.osd.3:Started 2023-10-06T05:14:02.166 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T05:14:02.166 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T05:14:02.170 INFO:tasks.ceph.osd.4:Started 2023-10-06T05:14:02.170 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T05:14:02.171 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T05:14:02.175 INFO:tasks.ceph.osd.5:Started 2023-10-06T05:14:02.175 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T05:14:02.175 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T05:14:02.181 INFO:tasks.ceph.osd.6:Started 2023-10-06T05:14:02.182 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T05:14:02.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T05:14:02.187 INFO:tasks.ceph.osd.7:Started 2023-10-06T05:14:02.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:02.192 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:02.191+0000 7fa0b8e7e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:14:02.328 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:02.326+0000 7fa0b8e7e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:14:02.415 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:02.413+0000 7fcf74e8b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:14:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.431+0000 7f4ac09fe700 1 Processor -- start 2023-10-06T05:14:02.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.431+0000 7f4ac09fe700 1 -- start start 2023-10-06T05:14:02.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.433+0000 7f4ac09fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4abc0ccfc0 0x7f4abc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.433+0000 7f4ac09fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0cf3b0 0x7f4abc1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4ac09fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc0cbd80 con 0x7f4abc0cc750 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4ac09fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc0af5f0 con 0x7f4abc0ccfc0 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4ac09fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0af8d0 con 0x7f4abc0cf3b0 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4abc0ccfc0 0x7f4abc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abb59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4abc0cc750 0x7f4abc0b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35234/0 (socket says 172.21.15.33:35234) 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abb59e700 1 -- 172.21.15.33:0/705644221 learned_addr learned my addr 172.21.15.33:0/705644221 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:02.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4aba59c700 1 --2- 172.21.15.33:0/705644221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0cf3b0 0x7f4abc1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4ab9d9b700 1 -- 172.21.15.33:0/705644221 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453598194 0 0) 0x7f4abc0cbd80 con 0x7f4abc0cc750 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abad9d700 1 -- 172.21.15.33:0/705644221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0cf3b0 msgr2=0x7f4abc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abad9d700 1 --2- 172.21.15.33:0/705644221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0cf3b0 0x7f4abc1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abad9d700 1 -- 172.21.15.33:0/705644221 >> v1:172.21.15.33:6789/0 conn(0x7f4abc0cc750 legacy=0x7f4abc0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.434+0000 7f4abad9d700 1 -- 172.21.15.33:0/705644221 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc0afbb0 con 0x7f4abc0ccfc0 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.435+0000 7f4abad9d700 1 --2- 172.21.15.33:0/705644221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0ccfc0 0x7f4abc0cc210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0098e0 tx=0x7f4aac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de617f022c4c668 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.435+0000 7f4ab9d9b700 1 -- 172.21.15.33:0/705644221 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00b640 con 0x7f4abc0ccfc0 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.435+0000 7f4ab9d9b700 1 -- 172.21.15.33:0/705644221 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac00b7a0 con 0x7f4abc0ccfc0 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.435+0000 7f4ab9d9b700 1 -- 172.21.15.33:0/705644221 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00bb40 con 0x7f4abc0ccfc0 2023-10-06T05:14:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.435+0000 7f4ac09fe700 1 -- 172.21.15.33:0/705644221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0ccfc0 msgr2=0x7f4abc0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.435+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/705644221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0ccfc0 0x7f4abc0cc210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0098e0 tx=0x7f4aac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.436+0000 7f4ac09fe700 1 -- 172.21.15.33:0/705644221 shutdown_connections 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.436+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/705644221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0ccfc0 0x7f4abc0cc210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.436+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/705644221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0cf3b0 0x7f4abc1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.436+0000 7f4ac09fe700 1 -- 172.21.15.33:0/705644221 >> 172.21.15.33:0/705644221 conn(0x7f4abc06cff0 msgr2=0x7f4abc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.436+0000 7f4ac09fe700 1 -- 172.21.15.33:0/705644221 shutdown_connections 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.436+0000 7f4ac09fe700 1 -- 172.21.15.33:0/705644221 wait complete. 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.437+0000 7f4ac09fe700 1 Processor -- start 2023-10-06T05:14:02.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.437+0000 7f4ac09fe700 1 -- start start 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4ac09fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 0x7f4abc1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4ac09fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0ccfc0 0x7f4abc0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4ac09fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc1555f0 0x7f4abc0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4ac09fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0b5ec0 con 0x7f4abc0cc750 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4ac09fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc148b90 con 0x7f4abc1555f0 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4ac09fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc148e70 con 0x7f4abc0ccfc0 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4abad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 0x7f4abc1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4abad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 0x7f4abc1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37372/0 (socket says 172.21.15.33:37372) 2023-10-06T05:14:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4abad9d700 1 -- 172.21.15.33:0/3337610545 learned_addr learned my addr 172.21.15.33:0/3337610545 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.438+0000 7f4aba59c700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0ccfc0 0x7f4abc0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abb59e700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc1555f0 0x7f4abc0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abad9d700 1 -- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0ccfc0 msgr2=0x7f4abc0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abad9d700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0ccfc0 0x7f4abc0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abad9d700 1 -- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc1555f0 msgr2=0x7f4abc0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abad9d700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc1555f0 0x7f4abc0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abad9d700 1 -- 172.21.15.33:0/3337610545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac009420 con 0x7f4abc0cc750 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4aba59c700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0ccfc0 0x7f4abc0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4abad9d700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 0x7f4abc1550b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4abc06be20 tx=0x7f4aac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00bb20 con 0x7f4abc0cc750 2023-10-06T05:14:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac000c90 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.439+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011630 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.440+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc149150 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.440+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc0b32a0 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.440+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4aac011c60 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.440+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f4aac025400 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.441+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc1489f0 con 0x7f4abc0cc750 2023-10-06T05:14:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.443+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4abc1489f0 con 0x7f4abc0cc750 2023-10-06T05:14:02.516 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:02.515+0000 7fcf74e8b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:14:02.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.517+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4abc04c380 con 0x7f4abc0cc750 2023-10-06T05:14:02.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.517+0000 7f4aa2ffd700 1 -- 172.21.15.33:0/3337610545 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f4aac02a060 con 0x7f4abc0cc750 2023-10-06T05:14:02.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:02.519 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:02.113264+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:02.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 msgr2=0x7f4abc1550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:02.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 0x7f4abc1550b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4abc06be20 tx=0x7f4aac017000 comp rx=0 tx=0).stop 2023-10-06T05:14:02.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 shutdown_connections 2023-10-06T05:14:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4abc1555f0 0x7f4abc0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4abc0cc750 0x7f4abc1550b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 --2- 172.21.15.33:0/3337610545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4abc0ccfc0 0x7f4abc0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.518+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 >> 172.21.15.33:0/3337610545 conn(0x7f4abc06cff0 msgr2=0x7f4abc0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.519+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 shutdown_connections 2023-10-06T05:14:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:02.519+0000 7f4ac09fe700 1 -- 172.21.15.33:0/3337610545 wait complete. 2023-10-06T05:14:02.537 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T05:14:02.537 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T05:14:02.618 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:02.616+0000 7fcf74e8b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:14:02.844 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T05:14:02.842+0000 7f6fcd3b4540 -1 Falling back to public interface 2023-10-06T05:14:02.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T05:14:02.845+0000 7fa31ec4e540 -1 Falling back to public interface 2023-10-06T05:14:02.854 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T05:14:02.853+0000 7f79aae59540 -1 Falling back to public interface 2023-10-06T05:14:02.855 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T05:14:02.853+0000 7fbe631da540 -1 Falling back to public interface 2023-10-06T05:14:02.865 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T05:14:02.863+0000 7f9f9eab3540 -1 Falling back to public interface 2023-10-06T05:14:02.871 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T05:14:02.870+0000 7fc831a96540 -1 Falling back to public interface 2023-10-06T05:14:02.879 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T05:14:02.878+0000 7f17783dc540 -1 Falling back to public interface 2023-10-06T05:14:02.884 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T05:14:02.882+0000 7f664afd8540 -1 Falling back to public interface 2023-10-06T05:14:03.054 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:03.052+0000 7fa0b8e7e180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:14:03.228 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:03.226+0000 7fa0b8e7e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:03.342 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:03.339+0000 7fa0b8e7e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:14:03.343 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:03.352 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:03.350+0000 7fcf74e8b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:14:03.508 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.506+0000 7fb9c53ff700 1 Processor -- start 2023-10-06T05:14:03.508 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.506+0000 7fb9c53ff700 1 -- start start 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.507+0000 7fb9c53ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9c0159690 0x7fb9c00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.507+0000 7fb9c53ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb9c00afaf0 0x7fb9c00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.507+0000 7fb9c53ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c015e080 con 0x7fb9c0159a60 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9c53ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c00aff70 con 0x7fb9c0159690 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9c53ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c015d490 con 0x7fb9c00afaf0 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9c0159690 0x7fb9c00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:03.510 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9c0159690 0x7fb9c00cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37378/0 (socket says 172.21.15.33:37378) 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 -- 172.21.15.33:0/3885149628 learned_addr learned my addr 172.21.15.33:0/3885149628 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9bf7fe700 1 --2- 172.21.15.33:0/3885149628 >> v2:172.21.15.167:3300/0 conn(0x7fb9c00afaf0 0x7fb9c00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 -- 172.21.15.33:0/3885149628 >> v2:172.21.15.167:3300/0 conn(0x7fb9c00afaf0 msgr2=0x7fb9c00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 --2- 172.21.15.33:0/3885149628 >> v2:172.21.15.167:3300/0 conn(0x7fb9c00afaf0 0x7fb9c00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 -- 172.21.15.33:0/3885149628 >> v1:172.21.15.167:6789/0 conn(0x7fb9c0159a60 legacy=0x7fb9c00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.508+0000 7fb9beffd700 1 -- 172.21.15.33:0/3885149628 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c015d750 con 0x7fb9c0159690 2023-10-06T05:14:03.511 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9beffd700 1 --2- 172.21.15.33:0/3885149628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c00cd010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b3130 tx=0x7fb9b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40c0f92f778237d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:03.512 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9bdffb700 1 -- 172.21.15.33:0/3885149628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0009e70 con 0x7fb9c0159690 2023-10-06T05:14:03.512 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9bdffb700 1 -- 172.21.15.33:0/3885149628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b000b860 con 0x7fb9c0159690 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9bdffb700 1 -- 172.21.15.33:0/3885149628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000bc00 con 0x7fb9c0159690 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9c53ff700 1 -- 172.21.15.33:0/3885149628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 msgr2=0x7fb9c00cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/3885149628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c00cd010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b3130 tx=0x7fb9b0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9c53ff700 1 -- 172.21.15.33:0/3885149628 shutdown_connections 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/3885149628 >> v2:172.21.15.167:3300/0 conn(0x7fb9c00afaf0 0x7fb9c00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/3885149628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c00cd010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.509+0000 7fb9c53ff700 1 -- 172.21.15.33:0/3885149628 >> 172.21.15.33:0/3885149628 conn(0x7fb9c006ed50 msgr2=0x7fb9c006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.510+0000 7fb9c53ff700 1 -- 172.21.15.33:0/3885149628 shutdown_connections 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.510+0000 7fb9c53ff700 1 -- 172.21.15.33:0/3885149628 wait complete. 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.511+0000 7fb9c53ff700 1 Processor -- start 2023-10-06T05:14:03.513 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.511+0000 7fb9c53ff700 1 -- start start 2023-10-06T05:14:03.516 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9c53ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00afaf0 0x7fb9c014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:03.516 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9c53ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:03.516 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9c53ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c0159a60 0x7fb9c015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:03.516 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9c53ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c014d6b0 con 0x7fb9c0159690 2023-10-06T05:14:03.516 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9c53ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00b1080 con 0x7fb9c0159a60 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9c53ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00b1360 con 0x7fb9c00afaf0 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9be7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9be7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c015e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37382/0 (socket says 172.21.15.33:37382) 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9be7fc700 1 -- 172.21.15.33:0/2424927687 learned_addr learned my addr 172.21.15.33:0/2424927687 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9beffd700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00afaf0 0x7fb9c014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.512+0000 7fb9bf7fe700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c0159a60 0x7fb9c015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9be7fc700 1 -- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00afaf0 msgr2=0x7fb9c014d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9be7fc700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00afaf0 0x7fb9c014d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.517 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9be7fc700 1 -- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c0159a60 msgr2=0x7fb9c015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9be7fc700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c0159a60 0x7fb9c015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9be7fc700 1 -- 172.21.15.33:0/2424927687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00094f0 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9beffd700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00afaf0 0x7fb9c014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9be7fc700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c015e950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c590 tx=0x7fb9b4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4011590 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4011710 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c00b1640 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4011ad0 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c00b0c70 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.513+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb9b4006aa0 con 0x7fb9c0159690 2023-10-06T05:14:03.518 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.514+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fb9b4006d50 con 0x7fb9c0159690 2023-10-06T05:14:03.519 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.514+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c00b3440 con 0x7fb9c0159690 2023-10-06T05:14:03.519 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.517+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb9c00b3440 con 0x7fb9c0159690 2023-10-06T05:14:03.533 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:03.531+0000 7fcf74e8b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:03.592 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.590+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9c0063e90 con 0x7fb9c0159690 2023-10-06T05:14:03.592 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.591+0000 7fb9a6ffd700 1 -- 172.21.15.33:0/2424927687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fb9b4017070 con 0x7fb9c0159690 2023-10-06T05:14:03.593 INFO:teuthology.misc.health.smithi033.stdout: 2023-10-06T05:14:03.593 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:02.113264+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 msgr2=0x7fb9c015e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c015e950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c590 tx=0x7fb9b4007f50 comp rx=0 tx=0).stop 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 shutdown_connections 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c0159a60 0x7fb9c015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00afaf0 0x7fb9c014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 --2- 172.21.15.33:0/2424927687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c0159690 0x7fb9c015e950 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:03.594 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 >> 172.21.15.33:0/2424927687 conn(0x7fb9c006ed50 msgr2=0x7fb9c00d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:03.595 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 shutdown_connections 2023-10-06T05:14:03.595 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:03.592+0000 7fb9c53ff700 1 -- 172.21.15.33:0/2424927687 wait complete. 2023-10-06T05:14:03.609 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T05:14:03.648 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:03.645+0000 7fa0b8e7e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:14:03.649 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:03.647+0000 7fcf74e8b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:14:03.748 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:03.745+0000 7fa0b8e7e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:14:03.953 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:03.952+0000 7fcf74e8b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:14:04.052 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:04.051+0000 7fcf74e8b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:14:04.776 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:04.773+0000 7fa0b8e7e180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:14:04.872 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:04.869+0000 7fa0b8e7e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:14:04.966 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:04.963+0000 7fa0b8e7e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:14:05.104 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.103+0000 7fcf74e8b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:14:05.133 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:05.130+0000 7fa0b8e7e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:14:05.200 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.199+0000 7fcf74e8b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:14:05.239 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:05.237+0000 7fa0b8e7e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:14:05.295 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.294+0000 7fcf74e8b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:14:05.454 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:05.450+0000 7fa0b8e7e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:14:05.479 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.478+0000 7fcf74e8b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:14:05.563 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:05.560+0000 7fa0b8e7e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:14:05.589 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.587+0000 7fcf74e8b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:14:05.680 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:05.677+0000 7fa0b8e7e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:14:05.795 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:05.792+0000 7fa0b8e7e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:14:05.816 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.815+0000 7fcf74e8b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:14:05.937 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:05.935+0000 7fcf74e8b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:14:06.034 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:06.032+0000 7fcf74e8b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:14:06.134 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T05:14:06.133+0000 7fcf74e8b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:14:06.182 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T05:14:06.179+0000 7fbe631da540 -1 osd.6 0 log_to_monitors true 2023-10-06T05:14:06.369 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T05:14:06.367+0000 7f6fcd3b4540 -1 osd.4 0 log_to_monitors true 2023-10-06T05:14:06.438 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T05:14:06.436+0000 7f9f9eab3540 -1 osd.5 0 log_to_monitors true 2023-10-06T05:14:06.447 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T05:14:06.445+0000 7fc831a96540 -1 osd.1 0 log_to_monitors true 2023-10-06T05:14:06.447 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T05:14:06.445+0000 7f664afd8540 -1 osd.7 0 log_to_monitors true 2023-10-06T05:14:06.448 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T05:14:06.447+0000 7f17783dc540 -1 osd.3 0 log_to_monitors true 2023-10-06T05:14:06.623 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T05:14:06.622+0000 7fa31ec4e540 -1 osd.0 0 log_to_monitors true 2023-10-06T05:14:06.670 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T05:14:06.669+0000 7f79aae59540 -1 osd.2 0 log_to_monitors true 2023-10-06T05:14:06.819 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:06.817+0000 7fa070ffa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T05:14:06.820 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T05:14:06.817+0000 7fa070ffa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T05:14:10.076 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T05:14:10.074+0000 7f66401f9700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T05:14:10.082 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T05:14:10.081+0000 7f66379e8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.104 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T05:14:10.103+0000 7fc8255f6700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T05:14:10.114 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T05:14:10.113+0000 7fc81e5e8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.147 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T05:14:10.145+0000 7fbe56bf6700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T05:14:10.154 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T05:14:10.152+0000 7fbe4fbe8700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.203 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T05:14:10.201+0000 7f799e9f6700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T05:14:10.210 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T05:14:10.208+0000 7f6fc0df6700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T05:14:10.211 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T05:14:10.210+0000 7f79979e8700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.214 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T05:14:10.213+0000 7f176bdf6700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T05:14:10.215 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T05:14:10.213+0000 7f6fb9de8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.224 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T05:14:10.222+0000 7f1764de8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.229 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T05:14:10.227+0000 7f9f925f6700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T05:14:10.235 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T05:14:10.233+0000 7f9f8b5e8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.252 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T05:14:10.251+0000 7fa3127f6700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T05:14:10.259 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T05:14:10.257+0000 7fa30b7e8700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:10.415 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:10.578 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.576+0000 7f159f59e700 1 Processor -- start 2023-10-06T05:14:10.578 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.576+0000 7f159f59e700 1 -- start start 2023-10-06T05:14:10.579 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15a006eef0 0x7f15a00c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:10.579 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a00c8530 con 0x7f15a00cb700 2023-10-06T05:14:10.580 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a00c9ca0 con 0x7f15a006e410 2023-10-06T05:14:10.580 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00c7f70 con 0x7f15a006eef0 2023-10-06T05:14:10.580 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f15a00cb700 0x7f15a00c71d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49522/0 (socket says 172.21.15.33:49522) 2023-10-06T05:14:10.580 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159dd9b700 1 -- 172.21.15.33:0/1744690216 learned_addr learned my addr 172.21.15.33:0/1744690216 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:10.580 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159ed9d700 1 --2- 172.21.15.33:0/1744690216 >> v2:172.21.15.167:3300/0 conn(0x7f15a006eef0 0x7f15a00c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:10.581 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159d59a700 1 -- 172.21.15.33:0/1744690216 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145168847 0 0) 0x7f15a00c8530 con 0x7f15a00cb700 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159d59a700 1 -- 172.21.15.33:0/1744690216 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1588003570 con 0x7f15a00cb700 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.578+0000 7f159ed9d700 1 -- 172.21.15.33:0/1744690216 >> v1:172.21.15.167:6789/0 conn(0x7f15a006e410 legacy=0x7f15a006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.579+0000 7f159ed9d700 1 -- 172.21.15.33:0/1744690216 >> v1:172.21.15.33:6789/0 conn(0x7f15a00cb700 legacy=0x7f15a00c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.579+0000 7f159ed9d700 1 -- 172.21.15.33:0/1744690216 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00c8250 con 0x7f15a006eef0 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.579+0000 7f159ed9d700 1 --2- 172.21.15.33:0/1744690216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a006eef0 0x7f15a00c78e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1598009e60 tx=0x7f1598009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c27196b054f4f6d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.579+0000 7f159d59a700 1 -- 172.21.15.33:0/1744690216 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800e070 con 0x7f15a006eef0 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.579+0000 7f159d59a700 1 -- 172.21.15.33:0/1744690216 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800bb60 con 0x7f15a006eef0 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.579+0000 7f159d59a700 1 -- 172.21.15.33:0/1744690216 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598012430 con 0x7f15a006eef0 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 -- 172.21.15.33:0/1744690216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a006eef0 msgr2=0x7f15a00c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.582 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 --2- 172.21.15.33:0/1744690216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a006eef0 0x7f15a00c78e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1598009e60 tx=0x7f1598009780 comp rx=0 tx=0).stop 2023-10-06T05:14:10.583 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 -- 172.21.15.33:0/1744690216 shutdown_connections 2023-10-06T05:14:10.583 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 --2- 172.21.15.33:0/1744690216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a006eef0 0x7f15a00c78e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.583 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 -- 172.21.15.33:0/1744690216 >> 172.21.15.33:0/1744690216 conn(0x7f15a00cc0d0 msgr2=0x7f15a00cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:10.583 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 -- 172.21.15.33:0/1744690216 shutdown_connections 2023-10-06T05:14:10.583 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.580+0000 7f159f59e700 1 -- 172.21.15.33:0/1744690216 wait complete. 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.581+0000 7f159f59e700 1 Processor -- start 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.581+0000 7f159f59e700 1 -- start start 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15a006e410 0x7f15a015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 0x7f15a015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a00cb700 0x7f15a0155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00c3b10 con 0x7f15a006eef0 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00af2b0 con 0x7f15a00cb700 2023-10-06T05:14:10.585 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00af590 con 0x7f15a006e410 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.582+0000 7f159dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 0x7f15a015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 0x7f15a015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56396/0 (socket says 172.21.15.33:56396) 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 -- 172.21.15.33:0/128031861 learned_addr learned my addr 172.21.15.33:0/128031861 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159ed9d700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a00cb700 0x7f15a0155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159e59c700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15a006e410 0x7f15a015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 -- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15a006e410 msgr2=0x7f15a015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15a006e410 0x7f15a015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 -- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a00cb700 msgr2=0x7f15a0155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a00cb700 0x7f15a0155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 -- 172.21.15.33:0/128031861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009a20 con 0x7f15a006eef0 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159dd9b700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 0x7f15a015e680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f159400c590 tx=0x7f1594007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:10.586 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594005cd0 con 0x7f15a006eef0 2023-10-06T05:14:10.587 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594005e50 con 0x7f15a006eef0 2023-10-06T05:14:10.587 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.583+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a00af870 con 0x7f15a006eef0 2023-10-06T05:14:10.587 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.584+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594007320 con 0x7f15a006eef0 2023-10-06T05:14:10.587 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.584+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a00aeed0 con 0x7f15a006eef0 2023-10-06T05:14:10.587 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.585+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a00c3cc0 con 0x7f15a006eef0 2023-10-06T05:14:10.588 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.585+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f159401b3b0 con 0x7f15a006eef0 2023-10-06T05:14:10.588 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.586+0000 7f158e7fc700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1580066ea0 0x7f1580069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:10.588 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.586+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f1594082d80 con 0x7f15a006eef0 2023-10-06T05:14:10.588 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.587+0000 7f159e59c700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1580066ea0 0x7f1580069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:10.588 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.587+0000 7f159e59c700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1580066ea0 0x7f1580069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15a015fec0 tx=0x7f1590009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:10.593 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.591+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15a00c3cc0 con 0x7f15a006eef0 2023-10-06T05:14:10.797 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.795+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a004c380 con 0x7f15a006eef0 2023-10-06T05:14:10.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.796+0000 7f158e7fc700 1 -- 172.21.15.33:0/128031861 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f1594017070 con 0x7f15a006eef0 2023-10-06T05:14:10.798 INFO:teuthology.misc.health.smithi033.stdout: 2023-10-06T05:14:10.798 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":12,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:07.874056+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:10.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1580066ea0 msgr2=0x7f1580069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1580066ea0 0x7f1580069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15a015fec0 tx=0x7f1590009040 comp rx=0 tx=0).stop 2023-10-06T05:14:10.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 msgr2=0x7f15a015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:10.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 0x7f15a015e680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f159400c590 tx=0x7f1594007e00 comp rx=0 tx=0).stop 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 shutdown_connections 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1580066ea0 0x7f1580069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15a00cb700 0x7f15a0155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15a006e410 0x7f15a015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 --2- 172.21.15.33:0/128031861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a006eef0 0x7f15a015e680 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 >> 172.21.15.33:0/128031861 conn(0x7f15a00cc0d0 msgr2=0x7f15a006aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 shutdown_connections 2023-10-06T05:14:10.801 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:10.798+0000 7f159f59e700 1 -- 172.21.15.33:0/128031861 wait complete. 2023-10-06T05:14:10.815 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T05:14:16.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:16.992 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:17.622 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:17.794 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.792+0000 7fd76f59e700 1 Processor -- start 2023-10-06T05:14:17.794 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.793+0000 7fd76f59e700 1 -- start start 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd7700b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7700c4110 0x7fd7700cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7700ca630 con 0x7fd7700b0af0 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7700c9a20 con 0x7fd7700c4110 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7700c9d00 con 0x7fd7700b0720 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7700c4110 0x7fd7700cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7700c4110 0x7fd7700cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56400/0 (socket says 172.21.15.33:56400) 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76ed9d700 1 -- 172.21.15.33:0/1946674871 learned_addr learned my addr 172.21.15.33:0/1946674871 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:17.796 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.794+0000 7fd76e59c700 1 --2- 172.21.15.33:0/1946674871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd7700b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:17.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76ed9d700 1 -- 172.21.15.33:0/1946674871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 msgr2=0x7fd7700b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:17.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76ed9d700 1 --2- 172.21.15.33:0/1946674871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd7700b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:17.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76ed9d700 1 -- 172.21.15.33:0/1946674871 >> v1:172.21.15.167:6789/0 conn(0x7fd7700b0af0 legacy=0x7fd7700c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:17.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76ed9d700 1 -- 172.21.15.33:0/1946674871 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700c9fe0 con 0x7fd7700c4110 2023-10-06T05:14:17.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76ed9d700 1 --2- 172.21.15.33:0/1946674871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd7700cc8d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd768009f20 tx=0x7fd768009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f056a1794dafa249 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:17.798 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76d59a700 1 -- 172.21.15.33:0/1946674871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76800e070 con 0x7fd7700c4110 2023-10-06T05:14:17.799 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76d59a700 1 -- 172.21.15.33:0/1946674871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76800ba60 con 0x7fd7700c4110 2023-10-06T05:14:17.799 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.795+0000 7fd76d59a700 1 -- 172.21.15.33:0/1946674871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7680092e0 con 0x7fd7700c4110 2023-10-06T05:14:17.799 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 -- 172.21.15.33:0/1946674871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 msgr2=0x7fd7700cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:17.799 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 --2- 172.21.15.33:0/1946674871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd7700cc8d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd768009f20 tx=0x7fd768009840 comp rx=0 tx=0).stop 2023-10-06T05:14:17.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 -- 172.21.15.33:0/1946674871 shutdown_connections 2023-10-06T05:14:17.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 --2- 172.21.15.33:0/1946674871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd7700cc8d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:17.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 --2- 172.21.15.33:0/1946674871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd7700b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:17.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 -- 172.21.15.33:0/1946674871 >> 172.21.15.33:0/1946674871 conn(0x7fd770097620 msgr2=0x7fd770067e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:17.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 -- 172.21.15.33:0/1946674871 shutdown_connections 2023-10-06T05:14:17.800 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.796+0000 7fd76f59e700 1 -- 172.21.15.33:0/1946674871 wait complete. 2023-10-06T05:14:17.802 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.797+0000 7fd76f59e700 1 Processor -- start 2023-10-06T05:14:17.802 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.797+0000 7fd76f59e700 1 -- start start 2023-10-06T05:14:17.802 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.798+0000 7fd76f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd770069870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.798+0000 7fd76f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd770146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.798+0000 7fd76f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd77015b020 0x7fd77015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.798+0000 7fd76f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7700687d0 con 0x7fd7700c4110 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.798+0000 7fd76f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd770068ab0 con 0x7fd77015b020 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.798+0000 7fd76f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd770068d90 con 0x7fd7700b0720 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd770146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd770146cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56414/0 (socket says 172.21.15.33:56414) 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 -- 172.21.15.33:0/880010017 learned_addr learned my addr 172.21.15.33:0/880010017 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76e59c700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd770069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76ed9d700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd77015b020 0x7fd77015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:17.803 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 -- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 msgr2=0x7fd770069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd770069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 -- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd77015b020 msgr2=0x7fd77015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd77015b020 0x7fd77015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 -- 172.21.15.33:0/880010017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768009ae0 con 0x7fd7700c4110 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.799+0000 7fd76dd9b700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd770146cd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c590 tx=0x7fd76400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.800+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764011590 con 0x7fd7700c4110 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.800+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640116f0 con 0x7fd7700c4110 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.800+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd770069070 con 0x7fd7700c4110 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.800+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764011ab0 con 0x7fd7700c4110 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.800+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd770069db0 con 0x7fd7700c4110 2023-10-06T05:14:17.804 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.801+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd770069590 con 0x7fd7700c4110 2023-10-06T05:14:17.805 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.802+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd76401b3b0 con 0x7fd7700c4110 2023-10-06T05:14:17.805 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.803+0000 7fd75e7fc700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd750066f00 0x7fd7500693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:17.805 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.803+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd764083830 con 0x7fd7700c4110 2023-10-06T05:14:17.805 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.803+0000 7fd76e59c700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd750066f00 0x7fd7500693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:17.806 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.804+0000 7fd76e59c700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd750066f00 0x7fd7500693b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd760009d40 tx=0x7fd760008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:17.809 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:17.807+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd770069590 con 0x7fd7700c4110 2023-10-06T05:14:18.013 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.011+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd770069590 con 0x7fd7700c4110 2023-10-06T05:14:18.014 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.012+0000 7fd75e7fc700 1 -- 172.21.15.33:0/880010017 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7fd770069590 con 0x7fd7700c4110 2023-10-06T05:14:18.015 INFO:teuthology.misc.health.smithi033.stdout: 2023-10-06T05:14:18.015 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":16,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:14.636077+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:18.018 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd750066f00 msgr2=0x7fd7500693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.018 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd750066f00 0x7fd7500693b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd760009d40 tx=0x7fd760008040 comp rx=0 tx=0).stop 2023-10-06T05:14:18.018 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 msgr2=0x7fd770146cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.018 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd770146cd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c590 tx=0x7fd76400c950 comp rx=0 tx=0).stop 2023-10-06T05:14:18.019 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 shutdown_connections 2023-10-06T05:14:18.019 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd750066f00 0x7fd7500693b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.019 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd77015b020 0x7fd77015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.019 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7700b0720 0x7fd770069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.019 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 --2- 172.21.15.33:0/880010017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7700c4110 0x7fd770146cd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.019 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 >> 172.21.15.33:0/880010017 conn(0x7fd770097620 msgr2=0x7fd7700ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:18.020 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 shutdown_connections 2023-10-06T05:14:18.020 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T05:14:18.015+0000 7fd76f59e700 1 -- 172.21.15.33:0/880010017 wait complete. 2023-10-06T05:14:18.031 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T05:14:18.031 INFO:tasks.ceph:Creating RBD pool 2023-10-06T05:14:18.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T05:14:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.210+0000 7f2946b9d700 1 Processor -- start 2023-10-06T05:14:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.210+0000 7f2946b9d700 1 -- start start 2023-10-06T05:14:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2946b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2940159850 0x7f294006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2946b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f294006d140 0x7f294006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2946b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29400b0cd0 con 0x7f294015acd0 2023-10-06T05:14:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2946b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294015a840 con 0x7f2940159850 2023-10-06T05:14:18.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2946b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2940159f10 con 0x7f294006d140 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2940159850 0x7f294006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2940159850 0x7f294006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56422/0 (socket says 172.21.15.33:56422) 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 -- 172.21.15.33:0/3767067573 learned_addr learned my addr 172.21.15.33:0/3767067573 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f294639c700 1 --2- 172.21.15.33:0/3767067573 >> v2:172.21.15.167:3300/0 conn(0x7f294006d140 0x7f294006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 -- 172.21.15.33:0/3767067573 >> v2:172.21.15.167:3300/0 conn(0x7f294006d140 msgr2=0x7f294006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 --2- 172.21.15.33:0/3767067573 >> v2:172.21.15.167:3300/0 conn(0x7f294006d140 0x7f294006edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 -- 172.21.15.33:0/3767067573 >> v1:172.21.15.167:6789/0 conn(0x7f294015acd0 legacy=0x7f294006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 -- 172.21.15.33:0/3767067573 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294015a1f0 con 0x7f2940159850 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.212+0000 7f2945b9b700 1 --2- 172.21.15.33:0/3767067573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2940159850 0x7f294006e890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f294006afb0 tx=0x7f293c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a1bbe345e82ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:18.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2944b99700 1 -- 172.21.15.33:0/3767067573 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00b820 con 0x7f2940159850 2023-10-06T05:14:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2944b99700 1 -- 172.21.15.33:0/3767067573 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c00b980 con 0x7f2940159850 2023-10-06T05:14:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2944b99700 1 -- 172.21.15.33:0/3767067573 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00bd20 con 0x7f2940159850 2023-10-06T05:14:18.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 -- 172.21.15.33:0/3767067573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2940159850 msgr2=0x7f294006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 --2- 172.21.15.33:0/3767067573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2940159850 0x7f294006e890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f294006afb0 tx=0x7f293c009250 comp rx=0 tx=0).stop 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 -- 172.21.15.33:0/3767067573 shutdown_connections 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 --2- 172.21.15.33:0/3767067573 >> v2:172.21.15.167:3300/0 conn(0x7f294006d140 0x7f294006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 --2- 172.21.15.33:0/3767067573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2940159850 0x7f294006e890 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 -- 172.21.15.33:0/3767067573 >> 172.21.15.33:0/3767067573 conn(0x7f294006cd40 msgr2=0x7f29400ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 -- 172.21.15.33:0/3767067573 shutdown_connections 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.213+0000 7f2946b9d700 1 -- 172.21.15.33:0/3767067573 wait complete. 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.215+0000 7f2946b9d700 1 Processor -- start 2023-10-06T05:14:18.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.215+0000 7f2946b9d700 1 -- start start 2023-10-06T05:14:18.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2946b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f294006d140 0x7f2940169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2946b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2940159850 0x7f29400ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2946b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 0x7f29400cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2946b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29400cdcb0 con 0x7f294015acd0 2023-10-06T05:14:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2946b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294016a220 con 0x7f294006d140 2023-10-06T05:14:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2946b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29400cd6f0 con 0x7f2940159850 2023-10-06T05:14:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f294639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 0x7f29400cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f294639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 0x7f29400cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56432/0 (socket says 172.21.15.33:56432) 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f294639c700 1 -- 172.21.15.33:0/2014102581 learned_addr learned my addr 172.21.15.33:0/2014102581 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f2945b9b700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f294006d140 0x7f2940169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.216+0000 7f294539a700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2940159850 0x7f29400ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f294639c700 1 -- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2940159850 msgr2=0x7f29400ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f294639c700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2940159850 0x7f29400ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f294639c700 1 -- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f294006d140 msgr2=0x7f2940169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f294639c700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f294006d140 0x7f2940169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f294639c700 1 -- 172.21.15.33:0/2014102581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0094f0 con 0x7f294015acd0 2023-10-06T05:14:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f2945b9b700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f294006d140 0x7f2940169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f294639c700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 0x7f29400cad10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2938007ef0 tx=0x7f2938009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293800eb10 con 0x7f294015acd0 2023-10-06T05:14:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293800ec70 con 0x7f294015acd0 2023-10-06T05:14:18.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29400cd9d0 con 0x7f294015acd0 2023-10-06T05:14:18.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938014420 con 0x7f294015acd0 2023-10-06T05:14:18.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.217+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29400ac2d0 con 0x7f294015acd0 2023-10-06T05:14:18.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.218+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294004c380 con 0x7f294015acd0 2023-10-06T05:14:18.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.219+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2938014830 con 0x7f294015acd0 2023-10-06T05:14:18.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.220+0000 7f2935ffb700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2924066f30 0x7f29240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.220+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2938084990 con 0x7f294015acd0 2023-10-06T05:14:18.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.220+0000 7f2945b9b700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2924066f30 0x7f29240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.221+0000 7f2945b9b700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2924066f30 0x7f29240693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f294006afb0 tx=0x7f293c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:18.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.224+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2938051eb0 con 0x7f294015acd0 2023-10-06T05:14:18.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.423+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f29400caaf0 con 0x7f294015acd0 2023-10-06T05:14:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.659+0000 7f2935ffb700 1 -- 172.21.15.33:0/2014102581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f29380554d0 con 0x7f294015acd0 2023-10-06T05:14:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2924066f30 msgr2=0x7f29240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2924066f30 0x7f29240693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f294006afb0 tx=0x7f293c016000 comp rx=0 tx=0).stop 2023-10-06T05:14:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 msgr2=0x7f29400cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 0x7f29400cad10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2938007ef0 tx=0x7f2938009840 comp rx=0 tx=0).stop 2023-10-06T05:14:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 shutdown_connections 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2924066f30 0x7f29240693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f294015acd0 0x7f29400cad10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f294006d140 0x7f2940169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 --2- 172.21.15.33:0/2014102581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2940159850 0x7f29400ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 >> 172.21.15.33:0/2014102581 conn(0x7f294006cd40 msgr2=0x7f29400af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 shutdown_connections 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.662+0000 7f2946b9d700 1 -- 172.21.15.33:0/2014102581 wait complete. 2023-10-06T05:14:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'rbd' created 2023-10-06T05:14:18.681 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T05:14:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.862+0000 7f12bffff700 1 Processor -- start 2023-10-06T05:14:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.862+0000 7f12bffff700 1 -- start start 2023-10-06T05:14:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.863+0000 7f12bffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12c0159400 0x7f12c00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.863+0000 7f12bffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f12c0068260 0x7f12c00af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.863+0000 7f12bffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c00a9dd0 con 0x7f12c006dbf0 2023-10-06T05:14:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.863+0000 7f12bffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c006d760 con 0x7f12c0159400 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.863+0000 7f12bffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c00aa8c0 con 0x7f12c0068260 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12beffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12c0159400 0x7f12c00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12be7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f12c006dbf0 0x7f12c0067b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49536/0 (socket says 172.21.15.33:49536) 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12be7fc700 1 -- 172.21.15.33:0/4265147400 learned_addr learned my addr 172.21.15.33:0/4265147400 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12bf7fe700 1 --2- 172.21.15.33:0/4265147400 >> v2:172.21.15.167:3300/0 conn(0x7f12c0068260 0x7f12c00af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12beffd700 1 -- 172.21.15.33:0/4265147400 >> v2:172.21.15.167:3300/0 conn(0x7f12c0068260 msgr2=0x7f12c00af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12beffd700 1 --2- 172.21.15.33:0/4265147400 >> v2:172.21.15.167:3300/0 conn(0x7f12c0068260 0x7f12c00af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12beffd700 1 -- 172.21.15.33:0/4265147400 >> v1:172.21.15.33:6789/0 conn(0x7f12c006dbf0 legacy=0x7f12c0067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12beffd700 1 -- 172.21.15.33:0/4265147400 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00aa5e0 con 0x7f12c0159400 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12bdffb700 1 -- 172.21.15.33:0/4265147400 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570761657 0 0) 0x7f12b4001110 con 0x7f12c006dbf0 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.864+0000 7f12beffd700 1 --2- 172.21.15.33:0/4265147400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0159400 0x7f12c00a9890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12b00098e0 tx=0x7f12b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac137bb9b5f46dd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bdffb700 1 -- 172.21.15.33:0/4265147400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0009e80 con 0x7f12c0159400 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bdffb700 1 -- 172.21.15.33:0/4265147400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b000b820 con 0x7f12c0159400 2023-10-06T05:14:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bdffb700 1 -- 172.21.15.33:0/4265147400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000bba0 con 0x7f12c0159400 2023-10-06T05:14:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 -- 172.21.15.33:0/4265147400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0159400 msgr2=0x7f12c00a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 --2- 172.21.15.33:0/4265147400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0159400 0x7f12c00a9890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12b00098e0 tx=0x7f12b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 -- 172.21.15.33:0/4265147400 shutdown_connections 2023-10-06T05:14:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 --2- 172.21.15.33:0/4265147400 >> v2:172.21.15.167:3300/0 conn(0x7f12c0068260 0x7f12c00af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 --2- 172.21.15.33:0/4265147400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0159400 0x7f12c00a9890 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 -- 172.21.15.33:0/4265147400 >> 172.21.15.33:0/4265147400 conn(0x7f12c006ebf0 msgr2=0x7f12c006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 -- 172.21.15.33:0/4265147400 shutdown_connections 2023-10-06T05:14:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.865+0000 7f12bffff700 1 -- 172.21.15.33:0/4265147400 wait complete. 2023-10-06T05:14:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.867+0000 7f12bffff700 1 Processor -- start 2023-10-06T05:14:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.867+0000 7f12bffff700 1 -- start start 2023-10-06T05:14:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 0x7f12c015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12c006dbf0 0x7f12c014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12c0159400 0x7f12c00b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c015d630 con 0x7f12c0068260 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c015d910 con 0x7f12c006dbf0 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c015dbf0 con 0x7f12c0159400 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 0x7f12c015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 0x7f12c015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56454/0 (socket says 172.21.15.33:56454) 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 -- 172.21.15.33:0/781441339 learned_addr learned my addr 172.21.15.33:0/781441339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12be7fc700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12c006dbf0 0x7f12c014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12bf7fe700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12c0159400 0x7f12c00b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 -- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12c0159400 msgr2=0x7f12c00b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12c0159400 0x7f12c00b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 -- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12c006dbf0 msgr2=0x7f12c014d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.868+0000 7f12beffd700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12c006dbf0 0x7f12c014d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12beffd700 1 -- 172.21.15.33:0/781441339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0009420 con 0x7f12c0068260 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12beffd700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 0x7f12c015cde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12c006b360 tx=0x7f12b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b001f070 con 0x7f12c0068260 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b000b990 con 0x7f12c0068260 2023-10-06T05:14:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c0152ef0 con 0x7f12c0068260 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011650 con 0x7f12c0068260 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.869+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c014d9c0 con 0x7f12c0068260 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.870+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c0152d50 con 0x7f12c0068260 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.871+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12b0011a40 con 0x7f12c0068260 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.872+0000 7f12a6ffd700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f12ac066f30 0x7f12ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.872+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f12b0085c70 con 0x7f12c0068260 2023-10-06T05:14:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.873+0000 7f12be7fc700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f12ac066f30 0x7f12ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.873+0000 7f12be7fc700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f12ac066f30 0x7f12ac0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12c0158a30 tx=0x7f12b4007b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:18.876+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12b0053170 con 0x7f12c0068260 2023-10-06T05:14:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.077+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12c0152d50 con 0x7f12c0068260 2023-10-06T05:14:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.667+0000 7f12a6ffd700 1 -- 172.21.15.33:0/781441339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f12c0152d50 con 0x7f12c0068260 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f12ac066f30 msgr2=0x7f12ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f12ac066f30 0x7f12ac0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12c0158a30 tx=0x7f12b4007b90 comp rx=0 tx=0).stop 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 msgr2=0x7f12c015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 0x7f12c015cde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12c006b360 tx=0x7f12b0017000 comp rx=0 tx=0).stop 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 shutdown_connections 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f12ac066f30 0x7f12ac0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12c0159400 0x7f12c00b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12c0068260 0x7f12c015cde0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 --2- 172.21.15.33:0/781441339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12c006dbf0 0x7f12c014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 >> 172.21.15.33:0/781441339 conn(0x7f12c006ebf0 msgr2=0x7f12c00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 shutdown_connections 2023-10-06T05:14:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.671+0000 7f12bffff700 1 -- 172.21.15.33:0/781441339 wait complete. 2023-10-06T05:14:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T05:14:19.694 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T05:14:19.695 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:14:19.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:14:19.699 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:14:19.700 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:14:19.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:14:19.704 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:14:19.704 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:14:19.704 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:14:19.709 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:14:19.709 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:14:19.709 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:14:19.714 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:14:19.714 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T05:14:19.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:19.845 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:14:19.845 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:14:19.846 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:14:19.846 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:14:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.915+0000 7fb9c639d700 1 Processor -- start 2023-10-06T05:14:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.915+0000 7fb9c639d700 1 -- start start 2023-10-06T05:14:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.916+0000 7fb9c639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00ccfc0 0x7fb9c00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.916+0000 7fb9c639d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00cbd80 con 0x7fb9c00cf3b0 2023-10-06T05:14:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.916+0000 7fb9c639d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00af5f0 con 0x7fb9c00cc750 2023-10-06T05:14:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.916+0000 7fb9c639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00af8d0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9c4b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb9c00cf3b0 0x7fb9c01594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49990/0 (socket says 172.21.15.33:49990) 2023-10-06T05:14:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9c4b9a700 1 -- 172.21.15.33:0/4063859172 learned_addr learned my addr 172.21.15.33:0/4063859172 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9c539b700 1 --2- 172.21.15.33:0/4063859172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00ccfc0 0x7fb9c00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9b7fff700 1 -- 172.21.15.33:0/4063859172 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912595705 0 0) 0x7fb9c00af5f0 con 0x7fb9c00cc750 2023-10-06T05:14:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9b7fff700 1 -- 172.21.15.33:0/4063859172 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ac003570 con 0x7fb9c00cc750 2023-10-06T05:14:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9b7fff700 1 -- 172.21.15.33:0/4063859172 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841352884 0 0) 0x7fb9c00cbd80 con 0x7fb9c00cf3b0 2023-10-06T05:14:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9c539b700 1 -- 172.21.15.33:0/4063859172 >> v1:172.21.15.167:6789/0 conn(0x7fb9c00cc750 legacy=0x7fb9c00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9c539b700 1 -- 172.21.15.33:0/4063859172 >> v1:172.21.15.33:6789/0 conn(0x7fb9c00cf3b0 legacy=0x7fb9c01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.917+0000 7fb9c539b700 1 -- 172.21.15.33:0/4063859172 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00afbb0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c539b700 1 --2- 172.21.15.33:0/4063859172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00ccfc0 0x7fb9c00cc210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009910 tx=0x7fb9bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a3860beee04f19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9b7fff700 1 -- 172.21.15.33:0/4063859172 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00b680 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9b7fff700 1 -- 172.21.15.33:0/4063859172 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc00b7e0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9b7fff700 1 -- 172.21.15.33:0/4063859172 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00bba0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c639d700 1 -- 172.21.15.33:0/4063859172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00ccfc0 msgr2=0x7fb9c00cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4063859172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00ccfc0 0x7fb9c00cc210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009910 tx=0x7fb9bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c639d700 1 -- 172.21.15.33:0/4063859172 shutdown_connections 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4063859172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00ccfc0 0x7fb9c00cc210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c639d700 1 -- 172.21.15.33:0/4063859172 >> 172.21.15.33:0/4063859172 conn(0x7fb9c006cff0 msgr2=0x7fb9c006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.918+0000 7fb9c639d700 1 -- 172.21.15.33:0/4063859172 shutdown_connections 2023-10-06T05:14:19.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.919+0000 7fb9c639d700 1 -- 172.21.15.33:0/4063859172 wait complete. 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.920+0000 7fb9c639d700 1 Processor -- start 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.920+0000 7fb9c639d700 1 -- start start 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00cc750 0x7fb9c0154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 0x7fb9c00c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c00c9c60 0x7fb9c0154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00c83c0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00c7e00 con 0x7fb9c00c9c60 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00c80e0 con 0x7fb9c00cc750 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c4b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 0x7fb9c00c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c4b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 0x7fb9c00c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32866/0 (socket says 172.21.15.33:32866) 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c539b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00cc750 0x7fb9c0154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c4b9a700 1 -- 172.21.15.33:0/4181230307 learned_addr learned my addr 172.21.15.33:0/4181230307 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.921+0000 7fb9c5b9c700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c00c9c60 0x7fb9c0154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c4b9a700 1 -- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00cc750 msgr2=0x7fb9c0154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c4b9a700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00cc750 0x7fb9c0154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c4b9a700 1 -- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c00c9c60 msgr2=0x7fb9c0154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c4b9a700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c00c9c60 0x7fb9c0154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c4b9a700 1 -- 172.21.15.33:0/4181230307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc009420 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c539b700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00cc750 0x7fb9c0154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c4b9a700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 0x7fb9c00c75b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000c5e0 tx=0x7fb9b0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0005b30 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0005c90 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c0169c20 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0007300 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.922+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c00c7af0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.923+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0169a80 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.924+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9b001b3b0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.926+0000 7fb9b57fa700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9a4066f30 0x7fb9a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.926+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb9b00837e0 con 0x7fb9c00ccfc0 2023-10-06T05:14:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.926+0000 7fb9c539b700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9a4066f30 0x7fb9a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.927+0000 7fb9c539b700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9a4066f30 0x7fb9a40693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0093f0 tx=0x7fb9bc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:19.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:19.930+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9c0169a80 con 0x7fb9c00ccfc0 2023-10-06T05:14:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.133+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9c016a140 con 0x7fb9c00ccfc0 2023-10-06T05:14:20.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.134+0000 7fb9b57fa700 1 -- 172.21.15.33:0/4181230307 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fb9c016a140 con 0x7fb9c00ccfc0 2023-10-06T05:14:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:20.136 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":18,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:19.658344+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9a4066f30 msgr2=0x7fb9a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9a4066f30 0x7fb9a40693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0093f0 tx=0x7fb9bc017000 comp rx=0 tx=0).stop 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 msgr2=0x7fb9c00c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 0x7fb9c00c75b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000c5e0 tx=0x7fb9b0000fe0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 shutdown_connections 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9a4066f30 0x7fb9a40693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9c00c9c60 0x7fb9c0154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c00cc750 0x7fb9c0154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 --2- 172.21.15.33:0/4181230307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9c00ccfc0 0x7fb9c00c75b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 >> 172.21.15.33:0/4181230307 conn(0x7fb9c006cff0 msgr2=0x7fb9c00c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:20.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 shutdown_connections 2023-10-06T05:14:20.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.136+0000 7fb9c639d700 1 -- 172.21.15.33:0/4181230307 wait complete. 2023-10-06T05:14:20.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:14:20.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:20.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.344+0000 7f1248d9d700 1 Processor -- start 2023-10-06T05:14:20.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.344+0000 7f1248d9d700 1 -- start start 2023-10-06T05:14:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f1248d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12440af540 0x7f12440af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f1248d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12440ce890 con 0x7f124415a650 2023-10-06T05:14:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f1248d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124415aac0 con 0x7f12440cd5e0 2023-10-06T05:14:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f1248d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124415d890 con 0x7f12440af540 2023-10-06T05:14:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f12437fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12440af540 0x7f12440af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f1242ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f124415a650 0x7f12440cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50000/0 (socket says 172.21.15.33:50000) 2023-10-06T05:14:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f1242ffd700 1 -- 172.21.15.33:0/729596849 learned_addr learned my addr 172.21.15.33:0/729596849 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.345+0000 7f12437fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12440af540 0x7f12440af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32872/0 (socket says 172.21.15.33:32872) 2023-10-06T05:14:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12437fe700 1 -- 172.21.15.33:0/729596849 >> v1:172.21.15.167:6789/0 conn(0x7f12440cd5e0 legacy=0x7f12440ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12437fe700 1 -- 172.21.15.33:0/729596849 >> v1:172.21.15.33:6789/0 conn(0x7f124415a650 legacy=0x7f12440cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12437fe700 1 -- 172.21.15.33:0/729596849 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124415db70 con 0x7f12440af540 2023-10-06T05:14:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12427fc700 1 -- 172.21.15.33:0/729596849 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420000007 0 0) 0x7f12440ce890 con 0x7f124415a650 2023-10-06T05:14:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12437fe700 1 --2- 172.21.15.33:0/729596849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12440af540 0x7f12440af910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12340098e0 tx=0x7f12340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f515a2ff6aada9e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12427fc700 1 -- 172.21.15.33:0/729596849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234009e80 con 0x7f12440af540 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12427fc700 1 -- 172.21.15.33:0/729596849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123400b860 con 0x7f12440af540 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.346+0000 7f12427fc700 1 -- 172.21.15.33:0/729596849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400bbe0 con 0x7f12440af540 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 -- 172.21.15.33:0/729596849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12440af540 msgr2=0x7f12440af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 --2- 172.21.15.33:0/729596849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12440af540 0x7f12440af910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12340098e0 tx=0x7f12340091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 -- 172.21.15.33:0/729596849 shutdown_connections 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 --2- 172.21.15.33:0/729596849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12440af540 0x7f12440af910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 -- 172.21.15.33:0/729596849 >> 172.21.15.33:0/729596849 conn(0x7f124406d9b0 msgr2=0x7f124406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 -- 172.21.15.33:0/729596849 shutdown_connections 2023-10-06T05:14:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.347+0000 7f1248d9d700 1 -- 172.21.15.33:0/729596849 wait complete. 2023-10-06T05:14:20.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.348+0000 7f1248d9d700 1 Processor -- start 2023-10-06T05:14:20.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.349+0000 7f1248d9d700 1 -- start start 2023-10-06T05:14:20.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.349+0000 7f1248d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12440af540 0x7f12440b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.349+0000 7f1248d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12440cd5e0 0x7f1244157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1248d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 0x7f12440b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1248d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124416d520 con 0x7f124415a650 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1248d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124416d770 con 0x7f12440cd5e0 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1248d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124416da50 con 0x7f12440af540 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 0x7f12440b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 0x7f12440b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32878/0 (socket says 172.21.15.33:32878) 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 -- 172.21.15.33:0/1395682613 learned_addr learned my addr 172.21.15.33:0/1395682613 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1242ffd700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12440cd5e0 0x7f1244157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f12437fe700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12440af540 0x7f12440b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 -- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12440af540 msgr2=0x7f12440b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12440af540 0x7f12440b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 -- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12440cd5e0 msgr2=0x7f1244157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12440cd5e0 0x7f1244157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 -- 172.21.15.33:0/1395682613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1234009420 con 0x7f124415a650 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.350+0000 7f1243fff700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 0x7f12440b2070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c910 tx=0x7f123c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.351+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00a7e0 con 0x7f124415a650 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.351+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00a940 con 0x7f124415a650 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.351+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00ad00 con 0x7f124415a650 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.351+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124416dd30 con 0x7f124415a650 2023-10-06T05:14:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.351+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12440b5f70 con 0x7f124415a650 2023-10-06T05:14:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.352+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124416e250 con 0x7f124415a650 2023-10-06T05:14:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.353+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f123c012630 con 0x7f124415a650 2023-10-06T05:14:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.353+0000 7f122b7fe700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1230066f30 0x7f12300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.353+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f123c084660 con 0x7f124415a650 2023-10-06T05:14:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.354+0000 7f12437fe700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1230066f30 0x7f12300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.354+0000 7f12437fe700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1230066f30 0x7f12300693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12340093f0 tx=0x7f1234017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:20.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.357+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f124416e250 con 0x7f124415a650 2023-10-06T05:14:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.558+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f124416e250 con 0x7f124415a650 2023-10-06T05:14:20.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.558+0000 7f122b7fe700 1 -- 172.21.15.33:0/1395682613 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f124416e250 con 0x7f124415a650 2023-10-06T05:14:20.560 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:14:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1230066f30 msgr2=0x7f12300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1230066f30 0x7f12300693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f12340093f0 tx=0x7f1234017000 comp rx=0 tx=0).stop 2023-10-06T05:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 msgr2=0x7f12440b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 0x7f12440b2070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f123c00c910 tx=0x7f123c0016f0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 shutdown_connections 2023-10-06T05:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1230066f30 0x7f12300693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124415a650 0x7f12440b2070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12440af540 0x7f12440b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 --2- 172.21.15.33:0/1395682613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12440cd5e0 0x7f1244157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 >> 172.21.15.33:0/1395682613 conn(0x7f124406d9b0 msgr2=0x7f12440cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 shutdown_connections 2023-10-06T05:14:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.561+0000 7f1248d9d700 1 -- 172.21.15.33:0/1395682613 wait complete. 2023-10-06T05:14:20.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:20.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.770+0000 7f050239d700 1 Processor -- start 2023-10-06T05:14:20.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.770+0000 7f050239d700 1 -- start start 2023-10-06T05:14:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050239d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04fc0cc7e0 0x7f04fc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050239d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04fc0b01f0 0x7f04fc0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc0ad430 con 0x7f04fc156370 2023-10-06T05:14:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050239d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0afe20 con 0x7f04fc0b01f0 2023-10-06T05:14:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050239d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0b0b70 con 0x7f04fc0cc7e0 2023-10-06T05:14:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050139b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f04fc156370 0x7f04fc0c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50004/0 (socket says 172.21.15.33:50004) 2023-10-06T05:14:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f050139b700 1 -- 172.21.15.33:0/4211236352 learned_addr learned my addr 172.21.15.33:0/4211236352 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f0501b9c700 1 --2- 172.21.15.33:0/4211236352 >> v2:172.21.15.33:3300/0 conn(0x7f04fc0b01f0 0x7f04fc0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f0500b9a700 1 --2- 172.21.15.33:0/4211236352 >> v2:172.21.15.167:3300/0 conn(0x7f04fc0cc7e0 0x7f04fc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f04f3fff700 1 -- 172.21.15.33:0/4211236352 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476307297 0 0) 0x7f04fc0ad430 con 0x7f04fc156370 2023-10-06T05:14:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.772+0000 7f04f3fff700 1 -- 172.21.15.33:0/4211236352 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e8003570 con 0x7f04fc156370 2023-10-06T05:14:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f0501b9c700 1 -- 172.21.15.33:0/4211236352 >> v2:172.21.15.167:3300/0 conn(0x7f04fc0cc7e0 msgr2=0x7f04fc0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f0501b9c700 1 --2- 172.21.15.33:0/4211236352 >> v2:172.21.15.167:3300/0 conn(0x7f04fc0cc7e0 0x7f04fc0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f0501b9c700 1 -- 172.21.15.33:0/4211236352 >> v1:172.21.15.33:6789/0 conn(0x7f04fc156370 legacy=0x7f04fc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f0501b9c700 1 -- 172.21.15.33:0/4211236352 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc06bc70 con 0x7f04fc0b01f0 2023-10-06T05:14:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f0501b9c700 1 --2- 172.21.15.33:0/4211236352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc0b01f0 0x7f04fc0b0630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0c94c0 tx=0x7f04f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c259fbd239e2988c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f04f3fff700 1 -- 172.21.15.33:0/4211236352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400e070 con 0x7f04fc0b01f0 2023-10-06T05:14:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f04f3fff700 1 -- 172.21.15.33:0/4211236352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f400bab0 con 0x7f04fc0b01f0 2023-10-06T05:14:20.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.773+0000 7f04f3fff700 1 -- 172.21.15.33:0/4211236352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400be70 con 0x7f04fc0b01f0 2023-10-06T05:14:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 -- 172.21.15.33:0/4211236352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc0b01f0 msgr2=0x7f04fc0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 --2- 172.21.15.33:0/4211236352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc0b01f0 0x7f04fc0b0630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0c94c0 tx=0x7f04f4009780 comp rx=0 tx=0).stop 2023-10-06T05:14:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 -- 172.21.15.33:0/4211236352 shutdown_connections 2023-10-06T05:14:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 --2- 172.21.15.33:0/4211236352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc0b01f0 0x7f04fc0b0630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 --2- 172.21.15.33:0/4211236352 >> v2:172.21.15.167:3300/0 conn(0x7f04fc0cc7e0 0x7f04fc0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 -- 172.21.15.33:0/4211236352 >> 172.21.15.33:0/4211236352 conn(0x7f04fc06d030 msgr2=0x7f04fc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 -- 172.21.15.33:0/4211236352 shutdown_connections 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.774+0000 7f050239d700 1 -- 172.21.15.33:0/4211236352 wait complete. 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.775+0000 7f050239d700 1 Processor -- start 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.775+0000 7f050239d700 1 -- start start 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f050239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04fc0b01f0 0x7f04fc16d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f050239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04fc0cc7e0 0x7f04fc14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f050239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 0x7f04fc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f050239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16dec0 con 0x7f04fc156370 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f050239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc157fb0 con 0x7f04fc0cc7e0 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f050239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc1573a0 con 0x7f04fc0b01f0 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f0501b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 0x7f04fc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f0501b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 0x7f04fc14e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32892/0 (socket says 172.21.15.33:32892) 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.776+0000 7f0501b9c700 1 -- 172.21.15.33:0/3227047749 learned_addr learned my addr 172.21.15.33:0/3227047749 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0500b9a700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04fc0cc7e0 0x7f04fc14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f050139b700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04fc0b01f0 0x7f04fc16d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0501b9c700 1 -- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04fc0b01f0 msgr2=0x7f04fc16d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0501b9c700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04fc0b01f0 0x7f04fc16d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0501b9c700 1 -- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04fc0cc7e0 msgr2=0x7f04fc14bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0501b9c700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04fc0cc7e0 0x7f04fc14bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0501b9c700 1 -- 172.21.15.33:0/3227047749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4009a20 con 0x7f04fc156370 2023-10-06T05:14:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f0501b9c700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 0x7f04fc14e0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04f400b610 tx=0x7f04f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4012de0 con 0x7f04fc156370 2023-10-06T05:14:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f4027470 con 0x7f04fc156370 2023-10-06T05:14:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc157680 con 0x7f04fc156370 2023-10-06T05:14:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.777+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4027830 con 0x7f04fc156370 2023-10-06T05:14:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.778+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc157c40 con 0x7f04fc156370 2023-10-06T05:14:20.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.778+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc04c380 con 0x7f04fc156370 2023-10-06T05:14:20.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.779+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04f4024070 con 0x7f04fc156370 2023-10-06T05:14:20.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.780+0000 7f04f17fa700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:20.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.780+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f04f40869e0 con 0x7f04fc156370 2023-10-06T05:14:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.781+0000 7f050139b700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.781+0000 7f050139b700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04e0066f30 0x7f04e00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04f80058d0 tx=0x7f04f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.785+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04f4053da0 con 0x7f04fc156370 2023-10-06T05:14:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.987+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f04fc0b13e0 con 0x7f04fc156370 2023-10-06T05:14:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.988+0000 7f04f17fa700 1 -- 172.21.15.33:0/3227047749 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f04f40573c0 con 0x7f04fc156370 2023-10-06T05:14:20.990 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:14:20.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04e0066f30 msgr2=0x7f04e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04e0066f30 0x7f04e00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04f80058d0 tx=0x7f04f8009040 comp rx=0 tx=0).stop 2023-10-06T05:14:20.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 msgr2=0x7f04fc14e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 0x7f04fc14e0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04f400b610 tx=0x7f04f4019000 comp rx=0 tx=0).stop 2023-10-06T05:14:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 shutdown_connections 2023-10-06T05:14:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04fc156370 0x7f04fc14e0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04fc0b01f0 0x7f04fc16d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 --2- 172.21.15.33:0/3227047749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04fc0cc7e0 0x7f04fc14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:20.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 >> 172.21.15.33:0/3227047749 conn(0x7f04fc06d030 msgr2=0x7f04fc14f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:20.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 shutdown_connections 2023-10-06T05:14:20.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:20.990+0000 7f050239d700 1 -- 172.21.15.33:0/3227047749 wait complete. 2023-10-06T05:14:21.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:14:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.194+0000 7fc28b59e700 1 Processor -- start 2023-10-06T05:14:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.194+0000 7fc28b59e700 1 -- start start 2023-10-06T05:14:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.195+0000 7fc28b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc28c0ccfc0 0x7fc28c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.195+0000 7fc28b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc28c0cf3b0 0x7fc28c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.195+0000 7fc28b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c0cbd80 con 0x7fc28c0cc750 2023-10-06T05:14:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.195+0000 7fc28b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c0af5f0 con 0x7fc28c0cf3b0 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.195+0000 7fc28b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c0af8d0 con 0x7fc28c0ccfc0 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc28c0cf3b0 0x7fc28c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc28c0cf3b0 0x7fc28c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32898/0 (socket says 172.21.15.33:32898) 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 -- 172.21.15.33:0/3872895209 learned_addr learned my addr 172.21.15.33:0/3872895209 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc28a59c700 1 --2- 172.21.15.33:0/3872895209 >> v2:172.21.15.167:3300/0 conn(0x7fc28c0ccfc0 0x7fc28c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 -- 172.21.15.33:0/3872895209 >> v2:172.21.15.167:3300/0 conn(0x7fc28c0ccfc0 msgr2=0x7fc28c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 --2- 172.21.15.33:0/3872895209 >> v2:172.21.15.167:3300/0 conn(0x7fc28c0ccfc0 0x7fc28c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 -- 172.21.15.33:0/3872895209 >> v1:172.21.15.33:6789/0 conn(0x7fc28c0cc750 legacy=0x7fc28c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 -- 172.21.15.33:0/3872895209 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0afbb0 con 0x7fc28c0cf3b0 2023-10-06T05:14:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc28959a700 1 -- 172.21.15.33:0/3872895209 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348906159 0 0) 0x7fc28c0cbd80 con 0x7fc28c0cc750 2023-10-06T05:14:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.196+0000 7fc289d9b700 1 --2- 172.21.15.33:0/3872895209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0cf3b0 0x7fc28c1594a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2800099b0 tx=0x7fc280009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2063d17275390 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28959a700 1 -- 172.21.15.33:0/3872895209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280009ea0 con 0x7fc28c0cf3b0 2023-10-06T05:14:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28959a700 1 -- 172.21.15.33:0/3872895209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28000b820 con 0x7fc28c0cf3b0 2023-10-06T05:14:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28959a700 1 -- 172.21.15.33:0/3872895209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28000bbc0 con 0x7fc28c0cf3b0 2023-10-06T05:14:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 -- 172.21.15.33:0/3872895209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0cf3b0 msgr2=0x7fc28c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 --2- 172.21.15.33:0/3872895209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0cf3b0 0x7fc28c1594a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2800099b0 tx=0x7fc280009250 comp rx=0 tx=0).stop 2023-10-06T05:14:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 -- 172.21.15.33:0/3872895209 shutdown_connections 2023-10-06T05:14:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 --2- 172.21.15.33:0/3872895209 >> v2:172.21.15.167:3300/0 conn(0x7fc28c0ccfc0 0x7fc28c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 --2- 172.21.15.33:0/3872895209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0cf3b0 0x7fc28c1594a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 -- 172.21.15.33:0/3872895209 >> 172.21.15.33:0/3872895209 conn(0x7fc28c06cff0 msgr2=0x7fc28c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 -- 172.21.15.33:0/3872895209 shutdown_connections 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.197+0000 7fc28b59e700 1 -- 172.21.15.33:0/3872895209 wait complete. 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.199+0000 7fc28b59e700 1 Processor -- start 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.199+0000 7fc28b59e700 1 -- start start 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc28c0cc750 0x7fc28c0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc28c0ccfc0 0x7fc28c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 0x7fc28c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c169ea0 con 0x7fc28c0b5b20 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0b4140 con 0x7fc28c0cc750 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0b4420 con 0x7fc28c0ccfc0 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 0x7fc28c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 0x7fc28c0cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32912/0 (socket says 172.21.15.33:32912) 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 -- 172.21.15.33:0/4106819828 learned_addr learned my addr 172.21.15.33:0/4106819828 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28a59c700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc28c0cc750 0x7fc28c0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc289d9b700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc28c0ccfc0 0x7fc28c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 -- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc28c0ccfc0 msgr2=0x7fc28c0cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc28c0ccfc0 0x7fc28c0cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 -- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc28c0cc750 msgr2=0x7fc28c0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc28c0cc750 0x7fc28c0b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc28ad9d700 1 -- 172.21.15.33:0/4106819828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2800094f0 con 0x7fc28c0b5b20 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.200+0000 7fc289d9b700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc28c0ccfc0 0x7fc28c0cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.201+0000 7fc28ad9d700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 0x7fc28c0cb080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc284001560 tx=0x7fc284001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.201+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28400a780 con 0x7fc28c0b5b20 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.201+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28400a8e0 con 0x7fc28c0b5b20 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.201+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28400aca0 con 0x7fc28c0b5b20 2023-10-06T05:14:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.201+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c0b4700 con 0x7fc28c0b5b20 2023-10-06T05:14:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.201+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c154390 con 0x7fc28c0b5b20 2023-10-06T05:14:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.202+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c16a010 con 0x7fc28c0b5b20 2023-10-06T05:14:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.203+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc284012630 con 0x7fc28c0b5b20 2023-10-06T05:14:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.204+0000 7fc27a7fc700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc26c066f30 0x7fc26c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.204+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc284084730 con 0x7fc28c0b5b20 2023-10-06T05:14:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.204+0000 7fc28a59c700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc26c066f30 0x7fc26c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.205+0000 7fc28a59c700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc26c066f30 0x7fc26c0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009d40 tx=0x7fc27c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.208+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc284051af0 con 0x7fc28c0b5b20 2023-10-06T05:14:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.456+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc28c04c380 con 0x7fc28c0b5b20 2023-10-06T05:14:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.457+0000 7fc27a7fc700 1 -- 172.21.15.33:0/4106819828 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7fc284055110 con 0x7fc28c0b5b20 2023-10-06T05:14:21.459 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:21.459 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc26c066f30 msgr2=0x7fc26c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc26c066f30 0x7fc26c0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009d40 tx=0x7fc27c008040 comp rx=0 tx=0).stop 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 msgr2=0x7fc28c0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 0x7fc28c0cb080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc284001560 tx=0x7fc284001770 comp rx=0 tx=0).stop 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 shutdown_connections 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc26c066f30 0x7fc26c0693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc28c0b5b20 0x7fc28c0cb080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc28c0cc750 0x7fc28c0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 --2- 172.21.15.33:0/4106819828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc28c0ccfc0 0x7fc28c0cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 >> 172.21.15.33:0/4106819828 conn(0x7fc28c06cff0 msgr2=0x7fc28c155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 shutdown_connections 2023-10-06T05:14:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.459+0000 7fc28b59e700 1 -- 172.21.15.33:0/4106819828 wait complete. 2023-10-06T05:14:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2023-10-06T05:14:21.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.662+0000 7fc2f3fff700 1 Processor -- start 2023-10-06T05:14:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.662+0000 7fc2f3fff700 1 -- start start 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2f40c7920 0x7fc2f4160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f406ab60 con 0x7fc2f4155af0 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f406ae40 con 0x7fc2f40c8b50 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f406b7e0 con 0x7fc2f40c7920 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc2f4155af0 0x7fc2f406a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50028/0 (socket says 172.21.15.33:50028) 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f2ffd700 1 -- 172.21.15.33:0/3259716847 learned_addr learned my addr 172.21.15.33:0/3259716847 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f37fe700 1 --2- 172.21.15.33:0/3259716847 >> v2:172.21.15.167:3300/0 conn(0x7fc2f40c7920 0x7fc2f4160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f1ffb700 1 -- 172.21.15.33:0/3259716847 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271207830 0 0) 0x7fc2f406ab60 con 0x7fc2f4155af0 2023-10-06T05:14:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f1ffb700 1 -- 172.21.15.33:0/3259716847 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc003570 con 0x7fc2f4155af0 2023-10-06T05:14:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3259716847 >> v1:172.21.15.167:6789/0 conn(0x7fc2f40c8b50 legacy=0x7fc2f40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3259716847 >> v1:172.21.15.33:6789/0 conn(0x7fc2f4155af0 legacy=0x7fc2f406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.664+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3259716847 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f406b120 con 0x7fc2f40c7920 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.665+0000 7fc2f37fe700 1 --2- 172.21.15.33:0/3259716847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c7920 0x7fc2f4160c30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00af70 tx=0x7fc2ec00a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4cc78164490ae97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.665+0000 7fc2f1ffb700 1 -- 172.21.15.33:0/3259716847 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00f070 con 0x7fc2f40c7920 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.665+0000 7fc2f1ffb700 1 -- 172.21.15.33:0/3259716847 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec00a2e0 con 0x7fc2f40c7920 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.665+0000 7fc2f1ffb700 1 -- 172.21.15.33:0/3259716847 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00cc70 con 0x7fc2f40c7920 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3259716847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c7920 msgr2=0x7fc2f4160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3259716847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c7920 0x7fc2f4160c30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00af70 tx=0x7fc2ec00a890 comp rx=0 tx=0).stop 2023-10-06T05:14:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3259716847 shutdown_connections 2023-10-06T05:14:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3259716847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c7920 0x7fc2f4160c30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3259716847 >> 172.21.15.33:0/3259716847 conn(0x7fc2f406d030 msgr2=0x7fc2f406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3259716847 shutdown_connections 2023-10-06T05:14:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.666+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3259716847 wait complete. 2023-10-06T05:14:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.667+0000 7fc2f3fff700 1 Processor -- start 2023-10-06T05:14:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.667+0000 7fc2f3fff700 1 -- start start 2023-10-06T05:14:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2f40c7920 0x7fc2f40aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c8b50 0x7fc2f4158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 0x7fc2f40c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41596b0 con 0x7fc2f4155af0 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40c40f0 con 0x7fc2f40c8b50 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40c3c60 con 0x7fc2f40c7920 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 0x7fc2f40c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.668+0000 7fc2f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 0x7fc2f40c20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32922/0 (socket says 172.21.15.33:32922) 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3000400285 learned_addr learned my addr 172.21.15.33:0/3000400285 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2f40c7920 msgr2=0x7fc2f40aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2f40c7920 0x7fc2f40aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c8b50 msgr2=0x7fc2f4158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f27fc700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c8b50 0x7fc2f4158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c8b50 0x7fc2f4158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 -- 172.21.15.33:0/3000400285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec00ab30 con 0x7fc2f4155af0 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f27fc700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c8b50 0x7fc2f4158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f37fe700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 0x7fc2f40c20b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec013000 tx=0x7fc2ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009c30 con 0x7fc2f4155af0 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec009d90 con 0x7fc2f4155af0 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01d690 con 0x7fc2f4155af0 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.669+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f40c3330 con 0x7fc2f4155af0 2023-10-06T05:14:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.670+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f40c38f0 con 0x7fc2f4155af0 2023-10-06T05:14:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.670+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f404c380 con 0x7fc2f4155af0 2023-10-06T05:14:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.671+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2ec01da00 con 0x7fc2f4155af0 2023-10-06T05:14:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.673+0000 7fc2daffd700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2e0066f30 0x7fc2e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.673+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc2ec085e20 con 0x7fc2f4155af0 2023-10-06T05:14:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.673+0000 7fc2f2ffd700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2e0066f30 0x7fc2e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.674+0000 7fc2f2ffd700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2e0066f30 0x7fc2e00693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40058d0 tx=0x7fc2e40079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.676+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2ec0531e0 con 0x7fc2f4155af0 2023-10-06T05:14:21.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.883+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2f40c3da0 con 0x7fc2f4155af0 2023-10-06T05:14:21.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.884+0000 7fc2daffd700 1 -- 172.21.15.33:0/3000400285 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13647 (secure 0 0 0) 0x7fc2ec01dcb0 con 0x7fc2f4155af0 2023-10-06T05:14:21.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:21.885 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:20.666643+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2e0066f30 msgr2=0x7fc2e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2e0066f30 0x7fc2e00693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40058d0 tx=0x7fc2e40079d0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 msgr2=0x7fc2f40c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 0x7fc2f40c20b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec013000 tx=0x7fc2ec019000 comp rx=0 tx=0).stop 2023-10-06T05:14:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 shutdown_connections 2023-10-06T05:14:21.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2e0066f30 0x7fc2e00693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f4155af0 0x7fc2f40c20b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2f40c7920 0x7fc2f40aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 --2- 172.21.15.33:0/3000400285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f40c8b50 0x7fc2f4158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 >> 172.21.15.33:0/3000400285 conn(0x7fc2f406d030 msgr2=0x7fc2f40c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 shutdown_connections 2023-10-06T05:14:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:21.886+0000 7fc2f3fff700 1 -- 172.21.15.33:0/3000400285 wait complete. 2023-10-06T05:14:21.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:14:21.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.093+0000 7f8d2679d700 1 Processor -- start 2023-10-06T05:14:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.093+0000 7f8d2679d700 1 -- start start 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.094+0000 7f8d2679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8d200cd5e0 0x7f8d200af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.094+0000 7f8d2679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d2015a650 0x7f8d2015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.094+0000 7f8d2679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2015d660 con 0x7f8d200af540 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.094+0000 7f8d2679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d200ce890 con 0x7f8d200cd5e0 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.094+0000 7f8d2679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015aab0 con 0x7f8d2015a650 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8d200cd5e0 0x7f8d200af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8d200cd5e0 0x7f8d200af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32934/0 (socket says 172.21.15.33:32934) 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 -- 172.21.15.33:0/1991106913 learned_addr learned my addr 172.21.15.33:0/1991106913 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d2579b700 1 --1- 172.21.15.33:0/1991106913 >> v1:172.21.15.167:6789/0 conn(0x7f8d200af540 0x7f8d200cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49890/0 (socket says 172.21.15.33:49890) 2023-10-06T05:14:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d25f9c700 1 --2- 172.21.15.33:0/1991106913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d2015a650 0x7f8d2015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d17fff700 1 -- 172.21.15.33:0/1991106913 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236432721 0 0) 0x7f8d2015d660 con 0x7f8d200af540 2023-10-06T05:14:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 -- 172.21.15.33:0/1991106913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d2015a650 msgr2=0x7f8d2015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 --2- 172.21.15.33:0/1991106913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d2015a650 0x7f8d2015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 -- 172.21.15.33:0/1991106913 >> v1:172.21.15.167:6789/0 conn(0x7f8d200af540 legacy=0x7f8d200cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 -- 172.21.15.33:0/1991106913 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2015d890 con 0x7f8d200cd5e0 2023-10-06T05:14:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.095+0000 7f8d24f9a700 1 --2- 172.21.15.33:0/1991106913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d200af910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099b0 tx=0x7f8d10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57694ac225ecba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d17fff700 1 -- 172.21.15.33:0/1991106913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10009ea0 con 0x7f8d200cd5e0 2023-10-06T05:14:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d17fff700 1 -- 172.21.15.33:0/1991106913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000b860 con 0x7f8d200cd5e0 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d17fff700 1 -- 172.21.15.33:0/1991106913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000bc00 con 0x7f8d200cd5e0 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 -- 172.21.15.33:0/1991106913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 msgr2=0x7f8d200af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 --2- 172.21.15.33:0/1991106913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d200af910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099b0 tx=0x7f8d10009250 comp rx=0 tx=0).stop 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 -- 172.21.15.33:0/1991106913 shutdown_connections 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 --2- 172.21.15.33:0/1991106913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d2015a650 0x7f8d2015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 --2- 172.21.15.33:0/1991106913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d200af910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 -- 172.21.15.33:0/1991106913 >> 172.21.15.33:0/1991106913 conn(0x7f8d2006d9b0 msgr2=0x7f8d2006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.096+0000 7f8d2679d700 1 -- 172.21.15.33:0/1991106913 shutdown_connections 2023-10-06T05:14:22.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.097+0000 7f8d2679d700 1 -- 172.21.15.33:0/1991106913 wait complete. 2023-10-06T05:14:22.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.098+0000 7f8d2679d700 1 Processor -- start 2023-10-06T05:14:22.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.098+0000 7f8d2679d700 1 -- start start 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d200af540 0x7f8d200b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d201579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2015a650 0x7f8d200b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200aec60 con 0x7f8d200cd5e0 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200b4580 con 0x7f8d2015a650 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200b3970 con 0x7f8d200af540 2023-10-06T05:14:22.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d24f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d201579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d24f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d201579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32950/0 (socket says 172.21.15.33:32950) 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d24f9a700 1 -- 172.21.15.33:0/2551303265 learned_addr learned my addr 172.21.15.33:0/2551303265 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d2579b700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d200af540 0x7f8d200b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.099+0000 7f8d25f9c700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2015a650 0x7f8d200b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d24f9a700 1 -- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d200af540 msgr2=0x7f8d200b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d24f9a700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d200af540 0x7f8d200b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d24f9a700 1 -- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2015a650 msgr2=0x7f8d200b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d24f9a700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2015a650 0x7f8d200b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d24f9a700 1 -- 172.21.15.33:0/2551303265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100094f0 con 0x7f8d200cd5e0 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d24f9a700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d201579f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d200caac0 tx=0x7f8d10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10000d10 con 0x7f8d200cd5e0 2023-10-06T05:14:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000bbe0 con 0x7f8d200cd5e0 2023-10-06T05:14:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10011690 con 0x7f8d200cd5e0 2023-10-06T05:14:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.100+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200b3c50 con 0x7f8d200cd5e0 2023-10-06T05:14:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.101+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d200b4210 con 0x7f8d200cd5e0 2023-10-06T05:14:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.101+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d200b4730 con 0x7f8d200cd5e0 2023-10-06T05:14:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.102+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d10011ac0 con 0x7f8d200cd5e0 2023-10-06T05:14:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.103+0000 7f8d157fa700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8d04066f30 0x7f8d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.103+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8d10085ca0 con 0x7f8d200cd5e0 2023-10-06T05:14:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.103+0000 7f8d2579b700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8d04066f30 0x7f8d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.104+0000 7f8d2579b700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8d04066f30 0x7f8d040693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007180 tx=0x7f8d1c007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.107+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d200b4730 con 0x7f8d200cd5e0 2023-10-06T05:14:22.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.308+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d2015d380 con 0x7f8d200cd5e0 2023-10-06T05:14:22.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.309+0000 7f8d157fa700 1 -- 172.21.15.33:0/2551303265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d100150a0 con 0x7f8d200cd5e0 2023-10-06T05:14:22.310 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8d04066f30 msgr2=0x7f8d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8d04066f30 0x7f8d040693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007180 tx=0x7f8d1c007ef0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 msgr2=0x7f8d201579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d201579f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d200caac0 tx=0x7f8d10017000 comp rx=0 tx=0).stop 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 shutdown_connections 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8d04066f30 0x7f8d040693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d2015a650 0x7f8d200b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d200af540 0x7f8d200b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 --2- 172.21.15.33:0/2551303265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d200cd5e0 0x7f8d201579f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 >> 172.21.15.33:0/2551303265 conn(0x7f8d2006d9b0 msgr2=0x7f8d200d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:22.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 shutdown_connections 2023-10-06T05:14:22.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.311+0000 7f8d2679d700 1 -- 172.21.15.33:0/2551303265 wait complete. 2023-10-06T05:14:22.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.517+0000 7fbbdffff700 1 Processor -- start 2023-10-06T05:14:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.518+0000 7fbbdffff700 1 -- start start 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbbe015a650 0x7fbbe015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe015d660 con 0x7fbbe00cced0 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe00ce890 con 0x7fbbe015a650 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe015aac0 con 0x7fbbe00af540 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdf7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbbe00cced0 0x7fbbe006c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50030/0 (socket says 172.21.15.33:50030) 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdf7fe700 1 -- 172.21.15.33:0/4051200171 learned_addr learned my addr 172.21.15.33:0/4051200171 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbde7fc700 1 --2- 172.21.15.33:0/4051200171 >> v2:172.21.15.167:3300/0 conn(0x7fbbe015a650 0x7fbbe015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.519+0000 7fbbdeffd700 1 --2- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbddffb700 1 -- 172.21.15.33:0/4051200171 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927307190 0 0) 0x7fbbe015d660 con 0x7fbbe00cced0 2023-10-06T05:14:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbddffb700 1 -- 172.21.15.33:0/4051200171 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc8003570 con 0x7fbbe00cced0 2023-10-06T05:14:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbde7fc700 1 -- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 msgr2=0x7fbbe00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbde7fc700 1 --2- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe00af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbde7fc700 1 -- 172.21.15.33:0/4051200171 >> v1:172.21.15.33:6789/0 conn(0x7fbbe00cced0 legacy=0x7fbbe006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbde7fc700 1 -- 172.21.15.33:0/4051200171 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe015d890 con 0x7fbbe015a650 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbde7fc700 1 --2- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe015d120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40098e0 tx=0x7fbbd40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7905b624e7dd5a14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbddffb700 1 -- 172.21.15.33:0/4051200171 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4011630 con 0x7fbbe015a650 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbddffb700 1 -- 172.21.15.33:0/4051200171 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4011790 con 0x7fbbe015a650 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.520+0000 7fbbddffb700 1 -- 172.21.15.33:0/4051200171 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4011b10 con 0x7fbbe015a650 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 -- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 msgr2=0x7fbbe015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 --2- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe015d120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40098e0 tx=0x7fbbd40091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 -- 172.21.15.33:0/4051200171 shutdown_connections 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 --2- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 --2- 172.21.15.33:0/4051200171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe015d120 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 -- 172.21.15.33:0/4051200171 >> 172.21.15.33:0/4051200171 conn(0x7fbbe006d9b0 msgr2=0x7fbbe006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 -- 172.21.15.33:0/4051200171 shutdown_connections 2023-10-06T05:14:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.521+0000 7fbbdffff700 1 -- 172.21.15.33:0/4051200171 wait complete. 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.522+0000 7fbbdffff700 1 Processor -- start 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.523+0000 7fbbdffff700 1 -- start start 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.523+0000 7fbbdffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe0151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.523+0000 7fbbdffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 0x7fbbe00b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe0156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0151640 con 0x7fbbe00cced0 2023-10-06T05:14:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00b2b00 con 0x7fbbe015a650 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00b1ef0 con 0x7fbbe00af540 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 0x7fbbe00b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 0x7fbbe00b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32952/0 (socket says 172.21.15.33:32952) 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 -- 172.21.15.33:0/2996477470 learned_addr learned my addr 172.21.15.33:0/2996477470 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdeffd700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe0151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 -- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 msgr2=0x7fbbe0151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe0151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 -- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 msgr2=0x7fbbe0156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdf7fe700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe0156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe0156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 -- 172.21.15.33:0/2996477470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4009420 con 0x7fbbe00cced0 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdeffd700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe0151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbdf7fe700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe0156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.524+0000 7fbbde7fc700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 0x7fbbe00b4170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00c7320 tx=0x7fbbd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.525+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4011b10 con 0x7fbbe00cced0 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.525+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd401a430 con 0x7fbbe00cced0 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.525+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd401a7f0 con 0x7fbbe00cced0 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.525+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe00b21d0 con 0x7fbbe00cced0 2023-10-06T05:14:22.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.525+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00b2790 con 0x7fbbe00cced0 2023-10-06T05:14:22.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.526+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe00b2c90 con 0x7fbbe00cced0 2023-10-06T05:14:22.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.527+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbbd401ab60 con 0x7fbbe00cced0 2023-10-06T05:14:22.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.529+0000 7fbbc6ffd700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbcc066f30 0x7fbbcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.529+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbbd40860e0 con 0x7fbbe00cced0 2023-10-06T05:14:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.529+0000 7fbbdeffd700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbcc066f30 0x7fbbcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.530+0000 7fbbdeffd700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbcc066f30 0x7fbbcc0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0006f00 tx=0x7fbbd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.533+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbd40534a0 con 0x7fbbe00cced0 2023-10-06T05:14:22.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.739+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbbe004c380 con 0x7fbbe00cced0 2023-10-06T05:14:22.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.739+0000 7fbbc6ffd700 1 -- 172.21.15.33:0/2996477470 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbbd4056ac0 con 0x7fbbe00cced0 2023-10-06T05:14:22.741 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:14:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbcc066f30 msgr2=0x7fbbcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbcc066f30 0x7fbbcc0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0006f00 tx=0x7fbbd0008040 comp rx=0 tx=0).stop 2023-10-06T05:14:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 msgr2=0x7fbbe00b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 0x7fbbe00b4170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00c7320 tx=0x7fbbd4017000 comp rx=0 tx=0).stop 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 shutdown_connections 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbcc066f30 0x7fbbcc0693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe015a650 0x7fbbe0156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbe00af540 0x7fbbe0151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 --2- 172.21.15.33:0/2996477470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbe00cced0 0x7fbbe00b4170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.741+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 >> 172.21.15.33:0/2996477470 conn(0x7fbbe006d9b0 msgr2=0x7fbbe015b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.742+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 shutdown_connections 2023-10-06T05:14:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.742+0000 7fbbdffff700 1 -- 172.21.15.33:0/2996477470 wait complete. 2023-10-06T05:14:22.760 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:14:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.946+0000 7f8f4df9d700 1 Processor -- start 2023-10-06T05:14:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.946+0000 7f8f4df9d700 1 -- start start 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.947+0000 7f8f4df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 0x7f8f4806da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.947+0000 7f8f4df9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8f4806df70 0x7f8f480afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.947+0000 7f8f4df9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4806e3e0 con 0x7f8f480b0310 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.947+0000 7f8f4df9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480b1e00 con 0x7f8f4806df70 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.947+0000 7f8f4df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4806f6b0 con 0x7f8f48159530 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.947+0000 7f8f4cf9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 0x7f8f4806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4d79c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f8f480b0310 0x7f8f48159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49894/0 (socket says 172.21.15.33:49894) 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4cf9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 0x7f8f4806da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:51908/0 (socket says 172.21.15.33:51908) 2023-10-06T05:14:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4d79c700 1 -- 172.21.15.33:0/4096612678 learned_addr learned my addr 172.21.15.33:0/4096612678 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f47fff700 1 --2- 172.21.15.33:0/4096612678 >> v2:172.21.15.167:3300/0 conn(0x7f8f4806df70 0x7f8f480afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f477fe700 1 -- 172.21.15.33:0/4096612678 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259050456 0 0) 0x7f8f4806e3e0 con 0x7f8f480b0310 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f477fe700 1 -- 172.21.15.33:0/4096612678 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f38003570 con 0x7f8f480b0310 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4cf9b700 1 -- 172.21.15.33:0/4096612678 >> v2:172.21.15.167:3300/0 conn(0x7f8f4806df70 msgr2=0x7f8f480afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4cf9b700 1 --2- 172.21.15.33:0/4096612678 >> v2:172.21.15.167:3300/0 conn(0x7f8f4806df70 0x7f8f480afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4cf9b700 1 -- 172.21.15.33:0/4096612678 >> v1:172.21.15.167:6789/0 conn(0x7f8f480b0310 legacy=0x7f8f48159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4cf9b700 1 -- 172.21.15.33:0/4096612678 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4806f220 con 0x7f8f48159530 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f47fff700 1 --2- 172.21.15.33:0/4096612678 >> v2:172.21.15.167:3300/0 conn(0x7f8f4806df70 0x7f8f480afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.948+0000 7f8f4cf9b700 1 --2- 172.21.15.33:0/4096612678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 0x7f8f4806da30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0098e0 tx=0x7f8f3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30655cfedbe413a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f477fe700 1 -- 172.21.15.33:0/4096612678 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c011630 con 0x7f8f48159530 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f477fe700 1 -- 172.21.15.33:0/4096612678 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c011790 con 0x7f8f48159530 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f477fe700 1 -- 172.21.15.33:0/4096612678 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c011b30 con 0x7f8f48159530 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 -- 172.21.15.33:0/4096612678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 msgr2=0x7f8f4806da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/4096612678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 0x7f8f4806da30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0098e0 tx=0x7f8f3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 -- 172.21.15.33:0/4096612678 shutdown_connections 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/4096612678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f48159530 0x7f8f4806da30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/4096612678 >> v2:172.21.15.167:3300/0 conn(0x7f8f4806df70 0x7f8f480afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 -- 172.21.15.33:0/4096612678 >> 172.21.15.33:0/4096612678 conn(0x7f8f4806c560 msgr2=0x7f8f4806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 -- 172.21.15.33:0/4096612678 shutdown_connections 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.949+0000 7f8f4df9d700 1 -- 172.21.15.33:0/4096612678 wait complete. 2023-10-06T05:14:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.951+0000 7f8f4df9d700 1 Processor -- start 2023-10-06T05:14:22.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.951+0000 7f8f4df9d700 1 -- start start 2023-10-06T05:14:22.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f4806df70 0x7f8f480ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4df9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f480b0310 0x7f8f48158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4df9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 0x7f8f48158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4df9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4816d800 con 0x7f8f48159530 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4df9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480cb370 con 0x7f8f480b0310 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480cb650 con 0x7f8f4806df70 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4d79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 0x7f8f48158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4d79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 0x7f8f48158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32958/0 (socket says 172.21.15.33:32958) 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4d79c700 1 -- 172.21.15.33:0/2484186252 learned_addr learned my addr 172.21.15.33:0/2484186252 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f4cf9b700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f4806df70 0x7f8f480ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.952+0000 7f8f47fff700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f480b0310 0x7f8f48158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4d79c700 1 -- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f4806df70 msgr2=0x7f8f480ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4d79c700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f4806df70 0x7f8f480ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4d79c700 1 -- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f480b0310 msgr2=0x7f8f48158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4d79c700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f480b0310 0x7f8f48158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4d79c700 1 -- 172.21.15.33:0/2484186252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009420 con 0x7f8f48159530 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4d79c700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 0x7f8f48158650 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f40001460 tx=0x7f8f40001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000a890 con 0x7f8f48159530 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000a9f0 con 0x7f8f48159530 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c009420 con 0x7f8f48159530 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000ad70 con 0x7f8f48159530 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.953+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f480ce620 con 0x7f8f48159530 2023-10-06T05:14:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.954+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f480cbb70 con 0x7f8f48159530 2023-10-06T05:14:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.955+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f40007300 con 0x7f8f48159530 2023-10-06T05:14:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.956+0000 7f8f44ff9700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8f2c066fb0 0x7f8f2c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.956+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8f400847b0 con 0x7f8f48159530 2023-10-06T05:14:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.956+0000 7f8f4cf9b700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8f2c066fb0 0x7f8f2c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:22.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.957+0000 7f8f4cf9b700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8f2c066fb0 0x7f8f2c069460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0091e0 tx=0x7f8f3c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:22.960+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f480cbb70 con 0x7f8f48159530 2023-10-06T05:14:23.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.167+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8f480cbb70 con 0x7f8f48159530 2023-10-06T05:14:23.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.168+0000 7f8f44ff9700 1 -- 172.21.15.33:0/2484186252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8f480cbb70 con 0x7f8f48159530 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:14:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:14:23.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:14:23.171 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8f2c066fb0 msgr2=0x7f8f2c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8f2c066fb0 0x7f8f2c069460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0091e0 tx=0x7f8f3c017000 comp rx=0 tx=0).stop 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 msgr2=0x7f8f48158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 0x7f8f48158650 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f40001460 tx=0x7f8f40001ae0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 shutdown_connections 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8f2c066fb0 0x7f8f2c069460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48159530 0x7f8f48158650 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f4806df70 0x7f8f480ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 --2- 172.21.15.33:0/2484186252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f480b0310 0x7f8f48158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 >> 172.21.15.33:0/2484186252 conn(0x7f8f4806c560 msgr2=0x7f8f480c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 shutdown_connections 2023-10-06T05:14:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.170+0000 7f8f4df9d700 1 -- 172.21.15.33:0/2484186252 wait complete. 2023-10-06T05:14:23.189 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:14:23.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:14:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.377+0000 7fb4c939d700 1 Processor -- start 2023-10-06T05:14:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.377+0000 7fb4c939d700 1 -- start start 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c4159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4c406ee20 0x7fb4c40ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40ad3a0 con 0x7fb4c40ad830 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c40ac580 con 0x7fb4c406ee20 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40afcc0 con 0x7fb4c4157320 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4c406ee20 0x7fb4c40ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4c406ee20 0x7fb4c40ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32972/0 (socket says 172.21.15.33:32972) 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 -- 172.21.15.33:0/3937686415 learned_addr learned my addr 172.21.15.33:0/3937686415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb4c40ad830 0x7fb4c406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50034/0 (socket says 172.21.15.33:50034) 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c3fff700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c4159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 -- 172.21.15.33:0/3937686415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 msgr2=0x7fb4c4159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c4159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 -- 172.21.15.33:0/3937686415 >> v1:172.21.15.33:6789/0 conn(0x7fb4c40ad830 legacy=0x7fb4c406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c8b9c700 1 -- 172.21.15.33:0/3937686415 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c41576f0 con 0x7fb4c406ee20 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.379+0000 7fb4c3fff700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c4159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:23.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.380+0000 7fb4c8b9c700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c406ee20 0x7fb4c40ac040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009f20 tx=0x7fb4bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e4ca3b31a14e06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.380+0000 7fb4c2ffd700 1 -- 172.21.15.33:0/3937686415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00e070 con 0x7fb4c406ee20 2023-10-06T05:14:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.380+0000 7fb4c2ffd700 1 -- 172.21.15.33:0/3937686415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc00ba80 con 0x7fb4c406ee20 2023-10-06T05:14:23.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.380+0000 7fb4c2ffd700 1 -- 172.21.15.33:0/3937686415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0092e0 con 0x7fb4c406ee20 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 -- 172.21.15.33:0/3937686415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c406ee20 msgr2=0x7fb4c40ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c406ee20 0x7fb4c40ac040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009f20 tx=0x7fb4bc009840 comp rx=0 tx=0).stop 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 -- 172.21.15.33:0/3937686415 shutdown_connections 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c406ee20 0x7fb4c40ac040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 --2- 172.21.15.33:0/3937686415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c4159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 -- 172.21.15.33:0/3937686415 >> 172.21.15.33:0/3937686415 conn(0x7fb4c406cd40 msgr2=0x7fb4c40ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 -- 172.21.15.33:0/3937686415 shutdown_connections 2023-10-06T05:14:23.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.381+0000 7fb4c939d700 1 -- 172.21.15.33:0/3937686415 wait complete. 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.382+0000 7fb4c939d700 1 Processor -- start 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 -- start start 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c406ee20 0x7fb4c40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 0x7fb4c416c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c416cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b2df0 con 0x7fb4c40ad830 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b30d0 con 0x7fb4c406ee20 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.383+0000 7fb4c939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b33b0 con 0x7fb4c4157320 2023-10-06T05:14:23.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 0x7fb4c416c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 0x7fb4c416c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32974/0 (socket says 172.21.15.33:32974) 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 -- 172.21.15.33:0/4289430610 learned_addr learned my addr 172.21.15.33:0/4289430610 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c3fff700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c406ee20 0x7fb4c40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c8b9c700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c416cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 -- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 msgr2=0x7fb4c416cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c416cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 -- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c406ee20 msgr2=0x7fb4c40b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c406ee20 0x7fb4c40b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 -- 172.21.15.33:0/4289430610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009ae0 con 0x7fb4c40ad830 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.384+0000 7fb4c37fe700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 0x7fb4c416c820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c790 tx=0x7fb4b8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.385+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8005a70 con 0x7fb4c40ad830 2023-10-06T05:14:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.385+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b8005bf0 con 0x7fb4c40ad830 2023-10-06T05:14:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.385+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80070d0 con 0x7fb4c40ad830 2023-10-06T05:14:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.385+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c40b3690 con 0x7fb4c40ad830 2023-10-06T05:14:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.385+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c40b2a50 con 0x7fb4c40ad830 2023-10-06T05:14:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.386+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c40b3bb0 con 0x7fb4c40ad830 2023-10-06T05:14:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.387+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4b8007480 con 0x7fb4c40ad830 2023-10-06T05:14:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.388+0000 7fb4abfff700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.388+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb4b80838d0 con 0x7fb4c40ad830 2023-10-06T05:14:23.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.389+0000 7fb4c3fff700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.389+0000 7fb4c3fff700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006f00 tx=0x7fb4b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.393+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c40b3bb0 con 0x7fb4c40ad830 2023-10-06T05:14:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.594+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb4c40b3bb0 con 0x7fb4c40ad830 2023-10-06T05:14:23.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.694+0000 7fb4abfff700 1 -- 172.21.15.33:0/4289430610 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb4c40b3bb0 con 0x7fb4c40ad830 2023-10-06T05:14:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.699+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 msgr2=0x7fb4b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.699+0000 7fb4c939d700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006f00 tx=0x7fb4b4008040 comp rx=0 tx=0).stop 2023-10-06T05:14:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.699+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 msgr2=0x7fb4c416c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.699+0000 7fb4c939d700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 0x7fb4c416c820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c790 tx=0x7fb4b8007f40 comp rx=0 tx=0).stop 2023-10-06T05:14:23.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.700+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 shutdown_connections 2023-10-06T05:14:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.700+0000 7fb4c939d700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.700+0000 7fb4c939d700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c4157320 0x7fb4c416cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.700+0000 7fb4c939d700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4c406ee20 0x7fb4c40b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.700+0000 7fb4c939d700 1 --2- 172.21.15.33:0/4289430610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c40ad830 0x7fb4c416c820 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.700+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 >> 172.21.15.33:0/4289430610 conn(0x7fb4c406cd40 msgr2=0x7fb4c40683f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.701+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 shutdown_connections 2023-10-06T05:14:23.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.701+0000 7fb4c939d700 1 -- 172.21.15.33:0/4289430610 wait complete. 2023-10-06T05:14:23.705 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:14:23.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:14:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.914+0000 7f4e149fe700 1 Processor -- start 2023-10-06T05:14:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.914+0000 7f4e149fe700 1 -- start start 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.915+0000 7f4e149fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.915+0000 7f4e149fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e10158f20 0x7f4e10159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.915+0000 7f4e149fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e100ce480 con 0x7f4e100cd4e0 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.915+0000 7f4e149fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e100cf920 con 0x7f4e10158f20 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.915+0000 7f4e149fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e100cb0e0 con 0x7f4e100cd110 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.915+0000 7f4e0f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e10158f20 0x7f4e10159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4e100cd4e0 0x7f4e1015da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50050/0 (socket says 172.21.15.33:50050) 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0e59c700 1 -- 172.21.15.33:0/3301334869 learned_addr learned my addr 172.21.15.33:0/3301334869 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e10158f20 0x7f4e10159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32990/0 (socket says 172.21.15.33:32990) 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0ed9d700 1 --2- 172.21.15.33:0/3301334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0f59e700 1 -- 172.21.15.33:0/3301334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 msgr2=0x7f4e100cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0f59e700 1 --2- 172.21.15.33:0/3301334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0f59e700 1 -- 172.21.15.33:0/3301334869 >> v1:172.21.15.33:6789/0 conn(0x7f4e100cd4e0 legacy=0x7f4e1015da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0f59e700 1 -- 172.21.15.33:0/3301334869 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e100cb3c0 con 0x7f4e10158f20 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0dd9b700 1 -- 172.21.15.33:0/3301334869 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692431351 0 0) 0x7f4e100ce480 con 0x7f4e100cd4e0 2023-10-06T05:14:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.916+0000 7f4e0f59e700 1 --2- 172.21.15.33:0/3301334869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e10159350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009e60 tx=0x7f4e08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5feaf73ef858d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e0dd9b700 1 -- 172.21.15.33:0/3301334869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800e070 con 0x7f4e10158f20 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e0dd9b700 1 -- 172.21.15.33:0/3301334869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0800bc30 con 0x7f4e10158f20 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e0dd9b700 1 -- 172.21.15.33:0/3301334869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08012430 con 0x7f4e10158f20 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 -- 172.21.15.33:0/3301334869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 msgr2=0x7f4e10159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3301334869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e10159350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009e60 tx=0x7f4e08009780 comp rx=0 tx=0).stop 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 -- 172.21.15.33:0/3301334869 shutdown_connections 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3301334869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e10159350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3301334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 -- 172.21.15.33:0/3301334869 >> 172.21.15.33:0/3301334869 conn(0x7f4e1006d9b0 msgr2=0x7f4e1006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 -- 172.21.15.33:0/3301334869 shutdown_connections 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.917+0000 7f4e149fe700 1 -- 172.21.15.33:0/3301334869 wait complete. 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.919+0000 7f4e149fe700 1 Processor -- start 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.919+0000 7f4e149fe700 1 -- start start 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e149fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e149fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e100cd4e0 0x7f4e100b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e149fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e100b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e149fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e100b56c0 con 0x7f4e10158f20 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e149fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1016a680 con 0x7f4e100cd4e0 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e149fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e10169a70 con 0x7f4e100cd110 2023-10-06T05:14:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e100b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:51944/0 (socket says 172.21.15.33:51944) 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0ed9d700 1 -- 172.21.15.33:0/3931636425 learned_addr learned my addr 172.21.15.33:0/3931636425 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0e59c700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e100cd4e0 0x7f4e100b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0f59e700 1 -- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 msgr2=0x7f4e100b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0f59e700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0f59e700 1 -- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e100cd4e0 msgr2=0x7f4e100b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0f59e700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e100cd4e0 0x7f4e100b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.920+0000 7f4e0f59e700 1 -- 172.21.15.33:0/3931636425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e08009a20 con 0x7f4e10158f20 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.921+0000 7f4e0f59e700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e100b15f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e08016000 tx=0x7f4e08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.921+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08027470 con 0x7f4e10158f20 2023-10-06T05:14:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.921+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08027630 con 0x7f4e10158f20 2023-10-06T05:14:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.921+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080279f0 con 0x7f4e10158f20 2023-10-06T05:14:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.921+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e10169d50 con 0x7f4e10158f20 2023-10-06T05:14:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.921+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1016a310 con 0x7f4e10158f20 2023-10-06T05:14:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.922+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1004c380 con 0x7f4e10158f20 2023-10-06T05:14:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.923+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e08024070 con 0x7f4e10158f20 2023-10-06T05:14:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.924+0000 7f4df6ffd700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4dfc066f30 0x7f4dfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.924+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4e08086c20 con 0x7f4e10158f20 2023-10-06T05:14:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.925+0000 7f4e0ed9d700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4dfc066f30 0x7f4dfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.925+0000 7f4e0ed9d700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4dfc066f30 0x7f4dfc0693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e00006fd0 tx=0x7f4e00008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:23.929+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e08053e70 con 0x7f4e10158f20 2023-10-06T05:14:24.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.131+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4e100cb6a0 con 0x7f4e10158f20 2023-10-06T05:14:24.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.702+0000 7f4df6ffd700 1 -- 172.21.15.33:0/3931636425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4e0802c0d0 con 0x7f4e10158f20 2023-10-06T05:14:24.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4dfc066f30 msgr2=0x7f4dfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4dfc066f30 0x7f4dfc0693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e00006fd0 tx=0x7f4e00008040 comp rx=0 tx=0).stop 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 msgr2=0x7f4e100b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e100b15f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e08016000 tx=0x7f4e08019000 comp rx=0 tx=0).stop 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 shutdown_connections 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4dfc066f30 0x7f4dfc0693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.719+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e10158f20 0x7f4e100b15f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.720+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e100cd110 0x7f4e100b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.720+0000 7f4e149fe700 1 --2- 172.21.15.33:0/3931636425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e100cd4e0 0x7f4e100b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.720+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 >> 172.21.15.33:0/3931636425 conn(0x7f4e1006d9b0 msgr2=0x7f4e10157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:24.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.721+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 shutdown_connections 2023-10-06T05:14:24.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.721+0000 7f4e149fe700 1 -- 172.21.15.33:0/3931636425 wait complete. 2023-10-06T05:14:24.727 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:14:24.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:14:24.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.937+0000 7fd73e59d700 1 Processor -- start 2023-10-06T05:14:24.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.937+0000 7fd73e59d700 1 -- start start 2023-10-06T05:14:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.938+0000 7fd73e59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7380c9490 0x7fd738155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.938+0000 7fd73e59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7380b06b0 con 0x7fd7380afb20 2023-10-06T05:14:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.938+0000 7fd73e59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7380c99e0 con 0x7fd738155a50 2023-10-06T05:14:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.938+0000 7fd73e59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380c9cc0 con 0x7fd7380c9490 2023-10-06T05:14:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd73dd9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd7380afb20 0x7fd7380caa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50058/0 (socket says 172.21.15.33:50058) 2023-10-06T05:14:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd73dd9c700 1 -- 172.21.15.33:0/142406683 learned_addr learned my addr 172.21.15.33:0/142406683 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd73cd9a700 1 --2- 172.21.15.33:0/142406683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7380c9490 0x7fd738155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371951254 0 0) 0x7fd7380b06b0 con 0x7fd7380afb20 2023-10-06T05:14:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd724004680 con 0x7fd7380afb20 2023-10-06T05:14:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286641044 0 0) 0x7fd7380c99e0 con 0x7fd738155a50 2023-10-06T05:14:24.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7380b06b0 con 0x7fd738155a50 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd73cd9a700 1 -- 172.21.15.33:0/142406683 >> v1:172.21.15.167:6789/0 conn(0x7fd738155a50 legacy=0x7fd7380af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd73cd9a700 1 -- 172.21.15.33:0/142406683 >> v1:172.21.15.33:6789/0 conn(0x7fd7380afb20 legacy=0x7fd7380caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.939+0000 7fd73cd9a700 1 -- 172.21.15.33:0/142406683 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380c9fa0 con 0x7fd7380c9490 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73cd9a700 1 --2- 172.21.15.33:0/142406683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7380c9490 0x7fd738155e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd728009940 tx=0x7fd7280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1271db4b9a9a40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728011630 con 0x7fd7380c9490 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728011790 con 0x7fd7380c9490 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd72ffff700 1 -- 172.21.15.33:0/142406683 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728011b30 con 0x7fd7380c9490 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73e59d700 1 -- 172.21.15.33:0/142406683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7380c9490 msgr2=0x7fd738155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73e59d700 1 --2- 172.21.15.33:0/142406683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7380c9490 0x7fd738155e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd728009940 tx=0x7fd7280091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73e59d700 1 -- 172.21.15.33:0/142406683 shutdown_connections 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73e59d700 1 --2- 172.21.15.33:0/142406683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7380c9490 0x7fd738155e20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73e59d700 1 -- 172.21.15.33:0/142406683 >> 172.21.15.33:0/142406683 conn(0x7fd73806d030 msgr2=0x7fd7380cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:24.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.940+0000 7fd73e59d700 1 -- 172.21.15.33:0/142406683 shutdown_connections 2023-10-06T05:14:24.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.941+0000 7fd73e59d700 1 -- 172.21.15.33:0/142406683 wait complete. 2023-10-06T05:14:24.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.942+0000 7fd73e59d700 1 Processor -- start 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.942+0000 7fd73e59d700 1 -- start start 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73e59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 0x7fd73814e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73e59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7380c9490 0x7fd73806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73e59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd738155a50 0x7fd738149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73e59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73814e600 con 0x7fd7380afb20 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73e59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380c7240 con 0x7fd7380c9490 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73e59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380c6630 con 0x7fd738155a50 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 0x7fd73814e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 0x7fd73814e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33016/0 (socket says 172.21.15.33:33016) 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.943+0000 7fd73d59b700 1 -- 172.21.15.33:0/2783214799 learned_addr learned my addr 172.21.15.33:0/2783214799 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73dd9c700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd738155a50 0x7fd738149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73cd9a700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7380c9490 0x7fd73806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73d59b700 1 -- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd738155a50 msgr2=0x7fd738149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73d59b700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd738155a50 0x7fd738149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73d59b700 1 -- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7380c9490 msgr2=0x7fd73806a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73d59b700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7380c9490 0x7fd73806a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73d59b700 1 -- 172.21.15.33:0/2783214799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728009420 con 0x7fd7380afb20 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73cd9a700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7380c9490 0x7fd73806a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73dd9c700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd738155a50 0x7fd738149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73d59b700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 0x7fd73814e0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c790 tx=0x7fd734007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:24.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734010510 con 0x7fd7380afb20 2023-10-06T05:14:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd734010670 con 0x7fd7380afb20 2023-10-06T05:14:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734010a10 con 0x7fd7380afb20 2023-10-06T05:14:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7380c6910 con 0x7fd7380afb20 2023-10-06T05:14:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.944+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7380c6ed0 con 0x7fd7380afb20 2023-10-06T05:14:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.945+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73804c380 con 0x7fd7380afb20 2023-10-06T05:14:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.946+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd73401b400 con 0x7fd7380afb20 2023-10-06T05:14:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.948+0000 7fd72d7fa700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd71c066f30 0x7fd71c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.948+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fd734084520 con 0x7fd7380afb20 2023-10-06T05:14:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.948+0000 7fd73cd9a700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd71c066f30 0x7fd71c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.949+0000 7fd73cd9a700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd71c066f30 0x7fd71c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd728009860 tx=0x7fd728017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:24.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:24.952+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7340515f0 con 0x7fd7380afb20 2023-10-06T05:14:25.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.201+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd73806b300 con 0x7fd7380afb20 2023-10-06T05:14:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.709+0000 7fd72d7fa700 1 -- 172.21.15.33:0/2783214799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd734054c10 con 0x7fd7380afb20 2023-10-06T05:14:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.726+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd71c066f30 msgr2=0x7fd71c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.726+0000 7fd73e59d700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd71c066f30 0x7fd71c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd728009860 tx=0x7fd728017000 comp rx=0 tx=0).stop 2023-10-06T05:14:25.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.726+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 msgr2=0x7fd73814e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:25.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.726+0000 7fd73e59d700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 0x7fd73814e0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c790 tx=0x7fd734007e10 comp rx=0 tx=0).stop 2023-10-06T05:14:25.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 shutdown_connections 2023-10-06T05:14:25.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd71c066f30 0x7fd71c0693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd738155a50 0x7fd738149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7380afb20 0x7fd73814e0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 --2- 172.21.15.33:0/2783214799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7380c9490 0x7fd73806a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 >> 172.21.15.33:0/2783214799 conn(0x7fd73806d030 msgr2=0x7fd738154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:25.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.730+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 shutdown_connections 2023-10-06T05:14:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.732+0000 7fd73e59d700 1 -- 172.21.15.33:0/2783214799 wait complete. 2023-10-06T05:14:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T05:14:25.760 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.947+0000 7f27e359e700 1 Processor -- start 2023-10-06T05:14:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.947+0000 7f27e359e700 1 -- start start 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.948+0000 7f27e359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e40cf3b0 0x7f27e41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.948+0000 7f27e359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27e40af0a0 0x7f27e41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.948+0000 7f27e359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e40cbd80 con 0x7f27e40ccfc0 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.948+0000 7f27e359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40af5f0 con 0x7f27e40af0a0 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.948+0000 7f27e359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40af8d0 con 0x7f27e40cf3b0 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27e40af0a0 0x7f27e41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27e40af0a0 0x7f27e41599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33020/0 (socket says 172.21.15.33:33020) 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 -- 172.21.15.33:0/4057668770 learned_addr learned my addr 172.21.15.33:0/4057668770 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f27e40ccfc0 0x7f27e40cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50062/0 (socket says 172.21.15.33:50062) 2023-10-06T05:14:25.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e1d9b700 1 --2- 172.21.15.33:0/4057668770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e40cf3b0 0x7f27e41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:25.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 -- 172.21.15.33:0/4057668770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e40cf3b0 msgr2=0x7f27e41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:25.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 --2- 172.21.15.33:0/4057668770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e40cf3b0 0x7f27e41594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 -- 172.21.15.33:0/4057668770 >> v1:172.21.15.33:6789/0 conn(0x7f27e40ccfc0 legacy=0x7f27e40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:25.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 -- 172.21.15.33:0/4057668770 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40afbb0 con 0x7f27e40af0a0 2023-10-06T05:14:25.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e159a700 1 -- 172.21.15.33:0/4057668770 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574114326 0 0) 0x7f27e40cbd80 con 0x7f27e40ccfc0 2023-10-06T05:14:25.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.949+0000 7f27e2d9d700 1 --2- 172.21.15.33:0/4057668770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e41599e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009f20 tx=0x7f27dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94696c58d91606a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e159a700 1 -- 172.21.15.33:0/4057668770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00e070 con 0x7f27e40af0a0 2023-10-06T05:14:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e159a700 1 -- 172.21.15.33:0/4057668770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc00b9d0 con 0x7f27e40af0a0 2023-10-06T05:14:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e159a700 1 -- 172.21.15.33:0/4057668770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0092e0 con 0x7f27e40af0a0 2023-10-06T05:14:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 -- 172.21.15.33:0/4057668770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 msgr2=0x7f27e41599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 --2- 172.21.15.33:0/4057668770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e41599e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009f20 tx=0x7f27dc009840 comp rx=0 tx=0).stop 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 -- 172.21.15.33:0/4057668770 shutdown_connections 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 --2- 172.21.15.33:0/4057668770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e41599e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 --2- 172.21.15.33:0/4057668770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e40cf3b0 0x7f27e41594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 -- 172.21.15.33:0/4057668770 >> 172.21.15.33:0/4057668770 conn(0x7f27e406cff0 msgr2=0x7f27e406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 -- 172.21.15.33:0/4057668770 shutdown_connections 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.950+0000 7f27e359e700 1 -- 172.21.15.33:0/4057668770 wait complete. 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.952+0000 7f27e359e700 1 Processor -- start 2023-10-06T05:14:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.952+0000 7f27e359e700 1 -- start start 2023-10-06T05:14:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e40ccfc0 0x7f27e40c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e4156340 0x7f27e41533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e416d190 con 0x7f27e40af0a0 2023-10-06T05:14:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e416d430 con 0x7f27e40ccfc0 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e416d710 con 0x7f27e4156340 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e4155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33036/0 (socket says 172.21.15.33:33036) 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e259c700 1 -- 172.21.15.33:0/58705589 learned_addr learned my addr 172.21.15.33:0/58705589 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e1d9b700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e40ccfc0 0x7f27e40c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.953+0000 7f27e2d9d700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e4156340 0x7f27e41533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e259c700 1 -- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e4156340 msgr2=0x7f27e41533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e259c700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e4156340 0x7f27e41533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e259c700 1 -- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e40ccfc0 msgr2=0x7f27e40c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e259c700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e40ccfc0 0x7f27e40c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e259c700 1 -- 172.21.15.33:0/58705589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc009ae0 con 0x7f27e40af0a0 2023-10-06T05:14:25.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e259c700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e4155e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c5b0 tx=0x7f27d4001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4005a70 con 0x7f27e40af0a0 2023-10-06T05:14:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d4005bd0 con 0x7f27e40af0a0 2023-10-06T05:14:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e416d9f0 con 0x7f27e40af0a0 2023-10-06T05:14:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40070d0 con 0x7f27e40af0a0 2023-10-06T05:14:25.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.954+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4154300 con 0x7f27e40af0a0 2023-10-06T05:14:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.955+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e416df10 con 0x7f27e40af0a0 2023-10-06T05:14:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.956+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27d4007480 con 0x7f27e40af0a0 2023-10-06T05:14:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.957+0000 7f27d27fc700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27c4066f30 0x7f27c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.957+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f27d4083b80 con 0x7f27e40af0a0 2023-10-06T05:14:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.957+0000 7f27e1d9b700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27c4066f30 0x7f27c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:25.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.958+0000 7f27e1d9b700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27c4066f30 0x7f27c40693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27e41547d0 tx=0x7f27d80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:25.961+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27e416df10 con 0x7f27e40af0a0 2023-10-06T05:14:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.164+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27e416df10 con 0x7f27e40af0a0 2023-10-06T05:14:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.165+0000 7f27d27fc700 1 -- 172.21.15.33:0/58705589 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f27e416df10 con 0x7f27e40af0a0 2023-10-06T05:14:26.168 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:26.168 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:25.203643+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27c4066f30 msgr2=0x7f27c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27c4066f30 0x7f27c40693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27e41547d0 tx=0x7f27d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 msgr2=0x7f27e4155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e4155e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c5b0 tx=0x7f27d4001040 comp rx=0 tx=0).stop 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 shutdown_connections 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27c4066f30 0x7f27c40693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27e4156340 0x7f27e41533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40af0a0 0x7f27e4155e00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 --2- 172.21.15.33:0/58705589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e40ccfc0 0x7f27e40c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 >> 172.21.15.33:0/58705589 conn(0x7f27e406cff0 msgr2=0x7f27e40ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 shutdown_connections 2023-10-06T05:14:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.167+0000 7f27e359e700 1 -- 172.21.15.33:0/58705589 wait complete. 2023-10-06T05:14:26.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.371+0000 7fdeff59e700 1 Processor -- start 2023-10-06T05:14:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.371+0000 7fdeff59e700 1 -- start start 2023-10-06T05:14:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdeff59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 0x7fdf0006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdeff59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdf000c7920 0x7fdf000c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdeff59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0006ab60 con 0x7fdf0006a6a0 2023-10-06T05:14:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdeff59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0006b7e0 con 0x7fdf000c7920 2023-10-06T05:14:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdeff59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0006ae40 con 0x7fdf00155af0 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 0x7fdf0006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 0x7fdf0006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:52000/0 (socket says 172.21.15.33:52000) 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdefe59c700 1 -- 172.21.15.33:0/2102908996 learned_addr learned my addr 172.21.15.33:0/2102908996 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdefed9d700 1 --2- 172.21.15.33:0/2102908996 >> v2:172.21.15.167:3300/0 conn(0x7fdf000c7920 0x7fdf000c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.373+0000 7fdefdd9b700 1 --1- 172.21.15.33:0/2102908996 >> v1:172.21.15.167:6789/0 conn(0x7fdf0006a6a0 0x7fdf000c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49924/0 (socket says 172.21.15.33:49924) 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefe59c700 1 -- 172.21.15.33:0/2102908996 >> v2:172.21.15.167:3300/0 conn(0x7fdf000c7920 msgr2=0x7fdf000c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefe59c700 1 --2- 172.21.15.33:0/2102908996 >> v2:172.21.15.167:3300/0 conn(0x7fdf000c7920 0x7fdf000c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefe59c700 1 -- 172.21.15.33:0/2102908996 >> v1:172.21.15.167:6789/0 conn(0x7fdf0006a6a0 legacy=0x7fdf000c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefe59c700 1 -- 172.21.15.33:0/2102908996 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0006b120 con 0x7fdf00155af0 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefe59c700 1 --2- 172.21.15.33:0/2102908996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 0x7fdf0006a160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdef00099b0 tx=0x7fdef0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfa48c23d97c25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefd59a700 1 -- 172.21.15.33:0/2102908996 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0009ea0 con 0x7fdf00155af0 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefd59a700 1 -- 172.21.15.33:0/2102908996 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000b860 con 0x7fdf00155af0 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.374+0000 7fdefd59a700 1 -- 172.21.15.33:0/2102908996 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000bc00 con 0x7fdf00155af0 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 -- 172.21.15.33:0/2102908996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 msgr2=0x7fdf0006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 --2- 172.21.15.33:0/2102908996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 0x7fdf0006a160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdef00099b0 tx=0x7fdef0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 -- 172.21.15.33:0/2102908996 shutdown_connections 2023-10-06T05:14:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 --2- 172.21.15.33:0/2102908996 >> v2:172.21.15.167:3300/0 conn(0x7fdf000c7920 0x7fdf000c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 --2- 172.21.15.33:0/2102908996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf00155af0 0x7fdf0006a160 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 -- 172.21.15.33:0/2102908996 >> 172.21.15.33:0/2102908996 conn(0x7fdf0006d030 msgr2=0x7fdf0006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 -- 172.21.15.33:0/2102908996 shutdown_connections 2023-10-06T05:14:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.375+0000 7fdeff59e700 1 -- 172.21.15.33:0/2102908996 wait complete. 2023-10-06T05:14:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.376+0000 7fdeff59e700 1 Processor -- start 2023-10-06T05:14:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.376+0000 7fdeff59e700 1 -- start start 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.377+0000 7fdeff59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0006a6a0 0x7fdf000c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.377+0000 7fdeff59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 0x7fdf000b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.377+0000 7fdeff59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf00155af0 0x7fdf000abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.377+0000 7fdeff59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0016db50 con 0x7fdf000c7920 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.377+0000 7fdeff59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0016cf40 con 0x7fdf00155af0 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.377+0000 7fdeff59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0016d220 con 0x7fdf0006a6a0 2023-10-06T05:14:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 0x7fdf000b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 0x7fdf000b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33044/0 (socket says 172.21.15.33:33044) 2023-10-06T05:14:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 -- 172.21.15.33:0/386926278 learned_addr learned my addr 172.21.15.33:0/386926278 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefed9d700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf00155af0 0x7fdf000abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefe59c700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0006a6a0 0x7fdf000c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 -- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0006a6a0 msgr2=0x7fdf000c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0006a6a0 0x7fdf000c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 -- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf00155af0 msgr2=0x7fdf000abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf00155af0 0x7fdf000abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 -- 172.21.15.33:0/386926278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00094f0 con 0x7fdf000c7920 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.378+0000 7fdefdd9b700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 0x7fdf000b16e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c790 tx=0x7fdef4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.379+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4005a70 con 0x7fdf000c7920 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.379+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4005bf0 con 0x7fdf000c7920 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.379+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40070d0 con 0x7fdf000c7920 2023-10-06T05:14:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.379+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0016d500 con 0x7fdf000c7920 2023-10-06T05:14:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.379+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00158ad0 con 0x7fdf000c7920 2023-10-06T05:14:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.380+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0004c380 con 0x7fdf000c7920 2023-10-06T05:14:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.380+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdef4007480 con 0x7fdf000c7920 2023-10-06T05:14:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.382+0000 7fdeee7fc700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee0066f30 0x7fdee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.382+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdef4083b60 con 0x7fdf000c7920 2023-10-06T05:14:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.382+0000 7fdefe59c700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee0066f30 0x7fdee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.383+0000 7fdefe59c700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee0066f30 0x7fdee00693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdef00099b0 tx=0x7fdef0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.385+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdef4050bd0 con 0x7fdf000c7920 2023-10-06T05:14:26.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.588+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf0016dc90 con 0x7fdf000c7920 2023-10-06T05:14:26.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.589+0000 7fdeee7fc700 1 -- 172.21.15.33:0/386926278 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fdef4017020 con 0x7fdf000c7920 2023-10-06T05:14:26.592 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:26.593 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:25.203643+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:26.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.591+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee0066f30 msgr2=0x7fdee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.591+0000 7fdeff59e700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee0066f30 0x7fdee00693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdef00099b0 tx=0x7fdef0017000 comp rx=0 tx=0).stop 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.591+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 msgr2=0x7fdf000b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.591+0000 7fdeff59e700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 0x7fdf000b16e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c790 tx=0x7fdef4007f40 comp rx=0 tx=0).stop 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 shutdown_connections 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee0066f30 0x7fdee00693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf00155af0 0x7fdf000abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf0006a6a0 0x7fdf000c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 --2- 172.21.15.33:0/386926278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf000c7920 0x7fdf000b16e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 >> 172.21.15.33:0/386926278 conn(0x7fdf0006d030 msgr2=0x7fdf000c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:26.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 shutdown_connections 2023-10-06T05:14:26.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.592+0000 7fdeff59e700 1 -- 172.21.15.33:0/386926278 wait complete. 2023-10-06T05:14:26.613 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:14:26.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.803+0000 7f51f959d700 1 Processor -- start 2023-10-06T05:14:26.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.803+0000 7f51f959d700 1 -- start start 2023-10-06T05:14:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f40af540 0x7f51f40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f959d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51f40cd5e0 0x7f51f415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f959d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f415d660 con 0x7f51f415a650 2023-10-06T05:14:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f959d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f40ce890 con 0x7f51f40cd5e0 2023-10-06T05:14:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f415aac0 con 0x7f51f40af540 2023-10-06T05:14:26.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f51f415a650 0x7f51f40cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50072/0 (socket says 172.21.15.33:50072) 2023-10-06T05:14:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f37fe700 1 -- 172.21.15.33:0/4294538489 learned_addr learned my addr 172.21.15.33:0/4294538489 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.804+0000 7f51f8d9c700 1 --2- 172.21.15.33:0/4294538489 >> v2:172.21.15.33:3300/0 conn(0x7f51f40cd5e0 0x7f51f415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f3fff700 1 --2- 172.21.15.33:0/4294538489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f40af540 0x7f51f40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f8d9c700 1 -- 172.21.15.33:0/4294538489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f40af540 msgr2=0x7f51f40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f8d9c700 1 --2- 172.21.15.33:0/4294538489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f40af540 0x7f51f40af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f8d9c700 1 -- 172.21.15.33:0/4294538489 >> v1:172.21.15.33:6789/0 conn(0x7f51f415a650 legacy=0x7f51f40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f8d9c700 1 -- 172.21.15.33:0/4294538489 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f415d890 con 0x7f51f40cd5e0 2023-10-06T05:14:26.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f2ffd700 1 -- 172.21.15.33:0/4294538489 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927582524 0 0) 0x7f51f415d660 con 0x7f51f415a650 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f8d9c700 1 --2- 172.21.15.33:0/4294538489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f415d120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009e60 tx=0x7f51ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c791f9a36319e35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f2ffd700 1 -- 172.21.15.33:0/4294538489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00e070 con 0x7f51f40cd5e0 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f2ffd700 1 -- 172.21.15.33:0/4294538489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00bc30 con 0x7f51f40cd5e0 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.805+0000 7f51f2ffd700 1 -- 172.21.15.33:0/4294538489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec012430 con 0x7f51f40cd5e0 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 -- 172.21.15.33:0/4294538489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 msgr2=0x7f51f415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 --2- 172.21.15.33:0/4294538489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f415d120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009e60 tx=0x7f51ec009780 comp rx=0 tx=0).stop 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 -- 172.21.15.33:0/4294538489 shutdown_connections 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 --2- 172.21.15.33:0/4294538489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f415d120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 --2- 172.21.15.33:0/4294538489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f40af540 0x7f51f40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 -- 172.21.15.33:0/4294538489 >> 172.21.15.33:0/4294538489 conn(0x7f51f406d9b0 msgr2=0x7f51f406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:26.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 -- 172.21.15.33:0/4294538489 shutdown_connections 2023-10-06T05:14:26.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.806+0000 7f51f959d700 1 -- 172.21.15.33:0/4294538489 wait complete. 2023-10-06T05:14:26.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.808+0000 7f51f959d700 1 Processor -- start 2023-10-06T05:14:26.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.808+0000 7f51f959d700 1 -- start start 2023-10-06T05:14:26.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f959d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f40af540 0x7f51f40c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f959d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f41695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f415a650 0x7f51f4169b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f959d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c7dd0 con 0x7f51f40cd5e0 2023-10-06T05:14:26.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f959d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c80b0 con 0x7f51f40af540 2023-10-06T05:14:26.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c8390 con 0x7f51f415a650 2023-10-06T05:14:26.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f41695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f41695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33076/0 (socket says 172.21.15.33:33076) 2023-10-06T05:14:26.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 -- 172.21.15.33:0/3371830658 learned_addr learned my addr 172.21.15.33:0/3371830658 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f3fff700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f40af540 0x7f51f40c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f8d9c700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f415a650 0x7f51f4169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 -- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f415a650 msgr2=0x7f51f4169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f415a650 0x7f51f4169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 -- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f40af540 msgr2=0x7f51f40c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f40af540 0x7f51f40c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.809+0000 7f51f37fe700 1 -- 172.21.15.33:0/3371830658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009a20 con 0x7f51f40cd5e0 2023-10-06T05:14:26.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.810+0000 7f51f37fe700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f41695f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c7e0 tx=0x7f51e8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:26.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.810+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8005b30 con 0x7f51f40cd5e0 2023-10-06T05:14:26.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.810+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e8005c90 con 0x7f51f40cd5e0 2023-10-06T05:14:26.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.810+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8007300 con 0x7f51f40cd5e0 2023-10-06T05:14:26.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.810+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f40c8670 con 0x7f51f40cd5e0 2023-10-06T05:14:26.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.810+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f40c7ac0 con 0x7f51f40cd5e0 2023-10-06T05:14:26.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.811+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f40c8b90 con 0x7f51f40cd5e0 2023-10-06T05:14:26.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.812+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51e801b400 con 0x7f51f40cd5e0 2023-10-06T05:14:26.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.813+0000 7f51dbfff700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51e0066f30 0x7f51e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:26.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.813+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51e8084230 con 0x7f51f40cd5e0 2023-10-06T05:14:26.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.814+0000 7f51f3fff700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51e0066f30 0x7f51e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:26.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.814+0000 7f51f3fff700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51e0066f30 0x7f51e00693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51e4006fd0 tx=0x7f51e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:26.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:26.817+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51f40c8b90 con 0x7f51f40cd5e0 2023-10-06T05:14:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.069+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f51f40c8b90 con 0x7f51f40cd5e0 2023-10-06T05:14:27.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.725+0000 7f51dbfff700 1 -- 172.21.15.33:0/3371830658 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f51f40c8b90 con 0x7f51f40cd5e0 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51e0066f30 msgr2=0x7f51e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51e0066f30 0x7f51e00693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51e4006fd0 tx=0x7f51e4008040 comp rx=0 tx=0).stop 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 msgr2=0x7f51f41695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f41695f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c7e0 tx=0x7f51e8007df0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 shutdown_connections 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51e0066f30 0x7f51e00693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51f415a650 0x7f51f4169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51f40af540 0x7f51f40c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 --2- 172.21.15.33:0/3371830658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f40cd5e0 0x7f51f41695f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 >> 172.21.15.33:0/3371830658 conn(0x7f51f406d9b0 msgr2=0x7f51f4159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 shutdown_connections 2023-10-06T05:14:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.729+0000 7f51f959d700 1 -- 172.21.15.33:0/3371830658 wait complete. 2023-10-06T05:14:27.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:14:27.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.946+0000 7fe88759d700 1 Processor -- start 2023-10-06T05:14:27.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.946+0000 7fe88759d700 1 -- start start 2023-10-06T05:14:27.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.947+0000 7fe88759d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8800cd5e0 0x7fe8800af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.947+0000 7fe88759d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe88015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.947+0000 7fe88759d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88015d660 con 0x7fe8800af540 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.947+0000 7fe88759d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800ce890 con 0x7fe8800cd5e0 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.947+0000 7fe88759d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88015aab0 con 0x7fe88015a650 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.947+0000 7fe885d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8800cd5e0 0x7fe8800af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe88659b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe8800af540 0x7fe8800cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50076/0 (socket says 172.21.15.33:50076) 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe88659b700 1 -- 172.21.15.33:0/2130304369 learned_addr learned my addr 172.21.15.33:0/2130304369 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe886d9c700 1 --2- 172.21.15.33:0/2130304369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe88015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885d9a700 1 -- 172.21.15.33:0/2130304369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 msgr2=0x7fe88015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885d9a700 1 --2- 172.21.15.33:0/2130304369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe88015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885d9a700 1 -- 172.21.15.33:0/2130304369 >> v1:172.21.15.33:6789/0 conn(0x7fe8800af540 legacy=0x7fe8800cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:27.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885d9a700 1 -- 172.21.15.33:0/2130304369 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88015d890 con 0x7fe8800cd5e0 2023-10-06T05:14:27.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885599700 1 -- 172.21.15.33:0/2130304369 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995616670 0 0) 0x7fe88015d660 con 0x7fe8800af540 2023-10-06T05:14:27.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885d9a700 1 --2- 172.21.15.33:0/2130304369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800cd5e0 0x7fe8800af910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098e0 tx=0x7fe8700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cf3fda6998ab56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.948+0000 7fe885599700 1 -- 172.21.15.33:0/2130304369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870009e80 con 0x7fe8800cd5e0 2023-10-06T05:14:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe885599700 1 -- 172.21.15.33:0/2130304369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000b860 con 0x7fe8800cd5e0 2023-10-06T05:14:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe885599700 1 -- 172.21.15.33:0/2130304369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000bbe0 con 0x7fe8800cd5e0 2023-10-06T05:14:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 -- 172.21.15.33:0/2130304369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800cd5e0 msgr2=0x7fe8800af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 --2- 172.21.15.33:0/2130304369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800cd5e0 0x7fe8800af910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098e0 tx=0x7fe8700091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 -- 172.21.15.33:0/2130304369 shutdown_connections 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 --2- 172.21.15.33:0/2130304369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe88015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 --2- 172.21.15.33:0/2130304369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800cd5e0 0x7fe8800af910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 -- 172.21.15.33:0/2130304369 >> 172.21.15.33:0/2130304369 conn(0x7fe88006d9b0 msgr2=0x7fe88006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 -- 172.21.15.33:0/2130304369 shutdown_connections 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.949+0000 7fe88759d700 1 -- 172.21.15.33:0/2130304369 wait complete. 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.951+0000 7fe88759d700 1 Processor -- start 2023-10-06T05:14:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.951+0000 7fe88759d700 1 -- start start 2023-10-06T05:14:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88759d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 0x7fe8800b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88759d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8800cd5e0 0x7fe8801579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88759d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe8800b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88759d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800aec60 con 0x7fe8800af540 2023-10-06T05:14:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88759d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800b4580 con 0x7fe8800cd5e0 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88759d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800b3970 con 0x7fe88015a650 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 0x7fe8800b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 0x7fe8800b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33090/0 (socket says 172.21.15.33:33090) 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 -- 172.21.15.33:0/3388943991 learned_addr learned my addr 172.21.15.33:0/3388943991 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe885d9a700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8800cd5e0 0x7fe8801579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe886d9c700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe8800b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 -- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 msgr2=0x7fe8800b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe8800b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 -- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8800cd5e0 msgr2=0x7fe8801579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8800cd5e0 0x7fe8801579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe88659b700 1 -- 172.21.15.33:0/3388943991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870009420 con 0x7fe8800af540 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.952+0000 7fe886d9c700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe8800b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.953+0000 7fe88659b700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 0x7fe8800b3270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c5b0 tx=0x7fe87c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.953+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c005b30 con 0x7fe8800af540 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.953+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c005c90 con 0x7fe8800af540 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.953+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c007300 con 0x7fe8800af540 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.953+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8800b3c50 con 0x7fe8800af540 2023-10-06T05:14:27.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.953+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8800b4210 con 0x7fe8800af540 2023-10-06T05:14:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.954+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88015d380 con 0x7fe8800af540 2023-10-06T05:14:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.955+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe87c01b3b0 con 0x7fe8800af540 2023-10-06T05:14:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.956+0000 7fe8767fc700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe864066f30 0x7fe8640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.956+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe87c083c60 con 0x7fe8800af540 2023-10-06T05:14:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.956+0000 7fe885d9a700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe864066f30 0x7fe8640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.957+0000 7fe885d9a700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe864066f30 0x7fe8640693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8700091e0 tx=0x7fe870017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:27.960+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe87c050cd0 con 0x7fe8800af540 2023-10-06T05:14:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.207+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe88004c380 con 0x7fe8800af540 2023-10-06T05:14:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.737+0000 7fe8767fc700 1 -- 172.21.15.33:0/3388943991 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe87c017020 con 0x7fe8800af540 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe864066f30 msgr2=0x7fe8640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe864066f30 0x7fe8640693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8700091e0 tx=0x7fe870017000 comp rx=0 tx=0).stop 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 msgr2=0x7fe8800b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 0x7fe8800b3270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c5b0 tx=0x7fe87c001070 comp rx=0 tx=0).stop 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 shutdown_connections 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe864066f30 0x7fe8640693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe88015a650 0x7fe8800b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8800af540 0x7fe8800b3270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 --2- 172.21.15.33:0/3388943991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8800cd5e0 0x7fe8801579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 >> 172.21.15.33:0/3388943991 conn(0x7fe88006d9b0 msgr2=0x7fe8800d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 shutdown_connections 2023-10-06T05:14:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.740+0000 7fe88759d700 1 -- 172.21.15.33:0/3388943991 wait complete. 2023-10-06T05:14:28.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:14:28.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.946+0000 7f9883fff700 1 Processor -- start 2023-10-06T05:14:28.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.946+0000 7f9883fff700 1 -- start start 2023-10-06T05:14:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.947+0000 7f9883fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9884159bb0 0x7f98840b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.947+0000 7f9883fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840cbd80 con 0x7f98840cf3b0 2023-10-06T05:14:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.947+0000 7f9883fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840af5f0 con 0x7f98840ccfc0 2023-10-06T05:14:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.947+0000 7f9883fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98840af8d0 con 0x7f9884159bb0 2023-10-06T05:14:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9884159bb0 0x7f98840b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9884159bb0 0x7f98840b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33094/0 (socket says 172.21.15.33:33094) 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 -- 172.21.15.33:0/1594927485 learned_addr learned my addr 172.21.15.33:0/1594927485 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98827fc700 1 --1- 172.21.15.33:0/1594927485 >> v1:172.21.15.33:6789/0 conn(0x7f98840cf3b0 0x7f98841594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50084/0 (socket says 172.21.15.33:50084) 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 -- 172.21.15.33:0/1594927485 >> v1:172.21.15.167:6789/0 conn(0x7f98840ccfc0 legacy=0x7f98840cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 -- 172.21.15.33:0/1594927485 >> v1:172.21.15.33:6789/0 conn(0x7f98840cf3b0 legacy=0x7f98841594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f9881ffb700 1 -- 172.21.15.33:0/1594927485 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323825720 0 0) 0x7f98840af5f0 con 0x7f98840ccfc0 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 -- 172.21.15.33:0/1594927485 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840afbb0 con 0x7f9884159bb0 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.948+0000 7f98837fe700 1 --2- 172.21.15.33:0/1594927485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9884159bb0 0x7f98840b0460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f987c009f50 tx=0x7f987c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f15361ff7ced server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9881ffb700 1 -- 172.21.15.33:0/1594927485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00e070 con 0x7f9884159bb0 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9881ffb700 1 -- 172.21.15.33:0/1594927485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c00b9b0 con 0x7f9884159bb0 2023-10-06T05:14:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9881ffb700 1 -- 172.21.15.33:0/1594927485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0092e0 con 0x7f9884159bb0 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 -- 172.21.15.33:0/1594927485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9884159bb0 msgr2=0x7f98840b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 --2- 172.21.15.33:0/1594927485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9884159bb0 0x7f98840b0460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f987c009f50 tx=0x7f987c009840 comp rx=0 tx=0).stop 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 -- 172.21.15.33:0/1594927485 shutdown_connections 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 --2- 172.21.15.33:0/1594927485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9884159bb0 0x7f98840b0460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 -- 172.21.15.33:0/1594927485 >> 172.21.15.33:0/1594927485 conn(0x7f988406cff0 msgr2=0x7f988406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 -- 172.21.15.33:0/1594927485 shutdown_connections 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.949+0000 7f9883fff700 1 -- 172.21.15.33:0/1594927485 wait complete. 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.951+0000 7f9883fff700 1 Processor -- start 2023-10-06T05:14:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.951+0000 7f9883fff700 1 -- start start 2023-10-06T05:14:28.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9883fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 0x7f98841541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:28.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9883fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9884159bb0 0x7f98840b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:28.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9883fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840c92a0 0x7f9884157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9883fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98840b45b0 con 0x7f98840ccfc0 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9883fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98840b4890 con 0x7f98840c92a0 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9883fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840b4b70 con 0x7f9884159bb0 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 0x7f98841541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 0x7f98841541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33098/0 (socket says 172.21.15.33:33098) 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 -- 172.21.15.33:0/3537994552 learned_addr learned my addr 172.21.15.33:0/3537994552 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f98827fc700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9884159bb0 0x7f98840b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f98837fe700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840c92a0 0x7f9884157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 -- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9884159bb0 msgr2=0x7f98840b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9884159bb0 0x7f98840b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 -- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840c92a0 msgr2=0x7f9884157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:28.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840c92a0 0x7f9884157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f9882ffd700 1 -- 172.21.15.33:0/3537994552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c009ae0 con 0x7f98840ccfc0 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f98837fe700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840c92a0 0x7f9884157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.952+0000 7f98827fc700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9884159bb0 0x7f98840b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f9882ffd700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 0x7f98841541e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f987400c7e0 tx=0x7f9874007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874012700 con 0x7f98840ccfc0 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874012880 con 0x7f98840ccfc0 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874012c40 con 0x7f98840ccfc0 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98840b4e50 con 0x7f98840ccfc0 2023-10-06T05:14:28.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.953+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98840b41a0 con 0x7f98840ccfc0 2023-10-06T05:14:28.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.954+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98840b49d0 con 0x7f98840ccfc0 2023-10-06T05:14:28.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.955+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f987401b400 con 0x7f98840ccfc0 2023-10-06T05:14:28.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.956+0000 7f986affd700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9870066f30 0x7f98700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:28.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.956+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9874084300 con 0x7f98840ccfc0 2023-10-06T05:14:28.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.956+0000 7f98827fc700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9870066f30 0x7f98700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:28.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.957+0000 7f98827fc700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9870066f30 0x7f98700693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f988406bdf0 tx=0x7f9878000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:28.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:28.960+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98840b49d0 con 0x7f98840ccfc0 2023-10-06T05:14:29.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.208+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98840b49d0 con 0x7f98840ccfc0 2023-10-06T05:14:29.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.209+0000 7f986affd700 1 -- 172.21.15.33:0/3537994552 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f98840b49d0 con 0x7f98840ccfc0 2023-10-06T05:14:29.212 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:29.212 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:14:29.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9870066f30 msgr2=0x7f98700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9870066f30 0x7f98700693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f988406bdf0 tx=0x7f9878000bc0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 msgr2=0x7f98841541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 0x7f98841541e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f987400c7e0 tx=0x7f9874007df0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 shutdown_connections 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9870066f30 0x7f98700693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98840c92a0 0x7f9884157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98840ccfc0 0x7f98841541e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 --2- 172.21.15.33:0/3537994552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9884159bb0 0x7f98840b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 >> 172.21.15.33:0/3537994552 conn(0x7f988406cff0 msgr2=0x7f98840cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 shutdown_connections 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.211+0000 7f9883fff700 1 -- 172.21.15.33:0/3537994552 wait complete. 2023-10-06T05:14:29.215 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:14:29.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.415+0000 7f320219d700 1 Processor -- start 2023-10-06T05:14:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.415+0000 7f320219d700 1 -- start start 2023-10-06T05:14:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.416+0000 7f320219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.416+0000 7f320219d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31fc06aad0 0x7f31fc06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.416+0000 7f320219d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0c9cb0 con 0x7f31fc0c7940 2023-10-06T05:14:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.416+0000 7f320219d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0ca270 con 0x7f31fc06aad0 2023-10-06T05:14:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.416+0000 7f320219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc06b840 con 0x7f31fc13faa0 2023-10-06T05:14:29.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.417+0000 7f320119b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.417+0000 7f320119b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:52072/0 (socket says 172.21.15.33:52072) 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.417+0000 7f320119b700 1 -- 172.21.15.33:0/973308927 learned_addr learned my addr 172.21.15.33:0/973308927 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.417+0000 7f320199c700 1 --2- 172.21.15.33:0/973308927 >> v2:172.21.15.167:3300/0 conn(0x7f31fc06aad0 0x7f31fc06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.417+0000 7f320099a700 1 --1- 172.21.15.33:0/973308927 >> v1:172.21.15.167:6789/0 conn(0x7f31fc0c7940 0x7f31fc0c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49932/0 (socket says 172.21.15.33:49932) 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320119b700 1 -- 172.21.15.33:0/973308927 >> v2:172.21.15.167:3300/0 conn(0x7f31fc06aad0 msgr2=0x7f31fc06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320119b700 1 --2- 172.21.15.33:0/973308927 >> v2:172.21.15.167:3300/0 conn(0x7f31fc06aad0 0x7f31fc06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320119b700 1 -- 172.21.15.33:0/973308927 >> v1:172.21.15.167:6789/0 conn(0x7f31fc0c7940 legacy=0x7f31fc0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f31f3fff700 1 -- 172.21.15.33:0/973308927 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440893707 0 0) 0x7f31fc0c9cb0 con 0x7f31fc0c7940 2023-10-06T05:14:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320119b700 1 -- 172.21.15.33:0/973308927 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc06bb20 con 0x7f31fc13faa0 2023-10-06T05:14:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320119b700 1 --2- 172.21.15.33:0/973308927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc13fe70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f31f80099b0 tx=0x7f31f8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e11a7d21a2ab9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f31f3fff700 1 -- 172.21.15.33:0/973308927 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8009ea0 con 0x7f31fc13faa0 2023-10-06T05:14:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f31f3fff700 1 -- 172.21.15.33:0/973308927 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f800b860 con 0x7f31fc13faa0 2023-10-06T05:14:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f31f3fff700 1 -- 172.21.15.33:0/973308927 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800bc00 con 0x7f31fc13faa0 2023-10-06T05:14:29.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320219d700 1 -- 172.21.15.33:0/973308927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 msgr2=0x7f31fc13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.418+0000 7f320219d700 1 --2- 172.21.15.33:0/973308927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc13fe70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f31f80099b0 tx=0x7f31f8009250 comp rx=0 tx=0).stop 2023-10-06T05:14:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.419+0000 7f320219d700 1 -- 172.21.15.33:0/973308927 shutdown_connections 2023-10-06T05:14:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.419+0000 7f320219d700 1 --2- 172.21.15.33:0/973308927 >> v2:172.21.15.167:3300/0 conn(0x7f31fc06aad0 0x7f31fc06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.419+0000 7f320219d700 1 --2- 172.21.15.33:0/973308927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc13fe70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.419+0000 7f320219d700 1 -- 172.21.15.33:0/973308927 >> 172.21.15.33:0/973308927 conn(0x7f31fc06d030 msgr2=0x7f31fc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.419+0000 7f320219d700 1 -- 172.21.15.33:0/973308927 shutdown_connections 2023-10-06T05:14:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.419+0000 7f320219d700 1 -- 172.21.15.33:0/973308927 wait complete. 2023-10-06T05:14:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.420+0000 7f320219d700 1 Processor -- start 2023-10-06T05:14:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.420+0000 7f320219d700 1 -- start start 2023-10-06T05:14:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31fc06aad0 0x7f31fc0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 0x7f31fc0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c45f0 con 0x7f31fc0c7940 2023-10-06T05:14:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c39e0 con 0x7f31fc06aad0 2023-10-06T05:14:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c3cc0 con 0x7f31fc13faa0 2023-10-06T05:14:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 0x7f31fc0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 0x7f31fc0c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33104/0 (socket says 172.21.15.33:33104) 2023-10-06T05:14:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320099a700 1 -- 172.21.15.33:0/2407024672 learned_addr learned my addr 172.21.15.33:0/2407024672 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.421+0000 7f320119b700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31fc06aad0 0x7f31fc0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320199c700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320099a700 1 -- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 msgr2=0x7f31fc0c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320099a700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc0c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320099a700 1 -- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31fc06aad0 msgr2=0x7f31fc0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320099a700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31fc06aad0 0x7f31fc0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320099a700 1 -- 172.21.15.33:0/2407024672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f80094f0 con 0x7f31fc0c7940 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320099a700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 0x7f31fc0c5620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0011e0 tx=0x7f31ec007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec005a70 con 0x7f31fc0c7940 2023-10-06T05:14:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec005bd0 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc0c3fa0 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0070d0 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.422+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc0c36d0 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.423+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc0c4760 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.424+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31ec0074c0 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.425+0000 7f31f17fa700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31e0066f30 0x7f31e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.425+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31ec083b60 con 0x7f31fc0c7940 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.426+0000 7f320119b700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31e0066f30 0x7f31e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.426+0000 7f320119b700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31e0066f30 0x7f31e00693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31f80099b0 tx=0x7f31f8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.429+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31ec050bd0 con 0x7f31fc0c7940 2023-10-06T05:14:29.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.631+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31fc04c380 con 0x7f31fc0c7940 2023-10-06T05:14:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.632+0000 7f31f17fa700 1 -- 172.21.15.33:0/2407024672 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f31ec017020 con 0x7f31fc0c7940 2023-10-06T05:14:29.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:29.635 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31e0066f30 msgr2=0x7f31e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31e0066f30 0x7f31e00693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31f80099b0 tx=0x7f31f8017000 comp rx=0 tx=0).stop 2023-10-06T05:14:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 msgr2=0x7f31fc0c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 0x7f31fc0c5620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0011e0 tx=0x7f31ec007e10 comp rx=0 tx=0).stop 2023-10-06T05:14:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 shutdown_connections 2023-10-06T05:14:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31e0066f30 0x7f31e00693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31fc13faa0 0x7f31fc0c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31fc06aad0 0x7f31fc0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 --2- 172.21.15.33:0/2407024672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc0c7940 0x7f31fc0c5620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 >> 172.21.15.33:0/2407024672 conn(0x7f31fc06d030 msgr2=0x7f31fc0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:29.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 shutdown_connections 2023-10-06T05:14:29.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.634+0000 7f320219d700 1 -- 172.21.15.33:0/2407024672 wait complete. 2023-10-06T05:14:29.655 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:14:29.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:14:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.840+0000 7f64f119d700 1 Processor -- start 2023-10-06T05:14:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.840+0000 7f64f119d700 1 -- start start 2023-10-06T05:14:29.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.841+0000 7f64f119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f64ec0c8b50 0x7f64ec0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f119d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64ec155ec0 0x7f64ec0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f119d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec06ab60 con 0x7f64ec155af0 2023-10-06T05:14:29.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f119d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec06b7e0 con 0x7f64ec155ec0 2023-10-06T05:14:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec06ae40 con 0x7f64ec0c8b50 2023-10-06T05:14:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64ebfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f64ec155af0 0x7f64ec06a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50092/0 (socket says 172.21.15.33:50092) 2023-10-06T05:14:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64ebfff700 1 -- 172.21.15.33:0/2111022901 learned_addr learned my addr 172.21.15.33:0/2111022901 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f099c700 1 --2- 172.21.15.33:0/2111022901 >> v2:172.21.15.33:3300/0 conn(0x7f64ec155ec0 0x7f64ec0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64eb7fe700 1 --2- 172.21.15.33:0/2111022901 >> v2:172.21.15.167:3300/0 conn(0x7f64ec0c8b50 0x7f64ec0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64eaffd700 1 -- 172.21.15.33:0/2111022901 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632344992 0 0) 0x7f64ec06ab60 con 0x7f64ec155af0 2023-10-06T05:14:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64eaffd700 1 -- 172.21.15.33:0/2111022901 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d4003570 con 0x7f64ec155af0 2023-10-06T05:14:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f099c700 1 -- 172.21.15.33:0/2111022901 >> v2:172.21.15.167:3300/0 conn(0x7f64ec0c8b50 msgr2=0x7f64ec0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f099c700 1 --2- 172.21.15.33:0/2111022901 >> v2:172.21.15.167:3300/0 conn(0x7f64ec0c8b50 0x7f64ec0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f099c700 1 -- 172.21.15.33:0/2111022901 >> v1:172.21.15.33:6789/0 conn(0x7f64ec155af0 legacy=0x7f64ec06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64f099c700 1 -- 172.21.15.33:0/2111022901 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec06b120 con 0x7f64ec155ec0 2023-10-06T05:14:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.842+0000 7f64eb7fe700 1 --2- 172.21.15.33:0/2111022901 >> v2:172.21.15.167:3300/0 conn(0x7f64ec0c8b50 0x7f64ec0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f099c700 1 --2- 172.21.15.33:0/2111022901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0c7750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f64ec06f250 tx=0x7f64e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2e3d832edb32cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64eaffd700 1 -- 172.21.15.33:0/2111022901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400e070 con 0x7f64ec155ec0 2023-10-06T05:14:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64eaffd700 1 -- 172.21.15.33:0/2111022901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400bb60 con 0x7f64ec155ec0 2023-10-06T05:14:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64eaffd700 1 -- 172.21.15.33:0/2111022901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4012430 con 0x7f64ec155ec0 2023-10-06T05:14:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f119d700 1 -- 172.21.15.33:0/2111022901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 msgr2=0x7f64ec0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f119d700 1 --2- 172.21.15.33:0/2111022901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0c7750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f64ec06f250 tx=0x7f64e4009780 comp rx=0 tx=0).stop 2023-10-06T05:14:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f119d700 1 -- 172.21.15.33:0/2111022901 shutdown_connections 2023-10-06T05:14:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f119d700 1 --2- 172.21.15.33:0/2111022901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0c7750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f119d700 1 --2- 172.21.15.33:0/2111022901 >> v2:172.21.15.167:3300/0 conn(0x7f64ec0c8b50 0x7f64ec0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.843+0000 7f64f119d700 1 -- 172.21.15.33:0/2111022901 >> 172.21.15.33:0/2111022901 conn(0x7f64ec06d030 msgr2=0x7f64ec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.844+0000 7f64f119d700 1 -- 172.21.15.33:0/2111022901 shutdown_connections 2023-10-06T05:14:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.844+0000 7f64f119d700 1 -- 172.21.15.33:0/2111022901 wait complete. 2023-10-06T05:14:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.845+0000 7f64f119d700 1 Processor -- start 2023-10-06T05:14:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.845+0000 7f64f119d700 1 -- start start 2023-10-06T05:14:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64ec0c8b50 0x7f64ec0aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64ec155af0 0x7f64ec1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0abad0 con 0x7f64ec155ec0 2023-10-06T05:14:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0a9dd0 con 0x7f64ec0c8b50 2023-10-06T05:14:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0aa050 con 0x7f64ec155af0 2023-10-06T05:14:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33126/0 (socket says 172.21.15.33:33126) 2023-10-06T05:14:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.846+0000 7f64f099c700 1 -- 172.21.15.33:0/1487613950 learned_addr learned my addr 172.21.15.33:0/1487613950 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64eb7fe700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64ec155af0 0x7f64ec1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f099c700 1 -- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64ec155af0 msgr2=0x7f64ec1501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64ebfff700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64ec0c8b50 0x7f64ec0aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f099c700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64ec155af0 0x7f64ec1501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f099c700 1 -- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64ec0c8b50 msgr2=0x7f64ec0aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f099c700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64ec0c8b50 0x7f64ec0aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f099c700 1 -- 172.21.15.33:0/1487613950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4009a20 con 0x7f64ec155ec0 2023-10-06T05:14:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64eb7fe700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64ec155af0 0x7f64ec1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64ebfff700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64ec0c8b50 0x7f64ec0aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f099c700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0a9890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e40032f0 con 0x7f64ec155ec0 2023-10-06T05:14:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4012e00 con 0x7f64ec155ec0 2023-10-06T05:14:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e401d880 con 0x7f64ec155ec0 2023-10-06T05:14:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0aa330 con 0x7f64ec155ec0 2023-10-06T05:14:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.847+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0c47c0 con 0x7f64ec155ec0 2023-10-06T05:14:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.848+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec0c49b0 con 0x7f64ec155ec0 2023-10-06T05:14:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.849+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64e4024070 con 0x7f64ec155ec0 2023-10-06T05:14:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.850+0000 7f64d3fff700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64d8066f30 0x7f64d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.851+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f64e4086e40 con 0x7f64ec155ec0 2023-10-06T05:14:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.851+0000 7f64ebfff700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64d8066f30 0x7f64d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.851+0000 7f64ebfff700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64d8066f30 0x7f64d80693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64dc007fd0 tx=0x7f64dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:29.855+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64e4053eb0 con 0x7f64ec155ec0 2023-10-06T05:14:30.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.054+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64ec04c380 con 0x7f64d8066f30 2023-10-06T05:14:30.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.065+0000 7f64d3fff700 1 -- 172.21.15.33:0/1487613950 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286679 (secure 0 0 0) 0x7f64ec04c380 con 0x7f64d8066f30 2023-10-06T05:14:30.071 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64d8066f30 msgr2=0x7f64d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64d8066f30 0x7f64d80693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64dc007fd0 tx=0x7f64dc009040 comp rx=0 tx=0).stop 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 msgr2=0x7f64ec0a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0a9890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e4019000 comp rx=0 tx=0).stop 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 shutdown_connections 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64d8066f30 0x7f64d80693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64ec155ec0 0x7f64ec0a9890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64ec0c8b50 0x7f64ec0aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 --2- 172.21.15.33:0/1487613950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64ec155af0 0x7f64ec1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 >> 172.21.15.33:0/1487613950 conn(0x7f64ec06d030 msgr2=0x7f64ec159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 shutdown_connections 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.069+0000 7f64f119d700 1 -- 172.21.15.33:0/1487613950 wait complete. 2023-10-06T05:14:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:14:30.089 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T05:14:29.818477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381320,"kb_used_data":3016,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593400,"statfs":{"total":773094113280,"available":770655641600,"internally_reserved":0,"allocated":3088384,"data_stored":1846772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001574"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733340+0000","last_change":"2023-10-06T05:14:25.733340+0000","last_active":"2023-10-06T05:14:25.733340+0000","last_peered":"2023-10-06T05:14:25.733340+0000","last_clean":"2023-10-06T05:14:25.733340+0000","last_became_active":"2023-10-06T05:14:25.732861+0000","last_became_peered":"2023-10-06T05:14:25.732861+0000","last_unstale":"2023-10-06T05:14:25.733340+0000","last_undegraded":"2023-10-06T05:14:25.733340+0000","last_fullsized":"2023-10-06T05:14:25.733340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:19.965653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733215+0000","last_change":"2023-10-06T05:14:25.730206+0000","last_active":"2023-10-06T05:14:25.733215+0000","last_peered":"2023-10-06T05:14:25.733215+0000","last_clean":"2023-10-06T05:14:25.733215+0000","last_became_active":"2023-10-06T05:14:25.729947+0000","last_became_peered":"2023-10-06T05:14:25.729947+0000","last_unstale":"2023-10-06T05:14:25.733215+0000","last_undegraded":"2023-10-06T05:14:25.733215+0000","last_fullsized":"2023-10-06T05:14:25.733215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744108+0000","last_change":"2023-10-06T05:14:25.744108+0000","last_active":"2023-10-06T05:14:25.744108+0000","last_peered":"2023-10-06T05:14:25.744108+0000","last_clean":"2023-10-06T05:14:25.744108+0000","last_became_active":"2023-10-06T05:14:25.741869+0000","last_became_peered":"2023-10-06T05:14:25.741869+0000","last_unstale":"2023-10-06T05:14:25.744108+0000","last_undegraded":"2023-10-06T05:14:25.744108+0000","last_fullsized":"2023-10-06T05:14:25.744108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:26.682160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727785+0000","last_change":"2023-10-06T05:14:25.724254+0000","last_active":"2023-10-06T05:14:25.727785+0000","last_peered":"2023-10-06T05:14:25.727785+0000","last_clean":"2023-10-06T05:14:25.727785+0000","last_became_active":"2023-10-06T05:14:25.723995+0000","last_became_peered":"2023-10-06T05:14:25.723995+0000","last_unstale":"2023-10-06T05:14:25.727785+0000","last_undegraded":"2023-10-06T05:14:25.727785+0000","last_fullsized":"2023-10-06T05:14:25.727785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:07.229350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726644+0000","last_change":"2023-10-06T05:14:25.726644+0000","last_active":"2023-10-06T05:14:25.726644+0000","last_peered":"2023-10-06T05:14:25.726644+0000","last_clean":"2023-10-06T05:14:25.726644+0000","last_became_active":"2023-10-06T05:14:25.726030+0000","last_became_peered":"2023-10-06T05:14:25.726030+0000","last_unstale":"2023-10-06T05:14:25.726644+0000","last_undegraded":"2023-10-06T05:14:25.726644+0000","last_fullsized":"2023-10-06T05:14:25.726644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743830+0000","last_change":"2023-10-06T05:14:25.743830+0000","last_active":"2023-10-06T05:14:25.743830+0000","last_peered":"2023-10-06T05:14:25.743830+0000","last_clean":"2023-10-06T05:14:25.743830+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.743830+0000","last_undegraded":"2023-10-06T05:14:25.743830+0000","last_fullsized":"2023-10-06T05:14:25.743830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:43.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737791+0000","last_change":"2023-10-06T05:14:25.737791+0000","last_active":"2023-10-06T05:14:25.737791+0000","last_peered":"2023-10-06T05:14:25.737791+0000","last_clean":"2023-10-06T05:14:25.737791+0000","last_became_active":"2023-10-06T05:14:25.737334+0000","last_became_peered":"2023-10-06T05:14:25.737334+0000","last_unstale":"2023-10-06T05:14:25.737791+0000","last_undegraded":"2023-10-06T05:14:25.737791+0000","last_fullsized":"2023-10-06T05:14:25.737791+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744210+0000","last_change":"2023-10-06T05:14:25.743380+0000","last_active":"2023-10-06T05:14:25.744210+0000","last_peered":"2023-10-06T05:14:25.744210+0000","last_clean":"2023-10-06T05:14:25.744210+0000","last_became_active":"2023-10-06T05:14:25.743115+0000","last_became_peered":"2023-10-06T05:14:25.743115+0000","last_unstale":"2023-10-06T05:14:25.744210+0000","last_undegraded":"2023-10-06T05:14:25.744210+0000","last_fullsized":"2023-10-06T05:14:25.744210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730260+0000","last_change":"2023-10-06T05:14:25.730260+0000","last_active":"2023-10-06T05:14:25.730260+0000","last_peered":"2023-10-06T05:14:25.730260+0000","last_clean":"2023-10-06T05:14:25.730260+0000","last_became_active":"2023-10-06T05:14:25.729967+0000","last_became_peered":"2023-10-06T05:14:25.729967+0000","last_unstale":"2023-10-06T05:14:25.730260+0000","last_undegraded":"2023-10-06T05:14:25.730260+0000","last_fullsized":"2023-10-06T05:14:25.730260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.734367+0000","last_change":"2023-10-06T05:14:25.732052+0000","last_active":"2023-10-06T05:14:25.734367+0000","last_peered":"2023-10-06T05:14:25.734367+0000","last_clean":"2023-10-06T05:14:25.734367+0000","last_became_active":"2023-10-06T05:14:25.731746+0000","last_became_peered":"2023-10-06T05:14:25.731746+0000","last_unstale":"2023-10-06T05:14:25.734367+0000","last_undegraded":"2023-10-06T05:14:25.734367+0000","last_fullsized":"2023-10-06T05:14:25.734367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741800+0000","last_change":"2023-10-06T05:14:25.741800+0000","last_active":"2023-10-06T05:14:25.741800+0000","last_peered":"2023-10-06T05:14:25.741800+0000","last_clean":"2023-10-06T05:14:25.741800+0000","last_became_active":"2023-10-06T05:14:25.741167+0000","last_became_peered":"2023-10-06T05:14:25.741167+0000","last_unstale":"2023-10-06T05:14:25.741800+0000","last_undegraded":"2023-10-06T05:14:25.741800+0000","last_fullsized":"2023-10-06T05:14:25.741800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743115+0000","last_change":"2023-10-06T05:14:25.743115+0000","last_active":"2023-10-06T05:14:25.743115+0000","last_peered":"2023-10-06T05:14:25.743115+0000","last_clean":"2023-10-06T05:14:25.743115+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.743115+0000","last_undegraded":"2023-10-06T05:14:25.743115+0000","last_fullsized":"2023-10-06T05:14:25.743115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.734027+0000","last_change":"2023-10-06T05:14:25.734027+0000","last_active":"2023-10-06T05:14:25.734027+0000","last_peered":"2023-10-06T05:14:25.734027+0000","last_clean":"2023-10-06T05:14:25.734027+0000","last_became_active":"2023-10-06T05:14:25.733743+0000","last_became_peered":"2023-10-06T05:14:25.733743+0000","last_unstale":"2023-10-06T05:14:25.734027+0000","last_undegraded":"2023-10-06T05:14:25.734027+0000","last_fullsized":"2023-10-06T05:14:25.734027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725222+0000","last_change":"2023-10-06T05:14:25.721669+0000","last_active":"2023-10-06T05:14:25.725222+0000","last_peered":"2023-10-06T05:14:25.725222+0000","last_clean":"2023-10-06T05:14:25.725222+0000","last_became_active":"2023-10-06T05:14:25.721412+0000","last_became_peered":"2023-10-06T05:14:25.721412+0000","last_unstale":"2023-10-06T05:14:25.725222+0000","last_undegraded":"2023-10-06T05:14:25.725222+0000","last_fullsized":"2023-10-06T05:14:25.725222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737798+0000","last_change":"2023-10-06T05:14:25.737798+0000","last_active":"2023-10-06T05:14:25.737798+0000","last_peered":"2023-10-06T05:14:25.737798+0000","last_clean":"2023-10-06T05:14:25.737798+0000","last_became_active":"2023-10-06T05:14:25.737342+0000","last_became_peered":"2023-10-06T05:14:25.737342+0000","last_unstale":"2023-10-06T05:14:25.737798+0000","last_undegraded":"2023-10-06T05:14:25.737798+0000","last_fullsized":"2023-10-06T05:14:25.737798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740804+0000","last_change":"2023-10-06T05:14:25.740804+0000","last_active":"2023-10-06T05:14:25.740804+0000","last_peered":"2023-10-06T05:14:25.740804+0000","last_clean":"2023-10-06T05:14:25.740804+0000","last_became_active":"2023-10-06T05:14:25.740330+0000","last_became_peered":"2023-10-06T05:14:25.740330+0000","last_unstale":"2023-10-06T05:14:25.740804+0000","last_undegraded":"2023-10-06T05:14:25.740804+0000","last_fullsized":"2023-10-06T05:14:25.740804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.955604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725672+0000","last_change":"2023-10-06T05:14:25.725672+0000","last_active":"2023-10-06T05:14:25.725672+0000","last_peered":"2023-10-06T05:14:25.725672+0000","last_clean":"2023-10-06T05:14:25.725672+0000","last_became_active":"2023-10-06T05:14:25.725392+0000","last_became_peered":"2023-10-06T05:14:25.725392+0000","last_unstale":"2023-10-06T05:14:25.725672+0000","last_undegraded":"2023-10-06T05:14:25.725672+0000","last_fullsized":"2023-10-06T05:14:25.725672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743525+0000","last_change":"2023-10-06T05:14:25.743525+0000","last_active":"2023-10-06T05:14:25.743525+0000","last_peered":"2023-10-06T05:14:25.743525+0000","last_clean":"2023-10-06T05:14:25.743525+0000","last_became_active":"2023-10-06T05:14:25.742820+0000","last_became_peered":"2023-10-06T05:14:25.742820+0000","last_unstale":"2023-10-06T05:14:25.743525+0000","last_undegraded":"2023-10-06T05:14:25.743525+0000","last_fullsized":"2023-10-06T05:14:25.743525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:42.341048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.718000+0000","last_change":"2023-10-06T05:14:25.718000+0000","last_active":"2023-10-06T05:14:25.718000+0000","last_peered":"2023-10-06T05:14:25.718000+0000","last_clean":"2023-10-06T05:14:25.718000+0000","last_became_active":"2023-10-06T05:14:25.717615+0000","last_became_peered":"2023-10-06T05:14:25.717615+0000","last_unstale":"2023-10-06T05:14:25.718000+0000","last_undegraded":"2023-10-06T05:14:25.718000+0000","last_fullsized":"2023-10-06T05:14:25.718000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:17.095036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742445+0000","last_change":"2023-10-06T05:14:25.742445+0000","last_active":"2023-10-06T05:14:25.742445+0000","last_peered":"2023-10-06T05:14:25.742445+0000","last_clean":"2023-10-06T05:14:25.742445+0000","last_became_active":"2023-10-06T05:14:25.741675+0000","last_became_peered":"2023-10-06T05:14:25.741675+0000","last_unstale":"2023-10-06T05:14:25.742445+0000","last_undegraded":"2023-10-06T05:14:25.742445+0000","last_fullsized":"2023-10-06T05:14:25.742445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733702+0000","last_change":"2023-10-06T05:14:25.733702+0000","last_active":"2023-10-06T05:14:25.733702+0000","last_peered":"2023-10-06T05:14:25.733702+0000","last_clean":"2023-10-06T05:14:25.733702+0000","last_became_active":"2023-10-06T05:14:25.733387+0000","last_became_peered":"2023-10-06T05:14:25.733387+0000","last_unstale":"2023-10-06T05:14:25.733702+0000","last_undegraded":"2023-10-06T05:14:25.733702+0000","last_fullsized":"2023-10-06T05:14:25.733702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.724100+0000","last_change":"2023-10-06T05:14:25.724100+0000","last_active":"2023-10-06T05:14:25.724100+0000","last_peered":"2023-10-06T05:14:25.724100+0000","last_clean":"2023-10-06T05:14:25.724100+0000","last_became_active":"2023-10-06T05:14:25.723750+0000","last_became_peered":"2023-10-06T05:14:25.723750+0000","last_unstale":"2023-10-06T05:14:25.724100+0000","last_undegraded":"2023-10-06T05:14:25.724100+0000","last_fullsized":"2023-10-06T05:14:25.724100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735739+0000","last_change":"2023-10-06T05:14:25.727464+0000","last_active":"2023-10-06T05:14:25.735739+0000","last_peered":"2023-10-06T05:14:25.735739+0000","last_clean":"2023-10-06T05:14:25.735739+0000","last_became_active":"2023-10-06T05:14:25.727166+0000","last_became_peered":"2023-10-06T05:14:25.727166+0000","last_unstale":"2023-10-06T05:14:25.735739+0000","last_undegraded":"2023-10-06T05:14:25.735739+0000","last_fullsized":"2023-10-06T05:14:25.735739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730521+0000","last_change":"2023-10-06T05:14:25.730521+0000","last_active":"2023-10-06T05:14:25.730521+0000","last_peered":"2023-10-06T05:14:25.730521+0000","last_clean":"2023-10-06T05:14:25.730521+0000","last_became_active":"2023-10-06T05:14:25.730234+0000","last_became_peered":"2023-10-06T05:14:25.730234+0000","last_unstale":"2023-10-06T05:14:25.730521+0000","last_undegraded":"2023-10-06T05:14:25.730521+0000","last_fullsized":"2023-10-06T05:14:25.730521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735194+0000","last_change":"2023-10-06T05:14:25.733204+0000","last_active":"2023-10-06T05:14:25.735194+0000","last_peered":"2023-10-06T05:14:25.735194+0000","last_clean":"2023-10-06T05:14:25.735194+0000","last_became_active":"2023-10-06T05:14:25.732877+0000","last_became_peered":"2023-10-06T05:14:25.732877+0000","last_unstale":"2023-10-06T05:14:25.735194+0000","last_undegraded":"2023-10-06T05:14:25.735194+0000","last_fullsized":"2023-10-06T05:14:25.735194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735005+0000","last_change":"2023-10-06T05:14:25.724268+0000","last_active":"2023-10-06T05:14:25.735005+0000","last_peered":"2023-10-06T05:14:25.735005+0000","last_clean":"2023-10-06T05:14:25.735005+0000","last_became_active":"2023-10-06T05:14:25.724009+0000","last_became_peered":"2023-10-06T05:14:25.724009+0000","last_unstale":"2023-10-06T05:14:25.735005+0000","last_undegraded":"2023-10-06T05:14:25.735005+0000","last_fullsized":"2023-10-06T05:14:25.735005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727218+0000","last_change":"2023-10-06T05:14:25.725754+0000","last_active":"2023-10-06T05:14:25.727218+0000","last_peered":"2023-10-06T05:14:25.727218+0000","last_clean":"2023-10-06T05:14:25.727218+0000","last_became_active":"2023-10-06T05:14:25.725516+0000","last_became_peered":"2023-10-06T05:14:25.725516+0000","last_unstale":"2023-10-06T05:14:25.727218+0000","last_undegraded":"2023-10-06T05:14:25.727218+0000","last_fullsized":"2023-10-06T05:14:25.727218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735114+0000","last_change":"2023-10-06T05:14:24.734577+0000","last_active":"2023-10-06T05:14:25.735114+0000","last_peered":"2023-10-06T05:14:25.735114+0000","last_clean":"2023-10-06T05:14:25.735114+0000","last_became_active":"2023-10-06T05:14:24.732575+0000","last_became_peered":"2023-10-06T05:14:24.732575+0000","last_unstale":"2023-10-06T05:14:25.735114+0000","last_undegraded":"2023-10-06T05:14:25.735114+0000","last_fullsized":"2023-10-06T05:14:25.735114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726043+0000","last_change":"2023-10-06T05:14:25.721117+0000","last_active":"2023-10-06T05:14:25.726043+0000","last_peered":"2023-10-06T05:14:25.726043+0000","last_clean":"2023-10-06T05:14:25.726043+0000","last_became_active":"2023-10-06T05:14:25.720806+0000","last_became_peered":"2023-10-06T05:14:25.720806+0000","last_unstale":"2023-10-06T05:14:25.726043+0000","last_undegraded":"2023-10-06T05:14:25.726043+0000","last_fullsized":"2023-10-06T05:14:25.726043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.714069+0000","last_change":"2023-10-06T05:14:24.723717+0000","last_active":"2023-10-06T05:14:25.714069+0000","last_peered":"2023-10-06T05:14:25.714069+0000","last_clean":"2023-10-06T05:14:25.714069+0000","last_became_active":"2023-10-06T05:14:24.723440+0000","last_became_peered":"2023-10-06T05:14:24.723440+0000","last_unstale":"2023-10-06T05:14:25.714069+0000","last_undegraded":"2023-10-06T05:14:25.714069+0000","last_fullsized":"2023-10-06T05:14:25.714069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726258+0000","last_change":"2023-10-06T05:14:25.726258+0000","last_active":"2023-10-06T05:14:25.726258+0000","last_peered":"2023-10-06T05:14:25.726258+0000","last_clean":"2023-10-06T05:14:25.726258+0000","last_became_active":"2023-10-06T05:14:25.725989+0000","last_became_peered":"2023-10-06T05:14:25.725989+0000","last_unstale":"2023-10-06T05:14:25.726258+0000","last_undegraded":"2023-10-06T05:14:25.726258+0000","last_fullsized":"2023-10-06T05:14:25.726258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708532+0000","last_change":"2023-10-06T05:14:24.722594+0000","last_active":"2023-10-06T05:14:25.708532+0000","last_peered":"2023-10-06T05:14:25.708532+0000","last_clean":"2023-10-06T05:14:25.708532+0000","last_became_active":"2023-10-06T05:14:24.722302+0000","last_became_peered":"2023-10-06T05:14:24.722302+0000","last_unstale":"2023-10-06T05:14:25.708532+0000","last_undegraded":"2023-10-06T05:14:25.708532+0000","last_fullsized":"2023-10-06T05:14:25.708532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728738+0000","last_change":"2023-10-06T05:14:25.722475+0000","last_active":"2023-10-06T05:14:25.728738+0000","last_peered":"2023-10-06T05:14:25.728738+0000","last_clean":"2023-10-06T05:14:25.728738+0000","last_became_active":"2023-10-06T05:14:25.721707+0000","last_became_peered":"2023-10-06T05:14:25.721707+0000","last_unstale":"2023-10-06T05:14:25.728738+0000","last_undegraded":"2023-10-06T05:14:25.728738+0000","last_fullsized":"2023-10-06T05:14:25.728738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.727634+0000","last_change":"2023-10-06T05:14:24.724581+0000","last_active":"2023-10-06T05:14:24.727634+0000","last_peered":"2023-10-06T05:14:24.727634+0000","last_clean":"2023-10-06T05:14:24.727634+0000","last_became_active":"2023-10-06T05:14:24.724293+0000","last_became_peered":"2023-10-06T05:14:24.724293+0000","last_unstale":"2023-10-06T05:14:24.727634+0000","last_undegraded":"2023-10-06T05:14:24.727634+0000","last_fullsized":"2023-10-06T05:14:24.727634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723275+0000","last_change":"2023-10-06T05:14:25.719638+0000","last_active":"2023-10-06T05:14:25.723275+0000","last_peered":"2023-10-06T05:14:25.723275+0000","last_clean":"2023-10-06T05:14:25.723275+0000","last_became_active":"2023-10-06T05:14:25.719347+0000","last_became_peered":"2023-10-06T05:14:25.719347+0000","last_unstale":"2023-10-06T05:14:25.723275+0000","last_undegraded":"2023-10-06T05:14:25.723275+0000","last_fullsized":"2023-10-06T05:14:25.723275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716142+0000","last_change":"2023-10-06T05:14:24.722842+0000","last_active":"2023-10-06T05:14:25.716142+0000","last_peered":"2023-10-06T05:14:25.716142+0000","last_clean":"2023-10-06T05:14:25.716142+0000","last_became_active":"2023-10-06T05:14:24.721323+0000","last_became_peered":"2023-10-06T05:14:24.721323+0000","last_unstale":"2023-10-06T05:14:25.716142+0000","last_undegraded":"2023-10-06T05:14:25.716142+0000","last_fullsized":"2023-10-06T05:14:25.716142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735759+0000","last_change":"2023-10-06T05:14:25.728303+0000","last_active":"2023-10-06T05:14:25.735759+0000","last_peered":"2023-10-06T05:14:25.735759+0000","last_clean":"2023-10-06T05:14:25.735759+0000","last_became_active":"2023-10-06T05:14:25.727775+0000","last_became_peered":"2023-10-06T05:14:25.727775+0000","last_unstale":"2023-10-06T05:14:25.735759+0000","last_undegraded":"2023-10-06T05:14:25.735759+0000","last_fullsized":"2023-10-06T05:14:25.735759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738654+0000","last_change":"2023-10-06T05:14:24.728587+0000","last_active":"2023-10-06T05:14:25.738654+0000","last_peered":"2023-10-06T05:14:25.738654+0000","last_clean":"2023-10-06T05:14:25.738654+0000","last_became_active":"2023-10-06T05:14:24.728292+0000","last_became_peered":"2023-10-06T05:14:24.728292+0000","last_unstale":"2023-10-06T05:14:25.738654+0000","last_undegraded":"2023-10-06T05:14:25.738654+0000","last_fullsized":"2023-10-06T05:14:25.738654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:11.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735121+0000","last_change":"2023-10-06T05:14:25.732130+0000","last_active":"2023-10-06T05:14:25.735121+0000","last_peered":"2023-10-06T05:14:25.735121+0000","last_clean":"2023-10-06T05:14:25.735121+0000","last_became_active":"2023-10-06T05:14:25.731459+0000","last_became_peered":"2023-10-06T05:14:25.731459+0000","last_unstale":"2023-10-06T05:14:25.735121+0000","last_undegraded":"2023-10-06T05:14:25.735121+0000","last_fullsized":"2023-10-06T05:14:25.735121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741850+0000","last_change":"2023-10-06T05:14:24.728538+0000","last_active":"2023-10-06T05:14:25.741850+0000","last_peered":"2023-10-06T05:14:25.741850+0000","last_clean":"2023-10-06T05:14:25.741850+0000","last_became_active":"2023-10-06T05:14:24.728200+0000","last_became_peered":"2023-10-06T05:14:24.728200+0000","last_unstale":"2023-10-06T05:14:25.741850+0000","last_undegraded":"2023-10-06T05:14:25.741850+0000","last_fullsized":"2023-10-06T05:14:25.741850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740582+0000","last_change":"2023-10-06T05:14:25.740582+0000","last_active":"2023-10-06T05:14:25.740582+0000","last_peered":"2023-10-06T05:14:25.740582+0000","last_clean":"2023-10-06T05:14:25.740582+0000","last_became_active":"2023-10-06T05:14:25.740298+0000","last_became_peered":"2023-10-06T05:14:25.740298+0000","last_unstale":"2023-10-06T05:14:25.740582+0000","last_undegraded":"2023-10-06T05:14:25.740582+0000","last_fullsized":"2023-10-06T05:14:25.740582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:31.489344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741847+0000","last_change":"2023-10-06T05:14:24.735437+0000","last_active":"2023-10-06T05:14:25.741847+0000","last_peered":"2023-10-06T05:14:25.741847+0000","last_clean":"2023-10-06T05:14:25.741847+0000","last_became_active":"2023-10-06T05:14:24.735168+0000","last_became_peered":"2023-10-06T05:14:24.735168+0000","last_unstale":"2023-10-06T05:14:25.741847+0000","last_undegraded":"2023-10-06T05:14:25.741847+0000","last_fullsized":"2023-10-06T05:14:25.741847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:06.220991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723244+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.723244+0000","last_peered":"2023-10-06T05:14:25.723244+0000","last_clean":"2023-10-06T05:14:25.723244+0000","last_became_active":"2023-10-06T05:14:25.719474+0000","last_became_peered":"2023-10-06T05:14:25.719474+0000","last_unstale":"2023-10-06T05:14:25.723244+0000","last_undegraded":"2023-10-06T05:14:25.723244+0000","last_fullsized":"2023-10-06T05:14:25.723244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.724763+0000","last_change":"2023-10-06T05:14:24.718646+0000","last_active":"2023-10-06T05:14:24.724763+0000","last_peered":"2023-10-06T05:14:24.724763+0000","last_clean":"2023-10-06T05:14:24.724763+0000","last_became_active":"2023-10-06T05:14:24.718324+0000","last_became_peered":"2023-10-06T05:14:24.718324+0000","last_unstale":"2023-10-06T05:14:24.724763+0000","last_undegraded":"2023-10-06T05:14:24.724763+0000","last_fullsized":"2023-10-06T05:14:24.724763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727199+0000","last_change":"2023-10-06T05:14:25.724883+0000","last_active":"2023-10-06T05:14:25.727199+0000","last_peered":"2023-10-06T05:14:25.727199+0000","last_clean":"2023-10-06T05:14:25.727199+0000","last_became_active":"2023-10-06T05:14:25.724633+0000","last_became_peered":"2023-10-06T05:14:25.724633+0000","last_unstale":"2023-10-06T05:14:25.727199+0000","last_undegraded":"2023-10-06T05:14:25.727199+0000","last_fullsized":"2023-10-06T05:14:25.727199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.740512+0000","last_change":"2023-10-06T05:14:24.736333+0000","last_active":"2023-10-06T05:14:24.740512+0000","last_peered":"2023-10-06T05:14:24.740512+0000","last_clean":"2023-10-06T05:14:24.740512+0000","last_became_active":"2023-10-06T05:14:24.736038+0000","last_became_peered":"2023-10-06T05:14:24.736038+0000","last_unstale":"2023-10-06T05:14:24.740512+0000","last_undegraded":"2023-10-06T05:14:24.740512+0000","last_fullsized":"2023-10-06T05:14:24.740512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725666+0000","last_change":"2023-10-06T05:14:25.725666+0000","last_active":"2023-10-06T05:14:25.725666+0000","last_peered":"2023-10-06T05:14:25.725666+0000","last_clean":"2023-10-06T05:14:25.725666+0000","last_became_active":"2023-10-06T05:14:25.725319+0000","last_became_peered":"2023-10-06T05:14:25.725319+0000","last_unstale":"2023-10-06T05:14:25.725666+0000","last_undegraded":"2023-10-06T05:14:25.725666+0000","last_fullsized":"2023-10-06T05:14:25.725666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733857+0000","last_change":"2023-10-06T05:14:24.721548+0000","last_active":"2023-10-06T05:14:25.733857+0000","last_peered":"2023-10-06T05:14:25.733857+0000","last_clean":"2023-10-06T05:14:25.733857+0000","last_became_active":"2023-10-06T05:14:24.721266+0000","last_became_peered":"2023-10-06T05:14:24.721266+0000","last_unstale":"2023-10-06T05:14:25.733857+0000","last_undegraded":"2023-10-06T05:14:25.733857+0000","last_fullsized":"2023-10-06T05:14:25.733857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727201+0000","last_change":"2023-10-06T05:14:25.725874+0000","last_active":"2023-10-06T05:14:25.727201+0000","last_peered":"2023-10-06T05:14:25.727201+0000","last_clean":"2023-10-06T05:14:25.727201+0000","last_became_active":"2023-10-06T05:14:25.725494+0000","last_became_peered":"2023-10-06T05:14:25.725494+0000","last_unstale":"2023-10-06T05:14:25.727201+0000","last_undegraded":"2023-10-06T05:14:25.727201+0000","last_fullsized":"2023-10-06T05:14:25.727201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719074+0000","last_change":"2023-10-06T05:14:24.733795+0000","last_active":"2023-10-06T05:14:25.719074+0000","last_peered":"2023-10-06T05:14:25.719074+0000","last_clean":"2023-10-06T05:14:25.719074+0000","last_became_active":"2023-10-06T05:14:24.733567+0000","last_became_peered":"2023-10-06T05:14:24.733567+0000","last_unstale":"2023-10-06T05:14:25.719074+0000","last_undegraded":"2023-10-06T05:14:25.719074+0000","last_fullsized":"2023-10-06T05:14:25.719074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732255+0000","last_change":"2023-10-06T05:14:25.727738+0000","last_active":"2023-10-06T05:14:25.732255+0000","last_peered":"2023-10-06T05:14:25.732255+0000","last_clean":"2023-10-06T05:14:25.732255+0000","last_became_active":"2023-10-06T05:14:25.727474+0000","last_became_peered":"2023-10-06T05:14:25.727474+0000","last_unstale":"2023-10-06T05:14:25.732255+0000","last_undegraded":"2023-10-06T05:14:25.732255+0000","last_fullsized":"2023-10-06T05:14:25.732255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742236+0000","last_change":"2023-10-06T05:14:24.733458+0000","last_active":"2023-10-06T05:14:25.742236+0000","last_peered":"2023-10-06T05:14:25.742236+0000","last_clean":"2023-10-06T05:14:25.742236+0000","last_became_active":"2023-10-06T05:14:24.733160+0000","last_became_peered":"2023-10-06T05:14:24.733160+0000","last_unstale":"2023-10-06T05:14:25.742236+0000","last_undegraded":"2023-10-06T05:14:25.742236+0000","last_fullsized":"2023-10-06T05:14:25.742236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741624+0000","last_change":"2023-10-06T05:14:25.741624+0000","last_active":"2023-10-06T05:14:25.741624+0000","last_peered":"2023-10-06T05:14:25.741624+0000","last_clean":"2023-10-06T05:14:25.741624+0000","last_became_active":"2023-10-06T05:14:25.740252+0000","last_became_peered":"2023-10-06T05:14:25.740252+0000","last_unstale":"2023-10-06T05:14:25.741624+0000","last_undegraded":"2023-10-06T05:14:25.741624+0000","last_fullsized":"2023-10-06T05:14:25.741624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723475+0000","last_change":"2023-10-06T05:14:24.721437+0000","last_active":"2023-10-06T05:14:25.723475+0000","last_peered":"2023-10-06T05:14:25.723475+0000","last_clean":"2023-10-06T05:14:25.723475+0000","last_became_active":"2023-10-06T05:14:24.721172+0000","last_became_peered":"2023-10-06T05:14:24.721172+0000","last_unstale":"2023-10-06T05:14:25.723475+0000","last_undegraded":"2023-10-06T05:14:25.723475+0000","last_fullsized":"2023-10-06T05:14:25.723475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:22.852582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727675+0000","last_change":"2023-10-06T05:14:25.719600+0000","last_active":"2023-10-06T05:14:25.727675+0000","last_peered":"2023-10-06T05:14:25.727675+0000","last_clean":"2023-10-06T05:14:25.727675+0000","last_became_active":"2023-10-06T05:14:25.719349+0000","last_became_peered":"2023-10-06T05:14:25.719349+0000","last_unstale":"2023-10-06T05:14:25.727675+0000","last_undegraded":"2023-10-06T05:14:25.727675+0000","last_fullsized":"2023-10-06T05:14:25.727675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716852+0000","last_change":"2023-10-06T05:14:24.728714+0000","last_active":"2023-10-06T05:14:25.716852+0000","last_peered":"2023-10-06T05:14:25.716852+0000","last_clean":"2023-10-06T05:14:25.716852+0000","last_became_active":"2023-10-06T05:14:24.728435+0000","last_became_peered":"2023-10-06T05:14:24.728435+0000","last_unstale":"2023-10-06T05:14:25.716852+0000","last_undegraded":"2023-10-06T05:14:25.716852+0000","last_fullsized":"2023-10-06T05:14:25.716852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.724993+0000","last_change":"2023-10-06T05:14:25.724993+0000","last_active":"2023-10-06T05:14:25.724993+0000","last_peered":"2023-10-06T05:14:25.724993+0000","last_clean":"2023-10-06T05:14:25.724993+0000","last_became_active":"2023-10-06T05:14:25.724538+0000","last_became_peered":"2023-10-06T05:14:25.724538+0000","last_unstale":"2023-10-06T05:14:25.724993+0000","last_undegraded":"2023-10-06T05:14:25.724993+0000","last_fullsized":"2023-10-06T05:14:25.724993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738370+0000","last_change":"2023-10-06T05:14:24.723321+0000","last_active":"2023-10-06T05:14:25.738370+0000","last_peered":"2023-10-06T05:14:25.738370+0000","last_clean":"2023-10-06T05:14:25.738370+0000","last_became_active":"2023-10-06T05:14:24.723063+0000","last_became_peered":"2023-10-06T05:14:24.723063+0000","last_unstale":"2023-10-06T05:14:25.738370+0000","last_undegraded":"2023-10-06T05:14:25.738370+0000","last_fullsized":"2023-10-06T05:14:25.738370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740648+0000","last_change":"2023-10-06T05:14:25.740648+0000","last_active":"2023-10-06T05:14:25.740648+0000","last_peered":"2023-10-06T05:14:25.740648+0000","last_clean":"2023-10-06T05:14:25.740648+0000","last_became_active":"2023-10-06T05:14:25.740351+0000","last_became_peered":"2023-10-06T05:14:25.740351+0000","last_unstale":"2023-10-06T05:14:25.740648+0000","last_undegraded":"2023-10-06T05:14:25.740648+0000","last_fullsized":"2023-10-06T05:14:25.740648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709955+0000","last_change":"2023-10-06T05:14:24.730263+0000","last_active":"2023-10-06T05:14:25.709955+0000","last_peered":"2023-10-06T05:14:25.709955+0000","last_clean":"2023-10-06T05:14:25.709955+0000","last_became_active":"2023-10-06T05:14:24.729911+0000","last_became_peered":"2023-10-06T05:14:24.729911+0000","last_unstale":"2023-10-06T05:14:25.709955+0000","last_undegraded":"2023-10-06T05:14:25.709955+0000","last_fullsized":"2023-10-06T05:14:25.709955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744620+0000","last_change":"2023-10-06T05:14:25.744620+0000","last_active":"2023-10-06T05:14:25.744620+0000","last_peered":"2023-10-06T05:14:25.744620+0000","last_clean":"2023-10-06T05:14:25.744620+0000","last_became_active":"2023-10-06T05:14:25.744323+0000","last_became_peered":"2023-10-06T05:14:25.744323+0000","last_unstale":"2023-10-06T05:14:25.744620+0000","last_undegraded":"2023-10-06T05:14:25.744620+0000","last_fullsized":"2023-10-06T05:14:25.744620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:39.850675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.735136+0000","last_change":"2023-10-06T05:14:24.731862+0000","last_active":"2023-10-06T05:14:24.735136+0000","last_peered":"2023-10-06T05:14:24.735136+0000","last_clean":"2023-10-06T05:14:24.735136+0000","last_became_active":"2023-10-06T05:14:24.729629+0000","last_became_peered":"2023-10-06T05:14:24.729629+0000","last_unstale":"2023-10-06T05:14:24.735136+0000","last_undegraded":"2023-10-06T05:14:24.735136+0000","last_fullsized":"2023-10-06T05:14:24.735136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742674+0000","last_change":"2023-10-06T05:14:25.742674+0000","last_active":"2023-10-06T05:14:25.742674+0000","last_peered":"2023-10-06T05:14:25.742674+0000","last_clean":"2023-10-06T05:14:25.742674+0000","last_became_active":"2023-10-06T05:14:25.741308+0000","last_became_peered":"2023-10-06T05:14:25.741308+0000","last_unstale":"2023-10-06T05:14:25.742674+0000","last_undegraded":"2023-10-06T05:14:25.742674+0000","last_fullsized":"2023-10-06T05:14:25.742674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:29.811666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708350+0000","last_change":"2023-10-06T05:14:24.724203+0000","last_active":"2023-10-06T05:14:25.708350+0000","last_peered":"2023-10-06T05:14:25.708350+0000","last_clean":"2023-10-06T05:14:25.708350+0000","last_became_active":"2023-10-06T05:14:24.723929+0000","last_became_peered":"2023-10-06T05:14:24.723929+0000","last_unstale":"2023-10-06T05:14:25.708350+0000","last_undegraded":"2023-10-06T05:14:25.708350+0000","last_fullsized":"2023-10-06T05:14:25.708350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735025+0000","last_change":"2023-10-06T05:14:25.726645+0000","last_active":"2023-10-06T05:14:25.735025+0000","last_peered":"2023-10-06T05:14:25.735025+0000","last_clean":"2023-10-06T05:14:25.735025+0000","last_became_active":"2023-10-06T05:14:25.724695+0000","last_became_peered":"2023-10-06T05:14:25.724695+0000","last_unstale":"2023-10-06T05:14:25.735025+0000","last_undegraded":"2023-10-06T05:14:25.735025+0000","last_fullsized":"2023-10-06T05:14:25.735025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713550+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:14:25.713550+0000","last_peered":"2023-10-06T05:14:25.713550+0000","last_clean":"2023-10-06T05:14:25.713550+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:14:25.713550+0000","last_undegraded":"2023-10-06T05:14:25.713550+0000","last_fullsized":"2023-10-06T05:14:25.713550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716529+0000","last_change":"2023-10-06T05:14:24.728757+0000","last_active":"2023-10-06T05:14:25.716529+0000","last_peered":"2023-10-06T05:14:25.716529+0000","last_clean":"2023-10-06T05:14:25.716529+0000","last_became_active":"2023-10-06T05:14:24.728480+0000","last_became_peered":"2023-10-06T05:14:24.728480+0000","last_unstale":"2023-10-06T05:14:25.716529+0000","last_undegraded":"2023-10-06T05:14:25.716529+0000","last_fullsized":"2023-10-06T05:14:25.716529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727990+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.727990+0000","last_peered":"2023-10-06T05:14:25.727990+0000","last_clean":"2023-10-06T05:14:25.727990+0000","last_became_active":"2023-10-06T05:14:25.719491+0000","last_became_peered":"2023-10-06T05:14:25.719491+0000","last_unstale":"2023-10-06T05:14:25.727990+0000","last_undegraded":"2023-10-06T05:14:25.727990+0000","last_fullsized":"2023-10-06T05:14:25.727990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715616+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:14:25.715616+0000","last_peered":"2023-10-06T05:14:25.715616+0000","last_clean":"2023-10-06T05:14:25.715616+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:14:25.715616+0000","last_undegraded":"2023-10-06T05:14:25.715616+0000","last_fullsized":"2023-10-06T05:14:25.715616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.737577+0000","last_change":"2023-10-06T05:14:24.732737+0000","last_active":"2023-10-06T05:14:24.737577+0000","last_peered":"2023-10-06T05:14:24.737577+0000","last_clean":"2023-10-06T05:14:24.737577+0000","last_became_active":"2023-10-06T05:14:24.732410+0000","last_became_peered":"2023-10-06T05:14:24.732410+0000","last_unstale":"2023-10-06T05:14:24.737577+0000","last_undegraded":"2023-10-06T05:14:24.737577+0000","last_fullsized":"2023-10-06T05:14:24.737577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736090+0000","last_change":"2023-10-06T05:14:25.723180+0000","last_active":"2023-10-06T05:14:25.736090+0000","last_peered":"2023-10-06T05:14:25.736090+0000","last_clean":"2023-10-06T05:14:25.736090+0000","last_became_active":"2023-10-06T05:14:25.722867+0000","last_became_peered":"2023-10-06T05:14:25.722867+0000","last_unstale":"2023-10-06T05:14:25.736090+0000","last_undegraded":"2023-10-06T05:14:25.736090+0000","last_fullsized":"2023-10-06T05:14:25.736090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.718276+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:14:24.718276+0000","last_peered":"2023-10-06T05:14:24.718276+0000","last_clean":"2023-10-06T05:14:24.718276+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:14:24.718276+0000","last_undegraded":"2023-10-06T05:14:24.718276+0000","last_fullsized":"2023-10-06T05:14:24.718276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713983+0000","last_change":"2023-10-06T05:14:24.719784+0000","last_active":"2023-10-06T05:14:25.713983+0000","last_peered":"2023-10-06T05:14:25.713983+0000","last_clean":"2023-10-06T05:14:25.713983+0000","last_became_active":"2023-10-06T05:14:24.719414+0000","last_became_peered":"2023-10-06T05:14:24.719414+0000","last_unstale":"2023-10-06T05:14:25.713983+0000","last_undegraded":"2023-10-06T05:14:25.713983+0000","last_fullsized":"2023-10-06T05:14:25.713983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735002+0000","last_change":"2023-10-06T05:14:25.725673+0000","last_active":"2023-10-06T05:14:25.735002+0000","last_peered":"2023-10-06T05:14:25.735002+0000","last_clean":"2023-10-06T05:14:25.735002+0000","last_became_active":"2023-10-06T05:14:25.725107+0000","last_became_peered":"2023-10-06T05:14:25.725107+0000","last_unstale":"2023-10-06T05:14:25.735002+0000","last_undegraded":"2023-10-06T05:14:25.735002+0000","last_fullsized":"2023-10-06T05:14:25.735002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:35.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715779+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:14:25.715779+0000","last_peered":"2023-10-06T05:14:25.715779+0000","last_clean":"2023-10-06T05:14:25.715779+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:14:25.715779+0000","last_undegraded":"2023-10-06T05:14:25.715779+0000","last_fullsized":"2023-10-06T05:14:25.715779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.710679+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:14:24.710679+0000","last_peered":"2023-10-06T05:14:24.710679+0000","last_clean":"2023-10-06T05:14:24.710679+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:14:24.710679+0000","last_undegraded":"2023-10-06T05:14:24.710679+0000","last_fullsized":"2023-10-06T05:14:24.710679+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708071+0000","last_change":"2023-10-06T05:14:24.717406+0000","last_active":"2023-10-06T05:14:25.708071+0000","last_peered":"2023-10-06T05:14:25.708071+0000","last_clean":"2023-10-06T05:14:25.708071+0000","last_became_active":"2023-10-06T05:14:24.716723+0000","last_became_peered":"2023-10-06T05:14:24.716723+0000","last_unstale":"2023-10-06T05:14:25.708071+0000","last_undegraded":"2023-10-06T05:14:25.708071+0000","last_fullsized":"2023-10-06T05:14:25.708071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.746536+0000","last_change":"2023-10-06T05:14:25.745372+0000","last_active":"2023-10-06T05:14:25.746536+0000","last_peered":"2023-10-06T05:14:25.746536+0000","last_clean":"2023-10-06T05:14:25.746536+0000","last_became_active":"2023-10-06T05:14:25.745163+0000","last_became_peered":"2023-10-06T05:14:25.745163+0000","last_unstale":"2023-10-06T05:14:25.746536+0000","last_undegraded":"2023-10-06T05:14:25.746536+0000","last_fullsized":"2023-10-06T05:14:25.746536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.860941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.710871+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:14:24.710871+0000","last_peered":"2023-10-06T05:14:24.710871+0000","last_clean":"2023-10-06T05:14:24.710871+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:14:24.710871+0000","last_undegraded":"2023-10-06T05:14:24.710871+0000","last_fullsized":"2023-10-06T05:14:24.710871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.737469+0000","last_change":"2023-10-06T05:14:24.736663+0000","last_active":"2023-10-06T05:14:24.737469+0000","last_peered":"2023-10-06T05:14:24.737469+0000","last_clean":"2023-10-06T05:14:24.737469+0000","last_became_active":"2023-10-06T05:14:24.736241+0000","last_became_peered":"2023-10-06T05:14:24.736241+0000","last_unstale":"2023-10-06T05:14:24.737469+0000","last_undegraded":"2023-10-06T05:14:24.737469+0000","last_fullsized":"2023-10-06T05:14:24.737469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.729949+0000","last_change":"2023-10-06T05:14:25.729949+0000","last_active":"2023-10-06T05:14:25.729949+0000","last_peered":"2023-10-06T05:14:25.729949+0000","last_clean":"2023-10-06T05:14:25.729949+0000","last_became_active":"2023-10-06T05:14:25.729600+0000","last_became_peered":"2023-10-06T05:14:25.729600+0000","last_unstale":"2023-10-06T05:14:25.729949+0000","last_undegraded":"2023-10-06T05:14:25.729949+0000","last_fullsized":"2023-10-06T05:14:25.729949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.711786+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:14:24.711786+0000","last_peered":"2023-10-06T05:14:24.711786+0000","last_clean":"2023-10-06T05:14:24.711786+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:14:24.711786+0000","last_undegraded":"2023-10-06T05:14:24.711786+0000","last_fullsized":"2023-10-06T05:14:24.711786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713950+0000","last_change":"2023-10-06T05:14:24.724312+0000","last_active":"2023-10-06T05:14:25.713950+0000","last_peered":"2023-10-06T05:14:25.713950+0000","last_clean":"2023-10-06T05:14:25.713950+0000","last_became_active":"2023-10-06T05:14:24.724011+0000","last_became_peered":"2023-10-06T05:14:24.724011+0000","last_unstale":"2023-10-06T05:14:25.713950+0000","last_undegraded":"2023-10-06T05:14:25.713950+0000","last_fullsized":"2023-10-06T05:14:25.713950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.729128+0000","last_change":"2023-10-06T05:14:25.729128+0000","last_active":"2023-10-06T05:14:25.729128+0000","last_peered":"2023-10-06T05:14:25.729128+0000","last_clean":"2023-10-06T05:14:25.729128+0000","last_became_active":"2023-10-06T05:14:25.728715+0000","last_became_peered":"2023-10-06T05:14:25.728715+0000","last_unstale":"2023-10-06T05:14:25.729128+0000","last_undegraded":"2023-10-06T05:14:25.729128+0000","last_fullsized":"2023-10-06T05:14:25.729128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727798+0000","last_change":"2023-10-06T05:14:24.721869+0000","last_active":"2023-10-06T05:14:25.727798+0000","last_peered":"2023-10-06T05:14:25.727798+0000","last_clean":"2023-10-06T05:14:25.727798+0000","last_became_active":"2023-10-06T05:14:24.719902+0000","last_became_peered":"2023-10-06T05:14:24.719902+0000","last_unstale":"2023-10-06T05:14:25.727798+0000","last_undegraded":"2023-10-06T05:14:25.727798+0000","last_fullsized":"2023-10-06T05:14:25.727798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713213+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:14:25.713213+0000","last_peered":"2023-10-06T05:14:25.713213+0000","last_clean":"2023-10-06T05:14:25.713213+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:14:25.713213+0000","last_undegraded":"2023-10-06T05:14:25.713213+0000","last_fullsized":"2023-10-06T05:14:25.713213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735685+0000","last_change":"2023-10-06T05:14:25.727038+0000","last_active":"2023-10-06T05:14:25.735685+0000","last_peered":"2023-10-06T05:14:25.735685+0000","last_clean":"2023-10-06T05:14:25.735685+0000","last_became_active":"2023-10-06T05:14:25.726717+0000","last_became_peered":"2023-10-06T05:14:25.726717+0000","last_unstale":"2023-10-06T05:14:25.735685+0000","last_undegraded":"2023-10-06T05:14:25.735685+0000","last_fullsized":"2023-10-06T05:14:25.735685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709571+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:14:25.709571+0000","last_peered":"2023-10-06T05:14:25.709571+0000","last_clean":"2023-10-06T05:14:25.709571+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:14:25.709571+0000","last_undegraded":"2023-10-06T05:14:25.709571+0000","last_fullsized":"2023-10-06T05:14:25.709571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.724706+0000","last_change":"2023-10-06T05:14:24.717582+0000","last_active":"2023-10-06T05:14:24.724706+0000","last_peered":"2023-10-06T05:14:24.724706+0000","last_clean":"2023-10-06T05:14:24.724706+0000","last_became_active":"2023-10-06T05:14:24.716871+0000","last_became_peered":"2023-10-06T05:14:24.716871+0000","last_unstale":"2023-10-06T05:14:24.724706+0000","last_undegraded":"2023-10-06T05:14:24.724706+0000","last_fullsized":"2023-10-06T05:14:24.724706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737874+0000","last_change":"2023-10-06T05:14:25.737874+0000","last_active":"2023-10-06T05:14:25.737874+0000","last_peered":"2023-10-06T05:14:25.737874+0000","last_clean":"2023-10-06T05:14:25.737874+0000","last_became_active":"2023-10-06T05:14:25.737569+0000","last_became_peered":"2023-10-06T05:14:25.737569+0000","last_unstale":"2023-10-06T05:14:25.737874+0000","last_undegraded":"2023-10-06T05:14:25.737874+0000","last_fullsized":"2023-10-06T05:14:25.737874+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716149+0000","last_change":"2023-10-06T05:14:24.736575+0000","last_active":"2023-10-06T05:14:25.716149+0000","last_peered":"2023-10-06T05:14:25.716149+0000","last_clean":"2023-10-06T05:14:25.716149+0000","last_became_active":"2023-10-06T05:14:24.736221+0000","last_became_peered":"2023-10-06T05:14:24.736221+0000","last_unstale":"2023-10-06T05:14:25.716149+0000","last_undegraded":"2023-10-06T05:14:25.716149+0000","last_fullsized":"2023-10-06T05:14:25.716149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719862+0000","last_change":"2023-10-06T05:14:25.719862+0000","last_active":"2023-10-06T05:14:25.719862+0000","last_peered":"2023-10-06T05:14:25.719862+0000","last_clean":"2023-10-06T05:14:25.719862+0000","last_became_active":"2023-10-06T05:14:25.719558+0000","last_became_peered":"2023-10-06T05:14:25.719558+0000","last_unstale":"2023-10-06T05:14:25.719862+0000","last_undegraded":"2023-10-06T05:14:25.719862+0000","last_fullsized":"2023-10-06T05:14:25.719862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:41.453897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742798+0000","last_change":"2023-10-06T05:14:24.734397+0000","last_active":"2023-10-06T05:14:25.742798+0000","last_peered":"2023-10-06T05:14:25.742798+0000","last_clean":"2023-10-06T05:14:25.742798+0000","last_became_active":"2023-10-06T05:14:24.734112+0000","last_became_peered":"2023-10-06T05:14:24.734112+0000","last_unstale":"2023-10-06T05:14:25.742798+0000","last_undegraded":"2023-10-06T05:14:25.742798+0000","last_fullsized":"2023-10-06T05:14:25.742798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:50.734968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736354+0000","last_change":"2023-10-06T05:14:25.736354+0000","last_active":"2023-10-06T05:14:25.736354+0000","last_peered":"2023-10-06T05:14:25.736354+0000","last_clean":"2023-10-06T05:14:25.736354+0000","last_became_active":"2023-10-06T05:14:25.735695+0000","last_became_peered":"2023-10-06T05:14:25.735695+0000","last_unstale":"2023-10-06T05:14:25.736354+0000","last_undegraded":"2023-10-06T05:14:25.736354+0000","last_fullsized":"2023-10-06T05:14:25.736354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738237+0000","last_change":"2023-10-06T05:14:24.733470+0000","last_active":"2023-10-06T05:14:25.738237+0000","last_peered":"2023-10-06T05:14:25.738237+0000","last_clean":"2023-10-06T05:14:25.738237+0000","last_became_active":"2023-10-06T05:14:24.733189+0000","last_became_peered":"2023-10-06T05:14:24.733189+0000","last_unstale":"2023-10-06T05:14:25.738237+0000","last_undegraded":"2023-10-06T05:14:25.738237+0000","last_fullsized":"2023-10-06T05:14:25.738237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744257+0000","last_change":"2023-10-06T05:14:25.744257+0000","last_active":"2023-10-06T05:14:25.744257+0000","last_peered":"2023-10-06T05:14:25.744257+0000","last_clean":"2023-10-06T05:14:25.744257+0000","last_became_active":"2023-10-06T05:14:25.743800+0000","last_became_peered":"2023-10-06T05:14:25.743800+0000","last_unstale":"2023-10-06T05:14:25.744257+0000","last_undegraded":"2023-10-06T05:14:25.744257+0000","last_fullsized":"2023-10-06T05:14:25.744257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737149+0000","last_change":"2023-10-06T05:14:24.719523+0000","last_active":"2023-10-06T05:14:25.737149+0000","last_peered":"2023-10-06T05:14:25.737149+0000","last_clean":"2023-10-06T05:14:25.737149+0000","last_became_active":"2023-10-06T05:14:24.717215+0000","last_became_peered":"2023-10-06T05:14:24.717215+0000","last_unstale":"2023-10-06T05:14:25.737149+0000","last_undegraded":"2023-10-06T05:14:25.737149+0000","last_fullsized":"2023-10-06T05:14:25.737149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733084+0000","last_change":"2023-10-06T05:14:25.726409+0000","last_active":"2023-10-06T05:14:25.733084+0000","last_peered":"2023-10-06T05:14:25.733084+0000","last_clean":"2023-10-06T05:14:25.733084+0000","last_became_active":"2023-10-06T05:14:25.725760+0000","last_became_peered":"2023-10-06T05:14:25.725760+0000","last_unstale":"2023-10-06T05:14:25.733084+0000","last_undegraded":"2023-10-06T05:14:25.733084+0000","last_fullsized":"2023-10-06T05:14:25.733084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730378+0000","last_change":"2023-10-06T05:14:24.724643+0000","last_active":"2023-10-06T05:14:25.730378+0000","last_peered":"2023-10-06T05:14:25.730378+0000","last_clean":"2023-10-06T05:14:25.730378+0000","last_became_active":"2023-10-06T05:14:24.724364+0000","last_became_peered":"2023-10-06T05:14:24.724364+0000","last_unstale":"2023-10-06T05:14:25.730378+0000","last_undegraded":"2023-10-06T05:14:25.730378+0000","last_fullsized":"2023-10-06T05:14:25.730378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741519+0000","last_change":"2023-10-06T05:14:25.741519+0000","last_active":"2023-10-06T05:14:25.741519+0000","last_peered":"2023-10-06T05:14:25.741519+0000","last_clean":"2023-10-06T05:14:25.741519+0000","last_became_active":"2023-10-06T05:14:25.741100+0000","last_became_peered":"2023-10-06T05:14:25.741100+0000","last_unstale":"2023-10-06T05:14:25.741519+0000","last_undegraded":"2023-10-06T05:14:25.741519+0000","last_fullsized":"2023-10-06T05:14:25.741519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728144+0000","last_change":"2023-10-06T05:14:25.721386+0000","last_active":"2023-10-06T05:14:25.728144+0000","last_peered":"2023-10-06T05:14:25.728144+0000","last_clean":"2023-10-06T05:14:25.728144+0000","last_became_active":"2023-10-06T05:14:25.720975+0000","last_became_peered":"2023-10-06T05:14:25.720975+0000","last_unstale":"2023-10-06T05:14:25.728144+0000","last_undegraded":"2023-10-06T05:14:25.728144+0000","last_fullsized":"2023-10-06T05:14:25.728144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:51.743327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733104+0000","last_change":"2023-10-06T05:14:25.726604+0000","last_active":"2023-10-06T05:14:25.733104+0000","last_peered":"2023-10-06T05:14:25.733104+0000","last_clean":"2023-10-06T05:14:25.733104+0000","last_became_active":"2023-10-06T05:14:25.726075+0000","last_became_peered":"2023-10-06T05:14:25.726075+0000","last_unstale":"2023-10-06T05:14:25.733104+0000","last_undegraded":"2023-10-06T05:14:25.733104+0000","last_fullsized":"2023-10-06T05:14:25.733104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743617+0000","last_change":"2023-10-06T05:14:25.743617+0000","last_active":"2023-10-06T05:14:25.743617+0000","last_peered":"2023-10-06T05:14:25.743617+0000","last_clean":"2023-10-06T05:14:25.743617+0000","last_became_active":"2023-10-06T05:14:25.741960+0000","last_became_peered":"2023-10-06T05:14:25.741960+0000","last_unstale":"2023-10-06T05:14:25.743617+0000","last_undegraded":"2023-10-06T05:14:25.743617+0000","last_fullsized":"2023-10-06T05:14:25.743617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:50.792266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735674+0000","last_change":"2023-10-06T05:14:25.728085+0000","last_active":"2023-10-06T05:14:25.735674+0000","last_peered":"2023-10-06T05:14:25.735674+0000","last_clean":"2023-10-06T05:14:25.735674+0000","last_became_active":"2023-10-06T05:14:25.727721+0000","last_became_peered":"2023-10-06T05:14:25.727721+0000","last_unstale":"2023-10-06T05:14:25.735674+0000","last_undegraded":"2023-10-06T05:14:25.735674+0000","last_fullsized":"2023-10-06T05:14:25.735674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739584+0000","last_change":"2023-10-06T05:14:25.735959+0000","last_active":"2023-10-06T05:14:25.739584+0000","last_peered":"2023-10-06T05:14:25.739584+0000","last_clean":"2023-10-06T05:14:25.739584+0000","last_became_active":"2023-10-06T05:14:25.735602+0000","last_became_peered":"2023-10-06T05:14:25.735602+0000","last_unstale":"2023-10-06T05:14:25.739584+0000","last_undegraded":"2023-10-06T05:14:25.739584+0000","last_fullsized":"2023-10-06T05:14:25.739584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":674798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":827392,"data_stored":673600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":82768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":84270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":83302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:14:30.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:14:30.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.313+0000 7f1612d9d700 1 Processor -- start 2023-10-06T05:14:30.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.313+0000 7f1612d9d700 1 -- start start 2023-10-06T05:14:30.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.314+0000 7f1612d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f160c0b0a40 0x7f160c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.314+0000 7f1612d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b0e10 0x7f160c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.314+0000 7f1612d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f160c0b2e30 0x7f160c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.314+0000 7f1612d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160c15e2d0 con 0x7f160c0b0a40 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.314+0000 7f1612d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160c15d6c0 con 0x7f160c0b2e30 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.314+0000 7f1612d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c15d9a0 con 0x7f160c0b0e10 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f160c0b0a40 0x7f160c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f160c0b0a40 0x7f160c06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37178/0 (socket says 172.21.15.33:37178) 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 -- 172.21.15.33:0/222547594 learned_addr learned my addr 172.21.15.33:0/222547594 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f161159a700 1 --2- 172.21.15.33:0/222547594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b0e10 0x7f160c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f161259c700 1 --2- 172.21.15.33:0/222547594 >> v2:172.21.15.167:3300/0 conn(0x7f160c0b2e30 0x7f160c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 -- 172.21.15.33:0/222547594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b0e10 msgr2=0x7f160c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:30.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 --2- 172.21.15.33:0/222547594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b0e10 0x7f160c0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 -- 172.21.15.33:0/222547594 >> v2:172.21.15.167:3300/0 conn(0x7f160c0b2e30 msgr2=0x7f160c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 --2- 172.21.15.33:0/222547594 >> v2:172.21.15.167:3300/0 conn(0x7f160c0b2e30 0x7f160c15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 -- 172.21.15.33:0/222547594 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c15dc80 con 0x7f160c0b0a40 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.315+0000 7f1611d9b700 1 --2- 172.21.15.33:0/222547594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c06ab00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f160c0b32b0 tx=0x7f1608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786ed4c186ea4836 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1610d99700 1 -- 172.21.15.33:0/222547594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800b640 con 0x7f160c0b0a40 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1610d99700 1 -- 172.21.15.33:0/222547594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160800b7a0 con 0x7f160c0b0a40 2023-10-06T05:14:30.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1610d99700 1 -- 172.21.15.33:0/222547594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800bb60 con 0x7f160c0b0a40 2023-10-06T05:14:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 -- 172.21.15.33:0/222547594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 msgr2=0x7f160c06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 --2- 172.21.15.33:0/222547594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c06ab00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f160c0b32b0 tx=0x7f1608009250 comp rx=0 tx=0).stop 2023-10-06T05:14:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 -- 172.21.15.33:0/222547594 shutdown_connections 2023-10-06T05:14:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 --2- 172.21.15.33:0/222547594 >> v2:172.21.15.167:3300/0 conn(0x7f160c0b2e30 0x7f160c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 --2- 172.21.15.33:0/222547594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c06ab00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 --2- 172.21.15.33:0/222547594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b0e10 0x7f160c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 -- 172.21.15.33:0/222547594 >> 172.21.15.33:0/222547594 conn(0x7f160c06e0f0 msgr2=0x7f160c0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 -- 172.21.15.33:0/222547594 shutdown_connections 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.316+0000 7f1612d9d700 1 -- 172.21.15.33:0/222547594 wait complete. 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.318+0000 7f1612d9d700 1 Processor -- start 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.318+0000 7f1612d9d700 1 -- start start 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1612d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c16ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1612d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160c0b0e10 0x7f160c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1612d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b2e30 0x7f160c0aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1612d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c15c060 con 0x7f160c0b0a40 2023-10-06T05:14:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1612d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c15c340 con 0x7f160c0b0e10 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1612d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c15c620 con 0x7f160c0b2e30 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1611d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c16ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1611d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c16ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37186/0 (socket says 172.21.15.33:37186) 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f1611d9b700 1 -- 172.21.15.33:0/4106088003 learned_addr learned my addr 172.21.15.33:0/4106088003 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f161259c700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b2e30 0x7f160c0aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.319+0000 7f161159a700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160c0b0e10 0x7f160c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1611d9b700 1 -- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b2e30 msgr2=0x7f160c0aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1611d9b700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b2e30 0x7f160c0aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1611d9b700 1 -- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160c0b0e10 msgr2=0x7f160c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1611d9b700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160c0b0e10 0x7f160c15b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1611d9b700 1 -- 172.21.15.33:0/4106088003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16080094f0 con 0x7f160c0b0a40 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1611d9b700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c16ced0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f160c0ce290 tx=0x7f1608017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800b910 con 0x7f160c0b0a40 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160800ba70 con 0x7f160c0b0a40 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608011630 con 0x7f160c0b0a40 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c15c900 con 0x7f160c0b0a40 2023-10-06T05:14:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.320+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c154870 con 0x7f160c0b0a40 2023-10-06T05:14:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.321+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c06f810 con 0x7f160c0b0a40 2023-10-06T05:14:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.322+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16080119a0 con 0x7f160c0b0a40 2023-10-06T05:14:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.323+0000 7f1601ffb700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15f0066f30 0x7f15f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.323+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1608085910 con 0x7f160c0b0a40 2023-10-06T05:14:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.323+0000 7f161159a700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15f0066f30 0x7f15f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.324+0000 7f161159a700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15f0066f30 0x7f15f00693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009db0 tx=0x7f15fc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:30.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.327+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1608052ab0 con 0x7f160c0b0a40 2023-10-06T05:14:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.524+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f160c0cd000 con 0x7f15f0066f30 2023-10-06T05:14:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.532+0000 7f1601ffb700 1 -- 172.21.15.33:0/4106088003 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286679 (secure 0 0 0) 0x7f160c0cd000 con 0x7f15f0066f30 2023-10-06T05:14:30.538 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15f0066f30 msgr2=0x7f15f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15f0066f30 0x7f15f00693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009db0 tx=0x7f15fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 msgr2=0x7f160c16ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c16ced0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f160c0ce290 tx=0x7f1608017000 comp rx=0 tx=0).stop 2023-10-06T05:14:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 shutdown_connections 2023-10-06T05:14:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15f0066f30 0x7f15f00693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f160c0b2e30 0x7f160c0aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0b0a40 0x7f160c16ced0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 --2- 172.21.15.33:0/4106088003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160c0b0e10 0x7f160c15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 >> 172.21.15.33:0/4106088003 conn(0x7f160c06e0f0 msgr2=0x7f160c157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 shutdown_connections 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.536+0000 7f1612d9d700 1 -- 172.21.15.33:0/4106088003 wait complete. 2023-10-06T05:14:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:14:30.556 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T05:14:29.818477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381320,"kb_used_data":3016,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593400,"statfs":{"total":773094113280,"available":770655641600,"internally_reserved":0,"allocated":3088384,"data_stored":1846772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001574"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733340+0000","last_change":"2023-10-06T05:14:25.733340+0000","last_active":"2023-10-06T05:14:25.733340+0000","last_peered":"2023-10-06T05:14:25.733340+0000","last_clean":"2023-10-06T05:14:25.733340+0000","last_became_active":"2023-10-06T05:14:25.732861+0000","last_became_peered":"2023-10-06T05:14:25.732861+0000","last_unstale":"2023-10-06T05:14:25.733340+0000","last_undegraded":"2023-10-06T05:14:25.733340+0000","last_fullsized":"2023-10-06T05:14:25.733340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:19.965653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733215+0000","last_change":"2023-10-06T05:14:25.730206+0000","last_active":"2023-10-06T05:14:25.733215+0000","last_peered":"2023-10-06T05:14:25.733215+0000","last_clean":"2023-10-06T05:14:25.733215+0000","last_became_active":"2023-10-06T05:14:25.729947+0000","last_became_peered":"2023-10-06T05:14:25.729947+0000","last_unstale":"2023-10-06T05:14:25.733215+0000","last_undegraded":"2023-10-06T05:14:25.733215+0000","last_fullsized":"2023-10-06T05:14:25.733215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744108+0000","last_change":"2023-10-06T05:14:25.744108+0000","last_active":"2023-10-06T05:14:25.744108+0000","last_peered":"2023-10-06T05:14:25.744108+0000","last_clean":"2023-10-06T05:14:25.744108+0000","last_became_active":"2023-10-06T05:14:25.741869+0000","last_became_peered":"2023-10-06T05:14:25.741869+0000","last_unstale":"2023-10-06T05:14:25.744108+0000","last_undegraded":"2023-10-06T05:14:25.744108+0000","last_fullsized":"2023-10-06T05:14:25.744108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:26.682160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727785+0000","last_change":"2023-10-06T05:14:25.724254+0000","last_active":"2023-10-06T05:14:25.727785+0000","last_peered":"2023-10-06T05:14:25.727785+0000","last_clean":"2023-10-06T05:14:25.727785+0000","last_became_active":"2023-10-06T05:14:25.723995+0000","last_became_peered":"2023-10-06T05:14:25.723995+0000","last_unstale":"2023-10-06T05:14:25.727785+0000","last_undegraded":"2023-10-06T05:14:25.727785+0000","last_fullsized":"2023-10-06T05:14:25.727785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:07.229350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726644+0000","last_change":"2023-10-06T05:14:25.726644+0000","last_active":"2023-10-06T05:14:25.726644+0000","last_peered":"2023-10-06T05:14:25.726644+0000","last_clean":"2023-10-06T05:14:25.726644+0000","last_became_active":"2023-10-06T05:14:25.726030+0000","last_became_peered":"2023-10-06T05:14:25.726030+0000","last_unstale":"2023-10-06T05:14:25.726644+0000","last_undegraded":"2023-10-06T05:14:25.726644+0000","last_fullsized":"2023-10-06T05:14:25.726644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743830+0000","last_change":"2023-10-06T05:14:25.743830+0000","last_active":"2023-10-06T05:14:25.743830+0000","last_peered":"2023-10-06T05:14:25.743830+0000","last_clean":"2023-10-06T05:14:25.743830+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.743830+0000","last_undegraded":"2023-10-06T05:14:25.743830+0000","last_fullsized":"2023-10-06T05:14:25.743830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:43.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737791+0000","last_change":"2023-10-06T05:14:25.737791+0000","last_active":"2023-10-06T05:14:25.737791+0000","last_peered":"2023-10-06T05:14:25.737791+0000","last_clean":"2023-10-06T05:14:25.737791+0000","last_became_active":"2023-10-06T05:14:25.737334+0000","last_became_peered":"2023-10-06T05:14:25.737334+0000","last_unstale":"2023-10-06T05:14:25.737791+0000","last_undegraded":"2023-10-06T05:14:25.737791+0000","last_fullsized":"2023-10-06T05:14:25.737791+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744210+0000","last_change":"2023-10-06T05:14:25.743380+0000","last_active":"2023-10-06T05:14:25.744210+0000","last_peered":"2023-10-06T05:14:25.744210+0000","last_clean":"2023-10-06T05:14:25.744210+0000","last_became_active":"2023-10-06T05:14:25.743115+0000","last_became_peered":"2023-10-06T05:14:25.743115+0000","last_unstale":"2023-10-06T05:14:25.744210+0000","last_undegraded":"2023-10-06T05:14:25.744210+0000","last_fullsized":"2023-10-06T05:14:25.744210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730260+0000","last_change":"2023-10-06T05:14:25.730260+0000","last_active":"2023-10-06T05:14:25.730260+0000","last_peered":"2023-10-06T05:14:25.730260+0000","last_clean":"2023-10-06T05:14:25.730260+0000","last_became_active":"2023-10-06T05:14:25.729967+0000","last_became_peered":"2023-10-06T05:14:25.729967+0000","last_unstale":"2023-10-06T05:14:25.730260+0000","last_undegraded":"2023-10-06T05:14:25.730260+0000","last_fullsized":"2023-10-06T05:14:25.730260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.734367+0000","last_change":"2023-10-06T05:14:25.732052+0000","last_active":"2023-10-06T05:14:25.734367+0000","last_peered":"2023-10-06T05:14:25.734367+0000","last_clean":"2023-10-06T05:14:25.734367+0000","last_became_active":"2023-10-06T05:14:25.731746+0000","last_became_peered":"2023-10-06T05:14:25.731746+0000","last_unstale":"2023-10-06T05:14:25.734367+0000","last_undegraded":"2023-10-06T05:14:25.734367+0000","last_fullsized":"2023-10-06T05:14:25.734367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741800+0000","last_change":"2023-10-06T05:14:25.741800+0000","last_active":"2023-10-06T05:14:25.741800+0000","last_peered":"2023-10-06T05:14:25.741800+0000","last_clean":"2023-10-06T05:14:25.741800+0000","last_became_active":"2023-10-06T05:14:25.741167+0000","last_became_peered":"2023-10-06T05:14:25.741167+0000","last_unstale":"2023-10-06T05:14:25.741800+0000","last_undegraded":"2023-10-06T05:14:25.741800+0000","last_fullsized":"2023-10-06T05:14:25.741800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743115+0000","last_change":"2023-10-06T05:14:25.743115+0000","last_active":"2023-10-06T05:14:25.743115+0000","last_peered":"2023-10-06T05:14:25.743115+0000","last_clean":"2023-10-06T05:14:25.743115+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.743115+0000","last_undegraded":"2023-10-06T05:14:25.743115+0000","last_fullsized":"2023-10-06T05:14:25.743115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.734027+0000","last_change":"2023-10-06T05:14:25.734027+0000","last_active":"2023-10-06T05:14:25.734027+0000","last_peered":"2023-10-06T05:14:25.734027+0000","last_clean":"2023-10-06T05:14:25.734027+0000","last_became_active":"2023-10-06T05:14:25.733743+0000","last_became_peered":"2023-10-06T05:14:25.733743+0000","last_unstale":"2023-10-06T05:14:25.734027+0000","last_undegraded":"2023-10-06T05:14:25.734027+0000","last_fullsized":"2023-10-06T05:14:25.734027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725222+0000","last_change":"2023-10-06T05:14:25.721669+0000","last_active":"2023-10-06T05:14:25.725222+0000","last_peered":"2023-10-06T05:14:25.725222+0000","last_clean":"2023-10-06T05:14:25.725222+0000","last_became_active":"2023-10-06T05:14:25.721412+0000","last_became_peered":"2023-10-06T05:14:25.721412+0000","last_unstale":"2023-10-06T05:14:25.725222+0000","last_undegraded":"2023-10-06T05:14:25.725222+0000","last_fullsized":"2023-10-06T05:14:25.725222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737798+0000","last_change":"2023-10-06T05:14:25.737798+0000","last_active":"2023-10-06T05:14:25.737798+0000","last_peered":"2023-10-06T05:14:25.737798+0000","last_clean":"2023-10-06T05:14:25.737798+0000","last_became_active":"2023-10-06T05:14:25.737342+0000","last_became_peered":"2023-10-06T05:14:25.737342+0000","last_unstale":"2023-10-06T05:14:25.737798+0000","last_undegraded":"2023-10-06T05:14:25.737798+0000","last_fullsized":"2023-10-06T05:14:25.737798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740804+0000","last_change":"2023-10-06T05:14:25.740804+0000","last_active":"2023-10-06T05:14:25.740804+0000","last_peered":"2023-10-06T05:14:25.740804+0000","last_clean":"2023-10-06T05:14:25.740804+0000","last_became_active":"2023-10-06T05:14:25.740330+0000","last_became_peered":"2023-10-06T05:14:25.740330+0000","last_unstale":"2023-10-06T05:14:25.740804+0000","last_undegraded":"2023-10-06T05:14:25.740804+0000","last_fullsized":"2023-10-06T05:14:25.740804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.955604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725672+0000","last_change":"2023-10-06T05:14:25.725672+0000","last_active":"2023-10-06T05:14:25.725672+0000","last_peered":"2023-10-06T05:14:25.725672+0000","last_clean":"2023-10-06T05:14:25.725672+0000","last_became_active":"2023-10-06T05:14:25.725392+0000","last_became_peered":"2023-10-06T05:14:25.725392+0000","last_unstale":"2023-10-06T05:14:25.725672+0000","last_undegraded":"2023-10-06T05:14:25.725672+0000","last_fullsized":"2023-10-06T05:14:25.725672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743525+0000","last_change":"2023-10-06T05:14:25.743525+0000","last_active":"2023-10-06T05:14:25.743525+0000","last_peered":"2023-10-06T05:14:25.743525+0000","last_clean":"2023-10-06T05:14:25.743525+0000","last_became_active":"2023-10-06T05:14:25.742820+0000","last_became_peered":"2023-10-06T05:14:25.742820+0000","last_unstale":"2023-10-06T05:14:25.743525+0000","last_undegraded":"2023-10-06T05:14:25.743525+0000","last_fullsized":"2023-10-06T05:14:25.743525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:42.341048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.718000+0000","last_change":"2023-10-06T05:14:25.718000+0000","last_active":"2023-10-06T05:14:25.718000+0000","last_peered":"2023-10-06T05:14:25.718000+0000","last_clean":"2023-10-06T05:14:25.718000+0000","last_became_active":"2023-10-06T05:14:25.717615+0000","last_became_peered":"2023-10-06T05:14:25.717615+0000","last_unstale":"2023-10-06T05:14:25.718000+0000","last_undegraded":"2023-10-06T05:14:25.718000+0000","last_fullsized":"2023-10-06T05:14:25.718000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:17.095036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742445+0000","last_change":"2023-10-06T05:14:25.742445+0000","last_active":"2023-10-06T05:14:25.742445+0000","last_peered":"2023-10-06T05:14:25.742445+0000","last_clean":"2023-10-06T05:14:25.742445+0000","last_became_active":"2023-10-06T05:14:25.741675+0000","last_became_peered":"2023-10-06T05:14:25.741675+0000","last_unstale":"2023-10-06T05:14:25.742445+0000","last_undegraded":"2023-10-06T05:14:25.742445+0000","last_fullsized":"2023-10-06T05:14:25.742445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733702+0000","last_change":"2023-10-06T05:14:25.733702+0000","last_active":"2023-10-06T05:14:25.733702+0000","last_peered":"2023-10-06T05:14:25.733702+0000","last_clean":"2023-10-06T05:14:25.733702+0000","last_became_active":"2023-10-06T05:14:25.733387+0000","last_became_peered":"2023-10-06T05:14:25.733387+0000","last_unstale":"2023-10-06T05:14:25.733702+0000","last_undegraded":"2023-10-06T05:14:25.733702+0000","last_fullsized":"2023-10-06T05:14:25.733702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.724100+0000","last_change":"2023-10-06T05:14:25.724100+0000","last_active":"2023-10-06T05:14:25.724100+0000","last_peered":"2023-10-06T05:14:25.724100+0000","last_clean":"2023-10-06T05:14:25.724100+0000","last_became_active":"2023-10-06T05:14:25.723750+0000","last_became_peered":"2023-10-06T05:14:25.723750+0000","last_unstale":"2023-10-06T05:14:25.724100+0000","last_undegraded":"2023-10-06T05:14:25.724100+0000","last_fullsized":"2023-10-06T05:14:25.724100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735739+0000","last_change":"2023-10-06T05:14:25.727464+0000","last_active":"2023-10-06T05:14:25.735739+0000","last_peered":"2023-10-06T05:14:25.735739+0000","last_clean":"2023-10-06T05:14:25.735739+0000","last_became_active":"2023-10-06T05:14:25.727166+0000","last_became_peered":"2023-10-06T05:14:25.727166+0000","last_unstale":"2023-10-06T05:14:25.735739+0000","last_undegraded":"2023-10-06T05:14:25.735739+0000","last_fullsized":"2023-10-06T05:14:25.735739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730521+0000","last_change":"2023-10-06T05:14:25.730521+0000","last_active":"2023-10-06T05:14:25.730521+0000","last_peered":"2023-10-06T05:14:25.730521+0000","last_clean":"2023-10-06T05:14:25.730521+0000","last_became_active":"2023-10-06T05:14:25.730234+0000","last_became_peered":"2023-10-06T05:14:25.730234+0000","last_unstale":"2023-10-06T05:14:25.730521+0000","last_undegraded":"2023-10-06T05:14:25.730521+0000","last_fullsized":"2023-10-06T05:14:25.730521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735194+0000","last_change":"2023-10-06T05:14:25.733204+0000","last_active":"2023-10-06T05:14:25.735194+0000","last_peered":"2023-10-06T05:14:25.735194+0000","last_clean":"2023-10-06T05:14:25.735194+0000","last_became_active":"2023-10-06T05:14:25.732877+0000","last_became_peered":"2023-10-06T05:14:25.732877+0000","last_unstale":"2023-10-06T05:14:25.735194+0000","last_undegraded":"2023-10-06T05:14:25.735194+0000","last_fullsized":"2023-10-06T05:14:25.735194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735005+0000","last_change":"2023-10-06T05:14:25.724268+0000","last_active":"2023-10-06T05:14:25.735005+0000","last_peered":"2023-10-06T05:14:25.735005+0000","last_clean":"2023-10-06T05:14:25.735005+0000","last_became_active":"2023-10-06T05:14:25.724009+0000","last_became_peered":"2023-10-06T05:14:25.724009+0000","last_unstale":"2023-10-06T05:14:25.735005+0000","last_undegraded":"2023-10-06T05:14:25.735005+0000","last_fullsized":"2023-10-06T05:14:25.735005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727218+0000","last_change":"2023-10-06T05:14:25.725754+0000","last_active":"2023-10-06T05:14:25.727218+0000","last_peered":"2023-10-06T05:14:25.727218+0000","last_clean":"2023-10-06T05:14:25.727218+0000","last_became_active":"2023-10-06T05:14:25.725516+0000","last_became_peered":"2023-10-06T05:14:25.725516+0000","last_unstale":"2023-10-06T05:14:25.727218+0000","last_undegraded":"2023-10-06T05:14:25.727218+0000","last_fullsized":"2023-10-06T05:14:25.727218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735114+0000","last_change":"2023-10-06T05:14:24.734577+0000","last_active":"2023-10-06T05:14:25.735114+0000","last_peered":"2023-10-06T05:14:25.735114+0000","last_clean":"2023-10-06T05:14:25.735114+0000","last_became_active":"2023-10-06T05:14:24.732575+0000","last_became_peered":"2023-10-06T05:14:24.732575+0000","last_unstale":"2023-10-06T05:14:25.735114+0000","last_undegraded":"2023-10-06T05:14:25.735114+0000","last_fullsized":"2023-10-06T05:14:25.735114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726043+0000","last_change":"2023-10-06T05:14:25.721117+0000","last_active":"2023-10-06T05:14:25.726043+0000","last_peered":"2023-10-06T05:14:25.726043+0000","last_clean":"2023-10-06T05:14:25.726043+0000","last_became_active":"2023-10-06T05:14:25.720806+0000","last_became_peered":"2023-10-06T05:14:25.720806+0000","last_unstale":"2023-10-06T05:14:25.726043+0000","last_undegraded":"2023-10-06T05:14:25.726043+0000","last_fullsized":"2023-10-06T05:14:25.726043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.714069+0000","last_change":"2023-10-06T05:14:24.723717+0000","last_active":"2023-10-06T05:14:25.714069+0000","last_peered":"2023-10-06T05:14:25.714069+0000","last_clean":"2023-10-06T05:14:25.714069+0000","last_became_active":"2023-10-06T05:14:24.723440+0000","last_became_peered":"2023-10-06T05:14:24.723440+0000","last_unstale":"2023-10-06T05:14:25.714069+0000","last_undegraded":"2023-10-06T05:14:25.714069+0000","last_fullsized":"2023-10-06T05:14:25.714069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726258+0000","last_change":"2023-10-06T05:14:25.726258+0000","last_active":"2023-10-06T05:14:25.726258+0000","last_peered":"2023-10-06T05:14:25.726258+0000","last_clean":"2023-10-06T05:14:25.726258+0000","last_became_active":"2023-10-06T05:14:25.725989+0000","last_became_peered":"2023-10-06T05:14:25.725989+0000","last_unstale":"2023-10-06T05:14:25.726258+0000","last_undegraded":"2023-10-06T05:14:25.726258+0000","last_fullsized":"2023-10-06T05:14:25.726258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708532+0000","last_change":"2023-10-06T05:14:24.722594+0000","last_active":"2023-10-06T05:14:25.708532+0000","last_peered":"2023-10-06T05:14:25.708532+0000","last_clean":"2023-10-06T05:14:25.708532+0000","last_became_active":"2023-10-06T05:14:24.722302+0000","last_became_peered":"2023-10-06T05:14:24.722302+0000","last_unstale":"2023-10-06T05:14:25.708532+0000","last_undegraded":"2023-10-06T05:14:25.708532+0000","last_fullsized":"2023-10-06T05:14:25.708532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728738+0000","last_change":"2023-10-06T05:14:25.722475+0000","last_active":"2023-10-06T05:14:25.728738+0000","last_peered":"2023-10-06T05:14:25.728738+0000","last_clean":"2023-10-06T05:14:25.728738+0000","last_became_active":"2023-10-06T05:14:25.721707+0000","last_became_peered":"2023-10-06T05:14:25.721707+0000","last_unstale":"2023-10-06T05:14:25.728738+0000","last_undegraded":"2023-10-06T05:14:25.728738+0000","last_fullsized":"2023-10-06T05:14:25.728738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.727634+0000","last_change":"2023-10-06T05:14:24.724581+0000","last_active":"2023-10-06T05:14:24.727634+0000","last_peered":"2023-10-06T05:14:24.727634+0000","last_clean":"2023-10-06T05:14:24.727634+0000","last_became_active":"2023-10-06T05:14:24.724293+0000","last_became_peered":"2023-10-06T05:14:24.724293+0000","last_unstale":"2023-10-06T05:14:24.727634+0000","last_undegraded":"2023-10-06T05:14:24.727634+0000","last_fullsized":"2023-10-06T05:14:24.727634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723275+0000","last_change":"2023-10-06T05:14:25.719638+0000","last_active":"2023-10-06T05:14:25.723275+0000","last_peered":"2023-10-06T05:14:25.723275+0000","last_clean":"2023-10-06T05:14:25.723275+0000","last_became_active":"2023-10-06T05:14:25.719347+0000","last_became_peered":"2023-10-06T05:14:25.719347+0000","last_unstale":"2023-10-06T05:14:25.723275+0000","last_undegraded":"2023-10-06T05:14:25.723275+0000","last_fullsized":"2023-10-06T05:14:25.723275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716142+0000","last_change":"2023-10-06T05:14:24.722842+0000","last_active":"2023-10-06T05:14:25.716142+0000","last_peered":"2023-10-06T05:14:25.716142+0000","last_clean":"2023-10-06T05:14:25.716142+0000","last_became_active":"2023-10-06T05:14:24.721323+0000","last_became_peered":"2023-10-06T05:14:24.721323+0000","last_unstale":"2023-10-06T05:14:25.716142+0000","last_undegraded":"2023-10-06T05:14:25.716142+0000","last_fullsized":"2023-10-06T05:14:25.716142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735759+0000","last_change":"2023-10-06T05:14:25.728303+0000","last_active":"2023-10-06T05:14:25.735759+0000","last_peered":"2023-10-06T05:14:25.735759+0000","last_clean":"2023-10-06T05:14:25.735759+0000","last_became_active":"2023-10-06T05:14:25.727775+0000","last_became_peered":"2023-10-06T05:14:25.727775+0000","last_unstale":"2023-10-06T05:14:25.735759+0000","last_undegraded":"2023-10-06T05:14:25.735759+0000","last_fullsized":"2023-10-06T05:14:25.735759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738654+0000","last_change":"2023-10-06T05:14:24.728587+0000","last_active":"2023-10-06T05:14:25.738654+0000","last_peered":"2023-10-06T05:14:25.738654+0000","last_clean":"2023-10-06T05:14:25.738654+0000","last_became_active":"2023-10-06T05:14:24.728292+0000","last_became_peered":"2023-10-06T05:14:24.728292+0000","last_unstale":"2023-10-06T05:14:25.738654+0000","last_undegraded":"2023-10-06T05:14:25.738654+0000","last_fullsized":"2023-10-06T05:14:25.738654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:11.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735121+0000","last_change":"2023-10-06T05:14:25.732130+0000","last_active":"2023-10-06T05:14:25.735121+0000","last_peered":"2023-10-06T05:14:25.735121+0000","last_clean":"2023-10-06T05:14:25.735121+0000","last_became_active":"2023-10-06T05:14:25.731459+0000","last_became_peered":"2023-10-06T05:14:25.731459+0000","last_unstale":"2023-10-06T05:14:25.735121+0000","last_undegraded":"2023-10-06T05:14:25.735121+0000","last_fullsized":"2023-10-06T05:14:25.735121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741850+0000","last_change":"2023-10-06T05:14:24.728538+0000","last_active":"2023-10-06T05:14:25.741850+0000","last_peered":"2023-10-06T05:14:25.741850+0000","last_clean":"2023-10-06T05:14:25.741850+0000","last_became_active":"2023-10-06T05:14:24.728200+0000","last_became_peered":"2023-10-06T05:14:24.728200+0000","last_unstale":"2023-10-06T05:14:25.741850+0000","last_undegraded":"2023-10-06T05:14:25.741850+0000","last_fullsized":"2023-10-06T05:14:25.741850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740582+0000","last_change":"2023-10-06T05:14:25.740582+0000","last_active":"2023-10-06T05:14:25.740582+0000","last_peered":"2023-10-06T05:14:25.740582+0000","last_clean":"2023-10-06T05:14:25.740582+0000","last_became_active":"2023-10-06T05:14:25.740298+0000","last_became_peered":"2023-10-06T05:14:25.740298+0000","last_unstale":"2023-10-06T05:14:25.740582+0000","last_undegraded":"2023-10-06T05:14:25.740582+0000","last_fullsized":"2023-10-06T05:14:25.740582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:31.489344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741847+0000","last_change":"2023-10-06T05:14:24.735437+0000","last_active":"2023-10-06T05:14:25.741847+0000","last_peered":"2023-10-06T05:14:25.741847+0000","last_clean":"2023-10-06T05:14:25.741847+0000","last_became_active":"2023-10-06T05:14:24.735168+0000","last_became_peered":"2023-10-06T05:14:24.735168+0000","last_unstale":"2023-10-06T05:14:25.741847+0000","last_undegraded":"2023-10-06T05:14:25.741847+0000","last_fullsized":"2023-10-06T05:14:25.741847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:06.220991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723244+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.723244+0000","last_peered":"2023-10-06T05:14:25.723244+0000","last_clean":"2023-10-06T05:14:25.723244+0000","last_became_active":"2023-10-06T05:14:25.719474+0000","last_became_peered":"2023-10-06T05:14:25.719474+0000","last_unstale":"2023-10-06T05:14:25.723244+0000","last_undegraded":"2023-10-06T05:14:25.723244+0000","last_fullsized":"2023-10-06T05:14:25.723244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.724763+0000","last_change":"2023-10-06T05:14:24.718646+0000","last_active":"2023-10-06T05:14:24.724763+0000","last_peered":"2023-10-06T05:14:24.724763+0000","last_clean":"2023-10-06T05:14:24.724763+0000","last_became_active":"2023-10-06T05:14:24.718324+0000","last_became_peered":"2023-10-06T05:14:24.718324+0000","last_unstale":"2023-10-06T05:14:24.724763+0000","last_undegraded":"2023-10-06T05:14:24.724763+0000","last_fullsized":"2023-10-06T05:14:24.724763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727199+0000","last_change":"2023-10-06T05:14:25.724883+0000","last_active":"2023-10-06T05:14:25.727199+0000","last_peered":"2023-10-06T05:14:25.727199+0000","last_clean":"2023-10-06T05:14:25.727199+0000","last_became_active":"2023-10-06T05:14:25.724633+0000","last_became_peered":"2023-10-06T05:14:25.724633+0000","last_unstale":"2023-10-06T05:14:25.727199+0000","last_undegraded":"2023-10-06T05:14:25.727199+0000","last_fullsized":"2023-10-06T05:14:25.727199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.740512+0000","last_change":"2023-10-06T05:14:24.736333+0000","last_active":"2023-10-06T05:14:24.740512+0000","last_peered":"2023-10-06T05:14:24.740512+0000","last_clean":"2023-10-06T05:14:24.740512+0000","last_became_active":"2023-10-06T05:14:24.736038+0000","last_became_peered":"2023-10-06T05:14:24.736038+0000","last_unstale":"2023-10-06T05:14:24.740512+0000","last_undegraded":"2023-10-06T05:14:24.740512+0000","last_fullsized":"2023-10-06T05:14:24.740512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725666+0000","last_change":"2023-10-06T05:14:25.725666+0000","last_active":"2023-10-06T05:14:25.725666+0000","last_peered":"2023-10-06T05:14:25.725666+0000","last_clean":"2023-10-06T05:14:25.725666+0000","last_became_active":"2023-10-06T05:14:25.725319+0000","last_became_peered":"2023-10-06T05:14:25.725319+0000","last_unstale":"2023-10-06T05:14:25.725666+0000","last_undegraded":"2023-10-06T05:14:25.725666+0000","last_fullsized":"2023-10-06T05:14:25.725666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733857+0000","last_change":"2023-10-06T05:14:24.721548+0000","last_active":"2023-10-06T05:14:25.733857+0000","last_peered":"2023-10-06T05:14:25.733857+0000","last_clean":"2023-10-06T05:14:25.733857+0000","last_became_active":"2023-10-06T05:14:24.721266+0000","last_became_peered":"2023-10-06T05:14:24.721266+0000","last_unstale":"2023-10-06T05:14:25.733857+0000","last_undegraded":"2023-10-06T05:14:25.733857+0000","last_fullsized":"2023-10-06T05:14:25.733857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727201+0000","last_change":"2023-10-06T05:14:25.725874+0000","last_active":"2023-10-06T05:14:25.727201+0000","last_peered":"2023-10-06T05:14:25.727201+0000","last_clean":"2023-10-06T05:14:25.727201+0000","last_became_active":"2023-10-06T05:14:25.725494+0000","last_became_peered":"2023-10-06T05:14:25.725494+0000","last_unstale":"2023-10-06T05:14:25.727201+0000","last_undegraded":"2023-10-06T05:14:25.727201+0000","last_fullsized":"2023-10-06T05:14:25.727201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719074+0000","last_change":"2023-10-06T05:14:24.733795+0000","last_active":"2023-10-06T05:14:25.719074+0000","last_peered":"2023-10-06T05:14:25.719074+0000","last_clean":"2023-10-06T05:14:25.719074+0000","last_became_active":"2023-10-06T05:14:24.733567+0000","last_became_peered":"2023-10-06T05:14:24.733567+0000","last_unstale":"2023-10-06T05:14:25.719074+0000","last_undegraded":"2023-10-06T05:14:25.719074+0000","last_fullsized":"2023-10-06T05:14:25.719074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732255+0000","last_change":"2023-10-06T05:14:25.727738+0000","last_active":"2023-10-06T05:14:25.732255+0000","last_peered":"2023-10-06T05:14:25.732255+0000","last_clean":"2023-10-06T05:14:25.732255+0000","last_became_active":"2023-10-06T05:14:25.727474+0000","last_became_peered":"2023-10-06T05:14:25.727474+0000","last_unstale":"2023-10-06T05:14:25.732255+0000","last_undegraded":"2023-10-06T05:14:25.732255+0000","last_fullsized":"2023-10-06T05:14:25.732255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742236+0000","last_change":"2023-10-06T05:14:24.733458+0000","last_active":"2023-10-06T05:14:25.742236+0000","last_peered":"2023-10-06T05:14:25.742236+0000","last_clean":"2023-10-06T05:14:25.742236+0000","last_became_active":"2023-10-06T05:14:24.733160+0000","last_became_peered":"2023-10-06T05:14:24.733160+0000","last_unstale":"2023-10-06T05:14:25.742236+0000","last_undegraded":"2023-10-06T05:14:25.742236+0000","last_fullsized":"2023-10-06T05:14:25.742236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741624+0000","last_change":"2023-10-06T05:14:25.741624+0000","last_active":"2023-10-06T05:14:25.741624+0000","last_peered":"2023-10-06T05:14:25.741624+0000","last_clean":"2023-10-06T05:14:25.741624+0000","last_became_active":"2023-10-06T05:14:25.740252+0000","last_became_peered":"2023-10-06T05:14:25.740252+0000","last_unstale":"2023-10-06T05:14:25.741624+0000","last_undegraded":"2023-10-06T05:14:25.741624+0000","last_fullsized":"2023-10-06T05:14:25.741624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723475+0000","last_change":"2023-10-06T05:14:24.721437+0000","last_active":"2023-10-06T05:14:25.723475+0000","last_peered":"2023-10-06T05:14:25.723475+0000","last_clean":"2023-10-06T05:14:25.723475+0000","last_became_active":"2023-10-06T05:14:24.721172+0000","last_became_peered":"2023-10-06T05:14:24.721172+0000","last_unstale":"2023-10-06T05:14:25.723475+0000","last_undegraded":"2023-10-06T05:14:25.723475+0000","last_fullsized":"2023-10-06T05:14:25.723475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:22.852582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727675+0000","last_change":"2023-10-06T05:14:25.719600+0000","last_active":"2023-10-06T05:14:25.727675+0000","last_peered":"2023-10-06T05:14:25.727675+0000","last_clean":"2023-10-06T05:14:25.727675+0000","last_became_active":"2023-10-06T05:14:25.719349+0000","last_became_peered":"2023-10-06T05:14:25.719349+0000","last_unstale":"2023-10-06T05:14:25.727675+0000","last_undegraded":"2023-10-06T05:14:25.727675+0000","last_fullsized":"2023-10-06T05:14:25.727675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716852+0000","last_change":"2023-10-06T05:14:24.728714+0000","last_active":"2023-10-06T05:14:25.716852+0000","last_peered":"2023-10-06T05:14:25.716852+0000","last_clean":"2023-10-06T05:14:25.716852+0000","last_became_active":"2023-10-06T05:14:24.728435+0000","last_became_peered":"2023-10-06T05:14:24.728435+0000","last_unstale":"2023-10-06T05:14:25.716852+0000","last_undegraded":"2023-10-06T05:14:25.716852+0000","last_fullsized":"2023-10-06T05:14:25.716852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.724993+0000","last_change":"2023-10-06T05:14:25.724993+0000","last_active":"2023-10-06T05:14:25.724993+0000","last_peered":"2023-10-06T05:14:25.724993+0000","last_clean":"2023-10-06T05:14:25.724993+0000","last_became_active":"2023-10-06T05:14:25.724538+0000","last_became_peered":"2023-10-06T05:14:25.724538+0000","last_unstale":"2023-10-06T05:14:25.724993+0000","last_undegraded":"2023-10-06T05:14:25.724993+0000","last_fullsized":"2023-10-06T05:14:25.724993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738370+0000","last_change":"2023-10-06T05:14:24.723321+0000","last_active":"2023-10-06T05:14:25.738370+0000","last_peered":"2023-10-06T05:14:25.738370+0000","last_clean":"2023-10-06T05:14:25.738370+0000","last_became_active":"2023-10-06T05:14:24.723063+0000","last_became_peered":"2023-10-06T05:14:24.723063+0000","last_unstale":"2023-10-06T05:14:25.738370+0000","last_undegraded":"2023-10-06T05:14:25.738370+0000","last_fullsized":"2023-10-06T05:14:25.738370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740648+0000","last_change":"2023-10-06T05:14:25.740648+0000","last_active":"2023-10-06T05:14:25.740648+0000","last_peered":"2023-10-06T05:14:25.740648+0000","last_clean":"2023-10-06T05:14:25.740648+0000","last_became_active":"2023-10-06T05:14:25.740351+0000","last_became_peered":"2023-10-06T05:14:25.740351+0000","last_unstale":"2023-10-06T05:14:25.740648+0000","last_undegraded":"2023-10-06T05:14:25.740648+0000","last_fullsized":"2023-10-06T05:14:25.740648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709955+0000","last_change":"2023-10-06T05:14:24.730263+0000","last_active":"2023-10-06T05:14:25.709955+0000","last_peered":"2023-10-06T05:14:25.709955+0000","last_clean":"2023-10-06T05:14:25.709955+0000","last_became_active":"2023-10-06T05:14:24.729911+0000","last_became_peered":"2023-10-06T05:14:24.729911+0000","last_unstale":"2023-10-06T05:14:25.709955+0000","last_undegraded":"2023-10-06T05:14:25.709955+0000","last_fullsized":"2023-10-06T05:14:25.709955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744620+0000","last_change":"2023-10-06T05:14:25.744620+0000","last_active":"2023-10-06T05:14:25.744620+0000","last_peered":"2023-10-06T05:14:25.744620+0000","last_clean":"2023-10-06T05:14:25.744620+0000","last_became_active":"2023-10-06T05:14:25.744323+0000","last_became_peered":"2023-10-06T05:14:25.744323+0000","last_unstale":"2023-10-06T05:14:25.744620+0000","last_undegraded":"2023-10-06T05:14:25.744620+0000","last_fullsized":"2023-10-06T05:14:25.744620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:39.850675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.735136+0000","last_change":"2023-10-06T05:14:24.731862+0000","last_active":"2023-10-06T05:14:24.735136+0000","last_peered":"2023-10-06T05:14:24.735136+0000","last_clean":"2023-10-06T05:14:24.735136+0000","last_became_active":"2023-10-06T05:14:24.729629+0000","last_became_peered":"2023-10-06T05:14:24.729629+0000","last_unstale":"2023-10-06T05:14:24.735136+0000","last_undegraded":"2023-10-06T05:14:24.735136+0000","last_fullsized":"2023-10-06T05:14:24.735136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742674+0000","last_change":"2023-10-06T05:14:25.742674+0000","last_active":"2023-10-06T05:14:25.742674+0000","last_peered":"2023-10-06T05:14:25.742674+0000","last_clean":"2023-10-06T05:14:25.742674+0000","last_became_active":"2023-10-06T05:14:25.741308+0000","last_became_peered":"2023-10-06T05:14:25.741308+0000","last_unstale":"2023-10-06T05:14:25.742674+0000","last_undegraded":"2023-10-06T05:14:25.742674+0000","last_fullsized":"2023-10-06T05:14:25.742674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:29.811666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708350+0000","last_change":"2023-10-06T05:14:24.724203+0000","last_active":"2023-10-06T05:14:25.708350+0000","last_peered":"2023-10-06T05:14:25.708350+0000","last_clean":"2023-10-06T05:14:25.708350+0000","last_became_active":"2023-10-06T05:14:24.723929+0000","last_became_peered":"2023-10-06T05:14:24.723929+0000","last_unstale":"2023-10-06T05:14:25.708350+0000","last_undegraded":"2023-10-06T05:14:25.708350+0000","last_fullsized":"2023-10-06T05:14:25.708350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735025+0000","last_change":"2023-10-06T05:14:25.726645+0000","last_active":"2023-10-06T05:14:25.735025+0000","last_peered":"2023-10-06T05:14:25.735025+0000","last_clean":"2023-10-06T05:14:25.735025+0000","last_became_active":"2023-10-06T05:14:25.724695+0000","last_became_peered":"2023-10-06T05:14:25.724695+0000","last_unstale":"2023-10-06T05:14:25.735025+0000","last_undegraded":"2023-10-06T05:14:25.735025+0000","last_fullsized":"2023-10-06T05:14:25.735025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713550+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:14:25.713550+0000","last_peered":"2023-10-06T05:14:25.713550+0000","last_clean":"2023-10-06T05:14:25.713550+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:14:25.713550+0000","last_undegraded":"2023-10-06T05:14:25.713550+0000","last_fullsized":"2023-10-06T05:14:25.713550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716529+0000","last_change":"2023-10-06T05:14:24.728757+0000","last_active":"2023-10-06T05:14:25.716529+0000","last_peered":"2023-10-06T05:14:25.716529+0000","last_clean":"2023-10-06T05:14:25.716529+0000","last_became_active":"2023-10-06T05:14:24.728480+0000","last_became_peered":"2023-10-06T05:14:24.728480+0000","last_unstale":"2023-10-06T05:14:25.716529+0000","last_undegraded":"2023-10-06T05:14:25.716529+0000","last_fullsized":"2023-10-06T05:14:25.716529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727990+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.727990+0000","last_peered":"2023-10-06T05:14:25.727990+0000","last_clean":"2023-10-06T05:14:25.727990+0000","last_became_active":"2023-10-06T05:14:25.719491+0000","last_became_peered":"2023-10-06T05:14:25.719491+0000","last_unstale":"2023-10-06T05:14:25.727990+0000","last_undegraded":"2023-10-06T05:14:25.727990+0000","last_fullsized":"2023-10-06T05:14:25.727990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715616+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:14:25.715616+0000","last_peered":"2023-10-06T05:14:25.715616+0000","last_clean":"2023-10-06T05:14:25.715616+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:14:25.715616+0000","last_undegraded":"2023-10-06T05:14:25.715616+0000","last_fullsized":"2023-10-06T05:14:25.715616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.737577+0000","last_change":"2023-10-06T05:14:24.732737+0000","last_active":"2023-10-06T05:14:24.737577+0000","last_peered":"2023-10-06T05:14:24.737577+0000","last_clean":"2023-10-06T05:14:24.737577+0000","last_became_active":"2023-10-06T05:14:24.732410+0000","last_became_peered":"2023-10-06T05:14:24.732410+0000","last_unstale":"2023-10-06T05:14:24.737577+0000","last_undegraded":"2023-10-06T05:14:24.737577+0000","last_fullsized":"2023-10-06T05:14:24.737577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736090+0000","last_change":"2023-10-06T05:14:25.723180+0000","last_active":"2023-10-06T05:14:25.736090+0000","last_peered":"2023-10-06T05:14:25.736090+0000","last_clean":"2023-10-06T05:14:25.736090+0000","last_became_active":"2023-10-06T05:14:25.722867+0000","last_became_peered":"2023-10-06T05:14:25.722867+0000","last_unstale":"2023-10-06T05:14:25.736090+0000","last_undegraded":"2023-10-06T05:14:25.736090+0000","last_fullsized":"2023-10-06T05:14:25.736090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.718276+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:14:24.718276+0000","last_peered":"2023-10-06T05:14:24.718276+0000","last_clean":"2023-10-06T05:14:24.718276+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:14:24.718276+0000","last_undegraded":"2023-10-06T05:14:24.718276+0000","last_fullsized":"2023-10-06T05:14:24.718276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713983+0000","last_change":"2023-10-06T05:14:24.719784+0000","last_active":"2023-10-06T05:14:25.713983+0000","last_peered":"2023-10-06T05:14:25.713983+0000","last_clean":"2023-10-06T05:14:25.713983+0000","last_became_active":"2023-10-06T05:14:24.719414+0000","last_became_peered":"2023-10-06T05:14:24.719414+0000","last_unstale":"2023-10-06T05:14:25.713983+0000","last_undegraded":"2023-10-06T05:14:25.713983+0000","last_fullsized":"2023-10-06T05:14:25.713983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735002+0000","last_change":"2023-10-06T05:14:25.725673+0000","last_active":"2023-10-06T05:14:25.735002+0000","last_peered":"2023-10-06T05:14:25.735002+0000","last_clean":"2023-10-06T05:14:25.735002+0000","last_became_active":"2023-10-06T05:14:25.725107+0000","last_became_peered":"2023-10-06T05:14:25.725107+0000","last_unstale":"2023-10-06T05:14:25.735002+0000","last_undegraded":"2023-10-06T05:14:25.735002+0000","last_fullsized":"2023-10-06T05:14:25.735002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:35.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715779+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:14:25.715779+0000","last_peered":"2023-10-06T05:14:25.715779+0000","last_clean":"2023-10-06T05:14:25.715779+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:14:25.715779+0000","last_undegraded":"2023-10-06T05:14:25.715779+0000","last_fullsized":"2023-10-06T05:14:25.715779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.710679+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:14:24.710679+0000","last_peered":"2023-10-06T05:14:24.710679+0000","last_clean":"2023-10-06T05:14:24.710679+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:14:24.710679+0000","last_undegraded":"2023-10-06T05:14:24.710679+0000","last_fullsized":"2023-10-06T05:14:24.710679+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708071+0000","last_change":"2023-10-06T05:14:24.717406+0000","last_active":"2023-10-06T05:14:25.708071+0000","last_peered":"2023-10-06T05:14:25.708071+0000","last_clean":"2023-10-06T05:14:25.708071+0000","last_became_active":"2023-10-06T05:14:24.716723+0000","last_became_peered":"2023-10-06T05:14:24.716723+0000","last_unstale":"2023-10-06T05:14:25.708071+0000","last_undegraded":"2023-10-06T05:14:25.708071+0000","last_fullsized":"2023-10-06T05:14:25.708071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.746536+0000","last_change":"2023-10-06T05:14:25.745372+0000","last_active":"2023-10-06T05:14:25.746536+0000","last_peered":"2023-10-06T05:14:25.746536+0000","last_clean":"2023-10-06T05:14:25.746536+0000","last_became_active":"2023-10-06T05:14:25.745163+0000","last_became_peered":"2023-10-06T05:14:25.745163+0000","last_unstale":"2023-10-06T05:14:25.746536+0000","last_undegraded":"2023-10-06T05:14:25.746536+0000","last_fullsized":"2023-10-06T05:14:25.746536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.860941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.710871+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:14:24.710871+0000","last_peered":"2023-10-06T05:14:24.710871+0000","last_clean":"2023-10-06T05:14:24.710871+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:14:24.710871+0000","last_undegraded":"2023-10-06T05:14:24.710871+0000","last_fullsized":"2023-10-06T05:14:24.710871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.737469+0000","last_change":"2023-10-06T05:14:24.736663+0000","last_active":"2023-10-06T05:14:24.737469+0000","last_peered":"2023-10-06T05:14:24.737469+0000","last_clean":"2023-10-06T05:14:24.737469+0000","last_became_active":"2023-10-06T05:14:24.736241+0000","last_became_peered":"2023-10-06T05:14:24.736241+0000","last_unstale":"2023-10-06T05:14:24.737469+0000","last_undegraded":"2023-10-06T05:14:24.737469+0000","last_fullsized":"2023-10-06T05:14:24.737469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.729949+0000","last_change":"2023-10-06T05:14:25.729949+0000","last_active":"2023-10-06T05:14:25.729949+0000","last_peered":"2023-10-06T05:14:25.729949+0000","last_clean":"2023-10-06T05:14:25.729949+0000","last_became_active":"2023-10-06T05:14:25.729600+0000","last_became_peered":"2023-10-06T05:14:25.729600+0000","last_unstale":"2023-10-06T05:14:25.729949+0000","last_undegraded":"2023-10-06T05:14:25.729949+0000","last_fullsized":"2023-10-06T05:14:25.729949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.711786+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:14:24.711786+0000","last_peered":"2023-10-06T05:14:24.711786+0000","last_clean":"2023-10-06T05:14:24.711786+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:14:24.711786+0000","last_undegraded":"2023-10-06T05:14:24.711786+0000","last_fullsized":"2023-10-06T05:14:24.711786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713950+0000","last_change":"2023-10-06T05:14:24.724312+0000","last_active":"2023-10-06T05:14:25.713950+0000","last_peered":"2023-10-06T05:14:25.713950+0000","last_clean":"2023-10-06T05:14:25.713950+0000","last_became_active":"2023-10-06T05:14:24.724011+0000","last_became_peered":"2023-10-06T05:14:24.724011+0000","last_unstale":"2023-10-06T05:14:25.713950+0000","last_undegraded":"2023-10-06T05:14:25.713950+0000","last_fullsized":"2023-10-06T05:14:25.713950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.729128+0000","last_change":"2023-10-06T05:14:25.729128+0000","last_active":"2023-10-06T05:14:25.729128+0000","last_peered":"2023-10-06T05:14:25.729128+0000","last_clean":"2023-10-06T05:14:25.729128+0000","last_became_active":"2023-10-06T05:14:25.728715+0000","last_became_peered":"2023-10-06T05:14:25.728715+0000","last_unstale":"2023-10-06T05:14:25.729128+0000","last_undegraded":"2023-10-06T05:14:25.729128+0000","last_fullsized":"2023-10-06T05:14:25.729128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727798+0000","last_change":"2023-10-06T05:14:24.721869+0000","last_active":"2023-10-06T05:14:25.727798+0000","last_peered":"2023-10-06T05:14:25.727798+0000","last_clean":"2023-10-06T05:14:25.727798+0000","last_became_active":"2023-10-06T05:14:24.719902+0000","last_became_peered":"2023-10-06T05:14:24.719902+0000","last_unstale":"2023-10-06T05:14:25.727798+0000","last_undegraded":"2023-10-06T05:14:25.727798+0000","last_fullsized":"2023-10-06T05:14:25.727798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713213+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:14:25.713213+0000","last_peered":"2023-10-06T05:14:25.713213+0000","last_clean":"2023-10-06T05:14:25.713213+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:14:25.713213+0000","last_undegraded":"2023-10-06T05:14:25.713213+0000","last_fullsized":"2023-10-06T05:14:25.713213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735685+0000","last_change":"2023-10-06T05:14:25.727038+0000","last_active":"2023-10-06T05:14:25.735685+0000","last_peered":"2023-10-06T05:14:25.735685+0000","last_clean":"2023-10-06T05:14:25.735685+0000","last_became_active":"2023-10-06T05:14:25.726717+0000","last_became_peered":"2023-10-06T05:14:25.726717+0000","last_unstale":"2023-10-06T05:14:25.735685+0000","last_undegraded":"2023-10-06T05:14:25.735685+0000","last_fullsized":"2023-10-06T05:14:25.735685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709571+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:14:25.709571+0000","last_peered":"2023-10-06T05:14:25.709571+0000","last_clean":"2023-10-06T05:14:25.709571+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:14:25.709571+0000","last_undegraded":"2023-10-06T05:14:25.709571+0000","last_fullsized":"2023-10-06T05:14:25.709571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T05:14:24.724706+0000","last_change":"2023-10-06T05:14:24.717582+0000","last_active":"2023-10-06T05:14:24.724706+0000","last_peered":"2023-10-06T05:14:24.724706+0000","last_clean":"2023-10-06T05:14:24.724706+0000","last_became_active":"2023-10-06T05:14:24.716871+0000","last_became_peered":"2023-10-06T05:14:24.716871+0000","last_unstale":"2023-10-06T05:14:24.724706+0000","last_undegraded":"2023-10-06T05:14:24.724706+0000","last_fullsized":"2023-10-06T05:14:24.724706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737874+0000","last_change":"2023-10-06T05:14:25.737874+0000","last_active":"2023-10-06T05:14:25.737874+0000","last_peered":"2023-10-06T05:14:25.737874+0000","last_clean":"2023-10-06T05:14:25.737874+0000","last_became_active":"2023-10-06T05:14:25.737569+0000","last_became_peered":"2023-10-06T05:14:25.737569+0000","last_unstale":"2023-10-06T05:14:25.737874+0000","last_undegraded":"2023-10-06T05:14:25.737874+0000","last_fullsized":"2023-10-06T05:14:25.737874+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716149+0000","last_change":"2023-10-06T05:14:24.736575+0000","last_active":"2023-10-06T05:14:25.716149+0000","last_peered":"2023-10-06T05:14:25.716149+0000","last_clean":"2023-10-06T05:14:25.716149+0000","last_became_active":"2023-10-06T05:14:24.736221+0000","last_became_peered":"2023-10-06T05:14:24.736221+0000","last_unstale":"2023-10-06T05:14:25.716149+0000","last_undegraded":"2023-10-06T05:14:25.716149+0000","last_fullsized":"2023-10-06T05:14:25.716149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719862+0000","last_change":"2023-10-06T05:14:25.719862+0000","last_active":"2023-10-06T05:14:25.719862+0000","last_peered":"2023-10-06T05:14:25.719862+0000","last_clean":"2023-10-06T05:14:25.719862+0000","last_became_active":"2023-10-06T05:14:25.719558+0000","last_became_peered":"2023-10-06T05:14:25.719558+0000","last_unstale":"2023-10-06T05:14:25.719862+0000","last_undegraded":"2023-10-06T05:14:25.719862+0000","last_fullsized":"2023-10-06T05:14:25.719862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:41.453897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742798+0000","last_change":"2023-10-06T05:14:24.734397+0000","last_active":"2023-10-06T05:14:25.742798+0000","last_peered":"2023-10-06T05:14:25.742798+0000","last_clean":"2023-10-06T05:14:25.742798+0000","last_became_active":"2023-10-06T05:14:24.734112+0000","last_became_peered":"2023-10-06T05:14:24.734112+0000","last_unstale":"2023-10-06T05:14:25.742798+0000","last_undegraded":"2023-10-06T05:14:25.742798+0000","last_fullsized":"2023-10-06T05:14:25.742798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:50.734968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736354+0000","last_change":"2023-10-06T05:14:25.736354+0000","last_active":"2023-10-06T05:14:25.736354+0000","last_peered":"2023-10-06T05:14:25.736354+0000","last_clean":"2023-10-06T05:14:25.736354+0000","last_became_active":"2023-10-06T05:14:25.735695+0000","last_became_peered":"2023-10-06T05:14:25.735695+0000","last_unstale":"2023-10-06T05:14:25.736354+0000","last_undegraded":"2023-10-06T05:14:25.736354+0000","last_fullsized":"2023-10-06T05:14:25.736354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738237+0000","last_change":"2023-10-06T05:14:24.733470+0000","last_active":"2023-10-06T05:14:25.738237+0000","last_peered":"2023-10-06T05:14:25.738237+0000","last_clean":"2023-10-06T05:14:25.738237+0000","last_became_active":"2023-10-06T05:14:24.733189+0000","last_became_peered":"2023-10-06T05:14:24.733189+0000","last_unstale":"2023-10-06T05:14:25.738237+0000","last_undegraded":"2023-10-06T05:14:25.738237+0000","last_fullsized":"2023-10-06T05:14:25.738237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744257+0000","last_change":"2023-10-06T05:14:25.744257+0000","last_active":"2023-10-06T05:14:25.744257+0000","last_peered":"2023-10-06T05:14:25.744257+0000","last_clean":"2023-10-06T05:14:25.744257+0000","last_became_active":"2023-10-06T05:14:25.743800+0000","last_became_peered":"2023-10-06T05:14:25.743800+0000","last_unstale":"2023-10-06T05:14:25.744257+0000","last_undegraded":"2023-10-06T05:14:25.744257+0000","last_fullsized":"2023-10-06T05:14:25.744257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737149+0000","last_change":"2023-10-06T05:14:24.719523+0000","last_active":"2023-10-06T05:14:25.737149+0000","last_peered":"2023-10-06T05:14:25.737149+0000","last_clean":"2023-10-06T05:14:25.737149+0000","last_became_active":"2023-10-06T05:14:24.717215+0000","last_became_peered":"2023-10-06T05:14:24.717215+0000","last_unstale":"2023-10-06T05:14:25.737149+0000","last_undegraded":"2023-10-06T05:14:25.737149+0000","last_fullsized":"2023-10-06T05:14:25.737149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733084+0000","last_change":"2023-10-06T05:14:25.726409+0000","last_active":"2023-10-06T05:14:25.733084+0000","last_peered":"2023-10-06T05:14:25.733084+0000","last_clean":"2023-10-06T05:14:25.733084+0000","last_became_active":"2023-10-06T05:14:25.725760+0000","last_became_peered":"2023-10-06T05:14:25.725760+0000","last_unstale":"2023-10-06T05:14:25.733084+0000","last_undegraded":"2023-10-06T05:14:25.733084+0000","last_fullsized":"2023-10-06T05:14:25.733084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730378+0000","last_change":"2023-10-06T05:14:24.724643+0000","last_active":"2023-10-06T05:14:25.730378+0000","last_peered":"2023-10-06T05:14:25.730378+0000","last_clean":"2023-10-06T05:14:25.730378+0000","last_became_active":"2023-10-06T05:14:24.724364+0000","last_became_peered":"2023-10-06T05:14:24.724364+0000","last_unstale":"2023-10-06T05:14:25.730378+0000","last_undegraded":"2023-10-06T05:14:25.730378+0000","last_fullsized":"2023-10-06T05:14:25.730378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741519+0000","last_change":"2023-10-06T05:14:25.741519+0000","last_active":"2023-10-06T05:14:25.741519+0000","last_peered":"2023-10-06T05:14:25.741519+0000","last_clean":"2023-10-06T05:14:25.741519+0000","last_became_active":"2023-10-06T05:14:25.741100+0000","last_became_peered":"2023-10-06T05:14:25.741100+0000","last_unstale":"2023-10-06T05:14:25.741519+0000","last_undegraded":"2023-10-06T05:14:25.741519+0000","last_fullsized":"2023-10-06T05:14:25.741519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728144+0000","last_change":"2023-10-06T05:14:25.721386+0000","last_active":"2023-10-06T05:14:25.728144+0000","last_peered":"2023-10-06T05:14:25.728144+0000","last_clean":"2023-10-06T05:14:25.728144+0000","last_became_active":"2023-10-06T05:14:25.720975+0000","last_became_peered":"2023-10-06T05:14:25.720975+0000","last_unstale":"2023-10-06T05:14:25.728144+0000","last_undegraded":"2023-10-06T05:14:25.728144+0000","last_fullsized":"2023-10-06T05:14:25.728144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:51.743327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733104+0000","last_change":"2023-10-06T05:14:25.726604+0000","last_active":"2023-10-06T05:14:25.733104+0000","last_peered":"2023-10-06T05:14:25.733104+0000","last_clean":"2023-10-06T05:14:25.733104+0000","last_became_active":"2023-10-06T05:14:25.726075+0000","last_became_peered":"2023-10-06T05:14:25.726075+0000","last_unstale":"2023-10-06T05:14:25.733104+0000","last_undegraded":"2023-10-06T05:14:25.733104+0000","last_fullsized":"2023-10-06T05:14:25.733104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743617+0000","last_change":"2023-10-06T05:14:25.743617+0000","last_active":"2023-10-06T05:14:25.743617+0000","last_peered":"2023-10-06T05:14:25.743617+0000","last_clean":"2023-10-06T05:14:25.743617+0000","last_became_active":"2023-10-06T05:14:25.741960+0000","last_became_peered":"2023-10-06T05:14:25.741960+0000","last_unstale":"2023-10-06T05:14:25.743617+0000","last_undegraded":"2023-10-06T05:14:25.743617+0000","last_fullsized":"2023-10-06T05:14:25.743617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:50.792266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735674+0000","last_change":"2023-10-06T05:14:25.728085+0000","last_active":"2023-10-06T05:14:25.735674+0000","last_peered":"2023-10-06T05:14:25.735674+0000","last_clean":"2023-10-06T05:14:25.735674+0000","last_became_active":"2023-10-06T05:14:25.727721+0000","last_became_peered":"2023-10-06T05:14:25.727721+0000","last_unstale":"2023-10-06T05:14:25.735674+0000","last_undegraded":"2023-10-06T05:14:25.735674+0000","last_fullsized":"2023-10-06T05:14:25.735674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739584+0000","last_change":"2023-10-06T05:14:25.735959+0000","last_active":"2023-10-06T05:14:25.739584+0000","last_peered":"2023-10-06T05:14:25.739584+0000","last_clean":"2023-10-06T05:14:25.739584+0000","last_became_active":"2023-10-06T05:14:25.735602+0000","last_became_peered":"2023-10-06T05:14:25.735602+0000","last_unstale":"2023-10-06T05:14:25.739584+0000","last_undegraded":"2023-10-06T05:14:25.739584+0000","last_fullsized":"2023-10-06T05:14:25.739584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":674798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":827392,"data_stored":673600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":82768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":84270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":83302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:14:30.563 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:14:30.563 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T05:14:30.563 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T05:14:30.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T05:14:30.564 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T05:14:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.781+0000 7f405b59d700 1 Processor -- start 2023-10-06T05:14:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.781+0000 7f405b59d700 1 -- start start 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.782+0000 7f405b59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f40540cf3b0 0x7f40541594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.782+0000 7f405b59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40540af0a0 0x7f40541599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.782+0000 7f405b59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40540cbd80 con 0x7f40540ccfc0 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.782+0000 7f405b59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40540af5f0 con 0x7f40540af0a0 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.782+0000 7f405b59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40540af8d0 con 0x7f40540cf3b0 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40540af0a0 0x7f40541599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f40540af0a0 0x7f40541599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37194/0 (socket says 172.21.15.33:37194) 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 -- 172.21.15.33:0/1521683473 learned_addr learned my addr 172.21.15.33:0/1521683473 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f4059d9a700 1 --2- 172.21.15.33:0/1521683473 >> v2:172.21.15.167:3300/0 conn(0x7f40540cf3b0 0x7f40541594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 -- 172.21.15.33:0/1521683473 >> v2:172.21.15.167:3300/0 conn(0x7f40540cf3b0 msgr2=0x7f40541594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 --2- 172.21.15.33:0/1521683473 >> v2:172.21.15.167:3300/0 conn(0x7f40540cf3b0 0x7f40541594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 -- 172.21.15.33:0/1521683473 >> v1:172.21.15.167:6789/0 conn(0x7f40540ccfc0 legacy=0x7f40540cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 -- 172.21.15.33:0/1521683473 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540afbb0 con 0x7f40540af0a0 2023-10-06T05:14:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.783+0000 7f405ad9c700 1 --2- 172.21.15.33:0/1521683473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40541599e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f404c009f20 tx=0x7f404c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505f2f9670df1e0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f4059599700 1 -- 172.21.15.33:0/1521683473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00e070 con 0x7f40540af0a0 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f4059599700 1 -- 172.21.15.33:0/1521683473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c00b990 con 0x7f40540af0a0 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f4059599700 1 -- 172.21.15.33:0/1521683473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0092e0 con 0x7f40540af0a0 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 -- 172.21.15.33:0/1521683473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 msgr2=0x7f40541599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 --2- 172.21.15.33:0/1521683473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40541599e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f404c009f20 tx=0x7f404c009840 comp rx=0 tx=0).stop 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 -- 172.21.15.33:0/1521683473 shutdown_connections 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 --2- 172.21.15.33:0/1521683473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40541599e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 --2- 172.21.15.33:0/1521683473 >> v2:172.21.15.167:3300/0 conn(0x7f40540cf3b0 0x7f40541594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 -- 172.21.15.33:0/1521683473 >> 172.21.15.33:0/1521683473 conn(0x7f405406cff0 msgr2=0x7f405406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:30.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 -- 172.21.15.33:0/1521683473 shutdown_connections 2023-10-06T05:14:30.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.784+0000 7f405b59d700 1 -- 172.21.15.33:0/1521683473 wait complete. 2023-10-06T05:14:30.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.786+0000 7f405b59d700 1 Processor -- start 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.786+0000 7f405b59d700 1 -- start start 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40540b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40540ccfc0 0x7f40540ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40540cad30 0x7f40540c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40540cb1d0 con 0x7f40540af0a0 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40540c93d0 con 0x7f40540cad30 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40540c96b0 con 0x7f40540ccfc0 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40540b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40540b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37204/0 (socket says 172.21.15.33:37204) 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 -- 172.21.15.33:0/2398729648 learned_addr learned my addr 172.21.15.33:0/2398729648 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:30.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 -- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40540ccfc0 msgr2=0x7f40540ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40540ccfc0 0x7f40540ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 -- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40540cad30 msgr2=0x7f40540c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40540cad30 0x7f40540c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.787+0000 7f405a59b700 1 -- 172.21.15.33:0/2398729648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c009ae0 con 0x7f40540af0a0 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.788+0000 7f405a59b700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40540b50b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f405000c4e0 tx=0x7f405000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.788+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050011590 con 0x7f40540af0a0 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.788+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4050011710 con 0x7f40540af0a0 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.788+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050011ad0 con 0x7f40540af0a0 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.788+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40540c9990 con 0x7f40540af0a0 2023-10-06T05:14:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.788+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40540c9f50 con 0x7f40540af0a0 2023-10-06T05:14:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.789+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40540ca170 con 0x7f40540af0a0 2023-10-06T05:14:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.790+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f405001b3b0 con 0x7f40540af0a0 2023-10-06T05:14:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.791+0000 7f404a7fc700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4038066f30 0x7f40380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.791+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4050083b10 con 0x7f40540af0a0 2023-10-06T05:14:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.791+0000 7f4059d9a700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4038066f30 0x7f40380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.792+0000 7f4059d9a700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4038066f30 0x7f40380693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f405406bdf0 tx=0x7f40440091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:30.795+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4050050c20 con 0x7f40540af0a0 2023-10-06T05:14:30.863 INFO:tasks.ceph.mds.d.smithi167.stderr:2023-10-06T05:14:30.861+0000 7f294aff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:14:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.057+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f405404c380 con 0x7f40540af0a0 2023-10-06T05:14:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.061+0000 7f404a7fc700 1 -- 172.21.15.33:0/2398729648 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7f4050017070 con 0x7f40540af0a0 2023-10-06T05:14:31.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4038066f30 msgr2=0x7f40380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4038066f30 0x7f40380693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f405406bdf0 tx=0x7f40440091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 msgr2=0x7f40540b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40540b50b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f405000c4e0 tx=0x7f405000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 shutdown_connections 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4038066f30 0x7f40380693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40540cad30 0x7f40540c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40540af0a0 0x7f40540b50b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 --2- 172.21.15.33:0/2398729648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40540ccfc0 0x7f40540ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 >> 172.21.15.33:0/2398729648 conn(0x7f405406cff0 msgr2=0x7f40540b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 shutdown_connections 2023-10-06T05:14:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.064+0000 7f405b59d700 1 -- 172.21.15.33:0/2398729648 wait complete. 2023-10-06T05:14:31.083 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":85179},{"type":"v1","addr":"172.21.15.167:6833","nonce":85179}]},"active_addr":"172.21.15.167:6833/85179","active_change":"2023-10-06T05:14:05.798345+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2007559130}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1817245224}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2507680897}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1439856044}]}]} 2023-10-06T05:14:31.090 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T05:14:31.090 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T05:14:31.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.273+0000 7fe7d319d700 1 Processor -- start 2023-10-06T05:14:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.273+0000 7fe7d319d700 1 -- start start 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.274+0000 7fe7d319d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0b0a40 0x7fe7cc06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.274+0000 7fe7d319d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7cc0b0e10 0x7fe7cc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.274+0000 7fe7d319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b2e30 0x7fe7cc15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.274+0000 7fe7d319d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc15e2d0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.274+0000 7fe7d319d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc15d6c0 con 0x7fe7cc0b0a40 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.274+0000 7fe7d319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc15d9a0 con 0x7fe7cc0b2e30 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7cc0b0e10 0x7fe7cc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7cc0b0e10 0x7fe7cc0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37218/0 (socket says 172.21.15.33:37218) 2023-10-06T05:14:31.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 -- 172.21.15.33:0/595515357 learned_addr learned my addr 172.21.15.33:0/595515357 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d219b700 1 --2- 172.21.15.33:0/595515357 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0b0a40 0x7fe7cc06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d299c700 1 --2- 172.21.15.33:0/595515357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b2e30 0x7fe7cc15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 -- 172.21.15.33:0/595515357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b2e30 msgr2=0x7fe7cc15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 --2- 172.21.15.33:0/595515357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b2e30 0x7fe7cc15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 -- 172.21.15.33:0/595515357 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0b0a40 msgr2=0x7fe7cc06ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 --2- 172.21.15.33:0/595515357 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0b0a40 0x7fe7cc06ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d199a700 1 -- 172.21.15.33:0/595515357 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc15dc80 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.275+0000 7fe7d219b700 1 --2- 172.21.15.33:0/595515357 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0b0a40 0x7fe7cc06ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:31.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.276+0000 7fe7d199a700 1 --2- 172.21.15.33:0/595515357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc0b28f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc06a230 tx=0x7fe7bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21cf7942f130042 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.276+0000 7fe7d1199700 1 -- 172.21.15.33:0/595515357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00b640 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.276+0000 7fe7d1199700 1 -- 172.21.15.33:0/595515357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00b7a0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.276+0000 7fe7d1199700 1 -- 172.21.15.33:0/595515357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00bb60 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.276+0000 7fe7d319d700 1 -- 172.21.15.33:0/595515357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 msgr2=0x7fe7cc0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.276+0000 7fe7d319d700 1 --2- 172.21.15.33:0/595515357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc0b28f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc06a230 tx=0x7fe7bc009250 comp rx=0 tx=0).stop 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 -- 172.21.15.33:0/595515357 shutdown_connections 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 --2- 172.21.15.33:0/595515357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b2e30 0x7fe7cc15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 --2- 172.21.15.33:0/595515357 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0b0a40 0x7fe7cc06ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 --2- 172.21.15.33:0/595515357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc0b28f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 -- 172.21.15.33:0/595515357 >> 172.21.15.33:0/595515357 conn(0x7fe7cc06e0f0 msgr2=0x7fe7cc0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 -- 172.21.15.33:0/595515357 shutdown_connections 2023-10-06T05:14:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.277+0000 7fe7d319d700 1 -- 172.21.15.33:0/595515357 wait complete. 2023-10-06T05:14:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.278+0000 7fe7d319d700 1 Processor -- start 2023-10-06T05:14:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.278+0000 7fe7d319d700 1 -- start start 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b0a40 0x7fe7cc15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b2e30 0x7fe7cc1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc15cea0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0c2570 con 0x7fe7cc0b2e30 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0c2850 con 0x7fe7cc0b0a40 2023-10-06T05:14:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc15bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37220/0 (socket says 172.21.15.33:37220) 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d199a700 1 -- 172.21.15.33:0/553491013 learned_addr learned my addr 172.21.15.33:0/553491013 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.279+0000 7fe7d299c700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b2e30 0x7fe7cc1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d219b700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b0a40 0x7fe7cc15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d199a700 1 -- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b0a40 msgr2=0x7fe7cc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d199a700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b0a40 0x7fe7cc15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d199a700 1 -- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b2e30 msgr2=0x7fe7cc1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d199a700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b2e30 0x7fe7cc1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d199a700 1 -- 172.21.15.33:0/553491013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8009420 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d199a700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc15bd70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0ce290 tx=0x7fe7bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00b910 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00ba70 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc011630 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.280+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0094f0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.281+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc0c2e10 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.281+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc06fcc0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.282+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7bc0119a0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.284+0000 7fe7c27fc700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.284+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe7bc085910 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.284+0000 7fe7d219b700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.285+0000 7fe7d219b700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009f40 tx=0x7fe7c80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.288+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7bc052ab0 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.490+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7cc04c380 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.491+0000 7fe7c27fc700 1 -- 172.21.15.33:0/553491013 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fe7bc015090 con 0x7fe7cc0b0e10 2023-10-06T05:14:31.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 msgr2=0x7fe7b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009f40 tx=0x7fe7c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 msgr2=0x7fe7cc15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc15bd70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0ce290 tx=0x7fe7bc017000 comp rx=0 tx=0).stop 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 shutdown_connections 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b2e30 0x7fe7cc1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc0b0a40 0x7fe7cc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 --2- 172.21.15.33:0/553491013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0b0e10 0x7fe7cc15bd70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.493+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 >> 172.21.15.33:0/553491013 conn(0x7fe7cc06e0f0 msgr2=0x7fe7cc06b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.494+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 shutdown_connections 2023-10-06T05:14:31.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.494+0000 7fe7d319d700 1 -- 172.21.15.33:0/553491013 wait complete. 2023-10-06T05:14:31.514 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T05:14:31.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.698+0000 7fca3139d700 1 Processor -- start 2023-10-06T05:14:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.698+0000 7fca3139d700 1 -- start start 2023-10-06T05:14:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca3139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca2c0b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca3139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c06f220 con 0x7fca2c159530 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca3139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c0b0730 con 0x7fca2c06e140 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca3139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0b1e00 con 0x7fca2c0afdd0 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca2bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca2c159530 0x7fca2c06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38946/0 (socket says 172.21.15.33:38946) 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca2bfff700 1 -- 172.21.15.33:0/3508216991 learned_addr learned my addr 172.21.15.33:0/3508216991 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca30b9c700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca2c0b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721270222 0 0) 0x7fca2c06f220 con 0x7fca2c159530 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.700+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca14003570 con 0x7fca2c159530 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941231022 0 0) 0x7fca2c0b0730 con 0x7fca2c06e140 2023-10-06T05:14:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca2c06f220 con 0x7fca2c06e140 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872664658 0 0) 0x7fca14003570 con 0x7fca2c159530 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 msgr2=0x7fca2c0b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca2c0b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 >> v1:172.21.15.167:6789/0 conn(0x7fca2c06e140 legacy=0x7fca2c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c06e610 con 0x7fca2c159530 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fca1c001470 con 0x7fca2c159530 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca30b9c700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca2c0b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 >> v1:172.21.15.33:6789/0 conn(0x7fca2c159530 legacy=0x7fca2c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca14003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca140041b0 0x7fca14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca14004d50 0x7fca140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0b1e00 con 0x7fca140041b0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1400bbf0 con 0x7fca14004d50 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1400bdb0 con 0x7fca2c0afdd0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca140041b0 0x7fca14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca2bfff700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca14003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.701+0000 7fca30b9c700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca14004d50 0x7fca140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 msgr2=0x7fca14003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca2c0afdd0 0x7fca14003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca14004d50 msgr2=0x7fca140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca14004d50 0x7fca140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3508216991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009a20 con 0x7fca140041b0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca140041b0 0x7fca14004810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fca20001a10 tx=0x7fca20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2f66181cf4774f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:31.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000c5d0 con 0x7fca140041b0 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000c730 con 0x7fca140041b0 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.702+0000 7fca2affd700 1 -- 172.21.15.33:0/3508216991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000cad0 con 0x7fca140041b0 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca3139d700 1 -- 172.21.15.33:0/3508216991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca140041b0 msgr2=0x7fca14004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca3139d700 1 --2- 172.21.15.33:0/3508216991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca140041b0 0x7fca14004810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fca20001a10 tx=0x7fca20007d30 comp rx=0 tx=0).stop 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca30b9c700 1 -- 172.21.15.33:0/3508216991 reap_dead start 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca3139d700 1 -- 172.21.15.33:0/3508216991 shutdown_connections 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca3139d700 1 -- 172.21.15.33:0/3508216991 >> 172.21.15.33:0/3508216991 conn(0x7fca2c06c560 msgr2=0x7fca2c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca3139d700 1 -- 172.21.15.33:0/3508216991 shutdown_connections 2023-10-06T05:14:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.703+0000 7fca3139d700 1 -- 172.21.15.33:0/3508216991 wait complete. 2023-10-06T05:14:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.704+0000 7fca3139d700 1 Processor -- start 2023-10-06T05:14:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.704+0000 7fca3139d700 1 -- start start 2023-10-06T05:14:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca3139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca140041b0 0x7fca2c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca3139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 0x7fca2c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca3139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca2c0ceda0 0x7fca2c0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca3139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0cac80 con 0x7fca2c15b100 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca3139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c9d90 con 0x7fca2c0ceda0 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca3139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0ca070 con 0x7fca140041b0 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 0x7fca2c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 0x7fca2c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37238/0 (socket says 172.21.15.33:37238) 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.705+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3179489452 learned_addr learned my addr 172.21.15.33:0/3179489452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca30b9c700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca2c0ceda0 0x7fca2c0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2bfff700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca140041b0 0x7fca2c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca140041b0 msgr2=0x7fca2c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca140041b0 0x7fca2c0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca2c0ceda0 msgr2=0x7fca2c0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca2c0ceda0 0x7fca2c0cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2b7fe700 1 -- 172.21.15.33:0/3179489452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0ca350 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca2b7fe700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 0x7fca2c0ce860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c3c0 tx=0x7fca20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000c5d0 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000c3f0 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000cad0 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.706+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c0ca630 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.707+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c0cb650 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.708+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c04c380 con 0x7fca2c15b100 2023-10-06T05:14:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.708+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca20019070 con 0x7fca2c15b100 2023-10-06T05:14:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.709+0000 7fca13fff700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca18066f30 0x7fca180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.709+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca200855c0 con 0x7fca2c15b100 2023-10-06T05:14:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.710+0000 7fca2bfff700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca18066f30 0x7fca180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.711+0000 7fca2bfff700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca18066f30 0x7fca180693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c001d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.714+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca20052630 con 0x7fca2c15b100 2023-10-06T05:14:31.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.919+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca2c155c90 con 0x7fca2c15b100 2023-10-06T05:14:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.920+0000 7fca13fff700 1 -- 172.21.15.33:0/3179489452 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fca20055c50 con 0x7fca2c15b100 2023-10-06T05:14:31.923 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:31.924 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:31.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca18066f30 msgr2=0x7fca180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca18066f30 0x7fca180693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c001d20 comp rx=0 tx=0).stop 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 msgr2=0x7fca2c0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 0x7fca2c0ce860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c3c0 tx=0x7fca20015000 comp rx=0 tx=0).stop 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 shutdown_connections 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca18066f30 0x7fca180693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca2c0ceda0 0x7fca2c0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca140041b0 0x7fca2c0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 --2- 172.21.15.33:0/3179489452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca2c15b100 0x7fca2c0ce860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 >> 172.21.15.33:0/3179489452 conn(0x7fca2c06c560 msgr2=0x7fca2c158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:31.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 shutdown_connections 2023-10-06T05:14:31.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:31.923+0000 7fca3139d700 1 -- 172.21.15.33:0/3179489452 wait complete. 2023-10-06T05:14:31.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T05:14:31.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T05:14:31.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T05:14:31.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T05:14:31.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T05:14:31.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T05:14:31.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T05:14:31.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.273+0000 7f1aa5d9d700 1 Processor -- start 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.273+0000 7f1aa5d9d700 1 -- start start 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.275+0000 7f1aa5d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a980e9a50 0x7f1a98006300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.275+0000 7f1aa5d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a98039160 0x7f1a9803bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.275+0000 7f1aa5d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a98006be0 con 0x7f1a980e9e20 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.275+0000 7f1aa5d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a98007c80 con 0x7f1a980e9a50 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.275+0000 7f1aa5d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a98006ec0 con 0x7f1a98039160 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.276+0000 7f1aa4d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a980e9a50 0x7f1a98006300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.276+0000 7f1aa4d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a980e9a50 0x7f1a98006300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37248/0 (socket says 172.21.15.33:37248) 2023-10-06T05:14:32.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.276+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2643581347 learned_addr learned my addr 172.21.15.33:0/2643581347 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.276+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2643581347 >> v2:172.21.15.167:3300/0 conn(0x7f1a98039160 msgr2=0x7f1a9803bf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.276+0000 7f1aa4d9b700 1 --2- 172.21.15.33:0/2643581347 >> v2:172.21.15.167:3300/0 conn(0x7f1a98039160 0x7f1a9803bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.277+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2643581347 >> v1:172.21.15.167:6789/0 conn(0x7f1a980e9e20 legacy=0x7f1a98038a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.277+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2643581347 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a980071a0 con 0x7f1a980e9a50 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.277+0000 7f1aa4d9b700 1 --2- 172.21.15.33:0/2643581347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9a50 0x7f1a98006300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e220 tx=0x7f1a8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7b936f2e2c112b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.278+0000 7f1a9f7fe700 1 -- 172.21.15.33:0/2643581347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00b7e0 con 0x7f1a980e9a50 2023-10-06T05:14:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.278+0000 7f1a9f7fe700 1 -- 172.21.15.33:0/2643581347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c00b940 con 0x7f1a980e9a50 2023-10-06T05:14:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1a9f7fe700 1 -- 172.21.15.33:0/2643581347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00bce0 con 0x7f1a980e9a50 2023-10-06T05:14:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2643581347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9a50 msgr2=0x7f1a98006300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2643581347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9a50 0x7f1a98006300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e220 tx=0x7f1a8c009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2643581347 shutdown_connections 2023-10-06T05:14:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2643581347 >> v2:172.21.15.167:3300/0 conn(0x7f1a98039160 0x7f1a9803bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2643581347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9a50 0x7f1a98006300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2643581347 >> 172.21.15.33:0/2643581347 conn(0x7f1a9800f1c0 msgr2=0x7f1a98059180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2643581347 shutdown_connections 2023-10-06T05:14:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.279+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2643581347 wait complete. 2023-10-06T05:14:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.280+0000 7f1f465ff700 1 Processor -- start 2023-10-06T05:14:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.282+0000 7f1aa5d9d700 1 Processor -- start 2023-10-06T05:14:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f1aa5d9d700 1 -- start start 2023-10-06T05:14:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.280+0000 7f89507ff700 1 Processor -- start 2023-10-06T05:14:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.281+0000 7f89507ff700 1 -- start start 2023-10-06T05:14:32.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.281+0000 7f1f465ff700 1 -- start start 2023-10-06T05:14:32.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f1f465ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f380e3e70 0x7f1f380e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f1f465ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 0x7f1f380ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f1f465ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f38006ea0 con 0x7f1f38006930 2023-10-06T05:14:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f1f465ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f380eab60 con 0x7f1f380e3e70 2023-10-06T05:14:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f1f465ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38007180 con 0x7f1f380ea1a0 2023-10-06T05:14:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f1f44b9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 0x7f1f380ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f1f44b9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 0x7f1f380ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41698/0 (socket says 172.21.15.33:41698) 2023-10-06T05:14:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f1f44b9c700 1 -- 172.21.15.33:0/2436288782 learned_addr learned my addr 172.21.15.33:0/2436288782 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1aa5d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a98039160 0x7f1a980da320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1f3ffff700 1 --2- 172.21.15.33:0/2436288782 >> v2:172.21.15.167:3300/0 conn(0x7f1f380e3e70 0x7f1f380e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1aa5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a980e9a50 0x7f1a980da860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1aa4d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a98039160 0x7f1a980da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1aa4d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a98039160 0x7f1a980da320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40106/0 (socket says 172.21.15.33:40106) 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1f44b9c700 1 -- 172.21.15.33:0/2436288782 >> v2:172.21.15.167:3300/0 conn(0x7f1f380e3e70 msgr2=0x7f1f380e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1f44b9c700 1 --2- 172.21.15.33:0/2436288782 >> v2:172.21.15.167:3300/0 conn(0x7f1f380e3e70 0x7f1f380e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1f44b9c700 1 -- 172.21.15.33:0/2436288782 >> v1:172.21.15.167:6789/0 conn(0x7f1f38006930 legacy=0x7f1f3804fe30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:14:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f1f44b9c700 1 -- 172.21.15.33:0/2436288782 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38007460 con 0x7f1f380ea1a0 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f89507ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89480b2200 0x7f89480d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f89507ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89480b25d0 0x7f8948161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f89507ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8948161fb0 con 0x7f8948161a50 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f89507ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480d1420 con 0x7f89480b25d0 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.283+0000 7f89507ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8948162290 con 0x7f89480b2200 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89480b2200 0x7f89480d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89480b2200 0x7f89480d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40094/0 (socket says 172.21.15.33:40094) 2023-10-06T05:14:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 -- 172.21.15.33:0/1150404452 learned_addr learned my addr 172.21.15.33:0/1150404452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2489959372 learned_addr learned my addr 172.21.15.33:0/2489959372 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894dd9a700 1 --2- 172.21.15.33:0/1150404452 >> v2:172.21.15.33:3300/0 conn(0x7f89480b25d0 0x7f8948161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 -- 172.21.15.33:0/1150404452 >> v2:172.21.15.33:3300/0 conn(0x7f89480b25d0 msgr2=0x7f8948161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 --2- 172.21.15.33:0/1150404452 >> v2:172.21.15.33:3300/0 conn(0x7f89480b25d0 0x7f8948161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 -- 172.21.15.33:0/1150404452 >> v1:172.21.15.33:6789/0 conn(0x7f8948161a50 legacy=0x7f89481578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.284+0000 7f894e59b700 1 -- 172.21.15.33:0/1150404452 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89480d0400 con 0x7f89480b2200 2023-10-06T05:14:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.285+0000 7f894e59b700 1 --2- 172.21.15.33:0/1150404452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b2200 0x7f89480d0ee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89480895c0 tx=0x7f8940009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdcade5fcc72cb55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9e20 0x7f1a9800ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1f44b9c700 1 --2- 172.21.15.33:0/2436288782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 0x7f1f380ea620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1f340099b0 tx=0x7f1f34009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c84d032b4f0b3d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1a9ffff700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a980e9a50 0x7f1a980da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1f3effd700 1 -- 172.21.15.33:0/2436288782 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400b820 con 0x7f1f380ea1a0 2023-10-06T05:14:32.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1f3effd700 1 -- 172.21.15.33:0/2436288782 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3400b980 con 0x7f1f380ea1a0 2023-10-06T05:14:32.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.287+0000 7f1aa559c700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9e20 0x7f1a9800ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.286+0000 7f1f3effd700 1 -- 172.21.15.33:0/2436288782 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400bd20 con 0x7f1f380ea1a0 2023-10-06T05:14:32.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.287+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980e6af0 con 0x7f1a980e9e20 2023-10-06T05:14:32.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.287+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9800d320 con 0x7f1a98039160 2023-10-06T05:14:32.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.287+0000 7f1f465ff700 1 -- 172.21.15.33:0/2436288782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 msgr2=0x7f1f380ea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.287+0000 7f1f465ff700 1 --2- 172.21.15.33:0/2436288782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 0x7f1f380ea620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1f340099b0 tx=0x7f1f34009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.287+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9800c710 con 0x7f1a980e9a50 2023-10-06T05:14:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1f465ff700 1 -- 172.21.15.33:0/2436288782 shutdown_connections 2023-10-06T05:14:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1f465ff700 1 --2- 172.21.15.33:0/2436288782 >> v2:172.21.15.167:3300/0 conn(0x7f1f380e3e70 0x7f1f380e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1f465ff700 1 --2- 172.21.15.33:0/2436288782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380ea1a0 0x7f1f380ea620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1f465ff700 1 -- 172.21.15.33:0/2436288782 >> 172.21.15.33:0/2436288782 conn(0x7f1f38054c90 msgr2=0x7f1f38055090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1f465ff700 1 -- 172.21.15.33:0/2436288782 shutdown_connections 2023-10-06T05:14:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1f465ff700 1 -- 172.21.15.33:0/2436288782 wait complete. 2023-10-06T05:14:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f894d599700 1 -- 172.21.15.33:0/1150404452 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894000b820 con 0x7f89480b2200 2023-10-06T05:14:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f894d599700 1 -- 172.21.15.33:0/1150404452 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894000b980 con 0x7f89480b2200 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a980e9a50 msgr2=0x7f1a980da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1aa4d9b700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a980e9a50 0x7f1a980da860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9e20 msgr2=0x7f1a9800ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1aa4d9b700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9e20 0x7f1a9800ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1aa4d9b700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c0094f0 con 0x7f1a98039160 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.288+0000 7f1aa4d9b700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a98039160 0x7f1a980da320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009290 tx=0x7f1a8c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.289+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c01b450 con 0x7f1a98039160 2023-10-06T05:14:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.289+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c020070 con 0x7f1a98039160 2023-10-06T05:14:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.289+0000 7f894d599700 1 -- 172.21.15.33:0/1150404452 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894000bd00 con 0x7f89480b2200 2023-10-06T05:14:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f1f465ff700 1 Processor -- start 2023-10-06T05:14:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 -- 172.21.15.33:0/1150404452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b2200 msgr2=0x7f89480d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 --2- 172.21.15.33:0/1150404452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b2200 0x7f89480d0ee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89480895c0 tx=0x7f8940009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 -- 172.21.15.33:0/1150404452 shutdown_connections 2023-10-06T05:14:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 --2- 172.21.15.33:0/1150404452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b2200 0x7f89480d0ee0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 --2- 172.21.15.33:0/1150404452 >> v2:172.21.15.33:3300/0 conn(0x7f89480b25d0 0x7f8948161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 -- 172.21.15.33:0/1150404452 >> 172.21.15.33:0/1150404452 conn(0x7f894806c560 msgr2=0x7f894806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f1f465ff700 1 -- start start 2023-10-06T05:14:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c01b820 con 0x7f1a98039160 2023-10-06T05:14:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f1aa559c700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a980e9e20 0x7f1a9800ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.291+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9800c9f0 con 0x7f1a98039160 2023-10-06T05:14:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.291+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9800cfb0 con 0x7f1a98039160 2023-10-06T05:14:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.290+0000 7f89507ff700 1 -- 172.21.15.33:0/1150404452 shutdown_connections 2023-10-06T05:14:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.291+0000 7f89507ff700 1 -- 172.21.15.33:0/1150404452 wait complete. 2023-10-06T05:14:32.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.292+0000 7f1f465ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f38006930 0x7f1f380ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.292+0000 7f1f465ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380e3e70 0x7f1f38037c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.292+0000 7f1f465ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f380ea1a0 0x7f1f380d4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f3f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380e3e70 0x7f1f38037c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f89507ff700 1 Processor -- start 2023-10-06T05:14:32.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f465ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380d3f80 con 0x7f1f38006930 2023-10-06T05:14:32.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a98005930 con 0x7f1a98039160 2023-10-06T05:14:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f465ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380d1f70 con 0x7f1f380ea1a0 2023-10-06T05:14:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f465ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380d3930 con 0x7f1f380e3e70 2023-10-06T05:14:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f3f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380e3e70 0x7f1f38037c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41718/0 (socket says 172.21.15.33:41718) 2023-10-06T05:14:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f3f7fe700 1 -- 172.21.15.33:0/3852623926 learned_addr learned my addr 172.21.15.33:0/3852623926 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f38006930 0x7f1f380ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f1f44b9c700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f380ea1a0 0x7f1f380d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f89507ff700 1 -- start start 2023-10-06T05:14:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.293+0000 7f49acdff700 1 Processor -- start 2023-10-06T05:14:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3f7fe700 1 -- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f380ea1a0 msgr2=0x7f1f380d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3f7fe700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f380ea1a0 0x7f1f380d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3f7fe700 1 -- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f38006930 msgr2=0x7f1f380ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3f7fe700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f38006930 0x7f1f380ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3f7fe700 1 -- 172.21.15.33:0/3852623926 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340094f0 con 0x7f1f380e3e70 2023-10-06T05:14:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f49acdff700 1 -- start start 2023-10-06T05:14:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f44b9c700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f380ea1a0 0x7f1f380d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3f7fe700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380e3e70 0x7f1f38037c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f4004c820 tx=0x7f1f40065f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f89507ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89480b2200 0x7f89481447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f89507ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b25d0 0x7f89481565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f2fc699d700 1 Processor -- start 2023-10-06T05:14:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f2fc699d700 1 -- start start 2023-10-06T05:14:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f89507ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8948161a50 0x7f89480cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f894e59b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89480b2200 0x7f89481447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f894e59b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89480b2200 0x7f89481447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41728/0 (socket says 172.21.15.33:41728) 2023-10-06T05:14:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f894e59b700 1 -- 172.21.15.33:0/2355467436 learned_addr learned my addr 172.21.15.33:0/2355467436 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a8c022030 con 0x7f1a98039160 2023-10-06T05:14:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400647c0 con 0x7f1f380e3e70 2023-10-06T05:14:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4006d6e0 con 0x7f1f380e3e70 2023-10-06T05:14:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4006d890 con 0x7f1f380e3e70 2023-10-06T05:14:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f1f465ff700 1 -- 172.21.15.33:0/3852623926 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380d3c10 con 0x7f1f380e3e70 2023-10-06T05:14:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.295+0000 7f1f465ff700 1 -- 172.21.15.33:0/3852623926 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f38038180 con 0x7f1f380e3e70 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.294+0000 7f1f3ffff700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f38006930 0x7f1f380ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f380d3790 con 0x7f1f380e3e70 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f894ed9c700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8948161a50 0x7f89480cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f894dd9a700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b25d0 0x7f89481565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8948156060 con 0x7f8948161a50 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8948155bd0 con 0x7f89480b25d0 2023-10-06T05:14:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8948154ce0 con 0x7f89480b2200 2023-10-06T05:14:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f49acdff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49a81596f0 0x7f49a80c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f49a6d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f49a8067c60 0x7f49a8068080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38962/0 (socket says 172.21.15.33:38962) 2023-10-06T05:14:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.296+0000 7f49a6d9d700 1 -- 172.21.15.33:0/3114769118 learned_addr learned my addr 172.21.15.33:0/3114769118 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a80c6e80 con 0x7f49a8067c60 2023-10-06T05:14:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a8156000 con 0x7f49a8155b60 2023-10-06T05:14:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a8159ac0 con 0x7f49a81596f0 2023-10-06T05:14:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f894e59b700 1 -- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b25d0 msgr2=0x7f89481565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f894e59b700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89480b25d0 0x7f89481565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f894e59b700 1 -- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8948161a50 msgr2=0x7f89480cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f894e59b700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8948161a50 0x7f89480cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f894e59b700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400094f0 con 0x7f89480b2200 2023-10-06T05:14:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f2fc699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2fb8037320 0x7f2fb80e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255729877 0 0) 0x7f49a80c6e80 con 0x7f49a8067c60 2023-10-06T05:14:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f49a659c700 1 --2- 172.21.15.33:0/3114769118 >> v2:172.21.15.33:3300/0 conn(0x7f49a81596f0 0x7f49a80c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f2fc699d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fb80e62d0 0x7f2fb80531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f894e59b700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89480b2200 0x7f89481447d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8940006fd0 tx=0x7f8940027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f1a9cff9700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a84067000 0x7f1a840694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.297+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a8c0867d0 con 0x7f1a98039160 2023-10-06T05:14:32.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f1a9cff9700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f1a8406f6c0 0x7f1a84071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a84072180 con 0x7f1a8406f6c0 2023-10-06T05:14:32.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a8c0538c0 con 0x7f1a98039160 2023-10-06T05:14:32.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f1a9ffff700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a84067000 0x7f1a840694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f2fc519a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2fb8037320 0x7f2fb80e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4990003570 con 0x7f49a8067c60 2023-10-06T05:14:32.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404822077 0 0) 0x7f49a8156000 con 0x7f49a8155b60 2023-10-06T05:14:32.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a80c6e80 con 0x7f49a8155b60 2023-10-06T05:14:32.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f1aa559c700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f1a8406f6c0 0x7f1a84071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f1aa559c700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f1a8406f6c0 0x7f1a84071ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940022b70 con 0x7f89480b2200 2023-10-06T05:14:32.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8940022cd0 con 0x7f89480b2200 2023-10-06T05:14:32.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f2fc699d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb80e6700 con 0x7f2fb8036f50 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f2fc699d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb80e6c00 con 0x7f2fb80e62d0 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f2fc519a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2fb8037320 0x7f2fb80e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40142/0 (socket says 172.21.15.33:40142) 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f2fc519a700 1 -- 172.21.15.33:0/2747000052 learned_addr learned my addr 172.21.15.33:0/2747000052 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f2fc619c700 1 --2- 172.21.15.33:0/2747000052 >> v2:172.21.15.33:3300/0 conn(0x7f2fb80e62d0 0x7f2fb80531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.298+0000 7f2fc599b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2fb8036f50 0x7f2fb804fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38978/0 (socket says 172.21.15.33:38978) 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f2fc699d700 1 -- 172.21.15.33:0/2747000052 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb80068f0 con 0x7f2fb8037320 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f2fc619c700 1 -- 172.21.15.33:0/2747000052 >> v2:172.21.15.167:3300/0 conn(0x7f2fb8037320 msgr2=0x7f2fb80e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f2fc619c700 1 --2- 172.21.15.33:0/2747000052 >> v2:172.21.15.167:3300/0 conn(0x7f2fb8037320 0x7f2fb80e5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f2fc619c700 1 -- 172.21.15.33:0/2747000052 >> v1:172.21.15.33:6789/0 conn(0x7f2fb8036f50 legacy=0x7f2fb804fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f2fc619c700 1 -- 172.21.15.33:0/2747000052 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8006b70 con 0x7f2fb80e62d0 2023-10-06T05:14:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f2fc619c700 1 --2- 172.21.15.33:0/2747000052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb80e62d0 0x7f2fb80531a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0000eb0 tx=0x7f2fc0065f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fcacc86575dcec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125062813 0 0) 0x7f4990003570 con 0x7f49a8067c60 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 >> v2:172.21.15.33:3300/0 conn(0x7f49a81596f0 msgr2=0x7f49a80c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 --2- 172.21.15.33:0/3114769118 >> v2:172.21.15.33:3300/0 conn(0x7f49a81596f0 0x7f49a80c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 >> v1:172.21.15.167:6789/0 conn(0x7f49a8155b60 legacy=0x7f49a8070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80c76d0 con 0x7f49a8067c60 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f499c001470 con 0x7f49a8067c60 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 >> v1:172.21.15.33:6789/0 conn(0x7f49a8067c60 legacy=0x7f49a8068080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4990004c50 0x7f4990005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49900037b0 0x7f4990003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f499000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8159ac0 con 0x7f4990004160 2023-10-06T05:14:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990014970 con 0x7f4990004c50 2023-10-06T05:14:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.299+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990014b30 con 0x7f49900037b0 2023-10-06T05:14:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f4008d020 con 0x7f1f380e3e70 2023-10-06T05:14:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894002c6c0 con 0x7f89480b2200 2023-10-06T05:14:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f2fc4999700 1 -- 172.21.15.33:0/2747000052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0065810 con 0x7f2fb80e62d0 2023-10-06T05:14:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8948154fc0 con 0x7f89480b2200 2023-10-06T05:14:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f2fc4999700 1 -- 172.21.15.33:0/2747000052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc00631f0 con 0x7f2fb80e62d0 2023-10-06T05:14:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8948155580 con 0x7f89480b2200 2023-10-06T05:14:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f2fc4999700 1 -- 172.21.15.33:0/2747000052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0065810 con 0x7f2fb80e62d0 2023-10-06T05:14:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f499000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== osd.1 v2:172.21.15.33:6816/85579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1a84072180 con 0x7f1a8406f6c0 2023-10-06T05:14:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f49a5d9b700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49900037b0 0x7f4990003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f1454b9d700 1 Processor -- start 2023-10-06T05:14:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.300+0000 7f1454b9d700 1 -- start start 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.301+0000 7f49a659c700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4990004c50 0x7f4990005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1f2bfff700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f24067000 0x7f1f240694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1f400fb500 con 0x7f1f380e3e70 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1f3ffff700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f24067000 0x7f1f240694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1f2bfff700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f1f2406f8b0 0x7f1f24071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 --> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f24072370 con 0x7f1f2406f8b0 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f400fb7f0 con 0x7f1f380e3e70 2023-10-06T05:14:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1f44b9c700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f1f2406f8b0 0x7f1f24071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f1a9ffff700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a84067000 0x7f1a840694b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800ddb0 tx=0x7f1a94009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f2fc699d700 1 -- 172.21.15.33:0/2747000052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb80e62d0 msgr2=0x7f2fb80531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f2fc699d700 1 --2- 172.21.15.33:0/2747000052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb80e62d0 0x7f2fb80531a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0000eb0 tx=0x7f2fc0065f70 comp rx=0 tx=0).stop 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f2fc699d700 1 -- 172.21.15.33:0/2747000052 shutdown_connections 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f2fc699d700 1 --2- 172.21.15.33:0/2747000052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb80e62d0 0x7f2fb80531a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f2fc699d700 1 --2- 172.21.15.33:0/2747000052 >> v2:172.21.15.167:3300/0 conn(0x7f2fb8037320 0x7f2fb80e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f2fc699d700 1 -- 172.21.15.33:0/2747000052 >> 172.21.15.33:0/2747000052 conn(0x7f2fb8054c80 msgr2=0x7f2fb8055080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 -- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49900037b0 msgr2=0x7f4990003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49900037b0 0x7f4990003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 -- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4990004c50 msgr2=0x7f4990005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4990004c50 0x7f4990005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 -- 172.21.15.33:0/3114769118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80c76d0 con 0x7f4990004160 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 -- 172.21.15.33:0/3114769118 reap_dead start 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f499000ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c320 tx=0x7f499c00c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ceabc679a452462 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00cc40 con 0x7f4990004160 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c00cdc0 con 0x7f4990004160 2023-10-06T05:14:32.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f49a559a700 1 -- 172.21.15.33:0/3114769118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c005b70 con 0x7f4990004160 2023-10-06T05:14:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7fef2cbff700 1 Processor -- start 2023-10-06T05:14:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.302+0000 7fef2cbff700 1 -- start start 2023-10-06T05:14:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1454b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 0x7f14480de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1454b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f144800c250 0x7f1448053330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1454b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1448053870 0x7f144804ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1454b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1448054690 con 0x7f1448053870 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1454b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480061f0 con 0x7f144800c250 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f1454b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1448053df0 con 0x7f14480de030 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f144f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 0x7f14480de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f144f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 0x7f14480de400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41752/0 (socket says 172.21.15.33:41752) 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f144f7fe700 1 -- 172.21.15.33:0/2330212731 learned_addr learned my addr 172.21.15.33:0/2330212731 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f144ffff700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.33:3300/0 conn(0x7f1448053870 0x7f144804ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.303+0000 7f144effd700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.167:3300/0 conn(0x7f144800c250 0x7f1448053330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144f7fe700 1 -- 172.21.15.33:0/2330212731 >> v2:172.21.15.167:3300/0 conn(0x7f144800c250 msgr2=0x7f1448053330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f894804c380 con 0x7f89480b2200 2023-10-06T05:14:32.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144f7fe700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.167:3300/0 conn(0x7f144800c250 0x7f1448053330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 msgr2=0x7f499000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f49acdff700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f499000ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c320 tx=0x7f499c00c630 comp rx=0 tx=0).stop 2023-10-06T05:14:32.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144f7fe700 1 -- 172.21.15.33:0/2330212731 >> v2:172.21.15.33:3300/0 conn(0x7f1448053870 msgr2=0x7f144804ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144f7fe700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.33:3300/0 conn(0x7f1448053870 0x7f144804ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144f7fe700 1 -- 172.21.15.33:0/2330212731 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480540d0 con 0x7f14480de030 2023-10-06T05:14:32.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144ffff700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.33:3300/0 conn(0x7f1448053870 0x7f144804ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:32.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f144f7fe700 1 --2- 172.21.15.33:0/2330212731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 0x7f14480de400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f144800b4c0 tx=0x7f1444009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caf34bab2a1b09a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.304+0000 7f1f44b9c700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f1f2406f8b0 0x7f1f24071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 shutdown_connections 2023-10-06T05:14:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f49acdff700 1 --2- 172.21.15.33:0/3114769118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f499000ba60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 >> 172.21.15.33:0/3114769118 conn(0x7f49a806f640 msgr2=0x7f49a80cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 shutdown_connections 2023-10-06T05:14:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f49acdff700 1 -- 172.21.15.33:0/3114769118 wait complete. 2023-10-06T05:14:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f2fc699d700 1 -- 172.21.15.33:0/2747000052 shutdown_connections 2023-10-06T05:14:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f2fc699d700 1 -- 172.21.15.33:0/2747000052 wait complete. 2023-10-06T05:14:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7fef2cbff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fef200de3a0 0x7fef2004ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7fef2cbff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef20050470 0x7fef20033b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== osd.0 v2:172.21.15.33:6800/85577 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1f24072370 con 0x7f1f2406f8b0 2023-10-06T05:14:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7fef2cbff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef20053cf0 con 0x7fef200ddfd0 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef20050470 0x7fef20033b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef20050470 0x7fef20033b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41758/0 (socket says 172.21.15.33:41758) 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7fef26d9d700 1 -- 172.21.15.33:0/3717083698 learned_addr learned my addr 172.21.15.33:0/3717083698 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7fef2659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fef200ddfd0 0x7fef200060c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38990/0 (socket says 172.21.15.33:38990) 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.305+0000 7f144e7fc700 1 -- 172.21.15.33:0/2330212731 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144400b640 con 0x7f14480de030 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7f144e7fc700 1 -- 172.21.15.33:0/2330212731 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144400b7a0 con 0x7f14480de030 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.306+0000 7f144e7fc700 1 -- 172.21.15.33:0/2330212731 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144400bb60 con 0x7f14480de030 2023-10-06T05:14:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 -- 172.21.15.33:0/2330212731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 msgr2=0x7f14480de400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2330212731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 0x7f14480de400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f144800b4c0 tx=0x7f1444009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200de820 con 0x7fef200de3a0 2023-10-06T05:14:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20050980 con 0x7fef20050470 2023-10-06T05:14:32.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 -- 172.21.15.33:0/2330212731 shutdown_connections 2023-10-06T05:14:32.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.33:3300/0 conn(0x7f1448053870 0x7f144804ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2330212731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14480de030 0x7f14480de400 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f894002ca30 con 0x7f89480b2200 2023-10-06T05:14:32.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2330212731 >> v2:172.21.15.167:3300/0 conn(0x7f144800c250 0x7f1448053330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7f1454b9d700 1 -- 172.21.15.33:0/2330212731 >> 172.21.15.33:0/2330212731 conn(0x7f1448054da0 msgr2=0x7f14480551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.307+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481214147 0 0) 0x7fef20053cf0 con 0x7fef200ddfd0 2023-10-06T05:14:32.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef25d9b700 1 --2- 172.21.15.33:0/3717083698 >> v2:172.21.15.167:3300/0 conn(0x7fef200de3a0 0x7fef2004ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef10003570 con 0x7fef200ddfd0 2023-10-06T05:14:32.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2085899447 0 0) 0x7fef10003570 con 0x7fef200ddfd0 2023-10-06T05:14:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef20050470 msgr2=0x7fef20033b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef20050470 0x7fef20033b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 >> v2:172.21.15.167:3300/0 conn(0x7fef200de3a0 msgr2=0x7fef2004ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 --2- 172.21.15.33:0/3717083698 >> v2:172.21.15.167:3300/0 conn(0x7fef200de3a0 0x7fef2004ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7f49acdff700 1 Processor -- start 2023-10-06T05:14:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.308+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200533c0 con 0x7fef200ddfd0 2023-10-06T05:14:32.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7f2fc699d700 1 Processor -- start 2023-10-06T05:14:32.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7f49acdff700 1 -- start start 2023-10-06T05:14:32.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fef1c001470 con 0x7fef200ddfd0 2023-10-06T05:14:32.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 >> v1:172.21.15.33:6789/0 conn(0x7fef200ddfd0 legacy=0x7fef200060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7f1f3ffff700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f24067000 0x7f1f240694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c006fd0 tx=0x7f1f2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7f2fc699d700 1 -- start start 2023-10-06T05:14:32.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7f89467fc700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8934067000 0x7f89340694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.309+0000 7fef2559a700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef10003b60 0x7fef10003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7f1454b9d700 1 -- 172.21.15.33:0/2330212731 shutdown_connections 2023-10-06T05:14:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7f1454b9d700 1 -- 172.21.15.33:0/2330212731 wait complete. 2023-10-06T05:14:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8940097380 con 0x7f89480b2200 2023-10-06T05:14:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2559a700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10004810 0x7fef10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2559a700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef10005310 0x7fef10009a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200de820 con 0x7fef10004810 2023-10-06T05:14:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef20050980 con 0x7fef10003b60 2023-10-06T05:14:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1000a220 con 0x7fef10005310 2023-10-06T05:14:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2659c700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef10003b60 0x7fef10003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7f89467fc700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7f893406f740 0x7f8934071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fef2659c700 1 -- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef10005310 msgr2=0x7fef10009a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:14:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f49acdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f49a80adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7fef2659c700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef10005310 0x7fef10009a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8934072200 con 0x7f893406f740 2023-10-06T05:14:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89400976b0 con 0x7f89480b2200 2023-10-06T05:14:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f49acdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a8153520 0x7f49a8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f894ed9c700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7f893406f740 0x7f8934071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7fef2659c700 1 -- 172.21.15.33:0/3717083698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10004810 msgr2=0x7fef10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7fef2659c700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10004810 0x7fef10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f49acdff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49a8152d30 0x7f49a80c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7fef2659c700 1 -- 172.21.15.33:0/3717083698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10003570 con 0x7fef10003b60 2023-10-06T05:14:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7fef2659c700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef10003b60 0x7fef10003f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef1c013d00 tx=0x7fef1c008ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b59ff68f5b6ed0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f49acdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a80ae1b0 con 0x7f4990004160 2023-10-06T05:14:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f49a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a8153520 0x7f49a8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f49acdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a80ad220 con 0x7f49a8153520 2023-10-06T05:14:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f49a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a8153520 0x7f49a8158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40202/0 (socket says 172.21.15.33:40202) 2023-10-06T05:14:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f49a5d9b700 1 -- 172.21.15.33:0/4036934934 learned_addr learned my addr 172.21.15.33:0/4036934934 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7f49a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f49a80adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49a8152d30 0x7f49a80c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c018070 con 0x7fef10003b60 2023-10-06T05:14:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8159270 con 0x7f49a8152d30 2023-10-06T05:14:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c013e00 con 0x7fef10003b60 2023-10-06T05:14:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f894ed9c700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7f893406f740 0x7f8934071b50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f894dd9a700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8934067000 0x7f89340694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7fef2559a700 1 -- 172.21.15.33:0/3717083698 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0054c0 con 0x7fef10003b60 2023-10-06T05:14:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f1454b9d700 1 Processor -- start 2023-10-06T05:14:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef10003b60 msgr2=0x7fef10003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7fef2cbff700 1 --2- 172.21.15.33:0/3717083698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef10003b60 0x7fef10003f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef1c013d00 tx=0x7fef1c008ae0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f49a5d9b700 1 -- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49a8152d30 msgr2=0x7f49a80c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f49a5d9b700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49a8152d30 0x7f49a80c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f49a5d9b700 1 -- 172.21.15.33:0/4036934934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 msgr2=0x7f49a80adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f49a5d9b700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f49a80adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f49a5d9b700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80c0cb0 con 0x7f49a8153520 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f49a659c700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4990004160 0x7f49a80adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== osd.2 v2:172.21.15.33:6808/85578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8934072200 con 0x7f893406f740 2023-10-06T05:14:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f49a5d9b700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a8153520 0x7f49a8158d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f499800f8e0 tx=0x7f4998007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f2fc699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8036f50 0x7f2fb80dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f2fc699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb8037320 0x7f2fb804bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f2fc699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fb80e62d0 0x7f2fb80e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f2fc699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb804a7b0 con 0x7f2fb8036f50 2023-10-06T05:14:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49a8152d30 0x7f49a80c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f2fc699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8049ba0 con 0x7f2fb8037320 2023-10-06T05:14:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7f2fc699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8049e80 con 0x7f2fb80e62d0 2023-10-06T05:14:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fef26d9d700 1 -- 172.21.15.33:0/3717083698 reap_dead start 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.312+0000 7f1454b9d700 1 -- start start 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f1454b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144800c250 0x7f144804e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f1454b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1448053870 0x7f144804c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f144f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144800c250 0x7f144804e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f144f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144800c250 0x7f144804e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37386/0 (socket says 172.21.15.33:37386) 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f2fc599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8036f50 0x7f2fb80dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 shutdown_connections 2023-10-06T05:14:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f1454b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14480de030 0x7f14480dc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 >> 172.21.15.33:0/3717083698 conn(0x7fef20054e30 msgr2=0x7fef20055230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f2fc599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8036f50 0x7f2fb80dc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37374/0 (socket says 172.21.15.33:37374) 2023-10-06T05:14:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f144effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1448053870 0x7f144804c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f144effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1448053870 0x7f144804c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41806/0 (socket says 172.21.15.33:41806) 2023-10-06T05:14:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 shutdown_connections 2023-10-06T05:14:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fef2cbff700 1 -- 172.21.15.33:0/3717083698 wait complete. 2023-10-06T05:14:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7f2fc599b700 1 -- 172.21.15.33:0/1680961132 learned_addr learned my addr 172.21.15.33:0/1680961132 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f1454b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144804cda0 con 0x7f144800c250 2023-10-06T05:14:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f1454b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144804d8a0 con 0x7f14480de030 2023-10-06T05:14:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f1454b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144804d000 con 0x7f1448053870 2023-10-06T05:14:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f144f7fe700 1 -- 172.21.15.33:0/2316200163 learned_addr learned my addr 172.21.15.33:0/2316200163 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f144ffff700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14480de030 0x7f14480dc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.310+0000 7fc05a5ff700 1 Processor -- start 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.311+0000 7fc05a5ff700 1 -- start start 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7fc05a5ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc05415dad0 0x7fc0540ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.313+0000 7fc05a5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc05406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc05a5ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05406ebb0 con 0x7fc05406e750 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc05a5ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0540cad80 con 0x7fc05415dad0 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc05a5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540c7330 con 0x7fc0540ca910 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc053fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc05415dad0 0x7fc0540ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc053fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc05415dad0 0x7fc0540ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40206/0 (socket says 172.21.15.33:40206) 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc053fff700 1 -- 172.21.15.33:0/1763602766 learned_addr learned my addr 172.21.15.33:0/1763602766 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.314+0000 7fc04bfff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc05406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc053fff700 1 -- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 msgr2=0x7fc05406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc053fff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc05406e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc053fff700 1 -- 172.21.15.33:0/1763602766 >> v1:172.21.15.167:6789/0 conn(0x7fc05406e750 legacy=0x7fc054161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc053fff700 1 -- 172.21.15.33:0/1763602766 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540c8190 con 0x7fc05415dad0 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc04bfff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc05406e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc053fff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415dad0 0x7fc0540ca3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc044009f20 tx=0x7fc044009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c232d748a6dfac25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7fc0537fe700 1 -- 172.21.15.33:0/1763602766 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400e070 con 0x7fc05415dad0 2023-10-06T05:14:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7fc0537fe700 1 -- 172.21.15.33:0/1763602766 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04400ba40 con 0x7fc05415dad0 2023-10-06T05:14:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f894dd9a700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8934067000 0x7f89340694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89481484d0 tx=0x7f893c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f144ffff700 1 -- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1448053870 msgr2=0x7f144804c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f144ffff700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1448053870 0x7f144804c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f144ffff700 1 -- 172.21.15.33:0/2316200163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144800c250 msgr2=0x7f144804e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f144ffff700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144800c250 0x7f144804e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f144ffff700 1 -- 172.21.15.33:0/2316200163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009420 con 0x7f14480de030 2023-10-06T05:14:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f144ffff700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14480de030 0x7f14480dc940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f143c00ea50 tx=0x7f143c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c00c920 con 0x7f14480de030 2023-10-06T05:14:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7fc05a5ff700 1 -- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415dad0 msgr2=0x7fc0540ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7fc05a5ff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415dad0 0x7fc0540ca3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc044009f20 tx=0x7fc044009840 comp rx=0 tx=0).stop 2023-10-06T05:14:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fc05a5ff700 1 -- 172.21.15.33:0/1763602766 shutdown_connections 2023-10-06T05:14:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fc05a5ff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415dad0 0x7fc0540ca3d0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc044009f20 tx=0x7fc044009840 comp rx=0 tx=0).stop 2023-10-06T05:14:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fc05a5ff700 1 --2- 172.21.15.33:0/1763602766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc05406e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fef2cbff700 1 Processor -- start 2023-10-06T05:14:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fc05a5ff700 1 -- 172.21.15.33:0/1763602766 >> 172.21.15.33:0/1763602766 conn(0x7fc0540ccd10 msgr2=0x7fc0540c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.319+0000 7fef2cbff700 1 -- start start 2023-10-06T05:14:32.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998006a30 con 0x7f49a8153520 2023-10-06T05:14:32.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14440094f0 con 0x7f14480de030 2023-10-06T05:14:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144804d5c0 con 0x7f14480de030 2023-10-06T05:14:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.319+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c00ca80 con 0x7f14480de030 2023-10-06T05:14:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.319+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c00ce00 con 0x7f14480de030 2023-10-06T05:14:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.319+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14480054e0 con 0x7f14480de030 2023-10-06T05:14:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4998006b90 con 0x7f49a8153520 2023-10-06T05:14:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998007030 con 0x7f49a8153520 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.321+0000 7fef2cbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10003b60 0x7fef20038590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc519a700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb8037320 0x7f2fb804bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc619c700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fb80e62d0 0x7f2fb80e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc599b700 1 -- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fb80e62d0 msgr2=0x7f2fb80e50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc599b700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fb80e62d0 0x7f2fb80e50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc599b700 1 -- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb8037320 msgr2=0x7f2fb804bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc599b700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fb8037320 0x7f2fb804bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.315+0000 7f2fc599b700 1 -- 172.21.15.33:0/1680961132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc004c960 con 0x7f2fb8036f50 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f2fc599b700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8036f50 0x7f2fb80dc2d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c7c0 tx=0x7f2fbc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc005a70 con 0x7f2fb8036f50 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.316+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc005bd0 con 0x7f2fb8036f50 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0070d0 con 0x7f2fb8036f50 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb804a160 con 0x7f2fb8036f50 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.317+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb804e770 con 0x7f2fb8036f50 2023-10-06T05:14:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.319+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fb8005380 con 0x7f2fb8036f50 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.321+0000 7fef2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10003b60 0x7fef20038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.321+0000 7fef2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10003b60 0x7fef20038590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37394/0 (socket says 172.21.15.33:37394) 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fef2659c700 1 -- 172.21.15.33:0/2594918331 learned_addr learned my addr 172.21.15.33:0/2594918331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fef2cbff700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef200e08b0 0x7fef200ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fef2cbff700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef200dcbb0 0x7fef200398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.320+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a80c0f90 con 0x7f49a8153520 2023-10-06T05:14:32.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.320+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a80ad750 con 0x7f49a8153520 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200e0d20 con 0x7fef10003b60 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200cae70 con 0x7fef200dcbb0 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef200da9f0 con 0x7fef200e08b0 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fef26d9d700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef200dcbb0 0x7fef200398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fef2659c700 1 -- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef200e08b0 msgr2=0x7fef200ca930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fef2659c700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef200e08b0 0x7fef200ca930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fef2659c700 1 -- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef200dcbb0 msgr2=0x7fef200398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fef2659c700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef200dcbb0 0x7fef200398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fef2659c700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20038ea0 con 0x7fef10003b60 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fef2659c700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10003b60 0x7fef20038590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fef1c008b20 tx=0x7fef1c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49a8068c80 con 0x7f49a8153520 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.321+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f143c005a00 con 0x7f14480de030 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7f143b7fe700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1434067000 0x7f14340694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f143c084b40 con 0x7f14480de030 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7f143b7fe700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f143406f8b0 0x7f1434071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 --> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1434072370 con 0x7f143406f8b0 2023-10-06T05:14:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f143c037ea0 con 0x7f14480de030 2023-10-06T05:14:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c01d420 con 0x7fef10003b60 2023-10-06T05:14:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c01d580 con 0x7fef10003b60 2023-10-06T05:14:32.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c01d6f0 con 0x7fef10003b60 2023-10-06T05:14:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49980073a0 con 0x7f49a8153520 2023-10-06T05:14:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.325+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef20038bc0 con 0x7fef10003b60 2023-10-06T05:14:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fc05a5ff700 1 -- 172.21.15.33:0/1763602766 shutdown_connections 2023-10-06T05:14:32.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.318+0000 7fc05a5ff700 1 -- 172.21.15.33:0/1763602766 wait complete. 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.320+0000 7fc05a5ff700 1 Processor -- start 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.321+0000 7fc05a5ff700 1 -- start start 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.322+0000 7fc05a5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05406e750 0x7fc0540afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc053fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05406e750 0x7fc0540afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc053fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05406e750 0x7fc0540afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37410/0 (socket says 172.21.15.33:37410) 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc053fff700 1 -- 172.21.15.33:0/3637514881 learned_addr learned my addr 172.21.15.33:0/3637514881 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc05a5ff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc0540b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc05a5ff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415ed10 0x7fc05414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05414c3a0 con 0x7fc05406e750 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05414bb00 con 0x7fc05415ed10 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.323+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05414bde0 con 0x7fc0540ca910 2023-10-06T05:14:32.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc053fff700 1 -- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 msgr2=0x7fc0540b19e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc053fff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0540ca910 0x7fc0540b19e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc053fff700 1 -- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415ed10 msgr2=0x7fc05414d8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc053fff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05415ed10 0x7fc05414d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc053fff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044009ae0 con 0x7fc05406e750 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc053fff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05406e750 0x7fc0540afd90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc044003000 tx=0x7fc044017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044012aa0 con 0x7fc05406e750 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.324+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc044012c20 con 0x7fc05406e750 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.325+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04401d420 con 0x7fc05406e750 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.325+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05414c0c0 con 0x7fc05406e750 2023-10-06T05:14:32.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.326+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc054087910 con 0x7fc05406e750 2023-10-06T05:14:32.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.325+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fbc007440 con 0x7f2fb8036f50 2023-10-06T05:14:32.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.326+0000 7f2fb5ffb700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fa4067000 0x7f2fa40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2fbc084390 con 0x7f2fb8036f50 2023-10-06T05:14:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7f2fb5ffb700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f2fa406f8b0 0x7f2fa4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 --> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fa4072370 con 0x7f2fa406f8b0 2023-10-06T05:14:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fbc084680 con 0x7f2fb8036f50 2023-10-06T05:14:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc054063ff0 con 0x7fc05406e750 2023-10-06T05:14:32.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f49967fc700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4988067080 0x7f4988069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f49a659c700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4988067080 0x7f4988069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f2fc519a700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fa4067000 0x7f2fa40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4998084080 con 0x7f49a8153520 2023-10-06T05:14:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f2fc619c700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f2fa406f8b0 0x7f2fa4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef1c027040 con 0x7fef10003b60 2023-10-06T05:14:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7f144f7fe700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1434067000 0x7f14340694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.327+0000 7f144effd700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f143406f8b0 0x7f1434071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f144f7fe700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1434067000 0x7f14340694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1444009490 tx=0x7f1444017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.328+0000 7f49967fc700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f498806f930 0x7f4988071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.329+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49880723f0 con 0x7f498806f930 2023-10-06T05:14:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.329+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4998086620 con 0x7f49a8153520 2023-10-06T05:14:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.329+0000 7f49a659c700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4988067080 0x7f4988069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.329+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f498806f930 0x7f4988071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.329+0000 7f144effd700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f143406f8b0 0x7f1434071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.330+0000 7f2fc519a700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fa4067000 0x7f2fa40694b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80e8c20 tx=0x7f2fb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.330+0000 7f49a6d9d700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f498806f930 0x7f4988071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.330+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef200da680 con 0x7fef10003b60 2023-10-06T05:14:32.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.329+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc04401d790 con 0x7fc05406e750 2023-10-06T05:14:32.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.330+0000 7f2fc619c700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f2fa406f8b0 0x7f2fa4071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== osd.4 v2:172.21.15.167:6800/85769 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f49880723f0 con 0x7f498806f930 2023-10-06T05:14:32.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fef167fc700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef08067000 0x7fef080694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fef1c0914e0 con 0x7fef10003b60 2023-10-06T05:14:32.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== osd.6 v2:172.21.15.167:6808/85771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1434072370 con 0x7f143406f8b0 2023-10-06T05:14:32.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fc050ff9700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc034066f80 0x7fc034069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc044085af0 con 0x7fc05406e750 2023-10-06T05:14:32.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fc050ff9700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7fc03406f6c0 0x7fc034071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc034072180 con 0x7fc03406f6c0 2023-10-06T05:14:32.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.331+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc04408a220 con 0x7fc05406e750 2023-10-06T05:14:32.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.332+0000 7fef25d9b700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef08067000 0x7fef080694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.332+0000 7fef2cbff700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7fef200d8a40 0x7fef200dae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.332+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef200dd330 con 0x7fef200d8a40 2023-10-06T05:14:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.332+0000 7fc04bfff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc034066f80 0x7fc034069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.332+0000 7fc058b9c700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7fc03406f6c0 0x7fc034071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.332+0000 7fef26d9d700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7fef200d8a40 0x7fef200dae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.333+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== osd.3 v2:172.21.15.33:6824/85580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2fa4072370 con 0x7f2fa406f8b0 2023-10-06T05:14:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.333+0000 7fc058b9c700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7fc03406f6c0 0x7fc034071ad0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.333+0000 7fc04bfff700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc034066f80 0x7fc034069430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc03c006fd0 tx=0x7fc03c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.333+0000 7fef25d9b700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef08067000 0x7fef080694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef28071ad0 tx=0x7fef28065f70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.334+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 --> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a98007480 con 0x7f1a8406f6c0 2023-10-06T05:14:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.334+0000 7fef26d9d700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7fef200d8a40 0x7fef200dae60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1a9cff9700 1 -- 172.21.15.33:0/2489959372 <== osd.1 v2:172.21.15.33:6816/85579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1a98007480 con 0x7f1a8406f6c0 2023-10-06T05:14:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.334+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 --> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f380d3da0 con 0x7f1f2406f8b0 2023-10-06T05:14:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f2bfff700 1 -- 172.21.15.33:0/3852623926 <== osd.0 v2:172.21.15.33:6800/85577 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f380d3da0 con 0x7f1f2406f8b0 2023-10-06T05:14:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f1f2406f8b0 msgr2=0x7f1f24071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f29ffb700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f1f2406f8b0 0x7f1f24071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== osd.5 v2:172.21.15.167:6816/85770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fef200dd330 con 0x7fef200d8a40 2023-10-06T05:14:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f24067000 msgr2=0x7f1f240694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f29ffb700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f24067000 0x7f1f240694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c006fd0 tx=0x7f1f2c008040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380e3e70 msgr2=0x7f1f38037c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.335+0000 7f1f29ffb700 1 --2- 172.21.15.33:0/3852623926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f380e3e70 0x7f1f38037c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f4004c820 tx=0x7f1f40065f70 comp rx=0 tx=0).stop 2023-10-06T05:14:32.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.334+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== osd.7 v2:172.21.15.167:6824/85794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc034072180 con 0x7fc03406f6c0 2023-10-06T05:14:32.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.337+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f1a8406f6c0 msgr2=0x7f1a84071ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.337+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f1a8406f6c0 0x7f1a84071ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.337+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a84067000 msgr2=0x7f1a840694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.338+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a84067000 0x7f1a840694b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800ddb0 tx=0x7f1a94009060 comp rx=0 tx=0).stop 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.338+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a98039160 msgr2=0x7f1a980da320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.338+0000 7f1aa5d9d700 1 --2- 172.21.15.33:0/2489959372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a98039160 0x7f1a980da320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009290 tx=0x7f1a8c016000 comp rx=0 tx=0).stop 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.338+0000 7f1aa559c700 1 -- 172.21.15.33:0/2489959372 reap_dead start 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.338+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 shutdown_connections 2023-10-06T05:14:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.338+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 >> 172.21.15.33:0/2489959372 conn(0x7f1a9800f1c0 msgr2=0x7f1a980e8410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.336+0000 7f1f44b9c700 1 -- 172.21.15.33:0/3852623926 reap_dead start 2023-10-06T05:14:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.336+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 shutdown_connections 2023-10-06T05:14:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.336+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 >> 172.21.15.33:0/3852623926 conn(0x7f1f38054c90 msgr2=0x7f1f3804fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.341+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 shutdown_connections 2023-10-06T05:14:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.341+0000 7f1aa5d9d700 1 -- 172.21.15.33:0/2489959372 wait complete. 2023-10-06T05:14:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.341+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 shutdown_connections 2023-10-06T05:14:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.342+0000 7f1f29ffb700 1 -- 172.21.15.33:0/3852623926 wait complete. 2023-10-06T05:14:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.355+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 --> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89480d06e0 con 0x7f893406f740 2023-10-06T05:14:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.356+0000 7f89467fc700 1 -- 172.21.15.33:0/2355467436 <== osd.2 v2:172.21.15.33:6808/85578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f89480d06e0 con 0x7f893406f740 2023-10-06T05:14:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.357+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7f893406f740 msgr2=0x7f8934071b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.357+0000 7f89507ff700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7f893406f740 0x7f8934071b50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.358+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8934067000 msgr2=0x7f89340694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.358+0000 7f89507ff700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8934067000 0x7f89340694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89481484d0 tx=0x7f893c008040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.358+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89480b2200 msgr2=0x7f89481447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.358+0000 7f89507ff700 1 --2- 172.21.15.33:0/2355467436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89480b2200 0x7f89481447d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8940006fd0 tx=0x7f8940027000 comp rx=0 tx=0).stop 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.358+0000 7f894ed9c700 1 -- 172.21.15.33:0/2355467436 reap_dead start 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.359+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 shutdown_connections 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.359+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 >> 172.21.15.33:0/2355467436 conn(0x7f894806c560 msgr2=0x7f89480b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.360+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 --> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49a804c380 con 0x7f498806f930 2023-10-06T05:14:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.361+0000 7f49967fc700 1 -- 172.21.15.33:0/4036934934 <== osd.4 v2:172.21.15.167:6800/85769 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f49a804c380 con 0x7f498806f930 2023-10-06T05:14:32.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 shutdown_connections 2023-10-06T05:14:32.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f89507ff700 1 -- 172.21.15.33:0/2355467436 wait complete. 2023-10-06T05:14:32.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7fc05a5ff700 1 -- 172.21.15.33:0/3637514881 --> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc05406ebb0 con 0x7fc03406f6c0 2023-10-06T05:14:32.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f498806f930 msgr2=0x7f4988071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f498806f930 0x7f4988071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4988067080 msgr2=0x7f4988069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4988067080 0x7f4988069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a000b040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a8153520 msgr2=0x7f49a8158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 --2- 172.21.15.33:0/4036934934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a8153520 0x7f49a8158d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f499800f8e0 tx=0x7f4998007d30 comp rx=0 tx=0).stop 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49a6d9d700 1 -- 172.21.15.33:0/4036934934 reap_dead start 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 shutdown_connections 2023-10-06T05:14:32.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 >> 172.21.15.33:0/4036934934 conn(0x7f49a806f640 msgr2=0x7f49a8068060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.362+0000 7fc050ff9700 1 -- 172.21.15.33:0/3637514881 <== osd.7 v2:172.21.15.167:6824/85794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc05406ebb0 con 0x7fc03406f6c0 2023-10-06T05:14:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7fc03406f6c0 msgr2=0x7fc034071ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc049ffb700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7fc03406f6c0 0x7fc034071ad0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc034066f80 msgr2=0x7fc034069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc049ffb700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc034066f80 0x7fc034069430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc03c006fd0 tx=0x7fc03c008040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.364+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 --> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f14480543b0 con 0x7f143406f8b0 2023-10-06T05:14:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05406e750 msgr2=0x7fc0540afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc049ffb700 1 --2- 172.21.15.33:0/3637514881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05406e750 0x7fc0540afd90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc044003000 tx=0x7fc044017000 comp rx=0 tx=0).stop 2023-10-06T05:14:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.363+0000 7fc058b9c700 1 -- 172.21.15.33:0/3637514881 reap_dead start 2023-10-06T05:14:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.364+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 shutdown_connections 2023-10-06T05:14:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.364+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 >> 172.21.15.33:0/3637514881 conn(0x7fc0540ccd10 msgr2=0x7fc05415cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.364+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 shutdown_connections 2023-10-06T05:14:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.364+0000 7fc049ffb700 1 -- 172.21.15.33:0/3637514881 wait complete. 2023-10-06T05:14:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.365+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 shutdown_connections 2023-10-06T05:14:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.364+0000 7f143b7fe700 1 -- 172.21.15.33:0/2316200163 <== osd.6 v2:172.21.15.167:6808/85771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f14480543b0 con 0x7f143406f8b0 2023-10-06T05:14:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.365+0000 7f49acdff700 1 -- 172.21.15.33:0/4036934934 wait complete. 2023-10-06T05:14:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.366+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f143406f8b0 msgr2=0x7f1434071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.366+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f143406f8b0 0x7f1434071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.368+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 --> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fb8006e50 con 0x7f2fa406f8b0 2023-10-06T05:14:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.367+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 --> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef200531a0 con 0x7fef200d8a40 2023-10-06T05:14:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.366+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1434067000 msgr2=0x7f14340694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.366+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1434067000 0x7f14340694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1444009490 tx=0x7f1444017000 comp rx=0 tx=0).stop 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.366+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14480de030 msgr2=0x7f14480dc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.366+0000 7f1454b9d700 1 --2- 172.21.15.33:0/2316200163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14480de030 0x7f14480dc940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f143c00ea50 tx=0x7f143c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.367+0000 7f144ffff700 1 -- 172.21.15.33:0/2316200163 reap_dead start 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.367+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 shutdown_connections 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.367+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 >> 172.21.15.33:0/2316200163 conn(0x7f1448054da0 msgr2=0x7f14480dd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.368+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 shutdown_connections 2023-10-06T05:14:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.368+0000 7f1454b9d700 1 -- 172.21.15.33:0/2316200163 wait complete. 2023-10-06T05:14:32.403 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2023-10-06T05:14:32.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T05:14:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.368+0000 7f2fb5ffb700 1 -- 172.21.15.33:0/1680961132 <== osd.3 v2:172.21.15.33:6824/85580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2fb8006e50 con 0x7f2fa406f8b0 2023-10-06T05:14:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.368+0000 7fef167fc700 1 -- 172.21.15.33:0/2594918331 <== osd.5 v2:172.21.15.167:6816/85770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef200531a0 con 0x7fef200d8a40 2023-10-06T05:14:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.369+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7fef200d8a40 msgr2=0x7fef200dae60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.369+0000 7fef2cbff700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7fef200d8a40 0x7fef200dae60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.369+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef08067000 msgr2=0x7fef080694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef08067000 0x7fef080694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef28071ad0 tx=0x7fef28065f70 comp rx=0 tx=0).stop 2023-10-06T05:14:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f2fa406f8b0 msgr2=0x7f2fa4071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc699d700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f2fa406f8b0 0x7f2fa4071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10003b60 msgr2=0x7fef20038590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 --2- 172.21.15.33:0/2594918331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef10003b60 0x7fef20038590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fef1c008b20 tx=0x7fef1c01e000 comp rx=0 tx=0).stop 2023-10-06T05:14:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fa4067000 msgr2=0x7f2fa40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef26d9d700 1 -- 172.21.15.33:0/2594918331 reap_dead start 2023-10-06T05:14:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 shutdown_connections 2023-10-06T05:14:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 >> 172.21.15.33:0/2594918331 conn(0x7fef20054e30 msgr2=0x7fef2004ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc699d700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fa4067000 0x7f2fa40694b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80e8c20 tx=0x7f2fb0008040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 shutdown_connections 2023-10-06T05:14:32.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8036f50 msgr2=0x7f2fb80dc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc699d700 1 --2- 172.21.15.33:0/1680961132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8036f50 0x7f2fb80dc2d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c7c0 tx=0x7f2fbc007f30 comp rx=0 tx=0).stop 2023-10-06T05:14:32.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7fef2cbff700 1 -- 172.21.15.33:0/2594918331 wait complete. 2023-10-06T05:14:32.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.370+0000 7f2fc619c700 1 -- 172.21.15.33:0/1680961132 reap_dead start 2023-10-06T05:14:32.409 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2023-10-06T05:14:32.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T05:14:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.371+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 shutdown_connections 2023-10-06T05:14:32.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.371+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 >> 172.21.15.33:0/1680961132 conn(0x7f2fb8054c80 msgr2=0x7f2fb80e0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.371+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 shutdown_connections 2023-10-06T05:14:32.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.371+0000 7f2fc699d700 1 -- 172.21.15.33:0/1680961132 wait complete. 2023-10-06T05:14:32.412 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2023-10-06T05:14:32.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T05:14:32.414 INFO:teuthology.orchestra.run.smithi033.stdout:55834574857 2023-10-06T05:14:32.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T05:14:32.416 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:32.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T05:14:32.417 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:32.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T05:14:32.419 INFO:teuthology.orchestra.run.smithi033.stdout:55834574857 2023-10-06T05:14:32.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T05:14:32.421 INFO:teuthology.orchestra.run.smithi033.stdout:55834574857 2023-10-06T05:14:32.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T05:14:32.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.781+0000 7fb367fff700 1 Processor -- start 2023-10-06T05:14:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.781+0000 7fb367fff700 1 -- start start 2023-10-06T05:14:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7fb367fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb36815d870 0x7fb36815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7fb367fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7fb367fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36815ef60 con 0x7fb3680c6ea0 2023-10-06T05:14:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7fb367fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3680c72f0 con 0x7fb36815d870 2023-10-06T05:14:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7fb367fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb368161940 con 0x7fb36815dc40 2023-10-06T05:14:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7fb3677fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb3680c6ea0 0x7fb3680c8970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38992/0 (socket says 172.21.15.33:38992) 2023-10-06T05:14:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7fb3677fe700 1 -- 172.21.15.33:0/3371850414 learned_addr learned my addr 172.21.15.33:0/3371850414 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7fb366ffd700 1 --2- 172.21.15.33:0/3371850414 >> v2:172.21.15.33:3300/0 conn(0x7fb36815d870 0x7fb36815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.781+0000 7f879e19d700 1 Processor -- start 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.781+0000 7f879e19d700 1 -- start start 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.783+0000 7f879e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87900df530 0x7f8790005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.783+0000 7f879e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.783+0000 7f879e19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8790053970 con 0x7f87900de150 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.783+0000 7f879e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879004f6a0 con 0x7f87900df530 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.783+0000 7f879e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8790006960 con 0x7f8790006500 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7f879c99a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87900df530 0x7f8790005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7f879c99a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87900df530 0x7f8790005fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40226/0 (socket says 172.21.15.33:40226) 2023-10-06T05:14:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.784+0000 7f879c99a700 1 -- 172.21.15.33:0/212938752 learned_addr learned my addr 172.21.15.33:0/212938752 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7f879d99c700 1 --2- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7f879c99a700 1 -- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 msgr2=0x7f87900e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7f879c99a700 1 --2- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900e1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7f879c99a700 1 -- 172.21.15.33:0/212938752 >> v1:172.21.15.167:6789/0 conn(0x7f87900de150 legacy=0x7f8790053e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.785+0000 7f879c99a700 1 -- 172.21.15.33:0/212938752 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790053360 con 0x7f87900df530 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.786+0000 7fb35e7fc700 1 --2- 172.21.15.33:0/3371850414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.786+0000 7f879c99a700 1 --2- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900df530 0x7f8790005fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87880099b0 tx=0x7f8788009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbcd1a433dbdf66f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.786+0000 7fb366ffd700 1 -- 172.21.15.33:0/3371850414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 msgr2=0x7fb3680c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.786+0000 7fb366ffd700 1 --2- 172.21.15.33:0/3371850414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.786+0000 7fb366ffd700 1 -- 172.21.15.33:0/3371850414 >> v1:172.21.15.33:6789/0 conn(0x7fb3680c6ea0 legacy=0x7fb3680c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.786+0000 7fb366ffd700 1 -- 172.21.15.33:0/3371850414 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb368161c20 con 0x7fb36815d870 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.787+0000 7fb366ffd700 1 --2- 172.21.15.33:0/3371850414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb36815ea20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb350009b40 tx=0x7fb3500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3de8f1f619f01f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.787+0000 7fb3667fc700 1 -- 172.21.15.33:0/3371850414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000b680 con 0x7fb36815d870 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.787+0000 7fb3667fc700 1 -- 172.21.15.33:0/3371850414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35000b800 con 0x7fb36815d870 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.787+0000 7fb3667fc700 1 -- 172.21.15.33:0/3371850414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000bbc0 con 0x7fb36815d870 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.788+0000 7f878ffff700 1 -- 172.21.15.33:0/212938752 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800b820 con 0x7f87900df530 2023-10-06T05:14:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.788+0000 7f878ffff700 1 -- 172.21.15.33:0/212938752 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878800b980 con 0x7f87900df530 2023-10-06T05:14:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.788+0000 7f878ffff700 1 -- 172.21.15.33:0/212938752 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800bd00 con 0x7f87900df530 2023-10-06T05:14:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.788+0000 7fb367fff700 1 -- 172.21.15.33:0/3371850414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 msgr2=0x7fb36815ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.788+0000 7fb367fff700 1 --2- 172.21.15.33:0/3371850414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb36815ea20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb350009b40 tx=0x7fb3500091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7fb367fff700 1 -- 172.21.15.33:0/3371850414 shutdown_connections 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7fb367fff700 1 --2- 172.21.15.33:0/3371850414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb36815ea20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7fb367fff700 1 --2- 172.21.15.33:0/3371850414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7fb367fff700 1 -- 172.21.15.33:0/3371850414 >> 172.21.15.33:0/3371850414 conn(0x7fb36806c560 msgr2=0x7fb36806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7fb367fff700 1 -- 172.21.15.33:0/3371850414 shutdown_connections 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7fb367fff700 1 -- 172.21.15.33:0/3371850414 wait complete. 2023-10-06T05:14:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f88b319d700 1 Processor -- start 2023-10-06T05:14:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 -- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900df530 msgr2=0x7f8790005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 --2- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900df530 0x7f8790005fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87880099b0 tx=0x7f8788009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 -- 172.21.15.33:0/212938752 shutdown_connections 2023-10-06T05:14:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 --2- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 --2- 172.21.15.33:0/212938752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900df530 0x7f8790005fc0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87880099b0 tx=0x7f8788009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 -- 172.21.15.33:0/212938752 >> 172.21.15.33:0/212938752 conn(0x7f8790054d30 msgr2=0x7f8790055130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.789+0000 7f879e19d700 1 -- 172.21.15.33:0/212938752 shutdown_connections 2023-10-06T05:14:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.790+0000 7f879e19d700 1 -- 172.21.15.33:0/212938752 wait complete. 2023-10-06T05:14:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.791+0000 7fb367fff700 1 Processor -- start 2023-10-06T05:14:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.790+0000 7f88b319d700 1 -- start start 2023-10-06T05:14:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7fb367fff700 1 -- start start 2023-10-06T05:14:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f88b319d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f88ac1553c0 0x7f88ac0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f88b319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f88b319d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88ac0ca6a0 0x7f88ac06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f88b319d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac06a340 con 0x7f88ac0ca6a0 2023-10-06T05:14:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f88b319d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac06b710 con 0x7f88ac1553c0 2023-10-06T05:14:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f88b319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0c95b0 con 0x7f88ac155790 2023-10-06T05:14:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f879e19d700 1 Processor -- start 2023-10-06T05:14:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.792+0000 7f879e19d700 1 -- start start 2023-10-06T05:14:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7f88b219b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f88ac1553c0 0x7f88ac0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7f88b219b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f88ac1553c0 0x7f88ac0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40230/0 (socket says 172.21.15.33:40230) 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7f88b299c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88ac0ca6a0 0x7f88ac06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7f88b299c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88ac0ca6a0 0x7f88ac06ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37438/0 (socket says 172.21.15.33:37438) 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7fb367fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3680c6ea0 0x7fb36814c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7fb367fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb3680c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7fb367fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7fb367fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36815a290 con 0x7fb36815d870 2023-10-06T05:14:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7fb367fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36815a4a0 con 0x7fb3680c6ea0 2023-10-06T05:14:32.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7fb367fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36815a780 con 0x7fb36815dc40 2023-10-06T05:14:32.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.793+0000 7f88b219b700 1 -- 172.21.15.33:0/2762777042 learned_addr learned my addr 172.21.15.33:0/2762777042 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7f88b199a700 1 --2- 172.21.15.33:0/2762777042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7fb35e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb3680c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7fb3677fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7fb3677fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41852/0 (socket says 172.21.15.33:41852) 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7fb3677fe700 1 -- 172.21.15.33:0/2474475903 learned_addr learned my addr 172.21.15.33:0/2474475903 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7f88b299c700 1 -- 172.21.15.33:0/2762777042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 msgr2=0x7f88ac06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7f88b299c700 1 --2- 172.21.15.33:0/2762777042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac06a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7f88b299c700 1 -- 172.21.15.33:0/2762777042 >> v2:172.21.15.167:3300/0 conn(0x7f88ac1553c0 msgr2=0x7f88ac0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.794+0000 7f88b299c700 1 --2- 172.21.15.33:0/2762777042 >> v2:172.21.15.167:3300/0 conn(0x7f88ac1553c0 0x7f88ac0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7f88b299c700 1 -- 172.21.15.33:0/2762777042 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0c9890 con 0x7f88ac0ca6a0 2023-10-06T05:14:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7fb35e7fc700 1 -- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 msgr2=0x7fb3680c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7fb35e7fc700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7fb35e7fc700 1 -- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3680c6ea0 msgr2=0x7fb36814c350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:14:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7fb35e7fc700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3680c6ea0 0x7fb36814c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7fb35e7fc700 1 -- 172.21.15.33:0/2474475903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb350009420 con 0x7fb36815d870 2023-10-06T05:14:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7f88b299c700 1 --2- 172.21.15.33:0/2762777042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac06ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0697c0 tx=0x7f88a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5569877330d2a019 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.795+0000 7f879e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f88b1199700 1 -- 172.21.15.33:0/2762777042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400bb40 con 0x7f88ac0ca6a0 2023-10-06T05:14:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 0x7f87900dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f88b1199700 1 -- 172.21.15.33:0/2762777042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a400bca0 con 0x7f88ac0ca6a0 2023-10-06T05:14:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f88b1199700 1 -- 172.21.15.33:0/2762777042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4010720 con 0x7f88ac0ca6a0 2023-10-06T05:14:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879d19b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879d19b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900f9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41864/0 (socket says 172.21.15.33:41864) 2023-10-06T05:14:32.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7fb35e7fc700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb3680c1c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c850 tx=0x7fb358008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 0x7f87900dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 0x7f87900dcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37456/0 (socket says 172.21.15.33:37456) 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900dd400 0x7f8790102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900db060 con 0x7f87900de150 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900dd970 con 0x7f87900dd400 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879d19b700 1 -- 172.21.15.33:0/3868815564 learned_addr learned my addr 172.21.15.33:0/3868815564 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.796+0000 7f879e19d700 1 -- 172.21.15.33:0/3868815564 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879004ce50 con 0x7f8790006500 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.797+0000 7f879d99c700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900dd400 0x7f8790102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.797+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011600 con 0x7fb36815d870 2023-10-06T05:14:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.797+0000 7f88b319d700 1 -- 172.21.15.33:0/2762777042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 msgr2=0x7f88ac06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.797+0000 7f88b319d700 1 --2- 172.21.15.33:0/2762777042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac06ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0697c0 tx=0x7f88a4009780 comp rx=0 tx=0).stop 2023-10-06T05:14:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.797+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36815aa60 con 0x7fb36815d870 2023-10-06T05:14:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36815b020 con 0x7fb36815d870 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f879c99a700 1 -- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 msgr2=0x7f87900f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f879c99a700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.799+0000 7f879c99a700 1 -- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900dd400 msgr2=0x7f8790102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.799+0000 7f879c99a700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900dd400 0x7f8790102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.799+0000 7f879c99a700 1 -- 172.21.15.33:0/3868815564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880094f0 con 0x7f87900de150 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.799+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358011760 con 0x7fb36815d870 2023-10-06T05:14:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.799+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011ae0 con 0x7fb36815d870 2023-10-06T05:14:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.799+0000 7f879c99a700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 0x7f87900dcec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8790052dd0 tx=0x7f8788016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.800+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788011c10 con 0x7f87900de150 2023-10-06T05:14:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.800+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788020070 con 0x7f87900de150 2023-10-06T05:14:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.800+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878801b450 con 0x7f87900de150 2023-10-06T05:14:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.800+0000 7f879e19d700 1 -- 172.21.15.33:0/3868815564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879004d130 con 0x7f87900de150 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f88b319d700 1 -- 172.21.15.33:0/2762777042 shutdown_connections 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f88b319d700 1 --2- 172.21.15.33:0/2762777042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac06ad10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f88b319d700 1 --2- 172.21.15.33:0/2762777042 >> v2:172.21.15.167:3300/0 conn(0x7f88ac1553c0 0x7f88ac0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f88b319d700 1 --2- 172.21.15.33:0/2762777042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac06a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.798+0000 7f88b319d700 1 -- 172.21.15.33:0/2762777042 >> 172.21.15.33:0/2762777042 conn(0x7f88ac06c560 msgr2=0x7f88ac06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.801+0000 7f88b319d700 1 -- 172.21.15.33:0/2762777042 shutdown_connections 2023-10-06T05:14:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.801+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c0052b0 con 0x7fb36815d870 2023-10-06T05:14:32.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.801+0000 7f879e19d700 1 -- 172.21.15.33:0/3868815564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879004d6f0 con 0x7f87900de150 2023-10-06T05:14:32.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.801+0000 7f88b319d700 1 -- 172.21.15.33:0/2762777042 wait complete. 2023-10-06T05:14:32.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7f88b319d700 1 Processor -- start 2023-10-06T05:14:32.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7f88b319d700 1 -- start start 2023-10-06T05:14:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7f52bdf9d700 1 Processor -- start 2023-10-06T05:14:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7f52bdf9d700 1 -- start start 2023-10-06T05:14:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7f28c13ff700 1 Processor -- start 2023-10-06T05:14:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7f28c13ff700 1 -- start start 2023-10-06T05:14:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f28c13ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28bc0ae820 0x7f28bc156450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f88b319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac1762a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f88b319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 0x7f88ac154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f28c13ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc156d50 con 0x7f28bc1530e0 2023-10-06T05:14:32.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f17941ff700 1 Processor -- start 2023-10-06T05:14:32.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f88b319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac0c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f88b319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0b0420 con 0x7f88ac0ca6a0 2023-10-06T05:14:32.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f28c13ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc0c89c0 con 0x7f28bc1534b0 2023-10-06T05:14:32.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f88b319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac16afc0 con 0x7f88ac1553c0 2023-10-06T05:14:32.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f88b319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0b06e0 con 0x7f88ac155790 2023-10-06T05:14:32.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f28c13ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc156a70 con 0x7f28bc0ae820 2023-10-06T05:14:32.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f52bdf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f52b00e3e60 0x7f52b00e4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f879e19d700 1 -- 172.21.15.33:0/3868815564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87780052b0 con 0x7f87900de150 2023-10-06T05:14:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f52bdf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b00ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f52bdf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b0006e90 con 0x7f52b0006920 2023-10-06T05:14:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f52bdf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b00eab50 con 0x7f52b00e3e60 2023-10-06T05:14:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7f52bdf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0007170 con 0x7f52b00ea190 2023-10-06T05:14:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f52b7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f52b0006920 0x7f52b004fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39000/0 (socket says 172.21.15.33:39000) 2023-10-06T05:14:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f52b7fff700 1 -- 172.21.15.33:0/1166737287 learned_addr learned my addr 172.21.15.33:0/1166737287 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8788022030 con 0x7f87900de150 2023-10-06T05:14:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f88b199a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 0x7f88ac154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f88b199a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 0x7f88ac154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40264/0 (socket says 172.21.15.33:40264) 2023-10-06T05:14:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f88b199a700 1 -- 172.21.15.33:0/400656336 learned_addr learned my addr 172.21.15.33:0/400656336 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f52bcf9b700 1 --2- 172.21.15.33:0/1166737287 >> v2:172.21.15.167:3300/0 conn(0x7f52b00e3e60 0x7f52b00e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.807+0000 7f17941ff700 1 -- start start 2023-10-06T05:14:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f52bcf9b700 1 -- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 msgr2=0x7f52b00ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f52bd79c700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b00ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.801+0000 7fc479b9d700 1 Processor -- start 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.801+0000 7fc479b9d700 1 -- start start 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7fc479b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc46c0df480 0x7fc46c005f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7fc479b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.802+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3580193b0 con 0x7fb36815d870 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb348067000 0x7fb3480694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.805+0000 7fb366ffd700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb348067000 0x7fb3480694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.806+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb358054db0 con 0x7fb36815d870 2023-10-06T05:14:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7fc479b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c0538c0 con 0x7fc46c0de0a0 2023-10-06T05:14:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7fc479b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c04f5f0 con 0x7fc46c0df480 2023-10-06T05:14:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.803+0000 7fc479b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0068b0 con 0x7fc46c006450 2023-10-06T05:14:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7fc473fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc46c0df480 0x7fc46c005f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7fc473fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc46c0df480 0x7fc46c005f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37472/0 (socket says 172.21.15.33:37472) 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.804+0000 7fc473fff700 1 -- 172.21.15.33:0/2732620790 learned_addr learned my addr 172.21.15.33:0/2732620790 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.805+0000 7fc473fff700 1 -- 172.21.15.33:0/2732620790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 msgr2=0x7fc46c0e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.805+0000 7fc473fff700 1 --2- 172.21.15.33:0/2732620790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.805+0000 7fc473fff700 1 -- 172.21.15.33:0/2732620790 >> v1:172.21.15.33:6789/0 conn(0x7fc46c0de0a0 legacy=0x7fc46c053d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.805+0000 7fc473fff700 1 -- 172.21.15.33:0/2732620790 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0532b0 con 0x7fc46c0df480 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.805+0000 7fc473fff700 1 --2- 172.21.15.33:0/2732620790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0df480 0x7fc46c005f10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4680099b0 tx=0x7fc468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e1d3cc58aba558 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f52bcf9b700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b00ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f52bcf9b700 1 -- 172.21.15.33:0/1166737287 >> v1:172.21.15.33:6789/0 conn(0x7f52b0006920 legacy=0x7f52b004fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f52bcf9b700 1 -- 172.21.15.33:0/1166737287 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0007450 con 0x7f52b00e3e60 2023-10-06T05:14:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f28ba7fc700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f28bc1534b0 0x7f28bc155d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57770/0 (socket says 172.21.15.33:57770) 2023-10-06T05:14:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f28ba7fc700 1 -- 172.21.15.33:0/3675368737 learned_addr learned my addr 172.21.15.33:0/3675368737 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f52bd79c700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b00ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.808+0000 7f28bb7fe700 1 --2- 172.21.15.33:0/3675368737 >> v2:172.21.15.33:3300/0 conn(0x7f28bc0ae820 0x7f28bc156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315565805 0 0) 0x7f28bc0c89c0 con 0x7f28bc1534b0 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28a4003570 con 0x7f28bc1534b0 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f88b199a700 1 -- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 msgr2=0x7f88ac0c1d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f88b199a700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac0c1d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f88b199a700 1 -- 172.21.15.33:0/400656336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 msgr2=0x7f88ac1762a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f52bcf9b700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00e4230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0098e0 tx=0x7f52ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3717ffe934e531cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f17941ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c06e920 0x7f178c168af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f17941ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0cb720 con 0x7f178c15dd10 2023-10-06T05:14:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f17941ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0ca870 con 0x7f178c15d940 2023-10-06T05:14:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f17941ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0cab10 con 0x7f178c06e920 2023-10-06T05:14:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7fb366ffd700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb348067000 0x7fb3480694b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb35000a010 tx=0x7fb350016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f878d7fa700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f877c067080 0x7f877c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8788086410 con 0x7f87900de150 2023-10-06T05:14:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f88b219b700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac1762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f88b199a700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac1762a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f88b199a700 1 -- 172.21.15.33:0/400656336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009a20 con 0x7f88ac1553c0 2023-10-06T05:14:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f1791f9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f178c15d940 0x7f178c0c6e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57782/0 (socket says 172.21.15.33:57782) 2023-10-06T05:14:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f1791f9b700 1 -- 172.21.15.33:0/2914214127 learned_addr learned my addr 172.21.15.33:0/2914214127 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229143812 0 0) 0x7f28a4003570 con 0x7f28bc1534b0 2023-10-06T05:14:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 >> v2:172.21.15.33:3300/0 conn(0x7f28bc0ae820 msgr2=0x7f28bc156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 --2- 172.21.15.33:0/3675368737 >> v2:172.21.15.33:3300/0 conn(0x7f28bc0ae820 0x7f28bc156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 >> v1:172.21.15.33:6789/0 conn(0x7f28bc1530e0 legacy=0x7f28bc0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0c8370 con 0x7f28bc1534b0 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 <== mon.1 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f28ac004cd0 con 0x7f28bc1534b0 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 >> v1:172.21.15.167:6789/0 conn(0x7f28bc1534b0 legacy=0x7f28bc155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a4003ae0 0x7f28a4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4004810 0x7f28a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.809+0000 7f28b9ffb700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4005210 0x7f28a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc156a70 con 0x7f28a4004810 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a400c1e0 con 0x7f28a4005210 2023-10-06T05:14:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a400c3a0 con 0x7f28a4003ae0 2023-10-06T05:14:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f879d19b700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f877c067080 0x7f877c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f28ba7fc700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4004810 0x7f28a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f28baffd700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a4003ae0 0x7f28a4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f52b77fe700 1 -- 172.21.15.33:0/1166737287 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac009e80 con 0x7f52b00e3e60 2023-10-06T05:14:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f52b77fe700 1 -- 172.21.15.33:0/1166737287 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00b860 con 0x7f52b00e3e60 2023-10-06T05:14:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f52b77fe700 1 -- 172.21.15.33:0/1166737287 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00bbe0 con 0x7f52b00e3e60 2023-10-06T05:14:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.810+0000 7f179279c700 1 --2- 172.21.15.33:0/2914214127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c06e920 0x7f178c168af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28ba7fc700 1 -- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a4003ae0 msgr2=0x7f28a4003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28ba7fc700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a4003ae0 0x7f28a4003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28ba7fc700 1 -- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4005210 msgr2=0x7f28a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28ba7fc700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a4005210 0x7f28a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28ba7fc700 1 -- 172.21.15.33:0/3675368737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0c8370 con 0x7f28a4004810 2023-10-06T05:14:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28baffd700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a4003ae0 0x7f28a4003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f1790f99700 1 -- 172.21.15.33:0/2914214127 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184756687 0 0) 0x7f178c0ca870 con 0x7f178c15d940 2023-10-06T05:14:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f1790f99700 1 -- 172.21.15.33:0/2914214127 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1774003570 con 0x7f178c15d940 2023-10-06T05:14:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 -- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 msgr2=0x7f52b00e4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00e4230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0098e0 tx=0x7f52ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28ba7fc700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4004810 0x7f28a4004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a4004790 tx=0x7f28ac00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab326e8c4b5991d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f28bb7fe700 1 -- 172.21.15.33:0/3675368737 reap_dead start 2023-10-06T05:14:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f88b199a700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 0x7f88ac154530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d860 tx=0x7f88a8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 -- 172.21.15.33:0/1166737287 shutdown_connections 2023-10-06T05:14:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f179279c700 1 -- 172.21.15.33:0/2914214127 >> v1:172.21.15.167:6789/0 conn(0x7f178c15d940 legacy=0x7f178c0c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f1790f99700 1 -- 172.21.15.33:0/2914214127 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850694504 0 0) 0x7f1774003570 con 0x7f178c15d940 2023-10-06T05:14:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f179279c700 1 -- 172.21.15.33:0/2914214127 >> v1:172.21.15.33:6789/0 conn(0x7f178c15dd10 legacy=0x7f178c06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f179279c700 1 -- 172.21.15.33:0/2914214127 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0cadf0 con 0x7f178c06e920 2023-10-06T05:14:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f879d19b700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f877c067080 0x7f877c069530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f879004b770 tx=0x7f8794008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8009610 con 0x7f88ac1553c0 2023-10-06T05:14:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a8009770 con 0x7f88ac1553c0 2023-10-06T05:14:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8009b30 con 0x7f88ac1553c0 2023-10-06T05:14:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac0b09c0 con 0x7f88ac1553c0 2023-10-06T05:14:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00e4230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac154a70 con 0x7f88ac1553c0 2023-10-06T05:14:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/1166737287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b00ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 -- 172.21.15.33:0/1166737287 >> 172.21.15.33:0/1166737287 conn(0x7f52b0054c80 msgr2=0x7f52b0055080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 -- 172.21.15.33:0/1166737287 shutdown_connections 2023-10-06T05:14:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.811+0000 7f52bdf9d700 1 -- 172.21.15.33:0/1166737287 wait complete. 2023-10-06T05:14:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f179279c700 1 --2- 172.21.15.33:0/2914214127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c06e920 0x7f178c168af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1780009890 tx=0x7f17800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff09cb8b3c011091 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f1790f99700 1 -- 172.21.15.33:0/2914214127 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0017d0 con 0x7f178c06e920 2023-10-06T05:14:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f1790f99700 1 -- 172.21.15.33:0/2914214127 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1780011420 con 0x7f178c06e920 2023-10-06T05:14:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.812+0000 7f1790f99700 1 -- 172.21.15.33:0/2914214127 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800117e0 con 0x7f178c06e920 2023-10-06T05:14:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.813+0000 7fc4737fe700 1 -- 172.21.15.33:0/2732620790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800b860 con 0x7fc46c0df480 2023-10-06T05:14:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.813+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00dce0 con 0x7f28a4004810 2023-10-06T05:14:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.813+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3580495a0 con 0x7fb36815d870 2023-10-06T05:14:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.813+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac04c380 con 0x7f88ac1553c0 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.813+0000 7f17941ff700 1 -- 172.21.15.33:0/2914214127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c06e920 msgr2=0x7f178c168af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.813+0000 7f17941ff700 1 --2- 172.21.15.33:0/2914214127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c06e920 0x7f178c168af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1780009890 tx=0x7f17800091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.814+0000 7f52bdf9d700 1 Processor -- start 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.814+0000 7f52bdf9d700 1 -- start start 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.815+0000 7f17941ff700 1 -- 172.21.15.33:0/2914214127 shutdown_connections 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.815+0000 7f17941ff700 1 --2- 172.21.15.33:0/2914214127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c06e920 0x7f178c168af0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.815+0000 7f17941ff700 1 -- 172.21.15.33:0/2914214127 >> 172.21.15.33:0/2914214127 conn(0x7f178c06d030 msgr2=0x7f178c0cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.815+0000 7f0a7139d700 1 Processor -- start 2023-10-06T05:14:32.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.815+0000 7f0a7139d700 1 -- start start 2023-10-06T05:14:32.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bdf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 0x7f52b00deae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bdf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88a801e460 con 0x7f88ac1553c0 2023-10-06T05:14:32.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f17941ff700 1 -- 172.21.15.33:0/2914214127 shutdown_connections 2023-10-06T05:14:32.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f17941ff700 1 -- 172.21.15.33:0/2914214127 wait complete. 2023-10-06T05:14:32.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a7139d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a640e6480 0x7f0a640e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a7139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a64008ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a7139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a64054260 con 0x7f0a640074c0 2023-10-06T05:14:32.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f88a27fc700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8890067080 0x7f8890069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a6b7fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0a640074c0 0x7f0a640510c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57788/0 (socket says 172.21.15.33:57788) 2023-10-06T05:14:32.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a6b7fe700 1 -- 172.21.15.33:0/2976996398 learned_addr learned my addr 172.21.15.33:0/2976996398 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:32.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f88a8085160 con 0x7f88ac1553c0 2023-10-06T05:14:32.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a6bfff700 1 --2- 172.21.15.33:0/2976996398 >> v2:172.21.15.33:3300/0 conn(0x7f0a640e6480 0x7f0a640e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a64051910 con 0x7f0a640e6480 2023-10-06T05:14:32.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a64053c10 con 0x7f0a64038c80 2023-10-06T05:14:32.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bcf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 0x7f52b00deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bcf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 0x7f52b00deae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37516/0 (socket says 172.21.15.33:37516) 2023-10-06T05:14:32.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bcf9b700 1 -- 172.21.15.33:0/272648866 learned_addr learned my addr 172.21.15.33:0/272648866 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b0038d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00f7600 con 0x7f52b0006920 2023-10-06T05:14:32.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0037a20 con 0x7f52b00e3e60 2023-10-06T05:14:32.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0038880 con 0x7f52b00ea190 2023-10-06T05:14:32.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.816+0000 7f52b7fff700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f52bd79c700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b0038d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.817+0000 7f0a70b9c700 1 --2- 172.21.15.33:0/2976996398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a64008ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f88b219b700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8890067080 0x7f8890069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f52bcf9b700 1 -- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 msgr2=0x7f52b0038d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f52bcf9b700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b0038d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f52bcf9b700 1 -- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 msgr2=0x7f52b00f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f52bcf9b700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f52bcf9b700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009420 con 0x7f52b0006920 2023-10-06T05:14:32.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a70b9c700 1 -- 172.21.15.33:0/2976996398 >> v2:172.21.15.33:3300/0 conn(0x7f0a640e6480 msgr2=0x7f0a640e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:32.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a70b9c700 1 --2- 172.21.15.33:0/2976996398 >> v2:172.21.15.33:3300/0 conn(0x7f0a640e6480 0x7f0a640e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a70b9c700 1 -- 172.21.15.33:0/2976996398 >> v1:172.21.15.167:6789/0 conn(0x7f0a640074c0 legacy=0x7f0a640510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a70b9c700 1 -- 172.21.15.33:0/2976996398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a64053ef0 con 0x7f0a64038c80 2023-10-06T05:14:32.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a70b9c700 1 --2- 172.21.15.33:0/2976996398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a64008ce0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0614b0 tx=0x7f0a6c065f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59cf36ecff028a3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a6affd700 1 -- 172.21.15.33:0/2976996398 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c065670 con 0x7f0a64038c80 2023-10-06T05:14:32.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f52bcf9b700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 0x7f52b00deae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52b000d9d0 tx=0x7f52ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f88b219b700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8890067080 0x7f8890069530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f889c006fd0 tx=0x7f889c00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a6affd700 1 -- 172.21.15.33:0/2976996398 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0657f0 con 0x7f0a64038c80 2023-10-06T05:14:32.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.818+0000 7f0a6affd700 1 -- 172.21.15.33:0/2976996398 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0631f0 con 0x7f0a64038c80 2023-10-06T05:14:32.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 msgr2=0x7f0a64008ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2976996398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a64008ce0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0614b0 tx=0x7f0a6c065f70 comp rx=0 tx=0).stop 2023-10-06T05:14:32.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 shutdown_connections 2023-10-06T05:14:32.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f17941ff700 1 Processor -- start 2023-10-06T05:14:32.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2976996398 >> v2:172.21.15.33:3300/0 conn(0x7f0a640e6480 0x7f0a640e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2976996398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a64008ce0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 >> 172.21.15.33:0/2976996398 conn(0x7f0a64052930 msgr2=0x7f0a64052d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.820+0000 7f17941ff700 1 -- start start 2023-10-06T05:14:32.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.819+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 shutdown_connections 2023-10-06T05:14:32.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.820+0000 7f0a7139d700 1 -- 172.21.15.33:0/2976996398 wait complete. 2023-10-06T05:14:32.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.820+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88a80522d0 con 0x7f88ac1553c0 2023-10-06T05:14:32.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.821+0000 7f17941ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f178c06e920 0x7f178c16f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f17941ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 0x7f178c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f17941ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f178c15dd10 0x7f178c0c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f0a7139d700 1 Processor -- start 2023-10-06T05:14:32.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f17941ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c3e40 con 0x7f178c06e920 2023-10-06T05:14:32.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac00de40 con 0x7f28a4004810 2023-10-06T05:14:32.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f28b9ffb700 1 -- 172.21.15.33:0/3675368737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac017670 con 0x7f28a4004810 2023-10-06T05:14:32.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f17941ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c16fc70 con 0x7f178c15dd10 2023-10-06T05:14:32.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f17941ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c3510 con 0x7f178c15d940 2023-10-06T05:14:32.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f179179a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 0x7f178c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f179179a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 0x7f178c0b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41940/0 (socket says 172.21.15.33:41940) 2023-10-06T05:14:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f179179a700 1 -- 172.21.15.33:0/92497807 learned_addr learned my addr 172.21.15.33:0/92497807 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f179279c700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f178c15dd10 0x7f178c0c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8788053540 con 0x7f87900de150 2023-10-06T05:14:32.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f0a7139d700 1 -- start start 2023-10-06T05:14:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7fc4737fe700 1 -- 172.21.15.33:0/2732620790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46800b9c0 con 0x7fc46c0df480 2023-10-06T05:14:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7fc4737fe700 1 -- 172.21.15.33:0/2732620790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800b860 con 0x7fc46c0df480 2023-10-06T05:14:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f1791f9b700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f178c06e920 0x7f178c16f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f28c13ff700 1 -- 172.21.15.33:0/3675368737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4004810 msgr2=0x7f28a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f28c13ff700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4004810 0x7f28a4004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a4004790 tx=0x7f28ac00f040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a7139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 0x7f0a640e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a7139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a640d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a7139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a640e6480 0x7f0a640e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a7139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640deb40 con 0x7f0a640e6480 2023-10-06T05:14:32.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac011420 con 0x7f52b0006920 2023-10-06T05:14:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.822+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac011580 con 0x7f52b0006920 2023-10-06T05:14:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac011420 con 0x7f52b0006920 2023-10-06T05:14:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b0037c70 con 0x7f52b0006920 2023-10-06T05:14:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b0038230 con 0x7f52b0006920 2023-10-06T05:14:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a7139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640e6d90 con 0x7f0a640074c0 2023-10-06T05:14:32.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a7139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640e2ca0 con 0x7f0a64038c80 2023-10-06T05:14:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 0x7f0a640e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f28c13ff700 1 -- 172.21.15.33:0/3675368737 shutdown_connections 2023-10-06T05:14:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f28c13ff700 1 --2- 172.21.15.33:0/3675368737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4004810 0x7f28a4004c80 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a4004790 tx=0x7f28ac00f040 comp rx=0 tx=0).stop 2023-10-06T05:14:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f28c13ff700 1 -- 172.21.15.33:0/3675368737 >> 172.21.15.33:0/3675368737 conn(0x7f28bc0675f0 msgr2=0x7f28bc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a6b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a640d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a6b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a640d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41956/0 (socket says 172.21.15.33:41956) 2023-10-06T05:14:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b0005380 con 0x7f52b0006920 2023-10-06T05:14:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f28c13ff700 1 -- 172.21.15.33:0/3675368737 shutdown_connections 2023-10-06T05:14:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a6b7fe700 1 -- 172.21.15.33:0/2769603849 learned_addr learned my addr 172.21.15.33:0/2769603849 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 0x7f0a640e6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40312/0 (socket says 172.21.15.33:40312) 2023-10-06T05:14:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7f0a70b9c700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a640e6480 0x7f0a640e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a6bfff700 1 -- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 msgr2=0x7f0a640d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a6bfff700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a640d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a6bfff700 1 -- 172.21.15.33:0/2769603849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a640e6480 msgr2=0x7f0a640e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a6bfff700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a640e6480 0x7f0a640e15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a6bfff700 1 -- 172.21.15.33:0/2769603849 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640e2f80 con 0x7f0a640074c0 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a70b9c700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a640e6480 0x7f0a640e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f0a6b7fe700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a640d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f179179a700 1 -- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f178c15dd10 msgr2=0x7f178c0c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f179179a700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f178c15dd10 0x7f178c0c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f179179a700 1 -- 172.21.15.33:0/92497807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f178c06e920 msgr2=0x7f178c16f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f179179a700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f178c06e920 0x7f178c16f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.823+0000 7f179179a700 1 -- 172.21.15.33:0/92497807 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009420 con 0x7f178c15d940 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7fc479b9d700 1 -- 172.21.15.33:0/2732620790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0df480 msgr2=0x7fc46c005f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.824+0000 7fc479b9d700 1 --2- 172.21.15.33:0/2732620790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0df480 0x7fc46c005f10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4680099b0 tx=0x7fc468009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7fc479b9d700 1 -- 172.21.15.33:0/2732620790 shutdown_connections 2023-10-06T05:14:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7fc479b9d700 1 --2- 172.21.15.33:0/2732620790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7fc479b9d700 1 --2- 172.21.15.33:0/2732620790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0df480 0x7fc46c005f10 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4680099b0 tx=0x7fc468009250 comp rx=0 tx=0).stop 2023-10-06T05:14:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7fc479b9d700 1 -- 172.21.15.33:0/2732620790 >> 172.21.15.33:0/2732620790 conn(0x7fc46c054c80 msgr2=0x7fc46c055080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.825+0000 7f28c13ff700 1 -- 172.21.15.33:0/3675368737 wait complete. 2023-10-06T05:14:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.827+0000 7fc479b9d700 1 -- 172.21.15.33:0/2732620790 shutdown_connections 2023-10-06T05:14:32.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.826+0000 7f28c13ff700 1 Processor -- start 2023-10-06T05:14:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f0a6bfff700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 0x7f0a640e6850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000e900 tx=0x7f0a600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60013680 con 0x7f0a640074c0 2023-10-06T05:14:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600137e0 con 0x7f0a640074c0 2023-10-06T05:14:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60013b80 con 0x7f0a640074c0 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.827+0000 7f28c13ff700 1 -- start start 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f28c13ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28bc14ebb0 0x7f28bc16d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f28c13ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28bc14ef80 0x7f28bc14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f28c13ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 0x7f28bc0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f28c13ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0aefb0 con 0x7f28bc14ebb0 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f28c13ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0af290 con 0x7f28bc14ef80 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f28c13ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc16e1e0 con 0x7f28bc14fa70 2023-10-06T05:14:32.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f28bb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 0x7f28bc0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f28bb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 0x7f28bc0af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41968/0 (socket says 172.21.15.33:41968) 2023-10-06T05:14:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f28bb7fe700 1 -- 172.21.15.33:0/847781022 learned_addr learned my addr 172.21.15.33:0/847781022 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a640e3260 con 0x7f0a640074c0 2023-10-06T05:14:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f28baffd700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28bc14ebb0 0x7f28bc16d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a640e3820 con 0x7f0a640074c0 2023-10-06T05:14:32.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f28ba7fc700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28bc14ef80 0x7f28bc14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28bb7fe700 1 -- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28bc14ef80 msgr2=0x7f28bc14f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28bb7fe700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28bc14ef80 0x7f28bc14f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28bb7fe700 1 -- 172.21.15.33:0/847781022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28bc14ebb0 msgr2=0x7f28bc16d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28bb7fe700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28bc14ebb0 0x7f28bc16d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28bb7fe700 1 -- 172.21.15.33:0/847781022 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc16e4c0 con 0x7f28bc14fa70 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28baffd700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28bc14ebb0 0x7f28bc16d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.830+0000 7f28bb7fe700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 0x7f28bc0af720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f28bc15a450 tx=0x7f28b000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00116d0 con 0x7f28bc14fa70 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0011830 con 0x7f28bc14fa70 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00116d0 con 0x7f28bc14fa70 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc16e7a0 con 0x7f28bc14fa70 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc0b15f0 con 0x7f28bc14fa70 2023-10-06T05:14:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7fc479b9d700 1 -- 172.21.15.33:0/2732620790 wait complete. 2023-10-06T05:14:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7fc479b9d700 1 Processor -- start 2023-10-06T05:14:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7fc479b9d700 1 -- start start 2023-10-06T05:14:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.832+0000 7f179179a700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 0x7f178c0b11f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f178400c4e0 tx=0x7f178400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784011590 con 0x7f178c15d940 2023-10-06T05:14:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1784011710 con 0x7f178c15d940 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784011ad0 con 0x7f178c15d940 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0c37f0 con 0x7f178c15d940 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.827+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52ac0116f0 con 0x7f52b0006920 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.828+0000 7f52b4ff9700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f529c066ff0 0x7f529c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f52b7fff700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f529c066ff0 0x7f529c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.829+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52ac057050 con 0x7f52b0006920 2023-10-06T05:14:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.831+0000 7f52b7fff700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f529c066ff0 0x7f529c0694a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52b804d170 tx=0x7f52b8065f70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7fc479b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7fc479b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 0x7fc46c04a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7fc479b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c10fae0 0x7fc46c036500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c0b1730 con 0x7f178c15d940 2023-10-06T05:14:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7fc478b9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7fc478b9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41984/0 (socket says 172.21.15.33:41984) 2023-10-06T05:14:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc04c380 con 0x7f28bc14fa70 2023-10-06T05:14:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.833+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a600123d0 con 0x7f0a640074c0 2023-10-06T05:14:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7f0a57fff700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0a50066fb0 0x7f0a50069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.835+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0a60084720 con 0x7f0a640074c0 2023-10-06T05:14:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.835+0000 7f0a55ffb700 1 -- 172.21.15.33:0/2769603849 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a440052b0 con 0x7f0a640074c0 2023-10-06T05:14:32.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.835+0000 7fc479b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c04cf90 con 0x7fc46c0de0a0 2023-10-06T05:14:32.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.835+0000 7fc479b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c04b440 con 0x7fc46c10fae0 2023-10-06T05:14:32.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.835+0000 7fc479b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c04a920 con 0x7fc46c006450 2023-10-06T05:14:32.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.834+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c068000 con 0x7f178c15d940 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 0x7fc46c04a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 0x7fc46c04a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37558/0 (socket says 172.21.15.33:37558) 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 -- 172.21.15.33:0/2476344341 learned_addr learned my addr 172.21.15.33:0/2476344341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 -- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 msgr2=0x7fc46c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 -- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c10fae0 msgr2=0x7fc46c036500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c10fae0 0x7fc46c036500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.836+0000 7fc473fff700 1 -- 172.21.15.33:0/2476344341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680094f0 con 0x7fc46c0de0a0 2023-10-06T05:14:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.837+0000 7fc473fff700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 0x7fc46c04a3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00d9d0 tx=0x7fc468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.838+0000 7fc478b9b700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-06T05:14:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7f0a6b7fe700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0a50066fb0 0x7f0a50069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.837+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f178401c440 con 0x7f178c15d940 2023-10-06T05:14:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.838+0000 7f1789ffb700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1778067080 0x7f1778069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.837+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28b0005a00 con 0x7f28bc14fa70 2023-10-06T05:14:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1784083e60 con 0x7f178c15d940 2023-10-06T05:14:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46801f070 con 0x7fc46c0de0a0 2023-10-06T05:14:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468011420 con 0x7fc46c0de0a0 2023-10-06T05:14:32.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800b860 con 0x7fc46c0de0a0 2023-10-06T05:14:32.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7f1791f9b700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1778067080 0x7f1778069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.839+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52ac053420 con 0x7f52b0006920 2023-10-06T05:14:32.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.840+0000 7fc479b9d700 1 -- 172.21.15.33:0/2476344341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c04cc60 con 0x7fc46c0de0a0 2023-10-06T05:14:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.840+0000 7f28a2ffd700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28a8066ff0 0x7f28a80694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.840+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f28b0015020 con 0x7f28bc14fa70 2023-10-06T05:14:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.840+0000 7fc479b9d700 1 -- 172.21.15.33:0/2476344341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c04ae80 con 0x7fc46c0de0a0 2023-10-06T05:14:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.840+0000 7f0a6b7fe700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0a50066fb0 0x7f0a50069460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a640e7680 tx=0x7f0a58007f40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.841+0000 7f28baffd700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28a8066ff0 0x7f28a80694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.841+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a60082e60 con 0x7f0a640074c0 2023-10-06T05:14:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.841+0000 7f1791f9b700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1778067080 0x7f1778069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f177c0013b0 tx=0x7f177c001680 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.842+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c04d100 con 0x7fc46c0de0a0 2023-10-06T05:14:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.843+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1784050fd0 con 0x7f178c15d940 2023-10-06T05:14:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.843+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4680119b0 con 0x7fc46c0de0a0 2023-10-06T05:14:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.843+0000 7f28baffd700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28a8066ff0 0x7f28a80694a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28b4007f40 tx=0x7f28b4010040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.844+0000 7fc470ff9700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc45806b340 0x7fc45806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.845+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc468056fc0 con 0x7fc46c0de0a0 2023-10-06T05:14:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.845+0000 7fc478b9b700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc45806b340 0x7fc45806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.845+0000 7fc478b9b700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc45806b340 0x7fc45806d7f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc4600017a0 tx=0x7fc460009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.846+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28b0050e30 con 0x7f28bc14fa70 2023-10-06T05:14:32.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:32.855+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc468053390 con 0x7fc46c0de0a0 2023-10-06T05:14:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.264+0000 7f879e19d700 1 -- 172.21.15.33:0/3868815564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8778005560 con 0x7f87900de150 2023-10-06T05:14:33.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.265+0000 7f878d7fa700 1 -- 172.21.15.33:0/3868815564 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8788056b60 con 0x7f87900de150 2023-10-06T05:14:33.268 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2023-10-06T05:14:33.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.270+0000 7f8782ffd700 1 -- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f877c067080 msgr2=0x7f877c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.270+0000 7f8782ffd700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f877c067080 0x7f877c069530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f879004b770 tx=0x7f8794008040 comp rx=0 tx=0).stop 2023-10-06T05:14:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.270+0000 7f8782ffd700 1 -- 172.21.15.33:0/3868815564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 msgr2=0x7f87900dcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.270+0000 7f8782ffd700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 0x7f87900dcec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8790052dd0 tx=0x7f8788016000 comp rx=0 tx=0).stop 2023-10-06T05:14:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.270+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb34c005830 con 0x7fb36815d870 2023-10-06T05:14:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7f8782ffd700 1 -- 172.21.15.33:0/3868815564 shutdown_connections 2023-10-06T05:14:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7f8782ffd700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f877c067080 0x7f877c069530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7f8782ffd700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87900dd400 0x7f8790102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7f8782ffd700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8790006500 0x7f87900f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7f8782ffd700 1 --2- 172.21.15.33:0/3868815564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87900de150 0x7f87900dcec0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7f8782ffd700 1 -- 172.21.15.33:0/3868815564 >> 172.21.15.33:0/3868815564 conn(0x7f8790054d30 msgr2=0x7f8790036fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.271+0000 7fb35f7fe700 1 -- 172.21.15.33:0/2474475903 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb358015090 con 0x7fb36815d870 2023-10-06T05:14:33.275 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2023-10-06T05:14:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.272+0000 7f8782ffd700 1 -- 172.21.15.33:0/3868815564 shutdown_connections 2023-10-06T05:14:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.272+0000 7f8782ffd700 1 -- 172.21.15.33:0/3868815564 wait complete. 2023-10-06T05:14:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.275+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb348067000 msgr2=0x7fb3480694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.275+0000 7fb367fff700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb348067000 0x7fb3480694b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb35000a010 tx=0x7fb350016000 comp rx=0 tx=0).stop 2023-10-06T05:14:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.276+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 msgr2=0x7fb3680c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.276+0000 7fb367fff700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb3680c1c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c850 tx=0x7fb358008040 comp rx=0 tx=0).stop 2023-10-06T05:14:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.276+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 shutdown_connections 2023-10-06T05:14:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.277+0000 7fb367fff700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb348067000 0x7fb3480694b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.277+0000 7fb367fff700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb36815dc40 0x7fb3680c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.277+0000 7fb367fff700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3680c6ea0 0x7fb36814c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.277+0000 7fb367fff700 1 --2- 172.21.15.33:0/2474475903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36815d870 0x7fb3680c1c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.277+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 >> 172.21.15.33:0/2474475903 conn(0x7fb36806c560 msgr2=0x7fb3680cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.278+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 shutdown_connections 2023-10-06T05:14:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.278+0000 7fb367fff700 1 -- 172.21.15.33:0/2474475903 wait complete. 2023-10-06T05:14:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.279+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f88ac069fd0 con 0x7f88ac1553c0 2023-10-06T05:14:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.280+0000 7f88a27fc700 1 -- 172.21.15.33:0/400656336 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88a80558f0 con 0x7f88ac1553c0 2023-10-06T05:14:33.282 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2023-10-06T05:14:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.284+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8890067080 msgr2=0x7f8890069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.284+0000 7f88b319d700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8890067080 0x7f8890069530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f889c006fd0 tx=0x7f889c00d040 comp rx=0 tx=0).stop 2023-10-06T05:14:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.285+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 msgr2=0x7f88ac154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.285+0000 7f88b319d700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 0x7f88ac154530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d860 tx=0x7f88a8008040 comp rx=0 tx=0).stop 2023-10-06T05:14:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.285+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 shutdown_connections 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.285+0000 7f88b319d700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8890067080 0x7f8890069530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.285+0000 7f88b319d700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88ac155790 0x7f88ac0c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.285+0000 7f88b319d700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac0ca6a0 0x7f88ac1762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.286+0000 7f88b319d700 1 --2- 172.21.15.33:0/400656336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88ac1553c0 0x7f88ac154530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.286+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 >> 172.21.15.33:0/400656336 conn(0x7f88ac06c560 msgr2=0x7f88ac0c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.286+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 shutdown_connections 2023-10-06T05:14:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.286+0000 7f88b319d700 1 -- 172.21.15.33:0/400656336 wait complete. 2023-10-06T05:14:33.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.292+0000 7f0a55ffb700 1 -- 172.21.15.33:0/2769603849 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a44005560 con 0x7f0a640074c0 2023-10-06T05:14:33.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.293+0000 7f0a57fff700 1 -- 172.21.15.33:0/2769603849 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a60021020 con 0x7f0a640074c0 2023-10-06T05:14:33.296 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:33.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.297+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f178c0c3650 con 0x7f178c15d940 2023-10-06T05:14:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.297+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0a50066fb0 msgr2=0x7f0a50069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.297+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0a50066fb0 0x7f0a50069460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a640e7680 tx=0x7f0a58007f40 comp rx=0 tx=0).stop 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.297+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 msgr2=0x7f0a640e6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.297+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 0x7f0a640e6850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000e900 tx=0x7f0a600091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.297+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 shutdown_connections 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0a50066fb0 0x7f0a50069460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a640e6480 0x7f0a640e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a640074c0 0x7f0a640e6850 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f0a7139d700 1 --2- 172.21.15.33:0/2769603849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a64038c80 0x7f0a640d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 >> 172.21.15.33:0/2769603849 conn(0x7f0a64052930 msgr2=0x7f0a6404ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f1789ffb700 1 -- 172.21.15.33:0/92497807 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f178c0c3650 con 0x7f178c15d940 2023-10-06T05:14:33.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.298+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f52b0007730 con 0x7f52b0006920 2023-10-06T05:14:33.303 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:33.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.299+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 shutdown_connections 2023-10-06T05:14:33.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.299+0000 7f52b4ff9700 1 -- 172.21.15.33:0/272648866 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f52ac053420 con 0x7f52b0006920 2023-10-06T05:14:33.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.299+0000 7f0a7139d700 1 -- 172.21.15.33:0/2769603849 wait complete. 2023-10-06T05:14:33.304 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.301+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1778067080 msgr2=0x7f1778069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.301+0000 7f17941ff700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1778067080 0x7f1778069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f177c0013b0 tx=0x7f177c001680 comp rx=0 tx=0).stop 2023-10-06T05:14:33.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.301+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 msgr2=0x7f178c0b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.301+0000 7f17941ff700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 0x7f178c0b11f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f178400c4e0 tx=0x7f178400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 shutdown_connections 2023-10-06T05:14:33.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1778067080 0x7f1778069530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f178c15dd10 0x7f178c0c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f178c06e920 0x7f178c16f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 --2- 172.21.15.33:0/92497807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178c15d940 0x7f178c0b11f0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f178400c4e0 tx=0x7f178400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 >> 172.21.15.33:0/92497807 conn(0x7f178c06d030 msgr2=0x7f178c0c10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.302+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 shutdown_connections 2023-10-06T05:14:33.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.303+0000 7f17941ff700 1 -- 172.21.15.33:0/92497807 wait complete. 2023-10-06T05:14:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.303+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f529c066ff0 msgr2=0x7f529c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.303+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f529c066ff0 0x7f529c0694a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52b804d170 tx=0x7f52b8065f70 comp rx=0 tx=0).stop 2023-10-06T05:14:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.303+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 msgr2=0x7f52b00deae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.303+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 0x7f52b00deae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52b000d9d0 tx=0x7f52ac017000 comp rx=0 tx=0).stop 2023-10-06T05:14:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f28bc069c10 con 0x7f28bc14fa70 2023-10-06T05:14:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 shutdown_connections 2023-10-06T05:14:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f529c066ff0 0x7f529c0694a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f52b00ea190 0x7f52b0038d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b0006920 0x7f52b00deae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f52bdf9d700 1 --2- 172.21.15.33:0/272648866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52b00e3e60 0x7f52b00f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 >> 172.21.15.33:0/272648866 conn(0x7f52b0054c80 msgr2=0x7f52b0049c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.304+0000 7f28a2ffd700 1 -- 172.21.15.33:0/847781022 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f28b0050e30 con 0x7f28bc14fa70 2023-10-06T05:14:33.311 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-10-06T05:14:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.305+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 shutdown_connections 2023-10-06T05:14:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.305+0000 7f52bdf9d700 1 -- 172.21.15.33:0/272648866 wait complete. 2023-10-06T05:14:33.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.1 2023-10-06T05:14:33.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.311+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc46c04d0d0 con 0x7fc46c0de0a0 2023-10-06T05:14:33.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.311+0000 7fc470ff9700 1 -- 172.21.15.33:0/2476344341 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc468053530 con 0x7fc46c0de0a0 2023-10-06T05:14:33.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.0 2023-10-06T05:14:33.315 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2023-10-06T05:14:33.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28a8066ff0 msgr2=0x7f28a80694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28a8066ff0 0x7f28a80694a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28b4007f40 tx=0x7f28b4010040 comp rx=0 tx=0).stop 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 msgr2=0x7f28bc0af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 0x7f28bc0af720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f28bc15a450 tx=0x7f28b000a330 comp rx=0 tx=0).stop 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 shutdown_connections 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28a8066ff0 0x7f28a80694a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28bc14fa70 0x7f28bc0af720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28bc14ebb0 0x7f28bc16d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 --2- 172.21.15.33:0/847781022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28bc14ef80 0x7f28bc14f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.312+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 >> 172.21.15.33:0/847781022 conn(0x7f28bc0675f0 msgr2=0x7f28bc06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.313+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 shutdown_connections 2023-10-06T05:14:33.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.313+0000 7f28c13ff700 1 -- 172.21.15.33:0/847781022 wait complete. 2023-10-06T05:14:33.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.2 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc45806b340 msgr2=0x7fc45806d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc45806b340 0x7fc45806d7f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc4600017a0 tx=0x7fc460009040 comp rx=0 tx=0).stop 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 msgr2=0x7fc46c04a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 0x7fc46c04a3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00d9d0 tx=0x7fc468017000 comp rx=0 tx=0).stop 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 shutdown_connections 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc45806b340 0x7fc45806d7f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c10fae0 0x7fc46c036500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46c006450 0x7fc46c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 --2- 172.21.15.33:0/2476344341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46c0de0a0 0x7fc46c04a3e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.316+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 >> 172.21.15.33:0/2476344341 conn(0x7fc46c054c80 msgr2=0x7fc46c051ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.317+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 shutdown_connections 2023-10-06T05:14:33.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:33.317+0000 7fc45e7fc700 1 -- 172.21.15.33:0/2476344341 wait complete. 2023-10-06T05:14:33.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.3 2023-10-06T05:14:33.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2023-10-06T05:14:33.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.7 2023-10-06T05:14:33.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-10-06T05:14:33.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.6 2023-10-06T05:14:34.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T05:14:34.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T05:14:34.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T05:14:34.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T05:14:34.334 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T05:14:34.345 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T05:14:34.345 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T05:14:34.345 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T05:14:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.616+0000 7f4073fff700 1 Processor -- start 2023-10-06T05:14:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.617+0000 7f4073fff700 1 -- start start 2023-10-06T05:14:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.619+0000 7f4073fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f406c0de0e0 0x7f406c053450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.619+0000 7f4073fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f406c04f450 0x7f406c033b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.619+0000 7f4073fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c006030 con 0x7f406c0de4b0 2023-10-06T05:14:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.619+0000 7f4073fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c053cc0 con 0x7f406c0de0e0 2023-10-06T05:14:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.619+0000 7f4073fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c053fa0 con 0x7f406c04f450 2023-10-06T05:14:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.620+0000 7f40727fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f406c04f450 0x7f406c033b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.620+0000 7f40737fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f406c0de4b0 0x7f406c0e1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39028/0 (socket says 172.21.15.33:39028) 2023-10-06T05:14:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.620+0000 7f40737fe700 1 -- 172.21.15.33:0/4267435972 learned_addr learned my addr 172.21.15.33:0/4267435972 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.621+0000 7f4071ffb700 1 -- 172.21.15.33:0/4267435972 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316427054 0 0) 0x7f406c006030 con 0x7f406c0de4b0 2023-10-06T05:14:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f4071ffb700 1 -- 172.21.15.33:0/4267435972 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f405c003570 con 0x7f406c0de4b0 2023-10-06T05:14:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f40727fc700 1 -- 172.21.15.33:0/4267435972 >> v2:172.21.15.33:3300/0 conn(0x7f406c0de0e0 msgr2=0x7f406c053450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f4072ffd700 1 --2- 172.21.15.33:0/4267435972 >> v2:172.21.15.33:3300/0 conn(0x7f406c0de0e0 0x7f406c053450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f4071ffb700 1 -- 172.21.15.33:0/4267435972 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522418093 0 0) 0x7f405c003570 con 0x7f406c0de4b0 2023-10-06T05:14:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f40727fc700 1 --2- 172.21.15.33:0/4267435972 >> v2:172.21.15.33:3300/0 conn(0x7f406c0de0e0 0x7f406c053450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f40727fc700 1 -- 172.21.15.33:0/4267435972 >> v1:172.21.15.33:6789/0 conn(0x7f406c0de4b0 legacy=0x7f406c0e1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.622+0000 7f40727fc700 1 -- 172.21.15.33:0/4267435972 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c054280 con 0x7f406c04f450 2023-10-06T05:14:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.623+0000 7f4072ffd700 1 --2- 172.21.15.33:0/4267435972 >> v2:172.21.15.33:3300/0 conn(0x7f406c0de0e0 0x7f406c053450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.623+0000 7f40727fc700 1 --2- 172.21.15.33:0/4267435972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c04f450 0x7f406c033b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40740614b0 tx=0x7f4074065f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df57d4cf814d2810 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.625+0000 7f4071ffb700 1 -- 172.21.15.33:0/4267435972 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064004cd0 con 0x7f406c04f450 2023-10-06T05:14:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.625+0000 7f4071ffb700 1 -- 172.21.15.33:0/4267435972 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407404c380 con 0x7f406c04f450 2023-10-06T05:14:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.625+0000 7f4071ffb700 1 -- 172.21.15.33:0/4267435972 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407406c0c0 con 0x7f406c04f450 2023-10-06T05:14:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f4073fff700 1 -- 172.21.15.33:0/4267435972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c04f450 msgr2=0x7f406c033b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f4073fff700 1 --2- 172.21.15.33:0/4267435972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c04f450 0x7f406c033b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40740614b0 tx=0x7f4074065f70 comp rx=0 tx=0).stop 2023-10-06T05:14:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f9b51b9d700 1 Processor -- start 2023-10-06T05:14:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f4073fff700 1 -- 172.21.15.33:0/4267435972 shutdown_connections 2023-10-06T05:14:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f4073fff700 1 --2- 172.21.15.33:0/4267435972 >> v2:172.21.15.33:3300/0 conn(0x7f406c0de0e0 0x7f406c053450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f4073fff700 1 --2- 172.21.15.33:0/4267435972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c04f450 0x7f406c033b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.626+0000 7f4073fff700 1 -- 172.21.15.33:0/4267435972 >> 172.21.15.33:0/4267435972 conn(0x7f406c051810 msgr2=0x7f406c051c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.627+0000 7f9b51b9d700 1 -- start start 2023-10-06T05:14:34.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.627+0000 7f4073fff700 1 -- 172.21.15.33:0/4267435972 shutdown_connections 2023-10-06T05:14:34.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.627+0000 7f4073fff700 1 -- 172.21.15.33:0/4267435972 wait complete. 2023-10-06T05:14:34.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.629+0000 7f9b51b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9b44038fb0 0x7f9b440e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.629+0000 7f9b51b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.629+0000 7f9b51b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440e2850 con 0x7f9b44039380 2023-10-06T05:14:34.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.629+0000 7f9b51b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b44007c60 con 0x7f9b44038fb0 2023-10-06T05:14:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.629+0000 7f9b51b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440070e0 con 0x7f9b44006a80 2023-10-06T05:14:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.630+0000 7f4073fff700 1 Processor -- start 2023-10-06T05:14:34.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.630+0000 7f9b5139c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.630+0000 7f9b5139c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440e2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41998/0 (socket says 172.21.15.33:41998) 2023-10-06T05:14:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.630+0000 7f9b5139c700 1 -- 172.21.15.33:0/416077735 learned_addr learned my addr 172.21.15.33:0/416077735 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.631+0000 7f9b5139c700 1 -- 172.21.15.33:0/416077735 >> v2:172.21.15.167:3300/0 conn(0x7f9b44038fb0 msgr2=0x7f9b440e1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.631+0000 7f9b5139c700 1 --2- 172.21.15.33:0/416077735 >> v2:172.21.15.167:3300/0 conn(0x7f9b44038fb0 0x7f9b440e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.631+0000 7f9b5139c700 1 -- 172.21.15.33:0/416077735 >> v1:172.21.15.33:6789/0 conn(0x7f9b44039380 legacy=0x7f9b44006370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.630+0000 7f4073fff700 1 -- start start 2023-10-06T05:14:34.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.632+0000 7f9b5139c700 1 -- 172.21.15.33:0/416077735 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440073c0 con 0x7f9b44006a80 2023-10-06T05:14:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.632+0000 7f4073fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f406c04f450 0x7f406c04c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.632+0000 7f4073fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 0x7f406c04c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f9b5139c700 1 --2- 172.21.15.33:0/416077735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440e2310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b380099b0 tx=0x7f9b38009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1b30467d936bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f4073fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f406c0de4b0 0x7f406c04ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f40727fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 0x7f406c04c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f40727fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 0x7f406c04c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40346/0 (socket says 172.21.15.33:40346) 2023-10-06T05:14:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f40727fc700 1 -- 172.21.15.33:0/3812461039 learned_addr learned my addr 172.21.15.33:0/3812461039 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f4072ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f406c04f450 0x7f406c04c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f9b4b7fe700 1 -- 172.21.15.33:0/416077735 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800b820 con 0x7f9b44006a80 2023-10-06T05:14:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7f40737fe700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f406c0de4b0 0x7f406c04ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.634+0000 7f9b4b7fe700 1 -- 172.21.15.33:0/416077735 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3800b980 con 0x7f9b44006a80 2023-10-06T05:14:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.634+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0dfc60 con 0x7f406c04f450 2023-10-06T05:14:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.634+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0d7840 con 0x7f406c0de0e0 2023-10-06T05:14:34.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.634+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0e0d80 con 0x7f406c0de4b0 2023-10-06T05:14:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 -- 172.21.15.33:0/416077735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 msgr2=0x7f9b440e2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 --2- 172.21.15.33:0/416077735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440e2310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b380099b0 tx=0x7f9b38009250 comp rx=0 tx=0).stop 2023-10-06T05:14:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 -- 172.21.15.33:0/416077735 shutdown_connections 2023-10-06T05:14:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 --2- 172.21.15.33:0/416077735 >> v2:172.21.15.167:3300/0 conn(0x7f9b44038fb0 0x7f9b440e1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 --2- 172.21.15.33:0/416077735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440e2310 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 -- 172.21.15.33:0/416077735 >> 172.21.15.33:0/416077735 conn(0x7f9b4400f1c0 msgr2=0x7f9b44059180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f40727fc700 1 -- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f406c0de4b0 msgr2=0x7f406c04ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 -- 172.21.15.33:0/416077735 shutdown_connections 2023-10-06T05:14:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7fe90b59e700 1 Processor -- start 2023-10-06T05:14:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.633+0000 7fe90b59e700 1 -- start start 2023-10-06T05:14:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7fe90b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040de030 0x7fe9040de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7fe90b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe9040506c0 0x7fe904053780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7fe90b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe904054690 con 0x7fe904053330 2023-10-06T05:14:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f40727fc700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f406c0de4b0 0x7f406c04ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f40727fc700 1 -- 172.21.15.33:0/3812461039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f406c04f450 msgr2=0x7f406c04c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7f40727fc700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f406c04f450 0x7f406c04c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7f40727fc700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0e1060 con 0x7f406c0de0e0 2023-10-06T05:14:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7f4072ffd700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f406c04f450 0x7f406c04c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7fe90b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040061f0 con 0x7fe9040506c0 2023-10-06T05:14:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.636+0000 7fe90b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe904053df0 con 0x7fe9040de030 2023-10-06T05:14:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.637+0000 7fe909d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe904053330 0x7fe90404ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39052/0 (socket says 172.21.15.33:39052) 2023-10-06T05:14:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.637+0000 7fe909d9b700 1 -- 172.21.15.33:0/2091023385 learned_addr learned my addr 172.21.15.33:0/2091023385 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.637+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040de030 0x7fe9040de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.637+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264486755 0 0) 0x7fe904054690 con 0x7fe904053330 2023-10-06T05:14:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.635+0000 7f9b51b9d700 1 -- 172.21.15.33:0/416077735 wait complete. 2023-10-06T05:14:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.637+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f4003570 con 0x7fe904053330 2023-10-06T05:14:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453552194 0 0) 0x7fe8f4003570 con 0x7fe904053330 2023-10-06T05:14:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040de030 msgr2=0x7fe9040de400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040de030 0x7fe9040de400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 >> v2:172.21.15.33:3300/0 conn(0x7fe9040506c0 msgr2=0x7fe904053780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:14:34.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 --2- 172.21.15.33:0/2091023385 >> v2:172.21.15.33:3300/0 conn(0x7fe9040506c0 0x7fe904053780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040540d0 con 0x7fe904053330 2023-10-06T05:14:34.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe9000048c0 con 0x7fe904053330 2023-10-06T05:14:34.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 >> v1:172.21.15.33:6789/0 conn(0x7fe904053330 legacy=0x7fe90404ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7f9b51b9d700 1 Processor -- start 2023-10-06T05:14:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4003a80 0x7fe8f4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f4004810 0x7fe8f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7f9b51b9d700 1 -- start start 2023-10-06T05:14:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f4005290 0x7fe8f4005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040061f0 con 0x7fe8f4003a80 2023-10-06T05:14:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904053df0 con 0x7fe8f4004810 2023-10-06T05:14:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.638+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4006220 con 0x7fe8f4005290 2023-10-06T05:14:34.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.639+0000 7f40727fc700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 0x7f406c04c770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4074061120 tx=0x7f407408c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.639+0000 7fe909d9b700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f4004810 0x7fe8f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.639+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4003a80 0x7fe8f4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.640+0000 7fe90a59c700 1 -- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f4005290 msgr2=0x7fe8f4005a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:14:34.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.640+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8f4005290 0x7fe8f4005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.640+0000 7f9b51b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.640+0000 7fe90a59c700 1 -- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f4004810 msgr2=0x7fe8f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.640+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f4004810 0x7fe8f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7fe90a59c700 1 -- 172.21.15.33:0/2091023385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040540d0 con 0x7fe8f4003a80 2023-10-06T05:14:34.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7fe909d9b700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8f4004810 0x7fe8f4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:34.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7f9b51b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b44038fb0 0x7f9b440520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4003a80 0x7fe8f4003eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe904007be0 tx=0x7fe8fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8604565af5a5ddd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7f9b50b9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7f9b50b9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:42030/0 (socket says 172.21.15.33:42030) 2023-10-06T05:14:34.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.641+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740646f0 con 0x7f406c0de0e0 2023-10-06T05:14:34.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074064870 con 0x7f406c0de0e0 2023-10-06T05:14:34.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407406d930 con 0x7f406c0de0e0 2023-10-06T05:14:34.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7f9b50b9b700 1 -- 172.21.15.33:0/2651773043 learned_addr learned my addr 172.21.15.33:0/2651773043 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c0e1340 con 0x7f406c0de0e0 2023-10-06T05:14:34.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c04d1f0 con 0x7f406c0de0e0 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc014430 con 0x7fe8f4003a80 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc014590 con 0x7fe8f4003a80 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.642+0000 7fe90959a700 1 -- 172.21.15.33:0/2091023385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc014950 con 0x7fe8f4003a80 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7fe90b59e700 1 -- 172.21.15.33:0/2091023385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4003a80 msgr2=0x7fe8f4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2091023385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8f4003a80 0x7fe8f4003eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe904007be0 tx=0x7fe8fc008040 comp rx=0 tx=0).stop 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7fe90ad9d700 1 -- 172.21.15.33:0/2091023385 reap_dead start 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7fe90b59e700 1 -- 172.21.15.33:0/2091023385 shutdown_connections 2023-10-06T05:14:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7fe90b59e700 1 -- 172.21.15.33:0/2091023385 >> 172.21.15.33:0/2091023385 conn(0x7fe904054da0 msgr2=0x7fe9040551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7f9b51b9d700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b44039380 0x7f9b440fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.643+0000 7f9b4bfff700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b44038fb0 0x7f9b440520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.644+0000 7f9b5139c700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b44039380 0x7f9b440fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.644+0000 7f9b51b9d700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4404faf0 con 0x7f9b44039380 2023-10-06T05:14:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.644+0000 7f9b51b9d700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440fef80 con 0x7f9b44038fb0 2023-10-06T05:14:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.644+0000 7f9b51b9d700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440533d0 con 0x7f9b44006a80 2023-10-06T05:14:34.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.644+0000 7fe90b59e700 1 -- 172.21.15.33:0/2091023385 shutdown_connections 2023-10-06T05:14:34.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.645+0000 7f9b5139c700 1 -- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 msgr2=0x7f9b440510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.645+0000 7f9b5139c700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.645+0000 7f9b5139c700 1 -- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b44038fb0 msgr2=0x7f9b440520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.645+0000 7f9b5139c700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b44038fb0 0x7f9b440520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.645+0000 7f9b5139c700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380094f0 con 0x7f9b44039380 2023-10-06T05:14:34.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.645+0000 7f9b4bfff700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b44038fb0 0x7f9b440520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:34.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.646+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c04d440 con 0x7f406c0de0e0 2023-10-06T05:14:34.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.644+0000 7fe90b59e700 1 -- 172.21.15.33:0/2091023385 wait complete. 2023-10-06T05:14:34.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.646+0000 7fe90b59e700 1 Processor -- start 2023-10-06T05:14:34.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.646+0000 7f9b5139c700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b44039380 0x7f9b440fea40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9b4404c950 tx=0x7f9b38016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.647+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38011e80 con 0x7f9b44039380 2023-10-06T05:14:34.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.647+0000 7fe90b59e700 1 -- start start 2023-10-06T05:14:34.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.647+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38020070 con 0x7f9b44039380 2023-10-06T05:14:34.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.647+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40740675f0 con 0x7f406c0de0e0 2023-10-06T05:14:34.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.648+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3801bb90 con 0x7f9b44039380 2023-10-06T05:14:34.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.648+0000 7f9b51b9d700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b440527c0 con 0x7f9b44039380 2023-10-06T05:14:34.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.649+0000 7fe90b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 0x7fe9040d6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.649+0000 7fe90a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 0x7fe9040d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.649+0000 7fe90a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 0x7fe9040d6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37592/0 (socket says 172.21.15.33:37592) 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.650+0000 7fe90a59c700 1 -- 172.21.15.33:0/2161747403 learned_addr learned my addr 172.21.15.33:0/2161747403 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.650+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040d28a0 0x7fe9040d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.650+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9040da280 0x7fe9040da700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.650+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040dac40 con 0x7fe9040d24d0 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.650+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040ee540 con 0x7fe9040da280 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.650+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040d7300 con 0x7fe9040d28a0 2023-10-06T05:14:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7f405affd700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4060067080 0x7f4060069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7fe90a59c700 1 -- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040d28a0 msgr2=0x7fe9040d6cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040d28a0 0x7fe9040d6cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7fe90a59c700 1 -- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9040da280 msgr2=0x7fe9040da700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:14:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9040da280 0x7fe9040da700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7fe90a59c700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040d75e0 con 0x7fe9040d24d0 2023-10-06T05:14:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f40740fb450 con 0x7f406c0de0e0 2023-10-06T05:14:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.651+0000 7fe90a59c700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 0x7fe9040d6790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006170 tx=0x7fe8fc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.652+0000 7f4072ffd700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4060067080 0x7f4060069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.652+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b38027480 con 0x7f9b44039380 2023-10-06T05:14:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.652+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc014430 con 0x7fe9040d24d0 2023-10-06T05:14:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.652+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00dca0 con 0x7fe9040d24d0 2023-10-06T05:14:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.653+0000 7f4072ffd700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4060067080 0x7f4060069530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f406800ab90 tx=0x7f4068008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.654+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9040d78c0 con 0x7fe9040d24d0 2023-10-06T05:14:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.654+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40740c8540 con 0x7f406c0de0e0 2023-10-06T05:14:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.654+0000 7f9b48ff9700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b30066f80 0x7f9b30069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.655+0000 7f9b50b9b700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b30066f80 0x7f9b30069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.655+0000 7f9b51b9d700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b44052d80 con 0x7f9b44039380 2023-10-06T05:14:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.654+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0273f0 con 0x7fe9040d24d0 2023-10-06T05:14:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.654+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9040d7e80 con 0x7fe9040d24d0 2023-10-06T05:14:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.657+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9b38022070 con 0x7f9b44039380 2023-10-06T05:14:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.657+0000 7f9b50b9b700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b30066f80 0x7f9b30069430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c066050 tx=0x7f9b4c06b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.657+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b44053760 con 0x7f9b44039380 2023-10-06T05:14:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.658+0000 7fe8ebfff700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90400b730 con 0x7fe9040d24d0 2023-10-06T05:14:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.659+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8fc030440 con 0x7fe9040d24d0 2023-10-06T05:14:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.661+0000 7fe8fa7fc700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8ec067000 0x7fe8ec0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.661+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8fc086230 con 0x7fe9040d24d0 2023-10-06T05:14:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.662+0000 7fe909d9b700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8ec067000 0x7fe8ec0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.664+0000 7fe909d9b700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8ec067000 0x7fe8ec0694b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9000011e0 tx=0x7fe900001130 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.668+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b44053760 con 0x7f9b44039380 2023-10-06T05:14:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.677+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8fc053320 con 0x7fe9040d24d0 2023-10-06T05:14:34.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.711+0000 7f535a19d700 1 Processor -- start 2023-10-06T05:14:34.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.712+0000 7f535a19d700 1 -- start start 2023-10-06T05:14:34.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.715+0000 7f535a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f535415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.715+0000 7f936f7ff700 1 Processor -- start 2023-10-06T05:14:34.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.715+0000 7f936f7ff700 1 -- start start 2023-10-06T05:14:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.715+0000 7f535a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53540c7070 0x7f53540ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.715+0000 7f535919b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f535415d870 0x7f53540c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57798/0 (socket says 172.21.15.33:57798) 2023-10-06T05:14:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.715+0000 7f535919b700 1 -- 172.21.15.33:0/392485623 learned_addr learned my addr 172.21.15.33:0/392485623 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.716+0000 7f535999c700 1 --2- 172.21.15.33:0/392485623 >> v2:172.21.15.33:3300/0 conn(0x7f53540c7070 0x7f53540ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.716+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535415ef60 con 0x7f535415d870 2023-10-06T05:14:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.716+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5354161940 con 0x7f53540c7070 2023-10-06T05:14:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.716+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5354161c20 con 0x7f535415dc40 2023-10-06T05:14:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f5353fff700 1 -- 172.21.15.33:0/392485623 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460355625 0 0) 0x7f5348001490 con 0x7f535415d870 2023-10-06T05:14:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f5353fff700 1 -- 172.21.15.33:0/392485623 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5344003570 con 0x7f535415d870 2023-10-06T05:14:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936f7ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f936815d9e0 0x7f9368150290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936f7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93681507d0 0x7f936815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936f7ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936815aed0 con 0x7f936815d610 2023-10-06T05:14:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936f7ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9368150c30 con 0x7f936815d9e0 2023-10-06T05:14:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936f7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93680cb680 con 0x7f93681507d0 2023-10-06T05:14:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.716+0000 7f535899a700 1 --2- 172.21.15.33:0/392485623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f535415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936d59b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f936815d610 0x7f93680cac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57806/0 (socket says 172.21.15.33:57806) 2023-10-06T05:14:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936d59b700 1 -- 172.21.15.33:0/2997690292 learned_addr learned my addr 172.21.15.33:0/2997690292 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936cd9a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f936815d9e0 0x7f9368150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f5353fff700 1 -- 172.21.15.33:0/392485623 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410899174 0 0) 0x7f5344003570 con 0x7f535415d870 2023-10-06T05:14:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.717+0000 7f936dd9c700 1 --2- 172.21.15.33:0/2997690292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93681507d0 0x7f936815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f535999c700 1 -- 172.21.15.33:0/392485623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 msgr2=0x7f535415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f37971ff700 1 Processor -- start 2023-10-06T05:14:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f535999c700 1 --2- 172.21.15.33:0/392485623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f535415ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f535999c700 1 -- 172.21.15.33:0/392485623 >> v1:172.21.15.167:6789/0 conn(0x7f535415d870 legacy=0x7f53540c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f535999c700 1 -- 172.21.15.33:0/392485623 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5354161f00 con 0x7f53540c7070 2023-10-06T05:14:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f935ffff700 1 -- 172.21.15.33:0/2997690292 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774572286 0 0) 0x7f936815aed0 con 0x7f936815d610 2023-10-06T05:14:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f936dd9c700 1 -- 172.21.15.33:0/2997690292 >> v2:172.21.15.167:3300/0 conn(0x7f936815d9e0 msgr2=0x7f9368150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f936dd9c700 1 --2- 172.21.15.33:0/2997690292 >> v2:172.21.15.167:3300/0 conn(0x7f936815d9e0 0x7f9368150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f936dd9c700 1 -- 172.21.15.33:0/2997690292 >> v1:172.21.15.167:6789/0 conn(0x7f936815d610 legacy=0x7f93680cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.718+0000 7f936dd9c700 1 -- 172.21.15.33:0/2997690292 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93680cb960 con 0x7f93681507d0 2023-10-06T05:14:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.719+0000 7f37971ff700 1 -- start start 2023-10-06T05:14:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.719+0000 7f535999c700 1 --2- 172.21.15.33:0/392485623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f53540ca050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f534c0098e0 tx=0x7f534c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa17ede309a9bf49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.720+0000 7f936dd9c700 1 --2- 172.21.15.33:0/2997690292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93681507d0 0x7f936815b380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93640098e0 tx=0x7f93640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f47cdc96934f85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.720+0000 7f5353fff700 1 -- 172.21.15.33:0/392485623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348001980 con 0x7f53540c7070 2023-10-06T05:14:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.720+0000 7fb16199d700 1 Processor -- start 2023-10-06T05:14:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.720+0000 7f5353fff700 1 -- 172.21.15.33:0/392485623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c011420 con 0x7f53540c7070 2023-10-06T05:14:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.720+0000 7f5353fff700 1 -- 172.21.15.33:0/392485623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0117a0 con 0x7f53540c7070 2023-10-06T05:14:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f37971ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f379015d9e0 0x7f3790150290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f37971ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37901507d0 0x7f379015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f37971ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379015aed0 con 0x7f379015d610 2023-10-06T05:14:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f37971ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3790150c30 con 0x7f379015d9e0 2023-10-06T05:14:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f37971ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900cb680 con 0x7f37901507d0 2023-10-06T05:14:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f3794f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f379015d610 0x7f37900cac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39068/0 (socket says 172.21.15.33:39068) 2023-10-06T05:14:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f3794f9b700 1 -- 172.21.15.33:0/3630167997 learned_addr learned my addr 172.21.15.33:0/3630167997 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 msgr2=0x7f53540ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f535a19d700 1 --2- 172.21.15.33:0/392485623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f53540ca050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f534c0098e0 tx=0x7f534c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.720+0000 7f935ffff700 1 -- 172.21.15.33:0/2997690292 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400b640 con 0x7f93681507d0 2023-10-06T05:14:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f935ffff700 1 -- 172.21.15.33:0/2997690292 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936400b7a0 con 0x7f93681507d0 2023-10-06T05:14:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7f935ffff700 1 -- 172.21.15.33:0/2997690292 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400bb60 con 0x7f93681507d0 2023-10-06T05:14:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828114811 0 0) 0x7f379015aed0 con 0x7f379015d610 2023-10-06T05:14:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3778003570 con 0x7f379015d610 2023-10-06T05:14:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 shutdown_connections 2023-10-06T05:14:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f535a19d700 1 --2- 172.21.15.33:0/392485623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f53540ca050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f535a19d700 1 --2- 172.21.15.33:0/392485623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f535415ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 >> 172.21.15.33:0/392485623 conn(0x7f535406c560 msgr2=0x7f535406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2835320306 0 0) 0x7f3778003570 con 0x7f379015d610 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 >> v2:172.21.15.167:3300/0 conn(0x7f37901507d0 msgr2=0x7f379015b380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 --2- 172.21.15.33:0/3630167997 >> v2:172.21.15.167:3300/0 conn(0x7f37901507d0 0x7f379015b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 >> v2:172.21.15.33:3300/0 conn(0x7f379015d9e0 msgr2=0x7f3790150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378ffff700 1 --2- 172.21.15.33:0/3630167997 >> v2:172.21.15.33:3300/0 conn(0x7f379015d9e0 0x7f3790150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 --2- 172.21.15.33:0/3630167997 >> v2:172.21.15.33:3300/0 conn(0x7f379015d9e0 0x7f3790150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.721+0000 7fb16199d700 1 -- start start 2023-10-06T05:14:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.722+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900cb960 con 0x7f379015d610 2023-10-06T05:14:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3788005260 con 0x7f379015d610 2023-10-06T05:14:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 >> v1:172.21.15.33:6789/0 conn(0x7f379015d610 legacy=0x7f37900cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3778003b30 0x7f3778003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7fb16199d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb15c1597d0 0x7fb15c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7fb16199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c0cc770 0x7fb15c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778004810 0x7f3778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7fb16199d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c0d0ed0 con 0x7fb15c159400 2023-10-06T05:14:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 shutdown_connections 2023-10-06T05:14:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778005410 0x7f377800bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790150c30 con 0x7f3778005410 2023-10-06T05:14:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900cb680 con 0x7f3778004810 2023-10-06T05:14:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377800c3c0 con 0x7f3778003b30 2023-10-06T05:14:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7fb16199d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c0b1160 con 0x7fb15c1597d0 2023-10-06T05:14:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7fb16199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0cb6d0 con 0x7fb15c0cc770 2023-10-06T05:14:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.723+0000 7f378ffff700 1 --2- 172.21.15.33:0/3630167997 >> v2:172.21.15.33:3300/0 conn(0x7f379015d9e0 0x7f3790150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f3794f9b700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3778003b30 0x7f3778003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f379579c700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778005410 0x7f377800bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f378ffff700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778004810 0x7f3778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f535a19d700 1 -- 172.21.15.33:0/392485623 wait complete. 2023-10-06T05:14:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7fb153fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb15c1597d0 0x7fb15c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7fb153fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb15c1597d0 0x7fb15c06c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40380/0 (socket says 172.21.15.33:40380) 2023-10-06T05:14:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7fb153fff700 1 -- 172.21.15.33:0/232661295 learned_addr learned my addr 172.21.15.33:0/232661295 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7fb16119c700 1 --2- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c0cc770 0x7fb15c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f936f7ff700 1 -- 172.21.15.33:0/2997690292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93681507d0 msgr2=0x7f936815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f936f7ff700 1 --2- 172.21.15.33:0/2997690292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93681507d0 0x7f936815b380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93640098e0 tx=0x7f93640091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f378ffff700 1 -- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3778003b30 msgr2=0x7f3778003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f378ffff700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3778003b30 0x7f3778003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f378ffff700 1 -- 172.21.15.33:0/3630167997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778005410 msgr2=0x7f377800bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f378ffff700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778005410 0x7f377800bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.724+0000 7f378ffff700 1 -- 172.21.15.33:0/3630167997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3778003570 con 0x7f3778004810 2023-10-06T05:14:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7fb153fff700 1 -- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c0cc770 msgr2=0x7fb15c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7fb153fff700 1 --2- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c0cc770 0x7fb15c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7fb153fff700 1 -- 172.21.15.33:0/232661295 >> v1:172.21.15.167:6789/0 conn(0x7fb15c159400 legacy=0x7fb15c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7fb153fff700 1 -- 172.21.15.33:0/232661295 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0cb9b0 con 0x7fb15c1597d0 2023-10-06T05:14:34.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f379579c700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778005410 0x7f377800bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:34.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f379579c700 1 -- 172.21.15.33:0/3630167997 reap_dead start 2023-10-06T05:14:34.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f936f7ff700 1 -- 172.21.15.33:0/2997690292 shutdown_connections 2023-10-06T05:14:34.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f936f7ff700 1 --2- 172.21.15.33:0/2997690292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93681507d0 0x7f936815b380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f936f7ff700 1 --2- 172.21.15.33:0/2997690292 >> v2:172.21.15.167:3300/0 conn(0x7f936815d9e0 0x7f9368150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f936f7ff700 1 -- 172.21.15.33:0/2997690292 >> 172.21.15.33:0/2997690292 conn(0x7f9368099960 msgr2=0x7f936806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f936f7ff700 1 -- 172.21.15.33:0/2997690292 shutdown_connections 2023-10-06T05:14:34.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f378ffff700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778004810 0x7f3778004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f378400abf0 tx=0x7f3784008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b832be093603682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7f936f7ff700 1 -- 172.21.15.33:0/2997690292 wait complete. 2023-10-06T05:14:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.725+0000 7fb153fff700 1 --2- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c06c560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1440099b0 tx=0x7fb144009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=729752148ec0789 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.726+0000 7facfd5ff700 1 Processor -- start 2023-10-06T05:14:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.726+0000 7facfd5ff700 1 -- start start 2023-10-06T05:14:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.726+0000 7fb15bfff700 1 -- 172.21.15.33:0/232661295 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14400b820 con 0x7fb15c1597d0 2023-10-06T05:14:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.726+0000 7fb15bfff700 1 -- 172.21.15.33:0/232661295 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14400b980 con 0x7fb15c1597d0 2023-10-06T05:14:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7f535a19d700 1 Processor -- start 2023-10-06T05:14:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7fb16199d700 1 -- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 msgr2=0x7fb15c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7fb16199d700 1 --2- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c06c560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1440099b0 tx=0x7fb144009250 comp rx=0 tx=0).stop 2023-10-06T05:14:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7f535a19d700 1 -- start start 2023-10-06T05:14:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7fb16199d700 1 -- 172.21.15.33:0/232661295 shutdown_connections 2023-10-06T05:14:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7fb16199d700 1 --2- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c0cc770 0x7fb15c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7fb16199d700 1 --2- 172.21.15.33:0/232661295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c06c560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.727+0000 7fb16199d700 1 -- 172.21.15.33:0/232661295 >> 172.21.15.33:0/232661295 conn(0x7fb15c06e4b0 msgr2=0x7fb15c0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7fb16199d700 1 -- 172.21.15.33:0/232661295 shutdown_connections 2023-10-06T05:14:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7f936f7ff700 1 Processor -- start 2023-10-06T05:14:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f936f7ff700 1 -- start start 2023-10-06T05:14:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7facfd5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7facfd5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7facf815dac0 0x7facf8159950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7facfd5ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf80ab390 con 0x7facf80aa250 2023-10-06T05:14:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7facfd5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf80cac60 con 0x7facf815dac0 2023-10-06T05:14:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7facfd5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80aa780 con 0x7facf815d6f0 2023-10-06T05:14:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7facf67fc700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7facf80aa250 0x7facf8159240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57826/0 (socket says 172.21.15.33:57826) 2023-10-06T05:14:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7facf67fc700 1 -- 172.21.15.33:0/4252072836 learned_addr learned my addr 172.21.15.33:0/4252072836 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7facf77fe700 1 --2- 172.21.15.33:0/4252072836 >> v2:172.21.15.33:3300/0 conn(0x7facf815dac0 0x7facf8159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.728+0000 7fb16199d700 1 -- 172.21.15.33:0/232661295 wait complete. 2023-10-06T05:14:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400a530 con 0x7f3778004810 2023-10-06T05:14:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400c6d0 con 0x7f3778004810 2023-10-06T05:14:34.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f378f7fe700 1 -- 172.21.15.33:0/3630167997 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400ca50 con 0x7f3778004810 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f535a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f535415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f535a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f535415d870 0x7f53540ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f535a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f53540add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f535a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540aebf0 con 0x7f53540c7070 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f535a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540af1e0 con 0x7f535415d870 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.729+0000 7f535a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540ae350 con 0x7f535415dc40 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f535919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f535415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f535919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f535415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37640/0 (socket says 172.21.15.33:37640) 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f535919b700 1 -- 172.21.15.33:0/3502465997 learned_addr learned my addr 172.21.15.33:0/3502465997 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f535899a700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f535415d870 0x7f53540ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f535999c700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f53540add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7f535899a700 1 -- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 msgr2=0x7f53540add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f535899a700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f53540add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f535899a700 1 -- 172.21.15.33:0/3502465997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 msgr2=0x7f535415a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f535899a700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f535415a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7fb16199d700 1 Processor -- start 2023-10-06T05:14:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7fb16199d700 1 -- start start 2023-10-06T05:14:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f936f7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 0x7f93681538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f535899a700 1 -- 172.21.15.33:0/3502465997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009420 con 0x7f535415d870 2023-10-06T05:14:34.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f936f7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 0x7f9368169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f535919b700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f535415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f37971ff700 1 -- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778004810 msgr2=0x7f3778004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f37971ff700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778004810 0x7f3778004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f378400abf0 tx=0x7f3784008040 comp rx=0 tx=0).stop 2023-10-06T05:14:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7facf77fe700 1 -- 172.21.15.33:0/4252072836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 msgr2=0x7facf80ca720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:14:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7facf77fe700 1 --2- 172.21.15.33:0/4252072836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80ca720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7facf77fe700 1 -- 172.21.15.33:0/4252072836 >> v1:172.21.15.167:6789/0 conn(0x7facf80aa250 legacy=0x7facf8159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.730+0000 7facf77fe700 1 -- 172.21.15.33:0/4252072836 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf80aaa60 con 0x7facf815dac0 2023-10-06T05:14:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f936d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 0x7f93681538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f936d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 0x7f93681538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37656/0 (socket says 172.21.15.33:37656) 2023-10-06T05:14:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f936cd9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 0x7f9368169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f936f7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f936815d9e0 0x7f93680c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7facf77fe700 1 --2- 172.21.15.33:0/4252072836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf8159950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7face80098e0 tx=0x7face80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cc61987708ad6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f37971ff700 1 -- 172.21.15.33:0/3630167997 shutdown_connections 2023-10-06T05:14:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f37971ff700 1 --2- 172.21.15.33:0/3630167997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778004810 0x7f3778004c80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.731+0000 7f37971ff700 1 -- 172.21.15.33:0/3630167997 >> 172.21.15.33:0/3630167997 conn(0x7f3790099960 msgr2=0x7f379006cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f37971ff700 1 -- 172.21.15.33:0/3630167997 shutdown_connections 2023-10-06T05:14:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f37971ff700 1 -- 172.21.15.33:0/3630167997 wait complete. 2023-10-06T05:14:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f936cd9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 0x7f9368169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:42112/0 (socket says 172.21.15.33:42112) 2023-10-06T05:14:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f936d59b700 1 -- 172.21.15.33:0/1895671138 learned_addr learned my addr 172.21.15.33:0/1895671138 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f936f7ff700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9368151650 con 0x7f93681507d0 2023-10-06T05:14:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f936f7ff700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93680b4810 con 0x7f936815d9e0 2023-10-06T05:14:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7f936f7ff700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936816a0e0 con 0x7f936815d610 2023-10-06T05:14:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.732+0000 7fb16199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 0x7fb15c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c159400 0x7fb15c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0b5540 con 0x7fb15c0cc770 2023-10-06T05:14:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7facf5ffb700 1 -- 172.21.15.33:0/4252072836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8009e80 con 0x7facf815dac0 2023-10-06T05:14:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 0x7fb15c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 0x7fb15c0c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37670/0 (socket says 172.21.15.33:37670) 2023-10-06T05:14:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16099b700 1 -- 172.21.15.33:0/2083790953 learned_addr learned my addr 172.21.15.33:0/2083790953 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0b5720 con 0x7fb15c1597d0 2023-10-06T05:14:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0b5a00 con 0x7fb15c159400 2023-10-06T05:14:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7fb16119c700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7fb16099b700 1 -- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c159400 msgr2=0x7fb15c1524c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7fb16099b700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c159400 0x7fb15c1524c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7fb16099b700 1 -- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 msgr2=0x7fb15c15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7fb16099b700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7fb16099b700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440094f0 con 0x7fb15c0cc770 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f535899a700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f535415d870 0x7f53540ad800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f534000d860 tx=0x7f5340008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340014620 con 0x7f535415d870 2023-10-06T05:14:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340014780 con 0x7f535415d870 2023-10-06T05:14:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340014620 con 0x7f535415d870 2023-10-06T05:14:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53540ae630 con 0x7f535415d870 2023-10-06T05:14:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7facf5ffb700 1 -- 172.21.15.33:0/4252072836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face800b8f0 con 0x7facf815dac0 2023-10-06T05:14:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7facfd5ff700 1 -- 172.21.15.33:0/4252072836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 msgr2=0x7facf8159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.734+0000 7fb16099b700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 0x7fb15c0c75d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c950 tx=0x7fb14c002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535415aa50 con 0x7f535415d870 2023-10-06T05:14:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f936dd9c700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f936815d9e0 0x7f93680c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f936dd9c700 1 -- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 msgr2=0x7f9368169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f936dd9c700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 0x7f9368169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.733+0000 7f936dd9c700 1 -- 172.21.15.33:0/1895671138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 msgr2=0x7f93681538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f37971ff700 1 Processor -- start 2023-10-06T05:14:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7facf5ffb700 1 -- 172.21.15.33:0/4252072836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8009e80 con 0x7facf815dac0 2023-10-06T05:14:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7facfd5ff700 1 --2- 172.21.15.33:0/4252072836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf8159950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7face80098e0 tx=0x7face80091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7facfd5ff700 1 -- 172.21.15.33:0/4252072836 shutdown_connections 2023-10-06T05:14:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7facfd5ff700 1 --2- 172.21.15.33:0/4252072836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf8159950 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7facfd5ff700 1 --2- 172.21.15.33:0/4252072836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7facfd5ff700 1 -- 172.21.15.33:0/4252072836 >> 172.21.15.33:0/4252072836 conn(0x7facf806d030 msgr2=0x7facf80cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f936dd9c700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 0x7f93681538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f936dd9c700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9364009420 con 0x7f936815d9e0 2023-10-06T05:14:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f936d59b700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 0x7f93681538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f936cd9a700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 0x7f9368169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7f37971ff700 1 -- start start 2023-10-06T05:14:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7f936dd9c700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f936815d9e0 0x7f93680c9830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9364016000 tx=0x7f9364017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00a7e0 con 0x7fb15c0cc770 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.735+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c00a960 con 0x7fb15c0cc770 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00ad20 con 0x7fb15c0cc770 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c0b5ce0 con 0x7fb15c0cc770 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c152a00 con 0x7fb15c0cc770 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400ba60 con 0x7f936815d9e0 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364026430 con 0x7f936815d9e0 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640267f0 con 0x7f936815d9e0 2023-10-06T05:14:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.736+0000 7f936f7ff700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936816a370 con 0x7f936815d9e0 2023-10-06T05:14:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.737+0000 7f936f7ff700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936816a930 con 0x7f936815d9e0 2023-10-06T05:14:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.737+0000 7f37971ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 0x7f3790152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f37971ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3790153c10 0x7f3790169a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f37971ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f379014f710 0x7f37900c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535404c380 con 0x7f535415d870 2023-10-06T05:14:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c152c50 con 0x7fb15c0cc770 2023-10-06T05:14:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f37971ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790153780 con 0x7f3778004810 2023-10-06T05:14:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f37971ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901500a0 con 0x7f3790153c10 2023-10-06T05:14:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f37971ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3790152b70 con 0x7f379014f710 2023-10-06T05:14:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7facfd5ff700 1 -- 172.21.15.33:0/4252072836 shutdown_connections 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f3794f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 0x7f3790152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f3794f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 0x7f3790152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37676/0 (socket says 172.21.15.33:37676) 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f3794f9b700 1 -- 172.21.15.33:0/3042573965 learned_addr learned my addr 172.21.15.33:0/3042573965 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.738+0000 7f378ffff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3790153c10 0x7f3790169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f379579c700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f379014f710 0x7f37900c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f3794f9b700 1 -- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f379014f710 msgr2=0x7f37900c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f3794f9b700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f379014f710 0x7f37900c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f3794f9b700 1 -- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3790153c10 msgr2=0x7f3790169a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f3794f9b700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3790153c10 0x7f3790169a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f3794f9b700 1 -- 172.21.15.33:0/3042573965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3790152e50 con 0x7f3778004810 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f379579c700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f379014f710 0x7f37900c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f378ffff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3790153c10 0x7f3790169a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f3794f9b700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 0x7f3790152520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f378800a570 tx=0x7f3788005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788013600 con 0x7f3778004810 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788013760 con 0x7f3778004810 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788013ae0 con 0x7f3778004810 2023-10-06T05:14:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3790153130 con 0x7f3778004810 2023-10-06T05:14:34.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.739+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9368063ff0 con 0x7f936815d9e0 2023-10-06T05:14:34.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb14c012630 con 0x7fb15c0cc770 2023-10-06T05:14:34.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53400148f0 con 0x7f535415d870 2023-10-06T05:14:34.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.740+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379014fb90 con 0x7f3778004810 2023-10-06T05:14:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.742+0000 7f53517fa700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5338066ff0 0x7f53380694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.742+0000 7fb1597fa700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb13c067000 0x7fb13c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.742+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f53400848d0 con 0x7f535415d870 2023-10-06T05:14:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.742+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb14c084510 con 0x7fb15c0cc770 2023-10-06T05:14:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.742+0000 7fb153fff700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb13c067000 0x7fb13c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.743+0000 7fb153fff700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb13c067000 0x7fb13c0694b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb1440099b0 tx=0x7fb144016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.743+0000 7f535919b700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5338066ff0 0x7f53380694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.743+0000 7facfd5ff700 1 -- 172.21.15.33:0/4252072836 wait complete. 2023-10-06T05:14:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.742+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9364023030 con 0x7f936815d9e0 2023-10-06T05:14:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.743+0000 7f935d7fa700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f934c067080 0x7f934c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.744+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37700052b0 con 0x7f3778004810 2023-10-06T05:14:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.743+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9364086040 con 0x7f936815d9e0 2023-10-06T05:14:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.744+0000 7f936d59b700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f934c067080 0x7f934c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.744+0000 7f535919b700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5338066ff0 0x7f53380694a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5348001890 tx=0x7f5348001610 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.745+0000 7f936d59b700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f934c067080 0x7f934c069530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f936000d780 tx=0x7f9360009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.745+0000 7facfd5ff700 1 Processor -- start 2023-10-06T05:14:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.745+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f378801c400 con 0x7f3778004810 2023-10-06T05:14:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.745+0000 7facfd5ff700 1 -- start start 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7f378cff9700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f377c067000 0x7f377c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facfd5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 0x7facf80b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facfd5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facfd5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf80ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facfd5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80acd10 con 0x7facf815dac0 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facfd5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80ad1a0 con 0x7facf80aa250 2023-10-06T05:14:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facfd5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80ac100 con 0x7facf815d6f0 2023-10-06T05:14:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9364053130 con 0x7f936815d9e0 2023-10-06T05:14:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7f378ffff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f377c067000 0x7f377c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f37880192c0 con 0x7f3778004810 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.747+0000 7facf6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 0x7facf80b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 0x7facf80b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40436/0 (socket says 172.21.15.33:40436) 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 -- 172.21.15.33:0/424527682 learned_addr learned my addr 172.21.15.33:0/424527682 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf67fc700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf77fe700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf80ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 -- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 msgr2=0x7facf80adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 -- 172.21.15.33:0/424527682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 msgr2=0x7facf80ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf80ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7facf6ffd700 1 -- 172.21.15.33:0/424527682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8009420 con 0x7facf80aa250 2023-10-06T05:14:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.749+0000 7facf77fe700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf80ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.749+0000 7facf6ffd700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 0x7facf80b17c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7facf000bbb0 tx=0x7facf0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7f378ffff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f377c067000 0x7f377c0694b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f378400abf0 tx=0x7f378401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.748+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb15c152c50 con 0x7fb15c0cc770 2023-10-06T05:14:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.752+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5340055470 con 0x7f535415d870 2023-10-06T05:14:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.753+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00092e0 con 0x7facf80aa250 2023-10-06T05:14:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.753+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf000c6f0 con 0x7facf80aa250 2023-10-06T05:14:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.753+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000cab0 con 0x7facf80aa250 2023-10-06T05:14:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.753+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3788052900 con 0x7f3778004810 2023-10-06T05:14:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.753+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf80ac3e0 con 0x7facf80aa250 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.753+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf80ac9a0 con 0x7facf80aa250 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.757+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7facf0012070 con 0x7facf80aa250 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.759+0000 7facdeffd700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7face4067000 0x7face40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.759+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7facf0055790 con 0x7facf80aa250 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.756+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf80acff0 con 0x7facf80aa250 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.761+0000 7facf67fc700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7face4067000 0x7face40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.764+0000 7facf67fc700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7face4067000 0x7face40694b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7facf80c9dc0 tx=0x7facec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:34.768+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facf0051b60 con 0x7facf80aa250 2023-10-06T05:14:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.103+0000 7f4073fff700 1 -- 172.21.15.33:0/3812461039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f406c0054e0 con 0x7f406c0de0e0 2023-10-06T05:14:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.104+0000 7f405affd700 1 -- 172.21.15.33:0/3812461039 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f407409c340 con 0x7f406c0de0e0 2023-10-06T05:14:35.107 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2023-10-06T05:14:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.108+0000 7f4058ff9700 1 -- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4060067080 msgr2=0x7f4060069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.108+0000 7f4058ff9700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4060067080 0x7f4060069530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f406800ab90 tx=0x7f4068008040 comp rx=0 tx=0).stop 2023-10-06T05:14:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.108+0000 7f4058ff9700 1 -- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 msgr2=0x7f406c04c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 0x7f406c04c770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4074061120 tx=0x7f407408c000 comp rx=0 tx=0).stop 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 -- 172.21.15.33:0/3812461039 shutdown_connections 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4060067080 0x7f4060069530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f406c0de4b0 0x7f406c04ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f406c04f450 0x7f406c04c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 --2- 172.21.15.33:0/3812461039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f406c0de0e0 0x7f406c04c770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.109+0000 7f4058ff9700 1 -- 172.21.15.33:0/3812461039 >> 172.21.15.33:0/3812461039 conn(0x7f406c051810 msgr2=0x7f406c04b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.110+0000 7f4058ff9700 1 -- 172.21.15.33:0/3812461039 shutdown_connections 2023-10-06T05:14:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.110+0000 7f4058ff9700 1 -- 172.21.15.33:0/3812461039 wait complete. 2023-10-06T05:14:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.122+0000 7fe8ebfff700 1 -- 172.21.15.33:0/2161747403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe9040d8070 con 0x7fe9040d24d0 2023-10-06T05:14:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.123+0000 7fe8fa7fc700 1 -- 172.21.15.33:0/2161747403 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe8fc056940 con 0x7fe9040d24d0 2023-10-06T05:14:35.126 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2023-10-06T05:14:35.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.126+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b44053760 con 0x7f9b44039380 2023-10-06T05:14:35.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.127+0000 7f9b48ff9700 1 -- 172.21.15.33:0/2651773043 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b44053760 con 0x7f9b44039380 2023-10-06T05:14:35.130 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2023-10-06T05:14:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.129+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8ec067000 msgr2=0x7fe8ec0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.129+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8ec067000 0x7fe8ec0694b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9000011e0 tx=0x7fe900001130 comp rx=0 tx=0).stop 2023-10-06T05:14:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.129+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 msgr2=0x7fe9040d6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.129+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 0x7fe9040d6790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006170 tx=0x7fe8fc01b000 comp rx=0 tx=0).stop 2023-10-06T05:14:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.130+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 shutdown_connections 2023-10-06T05:14:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.130+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8ec067000 0x7fe8ec0694b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.130+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9040da280 0x7fe9040da700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.130+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9040d24d0 0x7fe9040d6790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.130+0000 7fe90b59e700 1 --2- 172.21.15.33:0/2161747403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9040d28a0 0x7fe9040d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.130+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 >> 172.21.15.33:0/2161747403 conn(0x7fe904054da0 msgr2=0x7fe904035e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.131+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 shutdown_connections 2023-10-06T05:14:35.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.131+0000 7fe90b59e700 1 -- 172.21.15.33:0/2161747403 wait complete. 2023-10-06T05:14:35.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.131+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b30066f80 msgr2=0x7f9b30069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.131+0000 7f9b367fc700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b30066f80 0x7f9b30069430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c066050 tx=0x7f9b4c06b040 comp rx=0 tx=0).stop 2023-10-06T05:14:35.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b44039380 msgr2=0x7f9b440fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b44039380 0x7f9b440fea40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9b4404c950 tx=0x7f9b38016000 comp rx=0 tx=0).stop 2023-10-06T05:14:35.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 shutdown_connections 2023-10-06T05:14:35.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b30066f80 0x7f9b30069430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b44039380 0x7f9b440fea40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b44006a80 0x7f9b440510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 --2- 172.21.15.33:0/2651773043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b44038fb0 0x7f9b440520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.132+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 >> 172.21.15.33:0/2651773043 conn(0x7f9b4400f1c0 msgr2=0x7f9b4403d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.133+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 shutdown_connections 2023-10-06T05:14:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.133+0000 7f9b367fc700 1 -- 172.21.15.33:0/2651773043 wait complete. 2023-10-06T05:14:35.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-10-06T05:14:35.143 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-10-06T05:14:35.168 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-10-06T05:14:35.169 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.173+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb15c152c50 con 0x7fb15c0cc770 2023-10-06T05:14:35.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.174+0000 7fb1597fa700 1 -- 172.21.15.33:0/2083790953 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb15c152c50 con 0x7fb15c0cc770 2023-10-06T05:14:35.176 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:35.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.176+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb13c067000 msgr2=0x7fb13c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.176+0000 7fb16199d700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb13c067000 0x7fb13c0694b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb1440099b0 tx=0x7fb144016000 comp rx=0 tx=0).stop 2023-10-06T05:14:35.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.176+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 msgr2=0x7fb15c0c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.176+0000 7fb16199d700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 0x7fb15c0c75d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c950 tx=0x7fb14c002760 comp rx=0 tx=0).stop 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 shutdown_connections 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb13c067000 0x7fb13c0694b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb15c1597d0 0x7fb15c15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb15c0cc770 0x7fb15c0c75d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 --2- 172.21.15.33:0/2083790953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb15c159400 0x7fb15c1524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 >> 172.21.15.33:0/2083790953 conn(0x7fb15c06e4b0 msgr2=0x7fb15c0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 shutdown_connections 2023-10-06T05:14:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.177+0000 7fb16199d700 1 -- 172.21.15.33:0/2083790953 wait complete. 2023-10-06T05:14:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.179+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f93680b4950 con 0x7f936815d9e0 2023-10-06T05:14:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.180+0000 7f935d7fa700 1 -- 172.21.15.33:0/1895671138 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9364056750 con 0x7f936815d9e0 2023-10-06T05:14:35.184 INFO:teuthology.orchestra.run.smithi033.stdout:55834574857 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f934c067080 msgr2=0x7f934c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f934c067080 0x7f934c069530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f936000d780 tx=0x7f9360009040 comp rx=0 tx=0).stop 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f936815d9e0 msgr2=0x7f93680c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f936815d9e0 0x7f93680c9830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9364016000 tx=0x7f9364017000 comp rx=0 tx=0).stop 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 shutdown_connections 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f934c067080 0x7f934c069530 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f936815d9e0 0x7f93680c9830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93681507d0 0x7f93681538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 --2- 172.21.15.33:0/1895671138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f936815d610 0x7f9368169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 >> 172.21.15.33:0/1895671138 conn(0x7f9368099960 msgr2=0x7f93680ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 shutdown_connections 2023-10-06T05:14:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.184+0000 7f9352ffd700 1 -- 172.21.15.33:0/1895671138 wait complete. 2023-10-06T05:14:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.187+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3770005560 con 0x7f3778004810 2023-10-06T05:14:35.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.187+0000 7f378cff9700 1 -- 172.21.15.33:0/3042573965 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3788017020 con 0x7f3778004810 2023-10-06T05:14:35.190 INFO:teuthology.orchestra.run.smithi033.stdout:55834574857 2023-10-06T05:14:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f377c067000 msgr2=0x7f377c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f377c067000 0x7f377c0694b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f378400abf0 tx=0x7f378401a000 comp rx=0 tx=0).stop 2023-10-06T05:14:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 msgr2=0x7f3790152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 0x7f3790152520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f378800a570 tx=0x7f3788005b70 comp rx=0 tx=0).stop 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 shutdown_connections 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f377c067000 0x7f377c0694b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f379014f710 0x7f37900c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3778004810 0x7f3790152520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 --2- 172.21.15.33:0/3042573965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3790153c10 0x7f3790169a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.191+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 >> 172.21.15.33:0/3042573965 conn(0x7f3790099960 msgr2=0x7f37900cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.192+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 shutdown_connections 2023-10-06T05:14:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.192+0000 7f37971ff700 1 -- 172.21.15.33:0/3042573965 wait complete. 2023-10-06T05:14:35.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-06T05:14:35.200 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.203+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f535406d2b0 con 0x7f535415d870 2023-10-06T05:14:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.203+0000 7f53517fa700 1 -- 172.21.15.33:0/3502465997 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5340055470 con 0x7f535415d870 2023-10-06T05:14:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:55834574857 2023-10-06T05:14:35.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-10-06T05:14:35.209 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.206+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5338066ff0 msgr2=0x7f53380694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.206+0000 7f535a19d700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5338066ff0 0x7f53380694a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5348001890 tx=0x7f5348001610 comp rx=0 tx=0).stop 2023-10-06T05:14:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.206+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f535415d870 msgr2=0x7f53540ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.206+0000 7f535a19d700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f535415d870 0x7f53540ad800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f534000d860 tx=0x7f5340008040 comp rx=0 tx=0).stop 2023-10-06T05:14:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 shutdown_connections 2023-10-06T05:14:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5338066ff0 0x7f53380694a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f535415dc40 0x7f53540add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53540c7070 0x7f535415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 --2- 172.21.15.33:0/3502465997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f535415d870 0x7f53540ad800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 >> 172.21.15.33:0/3502465997 conn(0x7f535406c560 msgr2=0x7f53540c21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 shutdown_connections 2023-10-06T05:14:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.207+0000 7f535a19d700 1 -- 172.21.15.33:0/3502465997 wait complete. 2023-10-06T05:14:35.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-10-06T05:14:35.214 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.216+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7facf804c380 con 0x7facf80aa250 2023-10-06T05:14:35.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.217+0000 7facdeffd700 1 -- 172.21.15.33:0/424527682 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7facf0051b60 con 0x7facf80aa250 2023-10-06T05:14:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7face4067000 msgr2=0x7face40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7face4067000 0x7face40694b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7facf80c9dc0 tx=0x7facec009040 comp rx=0 tx=0).stop 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 msgr2=0x7facf80b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 0x7facf80b17c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7facf000bbb0 tx=0x7facf0009840 comp rx=0 tx=0).stop 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 shutdown_connections 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7face4067000 0x7face40694b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facf815dac0 0x7facf80ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf80aa250 0x7facf80b17c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 --2- 172.21.15.33:0/424527682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facf815d6f0 0x7facf80adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 >> 172.21.15.33:0/424527682 conn(0x7facf806d030 msgr2=0x7facf81528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 shutdown_connections 2023-10-06T05:14:35.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.221+0000 7facfd5ff700 1 -- 172.21.15.33:0/424527682 wait complete. 2023-10-06T05:14:35.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-10-06T05:14:35.233 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T05:14:35.243 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:35.243 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:14:35.243 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:14:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.466+0000 7fec6a19d700 1 Processor -- start 2023-10-06T05:14:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.466+0000 7fec6a19d700 1 -- start start 2023-10-06T05:14:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.467+0000 7fec6a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec640afdd0 0x7fec640b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.467+0000 7fec6a19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6406f220 con 0x7fec6406e140 2023-10-06T05:14:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.467+0000 7fec6a19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6406e610 con 0x7fec64159530 2023-10-06T05:14:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.467+0000 7fec6a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640b1e00 con 0x7fec640afdd0 2023-10-06T05:14:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec640afdd0 0x7fec640b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6899a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fec6406e140 0x7fec6415a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39082/0 (socket says 172.21.15.33:39082) 2023-10-06T05:14:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6899a700 1 -- 172.21.15.33:0/351462479 learned_addr learned my addr 172.21.15.33:0/351462479 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6999c700 1 -- 172.21.15.33:0/351462479 >> v1:172.21.15.167:6789/0 conn(0x7fec64159530 legacy=0x7fec6406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6999c700 1 -- 172.21.15.33:0/351462479 >> v1:172.21.15.33:6789/0 conn(0x7fec6406e140 legacy=0x7fec6415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec5bfff700 1 -- 172.21.15.33:0/351462479 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818454986 0 0) 0x7fec6406f220 con 0x7fec6406e140 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6999c700 1 -- 172.21.15.33:0/351462479 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6406e8f0 con 0x7fec640afdd0 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.468+0000 7fec6999c700 1 --2- 172.21.15.33:0/351462479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640b01a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009e60 tx=0x7fec5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af15044e5d5324b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec5bfff700 1 -- 172.21.15.33:0/351462479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00e070 con 0x7fec640afdd0 2023-10-06T05:14:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec5bfff700 1 -- 172.21.15.33:0/351462479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00bc30 con 0x7fec640afdd0 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec5bfff700 1 -- 172.21.15.33:0/351462479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012430 con 0x7fec640afdd0 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 -- 172.21.15.33:0/351462479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 msgr2=0x7fec640b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 --2- 172.21.15.33:0/351462479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640b01a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009e60 tx=0x7fec5c009780 comp rx=0 tx=0).stop 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 -- 172.21.15.33:0/351462479 shutdown_connections 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 --2- 172.21.15.33:0/351462479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640b01a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 -- 172.21.15.33:0/351462479 >> 172.21.15.33:0/351462479 conn(0x7fec6406c560 msgr2=0x7fec6406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 -- 172.21.15.33:0/351462479 shutdown_connections 2023-10-06T05:14:35.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.469+0000 7fec6a19d700 1 -- 172.21.15.33:0/351462479 wait complete. 2023-10-06T05:14:35.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.471+0000 7fec6a19d700 1 Processor -- start 2023-10-06T05:14:35.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.471+0000 7fec6a19d700 1 -- start start 2023-10-06T05:14:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406e140 0x7fec64154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640a8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64159530 0x7fec640a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6416f070 con 0x7fec640afdd0 2023-10-06T05:14:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6416f350 con 0x7fec64159530 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6416f630 con 0x7fec6406e140 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37704/0 (socket says 172.21.15.33:37704) 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6899a700 1 -- 172.21.15.33:0/2519104524 learned_addr learned my addr 172.21.15.33:0/2519104524 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6919b700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406e140 0x7fec64154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.472+0000 7fec6999c700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64159530 0x7fec640a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6899a700 1 -- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406e140 msgr2=0x7fec64154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6899a700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406e140 0x7fec64154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6899a700 1 -- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64159530 msgr2=0x7fec640a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6899a700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64159530 0x7fec640a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6899a700 1 -- 172.21.15.33:0/2519104524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009a20 con 0x7fec640afdd0 2023-10-06T05:14:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6999c700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64159530 0x7fec640a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6899a700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640a8190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c5b0 tx=0x7fec54001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54005b30 con 0x7fec640afdd0 2023-10-06T05:14:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54005c90 con 0x7fec640afdd0 2023-10-06T05:14:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6416f910 con 0x7fec640afdd0 2023-10-06T05:14:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54007300 con 0x7fec640afdd0 2023-10-06T05:14:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.473+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec640a57e0 con 0x7fec640afdd0 2023-10-06T05:14:35.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.474+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6416fe30 con 0x7fec640afdd0 2023-10-06T05:14:35.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.475+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec5401b3b0 con 0x7fec640afdd0 2023-10-06T05:14:35.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.476+0000 7fec597fa700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.476+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec54083c60 con 0x7fec640afdd0 2023-10-06T05:14:35.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.476+0000 7fec6919b700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.477+0000 7fec6919b700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec64157480 tx=0x7fec60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.480+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec6416fe30 con 0x7fec640afdd0 2023-10-06T05:14:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.678+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec6416fe30 con 0x7fec48066f30 2023-10-06T05:14:35.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.688+0000 7fec597fa700 1 -- 172.21.15.33:0/2519104524 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286595 (secure 0 0 0) 0x7fec6416fe30 con 0x7fec48066f30 2023-10-06T05:14:35.695 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 msgr2=0x7fec480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec64157480 tx=0x7fec60009040 comp rx=0 tx=0).stop 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 msgr2=0x7fec640a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640a8190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c5b0 tx=0x7fec54001070 comp rx=0 tx=0).stop 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 shutdown_connections 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64159530 0x7fec640a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406e140 0x7fec64154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 --2- 172.21.15.33:0/2519104524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec640afdd0 0x7fec640a8190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 >> 172.21.15.33:0/2519104524 conn(0x7fec6406c560 msgr2=0x7fec64158740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 shutdown_connections 2023-10-06T05:14:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.692+0000 7fec6a19d700 1 -- 172.21.15.33:0/2519104524 wait complete. 2023-10-06T05:14:35.698 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:14:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T05:14:33.819541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.114642"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740130+0000","last_change":"2023-10-06T05:14:25.733340+0000","last_active":"2023-10-06T05:14:25.740130+0000","last_peered":"2023-10-06T05:14:25.740130+0000","last_clean":"2023-10-06T05:14:25.740130+0000","last_became_active":"2023-10-06T05:14:25.732861+0000","last_became_peered":"2023-10-06T05:14:25.732861+0000","last_unstale":"2023-10-06T05:14:25.740130+0000","last_undegraded":"2023-10-06T05:14:25.740130+0000","last_fullsized":"2023-10-06T05:14:25.740130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:19.965653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733215+0000","last_change":"2023-10-06T05:14:25.730206+0000","last_active":"2023-10-06T05:14:25.733215+0000","last_peered":"2023-10-06T05:14:25.733215+0000","last_clean":"2023-10-06T05:14:25.733215+0000","last_became_active":"2023-10-06T05:14:25.729947+0000","last_became_peered":"2023-10-06T05:14:25.729947+0000","last_unstale":"2023-10-06T05:14:25.733215+0000","last_undegraded":"2023-10-06T05:14:25.733215+0000","last_fullsized":"2023-10-06T05:14:25.733215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747275+0000","last_change":"2023-10-06T05:14:25.744108+0000","last_active":"2023-10-06T05:14:25.747275+0000","last_peered":"2023-10-06T05:14:25.747275+0000","last_clean":"2023-10-06T05:14:25.747275+0000","last_became_active":"2023-10-06T05:14:25.741869+0000","last_became_peered":"2023-10-06T05:14:25.741869+0000","last_unstale":"2023-10-06T05:14:25.747275+0000","last_undegraded":"2023-10-06T05:14:25.747275+0000","last_fullsized":"2023-10-06T05:14:25.747275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:26.682160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727785+0000","last_change":"2023-10-06T05:14:25.724254+0000","last_active":"2023-10-06T05:14:25.727785+0000","last_peered":"2023-10-06T05:14:25.727785+0000","last_clean":"2023-10-06T05:14:25.727785+0000","last_became_active":"2023-10-06T05:14:25.723995+0000","last_became_peered":"2023-10-06T05:14:25.723995+0000","last_unstale":"2023-10-06T05:14:25.727785+0000","last_undegraded":"2023-10-06T05:14:25.727785+0000","last_fullsized":"2023-10-06T05:14:25.727785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:07.229350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735821+0000","last_change":"2023-10-06T05:14:25.726644+0000","last_active":"2023-10-06T05:14:25.735821+0000","last_peered":"2023-10-06T05:14:25.735821+0000","last_clean":"2023-10-06T05:14:25.735821+0000","last_became_active":"2023-10-06T05:14:25.726030+0000","last_became_peered":"2023-10-06T05:14:25.726030+0000","last_unstale":"2023-10-06T05:14:25.735821+0000","last_undegraded":"2023-10-06T05:14:25.735821+0000","last_fullsized":"2023-10-06T05:14:25.735821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747166+0000","last_change":"2023-10-06T05:14:25.743830+0000","last_active":"2023-10-06T05:14:25.747166+0000","last_peered":"2023-10-06T05:14:25.747166+0000","last_clean":"2023-10-06T05:14:25.747166+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.747166+0000","last_undegraded":"2023-10-06T05:14:25.747166+0000","last_fullsized":"2023-10-06T05:14:25.747166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:43.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739182+0000","last_change":"2023-10-06T05:14:25.737791+0000","last_active":"2023-10-06T05:14:25.739182+0000","last_peered":"2023-10-06T05:14:25.739182+0000","last_clean":"2023-10-06T05:14:25.739182+0000","last_became_active":"2023-10-06T05:14:25.737334+0000","last_became_peered":"2023-10-06T05:14:25.737334+0000","last_unstale":"2023-10-06T05:14:25.739182+0000","last_undegraded":"2023-10-06T05:14:25.739182+0000","last_fullsized":"2023-10-06T05:14:25.739182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744210+0000","last_change":"2023-10-06T05:14:25.743380+0000","last_active":"2023-10-06T05:14:25.744210+0000","last_peered":"2023-10-06T05:14:25.744210+0000","last_clean":"2023-10-06T05:14:25.744210+0000","last_became_active":"2023-10-06T05:14:25.743115+0000","last_became_peered":"2023-10-06T05:14:25.743115+0000","last_unstale":"2023-10-06T05:14:25.744210+0000","last_undegraded":"2023-10-06T05:14:25.744210+0000","last_fullsized":"2023-10-06T05:14:25.744210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736861+0000","last_change":"2023-10-06T05:14:25.730260+0000","last_active":"2023-10-06T05:14:25.736861+0000","last_peered":"2023-10-06T05:14:25.736861+0000","last_clean":"2023-10-06T05:14:25.736861+0000","last_became_active":"2023-10-06T05:14:25.729967+0000","last_became_peered":"2023-10-06T05:14:25.729967+0000","last_unstale":"2023-10-06T05:14:25.736861+0000","last_undegraded":"2023-10-06T05:14:25.736861+0000","last_fullsized":"2023-10-06T05:14:25.736861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.734367+0000","last_change":"2023-10-06T05:14:25.732052+0000","last_active":"2023-10-06T05:14:25.734367+0000","last_peered":"2023-10-06T05:14:25.734367+0000","last_clean":"2023-10-06T05:14:25.734367+0000","last_became_active":"2023-10-06T05:14:25.731746+0000","last_became_peered":"2023-10-06T05:14:25.731746+0000","last_unstale":"2023-10-06T05:14:25.734367+0000","last_undegraded":"2023-10-06T05:14:25.734367+0000","last_fullsized":"2023-10-06T05:14:25.734367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744812+0000","last_change":"2023-10-06T05:14:25.741800+0000","last_active":"2023-10-06T05:14:25.744812+0000","last_peered":"2023-10-06T05:14:25.744812+0000","last_clean":"2023-10-06T05:14:25.744812+0000","last_became_active":"2023-10-06T05:14:25.741167+0000","last_became_peered":"2023-10-06T05:14:25.741167+0000","last_unstale":"2023-10-06T05:14:25.744812+0000","last_undegraded":"2023-10-06T05:14:25.744812+0000","last_fullsized":"2023-10-06T05:14:25.744812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743990+0000","last_change":"2023-10-06T05:14:25.743115+0000","last_active":"2023-10-06T05:14:25.743990+0000","last_peered":"2023-10-06T05:14:25.743990+0000","last_clean":"2023-10-06T05:14:25.743990+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.743990+0000","last_undegraded":"2023-10-06T05:14:25.743990+0000","last_fullsized":"2023-10-06T05:14:25.743990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740187+0000","last_change":"2023-10-06T05:14:25.734027+0000","last_active":"2023-10-06T05:14:25.740187+0000","last_peered":"2023-10-06T05:14:25.740187+0000","last_clean":"2023-10-06T05:14:25.740187+0000","last_became_active":"2023-10-06T05:14:25.733743+0000","last_became_peered":"2023-10-06T05:14:25.733743+0000","last_unstale":"2023-10-06T05:14:25.740187+0000","last_undegraded":"2023-10-06T05:14:25.740187+0000","last_fullsized":"2023-10-06T05:14:25.740187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725222+0000","last_change":"2023-10-06T05:14:25.721669+0000","last_active":"2023-10-06T05:14:25.725222+0000","last_peered":"2023-10-06T05:14:25.725222+0000","last_clean":"2023-10-06T05:14:25.725222+0000","last_became_active":"2023-10-06T05:14:25.721412+0000","last_became_peered":"2023-10-06T05:14:25.721412+0000","last_unstale":"2023-10-06T05:14:25.725222+0000","last_undegraded":"2023-10-06T05:14:25.725222+0000","last_fullsized":"2023-10-06T05:14:25.725222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739202+0000","last_change":"2023-10-06T05:14:25.737798+0000","last_active":"2023-10-06T05:14:25.739202+0000","last_peered":"2023-10-06T05:14:25.739202+0000","last_clean":"2023-10-06T05:14:25.739202+0000","last_became_active":"2023-10-06T05:14:25.737342+0000","last_became_peered":"2023-10-06T05:14:25.737342+0000","last_unstale":"2023-10-06T05:14:25.739202+0000","last_undegraded":"2023-10-06T05:14:25.739202+0000","last_fullsized":"2023-10-06T05:14:25.739202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742612+0000","last_change":"2023-10-06T05:14:25.740804+0000","last_active":"2023-10-06T05:14:25.742612+0000","last_peered":"2023-10-06T05:14:25.742612+0000","last_clean":"2023-10-06T05:14:25.742612+0000","last_became_active":"2023-10-06T05:14:25.740330+0000","last_became_peered":"2023-10-06T05:14:25.740330+0000","last_unstale":"2023-10-06T05:14:25.742612+0000","last_undegraded":"2023-10-06T05:14:25.742612+0000","last_fullsized":"2023-10-06T05:14:25.742612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.955604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731300+0000","last_change":"2023-10-06T05:14:25.725672+0000","last_active":"2023-10-06T05:14:25.731300+0000","last_peered":"2023-10-06T05:14:25.731300+0000","last_clean":"2023-10-06T05:14:25.731300+0000","last_became_active":"2023-10-06T05:14:25.725392+0000","last_became_peered":"2023-10-06T05:14:25.725392+0000","last_unstale":"2023-10-06T05:14:25.731300+0000","last_undegraded":"2023-10-06T05:14:25.731300+0000","last_fullsized":"2023-10-06T05:14:25.731300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744646+0000","last_change":"2023-10-06T05:14:25.743525+0000","last_active":"2023-10-06T05:14:25.744646+0000","last_peered":"2023-10-06T05:14:25.744646+0000","last_clean":"2023-10-06T05:14:25.744646+0000","last_became_active":"2023-10-06T05:14:25.742820+0000","last_became_peered":"2023-10-06T05:14:25.742820+0000","last_unstale":"2023-10-06T05:14:25.744646+0000","last_undegraded":"2023-10-06T05:14:25.744646+0000","last_fullsized":"2023-10-06T05:14:25.744646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:42.341048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.720475+0000","last_change":"2023-10-06T05:14:25.718000+0000","last_active":"2023-10-06T05:14:25.720475+0000","last_peered":"2023-10-06T05:14:25.720475+0000","last_clean":"2023-10-06T05:14:25.720475+0000","last_became_active":"2023-10-06T05:14:25.717615+0000","last_became_peered":"2023-10-06T05:14:25.717615+0000","last_unstale":"2023-10-06T05:14:25.720475+0000","last_undegraded":"2023-10-06T05:14:25.720475+0000","last_fullsized":"2023-10-06T05:14:25.720475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:17.095036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743854+0000","last_change":"2023-10-06T05:14:25.742445+0000","last_active":"2023-10-06T05:14:25.743854+0000","last_peered":"2023-10-06T05:14:25.743854+0000","last_clean":"2023-10-06T05:14:25.743854+0000","last_became_active":"2023-10-06T05:14:25.741675+0000","last_became_peered":"2023-10-06T05:14:25.741675+0000","last_unstale":"2023-10-06T05:14:25.743854+0000","last_undegraded":"2023-10-06T05:14:25.743854+0000","last_fullsized":"2023-10-06T05:14:25.743854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735911+0000","last_change":"2023-10-06T05:14:25.733702+0000","last_active":"2023-10-06T05:14:25.735911+0000","last_peered":"2023-10-06T05:14:25.735911+0000","last_clean":"2023-10-06T05:14:25.735911+0000","last_became_active":"2023-10-06T05:14:25.733387+0000","last_became_peered":"2023-10-06T05:14:25.733387+0000","last_unstale":"2023-10-06T05:14:25.735911+0000","last_undegraded":"2023-10-06T05:14:25.735911+0000","last_fullsized":"2023-10-06T05:14:25.735911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731166+0000","last_change":"2023-10-06T05:14:25.724100+0000","last_active":"2023-10-06T05:14:25.731166+0000","last_peered":"2023-10-06T05:14:25.731166+0000","last_clean":"2023-10-06T05:14:25.731166+0000","last_became_active":"2023-10-06T05:14:25.723750+0000","last_became_peered":"2023-10-06T05:14:25.723750+0000","last_unstale":"2023-10-06T05:14:25.731166+0000","last_undegraded":"2023-10-06T05:14:25.731166+0000","last_fullsized":"2023-10-06T05:14:25.731166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735739+0000","last_change":"2023-10-06T05:14:25.727464+0000","last_active":"2023-10-06T05:14:25.735739+0000","last_peered":"2023-10-06T05:14:25.735739+0000","last_clean":"2023-10-06T05:14:25.735739+0000","last_became_active":"2023-10-06T05:14:25.727166+0000","last_became_peered":"2023-10-06T05:14:25.727166+0000","last_unstale":"2023-10-06T05:14:25.735739+0000","last_undegraded":"2023-10-06T05:14:25.735739+0000","last_fullsized":"2023-10-06T05:14:25.735739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732019+0000","last_change":"2023-10-06T05:14:25.730521+0000","last_active":"2023-10-06T05:14:25.732019+0000","last_peered":"2023-10-06T05:14:25.732019+0000","last_clean":"2023-10-06T05:14:25.732019+0000","last_became_active":"2023-10-06T05:14:25.730234+0000","last_became_peered":"2023-10-06T05:14:25.730234+0000","last_unstale":"2023-10-06T05:14:25.732019+0000","last_undegraded":"2023-10-06T05:14:25.732019+0000","last_fullsized":"2023-10-06T05:14:25.732019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735194+0000","last_change":"2023-10-06T05:14:25.733204+0000","last_active":"2023-10-06T05:14:25.735194+0000","last_peered":"2023-10-06T05:14:25.735194+0000","last_clean":"2023-10-06T05:14:25.735194+0000","last_became_active":"2023-10-06T05:14:25.732877+0000","last_became_peered":"2023-10-06T05:14:25.732877+0000","last_unstale":"2023-10-06T05:14:25.735194+0000","last_undegraded":"2023-10-06T05:14:25.735194+0000","last_fullsized":"2023-10-06T05:14:25.735194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735005+0000","last_change":"2023-10-06T05:14:25.724268+0000","last_active":"2023-10-06T05:14:25.735005+0000","last_peered":"2023-10-06T05:14:25.735005+0000","last_clean":"2023-10-06T05:14:25.735005+0000","last_became_active":"2023-10-06T05:14:25.724009+0000","last_became_peered":"2023-10-06T05:14:25.724009+0000","last_unstale":"2023-10-06T05:14:25.735005+0000","last_undegraded":"2023-10-06T05:14:25.735005+0000","last_fullsized":"2023-10-06T05:14:25.735005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727218+0000","last_change":"2023-10-06T05:14:25.725754+0000","last_active":"2023-10-06T05:14:25.727218+0000","last_peered":"2023-10-06T05:14:25.727218+0000","last_clean":"2023-10-06T05:14:25.727218+0000","last_became_active":"2023-10-06T05:14:25.725516+0000","last_became_peered":"2023-10-06T05:14:25.725516+0000","last_unstale":"2023-10-06T05:14:25.727218+0000","last_undegraded":"2023-10-06T05:14:25.727218+0000","last_fullsized":"2023-10-06T05:14:25.727218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741625+0000","last_change":"2023-10-06T05:14:24.734577+0000","last_active":"2023-10-06T05:14:25.741625+0000","last_peered":"2023-10-06T05:14:25.741625+0000","last_clean":"2023-10-06T05:14:25.741625+0000","last_became_active":"2023-10-06T05:14:24.732575+0000","last_became_peered":"2023-10-06T05:14:24.732575+0000","last_unstale":"2023-10-06T05:14:25.741625+0000","last_undegraded":"2023-10-06T05:14:25.741625+0000","last_fullsized":"2023-10-06T05:14:25.741625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726043+0000","last_change":"2023-10-06T05:14:25.721117+0000","last_active":"2023-10-06T05:14:25.726043+0000","last_peered":"2023-10-06T05:14:25.726043+0000","last_clean":"2023-10-06T05:14:25.726043+0000","last_became_active":"2023-10-06T05:14:25.720806+0000","last_became_peered":"2023-10-06T05:14:25.720806+0000","last_unstale":"2023-10-06T05:14:25.726043+0000","last_undegraded":"2023-10-06T05:14:25.726043+0000","last_fullsized":"2023-10-06T05:14:25.726043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.714069+0000","last_change":"2023-10-06T05:14:24.723717+0000","last_active":"2023-10-06T05:14:25.714069+0000","last_peered":"2023-10-06T05:14:25.714069+0000","last_clean":"2023-10-06T05:14:25.714069+0000","last_became_active":"2023-10-06T05:14:24.723440+0000","last_became_peered":"2023-10-06T05:14:24.723440+0000","last_unstale":"2023-10-06T05:14:25.714069+0000","last_undegraded":"2023-10-06T05:14:25.714069+0000","last_fullsized":"2023-10-06T05:14:25.714069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735391+0000","last_change":"2023-10-06T05:14:25.726258+0000","last_active":"2023-10-06T05:14:25.735391+0000","last_peered":"2023-10-06T05:14:25.735391+0000","last_clean":"2023-10-06T05:14:25.735391+0000","last_became_active":"2023-10-06T05:14:25.725989+0000","last_became_peered":"2023-10-06T05:14:25.725989+0000","last_unstale":"2023-10-06T05:14:25.735391+0000","last_undegraded":"2023-10-06T05:14:25.735391+0000","last_fullsized":"2023-10-06T05:14:25.735391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708532+0000","last_change":"2023-10-06T05:14:24.722594+0000","last_active":"2023-10-06T05:14:25.708532+0000","last_peered":"2023-10-06T05:14:25.708532+0000","last_clean":"2023-10-06T05:14:25.708532+0000","last_became_active":"2023-10-06T05:14:24.722302+0000","last_became_peered":"2023-10-06T05:14:24.722302+0000","last_unstale":"2023-10-06T05:14:25.708532+0000","last_undegraded":"2023-10-06T05:14:25.708532+0000","last_fullsized":"2023-10-06T05:14:25.708532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728738+0000","last_change":"2023-10-06T05:14:25.722475+0000","last_active":"2023-10-06T05:14:25.728738+0000","last_peered":"2023-10-06T05:14:25.728738+0000","last_clean":"2023-10-06T05:14:25.728738+0000","last_became_active":"2023-10-06T05:14:25.721707+0000","last_became_peered":"2023-10-06T05:14:25.721707+0000","last_unstale":"2023-10-06T05:14:25.728738+0000","last_undegraded":"2023-10-06T05:14:25.728738+0000","last_fullsized":"2023-10-06T05:14:25.728738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716173+0000","last_change":"2023-10-06T05:14:24.724581+0000","last_active":"2023-10-06T05:14:25.716173+0000","last_peered":"2023-10-06T05:14:25.716173+0000","last_clean":"2023-10-06T05:14:25.716173+0000","last_became_active":"2023-10-06T05:14:24.724293+0000","last_became_peered":"2023-10-06T05:14:24.724293+0000","last_unstale":"2023-10-06T05:14:25.716173+0000","last_undegraded":"2023-10-06T05:14:25.716173+0000","last_fullsized":"2023-10-06T05:14:25.716173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723275+0000","last_change":"2023-10-06T05:14:25.719638+0000","last_active":"2023-10-06T05:14:25.723275+0000","last_peered":"2023-10-06T05:14:25.723275+0000","last_clean":"2023-10-06T05:14:25.723275+0000","last_became_active":"2023-10-06T05:14:25.719347+0000","last_became_peered":"2023-10-06T05:14:25.719347+0000","last_unstale":"2023-10-06T05:14:25.723275+0000","last_undegraded":"2023-10-06T05:14:25.723275+0000","last_fullsized":"2023-10-06T05:14:25.723275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716142+0000","last_change":"2023-10-06T05:14:24.722842+0000","last_active":"2023-10-06T05:14:25.716142+0000","last_peered":"2023-10-06T05:14:25.716142+0000","last_clean":"2023-10-06T05:14:25.716142+0000","last_became_active":"2023-10-06T05:14:24.721323+0000","last_became_peered":"2023-10-06T05:14:24.721323+0000","last_unstale":"2023-10-06T05:14:25.716142+0000","last_undegraded":"2023-10-06T05:14:25.716142+0000","last_fullsized":"2023-10-06T05:14:25.716142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735759+0000","last_change":"2023-10-06T05:14:25.728303+0000","last_active":"2023-10-06T05:14:25.735759+0000","last_peered":"2023-10-06T05:14:25.735759+0000","last_clean":"2023-10-06T05:14:25.735759+0000","last_became_active":"2023-10-06T05:14:25.727775+0000","last_became_peered":"2023-10-06T05:14:25.727775+0000","last_unstale":"2023-10-06T05:14:25.735759+0000","last_undegraded":"2023-10-06T05:14:25.735759+0000","last_fullsized":"2023-10-06T05:14:25.735759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738830+0000","last_change":"2023-10-06T05:14:24.728587+0000","last_active":"2023-10-06T05:14:25.738830+0000","last_peered":"2023-10-06T05:14:25.738830+0000","last_clean":"2023-10-06T05:14:25.738830+0000","last_became_active":"2023-10-06T05:14:24.728292+0000","last_became_peered":"2023-10-06T05:14:24.728292+0000","last_unstale":"2023-10-06T05:14:25.738830+0000","last_undegraded":"2023-10-06T05:14:25.738830+0000","last_fullsized":"2023-10-06T05:14:25.738830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:11.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735121+0000","last_change":"2023-10-06T05:14:25.732130+0000","last_active":"2023-10-06T05:14:25.735121+0000","last_peered":"2023-10-06T05:14:25.735121+0000","last_clean":"2023-10-06T05:14:25.735121+0000","last_became_active":"2023-10-06T05:14:25.731459+0000","last_became_peered":"2023-10-06T05:14:25.731459+0000","last_unstale":"2023-10-06T05:14:25.735121+0000","last_undegraded":"2023-10-06T05:14:25.735121+0000","last_fullsized":"2023-10-06T05:14:25.735121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.748549+0000","last_change":"2023-10-06T05:14:24.728538+0000","last_active":"2023-10-06T05:14:25.748549+0000","last_peered":"2023-10-06T05:14:25.748549+0000","last_clean":"2023-10-06T05:14:25.748549+0000","last_became_active":"2023-10-06T05:14:24.728200+0000","last_became_peered":"2023-10-06T05:14:24.728200+0000","last_unstale":"2023-10-06T05:14:25.748549+0000","last_undegraded":"2023-10-06T05:14:25.748549+0000","last_fullsized":"2023-10-06T05:14:25.748549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741887+0000","last_change":"2023-10-06T05:14:25.740582+0000","last_active":"2023-10-06T05:14:25.741887+0000","last_peered":"2023-10-06T05:14:25.741887+0000","last_clean":"2023-10-06T05:14:25.741887+0000","last_became_active":"2023-10-06T05:14:25.740298+0000","last_became_peered":"2023-10-06T05:14:25.740298+0000","last_unstale":"2023-10-06T05:14:25.741887+0000","last_undegraded":"2023-10-06T05:14:25.741887+0000","last_fullsized":"2023-10-06T05:14:25.741887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:31.489344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742441+0000","last_change":"2023-10-06T05:14:24.735437+0000","last_active":"2023-10-06T05:14:25.742441+0000","last_peered":"2023-10-06T05:14:25.742441+0000","last_clean":"2023-10-06T05:14:25.742441+0000","last_became_active":"2023-10-06T05:14:24.735168+0000","last_became_peered":"2023-10-06T05:14:24.735168+0000","last_unstale":"2023-10-06T05:14:25.742441+0000","last_undegraded":"2023-10-06T05:14:25.742441+0000","last_fullsized":"2023-10-06T05:14:25.742441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:06.220991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723244+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.723244+0000","last_peered":"2023-10-06T05:14:25.723244+0000","last_clean":"2023-10-06T05:14:25.723244+0000","last_became_active":"2023-10-06T05:14:25.719474+0000","last_became_peered":"2023-10-06T05:14:25.719474+0000","last_unstale":"2023-10-06T05:14:25.723244+0000","last_undegraded":"2023-10-06T05:14:25.723244+0000","last_fullsized":"2023-10-06T05:14:25.723244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.712742+0000","last_change":"2023-10-06T05:14:24.718646+0000","last_active":"2023-10-06T05:14:25.712742+0000","last_peered":"2023-10-06T05:14:25.712742+0000","last_clean":"2023-10-06T05:14:25.712742+0000","last_became_active":"2023-10-06T05:14:24.718324+0000","last_became_peered":"2023-10-06T05:14:24.718324+0000","last_unstale":"2023-10-06T05:14:25.712742+0000","last_undegraded":"2023-10-06T05:14:25.712742+0000","last_fullsized":"2023-10-06T05:14:25.712742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727199+0000","last_change":"2023-10-06T05:14:25.724883+0000","last_active":"2023-10-06T05:14:25.727199+0000","last_peered":"2023-10-06T05:14:25.727199+0000","last_clean":"2023-10-06T05:14:25.727199+0000","last_became_active":"2023-10-06T05:14:25.724633+0000","last_became_peered":"2023-10-06T05:14:25.724633+0000","last_unstale":"2023-10-06T05:14:25.727199+0000","last_undegraded":"2023-10-06T05:14:25.727199+0000","last_fullsized":"2023-10-06T05:14:25.727199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.720059+0000","last_change":"2023-10-06T05:14:24.736333+0000","last_active":"2023-10-06T05:14:25.720059+0000","last_peered":"2023-10-06T05:14:25.720059+0000","last_clean":"2023-10-06T05:14:25.720059+0000","last_became_active":"2023-10-06T05:14:24.736038+0000","last_became_peered":"2023-10-06T05:14:24.736038+0000","last_unstale":"2023-10-06T05:14:25.720059+0000","last_undegraded":"2023-10-06T05:14:25.720059+0000","last_fullsized":"2023-10-06T05:14:25.720059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735371+0000","last_change":"2023-10-06T05:14:25.725666+0000","last_active":"2023-10-06T05:14:25.735371+0000","last_peered":"2023-10-06T05:14:25.735371+0000","last_clean":"2023-10-06T05:14:25.735371+0000","last_became_active":"2023-10-06T05:14:25.725319+0000","last_became_peered":"2023-10-06T05:14:25.725319+0000","last_unstale":"2023-10-06T05:14:25.735371+0000","last_undegraded":"2023-10-06T05:14:25.735371+0000","last_fullsized":"2023-10-06T05:14:25.735371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733857+0000","last_change":"2023-10-06T05:14:24.721548+0000","last_active":"2023-10-06T05:14:25.733857+0000","last_peered":"2023-10-06T05:14:25.733857+0000","last_clean":"2023-10-06T05:14:25.733857+0000","last_became_active":"2023-10-06T05:14:24.721266+0000","last_became_peered":"2023-10-06T05:14:24.721266+0000","last_unstale":"2023-10-06T05:14:25.733857+0000","last_undegraded":"2023-10-06T05:14:25.733857+0000","last_fullsized":"2023-10-06T05:14:25.733857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727201+0000","last_change":"2023-10-06T05:14:25.725874+0000","last_active":"2023-10-06T05:14:25.727201+0000","last_peered":"2023-10-06T05:14:25.727201+0000","last_clean":"2023-10-06T05:14:25.727201+0000","last_became_active":"2023-10-06T05:14:25.725494+0000","last_became_peered":"2023-10-06T05:14:25.725494+0000","last_unstale":"2023-10-06T05:14:25.727201+0000","last_undegraded":"2023-10-06T05:14:25.727201+0000","last_fullsized":"2023-10-06T05:14:25.727201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742053+0000","last_change":"2023-10-06T05:14:24.733795+0000","last_active":"2023-10-06T05:14:25.742053+0000","last_peered":"2023-10-06T05:14:25.742053+0000","last_clean":"2023-10-06T05:14:25.742053+0000","last_became_active":"2023-10-06T05:14:24.733567+0000","last_became_peered":"2023-10-06T05:14:24.733567+0000","last_unstale":"2023-10-06T05:14:25.742053+0000","last_undegraded":"2023-10-06T05:14:25.742053+0000","last_fullsized":"2023-10-06T05:14:25.742053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732255+0000","last_change":"2023-10-06T05:14:25.727738+0000","last_active":"2023-10-06T05:14:25.732255+0000","last_peered":"2023-10-06T05:14:25.732255+0000","last_clean":"2023-10-06T05:14:25.732255+0000","last_became_active":"2023-10-06T05:14:25.727474+0000","last_became_peered":"2023-10-06T05:14:25.727474+0000","last_unstale":"2023-10-06T05:14:25.732255+0000","last_undegraded":"2023-10-06T05:14:25.732255+0000","last_fullsized":"2023-10-06T05:14:25.732255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.745629+0000","last_change":"2023-10-06T05:14:24.733458+0000","last_active":"2023-10-06T05:14:25.745629+0000","last_peered":"2023-10-06T05:14:25.745629+0000","last_clean":"2023-10-06T05:14:25.745629+0000","last_became_active":"2023-10-06T05:14:24.733160+0000","last_became_peered":"2023-10-06T05:14:24.733160+0000","last_unstale":"2023-10-06T05:14:25.745629+0000","last_undegraded":"2023-10-06T05:14:25.745629+0000","last_fullsized":"2023-10-06T05:14:25.745629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744280+0000","last_change":"2023-10-06T05:14:25.741624+0000","last_active":"2023-10-06T05:14:25.744280+0000","last_peered":"2023-10-06T05:14:25.744280+0000","last_clean":"2023-10-06T05:14:25.744280+0000","last_became_active":"2023-10-06T05:14:25.740252+0000","last_became_peered":"2023-10-06T05:14:25.740252+0000","last_unstale":"2023-10-06T05:14:25.744280+0000","last_undegraded":"2023-10-06T05:14:25.744280+0000","last_fullsized":"2023-10-06T05:14:25.744280+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732612+0000","last_change":"2023-10-06T05:14:24.721437+0000","last_active":"2023-10-06T05:14:25.732612+0000","last_peered":"2023-10-06T05:14:25.732612+0000","last_clean":"2023-10-06T05:14:25.732612+0000","last_became_active":"2023-10-06T05:14:24.721172+0000","last_became_peered":"2023-10-06T05:14:24.721172+0000","last_unstale":"2023-10-06T05:14:25.732612+0000","last_undegraded":"2023-10-06T05:14:25.732612+0000","last_fullsized":"2023-10-06T05:14:25.732612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:22.852582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727675+0000","last_change":"2023-10-06T05:14:25.719600+0000","last_active":"2023-10-06T05:14:25.727675+0000","last_peered":"2023-10-06T05:14:25.727675+0000","last_clean":"2023-10-06T05:14:25.727675+0000","last_became_active":"2023-10-06T05:14:25.719349+0000","last_became_peered":"2023-10-06T05:14:25.719349+0000","last_unstale":"2023-10-06T05:14:25.727675+0000","last_undegraded":"2023-10-06T05:14:25.727675+0000","last_fullsized":"2023-10-06T05:14:25.727675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716852+0000","last_change":"2023-10-06T05:14:24.728714+0000","last_active":"2023-10-06T05:14:25.716852+0000","last_peered":"2023-10-06T05:14:25.716852+0000","last_clean":"2023-10-06T05:14:25.716852+0000","last_became_active":"2023-10-06T05:14:24.728435+0000","last_became_peered":"2023-10-06T05:14:24.728435+0000","last_unstale":"2023-10-06T05:14:25.716852+0000","last_undegraded":"2023-10-06T05:14:25.716852+0000","last_fullsized":"2023-10-06T05:14:25.716852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731329+0000","last_change":"2023-10-06T05:14:25.724993+0000","last_active":"2023-10-06T05:14:25.731329+0000","last_peered":"2023-10-06T05:14:25.731329+0000","last_clean":"2023-10-06T05:14:25.731329+0000","last_became_active":"2023-10-06T05:14:25.724538+0000","last_became_peered":"2023-10-06T05:14:25.724538+0000","last_unstale":"2023-10-06T05:14:25.731329+0000","last_undegraded":"2023-10-06T05:14:25.731329+0000","last_fullsized":"2023-10-06T05:14:25.731329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738370+0000","last_change":"2023-10-06T05:14:24.723321+0000","last_active":"2023-10-06T05:14:25.738370+0000","last_peered":"2023-10-06T05:14:25.738370+0000","last_clean":"2023-10-06T05:14:25.738370+0000","last_became_active":"2023-10-06T05:14:24.723063+0000","last_became_peered":"2023-10-06T05:14:24.723063+0000","last_unstale":"2023-10-06T05:14:25.738370+0000","last_undegraded":"2023-10-06T05:14:25.738370+0000","last_fullsized":"2023-10-06T05:14:25.738370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742001+0000","last_change":"2023-10-06T05:14:25.740648+0000","last_active":"2023-10-06T05:14:25.742001+0000","last_peered":"2023-10-06T05:14:25.742001+0000","last_clean":"2023-10-06T05:14:25.742001+0000","last_became_active":"2023-10-06T05:14:25.740351+0000","last_became_peered":"2023-10-06T05:14:25.740351+0000","last_unstale":"2023-10-06T05:14:25.742001+0000","last_undegraded":"2023-10-06T05:14:25.742001+0000","last_fullsized":"2023-10-06T05:14:25.742001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709955+0000","last_change":"2023-10-06T05:14:24.730263+0000","last_active":"2023-10-06T05:14:25.709955+0000","last_peered":"2023-10-06T05:14:25.709955+0000","last_clean":"2023-10-06T05:14:25.709955+0000","last_became_active":"2023-10-06T05:14:24.729911+0000","last_became_peered":"2023-10-06T05:14:24.729911+0000","last_unstale":"2023-10-06T05:14:25.709955+0000","last_undegraded":"2023-10-06T05:14:25.709955+0000","last_fullsized":"2023-10-06T05:14:25.709955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747672+0000","last_change":"2023-10-06T05:14:25.744620+0000","last_active":"2023-10-06T05:14:25.747672+0000","last_peered":"2023-10-06T05:14:25.747672+0000","last_clean":"2023-10-06T05:14:25.747672+0000","last_became_active":"2023-10-06T05:14:25.744323+0000","last_became_peered":"2023-10-06T05:14:25.744323+0000","last_unstale":"2023-10-06T05:14:25.747672+0000","last_undegraded":"2023-10-06T05:14:25.747672+0000","last_fullsized":"2023-10-06T05:14:25.747672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:39.850675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730362+0000","last_change":"2023-10-06T05:14:24.731862+0000","last_active":"2023-10-06T05:14:25.730362+0000","last_peered":"2023-10-06T05:14:25.730362+0000","last_clean":"2023-10-06T05:14:25.730362+0000","last_became_active":"2023-10-06T05:14:24.729629+0000","last_became_peered":"2023-10-06T05:14:24.729629+0000","last_unstale":"2023-10-06T05:14:25.730362+0000","last_undegraded":"2023-10-06T05:14:25.730362+0000","last_fullsized":"2023-10-06T05:14:25.730362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744587+0000","last_change":"2023-10-06T05:14:25.742674+0000","last_active":"2023-10-06T05:14:25.744587+0000","last_peered":"2023-10-06T05:14:25.744587+0000","last_clean":"2023-10-06T05:14:25.744587+0000","last_became_active":"2023-10-06T05:14:25.741308+0000","last_became_peered":"2023-10-06T05:14:25.741308+0000","last_unstale":"2023-10-06T05:14:25.744587+0000","last_undegraded":"2023-10-06T05:14:25.744587+0000","last_fullsized":"2023-10-06T05:14:25.744587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:29.811666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708350+0000","last_change":"2023-10-06T05:14:24.724203+0000","last_active":"2023-10-06T05:14:25.708350+0000","last_peered":"2023-10-06T05:14:25.708350+0000","last_clean":"2023-10-06T05:14:25.708350+0000","last_became_active":"2023-10-06T05:14:24.723929+0000","last_became_peered":"2023-10-06T05:14:24.723929+0000","last_unstale":"2023-10-06T05:14:25.708350+0000","last_undegraded":"2023-10-06T05:14:25.708350+0000","last_fullsized":"2023-10-06T05:14:25.708350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735025+0000","last_change":"2023-10-06T05:14:25.726645+0000","last_active":"2023-10-06T05:14:25.735025+0000","last_peered":"2023-10-06T05:14:25.735025+0000","last_clean":"2023-10-06T05:14:25.735025+0000","last_became_active":"2023-10-06T05:14:25.724695+0000","last_became_peered":"2023-10-06T05:14:25.724695+0000","last_unstale":"2023-10-06T05:14:25.735025+0000","last_undegraded":"2023-10-06T05:14:25.735025+0000","last_fullsized":"2023-10-06T05:14:25.735025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713550+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:14:25.713550+0000","last_peered":"2023-10-06T05:14:25.713550+0000","last_clean":"2023-10-06T05:14:25.713550+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:14:25.713550+0000","last_undegraded":"2023-10-06T05:14:25.713550+0000","last_fullsized":"2023-10-06T05:14:25.713550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716529+0000","last_change":"2023-10-06T05:14:24.728757+0000","last_active":"2023-10-06T05:14:25.716529+0000","last_peered":"2023-10-06T05:14:25.716529+0000","last_clean":"2023-10-06T05:14:25.716529+0000","last_became_active":"2023-10-06T05:14:24.728480+0000","last_became_peered":"2023-10-06T05:14:24.728480+0000","last_unstale":"2023-10-06T05:14:25.716529+0000","last_undegraded":"2023-10-06T05:14:25.716529+0000","last_fullsized":"2023-10-06T05:14:25.716529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727990+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.727990+0000","last_peered":"2023-10-06T05:14:25.727990+0000","last_clean":"2023-10-06T05:14:25.727990+0000","last_became_active":"2023-10-06T05:14:25.719491+0000","last_became_peered":"2023-10-06T05:14:25.719491+0000","last_unstale":"2023-10-06T05:14:25.727990+0000","last_undegraded":"2023-10-06T05:14:25.727990+0000","last_fullsized":"2023-10-06T05:14:25.727990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715616+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:14:25.715616+0000","last_peered":"2023-10-06T05:14:25.715616+0000","last_clean":"2023-10-06T05:14:25.715616+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:14:25.715616+0000","last_undegraded":"2023-10-06T05:14:25.715616+0000","last_fullsized":"2023-10-06T05:14:25.715616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731289+0000","last_change":"2023-10-06T05:14:24.732737+0000","last_active":"2023-10-06T05:14:25.731289+0000","last_peered":"2023-10-06T05:14:25.731289+0000","last_clean":"2023-10-06T05:14:25.731289+0000","last_became_active":"2023-10-06T05:14:24.732410+0000","last_became_peered":"2023-10-06T05:14:24.732410+0000","last_unstale":"2023-10-06T05:14:25.731289+0000","last_undegraded":"2023-10-06T05:14:25.731289+0000","last_fullsized":"2023-10-06T05:14:25.731289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736090+0000","last_change":"2023-10-06T05:14:25.723180+0000","last_active":"2023-10-06T05:14:25.736090+0000","last_peered":"2023-10-06T05:14:25.736090+0000","last_clean":"2023-10-06T05:14:25.736090+0000","last_became_active":"2023-10-06T05:14:25.722867+0000","last_became_peered":"2023-10-06T05:14:25.722867+0000","last_unstale":"2023-10-06T05:14:25.736090+0000","last_undegraded":"2023-10-06T05:14:25.736090+0000","last_fullsized":"2023-10-06T05:14:25.736090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.705335+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:14:25.705335+0000","last_peered":"2023-10-06T05:14:25.705335+0000","last_clean":"2023-10-06T05:14:25.705335+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:14:25.705335+0000","last_undegraded":"2023-10-06T05:14:25.705335+0000","last_fullsized":"2023-10-06T05:14:25.705335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713983+0000","last_change":"2023-10-06T05:14:24.719784+0000","last_active":"2023-10-06T05:14:25.713983+0000","last_peered":"2023-10-06T05:14:25.713983+0000","last_clean":"2023-10-06T05:14:25.713983+0000","last_became_active":"2023-10-06T05:14:24.719414+0000","last_became_peered":"2023-10-06T05:14:24.719414+0000","last_unstale":"2023-10-06T05:14:25.713983+0000","last_undegraded":"2023-10-06T05:14:25.713983+0000","last_fullsized":"2023-10-06T05:14:25.713983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735002+0000","last_change":"2023-10-06T05:14:25.725673+0000","last_active":"2023-10-06T05:14:25.735002+0000","last_peered":"2023-10-06T05:14:25.735002+0000","last_clean":"2023-10-06T05:14:25.735002+0000","last_became_active":"2023-10-06T05:14:25.725107+0000","last_became_peered":"2023-10-06T05:14:25.725107+0000","last_unstale":"2023-10-06T05:14:25.735002+0000","last_undegraded":"2023-10-06T05:14:25.735002+0000","last_fullsized":"2023-10-06T05:14:25.735002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:35.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715779+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:14:25.715779+0000","last_peered":"2023-10-06T05:14:25.715779+0000","last_clean":"2023-10-06T05:14:25.715779+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:14:25.715779+0000","last_undegraded":"2023-10-06T05:14:25.715779+0000","last_fullsized":"2023-10-06T05:14:25.715779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719147+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:14:25.719147+0000","last_peered":"2023-10-06T05:14:25.719147+0000","last_clean":"2023-10-06T05:14:25.719147+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:14:25.719147+0000","last_undegraded":"2023-10-06T05:14:25.719147+0000","last_fullsized":"2023-10-06T05:14:25.719147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708071+0000","last_change":"2023-10-06T05:14:24.717406+0000","last_active":"2023-10-06T05:14:25.708071+0000","last_peered":"2023-10-06T05:14:25.708071+0000","last_clean":"2023-10-06T05:14:25.708071+0000","last_became_active":"2023-10-06T05:14:24.716723+0000","last_became_peered":"2023-10-06T05:14:24.716723+0000","last_unstale":"2023-10-06T05:14:25.708071+0000","last_undegraded":"2023-10-06T05:14:25.708071+0000","last_fullsized":"2023-10-06T05:14:25.708071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.746536+0000","last_change":"2023-10-06T05:14:25.745372+0000","last_active":"2023-10-06T05:14:25.746536+0000","last_peered":"2023-10-06T05:14:25.746536+0000","last_clean":"2023-10-06T05:14:25.746536+0000","last_became_active":"2023-10-06T05:14:25.745163+0000","last_became_peered":"2023-10-06T05:14:25.745163+0000","last_unstale":"2023-10-06T05:14:25.746536+0000","last_undegraded":"2023-10-06T05:14:25.746536+0000","last_fullsized":"2023-10-06T05:14:25.746536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.860941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719614+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:14:25.719614+0000","last_peered":"2023-10-06T05:14:25.719614+0000","last_clean":"2023-10-06T05:14:25.719614+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:14:25.719614+0000","last_undegraded":"2023-10-06T05:14:25.719614+0000","last_fullsized":"2023-10-06T05:14:25.719614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.712654+0000","last_change":"2023-10-06T05:14:24.736663+0000","last_active":"2023-10-06T05:14:25.712654+0000","last_peered":"2023-10-06T05:14:25.712654+0000","last_clean":"2023-10-06T05:14:25.712654+0000","last_became_active":"2023-10-06T05:14:24.736241+0000","last_became_peered":"2023-10-06T05:14:24.736241+0000","last_unstale":"2023-10-06T05:14:25.712654+0000","last_undegraded":"2023-10-06T05:14:25.712654+0000","last_fullsized":"2023-10-06T05:14:25.712654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736890+0000","last_change":"2023-10-06T05:14:25.729949+0000","last_active":"2023-10-06T05:14:25.736890+0000","last_peered":"2023-10-06T05:14:25.736890+0000","last_clean":"2023-10-06T05:14:25.736890+0000","last_became_active":"2023-10-06T05:14:25.729600+0000","last_became_peered":"2023-10-06T05:14:25.729600+0000","last_unstale":"2023-10-06T05:14:25.736890+0000","last_undegraded":"2023-10-06T05:14:25.736890+0000","last_fullsized":"2023-10-06T05:14:25.736890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723259+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:14:25.723259+0000","last_peered":"2023-10-06T05:14:25.723259+0000","last_clean":"2023-10-06T05:14:25.723259+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:14:25.723259+0000","last_undegraded":"2023-10-06T05:14:25.723259+0000","last_fullsized":"2023-10-06T05:14:25.723259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713950+0000","last_change":"2023-10-06T05:14:24.724312+0000","last_active":"2023-10-06T05:14:25.713950+0000","last_peered":"2023-10-06T05:14:25.713950+0000","last_clean":"2023-10-06T05:14:25.713950+0000","last_became_active":"2023-10-06T05:14:24.724011+0000","last_became_peered":"2023-10-06T05:14:24.724011+0000","last_unstale":"2023-10-06T05:14:25.713950+0000","last_undegraded":"2023-10-06T05:14:25.713950+0000","last_fullsized":"2023-10-06T05:14:25.713950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735842+0000","last_change":"2023-10-06T05:14:25.729128+0000","last_active":"2023-10-06T05:14:25.735842+0000","last_peered":"2023-10-06T05:14:25.735842+0000","last_clean":"2023-10-06T05:14:25.735842+0000","last_became_active":"2023-10-06T05:14:25.728715+0000","last_became_peered":"2023-10-06T05:14:25.728715+0000","last_unstale":"2023-10-06T05:14:25.735842+0000","last_undegraded":"2023-10-06T05:14:25.735842+0000","last_fullsized":"2023-10-06T05:14:25.735842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727798+0000","last_change":"2023-10-06T05:14:24.721869+0000","last_active":"2023-10-06T05:14:25.727798+0000","last_peered":"2023-10-06T05:14:25.727798+0000","last_clean":"2023-10-06T05:14:25.727798+0000","last_became_active":"2023-10-06T05:14:24.719902+0000","last_became_peered":"2023-10-06T05:14:24.719902+0000","last_unstale":"2023-10-06T05:14:25.727798+0000","last_undegraded":"2023-10-06T05:14:25.727798+0000","last_fullsized":"2023-10-06T05:14:25.727798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713213+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:14:25.713213+0000","last_peered":"2023-10-06T05:14:25.713213+0000","last_clean":"2023-10-06T05:14:25.713213+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:14:25.713213+0000","last_undegraded":"2023-10-06T05:14:25.713213+0000","last_fullsized":"2023-10-06T05:14:25.713213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735685+0000","last_change":"2023-10-06T05:14:25.727038+0000","last_active":"2023-10-06T05:14:25.735685+0000","last_peered":"2023-10-06T05:14:25.735685+0000","last_clean":"2023-10-06T05:14:25.735685+0000","last_became_active":"2023-10-06T05:14:25.726717+0000","last_became_peered":"2023-10-06T05:14:25.726717+0000","last_unstale":"2023-10-06T05:14:25.735685+0000","last_undegraded":"2023-10-06T05:14:25.735685+0000","last_fullsized":"2023-10-06T05:14:25.735685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709571+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:14:25.709571+0000","last_peered":"2023-10-06T05:14:25.709571+0000","last_clean":"2023-10-06T05:14:25.709571+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:14:25.709571+0000","last_undegraded":"2023-10-06T05:14:25.709571+0000","last_fullsized":"2023-10-06T05:14:25.709571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:14:26.957001+0000","last_change":"2023-10-06T05:14:24.717582+0000","last_active":"2023-10-06T05:14:26.957001+0000","last_peered":"2023-10-06T05:14:26.957001+0000","last_clean":"2023-10-06T05:14:26.957001+0000","last_became_active":"2023-10-06T05:14:24.716871+0000","last_became_peered":"2023-10-06T05:14:24.716871+0000","last_unstale":"2023-10-06T05:14:26.957001+0000","last_undegraded":"2023-10-06T05:14:26.957001+0000","last_fullsized":"2023-10-06T05:14:26.957001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744258+0000","last_change":"2023-10-06T05:14:25.737874+0000","last_active":"2023-10-06T05:14:25.744258+0000","last_peered":"2023-10-06T05:14:25.744258+0000","last_clean":"2023-10-06T05:14:25.744258+0000","last_became_active":"2023-10-06T05:14:25.737569+0000","last_became_peered":"2023-10-06T05:14:25.737569+0000","last_unstale":"2023-10-06T05:14:25.744258+0000","last_undegraded":"2023-10-06T05:14:25.744258+0000","last_fullsized":"2023-10-06T05:14:25.744258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741554+0000","last_change":"2023-10-06T05:14:24.736575+0000","last_active":"2023-10-06T05:14:25.741554+0000","last_peered":"2023-10-06T05:14:25.741554+0000","last_clean":"2023-10-06T05:14:25.741554+0000","last_became_active":"2023-10-06T05:14:24.736221+0000","last_became_peered":"2023-10-06T05:14:24.736221+0000","last_unstale":"2023-10-06T05:14:25.741554+0000","last_undegraded":"2023-10-06T05:14:25.741554+0000","last_fullsized":"2023-10-06T05:14:25.741554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.724256+0000","last_change":"2023-10-06T05:14:25.719862+0000","last_active":"2023-10-06T05:14:25.724256+0000","last_peered":"2023-10-06T05:14:25.724256+0000","last_clean":"2023-10-06T05:14:25.724256+0000","last_became_active":"2023-10-06T05:14:25.719558+0000","last_became_peered":"2023-10-06T05:14:25.719558+0000","last_unstale":"2023-10-06T05:14:25.724256+0000","last_undegraded":"2023-10-06T05:14:25.724256+0000","last_fullsized":"2023-10-06T05:14:25.724256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:41.453897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742798+0000","last_change":"2023-10-06T05:14:24.734397+0000","last_active":"2023-10-06T05:14:25.742798+0000","last_peered":"2023-10-06T05:14:25.742798+0000","last_clean":"2023-10-06T05:14:25.742798+0000","last_became_active":"2023-10-06T05:14:24.734112+0000","last_became_peered":"2023-10-06T05:14:24.734112+0000","last_unstale":"2023-10-06T05:14:25.742798+0000","last_undegraded":"2023-10-06T05:14:25.742798+0000","last_fullsized":"2023-10-06T05:14:25.742798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:50.734968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741866+0000","last_change":"2023-10-06T05:14:25.736354+0000","last_active":"2023-10-06T05:14:25.741866+0000","last_peered":"2023-10-06T05:14:25.741866+0000","last_clean":"2023-10-06T05:14:25.741866+0000","last_became_active":"2023-10-06T05:14:25.735695+0000","last_became_peered":"2023-10-06T05:14:25.735695+0000","last_unstale":"2023-10-06T05:14:25.741866+0000","last_undegraded":"2023-10-06T05:14:25.741866+0000","last_fullsized":"2023-10-06T05:14:25.741866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738237+0000","last_change":"2023-10-06T05:14:24.733470+0000","last_active":"2023-10-06T05:14:25.738237+0000","last_peered":"2023-10-06T05:14:25.738237+0000","last_clean":"2023-10-06T05:14:25.738237+0000","last_became_active":"2023-10-06T05:14:24.733189+0000","last_became_peered":"2023-10-06T05:14:24.733189+0000","last_unstale":"2023-10-06T05:14:25.738237+0000","last_undegraded":"2023-10-06T05:14:25.738237+0000","last_fullsized":"2023-10-06T05:14:25.738237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747308+0000","last_change":"2023-10-06T05:14:25.744257+0000","last_active":"2023-10-06T05:14:25.747308+0000","last_peered":"2023-10-06T05:14:25.747308+0000","last_clean":"2023-10-06T05:14:25.747308+0000","last_became_active":"2023-10-06T05:14:25.743800+0000","last_became_peered":"2023-10-06T05:14:25.743800+0000","last_unstale":"2023-10-06T05:14:25.747308+0000","last_undegraded":"2023-10-06T05:14:25.747308+0000","last_fullsized":"2023-10-06T05:14:25.747308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737149+0000","last_change":"2023-10-06T05:14:24.719523+0000","last_active":"2023-10-06T05:14:25.737149+0000","last_peered":"2023-10-06T05:14:25.737149+0000","last_clean":"2023-10-06T05:14:25.737149+0000","last_became_active":"2023-10-06T05:14:24.717215+0000","last_became_peered":"2023-10-06T05:14:24.717215+0000","last_unstale":"2023-10-06T05:14:25.737149+0000","last_undegraded":"2023-10-06T05:14:25.737149+0000","last_fullsized":"2023-10-06T05:14:25.737149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733084+0000","last_change":"2023-10-06T05:14:25.726409+0000","last_active":"2023-10-06T05:14:25.733084+0000","last_peered":"2023-10-06T05:14:25.733084+0000","last_clean":"2023-10-06T05:14:25.733084+0000","last_became_active":"2023-10-06T05:14:25.725760+0000","last_became_peered":"2023-10-06T05:14:25.725760+0000","last_unstale":"2023-10-06T05:14:25.733084+0000","last_undegraded":"2023-10-06T05:14:25.733084+0000","last_fullsized":"2023-10-06T05:14:25.733084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730428+0000","last_change":"2023-10-06T05:14:24.724643+0000","last_active":"2023-10-06T05:14:25.730428+0000","last_peered":"2023-10-06T05:14:25.730428+0000","last_clean":"2023-10-06T05:14:25.730428+0000","last_became_active":"2023-10-06T05:14:24.724364+0000","last_became_peered":"2023-10-06T05:14:24.724364+0000","last_unstale":"2023-10-06T05:14:25.730428+0000","last_undegraded":"2023-10-06T05:14:25.730428+0000","last_fullsized":"2023-10-06T05:14:25.730428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742656+0000","last_change":"2023-10-06T05:14:25.741519+0000","last_active":"2023-10-06T05:14:25.742656+0000","last_peered":"2023-10-06T05:14:25.742656+0000","last_clean":"2023-10-06T05:14:25.742656+0000","last_became_active":"2023-10-06T05:14:25.741100+0000","last_became_peered":"2023-10-06T05:14:25.741100+0000","last_unstale":"2023-10-06T05:14:25.742656+0000","last_undegraded":"2023-10-06T05:14:25.742656+0000","last_fullsized":"2023-10-06T05:14:25.742656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728144+0000","last_change":"2023-10-06T05:14:25.721386+0000","last_active":"2023-10-06T05:14:25.728144+0000","last_peered":"2023-10-06T05:14:25.728144+0000","last_clean":"2023-10-06T05:14:25.728144+0000","last_became_active":"2023-10-06T05:14:25.720975+0000","last_became_peered":"2023-10-06T05:14:25.720975+0000","last_unstale":"2023-10-06T05:14:25.728144+0000","last_undegraded":"2023-10-06T05:14:25.728144+0000","last_fullsized":"2023-10-06T05:14:25.728144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:51.743327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733104+0000","last_change":"2023-10-06T05:14:25.726604+0000","last_active":"2023-10-06T05:14:25.733104+0000","last_peered":"2023-10-06T05:14:25.733104+0000","last_clean":"2023-10-06T05:14:25.733104+0000","last_became_active":"2023-10-06T05:14:25.726075+0000","last_became_peered":"2023-10-06T05:14:25.726075+0000","last_unstale":"2023-10-06T05:14:25.733104+0000","last_undegraded":"2023-10-06T05:14:25.733104+0000","last_fullsized":"2023-10-06T05:14:25.733104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744829+0000","last_change":"2023-10-06T05:14:25.743617+0000","last_active":"2023-10-06T05:14:25.744829+0000","last_peered":"2023-10-06T05:14:25.744829+0000","last_clean":"2023-10-06T05:14:25.744829+0000","last_became_active":"2023-10-06T05:14:25.741960+0000","last_became_peered":"2023-10-06T05:14:25.741960+0000","last_unstale":"2023-10-06T05:14:25.744829+0000","last_undegraded":"2023-10-06T05:14:25.744829+0000","last_fullsized":"2023-10-06T05:14:25.744829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:50.792266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735674+0000","last_change":"2023-10-06T05:14:25.728085+0000","last_active":"2023-10-06T05:14:25.735674+0000","last_peered":"2023-10-06T05:14:25.735674+0000","last_clean":"2023-10-06T05:14:25.735674+0000","last_became_active":"2023-10-06T05:14:25.727721+0000","last_became_peered":"2023-10-06T05:14:25.727721+0000","last_unstale":"2023-10-06T05:14:25.735674+0000","last_undegraded":"2023-10-06T05:14:25.735674+0000","last_fullsized":"2023-10-06T05:14:25.735674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739584+0000","last_change":"2023-10-06T05:14:25.735959+0000","last_active":"2023-10-06T05:14:25.739584+0000","last_peered":"2023-10-06T05:14:25.739584+0000","last_clean":"2023-10-06T05:14:25.739584+0000","last_became_active":"2023-10-06T05:14:25.735602+0000","last_became_peered":"2023-10-06T05:14:25.735602+0000","last_unstale":"2023-10-06T05:14:25.739584+0000","last_undegraded":"2023-10-06T05:14:25.739584+0000","last_fullsized":"2023-10-06T05:14:25.739584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7349999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7639999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:14:35.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:14:35.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.937+0000 7fefbdf9d700 1 Processor -- start 2023-10-06T05:14:35.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.937+0000 7fefbdf9d700 1 -- start start 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.938+0000 7fefbdf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fefb80ccfc0 0x7fefb80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.938+0000 7fefbdf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fefb80cc750 0x7fefb80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.938+0000 7fefbdf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb80cbd80 con 0x7fefb80cf3b0 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.938+0000 7fefbdf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb80af5f0 con 0x7fefb80ccfc0 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.938+0000 7fefbdf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb80af8d0 con 0x7fefb80cc750 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fefb80ccfc0 0x7fefb80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fefb80ccfc0 0x7fefb80cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37708/0 (socket says 172.21.15.33:37708) 2023-10-06T05:14:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 -- 172.21.15.33:0/4004218853 learned_addr learned my addr 172.21.15.33:0/4004218853 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 -- 172.21.15.33:0/4004218853 >> v2:172.21.15.167:3300/0 conn(0x7fefb80cc750 msgr2=0x7fefb80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbd79c700 1 --2- 172.21.15.33:0/4004218853 >> v2:172.21.15.167:3300/0 conn(0x7fefb80cc750 0x7fefb80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 --2- 172.21.15.33:0/4004218853 >> v2:172.21.15.167:3300/0 conn(0x7fefb80cc750 0x7fefb80b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 -- 172.21.15.33:0/4004218853 >> v1:172.21.15.167:6789/0 conn(0x7fefb80cf3b0 legacy=0x7fefb81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 -- 172.21.15.33:0/4004218853 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80afbb0 con 0x7fefb80ccfc0 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbd79c700 1 --2- 172.21.15.33:0/4004218853 >> v2:172.21.15.167:3300/0 conn(0x7fefb80cc750 0x7fefb80b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefbcf9b700 1 --2- 172.21.15.33:0/4004218853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80ccfc0 0x7fefb80cc210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefac0099b0 tx=0x7fefac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf5b4c4e48d97f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.939+0000 7fefb77fe700 1 -- 172.21.15.33:0/4004218853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00b7e0 con 0x7fefb80ccfc0 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefb77fe700 1 -- 172.21.15.33:0/4004218853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac00b940 con 0x7fefb80ccfc0 2023-10-06T05:14:35.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefb77fe700 1 -- 172.21.15.33:0/4004218853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00bce0 con 0x7fefb80ccfc0 2023-10-06T05:14:35.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 -- 172.21.15.33:0/4004218853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80ccfc0 msgr2=0x7fefb80cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/4004218853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80ccfc0 0x7fefb80cc210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefac0099b0 tx=0x7fefac009250 comp rx=0 tx=0).stop 2023-10-06T05:14:35.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 -- 172.21.15.33:0/4004218853 shutdown_connections 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/4004218853 >> v2:172.21.15.167:3300/0 conn(0x7fefb80cc750 0x7fefb80b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/4004218853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80ccfc0 0x7fefb80cc210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 -- 172.21.15.33:0/4004218853 >> 172.21.15.33:0/4004218853 conn(0x7fefb806cff0 msgr2=0x7fefb806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 -- 172.21.15.33:0/4004218853 shutdown_connections 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.940+0000 7fefbdf9d700 1 -- 172.21.15.33:0/4004218853 wait complete. 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.942+0000 7fefbdf9d700 1 Processor -- start 2023-10-06T05:14:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.942+0000 7fefbdf9d700 1 -- start start 2023-10-06T05:14:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbdf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 0x7fefb80c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbdf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb80ccfc0 0x7fefb8148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbdf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefb80c8d80 0x7fefb80b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbdf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb8154580 con 0x7fefb80cc750 2023-10-06T05:14:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbdf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80c88f0 con 0x7fefb80ccfc0 2023-10-06T05:14:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbdf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80ca300 con 0x7fefb80c8d80 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 0x7fefb80c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 0x7fefb80c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37710/0 (socket says 172.21.15.33:37710) 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 -- 172.21.15.33:0/3827372330 learned_addr learned my addr 172.21.15.33:0/3827372330 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbd79c700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefb80c8d80 0x7fefb80b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 -- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefb80c8d80 msgr2=0x7fefb80b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefb7fff700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb80ccfc0 0x7fefb8148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefb80c8d80 0x7fefb80b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 -- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb80ccfc0 msgr2=0x7fefb8148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb80ccfc0 0x7fefb8148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbcf9b700 1 -- 172.21.15.33:0/3827372330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0094f0 con 0x7fefb80cc750 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefbd79c700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefb80c8d80 0x7fefb80b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.943+0000 7fefb7fff700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb80ccfc0 0x7fefb8148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:35.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.944+0000 7fefbcf9b700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 0x7fefb80c9c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefb806be20 tx=0x7fefac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:35.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.944+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac01b680 con 0x7fefb80cc750 2023-10-06T05:14:35.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.944+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac01b7e0 con 0x7fefb80cc750 2023-10-06T05:14:35.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.944+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb80c7fc0 con 0x7fefb80cc750 2023-10-06T05:14:35.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.944+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac01bba0 con 0x7fefb80cc750 2023-10-06T05:14:35.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.944+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb80c8580 con 0x7fefb80cc750 2023-10-06T05:14:35.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.945+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb80c8bd0 con 0x7fefb80cc750 2023-10-06T05:14:35.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.946+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fefac01c450 con 0x7fefb80cc750 2023-10-06T05:14:35.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.947+0000 7fefb4ff9700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef9c066f30 0x7fef9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:35.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.947+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fefac086440 con 0x7fefb80cc750 2023-10-06T05:14:35.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.948+0000 7fefb7fff700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef9c066f30 0x7fef9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:35.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.948+0000 7fefb7fff700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef9c066f30 0x7fef9c0693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefa4006fd0 tx=0x7fefa4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:35.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:35.951+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefac0534b0 con 0x7fefb80cc750 2023-10-06T05:14:36.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.147+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fefb80d1390 con 0x7fef9c066f30 2023-10-06T05:14:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.156+0000 7fefb4ff9700 1 -- 172.21.15.33:0/3827372330 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286596 (secure 0 0 0) 0x7fefb80d1390 con 0x7fef9c066f30 2023-10-06T05:14:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:36.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef9c066f30 msgr2=0x7fef9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef9c066f30 0x7fef9c0693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefa4006fd0 tx=0x7fefa4008040 comp rx=0 tx=0).stop 2023-10-06T05:14:36.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 msgr2=0x7fefb80c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 0x7fefb80c9c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefb806be20 tx=0x7fefac016000 comp rx=0 tx=0).stop 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 shutdown_connections 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef9c066f30 0x7fef9c0693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefb80c8d80 0x7fefb80b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb80cc750 0x7fefb80c9c90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 --2- 172.21.15.33:0/3827372330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb80ccfc0 0x7fefb8148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 >> 172.21.15.33:0/3827372330 conn(0x7fefb806cff0 msgr2=0x7fefb81588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 shutdown_connections 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.160+0000 7fefbdf9d700 1 -- 172.21.15.33:0/3827372330 wait complete. 2023-10-06T05:14:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:14:36.180 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T05:14:35.820136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002863"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740130+0000","last_change":"2023-10-06T05:14:25.733340+0000","last_active":"2023-10-06T05:14:25.740130+0000","last_peered":"2023-10-06T05:14:25.740130+0000","last_clean":"2023-10-06T05:14:25.740130+0000","last_became_active":"2023-10-06T05:14:25.732861+0000","last_became_peered":"2023-10-06T05:14:25.732861+0000","last_unstale":"2023-10-06T05:14:25.740130+0000","last_undegraded":"2023-10-06T05:14:25.740130+0000","last_fullsized":"2023-10-06T05:14:25.740130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:19.965653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733215+0000","last_change":"2023-10-06T05:14:25.730206+0000","last_active":"2023-10-06T05:14:25.733215+0000","last_peered":"2023-10-06T05:14:25.733215+0000","last_clean":"2023-10-06T05:14:25.733215+0000","last_became_active":"2023-10-06T05:14:25.729947+0000","last_became_peered":"2023-10-06T05:14:25.729947+0000","last_unstale":"2023-10-06T05:14:25.733215+0000","last_undegraded":"2023-10-06T05:14:25.733215+0000","last_fullsized":"2023-10-06T05:14:25.733215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747275+0000","last_change":"2023-10-06T05:14:25.744108+0000","last_active":"2023-10-06T05:14:25.747275+0000","last_peered":"2023-10-06T05:14:25.747275+0000","last_clean":"2023-10-06T05:14:25.747275+0000","last_became_active":"2023-10-06T05:14:25.741869+0000","last_became_peered":"2023-10-06T05:14:25.741869+0000","last_unstale":"2023-10-06T05:14:25.747275+0000","last_undegraded":"2023-10-06T05:14:25.747275+0000","last_fullsized":"2023-10-06T05:14:25.747275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:26.682160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727785+0000","last_change":"2023-10-06T05:14:25.724254+0000","last_active":"2023-10-06T05:14:25.727785+0000","last_peered":"2023-10-06T05:14:25.727785+0000","last_clean":"2023-10-06T05:14:25.727785+0000","last_became_active":"2023-10-06T05:14:25.723995+0000","last_became_peered":"2023-10-06T05:14:25.723995+0000","last_unstale":"2023-10-06T05:14:25.727785+0000","last_undegraded":"2023-10-06T05:14:25.727785+0000","last_fullsized":"2023-10-06T05:14:25.727785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:07.229350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735821+0000","last_change":"2023-10-06T05:14:25.726644+0000","last_active":"2023-10-06T05:14:25.735821+0000","last_peered":"2023-10-06T05:14:25.735821+0000","last_clean":"2023-10-06T05:14:25.735821+0000","last_became_active":"2023-10-06T05:14:25.726030+0000","last_became_peered":"2023-10-06T05:14:25.726030+0000","last_unstale":"2023-10-06T05:14:25.735821+0000","last_undegraded":"2023-10-06T05:14:25.735821+0000","last_fullsized":"2023-10-06T05:14:25.735821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747166+0000","last_change":"2023-10-06T05:14:25.743830+0000","last_active":"2023-10-06T05:14:25.747166+0000","last_peered":"2023-10-06T05:14:25.747166+0000","last_clean":"2023-10-06T05:14:25.747166+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.747166+0000","last_undegraded":"2023-10-06T05:14:25.747166+0000","last_fullsized":"2023-10-06T05:14:25.747166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:43.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739182+0000","last_change":"2023-10-06T05:14:25.737791+0000","last_active":"2023-10-06T05:14:25.739182+0000","last_peered":"2023-10-06T05:14:25.739182+0000","last_clean":"2023-10-06T05:14:25.739182+0000","last_became_active":"2023-10-06T05:14:25.737334+0000","last_became_peered":"2023-10-06T05:14:25.737334+0000","last_unstale":"2023-10-06T05:14:25.739182+0000","last_undegraded":"2023-10-06T05:14:25.739182+0000","last_fullsized":"2023-10-06T05:14:25.739182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744210+0000","last_change":"2023-10-06T05:14:25.743380+0000","last_active":"2023-10-06T05:14:25.744210+0000","last_peered":"2023-10-06T05:14:25.744210+0000","last_clean":"2023-10-06T05:14:25.744210+0000","last_became_active":"2023-10-06T05:14:25.743115+0000","last_became_peered":"2023-10-06T05:14:25.743115+0000","last_unstale":"2023-10-06T05:14:25.744210+0000","last_undegraded":"2023-10-06T05:14:25.744210+0000","last_fullsized":"2023-10-06T05:14:25.744210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736861+0000","last_change":"2023-10-06T05:14:25.730260+0000","last_active":"2023-10-06T05:14:25.736861+0000","last_peered":"2023-10-06T05:14:25.736861+0000","last_clean":"2023-10-06T05:14:25.736861+0000","last_became_active":"2023-10-06T05:14:25.729967+0000","last_became_peered":"2023-10-06T05:14:25.729967+0000","last_unstale":"2023-10-06T05:14:25.736861+0000","last_undegraded":"2023-10-06T05:14:25.736861+0000","last_fullsized":"2023-10-06T05:14:25.736861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.734367+0000","last_change":"2023-10-06T05:14:25.732052+0000","last_active":"2023-10-06T05:14:25.734367+0000","last_peered":"2023-10-06T05:14:25.734367+0000","last_clean":"2023-10-06T05:14:25.734367+0000","last_became_active":"2023-10-06T05:14:25.731746+0000","last_became_peered":"2023-10-06T05:14:25.731746+0000","last_unstale":"2023-10-06T05:14:25.734367+0000","last_undegraded":"2023-10-06T05:14:25.734367+0000","last_fullsized":"2023-10-06T05:14:25.734367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744812+0000","last_change":"2023-10-06T05:14:25.741800+0000","last_active":"2023-10-06T05:14:25.744812+0000","last_peered":"2023-10-06T05:14:25.744812+0000","last_clean":"2023-10-06T05:14:25.744812+0000","last_became_active":"2023-10-06T05:14:25.741167+0000","last_became_peered":"2023-10-06T05:14:25.741167+0000","last_unstale":"2023-10-06T05:14:25.744812+0000","last_undegraded":"2023-10-06T05:14:25.744812+0000","last_fullsized":"2023-10-06T05:14:25.744812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743990+0000","last_change":"2023-10-06T05:14:25.743115+0000","last_active":"2023-10-06T05:14:25.743990+0000","last_peered":"2023-10-06T05:14:25.743990+0000","last_clean":"2023-10-06T05:14:25.743990+0000","last_became_active":"2023-10-06T05:14:25.742809+0000","last_became_peered":"2023-10-06T05:14:25.742809+0000","last_unstale":"2023-10-06T05:14:25.743990+0000","last_undegraded":"2023-10-06T05:14:25.743990+0000","last_fullsized":"2023-10-06T05:14:25.743990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.740187+0000","last_change":"2023-10-06T05:14:25.734027+0000","last_active":"2023-10-06T05:14:25.740187+0000","last_peered":"2023-10-06T05:14:25.740187+0000","last_clean":"2023-10-06T05:14:25.740187+0000","last_became_active":"2023-10-06T05:14:25.733743+0000","last_became_peered":"2023-10-06T05:14:25.733743+0000","last_unstale":"2023-10-06T05:14:25.740187+0000","last_undegraded":"2023-10-06T05:14:25.740187+0000","last_fullsized":"2023-10-06T05:14:25.740187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.725222+0000","last_change":"2023-10-06T05:14:25.721669+0000","last_active":"2023-10-06T05:14:25.725222+0000","last_peered":"2023-10-06T05:14:25.725222+0000","last_clean":"2023-10-06T05:14:25.725222+0000","last_became_active":"2023-10-06T05:14:25.721412+0000","last_became_peered":"2023-10-06T05:14:25.721412+0000","last_unstale":"2023-10-06T05:14:25.725222+0000","last_undegraded":"2023-10-06T05:14:25.725222+0000","last_fullsized":"2023-10-06T05:14:25.725222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739202+0000","last_change":"2023-10-06T05:14:25.737798+0000","last_active":"2023-10-06T05:14:25.739202+0000","last_peered":"2023-10-06T05:14:25.739202+0000","last_clean":"2023-10-06T05:14:25.739202+0000","last_became_active":"2023-10-06T05:14:25.737342+0000","last_became_peered":"2023-10-06T05:14:25.737342+0000","last_unstale":"2023-10-06T05:14:25.739202+0000","last_undegraded":"2023-10-06T05:14:25.739202+0000","last_fullsized":"2023-10-06T05:14:25.739202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742612+0000","last_change":"2023-10-06T05:14:25.740804+0000","last_active":"2023-10-06T05:14:25.742612+0000","last_peered":"2023-10-06T05:14:25.742612+0000","last_clean":"2023-10-06T05:14:25.742612+0000","last_became_active":"2023-10-06T05:14:25.740330+0000","last_became_peered":"2023-10-06T05:14:25.740330+0000","last_unstale":"2023-10-06T05:14:25.742612+0000","last_undegraded":"2023-10-06T05:14:25.742612+0000","last_fullsized":"2023-10-06T05:14:25.742612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.955604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731300+0000","last_change":"2023-10-06T05:14:25.725672+0000","last_active":"2023-10-06T05:14:25.731300+0000","last_peered":"2023-10-06T05:14:25.731300+0000","last_clean":"2023-10-06T05:14:25.731300+0000","last_became_active":"2023-10-06T05:14:25.725392+0000","last_became_peered":"2023-10-06T05:14:25.725392+0000","last_unstale":"2023-10-06T05:14:25.731300+0000","last_undegraded":"2023-10-06T05:14:25.731300+0000","last_fullsized":"2023-10-06T05:14:25.731300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744646+0000","last_change":"2023-10-06T05:14:25.743525+0000","last_active":"2023-10-06T05:14:25.744646+0000","last_peered":"2023-10-06T05:14:25.744646+0000","last_clean":"2023-10-06T05:14:25.744646+0000","last_became_active":"2023-10-06T05:14:25.742820+0000","last_became_peered":"2023-10-06T05:14:25.742820+0000","last_unstale":"2023-10-06T05:14:25.744646+0000","last_undegraded":"2023-10-06T05:14:25.744646+0000","last_fullsized":"2023-10-06T05:14:25.744646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:42.341048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.720475+0000","last_change":"2023-10-06T05:14:25.718000+0000","last_active":"2023-10-06T05:14:25.720475+0000","last_peered":"2023-10-06T05:14:25.720475+0000","last_clean":"2023-10-06T05:14:25.720475+0000","last_became_active":"2023-10-06T05:14:25.717615+0000","last_became_peered":"2023-10-06T05:14:25.717615+0000","last_unstale":"2023-10-06T05:14:25.720475+0000","last_undegraded":"2023-10-06T05:14:25.720475+0000","last_fullsized":"2023-10-06T05:14:25.720475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:17.095036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.743854+0000","last_change":"2023-10-06T05:14:25.742445+0000","last_active":"2023-10-06T05:14:25.743854+0000","last_peered":"2023-10-06T05:14:25.743854+0000","last_clean":"2023-10-06T05:14:25.743854+0000","last_became_active":"2023-10-06T05:14:25.741675+0000","last_became_peered":"2023-10-06T05:14:25.741675+0000","last_unstale":"2023-10-06T05:14:25.743854+0000","last_undegraded":"2023-10-06T05:14:25.743854+0000","last_fullsized":"2023-10-06T05:14:25.743854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735911+0000","last_change":"2023-10-06T05:14:25.733702+0000","last_active":"2023-10-06T05:14:25.735911+0000","last_peered":"2023-10-06T05:14:25.735911+0000","last_clean":"2023-10-06T05:14:25.735911+0000","last_became_active":"2023-10-06T05:14:25.733387+0000","last_became_peered":"2023-10-06T05:14:25.733387+0000","last_unstale":"2023-10-06T05:14:25.735911+0000","last_undegraded":"2023-10-06T05:14:25.735911+0000","last_fullsized":"2023-10-06T05:14:25.735911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731166+0000","last_change":"2023-10-06T05:14:25.724100+0000","last_active":"2023-10-06T05:14:25.731166+0000","last_peered":"2023-10-06T05:14:25.731166+0000","last_clean":"2023-10-06T05:14:25.731166+0000","last_became_active":"2023-10-06T05:14:25.723750+0000","last_became_peered":"2023-10-06T05:14:25.723750+0000","last_unstale":"2023-10-06T05:14:25.731166+0000","last_undegraded":"2023-10-06T05:14:25.731166+0000","last_fullsized":"2023-10-06T05:14:25.731166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735739+0000","last_change":"2023-10-06T05:14:25.727464+0000","last_active":"2023-10-06T05:14:25.735739+0000","last_peered":"2023-10-06T05:14:25.735739+0000","last_clean":"2023-10-06T05:14:25.735739+0000","last_became_active":"2023-10-06T05:14:25.727166+0000","last_became_peered":"2023-10-06T05:14:25.727166+0000","last_unstale":"2023-10-06T05:14:25.735739+0000","last_undegraded":"2023-10-06T05:14:25.735739+0000","last_fullsized":"2023-10-06T05:14:25.735739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732019+0000","last_change":"2023-10-06T05:14:25.730521+0000","last_active":"2023-10-06T05:14:25.732019+0000","last_peered":"2023-10-06T05:14:25.732019+0000","last_clean":"2023-10-06T05:14:25.732019+0000","last_became_active":"2023-10-06T05:14:25.730234+0000","last_became_peered":"2023-10-06T05:14:25.730234+0000","last_unstale":"2023-10-06T05:14:25.732019+0000","last_undegraded":"2023-10-06T05:14:25.732019+0000","last_fullsized":"2023-10-06T05:14:25.732019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735194+0000","last_change":"2023-10-06T05:14:25.733204+0000","last_active":"2023-10-06T05:14:25.735194+0000","last_peered":"2023-10-06T05:14:25.735194+0000","last_clean":"2023-10-06T05:14:25.735194+0000","last_became_active":"2023-10-06T05:14:25.732877+0000","last_became_peered":"2023-10-06T05:14:25.732877+0000","last_unstale":"2023-10-06T05:14:25.735194+0000","last_undegraded":"2023-10-06T05:14:25.735194+0000","last_fullsized":"2023-10-06T05:14:25.735194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735005+0000","last_change":"2023-10-06T05:14:25.724268+0000","last_active":"2023-10-06T05:14:25.735005+0000","last_peered":"2023-10-06T05:14:25.735005+0000","last_clean":"2023-10-06T05:14:25.735005+0000","last_became_active":"2023-10-06T05:14:25.724009+0000","last_became_peered":"2023-10-06T05:14:25.724009+0000","last_unstale":"2023-10-06T05:14:25.735005+0000","last_undegraded":"2023-10-06T05:14:25.735005+0000","last_fullsized":"2023-10-06T05:14:25.735005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727218+0000","last_change":"2023-10-06T05:14:25.725754+0000","last_active":"2023-10-06T05:14:25.727218+0000","last_peered":"2023-10-06T05:14:25.727218+0000","last_clean":"2023-10-06T05:14:25.727218+0000","last_became_active":"2023-10-06T05:14:25.725516+0000","last_became_peered":"2023-10-06T05:14:25.725516+0000","last_unstale":"2023-10-06T05:14:25.727218+0000","last_undegraded":"2023-10-06T05:14:25.727218+0000","last_fullsized":"2023-10-06T05:14:25.727218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741625+0000","last_change":"2023-10-06T05:14:24.734577+0000","last_active":"2023-10-06T05:14:25.741625+0000","last_peered":"2023-10-06T05:14:25.741625+0000","last_clean":"2023-10-06T05:14:25.741625+0000","last_became_active":"2023-10-06T05:14:24.732575+0000","last_became_peered":"2023-10-06T05:14:24.732575+0000","last_unstale":"2023-10-06T05:14:25.741625+0000","last_undegraded":"2023-10-06T05:14:25.741625+0000","last_fullsized":"2023-10-06T05:14:25.741625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.726043+0000","last_change":"2023-10-06T05:14:25.721117+0000","last_active":"2023-10-06T05:14:25.726043+0000","last_peered":"2023-10-06T05:14:25.726043+0000","last_clean":"2023-10-06T05:14:25.726043+0000","last_became_active":"2023-10-06T05:14:25.720806+0000","last_became_peered":"2023-10-06T05:14:25.720806+0000","last_unstale":"2023-10-06T05:14:25.726043+0000","last_undegraded":"2023-10-06T05:14:25.726043+0000","last_fullsized":"2023-10-06T05:14:25.726043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:38.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.714069+0000","last_change":"2023-10-06T05:14:24.723717+0000","last_active":"2023-10-06T05:14:25.714069+0000","last_peered":"2023-10-06T05:14:25.714069+0000","last_clean":"2023-10-06T05:14:25.714069+0000","last_became_active":"2023-10-06T05:14:24.723440+0000","last_became_peered":"2023-10-06T05:14:24.723440+0000","last_unstale":"2023-10-06T05:14:25.714069+0000","last_undegraded":"2023-10-06T05:14:25.714069+0000","last_fullsized":"2023-10-06T05:14:25.714069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735391+0000","last_change":"2023-10-06T05:14:25.726258+0000","last_active":"2023-10-06T05:14:25.735391+0000","last_peered":"2023-10-06T05:14:25.735391+0000","last_clean":"2023-10-06T05:14:25.735391+0000","last_became_active":"2023-10-06T05:14:25.725989+0000","last_became_peered":"2023-10-06T05:14:25.725989+0000","last_unstale":"2023-10-06T05:14:25.735391+0000","last_undegraded":"2023-10-06T05:14:25.735391+0000","last_fullsized":"2023-10-06T05:14:25.735391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708532+0000","last_change":"2023-10-06T05:14:24.722594+0000","last_active":"2023-10-06T05:14:25.708532+0000","last_peered":"2023-10-06T05:14:25.708532+0000","last_clean":"2023-10-06T05:14:25.708532+0000","last_became_active":"2023-10-06T05:14:24.722302+0000","last_became_peered":"2023-10-06T05:14:24.722302+0000","last_unstale":"2023-10-06T05:14:25.708532+0000","last_undegraded":"2023-10-06T05:14:25.708532+0000","last_fullsized":"2023-10-06T05:14:25.708532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728738+0000","last_change":"2023-10-06T05:14:25.722475+0000","last_active":"2023-10-06T05:14:25.728738+0000","last_peered":"2023-10-06T05:14:25.728738+0000","last_clean":"2023-10-06T05:14:25.728738+0000","last_became_active":"2023-10-06T05:14:25.721707+0000","last_became_peered":"2023-10-06T05:14:25.721707+0000","last_unstale":"2023-10-06T05:14:25.728738+0000","last_undegraded":"2023-10-06T05:14:25.728738+0000","last_fullsized":"2023-10-06T05:14:25.728738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:41.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716173+0000","last_change":"2023-10-06T05:14:24.724581+0000","last_active":"2023-10-06T05:14:25.716173+0000","last_peered":"2023-10-06T05:14:25.716173+0000","last_clean":"2023-10-06T05:14:25.716173+0000","last_became_active":"2023-10-06T05:14:24.724293+0000","last_became_peered":"2023-10-06T05:14:24.724293+0000","last_unstale":"2023-10-06T05:14:25.716173+0000","last_undegraded":"2023-10-06T05:14:25.716173+0000","last_fullsized":"2023-10-06T05:14:25.716173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723275+0000","last_change":"2023-10-06T05:14:25.719638+0000","last_active":"2023-10-06T05:14:25.723275+0000","last_peered":"2023-10-06T05:14:25.723275+0000","last_clean":"2023-10-06T05:14:25.723275+0000","last_became_active":"2023-10-06T05:14:25.719347+0000","last_became_peered":"2023-10-06T05:14:25.719347+0000","last_unstale":"2023-10-06T05:14:25.723275+0000","last_undegraded":"2023-10-06T05:14:25.723275+0000","last_fullsized":"2023-10-06T05:14:25.723275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:35.535565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716142+0000","last_change":"2023-10-06T05:14:24.722842+0000","last_active":"2023-10-06T05:14:25.716142+0000","last_peered":"2023-10-06T05:14:25.716142+0000","last_clean":"2023-10-06T05:14:25.716142+0000","last_became_active":"2023-10-06T05:14:24.721323+0000","last_became_peered":"2023-10-06T05:14:24.721323+0000","last_unstale":"2023-10-06T05:14:25.716142+0000","last_undegraded":"2023-10-06T05:14:25.716142+0000","last_fullsized":"2023-10-06T05:14:25.716142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735759+0000","last_change":"2023-10-06T05:14:25.728303+0000","last_active":"2023-10-06T05:14:25.735759+0000","last_peered":"2023-10-06T05:14:25.735759+0000","last_clean":"2023-10-06T05:14:25.735759+0000","last_became_active":"2023-10-06T05:14:25.727775+0000","last_became_peered":"2023-10-06T05:14:25.727775+0000","last_unstale":"2023-10-06T05:14:25.735759+0000","last_undegraded":"2023-10-06T05:14:25.735759+0000","last_fullsized":"2023-10-06T05:14:25.735759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738830+0000","last_change":"2023-10-06T05:14:24.728587+0000","last_active":"2023-10-06T05:14:25.738830+0000","last_peered":"2023-10-06T05:14:25.738830+0000","last_clean":"2023-10-06T05:14:25.738830+0000","last_became_active":"2023-10-06T05:14:24.728292+0000","last_became_peered":"2023-10-06T05:14:24.728292+0000","last_unstale":"2023-10-06T05:14:25.738830+0000","last_undegraded":"2023-10-06T05:14:25.738830+0000","last_fullsized":"2023-10-06T05:14:25.738830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:11.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735121+0000","last_change":"2023-10-06T05:14:25.732130+0000","last_active":"2023-10-06T05:14:25.735121+0000","last_peered":"2023-10-06T05:14:25.735121+0000","last_clean":"2023-10-06T05:14:25.735121+0000","last_became_active":"2023-10-06T05:14:25.731459+0000","last_became_peered":"2023-10-06T05:14:25.731459+0000","last_unstale":"2023-10-06T05:14:25.735121+0000","last_undegraded":"2023-10-06T05:14:25.735121+0000","last_fullsized":"2023-10-06T05:14:25.735121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:47.991500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.748549+0000","last_change":"2023-10-06T05:14:24.728538+0000","last_active":"2023-10-06T05:14:25.748549+0000","last_peered":"2023-10-06T05:14:25.748549+0000","last_clean":"2023-10-06T05:14:25.748549+0000","last_became_active":"2023-10-06T05:14:24.728200+0000","last_became_peered":"2023-10-06T05:14:24.728200+0000","last_unstale":"2023-10-06T05:14:25.748549+0000","last_undegraded":"2023-10-06T05:14:25.748549+0000","last_fullsized":"2023-10-06T05:14:25.748549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741887+0000","last_change":"2023-10-06T05:14:25.740582+0000","last_active":"2023-10-06T05:14:25.741887+0000","last_peered":"2023-10-06T05:14:25.741887+0000","last_clean":"2023-10-06T05:14:25.741887+0000","last_became_active":"2023-10-06T05:14:25.740298+0000","last_became_peered":"2023-10-06T05:14:25.740298+0000","last_unstale":"2023-10-06T05:14:25.741887+0000","last_undegraded":"2023-10-06T05:14:25.741887+0000","last_fullsized":"2023-10-06T05:14:25.741887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:31.489344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742441+0000","last_change":"2023-10-06T05:14:24.735437+0000","last_active":"2023-10-06T05:14:25.742441+0000","last_peered":"2023-10-06T05:14:25.742441+0000","last_clean":"2023-10-06T05:14:25.742441+0000","last_became_active":"2023-10-06T05:14:24.735168+0000","last_became_peered":"2023-10-06T05:14:24.735168+0000","last_unstale":"2023-10-06T05:14:25.742441+0000","last_undegraded":"2023-10-06T05:14:25.742441+0000","last_fullsized":"2023-10-06T05:14:25.742441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:06.220991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723244+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.723244+0000","last_peered":"2023-10-06T05:14:25.723244+0000","last_clean":"2023-10-06T05:14:25.723244+0000","last_became_active":"2023-10-06T05:14:25.719474+0000","last_became_peered":"2023-10-06T05:14:25.719474+0000","last_unstale":"2023-10-06T05:14:25.723244+0000","last_undegraded":"2023-10-06T05:14:25.723244+0000","last_fullsized":"2023-10-06T05:14:25.723244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.712742+0000","last_change":"2023-10-06T05:14:24.718646+0000","last_active":"2023-10-06T05:14:25.712742+0000","last_peered":"2023-10-06T05:14:25.712742+0000","last_clean":"2023-10-06T05:14:25.712742+0000","last_became_active":"2023-10-06T05:14:24.718324+0000","last_became_peered":"2023-10-06T05:14:24.718324+0000","last_unstale":"2023-10-06T05:14:25.712742+0000","last_undegraded":"2023-10-06T05:14:25.712742+0000","last_fullsized":"2023-10-06T05:14:25.712742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727199+0000","last_change":"2023-10-06T05:14:25.724883+0000","last_active":"2023-10-06T05:14:25.727199+0000","last_peered":"2023-10-06T05:14:25.727199+0000","last_clean":"2023-10-06T05:14:25.727199+0000","last_became_active":"2023-10-06T05:14:25.724633+0000","last_became_peered":"2023-10-06T05:14:25.724633+0000","last_unstale":"2023-10-06T05:14:25.727199+0000","last_undegraded":"2023-10-06T05:14:25.727199+0000","last_fullsized":"2023-10-06T05:14:25.727199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.720059+0000","last_change":"2023-10-06T05:14:24.736333+0000","last_active":"2023-10-06T05:14:25.720059+0000","last_peered":"2023-10-06T05:14:25.720059+0000","last_clean":"2023-10-06T05:14:25.720059+0000","last_became_active":"2023-10-06T05:14:24.736038+0000","last_became_peered":"2023-10-06T05:14:24.736038+0000","last_unstale":"2023-10-06T05:14:25.720059+0000","last_undegraded":"2023-10-06T05:14:25.720059+0000","last_fullsized":"2023-10-06T05:14:25.720059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.408081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735371+0000","last_change":"2023-10-06T05:14:25.725666+0000","last_active":"2023-10-06T05:14:25.735371+0000","last_peered":"2023-10-06T05:14:25.735371+0000","last_clean":"2023-10-06T05:14:25.735371+0000","last_became_active":"2023-10-06T05:14:25.725319+0000","last_became_peered":"2023-10-06T05:14:25.725319+0000","last_unstale":"2023-10-06T05:14:25.735371+0000","last_undegraded":"2023-10-06T05:14:25.735371+0000","last_fullsized":"2023-10-06T05:14:25.735371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733857+0000","last_change":"2023-10-06T05:14:24.721548+0000","last_active":"2023-10-06T05:14:25.733857+0000","last_peered":"2023-10-06T05:14:25.733857+0000","last_clean":"2023-10-06T05:14:25.733857+0000","last_became_active":"2023-10-06T05:14:24.721266+0000","last_became_peered":"2023-10-06T05:14:24.721266+0000","last_unstale":"2023-10-06T05:14:25.733857+0000","last_undegraded":"2023-10-06T05:14:25.733857+0000","last_fullsized":"2023-10-06T05:14:25.733857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727201+0000","last_change":"2023-10-06T05:14:25.725874+0000","last_active":"2023-10-06T05:14:25.727201+0000","last_peered":"2023-10-06T05:14:25.727201+0000","last_clean":"2023-10-06T05:14:25.727201+0000","last_became_active":"2023-10-06T05:14:25.725494+0000","last_became_peered":"2023-10-06T05:14:25.725494+0000","last_unstale":"2023-10-06T05:14:25.727201+0000","last_undegraded":"2023-10-06T05:14:25.727201+0000","last_fullsized":"2023-10-06T05:14:25.727201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742053+0000","last_change":"2023-10-06T05:14:24.733795+0000","last_active":"2023-10-06T05:14:25.742053+0000","last_peered":"2023-10-06T05:14:25.742053+0000","last_clean":"2023-10-06T05:14:25.742053+0000","last_became_active":"2023-10-06T05:14:24.733567+0000","last_became_peered":"2023-10-06T05:14:24.733567+0000","last_unstale":"2023-10-06T05:14:25.742053+0000","last_undegraded":"2023-10-06T05:14:25.742053+0000","last_fullsized":"2023-10-06T05:14:25.742053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732255+0000","last_change":"2023-10-06T05:14:25.727738+0000","last_active":"2023-10-06T05:14:25.732255+0000","last_peered":"2023-10-06T05:14:25.732255+0000","last_clean":"2023-10-06T05:14:25.732255+0000","last_became_active":"2023-10-06T05:14:25.727474+0000","last_became_peered":"2023-10-06T05:14:25.727474+0000","last_unstale":"2023-10-06T05:14:25.732255+0000","last_undegraded":"2023-10-06T05:14:25.732255+0000","last_fullsized":"2023-10-06T05:14:25.732255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.745629+0000","last_change":"2023-10-06T05:14:24.733458+0000","last_active":"2023-10-06T05:14:25.745629+0000","last_peered":"2023-10-06T05:14:25.745629+0000","last_clean":"2023-10-06T05:14:25.745629+0000","last_became_active":"2023-10-06T05:14:24.733160+0000","last_became_peered":"2023-10-06T05:14:24.733160+0000","last_unstale":"2023-10-06T05:14:25.745629+0000","last_undegraded":"2023-10-06T05:14:25.745629+0000","last_fullsized":"2023-10-06T05:14:25.745629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744280+0000","last_change":"2023-10-06T05:14:25.741624+0000","last_active":"2023-10-06T05:14:25.744280+0000","last_peered":"2023-10-06T05:14:25.744280+0000","last_clean":"2023-10-06T05:14:25.744280+0000","last_became_active":"2023-10-06T05:14:25.740252+0000","last_became_peered":"2023-10-06T05:14:25.740252+0000","last_unstale":"2023-10-06T05:14:25.744280+0000","last_undegraded":"2023-10-06T05:14:25.744280+0000","last_fullsized":"2023-10-06T05:14:25.744280+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.732612+0000","last_change":"2023-10-06T05:14:24.721437+0000","last_active":"2023-10-06T05:14:25.732612+0000","last_peered":"2023-10-06T05:14:25.732612+0000","last_clean":"2023-10-06T05:14:25.732612+0000","last_became_active":"2023-10-06T05:14:24.721172+0000","last_became_peered":"2023-10-06T05:14:24.721172+0000","last_unstale":"2023-10-06T05:14:25.732612+0000","last_undegraded":"2023-10-06T05:14:25.732612+0000","last_fullsized":"2023-10-06T05:14:25.732612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:22.852582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727675+0000","last_change":"2023-10-06T05:14:25.719600+0000","last_active":"2023-10-06T05:14:25.727675+0000","last_peered":"2023-10-06T05:14:25.727675+0000","last_clean":"2023-10-06T05:14:25.727675+0000","last_became_active":"2023-10-06T05:14:25.719349+0000","last_became_peered":"2023-10-06T05:14:25.719349+0000","last_unstale":"2023-10-06T05:14:25.727675+0000","last_undegraded":"2023-10-06T05:14:25.727675+0000","last_fullsized":"2023-10-06T05:14:25.727675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:20.416441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716852+0000","last_change":"2023-10-06T05:14:24.728714+0000","last_active":"2023-10-06T05:14:25.716852+0000","last_peered":"2023-10-06T05:14:25.716852+0000","last_clean":"2023-10-06T05:14:25.716852+0000","last_became_active":"2023-10-06T05:14:24.728435+0000","last_became_peered":"2023-10-06T05:14:24.728435+0000","last_unstale":"2023-10-06T05:14:25.716852+0000","last_undegraded":"2023-10-06T05:14:25.716852+0000","last_fullsized":"2023-10-06T05:14:25.716852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:40.445537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731329+0000","last_change":"2023-10-06T05:14:25.724993+0000","last_active":"2023-10-06T05:14:25.731329+0000","last_peered":"2023-10-06T05:14:25.731329+0000","last_clean":"2023-10-06T05:14:25.731329+0000","last_became_active":"2023-10-06T05:14:25.724538+0000","last_became_peered":"2023-10-06T05:14:25.724538+0000","last_unstale":"2023-10-06T05:14:25.731329+0000","last_undegraded":"2023-10-06T05:14:25.731329+0000","last_fullsized":"2023-10-06T05:14:25.731329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738370+0000","last_change":"2023-10-06T05:14:24.723321+0000","last_active":"2023-10-06T05:14:25.738370+0000","last_peered":"2023-10-06T05:14:25.738370+0000","last_clean":"2023-10-06T05:14:25.738370+0000","last_became_active":"2023-10-06T05:14:24.723063+0000","last_became_peered":"2023-10-06T05:14:24.723063+0000","last_unstale":"2023-10-06T05:14:25.738370+0000","last_undegraded":"2023-10-06T05:14:25.738370+0000","last_fullsized":"2023-10-06T05:14:25.738370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742001+0000","last_change":"2023-10-06T05:14:25.740648+0000","last_active":"2023-10-06T05:14:25.742001+0000","last_peered":"2023-10-06T05:14:25.742001+0000","last_clean":"2023-10-06T05:14:25.742001+0000","last_became_active":"2023-10-06T05:14:25.740351+0000","last_became_peered":"2023-10-06T05:14:25.740351+0000","last_unstale":"2023-10-06T05:14:25.742001+0000","last_undegraded":"2023-10-06T05:14:25.742001+0000","last_fullsized":"2023-10-06T05:14:25.742001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709955+0000","last_change":"2023-10-06T05:14:24.730263+0000","last_active":"2023-10-06T05:14:25.709955+0000","last_peered":"2023-10-06T05:14:25.709955+0000","last_clean":"2023-10-06T05:14:25.709955+0000","last_became_active":"2023-10-06T05:14:24.729911+0000","last_became_peered":"2023-10-06T05:14:24.729911+0000","last_unstale":"2023-10-06T05:14:25.709955+0000","last_undegraded":"2023-10-06T05:14:25.709955+0000","last_fullsized":"2023-10-06T05:14:25.709955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747672+0000","last_change":"2023-10-06T05:14:25.744620+0000","last_active":"2023-10-06T05:14:25.747672+0000","last_peered":"2023-10-06T05:14:25.747672+0000","last_clean":"2023-10-06T05:14:25.747672+0000","last_became_active":"2023-10-06T05:14:25.744323+0000","last_became_peered":"2023-10-06T05:14:25.744323+0000","last_unstale":"2023-10-06T05:14:25.747672+0000","last_undegraded":"2023-10-06T05:14:25.747672+0000","last_fullsized":"2023-10-06T05:14:25.747672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:39.850675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730362+0000","last_change":"2023-10-06T05:14:24.731862+0000","last_active":"2023-10-06T05:14:25.730362+0000","last_peered":"2023-10-06T05:14:25.730362+0000","last_clean":"2023-10-06T05:14:25.730362+0000","last_became_active":"2023-10-06T05:14:24.729629+0000","last_became_peered":"2023-10-06T05:14:24.729629+0000","last_unstale":"2023-10-06T05:14:25.730362+0000","last_undegraded":"2023-10-06T05:14:25.730362+0000","last_fullsized":"2023-10-06T05:14:25.730362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744587+0000","last_change":"2023-10-06T05:14:25.742674+0000","last_active":"2023-10-06T05:14:25.744587+0000","last_peered":"2023-10-06T05:14:25.744587+0000","last_clean":"2023-10-06T05:14:25.744587+0000","last_became_active":"2023-10-06T05:14:25.741308+0000","last_became_peered":"2023-10-06T05:14:25.741308+0000","last_unstale":"2023-10-06T05:14:25.744587+0000","last_undegraded":"2023-10-06T05:14:25.744587+0000","last_fullsized":"2023-10-06T05:14:25.744587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:29.811666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708350+0000","last_change":"2023-10-06T05:14:24.724203+0000","last_active":"2023-10-06T05:14:25.708350+0000","last_peered":"2023-10-06T05:14:25.708350+0000","last_clean":"2023-10-06T05:14:25.708350+0000","last_became_active":"2023-10-06T05:14:24.723929+0000","last_became_peered":"2023-10-06T05:14:24.723929+0000","last_unstale":"2023-10-06T05:14:25.708350+0000","last_undegraded":"2023-10-06T05:14:25.708350+0000","last_fullsized":"2023-10-06T05:14:25.708350+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735025+0000","last_change":"2023-10-06T05:14:25.726645+0000","last_active":"2023-10-06T05:14:25.735025+0000","last_peered":"2023-10-06T05:14:25.735025+0000","last_clean":"2023-10-06T05:14:25.735025+0000","last_became_active":"2023-10-06T05:14:25.724695+0000","last_became_peered":"2023-10-06T05:14:25.724695+0000","last_unstale":"2023-10-06T05:14:25.735025+0000","last_undegraded":"2023-10-06T05:14:25.735025+0000","last_fullsized":"2023-10-06T05:14:25.735025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:51.658034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713550+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:14:25.713550+0000","last_peered":"2023-10-06T05:14:25.713550+0000","last_clean":"2023-10-06T05:14:25.713550+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:14:25.713550+0000","last_undegraded":"2023-10-06T05:14:25.713550+0000","last_fullsized":"2023-10-06T05:14:25.713550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.716529+0000","last_change":"2023-10-06T05:14:24.728757+0000","last_active":"2023-10-06T05:14:25.716529+0000","last_peered":"2023-10-06T05:14:25.716529+0000","last_clean":"2023-10-06T05:14:25.716529+0000","last_became_active":"2023-10-06T05:14:24.728480+0000","last_became_peered":"2023-10-06T05:14:24.728480+0000","last_unstale":"2023-10-06T05:14:25.716529+0000","last_undegraded":"2023-10-06T05:14:25.716529+0000","last_fullsized":"2023-10-06T05:14:25.716529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727990+0000","last_change":"2023-10-06T05:14:25.719792+0000","last_active":"2023-10-06T05:14:25.727990+0000","last_peered":"2023-10-06T05:14:25.727990+0000","last_clean":"2023-10-06T05:14:25.727990+0000","last_became_active":"2023-10-06T05:14:25.719491+0000","last_became_peered":"2023-10-06T05:14:25.719491+0000","last_unstale":"2023-10-06T05:14:25.727990+0000","last_undegraded":"2023-10-06T05:14:25.727990+0000","last_fullsized":"2023-10-06T05:14:25.727990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715616+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:14:25.715616+0000","last_peered":"2023-10-06T05:14:25.715616+0000","last_clean":"2023-10-06T05:14:25.715616+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:14:25.715616+0000","last_undegraded":"2023-10-06T05:14:25.715616+0000","last_fullsized":"2023-10-06T05:14:25.715616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.731289+0000","last_change":"2023-10-06T05:14:24.732737+0000","last_active":"2023-10-06T05:14:25.731289+0000","last_peered":"2023-10-06T05:14:25.731289+0000","last_clean":"2023-10-06T05:14:25.731289+0000","last_became_active":"2023-10-06T05:14:24.732410+0000","last_became_peered":"2023-10-06T05:14:24.732410+0000","last_unstale":"2023-10-06T05:14:25.731289+0000","last_undegraded":"2023-10-06T05:14:25.731289+0000","last_fullsized":"2023-10-06T05:14:25.731289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736090+0000","last_change":"2023-10-06T05:14:25.723180+0000","last_active":"2023-10-06T05:14:25.736090+0000","last_peered":"2023-10-06T05:14:25.736090+0000","last_clean":"2023-10-06T05:14:25.736090+0000","last_became_active":"2023-10-06T05:14:25.722867+0000","last_became_peered":"2023-10-06T05:14:25.722867+0000","last_unstale":"2023-10-06T05:14:25.736090+0000","last_undegraded":"2023-10-06T05:14:25.736090+0000","last_fullsized":"2023-10-06T05:14:25.736090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.705335+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:14:25.705335+0000","last_peered":"2023-10-06T05:14:25.705335+0000","last_clean":"2023-10-06T05:14:25.705335+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:14:25.705335+0000","last_undegraded":"2023-10-06T05:14:25.705335+0000","last_fullsized":"2023-10-06T05:14:25.705335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713983+0000","last_change":"2023-10-06T05:14:24.719784+0000","last_active":"2023-10-06T05:14:25.713983+0000","last_peered":"2023-10-06T05:14:25.713983+0000","last_clean":"2023-10-06T05:14:25.713983+0000","last_became_active":"2023-10-06T05:14:24.719414+0000","last_became_peered":"2023-10-06T05:14:24.719414+0000","last_unstale":"2023-10-06T05:14:25.713983+0000","last_undegraded":"2023-10-06T05:14:25.713983+0000","last_fullsized":"2023-10-06T05:14:25.713983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735002+0000","last_change":"2023-10-06T05:14:25.725673+0000","last_active":"2023-10-06T05:14:25.735002+0000","last_peered":"2023-10-06T05:14:25.735002+0000","last_clean":"2023-10-06T05:14:25.735002+0000","last_became_active":"2023-10-06T05:14:25.725107+0000","last_became_peered":"2023-10-06T05:14:25.725107+0000","last_unstale":"2023-10-06T05:14:25.735002+0000","last_undegraded":"2023-10-06T05:14:25.735002+0000","last_fullsized":"2023-10-06T05:14:25.735002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:35.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.715779+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:14:25.715779+0000","last_peered":"2023-10-06T05:14:25.715779+0000","last_clean":"2023-10-06T05:14:25.715779+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:14:25.715779+0000","last_undegraded":"2023-10-06T05:14:25.715779+0000","last_fullsized":"2023-10-06T05:14:25.715779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719147+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:14:25.719147+0000","last_peered":"2023-10-06T05:14:25.719147+0000","last_clean":"2023-10-06T05:14:25.719147+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:14:25.719147+0000","last_undegraded":"2023-10-06T05:14:25.719147+0000","last_fullsized":"2023-10-06T05:14:25.719147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.708071+0000","last_change":"2023-10-06T05:14:24.717406+0000","last_active":"2023-10-06T05:14:25.708071+0000","last_peered":"2023-10-06T05:14:25.708071+0000","last_clean":"2023-10-06T05:14:25.708071+0000","last_became_active":"2023-10-06T05:14:24.716723+0000","last_became_peered":"2023-10-06T05:14:24.716723+0000","last_unstale":"2023-10-06T05:14:25.708071+0000","last_undegraded":"2023-10-06T05:14:25.708071+0000","last_fullsized":"2023-10-06T05:14:25.708071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:15.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.746536+0000","last_change":"2023-10-06T05:14:25.745372+0000","last_active":"2023-10-06T05:14:25.746536+0000","last_peered":"2023-10-06T05:14:25.746536+0000","last_clean":"2023-10-06T05:14:25.746536+0000","last_became_active":"2023-10-06T05:14:25.745163+0000","last_became_peered":"2023-10-06T05:14:25.745163+0000","last_unstale":"2023-10-06T05:14:25.746536+0000","last_undegraded":"2023-10-06T05:14:25.746536+0000","last_fullsized":"2023-10-06T05:14:25.746536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:23.860941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.719614+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:14:25.719614+0000","last_peered":"2023-10-06T05:14:25.719614+0000","last_clean":"2023-10-06T05:14:25.719614+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:14:25.719614+0000","last_undegraded":"2023-10-06T05:14:25.719614+0000","last_fullsized":"2023-10-06T05:14:25.719614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.712654+0000","last_change":"2023-10-06T05:14:24.736663+0000","last_active":"2023-10-06T05:14:25.712654+0000","last_peered":"2023-10-06T05:14:25.712654+0000","last_clean":"2023-10-06T05:14:25.712654+0000","last_became_active":"2023-10-06T05:14:24.736241+0000","last_became_peered":"2023-10-06T05:14:24.736241+0000","last_unstale":"2023-10-06T05:14:25.712654+0000","last_undegraded":"2023-10-06T05:14:25.712654+0000","last_fullsized":"2023-10-06T05:14:25.712654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.736890+0000","last_change":"2023-10-06T05:14:25.729949+0000","last_active":"2023-10-06T05:14:25.736890+0000","last_peered":"2023-10-06T05:14:25.736890+0000","last_clean":"2023-10-06T05:14:25.736890+0000","last_became_active":"2023-10-06T05:14:25.729600+0000","last_became_peered":"2023-10-06T05:14:25.729600+0000","last_unstale":"2023-10-06T05:14:25.736890+0000","last_undegraded":"2023-10-06T05:14:25.736890+0000","last_fullsized":"2023-10-06T05:14:25.736890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.723259+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:14:25.723259+0000","last_peered":"2023-10-06T05:14:25.723259+0000","last_clean":"2023-10-06T05:14:25.723259+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:14:25.723259+0000","last_undegraded":"2023-10-06T05:14:25.723259+0000","last_fullsized":"2023-10-06T05:14:25.723259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713950+0000","last_change":"2023-10-06T05:14:24.724312+0000","last_active":"2023-10-06T05:14:25.713950+0000","last_peered":"2023-10-06T05:14:25.713950+0000","last_clean":"2023-10-06T05:14:25.713950+0000","last_became_active":"2023-10-06T05:14:24.724011+0000","last_became_peered":"2023-10-06T05:14:24.724011+0000","last_unstale":"2023-10-06T05:14:25.713950+0000","last_undegraded":"2023-10-06T05:14:25.713950+0000","last_fullsized":"2023-10-06T05:14:25.713950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735842+0000","last_change":"2023-10-06T05:14:25.729128+0000","last_active":"2023-10-06T05:14:25.735842+0000","last_peered":"2023-10-06T05:14:25.735842+0000","last_clean":"2023-10-06T05:14:25.735842+0000","last_became_active":"2023-10-06T05:14:25.728715+0000","last_became_peered":"2023-10-06T05:14:25.728715+0000","last_unstale":"2023-10-06T05:14:25.735842+0000","last_undegraded":"2023-10-06T05:14:25.735842+0000","last_fullsized":"2023-10-06T05:14:25.735842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.727798+0000","last_change":"2023-10-06T05:14:24.721869+0000","last_active":"2023-10-06T05:14:25.727798+0000","last_peered":"2023-10-06T05:14:25.727798+0000","last_clean":"2023-10-06T05:14:25.727798+0000","last_became_active":"2023-10-06T05:14:24.719902+0000","last_became_peered":"2023-10-06T05:14:24.719902+0000","last_unstale":"2023-10-06T05:14:25.727798+0000","last_undegraded":"2023-10-06T05:14:25.727798+0000","last_fullsized":"2023-10-06T05:14:25.727798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:18.957293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.713213+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:14:25.713213+0000","last_peered":"2023-10-06T05:14:25.713213+0000","last_clean":"2023-10-06T05:14:25.713213+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:14:25.713213+0000","last_undegraded":"2023-10-06T05:14:25.713213+0000","last_fullsized":"2023-10-06T05:14:25.713213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735685+0000","last_change":"2023-10-06T05:14:25.727038+0000","last_active":"2023-10-06T05:14:25.735685+0000","last_peered":"2023-10-06T05:14:25.735685+0000","last_clean":"2023-10-06T05:14:25.735685+0000","last_became_active":"2023-10-06T05:14:25.726717+0000","last_became_peered":"2023-10-06T05:14:25.726717+0000","last_unstale":"2023-10-06T05:14:25.735685+0000","last_undegraded":"2023-10-06T05:14:25.735685+0000","last_fullsized":"2023-10-06T05:14:25.735685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:50.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.709571+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:14:25.709571+0000","last_peered":"2023-10-06T05:14:25.709571+0000","last_clean":"2023-10-06T05:14:25.709571+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:14:25.709571+0000","last_undegraded":"2023-10-06T05:14:25.709571+0000","last_fullsized":"2023-10-06T05:14:25.709571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:14:26.957001+0000","last_change":"2023-10-06T05:14:24.717582+0000","last_active":"2023-10-06T05:14:26.957001+0000","last_peered":"2023-10-06T05:14:26.957001+0000","last_clean":"2023-10-06T05:14:26.957001+0000","last_became_active":"2023-10-06T05:14:24.716871+0000","last_became_peered":"2023-10-06T05:14:24.716871+0000","last_unstale":"2023-10-06T05:14:26.957001+0000","last_undegraded":"2023-10-06T05:14:26.957001+0000","last_fullsized":"2023-10-06T05:14:26.957001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:15.211734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744258+0000","last_change":"2023-10-06T05:14:25.737874+0000","last_active":"2023-10-06T05:14:25.744258+0000","last_peered":"2023-10-06T05:14:25.744258+0000","last_clean":"2023-10-06T05:14:25.744258+0000","last_became_active":"2023-10-06T05:14:25.737569+0000","last_became_peered":"2023-10-06T05:14:25.737569+0000","last_unstale":"2023-10-06T05:14:25.744258+0000","last_undegraded":"2023-10-06T05:14:25.744258+0000","last_fullsized":"2023-10-06T05:14:25.744258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741554+0000","last_change":"2023-10-06T05:14:24.736575+0000","last_active":"2023-10-06T05:14:25.741554+0000","last_peered":"2023-10-06T05:14:25.741554+0000","last_clean":"2023-10-06T05:14:25.741554+0000","last_became_active":"2023-10-06T05:14:24.736221+0000","last_became_peered":"2023-10-06T05:14:24.736221+0000","last_unstale":"2023-10-06T05:14:25.741554+0000","last_undegraded":"2023-10-06T05:14:25.741554+0000","last_fullsized":"2023-10-06T05:14:25.741554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:47.796269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.724256+0000","last_change":"2023-10-06T05:14:25.719862+0000","last_active":"2023-10-06T05:14:25.724256+0000","last_peered":"2023-10-06T05:14:25.724256+0000","last_clean":"2023-10-06T05:14:25.724256+0000","last_became_active":"2023-10-06T05:14:25.719558+0000","last_became_peered":"2023-10-06T05:14:25.719558+0000","last_unstale":"2023-10-06T05:14:25.724256+0000","last_undegraded":"2023-10-06T05:14:25.724256+0000","last_fullsized":"2023-10-06T05:14:25.724256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:41.453897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742798+0000","last_change":"2023-10-06T05:14:24.734397+0000","last_active":"2023-10-06T05:14:25.742798+0000","last_peered":"2023-10-06T05:14:25.742798+0000","last_clean":"2023-10-06T05:14:25.742798+0000","last_became_active":"2023-10-06T05:14:24.734112+0000","last_became_peered":"2023-10-06T05:14:24.734112+0000","last_unstale":"2023-10-06T05:14:25.742798+0000","last_undegraded":"2023-10-06T05:14:25.742798+0000","last_fullsized":"2023-10-06T05:14:25.742798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:50.734968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.741866+0000","last_change":"2023-10-06T05:14:25.736354+0000","last_active":"2023-10-06T05:14:25.741866+0000","last_peered":"2023-10-06T05:14:25.741866+0000","last_clean":"2023-10-06T05:14:25.741866+0000","last_became_active":"2023-10-06T05:14:25.735695+0000","last_became_peered":"2023-10-06T05:14:25.735695+0000","last_unstale":"2023-10-06T05:14:25.741866+0000","last_undegraded":"2023-10-06T05:14:25.741866+0000","last_fullsized":"2023-10-06T05:14:25.741866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:43.562251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.738237+0000","last_change":"2023-10-06T05:14:24.733470+0000","last_active":"2023-10-06T05:14:25.738237+0000","last_peered":"2023-10-06T05:14:25.738237+0000","last_clean":"2023-10-06T05:14:25.738237+0000","last_became_active":"2023-10-06T05:14:24.733189+0000","last_became_peered":"2023-10-06T05:14:24.733189+0000","last_unstale":"2023-10-06T05:14:25.738237+0000","last_undegraded":"2023-10-06T05:14:25.738237+0000","last_fullsized":"2023-10-06T05:14:25.738237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:40.866431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.747308+0000","last_change":"2023-10-06T05:14:25.744257+0000","last_active":"2023-10-06T05:14:25.747308+0000","last_peered":"2023-10-06T05:14:25.747308+0000","last_clean":"2023-10-06T05:14:25.747308+0000","last_became_active":"2023-10-06T05:14:25.743800+0000","last_became_peered":"2023-10-06T05:14:25.743800+0000","last_unstale":"2023-10-06T05:14:25.747308+0000","last_undegraded":"2023-10-06T05:14:25.747308+0000","last_fullsized":"2023-10-06T05:14:25.747308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:25.269880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.737149+0000","last_change":"2023-10-06T05:14:24.719523+0000","last_active":"2023-10-06T05:14:25.737149+0000","last_peered":"2023-10-06T05:14:25.737149+0000","last_clean":"2023-10-06T05:14:25.737149+0000","last_became_active":"2023-10-06T05:14:24.717215+0000","last_became_peered":"2023-10-06T05:14:24.717215+0000","last_unstale":"2023-10-06T05:14:25.737149+0000","last_undegraded":"2023-10-06T05:14:25.737149+0000","last_fullsized":"2023-10-06T05:14:25.737149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.791052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733084+0000","last_change":"2023-10-06T05:14:25.726409+0000","last_active":"2023-10-06T05:14:25.733084+0000","last_peered":"2023-10-06T05:14:25.733084+0000","last_clean":"2023-10-06T05:14:25.733084+0000","last_became_active":"2023-10-06T05:14:25.725760+0000","last_became_peered":"2023-10-06T05:14:25.725760+0000","last_unstale":"2023-10-06T05:14:25.733084+0000","last_undegraded":"2023-10-06T05:14:25.733084+0000","last_fullsized":"2023-10-06T05:14:25.733084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.054887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.730428+0000","last_change":"2023-10-06T05:14:24.724643+0000","last_active":"2023-10-06T05:14:25.730428+0000","last_peered":"2023-10-06T05:14:25.730428+0000","last_clean":"2023-10-06T05:14:25.730428+0000","last_became_active":"2023-10-06T05:14:24.724364+0000","last_became_peered":"2023-10-06T05:14:24.724364+0000","last_unstale":"2023-10-06T05:14:25.730428+0000","last_undegraded":"2023-10-06T05:14:25.730428+0000","last_fullsized":"2023-10-06T05:14:25.730428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:23.686143+0000","last_clean_scrub_stamp":"2023-10-06T05:14:23.686143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:36.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.742656+0000","last_change":"2023-10-06T05:14:25.741519+0000","last_active":"2023-10-06T05:14:25.742656+0000","last_peered":"2023-10-06T05:14:25.742656+0000","last_clean":"2023-10-06T05:14:25.742656+0000","last_became_active":"2023-10-06T05:14:25.741100+0000","last_became_peered":"2023-10-06T05:14:25.741100+0000","last_unstale":"2023-10-06T05:14:25.742656+0000","last_undegraded":"2023-10-06T05:14:25.742656+0000","last_fullsized":"2023-10-06T05:14:25.742656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:37.716944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.728144+0000","last_change":"2023-10-06T05:14:25.721386+0000","last_active":"2023-10-06T05:14:25.728144+0000","last_peered":"2023-10-06T05:14:25.728144+0000","last_clean":"2023-10-06T05:14:25.728144+0000","last_became_active":"2023-10-06T05:14:25.720975+0000","last_became_peered":"2023-10-06T05:14:25.720975+0000","last_unstale":"2023-10-06T05:14:25.728144+0000","last_undegraded":"2023-10-06T05:14:25.728144+0000","last_fullsized":"2023-10-06T05:14:25.728144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:51.743327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.733104+0000","last_change":"2023-10-06T05:14:25.726604+0000","last_active":"2023-10-06T05:14:25.733104+0000","last_peered":"2023-10-06T05:14:25.733104+0000","last_clean":"2023-10-06T05:14:25.733104+0000","last_became_active":"2023-10-06T05:14:25.726075+0000","last_became_peered":"2023-10-06T05:14:25.726075+0000","last_unstale":"2023-10-06T05:14:25.733104+0000","last_undegraded":"2023-10-06T05:14:25.733104+0000","last_fullsized":"2023-10-06T05:14:25.733104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:54.241490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.744829+0000","last_change":"2023-10-06T05:14:25.743617+0000","last_active":"2023-10-06T05:14:25.744829+0000","last_peered":"2023-10-06T05:14:25.744829+0000","last_clean":"2023-10-06T05:14:25.744829+0000","last_became_active":"2023-10-06T05:14:25.741960+0000","last_became_peered":"2023-10-06T05:14:25.741960+0000","last_unstale":"2023-10-06T05:14:25.744829+0000","last_undegraded":"2023-10-06T05:14:25.744829+0000","last_fullsized":"2023-10-06T05:14:25.744829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:50.792266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.735674+0000","last_change":"2023-10-06T05:14:25.728085+0000","last_active":"2023-10-06T05:14:25.735674+0000","last_peered":"2023-10-06T05:14:25.735674+0000","last_clean":"2023-10-06T05:14:25.735674+0000","last_became_active":"2023-10-06T05:14:25.727721+0000","last_became_peered":"2023-10-06T05:14:25.727721+0000","last_unstale":"2023-10-06T05:14:25.735674+0000","last_undegraded":"2023-10-06T05:14:25.735674+0000","last_fullsized":"2023-10-06T05:14:25.735674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.287776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T05:14:25.739584+0000","last_change":"2023-10-06T05:14:25.735959+0000","last_active":"2023-10-06T05:14:25.739584+0000","last_peered":"2023-10-06T05:14:25.739584+0000","last_clean":"2023-10-06T05:14:25.739584+0000","last_became_active":"2023-10-06T05:14:25.735602+0000","last_became_peered":"2023-10-06T05:14:25.735602+0000","last_unstale":"2023-10-06T05:14:25.739584+0000","last_undegraded":"2023-10-06T05:14:25.739584+0000","last_fullsized":"2023-10-06T05:14:25.739584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:24.694503+0000","last_clean_scrub_stamp":"2023-10-06T05:14:24.694503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.263376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7349999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6520000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7639999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:14:36.196 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:14:36.196 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T05:14:36.196 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T05:14:36.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:14:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.402+0000 7f78ecb9d700 1 Processor -- start 2023-10-06T05:14:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.402+0000 7f78ecb9d700 1 -- start start 2023-10-06T05:14:36.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78ecb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78e806e140 0x7f78e815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78ecb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e80afdd0 0x7f78e80b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78ecb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e806f220 con 0x7f78e8159530 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78ecb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e80b1e00 con 0x7f78e806e140 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78ecb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e806f6b0 con 0x7f78e80afdd0 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78e77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f78e8159530 0x7f78e806da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39094/0 (socket says 172.21.15.33:39094) 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78e77fe700 1 -- 172.21.15.33:0/2074383391 learned_addr learned my addr 172.21.15.33:0/2074383391 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e80afdd0 0x7f78e80b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2074383391 >> v2:172.21.15.167:3300/0 conn(0x7f78e806e140 0x7f78e815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581689859 0 0) 0x7f78e806f220 con 0x7f78e8159530 2023-10-06T05:14:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.404+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d0003570 con 0x7f78e8159530 2023-10-06T05:14:36.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1240280337 0 0) 0x7f78d0003570 con 0x7f78e8159530 2023-10-06T05:14:36.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e80afdd0 msgr2=0x7f78e80b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e80afdd0 0x7f78e80b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 >> v2:172.21.15.167:3300/0 conn(0x7f78e806e140 msgr2=0x7f78e815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 --2- 172.21.15.33:0/2074383391 >> v2:172.21.15.167:3300/0 conn(0x7f78e806e140 0x7f78e815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e815ae50 con 0x7f78e8159530 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f78d8001470 con 0x7f78e8159530 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 >> v1:172.21.15.33:6789/0 conn(0x7f78e8159530 legacy=0x7f78e806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78d0003ae0 0x7f78d0003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78d0004810 0x7f78d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d00051c0 0x7f78d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80b1e00 con 0x7f78d00051c0 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e806f6b0 con 0x7f78d0003ae0 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d000bfd0 con 0x7f78d0004810 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e80afdd0 0x7f78e80b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2074383391 >> v2:172.21.15.167:3300/0 conn(0x7f78e806e140 0x7f78e815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d00051c0 0x7f78d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e77fe700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78d0003ae0 0x7f78d0003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.405+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78d0004810 0x7f78d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e7fff700 1 -- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78d0004810 msgr2=0x7f78d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78d0004810 0x7f78d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e7fff700 1 -- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78d0003ae0 msgr2=0x7f78d0003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78d0003ae0 0x7f78d0003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e7fff700 1 -- 172.21.15.33:0/2074383391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0003570 con 0x7f78d00051c0 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78d0004810 0x7f78d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d00051c0 0x7f78d000ba60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f78e80b01a0 tx=0x7f78e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21d5eeb04137b6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000ea40 con 0x7f78d00051c0 2023-10-06T05:14:36.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e000f040 con 0x7f78d00051c0 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.406+0000 7f78e67fc700 1 -- 172.21.15.33:0/2074383391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000edd0 con 0x7f78d00051c0 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2074383391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d00051c0 msgr2=0x7f78d000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2074383391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d00051c0 0x7f78d000ba60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f78e80b01a0 tx=0x7f78e000b040 comp rx=0 tx=0).stop 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78e7fff700 1 -- 172.21.15.33:0/2074383391 reap_dead start 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2074383391 shutdown_connections 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2074383391 >> 172.21.15.33:0/2074383391 conn(0x7f78e806c560 msgr2=0x7f78e806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2074383391 shutdown_connections 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.407+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2074383391 wait complete. 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.408+0000 7f78ecb9d700 1 Processor -- start 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.408+0000 7f78ecb9d700 1 -- start start 2023-10-06T05:14:36.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78ecb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e814d310 0x7f78e81698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78ecb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 0x7f78e815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78ecb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e814d860 0x7f78e814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78ecb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e815bf70 con 0x7f78e815b240 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78ecb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e815c250 con 0x7f78e814d310 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78ecb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e815c530 con 0x7f78e814d860 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 0x7f78e815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 0x7f78e815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37728/0 (socket says 172.21.15.33:37728) 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.409+0000 7f78e6ffd700 1 -- 172.21.15.33:0/2005977543 learned_addr learned my addr 172.21.15.33:0/2005977543 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e814d860 0x7f78e814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e77fe700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e814d310 0x7f78e81698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e6ffd700 1 -- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e814d860 msgr2=0x7f78e814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e814d860 0x7f78e814dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e6ffd700 1 -- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e814d310 msgr2=0x7f78e81698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e814d310 0x7f78e81698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e6ffd700 1 -- 172.21.15.33:0/2005977543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e815c810 con 0x7f78e815b240 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e7fff700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e814d860 0x7f78e814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e77fe700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e814d310 0x7f78e81698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78e6ffd700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 0x7f78e815b6b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f78dc011990 tx=0x7f78dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00f970 con 0x7f78e815b240 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc00fad0 con 0x7f78e815b240 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00fe50 con 0x7f78e815b240 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e815caf0 con 0x7f78e815b240 2023-10-06T05:14:36.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.410+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e814e220 con 0x7f78e815b240 2023-10-06T05:14:36.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.411+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e804c380 con 0x7f78e815b240 2023-10-06T05:14:36.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.412+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f78dc015670 con 0x7f78e815b240 2023-10-06T05:14:36.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.413+0000 7f78cf7fe700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f78d4066f60 0x7f78d4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.413+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f78dc084d70 con 0x7f78e815b240 2023-10-06T05:14:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.414+0000 7f78e77fe700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f78d4066f60 0x7f78d4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.414+0000 7f78e77fe700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f78d4066f60 0x7f78d4069410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78d8000e10 tx=0x7f78d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.417+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78dc051e90 con 0x7f78e815b240 2023-10-06T05:14:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.660+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f78e806e610 con 0x7f78e815b240 2023-10-06T05:14:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.660+0000 7f78cf7fe700 1 -- 172.21.15.33:0/2005977543 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f78dc022040 con 0x7f78e815b240 2023-10-06T05:14:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:36.664 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:14:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f78d4066f60 msgr2=0x7f78d4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f78d4066f60 0x7f78d4069410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78d8000e10 tx=0x7f78d8009040 comp rx=0 tx=0).stop 2023-10-06T05:14:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 msgr2=0x7f78e815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 0x7f78e815b6b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f78dc011990 tx=0x7f78dc008040 comp rx=0 tx=0).stop 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 shutdown_connections 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f78d4066f60 0x7f78d4069410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e814d860 0x7f78e814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e814d310 0x7f78e81698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 --2- 172.21.15.33:0/2005977543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78e815b240 0x7f78e815b6b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.662+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 >> 172.21.15.33:0/2005977543 conn(0x7f78e806c560 msgr2=0x7f78e80b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.663+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 shutdown_connections 2023-10-06T05:14:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.663+0000 7f78ecb9d700 1 -- 172.21.15.33:0/2005977543 wait complete. 2023-10-06T05:14:36.682 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T05:14:36.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.868+0000 7f354bfff700 1 Processor -- start 2023-10-06T05:14:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.868+0000 7f354bfff700 1 -- start start 2023-10-06T05:14:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 0x7f354c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f354c0b04e0 0x7f354c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c06e3e0 con 0x7f354c06df70 2023-10-06T05:14:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c0b1e00 con 0x7f354c0b04e0 2023-10-06T05:14:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c06f6b0 con 0x7f354c159530 2023-10-06T05:14:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354a7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f354c06df70 0x7f354c0afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39108/0 (socket says 172.21.15.33:39108) 2023-10-06T05:14:36.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354a7fc700 1 -- 172.21.15.33:0/2220390435 learned_addr learned my addr 172.21.15.33:0/2220390435 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855400508 0 0) 0x7f354c06e3e0 con 0x7f354c06df70 2023-10-06T05:14:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354b7fe700 1 --2- 172.21.15.33:0/2220390435 >> v2:172.21.15.167:3300/0 conn(0x7f354c0b04e0 0x7f354c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f354affd700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 0x7f354c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.870+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3534003570 con 0x7f354c06df70 2023-10-06T05:14:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3174352241 0 0) 0x7f3534003570 con 0x7f354c06df70 2023-10-06T05:14:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 msgr2=0x7f354c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 0x7f354c06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 >> v2:172.21.15.167:3300/0 conn(0x7f354c0b04e0 msgr2=0x7f354c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 --2- 172.21.15.33:0/2220390435 >> v2:172.21.15.167:3300/0 conn(0x7f354c0b04e0 0x7f354c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c06f220 con 0x7f354c06df70 2023-10-06T05:14:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3540001470 con 0x7f354c06df70 2023-10-06T05:14:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 >> v1:172.21.15.33:6789/0 conn(0x7f354c06df70 legacy=0x7f354c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3534003ae0 0x7f3534003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534004810 0x7f3534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 0x7f35340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0b1e00 con 0x7f3534004810 2023-10-06T05:14:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c06f6b0 con 0x7f3534003ae0 2023-10-06T05:14:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353400bb10 con 0x7f354c159530 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f354a7fc700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534004810 0x7f3534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f354affd700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3534003ae0 0x7f3534003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f354b7fe700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 0x7f35340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f354affd700 1 -- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 msgr2=0x7f35340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.871+0000 7f354affd700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c159530 0x7f35340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f354affd700 1 -- 172.21.15.33:0/2220390435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534004810 msgr2=0x7f3534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f354affd700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534004810 0x7f3534004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f354affd700 1 -- 172.21.15.33:0/2220390435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534003570 con 0x7f3534003ae0 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f354a7fc700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534004810 0x7f3534004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f354affd700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3534003ae0 0x7f3534003ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f353c009c00 tx=0x7f353c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92395cb01f053c68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c00e710 con 0x7f3534003ae0 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c00e870 con 0x7f3534003ae0 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.872+0000 7f3549ffb700 1 -- 172.21.15.33:0/2220390435 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c00ec30 con 0x7f3534003ae0 2023-10-06T05:14:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354bfff700 1 -- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3534003ae0 msgr2=0x7f3534003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354bfff700 1 --2- 172.21.15.33:0/2220390435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3534003ae0 0x7f3534003ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f353c009c00 tx=0x7f353c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354b7fe700 1 -- 172.21.15.33:0/2220390435 reap_dead start 2023-10-06T05:14:36.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354bfff700 1 -- 172.21.15.33:0/2220390435 shutdown_connections 2023-10-06T05:14:36.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354bfff700 1 -- 172.21.15.33:0/2220390435 >> 172.21.15.33:0/2220390435 conn(0x7f354c06c560 msgr2=0x7f354c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:36.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354bfff700 1 -- 172.21.15.33:0/2220390435 shutdown_connections 2023-10-06T05:14:36.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.873+0000 7f354bfff700 1 -- 172.21.15.33:0/2220390435 wait complete. 2023-10-06T05:14:36.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.874+0000 7f354bfff700 1 Processor -- start 2023-10-06T05:14:36.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.874+0000 7f354bfff700 1 -- start start 2023-10-06T05:14:36.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 0x7f354c140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f354c14d770 0x7f354c169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c14dbe0 0x7f354c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c141400 con 0x7f3534003ae0 2023-10-06T05:14:36.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c16a3f0 con 0x7f354c14d770 2023-10-06T05:14:36.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c16a6d0 con 0x7f354c14dbe0 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 0x7f354c140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 0x7f354c140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37750/0 (socket says 172.21.15.33:37750) 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.875+0000 7f354affd700 1 -- 172.21.15.33:0/3801323122 learned_addr learned my addr 172.21.15.33:0/3801323122 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354a7fc700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f354c14d770 0x7f354c169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354affd700 1 -- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c14dbe0 msgr2=0x7f354c15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354affd700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c14dbe0 0x7f354c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354affd700 1 -- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f354c14d770 msgr2=0x7f354c169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354affd700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f354c14d770 0x7f354c169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354affd700 1 -- 172.21.15.33:0/3801323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c15b170 con 0x7f3534003ae0 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354a7fc700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f354c14d770 0x7f354c169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354affd700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 0x7f354c140ec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f353400bae0 tx=0x7f353c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c022070 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c00ebf0 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c01c6a0 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.876+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c15b450 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.877+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c14d260 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.877+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c16a910 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.878+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f353c01ca50 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.879+0000 7f3532ffd700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.880+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f353c085fb0 con 0x7f3534003ae0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.880+0000 7f354a7fc700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.880+0000 7f354a7fc700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3538066f30 0x7f35380693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3540001cb0 tx=0x7f3540009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:36.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:36.883+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f353c053020 con 0x7f3534003ae0 2023-10-06T05:14:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.087+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f354c04c380 con 0x7f3534003ae0 2023-10-06T05:14:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.088+0000 7f3532ffd700 1 -- 172.21.15.33:0/3801323122 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f353c02b090 con 0x7f3534003ae0 2023-10-06T05:14:37.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:37.091 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3538066f30 msgr2=0x7f35380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3538066f30 0x7f35380693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3540001cb0 tx=0x7f3540009040 comp rx=0 tx=0).stop 2023-10-06T05:14:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 msgr2=0x7f354c140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 0x7f354c140ec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f353400bae0 tx=0x7f353c01a000 comp rx=0 tx=0).stop 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 shutdown_connections 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f354c14dbe0 0x7f354c15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3534003ae0 0x7f354c140ec0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 --2- 172.21.15.33:0/3801323122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f354c14d770 0x7f354c169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 >> 172.21.15.33:0/3801323122 conn(0x7f354c06c560 msgr2=0x7f354c155800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 shutdown_connections 2023-10-06T05:14:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.090+0000 7f354bfff700 1 -- 172.21.15.33:0/3801323122 wait complete. 2023-10-06T05:14:37.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:14:37.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.298+0000 7f3308f9d700 1 Processor -- start 2023-10-06T05:14:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.298+0000 7f3308f9d700 1 -- start start 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.299+0000 7f3308f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33040af540 0x7f33040af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.299+0000 7f3308f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33040cd5e0 0x7f330415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.299+0000 7f3308f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330415d660 con 0x7f330415a650 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.299+0000 7f3308f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33040ce890 con 0x7f33040af540 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.299+0000 7f3308f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330415aac0 con 0x7f33040cd5e0 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33040af540 0x7f33040af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f3302ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f330415a650 0x7f33040cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39124/0 (socket says 172.21.15.33:39124) 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33040af540 0x7f33040af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37752/0 (socket says 172.21.15.33:37752) 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f3302ffd700 1 -- 172.21.15.33:0/3378138005 learned_addr learned my addr 172.21.15.33:0/3378138005 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f3303fff700 1 --2- 172.21.15.33:0/3378138005 >> v2:172.21.15.167:3300/0 conn(0x7f33040cd5e0 0x7f330415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 -- 172.21.15.33:0/3378138005 >> v2:172.21.15.167:3300/0 conn(0x7f33040cd5e0 msgr2=0x7f330415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 --2- 172.21.15.33:0/3378138005 >> v2:172.21.15.167:3300/0 conn(0x7f33040cd5e0 0x7f330415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 -- 172.21.15.33:0/3378138005 >> v1:172.21.15.33:6789/0 conn(0x7f330415a650 legacy=0x7f33040cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 -- 172.21.15.33:0/3378138005 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330415d890 con 0x7f33040af540 2023-10-06T05:14:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33027fc700 1 -- 172.21.15.33:0/3378138005 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288656031 0 0) 0x7f330415d660 con 0x7f330415a650 2023-10-06T05:14:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.300+0000 7f33037fe700 1 --2- 172.21.15.33:0/3378138005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33040af540 0x7f33040af910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33040ce1c0 tx=0x7f32f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a84172b02452d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f33027fc700 1 -- 172.21.15.33:0/3378138005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4011630 con 0x7f33040af540 2023-10-06T05:14:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f33027fc700 1 -- 172.21.15.33:0/3378138005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4011790 con 0x7f33040af540 2023-10-06T05:14:37.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f33027fc700 1 -- 172.21.15.33:0/3378138005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4011b30 con 0x7f33040af540 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 -- 172.21.15.33:0/3378138005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33040af540 msgr2=0x7f33040af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 --2- 172.21.15.33:0/3378138005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33040af540 0x7f33040af910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33040ce1c0 tx=0x7f32f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 -- 172.21.15.33:0/3378138005 shutdown_connections 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 --2- 172.21.15.33:0/3378138005 >> v2:172.21.15.167:3300/0 conn(0x7f33040cd5e0 0x7f330415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 --2- 172.21.15.33:0/3378138005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33040af540 0x7f33040af910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 -- 172.21.15.33:0/3378138005 >> 172.21.15.33:0/3378138005 conn(0x7f330406d9b0 msgr2=0x7f330406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 -- 172.21.15.33:0/3378138005 shutdown_connections 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.301+0000 7f3308f9d700 1 -- 172.21.15.33:0/3378138005 wait complete. 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.303+0000 7f3308f9d700 1 Processor -- start 2023-10-06T05:14:37.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.303+0000 7f3308f9d700 1 -- start start 2023-10-06T05:14:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3308f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33040af540 0x7f33041551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3308f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040cd5e0 0x7f33040ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3308f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 0x7f33040ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3308f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3304155720 con 0x7f330415a650 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3308f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330416ac80 con 0x7f33040af540 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3308f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330416a070 con 0x7f33040cd5e0 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3303fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 0x7f33040ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3303fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 0x7f33040ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37756/0 (socket says 172.21.15.33:37756) 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3303fff700 1 -- 172.21.15.33:0/806709159 learned_addr learned my addr 172.21.15.33:0/806709159 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f33037fe700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33040af540 0x7f33041551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3303fff700 1 -- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040cd5e0 msgr2=0x7f33040ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.304+0000 7f3302ffd700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040cd5e0 0x7f33040ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3303fff700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040cd5e0 0x7f33040ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3303fff700 1 -- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33040af540 msgr2=0x7f33041551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3303fff700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33040af540 0x7f33041551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3303fff700 1 -- 172.21.15.33:0/806709159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f4009420 con 0x7f330415a650 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f33037fe700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33040af540 0x7f33041551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3302ffd700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040cd5e0 0x7f33040ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3303fff700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 0x7f33040ced20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009d70 tx=0x7f32fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00cb50 con 0x7f330415a650 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc00f040 con 0x7f330415a650 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc014480 con 0x7f330415a650 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330416a350 con 0x7f330415a650 2023-10-06T05:14:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.305+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330416a910 con 0x7f330415a650 2023-10-06T05:14:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.306+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330404c380 con 0x7f330415a650 2023-10-06T05:14:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.307+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32fc014830 con 0x7f330415a650 2023-10-06T05:14:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.308+0000 7f32eb7fe700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32f0066f30 0x7f32f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.308+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32fc0853b0 con 0x7f330415a650 2023-10-06T05:14:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.308+0000 7f33037fe700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32f0066f30 0x7f32f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.309+0000 7f33037fe700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32f0066f30 0x7f32f00693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32f4003080 tx=0x7f32f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.312+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32fc052420 con 0x7f330415a650 2023-10-06T05:14:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.514+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f330415db70 con 0x7f330415a650 2023-10-06T05:14:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.515+0000 7f32eb7fe700 1 -- 172.21.15.33:0/806709159 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32fc055a40 con 0x7f330415a650 2023-10-06T05:14:37.517 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:14:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32f0066f30 msgr2=0x7f32f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32f0066f30 0x7f32f00693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32f4003080 tx=0x7f32f4017000 comp rx=0 tx=0).stop 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 msgr2=0x7f33040ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 0x7f33040ced20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009d70 tx=0x7f32fc00b040 comp rx=0 tx=0).stop 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 shutdown_connections 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32f0066f30 0x7f32f00693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330415a650 0x7f33040ced20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33040af540 0x7f33041551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 --2- 172.21.15.33:0/806709159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040cd5e0 0x7f33040ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 >> 172.21.15.33:0/806709159 conn(0x7f330406d9b0 msgr2=0x7f33040c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 shutdown_connections 2023-10-06T05:14:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.517+0000 7f3308f9d700 1 -- 172.21.15.33:0/806709159 wait complete. 2023-10-06T05:14:37.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:37.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.723+0000 7f9957fff700 1 Processor -- start 2023-10-06T05:14:37.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.723+0000 7f9957fff700 1 -- start start 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9957fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99581552f0 0x7f99580ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9957fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f99580b18f0 0x7f99580cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9957fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99581586a0 con 0x7f9958154f20 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9957fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995806bdf0 con 0x7f99580b18f0 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9957fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9958157ab0 con 0x7f99581552f0 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9956ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9958154f20 0x7f99580b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39128/0 (socket says 172.21.15.33:39128) 2023-10-06T05:14:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f9956ffd700 1 -- 172.21.15.33:0/1856142142 learned_addr learned my addr 172.21.15.33:0/1856142142 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f99577fe700 1 --2- 172.21.15.33:0/1856142142 >> v2:172.21.15.167:3300/0 conn(0x7f99580b18f0 0x7f99580cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.724+0000 7f99567fc700 1 --2- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99581552f0 0x7f99580ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f9955ffb700 1 -- 172.21.15.33:0/1856142142 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551000626 0 0) 0x7f99581586a0 con 0x7f9958154f20 2023-10-06T05:14:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f9955ffb700 1 -- 172.21.15.33:0/1856142142 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9940003570 con 0x7f9958154f20 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f99577fe700 1 -- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99581552f0 msgr2=0x7f99580ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f99577fe700 1 --2- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99581552f0 0x7f99580ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f9955ffb700 1 -- 172.21.15.33:0/1856142142 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3505608405 0 0) 0x7f9940003570 con 0x7f9958154f20 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f99577fe700 1 -- 172.21.15.33:0/1856142142 >> v1:172.21.15.33:6789/0 conn(0x7f9958154f20 legacy=0x7f99580b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f99577fe700 1 -- 172.21.15.33:0/1856142142 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958157d70 con 0x7f99580b18f0 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.725+0000 7f99577fe700 1 --2- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580b18f0 0x7f99580cc7e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9950009e60 tx=0x7f9950009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff39c1145496a2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9955ffb700 1 -- 172.21.15.33:0/1856142142 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000e070 con 0x7f99580b18f0 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9955ffb700 1 -- 172.21.15.33:0/1856142142 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000b7a0 con 0x7f99580b18f0 2023-10-06T05:14:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9955ffb700 1 -- 172.21.15.33:0/1856142142 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948001470 con 0x7f99580b18f0 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 -- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580b18f0 msgr2=0x7f99580cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 --2- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580b18f0 0x7f99580cc7e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9950009e60 tx=0x7f9950009780 comp rx=0 tx=0).stop 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 -- 172.21.15.33:0/1856142142 shutdown_connections 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 --2- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580b18f0 0x7f99580cc7e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 --2- 172.21.15.33:0/1856142142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99581552f0 0x7f99580ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 -- 172.21.15.33:0/1856142142 >> 172.21.15.33:0/1856142142 conn(0x7f995806d030 msgr2=0x7f99580cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 -- 172.21.15.33:0/1856142142 shutdown_connections 2023-10-06T05:14:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.726+0000 7f9957fff700 1 -- 172.21.15.33:0/1856142142 wait complete. 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.728+0000 7f9957fff700 1 Processor -- start 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.728+0000 7f9957fff700 1 -- start start 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9957fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 0x7f995816a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9957fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9958154f20 0x7f99580b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9957fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995814b9a0 0x7f9958156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9957fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99580b1260 con 0x7f99580b18f0 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9957fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99580c7d40 con 0x7f995814b9a0 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9957fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99580aff50 con 0x7f9958154f20 2023-10-06T05:14:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 0x7f995816a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 0x7f995816a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37758/0 (socket says 172.21.15.33:37758) 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 -- 172.21.15.33:0/907998894 learned_addr learned my addr 172.21.15.33:0/907998894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f99567fc700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9958154f20 0x7f99580b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f99577fe700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995814b9a0 0x7f9958156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 -- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9958154f20 msgr2=0x7f99580b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9958154f20 0x7f99580b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 -- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995814b9a0 msgr2=0x7f9958156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995814b9a0 0x7f9958156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.729+0000 7f9956ffd700 1 -- 172.21.15.33:0/907998894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950009a20 con 0x7f99580b18f0 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.730+0000 7f9956ffd700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 0x7f995816a890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f994800cbc0 tx=0x7f9948001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.730+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948007150 con 0x7f99580b18f0 2023-10-06T05:14:37.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.730+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99480072b0 con 0x7f99580b18f0 2023-10-06T05:14:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.730+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99580af340 con 0x7f99580b18f0 2023-10-06T05:14:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.730+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948007650 con 0x7f99580b18f0 2023-10-06T05:14:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.730+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99580af900 con 0x7f99580b18f0 2023-10-06T05:14:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.731+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995804c380 con 0x7f99580b18f0 2023-10-06T05:14:37.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.732+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9948004a40 con 0x7f99580b18f0 2023-10-06T05:14:37.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.733+0000 7f993effd700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9944066f30 0x7f99440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:37.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.733+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9948083e40 con 0x7f99580b18f0 2023-10-06T05:14:37.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.733+0000 7f99567fc700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9944066f30 0x7f99440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:37.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.734+0000 7f99567fc700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9944066f30 0x7f99440693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9958154470 tx=0x7f994c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:37.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.737+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9948050f30 con 0x7f99580b18f0 2023-10-06T05:14:37.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.940+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f995806f370 con 0x7f99580b18f0 2023-10-06T05:14:37.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.940+0000 7f993effd700 1 -- 172.21.15.33:0/907998894 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9948018020 con 0x7f99580b18f0 2023-10-06T05:14:37.943 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9944066f30 msgr2=0x7f99440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9944066f30 0x7f99440693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9958154470 tx=0x7f994c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 msgr2=0x7f995816a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 0x7f995816a890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f994800cbc0 tx=0x7f9948001a80 comp rx=0 tx=0).stop 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 shutdown_connections 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9944066f30 0x7f99440693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995814b9a0 0x7f9958156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580b18f0 0x7f995816a890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 --2- 172.21.15.33:0/907998894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9958154f20 0x7f99580b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 >> 172.21.15.33:0/907998894 conn(0x7f995806d030 msgr2=0x7f995815f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 shutdown_connections 2023-10-06T05:14:37.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:37.943+0000 7f9957fff700 1 -- 172.21.15.33:0/907998894 wait complete. 2023-10-06T05:14:37.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:14:38.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.150+0000 7fef7639d700 1 Processor -- start 2023-10-06T05:14:38.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.150+0000 7fef7639d700 1 -- start start 2023-10-06T05:14:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef7639d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fef700b0900 0x7fef700af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef7639d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700ad430 con 0x7fef700cc7e0 2023-10-06T05:14:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef7639d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700afe20 con 0x7fef70156370 2023-10-06T05:14:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef7639d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7006bc70 con 0x7fef700b0900 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef74b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fef700cc7e0 0x7fef700b01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39132/0 (socket says 172.21.15.33:39132) 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef74b9a700 1 -- 172.21.15.33:0/3611650130 learned_addr learned my addr 172.21.15.33:0/3611650130 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef75b9c700 1 --2- 172.21.15.33:0/3611650130 >> v2:172.21.15.167:3300/0 conn(0x7fef700b0900 0x7fef700af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371193739 0 0) 0x7fef700ad430 con 0x7fef700cc7e0 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.152+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef5c003570 con 0x7fef700cc7e0 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752106507 0 0) 0x7fef700afe20 con 0x7fef70156370 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef700ad430 con 0x7fef70156370 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2915473388 0 0) 0x7fef5c003570 con 0x7fef700cc7e0 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef75b9c700 1 -- 172.21.15.33:0/3611650130 >> v1:172.21.15.167:6789/0 conn(0x7fef70156370 legacy=0x7fef700c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef75b9c700 1 -- 172.21.15.33:0/3611650130 >> v1:172.21.15.33:6789/0 conn(0x7fef700cc7e0 legacy=0x7fef700b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef75b9c700 1 -- 172.21.15.33:0/3611650130 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700cccc0 con 0x7fef700b0900 2023-10-06T05:14:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef75b9c700 1 --2- 172.21.15.33:0/3611650130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef700b0900 0x7fef700af8e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef68009e60 tx=0x7fef68009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ffb32712982029 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800e070 con 0x7fef700b0900 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6800b7a0 con 0x7fef700b0900 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.153+0000 7fef67fff700 1 -- 172.21.15.33:0/3611650130 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60001470 con 0x7fef700b0900 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 -- 172.21.15.33:0/3611650130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef700b0900 msgr2=0x7fef700af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 --2- 172.21.15.33:0/3611650130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef700b0900 0x7fef700af8e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef68009e60 tx=0x7fef68009780 comp rx=0 tx=0).stop 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 -- 172.21.15.33:0/3611650130 shutdown_connections 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 --2- 172.21.15.33:0/3611650130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef700b0900 0x7fef700af8e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 -- 172.21.15.33:0/3611650130 >> 172.21.15.33:0/3611650130 conn(0x7fef7006d030 msgr2=0x7fef700cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 -- 172.21.15.33:0/3611650130 shutdown_connections 2023-10-06T05:14:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.154+0000 7fef7639d700 1 -- 172.21.15.33:0/3611650130 wait complete. 2023-10-06T05:14:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.155+0000 7fef7639d700 1 Processor -- start 2023-10-06T05:14:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.155+0000 7fef7639d700 1 -- start start 2023-10-06T05:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.156+0000 7fef7639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef700b0900 0x7fef7006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.156+0000 7fef7639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 0x7fef70153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.156+0000 7fef7639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef70156370 0x7fef700ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.156+0000 7fef7639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70153650 con 0x7fef700cc7e0 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.156+0000 7fef7639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701538f0 con 0x7fef70156370 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.156+0000 7fef7639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef70153bd0 con 0x7fef700b0900 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 0x7fef70153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 0x7fef70153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37774/0 (socket says 172.21.15.33:37774) 2023-10-06T05:14:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 -- 172.21.15.33:0/1814938054 learned_addr learned my addr 172.21.15.33:0/1814938054 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef75b9c700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef70156370 0x7fef700ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef7539b700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef700b0900 0x7fef7006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 -- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef700b0900 msgr2=0x7fef7006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef700b0900 0x7fef7006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 -- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef70156370 msgr2=0x7fef700ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef70156370 0x7fef700ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 -- 172.21.15.33:0/1814938054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009a20 con 0x7fef700cc7e0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef74b9a700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 0x7fef70153110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef6000cb70 tx=0x7fef60001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60001d00 con 0x7fef700cc7e0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60007360 con 0x7fef700cc7e0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60004830 con 0x7fef700cc7e0 2023-10-06T05:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.157+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef70153eb0 con 0x7fef700cc7e0 2023-10-06T05:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.158+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef70154470 con 0x7fef700cc7e0 2023-10-06T05:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.158+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7004c380 con 0x7fef700cc7e0 2023-10-06T05:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.159+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef60004ba0 con 0x7fef700cc7e0 2023-10-06T05:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.160+0000 7fef657fa700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef54066f30 0x7fef540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.160+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fef60083ff0 con 0x7fef700cc7e0 2023-10-06T05:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.161+0000 7fef7539b700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef54066f30 0x7fef540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.161+0000 7fef7539b700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef54066f30 0x7fef540693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef6c000c00 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.164+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef600510e0 con 0x7fef700cc7e0 2023-10-06T05:14:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.364+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef7006f310 con 0x7fef700cc7e0 2023-10-06T05:14:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.365+0000 7fef657fa700 1 -- 172.21.15.33:0/1814938054 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef600180a0 con 0x7fef700cc7e0 2023-10-06T05:14:38.367 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:14:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef54066f30 msgr2=0x7fef540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef54066f30 0x7fef540693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef6c000c00 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2023-10-06T05:14:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 msgr2=0x7fef70153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 0x7fef70153110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef6000cb70 tx=0x7fef60001ab0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 shutdown_connections 2023-10-06T05:14:38.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef54066f30 0x7fef540693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef70156370 0x7fef700ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef700b0900 0x7fef7006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 --2- 172.21.15.33:0/1814938054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef700cc7e0 0x7fef70153110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 >> 172.21.15.33:0/1814938054 conn(0x7fef7006d030 msgr2=0x7fef700aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:38.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 shutdown_connections 2023-10-06T05:14:38.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.367+0000 7fef7639d700 1 -- 172.21.15.33:0/1814938054 wait complete. 2023-10-06T05:14:38.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:14:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.575+0000 7fc2e4d9d700 1 Processor -- start 2023-10-06T05:14:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.575+0000 7fc2e4d9d700 1 -- start start 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2e4d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2e0154f20 0x7fc2e00ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2e4d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2e01552f0 0x7fc2e00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2e4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e00b11e0 0x7fc2e00b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2e4d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e006bdf0 con 0x7fc2e0154f20 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2e4d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e01586a0 con 0x7fc2e01552f0 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2e4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00b1b60 con 0x7fc2e00b11e0 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2df7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2e0154f20 0x7fc2e00ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2df7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2e0154f20 0x7fc2e00ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37776/0 (socket says 172.21.15.33:37776) 2023-10-06T05:14:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.576+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2956185344 learned_addr learned my addr 172.21.15.33:0/2956185344 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2deffd700 1 --2- 172.21.15.33:0/2956185344 >> v2:172.21.15.167:3300/0 conn(0x7fc2e01552f0 0x7fc2e00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2dffff700 1 --2- 172.21.15.33:0/2956185344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e00b11e0 0x7fc2e00b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2956185344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e00b11e0 msgr2=0x7fc2e00b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2df7fe700 1 --2- 172.21.15.33:0/2956185344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e00b11e0 0x7fc2e00b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2956185344 >> v2:172.21.15.167:3300/0 conn(0x7fc2e01552f0 msgr2=0x7fc2e00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2df7fe700 1 --2- 172.21.15.33:0/2956185344 >> v2:172.21.15.167:3300/0 conn(0x7fc2e01552f0 0x7fc2e00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2956185344 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0157ab0 con 0x7fc2e0154f20 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2df7fe700 1 --2- 172.21.15.33:0/2956185344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e0154f20 0x7fc2e00ae720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00b21a0 tx=0x7fc2d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b174bfea09f8018 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2de7fc700 1 -- 172.21.15.33:0/2956185344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000b860 con 0x7fc2e0154f20 2023-10-06T05:14:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2de7fc700 1 -- 172.21.15.33:0/2956185344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000b9c0 con 0x7fc2e0154f20 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.577+0000 7fc2de7fc700 1 -- 172.21.15.33:0/2956185344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000bd60 con 0x7fc2e0154f20 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2956185344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e0154f20 msgr2=0x7fc2e00ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2956185344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e0154f20 0x7fc2e00ae720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00b21a0 tx=0x7fc2d0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2956185344 shutdown_connections 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2956185344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e00b11e0 0x7fc2e00b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2956185344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e0154f20 0x7fc2e00ae720 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2956185344 >> v2:172.21.15.167:3300/0 conn(0x7fc2e01552f0 0x7fc2e00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2956185344 >> 172.21.15.33:0/2956185344 conn(0x7fc2e006d030 msgr2=0x7fc2e00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2956185344 shutdown_connections 2023-10-06T05:14:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.578+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2956185344 wait complete. 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.579+0000 7fc2e4d9d700 1 Processor -- start 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.580+0000 7fc2e4d9d700 1 -- start start 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.580+0000 7fc2e4d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 0x7fc2e00c1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.580+0000 7fc2e4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e0154f20 0x7fc2e01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.580+0000 7fc2e4d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e0144d20 0x7fc2e0159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2e4d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00c2610 con 0x7fc2e00b11e0 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2e4d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00c1a00 con 0x7fc2e0144d20 2023-10-06T05:14:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2e4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00c1ce0 con 0x7fc2e0154f20 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 0x7fc2e00c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 0x7fc2e00c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37784/0 (socket says 172.21.15.33:37784) 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2644991149 learned_addr learned my addr 172.21.15.33:0/2644991149 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2deffd700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e0154f20 0x7fc2e01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2dffff700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e0144d20 0x7fc2e0159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e0154f20 msgr2=0x7fc2e01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e0154f20 0x7fc2e01572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e0144d20 msgr2=0x7fc2e0159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e0144d20 0x7fc2e0159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 -- 172.21.15.33:0/2644991149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00094f0 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2dffff700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e0144d20 0x7fc2e0159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.581+0000 7fc2df7fe700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 0x7fc2e00c1340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2e006d4d0 tx=0x7fc2d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.582+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0024440 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.582+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d00245a0 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.582+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0024960 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.582+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e00c1fc0 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.582+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e01451a0 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.583+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e01679c0 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.584+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2d001a420 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.584+0000 7fc2c77fe700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2cc066f30 0x7fc2cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.584+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc2d0086470 con 0x7fc2e00b11e0 2023-10-06T05:14:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.585+0000 7fc2deffd700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2cc066f30 0x7fc2cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.585+0000 7fc2deffd700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2cc066f30 0x7fc2cc0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4006fd0 tx=0x7fc2d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:38.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.589+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2d0053600 con 0x7fc2e00b11e0 2023-10-06T05:14:38.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.791+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2e004c380 con 0x7fc2e00b11e0 2023-10-06T05:14:38.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.791+0000 7fc2c77fe700 1 -- 172.21.15.33:0/2644991149 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2d0056c20 con 0x7fc2e00b11e0 2023-10-06T05:14:38.794 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2cc066f30 msgr2=0x7fc2cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2cc066f30 0x7fc2cc0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4006fd0 tx=0x7fc2d4008040 comp rx=0 tx=0).stop 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 msgr2=0x7fc2e00c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 0x7fc2e00c1340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2e006d4d0 tx=0x7fc2d0017000 comp rx=0 tx=0).stop 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 shutdown_connections 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc2cc066f30 0x7fc2cc0693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2e0144d20 0x7fc2e0159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e00b11e0 0x7fc2e00c1340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 --2- 172.21.15.33:0/2644991149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2e0154f20 0x7fc2e01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 >> 172.21.15.33:0/2644991149 conn(0x7fc2e006d030 msgr2=0x7fc2e006e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 shutdown_connections 2023-10-06T05:14:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.794+0000 7fc2e4d9d700 1 -- 172.21.15.33:0/2644991149 wait complete. 2023-10-06T05:14:38.814 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:14:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.997+0000 7f5cd179d700 1 Processor -- start 2023-10-06T05:14:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.997+0000 7f5cd179d700 1 -- start start 2023-10-06T05:14:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5cd179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ccc158700 0x7f5ccc0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5cd179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5ccc0699c0 0x7f5ccc069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5cd179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ccc158ba0 con 0x7f5ccc155140 2023-10-06T05:14:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5cd179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc0c8e40 con 0x7f5ccc158700 2023-10-06T05:14:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5cd179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc06a340 con 0x7f5ccc0699c0 2023-10-06T05:14:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ccc158700 0x7f5ccc0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ccc158700 0x7f5ccc0adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37788/0 (socket says 172.21.15.33:37788) 2023-10-06T05:14:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 -- 172.21.15.33:0/518390122 learned_addr learned my addr 172.21.15.33:0/518390122 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5cd0f9c700 1 --2- 172.21.15.33:0/518390122 >> v2:172.21.15.167:3300/0 conn(0x7f5ccc0699c0 0x7f5ccc069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 -- 172.21.15.33:0/518390122 >> v2:172.21.15.167:3300/0 conn(0x7f5ccc0699c0 msgr2=0x7f5ccc069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 --2- 172.21.15.33:0/518390122 >> v2:172.21.15.167:3300/0 conn(0x7f5ccc0699c0 0x7f5ccc069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 -- 172.21.15.33:0/518390122 >> v1:172.21.15.167:6789/0 conn(0x7f5ccc155140 legacy=0x7f5ccc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:38.999+0000 7f5ccb7fe700 1 -- 172.21.15.33:0/518390122 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc0ae490 con 0x7f5ccc158700 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5ccb7fe700 1 --2- 172.21.15.33:0/518390122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc0adf50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00099b0 tx=0x7f5cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afafa703ab45359 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5ccaffd700 1 -- 172.21.15.33:0/518390122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0009ea0 con 0x7f5ccc158700 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5ccaffd700 1 -- 172.21.15.33:0/518390122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc000b820 con 0x7f5ccc158700 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5ccaffd700 1 -- 172.21.15.33:0/518390122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000bbc0 con 0x7f5ccc158700 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 -- 172.21.15.33:0/518390122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 msgr2=0x7f5ccc0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 --2- 172.21.15.33:0/518390122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc0adf50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00099b0 tx=0x7f5cc0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 -- 172.21.15.33:0/518390122 shutdown_connections 2023-10-06T05:14:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 --2- 172.21.15.33:0/518390122 >> v2:172.21.15.167:3300/0 conn(0x7f5ccc0699c0 0x7f5ccc069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 --2- 172.21.15.33:0/518390122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc0adf50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 -- 172.21.15.33:0/518390122 >> 172.21.15.33:0/518390122 conn(0x7f5ccc06d030 msgr2=0x7f5ccc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 -- 172.21.15.33:0/518390122 shutdown_connections 2023-10-06T05:14:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.000+0000 7f5cd179d700 1 -- 172.21.15.33:0/518390122 wait complete. 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.002+0000 7f5cd179d700 1 Processor -- start 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.002+0000 7f5cd179d700 1 -- start start 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ccc0699c0 0x7f5ccc148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ccc155140 0x7f5ccc0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc150350 con 0x7f5ccc158700 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc0b0700 con 0x7f5ccc155140 2023-10-06T05:14:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc150070 con 0x7f5ccc0699c0 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd0f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd0f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37794/0 (socket says 172.21.15.33:37794) 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.003+0000 7f5cd0f9c700 1 -- 172.21.15.33:0/860984746 learned_addr learned my addr 172.21.15.33:0/860984746 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5ccb7fe700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ccc155140 0x7f5ccc0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd0f9c700 1 -- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ccc0699c0 msgr2=0x7f5ccc148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5ccbfff700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ccc0699c0 0x7f5ccc148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd0f9c700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ccc0699c0 0x7f5ccc148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd0f9c700 1 -- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ccc155140 msgr2=0x7f5ccc0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd0f9c700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ccc155140 0x7f5ccc0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd0f9c700 1 -- 172.21.15.33:0/860984746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc00094f0 con 0x7f5ccc158700 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5ccb7fe700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ccc155140 0x7f5ccc0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5ccbfff700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ccc0699c0 0x7f5ccc148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd0f9c700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc15c290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009e90 tx=0x7f5cc400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400cc20 con 0x7f5ccc158700 2023-10-06T05:14:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc400f040 con 0x7f5ccc158700 2023-10-06T05:14:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc0b00b0 con 0x7f5ccc158700 2023-10-06T05:14:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4014480 con 0x7f5ccc158700 2023-10-06T05:14:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.004+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc149440 con 0x7f5ccc158700 2023-10-06T05:14:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.005+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ccc0b0870 con 0x7f5ccc158700 2023-10-06T05:14:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.006+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5cc4014830 con 0x7f5ccc158700 2023-10-06T05:14:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.007+0000 7f5cb3fff700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5cb8066f30 0x7f5cb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.007+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5cc4085100 con 0x7f5ccc158700 2023-10-06T05:14:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.008+0000 7f5ccbfff700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5cb8066f30 0x7f5cb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.008+0000 7f5ccbfff700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5cb8066f30 0x7f5cb80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0017a0 tx=0x7f5cbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.012+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cc4052170 con 0x7f5ccc158700 2023-10-06T05:14:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.256+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ccc04c380 con 0x7f5ccc158700 2023-10-06T05:14:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.257+0000 7f5cb3fff700 1 -- 172.21.15.33:0/860984746 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f5cc4055790 con 0x7f5ccc158700 2023-10-06T05:14:39.259 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:14:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5cb8066f30 msgr2=0x7f5cb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5cb8066f30 0x7f5cb80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0017a0 tx=0x7f5cbc009040 comp rx=0 tx=0).stop 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 msgr2=0x7f5ccc15c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc15c290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009e90 tx=0x7f5cc400b040 comp rx=0 tx=0).stop 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 shutdown_connections 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5cb8066f30 0x7f5cb80693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ccc158700 0x7f5ccc15c290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ccc0699c0 0x7f5ccc148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 --2- 172.21.15.33:0/860984746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ccc155140 0x7f5ccc0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 >> 172.21.15.33:0/860984746 conn(0x7f5ccc06d030 msgr2=0x7f5ccc0c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 shutdown_connections 2023-10-06T05:14:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.259+0000 7f5cd179d700 1 -- 172.21.15.33:0/860984746 wait complete. 2023-10-06T05:14:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:14:39.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:39.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.462+0000 7fbe3a39d700 1 Processor -- start 2023-10-06T05:14:39.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.463+0000 7fbe3a39d700 1 -- start start 2023-10-06T05:14:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe3a39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe340c6340 0x7fbe340c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe3a39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe340c80d0 0x7fbe3406a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe3a39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3406c820 con 0x7fbe34159520 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe3a39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3406a160 con 0x7fbe340c80d0 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe3a39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe340c8800 con 0x7fbe340c6340 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe340c80d0 0x7fbe3406a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe340c80d0 0x7fbe3406a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37802/0 (socket says 172.21.15.33:37802) 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 -- 172.21.15.33:0/3442864415 learned_addr learned my addr 172.21.15.33:0/3442864415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe38b9a700 1 --2- 172.21.15.33:0/3442864415 >> v2:172.21.15.167:3300/0 conn(0x7fbe340c6340 0x7fbe340c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 -- 172.21.15.33:0/3442864415 >> v2:172.21.15.167:3300/0 conn(0x7fbe340c6340 msgr2=0x7fbe340c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 --2- 172.21.15.33:0/3442864415 >> v2:172.21.15.167:3300/0 conn(0x7fbe340c6340 0x7fbe340c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 -- 172.21.15.33:0/3442864415 >> v1:172.21.15.167:6789/0 conn(0x7fbe34159520 legacy=0x7fbe340c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.464+0000 7fbe39b9c700 1 -- 172.21.15.33:0/3442864415 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340c8ae0 con 0x7fbe340c80d0 2023-10-06T05:14:39.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.465+0000 7fbe39b9c700 1 --2- 172.21.15.33:0/3442864415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c80d0 0x7fbe3406a660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe340c6e50 tx=0x7fbe2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7150f9bf3136aab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:39.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.465+0000 7fbe2bfff700 1 -- 172.21.15.33:0/3442864415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00e070 con 0x7fbe340c80d0 2023-10-06T05:14:39.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.465+0000 7fbe2bfff700 1 -- 172.21.15.33:0/3442864415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00ba10 con 0x7fbe340c80d0 2023-10-06T05:14:39.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.465+0000 7fbe2bfff700 1 -- 172.21.15.33:0/3442864415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0092e0 con 0x7fbe340c80d0 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.465+0000 7fbe3a39d700 1 -- 172.21.15.33:0/3442864415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c80d0 msgr2=0x7fbe3406a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.465+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/3442864415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c80d0 0x7fbe3406a660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe340c6e50 tx=0x7fbe2c009840 comp rx=0 tx=0).stop 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.466+0000 7fbe3a39d700 1 -- 172.21.15.33:0/3442864415 shutdown_connections 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.466+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/3442864415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c80d0 0x7fbe3406a660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.466+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/3442864415 >> v2:172.21.15.167:3300/0 conn(0x7fbe340c6340 0x7fbe340c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.466+0000 7fbe3a39d700 1 -- 172.21.15.33:0/3442864415 >> 172.21.15.33:0/3442864415 conn(0x7fbe340cc930 msgr2=0x7fbe340ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.466+0000 7fbe3a39d700 1 -- 172.21.15.33:0/3442864415 shutdown_connections 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.466+0000 7fbe3a39d700 1 -- 172.21.15.33:0/3442864415 wait complete. 2023-10-06T05:14:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.467+0000 7fbe3a39d700 1 Processor -- start 2023-10-06T05:14:39.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.467+0000 7fbe3a39d700 1 -- start start 2023-10-06T05:14:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 0x7fbe3415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe340c80d0 0x7fbe3414a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe34159520 0x7fbe340acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3415b810 con 0x7fbe340c6340 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3415ac00 con 0x7fbe340c80d0 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3415aee0 con 0x7fbe34159520 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 0x7fbe3415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 0x7fbe3415a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37816/0 (socket says 172.21.15.33:37816) 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe3939b700 1 -- 172.21.15.33:0/554546945 learned_addr learned my addr 172.21.15.33:0/554546945 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe39b9c700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe34159520 0x7fbe340acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.468+0000 7fbe38b9a700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe340c80d0 0x7fbe3414a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3939b700 1 -- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe34159520 msgr2=0x7fbe340acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3939b700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe34159520 0x7fbe340acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3939b700 1 -- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe340c80d0 msgr2=0x7fbe3414a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3939b700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe340c80d0 0x7fbe3414a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3939b700 1 -- 172.21.15.33:0/554546945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c009ae0 con 0x7fbe340c6340 2023-10-06T05:14:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3939b700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 0x7fbe3415a2b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000b5e0 tx=0x7fbe30009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000f510 con 0x7fbe340c6340 2023-10-06T05:14:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3000f670 con 0x7fbe340c6340 2023-10-06T05:14:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000fa30 con 0x7fbe340c6340 2023-10-06T05:14:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3415b1c0 con 0x7fbe340c6340 2023-10-06T05:14:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.469+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3415a7f0 con 0x7fbe340c6340 2023-10-06T05:14:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.470+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3404c380 con 0x7fbe340c6340 2023-10-06T05:14:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.471+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe3001a400 con 0x7fbe340c6340 2023-10-06T05:14:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.472+0000 7fbe297fa700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe18066f30 0x7fbe180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.472+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbe300840d0 con 0x7fbe340c6340 2023-10-06T05:14:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.472+0000 7fbe38b9a700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe18066f30 0x7fbe180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.473+0000 7fbe38b9a700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe18066f30 0x7fbe180693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe3416cbe0 tx=0x7fbe24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:39.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.476+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe300511c0 con 0x7fbe340c6340 2023-10-06T05:14:39.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.678+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe340c8dc0 con 0x7fbe340c6340 2023-10-06T05:14:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.679+0000 7fbe297fa700 1 -- 172.21.15.33:0/554546945 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fbe300547e0 con 0x7fbe340c6340 2023-10-06T05:14:40.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:40.494 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe18066f30 msgr2=0x7fbe180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe18066f30 0x7fbe180693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe3416cbe0 tx=0x7fbe24008040 comp rx=0 tx=0).stop 2023-10-06T05:14:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 msgr2=0x7fbe3415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 0x7fbe3415a2b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000b5e0 tx=0x7fbe30009120 comp rx=0 tx=0).stop 2023-10-06T05:14:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 shutdown_connections 2023-10-06T05:14:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe18066f30 0x7fbe180693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe34159520 0x7fbe340acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe340c6340 0x7fbe3415a2b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 --2- 172.21.15.33:0/554546945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe340c80d0 0x7fbe3414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 >> 172.21.15.33:0/554546945 conn(0x7fbe340cc930 msgr2=0x7fbe34154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 shutdown_connections 2023-10-06T05:14:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:39.681+0000 7fbe3a39d700 1 -- 172.21.15.33:0/554546945 wait complete. 2023-10-06T05:14:40.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:14:40.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:40.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.697+0000 7febfaf9d700 1 Processor -- start 2023-10-06T05:14:40.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.697+0000 7febfaf9d700 1 -- start start 2023-10-06T05:14:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfaf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7febf4156370 0x7febf40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfaf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf40b01f0 0x7febf40b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfaf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf40ad430 con 0x7febf40cc7e0 2023-10-06T05:14:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfaf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf40afe20 con 0x7febf40b01f0 2023-10-06T05:14:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfaf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf40b0b70 con 0x7febf4156370 2023-10-06T05:14:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf40b01f0 0x7febf40b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf40b01f0 0x7febf40b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56210/0 (socket says 172.21.15.33:56210) 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 -- 172.21.15.33:0/3071290969 learned_addr learned my addr 172.21.15.33:0/3071290969 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febf9f9b700 1 --2- 172.21.15.33:0/3071290969 >> v2:172.21.15.167:3300/0 conn(0x7febf4156370 0x7febf40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 -- 172.21.15.33:0/3071290969 >> v2:172.21.15.167:3300/0 conn(0x7febf4156370 msgr2=0x7febf40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 --2- 172.21.15.33:0/3071290969 >> v2:172.21.15.167:3300/0 conn(0x7febf4156370 0x7febf40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 -- 172.21.15.33:0/3071290969 >> v1:172.21.15.167:6789/0 conn(0x7febf40cc7e0 legacy=0x7febf40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febfa79c700 1 -- 172.21.15.33:0/3071290969 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf406bc70 con 0x7febf40b01f0 2023-10-06T05:14:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.699+0000 7febf8f99700 1 -- 172.21.15.33:0/3071290969 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392579584 0 0) 0x7febf40ad430 con 0x7febf40cc7e0 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.700+0000 7febfa79c700 1 --2- 172.21.15.33:0/3071290969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40b01f0 0x7febf40b0630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febf40b21a0 tx=0x7febec00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c575f2766fd9a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.700+0000 7febf8f99700 1 -- 172.21.15.33:0/3071290969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec010070 con 0x7febf40b01f0 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.700+0000 7febf8f99700 1 -- 172.21.15.33:0/3071290969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00dde0 con 0x7febf40b01f0 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.700+0000 7febf8f99700 1 -- 172.21.15.33:0/3071290969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00b320 con 0x7febf40b01f0 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.700+0000 7febfaf9d700 1 -- 172.21.15.33:0/3071290969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40b01f0 msgr2=0x7febf40b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.700+0000 7febfaf9d700 1 --2- 172.21.15.33:0/3071290969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40b01f0 0x7febf40b0630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febf40b21a0 tx=0x7febec00ba50 comp rx=0 tx=0).stop 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.701+0000 7febfaf9d700 1 -- 172.21.15.33:0/3071290969 shutdown_connections 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.701+0000 7febfaf9d700 1 --2- 172.21.15.33:0/3071290969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40b01f0 0x7febf40b0630 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.701+0000 7febfaf9d700 1 --2- 172.21.15.33:0/3071290969 >> v2:172.21.15.167:3300/0 conn(0x7febf4156370 0x7febf40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.701+0000 7febfaf9d700 1 -- 172.21.15.33:0/3071290969 >> 172.21.15.33:0/3071290969 conn(0x7febf406d030 msgr2=0x7febf40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.701+0000 7febfaf9d700 1 -- 172.21.15.33:0/3071290969 shutdown_connections 2023-10-06T05:14:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.701+0000 7febfaf9d700 1 -- 172.21.15.33:0/3071290969 wait complete. 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.702+0000 7febfaf9d700 1 Processor -- start 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.702+0000 7febfaf9d700 1 -- start start 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfaf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf40b01f0 0x7febf40cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfaf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 0x7febf40aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfaf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf4156370 0x7febf416c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfaf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4157070 con 0x7febf40cc7e0 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfaf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40aa7e0 con 0x7febf4156370 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfaf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40aaa40 con 0x7febf40b01f0 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febf979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 0x7febf40aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febf979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 0x7febf40aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56224/0 (socket says 172.21.15.33:56224) 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febf979a700 1 -- 172.21.15.33:0/1877506866 learned_addr learned my addr 172.21.15.33:0/1877506866 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febfa79c700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf4156370 0x7febf416c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.703+0000 7febf9f9b700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf40b01f0 0x7febf40cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febf979a700 1 -- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf40b01f0 msgr2=0x7febf40cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febf979a700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf40b01f0 0x7febf40cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febf979a700 1 -- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf4156370 msgr2=0x7febf416c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febf979a700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf4156370 0x7febf416c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febf979a700 1 -- 172.21.15.33:0/1877506866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec00bcf0 con 0x7febf40cc7e0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febf979a700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 0x7febf40aa2a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febe400c7e0 tx=0x7febe4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe40105c0 con 0x7febf40cc7e0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4010740 con 0x7febf40cc7e0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4010b00 con 0x7febf40cc7e0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.704+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf4156740 con 0x7febf40cc7e0 2023-10-06T05:14:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.705+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf4156d00 con 0x7febf40cc7e0 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.705+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf404c380 con 0x7febf40cc7e0 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.706+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febe401b3b0 con 0x7febf40cc7e0 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.707+0000 7febe9ffb700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd8066f30 0x7febd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.707+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7febe4083ce0 con 0x7febf40cc7e0 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.708+0000 7febf9f9b700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd8066f30 0x7febd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.708+0000 7febf9f9b700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd8066f30 0x7febd80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febf0009db0 tx=0x7febf00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.711+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febe4050d50 con 0x7febf40cc7e0 2023-10-06T05:14:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.916+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febf41571b0 con 0x7febf40cc7e0 2023-10-06T05:14:41.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.916+0000 7febe9ffb700 1 -- 172.21.15.33:0/1877506866 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7febe4017070 con 0x7febf40cc7e0 2023-10-06T05:14:41.046 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:14:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.918+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd8066f30 msgr2=0x7febd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.918+0000 7febfaf9d700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd8066f30 0x7febd80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febf0009db0 tx=0x7febf00091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.918+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 msgr2=0x7febf40aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.918+0000 7febfaf9d700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 0x7febf40aa2a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febe400c7e0 tx=0x7febe4007df0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 shutdown_connections 2023-10-06T05:14:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd8066f30 0x7febd80693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf4156370 0x7febf416c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf40b01f0 0x7febf40cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 --2- 172.21.15.33:0/1877506866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf40cc7e0 0x7febf40aa2a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 >> 172.21.15.33:0/1877506866 conn(0x7febf406d030 msgr2=0x7febf414fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 shutdown_connections 2023-10-06T05:14:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:40.919+0000 7febfaf9d700 1 -- 172.21.15.33:0/1877506866 wait complete. 2023-10-06T05:14:41.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.238+0000 7fdbb939d700 1 Processor -- start 2023-10-06T05:14:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.238+0000 7fdbb939d700 1 -- start start 2023-10-06T05:14:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdbb4156370 0x7fdbb40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40ad430 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40afe20 con 0x7fdbb40b01f0 2023-10-06T05:14:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb406bc70 con 0x7fdbb4156370 2023-10-06T05:14:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdbb4156370 0x7fdbb40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdbb4156370 0x7fdbb40af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56232/0 (socket says 172.21.15.33:56232) 2023-10-06T05:14:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdbb40cc7e0 0x7fdbb40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41008/0 (socket says 172.21.15.33:41008) 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 -- 172.21.15.33:0/1096610776 learned_addr learned my addr 172.21.15.33:0/1096610776 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 -- 172.21.15.33:0/1096610776 >> v1:172.21.15.167:6789/0 conn(0x7fdbb40b01f0 legacy=0x7fdbb40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 -- 172.21.15.33:0/1096610776 >> v1:172.21.15.33:6789/0 conn(0x7fdbb40cc7e0 legacy=0x7fdbb40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 -- 172.21.15.33:0/1096610776 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40cccc0 con 0x7fdbb4156370 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb2ffd700 1 -- 172.21.15.33:0/1096610776 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815517429 0 0) 0x7fdbb40ad430 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb3fff700 1 --2- 172.21.15.33:0/1096610776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb4156370 0x7fdbb40af8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdba40099b0 tx=0x7fdba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cd9f72b208a8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb2ffd700 1 -- 172.21.15.33:0/1096610776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400b860 con 0x7fdbb4156370 2023-10-06T05:14:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb2ffd700 1 -- 172.21.15.33:0/1096610776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400b9c0 con 0x7fdbb4156370 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb2ffd700 1 -- 172.21.15.33:0/1096610776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400bd60 con 0x7fdbb4156370 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 -- 172.21.15.33:0/1096610776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb4156370 msgr2=0x7fdbb40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 --2- 172.21.15.33:0/1096610776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb4156370 0x7fdbb40af8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdba40099b0 tx=0x7fdba4009250 comp rx=0 tx=0).stop 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 -- 172.21.15.33:0/1096610776 shutdown_connections 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 --2- 172.21.15.33:0/1096610776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb4156370 0x7fdbb40af8e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.240+0000 7fdbb939d700 1 -- 172.21.15.33:0/1096610776 >> 172.21.15.33:0/1096610776 conn(0x7fdbb406d030 msgr2=0x7fdbb40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.241+0000 7fdbb939d700 1 -- 172.21.15.33:0/1096610776 shutdown_connections 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.241+0000 7fdbb939d700 1 -- 172.21.15.33:0/1096610776 wait complete. 2023-10-06T05:14:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.242+0000 7fdbb939d700 1 Processor -- start 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.242+0000 7fdbb939d700 1 -- start start 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40b01f0 0x7fdbb4151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 0x7fdbb415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbb4156370 0x7fdbb415f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb415dfb0 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb415e290 con 0x7fdbb40b01f0 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb415e570 con 0x7fdbb4156370 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 0x7fdbb415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 0x7fdbb415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56246/0 (socket says 172.21.15.33:56246) 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb37fe700 1 -- 172.21.15.33:0/4174705643 learned_addr learned my addr 172.21.15.33:0/4174705643 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb3fff700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40b01f0 0x7fdbb4151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb8b9c700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbb4156370 0x7fdbb415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.243+0000 7fdbb37fe700 1 -- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbb4156370 msgr2=0x7fdbb415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb37fe700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbb4156370 0x7fdbb415f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb37fe700 1 -- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40b01f0 msgr2=0x7fdbb4151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb37fe700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40b01f0 0x7fdbb4151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb37fe700 1 -- 172.21.15.33:0/4174705643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40094f0 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb3fff700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40b01f0 0x7fdbb4151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb8b9c700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbb4156370 0x7fdbb415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb37fe700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 0x7fdbb415d4f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c810 tx=0x7fdba8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba80073d0 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba8007550 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb415e850 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8004a80 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.244+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb41523a0 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.245+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb415ed70 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.246+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdba801c3b0 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.248+0000 7fdb9bfff700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdba0066f30 0x7fdba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.248+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdba8083c30 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.248+0000 7fdbb3fff700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdba0066f30 0x7fdba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.250+0000 7fdbb3fff700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdba0066f30 0x7fdba00693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdba40099b0 tx=0x7fdba4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.252+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbb415ed70 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.455+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdbb415ed70 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.456+0000 7fdb9bfff700 1 -- 172.21.15.33:0/4174705643 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdbb415ed70 con 0x7fdbb40cc7e0 2023-10-06T05:14:41.458 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdba0066f30 msgr2=0x7fdba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdba0066f30 0x7fdba00693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdba40099b0 tx=0x7fdba4017000 comp rx=0 tx=0).stop 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 msgr2=0x7fdbb415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 0x7fdbb415d4f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c810 tx=0x7fdba8007d90 comp rx=0 tx=0).stop 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 shutdown_connections 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdba0066f30 0x7fdba00693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbb4156370 0x7fdbb415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40b01f0 0x7fdbb4151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 --2- 172.21.15.33:0/4174705643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbb40cc7e0 0x7fdbb415d4f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 >> 172.21.15.33:0/4174705643 conn(0x7fdbb406d030 msgr2=0x7fdbb40ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 shutdown_connections 2023-10-06T05:14:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.458+0000 7fdbb939d700 1 -- 172.21.15.33:0/4174705643 wait complete. 2023-10-06T05:14:41.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:14:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.665+0000 7fe043fff700 1 Processor -- start 2023-10-06T05:14:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.665+0000 7fe043fff700 1 -- start start 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe043fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe0440af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe043fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe0440b01f0 0x7fe0440b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe043fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440ad430 con 0x7fe044156370 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe043fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0440afe20 con 0x7fe0440b01f0 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe043fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440b0b70 con 0x7fe0440cc7e0 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe042ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe044156370 0x7fe0440c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41012/0 (socket says 172.21.15.33:41012) 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe042ffd700 1 -- 172.21.15.33:0/927410068 learned_addr learned my addr 172.21.15.33:0/927410068 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0437fe700 1 --2- 172.21.15.33:0/927410068 >> v2:172.21.15.167:3300/0 conn(0x7fe0440b01f0 0x7fe0440b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0427fc700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe0440af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe041ffb700 1 -- 172.21.15.33:0/927410068 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642361679 0 0) 0x7fe0440ad430 con 0x7fe044156370 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe041ffb700 1 -- 172.21.15.33:0/927410068 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe02c003570 con 0x7fe044156370 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0437fe700 1 -- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 msgr2=0x7fe0440af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0437fe700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe0440af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0437fe700 1 -- 172.21.15.33:0/927410068 >> v1:172.21.15.33:6789/0 conn(0x7fe044156370 legacy=0x7fe0440c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe041ffb700 1 -- 172.21.15.33:0/927410068 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109740055 0 0) 0x7fe02c003570 con 0x7fe044156370 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0437fe700 1 -- 172.21.15.33:0/927410068 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04406bc70 con 0x7fe0440b01f0 2023-10-06T05:14:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.666+0000 7fe0427fc700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe0440af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.667+0000 7fe0437fe700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0440b01f0 0x7fe0440b0630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009e60 tx=0x7fe03c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d182afd86fdee56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.667+0000 7fe041ffb700 1 -- 172.21.15.33:0/927410068 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034001470 con 0x7fe0440b01f0 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.667+0000 7fe041ffb700 1 -- 172.21.15.33:0/927410068 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c00b7a0 con 0x7fe0440b01f0 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.667+0000 7fe041ffb700 1 -- 172.21.15.33:0/927410068 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00bb60 con 0x7fe0440b01f0 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.667+0000 7fe043fff700 1 -- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0440b01f0 msgr2=0x7fe0440b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.667+0000 7fe043fff700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0440b01f0 0x7fe0440b0630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009e60 tx=0x7fe03c009780 comp rx=0 tx=0).stop 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.668+0000 7fe043fff700 1 -- 172.21.15.33:0/927410068 shutdown_connections 2023-10-06T05:14:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.668+0000 7fe043fff700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0440b01f0 0x7fe0440b0630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.668+0000 7fe043fff700 1 --2- 172.21.15.33:0/927410068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe0440af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.668+0000 7fe043fff700 1 -- 172.21.15.33:0/927410068 >> 172.21.15.33:0/927410068 conn(0x7fe04406d030 msgr2=0x7fe0440cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.668+0000 7fe043fff700 1 -- 172.21.15.33:0/927410068 shutdown_connections 2023-10-06T05:14:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.668+0000 7fe043fff700 1 -- 172.21.15.33:0/927410068 wait complete. 2023-10-06T05:14:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.669+0000 7fe043fff700 1 Processor -- start 2023-10-06T05:14:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.669+0000 7fe043fff700 1 -- start start 2023-10-06T05:14:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 0x7fe04414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe04413cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe044156370 0x7fe044165150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe044166260 con 0x7fe0440b01f0 2023-10-06T05:14:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440c94e0 con 0x7fe044156370 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04414c440 con 0x7fe0440cc7e0 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 0x7fe04414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 0x7fe04414bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56250/0 (socket says 172.21.15.33:56250) 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 -- 172.21.15.33:0/4088761290 learned_addr learned my addr 172.21.15.33:0/4088761290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe0437fe700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe044156370 0x7fe044165150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe0427fc700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe04413cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 -- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 msgr2=0x7fe04413cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe04413cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 -- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe044156370 msgr2=0x7fe044165150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe044156370 0x7fe044165150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 -- 172.21.15.33:0/4088761290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009420 con 0x7fe0440b01f0 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe0427fc700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe04413cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe0437fe700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe044156370 0x7fe044165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe042ffd700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 0x7fe04414bc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe03400c7f0 tx=0x7fe0340019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034007360 con 0x7fe0440b01f0 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0340074c0 con 0x7fe0440b01f0 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034004830 con 0x7fe0440b01f0 2023-10-06T05:14:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c009a20 con 0x7fe0440b01f0 2023-10-06T05:14:41.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.671+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04414ca00 con 0x7fe0440b01f0 2023-10-06T05:14:41.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.673+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04404c380 con 0x7fe0440b01f0 2023-10-06T05:14:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.673+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe034004c60 con 0x7fe0440b01f0 2023-10-06T05:14:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.674+0000 7fe02affd700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe030066f30 0x7fe0300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.675+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe034083fa0 con 0x7fe0440b01f0 2023-10-06T05:14:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.675+0000 7fe0427fc700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe030066f30 0x7fe0300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.675+0000 7fe0427fc700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe030066f30 0x7fe0300693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0380093f0 tx=0x7fe038008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.679+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe034051090 con 0x7fe0440b01f0 2023-10-06T05:14:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.884+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe04406f310 con 0x7fe0440b01f0 2023-10-06T05:14:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.885+0000 7fe02affd700 1 -- 172.21.15.33:0/4088761290 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0340180a0 con 0x7fe0440b01f0 2023-10-06T05:14:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:14:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe030066f30 msgr2=0x7fe0300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe030066f30 0x7fe0300693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0380093f0 tx=0x7fe038008040 comp rx=0 tx=0).stop 2023-10-06T05:14:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 msgr2=0x7fe04414bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 0x7fe04414bc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe03400c7f0 tx=0x7fe0340019e0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 shutdown_connections 2023-10-06T05:14:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe030066f30 0x7fe0300693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe044156370 0x7fe044165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0440b01f0 0x7fe04414bc10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 --2- 172.21.15.33:0/4088761290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0440cc7e0 0x7fe04413cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 >> 172.21.15.33:0/4088761290 conn(0x7fe04406d030 msgr2=0x7fe04415a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 shutdown_connections 2023-10-06T05:14:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:41.887+0000 7fe043fff700 1 -- 172.21.15.33:0/4088761290 wait complete. 2023-10-06T05:14:41.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:14:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.095+0000 7f458ffff700 1 Processor -- start 2023-10-06T05:14:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.095+0000 7f458ffff700 1 -- start start 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.096+0000 7f458ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45900b0900 0x7f45900af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.096+0000 7f458ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45900ad430 con 0x7f45900cc7e0 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.096+0000 7f458ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45900afe20 con 0x7f4590156370 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.096+0000 7f458ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459006bc70 con 0x7f45900b0900 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.096+0000 7f458e7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f45900cc7e0 0x7f45900b01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41024/0 (socket says 172.21.15.33:41024) 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.096+0000 7f458e7fc700 1 -- 172.21.15.33:0/3968713123 learned_addr learned my addr 172.21.15.33:0/3968713123 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:42.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458f7fe700 1 --2- 172.21.15.33:0/3968713123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45900b0900 0x7f45900af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522702952 0 0) 0x7f45900ad430 con 0x7f45900cc7e0 2023-10-06T05:14:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4578003570 con 0x7f45900cc7e0 2023-10-06T05:14:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384728803 0 0) 0x7f45900afe20 con 0x7f4590156370 2023-10-06T05:14:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45900ad430 con 0x7f4590156370 2023-10-06T05:14:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458f7fe700 1 -- 172.21.15.33:0/3968713123 >> v1:172.21.15.167:6789/0 conn(0x7f4590156370 legacy=0x7f45900c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458f7fe700 1 -- 172.21.15.33:0/3968713123 >> v1:172.21.15.33:6789/0 conn(0x7f45900cc7e0 legacy=0x7f45900b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3009292289 0 0) 0x7f4578003570 con 0x7f45900cc7e0 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458f7fe700 1 -- 172.21.15.33:0/3968713123 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45900cccc0 con 0x7f45900b0900 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.097+0000 7f458f7fe700 1 --2- 172.21.15.33:0/3968713123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45900b0900 0x7f45900af8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4588009e90 tx=0x7f4588009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d8796d5eb7d6f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800e070 con 0x7f45900b0900 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458800b870 con 0x7f45900b0900 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458dffb700 1 -- 172.21.15.33:0/3968713123 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584001470 con 0x7f45900b0900 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 -- 172.21.15.33:0/3968713123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45900b0900 msgr2=0x7f45900af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 --2- 172.21.15.33:0/3968713123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45900b0900 0x7f45900af8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4588009e90 tx=0x7f4588009780 comp rx=0 tx=0).stop 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 -- 172.21.15.33:0/3968713123 shutdown_connections 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 --2- 172.21.15.33:0/3968713123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f45900b0900 0x7f45900af8e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 -- 172.21.15.33:0/3968713123 >> 172.21.15.33:0/3968713123 conn(0x7f459006d030 msgr2=0x7f45900cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 -- 172.21.15.33:0/3968713123 shutdown_connections 2023-10-06T05:14:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.098+0000 7f458ffff700 1 -- 172.21.15.33:0/3968713123 wait complete. 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.100+0000 7f458ffff700 1 Processor -- start 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.100+0000 7f458ffff700 1 -- start start 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 0x7f459014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45900cc7e0 0x7f4590150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4590156370 0x7f45901505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459014b800 con 0x7f45900b0900 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4590157cb0 con 0x7f45900cc7e0 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4590158270 con 0x7f4590156370 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 0x7f459014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 0x7f459014b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56262/0 (socket says 172.21.15.33:56262) 2023-10-06T05:14:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 -- 172.21.15.33:0/2357629413 learned_addr learned my addr 172.21.15.33:0/2357629413 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458e7fc700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45900cc7e0 0x7f4590150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458f7fe700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4590156370 0x7f45901505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 -- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4590156370 msgr2=0x7f45901505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4590156370 0x7f45901505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 -- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45900cc7e0 msgr2=0x7f4590150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.101+0000 7f458effd700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45900cc7e0 0x7f4590150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f458effd700 1 -- 172.21.15.33:0/2357629413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4588009a20 con 0x7f45900b0900 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f458e7fc700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45900cc7e0 0x7f4590150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f458effd700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 0x7f459014b2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f458000c7c0 tx=0x7f4580007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580010510 con 0x7f45900b0900 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4580010690 con 0x7f45900b0900 2023-10-06T05:14:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590153cf0 con 0x7f45900b0900 2023-10-06T05:14:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580010a50 con 0x7f45900b0900 2023-10-06T05:14:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.102+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590157f00 con 0x7f45900b0900 2023-10-06T05:14:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.103+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459004c380 con 0x7f45900b0900 2023-10-06T05:14:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.104+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f458001b400 con 0x7f45900b0900 2023-10-06T05:14:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.105+0000 7f4576ffd700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f457c066f30 0x7f457c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.105+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4580083ec0 con 0x7f45900b0900 2023-10-06T05:14:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.105+0000 7f458e7fc700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f457c066f30 0x7f457c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.106+0000 7f458e7fc700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f457c066f30 0x7f457c0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45840015a0 tx=0x7f4584000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.109+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f458004b5a0 con 0x7f45900b0900 2023-10-06T05:14:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.310+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f459006f310 con 0x7f45900b0900 2023-10-06T05:14:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.311+0000 7f4576ffd700 1 -- 172.21.15.33:0/2357629413 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45800160a0 con 0x7f45900b0900 2023-10-06T05:14:42.312 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f457c066f30 msgr2=0x7f457c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f457c066f30 0x7f457c0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45840015a0 tx=0x7f4584000bc0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 msgr2=0x7f459014b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 0x7f459014b2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f458000c7c0 tx=0x7f4580007e10 comp rx=0 tx=0).stop 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 shutdown_connections 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f457c066f30 0x7f457c0693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4590156370 0x7f45901505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45900b0900 0x7f459014b2c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 --2- 172.21.15.33:0/2357629413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45900cc7e0 0x7f4590150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 >> 172.21.15.33:0/2357629413 conn(0x7f459006d030 msgr2=0x7f45900ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 shutdown_connections 2023-10-06T05:14:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.313+0000 7f458ffff700 1 -- 172.21.15.33:0/2357629413 wait complete. 2023-10-06T05:14:42.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:14:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.516+0000 7fb122f9d700 1 Processor -- start 2023-10-06T05:14:42.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.516+0000 7fb122f9d700 1 -- start start 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.517+0000 7fb122f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c06e140 0x7fb11c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.517+0000 7fb122f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c06f220 con 0x7fb11c159530 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.517+0000 7fb122f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c15ae50 con 0x7fb11c0afdd0 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.517+0000 7fb122f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0b1e00 con 0x7fb11c06e140 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb121f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb11c159530 0x7fb11c06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41032/0 (socket says 172.21.15.33:41032) 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb121f9b700 1 -- 172.21.15.33:0/766856609 learned_addr learned my addr 172.21.15.33:0/766856609 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:42.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb12179a700 1 --2- 172.21.15.33:0/766856609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c06e140 0x7fb11c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833231086 0 0) 0x7fb11c06f220 con 0x7fb11c159530 2023-10-06T05:14:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb108003570 con 0x7fb11c159530 2023-10-06T05:14:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802973805 0 0) 0x7fb11c15ae50 con 0x7fb11c0afdd0 2023-10-06T05:14:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb11c06f220 con 0x7fb11c0afdd0 2023-10-06T05:14:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb12179a700 1 -- 172.21.15.33:0/766856609 >> v1:172.21.15.167:6789/0 conn(0x7fb11c0afdd0 legacy=0x7fb11c0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb12179a700 1 -- 172.21.15.33:0/766856609 >> v1:172.21.15.33:6789/0 conn(0x7fb11c159530 legacy=0x7fb11c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.518+0000 7fb12179a700 1 -- 172.21.15.33:0/766856609 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c06e610 con 0x7fb11c06e140 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb12179a700 1 --2- 172.21.15.33:0/766856609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c06e140 0x7fb11c15a910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009910 tx=0x7fb10c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e673fe9ff8d8811 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011630 con 0x7fb11c06e140 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c011790 con 0x7fb11c06e140 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb120f99700 1 -- 172.21.15.33:0/766856609 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011b30 con 0x7fb11c06e140 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 -- 172.21.15.33:0/766856609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c06e140 msgr2=0x7fb11c15a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 --2- 172.21.15.33:0/766856609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c06e140 0x7fb11c15a910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009910 tx=0x7fb10c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 -- 172.21.15.33:0/766856609 shutdown_connections 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 --2- 172.21.15.33:0/766856609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c06e140 0x7fb11c15a910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 -- 172.21.15.33:0/766856609 >> 172.21.15.33:0/766856609 conn(0x7fb11c06c560 msgr2=0x7fb11c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 -- 172.21.15.33:0/766856609 shutdown_connections 2023-10-06T05:14:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.519+0000 7fb122f9d700 1 -- 172.21.15.33:0/766856609 wait complete. 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.521+0000 7fb122f9d700 1 Processor -- start 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.521+0000 7fb122f9d700 1 -- start start 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb122f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11c06e140 0x7fb11c144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb122f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 0x7fb11c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb122f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c159530 0x7fb11c154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb122f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0c8610 con 0x7fb11c0afdd0 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb122f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0c8180 con 0x7fb11c06e140 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb122f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0c75b0 con 0x7fb11c159530 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb12179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 0x7fb11c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb12179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 0x7fb11c153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56274/0 (socket says 172.21.15.33:56274) 2023-10-06T05:14:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb12179a700 1 -- 172.21.15.33:0/1309159671 learned_addr learned my addr 172.21.15.33:0/1309159671 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb121f9b700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11c06e140 0x7fb11c144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb12179a700 1 -- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c159530 msgr2=0x7fb11c154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.522+0000 7fb12279c700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c159530 0x7fb11c154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb12179a700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c159530 0x7fb11c154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb12179a700 1 -- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11c06e140 msgr2=0x7fb11c144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb12179a700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11c06e140 0x7fb11c144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb12179a700 1 -- 172.21.15.33:0/1309159671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c009420 con 0x7fb11c0afdd0 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb121f9b700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11c06e140 0x7fb11c144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb12279c700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c159530 0x7fb11c154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb12179a700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 0x7fb11c153bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009f00 tx=0x7fb10c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c01f070 con 0x7fb11c0afdd0 2023-10-06T05:14:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c000bc0 con 0x7fb11c0afdd0 2023-10-06T05:14:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c0c7850 con 0x7fb11c0afdd0 2023-10-06T05:14:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c025420 con 0x7fb11c0afdd0 2023-10-06T05:14:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.523+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0c7e10 con 0x7fb11c0afdd0 2023-10-06T05:14:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.524+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c0cf250 con 0x7fb11c0afdd0 2023-10-06T05:14:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.525+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb10c023070 con 0x7fb11c0afdd0 2023-10-06T05:14:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.526+0000 7fb111ffb700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb100066f30 0x7fb1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.526+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb10c085b50 con 0x7fb11c0afdd0 2023-10-06T05:14:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.527+0000 7fb121f9b700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb100066f30 0x7fb1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.527+0000 7fb121f9b700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb100066f30 0x7fb1000693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb118007fd0 tx=0x7fb118009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.530+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb10c052bc0 con 0x7fb11c0afdd0 2023-10-06T05:14:42.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.778+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb11c04c380 con 0x7fb11c0afdd0 2023-10-06T05:14:42.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.778+0000 7fb111ffb700 1 -- 172.21.15.33:0/1309159671 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7fb10c015090 con 0x7fb11c0afdd0 2023-10-06T05:14:42.780 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:42.780 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:14:42.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.780+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb100066f30 msgr2=0x7fb1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.780+0000 7fb122f9d700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb100066f30 0x7fb1000693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb118007fd0 tx=0x7fb118009040 comp rx=0 tx=0).stop 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.780+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 msgr2=0x7fb11c153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.780+0000 7fb122f9d700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 0x7fb11c153bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009f00 tx=0x7fb10c017000 comp rx=0 tx=0).stop 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 shutdown_connections 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb100066f30 0x7fb1000693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb11c159530 0x7fb11c154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11c06e140 0x7fb11c144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 --2- 172.21.15.33:0/1309159671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb11c0afdd0 0x7fb11c153bc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 >> 172.21.15.33:0/1309159671 conn(0x7fb11c06c560 msgr2=0x7fb11c1559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 shutdown_connections 2023-10-06T05:14:42.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.781+0000 7fb122f9d700 1 -- 172.21.15.33:0/1309159671 wait complete. 2023-10-06T05:14:42.784 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:14:42.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.985+0000 7fca3b59e700 1 Processor -- start 2023-10-06T05:14:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.985+0000 7fca3b59e700 1 -- start start 2023-10-06T05:14:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.986+0000 7fca3b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c159bb0 0x7fca3c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.986+0000 7fca3b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0cbd80 con 0x7fca3c0ccfc0 2023-10-06T05:14:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.986+0000 7fca3b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0af5f0 con 0x7fca3c0cf3b0 2023-10-06T05:14:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.986+0000 7fca3b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0af8d0 con 0x7fca3c159bb0 2023-10-06T05:14:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca3c0ccfc0 0x7fca3c0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41034/0 (socket says 172.21.15.33:41034) 2023-10-06T05:14:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3a59c700 1 -- 172.21.15.33:0/3308550230 learned_addr learned my addr 172.21.15.33:0/3308550230 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:42.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c159bb0 0x7fca3c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563519914 0 0) 0x7fca3c0cbd80 con 0x7fca3c0ccfc0 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca24003570 con 0x7fca3c0ccfc0 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807402448 0 0) 0x7fca3c0af5f0 con 0x7fca3c0cf3b0 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c0cbd80 con 0x7fca3c0cf3b0 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8628881 0 0) 0x7fca24003570 con 0x7fca3c0ccfc0 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c159bb0 msgr2=0x7fca3c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:42.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c159bb0 0x7fca3c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 >> v1:172.21.15.167:6789/0 conn(0x7fca3c0cf3b0 legacy=0x7fca3c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0afbb0 con 0x7fca3c0ccfc0 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fca2c001470 con 0x7fca3c0ccfc0 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 >> v1:172.21.15.33:6789/0 conn(0x7fca3c0ccfc0 legacy=0x7fca3c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca24003c20 0x7fca24004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca24004810 0x7fca24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.987+0000 7fca3959a700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca240051c0 0x7fca2400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0af8d0 con 0x7fca24003c20 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2400c080 con 0x7fca240051c0 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2400c240 con 0x7fca24004810 2023-10-06T05:14:42.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c159bb0 0x7fca3c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca24003c20 0x7fca24004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca39d9b700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca24004810 0x7fca24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca240051c0 0x7fca2400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 -- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca24004810 msgr2=0x7fca24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca24004810 0x7fca24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 -- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca240051c0 msgr2=0x7fca2400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca240051c0 0x7fca2400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 -- 172.21.15.33:0/3308550230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24003570 con 0x7fca24003c20 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca240051c0 0x7fca2400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.988+0000 7fca3a59c700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca24003c20 0x7fca24004010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c1e0 tx=0x7fca2c001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5179572ca4d8121 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00cb10 con 0x7fca24003c20 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c00cc90 con 0x7fca24003c20 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3959a700 1 -- 172.21.15.33:0/3308550230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c005b70 con 0x7fca24003c20 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3b59e700 1 -- 172.21.15.33:0/3308550230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca24003c20 msgr2=0x7fca24004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3b59e700 1 --2- 172.21.15.33:0/3308550230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca24003c20 0x7fca24004010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c1e0 tx=0x7fca2c001d40 comp rx=0 tx=0).stop 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3ad9d700 1 -- 172.21.15.33:0/3308550230 reap_dead start 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3b59e700 1 -- 172.21.15.33:0/3308550230 shutdown_connections 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3b59e700 1 -- 172.21.15.33:0/3308550230 >> 172.21.15.33:0/3308550230 conn(0x7fca3c06cff0 msgr2=0x7fca3c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3b59e700 1 -- 172.21.15.33:0/3308550230 shutdown_connections 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.989+0000 7fca3b59e700 1 -- 172.21.15.33:0/3308550230 wait complete. 2023-10-06T05:14:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.991+0000 7fca3b59e700 1 Processor -- start 2023-10-06T05:14:42.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.991+0000 7fca3b59e700 1 -- start start 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca24003c20 0x7fca3c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c1508a0 0x7fca3c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 0x7fca3c0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0c8eb0 con 0x7fca3c150d10 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c14f740 con 0x7fca24003c20 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b4750 con 0x7fca3c1508a0 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 0x7fca3c0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca24003c20 0x7fca3c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 0x7fca3c0b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56286/0 (socket says 172.21.15.33:56286) 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 -- 172.21.15.33:0/1058191280 learned_addr learned my addr 172.21.15.33:0/1058191280 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca39d9b700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c1508a0 0x7fca3c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 -- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c1508a0 msgr2=0x7fca3c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c1508a0 0x7fca3c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 -- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca24003c20 msgr2=0x7fca3c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca24003c20 0x7fca3c14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.992+0000 7fca3ad9d700 1 -- 172.21.15.33:0/1058191280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0c8580 con 0x7fca3c150d10 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.993+0000 7fca3ad9d700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 0x7fca3c0b5470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0d0270 tx=0x7fca3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.993+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400fc20 con 0x7fca3c150d10 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.993+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400fd80 con 0x7fca3c150d10 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.993+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34016650 con 0x7fca3c150d10 2023-10-06T05:14:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.993+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c0c8860 con 0x7fca3c150d10 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.993+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c0d0f70 con 0x7fca3c150d10 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.994+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c0c9020 con 0x7fca3c150d10 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.995+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca34016a40 con 0x7fca3c150d10 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.996+0000 7fca2a7fc700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca1c066f30 0x7fca1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.996+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca34085230 con 0x7fca3c150d10 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.996+0000 7fca3a59c700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca1c066f30 0x7fca1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:42.997+0000 7fca3a59c700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca1c066f30 0x7fca1c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0c74b0 tx=0x7fca2c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.000+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca340522a0 con 0x7fca3c150d10 2023-10-06T05:14:43.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.200+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca3c04c380 con 0x7fca3c150d10 2023-10-06T05:14:43.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.201+0000 7fca2a7fc700 1 -- 172.21.15.33:0/1058191280 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fca340558c0 con 0x7fca3c150d10 2023-10-06T05:14:43.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:43.203 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca1c066f30 msgr2=0x7fca1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca1c066f30 0x7fca1c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0c74b0 tx=0x7fca2c015000 comp rx=0 tx=0).stop 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 msgr2=0x7fca3c0b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 0x7fca3c0b5470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0d0270 tx=0x7fca3400b040 comp rx=0 tx=0).stop 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 shutdown_connections 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca1c066f30 0x7fca1c0693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3c150d10 0x7fca3c0b5470 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca24003c20 0x7fca3c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 --2- 172.21.15.33:0/1058191280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca3c1508a0 0x7fca3c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 >> 172.21.15.33:0/1058191280 conn(0x7fca3c06cff0 msgr2=0x7fca3c157d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 shutdown_connections 2023-10-06T05:14:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.203+0000 7fca3b59e700 1 -- 172.21.15.33:0/1058191280 wait complete. 2023-10-06T05:14:43.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:14:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.408+0000 7fc4df3ff700 1 Processor -- start 2023-10-06T05:14:43.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.408+0000 7fc4df3ff700 1 -- start start 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.409+0000 7fc4df3ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4d806a160 0x7fc4d80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.409+0000 7fc4df3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d815b0c0 con 0x7fc4d8157740 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.409+0000 7fc4df3ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d815bc40 con 0x7fc4d8157370 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.409+0000 7fc4df3ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d815c0d0 con 0x7fc4d806a160 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dc99a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc4d8157740 0x7fc4d80c8b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41044/0 (socket says 172.21.15.33:41044) 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dd99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4d806a160 0x7fc4d80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dc99a700 1 -- 172.21.15.33:0/2603050390 learned_addr learned my addr 172.21.15.33:0/2603050390 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dd99c700 1 -- 172.21.15.33:0/2603050390 >> v1:172.21.15.167:6789/0 conn(0x7fc4d8157370 legacy=0x7fc4d8069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4cffff700 1 -- 172.21.15.33:0/2603050390 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757862639 0 0) 0x7fc4d815b0c0 con 0x7fc4d8157740 2023-10-06T05:14:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dd99c700 1 -- 172.21.15.33:0/2603050390 >> v1:172.21.15.33:6789/0 conn(0x7fc4d8157740 legacy=0x7fc4d80c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dd99c700 1 -- 172.21.15.33:0/2603050390 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d815b310 con 0x7fc4d806a160 2023-10-06T05:14:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4dd99c700 1 --2- 172.21.15.33:0/2603050390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d806a160 0x7fc4d80c9230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009e60 tx=0x7fc4d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b28655c25b38e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.410+0000 7fc4cffff700 1 -- 172.21.15.33:0/2603050390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000e070 con 0x7fc4d806a160 2023-10-06T05:14:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4cffff700 1 -- 172.21.15.33:0/2603050390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d000bbf0 con 0x7fc4d806a160 2023-10-06T05:14:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4cffff700 1 -- 172.21.15.33:0/2603050390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0012430 con 0x7fc4d806a160 2023-10-06T05:14:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2603050390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d806a160 msgr2=0x7fc4d80c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2603050390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d806a160 0x7fc4d80c9230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009e60 tx=0x7fc4d0009780 comp rx=0 tx=0).stop 2023-10-06T05:14:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2603050390 shutdown_connections 2023-10-06T05:14:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2603050390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d806a160 0x7fc4d80c9230 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2603050390 >> 172.21.15.33:0/2603050390 conn(0x7fc4d80675f0 msgr2=0x7fc4d80679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2603050390 shutdown_connections 2023-10-06T05:14:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.411+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2603050390 wait complete. 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.413+0000 7fc4df3ff700 1 Processor -- start 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.413+0000 7fc4df3ff700 1 -- start start 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4df3ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4d806a160 0x7fc4d814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4df3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 0x7fc4d8152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4df3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4d8157740 0x7fc4d814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4df3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d8153480 con 0x7fc4d8157370 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4df3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80b04c0 con 0x7fc4d8157740 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4df3ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d8152be0 con 0x7fc4d806a160 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 0x7fc4d8152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 0x7fc4d8152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56298/0 (socket says 172.21.15.33:56298) 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 -- 172.21.15.33:0/2822219489 learned_addr learned my addr 172.21.15.33:0/2822219489 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dd19b700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4d806a160 0x7fc4d814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dd99c700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4d8157740 0x7fc4d814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 -- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4d806a160 msgr2=0x7fc4d814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4d806a160 0x7fc4d814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 -- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4d8157740 msgr2=0x7fc4d814fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4d8157740 0x7fc4d814fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.414+0000 7fc4dc99a700 1 -- 172.21.15.33:0/2822219489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0009a20 con 0x7fc4d8157370 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.415+0000 7fc4dc99a700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 0x7fc4d8152390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800c5b0 tx=0x7fc4c8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.415+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8005b30 con 0x7fc4d8157370 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.415+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c8005c90 con 0x7fc4d8157370 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.415+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8007300 con 0x7fc4d8157370 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.415+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d8152ec0 con 0x7fc4d8157370 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.415+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d81528d0 con 0x7fc4d8157370 2023-10-06T05:14:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.416+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d80b0290 con 0x7fc4d8157370 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.417+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4c801b3b0 con 0x7fc4d8157370 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.418+0000 7fc4cd7fa700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4bc066f30 0x7fc4bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.418+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc4c8083c10 con 0x7fc4d8157370 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.418+0000 7fc4dd19b700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4bc066f30 0x7fc4bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.419+0000 7fc4dd19b700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4bc066f30 0x7fc4bc0693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4003a30 tx=0x7fc4d4007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.422+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4d80b0290 con 0x7fc4d8157370 2023-10-06T05:14:43.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.673+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4d80b0290 con 0x7fc4d8157370 2023-10-06T05:14:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.673+0000 7fc4cd7fa700 1 -- 172.21.15.33:0/2822219489 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7fc4d80b0290 con 0x7fc4d8157370 2023-10-06T05:14:43.675 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:43.675 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:14:43.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.675+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4bc066f30 msgr2=0x7fc4bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.675+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4bc066f30 0x7fc4bc0693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4003a30 tx=0x7fc4d4007ea0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.675+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 msgr2=0x7fc4d8152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.675+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 0x7fc4d8152390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800c5b0 tx=0x7fc4c8001070 comp rx=0 tx=0).stop 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 shutdown_connections 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4bc066f30 0x7fc4bc0693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4d8157740 0x7fc4d814fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4d806a160 0x7fc4d814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 --2- 172.21.15.33:0/2822219489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4d8157370 0x7fc4d8152390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 >> 172.21.15.33:0/2822219489 conn(0x7fc4d80675f0 msgr2=0x7fc4d815c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 shutdown_connections 2023-10-06T05:14:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:43.676+0000 7fc4df3ff700 1 -- 172.21.15.33:0/2822219489 wait complete. 2023-10-06T05:14:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:14:44.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:14:25.204019+0000', 'modified': '2023-10-06T05:14:28.726642+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/256792669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 256792669}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 256792669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:14:44.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:14:44.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:14:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.284+0000 7f97a899d700 1 Processor -- start 2023-10-06T05:14:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.285+0000 7f97a899d700 1 -- start start 2023-10-06T05:14:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97a40c8090 0x7f97a4156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a4159e10 0x7f97a4157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a899d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a406a820 con 0x7f97a4156880 2023-10-06T05:14:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a40cc290 con 0x7f97a40c8090 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a406bd10 con 0x7f97a4159e10 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97a4156880 0x7f97a40c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41050/0 (socket says 172.21.15.33:41050) 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a37fe700 1 -- 172.21.15.33:0/4145237464 learned_addr learned my addr 172.21.15.33:0/4145237464 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a3fff700 1 --2- 172.21.15.33:0/4145237464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a4159e10 0x7f97a4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.286+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/4145237464 >> v2:172.21.15.167:3300/0 conn(0x7f97a40c8090 0x7f97a4156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a27fc700 1 -- 172.21.15.33:0/4145237464 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718583685 0 0) 0x7f97a406a820 con 0x7f97a4156880 2023-10-06T05:14:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a27fc700 1 -- 172.21.15.33:0/4145237464 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f978c003570 con 0x7f97a4156880 2023-10-06T05:14:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a3fff700 1 -- 172.21.15.33:0/4145237464 >> v2:172.21.15.167:3300/0 conn(0x7f97a40c8090 msgr2=0x7f97a4156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a3fff700 1 --2- 172.21.15.33:0/4145237464 >> v2:172.21.15.167:3300/0 conn(0x7f97a40c8090 0x7f97a4156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a3fff700 1 -- 172.21.15.33:0/4145237464 >> v1:172.21.15.33:6789/0 conn(0x7f97a4156880 legacy=0x7f97a40c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a3fff700 1 -- 172.21.15.33:0/4145237464 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a415a2f0 con 0x7f97a4159e10 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/4145237464 >> v2:172.21.15.167:3300/0 conn(0x7f97a40c8090 0x7f97a4156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.287+0000 7f97a3fff700 1 --2- 172.21.15.33:0/4145237464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a4159e10 0x7f97a4157190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f979c009e90 tx=0x7f979c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0b1241bf8b22aa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a27fc700 1 -- 172.21.15.33:0/4145237464 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00e070 con 0x7f97a4159e10 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a27fc700 1 -- 172.21.15.33:0/4145237464 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00bb40 con 0x7f97a4159e10 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a27fc700 1 -- 172.21.15.33:0/4145237464 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c012430 con 0x7f97a4159e10 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 -- 172.21.15.33:0/4145237464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a4159e10 msgr2=0x7f97a4157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 --2- 172.21.15.33:0/4145237464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a4159e10 0x7f97a4157190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f979c009e90 tx=0x7f979c009780 comp rx=0 tx=0).stop 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 -- 172.21.15.33:0/4145237464 shutdown_connections 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 --2- 172.21.15.33:0/4145237464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a4159e10 0x7f97a4157190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 --2- 172.21.15.33:0/4145237464 >> v2:172.21.15.167:3300/0 conn(0x7f97a40c8090 0x7f97a4156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 -- 172.21.15.33:0/4145237464 >> 172.21.15.33:0/4145237464 conn(0x7f97a406c560 msgr2=0x7f97a406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 -- 172.21.15.33:0/4145237464 shutdown_connections 2023-10-06T05:14:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.288+0000 7f97a899d700 1 -- 172.21.15.33:0/4145237464 wait complete. 2023-10-06T05:14:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.290+0000 7f97a899d700 1 Processor -- start 2023-10-06T05:14:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.290+0000 7f97a899d700 1 -- start start 2023-10-06T05:14:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a40c8090 0x7f97a414da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a4156880 0x7f97a413b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 0x7f97a414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40af580 con 0x7f97a4159e10 2023-10-06T05:14:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4150620 con 0x7f97a4156880 2023-10-06T05:14:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40af840 con 0x7f97a40c8090 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 0x7f97a414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a40c8090 0x7f97a414da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a4156880 0x7f97a413b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 0x7f97a414fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56310/0 (socket says 172.21.15.33:56310) 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 -- 172.21.15.33:0/3158856742 learned_addr learned my addr 172.21.15.33:0/3158856742 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a40c8090 0x7f97a414da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:44956/0 (socket says 172.21.15.33:44956) 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 -- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a40c8090 msgr2=0x7f97a414da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a40c8090 0x7f97a414da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 -- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a4156880 msgr2=0x7f97a413b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a4156880 0x7f97a413b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.291+0000 7f97a3fff700 1 -- 172.21.15.33:0/3158856742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798009420 con 0x7f97a4159e10 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.292+0000 7f97a3fff700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 0x7f97a414fd10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f979c003440 tx=0x7f979c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.292+0000 7f978b7fe700 1 -- 172.21.15.33:0/3158856742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c009420 con 0x7f97a4159e10 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.292+0000 7f978b7fe700 1 -- 172.21.15.33:0/3158856742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c027470 con 0x7f97a4159e10 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.292+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c009a20 con 0x7f97a4159e10 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.292+0000 7f978b7fe700 1 -- 172.21.15.33:0/3158856742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c027810 con 0x7f97a4159e10 2023-10-06T05:14:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.292+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a40afe00 con 0x7f97a4159e10 2023-10-06T05:14:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.295+0000 7f978b7fe700 1 -- 172.21.15.33:0/3158856742 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f979c024070 con 0x7f97a4159e10 2023-10-06T05:14:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.296+0000 7f978b7fe700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9790066f30 0x7f97900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.296+0000 7f978b7fe700 1 -- 172.21.15.33:0/3158856742 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f979c086ee0 con 0x7f97a4159e10 2023-10-06T05:14:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.297+0000 7f97a37fe700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9790066f30 0x7f97900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.297+0000 7f97a37fe700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9790066f30 0x7f97900693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9794007fd0 tx=0x7f9794009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.297+0000 7f97b8310b80 1 Processor -- start 2023-10-06T05:14:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 -- start start 2023-10-06T05:14:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x56178436ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 0x561784393920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561784393e60 0x561784366990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617842eac80 con 0x5617843b4bc0 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617842c1a70 con 0x561784393e60 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.298+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617842bf270 con 0x56178436a950 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 0x561784393920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 0x561784393920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56314/0 (socket says 172.21.15.33:56314) 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 -- 172.21.15.33:0/1073314697 learned_addr learned my addr 172.21.15.33:0/1073314697 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a37fe700 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x56178436ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a3fff700 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561784393e60 0x561784366990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 -- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 msgr2=0x56178436ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x56178436ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 -- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561784393e60 msgr2=0x561784366990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561784393e60 0x561784366990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 -- 172.21.15.33:0/1073314697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561784155270 con 0x5617843b4bc0 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a3fff700 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561784393e60 0x561784366990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 0x561784393920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9798009da0 tx=0x7f97980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af26f73f91e497a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.299+0000 7f9788ff9700 1 -- 172.21.15.33:0/1073314697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798011e20 con 0x5617843b4bc0 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f9788ff9700 1 -- 172.21.15.33:0/1073314697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979801d480 con 0x5617843b4bc0 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f9788ff9700 1 -- 172.21.15.33:0/1073314697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979801d800 con 0x5617843b4bc0 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 -- 172.21.15.33:0/1073314697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 msgr2=0x561784393920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 0x561784393920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9798009da0 tx=0x7f97980091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 -- 172.21.15.33:0/1073314697 shutdown_connections 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561784393e60 0x561784366990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x56178436ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1073314697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617843b4bc0 0x561784393920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 -- 172.21.15.33:0/1073314697 >> 172.21.15.33:0/1073314697 conn(0x56178436c270 msgr2=0x56178436c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 -- 172.21.15.33:0/1073314697 shutdown_connections 2023-10-06T05:14:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.300+0000 7f97b8310b80 1 -- 172.21.15.33:0/1073314697 wait complete. 2023-10-06T05:14:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.301+0000 7f97b8310b80 1 -- start start 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.301+0000 7f97b8310b80 1 Processor -- start 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784392570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x5617843ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617842bb060 con 0x561784393e60 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561784266890 con 0x5617843b4bc0 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561784281ea0 con 0x56178436a950 2023-10-06T05:14:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97b8310b80 10 client.0 _ll_register_callbacks cb 0x561784157f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x5617843ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x5617843ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56316/0 (socket says 172.21.15.33:56316) 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.302+0000 7f97a2ffd700 1 -- 172.21.15.33:0/3479954906 learned_addr learned my addr 172.21.15.33:0/3479954906 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a37fe700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784392570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a3fff700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a2ffd700 1 -- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 msgr2=0x561784392570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784392570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a2ffd700 1 -- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 msgr2=0x5617843a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a2ffd700 1 -- 172.21.15.33:0/3479954906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9798009a80 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x5617843ac9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9798009390 tx=0x7f979801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1083048dd4f428 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f9781ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f9781ffb700 1 -- 172.21.15.33:0/3479954906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798028470 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f9781ffb700 1 -- 172.21.15.33:0/3479954906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9798024070 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.303+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56178435f690 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f9781ffb700 1 -- 172.21.15.33:0/3479954906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97980287e0 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 10 client.4710 fetch_fsmap learned FSMap version 8 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5617840f7fe0 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f9781ffb700 1 -- 172.21.15.33:0/3479954906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56178435f690 con 0x561784393e60 2023-10-06T05:14:44.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f9781ffb700 1 -- 172.21.15.33:0/3479954906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9798028940 con 0x561784393e60 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 10 client.4710 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 10 client.4710 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 20 client.4710 populate_metadata read hostname 'smithi033' 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ab480 0x5617843ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 4 client.4710 mds_command: new command op to 4303 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 --> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56178433c5c0 con 0x5617843ab480 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.304+0000 7f97a37fe700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ab480 0x5617843ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.305+0000 7f97a37fe700 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ab480 0x5617843ab850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b3cd869625f1a2 server_cookie=dafea9881e614c15 in_seq=0 out_seq=0 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.305+0000 7f9781ffb700 10 client.4710 ms_handle_connect on v2:172.21.15.167:6835/256792669 2023-10-06T05:14:44.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f9781ffb700 1 -- 172.21.15.33:0/3479954906 <== mds.0 v2:172.21.15.167:6835/256792669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56178433c5c0 con 0x5617843ab480 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f9781ffb700 10 client.4710 handle_command_reply: tid=0 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 client.4710 shutdown 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 msgr2=0x5617843ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x5617843ac9d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9798009390 tx=0x7f979801d000 comp rx=0 tx=0).stop 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 shutdown_connections 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ab480 0x5617843ab850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784392570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 --2- 172.21.15.33:0/3479954906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x5617843ac9d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f9781ffb700 0 client.4710 ms_handle_reset on v2:172.21.15.167:6835/256792669 2023-10-06T05:14:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 >> 172.21.15.33:0/3479954906 conn(0x56178436c270 msgr2=0x56178436b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 shutdown_connections 2023-10-06T05:14:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.306+0000 7f97b8310b80 1 -- 172.21.15.33:0/3479954906 wait complete. 2023-10-06T05:14:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.307+0000 7f97b8310b80 20 client.4710 trim_cache size 0 max 16384 2023-10-06T05:14:44.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.324+0000 7f97b8310b80 1 Processor -- start 2023-10-06T05:14:44.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.324+0000 7f97b8310b80 1 -- start start 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 0x561784390b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561784393e60 0x561784367c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97b8310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178413de30 con 0x56178436a950 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561784392330 con 0x5617843b4bc0 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561784368f30 con 0x561784393e60 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 0x561784390b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 0x561784390b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56328/0 (socket says 172.21.15.33:56328) 2023-10-06T05:14:44.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 -- 172.21.15.33:0/1895436539 learned_addr learned my addr 172.21.15.33:0/1895436539 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a3fff700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 -- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561784393e60 msgr2=0x561784367c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561784393e60 0x561784367c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561784393e60 0x561784367c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 -- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 msgr2=0x5617843ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a37fe700 1 -- 172.21.15.33:0/1895436539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561784289780 con 0x56178436a950 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a3fff700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.325+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561784393e60 0x561784367c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97a37fe700 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 0x561784390b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97940058d0 tx=0x7f9794014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d754e576aff9a6c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97897fa700 1 -- 172.21.15.33:0/1895436539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794004830 con 0x56178436a950 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97897fa700 1 -- 172.21.15.33:0/1895436539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979402b070 con 0x56178436a950 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97897fa700 1 -- 172.21.15.33:0/1895436539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794004990 con 0x56178436a950 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 -- 172.21.15.33:0/1895436539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 msgr2=0x561784390b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 0x561784390b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97940058d0 tx=0x7f9794014000 comp rx=0 tx=0).stop 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 -- 172.21.15.33:0/1895436539 shutdown_connections 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x5617843ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56178436a950 0x561784390b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 --2- 172.21.15.33:0/1895436539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561784393e60 0x561784367c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.326+0000 7f97b8310b80 1 -- 172.21.15.33:0/1895436539 >> 172.21.15.33:0/1895436539 conn(0x56178436c270 msgr2=0x561784368710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.327+0000 7f97b8310b80 1 -- 172.21.15.33:0/1895436539 shutdown_connections 2023-10-06T05:14:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.327+0000 7f97b8310b80 1 -- 172.21.15.33:0/1895436539 wait complete. 2023-10-06T05:14:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.327+0000 7f97b8310b80 1 -- start start 2023-10-06T05:14:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.327+0000 7f97b8310b80 1 Processor -- start 2023-10-06T05:14:44.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784408a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x56178439a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x56178439a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178440cd10 con 0x561784393e60 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561784409250 con 0x5617843b4bc0 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56178440b540 con 0x56178436a950 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.328+0000 7f97b8310b80 10 client.0 _ll_register_callbacks cb 0x561784157f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x56178439a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x56178439a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56332/0 (socket says 172.21.15.33:56332) 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 -- 172.21.15.33:0/2518695857 learned_addr learned my addr 172.21.15.33:0/2518695857 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a37fe700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784408a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a3fff700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x56178439a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 -- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 msgr2=0x561784408a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784408a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 -- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 msgr2=0x56178439a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x56178439a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 -- 172.21.15.33:0/2518695857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9794019040 con 0x561784393e60 2023-10-06T05:14:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a37fe700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784408a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97a2ffd700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x56178439a260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9798009390 tx=0x7f9798021050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3114dd71dd9509c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f9781ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f9781ffb700 1 -- 172.21.15.33:0/2518695857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798028af0 con 0x561784393e60 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f979801d480 con 0x561784393e60 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.329+0000 7f9781ffb700 1 -- 172.21.15.33:0/2518695857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9798024070 con 0x561784393e60 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f9781ffb700 1 -- 172.21.15.33:0/2518695857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9798014650 con 0x561784393e60 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 10 client.4716 fetch_fsmap learned FSMap version 8 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561784270470 con 0x561784393e60 2023-10-06T05:14:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f9781ffb700 1 -- 172.21.15.33:0/2518695857 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979801d480 con 0x561784393e60 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f9781ffb700 1 -- 172.21.15.33:0/2518695857 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9798011650 con 0x561784393e60 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 10 client.4716 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 10 client.4716 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 20 client.4716 populate_metadata read hostname 'smithi033' 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ad6f0 0x5617843adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 4 client.4716 mds_command: new command op to 4303 tid=0[{"prefix": "status"}] 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 --> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56178425de00 con 0x5617843ad6f0 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.330+0000 7f97a37fe700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ad6f0 0x5617843adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.331+0000 7f97a37fe700 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ad6f0 0x5617843adac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7e805b96430036 server_cookie=656889e9cecc0a42 in_seq=0 out_seq=0 2023-10-06T05:14:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.331+0000 7f9781ffb700 10 client.4716 ms_handle_connect on v2:172.21.15.167:6835/256792669 2023-10-06T05:14:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.331+0000 7f9781ffb700 1 -- 172.21.15.33:0/2518695857 <== mds.0 v2:172.21.15.167:6835/256792669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56178425de00 con 0x5617843ad6f0 2023-10-06T05:14:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.331+0000 7f9781ffb700 10 client.4716 handle_command_reply: tid=0 2023-10-06T05:14:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.331+0000 7f97b8310b80 1 client.4716 shutdown 2023-10-06T05:14:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 msgr2=0x56178439a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x56178439a260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9798009390 tx=0x7f9798021050 comp rx=0 tx=0).stop 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 shutdown_connections 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617843b4bc0 0x56178439a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:6835/256792669,v1:172.21.15.167:6837/256792669] conn(0x5617843ad6f0 0x5617843adac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56178436a950 0x561784408a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 --2- 172.21.15.33:0/2518695857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561784393e60 0x56178439a260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f9781ffb700 0 client.4716 ms_handle_reset on v2:172.21.15.167:6835/256792669 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 >> 172.21.15.33:0/2518695857 conn(0x56178436c270 msgr2=0x5617843ac750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 shutdown_connections 2023-10-06T05:14:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 1 -- 172.21.15.33:0/2518695857 wait complete. 2023-10-06T05:14:44.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97b8310b80 20 client.4716 trim_cache size 0 max 16384 2023-10-06T05:14:44.343 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:14:44.343 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:14:44.343 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:14:44.343 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4303, 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 18.627502964000001, 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 8, 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 23, 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 23, 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 24.481398564999999 2023-10-06T05:14:44.344 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:14:44.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9790066f30 msgr2=0x7f97900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97a899d700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9790066f30 0x7f97900693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9794007fd0 tx=0x7f9794009040 comp rx=0 tx=0).stop 2023-10-06T05:14:44.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 msgr2=0x7f97a414fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.332+0000 7f97a899d700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 0x7f97a414fd10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f979c003440 tx=0x7f979c019000 comp rx=0 tx=0).stop 2023-10-06T05:14:44.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 shutdown_connections 2023-10-06T05:14:44.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9790066f30 0x7f97900693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a4159e10 0x7f97a414fd10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a40c8090 0x7f97a414da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 --2- 172.21.15.33:0/3158856742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a4156880 0x7f97a413b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 >> 172.21.15.33:0/3158856742 conn(0x7f97a406c560 msgr2=0x7f97a406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 shutdown_connections 2023-10-06T05:14:44.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:44.333+0000 7f97a899d700 1 -- 172.21.15.33:0/3158856742 wait complete. 2023-10-06T05:14:44.351 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T05:14:44.373 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T05:14:44.374 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T05:14:44.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.374 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:14:44.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-10-06T05:14:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2023-10-06T05:14:44.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:14:44.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.522 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:14:44.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-10-06T05:14:44.662 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2023-10-06T05:14:44.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:14:44.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.665 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:14:44.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.706 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2023-10-06T05:14:44.817 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2023-10-06T05:14:44.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:14:44.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.820 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:14:44.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:44.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2023-10-06T05:14:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2023-10-06T05:14:44.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:14:44.963 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-06T05:14:44.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:14:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.149+0000 7fb22ffff700 1 Processor -- start 2023-10-06T05:14:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.149+0000 7fb22ffff700 1 -- start start 2023-10-06T05:14:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2300c7920 0x7fb230160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23006ab60 con 0x7fb2300c8b50 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23006ae40 con 0x7fb230155af0 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23006b7e0 con 0x7fb2300c7920 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2300c7920 0x7fb230160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2300c7920 0x7fb230160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56346/0 (socket says 172.21.15.33:56346) 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1846440977 learned_addr learned my addr 172.21.15.33:0/1846440977 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22e7fc700 1 --1- 172.21.15.33:0/1846440977 >> v1:172.21.15.33:6789/0 conn(0x7fb2300c8b50 0x7fb2300c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41064/0 (socket says 172.21.15.33:41064) 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1846440977 >> v1:172.21.15.167:6789/0 conn(0x7fb230155af0 legacy=0x7fb23006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1846440977 >> v1:172.21.15.33:6789/0 conn(0x7fb2300c8b50 legacy=0x7fb2300c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22dffb700 1 -- 172.21.15.33:0/1846440977 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193028483 0 0) 0x7fb23006ae40 con 0x7fb230155af0 2023-10-06T05:14:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.151+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1846440977 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23006b120 con 0x7fb2300c7920 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22f7fe700 1 --2- 172.21.15.33:0/1846440977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c7920 0x7fb230160c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb228009f20 tx=0x7fb228009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa11be895da542e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22dffb700 1 -- 172.21.15.33:0/1846440977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800e070 con 0x7fb2300c7920 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22dffb700 1 -- 172.21.15.33:0/1846440977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800ba80 con 0x7fb2300c7920 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22dffb700 1 -- 172.21.15.33:0/1846440977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280092e0 con 0x7fb2300c7920 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22ffff700 1 -- 172.21.15.33:0/1846440977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c7920 msgr2=0x7fb230160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1846440977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c7920 0x7fb230160c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb228009f20 tx=0x7fb228009840 comp rx=0 tx=0).stop 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22ffff700 1 -- 172.21.15.33:0/1846440977 shutdown_connections 2023-10-06T05:14:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1846440977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c7920 0x7fb230160c30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22ffff700 1 -- 172.21.15.33:0/1846440977 >> 172.21.15.33:0/1846440977 conn(0x7fb23006d030 msgr2=0x7fb23006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.152+0000 7fb22ffff700 1 -- 172.21.15.33:0/1846440977 shutdown_connections 2023-10-06T05:14:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.153+0000 7fb22ffff700 1 -- 172.21.15.33:0/1846440977 wait complete. 2023-10-06T05:14:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.154+0000 7fb22ffff700 1 Processor -- start 2023-10-06T05:14:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.154+0000 7fb22ffff700 1 -- start start 2023-10-06T05:14:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2300c7920 0x7fb2301455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2300c8b50 0x7fb2300c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 0x7fb2300c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300ac010 con 0x7fb230155af0 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300acb50 con 0x7fb2300c8b50 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300ac220 con 0x7fb2300c7920 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 0x7fb2300c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 0x7fb2300c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56360/0 (socket says 172.21.15.33:56360) 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.155+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1683935810 learned_addr learned my addr 172.21.15.33:0/1683935810 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22e7fc700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2300c8b50 0x7fb2300c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22effd700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2300c7920 0x7fb2301455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2300c7920 msgr2=0x7fb2301455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22f7fe700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2300c7920 0x7fb2301455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2300c8b50 msgr2=0x7fb2300c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22f7fe700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2300c8b50 0x7fb2300c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22f7fe700 1 -- 172.21.15.33:0/1683935810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009ae0 con 0x7fb230155af0 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22f7fe700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 0x7fb2300c6e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb228012000 tx=0x7fb228018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228007e70 con 0x7fb230155af0 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228026460 con 0x7fb230155af0 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228026820 con 0x7fb230155af0 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300ac500 con 0x7fb230155af0 2023-10-06T05:14:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.156+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300ab6f0 con 0x7fb230155af0 2023-10-06T05:14:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.157+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23004c380 con 0x7fb230155af0 2023-10-06T05:14:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.158+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb228026b90 con 0x7fb230155af0 2023-10-06T05:14:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.159+0000 7fb216ffd700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.159+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb2280867d0 con 0x7fb230155af0 2023-10-06T05:14:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.160+0000 7fb22effd700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.160+0000 7fb22effd700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb21c066f30 0x7fb21c0693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb230069b70 tx=0x7fb220009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.164+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb228053840 con 0x7fb230155af0 2023-10-06T05:14:45.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.412+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb23006b400 con 0x7fb230155af0 2023-10-06T05:14:45.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.413+0000 7fb216ffd700 1 -- 172.21.15.33:0/1683935810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb228056e60 con 0x7fb230155af0 2023-10-06T05:14:45.414 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:14:45.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb21c066f30 msgr2=0x7fb21c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb21c066f30 0x7fb21c0693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb230069b70 tx=0x7fb220009040 comp rx=0 tx=0).stop 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 msgr2=0x7fb2300c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 0x7fb2300c6e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb228012000 tx=0x7fb228018000 comp rx=0 tx=0).stop 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 shutdown_connections 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb230155af0 0x7fb2300c6e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2300c7920 0x7fb2301455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 --2- 172.21.15.33:0/1683935810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2300c8b50 0x7fb2300c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 >> 172.21.15.33:0/1683935810 conn(0x7fb23006d030 msgr2=0x7fb2300ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 shutdown_connections 2023-10-06T05:14:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:45.415+0000 7fb22ffff700 1 -- 172.21.15.33:0/1683935810 wait complete. 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:14:45.434 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:14:45.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:45.435 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:14:45.471 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:45.471 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:45.471 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:45.471 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:45.471 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:14:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42272sec preferred_lft 42272sec 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:14:45.473 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:14:45.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:14:45.474 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:45.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:45.927 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:14:45.992 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:14:45.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:45.995 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:14:46.048 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:14:46.048 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:14:46.049 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:14:46.049 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:14:46.049 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:14:46.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:46.050 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:46.050 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:46.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:14:46.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:14:46.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:14:46.051 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:46.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:46.734 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:14:46.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:46.772 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-10-06T05:14:46.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:46.850 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:46.850 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:46.850 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:14:46.850 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T05:14:46.850 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:47.019 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T05:14:47.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:47.019 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T05:14:47.020 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:47.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:47.096 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:47.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:47.296 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:47.296 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:47.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:14:47.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:14:47.297 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:47.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:47.455 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:14:47.491 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:14:47.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:47.492 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:14:47.568 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:14:47.568 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:14:47.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:14:47.712 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:14:47.712 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:14:47.713 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:14:48.141 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:14:48.141 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:14:48.141 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:14:48.141 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:48.141 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:48.164 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:14:48.165 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:14:48.170 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:14:48.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:48.170 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:14:48.233 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-06T05:14:48.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:14:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.418+0000 7f060659d700 1 Processor -- start 2023-10-06T05:14:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.418+0000 7f060659d700 1 -- start start 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f060659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060013faa0 0x7f060013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f060659d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f06000c7940 0x7f06000c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f060659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06000c9cb0 con 0x7f060006aad0 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f060659d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060006b660 con 0x7f06000c7940 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f060659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000ca270 con 0x7f060013faa0 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f060559b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060013faa0 0x7f060013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f0605d9c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f060006aad0 0x7f060006af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:45692/0 (socket says 172.21.15.33:45692) 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f0605d9c700 1 -- 172.21.15.33:0/498908776 learned_addr learned my addr 172.21.15.33:0/498908776 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.420+0000 7f0604d9a700 1 --2- 172.21.15.33:0/498908776 >> v2:172.21.15.167:3300/0 conn(0x7f06000c7940 0x7f06000c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f05f7fff700 1 -- 172.21.15.33:0/498908776 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233781179 0 0) 0x7f06000c9cb0 con 0x7f060006aad0 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f05f7fff700 1 -- 172.21.15.33:0/498908776 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec003570 con 0x7f060006aad0 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f0604d9a700 1 -- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060013faa0 msgr2=0x7f060013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f0604d9a700 1 --2- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060013faa0 0x7f060013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f0604d9a700 1 -- 172.21.15.33:0/498908776 >> v1:172.21.15.167:6789/0 conn(0x7f060006aad0 legacy=0x7f060006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f0604d9a700 1 -- 172.21.15.33:0/498908776 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060006b840 con 0x7f06000c7940 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f060559b700 1 --2- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060013faa0 0x7f060013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f0604d9a700 1 --2- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f06000c7db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05f00098e0 tx=0x7f05f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb680ace899e2c53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f05f7fff700 1 -- 172.21.15.33:0/498908776 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0011630 con 0x7f06000c7940 2023-10-06T05:14:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f05f7fff700 1 -- 172.21.15.33:0/498908776 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0011790 con 0x7f06000c7940 2023-10-06T05:14:48.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.421+0000 7f05f7fff700 1 -- 172.21.15.33:0/498908776 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0011b30 con 0x7f06000c7940 2023-10-06T05:14:48.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 -- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 msgr2=0x7f06000c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 --2- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f06000c7db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05f00098e0 tx=0x7f05f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 -- 172.21.15.33:0/498908776 shutdown_connections 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 --2- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060013faa0 0x7f060013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 --2- 172.21.15.33:0/498908776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f06000c7db0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 -- 172.21.15.33:0/498908776 >> 172.21.15.33:0/498908776 conn(0x7f060006d030 msgr2=0x7f060006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 -- 172.21.15.33:0/498908776 shutdown_connections 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.422+0000 7f060659d700 1 -- 172.21.15.33:0/498908776 wait complete. 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.424+0000 7f060659d700 1 Processor -- start 2023-10-06T05:14:48.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.424+0000 7f060659d700 1 -- start start 2023-10-06T05:14:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060006aad0 0x7f0600156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f060012f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 0x7f0600130d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0600157140 con 0x7f060013faa0 2023-10-06T05:14:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060012fc70 con 0x7f06000c7940 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060012ff50 con 0x7f060006aad0 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 0x7f0600130d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 0x7f0600130d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56366/0 (socket says 172.21.15.33:56366) 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 -- 172.21.15.33:0/1936552938 learned_addr learned my addr 172.21.15.33:0/1936552938 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f060559b700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060006aad0 0x7f0600156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0604d9a700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f060012f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 -- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060006aad0 msgr2=0x7f0600156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060006aad0 0x7f0600156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 -- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 msgr2=0x7f060012f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f060012f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.425+0000 7f0605d9c700 1 -- 172.21.15.33:0/1936552938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc009420 con 0x7f060013faa0 2023-10-06T05:14:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.426+0000 7f0605d9c700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 0x7f0600130d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05f8001460 tx=0x7f05f8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.426+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800a890 con 0x7f060013faa0 2023-10-06T05:14:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.426+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800a9f0 con 0x7f060013faa0 2023-10-06T05:14:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.426+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800ad70 con 0x7f060013faa0 2023-10-06T05:14:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.426+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f0009420 con 0x7f060013faa0 2023-10-06T05:14:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.426+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0600130510 con 0x7f060013faa0 2023-10-06T05:14:48.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.427+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060004c380 con 0x7f060013faa0 2023-10-06T05:14:48.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.428+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05f8007300 con 0x7f060013faa0 2023-10-06T05:14:48.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.429+0000 7f05f57fa700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05e4066f30 0x7f05e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:48.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.429+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f05f8084a70 con 0x7f060013faa0 2023-10-06T05:14:48.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.429+0000 7f060559b700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05e4066f30 0x7f05e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:48.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.430+0000 7f060559b700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05e4066f30 0x7f05e40693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009ea0 tx=0x7f05fc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:48.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.433+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05f8051b60 con 0x7f060013faa0 2023-10-06T05:14:48.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.683+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f060006bb20 con 0x7f060013faa0 2023-10-06T05:14:48.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.683+0000 7f05f57fa700 1 -- 172.21.15.33:0/1936552938 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f05f8055180 con 0x7f060013faa0 2023-10-06T05:14:48.685 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:14:48.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05e4066f30 msgr2=0x7f05e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05e4066f30 0x7f05e40693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009ea0 tx=0x7f05fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 msgr2=0x7f0600130d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 0x7f0600130d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05f8001460 tx=0x7f05f8001ae0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 shutdown_connections 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05e4066f30 0x7f05e40693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060013faa0 0x7f0600130d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f060006aad0 0x7f0600156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 --2- 172.21.15.33:0/1936552938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06000c7940 0x7f060012f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 >> 172.21.15.33:0/1936552938 conn(0x7f060006d030 msgr2=0x7f06000caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:48.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 shutdown_connections 2023-10-06T05:14:48.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:48.686+0000 7f060659d700 1 -- 172.21.15.33:0/1936552938 wait complete. 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:14:48.705 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T05:14:48.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:48.706 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:48.745 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42269sec preferred_lft 42269sec 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:48.746 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:14:48.747 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:48.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:48.748 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:14:48.845 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:14:48.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:48.848 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:14:48.887 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:14:48.887 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:14:48.888 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:14:48.888 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:14:48.888 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:14:48.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:48.888 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:48.889 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:48.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:14:48.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:14:48.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:14:48.889 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:49.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.087 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:14:49.123 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:14:49.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.124 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-10-06T05:14:49.200 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T05:14:49.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.200 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:49.201 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:49.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:14:49.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T05:14:49.201 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:49.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:14:49.431 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:e5:0e:f4:e3:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:14:49.432 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:14:49.432 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.432 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c8e5:eff:fef4:e3fd/64 scope link 2023-10-06T05:14:49.432 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:e5:0e:f4:e3:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:14:49.497 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.498 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c8e5:eff:fef4:e3fd/64 scope link 2023-10-06T05:14:49.498 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:49.500 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T05:14:49.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T05:14:49.500 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:49.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:49.738 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:14:49.738 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:14:49.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.1 up 2023-10-06T05:14:49.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T05:14:49.739 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:14:49.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:49.898 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T05:14:49.934 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T05:14:49.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:49.934 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T05:14:50.010 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:14:50.010 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:14:50.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T05:14:50.141 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T05:14:50.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T05:14:50.142 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:14:50.158 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T05:14:50.159 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T05:14:50.159 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:14:50.159 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:50.159 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:50.179 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T05:14:50.179 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T05:14:50.179 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:14:50.180 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:50.180 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:50.180 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:14:50.180 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:14:50.180 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:14:50.185 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:14:50.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:50.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T05:14:50.248 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-06T05:14:50.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:14:50.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.432+0000 7f44e759e700 1 Processor -- start 2023-10-06T05:14:50.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.432+0000 7f44e759e700 1 -- start start 2023-10-06T05:14:50.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f44e80c8050 0x7f44e813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 0x7f44e806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e80c9cb0 con 0x7f44e813faa0 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80ca270 con 0x7f44e80c8050 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e806b840 con 0x7f44e806aad0 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f44e813faa0 0x7f44e80c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46548/0 (socket says 172.21.15.33:46548) 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e659c700 1 -- 172.21.15.33:0/3539206398 learned_addr learned my addr 172.21.15.33:0/3539206398 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e6d9d700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 0x7f44e806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/3539206398 >> v2:172.21.15.167:3300/0 conn(0x7f44e80c8050 0x7f44e813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.434+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201969690 0 0) 0x7f44e80c9cb0 con 0x7f44e813faa0 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d0003570 con 0x7f44e813faa0 2023-10-06T05:14:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 516448675 0 0) 0x7f44d0003570 con 0x7f44e813faa0 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 msgr2=0x7f44e806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 0x7f44e806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 >> v2:172.21.15.167:3300/0 conn(0x7f44e80c8050 msgr2=0x7f44e813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 --2- 172.21.15.33:0/3539206398 >> v2:172.21.15.167:3300/0 conn(0x7f44e80c8050 0x7f44e813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e806bb20 con 0x7f44e813faa0 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f44d8001470 con 0x7f44e813faa0 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 >> v1:172.21.15.33:6789/0 conn(0x7f44e813faa0 legacy=0x7f44e80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0003a90 0x7f44d0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d0004810 0x7f44d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 0x7f44d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80ca270 con 0x7f44d0004810 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e806b840 con 0x7f44d0003a90 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d000ba60 con 0x7f44e806aad0 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d0004810 0x7f44d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e659c700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0003a90 0x7f44d0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.435+0000 7f44e6d9d700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 0x7f44d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e5d9b700 1 -- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 msgr2=0x7f44d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e806aad0 0x7f44d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e5d9b700 1 -- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0003a90 msgr2=0x7f44d0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0003a90 0x7f44d0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e5d9b700 1 -- 172.21.15.33:0/3539206398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0003570 con 0x7f44d0004810 2023-10-06T05:14:50.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d0004810 0x7f44d0004c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009c30 tx=0x7f44dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efdf2286854213e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00e710 con 0x7f44d0004810 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc00e870 con 0x7f44d0004810 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.436+0000 7f44e559a700 1 -- 172.21.15.33:0/3539206398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00ec30 con 0x7f44d0004810 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e759e700 1 -- 172.21.15.33:0/3539206398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d0004810 msgr2=0x7f44d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e759e700 1 --2- 172.21.15.33:0/3539206398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44d0004810 0x7f44d0004c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009c30 tx=0x7f44dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e6d9d700 1 -- 172.21.15.33:0/3539206398 reap_dead start 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e759e700 1 -- 172.21.15.33:0/3539206398 shutdown_connections 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e759e700 1 -- 172.21.15.33:0/3539206398 >> 172.21.15.33:0/3539206398 conn(0x7f44e806d030 msgr2=0x7f44e806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e759e700 1 -- 172.21.15.33:0/3539206398 shutdown_connections 2023-10-06T05:14:50.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.437+0000 7f44e759e700 1 -- 172.21.15.33:0/3539206398 wait complete. 2023-10-06T05:14:50.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.438+0000 7f44e759e700 1 Processor -- start 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.438+0000 7f44e759e700 1 -- start start 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0004810 0x7f44e815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 0x7f44e815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80c27f0 0x7f44e815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8160910 con 0x7f44e80c2380 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e815bcb0 con 0x7f44d0004810 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80c4730 con 0x7f44e80c27f0 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.439+0000 7f44e5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 0x7f44e815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 0x7f44e815b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46516/0 (socket says 172.21.15.33:46516) 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 -- 172.21.15.33:0/733222151 learned_addr learned my addr 172.21.15.33:0/733222151 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e659c700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0004810 0x7f44e815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 -- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80c27f0 msgr2=0x7f44e815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80c27f0 0x7f44e815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 -- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0004810 msgr2=0x7f44e815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0004810 0x7f44e815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 -- 172.21.15.33:0/733222151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80c49b0 con 0x7f44e80c2380 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e659c700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0004810 0x7f44e815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e5d9b700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 0x7f44e815b230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009fa0 tx=0x7f44dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc023070 con 0x7f44e80c2380 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc00e9e0 con 0x7f44e80c2380 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e80c4c90 con 0x7f44e80c2380 2023-10-06T05:14:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.440+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc01e030 con 0x7f44e80c2380 2023-10-06T05:14:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.441+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e80c5250 con 0x7f44e80c2380 2023-10-06T05:14:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.441+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e80c55c0 con 0x7f44e80c2380 2023-10-06T05:14:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.442+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44dc013470 con 0x7f44e80c2380 2023-10-06T05:14:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.443+0000 7f44d67fc700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f44c8066f30 0x7f44c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.444+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44dc085ee0 con 0x7f44e80c2380 2023-10-06T05:14:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.444+0000 7f44e659c700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f44c8066f30 0x7f44c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.444+0000 7f44e659c700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f44c8066f30 0x7f44c80693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000e10 tx=0x7f44d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.448+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44dc052f90 con 0x7f44e80c2380 2023-10-06T05:14:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.698+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f44e804c380 con 0x7f44e80c2380 2023-10-06T05:14:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.699+0000 7f44d67fc700 1 -- 172.21.15.33:0/733222151 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f44dc017070 con 0x7f44e80c2380 2023-10-06T05:14:50.701 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:14:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f44c8066f30 msgr2=0x7f44c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f44c8066f30 0x7f44c80693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000e10 tx=0x7f44d8009040 comp rx=0 tx=0).stop 2023-10-06T05:14:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 msgr2=0x7f44e815b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 0x7f44e815b230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009fa0 tx=0x7f44dc01a000 comp rx=0 tx=0).stop 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 shutdown_connections 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f44c8066f30 0x7f44c80693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44e80c27f0 0x7f44e815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d0004810 0x7f44e815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 --2- 172.21.15.33:0/733222151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44e80c2380 0x7f44e815b230 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.701+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 >> 172.21.15.33:0/733222151 conn(0x7f44e806d030 msgr2=0x7f44e80b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.702+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 shutdown_connections 2023-10-06T05:14:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:50.702+0000 7f44e759e700 1 -- 172.21.15.33:0/733222151 wait complete. 2023-10-06T05:14:50.721 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:14:50.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:14:50.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:14:50.721 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:14:50.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:14:50.721 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:14:50.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:14:50.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:14:50.722 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:14:50.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:50.722 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:14:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42270sec preferred_lft 42270sec 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:14:50.764 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:50.765 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:14:50.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:14:50.766 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:51.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:51.213 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:14:51.275 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:14:51.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:51.278 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:14:51.328 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:14:51.328 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:14:51.329 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:14:51.329 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:14:51.329 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:14:51.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:51.330 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:51.330 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:51.330 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:14:51.330 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:14:51.330 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:14:51.330 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:52.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:52.017 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:14:52.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:52.054 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2023-10-06T05:14:52.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:52.130 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:52.130 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:52.130 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:14:52.130 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T05:14:52.130 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:52.295 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T05:14:52.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:52.295 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:52.295 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:52.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T05:14:52.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:14:52.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T05:14:52.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T05:14:52.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T05:14:52.296 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:52.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:52.547 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:52.547 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:52.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:14:52.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:14:52.548 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:52.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:52.707 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:14:52.743 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:14:52.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:52.744 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:14:52.818 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:14:52.819 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:14:52.819 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:14:52.971 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:14:52.971 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:14:52.971 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:14:53.408 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:14:53.410 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:14:53.410 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:14:53.410 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:53.410 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:53.434 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:14:53.434 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:14:53.434 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:14:53.434 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:53.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:53.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:14:53.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:14:53.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:14:53.439 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:14:53.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:53.439 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:14:53.500 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-06T05:14:53.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:14:53.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.697+0000 7f9967fff700 1 Processor -- start 2023-10-06T05:14:53.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.697+0000 7f9967fff700 1 -- start start 2023-10-06T05:14:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9967fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99680ae7e0 0x7f99680c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9967fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996806a0d0 con 0x7f9968157400 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9967fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680c9210 con 0x7f996815be30 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9967fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996806a310 con 0x7f99680ae7e0 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9966ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9968157400 0x7f99680699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46554/0 (socket says 172.21.15.33:46554) 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9966ffd700 1 -- 172.21.15.33:0/1685120831 learned_addr learned my addr 172.21.15.33:0/1685120831 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f99677fe700 1 --2- 172.21.15.33:0/1685120831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99680ae7e0 0x7f99680c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.699+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550834308 0 0) 0x7f99680c9210 con 0x7f996815be30 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9950003570 con 0x7f996815be30 2023-10-06T05:14:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942728458 0 0) 0x7f9958001040 con 0x7f9968157400 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99680c9210 con 0x7f9968157400 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f99677fe700 1 -- 172.21.15.33:0/1685120831 >> v1:172.21.15.167:6789/0 conn(0x7f996815be30 legacy=0x7f99680ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f99677fe700 1 -- 172.21.15.33:0/1685120831 >> v1:172.21.15.33:6789/0 conn(0x7f9968157400 legacy=0x7f99680699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f99677fe700 1 -- 172.21.15.33:0/1685120831 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680c8bc0 con 0x7f99680ae7e0 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f99677fe700 1 --2- 172.21.15.33:0/1685120831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99680ae7e0 0x7f99680c96a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9960009e90 tx=0x7f9960009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb53cda088755f3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000e070 con 0x7f99680ae7e0 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996000bb80 con 0x7f99680ae7e0 2023-10-06T05:14:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.700+0000 7f9965ffb700 1 -- 172.21.15.33:0/1685120831 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960012430 con 0x7f99680ae7e0 2023-10-06T05:14:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 -- 172.21.15.33:0/1685120831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99680ae7e0 msgr2=0x7f99680c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 --2- 172.21.15.33:0/1685120831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99680ae7e0 0x7f99680c96a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9960009e90 tx=0x7f9960009780 comp rx=0 tx=0).stop 2023-10-06T05:14:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 -- 172.21.15.33:0/1685120831 shutdown_connections 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 --2- 172.21.15.33:0/1685120831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99680ae7e0 0x7f99680c96a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 -- 172.21.15.33:0/1685120831 >> 172.21.15.33:0/1685120831 conn(0x7f996806d030 msgr2=0x7f996806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 -- 172.21.15.33:0/1685120831 shutdown_connections 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.701+0000 7f9967fff700 1 -- 172.21.15.33:0/1685120831 wait complete. 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.702+0000 7f9967fff700 1 Processor -- start 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 -- start start 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 0x7f9968146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9968157400 0x7f996806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996815be30 0x7f996806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996815a020 con 0x7f99680ae7e0 2023-10-06T05:14:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996815a300 con 0x7f9968157400 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.703+0000 7f9967fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996815a5e0 con 0x7f996815be30 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 0x7f9968146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 0x7f9968146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46532/0 (socket says 172.21.15.33:46532) 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 -- 172.21.15.33:0/3523946502 learned_addr learned my addr 172.21.15.33:0/3523946502 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f99677fe700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996815be30 0x7f996806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f99667fc700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9968157400 0x7f996806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 -- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996815be30 msgr2=0x7f996806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996815be30 0x7f996806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 -- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9968157400 msgr2=0x7f996806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9968157400 0x7f996806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 -- 172.21.15.33:0/3523946502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960009a20 con 0x7f99680ae7e0 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f99667fc700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9968157400 0x7f996806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.704+0000 7f9966ffd700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 0x7f9968146f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f995800c790 tx=0x7f9958007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.705+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958005b30 con 0x7f99680ae7e0 2023-10-06T05:14:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.705+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9958005cb0 con 0x7f99680ae7e0 2023-10-06T05:14:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.705+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958007320 con 0x7f99680ae7e0 2023-10-06T05:14:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.705+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996815a8c0 con 0x7f99680ae7e0 2023-10-06T05:14:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.705+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99681474b0 con 0x7f99680ae7e0 2023-10-06T05:14:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.706+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996815ade0 con 0x7f99680ae7e0 2023-10-06T05:14:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.707+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f995801b3b0 con 0x7f99680ae7e0 2023-10-06T05:14:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.708+0000 7f994effd700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9954066f30 0x7f99540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.708+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9958083b30 con 0x7f99680ae7e0 2023-10-06T05:14:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.708+0000 7f99667fc700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9954066f30 0x7f99540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.709+0000 7f99667fc700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9954066f30 0x7f99540693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f99680ae7b0 tx=0x7f995c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.712+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9958050ba0 con 0x7f99680ae7e0 2023-10-06T05:14:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.960+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f996806a0d0 con 0x7f99680ae7e0 2023-10-06T05:14:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.960+0000 7f994effd700 1 -- 172.21.15.33:0/3523946502 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f996806a0d0 con 0x7f99680ae7e0 2023-10-06T05:14:53.962 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:14:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9954066f30 msgr2=0x7f99540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9954066f30 0x7f99540693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f99680ae7b0 tx=0x7f995c009040 comp rx=0 tx=0).stop 2023-10-06T05:14:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 msgr2=0x7f9968146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 0x7f9968146f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f995800c790 tx=0x7f9958007cf0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 shutdown_connections 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9954066f30 0x7f99540693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f996815be30 0x7f996806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99680ae7e0 0x7f9968146f70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 --2- 172.21.15.33:0/3523946502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9968157400 0x7f996806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 >> 172.21.15.33:0/3523946502 conn(0x7f996806d030 msgr2=0x7f9968147e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.962+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 shutdown_connections 2023-10-06T05:14:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:53.963+0000 7f9967fff700 1 -- 172.21.15.33:0/3523946502 wait complete. 2023-10-06T05:14:53.981 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:14:53.982 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T05:14:53.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:53.983 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:14:54.020 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42267sec preferred_lft 42267sec 2023-10-06T05:14:54.022 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:14:54.024 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:14:54.024 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link tentative 2023-10-06T05:14:54.024 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.024 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:14:54.119 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:14:54.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.122 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:14:54.160 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:14:54.160 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:14:54.160 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:14:54.160 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:14:54.160 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:14:54.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.161 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:54.162 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:54.162 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:14:54.162 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:14:54.162 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:14:54.162 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:54.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.356 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.394 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2023-10-06T05:14:54.469 INFO:teuthology.orchestra.run.smithi167.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T05:14:54.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.470 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:54.470 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:54.470 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:14:54.470 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T05:14:54.470 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:54.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.635 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T05:14:54.697 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:54.697 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 86:ef:ad:00:88:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.698 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::84ef:adff:fe00:8861/64 scope link 2023-10-06T05:14:54.699 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.700 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T05:14:54.762 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:14:54.762 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:14:54.763 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:14:54.763 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.763 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:14:54.763 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.763 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 86:ef:ad:00:88:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::84ef:adff:fe00:8861/64 scope link 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:14:54.766 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T05:14:54.768 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:54.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:54.999 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:14:54.999 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:14:54.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.1 up 2023-10-06T05:14:55.000 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T05:14:55.000 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:14:55.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:55.156 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T05:14:55.192 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T05:14:55.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:14:55.192 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T05:14:55.266 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:14:55.267 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:14:55.267 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T05:14:55.396 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T05:14:55.397 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T05:14:55.397 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:14:55.412 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T05:14:55.412 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T05:14:55.412 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:14:55.413 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:55.413 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:55.432 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T05:14:55.432 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T05:14:55.432 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:14:55.433 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:14:55.433 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:14:55.433 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:14:55.433 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:14:55.433 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:14:55.436 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:14:55.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:14:55.437 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T05:14:55.497 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T05:14:55.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:55.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.712+0000 7f681239d700 1 Processor -- start 2023-10-06T05:14:55.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.712+0000 7f681239d700 1 -- start start 2023-10-06T05:14:55.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f681239d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f680c0c8050 0x7f680c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f681239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0c9cb0 con 0x7f680c13faa0 2023-10-06T05:14:55.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f681239d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0ca270 con 0x7f680c06aad0 2023-10-06T05:14:55.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f681239d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c06b640 con 0x7f680c0c8050 2023-10-06T05:14:55.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f681139b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f680c13faa0 0x7f680c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46556/0 (socket says 172.21.15.33:46556) 2023-10-06T05:14:55.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f681139b700 1 -- 172.21.15.33:0/1414076760 learned_addr learned my addr 172.21.15.33:0/1414076760 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:55.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f6810b9a700 1 --2- 172.21.15.33:0/1414076760 >> v2:172.21.15.167:3300/0 conn(0x7f680c0c8050 0x7f680c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.714+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375175652 0 0) 0x7f680c0c9cb0 con 0x7f680c13faa0 2023-10-06T05:14:55.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67f8003570 con 0x7f680c13faa0 2023-10-06T05:14:55.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019396851 0 0) 0x7f680c0ca270 con 0x7f680c06aad0 2023-10-06T05:14:55.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f680c0c9cb0 con 0x7f680c06aad0 2023-10-06T05:14:55.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950370676 0 0) 0x7f67f8003570 con 0x7f680c13faa0 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 >> v2:172.21.15.167:3300/0 conn(0x7f680c0c8050 msgr2=0x7f680c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 --2- 172.21.15.33:0/1414076760 >> v2:172.21.15.167:3300/0 conn(0x7f680c0c8050 0x7f680c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 >> v1:172.21.15.167:6789/0 conn(0x7f680c06aad0 legacy=0x7f680c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c06b840 con 0x7f680c13faa0 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f6808001470 con 0x7f680c13faa0 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 >> v1:172.21.15.33:6789/0 conn(0x7f680c13faa0 legacy=0x7f680c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67f8003bf0 0x7f67f8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67f8004810 0x7f67f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f80051c0 0x7f67f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c06b640 con 0x7f67f8004810 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f800c0b0 con 0x7f67f8003bf0 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f800c270 con 0x7f67f80051c0 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6810b9a700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67f8004810 0x7f67f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f681139b700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67f8003bf0 0x7f67f8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.715+0000 7f6811b9c700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f80051c0 0x7f67f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6810b9a700 1 -- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f80051c0 msgr2=0x7f67f800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6810b9a700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f80051c0 0x7f67f800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6810b9a700 1 -- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67f8003bf0 msgr2=0x7f67f8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6810b9a700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67f8003bf0 0x7f67f8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6810b9a700 1 -- 172.21.15.33:0/1414076760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8003570 con 0x7f67f8004810 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f681139b700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67f8003bf0 0x7f67f8003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6811b9c700 1 -- 172.21.15.33:0/1414076760 reap_dead start 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6810b9a700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67f8004810 0x7f67f8004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009c00 tx=0x7f67fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361002db5217e0b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00e710 con 0x7f67f8004810 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00e870 con 0x7f67f8004810 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.716+0000 7f6803fff700 1 -- 172.21.15.33:0/1414076760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00ec30 con 0x7f67f8004810 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 -- 172.21.15.33:0/1414076760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67f8004810 msgr2=0x7f67f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67f8004810 0x7f67f8004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009c00 tx=0x7f67fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 -- 172.21.15.33:0/1414076760 shutdown_connections 2023-10-06T05:14:55.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 --2- 172.21.15.33:0/1414076760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67f8004810 0x7f67f8004c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 -- 172.21.15.33:0/1414076760 >> 172.21.15.33:0/1414076760 conn(0x7f680c06d030 msgr2=0x7f680c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 -- 172.21.15.33:0/1414076760 shutdown_connections 2023-10-06T05:14:55.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.717+0000 7f681239d700 1 -- 172.21.15.33:0/1414076760 wait complete. 2023-10-06T05:14:55.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.718+0000 7f681239d700 1 Processor -- start 2023-10-06T05:14:55.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.719+0000 7f681239d700 1 -- start start 2023-10-06T05:14:55.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f8004810 0x7f680c149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 0x7f680c13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f680c0c47e0 0x7f680c0c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0c5fa0 con 0x7f680c155720 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0c5670 con 0x7f680c0c47e0 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0c5950 con 0x7f67f8004810 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 0x7f680c13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 0x7f680c13e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46546/0 (socket says 172.21.15.33:46546) 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 -- 172.21.15.33:0/2823246813 learned_addr learned my addr 172.21.15.33:0/2823246813 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f681139b700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f8004810 0x7f680c149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6811b9c700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f680c0c47e0 0x7f680c0c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 -- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f8004810 msgr2=0x7f680c149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f8004810 0x7f680c149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 -- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f680c0c47e0 msgr2=0x7f680c0c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f680c0c47e0 0x7f680c0c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.720+0000 7f6810b9a700 1 -- 172.21.15.33:0/2823246813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0c5c30 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.721+0000 7f6810b9a700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 0x7f680c13e5f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f67f800c080 tx=0x7f67fc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.721+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc022070 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.721+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00e9e0 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.721+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0276c0 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.721+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c149c80 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.721+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c0c51a0 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.722+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c0c53c0 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.723+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67fc01c450 con 0x7f680c155720 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.724+0000 7f68017fa700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67f0066f30 0x7f67f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.724+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f67fc0860a0 con 0x7f680c155720 2023-10-06T05:14:55.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.724+0000 7f681139b700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67f0066f30 0x7f67f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.725+0000 7f681139b700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67f0066f30 0x7f67f00693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f680c065b00 tx=0x7f680c0c84c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.728+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67fc053110 con 0x7f680c155720 2023-10-06T05:14:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.934+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f680c04c380 con 0x7f680c155720 2023-10-06T05:14:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.935+0000 7f68017fa700 1 -- 172.21.15.33:0/2823246813 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f67fc006cb0 con 0x7f680c155720 2023-10-06T05:14:55.937 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:55.937 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67f0066f30 msgr2=0x7f67f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67f0066f30 0x7f67f00693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f680c065b00 tx=0x7f680c0c84c0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 msgr2=0x7f680c13e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 0x7f680c13e5f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f67f800c080 tx=0x7f67fc01a000 comp rx=0 tx=0).stop 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 shutdown_connections 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67f0066f30 0x7f67f00693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f680c0c47e0 0x7f680c0c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67f8004810 0x7f680c149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 --2- 172.21.15.33:0/2823246813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680c155720 0x7f680c13e5f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 >> 172.21.15.33:0/2823246813 conn(0x7f680c06d030 msgr2=0x7f680c156070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 shutdown_connections 2023-10-06T05:14:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:55.937+0000 7f681239d700 1 -- 172.21.15.33:0/2823246813 wait complete. 2023-10-06T05:14:55.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:14:55.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.147+0000 7f7189f9d700 1 Processor -- start 2023-10-06T05:14:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.147+0000 7f7189f9d700 1 -- start start 2023-10-06T05:14:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7189f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71841552f0 0x7f71840ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7189f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f71840b18f0 0x7f71840cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7189f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71841586a0 con 0x7f7184154f20 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7189f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718406bdf0 con 0x7f71840b18f0 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7189f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7184157ab0 con 0x7f71841552f0 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f718979c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f71840b18f0 0x7f71840cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7183fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71841552f0 0x7f71840ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7188f9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7184154f20 0x7f71840b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:40142/0 (socket says 172.21.15.33:40142) 2023-10-06T05:14:56.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.149+0000 7f7188f9b700 1 -- 172.21.15.33:0/2884574477 learned_addr learned my addr 172.21.15.33:0/2884574477 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f718979c700 1 -- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71841552f0 msgr2=0x7f71840ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f71837fe700 1 -- 172.21.15.33:0/2884574477 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570887501 0 0) 0x7f71841586a0 con 0x7f7184154f20 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f718979c700 1 --2- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71841552f0 0x7f71840ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f718979c700 1 -- 172.21.15.33:0/2884574477 >> v1:172.21.15.167:6789/0 conn(0x7f7184154f20 legacy=0x7f71840b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f718979c700 1 -- 172.21.15.33:0/2884574477 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7184157d70 con 0x7f71840b18f0 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f7183fff700 1 --2- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71841552f0 0x7f71840ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f718979c700 1 --2- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f71840cc7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f717c009e60 tx=0x7f717c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7045e8ceba58b299 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f71837fe700 1 -- 172.21.15.33:0/2884574477 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00e070 con 0x7f71840b18f0 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f71837fe700 1 -- 172.21.15.33:0/2884574477 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00bb60 con 0x7f71840b18f0 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.150+0000 7f71837fe700 1 -- 172.21.15.33:0/2884574477 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c012430 con 0x7f71840b18f0 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 -- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 msgr2=0x7f71840cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 --2- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f71840cc7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f717c009e60 tx=0x7f717c009780 comp rx=0 tx=0).stop 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 -- 172.21.15.33:0/2884574477 shutdown_connections 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 --2- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f71840cc7e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 --2- 172.21.15.33:0/2884574477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71841552f0 0x7f71840ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 -- 172.21.15.33:0/2884574477 >> 172.21.15.33:0/2884574477 conn(0x7f718406d030 msgr2=0x7f71840cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 -- 172.21.15.33:0/2884574477 shutdown_connections 2023-10-06T05:14:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.151+0000 7f7189f9d700 1 -- 172.21.15.33:0/2884574477 wait complete. 2023-10-06T05:14:56.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.152+0000 7f7189f9d700 1 Processor -- start 2023-10-06T05:14:56.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.152+0000 7f7189f9d700 1 -- start start 2023-10-06T05:14:56.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.153+0000 7f7189f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f7184158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.153+0000 7f7189f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7184154f20 0x7f7184169a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.153+0000 7f7189f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 0x7f7184169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.153+0000 7f7189f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718415a230 con 0x7f71840b0e00 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.153+0000 7f7189f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184159620 con 0x7f71840b18f0 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.153+0000 7f7189f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7184159900 con 0x7f7184154f20 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 0x7f7184169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 0x7f7184169f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46550/0 (socket says 172.21.15.33:46550) 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 -- 172.21.15.33:0/3865292786 learned_addr learned my addr 172.21.15.33:0/3865292786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f7188f9b700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f7184158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f7183fff700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7184154f20 0x7f7184169a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 -- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7184154f20 msgr2=0x7f7184169a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7184154f20 0x7f7184169a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 -- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 msgr2=0x7f7184158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f7184158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 -- 172.21.15.33:0/3865292786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7170009420 con 0x7f71840b0e00 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.154+0000 7f718979c700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 0x7f7184169f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f718406d4d0 tx=0x7f717c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.155+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c012e40 con 0x7f71840b0e00 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.155+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c01d630 con 0x7f71840b0e00 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.155+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c01d9f0 con 0x7f71840b0e00 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.155+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c009a20 con 0x7f71840b0e00 2023-10-06T05:14:56.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.155+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7184159ec0 con 0x7f71840b0e00 2023-10-06T05:14:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.156+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718415a3e0 con 0x7f71840b0e00 2023-10-06T05:14:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.157+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f717c024070 con 0x7f71840b0e00 2023-10-06T05:14:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.157+0000 7f7180ff9700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7168066f30 0x7f71680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.157+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f717c086eb0 con 0x7f71840b0e00 2023-10-06T05:14:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.158+0000 7f7188f9b700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7168066f30 0x7f71680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.158+0000 7f7188f9b700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7168066f30 0x7f71680693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7184157a80 tx=0x7f7178009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.162+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f718415a3e0 con 0x7f71840b0e00 2023-10-06T05:14:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.368+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f718404c380 con 0x7f71840b0e00 2023-10-06T05:14:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.368+0000 7f7180ff9700 1 -- 172.21.15.33:0/3865292786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f717c02c020 con 0x7f71840b0e00 2023-10-06T05:14:56.370 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:14:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7168066f30 msgr2=0x7f71680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7168066f30 0x7f71680693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7184157a80 tx=0x7f7178009040 comp rx=0 tx=0).stop 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 msgr2=0x7f7184169f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 0x7f7184169f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f718406d4d0 tx=0x7f717c019000 comp rx=0 tx=0).stop 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 shutdown_connections 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7168066f30 0x7f71680693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71840b0e00 0x7f7184169f40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71840b18f0 0x7f7184158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 --2- 172.21.15.33:0/3865292786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7184154f20 0x7f7184169a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.370+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 >> 172.21.15.33:0/3865292786 conn(0x7f718406d030 msgr2=0x7f71840acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.371+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 shutdown_connections 2023-10-06T05:14:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.371+0000 7f7189f9d700 1 -- 172.21.15.33:0/3865292786 wait complete. 2023-10-06T05:14:56.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.576+0000 7f0a0f19d700 1 Processor -- start 2023-10-06T05:14:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.577+0000 7f0a0f19d700 1 -- start start 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0f19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0a08156370 0x7f0a080af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a080cc7e0 0x7f0a080c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a080b01f0 0x7f0a080b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a080b0b90 con 0x7f0a080cc7e0 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0f19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a080ad430 con 0x7f0a08156370 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080afe20 con 0x7f0a080b01f0 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a080cc7e0 0x7f0a080c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a080cc7e0 0x7f0a080c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46552/0 (socket says 172.21.15.33:46552) 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0d99a700 1 -- 172.21.15.33:0/1116812015 learned_addr learned my addr 172.21.15.33:0/1116812015 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0e19b700 1 --2- 172.21.15.33:0/1116812015 >> v2:172.21.15.167:3300/0 conn(0x7f0a08156370 0x7f0a080af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.578+0000 7f0a0e99c700 1 --2- 172.21.15.33:0/1116812015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a080b01f0 0x7f0a080b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d99a700 1 -- 172.21.15.33:0/1116812015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a080b01f0 msgr2=0x7f0a080b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d99a700 1 --2- 172.21.15.33:0/1116812015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a080b01f0 0x7f0a080b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d99a700 1 -- 172.21.15.33:0/1116812015 >> v2:172.21.15.167:3300/0 conn(0x7f0a08156370 msgr2=0x7f0a080af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d99a700 1 --2- 172.21.15.33:0/1116812015 >> v2:172.21.15.167:3300/0 conn(0x7f0a08156370 0x7f0a080af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d99a700 1 -- 172.21.15.33:0/1116812015 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0806bc70 con 0x7f0a080cc7e0 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d99a700 1 --2- 172.21.15.33:0/1116812015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080cc7e0 0x7f0a080c8db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a080b21a0 tx=0x7f09f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a30ee833c32a971 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d199700 1 -- 172.21.15.33:0/1116812015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800f6b0 con 0x7f0a080cc7e0 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d199700 1 -- 172.21.15.33:0/1116812015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f800f810 con 0x7f0a080cc7e0 2023-10-06T05:14:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.579+0000 7f0a0d199700 1 -- 172.21.15.33:0/1116812015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800fbd0 con 0x7f0a080cc7e0 2023-10-06T05:14:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1116812015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080cc7e0 msgr2=0x7f0a080c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1116812015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080cc7e0 0x7f0a080c8db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a080b21a0 tx=0x7f09f8009250 comp rx=0 tx=0).stop 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1116812015 shutdown_connections 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1116812015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a080b01f0 0x7f0a080b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1116812015 >> v2:172.21.15.167:3300/0 conn(0x7f0a08156370 0x7f0a080af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1116812015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080cc7e0 0x7f0a080c8db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1116812015 >> 172.21.15.33:0/1116812015 conn(0x7f0a0806d030 msgr2=0x7f0a080cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1116812015 shutdown_connections 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.580+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1116812015 wait complete. 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.581+0000 7f0a0f19d700 1 Processor -- start 2023-10-06T05:14:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.581+0000 7f0a0f19d700 1 -- start start 2023-10-06T05:14:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.582+0000 7f0a0f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 0x7f0a08156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.582+0000 7f0a0f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a080cc7e0 0x7f0a081590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.582+0000 7f0a0f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a08156370 0x7f0a080a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.582+0000 7f0a0f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a08156c80 con 0x7f0a080b01f0 2023-10-06T05:14:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.582+0000 7f0a0f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a08153fb0 con 0x7f0a080cc7e0 2023-10-06T05:14:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.582+0000 7f0a0f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081571b0 con 0x7f0a08156370 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 0x7f0a08156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 0x7f0a08156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46560/0 (socket says 172.21.15.33:46560) 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 -- 172.21.15.33:0/1963119259 learned_addr learned my addr 172.21.15.33:0/1963119259 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e99c700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a08156370 0x7f0a080a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0d99a700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a080cc7e0 0x7f0a081590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 -- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a08156370 msgr2=0x7f0a080a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a08156370 0x7f0a080a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 -- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a080cc7e0 msgr2=0x7f0a081590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a080cc7e0 0x7f0a081590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 -- 172.21.15.33:0/1963119259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f80094f0 con 0x7f0a080b01f0 2023-10-06T05:14:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f0a0e19b700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 0x7f0a08156740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e930 tx=0x7f0a040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.583+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04013680 con 0x7f0a080b01f0 2023-10-06T05:14:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.584+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a040137e0 con 0x7f0a080b01f0 2023-10-06T05:14:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.584+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04013b80 con 0x7f0a080b01f0 2023-10-06T05:14:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.584+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08157490 con 0x7f0a080b01f0 2023-10-06T05:14:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.584+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a08154cc0 con 0x7f0a080b01f0 2023-10-06T05:14:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.585+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a08155030 con 0x7f0a080b01f0 2023-10-06T05:14:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.586+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a0400c3e0 con 0x7f0a080b01f0 2023-10-06T05:14:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.586+0000 7f09fe7fc700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09ec066f30 0x7f09ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.587+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0a040847e0 con 0x7f0a080b01f0 2023-10-06T05:14:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.587+0000 7f0a0d99a700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09ec066f30 0x7f09ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.588+0000 7f0a0d99a700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09ec066f30 0x7f09ec0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a080b21a0 tx=0x7f09f8019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.590+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a04051850 con 0x7f0a080b01f0 2023-10-06T05:14:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.793+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0a0804c380 con 0x7f0a080b01f0 2023-10-06T05:14:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.793+0000 7f09fe7fc700 1 -- 172.21.15.33:0/1963119259 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a04054e70 con 0x7f0a080b01f0 2023-10-06T05:14:56.795 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09ec066f30 msgr2=0x7f09ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09ec066f30 0x7f09ec0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a080b21a0 tx=0x7f09f8019000 comp rx=0 tx=0).stop 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 msgr2=0x7f0a08156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 0x7f0a08156740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e930 tx=0x7f0a040091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 shutdown_connections 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09ec066f30 0x7f09ec0693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a08156370 0x7f0a080a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a080b01f0 0x7f0a08156740 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 --2- 172.21.15.33:0/1963119259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a080cc7e0 0x7f0a081590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 >> 172.21.15.33:0/1963119259 conn(0x7f0a0806d030 msgr2=0x7f0a080ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 shutdown_connections 2023-10-06T05:14:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:56.796+0000 7f0a0f19d700 1 -- 172.21.15.33:0/1963119259 wait complete. 2023-10-06T05:14:56.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:14:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.005+0000 7f4450b9d700 1 Processor -- start 2023-10-06T05:14:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.005+0000 7f4450b9d700 1 -- start start 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.006+0000 7f4450b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f444c159850 0x7f444c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.006+0000 7f4450b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f444c15acd0 0x7f444c06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.006+0000 7f4450b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0b0cd0 con 0x7f444c06d140 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.006+0000 7f4450b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c15a840 con 0x7f444c15acd0 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.006+0000 7f4450b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c159f10 con 0x7f444c159850 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f444c15acd0 0x7f444c06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f444c06d140 0x7f444c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46564/0 (socket says 172.21.15.33:46564) 2023-10-06T05:14:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444bfff700 1 -- 172.21.15.33:0/3858286214 learned_addr learned my addr 172.21.15.33:0/3858286214 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444b7fe700 1 --2- 172.21.15.33:0/3858286214 >> v2:172.21.15.167:3300/0 conn(0x7f444c159850 0x7f444c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444a7fc700 1 -- 172.21.15.33:0/3858286214 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814711855 0 0) 0x7f444c0b0cd0 con 0x7f444c06d140 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444affd700 1 -- 172.21.15.33:0/3858286214 >> v2:172.21.15.167:3300/0 conn(0x7f444c159850 msgr2=0x7f444c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444affd700 1 --2- 172.21.15.33:0/3858286214 >> v2:172.21.15.167:3300/0 conn(0x7f444c159850 0x7f444c06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444affd700 1 -- 172.21.15.33:0/3858286214 >> v1:172.21.15.33:6789/0 conn(0x7f444c06d140 legacy=0x7f444c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444affd700 1 -- 172.21.15.33:0/3858286214 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c15a1f0 con 0x7f444c15acd0 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444affd700 1 --2- 172.21.15.33:0/3858286214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c06edd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f444c063620 tx=0x7f44400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dae1a192baffe90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.007+0000 7f444a7fc700 1 -- 172.21.15.33:0/3858286214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000b6b0 con 0x7f444c15acd0 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f444a7fc700 1 -- 172.21.15.33:0/3858286214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000b810 con 0x7f444c15acd0 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f444a7fc700 1 -- 172.21.15.33:0/3858286214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000bbb0 con 0x7f444c15acd0 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 -- 172.21.15.33:0/3858286214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 msgr2=0x7f444c06edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3858286214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c06edd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f444c063620 tx=0x7f44400091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 -- 172.21.15.33:0/3858286214 shutdown_connections 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3858286214 >> v2:172.21.15.167:3300/0 conn(0x7f444c159850 0x7f444c06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3858286214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c06edd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 -- 172.21.15.33:0/3858286214 >> 172.21.15.33:0/3858286214 conn(0x7f444c06cd40 msgr2=0x7f444c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 -- 172.21.15.33:0/3858286214 shutdown_connections 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.008+0000 7f4450b9d700 1 -- 172.21.15.33:0/3858286214 wait complete. 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.010+0000 7f4450b9d700 1 Processor -- start 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.010+0000 7f4450b9d700 1 -- start start 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f4450b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f444c06d140 0x7f444c0ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f4450b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f444c159850 0x7f444c0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f4450b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f4450b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0b20f0 con 0x7f444c15acd0 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f4450b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0abe70 con 0x7f444c159850 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f4450b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0ac150 con 0x7f444c06d140 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f444bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f444bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46588/0 (socket says 172.21.15.33:46588) 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f444bfff700 1 -- 172.21.15.33:0/3822383094 learned_addr learned my addr 172.21.15.33:0/3822383094 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f444affd700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f444c159850 0x7f444c0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.011+0000 7f444b7fe700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f444c06d140 0x7f444c0ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f444bfff700 1 -- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f444c06d140 msgr2=0x7f444c0ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f444bfff700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f444c06d140 0x7f444c0ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f444bfff700 1 -- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f444c159850 msgr2=0x7f444c0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f444bfff700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f444c159850 0x7f444c0b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f444bfff700 1 -- 172.21.15.33:0/3822383094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440009420 con 0x7f444c15acd0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f444bfff700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c146c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f444400cbc0 tx=0x7f4444001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400a7e0 con 0x7f444c15acd0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444400a940 con 0x7f444c15acd0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400ad00 con 0x7f444c15acd0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.012+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c0ac430 con 0x7f444c15acd0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.013+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c0b1e30 con 0x7f444c15acd0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.013+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c0b22a0 con 0x7f444c15acd0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.014+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4444007300 con 0x7f444c15acd0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.016+0000 7f44337fe700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4438066f30 0x7f44380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.016+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44440849d0 con 0x7f444c15acd0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.016+0000 7f444b7fe700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4438066f30 0x7f44380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.017+0000 7f444b7fe700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4438066f30 0x7f44380693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f443c006fd0 tx=0x7f443c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.020+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f444c0b22a0 con 0x7f444c15acd0 2023-10-06T05:14:57.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.223+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f444c0b22a0 con 0x7f444c15acd0 2023-10-06T05:14:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.223+0000 7f44337fe700 1 -- 172.21.15.33:0/3822383094 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f444c0b22a0 con 0x7f444c15acd0 2023-10-06T05:14:57.225 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4438066f30 msgr2=0x7f44380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4438066f30 0x7f44380693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f443c006fd0 tx=0x7f443c008040 comp rx=0 tx=0).stop 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 msgr2=0x7f444c146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c146c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f444400cbc0 tx=0x7f4444001980 comp rx=0 tx=0).stop 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 shutdown_connections 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4438066f30 0x7f44380693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444c15acd0 0x7f444c146c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f444c06d140 0x7f444c0ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 --2- 172.21.15.33:0/3822383094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f444c159850 0x7f444c0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 >> 172.21.15.33:0/3822383094 conn(0x7f444c06cd40 msgr2=0x7f444c0aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 shutdown_connections 2023-10-06T05:14:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.226+0000 7f4450b9d700 1 -- 172.21.15.33:0/3822383094 wait complete. 2023-10-06T05:14:57.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:14:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.432+0000 7fa72699d700 1 Processor -- start 2023-10-06T05:14:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.432+0000 7fa72699d700 1 -- start start 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.433+0000 7fa72699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7200af540 0x7fa7200af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.433+0000 7fa72699d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7200cd5e0 0x7fa72015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.433+0000 7fa72699d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72015d660 con 0x7fa72015a650 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.433+0000 7fa72699d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7200ce890 con 0x7fa7200cd5e0 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.433+0000 7fa72699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72015aac0 con 0x7fa7200af540 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.433+0000 7fa72619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7200cd5e0 0x7fa72015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7200cd5e0 0x7fa72015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46600/0 (socket says 172.21.15.33:46600) 2023-10-06T05:14:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 -- 172.21.15.33:0/4107030498 learned_addr learned my addr 172.21.15.33:0/4107030498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72599b700 1 --2- 172.21.15.33:0/4107030498 >> v2:172.21.15.167:3300/0 conn(0x7fa7200af540 0x7fa7200af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 -- 172.21.15.33:0/4107030498 >> v2:172.21.15.167:3300/0 conn(0x7fa7200af540 msgr2=0x7fa7200af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 --2- 172.21.15.33:0/4107030498 >> v2:172.21.15.167:3300/0 conn(0x7fa7200af540 0x7fa7200af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 -- 172.21.15.33:0/4107030498 >> v1:172.21.15.167:6789/0 conn(0x7fa72015a650 legacy=0x7fa7200cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 -- 172.21.15.33:0/4107030498 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72015d890 con 0x7fa7200cd5e0 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.434+0000 7fa72619c700 1 --2- 172.21.15.33:0/4107030498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200cd5e0 0x7fa72015d120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7200c7530 tx=0x7fa718009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9757ea89bd6e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa724999700 1 -- 172.21.15.33:0/4107030498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71800e070 con 0x7fa7200cd5e0 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa724999700 1 -- 172.21.15.33:0/4107030498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71800ba50 con 0x7fa7200cd5e0 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa724999700 1 -- 172.21.15.33:0/4107030498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180092e0 con 0x7fa7200cd5e0 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 -- 172.21.15.33:0/4107030498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200cd5e0 msgr2=0x7fa72015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 --2- 172.21.15.33:0/4107030498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200cd5e0 0x7fa72015d120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7200c7530 tx=0x7fa718009840 comp rx=0 tx=0).stop 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 -- 172.21.15.33:0/4107030498 shutdown_connections 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 --2- 172.21.15.33:0/4107030498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200cd5e0 0x7fa72015d120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 --2- 172.21.15.33:0/4107030498 >> v2:172.21.15.167:3300/0 conn(0x7fa7200af540 0x7fa7200af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 -- 172.21.15.33:0/4107030498 >> 172.21.15.33:0/4107030498 conn(0x7fa72006d9b0 msgr2=0x7fa72006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 -- 172.21.15.33:0/4107030498 shutdown_connections 2023-10-06T05:14:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.435+0000 7fa72699d700 1 -- 172.21.15.33:0/4107030498 wait complete. 2023-10-06T05:14:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.437+0000 7fa72699d700 1 Processor -- start 2023-10-06T05:14:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.437+0000 7fa72699d700 1 -- start start 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 0x7fa720148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7200cd5e0 0x7fa720158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa72015a650 0x7fa720159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72016a840 con 0x7fa7200af540 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa720169a00 con 0x7fa7200cd5e0 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa720169c70 con 0x7fa72015a650 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 0x7fa720148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 0x7fa720148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46602/0 (socket says 172.21.15.33:46602) 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72599b700 1 -- 172.21.15.33:0/1371928320 learned_addr learned my addr 172.21.15.33:0/1371928320 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72619c700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa72015a650 0x7fa720159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.438+0000 7fa72519a700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7200cd5e0 0x7fa720158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72599b700 1 -- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa72015a650 msgr2=0x7fa720159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72599b700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa72015a650 0x7fa720159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72599b700 1 -- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7200cd5e0 msgr2=0x7fa720158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72599b700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7200cd5e0 0x7fa720158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72599b700 1 -- 172.21.15.33:0/1371928320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718009ae0 con 0x7fa7200af540 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72519a700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7200cd5e0 0x7fa720158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72599b700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 0x7fa720148dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00d860 tx=0x7fa71c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0125f0 con 0x7fa7200af540 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c012770 con 0x7fa7200af540 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c012b30 con 0x7fa7200af540 2023-10-06T05:14:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa720169f10 con 0x7fa7200af540 2023-10-06T05:14:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.439+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72016a4d0 con 0x7fa7200af540 2023-10-06T05:14:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.440+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72004c380 con 0x7fa7200af540 2023-10-06T05:14:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.441+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa71c01e460 con 0x7fa7200af540 2023-10-06T05:14:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.442+0000 7fa715ffb700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa704066f30 0x7fa7040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.442+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa71c084d40 con 0x7fa7200af540 2023-10-06T05:14:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.442+0000 7fa72519a700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa704066f30 0x7fa7040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.443+0000 7fa72519a700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa704066f30 0x7fa7040693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa710007fd0 tx=0x7fa710009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.446+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa71c051db0 con 0x7fa7200af540 2023-10-06T05:14:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.646+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa72015db70 con 0x7fa7200af540 2023-10-06T05:14:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.647+0000 7fa715ffb700 1 -- 172.21.15.33:0/1371928320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa71c0553d0 con 0x7fa7200af540 2023-10-06T05:14:57.648 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa704066f30 msgr2=0x7fa7040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa704066f30 0x7fa7040693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa710007fd0 tx=0x7fa710009040 comp rx=0 tx=0).stop 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 msgr2=0x7fa720148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 0x7fa720148dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00d860 tx=0x7fa71c008040 comp rx=0 tx=0).stop 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 shutdown_connections 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa704066f30 0x7fa7040693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa72015a650 0x7fa720159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7200af540 0x7fa720148dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 --2- 172.21.15.33:0/1371928320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7200cd5e0 0x7fa720158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 >> 172.21.15.33:0/1371928320 conn(0x7fa72006d9b0 msgr2=0x7fa7201582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 shutdown_connections 2023-10-06T05:14:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.649+0000 7fa72699d700 1 -- 172.21.15.33:0/1371928320 wait complete. 2023-10-06T05:14:57.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.857+0000 7ff7aa39d700 1 Processor -- start 2023-10-06T05:14:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.857+0000 7ff7aa39d700 1 -- start start 2023-10-06T05:14:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.858+0000 7ff7aa39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff7a40b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.858+0000 7ff7aa39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a40cbd80 con 0x7ff7a40ccfc0 2023-10-06T05:14:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.858+0000 7ff7aa39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a40af5f0 con 0x7ff7a40cf3b0 2023-10-06T05:14:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.858+0000 7ff7aa39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40af8d0 con 0x7ff7a4159bb0 2023-10-06T05:14:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.858+0000 7ff7a939b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff7a40ccfc0 0x7ff7a40cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46578/0 (socket says 172.21.15.33:46578) 2023-10-06T05:14:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.858+0000 7ff7a939b700 1 -- 172.21.15.33:0/3114341874 learned_addr learned my addr 172.21.15.33:0/3114341874 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672675830 0 0) 0x7ff7a40cbd80 con 0x7ff7a40ccfc0 2023-10-06T05:14:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff790003570 con 0x7ff7a40ccfc0 2023-10-06T05:14:57.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff7a9b9c700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff7a40b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3758520574 0 0) 0x7ff790003570 con 0x7ff7a40ccfc0 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 msgr2=0x7ff7a40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff7a40b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 >> v1:172.21.15.167:6789/0 conn(0x7ff7a40cf3b0 legacy=0x7ff7a41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40afbb0 con 0x7ff7a40ccfc0 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7ff7a0001470 con 0x7ff7a40ccfc0 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 >> v1:172.21.15.33:6789/0 conn(0x7ff7a40ccfc0 legacy=0x7ff7a40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff790003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790004060 0x7ff790004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff790004d50 0x7ff7900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40af8d0 con 0x7ff790004060 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79000bc20 con 0x7ff790004d50 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.859+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79000bde0 con 0x7ff7a4159bb0 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790004060 0x7ff790004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a939b700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff790003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a9b9c700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff790004d50 0x7ff7900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 msgr2=0x7ff790003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff790003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff790004d50 msgr2=0x7ff7900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff790004d50 0x7ff7900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/3114341874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff790003570 con 0x7ff790004060 2023-10-06T05:14:57.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a939b700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7a4159bb0 0x7ff790003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a9b9c700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff790004d50 0x7ff7900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.860+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790004060 0x7ff790004810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40b0250 tx=0x7ff794007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd4796a6d8e012e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794012420 con 0x7ff790004060 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff794012580 con 0x7ff790004060 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff79bfff700 1 -- 172.21.15.33:0/3114341874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794012920 con 0x7ff790004060 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7aa39d700 1 -- 172.21.15.33:0/3114341874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790004060 msgr2=0x7ff790004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/3114341874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790004060 0x7ff790004810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40b0250 tx=0x7ff794007d30 comp rx=0 tx=0).stop 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7a9b9c700 1 -- 172.21.15.33:0/3114341874 reap_dead start 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7aa39d700 1 -- 172.21.15.33:0/3114341874 shutdown_connections 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7aa39d700 1 -- 172.21.15.33:0/3114341874 >> 172.21.15.33:0/3114341874 conn(0x7ff7a406cff0 msgr2=0x7ff7a406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7aa39d700 1 -- 172.21.15.33:0/3114341874 shutdown_connections 2023-10-06T05:14:57.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.861+0000 7ff7aa39d700 1 -- 172.21.15.33:0/3114341874 wait complete. 2023-10-06T05:14:57.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.863+0000 7ff7aa39d700 1 Processor -- start 2023-10-06T05:14:57.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.863+0000 7ff7aa39d700 1 -- start start 2023-10-06T05:14:57.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7aa39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff790004060 0x7ff7a414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7aa39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 0x7ff7a4150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7aa39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41511a0 0x7ff7a40c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7aa39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4151620 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7aa39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4151900 con 0x7ff7a41511a0 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7aa39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4151be0 con 0x7ff790004060 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 0x7ff7a4150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 0x7ff7a4150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46616/0 (socket says 172.21.15.33:46616) 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/181271237 learned_addr learned my addr 172.21.15.33:0/181271237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a939b700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff790004060 0x7ff7a414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a9b9c700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41511a0 0x7ff7a40c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff790004060 msgr2=0x7ff7a414e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff790004060 0x7ff7a414e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41511a0 msgr2=0x7ff7a40c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41511a0 0x7ff7a40c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a8b9a700 1 -- 172.21.15.33:0/181271237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4151ec0 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a939b700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff790004060 0x7ff7a414e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.864+0000 7ff7a9b9c700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41511a0 0x7ff7a40c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.865+0000 7ff7a8b9a700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 0x7ff7a4150c60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff794008fd0 tx=0x7ff794015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.865+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794012c90 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.865+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff794012e30 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.865+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794006820 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.865+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a41521a0 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.865+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a40c9ae0 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.866+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a4152510 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.867+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff794019070 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.868+0000 7ff7997fa700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff788066f30 0x7ff7880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.868+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff794085770 con 0x7ff7a40c8cb0 2023-10-06T05:14:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.868+0000 7ff7a939b700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff788066f30 0x7ff7880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.869+0000 7ff7a939b700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff788066f30 0x7ff7880693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0001ed0 tx=0x7ff7a0000f60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:57.872+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7940527e0 con 0x7ff7a40c8cb0 2023-10-06T05:14:58.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.077+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7a404c380 con 0x7ff7a40c8cb0 2023-10-06T05:14:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.078+0000 7ff7997fa700 1 -- 172.21.15.33:0/181271237 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7ff794055e00 con 0x7ff7a40c8cb0 2023-10-06T05:14:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:14:58.080 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:58.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff788066f30 msgr2=0x7ff7880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff788066f30 0x7ff7880693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0001ed0 tx=0x7ff7a0000f60 comp rx=0 tx=0).stop 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 msgr2=0x7ff7a4150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 0x7ff7a4150c60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff794008fd0 tx=0x7ff794015000 comp rx=0 tx=0).stop 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 shutdown_connections 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff788066f30 0x7ff7880693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41511a0 0x7ff7a40c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff790004060 0x7ff7a414e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 --2- 172.21.15.33:0/181271237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7a40c8cb0 0x7ff7a4150c60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 >> 172.21.15.33:0/181271237 conn(0x7ff7a406cff0 msgr2=0x7ff7a414ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 shutdown_connections 2023-10-06T05:14:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.080+0000 7ff7aa39d700 1 -- 172.21.15.33:0/181271237 wait complete. 2023-10-06T05:14:58.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:14:58.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.287+0000 7f9b0299d700 1 Processor -- start 2023-10-06T05:14:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.287+0000 7f9b0299d700 1 -- start start 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.288+0000 7f9b0299d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9afc0cc7e0 0x7f9afc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.288+0000 7f9b0299d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9afc0b01f0 0x7f9afc0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.288+0000 7f9b0299d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0ad430 con 0x7f9afc156370 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0299d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0afe20 con 0x7f9afc0cc7e0 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0299d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0b0b70 con 0x7f9afc0b01f0 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9afc0cc7e0 0x7f9afc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9afc0cc7e0 0x7f9afc0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46628/0 (socket says 172.21.15.33:46628) 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 -- 172.21.15.33:0/2742376984 learned_addr learned my addr 172.21.15.33:0/2742376984 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0219c700 1 --2- 172.21.15.33:0/2742376984 >> v2:172.21.15.167:3300/0 conn(0x7f9afc0b01f0 0x7f9afc0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 -- 172.21.15.33:0/2742376984 >> v2:172.21.15.167:3300/0 conn(0x7f9afc0b01f0 msgr2=0x7f9afc0b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 --2- 172.21.15.33:0/2742376984 >> v2:172.21.15.167:3300/0 conn(0x7f9afc0b01f0 0x7f9afc0b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 -- 172.21.15.33:0/2742376984 >> v1:172.21.15.167:6789/0 conn(0x7f9afc156370 legacy=0x7f9afc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0119a700 1 -- 172.21.15.33:0/2742376984 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc06bc70 con 0x7f9afc0cc7e0 2023-10-06T05:14:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.289+0000 7f9b0219c700 1 --2- 172.21.15.33:0/2742376984 >> v2:172.21.15.167:3300/0 conn(0x7f9afc0b01f0 0x7f9afc0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0119a700 1 --2- 172.21.15.33:0/2742376984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc0af8e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0b13b0 tx=0x7f9aec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3068e43bab5028c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b00999700 1 -- 172.21.15.33:0/2742376984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00b860 con 0x7f9afc0cc7e0 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b00999700 1 -- 172.21.15.33:0/2742376984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00b9c0 con 0x7f9afc0cc7e0 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b00999700 1 -- 172.21.15.33:0/2742376984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00bd60 con 0x7f9afc0cc7e0 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 -- 172.21.15.33:0/2742376984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0cc7e0 msgr2=0x7f9afc0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2742376984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc0af8e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0b13b0 tx=0x7f9aec009250 comp rx=0 tx=0).stop 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 -- 172.21.15.33:0/2742376984 shutdown_connections 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2742376984 >> v2:172.21.15.167:3300/0 conn(0x7f9afc0b01f0 0x7f9afc0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2742376984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc0af8e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 -- 172.21.15.33:0/2742376984 >> 172.21.15.33:0/2742376984 conn(0x7f9afc06d030 msgr2=0x7f9afc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 -- 172.21.15.33:0/2742376984 shutdown_connections 2023-10-06T05:14:58.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.290+0000 7f9b0299d700 1 -- 172.21.15.33:0/2742376984 wait complete. 2023-10-06T05:14:58.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.292+0000 7f9b0299d700 1 Processor -- start 2023-10-06T05:14:58.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.292+0000 7f9b0299d700 1 -- start start 2023-10-06T05:14:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 0x7f9afc1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc16ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9afc156370 0x7f9afc159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1597c0 con 0x7f9afc0b01f0 2023-10-06T05:14:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc16b140 con 0x7f9afc0cc7e0 2023-10-06T05:14:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1595b0 con 0x7f9afc156370 2023-10-06T05:14:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 0x7f9afc1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 0x7f9afc1548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46634/0 (socket says 172.21.15.33:46634) 2023-10-06T05:14:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0199b700 1 -- 172.21.15.33:0/2810513896 learned_addr learned my addr 172.21.15.33:0/2810513896 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0119a700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc16ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.293+0000 7f9b0219c700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9afc156370 0x7f9afc159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0199b700 1 -- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9afc156370 msgr2=0x7f9afc159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0199b700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9afc156370 0x7f9afc159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0199b700 1 -- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9afc0cc7e0 msgr2=0x7f9afc16ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0199b700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc16ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0199b700 1 -- 172.21.15.33:0/2810513896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0094f0 con 0x7f9afc0b01f0 2023-10-06T05:14:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0219c700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9afc156370 0x7f9afc159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:58.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0199b700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 0x7f9afc1548d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9af800c590 tx=0x7f9af800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8011590 con 0x7f9afc0b01f0 2023-10-06T05:14:58.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af8011710 con 0x7f9afc0b01f0 2023-10-06T05:14:58.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc153870 con 0x7f9afc0b01f0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8011ad0 con 0x7f9afc0b01f0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.294+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc153df0 con 0x7f9afc0b01f0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.295+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc154720 con 0x7f9afc0b01f0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.296+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9af801b3b0 con 0x7f9afc0b01f0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.297+0000 7f9af1ffb700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.297+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9af8083c60 con 0x7f9afc0b01f0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.298+0000 7f9b0119a700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.298+0000 7f9b0119a700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ae0066f30 0x7f9ae00693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0b13b0 tx=0x7f9aec019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.301+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9af8050cd0 con 0x7f9afc0b01f0 2023-10-06T05:14:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.501+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9afc04c380 con 0x7f9afc0b01f0 2023-10-06T05:14:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.501+0000 7f9af1ffb700 1 -- 172.21.15.33:0/2810513896 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9af8017070 con 0x7f9afc0b01f0 2023-10-06T05:14:58.503 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:14:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.503+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ae0066f30 msgr2=0x7f9ae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.503+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ae0066f30 0x7f9ae00693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0b13b0 tx=0x7f9aec019000 comp rx=0 tx=0).stop 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.503+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 msgr2=0x7f9afc1548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.503+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 0x7f9afc1548d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9af800c590 tx=0x7f9af800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 shutdown_connections 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9afc156370 0x7f9afc159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9afc0b01f0 0x7f9afc1548d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 --2- 172.21.15.33:0/2810513896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9afc0cc7e0 0x7f9afc16ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 >> 172.21.15.33:0/2810513896 conn(0x7f9afc06d030 msgr2=0x7f9afc155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 shutdown_connections 2023-10-06T05:14:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.504+0000 7f9b0299d700 1 -- 172.21.15.33:0/2810513896 wait complete. 2023-10-06T05:14:58.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:58.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.711+0000 7f358099d700 1 Processor -- start 2023-10-06T05:14:58.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.711+0000 7f358099d700 1 -- start start 2023-10-06T05:14:58.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.712+0000 7f358099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 0x7f357c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.712+0000 7f358099d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f357c0cc7e0 0x7f357c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.712+0000 7f358099d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357c0ad430 con 0x7f357c0b01f0 2023-10-06T05:14:58.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f358099d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357c0afe20 con 0x7f357c0cc7e0 2023-10-06T05:14:58.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f358099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c06bc70 con 0x7f357c156370 2023-10-06T05:14:58.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 0x7f357c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 0x7f357c0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:44786/0 (socket says 172.21.15.33:44786) 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357b7fe700 1 -- 172.21.15.33:0/369292744 learned_addr learned my addr 172.21.15.33:0/369292744 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357affd700 1 --2- 172.21.15.33:0/369292744 >> v2:172.21.15.167:3300/0 conn(0x7f357c0cc7e0 0x7f357c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357bfff700 1 --1- 172.21.15.33:0/369292744 >> v1:172.21.15.167:6789/0 conn(0x7f357c0b01f0 0x7f357c0ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:40168/0 (socket says 172.21.15.33:40168) 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357a7fc700 1 -- 172.21.15.33:0/369292744 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154076739 0 0) 0x7f357c0ad430 con 0x7f357c0b01f0 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357a7fc700 1 -- 172.21.15.33:0/369292744 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3564003570 con 0x7f357c0b01f0 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.713+0000 7f357b7fe700 1 -- 172.21.15.33:0/369292744 >> v2:172.21.15.167:3300/0 conn(0x7f357c0cc7e0 msgr2=0x7f357c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357b7fe700 1 --2- 172.21.15.33:0/369292744 >> v2:172.21.15.167:3300/0 conn(0x7f357c0cc7e0 0x7f357c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357b7fe700 1 -- 172.21.15.33:0/369292744 >> v1:172.21.15.167:6789/0 conn(0x7f357c0b01f0 legacy=0x7f357c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357b7fe700 1 -- 172.21.15.33:0/369292744 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c0c92f0 con 0x7f357c156370 2023-10-06T05:14:58.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357affd700 1 --2- 172.21.15.33:0/369292744 >> v2:172.21.15.167:3300/0 conn(0x7f357c0cc7e0 0x7f357c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:58.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357b7fe700 1 --2- 172.21.15.33:0/369292744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 0x7f357c0af8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f356c0099b0 tx=0x7f356c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b9d4a2338a6eb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357a7fc700 1 -- 172.21.15.33:0/369292744 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00b680 con 0x7f357c156370 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357a7fc700 1 -- 172.21.15.33:0/369292744 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c00b7e0 con 0x7f357c156370 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.714+0000 7f357a7fc700 1 -- 172.21.15.33:0/369292744 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00bb80 con 0x7f357c156370 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 -- 172.21.15.33:0/369292744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 msgr2=0x7f357c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 --2- 172.21.15.33:0/369292744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 0x7f357c0af8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f356c0099b0 tx=0x7f356c009250 comp rx=0 tx=0).stop 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 -- 172.21.15.33:0/369292744 shutdown_connections 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 --2- 172.21.15.33:0/369292744 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c156370 0x7f357c0af8e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 --2- 172.21.15.33:0/369292744 >> v2:172.21.15.167:3300/0 conn(0x7f357c0cc7e0 0x7f357c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 -- 172.21.15.33:0/369292744 >> 172.21.15.33:0/369292744 conn(0x7f357c06d030 msgr2=0x7f357c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 -- 172.21.15.33:0/369292744 shutdown_connections 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.715+0000 7f358099d700 1 -- 172.21.15.33:0/369292744 wait complete. 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.716+0000 7f358099d700 1 Processor -- start 2023-10-06T05:14:58.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.716+0000 7f358099d700 1 -- start start 2023-10-06T05:14:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.717+0000 7f358099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c0b01f0 0x7f357c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.717+0000 7f358099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357c0cc7e0 0x7f357c144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.717+0000 7f358099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 0x7f357c0a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.717+0000 7f358099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c1596d0 con 0x7f357c156370 2023-10-06T05:14:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.717+0000 7f358099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0a8dc0 con 0x7f357c0cc7e0 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.717+0000 7f358099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0a6a70 con 0x7f357c0b01f0 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 0x7f357c0a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 0x7f357c0a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46648/0 (socket says 172.21.15.33:46648) 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 -- 172.21.15.33:0/2702296160 learned_addr learned my addr 172.21.15.33:0/2702296160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357b7fe700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c0b01f0 0x7f357c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357affd700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357c0cc7e0 0x7f357c144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 -- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c0b01f0 msgr2=0x7f357c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c0b01f0 0x7f357c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 -- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357c0cc7e0 msgr2=0x7f357c144c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357c0cc7e0 0x7f357c144c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 -- 172.21.15.33:0/2702296160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570009420 con 0x7f357c156370 2023-10-06T05:14:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357b7fe700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c0b01f0 0x7f357c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357affd700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357c0cc7e0 0x7f357c144c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.718+0000 7f357bfff700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 0x7f357c0a6d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f357400cd30 tx=0x7f3574001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.719+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574005df0 con 0x7f357c156370 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.719+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357400a960 con 0x7f357c156370 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.719+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c0094f0 con 0x7f357c156370 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.719+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400ace0 con 0x7f357c156370 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.719+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c158030 con 0x7f357c156370 2023-10-06T05:14:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.720+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c0a8fc0 con 0x7f357c156370 2023-10-06T05:14:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.721+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3574007300 con 0x7f357c156370 2023-10-06T05:14:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.721+0000 7f35637fe700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3568066f30 0x7f35680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.721+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3574084940 con 0x7f357c156370 2023-10-06T05:14:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.722+0000 7f357b7fe700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3568066f30 0x7f35680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.723+0000 7f357b7fe700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3568066f30 0x7f35680693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f356c0099b0 tx=0x7f356c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.726+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f357c0a8fc0 con 0x7f357c156370 2023-10-06T05:14:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.929+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f357c0a8fc0 con 0x7f357c156370 2023-10-06T05:14:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.929+0000 7f35637fe700 1 -- 172.21.15.33:0/2702296160 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f357c0a8fc0 con 0x7f357c156370 2023-10-06T05:14:58.931 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:14:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3568066f30 msgr2=0x7f35680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3568066f30 0x7f35680693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f356c0099b0 tx=0x7f356c017000 comp rx=0 tx=0).stop 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 msgr2=0x7f357c0a6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 0x7f357c0a6d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f357400cd30 tx=0x7f3574001b60 comp rx=0 tx=0).stop 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 shutdown_connections 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3568066f30 0x7f35680693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f357c156370 0x7f357c0a6d50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f357c0b01f0 0x7f357c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 --2- 172.21.15.33:0/2702296160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357c0cc7e0 0x7f357c144c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 >> 172.21.15.33:0/2702296160 conn(0x7f357c06d030 msgr2=0x7f357c06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 shutdown_connections 2023-10-06T05:14:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:58.932+0000 7f358099d700 1 -- 172.21.15.33:0/2702296160 wait complete. 2023-10-06T05:14:58.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:14:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.139+0000 7f7c2ffff700 1 Processor -- start 2023-10-06T05:14:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.139+0000 7f7c2ffff700 1 -- start start 2023-10-06T05:14:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c3015acd0 0x7f7c300abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c3006efa0 0x7f7c300ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c300b0cd0 con 0x7f7c30159850 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3015a840 con 0x7f7c3015acd0 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c30159f10 con 0x7f7c3006efa0 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c3015acd0 0x7f7c300abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7c30159850 0x7f7c3006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46590/0 (socket says 172.21.15.33:46590) 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c3015acd0 0x7f7c300abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46656/0 (socket says 172.21.15.33:46656) 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2effd700 1 -- 172.21.15.33:0/1752587678 learned_addr learned my addr 172.21.15.33:0/1752587678 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2f7fe700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c3006efa0 0x7f7c300ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2e7fc700 1 -- 172.21.15.33:0/1752587678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c3006efa0 msgr2=0x7f7c300ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2e7fc700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c3006efa0 0x7f7c300ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2e7fc700 1 -- 172.21.15.33:0/1752587678 >> v1:172.21.15.33:6789/0 conn(0x7f7c30159850 legacy=0x7f7c3006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2e7fc700 1 -- 172.21.15.33:0/1752587678 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3015a1f0 con 0x7f7c3015acd0 2023-10-06T05:14:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.141+0000 7f7c2f7fe700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c3006efa0 0x7f7c300ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.142+0000 7f7c2e7fc700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3015acd0 0x7f7c300abe90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7c240098e0 tx=0x7f7c240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d6b7614de9aae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.142+0000 7f7c2dffb700 1 -- 172.21.15.33:0/1752587678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24009e80 con 0x7f7c3015acd0 2023-10-06T05:14:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.142+0000 7f7c2dffb700 1 -- 172.21.15.33:0/1752587678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2400b860 con 0x7f7c3015acd0 2023-10-06T05:14:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.142+0000 7f7c2dffb700 1 -- 172.21.15.33:0/1752587678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400bbe0 con 0x7f7c3015acd0 2023-10-06T05:14:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.142+0000 7f7c2ffff700 1 -- 172.21.15.33:0/1752587678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3015acd0 msgr2=0x7f7c300abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.142+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3015acd0 0x7f7c300abe90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7c240098e0 tx=0x7f7c240091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.143+0000 7f7c2ffff700 1 -- 172.21.15.33:0/1752587678 shutdown_connections 2023-10-06T05:14:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.143+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c3006efa0 0x7f7c300ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.143+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/1752587678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3015acd0 0x7f7c300abe90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.143+0000 7f7c2ffff700 1 -- 172.21.15.33:0/1752587678 >> 172.21.15.33:0/1752587678 conn(0x7f7c3006cd40 msgr2=0x7f7c300ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.143+0000 7f7c2ffff700 1 -- 172.21.15.33:0/1752587678 shutdown_connections 2023-10-06T05:14:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.143+0000 7f7c2ffff700 1 -- 172.21.15.33:0/1752587678 wait complete. 2023-10-06T05:14:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.144+0000 7f7c2ffff700 1 Processor -- start 2023-10-06T05:14:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.144+0000 7f7c2ffff700 1 -- start start 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 0x7f7c300cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c30159850 0x7f7c300ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c3015acd0 0x7f7c3015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300c87e0 con 0x7f7c3006efa0 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300cd700 con 0x7f7c3015acd0 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300cd420 con 0x7f7c30159850 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 0x7f7c300cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 0x7f7c300cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46672/0 (socket says 172.21.15.33:46672) 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.145+0000 7f7c2effd700 1 -- 172.21.15.33:0/665263052 learned_addr learned my addr 172.21.15.33:0/665263052 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2effd700 1 -- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c30159850 msgr2=0x7f7c300ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2effd700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c30159850 0x7f7c300ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2effd700 1 -- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c3015acd0 msgr2=0x7f7c3015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2effd700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c3015acd0 0x7f7c3015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2effd700 1 -- 172.21.15.33:0/665263052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24009420 con 0x7f7c3006efa0 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2effd700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 0x7f7c300cc690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c7c0 tx=0x7f7c20007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20005b30 con 0x7f7c3006efa0 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c20005cb0 con 0x7f7c3006efa0 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20004a80 con 0x7f7c3006efa0 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c300c8cf0 con 0x7f7c3006efa0 2023-10-06T05:14:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.146+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c300cd110 con 0x7f7c3006efa0 2023-10-06T05:14:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.147+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3004c380 con 0x7f7c3006efa0 2023-10-06T05:14:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.148+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c200070f0 con 0x7f7c3006efa0 2023-10-06T05:14:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.149+0000 7f7c16ffd700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c1c066f30 0x7f7c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.149+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7c20083b40 con 0x7f7c3006efa0 2023-10-06T05:14:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.150+0000 7f7c2e7fc700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c1c066f30 0x7f7c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.151+0000 7f7c2e7fc700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c1c066f30 0x7f7c1c0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7c3015bae0 tx=0x7f7c24017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.153+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c20050c30 con 0x7f7c3006efa0 2023-10-06T05:14:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.356+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c300caaf0 con 0x7f7c3006efa0 2023-10-06T05:14:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.357+0000 7f7c16ffd700 1 -- 172.21.15.33:0/665263052 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c20018020 con 0x7f7c3006efa0 2023-10-06T05:14:59.359 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:14:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c1c066f30 msgr2=0x7f7c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c1c066f30 0x7f7c1c0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7c3015bae0 tx=0x7f7c24017000 comp rx=0 tx=0).stop 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 msgr2=0x7f7c300cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 0x7f7c300cc690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c7c0 tx=0x7f7c20007f70 comp rx=0 tx=0).stop 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 shutdown_connections 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c1c066f30 0x7f7c1c0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c3015acd0 0x7f7c3015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3006efa0 0x7f7c300cc690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 --2- 172.21.15.33:0/665263052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c30159850 0x7f7c300ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 >> 172.21.15.33:0/665263052 conn(0x7f7c3006cd40 msgr2=0x7f7c3015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.359+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 shutdown_connections 2023-10-06T05:14:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.360+0000 7f7c2ffff700 1 -- 172.21.15.33:0/665263052 wait complete. 2023-10-06T05:14:59.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:14:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.566+0000 7f120a39d700 1 Processor -- start 2023-10-06T05:14:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.566+0000 7f120a39d700 1 -- start start 2023-10-06T05:14:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.567+0000 7f120a39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12040af540 0x7f12040af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120a39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f12040cd5e0 0x7f120415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120a39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120415d660 con 0x7f120415a650 2023-10-06T05:14:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120a39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12040ce890 con 0x7f12040af540 2023-10-06T05:14:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120a39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120415aac0 con 0x7f12040cd5e0 2023-10-06T05:14:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12040af540 0x7f12040af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f1208b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f120415a650 0x7f12040cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46604/0 (socket says 172.21.15.33:46604) 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12040af540 0x7f12040af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46678/0 (socket says 172.21.15.33:46678) 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f1208b9a700 1 -- 172.21.15.33:0/3395283012 learned_addr learned my addr 172.21.15.33:0/3395283012 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f1209b9c700 1 --2- 172.21.15.33:0/3395283012 >> v2:172.21.15.167:3300/0 conn(0x7f12040cd5e0 0x7f120415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120939b700 1 -- 172.21.15.33:0/3395283012 >> v2:172.21.15.167:3300/0 conn(0x7f12040cd5e0 msgr2=0x7f120415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120939b700 1 --2- 172.21.15.33:0/3395283012 >> v2:172.21.15.167:3300/0 conn(0x7f12040cd5e0 0x7f120415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120939b700 1 -- 172.21.15.33:0/3395283012 >> v1:172.21.15.33:6789/0 conn(0x7f120415a650 legacy=0x7f12040cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f120939b700 1 -- 172.21.15.33:0/3395283012 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120415d890 con 0x7f12040af540 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.568+0000 7f11fbfff700 1 -- 172.21.15.33:0/3395283012 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124095098 0 0) 0x7f120415d660 con 0x7f120415a650 2023-10-06T05:14:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.569+0000 7f120939b700 1 --2- 172.21.15.33:0/3395283012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040af540 0x7f12040af910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12040ce1c0 tx=0x7f12000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699296ed079c9c0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.569+0000 7f11fbfff700 1 -- 172.21.15.33:0/3395283012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000b6b0 con 0x7f12040af540 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.569+0000 7f11fbfff700 1 -- 172.21.15.33:0/3395283012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120000b810 con 0x7f12040af540 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.569+0000 7f11fbfff700 1 -- 172.21.15.33:0/3395283012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000bbb0 con 0x7f12040af540 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.569+0000 7f120a39d700 1 -- 172.21.15.33:0/3395283012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040af540 msgr2=0x7f12040af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.569+0000 7f120a39d700 1 --2- 172.21.15.33:0/3395283012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040af540 0x7f12040af910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12040ce1c0 tx=0x7f12000091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.570+0000 7f120a39d700 1 -- 172.21.15.33:0/3395283012 shutdown_connections 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.570+0000 7f120a39d700 1 --2- 172.21.15.33:0/3395283012 >> v2:172.21.15.167:3300/0 conn(0x7f12040cd5e0 0x7f120415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.570+0000 7f120a39d700 1 --2- 172.21.15.33:0/3395283012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040af540 0x7f12040af910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.570+0000 7f120a39d700 1 -- 172.21.15.33:0/3395283012 >> 172.21.15.33:0/3395283012 conn(0x7f120406d9b0 msgr2=0x7f120406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.570+0000 7f120a39d700 1 -- 172.21.15.33:0/3395283012 shutdown_connections 2023-10-06T05:14:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.570+0000 7f120a39d700 1 -- 172.21.15.33:0/3395283012 wait complete. 2023-10-06T05:14:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.571+0000 7f120a39d700 1 Processor -- start 2023-10-06T05:14:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.571+0000 7f120a39d700 1 -- start start 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f120a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12040af540 0x7f12041551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f120a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 0x7f12040ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f120a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120415a650 0x7f12040ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f120a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204155720 con 0x7f12040cd5e0 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f120a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120416ac80 con 0x7f120415a650 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f120a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120416a070 con 0x7f12040af540 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.572+0000 7f1208b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 0x7f12040ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 0x7f12040ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46684/0 (socket says 172.21.15.33:46684) 2023-10-06T05:14:59.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 -- 172.21.15.33:0/356487662 learned_addr learned my addr 172.21.15.33:0/356487662 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f120939b700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12040af540 0x7f12041551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1209b9c700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120415a650 0x7f12040ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 -- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12040af540 msgr2=0x7f12041551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12040af540 0x7f12041551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 -- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120415a650 msgr2=0x7f12040ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120415a650 0x7f12040ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 -- 172.21.15.33:0/356487662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200009420 con 0x7f12040cd5e0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f1208b9a700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 0x7f12040ca0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c7e0 tx=0x7f11f4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.573+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4005b30 con 0x7f12040cd5e0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.574+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4005c90 con 0x7f12040cd5e0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.574+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4007300 con 0x7f12040cd5e0 2023-10-06T05:14:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.574+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120416a350 con 0x7f12040cd5e0 2023-10-06T05:14:59.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.574+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120416a910 con 0x7f12040cd5e0 2023-10-06T05:14:59.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.575+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120404c380 con 0x7f12040cd5e0 2023-10-06T05:14:59.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.576+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11f401b3b0 con 0x7f12040cd5e0 2023-10-06T05:14:59.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.577+0000 7f11f97fa700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e8066f30 0x7f11e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.577+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f11f4083c50 con 0x7f12040cd5e0 2023-10-06T05:14:59.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.577+0000 7f120939b700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e8066f30 0x7f11e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.578+0000 7f120939b700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e8066f30 0x7f11e80693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f12000093f0 tx=0x7f1200017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.581+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f4050cc0 con 0x7f12040cd5e0 2023-10-06T05:14:59.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.783+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f120415db70 con 0x7f12040cd5e0 2023-10-06T05:14:59.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.784+0000 7f11f97fa700 1 -- 172.21.15.33:0/356487662 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11f4017020 con 0x7f12040cd5e0 2023-10-06T05:14:59.786 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:14:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e8066f30 msgr2=0x7f11e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e8066f30 0x7f11e80693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f12000093f0 tx=0x7f1200017000 comp rx=0 tx=0).stop 2023-10-06T05:14:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 msgr2=0x7f12040ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 0x7f12040ca0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c7e0 tx=0x7f11f4007df0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 shutdown_connections 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e8066f30 0x7f11e80693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120415a650 0x7f12040ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12040af540 0x7f12041551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 --2- 172.21.15.33:0/356487662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12040cd5e0 0x7f12040ca0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.786+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 >> 172.21.15.33:0/356487662 conn(0x7f120406d9b0 msgr2=0x7f12040c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.787+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 shutdown_connections 2023-10-06T05:14:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.787+0000 7f120a39d700 1 -- 172.21.15.33:0/356487662 wait complete. 2023-10-06T05:14:59.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:59.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.992+0000 7f787a99d700 1 Processor -- start 2023-10-06T05:14:59.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.992+0000 7f787a99d700 1 -- start start 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.993+0000 7f787a99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78740ccfc0 0x7f78740cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.993+0000 7f787a99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740cbd80 con 0x7f78740cc750 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.993+0000 7f787a99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740af5f0 con 0x7f78740cf3b0 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.993+0000 7f787a99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78740af8d0 con 0x7f78740ccfc0 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78740ccfc0 0x7f78740cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78740ccfc0 0x7f78740cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55720/0 (socket says 172.21.15.33:55720) 2023-10-06T05:14:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 -- 172.21.15.33:0/2249277419 learned_addr learned my addr 172.21.15.33:0/2249277419 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787a19c700 1 --1- 172.21.15.33:0/2249277419 >> v1:172.21.15.33:6789/0 conn(0x7f78740cc750 0x7f78740b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54764/0 (socket says 172.21.15.33:54764) 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 -- 172.21.15.33:0/2249277419 >> v1:172.21.15.167:6789/0 conn(0x7f78740cf3b0 legacy=0x7f78741594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 -- 172.21.15.33:0/2249277419 >> v1:172.21.15.33:6789/0 conn(0x7f78740cc750 legacy=0x7f78740b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 -- 172.21.15.33:0/2249277419 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740afbb0 con 0x7f78740ccfc0 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f7878999700 1 -- 172.21.15.33:0/2249277419 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096905374 0 0) 0x7f78740cbd80 con 0x7f78740cc750 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.994+0000 7f787999b700 1 --2- 172.21.15.33:0/2249277419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740ccfc0 0x7f78740cc210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f78700099e0 tx=0x7f7870009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abec47ed9b32d64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f7878999700 1 -- 172.21.15.33:0/2249277419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000b820 con 0x7f78740ccfc0 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f7878999700 1 -- 172.21.15.33:0/2249277419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787000b980 con 0x7f78740ccfc0 2023-10-06T05:14:59.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f7878999700 1 -- 172.21.15.33:0/2249277419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000bd20 con 0x7f78740ccfc0 2023-10-06T05:14:59.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 -- 172.21.15.33:0/2249277419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740ccfc0 msgr2=0x7f78740cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 --2- 172.21.15.33:0/2249277419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740ccfc0 0x7f78740cc210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f78700099e0 tx=0x7f7870009250 comp rx=0 tx=0).stop 2023-10-06T05:14:59.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 -- 172.21.15.33:0/2249277419 shutdown_connections 2023-10-06T05:14:59.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 --2- 172.21.15.33:0/2249277419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740ccfc0 0x7f78740cc210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 -- 172.21.15.33:0/2249277419 >> 172.21.15.33:0/2249277419 conn(0x7f787406cff0 msgr2=0x7f787406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 -- 172.21.15.33:0/2249277419 shutdown_connections 2023-10-06T05:15:00.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.995+0000 7f787a99d700 1 -- 172.21.15.33:0/2249277419 wait complete. 2023-10-06T05:15:00.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.997+0000 7f787a99d700 1 Processor -- start 2023-10-06T05:15:00.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.997+0000 7f787a99d700 1 -- start start 2023-10-06T05:15:00.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78740cc750 0x7f78740c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78740ccfc0 0x7f787414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 0x7f78740b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740ca1b0 con 0x7f78740c8890 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b4c70 con 0x7f78740cc750 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b4f50 con 0x7f78740ccfc0 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 0x7f78740b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 0x7f78740b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55730/0 (socket says 172.21.15.33:55730) 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 -- 172.21.15.33:0/3216073237 learned_addr learned my addr 172.21.15.33:0/3216073237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787919a700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78740ccfc0 0x7f787414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787999b700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78740cc750 0x7f78740c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 -- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78740ccfc0 msgr2=0x7f787414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78740ccfc0 0x7f787414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 -- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78740cc750 msgr2=0x7f78740c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78740cc750 0x7f78740c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787a19c700 1 -- 172.21.15.33:0/3216073237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700094f0 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787919a700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78740ccfc0 0x7f787414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.998+0000 7f787999b700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78740cc750 0x7f78740c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.999+0000 7f787a19c700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 0x7f78740b0420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f786c00cd50 tx=0x7f786c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.999+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c005df0 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.999+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00a960 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.999+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78740b5230 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.999+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00ad20 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:14:59.999+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787414b720 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.000+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787404c380 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.001+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f786c007300 con 0x7f78740c8890 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.002+0000 7f7869ffb700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7858066f30 0x7f78580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.002+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f786c084930 con 0x7f78740c8890 2023-10-06T05:15:00.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.002+0000 7f787999b700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7858066f30 0x7f78580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.003+0000 7f787999b700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7858066f30 0x7f78580693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78700099e0 tx=0x7f7870017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.006+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f786c051a20 con 0x7f78740c8890 2023-10-06T05:15:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.209+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7874159bb0 con 0x7f78740c8890 2023-10-06T05:15:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.210+0000 7f7869ffb700 1 -- 172.21.15.33:0/3216073237 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f786c055040 con 0x7f78740c8890 2023-10-06T05:15:00.213 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:00.213 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.212+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7858066f30 msgr2=0x7f78580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.212+0000 7f787a99d700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7858066f30 0x7f78580693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78700099e0 tx=0x7f7870017000 comp rx=0 tx=0).stop 2023-10-06T05:15:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.212+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 msgr2=0x7f78740b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.212+0000 7f787a99d700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 0x7f78740b0420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f786c00cd50 tx=0x7f786c001b10 comp rx=0 tx=0).stop 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 shutdown_connections 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7858066f30 0x7f78580693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78740c8890 0x7f78740b0420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78740cc750 0x7f78740c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 --2- 172.21.15.33:0/3216073237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78740ccfc0 0x7f787414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 >> 172.21.15.33:0/3216073237 conn(0x7f787406cff0 msgr2=0x7f7874155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 shutdown_connections 2023-10-06T05:15:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.213+0000 7f787a99d700 1 -- 172.21.15.33:0/3216073237 wait complete. 2023-10-06T05:15:00.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:00.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.421+0000 7f01a49fe700 1 Processor -- start 2023-10-06T05:15:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.421+0000 7f01a49fe700 1 -- start start 2023-10-06T05:15:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.422+0000 7f01a49fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01a0154f20 0x7f01a00ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.422+0000 7f01a49fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a01552f0 0x7f01a00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f01a49fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a01586a0 con 0x7f01a00b11e0 2023-10-06T05:15:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f01a49fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a006bdf0 con 0x7f01a0154f20 2023-10-06T05:15:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f01a49fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0157ab0 con 0x7f01a01552f0 2023-10-06T05:15:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01a0154f20 0x7f01a00ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019f59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01a00b11e0 0x7f01a015b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54766/0 (socket says 172.21.15.33:54766) 2023-10-06T05:15:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019f59e700 1 -- 172.21.15.33:0/815882825 learned_addr learned my addr 172.21.15.33:0/815882825 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019e59c700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a01552f0 0x7f01a00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019dd9b700 1 -- 172.21.15.33:0/815882825 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876760794 0 0) 0x7f01a01586a0 con 0x7f01a00b11e0 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019ed9d700 1 -- 172.21.15.33:0/815882825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a01552f0 msgr2=0x7f01a00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019ed9d700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a01552f0 0x7f01a00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019ed9d700 1 -- 172.21.15.33:0/815882825 >> v1:172.21.15.33:6789/0 conn(0x7f01a00b11e0 legacy=0x7f01a015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019ed9d700 1 -- 172.21.15.33:0/815882825 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0157d70 con 0x7f01a0154f20 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.423+0000 7f019e59c700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a01552f0 0x7f01a00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f019ed9d700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0154f20 0x7f01a00ae720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f01900098e0 tx=0x7f01900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f072bbd3b74b4e69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f019dd9b700 1 -- 172.21.15.33:0/815882825 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190009e80 con 0x7f01a0154f20 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f019dd9b700 1 -- 172.21.15.33:0/815882825 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019000b860 con 0x7f01a0154f20 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f019dd9b700 1 -- 172.21.15.33:0/815882825 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000bbe0 con 0x7f01a0154f20 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f01a49fe700 1 -- 172.21.15.33:0/815882825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0154f20 msgr2=0x7f01a00ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f01a49fe700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0154f20 0x7f01a00ae720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f01900098e0 tx=0x7f01900091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f01a49fe700 1 -- 172.21.15.33:0/815882825 shutdown_connections 2023-10-06T05:15:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f01a49fe700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0154f20 0x7f01a00ae720 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f01a49fe700 1 --2- 172.21.15.33:0/815882825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a01552f0 0x7f01a00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.424+0000 7f01a49fe700 1 -- 172.21.15.33:0/815882825 >> 172.21.15.33:0/815882825 conn(0x7f01a006d030 msgr2=0x7f01a00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.425+0000 7f01a49fe700 1 -- 172.21.15.33:0/815882825 shutdown_connections 2023-10-06T05:15:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.425+0000 7f01a49fe700 1 -- 172.21.15.33:0/815882825 wait complete. 2023-10-06T05:15:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.426+0000 7f01a49fe700 1 Processor -- start 2023-10-06T05:15:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.426+0000 7f01a49fe700 1 -- start start 2023-10-06T05:15:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f01a49fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 0x7f01a0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f01a49fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a0154f20 0x7f01a01532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f01a49fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01a0156e50 0x7f01a006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f01a49fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0167960 con 0x7f01a00b11e0 2023-10-06T05:15:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f01a49fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0156c10 con 0x7f01a0156e50 2023-10-06T05:15:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f01a49fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01681a0 con 0x7f01a0154f20 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f019ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 0x7f01a0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f019ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 0x7f01a0156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55750/0 (socket says 172.21.15.33:55750) 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f019ed9d700 1 -- 172.21.15.33:0/170708420 learned_addr learned my addr 172.21.15.33:0/170708420 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f019e59c700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a0154f20 0x7f01a01532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.427+0000 7f019f59e700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01a0156e50 0x7f01a006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019ed9d700 1 -- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a0154f20 msgr2=0x7f01a01532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019ed9d700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a0154f20 0x7f01a01532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019ed9d700 1 -- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01a0156e50 msgr2=0x7f01a006aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019ed9d700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01a0156e50 0x7f01a006aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019ed9d700 1 -- 172.21.15.33:0/170708420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190009420 con 0x7f01a00b11e0 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019e59c700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a0154f20 0x7f01a01532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019f59e700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01a0156e50 0x7f01a006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f019ed9d700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 0x7f01a0156090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f019000b3c0 tx=0x7f0190017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000bbe0 con 0x7f01a00b11e0 2023-10-06T05:15:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0190011420 con 0x7f01a00b11e0 2023-10-06T05:15:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01900117c0 con 0x7f01a00b11e0 2023-10-06T05:15:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0156930 con 0x7f01a00b11e0 2023-10-06T05:15:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.428+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a0167e30 con 0x7f01a00b11e0 2023-10-06T05:15:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.429+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a004c380 con 0x7f01a00b11e0 2023-10-06T05:15:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.430+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0190011b70 con 0x7f01a00b11e0 2023-10-06T05:15:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.431+0000 7f0186ffd700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f018c066f30 0x7f018c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.431+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01900860f0 con 0x7f01a00b11e0 2023-10-06T05:15:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.432+0000 7f019e59c700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f018c066f30 0x7f018c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.432+0000 7f019e59c700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f018c066f30 0x7f018c0693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01a0158730 tx=0x7f0194008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.435+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0190053160 con 0x7f01a00b11e0 2023-10-06T05:15:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.635+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01a006f370 con 0x7f01a00b11e0 2023-10-06T05:15:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.636+0000 7f0186ffd700 1 -- 172.21.15.33:0/170708420 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0190056780 con 0x7f01a00b11e0 2023-10-06T05:15:00.638 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:15:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f018c066f30 msgr2=0x7f018c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f018c066f30 0x7f018c0693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01a0158730 tx=0x7f0194008040 comp rx=0 tx=0).stop 2023-10-06T05:15:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 msgr2=0x7f01a0156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 0x7f01a0156090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f019000b3c0 tx=0x7f0190017000 comp rx=0 tx=0).stop 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 shutdown_connections 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f018c066f30 0x7f018c0693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01a0156e50 0x7f01a006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a00b11e0 0x7f01a0156090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 --2- 172.21.15.33:0/170708420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01a0154f20 0x7f01a01532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.638+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 >> 172.21.15.33:0/170708420 conn(0x7f01a006d030 msgr2=0x7f01a00aba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.639+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 shutdown_connections 2023-10-06T05:15:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.639+0000 7f01a49fe700 1 -- 172.21.15.33:0/170708420 wait complete. 2023-10-06T05:15:00.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:00.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.845+0000 7fcd7f19d700 1 Processor -- start 2023-10-06T05:15:00.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.846+0000 7fcd7f19d700 1 -- start start 2023-10-06T05:15:00.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd78156370 0x7fcd780af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd780cc7e0 0x7fcd780c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7f19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcd780b01f0 0x7fcd780b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780ad430 con 0x7fcd780cc7e0 2023-10-06T05:15:00.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7f19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780afe20 con 0x7fcd780b01f0 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780b0b70 con 0x7fcd78156370 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd780cc7e0 0x7fcd780c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd780cc7e0 0x7fcd780c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55756/0 (socket says 172.21.15.33:55756) 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.847+0000 7fcd7d99a700 1 -- 172.21.15.33:0/1791821464 learned_addr learned my addr 172.21.15.33:0/1791821464 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7e99c700 1 --2- 172.21.15.33:0/1791821464 >> v2:172.21.15.167:3300/0 conn(0x7fcd780b01f0 0x7fcd780b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7e19b700 1 --2- 172.21.15.33:0/1791821464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd78156370 0x7fcd780af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d99a700 1 -- 172.21.15.33:0/1791821464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd78156370 msgr2=0x7fcd780af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d99a700 1 --2- 172.21.15.33:0/1791821464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd78156370 0x7fcd780af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d99a700 1 -- 172.21.15.33:0/1791821464 >> v2:172.21.15.167:3300/0 conn(0x7fcd780b01f0 msgr2=0x7fcd780b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d99a700 1 --2- 172.21.15.33:0/1791821464 >> v2:172.21.15.167:3300/0 conn(0x7fcd780b01f0 0x7fcd780b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d99a700 1 -- 172.21.15.33:0/1791821464 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7806bc70 con 0x7fcd780cc7e0 2023-10-06T05:15:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d99a700 1 --2- 172.21.15.33:0/1791821464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd780cc7e0 0x7fcd780c8db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd78069810 tx=0x7fcd6800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e370a95de53f88f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d199700 1 -- 172.21.15.33:0/1791821464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800dc50 con 0x7fcd780cc7e0 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d199700 1 -- 172.21.15.33:0/1791821464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800ddb0 con 0x7fcd780cc7e0 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.848+0000 7fcd7d199700 1 -- 172.21.15.33:0/1791821464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd680196d0 con 0x7fcd780cc7e0 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 -- 172.21.15.33:0/1791821464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd780cc7e0 msgr2=0x7fcd780c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/1791821464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd780cc7e0 0x7fcd780c8db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd78069810 tx=0x7fcd6800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 -- 172.21.15.33:0/1791821464 shutdown_connections 2023-10-06T05:15:00.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/1791821464 >> v2:172.21.15.167:3300/0 conn(0x7fcd780b01f0 0x7fcd780b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/1791821464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd78156370 0x7fcd780af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/1791821464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd780cc7e0 0x7fcd780c8db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 -- 172.21.15.33:0/1791821464 >> 172.21.15.33:0/1791821464 conn(0x7fcd7806d030 msgr2=0x7fcd780cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 -- 172.21.15.33:0/1791821464 shutdown_connections 2023-10-06T05:15:00.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.849+0000 7fcd7f19d700 1 -- 172.21.15.33:0/1791821464 wait complete. 2023-10-06T05:15:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.850+0000 7fcd7f19d700 1 Processor -- start 2023-10-06T05:15:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 -- start start 2023-10-06T05:15:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd780b01f0 0x7fcd78151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780cc7e0 0x7fcd780abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 0x7fcd780ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78154390 con 0x7fcd78156370 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78153780 con 0x7fcd780cc7e0 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.851+0000 7fcd7f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78153a60 con 0x7fcd780b01f0 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 0x7fcd780ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 0x7fcd780ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55758/0 (socket says 172.21.15.33:55758) 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 -- 172.21.15.33:0/3524067344 learned_addr learned my addr 172.21.15.33:0/3524067344 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7d99a700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780cc7e0 0x7fcd780abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e19b700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd780b01f0 0x7fcd78151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 -- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd780b01f0 msgr2=0x7fcd78151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd780b01f0 0x7fcd78151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 -- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780cc7e0 msgr2=0x7fcd780abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780cc7e0 0x7fcd780abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 -- 172.21.15.33:0/3524067344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6800b790 con 0x7fcd78156370 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.852+0000 7fcd7e99c700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 0x7fcd780ac070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009fa0 tx=0x7fcd70009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.853+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000ed20 con 0x7fcd78156370 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.853+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000ee80 con 0x7fcd78156370 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.853+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70014670 con 0x7fcd78156370 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.853+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd78153d40 con 0x7fcd78156370 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.853+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd78151870 con 0x7fcd78156370 2023-10-06T05:15:00.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.854+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd78154500 con 0x7fcd78156370 2023-10-06T05:15:00.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.855+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd700149e0 con 0x7fcd78156370 2023-10-06T05:15:00.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.855+0000 7fcd6e7fc700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd5c066f30 0x7fcd5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.856+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcd70084f20 con 0x7fcd78156370 2023-10-06T05:15:00.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.856+0000 7fcd7e19b700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd5c066f30 0x7fcd5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.856+0000 7fcd7e19b700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd5c066f30 0x7fcd5c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd74006fd0 tx=0x7fcd74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:00.860+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd70051f90 con 0x7fcd78156370 2023-10-06T05:15:01.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.062+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd7804c380 con 0x7fcd78156370 2023-10-06T05:15:01.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.062+0000 7fcd6e7fc700 1 -- 172.21.15.33:0/3524067344 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd700555b0 con 0x7fcd78156370 2023-10-06T05:15:01.064 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd5c066f30 msgr2=0x7fcd5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd5c066f30 0x7fcd5c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd74006fd0 tx=0x7fcd74008040 comp rx=0 tx=0).stop 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 msgr2=0x7fcd780ac070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 0x7fcd780ac070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009fa0 tx=0x7fcd70009900 comp rx=0 tx=0).stop 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 shutdown_connections 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd5c066f30 0x7fcd5c0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78156370 0x7fcd780ac070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd780b01f0 0x7fcd78151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 --2- 172.21.15.33:0/3524067344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd780cc7e0 0x7fcd780abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 >> 172.21.15.33:0/3524067344 conn(0x7fcd7806d030 msgr2=0x7fcd780af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 shutdown_connections 2023-10-06T05:15:01.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.065+0000 7fcd7f19d700 1 -- 172.21.15.33:0/3524067344 wait complete. 2023-10-06T05:15:01.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:01.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.272+0000 7f17a199d700 1 Processor -- start 2023-10-06T05:15:01.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.272+0000 7f17a199d700 1 -- start start 2023-10-06T05:15:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f17a199d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f179c155a50 0x7f179c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f17a199d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f179c0afb20 0x7f179c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f17a199d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179c0b06b0 con 0x7f179c0c9490 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f17a199d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c0c99e0 con 0x7f179c0afb20 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f17a199d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c0c9cc0 con 0x7f179c155a50 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f179bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f179c0c9490 0x7f179c0af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54768/0 (socket says 172.21.15.33:54768) 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f179bfff700 1 -- 172.21.15.33:0/4097637781 learned_addr learned my addr 172.21.15.33:0/4097637781 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.273+0000 7f17a119c700 1 --2- 172.21.15.33:0/4097637781 >> v2:172.21.15.33:3300/0 conn(0x7f179c0afb20 0x7f179c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f17a099b700 1 --2- 172.21.15.33:0/4097637781 >> v2:172.21.15.167:3300/0 conn(0x7f179c155a50 0x7f179c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f17a119c700 1 -- 172.21.15.33:0/4097637781 >> v2:172.21.15.167:3300/0 conn(0x7f179c155a50 msgr2=0x7f179c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f17a119c700 1 --2- 172.21.15.33:0/4097637781 >> v2:172.21.15.167:3300/0 conn(0x7f179c155a50 0x7f179c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f17a119c700 1 -- 172.21.15.33:0/4097637781 >> v1:172.21.15.33:6789/0 conn(0x7f179c0c9490 legacy=0x7f179c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f17a119c700 1 -- 172.21.15.33:0/4097637781 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c0c9fa0 con 0x7f179c0afb20 2023-10-06T05:15:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f179b7fe700 1 -- 172.21.15.33:0/4097637781 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197304460 0 0) 0x7f179c0b06b0 con 0x7f179c0c9490 2023-10-06T05:15:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f17a119c700 1 --2- 172.21.15.33:0/4097637781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c0caa80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f179c0cd1e0 tx=0x7f1794009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36b2ad4f3ebddc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.274+0000 7f179b7fe700 1 -- 172.21.15.33:0/4097637781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179400e070 con 0x7f179c0afb20 2023-10-06T05:15:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f179b7fe700 1 -- 172.21.15.33:0/4097637781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179400bb60 con 0x7f179c0afb20 2023-10-06T05:15:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f179b7fe700 1 -- 172.21.15.33:0/4097637781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794012430 con 0x7f179c0afb20 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 -- 172.21.15.33:0/4097637781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 msgr2=0x7f179c0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 --2- 172.21.15.33:0/4097637781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c0caa80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f179c0cd1e0 tx=0x7f1794009780 comp rx=0 tx=0).stop 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 -- 172.21.15.33:0/4097637781 shutdown_connections 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 --2- 172.21.15.33:0/4097637781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c0caa80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 --2- 172.21.15.33:0/4097637781 >> v2:172.21.15.167:3300/0 conn(0x7f179c155a50 0x7f179c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 -- 172.21.15.33:0/4097637781 >> 172.21.15.33:0/4097637781 conn(0x7f179c06d030 msgr2=0x7f179c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 -- 172.21.15.33:0/4097637781 shutdown_connections 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.275+0000 7f17a199d700 1 -- 172.21.15.33:0/4097637781 wait complete. 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.277+0000 7f17a199d700 1 Processor -- start 2023-10-06T05:15:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.277+0000 7f17a199d700 1 -- start start 2023-10-06T05:15:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179c0c9490 0x7f179c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179c155a50 0x7f179c169970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c159410 con 0x7f179c0afb20 2023-10-06T05:15:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c158800 con 0x7f179c0c9490 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c158ae0 con 0x7f179c155a50 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55778/0 (socket says 172.21.15.33:55778) 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 -- 172.21.15.33:0/1371515465 learned_addr learned my addr 172.21.15.33:0/1371515465 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f179bfff700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179c0c9490 0x7f179c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a119c700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179c155a50 0x7f179c169970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 -- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179c155a50 msgr2=0x7f179c169970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179c155a50 0x7f179c169970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 -- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179c0c9490 msgr2=0x7f179c14de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179c0c9490 0x7f179c14de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.278+0000 7f17a099b700 1 -- 172.21.15.33:0/1371515465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794009a20 con 0x7f179c0afb20 2023-10-06T05:15:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.279+0000 7f17a099b700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c154020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f179000ec80 tx=0x7f17900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.279+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000c850 con 0x7f179c0afb20 2023-10-06T05:15:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.279+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179000c9b0 con 0x7f179c0afb20 2023-10-06T05:15:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.279+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c158dc0 con 0x7f179c0afb20 2023-10-06T05:15:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.279+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000cd50 con 0x7f179c0afb20 2023-10-06T05:15:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.279+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c14e360 con 0x7f179c0afb20 2023-10-06T05:15:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.280+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c1596f0 con 0x7f179c0afb20 2023-10-06T05:15:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.281+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1790013680 con 0x7f179c0afb20 2023-10-06T05:15:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.282+0000 7f1798ff9700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1780066f30 0x7f17800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.282+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1790084950 con 0x7f179c0afb20 2023-10-06T05:15:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.282+0000 7f179bfff700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1780066f30 0x7f17800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.283+0000 7f179bfff700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1780066f30 0x7f17800693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f179c0cb880 tx=0x7f1788007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.286+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17900519c0 con 0x7f179c0afb20 2023-10-06T05:15:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.485+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f179c04c380 con 0x7f179c0afb20 2023-10-06T05:15:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.485+0000 7f1798ff9700 1 -- 172.21.15.33:0/1371515465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1790054fe0 con 0x7f179c0afb20 2023-10-06T05:15:01.487 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:15:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.487+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1780066f30 msgr2=0x7f17800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.487+0000 7f17a199d700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1780066f30 0x7f17800693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f179c0cb880 tx=0x7f1788007d30 comp rx=0 tx=0).stop 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.487+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 msgr2=0x7f179c154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.487+0000 7f17a199d700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c154020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f179000ec80 tx=0x7f17900091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 shutdown_connections 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1780066f30 0x7f17800693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f179c155a50 0x7f179c169970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0afb20 0x7f179c154020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 --2- 172.21.15.33:0/1371515465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f179c0c9490 0x7f179c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 >> 172.21.15.33:0/1371515465 conn(0x7f179c06d030 msgr2=0x7f179c06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 shutdown_connections 2023-10-06T05:15:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.488+0000 7f17a199d700 1 -- 172.21.15.33:0/1371515465 wait complete. 2023-10-06T05:15:01.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.695+0000 7f84501ff700 1 Processor -- start 2023-10-06T05:15:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.696+0000 7f84501ff700 1 -- start start 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f84501ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8448159800 0x7f84480af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f84501ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84480aec30 con 0x7f84480af0c0 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f84501ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84480ae300 con 0x7f844806bb60 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f84501ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84480ae5e0 con 0x7f8448159800 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f844df9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8448159800 0x7f84480af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f844df9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8448159800 0x7f84480af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55794/0 (socket says 172.21.15.33:55794) 2023-10-06T05:15:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.697+0000 7f844df9b700 1 -- 172.21.15.33:0/4262195657 learned_addr learned my addr 172.21.15.33:0/4262195657 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844df9b700 1 -- 172.21.15.33:0/4262195657 >> v1:172.21.15.167:6789/0 conn(0x7f844806bb60 legacy=0x7f8448159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844df9b700 1 -- 172.21.15.33:0/4262195657 >> v1:172.21.15.33:6789/0 conn(0x7f84480af0c0 legacy=0x7f844806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844df9b700 1 -- 172.21.15.33:0/4262195657 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480ae8c0 con 0x7f8448159800 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844df9b700 1 --2- 172.21.15.33:0/4262195657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8448159800 0x7f84480af520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84440099b0 tx=0x7f8444009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93659ff59c68d87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844cf99700 1 -- 172.21.15.33:0/4262195657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400b860 con 0x7f8448159800 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844cf99700 1 -- 172.21.15.33:0/4262195657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844400b9c0 con 0x7f8448159800 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.698+0000 7f844cf99700 1 -- 172.21.15.33:0/4262195657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400bd80 con 0x7f8448159800 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 -- 172.21.15.33:0/4262195657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8448159800 msgr2=0x7f84480af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 --2- 172.21.15.33:0/4262195657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8448159800 0x7f84480af520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84440099b0 tx=0x7f8444009250 comp rx=0 tx=0).stop 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 -- 172.21.15.33:0/4262195657 shutdown_connections 2023-10-06T05:15:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 --2- 172.21.15.33:0/4262195657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8448159800 0x7f84480af520 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 -- 172.21.15.33:0/4262195657 >> 172.21.15.33:0/4262195657 conn(0x7f84480cde20 msgr2=0x7f844806ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 -- 172.21.15.33:0/4262195657 shutdown_connections 2023-10-06T05:15:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.699+0000 7f84501ff700 1 -- 172.21.15.33:0/4262195657 wait complete. 2023-10-06T05:15:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.700+0000 7f84501ff700 1 Processor -- start 2023-10-06T05:15:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.700+0000 7f84501ff700 1 -- start start 2023-10-06T05:15:01.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.701+0000 7f84501ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 0x7f844814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.701+0000 7f84501ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84480af0c0 0x7f844814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.701+0000 7f84501ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8448159800 0x7f8448148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.701+0000 7f84501ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844814b2a0 con 0x7f844806bb60 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.701+0000 7f84501ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481546b0 con 0x7f8448159800 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.701+0000 7f84501ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8448153d80 con 0x7f84480af0c0 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 0x7f844814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 0x7f844814d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55796/0 (socket says 172.21.15.33:55796) 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 -- 172.21.15.33:0/712269863 learned_addr learned my addr 172.21.15.33:0/712269863 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844e79c700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8448159800 0x7f8448148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844d79a700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84480af0c0 0x7f844814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 -- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84480af0c0 msgr2=0x7f844814aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84480af0c0 0x7f844814aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 -- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8448159800 msgr2=0x7f8448148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8448159800 0x7f8448148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 -- 172.21.15.33:0/712269863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440094f0 con 0x7f844806bb60 2023-10-06T05:15:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844d79a700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84480af0c0 0x7f844814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:01.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f844df9b700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 0x7f844814d780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f844806ef80 tx=0x7f8444017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844401f070 con 0x7f844806bb60 2023-10-06T05:15:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.702+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8444002f30 con 0x7f844806bb60 2023-10-06T05:15:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.703+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444025420 con 0x7f844806bb60 2023-10-06T05:15:01.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.703+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8448154060 con 0x7f844806bb60 2023-10-06T05:15:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.703+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844814afb0 con 0x7f844806bb60 2023-10-06T05:15:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.704+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84481560a0 con 0x7f844806bb60 2023-10-06T05:15:01.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.704+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8444023070 con 0x7f844806bb60 2023-10-06T05:15:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.705+0000 7f843dffb700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f842c066fb0 0x7f842c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.705+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8444085890 con 0x7f844806bb60 2023-10-06T05:15:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.706+0000 7f844d79a700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f842c066fb0 0x7f842c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.707+0000 7f844d79a700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f842c066fb0 0x7f842c069460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8448153b80 tx=0x7f8438009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.709+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8444052900 con 0x7f844806bb60 2023-10-06T05:15:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.914+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f844804c380 con 0x7f844806bb60 2023-10-06T05:15:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.914+0000 7f843dffb700 1 -- 172.21.15.33:0/712269863 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f844402a020 con 0x7f844806bb60 2023-10-06T05:15:01.916 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:15:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.916+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f842c066fb0 msgr2=0x7f842c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.916+0000 7f84501ff700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f842c066fb0 0x7f842c069460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8448153b80 tx=0x7f8438009040 comp rx=0 tx=0).stop 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 msgr2=0x7f844814d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 0x7f844814d780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f844806ef80 tx=0x7f8444017000 comp rx=0 tx=0).stop 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 shutdown_connections 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f842c066fb0 0x7f842c069460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8448159800 0x7f8448148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f844806bb60 0x7f844814d780 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 --2- 172.21.15.33:0/712269863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84480af0c0 0x7f844814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 >> 172.21.15.33:0/712269863 conn(0x7f84480cde20 msgr2=0x7f8448155990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 shutdown_connections 2023-10-06T05:15:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:01.917+0000 7f84501ff700 1 -- 172.21.15.33:0/712269863 wait complete. 2023-10-06T05:15:01.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.122+0000 7fabe9d9d700 1 Processor -- start 2023-10-06T05:15:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.122+0000 7fabe9d9d700 1 -- start start 2023-10-06T05:15:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.123+0000 7fabe9d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fabe40ccfc0 0x7fabe40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.123+0000 7fabe9d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 0x7fabe40b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.123+0000 7fabe9d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe40cbd80 con 0x7fabe40cf3b0 2023-10-06T05:15:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.123+0000 7fabe9d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe40af5f0 con 0x7fabe40ccfc0 2023-10-06T05:15:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.123+0000 7fabe9d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40af8d0 con 0x7fabe40cc750 2023-10-06T05:15:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fabe40cf3b0 0x7fabe41594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54786/0 (socket says 172.21.15.33:54786) 2023-10-06T05:15:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe3fff700 1 -- 172.21.15.33:0/339266158 learned_addr learned my addr 172.21.15.33:0/339266158 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe8d9b700 1 --2- 172.21.15.33:0/339266158 >> v2:172.21.15.167:3300/0 conn(0x7fabe40ccfc0 0x7fabe40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe959c700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 0x7fabe40b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802391553 0 0) 0x7fabe40cbd80 con 0x7fabe40cf3b0 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd4003570 con 0x7fabe40cf3b0 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2973501983 0 0) 0x7fabd4003570 con 0x7fabe40cf3b0 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 msgr2=0x7fabe40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 0x7fabe40b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 >> v2:172.21.15.167:3300/0 conn(0x7fabe40ccfc0 msgr2=0x7fabe40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 --2- 172.21.15.33:0/339266158 >> v2:172.21.15.167:3300/0 conn(0x7fabe40ccfc0 0x7fabe40cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe40afbb0 con 0x7fabe40cf3b0 2023-10-06T05:15:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fabd0001470 con 0x7fabe40cf3b0 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.124+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 >> v1:172.21.15.33:6789/0 conn(0x7fabe40cf3b0 legacy=0x7fabe41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe37fe700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd4003a60 0x7fabd4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe37fe700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 0x7fabd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe37fe700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40af5f0 con 0x7fabd4004d50 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40af8d0 con 0x7fabd4003a60 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd400ba90 con 0x7fabe40cc750 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe959c700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe8d9b700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd4003a60 0x7fabd4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe3fff700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 0x7fabd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe959c700 1 -- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 msgr2=0x7fabd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe959c700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe40cc750 0x7fabd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe959c700 1 -- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd4003a60 msgr2=0x7fabd4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe959c700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd4003a60 0x7fabd4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.125+0000 7fabe959c700 1 -- 172.21.15.33:0/339266158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4003570 con 0x7fabd4004d50 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe959c700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabd40051c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00e210 tx=0x7fabdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95a61e2b439ede0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00ea60 con 0x7fabd4004d50 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc00f040 con 0x7fabd4004d50 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe37fe700 1 -- 172.21.15.33:0/339266158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00ee10 con 0x7fabd4004d50 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe9d9d700 1 -- 172.21.15.33:0/339266158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 msgr2=0x7fabd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/339266158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabd40051c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00e210 tx=0x7fabdc00b040 comp rx=0 tx=0).stop 2023-10-06T05:15:02.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe959c700 1 -- 172.21.15.33:0/339266158 reap_dead start 2023-10-06T05:15:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe9d9d700 1 -- 172.21.15.33:0/339266158 shutdown_connections 2023-10-06T05:15:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe9d9d700 1 -- 172.21.15.33:0/339266158 >> 172.21.15.33:0/339266158 conn(0x7fabe406cff0 msgr2=0x7fabe406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.126+0000 7fabe9d9d700 1 -- 172.21.15.33:0/339266158 shutdown_connections 2023-10-06T05:15:02.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.127+0000 7fabe9d9d700 1 -- 172.21.15.33:0/339266158 wait complete. 2023-10-06T05:15:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.128+0000 7fabe9d9d700 1 Processor -- start 2023-10-06T05:15:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.128+0000 7fabe9d9d700 1 -- start start 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe9d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabe40b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe9d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabe40c9b00 0x7fabe40c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe9d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe4153c40 0x7fabe41540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe9d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40b4290 con 0x7fabd4004d50 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe9d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe4154600 con 0x7fabe40c9b00 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe9d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40b3610 con 0x7fabe4153c40 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe8d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabe40b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe8d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabe40b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55824/0 (socket says 172.21.15.33:55824) 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe8d9b700 1 -- 172.21.15.33:0/2248386486 learned_addr learned my addr 172.21.15.33:0/2248386486 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe3fff700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabe40c9b00 0x7fabe40c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.129+0000 7fabe959c700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe4153c40 0x7fabe41540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe8d9b700 1 -- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe4153c40 msgr2=0x7fabe41540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe8d9b700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe4153c40 0x7fabe41540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe8d9b700 1 -- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabe40c9b00 msgr2=0x7fabe40c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe8d9b700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabe40c9b00 0x7fabe40c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe8d9b700 1 -- 172.21.15.33:0/2248386486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe40b38f0 con 0x7fabd4004d50 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe3fff700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabe40c9b00 0x7fabe40c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe8d9b700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabe40b5190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011b10 tx=0x7fabd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd800f990 con 0x7fabd4004d50 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd800faf0 con 0x7fabd4004d50 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe40b3bd0 con 0x7fabd4004d50 2023-10-06T05:15:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd800fe70 con 0x7fabd4004d50 2023-10-06T05:15:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.130+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe40b32a0 con 0x7fabd4004d50 2023-10-06T05:15:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.131+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe404c380 con 0x7fabd4004d50 2023-10-06T05:15:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.132+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabd8014650 con 0x7fabd4004d50 2023-10-06T05:15:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.133+0000 7fabe0ff9700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc8066f30 0x7fabc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.133+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fabd8018030 con 0x7fabd4004d50 2023-10-06T05:15:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.134+0000 7fabe3fff700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc8066f30 0x7fabc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.134+0000 7fabe3fff700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc8066f30 0x7fabc80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fabd00015a0 tx=0x7fabd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.137+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabd8055c50 con 0x7fabd4004d50 2023-10-06T05:15:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.343+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabe4159bb0 con 0x7fabd4004d50 2023-10-06T05:15:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.344+0000 7fabe0ff9700 1 -- 172.21.15.33:0/2248386486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fabd8022090 con 0x7fabd4004d50 2023-10-06T05:15:02.346 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:02.346 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:14:26.713770+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc8066f30 msgr2=0x7fabc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc8066f30 0x7fabc80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fabd00015a0 tx=0x7fabd0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 msgr2=0x7fabe40b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabe40b5190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011b10 tx=0x7fabd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 shutdown_connections 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc8066f30 0x7fabc80693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabe4153c40 0x7fabe41540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd4004d50 0x7fabe40b5190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 --2- 172.21.15.33:0/2248386486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabe40c9b00 0x7fabe40c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 >> 172.21.15.33:0/2248386486 conn(0x7fabe406cff0 msgr2=0x7fabe40c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 shutdown_connections 2023-10-06T05:15:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.346+0000 7fabe9d9d700 1 -- 172.21.15.33:0/2248386486 wait complete. 2023-10-06T05:15:02.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:02.365 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.554+0000 7f324d19d700 1 Processor -- start 2023-10-06T05:15:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.555+0000 7f324d19d700 1 -- start start 2023-10-06T05:15:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f324d19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32480aff90 0x7f32480a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f324d19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f324806cb80 0x7f324815d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f324d19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324815d3a0 con 0x7f324806c7b0 2023-10-06T05:15:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f324d19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480686c0 con 0x7f32480aff90 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f324d19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480b0470 con 0x7f324806cb80 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f32477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32480aff90 0x7f32480a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f32477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32480aff90 0x7f32480a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55826/0 (socket says 172.21.15.33:55826) 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f32477fe700 1 -- 172.21.15.33:0/1247481266 learned_addr learned my addr 172.21.15.33:0/1247481266 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.556+0000 7f324c99c700 1 --2- 172.21.15.33:0/1247481266 >> v2:172.21.15.167:3300/0 conn(0x7f324806cb80 0x7f324815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f32477fe700 1 -- 172.21.15.33:0/1247481266 >> v2:172.21.15.167:3300/0 conn(0x7f324806cb80 msgr2=0x7f324815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f32477fe700 1 --2- 172.21.15.33:0/1247481266 >> v2:172.21.15.167:3300/0 conn(0x7f324806cb80 0x7f324815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f32477fe700 1 -- 172.21.15.33:0/1247481266 >> v1:172.21.15.167:6789/0 conn(0x7f324806c7b0 legacy=0x7f3248067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f32477fe700 1 -- 172.21.15.33:0/1247481266 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480b0750 con 0x7f32480aff90 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f324c99c700 1 --2- 172.21.15.33:0/1247481266 >> v2:172.21.15.167:3300/0 conn(0x7f324806cb80 0x7f324815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f32477fe700 1 --2- 172.21.15.33:0/1247481266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f32480a9890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32480b0ff0 tx=0x7f323c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c53c699aed9ac3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f3246ffd700 1 -- 172.21.15.33:0/1247481266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00b860 con 0x7f32480aff90 2023-10-06T05:15:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f3246ffd700 1 -- 172.21.15.33:0/1247481266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c00b9c0 con 0x7f32480aff90 2023-10-06T05:15:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.557+0000 7f3246ffd700 1 -- 172.21.15.33:0/1247481266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00bd60 con 0x7f32480aff90 2023-10-06T05:15:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 -- 172.21.15.33:0/1247481266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 msgr2=0x7f32480a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 --2- 172.21.15.33:0/1247481266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f32480a9890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32480b0ff0 tx=0x7f323c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 -- 172.21.15.33:0/1247481266 shutdown_connections 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 --2- 172.21.15.33:0/1247481266 >> v2:172.21.15.167:3300/0 conn(0x7f324806cb80 0x7f324815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 --2- 172.21.15.33:0/1247481266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f32480a9890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 -- 172.21.15.33:0/1247481266 >> 172.21.15.33:0/1247481266 conn(0x7f324806d670 msgr2=0x7f324806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 -- 172.21.15.33:0/1247481266 shutdown_connections 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.558+0000 7f324d19d700 1 -- 172.21.15.33:0/1247481266 wait complete. 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.559+0000 7f324d19d700 1 Processor -- start 2023-10-06T05:15:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.559+0000 7f324d19d700 1 -- start start 2023-10-06T05:15:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.560+0000 7f324d19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f324806c7b0 0x7f3248159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.560+0000 7f324d19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f324806cb80 0x7f324815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.560+0000 7f324d19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f324816ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.560+0000 7f324d19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324816c5e0 con 0x7f32480aff90 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.560+0000 7f324d19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324816b9d0 con 0x7f324806c7b0 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.560+0000 7f324d19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324816bcb0 con 0x7f324806cb80 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f324816ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f324816ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55838/0 (socket says 172.21.15.33:55838) 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 -- 172.21.15.33:0/1420306999 learned_addr learned my addr 172.21.15.33:0/1420306999 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f3247fff700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f324806c7b0 0x7f3248159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f32477fe700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f324806cb80 0x7f324815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 -- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f324806cb80 msgr2=0x7f324815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f324806cb80 0x7f324815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 -- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f324806c7b0 msgr2=0x7f3248159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f324806c7b0 0x7f3248159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 -- 172.21.15.33:0/1420306999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0094f0 con 0x7f32480aff90 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f324c99c700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f324816ca70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f324000d180 tx=0x7f3240009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.561+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000dcc0 con 0x7f32480aff90 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.562+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324000f040 con 0x7f32480aff90 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.562+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240015690 con 0x7f32480aff90 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.562+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324816bf90 con 0x7f32480aff90 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.562+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324816b660 con 0x7f32480aff90 2023-10-06T05:15:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.563+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324804c380 con 0x7f32480aff90 2023-10-06T05:15:02.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.564+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f324001e3b0 con 0x7f32480aff90 2023-10-06T05:15:02.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.564+0000 7f322ffff700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.564+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3240084bf0 con 0x7f32480aff90 2023-10-06T05:15:02.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.565+0000 7f3247fff700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.566+0000 7f3247fff700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3238007fd0 tx=0x7f3238007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.568+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3240051c60 con 0x7f32480aff90 2023-10-06T05:15:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.770+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f324816c720 con 0x7f32480aff90 2023-10-06T05:15:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.770+0000 7f322ffff700 1 -- 172.21.15.33:0/1420306999 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f324001a0a0 con 0x7f32480aff90 2023-10-06T05:15:02.772 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 msgr2=0x7f32340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3238007fd0 tx=0x7f3238007eb0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 msgr2=0x7f324816ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f324816ca70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f324000d180 tx=0x7f3240009a50 comp rx=0 tx=0).stop 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 shutdown_connections 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32480aff90 0x7f324816ca70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f324806c7b0 0x7f3248159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 --2- 172.21.15.33:0/1420306999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f324806cb80 0x7f324815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 >> 172.21.15.33:0/1420306999 conn(0x7f324806d670 msgr2=0x7f32480b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 shutdown_connections 2023-10-06T05:15:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.773+0000 7f324d19d700 1 -- 172.21.15.33:0/1420306999 wait complete. 2023-10-06T05:15:02.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.981+0000 7f1e80f9d700 1 Processor -- start 2023-10-06T05:15:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.981+0000 7f1e80f9d700 1 -- start start 2023-10-06T05:15:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e80f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e80f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1e7c0c9490 0x7f1e7c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e80f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0b06b0 con 0x7f1e7c0af950 2023-10-06T05:15:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e80f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0cba90 con 0x7f1e7c0c9490 2023-10-06T05:15:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e80f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c99e0 con 0x7f1e7c155a50 2023-10-06T05:15:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e7b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e7b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:42440/0 (socket says 172.21.15.33:42440) 2023-10-06T05:15:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e7b7fe700 1 -- 172.21.15.33:0/3386072938 learned_addr learned my addr 172.21.15.33:0/3386072938 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.983+0000 7f1e7affd700 1 --2- 172.21.15.33:0/3386072938 >> v2:172.21.15.167:3300/0 conn(0x7f1e7c0c9490 0x7f1e7c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7a7fc700 1 -- 172.21.15.33:0/3386072938 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515996692 0 0) 0x7f1e7c0b06b0 con 0x7f1e7c0af950 2023-10-06T05:15:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7a7fc700 1 -- 172.21.15.33:0/3386072938 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e64003570 con 0x7f1e7c0af950 2023-10-06T05:15:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7b7fe700 1 -- 172.21.15.33:0/3386072938 >> v2:172.21.15.167:3300/0 conn(0x7f1e7c0c9490 msgr2=0x7f1e7c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7b7fe700 1 --2- 172.21.15.33:0/3386072938 >> v2:172.21.15.167:3300/0 conn(0x7f1e7c0c9490 0x7f1e7c0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7b7fe700 1 -- 172.21.15.33:0/3386072938 >> v1:172.21.15.167:6789/0 conn(0x7f1e7c0af950 legacy=0x7f1e7c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7b7fe700 1 -- 172.21.15.33:0/3386072938 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0c9cc0 con 0x7f1e7c155a50 2023-10-06T05:15:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7affd700 1 --2- 172.21.15.33:0/3386072938 >> v2:172.21.15.167:3300/0 conn(0x7f1e7c0c9490 0x7f1e7c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7b7fe700 1 --2- 172.21.15.33:0/3386072938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c155e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0099e0 tx=0x7f1e6c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61c70c7b1f2c066 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.984+0000 7f1e7a7fc700 1 -- 172.21.15.33:0/3386072938 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00b640 con 0x7f1e7c155a50 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e7a7fc700 1 -- 172.21.15.33:0/3386072938 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00b7a0 con 0x7f1e7c155a50 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e7a7fc700 1 -- 172.21.15.33:0/3386072938 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00bb40 con 0x7f1e7c155a50 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 -- 172.21.15.33:0/3386072938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 msgr2=0x7f1e7c155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/3386072938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c155e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0099e0 tx=0x7f1e6c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 -- 172.21.15.33:0/3386072938 shutdown_connections 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/3386072938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c155e20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/3386072938 >> v2:172.21.15.167:3300/0 conn(0x7f1e7c0c9490 0x7f1e7c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 -- 172.21.15.33:0/3386072938 >> 172.21.15.33:0/3386072938 conn(0x7f1e7c06d030 msgr2=0x7f1e7c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 -- 172.21.15.33:0/3386072938 shutdown_connections 2023-10-06T05:15:02.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.985+0000 7f1e80f9d700 1 -- 172.21.15.33:0/3386072938 wait complete. 2023-10-06T05:15:02.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.987+0000 7f1e80f9d700 1 Processor -- start 2023-10-06T05:15:02.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.987+0000 7f1e80f9d700 1 -- start start 2023-10-06T05:15:02.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e80f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e7c0af950 0x7f1e7c167900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e80f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 0x7f1e7c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e80f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e80f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c167e40 con 0x7f1e7c0c9490 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e80f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c4fd0 con 0x7f1e7c0af950 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e80f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c5b50 con 0x7f1e7c155a50 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 0x7f1e7c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 0x7f1e7c1507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55842/0 (socket says 172.21.15.33:55842) 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 -- 172.21.15.33:0/2156364601 learned_addr learned my addr 172.21.15.33:0/2156364601 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7bfff700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7b7fe700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e7c0af950 0x7f1e7c167900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 -- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 msgr2=0x7f1e7c0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c0c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 -- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e7c0af950 msgr2=0x7f1e7c167900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e7c0af950 0x7f1e7c167900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.988+0000 7f1e7affd700 1 -- 172.21.15.33:0/2156364601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7000a530 con 0x7f1e7c0c9490 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.989+0000 7f1e7affd700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 0x7f1e7c1507a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000fcb0 tx=0x7f1e70009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.989+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70005c10 con 0x7f1e7c0c9490 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.989+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e70005d70 con 0x7f1e7c0c9490 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.989+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000d940 con 0x7f1e7c0c9490 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.989+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c0094f0 con 0x7f1e7c0c9490 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.989+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c0c5500 con 0x7f1e7c0c9490 2023-10-06T05:15:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.990+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c04c380 con 0x7f1e7c0c9490 2023-10-06T05:15:02.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.991+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e7000dd30 con 0x7f1e7c0c9490 2023-10-06T05:15:02.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.992+0000 7f1e637fe700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:02.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.992+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1e70084860 con 0x7f1e7c0c9490 2023-10-06T05:15:02.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.992+0000 7f1e7b7fe700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:02.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.993+0000 7f1e7b7fe700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1e68066f30 0x7f1e680693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0099e0 tx=0x7f1e6c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:02.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:02.996+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e70051970 con 0x7f1e7c0c9490 2023-10-06T05:15:03.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.198+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e7c06bb10 con 0x7f1e7c0c9490 2023-10-06T05:15:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.198+0000 7f1e637fe700 1 -- 172.21.15.33:0/2156364601 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e70054f90 con 0x7f1e7c0c9490 2023-10-06T05:15:03.200 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:15:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1e68066f30 msgr2=0x7f1e680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1e68066f30 0x7f1e680693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0099e0 tx=0x7f1e6c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 msgr2=0x7f1e7c1507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 0x7f1e7c1507a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000fcb0 tx=0x7f1e70009040 comp rx=0 tx=0).stop 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 shutdown_connections 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1e7c155a50 0x7f1e7c0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e7c0af950 0x7f1e7c167900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 --2- 172.21.15.33:0/2156364601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7c0c9490 0x7f1e7c1507a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 >> 172.21.15.33:0/2156364601 conn(0x7f1e7c06d030 msgr2=0x7f1e7c069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.200+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 shutdown_connections 2023-10-06T05:15:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.201+0000 7f1e80f9d700 1 -- 172.21.15.33:0/2156364601 wait complete. 2023-10-06T05:15:03.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.407+0000 7f3255f9d700 1 Processor -- start 2023-10-06T05:15:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.407+0000 7f3255f9d700 1 -- start start 2023-10-06T05:15:03.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f3255f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f32500cd5e0 0x7f32500af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f3255f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f325015a650 0x7f325015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f3255f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325015d660 con 0x7f32500af540 2023-10-06T05:15:03.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f3255f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32500ce890 con 0x7f325015a650 2023-10-06T05:15:03.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f3255f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325015aa60 con 0x7f32500cd5e0 2023-10-06T05:15:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f325579c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f325015a650 0x7f325015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f325579c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f325015a650 0x7f325015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55850/0 (socket says 172.21.15.33:55850) 2023-10-06T05:15:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.408+0000 7f325579c700 1 -- 172.21.15.33:0/79467667 learned_addr learned my addr 172.21.15.33:0/79467667 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f324ffff700 1 --2- 172.21.15.33:0/79467667 >> v2:172.21.15.167:3300/0 conn(0x7f32500cd5e0 0x7f32500af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f325579c700 1 -- 172.21.15.33:0/79467667 >> v2:172.21.15.167:3300/0 conn(0x7f32500cd5e0 msgr2=0x7f32500af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f325579c700 1 --2- 172.21.15.33:0/79467667 >> v2:172.21.15.167:3300/0 conn(0x7f32500cd5e0 0x7f32500af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f325579c700 1 -- 172.21.15.33:0/79467667 >> v1:172.21.15.167:6789/0 conn(0x7f32500af540 legacy=0x7f32500cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f325579c700 1 -- 172.21.15.33:0/79467667 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325015d890 con 0x7f325015a650 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f325579c700 1 --2- 172.21.15.33:0/79467667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f325015a650 0x7f325015d120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32500893f0 tx=0x7f3248009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57ceaa70232a456 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.409+0000 7f324f7fe700 1 -- 172.21.15.33:0/79467667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324800e070 con 0x7f325015a650 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f324f7fe700 1 -- 172.21.15.33:0/79467667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324800ba20 con 0x7f325015a650 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f324f7fe700 1 -- 172.21.15.33:0/79467667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32480092e0 con 0x7f325015a650 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 -- 172.21.15.33:0/79467667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f325015a650 msgr2=0x7f325015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 --2- 172.21.15.33:0/79467667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f325015a650 0x7f325015d120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32500893f0 tx=0x7f3248009840 comp rx=0 tx=0).stop 2023-10-06T05:15:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 -- 172.21.15.33:0/79467667 shutdown_connections 2023-10-06T05:15:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 --2- 172.21.15.33:0/79467667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f325015a650 0x7f325015d120 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 --2- 172.21.15.33:0/79467667 >> v2:172.21.15.167:3300/0 conn(0x7f32500cd5e0 0x7f32500af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 -- 172.21.15.33:0/79467667 >> 172.21.15.33:0/79467667 conn(0x7f325006d9b0 msgr2=0x7f325006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 -- 172.21.15.33:0/79467667 shutdown_connections 2023-10-06T05:15:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.410+0000 7f3255f9d700 1 -- 172.21.15.33:0/79467667 wait complete. 2023-10-06T05:15:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.412+0000 7f3255f9d700 1 Processor -- start 2023-10-06T05:15:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.412+0000 7f3255f9d700 1 -- start start 2023-10-06T05:15:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3255f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 0x7f32500b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3255f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32500cd5e0 0x7f325014c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3255f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f325015a650 0x7f32500ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3255f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325016a790 con 0x7f32500af540 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3255f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325014c580 con 0x7f325015a650 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3255f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32500ca6c0 con 0x7f32500cd5e0 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 0x7f32500b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 0x7f32500b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55856/0 (socket says 172.21.15.33:55856) 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 -- 172.21.15.33:0/3752392721 learned_addr learned my addr 172.21.15.33:0/3752392721 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f324ffff700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32500cd5e0 0x7f325014c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f325579c700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f325015a650 0x7f32500ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 -- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32500cd5e0 msgr2=0x7f325014c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32500cd5e0 0x7f325014c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 -- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f325015a650 msgr2=0x7f32500ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f325015a650 0x7f32500ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f3254f9b700 1 -- 172.21.15.33:0/3752392721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248009ae0 con 0x7f32500af540 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.413+0000 7f325579c700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f325015a650 0x7f32500ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.414+0000 7f3254f9b700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 0x7f32500b4650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f324400c4e0 tx=0x7f324400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.414+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244011590 con 0x7f32500af540 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.414+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244011710 con 0x7f32500af540 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.414+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3250169b80 con 0x7f32500af540 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.414+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244011ad0 con 0x7f32500af540 2023-10-06T05:15:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.414+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325016a140 con 0x7f32500af540 2023-10-06T05:15:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.415+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325004c380 con 0x7f32500af540 2023-10-06T05:15:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.416+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f324401b3b0 con 0x7f32500af540 2023-10-06T05:15:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.417+0000 7f324cff9700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.417+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3244083b20 con 0x7f32500af540 2023-10-06T05:15:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.417+0000 7f324ffff700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.418+0000 7f324ffff700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3250159610 tx=0x7f323c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.421+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3244050c30 con 0x7f32500af540 2023-10-06T05:15:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.622+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f325015db70 con 0x7f32500af540 2023-10-06T05:15:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.623+0000 7f324cff9700 1 -- 172.21.15.33:0/3752392721 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3244017020 con 0x7f32500af540 2023-10-06T05:15:03.625 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 msgr2=0x7f32340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3250159610 tx=0x7f323c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 msgr2=0x7f32500b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 0x7f32500b4650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f324400c4e0 tx=0x7f324400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 shutdown_connections 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f325015a650 0x7f32500ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32500af540 0x7f32500b4650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 --2- 172.21.15.33:0/3752392721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32500cd5e0 0x7f325014c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 >> 172.21.15.33:0/3752392721 conn(0x7f325006d9b0 msgr2=0x7f32500c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 shutdown_connections 2023-10-06T05:15:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.625+0000 7f3255f9d700 1 -- 172.21.15.33:0/3752392721 wait complete. 2023-10-06T05:15:03.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.832+0000 7fbcf7fff700 1 Processor -- start 2023-10-06T05:15:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.832+0000 7fbcf7fff700 1 -- start start 2023-10-06T05:15:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.833+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf8159850 0x7fbcf806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcf806d140 0x7fbcf806edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf80b0cd0 con 0x7fbcf815acd0 2023-10-06T05:15:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf815a840 con 0x7fbcf806d140 2023-10-06T05:15:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8159f10 con 0x7fbcf8159850 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcf806d140 0x7fbcf806edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcf806d140 0x7fbcf806edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55866/0 (socket says 172.21.15.33:55866) 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 -- 172.21.15.33:0/2162174072 learned_addr learned my addr 172.21.15.33:0/2162174072 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf6ffd700 1 --2- 172.21.15.33:0/2162174072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf8159850 0x7fbcf806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 -- 172.21.15.33:0/2162174072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf8159850 msgr2=0x7fbcf806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 --2- 172.21.15.33:0/2162174072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf8159850 0x7fbcf806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 -- 172.21.15.33:0/2162174072 >> v1:172.21.15.167:6789/0 conn(0x7fbcf815acd0 legacy=0x7fbcf806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.834+0000 7fbcf77fe700 1 -- 172.21.15.33:0/2162174072 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf815a1f0 con 0x7fbcf806d140 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf77fe700 1 --2- 172.21.15.33:0/2162174072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf806d140 0x7fbcf806edd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009f20 tx=0x7fbcf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d101c8e6d455aea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf5ffb700 1 -- 172.21.15.33:0/2162174072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf000e070 con 0x7fbcf806d140 2023-10-06T05:15:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf5ffb700 1 -- 172.21.15.33:0/2162174072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf000ba80 con 0x7fbcf806d140 2023-10-06T05:15:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf5ffb700 1 -- 172.21.15.33:0/2162174072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00092e0 con 0x7fbcf806d140 2023-10-06T05:15:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf7fff700 1 -- 172.21.15.33:0/2162174072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf806d140 msgr2=0x7fbcf806edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/2162174072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf806d140 0x7fbcf806edd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009f20 tx=0x7fbcf0009840 comp rx=0 tx=0).stop 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf7fff700 1 -- 172.21.15.33:0/2162174072 shutdown_connections 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/2162174072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf806d140 0x7fbcf806edd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/2162174072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf8159850 0x7fbcf806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.835+0000 7fbcf7fff700 1 -- 172.21.15.33:0/2162174072 >> 172.21.15.33:0/2162174072 conn(0x7fbcf806cd40 msgr2=0x7fbcf80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.836+0000 7fbcf7fff700 1 -- 172.21.15.33:0/2162174072 shutdown_connections 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.836+0000 7fbcf7fff700 1 -- 172.21.15.33:0/2162174072 wait complete. 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.837+0000 7fbcf7fff700 1 Processor -- start 2023-10-06T05:15:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.837+0000 7fbcf7fff700 1 -- start start 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf806d140 0x7fbcf8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf8159850 0x7fbcf80cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 0x7fbcf80b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf80cad10 con 0x7fbcf815acd0 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf80caff0 con 0x7fbcf8159850 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf80cb2d0 con 0x7fbcf806d140 2023-10-06T05:15:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 0x7fbcf80b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 0x7fbcf80b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55870/0 (socket says 172.21.15.33:55870) 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.838+0000 7fbcf77fe700 1 -- 172.21.15.33:0/104133649 learned_addr learned my addr 172.21.15.33:0/104133649 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf77fe700 1 -- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf806d140 msgr2=0x7fbcf8157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf77fe700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf806d140 0x7fbcf8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf77fe700 1 -- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf8159850 msgr2=0x7fbcf80cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf77fe700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf8159850 0x7fbcf80cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf77fe700 1 -- 172.21.15.33:0/104133649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf0009ae0 con 0x7fbcf815acd0 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf77fe700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 0x7fbcf80b2c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8065080 tx=0x7fbcf0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00032f0 con 0x7fbcf815acd0 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf0008e80 con 0x7fbcf815acd0 2023-10-06T05:15:03.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0026690 con 0x7fbcf815acd0 2023-10-06T05:15:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf80cb5b0 con 0x7fbcf815acd0 2023-10-06T05:15:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.839+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf80cbb70 con 0x7fbcf815acd0 2023-10-06T05:15:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.841+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf804c380 con 0x7fbcf815acd0 2023-10-06T05:15:03.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.841+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcf0003040 con 0x7fbcf815acd0 2023-10-06T05:15:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.842+0000 7fbcdeffd700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbce4066f30 0x7fbce40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.842+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbcf0086750 con 0x7fbcf815acd0 2023-10-06T05:15:03.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.843+0000 7fbcf6ffd700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbce4066f30 0x7fbce40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.844+0000 7fbcf6ffd700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbce4066f30 0x7fbce40693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009a60 tx=0x7fbce8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:03.847+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcf00537c0 con 0x7fbcf815acd0 2023-10-06T05:15:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.050+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcf81591e0 con 0x7fbcf815acd0 2023-10-06T05:15:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.051+0000 7fbcdeffd700 1 -- 172.21.15.33:0/104133649 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbcf0056de0 con 0x7fbcf815acd0 2023-10-06T05:15:04.053 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:15:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbce4066f30 msgr2=0x7fbce40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbce4066f30 0x7fbce40693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009a60 tx=0x7fbce8008040 comp rx=0 tx=0).stop 2023-10-06T05:15:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 msgr2=0x7fbcf80b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 0x7fbcf80b2c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8065080 tx=0x7fbcf0018000 comp rx=0 tx=0).stop 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 shutdown_connections 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbce4066f30 0x7fbce40693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf815acd0 0x7fbcf80b2c10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf806d140 0x7fbcf8157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 --2- 172.21.15.33:0/104133649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf8159850 0x7fbcf80cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 >> 172.21.15.33:0/104133649 conn(0x7fbcf806cd40 msgr2=0x7fbcf80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 shutdown_connections 2023-10-06T05:15:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.053+0000 7fbcf7fff700 1 -- 172.21.15.33:0/104133649 wait complete. 2023-10-06T05:15:04.108 INFO:tasks.cephfs_test_runner:Starting test: test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) Simulate software freeze or temporary network failure. 2023-10-06T05:15:04.108 INFO:tasks.cephfs_test_runner:test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) 2023-10-06T05:15:04.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death' 2023-10-06T05:15:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.294+0000 7f8b3359e700 1 Processor -- start 2023-10-06T05:15:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.294+0000 7f8b3359e700 1 -- start start 2023-10-06T05:15:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.295+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3413faa0 0x7f8b3413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.295+0000 7f8b3359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b340c7940 0x7f8b340c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.295+0000 7f8b3359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340c9cb0 con 0x7f8b3406aad0 2023-10-06T05:15:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.295+0000 7f8b3359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3406b660 con 0x7f8b340c7940 2023-10-06T05:15:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.295+0000 7f8b3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340ca270 con 0x7f8b3413faa0 2023-10-06T05:15:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.295+0000 7f8b31d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b340c7940 0x7f8b340c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b32d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b3406aad0 0x7f8b3406af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54802/0 (socket says 172.21.15.33:54802) 2023-10-06T05:15:04.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b32d9d700 1 -- 172.21.15.33:0/2601201840 learned_addr learned my addr 172.21.15.33:0/2601201840 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:04.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b31d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b340c7940 0x7f8b340c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55882/0 (socket says 172.21.15.33:55882) 2023-10-06T05:15:04.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b3259c700 1 --2- 172.21.15.33:0/2601201840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3413faa0 0x7f8b3413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2601201840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3413faa0 msgr2=0x7f8b3413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:04.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/2601201840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3413faa0 0x7f8b3413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2601201840 >> v1:172.21.15.33:6789/0 conn(0x7f8b3406aad0 legacy=0x7f8b3406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2601201840 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3406b840 con 0x7f8b340c7940 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b3159a700 1 -- 172.21.15.33:0/2601201840 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575295474 0 0) 0x7f8b340c9cb0 con 0x7f8b3406aad0 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/2601201840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340c7db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098e0 tx=0x7f8b280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8115f3378d5652 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.296+0000 7f8b3159a700 1 -- 172.21.15.33:0/2601201840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800b640 con 0x7f8b340c7940 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3159a700 1 -- 172.21.15.33:0/2601201840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800b7a0 con 0x7f8b340c7940 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3159a700 1 -- 172.21.15.33:0/2601201840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bb40 con 0x7f8b340c7940 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 -- 172.21.15.33:0/2601201840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 msgr2=0x7f8b340c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2601201840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340c7db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098e0 tx=0x7f8b280091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 -- 172.21.15.33:0/2601201840 shutdown_connections 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2601201840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3413faa0 0x7f8b3413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2601201840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340c7db0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 -- 172.21.15.33:0/2601201840 >> 172.21.15.33:0/2601201840 conn(0x7f8b3406d030 msgr2=0x7f8b3406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 -- 172.21.15.33:0/2601201840 shutdown_connections 2023-10-06T05:15:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.297+0000 7f8b3359e700 1 -- 172.21.15.33:0/2601201840 wait complete. 2023-10-06T05:15:04.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.299+0000 7f8b3359e700 1 Processor -- start 2023-10-06T05:15:04.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.299+0000 7f8b3359e700 1 -- start start 2023-10-06T05:15:04.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3406aad0 0x7f8b3415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b3413faa0 0x7f8b34156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b341594f0 con 0x7f8b340c7940 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b341591c0 con 0x7f8b3413faa0 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34157700 con 0x7f8b3406aad0 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b31d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b31d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55888/0 (socket says 172.21.15.33:55888) 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2305180757 learned_addr learned my addr 172.21.15.33:0/2305180757 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b3259c700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3406aad0 0x7f8b3415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b32d9d700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b3413faa0 0x7f8b34156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3406aad0 msgr2=0x7f8b3415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.300+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3406aad0 0x7f8b3415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b3413faa0 msgr2=0x7f8b34156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b3413faa0 0x7f8b34156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b31d9b700 1 -- 172.21.15.33:0/2305180757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28009420 con 0x7f8b340c7940 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b32d9d700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b3413faa0 0x7f8b34156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340a54f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8b340719c0 tx=0x7f8b28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bb20 con 0x7f8b340c7940 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b28011420 con 0x7f8b340c7940 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280117c0 con 0x7f8b340c7940 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b341579e0 con 0x7f8b340c7940 2023-10-06T05:15:04.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.301+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b34157cc0 con 0x7f8b340c7940 2023-10-06T05:15:04.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.302+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3404bb40 con 0x7f8b340c7940 2023-10-06T05:15:04.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.303+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b28011b70 con 0x7f8b340c7940 2023-10-06T05:15:04.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.304+0000 7f8b227fc700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.304+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b28085d60 con 0x7f8b340c7940 2023-10-06T05:15:04.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.304+0000 7f8b3259c700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.305+0000 7f8b3259c700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8b341607a0 tx=0x7f8b24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:04.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.308+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b28052dd0 con 0x7f8b340c7940 2023-10-06T05:15:04.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.590+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]} v 0) v1 -- 0x7f8b3404c380 con 0x7f8b340c7940 2023-10-06T05:15:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.939+0000 7f8b227fc700 1 -- 172.21.15.33:0/2305180757 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]}]=0 v64) v1 ==== 160+0+0 (secure 0 0 0) 0x7f8b280563f0 con 0x7f8b340c7940 2023-10-06T05:15:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 msgr2=0x7f8b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8b341607a0 tx=0x7f8b24008040 comp rx=0 tx=0).stop 2023-10-06T05:15:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 msgr2=0x7f8b340a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340a54f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8b340719c0 tx=0x7f8b28017000 comp rx=0 tx=0).stop 2023-10-06T05:15:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 shutdown_connections 2023-10-06T05:15:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b3413faa0 0x7f8b34156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b3406aad0 0x7f8b3415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 --2- 172.21.15.33:0/2305180757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340c7940 0x7f8b340a54f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 >> 172.21.15.33:0/2305180757 conn(0x7f8b3406d030 msgr2=0x7f8b3406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 shutdown_connections 2023-10-06T05:15:04.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:04.942+0000 7f8b3359e700 1 -- 172.21.15.33:0/2305180757 wait complete. 2023-10-06T05:15:04.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:15:05.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.151+0000 7fc6d4fff700 1 Processor -- start 2023-10-06T05:15:05.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.152+0000 7fc6d4fff700 1 -- start start 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6d4fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc6d0157740 0x7fc6d00ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6d4fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc6d00ae740 0x7fc6d0069f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6d4fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d00c75e0 con 0x7fc6d006a460 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6d4fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d00aebe0 con 0x7fc6d0157740 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6d4fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d00c8a80 con 0x7fc6d00ae740 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6ce59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc6d0157740 0x7fc6d00ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6ce59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc6d0157740 0x7fc6d00ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55904/0 (socket says 172.21.15.33:55904) 2023-10-06T05:15:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.153+0000 7fc6ce59c700 1 -- 172.21.15.33:0/3017999488 learned_addr learned my addr 172.21.15.33:0/3017999488 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6ce59c700 1 -- 172.21.15.33:0/3017999488 >> v2:172.21.15.167:3300/0 conn(0x7fc6d00ae740 msgr2=0x7fc6d0069f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6ce59c700 1 --2- 172.21.15.33:0/3017999488 >> v2:172.21.15.167:3300/0 conn(0x7fc6d00ae740 0x7fc6d0069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6ce59c700 1 -- 172.21.15.33:0/3017999488 >> v1:172.21.15.167:6789/0 conn(0x7fc6d006a460 legacy=0x7fc6d015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6ce59c700 1 -- 172.21.15.33:0/3017999488 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d00c69d0 con 0x7fc6d0157740 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6ce59c700 1 --2- 172.21.15.33:0/3017999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d0157740 0x7fc6d00ae200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc6d006fdb0 tx=0x7fc6c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c689cda3ebcc24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6cd59a700 1 -- 172.21.15.33:0/3017999488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000b820 con 0x7fc6d0157740 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6cd59a700 1 -- 172.21.15.33:0/3017999488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c000b980 con 0x7fc6d0157740 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.154+0000 7fc6cd59a700 1 -- 172.21.15.33:0/3017999488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000bd20 con 0x7fc6d0157740 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 -- 172.21.15.33:0/3017999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d0157740 msgr2=0x7fc6d00ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/3017999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d0157740 0x7fc6d00ae200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc6d006fdb0 tx=0x7fc6c0009250 comp rx=0 tx=0).stop 2023-10-06T05:15:05.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 -- 172.21.15.33:0/3017999488 shutdown_connections 2023-10-06T05:15:05.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/3017999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d0157740 0x7fc6d00ae200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/3017999488 >> v2:172.21.15.167:3300/0 conn(0x7fc6d00ae740 0x7fc6d0069f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 -- 172.21.15.33:0/3017999488 >> 172.21.15.33:0/3017999488 conn(0x7fc6d006d030 msgr2=0x7fc6d00cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 -- 172.21.15.33:0/3017999488 shutdown_connections 2023-10-06T05:15:05.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.155+0000 7fc6d4fff700 1 -- 172.21.15.33:0/3017999488 wait complete. 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.156+0000 7fc6d4fff700 1 Processor -- start 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.156+0000 7fc6d4fff700 1 -- start start 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.157+0000 7fc6d4fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 0x7fc6d0151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.157+0000 7fc6d4fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6d00ae740 0x7fc6d0169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.157+0000 7fc6d4fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d0157740 0x7fc6d00b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.157+0000 7fc6d4fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d0157c20 con 0x7fc6d006a460 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.157+0000 7fc6d4fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d00b1050 con 0x7fc6d0157740 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.157+0000 7fc6d4fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d016a340 con 0x7fc6d00ae740 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 0x7fc6d0151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 0x7fc6d0151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55906/0 (socket says 172.21.15.33:55906) 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 -- 172.21.15.33:0/1057128959 learned_addr learned my addr 172.21.15.33:0/1057128959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 -- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6d00ae740 msgr2=0x7fc6d0169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ced9d700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d0157740 0x7fc6d00b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6cdd9b700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6d00ae740 0x7fc6d0169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6d00ae740 0x7fc6d0169d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 -- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d0157740 msgr2=0x7fc6d00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d0157740 0x7fc6d00b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 -- 172.21.15.33:0/1057128959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00094f0 con 0x7fc6d006a460 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ced9d700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d0157740 0x7fc6d00b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6cdd9b700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6d00ae740 0x7fc6d0169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.158+0000 7fc6ce59c700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 0x7fc6d0151ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009290 tx=0x7fc6c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.159+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0026480 con 0x7fc6d006a460 2023-10-06T05:15:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.159+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c001f070 con 0x7fc6d006a460 2023-10-06T05:15:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.159+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d016a620 con 0x7fc6d006a460 2023-10-06T05:15:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.159+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0026850 con 0x7fc6d006a460 2023-10-06T05:15:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.159+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d016d3b0 con 0x7fc6d006a460 2023-10-06T05:15:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.160+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d016d5a0 con 0x7fc6d006a460 2023-10-06T05:15:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.161+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6c001a440 con 0x7fc6d006a460 2023-10-06T05:15:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.162+0000 7fc6be7fc700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6b0066f30 0x7fc6b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.162+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc6c0021070 con 0x7fc6d006a460 2023-10-06T05:15:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.162+0000 7fc6cdd9b700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6b0066f30 0x7fc6b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.163+0000 7fc6cdd9b700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6b0066f30 0x7fc6b00693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.166+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6c0053e70 con 0x7fc6d006a460 2023-10-06T05:15:05.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.368+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc6d004c380 con 0x7fc6d006a460 2023-10-06T05:15:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.372+0000 7fc6be7fc700 1 -- 172.21.15.33:0/1057128959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc6c0057490 con 0x7fc6d006a460 2023-10-06T05:15:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.374+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6b0066f30 msgr2=0x7fc6b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6b0066f30 0x7fc6b00693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4008040 comp rx=0 tx=0).stop 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 msgr2=0x7fc6d0151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 0x7fc6d0151ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009290 tx=0x7fc6c0016000 comp rx=0 tx=0).stop 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 shutdown_connections 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6b0066f30 0x7fc6b00693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d0157740 0x7fc6d00b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6d006a460 0x7fc6d0151ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 --2- 172.21.15.33:0/1057128959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6d00ae740 0x7fc6d0169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 >> 172.21.15.33:0/1057128959 conn(0x7fc6d006d030 msgr2=0x7fc6d00cabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 shutdown_connections 2023-10-06T05:15:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.375+0000 7fc6d4fff700 1 -- 172.21.15.33:0/1057128959 wait complete. 2023-10-06T05:15:05.394 INFO:tasks.cephfs_test_runner:Simulate software freeze or temporary network failure. ... skipped 'Require first client to be on separate server from MDSs' 2023-10-06T05:15:05.394 INFO:tasks.cephfs_test_runner:Starting test: test_abort_conn (tasks.cephfs.test_client_recovery.TestClientRecovery) Check that abort_conn() skips closing mds sessions. 2023-10-06T05:15:05.394 INFO:tasks.cephfs_test_runner:test_abort_conn (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:15:05.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn' 2023-10-06T05:15:05.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.580+0000 7f5da49fe700 1 Processor -- start 2023-10-06T05:15:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.580+0000 7f5da49fe700 1 -- start start 2023-10-06T05:15:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.581+0000 7f5da49fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5da0155af0 0x7f5da006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.581+0000 7f5da49fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5da00c7920 0x7f5da00c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.581+0000 7f5da49fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da006ab60 con 0x7f5da006a6a0 2023-10-06T05:15:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.581+0000 7f5da49fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da006b7e0 con 0x7f5da0155af0 2023-10-06T05:15:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.581+0000 7f5da49fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da006ae40 con 0x7f5da00c7920 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5da0155af0 0x7f5da006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5da0155af0 0x7f5da006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55920/0 (socket says 172.21.15.33:55920) 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/965350132 learned_addr learned my addr 172.21.15.33:0/965350132 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9e59c700 1 --1- 172.21.15.33:0/965350132 >> v1:172.21.15.33:6789/0 conn(0x7f5da006a6a0 0x7f5da00c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54806/0 (socket says 172.21.15.33:54806) 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9f59e700 1 --2- 172.21.15.33:0/965350132 >> v2:172.21.15.167:3300/0 conn(0x7f5da00c7920 0x7f5da00c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/965350132 >> v2:172.21.15.167:3300/0 conn(0x7f5da00c7920 msgr2=0x7f5da00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 --2- 172.21.15.33:0/965350132 >> v2:172.21.15.167:3300/0 conn(0x7f5da00c7920 0x7f5da00c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/965350132 >> v1:172.21.15.33:6789/0 conn(0x7f5da006a6a0 legacy=0x7f5da00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/965350132 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da006b120 con 0x7f5da0155af0 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9f59e700 1 --2- 172.21.15.33:0/965350132 >> v2:172.21.15.167:3300/0 conn(0x7f5da00c7920 0x7f5da00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9dd9b700 1 -- 172.21.15.33:0/965350132 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149132345 0 0) 0x7f5da006ab60 con 0x7f5da006a6a0 2023-10-06T05:15:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.582+0000 7f5d9ed9d700 1 --2- 172.21.15.33:0/965350132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0155af0 0x7f5da006a160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5da0070fb0 tx=0x7f5d9000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93c5013886c7f0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5d9dd9b700 1 -- 172.21.15.33:0/965350132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000db10 con 0x7f5da0155af0 2023-10-06T05:15:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5d9dd9b700 1 -- 172.21.15.33:0/965350132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9000dc70 con 0x7f5da0155af0 2023-10-06T05:15:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5d9dd9b700 1 -- 172.21.15.33:0/965350132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90017410 con 0x7f5da0155af0 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 -- 172.21.15.33:0/965350132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0155af0 msgr2=0x7f5da006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 --2- 172.21.15.33:0/965350132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0155af0 0x7f5da006a160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5da0070fb0 tx=0x7f5d9000b4e0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 -- 172.21.15.33:0/965350132 shutdown_connections 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 --2- 172.21.15.33:0/965350132 >> v2:172.21.15.167:3300/0 conn(0x7f5da00c7920 0x7f5da00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 --2- 172.21.15.33:0/965350132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da0155af0 0x7f5da006a160 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 -- 172.21.15.33:0/965350132 >> 172.21.15.33:0/965350132 conn(0x7f5da006d030 msgr2=0x7f5da006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 -- 172.21.15.33:0/965350132 shutdown_connections 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.583+0000 7f5da49fe700 1 -- 172.21.15.33:0/965350132 wait complete. 2023-10-06T05:15:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.585+0000 7f5da49fe700 1 Processor -- start 2023-10-06T05:15:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.585+0000 7f5da49fe700 1 -- start start 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5da49fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 0x7f5da00b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5da49fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5da00c7920 0x7f5da0148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5da49fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da0155af0 0x7f5da01491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5da49fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c3190 con 0x7f5da006a6a0 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5da49fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c3d10 con 0x7f5da0155af0 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5da49fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c33e0 con 0x7f5da00c7920 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5d9ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 0x7f5da00b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5d9ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 0x7f5da00b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55924/0 (socket says 172.21.15.33:55924) 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/1547111550 learned_addr learned my addr 172.21.15.33:0/1547111550 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5d9e59c700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5da00c7920 0x7f5da0148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.586+0000 7f5d9f59e700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da0155af0 0x7f5da01491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5da00c7920 msgr2=0x7f5da0148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d9ed9d700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5da00c7920 0x7f5da0148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da0155af0 msgr2=0x7f5da01491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d9ed9d700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da0155af0 0x7f5da01491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d9ed9d700 1 -- 172.21.15.33:0/1547111550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9000b780 con 0x7f5da006a6a0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d9ed9d700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 0x7f5da00b0720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000bfd0 tx=0x7f5d90019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9001f410 con 0x7f5da006a6a0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9001f590 con 0x7f5da006a6a0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da00c36c0 con 0x7f5da006a6a0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9001f950 con 0x7f5da006a6a0 2023-10-06T05:15:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.587+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da00b0c60 con 0x7f5da006a6a0 2023-10-06T05:15:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.588+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da004c380 con 0x7f5da006a6a0 2023-10-06T05:15:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.589+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d90015070 con 0x7f5da006a6a0 2023-10-06T05:15:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.590+0000 7f5d86ffd700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d8c066f30 0x7f5d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.590+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d90085de0 con 0x7f5da006a6a0 2023-10-06T05:15:05.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.590+0000 7f5d9e59c700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d8c066f30 0x7f5d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.591+0000 7f5d9e59c700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d8c066f30 0x7f5d8c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5da00cc360 tx=0x7f5d94007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.594+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d90052ed0 con 0x7f5da006a6a0 2023-10-06T05:15:05.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.884+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]} v 0) v1 -- 0x7f5da006b400 con 0x7f5da006a6a0 2023-10-06T05:15:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.942+0000 7f5d86ffd700 1 -- 172.21.15.33:0/1547111550 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]}]=0 v65) v1 ==== 150+0+0 (secure 0 0 0) 0x7f5d900564f0 con 0x7f5da006a6a0 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d8c066f30 msgr2=0x7f5d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d8c066f30 0x7f5d8c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5da00cc360 tx=0x7f5d94007ef0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 msgr2=0x7f5da00b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 0x7f5da00b0720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000bfd0 tx=0x7f5d90019000 comp rx=0 tx=0).stop 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 shutdown_connections 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d8c066f30 0x7f5d8c0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da0155af0 0x7f5da01491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da006a6a0 0x7f5da00b0720 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 --2- 172.21.15.33:0/1547111550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5da00c7920 0x7f5da0148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 >> 172.21.15.33:0/1547111550 conn(0x7f5da006d030 msgr2=0x7f5da00c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 shutdown_connections 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:05.944+0000 7f5da49fe700 1 -- 172.21.15.33:0/1547111550 wait complete. 2023-10-06T05:15:05.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:15:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.152+0000 7f27cdb9d700 1 Processor -- start 2023-10-06T05:15:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.152+0000 7f27cdb9d700 1 -- start start 2023-10-06T05:15:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27cdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27cdb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80b0230 con 0x7f27c8155a50 2023-10-06T05:15:06.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27cdb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c8156280 con 0x7f27c80c9490 2023-10-06T05:15:06.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27cdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80b06b0 con 0x7f27c8155e20 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27c7fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f27c80c9490 0x7f27c80afb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:46794/0 (socket says 172.21.15.33:46794) 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27c7fff700 1 -- 172.21.15.33:0/2292150535 learned_addr learned my addr 172.21.15.33:0/2292150535 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.153+0000 7f27cd39c700 1 --2- 172.21.15.33:0/2292150535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479546332 0 0) 0x7f27c8156280 con 0x7f27c80c9490 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8004680 con 0x7f27c80c9490 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985816394 0 0) 0x7f27c80b0230 con 0x7f27c8155a50 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c8156280 con 0x7f27c8155a50 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27cd39c700 1 -- 172.21.15.33:0/2292150535 >> v1:172.21.15.167:6789/0 conn(0x7f27c80c9490 legacy=0x7f27c80afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27cd39c700 1 -- 172.21.15.33:0/2292150535 >> v1:172.21.15.33:6789/0 conn(0x7f27c8155a50 legacy=0x7f27c80af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27cd39c700 1 -- 172.21.15.33:0/2292150535 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80cba90 con 0x7f27c8155e20 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.154+0000 7f27cd39c700 1 --2- 172.21.15.33:0/2292150535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8160ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009e90 tx=0x7f27c0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb43c3e70f12b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000bb80 con 0x7f27c8155e20 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c000bce0 con 0x7f27c8155e20 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27c77fe700 1 -- 172.21.15.33:0/2292150535 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00116b0 con 0x7f27c8155e20 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 -- 172.21.15.33:0/2292150535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 msgr2=0x7f27c8160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/2292150535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8160ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009e90 tx=0x7f27c0009780 comp rx=0 tx=0).stop 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 -- 172.21.15.33:0/2292150535 shutdown_connections 2023-10-06T05:15:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/2292150535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8160ae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 -- 172.21.15.33:0/2292150535 >> 172.21.15.33:0/2292150535 conn(0x7f27c806d030 msgr2=0x7f27c80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 -- 172.21.15.33:0/2292150535 shutdown_connections 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.155+0000 7f27cdb9d700 1 -- 172.21.15.33:0/2292150535 wait complete. 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.157+0000 7f27cdb9d700 1 Processor -- start 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.157+0000 7f27cdb9d700 1 -- start start 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cdb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 0x7f27c814e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cdb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8155a50 0x7f27c814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cdb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8150fd0 con 0x7f27c80c9490 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cdb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c814e830 con 0x7f27c8155a50 2023-10-06T05:15:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81503c0 con 0x7f27c8155e20 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27ccb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 0x7f27c814e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27ccb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 0x7f27c814e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55932/0 (socket says 172.21.15.33:55932) 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27ccb9b700 1 -- 172.21.15.33:0/395379570 learned_addr learned my addr 172.21.15.33:0/395379570 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27cd39c700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.158+0000 7f27c7fff700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8155a50 0x7f27c814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27ccb9b700 1 -- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 msgr2=0x7f27c8151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27ccb9b700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27ccb9b700 1 -- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8155a50 msgr2=0x7f27c814fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27ccb9b700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8155a50 0x7f27c814fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27ccb9b700 1 -- 172.21.15.33:0/395379570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0009a20 con 0x7f27c80c9490 2023-10-06T05:15:06.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27ccb9b700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 0x7f27c814e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c790 tx=0x7f27bc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc010510 con 0x7f27c80c9490 2023-10-06T05:15:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc010670 con 0x7f27c80c9490 2023-10-06T05:15:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c81506a0 con 0x7f27c80c9490 2023-10-06T05:15:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.159+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc010a10 con 0x7f27c80c9490 2023-10-06T05:15:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.160+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c8150c60 con 0x7f27c80c9490 2023-10-06T05:15:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.161+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c81512b0 con 0x7f27c80c9490 2023-10-06T05:15:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.161+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27bc01b3b0 con 0x7f27c80c9490 2023-10-06T05:15:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.163+0000 7f27c4ff9700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27ac066f30 0x7f27ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.163+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f27bc083f90 con 0x7f27c80c9490 2023-10-06T05:15:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.163+0000 7f27c7fff700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27ac066f30 0x7f27ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.164+0000 7f27c7fff700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27ac066f30 0x7f27ac0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27c814ef50 tx=0x7f27b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.167+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27bc051000 con 0x7f27c80c9490 2023-10-06T05:15:06.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.369+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f27c804c380 con 0x7f27c80c9490 2023-10-06T05:15:06.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.370+0000 7f27c4ff9700 1 -- 172.21.15.33:0/395379570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27bc017020 con 0x7f27c80c9490 2023-10-06T05:15:06.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27ac066f30 msgr2=0x7f27ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27ac066f30 0x7f27ac0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27c814ef50 tx=0x7f27b4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:06.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 msgr2=0x7f27c814e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 0x7f27c814e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c790 tx=0x7f27bc007e10 comp rx=0 tx=0).stop 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 shutdown_connections 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27ac066f30 0x7f27ac0693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27c8155e20 0x7f27c8151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27c80c9490 0x7f27c814e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 --2- 172.21.15.33:0/395379570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8155a50 0x7f27c814fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 >> 172.21.15.33:0/395379570 conn(0x7f27c806d030 msgr2=0x7f27c8069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 shutdown_connections 2023-10-06T05:15:06.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:06.372+0000 7f27cdb9d700 1 -- 172.21.15.33:0/395379570 wait complete. 2023-10-06T05:15:06.394 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:15:06.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:06.477 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:06.559 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:15:06.559 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:15:06.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:06.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:15:06.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:15:06.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:06.658 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:15:06.705 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:15:06.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:06.705 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T05:15:06.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:15:06.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:06.786 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:15:06.842 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T05:15:06.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:06.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T05:15:06.922 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:15:06.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:06.923 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:15:06.960 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T05:15:06.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:06.960 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T05:15:07.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:15:07.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:07.041 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:15:07.078 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:15:07.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.264+0000 7f1ca899d700 1 Processor -- start 2023-10-06T05:15:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.264+0000 7f1ca899d700 1 -- start start 2023-10-06T05:15:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.265+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca40ccfc0 0x7f1ca40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.265+0000 7f1ca899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ca40cf3b0 0x7f1ca41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.265+0000 7f1ca899d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca40cbd80 con 0x7f1ca40cc750 2023-10-06T05:15:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.265+0000 7f1ca899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40af5f0 con 0x7f1ca40cf3b0 2023-10-06T05:15:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.265+0000 7f1ca899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40af8d0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca40ccfc0 0x7f1ca40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ca40cf3b0 0x7f1ca41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ca40cf3b0 0x7f1ca41594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:42412/0 (socket says 172.21.15.33:42412) 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca3fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1ca40cc750 0x7f1ca40b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:46810/0 (socket says 172.21.15.33:46810) 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/3805849872 learned_addr learned my addr 172.21.15.33:0/3805849872 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca40ccfc0 0x7f1ca40cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:42518/0 (socket says 172.21.15.33:42518) 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca40ccfc0 msgr2=0x7f1ca40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 --2- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca40ccfc0 0x7f1ca40cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/3805849872 >> v1:172.21.15.167:6789/0 conn(0x7f1ca40cc750 legacy=0x7f1ca40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.266+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/3805849872 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40afbb0 con 0x7f1ca40cf3b0 2023-10-06T05:15:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.267+0000 7f1ca2ffd700 1 --2- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cf3b0 0x7f1ca41594a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c980099b0 tx=0x7f1c98009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2da4ebe096be5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.267+0000 7f1ca27fc700 1 -- 172.21.15.33:0/3805849872 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98009ea0 con 0x7f1ca40cf3b0 2023-10-06T05:15:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.267+0000 7f1ca27fc700 1 -- 172.21.15.33:0/3805849872 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9800b860 con 0x7f1ca40cf3b0 2023-10-06T05:15:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.267+0000 7f1ca27fc700 1 -- 172.21.15.33:0/3805849872 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800bc00 con 0x7f1ca40cf3b0 2023-10-06T05:15:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.267+0000 7f1ca899d700 1 -- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cf3b0 msgr2=0x7f1ca41594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.267+0000 7f1ca899d700 1 --2- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cf3b0 0x7f1ca41594a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c980099b0 tx=0x7f1c98009250 comp rx=0 tx=0).stop 2023-10-06T05:15:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.268+0000 7f1ca899d700 1 -- 172.21.15.33:0/3805849872 shutdown_connections 2023-10-06T05:15:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.268+0000 7f1ca899d700 1 --2- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca40ccfc0 0x7f1ca40cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.268+0000 7f1ca899d700 1 --2- 172.21.15.33:0/3805849872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cf3b0 0x7f1ca41594a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.268+0000 7f1ca899d700 1 -- 172.21.15.33:0/3805849872 >> 172.21.15.33:0/3805849872 conn(0x7f1ca406cff0 msgr2=0x7f1ca406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.268+0000 7f1ca899d700 1 -- 172.21.15.33:0/3805849872 shutdown_connections 2023-10-06T05:15:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.268+0000 7f1ca899d700 1 -- 172.21.15.33:0/3805849872 wait complete. 2023-10-06T05:15:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.269+0000 7f1ca899d700 1 Processor -- start 2023-10-06T05:15:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.269+0000 7f1ca899d700 1 -- start start 2023-10-06T05:15:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cc750 0x7f1ca40b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 0x7f1ca40b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca4154130 0x7f1ca40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4169e00 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b6010 con 0x7f1ca40cc750 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b43e0 con 0x7f1ca4154130 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.270+0000 7f1ca2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 0x7f1ca40b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 0x7f1ca40b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55942/0 (socket says 172.21.15.33:55942) 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/386179606 learned_addr learned my addr 172.21.15.33:0/386179606 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca37fe700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cc750 0x7f1ca40b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca3fff700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca4154130 0x7f1ca40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca4154130 msgr2=0x7f1ca40c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca4154130 0x7f1ca40c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cc750 msgr2=0x7f1ca40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cc750 0x7f1ca40b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 -- 172.21.15.33:0/386179606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980094f0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca3fff700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca4154130 0x7f1ca40c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca2ffd700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 0x7f1ca40b5ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009290 tx=0x7f1c98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800bbe0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98011420 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980117c0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.271+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca40b46c0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.272+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40b4c80 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.273+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca4169f70 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.273+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c98011bf0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.274+0000 7f1c8b7fe700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c90066f30 0x7f1c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.274+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f1c98085fc0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.275+0000 7f1ca37fe700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c90066f30 0x7f1c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.275+0000 7f1ca37fe700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c90066f30 0x7f1c900693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009db0 tx=0x7f1c940091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.278+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c98052ff0 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.527+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ca404c380 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.528+0000 7f1c8b7fe700 1 -- 172.21.15.33:0/386179606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f1c98056610 con 0x7f1ca40ccfc0 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c90066f30 msgr2=0x7f1c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c90066f30 0x7f1c900693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009db0 tx=0x7f1c940091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 msgr2=0x7f1ca40b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 0x7f1ca40b5ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009290 tx=0x7f1c98017000 comp rx=0 tx=0).stop 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 shutdown_connections 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c90066f30 0x7f1c900693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ca4154130 0x7f1ca40c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ca40cc750 0x7f1ca40b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 --2- 172.21.15.33:0/386179606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca40ccfc0 0x7f1ca40b5ad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 >> 172.21.15.33:0/386179606 conn(0x7f1ca406cff0 msgr2=0x7f1ca4157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 shutdown_connections 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.530+0000 7f1ca899d700 1 -- 172.21.15.33:0/386179606 wait complete. 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:15:07.550 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:07.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.736+0000 7fadef59e700 1 Processor -- start 2023-10-06T05:15:07.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.736+0000 7fadef59e700 1 -- start start 2023-10-06T05:15:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.737+0000 7fadef59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fadf00c8050 0x7fadf013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.737+0000 7fadef59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00c9cb0 con 0x7fadf006aad0 2023-10-06T05:15:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.737+0000 7fadef59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00ca270 con 0x7fadf013faa0 2023-10-06T05:15:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.737+0000 7fadef59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf006b640 con 0x7fadf00c8050 2023-10-06T05:15:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7fadeed9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fadf006aad0 0x7fadf006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54814/0 (socket says 172.21.15.33:54814) 2023-10-06T05:15:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7fadeed9d700 1 -- 172.21.15.33:0/319452579 learned_addr learned my addr 172.21.15.33:0/319452579 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7fadedd9b700 1 --2- 172.21.15.33:0/319452579 >> v2:172.21.15.167:3300/0 conn(0x7fadf00c8050 0x7fadf013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742750000 0 0) 0x7fadf00c9cb0 con 0x7fadf006aad0 2023-10-06T05:15:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd8003570 con 0x7fadf006aad0 2023-10-06T05:15:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843176012 0 0) 0x7fadf00ca270 con 0x7fadf013faa0 2023-10-06T05:15:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf00c9cb0 con 0x7fadf013faa0 2023-10-06T05:15:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7fadedd9b700 1 -- 172.21.15.33:0/319452579 >> v1:172.21.15.167:6789/0 conn(0x7fadf013faa0 legacy=0x7fadf00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7fadedd9b700 1 -- 172.21.15.33:0/319452579 >> v1:172.21.15.33:6789/0 conn(0x7fadf006aad0 legacy=0x7fadf006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.738+0000 7fadedd9b700 1 -- 172.21.15.33:0/319452579 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf006b840 con 0x7fadf00c8050 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.739+0000 7fadedd9b700 1 --2- 172.21.15.33:0/319452579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf00c8050 0x7fadf013fe70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fade40098e0 tx=0x7fade40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a280efc6f576e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.739+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4011630 con 0x7fadf00c8050 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.739+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade4011790 con 0x7fadf00c8050 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.739+0000 7faded59a700 1 -- 172.21.15.33:0/319452579 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4011b30 con 0x7fadf00c8050 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.739+0000 7fadef59e700 1 -- 172.21.15.33:0/319452579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf00c8050 msgr2=0x7fadf013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.739+0000 7fadef59e700 1 --2- 172.21.15.33:0/319452579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf00c8050 0x7fadf013fe70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fade40098e0 tx=0x7fade40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.740+0000 7fadef59e700 1 -- 172.21.15.33:0/319452579 shutdown_connections 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.740+0000 7fadef59e700 1 --2- 172.21.15.33:0/319452579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf00c8050 0x7fadf013fe70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.740+0000 7fadef59e700 1 -- 172.21.15.33:0/319452579 >> 172.21.15.33:0/319452579 conn(0x7fadf006d030 msgr2=0x7fadf006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.740+0000 7fadef59e700 1 -- 172.21.15.33:0/319452579 shutdown_connections 2023-10-06T05:15:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.740+0000 7fadef59e700 1 -- 172.21.15.33:0/319452579 wait complete. 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.741+0000 7fadef59e700 1 Processor -- start 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.741+0000 7fadef59e700 1 -- start start 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadef59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf006aad0 0x7fadf01686b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadef59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 0x7fadf013dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadef59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf013faa0 0x7fadf00a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadef59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf015ad60 con 0x7fadf00c8050 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadef59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf013e2e0 con 0x7fadf013faa0 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadef59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf01581f0 con 0x7fadf006aad0 2023-10-06T05:15:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 0x7fadf013dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 0x7fadf013dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55952/0 (socket says 172.21.15.33:55952) 2023-10-06T05:15:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.742+0000 7fadedd9b700 1 -- 172.21.15.33:0/2583675392 learned_addr learned my addr 172.21.15.33:0/2583675392 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadeed9d700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf013faa0 0x7fadf00a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadee59c700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf006aad0 0x7fadf01686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadedd9b700 1 -- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf006aad0 msgr2=0x7fadf01686b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadedd9b700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf006aad0 0x7fadf01686b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadedd9b700 1 -- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf013faa0 msgr2=0x7fadf00a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadedd9b700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf013faa0 0x7fadf00a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadedd9b700 1 -- 172.21.15.33:0/2583675392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009420 con 0x7fadf00c8050 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadee59c700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf006aad0 0x7fadf01686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadeed9d700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf013faa0 0x7fadf00a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadedd9b700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 0x7fadf013dda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fadf00719c0 tx=0x7fade4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade401f070 con 0x7fadf00c8050 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade4000c90 con 0x7fadf00c8050 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4025420 con 0x7fadf00c8050 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf01584d0 con 0x7fadf00c8050 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.743+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf0158a90 con 0x7fadf00c8050 2023-10-06T05:15:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.744+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf004c380 con 0x7fadf00c8050 2023-10-06T05:15:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.745+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fade4023070 con 0x7fadf00c8050 2023-10-06T05:15:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.746+0000 7fadde7fc700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadd0066f30 0x7fadd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.746+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fade4085b70 con 0x7fadf00c8050 2023-10-06T05:15:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.746+0000 7fadee59c700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadd0066f30 0x7fadd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.747+0000 7fadee59c700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadd0066f30 0x7fadd00693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fadf016aa40 tx=0x7fade0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.750+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fade4052ba0 con 0x7fadf00c8050 2023-10-06T05:15:07.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.954+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadf006bb20 con 0x7fadf00c8050 2023-10-06T05:15:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.955+0000 7fadde7fc700 1 -- 172.21.15.33:0/2583675392 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16961 (secure 0 0 0) 0x7fade402a020 con 0x7fadf00c8050 2023-10-06T05:15:07.957 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:07.957 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:07.375790+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadd0066f30 msgr2=0x7fadd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadd0066f30 0x7fadd00693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fadf016aa40 tx=0x7fade0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 msgr2=0x7fadf013dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 0x7fadf013dda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fadf00719c0 tx=0x7fade4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 shutdown_connections 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadd0066f30 0x7fadd00693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf013faa0 0x7fadf00a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadf006aad0 0x7fadf01686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 --2- 172.21.15.33:0/2583675392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf00c8050 0x7fadf013dda0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 >> 172.21.15.33:0/2583675392 conn(0x7fadf006d030 msgr2=0x7fadf006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 shutdown_connections 2023-10-06T05:15:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:07.957+0000 7fadef59e700 1 -- 172.21.15.33:0/2583675392 wait complete. 2023-10-06T05:15:07.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:23.634036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:24.170951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:07.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.165+0000 7f6b6ed9d700 1 Processor -- start 2023-10-06T05:15:08.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.165+0000 7f6b6ed9d700 1 -- start start 2023-10-06T05:15:08.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680b0900 0x7f6b680af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6ed9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680ad430 con 0x7f6b68156370 2023-10-06T05:15:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6ed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680afe20 con 0x7f6b680cc7e0 2023-10-06T05:15:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6806bc70 con 0x7f6b680b0900 2023-10-06T05:15:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6b68156370 0x7f6b680c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54830/0 (socket says 172.21.15.33:54830) 2023-10-06T05:15:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6dd9b700 1 -- 172.21.15.33:0/1783528323 learned_addr learned my addr 172.21.15.33:0/1783528323 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/1783528323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680b0900 0x7f6b680af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.167+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002533928 0 0) 0x7f6b680ad430 con 0x7f6b68156370 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b54003570 con 0x7f6b68156370 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789633944 0 0) 0x7f6b680afe20 con 0x7f6b680cc7e0 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b680ad430 con 0x7f6b680cc7e0 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1783528323 >> v1:172.21.15.167:6789/0 conn(0x7f6b680cc7e0 legacy=0x7f6b680b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1783528323 >> v1:172.21.15.33:6789/0 conn(0x7f6b68156370 legacy=0x7f6b680c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1783528323 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680cccc0 con 0x7f6b680b0900 2023-10-06T05:15:08.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/1783528323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680b0900 0x7f6b680af8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009e90 tx=0x7f6b60009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dedd64f3b6aafc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6000e070 con 0x7f6b680b0900 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6000bb80 con 0x7f6b680b0900 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.168+0000 7f6b6cd99700 1 -- 172.21.15.33:0/1783528323 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60012430 con 0x7f6b680b0900 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1783528323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680b0900 msgr2=0x7f6b680af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1783528323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680b0900 0x7f6b680af8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009e90 tx=0x7f6b60009780 comp rx=0 tx=0).stop 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1783528323 shutdown_connections 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1783528323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680b0900 0x7f6b680af8e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1783528323 >> 172.21.15.33:0/1783528323 conn(0x7f6b6806d030 msgr2=0x7f6b680cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1783528323 shutdown_connections 2023-10-06T05:15:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.169+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1783528323 wait complete. 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.171+0000 7f6b6ed9d700 1 Processor -- start 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.171+0000 7f6b6ed9d700 1 -- start start 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b680b0900 0x7f6b68169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680cc7e0 0x7f6b680ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 0x7f6b680aa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6ed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680a9ca0 con 0x7f6b68156370 2023-10-06T05:15:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6ed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6816a310 con 0x7f6b680b0900 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6816a5f0 con 0x7f6b680cc7e0 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 0x7f6b680aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 0x7f6b680aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55956/0 (socket says 172.21.15.33:55956) 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1215277706 learned_addr learned my addr 172.21.15.33:0/1215277706 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6dd9b700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b680b0900 0x7f6b68169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6d59a700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680cc7e0 0x7f6b680ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680cc7e0 msgr2=0x7f6b680ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680cc7e0 0x7f6b680ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b680b0900 msgr2=0x7f6b68169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b680b0900 0x7f6b68169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.172+0000 7f6b6e59c700 1 -- 172.21.15.33:0/1215277706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009a20 con 0x7f6b68156370 2023-10-06T05:15:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.173+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 0x7f6b680aa130 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b60013000 tx=0x7f6b60019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.173+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6001b420 con 0x7f6b68156370 2023-10-06T05:15:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.173+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6001b5a0 con 0x7f6b68156370 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.173+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6001b960 con 0x7f6b68156370 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.173+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b680a9930 con 0x7f6b68156370 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.173+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6816a8d0 con 0x7f6b68156370 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.174+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6804c380 con 0x7f6b68156370 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.175+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b6001bd50 con 0x7f6b68156370 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.176+0000 7f6b5dffb700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.176+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6b60086150 con 0x7f6b68156370 2023-10-06T05:15:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.176+0000 7f6b6dd9b700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.177+0000 7f6b6dd9b700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007fd0 tx=0x7f6b64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.180+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b60056cc0 con 0x7f6b68156370 2023-10-06T05:15:08.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.382+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b680a9de0 con 0x7f6b68156370 2023-10-06T05:15:08.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.383+0000 7f6b5dffb700 1 -- 172.21.15.33:0/1215277706 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b60056850 con 0x7f6b68156370 2023-10-06T05:15:08.385 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:15:08.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 msgr2=0x7f6b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007fd0 tx=0x7f6b64009040 comp rx=0 tx=0).stop 2023-10-06T05:15:08.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 msgr2=0x7f6b680aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 0x7f6b680aa130 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b60013000 tx=0x7f6b60019000 comp rx=0 tx=0).stop 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 shutdown_connections 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b68156370 0x7f6b680aa130 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b680b0900 0x7f6b68169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/1215277706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b680cc7e0 0x7f6b680ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.385+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 >> 172.21.15.33:0/1215277706 conn(0x7f6b6806d030 msgr2=0x7f6b6806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.386+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 shutdown_connections 2023-10-06T05:15:08.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.386+0000 7f6b6ed9d700 1 -- 172.21.15.33:0/1215277706 wait complete. 2023-10-06T05:15:08.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.592+0000 7f07e179d700 1 Processor -- start 2023-10-06T05:15:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.592+0000 7f07e179d700 1 -- start start 2023-10-06T05:15:08.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07e179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07dc0b1f90 0x7f07dc0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07e179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc070440 0x7f07dc067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07e179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc06ae10 con 0x7f07dc068140 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07e179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc06a2d0 con 0x7f07dc0b1f90 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07e179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc154290 con 0x7f07dc070440 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07dc0b1f90 0x7f07dc0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07dc0b1f90 0x7f07dc0abbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55972/0 (socket says 172.21.15.33:55972) 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 -- 172.21.15.33:0/2517642574 learned_addr learned my addr 172.21.15.33:0/2517642574 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 -- 172.21.15.33:0/2517642574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc070440 msgr2=0x7f07dc067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07db7fe700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc070440 0x7f07dc067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc070440 0x7f07dc067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 -- 172.21.15.33:0/2517642574 >> v1:172.21.15.167:6789/0 conn(0x7f07dc068140 legacy=0x7f07dc06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:08.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.594+0000 7f07dbfff700 1 -- 172.21.15.33:0/2517642574 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc068580 con 0x7f07dc0b1f90 2023-10-06T05:15:08.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07db7fe700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc070440 0x7f07dc067c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:08.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07dbfff700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc0b1f90 0x7f07dc0abbd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099b0 tx=0x7f07cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f7b28fbbfaa954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:08.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07daffd700 1 -- 172.21.15.33:0/2517642574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00b820 con 0x7f07dc0b1f90 2023-10-06T05:15:08.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07daffd700 1 -- 172.21.15.33:0/2517642574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00b980 con 0x7f07dc0b1f90 2023-10-06T05:15:08.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07daffd700 1 -- 172.21.15.33:0/2517642574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00bd20 con 0x7f07dc0b1f90 2023-10-06T05:15:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07e179d700 1 -- 172.21.15.33:0/2517642574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc0b1f90 msgr2=0x7f07dc0abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.595+0000 7f07e179d700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc0b1f90 0x7f07dc0abbd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099b0 tx=0x7f07cc009250 comp rx=0 tx=0).stop 2023-10-06T05:15:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.596+0000 7f07e179d700 1 -- 172.21.15.33:0/2517642574 shutdown_connections 2023-10-06T05:15:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.596+0000 7f07e179d700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc0b1f90 0x7f07dc0abbd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.596+0000 7f07e179d700 1 --2- 172.21.15.33:0/2517642574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc070440 0x7f07dc067c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.596+0000 7f07e179d700 1 -- 172.21.15.33:0/2517642574 >> 172.21.15.33:0/2517642574 conn(0x7f07dc06ea70 msgr2=0x7f07dc06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.596+0000 7f07e179d700 1 -- 172.21.15.33:0/2517642574 shutdown_connections 2023-10-06T05:15:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.596+0000 7f07e179d700 1 -- 172.21.15.33:0/2517642574 wait complete. 2023-10-06T05:15:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.597+0000 7f07e179d700 1 Processor -- start 2023-10-06T05:15:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.597+0000 7f07e179d700 1 -- start start 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07e179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc068140 0x7f07dc1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07e179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 0x7f07dc15db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07e179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc0b1f90 0x7f07dc13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07e179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc15e060 con 0x7f07dc070440 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07e179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc159980 con 0x7f07dc068140 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07e179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc159c60 con 0x7f07dc0b1f90 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 0x7f07dc15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 0x7f07dc15db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55984/0 (socket says 172.21.15.33:55984) 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.598+0000 7f07db7fe700 1 -- 172.21.15.33:0/2072585760 learned_addr learned my addr 172.21.15.33:0/2072585760 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07dbfff700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc068140 0x7f07dc1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07e0f9c700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc0b1f90 0x7f07dc13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07db7fe700 1 -- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc0b1f90 msgr2=0x7f07dc13e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07db7fe700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc0b1f90 0x7f07dc13e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07db7fe700 1 -- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc068140 msgr2=0x7f07dc1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07db7fe700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc068140 0x7f07dc1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07db7fe700 1 -- 172.21.15.33:0/2072585760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0094f0 con 0x7f07dc070440 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07db7fe700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 0x7f07dc15db20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c830 tx=0x7f07d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0011600 con 0x7f07dc070440 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0011760 con 0x7f07dc070440 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0011b20 con 0x7f07dc070440 2023-10-06T05:15:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.599+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc159f40 con 0x7f07dc070440 2023-10-06T05:15:08.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.600+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc169b60 con 0x7f07dc070440 2023-10-06T05:15:08.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.601+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc04c380 con 0x7f07dc070440 2023-10-06T05:15:08.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.602+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07d000a3e0 con 0x7f07dc070440 2023-10-06T05:15:08.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.603+0000 7f07c3fff700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07c8066f30 0x7f07c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:08.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.603+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f07d0084c00 con 0x7f07dc070440 2023-10-06T05:15:08.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.604+0000 7f07dbfff700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07c8066f30 0x7f07c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:08.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.604+0000 7f07dbfff700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07c8066f30 0x7f07c80693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099b0 tx=0x7f07cc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.607+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07d0051c30 con 0x7f07dc070440 2023-10-06T05:15:08.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.811+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07dc06a4e0 con 0x7f07dc070440 2023-10-06T05:15:08.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.812+0000 7f07c3fff700 1 -- 172.21.15.33:0/2072585760 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f07d00150a0 con 0x7f07dc070440 2023-10-06T05:15:08.813 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07c8066f30 msgr2=0x7f07c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07c8066f30 0x7f07c80693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099b0 tx=0x7f07cc016000 comp rx=0 tx=0).stop 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 msgr2=0x7f07dc15db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 0x7f07dc15db20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c830 tx=0x7f07d0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 shutdown_connections 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07c8066f30 0x7f07c80693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07dc0b1f90 0x7f07dc13e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07dc068140 0x7f07dc1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 --2- 172.21.15.33:0/2072585760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07dc070440 0x7f07dc15db20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:08.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 >> 172.21.15.33:0/2072585760 conn(0x7f07dc06ea70 msgr2=0x7f07dc0acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:08.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 shutdown_connections 2023-10-06T05:15:08.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:08.814+0000 7f07e179d700 1 -- 172.21.15.33:0/2072585760 wait complete. 2023-10-06T05:15:08.833 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.020+0000 7f2c3ffff700 1 Processor -- start 2023-10-06T05:15:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.020+0000 7f2c3ffff700 1 -- start start 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2c40156370 0x7f2c400af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c400c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c400ad430 con 0x7f2c400b01f0 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c400afe20 con 0x7f2c40156370 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4006bc70 con 0x7f2c400cc7e0 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2c40156370 0x7f2c400af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2c400b01f0 0x7f2c400ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54836/0 (socket says 172.21.15.33:54836) 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3f7fe700 1 -- 172.21.15.33:0/1306178227 learned_addr learned my addr 172.21.15.33:0/1306178227 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3e7fc700 1 --2- 172.21.15.33:0/1306178227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c400c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3effd700 1 -- 172.21.15.33:0/1306178227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 msgr2=0x7f2c400c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3effd700 1 --2- 172.21.15.33:0/1306178227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c400c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3effd700 1 -- 172.21.15.33:0/1306178227 >> v1:172.21.15.33:6789/0 conn(0x7f2c400b01f0 legacy=0x7f2c400ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3effd700 1 -- 172.21.15.33:0/1306178227 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c400c92f0 con 0x7f2c40156370 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.022+0000 7f2c3dffb700 1 -- 172.21.15.33:0/1306178227 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638482329 0 0) 0x7f2c400ad430 con 0x7f2c400b01f0 2023-10-06T05:15:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3effd700 1 --2- 172.21.15.33:0/1306178227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40156370 0x7f2c400af8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c300098e0 tx=0x7f2c300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a700b2cfa596c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3dffb700 1 -- 172.21.15.33:0/1306178227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30009e80 con 0x7f2c40156370 2023-10-06T05:15:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3dffb700 1 -- 172.21.15.33:0/1306178227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3000b860 con 0x7f2c40156370 2023-10-06T05:15:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3dffb700 1 -- 172.21.15.33:0/1306178227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000bbe0 con 0x7f2c40156370 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1306178227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40156370 msgr2=0x7f2c400af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1306178227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40156370 0x7f2c400af8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c300098e0 tx=0x7f2c300091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.023+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1306178227 shutdown_connections 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.024+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1306178227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c40156370 0x7f2c400af8e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.024+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1306178227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c400c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.024+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1306178227 >> 172.21.15.33:0/1306178227 conn(0x7f2c4006d030 msgr2=0x7f2c400cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.024+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1306178227 shutdown_connections 2023-10-06T05:15:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.024+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1306178227 wait complete. 2023-10-06T05:15:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.025+0000 7f2c3ffff700 1 Processor -- start 2023-10-06T05:15:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.025+0000 7f2c3ffff700 1 -- start start 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 0x7f2c40142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c401427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c40156370 0x7f2c40145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c40134680 con 0x7f2c400b01f0 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c401454c0 con 0x7f2c40156370 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c401448b0 con 0x7f2c400cc7e0 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 0x7f2c40142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 0x7f2c40142000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56000/0 (socket says 172.21.15.33:56000) 2023-10-06T05:15:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3effd700 1 -- 172.21.15.33:0/1076277549 learned_addr learned my addr 172.21.15.33:0/1076277549 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.026+0000 7f2c3e7fc700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c401427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3f7fe700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c40156370 0x7f2c40145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3effd700 1 -- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 msgr2=0x7f2c401427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3effd700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c401427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3effd700 1 -- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c40156370 msgr2=0x7f2c40145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3effd700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c40156370 0x7f2c40145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3effd700 1 -- 172.21.15.33:0/1076277549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c30009420 con 0x7f2c400b01f0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3effd700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 0x7f2c40142000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006d4d0 tx=0x7f2c30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000bbe0 con 0x7f2c400b01f0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30011420 con 0x7f2c400b01f0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30011800 con 0x7f2c400b01f0 2023-10-06T05:15:09.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c40144b90 con 0x7f2c400b01f0 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.027+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c40145150 con 0x7f2c400b01f0 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.028+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4004c380 con 0x7f2c400b01f0 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.029+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c30011bb0 con 0x7f2c400b01f0 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.030+0000 7f2c26ffd700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c2c066f30 0x7f2c2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.030+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f2c300860e0 con 0x7f2c400b01f0 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.031+0000 7f2c3e7fc700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c2c066f30 0x7f2c2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.032+0000 7f2c3e7fc700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c2c066f30 0x7f2c2c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006a6e0 tx=0x7f2c34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.034+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c30053110 con 0x7f2c400b01f0 2023-10-06T05:15:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.236+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c400b1420 con 0x7f2c400b01f0 2023-10-06T05:15:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.236+0000 7f2c26ffd700 1 -- 172.21.15.33:0/1076277549 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c30011e60 con 0x7f2c400b01f0 2023-10-06T05:15:09.238 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:15:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c2c066f30 msgr2=0x7f2c2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c2c066f30 0x7f2c2c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006a6e0 tx=0x7f2c34008040 comp rx=0 tx=0).stop 2023-10-06T05:15:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 msgr2=0x7f2c40142000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 0x7f2c40142000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006d4d0 tx=0x7f2c30017000 comp rx=0 tx=0).stop 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 shutdown_connections 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c2c066f30 0x7f2c2c0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c40156370 0x7f2c40145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c400b01f0 0x7f2c40142000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 --2- 172.21.15.33:0/1076277549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c400cc7e0 0x7f2c401427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.238+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 >> 172.21.15.33:0/1076277549 conn(0x7f2c4006d030 msgr2=0x7f2c400ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.239+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 shutdown_connections 2023-10-06T05:15:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.239+0000 7f2c3ffff700 1 -- 172.21.15.33:0/1076277549 wait complete. 2023-10-06T05:15:09.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.446+0000 7f0db8f9d700 1 Processor -- start 2023-10-06T05:15:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.446+0000 7f0db8f9d700 1 -- start start 2023-10-06T05:15:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.448+0000 7f0db8f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0db406c7b0 0x7f0db4067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.448+0000 7f0db8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 0x7f0db415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.448+0000 7f0db8f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db415d3a0 con 0x7f0db40aff90 2023-10-06T05:15:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.448+0000 7f0db8f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db40a9fa0 con 0x7f0db406c7b0 2023-10-06T05:15:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.448+0000 7f0db8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40b0470 con 0x7f0db406cb80 2023-10-06T05:15:09.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.448+0000 7f0db3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 0x7f0db415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 0x7f0db415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:42614/0 (socket says 172.21.15.33:42614) 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 -- 172.21.15.33:0/766178797 learned_addr learned my addr 172.21.15.33:0/766178797 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db2ffd700 1 --1- 172.21.15.33:0/766178797 >> v1:172.21.15.167:6789/0 conn(0x7f0db40aff90 0x7f0db40a9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:46832/0 (socket says 172.21.15.33:46832) 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db37fe700 1 --2- 172.21.15.33:0/766178797 >> v2:172.21.15.167:3300/0 conn(0x7f0db406c7b0 0x7f0db4067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 -- 172.21.15.33:0/766178797 >> v2:172.21.15.167:3300/0 conn(0x7f0db406c7b0 msgr2=0x7f0db4067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 --2- 172.21.15.33:0/766178797 >> v2:172.21.15.167:3300/0 conn(0x7f0db406c7b0 0x7f0db4067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 -- 172.21.15.33:0/766178797 >> v1:172.21.15.167:6789/0 conn(0x7f0db40aff90 legacy=0x7f0db40a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 -- 172.21.15.33:0/766178797 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db40b0750 con 0x7f0db406cb80 2023-10-06T05:15:09.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.449+0000 7f0db3fff700 1 --2- 172.21.15.33:0/766178797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 0x7f0db415d830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009f20 tx=0x7f0dac009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d317aa0aa0ddef8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db27fc700 1 -- 172.21.15.33:0/766178797 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00e070 con 0x7f0db406cb80 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db27fc700 1 -- 172.21.15.33:0/766178797 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac00b9d0 con 0x7f0db406cb80 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db27fc700 1 -- 172.21.15.33:0/766178797 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0092e0 con 0x7f0db406cb80 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 -- 172.21.15.33:0/766178797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 msgr2=0x7f0db415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/766178797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 0x7f0db415d830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009f20 tx=0x7f0dac009840 comp rx=0 tx=0).stop 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 -- 172.21.15.33:0/766178797 shutdown_connections 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/766178797 >> v2:172.21.15.167:3300/0 conn(0x7f0db406c7b0 0x7f0db4067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/766178797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406cb80 0x7f0db415d830 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 -- 172.21.15.33:0/766178797 >> 172.21.15.33:0/766178797 conn(0x7f0db406d670 msgr2=0x7f0db406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 -- 172.21.15.33:0/766178797 shutdown_connections 2023-10-06T05:15:09.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.450+0000 7f0db8f9d700 1 -- 172.21.15.33:0/766178797 wait complete. 2023-10-06T05:15:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.452+0000 7f0db8f9d700 1 Processor -- start 2023-10-06T05:15:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.452+0000 7f0db8f9d700 1 -- start start 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406c7b0 0x7f0db4151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db8f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 0x7f0db4152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db8f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0db40aff90 0x7f0db4157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db8f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db4153520 con 0x7f0db406cb80 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db8f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db414a1d0 con 0x7f0db40aff90 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db41526d0 con 0x7f0db406c7b0 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 0x7f0db4152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 0x7f0db4152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56008/0 (socket says 172.21.15.33:56008) 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db2ffd700 1 -- 172.21.15.33:0/499410274 learned_addr learned my addr 172.21.15.33:0/499410274 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db37fe700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406c7b0 0x7f0db4151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db3fff700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0db40aff90 0x7f0db4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.453+0000 7f0db2ffd700 1 -- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406c7b0 msgr2=0x7f0db4151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db2ffd700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406c7b0 0x7f0db4151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db2ffd700 1 -- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0db40aff90 msgr2=0x7f0db4157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db2ffd700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0db40aff90 0x7f0db4157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db2ffd700 1 -- 172.21.15.33:0/499410274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac009ae0 con 0x7f0db406cb80 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db37fe700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406c7b0 0x7f0db4151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db2ffd700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 0x7f0db4152190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c560 tx=0x7f0da8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8005a70 con 0x7f0db406cb80 2023-10-06T05:15:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8005bd0 con 0x7f0db406cb80 2023-10-06T05:15:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db41529a0 con 0x7f0db406cb80 2023-10-06T05:15:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80070d0 con 0x7f0db406cb80 2023-10-06T05:15:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.454+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db4152d30 con 0x7f0db406cb80 2023-10-06T05:15:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.455+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db414ef90 con 0x7f0db406cb80 2023-10-06T05:15:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.456+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0da8007480 con 0x7f0db406cb80 2023-10-06T05:15:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.457+0000 7f0d9b7fe700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0da0066f30 0x7f0da00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.457+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f0da8083be0 con 0x7f0db406cb80 2023-10-06T05:15:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.457+0000 7f0db37fe700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0da0066f30 0x7f0da00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.458+0000 7f0db37fe700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0da0066f30 0x7f0da00693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009d70 tx=0x7f0da4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.461+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0db414ef90 con 0x7f0db406cb80 2023-10-06T05:15:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.664+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0db404c380 con 0x7f0db406cb80 2023-10-06T05:15:09.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.664+0000 7f0d9b7fe700 1 -- 172.21.15.33:0/499410274 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0da8017020 con 0x7f0db406cb80 2023-10-06T05:15:09.666 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:15:09.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.666+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0da0066f30 msgr2=0x7f0da00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.666+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0da0066f30 0x7f0da00693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009d70 tx=0x7f0da4008040 comp rx=0 tx=0).stop 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.666+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 msgr2=0x7f0db4152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.666+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 0x7f0db4152190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c560 tx=0x7f0da8007f40 comp rx=0 tx=0).stop 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 shutdown_connections 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0da0066f30 0x7f0da00693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0db40aff90 0x7f0db4157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0db406c7b0 0x7f0db4151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 --2- 172.21.15.33:0/499410274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db406cb80 0x7f0db4152190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 >> 172.21.15.33:0/499410274 conn(0x7f0db406d670 msgr2=0x7f0db414a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 shutdown_connections 2023-10-06T05:15:09.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.667+0000 7f0db8f9d700 1 -- 172.21.15.33:0/499410274 wait complete. 2023-10-06T05:15:09.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:09.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.871+0000 7f21d7fff700 1 Processor -- start 2023-10-06T05:15:09.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.871+0000 7f21d7fff700 1 -- start start 2023-10-06T05:15:09.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f21d813faa0 0x7f21d813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80c9cb0 con 0x7f21d806aad0 2023-10-06T05:15:09.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d806b660 con 0x7f21d813faa0 2023-10-06T05:15:09.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80ca270 con 0x7f21d80c7940 2023-10-06T05:15:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d67fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d67fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55362/0 (socket says 172.21.15.33:55362) 2023-10-06T05:15:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d67fc700 1 -- 172.21.15.33:0/380982321 learned_addr learned my addr 172.21.15.33:0/380982321 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.873+0000 7f21d6ffd700 1 --2- 172.21.15.33:0/380982321 >> v2:172.21.15.167:3300/0 conn(0x7f21d813faa0 0x7f21d813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d67fc700 1 -- 172.21.15.33:0/380982321 >> v2:172.21.15.167:3300/0 conn(0x7f21d813faa0 msgr2=0x7f21d813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d67fc700 1 --2- 172.21.15.33:0/380982321 >> v2:172.21.15.167:3300/0 conn(0x7f21d813faa0 0x7f21d813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d67fc700 1 -- 172.21.15.33:0/380982321 >> v1:172.21.15.167:6789/0 conn(0x7f21d806aad0 legacy=0x7f21d806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d67fc700 1 -- 172.21.15.33:0/380982321 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d806b840 con 0x7f21d80c7940 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d5ffb700 1 -- 172.21.15.33:0/380982321 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487172772 0 0) 0x7f21d80c9cb0 con 0x7f21d806aad0 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d67fc700 1 --2- 172.21.15.33:0/380982321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d80c7db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0099b0 tx=0x7f21cc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6821b0e957461168 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d5ffb700 1 -- 172.21.15.33:0/380982321 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc009ea0 con 0x7f21d80c7940 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d5ffb700 1 -- 172.21.15.33:0/380982321 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc00b860 con 0x7f21d80c7940 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.874+0000 7f21d5ffb700 1 -- 172.21.15.33:0/380982321 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00bc00 con 0x7f21d80c7940 2023-10-06T05:15:09.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 -- 172.21.15.33:0/380982321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 msgr2=0x7f21d80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 --2- 172.21.15.33:0/380982321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d80c7db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0099b0 tx=0x7f21cc009250 comp rx=0 tx=0).stop 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 -- 172.21.15.33:0/380982321 shutdown_connections 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 --2- 172.21.15.33:0/380982321 >> v2:172.21.15.167:3300/0 conn(0x7f21d813faa0 0x7f21d813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 --2- 172.21.15.33:0/380982321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d80c7db0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 -- 172.21.15.33:0/380982321 >> 172.21.15.33:0/380982321 conn(0x7f21d806d030 msgr2=0x7f21d806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 -- 172.21.15.33:0/380982321 shutdown_connections 2023-10-06T05:15:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.875+0000 7f21d7fff700 1 -- 172.21.15.33:0/380982321 wait complete. 2023-10-06T05:15:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.876+0000 7f21d7fff700 1 Processor -- start 2023-10-06T05:15:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 -- start start 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 0x7f21d815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d813edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21d813faa0 0x7f21d815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d816abf0 con 0x7f21d806aad0 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d816a760 con 0x7f21d813faa0 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.877+0000 7f21d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8169b50 con 0x7f21d80c7940 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 0x7f21d815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 0x7f21d815dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59830/0 (socket says 172.21.15.33:59830) 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 -- 172.21.15.33:0/4009655048 learned_addr learned my addr 172.21.15.33:0/4009655048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d67fc700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d813edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d77fe700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21d813faa0 0x7f21d815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 -- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 msgr2=0x7f21d813edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d813edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 -- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21d813faa0 msgr2=0x7f21d815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21d813faa0 0x7f21d815ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.878+0000 7f21d6ffd700 1 -- 172.21.15.33:0/4009655048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc0094f0 con 0x7f21d806aad0 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.879+0000 7f21d6ffd700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 0x7f21d815dcd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21c800e950 tx=0x7f21c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.879+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800c850 con 0x7f21d806aad0 2023-10-06T05:15:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.879+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c800c9b0 con 0x7f21d806aad0 2023-10-06T05:15:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.879+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d8169e30 con 0x7f21d806aad0 2023-10-06T05:15:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.879+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800cd70 con 0x7f21d806aad0 2023-10-06T05:15:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.879+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d816a3f0 con 0x7f21d806aad0 2023-10-06T05:15:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.880+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d804c380 con 0x7f21d806aad0 2023-10-06T05:15:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.881+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21c8013680 con 0x7f21d806aad0 2023-10-06T05:15:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.882+0000 7f21beffd700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f21c4066f30 0x7f21c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.882+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f21c80849f0 con 0x7f21d806aad0 2023-10-06T05:15:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.882+0000 7f21d67fc700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f21c4066f30 0x7f21c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.883+0000 7f21d67fc700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f21c4066f30 0x7f21c40693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0099b0 tx=0x7f21cc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:09.886+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21c8051a20 con 0x7f21d806aad0 2023-10-06T05:15:10.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.131+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21d806bb20 con 0x7f21d806aad0 2023-10-06T05:15:10.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.132+0000 7f21beffd700 1 -- 172.21.15.33:0/4009655048 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f21c8055040 con 0x7f21d806aad0 2023-10-06T05:15:10.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:10.134 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f21c4066f30 msgr2=0x7f21c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f21c4066f30 0x7f21c40693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0099b0 tx=0x7f21cc017000 comp rx=0 tx=0).stop 2023-10-06T05:15:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 msgr2=0x7f21d815dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 0x7f21d815dcd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21c800e950 tx=0x7f21c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 shutdown_connections 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f21c4066f30 0x7f21c40693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21d813faa0 0x7f21d815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21d806aad0 0x7f21d815dcd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 --2- 172.21.15.33:0/4009655048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21d80c7940 0x7f21d813edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.134+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 >> 172.21.15.33:0/4009655048 conn(0x7f21d806d030 msgr2=0x7f21d806e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.135+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 shutdown_connections 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.135+0000 7f21d7fff700 1 -- 172.21.15.33:0/4009655048 wait complete. 2023-10-06T05:15:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:15:10.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.340+0000 7fe70db9d700 1 Processor -- start 2023-10-06T05:15:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.340+0000 7fe70db9d700 1 -- start start 2023-10-06T05:15:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.341+0000 7fe70db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 0x7fe7080b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.341+0000 7fe70db9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080cbd80 con 0x7fe7080cf3b0 2023-10-06T05:15:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.341+0000 7fe70db9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080af5f0 con 0x7fe7080ccfc0 2023-10-06T05:15:10.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.341+0000 7fe70db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080af8d0 con 0x7fe708159bb0 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.341+0000 7fe6fffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7080cf3b0 0x7fe7081594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35458/0 (socket says 172.21.15.33:35458) 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.341+0000 7fe6fffff700 1 -- 172.21.15.33:0/2767897128 learned_addr learned my addr 172.21.15.33:0/2767897128 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe70d39c700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 0x7fe7080b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246877546 0 0) 0x7fe7080cbd80 con 0x7fe7080cf3b0 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f0003570 con 0x7fe7080cf3b0 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135205254 0 0) 0x7fe6f0003570 con 0x7fe7080cf3b0 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 msgr2=0x7fe7080b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:10.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 0x7fe7080b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 >> v1:172.21.15.167:6789/0 conn(0x7fe7080ccfc0 legacy=0x7fe7080cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080afbb0 con 0x7fe7080cf3b0 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe6f8001470 con 0x7fe7080cf3b0 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 >> v1:172.21.15.33:6789/0 conn(0x7fe7080cf3b0 legacy=0x7fe7081594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 0x7fe6f0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe6f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f0004d50 0x7fe6f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080af8d0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f000bba0 con 0x7fe6f0004d50 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.342+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f000bd60 con 0x7fe708159bb0 2023-10-06T05:15:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe6f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe70cb9b700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 0x7fe6f0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe70d39c700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f0004d50 0x7fe6f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 -- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 msgr2=0x7fe6f0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe708159bb0 0x7fe6f0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 -- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f0004d50 msgr2=0x7fe6f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f0004d50 0x7fe6f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 -- 172.21.15.33:0/2767897128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0003570 con 0x7fe6f0003fd0 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe70d39c700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f0004d50 0x7fe6f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.343+0000 7fe6fffff700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe6f0004810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7080b0250 tx=0x7fe6f8001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc964b820c18bbec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800cb40 con 0x7fe6f0003fd0 2023-10-06T05:15:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f800ccc0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/2767897128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8005b70 con 0x7fe6f0003fd0 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70db9d700 1 -- 172.21.15.33:0/2767897128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 msgr2=0x7fe6f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70db9d700 1 --2- 172.21.15.33:0/2767897128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe6f0004810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7080b0250 tx=0x7fe6f8001da0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70d39c700 1 -- 172.21.15.33:0/2767897128 reap_dead start 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70db9d700 1 -- 172.21.15.33:0/2767897128 shutdown_connections 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70db9d700 1 -- 172.21.15.33:0/2767897128 >> 172.21.15.33:0/2767897128 conn(0x7fe70806cff0 msgr2=0x7fe70806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70db9d700 1 -- 172.21.15.33:0/2767897128 shutdown_connections 2023-10-06T05:15:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.344+0000 7fe70db9d700 1 -- 172.21.15.33:0/2767897128 wait complete. 2023-10-06T05:15:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.346+0000 7fe70db9d700 1 Processor -- start 2023-10-06T05:15:10.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.346+0000 7fe70db9d700 1 -- start start 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe70814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7080c5a10 0x7fe7080c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7080b32a0 0x7fe7080b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080ca1c0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080c63c0 con 0x7fe7080b32a0 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe708150f10 con 0x7fe7080c5a10 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe70814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe70814f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59842/0 (socket says 172.21.15.33:59842) 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70cb9b700 1 -- 172.21.15.33:0/3637700904 learned_addr learned my addr 172.21.15.33:0/3637700904 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe6fffff700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7080c5a10 0x7fe7080c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70d39c700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7080b32a0 0x7fe7080b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.347+0000 7fe70cb9b700 1 -- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7080c5a10 msgr2=0x7fe7080c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70cb9b700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7080c5a10 0x7fe7080c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70cb9b700 1 -- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7080b32a0 msgr2=0x7fe7080b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70cb9b700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7080b32a0 0x7fe7080b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70cb9b700 1 -- 172.21.15.33:0/3637700904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080b3c50 con 0x7fe6f0003fd0 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe6fffff700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7080c5a10 0x7fe7080c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70d39c700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7080b32a0 0x7fe7080b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70cb9b700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe70814f320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe704002be0 tx=0x7fe704007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400f9f0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70400fb50 con 0x7fe6f0003fd0 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040133f0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70814f860 con 0x7fe6f0003fd0 2023-10-06T05:15:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.348+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7080b4190 con 0x7fe6f0003fd0 2023-10-06T05:15:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.349+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70804c380 con 0x7fe6f0003fd0 2023-10-06T05:15:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.350+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe704013760 con 0x7fe6f0003fd0 2023-10-06T05:15:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.351+0000 7fe6fcff9700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6f4066f30 0x7fe6f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.351+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe704084ee0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.351+0000 7fe6fffff700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6f4066f30 0x7fe6f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.352+0000 7fe6fffff700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6f4066f30 0x7fe6f40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe7080cd880 tx=0x7fe6f8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.355+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe704051f10 con 0x7fe6f0003fd0 2023-10-06T05:15:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.558+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe708150ce0 con 0x7fe6f0003fd0 2023-10-06T05:15:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.559+0000 7fe6fcff9700 1 -- 172.21.15.33:0/3637700904 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16961 (secure 0 0 0) 0x7fe704055530 con 0x7fe6f0003fd0 2023-10-06T05:15:10.562 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:10.562 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:07.375790+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6f4066f30 msgr2=0x7fe6f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6f4066f30 0x7fe6f40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe7080cd880 tx=0x7fe6f8015000 comp rx=0 tx=0).stop 2023-10-06T05:15:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 msgr2=0x7fe70814f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe70814f320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe704002be0 tx=0x7fe704007d10 comp rx=0 tx=0).stop 2023-10-06T05:15:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 shutdown_connections 2023-10-06T05:15:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6f4066f30 0x7fe6f40693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7080b32a0 0x7fe7080b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0003fd0 0x7fe70814f320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 --2- 172.21.15.33:0/3637700904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7080c5a10 0x7fe7080c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 >> 172.21.15.33:0/3637700904 conn(0x7fe70806cff0 msgr2=0x7fe7081571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 shutdown_connections 2023-10-06T05:15:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.562+0000 7fe70db9d700 1 -- 172.21.15.33:0/3637700904 wait complete. 2023-10-06T05:15:10.582 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:15:10.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.767+0000 7f493f59e700 1 Processor -- start 2023-10-06T05:15:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.767+0000 7f493f59e700 1 -- start start 2023-10-06T05:15:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f494006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49400b0e10 0x7f49400b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494015e2d0 con 0x7f49400b2e30 2023-10-06T05:15:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494015d6c0 con 0x7f49400b0e10 2023-10-06T05:15:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494015d9a0 con 0x7f49400b0a40 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49400b0e10 0x7f49400b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49400b0e10 0x7f49400b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59854/0 (socket says 172.21.15.33:59854) 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493dd9b700 1 -- 172.21.15.33:0/597228722 learned_addr learned my addr 172.21.15.33:0/597228722 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.769+0000 7f493e59c700 1 --2- 172.21.15.33:0/597228722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f494006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493dd9b700 1 -- 172.21.15.33:0/597228722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 msgr2=0x7f494006ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493dd9b700 1 --2- 172.21.15.33:0/597228722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f494006ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493dd9b700 1 -- 172.21.15.33:0/597228722 >> v1:172.21.15.167:6789/0 conn(0x7f49400b2e30 legacy=0x7f494015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493dd9b700 1 -- 172.21.15.33:0/597228722 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494015dc80 con 0x7f49400b0e10 2023-10-06T05:15:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493dd9b700 1 --2- 172.21.15.33:0/597228722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b0e10 0x7f49400b28f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f49340099b0 tx=0x7f4934009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c114014d0ef110 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493d59a700 1 -- 172.21.15.33:0/597228722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400b860 con 0x7f49400b0e10 2023-10-06T05:15:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493d59a700 1 -- 172.21.15.33:0/597228722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400b9c0 con 0x7f49400b0e10 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.770+0000 7f493d59a700 1 -- 172.21.15.33:0/597228722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400bd60 con 0x7f49400b0e10 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 -- 172.21.15.33:0/597228722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b0e10 msgr2=0x7f49400b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 --2- 172.21.15.33:0/597228722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b0e10 0x7f49400b28f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f49340099b0 tx=0x7f4934009250 comp rx=0 tx=0).stop 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 -- 172.21.15.33:0/597228722 shutdown_connections 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 --2- 172.21.15.33:0/597228722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f494006ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 --2- 172.21.15.33:0/597228722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b0e10 0x7f49400b28f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 -- 172.21.15.33:0/597228722 >> 172.21.15.33:0/597228722 conn(0x7f494006e0f0 msgr2=0x7f49400675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 -- 172.21.15.33:0/597228722 shutdown_connections 2023-10-06T05:15:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.771+0000 7f493f59e700 1 -- 172.21.15.33:0/597228722 wait complete. 2023-10-06T05:15:10.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.772+0000 7f493f59e700 1 Processor -- start 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 -- start start 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f4940142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49400b0e10 0x7f49400c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 0x7f49400c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940145dd0 con 0x7f49400b2e30 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49401451c0 con 0x7f49400b0e10 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.773+0000 7f493f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49401454a0 con 0x7f49400b0a40 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 0x7f49400c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 0x7f49400c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59856/0 (socket says 172.21.15.33:59856) 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 -- 172.21.15.33:0/3281860272 learned_addr learned my addr 172.21.15.33:0/3281860272 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493e59c700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f4940142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493dd9b700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49400b0e10 0x7f49400c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 -- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 msgr2=0x7f4940142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f4940142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 -- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49400b0e10 msgr2=0x7f49400c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49400b0e10 0x7f49400c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 -- 172.21.15.33:0/3281860272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340094f0 con 0x7f49400b2e30 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493dd9b700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49400b0e10 0x7f49400c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.774+0000 7f493ed9d700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 0x7f49400c2510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f493800c840 tx=0x7f4938001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.775+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938012630 con 0x7f49400b2e30 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.775+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380127b0 con 0x7f49400b2e30 2023-10-06T05:15:10.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.775+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938012b70 con 0x7f49400b2e30 2023-10-06T05:15:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.775+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4940145780 con 0x7f49400b2e30 2023-10-06T05:15:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.775+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4940143340 con 0x7f49400b2e30 2023-10-06T05:15:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.776+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494006fcc0 con 0x7f49400b2e30 2023-10-06T05:15:10.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.777+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4938003030 con 0x7f49400b2e30 2023-10-06T05:15:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.778+0000 7f492e7fc700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4920066f30 0x7f49200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.779+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f49380845d0 con 0x7f49400b2e30 2023-10-06T05:15:10.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.779+0000 7f493e59c700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4920066f30 0x7f49200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:10.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.780+0000 7f493e59c700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4920066f30 0x7f49200693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4930006fd0 tx=0x7f4930008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:10.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:10.783+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4938055130 con 0x7f49400b2e30 2023-10-06T05:15:11.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.029+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f494004c380 con 0x7f49400b2e30 2023-10-06T05:15:11.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.030+0000 7f492e7fc700 1 -- 172.21.15.33:0/3281860272 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f4938054cc0 con 0x7f49400b2e30 2023-10-06T05:15:11.031 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:11.031 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4920066f30 msgr2=0x7f49200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4920066f30 0x7f49200693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4930006fd0 tx=0x7f4930008040 comp rx=0 tx=0).stop 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 msgr2=0x7f49400c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 0x7f49400c2510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f493800c840 tx=0x7f4938001670 comp rx=0 tx=0).stop 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 shutdown_connections 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4920066f30 0x7f49200693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49400b2e30 0x7f49400c2510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49400b0a40 0x7f4940142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 --2- 172.21.15.33:0/3281860272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49400b0e10 0x7f49400c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 >> 172.21.15.33:0/3281860272 conn(0x7f494006e0f0 msgr2=0x7f49400ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:11.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 shutdown_connections 2023-10-06T05:15:11.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.032+0000 7f493f59e700 1 -- 172.21.15.33:0/3281860272 wait complete. 2023-10-06T05:15:11.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:15:11.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.237+0000 7f568f39d700 1 Processor -- start 2023-10-06T05:15:11.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.237+0000 7f568f39d700 1 -- start start 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568f39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56880c6340 0x7f56880c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568f39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f56880c80d0 0x7f568806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568f39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568806c820 con 0x7f5688159520 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568f39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568806a160 con 0x7f56880c6340 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568f39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880c8800 con 0x7f56880c80d0 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568db9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56880c6340 0x7f56880c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568db9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56880c6340 0x7f56880c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59868/0 (socket says 172.21.15.33:59868) 2023-10-06T05:15:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.238+0000 7f568db9a700 1 -- 172.21.15.33:0/1165128874 learned_addr learned my addr 172.21.15.33:0/1165128874 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568eb9c700 1 --2- 172.21.15.33:0/1165128874 >> v2:172.21.15.167:3300/0 conn(0x7f56880c80d0 0x7f568806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568db9a700 1 -- 172.21.15.33:0/1165128874 >> v2:172.21.15.167:3300/0 conn(0x7f56880c80d0 msgr2=0x7f568806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568db9a700 1 --2- 172.21.15.33:0/1165128874 >> v2:172.21.15.167:3300/0 conn(0x7f56880c80d0 0x7f568806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568db9a700 1 -- 172.21.15.33:0/1165128874 >> v1:172.21.15.167:6789/0 conn(0x7f5688159520 legacy=0x7f56880c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568db9a700 1 -- 172.21.15.33:0/1165128874 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880c8ae0 con 0x7f56880c6340 2023-10-06T05:15:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568db9a700 1 --2- 172.21.15.33:0/1165128874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f56880c7b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f56880b29e0 tx=0x7f5678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0a7d70993d1ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568d399700 1 -- 172.21.15.33:0/1165128874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678009e70 con 0x7f56880c6340 2023-10-06T05:15:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568d399700 1 -- 172.21.15.33:0/1165128874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567800b860 con 0x7f56880c6340 2023-10-06T05:15:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.239+0000 7f568d399700 1 -- 172.21.15.33:0/1165128874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800bc00 con 0x7f56880c6340 2023-10-06T05:15:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 -- 172.21.15.33:0/1165128874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 msgr2=0x7f56880c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 --2- 172.21.15.33:0/1165128874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f56880c7b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f56880b29e0 tx=0x7f5678009250 comp rx=0 tx=0).stop 2023-10-06T05:15:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 -- 172.21.15.33:0/1165128874 shutdown_connections 2023-10-06T05:15:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 --2- 172.21.15.33:0/1165128874 >> v2:172.21.15.167:3300/0 conn(0x7f56880c80d0 0x7f568806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 --2- 172.21.15.33:0/1165128874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f56880c7b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 -- 172.21.15.33:0/1165128874 >> 172.21.15.33:0/1165128874 conn(0x7f56880cc930 msgr2=0x7f56880ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 -- 172.21.15.33:0/1165128874 shutdown_connections 2023-10-06T05:15:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.240+0000 7f568f39d700 1 -- 172.21.15.33:0/1165128874 wait complete. 2023-10-06T05:15:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.242+0000 7f568f39d700 1 Processor -- start 2023-10-06T05:15:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.242+0000 7f568f39d700 1 -- start start 2023-10-06T05:15:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.242+0000 7f568f39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f5688156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568f39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56880c80d0 0x7f56881697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568f39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5688159520 0x7f568815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568f39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568815b7e0 con 0x7f56880c6340 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568f39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568815abd0 con 0x7f5688159520 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568f39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568815aeb0 con 0x7f56880c80d0 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f5688156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f5688156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59880/0 (socket says 172.21.15.33:59880) 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 -- 172.21.15.33:0/1415259941 learned_addr learned my addr 172.21.15.33:0/1415259941 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568db9a700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56880c80d0 0x7f56881697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568eb9c700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5688159520 0x7f568815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 -- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56880c80d0 msgr2=0x7f56881697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56880c80d0 0x7f56881697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 -- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5688159520 msgr2=0x7f568815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5688159520 0x7f568815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.243+0000 7f568e39b700 1 -- 172.21.15.33:0/1415259941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780094f0 con 0x7f56880c6340 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.244+0000 7f568e39b700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f5688156070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f568400c4e0 tx=0x7f568400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.244+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684011590 con 0x7f56880c6340 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.244+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684011710 con 0x7f56880c6340 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.244+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684011ad0 con 0x7f56880c6340 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.244+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568815b190 con 0x7f56880c6340 2023-10-06T05:15:11.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.244+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881565b0 con 0x7f56880c6340 2023-10-06T05:15:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.245+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568804c380 con 0x7f56880c6340 2023-10-06T05:15:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.246+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f568401b3b0 con 0x7f56880c6340 2023-10-06T05:15:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.246+0000 7f567e7fc700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.247+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5684083ca0 con 0x7f56880c6340 2023-10-06T05:15:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.247+0000 7f568db9a700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.248+0000 7f568db9a700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f566c066f30 0x7f566c0693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f56880b29e0 tx=0x7f5678017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.251+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5684050cd0 con 0x7f56880c6340 2023-10-06T05:15:11.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.498+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56880c8dc0 con 0x7f56880c6340 2023-10-06T05:15:11.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.498+0000 7f567e7fc700 1 -- 172.21.15.33:0/1415259941 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f5684017070 con 0x7f56880c6340 2023-10-06T05:15:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:11.500 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/296316609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":296316609},{"type":"v1","addr":"172.21.15.33:6835","nonce":296316609}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/953266297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":953266297},{"type":"v1","addr":"172.21.15.167:6836","nonce":953266297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3788601500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3788601500},{"type":"v1","addr":"172.21.15.33:6834","nonce":3788601500}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:25.204019+0000","modified":"2023-10-06T05:14:28.726642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/256792669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":256792669},{"type":"v1","addr":"172.21.15.167:6837","nonce":256792669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f566c066f30 msgr2=0x7f566c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f566c066f30 0x7f566c0693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f56880b29e0 tx=0x7f5678017000 comp rx=0 tx=0).stop 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 msgr2=0x7f5688156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f5688156070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f568400c4e0 tx=0x7f568400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 shutdown_connections 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5688159520 0x7f568815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56880c6340 0x7f5688156070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 --2- 172.21.15.33:0/1415259941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56880c80d0 0x7f56881697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 >> 172.21.15.33:0/1415259941 conn(0x7f56880cc930 msgr2=0x7f5688156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 shutdown_connections 2023-10-06T05:15:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.501+0000 7f568f39d700 1 -- 172.21.15.33:0/1415259941 wait complete. 2023-10-06T05:15:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2023-10-06T05:15:11.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.704+0000 7fd8d559d700 1 Processor -- start 2023-10-06T05:15:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.704+0000 7fd8d559d700 1 -- start start 2023-10-06T05:15:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8d559d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8d00ca590 0x7fd8d00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8d559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8d559d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00c92d0 con 0x7fd8d0159650 2023-10-06T05:15:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8d559d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d00c7bc0 con 0x7fd8d00ca590 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8d559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c86c0 con 0x7fd8d0159a20 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8cffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd8d0159650 0x7fd8d00c9e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35468/0 (socket says 172.21.15.33:35468) 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8cffff700 1 -- 172.21.15.33:0/1720996913 learned_addr learned my addr 172.21.15.33:0/1720996913 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8d4d9c700 1 --2- 172.21.15.33:0/1720996913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8cf7fe700 1 --2- 172.21.15.33:0/1720996913 >> v2:172.21.15.167:3300/0 conn(0x7fd8d00ca590 0x7fd8d00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.706+0000 7fd8ceffd700 1 -- 172.21.15.33:0/1720996913 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510933775 0 0) 0x7fd8d00c92d0 con 0x7fd8d0159650 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8ceffd700 1 -- 172.21.15.33:0/1720996913 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b8003570 con 0x7fd8d0159650 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8d4d9c700 1 -- 172.21.15.33:0/1720996913 >> v2:172.21.15.167:3300/0 conn(0x7fd8d00ca590 msgr2=0x7fd8d00c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:11.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8d4d9c700 1 --2- 172.21.15.33:0/1720996913 >> v2:172.21.15.167:3300/0 conn(0x7fd8d00ca590 0x7fd8d00c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8d4d9c700 1 -- 172.21.15.33:0/1720996913 >> v1:172.21.15.33:6789/0 conn(0x7fd8d0159650 legacy=0x7fd8d00c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8d4d9c700 1 -- 172.21.15.33:0/1720996913 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00c89a0 con 0x7fd8d0159a20 2023-10-06T05:15:11.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8d4d9c700 1 --2- 172.21.15.33:0/1720996913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00c7680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009e60 tx=0x7fd8c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275f0a2903ac9689 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8ceffd700 1 -- 172.21.15.33:0/1720996913 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800e070 con 0x7fd8d0159a20 2023-10-06T05:15:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8ceffd700 1 -- 172.21.15.33:0/1720996913 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800bab0 con 0x7fd8d0159a20 2023-10-06T05:15:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.707+0000 7fd8ceffd700 1 -- 172.21.15.33:0/1720996913 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800be70 con 0x7fd8d0159a20 2023-10-06T05:15:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 -- 172.21.15.33:0/1720996913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 msgr2=0x7fd8d00c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 --2- 172.21.15.33:0/1720996913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00c7680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009e60 tx=0x7fd8c8009780 comp rx=0 tx=0).stop 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 -- 172.21.15.33:0/1720996913 shutdown_connections 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 --2- 172.21.15.33:0/1720996913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00c7680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 --2- 172.21.15.33:0/1720996913 >> v2:172.21.15.167:3300/0 conn(0x7fd8d00ca590 0x7fd8d00c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 -- 172.21.15.33:0/1720996913 >> 172.21.15.33:0/1720996913 conn(0x7fd8d006d030 msgr2=0x7fd8d006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 -- 172.21.15.33:0/1720996913 shutdown_connections 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.708+0000 7fd8d559d700 1 -- 172.21.15.33:0/1720996913 wait complete. 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.709+0000 7fd8d559d700 1 Processor -- start 2023-10-06T05:15:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.710+0000 7fd8d559d700 1 -- start start 2023-10-06T05:15:11.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.710+0000 7fd8d559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d00ca590 0x7fd8d00b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 0x7fd8d014a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00ae8d0 con 0x7fd8d0159650 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00b01b0 con 0x7fd8d00ca590 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00b0490 con 0x7fd8d0159a20 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 0x7fd8d014a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 0x7fd8d014a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59890/0 (socket says 172.21.15.33:59890) 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 -- 172.21.15.33:0/785008480 learned_addr learned my addr 172.21.15.33:0/785008480 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cffff700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d00ca590 0x7fd8d00b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d4d9c700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 -- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 msgr2=0x7fd8d00aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 -- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d00ca590 msgr2=0x7fd8d00b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d00ca590 0x7fd8d00b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8cf7fe700 1 -- 172.21.15.33:0/785008480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8009a20 con 0x7fd8d0159650 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.711+0000 7fd8d4d9c700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.712+0000 7fd8cf7fe700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 0x7fd8d014a4f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400ec50 tx=0x7fd8c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.712+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400c850 con 0x7fd8d0159650 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.712+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c400c9b0 con 0x7fd8d0159650 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.712+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400cd70 con 0x7fd8d0159650 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.712+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00b0770 con 0x7fd8d0159650 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.712+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d00b0d30 con 0x7fd8d0159650 2023-10-06T05:15:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.713+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d004c380 con 0x7fd8d0159650 2023-10-06T05:15:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.714+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8c4013680 con 0x7fd8d0159650 2023-10-06T05:15:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.715+0000 7fd8b7fff700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8bc066f30 0x7fd8bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.715+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fd8c4084a40 con 0x7fd8d0159650 2023-10-06T05:15:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.715+0000 7fd8cffff700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8bc066f30 0x7fd8bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:11.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.716+0000 7fd8cffff700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8bc066f30 0x7fd8bc0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007fd0 tx=0x7fd8c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.719+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8c4051a70 con 0x7fd8d0159650 2023-10-06T05:15:11.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.926+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8d00c8c80 con 0x7fd8d0159650 2023-10-06T05:15:11.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.927+0000 7fd8b7fff700 1 -- 172.21.15.33:0/785008480 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16961 (secure 0 0 0) 0x7fd8c4055090 con 0x7fd8d0159650 2023-10-06T05:15:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:11.929 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:07.375790+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:23.634036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:24.170951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8bc066f30 msgr2=0x7fd8bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8bc066f30 0x7fd8bc0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007fd0 tx=0x7fd8c0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 msgr2=0x7fd8d014a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 0x7fd8d014a4f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400ec50 tx=0x7fd8c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 shutdown_connections 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8bc066f30 0x7fd8bc0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8d0159a20 0x7fd8d00aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d00ca590 0x7fd8d00b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 --2- 172.21.15.33:0/785008480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d0159650 0x7fd8d014a4f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 >> 172.21.15.33:0/785008480 conn(0x7fd8d006d030 msgr2=0x7fd8d0154250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 shutdown_connections 2023-10-06T05:15:11.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:11.929+0000 7fd8d559d700 1 -- 172.21.15.33:0/785008480 wait complete. 2023-10-06T05:15:11.948 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:15:12.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.134+0000 7f006cf9d700 1 Processor -- start 2023-10-06T05:15:12.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.134+0000 7f006cf9d700 1 -- start start 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f006cf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00680ca7a0 0x7f0068159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f006cf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f006806c560 0x7f00681596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f006cf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f006806d480 0x7f006806a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f006cf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0068159c10 con 0x7f006806d480 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f006cf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0068070810 con 0x7f006806c560 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f006cf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006806da40 con 0x7f00680ca7a0 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.135+0000 7f0067fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f006806d480 0x7f006806a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f006806d480 0x7f006806a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59902/0 (socket says 172.21.15.33:59902) 2023-10-06T05:15:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 -- 172.21.15.33:0/2837226487 learned_addr learned my addr 172.21.15.33:0/2837226487 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f00677fe700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00680ca7a0 0x7f0068159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0066ffd700 1 --2- 172.21.15.33:0/2837226487 >> v2:172.21.15.167:3300/0 conn(0x7f006806c560 0x7f00681596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 -- 172.21.15.33:0/2837226487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00680ca7a0 msgr2=0x7f0068159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00680ca7a0 0x7f0068159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 -- 172.21.15.33:0/2837226487 >> v2:172.21.15.167:3300/0 conn(0x7f006806c560 msgr2=0x7f00681596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 --2- 172.21.15.33:0/2837226487 >> v2:172.21.15.167:3300/0 conn(0x7f006806c560 0x7f00681596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 -- 172.21.15.33:0/2837226487 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006806dd20 con 0x7f006806d480 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f00677fe700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00680ca7a0 0x7f0068159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0066ffd700 1 --2- 172.21.15.33:0/2837226487 >> v2:172.21.15.167:3300/0 conn(0x7f006806c560 0x7f00681596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f0067fff700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006806d480 0x7f006806a260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f006806f650 tx=0x7f0060009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83ab1aba9e4d7d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.136+0000 7f00667fc700 1 -- 172.21.15.33:0/2837226487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000e070 con 0x7f006806d480 2023-10-06T05:15:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f00667fc700 1 -- 172.21.15.33:0/2837226487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000ba50 con 0x7f006806d480 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f00667fc700 1 -- 172.21.15.33:0/2837226487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600092e0 con 0x7f006806d480 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 -- 172.21.15.33:0/2837226487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006806d480 msgr2=0x7f006806a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006806d480 0x7f006806a260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f006806f650 tx=0x7f0060009840 comp rx=0 tx=0).stop 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 -- 172.21.15.33:0/2837226487 shutdown_connections 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006806d480 0x7f006806a260 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 --2- 172.21.15.33:0/2837226487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00680ca7a0 0x7f0068159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 --2- 172.21.15.33:0/2837226487 >> v2:172.21.15.167:3300/0 conn(0x7f006806c560 0x7f00681596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 -- 172.21.15.33:0/2837226487 >> 172.21.15.33:0/2837226487 conn(0x7f006806e650 msgr2=0x7f006806ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 -- 172.21.15.33:0/2837226487 shutdown_connections 2023-10-06T05:15:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.137+0000 7f006cf9d700 1 -- 172.21.15.33:0/2837226487 wait complete. 2023-10-06T05:15:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.139+0000 7f006cf9d700 1 Processor -- start 2023-10-06T05:15:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.139+0000 7f006cf9d700 1 -- start start 2023-10-06T05:15:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f006cf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006806c560 0x7f006815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f006cf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006806d480 0x7f00681503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f006cf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 0x7f00680b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f006cf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006815cd80 con 0x7f00680ca7a0 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f006cf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006815aaa0 con 0x7f006806d480 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f006cf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006815ad80 con 0x7f006806c560 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0067fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 0x7f00680b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0067fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 0x7f00680b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59904/0 (socket says 172.21.15.33:59904) 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0067fff700 1 -- 172.21.15.33:0/1140709466 learned_addr learned my addr 172.21.15.33:0/1140709466 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f00677fe700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006806c560 0x7f006815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0066ffd700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006806d480 0x7f00681503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0067fff700 1 -- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006806c560 msgr2=0x7f006815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0067fff700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006806c560 0x7f006815c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.140+0000 7f0067fff700 1 -- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006806d480 msgr2=0x7f00681503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f0067fff700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006806d480 0x7f00681503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f0067fff700 1 -- 172.21.15.33:0/1140709466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009ae0 con 0x7f00680ca7a0 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f0067fff700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 0x7f00680b1b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0068069d40 tx=0x7f0060018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600032f0 con 0x7f00680ca7a0 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0060008ce0 con 0x7f00680ca7a0 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006815b060 con 0x7f00680ca7a0 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600266b0 con 0x7f00680ca7a0 2023-10-06T05:15:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.141+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006815b620 con 0x7f00680ca7a0 2023-10-06T05:15:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.142+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006804c380 con 0x7f00680ca7a0 2023-10-06T05:15:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.143+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0060003040 con 0x7f00680ca7a0 2023-10-06T05:15:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.144+0000 7f004f7fe700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0054066f30 0x7f00540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.144+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f00600865a0 con 0x7f00680ca7a0 2023-10-06T05:15:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.145+0000 7f00677fe700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0054066f30 0x7f00540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.145+0000 7f00677fe700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0054066f30 0x7f00540693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0058006fd0 tx=0x7f0058008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.148+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00600535d0 con 0x7f00680ca7a0 2023-10-06T05:15:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.397+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f00680c7060 con 0x7f00680ca7a0 2023-10-06T05:15:12.400 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:15:12.398+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:15:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.402+0000 7f004f7fe700 1 -- 172.21.15.33:0/1140709466 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0060056bf0 con 0x7f00680ca7a0 2023-10-06T05:15:12.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0054066f30 msgr2=0x7f00540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0054066f30 0x7f00540693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0058006fd0 tx=0x7f0058008040 comp rx=0 tx=0).stop 2023-10-06T05:15:12.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 msgr2=0x7f00680b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 0x7f00680b1b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0068069d40 tx=0x7f0060018000 comp rx=0 tx=0).stop 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 shutdown_connections 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0054066f30 0x7f00540693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00680ca7a0 0x7f00680b1b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006806c560 0x7f006815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 --2- 172.21.15.33:0/1140709466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006806d480 0x7f00681503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 >> 172.21.15.33:0/1140709466 conn(0x7f006806e650 msgr2=0x7f006806a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.404+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 shutdown_connections 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.405+0000 7f006cf9d700 1 -- 172.21.15.33:0/1140709466 wait complete. 2023-10-06T05:15:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:15:12.411 INFO:tasks.ceph.mds.d.smithi167.stderr: -2179> 2023-10-06T05:14:30.861+0000 7f294aff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:15:12.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:15:12.456 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:15:12.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.612+0000 7ff76359e700 1 Processor -- start 2023-10-06T05:15:12.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.612+0000 7ff76359e700 1 -- start start 2023-10-06T05:15:12.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.613+0000 7ff76359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7640af540 0x7ff7640af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.613+0000 7ff76359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 0x7ff76415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.613+0000 7ff76359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76415d660 con 0x7ff76415a650 2023-10-06T05:15:12.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.613+0000 7ff76359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640ce890 con 0x7ff7640af540 2023-10-06T05:15:12.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.613+0000 7ff76359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76415aac0 con 0x7ff7640cd5e0 2023-10-06T05:15:12.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff762d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 0x7ff76415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76259c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7640af540 0x7ff7640af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff762d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 0x7ff76415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55502/0 (socket says 172.21.15.33:55502) 2023-10-06T05:15:12.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff761d9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff76415a650 0x7ff7640cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:59944/0 (socket says 172.21.15.33:59944) 2023-10-06T05:15:12.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff762d9d700 1 -- 172.21.15.33:0/2582198189 learned_addr learned my addr 172.21.15.33:0/2582198189 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:12.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76259c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7640af540 0x7ff7640af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:50700/0 (socket says 172.21.15.33:50700) 2023-10-06T05:15:12.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76159a700 1 -- 172.21.15.33:0/2582198189 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960723770 0 0) 0x7ff76415d660 con 0x7ff76415a650 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76159a700 1 -- 172.21.15.33:0/2582198189 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c003570 con 0x7ff76415a650 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76259c700 1 -- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 msgr2=0x7ff76415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76259c700 1 --2- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 0x7ff76415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76259c700 1 -- 172.21.15.33:0/2582198189 >> v1:172.21.15.167:6789/0 conn(0x7ff76415a650 legacy=0x7ff7640cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff76259c700 1 -- 172.21.15.33:0/2582198189 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76415d890 con 0x7ff7640af540 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.614+0000 7ff762d9d700 1 --2- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 0x7ff76415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76259c700 1 --2- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640af540 0x7ff7640af910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7540098e0 tx=0x7ff7540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a883cffca98b0e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:12.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76159a700 1 -- 172.21.15.33:0/2582198189 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754011630 con 0x7ff7640af540 2023-10-06T05:15:12.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76159a700 1 -- 172.21.15.33:0/2582198189 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754011790 con 0x7ff7640af540 2023-10-06T05:15:12.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76159a700 1 -- 172.21.15.33:0/2582198189 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754011b30 con 0x7ff7640af540 2023-10-06T05:15:12.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76359e700 1 -- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640af540 msgr2=0x7ff7640af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76359e700 1 --2- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640af540 0x7ff7640af910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7540098e0 tx=0x7ff7540091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76359e700 1 -- 172.21.15.33:0/2582198189 shutdown_connections 2023-10-06T05:15:12.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76359e700 1 --2- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640af540 0x7ff7640af910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76359e700 1 --2- 172.21.15.33:0/2582198189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7640cd5e0 0x7ff76415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.615+0000 7ff76359e700 1 -- 172.21.15.33:0/2582198189 >> 172.21.15.33:0/2582198189 conn(0x7ff76406d9b0 msgr2=0x7ff76406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:12.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.616+0000 7ff76359e700 1 -- 172.21.15.33:0/2582198189 shutdown_connections 2023-10-06T05:15:12.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.616+0000 7ff76359e700 1 -- 172.21.15.33:0/2582198189 wait complete. 2023-10-06T05:15:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.617+0000 7ff76359e700 1 Processor -- start 2023-10-06T05:15:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.617+0000 7ff76359e700 1 -- start start 2023-10-06T05:15:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 0x7ff7640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640cd5e0 0x7ff76416da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76415a650 0x7ff7640c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640b6080 con 0x7ff7640af540 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640b5470 con 0x7ff7640cd5e0 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640b5750 con 0x7ff76415a650 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 0x7ff7640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 0x7ff7640b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59918/0 (socket says 172.21.15.33:59918) 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.618+0000 7ff76259c700 1 -- 172.21.15.33:0/556451157 learned_addr learned my addr 172.21.15.33:0/556451157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff761d9b700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640cd5e0 0x7ff76416da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff762d9d700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76415a650 0x7ff7640c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76259c700 1 -- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76415a650 msgr2=0x7ff7640c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76259c700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76415a650 0x7ff7640c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76259c700 1 -- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640cd5e0 msgr2=0x7ff76416da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76259c700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640cd5e0 0x7ff76416da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76259c700 1 -- 172.21.15.33:0/556451157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c009ae0 con 0x7ff7640af540 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76259c700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 0x7ff7640b4dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7640c7320 tx=0x7ff754017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754000d10 con 0x7ff7640af540 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754011b10 con 0x7ff7640af540 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754025660 con 0x7ff7640af540 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.619+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff754009420 con 0x7ff7640af540 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.620+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7640b5d10 con 0x7ff7640af540 2023-10-06T05:15:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.620+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76404c380 con 0x7ff7640af540 2023-10-06T05:15:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.621+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff754025a50 con 0x7ff7640af540 2023-10-06T05:15:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.622+0000 7ff7527fc700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.623+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7ff754085eb0 con 0x7ff7640af540 2023-10-06T05:15:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.623+0000 7ff761d9b700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.624+0000 7ff761d9b700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff744066f30 0x7ff7440693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7640c9f80 tx=0x7ff758007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.627+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff754052e90 con 0x7ff7640af540 2023-10-06T05:15:12.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:12.879+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff76415db70 con 0x7ff7640af540 2023-10-06T05:15:13.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.408+0000 7ff7527fc700 1 -- 172.21.15.33:0/556451157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7540564b0 con 0x7ff7640af540 2023-10-06T05:15:13.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff744066f30 msgr2=0x7ff7440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff744066f30 0x7ff7440693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7640c9f80 tx=0x7ff758007d40 comp rx=0 tx=0).stop 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 msgr2=0x7ff7640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 0x7ff7640b4dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7640c7320 tx=0x7ff754017000 comp rx=0 tx=0).stop 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 shutdown_connections 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff76415a650 0x7ff7640c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640af540 0x7ff7640b4dd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.410+0000 7ff76359e700 1 --2- 172.21.15.33:0/556451157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7640cd5e0 0x7ff76416da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.411+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 >> 172.21.15.33:0/556451157 conn(0x7ff76406d9b0 msgr2=0x7ff76415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:13.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.411+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 shutdown_connections 2023-10-06T05:15:13.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.411+0000 7ff76359e700 1 -- 172.21.15.33:0/556451157 wait complete. 2023-10-06T05:15:13.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:15:13.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.625+0000 7fe1bffff700 1 Processor -- start 2023-10-06T05:15:13.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.625+0000 7fe1bffff700 1 -- start start 2023-10-06T05:15:13.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1c0155aa0 0x7fe1c00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:13.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155e70 0x7fe1c00ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:13.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c0069ad0 con 0x7fe1c00acb30 2023-10-06T05:15:13.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c00acfb0 con 0x7fe1c0155aa0 2023-10-06T05:15:13.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c0069db0 con 0x7fe1c0155e70 2023-10-06T05:15:13.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1beffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1c0155aa0 0x7fe1c00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:13.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bf7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe1c00acb30 0x7fe1c00c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35484/0 (socket says 172.21.15.33:35484) 2023-10-06T05:15:13.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.626+0000 7fe1bf7fe700 1 -- 172.21.15.33:0/3053969101 learned_addr learned my addr 172.21.15.33:0/3053969101 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:13.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1be7fc700 1 --2- 172.21.15.33:0/3053969101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155e70 0x7fe1c00ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1bdffb700 1 -- 172.21.15.33:0/3053969101 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597922920 0 0) 0x7fe1c0069ad0 con 0x7fe1c00acb30 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1beffd700 1 -- 172.21.15.33:0/3053969101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155e70 msgr2=0x7fe1c00ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1beffd700 1 --2- 172.21.15.33:0/3053969101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155e70 0x7fe1c00ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1beffd700 1 -- 172.21.15.33:0/3053969101 >> v1:172.21.15.33:6789/0 conn(0x7fe1c00acb30 legacy=0x7fe1c00c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1beffd700 1 -- 172.21.15.33:0/3053969101 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c006a090 con 0x7fe1c0155aa0 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1beffd700 1 --2- 172.21.15.33:0/3053969101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155aa0 0x7fe1c00aef40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00098e0 tx=0x7fe1b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460b8343238088d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:13.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1bdffb700 1 -- 172.21.15.33:0/3053969101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0009e80 con 0x7fe1c0155aa0 2023-10-06T05:15:13.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1bdffb700 1 -- 172.21.15.33:0/3053969101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000b860 con 0x7fe1c0155aa0 2023-10-06T05:15:13.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.627+0000 7fe1bdffb700 1 -- 172.21.15.33:0/3053969101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000bbe0 con 0x7fe1c0155aa0 2023-10-06T05:15:13.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 -- 172.21.15.33:0/3053969101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155aa0 msgr2=0x7fe1c00aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:13.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 --2- 172.21.15.33:0/3053969101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155aa0 0x7fe1c00aef40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00098e0 tx=0x7fe1b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 -- 172.21.15.33:0/3053969101 shutdown_connections 2023-10-06T05:15:13.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 --2- 172.21.15.33:0/3053969101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155aa0 0x7fe1c00aef40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 --2- 172.21.15.33:0/3053969101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155e70 0x7fe1c00ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 -- 172.21.15.33:0/3053969101 >> 172.21.15.33:0/3053969101 conn(0x7fe1c0068090 msgr2=0x7fe1c006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:13.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 -- 172.21.15.33:0/3053969101 shutdown_connections 2023-10-06T05:15:13.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.628+0000 7fe1bffff700 1 -- 172.21.15.33:0/3053969101 wait complete. 2023-10-06T05:15:13.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.629+0000 7fe1bffff700 1 Processor -- start 2023-10-06T05:15:13.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.630+0000 7fe1bffff700 1 -- start start 2023-10-06T05:15:13.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.630+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1c00acb30 0x7fe1c00aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:13.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.630+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155aa0 0x7fe1c01690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 0x7fe1c0149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00af510 con 0x7fe1c0155e70 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00ab280 con 0x7fe1c00acb30 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00a7410 con 0x7fe1c0155aa0 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 0x7fe1c0149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 0x7fe1c0149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59928/0 (socket says 172.21.15.33:59928) 2023-10-06T05:15:13.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 -- 172.21.15.33:0/2041885551 learned_addr learned my addr 172.21.15.33:0/2041885551 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1beffd700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1c00acb30 0x7fe1c00aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1be7fc700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155aa0 0x7fe1c01690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 -- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155aa0 msgr2=0x7fe1c01690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155aa0 0x7fe1c01690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 -- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1c00acb30 msgr2=0x7fe1c00aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1c00acb30 0x7fe1c00aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.631+0000 7fe1bf7fe700 1 -- 172.21.15.33:0/2041885551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0009420 con 0x7fe1c0155e70 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.632+0000 7fe1bf7fe700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 0x7fe1c0149710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800cc90 tx=0x7fe1b8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.632+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800a870 con 0x7fe1c0155e70 2023-10-06T05:15:13.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.632+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b800a9d0 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.632+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800ad90 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.632+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c00a76f0 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.632+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c00af720 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.633+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c0063ff0 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.634+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1b8007300 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.635+0000 7fe1a6ffd700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ac066f30 0x7fe1ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.635+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fe1b8084a90 con 0x7fe1c0155e70 2023-10-06T05:15:13.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.636+0000 7fe1beffd700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ac066f30 0x7fe1ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.636+0000 7fe1beffd700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ac066f30 0x7fe1ac0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00091e0 tx=0x7fe1b0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:13.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.639+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1b8051af0 con 0x7fe1c0155e70 2023-10-06T05:15:13.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:13.847+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe1c0069640 con 0x7fe1c0155e70 2023-10-06T05:15:14.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.655+0000 7fe1a6ffd700 1 -- 172.21.15.33:0/2041885551 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe1b8055110 con 0x7fe1c0155e70 2023-10-06T05:15:14.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ac066f30 msgr2=0x7fe1ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ac066f30 0x7fe1ac0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00091e0 tx=0x7fe1b0017000 comp rx=0 tx=0).stop 2023-10-06T05:15:14.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 msgr2=0x7fe1c0149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 0x7fe1c0149710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800cc90 tx=0x7fe1b8001a20 comp rx=0 tx=0).stop 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 shutdown_connections 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ac066f30 0x7fe1ac0693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0155e70 0x7fe1c0149710 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1c00acb30 0x7fe1c00aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 --2- 172.21.15.33:0/2041885551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1c0155aa0 0x7fe1c01690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.659+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 >> 172.21.15.33:0/2041885551 conn(0x7fe1c0068090 msgr2=0x7fe1c00cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.660+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 shutdown_connections 2023-10-06T05:15:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.660+0000 7fe1bffff700 1 -- 172.21.15.33:0/2041885551 wait complete. 2023-10-06T05:15:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:15:14.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:15:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.879+0000 7f9e8f59d700 1 Processor -- start 2023-10-06T05:15:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.879+0000 7f9e8f59d700 1 -- start start 2023-10-06T05:15:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.880+0000 7f9e8f59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e881597f0 0x7f9e8815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8815d280 con 0x7f9e8806dc00 2023-10-06T05:15:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8f59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8815de00 con 0x7f9e8806dfd0 2023-10-06T05:15:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8f59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8815e290 con 0x7f9e881597f0 2023-10-06T05:15:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e881597f0 0x7f9e8815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e881597f0 0x7f9e8815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59932/0 (socket says 172.21.15.33:59932) 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8dd9a700 1 -- 172.21.15.33:0/3948368891 learned_addr learned my addr 172.21.15.33:0/3948368891 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e8806dc00 0x7f9e881590e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35490/0 (socket says 172.21.15.33:35490) 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8dd9a700 1 -- 172.21.15.33:0/3948368891 >> v1:172.21.15.167:6789/0 conn(0x7f9e8806dfd0 legacy=0x7f9e8815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8dd9a700 1 -- 172.21.15.33:0/3948368891 >> v1:172.21.15.33:6789/0 conn(0x7f9e8806dc00 legacy=0x7f9e881590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8dd9a700 1 -- 172.21.15.33:0/3948368891 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8815d4d0 con 0x7f9e881597f0 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.881+0000 7f9e8d599700 1 -- 172.21.15.33:0/3948368891 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343028269 0 0) 0x7f9e8815d280 con 0x7f9e8806dc00 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8dd9a700 1 --2- 172.21.15.33:0/3948368891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e8815a030 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e780099b0 tx=0x7f9e78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4e8e5b8d91cbd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8d599700 1 -- 172.21.15.33:0/3948368891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800b820 con 0x7f9e881597f0 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8d599700 1 -- 172.21.15.33:0/3948368891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7800b980 con 0x7f9e881597f0 2023-10-06T05:15:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8d599700 1 -- 172.21.15.33:0/3948368891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800bd20 con 0x7f9e881597f0 2023-10-06T05:15:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3948368891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 msgr2=0x7f9e8815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3948368891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e8815a030 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e780099b0 tx=0x7f9e78009250 comp rx=0 tx=0).stop 2023-10-06T05:15:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3948368891 shutdown_connections 2023-10-06T05:15:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3948368891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e8815a030 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3948368891 >> 172.21.15.33:0/3948368891 conn(0x7f9e880d0550 msgr2=0x7f9e880d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3948368891 shutdown_connections 2023-10-06T05:15:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.882+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3948368891 wait complete. 2023-10-06T05:15:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.884+0000 7f9e8f59d700 1 Processor -- start 2023-10-06T05:15:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.884+0000 7f9e8f59d700 1 -- start start 2023-10-06T05:15:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e8806dc00 0x7f9e880c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8806dfd0 0x7f9e880c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e880c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880ca090 con 0x7f9e881597f0 2023-10-06T05:15:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880ca370 con 0x7f9e8806dfd0 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8815b360 con 0x7f9e8806dc00 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e880c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e880c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59946/0 (socket says 172.21.15.33:59946) 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 -- 172.21.15.33:0/3114767231 learned_addr learned my addr 172.21.15.33:0/3114767231 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8e59b700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e8806dc00 0x7f9e880c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8dd9a700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8806dfd0 0x7f9e880c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 -- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e8806dc00 msgr2=0x7f9e880c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e8806dc00 0x7f9e880c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 -- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8806dfd0 msgr2=0x7f9e880c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8806dfd0 0x7f9e880c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8ed9c700 1 -- 172.21.15.33:0/3114767231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780094f0 con 0x7f9e881597f0 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.885+0000 7f9e8e59b700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e8806dc00 0x7f9e880c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.886+0000 7f9e8ed9c700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e880c9a50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c910 tx=0x7f9e80002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.886+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000a7e0 con 0x7f9e881597f0 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.886+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000a940 con 0x7f9e881597f0 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.886+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000ad00 con 0x7f9e881597f0 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.886+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8815b640 con 0x7f9e881597f0 2023-10-06T05:15:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.886+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e88156510 con 0x7f9e881597f0 2023-10-06T05:15:14.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.887+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8815bb60 con 0x7f9e881597f0 2023-10-06T05:15:14.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.888+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e80012630 con 0x7f9e881597f0 2023-10-06T05:15:14.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.889+0000 7f9e7e7fc700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e6c066f30 0x7f9e6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:14.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.889+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f9e800849b0 con 0x7f9e881597f0 2023-10-06T05:15:14.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.890+0000 7f9e8e59b700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e6c066f30 0x7f9e6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:14.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.891+0000 7f9e8e59b700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e6c066f30 0x7f9e6c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e840011e0 tx=0x7f9e84007c70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:14.894+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e8815bb60 con 0x7f9e881597f0 2023-10-06T05:15:15.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.096+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9e8815bb60 con 0x7f9e881597f0 2023-10-06T05:15:15.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.658+0000 7f9e7e7fc700 1 -- 172.21.15.33:0/3114767231 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9e8815bb60 con 0x7f9e881597f0 2023-10-06T05:15:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.666+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e6c066f30 msgr2=0x7f9e6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.666+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e6c066f30 0x7f9e6c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e840011e0 tx=0x7f9e84007c70 comp rx=0 tx=0).stop 2023-10-06T05:15:15.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.666+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 msgr2=0x7f9e880c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.666+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e880c9a50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c910 tx=0x7f9e80002720 comp rx=0 tx=0).stop 2023-10-06T05:15:15.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 shutdown_connections 2023-10-06T05:15:15.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e6c066f30 0x7f9e6c0693e0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e840011e0 tx=0x7f9e84007c70 comp rx=0 tx=0).stop 2023-10-06T05:15:15.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e881597f0 0x7f9e880c9a50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:15.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e8806dc00 0x7f9e880c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:15.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 --2- 172.21.15.33:0/3114767231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e8806dfd0 0x7f9e880c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:15.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 >> 172.21.15.33:0/3114767231 conn(0x7f9e880d0550 msgr2=0x7f9e880ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:15.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 shutdown_connections 2023-10-06T05:15:15.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.667+0000 7f9e8f59d700 1 -- 172.21.15.33:0/3114767231 wait complete. 2023-10-06T05:15:15.670 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:15:15.691 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:15:15.691 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:15:15.691 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:15:15.692 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:15.692 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:15:15.692 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:15:15.692 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:15:15.692 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:15.692 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:15:15.692 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:15:15.693 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:15:15.693 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:15.693 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:15:15.693 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:15:15.693 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:15:15.693 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:15.706 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:15:15.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:15:15.707 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:15:15.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:15:15.709 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:15:15.709 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:15:15.710 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:15:15.710 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:15:15.751 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:15:15.751 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:15.754 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:15:15.754 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:15.754 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:15:15.755 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:15.757 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:15:15.757 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:15.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:15:15.892 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:15:15.893 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:15:15.901 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:15:15.902 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:15:15.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.958+0000 7fe23f59e700 1 Processor -- start 2023-10-06T05:15:15.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.958+0000 7fe23f59e700 1 -- start start 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.959+0000 7fe23f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe240159e10 0x7fe240157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.959+0000 7fe23f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24006bd10 con 0x7fe2400c8090 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.959+0000 7fe23f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24006a820 con 0x7fe240156880 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.959+0000 7fe23f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400cc290 con 0x7fe240159e10 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.959+0000 7fe23dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe2400c8090 0x7fe240156c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35514/0 (socket says 172.21.15.33:35514) 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.959+0000 7fe23dd9b700 1 -- 172.21.15.33:0/3584388627 learned_addr learned my addr 172.21.15.33:0/3584388627 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:15.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23ed9d700 1 --2- 172.21.15.33:0/3584388627 >> v2:172.21.15.167:3300/0 conn(0x7fe240159e10 0x7fe240157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23d59a700 1 -- 172.21.15.33:0/3584388627 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198134385 0 0) 0x7fe24006bd10 con 0x7fe2400c8090 2023-10-06T05:15:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23d59a700 1 -- 172.21.15.33:0/3584388627 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe228003570 con 0x7fe2400c8090 2023-10-06T05:15:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23ed9d700 1 -- 172.21.15.33:0/3584388627 >> v1:172.21.15.167:6789/0 conn(0x7fe240156880 legacy=0x7fe2400c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23d59a700 1 -- 172.21.15.33:0/3584388627 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153100716 0 0) 0x7fe24006a820 con 0x7fe240156880 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23ed9d700 1 -- 172.21.15.33:0/3584388627 >> v1:172.21.15.33:6789/0 conn(0x7fe2400c8090 legacy=0x7fe240156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23ed9d700 1 -- 172.21.15.33:0/3584388627 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24015a2f0 con 0x7fe240159e10 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.960+0000 7fe23ed9d700 1 --2- 172.21.15.33:0/3584388627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240159e10 0x7fe240157360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe238009e90 tx=0x7fe238009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a885bb7ea59ca81e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23d59a700 1 -- 172.21.15.33:0/3584388627 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800e070 con 0x7fe240159e10 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23d59a700 1 -- 172.21.15.33:0/3584388627 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800bc40 con 0x7fe240159e10 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23d59a700 1 -- 172.21.15.33:0/3584388627 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012430 con 0x7fe240159e10 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 -- 172.21.15.33:0/3584388627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240159e10 msgr2=0x7fe240157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3584388627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240159e10 0x7fe240157360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe238009e90 tx=0x7fe238009780 comp rx=0 tx=0).stop 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 -- 172.21.15.33:0/3584388627 shutdown_connections 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3584388627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240159e10 0x7fe240157360 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 -- 172.21.15.33:0/3584388627 >> 172.21.15.33:0/3584388627 conn(0x7fe24006c560 msgr2=0x7fe24006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 -- 172.21.15.33:0/3584388627 shutdown_connections 2023-10-06T05:15:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.961+0000 7fe23f59e700 1 -- 172.21.15.33:0/3584388627 wait complete. 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.963+0000 7fe23f59e700 1 Processor -- start 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.963+0000 7fe23f59e700 1 -- start start 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2400c8090 0x7fe24014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 0x7fe24014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe240159e10 0x7fe2400b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c3c40 con 0x7fe240156880 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c3f20 con 0x7fe2400c8090 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c4200 con 0x7fe240159e10 2023-10-06T05:15:15.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 0x7fe24014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 0x7fe24014bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59984/0 (socket says 172.21.15.33:59984) 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 -- 172.21.15.33:0/3957631645 learned_addr learned my addr 172.21.15.33:0/3957631645 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23ed9d700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe240159e10 0x7fe2400b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23e59c700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2400c8090 0x7fe24014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 -- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe240159e10 msgr2=0x7fe2400b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe240159e10 0x7fe2400b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 -- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2400c8090 msgr2=0x7fe24014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2400c8090 0x7fe24014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23dd9b700 1 -- 172.21.15.33:0/3957631645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238009a20 con 0x7fe240156880 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.964+0000 7fe23ed9d700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe240159e10 0x7fe2400b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.965+0000 7fe23dd9b700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 0x7fe24014bdd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c560 tx=0x7fe234007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.965+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234005b30 con 0x7fe240156880 2023-10-06T05:15:15.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.965+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234005c90 con 0x7fe240156880 2023-10-06T05:15:15.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.965+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340072e0 con 0x7fe240156880 2023-10-06T05:15:15.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.965+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2400c44e0 con 0x7fe240156880 2023-10-06T05:15:15.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.965+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24014c310 con 0x7fe240156880 2023-10-06T05:15:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.966+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2400ccdc0 con 0x7fe240156880 2023-10-06T05:15:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.967+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe234007690 con 0x7fe240156880 2023-10-06T05:15:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.968+0000 7fe22e7fc700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe220066f30 0x7fe2200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.968+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe234083740 con 0x7fe240156880 2023-10-06T05:15:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.968+0000 7fe23e59c700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe220066f30 0x7fe2200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.969+0000 7fe23e59c700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe220066f30 0x7fe2200693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe230007fd0 tx=0x7fe230007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:15.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:15.972+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe234050ab0 con 0x7fe240156880 2023-10-06T05:15:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.200+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe2400c5f10 con 0x7fe240156880 2023-10-06T05:15:16.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.684+0000 7fe22e7fc700 1 -- 172.21.15.33:0/3957631645 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe234017080 con 0x7fe240156880 2023-10-06T05:15:16.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.686+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe220066f30 msgr2=0x7fe2200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe220066f30 0x7fe2200693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe230007fd0 tx=0x7fe230007d30 comp rx=0 tx=0).stop 2023-10-06T05:15:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 msgr2=0x7fe24014bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 0x7fe24014bdd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c560 tx=0x7fe234007d90 comp rx=0 tx=0).stop 2023-10-06T05:15:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 shutdown_connections 2023-10-06T05:15:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe220066f30 0x7fe2200693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe240159e10 0x7fe2400b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2400c8090 0x7fe24014c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 --2- 172.21.15.33:0/3957631645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe240156880 0x7fe24014bdd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 >> 172.21.15.33:0/3957631645 conn(0x7fe24006c560 msgr2=0x7fe2400c19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:16.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 shutdown_connections 2023-10-06T05:15:16.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.687+0000 7fe23f59e700 1 -- 172.21.15.33:0/3957631645 wait complete. 2023-10-06T05:15:16.691 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:15:16.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:15:16.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.897+0000 7fafdbfff700 1 Processor -- start 2023-10-06T05:15:16.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.897+0000 7fafdbfff700 1 -- start start 2023-10-06T05:15:16.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc158700 0x7fafdc0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafdc0699c0 0x7fafdc069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0ae490 con 0x7fafdc155140 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdbfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc06a380 con 0x7fafdc0699c0 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0c8e40 con 0x7fafdc158700 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafdc0699c0 0x7fafdc069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafdc0699c0 0x7fafdc069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59996/0 (socket says 172.21.15.33:59996) 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdb7fe700 1 -- 172.21.15.33:0/3862435248 learned_addr learned my addr 172.21.15.33:0/3862435248 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafda7fc700 1 --2- 172.21.15.33:0/3862435248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc158700 0x7fafdc0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdb7fe700 1 -- 172.21.15.33:0/3862435248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc158700 msgr2=0x7fafdc0adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.899+0000 7fafdb7fe700 1 --2- 172.21.15.33:0/3862435248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc158700 0x7fafdc0adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.900+0000 7fafdb7fe700 1 -- 172.21.15.33:0/3862435248 >> v1:172.21.15.167:6789/0 conn(0x7fafdc155140 legacy=0x7fafdc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.900+0000 7fafdb7fe700 1 -- 172.21.15.33:0/3862435248 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc158b70 con 0x7fafdc0699c0 2023-10-06T05:15:16.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.900+0000 7fafdb7fe700 1 --2- 172.21.15.33:0/3862435248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc0699c0 0x7fafdc069e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009f20 tx=0x7fafd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7c57cc70e27b51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.900+0000 7fafd9ffb700 1 -- 172.21.15.33:0/3862435248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400e070 con 0x7fafdc0699c0 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.900+0000 7fafd9ffb700 1 -- 172.21.15.33:0/3862435248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400ba80 con 0x7fafdc0699c0 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.900+0000 7fafd9ffb700 1 -- 172.21.15.33:0/3862435248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40092e0 con 0x7fafdc0699c0 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 -- 172.21.15.33:0/3862435248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc0699c0 msgr2=0x7fafdc069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 --2- 172.21.15.33:0/3862435248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc0699c0 0x7fafdc069e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009f20 tx=0x7fafd4009840 comp rx=0 tx=0).stop 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 -- 172.21.15.33:0/3862435248 shutdown_connections 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 --2- 172.21.15.33:0/3862435248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc0699c0 0x7fafdc069e40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 --2- 172.21.15.33:0/3862435248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc158700 0x7fafdc0adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 -- 172.21.15.33:0/3862435248 >> 172.21.15.33:0/3862435248 conn(0x7fafdc06d030 msgr2=0x7fafdc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 -- 172.21.15.33:0/3862435248 shutdown_connections 2023-10-06T05:15:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.901+0000 7fafdbfff700 1 -- 172.21.15.33:0/3862435248 wait complete. 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.902+0000 7fafdbfff700 1 Processor -- start 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.902+0000 7fafdbfff700 1 -- start start 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.903+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc0699c0 0x7fafdc0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.903+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 0x7fafdc159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.903+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc158700 0x7fafdc0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.903+0000 7fafdbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc15a3a0 con 0x7fafdc155140 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.903+0000 7fafdbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b1de0 con 0x7fafdc158700 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.903+0000 7fafdbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b0800 con 0x7fafdc0699c0 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 0x7fafdc159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 0x7fafdc159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59998/0 (socket says 172.21.15.33:59998) 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 -- 172.21.15.33:0/953522599 learned_addr learned my addr 172.21.15.33:0/953522599 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafdaffd700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc0699c0 0x7fafdc0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafdb7fe700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc158700 0x7fafdc0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 -- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc0699c0 msgr2=0x7fafdc0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc0699c0 0x7fafdc0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 -- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc158700 msgr2=0x7fafdc0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc158700 0x7fafdc0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 -- 172.21.15.33:0/953522599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009ae0 con 0x7fafdc155140 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafdaffd700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc0699c0 0x7fafdc0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.904+0000 7fafda7fc700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 0x7fafdc159e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d890 tx=0x7fafd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.905+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd00125f0 con 0x7fafdc155140 2023-10-06T05:15:16.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.905+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0012770 con 0x7fafdc155140 2023-10-06T05:15:16.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.905+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0012b30 con 0x7fafdc155140 2023-10-06T05:15:16.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.905+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc0b0ae0 con 0x7fafdc155140 2023-10-06T05:15:16.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.905+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc0b10a0 con 0x7fafdc155140 2023-10-06T05:15:16.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.906+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc0ae930 con 0x7fafdc155140 2023-10-06T05:15:16.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.907+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafd001e460 con 0x7fafdc155140 2023-10-06T05:15:16.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.908+0000 7fafc2ffd700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafc8066f30 0x7fafc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.908+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fafd0084a90 con 0x7fafdc155140 2023-10-06T05:15:16.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.908+0000 7fafdaffd700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafc8066f30 0x7fafc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.909+0000 7fafdaffd700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafc8066f30 0x7fafc80693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafcc007fd0 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:16.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:16.912+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafd0051e50 con 0x7fafdc155140 2023-10-06T05:15:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.195+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fafdc0c8400 con 0x7fafdc155140 2023-10-06T05:15:17.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.196+0000 7fafc2ffd700 1 -- 172.21.15.33:0/953522599 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fafd0055470 con 0x7fafdc155140 2023-10-06T05:15:17.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:17.197 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:15:17.197 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.199 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:15:17.200 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.203 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:15:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:17.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:15:17.210 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mds", 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWlx9lhEPqMBAAFB0ZkV7V2P+6leHOF+b5mw==", 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWlx9lTk3qMBAAI9xlyuVCL6sh2QJpl63vig==", 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.212 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-osd", 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWlx9lyVXqMBAApJFwXFq39MTXTx/MNvNj1g==", 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWlx9l3F7qMBAAJLEqMGQF4Nccnhc/ccD7yg==", 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWlx9lyWnqMBAAYr/OdTuKePxaghSvawMDMw==", 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWlx9lynLqMBAAy++UpVErRnqIodzNdy+qew==", 2023-10-06T05:15:17.214 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:15:17.216 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:15:17.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafc8066f30 msgr2=0x7fafc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafc8066f30 0x7fafc80693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafcc007fd0 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 msgr2=0x7fafdc159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 0x7fafdc159e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d890 tx=0x7fafd0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 shutdown_connections 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafc8066f30 0x7fafc80693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafdc158700 0x7fafdc0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafdc0699c0 0x7fafdc0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 --2- 172.21.15.33:0/953522599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafdc155140 0x7fafdc159e60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 >> 172.21.15.33:0/953522599 conn(0x7fafdc06d030 msgr2=0x7fafdc0c2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 shutdown_connections 2023-10-06T05:15:17.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.198+0000 7fafdbfff700 1 -- 172.21.15.33:0/953522599 wait complete. 2023-10-06T05:15:17.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T05:15:17.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.450+0000 7f899759e700 1 Processor -- start 2023-10-06T05:15:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.450+0000 7f899759e700 1 -- start start 2023-10-06T05:15:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8998159e10 0x7f8998157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899806bd10 con 0x7f8998156880 2023-10-06T05:15:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899806a820 con 0x7f89980c8090 2023-10-06T05:15:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89980cc290 con 0x7f8998159e10 2023-10-06T05:15:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8998156880 0x7f89980c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35516/0 (socket says 172.21.15.33:35516) 2023-10-06T05:15:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899659c700 1 -- 172.21.15.33:0/2763800641 learned_addr learned my addr 172.21.15.33:0/2763800641 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f8996d9d700 1 --2- 172.21.15.33:0/2763800641 >> v2:172.21.15.33:3300/0 conn(0x7f8998159e10 0x7f8998157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899559a700 1 -- 172.21.15.33:0/2763800641 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154235427 0 0) 0x7f899806bd10 con 0x7f8998156880 2023-10-06T05:15:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f899559a700 1 -- 172.21.15.33:0/2763800641 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8980003570 con 0x7f8998156880 2023-10-06T05:15:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f8996d9d700 1 -- 172.21.15.33:0/2763800641 >> v1:172.21.15.167:6789/0 conn(0x7f89980c8090 legacy=0x7f8998156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f8996d9d700 1 -- 172.21.15.33:0/2763800641 >> v1:172.21.15.33:6789/0 conn(0x7f8998156880 legacy=0x7f89980c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.452+0000 7f8996d9d700 1 -- 172.21.15.33:0/2763800641 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899815a2f0 con 0x7f8998159e10 2023-10-06T05:15:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.453+0000 7f8996d9d700 1 --2- 172.21.15.33:0/2763800641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998159e10 0x7f8998157360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8990009e90 tx=0x7f8990009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbbe618bf0fae61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.453+0000 7f899559a700 1 -- 172.21.15.33:0/2763800641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000e070 con 0x7f8998159e10 2023-10-06T05:15:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.453+0000 7f899559a700 1 -- 172.21.15.33:0/2763800641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000bbb0 con 0x7f8998159e10 2023-10-06T05:15:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.453+0000 7f899559a700 1 -- 172.21.15.33:0/2763800641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990012430 con 0x7f8998159e10 2023-10-06T05:15:17.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.453+0000 7f899759e700 1 -- 172.21.15.33:0/2763800641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998159e10 msgr2=0x7f8998157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.453+0000 7f899759e700 1 --2- 172.21.15.33:0/2763800641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998159e10 0x7f8998157360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8990009e90 tx=0x7f8990009780 comp rx=0 tx=0).stop 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.454+0000 7f899759e700 1 -- 172.21.15.33:0/2763800641 shutdown_connections 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.454+0000 7f899759e700 1 --2- 172.21.15.33:0/2763800641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998159e10 0x7f8998157360 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.454+0000 7f899759e700 1 -- 172.21.15.33:0/2763800641 >> 172.21.15.33:0/2763800641 conn(0x7f899806c560 msgr2=0x7f899806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.454+0000 7f899759e700 1 -- 172.21.15.33:0/2763800641 shutdown_connections 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.454+0000 7f899759e700 1 -- 172.21.15.33:0/2763800641 wait complete. 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.455+0000 7f899759e700 1 Processor -- start 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.455+0000 7f899759e700 1 -- start start 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 0x7f8998151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998156880 0x7f899813d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8998159e10 0x7f8998156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980c3f70 con 0x7f89980c8090 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980c3360 con 0x7f8998156880 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89980c3640 con 0x7f8998159e10 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 0x7f8998151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 0x7f8998151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60024/0 (socket says 172.21.15.33:60024) 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.456+0000 7f899659c700 1 -- 172.21.15.33:0/627091922 learned_addr learned my addr 172.21.15.33:0/627091922 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f8995d9b700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998156880 0x7f899813d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f8996d9d700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8998159e10 0x7f8998156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f899659c700 1 -- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8998159e10 msgr2=0x7f8998156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f899659c700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8998159e10 0x7f8998156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f899659c700 1 -- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998156880 msgr2=0x7f899813d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f899659c700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998156880 0x7f899813d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f899659c700 1 -- 172.21.15.33:0/627091922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990009a20 con 0x7f89980c8090 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f899659c700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 0x7f8998151210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f898800c7c0 tx=0x7f8988007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988005cd0 con 0x7f89980c8090 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.457+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988005e30 con 0x7f89980c8090 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.458+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988007320 con 0x7f89980c8090 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.458+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89980c3920 con 0x7f89980c8090 2023-10-06T05:15:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.458+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8998151750 con 0x7f89980c8090 2023-10-06T05:15:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.459+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89980c86f0 con 0x7f89980c8090 2023-10-06T05:15:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.460+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f898801b3b0 con 0x7f89980c8090 2023-10-06T05:15:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.461+0000 7f89867fc700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8978066f30 0x7f89780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.461+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8988083a80 con 0x7f89980c8090 2023-10-06T05:15:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.461+0000 7f8995d9b700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8978066f30 0x7f89780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.462+0000 7f8995d9b700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8978066f30 0x7f89780693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89981528e0 tx=0x7f898c009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.465+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8988050e40 con 0x7f89980c8090 2023-10-06T05:15:17.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.750+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f899804c380 con 0x7f89980c8090 2023-10-06T05:15:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.754+0000 7f89867fc700 1 -- 172.21.15.33:0/627091922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8988017020 con 0x7f89980c8090 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8978066f30 msgr2=0x7f89780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8978066f30 0x7f89780693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89981528e0 tx=0x7f898c009060 comp rx=0 tx=0).stop 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 msgr2=0x7f8998151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 0x7f8998151210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f898800c7c0 tx=0x7f8988007e00 comp rx=0 tx=0).stop 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 shutdown_connections 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8978066f30 0x7f89780693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8998159e10 0x7f8998156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89980c8090 0x7f8998151210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 --2- 172.21.15.33:0/627091922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8998156880 0x7f899813d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 >> 172.21.15.33:0/627091922 conn(0x7f899806c560 msgr2=0x7f899806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 shutdown_connections 2023-10-06T05:15:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.757+0000 7f899759e700 1 -- 172.21.15.33:0/627091922 wait complete. 2023-10-06T05:15:17.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T05:15:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.963+0000 7ff32739d700 1 Processor -- start 2023-10-06T05:15:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.963+0000 7ff32739d700 1 -- start start 2023-10-06T05:15:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff32739d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff320156880 0x7ff3200c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff32739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff3200c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff32739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff320156c50 0x7ff3201570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff32739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200cc290 con 0x7ff320156c50 2023-10-06T05:15:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff32739d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32006a820 con 0x7ff320156880 2023-10-06T05:15:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff32739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32015a2f0 con 0x7ff320159e10 2023-10-06T05:15:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff326b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff320156c50 0x7ff3201570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.964+0000 7ff326b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff320156c50 0x7ff3201570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60034/0 (socket says 172.21.15.33:60034) 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 -- 172.21.15.33:0/4239183153 learned_addr learned my addr 172.21.15.33:0/4239183153 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff32639b700 1 --2- 172.21.15.33:0/4239183153 >> v2:172.21.15.167:3300/0 conn(0x7ff320156880 0x7ff3200c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff325b9a700 1 --2- 172.21.15.33:0/4239183153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff3200c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 -- 172.21.15.33:0/4239183153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 msgr2=0x7ff3200c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 --2- 172.21.15.33:0/4239183153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff3200c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 -- 172.21.15.33:0/4239183153 >> v2:172.21.15.167:3300/0 conn(0x7ff320156880 msgr2=0x7ff3200c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 --2- 172.21.15.33:0/4239183153 >> v2:172.21.15.167:3300/0 conn(0x7ff320156880 0x7ff3200c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 -- 172.21.15.33:0/4239183153 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32015a5d0 con 0x7ff320156c50 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff32639b700 1 --2- 172.21.15.33:0/4239183153 >> v2:172.21.15.167:3300/0 conn(0x7ff320156880 0x7ff3200c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff326b9c700 1 --2- 172.21.15.33:0/4239183153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156c50 0x7ff3201570b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff32006dbe0 tx=0x7ff318009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18de24a5dcf7e525 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff325399700 1 -- 172.21.15.33:0/4239183153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800e070 con 0x7ff320156c50 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.965+0000 7ff325399700 1 -- 172.21.15.33:0/4239183153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800ba50 con 0x7ff320156c50 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff325399700 1 -- 172.21.15.33:0/4239183153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3180092e0 con 0x7ff320156c50 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 -- 172.21.15.33:0/4239183153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156c50 msgr2=0x7ff3201570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 --2- 172.21.15.33:0/4239183153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156c50 0x7ff3201570b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff32006dbe0 tx=0x7ff318009840 comp rx=0 tx=0).stop 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 -- 172.21.15.33:0/4239183153 shutdown_connections 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 --2- 172.21.15.33:0/4239183153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156c50 0x7ff3201570b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 --2- 172.21.15.33:0/4239183153 >> v2:172.21.15.167:3300/0 conn(0x7ff320156880 0x7ff3200c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 --2- 172.21.15.33:0/4239183153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff3200c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 -- 172.21.15.33:0/4239183153 >> 172.21.15.33:0/4239183153 conn(0x7ff32006c560 msgr2=0x7ff32006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 -- 172.21.15.33:0/4239183153 shutdown_connections 2023-10-06T05:15:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.966+0000 7ff32739d700 1 -- 172.21.15.33:0/4239183153 wait complete. 2023-10-06T05:15:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.968+0000 7ff32739d700 1 Processor -- start 2023-10-06T05:15:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.968+0000 7ff32739d700 1 -- start start 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 0x7ff320144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff320156c50 0x7ff32014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff32014c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32014fc40 con 0x7ff320156880 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320169d90 con 0x7ff320156c50 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320169460 con 0x7ff320159e10 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 0x7ff320144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 0x7ff320144530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60036/0 (socket says 172.21.15.33:60036) 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 -- 172.21.15.33:0/2456507723 learned_addr learned my addr 172.21.15.33:0/2456507723 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff326b9c700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff32014c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 -- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 msgr2=0x7ff32014c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff325b9a700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff320156c50 0x7ff32014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff32014c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 -- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff320156c50 msgr2=0x7ff32014f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff320156c50 0x7ff32014f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff32639b700 1 -- 172.21.15.33:0/2456507723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318009ae0 con 0x7ff320156880 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff325b9a700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff320156c50 0x7ff32014f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.969+0000 7ff326b9c700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff32014c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.970+0000 7ff32639b700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 0x7ff320144530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00d860 tx=0x7ff31c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.970+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0125f0 con 0x7ff320156880 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.970+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c012750 con 0x7ff320156880 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.970+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c012ad0 con 0x7ff320156880 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.970+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320169740 con 0x7ff320156880 2023-10-06T05:15:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.970+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32016a220 con 0x7ff320156880 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.971+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32004c380 con 0x7ff320156880 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.972+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff31c014430 con 0x7ff320156880 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.973+0000 7ff3167fc700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff304066f30 0x7ff3040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.973+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff31c084830 con 0x7ff320156880 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.973+0000 7ff325b9a700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff304066f30 0x7ff3040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.974+0000 7ff325b9a700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff304066f30 0x7ff3040693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3200c8880 tx=0x7ff310008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:17.977+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff31c055680 con 0x7ff320156880 2023-10-06T05:15:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.261+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff320169be0 con 0x7ff320156880 2023-10-06T05:15:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.265+0000 7ff3167fc700 1 -- 172.21.15.33:0/2456507723 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff31c055210 con 0x7ff320156880 2023-10-06T05:15:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.267+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff304066f30 msgr2=0x7ff3040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.267+0000 7ff32739d700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff304066f30 0x7ff3040693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3200c8880 tx=0x7ff310008040 comp rx=0 tx=0).stop 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.267+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 msgr2=0x7ff320144530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.267+0000 7ff32739d700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 0x7ff320144530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00d860 tx=0x7ff31c008040 comp rx=0 tx=0).stop 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 shutdown_connections 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff304066f30 0x7ff3040693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff320159e10 0x7ff32014c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff320156880 0x7ff320144530 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 --2- 172.21.15.33:0/2456507723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff320156c50 0x7ff32014f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 >> 172.21.15.33:0/2456507723 conn(0x7ff32006c560 msgr2=0x7ff3200c2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 shutdown_connections 2023-10-06T05:15:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.268+0000 7ff32739d700 1 -- 172.21.15.33:0/2456507723 wait complete. 2023-10-06T05:15:18.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T05:15:18.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.472+0000 7fd6fffff700 1 Processor -- start 2023-10-06T05:15:18.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.473+0000 7fd6fffff700 1 -- start start 2023-10-06T05:15:18.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6fffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7001592e0 0x7fd7000cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6fffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd70006c560 0x7fd7000aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6fffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7000b03a0 con 0x7fd7001596b0 2023-10-06T05:15:18.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6fffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7000b0f40 con 0x7fd7001592e0 2023-10-06T05:15:18.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6fffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70006ccd0 con 0x7fd70006c560 2023-10-06T05:15:18.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6feffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7001592e0 0x7fd7000cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6feffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7001592e0 0x7fd7000cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60042/0 (socket says 172.21.15.33:60042) 2023-10-06T05:15:18.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6feffd700 1 -- 172.21.15.33:0/1626988416 learned_addr learned my addr 172.21.15.33:0/1626988416 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6ff7fe700 1 --2- 172.21.15.33:0/1626988416 >> v2:172.21.15.167:3300/0 conn(0x7fd70006c560 0x7fd7000aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6feffd700 1 -- 172.21.15.33:0/1626988416 >> v2:172.21.15.167:3300/0 conn(0x7fd70006c560 msgr2=0x7fd7000aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.474+0000 7fd6feffd700 1 --2- 172.21.15.33:0/1626988416 >> v2:172.21.15.167:3300/0 conn(0x7fd70006c560 0x7fd7000aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.475+0000 7fd6feffd700 1 -- 172.21.15.33:0/1626988416 >> v1:172.21.15.167:6789/0 conn(0x7fd7001596b0 legacy=0x7fd7000b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.475+0000 7fd6feffd700 1 -- 172.21.15.33:0/1626988416 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70006cfb0 con 0x7fd7001592e0 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.475+0000 7fd6feffd700 1 --2- 172.21.15.33:0/1626988416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7001592e0 0x7fd7000cdd30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7000c9880 tx=0x7fd6f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbb5ff4fa468928 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.475+0000 7fd6fdffb700 1 -- 172.21.15.33:0/1626988416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000b860 con 0x7fd7001592e0 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.475+0000 7fd6fdffb700 1 -- 172.21.15.33:0/1626988416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000b9c0 con 0x7fd7001592e0 2023-10-06T05:15:18.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.475+0000 7fd6fdffb700 1 -- 172.21.15.33:0/1626988416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000bd60 con 0x7fd7001592e0 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 -- 172.21.15.33:0/1626988416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7001592e0 msgr2=0x7fd7000cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 --2- 172.21.15.33:0/1626988416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7001592e0 0x7fd7000cdd30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7000c9880 tx=0x7fd6f0009250 comp rx=0 tx=0).stop 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 -- 172.21.15.33:0/1626988416 shutdown_connections 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 --2- 172.21.15.33:0/1626988416 >> v2:172.21.15.167:3300/0 conn(0x7fd70006c560 0x7fd7000aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 --2- 172.21.15.33:0/1626988416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7001592e0 0x7fd7000cdd30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 -- 172.21.15.33:0/1626988416 >> 172.21.15.33:0/1626988416 conn(0x7fd7000675f0 msgr2=0x7fd7000679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 -- 172.21.15.33:0/1626988416 shutdown_connections 2023-10-06T05:15:18.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.476+0000 7fd6fffff700 1 -- 172.21.15.33:0/1626988416 wait complete. 2023-10-06T05:15:18.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.477+0000 7fd6fffff700 1 Processor -- start 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.477+0000 7fd6fffff700 1 -- start start 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 0x7fd7000aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6fffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7001592e0 0x7fd700157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6fffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7001596b0 0x7fd7001530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6fffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700156080 con 0x7fd70006c560 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6fffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70015c0f0 con 0x7fd7001592e0 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6fffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70015b4e0 con 0x7fd7001596b0 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.478+0000 7fd6feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 0x7fd7000aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 0x7fd7000aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60044/0 (socket says 172.21.15.33:60044) 2023-10-06T05:15:18.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 -- 172.21.15.33:0/2460347399 learned_addr learned my addr 172.21.15.33:0/2460347399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6fe7fc700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7001592e0 0x7fd700157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6ff7fe700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7001596b0 0x7fd7001530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 -- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7001596b0 msgr2=0x7fd7001530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7001596b0 0x7fd7001530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 -- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7001592e0 msgr2=0x7fd700157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7001592e0 0x7fd700157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 -- 172.21.15.33:0/2460347399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00094f0 con 0x7fd70006c560 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6ff7fe700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7001596b0 0x7fd7001530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6fe7fc700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7001592e0 0x7fd700157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6feffd700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 0x7fd7000aad10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd700069eb0 tx=0x7fd6f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0011e60 con 0x7fd70006c560 2023-10-06T05:15:18.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f001a630 con 0x7fd70006c560 2023-10-06T05:15:18.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.479+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70015b7c0 con 0x7fd70006c560 2023-10-06T05:15:18.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.480+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f001a9f0 con 0x7fd70006c560 2023-10-06T05:15:18.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.480+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70015bd80 con 0x7fd70006c560 2023-10-06T05:15:18.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.481+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70004c380 con 0x7fd70006c560 2023-10-06T05:15:18.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.481+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6f002d440 con 0x7fd70006c560 2023-10-06T05:15:18.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.482+0000 7fd6e6ffd700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.482+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd6f0085f20 con 0x7fd70006c560 2023-10-06T05:15:18.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.483+0000 7fd6fe7fc700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.484+0000 7fd6fe7fc700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6ec066f30 0x7fd6ec0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd70014e930 tx=0x7fd6f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.486+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6f00532e0 con 0x7fd70006c560 2023-10-06T05:15:18.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.770+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fd70006d290 con 0x7fd70006c560 2023-10-06T05:15:18.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.773+0000 7fd6e6ffd700 1 -- 172.21.15.33:0/2460347399 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd6f0056900 con 0x7fd70006c560 2023-10-06T05:15:18.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6ec066f30 msgr2=0x7fd6ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6ec066f30 0x7fd6ec0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd70014e930 tx=0x7fd6f4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:18.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 msgr2=0x7fd7000aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 0x7fd7000aad10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd700069eb0 tx=0x7fd6f0017000 comp rx=0 tx=0).stop 2023-10-06T05:15:18.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 shutdown_connections 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7001596b0 0x7fd7001530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd70006c560 0x7fd7000aad10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 --2- 172.21.15.33:0/2460347399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7001592e0 0x7fd700157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 >> 172.21.15.33:0/2460347399 conn(0x7fd7000675f0 msgr2=0x7fd7000aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 shutdown_connections 2023-10-06T05:15:18.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.775+0000 7fd6fffff700 1 -- 172.21.15.33:0/2460347399 wait complete. 2023-10-06T05:15:18.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T05:15:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.980+0000 7f4ad499d700 1 Processor -- start 2023-10-06T05:15:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.980+0000 7f4ad499d700 1 -- start start 2023-10-06T05:15:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4ad499d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ad0159a50 0x7f4ad00c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4ad499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad00c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4ad499d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad00a9ac0 con 0x7f4ad0159680 2023-10-06T05:15:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4ad499d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad00c7d50 con 0x7f4ad0159a50 2023-10-06T05:15:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4ad499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00caa10 con 0x7f4ad00699c0 2023-10-06T05:15:18.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4aceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ad0159a50 0x7f4ad00c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4aceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ad0159a50 0x7f4ad00c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60060/0 (socket says 172.21.15.33:60060) 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4aceffd700 1 -- 172.21.15.33:0/3146159788 learned_addr learned my addr 172.21.15.33:0/3146159788 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4acffff700 1 --2- 172.21.15.33:0/3146159788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad00c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4aceffd700 1 -- 172.21.15.33:0/3146159788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 msgr2=0x7f4ad00c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4aceffd700 1 --2- 172.21.15.33:0/3146159788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad00c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.982+0000 7f4aceffd700 1 -- 172.21.15.33:0/3146159788 >> v1:172.21.15.167:6789/0 conn(0x7f4ad0159680 legacy=0x7f4ad00c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.983+0000 7f4aceffd700 1 -- 172.21.15.33:0/3146159788 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00c74b0 con 0x7f4ad0159a50 2023-10-06T05:15:18.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.983+0000 7f4aceffd700 1 --2- 172.21.15.33:0/3146159788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159a50 0x7f4ad00c8b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40099e0 tx=0x7f4ac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee20d4382219c695 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.983+0000 7f4ace7fc700 1 -- 172.21.15.33:0/3146159788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400b820 con 0x7f4ad0159a50 2023-10-06T05:15:18.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.983+0000 7f4ace7fc700 1 -- 172.21.15.33:0/3146159788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac400b980 con 0x7f4ad0159a50 2023-10-06T05:15:18.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.983+0000 7f4ace7fc700 1 -- 172.21.15.33:0/3146159788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400bd20 con 0x7f4ad0159a50 2023-10-06T05:15:18.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 -- 172.21.15.33:0/3146159788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159a50 msgr2=0x7f4ad00c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 --2- 172.21.15.33:0/3146159788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159a50 0x7f4ad00c8b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40099e0 tx=0x7f4ac4009250 comp rx=0 tx=0).stop 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 -- 172.21.15.33:0/3146159788 shutdown_connections 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 --2- 172.21.15.33:0/3146159788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad00c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 --2- 172.21.15.33:0/3146159788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159a50 0x7f4ad00c8b10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 -- 172.21.15.33:0/3146159788 >> 172.21.15.33:0/3146159788 conn(0x7f4ad006c560 msgr2=0x7f4ad006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 -- 172.21.15.33:0/3146159788 shutdown_connections 2023-10-06T05:15:18.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.984+0000 7f4ad499d700 1 -- 172.21.15.33:0/3146159788 wait complete. 2023-10-06T05:15:18.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.985+0000 7f4ad499d700 1 Processor -- start 2023-10-06T05:15:18.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.985+0000 7f4ad499d700 1 -- start start 2023-10-06T05:15:18.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4ad499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad0144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4ad499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 0x7f4ad0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4ad499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ad0159a50 0x7f4ad015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4ad499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0156930 con 0x7f4ad0159680 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4ad499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0145100 con 0x7f4ad0159a50 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4ad499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0155d20 con 0x7f4ad00699c0 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4aceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 0x7f4ad0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4aceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 0x7f4ad0157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60062/0 (socket says 172.21.15.33:60062) 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.986+0000 7f4aceffd700 1 -- 172.21.15.33:0/2098389794 learned_addr learned my addr 172.21.15.33:0/2098389794 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4acffff700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ad0159a50 0x7f4ad015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4acf7fe700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad0144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4aceffd700 1 -- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 msgr2=0x7f4ad0144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4aceffd700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad0144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4aceffd700 1 -- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ad0159a50 msgr2=0x7f4ad015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4aceffd700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ad0159a50 0x7f4ad015e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4aceffd700 1 -- 172.21.15.33:0/2098389794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40094f0 con 0x7f4ad0159680 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4acf7fe700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad0144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4aceffd700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 0x7f4ad0157660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009290 tx=0x7f4ac4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4011e20 con 0x7f4ad0159680 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4024440 con 0x7f4ad0159680 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.987+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4024800 con 0x7f4ad0159680 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.988+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad0156000 con 0x7f4ad0159680 2023-10-06T05:15:18.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.988+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad01565c0 con 0x7f4ad0159680 2023-10-06T05:15:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.988+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0156aa0 con 0x7f4ad0159680 2023-10-06T05:15:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.989+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ac402d440 con 0x7f4ad0159680 2023-10-06T05:15:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.990+0000 7f4ab77fe700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4abc066f30 0x7f4abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.990+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4ac4085d40 con 0x7f4ad0159680 2023-10-06T05:15:18.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.991+0000 7f4acf7fe700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4abc066f30 0x7f4abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.991+0000 7f4acf7fe700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4abc066f30 0x7f4abc0693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0007fd0 tx=0x7f4ac0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:18.995+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ac4053100 con 0x7f4ad0159680 2023-10-06T05:15:19.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.277+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f4ad004c380 con 0x7f4ad0159680 2023-10-06T05:15:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.281+0000 7f4ab77fe700 1 -- 172.21.15.33:0/2098389794 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4ac4056720 con 0x7f4ad0159680 2023-10-06T05:15:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4abc066f30 msgr2=0x7f4abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4abc066f30 0x7f4abc0693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0007fd0 tx=0x7f4ac0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 msgr2=0x7f4ad0157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 0x7f4ad0157660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009290 tx=0x7f4ac4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 shutdown_connections 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4abc066f30 0x7f4abc0693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ad0159a50 0x7f4ad015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ad00699c0 0x7f4ad0144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 --2- 172.21.15.33:0/2098389794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad0159680 0x7f4ad0157660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 >> 172.21.15.33:0/2098389794 conn(0x7f4ad006c560 msgr2=0x7f4ad015fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 shutdown_connections 2023-10-06T05:15:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.283+0000 7f4ad499d700 1 -- 172.21.15.33:0/2098389794 wait complete. 2023-10-06T05:15:19.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T05:15:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.489+0000 7f269619d700 1 Processor -- start 2023-10-06T05:15:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.489+0000 7f269619d700 1 -- start start 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.490+0000 7f269619d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2690159400 0x7f26900cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.490+0000 7f269619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f26900cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.490+0000 7f269619d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900d0ed0 con 0x7f26901597d0 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.490+0000 7f269619d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26900b1160 con 0x7f2690159400 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.490+0000 7f269619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900cb6d0 con 0x7f269006cc70 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2690159400 0x7f26900cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2690159400 0x7f26900cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60072/0 (socket says 172.21.15.33:60072) 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 -- 172.21.15.33:0/3263430158 learned_addr learned my addr 172.21.15.33:0/3263430158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269599c700 1 --2- 172.21.15.33:0/3263430158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f26900cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 -- 172.21.15.33:0/3263430158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 msgr2=0x7f26900cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 --2- 172.21.15.33:0/3263430158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f26900cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 -- 172.21.15.33:0/3263430158 >> v1:172.21.15.167:6789/0 conn(0x7f26901597d0 legacy=0x7f269006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 -- 172.21.15.33:0/3263430158 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26900cb9b0 con 0x7f2690159400 2023-10-06T05:15:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.491+0000 7f269519b700 1 --2- 172.21.15.33:0/3263430158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f26900cab60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f268c0099b0 tx=0x7f268c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f878472e5f352ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f2687fff700 1 -- 172.21.15.33:0/3263430158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00b860 con 0x7f2690159400 2023-10-06T05:15:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f2687fff700 1 -- 172.21.15.33:0/3263430158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c00b9c0 con 0x7f2690159400 2023-10-06T05:15:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f2687fff700 1 -- 172.21.15.33:0/3263430158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00bd60 con 0x7f2690159400 2023-10-06T05:15:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 -- 172.21.15.33:0/3263430158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 msgr2=0x7f26900cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 --2- 172.21.15.33:0/3263430158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f26900cab60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f268c0099b0 tx=0x7f268c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 -- 172.21.15.33:0/3263430158 shutdown_connections 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 --2- 172.21.15.33:0/3263430158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f26900cab60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 --2- 172.21.15.33:0/3263430158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f26900cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 -- 172.21.15.33:0/3263430158 >> 172.21.15.33:0/3263430158 conn(0x7f269006e4b0 msgr2=0x7f26900d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 -- 172.21.15.33:0/3263430158 shutdown_connections 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.492+0000 7f269619d700 1 -- 172.21.15.33:0/3263430158 wait complete. 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.494+0000 7f269619d700 1 Processor -- start 2023-10-06T05:15:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.494+0000 7f269619d700 1 -- start start 2023-10-06T05:15:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f269016a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269619d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f269015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269619d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26901597d0 0x7f26900c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269619d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269016aea0 con 0x7f2690159400 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269619d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26900c9820 con 0x7f26901597d0 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900c8ef0 con 0x7f269006cc70 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f269015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f269015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60082/0 (socket says 172.21.15.33:60082) 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269499a700 1 -- 172.21.15.33:0/3227841899 learned_addr learned my addr 172.21.15.33:0/3227841899 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269519b700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f269016a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.495+0000 7f269599c700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26901597d0 0x7f26900c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269499a700 1 -- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 msgr2=0x7f269016a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269499a700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f269016a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269499a700 1 -- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26901597d0 msgr2=0x7f26900c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269499a700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26901597d0 0x7f26900c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269499a700 1 -- 172.21.15.33:0/3227841899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0094f0 con 0x7f2690159400 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269499a700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f269015d380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f268000c4e0 tx=0x7f268000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680011590 con 0x7f2690159400 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2680011710 con 0x7f2690159400 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680011ad0 con 0x7f2690159400 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26900c91d0 con 0x7f2690159400 2023-10-06T05:15:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.496+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269015d8c0 con 0x7f2690159400 2023-10-06T05:15:19.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.497+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26900c99b0 con 0x7f2690159400 2023-10-06T05:15:19.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.498+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f268001b3b0 con 0x7f2690159400 2023-10-06T05:15:19.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.499+0000 7f26857fa700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.499+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f26800837a0 con 0x7f2690159400 2023-10-06T05:15:19.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.500+0000 7f269519b700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.501+0000 7f269519b700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f268c0099b0 tx=0x7f268c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.504+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2680050b60 con 0x7f2690159400 2023-10-06T05:15:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.785+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f269004c380 con 0x7f2690159400 2023-10-06T05:15:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.789+0000 7f26857fa700 1 -- 172.21.15.33:0/3227841899 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2680017020 con 0x7f2690159400 2023-10-06T05:15:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 msgr2=0x7f26740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f268c0099b0 tx=0x7f268c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 msgr2=0x7f269015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f269015d380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f268000c4e0 tx=0x7f268000c7f0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 shutdown_connections 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26901597d0 0x7f26900c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.791+0000 7f269619d700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f269006cc70 0x7f269016a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.792+0000 7f269619d700 1 --2- 172.21.15.33:0/3227841899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2690159400 0x7f269015d380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.792+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 >> 172.21.15.33:0/3227841899 conn(0x7f269006e4b0 msgr2=0x7f2690152bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.792+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 shutdown_connections 2023-10-06T05:15:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.792+0000 7f269619d700 1 -- 172.21.15.33:0/3227841899 wait complete. 2023-10-06T05:15:19.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T05:15:20.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.998+0000 7fdefd79d700 1 Processor -- start 2023-10-06T05:15:20.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:19.998+0000 7fdefd79d700 1 -- start start 2023-10-06T05:15:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdefd79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdef81557f0 0x7fdef80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdefd79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8155bc0 0x7fdef80c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdefd79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef806a410 con 0x7fdef8157150 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdefd79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80cccd0 con 0x7fdef81557f0 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdefd79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80c7500 con 0x7fdef8155bc0 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdef81557f0 0x7fdef80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdef81557f0 0x7fdef80c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51956/0 (socket says 172.21.15.33:51956) 2023-10-06T05:15:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 -- 172.21.15.33:0/2203994660 learned_addr learned my addr 172.21.15.33:0/2203994660 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:20.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef77fe700 1 --2- 172.21.15.33:0/2203994660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8155bc0 0x7fdef80c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 -- 172.21.15.33:0/2203994660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8155bc0 msgr2=0x7fdef80c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 --2- 172.21.15.33:0/2203994660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8155bc0 0x7fdef80c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 -- 172.21.15.33:0/2203994660 >> v1:172.21.15.167:6789/0 conn(0x7fdef8157150 legacy=0x7fdef80c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.000+0000 7fdef7fff700 1 -- 172.21.15.33:0/2203994660 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80c77e0 con 0x7fdef81557f0 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.001+0000 7fdef7fff700 1 --2- 172.21.15.33:0/2203994660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef81557f0 0x7fdef80c6960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdee80099b0 tx=0x7fdee8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f15cf662574e9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.001+0000 7fdef6ffd700 1 -- 172.21.15.33:0/2203994660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8009ea0 con 0x7fdef81557f0 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.001+0000 7fdef6ffd700 1 -- 172.21.15.33:0/2203994660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee800b860 con 0x7fdef81557f0 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.001+0000 7fdef6ffd700 1 -- 172.21.15.33:0/2203994660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee800bc00 con 0x7fdef81557f0 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.001+0000 7fdefd79d700 1 -- 172.21.15.33:0/2203994660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef81557f0 msgr2=0x7fdef80c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.001+0000 7fdefd79d700 1 --2- 172.21.15.33:0/2203994660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef81557f0 0x7fdef80c6960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdee80099b0 tx=0x7fdee8009250 comp rx=0 tx=0).stop 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.002+0000 7fdefd79d700 1 -- 172.21.15.33:0/2203994660 shutdown_connections 2023-10-06T05:15:20.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.002+0000 7fdefd79d700 1 --2- 172.21.15.33:0/2203994660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef81557f0 0x7fdef80c6960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.002+0000 7fdefd79d700 1 --2- 172.21.15.33:0/2203994660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8155bc0 0x7fdef80c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.002+0000 7fdefd79d700 1 -- 172.21.15.33:0/2203994660 >> 172.21.15.33:0/2203994660 conn(0x7fdef806c560 msgr2=0x7fdef806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.002+0000 7fdefd79d700 1 -- 172.21.15.33:0/2203994660 shutdown_connections 2023-10-06T05:15:20.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.002+0000 7fdefd79d700 1 -- 172.21.15.33:0/2203994660 wait complete. 2023-10-06T05:15:20.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.003+0000 7fdefd79d700 1 Processor -- start 2023-10-06T05:15:20.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.003+0000 7fdefd79d700 1 -- start start 2023-10-06T05:15:20.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdefd79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdef81557f0 0x7fdef8149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdefd79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 0x7fdef814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdefd79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8157150 0x7fdef814b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdefd79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80ca630 con 0x7fdef8155bc0 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdefd79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80ca2e0 con 0x7fdef81557f0 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdefd79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8169a90 con 0x7fdef8157150 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.004+0000 7fdef77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 0x7fdef814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 0x7fdef814acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51964/0 (socket says 172.21.15.33:51964) 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 -- 172.21.15.33:0/1399731514 learned_addr learned my addr 172.21.15.33:0/1399731514 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef7fff700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdef81557f0 0x7fdef8149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdefcf9c700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8157150 0x7fdef814b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 -- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8157150 msgr2=0x7fdef814b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8157150 0x7fdef814b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 -- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdef81557f0 msgr2=0x7fdef8149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdef81557f0 0x7fdef8149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 -- 172.21.15.33:0/1399731514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80094f0 con 0x7fdef8155bc0 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdefcf9c700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8157150 0x7fdef814b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdef77fe700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 0x7fdef814acf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00ec80 tx=0x7fdeec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00c9d0 con 0x7fdef8155bc0 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.005+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00cb30 con 0x7fdef8155bc0 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.006+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef814b860 con 0x7fdef8155bc0 2023-10-06T05:15:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.006+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec013470 con 0x7fdef8155bc0 2023-10-06T05:15:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.006+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef814be20 con 0x7fdef8155bc0 2023-10-06T05:15:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.007+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef80ca450 con 0x7fdef8155bc0 2023-10-06T05:15:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.007+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdeec013820 con 0x7fdef8155bc0 2023-10-06T05:15:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.008+0000 7fdedffff700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee4066f30 0x7fdee40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.008+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdeec0847d0 con 0x7fdef8155bc0 2023-10-06T05:15:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.009+0000 7fdef7fff700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee4066f30 0x7fdee40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.009+0000 7fdef7fff700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee4066f30 0x7fdee40693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdee80099b0 tx=0x7fdee8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.012+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeec051b90 con 0x7fdef8155bc0 2023-10-06T05:15:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.293+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fdef804c380 con 0x7fdef8155bc0 2023-10-06T05:15:20.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.298+0000 7fdedffff700 1 -- 172.21.15.33:0/1399731514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdeec0551b0 con 0x7fdef8155bc0 2023-10-06T05:15:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee4066f30 msgr2=0x7fdee40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee4066f30 0x7fdee40693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdee80099b0 tx=0x7fdee8017000 comp rx=0 tx=0).stop 2023-10-06T05:15:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 msgr2=0x7fdef814acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 0x7fdef814acf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00ec80 tx=0x7fdeec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 shutdown_connections 2023-10-06T05:15:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdee4066f30 0x7fdee40693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdef8157150 0x7fdef814b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdef81557f0 0x7fdef8149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 --2- 172.21.15.33:0/1399731514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef8155bc0 0x7fdef814acf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 >> 172.21.15.33:0/1399731514 conn(0x7fdef806c560 msgr2=0x7fdef806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 shutdown_connections 2023-10-06T05:15:20.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.300+0000 7fdefd79d700 1 -- 172.21.15.33:0/1399731514 wait complete. 2023-10-06T05:15:20.322 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:20.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.507+0000 7f81b2f9d700 1 Processor -- start 2023-10-06T05:15:20.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.507+0000 7f81b2f9d700 1 -- start start 2023-10-06T05:15:20.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b2f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f81ac0cb710 0x7f81ac06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b2f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0c9cb0 con 0x7f81ac13faa0 2023-10-06T05:15:20.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b2f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0ca270 con 0x7f81ac0c8050 2023-10-06T05:15:20.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b2f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac06b840 con 0x7f81ac0cb710 2023-10-06T05:15:20.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b1f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81ac13faa0 0x7f81ac0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59952/0 (socket says 172.21.15.33:59952) 2023-10-06T05:15:20.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b1f9b700 1 -- 172.21.15.33:0/2916975085 learned_addr learned my addr 172.21.15.33:0/2916975085 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:20.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b279c700 1 --2- 172.21.15.33:0/2916975085 >> v2:172.21.15.167:3300/0 conn(0x7f81ac0cb710 0x7f81ac06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934512287 0 0) 0x7f81ac0c9cb0 con 0x7f81ac13faa0 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.509+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8198003570 con 0x7f81ac13faa0 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 302306737 0 0) 0x7f8198003570 con 0x7f81ac13faa0 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 >> v2:172.21.15.167:3300/0 conn(0x7f81ac0cb710 msgr2=0x7f81ac06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 --2- 172.21.15.33:0/2916975085 >> v2:172.21.15.167:3300/0 conn(0x7f81ac0cb710 0x7f81ac06b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 >> v1:172.21.15.167:6789/0 conn(0x7f81ac0c8050 legacy=0x7f81ac06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac06bb20 con 0x7f81ac13faa0 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f81a8001470 con 0x7f81ac13faa0 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 >> v1:172.21.15.33:6789/0 conn(0x7f81ac13faa0 legacy=0x7f81ac0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8198003a60 0x7f8198003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8198004810 0x7f8198004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81980051c0 0x7f819800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac06b840 con 0x7f8198003a60 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819800c0d0 con 0x7f81980051c0 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819800c290 con 0x7f8198004810 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8198003a60 0x7f8198003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b279c700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81980051c0 0x7f819800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.510+0000 7f81b179a700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8198004810 0x7f8198004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b1f9b700 1 -- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8198004810 msgr2=0x7f8198004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8198004810 0x7f8198004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b1f9b700 1 -- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81980051c0 msgr2=0x7f819800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81980051c0 0x7f819800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b1f9b700 1 -- 172.21.15.33:0/2916975085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198003570 con 0x7f8198003a60 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b279c700 1 -- 172.21.15.33:0/2916975085 reap_dead start 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b179a700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8198004810 0x7f8198004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8198003a60 0x7f8198003e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c390 tx=0x7f81ac0ca1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14812442031dd49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800cc60 con 0x7f8198003a60 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800cdc0 con 0x7f8198003a60 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.511+0000 7f81b0f99700 1 -- 172.21.15.33:0/2916975085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8005b70 con 0x7f8198003a60 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2916975085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8198003a60 msgr2=0x7f8198003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8198003a60 0x7f8198003e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c390 tx=0x7f81ac0ca1e0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2916975085 shutdown_connections 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2916975085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8198003a60 0x7f8198003e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2916975085 >> 172.21.15.33:0/2916975085 conn(0x7f81ac06d030 msgr2=0x7f81ac06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2916975085 shutdown_connections 2023-10-06T05:15:20.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.512+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2916975085 wait complete. 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.513+0000 7f81b2f9d700 1 Processor -- start 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.513+0000 7f81b2f9d700 1 -- start start 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8198003a60 0x7f81ac0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b2f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 0x7f81ac157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81ac0c5b30 0x7f81ac157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b2f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac156420 con 0x7f81ac0c36d0 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac157580 con 0x7f8198003a60 2023-10-06T05:15:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0c3b70 con 0x7f81ac0c5b30 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.514+0000 7f81b179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 0x7f81ac157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 0x7f81ac157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51976/0 (socket says 172.21.15.33:51976) 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 -- 172.21.15.33:0/2915928296 learned_addr learned my addr 172.21.15.33:0/2915928296 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8198003a60 0x7f81ac0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b279c700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81ac0c5b30 0x7f81ac157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 -- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81ac0c5b30 msgr2=0x7f81ac157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81ac0c5b30 0x7f81ac157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 -- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8198003a60 msgr2=0x7f81ac0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8198003a60 0x7f81ac0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 -- 172.21.15.33:0/2915928296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac0c3e50 con 0x7f81ac0c36d0 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81b179a700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 0x7f81ac157a10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f819c000c00 tx=0x7f819c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.515+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00ee60 con 0x7f81ac0c36d0 2023-10-06T05:15:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.516+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c0114f0 con 0x7f81ac0c36d0 2023-10-06T05:15:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.516+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0c4130 con 0x7f81ac0c36d0 2023-10-06T05:15:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.516+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0118b0 con 0x7f81ac0c36d0 2023-10-06T05:15:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.516+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac159970 con 0x7f81ac0c36d0 2023-10-06T05:15:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.517+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac159b60 con 0x7f81ac0c36d0 2023-10-06T05:15:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.518+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f819c011c20 con 0x7f81ac0c36d0 2023-10-06T05:15:20.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.518+0000 7f81a1ffb700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.519+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f819c083c10 con 0x7f81ac0c36d0 2023-10-06T05:15:20.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.519+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.520+0000 7f81b1f9b700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8190066f30 0x7f81900693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007cd0 tx=0x7f81a8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.522+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f819c04d5a0 con 0x7f81ac0c36d0 2023-10-06T05:15:20.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.726+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81ac04c380 con 0x7f81ac0c36d0 2023-10-06T05:15:20.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.727+0000 7f81a1ffb700 1 -- 172.21.15.33:0/2915928296 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+13650 (secure 0 0 0) 0x7f819c019020 con 0x7f81ac0c36d0 2023-10-06T05:15:20.730 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:20.730 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:16.692391+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:20.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8190066f30 msgr2=0x7f81900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8190066f30 0x7f81900693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007cd0 tx=0x7f81a8015000 comp rx=0 tx=0).stop 2023-10-06T05:15:20.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 msgr2=0x7f81ac157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 0x7f81ac157a10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f819c000c00 tx=0x7f819c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 shutdown_connections 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81ac0c5b30 0x7f81ac157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8198003a60 0x7f81ac0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 --2- 172.21.15.33:0/2915928296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ac0c36d0 0x7f81ac157a10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 >> 172.21.15.33:0/2915928296 conn(0x7f81ac06d030 msgr2=0x7f81ac06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 shutdown_connections 2023-10-06T05:15:20.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.729+0000 7f81b2f9d700 1 -- 172.21.15.33:0/2915928296 wait complete. 2023-10-06T05:15:20.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:15:20.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.939+0000 7f997b19d700 1 Processor -- start 2023-10-06T05:15:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.939+0000 7f997b19d700 1 -- start start 2023-10-06T05:15:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.940+0000 7f997b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f997406cb80 0x7f997415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.940+0000 7f997b19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997415d3a0 con 0x7f997406c7b0 2023-10-06T05:15:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.940+0000 7f997b19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740b0470 con 0x7f99740aff90 2023-10-06T05:15:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.940+0000 7f997b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740686c0 con 0x7f997406cb80 2023-10-06T05:15:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.940+0000 7f997a19b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f997406c7b0 0x7f9974067fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59966/0 (socket says 172.21.15.33:59966) 2023-10-06T05:15:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.940+0000 7f997a19b700 1 -- 172.21.15.33:0/2196037287 learned_addr learned my addr 172.21.15.33:0/2196037287 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:20.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f997a99c700 1 --2- 172.21.15.33:0/2196037287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f997406cb80 0x7f997415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128228572 0 0) 0x7f997415d3a0 con 0x7f997406c7b0 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9960003570 con 0x7f997406c7b0 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336705464 0 0) 0x7f99740b0470 con 0x7f99740aff90 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f997415d3a0 con 0x7f99740aff90 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f997a99c700 1 -- 172.21.15.33:0/2196037287 >> v1:172.21.15.167:6789/0 conn(0x7f99740aff90 legacy=0x7f99740a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f997a99c700 1 -- 172.21.15.33:0/2196037287 >> v1:172.21.15.33:6789/0 conn(0x7f997406c7b0 legacy=0x7f9974067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f997a99c700 1 -- 172.21.15.33:0/2196037287 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740b0750 con 0x7f997406cb80 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.941+0000 7f997a99c700 1 --2- 172.21.15.33:0/2196037287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f997406cb80 0x7f997415d830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f996c009e90 tx=0x7f996c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f651de9f18b1e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.942+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00e070 con 0x7f997406cb80 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.942+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00bb80 con 0x7f997406cb80 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.942+0000 7f9979199700 1 -- 172.21.15.33:0/2196037287 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c012430 con 0x7f997406cb80 2023-10-06T05:15:20.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.942+0000 7f997b19d700 1 -- 172.21.15.33:0/2196037287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f997406cb80 msgr2=0x7f997415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.942+0000 7f997b19d700 1 --2- 172.21.15.33:0/2196037287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f997406cb80 0x7f997415d830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f996c009e90 tx=0x7f996c009780 comp rx=0 tx=0).stop 2023-10-06T05:15:20.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.943+0000 7f997b19d700 1 -- 172.21.15.33:0/2196037287 shutdown_connections 2023-10-06T05:15:20.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.943+0000 7f997b19d700 1 --2- 172.21.15.33:0/2196037287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f997406cb80 0x7f997415d830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.943+0000 7f997b19d700 1 -- 172.21.15.33:0/2196037287 >> 172.21.15.33:0/2196037287 conn(0x7f997406d670 msgr2=0x7f997406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.943+0000 7f997b19d700 1 -- 172.21.15.33:0/2196037287 shutdown_connections 2023-10-06T05:15:20.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.943+0000 7f997b19d700 1 -- 172.21.15.33:0/2196037287 wait complete. 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.944+0000 7f997b19d700 1 Processor -- start 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.944+0000 7f997b19d700 1 -- start start 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f997406c7b0 0x7f997416a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 0x7f997414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99740aff90 0x7f99740ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997416aa50 con 0x7f997406cb80 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997416ad30 con 0x7f997406c7b0 2023-10-06T05:15:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997416b010 con 0x7f99740aff90 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 0x7f997414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 0x7f997414a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51992/0 (socket says 172.21.15.33:51992) 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997999a700 1 -- 172.21.15.33:0/1039990066 learned_addr learned my addr 172.21.15.33:0/1039990066 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997a99c700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99740aff90 0x7f99740ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.945+0000 7f997a19b700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f997406c7b0 0x7f997416a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997999a700 1 -- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99740aff90 msgr2=0x7f99740ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997999a700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99740aff90 0x7f99740ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997999a700 1 -- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f997406c7b0 msgr2=0x7f997416a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997999a700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f997406c7b0 0x7f997416a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997999a700 1 -- 172.21.15.33:0/1039990066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c009a20 con 0x7f997406cb80 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997999a700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 0x7f997414a830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f996400c5b0 tx=0x7f9964007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99640105c0 con 0x7f997406cb80 2023-10-06T05:15:20.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964010720 con 0x7f997406cb80 2023-10-06T05:15:20.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997414c8e0 con 0x7f997406cb80 2023-10-06T05:15:20.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964010ae0 con 0x7f997406cb80 2023-10-06T05:15:20.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.946+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997416a690 con 0x7f997406cb80 2023-10-06T05:15:20.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.947+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997414ce00 con 0x7f997406cb80 2023-10-06T05:15:20.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.948+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99640123e0 con 0x7f997406cb80 2023-10-06T05:15:20.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.949+0000 7f996a7fc700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9958066f60 0x7f9958069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.949+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9964083f00 con 0x7f997406cb80 2023-10-06T05:15:20.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.950+0000 7f997a19b700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9958066f60 0x7f9958069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.950+0000 7f997a19b700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9958066f60 0x7f9958069410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9974156c50 tx=0x7f9970009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:20.953+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f997414ce00 con 0x7f997406cb80 2023-10-06T05:15:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.153+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f997414ce00 con 0x7f997406cb80 2023-10-06T05:15:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.154+0000 7f996a7fc700 1 -- 172.21.15.33:0/1039990066 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f997414ce00 con 0x7f997406cb80 2023-10-06T05:15:21.155 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9958066f60 msgr2=0x7f9958069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9958066f60 0x7f9958069410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9974156c50 tx=0x7f9970009040 comp rx=0 tx=0).stop 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 msgr2=0x7f997414a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 0x7f997414a830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f996400c5b0 tx=0x7f9964007d90 comp rx=0 tx=0).stop 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 shutdown_connections 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9958066f60 0x7f9958069410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99740aff90 0x7f99740ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f997406c7b0 0x7f997416a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 --2- 172.21.15.33:0/1039990066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f997406cb80 0x7f997414a830 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 >> 172.21.15.33:0/1039990066 conn(0x7f997406d670 msgr2=0x7f99741562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 shutdown_connections 2023-10-06T05:15:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.156+0000 7f997b19d700 1 -- 172.21.15.33:0/1039990066 wait complete. 2023-10-06T05:15:21.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.362+0000 7fa29ffff700 1 Processor -- start 2023-10-06T05:15:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.363+0000 7fa29ffff700 1 -- start start 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a01552f0 0x7fa2a00ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a01586a0 con 0x7fa2a00b18f0 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a0157ab0 con 0x7fa2a0154f20 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a006bdf0 con 0x7fa2a01552f0 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa2a00b18f0 0x7fa2a00c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59972/0 (socket says 172.21.15.33:59972) 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29f7fe700 1 -- 172.21.15.33:0/3299024336 learned_addr learned my addr 172.21.15.33:0/3299024336 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.364+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/3299024336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a01552f0 0x7fa2a00ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185585874 0 0) 0x7fa2a01586a0 con 0x7fa2a00b18f0 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa288003570 con 0x7fa2a00b18f0 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995166151 0 0) 0x7fa2a0157ab0 con 0x7fa2a0154f20 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a01586a0 con 0x7fa2a0154f20 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29e7fc700 1 -- 172.21.15.33:0/3299024336 >> v1:172.21.15.167:6789/0 conn(0x7fa2a0154f20 legacy=0x7fa2a00b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29e7fc700 1 -- 172.21.15.33:0/3299024336 >> v1:172.21.15.33:6789/0 conn(0x7fa2a00b18f0 legacy=0x7fa2a00c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29e7fc700 1 -- 172.21.15.33:0/3299024336 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0157d70 con 0x7fa2a01552f0 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.365+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/3299024336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a01552f0 0x7fa2a00ae720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa294009910 tx=0x7fa2940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e393da237160b2b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294011630 con 0x7fa2a01552f0 2023-10-06T05:15:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa294011790 con 0x7fa2a01552f0 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29dffb700 1 -- 172.21.15.33:0/3299024336 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294011b30 con 0x7fa2a01552f0 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 -- 172.21.15.33:0/3299024336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a01552f0 msgr2=0x7fa2a00ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 --2- 172.21.15.33:0/3299024336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a01552f0 0x7fa2a00ae720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa294009910 tx=0x7fa2940091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 -- 172.21.15.33:0/3299024336 shutdown_connections 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 --2- 172.21.15.33:0/3299024336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a01552f0 0x7fa2a00ae720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 -- 172.21.15.33:0/3299024336 >> 172.21.15.33:0/3299024336 conn(0x7fa2a006d030 msgr2=0x7fa2a00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 -- 172.21.15.33:0/3299024336 shutdown_connections 2023-10-06T05:15:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.366+0000 7fa29ffff700 1 -- 172.21.15.33:0/3299024336 wait complete. 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.368+0000 7fa29ffff700 1 Processor -- start 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.368+0000 7fa29ffff700 1 -- start start 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 0x7fa2a0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a0154f20 0x7fa2a00b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a016b2b0 0x7fa2a014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a014c3b0 con 0x7fa2a00b18f0 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00c1780 con 0x7fa2a016b2b0 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a0156790 con 0x7fa2a0154f20 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 0x7fa2a0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 0x7fa2a0156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51996/0 (socket says 172.21.15.33:51996) 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29effd700 1 -- 172.21.15.33:0/44833049 learned_addr learned my addr 172.21.15.33:0/44833049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a0154f20 0x7fa2a00b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.369+0000 7fa29f7fe700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a016b2b0 0x7fa2a014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29effd700 1 -- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a0154f20 msgr2=0x7fa2a00b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29effd700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a0154f20 0x7fa2a00b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29effd700 1 -- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a016b2b0 msgr2=0x7fa2a014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29effd700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a016b2b0 0x7fa2a014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29effd700 1 -- 172.21.15.33:0/44833049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294009420 con 0x7fa2a00b18f0 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a0154f20 0x7fa2a00b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:21.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29f7fe700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a016b2b0 0x7fa2a014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29effd700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 0x7fa2a0156c20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c7f0 tx=0x7fa290007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290016070 con 0x7fa2a00b18f0 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa290011680 con 0x7fa2a00b18f0 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a0156180 con 0x7fa2a00b18f0 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290011a20 con 0x7fa2a00b18f0 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.370+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0156420 con 0x7fa2a00b18f0 2023-10-06T05:15:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.371+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a004c380 con 0x7fa2a00b18f0 2023-10-06T05:15:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.372+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa290012440 con 0x7fa2a00b18f0 2023-10-06T05:15:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.374+0000 7fa286ffd700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.374+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa2900841c0 con 0x7fa2a00b18f0 2023-10-06T05:15:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.374+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.375+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa294009860 tx=0x7fa294017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.378+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa290051580 con 0x7fa2a00b18f0 2023-10-06T05:15:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.581+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2a00ab7f0 con 0x7fa2a00b18f0 2023-10-06T05:15:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.582+0000 7fa286ffd700 1 -- 172.21.15.33:0/44833049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa290054ba0 con 0x7fa2a00b18f0 2023-10-06T05:15:21.584 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 msgr2=0x7fa28c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa294009860 tx=0x7fa294017000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 msgr2=0x7fa2a0156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 0x7fa2a0156c20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c7f0 tx=0x7fa290007dd0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 shutdown_connections 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a016b2b0 0x7fa2a014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a00b18f0 0x7fa2a0156c20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 --2- 172.21.15.33:0/44833049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a0154f20 0x7fa2a00b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 >> 172.21.15.33:0/44833049 conn(0x7fa2a006d030 msgr2=0x7fa2a00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 shutdown_connections 2023-10-06T05:15:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.584+0000 7fa29ffff700 1 -- 172.21.15.33:0/44833049 wait complete. 2023-10-06T05:15:21.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:15:21.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.789+0000 7fa635f9d700 1 Processor -- start 2023-10-06T05:15:21.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.789+0000 7fa635f9d700 1 -- start start 2023-10-06T05:15:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa635f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6300c8b50 0x7fa6300c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa635f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63006ab60 con 0x7fa630155ec0 2023-10-06T05:15:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa635f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63006ae40 con 0x7fa630155af0 2023-10-06T05:15:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa635f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63006b7e0 con 0x7fa6300c8b50 2023-10-06T05:15:21.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa63579c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa630155ec0 0x7fa6300c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59986/0 (socket says 172.21.15.33:59986) 2023-10-06T05:15:21.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa63579c700 1 -- 172.21.15.33:0/3861741717 learned_addr learned my addr 172.21.15.33:0/3861741717 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:21.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa634f9b700 1 --1- 172.21.15.33:0/3861741717 >> v1:172.21.15.167:6789/0 conn(0x7fa630155af0 0x7fa63006a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:50018/0 (socket says 172.21.15.33:50018) 2023-10-06T05:15:21.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.790+0000 7fa62ffff700 1 --2- 172.21.15.33:0/3861741717 >> v2:172.21.15.167:3300/0 conn(0x7fa6300c8b50 0x7fa6300c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.791+0000 7fa62ffff700 1 -- 172.21.15.33:0/3861741717 >> v1:172.21.15.167:6789/0 conn(0x7fa630155af0 legacy=0x7fa63006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.791+0000 7fa62ffff700 1 -- 172.21.15.33:0/3861741717 >> v1:172.21.15.33:6789/0 conn(0x7fa630155ec0 legacy=0x7fa6300c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.791+0000 7fa62ffff700 1 -- 172.21.15.33:0/3861741717 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63006b120 con 0x7fa6300c8b50 2023-10-06T05:15:21.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.791+0000 7fa62ffff700 1 --2- 172.21.15.33:0/3861741717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6300c8b50 0x7fa6300c7210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0098e0 tx=0x7fa61c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45d25d78d4da11d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.794+0000 7fa62f7fe700 1 -- 172.21.15.33:0/3861741717 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c009e80 con 0x7fa6300c8b50 2023-10-06T05:15:21.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.794+0000 7fa62f7fe700 1 -- 172.21.15.33:0/3861741717 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c00b860 con 0x7fa6300c8b50 2023-10-06T05:15:21.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.794+0000 7fa62f7fe700 1 -- 172.21.15.33:0/3861741717 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00bbe0 con 0x7fa6300c8b50 2023-10-06T05:15:21.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 -- 172.21.15.33:0/3861741717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6300c8b50 msgr2=0x7fa6300c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 --2- 172.21.15.33:0/3861741717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6300c8b50 0x7fa6300c7210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0098e0 tx=0x7fa61c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 -- 172.21.15.33:0/3861741717 shutdown_connections 2023-10-06T05:15:21.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 --2- 172.21.15.33:0/3861741717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6300c8b50 0x7fa6300c7210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 -- 172.21.15.33:0/3861741717 >> 172.21.15.33:0/3861741717 conn(0x7fa63006d030 msgr2=0x7fa63006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 -- 172.21.15.33:0/3861741717 shutdown_connections 2023-10-06T05:15:21.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.795+0000 7fa635f9d700 1 -- 172.21.15.33:0/3861741717 wait complete. 2023-10-06T05:15:21.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.796+0000 7fa635f9d700 1 Processor -- start 2023-10-06T05:15:21.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.797+0000 7fa635f9d700 1 -- start start 2023-10-06T05:15:21.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa635f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6300c8b50 0x7fa630148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa635f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 0x7fa63016a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa635f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa630155ec0 0x7fa6300a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa635f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63016a3f0 con 0x7fa630155af0 2023-10-06T05:15:21.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa635f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630169dc0 con 0x7fa630155ec0 2023-10-06T05:15:21.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa635f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300c3af0 con 0x7fa6300c8b50 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 0x7fa63016a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 0x7fa63016a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52006/0 (socket says 172.21.15.33:52006) 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 -- 172.21.15.33:0/2091017694 learned_addr learned my addr 172.21.15.33:0/2091017694 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa63579c700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa630155ec0 0x7fa6300a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa634f9b700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6300c8b50 0x7fa630148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 -- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6300c8b50 msgr2=0x7fa630148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6300c8b50 0x7fa630148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 -- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa630155ec0 msgr2=0x7fa6300a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa630155ec0 0x7fa6300a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa62ffff700 1 -- 172.21.15.33:0/2091017694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c009420 con 0x7fa630155af0 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.798+0000 7fa634f9b700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6300c8b50 0x7fa630148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.799+0000 7fa62ffff700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 0x7fa63016a880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa6300635d0 tx=0x7fa61c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.799+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c011420 con 0x7fa630155af0 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.799+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c011580 con 0x7fa630155af0 2023-10-06T05:15:21.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.799+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c011940 con 0x7fa630155af0 2023-10-06T05:15:21.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.799+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6300c3dd0 con 0x7fa630155af0 2023-10-06T05:15:21.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.799+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63016a080 con 0x7fa630155af0 2023-10-06T05:15:21.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.800+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63016a560 con 0x7fa630155af0 2023-10-06T05:15:21.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.801+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa61c011cf0 con 0x7fa630155af0 2023-10-06T05:15:21.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.802+0000 7fa62cff9700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa614066f30 0x7fa6140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.802+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa61c085d50 con 0x7fa630155af0 2023-10-06T05:15:21.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.802+0000 7fa634f9b700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa614066f30 0x7fa6140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.803+0000 7fa634f9b700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa614066f30 0x7fa6140693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa624007fd0 tx=0x7fa624007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:21.806+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa61c053110 con 0x7fa630155af0 2023-10-06T05:15:22.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.011+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa63004c380 con 0x7fa630155af0 2023-10-06T05:15:22.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.012+0000 7fa62cff9700 1 -- 172.21.15.33:0/2091017694 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa61c015090 con 0x7fa630155af0 2023-10-06T05:15:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:22.014 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:15:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa614066f30 msgr2=0x7fa6140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa614066f30 0x7fa6140693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa624007fd0 tx=0x7fa624007e10 comp rx=0 tx=0).stop 2023-10-06T05:15:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 msgr2=0x7fa63016a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 0x7fa63016a880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa6300635d0 tx=0x7fa61c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 shutdown_connections 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa614066f30 0x7fa6140693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa630155ec0 0x7fa6300a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6300c8b50 0x7fa630148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 --2- 172.21.15.33:0/2091017694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630155af0 0x7fa63016a880 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 >> 172.21.15.33:0/2091017694 conn(0x7fa63006d030 msgr2=0x7fa6300cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 shutdown_connections 2023-10-06T05:15:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.014+0000 7fa635f9d700 1 -- 172.21.15.33:0/2091017694 wait complete. 2023-10-06T05:15:22.033 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:15:22.033 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:15:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.222+0000 7f77fa99d700 1 Processor -- start 2023-10-06T05:15:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.223+0000 7f77fa99d700 1 -- start start 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77fa99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f4159400 0x7f77f40a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77fa99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f77f406dbf0 0x7f77f4067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77fa99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40a9dd0 con 0x7f77f41597d0 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77fa99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f406d760 con 0x7f77f406dbf0 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77fa99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40aa8c0 con 0x7f77f4159400 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77f919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f77f406dbf0 0x7f77f4067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77f919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f77f406dbf0 0x7f77f4067b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52010/0 (socket says 172.21.15.33:52010) 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77f919a700 1 -- 172.21.15.33:0/1608756551 learned_addr learned my addr 172.21.15.33:0/1608756551 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.224+0000 7f77f999b700 1 --2- 172.21.15.33:0/1608756551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f4159400 0x7f77f40a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f919a700 1 -- 172.21.15.33:0/1608756551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f4159400 msgr2=0x7f77f40a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f919a700 1 --2- 172.21.15.33:0/1608756551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f4159400 0x7f77f40a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f919a700 1 -- 172.21.15.33:0/1608756551 >> v1:172.21.15.167:6789/0 conn(0x7f77f41597d0 legacy=0x7f77f406a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f919a700 1 -- 172.21.15.33:0/1608756551 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40aa5e0 con 0x7f77f406dbf0 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f919a700 1 --2- 172.21.15.33:0/1608756551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f406dbf0 0x7f77f4067b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77e40099b0 tx=0x7f77e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4886e7597f62fd5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f8999700 1 -- 172.21.15.33:0/1608756551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4009ea0 con 0x7f77f406dbf0 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f8999700 1 -- 172.21.15.33:0/1608756551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e400b860 con 0x7f77f406dbf0 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.225+0000 7f77f8999700 1 -- 172.21.15.33:0/1608756551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400bc00 con 0x7f77f406dbf0 2023-10-06T05:15:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 -- 172.21.15.33:0/1608756551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f406dbf0 msgr2=0x7f77f4067b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1608756551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f406dbf0 0x7f77f4067b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77e40099b0 tx=0x7f77e4009250 comp rx=0 tx=0).stop 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 -- 172.21.15.33:0/1608756551 shutdown_connections 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1608756551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f4159400 0x7f77f40a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1608756551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f406dbf0 0x7f77f4067b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 -- 172.21.15.33:0/1608756551 >> 172.21.15.33:0/1608756551 conn(0x7f77f406ebf0 msgr2=0x7f77f406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 -- 172.21.15.33:0/1608756551 shutdown_connections 2023-10-06T05:15:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.226+0000 7f77fa99d700 1 -- 172.21.15.33:0/1608756551 wait complete. 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.227+0000 7f77fa99d700 1 Processor -- start 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 -- start start 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f406dbf0 0x7f77f40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 0x7f77f413eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f41597d0 0x7f77f4169320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f413c4a0 con 0x7f77f4159400 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40ae6b0 con 0x7f77f41597d0 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.228+0000 7f77fa99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40ae950 con 0x7f77f406dbf0 2023-10-06T05:15:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 0x7f77f413eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 0x7f77f413eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52026/0 (socket says 172.21.15.33:52026) 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 -- 172.21.15.33:0/1589281486 learned_addr learned my addr 172.21.15.33:0/1589281486 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f999b700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f406dbf0 0x7f77f40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77fa19c700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f41597d0 0x7f77f4169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 -- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f406dbf0 msgr2=0x7f77f40ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f406dbf0 0x7f77f40ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 -- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f41597d0 msgr2=0x7f77f4169320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f41597d0 0x7f77f4169320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 -- 172.21.15.33:0/1589281486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40094f0 con 0x7f77f4159400 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77fa19c700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f41597d0 0x7f77f4169320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.229+0000 7f77f919a700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 0x7f77f413eeb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f77f406b360 tx=0x7f77e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.230+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400bbe0 con 0x7f77f4159400 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.230+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4011420 con 0x7f77f4159400 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.230+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40117c0 con 0x7f77f4159400 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.230+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f40aec30 con 0x7f77f4159400 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.230+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f40af1f0 con 0x7f77f4159400 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.231+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f4063ff0 con 0x7f77f4159400 2023-10-06T05:15:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.232+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77e4011bf0 con 0x7f77f4159400 2023-10-06T05:15:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.233+0000 7f77e9ffb700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77d8066f30 0x7f77d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.233+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f77e4085d10 con 0x7f77f4159400 2023-10-06T05:15:22.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.233+0000 7f77f999b700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77d8066f30 0x7f77d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.234+0000 7f77f999b700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77d8066f30 0x7f77d80693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009d40 tx=0x7f77f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.237+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77e408b020 con 0x7f77f4159400 2023-10-06T05:15:22.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.440+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f77f4156f30 con 0x7f77f4159400 2023-10-06T05:15:22.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.800+0000 7f77e9ffb700 1 -- 172.21.15.33:0/1589281486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f77e40530d0 con 0x7f77f4159400 2023-10-06T05:15:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.808+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77d8066f30 msgr2=0x7f77d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77d8066f30 0x7f77d80693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009d40 tx=0x7f77f0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 msgr2=0x7f77f413eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 0x7f77f413eeb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f77f406b360 tx=0x7f77e4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 shutdown_connections 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77d8066f30 0x7f77d80693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f41597d0 0x7f77f4169320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77f406dbf0 0x7f77f40ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 --2- 172.21.15.33:0/1589281486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77f4159400 0x7f77f413eeb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 >> 172.21.15.33:0/1589281486 conn(0x7f77f406ebf0 msgr2=0x7f77f406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 shutdown_connections 2023-10-06T05:15:22.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:22.809+0000 7f77fa99d700 1 -- 172.21.15.33:0/1589281486 wait complete. 2023-10-06T05:15:22.812 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:15:22.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:15:23.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.021+0000 7f50ef59d700 1 Processor -- start 2023-10-06T05:15:23.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.021+0000 7f50ef59d700 1 -- start start 2023-10-06T05:15:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.022+0000 7f50ef59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50e80cd5e0 0x7f50e80af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.022+0000 7f50ef59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e815ddb0 con 0x7f50e815a7d0 2023-10-06T05:15:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.022+0000 7f50ef59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e815d1a0 con 0x7f50e80af540 2023-10-06T05:15:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.022+0000 7f50ef59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e815d480 con 0x7f50e80cd5e0 2023-10-06T05:15:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50edd9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50e815a7d0 0x7f50e80cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59990/0 (socket says 172.21.15.33:59990) 2023-10-06T05:15:23.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50edd9a700 1 -- 172.21.15.33:0/416423633 learned_addr learned my addr 172.21.15.33:0/416423633 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50eed9c700 1 --2- 172.21.15.33:0/416423633 >> v2:172.21.15.167:3300/0 conn(0x7f50e80cd5e0 0x7f50e80af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50ed599700 1 -- 172.21.15.33:0/416423633 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744685657 0 0) 0x7f50e815ddb0 con 0x7f50e815a7d0 2023-10-06T05:15:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50ed599700 1 -- 172.21.15.33:0/416423633 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d4003570 con 0x7f50e815a7d0 2023-10-06T05:15:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50eed9c700 1 -- 172.21.15.33:0/416423633 >> v1:172.21.15.167:6789/0 conn(0x7f50e80af540 legacy=0x7f50e80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50eed9c700 1 -- 172.21.15.33:0/416423633 >> v1:172.21.15.33:6789/0 conn(0x7f50e815a7d0 legacy=0x7f50e80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.023+0000 7f50eed9c700 1 -- 172.21.15.33:0/416423633 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e815d760 con 0x7f50e80cd5e0 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50eed9c700 1 --2- 172.21.15.33:0/416423633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80af910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e60 tx=0x7f50e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f73ae77a9e4856 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ed599700 1 -- 172.21.15.33:0/416423633 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000e070 con 0x7f50e80cd5e0 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ed599700 1 -- 172.21.15.33:0/416423633 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000bb60 con 0x7f50e80cd5e0 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ed599700 1 -- 172.21.15.33:0/416423633 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0012430 con 0x7f50e80cd5e0 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ef59d700 1 -- 172.21.15.33:0/416423633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 msgr2=0x7f50e80af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ef59d700 1 --2- 172.21.15.33:0/416423633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80af910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e60 tx=0x7f50e0009780 comp rx=0 tx=0).stop 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ef59d700 1 -- 172.21.15.33:0/416423633 shutdown_connections 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ef59d700 1 --2- 172.21.15.33:0/416423633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80af910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.024+0000 7f50ef59d700 1 -- 172.21.15.33:0/416423633 >> 172.21.15.33:0/416423633 conn(0x7f50e806d9b0 msgr2=0x7f50e806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.025+0000 7f50ef59d700 1 -- 172.21.15.33:0/416423633 shutdown_connections 2023-10-06T05:15:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.025+0000 7f50ef59d700 1 -- 172.21.15.33:0/416423633 wait complete. 2023-10-06T05:15:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.026+0000 7f50ef59d700 1 Processor -- start 2023-10-06T05:15:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.026+0000 7f50ef59d700 1 -- start start 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ef59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80af540 0x7f50e80b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ef59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ef59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 0x7f50e80b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ef59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81583a0 con 0x7f50e815a7d0 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ef59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80b14c0 con 0x7f50e80cd5e0 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ef59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80b0610 con 0x7f50e80af540 2023-10-06T05:15:23.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50eed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 0x7f50e80b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50eed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 0x7f50e80b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52028/0 (socket says 172.21.15.33:52028) 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50eed9c700 1 -- 172.21.15.33:0/2251661173 learned_addr learned my addr 172.21.15.33:0/2251661173 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50edd9a700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.027+0000 7f50ee59b700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80af540 0x7f50e80b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50eed9c700 1 -- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80af540 msgr2=0x7f50e80b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50eed9c700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80af540 0x7f50e80b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50eed9c700 1 -- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 msgr2=0x7f50e80b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50eed9c700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50eed9c700 1 -- 172.21.15.33:0/2251661173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009a20 con 0x7f50e815a7d0 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50eed9c700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 0x7f50e80b3b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50e80896d0 tx=0x7f50e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0027470 con 0x7f50e815a7d0 2023-10-06T05:15:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e00275f0 con 0x7f50e815a7d0 2023-10-06T05:15:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e00279b0 con 0x7f50e815a7d0 2023-10-06T05:15:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.028+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e80b08b0 con 0x7f50e815a7d0 2023-10-06T05:15:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.029+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e80b0e70 con 0x7f50e815a7d0 2023-10-06T05:15:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.029+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e804c380 con 0x7f50e815a7d0 2023-10-06T05:15:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.031+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50e0024070 con 0x7f50e815a7d0 2023-10-06T05:15:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.031+0000 7f50de7fc700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50cc066f30 0x7f50cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.032+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f50e00872b0 con 0x7f50e815a7d0 2023-10-06T05:15:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.032+0000 7f50ee59b700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50cc066f30 0x7f50cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.033+0000 7f50ee59b700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50cc066f30 0x7f50cc0693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50e40037c0 tx=0x7f50e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.035+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50e0054630 con 0x7f50e815a7d0 2023-10-06T05:15:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.237+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f50e815da40 con 0x7f50e815a7d0 2023-10-06T05:15:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.807+0000 7f50de7fc700 1 -- 172.21.15.33:0/2251661173 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f50e0057c50 con 0x7f50e815a7d0 2023-10-06T05:15:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.815+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50cc066f30 msgr2=0x7f50cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.815+0000 7f50ef59d700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50cc066f30 0x7f50cc0693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50e40037c0 tx=0x7f50e4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 msgr2=0x7f50e80b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 0x7f50e80b3b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50e80896d0 tx=0x7f50e0019000 comp rx=0 tx=0).stop 2023-10-06T05:15:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 shutdown_connections 2023-10-06T05:15:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50cc066f30 0x7f50cc0693e0 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50e40037c0 tx=0x7f50e4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e815a7d0 0x7f50e80b3b70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80af540 0x7f50e80b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 --2- 172.21.15.33:0/2251661173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cd5e0 0x7f50e80b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.816+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 >> 172.21.15.33:0/2251661173 conn(0x7f50e806d9b0 msgr2=0x7f50e815b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.817+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 shutdown_connections 2023-10-06T05:15:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:23.818+0000 7f50ef59d700 1 -- 172.21.15.33:0/2251661173 wait complete. 2023-10-06T05:15:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:15:23.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:15:24.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.035+0000 7efe4299d700 1 Processor -- start 2023-10-06T05:15:24.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.035+0000 7efe4299d700 1 -- start start 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4299d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe3c0cc7e0 0x7efe3c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4299d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efe3c0b01f0 0x7efe3c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4299d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0ad430 con 0x7efe3c156370 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4299d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0afe20 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4299d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0b0b70 con 0x7efe3c0b01f0 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4119a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe3c0cc7e0 0x7efe3c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4119a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe3c0cc7e0 0x7efe3c0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52044/0 (socket says 172.21.15.33:52044) 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4119a700 1 -- 172.21.15.33:0/356253508 learned_addr learned my addr 172.21.15.33:0/356253508 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.036+0000 7efe4219c700 1 --2- 172.21.15.33:0/356253508 >> v2:172.21.15.167:3300/0 conn(0x7efe3c0b01f0 0x7efe3c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe4119a700 1 -- 172.21.15.33:0/356253508 >> v2:172.21.15.167:3300/0 conn(0x7efe3c0b01f0 msgr2=0x7efe3c0b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe4119a700 1 --2- 172.21.15.33:0/356253508 >> v2:172.21.15.167:3300/0 conn(0x7efe3c0b01f0 0x7efe3c0b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe4119a700 1 -- 172.21.15.33:0/356253508 >> v1:172.21.15.167:6789/0 conn(0x7efe3c156370 legacy=0x7efe3c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe4119a700 1 -- 172.21.15.33:0/356253508 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c06bc70 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe4119a700 1 --2- 172.21.15.33:0/356253508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0af8e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0b13b0 tx=0x7efe2c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7247da8bcdce8f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe40999700 1 -- 172.21.15.33:0/356253508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00dba0 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe40999700 1 -- 172.21.15.33:0/356253508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c00dd00 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.037+0000 7efe40999700 1 -- 172.21.15.33:0/356253508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0196d0 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 -- 172.21.15.33:0/356253508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 msgr2=0x7efe3c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 --2- 172.21.15.33:0/356253508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0af8e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0b13b0 tx=0x7efe2c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 -- 172.21.15.33:0/356253508 shutdown_connections 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 --2- 172.21.15.33:0/356253508 >> v2:172.21.15.167:3300/0 conn(0x7efe3c0b01f0 0x7efe3c0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 --2- 172.21.15.33:0/356253508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0af8e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 -- 172.21.15.33:0/356253508 >> 172.21.15.33:0/356253508 conn(0x7efe3c06d030 msgr2=0x7efe3c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 -- 172.21.15.33:0/356253508 shutdown_connections 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.038+0000 7efe4299d700 1 -- 172.21.15.33:0/356253508 wait complete. 2023-10-06T05:15:24.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.040+0000 7efe4299d700 1 Processor -- start 2023-10-06T05:15:24.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.040+0000 7efe4299d700 1 -- start start 2023-10-06T05:15:24.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3c0b01f0 0x7efe3c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe3c156370 0x7efe3c0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c151530 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0c7ee0 con 0x7efe3c0b01f0 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c16b720 con 0x7efe3c156370 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52046/0 (socket says 172.21.15.33:52046) 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 -- 172.21.15.33:0/3901575799 learned_addr learned my addr 172.21.15.33:0/3901575799 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4199b700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3c0b01f0 0x7efe3c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4219c700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe3c156370 0x7efe3c0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 -- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe3c156370 msgr2=0x7efe3c0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe3c156370 0x7efe3c0aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 -- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3c0b01f0 msgr2=0x7efe3c150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3c0b01f0 0x7efe3c150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4119a700 1 -- 172.21.15.33:0/3901575799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c00b790 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.041+0000 7efe4219c700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe3c156370 0x7efe3c0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.042+0000 7efe4119a700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0aa530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efe3c06d4d0 tx=0x7efe2c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.042+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c010680 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.042+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0107e0 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.042+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c16b110 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.042+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c010ba0 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.042+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c16b3b0 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.043+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c04c380 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.044+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe2c02c430 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.045+0000 7efe31ffb700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efe20067000 0x7efe200694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.046+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5958+0+0 (secure 0 0 0) 0x7efe2c014070 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.046+0000 7efe4199b700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efe20067000 0x7efe200694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.047+0000 7efe4199b700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efe20067000 0x7efe200694b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efe38003a80 tx=0x7efe38007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.051+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe2c053490 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.299+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efe3c06f310 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.832+0000 7efe31ffb700 1 -- 172.21.15.33:0/3901575799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7efe2c056ab0 con 0x7efe3c0cc7e0 2023-10-06T05:15:24.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.837+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efe20067000 msgr2=0x7efe200694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.837+0000 7efe4299d700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efe20067000 0x7efe200694b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efe38003a80 tx=0x7efe38007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.837+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 msgr2=0x7efe3c0aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.837+0000 7efe4299d700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0aa530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efe3c06d4d0 tx=0x7efe2c01b000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 shutdown_connections 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efe20067000 0x7efe200694b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe3c156370 0x7efe3c0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3c0b01f0 0x7efe3c150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 --2- 172.21.15.33:0/3901575799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c0cc7e0 0x7efe3c0aa530 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 >> 172.21.15.33:0/3901575799 conn(0x7efe3c06d030 msgr2=0x7efe3c0abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 shutdown_connections 2023-10-06T05:15:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:24.838+0000 7efe4299d700 1 -- 172.21.15.33:0/3901575799 wait complete. 2023-10-06T05:15:24.843 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T05:15:24.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:25.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.056+0000 7fc781f9d700 1 Processor -- start 2023-10-06T05:15:25.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.056+0000 7fc781f9d700 1 -- start start 2023-10-06T05:15:25.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.057+0000 7fc781f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc77c159bb0 0x7fc77c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.057+0000 7fc781f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c0cbd80 con 0x7fc77c0cf3b0 2023-10-06T05:15:25.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.057+0000 7fc781f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c0af5f0 con 0x7fc77c0ccfc0 2023-10-06T05:15:25.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.057+0000 7fc781f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c0af8d0 con 0x7fc77c159bb0 2023-10-06T05:15:25.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc77bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc77c0cf3b0 0x7fc77c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60004/0 (socket says 172.21.15.33:60004) 2023-10-06T05:15:25.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc77bfff700 1 -- 172.21.15.33:0/1216485659 learned_addr learned my addr 172.21.15.33:0/1216485659 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:25.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc78179c700 1 --2- 172.21.15.33:0/1216485659 >> v2:172.21.15.33:3300/0 conn(0x7fc77c159bb0 0x7fc77c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc77b7fe700 1 -- 172.21.15.33:0/1216485659 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811833487 0 0) 0x7fc77c0cbd80 con 0x7fc77c0cf3b0 2023-10-06T05:15:25.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc77b7fe700 1 -- 172.21.15.33:0/1216485659 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76c003570 con 0x7fc77c0cf3b0 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc78179c700 1 -- 172.21.15.33:0/1216485659 >> v1:172.21.15.167:6789/0 conn(0x7fc77c0ccfc0 legacy=0x7fc77c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc78179c700 1 -- 172.21.15.33:0/1216485659 >> v1:172.21.15.33:6789/0 conn(0x7fc77c0cf3b0 legacy=0x7fc77c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc78179c700 1 -- 172.21.15.33:0/1216485659 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c0afbb0 con 0x7fc77c159bb0 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.058+0000 7fc78179c700 1 --2- 172.21.15.33:0/1216485659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c159bb0 0x7fc77c0b0460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc774009e90 tx=0x7fc774009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5be81170557b0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc77b7fe700 1 -- 172.21.15.33:0/1216485659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400e070 con 0x7fc77c159bb0 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc77b7fe700 1 -- 172.21.15.33:0/1216485659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77400bc40 con 0x7fc77c159bb0 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc77b7fe700 1 -- 172.21.15.33:0/1216485659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774012430 con 0x7fc77c159bb0 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 -- 172.21.15.33:0/1216485659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c159bb0 msgr2=0x7fc77c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 --2- 172.21.15.33:0/1216485659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c159bb0 0x7fc77c0b0460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc774009e90 tx=0x7fc774009780 comp rx=0 tx=0).stop 2023-10-06T05:15:25.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 -- 172.21.15.33:0/1216485659 shutdown_connections 2023-10-06T05:15:25.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 --2- 172.21.15.33:0/1216485659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c159bb0 0x7fc77c0b0460 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 -- 172.21.15.33:0/1216485659 >> 172.21.15.33:0/1216485659 conn(0x7fc77c06cff0 msgr2=0x7fc77c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 -- 172.21.15.33:0/1216485659 shutdown_connections 2023-10-06T05:15:25.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.059+0000 7fc781f9d700 1 -- 172.21.15.33:0/1216485659 wait complete. 2023-10-06T05:15:25.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.061+0000 7fc781f9d700 1 Processor -- start 2023-10-06T05:15:25.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.061+0000 7fc781f9d700 1 -- start start 2023-10-06T05:15:25.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc781f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc77c0ccfc0 0x7fc77c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc781f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc77c159bb0 0x7fc77c142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc781f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 0x7fc77c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc781f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0b58d0 con 0x7fc77c16a190 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc781f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c142690 con 0x7fc77c0ccfc0 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc781f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0b4fa0 con 0x7fc77c159bb0 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 0x7fc77c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 0x7fc77c0b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52054/0 (socket says 172.21.15.33:52054) 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 -- 172.21.15.33:0/3347340756 learned_addr learned my addr 172.21.15.33:0/3347340756 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc77bfff700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc77c159bb0 0x7fc77c142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 -- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc77c159bb0 msgr2=0x7fc77c142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc780f9b700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc77c0ccfc0 0x7fc77c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc77c159bb0 0x7fc77c142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 -- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc77c0ccfc0 msgr2=0x7fc77c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc77c0ccfc0 0x7fc77c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc78179c700 1 -- 172.21.15.33:0/3347340756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774009a20 con 0x7fc77c16a190 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc77bfff700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc77c159bb0 0x7fc77c142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.062+0000 7fc780f9b700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc77c0ccfc0 0x7fc77c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.063+0000 7fc78179c700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 0x7fc77c0b5d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc77c06be20 tx=0x7fc774019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.063+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77401b630 con 0x7fc77c16a190 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.063+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77401b7d0 con 0x7fc77c16a190 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.063+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77401bb90 con 0x7fc77c16a190 2023-10-06T05:15:25.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.063+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c0b5280 con 0x7fc77c16a190 2023-10-06T05:15:25.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.063+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c0b62a0 con 0x7fc77c16a190 2023-10-06T05:15:25.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.064+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c0b50e0 con 0x7fc77c16a190 2023-10-06T05:15:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.065+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc77401c400 con 0x7fc77c16a190 2023-10-06T05:15:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.066+0000 7fc778ff9700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc760066f30 0x7fc7600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.066+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc774086540 con 0x7fc77c16a190 2023-10-06T05:15:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.067+0000 7fc780f9b700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc760066f30 0x7fc7600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.067+0000 7fc780f9b700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc760066f30 0x7fc7600693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0cd730 tx=0x7fc770009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.070+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc77c0b50e0 con 0x7fc77c16a190 2023-10-06T05:15:25.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.278+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc77c0b50e0 con 0x7fc77c16a190 2023-10-06T05:15:25.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.279+0000 7fc778ff9700 1 -- 172.21.15.33:0/3347340756 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fc77c0b50e0 con 0x7fc77c16a190 2023-10-06T05:15:25.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:25.282 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:24.300976+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:22.479742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:23.276900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc760066f30 msgr2=0x7fc7600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc760066f30 0x7fc7600693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0cd730 tx=0x7fc770009040 comp rx=0 tx=0).stop 2023-10-06T05:15:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 msgr2=0x7fc77c0b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 0x7fc77c0b5d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc77c06be20 tx=0x7fc774019000 comp rx=0 tx=0).stop 2023-10-06T05:15:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 shutdown_connections 2023-10-06T05:15:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc760066f30 0x7fc7600693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c16a190 0x7fc77c0b5d60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc77c0ccfc0 0x7fc77c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 --2- 172.21.15.33:0/3347340756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc77c159bb0 0x7fc77c142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 >> 172.21.15.33:0/3347340756 conn(0x7fc77c06cff0 msgr2=0x7fc77c0cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 shutdown_connections 2023-10-06T05:15:25.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.282+0000 7fc781f9d700 1 -- 172.21.15.33:0/3347340756 wait complete. 2023-10-06T05:15:25.301 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:25.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.487+0000 7f61fb59e700 1 Processor -- start 2023-10-06T05:15:25.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.487+0000 7f61fb59e700 1 -- start start 2023-10-06T05:15:25.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61fc0b0a40 0x7f61fc06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc15e2d0 con 0x7f61fc0b0e10 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc15d6c0 con 0x7f61fc0b0a40 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc15d9a0 con 0x7f61fc15a0a0 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fa59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61fc0b0a40 0x7f61fc06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fa59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61fc0b0a40 0x7f61fc06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52056/0 (socket says 172.21.15.33:52056) 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fa59c700 1 -- 172.21.15.33:0/2706748527 learned_addr learned my addr 172.21.15.33:0/2706748527 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:25.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fad9d700 1 --2- 172.21.15.33:0/2706748527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fa59c700 1 -- 172.21.15.33:0/2706748527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 msgr2=0x7f61fc15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.489+0000 7f61fa59c700 1 --2- 172.21.15.33:0/2706748527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc15a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.490+0000 7f61fa59c700 1 -- 172.21.15.33:0/2706748527 >> v1:172.21.15.167:6789/0 conn(0x7f61fc0b0e10 legacy=0x7f61fc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.490+0000 7f61fa59c700 1 -- 172.21.15.33:0/2706748527 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc15dc80 con 0x7f61fc0b0a40 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.490+0000 7f61fa59c700 1 --2- 172.21.15.33:0/2706748527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc06ab00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0099b0 tx=0x7f61ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41064fc606a6aa88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.490+0000 7f61f959a700 1 -- 172.21.15.33:0/2706748527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00b860 con 0x7f61fc0b0a40 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.490+0000 7f61f959a700 1 -- 172.21.15.33:0/2706748527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec00b9c0 con 0x7f61fc0b0a40 2023-10-06T05:15:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.490+0000 7f61f959a700 1 -- 172.21.15.33:0/2706748527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00bd60 con 0x7f61fc0b0a40 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 -- 172.21.15.33:0/2706748527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 msgr2=0x7f61fc06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 --2- 172.21.15.33:0/2706748527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc06ab00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0099b0 tx=0x7f61ec009250 comp rx=0 tx=0).stop 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 -- 172.21.15.33:0/2706748527 shutdown_connections 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 --2- 172.21.15.33:0/2706748527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc06ab00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 --2- 172.21.15.33:0/2706748527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 -- 172.21.15.33:0/2706748527 >> 172.21.15.33:0/2706748527 conn(0x7f61fc06e0f0 msgr2=0x7f61fc0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 -- 172.21.15.33:0/2706748527 shutdown_connections 2023-10-06T05:15:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.491+0000 7f61fb59e700 1 -- 172.21.15.33:0/2706748527 wait complete. 2023-10-06T05:15:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.492+0000 7f61fb59e700 1 Processor -- start 2023-10-06T05:15:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.492+0000 7f61fb59e700 1 -- start start 2023-10-06T05:15:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc16cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0b0e10 0x7f61fc0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc16b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc16d480 con 0x7f61fc0b0a40 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc16a4b0 con 0x7f61fc0b0e10 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc16bea0 con 0x7f61fc15a0a0 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc16cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc16cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52062/0 (socket says 172.21.15.33:52062) 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.493+0000 7f61fa59c700 1 -- 172.21.15.33:0/3388623184 learned_addr learned my addr 172.21.15.33:0/3388623184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fa59c700 1 -- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 msgr2=0x7f61fc16b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fa59c700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc16b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fa59c700 1 -- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0b0e10 msgr2=0x7f61fc0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fa59c700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0b0e10 0x7f61fc0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fa59c700 1 -- 172.21.15.33:0/3388623184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0094f0 con 0x7f61fc0b0a40 2023-10-06T05:15:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fa59c700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc16cf40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ce290 tx=0x7f61ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec024440 con 0x7f61fc0b0a40 2023-10-06T05:15:25.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec0245c0 con 0x7f61fc0b0a40 2023-10-06T05:15:25.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc16c180 con 0x7f61fc0b0a40 2023-10-06T05:15:25.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.494+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec024980 con 0x7f61fc0b0a40 2023-10-06T05:15:25.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.495+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc16c740 con 0x7f61fc0b0a40 2023-10-06T05:15:25.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.495+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc06fcc0 con 0x7f61fc0b0a40 2023-10-06T05:15:25.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.496+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61ec01a420 con 0x7f61fc0b0a40 2023-10-06T05:15:25.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.497+0000 7f61ea7fc700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61dc066f30 0x7f61dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.497+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61ec0865b0 con 0x7f61fc0b0a40 2023-10-06T05:15:25.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.498+0000 7f61f9d9b700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61dc066f30 0x7f61dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.498+0000 7f61f9d9b700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61dc066f30 0x7f61dc0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61fc157700 tx=0x7f61f0007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.502+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61ec053620 con 0x7f61fc0b0a40 2023-10-06T05:15:25.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.706+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61fc04c380 con 0x7f61fc0b0a40 2023-10-06T05:15:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.707+0000 7f61ea7fc700 1 -- 172.21.15.33:0/3388623184 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f61ec056c40 con 0x7f61fc0b0a40 2023-10-06T05:15:25.709 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:25.709 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:24.300976+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:22.479742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:23.276900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61dc066f30 msgr2=0x7f61dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61dc066f30 0x7f61dc0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61fc157700 tx=0x7f61f0007b40 comp rx=0 tx=0).stop 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 msgr2=0x7f61fc16cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc16cf40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ce290 tx=0x7f61ec017000 comp rx=0 tx=0).stop 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 shutdown_connections 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61dc066f30 0x7f61dc0693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61fc15a0a0 0x7f61fc16b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61fc0b0a40 0x7f61fc16cf40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 --2- 172.21.15.33:0/3388623184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61fc0b0e10 0x7f61fc0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 >> 172.21.15.33:0/3388623184 conn(0x7f61fc06e0f0 msgr2=0x7f61fc0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 shutdown_connections 2023-10-06T05:15:25.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.710+0000 7f61fb59e700 1 -- 172.21.15.33:0/3388623184 wait complete. 2023-10-06T05:15:25.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:15:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.922+0000 7f9df4d9d700 1 Processor -- start 2023-10-06T05:15:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.922+0000 7f9df4d9d700 1 -- start start 2023-10-06T05:15:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.923+0000 7f9df4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.923+0000 7f9df4d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9df00cd5e0 0x7f9df015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.923+0000 7f9df4d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df015d660 con 0x7f9df015a650 2023-10-06T05:15:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.923+0000 7f9df4d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df00ce890 con 0x7f9df00cd5e0 2023-10-06T05:15:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.923+0000 7f9df4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df015aac0 con 0x7f9df00af540 2023-10-06T05:15:25.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9deffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9df00cd5e0 0x7f9df015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:47828/0 (socket says 172.21.15.33:47828) 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 -- 172.21.15.33:0/3414999656 learned_addr learned my addr 172.21.15.33:0/3414999656 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 -- 172.21.15.33:0/3414999656 >> v2:172.21.15.167:3300/0 conn(0x7f9df00cd5e0 msgr2=0x7f9df015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 --2- 172.21.15.33:0/3414999656 >> v2:172.21.15.167:3300/0 conn(0x7f9df00cd5e0 0x7f9df015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 -- 172.21.15.33:0/3414999656 >> v1:172.21.15.167:6789/0 conn(0x7f9df015a650 legacy=0x7f9df00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9def7fe700 1 -- 172.21.15.33:0/3414999656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df015d890 con 0x7f9df00af540 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.924+0000 7f9deffff700 1 --2- 172.21.15.33:0/3414999656 >> v2:172.21.15.167:3300/0 conn(0x7f9df00cd5e0 0x7f9df015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.925+0000 7f9def7fe700 1 --2- 172.21.15.33:0/3414999656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00af910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9de00099b0 tx=0x7f9de0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dd2015a758f8e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.925+0000 7f9dee7fc700 1 -- 172.21.15.33:0/3414999656 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0009ea0 con 0x7f9df00af540 2023-10-06T05:15:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.925+0000 7f9dee7fc700 1 -- 172.21.15.33:0/3414999656 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de000b860 con 0x7f9df00af540 2023-10-06T05:15:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.925+0000 7f9dee7fc700 1 -- 172.21.15.33:0/3414999656 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000bc00 con 0x7f9df00af540 2023-10-06T05:15:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.925+0000 7f9df4d9d700 1 -- 172.21.15.33:0/3414999656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 msgr2=0x7f9df00af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.925+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/3414999656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00af910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9de00099b0 tx=0x7f9de0009250 comp rx=0 tx=0).stop 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.926+0000 7f9df4d9d700 1 -- 172.21.15.33:0/3414999656 shutdown_connections 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.926+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/3414999656 >> v2:172.21.15.167:3300/0 conn(0x7f9df00cd5e0 0x7f9df015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.926+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/3414999656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00af910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.926+0000 7f9df4d9d700 1 -- 172.21.15.33:0/3414999656 >> 172.21.15.33:0/3414999656 conn(0x7f9df006d9b0 msgr2=0x7f9df006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.926+0000 7f9df4d9d700 1 -- 172.21.15.33:0/3414999656 shutdown_connections 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.926+0000 7f9df4d9d700 1 -- 172.21.15.33:0/3414999656 wait complete. 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.927+0000 7f9df4d9d700 1 Processor -- start 2023-10-06T05:15:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.927+0000 7f9df4d9d700 1 -- start start 2023-10-06T05:15:25.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9df4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9df4d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df00cd5e0 0x7f9df0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9df4d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 0x7f9df00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9df4d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b0e50 con 0x7f9df015a650 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9df4d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b1130 con 0x7f9df00cd5e0 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9df4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b1410 con 0x7f9df00af540 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.928+0000 7f9deffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 0x7f9df00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 0x7f9df00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52074/0 (socket says 172.21.15.33:52074) 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 -- 172.21.15.33:0/2889195183 learned_addr learned my addr 172.21.15.33:0/2889195183 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deeffd700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df00cd5e0 0x7f9df0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9def7fe700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 -- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 msgr2=0x7f9df00c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 -- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df00cd5e0 msgr2=0x7f9df0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df00cd5e0 0x7f9df0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 -- 172.21.15.33:0/2889195183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8009a20 con 0x7f9df015a650 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9deffff700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 0x7f9df00b4330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009f60 tx=0x7f9de800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.929+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800cb70 con 0x7f9df015a650 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.930+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de800f040 con 0x7f9df015a650 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.930+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de00094f0 con 0x7f9df015a650 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.930+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8014480 con 0x7f9df015a650 2023-10-06T05:15:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.930+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df00b19d0 con 0x7f9df015a650 2023-10-06T05:15:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.931+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df004c380 con 0x7f9df015a650 2023-10-06T05:15:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.932+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9de80147f0 con 0x7f9df015a650 2023-10-06T05:15:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.932+0000 7f9dd77fe700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ddc066f30 0x7f9ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.932+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9de80853c0 con 0x7f9df015a650 2023-10-06T05:15:25.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.933+0000 7f9def7fe700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ddc066f30 0x7f9ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.934+0000 7f9def7fe700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ddc066f30 0x7f9ddc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9de00099b0 tx=0x7f9de0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:25.936+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9de8052430 con 0x7f9df015a650 2023-10-06T05:15:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.184+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9df015db70 con 0x7f9df015a650 2023-10-06T05:15:26.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.844+0000 7f9dd77fe700 1 -- 172.21.15.33:0/2889195183 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9de8055a50 con 0x7f9df015a650 2023-10-06T05:15:26.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.847+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ddc066f30 msgr2=0x7f9ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.847+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ddc066f30 0x7f9ddc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9de00099b0 tx=0x7f9de0017000 comp rx=0 tx=0).stop 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.847+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 msgr2=0x7f9df00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.847+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 0x7f9df00b4330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009f60 tx=0x7f9de800b040 comp rx=0 tx=0).stop 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 shutdown_connections 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ddc066f30 0x7f9ddc0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9df015a650 0x7f9df00b4330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9df00af540 0x7f9df00c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 --2- 172.21.15.33:0/2889195183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9df00cd5e0 0x7f9df0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 >> 172.21.15.33:0/2889195183 conn(0x7f9df006d9b0 msgr2=0x7f9df00cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 shutdown_connections 2023-10-06T05:15:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:26.848+0000 7f9df4d9d700 1 -- 172.21.15.33:0/2889195183 wait complete. 2023-10-06T05:15:26.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:15:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.062+0000 7f902759e700 1 Processor -- start 2023-10-06T05:15:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.062+0000 7f902759e700 1 -- start start 2023-10-06T05:15:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.063+0000 7f902759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f902806c7b0 0x7f9028067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f902759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902815d3a0 con 0x7f90280aff90 2023-10-06T05:15:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f902759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90280b0470 con 0x7f902806cb80 2023-10-06T05:15:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f902759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90280a9fa0 con 0x7f902806c7b0 2023-10-06T05:15:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f9025d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f90280aff90 0x7f90280a9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60008/0 (socket says 172.21.15.33:60008) 2023-10-06T05:15:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f9025d9b700 1 -- 172.21.15.33:0/1509090904 learned_addr learned my addr 172.21.15.33:0/1509090904 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f902659c700 1 --2- 172.21.15.33:0/1509090904 >> v2:172.21.15.167:3300/0 conn(0x7f902806c7b0 0x7f9028067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167001313 0 0) 0x7f902815d3a0 con 0x7f90280aff90 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.064+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9010003570 con 0x7f90280aff90 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221182343 0 0) 0x7f9010003570 con 0x7f90280aff90 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 >> v2:172.21.15.167:3300/0 conn(0x7f902806c7b0 msgr2=0x7f9028067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 --2- 172.21.15.33:0/1509090904 >> v2:172.21.15.167:3300/0 conn(0x7f902806c7b0 0x7f9028067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 >> v1:172.21.15.167:6789/0 conn(0x7f902806cb80 legacy=0x7f902815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280b0750 con 0x7f90280aff90 2023-10-06T05:15:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f901c001470 con 0x7f90280aff90 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 >> v1:172.21.15.33:6789/0 conn(0x7f90280aff90 legacy=0x7f90280a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9010003ab0 0x7f9010003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9010004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9010005210 0x7f901000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90280a9fa0 con 0x7f9010004810 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901000c130 con 0x7f9010005210 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901000c2f0 con 0x7f9010003ab0 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9025d9b700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9010004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f902659c700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9010003ab0 0x7f9010003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9026d9d700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9010005210 0x7f901000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9025d9b700 1 -- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9010003ab0 msgr2=0x7f9010003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9025d9b700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9010003ab0 0x7f9010003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9025d9b700 1 -- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9010005210 msgr2=0x7f901000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9025d9b700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9010005210 0x7f901000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9025d9b700 1 -- 172.21.15.33:0/1509090904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9010003570 con 0x7f9010004810 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9026d9d700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9010005210 0x7f901000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.065+0000 7f9026d9d700 1 -- 172.21.15.33:0/1509090904 reap_dead start 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f9025d9b700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9010004c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f901c00c2a0 tx=0x7f901c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc16795dded14da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c00cb70 con 0x7f9010004810 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c00ccd0 con 0x7f9010004810 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902559a700 1 -- 172.21.15.33:0/1509090904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c005b70 con 0x7f9010004810 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902759e700 1 -- 172.21.15.33:0/1509090904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 msgr2=0x7f9010004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902759e700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9010004c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f901c00c2a0 tx=0x7f901c001d80 comp rx=0 tx=0).stop 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902759e700 1 -- 172.21.15.33:0/1509090904 shutdown_connections 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902759e700 1 --2- 172.21.15.33:0/1509090904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9010004c80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.066+0000 7f902759e700 1 -- 172.21.15.33:0/1509090904 >> 172.21.15.33:0/1509090904 conn(0x7f902806d670 msgr2=0x7f902806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.067+0000 7f902759e700 1 -- 172.21.15.33:0/1509090904 shutdown_connections 2023-10-06T05:15:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.067+0000 7f902759e700 1 -- 172.21.15.33:0/1509090904 wait complete. 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.068+0000 7f902759e700 1 Processor -- start 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.068+0000 7f902759e700 1 -- start start 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9028139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90280aeaf0 0x7f902814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90280ac6b0 0x7f90280acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90280ad090 con 0x7f9010004810 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028151090 con 0x7f90280ac6b0 2023-10-06T05:15:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9028150480 con 0x7f90280aeaf0 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9028139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9028139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52090/0 (socket says 172.21.15.33:52090) 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f902659c700 1 -- 172.21.15.33:0/3436490741 learned_addr learned my addr 172.21.15.33:0/3436490741 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f9026d9d700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90280ac6b0 0x7f90280acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.069+0000 7f9025d9b700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90280aeaf0 0x7f902814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902659c700 1 -- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90280aeaf0 msgr2=0x7f902814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902659c700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90280aeaf0 0x7f902814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902659c700 1 -- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90280ac6b0 msgr2=0x7f90280acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902659c700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90280ac6b0 0x7f90280acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902659c700 1 -- 172.21.15.33:0/3436490741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028150760 con 0x7f9010004810 2023-10-06T05:15:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f9026d9d700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90280ac6b0 0x7f90280acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902659c700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9028139740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f90180119c0 tx=0x7f9018008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018011e50 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901800f8e0 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9028150a40 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800fc60 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.070+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9028151520 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.071+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902804c380 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.072+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9018015670 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.073+0000 7f90167fc700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9008066f30 0x7f90080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.073+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9018084f90 con 0x7f9010004810 2023-10-06T05:15:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.074+0000 7f9025d9b700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9008066f30 0x7f90080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.075+0000 7f9025d9b700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9008066f30 0x7f90080693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f901c007c00 tx=0x7f901c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.077+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9018055a90 con 0x7f9010004810 2023-10-06T05:15:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.326+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f90280b0a30 con 0x7f9010004810 2023-10-06T05:15:27.689 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:15:27.685+0000 7f04abdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:15:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.849+0000 7f90167fc700 1 -- 172.21.15.33:0/3436490741 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9018055620 con 0x7f9010004810 2023-10-06T05:15:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9008066f30 msgr2=0x7f90080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9008066f30 0x7f90080693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f901c007c00 tx=0x7f901c015000 comp rx=0 tx=0).stop 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 msgr2=0x7f9028139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9028139740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f90180119c0 tx=0x7f9018008040 comp rx=0 tx=0).stop 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 shutdown_connections 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9008066f30 0x7f90080693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90280ac6b0 0x7f90280acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9010004810 0x7f9028139740 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 --2- 172.21.15.33:0/3436490741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90280aeaf0 0x7f902814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.852+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 >> 172.21.15.33:0/3436490741 conn(0x7f902806d670 msgr2=0x7f902814a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.853+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 shutdown_connections 2023-10-06T05:15:27.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:27.853+0000 7f902759e700 1 -- 172.21.15.33:0/3436490741 wait complete. 2023-10-06T05:15:27.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:28.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.061+0000 7fa7fe99d700 1 Processor -- start 2023-10-06T05:15:28.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.061+0000 7fa7fe99d700 1 -- start start 2023-10-06T05:15:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.062+0000 7fa7fe99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f806e140 0x7fa7f815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.062+0000 7fa7fe99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7f80afdd0 0x7fa7f80b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.062+0000 7fa7fe99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f806f220 con 0x7fa7f8159530 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.062+0000 7fa7fe99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f80b1e00 con 0x7fa7f80afdd0 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.062+0000 7fa7fe99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f806f6b0 con 0x7fa7f806e140 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fd99b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa7f8159530 0x7fa7f806da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60014/0 (socket says 172.21.15.33:60014) 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fd99b700 1 -- 172.21.15.33:0/1063547359 learned_addr learned my addr 172.21.15.33:0/1063547359 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fe19c700 1 --2- 172.21.15.33:0/1063547359 >> v2:172.21.15.167:3300/0 conn(0x7fa7f80afdd0 0x7fa7f80b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fd19a700 1 --2- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f806e140 0x7fa7f815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fc999700 1 -- 172.21.15.33:0/1063547359 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288730188 0 0) 0x7fa7f806f220 con 0x7fa7f8159530 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fc999700 1 -- 172.21.15.33:0/1063547359 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7e4003570 con 0x7fa7f8159530 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fe19c700 1 -- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f806e140 msgr2=0x7fa7f815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fe19c700 1 --2- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f806e140 0x7fa7f815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fe19c700 1 -- 172.21.15.33:0/1063547359 >> v1:172.21.15.33:6789/0 conn(0x7fa7f8159530 legacy=0x7fa7f806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.063+0000 7fa7fe19c700 1 -- 172.21.15.33:0/1063547359 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f815ae50 con 0x7fa7f80afdd0 2023-10-06T05:15:28.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe19c700 1 --2- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f80afdd0 0x7fa7f80b0210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009e60 tx=0x7fa7f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405bb64988cb965c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fc999700 1 -- 172.21.15.33:0/1063547359 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000e070 con 0x7fa7f80afdd0 2023-10-06T05:15:28.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fc999700 1 -- 172.21.15.33:0/1063547359 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000bab0 con 0x7fa7f80afdd0 2023-10-06T05:15:28.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fc999700 1 -- 172.21.15.33:0/1063547359 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000be30 con 0x7fa7f80afdd0 2023-10-06T05:15:28.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe99d700 1 -- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f80afdd0 msgr2=0x7fa7f80b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f80afdd0 0x7fa7f80b0210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009e60 tx=0x7fa7f0009780 comp rx=0 tx=0).stop 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe99d700 1 -- 172.21.15.33:0/1063547359 shutdown_connections 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f80afdd0 0x7fa7f80b0210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/1063547359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f806e140 0x7fa7f815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.064+0000 7fa7fe99d700 1 -- 172.21.15.33:0/1063547359 >> 172.21.15.33:0/1063547359 conn(0x7fa7f806c560 msgr2=0x7fa7f806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.065+0000 7fa7fe99d700 1 -- 172.21.15.33:0/1063547359 shutdown_connections 2023-10-06T05:15:28.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.065+0000 7fa7fe99d700 1 -- 172.21.15.33:0/1063547359 wait complete. 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.066+0000 7fa7fe99d700 1 Processor -- start 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.066+0000 7fa7fe99d700 1 -- start start 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fe99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f806e140 0x7fa7f80cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fe99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 0x7fa7f815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fe99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f8159530 0x7fa7f815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fe99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80c9a60 con 0x7fa7f80afdd0 2023-10-06T05:15:28.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fe99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80c9d40 con 0x7fa7f806e140 2023-10-06T05:15:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fe99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80ca020 con 0x7fa7f8159530 2023-10-06T05:15:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fd19a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 0x7fa7f815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fd19a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 0x7fa7f815f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52092/0 (socket says 172.21.15.33:52092) 2023-10-06T05:15:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fd19a700 1 -- 172.21.15.33:0/2654281338 learned_addr learned my addr 172.21.15.33:0/2654281338 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.067+0000 7fa7fd99b700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f806e140 0x7fa7f80cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fe19c700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f8159530 0x7fa7f815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fd19a700 1 -- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f8159530 msgr2=0x7fa7f815fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fd19a700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f8159530 0x7fa7f815fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fd19a700 1 -- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f806e140 msgr2=0x7fa7f80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fd19a700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f806e140 0x7fa7f80cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fd19a700 1 -- 172.21.15.33:0/2654281338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0009a20 con 0x7fa7f80afdd0 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fe19c700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f8159530 0x7fa7f815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fd19a700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 0x7fa7f815f740 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800e930 tx=0x7fa7e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8013680 con 0x7fa7f80afdd0 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e80137e0 con 0x7fa7f80afdd0 2023-10-06T05:15:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f80ca300 con 0x7fa7f80afdd0 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.068+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8013b80 con 0x7fa7f80afdd0 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.069+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f80cca10 con 0x7fa7f80afdd0 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.070+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f80ca950 con 0x7fa7f80afdd0 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.070+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7e800c460 con 0x7fa7f80afdd0 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.071+0000 7fa7edffb700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7dc066f30 0x7fa7dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.071+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7e80848e0 con 0x7fa7f80afdd0 2023-10-06T05:15:28.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.072+0000 7fa7fd99b700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7dc066f30 0x7fa7dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.072+0000 7fa7fd99b700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7dc066f30 0x7fa7dc0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80cef40 tx=0x7fa7f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.075+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7e8051950 con 0x7fa7f80afdd0 2023-10-06T05:15:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.324+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f804c380 con 0x7fa7f80afdd0 2023-10-06T05:15:28.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.324+0000 7fa7edffb700 1 -- 172.21.15.33:0/2654281338 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fa7e8054f70 con 0x7fa7f80afdd0 2023-10-06T05:15:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:28.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:24.301322+0000","modified":"2023-10-06T05:15:27.844263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6836/1338636713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1338636713},{"type":"v1","addr":"172.21.15.167:6836","nonce":1338636713}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:15:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7dc066f30 msgr2=0x7fa7dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7dc066f30 0x7fa7dc0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80cef40 tx=0x7fa7f4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 msgr2=0x7fa7f815f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 0x7fa7f815f740 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800e930 tx=0x7fa7e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 shutdown_connections 2023-10-06T05:15:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7dc066f30 0x7fa7dc0693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7f8159530 0x7fa7f815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7f806e140 0x7fa7f80cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 --2- 172.21.15.33:0/2654281338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7f80afdd0 0x7fa7f815f740 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 >> 172.21.15.33:0/2654281338 conn(0x7fa7f806c560 msgr2=0x7fa7f815b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 shutdown_connections 2023-10-06T05:15:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.327+0000 7fa7fe99d700 1 -- 172.21.15.33:0/2654281338 wait complete. 2023-10-06T05:15:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-10-06T05:15:28.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.532+0000 7f26bb59e700 1 Processor -- start 2023-10-06T05:15:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.532+0000 7f26bb59e700 1 -- start start 2023-10-06T05:15:28.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.533+0000 7f26bb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26bc0ccfc0 0x7f26bc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.533+0000 7f26bb59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26bc0cbd80 con 0x7f26bc0cc750 2023-10-06T05:15:28.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.533+0000 7f26bb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26bc0af5f0 con 0x7f26bc0cf3b0 2023-10-06T05:15:28.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.533+0000 7f26bb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc0af8d0 con 0x7f26bc0ccfc0 2023-10-06T05:15:28.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.533+0000 7f26ba59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26bc0ccfc0 0x7f26bc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26ba59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26bc0ccfc0 0x7f26bc0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52100/0 (socket says 172.21.15.33:52100) 2023-10-06T05:15:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26ba59c700 1 -- 172.21.15.33:0/1695638804 learned_addr learned my addr 172.21.15.33:0/1695638804 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26ba59c700 1 -- 172.21.15.33:0/1695638804 >> v1:172.21.15.167:6789/0 conn(0x7f26bc0cf3b0 legacy=0x7f26bc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26ba59c700 1 -- 172.21.15.33:0/1695638804 >> v1:172.21.15.33:6789/0 conn(0x7f26bc0cc750 legacy=0x7f26bc0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26ba59c700 1 -- 172.21.15.33:0/1695638804 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0afbb0 con 0x7f26bc0ccfc0 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26ba59c700 1 --2- 172.21.15.33:0/1695638804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0ccfc0 0x7f26bc0cc210 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0099e0 tx=0x7f26ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec28961d08c40e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.534+0000 7f26b959a700 1 -- 172.21.15.33:0/1695638804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00b820 con 0x7f26bc0ccfc0 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26b959a700 1 -- 172.21.15.33:0/1695638804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac00b980 con 0x7f26bc0ccfc0 2023-10-06T05:15:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26b959a700 1 -- 172.21.15.33:0/1695638804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00bd20 con 0x7f26bc0ccfc0 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 -- 172.21.15.33:0/1695638804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0ccfc0 msgr2=0x7f26bc0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1695638804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0ccfc0 0x7f26bc0cc210 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0099e0 tx=0x7f26ac009250 comp rx=0 tx=0).stop 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 -- 172.21.15.33:0/1695638804 shutdown_connections 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1695638804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0ccfc0 0x7f26bc0cc210 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 -- 172.21.15.33:0/1695638804 >> 172.21.15.33:0/1695638804 conn(0x7f26bc06cff0 msgr2=0x7f26bc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 -- 172.21.15.33:0/1695638804 shutdown_connections 2023-10-06T05:15:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.535+0000 7f26bb59e700 1 -- 172.21.15.33:0/1695638804 wait complete. 2023-10-06T05:15:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.537+0000 7f26bb59e700 1 Processor -- start 2023-10-06T05:15:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.537+0000 7f26bb59e700 1 -- start start 2023-10-06T05:15:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 0x7f26bc0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26bc0ccfc0 0x7f26bc0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc152980 0x7f26bc154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc169d90 con 0x7f26bc0cc750 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc16abe0 con 0x7f26bc152980 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc169fd0 con 0x7f26bc0ccfc0 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 0x7f26bc0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 0x7f26bc0c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52116/0 (socket says 172.21.15.33:52116) 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 -- 172.21.15.33:0/1097712591 learned_addr learned my addr 172.21.15.33:0/1097712591 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26bad9d700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc152980 0x7f26bc154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26b9d9b700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26bc0ccfc0 0x7f26bc0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 -- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26bc0ccfc0 msgr2=0x7f26bc0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26bc0ccfc0 0x7f26bc0b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 -- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc152980 msgr2=0x7f26bc154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc152980 0x7f26bc154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.538+0000 7f26ba59c700 1 -- 172.21.15.33:0/1097712591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0094f0 con 0x7f26bc0cc750 2023-10-06T05:15:28.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.539+0000 7f26ba59c700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 0x7f26bc0c75b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0094c0 tx=0x7f26ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.539+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01a420 con 0x7f26bc0cc750 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.539+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac01a5a0 con 0x7f26bc0cc750 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.539+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01a960 con 0x7f26bc0cc750 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.539+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc16a2b0 con 0x7f26bc0cc750 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.539+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc16a870 con 0x7f26bc0cc750 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.540+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc04c380 con 0x7f26bc0cc750 2023-10-06T05:15:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.541+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26ac01ad50 con 0x7f26bc0cc750 2023-10-06T05:15:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.542+0000 7f26aa7fc700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f269c066f30 0x7f269c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.543+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26ac086070 con 0x7f26bc0cc750 2023-10-06T05:15:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.543+0000 7f26b9d9b700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f269c066f30 0x7f269c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.544+0000 7f26b9d9b700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f269c066f30 0x7f269c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f26bc15f550 tx=0x7f26b0007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.547+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26ac0b75a0 con 0x7f26bc0cc750 2023-10-06T05:15:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.751+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26bc159bb0 con 0x7f26bc0cc750 2023-10-06T05:15:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.751+0000 7f26aa7fc700 1 -- 172.21.15.33:0/1097712591 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f26ac015020 con 0x7f26bc0cc750 2023-10-06T05:15:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:28.754 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:25.826634+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:22.479742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:23.276900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f269c066f30 msgr2=0x7f269c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f269c066f30 0x7f269c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f26bc15f550 tx=0x7f26b0007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 msgr2=0x7f26bc0c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 0x7f26bc0c75b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0094c0 tx=0x7f26ac017000 comp rx=0 tx=0).stop 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 shutdown_connections 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f269c066f30 0x7f269c0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc152980 0x7f26bc154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0cc750 0x7f26bc0c75b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 --2- 172.21.15.33:0/1097712591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26bc0ccfc0 0x7f26bc0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 >> 172.21.15.33:0/1097712591 conn(0x7f26bc06cff0 msgr2=0x7f26bc0c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 shutdown_connections 2023-10-06T05:15:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.754+0000 7f26bb59e700 1 -- 172.21.15.33:0/1097712591 wait complete. 2023-10-06T05:15:28.773 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:15:28.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:28.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.960+0000 7fa89ffff700 1 Processor -- start 2023-10-06T05:15:28.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.960+0000 7fa89ffff700 1 -- start start 2023-10-06T05:15:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a00c8050 0x7fa8a013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a00c9cb0 con 0x7fa8a013faa0 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00ca270 con 0x7fa8a00c8050 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a006b840 con 0x7fa8a006aad0 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a00c8050 0x7fa8a013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a00c8050 0x7fa8a013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52120/0 (socket says 172.21.15.33:52120) 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.961+0000 7fa89e7fc700 1 -- 172.21.15.33:0/3910017941 learned_addr learned my addr 172.21.15.33:0/3910017941 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89f7fe700 1 --2- 172.21.15.33:0/3910017941 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89e7fc700 1 -- 172.21.15.33:0/3910017941 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 msgr2=0x7fa8a006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89e7fc700 1 --2- 172.21.15.33:0/3910017941 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89e7fc700 1 -- 172.21.15.33:0/3910017941 >> v1:172.21.15.167:6789/0 conn(0x7fa8a013faa0 legacy=0x7fa8a00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89e7fc700 1 -- 172.21.15.33:0/3910017941 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a006bb20 con 0x7fa8a00c8050 2023-10-06T05:15:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89e7fc700 1 --2- 172.21.15.33:0/3910017941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00c8050 0x7fa8a013fe70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0156b90 tx=0x7fa89400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330725a5e8558ba0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.962+0000 7fa89dffb700 1 -- 172.21.15.33:0/3910017941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89400dc50 con 0x7fa8a00c8050 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89dffb700 1 -- 172.21.15.33:0/3910017941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89400ddb0 con 0x7fa8a00c8050 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89dffb700 1 -- 172.21.15.33:0/3910017941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8940196d0 con 0x7fa8a00c8050 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 -- 172.21.15.33:0/3910017941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00c8050 msgr2=0x7fa8a013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3910017941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00c8050 0x7fa8a013fe70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0156b90 tx=0x7fa89400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 -- 172.21.15.33:0/3910017941 shutdown_connections 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3910017941 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3910017941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00c8050 0x7fa8a013fe70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 -- 172.21.15.33:0/3910017941 >> 172.21.15.33:0/3910017941 conn(0x7fa8a006d030 msgr2=0x7fa8a006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 -- 172.21.15.33:0/3910017941 shutdown_connections 2023-10-06T05:15:28.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.963+0000 7fa89ffff700 1 -- 172.21.15.33:0/3910017941 wait complete. 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.965+0000 7fa89ffff700 1 Processor -- start 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.965+0000 7fa89ffff700 1 -- start start 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a00c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c8050 0x7fa8a00c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c5090 con 0x7fa8a013faa0 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c5370 con 0x7fa8a006aad0 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c5650 con 0x7fa8a00c8050 2023-10-06T05:15:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52124/0 (socket says 172.21.15.33:52124) 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3787271223 learned_addr learned my addr 172.21.15.33:0/3787271223 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89effd700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a00c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89e7fc700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c8050 0x7fa8a00c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c8050 msgr2=0x7fa8a00c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c8050 0x7fa8a00c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 msgr2=0x7fa8a00c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a00c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.966+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3787271223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89400b790 con 0x7fa8a013faa0 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.967+0000 7fa89f7fe700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0155af0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa898009fa0 tx=0x7fa898009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.967+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89800ebc0 con 0x7fa8a013faa0 2023-10-06T05:15:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.967+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89800ed20 con 0x7fa8a013faa0 2023-10-06T05:15:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.967+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a00c5930 con 0x7fa8a013faa0 2023-10-06T05:15:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.967+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898014670 con 0x7fa8a013faa0 2023-10-06T05:15:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.967+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a00c4020 con 0x7fa8a013faa0 2023-10-06T05:15:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.968+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00c5e50 con 0x7fa8a013faa0 2023-10-06T05:15:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.969+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8980149e0 con 0x7fa8a013faa0 2023-10-06T05:15:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.970+0000 7fa886ffd700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa88c066f30 0x7fa88c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.970+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa898084d90 con 0x7fa8a013faa0 2023-10-06T05:15:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.970+0000 7fa89effd700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa88c066f30 0x7fa88c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.971+0000 7fa89effd700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa88c066f30 0x7fa88c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00c9650 tx=0x7fa890009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:28.974+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8a00c5e50 con 0x7fa8a013faa0 2023-10-06T05:15:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.171+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8a00c5e50 con 0x7fa88c066f30 2023-10-06T05:15:29.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.181+0000 7fa886ffd700 1 -- 172.21.15.33:0/3787271223 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310296 (secure 0 0 0) 0x7fa8a00c5e50 con 0x7fa88c066f30 2023-10-06T05:15:29.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa88c066f30 msgr2=0x7fa88c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa88c066f30 0x7fa88c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00c9650 tx=0x7fa890009040 comp rx=0 tx=0).stop 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 msgr2=0x7fa8a0155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0155af0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa898009fa0 tx=0x7fa898009900 comp rx=0 tx=0).stop 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 shutdown_connections 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa88c066f30 0x7fa88c0693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0155af0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a00c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 --2- 172.21.15.33:0/3787271223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c8050 0x7fa8a00c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 >> 172.21.15.33:0/3787271223 conn(0x7fa8a006d030 msgr2=0x7fa8a00cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 shutdown_connections 2023-10-06T05:15:29.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.185+0000 7fa89ffff700 1 -- 172.21.15.33:0/3787271223 wait complete. 2023-10-06T05:15:29.190 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:15:29.204 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-10-06T05:15:27.831400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382680,"kb_used_data":4376,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592040,"statfs":{"total":773094113280,"available":770654248960,"internally_reserved":0,"allocated":4481024,"data_stored":2511664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.031934"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839597+0000","last_change":"2023-10-06T05:15:24.834229+0000","last_active":"2023-10-06T05:15:24.839597+0000","last_peered":"2023-10-06T05:15:24.839597+0000","last_clean":"2023-10-06T05:15:24.839597+0000","last_became_active":"2023-10-06T05:15:24.833960+0000","last_became_peered":"2023-10-06T05:15:24.833960+0000","last_unstale":"2023-10-06T05:15:24.839597+0000","last_undegraded":"2023-10-06T05:15:24.839597+0000","last_fullsized":"2023-10-06T05:15:24.839597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:47.981780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.835502+0000","last_change":"2023-10-06T05:15:24.829090+0000","last_active":"2023-10-06T05:15:24.835502+0000","last_peered":"2023-10-06T05:15:24.835502+0000","last_clean":"2023-10-06T05:15:24.835502+0000","last_became_active":"2023-10-06T05:15:24.828802+0000","last_became_peered":"2023-10-06T05:15:24.828802+0000","last_unstale":"2023-10-06T05:15:24.835502+0000","last_undegraded":"2023-10-06T05:15:24.835502+0000","last_fullsized":"2023-10-06T05:15:24.835502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:29.175187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.849845+0000","last_change":"2023-10-06T05:15:24.849845+0000","last_active":"2023-10-06T05:15:24.849845+0000","last_peered":"2023-10-06T05:15:24.849845+0000","last_clean":"2023-10-06T05:15:24.849845+0000","last_became_active":"2023-10-06T05:15:24.849568+0000","last_became_peered":"2023-10-06T05:15:24.849568+0000","last_unstale":"2023-10-06T05:15:24.849845+0000","last_undegraded":"2023-10-06T05:15:24.849845+0000","last_fullsized":"2023-10-06T05:15:24.849845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.851692+0000","last_change":"2023-10-06T05:15:24.849012+0000","last_active":"2023-10-06T05:15:24.851692+0000","last_peered":"2023-10-06T05:15:24.851692+0000","last_clean":"2023-10-06T05:15:24.851692+0000","last_became_active":"2023-10-06T05:15:24.848735+0000","last_became_peered":"2023-10-06T05:15:24.848735+0000","last_unstale":"2023-10-06T05:15:24.851692+0000","last_undegraded":"2023-10-06T05:15:24.851692+0000","last_fullsized":"2023-10-06T05:15:24.851692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.352940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843698+0000","last_change":"2023-10-06T05:15:24.842052+0000","last_active":"2023-10-06T05:15:24.843698+0000","last_peered":"2023-10-06T05:15:24.843698+0000","last_clean":"2023-10-06T05:15:24.843698+0000","last_became_active":"2023-10-06T05:15:24.841746+0000","last_became_peered":"2023-10-06T05:15:24.841746+0000","last_unstale":"2023-10-06T05:15:24.843698+0000","last_undegraded":"2023-10-06T05:15:24.843698+0000","last_fullsized":"2023-10-06T05:15:24.843698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:51.116140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.833658+0000","last_change":"2023-10-06T05:15:24.833658+0000","last_active":"2023-10-06T05:15:24.833658+0000","last_peered":"2023-10-06T05:15:24.833658+0000","last_clean":"2023-10-06T05:15:24.833658+0000","last_became_active":"2023-10-06T05:15:24.833305+0000","last_became_peered":"2023-10-06T05:15:24.833305+0000","last_unstale":"2023-10-06T05:15:24.833658+0000","last_undegraded":"2023-10-06T05:15:24.833658+0000","last_fullsized":"2023-10-06T05:15:24.833658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:18.757351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.850871+0000","last_change":"2023-10-06T05:15:24.843795+0000","last_active":"2023-10-06T05:15:24.850871+0000","last_peered":"2023-10-06T05:15:24.850871+0000","last_clean":"2023-10-06T05:15:24.850871+0000","last_became_active":"2023-10-06T05:15:24.843169+0000","last_became_peered":"2023-10-06T05:15:24.843169+0000","last_unstale":"2023-10-06T05:15:24.850871+0000","last_undegraded":"2023-10-06T05:15:24.850871+0000","last_fullsized":"2023-10-06T05:15:24.850871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845830+0000","last_change":"2023-10-06T05:15:24.842782+0000","last_active":"2023-10-06T05:15:24.845830+0000","last_peered":"2023-10-06T05:15:24.845830+0000","last_clean":"2023-10-06T05:15:24.845830+0000","last_became_active":"2023-10-06T05:15:24.842516+0000","last_became_peered":"2023-10-06T05:15:24.842516+0000","last_unstale":"2023-10-06T05:15:24.845830+0000","last_undegraded":"2023-10-06T05:15:24.845830+0000","last_fullsized":"2023-10-06T05:15:24.845830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.849099+0000","last_change":"2023-10-06T05:15:24.849099+0000","last_active":"2023-10-06T05:15:24.849099+0000","last_peered":"2023-10-06T05:15:24.849099+0000","last_clean":"2023-10-06T05:15:24.849099+0000","last_became_active":"2023-10-06T05:15:24.848817+0000","last_became_peered":"2023-10-06T05:15:24.848817+0000","last_unstale":"2023-10-06T05:15:24.849099+0000","last_undegraded":"2023-10-06T05:15:24.849099+0000","last_fullsized":"2023-10-06T05:15:24.849099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845307+0000","last_change":"2023-10-06T05:15:24.842313+0000","last_active":"2023-10-06T05:15:24.845307+0000","last_peered":"2023-10-06T05:15:24.845307+0000","last_clean":"2023-10-06T05:15:24.845307+0000","last_became_active":"2023-10-06T05:15:24.841715+0000","last_became_peered":"2023-10-06T05:15:24.841715+0000","last_unstale":"2023-10-06T05:15:24.845307+0000","last_undegraded":"2023-10-06T05:15:24.845307+0000","last_fullsized":"2023-10-06T05:15:24.845307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.686660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863441+0000","last_change":"2023-10-06T05:15:24.859053+0000","last_active":"2023-10-06T05:15:24.863441+0000","last_peered":"2023-10-06T05:15:24.863441+0000","last_clean":"2023-10-06T05:15:24.863441+0000","last_became_active":"2023-10-06T05:15:24.858752+0000","last_became_peered":"2023-10-06T05:15:24.858752+0000","last_unstale":"2023-10-06T05:15:24.863441+0000","last_undegraded":"2023-10-06T05:15:24.863441+0000","last_fullsized":"2023-10-06T05:15:24.863441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:07.029872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.828348+0000","last_change":"2023-10-06T05:15:24.828348+0000","last_active":"2023-10-06T05:15:24.828348+0000","last_peered":"2023-10-06T05:15:24.828348+0000","last_clean":"2023-10-06T05:15:24.828348+0000","last_became_active":"2023-10-06T05:15:24.828028+0000","last_became_peered":"2023-10-06T05:15:24.828028+0000","last_unstale":"2023-10-06T05:15:24.828348+0000","last_undegraded":"2023-10-06T05:15:24.828348+0000","last_fullsized":"2023-10-06T05:15:24.828348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845425+0000","last_change":"2023-10-06T05:15:24.842587+0000","last_active":"2023-10-06T05:15:24.845425+0000","last_peered":"2023-10-06T05:15:24.845425+0000","last_clean":"2023-10-06T05:15:24.845425+0000","last_became_active":"2023-10-06T05:15:24.841831+0000","last_became_peered":"2023-10-06T05:15:24.841831+0000","last_unstale":"2023-10-06T05:15:24.845425+0000","last_undegraded":"2023-10-06T05:15:24.845425+0000","last_fullsized":"2023-10-06T05:15:24.845425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.848254+0000","last_change":"2023-10-06T05:15:24.848254+0000","last_active":"2023-10-06T05:15:24.848254+0000","last_peered":"2023-10-06T05:15:24.848254+0000","last_clean":"2023-10-06T05:15:24.848254+0000","last_became_active":"2023-10-06T05:15:24.847953+0000","last_became_peered":"2023-10-06T05:15:24.847953+0000","last_unstale":"2023-10-06T05:15:24.848254+0000","last_undegraded":"2023-10-06T05:15:24.848254+0000","last_fullsized":"2023-10-06T05:15:24.848254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.850890+0000","last_change":"2023-10-06T05:15:24.843961+0000","last_active":"2023-10-06T05:15:24.850890+0000","last_peered":"2023-10-06T05:15:24.850890+0000","last_clean":"2023-10-06T05:15:24.850890+0000","last_became_active":"2023-10-06T05:15:24.843473+0000","last_became_peered":"2023-10-06T05:15:24.843473+0000","last_unstale":"2023-10-06T05:15:24.850890+0000","last_undegraded":"2023-10-06T05:15:24.850890+0000","last_fullsized":"2023-10-06T05:15:24.850890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:47.981780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845979+0000","last_change":"2023-10-06T05:15:24.845979+0000","last_active":"2023-10-06T05:15:24.845979+0000","last_peered":"2023-10-06T05:15:24.845979+0000","last_clean":"2023-10-06T05:15:24.845979+0000","last_became_active":"2023-10-06T05:15:24.845668+0000","last_became_peered":"2023-10-06T05:15:24.845668+0000","last_unstale":"2023-10-06T05:15:24.845979+0000","last_undegraded":"2023-10-06T05:15:24.845979+0000","last_fullsized":"2023-10-06T05:15:24.845979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:29.740946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844930+0000","last_change":"2023-10-06T05:15:24.844930+0000","last_active":"2023-10-06T05:15:24.844930+0000","last_peered":"2023-10-06T05:15:24.844930+0000","last_clean":"2023-10-06T05:15:24.844930+0000","last_became_active":"2023-10-06T05:15:24.844647+0000","last_became_peered":"2023-10-06T05:15:24.844647+0000","last_unstale":"2023-10-06T05:15:24.844930+0000","last_undegraded":"2023-10-06T05:15:24.844930+0000","last_fullsized":"2023-10-06T05:15:24.844930+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:07.029872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.857181+0000","last_change":"2023-10-06T05:15:24.851244+0000","last_active":"2023-10-06T05:15:24.857181+0000","last_peered":"2023-10-06T05:15:24.857181+0000","last_clean":"2023-10-06T05:15:24.857181+0000","last_became_active":"2023-10-06T05:15:24.850889+0000","last_became_peered":"2023-10-06T05:15:24.850889+0000","last_unstale":"2023-10-06T05:15:24.857181+0000","last_undegraded":"2023-10-06T05:15:24.857181+0000","last_fullsized":"2023-10-06T05:15:24.857181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:40.098336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.835666+0000","last_change":"2023-10-06T05:15:24.829961+0000","last_active":"2023-10-06T05:15:24.835666+0000","last_peered":"2023-10-06T05:15:24.835666+0000","last_clean":"2023-10-06T05:15:24.835666+0000","last_became_active":"2023-10-06T05:15:24.829560+0000","last_became_peered":"2023-10-06T05:15:24.829560+0000","last_unstale":"2023-10-06T05:15:24.835666+0000","last_undegraded":"2023-10-06T05:15:24.835666+0000","last_fullsized":"2023-10-06T05:15:24.835666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:49.108167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.857311+0000","last_change":"2023-10-06T05:15:24.851300+0000","last_active":"2023-10-06T05:15:24.857311+0000","last_peered":"2023-10-06T05:15:24.857311+0000","last_clean":"2023-10-06T05:15:24.857311+0000","last_became_active":"2023-10-06T05:15:24.851070+0000","last_became_peered":"2023-10-06T05:15:24.851070+0000","last_unstale":"2023-10-06T05:15:24.857311+0000","last_undegraded":"2023-10-06T05:15:24.857311+0000","last_fullsized":"2023-10-06T05:15:24.857311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:06.324525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845444+0000","last_change":"2023-10-06T05:15:24.842817+0000","last_active":"2023-10-06T05:15:24.845444+0000","last_peered":"2023-10-06T05:15:24.845444+0000","last_clean":"2023-10-06T05:15:24.845444+0000","last_became_active":"2023-10-06T05:15:24.841730+0000","last_became_peered":"2023-10-06T05:15:24.841730+0000","last_unstale":"2023-10-06T05:15:24.845444+0000","last_undegraded":"2023-10-06T05:15:24.845444+0000","last_fullsized":"2023-10-06T05:15:24.845444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:01.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846778+0000","last_change":"2023-10-06T05:15:24.846778+0000","last_active":"2023-10-06T05:15:24.846778+0000","last_peered":"2023-10-06T05:15:24.846778+0000","last_clean":"2023-10-06T05:15:24.846778+0000","last_became_active":"2023-10-06T05:15:24.846547+0000","last_became_peered":"2023-10-06T05:15:24.846547+0000","last_unstale":"2023-10-06T05:15:24.846778+0000","last_undegraded":"2023-10-06T05:15:24.846778+0000","last_fullsized":"2023-10-06T05:15:24.846778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:47.981780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843625+0000","last_change":"2023-10-06T05:15:24.843625+0000","last_active":"2023-10-06T05:15:24.843625+0000","last_peered":"2023-10-06T05:15:24.843625+0000","last_clean":"2023-10-06T05:15:24.843625+0000","last_became_active":"2023-10-06T05:15:24.841683+0000","last_became_peered":"2023-10-06T05:15:24.841683+0000","last_unstale":"2023-10-06T05:15:24.843625+0000","last_undegraded":"2023-10-06T05:15:24.843625+0000","last_fullsized":"2023-10-06T05:15:24.843625+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843508+0000","last_change":"2023-10-06T05:15:24.837480+0000","last_active":"2023-10-06T05:15:24.843508+0000","last_peered":"2023-10-06T05:15:24.843508+0000","last_clean":"2023-10-06T05:15:24.843508+0000","last_became_active":"2023-10-06T05:15:24.837094+0000","last_became_peered":"2023-10-06T05:15:24.837094+0000","last_unstale":"2023-10-06T05:15:24.843508+0000","last_undegraded":"2023-10-06T05:15:24.843508+0000","last_fullsized":"2023-10-06T05:15:24.843508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:57.105297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843333+0000","last_change":"2023-10-06T05:15:24.830962+0000","last_active":"2023-10-06T05:15:24.843333+0000","last_peered":"2023-10-06T05:15:24.843333+0000","last_clean":"2023-10-06T05:15:24.843333+0000","last_became_active":"2023-10-06T05:15:24.830644+0000","last_became_peered":"2023-10-06T05:15:24.830644+0000","last_unstale":"2023-10-06T05:15:24.843333+0000","last_undegraded":"2023-10-06T05:15:24.843333+0000","last_fullsized":"2023-10-06T05:15:24.843333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:49.108167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844750+0000","last_change":"2023-10-06T05:15:24.844750+0000","last_active":"2023-10-06T05:15:24.844750+0000","last_peered":"2023-10-06T05:15:24.844750+0000","last_clean":"2023-10-06T05:15:24.844750+0000","last_became_active":"2023-10-06T05:15:24.844454+0000","last_became_peered":"2023-10-06T05:15:24.844454+0000","last_unstale":"2023-10-06T05:15:24.844750+0000","last_undegraded":"2023-10-06T05:15:24.844750+0000","last_fullsized":"2023-10-06T05:15:24.844750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.686660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843327+0000","last_change":"2023-10-06T05:15:24.834415+0000","last_active":"2023-10-06T05:15:24.843327+0000","last_peered":"2023-10-06T05:15:24.843327+0000","last_clean":"2023-10-06T05:15:24.843327+0000","last_became_active":"2023-10-06T05:15:24.834113+0000","last_became_peered":"2023-10-06T05:15:24.834113+0000","last_unstale":"2023-10-06T05:15:24.843327+0000","last_undegraded":"2023-10-06T05:15:24.843327+0000","last_fullsized":"2023-10-06T05:15:24.843327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:40.366886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845363+0000","last_change":"2023-10-06T05:15:24.842623+0000","last_active":"2023-10-06T05:15:24.845363+0000","last_peered":"2023-10-06T05:15:24.845363+0000","last_clean":"2023-10-06T05:15:24.845363+0000","last_became_active":"2023-10-06T05:15:24.841851+0000","last_became_peered":"2023-10-06T05:15:24.841851+0000","last_unstale":"2023-10-06T05:15:24.845363+0000","last_undegraded":"2023-10-06T05:15:24.845363+0000","last_fullsized":"2023-10-06T05:15:24.845363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843899+0000","last_change":"2023-10-06T05:15:24.832222+0000","last_active":"2023-10-06T05:15:24.843899+0000","last_peered":"2023-10-06T05:15:24.843899+0000","last_clean":"2023-10-06T05:15:24.843899+0000","last_became_active":"2023-10-06T05:15:24.831908+0000","last_became_peered":"2023-10-06T05:15:24.831908+0000","last_unstale":"2023-10-06T05:15:24.843899+0000","last_undegraded":"2023-10-06T05:15:24.843899+0000","last_fullsized":"2023-10-06T05:15:24.843899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:51.116140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843253+0000","last_change":"2023-10-06T05:15:24.839018+0000","last_active":"2023-10-06T05:15:24.843253+0000","last_peered":"2023-10-06T05:15:24.843253+0000","last_clean":"2023-10-06T05:15:24.843253+0000","last_became_active":"2023-10-06T05:15:24.838354+0000","last_became_peered":"2023-10-06T05:15:24.838354+0000","last_unstale":"2023-10-06T05:15:24.843253+0000","last_undegraded":"2023-10-06T05:15:24.843253+0000","last_fullsized":"2023-10-06T05:15:24.843253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:04.635140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.838007+0000","last_change":"2023-10-06T05:15:24.838007+0000","last_active":"2023-10-06T05:15:24.838007+0000","last_peered":"2023-10-06T05:15:24.838007+0000","last_clean":"2023-10-06T05:15:24.838007+0000","last_became_active":"2023-10-06T05:15:24.837742+0000","last_became_peered":"2023-10-06T05:15:24.837742+0000","last_unstale":"2023-10-06T05:15:24.838007+0000","last_undegraded":"2023-10-06T05:15:24.838007+0000","last_fullsized":"2023-10-06T05:15:24.838007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.834712+0000","last_change":"2023-10-06T05:15:24.834712+0000","last_active":"2023-10-06T05:15:24.834712+0000","last_peered":"2023-10-06T05:15:24.834712+0000","last_clean":"2023-10-06T05:15:24.834712+0000","last_became_active":"2023-10-06T05:15:24.834457+0000","last_became_peered":"2023-10-06T05:15:24.834457+0000","last_unstale":"2023-10-06T05:15:24.834712+0000","last_undegraded":"2023-10-06T05:15:24.834712+0000","last_fullsized":"2023-10-06T05:15:24.834712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:59.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844484+0000","last_change":"2023-10-06T05:15:24.844484+0000","last_active":"2023-10-06T05:15:24.844484+0000","last_peered":"2023-10-06T05:15:24.844484+0000","last_clean":"2023-10-06T05:15:24.844484+0000","last_became_active":"2023-10-06T05:15:24.844204+0000","last_became_peered":"2023-10-06T05:15:24.844204+0000","last_unstale":"2023-10-06T05:15:24.844484+0000","last_undegraded":"2023-10-06T05:15:24.844484+0000","last_fullsized":"2023-10-06T05:15:24.844484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:38.333907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.861458+0000","last_change":"2023-10-06T05:15:23.823585+0000","last_active":"2023-10-06T05:15:24.861458+0000","last_peered":"2023-10-06T05:15:24.861458+0000","last_clean":"2023-10-06T05:15:24.861458+0000","last_became_active":"2023-10-06T05:15:23.823272+0000","last_became_peered":"2023-10-06T05:15:23.823272+0000","last_unstale":"2023-10-06T05:15:24.861458+0000","last_undegraded":"2023-10-06T05:15:24.861458+0000","last_fullsized":"2023-10-06T05:15:24.861458+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844394+0000","last_change":"2023-10-06T05:15:24.844394+0000","last_active":"2023-10-06T05:15:24.844394+0000","last_peered":"2023-10-06T05:15:24.844394+0000","last_clean":"2023-10-06T05:15:24.844394+0000","last_became_active":"2023-10-06T05:15:24.844116+0000","last_became_peered":"2023-10-06T05:15:24.844116+0000","last_unstale":"2023-10-06T05:15:24.844394+0000","last_undegraded":"2023-10-06T05:15:24.844394+0000","last_fullsized":"2023-10-06T05:15:24.844394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:14.757703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.816386+0000","last_change":"2023-10-06T05:15:23.817646+0000","last_active":"2023-10-06T05:15:24.816386+0000","last_peered":"2023-10-06T05:15:24.816386+0000","last_clean":"2023-10-06T05:15:24.816386+0000","last_became_active":"2023-10-06T05:15:23.817373+0000","last_became_peered":"2023-10-06T05:15:23.817373+0000","last_unstale":"2023-10-06T05:15:24.816386+0000","last_undegraded":"2023-10-06T05:15:24.816386+0000","last_fullsized":"2023-10-06T05:15:24.816386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:09.050751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846926+0000","last_change":"2023-10-06T05:15:24.846926+0000","last_active":"2023-10-06T05:15:24.846926+0000","last_peered":"2023-10-06T05:15:24.846926+0000","last_clean":"2023-10-06T05:15:24.846926+0000","last_became_active":"2023-10-06T05:15:24.846624+0000","last_became_peered":"2023-10-06T05:15:24.846624+0000","last_unstale":"2023-10-06T05:15:24.846926+0000","last_undegraded":"2023-10-06T05:15:24.846926+0000","last_fullsized":"2023-10-06T05:15:24.846926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:01.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.860349+0000","last_change":"2023-10-06T05:15:23.827475+0000","last_active":"2023-10-06T05:15:24.860349+0000","last_peered":"2023-10-06T05:15:24.860349+0000","last_clean":"2023-10-06T05:15:24.860349+0000","last_became_active":"2023-10-06T05:15:23.827183+0000","last_became_peered":"2023-10-06T05:15:23.827183+0000","last_unstale":"2023-10-06T05:15:24.860349+0000","last_undegraded":"2023-10-06T05:15:24.860349+0000","last_fullsized":"2023-10-06T05:15:24.860349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:39.353055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843230+0000","last_change":"2023-10-06T05:15:24.839504+0000","last_active":"2023-10-06T05:15:24.843230+0000","last_peered":"2023-10-06T05:15:24.843230+0000","last_clean":"2023-10-06T05:15:24.843230+0000","last_became_active":"2023-10-06T05:15:24.839085+0000","last_became_peered":"2023-10-06T05:15:24.839085+0000","last_unstale":"2023-10-06T05:15:24.843230+0000","last_undegraded":"2023-10-06T05:15:24.843230+0000","last_fullsized":"2023-10-06T05:15:24.843230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:12.285635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.859930+0000","last_change":"2023-10-06T05:15:23.821228+0000","last_active":"2023-10-06T05:15:24.859930+0000","last_peered":"2023-10-06T05:15:24.859930+0000","last_clean":"2023-10-06T05:15:24.859930+0000","last_became_active":"2023-10-06T05:15:23.820965+0000","last_became_peered":"2023-10-06T05:15:23.820965+0000","last_unstale":"2023-10-06T05:15:24.859930+0000","last_undegraded":"2023-10-06T05:15:24.859930+0000","last_fullsized":"2023-10-06T05:15:24.859930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.751768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.832128+0000","last_change":"2023-10-06T05:15:24.832128+0000","last_active":"2023-10-06T05:15:24.832128+0000","last_peered":"2023-10-06T05:15:24.832128+0000","last_clean":"2023-10-06T05:15:24.832128+0000","last_became_active":"2023-10-06T05:15:24.831864+0000","last_became_peered":"2023-10-06T05:15:24.831864+0000","last_unstale":"2023-10-06T05:15:24.832128+0000","last_undegraded":"2023-10-06T05:15:24.832128+0000","last_fullsized":"2023-10-06T05:15:24.832128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:07.029872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863351+0000","last_change":"2023-10-06T05:15:23.821870+0000","last_active":"2023-10-06T05:15:24.863351+0000","last_peered":"2023-10-06T05:15:24.863351+0000","last_clean":"2023-10-06T05:15:24.863351+0000","last_became_active":"2023-10-06T05:15:23.821587+0000","last_became_peered":"2023-10-06T05:15:23.821587+0000","last_unstale":"2023-10-06T05:15:24.863351+0000","last_undegraded":"2023-10-06T05:15:24.863351+0000","last_fullsized":"2023-10-06T05:15:24.863351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:35.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.847422+0000","last_change":"2023-10-06T05:15:24.842903+0000","last_active":"2023-10-06T05:15:24.847422+0000","last_peered":"2023-10-06T05:15:24.847422+0000","last_clean":"2023-10-06T05:15:24.847422+0000","last_became_active":"2023-10-06T05:15:24.841747+0000","last_became_peered":"2023-10-06T05:15:24.841747+0000","last_unstale":"2023-10-06T05:15:24.847422+0000","last_undegraded":"2023-10-06T05:15:24.847422+0000","last_fullsized":"2023-10-06T05:15:24.847422+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:32.781863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.830409+0000","last_change":"2023-10-06T05:15:23.816886+0000","last_active":"2023-10-06T05:15:24.830409+0000","last_peered":"2023-10-06T05:15:24.830409+0000","last_clean":"2023-10-06T05:15:24.830409+0000","last_became_active":"2023-10-06T05:15:23.816540+0000","last_became_peered":"2023-10-06T05:15:23.816540+0000","last_unstale":"2023-10-06T05:15:24.830409+0000","last_undegraded":"2023-10-06T05:15:24.830409+0000","last_fullsized":"2023-10-06T05:15:24.830409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.853398+0000","last_change":"2023-10-06T05:15:24.853398+0000","last_active":"2023-10-06T05:15:24.853398+0000","last_peered":"2023-10-06T05:15:24.853398+0000","last_clean":"2023-10-06T05:15:24.853398+0000","last_became_active":"2023-10-06T05:15:24.853116+0000","last_became_peered":"2023-10-06T05:15:24.853116+0000","last_unstale":"2023-10-06T05:15:24.853398+0000","last_undegraded":"2023-10-06T05:15:24.853398+0000","last_fullsized":"2023-10-06T05:15:24.853398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:12.285635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832878+0000","last_change":"2023-10-06T05:15:23.829293+0000","last_active":"2023-10-06T05:15:23.832878+0000","last_peered":"2023-10-06T05:15:23.832878+0000","last_clean":"2023-10-06T05:15:23.832878+0000","last_became_active":"2023-10-06T05:15:23.828962+0000","last_became_peered":"2023-10-06T05:15:23.828962+0000","last_unstale":"2023-10-06T05:15:23.832878+0000","last_undegraded":"2023-10-06T05:15:23.832878+0000","last_fullsized":"2023-10-06T05:15:23.832878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:35.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839234+0000","last_change":"2023-10-06T05:15:24.835973+0000","last_active":"2023-10-06T05:15:24.839234+0000","last_peered":"2023-10-06T05:15:24.839234+0000","last_clean":"2023-10-06T05:15:24.839234+0000","last_became_active":"2023-10-06T05:15:24.835583+0000","last_became_peered":"2023-10-06T05:15:24.835583+0000","last_unstale":"2023-10-06T05:15:24.839234+0000","last_undegraded":"2023-10-06T05:15:24.839234+0000","last_fullsized":"2023-10-06T05:15:24.839234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:01.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832390+0000","last_change":"2023-10-06T05:15:23.828406+0000","last_active":"2023-10-06T05:15:23.832390+0000","last_peered":"2023-10-06T05:15:23.832390+0000","last_clean":"2023-10-06T05:15:23.832390+0000","last_became_active":"2023-10-06T05:15:23.827803+0000","last_became_peered":"2023-10-06T05:15:23.827803+0000","last_unstale":"2023-10-06T05:15:23.832390+0000","last_undegraded":"2023-10-06T05:15:23.832390+0000","last_fullsized":"2023-10-06T05:15:23.832390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:31.565511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843650+0000","last_change":"2023-10-06T05:15:24.841850+0000","last_active":"2023-10-06T05:15:24.843650+0000","last_peered":"2023-10-06T05:15:24.843650+0000","last_clean":"2023-10-06T05:15:24.843650+0000","last_became_active":"2023-10-06T05:15:24.841600+0000","last_became_peered":"2023-10-06T05:15:24.841600+0000","last_unstale":"2023-10-06T05:15:24.843650+0000","last_undegraded":"2023-10-06T05:15:24.843650+0000","last_fullsized":"2023-10-06T05:15:24.843650+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:40.358638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.821925+0000","last_change":"2023-10-06T05:15:23.824145+0000","last_active":"2023-10-06T05:15:24.821925+0000","last_peered":"2023-10-06T05:15:24.821925+0000","last_clean":"2023-10-06T05:15:24.821925+0000","last_became_active":"2023-10-06T05:15:23.823966+0000","last_became_peered":"2023-10-06T05:15:23.823966+0000","last_unstale":"2023-10-06T05:15:24.821925+0000","last_undegraded":"2023-10-06T05:15:24.821925+0000","last_fullsized":"2023-10-06T05:15:24.821925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:07.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844397+0000","last_change":"2023-10-06T05:15:24.844397+0000","last_active":"2023-10-06T05:15:24.844397+0000","last_peered":"2023-10-06T05:15:24.844397+0000","last_clean":"2023-10-06T05:15:24.844397+0000","last_became_active":"2023-10-06T05:15:24.844088+0000","last_became_peered":"2023-10-06T05:15:24.844088+0000","last_unstale":"2023-10-06T05:15:24.844397+0000","last_undegraded":"2023-10-06T05:15:24.844397+0000","last_fullsized":"2023-10-06T05:15:24.844397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:32.781863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.827625+0000","last_change":"2023-10-06T05:15:23.824400+0000","last_active":"2023-10-06T05:15:24.827625+0000","last_peered":"2023-10-06T05:15:24.827625+0000","last_clean":"2023-10-06T05:15:24.827625+0000","last_became_active":"2023-10-06T05:15:23.824106+0000","last_became_peered":"2023-10-06T05:15:23.824106+0000","last_unstale":"2023-10-06T05:15:24.827625+0000","last_undegraded":"2023-10-06T05:15:24.827625+0000","last_fullsized":"2023-10-06T05:15:24.827625+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:56.099714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.836366+0000","last_change":"2023-10-06T05:15:24.836366+0000","last_active":"2023-10-06T05:15:24.836366+0000","last_peered":"2023-10-06T05:15:24.836366+0000","last_clean":"2023-10-06T05:15:24.836366+0000","last_became_active":"2023-10-06T05:15:24.836031+0000","last_became_peered":"2023-10-06T05:15:24.836031+0000","last_unstale":"2023-10-06T05:15:24.836366+0000","last_undegraded":"2023-10-06T05:15:24.836366+0000","last_fullsized":"2023-10-06T05:15:24.836366+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:32.781863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.865243+0000","last_change":"2023-10-06T05:15:23.825407+0000","last_active":"2023-10-06T05:15:24.865243+0000","last_peered":"2023-10-06T05:15:24.865243+0000","last_clean":"2023-10-06T05:15:24.865243+0000","last_became_active":"2023-10-06T05:15:23.825059+0000","last_became_peered":"2023-10-06T05:15:23.825059+0000","last_unstale":"2023-10-06T05:15:24.865243+0000","last_undegraded":"2023-10-06T05:15:24.865243+0000","last_fullsized":"2023-10-06T05:15:24.865243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846155+0000","last_change":"2023-10-06T05:15:24.846155+0000","last_active":"2023-10-06T05:15:24.846155+0000","last_peered":"2023-10-06T05:15:24.846155+0000","last_clean":"2023-10-06T05:15:24.846155+0000","last_became_active":"2023-10-06T05:15:24.845894+0000","last_became_peered":"2023-10-06T05:15:24.845894+0000","last_unstale":"2023-10-06T05:15:24.846155+0000","last_undegraded":"2023-10-06T05:15:24.846155+0000","last_fullsized":"2023-10-06T05:15:24.846155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.865115+0000","last_change":"2023-10-06T05:15:23.824972+0000","last_active":"2023-10-06T05:15:24.865115+0000","last_peered":"2023-10-06T05:15:24.865115+0000","last_clean":"2023-10-06T05:15:24.865115+0000","last_became_active":"2023-10-06T05:15:23.824690+0000","last_became_peered":"2023-10-06T05:15:23.824690+0000","last_unstale":"2023-10-06T05:15:24.865115+0000","last_undegraded":"2023-10-06T05:15:24.865115+0000","last_fullsized":"2023-10-06T05:15:24.865115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:16.309623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.842326+0000","last_change":"2023-10-06T05:15:24.842326+0000","last_active":"2023-10-06T05:15:24.842326+0000","last_peered":"2023-10-06T05:15:24.842326+0000","last_clean":"2023-10-06T05:15:24.842326+0000","last_became_active":"2023-10-06T05:15:24.841947+0000","last_became_peered":"2023-10-06T05:15:24.841947+0000","last_unstale":"2023-10-06T05:15:24.842326+0000","last_undegraded":"2023-10-06T05:15:24.842326+0000","last_fullsized":"2023-10-06T05:15:24.842326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.855637+0000","last_change":"2023-10-06T05:15:23.810649+0000","last_active":"2023-10-06T05:15:24.855637+0000","last_peered":"2023-10-06T05:15:24.855637+0000","last_clean":"2023-10-06T05:15:24.855637+0000","last_became_active":"2023-10-06T05:15:23.810340+0000","last_became_peered":"2023-10-06T05:15:23.810340+0000","last_unstale":"2023-10-06T05:15:24.855637+0000","last_undegraded":"2023-10-06T05:15:24.855637+0000","last_fullsized":"2023-10-06T05:15:24.855637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:33.300490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.831057+0000","last_change":"2023-10-06T05:15:24.826890+0000","last_active":"2023-10-06T05:15:24.831057+0000","last_peered":"2023-10-06T05:15:24.831057+0000","last_clean":"2023-10-06T05:15:24.831057+0000","last_became_active":"2023-10-06T05:15:24.826618+0000","last_became_peered":"2023-10-06T05:15:24.826618+0000","last_unstale":"2023-10-06T05:15:24.831057+0000","last_undegraded":"2023-10-06T05:15:24.831057+0000","last_fullsized":"2023-10-06T05:15:24.831057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:38.333907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832899+0000","last_change":"2023-10-06T05:15:23.829383+0000","last_active":"2023-10-06T05:15:23.832899+0000","last_peered":"2023-10-06T05:15:23.832899+0000","last_clean":"2023-10-06T05:15:23.832899+0000","last_became_active":"2023-10-06T05:15:23.828934+0000","last_became_peered":"2023-10-06T05:15:23.828934+0000","last_unstale":"2023-10-06T05:15:23.832899+0000","last_undegraded":"2023-10-06T05:15:23.832899+0000","last_fullsized":"2023-10-06T05:15:23.832899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.842018+0000","last_change":"2023-10-06T05:15:24.842018+0000","last_active":"2023-10-06T05:15:24.842018+0000","last_peered":"2023-10-06T05:15:24.842018+0000","last_clean":"2023-10-06T05:15:24.842018+0000","last_became_active":"2023-10-06T05:15:24.841744+0000","last_became_peered":"2023-10-06T05:15:24.841744+0000","last_unstale":"2023-10-06T05:15:24.842018+0000","last_undegraded":"2023-10-06T05:15:24.842018+0000","last_fullsized":"2023-10-06T05:15:24.842018+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.822818+0000","last_change":"2023-10-06T05:15:23.816494+0000","last_active":"2023-10-06T05:15:24.822818+0000","last_peered":"2023-10-06T05:15:24.822818+0000","last_clean":"2023-10-06T05:15:24.822818+0000","last_became_active":"2023-10-06T05:15:23.816245+0000","last_became_peered":"2023-10-06T05:15:23.816245+0000","last_unstale":"2023-10-06T05:15:24.822818+0000","last_undegraded":"2023-10-06T05:15:24.822818+0000","last_fullsized":"2023-10-06T05:15:24.822818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:09.136217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839251+0000","last_change":"2023-10-06T05:15:24.836863+0000","last_active":"2023-10-06T05:15:24.839251+0000","last_peered":"2023-10-06T05:15:24.839251+0000","last_clean":"2023-10-06T05:15:24.839251+0000","last_became_active":"2023-10-06T05:15:24.836605+0000","last_became_peered":"2023-10-06T05:15:24.836605+0000","last_unstale":"2023-10-06T05:15:24.839251+0000","last_undegraded":"2023-10-06T05:15:24.839251+0000","last_fullsized":"2023-10-06T05:15:24.839251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.860479+0000","last_change":"2023-10-06T05:15:23.816184+0000","last_active":"2023-10-06T05:15:24.860479+0000","last_peered":"2023-10-06T05:15:24.860479+0000","last_clean":"2023-10-06T05:15:24.860479+0000","last_became_active":"2023-10-06T05:15:23.815897+0000","last_became_peered":"2023-10-06T05:15:23.815897+0000","last_unstale":"2023-10-06T05:15:24.860479+0000","last_undegraded":"2023-10-06T05:15:24.860479+0000","last_fullsized":"2023-10-06T05:15:24.860479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839228+0000","last_change":"2023-10-06T05:15:24.837832+0000","last_active":"2023-10-06T05:15:24.839228+0000","last_peered":"2023-10-06T05:15:24.839228+0000","last_clean":"2023-10-06T05:15:24.839228+0000","last_became_active":"2023-10-06T05:15:24.837542+0000","last_became_peered":"2023-10-06T05:15:24.837542+0000","last_unstale":"2023-10-06T05:15:24.839228+0000","last_undegraded":"2023-10-06T05:15:24.839228+0000","last_fullsized":"2023-10-06T05:15:24.839228+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863612+0000","last_change":"2023-10-06T05:15:23.821032+0000","last_active":"2023-10-06T05:15:24.863612+0000","last_peered":"2023-10-06T05:15:24.863612+0000","last_clean":"2023-10-06T05:15:24.863612+0000","last_became_active":"2023-10-06T05:15:23.820789+0000","last_became_peered":"2023-10-06T05:15:23.820789+0000","last_unstale":"2023-10-06T05:15:24.863612+0000","last_undegraded":"2023-10-06T05:15:24.863612+0000","last_fullsized":"2023-10-06T05:15:24.863612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:59.538238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.855413+0000","last_change":"2023-10-06T05:15:24.845844+0000","last_active":"2023-10-06T05:15:24.855413+0000","last_peered":"2023-10-06T05:15:24.855413+0000","last_clean":"2023-10-06T05:15:24.855413+0000","last_became_active":"2023-10-06T05:15:24.845538+0000","last_became_peered":"2023-10-06T05:15:24.845538+0000","last_unstale":"2023-10-06T05:15:24.855413+0000","last_undegraded":"2023-10-06T05:15:24.855413+0000","last_fullsized":"2023-10-06T05:15:24.855413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.862733+0000","last_change":"2023-10-06T05:15:23.822015+0000","last_active":"2023-10-06T05:15:24.862733+0000","last_peered":"2023-10-06T05:15:24.862733+0000","last_clean":"2023-10-06T05:15:24.862733+0000","last_became_active":"2023-10-06T05:15:23.821690+0000","last_became_peered":"2023-10-06T05:15:23.821690+0000","last_unstale":"2023-10-06T05:15:24.862733+0000","last_undegraded":"2023-10-06T05:15:24.862733+0000","last_fullsized":"2023-10-06T05:15:24.862733+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:13.752120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.840551+0000","last_change":"2023-10-06T05:15:24.840551+0000","last_active":"2023-10-06T05:15:24.840551+0000","last_peered":"2023-10-06T05:15:24.840551+0000","last_clean":"2023-10-06T05:15:24.840551+0000","last_became_active":"2023-10-06T05:15:24.840254+0000","last_became_peered":"2023-10-06T05:15:24.840254+0000","last_unstale":"2023-10-06T05:15:24.840551+0000","last_undegraded":"2023-10-06T05:15:24.840551+0000","last_fullsized":"2023-10-06T05:15:24.840551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.862784+0000","last_change":"2023-10-06T05:15:23.827704+0000","last_active":"2023-10-06T05:15:24.862784+0000","last_peered":"2023-10-06T05:15:24.862784+0000","last_clean":"2023-10-06T05:15:24.862784+0000","last_became_active":"2023-10-06T05:15:23.827417+0000","last_became_peered":"2023-10-06T05:15:23.827417+0000","last_unstale":"2023-10-06T05:15:24.862784+0000","last_undegraded":"2023-10-06T05:15:24.862784+0000","last_fullsized":"2023-10-06T05:15:24.862784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.835636+0000","last_change":"2023-10-06T05:15:24.829282+0000","last_active":"2023-10-06T05:15:24.835636+0000","last_peered":"2023-10-06T05:15:24.835636+0000","last_clean":"2023-10-06T05:15:24.835636+0000","last_became_active":"2023-10-06T05:15:24.828978+0000","last_became_peered":"2023-10-06T05:15:24.828978+0000","last_unstale":"2023-10-06T05:15:24.835636+0000","last_undegraded":"2023-10-06T05:15:24.835636+0000","last_fullsized":"2023-10-06T05:15:24.835636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:40.366886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.830792+0000","last_change":"2023-10-06T05:15:23.828705+0000","last_active":"2023-10-06T05:15:23.830792+0000","last_peered":"2023-10-06T05:15:23.830792+0000","last_clean":"2023-10-06T05:15:23.830792+0000","last_became_active":"2023-10-06T05:15:23.828442+0000","last_became_peered":"2023-10-06T05:15:23.828442+0000","last_unstale":"2023-10-06T05:15:23.830792+0000","last_undegraded":"2023-10-06T05:15:23.830792+0000","last_fullsized":"2023-10-06T05:15:23.830792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:56.099714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.840261+0000","last_change":"2023-10-06T05:15:24.840261+0000","last_active":"2023-10-06T05:15:24.840261+0000","last_peered":"2023-10-06T05:15:24.840261+0000","last_clean":"2023-10-06T05:15:24.840261+0000","last_became_active":"2023-10-06T05:15:24.838724+0000","last_became_peered":"2023-10-06T05:15:24.838724+0000","last_unstale":"2023-10-06T05:15:24.840261+0000","last_undegraded":"2023-10-06T05:15:24.840261+0000","last_fullsized":"2023-10-06T05:15:24.840261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.857674+0000","last_change":"2023-10-06T05:15:23.820198+0000","last_active":"2023-10-06T05:15:24.857674+0000","last_peered":"2023-10-06T05:15:24.857674+0000","last_clean":"2023-10-06T05:15:24.857674+0000","last_became_active":"2023-10-06T05:15:23.819831+0000","last_became_peered":"2023-10-06T05:15:23.819831+0000","last_unstale":"2023-10-06T05:15:24.857674+0000","last_undegraded":"2023-10-06T05:15:24.857674+0000","last_fullsized":"2023-10-06T05:15:24.857674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:33.001187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.836805+0000","last_change":"2023-10-06T05:15:24.836805+0000","last_active":"2023-10-06T05:15:24.836805+0000","last_peered":"2023-10-06T05:15:24.836805+0000","last_clean":"2023-10-06T05:15:24.836805+0000","last_became_active":"2023-10-06T05:15:24.836512+0000","last_became_peered":"2023-10-06T05:15:24.836512+0000","last_unstale":"2023-10-06T05:15:24.836805+0000","last_undegraded":"2023-10-06T05:15:24.836805+0000","last_fullsized":"2023-10-06T05:15:24.836805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:15.984185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820229+0000","last_change":"2023-10-06T05:15:23.828529+0000","last_active":"2023-10-06T05:15:24.820229+0000","last_peered":"2023-10-06T05:15:24.820229+0000","last_clean":"2023-10-06T05:15:24.820229+0000","last_became_active":"2023-10-06T05:15:23.828273+0000","last_became_peered":"2023-10-06T05:15:23.828273+0000","last_unstale":"2023-10-06T05:15:24.820229+0000","last_undegraded":"2023-10-06T05:15:24.820229+0000","last_fullsized":"2023-10-06T05:15:24.820229+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:25.833446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845485+0000","last_change":"2023-10-06T05:15:24.842460+0000","last_active":"2023-10-06T05:15:24.845485+0000","last_peered":"2023-10-06T05:15:24.845485+0000","last_clean":"2023-10-06T05:15:24.845485+0000","last_became_active":"2023-10-06T05:15:24.842214+0000","last_became_peered":"2023-10-06T05:15:24.842214+0000","last_unstale":"2023-10-06T05:15:24.845485+0000","last_undegraded":"2023-10-06T05:15:24.845485+0000","last_fullsized":"2023-10-06T05:15:24.845485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:26.192843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.831017+0000","last_change":"2023-10-06T05:15:23.828196+0000","last_active":"2023-10-06T05:15:23.831017+0000","last_peered":"2023-10-06T05:15:23.831017+0000","last_clean":"2023-10-06T05:15:23.831017+0000","last_became_active":"2023-10-06T05:15:23.827716+0000","last_became_peered":"2023-10-06T05:15:23.827716+0000","last_unstale":"2023-10-06T05:15:23.831017+0000","last_undegraded":"2023-10-06T05:15:23.831017+0000","last_fullsized":"2023-10-06T05:15:23.831017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:56.099714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839213+0000","last_change":"2023-10-06T05:15:24.835538+0000","last_active":"2023-10-06T05:15:24.839213+0000","last_peered":"2023-10-06T05:15:24.839213+0000","last_clean":"2023-10-06T05:15:24.839213+0000","last_became_active":"2023-10-06T05:15:24.835234+0000","last_became_peered":"2023-10-06T05:15:24.835234+0000","last_unstale":"2023-10-06T05:15:24.839213+0000","last_undegraded":"2023-10-06T05:15:24.839213+0000","last_fullsized":"2023-10-06T05:15:24.839213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820016+0000","last_change":"2023-10-06T05:15:23.819215+0000","last_active":"2023-10-06T05:15:24.820016+0000","last_peered":"2023-10-06T05:15:24.820016+0000","last_clean":"2023-10-06T05:15:24.820016+0000","last_became_active":"2023-10-06T05:15:23.818917+0000","last_became_peered":"2023-10-06T05:15:23.818917+0000","last_unstale":"2023-10-06T05:15:24.820016+0000","last_undegraded":"2023-10-06T05:15:24.820016+0000","last_fullsized":"2023-10-06T05:15:24.820016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:37.775731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.827336+0000","last_change":"2023-10-06T05:15:24.827336+0000","last_active":"2023-10-06T05:15:24.827336+0000","last_peered":"2023-10-06T05:15:24.827336+0000","last_clean":"2023-10-06T05:15:24.827336+0000","last_became_active":"2023-10-06T05:15:24.827085+0000","last_became_peered":"2023-10-06T05:15:24.827085+0000","last_unstale":"2023-10-06T05:15:24.827336+0000","last_undegraded":"2023-10-06T05:15:24.827336+0000","last_fullsized":"2023-10-06T05:15:24.827336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:59.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832545+0000","last_change":"2023-10-06T05:15:23.828400+0000","last_active":"2023-10-06T05:15:23.832545+0000","last_peered":"2023-10-06T05:15:23.832545+0000","last_clean":"2023-10-06T05:15:23.832545+0000","last_became_active":"2023-10-06T05:15:23.828099+0000","last_became_peered":"2023-10-06T05:15:23.828099+0000","last_unstale":"2023-10-06T05:15:23.832545+0000","last_undegraded":"2023-10-06T05:15:23.832545+0000","last_fullsized":"2023-10-06T05:15:23.832545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:07.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.817672+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:15:23.817672+0000","last_peered":"2023-10-06T05:15:23.817672+0000","last_clean":"2023-10-06T05:15:23.817672+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:15:23.817672+0000","last_undegraded":"2023-10-06T05:15:23.817672+0000","last_fullsized":"2023-10-06T05:15:23.817672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845806+0000","last_change":"2023-10-06T05:15:24.841042+0000","last_active":"2023-10-06T05:15:24.845806+0000","last_peered":"2023-10-06T05:15:24.845806+0000","last_clean":"2023-10-06T05:15:24.845806+0000","last_became_active":"2023-10-06T05:15:24.840750+0000","last_became_peered":"2023-10-06T05:15:24.840750+0000","last_unstale":"2023-10-06T05:15:24.845806+0000","last_undegraded":"2023-10-06T05:15:24.845806+0000","last_fullsized":"2023-10-06T05:15:24.845806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:04.542305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.830219+0000","last_change":"2023-10-06T05:15:23.825201+0000","last_active":"2023-10-06T05:15:23.830219+0000","last_peered":"2023-10-06T05:15:23.830219+0000","last_clean":"2023-10-06T05:15:23.830219+0000","last_became_active":"2023-10-06T05:15:23.824900+0000","last_became_peered":"2023-10-06T05:15:23.824900+0000","last_unstale":"2023-10-06T05:15:23.830219+0000","last_undegraded":"2023-10-06T05:15:23.830219+0000","last_fullsized":"2023-10-06T05:15:23.830219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:33.300490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.825628+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:15:24.825628+0000","last_peered":"2023-10-06T05:15:24.825628+0000","last_clean":"2023-10-06T05:15:24.825628+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:15:24.825628+0000","last_undegraded":"2023-10-06T05:15:24.825628+0000","last_fullsized":"2023-10-06T05:15:24.825628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846155+0000","last_change":"2023-10-06T05:15:24.846155+0000","last_active":"2023-10-06T05:15:24.846155+0000","last_peered":"2023-10-06T05:15:24.846155+0000","last_clean":"2023-10-06T05:15:24.846155+0000","last_became_active":"2023-10-06T05:15:24.845842+0000","last_became_peered":"2023-10-06T05:15:24.845842+0000","last_unstale":"2023-10-06T05:15:24.846155+0000","last_undegraded":"2023-10-06T05:15:24.846155+0000","last_fullsized":"2023-10-06T05:15:24.846155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:26.192843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-06T05:15:25.836958+0000","last_change":"2023-10-06T05:15:23.829648+0000","last_active":"2023-10-06T05:15:25.836958+0000","last_peered":"2023-10-06T05:15:25.836958+0000","last_clean":"2023-10-06T05:15:25.836958+0000","last_became_active":"2023-10-06T05:15:23.829378+0000","last_became_peered":"2023-10-06T05:15:23.829378+0000","last_unstale":"2023-10-06T05:15:25.836958+0000","last_undegraded":"2023-10-06T05:15:25.836958+0000","last_fullsized":"2023-10-06T05:15:25.836958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:15.439776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820110+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:15:24.820110+0000","last_peered":"2023-10-06T05:15:24.820110+0000","last_clean":"2023-10-06T05:15:24.820110+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:15:24.820110+0000","last_undegraded":"2023-10-06T05:15:24.820110+0000","last_fullsized":"2023-10-06T05:15:24.820110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.851763+0000","last_change":"2023-10-06T05:15:24.842849+0000","last_active":"2023-10-06T05:15:24.851763+0000","last_peered":"2023-10-06T05:15:24.851763+0000","last_clean":"2023-10-06T05:15:24.851763+0000","last_became_active":"2023-10-06T05:15:24.841045+0000","last_became_peered":"2023-10-06T05:15:24.841045+0000","last_unstale":"2023-10-06T05:15:24.851763+0000","last_undegraded":"2023-10-06T05:15:24.851763+0000","last_fullsized":"2023-10-06T05:15:24.851763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.826811+0000","last_change":"2023-10-06T05:15:23.823464+0000","last_active":"2023-10-06T05:15:23.826811+0000","last_peered":"2023-10-06T05:15:23.826811+0000","last_clean":"2023-10-06T05:15:23.826811+0000","last_became_active":"2023-10-06T05:15:23.823272+0000","last_became_peered":"2023-10-06T05:15:23.823272+0000","last_unstale":"2023-10-06T05:15:23.826811+0000","last_undegraded":"2023-10-06T05:15:23.826811+0000","last_fullsized":"2023-10-06T05:15:23.826811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:35.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.829482+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:15:24.829482+0000","last_peered":"2023-10-06T05:15:24.829482+0000","last_clean":"2023-10-06T05:15:24.829482+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:15:24.829482+0000","last_undegraded":"2023-10-06T05:15:24.829482+0000","last_fullsized":"2023-10-06T05:15:24.829482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.807351+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:15:23.807351+0000","last_peered":"2023-10-06T05:15:23.807351+0000","last_clean":"2023-10-06T05:15:23.807351+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:15:23.807351+0000","last_undegraded":"2023-10-06T05:15:23.807351+0000","last_fullsized":"2023-10-06T05:15:23.807351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844649+0000","last_change":"2023-10-06T05:15:24.844649+0000","last_active":"2023-10-06T05:15:24.844649+0000","last_peered":"2023-10-06T05:15:24.844649+0000","last_clean":"2023-10-06T05:15:24.844649+0000","last_became_active":"2023-10-06T05:15:24.844385+0000","last_became_peered":"2023-10-06T05:15:24.844385+0000","last_unstale":"2023-10-06T05:15:24.844649+0000","last_undegraded":"2023-10-06T05:15:24.844649+0000","last_fullsized":"2023-10-06T05:15:24.844649+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:15.984185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863532+0000","last_change":"2023-10-06T05:15:23.825367+0000","last_active":"2023-10-06T05:15:24.863532+0000","last_peered":"2023-10-06T05:15:24.863532+0000","last_clean":"2023-10-06T05:15:24.863532+0000","last_became_active":"2023-10-06T05:15:23.825023+0000","last_became_peered":"2023-10-06T05:15:23.825023+0000","last_unstale":"2023-10-06T05:15:24.863532+0000","last_undegraded":"2023-10-06T05:15:24.863532+0000","last_fullsized":"2023-10-06T05:15:24.863532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.807437+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:15:23.807437+0000","last_peered":"2023-10-06T05:15:23.807437+0000","last_clean":"2023-10-06T05:15:23.807437+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:15:23.807437+0000","last_undegraded":"2023-10-06T05:15:23.807437+0000","last_fullsized":"2023-10-06T05:15:23.807437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.827359+0000","last_change":"2023-10-06T05:15:24.827359+0000","last_active":"2023-10-06T05:15:24.827359+0000","last_peered":"2023-10-06T05:15:24.827359+0000","last_clean":"2023-10-06T05:15:24.827359+0000","last_became_active":"2023-10-06T05:15:24.827081+0000","last_became_peered":"2023-10-06T05:15:24.827081+0000","last_unstale":"2023-10-06T05:15:24.827359+0000","last_undegraded":"2023-10-06T05:15:24.827359+0000","last_fullsized":"2023-10-06T05:15:24.827359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:32.571095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.823911+0000","last_change":"2023-10-06T05:15:23.819164+0000","last_active":"2023-10-06T05:15:24.823911+0000","last_peered":"2023-10-06T05:15:24.823911+0000","last_clean":"2023-10-06T05:15:24.823911+0000","last_became_active":"2023-10-06T05:15:23.818903+0000","last_became_peered":"2023-10-06T05:15:23.818903+0000","last_unstale":"2023-10-06T05:15:24.823911+0000","last_undegraded":"2023-10-06T05:15:24.823911+0000","last_fullsized":"2023-10-06T05:15:24.823911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:37.775731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.809579+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:15:23.809579+0000","last_peered":"2023-10-06T05:15:23.809579+0000","last_clean":"2023-10-06T05:15:23.809579+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:15:23.809579+0000","last_undegraded":"2023-10-06T05:15:23.809579+0000","last_fullsized":"2023-10-06T05:15:23.809579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.847750+0000","last_change":"2023-10-06T05:15:24.847750+0000","last_active":"2023-10-06T05:15:24.847750+0000","last_peered":"2023-10-06T05:15:24.847750+0000","last_clean":"2023-10-06T05:15:24.847750+0000","last_became_active":"2023-10-06T05:15:24.847118+0000","last_became_peered":"2023-10-06T05:15:24.847118+0000","last_unstale":"2023-10-06T05:15:24.847750+0000","last_undegraded":"2023-10-06T05:15:24.847750+0000","last_fullsized":"2023-10-06T05:15:24.847750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.826577+0000","last_change":"2023-10-06T05:15:23.819539+0000","last_active":"2023-10-06T05:15:23.826577+0000","last_peered":"2023-10-06T05:15:23.826577+0000","last_clean":"2023-10-06T05:15:23.826577+0000","last_became_active":"2023-10-06T05:15:23.819114+0000","last_became_peered":"2023-10-06T05:15:23.819114+0000","last_unstale":"2023-10-06T05:15:23.826577+0000","last_undegraded":"2023-10-06T05:15:23.826577+0000","last_fullsized":"2023-10-06T05:15:23.826577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:58.337389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.815010+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:15:23.815010+0000","last_peered":"2023-10-06T05:15:23.815010+0000","last_clean":"2023-10-06T05:15:23.815010+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:15:23.815010+0000","last_undegraded":"2023-10-06T05:15:23.815010+0000","last_fullsized":"2023-10-06T05:15:23.815010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846211+0000","last_change":"2023-10-06T05:15:24.846211+0000","last_active":"2023-10-06T05:15:24.846211+0000","last_peered":"2023-10-06T05:15:24.846211+0000","last_clean":"2023-10-06T05:15:24.846211+0000","last_became_active":"2023-10-06T05:15:24.845881+0000","last_became_peered":"2023-10-06T05:15:24.845881+0000","last_unstale":"2023-10-06T05:15:24.846211+0000","last_undegraded":"2023-10-06T05:15:24.846211+0000","last_fullsized":"2023-10-06T05:15:24.846211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820933+0000","last_change":"2023-10-06T05:15:23.823659+0000","last_active":"2023-10-06T05:15:24.820933+0000","last_peered":"2023-10-06T05:15:24.820933+0000","last_clean":"2023-10-06T05:15:24.820933+0000","last_became_active":"2023-10-06T05:15:23.823474+0000","last_became_peered":"2023-10-06T05:15:23.823474+0000","last_unstale":"2023-10-06T05:15:24.820933+0000","last_undegraded":"2023-10-06T05:15:24.820933+0000","last_fullsized":"2023-10-06T05:15:24.820933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:09.050751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.823265+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:15:24.823265+0000","last_peered":"2023-10-06T05:15:24.823265+0000","last_clean":"2023-10-06T05:15:24.823265+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:15:24.823265+0000","last_undegraded":"2023-10-06T05:15:24.823265+0000","last_fullsized":"2023-10-06T05:15:24.823265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":754443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.499},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.66000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297692,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074148,"statfs":{"total":96636764160,"available":96331927552,"internally_reserved":0,"allocated":413696,"data_stored":169664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":162559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.311},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.36599999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.624}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298260,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073580,"statfs":{"total":96636764160,"available":96331345920,"internally_reserved":0,"allocated":995328,"data_stored":752871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.314},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.749},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.878}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297692,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074148,"statfs":{"total":96636764160,"available":96331927552,"internally_reserved":0,"allocated":413696,"data_stored":169180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.37}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.80700000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.78000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":170680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.38200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.98399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.82499999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.91300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":163045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.625},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.81599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":417792,"data_stored":169222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.62},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.501},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:29.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.428+0000 7fd41ffff700 1 Processor -- start 2023-10-06T05:15:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.429+0000 7fd41ffff700 1 -- start start 2023-10-06T05:15:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4200cb710 0x7fd42006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200c9cb0 con 0x7fd42013faa0 2023-10-06T05:15:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200ca270 con 0x7fd4200c8050 2023-10-06T05:15:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42006b840 con 0x7fd4200cb710 2023-10-06T05:15:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd42013faa0 0x7fd4200c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60032/0 (socket says 172.21.15.33:60032) 2023-10-06T05:15:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41effd700 1 -- 172.21.15.33:0/451118077 learned_addr learned my addr 172.21.15.33:0/451118077 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.430+0000 7fd41f7fe700 1 --2- 172.21.15.33:0/451118077 >> v2:172.21.15.33:3300/0 conn(0x7fd4200cb710 0x7fd42006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41dffb700 1 -- 172.21.15.33:0/451118077 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730476164 0 0) 0x7fd4200c9cb0 con 0x7fd42013faa0 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41f7fe700 1 -- 172.21.15.33:0/451118077 >> v1:172.21.15.167:6789/0 conn(0x7fd4200c8050 legacy=0x7fd42006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41f7fe700 1 -- 172.21.15.33:0/451118077 >> v1:172.21.15.33:6789/0 conn(0x7fd42013faa0 legacy=0x7fd4200c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41f7fe700 1 -- 172.21.15.33:0/451118077 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42006bb20 con 0x7fd4200cb710 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41f7fe700 1 --2- 172.21.15.33:0/451118077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4200cb710 0x7fd42006b1e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd418009e60 tx=0x7fd418009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a728a9cc232431 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41dffb700 1 -- 172.21.15.33:0/451118077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800e070 con 0x7fd4200cb710 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41dffb700 1 -- 172.21.15.33:0/451118077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41800bc30 con 0x7fd4200cb710 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.431+0000 7fd41dffb700 1 -- 172.21.15.33:0/451118077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418012430 con 0x7fd4200cb710 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 -- 172.21.15.33:0/451118077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4200cb710 msgr2=0x7fd42006b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 --2- 172.21.15.33:0/451118077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4200cb710 0x7fd42006b1e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd418009e60 tx=0x7fd418009780 comp rx=0 tx=0).stop 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 -- 172.21.15.33:0/451118077 shutdown_connections 2023-10-06T05:15:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 --2- 172.21.15.33:0/451118077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4200cb710 0x7fd42006b1e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 -- 172.21.15.33:0/451118077 >> 172.21.15.33:0/451118077 conn(0x7fd42006d030 msgr2=0x7fd42006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 -- 172.21.15.33:0/451118077 shutdown_connections 2023-10-06T05:15:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.432+0000 7fd41ffff700 1 -- 172.21.15.33:0/451118077 wait complete. 2023-10-06T05:15:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.433+0000 7fd41ffff700 1 Processor -- start 2023-10-06T05:15:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.433+0000 7fd41ffff700 1 -- start start 2023-10-06T05:15:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.434+0000 7fd41ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4200c8050 0x7fd4201558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.434+0000 7fd41ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4200cb710 0x7fd420136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.434+0000 7fd41ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 0x7fd420137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.434+0000 7fd41ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200b0540 con 0x7fd42013faa0 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.434+0000 7fd41ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420137b40 con 0x7fd4200c8050 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.434+0000 7fd41ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd420137e20 con 0x7fd4200cb710 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 0x7fd420137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 0x7fd420137500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52142/0 (socket says 172.21.15.33:52142) 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 -- 172.21.15.33:0/2403336811 learned_addr learned my addr 172.21.15.33:0/2403336811 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41effd700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4200c8050 0x7fd4201558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41e7fc700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4200cb710 0x7fd420136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 -- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4200cb710 msgr2=0x7fd420136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4200cb710 0x7fd420136fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 -- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4200c8050 msgr2=0x7fd4201558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4200c8050 0x7fd4201558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 -- 172.21.15.33:0/2403336811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418009a20 con 0x7fd42013faa0 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41effd700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4200c8050 0x7fd4201558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.435+0000 7fd41f7fe700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 0x7fd420137500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd418013000 tx=0x7fd418019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.436+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41801b630 con 0x7fd42013faa0 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.436+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41801b7b0 con 0x7fd42013faa0 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.436+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41801bb70 con 0x7fd42013faa0 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.436+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd420138100 con 0x7fd42013faa0 2023-10-06T05:15:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.436+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd420155de0 con 0x7fd42013faa0 2023-10-06T05:15:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.437+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4200b06f0 con 0x7fd42013faa0 2023-10-06T05:15:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.438+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd41801c400 con 0x7fd42013faa0 2023-10-06T05:15:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.439+0000 7fd406ffd700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd40c066f30 0x7fd40c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.439+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4180869c0 con 0x7fd42013faa0 2023-10-06T05:15:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.440+0000 7fd41effd700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd40c066f30 0x7fd40c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.440+0000 7fd41effd700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd40c066f30 0x7fd40c0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd410001890 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.443+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd418053af0 con 0x7fd42013faa0 2023-10-06T05:15:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.641+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd42004c380 con 0x7fd40c066f30 2023-10-06T05:15:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.651+0000 7fd406ffd700 1 -- 172.21.15.33:0/2403336811 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310296 (secure 0 0 0) 0x7fd42004c380 con 0x7fd40c066f30 2023-10-06T05:15:29.658 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd40c066f30 msgr2=0x7fd40c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd40c066f30 0x7fd40c0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd410001890 tx=0x7fd410009040 comp rx=0 tx=0).stop 2023-10-06T05:15:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 msgr2=0x7fd420137500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 0x7fd420137500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd418013000 tx=0x7fd418019000 comp rx=0 tx=0).stop 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 shutdown_connections 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd40c066f30 0x7fd40c0693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd42013faa0 0x7fd420137500 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4200c8050 0x7fd4201558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 --2- 172.21.15.33:0/2403336811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4200cb710 0x7fd420136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.655+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 >> 172.21.15.33:0/2403336811 conn(0x7fd42006d030 msgr2=0x7fd4200c2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.656+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 shutdown_connections 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.656+0000 7fd41ffff700 1 -- 172.21.15.33:0/2403336811 wait complete. 2023-10-06T05:15:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:15:29.675 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-10-06T05:15:27.831400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382680,"kb_used_data":4376,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592040,"statfs":{"total":773094113280,"available":770654248960,"internally_reserved":0,"allocated":4481024,"data_stored":2511664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.031934"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839597+0000","last_change":"2023-10-06T05:15:24.834229+0000","last_active":"2023-10-06T05:15:24.839597+0000","last_peered":"2023-10-06T05:15:24.839597+0000","last_clean":"2023-10-06T05:15:24.839597+0000","last_became_active":"2023-10-06T05:15:24.833960+0000","last_became_peered":"2023-10-06T05:15:24.833960+0000","last_unstale":"2023-10-06T05:15:24.839597+0000","last_undegraded":"2023-10-06T05:15:24.839597+0000","last_fullsized":"2023-10-06T05:15:24.839597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:47.981780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.835502+0000","last_change":"2023-10-06T05:15:24.829090+0000","last_active":"2023-10-06T05:15:24.835502+0000","last_peered":"2023-10-06T05:15:24.835502+0000","last_clean":"2023-10-06T05:15:24.835502+0000","last_became_active":"2023-10-06T05:15:24.828802+0000","last_became_peered":"2023-10-06T05:15:24.828802+0000","last_unstale":"2023-10-06T05:15:24.835502+0000","last_undegraded":"2023-10-06T05:15:24.835502+0000","last_fullsized":"2023-10-06T05:15:24.835502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:29.175187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.849845+0000","last_change":"2023-10-06T05:15:24.849845+0000","last_active":"2023-10-06T05:15:24.849845+0000","last_peered":"2023-10-06T05:15:24.849845+0000","last_clean":"2023-10-06T05:15:24.849845+0000","last_became_active":"2023-10-06T05:15:24.849568+0000","last_became_peered":"2023-10-06T05:15:24.849568+0000","last_unstale":"2023-10-06T05:15:24.849845+0000","last_undegraded":"2023-10-06T05:15:24.849845+0000","last_fullsized":"2023-10-06T05:15:24.849845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.851692+0000","last_change":"2023-10-06T05:15:24.849012+0000","last_active":"2023-10-06T05:15:24.851692+0000","last_peered":"2023-10-06T05:15:24.851692+0000","last_clean":"2023-10-06T05:15:24.851692+0000","last_became_active":"2023-10-06T05:15:24.848735+0000","last_became_peered":"2023-10-06T05:15:24.848735+0000","last_unstale":"2023-10-06T05:15:24.851692+0000","last_undegraded":"2023-10-06T05:15:24.851692+0000","last_fullsized":"2023-10-06T05:15:24.851692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.352940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843698+0000","last_change":"2023-10-06T05:15:24.842052+0000","last_active":"2023-10-06T05:15:24.843698+0000","last_peered":"2023-10-06T05:15:24.843698+0000","last_clean":"2023-10-06T05:15:24.843698+0000","last_became_active":"2023-10-06T05:15:24.841746+0000","last_became_peered":"2023-10-06T05:15:24.841746+0000","last_unstale":"2023-10-06T05:15:24.843698+0000","last_undegraded":"2023-10-06T05:15:24.843698+0000","last_fullsized":"2023-10-06T05:15:24.843698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:51.116140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.833658+0000","last_change":"2023-10-06T05:15:24.833658+0000","last_active":"2023-10-06T05:15:24.833658+0000","last_peered":"2023-10-06T05:15:24.833658+0000","last_clean":"2023-10-06T05:15:24.833658+0000","last_became_active":"2023-10-06T05:15:24.833305+0000","last_became_peered":"2023-10-06T05:15:24.833305+0000","last_unstale":"2023-10-06T05:15:24.833658+0000","last_undegraded":"2023-10-06T05:15:24.833658+0000","last_fullsized":"2023-10-06T05:15:24.833658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:18.757351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.850871+0000","last_change":"2023-10-06T05:15:24.843795+0000","last_active":"2023-10-06T05:15:24.850871+0000","last_peered":"2023-10-06T05:15:24.850871+0000","last_clean":"2023-10-06T05:15:24.850871+0000","last_became_active":"2023-10-06T05:15:24.843169+0000","last_became_peered":"2023-10-06T05:15:24.843169+0000","last_unstale":"2023-10-06T05:15:24.850871+0000","last_undegraded":"2023-10-06T05:15:24.850871+0000","last_fullsized":"2023-10-06T05:15:24.850871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845830+0000","last_change":"2023-10-06T05:15:24.842782+0000","last_active":"2023-10-06T05:15:24.845830+0000","last_peered":"2023-10-06T05:15:24.845830+0000","last_clean":"2023-10-06T05:15:24.845830+0000","last_became_active":"2023-10-06T05:15:24.842516+0000","last_became_peered":"2023-10-06T05:15:24.842516+0000","last_unstale":"2023-10-06T05:15:24.845830+0000","last_undegraded":"2023-10-06T05:15:24.845830+0000","last_fullsized":"2023-10-06T05:15:24.845830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.849099+0000","last_change":"2023-10-06T05:15:24.849099+0000","last_active":"2023-10-06T05:15:24.849099+0000","last_peered":"2023-10-06T05:15:24.849099+0000","last_clean":"2023-10-06T05:15:24.849099+0000","last_became_active":"2023-10-06T05:15:24.848817+0000","last_became_peered":"2023-10-06T05:15:24.848817+0000","last_unstale":"2023-10-06T05:15:24.849099+0000","last_undegraded":"2023-10-06T05:15:24.849099+0000","last_fullsized":"2023-10-06T05:15:24.849099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845307+0000","last_change":"2023-10-06T05:15:24.842313+0000","last_active":"2023-10-06T05:15:24.845307+0000","last_peered":"2023-10-06T05:15:24.845307+0000","last_clean":"2023-10-06T05:15:24.845307+0000","last_became_active":"2023-10-06T05:15:24.841715+0000","last_became_peered":"2023-10-06T05:15:24.841715+0000","last_unstale":"2023-10-06T05:15:24.845307+0000","last_undegraded":"2023-10-06T05:15:24.845307+0000","last_fullsized":"2023-10-06T05:15:24.845307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.686660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863441+0000","last_change":"2023-10-06T05:15:24.859053+0000","last_active":"2023-10-06T05:15:24.863441+0000","last_peered":"2023-10-06T05:15:24.863441+0000","last_clean":"2023-10-06T05:15:24.863441+0000","last_became_active":"2023-10-06T05:15:24.858752+0000","last_became_peered":"2023-10-06T05:15:24.858752+0000","last_unstale":"2023-10-06T05:15:24.863441+0000","last_undegraded":"2023-10-06T05:15:24.863441+0000","last_fullsized":"2023-10-06T05:15:24.863441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:07.029872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.828348+0000","last_change":"2023-10-06T05:15:24.828348+0000","last_active":"2023-10-06T05:15:24.828348+0000","last_peered":"2023-10-06T05:15:24.828348+0000","last_clean":"2023-10-06T05:15:24.828348+0000","last_became_active":"2023-10-06T05:15:24.828028+0000","last_became_peered":"2023-10-06T05:15:24.828028+0000","last_unstale":"2023-10-06T05:15:24.828348+0000","last_undegraded":"2023-10-06T05:15:24.828348+0000","last_fullsized":"2023-10-06T05:15:24.828348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845425+0000","last_change":"2023-10-06T05:15:24.842587+0000","last_active":"2023-10-06T05:15:24.845425+0000","last_peered":"2023-10-06T05:15:24.845425+0000","last_clean":"2023-10-06T05:15:24.845425+0000","last_became_active":"2023-10-06T05:15:24.841831+0000","last_became_peered":"2023-10-06T05:15:24.841831+0000","last_unstale":"2023-10-06T05:15:24.845425+0000","last_undegraded":"2023-10-06T05:15:24.845425+0000","last_fullsized":"2023-10-06T05:15:24.845425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.848254+0000","last_change":"2023-10-06T05:15:24.848254+0000","last_active":"2023-10-06T05:15:24.848254+0000","last_peered":"2023-10-06T05:15:24.848254+0000","last_clean":"2023-10-06T05:15:24.848254+0000","last_became_active":"2023-10-06T05:15:24.847953+0000","last_became_peered":"2023-10-06T05:15:24.847953+0000","last_unstale":"2023-10-06T05:15:24.848254+0000","last_undegraded":"2023-10-06T05:15:24.848254+0000","last_fullsized":"2023-10-06T05:15:24.848254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.850890+0000","last_change":"2023-10-06T05:15:24.843961+0000","last_active":"2023-10-06T05:15:24.850890+0000","last_peered":"2023-10-06T05:15:24.850890+0000","last_clean":"2023-10-06T05:15:24.850890+0000","last_became_active":"2023-10-06T05:15:24.843473+0000","last_became_peered":"2023-10-06T05:15:24.843473+0000","last_unstale":"2023-10-06T05:15:24.850890+0000","last_undegraded":"2023-10-06T05:15:24.850890+0000","last_fullsized":"2023-10-06T05:15:24.850890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:47.981780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845979+0000","last_change":"2023-10-06T05:15:24.845979+0000","last_active":"2023-10-06T05:15:24.845979+0000","last_peered":"2023-10-06T05:15:24.845979+0000","last_clean":"2023-10-06T05:15:24.845979+0000","last_became_active":"2023-10-06T05:15:24.845668+0000","last_became_peered":"2023-10-06T05:15:24.845668+0000","last_unstale":"2023-10-06T05:15:24.845979+0000","last_undegraded":"2023-10-06T05:15:24.845979+0000","last_fullsized":"2023-10-06T05:15:24.845979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:29.740946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844930+0000","last_change":"2023-10-06T05:15:24.844930+0000","last_active":"2023-10-06T05:15:24.844930+0000","last_peered":"2023-10-06T05:15:24.844930+0000","last_clean":"2023-10-06T05:15:24.844930+0000","last_became_active":"2023-10-06T05:15:24.844647+0000","last_became_peered":"2023-10-06T05:15:24.844647+0000","last_unstale":"2023-10-06T05:15:24.844930+0000","last_undegraded":"2023-10-06T05:15:24.844930+0000","last_fullsized":"2023-10-06T05:15:24.844930+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:07.029872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.857181+0000","last_change":"2023-10-06T05:15:24.851244+0000","last_active":"2023-10-06T05:15:24.857181+0000","last_peered":"2023-10-06T05:15:24.857181+0000","last_clean":"2023-10-06T05:15:24.857181+0000","last_became_active":"2023-10-06T05:15:24.850889+0000","last_became_peered":"2023-10-06T05:15:24.850889+0000","last_unstale":"2023-10-06T05:15:24.857181+0000","last_undegraded":"2023-10-06T05:15:24.857181+0000","last_fullsized":"2023-10-06T05:15:24.857181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:40.098336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.835666+0000","last_change":"2023-10-06T05:15:24.829961+0000","last_active":"2023-10-06T05:15:24.835666+0000","last_peered":"2023-10-06T05:15:24.835666+0000","last_clean":"2023-10-06T05:15:24.835666+0000","last_became_active":"2023-10-06T05:15:24.829560+0000","last_became_peered":"2023-10-06T05:15:24.829560+0000","last_unstale":"2023-10-06T05:15:24.835666+0000","last_undegraded":"2023-10-06T05:15:24.835666+0000","last_fullsized":"2023-10-06T05:15:24.835666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:49.108167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.857311+0000","last_change":"2023-10-06T05:15:24.851300+0000","last_active":"2023-10-06T05:15:24.857311+0000","last_peered":"2023-10-06T05:15:24.857311+0000","last_clean":"2023-10-06T05:15:24.857311+0000","last_became_active":"2023-10-06T05:15:24.851070+0000","last_became_peered":"2023-10-06T05:15:24.851070+0000","last_unstale":"2023-10-06T05:15:24.857311+0000","last_undegraded":"2023-10-06T05:15:24.857311+0000","last_fullsized":"2023-10-06T05:15:24.857311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:06.324525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845444+0000","last_change":"2023-10-06T05:15:24.842817+0000","last_active":"2023-10-06T05:15:24.845444+0000","last_peered":"2023-10-06T05:15:24.845444+0000","last_clean":"2023-10-06T05:15:24.845444+0000","last_became_active":"2023-10-06T05:15:24.841730+0000","last_became_peered":"2023-10-06T05:15:24.841730+0000","last_unstale":"2023-10-06T05:15:24.845444+0000","last_undegraded":"2023-10-06T05:15:24.845444+0000","last_fullsized":"2023-10-06T05:15:24.845444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:01.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846778+0000","last_change":"2023-10-06T05:15:24.846778+0000","last_active":"2023-10-06T05:15:24.846778+0000","last_peered":"2023-10-06T05:15:24.846778+0000","last_clean":"2023-10-06T05:15:24.846778+0000","last_became_active":"2023-10-06T05:15:24.846547+0000","last_became_peered":"2023-10-06T05:15:24.846547+0000","last_unstale":"2023-10-06T05:15:24.846778+0000","last_undegraded":"2023-10-06T05:15:24.846778+0000","last_fullsized":"2023-10-06T05:15:24.846778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:47.981780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843625+0000","last_change":"2023-10-06T05:15:24.843625+0000","last_active":"2023-10-06T05:15:24.843625+0000","last_peered":"2023-10-06T05:15:24.843625+0000","last_clean":"2023-10-06T05:15:24.843625+0000","last_became_active":"2023-10-06T05:15:24.841683+0000","last_became_peered":"2023-10-06T05:15:24.841683+0000","last_unstale":"2023-10-06T05:15:24.843625+0000","last_undegraded":"2023-10-06T05:15:24.843625+0000","last_fullsized":"2023-10-06T05:15:24.843625+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843508+0000","last_change":"2023-10-06T05:15:24.837480+0000","last_active":"2023-10-06T05:15:24.843508+0000","last_peered":"2023-10-06T05:15:24.843508+0000","last_clean":"2023-10-06T05:15:24.843508+0000","last_became_active":"2023-10-06T05:15:24.837094+0000","last_became_peered":"2023-10-06T05:15:24.837094+0000","last_unstale":"2023-10-06T05:15:24.843508+0000","last_undegraded":"2023-10-06T05:15:24.843508+0000","last_fullsized":"2023-10-06T05:15:24.843508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:57.105297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843333+0000","last_change":"2023-10-06T05:15:24.830962+0000","last_active":"2023-10-06T05:15:24.843333+0000","last_peered":"2023-10-06T05:15:24.843333+0000","last_clean":"2023-10-06T05:15:24.843333+0000","last_became_active":"2023-10-06T05:15:24.830644+0000","last_became_peered":"2023-10-06T05:15:24.830644+0000","last_unstale":"2023-10-06T05:15:24.843333+0000","last_undegraded":"2023-10-06T05:15:24.843333+0000","last_fullsized":"2023-10-06T05:15:24.843333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:49.108167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844750+0000","last_change":"2023-10-06T05:15:24.844750+0000","last_active":"2023-10-06T05:15:24.844750+0000","last_peered":"2023-10-06T05:15:24.844750+0000","last_clean":"2023-10-06T05:15:24.844750+0000","last_became_active":"2023-10-06T05:15:24.844454+0000","last_became_peered":"2023-10-06T05:15:24.844454+0000","last_unstale":"2023-10-06T05:15:24.844750+0000","last_undegraded":"2023-10-06T05:15:24.844750+0000","last_fullsized":"2023-10-06T05:15:24.844750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.686660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843327+0000","last_change":"2023-10-06T05:15:24.834415+0000","last_active":"2023-10-06T05:15:24.843327+0000","last_peered":"2023-10-06T05:15:24.843327+0000","last_clean":"2023-10-06T05:15:24.843327+0000","last_became_active":"2023-10-06T05:15:24.834113+0000","last_became_peered":"2023-10-06T05:15:24.834113+0000","last_unstale":"2023-10-06T05:15:24.843327+0000","last_undegraded":"2023-10-06T05:15:24.843327+0000","last_fullsized":"2023-10-06T05:15:24.843327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:40.366886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845363+0000","last_change":"2023-10-06T05:15:24.842623+0000","last_active":"2023-10-06T05:15:24.845363+0000","last_peered":"2023-10-06T05:15:24.845363+0000","last_clean":"2023-10-06T05:15:24.845363+0000","last_became_active":"2023-10-06T05:15:24.841851+0000","last_became_peered":"2023-10-06T05:15:24.841851+0000","last_unstale":"2023-10-06T05:15:24.845363+0000","last_undegraded":"2023-10-06T05:15:24.845363+0000","last_fullsized":"2023-10-06T05:15:24.845363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843899+0000","last_change":"2023-10-06T05:15:24.832222+0000","last_active":"2023-10-06T05:15:24.843899+0000","last_peered":"2023-10-06T05:15:24.843899+0000","last_clean":"2023-10-06T05:15:24.843899+0000","last_became_active":"2023-10-06T05:15:24.831908+0000","last_became_peered":"2023-10-06T05:15:24.831908+0000","last_unstale":"2023-10-06T05:15:24.843899+0000","last_undegraded":"2023-10-06T05:15:24.843899+0000","last_fullsized":"2023-10-06T05:15:24.843899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:51.116140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843253+0000","last_change":"2023-10-06T05:15:24.839018+0000","last_active":"2023-10-06T05:15:24.843253+0000","last_peered":"2023-10-06T05:15:24.843253+0000","last_clean":"2023-10-06T05:15:24.843253+0000","last_became_active":"2023-10-06T05:15:24.838354+0000","last_became_peered":"2023-10-06T05:15:24.838354+0000","last_unstale":"2023-10-06T05:15:24.843253+0000","last_undegraded":"2023-10-06T05:15:24.843253+0000","last_fullsized":"2023-10-06T05:15:24.843253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:04.635140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.838007+0000","last_change":"2023-10-06T05:15:24.838007+0000","last_active":"2023-10-06T05:15:24.838007+0000","last_peered":"2023-10-06T05:15:24.838007+0000","last_clean":"2023-10-06T05:15:24.838007+0000","last_became_active":"2023-10-06T05:15:24.837742+0000","last_became_peered":"2023-10-06T05:15:24.837742+0000","last_unstale":"2023-10-06T05:15:24.838007+0000","last_undegraded":"2023-10-06T05:15:24.838007+0000","last_fullsized":"2023-10-06T05:15:24.838007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.834712+0000","last_change":"2023-10-06T05:15:24.834712+0000","last_active":"2023-10-06T05:15:24.834712+0000","last_peered":"2023-10-06T05:15:24.834712+0000","last_clean":"2023-10-06T05:15:24.834712+0000","last_became_active":"2023-10-06T05:15:24.834457+0000","last_became_peered":"2023-10-06T05:15:24.834457+0000","last_unstale":"2023-10-06T05:15:24.834712+0000","last_undegraded":"2023-10-06T05:15:24.834712+0000","last_fullsized":"2023-10-06T05:15:24.834712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:59.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844484+0000","last_change":"2023-10-06T05:15:24.844484+0000","last_active":"2023-10-06T05:15:24.844484+0000","last_peered":"2023-10-06T05:15:24.844484+0000","last_clean":"2023-10-06T05:15:24.844484+0000","last_became_active":"2023-10-06T05:15:24.844204+0000","last_became_peered":"2023-10-06T05:15:24.844204+0000","last_unstale":"2023-10-06T05:15:24.844484+0000","last_undegraded":"2023-10-06T05:15:24.844484+0000","last_fullsized":"2023-10-06T05:15:24.844484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:38.333907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.861458+0000","last_change":"2023-10-06T05:15:23.823585+0000","last_active":"2023-10-06T05:15:24.861458+0000","last_peered":"2023-10-06T05:15:24.861458+0000","last_clean":"2023-10-06T05:15:24.861458+0000","last_became_active":"2023-10-06T05:15:23.823272+0000","last_became_peered":"2023-10-06T05:15:23.823272+0000","last_unstale":"2023-10-06T05:15:24.861458+0000","last_undegraded":"2023-10-06T05:15:24.861458+0000","last_fullsized":"2023-10-06T05:15:24.861458+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844394+0000","last_change":"2023-10-06T05:15:24.844394+0000","last_active":"2023-10-06T05:15:24.844394+0000","last_peered":"2023-10-06T05:15:24.844394+0000","last_clean":"2023-10-06T05:15:24.844394+0000","last_became_active":"2023-10-06T05:15:24.844116+0000","last_became_peered":"2023-10-06T05:15:24.844116+0000","last_unstale":"2023-10-06T05:15:24.844394+0000","last_undegraded":"2023-10-06T05:15:24.844394+0000","last_fullsized":"2023-10-06T05:15:24.844394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:14.757703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.816386+0000","last_change":"2023-10-06T05:15:23.817646+0000","last_active":"2023-10-06T05:15:24.816386+0000","last_peered":"2023-10-06T05:15:24.816386+0000","last_clean":"2023-10-06T05:15:24.816386+0000","last_became_active":"2023-10-06T05:15:23.817373+0000","last_became_peered":"2023-10-06T05:15:23.817373+0000","last_unstale":"2023-10-06T05:15:24.816386+0000","last_undegraded":"2023-10-06T05:15:24.816386+0000","last_fullsized":"2023-10-06T05:15:24.816386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:09.050751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846926+0000","last_change":"2023-10-06T05:15:24.846926+0000","last_active":"2023-10-06T05:15:24.846926+0000","last_peered":"2023-10-06T05:15:24.846926+0000","last_clean":"2023-10-06T05:15:24.846926+0000","last_became_active":"2023-10-06T05:15:24.846624+0000","last_became_peered":"2023-10-06T05:15:24.846624+0000","last_unstale":"2023-10-06T05:15:24.846926+0000","last_undegraded":"2023-10-06T05:15:24.846926+0000","last_fullsized":"2023-10-06T05:15:24.846926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:01.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.860349+0000","last_change":"2023-10-06T05:15:23.827475+0000","last_active":"2023-10-06T05:15:24.860349+0000","last_peered":"2023-10-06T05:15:24.860349+0000","last_clean":"2023-10-06T05:15:24.860349+0000","last_became_active":"2023-10-06T05:15:23.827183+0000","last_became_peered":"2023-10-06T05:15:23.827183+0000","last_unstale":"2023-10-06T05:15:24.860349+0000","last_undegraded":"2023-10-06T05:15:24.860349+0000","last_fullsized":"2023-10-06T05:15:24.860349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:39.353055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843230+0000","last_change":"2023-10-06T05:15:24.839504+0000","last_active":"2023-10-06T05:15:24.843230+0000","last_peered":"2023-10-06T05:15:24.843230+0000","last_clean":"2023-10-06T05:15:24.843230+0000","last_became_active":"2023-10-06T05:15:24.839085+0000","last_became_peered":"2023-10-06T05:15:24.839085+0000","last_unstale":"2023-10-06T05:15:24.843230+0000","last_undegraded":"2023-10-06T05:15:24.843230+0000","last_fullsized":"2023-10-06T05:15:24.843230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:12.285635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.859930+0000","last_change":"2023-10-06T05:15:23.821228+0000","last_active":"2023-10-06T05:15:24.859930+0000","last_peered":"2023-10-06T05:15:24.859930+0000","last_clean":"2023-10-06T05:15:24.859930+0000","last_became_active":"2023-10-06T05:15:23.820965+0000","last_became_peered":"2023-10-06T05:15:23.820965+0000","last_unstale":"2023-10-06T05:15:24.859930+0000","last_undegraded":"2023-10-06T05:15:24.859930+0000","last_fullsized":"2023-10-06T05:15:24.859930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.751768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.832128+0000","last_change":"2023-10-06T05:15:24.832128+0000","last_active":"2023-10-06T05:15:24.832128+0000","last_peered":"2023-10-06T05:15:24.832128+0000","last_clean":"2023-10-06T05:15:24.832128+0000","last_became_active":"2023-10-06T05:15:24.831864+0000","last_became_peered":"2023-10-06T05:15:24.831864+0000","last_unstale":"2023-10-06T05:15:24.832128+0000","last_undegraded":"2023-10-06T05:15:24.832128+0000","last_fullsized":"2023-10-06T05:15:24.832128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:07.029872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863351+0000","last_change":"2023-10-06T05:15:23.821870+0000","last_active":"2023-10-06T05:15:24.863351+0000","last_peered":"2023-10-06T05:15:24.863351+0000","last_clean":"2023-10-06T05:15:24.863351+0000","last_became_active":"2023-10-06T05:15:23.821587+0000","last_became_peered":"2023-10-06T05:15:23.821587+0000","last_unstale":"2023-10-06T05:15:24.863351+0000","last_undegraded":"2023-10-06T05:15:24.863351+0000","last_fullsized":"2023-10-06T05:15:24.863351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:35.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.847422+0000","last_change":"2023-10-06T05:15:24.842903+0000","last_active":"2023-10-06T05:15:24.847422+0000","last_peered":"2023-10-06T05:15:24.847422+0000","last_clean":"2023-10-06T05:15:24.847422+0000","last_became_active":"2023-10-06T05:15:24.841747+0000","last_became_peered":"2023-10-06T05:15:24.841747+0000","last_unstale":"2023-10-06T05:15:24.847422+0000","last_undegraded":"2023-10-06T05:15:24.847422+0000","last_fullsized":"2023-10-06T05:15:24.847422+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:32.781863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.830409+0000","last_change":"2023-10-06T05:15:23.816886+0000","last_active":"2023-10-06T05:15:24.830409+0000","last_peered":"2023-10-06T05:15:24.830409+0000","last_clean":"2023-10-06T05:15:24.830409+0000","last_became_active":"2023-10-06T05:15:23.816540+0000","last_became_peered":"2023-10-06T05:15:23.816540+0000","last_unstale":"2023-10-06T05:15:24.830409+0000","last_undegraded":"2023-10-06T05:15:24.830409+0000","last_fullsized":"2023-10-06T05:15:24.830409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.853398+0000","last_change":"2023-10-06T05:15:24.853398+0000","last_active":"2023-10-06T05:15:24.853398+0000","last_peered":"2023-10-06T05:15:24.853398+0000","last_clean":"2023-10-06T05:15:24.853398+0000","last_became_active":"2023-10-06T05:15:24.853116+0000","last_became_peered":"2023-10-06T05:15:24.853116+0000","last_unstale":"2023-10-06T05:15:24.853398+0000","last_undegraded":"2023-10-06T05:15:24.853398+0000","last_fullsized":"2023-10-06T05:15:24.853398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:12.285635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832878+0000","last_change":"2023-10-06T05:15:23.829293+0000","last_active":"2023-10-06T05:15:23.832878+0000","last_peered":"2023-10-06T05:15:23.832878+0000","last_clean":"2023-10-06T05:15:23.832878+0000","last_became_active":"2023-10-06T05:15:23.828962+0000","last_became_peered":"2023-10-06T05:15:23.828962+0000","last_unstale":"2023-10-06T05:15:23.832878+0000","last_undegraded":"2023-10-06T05:15:23.832878+0000","last_fullsized":"2023-10-06T05:15:23.832878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:35.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839234+0000","last_change":"2023-10-06T05:15:24.835973+0000","last_active":"2023-10-06T05:15:24.839234+0000","last_peered":"2023-10-06T05:15:24.839234+0000","last_clean":"2023-10-06T05:15:24.839234+0000","last_became_active":"2023-10-06T05:15:24.835583+0000","last_became_peered":"2023-10-06T05:15:24.835583+0000","last_unstale":"2023-10-06T05:15:24.839234+0000","last_undegraded":"2023-10-06T05:15:24.839234+0000","last_fullsized":"2023-10-06T05:15:24.839234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:01.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832390+0000","last_change":"2023-10-06T05:15:23.828406+0000","last_active":"2023-10-06T05:15:23.832390+0000","last_peered":"2023-10-06T05:15:23.832390+0000","last_clean":"2023-10-06T05:15:23.832390+0000","last_became_active":"2023-10-06T05:15:23.827803+0000","last_became_peered":"2023-10-06T05:15:23.827803+0000","last_unstale":"2023-10-06T05:15:23.832390+0000","last_undegraded":"2023-10-06T05:15:23.832390+0000","last_fullsized":"2023-10-06T05:15:23.832390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:31.565511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.843650+0000","last_change":"2023-10-06T05:15:24.841850+0000","last_active":"2023-10-06T05:15:24.843650+0000","last_peered":"2023-10-06T05:15:24.843650+0000","last_clean":"2023-10-06T05:15:24.843650+0000","last_became_active":"2023-10-06T05:15:24.841600+0000","last_became_peered":"2023-10-06T05:15:24.841600+0000","last_unstale":"2023-10-06T05:15:24.843650+0000","last_undegraded":"2023-10-06T05:15:24.843650+0000","last_fullsized":"2023-10-06T05:15:24.843650+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:40.358638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.821925+0000","last_change":"2023-10-06T05:15:23.824145+0000","last_active":"2023-10-06T05:15:24.821925+0000","last_peered":"2023-10-06T05:15:24.821925+0000","last_clean":"2023-10-06T05:15:24.821925+0000","last_became_active":"2023-10-06T05:15:23.823966+0000","last_became_peered":"2023-10-06T05:15:23.823966+0000","last_unstale":"2023-10-06T05:15:24.821925+0000","last_undegraded":"2023-10-06T05:15:24.821925+0000","last_fullsized":"2023-10-06T05:15:24.821925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:07.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844397+0000","last_change":"2023-10-06T05:15:24.844397+0000","last_active":"2023-10-06T05:15:24.844397+0000","last_peered":"2023-10-06T05:15:24.844397+0000","last_clean":"2023-10-06T05:15:24.844397+0000","last_became_active":"2023-10-06T05:15:24.844088+0000","last_became_peered":"2023-10-06T05:15:24.844088+0000","last_unstale":"2023-10-06T05:15:24.844397+0000","last_undegraded":"2023-10-06T05:15:24.844397+0000","last_fullsized":"2023-10-06T05:15:24.844397+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:32.781863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.827625+0000","last_change":"2023-10-06T05:15:23.824400+0000","last_active":"2023-10-06T05:15:24.827625+0000","last_peered":"2023-10-06T05:15:24.827625+0000","last_clean":"2023-10-06T05:15:24.827625+0000","last_became_active":"2023-10-06T05:15:23.824106+0000","last_became_peered":"2023-10-06T05:15:23.824106+0000","last_unstale":"2023-10-06T05:15:24.827625+0000","last_undegraded":"2023-10-06T05:15:24.827625+0000","last_fullsized":"2023-10-06T05:15:24.827625+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:56.099714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.836366+0000","last_change":"2023-10-06T05:15:24.836366+0000","last_active":"2023-10-06T05:15:24.836366+0000","last_peered":"2023-10-06T05:15:24.836366+0000","last_clean":"2023-10-06T05:15:24.836366+0000","last_became_active":"2023-10-06T05:15:24.836031+0000","last_became_peered":"2023-10-06T05:15:24.836031+0000","last_unstale":"2023-10-06T05:15:24.836366+0000","last_undegraded":"2023-10-06T05:15:24.836366+0000","last_fullsized":"2023-10-06T05:15:24.836366+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:32.781863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.865243+0000","last_change":"2023-10-06T05:15:23.825407+0000","last_active":"2023-10-06T05:15:24.865243+0000","last_peered":"2023-10-06T05:15:24.865243+0000","last_clean":"2023-10-06T05:15:24.865243+0000","last_became_active":"2023-10-06T05:15:23.825059+0000","last_became_peered":"2023-10-06T05:15:23.825059+0000","last_unstale":"2023-10-06T05:15:24.865243+0000","last_undegraded":"2023-10-06T05:15:24.865243+0000","last_fullsized":"2023-10-06T05:15:24.865243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846155+0000","last_change":"2023-10-06T05:15:24.846155+0000","last_active":"2023-10-06T05:15:24.846155+0000","last_peered":"2023-10-06T05:15:24.846155+0000","last_clean":"2023-10-06T05:15:24.846155+0000","last_became_active":"2023-10-06T05:15:24.845894+0000","last_became_peered":"2023-10-06T05:15:24.845894+0000","last_unstale":"2023-10-06T05:15:24.846155+0000","last_undegraded":"2023-10-06T05:15:24.846155+0000","last_fullsized":"2023-10-06T05:15:24.846155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.865115+0000","last_change":"2023-10-06T05:15:23.824972+0000","last_active":"2023-10-06T05:15:24.865115+0000","last_peered":"2023-10-06T05:15:24.865115+0000","last_clean":"2023-10-06T05:15:24.865115+0000","last_became_active":"2023-10-06T05:15:23.824690+0000","last_became_peered":"2023-10-06T05:15:23.824690+0000","last_unstale":"2023-10-06T05:15:24.865115+0000","last_undegraded":"2023-10-06T05:15:24.865115+0000","last_fullsized":"2023-10-06T05:15:24.865115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:16.309623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.842326+0000","last_change":"2023-10-06T05:15:24.842326+0000","last_active":"2023-10-06T05:15:24.842326+0000","last_peered":"2023-10-06T05:15:24.842326+0000","last_clean":"2023-10-06T05:15:24.842326+0000","last_became_active":"2023-10-06T05:15:24.841947+0000","last_became_peered":"2023-10-06T05:15:24.841947+0000","last_unstale":"2023-10-06T05:15:24.842326+0000","last_undegraded":"2023-10-06T05:15:24.842326+0000","last_fullsized":"2023-10-06T05:15:24.842326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.855637+0000","last_change":"2023-10-06T05:15:23.810649+0000","last_active":"2023-10-06T05:15:24.855637+0000","last_peered":"2023-10-06T05:15:24.855637+0000","last_clean":"2023-10-06T05:15:24.855637+0000","last_became_active":"2023-10-06T05:15:23.810340+0000","last_became_peered":"2023-10-06T05:15:23.810340+0000","last_unstale":"2023-10-06T05:15:24.855637+0000","last_undegraded":"2023-10-06T05:15:24.855637+0000","last_fullsized":"2023-10-06T05:15:24.855637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:33.300490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.831057+0000","last_change":"2023-10-06T05:15:24.826890+0000","last_active":"2023-10-06T05:15:24.831057+0000","last_peered":"2023-10-06T05:15:24.831057+0000","last_clean":"2023-10-06T05:15:24.831057+0000","last_became_active":"2023-10-06T05:15:24.826618+0000","last_became_peered":"2023-10-06T05:15:24.826618+0000","last_unstale":"2023-10-06T05:15:24.831057+0000","last_undegraded":"2023-10-06T05:15:24.831057+0000","last_fullsized":"2023-10-06T05:15:24.831057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:38.333907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832899+0000","last_change":"2023-10-06T05:15:23.829383+0000","last_active":"2023-10-06T05:15:23.832899+0000","last_peered":"2023-10-06T05:15:23.832899+0000","last_clean":"2023-10-06T05:15:23.832899+0000","last_became_active":"2023-10-06T05:15:23.828934+0000","last_became_peered":"2023-10-06T05:15:23.828934+0000","last_unstale":"2023-10-06T05:15:23.832899+0000","last_undegraded":"2023-10-06T05:15:23.832899+0000","last_fullsized":"2023-10-06T05:15:23.832899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.842018+0000","last_change":"2023-10-06T05:15:24.842018+0000","last_active":"2023-10-06T05:15:24.842018+0000","last_peered":"2023-10-06T05:15:24.842018+0000","last_clean":"2023-10-06T05:15:24.842018+0000","last_became_active":"2023-10-06T05:15:24.841744+0000","last_became_peered":"2023-10-06T05:15:24.841744+0000","last_unstale":"2023-10-06T05:15:24.842018+0000","last_undegraded":"2023-10-06T05:15:24.842018+0000","last_fullsized":"2023-10-06T05:15:24.842018+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.822818+0000","last_change":"2023-10-06T05:15:23.816494+0000","last_active":"2023-10-06T05:15:24.822818+0000","last_peered":"2023-10-06T05:15:24.822818+0000","last_clean":"2023-10-06T05:15:24.822818+0000","last_became_active":"2023-10-06T05:15:23.816245+0000","last_became_peered":"2023-10-06T05:15:23.816245+0000","last_unstale":"2023-10-06T05:15:24.822818+0000","last_undegraded":"2023-10-06T05:15:24.822818+0000","last_fullsized":"2023-10-06T05:15:24.822818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:09.136217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839251+0000","last_change":"2023-10-06T05:15:24.836863+0000","last_active":"2023-10-06T05:15:24.839251+0000","last_peered":"2023-10-06T05:15:24.839251+0000","last_clean":"2023-10-06T05:15:24.839251+0000","last_became_active":"2023-10-06T05:15:24.836605+0000","last_became_peered":"2023-10-06T05:15:24.836605+0000","last_unstale":"2023-10-06T05:15:24.839251+0000","last_undegraded":"2023-10-06T05:15:24.839251+0000","last_fullsized":"2023-10-06T05:15:24.839251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.860479+0000","last_change":"2023-10-06T05:15:23.816184+0000","last_active":"2023-10-06T05:15:24.860479+0000","last_peered":"2023-10-06T05:15:24.860479+0000","last_clean":"2023-10-06T05:15:24.860479+0000","last_became_active":"2023-10-06T05:15:23.815897+0000","last_became_peered":"2023-10-06T05:15:23.815897+0000","last_unstale":"2023-10-06T05:15:24.860479+0000","last_undegraded":"2023-10-06T05:15:24.860479+0000","last_fullsized":"2023-10-06T05:15:24.860479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839228+0000","last_change":"2023-10-06T05:15:24.837832+0000","last_active":"2023-10-06T05:15:24.839228+0000","last_peered":"2023-10-06T05:15:24.839228+0000","last_clean":"2023-10-06T05:15:24.839228+0000","last_became_active":"2023-10-06T05:15:24.837542+0000","last_became_peered":"2023-10-06T05:15:24.837542+0000","last_unstale":"2023-10-06T05:15:24.839228+0000","last_undegraded":"2023-10-06T05:15:24.839228+0000","last_fullsized":"2023-10-06T05:15:24.839228+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863612+0000","last_change":"2023-10-06T05:15:23.821032+0000","last_active":"2023-10-06T05:15:24.863612+0000","last_peered":"2023-10-06T05:15:24.863612+0000","last_clean":"2023-10-06T05:15:24.863612+0000","last_became_active":"2023-10-06T05:15:23.820789+0000","last_became_peered":"2023-10-06T05:15:23.820789+0000","last_unstale":"2023-10-06T05:15:24.863612+0000","last_undegraded":"2023-10-06T05:15:24.863612+0000","last_fullsized":"2023-10-06T05:15:24.863612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:59.538238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.855413+0000","last_change":"2023-10-06T05:15:24.845844+0000","last_active":"2023-10-06T05:15:24.855413+0000","last_peered":"2023-10-06T05:15:24.855413+0000","last_clean":"2023-10-06T05:15:24.855413+0000","last_became_active":"2023-10-06T05:15:24.845538+0000","last_became_peered":"2023-10-06T05:15:24.845538+0000","last_unstale":"2023-10-06T05:15:24.855413+0000","last_undegraded":"2023-10-06T05:15:24.855413+0000","last_fullsized":"2023-10-06T05:15:24.855413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:46.975560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.862733+0000","last_change":"2023-10-06T05:15:23.822015+0000","last_active":"2023-10-06T05:15:24.862733+0000","last_peered":"2023-10-06T05:15:24.862733+0000","last_clean":"2023-10-06T05:15:24.862733+0000","last_became_active":"2023-10-06T05:15:23.821690+0000","last_became_peered":"2023-10-06T05:15:23.821690+0000","last_unstale":"2023-10-06T05:15:24.862733+0000","last_undegraded":"2023-10-06T05:15:24.862733+0000","last_fullsized":"2023-10-06T05:15:24.862733+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:13.752120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.840551+0000","last_change":"2023-10-06T05:15:24.840551+0000","last_active":"2023-10-06T05:15:24.840551+0000","last_peered":"2023-10-06T05:15:24.840551+0000","last_clean":"2023-10-06T05:15:24.840551+0000","last_became_active":"2023-10-06T05:15:24.840254+0000","last_became_peered":"2023-10-06T05:15:24.840254+0000","last_unstale":"2023-10-06T05:15:24.840551+0000","last_undegraded":"2023-10-06T05:15:24.840551+0000","last_fullsized":"2023-10-06T05:15:24.840551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:19.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.862784+0000","last_change":"2023-10-06T05:15:23.827704+0000","last_active":"2023-10-06T05:15:24.862784+0000","last_peered":"2023-10-06T05:15:24.862784+0000","last_clean":"2023-10-06T05:15:24.862784+0000","last_became_active":"2023-10-06T05:15:23.827417+0000","last_became_peered":"2023-10-06T05:15:23.827417+0000","last_unstale":"2023-10-06T05:15:24.862784+0000","last_undegraded":"2023-10-06T05:15:24.862784+0000","last_fullsized":"2023-10-06T05:15:24.862784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.835636+0000","last_change":"2023-10-06T05:15:24.829282+0000","last_active":"2023-10-06T05:15:24.835636+0000","last_peered":"2023-10-06T05:15:24.835636+0000","last_clean":"2023-10-06T05:15:24.835636+0000","last_became_active":"2023-10-06T05:15:24.828978+0000","last_became_peered":"2023-10-06T05:15:24.828978+0000","last_unstale":"2023-10-06T05:15:24.835636+0000","last_undegraded":"2023-10-06T05:15:24.835636+0000","last_fullsized":"2023-10-06T05:15:24.835636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:40.366886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.830792+0000","last_change":"2023-10-06T05:15:23.828705+0000","last_active":"2023-10-06T05:15:23.830792+0000","last_peered":"2023-10-06T05:15:23.830792+0000","last_clean":"2023-10-06T05:15:23.830792+0000","last_became_active":"2023-10-06T05:15:23.828442+0000","last_became_peered":"2023-10-06T05:15:23.828442+0000","last_unstale":"2023-10-06T05:15:23.830792+0000","last_undegraded":"2023-10-06T05:15:23.830792+0000","last_fullsized":"2023-10-06T05:15:23.830792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:56.099714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.840261+0000","last_change":"2023-10-06T05:15:24.840261+0000","last_active":"2023-10-06T05:15:24.840261+0000","last_peered":"2023-10-06T05:15:24.840261+0000","last_clean":"2023-10-06T05:15:24.840261+0000","last_became_active":"2023-10-06T05:15:24.838724+0000","last_became_peered":"2023-10-06T05:15:24.838724+0000","last_unstale":"2023-10-06T05:15:24.840261+0000","last_undegraded":"2023-10-06T05:15:24.840261+0000","last_fullsized":"2023-10-06T05:15:24.840261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.857674+0000","last_change":"2023-10-06T05:15:23.820198+0000","last_active":"2023-10-06T05:15:24.857674+0000","last_peered":"2023-10-06T05:15:24.857674+0000","last_clean":"2023-10-06T05:15:24.857674+0000","last_became_active":"2023-10-06T05:15:23.819831+0000","last_became_peered":"2023-10-06T05:15:23.819831+0000","last_unstale":"2023-10-06T05:15:24.857674+0000","last_undegraded":"2023-10-06T05:15:24.857674+0000","last_fullsized":"2023-10-06T05:15:24.857674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:33.001187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.836805+0000","last_change":"2023-10-06T05:15:24.836805+0000","last_active":"2023-10-06T05:15:24.836805+0000","last_peered":"2023-10-06T05:15:24.836805+0000","last_clean":"2023-10-06T05:15:24.836805+0000","last_became_active":"2023-10-06T05:15:24.836512+0000","last_became_peered":"2023-10-06T05:15:24.836512+0000","last_unstale":"2023-10-06T05:15:24.836805+0000","last_undegraded":"2023-10-06T05:15:24.836805+0000","last_fullsized":"2023-10-06T05:15:24.836805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:15.984185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820229+0000","last_change":"2023-10-06T05:15:23.828529+0000","last_active":"2023-10-06T05:15:24.820229+0000","last_peered":"2023-10-06T05:15:24.820229+0000","last_clean":"2023-10-06T05:15:24.820229+0000","last_became_active":"2023-10-06T05:15:23.828273+0000","last_became_peered":"2023-10-06T05:15:23.828273+0000","last_unstale":"2023-10-06T05:15:24.820229+0000","last_undegraded":"2023-10-06T05:15:24.820229+0000","last_fullsized":"2023-10-06T05:15:24.820229+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:25.833446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845485+0000","last_change":"2023-10-06T05:15:24.842460+0000","last_active":"2023-10-06T05:15:24.845485+0000","last_peered":"2023-10-06T05:15:24.845485+0000","last_clean":"2023-10-06T05:15:24.845485+0000","last_became_active":"2023-10-06T05:15:24.842214+0000","last_became_peered":"2023-10-06T05:15:24.842214+0000","last_unstale":"2023-10-06T05:15:24.845485+0000","last_undegraded":"2023-10-06T05:15:24.845485+0000","last_fullsized":"2023-10-06T05:15:24.845485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:26.192843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.831017+0000","last_change":"2023-10-06T05:15:23.828196+0000","last_active":"2023-10-06T05:15:23.831017+0000","last_peered":"2023-10-06T05:15:23.831017+0000","last_clean":"2023-10-06T05:15:23.831017+0000","last_became_active":"2023-10-06T05:15:23.827716+0000","last_became_peered":"2023-10-06T05:15:23.827716+0000","last_unstale":"2023-10-06T05:15:23.831017+0000","last_undegraded":"2023-10-06T05:15:23.831017+0000","last_fullsized":"2023-10-06T05:15:23.831017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:56.099714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.839213+0000","last_change":"2023-10-06T05:15:24.835538+0000","last_active":"2023-10-06T05:15:24.839213+0000","last_peered":"2023-10-06T05:15:24.839213+0000","last_clean":"2023-10-06T05:15:24.839213+0000","last_became_active":"2023-10-06T05:15:24.835234+0000","last_became_peered":"2023-10-06T05:15:24.835234+0000","last_unstale":"2023-10-06T05:15:24.839213+0000","last_undegraded":"2023-10-06T05:15:24.839213+0000","last_fullsized":"2023-10-06T05:15:24.839213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820016+0000","last_change":"2023-10-06T05:15:23.819215+0000","last_active":"2023-10-06T05:15:24.820016+0000","last_peered":"2023-10-06T05:15:24.820016+0000","last_clean":"2023-10-06T05:15:24.820016+0000","last_became_active":"2023-10-06T05:15:23.818917+0000","last_became_peered":"2023-10-06T05:15:23.818917+0000","last_unstale":"2023-10-06T05:15:24.820016+0000","last_undegraded":"2023-10-06T05:15:24.820016+0000","last_fullsized":"2023-10-06T05:15:24.820016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:37.775731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.827336+0000","last_change":"2023-10-06T05:15:24.827336+0000","last_active":"2023-10-06T05:15:24.827336+0000","last_peered":"2023-10-06T05:15:24.827336+0000","last_clean":"2023-10-06T05:15:24.827336+0000","last_became_active":"2023-10-06T05:15:24.827085+0000","last_became_peered":"2023-10-06T05:15:24.827085+0000","last_unstale":"2023-10-06T05:15:24.827336+0000","last_undegraded":"2023-10-06T05:15:24.827336+0000","last_fullsized":"2023-10-06T05:15:24.827336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:59.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.832545+0000","last_change":"2023-10-06T05:15:23.828400+0000","last_active":"2023-10-06T05:15:23.832545+0000","last_peered":"2023-10-06T05:15:23.832545+0000","last_clean":"2023-10-06T05:15:23.832545+0000","last_became_active":"2023-10-06T05:15:23.828099+0000","last_became_peered":"2023-10-06T05:15:23.828099+0000","last_unstale":"2023-10-06T05:15:23.832545+0000","last_undegraded":"2023-10-06T05:15:23.832545+0000","last_fullsized":"2023-10-06T05:15:23.832545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:07.019343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.817672+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:15:23.817672+0000","last_peered":"2023-10-06T05:15:23.817672+0000","last_clean":"2023-10-06T05:15:23.817672+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:15:23.817672+0000","last_undegraded":"2023-10-06T05:15:23.817672+0000","last_fullsized":"2023-10-06T05:15:23.817672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.845806+0000","last_change":"2023-10-06T05:15:24.841042+0000","last_active":"2023-10-06T05:15:24.845806+0000","last_peered":"2023-10-06T05:15:24.845806+0000","last_clean":"2023-10-06T05:15:24.845806+0000","last_became_active":"2023-10-06T05:15:24.840750+0000","last_became_peered":"2023-10-06T05:15:24.840750+0000","last_unstale":"2023-10-06T05:15:24.845806+0000","last_undegraded":"2023-10-06T05:15:24.845806+0000","last_fullsized":"2023-10-06T05:15:24.845806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:04.542305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.830219+0000","last_change":"2023-10-06T05:15:23.825201+0000","last_active":"2023-10-06T05:15:23.830219+0000","last_peered":"2023-10-06T05:15:23.830219+0000","last_clean":"2023-10-06T05:15:23.830219+0000","last_became_active":"2023-10-06T05:15:23.824900+0000","last_became_peered":"2023-10-06T05:15:23.824900+0000","last_unstale":"2023-10-06T05:15:23.830219+0000","last_undegraded":"2023-10-06T05:15:23.830219+0000","last_fullsized":"2023-10-06T05:15:23.830219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:33.300490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.825628+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:15:24.825628+0000","last_peered":"2023-10-06T05:15:24.825628+0000","last_clean":"2023-10-06T05:15:24.825628+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:15:24.825628+0000","last_undegraded":"2023-10-06T05:15:24.825628+0000","last_fullsized":"2023-10-06T05:15:24.825628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846155+0000","last_change":"2023-10-06T05:15:24.846155+0000","last_active":"2023-10-06T05:15:24.846155+0000","last_peered":"2023-10-06T05:15:24.846155+0000","last_clean":"2023-10-06T05:15:24.846155+0000","last_became_active":"2023-10-06T05:15:24.845842+0000","last_became_peered":"2023-10-06T05:15:24.845842+0000","last_unstale":"2023-10-06T05:15:24.846155+0000","last_undegraded":"2023-10-06T05:15:24.846155+0000","last_fullsized":"2023-10-06T05:15:24.846155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:26.192843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-06T05:15:25.836958+0000","last_change":"2023-10-06T05:15:23.829648+0000","last_active":"2023-10-06T05:15:25.836958+0000","last_peered":"2023-10-06T05:15:25.836958+0000","last_clean":"2023-10-06T05:15:25.836958+0000","last_became_active":"2023-10-06T05:15:23.829378+0000","last_became_peered":"2023-10-06T05:15:23.829378+0000","last_unstale":"2023-10-06T05:15:25.836958+0000","last_undegraded":"2023-10-06T05:15:25.836958+0000","last_fullsized":"2023-10-06T05:15:25.836958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:15.439776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820110+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:15:24.820110+0000","last_peered":"2023-10-06T05:15:24.820110+0000","last_clean":"2023-10-06T05:15:24.820110+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:15:24.820110+0000","last_undegraded":"2023-10-06T05:15:24.820110+0000","last_fullsized":"2023-10-06T05:15:24.820110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.851763+0000","last_change":"2023-10-06T05:15:24.842849+0000","last_active":"2023-10-06T05:15:24.851763+0000","last_peered":"2023-10-06T05:15:24.851763+0000","last_clean":"2023-10-06T05:15:24.851763+0000","last_became_active":"2023-10-06T05:15:24.841045+0000","last_became_peered":"2023-10-06T05:15:24.841045+0000","last_unstale":"2023-10-06T05:15:24.851763+0000","last_undegraded":"2023-10-06T05:15:24.851763+0000","last_fullsized":"2023-10-06T05:15:24.851763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:30.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.826811+0000","last_change":"2023-10-06T05:15:23.823464+0000","last_active":"2023-10-06T05:15:23.826811+0000","last_peered":"2023-10-06T05:15:23.826811+0000","last_clean":"2023-10-06T05:15:23.826811+0000","last_became_active":"2023-10-06T05:15:23.823272+0000","last_became_peered":"2023-10-06T05:15:23.823272+0000","last_unstale":"2023-10-06T05:15:23.826811+0000","last_undegraded":"2023-10-06T05:15:23.826811+0000","last_fullsized":"2023-10-06T05:15:23.826811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:35.347357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.829482+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:15:24.829482+0000","last_peered":"2023-10-06T05:15:24.829482+0000","last_clean":"2023-10-06T05:15:24.829482+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:15:24.829482+0000","last_undegraded":"2023-10-06T05:15:24.829482+0000","last_fullsized":"2023-10-06T05:15:24.829482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.807351+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:15:23.807351+0000","last_peered":"2023-10-06T05:15:23.807351+0000","last_clean":"2023-10-06T05:15:23.807351+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:15:23.807351+0000","last_undegraded":"2023-10-06T05:15:23.807351+0000","last_fullsized":"2023-10-06T05:15:23.807351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.844649+0000","last_change":"2023-10-06T05:15:24.844649+0000","last_active":"2023-10-06T05:15:24.844649+0000","last_peered":"2023-10-06T05:15:24.844649+0000","last_clean":"2023-10-06T05:15:24.844649+0000","last_became_active":"2023-10-06T05:15:24.844385+0000","last_became_peered":"2023-10-06T05:15:24.844385+0000","last_unstale":"2023-10-06T05:15:24.844649+0000","last_undegraded":"2023-10-06T05:15:24.844649+0000","last_fullsized":"2023-10-06T05:15:24.844649+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:15.984185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.863532+0000","last_change":"2023-10-06T05:15:23.825367+0000","last_active":"2023-10-06T05:15:24.863532+0000","last_peered":"2023-10-06T05:15:24.863532+0000","last_clean":"2023-10-06T05:15:24.863532+0000","last_became_active":"2023-10-06T05:15:23.825023+0000","last_became_peered":"2023-10-06T05:15:23.825023+0000","last_unstale":"2023-10-06T05:15:24.863532+0000","last_undegraded":"2023-10-06T05:15:24.863532+0000","last_fullsized":"2023-10-06T05:15:24.863532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.807437+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:15:23.807437+0000","last_peered":"2023-10-06T05:15:23.807437+0000","last_clean":"2023-10-06T05:15:23.807437+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:15:23.807437+0000","last_undegraded":"2023-10-06T05:15:23.807437+0000","last_fullsized":"2023-10-06T05:15:23.807437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.827359+0000","last_change":"2023-10-06T05:15:24.827359+0000","last_active":"2023-10-06T05:15:24.827359+0000","last_peered":"2023-10-06T05:15:24.827359+0000","last_clean":"2023-10-06T05:15:24.827359+0000","last_became_active":"2023-10-06T05:15:24.827081+0000","last_became_peered":"2023-10-06T05:15:24.827081+0000","last_unstale":"2023-10-06T05:15:24.827359+0000","last_undegraded":"2023-10-06T05:15:24.827359+0000","last_fullsized":"2023-10-06T05:15:24.827359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:32.571095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.823911+0000","last_change":"2023-10-06T05:15:23.819164+0000","last_active":"2023-10-06T05:15:24.823911+0000","last_peered":"2023-10-06T05:15:24.823911+0000","last_clean":"2023-10-06T05:15:24.823911+0000","last_became_active":"2023-10-06T05:15:23.818903+0000","last_became_peered":"2023-10-06T05:15:23.818903+0000","last_unstale":"2023-10-06T05:15:24.823911+0000","last_undegraded":"2023-10-06T05:15:24.823911+0000","last_fullsized":"2023-10-06T05:15:24.823911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:37.775731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.809579+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:15:23.809579+0000","last_peered":"2023-10-06T05:15:23.809579+0000","last_clean":"2023-10-06T05:15:23.809579+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:15:23.809579+0000","last_undegraded":"2023-10-06T05:15:23.809579+0000","last_fullsized":"2023-10-06T05:15:23.809579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.847750+0000","last_change":"2023-10-06T05:15:24.847750+0000","last_active":"2023-10-06T05:15:24.847750+0000","last_peered":"2023-10-06T05:15:24.847750+0000","last_clean":"2023-10-06T05:15:24.847750+0000","last_became_active":"2023-10-06T05:15:24.847118+0000","last_became_peered":"2023-10-06T05:15:24.847118+0000","last_unstale":"2023-10-06T05:15:24.847750+0000","last_undegraded":"2023-10-06T05:15:24.847750+0000","last_fullsized":"2023-10-06T05:15:24.847750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.412966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.826577+0000","last_change":"2023-10-06T05:15:23.819539+0000","last_active":"2023-10-06T05:15:23.826577+0000","last_peered":"2023-10-06T05:15:23.826577+0000","last_clean":"2023-10-06T05:15:23.826577+0000","last_became_active":"2023-10-06T05:15:23.819114+0000","last_became_peered":"2023-10-06T05:15:23.819114+0000","last_unstale":"2023-10-06T05:15:23.826577+0000","last_undegraded":"2023-10-06T05:15:23.826577+0000","last_fullsized":"2023-10-06T05:15:23.826577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:58.337389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T05:15:23.815010+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:15:23.815010+0000","last_peered":"2023-10-06T05:15:23.815010+0000","last_clean":"2023-10-06T05:15:23.815010+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:15:23.815010+0000","last_undegraded":"2023-10-06T05:15:23.815010+0000","last_fullsized":"2023-10-06T05:15:23.815010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.846211+0000","last_change":"2023-10-06T05:15:24.846211+0000","last_active":"2023-10-06T05:15:24.846211+0000","last_peered":"2023-10-06T05:15:24.846211+0000","last_clean":"2023-10-06T05:15:24.846211+0000","last_became_active":"2023-10-06T05:15:24.845881+0000","last_became_peered":"2023-10-06T05:15:24.845881+0000","last_unstale":"2023-10-06T05:15:24.846211+0000","last_undegraded":"2023-10-06T05:15:24.846211+0000","last_fullsized":"2023-10-06T05:15:24.846211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:23.798092+0000","last_clean_scrub_stamp":"2023-10-06T05:15:23.798092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:03.382517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.820933+0000","last_change":"2023-10-06T05:15:23.823659+0000","last_active":"2023-10-06T05:15:24.820933+0000","last_peered":"2023-10-06T05:15:24.820933+0000","last_clean":"2023-10-06T05:15:24.820933+0000","last_became_active":"2023-10-06T05:15:23.823474+0000","last_became_peered":"2023-10-06T05:15:23.823474+0000","last_unstale":"2023-10-06T05:15:24.820933+0000","last_undegraded":"2023-10-06T05:15:24.820933+0000","last_fullsized":"2023-10-06T05:15:24.820933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:22.792509+0000","last_clean_scrub_stamp":"2023-10-06T05:15:22.792509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:09.050751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T05:15:24.823265+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:15:24.823265+0000","last_peered":"2023-10-06T05:15:24.823265+0000","last_clean":"2023-10-06T05:15:24.823265+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:15:24.823265+0000","last_undegraded":"2023-10-06T05:15:24.823265+0000","last_fullsized":"2023-10-06T05:15:24.823265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":754443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.499},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.66000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297692,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074148,"statfs":{"total":96636764160,"available":96331927552,"internally_reserved":0,"allocated":413696,"data_stored":169664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":162559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.311},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.36599999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.624}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298260,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073580,"statfs":{"total":96636764160,"available":96331345920,"internally_reserved":0,"allocated":995328,"data_stored":752871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.314},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.749},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.878}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297692,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074148,"statfs":{"total":96636764160,"available":96331927552,"internally_reserved":0,"allocated":413696,"data_stored":169180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.37}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.80700000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.78000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":425984,"data_stored":170680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.38200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.98399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.82499999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.91300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":163045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.625},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.81599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297696,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074144,"statfs":{"total":96636764160,"available":96331923456,"internally_reserved":0,"allocated":417792,"data_stored":169222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.62},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.501},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:15:29.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.908+0000 7f0f73fff700 1 Processor -- start 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.908+0000 7f0f73fff700 1 -- start start 2023-10-06T05:15:29.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f73fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f740ac8e0 0x7f0f74161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f73fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f740c94d0 con 0x7f0f74155bd0 2023-10-06T05:15:29.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f73fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7406aab0 con 0x7f0f740ac460 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f73fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f74069ea0 con 0x7f0f740ac8e0 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f72ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0f74155bd0 0x7f0f740c8dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34552/0 (socket says 172.21.15.33:34552) 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f72ffd700 1 -- 172.21.15.33:0/446796986 learned_addr learned my addr 172.21.15.33:0/446796986 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.909+0000 7f0f737fe700 1 --2- 172.21.15.33:0/446796986 >> v2:172.21.15.33:3300/0 conn(0x7f0f740ac8e0 0x7f0f74161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f737fe700 1 -- 172.21.15.33:0/446796986 >> v1:172.21.15.167:6789/0 conn(0x7f0f740ac460 legacy=0x7f0f7415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f737fe700 1 -- 172.21.15.33:0/446796986 >> v1:172.21.15.33:6789/0 conn(0x7f0f74155bd0 legacy=0x7f0f740c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f737fe700 1 -- 172.21.15.33:0/446796986 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7406a180 con 0x7f0f740ac8e0 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f737fe700 1 --2- 172.21.15.33:0/446796986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac8e0 0x7f0f74161020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009e60 tx=0x7f0f6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b2cb8c3bda2d85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f71ffb700 1 -- 172.21.15.33:0/446796986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00bbf0 con 0x7f0f740ac8e0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f71ffb700 1 -- 172.21.15.33:0/446796986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c00bd50 con 0x7f0f740ac8e0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.910+0000 7f0f71ffb700 1 -- 172.21.15.33:0/446796986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c011650 con 0x7f0f740ac8e0 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 -- 172.21.15.33:0/446796986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac8e0 msgr2=0x7f0f74161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 --2- 172.21.15.33:0/446796986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac8e0 0x7f0f74161020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009e60 tx=0x7f0f6c009780 comp rx=0 tx=0).stop 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 -- 172.21.15.33:0/446796986 shutdown_connections 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 --2- 172.21.15.33:0/446796986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac8e0 0x7f0f74161020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 -- 172.21.15.33:0/446796986 >> 172.21.15.33:0/446796986 conn(0x7f0f7406c560 msgr2=0x7f0f7406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 -- 172.21.15.33:0/446796986 shutdown_connections 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.911+0000 7f0f73fff700 1 -- 172.21.15.33:0/446796986 wait complete. 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.912+0000 7f0f73fff700 1 Processor -- start 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.913+0000 7f0f73fff700 1 -- start start 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.913+0000 7f0f73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 0x7f0f741540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f73fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f740ac8e0 0x7f0f74169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f73fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f74155bd0 0x7f0f74159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f73fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7415b030 con 0x7f0f740ac460 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f73fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7415a7c0 con 0x7f0f74155bd0 2023-10-06T05:15:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f73fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7415a200 con 0x7f0f740ac8e0 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 0x7f0f741540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 0x7f0f741540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53916/0 (socket says 172.21.15.33:53916) 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 -- 172.21.15.33:0/61563677 learned_addr learned my addr 172.21.15.33:0/61563677 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f737fe700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f74155bd0 0x7f0f74159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f727fc700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f740ac8e0 0x7f0f74169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 -- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f740ac8e0 msgr2=0x7f0f74169ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f740ac8e0 0x7f0f74169ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 -- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f74155bd0 msgr2=0x7f0f74159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f74155bd0 0x7f0f74159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f72ffd700 1 -- 172.21.15.33:0/61563677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c009a20 con 0x7f0f740ac460 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.914+0000 7f0f727fc700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f740ac8e0 0x7f0f74169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.915+0000 7f0f72ffd700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 0x7f0f741540a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400c7c0 tx=0x7f0f64007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.915+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64005b30 con 0x7f0f740ac460 2023-10-06T05:15:29.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.915+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64005cb0 con 0x7f0f740ac460 2023-10-06T05:15:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.915+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64004a80 con 0x7f0f740ac460 2023-10-06T05:15:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.915+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7415a4e0 con 0x7f0f740ac460 2023-10-06T05:15:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.915+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f740abd50 con 0x7f0f740ac460 2023-10-06T05:15:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.916+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f740ac270 con 0x7f0f740ac460 2023-10-06T05:15:29.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.917+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f640070f0 con 0x7f0f740ac460 2023-10-06T05:15:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.919+0000 7f0f5affd700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f60066f30 0x7f0f600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.919+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0f640839e0 con 0x7f0f740ac460 2023-10-06T05:15:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.919+0000 7f0f727fc700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f60066f30 0x7f0f600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.920+0000 7f0f727fc700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f60066f30 0x7f0f600693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:29.923+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f64050ad0 con 0x7f0f740ac460 2023-10-06T05:15:30.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.210+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0f740c94d0 con 0x7f0f740ac460 2023-10-06T05:15:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.215+0000 7f0f5affd700 1 -- 172.21.15.33:0/61563677 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0f740c94d0 con 0x7f0f740ac460 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f60066f30 msgr2=0x7f0f600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f60066f30 0x7f0f600693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 msgr2=0x7f0f741540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 0x7f0f741540a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400c7c0 tx=0x7f0f64007f70 comp rx=0 tx=0).stop 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 shutdown_connections 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f60066f30 0x7f0f600693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f74155bd0 0x7f0f74159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f740ac460 0x7f0f741540a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 --2- 172.21.15.33:0/61563677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f740ac8e0 0x7f0f74169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 >> 172.21.15.33:0/61563677 conn(0x7f0f7406c560 msgr2=0x7f0f7406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 shutdown_connections 2023-10-06T05:15:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.217+0000 7f0f73fff700 1 -- 172.21.15.33:0/61563677 wait complete. 2023-10-06T05:15:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:15:30.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.421+0000 7f903899d700 1 Processor -- start 2023-10-06T05:15:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.421+0000 7f903899d700 1 -- start start 2023-10-06T05:15:30.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f903899d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9034155af0 0x7f903406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f903899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f903406a6a0 0x7f90340c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f903899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90340c8b50 0x7f90340c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f903899d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903406b7e0 con 0x7f9034155af0 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f903899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903406ab60 con 0x7f903406a6a0 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f903899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903406ae40 con 0x7f90340c8b50 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9034155af0 0x7f903406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9034155af0 0x7f903406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53920/0 (socket says 172.21.15.33:53920) 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 -- 172.21.15.33:0/278706851 learned_addr learned my addr 172.21.15.33:0/278706851 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f9032ffd700 1 --2- 172.21.15.33:0/278706851 >> v2:172.21.15.167:3300/0 conn(0x7f903406a6a0 0x7f90340c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f9033fff700 1 --2- 172.21.15.33:0/278706851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90340c8b50 0x7f90340c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 -- 172.21.15.33:0/278706851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90340c8b50 msgr2=0x7f90340c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:30.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 --2- 172.21.15.33:0/278706851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90340c8b50 0x7f90340c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 -- 172.21.15.33:0/278706851 >> v2:172.21.15.167:3300/0 conn(0x7f903406a6a0 msgr2=0x7f90340c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.423+0000 7f90337fe700 1 --2- 172.21.15.33:0/278706851 >> v2:172.21.15.167:3300/0 conn(0x7f903406a6a0 0x7f90340c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.424+0000 7f90337fe700 1 -- 172.21.15.33:0/278706851 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903406b120 con 0x7f9034155af0 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.424+0000 7f90337fe700 1 --2- 172.21.15.33:0/278706851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034155af0 0x7f903406a160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f903406f250 tx=0x7f9024009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db6e0028b7f5e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.424+0000 7f90327fc700 1 -- 172.21.15.33:0/278706851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902400b640 con 0x7f9034155af0 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.424+0000 7f90327fc700 1 -- 172.21.15.33:0/278706851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902400b7a0 con 0x7f9034155af0 2023-10-06T05:15:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.424+0000 7f90327fc700 1 -- 172.21.15.33:0/278706851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902400bb60 con 0x7f9034155af0 2023-10-06T05:15:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 -- 172.21.15.33:0/278706851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034155af0 msgr2=0x7f903406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 --2- 172.21.15.33:0/278706851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034155af0 0x7f903406a160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f903406f250 tx=0x7f9024009250 comp rx=0 tx=0).stop 2023-10-06T05:15:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 -- 172.21.15.33:0/278706851 shutdown_connections 2023-10-06T05:15:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 --2- 172.21.15.33:0/278706851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90340c8b50 0x7f90340c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 --2- 172.21.15.33:0/278706851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9034155af0 0x7f903406a160 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 --2- 172.21.15.33:0/278706851 >> v2:172.21.15.167:3300/0 conn(0x7f903406a6a0 0x7f90340c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 -- 172.21.15.33:0/278706851 >> 172.21.15.33:0/278706851 conn(0x7f903406d030 msgr2=0x7f903406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 -- 172.21.15.33:0/278706851 shutdown_connections 2023-10-06T05:15:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.425+0000 7f903899d700 1 -- 172.21.15.33:0/278706851 wait complete. 2023-10-06T05:15:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.426+0000 7f903899d700 1 Processor -- start 2023-10-06T05:15:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.426+0000 7f903899d700 1 -- start start 2023-10-06T05:15:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f903899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f903406a6a0 0x7f90341520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f903899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 0x7f903414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f903899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9034155af0 0x7f903414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f903899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90340b1350 con 0x7f90340c8b50 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f903899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90340b15f0 con 0x7f9034155af0 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f903899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90340b18d0 con 0x7f903406a6a0 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f9032ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 0x7f903414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f9032ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 0x7f903414a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53928/0 (socket says 172.21.15.33:53928) 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.427+0000 7f9032ffd700 1 -- 172.21.15.33:0/1121733448 learned_addr learned my addr 172.21.15.33:0/1121733448 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9033fff700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9034155af0 0x7f903414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f90337fe700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f903406a6a0 0x7f90341520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9032ffd700 1 -- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f903406a6a0 msgr2=0x7f90341520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9032ffd700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f903406a6a0 0x7f90341520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9032ffd700 1 -- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9034155af0 msgr2=0x7f903414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9032ffd700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9034155af0 0x7f903414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9032ffd700 1 -- 172.21.15.33:0/1121733448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90240094f0 con 0x7f90340c8b50 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f90337fe700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f903406a6a0 0x7f90341520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f9032ffd700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 0x7f903414a9b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f902800e900 tx=0x7f90280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028009510 con 0x7f90340c8b50 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902800c850 con 0x7f90340c8b50 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90340b1bb0 con 0x7f90340c8b50 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.428+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902800cbf0 con 0x7f90340c8b50 2023-10-06T05:15:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.429+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9034155340 con 0x7f90340c8b50 2023-10-06T05:15:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.429+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903404c380 con 0x7f90340c8b50 2023-10-06T05:15:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.430+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f902800cd50 con 0x7f90340c8b50 2023-10-06T05:15:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.431+0000 7f901b7fe700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9020066f60 0x7f9020069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.431+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9028084830 con 0x7f90340c8b50 2023-10-06T05:15:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.432+0000 7f90337fe700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9020066f60 0x7f9020069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.433+0000 7f90337fe700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9020066f60 0x7f9020069410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f903406f250 tx=0x7f9024017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.435+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90280518a0 con 0x7f90340c8b50 2023-10-06T05:15:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.683+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90340b1d40 con 0x7f90340c8b50 2023-10-06T05:15:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.684+0000 7f901b7fe700 1 -- 172.21.15.33:0/1121733448 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f9028054ec0 con 0x7f90340c8b50 2023-10-06T05:15:30.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:30.687 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:24.301322+0000","modified":"2023-10-06T05:15:27.844263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6836/1338636713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1338636713},{"type":"v1","addr":"172.21.15.167:6836","nonce":1338636713}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:15:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9020066f60 msgr2=0x7f9020069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9020066f60 0x7f9020069410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f903406f250 tx=0x7f9024017000 comp rx=0 tx=0).stop 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 msgr2=0x7f903414a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 0x7f903414a9b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f902800e900 tx=0x7f90280091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 shutdown_connections 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9020066f60 0x7f9020069410 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9034155af0 0x7f903414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f903406a6a0 0x7f90341520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 --2- 172.21.15.33:0/1121733448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90340c8b50 0x7f903414a9b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 >> 172.21.15.33:0/1121733448 conn(0x7f903406d030 msgr2=0x7f90340cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 shutdown_connections 2023-10-06T05:15:30.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:30.687+0000 7f903899d700 1 -- 172.21.15.33:0/1121733448 wait complete. 2023-10-06T05:15:30.691 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-10-06T05:15:31.110 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:15:24.301322+0000', 'modified': '2023-10-06T05:15:27.844263+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5028}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5028': {'gid': 5028, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.167:6836/1338636713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1338636713}, {'type': 'v1', 'addr': '172.21.15.167:6836', 'nonce': 1338636713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:15:31.110 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:15:31.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:15:31.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.298+0000 7fe80d79d700 1 Processor -- start 2023-10-06T05:15:31.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.298+0000 7fe80d79d700 1 -- start start 2023-10-06T05:15:31.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.299+0000 7fe80d79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe8080c93e0 0x7fe8080699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe808069f00 0x7fe8080c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80d79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80806a360 con 0x7fe808140d20 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80815bcb0 con 0x7fe808069f00 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80d79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8081410f0 con 0x7fe8080c93e0 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe807fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe808140d20 0x7fe8080c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34566/0 (socket says 172.21.15.33:34566) 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe807fff700 1 -- 172.21.15.33:0/2271419194 learned_addr learned my addr 172.21.15.33:0/2271419194 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/2271419194 >> v2:172.21.15.33:3300/0 conn(0x7fe808069f00 0x7fe8080c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe8077fe700 1 --2- 172.21.15.33:0/2271419194 >> v2:172.21.15.167:3300/0 conn(0x7fe8080c93e0 0x7fe8080699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe806ffd700 1 -- 172.21.15.33:0/2271419194 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476736440 0 0) 0x7fe7f8001110 con 0x7fe808140d20 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80cf9c700 1 -- 172.21.15.33:0/2271419194 >> v2:172.21.15.167:3300/0 conn(0x7fe8080c93e0 msgr2=0x7fe8080699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/2271419194 >> v2:172.21.15.167:3300/0 conn(0x7fe8080c93e0 0x7fe8080699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80cf9c700 1 -- 172.21.15.33:0/2271419194 >> v1:172.21.15.33:6789/0 conn(0x7fe808140d20 legacy=0x7fe8080c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.300+0000 7fe80cf9c700 1 -- 172.21.15.33:0/2271419194 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80815b0a0 con 0x7fe808069f00 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/2271419194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808069f00 0x7fe8080c6850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe8080cbf80 tx=0x7fe800009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29e5a3729f403e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe806ffd700 1 -- 172.21.15.33:0/2271419194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000e070 con 0x7fe808069f00 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe806ffd700 1 -- 172.21.15.33:0/2271419194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80000bc30 con 0x7fe808069f00 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe806ffd700 1 -- 172.21.15.33:0/2271419194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800012430 con 0x7fe808069f00 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80d79d700 1 -- 172.21.15.33:0/2271419194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808069f00 msgr2=0x7fe8080c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80d79d700 1 --2- 172.21.15.33:0/2271419194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808069f00 0x7fe8080c6850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe8080cbf80 tx=0x7fe800009780 comp rx=0 tx=0).stop 2023-10-06T05:15:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80d79d700 1 -- 172.21.15.33:0/2271419194 shutdown_connections 2023-10-06T05:15:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80d79d700 1 --2- 172.21.15.33:0/2271419194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808069f00 0x7fe8080c6850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80d79d700 1 --2- 172.21.15.33:0/2271419194 >> v2:172.21.15.167:3300/0 conn(0x7fe8080c93e0 0x7fe8080699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.301+0000 7fe80d79d700 1 -- 172.21.15.33:0/2271419194 >> 172.21.15.33:0/2271419194 conn(0x7fe80806c560 msgr2=0x7fe80806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.302+0000 7fe80d79d700 1 -- 172.21.15.33:0/2271419194 shutdown_connections 2023-10-06T05:15:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.302+0000 7fe80d79d700 1 -- 172.21.15.33:0/2271419194 wait complete. 2023-10-06T05:15:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.303+0000 7fe80d79d700 1 Processor -- start 2023-10-06T05:15:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.303+0000 7fe80d79d700 1 -- start start 2023-10-06T05:15:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe808069f00 0x7fe80813fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8080c93e0 0x7fe8080cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 0x7fe8080c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c64d0 con 0x7fe808140d20 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c41b0 con 0x7fe808069f00 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c35a0 con 0x7fe8080c93e0 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 0x7fe8080c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 0x7fe8080c1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53960/0 (socket says 172.21.15.33:53960) 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3574880993 learned_addr learned my addr 172.21.15.33:0/3574880993 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.304+0000 7fe807fff700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe808069f00 0x7fe80813fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8080c93e0 0x7fe8080cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8080c93e0 msgr2=0x7fe8080cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8080c93e0 0x7fe8080cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe808069f00 msgr2=0x7fe80813fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe808069f00 0x7fe80813fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3574880993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009a20 con 0x7fe808140d20 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8080c93e0 0x7fe8080cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 0x7fe8080c1b00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe800013000 tx=0x7fe800019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe7effff700 1 -- 172.21.15.33:0/3574880993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800027470 con 0x7fe808140d20 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe7effff700 1 -- 172.21.15.33:0/3574880993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8000275d0 con 0x7fe808140d20 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe7effff700 1 -- 172.21.15.33:0/3574880993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800027990 con 0x7fe808140d20 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.305+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8080c3880 con 0x7fe808140d20 2023-10-06T05:15:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.306+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8080c3e40 con 0x7fe808140d20 2023-10-06T05:15:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.307+0000 7fe7effff700 1 -- 172.21.15.33:0/3574880993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe800024070 con 0x7fe808140d20 2023-10-06T05:15:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.308+0000 7fe7effff700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7f4066f30 0x7fe7f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.308+0000 7fe7effff700 1 -- 172.21.15.33:0/3574880993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe800086e60 con 0x7fe808140d20 2023-10-06T05:15:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.309+0000 7fe807fff700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7f4066f30 0x7fe7f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.309+0000 7fe807fff700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7f4066f30 0x7fe7f40693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8007160 tx=0x7fe7f8007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.310+0000 7fe81d078b80 1 Processor -- start 2023-10-06T05:15:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.310+0000 7fe81d078b80 1 -- start start 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.311+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce1dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.311+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce46920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.311+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce19990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.311+0000 7fe81d078b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6cd9dc80 con 0x55be6ce67bc0 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.311+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6cd74a70 con 0x55be6ce1d950 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.311+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6cd72270 con 0x55be6ce46e60 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce46920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce46920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53970/0 (socket says 172.21.15.33:53970) 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 -- 172.21.15.33:0/3110878116 learned_addr learned my addr 172.21.15.33:0/3110878116 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe807fff700 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce1dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce19990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 -- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 msgr2=0x55be6ce19990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce19990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 -- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 msgr2=0x55be6ce1dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce1dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 -- 172.21.15.33:0/3110878116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6cc08270 con 0x55be6ce67bc0 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce19990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce46920 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe80806f890 tx=0x7fe7fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e41dfe4d49f44a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.312+0000 7fe7ed7fa700 1 -- 172.21.15.33:0/3110878116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00f6d0 con 0x55be6ce67bc0 2023-10-06T05:15:31.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe7ed7fa700 1 -- 172.21.15.33:0/3110878116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc00f830 con 0x55be6ce67bc0 2023-10-06T05:15:31.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe7ed7fa700 1 -- 172.21.15.33:0/3110878116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00fbf0 con 0x55be6ce67bc0 2023-10-06T05:15:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 -- 172.21.15.33:0/3110878116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 msgr2=0x55be6ce46920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce46920 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe80806f890 tx=0x7fe7fc008040 comp rx=0 tx=0).stop 2023-10-06T05:15:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 -- 172.21.15.33:0/3110878116 shutdown_connections 2023-10-06T05:15:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce19990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce1dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3110878116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce46920 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 -- 172.21.15.33:0/3110878116 >> 172.21.15.33:0/3110878116 conn(0x55be6ce1f270 msgr2=0x55be6ce1f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 -- 172.21.15.33:0/3110878116 shutdown_connections 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.313+0000 7fe81d078b80 1 -- 172.21.15.33:0/3110878116 wait complete. 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.314+0000 7fe81d078b80 1 -- start start 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.314+0000 7fe81d078b80 1 Processor -- start 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6cd6e060 con 0x55be6ce67bc0 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6cd19890 con 0x55be6ce1d950 2023-10-06T05:15:31.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6cd34ea0 con 0x55be6ce46e60 2023-10-06T05:15:31.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe81d078b80 10 client.0 _ll_register_callbacks cb 0x55be6cc0af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:31.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe80cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe80cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce5bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53972/0 (socket says 172.21.15.33:53972) 2023-10-06T05:15:31.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.315+0000 7fe80cf9c700 1 -- 172.21.15.33:0/1751440673 learned_addr learned my addr 172.21.15.33:0/1751440673 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe8077fe700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe807fff700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe80cf9c700 1 -- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 msgr2=0x55be6ce64fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce64fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe80cf9c700 1 -- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 msgr2=0x55be6ce64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce64aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe80cf9c700 1 -- 172.21.15.33:0/1751440673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc00e040 con 0x55be6ce67bc0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce5bf40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe800012fd0 tx=0x7fe800020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657f2eb56bac116 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe7e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe7e67fc700 1 -- 172.21.15.33:0/1751440673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000e040 con 0x55be6ce67bc0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe7e67fc700 1 -- 172.21.15.33:0/1751440673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe800057b90 con 0x55be6ce67bc0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.316+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be6ce12690 con 0x55be6ce67bc0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe7e67fc700 1 -- 172.21.15.33:0/1751440673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe80005ade0 con 0x55be6ce67bc0 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 10 client.5205 fetch_fsmap learned FSMap version 17 2023-10-06T05:15:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55be6cbaafe0 con 0x55be6ce67bc0 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe7e67fc700 1 -- 172.21.15.33:0/1751440673 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6ce12690 con 0x55be6ce67bc0 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe7e67fc700 1 -- 172.21.15.33:0/1751440673 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe80005e6a0 con 0x55be6ce67bc0 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 10 client.5205 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 10 client.5205 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 20 client.5205 populate_metadata read hostname 'smithi033' 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce452a0 0x55be6ce45670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 4 client.5205 mds_command: new command op to 5028 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 --> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be6cdef5c0 con 0x55be6ce452a0 2023-10-06T05:15:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.317+0000 7fe807fff700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce452a0 0x55be6ce45670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.318+0000 7fe807fff700 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce452a0 0x55be6ce45670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab58383bac70ace server_cookie=d7407a1637203b4a in_seq=0 out_seq=0 2023-10-06T05:15:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.318+0000 7fe7e67fc700 10 client.5205 ms_handle_connect on v2:172.21.15.167:6834/1338636713 2023-10-06T05:15:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.319+0000 7fe7e67fc700 1 -- 172.21.15.33:0/1751440673 <== mds.0 v2:172.21.15.167:6834/1338636713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55be6cdef5c0 con 0x55be6ce452a0 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.319+0000 7fe7e67fc700 10 client.5205 handle_command_reply: tid=0 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.319+0000 7fe81d078b80 1 client.5205 shutdown 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.319+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 msgr2=0x55be6ce5bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.319+0000 7fe81d078b80 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce5bf40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe800012fd0 tx=0x7fe800020040 comp rx=0 tx=0).stop 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 shutdown_connections 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce452a0 0x55be6ce45670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce67bc0 0x55be6ce5bf40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ce64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 --2- 172.21.15.33:0/1751440673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce46e60 0x55be6ce64fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe7e67fc700 0 client.5205 ms_handle_reset on v2:172.21.15.167:6834/1338636713 2023-10-06T05:15:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 >> 172.21.15.33:0/1751440673 conn(0x55be6ce1f270 msgr2=0x55be6ce141a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 shutdown_connections 2023-10-06T05:15:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 1 -- 172.21.15.33:0/1751440673 wait complete. 2023-10-06T05:15:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.320+0000 7fe81d078b80 20 client.5205 trim_cache size 0 max 16384 2023-10-06T05:15:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.337+0000 7fe81d078b80 1 Processor -- start 2023-10-06T05:15:31.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.337+0000 7fe81d078b80 1 -- start start 2023-10-06T05:15:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.338+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 0x55be6ce5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.338+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 0x55be6ce141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.338+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce46e60 0x55be6ce146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.338+0000 7fe81d078b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6cbf0e30 con 0x55be6ce1d950 2023-10-06T05:15:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.338+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6ce4d8d0 con 0x55be6ce46e60 2023-10-06T05:15:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.338+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6ce5cd90 con 0x55be6ce1f270 2023-10-06T05:15:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 0x55be6ce141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 0x55be6ce141a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53974/0 (socket says 172.21.15.33:53974) 2023-10-06T05:15:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 -- 172.21.15.33:0/3781364095 learned_addr learned my addr 172.21.15.33:0/3781364095 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 -- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 msgr2=0x55be6ce5bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 0x55be6ce5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 -- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce46e60 msgr2=0x55be6ce146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce46e60 0x55be6ce146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce46e60 0x55be6ce146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 -- 172.21.15.33:0/3781364095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6cd3c780 con 0x55be6ce1d950 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce46e60 0x55be6ce146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.339+0000 7fe8077fe700 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 0x55be6ce141a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc002000 tx=0x7fe7fc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc8a48dbe38107a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe7edffb700 1 -- 172.21.15.33:0/3781364095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc025070 con 0x55be6ce1d950 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe7edffb700 1 -- 172.21.15.33:0/3781364095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc000bc0 con 0x55be6ce1d950 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe7edffb700 1 -- 172.21.15.33:0/3781364095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00f6d0 con 0x55be6ce1d950 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 -- 172.21.15.33:0/3781364095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 msgr2=0x55be6ce141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 0x55be6ce141a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc002000 tx=0x7fe7fc01b000 comp rx=0 tx=0).stop 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 -- 172.21.15.33:0/3781364095 shutdown_connections 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce46e60 0x55be6ce146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 0x55be6ce5bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3781364095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce1d950 0x55be6ce141a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 -- 172.21.15.33:0/3781364095 >> 172.21.15.33:0/3781364095 conn(0x55be6ce1c9b0 msgr2=0x55be6ce63640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 -- 172.21.15.33:0/3781364095 shutdown_connections 2023-10-06T05:15:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.340+0000 7fe81d078b80 1 -- 172.21.15.33:0/3781364095 wait complete. 2023-10-06T05:15:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.341+0000 7fe81d078b80 1 -- start start 2023-10-06T05:15:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.341+0000 7fe81d078b80 1 Processor -- start 2023-10-06T05:15:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ceca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 0x55be6ceca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 0x55be6ce5f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6cecb0f0 con 0x55be6ce46e60 2023-10-06T05:15:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6ce3e360 con 0x55be6ce1d950 2023-10-06T05:15:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be6ce7dc40 con 0x55be6ce1f270 2023-10-06T05:15:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe81d078b80 10 client.0 _ll_register_callbacks cb 0x55be6cc0af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe80cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 0x55be6ce5f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe80cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 0x55be6ce5f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53976/0 (socket says 172.21.15.33:53976) 2023-10-06T05:15:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.342+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3331195637 learned_addr learned my addr 172.21.15.33:0/3331195637 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 msgr2=0x55be6ceca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 0x55be6ceca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe807fff700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ceca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 msgr2=0x55be6ceca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ceca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe80cf9c700 1 -- 172.21.15.33:0/3331195637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6cc08270 con 0x55be6ce46e60 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe807fff700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ceca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe80cf9c700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 0x55be6ce5f6f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe8000098f0 tx=0x7fe80006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7ae71013c75ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe7e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe7e67fc700 1 -- 172.21.15.33:0/3331195637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000e070 con 0x55be6ce46e60 2023-10-06T05:15:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be6ce1b180 con 0x55be6ce46e60 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.343+0000 7fe7e67fc700 1 -- 172.21.15.33:0/3331195637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe800057b90 con 0x55be6ce46e60 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe7e67fc700 1 -- 172.21.15.33:0/3331195637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe800057960 con 0x55be6ce46e60 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 10 client.5211 fetch_fsmap learned FSMap version 17 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55be6cd23470 con 0x55be6ce46e60 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe7e67fc700 1 -- 172.21.15.33:0/3331195637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be6ce1b180 con 0x55be6ce46e60 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe7e67fc700 1 -- 172.21.15.33:0/3331195637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe800058ba0 con 0x55be6ce46e60 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 10 client.5211 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T05:15:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 10 client.5211 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 20 client.5211 populate_metadata read hostname 'smithi033' 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce67bc0 0x55be6ce5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 4 client.5211 mds_command: new command op to 5028 tid=0[{"prefix": "status"}] 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 --> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fe7fc00f830 con 0x55be6ce67bc0 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.344+0000 7fe807fff700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce67bc0 0x55be6ce5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe807fff700 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce67bc0 0x55be6ce5fc30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1520858bc1c356 server_cookie=c3ffb7b902f22003 in_seq=0 out_seq=0 2023-10-06T05:15:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe7e67fc700 10 client.5211 ms_handle_connect on v2:172.21.15.167:6834/1338636713 2023-10-06T05:15:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe7e67fc700 1 -- 172.21.15.33:0/3331195637 <== mds.0 v2:172.21.15.167:6834/1338636713 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x7fe7fc00f830 con 0x55be6ce67bc0 2023-10-06T05:15:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe7e67fc700 10 client.5211 handle_command_reply: tid=0 2023-10-06T05:15:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 client.5211 shutdown 2023-10-06T05:15:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 msgr2=0x55be6ce5f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 0x55be6ce5f6f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe8000098f0 tx=0x7fe80006d000 comp rx=0 tx=0).stop 2023-10-06T05:15:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 shutdown_connections 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:6834/1338636713,v1:172.21.15.167:6836/1338636713] conn(0x55be6ce67bc0 0x55be6ce5fc30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be6ce46e60 0x55be6ce5f6f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be6ce1d950 0x55be6ceca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 --2- 172.21.15.33:0/3331195637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55be6ce1f270 0x55be6ceca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 >> 172.21.15.33:0/3331195637 conn(0x55be6ce1c9b0 msgr2=0x55be6ce45670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.345+0000 7fe7e67fc700 0 client.5211 ms_handle_reset on v2:172.21.15.167:6834/1338636713 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 shutdown_connections 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe81d078b80 1 -- 172.21.15.33:0/3331195637 wait complete. 2023-10-06T05:15:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe81d078b80 20 client.5211 trim_cache size 0 max 16384 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5028, 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.519252657, 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 17, 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 36, 2023-10-06T05:15:31.360 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 36, 2023-10-06T05:15:31.361 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 15.442380622 2023-10-06T05:15:31.361 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:15:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7f4066f30 msgr2=0x7fe7f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7f4066f30 0x7fe7f40693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8007160 tx=0x7fe7f8007f50 comp rx=0 tx=0).stop 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 msgr2=0x7fe8080c1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 0x7fe8080c1b00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe800013000 tx=0x7fe800019000 comp rx=0 tx=0).stop 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 shutdown_connections 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7f4066f30 0x7fe7f40693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe808140d20 0x7fe8080c1b00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe808069f00 0x7fe80813fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 --2- 172.21.15.33:0/3574880993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8080c93e0 0x7fe8080cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 >> 172.21.15.33:0/3574880993 conn(0x7fe80806c560 msgr2=0x7fe8080b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 shutdown_connections 2023-10-06T05:15:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.346+0000 7fe80d79d700 1 -- 172.21.15.33:0/3574880993 wait complete. 2023-10-06T05:15:31.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:31.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.553+0000 7f6eeaf9d700 1 Processor -- start 2023-10-06T05:15:31.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.553+0000 7f6eeaf9d700 1 -- start start 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eeaf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eeaf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ee40c7940 0x7f6ee40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eeaf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ee406aad0 0x7f6ee406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eeaf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40c9cb0 con 0x7f6ee40c7940 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eeaf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40ca270 con 0x7f6ee406aad0 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eeaf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee406b840 con 0x7f6ee413faa0 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6ee979a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ee40c7940 0x7f6ee40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6ee979a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ee40c7940 0x7f6ee40c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53978/0 (socket says 172.21.15.33:53978) 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6ee979a700 1 -- 172.21.15.33:0/973076869 learned_addr learned my addr 172.21.15.33:0/973076869 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6eea79c700 1 --2- 172.21.15.33:0/973076869 >> v2:172.21.15.167:3300/0 conn(0x7f6ee406aad0 0x7f6ee406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.555+0000 7f6ee9f9b700 1 --2- 172.21.15.33:0/973076869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee979a700 1 -- 172.21.15.33:0/973076869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 msgr2=0x7f6ee413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee979a700 1 --2- 172.21.15.33:0/973076869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee979a700 1 -- 172.21.15.33:0/973076869 >> v2:172.21.15.167:3300/0 conn(0x7f6ee406aad0 msgr2=0x7f6ee406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee979a700 1 --2- 172.21.15.33:0/973076869 >> v2:172.21.15.167:3300/0 conn(0x7f6ee406aad0 0x7f6ee406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee979a700 1 -- 172.21.15.33:0/973076869 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee406bb20 con 0x7f6ee40c7940 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6eea79c700 1 --2- 172.21.15.33:0/973076869 >> v2:172.21.15.167:3300/0 conn(0x7f6ee406aad0 0x7f6ee406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee979a700 1 --2- 172.21.15.33:0/973076869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c7db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40cc0b0 tx=0x7f6ed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebb78a230ef295f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee8f99700 1 -- 172.21.15.33:0/973076869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400b860 con 0x7f6ee40c7940 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee8f99700 1 -- 172.21.15.33:0/973076869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed400b9c0 con 0x7f6ee40c7940 2023-10-06T05:15:31.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.556+0000 7f6ee8f99700 1 -- 172.21.15.33:0/973076869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400bd60 con 0x7f6ee40c7940 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/973076869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee40c7940 msgr2=0x7f6ee40c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/973076869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c7db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40cc0b0 tx=0x7f6ed4009250 comp rx=0 tx=0).stop 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/973076869 shutdown_connections 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/973076869 >> v2:172.21.15.167:3300/0 conn(0x7f6ee406aad0 0x7f6ee406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/973076869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/973076869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c7db0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/973076869 >> 172.21.15.33:0/973076869 conn(0x7f6ee406d030 msgr2=0x7f6ee406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/973076869 shutdown_connections 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.557+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/973076869 wait complete. 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.558+0000 7f6eeaf9d700 1 Processor -- start 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.558+0000 7f6eeaf9d700 1 -- start start 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.559+0000 7f6eeaf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 0x7f6ee4155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.559+0000 7f6eeaf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.559+0000 7f6eeaf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee40c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.559+0000 7f6eeaf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee413efc0 con 0x7f6ee406aad0 2023-10-06T05:15:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.559+0000 7f6eeaf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40c4660 con 0x7f6ee40c7940 2023-10-06T05:15:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.559+0000 7f6eeaf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40c5490 con 0x7f6ee413faa0 2023-10-06T05:15:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 0x7f6ee4155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 0x7f6ee4155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53982/0 (socket says 172.21.15.33:53982) 2023-10-06T05:15:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 -- 172.21.15.33:0/1310112128 learned_addr learned my addr 172.21.15.33:0/1310112128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee979a700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6eea79c700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee40c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 -- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 msgr2=0x7f6ee40c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee40c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 -- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ee40c7940 msgr2=0x7f6ee40c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 -- 172.21.15.33:0/1310112128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed40094f0 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.560+0000 7f6ee9f9b700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 0x7f6ee4155d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000d860 tx=0x7f6ee0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.561+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00125f0 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.561+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee0012750 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.561+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0012b10 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.561+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee40c4880 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.561+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee40c4e40 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.562+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee404c380 con 0x7f6ee406aad0 2023-10-06T05:15:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.563+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ee001e460 con 0x7f6ee406aad0 2023-10-06T05:15:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.564+0000 7f6ed9ffb700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ec8066f30 0x7f6ec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.564+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ee0084bc0 con 0x7f6ee406aad0 2023-10-06T05:15:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.564+0000 7f6ee979a700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ec8066f30 0x7f6ec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.565+0000 7f6ee979a700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ec8066f30 0x7f6ec80693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40cc0b0 tx=0x7f6ed4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.568+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ee0051cd0 con 0x7f6ee406aad0 2023-10-06T05:15:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.815+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ee406f0b0 con 0x7f6ee406aad0 2023-10-06T05:15:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.816+0000 7f6ed9ffb700 1 -- 172.21.15.33:0/1310112128 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ee00552f0 con 0x7f6ee406aad0 2023-10-06T05:15:31.819 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:15:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ec8066f30 msgr2=0x7f6ec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ec8066f30 0x7f6ec80693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40cc0b0 tx=0x7f6ed4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:31.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 msgr2=0x7f6ee4155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 0x7f6ee4155d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000d860 tx=0x7f6ee0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 shutdown_connections 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ec8066f30 0x7f6ec80693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ee413faa0 0x7f6ee40c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ee406aad0 0x7f6ee4155d90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 --2- 172.21.15.33:0/1310112128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ee40c7940 0x7f6ee40c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.818+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 >> 172.21.15.33:0/1310112128 conn(0x7f6ee406d030 msgr2=0x7f6ee40c19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.819+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 shutdown_connections 2023-10-06T05:15:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:31.819+0000 7f6eeaf9d700 1 -- 172.21.15.33:0/1310112128 wait complete. 2023-10-06T05:15:31.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:15:31.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:31.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:15:31.840 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:15:31.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:31.841 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:15:31.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:15:31.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:31.841 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:15:31.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:31.841 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:15:31.882 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:31.882 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:31.882 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:31.882 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.882 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:31.883 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.883 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:31.883 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:31.883 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:15:31.883 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:15:31.884 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:15:31.885 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:15:31.885 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42225sec preferred_lft 42225sec 2023-10-06T05:15:31.885 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:15:31.885 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.886 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:31.886 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:31.886 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:15:31.886 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:31.887 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:31.887 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:15:31.887 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.887 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:15:31.887 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.888 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:15:31.888 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:15:31.888 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:15:31.888 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.888 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:15:31.888 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:15:31.889 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:15:31.889 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:31.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:31.889 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:15:31.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:31.985 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:15:32.023 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:15:32.023 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:32.023 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:15:32.024 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:15:32.024 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:32.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.025 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:15:32.025 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:15:32.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:15:32.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:15:32.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:15:32.025 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:15:32.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.225 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:15:32.261 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:15:32.261 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:15:32.262 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:15:32.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:15:32.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:32.364 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:32.400 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:15:32.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:32.400 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:32.476 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:15:32.477 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:32.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:15:32.609 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:15:32.609 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:15:32.609 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:15:32.625 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:15:32.625 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:15:32.626 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:15:32.626 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:15:32.626 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:15:32.648 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:15:32.652 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:15:32.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:32.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:32.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.904+0000 7f43d5f9d700 1 Processor -- start 2023-10-06T05:15:32.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.904+0000 7f43d5f9d700 1 -- start start 2023-10-06T05:15:32.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.905+0000 7f43d5f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f43d0159650 0x7f43d00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.905+0000 7f43d5f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00c92d0 con 0x7f43d00ca590 2023-10-06T05:15:32.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.905+0000 7f43d5f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00c86c0 con 0x7f43d0159a20 2023-10-06T05:15:32.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.905+0000 7f43d5f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d00c89a0 con 0x7f43d0159650 2023-10-06T05:15:32.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.906+0000 7f43d579c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43d00ca590 0x7f43d00a9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34570/0 (socket says 172.21.15.33:34570) 2023-10-06T05:15:32.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.906+0000 7f43d579c700 1 -- 172.21.15.33:0/1628926413 learned_addr learned my addr 172.21.15.33:0/1628926413 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:32.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.906+0000 7f43d4f9b700 1 --2- 172.21.15.33:0/1628926413 >> v2:172.21.15.167:3300/0 conn(0x7f43d0159650 0x7f43d00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.906+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680217157 0 0) 0x7f43d00c92d0 con 0x7f43d00ca590 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.906+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c0003570 con 0x7f43d00ca590 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.906+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300738901 0 0) 0x7f43d00c86c0 con 0x7f43d0159a20 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d00c92d0 con 0x7f43d0159a20 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1869553097 0 0) 0x7f43c0003570 con 0x7f43d00ca590 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 >> v2:172.21.15.167:3300/0 conn(0x7f43d0159650 msgr2=0x7f43d00c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 --2- 172.21.15.33:0/1628926413 >> v2:172.21.15.167:3300/0 conn(0x7f43d0159650 0x7f43d00c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 >> v1:172.21.15.167:6789/0 conn(0x7f43d0159a20 legacy=0x7f43d00c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00c8c80 con 0x7f43d00ca590 2023-10-06T05:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f43c80017f0 con 0x7f43d00ca590 2023-10-06T05:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 >> v1:172.21.15.33:6789/0 conn(0x7f43d00ca590 legacy=0x7f43d00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c0003bf0 0x7f43c0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c00051c0 0x7f43c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00c89a0 con 0x7f43c0004810 2023-10-06T05:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c000c160 con 0x7f43c0003bf0 2023-10-06T05:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c000c320 con 0x7f43c00051c0 2023-10-06T05:15:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43d4f9b700 1 --2- 172.21.15.33:0/1628926413 >> v2:172.21.15.167:3300/0 conn(0x7f43d0159650 0x7f43d00c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cffff700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43d579c700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c00051c0 0x7f43c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43d4f9b700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c0003bf0 0x7f43c0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cffff700 1 -- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c00051c0 msgr2=0x7f43c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cffff700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43c00051c0 0x7f43c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cffff700 1 -- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c0003bf0 msgr2=0x7f43c0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.907+0000 7f43cffff700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43c0003bf0 0x7f43c0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43cffff700 1 -- 172.21.15.33:0/1628926413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0003570 con 0x7f43c0004810 2023-10-06T05:15:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43d579c700 1 -- 172.21.15.33:0/1628926413 reap_dead start 2023-10-06T05:15:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43cffff700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43c0004c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f43d0159ea0 tx=0x7f43bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53913b357948da48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00c5d0 con 0x7f43c0004810 2023-10-06T05:15:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc00c730 con 0x7f43c0004810 2023-10-06T05:15:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43cf7fe700 1 -- 172.21.15.33:0/1628926413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00caf0 con 0x7f43c0004810 2023-10-06T05:15:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43d5f9d700 1 -- 172.21.15.33:0/1628926413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 msgr2=0x7f43c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.908+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43c0004c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f43d0159ea0 tx=0x7f43bc007d30 comp rx=0 tx=0).stop 2023-10-06T05:15:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.909+0000 7f43d5f9d700 1 -- 172.21.15.33:0/1628926413 shutdown_connections 2023-10-06T05:15:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.909+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/1628926413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43c0004c80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.909+0000 7f43d5f9d700 1 -- 172.21.15.33:0/1628926413 >> 172.21.15.33:0/1628926413 conn(0x7f43d006d030 msgr2=0x7f43d006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.909+0000 7f43d5f9d700 1 -- 172.21.15.33:0/1628926413 shutdown_connections 2023-10-06T05:15:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.909+0000 7f43d5f9d700 1 -- 172.21.15.33:0/1628926413 wait complete. 2023-10-06T05:15:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.910+0000 7f43d5f9d700 1 Processor -- start 2023-10-06T05:15:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.910+0000 7f43d5f9d700 1 -- start start 2023-10-06T05:15:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43d01530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d00c3e40 0x7f43d01585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d00c5700 0x7f43d00c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0158d90 con 0x7f43c0004810 2023-10-06T05:15:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00b1630 con 0x7f43d00c5700 2023-10-06T05:15:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0158af0 con 0x7f43d00c3e40 2023-10-06T05:15:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43d01530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43d01530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54002/0 (socket says 172.21.15.33:54002) 2023-10-06T05:15:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.911+0000 7f43d4f9b700 1 -- 172.21.15.33:0/652825531 learned_addr learned my addr 172.21.15.33:0/652825531 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43cffff700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d00c3e40 0x7f43d01585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d4f9b700 1 -- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d00c3e40 msgr2=0x7f43d01585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d579c700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d00c5700 0x7f43d00c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d4f9b700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d00c3e40 0x7f43d01585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d4f9b700 1 -- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d00c5700 msgr2=0x7f43d00c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d4f9b700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d00c5700 0x7f43d00c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d4f9b700 1 -- 172.21.15.33:0/652825531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00b0d00 con 0x7f43c0004810 2023-10-06T05:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43cffff700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d00c3e40 0x7f43d01585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d579c700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d00c5700 0x7f43d00c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d4f9b700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43d01530b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f43c4011da0 tx=0x7f43c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400fcd0 con 0x7f43c0004810 2023-10-06T05:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400fe30 con 0x7f43c0004810 2023-10-06T05:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4014690 con 0x7f43c0004810 2023-10-06T05:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d00b0fe0 con 0x7f43c0004810 2023-10-06T05:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.912+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d00c60e0 con 0x7f43c0004810 2023-10-06T05:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.914+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d004c380 con 0x7f43c0004810 2023-10-06T05:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.915+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43c4014a40 con 0x7f43c0004810 2023-10-06T05:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.915+0000 7f43ccff9700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b4066f30 0x7f43b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.915+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43c4018030 con 0x7f43c0004810 2023-10-06T05:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.916+0000 7f43cffff700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b4066f30 0x7f43b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.917+0000 7f43cffff700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b4066f30 0x7f43b40693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f43d00c6560 tx=0x7f43bc015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:32.919+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43c4052420 con 0x7f43c0004810 2023-10-06T05:15:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.168+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43d00c8f60 con 0x7f43c0004810 2023-10-06T05:15:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.169+0000 7f43ccff9700 1 -- 172.21.15.33:0/652825531 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f43c4055a40 con 0x7f43c0004810 2023-10-06T05:15:33.172 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b4066f30 msgr2=0x7f43b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b4066f30 0x7f43b40693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f43d00c6560 tx=0x7f43bc015000 comp rx=0 tx=0).stop 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 msgr2=0x7f43d01530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43d01530b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f43c4011da0 tx=0x7f43c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 shutdown_connections 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b4066f30 0x7f43b40693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d00c5700 0x7f43d00c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0004810 0x7f43d01530b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 --2- 172.21.15.33:0/652825531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d00c3e40 0x7f43d01585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 >> 172.21.15.33:0/652825531 conn(0x7f43d006d030 msgr2=0x7f43d0154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 shutdown_connections 2023-10-06T05:15:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:33.171+0000 7f43d5f9d700 1 -- 172.21.15.33:0/652825531 wait complete. 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:33.191 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:15:33.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.192 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.234 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:15:33.235 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42228sec preferred_lft 42228sec 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:15:33.236 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.237 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:15:33.237 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:15:33.237 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:15:33.237 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.237 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:33.336 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:15:33.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.339 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:15:33.377 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:15:33.378 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:33.378 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:15:33.378 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:15:33.378 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:33.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.379 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:15:33.379 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:15:33.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:15:33.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:15:33.380 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:15:33.380 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:15:33.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.575 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:15:33.611 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:15:33.612 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:15:33.612 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:15:33.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.612 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:15:33.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:33.714 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:33.750 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:15:33.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:33.750 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:33.826 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:15:33.826 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:33.826 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:15:33.957 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:15:33.958 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:15:33.958 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:15:33.974 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:15:33.974 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:15:33.974 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:15:33.974 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:15:33.974 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:15:33.995 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:15:33.995 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:15:33.995 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:15:33.995 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:15:33.995 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:15:33.996 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:15:33.996 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:15:33.996 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:15:33.999 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:15:33.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:34.000 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:34.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:15:34.277 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:15:34.278 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:15:34.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:15:34.482 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:15:34.483 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:15:34.483 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:15:34.483 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:34.562 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:34.643 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:15:34.644 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:15:34.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:34.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:34.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:15:34.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:34.724 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:15:34.762 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:15:34.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:34.762 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:34.848 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:15:34.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:34.848 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:15:34.887 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:35.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.071+0000 7f798f59e700 1 Processor -- start 2023-10-06T05:15:35.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.071+0000 7f798f59e700 1 -- start start 2023-10-06T05:15:35.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f799013faa0 0x7f799013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79900c7940 0x7f79900c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79900c9cb0 con 0x7f799006aad0 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799006b640 con 0x7f799013faa0 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900ca270 con 0x7f79900c7940 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f799013faa0 0x7f799013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798ed9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f799006aad0 0x7f799006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34582/0 (socket says 172.21.15.33:34582) 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f799013faa0 0x7f799013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54006/0 (socket says 172.21.15.33:54006) 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798ed9d700 1 -- 172.21.15.33:0/800840094 learned_addr learned my addr 172.21.15.33:0/800840094 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798dd9b700 1 --2- 172.21.15.33:0/800840094 >> v2:172.21.15.167:3300/0 conn(0x7f79900c7940 0x7f79900c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798e59c700 1 -- 172.21.15.33:0/800840094 >> v2:172.21.15.167:3300/0 conn(0x7f79900c7940 msgr2=0x7f79900c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798e59c700 1 --2- 172.21.15.33:0/800840094 >> v2:172.21.15.167:3300/0 conn(0x7f79900c7940 0x7f79900c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798e59c700 1 -- 172.21.15.33:0/800840094 >> v1:172.21.15.33:6789/0 conn(0x7f799006aad0 legacy=0x7f799006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798e59c700 1 -- 172.21.15.33:0/800840094 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799006b840 con 0x7f799013faa0 2023-10-06T05:15:35.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.073+0000 7f798d59a700 1 -- 172.21.15.33:0/800840094 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385897559 0 0) 0x7f79900c9cb0 con 0x7f799006aad0 2023-10-06T05:15:35.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.074+0000 7f798e59c700 1 --2- 172.21.15.33:0/800840094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f799013fe70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f79900cbef0 tx=0x7f798000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c79b14d00e9901 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.074+0000 7f798d59a700 1 -- 172.21.15.33:0/800840094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000db40 con 0x7f799013faa0 2023-10-06T05:15:35.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.074+0000 7f798d59a700 1 -- 172.21.15.33:0/800840094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798000dca0 con 0x7f799013faa0 2023-10-06T05:15:35.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.074+0000 7f798d59a700 1 -- 172.21.15.33:0/800840094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79800196b0 con 0x7f799013faa0 2023-10-06T05:15:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.074+0000 7f798f59e700 1 -- 172.21.15.33:0/800840094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 msgr2=0x7f799013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.074+0000 7f798f59e700 1 --2- 172.21.15.33:0/800840094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f799013fe70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f79900cbef0 tx=0x7f798000b430 comp rx=0 tx=0).stop 2023-10-06T05:15:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.075+0000 7f798f59e700 1 -- 172.21.15.33:0/800840094 shutdown_connections 2023-10-06T05:15:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.075+0000 7f798f59e700 1 --2- 172.21.15.33:0/800840094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f799013fe70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.075+0000 7f798f59e700 1 --2- 172.21.15.33:0/800840094 >> v2:172.21.15.167:3300/0 conn(0x7f79900c7940 0x7f79900c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.075+0000 7f798f59e700 1 -- 172.21.15.33:0/800840094 >> 172.21.15.33:0/800840094 conn(0x7f799006d030 msgr2=0x7f799006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.075+0000 7f798f59e700 1 -- 172.21.15.33:0/800840094 shutdown_connections 2023-10-06T05:15:35.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.075+0000 7f798f59e700 1 -- 172.21.15.33:0/800840094 wait complete. 2023-10-06T05:15:35.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.076+0000 7f798f59e700 1 Processor -- start 2023-10-06T05:15:35.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.076+0000 7f798f59e700 1 -- start start 2023-10-06T05:15:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f799006aad0 0x7f79900c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79900c7940 0x7f79900c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f79900c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900c5470 con 0x7f799013faa0 2023-10-06T05:15:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900c3bd0 con 0x7f799006aad0 2023-10-06T05:15:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79900c3e80 con 0x7f79900c7940 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.077+0000 7f798ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f79900c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f79900c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54020/0 (socket says 172.21.15.33:54020) 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 -- 172.21.15.33:0/4060849079 learned_addr learned my addr 172.21.15.33:0/4060849079 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798e59c700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f799006aad0 0x7f79900c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798dd9b700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79900c7940 0x7f79900c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 -- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79900c7940 msgr2=0x7f79900c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79900c7940 0x7f79900c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 -- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f799006aad0 msgr2=0x7f79900c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f799006aad0 0x7f79900c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 -- 172.21.15.33:0/4060849079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798000b6d0 con 0x7f799013faa0 2023-10-06T05:15:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798ed9d700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f79900c6af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f798800cc90 tx=0x7f7988001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800a8b0 con 0x7f799013faa0 2023-10-06T05:15:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798800aa10 con 0x7f799013faa0 2023-10-06T05:15:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79900c4160 con 0x7f799013faa0 2023-10-06T05:15:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.078+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800add0 con 0x7f799013faa0 2023-10-06T05:15:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.079+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79900c4a00 con 0x7f799013faa0 2023-10-06T05:15:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.080+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79900c4d70 con 0x7f799013faa0 2023-10-06T05:15:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.080+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7988007300 con 0x7f799013faa0 2023-10-06T05:15:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.082+0000 7f797e7fc700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7970066f30 0x7f79700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.082+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7988084a50 con 0x7f799013faa0 2023-10-06T05:15:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.082+0000 7f798e59c700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7970066f30 0x7f79700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.083+0000 7f798e59c700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7970066f30 0x7f79700693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f798000b6a0 tx=0x7f798001b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.086+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7988051b40 con 0x7f799013faa0 2023-10-06T05:15:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.333+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79900c4580 con 0x7f799013faa0 2023-10-06T05:15:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.334+0000 7f797e7fc700 1 -- 172.21.15.33:0/4060849079 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f79900c4580 con 0x7f799013faa0 2023-10-06T05:15:35.337 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:35.337 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:24.301322+0000","modified":"2023-10-06T05:15:27.844263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6836/1338636713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1338636713},{"type":"v1","addr":"172.21.15.167:6836","nonce":1338636713}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:15:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.336+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7970066f30 msgr2=0x7f79700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.336+0000 7f798f59e700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7970066f30 0x7f79700693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f798000b6a0 tx=0x7f798001b000 comp rx=0 tx=0).stop 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 msgr2=0x7f79900c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f79900c6af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f798800cc90 tx=0x7f7988001a80 comp rx=0 tx=0).stop 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 shutdown_connections 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7970066f30 0x7f79700693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f799013faa0 0x7f79900c6af0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f799006aad0 0x7f79900c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 --2- 172.21.15.33:0/4060849079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79900c7940 0x7f79900c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 >> 172.21.15.33:0/4060849079 conn(0x7f799006d030 msgr2=0x7f79900c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 shutdown_connections 2023-10-06T05:15:35.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.337+0000 7f798f59e700 1 -- 172.21.15.33:0/4060849079 wait complete. 2023-10-06T05:15:35.341 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-10-06T05:15:35.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.540+0000 7f035199d700 1 Processor -- start 2023-10-06T05:15:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.540+0000 7f035199d700 1 -- start start 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035199d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f034c159530 0x7f034c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c06df70 0x7f034c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035199d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c06e3e0 con 0x7f034c0b0310 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035199d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c0b1e00 con 0x7f034c159530 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c06f6b0 con 0x7f034c06df70 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f034c159530 0x7f034c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f034c159530 0x7f034c06da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54024/0 (socket says 172.21.15.33:54024) 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 -- 172.21.15.33:0/1175614435 learned_addr learned my addr 172.21.15.33:0/1175614435 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 -- 172.21.15.33:0/1175614435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c06df70 msgr2=0x7f034c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 --2- 172.21.15.33:0/1175614435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c06df70 0x7f034c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 -- 172.21.15.33:0/1175614435 >> v1:172.21.15.167:6789/0 conn(0x7f034c0b0310 legacy=0x7f034c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.542+0000 7f035099b700 1 -- 172.21.15.33:0/1175614435 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c06f220 con 0x7f034c159530 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.543+0000 7f035099b700 1 --2- 172.21.15.33:0/1175614435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c06da30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f03400099b0 tx=0x7f0340009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c97d44cb5888d4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.543+0000 7f034b7fe700 1 -- 172.21.15.33:0/1175614435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034000b820 con 0x7f034c159530 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.543+0000 7f034b7fe700 1 -- 172.21.15.33:0/1175614435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034000b980 con 0x7f034c159530 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.543+0000 7f034b7fe700 1 -- 172.21.15.33:0/1175614435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034000bd20 con 0x7f034c159530 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.543+0000 7f035199d700 1 -- 172.21.15.33:0/1175614435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 msgr2=0x7f034c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.543+0000 7f035199d700 1 --2- 172.21.15.33:0/1175614435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c06da30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f03400099b0 tx=0x7f0340009250 comp rx=0 tx=0).stop 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.544+0000 7f035199d700 1 -- 172.21.15.33:0/1175614435 shutdown_connections 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.544+0000 7f035199d700 1 --2- 172.21.15.33:0/1175614435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c06da30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.544+0000 7f035199d700 1 --2- 172.21.15.33:0/1175614435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c06df70 0x7f034c0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.544+0000 7f035199d700 1 -- 172.21.15.33:0/1175614435 >> 172.21.15.33:0/1175614435 conn(0x7f034c06c560 msgr2=0x7f034c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.544+0000 7f035199d700 1 -- 172.21.15.33:0/1175614435 shutdown_connections 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.544+0000 7f035199d700 1 -- 172.21.15.33:0/1175614435 wait complete. 2023-10-06T05:15:35.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.545+0000 7f035199d700 1 Processor -- start 2023-10-06T05:15:35.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.545+0000 7f035199d700 1 -- start start 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 0x7f034c1409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c0b0310 0x7f034c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c16b010 con 0x7f034c159530 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0ce830 con 0x7f034c06df70 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0ceb10 con 0x7f034c0b0310 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.546+0000 7f035119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035099b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 0x7f034c1409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f034bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c0b0310 0x7f034c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035099b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 0x7f034c1409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:52740/0 (socket says 172.21.15.33:52740) 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54036/0 (socket says 172.21.15.33:54036) 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035099b700 1 -- 172.21.15.33:0/766291832 learned_addr learned my addr 172.21.15.33:0/766291832 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 -- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c0b0310 msgr2=0x7f034c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c0b0310 0x7f034c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 -- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 msgr2=0x7f034c1409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 0x7f034c1409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 -- 172.21.15.33:0/766291832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400094f0 con 0x7f034c159530 2023-10-06T05:15:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035099b700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 0x7f034c1409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.547+0000 7f035119c700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c0c76b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f034400bb00 tx=0x7f0344009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.548+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034400c7c0 con 0x7f034c159530 2023-10-06T05:15:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.548+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034400c920 con 0x7f034c159530 2023-10-06T05:15:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.548+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034400cce0 con 0x7f034c159530 2023-10-06T05:15:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.548+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c0cedf0 con 0x7f034c159530 2023-10-06T05:15:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.548+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c140f30 con 0x7f034c159530 2023-10-06T05:15:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.549+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c16b1c0 con 0x7f034c159530 2023-10-06T05:15:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.549+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0344014620 con 0x7f034c159530 2023-10-06T05:15:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.550+0000 7f0348ff9700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.550+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0344084520 con 0x7f034c159530 2023-10-06T05:15:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.551+0000 7f035099b700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.551+0000 7f035099b700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0340003080 tx=0x7f0340016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.554+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f034404c5a0 con 0x7f034c159530 2023-10-06T05:15:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.760+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f034c04c380 con 0x7f034c159530 2023-10-06T05:15:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.761+0000 7f0348ff9700 1 -- 172.21.15.33:0/766291832 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f034c16b1c0 con 0x7f034c159530 2023-10-06T05:15:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:35.765 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:25.826634+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:22.479742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:23.276900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.763+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 msgr2=0x7f03300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.763+0000 7f035199d700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0340003080 tx=0x7f0340016000 comp rx=0 tx=0).stop 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 msgr2=0x7f034c0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c0c76b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f034400bb00 tx=0x7f0344009780 comp rx=0 tx=0).stop 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 shutdown_connections 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f034c159530 0x7f034c0c76b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f034c06df70 0x7f034c1409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 --2- 172.21.15.33:0/766291832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f034c0b0310 0x7f034c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 >> 172.21.15.33:0/766291832 conn(0x7f034c06c560 msgr2=0x7f034c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 shutdown_connections 2023-10-06T05:15:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.764+0000 7f035199d700 1 -- 172.21.15.33:0/766291832 wait complete. 2023-10-06T05:15:35.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:15:22.479742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:15:23.276900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:35.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:35.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.972+0000 7f2ee8f9d700 1 Processor -- start 2023-10-06T05:15:35.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.973+0000 7f2ee8f9d700 1 -- start start 2023-10-06T05:15:35.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee8f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2ee406d140 0x7f2ee406edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee8f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40b0cd0 con 0x7f2ee415acd0 2023-10-06T05:15:35.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee8f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee415a840 con 0x7f2ee406d140 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4159f10 con 0x7f2ee4159850 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:53630/0 (socket says 172.21.15.33:53630) 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee37fe700 1 -- 172.21.15.33:0/3789991964 learned_addr learned my addr 172.21.15.33:0/3789991964 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.974+0000 7f2ee3fff700 1 --2- 172.21.15.33:0/3789991964 >> v2:172.21.15.167:3300/0 conn(0x7f2ee406d140 0x7f2ee406edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee37fe700 1 -- 172.21.15.33:0/3789991964 >> v2:172.21.15.167:3300/0 conn(0x7f2ee406d140 msgr2=0x7f2ee406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee37fe700 1 --2- 172.21.15.33:0/3789991964 >> v2:172.21.15.167:3300/0 conn(0x7f2ee406d140 0x7f2ee406edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee37fe700 1 -- 172.21.15.33:0/3789991964 >> v1:172.21.15.167:6789/0 conn(0x7f2ee415acd0 legacy=0x7f2ee406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee37fe700 1 -- 172.21.15.33:0/3789991964 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee415a1f0 con 0x7f2ee4159850 2023-10-06T05:15:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee37fe700 1 --2- 172.21.15.33:0/3789991964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee406e890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40099b0 tx=0x7f2ed4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67dd958a6994cc52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee27fc700 1 -- 172.21.15.33:0/3789991964 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4009ea0 con 0x7f2ee4159850 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee27fc700 1 -- 172.21.15.33:0/3789991964 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed400b860 con 0x7f2ee4159850 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.975+0000 7f2ee27fc700 1 -- 172.21.15.33:0/3789991964 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400bc00 con 0x7f2ee4159850 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/3789991964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 msgr2=0x7f2ee406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/3789991964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee406e890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40099b0 tx=0x7f2ed4009250 comp rx=0 tx=0).stop 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/3789991964 shutdown_connections 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/3789991964 >> v2:172.21.15.167:3300/0 conn(0x7f2ee406d140 0x7f2ee406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/3789991964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee406e890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/3789991964 >> 172.21.15.33:0/3789991964 conn(0x7f2ee406cd40 msgr2=0x7f2ee40ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/3789991964 shutdown_connections 2023-10-06T05:15:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.976+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/3789991964 wait complete. 2023-10-06T05:15:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.977+0000 7f2ee8f9d700 1 Processor -- start 2023-10-06T05:15:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.978+0000 7f2ee8f9d700 1 -- start start 2023-10-06T05:15:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.978+0000 7f2ee8f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 0x7f2ee40cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee4143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee8f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee415acd0 0x7f2ee41582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee8f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cd1a0 con 0x7f2ee406d140 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee8f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b12a0 con 0x7f2ee415acd0 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cd390 con 0x7f2ee4159850 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 0x7f2ee40cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 0x7f2ee40cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54040/0 (socket says 172.21.15.33:54040) 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 -- 172.21.15.33:0/174793687 learned_addr learned my addr 172.21.15.33:0/174793687 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee3fff700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee415acd0 0x7f2ee41582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee2ffd700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee4143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 -- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 msgr2=0x7f2ee4143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee4143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 -- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee415acd0 msgr2=0x7f2ee41582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee415acd0 0x7f2ee41582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee37fe700 1 -- 172.21.15.33:0/174793687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40094f0 con 0x7f2ee406d140 2023-10-06T05:15:35.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.979+0000 7f2ee3fff700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee415acd0 0x7f2ee41582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:35.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.980+0000 7f2ee37fe700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 0x7f2ee40cad10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4006fd0 tx=0x7f2ed4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.980+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400bbe0 con 0x7f2ee406d140 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.980+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4000c90 con 0x7f2ee406d140 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.980+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4011650 con 0x7f2ee406d140 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.980+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee40cb250 con 0x7f2ee406d140 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.980+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee40cd910 con 0x7f2ee406d140 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.981+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee404c380 con 0x7f2ee406d140 2023-10-06T05:15:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.982+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ed4011a40 con 0x7f2ee406d140 2023-10-06T05:15:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.983+0000 7f2ecb7fe700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.983+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ed4085b50 con 0x7f2ee406d140 2023-10-06T05:15:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.983+0000 7f2ee2ffd700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.984+0000 7f2ee2ffd700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ed0066f30 0x7f2ed00693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b0330 tx=0x7f2ed8007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:35.987+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ed4052c40 con 0x7f2ee406d140 2023-10-06T05:15:36.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.188+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ee40ad830 con 0x7f2ee406d140 2023-10-06T05:15:36.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.189+0000 7f2ecb7fe700 1 -- 172.21.15.33:0/174793687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ed40150a0 con 0x7f2ee406d140 2023-10-06T05:15:36.192 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ed0066f30 msgr2=0x7f2ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ed0066f30 0x7f2ed00693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b0330 tx=0x7f2ed8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 msgr2=0x7f2ee40cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 0x7f2ee40cad10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4006fd0 tx=0x7f2ed4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 shutdown_connections 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ee415acd0 0x7f2ee41582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ee406d140 0x7f2ee40cad10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 --2- 172.21.15.33:0/174793687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ee4159850 0x7f2ee4143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 >> 172.21.15.33:0/174793687 conn(0x7f2ee406cd40 msgr2=0x7f2ee40abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:36.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 shutdown_connections 2023-10-06T05:15:36.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.191+0000 7f2ee8f9d700 1 -- 172.21.15.33:0/174793687 wait complete. 2023-10-06T05:15:36.211 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:36.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.397+0000 7fe3beb9d700 1 Processor -- start 2023-10-06T05:15:36.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.397+0000 7fe3beb9d700 1 -- start start 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3beb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3b80aff90 0x7fe3b80a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3beb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b806cb80 0x7fe3b815d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3beb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b815d3a0 con 0x7fe3b806c7b0 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3beb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80686c0 con 0x7fe3b80aff90 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3beb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80b0470 con 0x7fe3b806cb80 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bd39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3b80aff90 0x7fe3b80a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bdb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe3b806c7b0 0x7fe3b8067fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34596/0 (socket says 172.21.15.33:34596) 2023-10-06T05:15:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bd39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3b80aff90 0x7fe3b80a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54048/0 (socket says 172.21.15.33:54048) 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bdb9b700 1 -- 172.21.15.33:0/1654917339 learned_addr learned my addr 172.21.15.33:0/1654917339 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3be39c700 1 --2- 172.21.15.33:0/1654917339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b806cb80 0x7fe3b815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bd39a700 1 -- 172.21.15.33:0/1654917339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b806cb80 msgr2=0x7fe3b815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bd39a700 1 --2- 172.21.15.33:0/1654917339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b806cb80 0x7fe3b815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bd39a700 1 -- 172.21.15.33:0/1654917339 >> v1:172.21.15.33:6789/0 conn(0x7fe3b806c7b0 legacy=0x7fe3b8067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bd39a700 1 -- 172.21.15.33:0/1654917339 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80b0750 con 0x7fe3b80aff90 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.399+0000 7fe3bcb99700 1 -- 172.21.15.33:0/1654917339 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926619719 0 0) 0x7fe3b815d3a0 con 0x7fe3b806c7b0 2023-10-06T05:15:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.400+0000 7fe3bd39a700 1 --2- 172.21.15.33:0/1654917339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b80aff90 0x7fe3b80a9890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80098e0 tx=0x7fe3a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb7f84ecf4333c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.400+0000 7fe3bcb99700 1 -- 172.21.15.33:0/1654917339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8009e80 con 0x7fe3b80aff90 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.400+0000 7fe3bcb99700 1 -- 172.21.15.33:0/1654917339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a800b860 con 0x7fe3b80aff90 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.400+0000 7fe3bcb99700 1 -- 172.21.15.33:0/1654917339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a800bbe0 con 0x7fe3b80aff90 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.400+0000 7fe3beb9d700 1 -- 172.21.15.33:0/1654917339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b80aff90 msgr2=0x7fe3b80a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.400+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/1654917339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b80aff90 0x7fe3b80a9890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80098e0 tx=0x7fe3a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.401+0000 7fe3beb9d700 1 -- 172.21.15.33:0/1654917339 shutdown_connections 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.401+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/1654917339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b806cb80 0x7fe3b815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.401+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/1654917339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b80aff90 0x7fe3b80a9890 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.401+0000 7fe3beb9d700 1 -- 172.21.15.33:0/1654917339 >> 172.21.15.33:0/1654917339 conn(0x7fe3b806d670 msgr2=0x7fe3b806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.401+0000 7fe3beb9d700 1 -- 172.21.15.33:0/1654917339 shutdown_connections 2023-10-06T05:15:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.401+0000 7fe3beb9d700 1 -- 172.21.15.33:0/1654917339 wait complete. 2023-10-06T05:15:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.402+0000 7fe3beb9d700 1 Processor -- start 2023-10-06T05:15:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.402+0000 7fe3beb9d700 1 -- start start 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3beb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b806c7b0 0x7fe3b815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3beb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 0x7fe3b8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3beb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80aff90 0x7fe3b8145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3beb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8169bd0 con 0x7fe3b806cb80 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3beb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b815c170 con 0x7fe3b806c7b0 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3beb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8169dc0 con 0x7fe3b80aff90 2023-10-06T05:15:36.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3bd39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 0x7fe3b8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3bd39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 0x7fe3b8157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54060/0 (socket says 172.21.15.33:54060) 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3bd39a700 1 -- 172.21.15.33:0/3052280804 learned_addr learned my addr 172.21.15.33:0/3052280804 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.403+0000 7fe3be39c700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80aff90 0x7fe3b8145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bdb9b700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b806c7b0 0x7fe3b815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bd39a700 1 -- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80aff90 msgr2=0x7fe3b8145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bd39a700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80aff90 0x7fe3b8145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bd39a700 1 -- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b806c7b0 msgr2=0x7fe3b815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bd39a700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b806c7b0 0x7fe3b815a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bd39a700 1 -- 172.21.15.33:0/3052280804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8009420 con 0x7fe3b806cb80 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3bd39a700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 0x7fe3b8157910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8013000 tx=0x7fe3a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8000d10 con 0x7fe3b806cb80 2023-10-06T05:15:36.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.404+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a800b9d0 con 0x7fe3b806cb80 2023-10-06T05:15:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.405+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8025670 con 0x7fe3b806cb80 2023-10-06T05:15:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.405+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b815b8d0 con 0x7fe3b806cb80 2023-10-06T05:15:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.405+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b815be90 con 0x7fe3b806cb80 2023-10-06T05:15:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.406+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b8169f80 con 0x7fe3b806cb80 2023-10-06T05:15:36.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.406+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3a8023070 con 0x7fe3b806cb80 2023-10-06T05:15:36.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.408+0000 7fe3adffb700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe39c066f30 0x7fe39c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.408+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe3a80859f0 con 0x7fe3b806cb80 2023-10-06T05:15:36.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.408+0000 7fe3bdb9b700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe39c066f30 0x7fe39c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.409+0000 7fe3bdb9b700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe39c066f30 0x7fe39c0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4001360 tx=0x7fe3b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:36.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.412+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3a8052b00 con 0x7fe3b806cb80 2023-10-06T05:15:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.614+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3b80b0a30 con 0x7fe3b806cb80 2023-10-06T05:15:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.614+0000 7fe3adffb700 1 -- 172.21.15.33:0/3052280804 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3a8015020 con 0x7fe3b806cb80 2023-10-06T05:15:36.617 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:15:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe39c066f30 msgr2=0x7fe39c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe39c066f30 0x7fe39c0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4001360 tx=0x7fe3b4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 msgr2=0x7fe3b8157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 0x7fe3b8157910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8013000 tx=0x7fe3a8017000 comp rx=0 tx=0).stop 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 shutdown_connections 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe39c066f30 0x7fe39c0693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b80aff90 0x7fe3b8145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b806c7b0 0x7fe3b815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 --2- 172.21.15.33:0/3052280804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b806cb80 0x7fe3b8157910 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 >> 172.21.15.33:0/3052280804 conn(0x7fe3b806d670 msgr2=0x7fe3b80ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 shutdown_connections 2023-10-06T05:15:36.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.617+0000 7fe3beb9d700 1 -- 172.21.15.33:0/3052280804 wait complete. 2023-10-06T05:15:36.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.824+0000 7f0b3719d700 1 Processor -- start 2023-10-06T05:15:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.824+0000 7f0b3719d700 1 -- start start 2023-10-06T05:15:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.825+0000 7f0b3719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b300af540 0x7f0b300af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.825+0000 7f0b3719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0b300cd5e0 0x7f0b3015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.825+0000 7f0b3719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3015d660 con 0x7f0b3015a650 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.825+0000 7f0b3719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b300ce890 con 0x7f0b300af540 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.825+0000 7f0b3719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3015aac0 con 0x7f0b300cd5e0 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b300af540 0x7f0b300af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b300af540 0x7f0b300af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54064/0 (socket says 172.21.15.33:54064) 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 -- 172.21.15.33:0/1715732417 learned_addr learned my addr 172.21.15.33:0/1715732417 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3699c700 1 --2- 172.21.15.33:0/1715732417 >> v2:172.21.15.167:3300/0 conn(0x7f0b300cd5e0 0x7f0b3015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 -- 172.21.15.33:0/1715732417 >> v2:172.21.15.167:3300/0 conn(0x7f0b300cd5e0 msgr2=0x7f0b3015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 --2- 172.21.15.33:0/1715732417 >> v2:172.21.15.167:3300/0 conn(0x7f0b300cd5e0 0x7f0b3015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 -- 172.21.15.33:0/1715732417 >> v1:172.21.15.167:6789/0 conn(0x7f0b3015a650 legacy=0x7f0b300cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 -- 172.21.15.33:0/1715732417 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3015d890 con 0x7f0b300af540 2023-10-06T05:15:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.826+0000 7f0b3619b700 1 --2- 172.21.15.33:0/1715732417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300af540 0x7f0b300af910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b300ce1c0 tx=0x7f0b2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ad966646276415 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b35199700 1 -- 172.21.15.33:0/1715732417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c009e70 con 0x7f0b300af540 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b35199700 1 -- 172.21.15.33:0/1715732417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c00b860 con 0x7f0b300af540 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b35199700 1 -- 172.21.15.33:0/1715732417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00bc00 con 0x7f0b300af540 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 -- 172.21.15.33:0/1715732417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300af540 msgr2=0x7f0b300af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 --2- 172.21.15.33:0/1715732417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300af540 0x7f0b300af910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b300ce1c0 tx=0x7f0b2c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 -- 172.21.15.33:0/1715732417 shutdown_connections 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 --2- 172.21.15.33:0/1715732417 >> v2:172.21.15.167:3300/0 conn(0x7f0b300cd5e0 0x7f0b3015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 --2- 172.21.15.33:0/1715732417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300af540 0x7f0b300af910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 -- 172.21.15.33:0/1715732417 >> 172.21.15.33:0/1715732417 conn(0x7f0b3006d9b0 msgr2=0x7f0b3006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 -- 172.21.15.33:0/1715732417 shutdown_connections 2023-10-06T05:15:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.827+0000 7f0b3719d700 1 -- 172.21.15.33:0/1715732417 wait complete. 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.829+0000 7f0b3719d700 1 Processor -- start 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.829+0000 7f0b3719d700 1 -- start start 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b300af540 0x7f0b300aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 0x7f0b300b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b3015a650 0x7f0b300b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b3b70 con 0x7f0b300cd5e0 2023-10-06T05:15:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b3e50 con 0x7f0b3015a650 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b4130 con 0x7f0b300af540 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 0x7f0b300b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3619b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b300af540 0x7f0b300aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 0x7f0b300b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54066/0 (socket says 172.21.15.33:54066) 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 -- 172.21.15.33:0/2607294630 learned_addr learned my addr 172.21.15.33:0/2607294630 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3699c700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b3015a650 0x7f0b300b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 -- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b300af540 msgr2=0x7f0b300aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b300af540 0x7f0b300aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 -- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b3015a650 msgr2=0x7f0b300b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b3015a650 0x7f0b300b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.830+0000 7f0b3599a700 1 -- 172.21.15.33:0/2607294630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0094f0 con 0x7f0b300cd5e0 2023-10-06T05:15:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.831+0000 7f0b3599a700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 0x7f0b300b4c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000c590 tx=0x7f0b2000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.831+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20011590 con 0x7f0b300cd5e0 2023-10-06T05:15:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.831+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20011710 con 0x7f0b300cd5e0 2023-10-06T05:15:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.831+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b300b4410 con 0x7f0b300cd5e0 2023-10-06T05:15:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.831+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20011ad0 con 0x7f0b300cd5e0 2023-10-06T05:15:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.831+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b300b5150 con 0x7f0b300cd5e0 2023-10-06T05:15:36.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.832+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b300b4930 con 0x7f0b300cd5e0 2023-10-06T05:15:36.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.833+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b2001b3b0 con 0x7f0b300cd5e0 2023-10-06T05:15:36.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.834+0000 7f0b267fc700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b14066f30 0x7f0b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:36.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.834+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0b20083bc0 con 0x7f0b300cd5e0 2023-10-06T05:15:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.835+0000 7f0b3619b700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b14066f30 0x7f0b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.835+0000 7f0b3619b700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b14066f30 0x7f0b140693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b300ce1c0 tx=0x7f0b2c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:36.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:36.838+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b300b4930 con 0x7f0b300cd5e0 2023-10-06T05:15:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.042+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b300b4930 con 0x7f0b300cd5e0 2023-10-06T05:15:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.043+0000 7f0b267fc700 1 -- 172.21.15.33:0/2607294630 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b300b4930 con 0x7f0b300cd5e0 2023-10-06T05:15:37.046 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:15:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b14066f30 msgr2=0x7f0b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b14066f30 0x7f0b140693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b300ce1c0 tx=0x7f0b2c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 msgr2=0x7f0b300b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 0x7f0b300b4c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000c590 tx=0x7f0b2000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 shutdown_connections 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b14066f30 0x7f0b140693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b3015a650 0x7f0b300b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b300af540 0x7f0b300aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 --2- 172.21.15.33:0/2607294630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b300cd5e0 0x7f0b300b4c10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.045+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 >> 172.21.15.33:0/2607294630 conn(0x7f0b3006d9b0 msgr2=0x7f0b300c6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.046+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 shutdown_connections 2023-10-06T05:15:37.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.046+0000 7f0b3719d700 1 -- 172.21.15.33:0/2607294630 wait complete. 2023-10-06T05:15:37.065 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.253+0000 7f011ab9d700 1 Processor -- start 2023-10-06T05:15:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.253+0000 7f011ab9d700 1 -- start start 2023-10-06T05:15:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.254+0000 7f011ab9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01140af540 0x7f01140af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.254+0000 7f011ab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01140ce890 con 0x7f011415a650 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.254+0000 7f011ab9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011415aac0 con 0x7f01140cd5e0 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.254+0000 7f011ab9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011415d890 con 0x7f01140af540 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f011939a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f011415a650 0x7f01140cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34604/0 (socket says 172.21.15.33:34604) 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f011939a700 1 -- 172.21.15.33:0/1888753632 learned_addr learned my addr 172.21.15.33:0/1888753632 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0119b9b700 1 --2- 172.21.15.33:0/1888753632 >> v2:172.21.15.167:3300/0 conn(0x7f01140af540 0x7f01140af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162547877 0 0) 0x7f01140ce890 con 0x7f011415a650 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0100003570 con 0x7f011415a650 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799105143 0 0) 0x7f011415aac0 con 0x7f01140cd5e0 2023-10-06T05:15:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01140ce890 con 0x7f01140cd5e0 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727608300 0 0) 0x7f0100003570 con 0x7f011415a650 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 >> v2:172.21.15.167:3300/0 conn(0x7f01140af540 msgr2=0x7f01140af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 --2- 172.21.15.33:0/1888753632 >> v2:172.21.15.167:3300/0 conn(0x7f01140af540 0x7f01140af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 >> v1:172.21.15.167:6789/0 conn(0x7f01140cd5e0 legacy=0x7f01140ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011415db70 con 0x7f011415a650 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f0104001470 con 0x7f011415a650 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 >> v1:172.21.15.33:6789/0 conn(0x7f011415a650 legacy=0x7f01140cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.255+0000 7f0118b99700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0100003bf0 0x7f0100003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f0118b99700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0100004810 0x7f0100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f0118b99700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01000051c0 0x7f010000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011415d890 con 0x7f01000051c0 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010000c0b0 con 0x7f0100004810 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010000c270 con 0x7f0100003bf0 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01000051c0 0x7f010000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f0119b9b700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0100003bf0 0x7f0100003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 -- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0100003bf0 msgr2=0x7f0100003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0100003bf0 0x7f0100003fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 -- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0100004810 msgr2=0x7f0100004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0100004810 0x7f0100004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 -- 172.21.15.33:0/1888753632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100003570 con 0x7f01000051c0 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 -- 172.21.15.33:0/1888753632 reap_dead start 2023-10-06T05:15:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.256+0000 7f011a39c700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01000051c0 0x7f010000ba60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f01140cd9e0 tx=0x7f010c0017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2974bb54b7d2b78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c001ae0 con 0x7f01000051c0 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c00c980 con 0x7f01000051c0 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f0118b99700 1 -- 172.21.15.33:0/1888753632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00cd60 con 0x7f01000051c0 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 -- 172.21.15.33:0/1888753632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01000051c0 msgr2=0x7f010000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01000051c0 0x7f010000ba60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f01140cd9e0 tx=0x7f010c0017d0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 -- 172.21.15.33:0/1888753632 shutdown_connections 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1888753632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01000051c0 0x7f010000ba60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 -- 172.21.15.33:0/1888753632 >> 172.21.15.33:0/1888753632 conn(0x7f011406d9b0 msgr2=0x7f011406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 -- 172.21.15.33:0/1888753632 shutdown_connections 2023-10-06T05:15:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.257+0000 7f011ab9d700 1 -- 172.21.15.33:0/1888753632 wait complete. 2023-10-06T05:15:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.259+0000 7f011ab9d700 1 Processor -- start 2023-10-06T05:15:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.259+0000 7f011ab9d700 1 -- start start 2023-10-06T05:15:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011ab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01000051c0 0x7f01140c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011ab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 0x7f0114156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011ab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0114169580 0x7f01140b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011ab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140c8870 con 0x7f01140b5560 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011ab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01140c7c60 con 0x7f0114169580 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011ab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01140c7f40 con 0x7f01000051c0 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 0x7f0114156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 0x7f0114156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54070/0 (socket says 172.21.15.33:54070) 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 -- 172.21.15.33:0/1463150051 learned_addr learned my addr 172.21.15.33:0/1463150051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f0119b9b700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01000051c0 0x7f01140c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011a39c700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0114169580 0x7f01140b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 -- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01000051c0 msgr2=0x7f01140c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01000051c0 0x7f01140c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 -- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0114169580 msgr2=0x7f01140b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0114169580 0x7f01140b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f011939a700 1 -- 172.21.15.33:0/1463150051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140c8220 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.260+0000 7f0119b9b700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01000051c0 0x7f01140c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.261+0000 7f011939a700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 0x7f0114156be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f010400c7a0 tx=0x7f0104001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.261+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104007360 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.261+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01040074c0 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.261+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104004830 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.261+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01140c8500 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.261+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0114157120 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.262+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011404c380 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.263+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0104004ba0 con 0x7f01140b5560 2023-10-06T05:15:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.264+0000 7f0109ffb700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f8066f30 0x7f00f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.264+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0104083ec0 con 0x7f01140b5560 2023-10-06T05:15:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.264+0000 7f0119b9b700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f8066f30 0x7f00f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.265+0000 7f0119b9b700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f8066f30 0x7f00f80693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0110009170 tx=0x7f0110008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.268+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0104050fb0 con 0x7f01140b5560 2023-10-06T05:15:37.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.467+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f011415de50 con 0x7f01140b5560 2023-10-06T05:15:37.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.468+0000 7f0109ffb700 1 -- 172.21.15.33:0/1463150051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0104018020 con 0x7f01140b5560 2023-10-06T05:15:37.471 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:15:37.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f8066f30 msgr2=0x7f00f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f8066f30 0x7f00f80693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0110009170 tx=0x7f0110008040 comp rx=0 tx=0).stop 2023-10-06T05:15:37.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 msgr2=0x7f0114156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 0x7f0114156be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f010400c7a0 tx=0x7f0104001940 comp rx=0 tx=0).stop 2023-10-06T05:15:37.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 shutdown_connections 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f8066f30 0x7f00f80693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0114169580 0x7f01140b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01000051c0 0x7f01140c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 --2- 172.21.15.33:0/1463150051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01140b5560 0x7f0114156be0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 >> 172.21.15.33:0/1463150051 conn(0x7f011406d9b0 msgr2=0x7f01140c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 shutdown_connections 2023-10-06T05:15:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.470+0000 7f011ab9d700 1 -- 172.21.15.33:0/1463150051 wait complete. 2023-10-06T05:15:37.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:37.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.675+0000 7f528639d700 1 Processor -- start 2023-10-06T05:15:37.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.675+0000 7f528639d700 1 -- start start 2023-10-06T05:15:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.676+0000 7f528639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.676+0000 7f528639d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f528006aad0 0x7f528006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.676+0000 7f528639d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52800c9cb0 con 0x7f52800c7940 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.676+0000 7f528639d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800ca270 con 0x7f528006aad0 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f528639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528006b840 con 0x7f528013faa0 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5285b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f528006aad0 0x7f528006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5284b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f52800c7940 0x7f52800c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34606/0 (socket says 172.21.15.33:34606) 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5284b9a700 1 -- 172.21.15.33:0/1588653707 learned_addr learned my addr 172.21.15.33:0/1588653707 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f528539b700 1 --2- 172.21.15.33:0/1588653707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5285b9c700 1 -- 172.21.15.33:0/1588653707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 msgr2=0x7f528013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5285b9c700 1 --2- 172.21.15.33:0/1588653707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5285b9c700 1 -- 172.21.15.33:0/1588653707 >> v1:172.21.15.33:6789/0 conn(0x7f52800c7940 legacy=0x7f52800c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5285b9c700 1 -- 172.21.15.33:0/1588653707 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528006bb20 con 0x7f528006aad0 2023-10-06T05:15:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.677+0000 7f5277fff700 1 -- 172.21.15.33:0/1588653707 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467635299 0 0) 0x7f52800c9cb0 con 0x7f52800c7940 2023-10-06T05:15:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f5285b9c700 1 --2- 172.21.15.33:0/1588653707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f528006aad0 0x7f528006af50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5278009e60 tx=0x7f5278009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63c21231259d8f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f5277fff700 1 -- 172.21.15.33:0/1588653707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800e070 con 0x7f528006aad0 2023-10-06T05:15:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f5277fff700 1 -- 172.21.15.33:0/1588653707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527800bc30 con 0x7f528006aad0 2023-10-06T05:15:37.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f5277fff700 1 -- 172.21.15.33:0/1588653707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278012430 con 0x7f528006aad0 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f528639d700 1 -- 172.21.15.33:0/1588653707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f528006aad0 msgr2=0x7f528006af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f528639d700 1 --2- 172.21.15.33:0/1588653707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f528006aad0 0x7f528006af50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5278009e60 tx=0x7f5278009780 comp rx=0 tx=0).stop 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f528639d700 1 -- 172.21.15.33:0/1588653707 shutdown_connections 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f528639d700 1 --2- 172.21.15.33:0/1588653707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f528006aad0 0x7f528006af50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.678+0000 7f528639d700 1 --2- 172.21.15.33:0/1588653707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.679+0000 7f528639d700 1 -- 172.21.15.33:0/1588653707 >> 172.21.15.33:0/1588653707 conn(0x7f528006d030 msgr2=0x7f528006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.679+0000 7f528639d700 1 -- 172.21.15.33:0/1588653707 shutdown_connections 2023-10-06T05:15:37.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.679+0000 7f528639d700 1 -- 172.21.15.33:0/1588653707 wait complete. 2023-10-06T05:15:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.680+0000 7f528639d700 1 Processor -- start 2023-10-06T05:15:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.680+0000 7f528639d700 1 -- start start 2023-10-06T05:15:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f528639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528006aad0 0x7f528013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f528639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 0x7f528013d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f528639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528016cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f528639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528013db20 con 0x7f52800c7940 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f528639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528013f030 con 0x7f528006aad0 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f528639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528013e660 con 0x7f528013faa0 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f5284b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 0x7f528013d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f5284b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 0x7f528013d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54100/0 (socket says 172.21.15.33:54100) 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.681+0000 7f5284b9a700 1 -- 172.21.15.33:0/3388762900 learned_addr learned my addr 172.21.15.33:0/3388762900 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f528539b700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528006aad0 0x7f528013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5285b9c700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528016cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5284b9a700 1 -- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 msgr2=0x7f528016cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5284b9a700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528016cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5284b9a700 1 -- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528006aad0 msgr2=0x7f528013eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5284b9a700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528006aad0 0x7f528013eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5284b9a700 1 -- 172.21.15.33:0/3388762900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5278009a20 con 0x7f52800c7940 2023-10-06T05:15:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f528539b700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528006aad0 0x7f528013eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f5284b9a700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 0x7f528013d1d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f527000c7e0 tx=0x7f5270007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270005b30 con 0x7f52800c7940 2023-10-06T05:15:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270005c90 con 0x7f52800c7940 2023-10-06T05:15:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528013dd30 con 0x7f52800c7940 2023-10-06T05:15:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270007300 con 0x7f52800c7940 2023-10-06T05:15:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.682+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528013e2f0 con 0x7f52800c7940 2023-10-06T05:15:37.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.683+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528013e7d0 con 0x7f52800c7940 2023-10-06T05:15:37.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.684+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f527001b400 con 0x7f52800c7940 2023-10-06T05:15:37.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.685+0000 7f52757fa700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5264066f60 0x7f5264069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.685+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52700841d0 con 0x7f52800c7940 2023-10-06T05:15:37.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.685+0000 7f528539b700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5264066f60 0x7f5264069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.686+0000 7f528539b700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5264066f60 0x7f5264069410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f527c006fd0 tx=0x7f527c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.689+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52700512c0 con 0x7f52800c7940 2023-10-06T05:15:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.935+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f528004c380 con 0x7f52800c7940 2023-10-06T05:15:37.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.935+0000 7f52757fa700 1 -- 172.21.15.33:0/3388762900 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f52700548e0 con 0x7f52800c7940 2023-10-06T05:15:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:37.938 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:24.301322+0000","modified":"2023-10-06T05:15:27.844263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6836/1338636713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1338636713},{"type":"v1","addr":"172.21.15.167:6836","nonce":1338636713}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.937+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5264066f60 msgr2=0x7f5264069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.937+0000 7f528639d700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5264066f60 0x7f5264069410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f527c006fd0 tx=0x7f527c008040 comp rx=0 tx=0).stop 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.937+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 msgr2=0x7f528013d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.937+0000 7f528639d700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 0x7f528013d1d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f527000c7e0 tx=0x7f5270007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 shutdown_connections 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5264066f60 0x7f5264069410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528013faa0 0x7f528016cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528006aad0 0x7f528013eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 --2- 172.21.15.33:0/3388762900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52800c7940 0x7f528013d1d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 >> 172.21.15.33:0/3388762900 conn(0x7f528006d030 msgr2=0x7f52800c2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 shutdown_connections 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:37.938+0000 7f528639d700 1 -- 172.21.15.33:0/3388762900 wait complete. 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-10-06T05:15:37.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:38.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.143+0000 7f1aad99d700 1 Processor -- start 2023-10-06T05:15:38.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.143+0000 7f1aad99d700 1 -- start start 2023-10-06T05:15:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1aa813faa0 0x7f1aa813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80c7940 0x7f1aa80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa80c9cb0 con 0x7f1aa806aad0 2023-10-06T05:15:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa806b660 con 0x7f1aa813faa0 2023-10-06T05:15:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80ca270 con 0x7f1aa80c7940 2023-10-06T05:15:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1aa806aad0 0x7f1aa806af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34610/0 (socket says 172.21.15.33:34610) 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.144+0000 7f1aad19c700 1 -- 172.21.15.33:0/3844728791 learned_addr learned my addr 172.21.15.33:0/3844728791 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aa7fff700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80c7940 0x7f1aa80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aac99b700 1 --2- 172.21.15.33:0/3844728791 >> v2:172.21.15.167:3300/0 conn(0x7f1aa813faa0 0x7f1aa813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aa77fe700 1 -- 172.21.15.33:0/3844728791 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138252951 0 0) 0x7f1aa80c9cb0 con 0x7f1aa806aad0 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aa77fe700 1 -- 172.21.15.33:0/3844728791 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a98003570 con 0x7f1aa806aad0 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aac99b700 1 -- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80c7940 msgr2=0x7f1aa80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aac99b700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80c7940 0x7f1aa80c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aac99b700 1 -- 172.21.15.33:0/3844728791 >> v1:172.21.15.33:6789/0 conn(0x7f1aa806aad0 legacy=0x7f1aa806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aac99b700 1 -- 172.21.15.33:0/3844728791 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa806b840 con 0x7f1aa813faa0 2023-10-06T05:15:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.145+0000 7f1aa7fff700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80c7940 0x7f1aa80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:38.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aac99b700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa813fe70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098e0 tx=0x7f1a9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=626348a63ee19f26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aa77fe700 1 -- 172.21.15.33:0/3844728791 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011630 con 0x7f1aa813faa0 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aa77fe700 1 -- 172.21.15.33:0/3844728791 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c011790 con 0x7f1aa813faa0 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aa77fe700 1 -- 172.21.15.33:0/3844728791 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011b30 con 0x7f1aa813faa0 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aad99d700 1 -- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 msgr2=0x7f1aa813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa813fe70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098e0 tx=0x7f1a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aad99d700 1 -- 172.21.15.33:0/3844728791 shutdown_connections 2023-10-06T05:15:38.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa813fe70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3844728791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa80c7940 0x7f1aa80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.146+0000 7f1aad99d700 1 -- 172.21.15.33:0/3844728791 >> 172.21.15.33:0/3844728791 conn(0x7f1aa806d030 msgr2=0x7f1aa806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:38.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.147+0000 7f1aad99d700 1 -- 172.21.15.33:0/3844728791 shutdown_connections 2023-10-06T05:15:38.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.147+0000 7f1aad99d700 1 -- 172.21.15.33:0/3844728791 wait complete. 2023-10-06T05:15:38.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.148+0000 7f1aad99d700 1 Processor -- start 2023-10-06T05:15:38.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.148+0000 7f1aad99d700 1 -- start start 2023-10-06T05:15:38.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aad99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa806aad0 0x7f1aa80c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aad99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 0x7f1aa80c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aad99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa80c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aad99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80c64d0 con 0x7f1aa80c7940 2023-10-06T05:15:38.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aad99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa8169510 con 0x7f1aa813faa0 2023-10-06T05:15:38.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aad99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa81697d0 con 0x7f1aa806aad0 2023-10-06T05:15:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aa7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 0x7f1aa80c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aa7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 0x7f1aa80c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54108/0 (socket says 172.21.15.33:54108) 2023-10-06T05:15:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.149+0000 7f1aa7fff700 1 -- 172.21.15.33:0/3980692263 learned_addr learned my addr 172.21.15.33:0/3980692263 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aad19c700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa80c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aac99b700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa806aad0 0x7f1aa80c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa7fff700 1 -- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa806aad0 msgr2=0x7f1aa80c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa7fff700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa806aad0 0x7f1aa80c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa7fff700 1 -- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 msgr2=0x7f1aa80c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa7fff700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa80c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa7fff700 1 -- 172.21.15.33:0/3980692263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94009420 con 0x7f1aa80c7940 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa7fff700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 0x7f1aa80c1370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400ec80 tx=0x7f1a940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400c9d0 con 0x7f1aa80c7940 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400cb30 con 0x7f1aa80c7940 2023-10-06T05:15:38.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400ceb0 con 0x7f1aa80c7940 2023-10-06T05:15:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c009420 con 0x7f1aa80c7940 2023-10-06T05:15:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.150+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa8169d90 con 0x7f1aa80c7940 2023-10-06T05:15:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.151+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa816cb10 con 0x7f1aa80c7940 2023-10-06T05:15:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.152+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a94013680 con 0x7f1aa80c7940 2023-10-06T05:15:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.153+0000 7f1aa4ff9700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a8c066f30 0x7f1a8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.153+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a94084b90 con 0x7f1aa80c7940 2023-10-06T05:15:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.153+0000 7f1aac99b700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a8c066f30 0x7f1a8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.154+0000 7f1aac99b700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a8c066f30 0x7f1a8c0693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0091e0 tx=0x7f1a9c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:38.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.157+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1aa816cb10 con 0x7f1aa80c7940 2023-10-06T05:15:38.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.359+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aa816cb10 con 0x7f1aa80c7940 2023-10-06T05:15:38.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.360+0000 7f1aa4ff9700 1 -- 172.21.15.33:0/3980692263 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f1aa816cb10 con 0x7f1aa80c7940 2023-10-06T05:15:38.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:38.363 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:25.826634+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:22.479742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:23.276900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a8c066f30 msgr2=0x7f1a8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a8c066f30 0x7f1a8c0693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0091e0 tx=0x7f1a9c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 msgr2=0x7f1aa80c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 0x7f1aa80c1370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400ec80 tx=0x7f1a940091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 shutdown_connections 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a8c066f30 0x7f1a8c0693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1aa813faa0 0x7f1aa80c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1aa806aad0 0x7f1aa80c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 --2- 172.21.15.33:0/3980692263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa80c7940 0x7f1aa80c1370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.362+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 >> 172.21.15.33:0/3980692263 conn(0x7f1aa806d030 msgr2=0x7f1aa806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:38.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.363+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 shutdown_connections 2023-10-06T05:15:38.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.363+0000 7f1aad99d700 1 -- 172.21.15.33:0/3980692263 wait complete. 2023-10-06T05:15:38.382 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:15:38.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:38.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.567+0000 7fad8b59e700 1 Processor -- start 2023-10-06T05:15:38.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.567+0000 7fad8b59e700 1 -- start start 2023-10-06T05:15:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fad8c155af0 0x7fad8c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c06a6a0 0x7fad8c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fad8c0c8b50 0x7fad8c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06b7e0 con 0x7fad8c0c8b50 2023-10-06T05:15:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06ab60 con 0x7fad8c155af0 2023-10-06T05:15:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06ae40 con 0x7fad8c06a6a0 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fad8c0c8b50 0x7fad8c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fad8c0c8b50 0x7fad8c0c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54110/0 (socket says 172.21.15.33:54110) 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 -- 172.21.15.33:0/4123051092 learned_addr learned my addr 172.21.15.33:0/4123051092 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8a59c700 1 --2- 172.21.15.33:0/4123051092 >> v2:172.21.15.167:3300/0 conn(0x7fad8c155af0 0x7fad8c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad89d9b700 1 --2- 172.21.15.33:0/4123051092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c06a6a0 0x7fad8c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 -- 172.21.15.33:0/4123051092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c06a6a0 msgr2=0x7fad8c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 --2- 172.21.15.33:0/4123051092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c06a6a0 0x7fad8c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 -- 172.21.15.33:0/4123051092 >> v2:172.21.15.167:3300/0 conn(0x7fad8c155af0 msgr2=0x7fad8c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 --2- 172.21.15.33:0/4123051092 >> v2:172.21.15.167:3300/0 conn(0x7fad8c155af0 0x7fad8c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.569+0000 7fad8ad9d700 1 -- 172.21.15.33:0/4123051092 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c06b120 con 0x7fad8c0c8b50 2023-10-06T05:15:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.570+0000 7fad8ad9d700 1 --2- 172.21.15.33:0/4123051092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c0c8b50 0x7fad8c0c7750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0697c0 tx=0x7fad84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c7665bdd05fcdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.570+0000 7fad8959a700 1 -- 172.21.15.33:0/4123051092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400e070 con 0x7fad8c0c8b50 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.570+0000 7fad8959a700 1 -- 172.21.15.33:0/4123051092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8400ba20 con 0x7fad8c0c8b50 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.570+0000 7fad8959a700 1 -- 172.21.15.33:0/4123051092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840092e0 con 0x7fad8c0c8b50 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 -- 172.21.15.33:0/4123051092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c0c8b50 msgr2=0x7fad8c0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 --2- 172.21.15.33:0/4123051092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c0c8b50 0x7fad8c0c7750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0697c0 tx=0x7fad84009840 comp rx=0 tx=0).stop 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 -- 172.21.15.33:0/4123051092 shutdown_connections 2023-10-06T05:15:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 --2- 172.21.15.33:0/4123051092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c0c8b50 0x7fad8c0c7750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 --2- 172.21.15.33:0/4123051092 >> v2:172.21.15.167:3300/0 conn(0x7fad8c155af0 0x7fad8c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 --2- 172.21.15.33:0/4123051092 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c06a6a0 0x7fad8c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 -- 172.21.15.33:0/4123051092 >> 172.21.15.33:0/4123051092 conn(0x7fad8c06d030 msgr2=0x7fad8c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 -- 172.21.15.33:0/4123051092 shutdown_connections 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.571+0000 7fad8b59e700 1 -- 172.21.15.33:0/4123051092 wait complete. 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.572+0000 7fad8b59e700 1 Processor -- start 2023-10-06T05:15:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.572+0000 7fad8b59e700 1 -- start start 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 0x7fad8c14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c0c8b50 0x7fad8c0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8c155af0 0x7fad8c0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0c39c0 con 0x7fad8c06a6a0 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0aff50 con 0x7fad8c155af0 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0b0230 con 0x7fad8c0c8b50 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 0x7fad8c14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 0x7fad8c14ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54114/0 (socket says 172.21.15.33:54114) 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.573+0000 7fad8a59c700 1 -- 172.21.15.33:0/1008870129 learned_addr learned my addr 172.21.15.33:0/1008870129 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8a59c700 1 -- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c0c8b50 msgr2=0x7fad8c0ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8a59c700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c0c8b50 0x7fad8c0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8ad9d700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8c155af0 0x7fad8c0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8a59c700 1 -- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8c155af0 msgr2=0x7fad8c0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8a59c700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8c155af0 0x7fad8c0ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8a59c700 1 -- 172.21.15.33:0/1008870129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84009ae0 con 0x7fad8c06a6a0 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8a59c700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 0x7fad8c14ff50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e930 tx=0x7fad7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c009510 con 0x7fad8c06a6a0 2023-10-06T05:15:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00c850 con 0x7fad8c06a6a0 2023-10-06T05:15:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c0b0510 con 0x7fad8c06a6a0 2023-10-06T05:15:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00cc10 con 0x7fad8c06a6a0 2023-10-06T05:15:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.574+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c150490 con 0x7fad8c06a6a0 2023-10-06T05:15:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.576+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c0c3cf0 con 0x7fad8c06a6a0 2023-10-06T05:15:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.576+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad7c013470 con 0x7fad8c06a6a0 2023-10-06T05:15:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.577+0000 7fad7a7fc700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad6c066f30 0x7fad6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.577+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad7c084800 con 0x7fad8c06a6a0 2023-10-06T05:15:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.578+0000 7fad89d9b700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad6c066f30 0x7fad6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.578+0000 7fad89d9b700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad6c066f30 0x7fad6c0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0c2820 tx=0x7fad80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.581+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad7c051910 con 0x7fad8c06a6a0 2023-10-06T05:15:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.829+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad8c0c3b00 con 0x7fad8c06a6a0 2023-10-06T05:15:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.830+0000 7fad7a7fc700 1 -- 172.21.15.33:0/1008870129 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fad7c054f30 con 0x7fad8c06a6a0 2023-10-06T05:15:38.833 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:38.833 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:24.301322+0000","modified":"2023-10-06T05:15:27.844263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6836/1338636713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1338636713},{"type":"v1","addr":"172.21.15.167:6836","nonce":1338636713}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad6c066f30 msgr2=0x7fad6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad6c066f30 0x7fad6c0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0c2820 tx=0x7fad80008040 comp rx=0 tx=0).stop 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 msgr2=0x7fad8c14ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 0x7fad8c14ff50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e930 tx=0x7fad7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 shutdown_connections 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad6c066f30 0x7fad6c0693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad8c155af0 0x7fad8c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad8c06a6a0 0x7fad8c14ff50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 --2- 172.21.15.33:0/1008870129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad8c0c8b50 0x7fad8c0ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 >> 172.21.15.33:0/1008870129 conn(0x7fad8c06d030 msgr2=0x7fad8c159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 shutdown_connections 2023-10-06T05:15:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:38.832+0000 7fad8b59e700 1 -- 172.21.15.33:0/1008870129 wait complete. 2023-10-06T05:15:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-10-06T05:15:38.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:39.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.038+0000 7fdad0b9d700 1 Processor -- start 2023-10-06T05:15:39.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.038+0000 7fdad0b9d700 1 -- start start 2023-10-06T05:15:39.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.039+0000 7fdad0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 0x7fdacc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.039+0000 7fdad0b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdacc0cc750 0x7fdacc0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.039+0000 7fdad0b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdacc0cbd80 con 0x7fdacc0cf3b0 2023-10-06T05:15:39.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.039+0000 7fdad0b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc0af5f0 con 0x7fdacc0cc750 2023-10-06T05:15:39.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.039+0000 7fdad0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0af8d0 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 0x7fdacc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdacc0cc750 0x7fdacc0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdacc0cc750 0x7fdacc0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:52870/0 (socket says 172.21.15.33:52870) 2023-10-06T05:15:39.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacaffd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdacc0cf3b0 0x7fdacc1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:51540/0 (socket says 172.21.15.33:51540) 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 0x7fdacc0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:53732/0 (socket says 172.21.15.33:53732) 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 -- 172.21.15.33:0/275128580 learned_addr learned my addr 172.21.15.33:0/275128580 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 -- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 msgr2=0x7fdacc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 --2- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 0x7fdacc0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 -- 172.21.15.33:0/275128580 >> v1:172.21.15.167:6789/0 conn(0x7fdacc0cf3b0 legacy=0x7fdacc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacbfff700 1 -- 172.21.15.33:0/275128580 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0afbb0 con 0x7fdacc0cc750 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdacb7fe700 1 --2- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 0x7fdacc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.040+0000 7fdaca7fc700 1 -- 172.21.15.33:0/275128580 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940297476 0 0) 0x7fdacc0cbd80 con 0x7fdacc0cf3b0 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.041+0000 7fdacbfff700 1 --2- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc0b0460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009f20 tx=0x7fdac4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e19f45a7baef76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.041+0000 7fdaca7fc700 1 -- 172.21.15.33:0/275128580 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400e070 con 0x7fdacc0cc750 2023-10-06T05:15:39.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.041+0000 7fdaca7fc700 1 -- 172.21.15.33:0/275128580 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400b9d0 con 0x7fdacc0cc750 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.041+0000 7fdaca7fc700 1 -- 172.21.15.33:0/275128580 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40092e0 con 0x7fdacc0cc750 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.041+0000 7fdad0b9d700 1 -- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 msgr2=0x7fdacc0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.041+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc0b0460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009f20 tx=0x7fdac4009840 comp rx=0 tx=0).stop 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.042+0000 7fdad0b9d700 1 -- 172.21.15.33:0/275128580 shutdown_connections 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.042+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc0b0460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.042+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/275128580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc0ccfc0 0x7fdacc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.042+0000 7fdad0b9d700 1 -- 172.21.15.33:0/275128580 >> 172.21.15.33:0/275128580 conn(0x7fdacc06cff0 msgr2=0x7fdacc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.042+0000 7fdad0b9d700 1 -- 172.21.15.33:0/275128580 shutdown_connections 2023-10-06T05:15:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.042+0000 7fdad0b9d700 1 -- 172.21.15.33:0/275128580 wait complete. 2023-10-06T05:15:39.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.043+0000 7fdad0b9d700 1 Processor -- start 2023-10-06T05:15:39.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.043+0000 7fdad0b9d700 1 -- start start 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdad0b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdad0b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 0x7fdacc153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdad0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc16d850 0x7fdacc0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdad0b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc153fd0 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdad0b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc153cf0 con 0x7fdacc0cc750 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdad0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc16cd70 con 0x7fdacc16d850 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdacaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 0x7fdacc153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdacaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 0x7fdacc153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54124/0 (socket says 172.21.15.33:54124) 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.044+0000 7fdacaffd700 1 -- 172.21.15.33:0/1626691956 learned_addr learned my addr 172.21.15.33:0/1626691956 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacbfff700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc16d850 0x7fdacc0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacb7fe700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacaffd700 1 -- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc16d850 msgr2=0x7fdacc0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacaffd700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc16d850 0x7fdacc0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacaffd700 1 -- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 msgr2=0x7fdacc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacaffd700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacaffd700 1 -- 172.21.15.33:0/1626691956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc009420 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdacaffd700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 0x7fdacc153360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c510 tx=0x7fdac000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0005a70 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0005bf0 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.045+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac00070d0 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.046+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac4009ae0 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.046+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdacc16d330 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.047+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdacc16d520 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.048+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdac0007480 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.049+0000 7fdab37fe700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdab8066f30 0x7fdab80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.049+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdac0083b10 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.049+0000 7fdacb7fe700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdab8066f30 0x7fdab80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.050+0000 7fdacb7fe700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdab8066f30 0x7fdab80693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009f40 tx=0x7fdabc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.054+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdac0050b80 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.300+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdacc04c380 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.301+0000 7fdab37fe700 1 -- 172.21.15.33:0/1626691956 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fdac0017020 con 0x7fdacc0ccfc0 2023-10-06T05:15:39.304 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:39.304 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:24.301322+0000","modified":"2023-10-06T05:15:27.844263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5028},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5028":{"gid":5028,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6836/1338636713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1338636713},{"type":"v1","addr":"172.21.15.167:6836","nonce":1338636713}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:15:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdab8066f30 msgr2=0x7fdab80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdab8066f30 0x7fdab80693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009f40 tx=0x7fdabc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 msgr2=0x7fdacc153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 0x7fdacc153360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c510 tx=0x7fdac000c820 comp rx=0 tx=0).stop 2023-10-06T05:15:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 shutdown_connections 2023-10-06T05:15:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdab8066f30 0x7fdab80693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdacc16d850 0x7fdacc0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc0cc750 0x7fdacc154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 --2- 172.21.15.33:0/1626691956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdacc0ccfc0 0x7fdacc153360 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 >> 172.21.15.33:0/1626691956 conn(0x7fdacc06cff0 msgr2=0x7fdacc0c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 shutdown_connections 2023-10-06T05:15:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.303+0000 7fdad0b9d700 1 -- 172.21.15.33:0/1626691956 wait complete. 2023-10-06T05:15:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2023-10-06T05:15:39.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.509+0000 7f2a4d99d700 1 Processor -- start 2023-10-06T05:15:39.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.509+0000 7f2a4d99d700 1 -- start start 2023-10-06T05:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4813faa0 0x7f2a4813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a480c7940 0x7f2a480c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480c9cb0 con 0x7f2a4806aad0 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4806b660 con 0x7f2a480c7940 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480ca270 con 0x7f2a4813faa0 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2a4806aad0 0x7f2a4806af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34622/0 (socket says 172.21.15.33:34622) 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4d19c700 1 -- 172.21.15.33:0/3949290779 learned_addr learned my addr 172.21.15.33:0/3949290779 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.511+0000 7f2a4c99b700 1 --2- 172.21.15.33:0/3949290779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4813faa0 0x7f2a4813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a47fff700 1 --2- 172.21.15.33:0/3949290779 >> v2:172.21.15.167:3300/0 conn(0x7f2a480c7940 0x7f2a480c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a477fe700 1 -- 172.21.15.33:0/3949290779 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277737480 0 0) 0x7f2a480c9cb0 con 0x7f2a4806aad0 2023-10-06T05:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a477fe700 1 -- 172.21.15.33:0/3949290779 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a38003570 con 0x7f2a4806aad0 2023-10-06T05:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a4c99b700 1 -- 172.21.15.33:0/3949290779 >> v2:172.21.15.167:3300/0 conn(0x7f2a480c7940 msgr2=0x7f2a480c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a4c99b700 1 --2- 172.21.15.33:0/3949290779 >> v2:172.21.15.167:3300/0 conn(0x7f2a480c7940 0x7f2a480c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a4c99b700 1 -- 172.21.15.33:0/3949290779 >> v1:172.21.15.33:6789/0 conn(0x7f2a4806aad0 legacy=0x7f2a4806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a4c99b700 1 -- 172.21.15.33:0/3949290779 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4806b840 con 0x7f2a4813faa0 2023-10-06T05:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.512+0000 7f2a4c99b700 1 --2- 172.21.15.33:0/3949290779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4813faa0 0x7f2a4813fe70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0098e0 tx=0x7f2a3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65739dc1fd279063 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a477fe700 1 -- 172.21.15.33:0/3949290779 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c011630 con 0x7f2a4813faa0 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a477fe700 1 -- 172.21.15.33:0/3949290779 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c011790 con 0x7f2a4813faa0 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a477fe700 1 -- 172.21.15.33:0/3949290779 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c011b30 con 0x7f2a4813faa0 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3949290779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4813faa0 msgr2=0x7f2a4813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3949290779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4813faa0 0x7f2a4813fe70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0098e0 tx=0x7f2a3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3949290779 shutdown_connections 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3949290779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4813faa0 0x7f2a4813fe70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3949290779 >> v2:172.21.15.167:3300/0 conn(0x7f2a480c7940 0x7f2a480c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3949290779 >> 172.21.15.33:0/3949290779 conn(0x7f2a4806d030 msgr2=0x7f2a4806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3949290779 shutdown_connections 2023-10-06T05:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.513+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3949290779 wait complete. 2023-10-06T05:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.515+0000 7f2a4d99d700 1 Processor -- start 2023-10-06T05:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.515+0000 7f2a4d99d700 1 -- start start 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4806aad0 0x7f2a4813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 0x7f2a48156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4813faa0 0x7f2a4815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a481582d0 con 0x7f2a480c7940 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a481576c0 con 0x7f2a4813faa0 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a481579a0 con 0x7f2a4806aad0 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 0x7f2a48156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 0x7f2a48156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54138/0 (socket says 172.21.15.33:54138) 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 -- 172.21.15.33:0/3046170367 learned_addr learned my addr 172.21.15.33:0/3046170367 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4c99b700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4806aad0 0x7f2a4813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d19c700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4813faa0 0x7f2a4815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 -- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4806aad0 msgr2=0x7f2a4813ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4806aad0 0x7f2a4813ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 -- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4813faa0 msgr2=0x7f2a4815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4813faa0 0x7f2a4815dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a47fff700 1 -- 172.21.15.33:0/3046170367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c009420 con 0x7f2a480c7940 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.516+0000 7f2a4d19c700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4813faa0 0x7f2a4815dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.517+0000 7f2a47fff700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 0x7f2a48156c00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400ec80 tx=0x7f2a340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.517+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400c850 con 0x7f2a480c7940 2023-10-06T05:15:39.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.517+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3400c9b0 con 0x7f2a480c7940 2023-10-06T05:15:39.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.517+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a48157c80 con 0x7f2a480c7940 2023-10-06T05:15:39.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.517+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400cd50 con 0x7f2a480c7940 2023-10-06T05:15:39.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.517+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a480ae100 con 0x7f2a480c7940 2023-10-06T05:15:39.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.518+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4815b230 con 0x7f2a480c7940 2023-10-06T05:15:39.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.519+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a34013680 con 0x7f2a480c7940 2023-10-06T05:15:39.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.520+0000 7f2a44ff9700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a2c066f30 0x7f2a2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.520+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2a340849d0 con 0x7f2a480c7940 2023-10-06T05:15:39.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.521+0000 7f2a4c99b700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a2c066f30 0x7f2a2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.521+0000 7f2a4c99b700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a2c066f30 0x7f2a2c0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0091e0 tx=0x7f2a3c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.524+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a4815b230 con 0x7f2a480c7940 2023-10-06T05:15:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.728+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a4815b230 con 0x7f2a480c7940 2023-10-06T05:15:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.729+0000 7f2a44ff9700 1 -- 172.21.15.33:0/3046170367 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f2a4815b230 con 0x7f2a480c7940 2023-10-06T05:15:39.732 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:39.732 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:25.826634+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:22.479742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:23.276900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a2c066f30 msgr2=0x7f2a2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a2c066f30 0x7f2a2c0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0091e0 tx=0x7f2a3c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 msgr2=0x7f2a48156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 0x7f2a48156c00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400ec80 tx=0x7f2a340091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 shutdown_connections 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a2c066f30 0x7f2a2c0693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a4813faa0 0x7f2a4815dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a4806aad0 0x7f2a4813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 --2- 172.21.15.33:0/3046170367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a480c7940 0x7f2a48156c00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.731+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 >> 172.21.15.33:0/3046170367 conn(0x7f2a4806d030 msgr2=0x7f2a4806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.732+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 shutdown_connections 2023-10-06T05:15:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.732+0000 7f2a4d99d700 1 -- 172.21.15.33:0/3046170367 wait complete. 2023-10-06T05:15:39.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:15:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.939+0000 7f454d39d700 1 Processor -- start 2023-10-06T05:15:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.940+0000 7f454d39d700 1 -- start start 2023-10-06T05:15:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454d39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f45480c7070 0x7f45480c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454d39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4548155bc0 0x7f45480c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454d39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454806a410 con 0x7f45481557f0 2023-10-06T05:15:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454d39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480cccd0 con 0x7f4548155bc0 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454d39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480c7500 con 0x7f45480c7070 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454cb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4548155bc0 0x7f45480c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454cb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4548155bc0 0x7f45480c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41356/0 (socket says 172.21.15.33:41356) 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f454cb9c700 1 -- 172.21.15.33:0/3395953037 learned_addr learned my addr 172.21.15.33:0/3395953037 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.941+0000 7f45477fe700 1 --2- 172.21.15.33:0/3395953037 >> v2:172.21.15.167:3300/0 conn(0x7f45480c7070 0x7f45480c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f454cb9c700 1 -- 172.21.15.33:0/3395953037 >> v2:172.21.15.167:3300/0 conn(0x7f45480c7070 msgr2=0x7f45480c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f454cb9c700 1 --2- 172.21.15.33:0/3395953037 >> v2:172.21.15.167:3300/0 conn(0x7f45480c7070 0x7f45480c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f454cb9c700 1 -- 172.21.15.33:0/3395953037 >> v1:172.21.15.167:6789/0 conn(0x7f45481557f0 legacy=0x7f45480c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f454cb9c700 1 -- 172.21.15.33:0/3395953037 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45480c77e0 con 0x7f4548155bc0 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f45477fe700 1 --2- 172.21.15.33:0/3395953037 >> v2:172.21.15.167:3300/0 conn(0x7f45480c7070 0x7f45480c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f454cb9c700 1 --2- 172.21.15.33:0/3395953037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4548155bc0 0x7f45480c9050 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f45480b22f0 tx=0x7f4540009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d51c3ef3458ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f4546ffd700 1 -- 172.21.15.33:0/3395953037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000e070 con 0x7f4548155bc0 2023-10-06T05:15:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f4546ffd700 1 -- 172.21.15.33:0/3395953037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000ba00 con 0x7f4548155bc0 2023-10-06T05:15:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.942+0000 7f4546ffd700 1 -- 172.21.15.33:0/3395953037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45400092e0 con 0x7f4548155bc0 2023-10-06T05:15:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 -- 172.21.15.33:0/3395953037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4548155bc0 msgr2=0x7f45480c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 --2- 172.21.15.33:0/3395953037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4548155bc0 0x7f45480c9050 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f45480b22f0 tx=0x7f4540009840 comp rx=0 tx=0).stop 2023-10-06T05:15:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 -- 172.21.15.33:0/3395953037 shutdown_connections 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 --2- 172.21.15.33:0/3395953037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4548155bc0 0x7f45480c9050 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 --2- 172.21.15.33:0/3395953037 >> v2:172.21.15.167:3300/0 conn(0x7f45480c7070 0x7f45480c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 -- 172.21.15.33:0/3395953037 >> 172.21.15.33:0/3395953037 conn(0x7f454806c560 msgr2=0x7f454806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 -- 172.21.15.33:0/3395953037 shutdown_connections 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.943+0000 7f454d39d700 1 -- 172.21.15.33:0/3395953037 wait complete. 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.944+0000 7f454d39d700 1 Processor -- start 2023-10-06T05:15:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.945+0000 7f454d39d700 1 -- start start 2023-10-06T05:15:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.945+0000 7f454d39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45480c7070 0x7f45480c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f454d39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 0x7f45480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f454d39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4548155bc0 0x7f45480c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f454d39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c5bf0 con 0x7f45481557f0 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f454d39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c5ed0 con 0x7f45480c7070 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f454d39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c61b0 con 0x7f4548155bc0 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 0x7f45480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 0x7f45480caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41360/0 (socket says 172.21.15.33:41360) 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 -- 172.21.15.33:0/317730222 learned_addr learned my addr 172.21.15.33:0/317730222 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f4547fff700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45480c7070 0x7f45480c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f454cb9c700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4548155bc0 0x7f45480c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 -- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4548155bc0 msgr2=0x7f45480c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4548155bc0 0x7f45480c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 -- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45480c7070 msgr2=0x7f45480c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45480c7070 0x7f45480c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.946+0000 7f45477fe700 1 -- 172.21.15.33:0/317730222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540009ae0 con 0x7f45481557f0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.947+0000 7f45477fe700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 0x7f45480caa80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f453c00d860 tx=0x7f453c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.947+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c0125f0 con 0x7f45481557f0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.947+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c012750 con 0x7f45481557f0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.947+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c012ad0 con 0x7f45481557f0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.947+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45480c6490 con 0x7f45481557f0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.947+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45480cafc0 con 0x7f45481557f0 2023-10-06T05:15:39.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.948+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45480c66b0 con 0x7f45481557f0 2023-10-06T05:15:39.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.949+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f453c014430 con 0x7f45481557f0 2023-10-06T05:15:39.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.950+0000 7f452ffff700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4534066f30 0x7f45340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.950+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f453c084bb0 con 0x7f45481557f0 2023-10-06T05:15:39.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.950+0000 7f4547fff700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4534066f30 0x7f45340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.951+0000 7f4547fff700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4534066f30 0x7f45340693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4538003870 tx=0x7f4538009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:39.954+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f453c051c20 con 0x7f45481557f0 2023-10-06T05:15:40.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.203+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f45480cb2a0 con 0x7f45481557f0 2023-10-06T05:15:40.206 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:15:40.203+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:15:40.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.208+0000 7f452ffff700 1 -- 172.21.15.33:0/317730222 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f453c055240 con 0x7f45481557f0 2023-10-06T05:15:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4534066f30 msgr2=0x7f45340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4534066f30 0x7f45340693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4538003870 tx=0x7f4538009040 comp rx=0 tx=0).stop 2023-10-06T05:15:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 msgr2=0x7f45480caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 0x7f45480caa80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f453c00d860 tx=0x7f453c008040 comp rx=0 tx=0).stop 2023-10-06T05:15:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 shutdown_connections 2023-10-06T05:15:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4534066f30 0x7f45340693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4548155bc0 0x7f45480c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45480c7070 0x7f45480c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 --2- 172.21.15.33:0/317730222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45481557f0 0x7f45480caa80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 >> 172.21.15.33:0/317730222 conn(0x7f454806c560 msgr2=0x7f454806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 shutdown_connections 2023-10-06T05:15:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.211+0000 7f454d39d700 1 -- 172.21.15.33:0/317730222 wait complete. 2023-10-06T05:15:40.216 INFO:tasks.ceph.mds.b.smithi167.stderr: -687> 2023-10-06T05:15:27.685+0000 7f04abdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:15:40.216 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:15:40.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:15:40.254 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:15:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.419+0000 7f85c099d700 1 Processor -- start 2023-10-06T05:15:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.419+0000 7f85c099d700 1 -- start start 2023-10-06T05:15:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.420+0000 7f85c099d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85bc0af540 0x7f85bc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.420+0000 7f85c099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.420+0000 7f85c099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc15d660 con 0x7f85bc15a650 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.420+0000 7f85c099d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0ce890 con 0x7f85bc0af540 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.420+0000 7f85c099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc15aac0 con 0x7f85bc0cd5e0 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.420+0000 7f85bb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85bc0af540 0x7f85bc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85baffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f85bc15a650 0x7f85bc0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40414/0 (socket says 172.21.15.33:40414) 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85baffd700 1 -- 172.21.15.33:0/732580074 learned_addr learned my addr 172.21.15.33:0/732580074 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85bbfff700 1 --2- 172.21.15.33:0/732580074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85ba7fc700 1 -- 172.21.15.33:0/732580074 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209062758 0 0) 0x7f85bc15d660 con 0x7f85bc15a650 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85ba7fc700 1 -- 172.21.15.33:0/732580074 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a4003570 con 0x7f85bc15a650 2023-10-06T05:15:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85bb7fe700 1 -- 172.21.15.33:0/732580074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 msgr2=0x7f85bc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85bb7fe700 1 --2- 172.21.15.33:0/732580074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85bb7fe700 1 -- 172.21.15.33:0/732580074 >> v1:172.21.15.33:6789/0 conn(0x7f85bc15a650 legacy=0x7f85bc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85bb7fe700 1 -- 172.21.15.33:0/732580074 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc15d890 con 0x7f85bc0af540 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.421+0000 7f85bb7fe700 1 --2- 172.21.15.33:0/732580074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0af540 0x7f85bc0af910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098e0 tx=0x7f85ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a276b369b4218cd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85ba7fc700 1 -- 172.21.15.33:0/732580074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac011630 con 0x7f85bc0af540 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85ba7fc700 1 -- 172.21.15.33:0/732580074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac011790 con 0x7f85bc0af540 2023-10-06T05:15:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85ba7fc700 1 -- 172.21.15.33:0/732580074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac011b30 con 0x7f85bc0af540 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 -- 172.21.15.33:0/732580074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0af540 msgr2=0x7f85bc0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 --2- 172.21.15.33:0/732580074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0af540 0x7f85bc0af910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098e0 tx=0x7f85ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 -- 172.21.15.33:0/732580074 shutdown_connections 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 --2- 172.21.15.33:0/732580074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc0af540 0x7f85bc0af910 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 --2- 172.21.15.33:0/732580074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 -- 172.21.15.33:0/732580074 >> 172.21.15.33:0/732580074 conn(0x7f85bc06d9b0 msgr2=0x7f85bc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 -- 172.21.15.33:0/732580074 shutdown_connections 2023-10-06T05:15:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.422+0000 7f85c099d700 1 -- 172.21.15.33:0/732580074 wait complete. 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.424+0000 7f85c099d700 1 Processor -- start 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.424+0000 7f85c099d700 1 -- start start 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85c099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85bc0af540 0x7f85bc144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85c099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc0b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85c099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 0x7f85bc158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85c099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0c86d0 con 0x7f85bc15a650 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85c099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0c7ac0 con 0x7f85bc0af540 2023-10-06T05:15:40.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85c099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0c7da0 con 0x7f85bc0cd5e0 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 0x7f85bc158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 0x7f85bc158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41392/0 (socket says 172.21.15.33:41392) 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85bbfff700 1 -- 172.21.15.33:0/3348343690 learned_addr learned my addr 172.21.15.33:0/3348343690 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85bbfff700 1 -- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 msgr2=0x7f85bc0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.425+0000 7f85bbfff700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc0b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85bbfff700 1 -- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85bc0af540 msgr2=0x7f85bc144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85bbfff700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85bc0af540 0x7f85bc144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85bbfff700 1 -- 172.21.15.33:0/3348343690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac009420 con 0x7f85bc15a650 2023-10-06T05:15:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85bbfff700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 0x7f85bc158b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009e90 tx=0x7f85b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400cc20 con 0x7f85bc15a650 2023-10-06T05:15:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b400f040 con 0x7f85bc15a650 2023-10-06T05:15:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4014480 con 0x7f85bc15a650 2023-10-06T05:15:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc0c8080 con 0x7f85bc15a650 2023-10-06T05:15:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.426+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc145020 con 0x7f85bc15a650 2023-10-06T05:15:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.427+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc04c380 con 0x7f85bc15a650 2023-10-06T05:15:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.428+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85b40147f0 con 0x7f85bc15a650 2023-10-06T05:15:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.429+0000 7f85a37fe700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f85a8066f30 0x7f85a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.430+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f85b40854c0 con 0x7f85bc15a650 2023-10-06T05:15:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.430+0000 7f85bb7fe700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f85a8066f30 0x7f85a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.431+0000 7f85bb7fe700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f85a8066f30 0x7f85a80693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0b2100 tx=0x7f85ac017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.434+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85b4055f90 con 0x7f85bc15a650 2023-10-06T05:15:40.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:40.680+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f85bc0c8880 con 0x7f85bc15a650 2023-10-06T05:15:41.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.210+0000 7f85a37fe700 1 -- 172.21.15.33:0/3348343690 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85bc0c8880 con 0x7f85bc15a650 2023-10-06T05:15:41.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f85a8066f30 msgr2=0x7f85a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f85a8066f30 0x7f85a80693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0b2100 tx=0x7f85ac017000 comp rx=0 tx=0).stop 2023-10-06T05:15:41.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 msgr2=0x7f85bc158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 0x7f85bc158b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009e90 tx=0x7f85b400b040 comp rx=0 tx=0).stop 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 shutdown_connections 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f85a8066f30 0x7f85a80693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85bc15a650 0x7f85bc158b80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85bc0af540 0x7f85bc144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 --2- 172.21.15.33:0/3348343690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85bc0cd5e0 0x7f85bc0b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 >> 172.21.15.33:0/3348343690 conn(0x7f85bc06d9b0 msgr2=0x7f85bc156c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 shutdown_connections 2023-10-06T05:15:41.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.213+0000 7f85c099d700 1 -- 172.21.15.33:0/3348343690 wait complete. 2023-10-06T05:15:41.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:15:41.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.424+0000 7fd69619d700 1 Processor -- start 2023-10-06T05:15:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.425+0000 7fd69619d700 1 -- start start 2023-10-06T05:15:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.426+0000 7fd69619d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd69006dfd0 0x7fd69015a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.426+0000 7fd69619d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69015d280 con 0x7fd6901597f0 2023-10-06T05:15:41.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.426+0000 7fd69619d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69015de00 con 0x7fd69006dc00 2023-10-06T05:15:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.426+0000 7fd69619d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69015e290 con 0x7fd69006dfd0 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.426+0000 7fd69499a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd6901597f0 0x7fd69015a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40426/0 (socket says 172.21.15.33:40426) 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.426+0000 7fd69499a700 1 -- 172.21.15.33:0/2624447566 learned_addr learned my addr 172.21.15.33:0/2624447566 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd69599c700 1 --2- 172.21.15.33:0/2624447566 >> v2:172.21.15.167:3300/0 conn(0x7fd69006dfd0 0x7fd69015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744881229 0 0) 0x7fd69015d280 con 0x7fd6901597f0 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd67c003570 con 0x7fd6901597f0 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668267363 0 0) 0x7fd69015de00 con 0x7fd69006dc00 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd69015d280 con 0x7fd69006dc00 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996083228 0 0) 0x7fd67c003570 con 0x7fd6901597f0 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 >> v2:172.21.15.167:3300/0 conn(0x7fd69006dfd0 msgr2=0x7fd69015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 --2- 172.21.15.33:0/2624447566 >> v2:172.21.15.167:3300/0 conn(0x7fd69006dfd0 0x7fd69015a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 >> v1:172.21.15.167:6789/0 conn(0x7fd69006dc00 legacy=0x7fd6901590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69015d4d0 con 0x7fd6901597f0 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fd680001470 con 0x7fd6901597f0 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 >> v1:172.21.15.33:6789/0 conn(0x7fd6901597f0 legacy=0x7fd69015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c003bf0 0x7fd67c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67c004810 0x7fd67c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67c0051c0 0x7fd67c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c00c080 con 0x7fd67c0051c0 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c00c1f0 con 0x7fd67c003bf0 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.427+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c00c3b0 con 0x7fd67c004810 2023-10-06T05:15:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67c0051c0 0x7fd67c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69519b700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c003bf0 0x7fd67c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69499a700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67c004810 0x7fd67c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 -- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67c004810 msgr2=0x7fd67c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd67c004810 0x7fd67c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 -- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c003bf0 msgr2=0x7fd67c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c003bf0 0x7fd67c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 -- 172.21.15.33:0/2624447566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c003570 con 0x7fd67c0051c0 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 -- 172.21.15.33:0/2624447566 reap_dead start 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd69599c700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67c0051c0 0x7fd67c00ba60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd68800e210 tx=0x7fd68800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835f1a4223efaf60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800eac0 con 0x7fd67c0051c0 2023-10-06T05:15:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800f040 con 0x7fd67c0051c0 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.428+0000 7fd687fff700 1 -- 172.21.15.33:0/2624447566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800ee70 con 0x7fd67c0051c0 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 -- 172.21.15.33:0/2624447566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67c0051c0 msgr2=0x7fd67c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67c0051c0 0x7fd67c00ba60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd68800e210 tx=0x7fd68800b040 comp rx=0 tx=0).stop 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 -- 172.21.15.33:0/2624447566 shutdown_connections 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 --2- 172.21.15.33:0/2624447566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd67c0051c0 0x7fd67c00ba60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 -- 172.21.15.33:0/2624447566 >> 172.21.15.33:0/2624447566 conn(0x7fd6900d0550 msgr2=0x7fd6900d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 -- 172.21.15.33:0/2624447566 shutdown_connections 2023-10-06T05:15:41.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.429+0000 7fd69619d700 1 -- 172.21.15.33:0/2624447566 wait complete. 2023-10-06T05:15:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.430+0000 7fd69619d700 1 Processor -- start 2023-10-06T05:15:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 -- start start 2023-10-06T05:15:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c0051c0 0x7fd6900c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 0x7fd6900b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd69016d460 0x7fd69016d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900b0be0 con 0x7fd6900b5540 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900c87f0 con 0x7fd67c0051c0 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.431+0000 7fd69619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900ca450 con 0x7fd69016d460 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 0x7fd6900b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 0x7fd6900b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41402/0 (socket says 172.21.15.33:41402) 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 -- 172.21.15.33:0/3059272498 learned_addr learned my addr 172.21.15.33:0/3059272498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69599c700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd69016d460 0x7fd69016d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69519b700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c0051c0 0x7fd6900c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 -- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd69016d460 msgr2=0x7fd69016d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd69016d460 0x7fd69016d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 -- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c0051c0 msgr2=0x7fd6900c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c0051c0 0x7fd6900c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 -- 172.21.15.33:0/3059272498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900b5ef0 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69519b700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c0051c0 0x7fd6900c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.432+0000 7fd69499a700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 0x7fd6900b59b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd6800073c0 tx=0x7fd680001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.433+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680018030 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.433+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd680001040 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.433+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68000fb00 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.433+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6900c9c10 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.433+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6900c9e90 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.434+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd690063ff0 con 0x7fd6900b5540 2023-10-06T05:15:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.435+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd68001b070 con 0x7fd6900b5540 2023-10-06T05:15:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.436+0000 7fd6857fa700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd674066f30 0x7fd6740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.436+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd680019070 con 0x7fd6900b5540 2023-10-06T05:15:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.437+0000 7fd69519b700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd674066f30 0x7fd6740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.437+0000 7fd69519b700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd674066f30 0x7fd6740693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000c00 tx=0x7fd68c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.440+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd680051570 con 0x7fd6900b5540 2023-10-06T05:15:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:41.642+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd69015d7b0 con 0x7fd6900b5540 2023-10-06T05:15:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.218+0000 7fd6857fa700 1 -- 172.21.15.33:0/3059272498 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd680054b90 con 0x7fd6900b5540 2023-10-06T05:15:42.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd674066f30 msgr2=0x7fd6740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd674066f30 0x7fd6740693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000c00 tx=0x7fd68c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:42.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 msgr2=0x7fd6900b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 0x7fd6900b59b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd6800073c0 tx=0x7fd680001df0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 shutdown_connections 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd674066f30 0x7fd6740693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd69016d460 0x7fd69016d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c0051c0 0x7fd6900c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 --2- 172.21.15.33:0/3059272498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6900b5540 0x7fd6900b59b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.222+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 >> 172.21.15.33:0/3059272498 conn(0x7fd6900d0550 msgr2=0x7fd6900d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.223+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 shutdown_connections 2023-10-06T05:15:42.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.223+0000 7fd69619d700 1 -- 172.21.15.33:0/3059272498 wait complete. 2023-10-06T05:15:42.227 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:15:42.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:15:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.440+0000 7f8b3359e700 1 Processor -- start 2023-10-06T05:15:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.441+0000 7f8b3359e700 1 -- start start 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b3359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b340ceaf0 0x7f8b3415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b3359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340ad620 con 0x7f8b34154f60 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b3359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340adab0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b3359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340cac70 con 0x7f8b340ceaf0 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b31d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b340ceaf0 0x7f8b3415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b31d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b340ceaf0 0x7f8b3415b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41406/0 (socket says 172.21.15.33:41406) 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b3259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b34154f60 0x7f8b3415b0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40434/0 (socket says 172.21.15.33:40434) 2023-10-06T05:15:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.442+0000 7f8b31d9b700 1 -- 172.21.15.33:0/549648720 learned_addr learned my addr 172.21.15.33:0/549648720 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b31d9b700 1 -- 172.21.15.33:0/549648720 >> v1:172.21.15.167:6789/0 conn(0x7f8b3415bcf0 legacy=0x7f8b340adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b31d9b700 1 -- 172.21.15.33:0/549648720 >> v1:172.21.15.33:6789/0 conn(0x7f8b34154f60 legacy=0x7f8b3415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b31d9b700 1 -- 172.21.15.33:0/549648720 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340aca50 con 0x7f8b340ceaf0 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b3159a700 1 -- 172.21.15.33:0/549648720 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177774691 0 0) 0x7f8b340ad620 con 0x7f8b34154f60 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/549648720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340ceaf0 0x7f8b3415b7b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b280099b0 tx=0x7f8b28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d698efa8cadc2e77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b3159a700 1 -- 172.21.15.33:0/549648720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800b860 con 0x7f8b340ceaf0 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b3159a700 1 -- 172.21.15.33:0/549648720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800b9c0 con 0x7f8b340ceaf0 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.443+0000 7f8b3159a700 1 -- 172.21.15.33:0/549648720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bd60 con 0x7f8b340ceaf0 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 -- 172.21.15.33:0/549648720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340ceaf0 msgr2=0x7f8b3415b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 --2- 172.21.15.33:0/549648720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340ceaf0 0x7f8b3415b7b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b280099b0 tx=0x7f8b28009250 comp rx=0 tx=0).stop 2023-10-06T05:15:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 -- 172.21.15.33:0/549648720 shutdown_connections 2023-10-06T05:15:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 --2- 172.21.15.33:0/549648720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b340ceaf0 0x7f8b3415b7b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 -- 172.21.15.33:0/549648720 >> 172.21.15.33:0/549648720 conn(0x7f8b3406cd40 msgr2=0x7f8b340ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 -- 172.21.15.33:0/549648720 shutdown_connections 2023-10-06T05:15:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.444+0000 7f8b3359e700 1 -- 172.21.15.33:0/549648720 wait complete. 2023-10-06T05:15:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.445+0000 7f8b3359e700 1 Processor -- start 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.446+0000 7f8b3359e700 1 -- start start 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.446+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b340ceaf0 0x7f8b3414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.446+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b34154f60 0x7f8b340c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b3359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 0x7f8b341517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b3359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b2e40 con 0x7f8b3415bcf0 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b3359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c9b00 con 0x7f8b34154f60 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c9de0 con 0x7f8b340ceaf0 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 0x7f8b341517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 0x7f8b341517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41408/0 (socket says 172.21.15.33:41408) 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 -- 172.21.15.33:0/3513402810 learned_addr learned my addr 172.21.15.33:0/3513402810 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b31d9b700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b34154f60 0x7f8b340c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b3259c700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b340ceaf0 0x7f8b3414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 -- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b340ceaf0 msgr2=0x7f8b3414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b340ceaf0 0x7f8b3414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 -- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b34154f60 msgr2=0x7f8b340c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b34154f60 0x7f8b340c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b32d9d700 1 -- 172.21.15.33:0/3513402810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280094f0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.447+0000 7f8b3259c700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b340ceaf0 0x7f8b3414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.448+0000 7f8b32d9d700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 0x7f8b341517f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c810 tx=0x7f8b2c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.448+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c012630 con 0x7f8b3415bcf0 2023-10-06T05:15:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.448+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c012790 con 0x7f8b3415bcf0 2023-10-06T05:15:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.448+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c012b50 con 0x7f8b3415bcf0 2023-10-06T05:15:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.448+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b340ca0c0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.448+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340c97f0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.449+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b340b2fb0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.450+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b2c003030 con 0x7f8b3415bcf0 2023-10-06T05:15:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.451+0000 7f8b227fc700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.451+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8b2c0844e0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.451+0000 7f8b3259c700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.452+0000 7f8b3259c700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b240011e0 tx=0x7f8b24007c70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.455+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b340b2fb0 con 0x7f8b3415bcf0 2023-10-06T05:15:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:42.657+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b340b2f80 con 0x7f8b3415bcf0 2023-10-06T05:15:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.222+0000 7f8b227fc700 1 -- 172.21.15.33:0/3513402810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b2c055140 con 0x7f8b3415bcf0 2023-10-06T05:15:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.229+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 msgr2=0x7f8b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.229+0000 7f8b3359e700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b240011e0 tx=0x7f8b24007c70 comp rx=0 tx=0).stop 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.229+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 msgr2=0x7f8b341517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.229+0000 7f8b3359e700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 0x7f8b341517f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c810 tx=0x7f8b2c001650 comp rx=0 tx=0).stop 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.230+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 shutdown_connections 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.230+0000 7f8b3359e700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b14066f30 0x7f8b140693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.230+0000 7f8b3359e700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b3415bcf0 0x7f8b341517f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.230+0000 7f8b3359e700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b340ceaf0 0x7f8b3414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.230+0000 7f8b3359e700 1 --2- 172.21.15.33:0/3513402810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b34154f60 0x7f8b340c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.230+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 >> 172.21.15.33:0/3513402810 conn(0x7f8b3406cd40 msgr2=0x7f8b34152980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.231+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 shutdown_connections 2023-10-06T05:15:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.232+0000 7f8b3359e700 1 -- 172.21.15.33:0/3513402810 wait complete. 2023-10-06T05:15:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:15:43.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:15:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.444+0000 7fa28e19d700 1 Processor -- start 2023-10-06T05:15:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.444+0000 7fa28e19d700 1 -- start start 2023-10-06T05:15:43.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.445+0000 7fa28e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa2880af540 0x7fa2880af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.445+0000 7fa28e19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa28815a650 0x7fa28815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2880cced0 0x7fa2880cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28e19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2880cd890 con 0x7fa28815a650 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28815d660 con 0x7fa2880af540 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880ce890 con 0x7fa2880cced0 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa28815a650 0x7fa28815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa28815a650 0x7fa28815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41412/0 (socket says 172.21.15.33:41412) 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 -- 172.21.15.33:0/2162511463 learned_addr learned my addr 172.21.15.33:0/2162511463 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28d19b700 1 --2- 172.21.15.33:0/2162511463 >> v2:172.21.15.167:3300/0 conn(0x7fa2880af540 0x7fa2880af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28d99c700 1 --2- 172.21.15.33:0/2162511463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2880cced0 0x7fa2880cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 -- 172.21.15.33:0/2162511463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2880cced0 msgr2=0x7fa2880cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 --2- 172.21.15.33:0/2162511463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2880cced0 0x7fa2880cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 -- 172.21.15.33:0/2162511463 >> v2:172.21.15.167:3300/0 conn(0x7fa2880af540 msgr2=0x7fa2880af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 --2- 172.21.15.33:0/2162511463 >> v2:172.21.15.167:3300/0 conn(0x7fa2880af540 0x7fa2880af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28c99a700 1 -- 172.21.15.33:0/2162511463 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28815aac0 con 0x7fa28815a650 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.446+0000 7fa28d19b700 1 --2- 172.21.15.33:0/2162511463 >> v2:172.21.15.167:3300/0 conn(0x7fa2880af540 0x7fa2880af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28c99a700 1 --2- 172.21.15.33:0/2162511463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa28815a650 0x7fa28815d120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa2880c7530 tx=0x7fa278009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e634c81d8dc6d77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa27ffff700 1 -- 172.21.15.33:0/2162511463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27800b820 con 0x7fa28815a650 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa27ffff700 1 -- 172.21.15.33:0/2162511463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27800b980 con 0x7fa28815a650 2023-10-06T05:15:43.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa27ffff700 1 -- 172.21.15.33:0/2162511463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27800bd20 con 0x7fa28815a650 2023-10-06T05:15:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 -- 172.21.15.33:0/2162511463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa28815a650 msgr2=0x7fa28815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 --2- 172.21.15.33:0/2162511463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa28815a650 0x7fa28815d120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa2880c7530 tx=0x7fa278009250 comp rx=0 tx=0).stop 2023-10-06T05:15:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 -- 172.21.15.33:0/2162511463 shutdown_connections 2023-10-06T05:15:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 --2- 172.21.15.33:0/2162511463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2880cced0 0x7fa2880cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 --2- 172.21.15.33:0/2162511463 >> v2:172.21.15.167:3300/0 conn(0x7fa2880af540 0x7fa2880af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 --2- 172.21.15.33:0/2162511463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa28815a650 0x7fa28815d120 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.447+0000 7fa28e19d700 1 -- 172.21.15.33:0/2162511463 >> 172.21.15.33:0/2162511463 conn(0x7fa28806d9b0 msgr2=0x7fa28806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.448+0000 7fa28e19d700 1 -- 172.21.15.33:0/2162511463 shutdown_connections 2023-10-06T05:15:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.448+0000 7fa28e19d700 1 -- 172.21.15.33:0/2162511463 wait complete. 2023-10-06T05:15:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.449+0000 7fa28e19d700 1 Processor -- start 2023-10-06T05:15:43.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.449+0000 7fa28e19d700 1 -- start start 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 0x7fa2880c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2880cced0 0x7fa2880ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa28815a650 0x7fa28814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880b4790 con 0x7fa2880af540 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880ca7a0 con 0x7fa2880cced0 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880b4a50 con 0x7fa28815a650 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 0x7fa2880c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 0x7fa2880c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41418/0 (socket says 172.21.15.33:41418) 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.450+0000 7fa28d19b700 1 -- 172.21.15.33:0/465865960 learned_addr learned my addr 172.21.15.33:0/465865960 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28d19b700 1 -- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa28815a650 msgr2=0x7fa28814bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28d19b700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa28815a650 0x7fa28814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28d19b700 1 -- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2880cced0 msgr2=0x7fa2880ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28d19b700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2880cced0 0x7fa2880ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28d19b700 1 -- 172.21.15.33:0/465865960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780094f0 con 0x7fa2880af540 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28d19b700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 0x7fa2880c7580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa28400d860 tx=0x7fa284008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840125f0 con 0x7fa2880af540 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284012750 con 0x7fa2880af540 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284012ad0 con 0x7fa2880af540 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2880b4d30 con 0x7fa2880af540 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.451+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2880b52f0 con 0x7fa2880af540 2023-10-06T05:15:43.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.452+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2880b54e0 con 0x7fa2880af540 2023-10-06T05:15:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.453+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa284014430 con 0x7fa2880af540 2023-10-06T05:15:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.454+0000 7fa27d7fa700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa26c066f30 0x7fa26c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.454+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa2840848b0 con 0x7fa2880af540 2023-10-06T05:15:43.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.454+0000 7fa28c99a700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa26c066f30 0x7fa26c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.455+0000 7fa28c99a700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa26c066f30 0x7fa26c0693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2880c7530 tx=0x7fa278017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.458+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa284051c20 con 0x7fa2880af540 2023-10-06T05:15:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.665+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa28804c380 con 0x7fa2880af540 2023-10-06T05:15:43.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.668+0000 7fa27d7fa700 1 -- 172.21.15.33:0/465865960 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa284055240 con 0x7fa2880af540 2023-10-06T05:15:43.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa26c066f30 msgr2=0x7fa26c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa26c066f30 0x7fa26c0693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2880c7530 tx=0x7fa278017000 comp rx=0 tx=0).stop 2023-10-06T05:15:43.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 msgr2=0x7fa2880c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 0x7fa2880c7580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa28400d860 tx=0x7fa284008040 comp rx=0 tx=0).stop 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 shutdown_connections 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa26c066f30 0x7fa26c0693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa28815a650 0x7fa28814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2880af540 0x7fa2880c7580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 --2- 172.21.15.33:0/465865960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2880cced0 0x7fa2880ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.670+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 >> 172.21.15.33:0/465865960 conn(0x7fa28806d9b0 msgr2=0x7fa288157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.671+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 shutdown_connections 2023-10-06T05:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.671+0000 7fa28e19d700 1 -- 172.21.15.33:0/465865960 wait complete. 2023-10-06T05:15:43.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn' 2023-10-06T05:15:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.874+0000 7f564939d700 1 Processor -- start 2023-10-06T05:15:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.875+0000 7f564939d700 1 -- start start 2023-10-06T05:15:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.876+0000 7f564939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f56440c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.876+0000 7f564939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564406c820 con 0x7f5644159520 2023-10-06T05:15:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.876+0000 7f564939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564406a160 con 0x7f56440c80d0 2023-10-06T05:15:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.876+0000 7f564939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440c8800 con 0x7f56440c6340 2023-10-06T05:15:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.876+0000 7f5643fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5644159520 0x7f56440c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40436/0 (socket says 172.21.15.33:40436) 2023-10-06T05:15:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.876+0000 7f5643fff700 1 -- 172.21.15.33:0/3537024518 learned_addr learned my addr 172.21.15.33:0/3537024518 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f56437fe700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f56440c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247434972 0 0) 0x7f564406c820 con 0x7f5644159520 2023-10-06T05:15:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f562c003570 con 0x7f5644159520 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 162863259 0 0) 0x7f562c003570 con 0x7f5644159520 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 msgr2=0x7f56440c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f56440c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 >> v1:172.21.15.167:6789/0 conn(0x7f56440c80d0 legacy=0x7f564406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440c8ae0 con 0x7f5644159520 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f5634001470 con 0x7f5644159520 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 >> v1:172.21.15.33:6789/0 conn(0x7f5644159520 legacy=0x7f56440c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f562c003a90 0x7f562c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562c004810 0x7f562c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f562c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440c8800 con 0x7f562c003a90 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c00bbf0 con 0x7f562c004810 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c00bd80 con 0x7f56440c6340 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.877+0000 7f5643fff700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f562c003a90 0x7f562c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f56437fe700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562c004810 0x7f562c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5648b9c700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f562c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5643fff700 1 -- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 msgr2=0x7f562c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5643fff700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f562c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5643fff700 1 -- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562c004810 msgr2=0x7f562c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5643fff700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562c004810 0x7f562c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5643fff700 1 -- 172.21.15.33:0/3537024518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c003570 con 0x7f562c003a90 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5648b9c700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56440c6340 0x7f562c005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5643fff700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f562c003a90 0x7f562c003ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f563400c300 tx=0x7f5634001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4282557decd379 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400cb90 con 0x7f562c003a90 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563400ccf0 con 0x7f562c003a90 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.878+0000 7f5642ffd700 1 -- 172.21.15.33:0/3537024518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634005b70 con 0x7f562c003a90 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f564939d700 1 -- 172.21.15.33:0/3537024518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f562c003a90 msgr2=0x7f562c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f564939d700 1 --2- 172.21.15.33:0/3537024518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f562c003a90 0x7f562c003ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f563400c300 tx=0x7f5634001d80 comp rx=0 tx=0).stop 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f5648b9c700 1 -- 172.21.15.33:0/3537024518 reap_dead start 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f564939d700 1 -- 172.21.15.33:0/3537024518 shutdown_connections 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f564939d700 1 -- 172.21.15.33:0/3537024518 >> 172.21.15.33:0/3537024518 conn(0x7f56440cc930 msgr2=0x7f56440ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f564939d700 1 -- 172.21.15.33:0/3537024518 shutdown_connections 2023-10-06T05:15:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.879+0000 7f564939d700 1 -- 172.21.15.33:0/3537024518 wait complete. 2023-10-06T05:15:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.880+0000 7f564939d700 1 Processor -- start 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 -- start start 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562c003a90 0x7f564415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5644154390 0x7f564415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 0x7f56440ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440cae10 con 0x7f56440ca460 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564415bf80 con 0x7f5644154390 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.881+0000 7f564939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5644154b70 con 0x7f562c003a90 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 0x7f56440ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 0x7f56440ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41444/0 (socket says 172.21.15.33:41444) 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 -- 172.21.15.33:0/1004717920 learned_addr learned my addr 172.21.15.33:0/1004717920 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f56437fe700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5644154390 0x7f564415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5643fff700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562c003a90 0x7f564415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 -- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562c003a90 msgr2=0x7f564415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562c003a90 0x7f564415c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 -- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5644154390 msgr2=0x7f564415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5644154390 0x7f564415ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 -- 172.21.15.33:0/1004717920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644154e50 con 0x7f56440ca460 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.882+0000 7f5648b9c700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 0x7f56440ca8d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5644063690 tx=0x7f563c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.883+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c012590 con 0x7f56440ca460 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.883+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c012710 con 0x7f56440ca460 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.883+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c012ad0 con 0x7f56440ca460 2023-10-06T05:15:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.883+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5644155130 con 0x7f56440ca460 2023-10-06T05:15:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.883+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5644159bd0 con 0x7f56440ca460 2023-10-06T05:15:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.884+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5644159df0 con 0x7f56440ca460 2023-10-06T05:15:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.885+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f563c003030 con 0x7f56440ca460 2023-10-06T05:15:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.886+0000 7f562bfff700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5630066f30 0x7f56300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.886+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f563c0841c0 con 0x7f56440ca460 2023-10-06T05:15:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.887+0000 7f5643fff700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5630066f30 0x7f56300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.887+0000 7f5643fff700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5630066f30 0x7f56300693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f563400c2a0 tx=0x7f5634015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:43.891+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f563c0515d0 con 0x7f56440ca460 2023-10-06T05:15:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.171+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]} v 0) v1 -- 0x7f564404c380 con 0x7f56440ca460 2023-10-06T05:15:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.233+0000 7f562bfff700 1 -- 172.21.15.33:0/1004717920 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_abort_conn"]}]=0 v101) v1 ==== 147+0+0 (secure 0 0 0) 0x7f563c054bf0 con 0x7f56440ca460 2023-10-06T05:15:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5630066f30 msgr2=0x7f56300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5630066f30 0x7f56300693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f563400c2a0 tx=0x7f5634015000 comp rx=0 tx=0).stop 2023-10-06T05:15:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 msgr2=0x7f56440ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 0x7f56440ca8d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5644063690 tx=0x7f563c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 shutdown_connections 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5630066f30 0x7f56300693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56440ca460 0x7f56440ca8d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562c003a90 0x7f564415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 --2- 172.21.15.33:0/1004717920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5644154390 0x7f564415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 >> 172.21.15.33:0/1004717920 conn(0x7f56440cc930 msgr2=0x7f56440cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 shutdown_connections 2023-10-06T05:15:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.235+0000 7f564939d700 1 -- 172.21.15.33:0/1004717920 wait complete. 2023-10-06T05:15:44.255 INFO:tasks.cephfs_test_runner:Check that abort_conn() skips closing mds sessions. ... skipped 'Testing libcephfs function' 2023-10-06T05:15:44.255 INFO:tasks.cephfs_test_runner:Starting test: test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:15:44.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic' 2023-10-06T05:15:44.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.445+0000 7fc71219d700 1 Processor -- start 2023-10-06T05:15:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.445+0000 7fc71219d700 1 -- start start 2023-10-06T05:15:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc70c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71219d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc70c0cc750 0x7fc70c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71219d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c0cbd80 con 0x7fc70c0cf3b0 2023-10-06T05:15:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71219d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c0af5f0 con 0x7fc70c0cc750 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0af8d0 con 0x7fc70c0ccfc0 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71099a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc70c0cf3b0 0x7fc70c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40440/0 (socket says 172.21.15.33:40440) 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.446+0000 7fc71099a700 1 -- 172.21.15.33:0/2866914520 learned_addr learned my addr 172.21.15.33:0/2866914520 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc71119b700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc70c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc71199c700 1 --2- 172.21.15.33:0/2866914520 >> v2:172.21.15.167:3300/0 conn(0x7fc70c0cc750 0x7fc70c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818388071 0 0) 0x7fc70c0cbd80 con 0x7fc70c0cf3b0 2023-10-06T05:15:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f8003570 con 0x7fc70c0cf3b0 2023-10-06T05:15:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4122706094 0 0) 0x7fc6f8003570 con 0x7fc70c0cf3b0 2023-10-06T05:15:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 msgr2=0x7fc70c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc70c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 >> v2:172.21.15.167:3300/0 conn(0x7fc70c0cc750 msgr2=0x7fc70c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 --2- 172.21.15.33:0/2866914520 >> v2:172.21.15.167:3300/0 conn(0x7fc70c0cc750 0x7fc70c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0afbb0 con 0x7fc70c0cf3b0 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc6fc001470 con 0x7fc70c0cf3b0 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 >> v1:172.21.15.33:6789/0 conn(0x7fc70c0cf3b0 legacy=0x7fc70c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f8003ae0 0x7fc6f8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc6f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0af5f0 con 0x7fc6f8003ae0 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0af8d0 con 0x7fc6f8004d50 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.447+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f800bb10 con 0x7fc70c0ccfc0 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f8003ae0 0x7fc6f8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71099a700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc6f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71199c700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 -- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 msgr2=0x7fc6f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc6f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 -- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f8004d50 msgr2=0x7fc6f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 -- 172.21.15.33:0/2866914520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8003570 con 0x7fc6f8003ae0 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71099a700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c0ccfc0 0x7fc6f8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71199c700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f8004d50 0x7fc6f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.448+0000 7fc71119b700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f8003ae0 0x7fc6f8003ef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc708009c30 tx=0x7fc7080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c698d29d9c10397 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800e710 con 0x7fc6f8003ae0 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70800e870 con 0x7fc6f8003ae0 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc703fff700 1 -- 172.21.15.33:0/2866914520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800ec30 con 0x7fc6f8003ae0 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71219d700 1 -- 172.21.15.33:0/2866914520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f8003ae0 msgr2=0x7fc6f8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71219d700 1 --2- 172.21.15.33:0/2866914520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6f8003ae0 0x7fc6f8003ef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc708009c30 tx=0x7fc7080091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71199c700 1 -- 172.21.15.33:0/2866914520 reap_dead start 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71219d700 1 -- 172.21.15.33:0/2866914520 shutdown_connections 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71219d700 1 -- 172.21.15.33:0/2866914520 >> 172.21.15.33:0/2866914520 conn(0x7fc70c06cff0 msgr2=0x7fc70c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71219d700 1 -- 172.21.15.33:0/2866914520 shutdown_connections 2023-10-06T05:15:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.449+0000 7fc71219d700 1 -- 172.21.15.33:0/2866914520 wait complete. 2023-10-06T05:15:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.451+0000 7fc71219d700 1 Processor -- start 2023-10-06T05:15:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.451+0000 7fc71219d700 1 -- start start 2023-10-06T05:15:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6f8003ae0 0x7fc70c0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 0x7fc70c14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0b32a0 0x7fc70c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0b47b0 con 0x7fc70c0ca170 2023-10-06T05:15:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0b5180 con 0x7fc70c0b32a0 2023-10-06T05:15:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0b3ba0 con 0x7fc6f8003ae0 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 0x7fc70c14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 0x7fc70c14f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41464/0 (socket says 172.21.15.33:41464) 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 -- 172.21.15.33:0/121519873 learned_addr learned my addr 172.21.15.33:0/121519873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71199c700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0b32a0 0x7fc70c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71119b700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6f8003ae0 0x7fc70c0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 -- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6f8003ae0 msgr2=0x7fc70c0b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6f8003ae0 0x7fc70c0b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 -- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0b32a0 msgr2=0x7fc70c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0b32a0 0x7fc70c0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.452+0000 7fc71099a700 1 -- 172.21.15.33:0/121519873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0b3e80 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.453+0000 7fc71099a700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 0x7fc70c14f3d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00eb90 tx=0x7fc6fc0015a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.453+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00c550 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.453+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc00c6b0 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.453+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c0b4160 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.453+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00ca70 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.453+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c14f910 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.454+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c0b4920 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.455+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6fc01d460 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.456+0000 7fc7017fa700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6f0066f30 0x7fc6f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.456+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc6fc0842e0 con 0x7fc70c0ca170 2023-10-06T05:15:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.456+0000 7fc71119b700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6f0066f30 0x7fc6f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.457+0000 7fc71119b700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6f0066f30 0x7fc6f00693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7080091e0 tx=0x7fc70801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.460+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6fc0516f0 con 0x7fc70c0ca170 2023-10-06T05:15:44.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:44.745+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]} v 0) v1 -- 0x7fc70c04c380 con 0x7fc70c0ca170 2023-10-06T05:15:45.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.235+0000 7fc7017fa700 1 -- 172.21.15.33:0/121519873 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]}]=0 v102) v1 ==== 145+0+0 (secure 0 0 0) 0x7fc6fc054d10 con 0x7fc70c0ca170 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6f0066f30 msgr2=0x7fc6f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6f0066f30 0x7fc6f00693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7080091e0 tx=0x7fc70801a000 comp rx=0 tx=0).stop 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 msgr2=0x7fc70c14f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 0x7fc70c14f3d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00eb90 tx=0x7fc6fc0015a0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 shutdown_connections 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc6f0066f30 0x7fc6f00693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0b32a0 0x7fc70c0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6f8003ae0 0x7fc70c0b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 --2- 172.21.15.33:0/121519873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c0ca170 0x7fc70c14f3d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.237+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 >> 172.21.15.33:0/121519873 conn(0x7fc70c06cff0 msgr2=0x7fc70c156730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.238+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 shutdown_connections 2023-10-06T05:15:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.238+0000 7fc71219d700 1 -- 172.21.15.33:0/121519873 wait complete. 2023-10-06T05:15:45.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:15:45.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.449+0000 7fedcab9d700 1 Processor -- start 2023-10-06T05:15:45.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.449+0000 7fedcab9d700 1 -- start start 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedcab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fedc4156370 0x7fedc40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedcab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40ad430 con 0x7fedc40cc7e0 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedcab9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40afe20 con 0x7fedc40b01f0 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedcab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc406bc70 con 0x7fedc4156370 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedc9b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fedc4156370 0x7fedc40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedc939a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fedc40cc7e0 0x7fedc40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40444/0 (socket says 172.21.15.33:40444) 2023-10-06T05:15:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedc939a700 1 -- 172.21.15.33:0/2278907210 learned_addr learned my addr 172.21.15.33:0/2278907210 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedc9b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fedc4156370 0x7fedc40af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41476/0 (socket says 172.21.15.33:41476) 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.451+0000 7fedc9b9b700 1 -- 172.21.15.33:0/2278907210 >> v1:172.21.15.167:6789/0 conn(0x7fedc40b01f0 legacy=0x7fedc40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc9b9b700 1 -- 172.21.15.33:0/2278907210 >> v1:172.21.15.33:6789/0 conn(0x7fedc40cc7e0 legacy=0x7fedc40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc9b9b700 1 -- 172.21.15.33:0/2278907210 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc40cccc0 con 0x7fedc4156370 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc8b99700 1 -- 172.21.15.33:0/2278907210 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134672215 0 0) 0x7fedc40ad430 con 0x7fedc40cc7e0 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc9b9b700 1 --2- 172.21.15.33:0/2278907210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc40af8e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fedc000bb90 tx=0x7fedc000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d5f457a1e31778 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc8b99700 1 -- 172.21.15.33:0/2278907210 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc000db60 con 0x7fedc4156370 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc8b99700 1 -- 172.21.15.33:0/2278907210 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc000dcc0 con 0x7fedc4156370 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.452+0000 7fedc8b99700 1 -- 172.21.15.33:0/2278907210 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00106a0 con 0x7fedc4156370 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 -- 172.21.15.33:0/2278907210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 msgr2=0x7fedc40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 --2- 172.21.15.33:0/2278907210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc40af8e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fedc000bb90 tx=0x7fedc000b430 comp rx=0 tx=0).stop 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 -- 172.21.15.33:0/2278907210 shutdown_connections 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 --2- 172.21.15.33:0/2278907210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc40af8e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 -- 172.21.15.33:0/2278907210 >> 172.21.15.33:0/2278907210 conn(0x7fedc406d030 msgr2=0x7fedc40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 -- 172.21.15.33:0/2278907210 shutdown_connections 2023-10-06T05:15:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.453+0000 7fedcab9d700 1 -- 172.21.15.33:0/2278907210 wait complete. 2023-10-06T05:15:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.454+0000 7fedcab9d700 1 Processor -- start 2023-10-06T05:15:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 -- start start 2023-10-06T05:15:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc40b01f0 0x7fedc4151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc40cc7e0 0x7fedc4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc414fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4153200 con 0x7fedc4156370 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4153fc0 con 0x7fedc40b01f0 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.455+0000 7fedcab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4153440 con 0x7fedc40cc7e0 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc414fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc414fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41482/0 (socket says 172.21.15.33:41482) 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 -- 172.21.15.33:0/3274568331 learned_addr learned my addr 172.21.15.33:0/3274568331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedc9b9b700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc40b01f0 0x7fedc4151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedc939a700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc40cc7e0 0x7fedc4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 -- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc40cc7e0 msgr2=0x7fedc4152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc40cc7e0 0x7fedc4152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 -- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc40b01f0 msgr2=0x7fedc4151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc40b01f0 0x7fedc4151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.456+0000 7fedca39c700 1 -- 172.21.15.33:0/3274568331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc000b6d0 con 0x7fedc4156370 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.457+0000 7fedca39c700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc414fe40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00bc30 tx=0x7fedbc009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.457+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00c840 con 0x7fedc4156370 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.457+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc00c9e0 con 0x7fedc4156370 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.457+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc4153720 con 0x7fedc4156370 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.457+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00cda0 con 0x7fedc4156370 2023-10-06T05:15:45.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.457+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc4153ce0 con 0x7fedc4156370 2023-10-06T05:15:45.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.458+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc404c380 con 0x7fedc4156370 2023-10-06T05:15:45.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.459+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedbc014620 con 0x7fedc4156370 2023-10-06T05:15:45.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.460+0000 7fedb9ffb700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feda8066f30 0x7feda80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.460+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fedbc0842d0 con 0x7fedc4156370 2023-10-06T05:15:45.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.460+0000 7fedc9b9b700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feda8066f30 0x7feda80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.461+0000 7fedc9b9b700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feda8066f30 0x7feda80693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b2c0 tx=0x7fedc001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.464+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedbc04c5a0 con 0x7fedc4156370 2023-10-06T05:15:45.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.669+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fedc406f310 con 0x7fedc4156370 2023-10-06T05:15:45.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.673+0000 7fedb9ffb700 1 -- 172.21.15.33:0/3274568331 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedbc0516e0 con 0x7fedc4156370 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feda8066f30 msgr2=0x7feda80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feda8066f30 0x7feda80693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b2c0 tx=0x7fedc001a000 comp rx=0 tx=0).stop 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 msgr2=0x7fedc414fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc414fe40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00bc30 tx=0x7fedbc009820 comp rx=0 tx=0).stop 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 shutdown_connections 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feda8066f30 0x7feda80693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc4156370 0x7fedc414fe40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc40b01f0 0x7fedc4151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 --2- 172.21.15.33:0/3274568331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc40cc7e0 0x7fedc4152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 >> 172.21.15.33:0/3274568331 conn(0x7fedc406d030 msgr2=0x7fedc40cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 shutdown_connections 2023-10-06T05:15:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:45.676+0000 7fedcab9d700 1 -- 172.21.15.33:0/3274568331 wait complete. 2023-10-06T05:15:45.696 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:15:45.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:45.776 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:45.866 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:15:45.866 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:15:45.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 Processor -- start 2023-10-06T05:15:46.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 -- start start 2023-10-06T05:15:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f91640ccfc0 0x7f91640cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640cf3b0 0x7f91641594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91640cbd80 con 0x7f91640cc750 2023-10-06T05:15:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91640af5f0 con 0x7f91640ccfc0 2023-10-06T05:15:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.052+0000 7f916b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91640af8d0 con 0x7f91640cf3b0 2023-10-06T05:15:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f9169d9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640cf3b0 0x7f91641594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f916ad9c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f91640cc750 0x7f91640b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:41284/0 (socket says 172.21.15.33:41284) 2023-10-06T05:15:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f916a59b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f91640ccfc0 0x7f91640cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f916ad9c700 1 -- 172.21.15.33:0/3024960107 learned_addr learned my addr 172.21.15.33:0/3024960107 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f9169d9a700 1 -- 172.21.15.33:0/3024960107 >> v2:172.21.15.167:3300/0 conn(0x7f91640ccfc0 msgr2=0x7f91640cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f9169d9a700 1 --2- 172.21.15.33:0/3024960107 >> v2:172.21.15.167:3300/0 conn(0x7f91640ccfc0 0x7f91640cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f9169d9a700 1 -- 172.21.15.33:0/3024960107 >> v1:172.21.15.167:6789/0 conn(0x7f91640cc750 legacy=0x7f91640b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f9169d9a700 1 -- 172.21.15.33:0/3024960107 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640afbb0 con 0x7f91640cf3b0 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f916a59b700 1 --2- 172.21.15.33:0/3024960107 >> v2:172.21.15.167:3300/0 conn(0x7f91640ccfc0 0x7f91640cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.053+0000 7f9169599700 1 -- 172.21.15.33:0/3024960107 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734205380 0 0) 0x7f91640cbd80 con 0x7f91640cc750 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.054+0000 7f9169d9a700 1 --2- 172.21.15.33:0/3024960107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640cf3b0 0x7f91641594a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91540098e0 tx=0x7f91540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b7210954d76b01d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.054+0000 7f9169599700 1 -- 172.21.15.33:0/3024960107 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400b640 con 0x7f91640cf3b0 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.054+0000 7f9169599700 1 -- 172.21.15.33:0/3024960107 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915400b7a0 con 0x7f91640cf3b0 2023-10-06T05:15:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.054+0000 7f9169599700 1 -- 172.21.15.33:0/3024960107 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400bb40 con 0x7f91640cf3b0 2023-10-06T05:15:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.054+0000 7f916b59d700 1 -- 172.21.15.33:0/3024960107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640cf3b0 msgr2=0x7f91641594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 --2- 172.21.15.33:0/3024960107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640cf3b0 0x7f91641594a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91540098e0 tx=0x7f91540091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 -- 172.21.15.33:0/3024960107 shutdown_connections 2023-10-06T05:15:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 --2- 172.21.15.33:0/3024960107 >> v2:172.21.15.167:3300/0 conn(0x7f91640ccfc0 0x7f91640cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 --2- 172.21.15.33:0/3024960107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640cf3b0 0x7f91641594a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 -- 172.21.15.33:0/3024960107 >> 172.21.15.33:0/3024960107 conn(0x7f916406cff0 msgr2=0x7f916406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 -- 172.21.15.33:0/3024960107 shutdown_connections 2023-10-06T05:15:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.055+0000 7f916b59d700 1 -- 172.21.15.33:0/3024960107 wait complete. 2023-10-06T05:15:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.056+0000 7f916b59d700 1 Processor -- start 2023-10-06T05:15:46.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.056+0000 7f916b59d700 1 -- start start 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 0x7f91640b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640ccfc0 0x7f916416ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91640b32e0 0x7f9164156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91640b4680 con 0x7f91640cc750 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91640b4f20 con 0x7f91640b32e0 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91640b61a0 con 0x7f91640ccfc0 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916a59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 0x7f91640b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.057+0000 7f916a59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 0x7f91640b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41486/0 (socket says 172.21.15.33:41486) 2023-10-06T05:15:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 -- 172.21.15.33:0/2070462034 learned_addr learned my addr 172.21.15.33:0/2070462034 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916ad9c700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91640b32e0 0x7f9164156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f9169d9a700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640ccfc0 0x7f916416ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 -- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640ccfc0 msgr2=0x7f916416ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640ccfc0 0x7f916416ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 -- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91640b32e0 msgr2=0x7f9164156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91640b32e0 0x7f9164156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 -- 172.21.15.33:0/2070462034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9160009420 con 0x7f91640cc750 2023-10-06T05:15:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f916a59b700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 0x7f91640b5bc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f916000ec80 tx=0x7f91600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.058+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000c920 con 0x7f91640cc750 2023-10-06T05:15:46.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.059+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916000ca80 con 0x7f91640cc750 2023-10-06T05:15:46.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.059+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9154009420 con 0x7f91640cc750 2023-10-06T05:15:46.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.059+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000ce00 con 0x7f91640cc750 2023-10-06T05:15:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.059+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91640b4bb0 con 0x7f91640cc750 2023-10-06T05:15:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.060+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916404c380 con 0x7f91640cc750 2023-10-06T05:15:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.061+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9160013680 con 0x7f91640cc750 2023-10-06T05:15:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.061+0000 7f915a7fc700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9148066f30 0x7f91480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.061+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9160084b40 con 0x7f91640cc750 2023-10-06T05:15:46.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.063+0000 7f9169d9a700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9148066f30 0x7f91480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.064+0000 7f9169d9a700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9148066f30 0x7f91480693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91540098e0 tx=0x7f9154017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.066+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9160051eb0 con 0x7f91640cc750 2023-10-06T05:15:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.314+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91640af0a0 con 0x7f91640cc750 2023-10-06T05:15:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.315+0000 7f915a7fc700 1 -- 172.21.15.33:0/2070462034 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f9160021de0 con 0x7f91640cc750 2023-10-06T05:15:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:46.318 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1004549078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1004549078},{"type":"v1","addr":"172.21.15.167:6837","nonce":1004549078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2188521896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2188521896},{"type":"v1","addr":"172.21.15.33:6835","nonce":2188521896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5013,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3991307909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3991307909},{"type":"v1","addr":"172.21.15.33:6834","nonce":3991307909}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5173,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4066079221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4066079221},{"type":"v1","addr":"172.21.15.167:6836","nonce":4066079221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-10-06T05:15:46.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9148066f30 msgr2=0x7f91480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9148066f30 0x7f91480693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91540098e0 tx=0x7f9154017000 comp rx=0 tx=0).stop 2023-10-06T05:15:46.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 msgr2=0x7f91640b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 0x7f91640b5bc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f916000ec80 tx=0x7f91600091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 shutdown_connections 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9148066f30 0x7f91480693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91640b32e0 0x7f9164156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91640cc750 0x7f91640b5bc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 --2- 172.21.15.33:0/2070462034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91640ccfc0 0x7f916416ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 >> 172.21.15.33:0/2070462034 conn(0x7f916406cff0 msgr2=0x7f9164157f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 shutdown_connections 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.318+0000 7f916b59d700 1 -- 172.21.15.33:0/2070462034 wait complete. 2023-10-06T05:15:46.321 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2023-10-06T05:15:46.337 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:15:46.338 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:15:46.338 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:46.338 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:15:46.338 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:15:46.338 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:46.339 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:15:46.339 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:15:46.339 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:46.339 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:15:46.339 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:15:46.340 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:15:46.351 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:15:46.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:15:46.352 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:15:46.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:15:46.356 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:15:46.356 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:15:46.356 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:15:46.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:15:46.396 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:15:46.396 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:46.399 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:15:46.400 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:46.400 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:15:46.401 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:46.403 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:15:46.403 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:46.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:15:46.532 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:15:46.539 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:15:46.542 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:15:46.542 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:15:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.601+0000 7fdea15ff700 1 Processor -- start 2023-10-06T05:15:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.601+0000 7fdea15ff700 1 -- start start 2023-10-06T05:15:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fdea15ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde9c15d610 0x7fde9c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fdea15ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fdea15ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c15aed0 con 0x7fde9c0cb1b0 2023-10-06T05:15:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fdea15ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c0cb680 con 0x7fde9c15d610 2023-10-06T05:15:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fdea15ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0cb960 con 0x7fde9c1509a0 2023-10-06T05:15:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde9c15d610 0x7fde9c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde9c15d610 0x7fde9c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41530/0 (socket says 172.21.15.33:41530) 2023-10-06T05:15:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9affd700 1 -- 172.21.15.33:0/3587447861 learned_addr learned my addr 172.21.15.33:0/3587447861 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/3587447861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9b7fe700 1 -- 172.21.15.33:0/3587447861 >> v2:172.21.15.33:3300/0 conn(0x7fde9c15d610 msgr2=0x7fde9c0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/3587447861 >> v2:172.21.15.33:3300/0 conn(0x7fde9c15d610 0x7fde9c0cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9b7fe700 1 -- 172.21.15.33:0/3587447861 >> v1:172.21.15.167:6789/0 conn(0x7fde9c0cb1b0 legacy=0x7fde9c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9b7fe700 1 -- 172.21.15.33:0/3587447861 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0cbc40 con 0x7fde9c1509a0 2023-10-06T05:15:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.602+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/3587447861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c15b380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde94009e60 tx=0x7fde94009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f8468c31df54c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fde99ffb700 1 -- 172.21.15.33:0/3587447861 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400e070 con 0x7fde9c1509a0 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fde99ffb700 1 -- 172.21.15.33:0/3587447861 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400bb60 con 0x7fde9c1509a0 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fde99ffb700 1 -- 172.21.15.33:0/3587447861 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94012430 con 0x7fde9c1509a0 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 -- 172.21.15.33:0/3587447861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 msgr2=0x7fde9c15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 --2- 172.21.15.33:0/3587447861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c15b380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde94009e60 tx=0x7fde94009780 comp rx=0 tx=0).stop 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 -- 172.21.15.33:0/3587447861 shutdown_connections 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 --2- 172.21.15.33:0/3587447861 >> v2:172.21.15.33:3300/0 conn(0x7fde9c15d610 0x7fde9c0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 --2- 172.21.15.33:0/3587447861 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c15b380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 -- 172.21.15.33:0/3587447861 >> 172.21.15.33:0/3587447861 conn(0x7fde9c099960 msgr2=0x7fde9c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 -- 172.21.15.33:0/3587447861 shutdown_connections 2023-10-06T05:15:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.603+0000 7fdea15ff700 1 -- 172.21.15.33:0/3587447861 wait complete. 2023-10-06T05:15:46.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.605+0000 7fdea15ff700 1 Processor -- start 2023-10-06T05:15:46.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.605+0000 7fdea15ff700 1 -- start start 2023-10-06T05:15:46.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0cb1b0 0x7fde9c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c0d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 0x7fde9c1699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c16cfa0 con 0x7fde9c15d610 2023-10-06T05:15:46.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c169ee0 con 0x7fde9c0cb1b0 2023-10-06T05:15:46.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c16a180 con 0x7fde9c1509a0 2023-10-06T05:15:46.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 0x7fde9c1699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 0x7fde9c1699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41538/0 (socket says 172.21.15.33:41538) 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 -- 172.21.15.33:0/2735157773 learned_addr learned my addr 172.21.15.33:0/2735157773 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9affd700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0cb1b0 0x7fde9c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9a7fc700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c0d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 -- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 msgr2=0x7fde9c0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c0d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 -- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0cb1b0 msgr2=0x7fde9c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0cb1b0 0x7fde9c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 -- 172.21.15.33:0/2735157773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94009a20 con 0x7fde9c15d610 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9a7fc700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c0d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 0x7fde9c1699a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0cbaa0 tx=0x7fde94019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94027470 con 0x7fde9c15d610 2023-10-06T05:15:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde940275f0 con 0x7fde9c15d610 2023-10-06T05:15:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde940279b0 con 0x7fde9c15d610 2023-10-06T05:15:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c16a460 con 0x7fde9c15d610 2023-10-06T05:15:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.607+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c16d1f0 con 0x7fde9c15d610 2023-10-06T05:15:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.608+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c16d410 con 0x7fde9c15d610 2023-10-06T05:15:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.609+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde94024070 con 0x7fde9c15d610 2023-10-06T05:15:46.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.610+0000 7fde82ffd700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde88066f30 0x7fde880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.610+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fde94086e50 con 0x7fde9c15d610 2023-10-06T05:15:46.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.611+0000 7fde9affd700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde88066f30 0x7fde880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.611+0000 7fde9affd700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde88066f30 0x7fde880693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde9c154090 tx=0x7fde8c009220 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.615+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde940541c0 con 0x7fde9c15d610 2023-10-06T05:15:46.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:46.842+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fde9c063ff0 con 0x7fde9c15d610 2023-10-06T05:15:47.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:47.367 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:47.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.691+0000 7fde82ffd700 1 -- 172.21.15.33:0/2735157773 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7fde940577e0 con 0x7fde9c15d610 2023-10-06T05:15:47.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde88066f30 msgr2=0x7fde880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde88066f30 0x7fde880693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde9c154090 tx=0x7fde8c009220 comp rx=0 tx=0).stop 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 msgr2=0x7fde9c1699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 0x7fde9c1699a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0cbaa0 tx=0x7fde94019000 comp rx=0 tx=0).stop 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 shutdown_connections 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde88066f30 0x7fde880693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c15d610 0x7fde9c1699a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0cb1b0 0x7fde9c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 --2- 172.21.15.33:0/2735157773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c1509a0 0x7fde9c0d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 >> 172.21.15.33:0/2735157773 conn(0x7fde9c099960 msgr2=0x7fde9c0b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 shutdown_connections 2023-10-06T05:15:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.694+0000 7fdea15ff700 1 -- 172.21.15.33:0/2735157773 wait complete. 2023-10-06T05:15:47.697 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:15:47.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:15:47.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.901+0000 7fa9da99d700 1 Processor -- start 2023-10-06T05:15:47.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.901+0000 7fa9da99d700 1 -- start start 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9da99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa9d40c8050 0x7fa9d413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9da99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40c9cb0 con 0x7fa9d413faa0 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9da99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40ca270 con 0x7fa9d406aad0 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9da99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d406b640 con 0x7fa9d40c8050 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9d999b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa9d413faa0 0x7fa9d40c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40452/0 (socket says 172.21.15.33:40452) 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9d999b700 1 -- 172.21.15.33:0/2830929804 learned_addr learned my addr 172.21.15.33:0/2830929804 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9d919a700 1 --2- 172.21.15.33:0/2830929804 >> v2:172.21.15.167:3300/0 conn(0x7fa9d40c8050 0x7fa9d413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643216512 0 0) 0x7fa9d40c9cb0 con 0x7fa9d413faa0 2023-10-06T05:15:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.903+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c0003570 con 0x7fa9d413faa0 2023-10-06T05:15:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1606840412 0 0) 0x7fa9c0003570 con 0x7fa9d413faa0 2023-10-06T05:15:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 >> v2:172.21.15.167:3300/0 conn(0x7fa9d40c8050 msgr2=0x7fa9d413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 --2- 172.21.15.33:0/2830929804 >> v2:172.21.15.167:3300/0 conn(0x7fa9d40c8050 0x7fa9d413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 >> v1:172.21.15.167:6789/0 conn(0x7fa9d406aad0 legacy=0x7fa9d406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d406b840 con 0x7fa9d413faa0 2023-10-06T05:15:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa9d0001470 con 0x7fa9d413faa0 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 >> v1:172.21.15.33:6789/0 conn(0x7fa9d413faa0 legacy=0x7fa9d40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0003a60 0x7fa9c0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0004810 0x7fa9c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00051c0 0x7fa9c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d406b640 con 0x7fa9c0003a60 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c000c0d0 con 0x7fa9c00051c0 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c000c290 con 0x7fa9c0004810 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d999b700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0003a60 0x7fa9c0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9da19c700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00051c0 0x7fa9c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d919a700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0004810 0x7fa9c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d999b700 1 -- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0004810 msgr2=0x7fa9c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d999b700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0004810 0x7fa9c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d999b700 1 -- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00051c0 msgr2=0x7fa9c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d999b700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9c00051c0 0x7fa9c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9da19c700 1 -- 172.21.15.33:0/2830929804 reap_dead start 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.904+0000 7fa9d999b700 1 -- 172.21.15.33:0/2830929804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0003570 con 0x7fa9c0003a60 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.905+0000 7fa9d999b700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0003a60 0x7fa9c0003e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000c390 tx=0x7fa9d40c84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fcb6e4a2a48533 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.905+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000cc60 con 0x7fa9c0003a60 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.905+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000cdc0 con 0x7fa9c0003a60 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.905+0000 7fa9d8999700 1 -- 172.21.15.33:0/2830929804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0005b70 con 0x7fa9c0003a60 2023-10-06T05:15:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.905+0000 7fa9da99d700 1 -- 172.21.15.33:0/2830929804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0003a60 msgr2=0x7fa9c0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.905+0000 7fa9da99d700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0003a60 0x7fa9c0003e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000c390 tx=0x7fa9d40c84c0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.906+0000 7fa9da99d700 1 -- 172.21.15.33:0/2830929804 shutdown_connections 2023-10-06T05:15:47.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.906+0000 7fa9da99d700 1 --2- 172.21.15.33:0/2830929804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0003a60 0x7fa9c0003e70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.906+0000 7fa9da99d700 1 -- 172.21.15.33:0/2830929804 >> 172.21.15.33:0/2830929804 conn(0x7fa9d406d030 msgr2=0x7fa9d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.906+0000 7fa9da99d700 1 -- 172.21.15.33:0/2830929804 shutdown_connections 2023-10-06T05:15:47.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.906+0000 7fa9da99d700 1 -- 172.21.15.33:0/2830929804 wait complete. 2023-10-06T05:15:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.907+0000 7fa9da99d700 1 Processor -- start 2023-10-06T05:15:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.907+0000 7fa9da99d700 1 -- start start 2023-10-06T05:15:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.908+0000 7fa9da99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0003a60 0x7fa9d40c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.908+0000 7fa9da99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 0x7fa9d40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.908+0000 7fa9da99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d40c9530 0x7fa9d40aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.908+0000 7fa9da99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c6000 con 0x7fa9d41563f0 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.908+0000 7fa9da99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c99b0 con 0x7fa9d40c9530 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.908+0000 7fa9da99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c4540 con 0x7fa9c0003a60 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 0x7fa9d40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 0x7fa9d40c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41552/0 (socket says 172.21.15.33:41552) 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 -- 172.21.15.33:0/1375508324 learned_addr learned my addr 172.21.15.33:0/1375508324 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d999b700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0003a60 0x7fa9d40c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9da19c700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d40c9530 0x7fa9d40aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 -- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0003a60 msgr2=0x7fa9d40c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0003a60 0x7fa9d40c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 -- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d40c9530 msgr2=0x7fa9d40aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d40c9530 0x7fa9d40aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 -- 172.21.15.33:0/1375508324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40c4820 con 0x7fa9d41563f0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9da19c700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d40c9530 0x7fa9d40aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9d919a700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 0x7fa9d40c8ff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4011a40 tx=0x7fa9c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400f940 con 0x7fa9d41563f0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400faa0 con 0x7fa9d41563f0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400fe20 con 0x7fa9d41563f0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.909+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40c4b00 con 0x7fa9d41563f0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.910+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d40c5520 con 0x7fa9d41563f0 2023-10-06T05:15:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.911+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d40b0370 con 0x7fa9d41563f0 2023-10-06T05:15:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.912+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9c4014650 con 0x7fa9d41563f0 2023-10-06T05:15:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.913+0000 7fa9c9ffb700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9b8066f30 0x7fa9b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.913+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa9c4018030 con 0x7fa9d41563f0 2023-10-06T05:15:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.913+0000 7fa9d999b700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9b8066f30 0x7fa9b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.914+0000 7fa9d999b700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9b8066f30 0x7fa9b80693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0007cd0 tx=0x7fa9d0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:47.917+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9d40b0370 con 0x7fa9d41563f0 2023-10-06T05:15:48.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.197+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa9d40b0370 con 0x7fa9d41563f0 2023-10-06T05:15:48.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.198+0000 7fa9c9ffb700 1 -- 172.21.15.33:0/1375508324 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa9d40b0370 con 0x7fa9d41563f0 2023-10-06T05:15:48.200 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:48.200 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:15:48.200 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:15:48.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.207 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:15:48.208 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.210 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.214 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:15:48.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:15:48.218 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:15:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:15:48.220 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:15:48.222 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:15:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9b8066f30 msgr2=0x7fa9b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9b8066f30 0x7fa9b80693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0007cd0 tx=0x7fa9d0015000 comp rx=0 tx=0).stop 2023-10-06T05:15:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 msgr2=0x7fa9d40c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 0x7fa9d40c8ff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4011a40 tx=0x7fa9c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 shutdown_connections 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9b8066f30 0x7fa9b80693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9d40c9530 0x7fa9d40aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9c0003a60 0x7fa9d40c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 --2- 172.21.15.33:0/1375508324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9d41563f0 0x7fa9d40c8ff0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.200+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 >> 172.21.15.33:0/1375508324 conn(0x7fa9d406d030 msgr2=0x7fa9d40ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.201+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 shutdown_connections 2023-10-06T05:15:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.201+0000 7fa9da99d700 1 -- 172.21.15.33:0/1375508324 wait complete. 2023-10-06T05:15:48.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:48.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.412+0000 7fd74e59d700 1 Processor -- start 2023-10-06T05:15:48.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.412+0000 7fd74e59d700 1 -- start start 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74e59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd7480ccfc0 0x7fd7480cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74e59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7480cf3b0 0x7fd7481594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74e59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7480cbd80 con 0x7fd7480cc750 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74e59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7480af5f0 con 0x7fd7480cf3b0 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74e59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7480af8d0 con 0x7fd7480ccfc0 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74cd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7480cf3b0 0x7fd7481594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74cd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7480cf3b0 0x7fd7481594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41554/0 (socket says 172.21.15.33:41554) 2023-10-06T05:15:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74cd9a700 1 -- 172.21.15.33:0/1180918300 learned_addr learned my addr 172.21.15.33:0/1180918300 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74d59b700 1 --2- 172.21.15.33:0/1180918300 >> v2:172.21.15.167:3300/0 conn(0x7fd7480ccfc0 0x7fd7480cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.414+0000 7fd74cd9a700 1 -- 172.21.15.33:0/1180918300 >> v2:172.21.15.167:3300/0 conn(0x7fd7480ccfc0 msgr2=0x7fd7480cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd74cd9a700 1 --2- 172.21.15.33:0/1180918300 >> v2:172.21.15.167:3300/0 conn(0x7fd7480ccfc0 0x7fd7480cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd74cd9a700 1 -- 172.21.15.33:0/1180918300 >> v1:172.21.15.167:6789/0 conn(0x7fd7480cc750 legacy=0x7fd7480b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd74cd9a700 1 -- 172.21.15.33:0/1180918300 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480afbb0 con 0x7fd7480cf3b0 2023-10-06T05:15:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd74cd9a700 1 --2- 172.21.15.33:0/1180918300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cf3b0 0x7fd7481594a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7380099b0 tx=0x7fd738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711ecac374dca67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd73ffff700 1 -- 172.21.15.33:0/1180918300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738009ea0 con 0x7fd7480cf3b0 2023-10-06T05:15:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd73ffff700 1 -- 172.21.15.33:0/1180918300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73800b820 con 0x7fd7480cf3b0 2023-10-06T05:15:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.415+0000 7fd73ffff700 1 -- 172.21.15.33:0/1180918300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73800bbc0 con 0x7fd7480cf3b0 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 -- 172.21.15.33:0/1180918300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cf3b0 msgr2=0x7fd7481594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1180918300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cf3b0 0x7fd7481594a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7380099b0 tx=0x7fd738009250 comp rx=0 tx=0).stop 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 -- 172.21.15.33:0/1180918300 shutdown_connections 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1180918300 >> v2:172.21.15.167:3300/0 conn(0x7fd7480ccfc0 0x7fd7480cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1180918300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cf3b0 0x7fd7481594a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 -- 172.21.15.33:0/1180918300 >> 172.21.15.33:0/1180918300 conn(0x7fd74806cff0 msgr2=0x7fd74806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 -- 172.21.15.33:0/1180918300 shutdown_connections 2023-10-06T05:15:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.416+0000 7fd74e59d700 1 -- 172.21.15.33:0/1180918300 wait complete. 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.417+0000 7fd74e59d700 1 Processor -- start 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.417+0000 7fd74e59d700 1 -- start start 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74e59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 0x7fd7480c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74e59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7480ccfc0 0x7fd7480c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74e59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7480b32a0 0x7fd7480c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74e59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7480b4420 con 0x7fd7480cc750 2023-10-06T05:15:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74e59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7480b3810 con 0x7fd7480b32a0 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74e59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7480b3af0 con 0x7fd7480ccfc0 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 0x7fd7480c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.418+0000 7fd74d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 0x7fd7480c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41564/0 (socket says 172.21.15.33:41564) 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 -- 172.21.15.33:0/1673929789 learned_addr learned my addr 172.21.15.33:0/1673929789 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 -- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7480ccfc0 msgr2=0x7fd7480c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7480ccfc0 0x7fd7480c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 -- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7480b32a0 msgr2=0x7fd7480c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7480b32a0 0x7fd7480c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 -- 172.21.15.33:0/1673929789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380094f0 con 0x7fd7480cc750 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74d59b700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 0x7fd7480c9dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd74400ec80 tx=0x7fd7440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400c9d0 con 0x7fd7480cc750 2023-10-06T05:15:48.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74400cb30 con 0x7fd7480cc750 2023-10-06T05:15:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7480b3dd0 con 0x7fd7480cc750 2023-10-06T05:15:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.419+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744013470 con 0x7fd7480cc750 2023-10-06T05:15:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.420+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7480ca310 con 0x7fd7480cc750 2023-10-06T05:15:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.420+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74804c380 con 0x7fd7480cc750 2023-10-06T05:15:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.421+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd744013820 con 0x7fd7480cc750 2023-10-06T05:15:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.422+0000 7fd73d7fa700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd72c066f30 0x7fd72c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.422+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd744084770 con 0x7fd7480cc750 2023-10-06T05:15:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.423+0000 7fd74cd9a700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd72c066f30 0x7fd72c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.423+0000 7fd74cd9a700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd72c066f30 0x7fd72c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd7380099b0 tx=0x7fd738017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.426+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd744051b30 con 0x7fd7480cc750 2023-10-06T05:15:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.630+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7480af0a0 con 0x7fd7480cc750 2023-10-06T05:15:48.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.631+0000 7fd73d7fa700 1 -- 172.21.15.33:0/1673929789 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+13650 (secure 0 0 0) 0x7fd744055150 con 0x7fd7480cc750 2023-10-06T05:15:48.632 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:48.633 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":44,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:47.686991+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd72c066f30 msgr2=0x7fd72c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd72c066f30 0x7fd72c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd7380099b0 tx=0x7fd738017000 comp rx=0 tx=0).stop 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 msgr2=0x7fd7480c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 0x7fd7480c9dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd74400ec80 tx=0x7fd7440091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 shutdown_connections 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd72c066f30 0x7fd72c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7480b32a0 0x7fd7480c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7480cc750 0x7fd7480c9dd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 --2- 172.21.15.33:0/1673929789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7480ccfc0 0x7fd7480c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 >> 172.21.15.33:0/1673929789 conn(0x7fd74806cff0 msgr2=0x7fd7480ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 shutdown_connections 2023-10-06T05:15:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.633+0000 7fd74e59d700 1 -- 172.21.15.33:0/1673929789 wait complete. 2023-10-06T05:15:48.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:15:48.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.842+0000 7f3c1d99d700 1 Processor -- start 2023-10-06T05:15:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.842+0000 7f3c1d99d700 1 -- start start 2023-10-06T05:15:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.843+0000 7f3c1d99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c181552f0 0x7f3c180ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.843+0000 7f3c1d99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c181586a0 con 0x7f3c18154f20 2023-10-06T05:15:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.843+0000 7f3c1d99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c18157ab0 con 0x7f3c180b18f0 2023-10-06T05:15:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.843+0000 7f3c1d99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1806bdf0 con 0x7f3c181552f0 2023-10-06T05:15:48.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c17fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c181552f0 0x7f3c180ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c1c99b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c18154f20 0x7f3c180b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40454/0 (socket says 172.21.15.33:40454) 2023-10-06T05:15:48.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c1c99b700 1 -- 172.21.15.33:0/2551378895 learned_addr learned my addr 172.21.15.33:0/2551378895 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c177fe700 1 -- 172.21.15.33:0/2551378895 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417788253 0 0) 0x7f3c181586a0 con 0x7f3c18154f20 2023-10-06T05:15:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c177fe700 1 -- 172.21.15.33:0/2551378895 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c08003570 con 0x7f3c18154f20 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c17fff700 1 -- 172.21.15.33:0/2551378895 >> v1:172.21.15.167:6789/0 conn(0x7f3c180b18f0 legacy=0x7f3c180c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c17fff700 1 -- 172.21.15.33:0/2551378895 >> v1:172.21.15.33:6789/0 conn(0x7f3c18154f20 legacy=0x7f3c180b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.844+0000 7f3c17fff700 1 -- 172.21.15.33:0/2551378895 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18157d70 con 0x7f3c181552f0 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c17fff700 1 --2- 172.21.15.33:0/2551378895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c181552f0 0x7f3c180ae720 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3c040098e0 tx=0x7f3c040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b5ef7d0e24d5fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c177fe700 1 -- 172.21.15.33:0/2551378895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400b640 con 0x7f3c181552f0 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c177fe700 1 -- 172.21.15.33:0/2551378895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0400b7a0 con 0x7f3c181552f0 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c177fe700 1 -- 172.21.15.33:0/2551378895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400bb40 con 0x7f3c181552f0 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c1d99d700 1 -- 172.21.15.33:0/2551378895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c181552f0 msgr2=0x7f3c180ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/2551378895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c181552f0 0x7f3c180ae720 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3c040098e0 tx=0x7f3c040091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c1d99d700 1 -- 172.21.15.33:0/2551378895 shutdown_connections 2023-10-06T05:15:48.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/2551378895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c181552f0 0x7f3c180ae720 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.845+0000 7f3c1d99d700 1 -- 172.21.15.33:0/2551378895 >> 172.21.15.33:0/2551378895 conn(0x7f3c1806d030 msgr2=0x7f3c180cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.846+0000 7f3c1d99d700 1 -- 172.21.15.33:0/2551378895 shutdown_connections 2023-10-06T05:15:48.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.846+0000 7f3c1d99d700 1 -- 172.21.15.33:0/2551378895 wait complete. 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.847+0000 7f3c1d99d700 1 Processor -- start 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.847+0000 7f3c1d99d700 1 -- start start 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c1d99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c180b18f0 0x7f3c180b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c1d99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 0x7f3c18153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c1d99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c180ca1c0 0x7f3c180ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c1d99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cab60 con 0x7f3c18154f20 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c1d99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180afea0 con 0x7f3c180ca1c0 2023-10-06T05:15:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c1d99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180b0330 con 0x7f3c180b18f0 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 0x7f3c18153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 0x7f3c18153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41584/0 (socket says 172.21.15.33:41584) 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.848+0000 7f3c17fff700 1 -- 172.21.15.33:0/1522234328 learned_addr learned my addr 172.21.15.33:0/1522234328 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c1c99b700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c180b18f0 0x7f3c180b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c1d19c700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c180ca1c0 0x7f3c180ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c17fff700 1 -- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c180b18f0 msgr2=0x7f3c180b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c17fff700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c180b18f0 0x7f3c180b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c17fff700 1 -- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c180ca1c0 msgr2=0x7f3c180ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c17fff700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c180ca1c0 0x7f3c180ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c17fff700 1 -- 172.21.15.33:0/1522234328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c04009420 con 0x7f3c18154f20 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c1d19c700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c180ca1c0 0x7f3c180ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c17fff700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 0x7f3c18153e10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3c1806d4d0 tx=0x7f3c04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400bb20 con 0x7f3c18154f20 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c04011420 con 0x7f3c18154f20 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040117c0 con 0x7f3c18154f20 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.849+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c180b1650 con 0x7f3c18154f20 2023-10-06T05:15:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.850+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c180af290 con 0x7f3c18154f20 2023-10-06T05:15:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.851+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c180ae530 con 0x7f3c18154f20 2023-10-06T05:15:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.851+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c04011b70 con 0x7f3c18154f20 2023-10-06T05:15:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.852+0000 7f3c14ff9700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bfc066f60 0x7f3bfc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.852+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3c04085ab0 con 0x7f3c18154f20 2023-10-06T05:15:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.853+0000 7f3c1c99b700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bfc066f60 0x7f3bfc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.853+0000 7f3c1c99b700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bfc066f60 0x7f3bfc069410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0058d0 tx=0x7f3c0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:48.857+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c04052e70 con 0x7f3c18154f20 2023-10-06T05:15:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.058+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c1804c380 con 0x7f3c18154f20 2023-10-06T05:15:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.059+0000 7f3c14ff9700 1 -- 172.21.15.33:0/1522234328 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c04056490 con 0x7f3c18154f20 2023-10-06T05:15:49.061 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:15:49.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bfc066f60 msgr2=0x7f3bfc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bfc066f60 0x7f3bfc069410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0058d0 tx=0x7f3c0c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 msgr2=0x7f3c18153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 0x7f3c18153e10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3c1806d4d0 tx=0x7f3c04017000 comp rx=0 tx=0).stop 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 shutdown_connections 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bfc066f60 0x7f3bfc069410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.061+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c180ca1c0 0x7f3c180ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.062+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c180b18f0 0x7f3c180b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.062+0000 7f3c1d99d700 1 --2- 172.21.15.33:0/1522234328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c18154f20 0x7f3c18153e10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.062+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 >> 172.21.15.33:0/1522234328 conn(0x7f3c1806d030 msgr2=0x7f3c180abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.062+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 shutdown_connections 2023-10-06T05:15:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.062+0000 7f3c1d99d700 1 -- 172.21.15.33:0/1522234328 wait complete. 2023-10-06T05:15:49.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.270+0000 7f4e5f19d700 1 Processor -- start 2023-10-06T05:15:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.270+0000 7f4e5f19d700 1 -- start start 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e5815a650 0x7f4e5815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5f19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e580cced0 0x7f4e580cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580cd890 con 0x7f4e5815a650 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5f19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5815d660 con 0x7f4e580cced0 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580ce890 con 0x7f4e580af540 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.271+0000 7f4e5d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e5815a650 0x7f4e5815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e5815a650 0x7f4e5815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41598/0 (socket says 172.21.15.33:41598) 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 -- 172.21.15.33:0/4099836593 learned_addr learned my addr 172.21.15.33:0/4099836593 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5e99c700 1 --2- 172.21.15.33:0/4099836593 >> v2:172.21.15.167:3300/0 conn(0x7f4e580cced0 0x7f4e580cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5e19b700 1 --2- 172.21.15.33:0/4099836593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 -- 172.21.15.33:0/4099836593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 msgr2=0x7f4e580af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 --2- 172.21.15.33:0/4099836593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 -- 172.21.15.33:0/4099836593 >> v2:172.21.15.167:3300/0 conn(0x7f4e580cced0 msgr2=0x7f4e580cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 --2- 172.21.15.33:0/4099836593 >> v2:172.21.15.167:3300/0 conn(0x7f4e580cced0 0x7f4e580cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 -- 172.21.15.33:0/4099836593 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5815aac0 con 0x7f4e5815a650 2023-10-06T05:15:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.272+0000 7f4e5d99a700 1 --2- 172.21.15.33:0/4099836593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5815a650 0x7f4e5815d120 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4e580caac0 tx=0x7f4e48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3d2322d0a98000 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5d199700 1 -- 172.21.15.33:0/4099836593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800b640 con 0x7f4e5815a650 2023-10-06T05:15:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5d199700 1 -- 172.21.15.33:0/4099836593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4800b7a0 con 0x7f4e5815a650 2023-10-06T05:15:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5d199700 1 -- 172.21.15.33:0/4099836593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800bb40 con 0x7f4e5815a650 2023-10-06T05:15:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 -- 172.21.15.33:0/4099836593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5815a650 msgr2=0x7f4e5815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/4099836593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5815a650 0x7f4e5815d120 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4e580caac0 tx=0x7f4e48009250 comp rx=0 tx=0).stop 2023-10-06T05:15:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 -- 172.21.15.33:0/4099836593 shutdown_connections 2023-10-06T05:15:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/4099836593 >> v2:172.21.15.167:3300/0 conn(0x7f4e580cced0 0x7f4e580cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/4099836593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/4099836593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e5815a650 0x7f4e5815d120 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 -- 172.21.15.33:0/4099836593 >> 172.21.15.33:0/4099836593 conn(0x7f4e5806d9b0 msgr2=0x7f4e5806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 -- 172.21.15.33:0/4099836593 shutdown_connections 2023-10-06T05:15:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.273+0000 7f4e5f19d700 1 -- 172.21.15.33:0/4099836593 wait complete. 2023-10-06T05:15:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.275+0000 7f4e5f19d700 1 Processor -- start 2023-10-06T05:15:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.275+0000 7f4e5f19d700 1 -- start start 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 0x7f4e58149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e5815a650 0x7f4e580c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581553c0 con 0x7f4e580cced0 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b0ad0 con 0x7f4e5815a650 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b2a30 con 0x7f4e580af540 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 0x7f4e58149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 0x7f4e58149f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41600/0 (socket says 172.21.15.33:41600) 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5d99a700 1 -- 172.21.15.33:0/3722638091 learned_addr learned my addr 172.21.15.33:0/3722638091 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5e19b700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.276+0000 7f4e5e99c700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e5815a650 0x7f4e580c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5d99a700 1 -- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 msgr2=0x7f4e580b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5d99a700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5d99a700 1 -- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e5815a650 msgr2=0x7f4e580c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5d99a700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e5815a650 0x7f4e580c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5d99a700 1 -- 172.21.15.33:0/3722638091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480094f0 con 0x7f4e580cced0 2023-10-06T05:15:49.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5d99a700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 0x7f4e58149f00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4e480099b0 tx=0x7f4e48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800bb20 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48011420 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e580b01e0 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480117c0 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.277+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e580b0760 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.278+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5804c380 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.279+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e48011bb0 con 0x7f4e580cced0 2023-10-06T05:15:49.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.280+0000 7f4e4e7fc700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e3c066f30 0x7f4e3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.280+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4e48085810 con 0x7f4e580cced0 2023-10-06T05:15:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.280+0000 7f4e5e19b700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e3c066f30 0x7f4e3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.281+0000 7f4e5e19b700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e3c066f30 0x7f4e3c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009ca0 tx=0x7f4e54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.284+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e48052c50 con 0x7f4e580cced0 2023-10-06T05:15:49.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.486+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4e580b0c10 con 0x7f4e580cced0 2023-10-06T05:15:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.487+0000 7f4e4e7fc700 1 -- 172.21.15.33:0/3722638091 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4e48011e60 con 0x7f4e580cced0 2023-10-06T05:15:49.489 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:15:49.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e3c066f30 msgr2=0x7f4e3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e3c066f30 0x7f4e3c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009ca0 tx=0x7f4e54008040 comp rx=0 tx=0).stop 2023-10-06T05:15:49.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 msgr2=0x7f4e58149f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 0x7f4e58149f00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4e480099b0 tx=0x7f4e48017000 comp rx=0 tx=0).stop 2023-10-06T05:15:49.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 shutdown_connections 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e3c066f30 0x7f4e3c0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e5815a650 0x7f4e580c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e580af540 0x7f4e580b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 --2- 172.21.15.33:0/3722638091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e580cced0 0x7f4e58149f00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 >> 172.21.15.33:0/3722638091 conn(0x7f4e5806d9b0 msgr2=0x7f4e580c6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 shutdown_connections 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.489+0000 7f4e5f19d700 1 -- 172.21.15.33:0/3722638091 wait complete. 2023-10-06T05:15:49.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:15:49.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.694+0000 7f3e9ed9d700 1 Processor -- start 2023-10-06T05:15:49.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.694+0000 7f3e9ed9d700 1 -- start start 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.695+0000 7f3e9ed9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3e980ccfc0 0x7f3e980cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.695+0000 7f3e9ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e980cc750 0x7f3e980b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.695+0000 7f3e9ed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e980cbd80 con 0x7f3e980cf3b0 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.695+0000 7f3e9ed9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e980af5f0 con 0x7f3e980cc750 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.695+0000 7f3e9ed9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e980af8d0 con 0x7f3e980ccfc0 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e980cc750 0x7f3e980b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e980cc750 0x7f3e980b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41608/0 (socket says 172.21.15.33:41608) 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 -- 172.21.15.33:0/2167863290 learned_addr learned my addr 172.21.15.33:0/2167863290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9dd9b700 1 --2- 172.21.15.33:0/2167863290 >> v2:172.21.15.167:3300/0 conn(0x7f3e980ccfc0 0x7f3e980cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 -- 172.21.15.33:0/2167863290 >> v2:172.21.15.167:3300/0 conn(0x7f3e980ccfc0 msgr2=0x7f3e980cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 --2- 172.21.15.33:0/2167863290 >> v2:172.21.15.167:3300/0 conn(0x7f3e980ccfc0 0x7f3e980cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 -- 172.21.15.33:0/2167863290 >> v1:172.21.15.167:6789/0 conn(0x7f3e980cf3b0 legacy=0x7f3e981594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 -- 172.21.15.33:0/2167863290 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980afbb0 con 0x7f3e980cc750 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9dd9b700 1 --2- 172.21.15.33:0/2167863290 >> v2:172.21.15.167:3300/0 conn(0x7f3e980ccfc0 0x7f3e980cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.696+0000 7f3e9e59c700 1 --2- 172.21.15.33:0/2167863290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980cc750 0x7f3e980b0460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009f20 tx=0x7f3e90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d054dc3ba54367d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9cd99700 1 -- 172.21.15.33:0/2167863290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000e070 con 0x7f3e980cc750 2023-10-06T05:15:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9cd99700 1 -- 172.21.15.33:0/2167863290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000ba40 con 0x7f3e980cc750 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9cd99700 1 -- 172.21.15.33:0/2167863290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900092e0 con 0x7f3e980cc750 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/2167863290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980cc750 msgr2=0x7f3e980b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/2167863290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980cc750 0x7f3e980b0460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009f20 tx=0x7f3e90009840 comp rx=0 tx=0).stop 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/2167863290 shutdown_connections 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/2167863290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980cc750 0x7f3e980b0460 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/2167863290 >> v2:172.21.15.167:3300/0 conn(0x7f3e980ccfc0 0x7f3e980cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/2167863290 >> 172.21.15.33:0/2167863290 conn(0x7f3e9806cff0 msgr2=0x7f3e9806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/2167863290 shutdown_connections 2023-10-06T05:15:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.697+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/2167863290 wait complete. 2023-10-06T05:15:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.699+0000 7f3e9ed9d700 1 Processor -- start 2023-10-06T05:15:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.699+0000 7f3e9ed9d700 1 -- start start 2023-10-06T05:15:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e980cc750 0x7f3e980c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e980ccfc0 0x7f3e98156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 0x7f3e98157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9ed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c75b0 con 0x7f3e980ca120 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9ed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c8410 con 0x7f3e980ccfc0 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c7800 con 0x7f3e980cc750 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 0x7f3e98157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 0x7f3e98157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41612/0 (socket says 172.21.15.33:41612) 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 -- 172.21.15.33:0/341839741 learned_addr learned my addr 172.21.15.33:0/341839741 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9d59a700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e980ccfc0 0x7f3e98156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9dd9b700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e980cc750 0x7f3e980c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 -- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e980cc750 msgr2=0x7f3e980c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e980cc750 0x7f3e980c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 -- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e980ccfc0 msgr2=0x7f3e98156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e980ccfc0 0x7f3e98156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.700+0000 7f3e9e59c700 1 -- 172.21.15.33:0/341839741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90009ae0 con 0x7f3e980ca120 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.701+0000 7f3e9e59c700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 0x7f3e98157450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000b6c0 tx=0x7f3e90018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.701+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90008ce0 con 0x7f3e980ca120 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.701+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90008e60 con 0x7f3e980ca120 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.701+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900266b0 con 0x7f3e980ca120 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.701+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e980c7ae0 con 0x7f3e980ca120 2023-10-06T05:15:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.701+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e980c80a0 con 0x7f3e980ca120 2023-10-06T05:15:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.702+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e980c85a0 con 0x7f3e980ca120 2023-10-06T05:15:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.703+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e90003040 con 0x7f3e980ca120 2023-10-06T05:15:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.704+0000 7f3e8dffb700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3e7c066f30 0x7f3e7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.704+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3e90086260 con 0x7f3e980ca120 2023-10-06T05:15:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.705+0000 7f3e9dd9b700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3e7c066f30 0x7f3e7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.705+0000 7f3e9dd9b700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3e7c066f30 0x7f3e7c0693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e94006f00 tx=0x7f3e94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.708+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e90053620 con 0x7f3e980ca120 2023-10-06T05:15:49.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.915+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3e9804c3e0 con 0x7f3e980ca120 2023-10-06T05:15:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.915+0000 7f3e8dffb700 1 -- 172.21.15.33:0/341839741 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3e90056c40 con 0x7f3e980ca120 2023-10-06T05:15:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:15:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:15:49.918 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:15:49.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3e7c066f30 msgr2=0x7f3e7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3e7c066f30 0x7f3e7c0693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e94006f00 tx=0x7f3e94008040 comp rx=0 tx=0).stop 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 msgr2=0x7f3e98157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 0x7f3e98157450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000b6c0 tx=0x7f3e90018000 comp rx=0 tx=0).stop 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 shutdown_connections 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3e7c066f30 0x7f3e7c0693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e980ca120 0x7f3e98157450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e980cc750 0x7f3e980c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 --2- 172.21.15.33:0/341839741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e980ccfc0 0x7f3e98156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 >> 172.21.15.33:0/341839741 conn(0x7f3e9806cff0 msgr2=0x7f3e980c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 shutdown_connections 2023-10-06T05:15:49.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:49.918+0000 7f3e9ed9d700 1 -- 172.21.15.33:0/341839741 wait complete. 2023-10-06T05:15:49.937 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:15:49.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:15:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.127+0000 7f30cd39d700 1 Processor -- start 2023-10-06T05:15:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.127+0000 7f30cd39d700 1 -- start start 2023-10-06T05:15:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.128+0000 7f30cd39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f30c8153b30 0x7f30c8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.128+0000 7f30cd39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c8157dc0 con 0x7f30c81574d0 2023-10-06T05:15:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.128+0000 7f30cd39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c815a6c0 con 0x7f30c806b180 2023-10-06T05:15:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.128+0000 7f30cd39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c815a9a0 con 0x7f30c8153b30 2023-10-06T05:15:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f30c81574d0 0x7f30c806aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34614/0 (socket says 172.21.15.33:34614) 2023-10-06T05:15:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c77fe700 1 -- 172.21.15.33:0/3955062550 learned_addr learned my addr 172.21.15.33:0/3955062550 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c7fff700 1 --2- 172.21.15.33:0/3955062550 >> v2:172.21.15.167:3300/0 conn(0x7f30c8153b30 0x7f30c8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907993284 0 0) 0x7f30c8157dc0 con 0x7f30c81574d0 2023-10-06T05:15:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b0003570 con 0x7f30c81574d0 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811286752 0 0) 0x7f30c815a6c0 con 0x7f30c806b180 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30c8157dc0 con 0x7f30c806b180 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c7fff700 1 -- 172.21.15.33:0/3955062550 >> v1:172.21.15.167:6789/0 conn(0x7f30c806b180 legacy=0x7f30c8158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c7fff700 1 -- 172.21.15.33:0/3955062550 >> v1:172.21.15.33:6789/0 conn(0x7f30c81574d0 legacy=0x7f30c806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.129+0000 7f30c7fff700 1 -- 172.21.15.33:0/3955062550 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c815ac80 con 0x7f30c8153b30 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30c7fff700 1 --2- 172.21.15.33:0/3955062550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c8153b30 0x7f30c8159de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30b80098e0 tx=0x7f30b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c97bf852b16910 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8011630 con 0x7f30c8153b30 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8011790 con 0x7f30c8153b30 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30c6ffd700 1 -- 172.21.15.33:0/3955062550 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8011b30 con 0x7f30c8153b30 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30cd39d700 1 -- 172.21.15.33:0/3955062550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c8153b30 msgr2=0x7f30c8159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30cd39d700 1 --2- 172.21.15.33:0/3955062550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c8153b30 0x7f30c8159de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30b80098e0 tx=0x7f30b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.130+0000 7f30cd39d700 1 -- 172.21.15.33:0/3955062550 shutdown_connections 2023-10-06T05:15:50.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.131+0000 7f30cd39d700 1 --2- 172.21.15.33:0/3955062550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c8153b30 0x7f30c8159de0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.131+0000 7f30cd39d700 1 -- 172.21.15.33:0/3955062550 >> 172.21.15.33:0/3955062550 conn(0x7f30c806cfe0 msgr2=0x7f30c80cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.131+0000 7f30cd39d700 1 -- 172.21.15.33:0/3955062550 shutdown_connections 2023-10-06T05:15:50.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.131+0000 7f30cd39d700 1 -- 172.21.15.33:0/3955062550 wait complete. 2023-10-06T05:15:50.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.132+0000 7f30cd39d700 1 Processor -- start 2023-10-06T05:15:50.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.132+0000 7f30cd39d700 1 -- start start 2023-10-06T05:15:50.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30cd39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30c806b180 0x7f30c813e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30cd39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 0x7f30c80cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30cd39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c81574d0 0x7f30c80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30cd39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80b0540 con 0x7f30c8153b30 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30cd39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80cbfe0 con 0x7f30c81574d0 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30cd39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80cc2c0 con 0x7f30c806b180 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 0x7f30c80cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 0x7f30c80cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47400/0 (socket says 172.21.15.33:47400) 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30c77fe700 1 -- 172.21.15.33:0/2324868309 learned_addr learned my addr 172.21.15.33:0/2324868309 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30c7fff700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30c806b180 0x7f30c813e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.133+0000 7f30ccb9c700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c81574d0 0x7f30c80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30c77fe700 1 -- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30c806b180 msgr2=0x7f30c813e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30c77fe700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30c806b180 0x7f30c813e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30c77fe700 1 -- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c81574d0 msgr2=0x7f30c80c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30c77fe700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c81574d0 0x7f30c80c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30c77fe700 1 -- 172.21.15.33:0/2324868309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8009420 con 0x7f30c8153b30 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30c77fe700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 0x7f30c80cb590 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c790 tx=0x7f30bc007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc005b30 con 0x7f30c8153b30 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc005c90 con 0x7f30c8153b30 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc007300 con 0x7f30c8153b30 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.134+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c80b01d0 con 0x7f30c8153b30 2023-10-06T05:15:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.135+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c80af9f0 con 0x7f30c8153b30 2023-10-06T05:15:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.135+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c80cc4e0 con 0x7f30c8153b30 2023-10-06T05:15:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.137+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30bc01b3b0 con 0x7f30c8153b30 2023-10-06T05:15:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.137+0000 7f30affff700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.138+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f30bc083890 con 0x7f30c8153b30 2023-10-06T05:15:50.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.138+0000 7f30c7fff700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.139+0000 7f30c7fff700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30b4066f30 0x7f30b40693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30b80093f0 tx=0x7f30b8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.142+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30bc050c50 con 0x7f30c8153b30 2023-10-06T05:15:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:50.346+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f30c804c380 con 0x7f30c8153b30 2023-10-06T05:15:51.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.266+0000 7f30affff700 1 -- 172.21.15.33:0/2324868309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7f30bc017020 con 0x7f30c8153b30 2023-10-06T05:15:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30b4066f30 msgr2=0x7f30b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30b4066f30 0x7f30b40693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30b80093f0 tx=0x7f30b8017000 comp rx=0 tx=0).stop 2023-10-06T05:15:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 msgr2=0x7f30c80cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 0x7f30c80cb590 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c790 tx=0x7f30bc007ea0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 shutdown_connections 2023-10-06T05:15:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30c81574d0 0x7f30c80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30c806b180 0x7f30c813e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 --2- 172.21.15.33:0/2324868309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30c8153b30 0x7f30c80cb590 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 >> 172.21.15.33:0/2324868309 conn(0x7f30c806cfe0 msgr2=0x7f30c80acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 shutdown_connections 2023-10-06T05:15:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.273+0000 7f30cd39d700 1 -- 172.21.15.33:0/2324868309 wait complete. 2023-10-06T05:15:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:15:51.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:15:51.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.493+0000 7f334d79d700 1 Processor -- start 2023-10-06T05:15:51.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.494+0000 7f334d79d700 1 -- start start 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f334d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3348156370 0x7f33480aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f334d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3348156740 0x7f33480cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f334d79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334806b330 con 0x7f33480cc750 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f334d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334806beb0 con 0x7f3348156740 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f334d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334806b610 con 0x7f3348156370 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f33477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3348156740 0x7f33480cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f33477fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3348156740 0x7f33480cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47416/0 (socket says 172.21.15.33:47416) 2023-10-06T05:15:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.495+0000 7f33477fe700 1 -- 172.21.15.33:0/3907697318 learned_addr learned my addr 172.21.15.33:0/3907697318 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.496+0000 7f3347fff700 1 --2- 172.21.15.33:0/3907697318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3348156370 0x7f33480aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.496+0000 7f33477fe700 1 -- 172.21.15.33:0/3907697318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3348156370 msgr2=0x7f33480aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.496+0000 7f33477fe700 1 --2- 172.21.15.33:0/3907697318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3348156370 0x7f33480aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.496+0000 7f33477fe700 1 -- 172.21.15.33:0/3907697318 >> v1:172.21.15.167:6789/0 conn(0x7f33480cc750 legacy=0x7f33480c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.496+0000 7f33477fe700 1 -- 172.21.15.33:0/3907697318 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334806b8f0 con 0x7f3348156740 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.496+0000 7f33477fe700 1 --2- 172.21.15.33:0/3907697318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f33480cc210 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099b0 tx=0x7f333c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0de06409d81419 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f3346ffd700 1 -- 172.21.15.33:0/3907697318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c009ea0 con 0x7f3348156740 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f3346ffd700 1 -- 172.21.15.33:0/3907697318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c00b860 con 0x7f3348156740 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f3346ffd700 1 -- 172.21.15.33:0/3907697318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00bc00 con 0x7f3348156740 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 -- 172.21.15.33:0/3907697318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 msgr2=0x7f33480cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 --2- 172.21.15.33:0/3907697318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f33480cc210 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099b0 tx=0x7f333c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 -- 172.21.15.33:0/3907697318 shutdown_connections 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 --2- 172.21.15.33:0/3907697318 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3348156370 0x7f33480aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 --2- 172.21.15.33:0/3907697318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f33480cc210 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 -- 172.21.15.33:0/3907697318 >> 172.21.15.33:0/3907697318 conn(0x7f334806c560 msgr2=0x7f334806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 -- 172.21.15.33:0/3907697318 shutdown_connections 2023-10-06T05:15:51.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.497+0000 7f334d79d700 1 -- 172.21.15.33:0/3907697318 wait complete. 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.499+0000 7f334d79d700 1 Processor -- start 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.499+0000 7f334d79d700 1 -- start start 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33480cc750 0x7f334816b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3348156370 0x7f334816be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f3348141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334814d8a0 con 0x7f3348156740 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334814cc90 con 0x7f3348156370 2023-10-06T05:15:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334814cf70 con 0x7f33480cc750 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f3348141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334cf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f3348141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47428/0 (socket says 172.21.15.33:47428) 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f334cf9c700 1 -- 172.21.15.33:0/3062592036 learned_addr learned my addr 172.21.15.33:0/3062592036 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f3347fff700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33480cc750 0x7f334816b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.500+0000 7f33477fe700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3348156370 0x7f334816be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334cf9c700 1 -- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33480cc750 msgr2=0x7f334816b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334cf9c700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33480cc750 0x7f334816b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334cf9c700 1 -- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3348156370 msgr2=0x7f334816be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334cf9c700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3348156370 0x7f334816be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334cf9c700 1 -- 172.21.15.33:0/3062592036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0094f0 con 0x7f3348156740 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f33477fe700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3348156370 0x7f334816be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f3347fff700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33480cc750 0x7f334816b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334cf9c700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f3348141d50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f334000c8e0 tx=0x7f3340001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000ce80 con 0x7f3348156740 2023-10-06T05:15:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334000a7e0 con 0x7f3348156740 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000ab80 con 0x7f3348156740 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334814d250 con 0x7f3348156740 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.501+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3348140d10 con 0x7f3348156740 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.502+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334814da10 con 0x7f3348156740 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.504+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3340024420 con 0x7f3348156740 2023-10-06T05:15:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.504+0000 7f332ffff700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3334066f30 0x7f33340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.504+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f3340084fd0 con 0x7f3348156740 2023-10-06T05:15:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.505+0000 7f3347fff700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3334066f30 0x7f33340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.506+0000 7f3347fff700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3334066f30 0x7f33340693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d80 tx=0x7f3338008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.508+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33400522a0 con 0x7f3348156740 2023-10-06T05:15:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:51.711+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f334804c380 con 0x7f3348156740 2023-10-06T05:15:52.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.273+0000 7f332ffff700 1 -- 172.21.15.33:0/3062592036 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7f33400558c0 con 0x7f3348156740 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.280+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3334066f30 msgr2=0x7f33340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.280+0000 7f334d79d700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3334066f30 0x7f33340693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d80 tx=0x7f3338008040 comp rx=0 tx=0).stop 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.280+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 msgr2=0x7f3348141d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.280+0000 7f334d79d700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f3348141d50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f334000c8e0 tx=0x7f3340001710 comp rx=0 tx=0).stop 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.281+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 shutdown_connections 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.281+0000 7f334d79d700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3334066f30 0x7f33340693e0 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d80 tx=0x7f3338008040 comp rx=0 tx=0).stop 2023-10-06T05:15:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.281+0000 7f334d79d700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3348156740 0x7f3348141d50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.281+0000 7f334d79d700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33480cc750 0x7f334816b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.281+0000 7f334d79d700 1 --2- 172.21.15.33:0/3062592036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3348156370 0x7f334816be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.281+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 >> 172.21.15.33:0/3062592036 conn(0x7f334806c560 msgr2=0x7f33480cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:52.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.287+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 shutdown_connections 2023-10-06T05:15:52.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.287+0000 7f334d79d700 1 -- 172.21.15.33:0/3062592036 wait complete. 2023-10-06T05:15:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:15:52.315 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:15:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.503+0000 7fef9cf9d700 1 Processor -- start 2023-10-06T05:15:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.503+0000 7fef9cf9d700 1 -- start start 2023-10-06T05:15:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef9cf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef981552f0 0x7fef980ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef9cf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef981586a0 con 0x7fef980b18f0 2023-10-06T05:15:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef9cf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef98157ab0 con 0x7fef98154f20 2023-10-06T05:15:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef9cf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9806bdf0 con 0x7fef981552f0 2023-10-06T05:15:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef96ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef981552f0 0x7fef980ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef97fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fef980b18f0 0x7fef980c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34618/0 (socket says 172.21.15.33:34618) 2023-10-06T05:15:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef97fff700 1 -- 172.21.15.33:0/1075749269 learned_addr learned my addr 172.21.15.33:0/1075749269 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef96ffd700 1 -- 172.21.15.33:0/1075749269 >> v1:172.21.15.167:6789/0 conn(0x7fef98154f20 legacy=0x7fef980b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef96ffd700 1 -- 172.21.15.33:0/1075749269 >> v1:172.21.15.33:6789/0 conn(0x7fef980b18f0 legacy=0x7fef980c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.505+0000 7fef96ffd700 1 -- 172.21.15.33:0/1075749269 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef98157d70 con 0x7fef981552f0 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.506+0000 7fef96ffd700 1 --2- 172.21.15.33:0/1075749269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef981552f0 0x7fef980ae720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0098e0 tx=0x7fef8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a99cdc064bf5815 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.506+0000 7fef967fc700 1 -- 172.21.15.33:0/1075749269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c009e80 con 0x7fef981552f0 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.506+0000 7fef967fc700 1 -- 172.21.15.33:0/1075749269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00b860 con 0x7fef981552f0 2023-10-06T05:15:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.506+0000 7fef967fc700 1 -- 172.21.15.33:0/1075749269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00bbe0 con 0x7fef981552f0 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.506+0000 7fef9cf9d700 1 -- 172.21.15.33:0/1075749269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef981552f0 msgr2=0x7fef980ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.506+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/1075749269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef981552f0 0x7fef980ae720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0098e0 tx=0x7fef8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.507+0000 7fef9cf9d700 1 -- 172.21.15.33:0/1075749269 shutdown_connections 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.507+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/1075749269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef981552f0 0x7fef980ae720 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.507+0000 7fef9cf9d700 1 -- 172.21.15.33:0/1075749269 >> 172.21.15.33:0/1075749269 conn(0x7fef9806d030 msgr2=0x7fef980cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.507+0000 7fef9cf9d700 1 -- 172.21.15.33:0/1075749269 shutdown_connections 2023-10-06T05:15:52.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.507+0000 7fef9cf9d700 1 -- 172.21.15.33:0/1075749269 wait complete. 2023-10-06T05:15:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.508+0000 7fef9cf9d700 1 Processor -- start 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.508+0000 7fef9cf9d700 1 -- start start 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef9cf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 0x7fef981484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef9cf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef98154f20 0x7fef980a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef9cf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef98148a30 0x7fef980a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef9cf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980a6d20 con 0x7fef980b18f0 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef9cf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980a6470 con 0x7fef98148a30 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef9cf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980a5eb0 con 0x7fef98154f20 2023-10-06T05:15:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.509+0000 7fef977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 0x7fef981484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 0x7fef981484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47452/0 (socket says 172.21.15.33:47452) 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 -- 172.21.15.33:0/3730180178 learned_addr learned my addr 172.21.15.33:0/3730180178 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef96ffd700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef98154f20 0x7fef980a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef97fff700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef98148a30 0x7fef980a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 -- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef98154f20 msgr2=0x7fef980a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef98154f20 0x7fef980a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 -- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef98148a30 msgr2=0x7fef980a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef98148a30 0x7fef980a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 -- 172.21.15.33:0/3730180178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c009420 con 0x7fef980b18f0 2023-10-06T05:15:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef977fe700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 0x7fef981484f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c590 tx=0x7fef8800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88011590 con 0x7fef980b18f0 2023-10-06T05:15:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.510+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef880116f0 con 0x7fef980b18f0 2023-10-06T05:15:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.511+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88011ab0 con 0x7fef980b18f0 2023-10-06T05:15:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.511+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef980a6190 con 0x7fef980b18f0 2023-10-06T05:15:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.511+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef98149030 con 0x7fef980b18f0 2023-10-06T05:15:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.512+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9804c380 con 0x7fef980b18f0 2023-10-06T05:15:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.513+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef8801b3b0 con 0x7fef980b18f0 2023-10-06T05:15:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.513+0000 7fef7f7fe700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef84066f30 0x7fef840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.514+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fef88083c80 con 0x7fef980b18f0 2023-10-06T05:15:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.514+0000 7fef96ffd700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef84066f30 0x7fef840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.515+0000 7fef96ffd700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef84066f30 0x7fef840693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0093f0 tx=0x7fef8c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.518+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef88050d50 con 0x7fef980b18f0 2023-10-06T05:15:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:52.765+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fef9806f370 con 0x7fef980b18f0 2023-10-06T05:15:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.274+0000 7fef7f7fe700 1 -- 172.21.15.33:0/3730180178 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fef88017020 con 0x7fef980b18f0 2023-10-06T05:15:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.302+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef84066f30 msgr2=0x7fef840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef84066f30 0x7fef840693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0093f0 tx=0x7fef8c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 msgr2=0x7fef981484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 0x7fef981484f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c590 tx=0x7fef8800c950 comp rx=0 tx=0).stop 2023-10-06T05:15:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 shutdown_connections 2023-10-06T05:15:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fef84066f30 0x7fef840693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef98148a30 0x7fef980a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef980b18f0 0x7fef981484f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 --2- 172.21.15.33:0/3730180178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef98154f20 0x7fef980a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 >> 172.21.15.33:0/3730180178 conn(0x7fef9806d030 msgr2=0x7fef980ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.303+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 shutdown_connections 2023-10-06T05:15:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.304+0000 7fef9cf9d700 1 -- 172.21.15.33:0/3730180178 wait complete. 2023-10-06T05:15:53.308 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 7 and data pool 8 2023-10-06T05:15:53.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:53.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.514+0000 7f25c319d700 1 Processor -- start 2023-10-06T05:15:53.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.514+0000 7f25c319d700 1 -- start start 2023-10-06T05:15:53.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.515+0000 7f25c319d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25bc0c8b50 0x7f25bc06e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.515+0000 7f25c319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc155af0 0x7f25bc06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.515+0000 7f25c319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc06ab60 con 0x7f25bc06a6a0 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.515+0000 7f25c319d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc06b7e0 con 0x7f25bc0c8b50 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.515+0000 7f25c319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc06ae40 con 0x7f25bc155af0 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c299c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f25bc06a6a0 0x7f25bc0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34632/0 (socket says 172.21.15.33:34632) 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c299c700 1 -- 172.21.15.33:0/4105886855 learned_addr learned my addr 172.21.15.33:0/4105886855 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c219b700 1 --2- 172.21.15.33:0/4105886855 >> v2:172.21.15.167:3300/0 conn(0x7f25bc0c8b50 0x7f25bc06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c199a700 1 --2- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc155af0 0x7f25bc06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c1199700 1 -- 172.21.15.33:0/4105886855 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668850673 0 0) 0x7f25bc06ab60 con 0x7f25bc06a6a0 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c1199700 1 -- 172.21.15.33:0/4105886855 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25a8003570 con 0x7f25bc06a6a0 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c219b700 1 -- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc155af0 msgr2=0x7f25bc06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c219b700 1 --2- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc155af0 0x7f25bc06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c219b700 1 -- 172.21.15.33:0/4105886855 >> v1:172.21.15.33:6789/0 conn(0x7f25bc06a6a0 legacy=0x7f25bc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.516+0000 7f25c219b700 1 -- 172.21.15.33:0/4105886855 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc06b120 con 0x7f25bc0c8b50 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.517+0000 7f25c219b700 1 --2- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc06e960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25b80098e0 tx=0x7f25b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d308e6a90e9e90cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.517+0000 7f25c1199700 1 -- 172.21.15.33:0/4105886855 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8011630 con 0x7f25bc0c8b50 2023-10-06T05:15:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.517+0000 7f25c1199700 1 -- 172.21.15.33:0/4105886855 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8011790 con 0x7f25bc0c8b50 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.517+0000 7f25c1199700 1 -- 172.21.15.33:0/4105886855 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8011b30 con 0x7f25bc0c8b50 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.517+0000 7f25c319d700 1 -- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 msgr2=0x7f25bc06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 --2- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc06e960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25b80098e0 tx=0x7f25b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 -- 172.21.15.33:0/4105886855 shutdown_connections 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 --2- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc06e960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 --2- 172.21.15.33:0/4105886855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc155af0 0x7f25bc06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 -- 172.21.15.33:0/4105886855 >> 172.21.15.33:0/4105886855 conn(0x7f25bc06d030 msgr2=0x7f25bc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 -- 172.21.15.33:0/4105886855 shutdown_connections 2023-10-06T05:15:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.518+0000 7f25c319d700 1 -- 172.21.15.33:0/4105886855 wait complete. 2023-10-06T05:15:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.519+0000 7f25c319d700 1 Processor -- start 2023-10-06T05:15:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.519+0000 7f25c319d700 1 -- start start 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.520+0000 7f25c319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc06a6a0 0x7f25bc154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.520+0000 7f25c319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.520+0000 7f25c319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 0x7f25bc153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.520+0000 7f25c319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc154680 con 0x7f25bc155af0 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.520+0000 7f25c319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c4f00 con 0x7f25bc0c8b50 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.520+0000 7f25c319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c5130 con 0x7f25bc06a6a0 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 0x7f25bc153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 0x7f25bc153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47462/0 (socket says 172.21.15.33:47462) 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 -- 172.21.15.33:0/4179444250 learned_addr learned my addr 172.21.15.33:0/4179444250 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c219b700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc06a6a0 0x7f25bc154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c199a700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 -- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc06a6a0 msgr2=0x7f25bc154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc06a6a0 0x7f25bc154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 -- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 msgr2=0x7f25bc146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 -- 172.21.15.33:0/4179444250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8009420 con 0x7f25bc155af0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.521+0000 7f25c299c700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 0x7f25bc153910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f25b400cc40 tx=0x7f25b4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.522+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400a960 con 0x7f25bc155af0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.522+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b400aac0 con 0x7f25bc155af0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.522+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400ae40 con 0x7f25bc155af0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.522+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc0c5410 con 0x7f25bc155af0 2023-10-06T05:15:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.522+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc0c59d0 con 0x7f25bc155af0 2023-10-06T05:15:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.523+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc04c380 con 0x7f25bc155af0 2023-10-06T05:15:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.524+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25b4007300 con 0x7f25bc155af0 2023-10-06T05:15:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.524+0000 7f25b27fc700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25a0066f30 0x7f25a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.525+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f25b4084ab0 con 0x7f25bc155af0 2023-10-06T05:15:53.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.525+0000 7f25c219b700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25a0066f30 0x7f25a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.526+0000 7f25c219b700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25a0066f30 0x7f25a00693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f25b80091e0 tx=0x7f25b8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.529+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25b4051ba0 con 0x7f25bc155af0 2023-10-06T05:15:53.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.734+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25bc06fb40 con 0x7f25bc155af0 2023-10-06T05:15:53.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.735+0000 7f25b27fc700 1 -- 172.21.15.33:0/4179444250 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f25b40551c0 con 0x7f25bc155af0 2023-10-06T05:15:53.736 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:53.737 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:52.766386+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:50.385790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:51.751034+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25a0066f30 msgr2=0x7f25a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25a0066f30 0x7f25a00693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f25b80091e0 tx=0x7f25b8017000 comp rx=0 tx=0).stop 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 msgr2=0x7f25bc153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 0x7f25bc153910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f25b400cc40 tx=0x7f25b4001a00 comp rx=0 tx=0).stop 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 shutdown_connections 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25a0066f30 0x7f25a00693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25bc155af0 0x7f25bc153910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25bc06a6a0 0x7f25bc154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 --2- 172.21.15.33:0/4179444250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25bc0c8b50 0x7f25bc146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 >> 172.21.15.33:0/4179444250 conn(0x7f25bc06d030 msgr2=0x7f25bc0cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 shutdown_connections 2023-10-06T05:15:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.737+0000 7f25c319d700 1 -- 172.21.15.33:0/4179444250 wait complete. 2023-10-06T05:15:53.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.941+0000 7f24ad39d700 1 Processor -- start 2023-10-06T05:15:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.941+0000 7f24ad39d700 1 -- start start 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24ad39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f24a80ccfc0 0x7f24a80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24ad39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80cc750 0x7f24a80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24ad39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a80cbd80 con 0x7f24a80cf3b0 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24ad39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a80af5f0 con 0x7f24a80ccfc0 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24ad39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80af8d0 con 0x7f24a80cc750 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24a77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f24a80cf3b0 0x7f24a81594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34644/0 (socket says 172.21.15.33:34644) 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24a77fe700 1 -- 172.21.15.33:0/2758345893 learned_addr learned my addr 172.21.15.33:0/2758345893 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24a7fff700 1 --2- 172.21.15.33:0/2758345893 >> v2:172.21.15.167:3300/0 conn(0x7f24a80ccfc0 0x7f24a80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.943+0000 7f24acb9c700 1 --2- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80cc750 0x7f24a80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a6ffd700 1 -- 172.21.15.33:0/2758345893 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813373036 0 0) 0x7f24a80cbd80 con 0x7f24a80cf3b0 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a6ffd700 1 -- 172.21.15.33:0/2758345893 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2490003570 con 0x7f24a80cf3b0 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a7fff700 1 -- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80cc750 msgr2=0x7f24a80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a7fff700 1 --2- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80cc750 0x7f24a80b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a7fff700 1 -- 172.21.15.33:0/2758345893 >> v1:172.21.15.33:6789/0 conn(0x7f24a80cf3b0 legacy=0x7f24a81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a7fff700 1 -- 172.21.15.33:0/2758345893 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80afbb0 con 0x7f24a80ccfc0 2023-10-06T05:15:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a7fff700 1 --2- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80ccfc0 0x7f24a80cc210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2498009910 tx=0x7f24980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40c9bc73f5f882b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a6ffd700 1 -- 172.21.15.33:0/2758345893 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011630 con 0x7f24a80ccfc0 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a6ffd700 1 -- 172.21.15.33:0/2758345893 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498011790 con 0x7f24a80ccfc0 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.944+0000 7f24a6ffd700 1 -- 172.21.15.33:0/2758345893 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011b30 con 0x7f24a80ccfc0 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 -- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80ccfc0 msgr2=0x7f24a80cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 --2- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80ccfc0 0x7f24a80cc210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2498009910 tx=0x7f24980091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 -- 172.21.15.33:0/2758345893 shutdown_connections 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 --2- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80ccfc0 0x7f24a80cc210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 --2- 172.21.15.33:0/2758345893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80cc750 0x7f24a80b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 -- 172.21.15.33:0/2758345893 >> 172.21.15.33:0/2758345893 conn(0x7f24a806cff0 msgr2=0x7f24a806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 -- 172.21.15.33:0/2758345893 shutdown_connections 2023-10-06T05:15:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.945+0000 7f24ad39d700 1 -- 172.21.15.33:0/2758345893 wait complete. 2023-10-06T05:15:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.947+0000 7f24ad39d700 1 Processor -- start 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.947+0000 7f24ad39d700 1 -- start start 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.947+0000 7f24ad39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 0x7f24a8148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24ad39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80ccfc0 0x7f24a80c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24ad39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80c7af0 0x7f24a80c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24ad39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8148760 con 0x7f24a80cc750 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24ad39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80b36b0 con 0x7f24a80c7af0 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24ad39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80c8070 con 0x7f24a80ccfc0 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 0x7f24a8148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 0x7f24a8148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47470/0 (socket says 172.21.15.33:47470) 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 -- 172.21.15.33:0/4019083981 learned_addr learned my addr 172.21.15.33:0/4019083981 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24acb9c700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80c7af0 0x7f24a80c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a77fe700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80ccfc0 0x7f24a80c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 -- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80ccfc0 msgr2=0x7f24a80c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80ccfc0 0x7f24a80c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 -- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80c7af0 msgr2=0x7f24a80c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80c7af0 0x7f24a80c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a7fff700 1 -- 172.21.15.33:0/4019083981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009420 con 0x7f24a80cc750 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.948+0000 7f24a77fe700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80ccfc0 0x7f24a80c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.949+0000 7f24a7fff700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 0x7f24a8148220 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2498011fa0 tx=0x7f2498017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.949+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011b10 con 0x7f24a80cc750 2023-10-06T05:15:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.949+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24980253f0 con 0x7f24a80cc750 2023-10-06T05:15:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.949+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a80c8350 con 0x7f24a80cc750 2023-10-06T05:15:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.949+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24980257b0 con 0x7f24a80cc750 2023-10-06T05:15:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.949+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a80b3340 con 0x7f24a80cc750 2023-10-06T05:15:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.950+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a804c380 con 0x7f24a80cc750 2023-10-06T05:15:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.951+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2498025b60 con 0x7f24a80cc750 2023-10-06T05:15:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.952+0000 7f248ffff700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.952+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2498085ef0 con 0x7f24a80cc750 2023-10-06T05:15:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.952+0000 7f24a77fe700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.953+0000 7f24a77fe700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f24a80b6410 tx=0x7f249c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:53.956+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2498052f60 con 0x7f24a80cc750 2023-10-06T05:15:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.160+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24a80ce1e0 con 0x7f24a80cc750 2023-10-06T05:15:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.161+0000 7f248ffff700 1 -- 172.21.15.33:0/4019083981 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f2498056580 con 0x7f24a80cc750 2023-10-06T05:15:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:54.163 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:52.766386+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:50.385790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:51.751034+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.163+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 msgr2=0x7f24940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.163+0000 7f24ad39d700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f24a80b6410 tx=0x7f249c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.163+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 msgr2=0x7f24a8148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.163+0000 7f24ad39d700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 0x7f24a8148220 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2498011fa0 tx=0x7f2498017000 comp rx=0 tx=0).stop 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 shutdown_connections 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a80c7af0 0x7f24a80c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a80cc750 0x7f24a8148220 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 --2- 172.21.15.33:0/4019083981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a80ccfc0 0x7f24a80c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 >> 172.21.15.33:0/4019083981 conn(0x7f24a806cff0 msgr2=0x7f24a8158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 shutdown_connections 2023-10-06T05:15:54.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.164+0000 7f24ad39d700 1 -- 172.21.15.33:0/4019083981 wait complete. 2023-10-06T05:15:54.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:15:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.373+0000 7f89aa19d700 1 Processor -- start 2023-10-06T05:15:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.374+0000 7f89aa19d700 1 -- start start 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89aa19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89a40aff90 0x7f89a40a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89aa19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89a406cb80 0x7f89a415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89aa19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a415d3a0 con 0x7f89a406c7b0 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89aa19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a40686c0 con 0x7f89a406cb80 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89aa19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a40b0470 con 0x7f89a40aff90 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89a999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89a406cb80 0x7f89a415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89a999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89a406cb80 0x7f89a415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47484/0 (socket says 172.21.15.33:47484) 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89a999c700 1 -- 172.21.15.33:0/4275342871 learned_addr learned my addr 172.21.15.33:0/4275342871 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.375+0000 7f89a899a700 1 --2- 172.21.15.33:0/4275342871 >> v2:172.21.15.167:3300/0 conn(0x7f89a40aff90 0x7f89a40a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f89a999c700 1 -- 172.21.15.33:0/4275342871 >> v2:172.21.15.167:3300/0 conn(0x7f89a40aff90 msgr2=0x7f89a40a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f89a999c700 1 --2- 172.21.15.33:0/4275342871 >> v2:172.21.15.167:3300/0 conn(0x7f89a40aff90 0x7f89a40a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f89a999c700 1 -- 172.21.15.33:0/4275342871 >> v1:172.21.15.167:6789/0 conn(0x7f89a406c7b0 legacy=0x7f89a4067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f89a999c700 1 -- 172.21.15.33:0/4275342871 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40b0750 con 0x7f89a406cb80 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f89a999c700 1 --2- 172.21.15.33:0/4275342871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415d830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f89a406ee90 tx=0x7f899c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272b586380c322bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f899bfff700 1 -- 172.21.15.33:0/4275342871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c00e070 con 0x7f89a406cb80 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f899bfff700 1 -- 172.21.15.33:0/4275342871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c00ba20 con 0x7f89a406cb80 2023-10-06T05:15:54.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.376+0000 7f899bfff700 1 -- 172.21.15.33:0/4275342871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c0092e0 con 0x7f89a406cb80 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 -- 172.21.15.33:0/4275342871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 msgr2=0x7f89a415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 --2- 172.21.15.33:0/4275342871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415d830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f89a406ee90 tx=0x7f899c009840 comp rx=0 tx=0).stop 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 -- 172.21.15.33:0/4275342871 shutdown_connections 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 --2- 172.21.15.33:0/4275342871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415d830 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 --2- 172.21.15.33:0/4275342871 >> v2:172.21.15.167:3300/0 conn(0x7f89a40aff90 0x7f89a40a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 -- 172.21.15.33:0/4275342871 >> 172.21.15.33:0/4275342871 conn(0x7f89a406d670 msgr2=0x7f89a406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 -- 172.21.15.33:0/4275342871 shutdown_connections 2023-10-06T05:15:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.377+0000 7f89aa19d700 1 -- 172.21.15.33:0/4275342871 wait complete. 2023-10-06T05:15:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.378+0000 7f89aa19d700 1 Processor -- start 2023-10-06T05:15:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 -- start start 2023-10-06T05:15:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89a406c7b0 0x7f89a4145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a40aff90 0x7f89a4169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a41565a0 con 0x7f89a406cb80 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4157120 con 0x7f89a40aff90 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.379+0000 7f89aa19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a415a860 con 0x7f89a406c7b0 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47492/0 (socket says 172.21.15.33:47492) 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 -- 172.21.15.33:0/530334248 learned_addr learned my addr 172.21.15.33:0/530334248 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a999c700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a40aff90 0x7f89a4169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a919b700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89a406c7b0 0x7f89a4145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 -- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89a406c7b0 msgr2=0x7f89a4145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89a406c7b0 0x7f89a4145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 -- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a40aff90 msgr2=0x7f89a4169c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a40aff90 0x7f89a4169c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 -- 172.21.15.33:0/530334248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c009ae0 con 0x7f89a406cb80 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.380+0000 7f89a899a700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415a320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f899400e930 tx=0x7f89940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.381+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994009510 con 0x7f89a406cb80 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.381+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400c850 con 0x7f89a406cb80 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.381+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a41567f0 con 0x7f89a406cb80 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.381+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400cbf0 con 0x7f89a406cb80 2023-10-06T05:15:54.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.381+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a4156db0 con 0x7f89a406cb80 2023-10-06T05:15:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.382+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a41572b0 con 0x7f89a406cb80 2023-10-06T05:15:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.383+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8994013470 con 0x7f89a406cb80 2023-10-06T05:15:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.384+0000 7f89997fa700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8988066f30 0x7f89880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.384+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8994084850 con 0x7f89a406cb80 2023-10-06T05:15:54.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.384+0000 7f89a919b700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8988066f30 0x7f89880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.385+0000 7f89a919b700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8988066f30 0x7f89880693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f89a0003890 tx=0x7f89a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.388+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89940518c0 con 0x7f89a406cb80 2023-10-06T05:15:54.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:54.636+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89a404c380 con 0x7f89a406cb80 2023-10-06T05:15:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.293+0000 7f89997fa700 1 -- 172.21.15.33:0/530334248 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8994054ee0 con 0x7f89a406cb80 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8988066f30 msgr2=0x7f89880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8988066f30 0x7f89880693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f89a0003890 tx=0x7f89a0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 msgr2=0x7f89a415a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415a320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f899400e930 tx=0x7f89940091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 shutdown_connections 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8988066f30 0x7f89880693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a40aff90 0x7f89a4169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89a406c7b0 0x7f89a4145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 --2- 172.21.15.33:0/530334248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a406cb80 0x7f89a415a320 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 >> 172.21.15.33:0/530334248 conn(0x7f89a406d670 msgr2=0x7f89a40af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 shutdown_connections 2023-10-06T05:15:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.296+0000 7f89aa19d700 1 -- 172.21.15.33:0/530334248 wait complete. 2023-10-06T05:15:55.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:15:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.509+0000 7fa61559d700 1 Processor -- start 2023-10-06T05:15:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.509+0000 7fa61559d700 1 -- start start 2023-10-06T05:15:55.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa61559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa610155a50 0x7fa610155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa61559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100c9490 0x7fa6100af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa61559d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6100b06b0 con 0x7fa6100cc7e0 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa61559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100c99e0 con 0x7fa610155a50 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa61559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100c9cc0 con 0x7fa6100c9490 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa60ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa610155a50 0x7fa610155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa60ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa610155a50 0x7fa610155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47496/0 (socket says 172.21.15.33:47496) 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa60ffff700 1 -- 172.21.15.33:0/299964420 learned_addr learned my addr 172.21.15.33:0/299964420 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa60f7fe700 1 --2- 172.21.15.33:0/299964420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100c9490 0x7fa6100af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa60ffff700 1 -- 172.21.15.33:0/299964420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100c9490 msgr2=0x7fa6100af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.511+0000 7fa60ffff700 1 --2- 172.21.15.33:0/299964420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100c9490 0x7fa6100af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.512+0000 7fa60ffff700 1 -- 172.21.15.33:0/299964420 >> v1:172.21.15.33:6789/0 conn(0x7fa6100cc7e0 legacy=0x7fa6100af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.512+0000 7fa60ffff700 1 -- 172.21.15.33:0/299964420 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100c9fa0 con 0x7fa610155a50 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.512+0000 7fa60ffff700 1 --2- 172.21.15.33:0/299964420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa610155e20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa6000099e0 tx=0x7fa600009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5ad4aba3163de7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.512+0000 7fa60effd700 1 -- 172.21.15.33:0/299964420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000b820 con 0x7fa610155a50 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.512+0000 7fa60effd700 1 -- 172.21.15.33:0/299964420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60000b980 con 0x7fa610155a50 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.512+0000 7fa60effd700 1 -- 172.21.15.33:0/299964420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000bd20 con 0x7fa610155a50 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 -- 172.21.15.33:0/299964420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 msgr2=0x7fa610155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 --2- 172.21.15.33:0/299964420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa610155e20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa6000099e0 tx=0x7fa600009250 comp rx=0 tx=0).stop 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 -- 172.21.15.33:0/299964420 shutdown_connections 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 --2- 172.21.15.33:0/299964420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa610155e20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 --2- 172.21.15.33:0/299964420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100c9490 0x7fa6100af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 -- 172.21.15.33:0/299964420 >> 172.21.15.33:0/299964420 conn(0x7fa61006d030 msgr2=0x7fa6100cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 -- 172.21.15.33:0/299964420 shutdown_connections 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.513+0000 7fa61559d700 1 -- 172.21.15.33:0/299964420 wait complete. 2023-10-06T05:15:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.514+0000 7fa61559d700 1 Processor -- start 2023-10-06T05:15:55.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.515+0000 7fa61559d700 1 -- start start 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.515+0000 7fa61559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6100c9490 0x7fa6100c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.515+0000 7fa61559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100cc7e0 0x7fa61016a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa61559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa6100c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa61559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100c2870 con 0x7fa610155a50 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa61559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100c1d70 con 0x7fa6100c9490 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa61559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61016a940 con 0x7fa6100cc7e0 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa6100c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa6100c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47500/0 (socket says 172.21.15.33:47500) 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa60f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100cc7e0 0x7fa61016a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 -- 172.21.15.33:0/3091022588 learned_addr learned my addr 172.21.15.33:0/3091022588 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa60ffff700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6100c9490 0x7fa6100c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 -- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100cc7e0 msgr2=0x7fa61016a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100cc7e0 0x7fa61016a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 -- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6100c9490 msgr2=0x7fa6100c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6100c9490 0x7fa6100c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.516+0000 7fa614d9c700 1 -- 172.21.15.33:0/3091022588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000094f0 con 0x7fa610155a50 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.517+0000 7fa614d9c700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa6100c7830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c8e0 tx=0x7fa608001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.517+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608012630 con 0x7fa610155a50 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.517+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608012790 con 0x7fa610155a50 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.517+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608012b50 con 0x7fa610155a50 2023-10-06T05:15:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.517+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100c1fd0 con 0x7fa610155a50 2023-10-06T05:15:55.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.517+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100c2590 con 0x7fa610155a50 2023-10-06T05:15:55.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.518+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61004c380 con 0x7fa610155a50 2023-10-06T05:15:55.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.519+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa60801a3a0 con 0x7fa610155a50 2023-10-06T05:15:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.520+0000 7fa5f7fff700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5fc066f30 0x7fa5fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.520+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa608084940 con 0x7fa610155a50 2023-10-06T05:15:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.520+0000 7fa60ffff700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5fc066f30 0x7fa5fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.521+0000 7fa60ffff700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5fc066f30 0x7fa5fc0693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6000099e0 tx=0x7fa600017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.524+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6080519b0 con 0x7fa610155a50 2023-10-06T05:15:55.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:55.775+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa61006bb10 con 0x7fa610155a50 2023-10-06T05:15:56.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.303+0000 7fa5f7fff700 1 -- 172.21.15.33:0/3091022588 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa608054fd0 con 0x7fa610155a50 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5fc066f30 msgr2=0x7fa5fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5fc066f30 0x7fa5fc0693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6000099e0 tx=0x7fa600017000 comp rx=0 tx=0).stop 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 msgr2=0x7fa6100c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa6100c7830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c8e0 tx=0x7fa608001710 comp rx=0 tx=0).stop 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 shutdown_connections 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5fc066f30 0x7fa5fc0693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa610155a50 0x7fa6100c7830 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6100c9490 0x7fa6100c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 --2- 172.21.15.33:0/3091022588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6100cc7e0 0x7fa61016a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 >> 172.21.15.33:0/3091022588 conn(0x7fa61006d030 msgr2=0x7fa6100c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 shutdown_connections 2023-10-06T05:15:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.305+0000 7fa61559d700 1 -- 172.21.15.33:0/3091022588 wait complete. 2023-10-06T05:15:56.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.514+0000 7f3a31d9d700 1 Processor -- start 2023-10-06T05:15:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.514+0000 7f3a31d9d700 1 -- start start 2023-10-06T05:15:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.515+0000 7f3a31d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3a2c159520 0x7f3a2c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.515+0000 7f3a31d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a2c0c6170 0x7f3a2c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.515+0000 7f3a31d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c06c820 con 0x7f3a2c0c80d0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.515+0000 7f3a31d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0c6610 con 0x7f3a2c0c6170 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.515+0000 7f3a31d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c06b670 con 0x7f3a2c159520 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a2c0c6170 0x7f3a2c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3a2c0c6170 0x7f3a2c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47508/0 (socket says 172.21.15.33:47508) 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 -- 172.21.15.33:0/2119557198 learned_addr learned my addr 172.21.15.33:0/2119557198 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a30d9b700 1 --2- 172.21.15.33:0/2119557198 >> v2:172.21.15.167:3300/0 conn(0x7f3a2c159520 0x7f3a2c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 -- 172.21.15.33:0/2119557198 >> v2:172.21.15.167:3300/0 conn(0x7f3a2c159520 msgr2=0x7f3a2c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 --2- 172.21.15.33:0/2119557198 >> v2:172.21.15.167:3300/0 conn(0x7f3a2c159520 0x7f3a2c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 -- 172.21.15.33:0/2119557198 >> v1:172.21.15.167:6789/0 conn(0x7f3a2c0c80d0 legacy=0x7f3a2c067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 -- 172.21.15.33:0/2119557198 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c06a160 con 0x7f3a2c0c6170 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a30d9b700 1 --2- 172.21.15.33:0/2119557198 >> v2:172.21.15.167:3300/0 conn(0x7f3a2c159520 0x7f3a2c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.516+0000 7f3a2bfff700 1 --2- 172.21.15.33:0/2119557198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c0c7b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0c6e80 tx=0x7f3a18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab5becf593012c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a2b7fe700 1 -- 172.21.15.33:0/2119557198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800b860 con 0x7f3a2c0c6170 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a2b7fe700 1 -- 172.21.15.33:0/2119557198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1800b9c0 con 0x7f3a2c0c6170 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a2b7fe700 1 -- 172.21.15.33:0/2119557198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800bd60 con 0x7f3a2c0c6170 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 -- 172.21.15.33:0/2119557198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c0c6170 msgr2=0x7f3a2c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/2119557198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c0c7b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0c6e80 tx=0x7f3a18009250 comp rx=0 tx=0).stop 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 -- 172.21.15.33:0/2119557198 shutdown_connections 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/2119557198 >> v2:172.21.15.167:3300/0 conn(0x7f3a2c159520 0x7f3a2c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/2119557198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c0c7b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 -- 172.21.15.33:0/2119557198 >> 172.21.15.33:0/2119557198 conn(0x7f3a2c0cc930 msgr2=0x7f3a2c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 -- 172.21.15.33:0/2119557198 shutdown_connections 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.517+0000 7f3a31d9d700 1 -- 172.21.15.33:0/2119557198 wait complete. 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.519+0000 7f3a31d9d700 1 Processor -- start 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.519+0000 7f3a31d9d700 1 -- start start 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a31d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a31d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c0c80d0 0x7f3a2c143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a31d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 0x7f3a2c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a31d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c150320 con 0x7f3a2c159520 2023-10-06T05:15:56.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a31d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15b450 con 0x7f3a2c0c6170 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a31d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15b730 con 0x7f3a2c0c80d0 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a3159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 0x7f3a2c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a3159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 0x7f3a2c0b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47524/0 (socket says 172.21.15.33:47524) 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a3159c700 1 -- 172.21.15.33:0/1825369033 learned_addr learned my addr 172.21.15.33:0/1825369033 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a2bfff700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c0c80d0 0x7f3a2c143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.520+0000 7f3a30d9b700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a3159c700 1 -- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c0c80d0 msgr2=0x7f3a2c143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a3159c700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c0c80d0 0x7f3a2c143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a3159c700 1 -- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c0c6170 msgr2=0x7f3a2c14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a3159c700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a3159c700 1 -- 172.21.15.33:0/1825369033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180094f0 con 0x7f3a2c159520 2023-10-06T05:15:56.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a30d9b700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a2bfff700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c0c80d0 0x7f3a2c143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a3159c700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 0x7f3a2c0b13b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c8e0 tx=0x7f3a24001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400ce80 con 0x7f3a2c159520 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2400a7e0 con 0x7f3a2c159520 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c15ba10 con 0x7f3a2c159520 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400ab80 con 0x7f3a2c159520 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.521+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c143740 con 0x7f3a2c159520 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.522+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c15b2b0 con 0x7f3a2c159520 2023-10-06T05:15:56.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.523+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a24012420 con 0x7f3a2c159520 2023-10-06T05:15:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.524+0000 7f3a28ff9700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3a10066f30 0x7f3a100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.524+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3a24084a20 con 0x7f3a2c159520 2023-10-06T05:15:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.524+0000 7f3a30d9b700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3a10066f30 0x7f3a100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.525+0000 7f3a30d9b700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3a10066f30 0x7f3a100693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a20006f00 tx=0x7f3a20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.528+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a2c15b2b0 con 0x7f3a2c159520 2023-10-06T05:15:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.774+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a2c15b2b0 con 0x7f3a2c159520 2023-10-06T05:15:56.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.775+0000 7f3a28ff9700 1 -- 172.21.15.33:0/1825369033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f3a2c15b2b0 con 0x7f3a2c159520 2023-10-06T05:15:56.776 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:56.777 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3a10066f30 msgr2=0x7f3a100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3a10066f30 0x7f3a100693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a20006f00 tx=0x7f3a20008040 comp rx=0 tx=0).stop 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 msgr2=0x7f3a2c0b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 0x7f3a2c0b13b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c8e0 tx=0x7f3a24001710 comp rx=0 tx=0).stop 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 shutdown_connections 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3a10066f30 0x7f3a100693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a2c159520 0x7f3a2c0b13b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a2c0c6170 0x7f3a2c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 --2- 172.21.15.33:0/1825369033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a2c0c80d0 0x7f3a2c143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 >> 172.21.15.33:0/1825369033 conn(0x7f3a2c0cc930 msgr2=0x7f3a2c0cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 shutdown_connections 2023-10-06T05:15:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.777+0000 7f3a31d9d700 1 -- 172.21.15.33:0/1825369033 wait complete. 2023-10-06T05:15:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:15:56.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.982+0000 7f214d79d700 1 Processor -- start 2023-10-06T05:15:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.982+0000 7f214d79d700 1 -- start start 2023-10-06T05:15:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f214d79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f214806ab00 0x7f21480b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f214d79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214815e2d0 con 0x7f21480b2e30 2023-10-06T05:15:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f214d79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214806afa0 con 0x7f21480b0a40 2023-10-06T05:15:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f214d79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214815d6c0 con 0x7f214806ab00 2023-10-06T05:15:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f214cf9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f21480b2e30 0x7f214815a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34658/0 (socket says 172.21.15.33:34658) 2023-10-06T05:15:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f214cf9c700 1 -- 172.21.15.33:0/885659271 learned_addr learned my addr 172.21.15.33:0/885659271 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f21477fe700 1 --2- 172.21.15.33:0/885659271 >> v2:172.21.15.167:3300/0 conn(0x7f214806ab00 0x7f21480b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f2146ffd700 1 -- 172.21.15.33:0/885659271 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251549389 0 0) 0x7f214815e2d0 con 0x7f21480b2e30 2023-10-06T05:15:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.984+0000 7f2146ffd700 1 -- 172.21.15.33:0/885659271 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2130003570 con 0x7f21480b2e30 2023-10-06T05:15:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f21477fe700 1 -- 172.21.15.33:0/885659271 >> v1:172.21.15.167:6789/0 conn(0x7f21480b0a40 legacy=0x7f21480b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f21477fe700 1 -- 172.21.15.33:0/885659271 >> v1:172.21.15.33:6789/0 conn(0x7f21480b2e30 legacy=0x7f214815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f21477fe700 1 -- 172.21.15.33:0/885659271 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214815d9a0 con 0x7f214806ab00 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f21477fe700 1 --2- 172.21.15.33:0/885659271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214806ab00 0x7f21480b28f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f213c0098e0 tx=0x7f213c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f63e35248330d8c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f2146ffd700 1 -- 172.21.15.33:0/885659271 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00b640 con 0x7f214806ab00 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f2146ffd700 1 -- 172.21.15.33:0/885659271 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00b7a0 con 0x7f214806ab00 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.985+0000 7f2146ffd700 1 -- 172.21.15.33:0/885659271 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00bb40 con 0x7f214806ab00 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 -- 172.21.15.33:0/885659271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214806ab00 msgr2=0x7f21480b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 --2- 172.21.15.33:0/885659271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214806ab00 0x7f21480b28f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f213c0098e0 tx=0x7f213c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 -- 172.21.15.33:0/885659271 shutdown_connections 2023-10-06T05:15:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 --2- 172.21.15.33:0/885659271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214806ab00 0x7f21480b28f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 -- 172.21.15.33:0/885659271 >> 172.21.15.33:0/885659271 conn(0x7f214806e0f0 msgr2=0x7f21480675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 -- 172.21.15.33:0/885659271 shutdown_connections 2023-10-06T05:15:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.986+0000 7f214d79d700 1 -- 172.21.15.33:0/885659271 wait complete. 2023-10-06T05:15:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.987+0000 7f214d79d700 1 Processor -- start 2023-10-06T05:15:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.988+0000 7f214d79d700 1 -- start start 2023-10-06T05:15:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.988+0000 7f214d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f214806ab00 0x7f214815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.988+0000 7f214d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 0x7f214815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f214d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21480b2e30 0x7f214815fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f214d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21480c31c0 con 0x7f21480b0a40 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f214d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21480c29b0 con 0x7f21480b2e30 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f214d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214815b910 con 0x7f214806ab00 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 0x7f214815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 0x7f214815f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47532/0 (socket says 172.21.15.33:47532) 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 -- 172.21.15.33:0/3522741615 learned_addr learned my addr 172.21.15.33:0/3522741615 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f2147fff700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f214806ab00 0x7f214815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f214cf9c700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21480b2e30 0x7f214815fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 -- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f214806ab00 msgr2=0x7f214815ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f214806ab00 0x7f214815ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 -- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21480b2e30 msgr2=0x7f214815fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21480b2e30 0x7f214815fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 -- 172.21.15.33:0/3522741615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c009420 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f2147fff700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f214806ab00 0x7f214815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.989+0000 7f21477fe700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 0x7f214815f700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f213c0093f0 tx=0x7f213c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.990+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c000c40 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.990+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00b910 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.990+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c025670 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.990+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214815f470 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.990+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21480c2ee0 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.991+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214806fcc0 con 0x7f21480b0a40 2023-10-06T05:15:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.992+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f213c023070 con 0x7f21480b0a40 2023-10-06T05:15:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.992+0000 7f212ffff700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2134066f30 0x7f21340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.992+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f213c085a40 con 0x7f21480b0a40 2023-10-06T05:15:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.993+0000 7f2147fff700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2134066f30 0x7f21340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.993+0000 7f2147fff700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2134066f30 0x7f21340693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2138001820 tx=0x7f2138009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:56.997+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f213c052ab0 con 0x7f21480b0a40 2023-10-06T05:15:57.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.201+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f214815ba80 con 0x7f21480b0a40 2023-10-06T05:15:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.202+0000 7f212ffff700 1 -- 172.21.15.33:0/3522741615 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f213c015090 con 0x7f21480b0a40 2023-10-06T05:15:57.204 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:57.204 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":48,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:15:54.284633+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:50.385790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:51.751034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2134066f30 msgr2=0x7f21340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2134066f30 0x7f21340693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2138001820 tx=0x7f2138009040 comp rx=0 tx=0).stop 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 msgr2=0x7f214815f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 0x7f214815f700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f213c0093f0 tx=0x7f213c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 shutdown_connections 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2134066f30 0x7f21340693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21480b2e30 0x7f214815fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f214806ab00 0x7f214815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 --2- 172.21.15.33:0/3522741615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21480b0a40 0x7f214815f700 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 >> 172.21.15.33:0/3522741615 conn(0x7f214806e0f0 msgr2=0x7f21480abfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 shutdown_connections 2023-10-06T05:15:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.205+0000 7f214d79d700 1 -- 172.21.15.33:0/3522741615 wait complete. 2023-10-06T05:15:57.224 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:15:57.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:57.247 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:15:57.246+0000 7f3c49bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:15:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.412+0000 7f385f59d700 1 Processor -- start 2023-10-06T05:15:57.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.412+0000 7f385f59d700 1 -- start start 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385f59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38580c8b50 0x7f38580c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155ec0 0x7f38580c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385f59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385806ab60 con 0x7f3858155af0 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385f59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385806b7e0 con 0x7f38580c8b50 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385806ae40 con 0x7f3858155ec0 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38580c8b50 0x7f38580c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38580c8b50 0x7f38580c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47540/0 (socket says 172.21.15.33:47540) 2023-10-06T05:15:57.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.413+0000 7f385dd9a700 1 -- 172.21.15.33:0/2143298286 learned_addr learned my addr 172.21.15.33:0/2143298286 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:57.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385ed9c700 1 --2- 172.21.15.33:0/2143298286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155ec0 0x7f38580c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385dd9a700 1 -- 172.21.15.33:0/2143298286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155ec0 msgr2=0x7f38580c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385dd9a700 1 --2- 172.21.15.33:0/2143298286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155ec0 0x7f38580c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385dd9a700 1 -- 172.21.15.33:0/2143298286 >> v1:172.21.15.167:6789/0 conn(0x7f3858155af0 legacy=0x7f385806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385dd9a700 1 -- 172.21.15.33:0/2143298286 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385806b120 con 0x7f38580c8b50 2023-10-06T05:15:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385d599700 1 -- 172.21.15.33:0/2143298286 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477626115 0 0) 0x7f385806ab60 con 0x7f3858155af0 2023-10-06T05:15:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385dd9a700 1 --2- 172.21.15.33:0/2143298286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f38580c7210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38480099b0 tx=0x7f3848009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7696152c9439a63f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385d599700 1 -- 172.21.15.33:0/2143298286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848009ea0 con 0x7f38580c8b50 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385d599700 1 -- 172.21.15.33:0/2143298286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384800b860 con 0x7f38580c8b50 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.414+0000 7f385d599700 1 -- 172.21.15.33:0/2143298286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800bc00 con 0x7f38580c8b50 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 -- 172.21.15.33:0/2143298286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 msgr2=0x7f38580c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 --2- 172.21.15.33:0/2143298286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f38580c7210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38480099b0 tx=0x7f3848009250 comp rx=0 tx=0).stop 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 -- 172.21.15.33:0/2143298286 shutdown_connections 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 --2- 172.21.15.33:0/2143298286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155ec0 0x7f38580c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 --2- 172.21.15.33:0/2143298286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f38580c7210 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 -- 172.21.15.33:0/2143298286 >> 172.21.15.33:0/2143298286 conn(0x7f385806d030 msgr2=0x7f385806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 -- 172.21.15.33:0/2143298286 shutdown_connections 2023-10-06T05:15:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.415+0000 7f385f59d700 1 -- 172.21.15.33:0/2143298286 wait complete. 2023-10-06T05:15:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.417+0000 7f385f59d700 1 Processor -- start 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.417+0000 7f385f59d700 1 -- start start 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f3858144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155af0 0x7f385816aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858155ec0 0x7f385816b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581592e0 con 0x7f38580c8b50 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580c4b20 con 0x7f3858155ec0 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38581579d0 con 0x7f3858155af0 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f3858144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385dd9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155af0 0x7f385816aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385ed9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858155ec0 0x7f385816b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385dd9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155af0 0x7f385816aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:38976/0 (socket says 172.21.15.33:38976) 2023-10-06T05:15:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f3858144c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47544/0 (socket says 172.21.15.33:47544) 2023-10-06T05:15:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385dd9a700 1 -- 172.21.15.33:0/948934008 learned_addr learned my addr 172.21.15.33:0/948934008 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 -- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155af0 msgr2=0x7f385816aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155af0 0x7f385816aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 -- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858155ec0 msgr2=0x7f385816b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858155ec0 0x7f385816b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.418+0000 7f385e59b700 1 -- 172.21.15.33:0/948934008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480094f0 con 0x7f38580c8b50 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.419+0000 7f385e59b700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f3858144c70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f385400c510 tx=0x7f3854007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.419+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400ce80 con 0x7f38580c8b50 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.419+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854011590 con 0x7f38580c8b50 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.419+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011950 con 0x7f38580c8b50 2023-10-06T05:15:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.419+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3858157cb0 con 0x7f38580c8b50 2023-10-06T05:15:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.419+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3858168aa0 con 0x7f38580c8b50 2023-10-06T05:15:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.420+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385804c380 con 0x7f38580c8b50 2023-10-06T05:15:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.421+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3854011d40 con 0x7f38580c8b50 2023-10-06T05:15:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.422+0000 7f384e7fc700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f383c066f30 0x7f383c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.422+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3854083a20 con 0x7f38580c8b50 2023-10-06T05:15:57.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.422+0000 7f385dd9a700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f383c066f30 0x7f383c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.423+0000 7f385dd9a700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f383c066f30 0x7f383c0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3858158e70 tx=0x7f3848017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.426+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3854050a90 con 0x7f38580c8b50 2023-10-06T05:15:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.624+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f385806b400 con 0x7f383c066f30 2023-10-06T05:15:57.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.634+0000 7f384e7fc700 1 -- 172.21.15.33:0/948934008 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299646 (secure 0 0 0) 0x7f385806b400 con 0x7f383c066f30 2023-10-06T05:15:57.640 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f383c066f30 msgr2=0x7f383c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f383c066f30 0x7f383c0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3858158e70 tx=0x7f3848017000 comp rx=0 tx=0).stop 2023-10-06T05:15:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 msgr2=0x7f3858144c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f3858144c70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f385400c510 tx=0x7f3854007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 shutdown_connections 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f383c066f30 0x7f383c0693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858155ec0 0x7f385816b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38580c8b50 0x7f3858144c70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 --2- 172.21.15.33:0/948934008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3858155af0 0x7f385816aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.638+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 >> 172.21.15.33:0/948934008 conn(0x7f385806d030 msgr2=0x7f385806eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.639+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 shutdown_connections 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.639+0000 7f385f59d700 1 -- 172.21.15.33:0/948934008 wait complete. 2023-10-06T05:15:57.643 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:15:57.657 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-10-06T05:15:55.837853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383556,"kb_used_data":5252,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591164,"statfs":{"total":773094113280,"available":770653351936,"internally_reserved":0,"allocated":5378048,"data_stored":2933014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.575121"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265875+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:14.699179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.272106+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:42.503247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.272610+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:12.445672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.275128+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:25.153053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.269162+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:27.527005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268314+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:38.717480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265231+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:21.182085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268420+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:23.905925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268142+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:45.221625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268454+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:27.527005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268805+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:25.153053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265274+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:40.408958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268319+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:45.221625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265160+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:53.626357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268083+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:24.110662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265288+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.335935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268494+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:38.717480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.271108+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:04.134338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265207+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:50.909229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265164+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:06.682415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.271523+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:06.252867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265957+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:32.093973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268228+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:24.110662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.271228+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:50.909229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268862+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.335935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.168750+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:15:48.168750+0000","last_peered":"2023-10-06T05:15:48.168750+0000","last_clean":"2023-10-06T05:15:48.168750+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:15:48.168750+0000","last_undegraded":"2023-10-06T05:15:48.168750+0000","last_fullsized":"2023-10-06T05:15:48.168750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265246+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:04.134338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.573251+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:15:48.573251+0000","last_peered":"2023-10-06T05:15:48.573251+0000","last_clean":"2023-10-06T05:15:48.573251+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:15:48.573251+0000","last_undegraded":"2023-10-06T05:15:48.573251+0000","last_fullsized":"2023-10-06T05:15:48.573251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268313+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:55.046002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.249782+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:15:48.249782+0000","last_peered":"2023-10-06T05:15:48.249782+0000","last_clean":"2023-10-06T05:15:48.249782+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:15:48.249782+0000","last_undegraded":"2023-10-06T05:15:48.249782+0000","last_fullsized":"2023-10-06T05:15:48.249782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268337+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:25.153053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.573270+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:15:48.573270+0000","last_peered":"2023-10-06T05:15:48.573270+0000","last_clean":"2023-10-06T05:15:48.573270+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:15:48.573270+0000","last_undegraded":"2023-10-06T05:15:48.573270+0000","last_fullsized":"2023-10-06T05:15:48.573270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.267537+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:15:48.267537+0000","last_peered":"2023-10-06T05:15:48.267537+0000","last_clean":"2023-10-06T05:15:48.267537+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:15:48.267537+0000","last_undegraded":"2023-10-06T05:15:48.267537+0000","last_fullsized":"2023-10-06T05:15:48.267537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.268314+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:55.046002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.267508+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:15:48.267508+0000","last_peered":"2023-10-06T05:15:48.267508+0000","last_clean":"2023-10-06T05:15:48.267508+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:15:48.267508+0000","last_undegraded":"2023-10-06T05:15:48.267508+0000","last_fullsized":"2023-10-06T05:15:48.267508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.275173+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:23.905925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.267472+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:15:48.267472+0000","last_peered":"2023-10-06T05:15:48.267472+0000","last_clean":"2023-10-06T05:15:48.267472+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:15:48.267472+0000","last_undegraded":"2023-10-06T05:15:48.267472+0000","last_fullsized":"2023-10-06T05:15:48.267472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265344+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:38.992778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:48.168749+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:15:48.168749+0000","last_peered":"2023-10-06T05:15:48.168749+0000","last_clean":"2023-10-06T05:15:48.168749+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:15:48.168749+0000","last_undegraded":"2023-10-06T05:15:48.168749+0000","last_fullsized":"2023-10-06T05:15:48.168749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:15:52.261747+0000","last_change":"2023-10-06T05:15:52.261747+0000","last_active":"2023-10-06T05:15:52.261747+0000","last_peered":"2023-10-06T05:15:52.261747+0000","last_clean":"2023-10-06T05:15:52.261747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:52.261747+0000","last_undegraded":"2023-10-06T05:15:52.261747+0000","last_fullsized":"2023-10-06T05:15:52.261747+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:15:51.252385+0000","last_change":"2023-10-06T05:15:51.265289+0000","last_active":"2023-10-06T05:15:51.252385+0000","last_peered":"2023-10-06T05:15:51.252385+0000","last_clean":"2023-10-06T05:15:51.252385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:15:51.252385+0000","last_undegraded":"2023-10-06T05:15:51.252385+0000","last_fullsized":"2023-10-06T05:15:51.252385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:08.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:15:47.691659+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:15:47.691659+0000","last_peered":"2023-10-06T05:15:47.691659+0000","last_clean":"2023-10-06T05:15:47.691659+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:15:47.691659+0000","last_undegraded":"2023-10-06T05:15:47.691659+0000","last_fullsized":"2023-10-06T05:15:47.691659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":811712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81699999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.91700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.026}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.82699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.44},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.59799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.57799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":811712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.503},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.93600000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":215186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.629},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":528384,"data_stored":221344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.378},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.872}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":528384,"data_stored":221344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.042},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.62},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.88}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.85899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":528384,"data_stored":221344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.86399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.72499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.62},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:57.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:57.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.883+0000 7fc11f59e700 1 Processor -- start 2023-10-06T05:15:57.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.884+0000 7fc11f59e700 1 -- start start 2023-10-06T05:15:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc1200b0a40 0x7fc12006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1200b0e10 0x7fc1200b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b2e30 0x7fc12015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12015e2d0 con 0x7fc1200b0e10 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12015d6c0 con 0x7fc1200b0a40 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12015d9a0 con 0x7fc1200b2e30 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1200b0e10 0x7fc1200b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1200b0e10 0x7fc1200b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47558/0 (socket says 172.21.15.33:47558) 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11dd9b700 1 -- 172.21.15.33:0/2222698577 learned_addr learned my addr 172.21.15.33:0/2222698577 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11e59c700 1 --2- 172.21.15.33:0/2222698577 >> v2:172.21.15.167:3300/0 conn(0x7fc1200b0a40 0x7fc12006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.885+0000 7fc11ed9d700 1 --2- 172.21.15.33:0/2222698577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b2e30 0x7fc12015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11dd9b700 1 -- 172.21.15.33:0/2222698577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b2e30 msgr2=0x7fc12015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11dd9b700 1 --2- 172.21.15.33:0/2222698577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b2e30 0x7fc12015a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11dd9b700 1 -- 172.21.15.33:0/2222698577 >> v2:172.21.15.167:3300/0 conn(0x7fc1200b0a40 msgr2=0x7fc12006ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11dd9b700 1 --2- 172.21.15.33:0/2222698577 >> v2:172.21.15.167:3300/0 conn(0x7fc1200b0a40 0x7fc12006ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11dd9b700 1 -- 172.21.15.33:0/2222698577 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12015dc80 con 0x7fc1200b0e10 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11dd9b700 1 --2- 172.21.15.33:0/2222698577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc1200b28f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc12006a230 tx=0x7fc114009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07903c9afee9b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11d59a700 1 -- 172.21.15.33:0/2222698577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400b860 con 0x7fc1200b0e10 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11d59a700 1 -- 172.21.15.33:0/2222698577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400b9c0 con 0x7fc1200b0e10 2023-10-06T05:15:57.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.886+0000 7fc11d59a700 1 -- 172.21.15.33:0/2222698577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400bd60 con 0x7fc1200b0e10 2023-10-06T05:15:57.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 -- 172.21.15.33:0/2222698577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 msgr2=0x7fc1200b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 --2- 172.21.15.33:0/2222698577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc1200b28f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc12006a230 tx=0x7fc114009250 comp rx=0 tx=0).stop 2023-10-06T05:15:57.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 -- 172.21.15.33:0/2222698577 shutdown_connections 2023-10-06T05:15:57.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 --2- 172.21.15.33:0/2222698577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b2e30 0x7fc12015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 --2- 172.21.15.33:0/2222698577 >> v2:172.21.15.167:3300/0 conn(0x7fc1200b0a40 0x7fc12006ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 --2- 172.21.15.33:0/2222698577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc1200b28f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 -- 172.21.15.33:0/2222698577 >> 172.21.15.33:0/2222698577 conn(0x7fc12006e0f0 msgr2=0x7fc1200675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 -- 172.21.15.33:0/2222698577 shutdown_connections 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.887+0000 7fc11f59e700 1 -- 172.21.15.33:0/2222698577 wait complete. 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.888+0000 7fc11f59e700 1 Processor -- start 2023-10-06T05:15:57.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.888+0000 7fc11f59e700 1 -- start start 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.889+0000 7fc11f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b0a40 0x7fc12015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.889+0000 7fc11f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc12015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.889+0000 7fc11f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200b2e30 0x7fc1201564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.889+0000 7fc11f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12015cea0 con 0x7fc1200b0e10 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.889+0000 7fc11f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200c2570 con 0x7fc1200b2e30 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.889+0000 7fc11f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200c2850 con 0x7fc1200b0a40 2023-10-06T05:15:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc12015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc12015bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47572/0 (socket says 172.21.15.33:47572) 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 -- 172.21.15.33:0/273352381 learned_addr learned my addr 172.21.15.33:0/273352381 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 -- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b0a40 msgr2=0x7fc12015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b0a40 0x7fc12015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 -- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200b2e30 msgr2=0x7fc1201564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200b2e30 0x7fc1201564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 -- 172.21.15.33:0/273352381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140094f0 con 0x7fc1200b0e10 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc11dd9b700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc12015bd70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc1200ce290 tx=0x7fc114017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.890+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114024440 con 0x7fc1200b0e10 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.891+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1140245a0 con 0x7fc1200b0e10 2023-10-06T05:15:57.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.891+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1200c2b30 con 0x7fc1200b0e10 2023-10-06T05:15:57.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.891+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114024960 con 0x7fc1200b0e10 2023-10-06T05:15:57.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.891+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1200c30f0 con 0x7fc1200b0e10 2023-10-06T05:15:57.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.892+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12006fcc0 con 0x7fc1200b0e10 2023-10-06T05:15:57.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.893+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc11401a420 con 0x7fc1200b0e10 2023-10-06T05:15:57.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.894+0000 7fc10e7fc700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.894+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc114086590 con 0x7fc1200b0e10 2023-10-06T05:15:57.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.894+0000 7fc11e59c700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.895+0000 7fc11e59c700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc100066f30 0x7fc1000693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc110009a60 tx=0x7fc110008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:57.898+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc114053680 con 0x7fc1200b0e10 2023-10-06T05:15:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.096+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1200cd000 con 0x7fc100066f30 2023-10-06T05:15:58.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.106+0000 7fc10e7fc700 1 -- 172.21.15.33:0/273352381 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310281 (secure 0 0 0) 0x7fc1200cd000 con 0x7fc100066f30 2023-10-06T05:15:58.111 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc100066f30 msgr2=0x7fc1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc100066f30 0x7fc1000693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc110009a60 tx=0x7fc110008040 comp rx=0 tx=0).stop 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 msgr2=0x7fc12015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc12015bd70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc1200ce290 tx=0x7fc114017000 comp rx=0 tx=0).stop 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 shutdown_connections 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200b2e30 0x7fc1201564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1200b0a40 0x7fc12015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 --2- 172.21.15.33:0/273352381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1200b0e10 0x7fc12015bd70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 >> 172.21.15.33:0/273352381 conn(0x7fc12006e0f0 msgr2=0x7fc12006b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 shutdown_connections 2023-10-06T05:15:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.110+0000 7fc11f59e700 1 -- 172.21.15.33:0/273352381 wait complete. 2023-10-06T05:15:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:15:58.129 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-10-06T05:15:57.838206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2388040,"kb_used_data":5640,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752586680,"statfs":{"total":773094113280,"available":770648760320,"internally_reserved":0,"allocated":5775360,"data_stored":3119084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001091"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301252+0000","last_change":"2023-10-06T05:15:53.297065+0000","last_active":"2023-10-06T05:15:53.301252+0000","last_peered":"2023-10-06T05:15:53.301252+0000","last_clean":"2023-10-06T05:15:53.301252+0000","last_became_active":"2023-10-06T05:15:53.296768+0000","last_became_peered":"2023-10-06T05:15:53.296768+0000","last_unstale":"2023-10-06T05:15:53.301252+0000","last_undegraded":"2023-10-06T05:15:53.301252+0000","last_fullsized":"2023-10-06T05:15:53.301252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:47.821445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307893+0000","last_change":"2023-10-06T05:15:53.298215+0000","last_active":"2023-10-06T05:15:53.307893+0000","last_peered":"2023-10-06T05:15:53.307893+0000","last_clean":"2023-10-06T05:15:53.307893+0000","last_became_active":"2023-10-06T05:15:53.297809+0000","last_became_peered":"2023-10-06T05:15:53.297809+0000","last_unstale":"2023-10-06T05:15:53.307893+0000","last_undegraded":"2023-10-06T05:15:53.307893+0000","last_fullsized":"2023-10-06T05:15:53.307893+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:28.391326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.304143+0000","last_change":"2023-10-06T05:15:53.292570+0000","last_active":"2023-10-06T05:15:53.304143+0000","last_peered":"2023-10-06T05:15:53.304143+0000","last_clean":"2023-10-06T05:15:53.304143+0000","last_became_active":"2023-10-06T05:15:53.290099+0000","last_became_peered":"2023-10-06T05:15:53.290099+0000","last_unstale":"2023-10-06T05:15:53.304143+0000","last_undegraded":"2023-10-06T05:15:53.304143+0000","last_fullsized":"2023-10-06T05:15:53.304143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:29.640085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.292965+0000","last_change":"2023-10-06T05:15:53.287378+0000","last_active":"2023-10-06T05:15:53.292965+0000","last_peered":"2023-10-06T05:15:53.292965+0000","last_clean":"2023-10-06T05:15:53.292965+0000","last_became_active":"2023-10-06T05:15:53.287127+0000","last_became_peered":"2023-10-06T05:15:53.287127+0000","last_unstale":"2023-10-06T05:15:53.292965+0000","last_undegraded":"2023-10-06T05:15:53.292965+0000","last_fullsized":"2023-10-06T05:15:53.292965+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.959968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.299080+0000","last_change":"2023-10-06T05:15:53.295909+0000","last_active":"2023-10-06T05:15:53.299080+0000","last_peered":"2023-10-06T05:15:53.299080+0000","last_clean":"2023-10-06T05:15:53.299080+0000","last_became_active":"2023-10-06T05:15:53.295619+0000","last_became_peered":"2023-10-06T05:15:53.295619+0000","last_unstale":"2023-10-06T05:15:53.299080+0000","last_undegraded":"2023-10-06T05:15:53.299080+0000","last_fullsized":"2023-10-06T05:15:53.299080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.959968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301230+0000","last_change":"2023-10-06T05:15:53.296327+0000","last_active":"2023-10-06T05:15:53.301230+0000","last_peered":"2023-10-06T05:15:53.301230+0000","last_clean":"2023-10-06T05:15:53.301230+0000","last_became_active":"2023-10-06T05:15:53.296045+0000","last_became_peered":"2023-10-06T05:15:53.296045+0000","last_unstale":"2023-10-06T05:15:53.301230+0000","last_undegraded":"2023-10-06T05:15:53.301230+0000","last_fullsized":"2023-10-06T05:15:53.301230+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:48.431987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.295427+0000","last_change":"2023-10-06T05:15:53.288054+0000","last_active":"2023-10-06T05:15:53.295427+0000","last_peered":"2023-10-06T05:15:53.295427+0000","last_clean":"2023-10-06T05:15:53.295427+0000","last_became_active":"2023-10-06T05:15:53.287748+0000","last_became_peered":"2023-10-06T05:15:53.287748+0000","last_unstale":"2023-10-06T05:15:53.295427+0000","last_undegraded":"2023-10-06T05:15:53.295427+0000","last_fullsized":"2023-10-06T05:15:53.295427+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:45.223958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308318+0000","last_change":"2023-10-06T05:15:53.298060+0000","last_active":"2023-10-06T05:15:53.308318+0000","last_peered":"2023-10-06T05:15:53.308318+0000","last_clean":"2023-10-06T05:15:53.308318+0000","last_became_active":"2023-10-06T05:15:53.297351+0000","last_became_peered":"2023-10-06T05:15:53.297351+0000","last_unstale":"2023-10-06T05:15:53.308318+0000","last_undegraded":"2023-10-06T05:15:53.308318+0000","last_fullsized":"2023-10-06T05:15:53.308318+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:57.223358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307358+0000","last_change":"2023-10-06T05:15:53.297600+0000","last_active":"2023-10-06T05:15:53.307358+0000","last_peered":"2023-10-06T05:15:53.307358+0000","last_clean":"2023-10-06T05:15:53.307358+0000","last_became_active":"2023-10-06T05:15:53.297319+0000","last_became_peered":"2023-10-06T05:15:53.297319+0000","last_unstale":"2023-10-06T05:15:53.307358+0000","last_undegraded":"2023-10-06T05:15:53.307358+0000","last_fullsized":"2023-10-06T05:15:53.307358+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:56.067385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.302716+0000","last_change":"2023-10-06T05:15:53.299122+0000","last_active":"2023-10-06T05:15:53.302716+0000","last_peered":"2023-10-06T05:15:53.302716+0000","last_clean":"2023-10-06T05:15:53.302716+0000","last_became_active":"2023-10-06T05:15:53.298848+0000","last_became_peered":"2023-10-06T05:15:53.298848+0000","last_unstale":"2023-10-06T05:15:53.302716+0000","last_undegraded":"2023-10-06T05:15:53.302716+0000","last_fullsized":"2023-10-06T05:15:53.302716+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:09.656521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.293165+0000","last_change":"2023-10-06T05:15:53.291156+0000","last_active":"2023-10-06T05:15:53.293165+0000","last_peered":"2023-10-06T05:15:53.293165+0000","last_clean":"2023-10-06T05:15:53.293165+0000","last_became_active":"2023-10-06T05:15:53.290792+0000","last_became_peered":"2023-10-06T05:15:53.290792+0000","last_unstale":"2023-10-06T05:15:53.293165+0000","last_undegraded":"2023-10-06T05:15:53.293165+0000","last_fullsized":"2023-10-06T05:15:53.293165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:57.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308464+0000","last_change":"2023-10-06T05:15:53.302242+0000","last_active":"2023-10-06T05:15:53.308464+0000","last_peered":"2023-10-06T05:15:53.308464+0000","last_clean":"2023-10-06T05:15:53.308464+0000","last_became_active":"2023-10-06T05:15:53.301995+0000","last_became_peered":"2023-10-06T05:15:53.301995+0000","last_unstale":"2023-10-06T05:15:53.308464+0000","last_undegraded":"2023-10-06T05:15:53.308464+0000","last_fullsized":"2023-10-06T05:15:53.308464+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:43.938648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301029+0000","last_change":"2023-10-06T05:15:53.296312+0000","last_active":"2023-10-06T05:15:53.301029+0000","last_peered":"2023-10-06T05:15:53.301029+0000","last_clean":"2023-10-06T05:15:53.301029+0000","last_became_active":"2023-10-06T05:15:53.296000+0000","last_became_peered":"2023-10-06T05:15:53.296000+0000","last_unstale":"2023-10-06T05:15:53.301029+0000","last_undegraded":"2023-10-06T05:15:53.301029+0000","last_fullsized":"2023-10-06T05:15:53.301029+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:40.002140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.296398+0000","last_change":"2023-10-06T05:15:53.289506+0000","last_active":"2023-10-06T05:15:53.296398+0000","last_peered":"2023-10-06T05:15:53.296398+0000","last_clean":"2023-10-06T05:15:53.296398+0000","last_became_active":"2023-10-06T05:15:53.289235+0000","last_became_peered":"2023-10-06T05:15:53.289235+0000","last_unstale":"2023-10-06T05:15:53.296398+0000","last_undegraded":"2023-10-06T05:15:53.296398+0000","last_fullsized":"2023-10-06T05:15:53.296398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:44.963751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.304676+0000","last_change":"2023-10-06T05:15:53.293551+0000","last_active":"2023-10-06T05:15:53.304676+0000","last_peered":"2023-10-06T05:15:53.304676+0000","last_clean":"2023-10-06T05:15:53.304676+0000","last_became_active":"2023-10-06T05:15:53.293001+0000","last_became_peered":"2023-10-06T05:15:53.293001+0000","last_unstale":"2023-10-06T05:15:53.304676+0000","last_undegraded":"2023-10-06T05:15:53.304676+0000","last_fullsized":"2023-10-06T05:15:53.304676+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:43.391788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.300973+0000","last_change":"2023-10-06T05:15:53.295691+0000","last_active":"2023-10-06T05:15:53.300973+0000","last_peered":"2023-10-06T05:15:53.300973+0000","last_clean":"2023-10-06T05:15:53.300973+0000","last_became_active":"2023-10-06T05:15:53.295397+0000","last_became_peered":"2023-10-06T05:15:53.295397+0000","last_unstale":"2023-10-06T05:15:53.300973+0000","last_undegraded":"2023-10-06T05:15:53.300973+0000","last_fullsized":"2023-10-06T05:15:53.300973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:41.418320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.311901+0000","last_change":"2023-10-06T05:15:53.307797+0000","last_active":"2023-10-06T05:15:53.311901+0000","last_peered":"2023-10-06T05:15:53.311901+0000","last_clean":"2023-10-06T05:15:53.311901+0000","last_became_active":"2023-10-06T05:15:53.307526+0000","last_became_peered":"2023-10-06T05:15:53.307526+0000","last_unstale":"2023-10-06T05:15:53.311901+0000","last_undegraded":"2023-10-06T05:15:53.311901+0000","last_fullsized":"2023-10-06T05:15:53.311901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:09.139828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307867+0000","last_change":"2023-10-06T05:15:53.297964+0000","last_active":"2023-10-06T05:15:53.307867+0000","last_peered":"2023-10-06T05:15:53.307867+0000","last_clean":"2023-10-06T05:15:53.307867+0000","last_became_active":"2023-10-06T05:15:53.294111+0000","last_became_peered":"2023-10-06T05:15:53.294111+0000","last_unstale":"2023-10-06T05:15:53.307867+0000","last_undegraded":"2023-10-06T05:15:53.307867+0000","last_fullsized":"2023-10-06T05:15:53.307867+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:53.153757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.309072+0000","last_change":"2023-10-06T05:15:53.302955+0000","last_active":"2023-10-06T05:15:53.309072+0000","last_peered":"2023-10-06T05:15:53.309072+0000","last_clean":"2023-10-06T05:15:53.309072+0000","last_became_active":"2023-10-06T05:15:53.302607+0000","last_became_peered":"2023-10-06T05:15:53.302607+0000","last_unstale":"2023-10-06T05:15:53.309072+0000","last_undegraded":"2023-10-06T05:15:53.309072+0000","last_fullsized":"2023-10-06T05:15:53.309072+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:52.859550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303577+0000","last_change":"2023-10-06T05:15:53.298143+0000","last_active":"2023-10-06T05:15:53.303577+0000","last_peered":"2023-10-06T05:15:53.303577+0000","last_clean":"2023-10-06T05:15:53.303577+0000","last_became_active":"2023-10-06T05:15:53.297898+0000","last_became_peered":"2023-10-06T05:15:53.297898+0000","last_unstale":"2023-10-06T05:15:53.303577+0000","last_undegraded":"2023-10-06T05:15:53.303577+0000","last_fullsized":"2023-10-06T05:15:53.303577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:02.794844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.302360+0000","last_change":"2023-10-06T05:15:53.291606+0000","last_active":"2023-10-06T05:15:53.302360+0000","last_peered":"2023-10-06T05:15:53.302360+0000","last_clean":"2023-10-06T05:15:53.302360+0000","last_became_active":"2023-10-06T05:15:53.290789+0000","last_became_peered":"2023-10-06T05:15:53.290789+0000","last_unstale":"2023-10-06T05:15:53.302360+0000","last_undegraded":"2023-10-06T05:15:53.302360+0000","last_fullsized":"2023-10-06T05:15:53.302360+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:40.853911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.285814+0000","last_change":"2023-10-06T05:15:53.282325+0000","last_active":"2023-10-06T05:15:53.285814+0000","last_peered":"2023-10-06T05:15:53.285814+0000","last_clean":"2023-10-06T05:15:53.285814+0000","last_became_active":"2023-10-06T05:15:53.282077+0000","last_became_peered":"2023-10-06T05:15:53.282077+0000","last_unstale":"2023-10-06T05:15:53.285814+0000","last_undegraded":"2023-10-06T05:15:53.285814+0000","last_fullsized":"2023-10-06T05:15:53.285814+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:48.798497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.293784+0000","last_change":"2023-10-06T05:15:53.291576+0000","last_active":"2023-10-06T05:15:53.293784+0000","last_peered":"2023-10-06T05:15:53.293784+0000","last_clean":"2023-10-06T05:15:53.293784+0000","last_became_active":"2023-10-06T05:15:53.291290+0000","last_became_peered":"2023-10-06T05:15:53.291290+0000","last_unstale":"2023-10-06T05:15:53.293784+0000","last_undegraded":"2023-10-06T05:15:53.293784+0000","last_fullsized":"2023-10-06T05:15:53.293784+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:13.141988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.298141+0000","last_change":"2023-10-06T05:15:53.289649+0000","last_active":"2023-10-06T05:15:53.298141+0000","last_peered":"2023-10-06T05:15:53.298141+0000","last_clean":"2023-10-06T05:15:53.298141+0000","last_became_active":"2023-10-06T05:15:53.289399+0000","last_became_peered":"2023-10-06T05:15:53.289399+0000","last_unstale":"2023-10-06T05:15:53.298141+0000","last_undegraded":"2023-10-06T05:15:53.298141+0000","last_fullsized":"2023-10-06T05:15:53.298141+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:48.798497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.304042+0000","last_change":"2023-10-06T05:15:53.294671+0000","last_active":"2023-10-06T05:15:53.304042+0000","last_peered":"2023-10-06T05:15:53.304042+0000","last_clean":"2023-10-06T05:15:53.304042+0000","last_became_active":"2023-10-06T05:15:53.294421+0000","last_became_peered":"2023-10-06T05:15:53.294421+0000","last_unstale":"2023-10-06T05:15:53.304042+0000","last_undegraded":"2023-10-06T05:15:53.304042+0000","last_fullsized":"2023-10-06T05:15:53.304042+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:31.403723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.292211+0000","last_change":"2023-10-06T05:15:53.288417+0000","last_active":"2023-10-06T05:15:53.292211+0000","last_peered":"2023-10-06T05:15:53.292211+0000","last_clean":"2023-10-06T05:15:53.292211+0000","last_became_active":"2023-10-06T05:15:53.288091+0000","last_became_peered":"2023-10-06T05:15:53.288091+0000","last_unstale":"2023-10-06T05:15:53.292211+0000","last_undegraded":"2023-10-06T05:15:53.292211+0000","last_fullsized":"2023-10-06T05:15:53.292211+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:40.145031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.309309+0000","last_change":"2023-10-06T05:15:53.308448+0000","last_active":"2023-10-06T05:15:53.309309+0000","last_peered":"2023-10-06T05:15:53.309309+0000","last_clean":"2023-10-06T05:15:53.309309+0000","last_became_active":"2023-10-06T05:15:53.308151+0000","last_became_peered":"2023-10-06T05:15:53.308151+0000","last_unstale":"2023-10-06T05:15:53.309309+0000","last_undegraded":"2023-10-06T05:15:53.309309+0000","last_fullsized":"2023-10-06T05:15:53.309309+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.959968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.302446+0000","last_change":"2023-10-06T05:15:53.293190+0000","last_active":"2023-10-06T05:15:53.302446+0000","last_peered":"2023-10-06T05:15:53.302446+0000","last_clean":"2023-10-06T05:15:53.302446+0000","last_became_active":"2023-10-06T05:15:53.292944+0000","last_became_peered":"2023-10-06T05:15:53.292944+0000","last_unstale":"2023-10-06T05:15:53.302446+0000","last_undegraded":"2023-10-06T05:15:53.302446+0000","last_fullsized":"2023-10-06T05:15:53.302446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:13.455034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.311762+0000","last_change":"2023-10-06T05:15:53.305173+0000","last_active":"2023-10-06T05:15:53.311762+0000","last_peered":"2023-10-06T05:15:53.311762+0000","last_clean":"2023-10-06T05:15:53.311762+0000","last_became_active":"2023-10-06T05:15:53.304864+0000","last_became_peered":"2023-10-06T05:15:53.304864+0000","last_unstale":"2023-10-06T05:15:53.311762+0000","last_undegraded":"2023-10-06T05:15:53.311762+0000","last_fullsized":"2023-10-06T05:15:53.311762+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:41.133991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307825+0000","last_change":"2023-10-06T05:15:53.299028+0000","last_active":"2023-10-06T05:15:53.307825+0000","last_peered":"2023-10-06T05:15:53.307825+0000","last_clean":"2023-10-06T05:15:53.307825+0000","last_became_active":"2023-10-06T05:15:53.295042+0000","last_became_peered":"2023-10-06T05:15:53.295042+0000","last_unstale":"2023-10-06T05:15:53.307825+0000","last_undegraded":"2023-10-06T05:15:53.307825+0000","last_fullsized":"2023-10-06T05:15:53.307825+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:26.979030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.298065+0000","last_change":"2023-10-06T05:15:53.289546+0000","last_active":"2023-10-06T05:15:53.298065+0000","last_peered":"2023-10-06T05:15:53.298065+0000","last_clean":"2023-10-06T05:15:53.298065+0000","last_became_active":"2023-10-06T05:15:53.289248+0000","last_became_peered":"2023-10-06T05:15:53.289248+0000","last_unstale":"2023-10-06T05:15:53.298065+0000","last_undegraded":"2023-10-06T05:15:53.298065+0000","last_fullsized":"2023-10-06T05:15:53.298065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:56.067385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.295733+0000","last_change":"2023-10-06T05:15:53.289123+0000","last_active":"2023-10-06T05:15:53.295733+0000","last_peered":"2023-10-06T05:15:53.295733+0000","last_clean":"2023-10-06T05:15:53.295733+0000","last_became_active":"2023-10-06T05:15:53.288847+0000","last_became_peered":"2023-10-06T05:15:53.288847+0000","last_unstale":"2023-10-06T05:15:53.295733+0000","last_undegraded":"2023-10-06T05:15:53.295733+0000","last_fullsized":"2023-10-06T05:15:53.295733+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:26.979030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.304618+0000","last_change":"2023-10-06T05:15:53.293753+0000","last_active":"2023-10-06T05:15:53.304618+0000","last_peered":"2023-10-06T05:15:53.304618+0000","last_clean":"2023-10-06T05:15:53.304618+0000","last_became_active":"2023-10-06T05:15:53.293497+0000","last_became_peered":"2023-10-06T05:15:53.293497+0000","last_unstale":"2023-10-06T05:15:53.304618+0000","last_undegraded":"2023-10-06T05:15:53.304618+0000","last_fullsized":"2023-10-06T05:15:53.304618+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:52.975249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.302219+0000","last_change":"2023-10-06T05:15:52.290248+0000","last_active":"2023-10-06T05:15:53.302219+0000","last_peered":"2023-10-06T05:15:53.302219+0000","last_clean":"2023-10-06T05:15:53.302219+0000","last_became_active":"2023-10-06T05:15:52.288793+0000","last_became_peered":"2023-10-06T05:15:52.288793+0000","last_unstale":"2023-10-06T05:15:53.302219+0000","last_undegraded":"2023-10-06T05:15:53.302219+0000","last_fullsized":"2023-10-06T05:15:53.302219+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:50.244078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303258+0000","last_change":"2023-10-06T05:15:53.296107+0000","last_active":"2023-10-06T05:15:53.303258+0000","last_peered":"2023-10-06T05:15:53.303258+0000","last_clean":"2023-10-06T05:15:53.303258+0000","last_became_active":"2023-10-06T05:15:53.295834+0000","last_became_peered":"2023-10-06T05:15:53.295834+0000","last_unstale":"2023-10-06T05:15:53.303258+0000","last_undegraded":"2023-10-06T05:15:53.303258+0000","last_fullsized":"2023-10-06T05:15:53.303258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:18.959968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308790+0000","last_change":"2023-10-06T05:15:52.290452+0000","last_active":"2023-10-06T05:15:53.308790+0000","last_peered":"2023-10-06T05:15:53.308790+0000","last_clean":"2023-10-06T05:15:53.308790+0000","last_became_active":"2023-10-06T05:15:52.290004+0000","last_became_peered":"2023-10-06T05:15:52.290004+0000","last_unstale":"2023-10-06T05:15:53.308790+0000","last_undegraded":"2023-10-06T05:15:53.308790+0000","last_fullsized":"2023-10-06T05:15:53.308790+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:01.785482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303284+0000","last_change":"2023-10-06T05:15:53.294785+0000","last_active":"2023-10-06T05:15:53.303284+0000","last_peered":"2023-10-06T05:15:53.303284+0000","last_clean":"2023-10-06T05:15:53.303284+0000","last_became_active":"2023-10-06T05:15:53.291947+0000","last_became_peered":"2023-10-06T05:15:53.291947+0000","last_unstale":"2023-10-06T05:15:53.303284+0000","last_undegraded":"2023-10-06T05:15:53.303284+0000","last_fullsized":"2023-10-06T05:15:53.303284+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:15.311151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.306056+0000","last_change":"2023-10-06T05:15:52.294412+0000","last_active":"2023-10-06T05:15:53.306056+0000","last_peered":"2023-10-06T05:15:53.306056+0000","last_clean":"2023-10-06T05:15:53.306056+0000","last_became_active":"2023-10-06T05:15:52.293816+0000","last_became_peered":"2023-10-06T05:15:52.293816+0000","last_unstale":"2023-10-06T05:15:53.306056+0000","last_undegraded":"2023-10-06T05:15:53.306056+0000","last_fullsized":"2023-10-06T05:15:53.306056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:50.118121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.285563+0000","last_change":"2023-10-06T05:15:53.282869+0000","last_active":"2023-10-06T05:15:53.285563+0000","last_peered":"2023-10-06T05:15:53.285563+0000","last_clean":"2023-10-06T05:15:53.285563+0000","last_became_active":"2023-10-06T05:15:53.282600+0000","last_became_peered":"2023-10-06T05:15:53.282600+0000","last_unstale":"2023-10-06T05:15:53.285563+0000","last_undegraded":"2023-10-06T05:15:53.285563+0000","last_fullsized":"2023-10-06T05:15:53.285563+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:09.139828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.310767+0000","last_change":"2023-10-06T05:15:52.290015+0000","last_active":"2023-10-06T05:15:53.310767+0000","last_peered":"2023-10-06T05:15:53.310767+0000","last_clean":"2023-10-06T05:15:53.310767+0000","last_became_active":"2023-10-06T05:15:52.289717+0000","last_became_peered":"2023-10-06T05:15:52.289717+0000","last_unstale":"2023-10-06T05:15:53.310767+0000","last_undegraded":"2023-10-06T05:15:53.310767+0000","last_fullsized":"2023-10-06T05:15:53.310767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:27.527005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308119+0000","last_change":"2023-10-06T05:15:53.305263+0000","last_active":"2023-10-06T05:15:53.308119+0000","last_peered":"2023-10-06T05:15:53.308119+0000","last_clean":"2023-10-06T05:15:53.308119+0000","last_became_active":"2023-10-06T05:15:53.304261+0000","last_became_peered":"2023-10-06T05:15:53.304261+0000","last_unstale":"2023-10-06T05:15:53.308119+0000","last_undegraded":"2023-10-06T05:15:53.308119+0000","last_fullsized":"2023-10-06T05:15:53.308119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:53.153757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.314984+0000","last_change":"2023-10-06T05:15:52.285348+0000","last_active":"2023-10-06T05:15:53.314984+0000","last_peered":"2023-10-06T05:15:53.314984+0000","last_clean":"2023-10-06T05:15:53.314984+0000","last_became_active":"2023-10-06T05:15:52.283222+0000","last_became_peered":"2023-10-06T05:15:52.283222+0000","last_unstale":"2023-10-06T05:15:53.314984+0000","last_undegraded":"2023-10-06T05:15:53.314984+0000","last_fullsized":"2023-10-06T05:15:53.314984+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:47.422625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307971+0000","last_change":"2023-10-06T05:15:53.298481+0000","last_active":"2023-10-06T05:15:53.307971+0000","last_peered":"2023-10-06T05:15:53.307971+0000","last_clean":"2023-10-06T05:15:53.307971+0000","last_became_active":"2023-10-06T05:15:53.295064+0000","last_became_peered":"2023-10-06T05:15:53.295064+0000","last_unstale":"2023-10-06T05:15:53.307971+0000","last_undegraded":"2023-10-06T05:15:53.307971+0000","last_fullsized":"2023-10-06T05:15:53.307971+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:07.469800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.306283+0000","last_change":"2023-10-06T05:15:52.288778+0000","last_active":"2023-10-06T05:15:53.306283+0000","last_peered":"2023-10-06T05:15:53.306283+0000","last_clean":"2023-10-06T05:15:53.306283+0000","last_became_active":"2023-10-06T05:15:52.288498+0000","last_became_peered":"2023-10-06T05:15:52.288498+0000","last_unstale":"2023-10-06T05:15:53.306283+0000","last_undegraded":"2023-10-06T05:15:53.306283+0000","last_fullsized":"2023-10-06T05:15:53.306283+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:08.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.288965+0000","last_change":"2023-10-06T05:15:53.287749+0000","last_active":"2023-10-06T05:15:53.288965+0000","last_peered":"2023-10-06T05:15:53.288965+0000","last_clean":"2023-10-06T05:15:53.288965+0000","last_became_active":"2023-10-06T05:15:53.287485+0000","last_became_peered":"2023-10-06T05:15:53.287485+0000","last_unstale":"2023-10-06T05:15:53.288965+0000","last_undegraded":"2023-10-06T05:15:53.288965+0000","last_fullsized":"2023-10-06T05:15:53.288965+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:36.486288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.283657+0000","last_change":"2023-10-06T05:15:52.286785+0000","last_active":"2023-10-06T05:15:53.283657+0000","last_peered":"2023-10-06T05:15:53.283657+0000","last_clean":"2023-10-06T05:15:53.283657+0000","last_became_active":"2023-10-06T05:15:52.286495+0000","last_became_peered":"2023-10-06T05:15:52.286495+0000","last_unstale":"2023-10-06T05:15:53.283657+0000","last_undegraded":"2023-10-06T05:15:53.283657+0000","last_fullsized":"2023-10-06T05:15:53.283657+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:50.641843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307949+0000","last_change":"2023-10-06T05:15:53.299368+0000","last_active":"2023-10-06T05:15:53.307949+0000","last_peered":"2023-10-06T05:15:53.307949+0000","last_clean":"2023-10-06T05:15:53.307949+0000","last_became_active":"2023-10-06T05:15:53.294545+0000","last_became_peered":"2023-10-06T05:15:53.294545+0000","last_unstale":"2023-10-06T05:15:53.307949+0000","last_undegraded":"2023-10-06T05:15:53.307949+0000","last_fullsized":"2023-10-06T05:15:53.307949+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:18.229912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.274511+0000","last_change":"2023-10-06T05:15:52.292886+0000","last_active":"2023-10-06T05:15:53.274511+0000","last_peered":"2023-10-06T05:15:53.274511+0000","last_clean":"2023-10-06T05:15:53.274511+0000","last_became_active":"2023-10-06T05:15:52.287322+0000","last_became_peered":"2023-10-06T05:15:52.287322+0000","last_unstale":"2023-10-06T05:15:53.274511+0000","last_undegraded":"2023-10-06T05:15:53.274511+0000","last_fullsized":"2023-10-06T05:15:53.274511+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:32.093973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.296503+0000","last_change":"2023-10-06T05:15:53.288008+0000","last_active":"2023-10-06T05:15:53.296503+0000","last_peered":"2023-10-06T05:15:53.296503+0000","last_clean":"2023-10-06T05:15:53.296503+0000","last_became_active":"2023-10-06T05:15:53.287735+0000","last_became_peered":"2023-10-06T05:15:53.287735+0000","last_unstale":"2023-10-06T05:15:53.296503+0000","last_undegraded":"2023-10-06T05:15:53.296503+0000","last_fullsized":"2023-10-06T05:15:53.296503+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:26.979030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.275938+0000","last_change":"2023-10-06T05:15:52.292737+0000","last_active":"2023-10-06T05:15:53.275938+0000","last_peered":"2023-10-06T05:15:53.275938+0000","last_clean":"2023-10-06T05:15:53.275938+0000","last_became_active":"2023-10-06T05:15:52.292397+0000","last_became_peered":"2023-10-06T05:15:52.292397+0000","last_unstale":"2023-10-06T05:15:53.275938+0000","last_undegraded":"2023-10-06T05:15:53.275938+0000","last_fullsized":"2023-10-06T05:15:53.275938+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.335935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301530+0000","last_change":"2023-10-06T05:15:53.295035+0000","last_active":"2023-10-06T05:15:53.301530+0000","last_peered":"2023-10-06T05:15:53.301530+0000","last_clean":"2023-10-06T05:15:53.301530+0000","last_became_active":"2023-10-06T05:15:53.294739+0000","last_became_peered":"2023-10-06T05:15:53.294739+0000","last_unstale":"2023-10-06T05:15:53.301530+0000","last_undegraded":"2023-10-06T05:15:53.301530+0000","last_fullsized":"2023-10-06T05:15:53.301530+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:52.975249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.279764+0000","last_change":"2023-10-06T05:15:52.298195+0000","last_active":"2023-10-06T05:15:53.279764+0000","last_peered":"2023-10-06T05:15:53.279764+0000","last_clean":"2023-10-06T05:15:53.279764+0000","last_became_active":"2023-10-06T05:15:52.297550+0000","last_became_peered":"2023-10-06T05:15:52.297550+0000","last_unstale":"2023-10-06T05:15:53.279764+0000","last_undegraded":"2023-10-06T05:15:53.279764+0000","last_fullsized":"2023-10-06T05:15:53.279764+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:40.408958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.311718+0000","last_change":"2023-10-06T05:15:53.305770+0000","last_active":"2023-10-06T05:15:53.311718+0000","last_peered":"2023-10-06T05:15:53.311718+0000","last_clean":"2023-10-06T05:15:53.311718+0000","last_became_active":"2023-10-06T05:15:53.305507+0000","last_became_peered":"2023-10-06T05:15:53.305507+0000","last_unstale":"2023-10-06T05:15:53.311718+0000","last_undegraded":"2023-10-06T05:15:53.311718+0000","last_fullsized":"2023-10-06T05:15:53.311718+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:17.095627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.305280+0000","last_change":"2023-10-06T05:15:52.292685+0000","last_active":"2023-10-06T05:15:53.305280+0000","last_peered":"2023-10-06T05:15:53.305280+0000","last_clean":"2023-10-06T05:15:53.305280+0000","last_became_active":"2023-10-06T05:15:52.292345+0000","last_became_peered":"2023-10-06T05:15:52.292345+0000","last_unstale":"2023-10-06T05:15:53.305280+0000","last_undegraded":"2023-10-06T05:15:53.305280+0000","last_fullsized":"2023-10-06T05:15:53.305280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:27.527005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.296994+0000","last_change":"2023-10-06T05:15:53.293194+0000","last_active":"2023-10-06T05:15:53.296994+0000","last_peered":"2023-10-06T05:15:53.296994+0000","last_clean":"2023-10-06T05:15:53.296994+0000","last_became_active":"2023-10-06T05:15:53.292906+0000","last_became_peered":"2023-10-06T05:15:53.292906+0000","last_unstale":"2023-10-06T05:15:53.296994+0000","last_undegraded":"2023-10-06T05:15:53.296994+0000","last_fullsized":"2023-10-06T05:15:53.296994+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:52.975249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.316792+0000","last_change":"2023-10-06T05:15:52.295080+0000","last_active":"2023-10-06T05:15:53.316792+0000","last_peered":"2023-10-06T05:15:53.316792+0000","last_clean":"2023-10-06T05:15:53.316792+0000","last_became_active":"2023-10-06T05:15:52.294768+0000","last_became_peered":"2023-10-06T05:15:52.294768+0000","last_unstale":"2023-10-06T05:15:53.316792+0000","last_undegraded":"2023-10-06T05:15:53.316792+0000","last_fullsized":"2023-10-06T05:15:53.316792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:12.326255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301763+0000","last_change":"2023-10-06T05:15:53.296132+0000","last_active":"2023-10-06T05:15:53.301763+0000","last_peered":"2023-10-06T05:15:53.301763+0000","last_clean":"2023-10-06T05:15:53.301763+0000","last_became_active":"2023-10-06T05:15:53.295812+0000","last_became_peered":"2023-10-06T05:15:53.295812+0000","last_unstale":"2023-10-06T05:15:53.301763+0000","last_undegraded":"2023-10-06T05:15:53.301763+0000","last_fullsized":"2023-10-06T05:15:53.301763+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:43.391788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.315070+0000","last_change":"2023-10-06T05:15:52.291240+0000","last_active":"2023-10-06T05:15:53.315070+0000","last_peered":"2023-10-06T05:15:53.315070+0000","last_clean":"2023-10-06T05:15:53.315070+0000","last_became_active":"2023-10-06T05:15:52.287360+0000","last_became_peered":"2023-10-06T05:15:52.287360+0000","last_unstale":"2023-10-06T05:15:53.315070+0000","last_undegraded":"2023-10-06T05:15:53.315070+0000","last_fullsized":"2023-10-06T05:15:53.315070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.335935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.304593+0000","last_change":"2023-10-06T05:15:53.301367+0000","last_active":"2023-10-06T05:15:53.304593+0000","last_peered":"2023-10-06T05:15:53.304593+0000","last_clean":"2023-10-06T05:15:53.304593+0000","last_became_active":"2023-10-06T05:15:53.301083+0000","last_became_peered":"2023-10-06T05:15:53.301083+0000","last_unstale":"2023-10-06T05:15:53.304593+0000","last_undegraded":"2023-10-06T05:15:53.304593+0000","last_fullsized":"2023-10-06T05:15:53.304593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:57.223358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.282809+0000","last_change":"2023-10-06T05:15:52.298359+0000","last_active":"2023-10-06T05:15:53.282809+0000","last_peered":"2023-10-06T05:15:53.282809+0000","last_clean":"2023-10-06T05:15:53.282809+0000","last_became_active":"2023-10-06T05:15:52.298009+0000","last_became_peered":"2023-10-06T05:15:52.298009+0000","last_unstale":"2023-10-06T05:15:53.282809+0000","last_undegraded":"2023-10-06T05:15:53.282809+0000","last_fullsized":"2023-10-06T05:15:53.282809+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:59.682232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.299118+0000","last_change":"2023-10-06T05:15:53.294570+0000","last_active":"2023-10-06T05:15:53.299118+0000","last_peered":"2023-10-06T05:15:53.299118+0000","last_clean":"2023-10-06T05:15:53.299118+0000","last_became_active":"2023-10-06T05:15:53.294260+0000","last_became_peered":"2023-10-06T05:15:53.294260+0000","last_unstale":"2023-10-06T05:15:53.299118+0000","last_undegraded":"2023-10-06T05:15:53.299118+0000","last_fullsized":"2023-10-06T05:15:53.299118+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:45.892078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.275642+0000","last_change":"2023-10-06T05:15:52.291621+0000","last_active":"2023-10-06T05:15:53.275642+0000","last_peered":"2023-10-06T05:15:53.275642+0000","last_clean":"2023-10-06T05:15:53.275642+0000","last_became_active":"2023-10-06T05:15:52.291299+0000","last_became_peered":"2023-10-06T05:15:52.291299+0000","last_unstale":"2023-10-06T05:15:53.275642+0000","last_undegraded":"2023-10-06T05:15:53.275642+0000","last_fullsized":"2023-10-06T05:15:53.275642+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:23.905925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.311802+0000","last_change":"2023-10-06T05:15:53.306783+0000","last_active":"2023-10-06T05:15:53.311802+0000","last_peered":"2023-10-06T05:15:53.311802+0000","last_clean":"2023-10-06T05:15:53.311802+0000","last_became_active":"2023-10-06T05:15:53.306517+0000","last_became_peered":"2023-10-06T05:15:53.306517+0000","last_unstale":"2023-10-06T05:15:53.311802+0000","last_undegraded":"2023-10-06T05:15:53.311802+0000","last_fullsized":"2023-10-06T05:15:53.311802+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:19.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.311340+0000","last_change":"2023-10-06T05:15:52.292514+0000","last_active":"2023-10-06T05:15:53.311340+0000","last_peered":"2023-10-06T05:15:53.311340+0000","last_clean":"2023-10-06T05:15:53.311340+0000","last_became_active":"2023-10-06T05:15:52.292089+0000","last_became_peered":"2023-10-06T05:15:52.292089+0000","last_unstale":"2023-10-06T05:15:53.311340+0000","last_undegraded":"2023-10-06T05:15:53.311340+0000","last_fullsized":"2023-10-06T05:15:53.311340+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:58.887997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308340+0000","last_change":"2023-10-06T05:15:53.298260+0000","last_active":"2023-10-06T05:15:53.308340+0000","last_peered":"2023-10-06T05:15:53.308340+0000","last_clean":"2023-10-06T05:15:53.308340+0000","last_became_active":"2023-10-06T05:15:53.297788+0000","last_became_peered":"2023-10-06T05:15:53.297788+0000","last_unstale":"2023-10-06T05:15:53.308340+0000","last_undegraded":"2023-10-06T05:15:53.308340+0000","last_fullsized":"2023-10-06T05:15:53.308340+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:42.517049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.315025+0000","last_change":"2023-10-06T05:15:52.295542+0000","last_active":"2023-10-06T05:15:53.315025+0000","last_peered":"2023-10-06T05:15:53.315025+0000","last_clean":"2023-10-06T05:15:53.315025+0000","last_became_active":"2023-10-06T05:15:52.295264+0000","last_became_peered":"2023-10-06T05:15:52.295264+0000","last_unstale":"2023-10-06T05:15:53.315025+0000","last_undegraded":"2023-10-06T05:15:53.315025+0000","last_fullsized":"2023-10-06T05:15:53.315025+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:40.408958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308795+0000","last_change":"2023-10-06T05:15:53.307678+0000","last_active":"2023-10-06T05:15:53.308795+0000","last_peered":"2023-10-06T05:15:53.308795+0000","last_clean":"2023-10-06T05:15:53.308795+0000","last_became_active":"2023-10-06T05:15:53.307470+0000","last_became_peered":"2023-10-06T05:15:53.307470+0000","last_unstale":"2023-10-06T05:15:53.308795+0000","last_undegraded":"2023-10-06T05:15:53.308795+0000","last_fullsized":"2023-10-06T05:15:53.308795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:52.975249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307918+0000","last_change":"2023-10-06T05:15:52.280268+0000","last_active":"2023-10-06T05:15:53.307918+0000","last_peered":"2023-10-06T05:15:53.307918+0000","last_clean":"2023-10-06T05:15:53.307918+0000","last_became_active":"2023-10-06T05:15:52.279984+0000","last_became_peered":"2023-10-06T05:15:52.279984+0000","last_unstale":"2023-10-06T05:15:53.307918+0000","last_undegraded":"2023-10-06T05:15:53.307918+0000","last_fullsized":"2023-10-06T05:15:53.307918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:09.731404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308136+0000","last_change":"2023-10-06T05:15:53.304981+0000","last_active":"2023-10-06T05:15:53.308136+0000","last_peered":"2023-10-06T05:15:53.308136+0000","last_clean":"2023-10-06T05:15:53.308136+0000","last_became_active":"2023-10-06T05:15:53.304492+0000","last_became_peered":"2023-10-06T05:15:53.304492+0000","last_unstale":"2023-10-06T05:15:53.308136+0000","last_undegraded":"2023-10-06T05:15:53.308136+0000","last_fullsized":"2023-10-06T05:15:53.308136+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:28.391326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303436+0000","last_change":"2023-10-06T05:15:52.294447+0000","last_active":"2023-10-06T05:15:53.303436+0000","last_peered":"2023-10-06T05:15:53.303436+0000","last_clean":"2023-10-06T05:15:53.303436+0000","last_became_active":"2023-10-06T05:15:52.294069+0000","last_became_peered":"2023-10-06T05:15:52.294069+0000","last_unstale":"2023-10-06T05:15:53.303436+0000","last_undegraded":"2023-10-06T05:15:53.303436+0000","last_fullsized":"2023-10-06T05:15:53.303436+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.132626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301005+0000","last_change":"2023-10-06T05:15:53.295701+0000","last_active":"2023-10-06T05:15:53.301005+0000","last_peered":"2023-10-06T05:15:53.301005+0000","last_clean":"2023-10-06T05:15:53.301005+0000","last_became_active":"2023-10-06T05:15:53.295411+0000","last_became_peered":"2023-10-06T05:15:53.295411+0000","last_unstale":"2023-10-06T05:15:53.301005+0000","last_undegraded":"2023-10-06T05:15:53.301005+0000","last_fullsized":"2023-10-06T05:15:53.301005+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:32.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.283028+0000","last_change":"2023-10-06T05:15:52.291494+0000","last_active":"2023-10-06T05:15:53.283028+0000","last_peered":"2023-10-06T05:15:53.283028+0000","last_clean":"2023-10-06T05:15:53.283028+0000","last_became_active":"2023-10-06T05:15:52.291176+0000","last_became_peered":"2023-10-06T05:15:52.291176+0000","last_unstale":"2023-10-06T05:15:53.283028+0000","last_undegraded":"2023-10-06T05:15:53.283028+0000","last_fullsized":"2023-10-06T05:15:53.283028+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:09.731404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.295699+0000","last_change":"2023-10-06T05:15:53.292143+0000","last_active":"2023-10-06T05:15:53.295699+0000","last_peered":"2023-10-06T05:15:53.295699+0000","last_clean":"2023-10-06T05:15:53.295699+0000","last_became_active":"2023-10-06T05:15:53.291769+0000","last_became_peered":"2023-10-06T05:15:53.291769+0000","last_unstale":"2023-10-06T05:15:53.295699+0000","last_undegraded":"2023-10-06T05:15:53.295699+0000","last_fullsized":"2023-10-06T05:15:53.295699+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:53.153757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.308897+0000","last_change":"2023-10-06T05:15:52.292120+0000","last_active":"2023-10-06T05:15:53.308897+0000","last_peered":"2023-10-06T05:15:53.308897+0000","last_clean":"2023-10-06T05:15:53.308897+0000","last_became_active":"2023-10-06T05:15:52.291829+0000","last_became_peered":"2023-10-06T05:15:52.291829+0000","last_unstale":"2023-10-06T05:15:53.308897+0000","last_undegraded":"2023-10-06T05:15:53.308897+0000","last_fullsized":"2023-10-06T05:15:53.308897+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:35.623139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303488+0000","last_change":"2023-10-06T05:15:53.297085+0000","last_active":"2023-10-06T05:15:53.303488+0000","last_peered":"2023-10-06T05:15:53.303488+0000","last_clean":"2023-10-06T05:15:53.303488+0000","last_became_active":"2023-10-06T05:15:53.296807+0000","last_became_peered":"2023-10-06T05:15:53.296807+0000","last_unstale":"2023-10-06T05:15:53.303488+0000","last_undegraded":"2023-10-06T05:15:53.303488+0000","last_fullsized":"2023-10-06T05:15:53.303488+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:51.127483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.278447+0000","last_change":"2023-10-06T05:15:52.292649+0000","last_active":"2023-10-06T05:15:53.278447+0000","last_peered":"2023-10-06T05:15:53.278447+0000","last_clean":"2023-10-06T05:15:53.278447+0000","last_became_active":"2023-10-06T05:15:52.292321+0000","last_became_peered":"2023-10-06T05:15:52.292321+0000","last_unstale":"2023-10-06T05:15:53.278447+0000","last_undegraded":"2023-10-06T05:15:53.278447+0000","last_fullsized":"2023-10-06T05:15:53.278447+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:31.085646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.309899+0000","last_change":"2023-10-06T05:15:53.303671+0000","last_active":"2023-10-06T05:15:53.309899+0000","last_peered":"2023-10-06T05:15:53.309899+0000","last_clean":"2023-10-06T05:15:53.309899+0000","last_became_active":"2023-10-06T05:15:53.303393+0000","last_became_peered":"2023-10-06T05:15:53.303393+0000","last_unstale":"2023-10-06T05:15:53.309899+0000","last_undegraded":"2023-10-06T05:15:53.309899+0000","last_fullsized":"2023-10-06T05:15:53.309899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:28.663087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.274416+0000","last_change":"2023-10-06T05:15:52.297391+0000","last_active":"2023-10-06T05:15:53.274416+0000","last_peered":"2023-10-06T05:15:53.274416+0000","last_clean":"2023-10-06T05:15:53.274416+0000","last_became_active":"2023-10-06T05:15:52.297134+0000","last_became_peered":"2023-10-06T05:15:52.297134+0000","last_unstale":"2023-10-06T05:15:53.274416+0000","last_undegraded":"2023-10-06T05:15:53.274416+0000","last_fullsized":"2023-10-06T05:15:53.274416+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:58.887997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301138+0000","last_change":"2023-10-06T05:15:53.296472+0000","last_active":"2023-10-06T05:15:53.301138+0000","last_peered":"2023-10-06T05:15:53.301138+0000","last_clean":"2023-10-06T05:15:53.301138+0000","last_became_active":"2023-10-06T05:15:53.296107+0000","last_became_peered":"2023-10-06T05:15:53.296107+0000","last_unstale":"2023-10-06T05:15:53.301138+0000","last_undegraded":"2023-10-06T05:15:53.301138+0000","last_fullsized":"2023-10-06T05:15:53.301138+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:13.335617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.279628+0000","last_change":"2023-10-06T05:15:52.285488+0000","last_active":"2023-10-06T05:15:53.279628+0000","last_peered":"2023-10-06T05:15:53.279628+0000","last_clean":"2023-10-06T05:15:53.279628+0000","last_became_active":"2023-10-06T05:15:52.285187+0000","last_became_peered":"2023-10-06T05:15:52.285187+0000","last_unstale":"2023-10-06T05:15:53.279628+0000","last_undegraded":"2023-10-06T05:15:53.279628+0000","last_fullsized":"2023-10-06T05:15:53.279628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:14.699179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.301151+0000","last_change":"2023-10-06T05:15:53.289947+0000","last_active":"2023-10-06T05:15:53.301151+0000","last_peered":"2023-10-06T05:15:53.301151+0000","last_clean":"2023-10-06T05:15:53.301151+0000","last_became_active":"2023-10-06T05:15:53.289658+0000","last_became_peered":"2023-10-06T05:15:53.289658+0000","last_unstale":"2023-10-06T05:15:53.301151+0000","last_undegraded":"2023-10-06T05:15:53.301151+0000","last_fullsized":"2023-10-06T05:15:53.301151+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:43.391788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.279630+0000","last_change":"2023-10-06T05:15:52.290209+0000","last_active":"2023-10-06T05:15:53.279630+0000","last_peered":"2023-10-06T05:15:53.279630+0000","last_clean":"2023-10-06T05:15:53.279630+0000","last_became_active":"2023-10-06T05:15:52.289882+0000","last_became_peered":"2023-10-06T05:15:52.289882+0000","last_unstale":"2023-10-06T05:15:53.279630+0000","last_undegraded":"2023-10-06T05:15:53.279630+0000","last_fullsized":"2023-10-06T05:15:53.279630+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:38.992778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.284679+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:15:53.284679+0000","last_peered":"2023-10-06T05:15:53.284679+0000","last_clean":"2023-10-06T05:15:53.284679+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:15:53.284679+0000","last_undegraded":"2023-10-06T05:15:53.284679+0000","last_fullsized":"2023-10-06T05:15:53.284679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303343+0000","last_change":"2023-10-06T05:15:53.298725+0000","last_active":"2023-10-06T05:15:53.303343+0000","last_peered":"2023-10-06T05:15:53.303343+0000","last_clean":"2023-10-06T05:15:53.303343+0000","last_became_active":"2023-10-06T05:15:53.298393+0000","last_became_peered":"2023-10-06T05:15:53.298393+0000","last_unstale":"2023-10-06T05:15:53.303343+0000","last_undegraded":"2023-10-06T05:15:53.303343+0000","last_fullsized":"2023-10-06T05:15:53.303343+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:28.391326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.307446+0000","last_change":"2023-10-06T05:15:52.297972+0000","last_active":"2023-10-06T05:15:53.307446+0000","last_peered":"2023-10-06T05:15:53.307446+0000","last_clean":"2023-10-06T05:15:53.307446+0000","last_became_active":"2023-10-06T05:15:52.297454+0000","last_became_peered":"2023-10-06T05:15:52.297454+0000","last_unstale":"2023-10-06T05:15:53.307446+0000","last_undegraded":"2023-10-06T05:15:53.307446+0000","last_fullsized":"2023-10-06T05:15:53.307446+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:04.690643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.281749+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:15:53.281749+0000","last_peered":"2023-10-06T05:15:53.281749+0000","last_clean":"2023-10-06T05:15:53.281749+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:15:53.281749+0000","last_undegraded":"2023-10-06T05:15:53.281749+0000","last_fullsized":"2023-10-06T05:15:53.281749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.292940+0000","last_change":"2023-10-06T05:15:53.291269+0000","last_active":"2023-10-06T05:15:53.292940+0000","last_peered":"2023-10-06T05:15:53.292940+0000","last_clean":"2023-10-06T05:15:53.292940+0000","last_became_active":"2023-10-06T05:15:53.290982+0000","last_became_peered":"2023-10-06T05:15:53.290982+0000","last_unstale":"2023-10-06T05:15:53.292940+0000","last_undegraded":"2023-10-06T05:15:53.292940+0000","last_fullsized":"2023-10-06T05:15:53.292940+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:31.403723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.276630+0000","last_change":"2023-10-06T05:15:52.299565+0000","last_active":"2023-10-06T05:15:53.276630+0000","last_peered":"2023-10-06T05:15:53.276630+0000","last_clean":"2023-10-06T05:15:53.276630+0000","last_became_active":"2023-10-06T05:15:52.298445+0000","last_became_peered":"2023-10-06T05:15:52.298445+0000","last_unstale":"2023-10-06T05:15:53.276630+0000","last_undegraded":"2023-10-06T05:15:53.276630+0000","last_fullsized":"2023-10-06T05:15:53.276630+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:38.992778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.279617+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:15:53.279617+0000","last_peered":"2023-10-06T05:15:53.279617+0000","last_clean":"2023-10-06T05:15:53.279617+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:15:53.279617+0000","last_undegraded":"2023-10-06T05:15:53.279617+0000","last_fullsized":"2023-10-06T05:15:53.279617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303751+0000","last_change":"2023-10-06T05:15:53.299205+0000","last_active":"2023-10-06T05:15:53.303751+0000","last_peered":"2023-10-06T05:15:53.303751+0000","last_clean":"2023-10-06T05:15:53.303751+0000","last_became_active":"2023-10-06T05:15:53.298943+0000","last_became_peered":"2023-10-06T05:15:53.298943+0000","last_unstale":"2023-10-06T05:15:53.303751+0000","last_undegraded":"2023-10-06T05:15:53.303751+0000","last_fullsized":"2023-10-06T05:15:53.303751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:36.632501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"48'1","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-06T05:15:54.545514+0000","last_change":"2023-10-06T05:15:52.291261+0000","last_active":"2023-10-06T05:15:54.545514+0000","last_peered":"2023-10-06T05:15:54.545514+0000","last_clean":"2023-10-06T05:15:54.545514+0000","last_became_active":"2023-10-06T05:15:52.287569+0000","last_became_peered":"2023-10-06T05:15:52.287569+0000","last_unstale":"2023-10-06T05:15:54.545514+0000","last_undegraded":"2023-10-06T05:15:54.545514+0000","last_fullsized":"2023-10-06T05:15:54.545514+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:08.647159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":33,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.281319+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:15:53.281319+0000","last_peered":"2023-10-06T05:15:53.281319+0000","last_clean":"2023-10-06T05:15:53.281319+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:15:53.281319+0000","last_undegraded":"2023-10-06T05:15:53.281319+0000","last_fullsized":"2023-10-06T05:15:53.281319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.274835+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:15:53.274835+0000","last_peered":"2023-10-06T05:15:53.274835+0000","last_clean":"2023-10-06T05:15:53.274835+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:15:53.274835+0000","last_undegraded":"2023-10-06T05:15:53.274835+0000","last_fullsized":"2023-10-06T05:15:53.274835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.295891+0000","last_change":"2023-10-06T05:15:53.293135+0000","last_active":"2023-10-06T05:15:53.295891+0000","last_peered":"2023-10-06T05:15:53.295891+0000","last_clean":"2023-10-06T05:15:53.295891+0000","last_became_active":"2023-10-06T05:15:53.292895+0000","last_became_peered":"2023-10-06T05:15:53.292895+0000","last_unstale":"2023-10-06T05:15:53.295891+0000","last_undegraded":"2023-10-06T05:15:53.295891+0000","last_fullsized":"2023-10-06T05:15:53.295891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:59.897359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.275672+0000","last_change":"2023-10-06T05:15:52.291024+0000","last_active":"2023-10-06T05:15:53.275672+0000","last_peered":"2023-10-06T05:15:53.275672+0000","last_clean":"2023-10-06T05:15:53.275672+0000","last_became_active":"2023-10-06T05:15:52.290696+0000","last_became_peered":"2023-10-06T05:15:52.290696+0000","last_unstale":"2023-10-06T05:15:53.275672+0000","last_undegraded":"2023-10-06T05:15:53.275672+0000","last_fullsized":"2023-10-06T05:15:53.275672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:38.717480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.274941+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:15:53.274941+0000","last_peered":"2023-10-06T05:15:53.274941+0000","last_clean":"2023-10-06T05:15:53.274941+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:15:53.274941+0000","last_undegraded":"2023-10-06T05:15:53.274941+0000","last_fullsized":"2023-10-06T05:15:53.274941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.295183+0000","last_change":"2023-10-06T05:15:53.287043+0000","last_active":"2023-10-06T05:15:53.295183+0000","last_peered":"2023-10-06T05:15:53.295183+0000","last_clean":"2023-10-06T05:15:53.295183+0000","last_became_active":"2023-10-06T05:15:53.286387+0000","last_became_peered":"2023-10-06T05:15:53.286387+0000","last_unstale":"2023-10-06T05:15:53.295183+0000","last_undegraded":"2023-10-06T05:15:53.295183+0000","last_fullsized":"2023-10-06T05:15:53.295183+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:19.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.282756+0000","last_change":"2023-10-06T05:15:52.289887+0000","last_active":"2023-10-06T05:15:53.282756+0000","last_peered":"2023-10-06T05:15:53.282756+0000","last_clean":"2023-10-06T05:15:53.282756+0000","last_became_active":"2023-10-06T05:15:52.289528+0000","last_became_peered":"2023-10-06T05:15:52.289528+0000","last_unstale":"2023-10-06T05:15:53.282756+0000","last_undegraded":"2023-10-06T05:15:53.282756+0000","last_fullsized":"2023-10-06T05:15:53.282756+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:38.717480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.275770+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:15:53.275770+0000","last_peered":"2023-10-06T05:15:53.275770+0000","last_clean":"2023-10-06T05:15:53.275770+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:15:53.275770+0000","last_undegraded":"2023-10-06T05:15:53.275770+0000","last_fullsized":"2023-10-06T05:15:53.275770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.302425+0000","last_change":"2023-10-06T05:15:53.290600+0000","last_active":"2023-10-06T05:15:53.302425+0000","last_peered":"2023-10-06T05:15:53.302425+0000","last_clean":"2023-10-06T05:15:53.302425+0000","last_became_active":"2023-10-06T05:15:53.290336+0000","last_became_peered":"2023-10-06T05:15:53.290336+0000","last_unstale":"2023-10-06T05:15:53.302425+0000","last_undegraded":"2023-10-06T05:15:53.302425+0000","last_fullsized":"2023-10-06T05:15:53.302425+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:43.938648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.277007+0000","last_change":"2023-10-06T05:15:52.295204+0000","last_active":"2023-10-06T05:15:53.277007+0000","last_peered":"2023-10-06T05:15:53.277007+0000","last_clean":"2023-10-06T05:15:53.277007+0000","last_became_active":"2023-10-06T05:15:52.294934+0000","last_became_peered":"2023-10-06T05:15:52.294934+0000","last_unstale":"2023-10-06T05:15:53.277007+0000","last_undegraded":"2023-10-06T05:15:53.277007+0000","last_fullsized":"2023-10-06T05:15:53.277007+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:47.422625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.284915+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:15:53.284915+0000","last_peered":"2023-10-06T05:15:53.284915+0000","last_clean":"2023-10-06T05:15:53.284915+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:15:53.284915+0000","last_undegraded":"2023-10-06T05:15:53.284915+0000","last_fullsized":"2023-10-06T05:15:53.284915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.303314+0000","last_change":"2023-10-06T05:15:53.297469+0000","last_active":"2023-10-06T05:15:53.303314+0000","last_peered":"2023-10-06T05:15:53.303314+0000","last_clean":"2023-10-06T05:15:53.303314+0000","last_became_active":"2023-10-06T05:15:53.296950+0000","last_became_peered":"2023-10-06T05:15:53.296950+0000","last_unstale":"2023-10-06T05:15:53.303314+0000","last_undegraded":"2023-10-06T05:15:53.303314+0000","last_fullsized":"2023-10-06T05:15:53.303314+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:52.261747+0000","last_clean_scrub_stamp":"2023-10-06T05:15:52.261747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:07.469800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.278541+0000","last_change":"2023-10-06T05:15:52.296145+0000","last_active":"2023-10-06T05:15:53.278541+0000","last_peered":"2023-10-06T05:15:53.278541+0000","last_clean":"2023-10-06T05:15:53.278541+0000","last_became_active":"2023-10-06T05:15:52.295655+0000","last_became_peered":"2023-10-06T05:15:52.295655+0000","last_unstale":"2023-10-06T05:15:53.278541+0000","last_undegraded":"2023-10-06T05:15:53.278541+0000","last_fullsized":"2023-10-06T05:15:53.278541+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:51.252385+0000","last_clean_scrub_stamp":"2023-10-06T05:15:51.252385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:46.812083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T05:15:53.283809+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:15:53.283809+0000","last_peered":"2023-10-06T05:15:53.283809+0000","last_clean":"2023-10-06T05:15:53.283809+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:15:53.283809+0000","last_undegraded":"2023-10-06T05:15:53.283809+0000","last_fullsized":"2023-10-06T05:15:53.283809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":832209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.623},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.443}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":242341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80900000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.88900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":241877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.7349999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.63}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.70199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302524,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069316,"statfs":{"total":96636764160,"available":96326979584,"internally_reserved":0,"allocated":1167360,"data_stored":832141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.379},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.876}]},{"osd":6,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297852,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":577536,"data_stored":243253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.378},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.753},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297856,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":581632,"data_stored":243275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.312},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.505},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.84299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297844,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073996,"statfs":{"total":96636764160,"available":96331771904,"internally_reserved":0,"allocated":569344,"data_stored":242271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297840,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074000,"statfs":{"total":96636764160,"available":96331776000,"internally_reserved":0,"allocated":565248,"data_stored":241717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.71299999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.625}]},{"osd":2,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.505},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.76800000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.751}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:58.136 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:15:58.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:15:58.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.363+0000 7f2c91f9d700 1 Processor -- start 2023-10-06T05:15:58.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.363+0000 7f2c91f9d700 1 -- start start 2023-10-06T05:15:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c91f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2c8c14e580 0x7f2c8c0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c91f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c0c8ff0 0x7f2c8c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c91f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8c0b0870 con 0x7f2c8c0cc590 2023-10-06T05:15:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c91f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8c0c9470 con 0x7f2c8c14e580 2023-10-06T05:15:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c91f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0b0010 con 0x7f2c8c0c8ff0 2023-10-06T05:15:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2c8c14e580 0x7f2c8c0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2c8c14e580 0x7f2c8c0cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47576/0 (socket says 172.21.15.33:47576) 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 -- 172.21.15.33:0/3442915567 learned_addr learned my addr 172.21.15.33:0/3442915567 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c9179c700 1 --2- 172.21.15.33:0/3442915567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c0c8ff0 0x7f2c8c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 -- 172.21.15.33:0/3442915567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c0c8ff0 msgr2=0x7f2c8c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 --2- 172.21.15.33:0/3442915567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c0c8ff0 0x7f2c8c0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 -- 172.21.15.33:0/3442915567 >> v1:172.21.15.167:6789/0 conn(0x7f2c8c0cc590 legacy=0x7f2c8c0c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.365+0000 7f2c90f9b700 1 -- 172.21.15.33:0/3442915567 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0cc9e0 con 0x7f2c8c14e580 2023-10-06T05:15:58.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.366+0000 7f2c90f9b700 1 --2- 172.21.15.33:0/3442915567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c14e580 0x7f2c8c0cc050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c800099b0 tx=0x7f2c80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49c8f09d2763d58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.366+0000 7f2c8b7fe700 1 -- 172.21.15.33:0/3442915567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000b860 con 0x7f2c8c14e580 2023-10-06T05:15:58.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.366+0000 7f2c8b7fe700 1 -- 172.21.15.33:0/3442915567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8000b9c0 con 0x7f2c8c14e580 2023-10-06T05:15:58.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.366+0000 7f2c8b7fe700 1 -- 172.21.15.33:0/3442915567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000bd60 con 0x7f2c8c14e580 2023-10-06T05:15:58.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.366+0000 7f2c91f9d700 1 -- 172.21.15.33:0/3442915567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c14e580 msgr2=0x7f2c8c0cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.366+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/3442915567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c14e580 0x7f2c8c0cc050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c800099b0 tx=0x7f2c80009250 comp rx=0 tx=0).stop 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.367+0000 7f2c91f9d700 1 -- 172.21.15.33:0/3442915567 shutdown_connections 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.367+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/3442915567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c14e580 0x7f2c8c0cc050 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.367+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/3442915567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c0c8ff0 0x7f2c8c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.367+0000 7f2c91f9d700 1 -- 172.21.15.33:0/3442915567 >> 172.21.15.33:0/3442915567 conn(0x7f2c8c06c560 msgr2=0x7f2c8c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.367+0000 7f2c91f9d700 1 -- 172.21.15.33:0/3442915567 shutdown_connections 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.367+0000 7f2c91f9d700 1 -- 172.21.15.33:0/3442915567 wait complete. 2023-10-06T05:15:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.368+0000 7f2c91f9d700 1 Processor -- start 2023-10-06T05:15:58.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.368+0000 7f2c91f9d700 1 -- start start 2023-10-06T05:15:58.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c91f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c8c0c8ff0 0x7f2c8c148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c91f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 0x7f2c8c14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c91f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c14e580 0x7f2c8c0ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c91f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c14fcb0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c91f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c167630 con 0x7f2c8c0c8ff0 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c91f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c14f410 con 0x7f2c8c14e580 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 0x7f2c8c14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 0x7f2c8c14e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47584/0 (socket says 172.21.15.33:47584) 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.369+0000 7f2c8bfff700 1 -- 172.21.15.33:0/4207458789 learned_addr learned my addr 172.21.15.33:0/4207458789 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c90f9b700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c8c0c8ff0 0x7f2c8c148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c8bfff700 1 -- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c14e580 msgr2=0x7f2c8c0ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c9179c700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c14e580 0x7f2c8c0ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c8bfff700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c14e580 0x7f2c8c0ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c8bfff700 1 -- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c8c0c8ff0 msgr2=0x7f2c8c148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c8bfff700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c8c0c8ff0 0x7f2c8c148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c8bfff700 1 -- 172.21.15.33:0/4207458789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800094f0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c90f9b700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c8c0c8ff0 0x7f2c8c148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c9179c700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c14e580 0x7f2c8c0ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c8bfff700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 0x7f2c8c14e950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c5c0 tx=0x7f2c7800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78011590 con 0x7f2c8c0cc590 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78011710 con 0x7f2c8c0cc590 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78011ad0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.370+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c14f6f0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.371+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c14d2c0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.371+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c1677c0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.372+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c78012420 con 0x7f2c8c0cc590 2023-10-06T05:15:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.373+0000 7f2c88ff9700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c70066f30 0x7f2c700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.373+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c78083ae0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.374+0000 7f2c90f9b700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c70066f30 0x7f2c700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.374+0000 7f2c90f9b700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c70066f30 0x7f2c700693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c800099b0 tx=0x7f2c80017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.377+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c7804b5a0 con 0x7f2c8c0cc590 2023-10-06T05:15:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.664+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2c8c04c380 con 0x7f2c8c0cc590 2023-10-06T05:15:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.667+0000 7f2c88ff9700 1 -- 172.21.15.33:0/4207458789 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2c78017020 con 0x7f2c8c0cc590 2023-10-06T05:15:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c70066f30 msgr2=0x7f2c700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c70066f30 0x7f2c700693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c800099b0 tx=0x7f2c80017000 comp rx=0 tx=0).stop 2023-10-06T05:15:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 msgr2=0x7f2c8c14e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 0x7f2c8c14e950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c5c0 tx=0x7f2c7800c980 comp rx=0 tx=0).stop 2023-10-06T05:15:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 shutdown_connections 2023-10-06T05:15:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c70066f30 0x7f2c700693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c8c14e580 0x7f2c8c0ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c8c0c8ff0 0x7f2c8c148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 --2- 172.21.15.33:0/4207458789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0cc590 0x7f2c8c14e950 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 >> 172.21.15.33:0/4207458789 conn(0x7f2c8c06c560 msgr2=0x7f2c8c149d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 shutdown_connections 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.670+0000 7f2c91f9d700 1 -- 172.21.15.33:0/4207458789 wait complete. 2023-10-06T05:15:58.673 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:15:58.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:58.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.875+0000 7fa673fff700 1 Processor -- start 2023-10-06T05:15:58.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.875+0000 7fa673fff700 1 -- start start 2023-10-06T05:15:58.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa673fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa674155140 0x7fa674158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa673fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6740699c0 0x7fa674069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa673fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740c8e40 con 0x7fa6740adf50 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa673fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67406a380 con 0x7fa6740699c0 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa673fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740c99e0 con 0x7fa674155140 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa6737fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6740699c0 0x7fa674069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa6737fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6740699c0 0x7fa674069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47598/0 (socket says 172.21.15.33:47598) 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa6737fe700 1 -- 172.21.15.33:0/500997362 learned_addr learned my addr 172.21.15.33:0/500997362 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa6737fe700 1 -- 172.21.15.33:0/500997362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa674155140 msgr2=0x7fa674158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa6737fe700 1 --2- 172.21.15.33:0/500997362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa674155140 0x7fa674158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.877+0000 7fa6737fe700 1 -- 172.21.15.33:0/500997362 >> v1:172.21.15.167:6789/0 conn(0x7fa6740adf50 legacy=0x7fa6740c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:58.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.878+0000 7fa6737fe700 1 -- 172.21.15.33:0/500997362 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740ae6c0 con 0x7fa6740699c0 2023-10-06T05:15:58.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.878+0000 7fa6737fe700 1 --2- 172.21.15.33:0/500997362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa674069e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009f20 tx=0x7fa66c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03be0b7555041a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.878+0000 7fa671ffb700 1 -- 172.21.15.33:0/500997362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00e070 con 0x7fa6740699c0 2023-10-06T05:15:58.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.878+0000 7fa671ffb700 1 -- 172.21.15.33:0/500997362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c00ba40 con 0x7fa6740699c0 2023-10-06T05:15:58.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.878+0000 7fa671ffb700 1 -- 172.21.15.33:0/500997362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0092e0 con 0x7fa6740699c0 2023-10-06T05:15:58.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 -- 172.21.15.33:0/500997362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 msgr2=0x7fa674069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 --2- 172.21.15.33:0/500997362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa674069e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009f20 tx=0x7fa66c009840 comp rx=0 tx=0).stop 2023-10-06T05:15:58.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 -- 172.21.15.33:0/500997362 shutdown_connections 2023-10-06T05:15:58.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 --2- 172.21.15.33:0/500997362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa674069e40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 --2- 172.21.15.33:0/500997362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa674155140 0x7fa674158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 -- 172.21.15.33:0/500997362 >> 172.21.15.33:0/500997362 conn(0x7fa67406d030 msgr2=0x7fa67406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:58.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 -- 172.21.15.33:0/500997362 shutdown_connections 2023-10-06T05:15:58.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.879+0000 7fa673fff700 1 -- 172.21.15.33:0/500997362 wait complete. 2023-10-06T05:15:58.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.880+0000 7fa673fff700 1 Processor -- start 2023-10-06T05:15:58.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.880+0000 7fa673fff700 1 -- start start 2023-10-06T05:15:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa673fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa6741531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa673fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6740adf50 0x7fa6740c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa673fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa674155140 0x7fa674157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa673fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6741593d0 con 0x7fa6740699c0 2023-10-06T05:15:58.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa673fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740c65f0 con 0x7fa674155140 2023-10-06T05:15:58.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa673fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740c68d0 con 0x7fa6740adf50 2023-10-06T05:15:58.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.881+0000 7fa672ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa6741531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa6741531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47604/0 (socket says 172.21.15.33:47604) 2023-10-06T05:15:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 -- 172.21.15.33:0/810173379 learned_addr learned my addr 172.21.15.33:0/810173379 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa6737fe700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa674155140 0x7fa674157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa6727fc700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6740adf50 0x7fa6740c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 -- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6740adf50 msgr2=0x7fa6740c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6740adf50 0x7fa6740c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 -- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa674155140 msgr2=0x7fa674157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa674155140 0x7fa674157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 -- 172.21.15.33:0/810173379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c009ae0 con 0x7fa6740699c0 2023-10-06T05:15:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa672ffd700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa6741531b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa66400b810 tx=0x7fa664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.882+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66400f510 con 0x7fa6740699c0 2023-10-06T05:15:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.883+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66400f690 con 0x7fa6740699c0 2023-10-06T05:15:58.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.883+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740c6bb0 con 0x7fa6740699c0 2023-10-06T05:15:58.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.883+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66400fa50 con 0x7fa6740699c0 2023-10-06T05:15:58.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.883+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6741536f0 con 0x7fa6740699c0 2023-10-06T05:15:58.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.884+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa674159230 con 0x7fa6740699c0 2023-10-06T05:15:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.884+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa66401a400 con 0x7fa6740699c0 2023-10-06T05:15:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.886+0000 7fa65affd700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa660066f30 0x7fa6600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.886+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa664084200 con 0x7fa6740699c0 2023-10-06T05:15:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.886+0000 7fa6727fc700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa660066f30 0x7fa6600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.887+0000 7fa6727fc700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa660066f30 0x7fa6600693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa67415b540 tx=0x7fa668007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:58.890+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa674159230 con 0x7fa6740699c0 2023-10-06T05:15:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.136+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa674159230 con 0x7fa6740699c0 2023-10-06T05:15:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.137+0000 7fa65affd700 1 -- 172.21.15.33:0/810173379 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fa674159230 con 0x7fa6740699c0 2023-10-06T05:15:59.138 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:15:59.138 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa660066f30 msgr2=0x7fa6600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa660066f30 0x7fa6600693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa67415b540 tx=0x7fa668007ea0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 msgr2=0x7fa6741531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa6741531b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa66400b810 tx=0x7fa664009250 comp rx=0 tx=0).stop 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 shutdown_connections 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa660066f30 0x7fa6600693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa674155140 0x7fa674157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6740699c0 0x7fa6741531b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 --2- 172.21.15.33:0/810173379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6740adf50 0x7fa6740c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 >> 172.21.15.33:0/810173379 conn(0x7fa67406d030 msgr2=0x7fa674153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 shutdown_connections 2023-10-06T05:15:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.139+0000 7fa673fff700 1 -- 172.21.15.33:0/810173379 wait complete. 2023-10-06T05:15:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:15:59.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:15:52.766700+0000', 'modified': '2023-10-06T05:15:56.296376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5388': {'gid': 5388, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/1299702326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 1299702326}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 1299702326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:15:59.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:15:59.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:15:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.749+0000 7feaf73ff700 1 Processor -- start 2023-10-06T05:15:59.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.750+0000 7feaf73ff700 1 -- start start 2023-10-06T05:15:59.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.751+0000 7feaf73ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 0x7feaf015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.751+0000 7feaf73ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feaf00cc350 0x7feaf0142c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.751+0000 7feaf73ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf00cd460 con 0x7feaf006afd0 2023-10-06T05:15:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.751+0000 7feaf73ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf006f220 con 0x7feaf00cc350 2023-10-06T05:15:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.751+0000 7feaf73ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf015e270 con 0x7feaf006ac00 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf499a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feaf00cc350 0x7feaf0142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 0x7feaf015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf599c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7feaf006afd0 0x7feaf00c59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:54340/0 (socket says 172.21.15.33:54340) 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf599c700 1 -- 172.21.15.33:0/1982153964 learned_addr learned my addr 172.21.15.33:0/1982153964 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 0x7feaf015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:39058/0 (socket says 172.21.15.33:39058) 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feae7fff700 1 -- 172.21.15.33:0/1982153964 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838037647 0 0) 0x7feaf00cd460 con 0x7feaf006afd0 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 -- 172.21.15.33:0/1982153964 >> v2:172.21.15.167:3300/0 conn(0x7feaf00cc350 msgr2=0x7feaf0142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 --2- 172.21.15.33:0/1982153964 >> v2:172.21.15.167:3300/0 conn(0x7feaf00cc350 0x7feaf0142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 -- 172.21.15.33:0/1982153964 >> v1:172.21.15.167:6789/0 conn(0x7feaf006afd0 legacy=0x7feaf00c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 -- 172.21.15.33:0/1982153964 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00cc850 con 0x7feaf006ac00 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.752+0000 7feaf519b700 1 --2- 172.21.15.33:0/1982153964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 0x7feaf015dd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaec0098e0 tx=0x7feaec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809815227ec912ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feae7fff700 1 -- 172.21.15.33:0/1982153964 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00b640 con 0x7feaf006ac00 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feae7fff700 1 -- 172.21.15.33:0/1982153964 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec00b7a0 con 0x7feaf006ac00 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feae7fff700 1 -- 172.21.15.33:0/1982153964 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00bb40 con 0x7feaf006ac00 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 -- 172.21.15.33:0/1982153964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 msgr2=0x7feaf015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 --2- 172.21.15.33:0/1982153964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 0x7feaf015dd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaec0098e0 tx=0x7feaec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 -- 172.21.15.33:0/1982153964 shutdown_connections 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 --2- 172.21.15.33:0/1982153964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006ac00 0x7feaf015dd30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 --2- 172.21.15.33:0/1982153964 >> v2:172.21.15.167:3300/0 conn(0x7feaf00cc350 0x7feaf0142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 -- 172.21.15.33:0/1982153964 >> 172.21.15.33:0/1982153964 conn(0x7feaf006e3e0 msgr2=0x7feaf006e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 -- 172.21.15.33:0/1982153964 shutdown_connections 2023-10-06T05:15:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.753+0000 7feaf73ff700 1 -- 172.21.15.33:0/1982153964 wait complete. 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.755+0000 7feaf73ff700 1 Processor -- start 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.755+0000 7feaf73ff700 1 -- start start 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf73ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf006ac00 0x7feaf015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf73ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006afd0 0x7feaf00ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf73ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf00cc350 0x7feaf0142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf73ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00acb80 con 0x7feaf00cc350 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf73ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0169a80 con 0x7feaf006ac00 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf73ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00ad650 con 0x7feaf006afd0 2023-10-06T05:15:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf519b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf006ac00 0x7feaf015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf499a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006afd0 0x7feaf00ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf00cc350 0x7feaf0142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf519b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf006ac00 0x7feaf015b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:34470/0 (socket says 172.21.15.33:34470) 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.756+0000 7feaf519b700 1 -- 172.21.15.33:0/112375824 learned_addr learned my addr 172.21.15.33:0/112375824 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf499a700 1 -- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf006ac00 msgr2=0x7feaf015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf499a700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf006ac00 0x7feaf015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf499a700 1 -- 172.21.15.33:0/112375824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf00cc350 msgr2=0x7feaf0142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf499a700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf00cc350 0x7feaf0142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf499a700 1 -- 172.21.15.33:0/112375824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec009420 con 0x7feaf006afd0 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf599c700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf00cc350 0x7feaf0142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf499a700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006afd0 0x7feaf00ac640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feae000e8d0 tx=0x7feae00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feae57fa700 1 -- 172.21.15.33:0/112375824 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0013680 con 0x7feaf006afd0 2023-10-06T05:15:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feae57fa700 1 -- 172.21.15.33:0/112375824 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae00137e0 con 0x7feaf006afd0 2023-10-06T05:15:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf00acdb0 con 0x7feaf006afd0 2023-10-06T05:15:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feae57fa700 1 -- 172.21.15.33:0/112375824 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0013b80 con 0x7feaf006afd0 2023-10-06T05:15:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.757+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf00ad370 con 0x7feaf006afd0 2023-10-06T05:15:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.760+0000 7feae57fa700 1 -- 172.21.15.33:0/112375824 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feae00123d0 con 0x7feaf006afd0 2023-10-06T05:15:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.760+0000 7feae57fa700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fead4066f30 0x7fead40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.761+0000 7feae57fa700 1 -- 172.21.15.33:0/112375824 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feae0084760 con 0x7feaf006afd0 2023-10-06T05:15:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.761+0000 7feaf519b700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fead4066f30 0x7fead40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.762+0000 7feaf519b700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fead4066f30 0x7fead40693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feaec003080 tx=0x7feaec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.762+0000 7feb05ba0b80 1 Processor -- start 2023-10-06T05:15:59.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.762+0000 7feb05ba0b80 1 -- start start 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a7ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a35580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a35ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2199b5c80 con 0x55a219a389a0 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21998ca70 con 0x55a219a7ca90 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a21998a270 con 0x55a219a36bd0 2023-10-06T05:15:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.763+0000 7feaf499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a35580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a35580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47624/0 (socket says 172.21.15.33:47624) 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 -- 172.21.15.33:0/1587672289 learned_addr learned my addr 172.21.15.33:0/1587672289 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf519b700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a7ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf599c700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a35ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 -- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 msgr2=0x55a219a35ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a35ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 -- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 msgr2=0x55a219a7ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a7ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 -- 172.21.15.33:0/1587672289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a219820270 con 0x55a219a389a0 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf599c700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a35ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf519b700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a7ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:59.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feaf499a700 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a35580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feae0082a50 tx=0x7feae00829e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a87dd9ed1aeda3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feada7fc700 1 -- 172.21.15.33:0/1587672289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0058740 con 0x55a219a389a0 2023-10-06T05:15:59.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feada7fc700 1 -- 172.21.15.33:0/1587672289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae005c430 con 0x55a219a389a0 2023-10-06T05:15:59.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.764+0000 7feada7fc700 1 -- 172.21.15.33:0/1587672289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0053830 con 0x55a219a389a0 2023-10-06T05:15:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1587672289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 msgr2=0x55a219a35580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a35580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feae0082a50 tx=0x7feae00829e0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1587672289 shutdown_connections 2023-10-06T05:15:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a35ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a7ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1587672289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a35580 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1587672289 >> 172.21.15.33:0/1587672289 conn(0x55a219a37070 msgr2=0x55a219a37440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1587672289 shutdown_connections 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.765+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1587672289 wait complete. 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.766+0000 7feb05ba0b80 1 -- start start 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.766+0000 7feb05ba0b80 1 Processor -- start 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a77da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a219986060 con 0x55a219a7ca90 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a219931890 con 0x55a219a389a0 2023-10-06T05:15:59.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a21994cea0 con 0x55a219a36bd0 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.767+0000 7feb05ba0b80 10 client.0 _ll_register_callbacks cb 0x55a219822f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a7ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47628/0 (socket says 172.21.15.33:47628) 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf519b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 -- 172.21.15.33:0/4010734568 learned_addr learned my addr 172.21.15.33:0/4010734568 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf499a700 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a77da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 -- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 msgr2=0x55a219a61b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a61b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 -- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 msgr2=0x55a219a77da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a77da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 -- 172.21.15.33:0/4010734568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf00ad370 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feaf599c700 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a7ef40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feae8007980 tx=0x7feae8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6175bbad6b137338 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feacf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feacf7fe700 1 -- 172.21.15.33:0/4010734568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8019030 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.768+0000 7feacf7fe700 1 -- 172.21.15.33:0/4010734568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feae801a070 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a219a2a690 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feacf7fe700 1 -- 172.21.15.33:0/4010734568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feae8017b00 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 10 client.5520 fetch_fsmap learned FSMap version 24 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a2197c2fe0 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feacf7fe700 1 -- 172.21.15.33:0/4010734568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8024020 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feacf7fe700 1 -- 172.21.15.33:0/4010734568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feae8017c60 con 0x55a219a7ca90 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 10 client.5520 fetch_fsmap finished waiting for FSMap version 24 2023-10-06T05:15:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 10 client.5520 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:15:59.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 20 client.5520 populate_metadata read hostname 'smithi033' 2023-10-06T05:15:59.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a5dcb0 0x55a219a5e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 4 client.5520 mds_command: new command op to 5388 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:15:59.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.769+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 --> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a219a075c0 con 0x55a219a5dcb0 2023-10-06T05:15:59.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.770+0000 7feaf519b700 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a5dcb0 0x55a219a5e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.770+0000 7feaf519b700 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a5dcb0 0x55a219a5e080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a51af68e861d930 server_cookie=f6539968fbaee228 in_seq=0 out_seq=0 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.770+0000 7feacf7fe700 10 client.5520 ms_handle_connect on v2:172.21.15.167:6835/1299702326 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.771+0000 7feacf7fe700 1 -- 172.21.15.33:0/4010734568 <== mds.0 v2:172.21.15.167:6835/1299702326 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a219a075c0 con 0x55a219a5dcb0 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.771+0000 7feacf7fe700 10 client.5520 handle_command_reply: tid=0 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.771+0000 7feb05ba0b80 1 client.5520 shutdown 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.771+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 msgr2=0x55a219a7ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.771+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a7ef40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feae8007980 tx=0x7feae8009040 comp rx=0 tx=0).stop 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 shutdown_connections 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a5dcb0 0x55a219a5e080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a7ef40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a61b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/4010734568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a77da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feacf7fe700 0 client.5520 ms_handle_reset on v2:172.21.15.167:6835/1299702326 2023-10-06T05:15:59.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 >> 172.21.15.33:0/4010734568 conn(0x55a219a37070 msgr2=0x55a219a30740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 shutdown_connections 2023-10-06T05:15:59.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 1 -- 172.21.15.33:0/4010734568 wait complete. 2023-10-06T05:15:59.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.772+0000 7feb05ba0b80 20 client.5520 trim_cache size 0 max 16384 2023-10-06T05:15:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.789+0000 7feb05ba0b80 1 Processor -- start 2023-10-06T05:15:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.789+0000 7feb05ba0b80 1 -- start start 2023-10-06T05:15:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a7a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a7a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a5f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a219808e30 con 0x55a219a7ca90 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a219a30350 con 0x55a219a389a0 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a219a5e080 con 0x55a219a36bd0 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feaf599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a5f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feaf599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a5f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47636/0 (socket says 172.21.15.33:47636) 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feaf599c700 1 -- 172.21.15.33:0/3274294213 learned_addr learned my addr 172.21.15.33:0/3274294213 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feaf599c700 1 -- 172.21.15.33:0/3274294213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 msgr2=0x55a219a7a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.790+0000 7feaf599c700 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a7a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feaf599c700 1 -- 172.21.15.33:0/3274294213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 msgr2=0x55a219a7a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feaf599c700 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a7a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feaf599c700 1 -- 172.21.15.33:0/3274294213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a219954780 con 0x55a219a7ca90 2023-10-06T05:15:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feaf599c700 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a5f670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feae80020d0 tx=0x7feae8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93420d3f5de1b573 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feadaffd700 1 -- 172.21.15.33:0/3274294213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8019040 con 0x55a219a7ca90 2023-10-06T05:15:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feadaffd700 1 -- 172.21.15.33:0/3274294213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8024070 con 0x55a219a7ca90 2023-10-06T05:15:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.791+0000 7feadaffd700 1 -- 172.21.15.33:0/3274294213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8010950 con 0x55a219a7ca90 2023-10-06T05:15:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 -- 172.21.15.33:0/3274294213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 msgr2=0x55a219a5f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a5f670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feae80020d0 tx=0x7feae8016000 comp rx=0 tx=0).stop 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 -- 172.21.15.33:0/3274294213 shutdown_connections 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a7ca90 0x55a219a5f670 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a7a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/3274294213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a389a0 0x55a219a7a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 -- 172.21.15.33:0/3274294213 >> 172.21.15.33:0/3274294213 conn(0x55a219a37070 msgr2=0x55a219a59300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 -- 172.21.15.33:0/3274294213 shutdown_connections 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 -- 172.21.15.33:0/3274294213 wait complete. 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.792+0000 7feb05ba0b80 1 -- start start 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.793+0000 7feb05ba0b80 1 Processor -- start 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.793+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.793+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a7d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feb05ba0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a64d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a219a7e360 con 0x55a219a389a0 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a219a72860 con 0x55a219a7ca90 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feb05ba0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a219a64880 con 0x55a219a36bd0 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feb05ba0b80 10 client.0 _ll_register_callbacks cb 0x55a219822f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a7d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a7d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47648/0 (socket says 172.21.15.33:47648) 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 -- 172.21.15.33:0/1091741575 learned_addr learned my addr 172.21.15.33:0/1091741575 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 -- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 msgr2=0x55a219a72260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf599c700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a64d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf519b700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a72260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 -- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 msgr2=0x55a219a64d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a64d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 -- 172.21.15.33:0/1091741575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2197c2fe0 con 0x55a219a389a0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf519b700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a72260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf599c700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a64d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feaf499a700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a7d8e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feae000eec0 tx=0x7feae0069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e389dfc1d35e6df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.794+0000 7feacf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feacf7fe700 1 -- 172.21.15.33:0/1091741575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0071070 con 0x55a219a389a0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feacf7fe700 1 -- 172.21.15.33:0/1091741575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feae005bfa0 con 0x55a219a389a0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feae005c430 con 0x55a219a389a0 2023-10-06T05:15:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feacf7fe700 1 -- 172.21.15.33:0/1091741575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feae00536a0 con 0x55a219a389a0 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 10 client.5526 fetch_fsmap learned FSMap version 24 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a21993b470 con 0x55a219a389a0 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feacf7fe700 1 -- 172.21.15.33:0/1091741575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae005c430 con 0x55a219a389a0 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feacf7fe700 1 -- 172.21.15.33:0/1091741575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feae0051280 con 0x55a219a389a0 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 10 client.5526 fetch_fsmap finished waiting for FSMap version 24 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 10 client.5526 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 20 client.5526 populate_metadata read hostname 'smithi033' 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a7a2f0 0x55a219a7a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 4 client.5526 mds_command: new command op to 5388 tid=0[{"prefix": "status"}] 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.795+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 --> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a219928e00 con 0x55a219a7a2f0 2023-10-06T05:15:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feaf519b700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a7a2f0 0x55a219a7a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feaf519b700 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a7a2f0 0x55a219a7a6c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c44cdb521281816 server_cookie=9010f7e153c26b01 in_seq=0 out_seq=0 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feacf7fe700 10 client.5526 ms_handle_connect on v2:172.21.15.167:6835/1299702326 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feacf7fe700 1 -- 172.21.15.33:0/1091741575 <== mds.0 v2:172.21.15.167:6835/1299702326 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a219928e00 con 0x55a219a7a2f0 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feacf7fe700 10 client.5526 handle_command_reply: tid=0 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 client.5526 shutdown 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 msgr2=0x55a219a7d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a7d8e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feae000eec0 tx=0x7feae0069000 comp rx=0 tx=0).stop 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 shutdown_connections 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:6835/1299702326,v1:172.21.15.167:6837/1299702326] conn(0x55a219a7a2f0 0x55a219a7a6c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a219a7ca90 0x55a219a64d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a219a36bd0 0x55a219a72260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feacf7fe700 0 client.5526 ms_handle_reset on v2:172.21.15.167:6835/1299702326 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.796+0000 7feb05ba0b80 1 --2- 172.21.15.33:0/1091741575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a219a389a0 0x55a219a7d8e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 >> 172.21.15.33:0/1091741575 conn(0x55a219a37070 msgr2=0x55a219a7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 shutdown_connections 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feb05ba0b80 1 -- 172.21.15.33:0/1091741575 wait complete. 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feb05ba0b80 20 client.5526 trim_cache size 0 max 16384 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:15:59.804 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5388, 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5253620919999999, 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 48, 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 48, 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 13.251236708 2023-10-06T05:15:59.805 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:15:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fead4066f30 msgr2=0x7fead40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fead4066f30 0x7fead40693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feaec003080 tx=0x7feaec017000 comp rx=0 tx=0).stop 2023-10-06T05:15:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006afd0 msgr2=0x7feaf00ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006afd0 0x7feaf00ac640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feae000e8d0 tx=0x7feae00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 shutdown_connections 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fead4066f30 0x7fead40693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf00cc350 0x7feaf0142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf006ac00 0x7feaf015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 --2- 172.21.15.33:0/112375824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf006afd0 0x7feaf00ac640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.797+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 >> 172.21.15.33:0/112375824 conn(0x7feaf006e3e0 msgr2=0x7feaf00af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.798+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 shutdown_connections 2023-10-06T05:15:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:15:59.798+0000 7feaf73ff700 1 -- 172.21.15.33:0/112375824 wait complete. 2023-10-06T05:15:59.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:00.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.003+0000 7f4e2359e700 1 Processor -- start 2023-10-06T05:16:00.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.003+0000 7f4e2359e700 1 -- start start 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.004+0000 7f4e2359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e24155af0 0x7f4e2406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.004+0000 7f4e2359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e240c7920 0x7f4e240c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.004+0000 7f4e2359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2406ab60 con 0x7f4e2406a6a0 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.004+0000 7f4e2359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2406b7e0 con 0x7f4e24155af0 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.004+0000 7f4e2359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2406ae40 con 0x7f4e240c7920 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e24155af0 0x7f4e2406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e24155af0 0x7f4e2406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34868/0 (socket says 172.21.15.33:34868) 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 -- 172.21.15.33:0/728958074 learned_addr learned my addr 172.21.15.33:0/728958074 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e22d9d700 1 --2- 172.21.15.33:0/728958074 >> v2:172.21.15.167:3300/0 conn(0x7f4e240c7920 0x7f4e240c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 -- 172.21.15.33:0/728958074 >> v2:172.21.15.167:3300/0 conn(0x7f4e240c7920 msgr2=0x7f4e240c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:00.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 --2- 172.21.15.33:0/728958074 >> v2:172.21.15.167:3300/0 conn(0x7f4e240c7920 0x7f4e240c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 -- 172.21.15.33:0/728958074 >> v1:172.21.15.167:6789/0 conn(0x7f4e2406a6a0 legacy=0x7f4e240c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 -- 172.21.15.33:0/728958074 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2406b120 con 0x7f4e24155af0 2023-10-06T05:16:00.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.005+0000 7f4e2259c700 1 --2- 172.21.15.33:0/728958074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e24155af0 0x7f4e2406a160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4e24070fb0 tx=0x7f4e14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7026bb663d1918c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2159a700 1 -- 172.21.15.33:0/728958074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14009e70 con 0x7f4e24155af0 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2159a700 1 -- 172.21.15.33:0/728958074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1400b860 con 0x7f4e24155af0 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2159a700 1 -- 172.21.15.33:0/728958074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1400bc00 con 0x7f4e24155af0 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 -- 172.21.15.33:0/728958074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e24155af0 msgr2=0x7f4e2406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 --2- 172.21.15.33:0/728958074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e24155af0 0x7f4e2406a160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4e24070fb0 tx=0x7f4e14009250 comp rx=0 tx=0).stop 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 -- 172.21.15.33:0/728958074 shutdown_connections 2023-10-06T05:16:00.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 --2- 172.21.15.33:0/728958074 >> v2:172.21.15.167:3300/0 conn(0x7f4e240c7920 0x7f4e240c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 --2- 172.21.15.33:0/728958074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e24155af0 0x7f4e2406a160 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 -- 172.21.15.33:0/728958074 >> 172.21.15.33:0/728958074 conn(0x7f4e2406d030 msgr2=0x7f4e2406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 -- 172.21.15.33:0/728958074 shutdown_connections 2023-10-06T05:16:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.006+0000 7f4e2359e700 1 -- 172.21.15.33:0/728958074 wait complete. 2023-10-06T05:16:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.008+0000 7f4e2359e700 1 Processor -- start 2023-10-06T05:16:00.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.008+0000 7f4e2359e700 1 -- start start 2023-10-06T05:16:00.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 0x7f4e241567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e240c7920 0x7f4e240adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24155af0 0x7f4e241435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240ad830 con 0x7f4e2406a6a0 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240acc60 con 0x7f4e24155af0 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240acf00 con 0x7f4e240c7920 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 0x7f4e241567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 0x7f4e241567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34878/0 (socket says 172.21.15.33:34878) 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 -- 172.21.15.33:0/4180298647 learned_addr learned my addr 172.21.15.33:0/4180298647 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e22d9d700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24155af0 0x7f4e241435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e21d9b700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e240c7920 0x7f4e240adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 -- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e240c7920 msgr2=0x7f4e240adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e240c7920 0x7f4e240adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 -- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24155af0 msgr2=0x7f4e241435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.009+0000 7f4e2259c700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24155af0 0x7f4e241435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e2259c700 1 -- 172.21.15.33:0/4180298647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140094f0 con 0x7f4e2406a6a0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e2259c700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 0x7f4e241567e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4e240635d0 tx=0x7f4e14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14011420 con 0x7f4e2406a6a0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e14011580 con 0x7f4e2406a6a0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e240ad1e0 con 0x7f4e2406a6a0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14011940 con 0x7f4e2406a6a0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.010+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e241695a0 con 0x7f4e2406a6a0 2023-10-06T05:16:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.011+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2404c380 con 0x7f4e2406a6a0 2023-10-06T05:16:00.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.012+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e1402c3b0 con 0x7f4e2406a6a0 2023-10-06T05:16:00.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.013+0000 7f4e127fc700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e04066f30 0x7f4e040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.013+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4e14085cc0 con 0x7f4e2406a6a0 2023-10-06T05:16:00.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.014+0000 7f4e21d9b700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e04066f30 0x7f4e040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.014+0000 7f4e21d9b700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e04066f30 0x7f4e040693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e241597f0 tx=0x7f4e18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.017+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e1408c080 con 0x7f4e2406a6a0 2023-10-06T05:16:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.268+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e2406b400 con 0x7f4e2406a6a0 2023-10-06T05:16:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.269+0000 7f4e127fc700 1 -- 172.21.15.33:0/4180298647 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4e14052d30 con 0x7f4e2406a6a0 2023-10-06T05:16:00.271 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:00.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e04066f30 msgr2=0x7f4e040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e04066f30 0x7f4e040693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e241597f0 tx=0x7f4e18009040 comp rx=0 tx=0).stop 2023-10-06T05:16:00.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 msgr2=0x7f4e241567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 0x7f4e241567e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4e240635d0 tx=0x7f4e14017000 comp rx=0 tx=0).stop 2023-10-06T05:16:00.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 shutdown_connections 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e04066f30 0x7f4e040693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24155af0 0x7f4e241435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e2406a6a0 0x7f4e241567e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 --2- 172.21.15.33:0/4180298647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e240c7920 0x7f4e240adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.271+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 >> 172.21.15.33:0/4180298647 conn(0x7f4e2406d030 msgr2=0x7f4e240b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.272+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 shutdown_connections 2023-10-06T05:16:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:00.272+0000 7f4e2359e700 1 -- 172.21.15.33:0/4180298647 wait complete. 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:16:00.291 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:16:00.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:00.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:00.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:00.292 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:16:00.330 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:00.331 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:00.331 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:00.331 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:00.333 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:00.333 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:16:00.333 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:00.333 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:00.333 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:16:00.333 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42197sec preferred_lft 42197sec 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:16:00.334 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:16:00.336 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:16:00.336 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.336 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:00.336 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:16:00.336 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:16:00.336 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:00.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:00.337 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:00.432 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:16:00.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:00.435 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:16:00.474 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:16:00.474 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:00.475 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:16:00.475 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:16:00.475 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:00.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:00.476 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:16:00.476 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:16:00.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:16:00.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:16:00.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:16:00.476 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:16:00.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:00.676 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:16:00.713 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:16:00.713 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:16:00.713 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:00.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:00.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:16:00.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:00.816 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:00.853 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:00.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:00.853 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:00.929 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:16:00.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:00.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:01.077 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:01.078 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:01.078 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:01.078 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:01.078 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:01.099 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:01.099 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:01.099 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:01.099 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:01.099 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:01.100 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:16:01.100 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:16:01.100 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:16:01.104 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:01.104 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:01.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.354+0000 7f48d7fff700 1 Processor -- start 2023-10-06T05:16:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.355+0000 7f48d7fff700 1 -- start start 2023-10-06T05:16:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 0x7f48d815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d80afdd0 0x7f48d80b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d806f220 con 0x7f48d8159530 2023-10-06T05:16:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d80b1e00 con 0x7f48d806e140 2023-10-06T05:16:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d806f6b0 con 0x7f48d80afdd0 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d67fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 0x7f48d815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d80afdd0 0x7f48d80b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d67fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 0x7f48d815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:55118/0 (socket says 172.21.15.33:55118) 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.356+0000 7f48d67fc700 1 -- 172.21.15.33:0/1374624521 learned_addr learned my addr 172.21.15.33:0/1374624521 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d5ffb700 1 -- 172.21.15.33:0/1374624521 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744731842 0 0) 0x7f48d806f220 con 0x7f48d8159530 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d5ffb700 1 -- 172.21.15.33:0/1374624521 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c0003570 con 0x7f48d8159530 2023-10-06T05:16:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d77fe700 1 -- 172.21.15.33:0/1374624521 >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 msgr2=0x7f48d815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d77fe700 1 --2- 172.21.15.33:0/1374624521 >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 0x7f48d815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d77fe700 1 -- 172.21.15.33:0/1374624521 >> v1:172.21.15.167:6789/0 conn(0x7f48d8159530 legacy=0x7f48d806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d77fe700 1 -- 172.21.15.33:0/1374624521 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d815ae50 con 0x7f48d80afdd0 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d67fc700 1 --2- 172.21.15.33:0/1374624521 >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 0x7f48d815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.357+0000 7f48d77fe700 1 --2- 172.21.15.33:0/1374624521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d80afdd0 0x7f48d80b01f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009e60 tx=0x7f48d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cadabc7911afc68b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d5ffb700 1 -- 172.21.15.33:0/1374624521 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000e070 con 0x7f48d80afdd0 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d5ffb700 1 -- 172.21.15.33:0/1374624521 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d000bab0 con 0x7f48d80afdd0 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d5ffb700 1 -- 172.21.15.33:0/1374624521 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000be70 con 0x7f48d80afdd0 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 -- 172.21.15.33:0/1374624521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d80afdd0 msgr2=0x7f48d80b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 --2- 172.21.15.33:0/1374624521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d80afdd0 0x7f48d80b01f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009e60 tx=0x7f48d0009780 comp rx=0 tx=0).stop 2023-10-06T05:16:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 -- 172.21.15.33:0/1374624521 shutdown_connections 2023-10-06T05:16:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 --2- 172.21.15.33:0/1374624521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d80afdd0 0x7f48d80b01f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 --2- 172.21.15.33:0/1374624521 >> v2:172.21.15.167:3300/0 conn(0x7f48d806e140 0x7f48d815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 -- 172.21.15.33:0/1374624521 >> 172.21.15.33:0/1374624521 conn(0x7f48d806c560 msgr2=0x7f48d806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 -- 172.21.15.33:0/1374624521 shutdown_connections 2023-10-06T05:16:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.358+0000 7f48d7fff700 1 -- 172.21.15.33:0/1374624521 wait complete. 2023-10-06T05:16:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.360+0000 7f48d7fff700 1 Processor -- start 2023-10-06T05:16:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.360+0000 7f48d7fff700 1 -- start start 2023-10-06T05:16:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 0x7f48d8140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d80afdd0 0x7f48d8157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d8159530 0x7f48d8158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80ced40 con 0x7f48d806e140 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d816de20 con 0x7f48d80afdd0 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d816d990 con 0x7f48d8159530 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 0x7f48d8140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 0x7f48d8140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34880/0 (socket says 172.21.15.33:34880) 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.361+0000 7f48d6ffd700 1 -- 172.21.15.33:0/2305227217 learned_addr learned my addr 172.21.15.33:0/2305227217 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d77fe700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d8159530 0x7f48d8158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d67fc700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d80afdd0 0x7f48d8157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d6ffd700 1 -- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d8159530 msgr2=0x7f48d8158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d6ffd700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d8159530 0x7f48d8158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d6ffd700 1 -- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d80afdd0 msgr2=0x7f48d8157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d6ffd700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d80afdd0 0x7f48d8157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d6ffd700 1 -- 172.21.15.33:0/2305227217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0094f0 con 0x7f48d806e140 2023-10-06T05:16:01.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d6ffd700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 0x7f48d8140ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c7c0 tx=0x7f48c8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8010510 con 0x7f48d806e140 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8010690 con 0x7f48d806e140 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8010a50 con 0x7f48d806e140 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d0009a20 con 0x7f48d806e140 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.362+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d816d340 con 0x7f48d806e140 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.363+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d816dc70 con 0x7f48d806e140 2023-10-06T05:16:01.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.364+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48c801b400 con 0x7f48d806e140 2023-10-06T05:16:01.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.365+0000 7f48beffd700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48c4066f30 0x7f48c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.365+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f48c8083ee0 con 0x7f48d806e140 2023-10-06T05:16:01.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.366+0000 7f48d67fc700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48c4066f30 0x7f48c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.367+0000 7f48d67fc700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48c4066f30 0x7f48c40693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48d815c7d0 tx=0x7f48cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:01.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.369+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48c804b5a0 con 0x7f48d806e140 2023-10-06T05:16:01.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.618+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f48d804c380 con 0x7f48d806e140 2023-10-06T05:16:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.618+0000 7f48beffd700 1 -- 172.21.15.33:0/2305227217 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f48c8016070 con 0x7f48d806e140 2023-10-06T05:16:01.620 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48c4066f30 msgr2=0x7f48c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48c4066f30 0x7f48c40693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48d815c7d0 tx=0x7f48cc008040 comp rx=0 tx=0).stop 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 msgr2=0x7f48d8140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 0x7f48d8140ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c7c0 tx=0x7f48c8007e10 comp rx=0 tx=0).stop 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 shutdown_connections 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48c4066f30 0x7f48c40693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48d8159530 0x7f48d8158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d806e140 0x7f48d8140ff0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 --2- 172.21.15.33:0/2305227217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48d80afdd0 0x7f48d8157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 >> 172.21.15.33:0/2305227217 conn(0x7f48d806c560 msgr2=0x7f48d815b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 shutdown_connections 2023-10-06T05:16:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:01.621+0000 7f48d7fff700 1 -- 172.21.15.33:0/2305227217 wait complete. 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:01.640 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:16:01.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:01.641 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:16:01.682 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:16:01.683 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42199sec preferred_lft 42199sec 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:01.684 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:16:01.685 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:01.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:01.686 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:01.785 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:16:01.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:01.788 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:16:01.827 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:01.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:01.829 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:16:01.829 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:16:01.829 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:16:01.829 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:16:01.829 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:16:01.829 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:16:02.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:02.024 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:16:02.061 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:16:02.061 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:16:02.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:16:02.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:02.061 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:16:02.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:02.165 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:02.201 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:16:02.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:02.201 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:02.277 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:16:02.277 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:02.277 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:02.407 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:02.407 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:16:02.407 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:16:02.423 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:02.424 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:16:02.424 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:16:02.424 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:02.424 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:16:02.445 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:16:02.450 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:16:02.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:02.451 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:02.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:16:02.729 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:16:02.729 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:16:02.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:16:02.935 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:16:02.935 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:16:02.935 INFO:tasks.cephfs.mount:Creating file a 2023-10-06T05:16:02.935 DEBUG:teuthology.orchestra.run.smithi033:> touch /home/ubuntu/cephtest/mnt.0/a 2023-10-06T05:16:02.976 INFO:tasks.cephfs.mount:Creating file b 2023-10-06T05:16:02.977 DEBUG:teuthology.orchestra.run.smithi033:> touch /home/ubuntu/cephtest/mnt.0/b 2023-10-06T05:16:03.054 INFO:tasks.cephfs.mount:Creating file c 2023-10-06T05:16:03.054 DEBUG:teuthology.orchestra.run.smithi033:> touch /home/ubuntu/cephtest/mnt.0/c 2023-10-06T05:16:03.134 INFO:tasks.cephfs.mount:Checking file a 2023-10-06T05:16:03.134 DEBUG:teuthology.orchestra.run.smithi033:> ls /home/ubuntu/cephtest/mnt.0/a 2023-10-06T05:16:03.211 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0/a 2023-10-06T05:16:03.211 INFO:tasks.cephfs.mount:Checking file b 2023-10-06T05:16:03.211 DEBUG:teuthology.orchestra.run.smithi033:> ls /home/ubuntu/cephtest/mnt.0/b 2023-10-06T05:16:03.288 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0/b 2023-10-06T05:16:03.288 INFO:tasks.cephfs.mount:Checking file c 2023-10-06T05:16:03.288 DEBUG:teuthology.orchestra.run.smithi033:> ls /home/ubuntu/cephtest/mnt.0/c 2023-10-06T05:16:03.364 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0/c 2023-10-06T05:16:03.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:16:03.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:03.365 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:03.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:16:03.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:03.514 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:16:03.558 INFO:tasks.cephfs.mount:Checking file a 2023-10-06T05:16:03.558 DEBUG:teuthology.orchestra.run.smithi167:> ls /home/ubuntu/cephtest/mnt.1/a 2023-10-06T05:16:03.597 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1/a 2023-10-06T05:16:03.598 INFO:tasks.cephfs.mount:Checking file b 2023-10-06T05:16:03.598 DEBUG:teuthology.orchestra.run.smithi167:> ls /home/ubuntu/cephtest/mnt.1/b 2023-10-06T05:16:03.675 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1/b 2023-10-06T05:16:03.676 INFO:tasks.cephfs.mount:Checking file c 2023-10-06T05:16:03.676 DEBUG:teuthology.orchestra.run.smithi167:> ls /home/ubuntu/cephtest/mnt.1/c 2023-10-06T05:16:03.753 INFO:teuthology.orchestra.run.smithi167.stdout:/home/ubuntu/cephtest/mnt.1/c 2023-10-06T05:16:03.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:03.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.940+0000 7fe95d79d700 1 Processor -- start 2023-10-06T05:16:03.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.940+0000 7fe95d79d700 1 -- start start 2023-10-06T05:16:03.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe95d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe958155af0 0x7fe95806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe95d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe95806a6a0 0x7fe9580c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe95d79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95806ab60 con 0x7fe9580c8b50 2023-10-06T05:16:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe95d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95806b7e0 con 0x7fe958155af0 2023-10-06T05:16:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe95d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806ae40 con 0x7fe95806a6a0 2023-10-06T05:16:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe957fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe958155af0 0x7fe95806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe957fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe958155af0 0x7fe95806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34892/0 (socket says 172.21.15.33:34892) 2023-10-06T05:16:03.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe957fff700 1 -- 172.21.15.33:0/1065428786 learned_addr learned my addr 172.21.15.33:0/1065428786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe9577fe700 1 --2- 172.21.15.33:0/1065428786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe95806a6a0 0x7fe9580c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe957fff700 1 -- 172.21.15.33:0/1065428786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe95806a6a0 msgr2=0x7fe9580c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.942+0000 7fe957fff700 1 --2- 172.21.15.33:0/1065428786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe95806a6a0 0x7fe9580c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.943+0000 7fe957fff700 1 -- 172.21.15.33:0/1065428786 >> v1:172.21.15.167:6789/0 conn(0x7fe9580c8b50 legacy=0x7fe9581598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.943+0000 7fe957fff700 1 -- 172.21.15.33:0/1065428786 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95806b120 con 0x7fe958155af0 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.943+0000 7fe957fff700 1 --2- 172.21.15.33:0/1065428786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe958155af0 0x7fe95806a160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe9480099b0 tx=0x7fe948009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e15fd88d319bb4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.943+0000 7fe956ffd700 1 -- 172.21.15.33:0/1065428786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800b860 con 0x7fe958155af0 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.943+0000 7fe956ffd700 1 -- 172.21.15.33:0/1065428786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94800b9c0 con 0x7fe958155af0 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.943+0000 7fe956ffd700 1 -- 172.21.15.33:0/1065428786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800bd60 con 0x7fe958155af0 2023-10-06T05:16:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 -- 172.21.15.33:0/1065428786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe958155af0 msgr2=0x7fe95806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1065428786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe958155af0 0x7fe95806a160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe9480099b0 tx=0x7fe948009250 comp rx=0 tx=0).stop 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 -- 172.21.15.33:0/1065428786 shutdown_connections 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1065428786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe958155af0 0x7fe95806a160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1065428786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe95806a6a0 0x7fe9580c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 -- 172.21.15.33:0/1065428786 >> 172.21.15.33:0/1065428786 conn(0x7fe95806d030 msgr2=0x7fe95806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 -- 172.21.15.33:0/1065428786 shutdown_connections 2023-10-06T05:16:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.944+0000 7fe95d79d700 1 -- 172.21.15.33:0/1065428786 wait complete. 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.945+0000 7fe95d79d700 1 Processor -- start 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.945+0000 7fe95d79d700 1 -- start start 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.946+0000 7fe95d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe95806a6a0 0x7fe9580ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.946+0000 7fe95d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 0x7fe9580abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.946+0000 7fe95d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe958155af0 0x7fe9580ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.946+0000 7fe95d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580b19a0 con 0x7fe9580c8b50 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.946+0000 7fe95d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95814d580 con 0x7fe95806a6a0 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.946+0000 7fe95d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580b0960 con 0x7fe958155af0 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 0x7fe9580abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 0x7fe9580abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34904/0 (socket says 172.21.15.33:34904) 2023-10-06T05:16:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe957fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe95806a6a0 0x7fe9580ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 -- 172.21.15.33:0/1690742394 learned_addr learned my addr 172.21.15.33:0/1690742394 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe95cf9c700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe958155af0 0x7fe9580ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 -- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe958155af0 msgr2=0x7fe9580ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe958155af0 0x7fe9580ac030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 -- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe95806a6a0 msgr2=0x7fe9580ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe95806a6a0 0x7fe9580ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 -- 172.21.15.33:0/1690742394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480094f0 con 0x7fe9580c8b50 2023-10-06T05:16:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.947+0000 7fe9577fe700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 0x7fe9580abaf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00d890 tx=0x7fe94c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.948+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0125f0 con 0x7fe9580c8b50 2023-10-06T05:16:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.948+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c012770 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.948+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c012b30 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.948+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9580ae840 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.948+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9580b0ec0 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.949+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95804c380 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.950+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe94c01e460 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.951+0000 7fe93ffff700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe944066f30 0x7fe9440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.951+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe94c084cd0 con 0x7fe9580c8b50 2023-10-06T05:16:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.952+0000 7fe957fff700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe944066f30 0x7fe9440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.952+0000 7fe957fff700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe944066f30 0x7fe9440693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9480099b0 tx=0x7fe948017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:03.956+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe94c051d40 con 0x7fe9580c8b50 2023-10-06T05:16:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.203+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe95806fb40 con 0x7fe9580c8b50 2023-10-06T05:16:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.203+0000 7fe93ffff700 1 -- 172.21.15.33:0/1690742394 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe94c055360 con 0x7fe9580c8b50 2023-10-06T05:16:04.205 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe944066f30 msgr2=0x7fe9440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe944066f30 0x7fe9440693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9480099b0 tx=0x7fe948017000 comp rx=0 tx=0).stop 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 msgr2=0x7fe9580abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 0x7fe9580abaf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00d890 tx=0x7fe94c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 shutdown_connections 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe944066f30 0x7fe9440693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe958155af0 0x7fe9580ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe95806a6a0 0x7fe9580ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 --2- 172.21.15.33:0/1690742394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9580c8b50 0x7fe9580abaf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 >> 172.21.15.33:0/1690742394 conn(0x7fe95806d030 msgr2=0x7fe9580cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 shutdown_connections 2023-10-06T05:16:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:04.206+0000 7fe95d79d700 1 -- 172.21.15.33:0/1690742394 wait complete. 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:04.225 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:04.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:04.226 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42193sec preferred_lft 42193sec 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.265 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:04.266 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:16:04.266 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:16:04.266 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:04.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:04.266 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:04.363 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:16:04.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:04.367 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:16:04.406 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:16:04.406 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:04.406 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:16:04.406 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:16:04.406 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:04.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:04.408 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:16:04.408 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:16:04.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:16:04.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:16:04.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:16:04.408 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:16:04.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:04.608 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:16:04.644 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:16:04.644 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:16:04.645 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:04.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:04.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:16:04.747 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:04.747 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:04.784 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:04.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:04.784 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:04.860 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:16:04.861 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:04.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:04.993 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:04.993 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:04.993 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:05.009 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:05.009 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:05.010 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:05.010 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:05.010 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:16:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:16:05.038 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:16:05.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:05.038 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:05.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.286+0000 7f2d6bfff700 1 Processor -- start 2023-10-06T05:16:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.286+0000 7f2d6bfff700 1 -- start start 2023-10-06T05:16:05.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.287+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.287+0000 7f2d6bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2d6c0cc750 0x7f2d6c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.287+0000 7f2d6bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0cbd80 con 0x7f2d6c0cf3b0 2023-10-06T05:16:05.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.287+0000 7f2d6bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0af5f0 con 0x7f2d6c0cc750 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.287+0000 7f2d6bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0af8d0 con 0x7f2d6c0ccfc0 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6affd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6a7fc700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2d6c0cf3b0 0x7f2d6c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:34830/0 (socket says 172.21.15.33:34830) 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6a7fc700 1 -- 172.21.15.33:0/3571657358 learned_addr learned my addr 172.21.15.33:0/3571657358 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/3571657358 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c0cc750 0x7f2d6c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d69ffb700 1 -- 172.21.15.33:0/3571657358 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648795247 0 0) 0x7f2d6c0cbd80 con 0x7f2d6c0cf3b0 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d69ffb700 1 -- 172.21.15.33:0/3571657358 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d54003570 con 0x7f2d6c0cf3b0 2023-10-06T05:16:05.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6affd700 1 -- 172.21.15.33:0/3571657358 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c0cc750 msgr2=0x7f2d6c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6affd700 1 --2- 172.21.15.33:0/3571657358 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c0cc750 0x7f2d6c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6affd700 1 -- 172.21.15.33:0/3571657358 >> v1:172.21.15.167:6789/0 conn(0x7f2d6c0cf3b0 legacy=0x7f2d6c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6affd700 1 -- 172.21.15.33:0/3571657358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0afbb0 con 0x7f2d6c0ccfc0 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.288+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/3571657358 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c0cc750 0x7f2d6c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6affd700 1 --2- 172.21.15.33:0/3571657358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c0cc210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0098e0 tx=0x7f2d5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a595e671949ed0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d69ffb700 1 -- 172.21.15.33:0/3571657358 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c011630 con 0x7f2d6c0ccfc0 2023-10-06T05:16:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d69ffb700 1 -- 172.21.15.33:0/3571657358 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c011790 con 0x7f2d6c0ccfc0 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d69ffb700 1 -- 172.21.15.33:0/3571657358 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c011b30 con 0x7f2d6c0ccfc0 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3571657358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 msgr2=0x7f2d6c0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3571657358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c0cc210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0098e0 tx=0x7f2d5c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3571657358 shutdown_connections 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3571657358 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c0cc750 0x7f2d6c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3571657358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c0cc210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3571657358 >> 172.21.15.33:0/3571657358 conn(0x7f2d6c06cff0 msgr2=0x7f2d6c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.289+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3571657358 shutdown_connections 2023-10-06T05:16:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.290+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3571657358 wait complete. 2023-10-06T05:16:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.291+0000 7f2d6bfff700 1 Processor -- start 2023-10-06T05:16:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.291+0000 7f2d6bfff700 1 -- start start 2023-10-06T05:16:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6c0cc750 0x7f2d6c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c9940 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c8d30 con 0x7f2d6c0cc750 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c9010 con 0x7f2d6c0ccfc0 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34914/0 (socket says 172.21.15.33:34914) 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6b7fe700 1 -- 172.21.15.33:0/3590949108 learned_addr learned my addr 172.21.15.33:0/3590949108 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6affd700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6c0cc750 0x7f2d6c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.292+0000 7f2d6a7fc700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6b7fe700 1 -- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 msgr2=0x7f2d6c144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6b7fe700 1 -- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6c0cc750 msgr2=0x7f2d6c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6c0cc750 0x7f2d6c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6b7fe700 1 -- 172.21.15.33:0/3590949108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009a20 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0ce520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009f90 tx=0x7f2d6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400cbd0 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400f040 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64014480 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c009420 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.293+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c0c95d0 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.294+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0b6340 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.295+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d640147f0 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.296+0000 7f2d52ffd700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d58066f30 0x7f2d580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.296+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d640853f0 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.297+0000 7f2d6affd700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d58066f30 0x7f2d580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.298+0000 7f2d6affd700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d58066f30 0x7f2d580693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0091e0 tx=0x7f2d5c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.300+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d64052460 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.547+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d6c04c380 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.547+0000 7f2d52ffd700 1 -- 172.21.15.33:0/3590949108 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f2d64055a80 con 0x7f2d6c0c7f70 2023-10-06T05:16:05.549 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:05.549 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:16:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d58066f30 msgr2=0x7f2d580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d58066f30 0x7f2d580693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0091e0 tx=0x7f2d5c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 msgr2=0x7f2d6c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0ce520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009f90 tx=0x7f2d6400b040 comp rx=0 tx=0).stop 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 shutdown_connections 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d58066f30 0x7f2d580693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c0c7f70 0x7f2d6c0ce520 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6c0cc750 0x7f2d6c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/3590949108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c0ccfc0 0x7f2d6c144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 >> 172.21.15.33:0/3590949108 conn(0x7f2d6c06cff0 msgr2=0x7f2d6c0c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 shutdown_connections 2023-10-06T05:16:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:05.550+0000 7f2d6bfff700 1 -- 172.21.15.33:0/3590949108 wait complete. 2023-10-06T05:16:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:16:05.569 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T05:16:05.784 INFO:teuthology.orchestra.run.smithi167.stdout:[{"id":5395,"entity":{"name":{"type":"client","num":5395},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":2633773718}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":0.74198725600000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99829403358775781,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5395 v1:172.21.15.33:0/2633773718","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":5541,"entity":{"name":{"type":"client","num":5541},"addr":{"type":"v1","addr":"172.21.15.167:0","nonce":1515079690}},"state":"open","num_leases":0,"num_caps":4,"request_load_avg":3,"uptime":3.3299428099999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":5.9649597588916405,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5541 v1:172.21.15.167:0/1515079690","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:16:05.784 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 2633773718, "type": "v1" }, "name": { "num": 5395, "type": "client" } }, "id": 5395, "inst": "client.5395 v1:172.21.15.33:0/2633773718", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9982940335877578 }, "state": "open", "uptime": 0.741987256 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.167:0", "nonce": 1515079690, "type": "v1" }, "name": { "num": 5541, "type": "client" } }, "id": 5541, "inst": "client.5541 v1:172.21.15.167:0/1515079690", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 4, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 5.9649597588916405 }, "state": "open", "uptime": 3.32994281 } ] 2023-10-06T05:16:05.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:05.785 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:16:05.864 INFO:teuthology.orchestra.run.smithi033.stdout:client5395 2023-10-06T05:16:05.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:05.865 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:16:05.949 INFO:teuthology.orchestra.run.smithi167.stdout:client5541 2023-10-06T05:16:05.949 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:16:05.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:16:06.029 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:16:06.105 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:16:06.105 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:16:06.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:06.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:06.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:16:06.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:06.183 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:16:06.221 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:16:06.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:06.221 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:06.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:16:06.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:06.300 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:16:06.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.521+0000 7fec9559d700 1 Processor -- start 2023-10-06T05:16:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.521+0000 7fec9559d700 1 -- start start 2023-10-06T05:16:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec9559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90155af0 0x7fec9006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec9559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec9006a6a0 0x7fec900c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec9559d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9006ab60 con 0x7fec900c8b50 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec9559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9006b7e0 con 0x7fec9006a6a0 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec9559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9006ae40 con 0x7fec90155af0 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec8f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec9006a6a0 0x7fec900c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec94d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fec900c8b50 0x7fec901598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57318/0 (socket says 172.21.15.33:57318) 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec94d9c700 1 -- 172.21.15.33:0/757759041 learned_addr learned my addr 172.21.15.33:0/757759041 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec8ffff700 1 --2- 172.21.15.33:0/757759041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90155af0 0x7fec9006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec8f7fe700 1 -- 172.21.15.33:0/757759041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90155af0 msgr2=0x7fec9006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.523+0000 7fec8f7fe700 1 --2- 172.21.15.33:0/757759041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90155af0 0x7fec9006a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8f7fe700 1 -- 172.21.15.33:0/757759041 >> v1:172.21.15.33:6789/0 conn(0x7fec900c8b50 legacy=0x7fec901598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8f7fe700 1 -- 172.21.15.33:0/757759041 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9006b120 con 0x7fec9006a6a0 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8effd700 1 -- 172.21.15.33:0/757759041 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978998662 0 0) 0x7fec9006ab60 con 0x7fec900c8b50 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8f7fe700 1 --2- 172.21.15.33:0/757759041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec9006a6a0 0x7fec900c7210 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fec840098e0 tx=0x7fec840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7825c4688e6e0331 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8effd700 1 -- 172.21.15.33:0/757759041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400b640 con 0x7fec9006a6a0 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8effd700 1 -- 172.21.15.33:0/757759041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8400b7a0 con 0x7fec9006a6a0 2023-10-06T05:16:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.524+0000 7fec8effd700 1 -- 172.21.15.33:0/757759041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400bb40 con 0x7fec9006a6a0 2023-10-06T05:16:06.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 -- 172.21.15.33:0/757759041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec9006a6a0 msgr2=0x7fec900c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 --2- 172.21.15.33:0/757759041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec9006a6a0 0x7fec900c7210 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fec840098e0 tx=0x7fec840091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 -- 172.21.15.33:0/757759041 shutdown_connections 2023-10-06T05:16:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 --2- 172.21.15.33:0/757759041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90155af0 0x7fec9006a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 --2- 172.21.15.33:0/757759041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec9006a6a0 0x7fec900c7210 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 -- 172.21.15.33:0/757759041 >> 172.21.15.33:0/757759041 conn(0x7fec9006d030 msgr2=0x7fec9006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 -- 172.21.15.33:0/757759041 shutdown_connections 2023-10-06T05:16:06.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.525+0000 7fec9559d700 1 -- 172.21.15.33:0/757759041 wait complete. 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.526+0000 7fec9559d700 1 Processor -- start 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.526+0000 7fec9559d700 1 -- start start 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec9559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006a6a0 0x7fec901444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec9559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900c8b50 0x7fec900c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec9559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 0x7fec900c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec9559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec900adad0 con 0x7fec90155af0 2023-10-06T05:16:06.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec9559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec90169220 con 0x7fec9006a6a0 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec9559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900ad7f0 con 0x7fec900c8b50 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec94d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 0x7fec900c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.527+0000 7fec94d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 0x7fec900c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34934/0 (socket says 172.21.15.33:34934) 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 -- 172.21.15.33:0/3361668908 learned_addr learned my addr 172.21.15.33:0/3361668908 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec8ffff700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006a6a0 0x7fec901444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec8f7fe700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900c8b50 0x7fec900c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 -- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900c8b50 msgr2=0x7fec900c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900c8b50 0x7fec900c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 -- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006a6a0 msgr2=0x7fec901444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006a6a0 0x7fec901444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 -- 172.21.15.33:0/3361668908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84009420 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec8ffff700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006a6a0 0x7fec901444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec8f7fe700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900c8b50 0x7fec900c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec94d9c700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 0x7fec900c5bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec8800ccc0 tx=0x7fec88001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8800ab10 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8800ac70 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880070f0 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.528+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec90168bd0 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.529+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec90144a00 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.529+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90168a30 con 0x7fec90155af0 2023-10-06T05:16:06.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.530+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec88007520 con 0x7fec90155af0 2023-10-06T05:16:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.531+0000 7fec77fff700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.531+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec88084ae0 con 0x7fec90155af0 2023-10-06T05:16:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.532+0000 7fec8ffff700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.532+0000 7fec8ffff700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec7c066f30 0x7fec7c0693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec80006f00 tx=0x7fec80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.535+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec90168a30 con 0x7fec90155af0 2023-10-06T05:16:06.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.789+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec90168a30 con 0x7fec90155af0 2023-10-06T05:16:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.790+0000 7fec77fff700 1 -- 172.21.15.33:0/3361668908 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fec90168a30 con 0x7fec90155af0 2023-10-06T05:16:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:06.791 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec7c066f30 msgr2=0x7fec7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec7c066f30 0x7fec7c0693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec80006f00 tx=0x7fec80008040 comp rx=0 tx=0).stop 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 msgr2=0x7fec900c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 0x7fec900c5bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec8800ccc0 tx=0x7fec88001a20 comp rx=0 tx=0).stop 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 shutdown_connections 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec90155af0 0x7fec900c5bf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006a6a0 0x7fec901444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 --2- 172.21.15.33:0/3361668908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900c8b50 0x7fec900c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 >> 172.21.15.33:0/3361668908 conn(0x7fec9006d030 msgr2=0x7fec9006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 shutdown_connections 2023-10-06T05:16:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:06.792+0000 7fec9559d700 1 -- 172.21.15.33:0/3361668908 wait complete. 2023-10-06T05:16:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:16:06.813 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:07.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.000+0000 7fe251d9d700 1 Processor -- start 2023-10-06T05:16:07.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.000+0000 7fe251d9d700 1 -- start start 2023-10-06T05:16:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.001+0000 7fe251d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe24c0c8050 0x7fe24c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.001+0000 7fe251d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c06aad0 0x7fe24c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.001+0000 7fe251d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c0c9cb0 con 0x7fe24c13faa0 2023-10-06T05:16:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.001+0000 7fe251d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0ca270 con 0x7fe24c0c8050 2023-10-06T05:16:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.001+0000 7fe251d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c06b840 con 0x7fe24c06aad0 2023-10-06T05:16:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe24bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe24c0c8050 0x7fe24c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe250d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe24c13faa0 0x7fe24c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57320/0 (socket says 172.21.15.33:57320) 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe250d9b700 1 -- 172.21.15.33:0/3831011235 learned_addr learned my addr 172.21.15.33:0/3831011235 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe25159c700 1 --2- 172.21.15.33:0/3831011235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c06aad0 0x7fe24c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe24b7fe700 1 -- 172.21.15.33:0/3831011235 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028204541 0 0) 0x7fe24c0c9cb0 con 0x7fe24c13faa0 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe24bfff700 1 -- 172.21.15.33:0/3831011235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c06aad0 msgr2=0x7fe24c06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe24bfff700 1 --2- 172.21.15.33:0/3831011235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c06aad0 0x7fe24c06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe24bfff700 1 -- 172.21.15.33:0/3831011235 >> v1:172.21.15.33:6789/0 conn(0x7fe24c13faa0 legacy=0x7fe24c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.002+0000 7fe24bfff700 1 -- 172.21.15.33:0/3831011235 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c06bb20 con 0x7fe24c0c8050 2023-10-06T05:16:07.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe24bfff700 1 --2- 172.21.15.33:0/3831011235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c0c8050 0x7fe24c13fe70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe2380098e0 tx=0x7fe2380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d017a665c0ffb16d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe24b7fe700 1 -- 172.21.15.33:0/3831011235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800b640 con 0x7fe24c0c8050 2023-10-06T05:16:07.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe24b7fe700 1 -- 172.21.15.33:0/3831011235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800b7a0 con 0x7fe24c0c8050 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe24b7fe700 1 -- 172.21.15.33:0/3831011235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800bb40 con 0x7fe24c0c8050 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe251d9d700 1 -- 172.21.15.33:0/3831011235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c0c8050 msgr2=0x7fe24c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe251d9d700 1 --2- 172.21.15.33:0/3831011235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c0c8050 0x7fe24c13fe70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe2380098e0 tx=0x7fe2380091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe251d9d700 1 -- 172.21.15.33:0/3831011235 shutdown_connections 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe251d9d700 1 --2- 172.21.15.33:0/3831011235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c06aad0 0x7fe24c06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe251d9d700 1 --2- 172.21.15.33:0/3831011235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c0c8050 0x7fe24c13fe70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.003+0000 7fe251d9d700 1 -- 172.21.15.33:0/3831011235 >> 172.21.15.33:0/3831011235 conn(0x7fe24c06d030 msgr2=0x7fe24c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.004+0000 7fe251d9d700 1 -- 172.21.15.33:0/3831011235 shutdown_connections 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.004+0000 7fe251d9d700 1 -- 172.21.15.33:0/3831011235 wait complete. 2023-10-06T05:16:07.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.005+0000 7fe251d9d700 1 Processor -- start 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.005+0000 7fe251d9d700 1 -- start start 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe251d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 0x7fe24c138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe251d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0c8050 0x7fe24c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe251d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c13faa0 0x7fe24c1376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe251d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c138b40 con 0x7fe24c06aad0 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe251d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c137f30 con 0x7fe24c0c8050 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe251d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c138210 con 0x7fe24c13faa0 2023-10-06T05:16:07.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe250d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 0x7fe24c138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe250d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 0x7fe24c138fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34962/0 (socket says 172.21.15.33:34962) 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.006+0000 7fe250d9b700 1 -- 172.21.15.33:0/999185486 learned_addr learned my addr 172.21.15.33:0/999185486 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe25159c700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c13faa0 0x7fe24c1376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe24bfff700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0c8050 0x7fe24c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe250d9b700 1 -- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c13faa0 msgr2=0x7fe24c1376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe250d9b700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c13faa0 0x7fe24c1376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe250d9b700 1 -- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0c8050 msgr2=0x7fe24c1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe250d9b700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0c8050 0x7fe24c1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe250d9b700 1 -- 172.21.15.33:0/999185486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238009420 con 0x7fe24c06aad0 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe24bfff700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0c8050 0x7fe24c1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe250d9b700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 0x7fe24c138fd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c5b0 tx=0x7fe240001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240005b30 con 0x7fe24c06aad0 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe240005c90 con 0x7fe24c06aad0 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240007300 con 0x7fe24c06aad0 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.007+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c1384f0 con 0x7fe24c06aad0 2023-10-06T05:16:07.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.008+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c156930 con 0x7fe24c06aad0 2023-10-06T05:16:07.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.008+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c04c380 con 0x7fe24c06aad0 2023-10-06T05:16:07.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.009+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe24001b3b0 con 0x7fe24c06aad0 2023-10-06T05:16:07.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.011+0000 7fe248ff9700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.011+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe240083ca0 con 0x7fe24c06aad0 2023-10-06T05:16:07.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.011+0000 7fe24bfff700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.012+0000 7fe24bfff700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe230066f30 0x7fe2300693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe2380093f0 tx=0x7fe238017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.015+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe240050cd0 con 0x7fe24c06aad0 2023-10-06T05:16:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.217+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe24c138e20 con 0x7fe24c06aad0 2023-10-06T05:16:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.218+0000 7fe248ff9700 1 -- 172.21.15.33:0/999185486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16961 (secure 0 0 0) 0x7fe240017020 con 0x7fe24c06aad0 2023-10-06T05:16:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:07.220 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:06.666472+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:50.385790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:51.751034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe230066f30 msgr2=0x7fe2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe230066f30 0x7fe2300693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe2380093f0 tx=0x7fe238017000 comp rx=0 tx=0).stop 2023-10-06T05:16:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 msgr2=0x7fe24c138fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 0x7fe24c138fd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c5b0 tx=0x7fe240001070 comp rx=0 tx=0).stop 2023-10-06T05:16:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 shutdown_connections 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe24c13faa0 0x7fe24c1376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe24c06aad0 0x7fe24c138fd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 --2- 172.21.15.33:0/999185486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe24c0c8050 0x7fe24c1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 >> 172.21.15.33:0/999185486 conn(0x7fe24c06d030 msgr2=0x7fe24c0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 shutdown_connections 2023-10-06T05:16:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.220+0000 7fe251d9d700 1 -- 172.21.15.33:0/999185486 wait complete. 2023-10-06T05:16:07.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:15:50.385790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:15:51.751034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:16:07.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:16:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.429+0000 7efe06f9d700 1 Processor -- start 2023-10-06T05:16:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.429+0000 7efe06f9d700 1 -- start start 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe06f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efe00155a50 0x7efe00155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe06f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe000c9490 0x7efe000af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe06f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000af950 0x7efe000afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe06f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000b06b0 con 0x7efe000c9490 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe06f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000c99e0 con 0x7efe00155a50 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe06f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000c9cc0 con 0x7efe000af950 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe0579a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe000c9490 0x7efe000af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe0579a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe000c9490 0x7efe000af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34966/0 (socket says 172.21.15.33:34966) 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe0579a700 1 -- 172.21.15.33:0/953883367 learned_addr learned my addr 172.21.15.33:0/953883367 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe0679c700 1 --2- 172.21.15.33:0/953883367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000af950 0x7efe000afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.430+0000 7efe05f9b700 1 --2- 172.21.15.33:0/953883367 >> v2:172.21.15.167:3300/0 conn(0x7efe00155a50 0x7efe00155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe0579a700 1 -- 172.21.15.33:0/953883367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000af950 msgr2=0x7efe000afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe0579a700 1 --2- 172.21.15.33:0/953883367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000af950 0x7efe000afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe0579a700 1 -- 172.21.15.33:0/953883367 >> v2:172.21.15.167:3300/0 conn(0x7efe00155a50 msgr2=0x7efe00155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe0579a700 1 --2- 172.21.15.33:0/953883367 >> v2:172.21.15.167:3300/0 conn(0x7efe00155a50 0x7efe00155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe0579a700 1 -- 172.21.15.33:0/953883367 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000c9fa0 con 0x7efe000c9490 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe0579a700 1 --2- 172.21.15.33:0/953883367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000c9490 0x7efe000af410 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efe000cd1e0 tx=0x7efdf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a57ab817a1fd25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe04f99700 1 -- 172.21.15.33:0/953883367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000b640 con 0x7efe000c9490 2023-10-06T05:16:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe04f99700 1 -- 172.21.15.33:0/953883367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000b7a0 con 0x7efe000c9490 2023-10-06T05:16:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.431+0000 7efe04f99700 1 -- 172.21.15.33:0/953883367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000bb60 con 0x7efe000c9490 2023-10-06T05:16:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 -- 172.21.15.33:0/953883367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000c9490 msgr2=0x7efe000af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 --2- 172.21.15.33:0/953883367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000c9490 0x7efe000af410 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efe000cd1e0 tx=0x7efdf0009250 comp rx=0 tx=0).stop 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 -- 172.21.15.33:0/953883367 shutdown_connections 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 --2- 172.21.15.33:0/953883367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000af950 0x7efe000afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 --2- 172.21.15.33:0/953883367 >> v2:172.21.15.167:3300/0 conn(0x7efe00155a50 0x7efe00155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 --2- 172.21.15.33:0/953883367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000c9490 0x7efe000af410 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 -- 172.21.15.33:0/953883367 >> 172.21.15.33:0/953883367 conn(0x7efe0006d030 msgr2=0x7efe000cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 -- 172.21.15.33:0/953883367 shutdown_connections 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.432+0000 7efe06f9d700 1 -- 172.21.15.33:0/953883367 wait complete. 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.434+0000 7efe06f9d700 1 Processor -- start 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.434+0000 7efe06f9d700 1 -- start start 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe06f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 0x7efe000a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe06f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000c9490 0x7efe000a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe06f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe00155a50 0x7efe00150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe06f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000a74c0 con 0x7efe000af950 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe06f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000a9c00 con 0x7efe00155a50 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe06f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000a6bd0 con 0x7efe000c9490 2023-10-06T05:16:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 0x7efe000a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 0x7efe000a5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34972/0 (socket says 172.21.15.33:34972) 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 -- 172.21.15.33:0/845988408 learned_addr learned my addr 172.21.15.33:0/845988408 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe0579a700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000c9490 0x7efe000a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe0679c700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe00155a50 0x7efe00150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 -- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000c9490 msgr2=0x7efe000a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000c9490 0x7efe000a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 -- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe00155a50 msgr2=0x7efe00150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe00155a50 0x7efe00150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe05f9b700 1 -- 172.21.15.33:0/845988408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00094f0 con 0x7efe000af950 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe0579a700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000c9490 0x7efe000a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:07.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.435+0000 7efe0679c700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe00155a50 0x7efe00150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.436+0000 7efe05f9b700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 0x7efe000a5f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00e9d0 tx=0x7efdfc00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.436+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00a620 con 0x7efe000af950 2023-10-06T05:16:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.436+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc00ee40 con 0x7efe000af950 2023-10-06T05:16:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.436+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00cba0 con 0x7efe000af950 2023-10-06T05:16:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.436+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe000a6e70 con 0x7efe000af950 2023-10-06T05:16:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.436+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe00151530 con 0x7efe000af950 2023-10-06T05:16:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.437+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0004c380 con 0x7efe000af950 2023-10-06T05:16:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.438+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdfc0113f0 con 0x7efe000af950 2023-10-06T05:16:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.439+0000 7efdf5ffb700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efde4066f30 0x7efde40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.439+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6116+0+0 (secure 0 0 0) 0x7efdfc015030 con 0x7efe000af950 2023-10-06T05:16:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.440+0000 7efe0579a700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efde4066f30 0x7efde40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.440+0000 7efe0579a700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efde4066f30 0x7efde40693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe000cd1e0 tx=0x7efdf0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.443+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdfc051410 con 0x7efe000af950 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.644+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe000a77a0 con 0x7efe000af950 2023-10-06T05:16:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.644+0000 7efdf5ffb700 1 -- 172.21.15.33:0/845988408 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdfc054a30 con 0x7efe000af950 2023-10-06T05:16:07.646 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:16:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efde4066f30 msgr2=0x7efde40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efde4066f30 0x7efde40693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe000cd1e0 tx=0x7efdf0017000 comp rx=0 tx=0).stop 2023-10-06T05:16:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 msgr2=0x7efe000a5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 0x7efe000a5f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00e9d0 tx=0x7efdfc00a330 comp rx=0 tx=0).stop 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 shutdown_connections 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efde4066f30 0x7efde40693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe00155a50 0x7efe00150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe000af950 0x7efe000a5f90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 --2- 172.21.15.33:0/845988408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe000c9490 0x7efe000a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 >> 172.21.15.33:0/845988408 conn(0x7efe0006d030 msgr2=0x7efe000aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.646+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 shutdown_connections 2023-10-06T05:16:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.647+0000 7efe06f9d700 1 -- 172.21.15.33:0/845988408 wait complete. 2023-10-06T05:16:07.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:16:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.858+0000 7fc51b59e700 1 Processor -- start 2023-10-06T05:16:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.858+0000 7fc51b59e700 1 -- start start 2023-10-06T05:16:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.859+0000 7fc51b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc51c0af540 0x7fc51c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.859+0000 7fc51b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0cd5e0 0x7fc51c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.859+0000 7fc51b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c15d660 con 0x7fc51c15a650 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.859+0000 7fc51b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c0ce890 con 0x7fc51c0af540 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.859+0000 7fc51b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c15aac0 con 0x7fc51c0cd5e0 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc51c0af540 0x7fc51c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc51c0af540 0x7fc51c0af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34978/0 (socket says 172.21.15.33:34978) 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 -- 172.21.15.33:0/3825422466 learned_addr learned my addr 172.21.15.33:0/3825422466 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51ad9d700 1 --2- 172.21.15.33:0/3825422466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0cd5e0 0x7fc51c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 -- 172.21.15.33:0/3825422466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0cd5e0 msgr2=0x7fc51c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 --2- 172.21.15.33:0/3825422466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0cd5e0 0x7fc51c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 -- 172.21.15.33:0/3825422466 >> v1:172.21.15.167:6789/0 conn(0x7fc51c15a650 legacy=0x7fc51c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 -- 172.21.15.33:0/3825422466 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c15d890 con 0x7fc51c0af540 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.860+0000 7fc51a59c700 1 --2- 172.21.15.33:0/3825422466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c0af540 0x7fc51c0af910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099b0 tx=0x7fc50c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284e9b17e016c995 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51959a700 1 -- 172.21.15.33:0/3825422466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c009ea0 con 0x7fc51c0af540 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51959a700 1 -- 172.21.15.33:0/3825422466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c00b860 con 0x7fc51c0af540 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51959a700 1 -- 172.21.15.33:0/3825422466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00bc00 con 0x7fc51c0af540 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 -- 172.21.15.33:0/3825422466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c0af540 msgr2=0x7fc51c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 --2- 172.21.15.33:0/3825422466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c0af540 0x7fc51c0af910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099b0 tx=0x7fc50c009250 comp rx=0 tx=0).stop 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 -- 172.21.15.33:0/3825422466 shutdown_connections 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 --2- 172.21.15.33:0/3825422466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c0af540 0x7fc51c0af910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 --2- 172.21.15.33:0/3825422466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0cd5e0 0x7fc51c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 -- 172.21.15.33:0/3825422466 >> 172.21.15.33:0/3825422466 conn(0x7fc51c06d9b0 msgr2=0x7fc51c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 -- 172.21.15.33:0/3825422466 shutdown_connections 2023-10-06T05:16:07.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.861+0000 7fc51b59e700 1 -- 172.21.15.33:0/3825422466 wait complete. 2023-10-06T05:16:07.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.863+0000 7fc51b59e700 1 Processor -- start 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.863+0000 7fc51b59e700 1 -- start start 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0af540 0x7fc51c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc51c0cd5e0 0x7fc51c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 0x7fc51c0b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0aecb0 con 0x7fc51c15a650 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0b1ed0 con 0x7fc51c0cd5e0 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0b21b0 con 0x7fc51c0af540 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 0x7fc51c0b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 0x7fc51c0b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34988/0 (socket says 172.21.15.33:34988) 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51ad9d700 1 -- 172.21.15.33:0/4003689895 learned_addr learned my addr 172.21.15.33:0/4003689895 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:07.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51a59c700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0af540 0x7fc51c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc519d9b700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc51c0cd5e0 0x7fc51c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51ad9d700 1 -- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0af540 msgr2=0x7fc51c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.864+0000 7fc51ad9d700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0af540 0x7fc51c0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc51ad9d700 1 -- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc51c0cd5e0 msgr2=0x7fc51c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc51ad9d700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc51c0cd5e0 0x7fc51c0b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc51ad9d700 1 -- 172.21.15.33:0/4003689895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0094f0 con 0x7fc51c15a650 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc51ad9d700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 0x7fc51c0b37b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc514009e90 tx=0x7fc51400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400cb70 con 0x7fc51c15a650 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51400f040 con 0x7fc51c15a650 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514014480 con 0x7fc51c15a650 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c0b2490 con 0x7fc51c15a650 2023-10-06T05:16:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.865+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c0b1180 con 0x7fc51c15a650 2023-10-06T05:16:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.866+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c0b29b0 con 0x7fc51c15a650 2023-10-06T05:16:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.867+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5140147f0 con 0x7fc51c15a650 2023-10-06T05:16:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.868+0000 7fc50a7fc700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4fc066f60 0x7fc4fc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.868+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc514085400 con 0x7fc51c15a650 2023-10-06T05:16:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.869+0000 7fc51a59c700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4fc066f60 0x7fc4fc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.869+0000 7fc51a59c700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4fc066f60 0x7fc4fc069410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099b0 tx=0x7fc50c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:07.872+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc51c0b29b0 con 0x7fc51c15a650 2023-10-06T05:16:08.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.073+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc51c0b29b0 con 0x7fc51c15a650 2023-10-06T05:16:08.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.074+0000 7fc50a7fc700 1 -- 172.21.15.33:0/4003689895 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc51c0b29b0 con 0x7fc51c15a650 2023-10-06T05:16:08.075 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4fc066f60 msgr2=0x7fc4fc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4fc066f60 0x7fc4fc069410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099b0 tx=0x7fc50c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 msgr2=0x7fc51c0b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 0x7fc51c0b37b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc514009e90 tx=0x7fc51400b040 comp rx=0 tx=0).stop 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 shutdown_connections 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc4fc066f60 0x7fc4fc069410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc51c15a650 0x7fc51c0b37b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc51c0af540 0x7fc51c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 --2- 172.21.15.33:0/4003689895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc51c0cd5e0 0x7fc51c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 >> 172.21.15.33:0/4003689895 conn(0x7fc51c06d9b0 msgr2=0x7fc51c156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 shutdown_connections 2023-10-06T05:16:08.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.076+0000 7fc51b59e700 1 -- 172.21.15.33:0/4003689895 wait complete. 2023-10-06T05:16:08.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:16:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.285+0000 7f8aa1b9d700 1 Processor -- start 2023-10-06T05:16:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.285+0000 7f8aa1b9d700 1 -- start start 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa1b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a9c06c7b0 0x7f8a9c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa1b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a9c0aff90 0x7f8a9c0a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa1b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c15d3a0 con 0x7f8a9c06cb80 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa1b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c0b0470 con 0x7f8a9c06c7b0 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa1b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c0b0750 con 0x7f8a9c0aff90 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa0b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a9c06c7b0 0x7f8a9c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa0b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a9c06c7b0 0x7f8a9c067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35002/0 (socket says 172.21.15.33:35002) 2023-10-06T05:16:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.286+0000 7f8aa0b9b700 1 -- 172.21.15.33:0/3592960814 learned_addr learned my addr 172.21.15.33:0/3592960814 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8a9bfff700 1 --2- 172.21.15.33:0/3592960814 >> v2:172.21.15.167:3300/0 conn(0x7f8a9c0aff90 0x7f8a9c0a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8aa0b9b700 1 -- 172.21.15.33:0/3592960814 >> v2:172.21.15.167:3300/0 conn(0x7f8a9c0aff90 msgr2=0x7f8a9c0a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8aa0b9b700 1 --2- 172.21.15.33:0/3592960814 >> v2:172.21.15.167:3300/0 conn(0x7f8a9c0aff90 0x7f8a9c0a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8aa0b9b700 1 -- 172.21.15.33:0/3592960814 >> v1:172.21.15.167:6789/0 conn(0x7f8a9c06cb80 legacy=0x7f8a9c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8aa0b9b700 1 -- 172.21.15.33:0/3592960814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0b0a30 con 0x7f8a9c06c7b0 2023-10-06T05:16:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8a9bfff700 1 --2- 172.21.15.33:0/3592960814 >> v2:172.21.15.167:3300/0 conn(0x7f8a9c0aff90 0x7f8a9c0a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8aa0b9b700 1 --2- 172.21.15.33:0/3592960814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c067fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c06ee90 tx=0x7f8a90009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7280c73b00c708 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.287+0000 7f8a9b7fe700 1 -- 172.21.15.33:0/3592960814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005c10 con 0x7f8a9c06c7b0 2023-10-06T05:16:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8a9b7fe700 1 -- 172.21.15.33:0/3592960814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90005d70 con 0x7f8a9c06c7b0 2023-10-06T05:16:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8a9b7fe700 1 -- 172.21.15.33:0/3592960814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000b880 con 0x7f8a9c06c7b0 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/3592960814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c06c7b0 msgr2=0x7f8a9c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/3592960814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c067fb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c06ee90 tx=0x7f8a90009220 comp rx=0 tx=0).stop 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/3592960814 shutdown_connections 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/3592960814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c067fb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/3592960814 >> v2:172.21.15.167:3300/0 conn(0x7f8a9c0aff90 0x7f8a9c0a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/3592960814 >> 172.21.15.33:0/3592960814 conn(0x7f8a9c06d670 msgr2=0x7f8a9c06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/3592960814 shutdown_connections 2023-10-06T05:16:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.288+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/3592960814 wait complete. 2023-10-06T05:16:08.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.290+0000 7f8aa1b9d700 1 Processor -- start 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.290+0000 7f8aa1b9d700 1 -- start start 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa1b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa1b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a9c06cb80 0x7f8a9c0ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa1b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 0x7f8a9c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa1b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0aef00 con 0x7f8a9c0aff90 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa1b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1398d0 con 0x7f8a9c06c7b0 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa1b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0ae5d0 con 0x7f8a9c06cb80 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.291+0000 7f8aa139c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 0x7f8a9c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 0x7f8a9c0af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35010/0 (socket says 172.21.15.33:35010) 2023-10-06T05:16:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 -- 172.21.15.33:0/873840516 learned_addr learned my addr 172.21.15.33:0/873840516 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8a9bfff700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a9c06cb80 0x7f8a9c0ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa0b9b700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 -- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a9c06cb80 msgr2=0x7f8a9c0ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a9c06cb80 0x7f8a9c0ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 -- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c06c7b0 msgr2=0x7f8a9c139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 -- 172.21.15.33:0/873840516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90009480 con 0x7f8a9c0aff90 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.292+0000 7f8aa139c700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 0x7f8a9c0af390 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c8e0 tx=0x7f8a94001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.293+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94012630 con 0x7f8a9c0aff90 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.293+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a940127b0 con 0x7f8a9c0aff90 2023-10-06T05:16:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.293+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c0ae8b0 con 0x7f8a9c0aff90 2023-10-06T05:16:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.293+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94012b70 con 0x7f8a9c0aff90 2023-10-06T05:16:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.293+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c0acbf0 con 0x7f8a9c0aff90 2023-10-06T05:16:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.294+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c04c380 con 0x7f8a9c0aff90 2023-10-06T05:16:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.295+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a94003030 con 0x7f8a9c0aff90 2023-10-06T05:16:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.296+0000 7f8a98ff9700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a80066f30 0x7f8a800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.297+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f8a940845c0 con 0x7f8a9c0aff90 2023-10-06T05:16:08.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.297+0000 7f8aa0b9b700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a80066f30 0x7f8a800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.298+0000 7f8aa0b9b700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a80066f30 0x7f8a800693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8a90007fd0 tx=0x7f8a90016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.301+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a940515f0 con 0x7f8a9c0aff90 2023-10-06T05:16:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.503+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a9c0b0d10 con 0x7f8a9c0aff90 2023-10-06T05:16:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.503+0000 7f8a98ff9700 1 -- 172.21.15.33:0/873840516 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a94054c10 con 0x7f8a9c0aff90 2023-10-06T05:16:08.505 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:16:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a80066f30 msgr2=0x7f8a800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a80066f30 0x7f8a800693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8a90007fd0 tx=0x7f8a90016000 comp rx=0 tx=0).stop 2023-10-06T05:16:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 msgr2=0x7f8a9c0af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 0x7f8a9c0af390 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c8e0 tx=0x7f8a94001710 comp rx=0 tx=0).stop 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 shutdown_connections 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a80066f30 0x7f8a800693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a9c0aff90 0x7f8a9c0af390 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a9c06c7b0 0x7f8a9c139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 --2- 172.21.15.33:0/873840516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a9c06cb80 0x7f8a9c0ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.505+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 >> 172.21.15.33:0/873840516 conn(0x7f8a9c06d670 msgr2=0x7f8a9c14a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.506+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 shutdown_connections 2023-10-06T05:16:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.506+0000 7f8aa1b9d700 1 -- 172.21.15.33:0/873840516 wait complete. 2023-10-06T05:16:08.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:16:08.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.715+0000 7f53b999d700 1 Processor -- start 2023-10-06T05:16:08.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.715+0000 7f53b999d700 1 -- start start 2023-10-06T05:16:08.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.716+0000 7f53b999d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53b40cd5e0 0x7f53b40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.716+0000 7f53b999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b415a650 0x7f53b415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.716+0000 7f53b999d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b415d660 con 0x7f53b40af540 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.716+0000 7f53b999d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40ce890 con 0x7f53b40cd5e0 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.716+0000 7f53b999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b415aab0 con 0x7f53b415a650 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53b40cd5e0 0x7f53b40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53b40cd5e0 0x7f53b40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35026/0 (socket says 172.21.15.33:35026) 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 -- 172.21.15.33:0/1310797048 learned_addr learned my addr 172.21.15.33:0/1310797048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b919c700 1 --2- 172.21.15.33:0/1310797048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b415a650 0x7f53b415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 -- 172.21.15.33:0/1310797048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b415a650 msgr2=0x7f53b415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:08.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 --2- 172.21.15.33:0/1310797048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b415a650 0x7f53b415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 -- 172.21.15.33:0/1310797048 >> v1:172.21.15.167:6789/0 conn(0x7f53b40af540 legacy=0x7f53b40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 -- 172.21.15.33:0/1310797048 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b415d890 con 0x7f53b40cd5e0 2023-10-06T05:16:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.717+0000 7f53b3fff700 1 --2- 172.21.15.33:0/1310797048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b40cd5e0 0x7f53b40af910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f53a00099b0 tx=0x7f53a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e06aaa5d77931f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b37fe700 1 -- 172.21.15.33:0/1310797048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000b860 con 0x7f53b40cd5e0 2023-10-06T05:16:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b37fe700 1 -- 172.21.15.33:0/1310797048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a000b9c0 con 0x7f53b40cd5e0 2023-10-06T05:16:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b37fe700 1 -- 172.21.15.33:0/1310797048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000bd60 con 0x7f53b40cd5e0 2023-10-06T05:16:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 -- 172.21.15.33:0/1310797048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b40cd5e0 msgr2=0x7f53b40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 --2- 172.21.15.33:0/1310797048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b40cd5e0 0x7f53b40af910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f53a00099b0 tx=0x7f53a0009250 comp rx=0 tx=0).stop 2023-10-06T05:16:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 -- 172.21.15.33:0/1310797048 shutdown_connections 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 --2- 172.21.15.33:0/1310797048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b415a650 0x7f53b415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 --2- 172.21.15.33:0/1310797048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b40cd5e0 0x7f53b40af910 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 -- 172.21.15.33:0/1310797048 >> 172.21.15.33:0/1310797048 conn(0x7f53b406d9b0 msgr2=0x7f53b406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 -- 172.21.15.33:0/1310797048 shutdown_connections 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.718+0000 7f53b999d700 1 -- 172.21.15.33:0/1310797048 wait complete. 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.720+0000 7f53b999d700 1 Processor -- start 2023-10-06T05:16:08.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.720+0000 7f53b999d700 1 -- start start 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b999d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b40af540 0x7f53b414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b40cd5e0 0x7f53b415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b999d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 0x7f53b4158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b999d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41555d0 con 0x7f53b415a650 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b999d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40c8100 con 0x7f53b40af540 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40b1440 con 0x7f53b40cd5e0 2023-10-06T05:16:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 0x7f53b4158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 0x7f53b4158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35028/0 (socket says 172.21.15.33:35028) 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 -- 172.21.15.33:0/1015083016 learned_addr learned my addr 172.21.15.33:0/1015083016 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b3fff700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b40cd5e0 0x7f53b415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b899b700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b40af540 0x7f53b414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 -- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b40cd5e0 msgr2=0x7f53b415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b40cd5e0 0x7f53b415b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 -- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b40af540 msgr2=0x7f53b414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b40af540 0x7f53b414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b919c700 1 -- 172.21.15.33:0/1015083016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a00094f0 con 0x7f53b415a650 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.721+0000 7f53b3fff700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b40cd5e0 0x7f53b415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.722+0000 7f53b919c700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 0x7f53b4158c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009fa0 tx=0x7f53ac009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.722+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00ec70 con 0x7f53b415a650 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.722+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac00edd0 con 0x7f53b415a650 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.722+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac014670 con 0x7f53b415a650 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.722+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b40b1720 con 0x7f53b415a650 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.722+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b40b1ce0 con 0x7f53b415a650 2023-10-06T05:16:08.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.723+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b40b3300 con 0x7f53b415a650 2023-10-06T05:16:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.724+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53ac0149e0 con 0x7f53b415a650 2023-10-06T05:16:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.725+0000 7f53b0ff9700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5398066f30 0x7f53980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.725+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f53ac084e60 con 0x7f53b415a650 2023-10-06T05:16:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.725+0000 7f53b899b700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5398066f30 0x7f53980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.726+0000 7f53b899b700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5398066f30 0x7f53980693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f53a80039e0 tx=0x7f53a8007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.729+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53b40b3300 con 0x7f53b415a650 2023-10-06T05:16:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.931+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53b40b3300 con 0x7f53b415a650 2023-10-06T05:16:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.931+0000 7f53b0ff9700 1 -- 172.21.15.33:0/1015083016 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53b40b3300 con 0x7f53b415a650 2023-10-06T05:16:08.934 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:16:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5398066f30 msgr2=0x7f53980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5398066f30 0x7f53980693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f53a80039e0 tx=0x7f53a8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 msgr2=0x7f53b4158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 0x7f53b4158c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009fa0 tx=0x7f53ac009900 comp rx=0 tx=0).stop 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 shutdown_connections 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5398066f30 0x7f53980693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53b415a650 0x7f53b4158c70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b40af540 0x7f53b414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 --2- 172.21.15.33:0/1015083016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b40cd5e0 0x7f53b415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 >> 172.21.15.33:0/1015083016 conn(0x7f53b406d9b0 msgr2=0x7f53b415bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 shutdown_connections 2023-10-06T05:16:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:08.934+0000 7f53b999d700 1 -- 172.21.15.33:0/1015083016 wait complete. 2023-10-06T05:16:08.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.139+0000 7f28ea59d700 1 Processor -- start 2023-10-06T05:16:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.139+0000 7f28ea59d700 1 -- start start 2023-10-06T05:16:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28ea59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28e413faa0 0x7f28e413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28ea59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28e40c7940 0x7f28e40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28ea59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e40c9cb0 con 0x7f28e406aad0 2023-10-06T05:16:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28ea59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e406b640 con 0x7f28e413faa0 2023-10-06T05:16:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28ea59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e40ca270 con 0x7f28e40c7940 2023-10-06T05:16:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28e959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28e413faa0 0x7f28e413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28e9d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f28e406aad0 0x7f28e406af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57326/0 (socket says 172.21.15.33:57326) 2023-10-06T05:16:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.140+0000 7f28e9d9c700 1 -- 172.21.15.33:0/4216767580 learned_addr learned my addr 172.21.15.33:0/4216767580 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e8d9a700 1 --2- 172.21.15.33:0/4216767580 >> v2:172.21.15.167:3300/0 conn(0x7f28e40c7940 0x7f28e40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28dbfff700 1 -- 172.21.15.33:0/4216767580 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963077979 0 0) 0x7f28e40c9cb0 con 0x7f28e406aad0 2023-10-06T05:16:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e959b700 1 -- 172.21.15.33:0/4216767580 >> v2:172.21.15.167:3300/0 conn(0x7f28e40c7940 msgr2=0x7f28e40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e959b700 1 --2- 172.21.15.33:0/4216767580 >> v2:172.21.15.167:3300/0 conn(0x7f28e40c7940 0x7f28e40c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e959b700 1 -- 172.21.15.33:0/4216767580 >> v1:172.21.15.33:6789/0 conn(0x7f28e406aad0 legacy=0x7f28e406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e959b700 1 -- 172.21.15.33:0/4216767580 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e406b840 con 0x7f28e413faa0 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e8d9a700 1 --2- 172.21.15.33:0/4216767580 >> v2:172.21.15.167:3300/0 conn(0x7f28e40c7940 0x7f28e40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28e959b700 1 --2- 172.21.15.33:0/4216767580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e413faa0 0x7f28e413fe70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f28e40cbef0 tx=0x7f28e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f6e169b0b876e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.141+0000 7f28dbfff700 1 -- 172.21.15.33:0/4216767580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000b6b0 con 0x7f28e413faa0 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28dbfff700 1 -- 172.21.15.33:0/4216767580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e000b810 con 0x7f28e413faa0 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28dbfff700 1 -- 172.21.15.33:0/4216767580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000bbb0 con 0x7f28e413faa0 2023-10-06T05:16:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 -- 172.21.15.33:0/4216767580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e413faa0 msgr2=0x7f28e413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 --2- 172.21.15.33:0/4216767580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e413faa0 0x7f28e413fe70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f28e40cbef0 tx=0x7f28e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 -- 172.21.15.33:0/4216767580 shutdown_connections 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 --2- 172.21.15.33:0/4216767580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e413faa0 0x7f28e413fe70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 --2- 172.21.15.33:0/4216767580 >> v2:172.21.15.167:3300/0 conn(0x7f28e40c7940 0x7f28e40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 -- 172.21.15.33:0/4216767580 >> 172.21.15.33:0/4216767580 conn(0x7f28e406d030 msgr2=0x7f28e406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 -- 172.21.15.33:0/4216767580 shutdown_connections 2023-10-06T05:16:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.142+0000 7f28ea59d700 1 -- 172.21.15.33:0/4216767580 wait complete. 2023-10-06T05:16:09.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.144+0000 7f28ea59d700 1 Processor -- start 2023-10-06T05:16:09.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.144+0000 7f28ea59d700 1 -- start start 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28ea59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 0x7f28e4159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28ea59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28e40c7940 0x7f28e4155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28ea59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e413faa0 0x7f28e415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28ea59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e4156d10 con 0x7f28e406aad0 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28ea59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e415e610 con 0x7f28e413faa0 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28ea59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e415dd70 con 0x7f28e40c7940 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 0x7f28e4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 0x7f28e4159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35038/0 (socket says 172.21.15.33:35038) 2023-10-06T05:16:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 -- 172.21.15.33:0/1893055095 learned_addr learned my addr 172.21.15.33:0/1893055095 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 -- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28e40c7940 msgr2=0x7f28e4155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28e40c7940 0x7f28e4155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 -- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e413faa0 msgr2=0x7f28e415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e413faa0 0x7f28e415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.145+0000 7f28e959b700 1 -- 172.21.15.33:0/1893055095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009420 con 0x7f28e406aad0 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.146+0000 7f28e959b700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 0x7f28e4159100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f28e40719c0 tx=0x7f28e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.146+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000bb90 con 0x7f28e406aad0 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.146+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e0011420 con 0x7f28e406aad0 2023-10-06T05:16:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.146+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e415e050 con 0x7f28e406aad0 2023-10-06T05:16:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.146+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00117a0 con 0x7f28e406aad0 2023-10-06T05:16:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.146+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e413ede0 con 0x7f28e406aad0 2023-10-06T05:16:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.147+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e404c380 con 0x7f28e406aad0 2023-10-06T05:16:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.148+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28e0011b90 con 0x7f28e406aad0 2023-10-06T05:16:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.149+0000 7f28d97fa700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28c8066f30 0x7f28c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.149+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f28e0085ff0 con 0x7f28e406aad0 2023-10-06T05:16:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.149+0000 7f28e8d9a700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28c8066f30 0x7f28c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.150+0000 7f28e8d9a700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28c8066f30 0x7f28c80693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006fd0 tx=0x7f28d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.153+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28e00b75a0 con 0x7f28e406aad0 2023-10-06T05:16:09.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.412+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e406bb20 con 0x7f28e406aad0 2023-10-06T05:16:09.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.412+0000 7f28d97fa700 1 -- 172.21.15.33:0/1893055095 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f28e0053040 con 0x7f28e406aad0 2023-10-06T05:16:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:09.414 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28c8066f30 msgr2=0x7f28c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28c8066f30 0x7f28c80693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006fd0 tx=0x7f28d4008040 comp rx=0 tx=0).stop 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 msgr2=0x7f28e4159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 0x7f28e4159100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f28e40719c0 tx=0x7f28e0017000 comp rx=0 tx=0).stop 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 shutdown_connections 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f28c8066f30 0x7f28c80693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e413faa0 0x7f28e415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28e406aad0 0x7f28e4159100 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 --2- 172.21.15.33:0/1893055095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28e40c7940 0x7f28e4155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 >> 172.21.15.33:0/1893055095 conn(0x7f28e406d030 msgr2=0x7f28e406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:09.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 shutdown_connections 2023-10-06T05:16:09.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.415+0000 7f28ea59d700 1 -- 172.21.15.33:0/1893055095 wait complete. 2023-10-06T05:16:09.418 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:16:09.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.619+0000 7f0fbb59e700 1 Processor -- start 2023-10-06T05:16:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.619+0000 7f0fbb59e700 1 -- start start 2023-10-06T05:16:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fbb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fbc0c6340 0x7f0fbc0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fbb59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc06c820 con 0x7f0fbc159520 2023-10-06T05:16:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fbb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc06a160 con 0x7f0fbc0c80d0 2023-10-06T05:16:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fbb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0c8800 con 0x7f0fbc0c6340 2023-10-06T05:16:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fba59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0fbc159520 0x7f0fbc0c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57332/0 (socket says 172.21.15.33:57332) 2023-10-06T05:16:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fba59c700 1 -- 172.21.15.33:0/385362042 learned_addr learned my addr 172.21.15.33:0/385362042 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fb9d9b700 1 --2- 172.21.15.33:0/385362042 >> v2:172.21.15.167:3300/0 conn(0x7f0fbc0c6340 0x7f0fbc0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.621+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331477586 0 0) 0x7f0fbc06c820 con 0x7f0fbc159520 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fa4003570 con 0x7f0fbc159520 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355812397 0 0) 0x7f0fbc06a160 con 0x7f0fbc0c80d0 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fbc06c820 con 0x7f0fbc0c80d0 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951140936 0 0) 0x7f0fa4003570 con 0x7f0fbc159520 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb9d9b700 1 -- 172.21.15.33:0/385362042 >> v1:172.21.15.167:6789/0 conn(0x7f0fbc0c80d0 legacy=0x7f0fbc06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb9d9b700 1 -- 172.21.15.33:0/385362042 >> v1:172.21.15.33:6789/0 conn(0x7f0fbc159520 legacy=0x7f0fbc0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb9d9b700 1 -- 172.21.15.33:0/385362042 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0c8ae0 con 0x7f0fbc0c6340 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.622+0000 7f0fb9d9b700 1 --2- 172.21.15.33:0/385362042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c6340 0x7f0fbc0c7b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00098e0 tx=0x7f0fb00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d08f2f678eabe30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac001470 con 0x7f0fbc0c6340 2023-10-06T05:16:09.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb0011420 con 0x7f0fbc0c6340 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fb959a700 1 -- 172.21.15.33:0/385362042 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00117a0 con 0x7f0fbc0c6340 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 -- 172.21.15.33:0/385362042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c6340 msgr2=0x7f0fbc0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/385362042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c6340 0x7f0fbc0c7b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00098e0 tx=0x7f0fb00091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 -- 172.21.15.33:0/385362042 shutdown_connections 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/385362042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c6340 0x7f0fbc0c7b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 -- 172.21.15.33:0/385362042 >> 172.21.15.33:0/385362042 conn(0x7f0fbc0cc930 msgr2=0x7f0fbc0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 -- 172.21.15.33:0/385362042 shutdown_connections 2023-10-06T05:16:09.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.623+0000 7f0fbb59e700 1 -- 172.21.15.33:0/385362042 wait complete. 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.625+0000 7f0fbb59e700 1 Processor -- start 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.625+0000 7f0fbb59e700 1 -- start start 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fbc0c6340 0x7f0fbc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c80d0 0x7f0fbc0ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 0x7f0fbc0ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc15a500 con 0x7f0fbc159520 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0ad3c0 con 0x7f0fbc0c80d0 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc1598f0 con 0x7f0fbc0c6340 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 0x7f0fbc0ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 0x7f0fbc0ace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35042/0 (socket says 172.21.15.33:35042) 2023-10-06T05:16:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 -- 172.21.15.33:0/2901449492 learned_addr learned my addr 172.21.15.33:0/2901449492 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fb9d9b700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c80d0 0x7f0fbc0ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fba59c700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fbc0c6340 0x7f0fbc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 -- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fbc0c6340 msgr2=0x7f0fbc0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fbc0c6340 0x7f0fbc0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 -- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c80d0 msgr2=0x7f0fbc0ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c80d0 0x7f0fbc0ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fbad9d700 1 -- 172.21.15.33:0/2901449492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0009420 con 0x7f0fbc159520 2023-10-06T05:16:09.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.626+0000 7f0fb9d9b700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c80d0 0x7f0fbc0ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.627+0000 7f0fbad9d700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 0x7f0fbc0ace80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c910 tx=0x7f0fb40027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.627+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400a7e0 con 0x7f0fbc159520 2023-10-06T05:16:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.627+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb400a960 con 0x7f0fbc159520 2023-10-06T05:16:09.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.627+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc159bd0 con 0x7f0fbc159520 2023-10-06T05:16:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.627+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400ad20 con 0x7f0fbc159520 2023-10-06T05:16:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.627+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc15a190 con 0x7f0fbc159520 2023-10-06T05:16:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.628+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc15a670 con 0x7f0fbc159520 2023-10-06T05:16:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.629+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fb4012630 con 0x7f0fbc159520 2023-10-06T05:16:09.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.630+0000 7f0faa7fc700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f9c066f30 0x7f0f9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.630+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f0fb4084c90 con 0x7f0fbc159520 2023-10-06T05:16:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.630+0000 7f0fba59c700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f9c066f30 0x7f0f9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.631+0000 7f0fba59c700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f9c066f30 0x7f0f9c0693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005b80 tx=0x7f0fac000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.634+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fb4051d70 con 0x7f0fbc159520 2023-10-06T05:16:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.837+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fbc04c380 con 0x7f0fbc159520 2023-10-06T05:16:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.838+0000 7f0faa7fc700 1 -- 172.21.15.33:0/2901449492 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16961 (secure 0 0 0) 0x7f0fb4055390 con 0x7f0fbc159520 2023-10-06T05:16:09.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:09.840 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":50,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:07.670746+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:50.385790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:51.751034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f9c066f30 msgr2=0x7f0f9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f9c066f30 0x7f0f9c0693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005b80 tx=0x7f0fac000bc0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 msgr2=0x7f0fbc0ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 0x7f0fbc0ace80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c910 tx=0x7f0fb40027a0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 shutdown_connections 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f9c066f30 0x7f0f9c0693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fbc159520 0x7f0fbc0ace80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fbc0c6340 0x7f0fbc0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 --2- 172.21.15.33:0/2901449492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fbc0c80d0 0x7f0fbc0ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 >> 172.21.15.33:0/2901449492 conn(0x7f0fbc0cc930 msgr2=0x7f0fbc0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:09.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 shutdown_connections 2023-10-06T05:16:09.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:09.840+0000 7f0fbb59e700 1 -- 172.21.15.33:0/2901449492 wait complete. 2023-10-06T05:16:09.859 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:16:09.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.046+0000 7fa241d9d700 1 Processor -- start 2023-10-06T05:16:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.046+0000 7fa241d9d700 1 -- start start 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.047+0000 7fa241d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa23c0ccfc0 0x7fa23c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.047+0000 7fa241d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23c0cbd80 con 0x7fa23c0cf3b0 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.047+0000 7fa241d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23c0af5f0 con 0x7fa23c0cc750 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.047+0000 7fa241d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23c0af8d0 con 0x7fa23c0ccfc0 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa23c0cf3b0 0x7fa23c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51342/0 (socket says 172.21.15.33:51342) 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23bfff700 1 -- 172.21.15.33:0/3341389564 learned_addr learned my addr 172.21.15.33:0/3341389564 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa240d9b700 1 --2- 172.21.15.33:0/3341389564 >> v2:172.21.15.167:3300/0 conn(0x7fa23c0ccfc0 0x7fa23c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354571230 0 0) 0x7fa23c0cbd80 con 0x7fa23c0cf3b0 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa22c003570 con 0x7fa23c0cf3b0 2023-10-06T05:16:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1797689780 0 0) 0x7fa22c003570 con 0x7fa23c0cf3b0 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 >> v2:172.21.15.167:3300/0 conn(0x7fa23c0ccfc0 msgr2=0x7fa23c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 --2- 172.21.15.33:0/3341389564 >> v2:172.21.15.167:3300/0 conn(0x7fa23c0ccfc0 0x7fa23c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 >> v1:172.21.15.167:6789/0 conn(0x7fa23c0cc750 legacy=0x7fa23c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0afbb0 con 0x7fa23c0cf3b0 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa228001470 con 0x7fa23c0cf3b0 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 >> v1:172.21.15.33:6789/0 conn(0x7fa23c0cf3b0 legacy=0x7fa23c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c003ab0 0x7fa22c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.048+0000 7fa23b7fe700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c004810 0x7fa22c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa23b7fe700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa22c005210 0x7fa22c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0af8d0 con 0x7fa22c003ab0 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c00c1b0 con 0x7fa22c005210 2023-10-06T05:16:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c00c370 con 0x7fa22c004810 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c003ab0 0x7fa22c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa24159c700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa22c005210 0x7fa22c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa23bfff700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c004810 0x7fa22c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 -- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c004810 msgr2=0x7fa22c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c004810 0x7fa22c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 -- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa22c005210 msgr2=0x7fa22c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa22c005210 0x7fa22c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 -- 172.21.15.33:0/3341389564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c003570 con 0x7fa22c003ab0 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa24159c700 1 -- 172.21.15.33:0/3341389564 reap_dead start 2023-10-06T05:16:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.049+0000 7fa240d9b700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c003ab0 0x7fa22c003ec0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa230009c00 tx=0x7fa2300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49936aefb456c9cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000e710 con 0x7fa22c003ab0 2023-10-06T05:16:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23000e870 con 0x7fa22c003ab0 2023-10-06T05:16:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa23b7fe700 1 -- 172.21.15.33:0/3341389564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000ec30 con 0x7fa22c003ab0 2023-10-06T05:16:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 -- 172.21.15.33:0/3341389564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c003ab0 msgr2=0x7fa22c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c003ab0 0x7fa22c003ec0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa230009c00 tx=0x7fa2300091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 -- 172.21.15.33:0/3341389564 shutdown_connections 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 --2- 172.21.15.33:0/3341389564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c003ab0 0x7fa22c003ec0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 -- 172.21.15.33:0/3341389564 >> 172.21.15.33:0/3341389564 conn(0x7fa23c06cff0 msgr2=0x7fa23c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 -- 172.21.15.33:0/3341389564 shutdown_connections 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.050+0000 7fa241d9d700 1 -- 172.21.15.33:0/3341389564 wait complete. 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.052+0000 7fa241d9d700 1 Processor -- start 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.052+0000 7fa241d9d700 1 -- start start 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa241d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c003ab0 0x7fa23c148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa241d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23c154000 0x7fa23c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa241d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 0x7fa23c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa241d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c157710 con 0x7fa23c156d60 2023-10-06T05:16:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa241d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0b58c0 con 0x7fa23c154000 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa241d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0b47e0 con 0x7fa22c003ab0 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 0x7fa23c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 0x7fa23c1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47692/0 (socket says 172.21.15.33:47692) 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 -- 172.21.15.33:0/869488956 learned_addr learned my addr 172.21.15.33:0/869488956 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa240d9b700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c003ab0 0x7fa23c148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa23bfff700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23c154000 0x7fa23c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 -- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c003ab0 msgr2=0x7fa23c148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c003ab0 0x7fa23c148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 -- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23c154000 msgr2=0x7fa23c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.053+0000 7fa24159c700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23c154000 0x7fa23c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa24159c700 1 -- 172.21.15.33:0/869488956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0b4c70 con 0x7fa23c156d60 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa240d9b700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c003ab0 0x7fa23c148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa24159c700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 0x7fa23c1571d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0cbcf0 tx=0x7fa234009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234006db0 con 0x7fa23c156d60 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa234003070 con 0x7fa23c156d60 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234003430 con 0x7fa23c156d60 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c0b60d0 con 0x7fa23c156d60 2023-10-06T05:16:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.054+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c0b5b10 con 0x7fa23c156d60 2023-10-06T05:16:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.055+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c0b4950 con 0x7fa23c156d60 2023-10-06T05:16:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.056+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa234001790 con 0x7fa23c156d60 2023-10-06T05:16:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.057+0000 7fa238ff9700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa220066f30 0x7fa2200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.057+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa234084110 con 0x7fa23c156d60 2023-10-06T05:16:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.058+0000 7fa240d9b700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa220066f30 0x7fa2200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.058+0000 7fa240d9b700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa220066f30 0x7fa2200693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa2300091e0 tx=0x7fa23001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.061+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa234051140 con 0x7fa23c156d60 2023-10-06T05:16:10.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.313+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa23c04c380 con 0x7fa23c156d60 2023-10-06T05:16:10.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.314+0000 7fa238ff9700 1 -- 172.21.15.33:0/869488956 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fa234054760 con 0x7fa23c156d60 2023-10-06T05:16:10.315 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:10.316 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:16:10.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa220066f30 msgr2=0x7fa2200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa220066f30 0x7fa2200693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa2300091e0 tx=0x7fa23001a000 comp rx=0 tx=0).stop 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 msgr2=0x7fa23c1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 0x7fa23c1571d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0cbcf0 tx=0x7fa234009040 comp rx=0 tx=0).stop 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 shutdown_connections 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa220066f30 0x7fa2200693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa23c156d60 0x7fa23c1571d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa22c003ab0 0x7fa23c148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 --2- 172.21.15.33:0/869488956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23c154000 0x7fa23c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 >> 172.21.15.33:0/869488956 conn(0x7fa23c06cff0 msgr2=0x7fa23c0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 shutdown_connections 2023-10-06T05:16:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.316+0000 7fa241d9d700 1 -- 172.21.15.33:0/869488956 wait complete. 2023-10-06T05:16:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:16:10.335 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.521+0000 7f0b0279d700 1 Processor -- start 2023-10-06T05:16:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.521+0000 7f0b0279d700 1 -- start start 2023-10-06T05:16:10.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b0279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0afc0cb710 0x7f0afc06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b0279d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc0c9cb0 con 0x7f0afc13faa0 2023-10-06T05:16:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b0279d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc0ca270 con 0x7f0afc0c8050 2023-10-06T05:16:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b0279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0afc06b840 con 0x7f0afc0cb710 2023-10-06T05:16:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b0179b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0afc13faa0 0x7f0afc0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51352/0 (socket says 172.21.15.33:51352) 2023-10-06T05:16:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b0179b700 1 -- 172.21.15.33:0/3809518097 learned_addr learned my addr 172.21.15.33:0/3809518097 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0b01f9c700 1 --2- 172.21.15.33:0/3809518097 >> v2:172.21.15.167:3300/0 conn(0x7f0afc0cb710 0x7f0afc06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.523+0000 7f0af3fff700 1 -- 172.21.15.33:0/3809518097 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455188987 0 0) 0x7f0afc0c9cb0 con 0x7f0afc13faa0 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0af3fff700 1 -- 172.21.15.33:0/3809518097 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae8003570 con 0x7f0afc13faa0 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0b01f9c700 1 -- 172.21.15.33:0/3809518097 >> v1:172.21.15.167:6789/0 conn(0x7f0afc0c8050 legacy=0x7f0afc06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0b01f9c700 1 -- 172.21.15.33:0/3809518097 >> v1:172.21.15.33:6789/0 conn(0x7f0afc13faa0 legacy=0x7f0afc0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0b01f9c700 1 -- 172.21.15.33:0/3809518097 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc06bb20 con 0x7f0afc0cb710 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0b01f9c700 1 --2- 172.21.15.33:0/3809518097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc0cb710 0x7f0afc06b1e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009e60 tx=0x7f0af4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97010440ab7ae43a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0af3fff700 1 -- 172.21.15.33:0/3809518097 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400bb60 con 0x7f0afc0cb710 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0af3fff700 1 -- 172.21.15.33:0/3809518097 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af400bcc0 con 0x7f0afc0cb710 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.524+0000 7f0af3fff700 1 -- 172.21.15.33:0/3809518097 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4011670 con 0x7f0afc0cb710 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 -- 172.21.15.33:0/3809518097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc0cb710 msgr2=0x7f0afc06b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3809518097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc0cb710 0x7f0afc06b1e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009e60 tx=0x7f0af4009780 comp rx=0 tx=0).stop 2023-10-06T05:16:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 -- 172.21.15.33:0/3809518097 shutdown_connections 2023-10-06T05:16:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3809518097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc0cb710 0x7f0afc06b1e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 -- 172.21.15.33:0/3809518097 >> 172.21.15.33:0/3809518097 conn(0x7f0afc06d030 msgr2=0x7f0afc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 -- 172.21.15.33:0/3809518097 shutdown_connections 2023-10-06T05:16:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.525+0000 7f0b0279d700 1 -- 172.21.15.33:0/3809518097 wait complete. 2023-10-06T05:16:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.526+0000 7f0b0279d700 1 Processor -- start 2023-10-06T05:16:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 -- start start 2023-10-06T05:16:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0afc0c8050 0x7f0afc13c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 0x7f0afc0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc13faa0 0x7f0afc169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc159b80 con 0x7f0afc0cb710 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc158f70 con 0x7f0afc13faa0 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.527+0000 7f0b0279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc159250 con 0x7f0afc0c8050 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 0x7f0afc0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 0x7f0afc0c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47700/0 (socket says 172.21.15.33:47700) 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 -- 172.21.15.33:0/3918468059 learned_addr learned my addr 172.21.15.33:0/3918468059 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b0179b700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0afc0c8050 0x7f0afc13c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b01f9c700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc13faa0 0x7f0afc169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 -- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0afc0c8050 msgr2=0x7f0afc13c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0afc0c8050 0x7f0afc13c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 -- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc13faa0 msgr2=0x7f0afc169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc13faa0 0x7f0afc169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 -- 172.21.15.33:0/3918468059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af4009a20 con 0x7f0afc0cb710 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.528+0000 7f0b00f9a700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 0x7f0afc0c50f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00c790 tx=0x7f0aec007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.529+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec011590 con 0x7f0afc0cb710 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.529+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec011710 con 0x7f0afc0cb710 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.529+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc159530 con 0x7f0afc0cb710 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.529+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec011ad0 con 0x7f0afc0cb710 2023-10-06T05:16:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.529+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc0c69c0 con 0x7f0afc0cb710 2023-10-06T05:16:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.530+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc159cc0 con 0x7f0afc0cb710 2023-10-06T05:16:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.531+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0aec01b3b0 con 0x7f0afc0cb710 2023-10-06T05:16:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.531+0000 7f0af17fa700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ae0066f30 0x7f0ae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.532+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f0aec083b60 con 0x7f0afc0cb710 2023-10-06T05:16:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.532+0000 7f0b0179b700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ae0066f30 0x7f0ae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.533+0000 7f0b0179b700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ae0066f30 0x7f0ae00693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0af8007fd0 tx=0x7f0af8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.536+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0afc159cc0 con 0x7f0afc0cb710 2023-10-06T05:16:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.783+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0afc159cc0 con 0x7f0afc0cb710 2023-10-06T05:16:10.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.784+0000 7f0af17fa700 1 -- 172.21.15.33:0/3918468059 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f0afc159cc0 con 0x7f0afc0cb710 2023-10-06T05:16:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:10.786 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:15:52.766700+0000","modified":"2023-10-06T05:15:56.296376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/1299702326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1299702326},{"type":"v1","addr":"172.21.15.167:6837","nonce":1299702326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:16:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ae0066f30 msgr2=0x7f0ae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ae0066f30 0x7f0ae00693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0af8007fd0 tx=0x7f0af8009040 comp rx=0 tx=0).stop 2023-10-06T05:16:10.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 msgr2=0x7f0afc0c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 0x7f0afc0c50f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00c790 tx=0x7f0aec007f40 comp rx=0 tx=0).stop 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 shutdown_connections 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ae0066f30 0x7f0ae00693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0afc13faa0 0x7f0afc169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0afc0c8050 0x7f0afc13c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 --2- 172.21.15.33:0/3918468059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0afc0cb710 0x7f0afc0c50f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 >> 172.21.15.33:0/3918468059 conn(0x7f0afc06d030 msgr2=0x7f0afc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.786+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 shutdown_connections 2023-10-06T05:16:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.787+0000 7f0b0279d700 1 -- 172.21.15.33:0/3918468059 wait complete. 2023-10-06T05:16:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2023-10-06T05:16:10.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:10.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.990+0000 7ff2bdb9d700 1 Processor -- start 2023-10-06T05:16:10.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.991+0000 7ff2bdb9d700 1 -- start start 2023-10-06T05:16:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bdb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff2b813faa0 0x7ff2b813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bdb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2b80c7940 0x7ff2b80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bdb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b80c9cb0 con 0x7ff2b80c7940 2023-10-06T05:16:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bdb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b80ca270 con 0x7ff2b813faa0 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b806b840 con 0x7ff2b806aad0 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2b7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2b80c7940 0x7ff2b80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2b7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2b80c7940 0x7ff2b80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47710/0 (socket says 172.21.15.33:47710) 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2b7fff700 1 -- 172.21.15.33:0/3520909227 learned_addr learned my addr 172.21.15.33:0/3520909227 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bcb9b700 1 --2- 172.21.15.33:0/3520909227 >> v2:172.21.15.167:3300/0 conn(0x7ff2b813faa0 0x7ff2b813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.992+0000 7ff2bd39c700 1 --2- 172.21.15.33:0/3520909227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b7fff700 1 -- 172.21.15.33:0/3520909227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 msgr2=0x7ff2b806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b7fff700 1 --2- 172.21.15.33:0/3520909227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b7fff700 1 -- 172.21.15.33:0/3520909227 >> v2:172.21.15.167:3300/0 conn(0x7ff2b813faa0 msgr2=0x7ff2b813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b7fff700 1 --2- 172.21.15.33:0/3520909227 >> v2:172.21.15.167:3300/0 conn(0x7ff2b813faa0 0x7ff2b813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b7fff700 1 -- 172.21.15.33:0/3520909227 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b806bb20 con 0x7ff2b80c7940 2023-10-06T05:16:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b7fff700 1 --2- 172.21.15.33:0/3520909227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80c7db0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff2b806fe50 tx=0x7ff2a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2929e84a5b35f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b77fe700 1 -- 172.21.15.33:0/3520909227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400b640 con 0x7ff2b80c7940 2023-10-06T05:16:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b77fe700 1 -- 172.21.15.33:0/3520909227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a400b7a0 con 0x7ff2b80c7940 2023-10-06T05:16:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.993+0000 7ff2b77fe700 1 -- 172.21.15.33:0/3520909227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400bb40 con 0x7ff2b80c7940 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/3520909227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 msgr2=0x7ff2b80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/3520909227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80c7db0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff2b806fe50 tx=0x7ff2a4009250 comp rx=0 tx=0).stop 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/3520909227 shutdown_connections 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/3520909227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/3520909227 >> v2:172.21.15.167:3300/0 conn(0x7ff2b813faa0 0x7ff2b813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/3520909227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80c7db0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/3520909227 >> 172.21.15.33:0/3520909227 conn(0x7ff2b806d030 msgr2=0x7ff2b806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/3520909227 shutdown_connections 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.994+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/3520909227 wait complete. 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.995+0000 7ff2bdb9d700 1 Processor -- start 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.996+0000 7ff2bdb9d700 1 -- start start 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.996+0000 7ff2bdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b8158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.996+0000 7ff2bdb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2bdb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2b813faa0 0x7ff2b80adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2bdb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80a6730 con 0x7ff2b80c7940 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2bdb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80ae240 con 0x7ff2b813faa0 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2bdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80ae520 con 0x7ff2b806aad0 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47712/0 (socket says 172.21.15.33:47712) 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 -- 172.21.15.33:0/1405741434 learned_addr learned my addr 172.21.15.33:0/1405741434 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2bd39c700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2b813faa0 0x7ff2b80adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2bcb9b700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b8158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 -- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 msgr2=0x7ff2b8158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b8158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 -- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2b813faa0 msgr2=0x7ff2b80adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2b813faa0 0x7ff2b80adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 -- 172.21.15.33:0/1405741434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40094f0 con 0x7ff2b80c7940 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.997+0000 7ff2b7fff700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80ad710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40099b0 tx=0x7ff2a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.998+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400bb20 con 0x7ff2b80c7940 2023-10-06T05:16:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.998+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4011420 con 0x7ff2b80c7940 2023-10-06T05:16:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.998+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b80ae800 con 0x7ff2b80c7940 2023-10-06T05:16:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.998+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40117c0 con 0x7ff2b80c7940 2023-10-06T05:16:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.998+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b8159360 con 0x7ff2b80c7940 2023-10-06T05:16:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:10.999+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b80a68e0 con 0x7ff2b80c7940 2023-10-06T05:16:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.000+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2a4011bb0 con 0x7ff2b80c7940 2023-10-06T05:16:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.001+0000 7ff2b4ff9700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff29c066f60 0x7ff29c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.001+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff2a40861f0 con 0x7ff2b80c7940 2023-10-06T05:16:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.001+0000 7ff2bcb9b700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff29c066f60 0x7ff29c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.002+0000 7ff2bcb9b700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff29c066f60 0x7ff29c069410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009ca0 tx=0x7ff2ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.005+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2b80a68e0 con 0x7ff2b80c7940 2023-10-06T05:16:11.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.209+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2b804c380 con 0x7ff2b80c7940 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.210+0000 7ff2b4ff9700 1 -- 172.21.15.33:0/1405741434 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16961 (secure 0 0 0) 0x7ff2a4011e60 con 0x7ff2b80c7940 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":50,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:07.670746+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:15:50.385790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:15:51.751034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.2d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff29c066f60 msgr2=0x7ff29c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff29c066f60 0x7ff29c069410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009ca0 tx=0x7ff2ac008040 comp rx=0 tx=0).stop 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 msgr2=0x7ff2b80ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80ad710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40099b0 tx=0x7ff2a4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 shutdown_connections 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff29c066f60 0x7ff29c069410 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2b813faa0 0x7ff2b80adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2b806aad0 0x7ff2b8158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 --2- 172.21.15.33:0/1405741434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b80c7940 0x7ff2b80ad710 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 >> 172.21.15.33:0/1405741434 conn(0x7ff2b806d030 msgr2=0x7ff2b80ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 shutdown_connections 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.212+0000 7ff2bdb9d700 1 -- 172.21.15.33:0/1405741434 wait complete. 2023-10-06T05:16:11.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:16:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.526+0000 7f7b0599d700 1 Processor -- start 2023-10-06T05:16:11.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.526+0000 7f7b0599d700 1 -- start start 2023-10-06T05:16:11.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0599d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b00156880 0x7f7b000c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0599d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7b00159e10 0x7f7b000c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0599d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b00157800 con 0x7f7b00156c50 2023-10-06T05:16:11.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0599d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000cc290 con 0x7f7b00156880 2023-10-06T05:16:11.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0599d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0006bd10 con 0x7f7b00159e10 2023-10-06T05:16:11.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0499b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b00156880 0x7f7b000c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0519c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7b00156c50 0x7f7b001570f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51368/0 (socket says 172.21.15.33:51368) 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.527+0000 7f7b0519c700 1 -- 172.21.15.33:0/43198911 learned_addr learned my addr 172.21.15.33:0/43198911 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7afffff700 1 --2- 172.21.15.33:0/43198911 >> v2:172.21.15.167:3300/0 conn(0x7f7b00159e10 0x7f7b000c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7b0499b700 1 -- 172.21.15.33:0/43198911 >> v2:172.21.15.167:3300/0 conn(0x7f7b00159e10 msgr2=0x7f7b000c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7b0499b700 1 --2- 172.21.15.33:0/43198911 >> v2:172.21.15.167:3300/0 conn(0x7f7b00159e10 0x7f7b000c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7b0499b700 1 -- 172.21.15.33:0/43198911 >> v1:172.21.15.33:6789/0 conn(0x7f7b00156c50 legacy=0x7f7b001570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7b0499b700 1 -- 172.21.15.33:0/43198911 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0006a820 con 0x7f7b00156880 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7aff7fe700 1 -- 172.21.15.33:0/43198911 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035002555 0 0) 0x7f7b00157800 con 0x7f7b00156c50 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7b0499b700 1 --2- 172.21.15.33:0/43198911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00156880 0x7f7b000c7980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7b0006a390 tx=0x7f7af40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d425e816b8bf599 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7aff7fe700 1 -- 172.21.15.33:0/43198911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011630 con 0x7f7b00156880 2023-10-06T05:16:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.528+0000 7f7aff7fe700 1 -- 172.21.15.33:0/43198911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4011790 con 0x7f7b00156880 2023-10-06T05:16:11.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7aff7fe700 1 -- 172.21.15.33:0/43198911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011b30 con 0x7f7b00156880 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 -- 172.21.15.33:0/43198911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00156880 msgr2=0x7f7b000c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 --2- 172.21.15.33:0/43198911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00156880 0x7f7b000c7980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7b0006a390 tx=0x7f7af40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 -- 172.21.15.33:0/43198911 shutdown_connections 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 --2- 172.21.15.33:0/43198911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00156880 0x7f7b000c7980 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 --2- 172.21.15.33:0/43198911 >> v2:172.21.15.167:3300/0 conn(0x7f7b00159e10 0x7f7b000c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 -- 172.21.15.33:0/43198911 >> 172.21.15.33:0/43198911 conn(0x7f7b0006c560 msgr2=0x7f7b0006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 -- 172.21.15.33:0/43198911 shutdown_connections 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.529+0000 7f7b0599d700 1 -- 172.21.15.33:0/43198911 wait complete. 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.531+0000 7f7b0599d700 1 Processor -- start 2023-10-06T05:16:11.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.531+0000 7f7b0599d700 1 -- start start 2023-10-06T05:16:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0599d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b00156880 0x7f7b0015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b00156c50 0x7f7b00158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0599d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 0x7f7b001592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0599d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000afff0 con 0x7f7b00159e10 2023-10-06T05:16:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0599d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000b0c80 con 0x7f7b00156880 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000b0f60 con 0x7f7b00156c50 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 0x7f7b001592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0499b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b00156880 0x7f7b0015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7afffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b00156c50 0x7f7b00158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 0x7f7b001592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47730/0 (socket says 172.21.15.33:47730) 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 -- 172.21.15.33:0/3287792381 learned_addr learned my addr 172.21.15.33:0/3287792381 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 -- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b00156c50 msgr2=0x7f7b00158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b00156c50 0x7f7b00158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 -- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b00156880 msgr2=0x7f7b0015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b00156880 0x7f7b0015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.532+0000 7f7b0519c700 1 -- 172.21.15.33:0/3287792381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009420 con 0x7f7b00159e10 2023-10-06T05:16:11.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.533+0000 7f7b0519c700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 0x7f7b001592d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7af800cc90 tx=0x7f7af8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.533+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800a8b0 con 0x7f7b00159e10 2023-10-06T05:16:11.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.533+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800aa10 con 0x7f7b00159e10 2023-10-06T05:16:11.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.533+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800add0 con 0x7f7b00159e10 2023-10-06T05:16:11.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.533+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b000b1240 con 0x7f7b00159e10 2023-10-06T05:16:11.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.533+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0015ccf0 con 0x7f7b00159e10 2023-10-06T05:16:11.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.534+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0004c380 con 0x7f7b00159e10 2023-10-06T05:16:11.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.535+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7af8007300 con 0x7f7b00159e10 2023-10-06T05:16:11.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.536+0000 7f7afcff9700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae4066fb0 0x7f7ae4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.536+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f7af80849f0 con 0x7f7b00159e10 2023-10-06T05:16:11.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.536+0000 7f7b0499b700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae4066fb0 0x7f7ae4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.537+0000 7f7b0499b700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae4066fb0 0x7f7ae4069460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7af40093f0 tx=0x7f7af4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.540+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7af8051aa0 con 0x7f7b00159e10 2023-10-06T05:16:11.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.786+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7b0015a2f0 con 0x7f7b00159e10 2023-10-06T05:16:12.709 INFO:tasks.ceph.mds.b.smithi167.stderr: -3595> 2023-10-06T05:15:57.246+0000 7f3c49bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:16:12.709 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:16:11.787+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:16:12.709 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:16:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.792+0000 7f7afcff9700 1 -- 172.21.15.33:0/3287792381 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7af80550c0 con 0x7f7b00159e10 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae4066fb0 msgr2=0x7f7ae4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae4066fb0 0x7f7ae4069460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7af40093f0 tx=0x7f7af4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 msgr2=0x7f7b001592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 0x7f7b001592d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7af800cc90 tx=0x7f7af8001a80 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 shutdown_connections 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae4066fb0 0x7f7ae4069460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b00159e10 0x7f7b001592d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b00156880 0x7f7b0015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 --2- 172.21.15.33:0/3287792381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b00156c50 0x7f7b00158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 >> 172.21.15.33:0/3287792381 conn(0x7f7b0006c560 msgr2=0x7f7b0014eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 shutdown_connections 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:11.794+0000 7f7b0599d700 1 -- 172.21.15.33:0/3287792381 wait complete. 2023-10-06T05:16:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:16:12.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:16:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.913+0000 7fc5ad19d700 1 Processor -- start 2023-10-06T05:16:13.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.913+0000 7fc5ad19d700 1 -- start start 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5ad19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 0x7fc5a80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5ad19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80ad430 con 0x7fc5a8156370 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5ad19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80afe20 con 0x7fc5a80b01f0 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5ad19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a806bc70 con 0x7fc5a80cc7e0 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5a7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc5a8156370 0x7fc5a80c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51378/0 (socket says 172.21.15.33:51378) 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5a7fff700 1 -- 172.21.15.33:0/2056975061 learned_addr learned my addr 172.21.15.33:0/2056975061 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5a77fe700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 0x7fc5a80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.915+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179662044 0 0) 0x7fc5a80ad430 con 0x7fc5a8156370 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc590003570 con 0x7fc5a8156370 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227020979 0 0) 0x7fc5a80afe20 con 0x7fc5a80b01f0 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a80ad430 con 0x7fc5a80b01f0 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034105825 0 0) 0x7fc590003570 con 0x7fc5a8156370 2023-10-06T05:16:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 msgr2=0x7fc5a80af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 0x7fc5a80af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 >> v1:172.21.15.167:6789/0 conn(0x7fc5a80b01f0 legacy=0x7fc5a80ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80cccc0 con 0x7fc5a8156370 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc598001470 con 0x7fc5a8156370 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 >> v1:172.21.15.33:6789/0 conn(0x7fc5a8156370 legacy=0x7fc5a80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 0x7fc590003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5900041d0 0x7fc590004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc590004d50 0x7fc5900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a806bc70 con 0x7fc590004d50 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000de00 con 0x7fc5900041d0 2023-10-06T05:16:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000dfc0 con 0x7fc5a80cc7e0 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a7fff700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 0x7fc590003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5a77fe700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5900041d0 0x7fc590004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.916+0000 7fc5ac99c700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc590004d50 0x7fc5900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5ac99c700 1 -- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 msgr2=0x7fc590003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5ac99c700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a80cc7e0 0x7fc590003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5ac99c700 1 -- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5900041d0 msgr2=0x7fc590004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5ac99c700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5900041d0 0x7fc590004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5ac99c700 1 -- 172.21.15.33:0/2056975061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590003570 con 0x7fc590004d50 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5ac99c700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc590004d50 0x7fc5900051c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0002120 tx=0x7fc5a00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13b2f5a845ac93d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0001ab0 con 0x7fc590004d50 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000c980 con 0x7fc590004d50 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.917+0000 7fc5a6ffd700 1 -- 172.21.15.33:0/2056975061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000cd20 con 0x7fc590004d50 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ad19d700 1 -- 172.21.15.33:0/2056975061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc590004d50 msgr2=0x7fc5900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/2056975061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc590004d50 0x7fc5900051c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0002120 tx=0x7fc5a00017a0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ac99c700 1 -- 172.21.15.33:0/2056975061 reap_dead start 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ad19d700 1 -- 172.21.15.33:0/2056975061 shutdown_connections 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ad19d700 1 -- 172.21.15.33:0/2056975061 >> 172.21.15.33:0/2056975061 conn(0x7fc5a806d030 msgr2=0x7fc5a80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ad19d700 1 -- 172.21.15.33:0/2056975061 shutdown_connections 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.918+0000 7fc5ad19d700 1 -- 172.21.15.33:0/2056975061 wait complete. 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.919+0000 7fc5ad19d700 1 Processor -- start 2023-10-06T05:16:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.920+0000 7fc5ad19d700 1 -- start start 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.920+0000 7fc5ad19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8146eb0 0x7fc5a815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.920+0000 7fc5ad19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 0x7fc5a815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.920+0000 7fc5ad19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8156fa0 0x7fc5a8157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5ad19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a815eb10 con 0x7fc5a8147280 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5ad19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8156b10 con 0x7fc5a8156fa0 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5ad19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81567e0 con 0x7fc5a8146eb0 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 0x7fc5a815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 0x7fc5a815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47742/0 (socket says 172.21.15.33:47742) 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 -- 172.21.15.33:0/960434497 learned_addr learned my addr 172.21.15.33:0/960434497 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a7fff700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8146eb0 0x7fc5a815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5ac99c700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8156fa0 0x7fc5a8157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 -- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8146eb0 msgr2=0x7fc5a815e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8146eb0 0x7fc5a815e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 -- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8156fa0 msgr2=0x7fc5a8157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8156fa0 0x7fc5a8157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 -- 172.21.15.33:0/960434497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a815ed50 con 0x7fc5a8147280 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.921+0000 7fc5a77fe700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 0x7fc5a815e5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc59c011b40 tx=0x7fc59c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.922+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00f9b0 con 0x7fc5a8147280 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.922+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00fb10 con 0x7fc5a8147280 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.922+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a815f030 con 0x7fc5a8147280 2023-10-06T05:16:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.922+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00fe90 con 0x7fc5a8147280 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.922+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a8157960 con 0x7fc5a8147280 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.923+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8156cc0 con 0x7fc5a8147280 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.924+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc59c014650 con 0x7fc5a8147280 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.925+0000 7fc58ffff700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc594066f30 0x7fc5940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.925+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fc59c018030 con 0x7fc5a8147280 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.925+0000 7fc5a7fff700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc594066f30 0x7fc5940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.926+0000 7fc5a7fff700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc594066f30 0x7fc5940693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc598001200 tx=0x7fc598009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:12.929+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5a8156cc0 con 0x7fc5a8147280 2023-10-06T05:16:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.177+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc5a8156c80 con 0x7fc5a8147280 2023-10-06T05:16:13.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.799+0000 7fc58ffff700 1 -- 172.21.15.33:0/960434497 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc59c0521e0 con 0x7fc5a8147280 2023-10-06T05:16:13.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc594066f30 msgr2=0x7fc5940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc594066f30 0x7fc5940693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc598001200 tx=0x7fc598009040 comp rx=0 tx=0).stop 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 msgr2=0x7fc5a815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 0x7fc5a815e5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc59c011b40 tx=0x7fc59c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 shutdown_connections 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc594066f30 0x7fc5940693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8156fa0 0x7fc5a8157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a8146eb0 0x7fc5a815e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 --2- 172.21.15.33:0/960434497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a8147280 0x7fc5a815e5d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 >> 172.21.15.33:0/960434497 conn(0x7fc5a806d030 msgr2=0x7fc5a80ac560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 shutdown_connections 2023-10-06T05:16:13.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:13.802+0000 7fc5ad19d700 1 -- 172.21.15.33:0/960434497 wait complete. 2023-10-06T05:16:13.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:16:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.017+0000 7f1d50f9d700 1 Processor -- start 2023-10-06T05:16:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.017+0000 7f1d50f9d700 1 -- start start 2023-10-06T05:16:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.018+0000 7f1d50f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1d4c157320 0x7f1d4c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.018+0000 7f1d50f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c0b0840 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.018+0000 7f1d50f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c1585c0 con 0x7f1d4c1576f0 2023-10-06T05:16:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.018+0000 7f1d50f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b1ff0 con 0x7f1d4c157320 2023-10-06T05:16:14.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1d4c0b1ae0 0x7f1d4c158a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51380/0 (socket says 172.21.15.33:51380) 2023-10-06T05:16:14.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4bfff700 1 -- 172.21.15.33:0/1909024429 learned_addr learned my addr 172.21.15.33:0/1909024429 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:14.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4b7fe700 1 --2- 172.21.15.33:0/1909024429 >> v2:172.21.15.167:3300/0 conn(0x7f1d4c157320 0x7f1d4c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441052094 0 0) 0x7f1d4c0b0840 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d34003570 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774382090 0 0) 0x7f1d4c1585c0 con 0x7f1d4c1576f0 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d4c0b0840 con 0x7f1d4c1576f0 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/1909024429 >> v1:172.21.15.167:6789/0 conn(0x7f1d4c1576f0 legacy=0x7f1d4c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/1909024429 >> v1:172.21.15.33:6789/0 conn(0x7f1d4c0b1ae0 legacy=0x7f1d4c158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.019+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/1909024429 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c157c90 con 0x7f1d4c157320 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.020+0000 7f1d4b7fe700 1 --2- 172.21.15.33:0/1909024429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 0x7f1d4c0adf10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0098e0 tx=0x7f1d3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb8cdae34ae3ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.020+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c011630 con 0x7f1d4c157320 2023-10-06T05:16:14.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.020+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c011790 con 0x7f1d4c157320 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.020+0000 7f1d4a7fc700 1 -- 172.21.15.33:0/1909024429 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c011b30 con 0x7f1d4c157320 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.020+0000 7f1d50f9d700 1 -- 172.21.15.33:0/1909024429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 msgr2=0x7f1d4c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.020+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/1909024429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 0x7f1d4c0adf10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0098e0 tx=0x7f1d3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.021+0000 7f1d50f9d700 1 -- 172.21.15.33:0/1909024429 shutdown_connections 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.021+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/1909024429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 0x7f1d4c0adf10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.021+0000 7f1d50f9d700 1 -- 172.21.15.33:0/1909024429 >> 172.21.15.33:0/1909024429 conn(0x7f1d4c068090 msgr2=0x7f1d4c06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.021+0000 7f1d50f9d700 1 -- 172.21.15.33:0/1909024429 shutdown_connections 2023-10-06T05:16:14.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.021+0000 7f1d50f9d700 1 -- 172.21.15.33:0/1909024429 wait complete. 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.022+0000 7f1d50f9d700 1 Processor -- start 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.022+0000 7f1d50f9d700 1 -- start start 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d50f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 0x7f1d4c1680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d50f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 0x7f1d4c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d50f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d4c1576f0 0x7f1d4c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d50f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0a79a0 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d50f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c14ff80 con 0x7f1d4c157320 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d50f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0a7510 con 0x7f1d4c1576f0 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d4b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 0x7f1d4c1680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d4b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 0x7f1d4c1680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47758/0 (socket says 172.21.15.33:47758) 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.023+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/2742489530 learned_addr learned my addr 172.21.15.33:0/2742489530 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d4c1576f0 msgr2=0x7f1d4c0a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d4b7fe700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d4c1576f0 0x7f1d4c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 msgr2=0x7f1d4c0a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d4b7fe700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 0x7f1d4c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d4b7fe700 1 -- 172.21.15.33:0/2742489530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c009420 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d4b7fe700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 0x7f1d4c1680b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009ed0 tx=0x7f1d3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c01f070 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c000c90 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c025420 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c0a6900 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.024+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c0a6ec0 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.025+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c063ff0 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.026+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d3c023070 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.027+0000 7f1d337fe700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d38066f30 0x7f1d380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.027+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f1d3c085c70 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.028+0000 7f1d4affd700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d38066f30 0x7f1d380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.028+0000 7f1d4affd700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d38066f30 0x7f1d380693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0c9990 tx=0x7f1d400079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.032+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d3c052c50 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.234+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d4c06bcf0 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.818+0000 7f1d337fe700 1 -- 172.21.15.33:0/2742489530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1d3c02a020 con 0x7f1d4c0b1ae0 2023-10-06T05:16:14.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d38066f30 msgr2=0x7f1d380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d38066f30 0x7f1d380693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0c9990 tx=0x7f1d400079d0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 msgr2=0x7f1d4c1680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 0x7f1d4c1680b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009ed0 tx=0x7f1d3c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 shutdown_connections 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d38066f30 0x7f1d380693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d4c1576f0 0x7f1d4c0a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d4c0b1ae0 0x7f1d4c1680b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.822+0000 7f1d50f9d700 1 --2- 172.21.15.33:0/2742489530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d4c157320 0x7f1d4c0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.823+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 >> 172.21.15.33:0/2742489530 conn(0x7f1d4c068090 msgr2=0x7f1d4c151760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.824+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 shutdown_connections 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:14.824+0000 7f1d50f9d700 1 -- 172.21.15.33:0/2742489530 wait complete. 2023-10-06T05:16:14.955 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:16:14.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:16:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.154+0000 7f4d0299d700 1 Processor -- start 2023-10-06T05:16:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.154+0000 7f4d0299d700 1 -- start start 2023-10-06T05:16:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.155+0000 7f4d0299d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4cfc0ad010 0x7f4cfc06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.155+0000 7f4d0299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc15acd0 0x7f4cfc158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.155+0000 7f4d0299d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc06b1f0 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.155+0000 7f4d0299d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc06a3f0 con 0x7f4cfc0ad010 2023-10-06T05:16:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.155+0000 7f4d0299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc06a5e0 con 0x7f4cfc15acd0 2023-10-06T05:16:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4cfc0ad010 0x7f4cfc06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4cfc0ad010 0x7f4cfc06ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47774/0 (socket says 172.21.15.33:47774) 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 -- 172.21.15.33:0/2768763096 learned_addr learned my addr 172.21.15.33:0/2768763096 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0219c700 1 --2- 172.21.15.33:0/2768763096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc15acd0 0x7f4cfc158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 -- 172.21.15.33:0/2768763096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc15acd0 msgr2=0x7f4cfc158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 --2- 172.21.15.33:0/2768763096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc15acd0 0x7f4cfc158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 -- 172.21.15.33:0/2768763096 >> v1:172.21.15.33:6789/0 conn(0x7f4cfc0ad3e0 legacy=0x7f4cfc15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 -- 172.21.15.33:0/2768763096 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc06a8c0 con 0x7f4cfc0ad010 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d00999700 1 -- 172.21.15.33:0/2768763096 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123381990 0 0) 0x7f4cfc06b1f0 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.156+0000 7f4d0199b700 1 --2- 172.21.15.33:0/2768763096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad010 0x7f4cfc06ea00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80099b0 tx=0x7f4cf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6aac4fd302d7257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d00999700 1 -- 172.21.15.33:0/2768763096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800b8f0 con 0x7f4cfc0ad010 2023-10-06T05:16:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d00999700 1 -- 172.21.15.33:0/2768763096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf800ba50 con 0x7f4cfc0ad010 2023-10-06T05:16:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d00999700 1 -- 172.21.15.33:0/2768763096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800bdf0 con 0x7f4cfc0ad010 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 -- 172.21.15.33:0/2768763096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad010 msgr2=0x7f4cfc06ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 --2- 172.21.15.33:0/2768763096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad010 0x7f4cfc06ea00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80099b0 tx=0x7f4cf8009250 comp rx=0 tx=0).stop 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 -- 172.21.15.33:0/2768763096 shutdown_connections 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 --2- 172.21.15.33:0/2768763096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad010 0x7f4cfc06ea00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 --2- 172.21.15.33:0/2768763096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc15acd0 0x7f4cfc158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 -- 172.21.15.33:0/2768763096 >> 172.21.15.33:0/2768763096 conn(0x7f4cfc068090 msgr2=0x7f4cfc06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 -- 172.21.15.33:0/2768763096 shutdown_connections 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.157+0000 7f4d0299d700 1 -- 172.21.15.33:0/2768763096 wait complete. 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.159+0000 7f4d0299d700 1 Processor -- start 2023-10-06T05:16:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.159+0000 7f4d0299d700 1 -- start start 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc0ad010 0x7f4cfc151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 0x7f4cfc15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cfc15acd0 0x7f4cfc15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0cd030 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0aa080 con 0x7f4cfc15acd0 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0ccd50 con 0x7f4cfc0ad010 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 0x7f4cfc15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 0x7f4cfc15d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47776/0 (socket says 172.21.15.33:47776) 2023-10-06T05:16:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0119a700 1 -- 172.21.15.33:0/1934276935 learned_addr learned my addr 172.21.15.33:0/1934276935 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0199b700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc0ad010 0x7f4cfc151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.160+0000 7f4d0219c700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cfc15acd0 0x7f4cfc15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0119a700 1 -- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc0ad010 msgr2=0x7f4cfc151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0119a700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc0ad010 0x7f4cfc151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0119a700 1 -- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cfc15acd0 msgr2=0x7f4cfc15d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0119a700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cfc15acd0 0x7f4cfc15d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0119a700 1 -- 172.21.15.33:0/1934276935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80094f0 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0119a700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 0x7f4cfc15d260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00c7e0 tx=0x7f4cec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0105c0 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec010740 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc0a9a30 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec010b00 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.161+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc152360 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.162+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc0a9890 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.163+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cec01b3b0 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.164+0000 7f4cf1ffb700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ce0066f30 0x7f4ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.164+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f4cec083bc0 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.164+0000 7f4d0199b700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ce0066f30 0x7f4ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.165+0000 7f4d0199b700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ce0066f30 0x7f4ce00693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80099b0 tx=0x7f4cf8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.168+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cfc0a9890 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.370+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4cfc0a9890 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.817+0000 7f4cf1ffb700 1 -- 172.21.15.33:0/1934276935 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4cfc0a9890 con 0x7f4cfc0ad3e0 2023-10-06T05:16:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.823+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ce0066f30 msgr2=0x7f4ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.823+0000 7f4d0299d700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ce0066f30 0x7f4ce00693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80099b0 tx=0x7f4cf8017000 comp rx=0 tx=0).stop 2023-10-06T05:16:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.823+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 msgr2=0x7f4cfc15d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.823+0000 7f4d0299d700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 0x7f4cfc15d260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00c7e0 tx=0x7f4cec007df0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.824+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 shutdown_connections 2023-10-06T05:16:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.824+0000 7f4d0299d700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ce0066f30 0x7f4ce00693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.824+0000 7f4d0299d700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cfc15acd0 0x7f4cfc15d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.824+0000 7f4d0299d700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cfc0ad010 0x7f4cfc151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.824+0000 7f4d0299d700 1 --2- 172.21.15.33:0/1934276935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cfc0ad3e0 0x7f4cfc15d260 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.824+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 >> 172.21.15.33:0/1934276935 conn(0x7f4cfc068090 msgr2=0x7f4cfc0b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.825+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 shutdown_connections 2023-10-06T05:16:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:15.825+0000 7f4d0299d700 1 -- 172.21.15.33:0/1934276935 wait complete. 2023-10-06T05:16:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:16:15.850 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:16:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.047+0000 7f8fe359e700 1 Processor -- start 2023-10-06T05:16:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.047+0000 7f8fe359e700 1 -- start start 2023-10-06T05:16:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.048+0000 7f8fe359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8fe40cd5e0 0x7f8fe40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.048+0000 7f8fe359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8fe415a650 0x7f8fe415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.048+0000 7f8fe359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe415d660 con 0x7f8fe40af540 2023-10-06T05:16:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.048+0000 7f8fe359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe40ce890 con 0x7f8fe40cd5e0 2023-10-06T05:16:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.048+0000 7f8fe359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe415aa60 con 0x7f8fe415a650 2023-10-06T05:16:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8fe40af540 0x7f8fe40cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51388/0 (socket says 172.21.15.33:51388) 2023-10-06T05:16:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe259c700 1 -- 172.21.15.33:0/1700654356 learned_addr learned my addr 172.21.15.33:0/1700654356 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe1d9b700 1 --2- 172.21.15.33:0/1700654356 >> v2:172.21.15.33:3300/0 conn(0x7f8fe40cd5e0 0x7f8fe40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe2d9d700 1 --2- 172.21.15.33:0/1700654356 >> v2:172.21.15.167:3300/0 conn(0x7f8fe415a650 0x7f8fe415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe159a700 1 -- 172.21.15.33:0/1700654356 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172829981 0 0) 0x7f8fe415d660 con 0x7f8fe40af540 2023-10-06T05:16:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe1d9b700 1 -- 172.21.15.33:0/1700654356 >> v2:172.21.15.167:3300/0 conn(0x7f8fe415a650 msgr2=0x7f8fe415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe1d9b700 1 --2- 172.21.15.33:0/1700654356 >> v2:172.21.15.167:3300/0 conn(0x7f8fe415a650 0x7f8fe415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe1d9b700 1 -- 172.21.15.33:0/1700654356 >> v1:172.21.15.33:6789/0 conn(0x7f8fe40af540 legacy=0x7f8fe40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe1d9b700 1 -- 172.21.15.33:0/1700654356 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe415d890 con 0x7f8fe40cd5e0 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.049+0000 7f8fe1d9b700 1 --2- 172.21.15.33:0/1700654356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40af910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40caa90 tx=0x7f8fd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7168907f4a7cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe159a700 1 -- 172.21.15.33:0/1700654356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800b6b0 con 0x7f8fe40cd5e0 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe159a700 1 -- 172.21.15.33:0/1700654356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd800b810 con 0x7f8fe40cd5e0 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe159a700 1 -- 172.21.15.33:0/1700654356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800bbb0 con 0x7f8fe40cd5e0 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 -- 172.21.15.33:0/1700654356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe40cd5e0 msgr2=0x7f8fe40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 --2- 172.21.15.33:0/1700654356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40af910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40caa90 tx=0x7f8fd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 -- 172.21.15.33:0/1700654356 shutdown_connections 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 --2- 172.21.15.33:0/1700654356 >> v2:172.21.15.167:3300/0 conn(0x7f8fe415a650 0x7f8fe415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 --2- 172.21.15.33:0/1700654356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40af910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 -- 172.21.15.33:0/1700654356 >> 172.21.15.33:0/1700654356 conn(0x7f8fe406d9b0 msgr2=0x7f8fe406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 -- 172.21.15.33:0/1700654356 shutdown_connections 2023-10-06T05:16:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.050+0000 7f8fe359e700 1 -- 172.21.15.33:0/1700654356 wait complete. 2023-10-06T05:16:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.052+0000 7f8fe359e700 1 Processor -- start 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.052+0000 7f8fe359e700 1 -- start start 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe40af540 0x7f8fe40c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 0x7f8fe4157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40b44d0 con 0x7f8fe415a650 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40b47b0 con 0x7f8fe40cd5e0 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40b4a90 con 0x7f8fe40af540 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 0x7f8fe4157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 0x7f8fe4157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47794/0 (socket says 172.21.15.33:47794) 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe2d9d700 1 -- 172.21.15.33:0/635887513 learned_addr learned my addr 172.21.15.33:0/635887513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe1d9b700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe259c700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe40af540 0x7f8fe40c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.053+0000 7f8fe2d9d700 1 -- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe40af540 msgr2=0x7f8fe40c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe2d9d700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe40af540 0x7f8fe40c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe2d9d700 1 -- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe40cd5e0 msgr2=0x7f8fe40b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe2d9d700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe2d9d700 1 -- 172.21.15.33:0/635887513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009420 con 0x7f8fe415a650 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe1d9b700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe2d9d700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 0x7f8fe4157300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009fa0 tx=0x7f8fdc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00ed20 con 0x7f8fe415a650 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc00ee80 con 0x7f8fe415a650 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc014670 con 0x7f8fe415a650 2023-10-06T05:16:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe40c9580 con 0x7f8fe415a650 2023-10-06T05:16:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.054+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe4157840 con 0x7f8fe415a650 2023-10-06T05:16:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.055+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe4157a30 con 0x7f8fe415a650 2023-10-06T05:16:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.056+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fdc014a20 con 0x7f8fe415a650 2023-10-06T05:16:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.058+0000 7f8fd27fc700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8fc4066f30 0x7f8fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.058+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f8fdc084b90 con 0x7f8fe415a650 2023-10-06T05:16:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.059+0000 7f8fe259c700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8fc4066f30 0x7f8fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.059+0000 7f8fe259c700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8fc4066f30 0x7f8fc40693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c8f0 tx=0x7f8fd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.062+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fdc051f00 con 0x7f8fe415a650 2023-10-06T05:16:16.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.265+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8fe40c93e0 con 0x7f8fe415a650 2023-10-06T05:16:16.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.269+0000 7f8fd27fc700 1 -- 172.21.15.33:0/635887513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8fe40c93e0 con 0x7f8fe415a650 2023-10-06T05:16:16.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8fc4066f30 msgr2=0x7f8fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8fc4066f30 0x7f8fc40693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c8f0 tx=0x7f8fd4009040 comp rx=0 tx=0).stop 2023-10-06T05:16:16.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 msgr2=0x7f8fe4157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 0x7f8fe4157300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009fa0 tx=0x7f8fdc009900 comp rx=0 tx=0).stop 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 shutdown_connections 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8fc4066f30 0x7f8fc40693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fe415a650 0x7f8fe4157300 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe40af540 0x7f8fe40c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.271+0000 7f8fe359e700 1 --2- 172.21.15.33:0/635887513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe40cd5e0 0x7f8fe40b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.272+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 >> 172.21.15.33:0/635887513 conn(0x7f8fe406d9b0 msgr2=0x7f8fe4158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.272+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 shutdown_connections 2023-10-06T05:16:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.272+0000 7f8fe359e700 1 -- 172.21.15.33:0/635887513 wait complete. 2023-10-06T05:16:16.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic' 2023-10-06T05:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.475+0000 7f1ad5f9d700 1 Processor -- start 2023-10-06T05:16:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.475+0000 7f1ad5f9d700 1 -- start start 2023-10-06T05:16:16.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.476+0000 7f1ad5f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ad01573c0 0x7f1ad015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.476+0000 7f1ad5f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ad0157790 0x7f1ad00af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.476+0000 7f1ad5f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad015a8b0 con 0x7f1ad00af600 2023-10-06T05:16:16.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.476+0000 7f1ad5f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00afb40 con 0x7f1ad01573c0 2023-10-06T05:16:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.476+0000 7f1ad5f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad015a260 con 0x7f1ad0157790 2023-10-06T05:16:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ad01573c0 0x7f1ad015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ad01573c0 0x7f1ad015ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47804/0 (socket says 172.21.15.33:47804) 2023-10-06T05:16:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/3231063184 learned_addr learned my addr 172.21.15.33:0/3231063184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1acffff700 1 --2- 172.21.15.33:0/3231063184 >> v2:172.21.15.167:3300/0 conn(0x7f1ad0157790 0x7f1ad00af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/3231063184 >> v2:172.21.15.167:3300/0 conn(0x7f1ad0157790 msgr2=0x7f1ad00af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 --2- 172.21.15.33:0/3231063184 >> v2:172.21.15.167:3300/0 conn(0x7f1ad0157790 0x7f1ad00af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/3231063184 >> v1:172.21.15.167:6789/0 conn(0x7f1ad00af600 legacy=0x7f1ad006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/3231063184 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad015a540 con 0x7f1ad01573c0 2023-10-06T05:16:16.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.477+0000 7f1ad4f9b700 1 --2- 172.21.15.33:0/3231063184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad01573c0 0x7f1ad015ad40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00719c0 tx=0x7f1ac4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201f2e1dbd9d1146 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1acf7fe700 1 -- 172.21.15.33:0/3231063184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4005c10 con 0x7f1ad01573c0 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1acf7fe700 1 -- 172.21.15.33:0/3231063184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac4005d70 con 0x7f1ad01573c0 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1acf7fe700 1 -- 172.21.15.33:0/3231063184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400b840 con 0x7f1ad01573c0 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/3231063184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad01573c0 msgr2=0x7f1ad015ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/3231063184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad01573c0 0x7f1ad015ad40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00719c0 tx=0x7f1ac4009220 comp rx=0 tx=0).stop 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/3231063184 shutdown_connections 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/3231063184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad01573c0 0x7f1ad015ad40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/3231063184 >> v2:172.21.15.167:3300/0 conn(0x7f1ad0157790 0x7f1ad00af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/3231063184 >> 172.21.15.33:0/3231063184 conn(0x7f1ad006e430 msgr2=0x7f1ad00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/3231063184 shutdown_connections 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.478+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/3231063184 wait complete. 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.480+0000 7f1ad5f9d700 1 Processor -- start 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.480+0000 7f1ad5f9d700 1 -- start start 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 0x7f1ad00cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad01573c0 0x7f1ad00c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ad0157790 0x7f1ad00ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ca790 con 0x7f1ad00af600 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ccf40 con 0x7f1ad01573c0 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00cd200 con 0x7f1ad0157790 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 0x7f1ad00cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 0x7f1ad00cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47806/0 (socket says 172.21.15.33:47806) 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/1814087021 learned_addr learned my addr 172.21.15.33:0/1814087021 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ad0157790 msgr2=0x7f1ad00ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ad0157790 0x7f1ad00ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad01573c0 msgr2=0x7f1ad00c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad01573c0 0x7f1ad00c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.481+0000 7f1ad4f9b700 1 -- 172.21.15.33:0/1814087021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4009480 con 0x7f1ad00af600 2023-10-06T05:16:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.482+0000 7f1ad4f9b700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 0x7f1ad00cdfa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009260 tx=0x7f1ac4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.482+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4011ae0 con 0x7f1ad00af600 2023-10-06T05:16:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.482+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac4011c60 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.482+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac401d430 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.482+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00cd4e0 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.482+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad00cdaa0 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.483+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad00cde10 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.484+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ac401d7a0 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.485+0000 7f1accff9700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ab4066f30 0x7f1ab40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.485+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1ac40860a0 con 0x7f1ad00af600 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.485+0000 7f1acffff700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ab4066f30 0x7f1ab40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.486+0000 7f1acffff700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ab4066f30 0x7f1ab40693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009a40 tx=0x7f1abc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.489+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ac4053410 con 0x7f1ad00af600 2023-10-06T05:16:16.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.769+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]} v 0) v1 -- 0x7f1ad0064510 con 0x7f1ad00af600 2023-10-06T05:16:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.831+0000 7f1accff9700 1 -- 172.21.15.33:0/1814087021 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_basic"]}]=0 v132) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1ac4056a30 con 0x7f1ad00af600 2023-10-06T05:16:16.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ab4066f30 msgr2=0x7f1ab40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ab4066f30 0x7f1ab40693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009a40 tx=0x7f1abc008040 comp rx=0 tx=0).stop 2023-10-06T05:16:16.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 msgr2=0x7f1ad00cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 0x7f1ad00cdfa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009260 tx=0x7f1ac4015000 comp rx=0 tx=0).stop 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 shutdown_connections 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ab4066f30 0x7f1ab40693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ad0157790 0x7f1ad00ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ad00af600 0x7f1ad00cdfa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 --2- 172.21.15.33:0/1814087021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad01573c0 0x7f1ad00c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.833+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 >> 172.21.15.33:0/1814087021 conn(0x7f1ad006e430 msgr2=0x7f1ad0157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.834+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 shutdown_connections 2023-10-06T05:16:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:16.834+0000 7f1ad5f9d700 1 -- 172.21.15.33:0/1814087021 wait complete. 2023-10-06T05:16:16.854 INFO:tasks.cephfs_test_runner:test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:16:16.854 INFO:tasks.cephfs_test_runner:Starting test: test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:16:16.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout' 2023-10-06T05:16:17.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.041+0000 7f98a89fe700 1 Processor -- start 2023-10-06T05:16:17.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.041+0000 7f98a89fe700 1 -- start start 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.042+0000 7f98a89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a4155af0 0x7f98a406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.042+0000 7f98a89fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a406ab60 con 0x7f98a406a6a0 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.042+0000 7f98a89fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a406ae40 con 0x7f98a40c7920 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.042+0000 7f98a89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a406b7e0 con 0x7f98a4155af0 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.042+0000 7f98a259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f98a406a6a0 0x7f98a40c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51404/0 (socket says 172.21.15.33:51404) 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.042+0000 7f98a259c700 1 -- 172.21.15.33:0/1656120197 learned_addr learned my addr 172.21.15.33:0/1656120197 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a2d9d700 1 --2- 172.21.15.33:0/1656120197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a4155af0 0x7f98a406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944133865 0 0) 0x7f98a406ab60 con 0x7f98a406a6a0 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f988c003570 con 0x7f98a406a6a0 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438751681 0 0) 0x7f98a406ae40 con 0x7f98a40c7920 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a406ab60 con 0x7f98a40c7920 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a2d9d700 1 -- 172.21.15.33:0/1656120197 >> v1:172.21.15.167:6789/0 conn(0x7f98a40c7920 legacy=0x7f98a40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a2d9d700 1 -- 172.21.15.33:0/1656120197 >> v1:172.21.15.33:6789/0 conn(0x7f98a406a6a0 legacy=0x7f98a40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a2d9d700 1 -- 172.21.15.33:0/1656120197 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a406b120 con 0x7f98a4155af0 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.043+0000 7f98a2d9d700 1 --2- 172.21.15.33:0/1656120197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a4155af0 0x7f98a406a160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9894009910 tx=0x7f98940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc22aa7c1f458a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989400f6b0 con 0x7f98a4155af0 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989400f810 con 0x7f98a4155af0 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a1d9b700 1 -- 172.21.15.33:0/1656120197 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989400fbb0 con 0x7f98a4155af0 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 -- 172.21.15.33:0/1656120197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a4155af0 msgr2=0x7f98a406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 --2- 172.21.15.33:0/1656120197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a4155af0 0x7f98a406a160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9894009910 tx=0x7f98940091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 -- 172.21.15.33:0/1656120197 shutdown_connections 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 --2- 172.21.15.33:0/1656120197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a4155af0 0x7f98a406a160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 -- 172.21.15.33:0/1656120197 >> 172.21.15.33:0/1656120197 conn(0x7f98a406d030 msgr2=0x7f98a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 -- 172.21.15.33:0/1656120197 shutdown_connections 2023-10-06T05:16:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.044+0000 7f98a89fe700 1 -- 172.21.15.33:0/1656120197 wait complete. 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.046+0000 7f98a89fe700 1 Processor -- start 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.046+0000 7f98a89fe700 1 -- start start 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a406a6a0 0x7f98a40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a89fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 0x7f98a414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a89fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98a4155af0 0x7f98a41587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a89fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40ae1b0 con 0x7f98a40c7920 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a89fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40acb00 con 0x7f98a4155af0 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40acde0 con 0x7f98a406a6a0 2023-10-06T05:16:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 0x7f98a414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 0x7f98a414cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47814/0 (socket says 172.21.15.33:47814) 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a259c700 1 -- 172.21.15.33:0/3155539089 learned_addr learned my addr 172.21.15.33:0/3155539089 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a2d9d700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a406a6a0 0x7f98a40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.047+0000 7f98a359e700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98a4155af0 0x7f98a41587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a259c700 1 -- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a406a6a0 msgr2=0x7f98a40ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a259c700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a406a6a0 0x7f98a40ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a259c700 1 -- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98a4155af0 msgr2=0x7f98a41587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a259c700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98a4155af0 0x7f98a41587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a259c700 1 -- 172.21.15.33:0/3155539089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894009420 con 0x7f98a40c7920 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a259c700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 0x7f98a414cdb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f989800c580 tx=0x7f9898007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:17.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898005b30 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898005c90 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40ad0c0 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898007320 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.048+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a40ade10 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.049+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a406c1b0 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.050+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f989801b3b0 con 0x7f98a40c7920 2023-10-06T05:16:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.051+0000 7f988affd700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9890066f30 0x7f98900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:17.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.051+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9898083820 con 0x7f98a40c7920 2023-10-06T05:16:17.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.052+0000 7f98a2d9d700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9890066f30 0x7f98900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:17.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.052+0000 7f98a2d9d700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9890066f30 0x7f98900693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f98940093f0 tx=0x7f9894019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:17.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.055+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9898050c30 con 0x7f98a40c7920 2023-10-06T05:16:17.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.340+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]} v 0) v1 -- 0x7f98a404c380 con 0x7f98a40c7920 2023-10-06T05:16:17.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:17.482 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.833+0000 7f988affd700 1 -- 172.21.15.33:0/3155539089 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]}]=0 v133) v1 ==== 162+0+0 (secure 0 0 0) 0x7f9898054250 con 0x7f98a40c7920 2023-10-06T05:16:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9890066f30 msgr2=0x7f98900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9890066f30 0x7f98900693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f98940093f0 tx=0x7f9894019000 comp rx=0 tx=0).stop 2023-10-06T05:16:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 msgr2=0x7f98a414cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 0x7f98a414cdb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f989800c580 tx=0x7f9898007e40 comp rx=0 tx=0).stop 2023-10-06T05:16:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 shutdown_connections 2023-10-06T05:16:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9890066f30 0x7f98900693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98a4155af0 0x7f98a41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98a406a6a0 0x7f98a40ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 --2- 172.21.15.33:0/3155539089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40c7920 0x7f98a414cdb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 >> 172.21.15.33:0/3155539089 conn(0x7f98a406d030 msgr2=0x7f98a40c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 shutdown_connections 2023-10-06T05:16:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:17.836+0000 7f98a89fe700 1 -- 172.21.15.33:0/3155539089 wait complete. 2023-10-06T05:16:17.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:16:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.045+0000 7f13d89fe700 1 Processor -- start 2023-10-06T05:16:18.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.045+0000 7f13d89fe700 1 -- start start 2023-10-06T05:16:18.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.046+0000 7f13d89fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f13d4155a50 0x7f13d4155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.046+0000 7f13d89fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40b06b0 con 0x7f13d40c9490 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.046+0000 7f13d89fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40c99e0 con 0x7f13d40afb20 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.046+0000 7f13d89fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40c9cc0 con 0x7f13d4155a50 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f13d40c9490 0x7f13d40af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51418/0 (socket says 172.21.15.33:51418) 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d259c700 1 -- 172.21.15.33:0/106393701 learned_addr learned my addr 172.21.15.33:0/106393701 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d2d9d700 1 --2- 172.21.15.33:0/106393701 >> v2:172.21.15.167:3300/0 conn(0x7f13d4155a50 0x7f13d4155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156151249 0 0) 0x7f13d40b06b0 con 0x7f13d40c9490 2023-10-06T05:16:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc003570 con 0x7f13d40c9490 2023-10-06T05:16:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441282904 0 0) 0x7f13bc003570 con 0x7f13d40c9490 2023-10-06T05:16:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 >> v2:172.21.15.167:3300/0 conn(0x7f13d4155a50 msgr2=0x7f13d4155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 --2- 172.21.15.33:0/106393701 >> v2:172.21.15.167:3300/0 conn(0x7f13d4155a50 0x7f13d4155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 >> v1:172.21.15.167:6789/0 conn(0x7f13d40afb20 legacy=0x7f13d40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40c9fa0 con 0x7f13d40c9490 2023-10-06T05:16:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f13c8001470 con 0x7f13d40c9490 2023-10-06T05:16:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.047+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 >> v1:172.21.15.33:6789/0 conn(0x7f13d40c9490 legacy=0x7f13d40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d1d9b700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc003a60 0x7f13bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d1d9b700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004810 0x7f13bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d1d9b700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc0051c0 0x7f13bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c9cc0 con 0x7f13bc004810 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc00c160 con 0x7f13bc0051c0 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc00c320 con 0x7f13bc003a60 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d259c700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004810 0x7f13bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d2d9d700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc003a60 0x7f13bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d359e700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc0051c0 0x7f13bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d259c700 1 -- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc003a60 msgr2=0x7f13bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d259c700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc003a60 0x7f13bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d259c700 1 -- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc0051c0 msgr2=0x7f13bc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d259c700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc0051c0 0x7f13bc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d259c700 1 -- 172.21.15.33:0/106393701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc003570 con 0x7f13bc004810 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d359e700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc0051c0 0x7f13bc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.048+0000 7f13d359e700 1 -- 172.21.15.33:0/106393701 reap_dead start 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d259c700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004810 0x7f13bc004c80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e5d0 tx=0x7f13c8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7542eb1f23d4e48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8007ba0 con 0x7f13bc004810 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8005920 con 0x7f13bc004810 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d1d9b700 1 -- 172.21.15.33:0/106393701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8005cc0 con 0x7f13bc004810 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 -- 172.21.15.33:0/106393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004810 msgr2=0x7f13bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004810 0x7f13bc004c80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e5d0 tx=0x7f13c8001e50 comp rx=0 tx=0).stop 2023-10-06T05:16:18.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 -- 172.21.15.33:0/106393701 shutdown_connections 2023-10-06T05:16:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 --2- 172.21.15.33:0/106393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004810 0x7f13bc004c80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 -- 172.21.15.33:0/106393701 >> 172.21.15.33:0/106393701 conn(0x7f13d406d030 msgr2=0x7f13d40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 -- 172.21.15.33:0/106393701 shutdown_connections 2023-10-06T05:16:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.049+0000 7f13d89fe700 1 -- 172.21.15.33:0/106393701 wait complete. 2023-10-06T05:16:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.051+0000 7f13d89fe700 1 Processor -- start 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.051+0000 7f13d89fe700 1 -- start start 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d89fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc004810 0x7f13d413dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d415ea60 0x7f13d415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d89fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 0x7f13d414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d89fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d414e450 con 0x7f13d414daa0 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d89fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d415dd70 con 0x7f13bc004810 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d415e5d0 con 0x7f13d415ea60 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 0x7f13d414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 0x7f13d414df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47830/0 (socket says 172.21.15.33:47830) 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d359e700 1 -- 172.21.15.33:0/4226294440 learned_addr learned my addr 172.21.15.33:0/4226294440 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d2d9d700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc004810 0x7f13d413dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.052+0000 7f13d259c700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d415ea60 0x7f13d415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d359e700 1 -- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d415ea60 msgr2=0x7f13d415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d359e700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d415ea60 0x7f13d415eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d359e700 1 -- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc004810 msgr2=0x7f13d413dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d359e700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc004810 0x7f13d413dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d359e700 1 -- 172.21.15.33:0/4226294440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d416ac40 con 0x7f13d414daa0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d359e700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 0x7f13d414df10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f13d406ddb0 tx=0x7f13cc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc012590 con 0x7f13d414daa0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0126f0 con 0x7f13d414daa0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d415df80 con 0x7f13d414daa0 2023-10-06T05:16:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc012ab0 con 0x7f13d414daa0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.053+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d413e260 con 0x7f13d414daa0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.054+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d415e780 con 0x7f13d414daa0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.055+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13cc003030 con 0x7f13d414daa0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.056+0000 7f13baffd700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13c0066f30 0x7f13c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.056+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f13cc0842b0 con 0x7f13d414daa0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.057+0000 7f13d2d9d700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13c0066f30 0x7f13c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.058+0000 7f13d2d9d700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13c0066f30 0x7f13c00693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13c4006f00 tx=0x7f13c40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.060+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13d415e780 con 0x7f13d414daa0 2023-10-06T05:16:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.262+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f13d415e740 con 0x7f13d414daa0 2023-10-06T05:16:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.266+0000 7f13baffd700 1 -- 172.21.15.33:0/4226294440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f13cc0550b0 con 0x7f13d414daa0 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13c0066f30 msgr2=0x7f13c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13c0066f30 0x7f13c00693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13c4006f00 tx=0x7f13c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 msgr2=0x7f13d414df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 0x7f13d414df10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f13d406ddb0 tx=0x7f13cc009040 comp rx=0 tx=0).stop 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 shutdown_connections 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13c0066f30 0x7f13c00693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d414daa0 0x7f13d414df10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc004810 0x7f13d413dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 --2- 172.21.15.33:0/4226294440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d415ea60 0x7f13d415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 >> 172.21.15.33:0/4226294440 conn(0x7f13d406d030 msgr2=0x7f13d40cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 shutdown_connections 2023-10-06T05:16:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.268+0000 7f13d89fe700 1 -- 172.21.15.33:0/4226294440 wait complete. 2023-10-06T05:16:18.288 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:16:18.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:16:18.370 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:16:18.463 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:16:18.463 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:16:18.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.649+0000 7fab7899d700 1 Processor -- start 2023-10-06T05:16:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.649+0000 7fab7899d700 1 -- start start 2023-10-06T05:16:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.650+0000 7fab7899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab74155af0 0x7fab7406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.650+0000 7fab7899d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab740c7920 0x7fab740c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.650+0000 7fab7899d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7406ab60 con 0x7fab7406a6a0 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab7899d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7406b7e0 con 0x7fab740c7920 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab7899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7406ae40 con 0x7fab74155af0 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab73fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab740c7920 0x7fab740c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab72ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab7406a6a0 0x7fab740c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51426/0 (socket says 172.21.15.33:51426) 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab72ffd700 1 -- 172.21.15.33:0/3851391601 learned_addr learned my addr 172.21.15.33:0/3851391601 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab737fe700 1 --2- 172.21.15.33:0/3851391601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab74155af0 0x7fab7406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab727fc700 1 -- 172.21.15.33:0/3851391601 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894377708 0 0) 0x7fab7406ab60 con 0x7fab7406a6a0 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab727fc700 1 -- 172.21.15.33:0/3851391601 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab5c003570 con 0x7fab7406a6a0 2023-10-06T05:16:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab73fff700 1 -- 172.21.15.33:0/3851391601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab74155af0 msgr2=0x7fab7406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab73fff700 1 --2- 172.21.15.33:0/3851391601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab74155af0 0x7fab7406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab73fff700 1 -- 172.21.15.33:0/3851391601 >> v1:172.21.15.33:6789/0 conn(0x7fab7406a6a0 legacy=0x7fab740c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.651+0000 7fab73fff700 1 -- 172.21.15.33:0/3851391601 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7406b120 con 0x7fab740c7920 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.652+0000 7fab73fff700 1 --2- 172.21.15.33:0/3851391601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740c9e20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009e60 tx=0x7fab6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1dc444100c4772 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.652+0000 7fab727fc700 1 -- 172.21.15.33:0/3851391601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00bab0 con 0x7fab740c7920 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.652+0000 7fab727fc700 1 -- 172.21.15.33:0/3851391601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c00bc10 con 0x7fab740c7920 2023-10-06T05:16:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.652+0000 7fab727fc700 1 -- 172.21.15.33:0/3851391601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c011400 con 0x7fab740c7920 2023-10-06T05:16:18.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.652+0000 7fab7899d700 1 -- 172.21.15.33:0/3851391601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 msgr2=0x7fab740c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.652+0000 7fab7899d700 1 --2- 172.21.15.33:0/3851391601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740c9e20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009e60 tx=0x7fab6c009780 comp rx=0 tx=0).stop 2023-10-06T05:16:18.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.653+0000 7fab7899d700 1 -- 172.21.15.33:0/3851391601 shutdown_connections 2023-10-06T05:16:18.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.653+0000 7fab7899d700 1 --2- 172.21.15.33:0/3851391601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740c9e20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.653+0000 7fab7899d700 1 --2- 172.21.15.33:0/3851391601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab74155af0 0x7fab7406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.653+0000 7fab7899d700 1 -- 172.21.15.33:0/3851391601 >> 172.21.15.33:0/3851391601 conn(0x7fab7406d030 msgr2=0x7fab7406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:18.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.653+0000 7fab7899d700 1 -- 172.21.15.33:0/3851391601 shutdown_connections 2023-10-06T05:16:18.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.653+0000 7fab7899d700 1 -- 172.21.15.33:0/3851391601 wait complete. 2023-10-06T05:16:18.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.654+0000 7fab7899d700 1 Processor -- start 2023-10-06T05:16:18.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.654+0000 7fab7899d700 1 -- start start 2023-10-06T05:16:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab7899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab7406a6a0 0x7fab74169320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab7899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab7899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74155af0 0x7fab740adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab7899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740ac800 con 0x7fab740c7920 2023-10-06T05:16:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab7899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740c4180 con 0x7fab74155af0 2023-10-06T05:16:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab7899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab74169860 con 0x7fab7406a6a0 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47852/0 (socket says 172.21.15.33:47852) 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.655+0000 7fab72ffd700 1 -- 172.21.15.33:0/609053427 learned_addr learned my addr 172.21.15.33:0/609053427 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab72ffd700 1 -- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab7406a6a0 msgr2=0x7fab74169320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab72ffd700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab7406a6a0 0x7fab74169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab72ffd700 1 -- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74155af0 msgr2=0x7fab740adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab72ffd700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74155af0 0x7fab740adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab72ffd700 1 -- 172.21.15.33:0/609053427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c009a20 con 0x7fab740c7920 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab72ffd700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740abaf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c560 tx=0x7fab68007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68005c00 con 0x7fab740c7920 2023-10-06T05:16:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68005d60 con 0x7fab740c7920 2023-10-06T05:16:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68007300 con 0x7fab740c7920 2023-10-06T05:16:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab740ac030 con 0x7fab740c7920 2023-10-06T05:16:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.656+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab740c3850 con 0x7fab740c7920 2023-10-06T05:16:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.658+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7404c380 con 0x7fab740c7920 2023-10-06T05:16:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.658+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab680076b0 con 0x7fab740c7920 2023-10-06T05:16:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.659+0000 7fab5b7fe700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab60066f30 0x7fab600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.659+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fab68083920 con 0x7fab740c7920 2023-10-06T05:16:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.660+0000 7fab737fe700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab60066f30 0x7fab600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.660+0000 7fab737fe700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab60066f30 0x7fab600693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fab64009bb0 tx=0x7fab640091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.663+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab68050c90 con 0x7fab740c7920 2023-10-06T05:16:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.911+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab7406b460 con 0x7fab740c7920 2023-10-06T05:16:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.912+0000 7fab5b7fe700 1 -- 172.21.15.33:0/609053427 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fab68017020 con 0x7fab740c7920 2023-10-06T05:16:18.914 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:18.914 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1657141302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1657141302},{"type":"v1","addr":"172.21.15.33:6835","nonce":1657141302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1197489941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1197489941},{"type":"v1","addr":"172.21.15.33:6833","nonce":1197489941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1659496492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1659496492},{"type":"v1","addr":"172.21.15.167:6836","nonce":1659496492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2833058561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2833058561},{"type":"v1","addr":"172.21.15.167:6837","nonce":2833058561}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-06T05:16:18.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab60066f30 msgr2=0x7fab600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab60066f30 0x7fab600693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fab64009bb0 tx=0x7fab640091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 msgr2=0x7fab740abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:18.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740abaf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fab6800c560 tx=0x7fab68007dd0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 shutdown_connections 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab60066f30 0x7fab600693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74155af0 0x7fab740adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab7406a6a0 0x7fab74169320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 --2- 172.21.15.33:0/609053427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab740c7920 0x7fab740abaf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 >> 172.21.15.33:0/609053427 conn(0x7fab7406d030 msgr2=0x7fab740cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 shutdown_connections 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:18.914+0000 7fab7899d700 1 -- 172.21.15.33:0/609053427 wait complete. 2023-10-06T05:16:18.917 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2023-10-06T05:16:18.933 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:16:18.933 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:16:18.934 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:16:18.934 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:18.934 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:16:18.934 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:16:18.934 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:16:18.934 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:18.935 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:16:18.935 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:16:18.935 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:16:18.935 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:18.935 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:16:18.935 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:16:18.935 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:16:18.936 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:18.949 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:16:18.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:16:18.950 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:16:18.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:16:18.953 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:16:18.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:16:18.994 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:16:18.994 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:18.997 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:16:18.997 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:18.998 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:16:18.999 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:19.127 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:16:19.127 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:16:19.129 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:16:19.127+0000 7f4a013fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-06T05:16:19.135 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:16:20.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T05:16:24.953 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:16:24.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:16:24.958 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:16:24.959 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:24.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:16:25.095 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:16:25.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.158+0000 7f115999d700 1 Processor -- start 2023-10-06T05:16:25.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.158+0000 7f115999d700 1 -- start start 2023-10-06T05:16:25.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.159+0000 7f115999d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f11540c7070 0x7f11540c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.159+0000 7f115999d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1154155bc0 0x7f11540c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.159+0000 7f115999d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115406a410 con 0x7f11541557f0 2023-10-06T05:16:25.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.159+0000 7f115999d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11540cccd0 con 0x7f1154155bc0 2023-10-06T05:16:25.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.159+0000 7f115999d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11540c7500 con 0x7f11540c7070 2023-10-06T05:16:25.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115919c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1154155bc0 0x7f11540c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.159+0000 7f115899b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11541557f0 0x7f11540c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57354/0 (socket says 172.21.15.33:57354) 2023-10-06T05:16:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115899b700 1 -- 172.21.15.33:0/3011170455 learned_addr learned my addr 172.21.15.33:0/3011170455 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:25.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f1153fff700 1 --2- 172.21.15.33:0/3011170455 >> v2:172.21.15.167:3300/0 conn(0x7f11540c7070 0x7f11540c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115919c700 1 -- 172.21.15.33:0/3011170455 >> v2:172.21.15.167:3300/0 conn(0x7f11540c7070 msgr2=0x7f11540c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115919c700 1 --2- 172.21.15.33:0/3011170455 >> v2:172.21.15.167:3300/0 conn(0x7f11540c7070 0x7f11540c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115919c700 1 -- 172.21.15.33:0/3011170455 >> v1:172.21.15.33:6789/0 conn(0x7f11541557f0 legacy=0x7f11540c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115919c700 1 -- 172.21.15.33:0/3011170455 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540c77e0 con 0x7f1154155bc0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f11537fe700 1 -- 172.21.15.33:0/3011170455 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833215969 0 0) 0x7f115406a410 con 0x7f11541557f0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.160+0000 7f115919c700 1 --2- 172.21.15.33:0/3011170455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1154155bc0 0x7f11540c9050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11540b22f0 tx=0x7f114c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39008a80c108ca0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f11537fe700 1 -- 172.21.15.33:0/3011170455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c00e070 con 0x7f1154155bc0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f11537fe700 1 -- 172.21.15.33:0/3011170455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c00bb60 con 0x7f1154155bc0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f11537fe700 1 -- 172.21.15.33:0/3011170455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c012430 con 0x7f1154155bc0 2023-10-06T05:16:25.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 -- 172.21.15.33:0/3011170455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1154155bc0 msgr2=0x7f11540c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 --2- 172.21.15.33:0/3011170455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1154155bc0 0x7f11540c9050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f11540b22f0 tx=0x7f114c009780 comp rx=0 tx=0).stop 2023-10-06T05:16:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 -- 172.21.15.33:0/3011170455 shutdown_connections 2023-10-06T05:16:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 --2- 172.21.15.33:0/3011170455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1154155bc0 0x7f11540c9050 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 --2- 172.21.15.33:0/3011170455 >> v2:172.21.15.167:3300/0 conn(0x7f11540c7070 0x7f11540c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 -- 172.21.15.33:0/3011170455 >> 172.21.15.33:0/3011170455 conn(0x7f115406c560 msgr2=0x7f115406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 -- 172.21.15.33:0/3011170455 shutdown_connections 2023-10-06T05:16:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.161+0000 7f115999d700 1 -- 172.21.15.33:0/3011170455 wait complete. 2023-10-06T05:16:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.163+0000 7f115999d700 1 Processor -- start 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.163+0000 7f115999d700 1 -- start start 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115999d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 0x7f11540c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11541557f0 0x7f115406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115999d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1154155bc0 0x7f1154145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115999d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11540af020 con 0x7f11540c7070 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115999d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11540afba0 con 0x7f1154155bc0 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11540af270 con 0x7f11541557f0 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 0x7f11540c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 0x7f11540c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60128/0 (socket says 172.21.15.33:60128) 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 -- 172.21.15.33:0/2793463628 learned_addr learned my addr 172.21.15.33:0/2793463628 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115919c700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1154155bc0 0x7f1154145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f1153fff700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11541557f0 0x7f115406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 -- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11541557f0 msgr2=0x7f115406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11541557f0 0x7f115406e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 -- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1154155bc0 msgr2=0x7f1154145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1154155bc0 0x7f1154145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115899b700 1 -- 172.21.15.33:0/2793463628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c009a20 con 0x7f11540c7070 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.164+0000 7f115919c700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1154155bc0 0x7f1154145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.165+0000 7f115899b700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 0x7f11540c4ff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f114800c7e0 tx=0x7f1148007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.165+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148005b30 con 0x7f11540c7070 2023-10-06T05:16:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.165+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148005c90 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.165+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148007320 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.165+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11540af550 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.165+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11540c65a0 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.166+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115404c380 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.167+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f114801b3b0 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.168+0000 7f1150ff9700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1138066f30 0x7f11380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.168+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1148083950 con 0x7f11540c7070 2023-10-06T05:16:25.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.168+0000 7f1153fff700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1138066f30 0x7f11380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.169+0000 7f1153fff700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1138066f30 0x7f11380693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f11540c42d0 tx=0x7f11400091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.172+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1148050cc0 con 0x7f11540c7070 2023-10-06T05:16:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:25.390+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f115414aab0 con 0x7f11540c7070 2023-10-06T05:16:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.680+0000 7f1150ff9700 1 -- 172.21.15.33:0/2793463628 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1148017080 con 0x7f11540c7070 2023-10-06T05:16:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1138066f30 msgr2=0x7f11380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1138066f30 0x7f11380693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f11540c42d0 tx=0x7f11400091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 msgr2=0x7f11540c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 0x7f11540c4ff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f114800c7e0 tx=0x7f1148007df0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 shutdown_connections 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1138066f30 0x7f11380693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1154155bc0 0x7f1154145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11540c7070 0x7f11540c4ff0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 --2- 172.21.15.33:0/2793463628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11541557f0 0x7f115406e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.684+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 >> 172.21.15.33:0/2793463628 conn(0x7f115406c560 msgr2=0x7f11540cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.685+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 shutdown_connections 2023-10-06T05:16:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.685+0000 7f115999d700 1 -- 172.21.15.33:0/2793463628 wait complete. 2023-10-06T05:16:26.688 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:16:26.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:16:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.899+0000 7fddff59e700 1 Processor -- start 2023-10-06T05:16:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.899+0000 7fddff59e700 1 -- start start 2023-10-06T05:16:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddff59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde00159bb0 0x7fde000b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddff59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000cbd80 con 0x7fde000ccfc0 2023-10-06T05:16:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddff59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000af5f0 con 0x7fde000cf3b0 2023-10-06T05:16:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddff59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000af8d0 con 0x7fde00159bb0 2023-10-06T05:16:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddfed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde00159bb0 0x7fde000b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddfe59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fde000ccfc0 0x7fde000cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57358/0 (socket says 172.21.15.33:57358) 2023-10-06T05:16:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.900+0000 7fddfe59c700 1 -- 172.21.15.33:0/3864703300 learned_addr learned my addr 172.21.15.33:0/3864703300 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfed9d700 1 -- 172.21.15.33:0/3864703300 >> v1:172.21.15.167:6789/0 conn(0x7fde000cf3b0 legacy=0x7fde001594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfed9d700 1 -- 172.21.15.33:0/3864703300 >> v1:172.21.15.33:6789/0 conn(0x7fde000ccfc0 legacy=0x7fde000cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfed9d700 1 -- 172.21.15.33:0/3864703300 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000afbb0 con 0x7fde00159bb0 2023-10-06T05:16:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfd59a700 1 -- 172.21.15.33:0/3864703300 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735496660 0 0) 0x7fde000cbd80 con 0x7fde000ccfc0 2023-10-06T05:16:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfed9d700 1 --2- 172.21.15.33:0/3864703300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159bb0 0x7fde000b0460 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009e90 tx=0x7fddf8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8f3b7f38da30b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfd59a700 1 -- 172.21.15.33:0/3864703300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800e070 con 0x7fde00159bb0 2023-10-06T05:16:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfd59a700 1 -- 172.21.15.33:0/3864703300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800bbb0 con 0x7fde00159bb0 2023-10-06T05:16:26.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.901+0000 7fddfd59a700 1 -- 172.21.15.33:0/3864703300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8012430 con 0x7fde00159bb0 2023-10-06T05:16:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 -- 172.21.15.33:0/3864703300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159bb0 msgr2=0x7fde000b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 --2- 172.21.15.33:0/3864703300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159bb0 0x7fde000b0460 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009e90 tx=0x7fddf8009780 comp rx=0 tx=0).stop 2023-10-06T05:16:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 -- 172.21.15.33:0/3864703300 shutdown_connections 2023-10-06T05:16:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 --2- 172.21.15.33:0/3864703300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159bb0 0x7fde000b0460 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 -- 172.21.15.33:0/3864703300 >> 172.21.15.33:0/3864703300 conn(0x7fde0006cff0 msgr2=0x7fde0006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 -- 172.21.15.33:0/3864703300 shutdown_connections 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.902+0000 7fddff59e700 1 -- 172.21.15.33:0/3864703300 wait complete. 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.903+0000 7fddff59e700 1 Processor -- start 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.904+0000 7fddff59e700 1 -- start start 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.904+0000 7fddff59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde000ccfc0 0x7fde00153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.904+0000 7fddff59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde00159bb0 0x7fde0016a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.904+0000 7fddff59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 0x7fde0015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddff59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0016ab20 con 0x7fde000c8c20 2023-10-06T05:16:26.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddff59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0016d6c0 con 0x7fde000ccfc0 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddff59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000c9f20 con 0x7fde00159bb0 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 0x7fde0015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 0x7fde0015e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60160/0 (socket says 172.21.15.33:60160) 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 -- 172.21.15.33:0/2399693355 learned_addr learned my addr 172.21.15.33:0/2399693355 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfe59c700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde000ccfc0 0x7fde00153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfdd9b700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde00159bb0 0x7fde0016a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 -- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde00159bb0 msgr2=0x7fde0016a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde00159bb0 0x7fde0016a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 -- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde000ccfc0 msgr2=0x7fde00153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde000ccfc0 0x7fde00153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 -- 172.21.15.33:0/2399693355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8009a20 con 0x7fde000c8c20 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfdd9b700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde00159bb0 0x7fde0016a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.905+0000 7fddfed9d700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 0x7fde0015e6a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fddf80099f0 tx=0x7fddf8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.906+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf801d640 con 0x7fde000c8c20 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.906+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf801d7e0 con 0x7fde000c8c20 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.906+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000ca1e0 con 0x7fde000c8c20 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.906+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf801dba0 con 0x7fde000c8c20 2023-10-06T05:16:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.906+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde000c9090 con 0x7fde000c8c20 2023-10-06T05:16:26.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.907+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0016d190 con 0x7fde000c8c20 2023-10-06T05:16:26.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.908+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddf8024070 con 0x7fde000c8c20 2023-10-06T05:16:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.909+0000 7fddee7fc700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde0066f30 0x7fdde00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.909+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fddf8086ec0 con 0x7fde000c8c20 2023-10-06T05:16:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.909+0000 7fddfe59c700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde0066f30 0x7fdde00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.910+0000 7fddfe59c700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde0066f30 0x7fdde00693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddf000c8f0 tx=0x7fddf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:26.913+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddf8054330 con 0x7fde000c8c20 2023-10-06T05:16:27.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.195+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fde0004c380 con 0x7fde000c8c20 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.195+0000 7fddee7fc700 1 -- 172.21.15.33:0/2399693355 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fddf8057950 con 0x7fde000c8c20 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:16:27.197 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:16:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:16:27.200 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:16:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:16:27.207 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.210 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.213 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:16:27.213 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:16:27.213 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:16:27.213 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:27.215 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:16:27.215 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:16:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde0066f30 msgr2=0x7fdde00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde0066f30 0x7fdde00693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddf000c8f0 tx=0x7fddf0009040 comp rx=0 tx=0).stop 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 msgr2=0x7fde0015e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 0x7fde0015e6a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fddf80099f0 tx=0x7fddf8018000 comp rx=0 tx=0).stop 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 shutdown_connections 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde0066f30 0x7fdde00693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde000c8c20 0x7fde0015e6a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde000ccfc0 0x7fde00153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 --2- 172.21.15.33:0/2399693355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde00159bb0 0x7fde0016a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.197+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 >> 172.21.15.33:0/2399693355 conn(0x7fde0006cff0 msgr2=0x7fde00155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.198+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 shutdown_connections 2023-10-06T05:16:27.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.198+0000 7fddff59e700 1 -- 172.21.15.33:0/2399693355 wait complete. 2023-10-06T05:16:27.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:27.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.403+0000 7fe7d1f9d700 1 Processor -- start 2023-10-06T05:16:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.403+0000 7fe7d1f9d700 1 -- start start 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d1f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0c8b50 0x7fe7cc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7cc0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d1f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc06ab60 con 0x7fe7cc155af0 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d1f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc06b7e0 con 0x7fe7cc0c8b50 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06ae40 con 0x7fe7cc155ec0 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d0f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7cc155af0 0x7fe7cc06a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57362/0 (socket says 172.21.15.33:57362) 2023-10-06T05:16:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.404+0000 7fe7d0f9b700 1 -- 172.21.15.33:0/505158054 learned_addr learned my addr 172.21.15.33:0/505158054 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/505158054 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0c8b50 0x7fe7cc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7d179c700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7cc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176013565 0 0) 0x7fe7cc06ab60 con 0x7fe7cc155af0 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc003570 con 0x7fe7cc155af0 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3171731892 0 0) 0x7fe7bc003570 con 0x7fe7cc155af0 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 msgr2=0x7fe7cc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7cc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0c8b50 msgr2=0x7fe7cc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 --2- 172.21.15.33:0/505158054 >> v2:172.21.15.167:3300/0 conn(0x7fe7cc0c8b50 0x7fe7cc0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc06b120 con 0x7fe7cc155af0 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe7c0001470 con 0x7fe7cc155af0 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 >> v1:172.21.15.33:6789/0 conn(0x7fe7cc155af0 legacy=0x7fe7cc06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc003a60 0x7fe7bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc004810 0x7fe7bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06b7e0 con 0x7fe7bc004810 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06ae40 con 0x7fe7bc003a60 2023-10-06T05:16:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.405+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc00ba60 con 0x7fe7cc155ec0 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc004810 0x7fe7bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7d0f9b700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc003a60 0x7fe7bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7d179c700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 -- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 msgr2=0x7fe7bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 -- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc003a60 msgr2=0x7fe7bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc003a60 0x7fe7bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 -- 172.21.15.33:0/505158054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc003570 con 0x7fe7bc004810 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7d179c700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7cc155ec0 0x7fe7bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7d0f9b700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc003a60 0x7fe7bc003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.406+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc004810 0x7fe7bc004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009c30 tx=0x7fe7b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7ddf06cf8569b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800e710 con 0x7fe7bc004810 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b800e870 con 0x7fe7bc004810 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7cb7fe700 1 -- 172.21.15.33:0/505158054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ec30 con 0x7fe7bc004810 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/505158054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc004810 msgr2=0x7fe7bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/505158054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc004810 0x7fe7bc004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009c30 tx=0x7fe7b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d179c700 1 -- 172.21.15.33:0/505158054 reap_dead start 2023-10-06T05:16:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/505158054 shutdown_connections 2023-10-06T05:16:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/505158054 >> 172.21.15.33:0/505158054 conn(0x7fe7cc06d030 msgr2=0x7fe7cc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/505158054 shutdown_connections 2023-10-06T05:16:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.407+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/505158054 wait complete. 2023-10-06T05:16:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.409+0000 7fe7d1f9d700 1 Processor -- start 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.409+0000 7fe7d1f9d700 1 -- start start 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc004810 0x7fe7cc14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d1f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 0x7fe7cc0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d1f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b1100 0x7fe7cc0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d1f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0c4210 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d1f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc14c460 con 0x7fe7cc0b1100 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0ac790 con 0x7fe7bc004810 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 0x7fe7cc0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 0x7fe7cc0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60172/0 (socket says 172.21.15.33:60172) 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 -- 172.21.15.33:0/3475603639 learned_addr learned my addr 172.21.15.33:0/3475603639 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d0f9b700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc004810 0x7fe7cc14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7d179c700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b1100 0x7fe7cc0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 -- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc004810 msgr2=0x7fe7cc14bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc004810 0x7fe7cc14bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 -- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b1100 msgr2=0x7fe7cc0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b1100 0x7fe7cc0b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.410+0000 7fe7cbfff700 1 -- 172.21.15.33:0/3475603639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0b1ab0 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.411+0000 7fe7cbfff700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 0x7fe7cc0c3190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e4b0 tx=0x7fe7b801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.411+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8022070 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.411+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b800e9e0 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.411+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800e710 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.411+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc0aca50 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.411+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc0c3970 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.412+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc0acbe0 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.413+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7b801c660 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.414+0000 7fe7c8ff9700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.415+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe7b8085b40 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.415+0000 7fe7d0f9b700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T05:16:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.415+0000 7fe7d0f9b700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0001ed0 tx=0x7fe7c0007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.419+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7b8052f00 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.626+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7cc04c380 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.627+0000 7fe7c8ff9700 1 -- 172.21.15.33:0/3475603639 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13650 (secure 0 0 0) 0x7fe7b8017090 con 0x7fe7cc0c6e80 2023-10-06T05:16:27.629 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:27.629 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:26.676798+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 msgr2=0x7fe7b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0001ed0 tx=0x7fe7c0007040 comp rx=0 tx=0).stop 2023-10-06T05:16:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 msgr2=0x7fe7cc0c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 0x7fe7cc0c3190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e4b0 tx=0x7fe7b801a000 comp rx=0 tx=0).stop 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 shutdown_connections 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7b0066f30 0x7fe7b00693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7cc0b1100 0x7fe7cc0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc004810 0x7fe7cc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 --2- 172.21.15.33:0/3475603639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0c6e80 0x7fe7cc0c3190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.629+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 >> 172.21.15.33:0/3475603639 conn(0x7fe7cc06d030 msgr2=0x7fe7cc0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.630+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 shutdown_connections 2023-10-06T05:16:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.630+0000 7fe7d1f9d700 1 -- 172.21.15.33:0/3475603639 wait complete. 2023-10-06T05:16:27.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:16:27.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:16:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.838+0000 7ff34bfff700 1 Processor -- start 2023-10-06T05:16:27.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.839+0000 7ff34bfff700 1 -- start start 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff34c159850 0x7ff34c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34c0b0cd0 con 0x7ff34c06d140 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34c15a840 con 0x7ff34c15acd0 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34c06e490 con 0x7ff34c159850 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff34c159850 0x7ff34c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff34c06d140 0x7ff34c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57366/0 (socket says 172.21.15.33:57366) 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff34c159850 0x7ff34c06e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60180/0 (socket says 172.21.15.33:60180) 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.840+0000 7ff34b7fe700 1 -- 172.21.15.33:0/4112111380 learned_addr learned my addr 172.21.15.33:0/4112111380 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff34affd700 1 -- 172.21.15.33:0/4112111380 >> v1:172.21.15.167:6789/0 conn(0x7ff34c15acd0 legacy=0x7ff34c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff34affd700 1 -- 172.21.15.33:0/4112111380 >> v1:172.21.15.33:6789/0 conn(0x7ff34c06d140 legacy=0x7ff34c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff34affd700 1 -- 172.21.15.33:0/4112111380 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c159f10 con 0x7ff34c159850 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff349ffb700 1 -- 172.21.15.33:0/4112111380 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801777831 0 0) 0x7ff34c0b0cd0 con 0x7ff34c06d140 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff34affd700 1 --2- 172.21.15.33:0/4112111380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c159850 0x7ff34c06e890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098e0 tx=0x7ff33c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc06da3c5bc5848 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff349ffb700 1 -- 172.21.15.33:0/4112111380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c009e80 con 0x7ff34c159850 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff349ffb700 1 -- 172.21.15.33:0/4112111380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c00b820 con 0x7ff34c159850 2023-10-06T05:16:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.841+0000 7ff349ffb700 1 -- 172.21.15.33:0/4112111380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00bba0 con 0x7ff34c159850 2023-10-06T05:16:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 -- 172.21.15.33:0/4112111380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c159850 msgr2=0x7ff34c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 --2- 172.21.15.33:0/4112111380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c159850 0x7ff34c06e890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098e0 tx=0x7ff33c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 -- 172.21.15.33:0/4112111380 shutdown_connections 2023-10-06T05:16:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 --2- 172.21.15.33:0/4112111380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c159850 0x7ff34c06e890 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 -- 172.21.15.33:0/4112111380 >> 172.21.15.33:0/4112111380 conn(0x7ff34c06cd40 msgr2=0x7ff34c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 -- 172.21.15.33:0/4112111380 shutdown_connections 2023-10-06T05:16:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.842+0000 7ff34bfff700 1 -- 172.21.15.33:0/4112111380 wait complete. 2023-10-06T05:16:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.844+0000 7ff34bfff700 1 Processor -- start 2023-10-06T05:16:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.844+0000 7ff34bfff700 1 -- start start 2023-10-06T05:16:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34c06d140 0x7ff34c146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34c159850 0x7ff34c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 0x7ff34c16ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0cd2a0 con 0x7ff34c15acd0 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0ab290 con 0x7ff34c06d140 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c16a600 con 0x7ff34c159850 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 0x7ff34c16ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 0x7ff34c16ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60182/0 (socket says 172.21.15.33:60182) 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 -- 172.21.15.33:0/1270461869 learned_addr learned my addr 172.21.15.33:0/1270461869 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34affd700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34c06d140 0x7ff34c146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34a7fc700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34c159850 0x7ff34c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 -- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34c159850 msgr2=0x7ff34c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34c159850 0x7ff34c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 -- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34c06d140 msgr2=0x7ff34c146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34c06d140 0x7ff34c146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34b7fe700 1 -- 172.21.15.33:0/1270461869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c009420 con 0x7ff34c15acd0 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.845+0000 7ff34a7fc700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34c159850 0x7ff34c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.846+0000 7ff34b7fe700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 0x7ff34c16ca30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff34400cbf0 tx=0x7ff344001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.846+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400a8d0 con 0x7ff34c15acd0 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.846+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34400aa30 con 0x7ff34c15acd0 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.846+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400adf0 con 0x7ff34c15acd0 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.846+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c0cd4b0 con 0x7ff34c15acd0 2023-10-06T05:16:27.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.846+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c0aaf20 con 0x7ff34c15acd0 2023-10-06T05:16:27.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.847+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34c0b4200 con 0x7ff34c15acd0 2023-10-06T05:16:27.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.848+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff344007300 con 0x7ff34c15acd0 2023-10-06T05:16:27.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.849+0000 7ff332ffd700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff338066f30 0x7ff3380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.849+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff344084720 con 0x7ff34c15acd0 2023-10-06T05:16:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.850+0000 7ff34affd700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff338066f30 0x7ff3380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.850+0000 7ff34affd700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff338066f30 0x7ff3380693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0030c0 tx=0x7ff33c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:27.853+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff344051b60 con 0x7ff34c15acd0 2023-10-06T05:16:28.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.055+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff34c04c380 con 0x7ff34c15acd0 2023-10-06T05:16:28.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.056+0000 7ff332ffd700 1 -- 172.21.15.33:0/1270461869 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff344055180 con 0x7ff34c15acd0 2023-10-06T05:16:28.057 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff338066f30 msgr2=0x7ff3380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff338066f30 0x7ff3380693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0030c0 tx=0x7ff33c016000 comp rx=0 tx=0).stop 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 msgr2=0x7ff34c16ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 0x7ff34c16ca30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff34400cbf0 tx=0x7ff344001980 comp rx=0 tx=0).stop 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 shutdown_connections 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff338066f30 0x7ff3380693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff34c15acd0 0x7ff34c16ca30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff34c06d140 0x7ff34c146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 --2- 172.21.15.33:0/1270461869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff34c159850 0x7ff34c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 >> 172.21.15.33:0/1270461869 conn(0x7ff34c06cd40 msgr2=0x7ff34c15d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 shutdown_connections 2023-10-06T05:16:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.058+0000 7ff34bfff700 1 -- 172.21.15.33:0/1270461869 wait complete. 2023-10-06T05:16:28.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:16:28.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.265+0000 7f11fd59d700 1 Processor -- start 2023-10-06T05:16:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.265+0000 7f11fd59d700 1 -- start start 2023-10-06T05:16:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.266+0000 7f11fd59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 0x7f11f815a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.266+0000 7f11fd59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80ce890 con 0x7f11f80cd5e0 2023-10-06T05:16:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.266+0000 7f11fd59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f815d890 con 0x7f11f80af540 2023-10-06T05:16:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.266+0000 7f11fd59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80afd50 con 0x7f11f80af910 2023-10-06T05:16:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.266+0000 7f11f77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11f80cd5e0 0x7f11f80ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57370/0 (socket says 172.21.15.33:57370) 2023-10-06T05:16:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.266+0000 7f11f77fe700 1 -- 172.21.15.33:0/4263412287 learned_addr learned my addr 172.21.15.33:0/4263412287 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 0x7f11f815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090946030 0 0) 0x7f11f80ce890 con 0x7f11f80cd5e0 2023-10-06T05:16:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e0003570 con 0x7f11f80cd5e0 2023-10-06T05:16:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433552619 0 0) 0x7f11e0003570 con 0x7f11f80cd5e0 2023-10-06T05:16:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 msgr2=0x7f11f815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 0x7f11f815a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 >> v1:172.21.15.167:6789/0 conn(0x7f11f80af540 legacy=0x7f11f80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f815db70 con 0x7f11f80cd5e0 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f11ec001470 con 0x7f11f80cd5e0 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 >> v1:172.21.15.33:6789/0 conn(0x7f11f80cd5e0 legacy=0x7f11f80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 0x7f11e0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e0004010 0x7f11e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.267+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80afd50 con 0x7f11e0004d50 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e000bbf0 con 0x7f11e0004010 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e000bdb0 con 0x7f11f80af910 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11f7fff700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 0x7f11e0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11f77fe700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e0004010 0x7f11e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11fcd9c700 1 -- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 msgr2=0x7f11e0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80af910 0x7f11e0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11fcd9c700 1 -- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e0004010 msgr2=0x7f11e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e0004010 0x7f11e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.268+0000 7f11fcd9c700 1 -- 172.21.15.33:0/4263412287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0003570 con 0x7f11e0004d50 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11e00051c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f11f00094d0 tx=0x7f11f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd3c5e1137d7da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000ea90 con 0x7f11e0004d50 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000f040 con 0x7f11e0004d50 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11f6ffd700 1 -- 172.21.15.33:0/4263412287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000ee20 con 0x7f11e0004d50 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fd59d700 1 -- 172.21.15.33:0/4263412287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 msgr2=0x7f11e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fd59d700 1 --2- 172.21.15.33:0/4263412287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11e00051c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f11f00094d0 tx=0x7f11f000b040 comp rx=0 tx=0).stop 2023-10-06T05:16:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fcd9c700 1 -- 172.21.15.33:0/4263412287 reap_dead start 2023-10-06T05:16:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fd59d700 1 -- 172.21.15.33:0/4263412287 shutdown_connections 2023-10-06T05:16:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fd59d700 1 -- 172.21.15.33:0/4263412287 >> 172.21.15.33:0/4263412287 conn(0x7f11f806d9b0 msgr2=0x7f11f806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.269+0000 7f11fd59d700 1 -- 172.21.15.33:0/4263412287 shutdown_connections 2023-10-06T05:16:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.270+0000 7f11fd59d700 1 -- 172.21.15.33:0/4263412287 wait complete. 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.271+0000 7f11fd59d700 1 Processor -- start 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.271+0000 7f11fd59d700 1 -- start start 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fd59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11f8157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fd59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80b3960 0x7f11f80cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fd59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80b3db0 0x7f11f80cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fd59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c8d90 con 0x7f11e0004d50 2023-10-06T05:16:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fd59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b4bc0 con 0x7f11f80b3db0 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fd59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b4320 con 0x7f11f80b3960 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11f8157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11f8157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60190/0 (socket says 172.21.15.33:60190) 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11f7fff700 1 -- 172.21.15.33:0/3702381850 learned_addr learned my addr 172.21.15.33:0/3702381850 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11f77fe700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80b3960 0x7f11f80cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.272+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80b3db0 0x7f11f80cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11f7fff700 1 -- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80b3960 msgr2=0x7f11f80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11f7fff700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80b3960 0x7f11f80cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11f7fff700 1 -- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80b3db0 msgr2=0x7f11f80cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11f7fff700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80b3db0 0x7f11f80cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11f7fff700 1 -- 172.21.15.33:0/3702381850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80b4600 con 0x7f11e0004d50 2023-10-06T05:16:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11fcd9c700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80b3db0 0x7f11f80cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11f7fff700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11f8157070 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f11e8002950 tx=0x7f11e8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800f670 con 0x7f11e0004d50 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800f7d0 con 0x7f11e0004d50 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f80b48e0 con 0x7f11e0004d50 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800fb50 con 0x7f11e0004d50 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.273+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80b33c0 con 0x7f11e0004d50 2023-10-06T05:16:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.274+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f80c8b60 con 0x7f11e0004d50 2023-10-06T05:16:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.275+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11e801c400 con 0x7f11e0004d50 2023-10-06T05:16:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.276+0000 7f11dffff700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f60 0x7f11e4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.276+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f11e8084820 con 0x7f11e0004d50 2023-10-06T05:16:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.277+0000 7f11f77fe700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f60 0x7f11e4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.277+0000 7f11f77fe700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f60 0x7f11e4069410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000e50 tx=0x7f11ec001e50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.280+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f80c8b60 con 0x7f11e0004d50 2023-10-06T05:16:28.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.482+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f11f80c8b60 con 0x7f11e0004d50 2023-10-06T05:16:28.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.482+0000 7f11dffff700 1 -- 172.21.15.33:0/3702381850 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f11f80c8b60 con 0x7f11e0004d50 2023-10-06T05:16:28.484 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f60 msgr2=0x7f11e4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f60 0x7f11e4069410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000e50 tx=0x7f11ec001e50 comp rx=0 tx=0).stop 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 msgr2=0x7f11f8157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11f8157070 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f11e8002950 tx=0x7f11e8007d30 comp rx=0 tx=0).stop 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 shutdown_connections 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f60 0x7f11e4069410 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80b3db0 0x7f11f80cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11e0004d50 0x7f11f8157070 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 --2- 172.21.15.33:0/3702381850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80b3960 0x7f11f80cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 >> 172.21.15.33:0/3702381850 conn(0x7f11f806d9b0 msgr2=0x7f11f80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:28.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 shutdown_connections 2023-10-06T05:16:28.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.484+0000 7f11fd59d700 1 -- 172.21.15.33:0/3702381850 wait complete. 2023-10-06T05:16:28.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:16:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.690+0000 7f739df9d700 1 Processor -- start 2023-10-06T05:16:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.690+0000 7f739df9d700 1 -- start start 2023-10-06T05:16:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.691+0000 7f739df9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7398159520 0x7f73980c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.691+0000 7f739df9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739806c820 con 0x7f73980c82a0 2023-10-06T05:16:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.691+0000 7f739df9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739806a160 con 0x7f73980c6170 2023-10-06T05:16:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.691+0000 7f739df9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73980c6610 con 0x7f7398159520 2023-10-06T05:16:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f739d79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73980c82a0 0x7f739806a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57372/0 (socket says 172.21.15.33:57372) 2023-10-06T05:16:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f739d79c700 1 -- 172.21.15.33:0/4015868152 learned_addr learned my addr 172.21.15.33:0/4015868152 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f739cf9b700 1 --2- 172.21.15.33:0/4015868152 >> v2:172.21.15.167:3300/0 conn(0x7f7398159520 0x7f73980c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f73977fe700 1 -- 172.21.15.33:0/4015868152 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236695118 0 0) 0x7f739806c820 con 0x7f73980c82a0 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f73977fe700 1 -- 172.21.15.33:0/4015868152 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7388003570 con 0x7f73980c82a0 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f739cf9b700 1 -- 172.21.15.33:0/4015868152 >> v1:172.21.15.167:6789/0 conn(0x7f73980c6170 legacy=0x7f73980c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f739cf9b700 1 -- 172.21.15.33:0/4015868152 >> v1:172.21.15.33:6789/0 conn(0x7f73980c82a0 legacy=0x7f739806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.692+0000 7f739cf9b700 1 -- 172.21.15.33:0/4015868152 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980c8800 con 0x7f7398159520 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.693+0000 7f739cf9b700 1 --2- 172.21.15.33:0/4015868152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f73980c5c30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f738c0098e0 tx=0x7f738c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8867d71c914b57c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.693+0000 7f73977fe700 1 -- 172.21.15.33:0/4015868152 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00b640 con 0x7f7398159520 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.693+0000 7f73977fe700 1 -- 172.21.15.33:0/4015868152 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c00b7a0 con 0x7f7398159520 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.693+0000 7f73977fe700 1 -- 172.21.15.33:0/4015868152 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00bb40 con 0x7f7398159520 2023-10-06T05:16:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 -- 172.21.15.33:0/4015868152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 msgr2=0x7f73980c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 --2- 172.21.15.33:0/4015868152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f73980c5c30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f738c0098e0 tx=0x7f738c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 -- 172.21.15.33:0/4015868152 shutdown_connections 2023-10-06T05:16:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 --2- 172.21.15.33:0/4015868152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f73980c5c30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 -- 172.21.15.33:0/4015868152 >> 172.21.15.33:0/4015868152 conn(0x7f73980cc930 msgr2=0x7f73980ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 -- 172.21.15.33:0/4015868152 shutdown_connections 2023-10-06T05:16:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.694+0000 7f739df9d700 1 -- 172.21.15.33:0/4015868152 wait complete. 2023-10-06T05:16:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.695+0000 7f739df9d700 1 Processor -- start 2023-10-06T05:16:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 -- start start 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73980c6170 0x7f739814a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 0x7f739815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f739815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398159d00 con 0x7f73980c82a0 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73980ad050 con 0x7f7398159520 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.696+0000 7f739df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73980ad330 con 0x7f73980c6170 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 0x7f739815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 0x7f739815aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60198/0 (socket says 172.21.15.33:60198) 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 -- 172.21.15.33:0/3426300326 learned_addr learned my addr 172.21.15.33:0/3426300326 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f739cf9b700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73980c6170 0x7f739814a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f739d79c700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f739815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 -- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73980c6170 msgr2=0x7f739814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73980c6170 0x7f739814a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 -- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 msgr2=0x7f739815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f739815a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 -- 172.21.15.33:0/3426300326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c009420 con 0x7f73980c82a0 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f739d79c700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f739815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.697+0000 7f7397fff700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 0x7f739815aef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f738400c590 tx=0x7f738400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.698+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384011590 con 0x7f73980c82a0 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.698+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384011710 con 0x7f73980c82a0 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.698+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7398159990 con 0x7f73980c82a0 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.698+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384011ad0 con 0x7f73980c82a0 2023-10-06T05:16:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.698+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73980ad610 con 0x7f73980c82a0 2023-10-06T05:16:28.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.699+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7398159e70 con 0x7f73980c82a0 2023-10-06T05:16:28.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.700+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f738401b3b0 con 0x7f73980c82a0 2023-10-06T05:16:28.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.700+0000 7f7394ff9700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f737c066f30 0x7f737c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.701+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7384083920 con 0x7f73980c82a0 2023-10-06T05:16:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.701+0000 7f739cf9b700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f737c066f30 0x7f737c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.702+0000 7f739cf9b700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f737c066f30 0x7f737c0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f738c0097b0 tx=0x7f738c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:28.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.705+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7384050ce0 con 0x7f73980c82a0 2023-10-06T05:16:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.911+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f739804c380 con 0x7f73980c82a0 2023-10-06T05:16:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.911+0000 7f7394ff9700 1 -- 172.21.15.33:0/3426300326 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7384017070 con 0x7f73980c82a0 2023-10-06T05:16:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:28.913 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:16:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:16:28.914 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f737c066f30 msgr2=0x7f737c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f737c066f30 0x7f737c0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f738c0097b0 tx=0x7f738c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 msgr2=0x7f739815aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 0x7f739815aef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f738400c590 tx=0x7f738400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 shutdown_connections 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f737c066f30 0x7f737c0693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398159520 0x7f739815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73980c6170 0x7f739814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 --2- 172.21.15.33:0/3426300326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73980c82a0 0x7f739815aef0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 >> 172.21.15.33:0/3426300326 conn(0x7f73980cc930 msgr2=0x7f73980b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 shutdown_connections 2023-10-06T05:16:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:28.913+0000 7f739df9d700 1 -- 172.21.15.33:0/3426300326 wait complete. 2023-10-06T05:16:28.932 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:16:28.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:16:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.121+0000 7ff922d9d700 1 Processor -- start 2023-10-06T05:16:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.121+0000 7ff922d9d700 1 -- start start 2023-10-06T05:16:29.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff922d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff91c0ad010 0x7ff91c06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff922d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad3e0 0x7ff91c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff922d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff91c15ab00 0x7ff91c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff922d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c06a3f0 con 0x7ff91c0ad010 2023-10-06T05:16:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff922d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c06b1f0 con 0x7ff91c15ab00 2023-10-06T05:16:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff922d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c06a5e0 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff91c0ad010 0x7ff91c06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff91c0ad010 0x7ff91c06ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60202/0 (socket says 172.21.15.33:60202) 2023-10-06T05:16:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 -- 172.21.15.33:0/4096320687 learned_addr learned my addr 172.21.15.33:0/4096320687 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff92259c700 1 --2- 172.21.15.33:0/4096320687 >> v2:172.21.15.167:3300/0 conn(0x7ff91c15ab00 0x7ff91c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff92159a700 1 --2- 172.21.15.33:0/4096320687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad3e0 0x7ff91c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 -- 172.21.15.33:0/4096320687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad3e0 msgr2=0x7ff91c15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 --2- 172.21.15.33:0/4096320687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad3e0 0x7ff91c15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 -- 172.21.15.33:0/4096320687 >> v2:172.21.15.167:3300/0 conn(0x7ff91c15ab00 msgr2=0x7ff91c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 --2- 172.21.15.33:0/4096320687 >> v2:172.21.15.167:3300/0 conn(0x7ff91c15ab00 0x7ff91c15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff921d9b700 1 -- 172.21.15.33:0/4096320687 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c06a8c0 con 0x7ff91c0ad010 2023-10-06T05:16:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.123+0000 7ff92259c700 1 --2- 172.21.15.33:0/4096320687 >> v2:172.21.15.167:3300/0 conn(0x7ff91c15ab00 0x7ff91c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.124+0000 7ff921d9b700 1 --2- 172.21.15.33:0/4096320687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad010 0x7ff91c06ea00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0699c0 tx=0x7ff918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881be6409ee5e74d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.124+0000 7ff920d99700 1 -- 172.21.15.33:0/4096320687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800b8f0 con 0x7ff91c0ad010 2023-10-06T05:16:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.124+0000 7ff920d99700 1 -- 172.21.15.33:0/4096320687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91800ba50 con 0x7ff91c0ad010 2023-10-06T05:16:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.124+0000 7ff920d99700 1 -- 172.21.15.33:0/4096320687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800bdf0 con 0x7ff91c0ad010 2023-10-06T05:16:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.124+0000 7ff922d9d700 1 -- 172.21.15.33:0/4096320687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad010 msgr2=0x7ff91c06ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.124+0000 7ff922d9d700 1 --2- 172.21.15.33:0/4096320687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad010 0x7ff91c06ea00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0699c0 tx=0x7ff918009250 comp rx=0 tx=0).stop 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 -- 172.21.15.33:0/4096320687 shutdown_connections 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 --2- 172.21.15.33:0/4096320687 >> v2:172.21.15.167:3300/0 conn(0x7ff91c15ab00 0x7ff91c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 --2- 172.21.15.33:0/4096320687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad010 0x7ff91c06ea00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 --2- 172.21.15.33:0/4096320687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad3e0 0x7ff91c15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 -- 172.21.15.33:0/4096320687 >> 172.21.15.33:0/4096320687 conn(0x7ff91c068090 msgr2=0x7ff91c06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 -- 172.21.15.33:0/4096320687 shutdown_connections 2023-10-06T05:16:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.125+0000 7ff922d9d700 1 -- 172.21.15.33:0/4096320687 wait complete. 2023-10-06T05:16:29.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.126+0000 7ff922d9d700 1 Processor -- start 2023-10-06T05:16:29.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.126+0000 7ff922d9d700 1 -- start start 2023-10-06T05:16:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff922d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad010 0x7ff91c169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff922d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 0x7ff91c0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff922d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c15ab00 0x7ff91c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff922d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c16ac50 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff922d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c16b040 con 0x7ff91c15ab00 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff922d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c16a150 con 0x7ff91c0ad010 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.127+0000 7ff92159a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 0x7ff91c0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 0x7ff91c0cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60216/0 (socket says 172.21.15.33:60216) 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 -- 172.21.15.33:0/484424217 learned_addr learned my addr 172.21.15.33:0/484424217 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff921d9b700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad010 0x7ff91c169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92259c700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c15ab00 0x7ff91c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 -- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad010 msgr2=0x7ff91c169c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad010 0x7ff91c169c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 -- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c15ab00 msgr2=0x7ff91c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c15ab00 0x7ff91c0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 -- 172.21.15.33:0/484424217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180094f0 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff921d9b700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad010 0x7ff91c169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92259c700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c15ab00 0x7ff91c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff92159a700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 0x7ff91c0cc580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00d890 tx=0x7ff90c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.128+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00b600 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.129+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c00b760 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.129+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00bae0 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.129+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c16a3b0 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.129+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c16a970 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.130+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c16ae10 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.130+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff90c01e460 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.131+0000 7ff911ffb700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff900066f30 0x7ff9000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.131+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff90c084a90 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.132+0000 7ff921d9b700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff900066f30 0x7ff9000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.132+0000 7ff921d9b700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff900066f30 0x7ff9000693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0699c0 tx=0x7ff918017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.135+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff91c16ae10 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.339+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff91c16ae10 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.706+0000 7ff911ffb700 1 -- 172.21.15.33:0/484424217 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff91c16ae10 con 0x7ff91c0ad3e0 2023-10-06T05:16:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.710+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff900066f30 msgr2=0x7ff9000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.710+0000 7ff922d9d700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff900066f30 0x7ff9000693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0699c0 tx=0x7ff918017000 comp rx=0 tx=0).stop 2023-10-06T05:16:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.710+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 msgr2=0x7ff91c0cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.710+0000 7ff922d9d700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 0x7ff91c0cc580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00d890 tx=0x7ff90c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.711+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 shutdown_connections 2023-10-06T05:16:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.711+0000 7ff922d9d700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff900066f30 0x7ff9000693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.711+0000 7ff922d9d700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff91c15ab00 0x7ff91c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.711+0000 7ff922d9d700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff91c0ad010 0x7ff91c169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.711+0000 7ff922d9d700 1 --2- 172.21.15.33:0/484424217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c0ad3e0 0x7ff91c0cc580 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.711+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 >> 172.21.15.33:0/484424217 conn(0x7ff91c068090 msgr2=0x7ff91c0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.713+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 shutdown_connections 2023-10-06T05:16:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.713+0000 7ff922d9d700 1 -- 172.21.15.33:0/484424217 wait complete. 2023-10-06T05:16:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:16:29.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:16:29.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.928+0000 7fa0f919d700 1 Processor -- start 2023-10-06T05:16:29.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.928+0000 7fa0f919d700 1 -- start start 2023-10-06T05:16:29.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.929+0000 7fa0f919d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0f4156370 0x7fa0f40aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f40c7f60 0x7fa0f40c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f919d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f40c8900 con 0x7fa0f4156740 2023-10-06T05:16:29.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f919d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f406beb0 con 0x7fa0f4156370 2023-10-06T05:16:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f406b330 con 0x7fa0f40c7f60 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0f4156370 0x7fa0f40aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f37fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa0f4156740 0x7fa0f40cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:52518/0 (socket says 172.21.15.33:52518) 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f899c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f40c7f60 0x7fa0f40c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.930+0000 7fa0f37fe700 1 -- 172.21.15.33:0/3578954650 learned_addr learned my addr 172.21.15.33:0/3578954650 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.931+0000 7fa0f3fff700 1 -- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f40c7f60 msgr2=0x7fa0f40c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.931+0000 7fa0f3fff700 1 --2- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f40c7f60 0x7fa0f40c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.931+0000 7fa0f3fff700 1 -- 172.21.15.33:0/3578954650 >> v1:172.21.15.167:6789/0 conn(0x7fa0f4156740 legacy=0x7fa0f40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.931+0000 7fa0f3fff700 1 -- 172.21.15.33:0/3578954650 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f406b610 con 0x7fa0f4156370 2023-10-06T05:16:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.931+0000 7fa0f3fff700 1 --2- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156370 0x7fa0f40aef80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40098e0 tx=0x7fa0e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1600c4c5c0ea8b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.931+0000 7fa0f2ffd700 1 -- 172.21.15.33:0/3578954650 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400b640 con 0x7fa0f4156370 2023-10-06T05:16:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f2ffd700 1 -- 172.21.15.33:0/3578954650 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400b7a0 con 0x7fa0f4156370 2023-10-06T05:16:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f2ffd700 1 -- 172.21.15.33:0/3578954650 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400bb40 con 0x7fa0f4156370 2023-10-06T05:16:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 -- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156370 msgr2=0x7fa0f40aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 --2- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156370 0x7fa0f40aef80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40098e0 tx=0x7fa0e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 -- 172.21.15.33:0/3578954650 shutdown_connections 2023-10-06T05:16:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 --2- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156370 0x7fa0f40aef80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 --2- 172.21.15.33:0/3578954650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f40c7f60 0x7fa0f40c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 -- 172.21.15.33:0/3578954650 >> 172.21.15.33:0/3578954650 conn(0x7fa0f406c560 msgr2=0x7fa0f406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 -- 172.21.15.33:0/3578954650 shutdown_connections 2023-10-06T05:16:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.932+0000 7fa0f919d700 1 -- 172.21.15.33:0/3578954650 wait complete. 2023-10-06T05:16:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.934+0000 7fa0f919d700 1 Processor -- start 2023-10-06T05:16:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.934+0000 7fa0f919d700 1 -- start start 2023-10-06T05:16:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f919d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 0x7fa0f415b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f4156370 0x7fa0f416dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156740 0x7fa0f414a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f919d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f415dac0 con 0x7fa0f40c7f60 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40cc620 con 0x7fa0f4156740 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f416d760 con 0x7fa0f4156370 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 0x7fa0f415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 0x7fa0f415b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52304/0 (socket says 172.21.15.33:52304) 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 -- 172.21.15.33:0/1529928422 learned_addr learned my addr 172.21.15.33:0/1529928422 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f899c700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156740 0x7fa0f414a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f37fe700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f4156370 0x7fa0f416dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 -- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f4156370 msgr2=0x7fa0f416dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f4156370 0x7fa0f416dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 -- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156740 msgr2=0x7fa0f414a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156740 0x7fa0f414a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.935+0000 7fa0f3fff700 1 -- 172.21.15.33:0/1529928422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4009420 con 0x7fa0f40c7f60 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.936+0000 7fa0f3fff700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 0x7fa0f415b800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4069740 tx=0x7fa0e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.936+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e401bb70 con 0x7fa0f40c7f60 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.936+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e401bcd0 con 0x7fa0f40c7f60 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.936+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4011670 con 0x7fa0f40c7f60 2023-10-06T05:16:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.936+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f415c920 con 0x7fa0f40c7f60 2023-10-06T05:16:29.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.936+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f416d3f0 con 0x7fa0f40c7f60 2023-10-06T05:16:29.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.937+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f404c380 con 0x7fa0f40c7f60 2023-10-06T05:16:29.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.938+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0e4011a20 con 0x7fa0f40c7f60 2023-10-06T05:16:29.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.939+0000 7fa0dbfff700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0e0066f30 0x7fa0e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.940+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fa0e4085970 con 0x7fa0f40c7f60 2023-10-06T05:16:29.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.940+0000 7fa0f37fe700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0e0066f30 0x7fa0e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.941+0000 7fa0f37fe700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0e0066f30 0x7fa0e00693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa0f415c4b0 tx=0x7fa0e8007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:29.944+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0e4052bc0 con 0x7fa0f40c7f60 2023-10-06T05:16:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.149+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa0f406dec0 con 0x7fa0f40c7f60 2023-10-06T05:16:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.713+0000 7fa0dbfff700 1 -- 172.21.15.33:0/1529928422 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa0e40561e0 con 0x7fa0f40c7f60 2023-10-06T05:16:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.719+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0e0066f30 msgr2=0x7fa0e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.719+0000 7fa0f919d700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0e0066f30 0x7fa0e00693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa0f415c4b0 tx=0x7fa0e8007eb0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.719+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 msgr2=0x7fa0f415b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.719+0000 7fa0f919d700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 0x7fa0f415b800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4069740 tx=0x7fa0e4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.721+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 shutdown_connections 2023-10-06T05:16:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.721+0000 7fa0f919d700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0e0066f30 0x7fa0e00693e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.721+0000 7fa0f919d700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0f4156740 0x7fa0f414a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.721+0000 7fa0f919d700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0f40c7f60 0x7fa0f415b800 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.721+0000 7fa0f919d700 1 --2- 172.21.15.33:0/1529928422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0f4156370 0x7fa0f416dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.721+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 >> 172.21.15.33:0/1529928422 conn(0x7fa0f406c560 msgr2=0x7fa0f414d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.724+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 shutdown_connections 2023-10-06T05:16:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.724+0000 7fa0f919d700 1 -- 172.21.15.33:0/1529928422 wait complete. 2023-10-06T05:16:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:16:30.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:16:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.946+0000 7f86bdb9d700 1 Processor -- start 2023-10-06T05:16:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.946+0000 7f86bdb9d700 1 -- start start 2023-10-06T05:16:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b80af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bdb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f86b815a650 0x7f86b815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bdb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b815d660 con 0x7f86b80af540 2023-10-06T05:16:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bdb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b80ce890 con 0x7f86b815a650 2023-10-06T05:16:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b815aad0 con 0x7f86b80cd5e0 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bd39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f86b815a650 0x7f86b815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bcb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f86b80af540 0x7f86b80cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58708/0 (socket says 172.21.15.33:58708) 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.947+0000 7f86bcb9b700 1 -- 172.21.15.33:0/1328961337 learned_addr learned my addr 172.21.15.33:0/1328961337 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86b7fff700 1 --2- 172.21.15.33:0/1328961337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b80af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86bd39c700 1 -- 172.21.15.33:0/1328961337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 msgr2=0x7f86b80af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86bd39c700 1 --2- 172.21.15.33:0/1328961337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b80af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86bd39c700 1 -- 172.21.15.33:0/1328961337 >> v1:172.21.15.33:6789/0 conn(0x7f86b80af540 legacy=0x7f86b80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86bd39c700 1 -- 172.21.15.33:0/1328961337 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b815d890 con 0x7f86b815a650 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86b77fe700 1 -- 172.21.15.33:0/1328961337 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075107479 0 0) 0x7f86b815d660 con 0x7f86b80af540 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86bd39c700 1 --2- 172.21.15.33:0/1328961337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b815d120 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009e60 tx=0x7f86b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a770092ebc4eb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:30.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86b77fe700 1 -- 172.21.15.33:0/1328961337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000e070 con 0x7f86b815a650 2023-10-06T05:16:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86b77fe700 1 -- 172.21.15.33:0/1328961337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b000bb60 con 0x7f86b815a650 2023-10-06T05:16:30.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.948+0000 7f86b77fe700 1 -- 172.21.15.33:0/1328961337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0012430 con 0x7f86b815a650 2023-10-06T05:16:30.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 -- 172.21.15.33:0/1328961337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 msgr2=0x7f86b815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/1328961337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b815d120 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009e60 tx=0x7f86b0009780 comp rx=0 tx=0).stop 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 -- 172.21.15.33:0/1328961337 shutdown_connections 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/1328961337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b815d120 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/1328961337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b80af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 -- 172.21.15.33:0/1328961337 >> 172.21.15.33:0/1328961337 conn(0x7f86b806d9b0 msgr2=0x7f86b806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 -- 172.21.15.33:0/1328961337 shutdown_connections 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.949+0000 7f86bdb9d700 1 -- 172.21.15.33:0/1328961337 wait complete. 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.950+0000 7f86bdb9d700 1 Processor -- start 2023-10-06T05:16:30.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 -- start start 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86b80af540 0x7f86b80c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8159650 con 0x7f86b815a650 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b80aef20 con 0x7f86b80af540 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.951+0000 7f86bdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b80c85e0 con 0x7f86b80cd5e0 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b80b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52318/0 (socket says 172.21.15.33:52318) 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 -- 172.21.15.33:0/602655252 learned_addr learned my addr 172.21.15.33:0/602655252 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:30.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86b7fff700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bcb9b700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86b80af540 0x7f86b80c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 -- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 msgr2=0x7f86b8159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b8159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 -- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86b80af540 msgr2=0x7f86b80c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86b80af540 0x7f86b80c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.952+0000 7f86bd39c700 1 -- 172.21.15.33:0/602655252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0009a20 con 0x7f86b815a650 2023-10-06T05:16:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.953+0000 7f86bd39c700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b80b40d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86b80c7320 tx=0x7f86b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.953+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0012e60 con 0x7f86b815a650 2023-10-06T05:16:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.953+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b0027470 con 0x7f86b815a650 2023-10-06T05:16:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.953+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0027850 con 0x7f86b815a650 2023-10-06T05:16:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.953+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b80b3310 con 0x7f86b815a650 2023-10-06T05:16:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.953+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b80b38d0 con 0x7f86b815a650 2023-10-06T05:16:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.954+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b804c380 con 0x7f86b815a650 2023-10-06T05:16:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.955+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86b0024070 con 0x7f86b815a650 2023-10-06T05:16:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.956+0000 7f86b4ff9700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f869c066f30 0x7f869c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.956+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f86b0086d70 con 0x7f86b815a650 2023-10-06T05:16:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.957+0000 7f86bcb9b700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f869c066f30 0x7f869c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.957+0000 7f86bcb9b700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f869c066f30 0x7f869c0693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86ac007160 tx=0x7f86ac007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:30.961+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86b0053e40 con 0x7f86b815a650 2023-10-06T05:16:31.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.210+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86b815db70 con 0x7f86b815a650 2023-10-06T05:16:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.716+0000 7f86b4ff9700 1 -- 172.21.15.33:0/602655252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) v1 ==== 173+0+0 (secure 0 0 0) 0x7f86b002c0c0 con 0x7f86b815a650 2023-10-06T05:16:31.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.730+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f869c066f30 msgr2=0x7f869c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.730+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f869c066f30 0x7f869c0693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86ac007160 tx=0x7f86ac007f50 comp rx=0 tx=0).stop 2023-10-06T05:16:31.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.730+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 msgr2=0x7f86b80b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.730+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b80b40d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86b80c7320 tx=0x7f86b0019000 comp rx=0 tx=0).stop 2023-10-06T05:16:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.734+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 shutdown_connections 2023-10-06T05:16:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.734+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f869c066f30 0x7f869c0693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.734+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b815a650 0x7f86b80b40d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.734+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86b80af540 0x7f86b80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.734+0000 7f86bdb9d700 1 --2- 172.21.15.33:0/602655252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86b80cd5e0 0x7f86b8159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.734+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 >> 172.21.15.33:0/602655252 conn(0x7f86b806d9b0 msgr2=0x7f86b80cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.738+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 shutdown_connections 2023-10-06T05:16:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.739+0000 7f86bdb9d700 1 -- 172.21.15.33:0/602655252 wait complete. 2023-10-06T05:16:31.747 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T05:16:31.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:31.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.956+0000 7f931bfff700 1 Processor -- start 2023-10-06T05:16:31.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.956+0000 7f931bfff700 1 -- start start 2023-10-06T05:16:31.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.957+0000 7f931bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f931c0afdd0 0x7f931c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.957+0000 7f931bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c06f220 con 0x7f931c159530 2023-10-06T05:16:31.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.957+0000 7f931bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c06e610 con 0x7f931c06e140 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.957+0000 7f931bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c0b1e00 con 0x7f931c0afdd0 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f931c159530 0x7f931c06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58710/0 (socket says 172.21.15.33:58710) 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931affd700 1 -- 172.21.15.33:0/4121318388 learned_addr learned my addr 172.21.15.33:0/4121318388 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931b7fe700 1 --2- 172.21.15.33:0/4121318388 >> v2:172.21.15.33:3300/0 conn(0x7f931c0afdd0 0x7f931c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f9319ffb700 1 -- 172.21.15.33:0/4121318388 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732632973 0 0) 0x7f931c06f220 con 0x7f931c159530 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f9319ffb700 1 -- 172.21.15.33:0/4121318388 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9304003570 con 0x7f931c159530 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931b7fe700 1 -- 172.21.15.33:0/4121318388 >> v1:172.21.15.167:6789/0 conn(0x7f931c06e140 legacy=0x7f931c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931b7fe700 1 -- 172.21.15.33:0/4121318388 >> v1:172.21.15.33:6789/0 conn(0x7f931c159530 legacy=0x7f931c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931b7fe700 1 -- 172.21.15.33:0/4121318388 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c06e8f0 con 0x7f931c0afdd0 2023-10-06T05:16:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.958+0000 7f931b7fe700 1 --2- 172.21.15.33:0/4121318388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0afdd0 0x7f931c0b01a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9314009e60 tx=0x7f9314009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d13cc9209693e46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f9319ffb700 1 -- 172.21.15.33:0/4121318388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400e070 con 0x7f931c0afdd0 2023-10-06T05:16:31.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f9319ffb700 1 -- 172.21.15.33:0/4121318388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400bb60 con 0x7f931c0afdd0 2023-10-06T05:16:31.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f9319ffb700 1 -- 172.21.15.33:0/4121318388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314012430 con 0x7f931c0afdd0 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 -- 172.21.15.33:0/4121318388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0afdd0 msgr2=0x7f931c0b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 --2- 172.21.15.33:0/4121318388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0afdd0 0x7f931c0b01a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9314009e60 tx=0x7f9314009780 comp rx=0 tx=0).stop 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 -- 172.21.15.33:0/4121318388 shutdown_connections 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 --2- 172.21.15.33:0/4121318388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c0afdd0 0x7f931c0b01a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 -- 172.21.15.33:0/4121318388 >> 172.21.15.33:0/4121318388 conn(0x7f931c06c560 msgr2=0x7f931c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 -- 172.21.15.33:0/4121318388 shutdown_connections 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.959+0000 7f931bfff700 1 -- 172.21.15.33:0/4121318388 wait complete. 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.961+0000 7f931bfff700 1 Processor -- start 2023-10-06T05:16:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.961+0000 7f931bfff700 1 -- start start 2023-10-06T05:16:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f931c06e140 0x7f931c149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f931c0afdd0 0x7f931c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 0x7f931c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c153610 con 0x7f931c159530 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0cd030 con 0x7f931c06e140 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0c8d00 con 0x7f931c0afdd0 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 0x7f931c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 0x7f931c0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52338/0 (socket says 172.21.15.33:52338) 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931b7fe700 1 -- 172.21.15.33:0/260696826 learned_addr learned my addr 172.21.15.33:0/260696826 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931affd700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f931c06e140 0x7f931c149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.962+0000 7f931a7fc700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f931c0afdd0 0x7f931c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931b7fe700 1 -- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f931c0afdd0 msgr2=0x7f931c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931b7fe700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f931c0afdd0 0x7f931c0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931b7fe700 1 -- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f931c06e140 msgr2=0x7f931c149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931b7fe700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f931c06e140 0x7f931c149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931b7fe700 1 -- 172.21.15.33:0/260696826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009a20 con 0x7f931c159530 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931a7fc700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f931c0afdd0 0x7f931c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931b7fe700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 0x7f931c0cdc20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f931400b6c0 tx=0x7f9314019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314012e30 con 0x7f931c159530 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931401d630 con 0x7f931c159530 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931401d9f0 con 0x7f931c159530 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0ccd50 con 0x7f931c159530 2023-10-06T05:16:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.963+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0c8990 con 0x7f931c159530 2023-10-06T05:16:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.964+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c158dc0 con 0x7f931c159530 2023-10-06T05:16:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.965+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9314024070 con 0x7f931c159530 2023-10-06T05:16:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.967+0000 7f9302ffd700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9308066f30 0x7f93080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.967+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9314086f10 con 0x7f931c159530 2023-10-06T05:16:31.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.968+0000 7f931affd700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9308066f30 0x7f93080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.968+0000 7f931affd700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9308066f30 0x7f93080693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f930c0058d0 tx=0x7f930c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:31.971+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9314053f80 con 0x7f931c159530 2023-10-06T05:16:32.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.173+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f931c0c8e40 con 0x7f931c159530 2023-10-06T05:16:32.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.174+0000 7f9302ffd700 1 -- 172.21.15.33:0/260696826 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16881 (secure 0 0 0) 0x7f93140575a0 con 0x7f931c159530 2023-10-06T05:16:32.176 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:32.176 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":61,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:31.211370+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:29.379192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:30.188919+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9308066f30 msgr2=0x7f93080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9308066f30 0x7f93080693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f930c0058d0 tx=0x7f930c009040 comp rx=0 tx=0).stop 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 msgr2=0x7f931c0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 0x7f931c0cdc20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f931400b6c0 tx=0x7f9314019000 comp rx=0 tx=0).stop 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 shutdown_connections 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9308066f30 0x7f93080693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f931c159530 0x7f931c0cdc20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f931c06e140 0x7f931c149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 --2- 172.21.15.33:0/260696826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f931c0afdd0 0x7f931c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 >> 172.21.15.33:0/260696826 conn(0x7f931c06c560 msgr2=0x7f931c153940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 shutdown_connections 2023-10-06T05:16:32.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.177+0000 7f931bfff700 1 -- 172.21.15.33:0/260696826 wait complete. 2023-10-06T05:16:32.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.382+0000 7fcfd279d700 1 Processor -- start 2023-10-06T05:16:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.382+0000 7fcfd279d700 1 -- start start 2023-10-06T05:16:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.383+0000 7fcfd279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc155140 0x7fcfcc158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.383+0000 7fcfd279d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc0c8e40 con 0x7fcfcc0699c0 2023-10-06T05:16:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.383+0000 7fcfd279d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc0c99e0 con 0x7fcfcc0adf50 2023-10-06T05:16:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.383+0000 7fcfd279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0ae6c0 con 0x7fcfcc155140 2023-10-06T05:16:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.383+0000 7fcfd1f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcfcc0699c0 0x7fcfcc0c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58712/0 (socket says 172.21.15.33:58712) 2023-10-06T05:16:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.383+0000 7fcfd1f9c700 1 -- 172.21.15.33:0/2338482728 learned_addr learned my addr 172.21.15.33:0/2338482728 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfd179b700 1 --2- 172.21.15.33:0/2338482728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc155140 0x7fcfcc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463109480 0 0) 0x7fcfcc0c8e40 con 0x7fcfcc0699c0 2023-10-06T05:16:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb8003570 con 0x7fcfcc0699c0 2023-10-06T05:16:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076518849 0 0) 0x7fcfcc0c99e0 con 0x7fcfcc0adf50 2023-10-06T05:16:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfcc0c8e40 con 0x7fcfcc0adf50 2023-10-06T05:16:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242119519 0 0) 0x7fcfb8003570 con 0x7fcfcc0699c0 2023-10-06T05:16:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfd179b700 1 -- 172.21.15.33:0/2338482728 >> v1:172.21.15.167:6789/0 conn(0x7fcfcc0adf50 legacy=0x7fcfcc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfd179b700 1 -- 172.21.15.33:0/2338482728 >> v1:172.21.15.33:6789/0 conn(0x7fcfcc0699c0 legacy=0x7fcfcc0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.384+0000 7fcfd179b700 1 -- 172.21.15.33:0/2338482728 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0ae9a0 con 0x7fcfcc155140 2023-10-06T05:16:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.385+0000 7fcfd179b700 1 --2- 172.21.15.33:0/2338482728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc155140 0x7fcfcc158700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009910 tx=0x7fcfc80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97e7a0e57077f27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:32.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.385+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40017f0 con 0x7fcfcc155140 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.385+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc8011420 con 0x7fcfcc155140 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.385+0000 7fcfc3fff700 1 -- 172.21.15.33:0/2338482728 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80117c0 con 0x7fcfcc155140 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.385+0000 7fcfd279d700 1 -- 172.21.15.33:0/2338482728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc155140 msgr2=0x7fcfcc158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.385+0000 7fcfd279d700 1 --2- 172.21.15.33:0/2338482728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc155140 0x7fcfcc158700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009910 tx=0x7fcfc80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.386+0000 7fcfd279d700 1 -- 172.21.15.33:0/2338482728 shutdown_connections 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.386+0000 7fcfd279d700 1 --2- 172.21.15.33:0/2338482728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc155140 0x7fcfcc158700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.386+0000 7fcfd279d700 1 -- 172.21.15.33:0/2338482728 >> 172.21.15.33:0/2338482728 conn(0x7fcfcc06d030 msgr2=0x7fcfcc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.386+0000 7fcfd279d700 1 -- 172.21.15.33:0/2338482728 shutdown_connections 2023-10-06T05:16:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.386+0000 7fcfd279d700 1 -- 172.21.15.33:0/2338482728 wait complete. 2023-10-06T05:16:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.387+0000 7fcfd279d700 1 Processor -- start 2023-10-06T05:16:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.387+0000 7fcfd279d700 1 -- start start 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.388+0000 7fcfd279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0699c0 0x7fcfcc0c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.388+0000 7fcfd279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0adf50 0x7fcfcc0c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.388+0000 7fcfd279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 0x7fcfcc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.388+0000 7fcfd279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0b00c0 con 0x7fcfcc155140 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.388+0000 7fcfd279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0af4b0 con 0x7fcfcc0adf50 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.388+0000 7fcfd279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0af790 con 0x7fcfcc0699c0 2023-10-06T05:16:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 0x7fcfcc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 0x7fcfcc15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52342/0 (socket says 172.21.15.33:52342) 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 -- 172.21.15.33:0/3873803702 learned_addr learned my addr 172.21.15.33:0/3873803702 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd179b700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0699c0 0x7fcfcc0c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd0f9a700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0adf50 0x7fcfcc0c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 -- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0699c0 msgr2=0x7fcfcc0c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0699c0 0x7fcfcc0c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 -- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0adf50 msgr2=0x7fcfcc0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0adf50 0x7fcfcc0c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 -- 172.21.15.33:0/3873803702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8009420 con 0x7fcfcc155140 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.389+0000 7fcfd1f9c700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 0x7fcfcc15e430 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4001560 tx=0x7fcfc4007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.390+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004dc0 con 0x7fcfcc155140 2023-10-06T05:16:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.390+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc400a710 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.390+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400aad0 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.390+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0afa70 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.390+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc0aefb0 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.391+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc0af230 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.392+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfc4007150 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.393+0000 7fcfc17fa700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfb0066f30 0x7fcfb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.393+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcfc4084200 con 0x7fcfcc155140 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.394+0000 7fcfd179b700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfb0066f30 0x7fcfb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.394+0000 7fcfd179b700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfb0066f30 0x7fcfb00693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80030c0 tx=0x7fcfc8016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.397+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfc40512f0 con 0x7fcfcc155140 2023-10-06T05:16:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.600+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfcc04c380 con 0x7fcfcc155140 2023-10-06T05:16:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.601+0000 7fcfc17fa700 1 -- 172.21.15.33:0/3873803702 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16881 (secure 0 0 0) 0x7fcfc4054910 con 0x7fcfcc155140 2023-10-06T05:16:32.603 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:32.603 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":61,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:31.211370+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:29.379192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:30.188919+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfb0066f30 msgr2=0x7fcfb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfb0066f30 0x7fcfb00693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80030c0 tx=0x7fcfc8016000 comp rx=0 tx=0).stop 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 msgr2=0x7fcfcc15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 0x7fcfcc15e430 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4001560 tx=0x7fcfc4007d20 comp rx=0 tx=0).stop 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 shutdown_connections 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfb0066f30 0x7fcfb00693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfcc155140 0x7fcfcc15e430 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0699c0 0x7fcfcc0c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 --2- 172.21.15.33:0/3873803702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0adf50 0x7fcfcc0c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 >> 172.21.15.33:0/3873803702 conn(0x7fcfcc06d030 msgr2=0x7fcfcc150d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 shutdown_connections 2023-10-06T05:16:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.603+0000 7fcfd279d700 1 -- 172.21.15.33:0/3873803702 wait complete. 2023-10-06T05:16:32.622 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:16:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.816+0000 7f187759e700 1 Processor -- start 2023-10-06T05:16:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.816+0000 7f187759e700 1 -- start start 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1878159850 0x7f187806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18780b0cd0 con 0x7f187815acd0 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187815a840 con 0x7f187806d140 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187806e490 con 0x7f1878159850 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1878159850 0x7f187806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1878159850 0x7f187806e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52358/0 (socket says 172.21.15.33:52358) 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f1875d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f187815acd0 0x7f187806d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58722/0 (socket says 172.21.15.33:58722) 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187659c700 1 -- 172.21.15.33:0/3184100212 learned_addr learned my addr 172.21.15.33:0/3184100212 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187659c700 1 -- 172.21.15.33:0/3184100212 >> v1:172.21.15.167:6789/0 conn(0x7f187806d140 legacy=0x7f187806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187659c700 1 -- 172.21.15.33:0/3184100212 >> v1:172.21.15.33:6789/0 conn(0x7f187815acd0 legacy=0x7f187806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187659c700 1 -- 172.21.15.33:0/3184100212 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878159f10 con 0x7f1878159850 2023-10-06T05:16:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.818+0000 7f187559a700 1 -- 172.21.15.33:0/3184100212 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779065068 0 0) 0x7f18780b0cd0 con 0x7f187815acd0 2023-10-06T05:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.819+0000 7f187659c700 1 --2- 172.21.15.33:0/3184100212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1878159850 0x7f187806e890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18680099b0 tx=0x7f1868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a2fd89d0dd5107 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.819+0000 7f187559a700 1 -- 172.21.15.33:0/3184100212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186800b820 con 0x7f1878159850 2023-10-06T05:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.819+0000 7f187559a700 1 -- 172.21.15.33:0/3184100212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186800b980 con 0x7f1878159850 2023-10-06T05:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.819+0000 7f187559a700 1 -- 172.21.15.33:0/3184100212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186800bd20 con 0x7f1878159850 2023-10-06T05:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.819+0000 7f187759e700 1 -- 172.21.15.33:0/3184100212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1878159850 msgr2=0x7f187806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.819+0000 7f187759e700 1 --2- 172.21.15.33:0/3184100212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1878159850 0x7f187806e890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18680099b0 tx=0x7f1868009250 comp rx=0 tx=0).stop 2023-10-06T05:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.820+0000 7f187759e700 1 -- 172.21.15.33:0/3184100212 shutdown_connections 2023-10-06T05:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.820+0000 7f187759e700 1 --2- 172.21.15.33:0/3184100212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1878159850 0x7f187806e890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.820+0000 7f187759e700 1 -- 172.21.15.33:0/3184100212 >> 172.21.15.33:0/3184100212 conn(0x7f187806cd40 msgr2=0x7f18780ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.820+0000 7f187759e700 1 -- 172.21.15.33:0/3184100212 shutdown_connections 2023-10-06T05:16:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.820+0000 7f187759e700 1 -- 172.21.15.33:0/3184100212 wait complete. 2023-10-06T05:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.821+0000 7f187759e700 1 Processor -- start 2023-10-06T05:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.821+0000 7f187759e700 1 -- start start 2023-10-06T05:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.822+0000 7f187759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187806d140 0x7f18780cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.822+0000 7f187759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1878159850 0x7f18780b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.822+0000 7f187759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 0x7f187814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.822+0000 7f187759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780ac8f0 con 0x7f187815acd0 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.822+0000 7f187759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780b3ed0 con 0x7f1878159850 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.822+0000 7f187759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18780b4170 con 0x7f187806d140 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 0x7f187814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 0x7f187814dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52370/0 (socket says 172.21.15.33:52370) 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 -- 172.21.15.33:0/644881505 learned_addr learned my addr 172.21.15.33:0/644881505 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f187659c700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187806d140 0x7f18780cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1875d9b700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1878159850 0x7f18780b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 -- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187806d140 msgr2=0x7f18780cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187806d140 0x7f18780cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 -- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1878159850 msgr2=0x7f18780b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1878159850 0x7f18780b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 -- 172.21.15.33:0/644881505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18680094f0 con 0x7f187815acd0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.823+0000 7f1876d9d700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 0x7f187814dfc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f187000bb60 tx=0x7f18700097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.824+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187000c7e0 con 0x7f187815acd0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.824+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187000c940 con 0x7f187815acd0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.824+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18780cd190 con 0x7f187815acd0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.824+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187000cd00 con 0x7f187815acd0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.824+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18780cd750 con 0x7f187815acd0 2023-10-06T05:16:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.825+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18780aca60 con 0x7f187815acd0 2023-10-06T05:16:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.826+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1870014620 con 0x7f187815acd0 2023-10-06T05:16:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.827+0000 7f18667fc700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1858066f30 0x7f18580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.827+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1870084580 con 0x7f187815acd0 2023-10-06T05:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.827+0000 7f187659c700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1858066f30 0x7f18580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.828+0000 7f187659c700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1858066f30 0x7f18580693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f18680099b0 tx=0x7f1868016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:32.831+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f187004c5a0 con 0x7f187815acd0 2023-10-06T05:16:33.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.079+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f187804c380 con 0x7f187815acd0 2023-10-06T05:16:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.746+0000 7f18667fc700 1 -- 172.21.15.33:0/644881505 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1870055120 con 0x7f187815acd0 2023-10-06T05:16:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1858066f30 msgr2=0x7f18580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1858066f30 0x7f18580693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f18680099b0 tx=0x7f1868016000 comp rx=0 tx=0).stop 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 msgr2=0x7f187814dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 0x7f187814dfc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f187000bb60 tx=0x7f18700097f0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 shutdown_connections 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1858066f30 0x7f18580693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187815acd0 0x7f187814dfc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187806d140 0x7f18780cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 --2- 172.21.15.33:0/644881505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1878159850 0x7f18780b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 >> 172.21.15.33:0/644881505 conn(0x7f187806cd40 msgr2=0x7f187815b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 shutdown_connections 2023-10-06T05:16:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.749+0000 7f187759e700 1 -- 172.21.15.33:0/644881505 wait complete. 2023-10-06T05:16:33.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:16:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.963+0000 7fd8fbfff700 1 Processor -- start 2023-10-06T05:16:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.963+0000 7fd8fbfff700 1 -- start start 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.964+0000 7fd8fbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd8fc156370 0x7fd8fc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.964+0000 7fd8fbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8fc0b01f0 0x7fd8fc0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.964+0000 7fd8fbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0ad430 con 0x7fd8fc0cc7e0 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.964+0000 7fd8fbfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc0afe20 con 0x7fd8fc0b01f0 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.964+0000 7fd8fbfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc0b0b70 con 0x7fd8fc156370 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8fc0b01f0 0x7fd8fc0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8fc0b01f0 0x7fd8fc0b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52378/0 (socket says 172.21.15.33:52378) 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/177295531 learned_addr learned my addr 172.21.15.33:0/177295531 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8faffd700 1 --2- 172.21.15.33:0/177295531 >> v2:172.21.15.167:3300/0 conn(0x7fd8fc156370 0x7fd8fc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:33.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/177295531 >> v2:172.21.15.167:3300/0 conn(0x7fd8fc156370 msgr2=0x7fd8fc0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:33.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 --2- 172.21.15.33:0/177295531 >> v2:172.21.15.167:3300/0 conn(0x7fd8fc156370 0x7fd8fc0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/177295531 >> v1:172.21.15.167:6789/0 conn(0x7fd8fc0cc7e0 legacy=0x7fd8fc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:33.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/177295531 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc06bc70 con 0x7fd8fc0b01f0 2023-10-06T05:16:33.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.965+0000 7fd8faffd700 1 --2- 172.21.15.33:0/177295531 >> v2:172.21.15.167:3300/0 conn(0x7fd8fc156370 0x7fd8fc0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:33.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fb7fe700 1 --2- 172.21.15.33:0/177295531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc0b01f0 0x7fd8fc0b0630 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0b21a0 tx=0x7fd8f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529c2e28ccc5ae30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:33.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8f9ffb700 1 -- 172.21.15.33:0/177295531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400e070 con 0x7fd8fc0b01f0 2023-10-06T05:16:33.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8f9ffb700 1 -- 172.21.15.33:0/177295531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f400ba10 con 0x7fd8fc0b01f0 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8f9ffb700 1 -- 172.21.15.33:0/177295531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40092e0 con 0x7fd8fc0b01f0 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fbfff700 1 -- 172.21.15.33:0/177295531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc0b01f0 msgr2=0x7fd8fc0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/177295531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc0b01f0 0x7fd8fc0b0630 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0b21a0 tx=0x7fd8f4009840 comp rx=0 tx=0).stop 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fbfff700 1 -- 172.21.15.33:0/177295531 shutdown_connections 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/177295531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc0b01f0 0x7fd8fc0b0630 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/177295531 >> v2:172.21.15.167:3300/0 conn(0x7fd8fc156370 0x7fd8fc0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.966+0000 7fd8fbfff700 1 -- 172.21.15.33:0/177295531 >> 172.21.15.33:0/177295531 conn(0x7fd8fc06d030 msgr2=0x7fd8fc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.967+0000 7fd8fbfff700 1 -- 172.21.15.33:0/177295531 shutdown_connections 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.967+0000 7fd8fbfff700 1 -- 172.21.15.33:0/177295531 wait complete. 2023-10-06T05:16:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.968+0000 7fd8fbfff700 1 Processor -- start 2023-10-06T05:16:33.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.968+0000 7fd8fbfff700 1 -- start start 2023-10-06T05:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc0b01f0 0x7fd8fc0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0cc7e0 0x7fd8fc1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 0x7fd8fc150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc1553c0 con 0x7fd8fc156370 2023-10-06T05:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc1547b0 con 0x7fd8fc0cc7e0 2023-10-06T05:16:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc154a90 con 0x7fd8fc0b01f0 2023-10-06T05:16:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 0x7fd8fc150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 0x7fd8fc150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52382/0 (socket says 172.21.15.33:52382) 2023-10-06T05:16:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.969+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/2821329360 learned_addr learned my addr 172.21.15.33:0/2821329360 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8faffd700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc0b01f0 0x7fd8fc0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fa7fc700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0cc7e0 0x7fd8fc1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:33.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc0b01f0 msgr2=0x7fd8fc0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fb7fe700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc0b01f0 0x7fd8fc0c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0cc7e0 msgr2=0x7fd8fc1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fb7fe700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0cc7e0 0x7fd8fc1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fb7fe700 1 -- 172.21.15.33:0/2821329360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f4009ae0 con 0x7fd8fc156370 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8faffd700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc0b01f0 0x7fd8fc0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fa7fc700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0cc7e0 0x7fd8fc1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fb7fe700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 0x7fd8fc150e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc06d4d0 tx=0x7fd8f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40032f0 con 0x7fd8fc156370 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4012e60 con 0x7fd8fc156370 2023-10-06T05:16:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f401d670 con 0x7fd8fc156370 2023-10-06T05:16:33.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc154d70 con 0x7fd8fc156370 2023-10-06T05:16:33.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.970+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc151390 con 0x7fd8fc156370 2023-10-06T05:16:33.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.971+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc04c380 con 0x7fd8fc156370 2023-10-06T05:16:33.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.972+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8f401daa0 con 0x7fd8fc156370 2023-10-06T05:16:33.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.973+0000 7fd8e2ffd700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:33.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.973+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8f4085ff0 con 0x7fd8fc156370 2023-10-06T05:16:33.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.974+0000 7fd8faffd700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:33.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.974+0000 7fd8faffd700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8e8066f30 0x7fd8e80693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec008b20 tx=0x7fd8ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:33.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:33.977+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8f40b75a0 con 0x7fd8fc156370 2023-10-06T05:16:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.227+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd8fc155ba0 con 0x7fd8fc156370 2023-10-06T05:16:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.754+0000 7fd8e2ffd700 1 -- 172.21.15.33:0/2821329360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8f4053080 con 0x7fd8fc156370 2023-10-06T05:16:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8e8066f30 msgr2=0x7fd8e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8e8066f30 0x7fd8e80693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec008b20 tx=0x7fd8ec008040 comp rx=0 tx=0).stop 2023-10-06T05:16:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 msgr2=0x7fd8fc150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 0x7fd8fc150e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc06d4d0 tx=0x7fd8f4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 shutdown_connections 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8fc156370 0x7fd8fc150e50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc0b01f0 0x7fd8fc0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 --2- 172.21.15.33:0/2821329360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0cc7e0 0x7fd8fc1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 >> 172.21.15.33:0/2821329360 conn(0x7fd8fc06d030 msgr2=0x7fd8fc0abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 shutdown_connections 2023-10-06T05:16:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.757+0000 7fd8fbfff700 1 -- 172.21.15.33:0/2821329360 wait complete. 2023-10-06T05:16:34.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:34.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.965+0000 7f9908b9d700 1 Processor -- start 2023-10-06T05:16:34.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.965+0000 7f9908b9d700 1 -- start start 2023-10-06T05:16:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f9908b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9904153b10 0x7f99040c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f9908b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040c6b30 0x7f99040c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f9908b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990415abd0 con 0x7f99040675f0 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f9908b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040c74f0 con 0x7f9904153b10 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f9908b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040c9b00 con 0x7f99040c6b30 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f99037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9904153b10 0x7f99040c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f99037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9904153b10 0x7f99040c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52390/0 (socket says 172.21.15.33:52390) 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f99037fe700 1 -- 172.21.15.33:0/1986695399 learned_addr learned my addr 172.21.15.33:0/1986695399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f9902ffd700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040c6b30 0x7f99040c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f99037fe700 1 -- 172.21.15.33:0/1986695399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040c6b30 msgr2=0x7f99040c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f99037fe700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040c6b30 0x7f99040c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.967+0000 7f99037fe700 1 -- 172.21.15.33:0/1986695399 >> v1:172.21.15.167:6789/0 conn(0x7f99040675f0 legacy=0x7f9904067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.968+0000 7f99037fe700 1 -- 172.21.15.33:0/1986695399 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990415a050 con 0x7f9904153b10 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.968+0000 7f9902ffd700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040c6b30 0x7f99040c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.968+0000 7f99037fe700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9904153b10 0x7f99040c95c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f98f40099b0 tx=0x7f98f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64980c89625f6a07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.968+0000 7f99027fc700 1 -- 172.21.15.33:0/1986695399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400b860 con 0x7f9904153b10 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.968+0000 7f99027fc700 1 -- 172.21.15.33:0/1986695399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400b9c0 con 0x7f9904153b10 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.968+0000 7f99027fc700 1 -- 172.21.15.33:0/1986695399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400bd60 con 0x7f9904153b10 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 -- 172.21.15.33:0/1986695399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9904153b10 msgr2=0x7f99040c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9904153b10 0x7f99040c95c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f98f40099b0 tx=0x7f98f4009250 comp rx=0 tx=0).stop 2023-10-06T05:16:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 -- 172.21.15.33:0/1986695399 shutdown_connections 2023-10-06T05:16:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9904153b10 0x7f99040c95c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1986695399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040c6b30 0x7f99040c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 -- 172.21.15.33:0/1986695399 >> 172.21.15.33:0/1986695399 conn(0x7f990406c560 msgr2=0x7f990406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 -- 172.21.15.33:0/1986695399 shutdown_connections 2023-10-06T05:16:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.969+0000 7f9908b9d700 1 -- 172.21.15.33:0/1986695399 wait complete. 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.970+0000 7f9908b9d700 1 Processor -- start 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.970+0000 7f9908b9d700 1 -- start start 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9908b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040675f0 0x7f9904169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9908b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 0x7f99040ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9908b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9904153b10 0x7f99040b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9908b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040c44f0 con 0x7f99040c6b30 2023-10-06T05:16:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9908b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040b2200 con 0x7f9904153b10 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9908b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040b1f20 con 0x7f99040675f0 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.971+0000 7f9902ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 0x7f99040ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 0x7f99040ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52394/0 (socket says 172.21.15.33:52394) 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 -- 172.21.15.33:0/1188792090 learned_addr learned my addr 172.21.15.33:0/1188792090 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f99037fe700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040675f0 0x7f9904169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9903fff700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9904153b10 0x7f99040b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 -- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040675f0 msgr2=0x7f9904169f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040675f0 0x7f9904169f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 -- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9904153b10 msgr2=0x7f99040b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9904153b10 0x7f99040b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 -- 172.21.15.33:0/1188792090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40094f0 con 0x7f99040c6b30 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9902ffd700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 0x7f99040ade80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f98f800d890 tx=0x7f98f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f800b600 con 0x7f99040c6b30 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f800b760 con 0x7f99040c6b30 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f800bb00 con 0x7f99040c6b30 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.972+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99040c3ea0 con 0x7f99040c6b30 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.973+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990416a4a0 con 0x7f99040c6b30 2023-10-06T05:16:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.974+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99040c46a0 con 0x7f99040c6b30 2023-10-06T05:16:34.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.974+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98f8014430 con 0x7f99040c6b30 2023-10-06T05:16:34.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.975+0000 7f98eb7fe700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f0066f30 0x7f98f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.976+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98f8084b30 con 0x7f99040c6b30 2023-10-06T05:16:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.976+0000 7f99037fe700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f0066f30 0x7f98f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.977+0000 7f99037fe700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f0066f30 0x7f98f00693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98f40099b0 tx=0x7f98f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:34.979+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99040c46a0 con 0x7f99040c6b30 2023-10-06T05:16:35.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.224+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f990404c380 con 0x7f99040c6b30 2023-10-06T05:16:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.225+0000 7f98eb7fe700 1 -- 172.21.15.33:0/1188792090 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f98f8051ba0 con 0x7f99040c6b30 2023-10-06T05:16:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:35.227 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f0066f30 msgr2=0x7f98f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f0066f30 0x7f98f00693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98f40099b0 tx=0x7f98f4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 msgr2=0x7f99040ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 0x7f99040ade80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f98f800d890 tx=0x7f98f8008040 comp rx=0 tx=0).stop 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 shutdown_connections 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f0066f30 0x7f98f00693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9904153b10 0x7f99040b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99040675f0 0x7f9904169f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 --2- 172.21.15.33:0/1188792090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c6b30 0x7f99040ade80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 >> 172.21.15.33:0/1188792090 conn(0x7f990406c560 msgr2=0x7f99041533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 shutdown_connections 2023-10-06T05:16:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.227+0000 7f9908b9d700 1 -- 172.21.15.33:0/1188792090 wait complete. 2023-10-06T05:16:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:16:35.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:35.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.431+0000 7fe1bb59e700 1 Processor -- start 2023-10-06T05:16:35.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.432+0000 7fe1bb59e700 1 -- start start 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1bb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1bc13faa0 0x7fe1bc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1bb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 0x7fe1bc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1bb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0c9cb0 con 0x7fe1bc06aad0 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1bb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc06b660 con 0x7fe1bc13faa0 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1bb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0ca270 con 0x7fe1bc0c7940 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1b9d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 0x7fe1bc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1ba59c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1bc13faa0 0x7fe1bc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1b9d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 0x7fe1bc0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:37716/0 (socket says 172.21.15.33:37716) 2023-10-06T05:16:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1714419577 learned_addr learned my addr 172.21.15.33:0/1714419577 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.433+0000 7fe1bad9d700 1 --1- 172.21.15.33:0/1714419577 >> v1:172.21.15.167:6789/0 conn(0x7fe1bc06aad0 0x7fe1bc06af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:52606/0 (socket says 172.21.15.33:52606) 2023-10-06T05:16:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.434+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1714419577 >> v2:172.21.15.167:3300/0 conn(0x7fe1bc13faa0 msgr2=0x7fe1bc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.434+0000 7fe1b9d9b700 1 --2- 172.21.15.33:0/1714419577 >> v2:172.21.15.167:3300/0 conn(0x7fe1bc13faa0 0x7fe1bc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.434+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1714419577 >> v1:172.21.15.167:6789/0 conn(0x7fe1bc06aad0 legacy=0x7fe1bc06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.434+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1714419577 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc06b840 con 0x7fe1bc0c7940 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.434+0000 7fe1b9d9b700 1 --2- 172.21.15.33:0/1714419577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 0x7fe1bc0c7db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00099b0 tx=0x7fe1b0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704627ae5a977f5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.434+0000 7fe1b959a700 1 -- 172.21.15.33:0/1714419577 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0009ea0 con 0x7fe1bc0c7940 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1b959a700 1 -- 172.21.15.33:0/1714419577 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000b860 con 0x7fe1bc0c7940 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1b959a700 1 -- 172.21.15.33:0/1714419577 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000bc00 con 0x7fe1bc0c7940 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1714419577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 msgr2=0x7fe1bc0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1714419577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 0x7fe1bc0c7db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00099b0 tx=0x7fe1b0009250 comp rx=0 tx=0).stop 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1714419577 shutdown_connections 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1714419577 >> v2:172.21.15.167:3300/0 conn(0x7fe1bc13faa0 0x7fe1bc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1714419577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc0c7940 0x7fe1bc0c7db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1714419577 >> 172.21.15.33:0/1714419577 conn(0x7fe1bc06d030 msgr2=0x7fe1bc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1714419577 shutdown_connections 2023-10-06T05:16:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.435+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1714419577 wait complete. 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.437+0000 7fe1bb59e700 1 Processor -- start 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.437+0000 7fe1bb59e700 1 -- start start 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc06aad0 0x7fe1bc13ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 0x7fe1bc0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1bc13faa0 0x7fe1bc158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc16dc60 con 0x7fe1bc0c7940 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc16d050 con 0x7fe1bc13faa0 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc16d330 con 0x7fe1bc06aad0 2023-10-06T05:16:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 0x7fe1bc0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1ba59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc06aad0 0x7fe1bc13ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 0x7fe1bc0b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52408/0 (socket says 172.21.15.33:52408) 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1266460478 learned_addr learned my addr 172.21.15.33:0/1266460478 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1bad9d700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1bc13faa0 0x7fe1bc158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc06aad0 msgr2=0x7fe1bc13ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc06aad0 0x7fe1bc13ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1bc13faa0 msgr2=0x7fe1bc158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1bc13faa0 0x7fe1bc158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.438+0000 7fe1b9d9b700 1 -- 172.21.15.33:0/1266460478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00094f0 con 0x7fe1bc0c7940 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.439+0000 7fe1b9d9b700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 0x7fe1bc0b16a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009290 tx=0x7fe1b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.439+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000b9d0 con 0x7fe1bc0c7940 2023-10-06T05:16:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.439+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000bb70 con 0x7fe1bc0c7940 2023-10-06T05:16:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.439+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00106c0 con 0x7fe1bc0c7940 2023-10-06T05:16:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.439+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc16d610 con 0x7fe1bc0c7940 2023-10-06T05:16:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.439+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc0ae910 con 0x7fe1bc0c7940 2023-10-06T05:16:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.440+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc16de10 con 0x7fe1bc0c7940 2023-10-06T05:16:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.441+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1b002c440 con 0x7fe1bc0c7940 2023-10-06T05:16:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.442+0000 7fe1aa7fc700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe19c066f30 0x7fe19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.442+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1b0014070 con 0x7fe1bc0c7940 2023-10-06T05:16:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.443+0000 7fe1ba59c700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe19c066f30 0x7fe19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.443+0000 7fe1ba59c700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe19c066f30 0x7fe19c0693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009d10 tx=0x7fe1ac0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.447+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1bc16de10 con 0x7fe1bc0c7940 2023-10-06T05:16:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.651+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1bc04c380 con 0x7fe1bc0c7940 2023-10-06T05:16:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.652+0000 7fe1aa7fc700 1 -- 172.21.15.33:0/1266460478 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16868 (secure 0 0 0) 0x7fe1b0053050 con 0x7fe1bc0c7940 2023-10-06T05:16:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:35.656 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:16:32.722661+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:29.379192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:30.188919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe19c066f30 msgr2=0x7fe19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe19c066f30 0x7fe19c0693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009d10 tx=0x7fe1ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 msgr2=0x7fe1bc0b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 0x7fe1bc0b16a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009290 tx=0x7fe1b0019000 comp rx=0 tx=0).stop 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 shutdown_connections 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe19c066f30 0x7fe19c0693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1bc13faa0 0x7fe1bc158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1bc06aad0 0x7fe1bc13ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 --2- 172.21.15.33:0/1266460478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1bc0c7940 0x7fe1bc0b16a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 >> 172.21.15.33:0/1266460478 conn(0x7fe1bc06d030 msgr2=0x7fe1bc15a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 shutdown_connections 2023-10-06T05:16:35.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.655+0000 7fe1bb59e700 1 -- 172.21.15.33:0/1266460478 wait complete. 2023-10-06T05:16:35.675 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:16:35.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:16:35.859 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:16:35.856+0000 7fda483f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:16:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.862+0000 7f43fa79d700 1 Processor -- start 2023-10-06T05:16:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.862+0000 7f43fa79d700 1 -- start start 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.863+0000 7f43fa79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43f4159520 0x7f43f40c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.863+0000 7f43fa79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f406a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.863+0000 7f43fa79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f406c820 con 0x7f43f40c6170 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.863+0000 7f43fa79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f40c6610 con 0x7f43f4159520 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.863+0000 7f43fa79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f406a160 con 0x7f43f40c82a0 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.863+0000 7f43f979b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43f4159520 0x7f43f40c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43f4159520 0x7f43f40c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52418/0 (socket says 172.21.15.33:52418) 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 -- 172.21.15.33:0/3117497791 learned_addr learned my addr 172.21.15.33:0/3117497791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f9f9c700 1 --2- 172.21.15.33:0/3117497791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f406a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 -- 172.21.15.33:0/3117497791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 msgr2=0x7f43f406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 --2- 172.21.15.33:0/3117497791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f406a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 -- 172.21.15.33:0/3117497791 >> v1:172.21.15.167:6789/0 conn(0x7f43f40c6170 legacy=0x7f43f40c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 -- 172.21.15.33:0/3117497791 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f40c8800 con 0x7f43f4159520 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43f979b700 1 --2- 172.21.15.33:0/3117497791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f4159520 0x7f43f40c5c30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f43f00099b0 tx=0x7f43f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d57a22dd352cb0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43ebfff700 1 -- 172.21.15.33:0/3117497791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000b860 con 0x7f43f4159520 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43ebfff700 1 -- 172.21.15.33:0/3117497791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f000b9c0 con 0x7f43f4159520 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.864+0000 7f43ebfff700 1 -- 172.21.15.33:0/3117497791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000bd60 con 0x7f43f4159520 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 -- 172.21.15.33:0/3117497791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f4159520 msgr2=0x7f43f40c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 --2- 172.21.15.33:0/3117497791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f4159520 0x7f43f40c5c30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f43f00099b0 tx=0x7f43f0009250 comp rx=0 tx=0).stop 2023-10-06T05:16:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 -- 172.21.15.33:0/3117497791 shutdown_connections 2023-10-06T05:16:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 --2- 172.21.15.33:0/3117497791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f4159520 0x7f43f40c5c30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 --2- 172.21.15.33:0/3117497791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 -- 172.21.15.33:0/3117497791 >> 172.21.15.33:0/3117497791 conn(0x7f43f40cc930 msgr2=0x7f43f40ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 -- 172.21.15.33:0/3117497791 shutdown_connections 2023-10-06T05:16:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.865+0000 7f43fa79d700 1 -- 172.21.15.33:0/3117497791 wait complete. 2023-10-06T05:16:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.867+0000 7f43fa79d700 1 Processor -- start 2023-10-06T05:16:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.867+0000 7f43fa79d700 1 -- start start 2023-10-06T05:16:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43fa79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 0x7f43f414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43fa79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f40b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43fa79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f4159520 0x7f43f4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43fa79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f414ec90 con 0x7f43f40c6170 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43fa79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f40b0900 con 0x7f43f4159520 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43fa79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f414ee90 con 0x7f43f40c82a0 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 0x7f43f414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 0x7f43f414e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52428/0 (socket says 172.21.15.33:52428) 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 -- 172.21.15.33:0/1168324290 learned_addr learned my addr 172.21.15.33:0/1168324290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f8f9a700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f40b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f9f9c700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f4159520 0x7f43f4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 -- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 msgr2=0x7f43f40b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f40b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 -- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f4159520 msgr2=0x7f43f4152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f4159520 0x7f43f4152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.868+0000 7f43f979b700 1 -- 172.21.15.33:0/1168324290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00094f0 con 0x7f43f40c6170 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.869+0000 7f43f979b700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 0x7f43f414e750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f43f000b3c0 tx=0x7f43f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.869+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0011df0 con 0x7f43f40c6170 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.869+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f0024440 con 0x7f43f40c6170 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.869+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00247e0 con 0x7f43f40c6170 2023-10-06T05:16:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.869+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f41503e0 con 0x7f43f40c6170 2023-10-06T05:16:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.869+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f41509a0 con 0x7f43f40c6170 2023-10-06T05:16:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.870+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f404c380 con 0x7f43f40c6170 2023-10-06T05:16:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.871+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43f001a420 con 0x7f43f40c6170 2023-10-06T05:16:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.872+0000 7f43e97fa700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.872+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43f0086490 con 0x7f43f40c6170 2023-10-06T05:16:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.872+0000 7f43f8f9a700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.873+0000 7f43f8f9a700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43f40ca8d0 tx=0x7f43e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:35.876+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43f0053580 con 0x7f43f40c6170 2023-10-06T05:16:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.077+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43f40c8ae0 con 0x7f43d8066f30 2023-10-06T05:16:36.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.088+0000 7f43e97fa700 1 -- 172.21.15.33:0/1168324290 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310777 (secure 0 0 0) 0x7f43f40c8ae0 con 0x7f43d8066f30 2023-10-06T05:16:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.092+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43d8066f30 msgr2=0x7f43d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.092+0000 7f43fa79d700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43f40ca8d0 tx=0x7f43e4009040 comp rx=0 tx=0).stop 2023-10-06T05:16:36.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.092+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 msgr2=0x7f43f414e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.092+0000 7f43fa79d700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 0x7f43f414e750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f43f000b3c0 tx=0x7f43f0017000 comp rx=0 tx=0).stop 2023-10-06T05:16:36.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 shutdown_connections 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f4159520 0x7f43f4152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f40c6170 0x7f43f414e750 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 --2- 172.21.15.33:0/1168324290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f40c82a0 0x7f43f40b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 >> 172.21.15.33:0/1168324290 conn(0x7f43f40cc930 msgr2=0x7f43f4151a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 shutdown_connections 2023-10-06T05:16:36.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.093+0000 7f43fa79d700 1 -- 172.21.15.33:0/1168324290 wait complete. 2023-10-06T05:16:36.100 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:16:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-10-06T05:16:35.845649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389208,"kb_used_data":6808,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752585512,"statfs":{"total":773094113280,"available":770647564288,"internally_reserved":0,"allocated":6971392,"data_stored":3695066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001255"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743090+0000","last_change":"2023-10-06T05:16:31.743090+0000","last_active":"2023-10-06T05:16:31.743090+0000","last_peered":"2023-10-06T05:16:31.743090+0000","last_clean":"2023-10-06T05:16:31.743090+0000","last_became_active":"2023-10-06T05:16:31.742824+0000","last_became_peered":"2023-10-06T05:16:31.742824+0000","last_unstale":"2023-10-06T05:16:31.743090+0000","last_undegraded":"2023-10-06T05:16:31.743090+0000","last_fullsized":"2023-10-06T05:16:31.743090+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:51.707463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741207+0000","last_change":"2023-10-06T05:16:31.741207+0000","last_active":"2023-10-06T05:16:31.741207+0000","last_peered":"2023-10-06T05:16:31.741207+0000","last_clean":"2023-10-06T05:16:31.741207+0000","last_became_active":"2023-10-06T05:16:31.740896+0000","last_became_peered":"2023-10-06T05:16:31.740896+0000","last_unstale":"2023-10-06T05:16:31.741207+0000","last_undegraded":"2023-10-06T05:16:31.741207+0000","last_fullsized":"2023-10-06T05:16:31.741207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:10.574150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.751006+0000","last_change":"2023-10-06T05:16:31.751006+0000","last_active":"2023-10-06T05:16:31.751006+0000","last_peered":"2023-10-06T05:16:31.751006+0000","last_clean":"2023-10-06T05:16:31.751006+0000","last_became_active":"2023-10-06T05:16:31.750750+0000","last_became_peered":"2023-10-06T05:16:31.750750+0000","last_unstale":"2023-10-06T05:16:31.751006+0000","last_undegraded":"2023-10-06T05:16:31.751006+0000","last_fullsized":"2023-10-06T05:16:31.751006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:31.817743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743376+0000","last_change":"2023-10-06T05:16:31.743376+0000","last_active":"2023-10-06T05:16:31.743376+0000","last_peered":"2023-10-06T05:16:31.743376+0000","last_clean":"2023-10-06T05:16:31.743376+0000","last_became_active":"2023-10-06T05:16:31.741593+0000","last_became_peered":"2023-10-06T05:16:31.741593+0000","last_unstale":"2023-10-06T05:16:31.743376+0000","last_undegraded":"2023-10-06T05:16:31.743376+0000","last_fullsized":"2023-10-06T05:16:31.743376+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:19.950359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735851+0000","last_change":"2023-10-06T05:16:31.735851+0000","last_active":"2023-10-06T05:16:31.735851+0000","last_peered":"2023-10-06T05:16:31.735851+0000","last_clean":"2023-10-06T05:16:31.735851+0000","last_became_active":"2023-10-06T05:16:31.735563+0000","last_became_peered":"2023-10-06T05:16:31.735563+0000","last_unstale":"2023-10-06T05:16:31.735851+0000","last_undegraded":"2023-10-06T05:16:31.735851+0000","last_fullsized":"2023-10-06T05:16:31.735851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:06.572971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737822+0000","last_change":"2023-10-06T05:16:31.737822+0000","last_active":"2023-10-06T05:16:31.737822+0000","last_peered":"2023-10-06T05:16:31.737822+0000","last_clean":"2023-10-06T05:16:31.737822+0000","last_became_active":"2023-10-06T05:16:31.737523+0000","last_became_peered":"2023-10-06T05:16:31.737523+0000","last_unstale":"2023-10-06T05:16:31.737822+0000","last_undegraded":"2023-10-06T05:16:31.737822+0000","last_fullsized":"2023-10-06T05:16:31.737822+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:36.127945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735182+0000","last_change":"2023-10-06T05:16:31.735182+0000","last_active":"2023-10-06T05:16:31.735182+0000","last_peered":"2023-10-06T05:16:31.735182+0000","last_clean":"2023-10-06T05:16:31.735182+0000","last_became_active":"2023-10-06T05:16:31.734906+0000","last_became_peered":"2023-10-06T05:16:31.734906+0000","last_unstale":"2023-10-06T05:16:31.735182+0000","last_undegraded":"2023-10-06T05:16:31.735182+0000","last_fullsized":"2023-10-06T05:16:31.735182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:15.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739783+0000","last_change":"2023-10-06T05:16:31.739783+0000","last_active":"2023-10-06T05:16:31.739783+0000","last_peered":"2023-10-06T05:16:31.739783+0000","last_clean":"2023-10-06T05:16:31.739783+0000","last_became_active":"2023-10-06T05:16:31.739491+0000","last_became_peered":"2023-10-06T05:16:31.739491+0000","last_unstale":"2023-10-06T05:16:31.739783+0000","last_undegraded":"2023-10-06T05:16:31.739783+0000","last_fullsized":"2023-10-06T05:16:31.739783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:19.950359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.752032+0000","last_change":"2023-10-06T05:16:31.752032+0000","last_active":"2023-10-06T05:16:31.752032+0000","last_peered":"2023-10-06T05:16:31.752032+0000","last_clean":"2023-10-06T05:16:31.752032+0000","last_became_active":"2023-10-06T05:16:31.751719+0000","last_became_peered":"2023-10-06T05:16:31.751719+0000","last_unstale":"2023-10-06T05:16:31.752032+0000","last_undegraded":"2023-10-06T05:16:31.752032+0000","last_fullsized":"2023-10-06T05:16:31.752032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:26.456768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.745966+0000","last_change":"2023-10-06T05:16:31.745966+0000","last_active":"2023-10-06T05:16:31.745966+0000","last_peered":"2023-10-06T05:16:31.745966+0000","last_clean":"2023-10-06T05:16:31.745966+0000","last_became_active":"2023-10-06T05:16:31.745659+0000","last_became_peered":"2023-10-06T05:16:31.745659+0000","last_unstale":"2023-10-06T05:16:31.745966+0000","last_undegraded":"2023-10-06T05:16:31.745966+0000","last_fullsized":"2023-10-06T05:16:31.745966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:43.017359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737114+0000","last_change":"2023-10-06T05:16:31.737114+0000","last_active":"2023-10-06T05:16:31.737114+0000","last_peered":"2023-10-06T05:16:31.737114+0000","last_clean":"2023-10-06T05:16:31.737114+0000","last_became_active":"2023-10-06T05:16:31.736830+0000","last_became_peered":"2023-10-06T05:16:31.736830+0000","last_unstale":"2023-10-06T05:16:31.737114+0000","last_undegraded":"2023-10-06T05:16:31.737114+0000","last_fullsized":"2023-10-06T05:16:31.737114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:33.246474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740884+0000","last_change":"2023-10-06T05:16:31.740884+0000","last_active":"2023-10-06T05:16:31.740884+0000","last_peered":"2023-10-06T05:16:31.740884+0000","last_clean":"2023-10-06T05:16:31.740884+0000","last_became_active":"2023-10-06T05:16:31.740364+0000","last_became_peered":"2023-10-06T05:16:31.740364+0000","last_unstale":"2023-10-06T05:16:31.740884+0000","last_undegraded":"2023-10-06T05:16:31.740884+0000","last_fullsized":"2023-10-06T05:16:31.740884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:34.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.734272+0000","last_change":"2023-10-06T05:16:31.734272+0000","last_active":"2023-10-06T05:16:31.734272+0000","last_peered":"2023-10-06T05:16:31.734272+0000","last_clean":"2023-10-06T05:16:31.734272+0000","last_became_active":"2023-10-06T05:16:31.733835+0000","last_became_peered":"2023-10-06T05:16:31.733835+0000","last_unstale":"2023-10-06T05:16:31.734272+0000","last_undegraded":"2023-10-06T05:16:31.734272+0000","last_fullsized":"2023-10-06T05:16:31.734272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:33.246474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.750128+0000","last_change":"2023-10-06T05:16:31.750128+0000","last_active":"2023-10-06T05:16:31.750128+0000","last_peered":"2023-10-06T05:16:31.750128+0000","last_clean":"2023-10-06T05:16:31.750128+0000","last_became_active":"2023-10-06T05:16:31.749844+0000","last_became_peered":"2023-10-06T05:16:31.749844+0000","last_unstale":"2023-10-06T05:16:31.750128+0000","last_undegraded":"2023-10-06T05:16:31.750128+0000","last_fullsized":"2023-10-06T05:16:31.750128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:05.113348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.744622+0000","last_change":"2023-10-06T05:16:31.744622+0000","last_active":"2023-10-06T05:16:31.744622+0000","last_peered":"2023-10-06T05:16:31.744622+0000","last_clean":"2023-10-06T05:16:31.744622+0000","last_became_active":"2023-10-06T05:16:31.744363+0000","last_became_peered":"2023-10-06T05:16:31.744363+0000","last_unstale":"2023-10-06T05:16:31.744622+0000","last_undegraded":"2023-10-06T05:16:31.744622+0000","last_fullsized":"2023-10-06T05:16:31.744622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:34.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735101+0000","last_change":"2023-10-06T05:16:31.735101+0000","last_active":"2023-10-06T05:16:31.735101+0000","last_peered":"2023-10-06T05:16:31.735101+0000","last_clean":"2023-10-06T05:16:31.735101+0000","last_became_active":"2023-10-06T05:16:31.734802+0000","last_became_peered":"2023-10-06T05:16:31.734802+0000","last_unstale":"2023-10-06T05:16:31.735101+0000","last_undegraded":"2023-10-06T05:16:31.735101+0000","last_fullsized":"2023-10-06T05:16:31.735101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:47.915171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732034+0000","last_change":"2023-10-06T05:16:31.732034+0000","last_active":"2023-10-06T05:16:31.732034+0000","last_peered":"2023-10-06T05:16:31.732034+0000","last_clean":"2023-10-06T05:16:31.732034+0000","last_became_active":"2023-10-06T05:16:31.731781+0000","last_became_peered":"2023-10-06T05:16:31.731781+0000","last_unstale":"2023-10-06T05:16:31.732034+0000","last_undegraded":"2023-10-06T05:16:31.732034+0000","last_fullsized":"2023-10-06T05:16:31.732034+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:36.864601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743248+0000","last_change":"2023-10-06T05:16:31.743248+0000","last_active":"2023-10-06T05:16:31.743248+0000","last_peered":"2023-10-06T05:16:31.743248+0000","last_clean":"2023-10-06T05:16:31.743248+0000","last_became_active":"2023-10-06T05:16:31.742045+0000","last_became_peered":"2023-10-06T05:16:31.742045+0000","last_unstale":"2023-10-06T05:16:31.743248+0000","last_undegraded":"2023-10-06T05:16:31.743248+0000","last_fullsized":"2023-10-06T05:16:31.743248+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:48.881030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732378+0000","last_change":"2023-10-06T05:16:31.732378+0000","last_active":"2023-10-06T05:16:31.732378+0000","last_peered":"2023-10-06T05:16:31.732378+0000","last_clean":"2023-10-06T05:16:31.732378+0000","last_became_active":"2023-10-06T05:16:31.732110+0000","last_became_peered":"2023-10-06T05:16:31.732110+0000","last_unstale":"2023-10-06T05:16:31.732378+0000","last_undegraded":"2023-10-06T05:16:31.732378+0000","last_fullsized":"2023-10-06T05:16:31.732378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:08.607604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732373+0000","last_change":"2023-10-06T05:16:31.732373+0000","last_active":"2023-10-06T05:16:31.732373+0000","last_peered":"2023-10-06T05:16:31.732373+0000","last_clean":"2023-10-06T05:16:31.732373+0000","last_became_active":"2023-10-06T05:16:31.732117+0000","last_became_peered":"2023-10-06T05:16:31.732117+0000","last_unstale":"2023-10-06T05:16:31.732373+0000","last_undegraded":"2023-10-06T05:16:31.732373+0000","last_fullsized":"2023-10-06T05:16:31.732373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:58.652264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.738202+0000","last_change":"2023-10-06T05:16:31.738202+0000","last_active":"2023-10-06T05:16:31.738202+0000","last_peered":"2023-10-06T05:16:31.738202+0000","last_clean":"2023-10-06T05:16:31.738202+0000","last_became_active":"2023-10-06T05:16:31.737795+0000","last_became_peered":"2023-10-06T05:16:31.737795+0000","last_unstale":"2023-10-06T05:16:31.738202+0000","last_undegraded":"2023-10-06T05:16:31.738202+0000","last_fullsized":"2023-10-06T05:16:31.738202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:58.237697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735215+0000","last_change":"2023-10-06T05:16:31.735215+0000","last_active":"2023-10-06T05:16:31.735215+0000","last_peered":"2023-10-06T05:16:31.735215+0000","last_clean":"2023-10-06T05:16:31.735215+0000","last_became_active":"2023-10-06T05:16:31.734936+0000","last_became_peered":"2023-10-06T05:16:31.734936+0000","last_unstale":"2023-10-06T05:16:31.735215+0000","last_undegraded":"2023-10-06T05:16:31.735215+0000","last_fullsized":"2023-10-06T05:16:31.735215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:57.549236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.733059+0000","last_change":"2023-10-06T05:16:31.733059+0000","last_active":"2023-10-06T05:16:31.733059+0000","last_peered":"2023-10-06T05:16:31.733059+0000","last_clean":"2023-10-06T05:16:31.733059+0000","last_became_active":"2023-10-06T05:16:31.732798+0000","last_became_peered":"2023-10-06T05:16:31.732798+0000","last_unstale":"2023-10-06T05:16:31.733059+0000","last_undegraded":"2023-10-06T05:16:31.733059+0000","last_fullsized":"2023-10-06T05:16:31.733059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:37.068605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735301+0000","last_change":"2023-10-06T05:16:31.735301+0000","last_active":"2023-10-06T05:16:31.735301+0000","last_peered":"2023-10-06T05:16:31.735301+0000","last_clean":"2023-10-06T05:16:31.735301+0000","last_became_active":"2023-10-06T05:16:31.735050+0000","last_became_peered":"2023-10-06T05:16:31.735050+0000","last_unstale":"2023-10-06T05:16:31.735301+0000","last_undegraded":"2023-10-06T05:16:31.735301+0000","last_fullsized":"2023-10-06T05:16:31.735301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:33.246474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735397+0000","last_change":"2023-10-06T05:16:31.735397+0000","last_active":"2023-10-06T05:16:31.735397+0000","last_peered":"2023-10-06T05:16:31.735397+0000","last_clean":"2023-10-06T05:16:31.735397+0000","last_became_active":"2023-10-06T05:16:31.735120+0000","last_became_peered":"2023-10-06T05:16:31.735120+0000","last_unstale":"2023-10-06T05:16:31.735397+0000","last_undegraded":"2023-10-06T05:16:31.735397+0000","last_fullsized":"2023-10-06T05:16:31.735397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:36.864601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739844+0000","last_change":"2023-10-06T05:16:31.739844+0000","last_active":"2023-10-06T05:16:31.739844+0000","last_peered":"2023-10-06T05:16:31.739844+0000","last_clean":"2023-10-06T05:16:31.739844+0000","last_became_active":"2023-10-06T05:16:31.739560+0000","last_became_peered":"2023-10-06T05:16:31.739560+0000","last_unstale":"2023-10-06T05:16:31.739844+0000","last_undegraded":"2023-10-06T05:16:31.739844+0000","last_fullsized":"2023-10-06T05:16:31.739844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:10.574150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737270+0000","last_change":"2023-10-06T05:16:31.737270+0000","last_active":"2023-10-06T05:16:31.737270+0000","last_peered":"2023-10-06T05:16:31.737270+0000","last_clean":"2023-10-06T05:16:31.737270+0000","last_became_active":"2023-10-06T05:16:31.734035+0000","last_became_peered":"2023-10-06T05:16:31.734035+0000","last_unstale":"2023-10-06T05:16:31.737270+0000","last_undegraded":"2023-10-06T05:16:31.737270+0000","last_fullsized":"2023-10-06T05:16:31.737270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:18.230946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.733722+0000","last_change":"2023-10-06T05:16:31.733722+0000","last_active":"2023-10-06T05:16:31.733722+0000","last_peered":"2023-10-06T05:16:31.733722+0000","last_clean":"2023-10-06T05:16:31.733722+0000","last_became_active":"2023-10-06T05:16:31.733455+0000","last_became_peered":"2023-10-06T05:16:31.733455+0000","last_unstale":"2023-10-06T05:16:31.733722+0000","last_undegraded":"2023-10-06T05:16:31.733722+0000","last_fullsized":"2023-10-06T05:16:31.733722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:24.781580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743127+0000","last_change":"2023-10-06T05:16:31.743127+0000","last_active":"2023-10-06T05:16:31.743127+0000","last_peered":"2023-10-06T05:16:31.743127+0000","last_clean":"2023-10-06T05:16:31.743127+0000","last_became_active":"2023-10-06T05:16:31.742862+0000","last_became_peered":"2023-10-06T05:16:31.742862+0000","last_unstale":"2023-10-06T05:16:31.743127+0000","last_undegraded":"2023-10-06T05:16:31.743127+0000","last_fullsized":"2023-10-06T05:16:31.743127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:58.848697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.738429+0000","last_change":"2023-10-06T05:16:31.738429+0000","last_active":"2023-10-06T05:16:31.738429+0000","last_peered":"2023-10-06T05:16:31.738429+0000","last_clean":"2023-10-06T05:16:31.738429+0000","last_became_active":"2023-10-06T05:16:31.737049+0000","last_became_peered":"2023-10-06T05:16:31.737049+0000","last_unstale":"2023-10-06T05:16:31.738429+0000","last_undegraded":"2023-10-06T05:16:31.738429+0000","last_fullsized":"2023-10-06T05:16:31.738429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:48.881030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.736253+0000","last_change":"2023-10-06T05:16:31.736253+0000","last_active":"2023-10-06T05:16:31.736253+0000","last_peered":"2023-10-06T05:16:31.736253+0000","last_clean":"2023-10-06T05:16:31.736253+0000","last_became_active":"2023-10-06T05:16:31.735438+0000","last_became_peered":"2023-10-06T05:16:31.735438+0000","last_unstale":"2023-10-06T05:16:31.736253+0000","last_undegraded":"2023-10-06T05:16:31.736253+0000","last_fullsized":"2023-10-06T05:16:31.736253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:34.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.729996+0000","last_change":"2023-10-06T05:16:31.729996+0000","last_active":"2023-10-06T05:16:31.729996+0000","last_peered":"2023-10-06T05:16:31.729996+0000","last_clean":"2023-10-06T05:16:31.729996+0000","last_became_active":"2023-10-06T05:16:31.729760+0000","last_became_peered":"2023-10-06T05:16:31.729760+0000","last_unstale":"2023-10-06T05:16:31.729996+0000","last_undegraded":"2023-10-06T05:16:31.729996+0000","last_fullsized":"2023-10-06T05:16:31.729996+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.397759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.742987+0000","last_change":"2023-10-06T05:16:31.742987+0000","last_active":"2023-10-06T05:16:31.742987+0000","last_peered":"2023-10-06T05:16:31.742987+0000","last_clean":"2023-10-06T05:16:31.742987+0000","last_became_active":"2023-10-06T05:16:31.742674+0000","last_became_peered":"2023-10-06T05:16:31.742674+0000","last_unstale":"2023-10-06T05:16:31.742987+0000","last_undegraded":"2023-10-06T05:16:31.742987+0000","last_fullsized":"2023-10-06T05:16:31.742987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:46.338128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.758701+0000","last_change":"2023-10-06T05:16:30.741190+0000","last_active":"2023-10-06T05:16:31.758701+0000","last_peered":"2023-10-06T05:16:31.758701+0000","last_clean":"2023-10-06T05:16:31.758701+0000","last_became_active":"2023-10-06T05:16:30.740682+0000","last_became_peered":"2023-10-06T05:16:30.740682+0000","last_unstale":"2023-10-06T05:16:31.758701+0000","last_undegraded":"2023-10-06T05:16:31.758701+0000","last_fullsized":"2023-10-06T05:16:31.758701+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.746933+0000","last_change":"2023-10-06T05:16:31.746933+0000","last_active":"2023-10-06T05:16:31.746933+0000","last_peered":"2023-10-06T05:16:31.746933+0000","last_clean":"2023-10-06T05:16:31.746933+0000","last_became_active":"2023-10-06T05:16:31.746609+0000","last_became_peered":"2023-10-06T05:16:31.746609+0000","last_unstale":"2023-10-06T05:16:31.746933+0000","last_undegraded":"2023-10-06T05:16:31.746933+0000","last_fullsized":"2023-10-06T05:16:31.746933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:58.237697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.716589+0000","last_change":"2023-10-06T05:16:30.732833+0000","last_active":"2023-10-06T05:16:31.716589+0000","last_peered":"2023-10-06T05:16:31.716589+0000","last_clean":"2023-10-06T05:16:31.716589+0000","last_became_active":"2023-10-06T05:16:30.732003+0000","last_became_peered":"2023-10-06T05:16:30.732003+0000","last_unstale":"2023-10-06T05:16:31.716589+0000","last_undegraded":"2023-10-06T05:16:31.716589+0000","last_fullsized":"2023-10-06T05:16:31.716589+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:04.104858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735391+0000","last_change":"2023-10-06T05:16:31.735391+0000","last_active":"2023-10-06T05:16:31.735391+0000","last_peered":"2023-10-06T05:16:31.735391+0000","last_clean":"2023-10-06T05:16:31.735391+0000","last_became_active":"2023-10-06T05:16:31.735110+0000","last_became_peered":"2023-10-06T05:16:31.735110+0000","last_unstale":"2023-10-06T05:16:31.735391+0000","last_undegraded":"2023-10-06T05:16:31.735391+0000","last_fullsized":"2023-10-06T05:16:31.735391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:57.549236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.749881+0000","last_change":"2023-10-06T05:16:30.728961+0000","last_active":"2023-10-06T05:16:31.749881+0000","last_peered":"2023-10-06T05:16:31.749881+0000","last_clean":"2023-10-06T05:16:31.749881+0000","last_became_active":"2023-10-06T05:16:30.728658+0000","last_became_peered":"2023-10-06T05:16:30.728658+0000","last_unstale":"2023-10-06T05:16:31.749881+0000","last_undegraded":"2023-10-06T05:16:31.749881+0000","last_fullsized":"2023-10-06T05:16:31.749881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:42.008870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.746057+0000","last_change":"2023-10-06T05:16:31.746057+0000","last_active":"2023-10-06T05:16:31.746057+0000","last_peered":"2023-10-06T05:16:31.746057+0000","last_clean":"2023-10-06T05:16:31.746057+0000","last_became_active":"2023-10-06T05:16:31.745747+0000","last_became_peered":"2023-10-06T05:16:31.745747+0000","last_unstale":"2023-10-06T05:16:31.746057+0000","last_undegraded":"2023-10-06T05:16:31.746057+0000","last_fullsized":"2023-10-06T05:16:31.746057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:50.174715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.751527+0000","last_change":"2023-10-06T05:16:30.735164+0000","last_active":"2023-10-06T05:16:31.751527+0000","last_peered":"2023-10-06T05:16:31.751527+0000","last_clean":"2023-10-06T05:16:31.751527+0000","last_became_active":"2023-10-06T05:16:30.734864+0000","last_became_peered":"2023-10-06T05:16:30.734864+0000","last_unstale":"2023-10-06T05:16:31.751527+0000","last_undegraded":"2023-10-06T05:16:31.751527+0000","last_fullsized":"2023-10-06T05:16:31.751527+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:33.098566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740389+0000","last_change":"2023-10-06T05:16:31.740389+0000","last_active":"2023-10-06T05:16:31.740389+0000","last_peered":"2023-10-06T05:16:31.740389+0000","last_clean":"2023-10-06T05:16:31.740389+0000","last_became_active":"2023-10-06T05:16:31.739307+0000","last_became_peered":"2023-10-06T05:16:31.739307+0000","last_unstale":"2023-10-06T05:16:31.740389+0000","last_undegraded":"2023-10-06T05:16:31.740389+0000","last_fullsized":"2023-10-06T05:16:31.740389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.375266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.719386+0000","last_change":"2023-10-06T05:16:30.736788+0000","last_active":"2023-10-06T05:16:31.719386+0000","last_peered":"2023-10-06T05:16:31.719386+0000","last_clean":"2023-10-06T05:16:31.719386+0000","last_became_active":"2023-10-06T05:16:30.736509+0000","last_became_peered":"2023-10-06T05:16:30.736509+0000","last_unstale":"2023-10-06T05:16:31.719386+0000","last_undegraded":"2023-10-06T05:16:31.719386+0000","last_fullsized":"2023-10-06T05:16:31.719386+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:57.840207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.736892+0000","last_change":"2023-10-06T05:16:31.736892+0000","last_active":"2023-10-06T05:16:31.736892+0000","last_peered":"2023-10-06T05:16:31.736892+0000","last_clean":"2023-10-06T05:16:31.736892+0000","last_became_active":"2023-10-06T05:16:31.736622+0000","last_became_peered":"2023-10-06T05:16:31.736622+0000","last_unstale":"2023-10-06T05:16:31.736892+0000","last_undegraded":"2023-10-06T05:16:31.736892+0000","last_fullsized":"2023-10-06T05:16:31.736892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:18.230946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.719579+0000","last_change":"2023-10-06T05:16:30.731764+0000","last_active":"2023-10-06T05:16:31.719579+0000","last_peered":"2023-10-06T05:16:31.719579+0000","last_clean":"2023-10-06T05:16:31.719579+0000","last_became_active":"2023-10-06T05:16:30.731454+0000","last_became_peered":"2023-10-06T05:16:30.731454+0000","last_unstale":"2023-10-06T05:16:31.719579+0000","last_undegraded":"2023-10-06T05:16:31.719579+0000","last_fullsized":"2023-10-06T05:16:31.719579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:57.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.730126+0000","last_change":"2023-10-06T05:16:31.730126+0000","last_active":"2023-10-06T05:16:31.730126+0000","last_peered":"2023-10-06T05:16:31.730126+0000","last_clean":"2023-10-06T05:16:31.730126+0000","last_became_active":"2023-10-06T05:16:31.729644+0000","last_became_peered":"2023-10-06T05:16:31.729644+0000","last_unstale":"2023-10-06T05:16:31.730126+0000","last_undegraded":"2023-10-06T05:16:31.730126+0000","last_fullsized":"2023-10-06T05:16:31.730126+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:12.346621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.741242+0000","last_change":"2023-10-06T05:16:30.738916+0000","last_active":"2023-10-06T05:16:30.741242+0000","last_peered":"2023-10-06T05:16:30.741242+0000","last_clean":"2023-10-06T05:16:30.741242+0000","last_became_active":"2023-10-06T05:16:30.738650+0000","last_became_peered":"2023-10-06T05:16:30.738650+0000","last_unstale":"2023-10-06T05:16:30.741242+0000","last_undegraded":"2023-10-06T05:16:30.741242+0000","last_fullsized":"2023-10-06T05:16:30.741242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:47.872540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.729217+0000","last_change":"2023-10-06T05:16:31.729217+0000","last_active":"2023-10-06T05:16:31.729217+0000","last_peered":"2023-10-06T05:16:31.729217+0000","last_clean":"2023-10-06T05:16:31.729217+0000","last_became_active":"2023-10-06T05:16:31.728940+0000","last_became_peered":"2023-10-06T05:16:31.728940+0000","last_unstale":"2023-10-06T05:16:31.729217+0000","last_undegraded":"2023-10-06T05:16:31.729217+0000","last_fullsized":"2023-10-06T05:16:31.729217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.397759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.736725+0000","last_change":"2023-10-06T05:16:30.730313+0000","last_active":"2023-10-06T05:16:30.736725+0000","last_peered":"2023-10-06T05:16:30.736725+0000","last_clean":"2023-10-06T05:16:30.736725+0000","last_became_active":"2023-10-06T05:16:30.730043+0000","last_became_peered":"2023-10-06T05:16:30.730043+0000","last_unstale":"2023-10-06T05:16:30.736725+0000","last_undegraded":"2023-10-06T05:16:30.736725+0000","last_fullsized":"2023-10-06T05:16:30.736725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:14.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732519+0000","last_change":"2023-10-06T05:16:31.732519+0000","last_active":"2023-10-06T05:16:31.732519+0000","last_peered":"2023-10-06T05:16:31.732519+0000","last_clean":"2023-10-06T05:16:31.732519+0000","last_became_active":"2023-10-06T05:16:31.732233+0000","last_became_peered":"2023-10-06T05:16:31.732233+0000","last_unstale":"2023-10-06T05:16:31.732519+0000","last_undegraded":"2023-10-06T05:16:31.732519+0000","last_fullsized":"2023-10-06T05:16:31.732519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:23.009115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.744537+0000","last_change":"2023-10-06T05:16:30.741247+0000","last_active":"2023-10-06T05:16:30.744537+0000","last_peered":"2023-10-06T05:16:30.744537+0000","last_clean":"2023-10-06T05:16:30.744537+0000","last_became_active":"2023-10-06T05:16:30.740896+0000","last_became_peered":"2023-10-06T05:16:30.740896+0000","last_unstale":"2023-10-06T05:16:30.744537+0000","last_undegraded":"2023-10-06T05:16:30.744537+0000","last_fullsized":"2023-10-06T05:16:30.744537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:22.000626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740432+0000","last_change":"2023-10-06T05:16:31.740432+0000","last_active":"2023-10-06T05:16:31.740432+0000","last_peered":"2023-10-06T05:16:31.740432+0000","last_clean":"2023-10-06T05:16:31.740432+0000","last_became_active":"2023-10-06T05:16:31.740166+0000","last_became_peered":"2023-10-06T05:16:31.740166+0000","last_unstale":"2023-10-06T05:16:31.740432+0000","last_undegraded":"2023-10-06T05:16:31.740432+0000","last_fullsized":"2023-10-06T05:16:31.740432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:07.515410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.735189+0000","last_change":"2023-10-06T05:16:30.733275+0000","last_active":"2023-10-06T05:16:30.735189+0000","last_peered":"2023-10-06T05:16:30.735189+0000","last_clean":"2023-10-06T05:16:30.735189+0000","last_became_active":"2023-10-06T05:16:30.733021+0000","last_became_peered":"2023-10-06T05:16:30.733021+0000","last_unstale":"2023-10-06T05:16:30.735189+0000","last_undegraded":"2023-10-06T05:16:30.735189+0000","last_fullsized":"2023-10-06T05:16:30.735189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:19.056144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.747476+0000","last_change":"2023-10-06T05:16:31.747476+0000","last_active":"2023-10-06T05:16:31.747476+0000","last_peered":"2023-10-06T05:16:31.747476+0000","last_clean":"2023-10-06T05:16:31.747476+0000","last_became_active":"2023-10-06T05:16:31.744118+0000","last_became_peered":"2023-10-06T05:16:31.744118+0000","last_unstale":"2023-10-06T05:16:31.747476+0000","last_undegraded":"2023-10-06T05:16:31.747476+0000","last_fullsized":"2023-10-06T05:16:31.747476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:59.831684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.754096+0000","last_change":"2023-10-06T05:16:30.742192+0000","last_active":"2023-10-06T05:16:31.754096+0000","last_peered":"2023-10-06T05:16:31.754096+0000","last_clean":"2023-10-06T05:16:31.754096+0000","last_became_active":"2023-10-06T05:16:30.741945+0000","last_became_peered":"2023-10-06T05:16:30.741945+0000","last_unstale":"2023-10-06T05:16:31.754096+0000","last_undegraded":"2023-10-06T05:16:31.754096+0000","last_fullsized":"2023-10-06T05:16:31.754096+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:37.425290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732651+0000","last_change":"2023-10-06T05:16:31.732651+0000","last_active":"2023-10-06T05:16:31.732651+0000","last_peered":"2023-10-06T05:16:31.732651+0000","last_clean":"2023-10-06T05:16:31.732651+0000","last_became_active":"2023-10-06T05:16:31.732329+0000","last_became_peered":"2023-10-06T05:16:31.732329+0000","last_unstale":"2023-10-06T05:16:31.732651+0000","last_undegraded":"2023-10-06T05:16:31.732651+0000","last_fullsized":"2023-10-06T05:16:31.732651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:10.574150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.762350+0000","last_change":"2023-10-06T05:16:30.734398+0000","last_active":"2023-10-06T05:16:31.762350+0000","last_peered":"2023-10-06T05:16:31.762350+0000","last_clean":"2023-10-06T05:16:31.762350+0000","last_became_active":"2023-10-06T05:16:30.734116+0000","last_became_peered":"2023-10-06T05:16:30.734116+0000","last_unstale":"2023-10-06T05:16:31.762350+0000","last_undegraded":"2023-10-06T05:16:31.762350+0000","last_fullsized":"2023-10-06T05:16:31.762350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:11.338131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739179+0000","last_change":"2023-10-06T05:16:31.739179+0000","last_active":"2023-10-06T05:16:31.739179+0000","last_peered":"2023-10-06T05:16:31.739179+0000","last_clean":"2023-10-06T05:16:31.739179+0000","last_became_active":"2023-10-06T05:16:31.738813+0000","last_became_peered":"2023-10-06T05:16:31.738813+0000","last_unstale":"2023-10-06T05:16:31.739179+0000","last_undegraded":"2023-10-06T05:16:31.739179+0000","last_fullsized":"2023-10-06T05:16:31.739179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:26.456768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.761628+0000","last_change":"2023-10-06T05:16:30.739499+0000","last_active":"2023-10-06T05:16:31.761628+0000","last_peered":"2023-10-06T05:16:31.761628+0000","last_clean":"2023-10-06T05:16:31.761628+0000","last_became_active":"2023-10-06T05:16:30.739223+0000","last_became_peered":"2023-10-06T05:16:30.739223+0000","last_unstale":"2023-10-06T05:16:31.761628+0000","last_undegraded":"2023-10-06T05:16:31.761628+0000","last_fullsized":"2023-10-06T05:16:31.761628+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:08.111445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735292+0000","last_change":"2023-10-06T05:16:31.735292+0000","last_active":"2023-10-06T05:16:31.735292+0000","last_peered":"2023-10-06T05:16:31.735292+0000","last_clean":"2023-10-06T05:16:31.735292+0000","last_became_active":"2023-10-06T05:16:31.734566+0000","last_became_peered":"2023-10-06T05:16:31.734566+0000","last_unstale":"2023-10-06T05:16:31.735292+0000","last_undegraded":"2023-10-06T05:16:31.735292+0000","last_fullsized":"2023-10-06T05:16:31.735292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:15.070344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734374+0000","last_change":"2023-10-06T05:16:30.731789+0000","last_active":"2023-10-06T05:16:30.734374+0000","last_peered":"2023-10-06T05:16:30.734374+0000","last_clean":"2023-10-06T05:16:30.734374+0000","last_became_active":"2023-10-06T05:16:30.731455+0000","last_became_peered":"2023-10-06T05:16:30.731455+0000","last_unstale":"2023-10-06T05:16:30.734374+0000","last_undegraded":"2023-10-06T05:16:30.734374+0000","last_fullsized":"2023-10-06T05:16:30.734374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.731174+0000","last_change":"2023-10-06T05:16:31.731174+0000","last_active":"2023-10-06T05:16:31.731174+0000","last_peered":"2023-10-06T05:16:31.731174+0000","last_clean":"2023-10-06T05:16:31.731174+0000","last_became_active":"2023-10-06T05:16:31.730875+0000","last_became_peered":"2023-10-06T05:16:31.730875+0000","last_unstale":"2023-10-06T05:16:31.731174+0000","last_undegraded":"2023-10-06T05:16:31.731174+0000","last_fullsized":"2023-10-06T05:16:31.731174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:53.375917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.744141+0000","last_change":"2023-10-06T05:16:30.740289+0000","last_active":"2023-10-06T05:16:30.744141+0000","last_peered":"2023-10-06T05:16:30.744141+0000","last_clean":"2023-10-06T05:16:30.744141+0000","last_became_active":"2023-10-06T05:16:30.740024+0000","last_became_peered":"2023-10-06T05:16:30.740024+0000","last_unstale":"2023-10-06T05:16:30.744141+0000","last_undegraded":"2023-10-06T05:16:30.744141+0000","last_fullsized":"2023-10-06T05:16:30.744141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:24.724584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741576+0000","last_change":"2023-10-06T05:16:31.741576+0000","last_active":"2023-10-06T05:16:31.741576+0000","last_peered":"2023-10-06T05:16:31.741576+0000","last_clean":"2023-10-06T05:16:31.741576+0000","last_became_active":"2023-10-06T05:16:31.741225+0000","last_became_peered":"2023-10-06T05:16:31.741225+0000","last_unstale":"2023-10-06T05:16:31.741576+0000","last_undegraded":"2023-10-06T05:16:31.741576+0000","last_fullsized":"2023-10-06T05:16:31.741576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:15.070344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.758673+0000","last_change":"2023-10-06T05:16:30.736552+0000","last_active":"2023-10-06T05:16:31.758673+0000","last_peered":"2023-10-06T05:16:31.758673+0000","last_clean":"2023-10-06T05:16:31.758673+0000","last_became_active":"2023-10-06T05:16:30.736295+0000","last_became_peered":"2023-10-06T05:16:30.736295+0000","last_unstale":"2023-10-06T05:16:31.758673+0000","last_undegraded":"2023-10-06T05:16:31.758673+0000","last_fullsized":"2023-10-06T05:16:31.758673+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:05.392771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739104+0000","last_change":"2023-10-06T05:16:31.739104+0000","last_active":"2023-10-06T05:16:31.739104+0000","last_peered":"2023-10-06T05:16:31.739104+0000","last_clean":"2023-10-06T05:16:31.739104+0000","last_became_active":"2023-10-06T05:16:31.738815+0000","last_became_peered":"2023-10-06T05:16:31.738815+0000","last_unstale":"2023-10-06T05:16:31.739104+0000","last_undegraded":"2023-10-06T05:16:31.739104+0000","last_fullsized":"2023-10-06T05:16:31.739104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:29.512000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.758349+0000","last_change":"2023-10-06T05:16:30.736868+0000","last_active":"2023-10-06T05:16:31.758349+0000","last_peered":"2023-10-06T05:16:31.758349+0000","last_clean":"2023-10-06T05:16:31.758349+0000","last_became_active":"2023-10-06T05:16:30.736569+0000","last_became_peered":"2023-10-06T05:16:30.736569+0000","last_unstale":"2023-10-06T05:16:31.758349+0000","last_undegraded":"2023-10-06T05:16:31.758349+0000","last_fullsized":"2023-10-06T05:16:31.758349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:24.192599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740118+0000","last_change":"2023-10-06T05:16:31.740118+0000","last_active":"2023-10-06T05:16:31.740118+0000","last_peered":"2023-10-06T05:16:31.740118+0000","last_clean":"2023-10-06T05:16:31.740118+0000","last_became_active":"2023-10-06T05:16:31.739833+0000","last_became_peered":"2023-10-06T05:16:31.739833+0000","last_unstale":"2023-10-06T05:16:31.740118+0000","last_undegraded":"2023-10-06T05:16:31.740118+0000","last_fullsized":"2023-10-06T05:16:31.740118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:19.575837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734329+0000","last_change":"2023-10-06T05:16:30.731142+0000","last_active":"2023-10-06T05:16:30.734329+0000","last_peered":"2023-10-06T05:16:30.734329+0000","last_clean":"2023-10-06T05:16:30.734329+0000","last_became_active":"2023-10-06T05:16:30.730810+0000","last_became_peered":"2023-10-06T05:16:30.730810+0000","last_unstale":"2023-10-06T05:16:30.734329+0000","last_undegraded":"2023-10-06T05:16:30.734329+0000","last_fullsized":"2023-10-06T05:16:30.734329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:54.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741743+0000","last_change":"2023-10-06T05:16:31.741743+0000","last_active":"2023-10-06T05:16:31.741743+0000","last_peered":"2023-10-06T05:16:31.741743+0000","last_clean":"2023-10-06T05:16:31.741743+0000","last_became_active":"2023-10-06T05:16:31.741482+0000","last_became_peered":"2023-10-06T05:16:31.741482+0000","last_unstale":"2023-10-06T05:16:31.741743+0000","last_undegraded":"2023-10-06T05:16:31.741743+0000","last_fullsized":"2023-10-06T05:16:31.741743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:50.822288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734526+0000","last_change":"2023-10-06T05:16:30.728891+0000","last_active":"2023-10-06T05:16:30.734526+0000","last_peered":"2023-10-06T05:16:30.734526+0000","last_clean":"2023-10-06T05:16:30.734526+0000","last_became_active":"2023-10-06T05:16:30.728632+0000","last_became_peered":"2023-10-06T05:16:30.728632+0000","last_unstale":"2023-10-06T05:16:30.734526+0000","last_undegraded":"2023-10-06T05:16:30.734526+0000","last_fullsized":"2023-10-06T05:16:30.734526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:49.166225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.728990+0000","last_change":"2023-10-06T05:16:31.728990+0000","last_active":"2023-10-06T05:16:31.728990+0000","last_peered":"2023-10-06T05:16:31.728990+0000","last_clean":"2023-10-06T05:16:31.728990+0000","last_became_active":"2023-10-06T05:16:31.728689+0000","last_became_peered":"2023-10-06T05:16:31.728689+0000","last_unstale":"2023-10-06T05:16:31.728990+0000","last_undegraded":"2023-10-06T05:16:31.728990+0000","last_fullsized":"2023-10-06T05:16:31.728990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:55.445215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.736753+0000","last_change":"2023-10-06T05:16:30.734176+0000","last_active":"2023-10-06T05:16:30.736753+0000","last_peered":"2023-10-06T05:16:30.736753+0000","last_clean":"2023-10-06T05:16:30.736753+0000","last_became_active":"2023-10-06T05:16:30.733627+0000","last_became_peered":"2023-10-06T05:16:30.733627+0000","last_unstale":"2023-10-06T05:16:30.736753+0000","last_undegraded":"2023-10-06T05:16:30.736753+0000","last_fullsized":"2023-10-06T05:16:30.736753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:38.868667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.742873+0000","last_change":"2023-10-06T05:16:31.742873+0000","last_active":"2023-10-06T05:16:31.742873+0000","last_peered":"2023-10-06T05:16:31.742873+0000","last_clean":"2023-10-06T05:16:31.742873+0000","last_became_active":"2023-10-06T05:16:31.742603+0000","last_became_peered":"2023-10-06T05:16:31.742603+0000","last_unstale":"2023-10-06T05:16:31.742873+0000","last_undegraded":"2023-10-06T05:16:31.742873+0000","last_fullsized":"2023-10-06T05:16:31.742873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:37.210002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.744109+0000","last_change":"2023-10-06T05:16:30.739638+0000","last_active":"2023-10-06T05:16:30.744109+0000","last_peered":"2023-10-06T05:16:30.744109+0000","last_clean":"2023-10-06T05:16:30.744109+0000","last_became_active":"2023-10-06T05:16:30.739271+0000","last_became_peered":"2023-10-06T05:16:30.739271+0000","last_unstale":"2023-10-06T05:16:30.744109+0000","last_undegraded":"2023-10-06T05:16:30.744109+0000","last_fullsized":"2023-10-06T05:16:30.744109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:50.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.738940+0000","last_change":"2023-10-06T05:16:31.738940+0000","last_active":"2023-10-06T05:16:31.738940+0000","last_peered":"2023-10-06T05:16:31.738940+0000","last_clean":"2023-10-06T05:16:31.738940+0000","last_became_active":"2023-10-06T05:16:31.738672+0000","last_became_peered":"2023-10-06T05:16:31.738672+0000","last_unstale":"2023-10-06T05:16:31.738940+0000","last_undegraded":"2023-10-06T05:16:31.738940+0000","last_fullsized":"2023-10-06T05:16:31.738940+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:05.113348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734023+0000","last_change":"2023-10-06T05:16:30.731846+0000","last_active":"2023-10-06T05:16:30.734023+0000","last_peered":"2023-10-06T05:16:30.734023+0000","last_clean":"2023-10-06T05:16:30.734023+0000","last_became_active":"2023-10-06T05:16:30.731573+0000","last_became_peered":"2023-10-06T05:16:30.731573+0000","last_unstale":"2023-10-06T05:16:30.734023+0000","last_undegraded":"2023-10-06T05:16:30.734023+0000","last_fullsized":"2023-10-06T05:16:30.734023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:33.098566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737546+0000","last_change":"2023-10-06T05:16:31.737546+0000","last_active":"2023-10-06T05:16:31.737546+0000","last_peered":"2023-10-06T05:16:31.737546+0000","last_clean":"2023-10-06T05:16:31.737546+0000","last_became_active":"2023-10-06T05:16:31.737283+0000","last_became_peered":"2023-10-06T05:16:31.737283+0000","last_unstale":"2023-10-06T05:16:31.737546+0000","last_undegraded":"2023-10-06T05:16:31.737546+0000","last_fullsized":"2023-10-06T05:16:31.737546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:15.536940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.741222+0000","last_change":"2023-10-06T05:16:30.738233+0000","last_active":"2023-10-06T05:16:30.741222+0000","last_peered":"2023-10-06T05:16:30.741222+0000","last_clean":"2023-10-06T05:16:30.741222+0000","last_became_active":"2023-10-06T05:16:30.737891+0000","last_became_peered":"2023-10-06T05:16:30.737891+0000","last_unstale":"2023-10-06T05:16:30.741222+0000","last_undegraded":"2023-10-06T05:16:30.741222+0000","last_fullsized":"2023-10-06T05:16:30.741222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:18.941870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741777+0000","last_change":"2023-10-06T05:16:31.741777+0000","last_active":"2023-10-06T05:16:31.741777+0000","last_peered":"2023-10-06T05:16:31.741777+0000","last_clean":"2023-10-06T05:16:31.741777+0000","last_became_active":"2023-10-06T05:16:31.741511+0000","last_became_peered":"2023-10-06T05:16:31.741511+0000","last_unstale":"2023-10-06T05:16:31.741777+0000","last_undegraded":"2023-10-06T05:16:31.741777+0000","last_fullsized":"2023-10-06T05:16:31.741777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:31.301396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.733680+0000","last_change":"2023-10-06T05:16:30.730536+0000","last_active":"2023-10-06T05:16:30.733680+0000","last_peered":"2023-10-06T05:16:30.733680+0000","last_clean":"2023-10-06T05:16:30.733680+0000","last_became_active":"2023-10-06T05:16:30.730208+0000","last_became_peered":"2023-10-06T05:16:30.730208+0000","last_unstale":"2023-10-06T05:16:30.733680+0000","last_undegraded":"2023-10-06T05:16:30.733680+0000","last_fullsized":"2023-10-06T05:16:30.733680+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:07.599115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.744434+0000","last_change":"2023-10-06T05:16:31.744434+0000","last_active":"2023-10-06T05:16:31.744434+0000","last_peered":"2023-10-06T05:16:31.744434+0000","last_clean":"2023-10-06T05:16:31.744434+0000","last_became_active":"2023-10-06T05:16:31.744155+0000","last_became_peered":"2023-10-06T05:16:31.744155+0000","last_unstale":"2023-10-06T05:16:31.744434+0000","last_undegraded":"2023-10-06T05:16:31.744434+0000","last_fullsized":"2023-10-06T05:16:31.744434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:08.022240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.730270+0000","last_change":"2023-10-06T05:16:30.732484+0000","last_active":"2023-10-06T05:16:31.730270+0000","last_peered":"2023-10-06T05:16:31.730270+0000","last_clean":"2023-10-06T05:16:31.730270+0000","last_became_active":"2023-10-06T05:16:30.732229+0000","last_became_peered":"2023-10-06T05:16:30.732229+0000","last_unstale":"2023-10-06T05:16:31.730270+0000","last_undegraded":"2023-10-06T05:16:31.730270+0000","last_fullsized":"2023-10-06T05:16:31.730270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:11.338131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.718878+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:16:30.718878+0000","last_peered":"2023-10-06T05:16:30.718878+0000","last_clean":"2023-10-06T05:16:30.718878+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:16:30.718878+0000","last_undegraded":"2023-10-06T05:16:30.718878+0000","last_fullsized":"2023-10-06T05:16:30.718878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.729942+0000","last_change":"2023-10-06T05:16:31.729942+0000","last_active":"2023-10-06T05:16:31.729942+0000","last_peered":"2023-10-06T05:16:31.729942+0000","last_clean":"2023-10-06T05:16:31.729942+0000","last_became_active":"2023-10-06T05:16:31.729340+0000","last_became_peered":"2023-10-06T05:16:31.729340+0000","last_unstale":"2023-10-06T05:16:31.729942+0000","last_undegraded":"2023-10-06T05:16:31.729942+0000","last_fullsized":"2023-10-06T05:16:31.729942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:39.877157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.739304+0000","last_change":"2023-10-06T05:16:30.736587+0000","last_active":"2023-10-06T05:16:30.739304+0000","last_peered":"2023-10-06T05:16:30.739304+0000","last_clean":"2023-10-06T05:16:30.739304+0000","last_became_active":"2023-10-06T05:16:30.736309+0000","last_became_peered":"2023-10-06T05:16:30.736309+0000","last_unstale":"2023-10-06T05:16:30.739304+0000","last_undegraded":"2023-10-06T05:16:30.739304+0000","last_fullsized":"2023-10-06T05:16:30.739304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:57.840207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.722105+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:16:30.722105+0000","last_peered":"2023-10-06T05:16:30.722105+0000","last_clean":"2023-10-06T05:16:30.722105+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:16:30.722105+0000","last_undegraded":"2023-10-06T05:16:30.722105+0000","last_fullsized":"2023-10-06T05:16:30.722105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740698+0000","last_change":"2023-10-06T05:16:31.740698+0000","last_active":"2023-10-06T05:16:31.740698+0000","last_peered":"2023-10-06T05:16:31.740698+0000","last_clean":"2023-10-06T05:16:31.740698+0000","last_became_active":"2023-10-06T05:16:31.739538+0000","last_became_peered":"2023-10-06T05:16:31.739538+0000","last_unstale":"2023-10-06T05:16:31.740698+0000","last_undegraded":"2023-10-06T05:16:31.740698+0000","last_fullsized":"2023-10-06T05:16:31.740698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:18.230946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734432+0000","last_change":"2023-10-06T05:16:30.733013+0000","last_active":"2023-10-06T05:16:30.734432+0000","last_peered":"2023-10-06T05:16:30.734432+0000","last_clean":"2023-10-06T05:16:30.734432+0000","last_became_active":"2023-10-06T05:16:30.732802+0000","last_became_peered":"2023-10-06T05:16:30.732802+0000","last_unstale":"2023-10-06T05:16:30.734432+0000","last_undegraded":"2023-10-06T05:16:30.734432+0000","last_fullsized":"2023-10-06T05:16:30.734432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:38.868667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.713460+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:16:30.713460+0000","last_peered":"2023-10-06T05:16:30.713460+0000","last_clean":"2023-10-06T05:16:30.713460+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:16:30.713460+0000","last_undegraded":"2023-10-06T05:16:30.713460+0000","last_fullsized":"2023-10-06T05:16:30.713460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741903+0000","last_change":"2023-10-06T05:16:31.741903+0000","last_active":"2023-10-06T05:16:31.741903+0000","last_peered":"2023-10-06T05:16:31.741903+0000","last_clean":"2023-10-06T05:16:31.741903+0000","last_became_active":"2023-10-06T05:16:31.741646+0000","last_became_peered":"2023-10-06T05:16:31.741646+0000","last_unstale":"2023-10-06T05:16:31.741903+0000","last_undegraded":"2023-10-06T05:16:31.741903+0000","last_fullsized":"2023-10-06T05:16:31.741903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:31:13.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.739103+0000","last_change":"2023-10-06T05:16:30.737214+0000","last_active":"2023-10-06T05:16:30.739103+0000","last_peered":"2023-10-06T05:16:30.739103+0000","last_clean":"2023-10-06T05:16:30.739103+0000","last_became_active":"2023-10-06T05:16:30.736958+0000","last_became_peered":"2023-10-06T05:16:30.736958+0000","last_unstale":"2023-10-06T05:16:30.739103+0000","last_undegraded":"2023-10-06T05:16:30.739103+0000","last_fullsized":"2023-10-06T05:16:30.739103+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:25.448278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.722258+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:16:30.722258+0000","last_peered":"2023-10-06T05:16:30.722258+0000","last_clean":"2023-10-06T05:16:30.722258+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:16:30.722258+0000","last_undegraded":"2023-10-06T05:16:30.722258+0000","last_fullsized":"2023-10-06T05:16:30.722258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.730455+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:16:30.730455+0000","last_peered":"2023-10-06T05:16:30.730455+0000","last_clean":"2023-10-06T05:16:30.730455+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:16:30.730455+0000","last_undegraded":"2023-10-06T05:16:30.730455+0000","last_fullsized":"2023-10-06T05:16:30.730455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.734088+0000","last_change":"2023-10-06T05:16:31.734088+0000","last_active":"2023-10-06T05:16:31.734088+0000","last_peered":"2023-10-06T05:16:31.734088+0000","last_clean":"2023-10-06T05:16:31.734088+0000","last_became_active":"2023-10-06T05:16:31.733822+0000","last_became_peered":"2023-10-06T05:16:31.733822+0000","last_unstale":"2023-10-06T05:16:31.734088+0000","last_undegraded":"2023-10-06T05:16:31.734088+0000","last_fullsized":"2023-10-06T05:16:31.734088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:37.068605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.754018+0000","last_change":"2023-10-06T05:16:30.738187+0000","last_active":"2023-10-06T05:16:31.754018+0000","last_peered":"2023-10-06T05:16:31.754018+0000","last_clean":"2023-10-06T05:16:31.754018+0000","last_became_active":"2023-10-06T05:16:30.737609+0000","last_became_peered":"2023-10-06T05:16:30.737609+0000","last_unstale":"2023-10-06T05:16:31.754018+0000","last_undegraded":"2023-10-06T05:16:31.754018+0000","last_fullsized":"2023-10-06T05:16:31.754018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:25.448278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.730535+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:16:30.730535+0000","last_peered":"2023-10-06T05:16:30.730535+0000","last_clean":"2023-10-06T05:16:30.730535+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:16:30.730535+0000","last_undegraded":"2023-10-06T05:16:30.730535+0000","last_fullsized":"2023-10-06T05:16:30.730535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.727276+0000","last_change":"2023-10-06T05:16:31.727276+0000","last_active":"2023-10-06T05:16:31.727276+0000","last_peered":"2023-10-06T05:16:31.727276+0000","last_clean":"2023-10-06T05:16:31.727276+0000","last_became_active":"2023-10-06T05:16:31.726986+0000","last_became_peered":"2023-10-06T05:16:31.726986+0000","last_unstale":"2023-10-06T05:16:31.727276+0000","last_undegraded":"2023-10-06T05:16:31.727276+0000","last_fullsized":"2023-10-06T05:16:31.727276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:54.421560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.762070+0000","last_change":"2023-10-06T05:16:30.739656+0000","last_active":"2023-10-06T05:16:31.762070+0000","last_peered":"2023-10-06T05:16:31.762070+0000","last_clean":"2023-10-06T05:16:31.762070+0000","last_became_active":"2023-10-06T05:16:30.739319+0000","last_became_peered":"2023-10-06T05:16:30.739319+0000","last_unstale":"2023-10-06T05:16:31.762070+0000","last_undegraded":"2023-10-06T05:16:31.762070+0000","last_fullsized":"2023-10-06T05:16:31.762070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:01.657888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.735586+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:16:30.735586+0000","last_peered":"2023-10-06T05:16:30.735586+0000","last_clean":"2023-10-06T05:16:30.735586+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:16:30.735586+0000","last_undegraded":"2023-10-06T05:16:30.735586+0000","last_fullsized":"2023-10-06T05:16:30.735586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.747497+0000","last_change":"2023-10-06T05:16:31.747497+0000","last_active":"2023-10-06T05:16:31.747497+0000","last_peered":"2023-10-06T05:16:31.747497+0000","last_clean":"2023-10-06T05:16:31.747497+0000","last_became_active":"2023-10-06T05:16:31.744893+0000","last_became_peered":"2023-10-06T05:16:31.744893+0000","last_unstale":"2023-10-06T05:16:31.747497+0000","last_undegraded":"2023-10-06T05:16:31.747497+0000","last_fullsized":"2023-10-06T05:16:31.747497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:50.822288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.730425+0000","last_change":"2023-10-06T05:16:30.741488+0000","last_active":"2023-10-06T05:16:31.730425+0000","last_peered":"2023-10-06T05:16:31.730425+0000","last_clean":"2023-10-06T05:16:31.730425+0000","last_became_active":"2023-10-06T05:16:30.741272+0000","last_became_peered":"2023-10-06T05:16:30.741272+0000","last_unstale":"2023-10-06T05:16:31.730425+0000","last_undegraded":"2023-10-06T05:16:31.730425+0000","last_fullsized":"2023-10-06T05:16:31.730425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:59.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.720297+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:16:30.720297+0000","last_peered":"2023-10-06T05:16:30.720297+0000","last_clean":"2023-10-06T05:16:30.720297+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:16:30.720297+0000","last_undegraded":"2023-10-06T05:16:30.720297+0000","last_fullsized":"2023-10-06T05:16:30.720297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.724634+0000","last_change":"2023-10-06T05:16:31.724634+0000","last_active":"2023-10-06T05:16:31.724634+0000","last_peered":"2023-10-06T05:16:31.724634+0000","last_clean":"2023-10-06T05:16:31.724634+0000","last_became_active":"2023-10-06T05:16:31.724340+0000","last_became_peered":"2023-10-06T05:16:31.724340+0000","last_unstale":"2023-10-06T05:16:31.724634+0000","last_undegraded":"2023-10-06T05:16:31.724634+0000","last_fullsized":"2023-10-06T05:16:31.724634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:46.935931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.738362+0000","last_change":"2023-10-06T05:16:30.736044+0000","last_active":"2023-10-06T05:16:30.738362+0000","last_peered":"2023-10-06T05:16:30.738362+0000","last_clean":"2023-10-06T05:16:30.738362+0000","last_became_active":"2023-10-06T05:16:30.735765+0000","last_became_peered":"2023-10-06T05:16:30.735765+0000","last_unstale":"2023-10-06T05:16:30.738362+0000","last_undegraded":"2023-10-06T05:16:30.738362+0000","last_fullsized":"2023-10-06T05:16:30.738362+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:04.104858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.725184+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:16:30.725184+0000","last_peered":"2023-10-06T05:16:30.725184+0000","last_clean":"2023-10-06T05:16:30.725184+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:16:30.725184+0000","last_undegraded":"2023-10-06T05:16:30.725184+0000","last_fullsized":"2023-10-06T05:16:30.725184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298572,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073268,"statfs":{"total":96636764160,"available":96331026432,"internally_reserved":0,"allocated":1314816,"data_stored":903767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85999999999999999,"5min":1.113,"15min":1.113},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":1.079,"15min":1.079},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.85299999999999998,"5min":1.202,"15min":1.202},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.626,"15min":0.621},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85999999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.67700000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0009999999999999,"5min":1.407,"15min":1.407},"last":0.747}]},{"osd":6,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.99299999999999999,"5min":1.05,"15min":1.05},"last":0.627},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78300000000000003,"5min":1.034,"15min":1.034},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71699999999999997,"15min":0.72999999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91500000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89600000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.82699999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.745}]},{"osd":2,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.80000000000000004,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86299999999999999,"5min":0.876,"15min":0.876},"last":0.496},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81100000000000005,"15min":0.82299999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.90800000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92000000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76700000000000002,"15min":0.78600000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.81200000000000006},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.86699999999999999,"15min":0.88900000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.53000000000000003,"15min":0.55400000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57099999999999995,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.57799999999999996,"15min":0.60199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.38400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.55400000000000005,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66000000000000003,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68400000000000005,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.61799999999999999,"15min":0.64200000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71699999999999997,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.437}]},{"osd":4,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.79900000000000004,"15min":0.81499999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98099999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.83899999999999997,"15min":0.85999999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.016,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95099999999999996,"5min":1.349,"15min":1.349},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98599999999999999,"5min":1.669,"15min":1.669},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.87,"15min":0.88700000000000001},"min":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1020000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.89800000000000002,"15min":0.92000000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0680000000000001,"5min":1.573,"15min":1.573},"last":0.503}]},{"osd":7,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.86599999999999999,"15min":0.88800000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.90400000000000003,"5min":1.528,"15min":1.528},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.874,"15min":0.89500000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98899999999999999,"5min":1.75,"15min":1.75},"last":0.48699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302668,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069172,"statfs":{"total":96636764160,"available":96326832128,"internally_reserved":0,"allocated":1314816,"data_stored":903767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42999999999999999,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.627},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435}]},{"osd":3,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.746,"15min":0.746},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.748,"15min":0.745},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91700000000000004,"5min":1.04,"15min":1.04},"last":0.75},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1,"5min":1.01,"15min":1.01},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.91200000000000003,"5min":0.999,"15min":0.999},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.432,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.55000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.442,"15min":0.442},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57699999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.63500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.502}]},{"osd":4,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.006,"5min":1.044,"15min":1.044},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.57399999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.93600000000000005,"5min":1.056,"15min":1.056},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79400000000000004,"15min":0.80400000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.86399999999999999,"5min":1.319,"15min":1.319},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.81799999999999995,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92200000000000004,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.92500000000000004,"5min":1.141,"15min":1.141},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.61099999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96499999999999997,"5min":1.282,"15min":1.282},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.96999999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.77500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0720000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.48899999999999999,"15min":0.502},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53700000000000003,"5min":1.772,"15min":1.772},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.55000000000000004,"5min":1.913,"15min":1.913},"last":0.374},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.61199999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.624,"5min":1.954,"15min":1.954},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.622,"15min":0.63900000000000001},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.58799999999999997,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.55800000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1619999999999999,"5min":1.282,"15min":1.282},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.55800000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.85599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86299999999999999,"15min":0.875},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2250000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.343,"15min":1.343},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83699999999999997},"min":{"1min":0.64100000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3109999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.206,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.143,"5min":1.214,"15min":1.214},"last":0.81000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97099999999999997,"5min":1.073,"15min":1.073},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":1.169,"15min":1.169},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.042,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92900000000000005,"5min":1.117,"15min":1.117},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89600000000000002,"5min":0.998,"15min":0.998},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.74199999999999999,"15min":0.752},"min":{"1min":0.501,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.751},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.505,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.196},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.75700000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81299999999999994,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.495},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.375,"15min":1.375},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":1.004,"15min":1.004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.82299999999999995,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1639999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.83799999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0209999999999999,"5min":1.466,"15min":1.466},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1339999999999999,"5min":1.218,"15min":1.218},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298008,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073832,"statfs":{"total":96636764160,"available":96331603968,"internally_reserved":0,"allocated":737280,"data_stored":320537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.872,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86399999999999999,"5min":1.028,"15min":1.028},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.84599999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0149999999999999,"5min":1.087,"15min":1.087},"last":0.505},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.278,"15min":1.278},"last":0.433}]},{"osd":4,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94299999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.45900000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85099999999999998,"5min":1.294,"15min":1.294},"last":0.59799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81699999999999995,"5min":1.006,"15min":1.006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":1.254,"15min":1.254},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.337,"15min":1.337},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91100000000000003,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:16:36.122 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:16:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.338+0000 7f312e39d700 1 Processor -- start 2023-10-06T05:16:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.338+0000 7f312e39d700 1 -- start start 2023-10-06T05:16:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.339+0000 7f312e39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31280c8050 0x7f312813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.339+0000 7f312e39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f312806aad0 0x7f312806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.339+0000 7f312e39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280c9cb0 con 0x7f312813faa0 2023-10-06T05:16:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.339+0000 7f312e39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31280ca270 con 0x7f312806aad0 2023-10-06T05:16:36.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.339+0000 7f312e39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312806b840 con 0x7f31280c8050 2023-10-06T05:16:36.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312db9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f312806aad0 0x7f312806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312d39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f312813faa0 0x7f31280c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58728/0 (socket says 172.21.15.33:58728) 2023-10-06T05:16:36.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312d39b700 1 -- 172.21.15.33:0/3005504822 learned_addr learned my addr 172.21.15.33:0/3005504822 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:36.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312cb9a700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31280c8050 0x7f312813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312db9c700 1 -- 172.21.15.33:0/3005504822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31280c8050 msgr2=0x7f312813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:36.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312db9c700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31280c8050 0x7f312813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312db9c700 1 -- 172.21.15.33:0/3005504822 >> v1:172.21.15.33:6789/0 conn(0x7f312813faa0 legacy=0x7f31280c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312db9c700 1 -- 172.21.15.33:0/3005504822 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312806bb20 con 0x7f312806aad0 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f311ffff700 1 -- 172.21.15.33:0/3005504822 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185937418 0 0) 0x7f31280c9cb0 con 0x7f312813faa0 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312cb9a700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31280c8050 0x7f312813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.340+0000 7f312db9c700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312806aad0 0x7f312806af50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3120009e60 tx=0x7f3120009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20962b344384bdc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f311ffff700 1 -- 172.21.15.33:0/3005504822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312000e070 con 0x7f312806aad0 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f311ffff700 1 -- 172.21.15.33:0/3005504822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312000bc30 con 0x7f312806aad0 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f311ffff700 1 -- 172.21.15.33:0/3005504822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120012430 con 0x7f312806aad0 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 -- 172.21.15.33:0/3005504822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312806aad0 msgr2=0x7f312806af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312806aad0 0x7f312806af50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3120009e60 tx=0x7f3120009780 comp rx=0 tx=0).stop 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 -- 172.21.15.33:0/3005504822 shutdown_connections 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312806aad0 0x7f312806af50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 --2- 172.21.15.33:0/3005504822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31280c8050 0x7f312813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 -- 172.21.15.33:0/3005504822 >> 172.21.15.33:0/3005504822 conn(0x7f312806d030 msgr2=0x7f312806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:36.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 -- 172.21.15.33:0/3005504822 shutdown_connections 2023-10-06T05:16:36.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.341+0000 7f312e39d700 1 -- 172.21.15.33:0/3005504822 wait complete. 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.343+0000 7f312e39d700 1 Processor -- start 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.343+0000 7f312e39d700 1 -- start start 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312e39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f312806aad0 0x7f3128155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312e39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 0x7f312816cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312e39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f312813faa0 0x7f3128157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312e39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312816d4c0 con 0x7f31280c8050 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312e39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312816d7a0 con 0x7f312813faa0 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312e39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3128157050 con 0x7f312806aad0 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312cb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 0x7f312816cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312cb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 0x7f312816cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52448/0 (socket says 172.21.15.33:52448) 2023-10-06T05:16:36.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312cb9a700 1 -- 172.21.15.33:0/1878673123 learned_addr learned my addr 172.21.15.33:0/1878673123 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312d39b700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f312806aad0 0x7f3128155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.344+0000 7f312db9c700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f312813faa0 0x7f3128157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312cb9a700 1 -- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f312806aad0 msgr2=0x7f3128155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312cb9a700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f312806aad0 0x7f3128155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312cb9a700 1 -- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f312813faa0 msgr2=0x7f3128157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312cb9a700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f312813faa0 0x7f3128157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312cb9a700 1 -- 172.21.15.33:0/1878673123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120009a20 con 0x7f31280c8050 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312cb9a700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 0x7f312816cb00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f311800e8d0 tx=0x7f3118008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118013410 con 0x7f31280c8050 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118013570 con 0x7f31280c8050 2023-10-06T05:16:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118013910 con 0x7f31280c8050 2023-10-06T05:16:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3128157330 con 0x7f31280c8050 2023-10-06T05:16:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.345+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31281599b0 con 0x7f31280c8050 2023-10-06T05:16:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.346+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3128156eb0 con 0x7f31280c8050 2023-10-06T05:16:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.347+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3118013d00 con 0x7f31280c8050 2023-10-06T05:16:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.348+0000 7f311d7fa700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f310c066f30 0x7f310c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.348+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3118084440 con 0x7f31280c8050 2023-10-06T05:16:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.348+0000 7f312d39b700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f310c066f30 0x7f310c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.349+0000 7f312d39b700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f310c066f30 0x7f310c0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3128167050 tx=0x7f3124007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.352+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3128156eb0 con 0x7f31280c8050 2023-10-06T05:16:36.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.547+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3128156eb0 con 0x7f310c066f30 2023-10-06T05:16:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.556+0000 7f311d7fa700 1 -- 172.21.15.33:0/1878673123 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310777 (secure 0 0 0) 0x7f3128156eb0 con 0x7f310c066f30 2023-10-06T05:16:36.563 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.560+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f310c066f30 msgr2=0x7f310c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f310c066f30 0x7f310c0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3128167050 tx=0x7f3124007f50 comp rx=0 tx=0).stop 2023-10-06T05:16:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 msgr2=0x7f312816cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 0x7f312816cb00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f311800e8d0 tx=0x7f3118008040 comp rx=0 tx=0).stop 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 shutdown_connections 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f310c066f30 0x7f310c0693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f312813faa0 0x7f3128157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f312806aad0 0x7f3128155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 --2- 172.21.15.33:0/1878673123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31280c8050 0x7f312816cb00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 >> 172.21.15.33:0/1878673123 conn(0x7f312806d030 msgr2=0x7f31280cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 shutdown_connections 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.561+0000 7f312e39d700 1 -- 172.21.15.33:0/1878673123 wait complete. 2023-10-06T05:16:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:16:36.581 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-10-06T05:16:35.845649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389208,"kb_used_data":6808,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752585512,"statfs":{"total":773094113280,"available":770647564288,"internally_reserved":0,"allocated":6971392,"data_stored":3695066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001255"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743090+0000","last_change":"2023-10-06T05:16:31.743090+0000","last_active":"2023-10-06T05:16:31.743090+0000","last_peered":"2023-10-06T05:16:31.743090+0000","last_clean":"2023-10-06T05:16:31.743090+0000","last_became_active":"2023-10-06T05:16:31.742824+0000","last_became_peered":"2023-10-06T05:16:31.742824+0000","last_unstale":"2023-10-06T05:16:31.743090+0000","last_undegraded":"2023-10-06T05:16:31.743090+0000","last_fullsized":"2023-10-06T05:16:31.743090+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:51.707463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741207+0000","last_change":"2023-10-06T05:16:31.741207+0000","last_active":"2023-10-06T05:16:31.741207+0000","last_peered":"2023-10-06T05:16:31.741207+0000","last_clean":"2023-10-06T05:16:31.741207+0000","last_became_active":"2023-10-06T05:16:31.740896+0000","last_became_peered":"2023-10-06T05:16:31.740896+0000","last_unstale":"2023-10-06T05:16:31.741207+0000","last_undegraded":"2023-10-06T05:16:31.741207+0000","last_fullsized":"2023-10-06T05:16:31.741207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:10.574150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.751006+0000","last_change":"2023-10-06T05:16:31.751006+0000","last_active":"2023-10-06T05:16:31.751006+0000","last_peered":"2023-10-06T05:16:31.751006+0000","last_clean":"2023-10-06T05:16:31.751006+0000","last_became_active":"2023-10-06T05:16:31.750750+0000","last_became_peered":"2023-10-06T05:16:31.750750+0000","last_unstale":"2023-10-06T05:16:31.751006+0000","last_undegraded":"2023-10-06T05:16:31.751006+0000","last_fullsized":"2023-10-06T05:16:31.751006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:31.817743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743376+0000","last_change":"2023-10-06T05:16:31.743376+0000","last_active":"2023-10-06T05:16:31.743376+0000","last_peered":"2023-10-06T05:16:31.743376+0000","last_clean":"2023-10-06T05:16:31.743376+0000","last_became_active":"2023-10-06T05:16:31.741593+0000","last_became_peered":"2023-10-06T05:16:31.741593+0000","last_unstale":"2023-10-06T05:16:31.743376+0000","last_undegraded":"2023-10-06T05:16:31.743376+0000","last_fullsized":"2023-10-06T05:16:31.743376+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:19.950359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735851+0000","last_change":"2023-10-06T05:16:31.735851+0000","last_active":"2023-10-06T05:16:31.735851+0000","last_peered":"2023-10-06T05:16:31.735851+0000","last_clean":"2023-10-06T05:16:31.735851+0000","last_became_active":"2023-10-06T05:16:31.735563+0000","last_became_peered":"2023-10-06T05:16:31.735563+0000","last_unstale":"2023-10-06T05:16:31.735851+0000","last_undegraded":"2023-10-06T05:16:31.735851+0000","last_fullsized":"2023-10-06T05:16:31.735851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:06.572971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737822+0000","last_change":"2023-10-06T05:16:31.737822+0000","last_active":"2023-10-06T05:16:31.737822+0000","last_peered":"2023-10-06T05:16:31.737822+0000","last_clean":"2023-10-06T05:16:31.737822+0000","last_became_active":"2023-10-06T05:16:31.737523+0000","last_became_peered":"2023-10-06T05:16:31.737523+0000","last_unstale":"2023-10-06T05:16:31.737822+0000","last_undegraded":"2023-10-06T05:16:31.737822+0000","last_fullsized":"2023-10-06T05:16:31.737822+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:36.127945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735182+0000","last_change":"2023-10-06T05:16:31.735182+0000","last_active":"2023-10-06T05:16:31.735182+0000","last_peered":"2023-10-06T05:16:31.735182+0000","last_clean":"2023-10-06T05:16:31.735182+0000","last_became_active":"2023-10-06T05:16:31.734906+0000","last_became_peered":"2023-10-06T05:16:31.734906+0000","last_unstale":"2023-10-06T05:16:31.735182+0000","last_undegraded":"2023-10-06T05:16:31.735182+0000","last_fullsized":"2023-10-06T05:16:31.735182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:15.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739783+0000","last_change":"2023-10-06T05:16:31.739783+0000","last_active":"2023-10-06T05:16:31.739783+0000","last_peered":"2023-10-06T05:16:31.739783+0000","last_clean":"2023-10-06T05:16:31.739783+0000","last_became_active":"2023-10-06T05:16:31.739491+0000","last_became_peered":"2023-10-06T05:16:31.739491+0000","last_unstale":"2023-10-06T05:16:31.739783+0000","last_undegraded":"2023-10-06T05:16:31.739783+0000","last_fullsized":"2023-10-06T05:16:31.739783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:19.950359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.752032+0000","last_change":"2023-10-06T05:16:31.752032+0000","last_active":"2023-10-06T05:16:31.752032+0000","last_peered":"2023-10-06T05:16:31.752032+0000","last_clean":"2023-10-06T05:16:31.752032+0000","last_became_active":"2023-10-06T05:16:31.751719+0000","last_became_peered":"2023-10-06T05:16:31.751719+0000","last_unstale":"2023-10-06T05:16:31.752032+0000","last_undegraded":"2023-10-06T05:16:31.752032+0000","last_fullsized":"2023-10-06T05:16:31.752032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:26.456768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.745966+0000","last_change":"2023-10-06T05:16:31.745966+0000","last_active":"2023-10-06T05:16:31.745966+0000","last_peered":"2023-10-06T05:16:31.745966+0000","last_clean":"2023-10-06T05:16:31.745966+0000","last_became_active":"2023-10-06T05:16:31.745659+0000","last_became_peered":"2023-10-06T05:16:31.745659+0000","last_unstale":"2023-10-06T05:16:31.745966+0000","last_undegraded":"2023-10-06T05:16:31.745966+0000","last_fullsized":"2023-10-06T05:16:31.745966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:43.017359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737114+0000","last_change":"2023-10-06T05:16:31.737114+0000","last_active":"2023-10-06T05:16:31.737114+0000","last_peered":"2023-10-06T05:16:31.737114+0000","last_clean":"2023-10-06T05:16:31.737114+0000","last_became_active":"2023-10-06T05:16:31.736830+0000","last_became_peered":"2023-10-06T05:16:31.736830+0000","last_unstale":"2023-10-06T05:16:31.737114+0000","last_undegraded":"2023-10-06T05:16:31.737114+0000","last_fullsized":"2023-10-06T05:16:31.737114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:33.246474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740884+0000","last_change":"2023-10-06T05:16:31.740884+0000","last_active":"2023-10-06T05:16:31.740884+0000","last_peered":"2023-10-06T05:16:31.740884+0000","last_clean":"2023-10-06T05:16:31.740884+0000","last_became_active":"2023-10-06T05:16:31.740364+0000","last_became_peered":"2023-10-06T05:16:31.740364+0000","last_unstale":"2023-10-06T05:16:31.740884+0000","last_undegraded":"2023-10-06T05:16:31.740884+0000","last_fullsized":"2023-10-06T05:16:31.740884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:34.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.734272+0000","last_change":"2023-10-06T05:16:31.734272+0000","last_active":"2023-10-06T05:16:31.734272+0000","last_peered":"2023-10-06T05:16:31.734272+0000","last_clean":"2023-10-06T05:16:31.734272+0000","last_became_active":"2023-10-06T05:16:31.733835+0000","last_became_peered":"2023-10-06T05:16:31.733835+0000","last_unstale":"2023-10-06T05:16:31.734272+0000","last_undegraded":"2023-10-06T05:16:31.734272+0000","last_fullsized":"2023-10-06T05:16:31.734272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:33.246474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.750128+0000","last_change":"2023-10-06T05:16:31.750128+0000","last_active":"2023-10-06T05:16:31.750128+0000","last_peered":"2023-10-06T05:16:31.750128+0000","last_clean":"2023-10-06T05:16:31.750128+0000","last_became_active":"2023-10-06T05:16:31.749844+0000","last_became_peered":"2023-10-06T05:16:31.749844+0000","last_unstale":"2023-10-06T05:16:31.750128+0000","last_undegraded":"2023-10-06T05:16:31.750128+0000","last_fullsized":"2023-10-06T05:16:31.750128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:05.113348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.744622+0000","last_change":"2023-10-06T05:16:31.744622+0000","last_active":"2023-10-06T05:16:31.744622+0000","last_peered":"2023-10-06T05:16:31.744622+0000","last_clean":"2023-10-06T05:16:31.744622+0000","last_became_active":"2023-10-06T05:16:31.744363+0000","last_became_peered":"2023-10-06T05:16:31.744363+0000","last_unstale":"2023-10-06T05:16:31.744622+0000","last_undegraded":"2023-10-06T05:16:31.744622+0000","last_fullsized":"2023-10-06T05:16:31.744622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:34.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735101+0000","last_change":"2023-10-06T05:16:31.735101+0000","last_active":"2023-10-06T05:16:31.735101+0000","last_peered":"2023-10-06T05:16:31.735101+0000","last_clean":"2023-10-06T05:16:31.735101+0000","last_became_active":"2023-10-06T05:16:31.734802+0000","last_became_peered":"2023-10-06T05:16:31.734802+0000","last_unstale":"2023-10-06T05:16:31.735101+0000","last_undegraded":"2023-10-06T05:16:31.735101+0000","last_fullsized":"2023-10-06T05:16:31.735101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:47.915171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732034+0000","last_change":"2023-10-06T05:16:31.732034+0000","last_active":"2023-10-06T05:16:31.732034+0000","last_peered":"2023-10-06T05:16:31.732034+0000","last_clean":"2023-10-06T05:16:31.732034+0000","last_became_active":"2023-10-06T05:16:31.731781+0000","last_became_peered":"2023-10-06T05:16:31.731781+0000","last_unstale":"2023-10-06T05:16:31.732034+0000","last_undegraded":"2023-10-06T05:16:31.732034+0000","last_fullsized":"2023-10-06T05:16:31.732034+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:36.864601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743248+0000","last_change":"2023-10-06T05:16:31.743248+0000","last_active":"2023-10-06T05:16:31.743248+0000","last_peered":"2023-10-06T05:16:31.743248+0000","last_clean":"2023-10-06T05:16:31.743248+0000","last_became_active":"2023-10-06T05:16:31.742045+0000","last_became_peered":"2023-10-06T05:16:31.742045+0000","last_unstale":"2023-10-06T05:16:31.743248+0000","last_undegraded":"2023-10-06T05:16:31.743248+0000","last_fullsized":"2023-10-06T05:16:31.743248+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:48.881030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732378+0000","last_change":"2023-10-06T05:16:31.732378+0000","last_active":"2023-10-06T05:16:31.732378+0000","last_peered":"2023-10-06T05:16:31.732378+0000","last_clean":"2023-10-06T05:16:31.732378+0000","last_became_active":"2023-10-06T05:16:31.732110+0000","last_became_peered":"2023-10-06T05:16:31.732110+0000","last_unstale":"2023-10-06T05:16:31.732378+0000","last_undegraded":"2023-10-06T05:16:31.732378+0000","last_fullsized":"2023-10-06T05:16:31.732378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:08.607604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732373+0000","last_change":"2023-10-06T05:16:31.732373+0000","last_active":"2023-10-06T05:16:31.732373+0000","last_peered":"2023-10-06T05:16:31.732373+0000","last_clean":"2023-10-06T05:16:31.732373+0000","last_became_active":"2023-10-06T05:16:31.732117+0000","last_became_peered":"2023-10-06T05:16:31.732117+0000","last_unstale":"2023-10-06T05:16:31.732373+0000","last_undegraded":"2023-10-06T05:16:31.732373+0000","last_fullsized":"2023-10-06T05:16:31.732373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:58.652264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.738202+0000","last_change":"2023-10-06T05:16:31.738202+0000","last_active":"2023-10-06T05:16:31.738202+0000","last_peered":"2023-10-06T05:16:31.738202+0000","last_clean":"2023-10-06T05:16:31.738202+0000","last_became_active":"2023-10-06T05:16:31.737795+0000","last_became_peered":"2023-10-06T05:16:31.737795+0000","last_unstale":"2023-10-06T05:16:31.738202+0000","last_undegraded":"2023-10-06T05:16:31.738202+0000","last_fullsized":"2023-10-06T05:16:31.738202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:58.237697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735215+0000","last_change":"2023-10-06T05:16:31.735215+0000","last_active":"2023-10-06T05:16:31.735215+0000","last_peered":"2023-10-06T05:16:31.735215+0000","last_clean":"2023-10-06T05:16:31.735215+0000","last_became_active":"2023-10-06T05:16:31.734936+0000","last_became_peered":"2023-10-06T05:16:31.734936+0000","last_unstale":"2023-10-06T05:16:31.735215+0000","last_undegraded":"2023-10-06T05:16:31.735215+0000","last_fullsized":"2023-10-06T05:16:31.735215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:57.549236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.733059+0000","last_change":"2023-10-06T05:16:31.733059+0000","last_active":"2023-10-06T05:16:31.733059+0000","last_peered":"2023-10-06T05:16:31.733059+0000","last_clean":"2023-10-06T05:16:31.733059+0000","last_became_active":"2023-10-06T05:16:31.732798+0000","last_became_peered":"2023-10-06T05:16:31.732798+0000","last_unstale":"2023-10-06T05:16:31.733059+0000","last_undegraded":"2023-10-06T05:16:31.733059+0000","last_fullsized":"2023-10-06T05:16:31.733059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:37.068605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735301+0000","last_change":"2023-10-06T05:16:31.735301+0000","last_active":"2023-10-06T05:16:31.735301+0000","last_peered":"2023-10-06T05:16:31.735301+0000","last_clean":"2023-10-06T05:16:31.735301+0000","last_became_active":"2023-10-06T05:16:31.735050+0000","last_became_peered":"2023-10-06T05:16:31.735050+0000","last_unstale":"2023-10-06T05:16:31.735301+0000","last_undegraded":"2023-10-06T05:16:31.735301+0000","last_fullsized":"2023-10-06T05:16:31.735301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:33.246474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735397+0000","last_change":"2023-10-06T05:16:31.735397+0000","last_active":"2023-10-06T05:16:31.735397+0000","last_peered":"2023-10-06T05:16:31.735397+0000","last_clean":"2023-10-06T05:16:31.735397+0000","last_became_active":"2023-10-06T05:16:31.735120+0000","last_became_peered":"2023-10-06T05:16:31.735120+0000","last_unstale":"2023-10-06T05:16:31.735397+0000","last_undegraded":"2023-10-06T05:16:31.735397+0000","last_fullsized":"2023-10-06T05:16:31.735397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:36.864601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739844+0000","last_change":"2023-10-06T05:16:31.739844+0000","last_active":"2023-10-06T05:16:31.739844+0000","last_peered":"2023-10-06T05:16:31.739844+0000","last_clean":"2023-10-06T05:16:31.739844+0000","last_became_active":"2023-10-06T05:16:31.739560+0000","last_became_peered":"2023-10-06T05:16:31.739560+0000","last_unstale":"2023-10-06T05:16:31.739844+0000","last_undegraded":"2023-10-06T05:16:31.739844+0000","last_fullsized":"2023-10-06T05:16:31.739844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:10.574150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737270+0000","last_change":"2023-10-06T05:16:31.737270+0000","last_active":"2023-10-06T05:16:31.737270+0000","last_peered":"2023-10-06T05:16:31.737270+0000","last_clean":"2023-10-06T05:16:31.737270+0000","last_became_active":"2023-10-06T05:16:31.734035+0000","last_became_peered":"2023-10-06T05:16:31.734035+0000","last_unstale":"2023-10-06T05:16:31.737270+0000","last_undegraded":"2023-10-06T05:16:31.737270+0000","last_fullsized":"2023-10-06T05:16:31.737270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:18.230946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.733722+0000","last_change":"2023-10-06T05:16:31.733722+0000","last_active":"2023-10-06T05:16:31.733722+0000","last_peered":"2023-10-06T05:16:31.733722+0000","last_clean":"2023-10-06T05:16:31.733722+0000","last_became_active":"2023-10-06T05:16:31.733455+0000","last_became_peered":"2023-10-06T05:16:31.733455+0000","last_unstale":"2023-10-06T05:16:31.733722+0000","last_undegraded":"2023-10-06T05:16:31.733722+0000","last_fullsized":"2023-10-06T05:16:31.733722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:24.781580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.743127+0000","last_change":"2023-10-06T05:16:31.743127+0000","last_active":"2023-10-06T05:16:31.743127+0000","last_peered":"2023-10-06T05:16:31.743127+0000","last_clean":"2023-10-06T05:16:31.743127+0000","last_became_active":"2023-10-06T05:16:31.742862+0000","last_became_peered":"2023-10-06T05:16:31.742862+0000","last_unstale":"2023-10-06T05:16:31.743127+0000","last_undegraded":"2023-10-06T05:16:31.743127+0000","last_fullsized":"2023-10-06T05:16:31.743127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:58.848697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.738429+0000","last_change":"2023-10-06T05:16:31.738429+0000","last_active":"2023-10-06T05:16:31.738429+0000","last_peered":"2023-10-06T05:16:31.738429+0000","last_clean":"2023-10-06T05:16:31.738429+0000","last_became_active":"2023-10-06T05:16:31.737049+0000","last_became_peered":"2023-10-06T05:16:31.737049+0000","last_unstale":"2023-10-06T05:16:31.738429+0000","last_undegraded":"2023-10-06T05:16:31.738429+0000","last_fullsized":"2023-10-06T05:16:31.738429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:48.881030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.736253+0000","last_change":"2023-10-06T05:16:31.736253+0000","last_active":"2023-10-06T05:16:31.736253+0000","last_peered":"2023-10-06T05:16:31.736253+0000","last_clean":"2023-10-06T05:16:31.736253+0000","last_became_active":"2023-10-06T05:16:31.735438+0000","last_became_peered":"2023-10-06T05:16:31.735438+0000","last_unstale":"2023-10-06T05:16:31.736253+0000","last_undegraded":"2023-10-06T05:16:31.736253+0000","last_fullsized":"2023-10-06T05:16:31.736253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:34.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.729996+0000","last_change":"2023-10-06T05:16:31.729996+0000","last_active":"2023-10-06T05:16:31.729996+0000","last_peered":"2023-10-06T05:16:31.729996+0000","last_clean":"2023-10-06T05:16:31.729996+0000","last_became_active":"2023-10-06T05:16:31.729760+0000","last_became_peered":"2023-10-06T05:16:31.729760+0000","last_unstale":"2023-10-06T05:16:31.729996+0000","last_undegraded":"2023-10-06T05:16:31.729996+0000","last_fullsized":"2023-10-06T05:16:31.729996+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.397759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.742987+0000","last_change":"2023-10-06T05:16:31.742987+0000","last_active":"2023-10-06T05:16:31.742987+0000","last_peered":"2023-10-06T05:16:31.742987+0000","last_clean":"2023-10-06T05:16:31.742987+0000","last_became_active":"2023-10-06T05:16:31.742674+0000","last_became_peered":"2023-10-06T05:16:31.742674+0000","last_unstale":"2023-10-06T05:16:31.742987+0000","last_undegraded":"2023-10-06T05:16:31.742987+0000","last_fullsized":"2023-10-06T05:16:31.742987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:46.338128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.758701+0000","last_change":"2023-10-06T05:16:30.741190+0000","last_active":"2023-10-06T05:16:31.758701+0000","last_peered":"2023-10-06T05:16:31.758701+0000","last_clean":"2023-10-06T05:16:31.758701+0000","last_became_active":"2023-10-06T05:16:30.740682+0000","last_became_peered":"2023-10-06T05:16:30.740682+0000","last_unstale":"2023-10-06T05:16:31.758701+0000","last_undegraded":"2023-10-06T05:16:31.758701+0000","last_fullsized":"2023-10-06T05:16:31.758701+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.746933+0000","last_change":"2023-10-06T05:16:31.746933+0000","last_active":"2023-10-06T05:16:31.746933+0000","last_peered":"2023-10-06T05:16:31.746933+0000","last_clean":"2023-10-06T05:16:31.746933+0000","last_became_active":"2023-10-06T05:16:31.746609+0000","last_became_peered":"2023-10-06T05:16:31.746609+0000","last_unstale":"2023-10-06T05:16:31.746933+0000","last_undegraded":"2023-10-06T05:16:31.746933+0000","last_fullsized":"2023-10-06T05:16:31.746933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:58.237697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.716589+0000","last_change":"2023-10-06T05:16:30.732833+0000","last_active":"2023-10-06T05:16:31.716589+0000","last_peered":"2023-10-06T05:16:31.716589+0000","last_clean":"2023-10-06T05:16:31.716589+0000","last_became_active":"2023-10-06T05:16:30.732003+0000","last_became_peered":"2023-10-06T05:16:30.732003+0000","last_unstale":"2023-10-06T05:16:31.716589+0000","last_undegraded":"2023-10-06T05:16:31.716589+0000","last_fullsized":"2023-10-06T05:16:31.716589+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:04.104858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735391+0000","last_change":"2023-10-06T05:16:31.735391+0000","last_active":"2023-10-06T05:16:31.735391+0000","last_peered":"2023-10-06T05:16:31.735391+0000","last_clean":"2023-10-06T05:16:31.735391+0000","last_became_active":"2023-10-06T05:16:31.735110+0000","last_became_peered":"2023-10-06T05:16:31.735110+0000","last_unstale":"2023-10-06T05:16:31.735391+0000","last_undegraded":"2023-10-06T05:16:31.735391+0000","last_fullsized":"2023-10-06T05:16:31.735391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:57.549236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.749881+0000","last_change":"2023-10-06T05:16:30.728961+0000","last_active":"2023-10-06T05:16:31.749881+0000","last_peered":"2023-10-06T05:16:31.749881+0000","last_clean":"2023-10-06T05:16:31.749881+0000","last_became_active":"2023-10-06T05:16:30.728658+0000","last_became_peered":"2023-10-06T05:16:30.728658+0000","last_unstale":"2023-10-06T05:16:31.749881+0000","last_undegraded":"2023-10-06T05:16:31.749881+0000","last_fullsized":"2023-10-06T05:16:31.749881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:42.008870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.746057+0000","last_change":"2023-10-06T05:16:31.746057+0000","last_active":"2023-10-06T05:16:31.746057+0000","last_peered":"2023-10-06T05:16:31.746057+0000","last_clean":"2023-10-06T05:16:31.746057+0000","last_became_active":"2023-10-06T05:16:31.745747+0000","last_became_peered":"2023-10-06T05:16:31.745747+0000","last_unstale":"2023-10-06T05:16:31.746057+0000","last_undegraded":"2023-10-06T05:16:31.746057+0000","last_fullsized":"2023-10-06T05:16:31.746057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:50.174715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.751527+0000","last_change":"2023-10-06T05:16:30.735164+0000","last_active":"2023-10-06T05:16:31.751527+0000","last_peered":"2023-10-06T05:16:31.751527+0000","last_clean":"2023-10-06T05:16:31.751527+0000","last_became_active":"2023-10-06T05:16:30.734864+0000","last_became_peered":"2023-10-06T05:16:30.734864+0000","last_unstale":"2023-10-06T05:16:31.751527+0000","last_undegraded":"2023-10-06T05:16:31.751527+0000","last_fullsized":"2023-10-06T05:16:31.751527+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:33.098566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740389+0000","last_change":"2023-10-06T05:16:31.740389+0000","last_active":"2023-10-06T05:16:31.740389+0000","last_peered":"2023-10-06T05:16:31.740389+0000","last_clean":"2023-10-06T05:16:31.740389+0000","last_became_active":"2023-10-06T05:16:31.739307+0000","last_became_peered":"2023-10-06T05:16:31.739307+0000","last_unstale":"2023-10-06T05:16:31.740389+0000","last_undegraded":"2023-10-06T05:16:31.740389+0000","last_fullsized":"2023-10-06T05:16:31.740389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.375266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.719386+0000","last_change":"2023-10-06T05:16:30.736788+0000","last_active":"2023-10-06T05:16:31.719386+0000","last_peered":"2023-10-06T05:16:31.719386+0000","last_clean":"2023-10-06T05:16:31.719386+0000","last_became_active":"2023-10-06T05:16:30.736509+0000","last_became_peered":"2023-10-06T05:16:30.736509+0000","last_unstale":"2023-10-06T05:16:31.719386+0000","last_undegraded":"2023-10-06T05:16:31.719386+0000","last_fullsized":"2023-10-06T05:16:31.719386+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:57.840207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.736892+0000","last_change":"2023-10-06T05:16:31.736892+0000","last_active":"2023-10-06T05:16:31.736892+0000","last_peered":"2023-10-06T05:16:31.736892+0000","last_clean":"2023-10-06T05:16:31.736892+0000","last_became_active":"2023-10-06T05:16:31.736622+0000","last_became_peered":"2023-10-06T05:16:31.736622+0000","last_unstale":"2023-10-06T05:16:31.736892+0000","last_undegraded":"2023-10-06T05:16:31.736892+0000","last_fullsized":"2023-10-06T05:16:31.736892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:18.230946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.719579+0000","last_change":"2023-10-06T05:16:30.731764+0000","last_active":"2023-10-06T05:16:31.719579+0000","last_peered":"2023-10-06T05:16:31.719579+0000","last_clean":"2023-10-06T05:16:31.719579+0000","last_became_active":"2023-10-06T05:16:30.731454+0000","last_became_peered":"2023-10-06T05:16:30.731454+0000","last_unstale":"2023-10-06T05:16:31.719579+0000","last_undegraded":"2023-10-06T05:16:31.719579+0000","last_fullsized":"2023-10-06T05:16:31.719579+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:57.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.730126+0000","last_change":"2023-10-06T05:16:31.730126+0000","last_active":"2023-10-06T05:16:31.730126+0000","last_peered":"2023-10-06T05:16:31.730126+0000","last_clean":"2023-10-06T05:16:31.730126+0000","last_became_active":"2023-10-06T05:16:31.729644+0000","last_became_peered":"2023-10-06T05:16:31.729644+0000","last_unstale":"2023-10-06T05:16:31.730126+0000","last_undegraded":"2023-10-06T05:16:31.730126+0000","last_fullsized":"2023-10-06T05:16:31.730126+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:12.346621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.741242+0000","last_change":"2023-10-06T05:16:30.738916+0000","last_active":"2023-10-06T05:16:30.741242+0000","last_peered":"2023-10-06T05:16:30.741242+0000","last_clean":"2023-10-06T05:16:30.741242+0000","last_became_active":"2023-10-06T05:16:30.738650+0000","last_became_peered":"2023-10-06T05:16:30.738650+0000","last_unstale":"2023-10-06T05:16:30.741242+0000","last_undegraded":"2023-10-06T05:16:30.741242+0000","last_fullsized":"2023-10-06T05:16:30.741242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:47.872540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.729217+0000","last_change":"2023-10-06T05:16:31.729217+0000","last_active":"2023-10-06T05:16:31.729217+0000","last_peered":"2023-10-06T05:16:31.729217+0000","last_clean":"2023-10-06T05:16:31.729217+0000","last_became_active":"2023-10-06T05:16:31.728940+0000","last_became_peered":"2023-10-06T05:16:31.728940+0000","last_unstale":"2023-10-06T05:16:31.729217+0000","last_undegraded":"2023-10-06T05:16:31.729217+0000","last_fullsized":"2023-10-06T05:16:31.729217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.397759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.736725+0000","last_change":"2023-10-06T05:16:30.730313+0000","last_active":"2023-10-06T05:16:30.736725+0000","last_peered":"2023-10-06T05:16:30.736725+0000","last_clean":"2023-10-06T05:16:30.736725+0000","last_became_active":"2023-10-06T05:16:30.730043+0000","last_became_peered":"2023-10-06T05:16:30.730043+0000","last_unstale":"2023-10-06T05:16:30.736725+0000","last_undegraded":"2023-10-06T05:16:30.736725+0000","last_fullsized":"2023-10-06T05:16:30.736725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:14.879784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732519+0000","last_change":"2023-10-06T05:16:31.732519+0000","last_active":"2023-10-06T05:16:31.732519+0000","last_peered":"2023-10-06T05:16:31.732519+0000","last_clean":"2023-10-06T05:16:31.732519+0000","last_became_active":"2023-10-06T05:16:31.732233+0000","last_became_peered":"2023-10-06T05:16:31.732233+0000","last_unstale":"2023-10-06T05:16:31.732519+0000","last_undegraded":"2023-10-06T05:16:31.732519+0000","last_fullsized":"2023-10-06T05:16:31.732519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:23.009115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.744537+0000","last_change":"2023-10-06T05:16:30.741247+0000","last_active":"2023-10-06T05:16:30.744537+0000","last_peered":"2023-10-06T05:16:30.744537+0000","last_clean":"2023-10-06T05:16:30.744537+0000","last_became_active":"2023-10-06T05:16:30.740896+0000","last_became_peered":"2023-10-06T05:16:30.740896+0000","last_unstale":"2023-10-06T05:16:30.744537+0000","last_undegraded":"2023-10-06T05:16:30.744537+0000","last_fullsized":"2023-10-06T05:16:30.744537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:22.000626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740432+0000","last_change":"2023-10-06T05:16:31.740432+0000","last_active":"2023-10-06T05:16:31.740432+0000","last_peered":"2023-10-06T05:16:31.740432+0000","last_clean":"2023-10-06T05:16:31.740432+0000","last_became_active":"2023-10-06T05:16:31.740166+0000","last_became_peered":"2023-10-06T05:16:31.740166+0000","last_unstale":"2023-10-06T05:16:31.740432+0000","last_undegraded":"2023-10-06T05:16:31.740432+0000","last_fullsized":"2023-10-06T05:16:31.740432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:07.515410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.735189+0000","last_change":"2023-10-06T05:16:30.733275+0000","last_active":"2023-10-06T05:16:30.735189+0000","last_peered":"2023-10-06T05:16:30.735189+0000","last_clean":"2023-10-06T05:16:30.735189+0000","last_became_active":"2023-10-06T05:16:30.733021+0000","last_became_peered":"2023-10-06T05:16:30.733021+0000","last_unstale":"2023-10-06T05:16:30.735189+0000","last_undegraded":"2023-10-06T05:16:30.735189+0000","last_fullsized":"2023-10-06T05:16:30.735189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:19.056144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.747476+0000","last_change":"2023-10-06T05:16:31.747476+0000","last_active":"2023-10-06T05:16:31.747476+0000","last_peered":"2023-10-06T05:16:31.747476+0000","last_clean":"2023-10-06T05:16:31.747476+0000","last_became_active":"2023-10-06T05:16:31.744118+0000","last_became_peered":"2023-10-06T05:16:31.744118+0000","last_unstale":"2023-10-06T05:16:31.747476+0000","last_undegraded":"2023-10-06T05:16:31.747476+0000","last_fullsized":"2023-10-06T05:16:31.747476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:59.831684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.754096+0000","last_change":"2023-10-06T05:16:30.742192+0000","last_active":"2023-10-06T05:16:31.754096+0000","last_peered":"2023-10-06T05:16:31.754096+0000","last_clean":"2023-10-06T05:16:31.754096+0000","last_became_active":"2023-10-06T05:16:30.741945+0000","last_became_peered":"2023-10-06T05:16:30.741945+0000","last_unstale":"2023-10-06T05:16:31.754096+0000","last_undegraded":"2023-10-06T05:16:31.754096+0000","last_fullsized":"2023-10-06T05:16:31.754096+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:37.425290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.732651+0000","last_change":"2023-10-06T05:16:31.732651+0000","last_active":"2023-10-06T05:16:31.732651+0000","last_peered":"2023-10-06T05:16:31.732651+0000","last_clean":"2023-10-06T05:16:31.732651+0000","last_became_active":"2023-10-06T05:16:31.732329+0000","last_became_peered":"2023-10-06T05:16:31.732329+0000","last_unstale":"2023-10-06T05:16:31.732651+0000","last_undegraded":"2023-10-06T05:16:31.732651+0000","last_fullsized":"2023-10-06T05:16:31.732651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:10.574150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.762350+0000","last_change":"2023-10-06T05:16:30.734398+0000","last_active":"2023-10-06T05:16:31.762350+0000","last_peered":"2023-10-06T05:16:31.762350+0000","last_clean":"2023-10-06T05:16:31.762350+0000","last_became_active":"2023-10-06T05:16:30.734116+0000","last_became_peered":"2023-10-06T05:16:30.734116+0000","last_unstale":"2023-10-06T05:16:31.762350+0000","last_undegraded":"2023-10-06T05:16:31.762350+0000","last_fullsized":"2023-10-06T05:16:31.762350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:11.338131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739179+0000","last_change":"2023-10-06T05:16:31.739179+0000","last_active":"2023-10-06T05:16:31.739179+0000","last_peered":"2023-10-06T05:16:31.739179+0000","last_clean":"2023-10-06T05:16:31.739179+0000","last_became_active":"2023-10-06T05:16:31.738813+0000","last_became_peered":"2023-10-06T05:16:31.738813+0000","last_unstale":"2023-10-06T05:16:31.739179+0000","last_undegraded":"2023-10-06T05:16:31.739179+0000","last_fullsized":"2023-10-06T05:16:31.739179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:26.456768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.761628+0000","last_change":"2023-10-06T05:16:30.739499+0000","last_active":"2023-10-06T05:16:31.761628+0000","last_peered":"2023-10-06T05:16:31.761628+0000","last_clean":"2023-10-06T05:16:31.761628+0000","last_became_active":"2023-10-06T05:16:30.739223+0000","last_became_peered":"2023-10-06T05:16:30.739223+0000","last_unstale":"2023-10-06T05:16:31.761628+0000","last_undegraded":"2023-10-06T05:16:31.761628+0000","last_fullsized":"2023-10-06T05:16:31.761628+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:08.111445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.735292+0000","last_change":"2023-10-06T05:16:31.735292+0000","last_active":"2023-10-06T05:16:31.735292+0000","last_peered":"2023-10-06T05:16:31.735292+0000","last_clean":"2023-10-06T05:16:31.735292+0000","last_became_active":"2023-10-06T05:16:31.734566+0000","last_became_peered":"2023-10-06T05:16:31.734566+0000","last_unstale":"2023-10-06T05:16:31.735292+0000","last_undegraded":"2023-10-06T05:16:31.735292+0000","last_fullsized":"2023-10-06T05:16:31.735292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:15.070344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734374+0000","last_change":"2023-10-06T05:16:30.731789+0000","last_active":"2023-10-06T05:16:30.734374+0000","last_peered":"2023-10-06T05:16:30.734374+0000","last_clean":"2023-10-06T05:16:30.734374+0000","last_became_active":"2023-10-06T05:16:30.731455+0000","last_became_peered":"2023-10-06T05:16:30.731455+0000","last_unstale":"2023-10-06T05:16:30.734374+0000","last_undegraded":"2023-10-06T05:16:30.734374+0000","last_fullsized":"2023-10-06T05:16:30.734374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:55.968968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.731174+0000","last_change":"2023-10-06T05:16:31.731174+0000","last_active":"2023-10-06T05:16:31.731174+0000","last_peered":"2023-10-06T05:16:31.731174+0000","last_clean":"2023-10-06T05:16:31.731174+0000","last_became_active":"2023-10-06T05:16:31.730875+0000","last_became_peered":"2023-10-06T05:16:31.730875+0000","last_unstale":"2023-10-06T05:16:31.731174+0000","last_undegraded":"2023-10-06T05:16:31.731174+0000","last_fullsized":"2023-10-06T05:16:31.731174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:53.375917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.744141+0000","last_change":"2023-10-06T05:16:30.740289+0000","last_active":"2023-10-06T05:16:30.744141+0000","last_peered":"2023-10-06T05:16:30.744141+0000","last_clean":"2023-10-06T05:16:30.744141+0000","last_became_active":"2023-10-06T05:16:30.740024+0000","last_became_peered":"2023-10-06T05:16:30.740024+0000","last_unstale":"2023-10-06T05:16:30.744141+0000","last_undegraded":"2023-10-06T05:16:30.744141+0000","last_fullsized":"2023-10-06T05:16:30.744141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:24.724584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741576+0000","last_change":"2023-10-06T05:16:31.741576+0000","last_active":"2023-10-06T05:16:31.741576+0000","last_peered":"2023-10-06T05:16:31.741576+0000","last_clean":"2023-10-06T05:16:31.741576+0000","last_became_active":"2023-10-06T05:16:31.741225+0000","last_became_peered":"2023-10-06T05:16:31.741225+0000","last_unstale":"2023-10-06T05:16:31.741576+0000","last_undegraded":"2023-10-06T05:16:31.741576+0000","last_fullsized":"2023-10-06T05:16:31.741576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:15.070344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.758673+0000","last_change":"2023-10-06T05:16:30.736552+0000","last_active":"2023-10-06T05:16:31.758673+0000","last_peered":"2023-10-06T05:16:31.758673+0000","last_clean":"2023-10-06T05:16:31.758673+0000","last_became_active":"2023-10-06T05:16:30.736295+0000","last_became_peered":"2023-10-06T05:16:30.736295+0000","last_unstale":"2023-10-06T05:16:31.758673+0000","last_undegraded":"2023-10-06T05:16:31.758673+0000","last_fullsized":"2023-10-06T05:16:31.758673+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:05.392771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.739104+0000","last_change":"2023-10-06T05:16:31.739104+0000","last_active":"2023-10-06T05:16:31.739104+0000","last_peered":"2023-10-06T05:16:31.739104+0000","last_clean":"2023-10-06T05:16:31.739104+0000","last_became_active":"2023-10-06T05:16:31.738815+0000","last_became_peered":"2023-10-06T05:16:31.738815+0000","last_unstale":"2023-10-06T05:16:31.739104+0000","last_undegraded":"2023-10-06T05:16:31.739104+0000","last_fullsized":"2023-10-06T05:16:31.739104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:29.512000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.758349+0000","last_change":"2023-10-06T05:16:30.736868+0000","last_active":"2023-10-06T05:16:31.758349+0000","last_peered":"2023-10-06T05:16:31.758349+0000","last_clean":"2023-10-06T05:16:31.758349+0000","last_became_active":"2023-10-06T05:16:30.736569+0000","last_became_peered":"2023-10-06T05:16:30.736569+0000","last_unstale":"2023-10-06T05:16:31.758349+0000","last_undegraded":"2023-10-06T05:16:31.758349+0000","last_fullsized":"2023-10-06T05:16:31.758349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:24.192599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740118+0000","last_change":"2023-10-06T05:16:31.740118+0000","last_active":"2023-10-06T05:16:31.740118+0000","last_peered":"2023-10-06T05:16:31.740118+0000","last_clean":"2023-10-06T05:16:31.740118+0000","last_became_active":"2023-10-06T05:16:31.739833+0000","last_became_peered":"2023-10-06T05:16:31.739833+0000","last_unstale":"2023-10-06T05:16:31.740118+0000","last_undegraded":"2023-10-06T05:16:31.740118+0000","last_fullsized":"2023-10-06T05:16:31.740118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:19.575837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734329+0000","last_change":"2023-10-06T05:16:30.731142+0000","last_active":"2023-10-06T05:16:30.734329+0000","last_peered":"2023-10-06T05:16:30.734329+0000","last_clean":"2023-10-06T05:16:30.734329+0000","last_became_active":"2023-10-06T05:16:30.730810+0000","last_became_peered":"2023-10-06T05:16:30.730810+0000","last_unstale":"2023-10-06T05:16:30.734329+0000","last_undegraded":"2023-10-06T05:16:30.734329+0000","last_fullsized":"2023-10-06T05:16:30.734329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:54.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741743+0000","last_change":"2023-10-06T05:16:31.741743+0000","last_active":"2023-10-06T05:16:31.741743+0000","last_peered":"2023-10-06T05:16:31.741743+0000","last_clean":"2023-10-06T05:16:31.741743+0000","last_became_active":"2023-10-06T05:16:31.741482+0000","last_became_peered":"2023-10-06T05:16:31.741482+0000","last_unstale":"2023-10-06T05:16:31.741743+0000","last_undegraded":"2023-10-06T05:16:31.741743+0000","last_fullsized":"2023-10-06T05:16:31.741743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:50.822288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734526+0000","last_change":"2023-10-06T05:16:30.728891+0000","last_active":"2023-10-06T05:16:30.734526+0000","last_peered":"2023-10-06T05:16:30.734526+0000","last_clean":"2023-10-06T05:16:30.734526+0000","last_became_active":"2023-10-06T05:16:30.728632+0000","last_became_peered":"2023-10-06T05:16:30.728632+0000","last_unstale":"2023-10-06T05:16:30.734526+0000","last_undegraded":"2023-10-06T05:16:30.734526+0000","last_fullsized":"2023-10-06T05:16:30.734526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:49.166225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.728990+0000","last_change":"2023-10-06T05:16:31.728990+0000","last_active":"2023-10-06T05:16:31.728990+0000","last_peered":"2023-10-06T05:16:31.728990+0000","last_clean":"2023-10-06T05:16:31.728990+0000","last_became_active":"2023-10-06T05:16:31.728689+0000","last_became_peered":"2023-10-06T05:16:31.728689+0000","last_unstale":"2023-10-06T05:16:31.728990+0000","last_undegraded":"2023-10-06T05:16:31.728990+0000","last_fullsized":"2023-10-06T05:16:31.728990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:55.445215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.736753+0000","last_change":"2023-10-06T05:16:30.734176+0000","last_active":"2023-10-06T05:16:30.736753+0000","last_peered":"2023-10-06T05:16:30.736753+0000","last_clean":"2023-10-06T05:16:30.736753+0000","last_became_active":"2023-10-06T05:16:30.733627+0000","last_became_peered":"2023-10-06T05:16:30.733627+0000","last_unstale":"2023-10-06T05:16:30.736753+0000","last_undegraded":"2023-10-06T05:16:30.736753+0000","last_fullsized":"2023-10-06T05:16:30.736753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:38.868667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.742873+0000","last_change":"2023-10-06T05:16:31.742873+0000","last_active":"2023-10-06T05:16:31.742873+0000","last_peered":"2023-10-06T05:16:31.742873+0000","last_clean":"2023-10-06T05:16:31.742873+0000","last_became_active":"2023-10-06T05:16:31.742603+0000","last_became_peered":"2023-10-06T05:16:31.742603+0000","last_unstale":"2023-10-06T05:16:31.742873+0000","last_undegraded":"2023-10-06T05:16:31.742873+0000","last_fullsized":"2023-10-06T05:16:31.742873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:37.210002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.744109+0000","last_change":"2023-10-06T05:16:30.739638+0000","last_active":"2023-10-06T05:16:30.744109+0000","last_peered":"2023-10-06T05:16:30.744109+0000","last_clean":"2023-10-06T05:16:30.744109+0000","last_became_active":"2023-10-06T05:16:30.739271+0000","last_became_peered":"2023-10-06T05:16:30.739271+0000","last_unstale":"2023-10-06T05:16:30.744109+0000","last_undegraded":"2023-10-06T05:16:30.744109+0000","last_fullsized":"2023-10-06T05:16:30.744109+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:50.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.738940+0000","last_change":"2023-10-06T05:16:31.738940+0000","last_active":"2023-10-06T05:16:31.738940+0000","last_peered":"2023-10-06T05:16:31.738940+0000","last_clean":"2023-10-06T05:16:31.738940+0000","last_became_active":"2023-10-06T05:16:31.738672+0000","last_became_peered":"2023-10-06T05:16:31.738672+0000","last_unstale":"2023-10-06T05:16:31.738940+0000","last_undegraded":"2023-10-06T05:16:31.738940+0000","last_fullsized":"2023-10-06T05:16:31.738940+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:05.113348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734023+0000","last_change":"2023-10-06T05:16:30.731846+0000","last_active":"2023-10-06T05:16:30.734023+0000","last_peered":"2023-10-06T05:16:30.734023+0000","last_clean":"2023-10-06T05:16:30.734023+0000","last_became_active":"2023-10-06T05:16:30.731573+0000","last_became_peered":"2023-10-06T05:16:30.731573+0000","last_unstale":"2023-10-06T05:16:30.734023+0000","last_undegraded":"2023-10-06T05:16:30.734023+0000","last_fullsized":"2023-10-06T05:16:30.734023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:33.098566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.737546+0000","last_change":"2023-10-06T05:16:31.737546+0000","last_active":"2023-10-06T05:16:31.737546+0000","last_peered":"2023-10-06T05:16:31.737546+0000","last_clean":"2023-10-06T05:16:31.737546+0000","last_became_active":"2023-10-06T05:16:31.737283+0000","last_became_peered":"2023-10-06T05:16:31.737283+0000","last_unstale":"2023-10-06T05:16:31.737546+0000","last_undegraded":"2023-10-06T05:16:31.737546+0000","last_fullsized":"2023-10-06T05:16:31.737546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:15.536940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.741222+0000","last_change":"2023-10-06T05:16:30.738233+0000","last_active":"2023-10-06T05:16:30.741222+0000","last_peered":"2023-10-06T05:16:30.741222+0000","last_clean":"2023-10-06T05:16:30.741222+0000","last_became_active":"2023-10-06T05:16:30.737891+0000","last_became_peered":"2023-10-06T05:16:30.737891+0000","last_unstale":"2023-10-06T05:16:30.741222+0000","last_undegraded":"2023-10-06T05:16:30.741222+0000","last_fullsized":"2023-10-06T05:16:30.741222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:18.941870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741777+0000","last_change":"2023-10-06T05:16:31.741777+0000","last_active":"2023-10-06T05:16:31.741777+0000","last_peered":"2023-10-06T05:16:31.741777+0000","last_clean":"2023-10-06T05:16:31.741777+0000","last_became_active":"2023-10-06T05:16:31.741511+0000","last_became_peered":"2023-10-06T05:16:31.741511+0000","last_unstale":"2023-10-06T05:16:31.741777+0000","last_undegraded":"2023-10-06T05:16:31.741777+0000","last_fullsized":"2023-10-06T05:16:31.741777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:31.301396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.733680+0000","last_change":"2023-10-06T05:16:30.730536+0000","last_active":"2023-10-06T05:16:30.733680+0000","last_peered":"2023-10-06T05:16:30.733680+0000","last_clean":"2023-10-06T05:16:30.733680+0000","last_became_active":"2023-10-06T05:16:30.730208+0000","last_became_peered":"2023-10-06T05:16:30.730208+0000","last_unstale":"2023-10-06T05:16:30.733680+0000","last_undegraded":"2023-10-06T05:16:30.733680+0000","last_fullsized":"2023-10-06T05:16:30.733680+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:07.599115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.744434+0000","last_change":"2023-10-06T05:16:31.744434+0000","last_active":"2023-10-06T05:16:31.744434+0000","last_peered":"2023-10-06T05:16:31.744434+0000","last_clean":"2023-10-06T05:16:31.744434+0000","last_became_active":"2023-10-06T05:16:31.744155+0000","last_became_peered":"2023-10-06T05:16:31.744155+0000","last_unstale":"2023-10-06T05:16:31.744434+0000","last_undegraded":"2023-10-06T05:16:31.744434+0000","last_fullsized":"2023-10-06T05:16:31.744434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:08.022240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.730270+0000","last_change":"2023-10-06T05:16:30.732484+0000","last_active":"2023-10-06T05:16:31.730270+0000","last_peered":"2023-10-06T05:16:31.730270+0000","last_clean":"2023-10-06T05:16:31.730270+0000","last_became_active":"2023-10-06T05:16:30.732229+0000","last_became_peered":"2023-10-06T05:16:30.732229+0000","last_unstale":"2023-10-06T05:16:31.730270+0000","last_undegraded":"2023-10-06T05:16:31.730270+0000","last_fullsized":"2023-10-06T05:16:31.730270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:11.338131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.718878+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:16:30.718878+0000","last_peered":"2023-10-06T05:16:30.718878+0000","last_clean":"2023-10-06T05:16:30.718878+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:16:30.718878+0000","last_undegraded":"2023-10-06T05:16:30.718878+0000","last_fullsized":"2023-10-06T05:16:30.718878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.729942+0000","last_change":"2023-10-06T05:16:31.729942+0000","last_active":"2023-10-06T05:16:31.729942+0000","last_peered":"2023-10-06T05:16:31.729942+0000","last_clean":"2023-10-06T05:16:31.729942+0000","last_became_active":"2023-10-06T05:16:31.729340+0000","last_became_peered":"2023-10-06T05:16:31.729340+0000","last_unstale":"2023-10-06T05:16:31.729942+0000","last_undegraded":"2023-10-06T05:16:31.729942+0000","last_fullsized":"2023-10-06T05:16:31.729942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:39.877157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.739304+0000","last_change":"2023-10-06T05:16:30.736587+0000","last_active":"2023-10-06T05:16:30.739304+0000","last_peered":"2023-10-06T05:16:30.739304+0000","last_clean":"2023-10-06T05:16:30.739304+0000","last_became_active":"2023-10-06T05:16:30.736309+0000","last_became_peered":"2023-10-06T05:16:30.736309+0000","last_unstale":"2023-10-06T05:16:30.739304+0000","last_undegraded":"2023-10-06T05:16:30.739304+0000","last_fullsized":"2023-10-06T05:16:30.739304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:57.840207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.722105+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:16:30.722105+0000","last_peered":"2023-10-06T05:16:30.722105+0000","last_clean":"2023-10-06T05:16:30.722105+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:16:30.722105+0000","last_undegraded":"2023-10-06T05:16:30.722105+0000","last_fullsized":"2023-10-06T05:16:30.722105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.740698+0000","last_change":"2023-10-06T05:16:31.740698+0000","last_active":"2023-10-06T05:16:31.740698+0000","last_peered":"2023-10-06T05:16:31.740698+0000","last_clean":"2023-10-06T05:16:31.740698+0000","last_became_active":"2023-10-06T05:16:31.739538+0000","last_became_peered":"2023-10-06T05:16:31.739538+0000","last_unstale":"2023-10-06T05:16:31.740698+0000","last_undegraded":"2023-10-06T05:16:31.740698+0000","last_fullsized":"2023-10-06T05:16:31.740698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:18.230946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.734432+0000","last_change":"2023-10-06T05:16:30.733013+0000","last_active":"2023-10-06T05:16:30.734432+0000","last_peered":"2023-10-06T05:16:30.734432+0000","last_clean":"2023-10-06T05:16:30.734432+0000","last_became_active":"2023-10-06T05:16:30.732802+0000","last_became_peered":"2023-10-06T05:16:30.732802+0000","last_unstale":"2023-10-06T05:16:30.734432+0000","last_undegraded":"2023-10-06T05:16:30.734432+0000","last_fullsized":"2023-10-06T05:16:30.734432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:38.868667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.713460+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:16:30.713460+0000","last_peered":"2023-10-06T05:16:30.713460+0000","last_clean":"2023-10-06T05:16:30.713460+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:16:30.713460+0000","last_undegraded":"2023-10-06T05:16:30.713460+0000","last_fullsized":"2023-10-06T05:16:30.713460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.741903+0000","last_change":"2023-10-06T05:16:31.741903+0000","last_active":"2023-10-06T05:16:31.741903+0000","last_peered":"2023-10-06T05:16:31.741903+0000","last_clean":"2023-10-06T05:16:31.741903+0000","last_became_active":"2023-10-06T05:16:31.741646+0000","last_became_peered":"2023-10-06T05:16:31.741646+0000","last_unstale":"2023-10-06T05:16:31.741903+0000","last_undegraded":"2023-10-06T05:16:31.741903+0000","last_fullsized":"2023-10-06T05:16:31.741903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:31:13.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.739103+0000","last_change":"2023-10-06T05:16:30.737214+0000","last_active":"2023-10-06T05:16:30.739103+0000","last_peered":"2023-10-06T05:16:30.739103+0000","last_clean":"2023-10-06T05:16:30.739103+0000","last_became_active":"2023-10-06T05:16:30.736958+0000","last_became_peered":"2023-10-06T05:16:30.736958+0000","last_unstale":"2023-10-06T05:16:30.739103+0000","last_undegraded":"2023-10-06T05:16:30.739103+0000","last_fullsized":"2023-10-06T05:16:30.739103+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:25.448278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.722258+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:16:30.722258+0000","last_peered":"2023-10-06T05:16:30.722258+0000","last_clean":"2023-10-06T05:16:30.722258+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:16:30.722258+0000","last_undegraded":"2023-10-06T05:16:30.722258+0000","last_fullsized":"2023-10-06T05:16:30.722258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.730455+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:16:30.730455+0000","last_peered":"2023-10-06T05:16:30.730455+0000","last_clean":"2023-10-06T05:16:30.730455+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:16:30.730455+0000","last_undegraded":"2023-10-06T05:16:30.730455+0000","last_fullsized":"2023-10-06T05:16:30.730455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.734088+0000","last_change":"2023-10-06T05:16:31.734088+0000","last_active":"2023-10-06T05:16:31.734088+0000","last_peered":"2023-10-06T05:16:31.734088+0000","last_clean":"2023-10-06T05:16:31.734088+0000","last_became_active":"2023-10-06T05:16:31.733822+0000","last_became_peered":"2023-10-06T05:16:31.733822+0000","last_unstale":"2023-10-06T05:16:31.734088+0000","last_undegraded":"2023-10-06T05:16:31.734088+0000","last_fullsized":"2023-10-06T05:16:31.734088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:37.068605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.754018+0000","last_change":"2023-10-06T05:16:30.738187+0000","last_active":"2023-10-06T05:16:31.754018+0000","last_peered":"2023-10-06T05:16:31.754018+0000","last_clean":"2023-10-06T05:16:31.754018+0000","last_became_active":"2023-10-06T05:16:30.737609+0000","last_became_peered":"2023-10-06T05:16:30.737609+0000","last_unstale":"2023-10-06T05:16:31.754018+0000","last_undegraded":"2023-10-06T05:16:31.754018+0000","last_fullsized":"2023-10-06T05:16:31.754018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:25.448278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.730535+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:16:30.730535+0000","last_peered":"2023-10-06T05:16:30.730535+0000","last_clean":"2023-10-06T05:16:30.730535+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:16:30.730535+0000","last_undegraded":"2023-10-06T05:16:30.730535+0000","last_fullsized":"2023-10-06T05:16:30.730535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.727276+0000","last_change":"2023-10-06T05:16:31.727276+0000","last_active":"2023-10-06T05:16:31.727276+0000","last_peered":"2023-10-06T05:16:31.727276+0000","last_clean":"2023-10-06T05:16:31.727276+0000","last_became_active":"2023-10-06T05:16:31.726986+0000","last_became_peered":"2023-10-06T05:16:31.726986+0000","last_unstale":"2023-10-06T05:16:31.727276+0000","last_undegraded":"2023-10-06T05:16:31.727276+0000","last_fullsized":"2023-10-06T05:16:31.727276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:54.421560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.762070+0000","last_change":"2023-10-06T05:16:30.739656+0000","last_active":"2023-10-06T05:16:31.762070+0000","last_peered":"2023-10-06T05:16:31.762070+0000","last_clean":"2023-10-06T05:16:31.762070+0000","last_became_active":"2023-10-06T05:16:30.739319+0000","last_became_peered":"2023-10-06T05:16:30.739319+0000","last_unstale":"2023-10-06T05:16:31.762070+0000","last_undegraded":"2023-10-06T05:16:31.762070+0000","last_fullsized":"2023-10-06T05:16:31.762070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:01.657888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.735586+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:16:30.735586+0000","last_peered":"2023-10-06T05:16:30.735586+0000","last_clean":"2023-10-06T05:16:30.735586+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:16:30.735586+0000","last_undegraded":"2023-10-06T05:16:30.735586+0000","last_fullsized":"2023-10-06T05:16:30.735586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.747497+0000","last_change":"2023-10-06T05:16:31.747497+0000","last_active":"2023-10-06T05:16:31.747497+0000","last_peered":"2023-10-06T05:16:31.747497+0000","last_clean":"2023-10-06T05:16:31.747497+0000","last_became_active":"2023-10-06T05:16:31.744893+0000","last_became_peered":"2023-10-06T05:16:31.744893+0000","last_unstale":"2023-10-06T05:16:31.747497+0000","last_undegraded":"2023-10-06T05:16:31.747497+0000","last_fullsized":"2023-10-06T05:16:31.747497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:50.822288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.730425+0000","last_change":"2023-10-06T05:16:30.741488+0000","last_active":"2023-10-06T05:16:31.730425+0000","last_peered":"2023-10-06T05:16:31.730425+0000","last_clean":"2023-10-06T05:16:31.730425+0000","last_became_active":"2023-10-06T05:16:30.741272+0000","last_became_peered":"2023-10-06T05:16:30.741272+0000","last_unstale":"2023-10-06T05:16:31.730425+0000","last_undegraded":"2023-10-06T05:16:31.730425+0000","last_fullsized":"2023-10-06T05:16:31.730425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:59.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.720297+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:16:30.720297+0000","last_peered":"2023-10-06T05:16:30.720297+0000","last_clean":"2023-10-06T05:16:30.720297+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:16:30.720297+0000","last_undegraded":"2023-10-06T05:16:30.720297+0000","last_fullsized":"2023-10-06T05:16:30.720297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T05:16:31.724634+0000","last_change":"2023-10-06T05:16:31.724634+0000","last_active":"2023-10-06T05:16:31.724634+0000","last_peered":"2023-10-06T05:16:31.724634+0000","last_clean":"2023-10-06T05:16:31.724634+0000","last_became_active":"2023-10-06T05:16:31.724340+0000","last_became_peered":"2023-10-06T05:16:31.724340+0000","last_unstale":"2023-10-06T05:16:31.724634+0000","last_undegraded":"2023-10-06T05:16:31.724634+0000","last_fullsized":"2023-10-06T05:16:31.724634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:30.703593+0000","last_clean_scrub_stamp":"2023-10-06T05:16:30.703593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:46.935931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.738362+0000","last_change":"2023-10-06T05:16:30.736044+0000","last_active":"2023-10-06T05:16:30.738362+0000","last_peered":"2023-10-06T05:16:30.738362+0000","last_clean":"2023-10-06T05:16:30.738362+0000","last_became_active":"2023-10-06T05:16:30.735765+0000","last_became_peered":"2023-10-06T05:16:30.735765+0000","last_unstale":"2023-10-06T05:16:30.738362+0000","last_undegraded":"2023-10-06T05:16:30.738362+0000","last_fullsized":"2023-10-06T05:16:30.738362+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:29.695104+0000","last_clean_scrub_stamp":"2023-10-06T05:16:29.695104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:04.104858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-06T05:16:30.725184+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:16:30.725184+0000","last_peered":"2023-10-06T05:16:30.725184+0000","last_clean":"2023-10-06T05:16:30.725184+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:16:30.725184+0000","last_undegraded":"2023-10-06T05:16:30.725184+0000","last_fullsized":"2023-10-06T05:16:30.725184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298572,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073268,"statfs":{"total":96636764160,"available":96331026432,"internally_reserved":0,"allocated":1314816,"data_stored":903767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.623,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85999999999999999,"5min":1.113,"15min":1.113},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":1.079,"15min":1.079},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.85299999999999998,"5min":1.202,"15min":1.202},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.626,"15min":0.621},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85999999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76000000000000001,"15min":0.754},"min":{"1min":0.67700000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0009999999999999,"5min":1.407,"15min":1.407},"last":0.747}]},{"osd":6,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.99299999999999999,"5min":1.05,"15min":1.05},"last":0.627},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78300000000000003,"5min":1.034,"15min":1.034},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71699999999999997,"15min":0.72999999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91500000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89600000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.82699999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.745}]},{"osd":2,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.80000000000000004,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86299999999999999,"5min":0.876,"15min":0.876},"last":0.496},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81100000000000005,"15min":0.82299999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.90800000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92000000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.76700000000000002,"15min":0.78600000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.81699999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 05:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.81200000000000006},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.86699999999999999,"15min":0.88900000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.53000000000000003,"15min":0.55400000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57099999999999995,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.57799999999999996,"15min":0.60199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.38400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.55400000000000005,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66000000000000003,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.66600000000000004,"15min":0.69499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68400000000000005,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.61799999999999999,"15min":0.64200000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71699999999999997,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.437}]},{"osd":4,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.79900000000000004,"15min":0.81499999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98099999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.83899999999999997,"15min":0.85999999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.016,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95099999999999996,"5min":1.349,"15min":1.349},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98599999999999999,"5min":1.669,"15min":1.669},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.87,"15min":0.88700000000000001},"min":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1020000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.89800000000000002,"15min":0.92000000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0680000000000001,"5min":1.573,"15min":1.573},"last":0.503}]},{"osd":7,"last update":"Fri Oct 6 05:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.86599999999999999,"15min":0.88800000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.90400000000000003,"5min":1.528,"15min":1.528},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.874,"15min":0.89500000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98899999999999999,"5min":1.75,"15min":1.75},"last":0.48699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302668,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069172,"statfs":{"total":96636764160,"available":96326832128,"internally_reserved":0,"allocated":1314816,"data_stored":903767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42999999999999999,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48699999999999999,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.51900000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.627},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435}]},{"osd":3,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.746,"15min":0.746},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.748,"15min":0.745},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91700000000000004,"5min":1.04,"15min":1.04},"last":0.75},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.77800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1,"5min":1.01,"15min":1.01},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.91200000000000003,"5min":0.999,"15min":0.999},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.432,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.55000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.442,"15min":0.442},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57699999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.63500000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.502}]},{"osd":4,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.006,"5min":1.044,"15min":1.044},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.57399999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.93600000000000005,"5min":1.056,"15min":1.056},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79400000000000004,"15min":0.80400000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.86399999999999999,"5min":1.319,"15min":1.319},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.81799999999999995,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92200000000000004,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.624,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.92500000000000004,"5min":1.141,"15min":1.141},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.61099999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96499999999999997,"5min":1.282,"15min":1.282},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.96999999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.77500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0720000000000001,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.48899999999999999,"15min":0.502},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53700000000000003,"5min":1.772,"15min":1.772},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.55000000000000004,"5min":1.913,"15min":1.913},"last":0.374},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.61199999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.624,"5min":1.954,"15min":1.954},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.622,"15min":0.63900000000000001},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.58799999999999997,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.55800000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1619999999999999,"5min":1.282,"15min":1.282},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.55800000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.85599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86299999999999999,"15min":0.875},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2250000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.157,"5min":1.343,"15min":1.343},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83699999999999997},"min":{"1min":0.64100000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3109999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.206,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.143,"5min":1.214,"15min":1.214},"last":0.81000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297992,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073848,"statfs":{"total":96636764160,"available":96331620352,"internally_reserved":0,"allocated":720896,"data_stored":313399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97099999999999997,"5min":1.073,"15min":1.073},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":1.169,"15min":1.169},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.042,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92900000000000005,"5min":1.117,"15min":1.117},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89600000000000002,"5min":0.998,"15min":0.998},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.74199999999999999,"15min":0.752},"min":{"1min":0.501,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.751},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.505,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.196},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.74099999999999999,"15min":0.75700000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81299999999999994,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.495},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.375,"15min":1.375},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":1.004,"15min":1.004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.82299999999999995,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1639999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.83799999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0209999999999999,"5min":1.466,"15min":1.466},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1339999999999999,"5min":1.218,"15min":1.218},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298008,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073832,"statfs":{"total":96636764160,"available":96331603968,"internally_reserved":0,"allocated":737280,"data_stored":320537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.872,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.50600000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86399999999999999,"5min":1.028,"15min":1.028},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.84599999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0149999999999999,"5min":1.087,"15min":1.087},"last":0.505},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.278,"15min":1.278},"last":0.433}]},{"osd":4,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94299999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.45900000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85099999999999998,"5min":1.294,"15min":1.294},"last":0.59799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81699999999999995,"5min":1.006,"15min":1.006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76000000000000001,"15min":0.76900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":1.254,"15min":1.254},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89600000000000002,"5min":1.337,"15min":1.337},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91100000000000003,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:16:36.588 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:16:36.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:16:36.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.813+0000 7f66b759e700 1 Processor -- start 2023-10-06T05:16:36.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.813+0000 7f66b759e700 1 -- start start 2023-10-06T05:16:36.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.814+0000 7f66b759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b8156370 0x7f66b80aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.814+0000 7f66b759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66b8156740 0x7f66b80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.814+0000 7f66b759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66b80cc750 0x7f66b80c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.814+0000 7f66b759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b806beb0 con 0x7f66b8156740 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.814+0000 7f66b759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b806b330 con 0x7f66b80cc750 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.814+0000 7f66b759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b806b610 con 0x7f66b8156370 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66b8156740 0x7f66b80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66b8156740 0x7f66b80cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52458/0 (socket says 172.21.15.33:52458) 2023-10-06T05:16:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 -- 172.21.15.33:0/2099648841 learned_addr learned my addr 172.21.15.33:0/2099648841 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b659c700 1 --2- 172.21.15.33:0/2099648841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b8156370 0x7f66b80aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b6d9d700 1 --2- 172.21.15.33:0/2099648841 >> v2:172.21.15.167:3300/0 conn(0x7f66b80cc750 0x7f66b80c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 -- 172.21.15.33:0/2099648841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b8156370 msgr2=0x7f66b80aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 --2- 172.21.15.33:0/2099648841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b8156370 0x7f66b80aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 -- 172.21.15.33:0/2099648841 >> v2:172.21.15.167:3300/0 conn(0x7f66b80cc750 msgr2=0x7f66b80c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 --2- 172.21.15.33:0/2099648841 >> v2:172.21.15.167:3300/0 conn(0x7f66b80cc750 0x7f66b80c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 -- 172.21.15.33:0/2099648841 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b806b8f0 con 0x7f66b8156740 2023-10-06T05:16:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.815+0000 7f66b5d9b700 1 --2- 172.21.15.33:0/2099648841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80cc210 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f66b80694d0 tx=0x7f66ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a66c0395614bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b559a700 1 -- 172.21.15.33:0/2099648841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00b640 con 0x7f66b8156740 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b559a700 1 -- 172.21.15.33:0/2099648841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac00b7a0 con 0x7f66b8156740 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b559a700 1 -- 172.21.15.33:0/2099648841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00bb60 con 0x7f66b8156740 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 -- 172.21.15.33:0/2099648841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 msgr2=0x7f66b80cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 --2- 172.21.15.33:0/2099648841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80cc210 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f66b80694d0 tx=0x7f66ac009250 comp rx=0 tx=0).stop 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 -- 172.21.15.33:0/2099648841 shutdown_connections 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 --2- 172.21.15.33:0/2099648841 >> v2:172.21.15.167:3300/0 conn(0x7f66b80cc750 0x7f66b80c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 --2- 172.21.15.33:0/2099648841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b8156370 0x7f66b80aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 --2- 172.21.15.33:0/2099648841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80cc210 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 -- 172.21.15.33:0/2099648841 >> 172.21.15.33:0/2099648841 conn(0x7f66b806c560 msgr2=0x7f66b806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:36.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 -- 172.21.15.33:0/2099648841 shutdown_connections 2023-10-06T05:16:36.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.816+0000 7f66b759e700 1 -- 172.21.15.33:0/2099648841 wait complete. 2023-10-06T05:16:36.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.818+0000 7f66b759e700 1 Processor -- start 2023-10-06T05:16:36.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.818+0000 7f66b759e700 1 -- start start 2023-10-06T05:16:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b80cc750 0x7f66b81524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66b8156370 0x7f66b80aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80ab180 con 0x7f66b8156740 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b81544f0 con 0x7f66b8156370 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b81538e0 con 0x7f66b80cc750 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52462/0 (socket says 172.21.15.33:52462) 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b6d9d700 1 -- 172.21.15.33:0/2354697828 learned_addr learned my addr 172.21.15.33:0/2354697828 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b659c700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b80cc750 0x7f66b81524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.819+0000 7f66b5d9b700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66b8156370 0x7f66b80aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b6d9d700 1 -- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b80cc750 msgr2=0x7f66b81524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b6d9d700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b80cc750 0x7f66b81524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b6d9d700 1 -- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66b8156370 msgr2=0x7f66b80aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b6d9d700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66b8156370 0x7f66b80aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b6d9d700 1 -- 172.21.15.33:0/2354697828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0094f0 con 0x7f66b8156740 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b6d9d700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80aac40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009e90 tx=0x7f66b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000cb70 con 0x7f66b8156740 2023-10-06T05:16:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b000f040 con 0x7f66b8156740 2023-10-06T05:16:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0014480 con 0x7f66b8156740 2023-10-06T05:16:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b8153bc0 con 0x7f66b8156740 2023-10-06T05:16:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.820+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b8154180 con 0x7f66b8156740 2023-10-06T05:16:36.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.821+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b81546a0 con 0x7f66b8156740 2023-10-06T05:16:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.822+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66b00147f0 con 0x7f66b8156740 2023-10-06T05:16:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.823+0000 7f66a67fc700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6698066f30 0x7f66980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.824+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f66b00854b0 con 0x7f66b8156740 2023-10-06T05:16:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.824+0000 7f66b659c700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6698066f30 0x7f66980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.825+0000 7f66b659c700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6698066f30 0x7f66980693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009d20 tx=0x7f66a80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:36.827+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66b81546a0 con 0x7f66b8156740 2023-10-06T05:16:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.111+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f66b81546a0 con 0x7f66b8156740 2023-10-06T05:16:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.117+0000 7f66a67fc700 1 -- 172.21.15.33:0/2354697828 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f66b81546a0 con 0x7f66b8156740 2023-10-06T05:16:37.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6698066f30 msgr2=0x7f66980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6698066f30 0x7f66980693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009d20 tx=0x7f66a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 msgr2=0x7f66b80aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80aac40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009e90 tx=0x7f66b000b040 comp rx=0 tx=0).stop 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 shutdown_connections 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6698066f30 0x7f66980693e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8156740 0x7f66b80aac40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66b80cc750 0x7f66b81524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 --2- 172.21.15.33:0/2354697828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66b8156370 0x7f66b80aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 >> 172.21.15.33:0/2354697828 conn(0x7f66b806c560 msgr2=0x7f66b8149d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 shutdown_connections 2023-10-06T05:16:37.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.119+0000 7f66b759e700 1 -- 172.21.15.33:0/2354697828 wait complete. 2023-10-06T05:16:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:16:37.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:37.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.323+0000 7f2758b9d700 1 Processor -- start 2023-10-06T05:16:37.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.323+0000 7f2758b9d700 1 -- start start 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2758b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27540b0a40 0x7f275406ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2758b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f275415a0a0 0x7f275415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2758b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275415e2d0 con 0x7f27540b0e10 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2758b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275415d6c0 con 0x7f275415a0a0 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2758b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275415d9a0 con 0x7f27540b0a40 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2753fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f275415a0a0 0x7f275415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2753fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f275415a0a0 0x7f275415a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52472/0 (socket says 172.21.15.33:52472) 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f2753fff700 1 -- 172.21.15.33:0/3997825990 learned_addr learned my addr 172.21.15.33:0/3997825990 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.325+0000 7f27537fe700 1 --2- 172.21.15.33:0/3997825990 >> v2:172.21.15.167:3300/0 conn(0x7f27540b0a40 0x7f275406ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f2753fff700 1 -- 172.21.15.33:0/3997825990 >> v2:172.21.15.167:3300/0 conn(0x7f27540b0a40 msgr2=0x7f275406ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f2753fff700 1 --2- 172.21.15.33:0/3997825990 >> v2:172.21.15.167:3300/0 conn(0x7f27540b0a40 0x7f275406ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f2753fff700 1 -- 172.21.15.33:0/3997825990 >> v1:172.21.15.167:6789/0 conn(0x7f27540b0e10 legacy=0x7f27540b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f2753fff700 1 -- 172.21.15.33:0/3997825990 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275415dc80 con 0x7f275415a0a0 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f2753fff700 1 --2- 172.21.15.33:0/3997825990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f275415a0a0 0x7f275415a4e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f275406a230 tx=0x7f274c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db4a069644db14d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f27527fc700 1 -- 172.21.15.33:0/3997825990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c00e070 con 0x7f275415a0a0 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f27527fc700 1 -- 172.21.15.33:0/3997825990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c00ba20 con 0x7f275415a0a0 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.326+0000 7f27527fc700 1 -- 172.21.15.33:0/3997825990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c0092e0 con 0x7f275415a0a0 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 -- 172.21.15.33:0/3997825990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f275415a0a0 msgr2=0x7f275415a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3997825990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f275415a0a0 0x7f275415a4e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f275406a230 tx=0x7f274c009840 comp rx=0 tx=0).stop 2023-10-06T05:16:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 -- 172.21.15.33:0/3997825990 shutdown_connections 2023-10-06T05:16:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3997825990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f275415a0a0 0x7f275415a4e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3997825990 >> v2:172.21.15.167:3300/0 conn(0x7f27540b0a40 0x7f275406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 -- 172.21.15.33:0/3997825990 >> 172.21.15.33:0/3997825990 conn(0x7f275406e0f0 msgr2=0x7f27540675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 -- 172.21.15.33:0/3997825990 shutdown_connections 2023-10-06T05:16:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.327+0000 7f2758b9d700 1 -- 172.21.15.33:0/3997825990 wait complete. 2023-10-06T05:16:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.328+0000 7f2758b9d700 1 Processor -- start 2023-10-06T05:16:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.328+0000 7f2758b9d700 1 -- start start 2023-10-06T05:16:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.329+0000 7f2758b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27540b0a40 0x7f27541540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.329+0000 7f2758b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 0x7f2754151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.329+0000 7f2758b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275415a0a0 0x7f275414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.329+0000 7f2758b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275415d490 con 0x7f27540b0e10 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.329+0000 7f2758b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27541536e0 con 0x7f275415a0a0 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.329+0000 7f2758b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275415ced0 con 0x7f27540b0a40 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 0x7f2754151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 0x7f2754151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52484/0 (socket says 172.21.15.33:52484) 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 -- 172.21.15.33:0/3597364055 learned_addr learned my addr 172.21.15.33:0/3597364055 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f27537fe700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27540b0a40 0x7f27541540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2753fff700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275415a0a0 0x7f275414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 -- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27540b0a40 msgr2=0x7f27541540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27540b0a40 0x7f27541540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 -- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275415a0a0 msgr2=0x7f275414d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275415a0a0 0x7f275414d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 -- 172.21.15.33:0/3597364055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c009ae0 con 0x7f27540b0e10 2023-10-06T05:16:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f27537fe700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27540b0a40 0x7f27541540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2753fff700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275415a0a0 0x7f275414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f2752ffd700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 0x7f2754151150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f274800c790 tx=0x7f2748007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.330+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011590 con 0x7f27540b0e10 2023-10-06T05:16:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.331+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2748011710 con 0x7f27540b0e10 2023-10-06T05:16:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.331+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011ad0 con 0x7f27540b0e10 2023-10-06T05:16:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.331+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275415d1b0 con 0x7f27540b0e10 2023-10-06T05:16:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.331+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2754153bb0 con 0x7f27540b0e10 2023-10-06T05:16:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.332+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275406fcc0 con 0x7f27540b0e10 2023-10-06T05:16:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.333+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f274801b3b0 con 0x7f27540b0e10 2023-10-06T05:16:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.334+0000 7f273b7fe700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2740066f30 0x7f27400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.334+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2748083c60 con 0x7f27540b0e10 2023-10-06T05:16:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.334+0000 7f27537fe700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2740066f30 0x7f27400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.335+0000 7f27537fe700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2740066f30 0x7f27400693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2744009d80 tx=0x7f2744008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:37.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.338+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2748050cd0 con 0x7f27540b0e10 2023-10-06T05:16:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.587+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2754153dd0 con 0x7f27540b0e10 2023-10-06T05:16:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.588+0000 7f273b7fe700 1 -- 172.21.15.33:0/3597364055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f2748017070 con 0x7f27540b0e10 2023-10-06T05:16:37.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:37.590 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2740066f30 msgr2=0x7f27400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2740066f30 0x7f27400693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2744009d80 tx=0x7f2744008040 comp rx=0 tx=0).stop 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 msgr2=0x7f2754151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 0x7f2754151150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f274800c790 tx=0x7f2748007f40 comp rx=0 tx=0).stop 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 shutdown_connections 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2740066f30 0x7f27400693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275415a0a0 0x7f275414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27540b0a40 0x7f27541540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 --2- 172.21.15.33:0/3597364055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27540b0e10 0x7f2754151150 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 >> 172.21.15.33:0/3597364055 conn(0x7f275406e0f0 msgr2=0x7f27540b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 shutdown_connections 2023-10-06T05:16:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:37.590+0000 7f2758b9d700 1 -- 172.21.15.33:0/3597364055 wait complete. 2023-10-06T05:16:37.594 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:16:38.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:16:31.211707+0000', 'modified': '2023-10-06T05:16:34.745629+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5715}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5715': {'gid': 5715, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.33:6834/103971342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 103971342}, {'type': 'v1', 'addr': '172.21.15.33:6834', 'nonce': 103971342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:16:38.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:16:38.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:16:38.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.200+0000 7fdb5cb9d700 1 Processor -- start 2023-10-06T05:16:38.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.200+0000 7fdb5cb9d700 1 -- start start 2023-10-06T05:16:38.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.201+0000 7fdb5cb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdb58156880 0x7fdb580c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb5cb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5806a820 con 0x7fdb580c7ec0 2023-10-06T05:16:38.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb5cb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5815a2f0 con 0x7fdb58159e10 2023-10-06T05:16:38.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb5cb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb580cc290 con 0x7fdb58156880 2023-10-06T05:16:38.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb577fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdb58156880 0x7fdb580c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb577fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdb58156880 0x7fdb580c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52494/0 (socket says 172.21.15.33:52494) 2023-10-06T05:16:38.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb57fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb580c7ec0 0x7fdb5806ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58736/0 (socket says 172.21.15.33:58736) 2023-10-06T05:16:38.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb577fe700 1 -- 172.21.15.33:0/3816421767 learned_addr learned my addr 172.21.15.33:0/3816421767 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb577fe700 1 -- 172.21.15.33:0/3816421767 >> v1:172.21.15.167:6789/0 conn(0x7fdb58159e10 legacy=0x7fdb58156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb577fe700 1 -- 172.21.15.33:0/3816421767 >> v1:172.21.15.33:6789/0 conn(0x7fdb580c7ec0 legacy=0x7fdb5806ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb577fe700 1 -- 172.21.15.33:0/3816421767 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5815a5d0 con 0x7fdb58156880 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.202+0000 7fdb567fc700 1 -- 172.21.15.33:0/3816421767 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210690389 0 0) 0x7fdb5806a820 con 0x7fdb580c7ec0 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3816421767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb58156880 0x7fdb580c7980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800bc50 tx=0x7fdb4800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b172eb8719c0ed3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb567fc700 1 -- 172.21.15.33:0/3816421767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800be60 con 0x7fdb58156880 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb567fc700 1 -- 172.21.15.33:0/3816421767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4800ddc0 con 0x7fdb58156880 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb567fc700 1 -- 172.21.15.33:0/3816421767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480106a0 con 0x7fdb58156880 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3816421767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb58156880 msgr2=0x7fdb580c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3816421767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb58156880 0x7fdb580c7980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800bc50 tx=0x7fdb4800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3816421767 shutdown_connections 2023-10-06T05:16:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3816421767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb58156880 0x7fdb580c7980 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3816421767 >> 172.21.15.33:0/3816421767 conn(0x7fdb5806c560 msgr2=0x7fdb5806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.203+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3816421767 shutdown_connections 2023-10-06T05:16:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.204+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3816421767 wait complete. 2023-10-06T05:16:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.205+0000 7fdb5cb9d700 1 Processor -- start 2023-10-06T05:16:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.205+0000 7fdb5cb9d700 1 -- start start 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb5cb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 0x7fdb580b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb5cb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb58156880 0x7fdb58143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb5cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb58159e10 0x7fdb5814f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb5cb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580b0a80 con 0x7fdb580c7ec0 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb5cb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580afe70 con 0x7fdb58156880 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb5cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580b0150 con 0x7fdb58159e10 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 0x7fdb580b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 0x7fdb580b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52502/0 (socket says 172.21.15.33:52502) 2023-10-06T05:16:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb577fe700 1 -- 172.21.15.33:0/3744824417 learned_addr learned my addr 172.21.15.33:0/3744824417 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb57fff700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb58159e10 0x7fdb5814f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.206+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb58156880 0x7fdb58143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb577fe700 1 -- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb58159e10 msgr2=0x7fdb5814f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb58159e10 0x7fdb5814f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb577fe700 1 -- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb58156880 msgr2=0x7fdb58143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb58156880 0x7fdb58143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb577fe700 1 -- 172.21.15.33:0/3744824417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4800b790 con 0x7fdb580c7ec0 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb58156880 0x7fdb58143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 0x7fdb580b0f10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b760 tx=0x7fdb4801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb3f7fe700 1 -- 172.21.15.33:0/3744824417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48018880 con 0x7fdb580c7ec0 2023-10-06T05:16:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb3f7fe700 1 -- 172.21.15.33:0/3744824417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb480189e0 con 0x7fdb580c7ec0 2023-10-06T05:16:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb3f7fe700 1 -- 172.21.15.33:0/3744824417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48018da0 con 0x7fdb580c7ec0 2023-10-06T05:16:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb580b0430 con 0x7fdb580c7ec0 2023-10-06T05:16:38.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.207+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb58144100 con 0x7fdb580c7ec0 2023-10-06T05:16:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.209+0000 7fdb3f7fe700 1 -- 172.21.15.33:0/3744824417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb480233d0 con 0x7fdb580c7ec0 2023-10-06T05:16:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.210+0000 7fdb3f7fe700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb44066f30 0x7fdb440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.210+0000 7fdb3f7fe700 1 -- 172.21.15.33:0/3744824417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb48014070 con 0x7fdb580c7ec0 2023-10-06T05:16:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.211+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb44066f30 0x7fdb440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.211+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb44066f30 0x7fdb440693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb58150750 tx=0x7fdb4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.212+0000 7fdb6c46cb80 1 Processor -- start 2023-10-06T05:16:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 -- start start 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb2e0950 0x560bcb2e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 0x560bcb309920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb2dc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb260c80 con 0x560bcb32abc0 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb237a70 con 0x560bcb2e0950 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.213+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bcb235270 con 0x560bcb309e60 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 0x560bcb309920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 0x560bcb309920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52516/0 (socket says 172.21.15.33:52516) 2023-10-06T05:16:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 -- 172.21.15.33:0/1801148954 learned_addr learned my addr 172.21.15.33:0/1801148954 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb577fe700 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb2e0950 0x560bcb2e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb57fff700 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb2dc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 -- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 msgr2=0x560bcb2dc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb2dc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 -- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb2e0950 msgr2=0x560bcb2e0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb2e0950 0x560bcb2e0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 -- 172.21.15.33:0/1801148954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bcb0cb270 con 0x560bcb32abc0 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.214+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 0x560bcb309920 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000fd0 tx=0x7fdb4c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc576b34ad23eab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb3cff9700 1 -- 172.21.15.33:0/1801148954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c014540 con 0x560bcb32abc0 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb3cff9700 1 -- 172.21.15.33:0/1801148954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c0146a0 con 0x560bcb32abc0 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb3cff9700 1 -- 172.21.15.33:0/1801148954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c014a20 con 0x560bcb32abc0 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/1801148954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 msgr2=0x560bcb309920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 0x560bcb309920 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000fd0 tx=0x7fdb4c012000 comp rx=0 tx=0).stop 2023-10-06T05:16:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/1801148954 shutdown_connections 2023-10-06T05:16:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb2dc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb2e0950 0x560bcb2e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/1801148954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb32abc0 0x560bcb309920 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/1801148954 >> 172.21.15.33:0/1801148954 conn(0x560bcb2e2270 msgr2=0x560bcb2e2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/1801148954 shutdown_connections 2023-10-06T05:16:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.215+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/1801148954 wait complete. 2023-10-06T05:16:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.216+0000 7fdb6c46cb80 1 -- start start 2023-10-06T05:16:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.216+0000 7fdb6c46cb80 1 Processor -- start 2023-10-06T05:16:38.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb336c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb3217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb31eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb231060 con 0x560bcb2e0950 2023-10-06T05:16:38.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb1dc890 con 0x560bcb32abc0 2023-10-06T05:16:38.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bcb1f7ea0 con 0x560bcb309e60 2023-10-06T05:16:38.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.217+0000 7fdb6c46cb80 10 client.0 _ll_register_callbacks cb 0x560bcb0cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:38.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb336c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb336c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52518/0 (socket says 172.21.15.33:52518) 2023-10-06T05:16:38.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 -- 172.21.15.33:0/2570981251 learned_addr learned my addr 172.21.15.33:0/2570981251 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb3217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb57fff700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb31eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 -- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 msgr2=0x560bcb3217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb3217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 -- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 msgr2=0x560bcb31eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb31eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 -- 172.21.15.33:0/2570981251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c00e040 con 0x560bcb2e0950 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb3217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb57fff700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb31eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb577fe700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb336c20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800e010 tx=0x7fdb48086700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ba5ada985d62d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.218+0000 7fdb35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb35ffb700 1 -- 172.21.15.33:0/2570981251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48059270 con 0x560bcb2e0950 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb35ffb700 1 -- 172.21.15.33:0/2570981251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb4805c4c0 con 0x560bcb2e0950 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bcb2d5690 con 0x560bcb2e0950 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb35ffb700 1 -- 172.21.15.33:0/2570981251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb480882e0 con 0x560bcb2e0950 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 10 client.5871 fetch_fsmap learned FSMap version 35 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x560bcb06dfe0 con 0x560bcb2e0950 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb35ffb700 1 -- 172.21.15.33:0/2570981251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560bcb2d5690 con 0x560bcb2e0950 2023-10-06T05:16:38.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb35ffb700 1 -- 172.21.15.33:0/2570981251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdb4805fd80 con 0x560bcb2e0950 2023-10-06T05:16:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 10 client.5871 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T05:16:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 10 client.5871 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 20 client.5871 populate_metadata read hostname 'smithi033' 2023-10-06T05:16:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb307230 0x560bcb307600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 4 client.5871 mds_command: new command op to 5715 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:16:38.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 --> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bcb2b25c0 con 0x560bcb307230 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.219+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb307230 0x560bcb307600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.220+0000 7fdb35ffb700 10 client.5871 ms_handle_connect on v2:172.21.15.33:6832/103971342 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.220+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb307230 0x560bcb307600 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6330305454e7ec4 server_cookie=d6e76afd8aa812b4 in_seq=0 out_seq=0 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb35ffb700 1 -- 172.21.15.33:0/2570981251 <== mds.0 v2:172.21.15.33:6832/103971342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560bcb2b25c0 con 0x560bcb307230 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb35ffb700 10 client.5871 handle_command_reply: tid=0 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 client.5871 shutdown 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 msgr2=0x560bcb336c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb336c20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800e010 tx=0x7fdb48086700 comp rx=0 tx=0).stop 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 shutdown_connections 2023-10-06T05:16:38.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb307230 0x560bcb307600 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb31eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb336c20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb35ffb700 0 client.5871 ms_handle_reset on v2:172.21.15.33:6832/103971342 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/2570981251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb3217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 >> 172.21.15.33:0/2570981251 conn(0x560bcb2e2270 msgr2=0x560bcb2d71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 shutdown_connections 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/2570981251 wait complete. 2023-10-06T05:16:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.221+0000 7fdb6c46cb80 20 client.5871 trim_cache size 0 max 16384 2023-10-06T05:16:38.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.239+0000 7fdb6c46cb80 1 Processor -- start 2023-10-06T05:16:38.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.239+0000 7fdb6c46cb80 1 -- start start 2023-10-06T05:16:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb340090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb0b3e30 con 0x560bcb2e0950 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb2de960 con 0x560bcb32abc0 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bcb2dd8d0 con 0x560bcb309e60 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52520/0 (socket says 172.21.15.33:52520) 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 -- 172.21.15.33:0/4264070718 learned_addr learned my addr 172.21.15.33:0/4264070718 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb57fff700 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb340090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 -- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 msgr2=0x560bcb340090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb340090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 -- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 msgr2=0x560bcb2d71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 -- 172.21.15.33:0/4264070718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bcb1ff780 con 0x560bcb2e0950 2023-10-06T05:16:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.240+0000 7fdb577fe700 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3255c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdb480591b0 tx=0x7fdb48070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d65c798949335d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb3d7fa700 1 -- 172.21.15.33:0/4264070718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480882e0 con 0x560bcb2e0950 2023-10-06T05:16:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb3d7fa700 1 -- 172.21.15.33:0/4264070718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb480555d0 con 0x560bcb2e0950 2023-10-06T05:16:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb3d7fa700 1 -- 172.21.15.33:0/4264070718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48059270 con 0x560bcb2e0950 2023-10-06T05:16:38.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/4264070718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 msgr2=0x560bcb3255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3255c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdb480591b0 tx=0x7fdb48070000 comp rx=0 tx=0).stop 2023-10-06T05:16:38.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/4264070718 shutdown_connections 2023-10-06T05:16:38.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3255c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/4264070718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb309e60 0x560bcb340090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/4264070718 >> 172.21.15.33:0/4264070718 conn(0x560bcb2e2270 msgr2=0x560bcb2e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/4264070718 shutdown_connections 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.241+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/4264070718 wait complete. 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.242+0000 7fdb6c46cb80 1 -- start start 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.242+0000 7fdb6c46cb80 1 Processor -- start 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb309e60 0x560bcb390a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb3264a0 con 0x560bcb2e0950 2023-10-06T05:16:38.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bcb2dd270 con 0x560bcb309e60 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bcb2dd550 con 0x560bcb32abc0 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb6c46cb80 10 client.0 _ll_register_callbacks cb 0x560bcb0cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.243+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3904e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52526/0 (socket says 172.21.15.33:52526) 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 -- 172.21.15.33:0/3156759523 learned_addr learned my addr 172.21.15.33:0/3156759523 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb57fff700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb309e60 0x560bcb390a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 -- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb32abc0 msgr2=0x560bcb2d71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 -- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb309e60 msgr2=0x560bcb390a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb309e60 0x560bcb390a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 -- 172.21.15.33:0/3156759523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bcb06dfe0 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb577fe700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3904e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdb48059200 tx=0x7fdb48008050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3b8e61c1161fad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb35ffb700 1 -- 172.21.15.33:0/3156759523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48059270 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb35ffb700 1 -- 172.21.15.33:0/3156759523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb4805c4c0 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.244+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bcb1d3e00 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb35ffb700 1 -- 172.21.15.33:0/3156759523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb480882e0 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 10 client.5877 fetch_fsmap learned FSMap version 35 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x560bcb1e6470 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb35ffb700 1 -- 172.21.15.33:0/3156759523 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560bcb1d3e00 con 0x560bcb2e0950 2023-10-06T05:16:38.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb35ffb700 1 -- 172.21.15.33:0/3156759523 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdb4805e480 con 0x560bcb2e0950 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 10 client.5877 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 10 client.5877 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 20 client.5877 populate_metadata read hostname 'smithi033' 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb338f10 0x560bcb3392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 4 client.5877 mds_command: new command op to 5715 tid=0[{"prefix": "status"}] 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 --> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560bcb1d3b20 con 0x560bcb338f10 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.245+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb338f10 0x560bcb3392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb338f10 0x560bcb3392e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c1676349d267d server_cookie=e0c65af6f9e62773 in_seq=0 out_seq=0 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb35ffb700 10 client.5877 ms_handle_connect on v2:172.21.15.33:6832/103971342 2023-10-06T05:16:38.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb35ffb700 1 -- 172.21.15.33:0/3156759523 <== mds.0 v2:172.21.15.33:6832/103971342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560bcb1d3b20 con 0x560bcb338f10 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb35ffb700 10 client.5877 handle_command_reply: tid=0 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 client.5877 shutdown 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 msgr2=0x560bcb3904e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3904e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdb48059200 tx=0x7fdb48008050 comp rx=0 tx=0).stop 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 shutdown_connections 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:6832/103971342,v1:172.21.15.33:6834/103971342] conn(0x560bcb338f10 0x560bcb3392e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x560bcb32abc0 0x560bcb2d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560bcb2e0950 0x560bcb3904e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 --2- 172.21.15.33:0/3156759523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcb309e60 0x560bcb390a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb35ffb700 0 client.5877 ms_handle_reset on v2:172.21.15.33:6832/103971342 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 >> 172.21.15.33:0/3156759523 conn(0x560bcb2e2270 msgr2=0x560bcb3068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 shutdown_connections 2023-10-06T05:16:38.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.246+0000 7fdb6c46cb80 1 -- 172.21.15.33:0/3156759523 wait complete. 2023-10-06T05:16:38.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb6c46cb80 20 client.5877 trim_cache size 0 max 16384 2023-10-06T05:16:38.257 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:16:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:16:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:16:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5715, 2023-10-06T05:16:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:16:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:16:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:16:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5339195979999998, 2023-10-06T05:16:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 35, 2023-10-06T05:16:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 62, 2023-10-06T05:16:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 62, 2023-10-06T05:16:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 19.117094569999999 2023-10-06T05:16:38.259 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:16:38.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb44066f30 msgr2=0x7fdb440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb44066f30 0x7fdb440693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb58150750 tx=0x7fdb4c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 msgr2=0x7fdb580b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 0x7fdb580b0f10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b760 tx=0x7fdb4801a000 comp rx=0 tx=0).stop 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 shutdown_connections 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb44066f30 0x7fdb440693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb58159e10 0x7fdb5814f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb580c7ec0 0x7fdb580b0f10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 --2- 172.21.15.33:0/3744824417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb58156880 0x7fdb58143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 >> 172.21.15.33:0/3744824417 conn(0x7fdb5806c560 msgr2=0x7fdb58159010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 shutdown_connections 2023-10-06T05:16:38.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.247+0000 7fdb5cb9d700 1 -- 172.21.15.33:0/3744824417 wait complete. 2023-10-06T05:16:38.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:38.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.451+0000 7f571db9d700 1 Processor -- start 2023-10-06T05:16:38.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.451+0000 7f571db9d700 1 -- start start 2023-10-06T05:16:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.452+0000 7f571db9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f571813faa0 0x7f571813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.452+0000 7f571db9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57180c7940 0x7f57180c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.452+0000 7f571db9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57180c9cb0 con 0x7f571806aad0 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.452+0000 7f571db9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571806b640 con 0x7f57180c7940 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.452+0000 7f571db9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57180ca270 con 0x7f571813faa0 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f5717fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57180c7940 0x7f57180c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f571d39c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f571806aad0 0x7f571806af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58750/0 (socket says 172.21.15.33:58750) 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f571d39c700 1 -- 172.21.15.33:0/1000538443 learned_addr learned my addr 172.21.15.33:0/1000538443 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f571cb9b700 1 --2- 172.21.15.33:0/1000538443 >> v2:172.21.15.167:3300/0 conn(0x7f571813faa0 0x7f571813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f5717fff700 1 -- 172.21.15.33:0/1000538443 >> v2:172.21.15.167:3300/0 conn(0x7f571813faa0 msgr2=0x7f571813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f5717fff700 1 --2- 172.21.15.33:0/1000538443 >> v2:172.21.15.167:3300/0 conn(0x7f571813faa0 0x7f571813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f5717fff700 1 -- 172.21.15.33:0/1000538443 >> v1:172.21.15.33:6789/0 conn(0x7f571806aad0 legacy=0x7f571806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f5717fff700 1 -- 172.21.15.33:0/1000538443 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571806b840 con 0x7f57180c7940 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.453+0000 7f5717fff700 1 --2- 172.21.15.33:0/1000538443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c7940 0x7f57180c7db0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f571806fe50 tx=0x7f57040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c107c332c662d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f57177fe700 1 -- 172.21.15.33:0/1000538443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704011630 con 0x7f57180c7940 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f57177fe700 1 -- 172.21.15.33:0/1000538443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704011790 con 0x7f57180c7940 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f57177fe700 1 -- 172.21.15.33:0/1000538443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704011b30 con 0x7f57180c7940 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 -- 172.21.15.33:0/1000538443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c7940 msgr2=0x7f57180c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 --2- 172.21.15.33:0/1000538443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c7940 0x7f57180c7db0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f571806fe50 tx=0x7f57040091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 -- 172.21.15.33:0/1000538443 shutdown_connections 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 --2- 172.21.15.33:0/1000538443 >> v2:172.21.15.167:3300/0 conn(0x7f571813faa0 0x7f571813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 --2- 172.21.15.33:0/1000538443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c7940 0x7f57180c7db0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 -- 172.21.15.33:0/1000538443 >> 172.21.15.33:0/1000538443 conn(0x7f571806d030 msgr2=0x7f571806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 -- 172.21.15.33:0/1000538443 shutdown_connections 2023-10-06T05:16:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.454+0000 7f571db9d700 1 -- 172.21.15.33:0/1000538443 wait complete. 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.456+0000 7f571db9d700 1 Processor -- start 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.456+0000 7f571db9d700 1 -- start start 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571806aad0 0x7f57180c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180c7940 0x7f5718155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 0x7f57181571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180c3190 con 0x7f571813faa0 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180c3cd0 con 0x7f571806aad0 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57180c33a0 con 0x7f57180c7940 2023-10-06T05:16:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 0x7f57181571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 0x7f57181571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52534/0 (socket says 172.21.15.33:52534) 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571d39c700 1 -- 172.21.15.33:0/3894472928 learned_addr learned my addr 172.21.15.33:0/3894472928 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f571cb9b700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571806aad0 0x7f57180c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.457+0000 7f5717fff700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180c7940 0x7f5718155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571d39c700 1 -- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180c7940 msgr2=0x7f5718155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571d39c700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180c7940 0x7f5718155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571d39c700 1 -- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571806aad0 msgr2=0x7f57180c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571d39c700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571806aad0 0x7f57180c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571d39c700 1 -- 172.21.15.33:0/3894472928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704009420 con 0x7f571813faa0 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571cb9b700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571806aad0 0x7f57180c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f5717fff700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180c7940 0x7f5718155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571d39c700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 0x7f57181571f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f571000cc40 tx=0x7f5710001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000a9c0 con 0x7f571813faa0 2023-10-06T05:16:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571000ab20 con 0x7f571813faa0 2023-10-06T05:16:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100070f0 con 0x7f571813faa0 2023-10-06T05:16:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57180c3680 con 0x7f571813faa0 2023-10-06T05:16:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.458+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57180c5f80 con 0x7f571813faa0 2023-10-06T05:16:38.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.459+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57180c3e60 con 0x7f571813faa0 2023-10-06T05:16:38.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.460+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57100074e0 con 0x7f571813faa0 2023-10-06T05:16:38.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.461+0000 7f5714ff9700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:38.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.461+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5710084ab0 con 0x7f571813faa0 2023-10-06T05:16:38.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.462+0000 7f571cb9b700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:38.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.462+0000 7f571cb9b700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f570c009ce0 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:38.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.465+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5710051c40 con 0x7f571813faa0 2023-10-06T05:16:38.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.716+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f571804c380 con 0x7f571813faa0 2023-10-06T05:16:38.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.717+0000 7f5714ff9700 1 -- 172.21.15.33:0/3894472928 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5710055260 con 0x7f571813faa0 2023-10-06T05:16:38.719 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56fc066f30 msgr2=0x7f56fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f570c009ce0 tx=0x7f570c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 msgr2=0x7f57181571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 0x7f57181571f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f571000cc40 tx=0x7f5710001a50 comp rx=0 tx=0).stop 2023-10-06T05:16:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 shutdown_connections 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f571813faa0 0x7f57181571f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571806aad0 0x7f57180c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 --2- 172.21.15.33:0/3894472928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180c7940 0x7f5718155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 >> 172.21.15.33:0/3894472928 conn(0x7f571806d030 msgr2=0x7f57180ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 shutdown_connections 2023-10-06T05:16:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:38.719+0000 7f571db9d700 1 -- 172.21.15.33:0/3894472928 wait complete. 2023-10-06T05:16:38.740 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:16:38.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:38.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:16:38.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:16:38.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:38.740 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:16:38.741 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:16:38.741 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:38.741 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:38.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:38.741 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:16:38.779 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:38.779 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:38.779 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:38.779 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.779 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:38.779 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:16:38.780 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42158sec preferred_lft 42158sec 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:16:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:38.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:38.783 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:38.879 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:16:38.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:38.882 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:16:38.921 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:16:38.921 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:38.921 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:16:38.922 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:16:38.922 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:38.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:38.923 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:16:38.923 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:16:38.923 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:16:38.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:16:38.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:16:38.924 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:16:39.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:39.122 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:16:39.159 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:16:39.159 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:16:39.159 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:39.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:39.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:16:39.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:39.262 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:39.299 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:39.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:39.299 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:39.376 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:16:39.376 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:39.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:39.508 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:39.508 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:39.508 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:39.524 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:39.525 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:39.525 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:39.525 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:39.525 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:39.545 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:16:39.545 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:16:39.546 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:16:39.546 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:39.546 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:39.546 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:16:39.546 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:16:39.546 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:16:39.550 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:16:39.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:39.550 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:39.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.799+0000 7fc80559d700 1 Processor -- start 2023-10-06T05:16:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.799+0000 7fc80559d700 1 -- start start 2023-10-06T05:16:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc80559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc800155af0 0x7fc80006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc80559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc80006a6a0 0x7fc8000c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc80559d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80006ab60 con 0x7fc8000c8b50 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc80559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80006b7e0 con 0x7fc80006a6a0 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc80559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80006ae40 con 0x7fc800155af0 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc7ff7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc80006a6a0 0x7fc8000c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc804d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc8000c8b50 0x7fc8001598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58758/0 (socket says 172.21.15.33:58758) 2023-10-06T05:16:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc804d9c700 1 -- 172.21.15.33:0/1127966393 learned_addr learned my addr 172.21.15.33:0/1127966393 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:39.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc7fffff700 1 --2- 172.21.15.33:0/1127966393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc800155af0 0x7fc80006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1127966393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc800155af0 msgr2=0x7fc80006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc7ff7fe700 1 --2- 172.21.15.33:0/1127966393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc800155af0 0x7fc80006a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1127966393 >> v1:172.21.15.33:6789/0 conn(0x7fc8000c8b50 legacy=0x7fc8001598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.801+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1127966393 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80006b120 con 0x7fc80006a6a0 2023-10-06T05:16:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.802+0000 7fc7ff7fe700 1 --2- 172.21.15.33:0/1127966393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80006a6a0 0x7fc8000c7210 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bb90 tx=0x7fc7f400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800a1e25b05c3638 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:39.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.802+0000 7fc7feffd700 1 -- 172.21.15.33:0/1127966393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400da90 con 0x7fc80006a6a0 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.802+0000 7fc7feffd700 1 -- 172.21.15.33:0/1127966393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f400dbf0 con 0x7fc80006a6a0 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.802+0000 7fc7feffd700 1 -- 172.21.15.33:0/1127966393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4017410 con 0x7fc80006a6a0 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.802+0000 7fc80559d700 1 -- 172.21.15.33:0/1127966393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80006a6a0 msgr2=0x7fc8000c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.802+0000 7fc80559d700 1 --2- 172.21.15.33:0/1127966393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80006a6a0 0x7fc8000c7210 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bb90 tx=0x7fc7f400b430 comp rx=0 tx=0).stop 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.803+0000 7fc80559d700 1 -- 172.21.15.33:0/1127966393 shutdown_connections 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.803+0000 7fc80559d700 1 --2- 172.21.15.33:0/1127966393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc800155af0 0x7fc80006a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.803+0000 7fc80559d700 1 --2- 172.21.15.33:0/1127966393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80006a6a0 0x7fc8000c7210 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.803+0000 7fc80559d700 1 -- 172.21.15.33:0/1127966393 >> 172.21.15.33:0/1127966393 conn(0x7fc80006d030 msgr2=0x7fc80006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.803+0000 7fc80559d700 1 -- 172.21.15.33:0/1127966393 shutdown_connections 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.803+0000 7fc80559d700 1 -- 172.21.15.33:0/1127966393 wait complete. 2023-10-06T05:16:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.804+0000 7fc80559d700 1 Processor -- start 2023-10-06T05:16:39.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.804+0000 7fc80559d700 1 -- start start 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc80559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80006a6a0 0x7fc800159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc80559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 0x7fc8000abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc80559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc800155af0 0x7fc800154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc80559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000adbb0 con 0x7fc8000c8b50 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc80559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000ade00 con 0x7fc800155af0 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc80559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000ae0e0 con 0x7fc80006a6a0 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc7ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 0x7fc8000abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc7ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 0x7fc8000abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52560/0 (socket says 172.21.15.33:52560) 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.805+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1269328340 learned_addr learned my addr 172.21.15.33:0/1269328340 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:39.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7fffff700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80006a6a0 0x7fc800159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc804d9c700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc800155af0 0x7fc800154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80006a6a0 msgr2=0x7fc800159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7ff7fe700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80006a6a0 0x7fc800159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc800155af0 msgr2=0x7fc800154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7ff7fe700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc800155af0 0x7fc800154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7ff7fe700 1 -- 172.21.15.33:0/1269328340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f400b6d0 con 0x7fc8000c8b50 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7ff7fe700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 0x7fc8000abaf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400b470 tx=0x7fc7f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4009070 con 0x7fc8000c8b50 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f400a040 con 0x7fc8000c8b50 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4020970 con 0x7fc8000c8b50 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.806+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8000c3330 con 0x7fc8000c8b50 2023-10-06T05:16:39.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.807+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc800159580 con 0x7fc8000c8b50 2023-10-06T05:16:39.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.808+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8000c3850 con 0x7fc8000c8b50 2023-10-06T05:16:39.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.809+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7f4015070 con 0x7fc8000c8b50 2023-10-06T05:16:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.810+0000 7fc7e7fff700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7ec066f30 0x7fc7ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.810+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc7f4085ff0 con 0x7fc8000c8b50 2023-10-06T05:16:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.810+0000 7fc7fffff700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7ec066f30 0x7fc7ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:39.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.811+0000 7fc7fffff700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7ec066f30 0x7fc7ec0693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006fd0 tx=0x7fc7f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:39.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:39.814+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8000c3850 con 0x7fc8000c8b50 2023-10-06T05:16:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.064+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8000c3850 con 0x7fc8000c8b50 2023-10-06T05:16:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.065+0000 7fc7e7fff700 1 -- 172.21.15.33:0/1269328340 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc8000c3850 con 0x7fc8000c8b50 2023-10-06T05:16:40.068 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7ec066f30 msgr2=0x7fc7ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7ec066f30 0x7fc7ec0693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006fd0 tx=0x7fc7f0008040 comp rx=0 tx=0).stop 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 msgr2=0x7fc8000abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 0x7fc8000abaf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400b470 tx=0x7fc7f4019000 comp rx=0 tx=0).stop 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 shutdown_connections 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7ec066f30 0x7fc7ec0693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc800155af0 0x7fc800154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80006a6a0 0x7fc800159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 --2- 172.21.15.33:0/1269328340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8000c8b50 0x7fc8000abaf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 >> 172.21.15.33:0/1269328340 conn(0x7fc80006d030 msgr2=0x7fc800159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 shutdown_connections 2023-10-06T05:16:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:40.067+0000 7fc80559d700 1 -- 172.21.15.33:0/1269328340 wait complete. 2023-10-06T05:16:40.087 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:16:40.087 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:40.087 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:16:40.087 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:16:40.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:40.087 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:16:40.088 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:16:40.088 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:40.088 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:16:40.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.088 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:16:40.129 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:40.130 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42161sec preferred_lft 42161sec 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:16:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:16:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:40.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.133 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:40.237 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:16:40.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.240 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:16:40.279 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:16:40.280 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:40.280 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:16:40.280 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:16:40.280 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:40.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.280 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:16:40.280 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:16:40.281 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:16:40.281 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:16:40.281 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:16:40.281 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:16:40.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.475 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:16:40.511 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:16:40.511 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:16:40.511 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:16:40.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:16:40.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:40.613 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:40.649 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:16:40.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:40.649 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:40.725 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:16:40.725 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:40.725 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:40.856 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:40.856 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:16:40.856 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:16:40.872 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:40.873 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:16:40.873 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:16:40.873 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:40.873 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:40.894 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:16:40.895 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:16:40.899 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:16:40.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:40.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:16:40.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:16:41.176 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:16:41.176 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:16:41.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:16:41.380 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:16:41.381 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:16:41.381 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.565+0000 7fb37e19d700 1 Processor -- start 2023-10-06T05:16:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.565+0000 7fb37e19d700 1 -- start start 2023-10-06T05:16:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37e19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3780c8b50 0x7fb3780c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb378155ec0 0x7fb3780c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37e19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37806ab60 con 0x7fb378155af0 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37e19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37806b7e0 con 0x7fb3780c8b50 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37806ae40 con 0x7fb378155ec0 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3780c8b50 0x7fb3780c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3780c8b50 0x7fb3780c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45890/0 (socket says 172.21.15.33:45890) 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 -- 172.21.15.33:0/2433425615 learned_addr learned my addr 172.21.15.33:0/2433425615 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37d99c700 1 --2- 172.21.15.33:0/2433425615 >> v2:172.21.15.167:3300/0 conn(0x7fb378155ec0 0x7fb3780c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 -- 172.21.15.33:0/2433425615 >> v2:172.21.15.167:3300/0 conn(0x7fb378155ec0 msgr2=0x7fb3780c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 --2- 172.21.15.33:0/2433425615 >> v2:172.21.15.167:3300/0 conn(0x7fb378155ec0 0x7fb3780c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 -- 172.21.15.33:0/2433425615 >> v1:172.21.15.167:6789/0 conn(0x7fb378155af0 legacy=0x7fb37806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.567+0000 7fb37c99a700 1 -- 172.21.15.33:0/2433425615 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37806b120 con 0x7fb3780c8b50 2023-10-06T05:16:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.568+0000 7fb37c99a700 1 --2- 172.21.15.33:0/2433425615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3780c8b50 0x7fb3780c7210 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb37806fdf0 tx=0x7fb368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5118f2f535b7927 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.568+0000 7fb36ffff700 1 -- 172.21.15.33:0/2433425615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800b820 con 0x7fb3780c8b50 2023-10-06T05:16:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.568+0000 7fb36ffff700 1 -- 172.21.15.33:0/2433425615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36800b980 con 0x7fb3780c8b50 2023-10-06T05:16:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.568+0000 7fb36ffff700 1 -- 172.21.15.33:0/2433425615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800bd20 con 0x7fb3780c8b50 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.568+0000 7fb37e19d700 1 -- 172.21.15.33:0/2433425615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3780c8b50 msgr2=0x7fb3780c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.568+0000 7fb37e19d700 1 --2- 172.21.15.33:0/2433425615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3780c8b50 0x7fb3780c7210 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb37806fdf0 tx=0x7fb368009250 comp rx=0 tx=0).stop 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.569+0000 7fb37e19d700 1 -- 172.21.15.33:0/2433425615 shutdown_connections 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.569+0000 7fb37e19d700 1 --2- 172.21.15.33:0/2433425615 >> v2:172.21.15.167:3300/0 conn(0x7fb378155ec0 0x7fb3780c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.569+0000 7fb37e19d700 1 --2- 172.21.15.33:0/2433425615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3780c8b50 0x7fb3780c7210 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.569+0000 7fb37e19d700 1 -- 172.21.15.33:0/2433425615 >> 172.21.15.33:0/2433425615 conn(0x7fb37806d030 msgr2=0x7fb37806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.569+0000 7fb37e19d700 1 -- 172.21.15.33:0/2433425615 shutdown_connections 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.569+0000 7fb37e19d700 1 -- 172.21.15.33:0/2433425615 wait complete. 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.570+0000 7fb37e19d700 1 Processor -- start 2023-10-06T05:16:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.570+0000 7fb37e19d700 1 -- start start 2023-10-06T05:16:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3780c8b50 0x7fb378157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb378155af0 0x7fb3781587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 0x7fb378158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb378158220 con 0x7fb378155ec0 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780abbf0 con 0x7fb378155af0 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780c52e0 con 0x7fb3780c8b50 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 0x7fb378158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 0x7fb378158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45906/0 (socket says 172.21.15.33:45906) 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37d99c700 1 -- 172.21.15.33:0/3324440716 learned_addr learned my addr 172.21.15.33:0/3324440716 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37c99a700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb378155af0 0x7fb3781587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.571+0000 7fb37d19b700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3780c8b50 0x7fb378157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d99c700 1 -- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3780c8b50 msgr2=0x7fb378157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d99c700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3780c8b50 0x7fb378157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d99c700 1 -- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb378155af0 msgr2=0x7fb3781587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d99c700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb378155af0 0x7fb3781587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d99c700 1 -- 172.21.15.33:0/3324440716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680094f0 con 0x7fb378155ec0 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d19b700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3780c8b50 0x7fb378157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37d99c700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 0x7fb378158d20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb370007f20 tx=0x7fb370009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000ebf0 con 0x7fb378155ec0 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37000ed70 con 0x7fb378155ec0 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370014670 con 0x7fb378155ec0 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3780c49b0 con 0x7fb378155ec0 2023-10-06T05:16:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.572+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3780c4f70 con 0x7fb378155ec0 2023-10-06T05:16:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.573+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37804c380 con 0x7fb378155ec0 2023-10-06T05:16:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.574+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3700149e0 con 0x7fb378155ec0 2023-10-06T05:16:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.575+0000 7fb36d7fa700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb35c066f30 0x7fb35c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.575+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb370084dc0 con 0x7fb378155ec0 2023-10-06T05:16:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.576+0000 7fb37d19b700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb35c066f30 0x7fb35c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.576+0000 7fb37d19b700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb35c066f30 0x7fb35c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb374006fd0 tx=0x7fb374008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.580+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb370051e30 con 0x7fb378155ec0 2023-10-06T05:16:41.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.825+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3780c5420 con 0x7fb378155ec0 2023-10-06T05:16:41.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.826+0000 7fb36d7fa700 1 -- 172.21.15.33:0/3324440716 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fb370055450 con 0x7fb378155ec0 2023-10-06T05:16:41.828 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:41.829 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb35c066f30 msgr2=0x7fb35c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb35c066f30 0x7fb35c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb374006fd0 tx=0x7fb374008040 comp rx=0 tx=0).stop 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 msgr2=0x7fb378158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 0x7fb378158d20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb370007f20 tx=0x7fb370009860 comp rx=0 tx=0).stop 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 shutdown_connections 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb35c066f30 0x7fb35c0693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb378155ec0 0x7fb378158d20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3780c8b50 0x7fb378157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 --2- 172.21.15.33:0/3324440716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb378155af0 0x7fb3781587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 >> 172.21.15.33:0/3324440716 conn(0x7fb37806d030 msgr2=0x7fb3780c7240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 shutdown_connections 2023-10-06T05:16:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:41.828+0000 7fb37e19d700 1 -- 172.21.15.33:0/3324440716 wait complete. 2023-10-06T05:16:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:16:41.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_defer_session_stale false 2023-10-06T05:16:42.057 INFO:teuthology.orchestra.run.smithi033.stdout:{"success":"mds_defer_session_stale = 'false' (not observed, change may require restart) "} 2023-10-06T05:16:42.058 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_defer_session_stale = 'false' (not observed, change may require restart) " } 2023-10-06T05:16:42.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.242+0000 7fc51e1ff700 1 Processor -- start 2023-10-06T05:16:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.243+0000 7fc51e1ff700 1 -- start start 2023-10-06T05:16:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc51e1ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518069a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc51e1ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc518157740 0x7fc518069f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc51e1ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51815b0c0 con 0x7fc5180c8b20 2023-10-06T05:16:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc51e1ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51815c0d0 con 0x7fc518157740 2023-10-06T05:16:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc51e1ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51815bc40 con 0x7fc518157370 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc516ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc518157740 0x7fc518069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc517fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc5180c8b20 0x7fc5180c8f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46790/0 (socket says 172.21.15.33:46790) 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc517fff700 1 -- 172.21.15.33:0/4140795122 learned_addr learned my addr 172.21.15.33:0/4140795122 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.244+0000 7fc5177fe700 1 --2- 172.21.15.33:0/4140795122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc5167fc700 1 -- 172.21.15.33:0/4140795122 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747284011 0 0) 0x7fc51815b0c0 con 0x7fc5180c8b20 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc516ffd700 1 -- 172.21.15.33:0/4140795122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 msgr2=0x7fc518069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc516ffd700 1 --2- 172.21.15.33:0/4140795122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518069a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc516ffd700 1 -- 172.21.15.33:0/4140795122 >> v1:172.21.15.33:6789/0 conn(0x7fc5180c8b20 legacy=0x7fc5180c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc516ffd700 1 -- 172.21.15.33:0/4140795122 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51815b310 con 0x7fc518157740 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc516ffd700 1 --2- 172.21.15.33:0/4140795122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc518069f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0098e0 tx=0x7fc50c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f013c85188cdce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc5167fc700 1 -- 172.21.15.33:0/4140795122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00b640 con 0x7fc518157740 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc5167fc700 1 -- 172.21.15.33:0/4140795122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c00b7a0 con 0x7fc518157740 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.245+0000 7fc5167fc700 1 -- 172.21.15.33:0/4140795122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00bb40 con 0x7fc518157740 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4140795122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 msgr2=0x7fc518069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4140795122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc518069f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0098e0 tx=0x7fc50c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4140795122 shutdown_connections 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4140795122 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4140795122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc518069f90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4140795122 >> 172.21.15.33:0/4140795122 conn(0x7fc5180675f0 msgr2=0x7fc5180679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4140795122 shutdown_connections 2023-10-06T05:16:42.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.246+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4140795122 wait complete. 2023-10-06T05:16:42.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.247+0000 7fc51e1ff700 1 Processor -- start 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.248+0000 7fc51e1ff700 1 -- start start 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc51e1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5180c8b20 0x7fc518150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc51e1ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc51e1ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc5180c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc51e1ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc518159390 con 0x7fc518157740 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc51e1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180c5640 con 0x7fc5180c8b20 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc51e1ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180c5920 con 0x7fc518157370 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc5180c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc5180c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45936/0 (socket says 172.21.15.33:45936) 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 -- 172.21.15.33:0/4227003922 learned_addr learned my addr 172.21.15.33:0/4227003922 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc5177fe700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5180c8b20 0x7fc518150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc516ffd700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 -- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 msgr2=0x7fc518158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 -- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5180c8b20 msgr2=0x7fc518150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5180c8b20 0x7fc518150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.249+0000 7fc517fff700 1 -- 172.21.15.33:0/4227003922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c009420 con 0x7fc518157740 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.250+0000 7fc517fff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc5180c4180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc51000cbf0 tx=0x7fc5100019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.250+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000a8b0 con 0x7fc518157740 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.250+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000aa10 con 0x7fc518157740 2023-10-06T05:16:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.250+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000add0 con 0x7fc518157740 2023-10-06T05:16:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.250+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5180c5c00 con 0x7fc518157740 2023-10-06T05:16:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.250+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5180c46c0 con 0x7fc518157740 2023-10-06T05:16:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.251+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518063ff0 con 0x7fc518157740 2023-10-06T05:16:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.252+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc510007300 con 0x7fc518157740 2023-10-06T05:16:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.253+0000 7fc4ff7fe700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc504066f30 0x7fc5040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.253+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc5100849a0 con 0x7fc518157740 2023-10-06T05:16:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.253+0000 7fc5177fe700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc504066f30 0x7fc5040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.254+0000 7fc5177fe700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc504066f30 0x7fc5040693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc508009db0 tx=0x7fc5080091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.257+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc510051a90 con 0x7fc518157740 2023-10-06T05:16:42.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.503+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5180c7eb0 con 0x7fc518157740 2023-10-06T05:16:42.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.504+0000 7fc4ff7fe700 1 -- 172.21.15.33:0/4227003922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fc5100550b0 con 0x7fc518157740 2023-10-06T05:16:42.507 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:42.507 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:16:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc504066f30 msgr2=0x7fc5040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc504066f30 0x7fc5040693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc508009db0 tx=0x7fc5080091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 msgr2=0x7fc5180c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc5180c4180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc51000cbf0 tx=0x7fc5100019b0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 shutdown_connections 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc504066f30 0x7fc5040693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc518157740 0x7fc5180c4180 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5180c8b20 0x7fc518150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 --2- 172.21.15.33:0/4227003922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc518157370 0x7fc518158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.506+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 >> 172.21.15.33:0/4227003922 conn(0x7fc5180675f0 msgr2=0x7fc518150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.507+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 shutdown_connections 2023-10-06T05:16:42.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.507+0000 7fc51e1ff700 1 -- 172.21.15.33:0/4227003922 wait complete. 2023-10-06T05:16:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:16:42.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:42.527 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:16:42.599 INFO:teuthology.orchestra.run.smithi033.stdout:client5889 2023-10-06T05:16:42.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:42.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.783+0000 7fb1a279d700 1 Processor -- start 2023-10-06T05:16:42.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.783+0000 7fb1a279d700 1 -- start start 2023-10-06T05:16:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb19c0b0a40 0x7fb19c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb19c15a0a0 0x7fb19c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a279d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19c15e2d0 con 0x7fb19c0b0e10 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c15d6c0 con 0x7fb19c0b0a40 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c15d9a0 con 0x7fb19c15a0a0 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb19c0b0a40 0x7fb19c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb19c0b0a40 0x7fb19c06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45946/0 (socket says 172.21.15.33:45946) 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 -- 172.21.15.33:0/1787206177 learned_addr learned my addr 172.21.15.33:0/1787206177 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a1f9c700 1 --2- 172.21.15.33:0/1787206177 >> v2:172.21.15.167:3300/0 conn(0x7fb19c15a0a0 0x7fb19c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 -- 172.21.15.33:0/1787206177 >> v2:172.21.15.167:3300/0 conn(0x7fb19c15a0a0 msgr2=0x7fb19c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 --2- 172.21.15.33:0/1787206177 >> v2:172.21.15.167:3300/0 conn(0x7fb19c15a0a0 0x7fb19c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 -- 172.21.15.33:0/1787206177 >> v1:172.21.15.167:6789/0 conn(0x7fb19c0b0e10 legacy=0x7fb19c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a179b700 1 -- 172.21.15.33:0/1787206177 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c15dc80 con 0x7fb19c0b0a40 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.785+0000 7fb1a1f9c700 1 --2- 172.21.15.33:0/1787206177 >> v2:172.21.15.167:3300/0 conn(0x7fb19c15a0a0 0x7fb19c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.786+0000 7fb1a179b700 1 --2- 172.21.15.33:0/1787206177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c06ab00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0b0740 tx=0x7fb198009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60caceb3c26745ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.786+0000 7fb193fff700 1 -- 172.21.15.33:0/1787206177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800b860 con 0x7fb19c0b0a40 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.786+0000 7fb193fff700 1 -- 172.21.15.33:0/1787206177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19800b9c0 con 0x7fb19c0b0a40 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.786+0000 7fb193fff700 1 -- 172.21.15.33:0/1787206177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800bd60 con 0x7fb19c0b0a40 2023-10-06T05:16:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.786+0000 7fb1a279d700 1 -- 172.21.15.33:0/1787206177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 msgr2=0x7fb19c06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.786+0000 7fb1a279d700 1 --2- 172.21.15.33:0/1787206177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c06ab00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0b0740 tx=0x7fb198009250 comp rx=0 tx=0).stop 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.787+0000 7fb1a279d700 1 -- 172.21.15.33:0/1787206177 shutdown_connections 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.787+0000 7fb1a279d700 1 --2- 172.21.15.33:0/1787206177 >> v2:172.21.15.167:3300/0 conn(0x7fb19c15a0a0 0x7fb19c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.787+0000 7fb1a279d700 1 --2- 172.21.15.33:0/1787206177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c06ab00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.787+0000 7fb1a279d700 1 -- 172.21.15.33:0/1787206177 >> 172.21.15.33:0/1787206177 conn(0x7fb19c06e0f0 msgr2=0x7fb19c0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.787+0000 7fb1a279d700 1 -- 172.21.15.33:0/1787206177 shutdown_connections 2023-10-06T05:16:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.787+0000 7fb1a279d700 1 -- 172.21.15.33:0/1787206177 wait complete. 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.788+0000 7fb1a279d700 1 Processor -- start 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.788+0000 7fb1a279d700 1 -- start start 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb19c0b0e10 0x7fb19c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb19c15a0a0 0x7fb19c0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0c5ef0 con 0x7fb19c0b0a40 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0c3e00 con 0x7fb19c0b0e10 2023-10-06T05:16:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0c34d0 con 0x7fb19c15a0a0 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c145600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45954/0 (socket says 172.21.15.33:45954) 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.789+0000 7fb1a179b700 1 -- 172.21.15.33:0/3580728524 learned_addr learned my addr 172.21.15.33:0/3580728524 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a179b700 1 -- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb19c15a0a0 msgr2=0x7fb19c0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a179b700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb19c15a0a0 0x7fb19c0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a179b700 1 -- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb19c0b0e10 msgr2=0x7fb19c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a179b700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb19c0b0e10 0x7fb19c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a179b700 1 -- 172.21.15.33:0/3580728524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980094f0 con 0x7fb19c0b0a40 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a179b700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c145600 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb1980090b0 tx=0x7fb198017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198011e20 con 0x7fb19c0b0a40 2023-10-06T05:16:42.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19801a660 con 0x7fb19c0b0a40 2023-10-06T05:16:42.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c0c37b0 con 0x7fb19c0b0a40 2023-10-06T05:16:42.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19801aa20 con 0x7fb19c0b0a40 2023-10-06T05:16:42.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.790+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c145b40 con 0x7fb19c0b0a40 2023-10-06T05:16:42.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.791+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c0b1bc0 con 0x7fb19c0b0a40 2023-10-06T05:16:42.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.792+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb19801b430 con 0x7fb19c0b0a40 2023-10-06T05:16:42.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.793+0000 7fb1917fa700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb180066f30 0x7fb1800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.793+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb198085e90 con 0x7fb19c0b0a40 2023-10-06T05:16:42.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.794+0000 7fb1a0f9a700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb180066f30 0x7fb1800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.794+0000 7fb1a0f9a700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb180066f30 0x7fb1800693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb19c149030 tx=0x7fb18c007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:42.797+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb198052f00 con 0x7fb19c0b0a40 2023-10-06T05:16:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.045+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb19c06fcc0 con 0x7fb19c0b0a40 2023-10-06T05:16:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.045+0000 7fb1917fa700 1 -- 172.21.15.33:0/3580728524 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fb198056520 con 0x7fb19c0b0a40 2023-10-06T05:16:43.048 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:43.048 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb180066f30 msgr2=0x7fb1800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb180066f30 0x7fb1800693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb19c149030 tx=0x7fb18c007b40 comp rx=0 tx=0).stop 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 msgr2=0x7fb19c145600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c145600 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb1980090b0 tx=0x7fb198017000 comp rx=0 tx=0).stop 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 shutdown_connections 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb180066f30 0x7fb1800693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb19c15a0a0 0x7fb19c0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb19c0b0a40 0x7fb19c145600 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 --2- 172.21.15.33:0/3580728524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb19c0b0e10 0x7fb19c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 >> 172.21.15.33:0/3580728524 conn(0x7fb19c06e0f0 msgr2=0x7fb19c0aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 shutdown_connections 2023-10-06T05:16:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.048+0000 7fb1a279d700 1 -- 172.21.15.33:0/3580728524 wait complete. 2023-10-06T05:16:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:16:43.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session config 5889 timeout 120 2023-10-06T05:16:43.278 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:16:43.278 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T05:16:43.278 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:43.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:43.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:16:43.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.533+0000 7fb43179d700 1 Processor -- start 2023-10-06T05:16:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.533+0000 7fb43179d700 1 -- start start 2023-10-06T05:16:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.534+0000 7fb43179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb42c159530 0x7fb42c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.534+0000 7fb43179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.534+0000 7fb43179d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c06e3e0 con 0x7fb42c0b0310 2023-10-06T05:16:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.534+0000 7fb43179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42c0b1e00 con 0x7fb42c159530 2023-10-06T05:16:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.534+0000 7fb43179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c06f6b0 con 0x7fb42c06df70 2023-10-06T05:16:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb42c159530 0x7fb42c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb430f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb42c0b0310 0x7fb42c159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46804/0 (socket says 172.21.15.33:46804) 2023-10-06T05:16:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb430f9c700 1 -- 172.21.15.33:0/3938494210 learned_addr learned my addr 172.21.15.33:0/3938494210 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:16:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42b7fe700 1 --2- 172.21.15.33:0/3938494210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42bfff700 1 -- 172.21.15.33:0/3938494210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 msgr2=0x7fb42c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42bfff700 1 --2- 172.21.15.33:0/3938494210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c0afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42bfff700 1 -- 172.21.15.33:0/3938494210 >> v1:172.21.15.33:6789/0 conn(0x7fb42c0b0310 legacy=0x7fb42c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42bfff700 1 -- 172.21.15.33:0/3938494210 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c06f220 con 0x7fb42c159530 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.535+0000 7fb42affd700 1 -- 172.21.15.33:0/3938494210 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051321011 0 0) 0x7fb42c06e3e0 con 0x7fb42c0b0310 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb42bfff700 1 --2- 172.21.15.33:0/3938494210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c159530 0x7fb42c06da30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0098e0 tx=0x7fb41c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188ab65913689b8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb42affd700 1 -- 172.21.15.33:0/3938494210 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00b640 con 0x7fb42c159530 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb42affd700 1 -- 172.21.15.33:0/3938494210 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c00b7a0 con 0x7fb42c159530 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb42affd700 1 -- 172.21.15.33:0/3938494210 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00bb40 con 0x7fb42c159530 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 -- 172.21.15.33:0/3938494210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c159530 msgr2=0x7fb42c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 --2- 172.21.15.33:0/3938494210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c159530 0x7fb42c06da30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0098e0 tx=0x7fb41c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 -- 172.21.15.33:0/3938494210 shutdown_connections 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 --2- 172.21.15.33:0/3938494210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c159530 0x7fb42c06da30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 --2- 172.21.15.33:0/3938494210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 -- 172.21.15.33:0/3938494210 >> 172.21.15.33:0/3938494210 conn(0x7fb42c06c560 msgr2=0x7fb42c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 -- 172.21.15.33:0/3938494210 shutdown_connections 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.536+0000 7fb43179d700 1 -- 172.21.15.33:0/3938494210 wait complete. 2023-10-06T05:16:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.538+0000 7fb43179d700 1 Processor -- start 2023-10-06T05:16:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.538+0000 7fb43179d700 1 -- start start 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb43179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb43179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 0x7fb42c16d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb43179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c159530 0x7fb42c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb43179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1521e0 con 0x7fb42c0b0310 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb43179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c16dd00 con 0x7fb42c159530 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb43179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c16dfe0 con 0x7fb42c06df70 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb42b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 0x7fb42c16d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb42b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 0x7fb42c16d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45968/0 (socket says 172.21.15.33:45968) 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb42b7fe700 1 -- 172.21.15.33:0/3147094935 learned_addr learned my addr 172.21.15.33:0/3147094935 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb430f9c700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c159530 0x7fb42c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.539+0000 7fb42bfff700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42b7fe700 1 -- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 msgr2=0x7fb42c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42b7fe700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c15e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42b7fe700 1 -- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c159530 msgr2=0x7fb42c14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42b7fe700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c159530 0x7fb42c14dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42b7fe700 1 -- 172.21.15.33:0/3147094935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009420 con 0x7fb42c0b0310 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42bfff700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb42b7fe700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 0x7fb42c16d240 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb42000ec80 tx=0x7fb4200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42000c850 con 0x7fb42c0b0310 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42000c9b0 con 0x7fb42c0b0310 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c152430 con 0x7fb42c0b0310 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42000cd50 con 0x7fb42c0b0310 2023-10-06T05:16:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.540+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c0afb60 con 0x7fb42c0b0310 2023-10-06T05:16:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.541+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c152a80 con 0x7fb42c0b0310 2023-10-06T05:16:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.542+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb420013680 con 0x7fb42c0b0310 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.543+0000 7fb413fff700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb418066f30 0x7fb4180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.544+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb4200849d0 con 0x7fb42c0b0310 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.544+0000 7fb42bfff700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb418066f30 0x7fb4180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.545+0000 7fb42bfff700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb418066f30 0x7fb4180693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0091e0 tx=0x7fb41c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.548+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb420051a40 con 0x7fb42c0b0310 2023-10-06T05:16:43.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.796+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb42c04c380 con 0x7fb42c0b0310 2023-10-06T05:16:43.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.797+0000 7fb413fff700 1 -- 172.21.15.33:0/3147094935 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fb420055060 con 0x7fb42c0b0310 2023-10-06T05:16:43.800 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:16:43.800 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb418066f30 msgr2=0x7fb4180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb418066f30 0x7fb4180693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0091e0 tx=0x7fb41c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 msgr2=0x7fb42c16d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 0x7fb42c16d240 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb42000ec80 tx=0x7fb4200091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 shutdown_connections 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb418066f30 0x7fb4180693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c159530 0x7fb42c14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06df70 0x7fb42c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 --2- 172.21.15.33:0/3147094935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0b0310 0x7fb42c16d240 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 >> 172.21.15.33:0/3147094935 conn(0x7fb42c06c560 msgr2=0x7fb42c159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 shutdown_connections 2023-10-06T05:16:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:16:43.799+0000 7fb43179d700 1 -- 172.21.15.33:0/3147094935 wait complete. 2023-10-06T05:16:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:16:43.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:16:44.031 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":5901,"entity":{"name":{"type":"client","num":5901},"addr":{"type":"v1","addr":"172.21.15.167:0","nonce":2450314073}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.1259516930000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99281041894333733,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5901 v1:172.21.15.167:0/2450314073","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":5889,"entity":{"name":{"type":"client","num":5889},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4010096267}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":4,"uptime":4.4739308619999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.98972312781237004,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5889 v1:192.168.0.1:0/4010096267","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/","timeout":"120"}}] 2023-10-06T05:16:44.032 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.167:0", "nonce": 2450314073, "type": "v1" }, "name": { "num": 5901, "type": "client" } }, "id": 5901, "inst": "client.5901 v1:172.21.15.167:0/2450314073", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9928104189433373 }, "state": "open", "uptime": 3.125951693 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/", "timeout": "120" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4010096267, "type": "v1" }, "name": { "num": 5889, "type": "client" } }, "id": 5889, "inst": "client.5889 v1:192.168.0.1:0/4010096267", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 4, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.98972312781237 }, "state": "open", "uptime": 4.473930862 } ] 2023-10-06T05:16:47.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:47.591 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:17.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:17.693 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:47.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:47.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:14.033 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.231+0000 7f830219d700 1 Processor -- start 2023-10-06T05:18:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.231+0000 7f830219d700 1 -- start start 2023-10-06T05:18:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830219d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82fc155140 0x7f82fc158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830219d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0c8e40 con 0x7f82fc0699c0 2023-10-06T05:18:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830219d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0c99e0 con 0x7f82fc0adf50 2023-10-06T05:18:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830219d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc0ae3f0 con 0x7f82fc155140 2023-10-06T05:18:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82fc155140 0x7f82fc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82fc155140 0x7f82fc158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32792/0 (socket says 172.21.15.33:32792) 2023-10-06T05:18:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830119b700 1 -- 172.21.15.33:0/482298313 learned_addr learned my addr 172.21.15.33:0/482298313 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:18:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830119b700 1 -- 172.21.15.33:0/482298313 >> v1:172.21.15.167:6789/0 conn(0x7f82fc0adf50 legacy=0x7f82fc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830119b700 1 -- 172.21.15.33:0/482298313 >> v1:172.21.15.33:6789/0 conn(0x7f82fc0699c0 legacy=0x7f82fc0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.233+0000 7f830119b700 1 -- 172.21.15.33:0/482298313 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0ae6c0 con 0x7f82fc155140 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.234+0000 7f830119b700 1 --2- 172.21.15.33:0/482298313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc155140 0x7f82fc158700 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f82f80099e0 tx=0x7f82f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c410684e9221abcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.234+0000 7f82f3fff700 1 -- 172.21.15.33:0/482298313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800b7e0 con 0x7f82fc155140 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.234+0000 7f82f3fff700 1 -- 172.21.15.33:0/482298313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f800b940 con 0x7f82fc155140 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.234+0000 7f82f3fff700 1 -- 172.21.15.33:0/482298313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800bce0 con 0x7f82fc155140 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.234+0000 7f830219d700 1 -- 172.21.15.33:0/482298313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc155140 msgr2=0x7f82fc158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.234+0000 7f830219d700 1 --2- 172.21.15.33:0/482298313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc155140 0x7f82fc158700 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f82f80099e0 tx=0x7f82f8009250 comp rx=0 tx=0).stop 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.235+0000 7f830219d700 1 -- 172.21.15.33:0/482298313 shutdown_connections 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.235+0000 7f830219d700 1 --2- 172.21.15.33:0/482298313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc155140 0x7f82fc158700 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.235+0000 7f830219d700 1 -- 172.21.15.33:0/482298313 >> 172.21.15.33:0/482298313 conn(0x7f82fc06d030 msgr2=0x7f82fc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.235+0000 7f830219d700 1 -- 172.21.15.33:0/482298313 shutdown_connections 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.235+0000 7f830219d700 1 -- 172.21.15.33:0/482298313 wait complete. 2023-10-06T05:18:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.236+0000 7f830219d700 1 Processor -- start 2023-10-06T05:18:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.236+0000 7f830219d700 1 -- start start 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 0x7f82fc0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc0adf50 0x7f82fc14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82fc155140 0x7f82fc0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c4680 con 0x7f82fc0699c0 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c3a70 con 0x7f82fc0adf50 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c3d50 con 0x7f82fc155140 2023-10-06T05:18:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 0x7f82fc0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 0x7f82fc0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32804/0 (socket says 172.21.15.33:32804) 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.237+0000 7f830119b700 1 -- 172.21.15.33:0/2980789086 learned_addr learned my addr 172.21.15.33:0/2980789086 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830099a700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc0adf50 0x7f82fc14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830199c700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82fc155140 0x7f82fc0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830119b700 1 -- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82fc155140 msgr2=0x7f82fc0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830119b700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82fc155140 0x7f82fc0b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830119b700 1 -- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc0adf50 msgr2=0x7f82fc14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830119b700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc0adf50 0x7f82fc14efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830119b700 1 -- 172.21.15.33:0/2980789086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80094f0 con 0x7f82fc0699c0 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830119b700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 0x7f82fc0c3190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f82f80099b0 tx=0x7f82f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8011db0 con 0x7f82fc0699c0 2023-10-06T05:18:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f8020070 con 0x7f82fc0699c0 2023-10-06T05:18:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f801b6a0 con 0x7f82fc0699c0 2023-10-06T05:18:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.238+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc0c4030 con 0x7f82fc0699c0 2023-10-06T05:18:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.239+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc0c36d0 con 0x7f82fc0699c0 2023-10-06T05:18:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.240+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc04c380 con 0x7f82fc0699c0 2023-10-06T05:18:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.240+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82f8022030 con 0x7f82fc0699c0 2023-10-06T05:18:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.242+0000 7f82f17fa700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.242+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f82f8086630 con 0x7f82fc0699c0 2023-10-06T05:18:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.242+0000 7f830099a700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.243+0000 7f830099a700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82e0066f30 0x7f82e00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f82ec006fd0 tx=0x7f82ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.247+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82f80536d0 con 0x7f82fc0699c0 2023-10-06T05:18:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.497+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82fc04c380 con 0x7f82fc0699c0 2023-10-06T05:18:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.498+0000 7f82f17fa700 1 -- 172.21.15.33:0/2980789086 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f82f8056cf0 con 0x7f82fc0699c0 2023-10-06T05:18:14.500 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:18:14.500 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82e0066f30 msgr2=0x7f82e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82e0066f30 0x7f82e00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f82ec006fd0 tx=0x7f82ec008040 comp rx=0 tx=0).stop 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 msgr2=0x7f82fc0c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 0x7f82fc0c3190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f82f80099b0 tx=0x7f82f8016000 comp rx=0 tx=0).stop 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 shutdown_connections 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82fc155140 0x7f82fc0b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82fc0699c0 0x7f82fc0c3190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 --2- 172.21.15.33:0/2980789086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82fc0adf50 0x7f82fc14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 >> 172.21.15.33:0/2980789086 conn(0x7f82fc06d030 msgr2=0x7f82fc14ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 shutdown_connections 2023-10-06T05:18:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:18:14.500+0000 7f830219d700 1 -- 172.21.15.33:0/2980789086 wait complete. 2023-10-06T05:18:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:18:14.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:18:14.730 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":5901,"entity":{"name":{"type":"client","num":5901},"addr":{"type":"v1","addr":"172.21.15.167:0","nonce":2450314073}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":93.825549742000007,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.80511019758023772,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5901 v1:172.21.15.167:0/2450314073","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":5889,"entity":{"name":{"type":"client","num":5889},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4010096267}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":1,"uptime":95.173528911000005,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.80260658810453711,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5889 v1:192.168.0.1:0/4010096267","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/","timeout":"120"}}] 2023-10-06T05:18:14.730 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.167:0", "nonce": 2450314073, "type": "v1" }, "name": { "num": 5901, "type": "client" } }, "id": 5901, "inst": "client.5901 v1:172.21.15.167:0/2450314073", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.8051101975802377 }, "state": "open", "uptime": 93.825549742 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/", "timeout": "120" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4010096267, "type": "v1" }, "name": { "num": 5889, "type": "client" } }, "id": 5889, "inst": "client.5889 v1:192.168.0.1:0/4010096267", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.8026065881045371 }, "state": "open", "uptime": 95.173528911 } ] 2023-10-06T05:18:17.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:17.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:48.060 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:48.065 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:14.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:14.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.930+0000 7f9fd0b9d700 1 Processor -- start 2023-10-06T05:19:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.930+0000 7f9fd0b9d700 1 -- start start 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fd0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fd0b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fcc0cf3b0 0x7f9fcc1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fd0b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9fcc0cc750 0x7f9fcc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fd0b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fcc0cbd80 con 0x7f9fcc0cf3b0 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fd0b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fcc0af5f0 con 0x7f9fcc0cc750 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fd0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc0af8d0 con 0x7f9fcc0ccfc0 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.931+0000 7f9fcaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fcc0cf3b0 0x7f9fcc1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fcc0cf3b0 0x7f9fcc1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51244/0 (socket says 172.21.15.33:51244) 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 -- 172.21.15.33:0/2561882714 learned_addr learned my addr 172.21.15.33:0/2561882714 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcb7fe700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcbfff700 1 --2- 172.21.15.33:0/2561882714 >> v2:172.21.15.167:3300/0 conn(0x7f9fcc0cc750 0x7f9fcc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 -- 172.21.15.33:0/2561882714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 msgr2=0x7f9fcc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 -- 172.21.15.33:0/2561882714 >> v2:172.21.15.167:3300/0 conn(0x7f9fcc0cc750 msgr2=0x7f9fcc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 --2- 172.21.15.33:0/2561882714 >> v2:172.21.15.167:3300/0 conn(0x7f9fcc0cc750 0x7f9fcc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 -- 172.21.15.33:0/2561882714 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc0afbb0 con 0x7f9fcc0cf3b0 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcb7fe700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcbfff700 1 --2- 172.21.15.33:0/2561882714 >> v2:172.21.15.167:3300/0 conn(0x7f9fcc0cc750 0x7f9fcc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.932+0000 7f9fcaffd700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cf3b0 0x7f9fcc1594a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0cdb60 tx=0x7f9fc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac1dbd18f887b67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fca7fc700 1 -- 172.21.15.33:0/2561882714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000b860 con 0x7f9fcc0cf3b0 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fca7fc700 1 -- 172.21.15.33:0/2561882714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000b9c0 con 0x7f9fcc0cf3b0 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fca7fc700 1 -- 172.21.15.33:0/2561882714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000bd60 con 0x7f9fcc0cf3b0 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/2561882714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cf3b0 msgr2=0x7f9fcc1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cf3b0 0x7f9fcc1594a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0cdb60 tx=0x7f9fc0009250 comp rx=0 tx=0).stop 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/2561882714 shutdown_connections 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/2561882714 >> v2:172.21.15.167:3300/0 conn(0x7f9fcc0cc750 0x7f9fcc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/2561882714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cf3b0 0x7f9fcc1594a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/2561882714 >> 172.21.15.33:0/2561882714 conn(0x7f9fcc06cff0 msgr2=0x7f9fcc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/2561882714 shutdown_connections 2023-10-06T05:19:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.933+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/2561882714 wait complete. 2023-10-06T05:19:14.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.935+0000 7f9fd0b9d700 1 Processor -- start 2023-10-06T05:19:14.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.935+0000 7f9fd0b9d700 1 -- start start 2023-10-06T05:19:14.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fd0b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 0x7f9fcc16d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fd0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc16dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fd0b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fcc1694a0 0x7f9fcc169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fd0b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc0c82d0 con 0x7f9fcc0cc750 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fd0b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc0c79e0 con 0x7f9fcc1694a0 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fd0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc0c7c80 con 0x7f9fcc0ccfc0 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fcb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 0x7f9fcc16d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fcb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 0x7f9fcc16d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51250/0 (socket says 172.21.15.33:51250) 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fcb7fe700 1 -- 172.21.15.33:0/539728803 learned_addr learned my addr 172.21.15.33:0/539728803 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fcaffd700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc16dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.936+0000 7f9fcbfff700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fcc1694a0 0x7f9fcc169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcb7fe700 1 -- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 msgr2=0x7f9fcc16dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcb7fe700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc16dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcb7fe700 1 -- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fcc1694a0 msgr2=0x7f9fcc169920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcb7fe700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fcc1694a0 0x7f9fcc169920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:14.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcb7fe700 1 -- 172.21.15.33:0/539728803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00094f0 con 0x7f9fcc0cc750 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcbfff700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fcc1694a0 0x7f9fcc169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fcb7fe700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 0x7f9fcc16d980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00e850 tx=0x7f9fbc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc013410 con 0x7f9fcc0cc750 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc013570 con 0x7f9fcc0cc750 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fcc0c7f60 con 0x7f9fcc0cc750 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc013910 con 0x7f9fcc0cc750 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.937+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fcc169e60 con 0x7f9fcc0cc750 2023-10-06T05:19:14.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.938+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fcc0b2e00 con 0x7f9fcc0cc750 2023-10-06T05:19:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.939+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fbc013d00 con 0x7f9fcc0cc750 2023-10-06T05:19:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.940+0000 7f9fb37fe700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb8066f30 0x7f9fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.940+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9fbc084450 con 0x7f9fcc0cc750 2023-10-06T05:19:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.941+0000 7f9fcaffd700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb8066f30 0x7f9fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.941+0000 7f9fcaffd700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb8066f30 0x7f9fb80693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0b0250 tx=0x7f9fc0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:14.944+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fbc0514a0 con 0x7f9fcc0cc750 2023-10-06T05:19:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.195+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fcc0c85b0 con 0x7f9fcc0cc750 2023-10-06T05:19:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.195+0000 7f9fb37fe700 1 -- 172.21.15.33:0/539728803 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f9fbc054ac0 con 0x7f9fcc0cc750 2023-10-06T05:19:15.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:15.197 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb8066f30 msgr2=0x7f9fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb8066f30 0x7f9fb80693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0b0250 tx=0x7f9fc0017000 comp rx=0 tx=0).stop 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 msgr2=0x7f9fcc16d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 0x7f9fcc16d980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00e850 tx=0x7f9fbc008040 comp rx=0 tx=0).stop 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 shutdown_connections 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb8066f30 0x7f9fb80693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fcc1694a0 0x7f9fcc169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fcc0cc750 0x7f9fcc16d980 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 --2- 172.21.15.33:0/539728803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fcc0ccfc0 0x7f9fcc16dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 >> 172.21.15.33:0/539728803 conn(0x7f9fcc06cff0 msgr2=0x7f9fcc1563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 shutdown_connections 2023-10-06T05:19:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:15.198+0000 7f9fd0b9d700 1 -- 172.21.15.33:0/539728803 wait complete. 2023-10-06T05:19:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:19:15.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:15.426 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":5901,"entity":{"name":{"type":"client","num":5901},"addr":{"type":"v1","addr":"172.21.15.167:0","nonce":2450314073}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":154.522611856,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.69976122371820193,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5901 v1:172.21.15.167:0/2450314073","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:19:15.427 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.167:0", "nonce": 2450314073, "type": "v1" }, "name": { "num": 5901, "type": "client" } }, "id": 5901, "inst": "client.5901 v1:172.21.15.167:0/2450314073", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6997612237182019 }, "state": "open", "uptime": 154.522611856 } ] 2023-10-06T05:19:15.427 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:19:15.427 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:19:15.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:19:15.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:19:15.502 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:19:15.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:15.502 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:19:15.539 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:19:15.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:19:15.668 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:19:15.754 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:19:15.754 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:19:15.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:19:15.755 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:19:15.841 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:19:15.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:15.842 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:19:15.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.065+0000 7f984dd9d700 1 Processor -- start 2023-10-06T05:19:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.065+0000 7f984dd9d700 1 -- start start 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.066+0000 7f984dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98480c8050 0x7f984813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.066+0000 7f984dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984806aad0 0x7f984806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.066+0000 7f984dd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480c9cb0 con 0x7f984813faa0 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.066+0000 7f984dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98480ca270 con 0x7f98480c8050 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.066+0000 7f984dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984806b840 con 0x7f984806aad0 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f9847fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98480c8050 0x7f984813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f984cd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f984813faa0 0x7f98480c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48682/0 (socket says 172.21.15.33:48682) 2023-10-06T05:19:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f984cd9b700 1 -- 172.21.15.33:0/4288667644 learned_addr learned my addr 172.21.15.33:0/4288667644 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f984d59c700 1 --2- 172.21.15.33:0/4288667644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984806aad0 0x7f984806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f9847fff700 1 -- 172.21.15.33:0/4288667644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984806aad0 msgr2=0x7f984806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f9847fff700 1 --2- 172.21.15.33:0/4288667644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984806aad0 0x7f984806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f9847fff700 1 -- 172.21.15.33:0/4288667644 >> v1:172.21.15.33:6789/0 conn(0x7f984813faa0 legacy=0x7f98480c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f9847fff700 1 -- 172.21.15.33:0/4288667644 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984806bb20 con 0x7f98480c8050 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.067+0000 7f98477fe700 1 -- 172.21.15.33:0/4288667644 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588560456 0 0) 0x7f98480c9cb0 con 0x7f984813faa0 2023-10-06T05:19:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f9847fff700 1 --2- 172.21.15.33:0/4288667644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98480c8050 0x7f984813fe70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98340098e0 tx=0x7f98340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19442b082fba2587 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f98477fe700 1 -- 172.21.15.33:0/4288667644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400b640 con 0x7f98480c8050 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f98477fe700 1 -- 172.21.15.33:0/4288667644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983400b7a0 con 0x7f98480c8050 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f98477fe700 1 -- 172.21.15.33:0/4288667644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400bb40 con 0x7f98480c8050 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f984dd9d700 1 -- 172.21.15.33:0/4288667644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98480c8050 msgr2=0x7f984813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f984dd9d700 1 --2- 172.21.15.33:0/4288667644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98480c8050 0x7f984813fe70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98340098e0 tx=0x7f98340091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f984dd9d700 1 -- 172.21.15.33:0/4288667644 shutdown_connections 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f984dd9d700 1 --2- 172.21.15.33:0/4288667644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984806aad0 0x7f984806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f984dd9d700 1 --2- 172.21.15.33:0/4288667644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98480c8050 0x7f984813fe70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.068+0000 7f984dd9d700 1 -- 172.21.15.33:0/4288667644 >> 172.21.15.33:0/4288667644 conn(0x7f984806d030 msgr2=0x7f984806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.069+0000 7f984dd9d700 1 -- 172.21.15.33:0/4288667644 shutdown_connections 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.069+0000 7f984dd9d700 1 -- 172.21.15.33:0/4288667644 wait complete. 2023-10-06T05:19:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.070+0000 7f984dd9d700 1 Processor -- start 2023-10-06T05:19:16.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.070+0000 7f984dd9d700 1 -- start start 2023-10-06T05:19:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 0x7f984813e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98480c8050 0x7f98480c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984813faa0 0x7f98480a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984815bd80 con 0x7f984806aad0 2023-10-06T05:19:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984815b1b0 con 0x7f98480c8050 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984815b450 con 0x7f984813faa0 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 0x7f984813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 0x7f984813e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51278/0 (socket says 172.21.15.33:51278) 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.071+0000 7f984cd9b700 1 -- 172.21.15.33:0/3576028633 learned_addr learned my addr 172.21.15.33:0/3576028633 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984d59c700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984813faa0 0x7f98480a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f9847fff700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98480c8050 0x7f98480c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984cd9b700 1 -- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984813faa0 msgr2=0x7f98480a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984cd9b700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984813faa0 0x7f98480a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984cd9b700 1 -- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98480c8050 msgr2=0x7f98480c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984cd9b700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98480c8050 0x7f98480c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984cd9b700 1 -- 172.21.15.33:0/3576028633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9834009420 con 0x7f984806aad0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984cd9b700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 0x7f984813e320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c5b0 tx=0x7f983c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c005b30 con 0x7f984806aad0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c005c90 con 0x7f984806aad0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c007300 con 0x7f984806aad0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984815b730 con 0x7f984806aad0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.072+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984813e860 con 0x7f984806aad0 2023-10-06T05:19:16.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.073+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984804c380 con 0x7f984806aad0 2023-10-06T05:19:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.074+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f983c01b3b0 con 0x7f984806aad0 2023-10-06T05:19:16.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.075+0000 7f9844ff9700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f982c066f30 0x7f982c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.075+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f983c083d00 con 0x7f984806aad0 2023-10-06T05:19:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.075+0000 7f9847fff700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f982c066f30 0x7f982c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.076+0000 7f9847fff700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f982c066f30 0x7f982c0693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f98340093f0 tx=0x7f9834017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.079+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f983c050cd0 con 0x7f984806aad0 2023-10-06T05:19:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.329+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98480c9f00 con 0x7f984806aad0 2023-10-06T05:19:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.329+0000 7f9844ff9700 1 -- 172.21.15.33:0/3576028633 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f983c017090 con 0x7f984806aad0 2023-10-06T05:19:16.331 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f982c066f30 msgr2=0x7f982c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f982c066f30 0x7f982c0693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f98340093f0 tx=0x7f9834017000 comp rx=0 tx=0).stop 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 msgr2=0x7f984813e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 0x7f984813e320 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c5b0 tx=0x7f983c001070 comp rx=0 tx=0).stop 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 shutdown_connections 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f982c066f30 0x7f982c0693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984813faa0 0x7f98480a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984806aad0 0x7f984813e320 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 --2- 172.21.15.33:0/3576028633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98480c8050 0x7f98480c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 >> 172.21.15.33:0/3576028633 conn(0x7f984806d030 msgr2=0x7f984806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 shutdown_connections 2023-10-06T05:19:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.332+0000 7f984dd9d700 1 -- 172.21.15.33:0/3576028633 wait complete. 2023-10-06T05:19:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:19:16.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.535+0000 7f1a9899d700 1 Processor -- start 2023-10-06T05:19:16.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.535+0000 7f1a9899d700 1 -- start start 2023-10-06T05:19:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.536+0000 7f1a9899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a940c8050 0x7f1a9413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.536+0000 7f1a9899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9406aad0 0x7f1a9406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.536+0000 7f1a9899d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a940c9cb0 con 0x7f1a9413faa0 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.536+0000 7f1a9899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a940ca270 con 0x7f1a940c8050 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.536+0000 7f1a9899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9406b840 con 0x7f1a9406aad0 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a940c8050 0x7f1a9413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a93fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9406aad0 0x7f1a9406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1a940c8050 0x7f1a9413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44316/0 (socket says 172.21.15.33:44316) 2023-10-06T05:19:16.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 -- 172.21.15.33:0/1722125281 learned_addr learned my addr 172.21.15.33:0/1722125281 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:16.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 -- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9406aad0 msgr2=0x7f1a9406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 --2- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9406aad0 0x7f1a9406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 -- 172.21.15.33:0/1722125281 >> v1:172.21.15.167:6789/0 conn(0x7f1a9413faa0 legacy=0x7f1a940c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a92ffd700 1 -- 172.21.15.33:0/1722125281 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9406bb20 con 0x7f1a940c8050 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.537+0000 7f1a93fff700 1 --2- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9406aad0 0x7f1a9406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.538+0000 7f1a92ffd700 1 --2- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a9413fe70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800bc50 tx=0x7f1a8800b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62082b9884ad5062 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.538+0000 7f1a927fc700 1 -- 172.21.15.33:0/1722125281 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8800dbf0 con 0x7f1a940c8050 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.538+0000 7f1a927fc700 1 -- 172.21.15.33:0/1722125281 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8800dd50 con 0x7f1a940c8050 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.538+0000 7f1a927fc700 1 -- 172.21.15.33:0/1722125281 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a880196b0 con 0x7f1a940c8050 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.538+0000 7f1a9899d700 1 -- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 msgr2=0x7f1a9413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.538+0000 7f1a9899d700 1 --2- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a9413fe70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800bc50 tx=0x7f1a8800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.539+0000 7f1a9899d700 1 -- 172.21.15.33:0/1722125281 shutdown_connections 2023-10-06T05:19:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.539+0000 7f1a9899d700 1 --2- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9406aad0 0x7f1a9406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.539+0000 7f1a9899d700 1 --2- 172.21.15.33:0/1722125281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a9413fe70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.539+0000 7f1a9899d700 1 -- 172.21.15.33:0/1722125281 >> 172.21.15.33:0/1722125281 conn(0x7f1a9406d030 msgr2=0x7f1a9406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.539+0000 7f1a9899d700 1 -- 172.21.15.33:0/1722125281 shutdown_connections 2023-10-06T05:19:16.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.539+0000 7f1a9899d700 1 -- 172.21.15.33:0/1722125281 wait complete. 2023-10-06T05:19:16.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.540+0000 7f1a9899d700 1 Processor -- start 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.540+0000 7f1a9899d700 1 -- start start 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.541+0000 7f1a9899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 0x7f1a940c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.541+0000 7f1a9899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a94158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.541+0000 7f1a9899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9413faa0 0x7f1a9413cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.541+0000 7f1a9899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94159280 con 0x7f1a9406aad0 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.541+0000 7f1a9899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c3630 con 0x7f1a940c8050 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.541+0000 7f1a9899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c4fa0 con 0x7f1a9413faa0 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 0x7f1a940c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 0x7f1a940c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51294/0 (socket says 172.21.15.33:51294) 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 -- 172.21.15.33:0/3218130071 learned_addr learned my addr 172.21.15.33:0/3218130071 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:16.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a92ffd700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a94158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a93fff700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9413faa0 0x7f1a9413cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 -- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9413faa0 msgr2=0x7f1a9413cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9413faa0 0x7f1a9413cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 -- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 msgr2=0x7f1a94158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a94158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 -- 172.21.15.33:0/3218130071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c009a20 con 0x7f1a9406aad0 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a93fff700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9413faa0 0x7f1a9413cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:16.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.542+0000 7f1a937fe700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 0x7f1a940c48a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400c4e0 tx=0x7f1a8400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.543+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84011590 con 0x7f1a9406aad0 2023-10-06T05:19:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.543+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a84011710 con 0x7f1a9406aad0 2023-10-06T05:19:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.543+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8800b790 con 0x7f1a9406aad0 2023-10-06T05:19:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.543+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84011ad0 con 0x7f1a9406aad0 2023-10-06T05:19:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.543+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a940c32c0 con 0x7f1a9406aad0 2023-10-06T05:19:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.544+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9404c380 con 0x7f1a9406aad0 2023-10-06T05:19:16.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.545+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a8401b3b0 con 0x7f1a9406aad0 2023-10-06T05:19:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.545+0000 7f1a7b7fe700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a80066f30 0x7f1a800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.545+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1a84083c40 con 0x7f1a9406aad0 2023-10-06T05:19:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.546+0000 7f1a92ffd700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a80066f30 0x7f1a800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.547+0000 7f1a92ffd700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a80066f30 0x7f1a800693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800bc50 tx=0x7f1a8801b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.549+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a84050c10 con 0x7f1a9406aad0 2023-10-06T05:19:16.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.753+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a940c9f00 con 0x7f1a9406aad0 2023-10-06T05:19:16.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.754+0000 7f1a7b7fe700 1 -- 172.21.15.33:0/3218130071 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17120 (secure 0 0 0) 0x7f1a84017070 con 0x7f1a9406aad0 2023-10-06T05:19:16.756 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:16.756 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":65,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:18:44.699297+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:29.379192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:30.188919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/4010096267":"2023-10-06T06:18:44.138165+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a80066f30 msgr2=0x7f1a800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a80066f30 0x7f1a800693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800bc50 tx=0x7f1a8801b000 comp rx=0 tx=0).stop 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 msgr2=0x7f1a940c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 0x7f1a940c48a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400c4e0 tx=0x7f1a8400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 shutdown_connections 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a80066f30 0x7f1a800693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9413faa0 0x7f1a9413cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9406aad0 0x7f1a940c48a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 --2- 172.21.15.33:0/3218130071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a940c8050 0x7f1a94158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 >> 172.21.15.33:0/3218130071 conn(0x7f1a9406d030 msgr2=0x7f1a940c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:16.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 shutdown_connections 2023-10-06T05:19:16.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.757+0000 7f1a9899d700 1 -- 172.21.15.33:0/3218130071 wait complete. 2023-10-06T05:19:16.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:29.379192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:16:30.188919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:19:16.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:19:16.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.964+0000 7febd979d700 1 Processor -- start 2023-10-06T05:19:16.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.965+0000 7febd979d700 1 -- start start 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd415acd0 0x7febd40abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd979d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febd406efa0 0x7febd40ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40b0cd0 con 0x7febd4159850 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd979d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd415a840 con 0x7febd406efa0 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd4159f10 con 0x7febd415acd0 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd8f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febd406efa0 0x7febd40ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd8f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febd406efa0 0x7febd40ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51302/0 (socket says 172.21.15.33:51302) 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd8f9c700 1 -- 172.21.15.33:0/1002471301 learned_addr learned my addr 172.21.15.33:0/1002471301 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:16.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.966+0000 7febd37fe700 1 --2- 172.21.15.33:0/1002471301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd415acd0 0x7febd40abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd8f9c700 1 -- 172.21.15.33:0/1002471301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd415acd0 msgr2=0x7febd40abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd8f9c700 1 --2- 172.21.15.33:0/1002471301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd415acd0 0x7febd40abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd8f9c700 1 -- 172.21.15.33:0/1002471301 >> v1:172.21.15.167:6789/0 conn(0x7febd4159850 legacy=0x7febd406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd8f9c700 1 -- 172.21.15.33:0/1002471301 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd415a1f0 con 0x7febd406efa0 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd8f9c700 1 --2- 172.21.15.33:0/1002471301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd406efa0 0x7febd40ac3d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7febcc009f20 tx=0x7febcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85df18b8dc25e3b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd2ffd700 1 -- 172.21.15.33:0/1002471301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00e070 con 0x7febd406efa0 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd2ffd700 1 -- 172.21.15.33:0/1002471301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00b9d0 con 0x7febd406efa0 2023-10-06T05:19:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.967+0000 7febd2ffd700 1 -- 172.21.15.33:0/1002471301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0092e0 con 0x7febd406efa0 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 -- 172.21.15.33:0/1002471301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd406efa0 msgr2=0x7febd40ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 --2- 172.21.15.33:0/1002471301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd406efa0 0x7febd40ac3d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7febcc009f20 tx=0x7febcc009840 comp rx=0 tx=0).stop 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 -- 172.21.15.33:0/1002471301 shutdown_connections 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 --2- 172.21.15.33:0/1002471301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd406efa0 0x7febd40ac3d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 --2- 172.21.15.33:0/1002471301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd415acd0 0x7febd40abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 -- 172.21.15.33:0/1002471301 >> 172.21.15.33:0/1002471301 conn(0x7febd406cd40 msgr2=0x7febd40ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 -- 172.21.15.33:0/1002471301 shutdown_connections 2023-10-06T05:19:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.968+0000 7febd979d700 1 -- 172.21.15.33:0/1002471301 wait complete. 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.970+0000 7febd979d700 1 Processor -- start 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.970+0000 7febd979d700 1 -- start start 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd406efa0 0x7febd4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 0x7febd40c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd415acd0 0x7febd4068530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40caff0 con 0x7febd4159850 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cb290 con 0x7febd415acd0 2023-10-06T05:19:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cb570 con 0x7febd406efa0 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 0x7febd40c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 0x7febd40c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51308/0 (socket says 172.21.15.33:51308) 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 -- 172.21.15.33:0/906559646 learned_addr learned my addr 172.21.15.33:0/906559646 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd8f9c700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd415acd0 0x7febd4068530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd3fff700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd406efa0 0x7febd4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 -- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd406efa0 msgr2=0x7febd4149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd406efa0 0x7febd4149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 -- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd415acd0 msgr2=0x7febd4068530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd415acd0 0x7febd4068530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.971+0000 7febd37fe700 1 -- 172.21.15.33:0/906559646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009ae0 con 0x7febd4159850 2023-10-06T05:19:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.972+0000 7febd37fe700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 0x7febd40c9a80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7febc800e930 tx=0x7febc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.972+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8013680 con 0x7febd4159850 2023-10-06T05:19:16.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.972+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc80137e0 con 0x7febd4159850 2023-10-06T05:19:16.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.972+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40cb850 con 0x7febd4159850 2023-10-06T05:19:16.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.972+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8013b80 con 0x7febd4159850 2023-10-06T05:19:16.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.972+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd40c7880 con 0x7febd4159850 2023-10-06T05:19:16.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.973+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd404c380 con 0x7febd4159850 2023-10-06T05:19:16.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.974+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febc800c3e0 con 0x7febd4159850 2023-10-06T05:19:16.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.975+0000 7febbbfff700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febc0067000 0x7febc00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:16.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.975+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7febc80847b0 con 0x7febd4159850 2023-10-06T05:19:16.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.975+0000 7febd3fff700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febc0067000 0x7febc00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:16.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.976+0000 7febd3fff700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febc0067000 0x7febc00694b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7febc4003a80 tx=0x7febc4007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:16.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:16.979+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febc8051800 con 0x7febd4159850 2023-10-06T05:19:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.183+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febd40ca7c0 con 0x7febd4159850 2023-10-06T05:19:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.183+0000 7febbbfff700 1 -- 172.21.15.33:0/906559646 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7febc8054e20 con 0x7febd4159850 2023-10-06T05:19:17.185 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:19:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febc0067000 msgr2=0x7febc00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febc0067000 0x7febc00694b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7febc4003a80 tx=0x7febc4007ec0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 msgr2=0x7febd40c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 0x7febd40c9a80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7febc800e930 tx=0x7febc80091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 shutdown_connections 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febc0067000 0x7febc00694b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd415acd0 0x7febd4068530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febd406efa0 0x7febd4149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 --2- 172.21.15.33:0/906559646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febd4159850 0x7febd40c9a80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 >> 172.21.15.33:0/906559646 conn(0x7febd406cd40 msgr2=0x7febd40aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 shutdown_connections 2023-10-06T05:19:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.186+0000 7febd979d700 1 -- 172.21.15.33:0/906559646 wait complete. 2023-10-06T05:19:17.205 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:19:17.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.393+0000 7f7f7db9d700 1 Processor -- start 2023-10-06T05:19:17.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.393+0000 7f7f7db9d700 1 -- start start 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7db9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f780af540 0x7f7f780af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7db9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f7815a650 0x7f7f7815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7db9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f780cd890 con 0x7f7f780af540 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7db9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7815d660 con 0x7f7f7815a650 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780ce890 con 0x7f7f780cced0 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7cb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f780af540 0x7f7f780af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7cb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f780af540 0x7f7f780af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51324/0 (socket says 172.21.15.33:51324) 2023-10-06T05:19:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.394+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/87058246 learned_addr learned my addr 172.21.15.33:0/87058246 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f77fff700 1 --2- 172.21.15.33:0/87058246 >> v2:172.21.15.167:3300/0 conn(0x7f7f7815a650 0x7f7f7815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7d39c700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/87058246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 msgr2=0x7f7f780cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7cb9b700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/87058246 >> v2:172.21.15.167:3300/0 conn(0x7f7f7815a650 msgr2=0x7f7f7815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7cb9b700 1 --2- 172.21.15.33:0/87058246 >> v2:172.21.15.167:3300/0 conn(0x7f7f7815a650 0x7f7f7815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/87058246 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7815aac0 con 0x7f7f780af540 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7d39c700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f7cb9b700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780af910 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7f780893f0 tx=0x7f7f6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8e009ab79e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f777fe700 1 -- 172.21.15.33:0/87058246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00b820 con 0x7f7f780af540 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f777fe700 1 -- 172.21.15.33:0/87058246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00b980 con 0x7f7f780af540 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.395+0000 7f7f777fe700 1 -- 172.21.15.33:0/87058246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00bd20 con 0x7f7f780af540 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 -- 172.21.15.33:0/87058246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 msgr2=0x7f7f780af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780af910 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7f780893f0 tx=0x7f7f6c009250 comp rx=0 tx=0).stop 2023-10-06T05:19:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 -- 172.21.15.33:0/87058246 shutdown_connections 2023-10-06T05:19:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/87058246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780af910 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/87058246 >> v2:172.21.15.167:3300/0 conn(0x7f7f7815a650 0x7f7f7815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 -- 172.21.15.33:0/87058246 >> 172.21.15.33:0/87058246 conn(0x7f7f7806d9b0 msgr2=0x7f7f7806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 -- 172.21.15.33:0/87058246 shutdown_connections 2023-10-06T05:19:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.396+0000 7f7f7db9d700 1 -- 172.21.15.33:0/87058246 wait complete. 2023-10-06T05:19:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.398+0000 7f7f7db9d700 1 Processor -- start 2023-10-06T05:19:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.398+0000 7f7f7db9d700 1 -- start start 2023-10-06T05:19:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.398+0000 7f7f7db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f7815a650 0x7f7f7814be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780b5af0 con 0x7f7f780af540 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780c8a30 con 0x7f7f7815a650 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f780b54a0 con 0x7f7f780cced0 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51334/0 (socket says 172.21.15.33:51334) 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/1089534029 learned_addr learned my addr 172.21.15.33:0/1089534029 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f77fff700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7d39c700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f7815a650 0x7f7f7814be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 msgr2=0x7f7f780c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f7815a650 msgr2=0x7f7f7814be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f7815a650 0x7f7f7814be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 -- 172.21.15.33:0/1089534029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0094f0 con 0x7f7f780af540 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7d39c700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f7815a650 0x7f7f7814be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.399+0000 7f7f7cb9b700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780b5f80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0094c0 tx=0x7f7f6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.400+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c024440 con 0x7f7f780af540 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.400+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c0245a0 con 0x7f7f780af540 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.400+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f780b5780 con 0x7f7f780af540 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.400+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c024960 con 0x7f7f780af540 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.400+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7814c350 con 0x7f7f780af540 2023-10-06T05:19:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.401+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f780b5dd0 con 0x7f7f780af540 2023-10-06T05:19:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.402+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f6c01a420 con 0x7f7f780af540 2023-10-06T05:19:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.402+0000 7f7f74ff9700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f5c066f30 0x7f7f5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.403+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7f6c086510 con 0x7f7f780af540 2023-10-06T05:19:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.403+0000 7f7f77fff700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f5c066f30 0x7f7f5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.403+0000 7f7f77fff700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f5c066f30 0x7f7f5c0693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7f7816cd30 tx=0x7f7f64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.407+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f6c053560 con 0x7f7f780af540 2023-10-06T05:19:17.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.607+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f7804c380 con 0x7f7f780af540 2023-10-06T05:19:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.607+0000 7f7f74ff9700 1 -- 172.21.15.33:0/1089534029 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f6c056b80 con 0x7f7f780af540 2023-10-06T05:19:17.609 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:19:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.609+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f5c066f30 msgr2=0x7f7f5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.609+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f5c066f30 0x7f7f5c0693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7f7816cd30 tx=0x7f7f64008040 comp rx=0 tx=0).stop 2023-10-06T05:19:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.609+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 msgr2=0x7f7f780b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.609+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780b5f80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0094c0 tx=0x7f7f6c017000 comp rx=0 tx=0).stop 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 shutdown_connections 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f5c066f30 0x7f7f5c0693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f7815a650 0x7f7f7814be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f780af540 0x7f7f780b5f80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 --2- 172.21.15.33:0/1089534029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f780cced0 0x7f7f780c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 >> 172.21.15.33:0/1089534029 conn(0x7f7f7806d9b0 msgr2=0x7f7f780c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 shutdown_connections 2023-10-06T05:19:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.610+0000 7f7f7db9d700 1 -- 172.21.15.33:0/1089534029 wait complete. 2023-10-06T05:19:17.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:19:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.817+0000 7fc5a359e700 1 Processor -- start 2023-10-06T05:19:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.817+0000 7fc5a359e700 1 -- start start 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a4156370 0x7fc5a40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5a40cc7e0 0x7fc5a40c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40ad430 con 0x7fc5a40b01f0 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40afe20 con 0x7fc5a40cc7e0 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a406bc70 con 0x7fc5a4156370 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5a40cc7e0 0x7fc5a40c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5a40cc7e0 0x7fc5a40c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51336/0 (socket says 172.21.15.33:51336) 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a1d9b700 1 -- 172.21.15.33:0/3077416970 learned_addr learned my addr 172.21.15.33:0/3077416970 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a259c700 1 --2- 172.21.15.33:0/3077416970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a4156370 0x7fc5a40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a1d9b700 1 -- 172.21.15.33:0/3077416970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a4156370 msgr2=0x7fc5a40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:17.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a1d9b700 1 --2- 172.21.15.33:0/3077416970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a4156370 0x7fc5a40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.819+0000 7fc5a1d9b700 1 -- 172.21.15.33:0/3077416970 >> v1:172.21.15.167:6789/0 conn(0x7fc5a40b01f0 legacy=0x7fc5a40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.820+0000 7fc5a1d9b700 1 -- 172.21.15.33:0/3077416970 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40c92f0 con 0x7fc5a40cc7e0 2023-10-06T05:19:17.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.820+0000 7fc5a1d9b700 1 --2- 172.21.15.33:0/3077416970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a40c8db0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc5980099b0 tx=0x7fc598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3d1a27ec0cbb51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:17.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.820+0000 7fc5a159a700 1 -- 172.21.15.33:0/3077416970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800b860 con 0x7fc5a40cc7e0 2023-10-06T05:19:17.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.820+0000 7fc5a159a700 1 -- 172.21.15.33:0/3077416970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59800b9c0 con 0x7fc5a40cc7e0 2023-10-06T05:19:17.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.820+0000 7fc5a159a700 1 -- 172.21.15.33:0/3077416970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800bd60 con 0x7fc5a40cc7e0 2023-10-06T05:19:17.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 -- 172.21.15.33:0/3077416970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40cc7e0 msgr2=0x7fc5a40c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3077416970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a40c8db0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc5980099b0 tx=0x7fc598009250 comp rx=0 tx=0).stop 2023-10-06T05:19:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 -- 172.21.15.33:0/3077416970 shutdown_connections 2023-10-06T05:19:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3077416970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a4156370 0x7fc5a40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3077416970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a40c8db0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 -- 172.21.15.33:0/3077416970 >> 172.21.15.33:0/3077416970 conn(0x7fc5a406d030 msgr2=0x7fc5a40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 -- 172.21.15.33:0/3077416970 shutdown_connections 2023-10-06T05:19:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.821+0000 7fc5a359e700 1 -- 172.21.15.33:0/3077416970 wait complete. 2023-10-06T05:19:17.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.822+0000 7fc5a359e700 1 Processor -- start 2023-10-06T05:19:17.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.822+0000 7fc5a359e700 1 -- start start 2023-10-06T05:19:17.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.823+0000 7fc5a359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a40b01f0 0x7fc5a4149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.823+0000 7fc5a359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a416a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.823+0000 7fc5a359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 0x7fc5a414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.823+0000 7fc5a359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4149c90 con 0x7fc5a4156370 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.823+0000 7fc5a359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a414a590 con 0x7fc5a40cc7e0 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.823+0000 7fc5a359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40ab590 con 0x7fc5a40b01f0 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 0x7fc5a414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 0x7fc5a414aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51352/0 (socket says 172.21.15.33:51352) 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 -- 172.21.15.33:0/3937742753 learned_addr learned my addr 172.21.15.33:0/3937742753 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a1d9b700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a416a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a259c700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a40b01f0 0x7fc5a4149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 -- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a40b01f0 msgr2=0x7fc5a4149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a40b01f0 0x7fc5a4149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 -- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a40cc7e0 msgr2=0x7fc5a416a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a416a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 -- 172.21.15.33:0/3937742753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5980094f0 con 0x7fc5a4156370 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.824+0000 7fc5a2d9d700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 0x7fc5a414aa20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00c910 tx=0x7fc59c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.825+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c012630 con 0x7fc5a4156370 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.825+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c012790 con 0x7fc5a4156370 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.825+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c012b50 con 0x7fc5a4156370 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.825+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a416aa70 con 0x7fc5a4156370 2023-10-06T05:19:17.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.825+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4149f40 con 0x7fc5a4156370 2023-10-06T05:19:17.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.826+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a414a720 con 0x7fc5a4156370 2023-10-06T05:19:17.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.827+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc59c01a3a0 con 0x7fc5a4156370 2023-10-06T05:19:17.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.828+0000 7fc5927fc700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc584066f30 0x7fc5840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:17.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.828+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc59c0849f0 con 0x7fc5a4156370 2023-10-06T05:19:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.828+0000 7fc5a259c700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc584066f30 0x7fc5840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.829+0000 7fc5a259c700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc584066f30 0x7fc5840693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc594006fd0 tx=0x7fc594008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:17.832+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc59c0519c0 con 0x7fc5a4156370 2023-10-06T05:19:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.034+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5a404c380 con 0x7fc5a4156370 2023-10-06T05:19:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.035+0000 7fc5927fc700 1 -- 172.21.15.33:0/3937742753 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc59c054fe0 con 0x7fc5a4156370 2023-10-06T05:19:18.037 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc584066f30 msgr2=0x7fc5840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc584066f30 0x7fc5840693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc594006fd0 tx=0x7fc594008040 comp rx=0 tx=0).stop 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 msgr2=0x7fc5a414aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 0x7fc5a414aa20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00c910 tx=0x7fc59c001710 comp rx=0 tx=0).stop 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 shutdown_connections 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc584066f30 0x7fc5840693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a4156370 0x7fc5a414aa20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5a40b01f0 0x7fc5a4149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 --2- 172.21.15.33:0/3937742753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a40cc7e0 0x7fc5a416a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 >> 172.21.15.33:0/3937742753 conn(0x7fc5a406d030 msgr2=0x7fc5a406a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 shutdown_connections 2023-10-06T05:19:18.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.037+0000 7fc5a359e700 1 -- 172.21.15.33:0/3937742753 wait complete. 2023-10-06T05:19:18.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:19:18.164 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:18.169 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.246+0000 7fde291ff700 1 Processor -- start 2023-10-06T05:19:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.246+0000 7fde291ff700 1 -- start start 2023-10-06T05:19:18.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.247+0000 7fde291ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde2406f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.247+0000 7fde291ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240d1120 con 0x7fde240cabf0 2023-10-06T05:19:18.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.247+0000 7fde291ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240d1400 con 0x7fde2406d140 2023-10-06T05:19:18.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.247+0000 7fde291ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2406e920 con 0x7fde24159b10 2023-10-06T05:19:18.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.247+0000 7fde22d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fde240cabf0 0x7fde240c7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48688/0 (socket says 172.21.15.33:48688) 2023-10-06T05:19:18.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.247+0000 7fde22d9d700 1 -- 172.21.15.33:0/3138602298 learned_addr learned my addr 172.21.15.33:0/3138602298 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde2259c700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde2406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022555153 0 0) 0x7fde240d1120 con 0x7fde240cabf0 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde0c003570 con 0x7fde240cabf0 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278525534 0 0) 0x7fde240d1400 con 0x7fde2406d140 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde240d1120 con 0x7fde2406d140 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3103072844 0 0) 0x7fde0c003570 con 0x7fde240cabf0 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 msgr2=0x7fde2406f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde2406f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 >> v1:172.21.15.167:6789/0 conn(0x7fde2406d140 legacy=0x7fde2406f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2406ec00 con 0x7fde240cabf0 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fde140015f0 con 0x7fde240cabf0 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 >> v1:172.21.15.33:6789/0 conn(0x7fde240cabf0 legacy=0x7fde240c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde0c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c0041d0 0x7fde0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.248+0000 7fde21d9b700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c004d50 0x7fde0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2406e920 con 0x7fde0c0041d0 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c00bbf0 con 0x7fde0c004d50 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c00bdb0 con 0x7fde24159b10 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2259c700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c0041d0 0x7fde0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde22d9d700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde0c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2359e700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c004d50 0x7fde0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2259c700 1 -- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 msgr2=0x7fde0c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2259c700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde0c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2259c700 1 -- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c004d50 msgr2=0x7fde0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2259c700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c004d50 0x7fde0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde2259c700 1 -- 172.21.15.33:0/3138602298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c003570 con 0x7fde0c0041d0 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.249+0000 7fde22d9d700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24159b10 0x7fde0c003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde2259c700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c0041d0 0x7fde0c004810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fde18009c00 tx=0x7fde180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7be4a09fce85526 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800e710 con 0x7fde0c0041d0 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800e870 con 0x7fde0c0041d0 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde21d9b700 1 -- 172.21.15.33:0/3138602298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800ec30 con 0x7fde0c0041d0 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde291ff700 1 -- 172.21.15.33:0/3138602298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c0041d0 msgr2=0x7fde0c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde291ff700 1 --2- 172.21.15.33:0/3138602298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c0041d0 0x7fde0c004810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fde18009c00 tx=0x7fde180091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde2359e700 1 -- 172.21.15.33:0/3138602298 reap_dead start 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde291ff700 1 -- 172.21.15.33:0/3138602298 shutdown_connections 2023-10-06T05:19:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde291ff700 1 -- 172.21.15.33:0/3138602298 >> 172.21.15.33:0/3138602298 conn(0x7fde2406cd40 msgr2=0x7fde240d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.250+0000 7fde291ff700 1 -- 172.21.15.33:0/3138602298 shutdown_connections 2023-10-06T05:19:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.251+0000 7fde291ff700 1 -- 172.21.15.33:0/3138602298 wait complete. 2023-10-06T05:19:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.252+0000 7fde291ff700 1 Processor -- start 2023-10-06T05:19:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.252+0000 7fde291ff700 1 -- start start 2023-10-06T05:19:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde291ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c0041d0 0x7fde240c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde291ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 0x7fde24157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde291ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24157b10 0x7fde2415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde291ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde240b4eb0 con 0x7fde241580e0 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde291ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde240c50a0 con 0x7fde0c0041d0 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde291ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2415f3a0 con 0x7fde24157b10 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde2259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 0x7fde24157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde2259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 0x7fde24157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51372/0 (socket says 172.21.15.33:51372) 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde2259c700 1 -- 172.21.15.33:0/250547116 learned_addr learned my addr 172.21.15.33:0/250547116 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.253+0000 7fde22d9d700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c0041d0 0x7fde240c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2359e700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24157b10 0x7fde2415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2259c700 1 -- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24157b10 msgr2=0x7fde2415c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2259c700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24157b10 0x7fde2415c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2259c700 1 -- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c0041d0 msgr2=0x7fde240c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2259c700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c0041d0 0x7fde240c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2259c700 1 -- 172.21.15.33:0/250547116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240b4580 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2359e700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24157b10 0x7fde2415c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde2259c700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 0x7fde24157350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0057f0 tx=0x7fde1801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18022070 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800ebf0 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1801c6a0 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde240b4860 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.254+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde24157890 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.255+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2404c380 con 0x7fde241580e0 2023-10-06T05:19:18.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.256+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde1801ca50 con 0x7fde241580e0 2023-10-06T05:19:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.258+0000 7fde0affd700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde10066f60 0x7fde10069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.258+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fde18085ef0 con 0x7fde241580e0 2023-10-06T05:19:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.258+0000 7fde22d9d700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde10066f60 0x7fde10069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.259+0000 7fde22d9d700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde10066f60 0x7fde10069410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde14007fd0 tx=0x7fde14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.262+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde18052f00 con 0x7fde241580e0 2023-10-06T05:19:18.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.464+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde240ca450 con 0x7fde241580e0 2023-10-06T05:19:18.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.465+0000 7fde0affd700 1 -- 172.21.15.33:0/250547116 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde1802b0a0 con 0x7fde241580e0 2023-10-06T05:19:18.467 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde10066f60 msgr2=0x7fde10069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde10066f60 0x7fde10069410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde14007fd0 tx=0x7fde14009040 comp rx=0 tx=0).stop 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 msgr2=0x7fde24157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 0x7fde24157350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0057f0 tx=0x7fde1801a000 comp rx=0 tx=0).stop 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 shutdown_connections 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde10066f60 0x7fde10069410 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde24157b10 0x7fde2415c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0c0041d0 0x7fde240c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 --2- 172.21.15.33:0/250547116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde241580e0 0x7fde24157350 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 >> 172.21.15.33:0/250547116 conn(0x7fde2406cd40 msgr2=0x7fde240ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 shutdown_connections 2023-10-06T05:19:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.467+0000 7fde291ff700 1 -- 172.21.15.33:0/250547116 wait complete. 2023-10-06T05:19:18.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.671+0000 7f155239d700 1 Processor -- start 2023-10-06T05:19:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.671+0000 7f155239d700 1 -- start start 2023-10-06T05:19:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155239d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f154c155140 0x7f154c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155239d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f154c0699c0 0x7f154c069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155239d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0c8e40 con 0x7f154c0adf50 2023-10-06T05:19:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155239d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154c06a340 con 0x7f154c0699c0 2023-10-06T05:19:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155239d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154c0c99e0 con 0x7f154c155140 2023-10-06T05:19:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f154c0699c0 0x7f154c069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f154c0699c0 0x7f154c069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51388/0 (socket says 172.21.15.33:51388) 2023-10-06T05:19:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 -- 172.21.15.33:0/1934146381 learned_addr learned my addr 172.21.15.33:0/1934146381 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 -- 172.21.15.33:0/1934146381 >> v2:172.21.15.167:3300/0 conn(0x7f154c155140 msgr2=0x7f154c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155139b700 1 --2- 172.21.15.33:0/1934146381 >> v2:172.21.15.167:3300/0 conn(0x7f154c155140 0x7f154c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 --2- 172.21.15.33:0/1934146381 >> v2:172.21.15.167:3300/0 conn(0x7f154c155140 0x7f154c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 -- 172.21.15.33:0/1934146381 >> v1:172.21.15.167:6789/0 conn(0x7f154c0adf50 legacy=0x7f154c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f1551b9c700 1 -- 172.21.15.33:0/1934146381 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0ae3f0 con 0x7f154c0699c0 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.673+0000 7f155139b700 1 --2- 172.21.15.33:0/1934146381 >> v2:172.21.15.167:3300/0 conn(0x7f154c155140 0x7f154c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.674+0000 7f1551b9c700 1 --2- 172.21.15.33:0/1934146381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c0699c0 0x7f154c069e00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1544009f20 tx=0x7f1544009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d621432aeda1c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.674+0000 7f1543fff700 1 -- 172.21.15.33:0/1934146381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400e070 con 0x7f154c0699c0 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.674+0000 7f1543fff700 1 -- 172.21.15.33:0/1934146381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154400ba00 con 0x7f154c0699c0 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.674+0000 7f1543fff700 1 -- 172.21.15.33:0/1934146381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440092e0 con 0x7f154c0699c0 2023-10-06T05:19:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.674+0000 7f155239d700 1 -- 172.21.15.33:0/1934146381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c0699c0 msgr2=0x7f154c069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.674+0000 7f155239d700 1 --2- 172.21.15.33:0/1934146381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c0699c0 0x7f154c069e00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1544009f20 tx=0x7f1544009840 comp rx=0 tx=0).stop 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.675+0000 7f155239d700 1 -- 172.21.15.33:0/1934146381 shutdown_connections 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.675+0000 7f155239d700 1 --2- 172.21.15.33:0/1934146381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c0699c0 0x7f154c069e00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.675+0000 7f155239d700 1 --2- 172.21.15.33:0/1934146381 >> v2:172.21.15.167:3300/0 conn(0x7f154c155140 0x7f154c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.675+0000 7f155239d700 1 -- 172.21.15.33:0/1934146381 >> 172.21.15.33:0/1934146381 conn(0x7f154c06d030 msgr2=0x7f154c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.675+0000 7f155239d700 1 -- 172.21.15.33:0/1934146381 shutdown_connections 2023-10-06T05:19:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.675+0000 7f155239d700 1 -- 172.21.15.33:0/1934146381 wait complete. 2023-10-06T05:19:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.676+0000 7f155239d700 1 Processor -- start 2023-10-06T05:19:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.676+0000 7f155239d700 1 -- start start 2023-10-06T05:19:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f154c0699c0 0x7f154c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f154c0adf50 0x7f154c0c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 0x7f154c0c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c154550 con 0x7f154c155140 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0c7080 con 0x7f154c0adf50 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0c83c0 con 0x7f154c0699c0 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f1551b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 0x7f154c0c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f1551b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 0x7f154c0c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51404/0 (socket says 172.21.15.33:51404) 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f1551b9c700 1 -- 172.21.15.33:0/3885631114 learned_addr learned my addr 172.21.15.33:0/3885631114 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f155139b700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f154c0699c0 0x7f154c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.677+0000 7f1550b9a700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f154c0adf50 0x7f154c0c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f1551b9c700 1 -- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f154c0699c0 msgr2=0x7f154c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f1551b9c700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f154c0699c0 0x7f154c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f1551b9c700 1 -- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f154c0adf50 msgr2=0x7f154c0c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f1551b9c700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f154c0adf50 0x7f154c0c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f1551b9c700 1 -- 172.21.15.33:0/3885631114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544009ae0 con 0x7f154c155140 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f1551b9c700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 0x7f154c0c4d70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f154c063610 tx=0x7f1544017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440032f0 con 0x7f154c155140 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1544012d00 con 0x7f154c155140 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c0c7320 con 0x7f154c155140 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.678+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154401d650 con 0x7f154c155140 2023-10-06T05:19:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.679+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c0c78e0 con 0x7f154c155140 2023-10-06T05:19:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.680+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c04c380 con 0x7f154c155140 2023-10-06T05:19:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.680+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f154401da80 con 0x7f154c155140 2023-10-06T05:19:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.681+0000 7f15417fa700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1530066f30 0x7f15300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.681+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1544085f60 con 0x7f154c155140 2023-10-06T05:19:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.682+0000 7f155139b700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1530066f30 0x7f15300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.682+0000 7f155139b700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1530066f30 0x7f15300693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1548006fd0 tx=0x7f1548008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:18.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.686+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1544052f30 con 0x7f154c155140 2023-10-06T05:19:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.939+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f154c154930 con 0x7f154c155140 2023-10-06T05:19:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.940+0000 7f15417fa700 1 -- 172.21.15.33:0/3885631114 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f1544056550 con 0x7f154c155140 2023-10-06T05:19:18.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:18.942 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:18.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.942+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1530066f30 msgr2=0x7f15300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.942+0000 7f155239d700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1530066f30 0x7f15300693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1548006fd0 tx=0x7f1548008040 comp rx=0 tx=0).stop 2023-10-06T05:19:18.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.942+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 msgr2=0x7f154c0c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:18.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.942+0000 7f155239d700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 0x7f154c0c4d70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f154c063610 tx=0x7f1544017000 comp rx=0 tx=0).stop 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 shutdown_connections 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1530066f30 0x7f15300693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f154c155140 0x7f154c0c4d70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f154c0699c0 0x7f154c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 --2- 172.21.15.33:0/3885631114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f154c0adf50 0x7f154c0c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 >> 172.21.15.33:0/3885631114 conn(0x7f154c06d030 msgr2=0x7f154c0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 shutdown_connections 2023-10-06T05:19:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:18.943+0000 7f155239d700 1 -- 172.21.15.33:0/3885631114 wait complete. 2023-10-06T05:19:18.946 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:19:18.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.147+0000 7f25d1fff700 1 Processor -- start 2023-10-06T05:19:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.147+0000 7f25d1fff700 1 -- start start 2023-10-06T05:19:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.148+0000 7f25d1fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25cc13fb70 0x7f25cc06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25d1fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25cc0af000 0x7f25cc0af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25d1fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25cc0af960 con 0x7f25cc06ba40 2023-10-06T05:19:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25d1fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc0ca8d0 con 0x7f25cc13fb70 2023-10-06T05:19:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25d1fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc0cabb0 con 0x7f25cc0af000 2023-10-06T05:19:19.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25cc13fb70 0x7f25cc06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25cc13fb70 0x7f25cc06b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51410/0 (socket says 172.21.15.33:51410) 2023-10-06T05:19:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 -- 172.21.15.33:0/2273286515 learned_addr learned my addr 172.21.15.33:0/2273286515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cbfff700 1 --2- 172.21.15.33:0/2273286515 >> v2:172.21.15.167:3300/0 conn(0x7f25cc0af000 0x7f25cc0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 -- 172.21.15.33:0/2273286515 >> v2:172.21.15.167:3300/0 conn(0x7f25cc0af000 msgr2=0x7f25cc0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 --2- 172.21.15.33:0/2273286515 >> v2:172.21.15.167:3300/0 conn(0x7f25cc0af000 0x7f25cc0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 -- 172.21.15.33:0/2273286515 >> v1:172.21.15.167:6789/0 conn(0x7f25cc06ba40 legacy=0x7f25cc0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cb7fe700 1 -- 172.21.15.33:0/2273286515 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc06ad00 con 0x7f25cc13fb70 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.149+0000 7f25cbfff700 1 --2- 172.21.15.33:0/2273286515 >> v2:172.21.15.167:3300/0 conn(0x7f25cc0af000 0x7f25cc0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.150+0000 7f25cb7fe700 1 --2- 172.21.15.33:0/2273286515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc06b500 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06a250 tx=0x7f25bc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d39c663e33ae150 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.150+0000 7f25ca7fc700 1 -- 172.21.15.33:0/2273286515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00dc10 con 0x7f25cc13fb70 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.150+0000 7f25ca7fc700 1 -- 172.21.15.33:0/2273286515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc00dd70 con 0x7f25cc13fb70 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.150+0000 7f25ca7fc700 1 -- 172.21.15.33:0/2273286515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0106c0 con 0x7f25cc13fb70 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.150+0000 7f25d1fff700 1 -- 172.21.15.33:0/2273286515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 msgr2=0x7f25cc06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.150+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2273286515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc06b500 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06a250 tx=0x7f25bc00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.151+0000 7f25d1fff700 1 -- 172.21.15.33:0/2273286515 shutdown_connections 2023-10-06T05:19:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.151+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2273286515 >> v2:172.21.15.167:3300/0 conn(0x7f25cc0af000 0x7f25cc0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.151+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2273286515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc06b500 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.151+0000 7f25d1fff700 1 -- 172.21.15.33:0/2273286515 >> 172.21.15.33:0/2273286515 conn(0x7f25cc0675f0 msgr2=0x7f25cc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.151+0000 7f25d1fff700 1 -- 172.21.15.33:0/2273286515 shutdown_connections 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.151+0000 7f25d1fff700 1 -- 172.21.15.33:0/2273286515 wait complete. 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.152+0000 7f25d1fff700 1 Processor -- start 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.152+0000 7f25d1fff700 1 -- start start 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25d1fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25cc06ba40 0x7f25cc0abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25d1fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25cc0af000 0x7f25cc131a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25d1fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc0aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25d1fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0ad0e0 con 0x7f25cc13fb70 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25d1fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0ac350 con 0x7f25cc06ba40 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25d1fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0ac630 con 0x7f25cc0af000 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc0aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.153+0000 7f25cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc0aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51422/0 (socket says 172.21.15.33:51422) 2023-10-06T05:19:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 -- 172.21.15.33:0/2784299943 learned_addr learned my addr 172.21.15.33:0/2784299943 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cb7fe700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25cc06ba40 0x7f25cc0abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 -- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25cc0af000 msgr2=0x7f25cc131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25caffd700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25cc0af000 0x7f25cc131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25cc0af000 0x7f25cc131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 -- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25cc06ba40 msgr2=0x7f25cc0abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25cc06ba40 0x7f25cc0abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 -- 172.21.15.33:0/2784299943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc00b790 con 0x7f25cc13fb70 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cb7fe700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25cc06ba40 0x7f25cc0abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25caffd700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25cc0af000 0x7f25cc131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25cbfff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc0aac80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f25c400a380 tx=0x7f25c4007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400ae10 con 0x7f25cc13fb70 2023-10-06T05:19:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400d650 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.154+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400da10 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.155+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc0ac910 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.155+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc0ac040 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.156+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc063ff0 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.157+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25c401e3b0 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.157+0000 7f25b37fe700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25b8066f30 0x7f25b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.157+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f25c40851e0 con 0x7f25cc13fb70 2023-10-06T05:19:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.158+0000 7f25cb7fe700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25b8066f30 0x7f25b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.158+0000 7f25cb7fe700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25b8066f30 0x7f25b80693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06a250 tx=0x7f25bc01a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:19.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.162+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25c40521b0 con 0x7f25cc13fb70 2023-10-06T05:19:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.366+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25cc0aa760 con 0x7f25cc13fb70 2023-10-06T05:19:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.367+0000 7f25b37fe700 1 -- 172.21.15.33:0/2784299943 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17120 (secure 0 0 0) 0x7f25c40557d0 con 0x7f25cc13fb70 2023-10-06T05:19:19.370 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:19.370 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":65,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:18:44.699297+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:29.379192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:30.188919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/4010096267":"2023-10-06T06:18:44.138165+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25b8066f30 msgr2=0x7f25b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25b8066f30 0x7f25b80693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06a250 tx=0x7f25bc01a000 comp rx=0 tx=0).stop 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 msgr2=0x7f25cc0aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc0aac80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f25c400a380 tx=0x7f25c4007900 comp rx=0 tx=0).stop 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 shutdown_connections 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25b8066f30 0x7f25b80693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25cc13fb70 0x7f25cc0aac80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25cc06ba40 0x7f25cc0abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 --2- 172.21.15.33:0/2784299943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25cc0af000 0x7f25cc131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 >> 172.21.15.33:0/2784299943 conn(0x7f25cc0675f0 msgr2=0x7f25cc0cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 shutdown_connections 2023-10-06T05:19:19.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.369+0000 7f25d1fff700 1 -- 172.21.15.33:0/2784299943 wait complete. 2023-10-06T05:19:19.388 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:19:19.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.573+0000 7f269739d700 1 Processor -- start 2023-10-06T05:19:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.573+0000 7f269739d700 1 -- start start 2023-10-06T05:19:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.574+0000 7f269739d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f26900c8050 0x7f269013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.574+0000 7f269739d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900c9cb0 con 0x7f269013faa0 2023-10-06T05:19:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.574+0000 7f269739d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900ca270 con 0x7f269006aad0 2023-10-06T05:19:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.574+0000 7f269739d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269006b640 con 0x7f26900c8050 2023-10-06T05:19:19.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f269639b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f269013faa0 0x7f26900c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48696/0 (socket says 172.21.15.33:48696) 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f269639b700 1 -- 172.21.15.33:0/1692004321 learned_addr learned my addr 172.21.15.33:0/1692004321 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695b9a700 1 --2- 172.21.15.33:0/1692004321 >> v2:172.21.15.167:3300/0 conn(0x7f26900c8050 0x7f269013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836090456 0 0) 0x7f26900c9cb0 con 0x7f269013faa0 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f267c003570 con 0x7f269013faa0 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543093037 0 0) 0x7f267c003570 con 0x7f269013faa0 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 >> v2:172.21.15.167:3300/0 conn(0x7f26900c8050 msgr2=0x7f269013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 --2- 172.21.15.33:0/1692004321 >> v2:172.21.15.167:3300/0 conn(0x7f26900c8050 0x7f269013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 >> v1:172.21.15.167:6789/0 conn(0x7f269006aad0 legacy=0x7f269006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269006b840 con 0x7f269013faa0 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f268c001470 con 0x7f269013faa0 2023-10-06T05:19:19.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 >> v1:172.21.15.33:6789/0 conn(0x7f269013faa0 legacy=0x7f26900c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.575+0000 7f2695399700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267c003a60 0x7f267c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2695399700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2695399700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f267c0051c0 0x7f267c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269006b640 con 0x7f267c003a60 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c00c050 con 0x7f267c0051c0 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c00c210 con 0x7f267c004810 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267c003a60 0x7f267c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2695b9a700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2696b9c700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f267c0051c0 0x7f267c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 -- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c004810 msgr2=0x7f267c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 -- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f267c0051c0 msgr2=0x7f267c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f267c0051c0 0x7f267c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 -- 172.21.15.33:0/1692004321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c003570 con 0x7f267c003a60 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f2696b9c700 1 -- 172.21.15.33:0/1692004321 reap_dead start 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.576+0000 7f269639b700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267c003a60 0x7f267c003e30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c2d0 tx=0x7f26900c84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351380645257444b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00cc00 con 0x7f267c003a60 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c00cd60 con 0x7f267c003a60 2023-10-06T05:19:19.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f2695399700 1 -- 172.21.15.33:0/1692004321 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c005b70 con 0x7f267c003a60 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 -- 172.21.15.33:0/1692004321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267c003a60 msgr2=0x7f267c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267c003a60 0x7f267c003e30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c2d0 tx=0x7f26900c84c0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 -- 172.21.15.33:0/1692004321 shutdown_connections 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 --2- 172.21.15.33:0/1692004321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f267c003a60 0x7f267c003e30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 -- 172.21.15.33:0/1692004321 >> 172.21.15.33:0/1692004321 conn(0x7f269006d030 msgr2=0x7f269006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 -- 172.21.15.33:0/1692004321 shutdown_connections 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.577+0000 7f269739d700 1 -- 172.21.15.33:0/1692004321 wait complete. 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.579+0000 7f269739d700 1 Processor -- start 2023-10-06T05:19:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.579+0000 7f269739d700 1 -- start start 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c003a60 0x7f2690155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269013e090 0x7f2690156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 0x7f26900b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26900b1ae0 con 0x7f26900b1100 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269013bf90 con 0x7f269013e090 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269013e590 con 0x7f267c003a60 2023-10-06T05:19:19.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 0x7f26900b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 0x7f26900b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51440/0 (socket says 172.21.15.33:51440) 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 -- 172.21.15.33:0/3893248024 learned_addr learned my addr 172.21.15.33:0/3893248024 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2695b9a700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269013e090 0x7f2690156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f269639b700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c003a60 0x7f2690155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 -- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c003a60 msgr2=0x7f2690155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c003a60 0x7f2690155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 -- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269013e090 msgr2=0x7f2690156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269013e090 0x7f2690156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.580+0000 7f2696b9c700 1 -- 172.21.15.33:0/3893248024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269013e870 con 0x7f26900b1100 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f2695b9a700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269013e090 0x7f2690156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:19.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f2696b9c700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 0x7f26900b15a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f269015a070 tx=0x7f2688007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880032e0 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688014040 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688001710 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269013eb50 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.581+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2690157140 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.582+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269013c100 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.583+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2688001a80 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.584+0000 7f26867fc700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.584+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f26880879b0 con 0x7f26900b1100 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.585+0000 7f269639b700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:19.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.585+0000 7f269639b700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f268c001ed0 tx=0x7f268c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:19.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.588+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f269013c100 con 0x7f26900b1100 2023-10-06T05:19:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.842+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f269013c100 con 0x7f26900b1100 2023-10-06T05:19:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.843+0000 7f26867fc700 1 -- 172.21.15.33:0/3893248024 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f269013c100 con 0x7f26900b1100 2023-10-06T05:19:19.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:19.845 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:19.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.845+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 msgr2=0x7f26740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.845+0000 7f269739d700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f268c001ed0 tx=0x7f268c015000 comp rx=0 tx=0).stop 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.845+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 msgr2=0x7f26900b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.845+0000 7f269739d700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 0x7f26900b15a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f269015a070 tx=0x7f2688007ed0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 shutdown_connections 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2674066f30 0x7f26740693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26900b1100 0x7f26900b15a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f267c003a60 0x7f2690155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 --2- 172.21.15.33:0/3893248024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f269013e090 0x7f2690156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 >> 172.21.15.33:0/3893248024 conn(0x7f269006d030 msgr2=0x7f26900c51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 shutdown_connections 2023-10-06T05:19:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:19.846+0000 7f269739d700 1 -- 172.21.15.33:0/3893248024 wait complete. 2023-10-06T05:19:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:19:19.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:20.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.052+0000 7fdc8759e700 1 Processor -- start 2023-10-06T05:19:20.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.052+0000 7fdc8759e700 1 -- start start 2023-10-06T05:19:20.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.053+0000 7fdc8759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc88155af0 0x7fdc8806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.053+0000 7fdc8759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8806a6a0 0x7fdc880c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.053+0000 7fdc8759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8806ab60 con 0x7fdc880c8b50 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.053+0000 7fdc8759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8806b7e0 con 0x7fdc88155af0 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.053+0000 7fdc8759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8806ae40 con 0x7fdc8806a6a0 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc88155af0 0x7fdc8806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc85d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8806a6a0 0x7fdc880c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc88155af0 0x7fdc8806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:48618/0 (socket says 172.21.15.33:48618) 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 -- 172.21.15.33:0/2717476995 learned_addr learned my addr 172.21.15.33:0/2717476995 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 -- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8806a6a0 msgr2=0x7fdc880c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 --2- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8806a6a0 0x7fdc880c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 -- 172.21.15.33:0/2717476995 >> v1:172.21.15.167:6789/0 conn(0x7fdc880c8b50 legacy=0x7fdc881598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 -- 172.21.15.33:0/2717476995 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8806b120 con 0x7fdc88155af0 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.054+0000 7fdc8659c700 1 --2- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc8806a160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc780099b0 tx=0x7fdc78009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ee85caa36996d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8559a700 1 -- 172.21.15.33:0/2717476995 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78009ea0 con 0x7fdc88155af0 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8559a700 1 -- 172.21.15.33:0/2717476995 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7800b860 con 0x7fdc88155af0 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8559a700 1 -- 172.21.15.33:0/2717476995 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800bc00 con 0x7fdc88155af0 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 -- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 msgr2=0x7fdc8806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 --2- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc8806a160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc780099b0 tx=0x7fdc78009250 comp rx=0 tx=0).stop 2023-10-06T05:19:20.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 -- 172.21.15.33:0/2717476995 shutdown_connections 2023-10-06T05:19:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 --2- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc8806a160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 --2- 172.21.15.33:0/2717476995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc8806a6a0 0x7fdc880c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 -- 172.21.15.33:0/2717476995 >> 172.21.15.33:0/2717476995 conn(0x7fdc8806d030 msgr2=0x7fdc8806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 -- 172.21.15.33:0/2717476995 shutdown_connections 2023-10-06T05:19:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.055+0000 7fdc8759e700 1 -- 172.21.15.33:0/2717476995 wait complete. 2023-10-06T05:19:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.057+0000 7fdc8759e700 1 Processor -- start 2023-10-06T05:19:20.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.057+0000 7fdc8759e700 1 -- start start 2023-10-06T05:19:20.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 0x7fdc8816de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc880c8b50 0x7fdc8816a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc880aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88169e10 con 0x7fdc8806a6a0 2023-10-06T05:19:20.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8816a3c0 con 0x7fdc88155af0 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8816e3b0 con 0x7fdc880c8b50 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 0x7fdc8816de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 0x7fdc8816de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36186/0 (socket says 172.21.15.33:36186) 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc8659c700 1 -- 172.21.15.33:0/65467898 learned_addr learned my addr 172.21.15.33:0/65467898 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc85d9b700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc880c8b50 0x7fdc8816a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.058+0000 7fdc86d9d700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc880aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8659c700 1 -- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc880c8b50 msgr2=0x7fdc8816a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8659c700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc880c8b50 0x7fdc8816a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8659c700 1 -- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 msgr2=0x7fdc880aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8659c700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc880aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8659c700 1 -- 172.21.15.33:0/65467898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc780094f0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8659c700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 0x7fdc8816de70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdc880635d0 tx=0x7fdc78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800bbe0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc78011420 con 0x7fdc8806a6a0 2023-10-06T05:19:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc88157cc0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780117c0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.059+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8816a050 con 0x7fdc8806a6a0 2023-10-06T05:19:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.060+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc881581e0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.061+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc78011bf0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.062+0000 7fdc767fc700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdc68066f30 0x7fdc680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.062+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fdc78085cc0 con 0x7fdc8806a6a0 2023-10-06T05:19:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.063+0000 7fdc85d9b700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdc68066f30 0x7fdc680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.064+0000 7fdc85d9b700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdc68066f30 0x7fdc680693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc8806aa90 tx=0x7fdc7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.066+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc7808c080 con 0x7fdc8806a6a0 2023-10-06T05:19:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.314+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc8804c380 con 0x7fdc8806a6a0 2023-10-06T05:19:20.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.315+0000 7fdc767fc700 1 -- 172.21.15.33:0/65467898 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fdc78052d10 con 0x7fdc8806a6a0 2023-10-06T05:19:20.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:20.317 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:31.211707+0000","modified":"2023-10-06T05:16:34.745629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.33:6834/103971342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":103971342},{"type":"v1","addr":"172.21.15.33:6834","nonce":103971342}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdc68066f30 msgr2=0x7fdc680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdc68066f30 0x7fdc680693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc8806aa90 tx=0x7fdc7c008040 comp rx=0 tx=0).stop 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 msgr2=0x7fdc8816de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 0x7fdc8816de70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdc880635d0 tx=0x7fdc78017000 comp rx=0 tx=0).stop 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 shutdown_connections 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdc68066f30 0x7fdc680693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc88155af0 0x7fdc880aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc8806a6a0 0x7fdc8816de70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 --2- 172.21.15.33:0/65467898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc880c8b50 0x7fdc8816a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 >> 172.21.15.33:0/65467898 conn(0x7fdc8806d030 msgr2=0x7fdc88159100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 shutdown_connections 2023-10-06T05:19:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.317+0000 7fdc8759e700 1 -- 172.21.15.33:0/65467898 wait complete. 2023-10-06T05:19:20.320 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2023-10-06T05:19:20.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:20.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.522+0000 7f300b39d700 1 Processor -- start 2023-10-06T05:19:20.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.522+0000 7f300b39d700 1 -- start start 2023-10-06T05:19:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.523+0000 7f300b39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3004159bb0 0x7f30040b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.523+0000 7f300b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040cbd80 con 0x7f30040ccfc0 2023-10-06T05:19:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.523+0000 7f300b39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040af5f0 con 0x7f30040cf3b0 2023-10-06T05:19:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.523+0000 7f300b39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30040af8d0 con 0x7f3004159bb0 2023-10-06T05:19:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300a39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f30040ccfc0 0x7f30040cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59358/0 (socket says 172.21.15.33:59358) 2023-10-06T05:19:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300a39b700 1 -- 172.21.15.33:0/3067788220 learned_addr learned my addr 172.21.15.33:0/3067788220 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:20.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300ab9c700 1 --2- 172.21.15.33:0/3067788220 >> v2:172.21.15.33:3300/0 conn(0x7f3004159bb0 0x7f30040b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f3009399700 1 -- 172.21.15.33:0/3067788220 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873386518 0 0) 0x7f30040cbd80 con 0x7f30040ccfc0 2023-10-06T05:19:20.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f3009399700 1 -- 172.21.15.33:0/3067788220 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff0003570 con 0x7f30040ccfc0 2023-10-06T05:19:20.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300ab9c700 1 -- 172.21.15.33:0/3067788220 >> v1:172.21.15.167:6789/0 conn(0x7f30040cf3b0 legacy=0x7f30041594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300ab9c700 1 -- 172.21.15.33:0/3067788220 >> v1:172.21.15.33:6789/0 conn(0x7f30040ccfc0 legacy=0x7f30040cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300ab9c700 1 -- 172.21.15.33:0/3067788220 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040afbb0 con 0x7f3004159bb0 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.524+0000 7f300ab9c700 1 --2- 172.21.15.33:0/3067788220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004159bb0 0x7f30040b0460 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009e90 tx=0x7f2ffc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e291e8fca86a4907 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f3009399700 1 -- 172.21.15.33:0/3067788220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00e070 con 0x7f3004159bb0 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f3009399700 1 -- 172.21.15.33:0/3067788220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc00bc40 con 0x7f3004159bb0 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f3009399700 1 -- 172.21.15.33:0/3067788220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc012430 con 0x7f3004159bb0 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 -- 172.21.15.33:0/3067788220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004159bb0 msgr2=0x7f30040b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 --2- 172.21.15.33:0/3067788220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004159bb0 0x7f30040b0460 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009e90 tx=0x7f2ffc009780 comp rx=0 tx=0).stop 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 -- 172.21.15.33:0/3067788220 shutdown_connections 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 --2- 172.21.15.33:0/3067788220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004159bb0 0x7f30040b0460 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 -- 172.21.15.33:0/3067788220 >> 172.21.15.33:0/3067788220 conn(0x7f300406cff0 msgr2=0x7f300406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:20.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 -- 172.21.15.33:0/3067788220 shutdown_connections 2023-10-06T05:19:20.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.525+0000 7f300b39d700 1 -- 172.21.15.33:0/3067788220 wait complete. 2023-10-06T05:19:20.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.527+0000 7f300b39d700 1 Processor -- start 2023-10-06T05:19:20.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.527+0000 7f300b39d700 1 -- start start 2023-10-06T05:19:20.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30040ccfc0 0x7f30040c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3004159bb0 0x7f30040c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 0x7f30040c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040ca2a0 con 0x7f3004169c90 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040b3d50 con 0x7f30040ccfc0 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30040b3420 con 0x7f3004159bb0 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 0x7f30040c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 0x7f30040c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36190/0 (socket says 172.21.15.33:36190) 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 -- 172.21.15.33:0/4006335937 learned_addr learned my addr 172.21.15.33:0/4006335937 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 -- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3004159bb0 msgr2=0x7f30040c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3004159bb0 0x7f30040c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 -- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30040ccfc0 msgr2=0x7f30040c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30040ccfc0 0x7f30040c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.528+0000 7f300ab9c700 1 -- 172.21.15.33:0/4006335937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc009a20 con 0x7f3004169c90 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.529+0000 7f300ab9c700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 0x7f30040c6810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f300406be20 tx=0x7f2ffc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.529+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc012e40 con 0x7f3004169c90 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.529+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc01b630 con 0x7f3004169c90 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.529+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc01b9f0 con 0x7f3004169c90 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.529+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30040b3700 con 0x7f3004169c90 2023-10-06T05:19:20.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.529+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30040c6d50 con 0x7f3004169c90 2023-10-06T05:19:20.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.530+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300404c380 con 0x7f3004169c90 2023-10-06T05:19:20.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.531+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ffc01c400 con 0x7f3004169c90 2023-10-06T05:19:20.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.532+0000 7f2ffa7fc700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fe8066f30 0x7f2fe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.532+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2ffc086270 con 0x7f3004169c90 2023-10-06T05:19:20.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.532+0000 7f300a39b700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fe8066f30 0x7f2fe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.533+0000 7f300a39b700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fe8066f30 0x7f2fe80693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f30000058d0 tx=0x7f30000079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.536+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ffc053240 con 0x7f3004169c90 2023-10-06T05:19:20.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.740+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30040cc920 con 0x7f3004169c90 2023-10-06T05:19:20.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.741+0000 7f2ffa7fc700 1 -- 172.21.15.33:0/4006335937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17120 (secure 0 0 0) 0x7f2ffc056860 con 0x7f3004169c90 2023-10-06T05:19:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:20.743 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":65,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:18:44.699297+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:29.379192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:30.188919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/4010096267":"2023-10-06T06:18:44.138165+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fe8066f30 msgr2=0x7f2fe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fe8066f30 0x7f2fe80693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f30000058d0 tx=0x7f30000079d0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 msgr2=0x7f30040c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 0x7f30040c6810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f300406be20 tx=0x7f2ffc019000 comp rx=0 tx=0).stop 2023-10-06T05:19:20.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 shutdown_connections 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2fe8066f30 0x7f2fe80693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3004169c90 0x7f30040c6810 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30040ccfc0 0x7f30040c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 --2- 172.21.15.33:0/4006335937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3004159bb0 0x7f30040c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 >> 172.21.15.33:0/4006335937 conn(0x7f300406cff0 msgr2=0x7f3004150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 shutdown_connections 2023-10-06T05:19:20.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.743+0000 7f300b39d700 1 -- 172.21.15.33:0/4006335937 wait complete. 2023-10-06T05:19:20.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:19:20.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.950+0000 7ff25a19d700 1 Processor -- start 2023-10-06T05:19:20.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.951+0000 7ff25a19d700 1 -- start start 2023-10-06T05:19:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff254156880 0x7ff2540c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2540c7ec0 0x7ff254157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25a19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2540c8320 con 0x7ff254159e10 2023-10-06T05:19:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540cc290 con 0x7ff2540c7ec0 2023-10-06T05:19:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25406bd10 con 0x7ff254156880 2023-10-06T05:19:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2540c7ec0 0x7ff254157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25899a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff254159e10 0x7ff254156c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59374/0 (socket says 172.21.15.33:59374) 2023-10-06T05:19:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25899a700 1 -- 172.21.15.33:0/2439503256 learned_addr learned my addr 172.21.15.33:0/2439503256 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.952+0000 7ff25919b700 1 --2- 172.21.15.33:0/2439503256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff254156880 0x7ff2540c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff24bfff700 1 -- 172.21.15.33:0/2439503256 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233118426 0 0) 0x7ff244001110 con 0x7ff254159e10 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff25999c700 1 -- 172.21.15.33:0/2439503256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff254156880 msgr2=0x7ff2540c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff25999c700 1 --2- 172.21.15.33:0/2439503256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff254156880 0x7ff2540c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff25999c700 1 -- 172.21.15.33:0/2439503256 >> v1:172.21.15.33:6789/0 conn(0x7ff254159e10 legacy=0x7ff254156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff25999c700 1 -- 172.21.15.33:0/2439503256 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25406a820 con 0x7ff2540c7ec0 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff25999c700 1 --2- 172.21.15.33:0/2439503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2540c7ec0 0x7ff254157360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009e60 tx=0x7ff24c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906afdd97ed60d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff24bfff700 1 -- 172.21.15.33:0/2439503256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00e070 con 0x7ff2540c7ec0 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff24bfff700 1 -- 172.21.15.33:0/2439503256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c00bb60 con 0x7ff2540c7ec0 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.953+0000 7ff24bfff700 1 -- 172.21.15.33:0/2439503256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c012430 con 0x7ff2540c7ec0 2023-10-06T05:19:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 -- 172.21.15.33:0/2439503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2540c7ec0 msgr2=0x7ff254157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2439503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2540c7ec0 0x7ff254157360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009e60 tx=0x7ff24c009780 comp rx=0 tx=0).stop 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 -- 172.21.15.33:0/2439503256 shutdown_connections 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2439503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2540c7ec0 0x7ff254157360 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2439503256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff254156880 0x7ff2540c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 -- 172.21.15.33:0/2439503256 >> 172.21.15.33:0/2439503256 conn(0x7ff25406c560 msgr2=0x7ff25406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 -- 172.21.15.33:0/2439503256 shutdown_connections 2023-10-06T05:19:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.954+0000 7ff25a19d700 1 -- 172.21.15.33:0/2439503256 wait complete. 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.955+0000 7ff25a19d700 1 Processor -- start 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.955+0000 7ff25a19d700 1 -- start start 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.956+0000 7ff25a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2540c7ec0 0x7ff25413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.956+0000 7ff25a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff254156880 0x7ff2540cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.956+0000 7ff25a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254159e10 0x7ff25415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.956+0000 7ff25a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff254140120 con 0x7ff254159e10 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.956+0000 7ff25a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25415b290 con 0x7ff254156880 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.956+0000 7ff25a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540ca5d0 con 0x7ff2540c7ec0 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254159e10 0x7ff25415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254159e10 0x7ff25415c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36208/0 (socket says 172.21.15.33:36208) 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25919b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2540c7ec0 0x7ff25413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25999c700 1 -- 172.21.15.33:0/2602923480 learned_addr learned my addr 172.21.15.33:0/2602923480 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25899a700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff254156880 0x7ff2540cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25919b700 1 -- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff254156880 msgr2=0x7ff2540cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25919b700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff254156880 0x7ff2540cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25919b700 1 -- 172.21.15.33:0/2602923480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254159e10 msgr2=0x7ff25415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25919b700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254159e10 0x7ff25415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.957+0000 7ff25919b700 1 -- 172.21.15.33:0/2602923480 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009a20 con 0x7ff2540c7ec0 2023-10-06T05:19:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.958+0000 7ff25919b700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2540c7ec0 0x7ff25413fbe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff25000ea80 tx=0x7ff2500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.958+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000eec0 con 0x7ff2540c7ec0 2023-10-06T05:19:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.958+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25000c7f0 con 0x7ff2540c7ec0 2023-10-06T05:19:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.958+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540c99c0 con 0x7ff2540c7ec0 2023-10-06T05:19:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.958+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000cb70 con 0x7ff2540c7ec0 2023-10-06T05:19:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.958+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2540ca000 con 0x7ff2540c7ec0 2023-10-06T05:19:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.959+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25404c380 con 0x7ff2540c7ec0 2023-10-06T05:19:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.960+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff250013470 con 0x7ff2540c7ec0 2023-10-06T05:19:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.961+0000 7ff2497fa700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff238066f30 0x7ff2380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.961+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff250084910 con 0x7ff2540c7ec0 2023-10-06T05:19:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.961+0000 7ff25899a700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff238066f30 0x7ff2380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.962+0000 7ff25899a700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff238066f30 0x7ff2380693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff2540cab20 tx=0x7ff244007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:20.967+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2500518e0 con 0x7ff2540c7ec0 2023-10-06T05:19:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.216+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff2540ca320 con 0x7ff2540c7ec0 2023-10-06T05:19:21.219 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:19:21.217+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:19:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.227+0000 7ff2497fa700 1 -- 172.21.15.33:0/2602923480 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff250054f00 con 0x7ff2540c7ec0 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.230+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff238066f30 msgr2=0x7ff2380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.230+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff238066f30 0x7ff2380693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff2540cab20 tx=0x7ff244007f50 comp rx=0 tx=0).stop 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2540c7ec0 msgr2=0x7ff25413fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2540c7ec0 0x7ff25413fbe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff25000ea80 tx=0x7ff2500091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 shutdown_connections 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff238066f30 0x7ff2380693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff254159e10 0x7ff25415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2540c7ec0 0x7ff25413fbe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 --2- 172.21.15.33:0/2602923480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff254156880 0x7ff2540cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 >> 172.21.15.33:0/2602923480 conn(0x7ff25406c560 msgr2=0x7ff25414ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 shutdown_connections 2023-10-06T05:19:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.231+0000 7ff25a19d700 1 -- 172.21.15.33:0/2602923480 wait complete. 2023-10-06T05:19:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:19:21.237 INFO:tasks.ceph.mds.c.smithi033.stderr: -6409> 2023-10-06T05:16:35.856+0000 7fda483f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:19:21.258 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:19:21.292 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:19:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.447+0000 7f5b2ab9d700 1 Processor -- start 2023-10-06T05:19:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.447+0000 7f5b2ab9d700 1 -- start start 2023-10-06T05:19:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.448+0000 7f5b2ab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240af540 0x7f5b240af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.448+0000 7f5b2ab9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b2415a650 0x7f5b2415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.448+0000 7f5b2ab9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2415d660 con 0x7f5b240cced0 2023-10-06T05:19:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.448+0000 7f5b2ab9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240ce890 con 0x7f5b2415a650 2023-10-06T05:19:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.448+0000 7f5b2ab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2415aac0 con 0x7f5b240af540 2023-10-06T05:19:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b2939a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b2415a650 0x7f5b2415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b2a39c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5b240cced0 0x7f5b2406c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:41172/0 (socket says 172.21.15.33:41172) 2023-10-06T05:19:21.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b29b9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240af540 0x7f5b240af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b2a39c700 1 -- 172.21.15.33:0/827194325 learned_addr learned my addr 172.21.15.33:0/827194325 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b28b99700 1 -- 172.21.15.33:0/827194325 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541117838 0 0) 0x7f5b2415d660 con 0x7f5b240cced0 2023-10-06T05:19:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b28b99700 1 -- 172.21.15.33:0/827194325 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b10003570 con 0x7f5b240cced0 2023-10-06T05:19:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b29b9b700 1 -- 172.21.15.33:0/827194325 >> v2:172.21.15.167:3300/0 conn(0x7f5b2415a650 msgr2=0x7f5b2415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b29b9b700 1 --2- 172.21.15.33:0/827194325 >> v2:172.21.15.167:3300/0 conn(0x7f5b2415a650 0x7f5b2415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b29b9b700 1 -- 172.21.15.33:0/827194325 >> v1:172.21.15.167:6789/0 conn(0x7f5b240cced0 legacy=0x7f5b2406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b29b9b700 1 -- 172.21.15.33:0/827194325 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2415d890 con 0x7f5b240af540 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.449+0000 7f5b2939a700 1 --2- 172.21.15.33:0/827194325 >> v2:172.21.15.167:3300/0 conn(0x7f5b2415a650 0x7f5b2415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.450+0000 7f5b29b9b700 1 --2- 172.21.15.33:0/827194325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240af540 0x7f5b240af910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b200098e0 tx=0x7f5b200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f27e3c56b185663 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.450+0000 7f5b28b99700 1 -- 172.21.15.33:0/827194325 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20011630 con 0x7f5b240af540 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.450+0000 7f5b28b99700 1 -- 172.21.15.33:0/827194325 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b20011790 con 0x7f5b240af540 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.450+0000 7f5b28b99700 1 -- 172.21.15.33:0/827194325 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20011b30 con 0x7f5b240af540 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.450+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/827194325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240af540 msgr2=0x7f5b240af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.450+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/827194325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240af540 0x7f5b240af910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b200098e0 tx=0x7f5b200091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.451+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/827194325 shutdown_connections 2023-10-06T05:19:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.451+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/827194325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240af540 0x7f5b240af910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.451+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/827194325 >> v2:172.21.15.167:3300/0 conn(0x7f5b2415a650 0x7f5b2415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.451+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/827194325 >> 172.21.15.33:0/827194325 conn(0x7f5b2406d9b0 msgr2=0x7f5b2406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.451+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/827194325 shutdown_connections 2023-10-06T05:19:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.451+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/827194325 wait complete. 2023-10-06T05:19:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.452+0000 7f5b2ab9d700 1 Processor -- start 2023-10-06T05:19:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.452+0000 7f5b2ab9d700 1 -- start start 2023-10-06T05:19:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b2ab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 0x7f5b240b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b2ab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240cced0 0x7f5b240b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b2ab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b2415a650 0x7f5b24147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b2ab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240ca200 con 0x7f5b240af540 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b2ab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b6320 con 0x7f5b2415a650 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b2ab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b5e90 con 0x7f5b240cced0 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b29b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 0x7f5b240b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b29b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 0x7f5b240b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36234/0 (socket says 172.21.15.33:36234) 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.453+0000 7f5b29b9b700 1 -- 172.21.15.33:0/3303622128 learned_addr learned my addr 172.21.15.33:0/3303622128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b2939a700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240cced0 0x7f5b240b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b2a39c700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b2415a650 0x7f5b24147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b29b9b700 1 -- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240cced0 msgr2=0x7f5b240b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b29b9b700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240cced0 0x7f5b240b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b29b9b700 1 -- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b2415a650 msgr2=0x7f5b24147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b29b9b700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b2415a650 0x7f5b24147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b29b9b700 1 -- 172.21.15.33:0/3303622128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14009420 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b2a39c700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b2415a650 0x7f5b24147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b29b9b700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 0x7f5b240b4c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009ed0 tx=0x7f5b20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20011b10 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b200253f0 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.454+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b20009420 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.455+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20025790 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.455+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b240b5560 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.455+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b24157180 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.456+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b20025b40 con 0x7f5b240af540 2023-10-06T05:19:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.457+0000 7f5b19ffb700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.457+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5b20085fe0 con 0x7f5b240af540 2023-10-06T05:19:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.458+0000 7f5b2939a700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.458+0000 7f5b2939a700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5b240c9bd0 tx=0x7f5b14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.461+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b20052f60 con 0x7f5b240af540 2023-10-06T05:19:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:21.708+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5b240b6170 con 0x7f5b240af540 2023-10-06T05:19:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.223+0000 7f5b19ffb700 1 -- 172.21.15.33:0/3303622128 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5b20056580 con 0x7f5b240af540 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 msgr2=0x7f5b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5b240c9bd0 tx=0x7f5b14008040 comp rx=0 tx=0).stop 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 msgr2=0x7f5b240b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 0x7f5b240b4c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009ed0 tx=0x7f5b20017000 comp rx=0 tx=0).stop 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 shutdown_connections 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b2415a650 0x7f5b24147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240af540 0x7f5b240b4c80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 --2- 172.21.15.33:0/3303622128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240cced0 0x7f5b240b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 >> 172.21.15.33:0/3303622128 conn(0x7f5b2406d9b0 msgr2=0x7f5b240caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 shutdown_connections 2023-10-06T05:19:22.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.226+0000 7f5b2ab9d700 1 -- 172.21.15.33:0/3303622128 wait complete. 2023-10-06T05:19:22.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:19:22.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.481+0000 7f7b0179d700 1 Processor -- start 2023-10-06T05:19:22.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.481+0000 7f7b0179d700 1 -- start start 2023-10-06T05:19:22.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.482+0000 7f7b0179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7afc0ad010 0x7f7afc06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:22.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.482+0000 7f7b0179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc15acd0 0x7f7afc158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:22.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.482+0000 7f7b0179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc06b1f0 con 0x7f7afc0ad3e0 2023-10-06T05:19:22.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.482+0000 7f7b0179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc06a3f0 con 0x7f7afc0ad010 2023-10-06T05:19:22.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.482+0000 7f7b0179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc06a5e0 con 0x7f7afc15acd0 2023-10-06T05:19:22.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7afc0ad010 0x7f7afc06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:22.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7afc0ad010 0x7f7afc06ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36240/0 (socket says 172.21.15.33:36240) 2023-10-06T05:19:22.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 -- 172.21.15.33:0/4053856026 learned_addr learned my addr 172.21.15.33:0/4053856026 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:22.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7b00f9c700 1 --2- 172.21.15.33:0/4053856026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc15acd0 0x7f7afc158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:22.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 -- 172.21.15.33:0/4053856026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc15acd0 msgr2=0x7f7afc158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 --2- 172.21.15.33:0/4053856026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc15acd0 0x7f7afc158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 -- 172.21.15.33:0/4053856026 >> v1:172.21.15.167:6789/0 conn(0x7f7afc0ad3e0 legacy=0x7f7afc15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 -- 172.21.15.33:0/4053856026 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc06a8c0 con 0x7f7afc0ad010 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.483+0000 7f7afbfff700 1 --2- 172.21.15.33:0/4053856026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc06ea00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0099b0 tx=0x7f7aec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec80735e183ee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7afaffd700 1 -- 172.21.15.33:0/4053856026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00b8b0 con 0x7f7afc0ad010 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7afaffd700 1 -- 172.21.15.33:0/4053856026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec00ba10 con 0x7f7afc0ad010 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7afaffd700 1 -- 172.21.15.33:0/4053856026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00bdb0 con 0x7f7afc0ad010 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 -- 172.21.15.33:0/4053856026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 msgr2=0x7f7afc06ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 --2- 172.21.15.33:0/4053856026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc06ea00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0099b0 tx=0x7f7aec009250 comp rx=0 tx=0).stop 2023-10-06T05:19:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 -- 172.21.15.33:0/4053856026 shutdown_connections 2023-10-06T05:19:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 --2- 172.21.15.33:0/4053856026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc06ea00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 --2- 172.21.15.33:0/4053856026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc15acd0 0x7f7afc158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 -- 172.21.15.33:0/4053856026 >> 172.21.15.33:0/4053856026 conn(0x7f7afc068090 msgr2=0x7f7afc06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 -- 172.21.15.33:0/4053856026 shutdown_connections 2023-10-06T05:19:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.484+0000 7f7b0179d700 1 -- 172.21.15.33:0/4053856026 wait complete. 2023-10-06T05:19:22.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.486+0000 7f7b0179d700 1 Processor -- start 2023-10-06T05:19:22.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.486+0000 7f7b0179d700 1 -- start start 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7b0179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc0a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7b0179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc0ad3e0 0x7f7afc14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7b0179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7afc15acd0 0x7f7afc0abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7b0179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0cd150 con 0x7f7afc0ad010 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7b0179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc150c00 con 0x7f7afc15acd0 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7b0179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1508d0 con 0x7f7afc0ad3e0 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc0a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc0a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36244/0 (socket says 172.21.15.33:36244) 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.487+0000 7f7afbfff700 1 -- 172.21.15.33:0/2987559999 learned_addr learned my addr 172.21.15.33:0/2987559999 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:22.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7b00f9c700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7afc15acd0 0x7f7afc0abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afb7fe700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc0ad3e0 0x7f7afc14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afbfff700 1 -- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc0ad3e0 msgr2=0x7f7afc14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afbfff700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc0ad3e0 0x7f7afc14f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afbfff700 1 -- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7afc15acd0 msgr2=0x7f7afc0abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afbfff700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7afc15acd0 0x7f7afc0abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afbfff700 1 -- 172.21.15.33:0/2987559999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0094f0 con 0x7f7afc0ad010 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afb7fe700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc0ad3e0 0x7f7afc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7afbfff700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc0a9890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0094c0 tx=0x7f7aec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec011d50 con 0x7f7afc0ad010 2023-10-06T05:19:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec020070 con 0x7f7afc0ad010 2023-10-06T05:19:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec01b6a0 con 0x7f7afc0ad010 2023-10-06T05:19:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0a9f90 con 0x7f7afc0ad010 2023-10-06T05:19:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.488+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc0aa550 con 0x7f7afc0ad010 2023-10-06T05:19:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.489+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc0aa770 con 0x7f7afc0ad010 2023-10-06T05:19:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.490+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7aec022030 con 0x7f7afc0ad010 2023-10-06T05:19:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.491+0000 7f7ae3fff700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae8066f30 0x7f7ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.491+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7aec086690 con 0x7f7afc0ad010 2023-10-06T05:19:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.492+0000 7f7afb7fe700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae8066f30 0x7f7ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.492+0000 7f7afb7fe700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae8066f30 0x7f7ae80693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0ca250 tx=0x7f7af0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.495+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7afc0aa770 con 0x7f7afc0ad010 2023-10-06T05:19:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:22.698+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7afc0aa770 con 0x7f7afc0ad010 2023-10-06T05:19:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.230+0000 7f7ae3fff700 1 -- 172.21.15.33:0/2987559999 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7afc0aa770 con 0x7f7afc0ad010 2023-10-06T05:19:23.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.234+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae8066f30 msgr2=0x7f7ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:23.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.234+0000 7f7b0179d700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae8066f30 0x7f7ae80693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0ca250 tx=0x7f7af0009040 comp rx=0 tx=0).stop 2023-10-06T05:19:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.234+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 msgr2=0x7f7afc0a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.234+0000 7f7b0179d700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc0a9890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0094c0 tx=0x7f7aec016000 comp rx=0 tx=0).stop 2023-10-06T05:19:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 shutdown_connections 2023-10-06T05:19:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ae8066f30 0x7f7ae80693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7afc15acd0 0x7f7afc0abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7afc0ad010 0x7f7afc0a9890 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 --2- 172.21.15.33:0/2987559999 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7afc0ad3e0 0x7f7afc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 >> 172.21.15.33:0/2987559999 conn(0x7f7afc068090 msgr2=0x7f7afc0af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 shutdown_connections 2023-10-06T05:19:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.235+0000 7f7b0179d700 1 -- 172.21.15.33:0/2987559999 wait complete. 2023-10-06T05:19:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:19:23.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:19:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.452+0000 7f3f6359e700 1 Processor -- start 2023-10-06T05:19:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.452+0000 7f3f6359e700 1 -- start start 2023-10-06T05:19:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 0x7f3f64156b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3f641570a0 0x7f3f64157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6406b1f0 con 0x7f3f640ad010 2023-10-06T05:19:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6406a3f0 con 0x7f3f641570a0 2023-10-06T05:19:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6406f110 con 0x7f3f640ad3e0 2023-10-06T05:19:23.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3f640ad010 0x7f3f6406ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59380/0 (socket says 172.21.15.33:59380) 2023-10-06T05:19:23.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6259c700 1 -- 172.21.15.33:0/1777901912 learned_addr learned my addr 172.21.15.33:0/1777901912 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:23.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f62d9d700 1 --2- 172.21.15.33:0/1777901912 >> v2:172.21.15.167:3300/0 conn(0x7f3f641570a0 0x7f3f64157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f61d9b700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 0x7f3f64156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826647592 0 0) 0x7f3f6406b1f0 con 0x7f3f640ad010 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.454+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f4c003570 con 0x7f3f640ad010 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991975298 0 0) 0x7f3f4c003570 con 0x7f3f640ad010 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 msgr2=0x7f3f64156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 0x7f3f64156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 >> v2:172.21.15.167:3300/0 conn(0x7f3f641570a0 msgr2=0x7f3f64157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 --2- 172.21.15.33:0/1777901912 >> v2:172.21.15.167:3300/0 conn(0x7f3f641570a0 0x7f3f64157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64157a40 con 0x7f3f640ad010 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3f54001470 con 0x7f3f640ad010 2023-10-06T05:19:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 >> v1:172.21.15.33:6789/0 conn(0x7f3f640ad010 legacy=0x7f3f6406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f4c003a60 0x7f3f4c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 0x7f3f4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f4c004d50 0x7f3f4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6406f110 con 0x7f3f4c003a60 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c00ba60 con 0x7f3f4c004d50 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c00bba0 con 0x7f3f640ad3e0 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f6259c700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f4c003a60 0x7f3f4c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f62d9d700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f4c004d50 0x7f3f4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.455+0000 7f3f61d9b700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 0x7f3f4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6259c700 1 -- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 msgr2=0x7f3f4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6259c700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f640ad3e0 0x7f3f4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6259c700 1 -- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f4c004d50 msgr2=0x7f3f4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6259c700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f4c004d50 0x7f3f4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6259c700 1 -- 172.21.15.33:0/1777901912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c003570 con 0x7f3f4c003a60 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f62d9d700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f4c004d50 0x7f3f4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6259c700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f4c003a60 0x7f3f4c003e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400c3c0 tx=0x7f3f6406b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29157a9a5df32d9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400cc50 con 0x7f3f4c003a60 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400cdb0 con 0x7f3f4c003a60 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6159a700 1 -- 172.21.15.33:0/1777901912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54005b70 con 0x7f3f4c003a60 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6359e700 1 -- 172.21.15.33:0/1777901912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f4c003a60 msgr2=0x7f3f4c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f6359e700 1 --2- 172.21.15.33:0/1777901912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f4c003a60 0x7f3f4c003e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400c3c0 tx=0x7f3f6406b160 comp rx=0 tx=0).stop 2023-10-06T05:19:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.456+0000 7f3f62d9d700 1 -- 172.21.15.33:0/1777901912 reap_dead start 2023-10-06T05:19:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.457+0000 7f3f6359e700 1 -- 172.21.15.33:0/1777901912 shutdown_connections 2023-10-06T05:19:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.457+0000 7f3f6359e700 1 -- 172.21.15.33:0/1777901912 >> 172.21.15.33:0/1777901912 conn(0x7f3f64068090 msgr2=0x7f3f6406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.457+0000 7f3f6359e700 1 -- 172.21.15.33:0/1777901912 shutdown_connections 2023-10-06T05:19:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.457+0000 7f3f6359e700 1 -- 172.21.15.33:0/1777901912 wait complete. 2023-10-06T05:19:23.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.458+0000 7f3f6359e700 1 Processor -- start 2023-10-06T05:19:23.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.458+0000 7f3f6359e700 1 -- start start 2023-10-06T05:19:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f6359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f4c003a60 0x7f3f640c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f6359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f640cc6b0 0x7f3f640c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f6359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 0x7f3f640c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f6359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640cc220 con 0x7f3f640cb050 2023-10-06T05:19:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f6359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640cb610 con 0x7f3f640cc6b0 2023-10-06T05:19:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f6359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640cb8f0 con 0x7f3f4c003a60 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 0x7f3f640c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 0x7f3f640c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36272/0 (socket says 172.21.15.33:36272) 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.459+0000 7f3f62d9d700 1 -- 172.21.15.33:0/3690195574 learned_addr learned my addr 172.21.15.33:0/3690195574 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f61d9b700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f640cc6b0 0x7f3f640c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f6259c700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f4c003a60 0x7f3f640c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f62d9d700 1 -- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f4c003a60 msgr2=0x7f3f640c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f62d9d700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f4c003a60 0x7f3f640c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f62d9d700 1 -- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f640cc6b0 msgr2=0x7f3f640c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f62d9d700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f640cc6b0 0x7f3f640c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f62d9d700 1 -- 172.21.15.33:0/3690195574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640cbbd0 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f61d9b700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f640cc6b0 0x7f3f640c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f62d9d700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 0x7f3f640c3a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f6406beb0 tx=0x7f3f5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c01b070 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c00f040 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00e050 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f640cbeb0 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.460+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6414a5e0 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.461+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f64063ff0 con 0x7f3f640cb050 2023-10-06T05:19:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.462+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f5c003020 con 0x7f3f640cb050 2023-10-06T05:19:23.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.464+0000 7f3f527fc700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f44066f30 0x7f3f440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:23.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.464+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5836+0+0 (secure 0 0 0) 0x7f3f5c084b80 con 0x7f3f640cb050 2023-10-06T05:19:23.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.465+0000 7f3f6259c700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f44066f30 0x7f3f440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:23.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.465+0000 7f3f6259c700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f44066f30 0x7f3f440693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f54015040 tx=0x7f3f54012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:23.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.468+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f5c051cd0 con 0x7f3f640cb050 2023-10-06T05:19:23.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:23.672+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f64064790 con 0x7f3f640cb050 2023-10-06T05:19:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.229+0000 7f3f527fc700 1 -- 172.21.15.33:0/3690195574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3f5c084e60 con 0x7f3f640cb050 2023-10-06T05:19:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.234+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f44066f30 msgr2=0x7f3f440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.234+0000 7f3f6359e700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f44066f30 0x7f3f440693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f54015040 tx=0x7f3f54012000 comp rx=0 tx=0).stop 2023-10-06T05:19:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.234+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 msgr2=0x7f3f640c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.234+0000 7f3f6359e700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 0x7f3f640c3a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f6406beb0 tx=0x7f3f5c00b040 comp rx=0 tx=0).stop 2023-10-06T05:19:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 shutdown_connections 2023-10-06T05:19:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f44066f30 0x7f3f440693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f640cb050 0x7f3f640c3a20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f4c003a60 0x7f3f640c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 --2- 172.21.15.33:0/3690195574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f640cc6b0 0x7f3f640c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 >> 172.21.15.33:0/3690195574 conn(0x7f3f64068090 msgr2=0x7f3f64155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.235+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 shutdown_connections 2023-10-06T05:19:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.237+0000 7f3f6359e700 1 -- 172.21.15.33:0/3690195574 wait complete. 2023-10-06T05:19:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:19:24.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:19:24.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.451+0000 7f2afffff700 1 Processor -- start 2023-10-06T05:19:24.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.451+0000 7f2afffff700 1 -- start start 2023-10-06T05:19:24.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.452+0000 7f2afffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2b000b1f90 0x7f2b000abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.452+0000 7f2afffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b00070440 0x7f2b00067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.452+0000 7f2afffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0006ae10 con 0x7f2b00068140 2023-10-06T05:19:24.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.452+0000 7f2afffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0006a2d0 con 0x7f2b000b1f90 2023-10-06T05:19:24.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b00154290 con 0x7f2b00070440 2023-10-06T05:19:24.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2aff7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2b00068140 0x7f2b0006c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59394/0 (socket says 172.21.15.33:59394) 2023-10-06T05:19:24.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2aff7fe700 1 -- 172.21.15.33:0/2127536007 learned_addr learned my addr 172.21.15.33:0/2127536007 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:24.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afeffd700 1 --2- 172.21.15.33:0/2127536007 >> v2:172.21.15.33:3300/0 conn(0x7f2b000b1f90 0x7f2b000abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afe7fc700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b00070440 0x7f2b00067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afdffb700 1 -- 172.21.15.33:0/2127536007 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176397969 0 0) 0x7f2b0006ae10 con 0x7f2b00068140 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afdffb700 1 -- 172.21.15.33:0/2127536007 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae8003570 con 0x7f2b00068140 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afeffd700 1 -- 172.21.15.33:0/2127536007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b00070440 msgr2=0x7f2b00067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afeffd700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b00070440 0x7f2b00067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afeffd700 1 -- 172.21.15.33:0/2127536007 >> v1:172.21.15.33:6789/0 conn(0x7f2b00068140 legacy=0x7f2b0006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afeffd700 1 -- 172.21.15.33:0/2127536007 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00068580 con 0x7f2b000b1f90 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.453+0000 7f2afe7fc700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b00070440 0x7f2b00067c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afeffd700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b000b1f90 0x7f2b000abbd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098e0 tx=0x7f2af00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dede21479740ebe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afdffb700 1 -- 172.21.15.33:0/2127536007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000b640 con 0x7f2b000b1f90 2023-10-06T05:19:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afdffb700 1 -- 172.21.15.33:0/2127536007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000b7a0 con 0x7f2b000b1f90 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afdffb700 1 -- 172.21.15.33:0/2127536007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000bb40 con 0x7f2b000b1f90 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afffff700 1 -- 172.21.15.33:0/2127536007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b000b1f90 msgr2=0x7f2b000abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afffff700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b000b1f90 0x7f2b000abbd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098e0 tx=0x7f2af00091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afffff700 1 -- 172.21.15.33:0/2127536007 shutdown_connections 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afffff700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b000b1f90 0x7f2b000abbd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afffff700 1 --2- 172.21.15.33:0/2127536007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b00070440 0x7f2b00067c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.454+0000 7f2afffff700 1 -- 172.21.15.33:0/2127536007 >> 172.21.15.33:0/2127536007 conn(0x7f2b0006ea70 msgr2=0x7f2b0006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.455+0000 7f2afffff700 1 -- 172.21.15.33:0/2127536007 shutdown_connections 2023-10-06T05:19:24.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.455+0000 7f2afffff700 1 -- 172.21.15.33:0/2127536007 wait complete. 2023-10-06T05:19:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.456+0000 7f2afffff700 1 Processor -- start 2023-10-06T05:19:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.456+0000 7f2afffff700 1 -- start start 2023-10-06T05:19:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 0x7f2b0013a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b00070440 0x7f2b001512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b000b1f90 0x7f2b0014f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0013a6f0 con 0x7f2b00068140 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b00155230 con 0x7f2b00070440 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b00151cd0 con 0x7f2b000b1f90 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 0x7f2b0013a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 0x7f2b0013a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36300/0 (socket says 172.21.15.33:36300) 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.457+0000 7f2afeffd700 1 -- 172.21.15.33:0/1587228631 learned_addr learned my addr 172.21.15.33:0/1587228631 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afeffd700 1 -- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b000b1f90 msgr2=0x7f2b0014f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afeffd700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b000b1f90 0x7f2b0014f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afeffd700 1 -- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b00070440 msgr2=0x7f2b001512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afeffd700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b00070440 0x7f2b001512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afeffd700 1 -- 172.21.15.33:0/1587228631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0009420 con 0x7f2b00068140 2023-10-06T05:19:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afeffd700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 0x7f2b0013a1b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009f80 tx=0x7f2af0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000bb20 con 0x7f2b00068140 2023-10-06T05:19:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af0011420 con 0x7f2b00068140 2023-10-06T05:19:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b00151fb0 con 0x7f2b00068140 2023-10-06T05:19:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af00117a0 con 0x7f2b00068140 2023-10-06T05:19:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.458+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b00152570 con 0x7f2b00068140 2023-10-06T05:19:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.459+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0004c380 con 0x7f2b00068140 2023-10-06T05:19:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.460+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2af0011b10 con 0x7f2b00068140 2023-10-06T05:19:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.461+0000 7f2ae6ffd700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2aec066fb0 0x7f2aec069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.461+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2af0085be0 con 0x7f2b00068140 2023-10-06T05:19:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.462+0000 7f2afe7fc700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2aec066fb0 0x7f2aec069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.462+0000 7f2afe7fc700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2aec066fb0 0x7f2aec069460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2af4006f00 tx=0x7f2af4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.466+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2af0052f20 con 0x7f2b00068140 2023-10-06T05:19:24.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.667+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2b0006a4e0 con 0x7f2b00068140 2023-10-06T05:19:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.671+0000 7f2ae6ffd700 1 -- 172.21.15.33:0/1587228631 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2af00150a0 con 0x7f2b00068140 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2aec066fb0 msgr2=0x7f2aec069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2aec066fb0 0x7f2aec069460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2af4006f00 tx=0x7f2af4008040 comp rx=0 tx=0).stop 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 msgr2=0x7f2b0013a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 0x7f2b0013a1b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009f80 tx=0x7f2af0017000 comp rx=0 tx=0).stop 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 shutdown_connections 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2aec066fb0 0x7f2aec069460 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b000b1f90 0x7f2b0014f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b00068140 0x7f2b0013a1b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 --2- 172.21.15.33:0/1587228631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b00070440 0x7f2b001512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 >> 172.21.15.33:0/1587228631 conn(0x7f2b0006ea70 msgr2=0x7f2b0006d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 shutdown_connections 2023-10-06T05:19:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.673+0000 7f2afffff700 1 -- 172.21.15.33:0/1587228631 wait complete. 2023-10-06T05:19:24.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout' 2023-10-06T05:19:24.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.878+0000 7f90c3fff700 1 Processor -- start 2023-10-06T05:19:24.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.878+0000 7f90c3fff700 1 -- start start 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90c406df70 0x7f90c40afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f90c40b0310 0x7f90c4159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40b1e00 con 0x7f90c406df70 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c406f6b0 con 0x7f90c40b0310 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c406e3e0 con 0x7f90c4159530 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c27fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90c406df70 0x7f90c40afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c27fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90c406df70 0x7f90c40afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36312/0 (socket says 172.21.15.33:36312) 2023-10-06T05:19:24.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.879+0000 7f90c27fc700 1 -- 172.21.15.33:0/284708692 learned_addr learned my addr 172.21.15.33:0/284708692 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:24.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c37fe700 1 --2- 172.21.15.33:0/284708692 >> v2:172.21.15.167:3300/0 conn(0x7f90c40b0310 0x7f90c4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/284708692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c27fc700 1 -- 172.21.15.33:0/284708692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 msgr2=0x7f90c406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c27fc700 1 --2- 172.21.15.33:0/284708692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c27fc700 1 -- 172.21.15.33:0/284708692 >> v2:172.21.15.167:3300/0 conn(0x7f90c40b0310 msgr2=0x7f90c4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c27fc700 1 --2- 172.21.15.33:0/284708692 >> v2:172.21.15.167:3300/0 conn(0x7f90c40b0310 0x7f90c4159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c27fc700 1 -- 172.21.15.33:0/284708692 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c406f220 con 0x7f90c406df70 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c37fe700 1 --2- 172.21.15.33:0/284708692 >> v2:172.21.15.167:3300/0 conn(0x7f90c40b0310 0x7f90c4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c27fc700 1 --2- 172.21.15.33:0/284708692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c40afdd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f90c40b09e0 tx=0x7f90b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ecc8486ef21331 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:24.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.880+0000 7f90c1ffb700 1 -- 172.21.15.33:0/284708692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800b820 con 0x7f90c406df70 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c1ffb700 1 -- 172.21.15.33:0/284708692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800b980 con 0x7f90c406df70 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c1ffb700 1 -- 172.21.15.33:0/284708692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800bd20 con 0x7f90c406df70 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 -- 172.21.15.33:0/284708692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 msgr2=0x7f90c40afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 --2- 172.21.15.33:0/284708692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c40afdd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f90c40b09e0 tx=0x7f90b8009250 comp rx=0 tx=0).stop 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 -- 172.21.15.33:0/284708692 shutdown_connections 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 --2- 172.21.15.33:0/284708692 >> v2:172.21.15.167:3300/0 conn(0x7f90c40b0310 0x7f90c4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 --2- 172.21.15.33:0/284708692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 --2- 172.21.15.33:0/284708692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c40afdd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 -- 172.21.15.33:0/284708692 >> 172.21.15.33:0/284708692 conn(0x7f90c406c560 msgr2=0x7f90c406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 -- 172.21.15.33:0/284708692 shutdown_connections 2023-10-06T05:19:24.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.881+0000 7f90c3fff700 1 -- 172.21.15.33:0/284708692 wait complete. 2023-10-06T05:19:24.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.883+0000 7f90c3fff700 1 Processor -- start 2023-10-06T05:19:24.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.883+0000 7f90c3fff700 1 -- start start 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c4149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c40b0310 0x7f90c41581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40cda80 con 0x7f90c406df70 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40cce70 con 0x7f90c40b0310 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40cd150 con 0x7f90c4159530 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c4149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c4149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36324/0 (socket says 172.21.15.33:36324) 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 -- 172.21.15.33:0/846824230 learned_addr learned my addr 172.21.15.33:0/846824230 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c27fc700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c40b0310 0x7f90c41581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c37fe700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 -- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 msgr2=0x7f90c41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c41586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 -- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c40b0310 msgr2=0x7f90c41581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c40b0310 0x7f90c41581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.884+0000 7f90c2ffd700 1 -- 172.21.15.33:0/846824230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80094f0 con 0x7f90c406df70 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.885+0000 7f90c2ffd700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c4149410 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f90b400d890 tx=0x7f90b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.885+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40125f0 con 0x7f90c406df70 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.885+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4012770 con 0x7f90c406df70 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.885+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c40cd430 con 0x7f90c406df70 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.885+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4012b30 con 0x7f90c406df70 2023-10-06T05:19:24.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.885+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c4149950 con 0x7f90c406df70 2023-10-06T05:19:24.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.886+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c404c380 con 0x7f90c406df70 2023-10-06T05:19:24.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.887+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90b401e460 con 0x7f90c406df70 2023-10-06T05:19:24.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.888+0000 7f90aaffd700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90b0066f30 0x7f90b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:24.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.888+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f90b4084b80 con 0x7f90c406df70 2023-10-06T05:19:24.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.888+0000 7f90c27fc700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90b0066f30 0x7f90b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:24.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.889+0000 7f90c27fc700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90b0066f30 0x7f90b00693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90c40b09e0 tx=0x7f90b8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:24.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:24.892+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90b4051ec0 con 0x7f90c406df70 2023-10-06T05:19:25.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.181+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]} v 0) v1 -- 0x7f90c406dd50 con 0x7f90c406df70 2023-10-06T05:19:25.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.238+0000 7f90aaffd700 1 -- 172.21.15.33:0/846824230 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_config_session_timeout"]}]=0 v246) v1 ==== 159+0+0 (secure 0 0 0) 0x7f90b4084e60 con 0x7f90c406df70 2023-10-06T05:19:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.240+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90b0066f30 msgr2=0x7f90b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.240+0000 7f90c3fff700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90b0066f30 0x7f90b00693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90c40b09e0 tx=0x7f90b8017000 comp rx=0 tx=0).stop 2023-10-06T05:19:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.240+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 msgr2=0x7f90c4149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.240+0000 7f90c3fff700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c4149410 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f90b400d890 tx=0x7f90b4008040 comp rx=0 tx=0).stop 2023-10-06T05:19:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 shutdown_connections 2023-10-06T05:19:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90b0066f30 0x7f90b00693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90c4159530 0x7f90c41586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90c406df70 0x7f90c4149410 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 --2- 172.21.15.33:0/846824230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c40b0310 0x7f90c41581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 >> 172.21.15.33:0/846824230 conn(0x7f90c406c560 msgr2=0x7f90c415c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 shutdown_connections 2023-10-06T05:19:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.241+0000 7f90c3fff700 1 -- 172.21.15.33:0/846824230 wait complete. 2023-10-06T05:19:25.260 INFO:tasks.cephfs_test_runner:test_config_session_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:19:25.260 INFO:tasks.cephfs_test_runner:Starting test: test_create_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:19:25.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync' 2023-10-06T05:19:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.445+0000 7f7c2f59d700 1 Processor -- start 2023-10-06T05:19:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.445+0000 7f7c2f59d700 1 -- start start 2023-10-06T05:19:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c280cf3b0 0x7f7c281594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c280cbd80 con 0x7f7c280af0a0 2023-10-06T05:19:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2f59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c280af5f0 con 0x7f7c280ccfc0 2023-10-06T05:19:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280af8d0 con 0x7f7c280cf3b0 2023-10-06T05:19:25.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2ed9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7c280af0a0 0x7f7c280b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59408/0 (socket says 172.21.15.33:59408) 2023-10-06T05:19:25.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2ed9c700 1 -- 172.21.15.33:0/1665429346 learned_addr learned my addr 172.21.15.33:0/1665429346 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c280cf3b0 0x7f7c281594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661327591 0 0) 0x7f7c280cbd80 con 0x7f7c280af0a0 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.447+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c14003570 con 0x7f7c280af0a0 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1607544859 0 0) 0x7f7c14003570 con 0x7f7c280af0a0 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c280cf3b0 msgr2=0x7f7c281594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c280cf3b0 0x7f7c281594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 >> v1:172.21.15.167:6789/0 conn(0x7f7c280ccfc0 legacy=0x7f7c280cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280afbb0 con 0x7f7c280af0a0 2023-10-06T05:19:25.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f7c200017f0 con 0x7f7c280af0a0 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 >> v1:172.21.15.33:6789/0 conn(0x7f7c280af0a0 legacy=0x7f7c280b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14003a90 0x7f7c14003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c14004810 0x7f7c14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14005180 0x7f7c1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280af8d0 con 0x7f7c14005180 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1400bfa0 con 0x7f7c14003a90 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1400c0e0 con 0x7f7c14004810 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2ed9c700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14005180 0x7f7c1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c14004810 0x7f7c14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.448+0000 7f7c2e59b700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14003a90 0x7f7c14003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14003a90 msgr2=0x7f7c14003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14003a90 0x7f7c14003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/1665429346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14005180 msgr2=0x7f7c1400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14005180 0x7f7c1400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/1665429346 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c14003570 con 0x7f7c14004810 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2ed9c700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c14005180 0x7f7c1400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c14004810 0x7f7c14004c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c280b0250 tx=0x7f7c180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5299a5ca83dddb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800e710 con 0x7f7c14004810 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800e870 con 0x7f7c14004810 2023-10-06T05:19:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.449+0000 7f7c2d599700 1 -- 172.21.15.33:0/1665429346 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800ec30 con 0x7f7c14004810 2023-10-06T05:19:25.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2f59d700 1 -- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c14004810 msgr2=0x7f7c14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/1665429346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c14004810 0x7f7c14004c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c280b0250 tx=0x7f7c180091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2ed9c700 1 -- 172.21.15.33:0/1665429346 reap_dead start 2023-10-06T05:19:25.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2f59d700 1 -- 172.21.15.33:0/1665429346 shutdown_connections 2023-10-06T05:19:25.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2f59d700 1 -- 172.21.15.33:0/1665429346 >> 172.21.15.33:0/1665429346 conn(0x7f7c2806cff0 msgr2=0x7f7c2806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:25.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2f59d700 1 -- 172.21.15.33:0/1665429346 shutdown_connections 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.450+0000 7f7c2f59d700 1 -- 172.21.15.33:0/1665429346 wait complete. 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.451+0000 7f7c2f59d700 1 Processor -- start 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.451+0000 7f7c2f59d700 1 -- start start 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.452+0000 7f7c2f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14004810 0x7f7c280ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.452+0000 7f7c2f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 0x7f7c28143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.452+0000 7f7c2f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c28158350 0x7f7c281587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.452+0000 7f7c2f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280c9ba0 con 0x7f7c280b32a0 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.452+0000 7f7c2f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28151710 con 0x7f7c14004810 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.452+0000 7f7c2f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28158d00 con 0x7f7c28158350 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 0x7f7c28143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 0x7f7c28143170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36354/0 (socket says 172.21.15.33:36354) 2023-10-06T05:19:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/666960128 learned_addr learned my addr 172.21.15.33:0/666960128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2ed9c700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c28158350 0x7f7c281587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2e59b700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14004810 0x7f7c280ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c28158350 msgr2=0x7f7c281587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c28158350 0x7f7c281587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14004810 msgr2=0x7f7c280ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14004810 0x7f7c280ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 -- 172.21.15.33:0/666960128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c281518a0 con 0x7f7c280b32a0 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2ed9c700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c28158350 0x7f7c281587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2e59b700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14004810 0x7f7c280ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.453+0000 7f7c2dd9a700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 0x7f7c28143170 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7c1801eef0 tx=0x7f7c1801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:25.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.454+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18023070 con 0x7f7c280b32a0 2023-10-06T05:19:25.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.454+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800e9e0 con 0x7f7c280b32a0 2023-10-06T05:19:25.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.454+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c280b3710 con 0x7f7c280b32a0 2023-10-06T05:19:25.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.454+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1801e030 con 0x7f7c280b32a0 2023-10-06T05:19:25.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.454+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c280b3960 con 0x7f7c280b32a0 2023-10-06T05:19:25.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.455+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c28151a30 con 0x7f7c280b32a0 2023-10-06T05:19:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.456+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c18013470 con 0x7f7c280b32a0 2023-10-06T05:19:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.456+0000 7f7c1e7fc700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c0c066f30 0x7f7c0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.457+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7c18085c10 con 0x7f7c280b32a0 2023-10-06T05:19:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.457+0000 7f7c2e59b700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c0c066f30 0x7f7c0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.458+0000 7f7c2e59b700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c0c066f30 0x7f7c0c0693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007bc0 tx=0x7f7c24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:25.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.461+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c18052f50 con 0x7f7c280b32a0 2023-10-06T05:19:25.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:25.744+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]} v 0) v1 -- 0x7f7c2804c380 con 0x7f7c280b32a0 2023-10-06T05:19:26.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.243+0000 7f7c1e7fc700 1 -- 172.21.15.33:0/666960128 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]}]=0 v247) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7c18056570 con 0x7f7c280b32a0 2023-10-06T05:19:26.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c0c066f30 msgr2=0x7f7c0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c0c066f30 0x7f7c0c0693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007bc0 tx=0x7f7c24009040 comp rx=0 tx=0).stop 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 msgr2=0x7f7c28143170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 0x7f7c28143170 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7c1801eef0 tx=0x7f7c1801a000 comp rx=0 tx=0).stop 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 shutdown_connections 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c0c066f30 0x7f7c0c0693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c28158350 0x7f7c281587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c14004810 0x7f7c280ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 --2- 172.21.15.33:0/666960128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c280b32a0 0x7f7c28143170 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.246+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 >> 172.21.15.33:0/666960128 conn(0x7f7c2806cff0 msgr2=0x7f7c280ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.247+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 shutdown_connections 2023-10-06T05:19:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.247+0000 7f7c2f59d700 1 -- 172.21.15.33:0/666960128 wait complete. 2023-10-06T05:19:26.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:19:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.462+0000 7f00881ff700 1 Processor -- start 2023-10-06T05:19:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.462+0000 7f00881ff700 1 -- start start 2023-10-06T05:19:26.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.463+0000 7f00881ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0080159b10 0x7f008006f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.463+0000 7f00881ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00800d1120 con 0x7f00800cabf0 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.463+0000 7f00881ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00800d1400 con 0x7f008006d140 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.463+0000 7f00881ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008006e920 con 0x7f0080159b10 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0085f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f00800cabf0 0x7f00800c7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59424/0 (socket says 172.21.15.33:59424) 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0085f9b700 1 -- 172.21.15.33:0/1891952328 learned_addr learned my addr 172.21.15.33:0/1891952328 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f008579a700 1 --2- 172.21.15.33:0/1891952328 >> v2:172.21.15.167:3300/0 conn(0x7f0080159b10 0x7f008006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390472043 0 0) 0x7f00800d1120 con 0x7f00800cabf0 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f006c003570 con 0x7f00800cabf0 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034343978 0 0) 0x7f00800d1400 con 0x7f008006d140 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00800d1120 con 0x7f008006d140 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944357145 0 0) 0x7f006c003570 con 0x7f00800cabf0 2023-10-06T05:19:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 >> v2:172.21.15.167:3300/0 conn(0x7f0080159b10 msgr2=0x7f008006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 --2- 172.21.15.33:0/1891952328 >> v2:172.21.15.167:3300/0 conn(0x7f0080159b10 0x7f008006f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 >> v1:172.21.15.167:6789/0 conn(0x7f008006d140 legacy=0x7f008006f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.464+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008006ec00 con 0x7f00800cabf0 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f007c0015f0 con 0x7f00800cabf0 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 >> v1:172.21.15.33:6789/0 conn(0x7f00800cabf0 legacy=0x7f00800c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006c003c40 0x7f006c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c004810 0x7f006c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c005210 0x7f006c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008006e920 con 0x7f006c003c40 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c00c1b0 con 0x7f006c005210 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c00c370 con 0x7f006c004810 2023-10-06T05:19:26.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0085f9b700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006c003c40 0x7f006c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f008679c700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c005210 0x7f006c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f008579a700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c004810 0x7f006c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0085f9b700 1 -- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c004810 msgr2=0x7f006c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0085f9b700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c004810 0x7f006c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0085f9b700 1 -- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c005210 msgr2=0x7f006c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0085f9b700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c005210 0x7f006c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f0085f9b700 1 -- 172.21.15.33:0/1891952328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c003570 con 0x7f006c003c40 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f008679c700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c005210 0x7f006c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.465+0000 7f008679c700 1 -- 172.21.15.33:0/1891952328 reap_dead start 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f0085f9b700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006c003c40 0x7f006c004030 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f007c00c1e0 tx=0x7f007c007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a131190237fff31c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c00cad0 con 0x7f006c003c40 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c00cc50 con 0x7f006c003c40 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f0084f99700 1 -- 172.21.15.33:0/1891952328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0070d0 con 0x7f006c003c40 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f00881ff700 1 -- 172.21.15.33:0/1891952328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006c003c40 msgr2=0x7f006c004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f00881ff700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006c003c40 0x7f006c004030 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f007c00c1e0 tx=0x7f007c007810 comp rx=0 tx=0).stop 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f00881ff700 1 -- 172.21.15.33:0/1891952328 shutdown_connections 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f00881ff700 1 --2- 172.21.15.33:0/1891952328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f006c003c40 0x7f006c004030 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.466+0000 7f00881ff700 1 -- 172.21.15.33:0/1891952328 >> 172.21.15.33:0/1891952328 conn(0x7f008006cd40 msgr2=0x7f00800d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.467+0000 7f00881ff700 1 -- 172.21.15.33:0/1891952328 shutdown_connections 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.467+0000 7f00881ff700 1 -- 172.21.15.33:0/1891952328 wait complete. 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.468+0000 7f00881ff700 1 Processor -- start 2023-10-06T05:19:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.468+0000 7f00881ff700 1 -- start start 2023-10-06T05:19:26.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f00881ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c003c40 0x7f008016cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f00881ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 0x7f00800c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f00881ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00800c9440 0x7f008015d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f00881ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800c6e90 con 0x7f00800b5ce0 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f00881ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800c4580 con 0x7f00800c9440 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f00881ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008016d500 con 0x7f006c003c40 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.469+0000 7f008579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 0x7f00800c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 0x7f00800c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36376/0 (socket says 172.21.15.33:36376) 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 -- 172.21.15.33:0/234649564 learned_addr learned my addr 172.21.15.33:0/234649564 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f0085f9b700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c003c40 0x7f008016cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008679c700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00800c9440 0x7f008015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 -- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c003c40 msgr2=0x7f008016cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c003c40 0x7f008016cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 -- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00800c9440 msgr2=0x7f008015d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00800c9440 0x7f008015d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 -- 172.21.15.33:0/234649564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008016d7a0 con 0x7f00800b5ce0 2023-10-06T05:19:26.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008679c700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00800c9440 0x7f008015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f008579a700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 0x7f00800c8f00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0070011b40 tx=0x7f00700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.470+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000fa70 con 0x7f00800b5ce0 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.471+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007000fbd0 con 0x7f00800b5ce0 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.471+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070014440 con 0x7f00800b5ce0 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.471+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008016da80 con 0x7f00800b5ce0 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.471+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00800c47d0 con 0x7f00800b5ce0 2023-10-06T05:19:26.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.472+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008004c380 con 0x7f00800b5ce0 2023-10-06T05:19:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.473+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0070014870 con 0x7f00800b5ce0 2023-10-06T05:19:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.473+0000 7f0075ffb700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.473+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0070018030 con 0x7f00800b5ce0 2023-10-06T05:19:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.474+0000 7f0085f9b700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.475+0000 7f0085f9b700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f007c00c1e0 tx=0x7f007c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.477+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00700522d0 con 0x7f00800b5ce0 2023-10-06T05:19:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.679+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f00800ca470 con 0x7f00800b5ce0 2023-10-06T05:19:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.684+0000 7f0075ffb700 1 -- 172.21.15.33:0/234649564 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00700220c0 con 0x7f00800b5ce0 2023-10-06T05:19:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 msgr2=0x7f00640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f007c00c1e0 tx=0x7f007c015000 comp rx=0 tx=0).stop 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 msgr2=0x7f00800c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 0x7f00800c8f00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0070011b40 tx=0x7f00700091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 shutdown_connections 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00800c9440 0x7f008015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f006c003c40 0x7f008016cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 --2- 172.21.15.33:0/234649564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00800b5ce0 0x7f00800c8f00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:26.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 >> 172.21.15.33:0/234649564 conn(0x7f008006cd40 msgr2=0x7f00800cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:26.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 shutdown_connections 2023-10-06T05:19:26.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:26.686+0000 7f00881ff700 1 -- 172.21.15.33:0/234649564 wait complete. 2023-10-06T05:19:26.705 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:19:26.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:19:26.789 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:19:26.879 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:19:26.880 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:19:26.880 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.065+0000 7f9be719d700 1 Processor -- start 2023-10-06T05:19:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.065+0000 7f9be719d700 1 -- start start 2023-10-06T05:19:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be0155140 0x7f9be0158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9be00699c0 0x7f9be0069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be00c8e40 con 0x7f9be00adf50 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be006a380 con 0x7f9be00699c0 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00c99e0 con 0x7f9be0155140 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9be00699c0 0x7f9be0069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9be00699c0 0x7f9be0069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36392/0 (socket says 172.21.15.33:36392) 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 -- 172.21.15.33:0/4010943059 learned_addr learned my addr 172.21.15.33:0/4010943059 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be619b700 1 --2- 172.21.15.33:0/4010943059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be0155140 0x7f9be0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 -- 172.21.15.33:0/4010943059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be0155140 msgr2=0x7f9be0158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 --2- 172.21.15.33:0/4010943059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be0155140 0x7f9be0158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 -- 172.21.15.33:0/4010943059 >> v1:172.21.15.167:6789/0 conn(0x7f9be00adf50 legacy=0x7f9be00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.067+0000 7f9be699c700 1 -- 172.21.15.33:0/4010943059 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00ae6c0 con 0x7f9be00699c0 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.068+0000 7f9be699c700 1 --2- 172.21.15.33:0/4010943059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be00699c0 0x7f9be0069e40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009f20 tx=0x7f9bd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f3e9722afa296a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.068+0000 7f9be5199700 1 -- 172.21.15.33:0/4010943059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800e070 con 0x7f9be00699c0 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.068+0000 7f9be5199700 1 -- 172.21.15.33:0/4010943059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd800ba80 con 0x7f9be00699c0 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.068+0000 7f9be5199700 1 -- 172.21.15.33:0/4010943059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80092e0 con 0x7f9be00699c0 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.068+0000 7f9be719d700 1 -- 172.21.15.33:0/4010943059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be00699c0 msgr2=0x7f9be0069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.068+0000 7f9be719d700 1 --2- 172.21.15.33:0/4010943059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be00699c0 0x7f9be0069e40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009f20 tx=0x7f9bd8009840 comp rx=0 tx=0).stop 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.069+0000 7f9be719d700 1 -- 172.21.15.33:0/4010943059 shutdown_connections 2023-10-06T05:19:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.069+0000 7f9be719d700 1 --2- 172.21.15.33:0/4010943059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be00699c0 0x7f9be0069e40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.069+0000 7f9be719d700 1 --2- 172.21.15.33:0/4010943059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be0155140 0x7f9be0158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.069+0000 7f9be719d700 1 -- 172.21.15.33:0/4010943059 >> 172.21.15.33:0/4010943059 conn(0x7f9be006d030 msgr2=0x7f9be006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.069+0000 7f9be719d700 1 -- 172.21.15.33:0/4010943059 shutdown_connections 2023-10-06T05:19:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.069+0000 7f9be719d700 1 -- 172.21.15.33:0/4010943059 wait complete. 2023-10-06T05:19:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.070+0000 7f9be719d700 1 Processor -- start 2023-10-06T05:19:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.070+0000 7f9be719d700 1 -- start start 2023-10-06T05:19:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be00699c0 0x7f9be00c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be00adf50 0x7f9be00c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 0x7f9be00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00c6700 con 0x7f9be0155140 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00c5ee0 con 0x7f9be00699c0 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00c6140 con 0x7f9be00adf50 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 0x7f9be00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 0x7f9be00caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36404/0 (socket says 172.21.15.33:36404) 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.071+0000 7f9be699c700 1 -- 172.21.15.33:0/766189080 learned_addr learned my addr 172.21.15.33:0/766189080 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be699c700 1 -- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be00adf50 msgr2=0x7f9be00c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be699c700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be00adf50 0x7f9be00c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be699c700 1 -- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be00699c0 msgr2=0x7f9be00c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be699c700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be00699c0 0x7f9be00c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be699c700 1 -- 172.21.15.33:0/766189080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8009ae0 con 0x7f9be0155140 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be699c700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 0x7f9be00caae0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9be0063610 tx=0x7f9bd8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8008e80 con 0x7f9be0155140 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80092e0 con 0x7f9be0155140 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8025690 con 0x7f9be0155140 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.072+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be00c6420 con 0x7f9be0155140 2023-10-06T05:19:27.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.073+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be00a9ca0 con 0x7f9be0155140 2023-10-06T05:19:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.074+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be004c380 con 0x7f9be0155140 2023-10-06T05:19:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.074+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bd8003040 con 0x7f9be0155140 2023-10-06T05:19:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.075+0000 7f9bd67fc700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9bc4066f30 0x7f9bc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.075+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9bd80861b0 con 0x7f9be0155140 2023-10-06T05:19:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.076+0000 7f9be619b700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9bc4066f30 0x7f9bc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.076+0000 7f9be619b700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9bc4066f30 0x7f9bc40693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009a60 tx=0x7f9bdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.079+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bd80535a0 con 0x7f9be0155140 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.323+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9be006e740 con 0x7f9be0155140 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.324+0000 7f9bd67fc700 1 -- 172.21.15.33:0/766189080 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f9bd8020de0 con 0x7f9be0155140 2023-10-06T05:19:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:27.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/533903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":533903849},{"type":"v1","addr":"172.21.15.167:6835","nonce":533903849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3289996434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3289996434},{"type":"v1","addr":"172.21.15.167:6837","nonce":3289996434}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1187554690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1187554690},{"type":"v1","addr":"172.21.15.33:6835","nonce":1187554690}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3492692814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3492692814},{"type":"v1","addr":"172.21.15.33:6834","nonce":3492692814}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9bc4066f30 msgr2=0x7f9bc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9bc4066f30 0x7f9bc40693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009a60 tx=0x7f9bdc008040 comp rx=0 tx=0).stop 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 msgr2=0x7f9be00caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 0x7f9be00caae0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9be0063610 tx=0x7f9bd8018000 comp rx=0 tx=0).stop 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 shutdown_connections 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9bc4066f30 0x7f9bc40693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9be0155140 0x7f9be00caae0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9be00699c0 0x7f9be00c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.326+0000 7f9be719d700 1 --2- 172.21.15.33:0/766189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9be00adf50 0x7f9be00c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.327+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 >> 172.21.15.33:0/766189080 conn(0x7f9be006d030 msgr2=0x7f9be01541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.327+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 shutdown_connections 2023-10-06T05:19:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.327+0000 7f9be719d700 1 -- 172.21.15.33:0/766189080 wait complete. 2023-10-06T05:19:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2023-10-06T05:19:27.347 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:19:27.347 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:19:27.347 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:19:27.347 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:19:27.347 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:19:27.347 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:19:27.348 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:19:27.348 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:19:27.348 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:19:27.348 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:19:27.348 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:19:27.348 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:19:27.348 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:19:27.348 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:19:27.349 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:19:27.349 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:19:27.361 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:19:27.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:19:27.362 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:19:27.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:19:27.366 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:19:27.366 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:19:27.367 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:19:27.367 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:19:27.406 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:19:27.407 DEBUG:teuthology.parallel:result is None 2023-10-06T05:19:27.411 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:19:27.411 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:19:27.412 DEBUG:teuthology.parallel:result is None 2023-10-06T05:19:27.412 DEBUG:teuthology.parallel:result is None 2023-10-06T05:19:27.413 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:19:27.414 DEBUG:teuthology.parallel:result is None 2023-10-06T05:19:27.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:19:27.544 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:19:27.551 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:19:27.551 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:19:27.551 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:19:27.552 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:19:27.551+0000 7fe06cdfd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T05:19:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.615+0000 7f2c2099d700 1 Processor -- start 2023-10-06T05:19:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.615+0000 7f2c2099d700 1 -- start start 2023-10-06T05:19:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.616+0000 7f2c2099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.616+0000 7f2c2099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1c06bd10 con 0x7f2c1c156880 2023-10-06T05:19:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.616+0000 7f2c2099d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1c06a820 con 0x7f2c1c0c8090 2023-10-06T05:19:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.616+0000 7f2c2099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0cc290 con 0x7f2c1c159e10 2023-10-06T05:19:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2c1c156880 0x7f2c1c0c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59446/0 (socket says 172.21.15.33:59446) 2023-10-06T05:19:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1b7fe700 1 -- 172.21.15.33:0/687933110 learned_addr learned my addr 172.21.15.33:0/687933110 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1bfff700 1 --2- 172.21.15.33:0/687933110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576189658 0 0) 0x7f2c1c06bd10 con 0x7f2c1c156880 2023-10-06T05:19:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c04003570 con 0x7f2c1c156880 2023-10-06T05:19:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182903168 0 0) 0x7f2c1c06a820 con 0x7f2c1c0c8090 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c1c06bd10 con 0x7f2c1c0c8090 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1bfff700 1 -- 172.21.15.33:0/687933110 >> v1:172.21.15.167:6789/0 conn(0x7f2c1c0c8090 legacy=0x7f2c1c156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1bfff700 1 -- 172.21.15.33:0/687933110 >> v1:172.21.15.33:6789/0 conn(0x7f2c1c156880 legacy=0x7f2c1c0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2604266114 0 0) 0x7f2c04003570 con 0x7f2c1c156880 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.617+0000 7f2c1bfff700 1 -- 172.21.15.33:0/687933110 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c15a2f0 con 0x7f2c1c159e10 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c1bfff700 1 --2- 172.21.15.33:0/687933110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c157360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009ec0 tx=0x7f2c14009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb19f224dc76c1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e070 con 0x7f2c1c159e10 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1400b870 con 0x7f2c1c159e10 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c1a7fc700 1 -- 172.21.15.33:0/687933110 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c001470 con 0x7f2c1c159e10 2023-10-06T05:19:27.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c2099d700 1 -- 172.21.15.33:0/687933110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 msgr2=0x7f2c1c157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c2099d700 1 --2- 172.21.15.33:0/687933110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c157360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009ec0 tx=0x7f2c14009780 comp rx=0 tx=0).stop 2023-10-06T05:19:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c2099d700 1 -- 172.21.15.33:0/687933110 shutdown_connections 2023-10-06T05:19:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c2099d700 1 --2- 172.21.15.33:0/687933110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c157360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.618+0000 7f2c2099d700 1 -- 172.21.15.33:0/687933110 >> 172.21.15.33:0/687933110 conn(0x7f2c1c06c560 msgr2=0x7f2c1c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.619+0000 7f2c2099d700 1 -- 172.21.15.33:0/687933110 shutdown_connections 2023-10-06T05:19:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.619+0000 7f2c2099d700 1 -- 172.21.15.33:0/687933110 wait complete. 2023-10-06T05:19:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.620+0000 7f2c2099d700 1 Processor -- start 2023-10-06T05:19:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.620+0000 7f2c2099d700 1 -- start start 2023-10-06T05:19:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c2099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c0c8090 0x7f2c1c16c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c2099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 0x7f2c1c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c2099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c2099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0cab80 con 0x7f2c1c156880 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c2099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c16ca80 con 0x7f2c1c0c8090 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c2099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0c9f70 con 0x7f2c1c159e10 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 0x7f2c1c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 0x7f2c1c0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36434/0 (socket says 172.21.15.33:36434) 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.621+0000 7f2c1affd700 1 -- 172.21.15.33:0/3547013191 learned_addr learned my addr 172.21.15.33:0/3547013191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1bfff700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1b7fe700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c0c8090 0x7f2c1c16c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1affd700 1 -- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 msgr2=0x7f2c1c14ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1affd700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c14ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1affd700 1 -- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c0c8090 msgr2=0x7f2c1c16c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1affd700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c0c8090 0x7f2c1c16c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1affd700 1 -- 172.21.15.33:0/3547013191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009a20 con 0x7f2c1c156880 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c1affd700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 0x7f2c1c0c9590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000c7c0 tx=0x7f2c10007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10010510 con 0x7f2c1c156880 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c10010690 con 0x7f2c1c156880 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10010a50 con 0x7f2c1c156880 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.622+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c0ca250 con 0x7f2c1c156880 2023-10-06T05:19:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.623+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1c0ca810 con 0x7f2c1c156880 2023-10-06T05:19:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.623+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1c04c380 con 0x7f2c1c156880 2023-10-06T05:19:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.624+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c1001b400 con 0x7f2c1c156880 2023-10-06T05:19:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.626+0000 7f2c037fe700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c08066f30 0x7f2c080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.626+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2c10083c30 con 0x7f2c1c156880 2023-10-06T05:19:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.626+0000 7f2c1b7fe700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c08066f30 0x7f2c080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.627+0000 7f2c1b7fe700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c08066f30 0x7f2c080693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0015a0 tx=0x7f2c0c000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.630+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c1004b5a0 con 0x7f2c1c156880 2023-10-06T05:19:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:27.846+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2c1c0c7760 con 0x7f2c1c156880 2023-10-06T05:19:28.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.254+0000 7f2c037fe700 1 -- 172.21.15.33:0/3547013191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2c10016080 con 0x7f2c1c156880 2023-10-06T05:19:28.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c08066f30 msgr2=0x7f2c080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c08066f30 0x7f2c080693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0015a0 tx=0x7f2c0c000bc0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 msgr2=0x7f2c1c0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 0x7f2c1c0c9590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000c7c0 tx=0x7f2c10007db0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 shutdown_connections 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c08066f30 0x7f2c080693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c1c159e10 0x7f2c1c14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c0c8090 0x7f2c1c16c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 --2- 172.21.15.33:0/3547013191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c156880 0x7f2c1c0c9590 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 >> 172.21.15.33:0/3547013191 conn(0x7f2c1c06c560 msgr2=0x7f2c1c1504d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:28.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 shutdown_connections 2023-10-06T05:19:28.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.258+0000 7f2c2099d700 1 -- 172.21.15.33:0/3547013191 wait complete. 2023-10-06T05:19:28.261 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:19:28.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.466+0000 7f406dd9d700 1 Processor -- start 2023-10-06T05:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.466+0000 7f406dd9d700 1 -- start start 2023-10-06T05:19:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.467+0000 7f406dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680cf3b0 0x7f40681594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.467+0000 7f406dd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680cbd80 con 0x7f40680ccfc0 2023-10-06T05:19:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.467+0000 7f406dd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680af5f0 con 0x7f40680af0a0 2023-10-06T05:19:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.467+0000 7f406dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680af8d0 con 0x7f40680cf3b0 2023-10-06T05:19:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f406cd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f40680ccfc0 0x7f40680cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59456/0 (socket says 172.21.15.33:59456) 2023-10-06T05:19:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f406cd9b700 1 -- 172.21.15.33:0/2960570840 learned_addr learned my addr 172.21.15.33:0/2960570840 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f4067fff700 1 --2- 172.21.15.33:0/2960570840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680cf3b0 0x7f40681594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209786263 0 0) 0x7f40680cbd80 con 0x7f40680ccfc0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4058003570 con 0x7f40680ccfc0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325597079 0 0) 0x7f40680af5f0 con 0x7f40680af0a0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40680cbd80 con 0x7f40680af0a0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f4067fff700 1 -- 172.21.15.33:0/2960570840 >> v1:172.21.15.167:6789/0 conn(0x7f40680af0a0 legacy=0x7f40680b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f4067fff700 1 -- 172.21.15.33:0/2960570840 >> v1:172.21.15.33:6789/0 conn(0x7f40680ccfc0 legacy=0x7f40680cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.468+0000 7f4067fff700 1 -- 172.21.15.33:0/2960570840 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680afbb0 con 0x7f40680cf3b0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.469+0000 7f4067fff700 1 --2- 172.21.15.33:0/2960570840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680cf3b0 0x7f40681594a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4054009910 tx=0x7f40540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658184eb9e769ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.469+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054011630 con 0x7f40680cf3b0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.469+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054011790 con 0x7f40680cf3b0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.469+0000 7f40677fe700 1 -- 172.21.15.33:0/2960570840 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054011b30 con 0x7f40680cf3b0 2023-10-06T05:19:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.469+0000 7f406dd9d700 1 -- 172.21.15.33:0/2960570840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680cf3b0 msgr2=0x7f40681594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.469+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2960570840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680cf3b0 0x7f40681594a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4054009910 tx=0x7f40540091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.470+0000 7f406dd9d700 1 -- 172.21.15.33:0/2960570840 shutdown_connections 2023-10-06T05:19:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.470+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2960570840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680cf3b0 0x7f40681594a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.470+0000 7f406dd9d700 1 -- 172.21.15.33:0/2960570840 >> 172.21.15.33:0/2960570840 conn(0x7f406806cff0 msgr2=0x7f406806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.470+0000 7f406dd9d700 1 -- 172.21.15.33:0/2960570840 shutdown_connections 2023-10-06T05:19:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.470+0000 7f406dd9d700 1 -- 172.21.15.33:0/2960570840 wait complete. 2023-10-06T05:19:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.471+0000 7f406dd9d700 1 Processor -- start 2023-10-06T05:19:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.471+0000 7f406dd9d700 1 -- start start 2023-10-06T05:19:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680af0a0 0x7f40681400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 0x7f4068152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4068151160 0x7f40680b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680c7640 con 0x7f40680ccfc0 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068151670 con 0x7f4068151160 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4068151950 con 0x7f40680af0a0 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f4067fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 0x7f4068152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f4067fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 0x7f4068152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36448/0 (socket says 172.21.15.33:36448) 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f4067fff700 1 -- 172.21.15.33:0/2518148349 learned_addr learned my addr 172.21.15.33:0/2518148349 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406cd9b700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680af0a0 0x7f40681400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.472+0000 7f406d59c700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4068151160 0x7f40680b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4067fff700 1 -- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680af0a0 msgr2=0x7f40681400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4067fff700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680af0a0 0x7f40681400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4067fff700 1 -- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4068151160 msgr2=0x7f40680b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4067fff700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4068151160 0x7f40680b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4067fff700 1 -- 172.21.15.33:0/2518148349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054009420 con 0x7f40680ccfc0 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4067fff700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 0x7f4068152950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4054013000 tx=0x7f4054017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405401f070 con 0x7f40680ccfc0 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054002f70 con 0x7f40680ccfc0 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054011900 con 0x7f40680ccfc0 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.473+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4068151c30 con 0x7f40680ccfc0 2023-10-06T05:19:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.474+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4068140600 con 0x7f40680ccfc0 2023-10-06T05:19:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.475+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40680c77b0 con 0x7f40680ccfc0 2023-10-06T05:19:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.475+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4054023070 con 0x7f40680ccfc0 2023-10-06T05:19:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.477+0000 7f4064ff9700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f404c066f60 0x7f404c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.477+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4054085240 con 0x7f40680ccfc0 2023-10-06T05:19:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.477+0000 7f406cd9b700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f404c066f60 0x7f404c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.478+0000 7f406cd9b700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f404c066f60 0x7f404c069410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f405c007fd0 tx=0x7f405c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:28.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.481+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4054052600 con 0x7f40680ccfc0 2023-10-06T05:19:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.770+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f406804c380 con 0x7f40680ccfc0 2023-10-06T05:19:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.770+0000 7f4064ff9700 1 -- 172.21.15.33:0/2518148349 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4054055c20 con 0x7f40680ccfc0 2023-10-06T05:19:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:28.772 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:19:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:19:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:19:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:19:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:19:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:19:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:19:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f404c066f60 msgr2=0x7f404c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f404c066f60 0x7f404c069410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f405c007fd0 tx=0x7f405c009040 comp rx=0 tx=0).stop 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 msgr2=0x7f4068152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 0x7f4068152950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4054013000 tx=0x7f4054017000 comp rx=0 tx=0).stop 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 shutdown_connections 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f404c066f60 0x7f404c069410 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4068151160 0x7f40680b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40680af0a0 0x7f40681400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 --2- 172.21.15.33:0/2518148349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40680ccfc0 0x7f4068152950 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 >> 172.21.15.33:0/2518148349 conn(0x7f406806cff0 msgr2=0x7f4068155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:28.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 shutdown_connections 2023-10-06T05:19:28.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.773+0000 7f406dd9d700 1 -- 172.21.15.33:0/2518148349 wait complete. 2023-10-06T05:19:28.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.978+0000 7f357a79d700 1 Processor -- start 2023-10-06T05:19:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.979+0000 7f357a79d700 1 -- start start 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f357a79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3574158700 0x7f35740adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f357a79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35740699c0 0x7f3574069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f357a79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3574158ba0 con 0x7f3574155140 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f357a79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357406a380 con 0x7f3574158700 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f357a79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c8e40 con 0x7f35740699c0 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f3578f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3574158700 0x7f35740adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f3578f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3574158700 0x7f35740adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36458/0 (socket says 172.21.15.33:36458) 2023-10-06T05:19:28.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f3578f9a700 1 -- 172.21.15.33:0/2560590789 learned_addr learned my addr 172.21.15.33:0/2560590789 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:28.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.980+0000 7f3579f9c700 1 --2- 172.21.15.33:0/2560590789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35740699c0 0x7f3574069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f3578f9a700 1 -- 172.21.15.33:0/2560590789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35740699c0 msgr2=0x7f3574069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f3578f9a700 1 --2- 172.21.15.33:0/2560590789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35740699c0 0x7f3574069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f3578f9a700 1 -- 172.21.15.33:0/2560590789 >> v1:172.21.15.33:6789/0 conn(0x7f3574155140 legacy=0x7f35740c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f3578f9a700 1 -- 172.21.15.33:0/2560590789 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740ae490 con 0x7f3574158700 2023-10-06T05:19:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f3578f9a700 1 --2- 172.21.15.33:0/2560590789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3574158700 0x7f35740adf50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f35640099e0 tx=0x7f3564009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764a8f7fd8fdbc95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f356bfff700 1 -- 172.21.15.33:0/2560590789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400b820 con 0x7f3574158700 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f356bfff700 1 -- 172.21.15.33:0/2560590789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356400b980 con 0x7f3574158700 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.981+0000 7f356bfff700 1 -- 172.21.15.33:0/2560590789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400bd20 con 0x7f3574158700 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 -- 172.21.15.33:0/2560590789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3574158700 msgr2=0x7f35740adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 --2- 172.21.15.33:0/2560590789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3574158700 0x7f35740adf50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f35640099e0 tx=0x7f3564009250 comp rx=0 tx=0).stop 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 -- 172.21.15.33:0/2560590789 shutdown_connections 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 --2- 172.21.15.33:0/2560590789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35740699c0 0x7f3574069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 --2- 172.21.15.33:0/2560590789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3574158700 0x7f35740adf50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 -- 172.21.15.33:0/2560590789 >> 172.21.15.33:0/2560590789 conn(0x7f357406d030 msgr2=0x7f357406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 -- 172.21.15.33:0/2560590789 shutdown_connections 2023-10-06T05:19:28.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.982+0000 7f357a79d700 1 -- 172.21.15.33:0/2560590789 wait complete. 2023-10-06T05:19:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.983+0000 7f357a79d700 1 Processor -- start 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 -- start start 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 0x7f35740c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574155140 0x7f35740acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3574158700 0x7f35740b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357415c800 con 0x7f35740699c0 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357415c370 con 0x7f3574155140 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.984+0000 7f357a79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c2230 con 0x7f3574158700 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 0x7f35740c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 0x7f35740c1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36464/0 (socket says 172.21.15.33:36464) 2023-10-06T05:19:28.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 -- 172.21.15.33:0/3860959801 learned_addr learned my addr 172.21.15.33:0/3860959801 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f3579f9c700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3574158700 0x7f35740b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f3578f9a700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574155140 0x7f35740acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 -- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3574158700 msgr2=0x7f35740b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3574158700 0x7f35740b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 -- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574155140 msgr2=0x7f35740acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574155140 0x7f35740acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 -- 172.21.15.33:0/3860959801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640094f0 con 0x7f35740699c0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.985+0000 7f357979b700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 0x7f35740c1cf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f357000c5c0 tx=0x7f357000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.986+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570011590 con 0x7f35740699c0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.986+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3570011710 con 0x7f35740699c0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.986+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570011ad0 con 0x7f35740699c0 2023-10-06T05:19:28.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.986+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357415b760 con 0x7f35740699c0 2023-10-06T05:19:28.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.986+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357415bd20 con 0x7f35740699c0 2023-10-06T05:19:28.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.987+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357415c650 con 0x7f35740699c0 2023-10-06T05:19:28.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.988+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f357001b3b0 con 0x7f35740699c0 2023-10-06T05:19:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.988+0000 7f35697fa700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3558066f60 0x7f3558069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.988+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f35700839c0 con 0x7f35740699c0 2023-10-06T05:19:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.989+0000 7f3578f9a700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3558066f60 0x7f3558069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.990+0000 7f3578f9a700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3558066f60 0x7f3558069410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35640099e0 tx=0x7f3564017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:28.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:28.993+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3570050d80 con 0x7f35740699c0 2023-10-06T05:19:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.199+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f357404c380 con 0x7f35740699c0 2023-10-06T05:19:29.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.200+0000 7f35697fa700 1 -- 172.21.15.33:0/3860959801 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13651 (secure 0 0 0) 0x7f3570017020 con 0x7f35740699c0 2023-10-06T05:19:29.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:29.202 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":72,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:19:28.260632+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3558066f60 msgr2=0x7f3558069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3558066f60 0x7f3558069410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35640099e0 tx=0x7f3564017000 comp rx=0 tx=0).stop 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 msgr2=0x7f35740c1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 0x7f35740c1cf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f357000c5c0 tx=0x7f357000c980 comp rx=0 tx=0).stop 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 shutdown_connections 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3558066f60 0x7f3558069410 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3574158700 0x7f35740b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35740699c0 0x7f35740c1cf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 --2- 172.21.15.33:0/3860959801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574155140 0x7f35740acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.202+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 >> 172.21.15.33:0/3860959801 conn(0x7f357406d030 msgr2=0x7f35740c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.203+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 shutdown_connections 2023-10-06T05:19:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.203+0000 7f357a79d700 1 -- 172.21.15.33:0/3860959801 wait complete. 2023-10-06T05:19:29.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:19:29.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:19:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.411+0000 7f728319d700 1 Processor -- start 2023-10-06T05:19:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.411+0000 7f728319d700 1 -- start start 2023-10-06T05:19:29.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.413+0000 7f728319d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f727c1595d0 0x7f727c0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.413+0000 7f728319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c0aa7d0 con 0x7f727c0ac180 2023-10-06T05:19:29.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.413+0000 7f728319d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c0a9bc0 con 0x7f727c0ad460 2023-10-06T05:19:29.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.413+0000 7f728319d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c0a9ea0 con 0x7f727c1595d0 2023-10-06T05:19:29.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.413+0000 7f728199a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f727c0ac180 0x7f727c0acd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59482/0 (socket says 172.21.15.33:59482) 2023-10-06T05:19:29.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.413+0000 7f728199a700 1 -- 172.21.15.33:0/2205408462 learned_addr learned my addr 172.21.15.33:0/2205408462 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f728219b700 1 --2- 172.21.15.33:0/2205408462 >> v2:172.21.15.167:3300/0 conn(0x7f727c1595d0 0x7f727c0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131673065 0 0) 0x7f727c0aa7d0 con 0x7f727c0ac180 2023-10-06T05:19:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7268003570 con 0x7f727c0ac180 2023-10-06T05:19:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4086150417 0 0) 0x7f7268003570 con 0x7f727c0ac180 2023-10-06T05:19:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 >> v2:172.21.15.167:3300/0 conn(0x7f727c1595d0 msgr2=0x7f727c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:29.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 --2- 172.21.15.33:0/2205408462 >> v2:172.21.15.167:3300/0 conn(0x7f727c1595d0 0x7f727c0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 >> v1:172.21.15.167:6789/0 conn(0x7f727c0ad460 legacy=0x7f727c0af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c0aa180 con 0x7f727c0ac180 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f726c001470 con 0x7f727c0ac180 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 >> v1:172.21.15.33:6789/0 conn(0x7f727c0ac180 legacy=0x7f727c0acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f7268003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7268004810 0x7f7268004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72680051c0 0x7f726800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0a9ea0 con 0x7f7268003a60 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726800c0e0 con 0x7f72680051c0 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726800c2a0 con 0x7f7268004810 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.414+0000 7f728219b700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f7268003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728199a700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7268004810 0x7f7268004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728299c700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72680051c0 0x7f726800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728219b700 1 -- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7268004810 msgr2=0x7f7268004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728219b700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7268004810 0x7f7268004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728219b700 1 -- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72680051c0 msgr2=0x7f726800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728219b700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72680051c0 0x7f726800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728219b700 1 -- 172.21.15.33:0/2205408462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268003570 con 0x7f7268003a60 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728299c700 1 -- 172.21.15.33:0/2205408462 reap_dead start 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.415+0000 7f728219b700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f7268003e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7278009c00 tx=0x7f72780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb6669bb93e9ce5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800e710 con 0x7f7268003a60 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800e870 con 0x7f7268003a60 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f7281199700 1 -- 172.21.15.33:0/2205408462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800ec30 con 0x7f7268003a60 2023-10-06T05:19:29.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 -- 172.21.15.33:0/2205408462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 msgr2=0x7f7268003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f7268003e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7278009c00 tx=0x7f72780091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 -- 172.21.15.33:0/2205408462 shutdown_connections 2023-10-06T05:19:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 --2- 172.21.15.33:0/2205408462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f7268003e30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 -- 172.21.15.33:0/2205408462 >> 172.21.15.33:0/2205408462 conn(0x7f727c06d030 msgr2=0x7f727c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 -- 172.21.15.33:0/2205408462 shutdown_connections 2023-10-06T05:19:29.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.416+0000 7f728319d700 1 -- 172.21.15.33:0/2205408462 wait complete. 2023-10-06T05:19:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.418+0000 7f728319d700 1 Processor -- start 2023-10-06T05:19:29.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.418+0000 7f728319d700 1 -- start start 2023-10-06T05:19:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f727c146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727c1437d0 0x7f727c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c15bd30 0x7f727c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c15a100 con 0x7f7268003a60 2023-10-06T05:19:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c15c1a0 con 0x7f727c15bd30 2023-10-06T05:19:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0b1950 con 0x7f727c1437d0 2023-10-06T05:19:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728219b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f727c146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728219b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f727c146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36490/0 (socket says 172.21.15.33:36490) 2023-10-06T05:19:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728219b700 1 -- 172.21.15.33:0/406957064 learned_addr learned my addr 172.21.15.33:0/406957064 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728199a700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727c1437d0 0x7f727c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.419+0000 7f728299c700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c15bd30 0x7f727c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728219b700 1 -- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727c1437d0 msgr2=0x7f727c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728219b700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727c1437d0 0x7f727c15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728219b700 1 -- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c15bd30 msgr2=0x7f727c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728219b700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c15bd30 0x7f727c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728219b700 1 -- 172.21.15.33:0/406957064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c1467a0 con 0x7f7268003a60 2023-10-06T05:19:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728219b700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f727c146260 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f726800c0b0 tx=0x7f727801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278022070 con 0x7f7268003a60 2023-10-06T05:19:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800e9e0 con 0x7f7268003a60 2023-10-06T05:19:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727801c450 con 0x7f7268003a60 2023-10-06T05:19:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c0b0ab0 con 0x7f7268003a60 2023-10-06T05:19:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.420+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c159a40 con 0x7f7268003a60 2023-10-06T05:19:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.422+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c04c380 con 0x7f7268003a60 2023-10-06T05:19:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.422+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f727801c880 con 0x7f7268003a60 2023-10-06T05:19:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.423+0000 7f72727fc700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7260066f30 0x7f72600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.423+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7278085ae0 con 0x7f7268003a60 2023-10-06T05:19:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.424+0000 7f728199a700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7260066f30 0x7f72600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.424+0000 7f728199a700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7260066f30 0x7f72600693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f727c15d7d0 tx=0x7f726c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.427+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7278052f50 con 0x7f7268003a60 2023-10-06T05:19:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.629+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f727c0aa460 con 0x7f7268003a60 2023-10-06T05:19:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.630+0000 7f72727fc700 1 -- 172.21.15.33:0/406957064 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72780170a0 con 0x7f7268003a60 2023-10-06T05:19:29.631 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7260066f30 msgr2=0x7f72600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7260066f30 0x7f72600693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f727c15d7d0 tx=0x7f726c009040 comp rx=0 tx=0).stop 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 msgr2=0x7f727c146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f727c146260 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f726800c0b0 tx=0x7f727801a000 comp rx=0 tx=0).stop 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 shutdown_connections 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7260066f30 0x7f72600693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c15bd30 0x7f727c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7268003a60 0x7f727c146260 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 --2- 172.21.15.33:0/406957064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727c1437d0 0x7f727c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 >> 172.21.15.33:0/406957064 conn(0x7f727c06d030 msgr2=0x7f727c0ab770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:29.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 shutdown_connections 2023-10-06T05:19:29.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.632+0000 7f728319d700 1 -- 172.21.15.33:0/406957064 wait complete. 2023-10-06T05:19:29.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:19:29.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.840+0000 7f1bf119d700 1 Processor -- start 2023-10-06T05:19:29.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.841+0000 7f1bf119d700 1 -- start start 2023-10-06T05:19:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1bec154f20 0x7f1bec0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec1552f0 0x7f1bec0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf119d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec1586a0 con 0x7f1bec0b11e0 2023-10-06T05:19:29.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec06bdf0 con 0x7f1bec154f20 2023-10-06T05:19:29.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec157ab0 con 0x7f1bec1552f0 2023-10-06T05:19:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1beb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec1552f0 0x7f1bec0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf099c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1bec0b11e0 0x7f1bec15b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:41266/0 (socket says 172.21.15.33:41266) 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bf099c700 1 -- 172.21.15.33:0/744889498 learned_addr learned my addr 172.21.15.33:0/744889498 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.842+0000 7f1bebfff700 1 --2- 172.21.15.33:0/744889498 >> v2:172.21.15.167:3300/0 conn(0x7f1bec154f20 0x7f1bec0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beaffd700 1 -- 172.21.15.33:0/744889498 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272068695 0 0) 0x7f1bec1586a0 con 0x7f1bec0b11e0 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beb7fe700 1 -- 172.21.15.33:0/744889498 >> v2:172.21.15.167:3300/0 conn(0x7f1bec154f20 msgr2=0x7f1bec0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beb7fe700 1 --2- 172.21.15.33:0/744889498 >> v2:172.21.15.167:3300/0 conn(0x7f1bec154f20 0x7f1bec0ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beb7fe700 1 -- 172.21.15.33:0/744889498 >> v1:172.21.15.167:6789/0 conn(0x7f1bec0b11e0 legacy=0x7f1bec15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beb7fe700 1 -- 172.21.15.33:0/744889498 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec157d70 con 0x7f1bec1552f0 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1bebfff700 1 --2- 172.21.15.33:0/744889498 >> v2:172.21.15.167:3300/0 conn(0x7f1bec154f20 0x7f1bec0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beb7fe700 1 --2- 172.21.15.33:0/744889498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec1552f0 0x7f1bec0cc7e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098e0 tx=0x7f1be00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1ab9e522ddae7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beaffd700 1 -- 172.21.15.33:0/744889498 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000b640 con 0x7f1bec1552f0 2023-10-06T05:19:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beaffd700 1 -- 172.21.15.33:0/744889498 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be000b7a0 con 0x7f1bec1552f0 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.843+0000 7f1beaffd700 1 -- 172.21.15.33:0/744889498 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000bb40 con 0x7f1bec1552f0 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 -- 172.21.15.33:0/744889498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec1552f0 msgr2=0x7f1bec0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 --2- 172.21.15.33:0/744889498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec1552f0 0x7f1bec0cc7e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098e0 tx=0x7f1be00091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 -- 172.21.15.33:0/744889498 shutdown_connections 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 --2- 172.21.15.33:0/744889498 >> v2:172.21.15.167:3300/0 conn(0x7f1bec154f20 0x7f1bec0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 --2- 172.21.15.33:0/744889498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec1552f0 0x7f1bec0cc7e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 -- 172.21.15.33:0/744889498 >> 172.21.15.33:0/744889498 conn(0x7f1bec06d030 msgr2=0x7f1bec0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 -- 172.21.15.33:0/744889498 shutdown_connections 2023-10-06T05:19:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.844+0000 7f1bf119d700 1 -- 172.21.15.33:0/744889498 wait complete. 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.845+0000 7f1bf119d700 1 Processor -- start 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.846+0000 7f1bf119d700 1 -- start start 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.846+0000 7f1bf119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec0b11e0 0x7f1bec06ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.846+0000 7f1bf119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bec154f20 0x7f1bec1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.846+0000 7f1bf119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 0x7f1bec148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0c2230 con 0x7f1bec157380 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec156ef0 con 0x7f1bec154f20 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1562e0 con 0x7f1bec0b11e0 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 0x7f1bec148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 0x7f1bec148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36498/0 (socket says 172.21.15.33:36498) 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 -- 172.21.15.33:0/4211780393 learned_addr learned my addr 172.21.15.33:0/4211780393 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1beb7fe700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bec154f20 0x7f1bec1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bebfff700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec0b11e0 0x7f1bec06ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 -- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec0b11e0 msgr2=0x7f1bec06ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec0b11e0 0x7f1bec06ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 -- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bec154f20 msgr2=0x7f1bec1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bec154f20 0x7f1bec1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 -- 172.21.15.33:0/4211780393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc009420 con 0x7f1bec157380 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1beb7fe700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bec154f20 0x7f1bec1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bebfff700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec0b11e0 0x7f1bec06ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.847+0000 7f1bf099c700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 0x7f1bec148f00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1be400cbf0 tx=0x7f1be40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:29.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.848+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400a9c0 con 0x7f1bec157380 2023-10-06T05:19:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.848+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be400ab20 con 0x7f1bec157380 2023-10-06T05:19:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.848+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0009420 con 0x7f1bec157380 2023-10-06T05:19:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.848+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be40070f0 con 0x7f1bec157380 2023-10-06T05:19:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.848+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1568a0 con 0x7f1bec157380 2023-10-06T05:19:29.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.849+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec157060 con 0x7f1bec157380 2023-10-06T05:19:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.850+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1be4007520 con 0x7f1bec157380 2023-10-06T05:19:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.851+0000 7f1bd3fff700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1bd8066f30 0x7f1bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.851+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1be40847b0 con 0x7f1bec157380 2023-10-06T05:19:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.852+0000 7f1bebfff700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1bd8066f30 0x7f1bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.852+0000 7f1bebfff700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1bd8066f30 0x7f1bd80693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0093f0 tx=0x7f1bdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:29.855+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1be4051bf0 con 0x7f1bec157380 2023-10-06T05:19:30.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.057+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1bec04c380 con 0x7f1bec157380 2023-10-06T05:19:30.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.058+0000 7f1bd3fff700 1 -- 172.21.15.33:0/4211780393 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1be4055210 con 0x7f1bec157380 2023-10-06T05:19:30.060 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1bd8066f30 msgr2=0x7f1bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1bd8066f30 0x7f1bd80693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0093f0 tx=0x7f1bdc008040 comp rx=0 tx=0).stop 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 msgr2=0x7f1bec148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 0x7f1bec148f00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1be400cbf0 tx=0x7f1be40019b0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 shutdown_connections 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1bd8066f30 0x7f1bd80693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bec157380 0x7f1bec148f00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bec0b11e0 0x7f1bec06ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.060+0000 7f1bf119d700 1 --2- 172.21.15.33:0/4211780393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bec154f20 0x7f1bec1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.061+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 >> 172.21.15.33:0/4211780393 conn(0x7f1bec06d030 msgr2=0x7f1bec0ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:30.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.061+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 shutdown_connections 2023-10-06T05:19:30.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.061+0000 7f1bf119d700 1 -- 172.21.15.33:0/4211780393 wait complete. 2023-10-06T05:19:30.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:19:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.266+0000 7faf6e79d700 1 Processor -- start 2023-10-06T05:19:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.266+0000 7faf6e79d700 1 -- start start 2023-10-06T05:19:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.267+0000 7faf6e79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf680ccfc0 0x7faf680cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.267+0000 7faf6e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680cf3b0 0x7faf681594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.267+0000 7faf6e79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf680cc750 0x7faf681599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.267+0000 7faf6e79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680cbd80 con 0x7faf680cc750 2023-10-06T05:19:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.267+0000 7faf6e79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680af5f0 con 0x7faf680ccfc0 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.267+0000 7faf6e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf680af8d0 con 0x7faf680cf3b0 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf680cc750 0x7faf681599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf680cc750 0x7faf681599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40566/0 (socket says 172.21.15.33:40566) 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 -- 172.21.15.33:0/2283402629 learned_addr learned my addr 172.21.15.33:0/2283402629 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6cf9a700 1 --2- 172.21.15.33:0/2283402629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680cf3b0 0x7faf681594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6d79b700 1 --2- 172.21.15.33:0/2283402629 >> v2:172.21.15.167:3300/0 conn(0x7faf680ccfc0 0x7faf680cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 -- 172.21.15.33:0/2283402629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680cf3b0 msgr2=0x7faf681594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 --2- 172.21.15.33:0/2283402629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680cf3b0 0x7faf681594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 -- 172.21.15.33:0/2283402629 >> v2:172.21.15.167:3300/0 conn(0x7faf680ccfc0 msgr2=0x7faf680cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:30.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 --2- 172.21.15.33:0/2283402629 >> v2:172.21.15.167:3300/0 conn(0x7faf680ccfc0 0x7faf680cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 -- 172.21.15.33:0/2283402629 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf680afbb0 con 0x7faf680cc750 2023-10-06T05:19:30.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.268+0000 7faf6df9c700 1 --2- 172.21.15.33:0/2283402629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf681599e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faf6806d790 tx=0x7faf60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da341471b987af44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:30.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf5ffff700 1 -- 172.21.15.33:0/2283402629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000e070 con 0x7faf680cc750 2023-10-06T05:19:30.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf5ffff700 1 -- 172.21.15.33:0/2283402629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000b970 con 0x7faf680cc750 2023-10-06T05:19:30.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf5ffff700 1 -- 172.21.15.33:0/2283402629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600092e0 con 0x7faf680cc750 2023-10-06T05:19:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 -- 172.21.15.33:0/2283402629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 msgr2=0x7faf681599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 --2- 172.21.15.33:0/2283402629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf681599e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faf6806d790 tx=0x7faf60009840 comp rx=0 tx=0).stop 2023-10-06T05:19:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 -- 172.21.15.33:0/2283402629 shutdown_connections 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 --2- 172.21.15.33:0/2283402629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf681599e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 --2- 172.21.15.33:0/2283402629 >> v2:172.21.15.167:3300/0 conn(0x7faf680ccfc0 0x7faf680cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 --2- 172.21.15.33:0/2283402629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680cf3b0 0x7faf681594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.269+0000 7faf6e79d700 1 -- 172.21.15.33:0/2283402629 >> 172.21.15.33:0/2283402629 conn(0x7faf6806cff0 msgr2=0x7faf6806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.270+0000 7faf6e79d700 1 -- 172.21.15.33:0/2283402629 shutdown_connections 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.270+0000 7faf6e79d700 1 -- 172.21.15.33:0/2283402629 wait complete. 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.271+0000 7faf6e79d700 1 Processor -- start 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.271+0000 7faf6e79d700 1 -- start start 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6e79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf680b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccfc0 0x7faf68153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6e79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf680c8f60 0x7faf681558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6e79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6815f040 con 0x7faf680cc750 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6e79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680b41d0 con 0x7faf680c8f60 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68169a10 con 0x7faf680ccfc0 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6d79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf680b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6d79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf680b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40578/0 (socket says 172.21.15.33:40578) 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.272+0000 7faf6d79b700 1 -- 172.21.15.33:0/1353940551 learned_addr learned my addr 172.21.15.33:0/1353940551 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:30.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6cf9a700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccfc0 0x7faf68153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6df9c700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf680c8f60 0x7faf681558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6d79b700 1 -- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccfc0 msgr2=0x7faf68153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6d79b700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccfc0 0x7faf68153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6d79b700 1 -- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf680c8f60 msgr2=0x7faf681558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6d79b700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf680c8f60 0x7faf681558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6d79b700 1 -- 172.21.15.33:0/1353940551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009ae0 con 0x7faf680cc750 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6cf9a700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccfc0 0x7faf68153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6d79b700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf680b32a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faf6400ec80 tx=0x7faf640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400c850 con 0x7faf680cc750 2023-10-06T05:19:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6400c9b0 con 0x7faf680cc750 2023-10-06T05:19:30.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400cd50 con 0x7faf680cc750 2023-10-06T05:19:30.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.273+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf68169c80 con 0x7faf680cc750 2023-10-06T05:19:30.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.274+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf680b3930 con 0x7faf680cc750 2023-10-06T05:19:30.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.275+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6804c380 con 0x7faf680cc750 2023-10-06T05:19:30.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.275+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf64013680 con 0x7faf680cc750 2023-10-06T05:19:30.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.276+0000 7faf5d7fa700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.276+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5206+0+0 (secure 0 0 0) 0x7faf64084600 con 0x7faf680cc750 2023-10-06T05:19:30.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.277+0000 7faf6cf9a700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.278+0000 7faf6cf9a700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf680cf1a0 tx=0x7faf58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:30.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.280+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf64051a60 con 0x7faf680cc750 2023-10-06T05:19:30.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.484+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faf680cb940 con 0x7faf680cc750 2023-10-06T05:19:30.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.484+0000 7faf5d7fa700 1 -- 172.21.15.33:0/1353940551 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+126 (secure 0 0 0) 0x7faf64055080 con 0x7faf680cc750 2023-10-06T05:19:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:30.487 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:19:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:19:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:19:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:19:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:19:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:19:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:19:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:19:30.489 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:19:30.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 msgr2=0x7faf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf680cf1a0 tx=0x7faf58008040 comp rx=0 tx=0).stop 2023-10-06T05:19:30.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 msgr2=0x7faf680b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf680b32a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faf6400ec80 tx=0x7faf640091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 shutdown_connections 2023-10-06T05:19:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf680c8f60 0x7faf681558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680cc750 0x7faf680b32a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 --2- 172.21.15.33:0/1353940551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccfc0 0x7faf68153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 >> 172.21.15.33:0/1353940551 conn(0x7faf6806cff0 msgr2=0x7faf680c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:30.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 shutdown_connections 2023-10-06T05:19:30.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.486+0000 7faf6e79d700 1 -- 172.21.15.33:0/1353940551 wait complete. 2023-10-06T05:19:30.505 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:19:30.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:19:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.696+0000 7ff8e519d700 1 Processor -- start 2023-10-06T05:19:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.696+0000 7ff8e519d700 1 -- start start 2023-10-06T05:19:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8e519d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8e00ad3e0 0x7ff8e0156b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8e519d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff8e01570a0 0x7ff8e0157490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8e519d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e006b1f0 con 0x7ff8e00ad010 2023-10-06T05:19:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8e519d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e006a3f0 con 0x7ff8e00ad3e0 2023-10-06T05:19:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8e519d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e006f110 con 0x7ff8e01570a0 2023-10-06T05:19:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8e00ad3e0 0x7ff8e0156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff8e00ad3e0 0x7ff8e0156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40586/0 (socket says 172.21.15.33:40586) 2023-10-06T05:19:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 -- 172.21.15.33:0/3610458442 learned_addr learned my addr 172.21.15.33:0/3610458442 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 -- 172.21.15.33:0/3610458442 >> v2:172.21.15.167:3300/0 conn(0x7ff8e01570a0 msgr2=0x7ff8e0157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 --2- 172.21.15.33:0/3610458442 >> v2:172.21.15.167:3300/0 conn(0x7ff8e01570a0 0x7ff8e0157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 -- 172.21.15.33:0/3610458442 >> v1:172.21.15.167:6789/0 conn(0x7ff8e00ad010 legacy=0x7ff8e006ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.698+0000 7ff8df7fe700 1 -- 172.21.15.33:0/3610458442 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e01579d0 con 0x7ff8e00ad3e0 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.699+0000 7ff8df7fe700 1 --2- 172.21.15.33:0/3610458442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e0156b60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff8e006beb0 tx=0x7ff8d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc5bdf3a16047e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.699+0000 7ff8deffd700 1 -- 172.21.15.33:0/3610458442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400b8b0 con 0x7ff8e00ad3e0 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.699+0000 7ff8deffd700 1 -- 172.21.15.33:0/3610458442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d400ba10 con 0x7ff8e00ad3e0 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.699+0000 7ff8deffd700 1 -- 172.21.15.33:0/3610458442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400bdb0 con 0x7ff8e00ad3e0 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.699+0000 7ff8e519d700 1 -- 172.21.15.33:0/3610458442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e00ad3e0 msgr2=0x7ff8e0156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.699+0000 7ff8e519d700 1 --2- 172.21.15.33:0/3610458442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e0156b60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff8e006beb0 tx=0x7ff8d4009250 comp rx=0 tx=0).stop 2023-10-06T05:19:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.700+0000 7ff8e519d700 1 -- 172.21.15.33:0/3610458442 shutdown_connections 2023-10-06T05:19:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.700+0000 7ff8e519d700 1 --2- 172.21.15.33:0/3610458442 >> v2:172.21.15.167:3300/0 conn(0x7ff8e01570a0 0x7ff8e0157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.700+0000 7ff8e519d700 1 --2- 172.21.15.33:0/3610458442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e0156b60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.700+0000 7ff8e519d700 1 -- 172.21.15.33:0/3610458442 >> 172.21.15.33:0/3610458442 conn(0x7ff8e0068090 msgr2=0x7ff8e006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.700+0000 7ff8e519d700 1 -- 172.21.15.33:0/3610458442 shutdown_connections 2023-10-06T05:19:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.700+0000 7ff8e519d700 1 -- 172.21.15.33:0/3610458442 wait complete. 2023-10-06T05:19:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.701+0000 7ff8e519d700 1 Processor -- start 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.701+0000 7ff8e519d700 1 -- start start 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8e00ad010 0x7ff8e016c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e519d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e519d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 0x7ff8e01696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e519d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0155f30 con 0x7ff8e01570a0 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e519d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0155320 con 0x7ff8e00ad3e0 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0155600 con 0x7ff8e00ad010 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 0x7ff8e01696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 0x7ff8e01696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40590/0 (socket says 172.21.15.33:40590) 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.702+0000 7ff8e499c700 1 -- 172.21.15.33:0/2302900732 learned_addr learned my addr 172.21.15.33:0/2302900732 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8df7fe700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8dffff700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8e00ad010 0x7ff8e016c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e499c700 1 -- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8e00ad010 msgr2=0x7ff8e016c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e499c700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8e00ad010 0x7ff8e016c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e499c700 1 -- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8e00ad3e0 msgr2=0x7ff8e015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e499c700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e499c700 1 -- 172.21.15.33:0/2302900732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d40094f0 con 0x7ff8e01570a0 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e499c700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 0x7ff8e01696e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800bc30 tx=0x7ff8d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80092e0 con 0x7ff8e01570a0 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d800c6f0 con 0x7ff8e01570a0 2023-10-06T05:19:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800cad0 con 0x7ff8e01570a0 2023-10-06T05:19:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.703+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e01558e0 con 0x7ff8e01570a0 2023-10-06T05:19:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.704+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e0154f80 con 0x7ff8e01570a0 2023-10-06T05:19:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.705+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e016a870 con 0x7ff8e01570a0 2023-10-06T05:19:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.705+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8d8012070 con 0x7ff8e01570a0 2023-10-06T05:19:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.706+0000 7ff8c7fff700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff8cc066f30 0x7ff8cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.706+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff8d8083ff0 con 0x7ff8e01570a0 2023-10-06T05:19:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.707+0000 7ff8dffff700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff8cc066f30 0x7ff8cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.707+0000 7ff8dffff700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff8cc066f30 0x7ff8cc0693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00ac3e0 tx=0x7ff8d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.710+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8d80513d0 con 0x7ff8e01570a0 2023-10-06T05:19:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:30.916+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff8e0063ff0 con 0x7ff8e01570a0 2023-10-06T05:19:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.272+0000 7ff8c7fff700 1 -- 172.21.15.33:0/2302900732 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff8d80549f0 con 0x7ff8e01570a0 2023-10-06T05:19:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff8cc066f30 msgr2=0x7ff8cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff8cc066f30 0x7ff8cc0693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00ac3e0 tx=0x7ff8d0009040 comp rx=0 tx=0).stop 2023-10-06T05:19:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 msgr2=0x7ff8e01696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 0x7ff8e01696e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800bc30 tx=0x7ff8d8009840 comp rx=0 tx=0).stop 2023-10-06T05:19:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 shutdown_connections 2023-10-06T05:19:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff8cc066f30 0x7ff8cc0693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8e01570a0 0x7ff8e01696e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8e00ad010 0x7ff8e016c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 --2- 172.21.15.33:0/2302900732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8e00ad3e0 0x7ff8e015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.277+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 >> 172.21.15.33:0/2302900732 conn(0x7ff8e0068090 msgr2=0x7ff8e00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.278+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 shutdown_connections 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.278+0000 7ff8e519d700 1 -- 172.21.15.33:0/2302900732 wait complete. 2023-10-06T05:19:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:19:31.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:19:31.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.514+0000 7f263559d700 1 Processor -- start 2023-10-06T05:19:31.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.514+0000 7f263559d700 1 -- start start 2023-10-06T05:19:31.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f263559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26300c7f60 0x7f26300c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f263559d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26300c8900 con 0x7f2630156740 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f263559d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263006b330 con 0x7f2630156370 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f263559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263006beb0 con 0x7f26300c7f60 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f262f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2630156740 0x7f26300cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33172/0 (socket says 172.21.15.33:33172) 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f262f7fe700 1 -- 172.21.15.33:0/670334312 learned_addr learned my addr 172.21.15.33:0/670334312 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f2634d9c700 1 --2- 172.21.15.33:0/670334312 >> v2:172.21.15.33:3300/0 conn(0x7f26300c7f60 0x7f26300c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f2634d9c700 1 -- 172.21.15.33:0/670334312 >> v1:172.21.15.167:6789/0 conn(0x7f2630156370 legacy=0x7f26300aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f2634d9c700 1 -- 172.21.15.33:0/670334312 >> v1:172.21.15.33:6789/0 conn(0x7f2630156740 legacy=0x7f26300cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f2634d9c700 1 -- 172.21.15.33:0/670334312 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263006b610 con 0x7f26300c7f60 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.516+0000 7f262effd700 1 -- 172.21.15.33:0/670334312 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875691047 0 0) 0x7f263006b330 con 0x7f2630156370 2023-10-06T05:19:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.517+0000 7f2634d9c700 1 --2- 172.21.15.33:0/670334312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300c7f60 0x7f26300c83c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2628009e60 tx=0x7f2628009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2b2ece3b72b902 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.517+0000 7f262effd700 1 -- 172.21.15.33:0/670334312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800e070 con 0x7f26300c7f60 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.517+0000 7f262effd700 1 -- 172.21.15.33:0/670334312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262800bc30 con 0x7f26300c7f60 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.517+0000 7f262effd700 1 -- 172.21.15.33:0/670334312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628012430 con 0x7f26300c7f60 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.517+0000 7f263559d700 1 -- 172.21.15.33:0/670334312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300c7f60 msgr2=0x7f26300c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.517+0000 7f263559d700 1 --2- 172.21.15.33:0/670334312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300c7f60 0x7f26300c83c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2628009e60 tx=0x7f2628009780 comp rx=0 tx=0).stop 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.518+0000 7f263559d700 1 -- 172.21.15.33:0/670334312 shutdown_connections 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.518+0000 7f263559d700 1 --2- 172.21.15.33:0/670334312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300c7f60 0x7f26300c83c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.518+0000 7f263559d700 1 -- 172.21.15.33:0/670334312 >> 172.21.15.33:0/670334312 conn(0x7f263006c560 msgr2=0x7f263006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.518+0000 7f263559d700 1 -- 172.21.15.33:0/670334312 shutdown_connections 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.518+0000 7f263559d700 1 -- 172.21.15.33:0/670334312 wait complete. 2023-10-06T05:19:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.519+0000 7f263559d700 1 Processor -- start 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.519+0000 7f263559d700 1 -- start start 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f263559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26300c7f60 0x7f2630143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f263559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2630156370 0x7f2630146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f263559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 0x7f26300b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f263559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263016cfa0 con 0x7f2630156740 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f263559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263016cc70 con 0x7f26300c7f60 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f263559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26301472d0 con 0x7f2630156370 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f2634d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 0x7f26300b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.520+0000 7f2634d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 0x7f26300b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40602/0 (socket says 172.21.15.33:40602) 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 -- 172.21.15.33:0/2140392728 learned_addr learned my addr 172.21.15.33:0/2140392728 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f262ffff700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26300c7f60 0x7f2630143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 -- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2630156370 msgr2=0x7f2630146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f262f7fe700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2630156370 0x7f2630146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2630156370 0x7f2630146ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 -- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26300c7f60 msgr2=0x7f2630143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26300c7f60 0x7f2630143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 -- 172.21.15.33:0/2140392728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628009a20 con 0x7f2630156740 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f262ffff700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26300c7f60 0x7f2630143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f262f7fe700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2630156370 0x7f2630146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2634d9c700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 0x7f26300b1130 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2630069740 tx=0x7f2628019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.521+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262801b630 con 0x7f2630156740 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.522+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262801b790 con 0x7f2630156740 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.522+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26301475b0 con 0x7f2630156740 2023-10-06T05:19:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.522+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262801bb50 con 0x7f2630156740 2023-10-06T05:19:31.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.522+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26300b1670 con 0x7f2630156740 2023-10-06T05:19:31.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.523+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263016d110 con 0x7f2630156740 2023-10-06T05:19:31.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.524+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f262801c400 con 0x7f2630156740 2023-10-06T05:19:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.525+0000 7f2617fff700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.525+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f2628086900 con 0x7f2630156740 2023-10-06T05:19:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.525+0000 7f262ffff700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.526+0000 7f262ffff700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f261c066f30 0x7f261c0693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2620007fd0 tx=0x7f2620009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.529+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2628053bd0 con 0x7f2630156740 2023-10-06T05:19:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:31.733+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f263004c380 con 0x7f2630156740 2023-10-06T05:19:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.283+0000 7f2617fff700 1 -- 172.21.15.33:0/2140392728 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) v1 ==== 170+0+0 (secure 0 0 0) 0x7f262802a0d0 con 0x7f2630156740 2023-10-06T05:19:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.295+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f261c066f30 msgr2=0x7f261c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.295+0000 7f263559d700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f261c066f30 0x7f261c0693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2620007fd0 tx=0x7f2620009040 comp rx=0 tx=0).stop 2023-10-06T05:19:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.295+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 msgr2=0x7f26300b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.295+0000 7f263559d700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 0x7f26300b1130 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2630069740 tx=0x7f2628019000 comp rx=0 tx=0).stop 2023-10-06T05:19:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.296+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 shutdown_connections 2023-10-06T05:19:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.296+0000 7f263559d700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.296+0000 7f263559d700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2630156740 0x7f26300b1130 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.296+0000 7f263559d700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26300c7f60 0x7f2630143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.296+0000 7f263559d700 1 --2- 172.21.15.33:0/2140392728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2630156370 0x7f2630146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.296+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 >> 172.21.15.33:0/2140392728 conn(0x7f263006c560 msgr2=0x7f26300cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.297+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 shutdown_connections 2023-10-06T05:19:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.297+0000 7f263559d700 1 -- 172.21.15.33:0/2140392728 wait complete. 2023-10-06T05:19:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:19:32.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:19:32.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.513+0000 7fd0effff700 1 Processor -- start 2023-10-06T05:19:32.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.513+0000 7fd0effff700 1 -- start start 2023-10-06T05:19:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.514+0000 7fd0effff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0f0159190 0x7fd0f0159560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.514+0000 7fd0effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 0x7fd0f00680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.514+0000 7fd0effff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f00b2040 con 0x7fd0f0067c40 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.514+0000 7fd0effff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f0068620 con 0x7fd0f0159190 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.514+0000 7fd0effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f006ebe0 con 0x7fd0f00b1c00 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 0x7fd0f00680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 0x7fd0f00680e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:39084/0 (socket says 172.21.15.33:39084) 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 -- 172.21.15.33:0/1206155360 learned_addr learned my addr 172.21.15.33:0/1206155360 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0eeffd700 1 --2- 172.21.15.33:0/1206155360 >> v2:172.21.15.167:3300/0 conn(0x7fd0f0159190 0x7fd0f0159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 -- 172.21.15.33:0/1206155360 >> v2:172.21.15.167:3300/0 conn(0x7fd0f0159190 msgr2=0x7fd0f0159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 --2- 172.21.15.33:0/1206155360 >> v2:172.21.15.167:3300/0 conn(0x7fd0f0159190 0x7fd0f0159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0edffb700 1 -- 172.21.15.33:0/1206155360 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006550961 0 0) 0x7fd0f00b2040 con 0x7fd0f0067c40 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 -- 172.21.15.33:0/1206155360 >> v1:172.21.15.167:6789/0 conn(0x7fd0f0067c40 legacy=0x7fd0f00b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.515+0000 7fd0ef7fe700 1 -- 172.21.15.33:0/1206155360 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0159c50 con 0x7fd0f00b1c00 2023-10-06T05:19:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.516+0000 7fd0ef7fe700 1 --2- 172.21.15.33:0/1206155360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 0x7fd0f00680e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009f20 tx=0x7fd0e8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94cb5933119f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.516+0000 7fd0edffb700 1 -- 172.21.15.33:0/1206155360 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800e070 con 0x7fd0f00b1c00 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.516+0000 7fd0edffb700 1 -- 172.21.15.33:0/1206155360 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e800b9d0 con 0x7fd0f00b1c00 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.516+0000 7fd0edffb700 1 -- 172.21.15.33:0/1206155360 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80092e0 con 0x7fd0f00b1c00 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 -- 172.21.15.33:0/1206155360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 msgr2=0x7fd0f00680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 --2- 172.21.15.33:0/1206155360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 0x7fd0f00680e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009f20 tx=0x7fd0e8009840 comp rx=0 tx=0).stop 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 -- 172.21.15.33:0/1206155360 shutdown_connections 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 --2- 172.21.15.33:0/1206155360 >> v2:172.21.15.167:3300/0 conn(0x7fd0f0159190 0x7fd0f0159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 --2- 172.21.15.33:0/1206155360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f00b1c00 0x7fd0f00680e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 -- 172.21.15.33:0/1206155360 >> 172.21.15.33:0/1206155360 conn(0x7fd0f006c560 msgr2=0x7fd0f006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 -- 172.21.15.33:0/1206155360 shutdown_connections 2023-10-06T05:19:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.517+0000 7fd0effff700 1 -- 172.21.15.33:0/1206155360 wait complete. 2023-10-06T05:19:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.518+0000 7fd0effff700 1 Processor -- start 2023-10-06T05:19:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.518+0000 7fd0effff700 1 -- start start 2023-10-06T05:19:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.519+0000 7fd0effff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f0067c40 0x7fd0f00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.519+0000 7fd0effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 0x7fd0f016c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.519+0000 7fd0effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f0159190 0x7fd0f0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.519+0000 7fd0effff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00cb890 con 0x7fd0f00b1c00 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.519+0000 7fd0effff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00cbb70 con 0x7fd0f0067c40 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.519+0000 7fd0effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00cbe50 con 0x7fd0f0159190 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 0x7fd0f016c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 0x7fd0f016c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40612/0 (socket says 172.21.15.33:40612) 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 -- 172.21.15.33:0/379333288 learned_addr learned my addr 172.21.15.33:0/379333288 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ef7fe700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f0159190 0x7fd0f0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0eeffd700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f0067c40 0x7fd0f00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 -- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f0159190 msgr2=0x7fd0f0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f0159190 0x7fd0f0155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 -- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f0067c40 msgr2=0x7fd0f00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f0067c40 0x7fd0f00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 -- 172.21.15.33:0/379333288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8009ae0 con 0x7fd0f00b1c00 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0ee7fc700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 0x7fd0f016c9b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c7e0 tx=0x7fd0e4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.520+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4012700 con 0x7fd0f00b1c00 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.521+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e4012880 con 0x7fd0f00b1c00 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.521+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f00cc130 con 0x7fd0f00b1c00 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.521+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4012c40 con 0x7fd0f00b1c00 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.521+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f00cb640 con 0x7fd0f00b1c00 2023-10-06T05:19:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.522+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00cc650 con 0x7fd0f00b1c00 2023-10-06T05:19:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.523+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0e401b400 con 0x7fd0f00b1c00 2023-10-06T05:19:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.524+0000 7fd0d6ffd700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.524+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fd0e4084290 con 0x7fd0f00b1c00 2023-10-06T05:19:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.524+0000 7fd0eeffd700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.525+0000 7fd0eeffd700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0dc066f30 0x7fd0dc0693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009ca0 tx=0x7fd0e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.528+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0f00cc650 con 0x7fd0f00b1c00 2023-10-06T05:19:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:32.777+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd0f00cc650 con 0x7fd0f00b1c00 2023-10-06T05:19:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.288+0000 7fd0d6ffd700 1 -- 172.21.15.33:0/379333288 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd0f00cc650 con 0x7fd0f00b1c00 2023-10-06T05:19:33.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.309+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0dc066f30 msgr2=0x7fd0dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.309+0000 7fd0effff700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0dc066f30 0x7fd0dc0693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009ca0 tx=0x7fd0e0008040 comp rx=0 tx=0).stop 2023-10-06T05:19:33.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.309+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 msgr2=0x7fd0f016c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.309+0000 7fd0effff700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 0x7fd0f016c9b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c7e0 tx=0x7fd0e4007df0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.310+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 shutdown_connections 2023-10-06T05:19:33.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.310+0000 7fd0effff700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.310+0000 7fd0effff700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f0159190 0x7fd0f0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.310+0000 7fd0effff700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f0067c40 0x7fd0f00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.310+0000 7fd0effff700 1 --2- 172.21.15.33:0/379333288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f00b1c00 0x7fd0f016c9b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.310+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 >> 172.21.15.33:0/379333288 conn(0x7fd0f006c560 msgr2=0x7fd0f00b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:33.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.317+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 shutdown_connections 2023-10-06T05:19:33.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.317+0000 7fd0effff700 1 -- 172.21.15.33:0/379333288 wait complete. 2023-10-06T05:19:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T05:19:33.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.527+0000 7fa738d9d700 1 Processor -- start 2023-10-06T05:19:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.527+0000 7fa738d9d700 1 -- start start 2023-10-06T05:19:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa738d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7340ccfc0 0x7fa7340cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa738d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7340cf3b0 0x7fa7341594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa738d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7341599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa738d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340cbd80 con 0x7fa7340ccfc0 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa738d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340af5f0 con 0x7fa7340cf3b0 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa738d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340af8d0 con 0x7fa7340cc750 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.528+0000 7fa7337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7340ccfc0 0x7fa7340cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7340ccfc0 0x7fa7340cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40626/0 (socket says 172.21.15.33:40626) 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 -- 172.21.15.33:0/3398797413 learned_addr learned my addr 172.21.15.33:0/3398797413 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa732ffd700 1 --2- 172.21.15.33:0/3398797413 >> v2:172.21.15.167:3300/0 conn(0x7fa7340cf3b0 0x7fa7341594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa733fff700 1 --2- 172.21.15.33:0/3398797413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7341599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 -- 172.21.15.33:0/3398797413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 msgr2=0x7fa7341599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 --2- 172.21.15.33:0/3398797413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7341599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 -- 172.21.15.33:0/3398797413 >> v2:172.21.15.167:3300/0 conn(0x7fa7340cf3b0 msgr2=0x7fa7341594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 --2- 172.21.15.33:0/3398797413 >> v2:172.21.15.167:3300/0 conn(0x7fa7340cf3b0 0x7fa7341594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 -- 172.21.15.33:0/3398797413 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340afbb0 con 0x7fa7340ccfc0 2023-10-06T05:19:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.529+0000 7fa7337fe700 1 --2- 172.21.15.33:0/3398797413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa7340cc210 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa7240099b0 tx=0x7fa724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadbf38fd13a889c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa7327fc700 1 -- 172.21.15.33:0/3398797413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400b820 con 0x7fa7340ccfc0 2023-10-06T05:19:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa7327fc700 1 -- 172.21.15.33:0/3398797413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72400b980 con 0x7fa7340ccfc0 2023-10-06T05:19:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa7327fc700 1 -- 172.21.15.33:0/3398797413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400bd20 con 0x7fa7340ccfc0 2023-10-06T05:19:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 -- 172.21.15.33:0/3398797413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 msgr2=0x7fa7340cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3398797413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa7340cc210 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa7240099b0 tx=0x7fa724009250 comp rx=0 tx=0).stop 2023-10-06T05:19:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 -- 172.21.15.33:0/3398797413 shutdown_connections 2023-10-06T05:19:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3398797413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7341599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3398797413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa7340cc210 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3398797413 >> v2:172.21.15.167:3300/0 conn(0x7fa7340cf3b0 0x7fa7341594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 -- 172.21.15.33:0/3398797413 >> 172.21.15.33:0/3398797413 conn(0x7fa73406cff0 msgr2=0x7fa73406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 -- 172.21.15.33:0/3398797413 shutdown_connections 2023-10-06T05:19:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.530+0000 7fa738d9d700 1 -- 172.21.15.33:0/3398797413 wait complete. 2023-10-06T05:19:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.532+0000 7fa738d9d700 1 Processor -- start 2023-10-06T05:19:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.532+0000 7fa738d9d700 1 -- start start 2023-10-06T05:19:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa738d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7340c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa738d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa734169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa738d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7340c85e0 0x7fa73416a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa738d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340ca400 con 0x7fa7340ccfc0 2023-10-06T05:19:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa738d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340b1f30 con 0x7fa7340c85e0 2023-10-06T05:19:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa738d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c9360 con 0x7fa7340cc750 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa734169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa734169c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40638/0 (socket says 172.21.15.33:40638) 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 -- 172.21.15.33:0/3751731049 learned_addr learned my addr 172.21.15.33:0/3751731049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa733fff700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7340c85e0 0x7fa73416a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa7337fe700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7340c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 -- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 msgr2=0x7fa7340c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7340c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 -- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7340c85e0 msgr2=0x7fa73416a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7340c85e0 0x7fa73416a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.533+0000 7fa732ffd700 1 -- 172.21.15.33:0/3751731049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240094f0 con 0x7fa7340ccfc0 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.534+0000 7fa732ffd700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa734169c90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa72800d890 tx=0x7fa728008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.534+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728012630 con 0x7fa7340ccfc0 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.534+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa728012790 con 0x7fa7340ccfc0 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.534+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728012b30 con 0x7fa7340ccfc0 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.534+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7340c9640 con 0x7fa7340ccfc0 2023-10-06T05:19:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.534+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7340c9c00 con 0x7fa7340ccfc0 2023-10-06T05:19:33.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.535+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73404c380 con 0x7fa7340ccfc0 2023-10-06T05:19:33.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.536+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa72801e460 con 0x7fa7340ccfc0 2023-10-06T05:19:33.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.537+0000 7fa71b7fe700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa720066f30 0x7fa7200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.537+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa728084df0 con 0x7fa7340ccfc0 2023-10-06T05:19:33.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.537+0000 7fa7337fe700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa720066f30 0x7fa7200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.538+0000 7fa7337fe700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa720066f30 0x7fa7200693e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa7240099b0 tx=0x7fa724017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:33.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.541+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa728051e60 con 0x7fa7340ccfc0 2023-10-06T05:19:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.746+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa734157e80 con 0x7fa7340ccfc0 2023-10-06T05:19:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.747+0000 7fa71b7fe700 1 -- 172.21.15.33:0/3751731049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16882 (secure 0 0 0) 0x7fa728055480 con 0x7fa7340ccfc0 2023-10-06T05:19:33.750 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:33.750 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:19:32.778147+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:19:30.956591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:19:31.773352+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa720066f30 msgr2=0x7fa7200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa720066f30 0x7fa7200693e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa7240099b0 tx=0x7fa724017000 comp rx=0 tx=0).stop 2023-10-06T05:19:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 msgr2=0x7fa734169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa734169c90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa72800d890 tx=0x7fa728008040 comp rx=0 tx=0).stop 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 shutdown_connections 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa720066f30 0x7fa7200693e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7340c85e0 0x7fa73416a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7340cc750 0x7fa7340c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 --2- 172.21.15.33:0/3751731049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7340ccfc0 0x7fa734169c90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 >> 172.21.15.33:0/3751731049 conn(0x7fa73406cff0 msgr2=0x7fa7340c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 shutdown_connections 2023-10-06T05:19:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.750+0000 7fa738d9d700 1 -- 172.21.15.33:0/3751731049 wait complete. 2023-10-06T05:19:33.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.956+0000 7fc3f679d700 1 Processor -- start 2023-10-06T05:19:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.956+0000 7fc3f679d700 1 -- start start 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f0158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3f00adf50 0x7fc3f00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f00c8e40 con 0x7fc3f00c9810 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f00ae6c0 con 0x7fc3f00adf50 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00ae9a0 con 0x7fc3f0155140 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f4f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3f00adf50 0x7fc3f00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f4f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3f00adf50 0x7fc3f00c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40644/0 (socket says 172.21.15.33:40644) 2023-10-06T05:19:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.957+0000 7fc3f4f9a700 1 -- 172.21.15.33:0/3609904814 learned_addr learned my addr 172.21.15.33:0/3609904814 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3f579b700 1 --2- 172.21.15.33:0/3609904814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3f4f9a700 1 -- 172.21.15.33:0/3609904814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 msgr2=0x7fc3f0158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3f4f9a700 1 --2- 172.21.15.33:0/3609904814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f0158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3f4f9a700 1 -- 172.21.15.33:0/3609904814 >> v1:172.21.15.33:6789/0 conn(0x7fc3f00c9810 legacy=0x7fc3f00699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3f4f9a700 1 -- 172.21.15.33:0/3609904814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00c8ad0 con 0x7fc3f00adf50 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3f4f9a700 1 --2- 172.21.15.33:0/3609904814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c92d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00099b0 tx=0x7fc3e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0453a6c40ead8ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3e7fff700 1 -- 172.21.15.33:0/3609904814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000b860 con 0x7fc3f00adf50 2023-10-06T05:19:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3e7fff700 1 -- 172.21.15.33:0/3609904814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000b9c0 con 0x7fc3f00adf50 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.958+0000 7fc3e7fff700 1 -- 172.21.15.33:0/3609904814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000bd60 con 0x7fc3f00adf50 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 -- 172.21.15.33:0/3609904814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 msgr2=0x7fc3f00c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 --2- 172.21.15.33:0/3609904814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c92d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00099b0 tx=0x7fc3e0009250 comp rx=0 tx=0).stop 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 -- 172.21.15.33:0/3609904814 shutdown_connections 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 --2- 172.21.15.33:0/3609904814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f0158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 --2- 172.21.15.33:0/3609904814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c92d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 -- 172.21.15.33:0/3609904814 >> 172.21.15.33:0/3609904814 conn(0x7fc3f006d030 msgr2=0x7fc3f006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 -- 172.21.15.33:0/3609904814 shutdown_connections 2023-10-06T05:19:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.959+0000 7fc3f679d700 1 -- 172.21.15.33:0/3609904814 wait complete. 2023-10-06T05:19:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.960+0000 7fc3f679d700 1 Processor -- start 2023-10-06T05:19:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.961+0000 7fc3f679d700 1 -- start start 2023-10-06T05:19:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.961+0000 7fc3f679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f00c9810 0x7fc3f014df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f016afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0169df0 con 0x7fc3f00adf50 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00cb5c0 con 0x7fc3f00c9810 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f014e450 con 0x7fc3f0155140 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40656/0 (socket says 172.21.15.33:40656) 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 -- 172.21.15.33:0/1592985467 learned_addr learned my addr 172.21.15.33:0/1592985467 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f5f9c700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f016afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f4f9a700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f00c9810 0x7fc3f014df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 -- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 msgr2=0x7fc3f016afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f016afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 -- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f00c9810 msgr2=0x7fc3f014df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f00c9810 0x7fc3f014df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 -- 172.21.15.33:0/1592985467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e00094f0 con 0x7fc3f00adf50 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.962+0000 7fc3f579b700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c9cb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d860 tx=0x7fc3ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.963+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec009610 con 0x7fc3f00adf50 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.963+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec009770 con 0x7fc3f00adf50 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.963+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec009b30 con 0x7fc3f00adf50 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.963+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f00cad20 con 0x7fc3f00adf50 2023-10-06T05:19:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.963+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f016a000 con 0x7fc3f00adf50 2023-10-06T05:19:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.964+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f004c380 con 0x7fc3f00adf50 2023-10-06T05:19:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.965+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3ec01e460 con 0x7fc3f00adf50 2023-10-06T05:19:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.966+0000 7fc3e57fa700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3d4066f30 0x7fc3d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.966+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc3ec084da0 con 0x7fc3f00adf50 2023-10-06T05:19:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.966+0000 7fc3f4f9a700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3d4066f30 0x7fc3d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:33.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.967+0000 7fc3f4f9a700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3d4066f30 0x7fc3d40693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00099b0 tx=0x7fc3e0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:33.970+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3ec051e10 con 0x7fc3f00adf50 2023-10-06T05:19:34.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.174+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3f00c7b80 con 0x7fc3f00adf50 2023-10-06T05:19:34.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.175+0000 7fc3e57fa700 1 -- 172.21.15.33:0/1592985467 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16882 (secure 0 0 0) 0x7fc3ec055430 con 0x7fc3f00adf50 2023-10-06T05:19:34.177 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:34.177 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:19:32.778147+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:19:30.956591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:19:31.773352+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:34.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3d4066f30 msgr2=0x7fc3d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3d4066f30 0x7fc3d40693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00099b0 tx=0x7fc3e0017000 comp rx=0 tx=0).stop 2023-10-06T05:19:34.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 msgr2=0x7fc3f00c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c9cb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d860 tx=0x7fc3ec008040 comp rx=0 tx=0).stop 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 shutdown_connections 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3d4066f30 0x7fc3d40693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3f0155140 0x7fc3f016afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f00adf50 0x7fc3f00c9cb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 --2- 172.21.15.33:0/1592985467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3f00c9810 0x7fc3f014df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 >> 172.21.15.33:0/1592985467 conn(0x7fc3f006d030 msgr2=0x7fc3f00ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 shutdown_connections 2023-10-06T05:19:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.177+0000 7fc3f679d700 1 -- 172.21.15.33:0/1592985467 wait complete. 2023-10-06T05:19:34.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:19:34.266 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:19:34.264+0000 7f6bd63f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:19:34.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.389+0000 7f36ae99d700 1 Processor -- start 2023-10-06T05:19:34.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.389+0000 7f36ae99d700 1 -- start start 2023-10-06T05:19:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.390+0000 7f36ae99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 0x7f36a80abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.390+0000 7f36ae99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a80b0cd0 con 0x7f36a8159850 2023-10-06T05:19:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.390+0000 7f36ae99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a815a840 con 0x7f36a806efa0 2023-10-06T05:19:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.390+0000 7f36ae99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8159f10 con 0x7f36a815acd0 2023-10-06T05:19:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.390+0000 7f36ad99b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f36a8159850 0x7f36a806e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33194/0 (socket says 172.21.15.33:33194) 2023-10-06T05:19:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.390+0000 7f36ad99b700 1 -- 172.21.15.33:0/2667447339 learned_addr learned my addr 172.21.15.33:0/2667447339 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ad19a700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 0x7f36a80abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178250217 0 0) 0x7f36a80b0cd0 con 0x7f36a8159850 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3694003570 con 0x7f36a8159850 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786018853 0 0) 0x7f36a815a840 con 0x7f36a806efa0 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36a80b0cd0 con 0x7f36a806efa0 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474587844 0 0) 0x7f3694003570 con 0x7f36a8159850 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 msgr2=0x7f36a80abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 0x7f36a80abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 >> v1:172.21.15.167:6789/0 conn(0x7f36a806efa0 legacy=0x7f36a80ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a815a1f0 con 0x7f36a8159850 2023-10-06T05:19:34.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f36a4001470 con 0x7f36a8159850 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 >> v1:172.21.15.33:6789/0 conn(0x7f36a8159850 legacy=0x7f36a806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 0x7f3694003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3694004180 0x7f3694004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3694004d50 0x7f36940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8159f10 con 0x7f3694004d50 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369400bba0 con 0x7f3694004180 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.391+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369400bd60 con 0x7f36a815acd0 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3694004d50 0x7f36940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ad19a700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3694004180 0x7f3694004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ad99b700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 0x7f3694003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 -- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 msgr2=0x7f3694003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:34.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36a815acd0 0x7f3694003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 -- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3694004180 msgr2=0x7f3694004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3694004180 0x7f3694004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 -- 172.21.15.33:0/2667447339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694003570 con 0x7f3694004d50 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.392+0000 7f36ae19c700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3694004d50 0x7f36940051c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f36a806f3a0 tx=0x7f36a00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c80fcef7eed826 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0001ab0 con 0x7f3694004d50 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000c8b0 con 0x7f3694004d50 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ac999700 1 -- 172.21.15.33:0/2667447339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000cc50 con 0x7f3694004d50 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae99d700 1 -- 172.21.15.33:0/2667447339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3694004d50 msgr2=0x7f36940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae99d700 1 --2- 172.21.15.33:0/2667447339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3694004d50 0x7f36940051c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f36a806f3a0 tx=0x7f36a00017a0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae19c700 1 -- 172.21.15.33:0/2667447339 reap_dead start 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae99d700 1 -- 172.21.15.33:0/2667447339 shutdown_connections 2023-10-06T05:19:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae99d700 1 -- 172.21.15.33:0/2667447339 >> 172.21.15.33:0/2667447339 conn(0x7f36a806cd40 msgr2=0x7f36a80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae99d700 1 -- 172.21.15.33:0/2667447339 shutdown_connections 2023-10-06T05:19:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.393+0000 7f36ae99d700 1 -- 172.21.15.33:0/2667447339 wait complete. 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.395+0000 7f36ae99d700 1 Processor -- start 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.395+0000 7f36ae99d700 1 -- start start 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3694004d50 0x7f36a81552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 0x7f36a81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80c8d60 0x7f36a80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80ca330 con 0x7f36a8157e40 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a81587f0 con 0x7f36a80c8d60 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80c9a90 con 0x7f3694004d50 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 0x7f36a81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 0x7f36a81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40686/0 (socket says 172.21.15.33:40686) 2023-10-06T05:19:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 -- 172.21.15.33:0/3758017943 learned_addr learned my addr 172.21.15.33:0/3758017943 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad99b700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3694004d50 0x7f36a81552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae19c700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80c8d60 0x7f36a80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 -- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3694004d50 msgr2=0x7f36a81552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3694004d50 0x7f36a81552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 -- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80c8d60 msgr2=0x7f36a80c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80c8d60 0x7f36a80c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad19a700 1 -- 172.21.15.33:0/3758017943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80c9d70 con 0x7f36a8157e40 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ae19c700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80c8d60 0x7f36a80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.396+0000 7f36ad99b700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3694004d50 0x7f36a81552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.397+0000 7f36ad19a700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 0x7f36a81582b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3698011b10 tx=0x7f36980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.397+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800f9f0 con 0x7f36a8157e40 2023-10-06T05:19:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.397+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369800fb50 con 0x7f36a8157e40 2023-10-06T05:19:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.397+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800fed0 con 0x7f36a8157e40 2023-10-06T05:19:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.397+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a80ca050 con 0x7f36a8157e40 2023-10-06T05:19:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.397+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a8155800 con 0x7f36a8157e40 2023-10-06T05:19:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.398+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a815b1a0 con 0x7f36a8157e40 2023-10-06T05:19:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.399+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3698014650 con 0x7f36a8157e40 2023-10-06T05:19:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.400+0000 7f369dffb700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.400+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3698018030 con 0x7f36a8157e40 2023-10-06T05:19:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.401+0000 7f36ad99b700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.401+0000 7f36ad99b700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f368c066f30 0x7f368c0693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36a4000e10 tx=0x7f36a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:34.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.404+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36980522d0 con 0x7f36a8157e40 2023-10-06T05:19:34.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:34.654+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f36a804c380 con 0x7f36a8157e40 2023-10-06T05:19:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.303+0000 7f369dffb700 1 -- 172.21.15.33:0/3758017943 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f36980558f0 con 0x7f36a8157e40 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f368c066f30 msgr2=0x7f368c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f368c066f30 0x7f368c0693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36a4000e10 tx=0x7f36a4009040 comp rx=0 tx=0).stop 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 msgr2=0x7f36a81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 0x7f36a81582b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3698011b10 tx=0x7f36980091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 shutdown_connections 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36a80c8d60 0x7f36a80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3694004d50 0x7f36a81552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 --2- 172.21.15.33:0/3758017943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36a8157e40 0x7f36a81582b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 >> 172.21.15.33:0/3758017943 conn(0x7f36a806cd40 msgr2=0x7f36a806e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 shutdown_connections 2023-10-06T05:19:35.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.306+0000 7f36ae99d700 1 -- 172.21.15.33:0/3758017943 wait complete. 2023-10-06T05:19:35.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:19:35.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.521+0000 7ff95979d700 1 Processor -- start 2023-10-06T05:19:35.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.521+0000 7ff95979d700 1 -- start start 2023-10-06T05:19:35.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.522+0000 7ff95979d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff95415acd0 0x7ff9540abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:35.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.522+0000 7ff95979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 0x7ff9540ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:35.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.522+0000 7ff95979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9540b0cd0 con 0x7ff954159850 2023-10-06T05:19:35.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.522+0000 7ff95979d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95415a840 con 0x7ff95415acd0 2023-10-06T05:19:35.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.522+0000 7ff95979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff954159f10 con 0x7ff95406efa0 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 0x7ff9540ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 0x7ff9540ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:39164/0 (socket says 172.21.15.33:39164) 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 -- 172.21.15.33:0/2713783891 learned_addr learned my addr 172.21.15.33:0/2713783891 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff953fff700 1 --1- 172.21.15.33:0/2713783891 >> v1:172.21.15.167:6789/0 conn(0x7ff954159850 0x7ff95406e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:43180/0 (socket says 172.21.15.33:43180) 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff9537fe700 1 --2- 172.21.15.33:0/2713783891 >> v2:172.21.15.167:3300/0 conn(0x7ff95415acd0 0x7ff9540abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff952ffd700 1 -- 172.21.15.33:0/2713783891 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099775634 0 0) 0x7ff9540b0cd0 con 0x7ff954159850 2023-10-06T05:19:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 -- 172.21.15.33:0/2713783891 >> v2:172.21.15.167:3300/0 conn(0x7ff95415acd0 msgr2=0x7ff9540abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 --2- 172.21.15.33:0/2713783891 >> v2:172.21.15.167:3300/0 conn(0x7ff95415acd0 0x7ff9540abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 -- 172.21.15.33:0/2713783891 >> v1:172.21.15.167:6789/0 conn(0x7ff954159850 legacy=0x7ff95406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 -- 172.21.15.33:0/2713783891 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95415a1f0 con 0x7ff95406efa0 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.523+0000 7ff958f9c700 1 --2- 172.21.15.33:0/2713783891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 0x7ff9540ac3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009f20 tx=0x7ff94c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2060a268d07a8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff952ffd700 1 -- 172.21.15.33:0/2713783891 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00e070 con 0x7ff95406efa0 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff952ffd700 1 -- 172.21.15.33:0/2713783891 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00b9d0 con 0x7ff95406efa0 2023-10-06T05:19:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff952ffd700 1 -- 172.21.15.33:0/2713783891 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0092e0 con 0x7ff95406efa0 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 -- 172.21.15.33:0/2713783891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 msgr2=0x7ff9540ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 --2- 172.21.15.33:0/2713783891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 0x7ff9540ac3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009f20 tx=0x7ff94c009840 comp rx=0 tx=0).stop 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 -- 172.21.15.33:0/2713783891 shutdown_connections 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 --2- 172.21.15.33:0/2713783891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95406efa0 0x7ff9540ac3d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 --2- 172.21.15.33:0/2713783891 >> v2:172.21.15.167:3300/0 conn(0x7ff95415acd0 0x7ff9540abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 -- 172.21.15.33:0/2713783891 >> 172.21.15.33:0/2713783891 conn(0x7ff95406cd40 msgr2=0x7ff9540ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.524+0000 7ff95979d700 1 -- 172.21.15.33:0/2713783891 shutdown_connections 2023-10-06T05:19:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.525+0000 7ff95979d700 1 -- 172.21.15.33:0/2713783891 wait complete. 2023-10-06T05:19:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.526+0000 7ff95979d700 1 Processor -- start 2023-10-06T05:19:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.526+0000 7ff95979d700 1 -- start start 2023-10-06T05:19:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff95979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 0x7ff9540cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff95979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff954159850 0x7ff954148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff95979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95415acd0 0x7ff954068810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff95979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540cd6c0 con 0x7ff95406efa0 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff95979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540cc460 con 0x7ff954159850 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff95979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95416a3f0 con 0x7ff95415acd0 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff953fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 0x7ff9540cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff953fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 0x7ff9540cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40690/0 (socket says 172.21.15.33:40690) 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.527+0000 7ff953fff700 1 -- 172.21.15.33:0/80549280 learned_addr learned my addr 172.21.15.33:0/80549280 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff958f9c700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95415acd0 0x7ff954068810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff9537fe700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff954159850 0x7ff954148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff953fff700 1 -- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95415acd0 msgr2=0x7ff954068810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff953fff700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95415acd0 0x7ff954068810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff953fff700 1 -- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff954159850 msgr2=0x7ff954148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:35.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff953fff700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff954159850 0x7ff954148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:35.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff953fff700 1 -- 172.21.15.33:0/80549280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c009ae0 con 0x7ff95406efa0 2023-10-06T05:19:35.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff953fff700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 0x7ff9540cc8f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff94400c810 tx=0x7ff944007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944005a70 con 0x7ff95406efa0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944005bd0 con 0x7ff95406efa0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440070d0 con 0x7ff95406efa0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.528+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95416a110 con 0x7ff95406efa0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.529+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9540cc0f0 con 0x7ff95406efa0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.530+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9540cf2b0 con 0x7ff95406efa0 2023-10-06T05:19:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.531+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff944007480 con 0x7ff95406efa0 2023-10-06T05:19:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.532+0000 7ff93bfff700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff940066f30 0x7ff9400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.532+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff944083bf0 con 0x7ff95406efa0 2023-10-06T05:19:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.533+0000 7ff9537fe700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff940066f30 0x7ff9400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.533+0000 7ff9537fe700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff940066f30 0x7ff9400693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff948009d10 tx=0x7ff9480091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.536+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff944050c60 con 0x7ff95406efa0 2023-10-06T05:19:35.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:35.787+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff95404c380 con 0x7ff95406efa0 2023-10-06T05:19:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.309+0000 7ff93bfff700 1 -- 172.21.15.33:0/80549280 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff944017070 con 0x7ff95406efa0 2023-10-06T05:19:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.311+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff940066f30 msgr2=0x7ff9400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.311+0000 7ff95979d700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff940066f30 0x7ff9400693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff948009d10 tx=0x7ff9480091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.311+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 msgr2=0x7ff9540cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.311+0000 7ff95979d700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 0x7ff9540cc8f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff94400c810 tx=0x7ff944007e10 comp rx=0 tx=0).stop 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.311+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 shutdown_connections 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.311+0000 7ff95979d700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff940066f30 0x7ff9400693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.312+0000 7ff95979d700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff95415acd0 0x7ff954068810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.312+0000 7ff95979d700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95406efa0 0x7ff9540cc8f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.312+0000 7ff95979d700 1 --2- 172.21.15.33:0/80549280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff954159850 0x7ff954148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.312+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 >> 172.21.15.33:0/80549280 conn(0x7ff95406cd40 msgr2=0x7ff95415b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.312+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 shutdown_connections 2023-10-06T05:19:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.312+0000 7ff95979d700 1 -- 172.21.15.33:0/80549280 wait complete. 2023-10-06T05:19:36.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.521+0000 7f822c9fe700 1 Processor -- start 2023-10-06T05:19:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.521+0000 7f822c9fe700 1 -- start start 2023-10-06T05:19:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.522+0000 7f822c9fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82280c82a0 0x7f82281641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.522+0000 7f822c9fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822806c820 con 0x7f82280c6340 2023-10-06T05:19:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.522+0000 7f822c9fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822806b670 con 0x7f8228159520 2023-10-06T05:19:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.522+0000 7f822c9fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822806a160 con 0x7f82280c82a0 2023-10-06T05:19:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.522+0000 7f822659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f82280c6340 0x7f82280c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33210/0 (socket says 172.21.15.33:33210) 2023-10-06T05:19:36.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.522+0000 7f822659c700 1 -- 172.21.15.33:0/1990945580 learned_addr learned my addr 172.21.15.33:0/1990945580 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f822759e700 1 --2- 172.21.15.33:0/1990945580 >> v2:172.21.15.167:3300/0 conn(0x7f82280c82a0 0x7f82281641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616455362 0 0) 0x7f822806c820 con 0x7f82280c6340 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8210003570 con 0x7f82280c6340 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215611652 0 0) 0x7f8210003570 con 0x7f82280c6340 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 >> v2:172.21.15.167:3300/0 conn(0x7f82280c82a0 msgr2=0x7f82281641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 --2- 172.21.15.33:0/1990945580 >> v2:172.21.15.167:3300/0 conn(0x7f82280c82a0 0x7f82281641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 >> v1:172.21.15.167:6789/0 conn(0x7f8228159520 legacy=0x7f82280c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280c8800 con 0x7f82280c6340 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f821c001470 con 0x7f82280c6340 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 >> v1:172.21.15.33:6789/0 conn(0x7f82280c6340 legacy=0x7f82280c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210003ab0 0x7f8210003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8210004810 0x7f8210004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8210005210 0x7f821000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822806a160 con 0x7f8210003ab0 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821000c1b0 con 0x7f8210004810 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821000c370 con 0x7f8210005210 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.523+0000 7f8226d9d700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210003ab0 0x7f8210003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8210004810 0x7f8210004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822759e700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8210005210 0x7f821000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 -- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8210005210 msgr2=0x7f821000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8210005210 0x7f821000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 -- 172.21.15.33:0/1990945580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210003ab0 msgr2=0x7f8210003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210003ab0 0x7f8210003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 -- 172.21.15.33:0/1990945580 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210003570 con 0x7f8210004810 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f8226d9d700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210003ab0 0x7f8210003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822759e700 1 -- 172.21.15.33:0/1990945580 reap_dead start 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.524+0000 7f822659c700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8210004810 0x7f8210004c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c2a0 tx=0x7f821c001d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea01d7ec963f88f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00cb70 con 0x7f8210004810 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c00ccf0 con 0x7f8210004810 2023-10-06T05:19:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f8225d9b700 1 -- 172.21.15.33:0/1990945580 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c005b70 con 0x7f8210004810 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 -- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8210004810 msgr2=0x7f8210004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8210004810 0x7f8210004c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c2a0 tx=0x7f821c001d80 comp rx=0 tx=0).stop 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 -- 172.21.15.33:0/1990945580 shutdown_connections 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 --2- 172.21.15.33:0/1990945580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8210004810 0x7f8210004c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 -- 172.21.15.33:0/1990945580 >> 172.21.15.33:0/1990945580 conn(0x7f82280cc930 msgr2=0x7f82280ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 -- 172.21.15.33:0/1990945580 shutdown_connections 2023-10-06T05:19:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.525+0000 7f822c9fe700 1 -- 172.21.15.33:0/1990945580 wait complete. 2023-10-06T05:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.527+0000 7f822c9fe700 1 Processor -- start 2023-10-06T05:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.527+0000 7f822c9fe700 1 -- start start 2023-10-06T05:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822c9fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 0x7f82281696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822815d0f0 0x7f822815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822c9fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8228169c30 0x7f822815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822c9fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822815e190 con 0x7f8210004810 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822c9fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822815d580 con 0x7f8228169c30 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822815d860 con 0x7f822815d0f0 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f8226d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 0x7f82281696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f8226d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 0x7f82281696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40716/0 (socket says 172.21.15.33:40716) 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f8226d9d700 1 -- 172.21.15.33:0/4064379265 learned_addr learned my addr 172.21.15.33:0/4064379265 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822659c700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822815d0f0 0x7f822815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.528+0000 7f822759e700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8228169c30 0x7f822815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f8226d9d700 1 -- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822815d0f0 msgr2=0x7f822815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f8226d9d700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822815d0f0 0x7f822815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f8226d9d700 1 -- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8228169c30 msgr2=0x7f822815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f8226d9d700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8228169c30 0x7f822815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f8226d9d700 1 -- 172.21.15.33:0/4064379265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822815db40 con 0x7f8210004810 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f822759e700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8228169c30 0x7f822815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f8226d9d700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 0x7f82281696f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f82180078a0 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800fe90 con 0x7f8210004810 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821800d850 con 0x7f8210004810 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800dbf0 con 0x7f8210004810 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822815de20 con 0x7f8210004810 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.529+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822815e7e0 con 0x7f8210004810 2023-10-06T05:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.530+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822804c380 con 0x7f8210004810 2023-10-06T05:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.531+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f821801d460 con 0x7f8210004810 2023-10-06T05:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.532+0000 7f820effd700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8214066f30 0x7f82140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.532+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8218084960 con 0x7f8210004810 2023-10-06T05:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.533+0000 7f822659c700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8214066f30 0x7f82140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.533+0000 7f822659c700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8214066f30 0x7f82140693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f821c007c00 tx=0x7f821c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.536+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82180519d0 con 0x7f8210004810 2023-10-06T05:19:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.784+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f822815e470 con 0x7f8210004810 2023-10-06T05:19:36.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.784+0000 7f820effd700 1 -- 172.21.15.33:0/4064379265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f8218054ff0 con 0x7f8210004810 2023-10-06T05:19:36.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:36.786 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:36.302879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2242593899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2242593899},{"type":"v1","addr":"172.21.15.33:6835","nonce":2242593899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8214066f30 msgr2=0x7f82140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8214066f30 0x7f82140693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f821c007c00 tx=0x7f821c015000 comp rx=0 tx=0).stop 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 msgr2=0x7f82281696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 0x7f82281696f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f82180078a0 tx=0x7f8218009040 comp rx=0 tx=0).stop 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 shutdown_connections 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8214066f30 0x7f82140693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8228169c30 0x7f822815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8210004810 0x7f82281696f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 --2- 172.21.15.33:0/4064379265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f822815d0f0 0x7f822815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 >> 172.21.15.33:0/4064379265 conn(0x7f82280cc930 msgr2=0x7f82280ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 shutdown_connections 2023-10-06T05:19:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.787+0000 7f822c9fe700 1 -- 172.21.15.33:0/4064379265 wait complete. 2023-10-06T05:19:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 43 2023-10-06T05:19:36.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:19:36.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.991+0000 7f1ec299d700 1 Processor -- start 2023-10-06T05:19:36.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.991+0000 7f1ec299d700 1 -- start start 2023-10-06T05:19:36.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.992+0000 7f1ec299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:36.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.992+0000 7f1ec299d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0cbd80 con 0x7f1ebc0cf3b0 2023-10-06T05:19:36.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.992+0000 7f1ec299d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0af5f0 con 0x7f1ebc0cc750 2023-10-06T05:19:36.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.992+0000 7f1ec299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0af8d0 con 0x7f1ebc0ccfc0 2023-10-06T05:19:36.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec119a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1ebc0cf3b0 0x7f1ebc1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33220/0 (socket says 172.21.15.33:33220) 2023-10-06T05:19:36.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec119a700 1 -- 172.21.15.33:0/1159024431 learned_addr learned my addr 172.21.15.33:0/1159024431 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:36.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec199b700 1 --2- 172.21.15.33:0/1159024431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:36.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920285880 0 0) 0x7f1ebc0cbd80 con 0x7f1ebc0cf3b0 2023-10-06T05:19:36.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea8003570 con 0x7f1ebc0cf3b0 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478000586 0 0) 0x7f1ebc0af5f0 con 0x7f1ebc0cc750 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ebc0cbd80 con 0x7f1ebc0cc750 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec199b700 1 -- 172.21.15.33:0/1159024431 >> v1:172.21.15.167:6789/0 conn(0x7f1ebc0cc750 legacy=0x7f1ebc0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec199b700 1 -- 172.21.15.33:0/1159024431 >> v1:172.21.15.33:6789/0 conn(0x7f1ebc0cf3b0 legacy=0x7f1ebc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.993+0000 7f1ec199b700 1 -- 172.21.15.33:0/1159024431 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0afbb0 con 0x7f1ebc0ccfc0 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.994+0000 7f1ec199b700 1 --2- 172.21.15.33:0/1159024431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc0cc210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009910 tx=0x7f1eb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=244639166b836add server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.994+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8011630 con 0x7f1ebc0ccfc0 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.994+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8011790 con 0x7f1ebc0ccfc0 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.994+0000 7f1ec0999700 1 -- 172.21.15.33:0/1159024431 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8011b30 con 0x7f1ebc0ccfc0 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.994+0000 7f1ec299d700 1 -- 172.21.15.33:0/1159024431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 msgr2=0x7f1ebc0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.994+0000 7f1ec299d700 1 --2- 172.21.15.33:0/1159024431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc0cc210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009910 tx=0x7f1eb80091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.995+0000 7f1ec299d700 1 -- 172.21.15.33:0/1159024431 shutdown_connections 2023-10-06T05:19:36.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.995+0000 7f1ec299d700 1 --2- 172.21.15.33:0/1159024431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc0cc210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:36.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.995+0000 7f1ec299d700 1 -- 172.21.15.33:0/1159024431 >> 172.21.15.33:0/1159024431 conn(0x7f1ebc06cff0 msgr2=0x7f1ebc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:36.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.995+0000 7f1ec299d700 1 -- 172.21.15.33:0/1159024431 shutdown_connections 2023-10-06T05:19:36.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.995+0000 7f1ec299d700 1 -- 172.21.15.33:0/1159024431 wait complete. 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.996+0000 7f1ec299d700 1 Processor -- start 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.996+0000 7f1ec299d700 1 -- start start 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 0x7f1ebc1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0b3cb0 0x7f1ebc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc169f70 con 0x7f1ebc0cc750 2023-10-06T05:19:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c7d60 con 0x7f1ebc0b3cb0 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c7730 con 0x7f1ebc0ccfc0 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 0x7f1ebc1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 0x7f1ebc1459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40724/0 (socket says 172.21.15.33:40724) 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.997+0000 7f1ec199b700 1 -- 172.21.15.33:0/2376103509 learned_addr learned my addr 172.21.15.33:0/2376103509 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec119a700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec219c700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0b3cb0 0x7f1ebc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec199b700 1 -- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 msgr2=0x7f1ebc1483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec199b700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc1483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec199b700 1 -- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0b3cb0 msgr2=0x7f1ebc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec199b700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0b3cb0 0x7f1ebc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec199b700 1 -- 172.21.15.33:0/2376103509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8009420 con 0x7f1ebc0cc750 2023-10-06T05:19:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec199b700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 0x7f1ebc1459b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fd0 tx=0x7f1eb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb801f070 con 0x7f1ebc0cc750 2023-10-06T05:19:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8002f70 con 0x7f1ebc0cc750 2023-10-06T05:19:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8011900 con 0x7f1ebc0cc750 2023-10-06T05:19:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.998+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc0b4770 con 0x7f1ebc0cc750 2023-10-06T05:19:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:36.999+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc148910 con 0x7f1ebc0cc750 2023-10-06T05:19:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.000+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc04c380 con 0x7f1ebc0cc750 2023-10-06T05:19:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.000+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1eb8023070 con 0x7f1ebc0cc750 2023-10-06T05:19:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.001+0000 7f1eb1ffb700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ea0066f30 0x7f1ea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.001+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1eb80859f0 con 0x7f1ebc0cc750 2023-10-06T05:19:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.002+0000 7f1ec119a700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ea0066f30 0x7f1ea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.002+0000 7f1ec119a700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ea0066f30 0x7f1ea00693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0b62a0 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.005+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1eb8052a60 con 0x7f1ebc0cc750 2023-10-06T05:19:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.209+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ebc159bb0 con 0x7f1ebc0cc750 2023-10-06T05:19:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.210+0000 7f1eb1ffb700 1 -- 172.21.15.33:0/2376103509 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f1eb802a090 con 0x7f1ebc0cc750 2023-10-06T05:19:37.212 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:37.212 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":76,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:19:34.282039+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:19:30.956591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:19:31.773352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:19:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ea0066f30 msgr2=0x7f1ea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ea0066f30 0x7f1ea00693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0b62a0 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 msgr2=0x7f1ebc1459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 0x7f1ebc1459b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fd0 tx=0x7f1eb8017000 comp rx=0 tx=0).stop 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 shutdown_connections 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ea0066f30 0x7f1ea00693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ebc0b3cb0 0x7f1ebc0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc0cc750 0x7f1ebc1459b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 --2- 172.21.15.33:0/2376103509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ebc0ccfc0 0x7f1ebc1483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 >> 172.21.15.33:0/2376103509 conn(0x7f1ebc06cff0 msgr2=0x7f1ebc0c6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:37.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 shutdown_connections 2023-10-06T05:19:37.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.213+0000 7f1ec299d700 1 -- 172.21.15.33:0/2376103509 wait complete. 2023-10-06T05:19:37.232 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:19:37.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:19:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.418+0000 7ff4b599d700 1 Processor -- start 2023-10-06T05:19:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.418+0000 7ff4b599d700 1 -- start start 2023-10-06T05:19:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4b599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4b599d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b006e3e0 con 0x7ff4b006df70 2023-10-06T05:19:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4b599d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b006f220 con 0x7ff4b00b04e0 2023-10-06T05:19:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4b599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00b1e00 con 0x7ff4b0159530 2023-10-06T05:19:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4affff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff4b006df70 0x7ff4b00afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33236/0 (socket says 172.21.15.33:33236) 2023-10-06T05:19:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4affff700 1 -- 172.21.15.33:0/1121324904 learned_addr learned my addr 172.21.15.33:0/1121324904 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.420+0000 7ff4b499b700 1 --2- 172.21.15.33:0/1121324904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981844575 0 0) 0x7ff4b006e3e0 con 0x7ff4b006df70 2023-10-06T05:19:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a0003570 con 0x7ff4b006df70 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053624972 0 0) 0x7ff4b006f220 con 0x7ff4b00b04e0 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b006e3e0 con 0x7ff4b00b04e0 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4b499b700 1 -- 172.21.15.33:0/1121324904 >> v1:172.21.15.167:6789/0 conn(0x7ff4b00b04e0 legacy=0x7ff4b015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4b499b700 1 -- 172.21.15.33:0/1121324904 >> v1:172.21.15.33:6789/0 conn(0x7ff4b006df70 legacy=0x7ff4b00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2160161435 0 0) 0x7ff4a0003570 con 0x7ff4b006df70 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4b499b700 1 -- 172.21.15.33:0/1121324904 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b006e610 con 0x7ff4b0159530 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4b499b700 1 --2- 172.21.15.33:0/1121324904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b006da30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009910 tx=0x7ff4a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de000c0eb2329d8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c001470 con 0x7ff4b0159530 2023-10-06T05:19:37.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4011420 con 0x7ff4b0159530 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.421+0000 7ff4af7fe700 1 -- 172.21.15.33:0/1121324904 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40117c0 con 0x7ff4b0159530 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 -- 172.21.15.33:0/1121324904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 msgr2=0x7ff4b006da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1121324904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b006da30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009910 tx=0x7ff4a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 -- 172.21.15.33:0/1121324904 shutdown_connections 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1121324904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b006da30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 -- 172.21.15.33:0/1121324904 >> 172.21.15.33:0/1121324904 conn(0x7ff4b006c560 msgr2=0x7ff4b006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 -- 172.21.15.33:0/1121324904 shutdown_connections 2023-10-06T05:19:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.422+0000 7ff4b599d700 1 -- 172.21.15.33:0/1121324904 wait complete. 2023-10-06T05:19:37.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.424+0000 7ff4b599d700 1 Processor -- start 2023-10-06T05:19:37.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.424+0000 7ff4b599d700 1 -- start start 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b599d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 0x7ff4b0152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b599d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b00b04e0 0x7ff4b0155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b016db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b599d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b0155660 con 0x7ff4b006df70 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b599d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b016d6d0 con 0x7ff4b00b04e0 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b0153220 con 0x7ff4b0159530 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 0x7ff4b0152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 0x7ff4b0152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40732/0 (socket says 172.21.15.33:40732) 2023-10-06T05:19:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 -- 172.21.15.33:0/1227902816 learned_addr learned my addr 172.21.15.33:0/1227902816 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4affff700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b00b04e0 0x7ff4b0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b519c700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b016db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 -- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 msgr2=0x7ff4b016db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b016db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 -- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b00b04e0 msgr2=0x7ff4b0155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b00b04e0 0x7ff4b0155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.425+0000 7ff4b499b700 1 -- 172.21.15.33:0/1227902816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009420 con 0x7ff4b006df70 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.426+0000 7ff4b499b700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 0x7ff4b0152b20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40091e0 tx=0x7ff4a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.426+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a401cb30 con 0x7ff4b006df70 2023-10-06T05:19:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.426+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a401cc90 con 0x7ff4b006df70 2023-10-06T05:19:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.426+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a401a6a0 con 0x7ff4b006df70 2023-10-06T05:19:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.426+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b0153500 con 0x7ff4b006df70 2023-10-06T05:19:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.426+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b016e0a0 con 0x7ff4b006df70 2023-10-06T05:19:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.427+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b016d1c0 con 0x7ff4b006df70 2023-10-06T05:19:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.428+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4a401aa90 con 0x7ff4b006df70 2023-10-06T05:19:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.429+0000 7ff4acff9700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff494066f30 0x7ff4940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.429+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff4a4086480 con 0x7ff4b006df70 2023-10-06T05:19:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.430+0000 7ff4affff700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff494066f30 0x7ff4940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.430+0000 7ff4affff700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff494066f30 0x7ff4940693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff4b016d120 tx=0x7ff49c001620 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.433+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4b016d1c0 con 0x7ff4b006df70 2023-10-06T05:19:37.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.630+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4b016d1c0 con 0x7ff494066f30 2023-10-06T05:19:37.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.641+0000 7ff4acff9700 1 -- 172.21.15.33:0/1227902816 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299830 (secure 0 0 0) 0x7ff4b016d1c0 con 0x7ff494066f30 2023-10-06T05:19:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.645+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff494066f30 msgr2=0x7ff4940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff494066f30 0x7ff4940693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff4b016d120 tx=0x7ff49c001620 comp rx=0 tx=0).stop 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 msgr2=0x7ff4b0152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 0x7ff4b0152b20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40091e0 tx=0x7ff4a4016000 comp rx=0 tx=0).stop 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 shutdown_connections 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff494066f30 0x7ff4940693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b0159530 0x7ff4b016db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b006df70 0x7ff4b0152b20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 --2- 172.21.15.33:0/1227902816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b00b04e0 0x7ff4b0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 >> 172.21.15.33:0/1227902816 conn(0x7ff4b006c560 msgr2=0x7ff4b00c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 shutdown_connections 2023-10-06T05:19:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.646+0000 7ff4b599d700 1 -- 172.21.15.33:0/1227902816 wait complete. 2023-10-06T05:19:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:19:37.665 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2023-10-06T05:19:35.891809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2394552,"kb_used_data":8056,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752580168,"statfs":{"total":773094113280,"available":770642092032,"internally_reserved":0,"allocated":8249344,"data_stored":4304328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.618641"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.279487+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:15.390763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.280437+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:55.278266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.285489+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:57.546139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.276562+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:12.764037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.282802+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:14.668376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.290879+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:13.352226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.274777+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:52.246863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.279532+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:25.427815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.285398+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.615275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.291743+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:46.662138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.279864+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:32.986882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.276025+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:38.459103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.291789+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:19.528070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.280076+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:14.668376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.279738+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:04.458091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.274817+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:37.384528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.291772+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:00.364487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.274832+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:14.615275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.280311+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:14.892927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.280000+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:42.637527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.275759+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:54.375395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.279385+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:45.880231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.275759+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:04.071708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.277980+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:30.026577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.279257+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:39.958288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.606955+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:19:28.606955+0000","last_peered":"2023-10-06T05:19:28.606955+0000","last_clean":"2023-10-06T05:19:28.606955+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:19:28.606955+0000","last_undegraded":"2023-10-06T05:19:28.606955+0000","last_fullsized":"2023-10-06T05:19:28.606955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.275141+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:43.194390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.608255+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:19:28.608255+0000","last_peered":"2023-10-06T05:19:28.608255+0000","last_clean":"2023-10-06T05:19:28.608255+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:19:28.608255+0000","last_undegraded":"2023-10-06T05:19:28.608255+0000","last_fullsized":"2023-10-06T05:19:28.608255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.275155+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:27.402626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.607275+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:19:28.607275+0000","last_peered":"2023-10-06T05:19:28.607275+0000","last_clean":"2023-10-06T05:19:28.607275+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:19:28.607275+0000","last_undegraded":"2023-10-06T05:19:28.607275+0000","last_fullsized":"2023-10-06T05:19:28.607275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.275952+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:18.728829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.608278+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:19:28.608278+0000","last_peered":"2023-10-06T05:19:28.608278+0000","last_clean":"2023-10-06T05:19:28.608278+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:19:28.608278+0000","last_undegraded":"2023-10-06T05:19:28.608278+0000","last_fullsized":"2023-10-06T05:19:28.608278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.510946+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:19:28.510946+0000","last_peered":"2023-10-06T05:19:28.510946+0000","last_clean":"2023-10-06T05:19:28.510946+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:19:28.510946+0000","last_undegraded":"2023-10-06T05:19:28.510946+0000","last_fullsized":"2023-10-06T05:19:28.510946+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.275204+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:25.005344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.510885+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:19:28.510885+0000","last_peered":"2023-10-06T05:19:28.510885+0000","last_clean":"2023-10-06T05:19:28.510885+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:19:28.510885+0000","last_undegraded":"2023-10-06T05:19:28.510885+0000","last_fullsized":"2023-10-06T05:19:28.510885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.278004+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:30.867050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.510965+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:19:28.510965+0000","last_peered":"2023-10-06T05:19:28.510965+0000","last_clean":"2023-10-06T05:19:28.510965+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:19:28.510965+0000","last_undegraded":"2023-10-06T05:19:28.510965+0000","last_fullsized":"2023-10-06T05:19:28.510965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.280097+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:20.361257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.606930+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:19:28.606930+0000","last_peered":"2023-10-06T05:19:28.606930+0000","last_clean":"2023-10-06T05:19:28.606930+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:19:28.606930+0000","last_undegraded":"2023-10-06T05:19:28.606930+0000","last_fullsized":"2023-10-06T05:19:28.606930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:19:32.272370+0000","last_change":"2023-10-06T05:19:32.272370+0000","last_active":"2023-10-06T05:19:32.272370+0000","last_peered":"2023-10-06T05:19:32.272370+0000","last_clean":"2023-10-06T05:19:32.272370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:32.272370+0000","last_undegraded":"2023-10-06T05:19:32.272370+0000","last_fullsized":"2023-10-06T05:19:32.272370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:19:31.265938+0000","last_change":"2023-10-06T05:19:31.282656+0000","last_active":"2023-10-06T05:19:31.265938+0000","last_peered":"2023-10-06T05:19:31.265938+0000","last_clean":"2023-10-06T05:19:31.265938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:19:31.265938+0000","last_undegraded":"2023-10-06T05:19:31.265938+0000","last_fullsized":"2023-10-06T05:19:31.265938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:14.892927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":61,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-06T05:19:28.265230+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:19:28.265230+0000","last_peered":"2023-10-06T05:19:28.265230+0000","last_clean":"2023-10-06T05:19:28.265230+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:19:28.265230+0000","last_undegraded":"2023-10-06T05:19:28.265230+0000","last_fullsized":"2023-10-06T05:19:28.265230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298736,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073104,"statfs":{"total":96636764160,"available":96330858496,"internally_reserved":0,"allocated":1482752,"data_stored":983896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68100000000000005,"15min":0.64100000000000001},"min":{"1min":0.51300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.70999999999999996,"15min":0.68500000000000005},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.67400000000000004,"15min":0.621},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.58099999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.68000000000000005},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.82699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.751,"15min":0.78000000000000003},"min":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.117,"5min":1.202,"15min":1.202},"last":0.85699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.626,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2050000000000001,"5min":1.407,"15min":1.407},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.24,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.98899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":387370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.44800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92100000000000004,"5min":1.034,"15min":1.034},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.94099999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.625,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.52300000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72799999999999998,"15min":0.76800000000000002},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74299999999999999,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86299999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86499999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77000000000000002,"15min":0.81000000000000005},"min":{"1min":0.66200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64100000000000001,"15min":0.68300000000000005},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.998,"15min":0.998},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71499999999999997,"15min":0.78500000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":1.038,"15min":1.038},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.76600000000000001,"15min":0.85599999999999998},"min":{"1min":0.59099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.90200000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":387370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69799999999999995,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.58099999999999996},"min":{"1min":0.378,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71799999999999997,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.377},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.622},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89500000000000002,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70899999999999996,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.623},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73099999999999998,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.60799999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96799999999999997,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.83599999999999997},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.998,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.018,"5min":1.349,"15min":1.349},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79500000000000004,"15min":0.83899999999999997},"min":{"1min":0.61299999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97099999999999997,"5min":1.669,"15min":1.669},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82699999999999996,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98099999999999998,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83099999999999996,"15min":0.89700000000000002},"min":{"1min":0.625,"5min":0.503,"15min":0.503},"max":{"1min":0.995,"5min":1.573,"15min":1.573},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82399999999999995,"15min":0.874},"min":{"1min":0.65700000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.02,"5min":1.528,"15min":1.528},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.877},"min":{"1min":0.54600000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0249999999999999,"5min":1.75,"15min":1.75},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302820,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069020,"statfs":{"total":96636764160,"available":96326676480,"internally_reserved":0,"allocated":1470464,"data_stored":977738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71399999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77600000000000002,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.77200000000000002,"15min":0.74199999999999999},"min":{"1min":0.63700000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.79800000000000004,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.76400000000000001},"min":{"1min":0.56200000000000006,"5min":0.503,"15min":0.503},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78800000000000003,"15min":0.77700000000000002},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1679999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.82899999999999996,"15min":0.79500000000000004},"min":{"1min":0.68100000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.96299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.65900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.93799999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":888832,"data_stored":393528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.49099999999999999,"15min":0.45800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.74299999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78800000000000003,"15min":0.76400000000000001},"min":{"1min":0.64000000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.077,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.63300000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.044,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80800000000000005,"15min":0.80900000000000005},"min":{"1min":0.66200000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0009999999999999,"5min":1.319,"15min":1.319},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.66000000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1399999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.79100000000000004},"min":{"1min":0.70999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0860000000000001,"5min":1.141,"15min":1.141},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.69599999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0980000000000001,"5min":1.282,"15min":1.282},"last":0.629},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.69999999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0580000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.68100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":888832,"data_stored":393528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.72499999999999998,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.436},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.495,"15min":0.504},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72099999999999997,"5min":1.772,"15min":1.772},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":1.913,"15min":1.913},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.59199999999999997},"min":{"1min":0.42299999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.46700000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":1.954,"15min":1.954},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.629},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.75900000000000001,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.48399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0640000000000001,"5min":1.282,"15min":1.282},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1579999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1060000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.85399999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.343,"15min":1.343},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.61199999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.82099999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.83699999999999997,"15min":0.86399999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.129,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.65300000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.044,"5min":1.214,"15min":1.214},"last":0.746}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":888832,"data_stored":393528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.73499999999999999,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0249999999999999,"5min":1.073,"15min":1.073},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.51000000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.169,"15min":1.169},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.54500000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91200000000000003,"5min":1.117,"15min":1.117},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71699999999999997,"15min":0.72199999999999998},"min":{"1min":0.52400000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96999999999999997,"5min":1.002,"15min":1.002},"last":0.873}]},{"osd":3,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98799999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.996,"5min":1.196,"15min":1.196},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.72699999999999998},"min":{"1min":0.46400000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.81999999999999995,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.77000000000000002},"min":{"1min":0.46000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":1.375,"15min":1.375},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69099999999999995,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96899999999999997,"5min":1.004,"15min":1.004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92900000000000005,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.92100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.82699999999999996},"min":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.466,"15min":1.466},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.84699999999999998},"min":{"1min":0.53600000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.018,"5min":1.218,"15min":1.218},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":387370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71699999999999997,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93500000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.92800000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.038,"15min":1.038},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.73699999999999999,"15min":0.70299999999999996},"min":{"1min":0.54300000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.002,"5min":1.087,"15min":1.087},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.495,"5min":0.433,"15min":0.433},"max":{"1min":1.2769999999999999,"5min":1.278,"15min":1.278},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63800000000000001,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.76500000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82599999999999996,"5min":1.294,"15min":1.294},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65200000000000002,"15min":0.69099999999999995},"min":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71499999999999997,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.873,"5min":1.254,"15min":1.254},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.78600000000000003},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":1.337,"15min":1.337},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.78400000000000003},"min":{"1min":0.501,"5min":0.495,"15min":0.495},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.68700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:19:37.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:19:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.890+0000 7f2fa359d700 1 Processor -- start 2023-10-06T05:19:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.890+0000 7f2fa359d700 1 -- start start 2023-10-06T05:19:37.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 0x7f2f9c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa359d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f9c0c7940 0x7f2f9c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa359d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9c0c9cb0 con 0x7f2f9c06aad0 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa359d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9c06b660 con 0x7f2f9c0c7940 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0ca270 con 0x7f2f9c13faa0 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa259b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 0x7f2f9c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa1d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f9c0c7940 0x7f2f9c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa259b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 0x7f2f9c13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:39226/0 (socket says 172.21.15.33:39226) 2023-10-06T05:19:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa259b700 1 -- 172.21.15.33:0/2900551769 learned_addr learned my addr 172.21.15.33:0/2900551769 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:37.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.892+0000 7f2fa1d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f9c0c7940 0x7f2f9c0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40738/0 (socket says 172.21.15.33:40738) 2023-10-06T05:19:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa259b700 1 -- 172.21.15.33:0/2900551769 >> v2:172.21.15.33:3300/0 conn(0x7f2f9c0c7940 msgr2=0x7f2f9c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa259b700 1 --2- 172.21.15.33:0/2900551769 >> v2:172.21.15.33:3300/0 conn(0x7f2f9c0c7940 0x7f2f9c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa259b700 1 -- 172.21.15.33:0/2900551769 >> v1:172.21.15.33:6789/0 conn(0x7f2f9c06aad0 legacy=0x7f2f9c06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa259b700 1 -- 172.21.15.33:0/2900551769 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c06b840 con 0x7f2f9c13faa0 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa259b700 1 --2- 172.21.15.33:0/2900551769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 0x7f2f9c13fe70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f980099b0 tx=0x7f2f98009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb38887a0c3cfda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa1599700 1 -- 172.21.15.33:0/2900551769 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98009ea0 con 0x7f2f9c13faa0 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa1599700 1 -- 172.21.15.33:0/2900551769 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9800b860 con 0x7f2f9c13faa0 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.893+0000 7f2fa1599700 1 -- 172.21.15.33:0/2900551769 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800bc00 con 0x7f2f9c13faa0 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 -- 172.21.15.33:0/2900551769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 msgr2=0x7f2f9c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 --2- 172.21.15.33:0/2900551769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 0x7f2f9c13fe70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f980099b0 tx=0x7f2f98009250 comp rx=0 tx=0).stop 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 -- 172.21.15.33:0/2900551769 shutdown_connections 2023-10-06T05:19:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 --2- 172.21.15.33:0/2900551769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c13faa0 0x7f2f9c13fe70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 --2- 172.21.15.33:0/2900551769 >> v2:172.21.15.33:3300/0 conn(0x7f2f9c0c7940 0x7f2f9c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 -- 172.21.15.33:0/2900551769 >> 172.21.15.33:0/2900551769 conn(0x7f2f9c06d030 msgr2=0x7f2f9c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 -- 172.21.15.33:0/2900551769 shutdown_connections 2023-10-06T05:19:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.894+0000 7f2fa359d700 1 -- 172.21.15.33:0/2900551769 wait complete. 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.895+0000 7f2fa359d700 1 Processor -- start 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.895+0000 7f2fa359d700 1 -- start start 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.896+0000 7f2fa359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c06aad0 0x7f2f9c13e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.896+0000 7f2fa359d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9c0c7940 0x7f2f9c13ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.896+0000 7f2fa359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 0x7f2f9c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.896+0000 7f2fa359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0c3d80 con 0x7f2f9c13faa0 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.896+0000 7f2fa359d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1592e0 con 0x7f2f9c0c7940 2023-10-06T05:19:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.896+0000 7f2fa359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c169b70 con 0x7f2f9c06aad0 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 0x7f2f9c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 0x7f2f9c0af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40742/0 (socket says 172.21.15.33:40742) 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 -- 172.21.15.33:0/3887034463 learned_addr learned my addr 172.21.15.33:0/3887034463 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa259b700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c06aad0 0x7f2f9c13e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa1d9a700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9c0c7940 0x7f2f9c13ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 -- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c06aad0 msgr2=0x7f2f9c13e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c06aad0 0x7f2f9c13e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 -- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9c0c7940 msgr2=0x7f2f9c13ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9c0c7940 0x7f2f9c13ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.897+0000 7f2fa2d9c700 1 -- 172.21.15.33:0/3887034463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980094f0 con 0x7f2f9c13faa0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.898+0000 7f2fa2d9c700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 0x7f2f9c0af060 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c8e0 tx=0x7f2f94001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.898+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94012630 con 0x7f2f9c13faa0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.898+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f94012790 con 0x7f2f9c13faa0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.898+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9c0c7030 con 0x7f2f9c13faa0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.898+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94012b50 con 0x7f2f9c13faa0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.898+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c158c90 con 0x7f2f9c13faa0 2023-10-06T05:19:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.899+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c159470 con 0x7f2f9c13faa0 2023-10-06T05:19:37.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.900+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f9401a3a0 con 0x7f2f9c13faa0 2023-10-06T05:19:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.901+0000 7f2f927fc700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f80066f30 0x7f2f800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.901+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2f940848a0 con 0x7f2f9c13faa0 2023-10-06T05:19:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.901+0000 7f2fa259b700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f80066f30 0x7f2f800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.902+0000 7f2fa259b700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f80066f30 0x7f2f800693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2f980099b0 tx=0x7f2f98017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:37.905+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f94051910 con 0x7f2f9c13faa0 2023-10-06T05:19:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.103+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f9c06bb20 con 0x7f2f80066f30 2023-10-06T05:19:38.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.113+0000 7f2f927fc700 1 -- 172.21.15.33:0/3887034463 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310439 (secure 0 0 0) 0x7f2f9c06bb20 con 0x7f2f80066f30 2023-10-06T05:19:38.118 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f80066f30 msgr2=0x7f2f800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f80066f30 0x7f2f800693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2f980099b0 tx=0x7f2f98017000 comp rx=0 tx=0).stop 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 msgr2=0x7f2f9c0af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 0x7f2f9c0af060 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c8e0 tx=0x7f2f94001710 comp rx=0 tx=0).stop 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 shutdown_connections 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f80066f30 0x7f2f800693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9c13faa0 0x7f2f9c0af060 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f9c06aad0 0x7f2f9c13e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 --2- 172.21.15.33:0/3887034463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9c0c7940 0x7f2f9c13ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 >> 172.21.15.33:0/3887034463 conn(0x7f2f9c06d030 msgr2=0x7f2f9c0c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:38.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 shutdown_connections 2023-10-06T05:19:38.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.117+0000 7f2fa359d700 1 -- 172.21.15.33:0/3887034463 wait complete. 2023-10-06T05:19:38.122 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:19:38.136 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2023-10-06T05:19:37.892330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2394952,"kb_used_data":8456,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752579768,"statfs":{"total":773094113280,"available":770641682432,"internally_reserved":0,"allocated":8658944,"data_stored":4496556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001489"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.314321+0000","last_change":"2023-10-06T05:19:33.308925+0000","last_active":"2023-10-06T05:19:33.314321+0000","last_peered":"2023-10-06T05:19:33.314321+0000","last_clean":"2023-10-06T05:19:33.314321+0000","last_became_active":"2023-10-06T05:19:33.308565+0000","last_became_peered":"2023-10-06T05:19:33.308565+0000","last_unstale":"2023-10-06T05:19:33.314321+0000","last_undegraded":"2023-10-06T05:19:33.314321+0000","last_fullsized":"2023-10-06T05:19:33.314321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:57.760255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.320923+0000","last_change":"2023-10-06T05:19:33.313045+0000","last_active":"2023-10-06T05:19:33.320923+0000","last_peered":"2023-10-06T05:19:33.320923+0000","last_clean":"2023-10-06T05:19:33.320923+0000","last_became_active":"2023-10-06T05:19:33.312804+0000","last_became_peered":"2023-10-06T05:19:33.312804+0000","last_unstale":"2023-10-06T05:19:33.320923+0000","last_undegraded":"2023-10-06T05:19:33.320923+0000","last_fullsized":"2023-10-06T05:19:33.320923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:07.470200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318734+0000","last_change":"2023-10-06T05:19:33.312747+0000","last_active":"2023-10-06T05:19:33.318734+0000","last_peered":"2023-10-06T05:19:33.318734+0000","last_clean":"2023-10-06T05:19:33.318734+0000","last_became_active":"2023-10-06T05:19:33.312456+0000","last_became_peered":"2023-10-06T05:19:33.312456+0000","last_unstale":"2023-10-06T05:19:33.318734+0000","last_undegraded":"2023-10-06T05:19:33.318734+0000","last_fullsized":"2023-10-06T05:19:33.318734+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:53.253295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317260+0000","last_change":"2023-10-06T05:19:33.315140+0000","last_active":"2023-10-06T05:19:33.317260+0000","last_peered":"2023-10-06T05:19:33.317260+0000","last_clean":"2023-10-06T05:19:33.317260+0000","last_became_active":"2023-10-06T05:19:33.314805+0000","last_became_peered":"2023-10-06T05:19:33.314805+0000","last_unstale":"2023-10-06T05:19:33.317260+0000","last_undegraded":"2023-10-06T05:19:33.317260+0000","last_fullsized":"2023-10-06T05:19:33.317260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:43.055363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318123+0000","last_change":"2023-10-06T05:19:33.307859+0000","last_active":"2023-10-06T05:19:33.318123+0000","last_peered":"2023-10-06T05:19:33.318123+0000","last_clean":"2023-10-06T05:19:33.318123+0000","last_became_active":"2023-10-06T05:19:33.307464+0000","last_became_peered":"2023-10-06T05:19:33.307464+0000","last_unstale":"2023-10-06T05:19:33.318123+0000","last_undegraded":"2023-10-06T05:19:33.318123+0000","last_fullsized":"2023-10-06T05:19:33.318123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:39.465535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.320878+0000","last_change":"2023-10-06T05:19:33.315883+0000","last_active":"2023-10-06T05:19:33.320878+0000","last_peered":"2023-10-06T05:19:33.320878+0000","last_clean":"2023-10-06T05:19:33.320878+0000","last_became_active":"2023-10-06T05:19:33.315396+0000","last_became_peered":"2023-10-06T05:19:33.315396+0000","last_unstale":"2023-10-06T05:19:33.320878+0000","last_undegraded":"2023-10-06T05:19:33.320878+0000","last_fullsized":"2023-10-06T05:19:33.320878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:46.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.315854+0000","last_change":"2023-10-06T05:19:33.311373+0000","last_active":"2023-10-06T05:19:33.315854+0000","last_peered":"2023-10-06T05:19:33.315854+0000","last_clean":"2023-10-06T05:19:33.315854+0000","last_became_active":"2023-10-06T05:19:33.310991+0000","last_became_peered":"2023-10-06T05:19:33.310991+0000","last_unstale":"2023-10-06T05:19:33.315854+0000","last_undegraded":"2023-10-06T05:19:33.315854+0000","last_fullsized":"2023-10-06T05:19:33.315854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:14.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317498+0000","last_change":"2023-10-06T05:19:33.307511+0000","last_active":"2023-10-06T05:19:33.317498+0000","last_peered":"2023-10-06T05:19:33.317498+0000","last_clean":"2023-10-06T05:19:33.317498+0000","last_became_active":"2023-10-06T05:19:33.307208+0000","last_became_peered":"2023-10-06T05:19:33.307208+0000","last_unstale":"2023-10-06T05:19:33.317498+0000","last_undegraded":"2023-10-06T05:19:33.317498+0000","last_fullsized":"2023-10-06T05:19:33.317498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:53.309974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317799+0000","last_change":"2023-10-06T05:19:33.305612+0000","last_active":"2023-10-06T05:19:33.317799+0000","last_peered":"2023-10-06T05:19:33.317799+0000","last_clean":"2023-10-06T05:19:33.317799+0000","last_became_active":"2023-10-06T05:19:33.305340+0000","last_became_peered":"2023-10-06T05:19:33.305340+0000","last_unstale":"2023-10-06T05:19:33.317799+0000","last_undegraded":"2023-10-06T05:19:33.317799+0000","last_fullsized":"2023-10-06T05:19:33.317799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:44.200822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308251+0000","last_change":"2023-10-06T05:19:33.300384+0000","last_active":"2023-10-06T05:19:33.308251+0000","last_peered":"2023-10-06T05:19:33.308251+0000","last_clean":"2023-10-06T05:19:33.308251+0000","last_became_active":"2023-10-06T05:19:33.300054+0000","last_became_peered":"2023-10-06T05:19:33.300054+0000","last_unstale":"2023-10-06T05:19:33.308251+0000","last_undegraded":"2023-10-06T05:19:33.308251+0000","last_fullsized":"2023-10-06T05:19:33.308251+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:12.049971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.316644+0000","last_change":"2023-10-06T05:19:33.311548+0000","last_active":"2023-10-06T05:19:33.316644+0000","last_peered":"2023-10-06T05:19:33.316644+0000","last_clean":"2023-10-06T05:19:33.316644+0000","last_became_active":"2023-10-06T05:19:33.311305+0000","last_became_peered":"2023-10-06T05:19:33.311305+0000","last_unstale":"2023-10-06T05:19:33.316644+0000","last_undegraded":"2023-10-06T05:19:33.316644+0000","last_fullsized":"2023-10-06T05:19:33.316644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:38.977301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308414+0000","last_change":"2023-10-06T05:19:33.301542+0000","last_active":"2023-10-06T05:19:33.308414+0000","last_peered":"2023-10-06T05:19:33.308414+0000","last_clean":"2023-10-06T05:19:33.308414+0000","last_became_active":"2023-10-06T05:19:33.301255+0000","last_became_peered":"2023-10-06T05:19:33.301255+0000","last_unstale":"2023-10-06T05:19:33.308414+0000","last_undegraded":"2023-10-06T05:19:33.308414+0000","last_fullsized":"2023-10-06T05:19:33.308414+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:44.096526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318144+0000","last_change":"2023-10-06T05:19:33.308478+0000","last_active":"2023-10-06T05:19:33.318144+0000","last_peered":"2023-10-06T05:19:33.318144+0000","last_clean":"2023-10-06T05:19:33.318144+0000","last_became_active":"2023-10-06T05:19:33.307129+0000","last_became_peered":"2023-10-06T05:19:33.307129+0000","last_unstale":"2023-10-06T05:19:33.318144+0000","last_undegraded":"2023-10-06T05:19:33.318144+0000","last_fullsized":"2023-10-06T05:19:33.318144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:55.381827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.320899+0000","last_change":"2023-10-06T05:19:33.316399+0000","last_active":"2023-10-06T05:19:33.320899+0000","last_peered":"2023-10-06T05:19:33.320899+0000","last_clean":"2023-10-06T05:19:33.320899+0000","last_became_active":"2023-10-06T05:19:33.315664+0000","last_became_peered":"2023-10-06T05:19:33.315664+0000","last_unstale":"2023-10-06T05:19:33.320899+0000","last_undegraded":"2023-10-06T05:19:33.320899+0000","last_fullsized":"2023-10-06T05:19:33.320899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:46.602601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.310357+0000","last_change":"2023-10-06T05:19:33.307379+0000","last_active":"2023-10-06T05:19:33.310357+0000","last_peered":"2023-10-06T05:19:33.310357+0000","last_clean":"2023-10-06T05:19:33.310357+0000","last_became_active":"2023-10-06T05:19:33.306986+0000","last_became_peered":"2023-10-06T05:19:33.306986+0000","last_unstale":"2023-10-06T05:19:33.310357+0000","last_undegraded":"2023-10-06T05:19:33.310357+0000","last_fullsized":"2023-10-06T05:19:33.310357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:46.022387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.300548+0000","last_change":"2023-10-06T05:19:33.298456+0000","last_active":"2023-10-06T05:19:33.300548+0000","last_peered":"2023-10-06T05:19:33.300548+0000","last_clean":"2023-10-06T05:19:33.300548+0000","last_became_active":"2023-10-06T05:19:33.298113+0000","last_became_peered":"2023-10-06T05:19:33.298113+0000","last_unstale":"2023-10-06T05:19:33.300548+0000","last_undegraded":"2023-10-06T05:19:33.300548+0000","last_fullsized":"2023-10-06T05:19:33.300548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:15.673947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318948+0000","last_change":"2023-10-06T05:19:33.313153+0000","last_active":"2023-10-06T05:19:33.318948+0000","last_peered":"2023-10-06T05:19:33.318948+0000","last_clean":"2023-10-06T05:19:33.318948+0000","last_became_active":"2023-10-06T05:19:33.312589+0000","last_became_peered":"2023-10-06T05:19:33.312589+0000","last_unstale":"2023-10-06T05:19:33.318948+0000","last_undegraded":"2023-10-06T05:19:33.318948+0000","last_fullsized":"2023-10-06T05:19:33.318948+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:05.078140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.322237+0000","last_change":"2023-10-06T05:19:33.318474+0000","last_active":"2023-10-06T05:19:33.322237+0000","last_peered":"2023-10-06T05:19:33.322237+0000","last_clean":"2023-10-06T05:19:33.322237+0000","last_became_active":"2023-10-06T05:19:33.318154+0000","last_became_peered":"2023-10-06T05:19:33.318154+0000","last_unstale":"2023-10-06T05:19:33.322237+0000","last_undegraded":"2023-10-06T05:19:33.322237+0000","last_fullsized":"2023-10-06T05:19:33.322237+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:16.114953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.321001+0000","last_change":"2023-10-06T05:19:33.316456+0000","last_active":"2023-10-06T05:19:33.321001+0000","last_peered":"2023-10-06T05:19:33.321001+0000","last_clean":"2023-10-06T05:19:33.321001+0000","last_became_active":"2023-10-06T05:19:33.316101+0000","last_became_peered":"2023-10-06T05:19:33.316101+0000","last_unstale":"2023-10-06T05:19:33.321001+0000","last_undegraded":"2023-10-06T05:19:33.321001+0000","last_fullsized":"2023-10-06T05:19:33.321001+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:07.470200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.322346+0000","last_change":"2023-10-06T05:19:33.317151+0000","last_active":"2023-10-06T05:19:33.322346+0000","last_peered":"2023-10-06T05:19:33.322346+0000","last_clean":"2023-10-06T05:19:33.322346+0000","last_became_active":"2023-10-06T05:19:33.311158+0000","last_became_peered":"2023-10-06T05:19:33.311158+0000","last_unstale":"2023-10-06T05:19:33.322346+0000","last_undegraded":"2023-10-06T05:19:33.322346+0000","last_fullsized":"2023-10-06T05:19:33.322346+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:13.770470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318630+0000","last_change":"2023-10-06T05:19:33.313274+0000","last_active":"2023-10-06T05:19:33.318630+0000","last_peered":"2023-10-06T05:19:33.318630+0000","last_clean":"2023-10-06T05:19:33.318630+0000","last_became_active":"2023-10-06T05:19:33.313017+0000","last_became_peered":"2023-10-06T05:19:33.313017+0000","last_unstale":"2023-10-06T05:19:33.318630+0000","last_undegraded":"2023-10-06T05:19:33.318630+0000","last_fullsized":"2023-10-06T05:19:33.318630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:15.673947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318994+0000","last_change":"2023-10-06T05:19:33.314896+0000","last_active":"2023-10-06T05:19:33.318994+0000","last_peered":"2023-10-06T05:19:33.318994+0000","last_clean":"2023-10-06T05:19:33.318994+0000","last_became_active":"2023-10-06T05:19:33.314551+0000","last_became_peered":"2023-10-06T05:19:33.314551+0000","last_unstale":"2023-10-06T05:19:33.318994+0000","last_undegraded":"2023-10-06T05:19:33.318994+0000","last_fullsized":"2023-10-06T05:19:33.318994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:01.769827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318141+0000","last_change":"2023-10-06T05:19:33.307717+0000","last_active":"2023-10-06T05:19:33.318141+0000","last_peered":"2023-10-06T05:19:33.318141+0000","last_clean":"2023-10-06T05:19:33.318141+0000","last_became_active":"2023-10-06T05:19:33.307452+0000","last_became_peered":"2023-10-06T05:19:33.307452+0000","last_unstale":"2023-10-06T05:19:33.318141+0000","last_undegraded":"2023-10-06T05:19:33.318141+0000","last_fullsized":"2023-10-06T05:19:33.318141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:57.790840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308490+0000","last_change":"2023-10-06T05:19:33.301338+0000","last_active":"2023-10-06T05:19:33.308490+0000","last_peered":"2023-10-06T05:19:33.308490+0000","last_clean":"2023-10-06T05:19:33.308490+0000","last_became_active":"2023-10-06T05:19:33.301038+0000","last_became_peered":"2023-10-06T05:19:33.301038+0000","last_unstale":"2023-10-06T05:19:33.308490+0000","last_undegraded":"2023-10-06T05:19:33.308490+0000","last_fullsized":"2023-10-06T05:19:33.308490+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:10.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308358+0000","last_change":"2023-10-06T05:19:33.299986+0000","last_active":"2023-10-06T05:19:33.308358+0000","last_peered":"2023-10-06T05:19:33.308358+0000","last_clean":"2023-10-06T05:19:33.308358+0000","last_became_active":"2023-10-06T05:19:33.299677+0000","last_became_peered":"2023-10-06T05:19:33.299677+0000","last_unstale":"2023-10-06T05:19:33.308358+0000","last_undegraded":"2023-10-06T05:19:33.308358+0000","last_fullsized":"2023-10-06T05:19:33.308358+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:01.295805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308353+0000","last_change":"2023-10-06T05:19:33.301664+0000","last_active":"2023-10-06T05:19:33.308353+0000","last_peered":"2023-10-06T05:19:33.308353+0000","last_clean":"2023-10-06T05:19:33.308353+0000","last_became_active":"2023-10-06T05:19:33.301385+0000","last_became_peered":"2023-10-06T05:19:33.301385+0000","last_unstale":"2023-10-06T05:19:33.308353+0000","last_undegraded":"2023-10-06T05:19:33.308353+0000","last_fullsized":"2023-10-06T05:19:33.308353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:56.415829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.301266+0000","last_change":"2023-10-06T05:19:33.299173+0000","last_active":"2023-10-06T05:19:33.301266+0000","last_peered":"2023-10-06T05:19:33.301266+0000","last_clean":"2023-10-06T05:19:33.301266+0000","last_became_active":"2023-10-06T05:19:33.298864+0000","last_became_peered":"2023-10-06T05:19:33.298864+0000","last_unstale":"2023-10-06T05:19:33.301266+0000","last_undegraded":"2023-10-06T05:19:33.301266+0000","last_fullsized":"2023-10-06T05:19:33.301266+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:33.892061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317521+0000","last_change":"2023-10-06T05:19:33.312295+0000","last_active":"2023-10-06T05:19:33.317521+0000","last_peered":"2023-10-06T05:19:33.317521+0000","last_clean":"2023-10-06T05:19:33.317521+0000","last_became_active":"2023-10-06T05:19:33.310867+0000","last_became_peered":"2023-10-06T05:19:33.310867+0000","last_unstale":"2023-10-06T05:19:33.317521+0000","last_undegraded":"2023-10-06T05:19:33.317521+0000","last_fullsized":"2023-10-06T05:19:33.317521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:38.448214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.309160+0000","last_change":"2023-10-06T05:19:33.300539+0000","last_active":"2023-10-06T05:19:33.309160+0000","last_peered":"2023-10-06T05:19:33.309160+0000","last_clean":"2023-10-06T05:19:33.309160+0000","last_became_active":"2023-10-06T05:19:33.300290+0000","last_became_peered":"2023-10-06T05:19:33.300290+0000","last_unstale":"2023-10-06T05:19:33.309160+0000","last_undegraded":"2023-10-06T05:19:33.309160+0000","last_fullsized":"2023-10-06T05:19:33.309160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:53.952874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.296933+0000","last_change":"2023-10-06T05:19:33.294730+0000","last_active":"2023-10-06T05:19:33.296933+0000","last_peered":"2023-10-06T05:19:33.296933+0000","last_clean":"2023-10-06T05:19:33.296933+0000","last_became_active":"2023-10-06T05:19:33.294464+0000","last_became_peered":"2023-10-06T05:19:33.294464+0000","last_unstale":"2023-10-06T05:19:33.296933+0000","last_undegraded":"2023-10-06T05:19:33.296933+0000","last_fullsized":"2023-10-06T05:19:33.296933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:18.495881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317828+0000","last_change":"2023-10-06T05:19:33.308170+0000","last_active":"2023-10-06T05:19:33.317828+0000","last_peered":"2023-10-06T05:19:33.317828+0000","last_clean":"2023-10-06T05:19:33.317828+0000","last_became_active":"2023-10-06T05:19:33.307903+0000","last_became_peered":"2023-10-06T05:19:33.307903+0000","last_unstale":"2023-10-06T05:19:33.317828+0000","last_undegraded":"2023-10-06T05:19:33.317828+0000","last_fullsized":"2023-10-06T05:19:33.317828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:56.284698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.316580+0000","last_change":"2023-10-06T05:19:33.314801+0000","last_active":"2023-10-06T05:19:33.316580+0000","last_peered":"2023-10-06T05:19:33.316580+0000","last_clean":"2023-10-06T05:19:33.316580+0000","last_became_active":"2023-10-06T05:19:33.314551+0000","last_became_peered":"2023-10-06T05:19:33.314551+0000","last_unstale":"2023-10-06T05:19:33.316580+0000","last_undegraded":"2023-10-06T05:19:33.316580+0000","last_fullsized":"2023-10-06T05:19:33.316580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:32.197256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.322283+0000","last_change":"2023-10-06T05:19:33.319129+0000","last_active":"2023-10-06T05:19:33.322283+0000","last_peered":"2023-10-06T05:19:33.322283+0000","last_clean":"2023-10-06T05:19:33.322283+0000","last_became_active":"2023-10-06T05:19:33.318873+0000","last_became_peered":"2023-10-06T05:19:33.318873+0000","last_unstale":"2023-10-06T05:19:33.322283+0000","last_undegraded":"2023-10-06T05:19:33.322283+0000","last_fullsized":"2023-10-06T05:19:33.322283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:15.621708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"75'4","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.326819+0000","last_change":"2023-10-06T05:19:32.297507+0000","last_active":"2023-10-06T05:19:33.326819+0000","last_peered":"2023-10-06T05:19:33.326819+0000","last_clean":"2023-10-06T05:19:33.326819+0000","last_became_active":"2023-10-06T05:19:32.297131+0000","last_became_peered":"2023-10-06T05:19:32.297131+0000","last_unstale":"2023-10-06T05:19:33.326819+0000","last_undegraded":"2023-10-06T05:19:33.326819+0000","last_fullsized":"2023-10-06T05:19:33.326819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:26.883648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.304459+0000","last_change":"2023-10-06T05:19:33.297525+0000","last_active":"2023-10-06T05:19:33.304459+0000","last_peered":"2023-10-06T05:19:33.304459+0000","last_clean":"2023-10-06T05:19:33.304459+0000","last_became_active":"2023-10-06T05:19:33.297230+0000","last_became_peered":"2023-10-06T05:19:33.297230+0000","last_unstale":"2023-10-06T05:19:33.304459+0000","last_undegraded":"2023-10-06T05:19:33.304459+0000","last_fullsized":"2023-10-06T05:19:33.304459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:57.393956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.313891+0000","last_change":"2023-10-06T05:19:32.309652+0000","last_active":"2023-10-06T05:19:33.313891+0000","last_peered":"2023-10-06T05:19:33.313891+0000","last_clean":"2023-10-06T05:19:33.313891+0000","last_became_active":"2023-10-06T05:19:32.309243+0000","last_became_peered":"2023-10-06T05:19:32.309243+0000","last_unstale":"2023-10-06T05:19:33.313891+0000","last_undegraded":"2023-10-06T05:19:33.313891+0000","last_fullsized":"2023-10-06T05:19:33.313891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:52.246863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318059+0000","last_change":"2023-10-06T05:19:33.306006+0000","last_active":"2023-10-06T05:19:33.318059+0000","last_peered":"2023-10-06T05:19:33.318059+0000","last_clean":"2023-10-06T05:19:33.318059+0000","last_became_active":"2023-10-06T05:19:33.305522+0000","last_became_peered":"2023-10-06T05:19:33.305522+0000","last_unstale":"2023-10-06T05:19:33.318059+0000","last_undegraded":"2023-10-06T05:19:33.318059+0000","last_fullsized":"2023-10-06T05:19:33.318059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:26.011776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.313414+0000","last_change":"2023-10-06T05:19:32.307508+0000","last_active":"2023-10-06T05:19:33.313414+0000","last_peered":"2023-10-06T05:19:33.313414+0000","last_clean":"2023-10-06T05:19:33.313414+0000","last_became_active":"2023-10-06T05:19:32.307217+0000","last_became_peered":"2023-10-06T05:19:32.307217+0000","last_unstale":"2023-10-06T05:19:33.313414+0000","last_undegraded":"2023-10-06T05:19:33.313414+0000","last_fullsized":"2023-10-06T05:19:33.313414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:11.043539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318276+0000","last_change":"2023-10-06T05:19:33.309929+0000","last_active":"2023-10-06T05:19:33.318276+0000","last_peered":"2023-10-06T05:19:33.318276+0000","last_clean":"2023-10-06T05:19:33.318276+0000","last_became_active":"2023-10-06T05:19:33.309647+0000","last_became_peered":"2023-10-06T05:19:33.309647+0000","last_unstale":"2023-10-06T05:19:33.318276+0000","last_undegraded":"2023-10-06T05:19:33.318276+0000","last_fullsized":"2023-10-06T05:19:33.318276+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.596144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.310422+0000","last_change":"2023-10-06T05:19:32.305777+0000","last_active":"2023-10-06T05:19:33.310422+0000","last_peered":"2023-10-06T05:19:33.310422+0000","last_clean":"2023-10-06T05:19:33.310422+0000","last_became_active":"2023-10-06T05:19:32.305511+0000","last_became_peered":"2023-10-06T05:19:32.305511+0000","last_unstale":"2023-10-06T05:19:33.310422+0000","last_undegraded":"2023-10-06T05:19:33.310422+0000","last_fullsized":"2023-10-06T05:19:33.310422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:56.888883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.321909+0000","last_change":"2023-10-06T05:19:33.314644+0000","last_active":"2023-10-06T05:19:33.321909+0000","last_peered":"2023-10-06T05:19:33.321909+0000","last_clean":"2023-10-06T05:19:33.321909+0000","last_became_active":"2023-10-06T05:19:33.314271+0000","last_became_peered":"2023-10-06T05:19:33.314271+0000","last_unstale":"2023-10-06T05:19:33.321909+0000","last_undegraded":"2023-10-06T05:19:33.321909+0000","last_fullsized":"2023-10-06T05:19:33.321909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:55.627964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318557+0000","last_change":"2023-10-06T05:19:32.307678+0000","last_active":"2023-10-06T05:19:33.318557+0000","last_peered":"2023-10-06T05:19:33.318557+0000","last_clean":"2023-10-06T05:19:33.318557+0000","last_became_active":"2023-10-06T05:19:32.307365+0000","last_became_peered":"2023-10-06T05:19:32.307365+0000","last_unstale":"2023-10-06T05:19:33.318557+0000","last_undegraded":"2023-10-06T05:19:33.318557+0000","last_fullsized":"2023-10-06T05:19:33.318557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:55.278266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.316567+0000","last_change":"2023-10-06T05:19:33.310984+0000","last_active":"2023-10-06T05:19:33.316567+0000","last_peered":"2023-10-06T05:19:33.316567+0000","last_clean":"2023-10-06T05:19:33.316567+0000","last_became_active":"2023-10-06T05:19:33.310705+0000","last_became_peered":"2023-10-06T05:19:33.310705+0000","last_unstale":"2023-10-06T05:19:33.316567+0000","last_undegraded":"2023-10-06T05:19:33.316567+0000","last_fullsized":"2023-10-06T05:19:33.316567+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:24.083792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.323659+0000","last_change":"2023-10-06T05:19:32.302987+0000","last_active":"2023-10-06T05:19:33.323659+0000","last_peered":"2023-10-06T05:19:33.323659+0000","last_clean":"2023-10-06T05:19:33.323659+0000","last_became_active":"2023-10-06T05:19:32.302426+0000","last_became_peered":"2023-10-06T05:19:32.302426+0000","last_unstale":"2023-10-06T05:19:33.323659+0000","last_undegraded":"2023-10-06T05:19:33.323659+0000","last_fullsized":"2023-10-06T05:19:33.323659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:07.699762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318586+0000","last_change":"2023-10-06T05:19:33.312932+0000","last_active":"2023-10-06T05:19:33.318586+0000","last_peered":"2023-10-06T05:19:33.318586+0000","last_clean":"2023-10-06T05:19:33.318586+0000","last_became_active":"2023-10-06T05:19:33.312656+0000","last_became_peered":"2023-10-06T05:19:33.312656+0000","last_unstale":"2023-10-06T05:19:33.318586+0000","last_undegraded":"2023-10-06T05:19:33.318586+0000","last_fullsized":"2023-10-06T05:19:33.318586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:19.735261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.285801+0000","last_change":"2023-10-06T05:19:32.304052+0000","last_active":"2023-10-06T05:19:33.285801+0000","last_peered":"2023-10-06T05:19:33.285801+0000","last_clean":"2023-10-06T05:19:33.285801+0000","last_became_active":"2023-10-06T05:19:32.303777+0000","last_became_peered":"2023-10-06T05:19:32.303777+0000","last_unstale":"2023-10-06T05:19:33.285801+0000","last_undegraded":"2023-10-06T05:19:33.285801+0000","last_fullsized":"2023-10-06T05:19:33.285801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:11.043539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.316424+0000","last_change":"2023-10-06T05:19:33.311153+0000","last_active":"2023-10-06T05:19:33.316424+0000","last_peered":"2023-10-06T05:19:33.316424+0000","last_clean":"2023-10-06T05:19:33.316424+0000","last_became_active":"2023-10-06T05:19:33.310802+0000","last_became_peered":"2023-10-06T05:19:33.310802+0000","last_unstale":"2023-10-06T05:19:33.316424+0000","last_undegraded":"2023-10-06T05:19:33.316424+0000","last_fullsized":"2023-10-06T05:19:33.316424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:31.873482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.292686+0000","last_change":"2023-10-06T05:19:32.297440+0000","last_active":"2023-10-06T05:19:33.292686+0000","last_peered":"2023-10-06T05:19:33.292686+0000","last_clean":"2023-10-06T05:19:33.292686+0000","last_became_active":"2023-10-06T05:19:32.297172+0000","last_became_peered":"2023-10-06T05:19:32.297172+0000","last_unstale":"2023-10-06T05:19:33.292686+0000","last_undegraded":"2023-10-06T05:19:33.292686+0000","last_fullsized":"2023-10-06T05:19:33.292686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:20.944886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.301203+0000","last_change":"2023-10-06T05:19:33.291806+0000","last_active":"2023-10-06T05:19:33.301203+0000","last_peered":"2023-10-06T05:19:33.301203+0000","last_clean":"2023-10-06T05:19:33.301203+0000","last_became_active":"2023-10-06T05:19:33.291549+0000","last_became_peered":"2023-10-06T05:19:33.291549+0000","last_unstale":"2023-10-06T05:19:33.301203+0000","last_undegraded":"2023-10-06T05:19:33.301203+0000","last_fullsized":"2023-10-06T05:19:33.301203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:46.489275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.286834+0000","last_change":"2023-10-06T05:19:32.310093+0000","last_active":"2023-10-06T05:19:33.286834+0000","last_peered":"2023-10-06T05:19:33.286834+0000","last_clean":"2023-10-06T05:19:33.286834+0000","last_became_active":"2023-10-06T05:19:32.309674+0000","last_became_peered":"2023-10-06T05:19:32.309674+0000","last_unstale":"2023-10-06T05:19:33.286834+0000","last_undegraded":"2023-10-06T05:19:33.286834+0000","last_fullsized":"2023-10-06T05:19:33.286834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:52.524343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.316148+0000","last_change":"2023-10-06T05:19:33.313568+0000","last_active":"2023-10-06T05:19:33.316148+0000","last_peered":"2023-10-06T05:19:33.316148+0000","last_clean":"2023-10-06T05:19:33.316148+0000","last_became_active":"2023-10-06T05:19:33.313270+0000","last_became_peered":"2023-10-06T05:19:33.313270+0000","last_unstale":"2023-10-06T05:19:33.316148+0000","last_undegraded":"2023-10-06T05:19:33.316148+0000","last_fullsized":"2023-10-06T05:19:33.316148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:16.114953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.293454+0000","last_change":"2023-10-06T05:19:32.305424+0000","last_active":"2023-10-06T05:19:33.293454+0000","last_peered":"2023-10-06T05:19:33.293454+0000","last_clean":"2023-10-06T05:19:33.293454+0000","last_became_active":"2023-10-06T05:19:32.305014+0000","last_became_peered":"2023-10-06T05:19:32.305014+0000","last_unstale":"2023-10-06T05:19:33.293454+0000","last_undegraded":"2023-10-06T05:19:33.293454+0000","last_fullsized":"2023-10-06T05:19:33.293454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:33.170277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.309294+0000","last_change":"2023-10-06T05:19:33.303174+0000","last_active":"2023-10-06T05:19:33.309294+0000","last_peered":"2023-10-06T05:19:33.309294+0000","last_clean":"2023-10-06T05:19:33.309294+0000","last_became_active":"2023-10-06T05:19:33.302910+0000","last_became_peered":"2023-10-06T05:19:33.302910+0000","last_unstale":"2023-10-06T05:19:33.309294+0000","last_undegraded":"2023-10-06T05:19:33.309294+0000","last_fullsized":"2023-10-06T05:19:33.309294+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:53.952874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"75'3","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.326468+0000","last_change":"2023-10-06T05:19:32.293315+0000","last_active":"2023-10-06T05:19:33.326468+0000","last_peered":"2023-10-06T05:19:33.326468+0000","last_clean":"2023-10-06T05:19:33.326468+0000","last_became_active":"2023-10-06T05:19:32.293037+0000","last_became_peered":"2023-10-06T05:19:32.293037+0000","last_unstale":"2023-10-06T05:19:33.326468+0000","last_undegraded":"2023-10-06T05:19:33.326468+0000","last_fullsized":"2023-10-06T05:19:33.326468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:39.966948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317812+0000","last_change":"2023-10-06T05:19:33.307416+0000","last_active":"2023-10-06T05:19:33.317812+0000","last_peered":"2023-10-06T05:19:33.317812+0000","last_clean":"2023-10-06T05:19:33.317812+0000","last_became_active":"2023-10-06T05:19:33.307167+0000","last_became_peered":"2023-10-06T05:19:33.307167+0000","last_unstale":"2023-10-06T05:19:33.317812+0000","last_undegraded":"2023-10-06T05:19:33.317812+0000","last_fullsized":"2023-10-06T05:19:33.317812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:41.297753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"75'6","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.321373+0000","last_change":"2023-10-06T05:19:32.300608+0000","last_active":"2023-10-06T05:19:33.321373+0000","last_peered":"2023-10-06T05:19:33.321373+0000","last_clean":"2023-10-06T05:19:33.321373+0000","last_became_active":"2023-10-06T05:19:32.300287+0000","last_became_peered":"2023-10-06T05:19:32.300287+0000","last_unstale":"2023-10-06T05:19:33.321373+0000","last_undegraded":"2023-10-06T05:19:33.321373+0000","last_fullsized":"2023-10-06T05:19:33.321373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:03.151544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308603+0000","last_change":"2023-10-06T05:19:33.303883+0000","last_active":"2023-10-06T05:19:33.308603+0000","last_peered":"2023-10-06T05:19:33.308603+0000","last_clean":"2023-10-06T05:19:33.308603+0000","last_became_active":"2023-10-06T05:19:33.303586+0000","last_became_peered":"2023-10-06T05:19:33.303586+0000","last_unstale":"2023-10-06T05:19:33.308603+0000","last_undegraded":"2023-10-06T05:19:33.308603+0000","last_fullsized":"2023-10-06T05:19:33.308603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:53.530775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.319053+0000","last_change":"2023-10-06T05:19:32.304799+0000","last_active":"2023-10-06T05:19:33.319053+0000","last_peered":"2023-10-06T05:19:33.319053+0000","last_clean":"2023-10-06T05:19:33.319053+0000","last_became_active":"2023-10-06T05:19:32.304447+0000","last_became_peered":"2023-10-06T05:19:32.304447+0000","last_unstale":"2023-10-06T05:19:33.319053+0000","last_undegraded":"2023-10-06T05:19:33.319053+0000","last_fullsized":"2023-10-06T05:19:33.319053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:14.892927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.308339+0000","last_change":"2023-10-06T05:19:33.305940+0000","last_active":"2023-10-06T05:19:33.308339+0000","last_peered":"2023-10-06T05:19:33.308339+0000","last_clean":"2023-10-06T05:19:33.308339+0000","last_became_active":"2023-10-06T05:19:33.304850+0000","last_became_peered":"2023-10-06T05:19:33.304850+0000","last_unstale":"2023-10-06T05:19:33.308339+0000","last_undegraded":"2023-10-06T05:19:33.308339+0000","last_fullsized":"2023-10-06T05:19:33.308339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:20.057364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.286458+0000","last_change":"2023-10-06T05:19:32.308383+0000","last_active":"2023-10-06T05:19:33.286458+0000","last_peered":"2023-10-06T05:19:33.286458+0000","last_clean":"2023-10-06T05:19:33.286458+0000","last_became_active":"2023-10-06T05:19:32.308117+0000","last_became_peered":"2023-10-06T05:19:32.308117+0000","last_unstale":"2023-10-06T05:19:33.286458+0000","last_undegraded":"2023-10-06T05:19:33.286458+0000","last_fullsized":"2023-10-06T05:19:33.286458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:37.384528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.309206+0000","last_change":"2023-10-06T05:19:33.307341+0000","last_active":"2023-10-06T05:19:33.309206+0000","last_peered":"2023-10-06T05:19:33.309206+0000","last_clean":"2023-10-06T05:19:33.309206+0000","last_became_active":"2023-10-06T05:19:33.307036+0000","last_became_peered":"2023-10-06T05:19:33.307036+0000","last_unstale":"2023-10-06T05:19:33.309206+0000","last_undegraded":"2023-10-06T05:19:33.309206+0000","last_fullsized":"2023-10-06T05:19:33.309206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:21.934541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.294905+0000","last_change":"2023-10-06T05:19:32.300431+0000","last_active":"2023-10-06T05:19:33.294905+0000","last_peered":"2023-10-06T05:19:33.294905+0000","last_clean":"2023-10-06T05:19:33.294905+0000","last_became_active":"2023-10-06T05:19:32.297758+0000","last_became_peered":"2023-10-06T05:19:32.297758+0000","last_unstale":"2023-10-06T05:19:33.294905+0000","last_undegraded":"2023-10-06T05:19:33.294905+0000","last_fullsized":"2023-10-06T05:19:33.294905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:53.370267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.309300+0000","last_change":"2023-10-06T05:19:33.307274+0000","last_active":"2023-10-06T05:19:33.309300+0000","last_peered":"2023-10-06T05:19:33.309300+0000","last_clean":"2023-10-06T05:19:33.309300+0000","last_became_active":"2023-10-06T05:19:33.306990+0000","last_became_peered":"2023-10-06T05:19:33.306990+0000","last_unstale":"2023-10-06T05:19:33.309300+0000","last_undegraded":"2023-10-06T05:19:33.309300+0000","last_fullsized":"2023-10-06T05:19:33.309300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:07.806789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"75'8","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.309687+0000","last_change":"2023-10-06T05:19:32.298979+0000","last_active":"2023-10-06T05:19:33.309687+0000","last_peered":"2023-10-06T05:19:33.309687+0000","last_clean":"2023-10-06T05:19:33.309687+0000","last_became_active":"2023-10-06T05:19:32.298118+0000","last_became_peered":"2023-10-06T05:19:32.298118+0000","last_unstale":"2023-10-06T05:19:33.309687+0000","last_undegraded":"2023-10-06T05:19:33.309687+0000","last_fullsized":"2023-10-06T05:19:33.309687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:57.546139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318220+0000","last_change":"2023-10-06T05:19:33.306913+0000","last_active":"2023-10-06T05:19:33.318220+0000","last_peered":"2023-10-06T05:19:33.318220+0000","last_clean":"2023-10-06T05:19:33.318220+0000","last_became_active":"2023-10-06T05:19:33.306372+0000","last_became_peered":"2023-10-06T05:19:33.306372+0000","last_unstale":"2023-10-06T05:19:33.318220+0000","last_undegraded":"2023-10-06T05:19:33.318220+0000","last_fullsized":"2023-10-06T05:19:33.318220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:28.409058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.322273+0000","last_change":"2023-10-06T05:19:32.301044+0000","last_active":"2023-10-06T05:19:33.322273+0000","last_peered":"2023-10-06T05:19:33.322273+0000","last_clean":"2023-10-06T05:19:33.322273+0000","last_became_active":"2023-10-06T05:19:32.300727+0000","last_became_peered":"2023-10-06T05:19:32.300727+0000","last_unstale":"2023-10-06T05:19:33.322273+0000","last_undegraded":"2023-10-06T05:19:33.322273+0000","last_fullsized":"2023-10-06T05:19:33.322273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:42.637527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.304981+0000","last_change":"2023-10-06T05:19:33.298523+0000","last_active":"2023-10-06T05:19:33.304981+0000","last_peered":"2023-10-06T05:19:33.304981+0000","last_clean":"2023-10-06T05:19:33.304981+0000","last_became_active":"2023-10-06T05:19:33.298061+0000","last_became_peered":"2023-10-06T05:19:33.298061+0000","last_unstale":"2023-10-06T05:19:33.304981+0000","last_undegraded":"2023-10-06T05:19:33.304981+0000","last_fullsized":"2023-10-06T05:19:33.304981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:36.459476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.286646+0000","last_change":"2023-10-06T05:19:32.302153+0000","last_active":"2023-10-06T05:19:33.286646+0000","last_peered":"2023-10-06T05:19:33.286646+0000","last_clean":"2023-10-06T05:19:33.286646+0000","last_became_active":"2023-10-06T05:19:32.301887+0000","last_became_peered":"2023-10-06T05:19:32.301887+0000","last_unstale":"2023-10-06T05:19:33.286646+0000","last_undegraded":"2023-10-06T05:19:33.286646+0000","last_fullsized":"2023-10-06T05:19:33.286646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:15.108521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318013+0000","last_change":"2023-10-06T05:19:33.305792+0000","last_active":"2023-10-06T05:19:33.318013+0000","last_peered":"2023-10-06T05:19:33.318013+0000","last_clean":"2023-10-06T05:19:33.318013+0000","last_became_active":"2023-10-06T05:19:33.305150+0000","last_became_peered":"2023-10-06T05:19:33.305150+0000","last_unstale":"2023-10-06T05:19:33.318013+0000","last_undegraded":"2023-10-06T05:19:33.318013+0000","last_fullsized":"2023-10-06T05:19:33.318013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:44.096526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.288044+0000","last_change":"2023-10-06T05:19:32.311240+0000","last_active":"2023-10-06T05:19:33.288044+0000","last_peered":"2023-10-06T05:19:33.288044+0000","last_clean":"2023-10-06T05:19:33.288044+0000","last_became_active":"2023-10-06T05:19:32.310754+0000","last_became_peered":"2023-10-06T05:19:32.310754+0000","last_unstale":"2023-10-06T05:19:33.288044+0000","last_undegraded":"2023-10-06T05:19:33.288044+0000","last_fullsized":"2023-10-06T05:19:33.288044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:42.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.293046+0000","last_change":"2023-10-06T05:19:33.291211+0000","last_active":"2023-10-06T05:19:33.293046+0000","last_peered":"2023-10-06T05:19:33.293046+0000","last_clean":"2023-10-06T05:19:33.293046+0000","last_became_active":"2023-10-06T05:19:33.290955+0000","last_became_peered":"2023-10-06T05:19:33.290955+0000","last_unstale":"2023-10-06T05:19:33.293046+0000","last_undegraded":"2023-10-06T05:19:33.293046+0000","last_fullsized":"2023-10-06T05:19:33.293046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:17.965182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.286866+0000","last_change":"2023-10-06T05:19:32.309776+0000","last_active":"2023-10-06T05:19:33.286866+0000","last_peered":"2023-10-06T05:19:33.286866+0000","last_clean":"2023-10-06T05:19:33.286866+0000","last_became_active":"2023-10-06T05:19:32.309339+0000","last_became_peered":"2023-10-06T05:19:32.309339+0000","last_unstale":"2023-10-06T05:19:33.286866+0000","last_undegraded":"2023-10-06T05:19:33.286866+0000","last_fullsized":"2023-10-06T05:19:33.286866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:40.291321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.303708+0000","last_change":"2023-10-06T05:19:33.298749+0000","last_active":"2023-10-06T05:19:33.303708+0000","last_peered":"2023-10-06T05:19:33.303708+0000","last_clean":"2023-10-06T05:19:33.303708+0000","last_became_active":"2023-10-06T05:19:33.298488+0000","last_became_peered":"2023-10-06T05:19:33.298488+0000","last_unstale":"2023-10-06T05:19:33.303708+0000","last_undegraded":"2023-10-06T05:19:33.303708+0000","last_fullsized":"2023-10-06T05:19:33.303708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:00.493326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.290165+0000","last_change":"2023-10-06T05:19:32.306525+0000","last_active":"2023-10-06T05:19:33.290165+0000","last_peered":"2023-10-06T05:19:33.290165+0000","last_clean":"2023-10-06T05:19:33.290165+0000","last_became_active":"2023-10-06T05:19:32.306225+0000","last_became_peered":"2023-10-06T05:19:32.306225+0000","last_unstale":"2023-10-06T05:19:33.290165+0000","last_undegraded":"2023-10-06T05:19:33.290165+0000","last_fullsized":"2023-10-06T05:19:33.290165+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:11.651277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.315788+0000","last_change":"2023-10-06T05:19:33.311217+0000","last_active":"2023-10-06T05:19:33.315788+0000","last_peered":"2023-10-06T05:19:33.315788+0000","last_clean":"2023-10-06T05:19:33.315788+0000","last_became_active":"2023-10-06T05:19:33.310881+0000","last_became_peered":"2023-10-06T05:19:33.310881+0000","last_unstale":"2023-10-06T05:19:33.315788+0000","last_undegraded":"2023-10-06T05:19:33.315788+0000","last_fullsized":"2023-10-06T05:19:33.315788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:31.033010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.300525+0000","last_change":"2023-10-06T05:19:32.300076+0000","last_active":"2023-10-06T05:19:33.300525+0000","last_peered":"2023-10-06T05:19:33.300525+0000","last_clean":"2023-10-06T05:19:33.300525+0000","last_became_active":"2023-10-06T05:19:32.297674+0000","last_became_peered":"2023-10-06T05:19:32.297674+0000","last_unstale":"2023-10-06T05:19:33.300525+0000","last_undegraded":"2023-10-06T05:19:33.300525+0000","last_fullsized":"2023-10-06T05:19:33.300525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:04.679607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.318720+0000","last_change":"2023-10-06T05:19:33.312631+0000","last_active":"2023-10-06T05:19:33.318720+0000","last_peered":"2023-10-06T05:19:33.318720+0000","last_clean":"2023-10-06T05:19:33.318720+0000","last_became_active":"2023-10-06T05:19:33.312373+0000","last_became_peered":"2023-10-06T05:19:33.312373+0000","last_unstale":"2023-10-06T05:19:33.318720+0000","last_undegraded":"2023-10-06T05:19:33.318720+0000","last_fullsized":"2023-10-06T05:19:33.318720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:38.390961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.290656+0000","last_change":"2023-10-06T05:19:32.299279+0000","last_active":"2023-10-06T05:19:33.290656+0000","last_peered":"2023-10-06T05:19:33.290656+0000","last_clean":"2023-10-06T05:19:33.290656+0000","last_became_active":"2023-10-06T05:19:32.298988+0000","last_became_peered":"2023-10-06T05:19:32.298988+0000","last_unstale":"2023-10-06T05:19:33.290656+0000","last_undegraded":"2023-10-06T05:19:33.290656+0000","last_fullsized":"2023-10-06T05:19:33.290656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:08.989668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.304215+0000","last_change":"2023-10-06T05:19:33.303185+0000","last_active":"2023-10-06T05:19:33.304215+0000","last_peered":"2023-10-06T05:19:33.304215+0000","last_clean":"2023-10-06T05:19:33.304215+0000","last_became_active":"2023-10-06T05:19:33.302904+0000","last_became_peered":"2023-10-06T05:19:33.302904+0000","last_unstale":"2023-10-06T05:19:33.304215+0000","last_undegraded":"2023-10-06T05:19:33.304215+0000","last_fullsized":"2023-10-06T05:19:33.304215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:58.552571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.295166+0000","last_change":"2023-10-06T05:19:32.295423+0000","last_active":"2023-10-06T05:19:33.295166+0000","last_peered":"2023-10-06T05:19:33.295166+0000","last_clean":"2023-10-06T05:19:33.295166+0000","last_became_active":"2023-10-06T05:19:32.295172+0000","last_became_peered":"2023-10-06T05:19:32.295172+0000","last_unstale":"2023-10-06T05:19:33.295166+0000","last_undegraded":"2023-10-06T05:19:33.295166+0000","last_fullsized":"2023-10-06T05:19:33.295166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:13.352226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.317435+0000","last_change":"2023-10-06T05:19:33.308029+0000","last_active":"2023-10-06T05:19:33.317435+0000","last_peered":"2023-10-06T05:19:33.317435+0000","last_clean":"2023-10-06T05:19:33.317435+0000","last_became_active":"2023-10-06T05:19:33.303757+0000","last_became_peered":"2023-10-06T05:19:33.303757+0000","last_unstale":"2023-10-06T05:19:33.317435+0000","last_undegraded":"2023-10-06T05:19:33.317435+0000","last_fullsized":"2023-10-06T05:19:33.317435+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:24.696507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.326529+0000","last_change":"2023-10-06T05:19:32.301699+0000","last_active":"2023-10-06T05:19:33.326529+0000","last_peered":"2023-10-06T05:19:33.326529+0000","last_clean":"2023-10-06T05:19:33.326529+0000","last_became_active":"2023-10-06T05:19:32.301441+0000","last_became_peered":"2023-10-06T05:19:32.301441+0000","last_unstale":"2023-10-06T05:19:33.326529+0000","last_undegraded":"2023-10-06T05:19:33.326529+0000","last_fullsized":"2023-10-06T05:19:33.326529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:20.482395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.295514+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:19:33.295514+0000","last_peered":"2023-10-06T05:19:33.295514+0000","last_clean":"2023-10-06T05:19:33.295514+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:19:33.295514+0000","last_undegraded":"2023-10-06T05:19:33.295514+0000","last_fullsized":"2023-10-06T05:19:33.295514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.309893+0000","last_change":"2023-10-06T05:19:33.303055+0000","last_active":"2023-10-06T05:19:33.309893+0000","last_peered":"2023-10-06T05:19:33.309893+0000","last_clean":"2023-10-06T05:19:33.309893+0000","last_became_active":"2023-10-06T05:19:33.302803+0000","last_became_peered":"2023-10-06T05:19:33.302803+0000","last_unstale":"2023-10-06T05:19:33.309893+0000","last_undegraded":"2023-10-06T05:19:33.309893+0000","last_fullsized":"2023-10-06T05:19:33.309893+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:03.200143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.320994+0000","last_change":"2023-10-06T05:19:32.293558+0000","last_active":"2023-10-06T05:19:33.320994+0000","last_peered":"2023-10-06T05:19:33.320994+0000","last_clean":"2023-10-06T05:19:33.320994+0000","last_became_active":"2023-10-06T05:19:32.293256+0000","last_became_peered":"2023-10-06T05:19:32.293256+0000","last_unstale":"2023-10-06T05:19:33.320994+0000","last_undegraded":"2023-10-06T05:19:33.320994+0000","last_fullsized":"2023-10-06T05:19:33.320994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:31.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.291427+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:19:33.291427+0000","last_peered":"2023-10-06T05:19:33.291427+0000","last_clean":"2023-10-06T05:19:33.291427+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:19:33.291427+0000","last_undegraded":"2023-10-06T05:19:33.291427+0000","last_fullsized":"2023-10-06T05:19:33.291427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.307373+0000","last_change":"2023-10-06T05:19:33.300362+0000","last_active":"2023-10-06T05:19:33.307373+0000","last_peered":"2023-10-06T05:19:33.307373+0000","last_clean":"2023-10-06T05:19:33.307373+0000","last_became_active":"2023-10-06T05:19:33.299980+0000","last_became_peered":"2023-10-06T05:19:33.299980+0000","last_unstale":"2023-10-06T05:19:33.307373+0000","last_undegraded":"2023-10-06T05:19:33.307373+0000","last_fullsized":"2023-10-06T05:19:33.307373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:16.584882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.287625+0000","last_change":"2023-10-06T05:19:32.301159+0000","last_active":"2023-10-06T05:19:33.287625+0000","last_peered":"2023-10-06T05:19:33.287625+0000","last_clean":"2023-10-06T05:19:33.287625+0000","last_became_active":"2023-10-06T05:19:32.300868+0000","last_became_peered":"2023-10-06T05:19:32.300868+0000","last_unstale":"2023-10-06T05:19:33.287625+0000","last_undegraded":"2023-10-06T05:19:33.287625+0000","last_fullsized":"2023-10-06T05:19:33.287625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:31.589711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.294992+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:19:33.294992+0000","last_peered":"2023-10-06T05:19:33.294992+0000","last_clean":"2023-10-06T05:19:33.294992+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:19:33.294992+0000","last_undegraded":"2023-10-06T05:19:33.294992+0000","last_fullsized":"2023-10-06T05:19:33.294992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.313463+0000","last_change":"2023-10-06T05:19:33.308373+0000","last_active":"2023-10-06T05:19:33.313463+0000","last_peered":"2023-10-06T05:19:33.313463+0000","last_clean":"2023-10-06T05:19:33.313463+0000","last_became_active":"2023-10-06T05:19:33.307863+0000","last_became_peered":"2023-10-06T05:19:33.307863+0000","last_unstale":"2023-10-06T05:19:33.313463+0000","last_undegraded":"2023-10-06T05:19:33.313463+0000","last_fullsized":"2023-10-06T05:19:33.313463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.596144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.288405+0000","last_change":"2023-10-06T05:19:32.301178+0000","last_active":"2023-10-06T05:19:33.288405+0000","last_peered":"2023-10-06T05:19:33.288405+0000","last_clean":"2023-10-06T05:19:33.288405+0000","last_became_active":"2023-10-06T05:19:32.300847+0000","last_became_peered":"2023-10-06T05:19:32.300847+0000","last_unstale":"2023-10-06T05:19:33.288405+0000","last_undegraded":"2023-10-06T05:19:33.288405+0000","last_fullsized":"2023-10-06T05:19:33.288405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:17.489448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.292450+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:19:33.292450+0000","last_peered":"2023-10-06T05:19:33.292450+0000","last_clean":"2023-10-06T05:19:33.292450+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:19:33.292450+0000","last_undegraded":"2023-10-06T05:19:33.292450+0000","last_fullsized":"2023-10-06T05:19:33.292450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.287352+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:19:33.287352+0000","last_peered":"2023-10-06T05:19:33.287352+0000","last_clean":"2023-10-06T05:19:33.287352+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:19:33.287352+0000","last_undegraded":"2023-10-06T05:19:33.287352+0000","last_fullsized":"2023-10-06T05:19:33.287352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.322190+0000","last_change":"2023-10-06T05:19:33.316635+0000","last_active":"2023-10-06T05:19:33.322190+0000","last_peered":"2023-10-06T05:19:33.322190+0000","last_clean":"2023-10-06T05:19:33.322190+0000","last_became_active":"2023-10-06T05:19:33.316334+0000","last_became_peered":"2023-10-06T05:19:33.316334+0000","last_unstale":"2023-10-06T05:19:33.322190+0000","last_undegraded":"2023-10-06T05:19:33.322190+0000","last_fullsized":"2023-10-06T05:19:33.322190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:46.602601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.285851+0000","last_change":"2023-10-06T05:19:32.301200+0000","last_active":"2023-10-06T05:19:33.285851+0000","last_peered":"2023-10-06T05:19:33.285851+0000","last_clean":"2023-10-06T05:19:33.285851+0000","last_became_active":"2023-10-06T05:19:32.300936+0000","last_became_peered":"2023-10-06T05:19:32.300936+0000","last_unstale":"2023-10-06T05:19:33.285851+0000","last_undegraded":"2023-10-06T05:19:33.285851+0000","last_fullsized":"2023-10-06T05:19:33.285851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:14.667515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.286904+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:19:33.286904+0000","last_peered":"2023-10-06T05:19:33.286904+0000","last_clean":"2023-10-06T05:19:33.286904+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:19:33.286904+0000","last_undegraded":"2023-10-06T05:19:33.286904+0000","last_fullsized":"2023-10-06T05:19:33.286904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.321026+0000","last_change":"2023-10-06T05:19:33.317150+0000","last_active":"2023-10-06T05:19:33.321026+0000","last_peered":"2023-10-06T05:19:33.321026+0000","last_clean":"2023-10-06T05:19:33.321026+0000","last_became_active":"2023-10-06T05:19:33.316864+0000","last_became_peered":"2023-10-06T05:19:33.316864+0000","last_unstale":"2023-10-06T05:19:33.321026+0000","last_undegraded":"2023-10-06T05:19:33.321026+0000","last_fullsized":"2023-10-06T05:19:33.321026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:17.965182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.295426+0000","last_change":"2023-10-06T05:19:32.302108+0000","last_active":"2023-10-06T05:19:33.295426+0000","last_peered":"2023-10-06T05:19:33.295426+0000","last_clean":"2023-10-06T05:19:33.295426+0000","last_became_active":"2023-10-06T05:19:32.301860+0000","last_became_peered":"2023-10-06T05:19:32.301860+0000","last_unstale":"2023-10-06T05:19:33.295426+0000","last_undegraded":"2023-10-06T05:19:33.295426+0000","last_fullsized":"2023-10-06T05:19:33.295426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:46.662138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.288096+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:19:33.288096+0000","last_peered":"2023-10-06T05:19:33.288096+0000","last_clean":"2023-10-06T05:19:33.288096+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:19:33.288096+0000","last_undegraded":"2023-10-06T05:19:33.288096+0000","last_fullsized":"2023-10-06T05:19:33.288096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.321234+0000","last_change":"2023-10-06T05:19:33.317974+0000","last_active":"2023-10-06T05:19:33.321234+0000","last_peered":"2023-10-06T05:19:33.321234+0000","last_clean":"2023-10-06T05:19:33.321234+0000","last_became_active":"2023-10-06T05:19:33.317555+0000","last_became_peered":"2023-10-06T05:19:33.317555+0000","last_unstale":"2023-10-06T05:19:33.321234+0000","last_undegraded":"2023-10-06T05:19:33.321234+0000","last_fullsized":"2023-10-06T05:19:33.321234+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:55.627964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"75'1","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.319620+0000","last_change":"2023-10-06T05:19:32.309344+0000","last_active":"2023-10-06T05:19:33.319620+0000","last_peered":"2023-10-06T05:19:33.319620+0000","last_clean":"2023-10-06T05:19:33.319620+0000","last_became_active":"2023-10-06T05:19:32.309068+0000","last_became_peered":"2023-10-06T05:19:32.309068+0000","last_unstale":"2023-10-06T05:19:33.319620+0000","last_undegraded":"2023-10-06T05:19:33.319620+0000","last_fullsized":"2023-10-06T05:19:33.319620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:15.108521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.297333+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:19:33.297333+0000","last_peered":"2023-10-06T05:19:33.297333+0000","last_clean":"2023-10-06T05:19:33.297333+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:19:33.297333+0000","last_undegraded":"2023-10-06T05:19:33.297333+0000","last_fullsized":"2023-10-06T05:19:33.297333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.319369+0000","last_change":"2023-10-06T05:19:33.311398+0000","last_active":"2023-10-06T05:19:33.319369+0000","last_peered":"2023-10-06T05:19:33.319369+0000","last_clean":"2023-10-06T05:19:33.319369+0000","last_became_active":"2023-10-06T05:19:33.311124+0000","last_became_peered":"2023-10-06T05:19:33.311124+0000","last_unstale":"2023-10-06T05:19:33.319369+0000","last_undegraded":"2023-10-06T05:19:33.319369+0000","last_fullsized":"2023-10-06T05:19:33.319369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:32.272370+0000","last_clean_scrub_stamp":"2023-10-06T05:19:32.272370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:48.907951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"75'2","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.321861+0000","last_change":"2023-10-06T05:19:32.303086+0000","last_active":"2023-10-06T05:19:33.321861+0000","last_peered":"2023-10-06T05:19:33.321861+0000","last_clean":"2023-10-06T05:19:33.321861+0000","last_became_active":"2023-10-06T05:19:32.302806+0000","last_became_peered":"2023-10-06T05:19:32.302806+0000","last_unstale":"2023-10-06T05:19:33.321861+0000","last_undegraded":"2023-10-06T05:19:33.321861+0000","last_fullsized":"2023-10-06T05:19:33.321861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:19:31.265938+0000","last_clean_scrub_stamp":"2023-10-06T05:19:31.265938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:20.361257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:19:33.291903+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:19:33.291903+0000","last_peered":"2023-10-06T05:19:33.291903+0000","last_clean":"2023-10-06T05:19:33.291903+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:19:33.291903+0000","last_undegraded":"2023-10-06T05:19:33.291903+0000","last_fullsized":"2023-10-06T05:19:33.291903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298776,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073064,"statfs":{"total":96636764160,"available":96330817536,"internally_reserved":0,"allocated":1523712,"data_stored":1004291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68100000000000005,"15min":0.64100000000000001},"min":{"1min":0.51300000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.70999999999999996,"15min":0.68500000000000005},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.67400000000000004,"15min":0.621},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.52900000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.58099999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.68000000000000005},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.875}]},{"osd":3,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.751,"15min":0.78000000000000003},"min":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.117,"5min":1.202,"15min":1.202},"last":0.93000000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.626,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.502},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2050000000000001,"5min":1.407,"15min":1.407},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.24,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":937984,"data_stored":415437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.44800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92100000000000004,"5min":1.034,"15min":1.034},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.94099999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.625,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.52300000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72799999999999998,"15min":0.76800000000000002},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74299999999999999,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.86299999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86499999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77000000000000002,"15min":0.81000000000000005},"min":{"1min":0.66200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64100000000000001,"15min":0.68300000000000005},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90500000000000003,"5min":0.998,"15min":0.998},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67900000000000005,"15min":0.75700000000000001},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71499999999999997,"15min":0.78500000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96699999999999997,"5min":1.038,"15min":1.038},"last":0.753},{"interface":"front","average":{"1min":0.75,"5min":0.76600000000000001,"15min":0.85599999999999998},"min":{"1min":0.59099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.90200000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":414013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69799999999999995,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.434},{"interface":"front","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.58099999999999996},"min":{"1min":0.378,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71799999999999997,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.622},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89500000000000002,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70899999999999996,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.623},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73099999999999998,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.60799999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96799999999999997,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.83599999999999997},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.998,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.748}]},{"osd":5,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.018,"5min":1.349,"15min":1.349},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79500000000000004,"15min":0.83899999999999997},"min":{"1min":0.61299999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97099999999999997,"5min":1.669,"15min":1.669},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82699999999999996,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98099999999999998,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83099999999999996,"15min":0.89700000000000002},"min":{"1min":0.625,"5min":0.503,"15min":0.503},"max":{"1min":0.995,"5min":1.573,"15min":1.573},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82399999999999995,"15min":0.874},"min":{"1min":0.65700000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.02,"5min":1.528,"15min":1.528},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.877},"min":{"1min":0.54600000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0249999999999999,"5min":1.75,"15min":1.75},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302868,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068972,"statfs":{"total":96636764160,"available":96326627328,"internally_reserved":0,"allocated":1519616,"data_stored":1004269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46800000000000003,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70099999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71399999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77600000000000002,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.77200000000000002,"15min":0.74199999999999999},"min":{"1min":0.63700000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.79800000000000004,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.76400000000000001},"min":{"1min":0.56200000000000006,"5min":0.503,"15min":0.503},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78800000000000003,"15min":0.77700000000000002},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1679999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.82899999999999996,"15min":0.79500000000000004},"min":{"1min":0.68100000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.65900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298204,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073636,"statfs":{"total":96636764160,"available":96331403264,"internally_reserved":0,"allocated":937984,"data_stored":414615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.49099999999999999,"15min":0.45800000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.35899999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.74299999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78800000000000003,"15min":0.76400000000000001},"min":{"1min":0.64000000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.077,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.63300000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.044,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80800000000000005,"15min":0.80900000000000005},"min":{"1min":0.66200000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0009999999999999,"5min":1.319,"15min":1.319},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.66000000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1399999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.81000000000000005,"15min":0.79100000000000004},"min":{"1min":0.70999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0860000000000001,"5min":1.141,"15min":1.141},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.69599999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0980000000000001,"5min":1.282,"15min":1.282},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.69999999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0580000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302292,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069548,"statfs":{"total":96636764160,"available":96327217152,"internally_reserved":0,"allocated":929792,"data_stored":414455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.72499999999999998,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.495,"15min":0.504},"min":{"1min":0.372,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72099999999999997,"5min":1.772,"15min":1.772},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69099999999999995,"5min":1.913,"15min":1.913},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.59199999999999997},"min":{"1min":0.42299999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.46700000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":1.954,"15min":1.954},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.629},"min":{"1min":0.499,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.75900000000000001,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.621}]},{"osd":4,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.48399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0640000000000001,"5min":1.282,"15min":1.282},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1579999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1060000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.85399999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.343,"15min":1.343},"last":0.749},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.61199999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.83699999999999997,"15min":0.86399999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.129,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.65300000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.044,"5min":1.214,"15min":1.214},"last":0.79800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":415541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.73499999999999999,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0249999999999999,"5min":1.073,"15min":1.073},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.51000000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.169,"15min":1.169},"last":0.755}]},{"osd":1,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.54500000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91200000000000003,"5min":1.117,"15min":1.117},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71699999999999997,"15min":0.72199999999999998},"min":{"1min":0.52400000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96999999999999997,"5min":1.002,"15min":1.002},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98799999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.996,"5min":1.196,"15min":1.196},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.72699999999999998},"min":{"1min":0.46400000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.81999999999999995,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.77000000000000002},"min":{"1min":0.46000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":1.375,"15min":1.375},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69099999999999995,"15min":0.75900000000000001},"min":{"1min":0.498,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96899999999999997,"5min":1.004,"15min":1.004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92900000000000005,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.82699999999999996},"min":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.466,"15min":1.466},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.84699999999999998},"min":{"1min":0.53600000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.018,"5min":1.218,"15min":1.218},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":929792,"data_stored":413935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71699999999999997,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93500000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.92800000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.53400000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.038,"15min":1.038},"last":0.89500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.67500000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.73699999999999999,"15min":0.70299999999999996},"min":{"1min":0.54300000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.94899999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.002,"5min":1.087,"15min":1.087},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.495,"5min":0.433,"15min":0.433},"max":{"1min":1.2769999999999999,"5min":1.278,"15min":1.278},"last":0.97299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63800000000000001,"15min":0.66800000000000004},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.76500000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82599999999999996,"5min":1.294,"15min":1.294},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65200000000000002,"15min":0.69099999999999995},"min":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71499999999999997,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.873,"5min":1.254,"15min":1.254},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.78600000000000003},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":1.337,"15min":1.337},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.78400000000000003},"min":{"1min":0.501,"5min":0.495,"15min":0.495},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":1.0609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:19:38.142 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:19:38.142 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:19:38.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.370+0000 7f05baf9d700 1 Processor -- start 2023-10-06T05:19:38.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.370+0000 7f05baf9d700 1 -- start start 2023-10-06T05:19:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05baf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f05b40cc600 0x7f05b40cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05baf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05b4067e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05baf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b40b0350 con 0x7f05b4068340 2023-10-06T05:19:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05baf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b4159d10 con 0x7f05b40cc600 2023-10-06T05:19:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05baf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40b0630 con 0x7f05b40ccf30 2023-10-06T05:19:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05ba79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f05b4068340 0x7f05b406c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33250/0 (socket says 172.21.15.33:33250) 2023-10-06T05:19:38.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05ba79c700 1 -- 172.21.15.33:0/1455241776 learned_addr learned my addr 172.21.15.33:0/1455241776 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05b979a700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05b4067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1455241776 >> v2:172.21.15.167:3300/0 conn(0x7f05b40cc600 0x7f05b40cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422676394 0 0) 0x7f05b40b0350 con 0x7f05b4068340 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.372+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a0003570 con 0x7f05b4068340 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 416789134 0 0) 0x7f05a0003570 con 0x7f05b4068340 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 msgr2=0x7f05b4067e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05b4067e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 >> v2:172.21.15.167:3300/0 conn(0x7f05b40cc600 msgr2=0x7f05b40cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 --2- 172.21.15.33:0/1455241776 >> v2:172.21.15.167:3300/0 conn(0x7f05b40cc600 0x7f05b40cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40b0910 con 0x7f05b4068340 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f05ac0017f0 con 0x7f05b4068340 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b979a700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05b4067e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 >> v1:172.21.15.33:6789/0 conn(0x7f05b4068340 legacy=0x7f05b406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a0004810 0x7f05a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4159d10 con 0x7f05a0003a60 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40b0630 con 0x7f05a0004810 2023-10-06T05:19:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a000ba60 con 0x7f05b40ccf30 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05b979a700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a0004810 0x7f05a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.373+0000 7f05ba79c700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 msgr2=0x7f05a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b40ccf30 0x7f05a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a0004810 msgr2=0x7f05a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a0004810 0x7f05a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1455241776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009420 con 0x7f05a0003a60 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05a0003e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009c00 tx=0x7f05b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e92782469d21b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000e710 con 0x7f05a0003a60 2023-10-06T05:19:38.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000e870 con 0x7f05a0003a60 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.374+0000 7f05b8f99700 1 -- 172.21.15.33:0/1455241776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000ec30 con 0x7f05a0003a60 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05baf9d700 1 -- 172.21.15.33:0/1455241776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 msgr2=0x7f05a0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1455241776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05a0003e70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009c00 tx=0x7f05b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05ba79c700 1 -- 172.21.15.33:0/1455241776 reap_dead start 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05baf9d700 1 -- 172.21.15.33:0/1455241776 shutdown_connections 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05baf9d700 1 -- 172.21.15.33:0/1455241776 >> 172.21.15.33:0/1455241776 conn(0x7f05b406d670 msgr2=0x7f05b406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05baf9d700 1 -- 172.21.15.33:0/1455241776 shutdown_connections 2023-10-06T05:19:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.375+0000 7f05baf9d700 1 -- 172.21.15.33:0/1455241776 wait complete. 2023-10-06T05:19:38.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.376+0000 7f05baf9d700 1 Processor -- start 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.376+0000 7f05baf9d700 1 -- start start 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.377+0000 7f05baf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05b4157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.377+0000 7f05baf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b4158910 0x7f05b40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.377+0000 7f05baf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b4142dd0 0x7f05b40acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.377+0000 7f05baf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4158480 con 0x7f05a0003a60 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.377+0000 7f05baf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4157870 con 0x7f05b4142dd0 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.377+0000 7f05baf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4157b50 con 0x7f05b4158910 2023-10-06T05:19:38.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05b4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05b4157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40770/0 (socket says 172.21.15.33:40770) 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1534363820 learned_addr learned my addr 172.21.15.33:0/1534363820 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05ba79c700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b4142dd0 0x7f05b40acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b979a700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b4158910 0x7f05b40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b4158910 msgr2=0x7f05b40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b4158910 0x7f05b40af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b4142dd0 msgr2=0x7f05b40acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b4142dd0 0x7f05b40acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 -- 172.21.15.33:0/1534363820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4157e30 con 0x7f05a0003a60 2023-10-06T05:19:38.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05ba79c700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b4142dd0 0x7f05b40acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b979a700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b4158910 0x7f05b40af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05b9f9b700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05b4157190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f05a0004690 tx=0x7f05b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0022070 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000ebf0 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.378+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b001c6a0 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.379+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b4158110 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.379+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b4143400 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.380+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b404c380 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.380+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05b001ca90 con 0x7f05a0003a60 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.381+0000 7f05a9ffb700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0598066f30 0x7f05980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.381+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f05b0085ee0 con 0x7f05a0003a60 2023-10-06T05:19:38.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.382+0000 7f05b979a700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0598066f30 0x7f05980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.383+0000 7f05b979a700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0598066f30 0x7f05980693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f05a40093f0 tx=0x7f05a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.386+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05b0052f90 con 0x7f05a0003a60 2023-10-06T05:19:38.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.666+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f05b40b0bf0 con 0x7f05a0003a60 2023-10-06T05:19:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.670+0000 7f05a9ffb700 1 -- 172.21.15.33:0/1534363820 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f05b002b0e0 con 0x7f05a0003a60 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0598066f30 msgr2=0x7f05980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0598066f30 0x7f05980693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f05a40093f0 tx=0x7f05a4008040 comp rx=0 tx=0).stop 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 msgr2=0x7f05b4157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05b4157190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f05a0004690 tx=0x7f05b001a000 comp rx=0 tx=0).stop 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 shutdown_connections 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0598066f30 0x7f05980693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b4142dd0 0x7f05b40acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0003a60 0x7f05b4157190 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 --2- 172.21.15.33:0/1534363820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05b4158910 0x7f05b40af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 >> 172.21.15.33:0/1534363820 conn(0x7f05b406d670 msgr2=0x7f05b40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 shutdown_connections 2023-10-06T05:19:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.673+0000 7f05baf9d700 1 -- 172.21.15.33:0/1534363820 wait complete. 2023-10-06T05:19:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:19:38.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.877+0000 7fe8ddb9d700 1 Processor -- start 2023-10-06T05:19:38.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.877+0000 7fe8ddb9d700 1 -- start start 2023-10-06T05:19:38.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.878+0000 7fe8ddb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8d81557a0 0x7fe8d806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.878+0000 7fe8ddb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d815a090 0x7fe8d8067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.878+0000 7fe8ddb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d8068540 con 0x7fe8d80680e0 2023-10-06T05:19:38.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.878+0000 7fe8ddb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d806c770 con 0x7fe8d81557a0 2023-10-06T05:19:38.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.878+0000 7fe8ddb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d815a500 con 0x7fe8d815a090 2023-10-06T05:19:38.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8d81557a0 0x7fe8d806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8d81557a0 0x7fe8d806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40782/0 (socket says 172.21.15.33:40782) 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 -- 172.21.15.33:0/4006734975 learned_addr learned my addr 172.21.15.33:0/4006734975 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dd39c700 1 --1- 172.21.15.33:0/4006734975 >> v1:172.21.15.33:6789/0 conn(0x7fe8d80680e0 0x7fe8d806cc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33262/0 (socket says 172.21.15.33:33262) 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8d7fff700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d815a090 0x7fe8d8067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 -- 172.21.15.33:0/4006734975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d815a090 msgr2=0x7fe8d8067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d815a090 0x7fe8d8067ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 -- 172.21.15.33:0/4006734975 >> v1:172.21.15.33:6789/0 conn(0x7fe8d80680e0 legacy=0x7fe8d806cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 -- 172.21.15.33:0/4006734975 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d806a820 con 0x7fe8d81557a0 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8d7fff700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d815a090 0x7fe8d8067ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.879+0000 7fe8dcb9b700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d81557a0 0x7fe8d806a160 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0099b0 tx=0x7fe8cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b969eafccfaf2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8d77fe700 1 -- 172.21.15.33:0/4006734975 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00b860 con 0x7fe8d81557a0 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8d77fe700 1 -- 172.21.15.33:0/4006734975 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc00b9c0 con 0x7fe8d81557a0 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8d77fe700 1 -- 172.21.15.33:0/4006734975 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00bd60 con 0x7fe8d81557a0 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/4006734975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d81557a0 msgr2=0x7fe8d806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d81557a0 0x7fe8d806a160 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0099b0 tx=0x7fe8cc009250 comp rx=0 tx=0).stop 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/4006734975 shutdown_connections 2023-10-06T05:19:38.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d81557a0 0x7fe8d806a160 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/4006734975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d815a090 0x7fe8d8067ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/4006734975 >> 172.21.15.33:0/4006734975 conn(0x7fe8d80cc930 msgr2=0x7fe8d80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/4006734975 shutdown_connections 2023-10-06T05:19:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.880+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/4006734975 wait complete. 2023-10-06T05:19:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.882+0000 7fe8ddb9d700 1 Processor -- start 2023-10-06T05:19:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.882+0000 7fe8ddb9d700 1 -- start start 2023-10-06T05:19:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8ddb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d80680e0 0x7fe8d8169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8ddb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8d81557a0 0x7fe8d81582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8ddb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 0x7fe8d80c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8ddb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80b0950 con 0x7fe8d815a090 2023-10-06T05:19:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8ddb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80b0c30 con 0x7fe8d81557a0 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8ddb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80b0f10 con 0x7fe8d80680e0 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8dd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 0x7fe8d80c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8dd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 0x7fe8d80c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40784/0 (socket says 172.21.15.33:40784) 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.883+0000 7fe8dd39c700 1 -- 172.21.15.33:0/3533251084 learned_addr learned my addr 172.21.15.33:0/3533251084 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dcb9b700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d80680e0 0x7fe8d8169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8d7fff700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8d81557a0 0x7fe8d81582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dd39c700 1 -- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d80680e0 msgr2=0x7fe8d8169b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dd39c700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d80680e0 0x7fe8d8169b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dd39c700 1 -- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8d81557a0 msgr2=0x7fe8d81582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dd39c700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8d81557a0 0x7fe8d81582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dd39c700 1 -- 172.21.15.33:0/3533251084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc0094f0 con 0x7fe8d815a090 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8dd39c700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 0x7fe8d80c5e80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c910 tx=0x7fe8d0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0004da0 con 0x7fe8d815a090 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d00060f0 con 0x7fe8d815a090 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d80b11f0 con 0x7fe8d815a090 2023-10-06T05:19:38.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.884+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00064b0 con 0x7fe8d815a090 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.885+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d8158830 con 0x7fe8d815a090 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.885+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80b1710 con 0x7fe8d815a090 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.886+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8d0006860 con 0x7fe8d815a090 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.887+0000 7fe8d4ff9700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.887+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8d0084260 con 0x7fe8d815a090 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.888+0000 7fe8dcb9b700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.889+0000 7fe8dcb9b700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8bc066f30 0x7fe8bc0693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0099b0 tx=0x7fe8cc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:38.891+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8d80b1710 con 0x7fe8d815a090 2023-10-06T05:19:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.137+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d8068540 con 0x7fe8d815a090 2023-10-06T05:19:39.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.137+0000 7fe8d4ff9700 1 -- 172.21.15.33:0/3533251084 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fe8d8068540 con 0x7fe8d815a090 2023-10-06T05:19:39.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:39.140 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:36.302879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2242593899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2242593899},{"type":"v1","addr":"172.21.15.33:6835","nonce":2242593899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.139+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8bc066f30 msgr2=0x7fe8bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.139+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8bc066f30 0x7fe8bc0693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0099b0 tx=0x7fe8cc017000 comp rx=0 tx=0).stop 2023-10-06T05:19:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.139+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 msgr2=0x7fe8d80c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.139+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 0x7fe8d80c5e80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c910 tx=0x7fe8d0001930 comp rx=0 tx=0).stop 2023-10-06T05:19:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 shutdown_connections 2023-10-06T05:19:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8d815a090 0x7fe8d80c5e80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8d80680e0 0x7fe8d8169b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 --2- 172.21.15.33:0/3533251084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8d81557a0 0x7fe8d81582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 >> 172.21.15.33:0/3533251084 conn(0x7fe8d80cc930 msgr2=0x7fe8d80ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 shutdown_connections 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.140+0000 7fe8ddb9d700 1 -- 172.21.15.33:0/3533251084 wait complete. 2023-10-06T05:19:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 43 2023-10-06T05:19:39.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:19:32.778469+0000', 'modified': '2023-10-06T05:19:36.302879+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6093': {'gid': 6093, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2242593899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 2242593899}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2242593899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:19:39.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:19:39.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:19:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.750+0000 7fab6b59e700 1 Processor -- start 2023-10-06T05:19:39.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.751+0000 7fab6b59e700 1 -- start start 2023-10-06T05:19:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.752+0000 7fab6b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fab6c0af3b0 0x7fab6c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.752+0000 7fab6b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0b03a0 con 0x7fab6c1596b0 2023-10-06T05:19:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.752+0000 7fab6b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0b0f40 con 0x7fab6c1592e0 2023-10-06T05:19:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.752+0000 7fab6b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c06ccd0 con 0x7fab6c0af3b0 2023-10-06T05:19:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.752+0000 7fab69d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab6c1596b0 0x7fab6c0aeca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33264/0 (socket says 172.21.15.33:33264) 2023-10-06T05:19:39.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.752+0000 7fab69d9b700 1 -- 172.21.15.33:0/1886976694 learned_addr learned my addr 172.21.15.33:0/1886976694 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/1886976694 >> v2:172.21.15.167:3300/0 conn(0x7fab6c0af3b0 0x7fab6c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480039039 0 0) 0x7fab6c0b03a0 con 0x7fab6c1596b0 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab54003570 con 0x7fab6c1596b0 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232225073 0 0) 0x7fab6c0b0f40 con 0x7fab6c1592e0 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab6c0b03a0 con 0x7fab6c1592e0 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713901194 0 0) 0x7fab54003570 con 0x7fab6c1596b0 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 >> v2:172.21.15.167:3300/0 conn(0x7fab6c0af3b0 msgr2=0x7fab6c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 --2- 172.21.15.33:0/1886976694 >> v2:172.21.15.167:3300/0 conn(0x7fab6c0af3b0 0x7fab6c0cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 >> v1:172.21.15.167:6789/0 conn(0x7fab6c1592e0 legacy=0x7fab6c0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c06cfb0 con 0x7fab6c1596b0 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fab60001470 con 0x7fab6c1596b0 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 >> v1:172.21.15.33:6789/0 conn(0x7fab6c1596b0 legacy=0x7fab6c0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54003c40 0x7fab54004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab54004810 0x7fab54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab54005210 0x7fab5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c06ccd0 con 0x7fab54005210 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5400c1b0 con 0x7fab54004810 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.753+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5400c370 con 0x7fab54003c40 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab54005210 0x7fab5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab69d9b700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab54004810 0x7fab54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6a59c700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54003c40 0x7fab54004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 -- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54003c40 msgr2=0x7fab54004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54003c40 0x7fab54004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 -- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab54004810 msgr2=0x7fab54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab54004810 0x7fab54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 -- 172.21.15.33:0/1886976694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54003570 con 0x7fab54005210 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab69d9b700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab54004810 0x7fab54004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 -- 172.21.15.33:0/1886976694 reap_dead start 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab54005210 0x7fab5400ba60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab640094d0 tx=0x7fab6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80a9ec7a25507c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.754+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400ea60 con 0x7fab54005210 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400f040 con 0x7fab54005210 2023-10-06T05:19:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6959a700 1 -- 172.21.15.33:0/1886976694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400ee30 con 0x7fab54005210 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 -- 172.21.15.33:0/1886976694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab54005210 msgr2=0x7fab5400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab54005210 0x7fab5400ba60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab640094d0 tx=0x7fab6400b040 comp rx=0 tx=0).stop 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 -- 172.21.15.33:0/1886976694 shutdown_connections 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 --2- 172.21.15.33:0/1886976694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab54005210 0x7fab5400ba60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 -- 172.21.15.33:0/1886976694 >> 172.21.15.33:0/1886976694 conn(0x7fab6c0675f0 msgr2=0x7fab6c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 -- 172.21.15.33:0/1886976694 shutdown_connections 2023-10-06T05:19:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.755+0000 7fab6b59e700 1 -- 172.21.15.33:0/1886976694 wait complete. 2023-10-06T05:19:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.757+0000 7fab6b59e700 1 Processor -- start 2023-10-06T05:19:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.757+0000 7fab6b59e700 1 -- start start 2023-10-06T05:19:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54005210 0x7fab6c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 0x7fab6c0c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0ca3d0 0x7fab6c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c93a0 con 0x7fab6c156420 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c156200 con 0x7fab6c0ca3d0 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c9610 con 0x7fab54005210 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 0x7fab6c0c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 0x7fab6c0c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40800/0 (socket says 172.21.15.33:40800) 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab69d9b700 1 -- 172.21.15.33:0/2766679211 learned_addr learned my addr 172.21.15.33:0/2766679211 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6a59c700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54005210 0x7fab6c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.758+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0ca3d0 0x7fab6c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab69d9b700 1 -- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54005210 msgr2=0x7fab6c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab69d9b700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54005210 0x7fab6c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab69d9b700 1 -- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0ca3d0 msgr2=0x7fab6c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab69d9b700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0ca3d0 0x7fab6c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab69d9b700 1 -- 172.21.15.33:0/2766679211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0c98f0 con 0x7fab6c156420 2023-10-06T05:19:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab69d9b700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 0x7fab6c0c8a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab60001e30 tx=0x7fab60001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab5a7fc700 1 -- 172.21.15.33:0/2766679211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60018030 con 0x7fab6c156420 2023-10-06T05:19:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab5a7fc700 1 -- 172.21.15.33:0/2766679211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6000f8d0 con 0x7fab6c156420 2023-10-06T05:19:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab5a7fc700 1 -- 172.21.15.33:0/2766679211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6000fc50 con 0x7fab6c156420 2023-10-06T05:19:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c0c9bd0 con 0x7fab6c156420 2023-10-06T05:19:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.759+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c0c8f70 con 0x7fab6c156420 2023-10-06T05:19:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.761+0000 7fab5a7fc700 1 -- 172.21.15.33:0/2766679211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab6001b070 con 0x7fab6c156420 2023-10-06T05:19:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.762+0000 7fab5a7fc700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab4c066f60 0x7fab4c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.762+0000 7fab5a7fc700 1 -- 172.21.15.33:0/2766679211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab60019070 con 0x7fab6c156420 2023-10-06T05:19:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.763+0000 7fab6a59c700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab4c066f60 0x7fab4c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.763+0000 7fab6a59c700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab4c066f60 0x7fab4c069410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.764+0000 7fab7f61fb80 1 Processor -- start 2023-10-06T05:19:39.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.764+0000 7fab7f61fb80 1 -- start start 2023-10-06T05:19:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d27206d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27250bc0 0x561d2722f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d2722fe60 0x561d27202990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d27186c80 con 0x561d27206950 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2715da70 con 0x561d2722fe60 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d2715b270 con 0x561d27250bc0 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d27206d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d27206d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40812/0 (socket says 172.21.15.33:40812) 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.765+0000 7fab6a59c700 1 -- 172.21.15.33:0/3546873792 learned_addr learned my addr 172.21.15.33:0/3546873792 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6a59c700 1 -- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27250bc0 msgr2=0x561d2722f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab69d9b700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27250bc0 0x561d2722f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d2722fe60 0x561d27202990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27250bc0 0x561d2722f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6a59c700 1 -- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d2722fe60 msgr2=0x561d27202990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d2722fe60 0x561d27202990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6a59c700 1 -- 172.21.15.33:0/3546873792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d26ff1270 con 0x561d27206950 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab69d9b700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27250bc0 0x561d2722f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d2722fe60 0x561d27202990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d27206d20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002cf0 tx=0x7fab5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4809967fbc8fb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab4b7fe700 1 -- 172.21.15.33:0/3546873792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c015c60 con 0x561d27206950 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab4b7fe700 1 -- 172.21.15.33:0/3546873792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c015dc0 con 0x561d27206950 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.766+0000 7fab4b7fe700 1 -- 172.21.15.33:0/3546873792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c021620 con 0x561d27206950 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3546873792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 msgr2=0x561d27206d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d27206d20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002cf0 tx=0x7fab5c018000 comp rx=0 tx=0).stop 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3546873792 shutdown_connections 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d2722fe60 0x561d27202990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d27206d20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3546873792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27250bc0 0x561d2722f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3546873792 >> 172.21.15.33:0/3546873792 conn(0x561d27208270 msgr2=0x561d27208640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3546873792 shutdown_connections 2023-10-06T05:19:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.767+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3546873792 wait complete. 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.768+0000 7fab7f61fb80 1 -- start start 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.768+0000 7fab7f61fb80 1 Processor -- start 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d2722ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27250bc0 0x561d271fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d27157060 con 0x561d2722fe60 2023-10-06T05:19:39.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d27102890 con 0x561d27250bc0 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d2711dea0 con 0x561d27206950 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab7f61fb80 10 client.0 _ll_register_callbacks cb 0x561d26ff3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40822/0 (socket says 172.21.15.33:40822) 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab69d9b700 1 -- 172.21.15.33:0/3936822962 learned_addr learned my addr 172.21.15.33:0/3936822962 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d2722ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27250bc0 0x561d271fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.769+0000 7fab69d9b700 1 -- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 msgr2=0x561d2722ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab69d9b700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d2722ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab69d9b700 1 -- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27250bc0 msgr2=0x561d271fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab69d9b700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27250bc0 0x561d271fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab69d9b700 1 -- 172.21.15.33:0/3936822962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c0b03a0 con 0x561d2722fe60 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d2722ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab69d9b700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722c1f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab600827f0 tx=0x7fab60082750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808c555e78421692 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab48ff9700 1 -- 172.21.15.33:0/3936822962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6005c1a0 con 0x561d2722fe60 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab48ff9700 1 -- 172.21.15.33:0/3936822962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab6005a040 con 0x561d2722fe60 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d271fb690 con 0x561d2722fe60 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab48ff9700 1 -- 172.21.15.33:0/3936822962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab600842f0 con 0x561d2722fe60 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab7f61fb80 10 client.6225 fetch_fsmap learned FSMap version 43 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x561d26f93fe0 con 0x561d2722fe60 2023-10-06T05:19:39.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.770+0000 7fab48ff9700 1 -- 172.21.15.33:0/3936822962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d271fb690 con 0x561d2722fe60 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab48ff9700 1 -- 172.21.15.33:0/3936822962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fab600547a0 con 0x561d2722fe60 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab7f61fb80 10 client.6225 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab7f61fb80 10 client.6225 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab7f61fb80 20 client.6225 populate_metadata read hostname 'smithi033' 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d2724a730 0x561d2724ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab7f61fb80 4 client.6225 mds_command: new command op to 6093 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 --> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d271d85c0 con 0x561d2724a730 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d2724a730 0x561d2724ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab6a59c700 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d2724a730 0x561d2724ab00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd5b8bd610a3 server_cookie=9649646bff9cb0eb in_seq=0 out_seq=0 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.771+0000 7fab48ff9700 10 client.6225 ms_handle_connect on v2:172.21.15.33:6834/2242593899 2023-10-06T05:19:39.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.772+0000 7fab48ff9700 1 -- 172.21.15.33:0/3936822962 <== mds.0 v2:172.21.15.33:6834/2242593899 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561d271d85c0 con 0x561d2724a730 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.772+0000 7fab48ff9700 10 client.6225 handle_command_reply: tid=0 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.772+0000 7fab7f61fb80 1 client.6225 shutdown 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.772+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 msgr2=0x561d2722c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.772+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722c1f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab600827f0 tx=0x7fab60082750 comp rx=0 tx=0).stop 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 shutdown_connections 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27250bc0 0x561d271fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d2724a730 0x561d2724ab00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d2722ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/3936822962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722c1f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 >> 172.21.15.33:0/3936822962 conn(0x561d27208270 msgr2=0x561d272071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab48ff9700 0 client.6225 ms_handle_reset on v2:172.21.15.33:6834/2242593899 2023-10-06T05:19:39.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 shutdown_connections 2023-10-06T05:19:39.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 1 -- 172.21.15.33:0/3936822962 wait complete. 2023-10-06T05:19:39.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.773+0000 7fab7f61fb80 20 client.6225 trim_cache size 0 max 16384 2023-10-06T05:19:39.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.791+0000 7fab7f61fb80 1 Processor -- start 2023-10-06T05:19:39.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.791+0000 7fab7f61fb80 1 -- start start 2023-10-06T05:19:39.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.791+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d27236260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.791+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d272367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.791+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d26fd9e30 con 0x561d2722fe60 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d27236ce0 con 0x561d27208270 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d271e6030 con 0x561d27206950 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40838/0 (socket says 172.21.15.33:40838) 2023-10-06T05:19:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 -- 172.21.15.33:0/676793454 learned_addr learned my addr 172.21.15.33:0/676793454 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab69d9b700 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d272367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6a59c700 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d27236260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 -- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 msgr2=0x561d272367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d272367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 -- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 msgr2=0x561d27236260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d27236260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6ad9d700 1 -- 172.21.15.33:0/676793454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d27125780 con 0x561d2722fe60 2023-10-06T05:19:39.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.792+0000 7fab6a59c700 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d27236260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722a510 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fab6c068300 tx=0x7fab64019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e97ce32866614c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab4bfff700 1 -- 172.21.15.33:0/676793454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64025070 con 0x561d2722fe60 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab4bfff700 1 -- 172.21.15.33:0/676793454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab64009420 con 0x561d2722fe60 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab4bfff700 1 -- 172.21.15.33:0/676793454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400ec90 con 0x561d2722fe60 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 -- 172.21.15.33:0/676793454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 msgr2=0x561d2722a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722a510 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fab6c068300 tx=0x7fab64019000 comp rx=0 tx=0).stop 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 -- 172.21.15.33:0/676793454 shutdown_connections 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2722fe60 0x561d2722a510 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d27236260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/676793454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d27206950 0x561d272367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.793+0000 7fab7f61fb80 1 -- 172.21.15.33:0/676793454 >> 172.21.15.33:0/676793454 conn(0x561d272092b0 msgr2=0x561d272035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.794+0000 7fab7f61fb80 1 -- 172.21.15.33:0/676793454 shutdown_connections 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.794+0000 7fab7f61fb80 1 -- 172.21.15.33:0/676793454 wait complete. 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.794+0000 7fab7f61fb80 1 -- start start 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.794+0000 7fab7f61fb80 1 Processor -- start 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d2726af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d2726b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d2722fe60 0x561d27250f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2726e560 con 0x561d27206950 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2722b2a0 con 0x561d27208270 2023-10-06T05:19:39.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d2724b060 con 0x561d2722fe60 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.795+0000 7fab7f61fb80 10 client.0 _ll_register_callbacks cb 0x561d26ff3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d2726af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d2726af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40844/0 (socket says 172.21.15.33:40844) 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 -- 172.21.15.33:0/147556832 learned_addr learned my addr 172.21.15.33:0/147556832 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6ad9d700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d2722fe60 0x561d27250f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab69d9b700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d2726b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 -- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d2722fe60 msgr2=0x561d27250f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d2722fe60 0x561d27250f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 -- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 msgr2=0x561d2726b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d2726b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 -- 172.21.15.33:0/147556832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab54003570 con 0x561d27206950 2023-10-06T05:19:39.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab69d9b700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d2726b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab6a59c700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d2726af60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002a40 tx=0x7fab5c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39738bf0bc38dbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.796+0000 7fab48ff9700 1 -- 172.21.15.33:0/147556832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c03c070 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab48ff9700 1 -- 172.21.15.33:0/147556832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab5c0219d0 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab5c015dc0 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab48ff9700 1 -- 172.21.15.33:0/147556832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab5c032040 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 10 client.6231 fetch_fsmap learned FSMap version 43 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x561d2710c470 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab48ff9700 1 -- 172.21.15.33:0/147556832 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c045020 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab48ff9700 1 -- 172.21.15.33:0/147556832 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fab5c03f020 con 0x561d27206950 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 10 client.6231 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T05:19:39.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 10 client.6231 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:19:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 20 client.6231 populate_metadata read hostname 'smithi033' 2023-10-06T05:19:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d27250bc0 0x561d272514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 4 client.6231 mds_command: new command op to 6093 tid=0[{"prefix": "status"}] 2023-10-06T05:19:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 --> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d270f9e00 con 0x561d27250bc0 2023-10-06T05:19:39.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.797+0000 7fab69d9b700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d27250bc0 0x561d272514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:39.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab69d9b700 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d27250bc0 0x561d272514d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c6501fbc564fdf server_cookie=80f43fb2c33510ea in_seq=0 out_seq=0 2023-10-06T05:19:39.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab48ff9700 10 client.6231 ms_handle_connect on v2:172.21.15.33:6834/2242593899 2023-10-06T05:19:39.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab48ff9700 1 -- 172.21.15.33:0/147556832 <== mds.0 v2:172.21.15.33:6834/2242593899 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561d270f9e00 con 0x561d27250bc0 2023-10-06T05:19:39.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab48ff9700 10 client.6231 handle_command_reply: tid=0 2023-10-06T05:19:39.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 client.6231 shutdown 2023-10-06T05:19:39.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 msgr2=0x561d2726af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d2726af60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002a40 tx=0x7fab5c02e040 comp rx=0 tx=0).stop 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 shutdown_connections 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:6834/2242593899,v1:172.21.15.33:6835/2242593899] conn(0x561d27250bc0 0x561d272514d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561d2722fe60 0x561d27250f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d27206950 0x561d2726af60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 --2- 172.21.15.33:0/147556832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d27208270 0x561d2726b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab48ff9700 0 client.6231 ms_handle_reset on v2:172.21.15.33:6834/2242593899 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.798+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 >> 172.21.15.33:0/147556832 conn(0x561d272092b0 msgr2=0x561d2725c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 shutdown_connections 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab7f61fb80 1 -- 172.21.15.33:0/147556832 wait complete. 2023-10-06T05:19:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab7f61fb80 20 client.6231 trim_cache size 0 max 16384 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 6093, 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5164478600000004, 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 43, 2023-10-06T05:19:39.806 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 76, 2023-10-06T05:19:39.807 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 76, 2023-10-06T05:19:39.807 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.244852611000001 2023-10-06T05:19:39.807 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab4c066f60 msgr2=0x7fab4c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab4c066f60 0x7fab4c069410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c007d30 comp rx=0 tx=0).stop 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 msgr2=0x7fab6c0c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 0x7fab6c0c8a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab60001e30 tx=0x7fab60001d40 comp rx=0 tx=0).stop 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 shutdown_connections 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab4c066f60 0x7fab4c069410 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0ca3d0 0x7fab6c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab54005210 0x7fab6c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 --2- 172.21.15.33:0/2766679211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6c156420 0x7fab6c0c8a30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:39.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 >> 172.21.15.33:0/2766679211 conn(0x7fab6c0675f0 msgr2=0x7fab6c158320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:39.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.799+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 shutdown_connections 2023-10-06T05:19:39.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:39.800+0000 7fab6b59e700 1 -- 172.21.15.33:0/2766679211 wait complete. 2023-10-06T05:19:39.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:19:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.004+0000 7f90fef9d700 1 Processor -- start 2023-10-06T05:19:40.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.004+0000 7f90fef9d700 1 -- start start 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.005+0000 7f90fef9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90f8159530 0x7f90f806da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.005+0000 7f90fef9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f90f806df70 0x7f90f80afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.005+0000 7f90fef9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f806e3e0 con 0x7f90f80b0310 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.005+0000 7f90fef9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80b1e00 con 0x7f90f8159530 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.005+0000 7f90fef9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f806f6b0 con 0x7f90f806df70 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90f8159530 0x7f90f806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90f8159530 0x7f90f806da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43848/0 (socket says 172.21.15.33:43848) 2023-10-06T05:19:40.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 -- 172.21.15.33:0/519619952 learned_addr learned my addr 172.21.15.33:0/519619952 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fe79c700 1 --1- 172.21.15.33:0/519619952 >> v1:172.21.15.33:6789/0 conn(0x7f90f80b0310 0x7f90f8159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45766/0 (socket says 172.21.15.33:45766) 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fd79a700 1 --2- 172.21.15.33:0/519619952 >> v2:172.21.15.167:3300/0 conn(0x7f90f806df70 0x7f90f80afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 -- 172.21.15.33:0/519619952 >> v2:172.21.15.167:3300/0 conn(0x7f90f806df70 msgr2=0x7f90f80afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 --2- 172.21.15.33:0/519619952 >> v2:172.21.15.167:3300/0 conn(0x7f90f806df70 0x7f90f80afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 -- 172.21.15.33:0/519619952 >> v1:172.21.15.33:6789/0 conn(0x7f90f80b0310 legacy=0x7f90f8159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 -- 172.21.15.33:0/519619952 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f806f220 con 0x7f90f8159530 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fd79a700 1 --2- 172.21.15.33:0/519619952 >> v2:172.21.15.167:3300/0 conn(0x7f90f806df70 0x7f90f80afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:40.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.006+0000 7f90fdf9b700 1 --2- 172.21.15.33:0/519619952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8159530 0x7f90f806da30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f90f80add90 tx=0x7f90f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf8d3f914670687 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:40.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fcf99700 1 -- 172.21.15.33:0/519619952 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f400b820 con 0x7f90f8159530 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fcf99700 1 -- 172.21.15.33:0/519619952 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f400b980 con 0x7f90f8159530 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fcf99700 1 -- 172.21.15.33:0/519619952 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f400bd20 con 0x7f90f8159530 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 -- 172.21.15.33:0/519619952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8159530 msgr2=0x7f90f806da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 --2- 172.21.15.33:0/519619952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8159530 0x7f90f806da30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f90f80add90 tx=0x7f90f4009250 comp rx=0 tx=0).stop 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 -- 172.21.15.33:0/519619952 shutdown_connections 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 --2- 172.21.15.33:0/519619952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8159530 0x7f90f806da30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 --2- 172.21.15.33:0/519619952 >> v2:172.21.15.167:3300/0 conn(0x7f90f806df70 0x7f90f80afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 -- 172.21.15.33:0/519619952 >> 172.21.15.33:0/519619952 conn(0x7f90f806c560 msgr2=0x7f90f806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.007+0000 7f90fef9d700 1 -- 172.21.15.33:0/519619952 shutdown_connections 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.008+0000 7f90fef9d700 1 -- 172.21.15.33:0/519619952 wait complete. 2023-10-06T05:19:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.009+0000 7f90fef9d700 1 Processor -- start 2023-10-06T05:19:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.009+0000 7f90fef9d700 1 -- start start 2023-10-06T05:19:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fef9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f806df70 0x7f90f80cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fef9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 0x7f90f815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90f8159530 0x7f90f8169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fef9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80ce3f0 con 0x7f90f80b0310 2023-10-06T05:19:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fef9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f816aba0 con 0x7f90f806df70 2023-10-06T05:19:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f815c1b0 con 0x7f90f8159530 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fd79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 0x7f90f815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fd79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 0x7f90f815c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43858/0 (socket says 172.21.15.33:43858) 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fd79a700 1 -- 172.21.15.33:0/1013399882 learned_addr learned my addr 172.21.15.33:0/1013399882 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fdf9b700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f806df70 0x7f90f80cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.010+0000 7f90fe79c700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90f8159530 0x7f90f8169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fd79a700 1 -- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90f8159530 msgr2=0x7f90f8169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fd79a700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90f8159530 0x7f90f8169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fd79a700 1 -- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f806df70 msgr2=0x7f90f80cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fd79a700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f806df70 0x7f90f80cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fd79a700 1 -- 172.21.15.33:0/1013399882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40094f0 con 0x7f90f80b0310 2023-10-06T05:19:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fd79a700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 0x7f90f815c640 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e8d0 tx=0x7f90e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8013410 con 0x7f90f80b0310 2023-10-06T05:19:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e8013570 con 0x7f90f80b0310 2023-10-06T05:19:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8013910 con 0x7f90f80b0310 2023-10-06T05:19:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f816a5e0 con 0x7f90f80b0310 2023-10-06T05:19:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.011+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f815be40 con 0x7f90f80b0310 2023-10-06T05:19:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.012+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f815bca0 con 0x7f90f80b0310 2023-10-06T05:19:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.013+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90e8013d00 con 0x7f90f80b0310 2023-10-06T05:19:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.014+0000 7f90edffb700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90dc066f30 0x7f90dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.014+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f90e8084470 con 0x7f90f80b0310 2023-10-06T05:19:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.015+0000 7f90fdf9b700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90dc066f30 0x7f90dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:40.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.015+0000 7f90fdf9b700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90dc066f30 0x7f90dc0693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90f80add90 tx=0x7f90f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.019+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90f815bca0 con 0x7f90f80b0310 2023-10-06T05:19:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.267+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f90f815bca0 con 0x7f90f80b0310 2023-10-06T05:19:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.267+0000 7f90edffb700 1 -- 172.21.15.33:0/1013399882 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f90f815bca0 con 0x7f90f80b0310 2023-10-06T05:19:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:19:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.269+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90dc066f30 msgr2=0x7f90dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.269+0000 7f90fef9d700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90dc066f30 0x7f90dc0693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90f80add90 tx=0x7f90f4017000 comp rx=0 tx=0).stop 2023-10-06T05:19:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.269+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 msgr2=0x7f90f815c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.269+0000 7f90fef9d700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 0x7f90f815c640 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e8d0 tx=0x7f90e8008040 comp rx=0 tx=0).stop 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 shutdown_connections 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f90dc066f30 0x7f90dc0693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90f8159530 0x7f90f8169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f806df70 0x7f90f80cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 --2- 172.21.15.33:0/1013399882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80b0310 0x7f90f815c640 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 >> 172.21.15.33:0/1013399882 conn(0x7f90f806c560 msgr2=0x7f90f806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 shutdown_connections 2023-10-06T05:19:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:40.270+0000 7f90fef9d700 1 -- 172.21.15.33:0/1013399882 wait complete. 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:19:40.289 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:19:40.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:40.290 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:19:40.329 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:19:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:19:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:19:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:19:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41977sec preferred_lft 41977sec 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:19:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:40.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:40.333 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:19:40.429 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:19:40.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:40.433 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:19:40.471 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:19:40.471 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:19:40.471 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:19:40.471 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:19:40.472 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:19:40.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:40.473 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:19:40.473 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:19:40.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:19:40.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:19:40.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:19:40.473 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:19:40.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:40.673 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:19:40.710 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:19:40.710 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:19:40.710 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:19:40.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:40.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:19:40.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:19:40.819 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:19:40.856 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:19:40.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:19:40.856 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:19:40.933 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:19:40.933 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:19:40.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:19:41.064 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:19:41.064 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:19:41.064 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:19:41.080 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:19:41.080 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:19:41.081 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:19:41.081 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:19:41.081 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:19:41.102 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:19:41.102 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:19:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:19:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:19:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:19:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:19:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:19:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:19:41.109 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:19:41.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:41.109 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:19:41.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:19:41.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.361+0000 7f9edd79d700 1 Processor -- start 2023-10-06T05:19:41.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.361+0000 7f9edd79d700 1 -- start start 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.362+0000 7f9edd79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ed813faa0 0x7f9ed813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.362+0000 7f9edd79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.362+0000 7f9edd79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed80c9cb0 con 0x7f9ed806aad0 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.362+0000 7f9edd79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed806b660 con 0x7f9ed813faa0 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.362+0000 7f9edd79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80ca270 con 0x7f9ed80c7940 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9edcf9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ed806aad0 0x7f9ed806af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45780/0 (socket says 172.21.15.33:45780) 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9edcf9c700 1 -- 172.21.15.33:0/2108623111 learned_addr learned my addr 172.21.15.33:0/2108623111 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/2108623111 >> v2:172.21.15.33:3300/0 conn(0x7f9ed813faa0 0x7f9ed813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed77fe700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/2108623111 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130623575 0 0) 0x7f9ed80c9cb0 con 0x7f9ed806aad0 2023-10-06T05:19:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed7fff700 1 -- 172.21.15.33:0/2108623111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 msgr2=0x7f9ed80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed7fff700 1 -- 172.21.15.33:0/2108623111 >> v1:172.21.15.33:6789/0 conn(0x7f9ed806aad0 legacy=0x7f9ed806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed7fff700 1 -- 172.21.15.33:0/2108623111 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed806b840 con 0x7f9ed813faa0 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed77fe700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.363+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813fe70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098e0 tx=0x7f9ec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75839ed252536122 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/2108623111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8009e80 con 0x7f9ed813faa0 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/2108623111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800b860 con 0x7f9ed813faa0 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/2108623111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800bbe0 con 0x7f9ed813faa0 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 -- 172.21.15.33:0/2108623111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 msgr2=0x7f9ed813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813fe70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098e0 tx=0x7f9ec80091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 -- 172.21.15.33:0/2108623111 shutdown_connections 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813fe70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 --2- 172.21.15.33:0/2108623111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 -- 172.21.15.33:0/2108623111 >> 172.21.15.33:0/2108623111 conn(0x7f9ed806d030 msgr2=0x7f9ed806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:41.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 -- 172.21.15.33:0/2108623111 shutdown_connections 2023-10-06T05:19:41.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.364+0000 7f9edd79d700 1 -- 172.21.15.33:0/2108623111 wait complete. 2023-10-06T05:19:41.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.366+0000 7f9edd79d700 1 Processor -- start 2023-10-06T05:19:41.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.366+0000 7f9edd79d700 1 -- start start 2023-10-06T05:19:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.367+0000 7f9edd79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed806aad0 0x7f9ed80c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.367+0000 7f9edd79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.367+0000 7f9edd79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.367+0000 7f9edd79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80c4080 con 0x7f9ed813faa0 2023-10-06T05:19:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.367+0000 7f9edd79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80c4290 con 0x7f9ed806aad0 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.367+0000 7f9edd79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80b0020 con 0x7f9ed80c7940 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43876/0 (socket says 172.21.15.33:43876) 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 -- 172.21.15.33:0/1281132497 learned_addr learned my addr 172.21.15.33:0/1281132497 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9ed77fe700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed806aad0 0x7f9ed80c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 -- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 msgr2=0x7f9ed80c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 -- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed806aad0 msgr2=0x7f9ed80c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed806aad0 0x7f9ed80c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 -- 172.21.15.33:0/1281132497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009420 con 0x7f9ed813faa0 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed806aad0 0x7f9ed80c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9edcf9c700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813da00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000cc90 tx=0x7f9ed0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.368+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000a960 con 0x7f9ed813faa0 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.369+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed000aac0 con 0x7f9ed813faa0 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.369+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000ae80 con 0x7f9ed813faa0 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.369+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed80b0300 con 0x7f9ed813faa0 2023-10-06T05:19:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.369+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed80b08c0 con 0x7f9ed813faa0 2023-10-06T05:19:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.370+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed80b0ae0 con 0x7f9ed813faa0 2023-10-06T05:19:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.371+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ed0007300 con 0x7f9ed813faa0 2023-10-06T05:19:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.371+0000 7f9ebffff700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ec4066f30 0x7f9ec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.371+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ed0084b10 con 0x7f9ed813faa0 2023-10-06T05:19:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.372+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ec4066f30 0x7f9ec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:41.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.373+0000 7f9ed7fff700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ec4066f30 0x7f9ec40693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80093f0 tx=0x7f9ec8019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:41.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.375+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ed0051c00 con 0x7f9ed813faa0 2023-10-06T05:19:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.622+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ed804c380 con 0x7f9ed813faa0 2023-10-06T05:19:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.623+0000 7f9ebffff700 1 -- 172.21.15.33:0/1281132497 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ed0055220 con 0x7f9ed813faa0 2023-10-06T05:19:41.625 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:19:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ec4066f30 msgr2=0x7f9ec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ec4066f30 0x7f9ec40693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80093f0 tx=0x7f9ec8019000 comp rx=0 tx=0).stop 2023-10-06T05:19:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 msgr2=0x7f9ed813da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813da00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000cc90 tx=0x7f9ed0001a80 comp rx=0 tx=0).stop 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 shutdown_connections 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ec4066f30 0x7f9ec40693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed813faa0 0x7f9ed813da00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed806aad0 0x7f9ed80c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 --2- 172.21.15.33:0/1281132497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80c7940 0x7f9ed80c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.625+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 >> 172.21.15.33:0/1281132497 conn(0x7f9ed806d030 msgr2=0x7f9ed806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.626+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 shutdown_connections 2023-10-06T05:19:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:41.626+0000 7f9edd79d700 1 -- 172.21.15.33:0/1281132497 wait complete. 2023-10-06T05:19:41.644 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:19:41.645 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:19:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:41.646 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:19:41.687 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:19:41.687 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:19:41.687 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:41.688 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41979sec preferred_lft 41979sec 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:19:41.689 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:19:41.690 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:19:41.690 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.690 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:19:41.690 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:19:41.690 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:19:41.690 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:19:41.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:41.691 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:19:41.794 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:19:41.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:41.797 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:19:41.836 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:19:41.836 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:19:41.836 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:19:41.836 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:19:41.837 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:19:41.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:41.837 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:19:41.837 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:19:41.837 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:19:41.837 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:19:41.837 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:19:41.837 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:19:42.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:42.033 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:19:42.069 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:19:42.069 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:19:42.069 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:19:42.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:42.069 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:19:42.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:19:42.171 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:19:42.208 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:19:42.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:19:42.209 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:19:42.284 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:19:42.284 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:19:42.284 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:19:42.413 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:19:42.413 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:19:42.413 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:19:42.429 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:19:42.430 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:19:42.430 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:19:42.430 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:19:42.430 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:19:42.451 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:19:42.451 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:19:42.451 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:19:42.451 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:19:42.452 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:19:42.452 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:19:42.452 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:19:42.452 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:19:42.456 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:19:42.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:42.457 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:19:42.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:19:42.737 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:19:42.737 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:19:42.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:19:42.943 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:19:42.944 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:19:42.944 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:19:42.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:19:42.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:19:43.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:19:43.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:43.024 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:19:43.060 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:19:43.060 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T05:19:43.060 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:19:43.060 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> print("Starting creation...") 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> start = time.time() 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> os.mkdir(path) 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> dfd = os.open(path, os.O_DIRECTORY) 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> fd = open(os.path.join(path, "childfile"), "w") 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> print("Finished creation in {0}s".format(time.time() - start)) 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> print("Starting fsync...") 2023-10-06T05:19:43.061 DEBUG:teuthology.orchestra.run.smithi033:> start = time.time() 2023-10-06T05:19:43.062 DEBUG:teuthology.orchestra.run.smithi033:> if False: 2023-10-06T05:19:43.062 DEBUG:teuthology.orchestra.run.smithi033:> os.fsync(dfd) 2023-10-06T05:19:43.062 DEBUG:teuthology.orchestra.run.smithi033:> else: 2023-10-06T05:19:43.062 DEBUG:teuthology.orchestra.run.smithi033:> os.fsync(fd) 2023-10-06T05:19:43.062 DEBUG:teuthology.orchestra.run.smithi033:> print("Finished fsync in {0}s".format(time.time() - start)) 2023-10-06T05:19:43.062 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:19:43.176 INFO:teuthology.orchestra.run.smithi033.stdout:Starting creation... 2023-10-06T05:19:43.176 INFO:teuthology.orchestra.run.smithi033.stdout:Finished creation in 0.005806922912597656s 2023-10-06T05:19:43.176 INFO:teuthology.orchestra.run.smithi033.stdout:Starting fsync... 2023-10-06T05:19:43.176 INFO:teuthology.orchestra.run.smithi033.stdout:Finished fsync in 0.001954793930053711s 2023-10-06T05:19:43.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:19:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.580+0000 7fddbbfff700 1 Processor -- start 2023-10-06T05:19:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.580+0000 7fddbbfff700 1 -- start start 2023-10-06T05:19:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fddbc156880 0x7fddbc0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc06a820 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc15a2f0 con 0x7fddbc159e10 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbbfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc0cc290 con 0x7fddbc156880 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fddbc0c7ec0 0x7fddbc06ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45790/0 (socket says 172.21.15.33:45790) 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbb7fe700 1 -- 172.21.15.33:0/1397793134 learned_addr learned my addr 172.21.15.33:0/1397793134 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.582+0000 7fddbaffd700 1 --2- 172.21.15.33:0/1397793134 >> v2:172.21.15.167:3300/0 conn(0x7fddbc156880 0x7fddbc0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214237331 0 0) 0x7fddbc06a820 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdda4003570 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48352601 0 0) 0x7fddbc15a2f0 con 0x7fddbc159e10 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddbc06a820 con 0x7fddbc159e10 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddbaffd700 1 -- 172.21.15.33:0/1397793134 >> v1:172.21.15.167:6789/0 conn(0x7fddbc159e10 legacy=0x7fddbc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddbaffd700 1 -- 172.21.15.33:0/1397793134 >> v1:172.21.15.33:6789/0 conn(0x7fddbc0c7ec0 legacy=0x7fddbc06ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddbaffd700 1 -- 172.21.15.33:0/1397793134 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc15a5d0 con 0x7fddbc156880 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddbaffd700 1 --2- 172.21.15.33:0/1397793134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0c7980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddac0098e0 tx=0x7fddac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f46d3e1c8e64710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac011630 con 0x7fddbc156880 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac011790 con 0x7fddbc156880 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.583+0000 7fddb9ffb700 1 -- 172.21.15.33:0/1397793134 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac011b30 con 0x7fddbc156880 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 -- 172.21.15.33:0/1397793134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 msgr2=0x7fddbc0c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1397793134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0c7980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddac0098e0 tx=0x7fddac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 -- 172.21.15.33:0/1397793134 shutdown_connections 2023-10-06T05:19:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1397793134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0c7980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 -- 172.21.15.33:0/1397793134 >> 172.21.15.33:0/1397793134 conn(0x7fddbc06c560 msgr2=0x7fddbc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 -- 172.21.15.33:0/1397793134 shutdown_connections 2023-10-06T05:19:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.584+0000 7fddbbfff700 1 -- 172.21.15.33:0/1397793134 wait complete. 2023-10-06T05:19:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.586+0000 7fddbbfff700 1 Processor -- start 2023-10-06T05:19:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.586+0000 7fddbbfff700 1 -- start start 2023-10-06T05:19:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 0x7fddbc14f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddbc159e10 0x7fddbc16d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc0afce0 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc0affc0 con 0x7fddbc156880 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc0b02a0 con 0x7fddbc159e10 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 0x7fddbc14f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 0x7fddbc14f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43878/0 (socket says 172.21.15.33:43878) 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 -- 172.21.15.33:0/1765278877 learned_addr learned my addr 172.21.15.33:0/1765278877 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddba7fc700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbb7fe700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddbc159e10 0x7fddbc16d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 -- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddbc159e10 msgr2=0x7fddbc16d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddbc159e10 0x7fddbc16d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 -- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 msgr2=0x7fddbc0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.587+0000 7fddbaffd700 1 -- 172.21.15.33:0/1765278877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac009420 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.588+0000 7fddbaffd700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 0x7fddbc14f360 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0636f0 tx=0x7fddac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.588+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac01f070 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.588+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac000bc0 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.588+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc156df0 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.588+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac025420 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.588+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc0b20b0 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.589+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc156c50 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.590+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddac023070 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.591+0000 7fdda2ffd700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdda8066f30 0x7fdda80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.591+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fddac085a20 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.591+0000 7fddba7fc700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdda8066f30 0x7fdda80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.592+0000 7fddba7fc700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdda8066f30 0x7fdda80693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fddbc16d100 tx=0x7fddb0007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.595+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddbc156c50 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.843+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fddbc157030 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.845 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:19:43.843+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:19:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.850+0000 7fdda2ffd700 1 -- 172.21.15.33:0/1765278877 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7fddbc157030 con 0x7fddbc0c7ec0 2023-10-06T05:19:43.853 INFO:tasks.ceph.mds.c.smithi033.stderr: -1090> 2023-10-06T05:19:34.264+0000 7f6bd63f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:19:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdda8066f30 msgr2=0x7fdda80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdda8066f30 0x7fdda80693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fddbc16d100 tx=0x7fddb0007d50 comp rx=0 tx=0).stop 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 msgr2=0x7fddbc14f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 0x7fddbc14f360 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0636f0 tx=0x7fddac017000 comp rx=0 tx=0).stop 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 shutdown_connections 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdda8066f30 0x7fdda80693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddbc159e10 0x7fddbc16d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0c7ec0 0x7fddbc14f360 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 --2- 172.21.15.33:0/1765278877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddbc156880 0x7fddbc0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.852+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 >> 172.21.15.33:0/1765278877 conn(0x7fddbc06c560 msgr2=0x7fddbc0c0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.853+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 shutdown_connections 2023-10-06T05:19:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:43.853+0000 7fddbbfff700 1 -- 172.21.15.33:0/1765278877 wait complete. 2023-10-06T05:19:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:19:43.872 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T05:19:43.872 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:19:43.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:43.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:19:43.890 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:19:43.945 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:19:43.945 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:19:43.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:19:43.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:19:44.061 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:19:44.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:19:44.061 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:19:44.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:19:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.329+0000 7f76f4f9d700 1 Processor -- start 2023-10-06T05:19:44.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.329+0000 7f76f4f9d700 1 -- start start 2023-10-06T05:19:44.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.330+0000 7f76f4f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76f006c7b0 0x7f76f0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.330+0000 7f76f4f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f006cb80 0x7f76f015d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.330+0000 7f76f4f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f015d3a0 con 0x7f76f00aff90 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.330+0000 7f76f4f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00a9fa0 con 0x7f76f006c7b0 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.330+0000 7f76f4f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00b0470 con 0x7f76f006cb80 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.330+0000 7f76ef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76f006c7b0 0x7f76f0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76f006c7b0 0x7f76f0067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43888/0 (socket says 172.21.15.33:43888) 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 -- 172.21.15.33:0/1950346315 learned_addr learned my addr 172.21.15.33:0/1950346315 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76effff700 1 --2- 172.21.15.33:0/1950346315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f006cb80 0x7f76f015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 -- 172.21.15.33:0/1950346315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f006cb80 msgr2=0x7f76f015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 --2- 172.21.15.33:0/1950346315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f006cb80 0x7f76f015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 -- 172.21.15.33:0/1950346315 >> v1:172.21.15.167:6789/0 conn(0x7f76f00aff90 legacy=0x7f76f00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 -- 172.21.15.33:0/1950346315 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00b0750 con 0x7f76f006c7b0 2023-10-06T05:19:44.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ef7fe700 1 --2- 172.21.15.33:0/1950346315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006c7b0 0x7f76f0067fb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009990 tx=0x7f76e0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e418901680e11ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.331+0000 7f76ee7fc700 1 -- 172.21.15.33:0/1950346315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0005c10 con 0x7f76f006c7b0 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76ee7fc700 1 -- 172.21.15.33:0/1950346315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0005d70 con 0x7f76f006c7b0 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76ee7fc700 1 -- 172.21.15.33:0/1950346315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000b880 con 0x7f76f006c7b0 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 -- 172.21.15.33:0/1950346315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006c7b0 msgr2=0x7f76f0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/1950346315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006c7b0 0x7f76f0067fb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009990 tx=0x7f76e0009220 comp rx=0 tx=0).stop 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 -- 172.21.15.33:0/1950346315 shutdown_connections 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/1950346315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006c7b0 0x7f76f0067fb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/1950346315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f006cb80 0x7f76f015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 -- 172.21.15.33:0/1950346315 >> 172.21.15.33:0/1950346315 conn(0x7f76f006d670 msgr2=0x7f76f006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:44.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 -- 172.21.15.33:0/1950346315 shutdown_connections 2023-10-06T05:19:44.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.332+0000 7f76f4f9d700 1 -- 172.21.15.33:0/1950346315 wait complete. 2023-10-06T05:19:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.334+0000 7f76f4f9d700 1 Processor -- start 2023-10-06T05:19:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.334+0000 7f76f4f9d700 1 -- start start 2023-10-06T05:19:44.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76f4f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f006c7b0 0x7f76f016b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76f4f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 0x7f76f0152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76f4f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f00aff90 0x7f76f015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76f4f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0153e50 con 0x7f76f006cb80 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76f4f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00cdf00 con 0x7f76f006c7b0 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76f4f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00ad230 con 0x7f76f00aff90 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 0x7f76f0152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 0x7f76f0152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43894/0 (socket says 172.21.15.33:43894) 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76eeffd700 1 -- 172.21.15.33:0/389591215 learned_addr learned my addr 172.21.15.33:0/389591215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76ef7fe700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f006c7b0 0x7f76f016b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.335+0000 7f76effff700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f00aff90 0x7f76f015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76eeffd700 1 -- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f00aff90 msgr2=0x7f76f015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76eeffd700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f00aff90 0x7f76f015bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76eeffd700 1 -- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f006c7b0 msgr2=0x7f76f016b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76eeffd700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f006c7b0 0x7f76f016b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76eeffd700 1 -- 172.21.15.33:0/389591215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009480 con 0x7f76f006cb80 2023-10-06T05:19:44.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76eeffd700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 0x7f76f0152cc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c5c0 tx=0x7f76e4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4011590 con 0x7f76f006cb80 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4011710 con 0x7f76f006cb80 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f00ad510 con 0x7f76f006cb80 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4011ad0 con 0x7f76f006cb80 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.336+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f00adb50 con 0x7f76f006cb80 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.337+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f00adce0 con 0x7f76f006cb80 2023-10-06T05:19:44.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.338+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76e401a360 con 0x7f76f006cb80 2023-10-06T05:19:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.339+0000 7f76d77fe700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f76dc066f30 0x7f76dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.339+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f76e4083590 con 0x7f76f006cb80 2023-10-06T05:19:44.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.340+0000 7f76ef7fe700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f76dc066f30 0x7f76dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.341+0000 7f76ef7fe700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f76dc066f30 0x7f76dc0693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009260 tx=0x7f76e0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:44.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.344+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76e40505d0 con 0x7f76f006cb80 2023-10-06T05:19:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.595+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f76f004c380 con 0x7f76f006cb80 2023-10-06T05:19:44.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.849+0000 7f76d77fe700 1 -- 172.21.15.33:0/389591215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v45) v1 ==== 162+0+0 (secure 0 0 0) 0x7f76e4053bf0 con 0x7f76f006cb80 2023-10-06T05:19:44.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f76dc066f30 msgr2=0x7f76dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f76dc066f30 0x7f76dc0693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009260 tx=0x7f76e0016000 comp rx=0 tx=0).stop 2023-10-06T05:19:44.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 msgr2=0x7f76f0152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 0x7f76f0152cc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c5c0 tx=0x7f76e4007f50 comp rx=0 tx=0).stop 2023-10-06T05:19:44.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 shutdown_connections 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f76dc066f30 0x7f76dc0693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76f00aff90 0x7f76f015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f006c7b0 0x7f76f016b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 --2- 172.21.15.33:0/389591215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f006cb80 0x7f76f0152cc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 >> 172.21.15.33:0/389591215 conn(0x7f76f006d670 msgr2=0x7f76f0157530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 shutdown_connections 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:44.852+0000 7f76f4f9d700 1 -- 172.21.15.33:0/389591215 wait complete. 2023-10-06T05:19:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:19:44.875 INFO:tasks.cephfs.test_client_recovery:Waiting for reconnect... 2023-10-06T05:19:44.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.062+0000 7f6d99b9d700 1 Processor -- start 2023-10-06T05:19:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.062+0000 7f6d99b9d700 1 -- start start 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d99b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d94155140 0x7f6d94158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d99b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d940c8e40 con 0x7f6d940699c0 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d99b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d940c99e0 con 0x7f6d940adf50 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d99b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d940ae3f0 con 0x7f6d94155140 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d98b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d94155140 0x7f6d94158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d9939c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d940699c0 0x7f6d940c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45802/0 (socket says 172.21.15.33:45802) 2023-10-06T05:19:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.063+0000 7f6d9939c700 1 -- 172.21.15.33:0/660539106 learned_addr learned my addr 172.21.15.33:0/660539106 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d98b9b700 1 -- 172.21.15.33:0/660539106 >> v1:172.21.15.167:6789/0 conn(0x7f6d940adf50 legacy=0x7f6d940c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d98b9b700 1 -- 172.21.15.33:0/660539106 >> v1:172.21.15.33:6789/0 conn(0x7f6d940699c0 legacy=0x7f6d940c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d98b9b700 1 -- 172.21.15.33:0/660539106 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940ae6c0 con 0x7f6d94155140 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d937fe700 1 -- 172.21.15.33:0/660539106 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064848440 0 0) 0x7f6d940c8e40 con 0x7f6d940699c0 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d98b9b700 1 --2- 172.21.15.33:0/660539106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94158700 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009910 tx=0x7f6d880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861fa7e77814548a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d937fe700 1 -- 172.21.15.33:0/660539106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800b7e0 con 0x7f6d94155140 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d937fe700 1 -- 172.21.15.33:0/660539106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800b940 con 0x7f6d94155140 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.064+0000 7f6d937fe700 1 -- 172.21.15.33:0/660539106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800bce0 con 0x7f6d94155140 2023-10-06T05:19:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 -- 172.21.15.33:0/660539106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 msgr2=0x7f6d94158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/660539106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94158700 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009910 tx=0x7f6d880091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 -- 172.21.15.33:0/660539106 shutdown_connections 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/660539106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94158700 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 -- 172.21.15.33:0/660539106 >> 172.21.15.33:0/660539106 conn(0x7f6d9406d030 msgr2=0x7f6d9406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 -- 172.21.15.33:0/660539106 shutdown_connections 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.065+0000 7f6d99b9d700 1 -- 172.21.15.33:0/660539106 wait complete. 2023-10-06T05:19:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.067+0000 7f6d99b9d700 1 Processor -- start 2023-10-06T05:19:45.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.067+0000 7f6d99b9d700 1 -- start start 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d99b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d940699c0 0x7f6d94151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d99b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d940adf50 0x7f6d941515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d99b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94167810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d99b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c7730 con 0x7f6d94155140 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d99b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c3560 con 0x7f6d940adf50 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d99b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940c6be0 con 0x7f6d940699c0 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94167810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43912/0 (socket says 172.21.15.33:43912) 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 -- 172.21.15.33:0/453336603 learned_addr learned my addr 172.21.15.33:0/453336603 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d93fff700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d940adf50 0x7f6d941515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d98b9b700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d940699c0 0x7f6d94151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 -- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d940699c0 msgr2=0x7f6d94151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d940699c0 0x7f6d94151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 -- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d940adf50 msgr2=0x7f6d941515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d940adf50 0x7f6d941515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d9939c700 1 -- 172.21.15.33:0/453336603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009420 con 0x7f6d94155140 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.068+0000 7f6d93fff700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d940adf50 0x7f6d941515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.069+0000 7f6d9939c700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94167810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00cc20 tx=0x7f6d8c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.069+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c004bb0 con 0x7f6d94155140 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.069+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c004d10 con 0x7f6d94155140 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.069+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00abd0 con 0x7f6d94155140 2023-10-06T05:19:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.069+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d940c6e90 con 0x7f6d94155140 2023-10-06T05:19:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.069+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d940c7450 con 0x7f6d94155140 2023-10-06T05:19:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.070+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9415af60 con 0x7f6d94155140 2023-10-06T05:19:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.071+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d8c0070f0 con 0x7f6d94155140 2023-10-06T05:19:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.072+0000 7f6d90ff9700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.072+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6d8c084640 con 0x7f6d94155140 2023-10-06T05:19:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.072+0000 7f6d98b9b700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.073+0000 7f6d98b9b700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d880093f0 tx=0x7f6d88016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:45.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.076+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d8c051700 con 0x7f6d94155140 2023-10-06T05:19:45.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.326+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d9404c380 con 0x7f6d94155140 2023-10-06T05:19:45.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.327+0000 7f6d90ff9700 1 -- 172.21.15.33:0/453336603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f6d8c054d20 con 0x7f6d94155140 2023-10-06T05:19:45.328 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:45.328 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:44.850459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 msgr2=0x7f6d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d880093f0 tx=0x7f6d88016000 comp rx=0 tx=0).stop 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 msgr2=0x7f6d94167810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94167810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00cc20 tx=0x7f6d8c0019b0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 shutdown_connections 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d94155140 0x7f6d94167810 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d940699c0 0x7f6d94151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 --2- 172.21.15.33:0/453336603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d940adf50 0x7f6d941515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 >> 172.21.15.33:0/453336603 conn(0x7f6d9406d030 msgr2=0x7f6d94153ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 shutdown_connections 2023-10-06T05:19:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:45.329+0000 7f6d99b9d700 1 -- 172.21.15.33:0/453336603 wait complete. 2023-10-06T05:19:45.332 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 46 2023-10-06T05:19:45.348 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2023-10-06T05:19:46.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:46.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.546+0000 7fe11b59e700 1 Processor -- start 2023-10-06T05:19:46.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.546+0000 7fe11b59e700 1 -- start start 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0ccfc0 0x7fe11c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe11c0cf3b0 0x7fe11c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe11c0cc750 0x7fe11c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0cbd80 con 0x7fe11c0cc750 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0af5f0 con 0x7fe11c0cf3b0 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0af8d0 con 0x7fe11c0ccfc0 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.547+0000 7fe11ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe11c0cc750 0x7fe11c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe11c0cc750 0x7fe11c1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43916/0 (socket says 172.21.15.33:43916) 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 -- 172.21.15.33:0/840120060 learned_addr learned my addr 172.21.15.33:0/840120060 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe119d9b700 1 --2- 172.21.15.33:0/840120060 >> v2:172.21.15.167:3300/0 conn(0x7fe11c0cf3b0 0x7fe11c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11a59c700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0ccfc0 0x7fe11c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 -- 172.21.15.33:0/840120060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0ccfc0 msgr2=0x7fe11c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0ccfc0 0x7fe11c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 -- 172.21.15.33:0/840120060 >> v2:172.21.15.167:3300/0 conn(0x7fe11c0cf3b0 msgr2=0x7fe11c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 --2- 172.21.15.33:0/840120060 >> v2:172.21.15.167:3300/0 conn(0x7fe11c0cf3b0 0x7fe11c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 -- 172.21.15.33:0/840120060 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c0afbb0 con 0x7fe11c0cc750 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe119d9b700 1 --2- 172.21.15.33:0/840120060 >> v2:172.21.15.167:3300/0 conn(0x7fe11c0cf3b0 0x7fe11c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11a59c700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0ccfc0 0x7fe11c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.548+0000 7fe11ad9d700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0cc750 0x7fe11c1599e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe11c071260 tx=0x7fe114009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3489932998bb52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11959a700 1 -- 172.21.15.33:0/840120060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400e070 con 0x7fe11c0cc750 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11959a700 1 -- 172.21.15.33:0/840120060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11400ba50 con 0x7fe11c0cc750 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11959a700 1 -- 172.21.15.33:0/840120060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140092e0 con 0x7fe11c0cc750 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 -- 172.21.15.33:0/840120060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0cc750 msgr2=0x7fe11c1599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0cc750 0x7fe11c1599e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe11c071260 tx=0x7fe114009840 comp rx=0 tx=0).stop 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 -- 172.21.15.33:0/840120060 shutdown_connections 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0cc750 0x7fe11c1599e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 --2- 172.21.15.33:0/840120060 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0ccfc0 0x7fe11c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 --2- 172.21.15.33:0/840120060 >> v2:172.21.15.167:3300/0 conn(0x7fe11c0cf3b0 0x7fe11c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 -- 172.21.15.33:0/840120060 >> 172.21.15.33:0/840120060 conn(0x7fe11c06cff0 msgr2=0x7fe11c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 -- 172.21.15.33:0/840120060 shutdown_connections 2023-10-06T05:19:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.549+0000 7fe11b59e700 1 -- 172.21.15.33:0/840120060 wait complete. 2023-10-06T05:19:46.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.551+0000 7fe11b59e700 1 Processor -- start 2023-10-06T05:19:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.551+0000 7fe11b59e700 1 -- start start 2023-10-06T05:19:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0cc750 0x7fe11c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c0ccfc0 0x7fe11c144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 0x7fe11c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c150b30 con 0x7fe11c0b32a0 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0c9260 con 0x7fe11c0ccfc0 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0c9540 con 0x7fe11c0cc750 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 0x7fe11c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 0x7fe11c0b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43922/0 (socket says 172.21.15.33:43922) 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11ad9d700 1 -- 172.21.15.33:0/611454776 learned_addr learned my addr 172.21.15.33:0/611454776 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe11a59c700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0cc750 0x7fe11c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.552+0000 7fe119d9b700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c0ccfc0 0x7fe11c144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11ad9d700 1 -- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0cc750 msgr2=0x7fe11c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11ad9d700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0cc750 0x7fe11c0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11ad9d700 1 -- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c0ccfc0 msgr2=0x7fe11c144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11ad9d700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c0ccfc0 0x7fe11c144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11ad9d700 1 -- 172.21.15.33:0/611454776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114009ae0 con 0x7fe11c0b32a0 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe119d9b700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c0ccfc0 0x7fe11c144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11ad9d700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 0x7fe11c0b3e70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe114009880 tx=0x7fe114018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140032f0 con 0x7fe11c0b32a0 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114008dd0 con 0x7fe11c0b32a0 2023-10-06T05:19:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114026690 con 0x7fe11c0b32a0 2023-10-06T05:19:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c0c9820 con 0x7fe11c0b32a0 2023-10-06T05:19:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.553+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c0b2c70 con 0x7fe11c0b32a0 2023-10-06T05:19:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.554+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c04c380 con 0x7fe11c0b32a0 2023-10-06T05:19:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.555+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe114003040 con 0x7fe11c0b32a0 2023-10-06T05:19:46.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.556+0000 7fe10a7fc700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe0fc066f30 0x7fe0fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:46.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.556+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe1140866c0 con 0x7fe11c0b32a0 2023-10-06T05:19:46.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.556+0000 7fe11a59c700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe0fc066f30 0x7fe0fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:46.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.557+0000 7fe11a59c700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe0fc066f30 0x7fe0fc0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0b6330 tx=0x7fe10c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:46.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.560+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe114053700 con 0x7fe11c0b32a0 2023-10-06T05:19:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.808+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe11c089560 con 0x7fe11c0b32a0 2023-10-06T05:19:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.809+0000 7fe10a7fc700 1 -- 172.21.15.33:0/611454776 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fe114056d20 con 0x7fe11c0b32a0 2023-10-06T05:19:46.810 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:46.810 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:46.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe0fc066f30 msgr2=0x7fe0fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:46.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe0fc066f30 0x7fe0fc0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0b6330 tx=0x7fe10c008040 comp rx=0 tx=0).stop 2023-10-06T05:19:46.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 msgr2=0x7fe11c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 0x7fe11c0b3e70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe114009880 tx=0x7fe114018000 comp rx=0 tx=0).stop 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 shutdown_connections 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe0fc066f30 0x7fe0fc0693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c0b32a0 0x7fe11c0b3e70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c0cc750 0x7fe11c0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 --2- 172.21.15.33:0/611454776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c0ccfc0 0x7fe11c144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 >> 172.21.15.33:0/611454776 conn(0x7fe11c06cff0 msgr2=0x7fe11c1523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 shutdown_connections 2023-10-06T05:19:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:46.811+0000 7fe11b59e700 1 -- 172.21.15.33:0/611454776 wait complete. 2023-10-06T05:19:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:46.830 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to up:reconnect 2023-10-06T05:19:46.831 DEBUG:tasks.cephfs.filesystem:reached state 'up:reconnect' in 1.955246925354004s 2023-10-06T05:19:46.831 INFO:tasks.cephfs.test_client_recovery:Waiting for active... 2023-10-06T05:19:46.831 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.017+0000 7fd41db9d700 1 Processor -- start 2023-10-06T05:19:47.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.017+0000 7fd41db9d700 1 -- start start 2023-10-06T05:19:47.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd41db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd41806ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:47.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd41db9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4180b0e10 0x7fd4180b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:47.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd41db9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41815e2d0 con 0x7fd4180b2e30 2023-10-06T05:19:47.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd41db9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41815d6c0 con 0x7fd4180b0e10 2023-10-06T05:19:47.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd41db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815d9a0 con 0x7fd4180b0a40 2023-10-06T05:19:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd417fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4180b0e10 0x7fd4180b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd417fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4180b0e10 0x7fd4180b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43930/0 (socket says 172.21.15.33:43930) 2023-10-06T05:19:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd417fff700 1 -- 172.21.15.33:0/3561447943 learned_addr learned my addr 172.21.15.33:0/3561447943 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd41cb9b700 1 --2- 172.21.15.33:0/3561447943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd41806ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd417fff700 1 -- 172.21.15.33:0/3561447943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 msgr2=0x7fd41806ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd417fff700 1 --2- 172.21.15.33:0/3561447943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd41806ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.019+0000 7fd417fff700 1 -- 172.21.15.33:0/3561447943 >> v1:172.21.15.167:6789/0 conn(0x7fd4180b2e30 legacy=0x7fd41815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:47.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd417fff700 1 -- 172.21.15.33:0/3561447943 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41815dc80 con 0x7fd4180b0e10 2023-10-06T05:19:47.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd417fff700 1 --2- 172.21.15.33:0/3561447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b0e10 0x7fd4180b28f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd4040099b0 tx=0x7fd404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2e2d3b9a81146 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:47.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd4177fe700 1 -- 172.21.15.33:0/3561447943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400b860 con 0x7fd4180b0e10 2023-10-06T05:19:47.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd4177fe700 1 -- 172.21.15.33:0/3561447943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400b9c0 con 0x7fd4180b0e10 2023-10-06T05:19:47.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd4177fe700 1 -- 172.21.15.33:0/3561447943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400bd60 con 0x7fd4180b0e10 2023-10-06T05:19:47.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd41db9d700 1 -- 172.21.15.33:0/3561447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b0e10 msgr2=0x7fd4180b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:47.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.020+0000 7fd41db9d700 1 --2- 172.21.15.33:0/3561447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b0e10 0x7fd4180b28f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd4040099b0 tx=0x7fd404009250 comp rx=0 tx=0).stop 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.021+0000 7fd41db9d700 1 -- 172.21.15.33:0/3561447943 shutdown_connections 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.021+0000 7fd41db9d700 1 --2- 172.21.15.33:0/3561447943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd41806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.021+0000 7fd41db9d700 1 --2- 172.21.15.33:0/3561447943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b0e10 0x7fd4180b28f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.021+0000 7fd41db9d700 1 -- 172.21.15.33:0/3561447943 >> 172.21.15.33:0/3561447943 conn(0x7fd41806e0f0 msgr2=0x7fd4180675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.021+0000 7fd41db9d700 1 -- 172.21.15.33:0/3561447943 shutdown_connections 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.021+0000 7fd41db9d700 1 -- 172.21.15.33:0/3561447943 wait complete. 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.022+0000 7fd41db9d700 1 Processor -- start 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.022+0000 7fd41db9d700 1 -- start start 2023-10-06T05:19:47.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd4181566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4180b0e10 0x7fd418169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 0x7fd4180c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815ba90 con 0x7fd4180b2e30 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815aec0 con 0x7fd4180b0e10 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41815b160 con 0x7fd4180b0a40 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 0x7fd4180c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 0x7fd4180c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43932/0 (socket says 172.21.15.33:43932) 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.023+0000 7fd41d39c700 1 -- 172.21.15.33:0/2399326578 learned_addr learned my addr 172.21.15.33:0/2399326578 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:47.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41cb9b700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd4181566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:47.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd417fff700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4180b0e10 0x7fd418169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:47.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41d39c700 1 -- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 msgr2=0x7fd4181566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:47.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41d39c700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd4181566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41d39c700 1 -- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4180b0e10 msgr2=0x7fd418169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:47.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41d39c700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4180b0e10 0x7fd418169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41d39c700 1 -- 172.21.15.33:0/2399326578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4040094f0 con 0x7fd4180b2e30 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd417fff700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4180b0e10 0x7fd418169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41cb9b700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd4181566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41d39c700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 0x7fd4180c31d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c840 tx=0x7fd410001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410012630 con 0x7fd4180b2e30 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4100127b0 con 0x7fd4180b2e30 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410012b70 con 0x7fd4180b2e30 2023-10-06T05:19:47.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.024+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41815b440 con 0x7fd4180b2e30 2023-10-06T05:19:47.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.025+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4180c2280 con 0x7fd4180b2e30 2023-10-06T05:19:47.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.026+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41806fcc0 con 0x7fd4180b2e30 2023-10-06T05:19:47.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.026+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd410003030 con 0x7fd4180b2e30 2023-10-06T05:19:47.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.027+0000 7fd414ff9700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3fc066f30 0x7fd3fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:47.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.027+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd4100846a0 con 0x7fd4180b2e30 2023-10-06T05:19:47.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.028+0000 7fd41cb9b700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3fc066f30 0x7fd3fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.029+0000 7fd41cb9b700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3fc066f30 0x7fd3fc0693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd40c006fd0 tx=0x7fd40c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.032+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd410051760 con 0x7fd4180b2e30 2023-10-06T05:19:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.277+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd41804c380 con 0x7fd4180b2e30 2023-10-06T05:19:47.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.278+0000 7fd414ff9700 1 -- 172.21.15.33:0/2399326578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd410054d80 con 0x7fd4180b2e30 2023-10-06T05:19:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:47.280 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3fc066f30 msgr2=0x7fd3fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3fc066f30 0x7fd3fc0693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd40c006fd0 tx=0x7fd40c008040 comp rx=0 tx=0).stop 2023-10-06T05:19:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 msgr2=0x7fd4180c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:47.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 0x7fd4180c31d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c840 tx=0x7fd410001670 comp rx=0 tx=0).stop 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 shutdown_connections 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3fc066f30 0x7fd3fc0693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4180b2e30 0x7fd4180c31d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4180b0a40 0x7fd4181566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 --2- 172.21.15.33:0/2399326578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4180b0e10 0x7fd418169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 >> 172.21.15.33:0/2399326578 conn(0x7fd41806e0f0 msgr2=0x7fd4180675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 shutdown_connections 2023-10-06T05:19:47.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:47.280+0000 7fd41db9d700 1 -- 172.21.15.33:0/2399326578 wait complete. 2023-10-06T05:19:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:47.302 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:48.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:48.297 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:48.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:48.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.529+0000 7f27997ff700 1 Processor -- start 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.530+0000 7f27997ff700 1 -- start start 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27997ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f279413fb70 0x7f279406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27997ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f279406ba40 0x7f27940c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27997ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27940ca8d0 con 0x7f27940ca350 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27997ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27940cabb0 con 0x7f279406ba40 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27997ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279406ad00 con 0x7f279413fb70 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27927fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f279406ba40 0x7f27940c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27937fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f27940ca350 0x7f27940af000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45810/0 (socket says 172.21.15.33:45810) 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f27937fe700 1 -- 172.21.15.33:0/3426488337 learned_addr learned my addr 172.21.15.33:0/3426488337 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.531+0000 7f2792ffd700 1 --2- 172.21.15.33:0/3426488337 >> v2:172.21.15.167:3300/0 conn(0x7f279413fb70 0x7f279406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f27927fc700 1 -- 172.21.15.33:0/3426488337 >> v2:172.21.15.167:3300/0 conn(0x7f279413fb70 msgr2=0x7f279406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f27927fc700 1 --2- 172.21.15.33:0/3426488337 >> v2:172.21.15.167:3300/0 conn(0x7f279413fb70 0x7f279406b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f27927fc700 1 -- 172.21.15.33:0/3426488337 >> v1:172.21.15.33:6789/0 conn(0x7f27940ca350 legacy=0x7f27940af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f27927fc700 1 -- 172.21.15.33:0/3426488337 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279406afe0 con 0x7f279406ba40 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f2791ffb700 1 -- 172.21.15.33:0/3426488337 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993797895 0 0) 0x7f27940ca8d0 con 0x7f27940ca350 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f27927fc700 1 --2- 172.21.15.33:0/3426488337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279406ba40 0x7f27940c9e10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f279406bf00 tx=0x7f27880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca562cce95fe27b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f2791ffb700 1 -- 172.21.15.33:0/3426488337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788011630 con 0x7f279406ba40 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f2791ffb700 1 -- 172.21.15.33:0/3426488337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788011790 con 0x7f279406ba40 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.532+0000 7f2791ffb700 1 -- 172.21.15.33:0/3426488337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788011b30 con 0x7f279406ba40 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 -- 172.21.15.33:0/3426488337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279406ba40 msgr2=0x7f27940c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 --2- 172.21.15.33:0/3426488337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279406ba40 0x7f27940c9e10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f279406bf00 tx=0x7f27880091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 -- 172.21.15.33:0/3426488337 shutdown_connections 2023-10-06T05:19:48.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 --2- 172.21.15.33:0/3426488337 >> v2:172.21.15.167:3300/0 conn(0x7f279413fb70 0x7f279406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 --2- 172.21.15.33:0/3426488337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279406ba40 0x7f27940c9e10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 -- 172.21.15.33:0/3426488337 >> 172.21.15.33:0/3426488337 conn(0x7f27940675f0 msgr2=0x7f27940679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:48.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 -- 172.21.15.33:0/3426488337 shutdown_connections 2023-10-06T05:19:48.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.533+0000 7f27997ff700 1 -- 172.21.15.33:0/3426488337 wait complete. 2023-10-06T05:19:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.534+0000 7f27997ff700 1 Processor -- start 2023-10-06T05:19:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.534+0000 7f27997ff700 1 -- start start 2023-10-06T05:19:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.535+0000 7f27997ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279406ba40 0x7f279413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.535+0000 7f27997ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27940ca350 0x7f27940ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.535+0000 7f27997ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 0x7f27940afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.535+0000 7f27997ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794159180 con 0x7f279413fb70 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.535+0000 7f27997ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27941673f0 con 0x7f27940ca350 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.535+0000 7f27997ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2794158600 con 0x7f279406ba40 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 0x7f27940afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 0x7f27940afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43946/0 (socket says 172.21.15.33:43946) 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 -- 172.21.15.33:0/2883590924 learned_addr learned my addr 172.21.15.33:0/2883590924 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 -- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279406ba40 msgr2=0x7f279413ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279406ba40 0x7f279413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 -- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27940ca350 msgr2=0x7f27940ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:48.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27927fc700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27940ca350 0x7f27940ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27940ca350 0x7f27940ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 -- 172.21.15.33:0/2883590924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2788009420 con 0x7f279413fb70 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27927fc700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27940ca350 0x7f27940ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27937fe700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 0x7f27940afc90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f278c00cc90 tx=0x7f278c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00cea0 con 0x7f279413fb70 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c00ab70 con 0x7f279413fb70 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c0070f0 con 0x7f279413fb70 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.536+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27941588e0 con 0x7f279413fb70 2023-10-06T05:19:48.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.537+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2794158ea0 con 0x7f279413fb70 2023-10-06T05:19:48.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.538+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27941675c0 con 0x7f279413fb70 2023-10-06T05:19:48.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.539+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f278c0074a0 con 0x7f279413fb70 2023-10-06T05:19:48.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.540+0000 7f277affd700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2780066f30 0x7f27800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:48.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.540+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f278c084be0 con 0x7f279413fb70 2023-10-06T05:19:48.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.540+0000 7f2792ffd700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2780066f30 0x7f27800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:48.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.541+0000 7f2792ffd700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2780066f30 0x7f27800693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2784009b10 tx=0x7f27840091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:48.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.544+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f278c051ca0 con 0x7f279413fb70 2023-10-06T05:19:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.790+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2794063ff0 con 0x7f279413fb70 2023-10-06T05:19:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.790+0000 7f277affd700 1 -- 172.21.15.33:0/2883590924 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f278c0552c0 con 0x7f279413fb70 2023-10-06T05:19:48.793 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:48.793 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2780066f30 msgr2=0x7f27800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2780066f30 0x7f27800693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2784009b10 tx=0x7f27840091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 msgr2=0x7f27940afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 0x7f27940afc90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f278c00cc90 tx=0x7f278c001a80 comp rx=0 tx=0).stop 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 shutdown_connections 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2780066f30 0x7f27800693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f279413fb70 0x7f27940afc90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f279406ba40 0x7f279413ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 --2- 172.21.15.33:0/2883590924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27940ca350 0x7f27940ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 >> 172.21.15.33:0/2883590924 conn(0x7f27940675f0 msgr2=0x7f27940af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 shutdown_connections 2023-10-06T05:19:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:48.793+0000 7f27997ff700 1 -- 172.21.15.33:0/2883590924 wait complete. 2023-10-06T05:19:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:48.813 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:49.814 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:50.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.010+0000 7f25ddb9d700 1 Processor -- start 2023-10-06T05:19:50.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.010+0000 7f25ddb9d700 1 -- start start 2023-10-06T05:19:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25ddb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25d8159520 0x7f25d80c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25ddb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c82a0 0x7f25d806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25ddb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d806c820 con 0x7f25d80c6170 2023-10-06T05:19:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25ddb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d80c6610 con 0x7f25d8159520 2023-10-06T05:19:50.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25ddb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d806a160 con 0x7f25d80c82a0 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25dcb9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25d8159520 0x7f25d80c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25dd39c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c82a0 0x7f25d806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25d7fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f25d80c6170 0x7f25d80c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:42592/0 (socket says 172.21.15.33:42592) 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.012+0000 7f25d7fff700 1 -- 172.21.15.33:0/3207154840 learned_addr learned my addr 172.21.15.33:0/3207154840 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25d77fe700 1 -- 172.21.15.33:0/3207154840 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510099986 0 0) 0x7f25d806c820 con 0x7f25d80c6170 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25d77fe700 1 -- 172.21.15.33:0/3207154840 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c8003570 con 0x7f25d80c6170 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25dcb9b700 1 -- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c82a0 msgr2=0x7f25d806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25dcb9b700 1 --2- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c82a0 0x7f25d806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25dcb9b700 1 -- 172.21.15.33:0/3207154840 >> v1:172.21.15.167:6789/0 conn(0x7f25d80c6170 legacy=0x7f25d80c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25dcb9b700 1 -- 172.21.15.33:0/3207154840 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80c8800 con 0x7f25d8159520 2023-10-06T05:19:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25dd39c700 1 --2- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c82a0 0x7f25d806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25dcb9b700 1 --2- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d8159520 0x7f25d80c5c30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0098e0 tx=0x7f25cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ff5734fb7321c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25d77fe700 1 -- 172.21.15.33:0/3207154840 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc011630 con 0x7f25d8159520 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25d77fe700 1 -- 172.21.15.33:0/3207154840 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc011790 con 0x7f25d8159520 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.013+0000 7f25d77fe700 1 -- 172.21.15.33:0/3207154840 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc011b30 con 0x7f25d8159520 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d8159520 msgr2=0x7f25d80c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d8159520 0x7f25d80c5c30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0098e0 tx=0x7f25cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3207154840 shutdown_connections 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d8159520 0x7f25d80c5c30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3207154840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c82a0 0x7f25d806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3207154840 >> 172.21.15.33:0/3207154840 conn(0x7f25d80cc930 msgr2=0x7f25d80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3207154840 shutdown_connections 2023-10-06T05:19:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.014+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3207154840 wait complete. 2023-10-06T05:19:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.015+0000 7f25ddb9d700 1 Processor -- start 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.016+0000 7f25ddb9d700 1 -- start start 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.016+0000 7f25ddb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c6170 0x7f25d8147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.016+0000 7f25ddb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d80c82a0 0x7f25d80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25ddb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 0x7f25d806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25ddb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8169690 con 0x7f25d8159520 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25ddb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d81479b0 con 0x7f25d80c82a0 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25ddb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d815b270 con 0x7f25d80c6170 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 0x7f25d806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 0x7f25d806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46072/0 (socket says 172.21.15.33:46072) 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 -- 172.21.15.33:0/3297658265 learned_addr learned my addr 172.21.15.33:0/3297658265 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dcb9b700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c6170 0x7f25d8147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25d7fff700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d80c82a0 0x7f25d80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 -- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c6170 msgr2=0x7f25d8147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c6170 0x7f25d8147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 -- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d80c82a0 msgr2=0x7f25d80aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d80c82a0 0x7f25d80aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.017+0000 7f25dd39c700 1 -- 172.21.15.33:0/3297658265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0009a20 con 0x7f25d8159520 2023-10-06T05:19:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.018+0000 7f25dd39c700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 0x7f25d806ca70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c0f0 tx=0x7f25d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.018+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000cc30 con 0x7f25d8159520 2023-10-06T05:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.018+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d000f040 con 0x7f25d8159520 2023-10-06T05:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.018+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0014480 con 0x7f25d8159520 2023-10-06T05:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.018+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc009420 con 0x7f25d8159520 2023-10-06T05:19:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.018+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d8159e90 con 0x7f25d8159520 2023-10-06T05:19:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.019+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d804c380 con 0x7f25d8159520 2023-10-06T05:19:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.020+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25d0003020 con 0x7f25d8159520 2023-10-06T05:19:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.021+0000 7f25d4ff9700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25bc066f30 0x7f25bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.021+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f25d00850a0 con 0x7f25d8159520 2023-10-06T05:19:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.021+0000 7f25dcb9b700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25bc066f30 0x7f25bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.022+0000 7f25dcb9b700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25bc066f30 0x7f25bc0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0091e0 tx=0x7f25cc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.025+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25d00520e0 con 0x7f25d8159520 2023-10-06T05:19:50.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.273+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25d80c8ae0 con 0x7f25d8159520 2023-10-06T05:19:50.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.273+0000 7f25d4ff9700 1 -- 172.21.15.33:0/3297658265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f25d0055700 con 0x7f25d8159520 2023-10-06T05:19:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:50.276 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25bc066f30 msgr2=0x7f25bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25bc066f30 0x7f25bc0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0091e0 tx=0x7f25cc017000 comp rx=0 tx=0).stop 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 msgr2=0x7f25d806ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 0x7f25d806ca70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c0f0 tx=0x7f25d000b040 comp rx=0 tx=0).stop 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 shutdown_connections 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25bc066f30 0x7f25bc0693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25d8159520 0x7f25d806ca70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25d80c6170 0x7f25d8147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 --2- 172.21.15.33:0/3297658265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25d80c82a0 0x7f25d80aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 >> 172.21.15.33:0/3297658265 conn(0x7f25d80cc930 msgr2=0x7f25d8157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:50.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 shutdown_connections 2023-10-06T05:19:50.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:50.276+0000 7f25ddb9d700 1 -- 172.21.15.33:0/3297658265 wait complete. 2023-10-06T05:19:50.280 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:50.296 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:51.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.493+0000 7fbd6f59d700 1 Processor -- start 2023-10-06T05:19:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.493+0000 7fbd6f59d700 1 -- start start 2023-10-06T05:19:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd680cb710 0x7fbd6806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd680c9cb0 con 0x7fbd6813faa0 2023-10-06T05:19:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6f59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd680ca270 con 0x7fbd680c8050 2023-10-06T05:19:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6806b840 con 0x7fbd680cb710 2023-10-06T05:19:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbd6813faa0 0x7fbd680c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55878/0 (socket says 172.21.15.33:55878) 2023-10-06T05:19:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6e59b700 1 -- 172.21.15.33:0/1412750928 learned_addr learned my addr 172.21.15.33:0/1412750928 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:51.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6ed9c700 1 --2- 172.21.15.33:0/1412750928 >> v2:172.21.15.167:3300/0 conn(0x7fbd680cb710 0x7fbd6806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250165637 0 0) 0x7fbd680c9cb0 con 0x7fbd6813faa0 2023-10-06T05:19:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.495+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd54003570 con 0x7fbd6813faa0 2023-10-06T05:19:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349926525 0 0) 0x7fbd680ca270 con 0x7fbd680c8050 2023-10-06T05:19:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd680c9cb0 con 0x7fbd680c8050 2023-10-06T05:19:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1259747903 0 0) 0x7fbd54003570 con 0x7fbd6813faa0 2023-10-06T05:19:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 >> v2:172.21.15.167:3300/0 conn(0x7fbd680cb710 msgr2=0x7fbd6806b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 --2- 172.21.15.33:0/1412750928 >> v2:172.21.15.167:3300/0 conn(0x7fbd680cb710 0x7fbd6806b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 >> v1:172.21.15.167:6789/0 conn(0x7fbd680c8050 legacy=0x7fbd6806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6806bb20 con 0x7fbd6813faa0 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fbd64001470 con 0x7fbd6813faa0 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 >> v1:172.21.15.33:6789/0 conn(0x7fbd6813faa0 legacy=0x7fbd680c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd54003c40 0x7fbd54004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd54005210 0x7fbd5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6806b840 con 0x7fbd54004810 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5400c130 con 0x7fbd54003c40 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5400c2c0 con 0x7fbd54005210 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6ed9c700 1 --2- 172.21.15.33:0/1412750928 >> v2:172.21.15.167:3300/0 conn(0x7fbd680cb710 0x7fbd6806b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6e59b700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd54003c40 0x7fbd54004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.496+0000 7fbd6ed9c700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd54005210 0x7fbd5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6dd9a700 1 -- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd54005210 msgr2=0x7fbd5400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd54005210 0x7fbd5400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6dd9a700 1 -- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd54003c40 msgr2=0x7fbd54004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd54003c40 0x7fbd54004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6dd9a700 1 -- 172.21.15.33:0/1412750928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54003570 con 0x7fbd54004810 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6ed9c700 1 -- 172.21.15.33:0/1412750928 reap_dead start 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbd58001a40 tx=0x7fbd58007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90e864b0cee9167 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800c590 con 0x7fbd54004810 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5800c6f0 con 0x7fbd54004810 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.497+0000 7fbd6d599700 1 -- 172.21.15.33:0/1412750928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800cab0 con 0x7fbd54004810 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 -- 172.21.15.33:0/1412750928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 msgr2=0x7fbd54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbd58001a40 tx=0x7fbd58007d30 comp rx=0 tx=0).stop 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 -- 172.21.15.33:0/1412750928 shutdown_connections 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/1412750928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 -- 172.21.15.33:0/1412750928 >> 172.21.15.33:0/1412750928 conn(0x7fbd6806d030 msgr2=0x7fbd6806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 -- 172.21.15.33:0/1412750928 shutdown_connections 2023-10-06T05:19:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.498+0000 7fbd6f59d700 1 -- 172.21.15.33:0/1412750928 wait complete. 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.499+0000 7fbd6f59d700 1 Processor -- start 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.500+0000 7fbd6f59d700 1 -- start start 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.500+0000 7fbd6f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd6813db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.500+0000 7fbd6f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd680a5480 0x7fbd68157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd68158470 0x7fbd680b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680a5de0 con 0x7fbd54004810 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68167720 con 0x7fbd68158470 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6816b960 con 0x7fbd680a5480 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd6813db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd6813db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46084/0 (socket says 172.21.15.33:46084) 2023-10-06T05:19:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 -- 172.21.15.33:0/2909293981 learned_addr learned my addr 172.21.15.33:0/2909293981 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd680a5480 0x7fbd68157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6ed9c700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd68158470 0x7fbd680b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 -- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd680a5480 msgr2=0x7fbd68157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd680a5480 0x7fbd68157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 -- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd68158470 msgr2=0x7fbd680b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd68158470 0x7fbd680b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 -- 172.21.15.33:0/2909293981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd680a6030 con 0x7fbd54004810 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.501+0000 7fbd6e59b700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd6813db70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004fb0 tx=0x7fbd680ca1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.502+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64000be0 con 0x7fbd54004810 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.502+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd64011590 con 0x7fbd54004810 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.502+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64011930 con 0x7fbd54004810 2023-10-06T05:19:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.502+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd680a6310 con 0x7fbd54004810 2023-10-06T05:19:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.502+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6816b5e0 con 0x7fbd54004810 2023-10-06T05:19:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.503+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6804c380 con 0x7fbd54004810 2023-10-06T05:19:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.504+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd64011d20 con 0x7fbd54004810 2023-10-06T05:19:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.505+0000 7fbd5e7fc700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd4c066f30 0x7fbd4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.505+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fbd640848e0 con 0x7fbd54004810 2023-10-06T05:19:51.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.506+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd4c066f30 0x7fbd4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:51.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.506+0000 7fbd6dd9a700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd4c066f30 0x7fbd4c0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbd580019e0 tx=0x7fbd58015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:51.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.510+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd64051920 con 0x7fbd54004810 2023-10-06T05:19:51.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.758+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd680c9f00 con 0x7fbd54004810 2023-10-06T05:19:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.758+0000 7fbd5e7fc700 1 -- 172.21.15.33:0/2909293981 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fbd64054f40 con 0x7fbd54004810 2023-10-06T05:19:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:51.761 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd4c066f30 msgr2=0x7fbd4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd4c066f30 0x7fbd4c0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbd580019e0 tx=0x7fbd58015000 comp rx=0 tx=0).stop 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 msgr2=0x7fbd6813db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd6813db70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004fb0 tx=0x7fbd680ca1e0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 shutdown_connections 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd4c066f30 0x7fbd4c0693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd68158470 0x7fbd680b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd6813db70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 --2- 172.21.15.33:0/2909293981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd680a5480 0x7fbd68157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:51.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 >> 172.21.15.33:0/2909293981 conn(0x7fbd6806d030 msgr2=0x7fbd680c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:51.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 shutdown_connections 2023-10-06T05:19:51.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:51.761+0000 7fbd6f59d700 1 -- 172.21.15.33:0/2909293981 wait complete. 2023-10-06T05:19:51.765 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:51.781 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:52.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.979+0000 7f944b59d700 1 Processor -- start 2023-10-06T05:19:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.979+0000 7f944b59d700 1 -- start start 2023-10-06T05:19:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 0x7f944413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944b59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f944406aad0 0x7f944406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944b59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94440c9cb0 con 0x7f944413faa0 2023-10-06T05:19:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944b59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94440ca270 con 0x7f944406aad0 2023-10-06T05:19:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944406b840 con 0x7f94440c8050 2023-10-06T05:19:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944a59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f944413faa0 0x7f94440c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55888/0 (socket says 172.21.15.33:55888) 2023-10-06T05:19:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944a59b700 1 -- 172.21.15.33:0/2349800280 learned_addr learned my addr 172.21.15.33:0/2349800280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f9449d9a700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 0x7f944413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.981+0000 7f944ad9c700 1 --2- 172.21.15.33:0/2349800280 >> v2:172.21.15.167:3300/0 conn(0x7f944406aad0 0x7f944406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515652123 0 0) 0x7f94440c9cb0 con 0x7f944413faa0 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9430003570 con 0x7f944413faa0 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3745348282 0 0) 0x7f9430003570 con 0x7f944413faa0 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 msgr2=0x7f944413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 0x7f944413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 >> v2:172.21.15.167:3300/0 conn(0x7f944406aad0 msgr2=0x7f944406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 --2- 172.21.15.33:0/2349800280 >> v2:172.21.15.167:3300/0 conn(0x7f944406aad0 0x7f944406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944406bb20 con 0x7f944413faa0 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9440001470 con 0x7f944413faa0 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 >> v1:172.21.15.33:6789/0 conn(0x7f944413faa0 legacy=0x7f94440c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 0x7f9430003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94300040b0 0x7f9430004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9430004d50 0x7f94300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440ca270 con 0x7f9430004d50 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944406b840 con 0x7f94300040b0 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943000bb60 con 0x7f94440c8050 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f944ad9c700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9430004d50 0x7f94300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f9449d9a700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94300040b0 0x7f9430004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.982+0000 7f944a59b700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 0x7f9430003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f944ad9c700 1 -- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 msgr2=0x7f9430003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f944ad9c700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94440c8050 0x7f9430003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f944ad9c700 1 -- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94300040b0 msgr2=0x7f9430004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f944ad9c700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94300040b0 0x7f9430004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f944ad9c700 1 -- 172.21.15.33:0/2349800280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9430003570 con 0x7f9430004d50 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f944ad9c700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9430004d50 0x7f94300051c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f943c00b6e0 tx=0x7f943c00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4560a87984282d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c010cc0 con 0x7f9430004d50 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c011040 con 0x7f9430004d50 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.983+0000 7f9449599700 1 -- 172.21.15.33:0/2349800280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00b0d0 con 0x7f9430004d50 2023-10-06T05:19:52.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944b59d700 1 -- 172.21.15.33:0/2349800280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9430004d50 msgr2=0x7f94300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944b59d700 1 --2- 172.21.15.33:0/2349800280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9430004d50 0x7f94300051c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f943c00b6e0 tx=0x7f943c00d040 comp rx=0 tx=0).stop 2023-10-06T05:19:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944ad9c700 1 -- 172.21.15.33:0/2349800280 reap_dead start 2023-10-06T05:19:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944b59d700 1 -- 172.21.15.33:0/2349800280 shutdown_connections 2023-10-06T05:19:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944b59d700 1 -- 172.21.15.33:0/2349800280 >> 172.21.15.33:0/2349800280 conn(0x7f944406d030 msgr2=0x7f944406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944b59d700 1 -- 172.21.15.33:0/2349800280 shutdown_connections 2023-10-06T05:19:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.984+0000 7f944b59d700 1 -- 172.21.15.33:0/2349800280 wait complete. 2023-10-06T05:19:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.985+0000 7f944b59d700 1 Processor -- start 2023-10-06T05:19:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.985+0000 7f944b59d700 1 -- start start 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.986+0000 7f944b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9430004d50 0x7f94440ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.986+0000 7f944b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9444156060 0x7f94441564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.986+0000 7f944b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 0x7f944413d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.986+0000 7f944b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440ae3a0 con 0x7f944413d460 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.986+0000 7f944b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944413edb0 con 0x7f9430004d50 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.986+0000 7f944b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9444156a10 con 0x7f9444156060 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 0x7f944413d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 0x7f944413d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46092/0 (socket says 172.21.15.33:46092) 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 -- 172.21.15.33:0/803463918 learned_addr learned my addr 172.21.15.33:0/803463918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f9449d9a700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9444156060 0x7f94441564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944a59b700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9430004d50 0x7f94440ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 -- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9444156060 msgr2=0x7f94441564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9444156060 0x7f94441564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 -- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9430004d50 msgr2=0x7f94440ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9430004d50 0x7f94440ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 -- 172.21.15.33:0/803463918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944413e1a0 con 0x7f944413d460 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.987+0000 7f944ad9c700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 0x7f944413d8d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9430004760 tx=0x7f943c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.988+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c010ed0 con 0x7f944413d460 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.988+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c00b0d0 con 0x7f944413d460 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.988+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c0166f0 con 0x7f944413d460 2023-10-06T05:19:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.988+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944413e480 con 0x7f944413d460 2023-10-06T05:19:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.988+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944413ea40 con 0x7f944413d460 2023-10-06T05:19:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.989+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944404c380 con 0x7f944413d460 2023-10-06T05:19:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.990+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f943c016aa0 con 0x7f944413d460 2023-10-06T05:19:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.990+0000 7f943a7fc700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9428066f30 0x7f94280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:52.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.991+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f943c085940 con 0x7f944413d460 2023-10-06T05:19:52.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.991+0000 7f944a59b700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9428066f30 0x7f94280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:52.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.992+0000 7f944a59b700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9428066f30 0x7f94280693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9440001050 tx=0x7f9440009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:52.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:52.995+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f943c052980 con 0x7f944413d460 2023-10-06T05:19:53.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.242+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94440c9f00 con 0x7f944413d460 2023-10-06T05:19:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.242+0000 7f943a7fc700 1 -- 172.21.15.33:0/803463918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f943c025020 con 0x7f944413d460 2023-10-06T05:19:53.315 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:53.315 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9428066f30 msgr2=0x7f94280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9428066f30 0x7f94280693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9440001050 tx=0x7f9440009040 comp rx=0 tx=0).stop 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 msgr2=0x7f944413d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 0x7f944413d8d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9430004760 tx=0x7f943c01b000 comp rx=0 tx=0).stop 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 shutdown_connections 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9428066f30 0x7f94280693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f944413d460 0x7f944413d8d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9430004d50 0x7f94440ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 --2- 172.21.15.33:0/803463918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9444156060 0x7f94441564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 >> 172.21.15.33:0/803463918 conn(0x7f944406d030 msgr2=0x7f94440c27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 shutdown_connections 2023-10-06T05:19:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:53.245+0000 7f944b59d700 1 -- 172.21.15.33:0/803463918 wait complete. 2023-10-06T05:19:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:53.318 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:54.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:54.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.516+0000 7fa28319d700 1 Processor -- start 2023-10-06T05:19:54.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.516+0000 7fa28319d700 1 -- start start 2023-10-06T05:19:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.517+0000 7fa28319d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa27c0c8050 0x7fa27c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.517+0000 7fa28319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c0c9cb0 con 0x7fa27c13faa0 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.517+0000 7fa28319d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c0ca270 con 0x7fa27c06aad0 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.517+0000 7fa28319d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c06b640 con 0x7fa27c0c8050 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.517+0000 7fa28219b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa27c13faa0 0x7fa27c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55904/0 (socket says 172.21.15.33:55904) 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.518+0000 7fa28219b700 1 -- 172.21.15.33:0/4085021197 learned_addr learned my addr 172.21.15.33:0/4085021197 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.518+0000 7fa28199a700 1 --2- 172.21.15.33:0/4085021197 >> v2:172.21.15.33:3300/0 conn(0x7fa27c0c8050 0x7fa27c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.518+0000 7fa28199a700 1 -- 172.21.15.33:0/4085021197 >> v1:172.21.15.167:6789/0 conn(0x7fa27c06aad0 legacy=0x7fa27c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.518+0000 7fa28199a700 1 -- 172.21.15.33:0/4085021197 >> v1:172.21.15.33:6789/0 conn(0x7fa27c13faa0 legacy=0x7fa27c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.518+0000 7fa28199a700 1 -- 172.21.15.33:0/4085021197 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c06b840 con 0x7fa27c0c8050 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.518+0000 7fa28199a700 1 --2- 172.21.15.33:0/4085021197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c0c8050 0x7fa27c13fe70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0098e0 tx=0x7fa26c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81ae27519aab291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa281199700 1 -- 172.21.15.33:0/4085021197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c009e80 con 0x7fa27c0c8050 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa281199700 1 -- 172.21.15.33:0/4085021197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00b860 con 0x7fa27c0c8050 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa281199700 1 -- 172.21.15.33:0/4085021197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00bbe0 con 0x7fa27c0c8050 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 -- 172.21.15.33:0/4085021197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c0c8050 msgr2=0x7fa27c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 --2- 172.21.15.33:0/4085021197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c0c8050 0x7fa27c13fe70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0098e0 tx=0x7fa26c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 -- 172.21.15.33:0/4085021197 shutdown_connections 2023-10-06T05:19:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 --2- 172.21.15.33:0/4085021197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c0c8050 0x7fa27c13fe70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 -- 172.21.15.33:0/4085021197 >> 172.21.15.33:0/4085021197 conn(0x7fa27c06d030 msgr2=0x7fa27c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 -- 172.21.15.33:0/4085021197 shutdown_connections 2023-10-06T05:19:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.519+0000 7fa28319d700 1 -- 172.21.15.33:0/4085021197 wait complete. 2023-10-06T05:19:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.521+0000 7fa28319d700 1 Processor -- start 2023-10-06T05:19:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.521+0000 7fa28319d700 1 -- start start 2023-10-06T05:19:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 0x7fa27c0c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27c0c8050 0x7fa27c13ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa27c13faa0 0x7fa27c13cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c13d4a0 con 0x7fa27c06aad0 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c13e2a0 con 0x7fa27c0c8050 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c13a880 con 0x7fa27c13faa0 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28219b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 0x7fa27c0c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28219b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 0x7fa27c0c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46108/0 (socket says 172.21.15.33:46108) 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.522+0000 7fa28219b700 1 -- 172.21.15.33:0/938668045 learned_addr learned my addr 172.21.15.33:0/938668045 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28299c700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa27c13faa0 0x7fa27c13cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28199a700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27c0c8050 0x7fa27c13ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28219b700 1 -- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa27c13faa0 msgr2=0x7fa27c13cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28219b700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa27c13faa0 0x7fa27c13cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28219b700 1 -- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27c0c8050 msgr2=0x7fa27c13ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28219b700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27c0c8050 0x7fa27c13ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28219b700 1 -- 172.21.15.33:0/938668045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009420 con 0x7fa27c06aad0 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28299c700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa27c13faa0 0x7fa27c13cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28219b700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 0x7fa27c0c38f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c560 tx=0x7fa278001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278005b30 con 0x7fa27c06aad0 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa278005cb0 con 0x7fa27c06aad0 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278007320 con 0x7fa27c06aad0 2023-10-06T05:19:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c13d690 con 0x7fa27c06aad0 2023-10-06T05:19:54.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.523+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c13dcd0 con 0x7fa27c06aad0 2023-10-06T05:19:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.524+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c13e050 con 0x7fa27c06aad0 2023-10-06T05:19:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.525+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa278007690 con 0x7fa27c06aad0 2023-10-06T05:19:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.526+0000 7fa2727fc700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa260066f30 0x7fa2600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.526+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa278083c60 con 0x7fa27c06aad0 2023-10-06T05:19:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.527+0000 7fa28199a700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa260066f30 0x7fa2600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.527+0000 7fa28199a700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa260066f30 0x7fa2600693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0093f0 tx=0x7fa26c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.531+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa278050ca0 con 0x7fa27c06aad0 2023-10-06T05:19:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.776+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa27c04c380 con 0x7fa27c06aad0 2023-10-06T05:19:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.777+0000 7fa2727fc700 1 -- 172.21.15.33:0/938668045 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fa278017020 con 0x7fa27c06aad0 2023-10-06T05:19:54.780 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:54.780 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:54.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa260066f30 msgr2=0x7fa2600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa260066f30 0x7fa2600693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0093f0 tx=0x7fa26c017000 comp rx=0 tx=0).stop 2023-10-06T05:19:54.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 msgr2=0x7fa27c0c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:54.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 0x7fa27c0c38f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c560 tx=0x7fa278001020 comp rx=0 tx=0).stop 2023-10-06T05:19:54.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 shutdown_connections 2023-10-06T05:19:54.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa260066f30 0x7fa2600693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa27c13faa0 0x7fa27c13cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa27c06aad0 0x7fa27c0c38f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 --2- 172.21.15.33:0/938668045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27c0c8050 0x7fa27c13ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 >> 172.21.15.33:0/938668045 conn(0x7fa27c06d030 msgr2=0x7fa27c0c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 shutdown_connections 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:54.779+0000 7fa28319d700 1 -- 172.21.15.33:0/938668045 wait complete. 2023-10-06T05:19:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:54.800 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:55.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.991+0000 7f8b91b9d700 1 Processor -- start 2023-10-06T05:19:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.992+0000 7f8b91b9d700 1 -- start start 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b91b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b8c0c8050 0x7f8b8c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b91b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0c9cb0 con 0x7f8b8c06aad0 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b91b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0ca270 con 0x7f8b8c13faa0 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b91b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c06b640 con 0x7f8b8c0c8050 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b8bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b8c0c8050 0x7f8b8c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b9139c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b8c06aad0 0x7f8b8c06af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55918/0 (socket says 172.21.15.33:55918) 2023-10-06T05:19:55.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.993+0000 7f8b9139c700 1 -- 172.21.15.33:0/2373984671 learned_addr learned my addr 172.21.15.33:0/2373984671 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:19:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8bfff700 1 -- 172.21.15.33:0/2373984671 >> v1:172.21.15.167:6789/0 conn(0x7f8b8c13faa0 legacy=0x7f8b8c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8bfff700 1 -- 172.21.15.33:0/2373984671 >> v1:172.21.15.33:6789/0 conn(0x7f8b8c06aad0 legacy=0x7f8b8c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8bfff700 1 -- 172.21.15.33:0/2373984671 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c06b840 con 0x7f8b8c0c8050 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8bfff700 1 --2- 172.21.15.33:0/2373984671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c0c8050 0x7f8b8c13fe70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8b780098e0 tx=0x7f8b780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96268b7520bab6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8b7fe700 1 -- 172.21.15.33:0/2373984671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78009e80 con 0x7f8b8c0c8050 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8b7fe700 1 -- 172.21.15.33:0/2373984671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7800b820 con 0x7f8b8c0c8050 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.994+0000 7f8b8b7fe700 1 -- 172.21.15.33:0/2373984671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800bba0 con 0x7f8b8c0c8050 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 -- 172.21.15.33:0/2373984671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c0c8050 msgr2=0x7f8b8c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/2373984671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c0c8050 0x7f8b8c13fe70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8b780098e0 tx=0x7f8b780091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 -- 172.21.15.33:0/2373984671 shutdown_connections 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/2373984671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c0c8050 0x7f8b8c13fe70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 -- 172.21.15.33:0/2373984671 >> 172.21.15.33:0/2373984671 conn(0x7f8b8c06d030 msgr2=0x7f8b8c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 -- 172.21.15.33:0/2373984671 shutdown_connections 2023-10-06T05:19:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.995+0000 7f8b91b9d700 1 -- 172.21.15.33:0/2373984671 wait complete. 2023-10-06T05:19:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.996+0000 7f8b91b9d700 1 Processor -- start 2023-10-06T05:19:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 -- start start 2023-10-06T05:19:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 0x7f8b8c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b8c0c8050 0x7f8b8c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b8c13faa0 0x7f8b8c16b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c157c30 con 0x7f8b8c06aad0 2023-10-06T05:19:56.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c157ed0 con 0x7f8b8c13faa0 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.997+0000 7f8b91b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c1581b0 con 0x7f8b8c0c8050 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 0x7f8b8c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 0x7f8b8c156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46122/0 (socket says 172.21.15.33:46122) 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 -- 172.21.15.33:0/87185410 learned_addr learned my addr 172.21.15.33:0/87185410 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b9139c700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b8c13faa0 0x7f8b8c16b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b8bfff700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b8c0c8050 0x7f8b8c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 -- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b8c0c8050 msgr2=0x7f8b8c158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b8c0c8050 0x7f8b8c158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 -- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b8c13faa0 msgr2=0x7f8b8c16b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b8c13faa0 0x7f8b8c16b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:56.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 -- 172.21.15.33:0/87185410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b78009420 con 0x7f8b8c06aad0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b90b9b700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 0x7f8b8c156210 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000b5e0 tx=0x7f8b80009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.998+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000f510 con 0x7f8b8c06aad0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.999+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8000f670 con 0x7f8b8c06aad0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.999+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c158490 con 0x7f8b8c06aad0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.999+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000fa30 con 0x7f8b8c06aad0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:55.999+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c156750 con 0x7f8b8c06aad0 2023-10-06T05:19:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.000+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c04c380 con 0x7f8b8c06aad0 2023-10-06T05:19:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.001+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b8001a400 con 0x7f8b8c06aad0 2023-10-06T05:19:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.002+0000 7f8b88ff9700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b70066f30 0x7f8b700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.002+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8b80084310 con 0x7f8b8c06aad0 2023-10-06T05:19:56.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.002+0000 7f8b8bfff700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b70066f30 0x7f8b700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.003+0000 7f8b8bfff700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b70066f30 0x7f8b700693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b780030c0 tx=0x7f8b78016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:56.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.006+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b80051350 con 0x7f8b8c06aad0 2023-10-06T05:19:56.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.253+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b8c06bb20 con 0x7f8b8c06aad0 2023-10-06T05:19:56.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.253+0000 7f8b88ff9700 1 -- 172.21.15.33:0/87185410 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f8b80054970 con 0x7f8b8c06aad0 2023-10-06T05:19:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:56.256 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b70066f30 msgr2=0x7f8b700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b70066f30 0x7f8b700693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b780030c0 tx=0x7f8b78016000 comp rx=0 tx=0).stop 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 msgr2=0x7f8b8c156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 0x7f8b8c156210 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000b5e0 tx=0x7f8b80009120 comp rx=0 tx=0).stop 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 shutdown_connections 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b70066f30 0x7f8b700693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b8c13faa0 0x7f8b8c16b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b8c06aad0 0x7f8b8c156210 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 --2- 172.21.15.33:0/87185410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b8c0c8050 0x7f8b8c158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 >> 172.21.15.33:0/87185410 conn(0x7f8b8c06d030 msgr2=0x7f8b8c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 shutdown_connections 2023-10-06T05:19:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:56.256+0000 7f8b91b9d700 1 -- 172.21.15.33:0/87185410 wait complete. 2023-10-06T05:19:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:56.276 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:57.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:57.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.476+0000 7f396159d700 1 Processor -- start 2023-10-06T05:19:57.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.476+0000 7f396159d700 1 -- start start 2023-10-06T05:19:57.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f396159d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f395c155140 0x7f395c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f396159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0adf50 0x7f395c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f396159d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f395c0c9810 0x7f395c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f396159d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395c0c8e40 con 0x7f395c0c9810 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f396159d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395c0ae6c0 con 0x7f395c155140 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f396159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c0ae9a0 con 0x7f395c0adf50 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f3960d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f395c0c9810 0x7f395c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f3960d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f395c0c9810 0x7f395c0699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46126/0 (socket says 172.21.15.33:46126) 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f3960d9c700 1 -- 172.21.15.33:0/2138683888 learned_addr learned my addr 172.21.15.33:0/2138683888 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f395b7fe700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0adf50 0x7f395c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.478+0000 7f395bfff700 1 --2- 172.21.15.33:0/2138683888 >> v2:172.21.15.167:3300/0 conn(0x7f395c155140 0x7f395c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f3960d9c700 1 -- 172.21.15.33:0/2138683888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0adf50 msgr2=0x7f395c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f3960d9c700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0adf50 0x7f395c0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f3960d9c700 1 -- 172.21.15.33:0/2138683888 >> v2:172.21.15.167:3300/0 conn(0x7f395c155140 msgr2=0x7f395c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f3960d9c700 1 --2- 172.21.15.33:0/2138683888 >> v2:172.21.15.167:3300/0 conn(0x7f395c155140 0x7f395c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f3960d9c700 1 -- 172.21.15.33:0/2138683888 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c0c8ad0 con 0x7f395c0c9810 2023-10-06T05:19:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f395b7fe700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0adf50 0x7f395c0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f3960d9c700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0c9810 0x7f395c0699c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f395c071970 tx=0x7f3954009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db07f8a264a86b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f395affd700 1 -- 172.21.15.33:0/2138683888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395400e070 con 0x7f395c0c9810 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f395affd700 1 -- 172.21.15.33:0/2138683888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395400ba20 con 0x7f395c0c9810 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.479+0000 7f395affd700 1 -- 172.21.15.33:0/2138683888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39540092e0 con 0x7f395c0c9810 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 -- 172.21.15.33:0/2138683888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0c9810 msgr2=0x7f395c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0c9810 0x7f395c0699c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f395c071970 tx=0x7f3954009840 comp rx=0 tx=0).stop 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 -- 172.21.15.33:0/2138683888 shutdown_connections 2023-10-06T05:19:57.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0c9810 0x7f395c0699c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 --2- 172.21.15.33:0/2138683888 >> v2:172.21.15.167:3300/0 conn(0x7f395c155140 0x7f395c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 --2- 172.21.15.33:0/2138683888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0adf50 0x7f395c0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 -- 172.21.15.33:0/2138683888 >> 172.21.15.33:0/2138683888 conn(0x7f395c06d030 msgr2=0x7f395c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 -- 172.21.15.33:0/2138683888 shutdown_connections 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.480+0000 7f396159d700 1 -- 172.21.15.33:0/2138683888 wait complete. 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.481+0000 7f396159d700 1 Processor -- start 2023-10-06T05:19:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.481+0000 7f396159d700 1 -- start start 2023-10-06T05:19:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.482+0000 7f396159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 0x7f395c0b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.482+0000 7f396159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0c9810 0x7f395c0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.482+0000 7f396159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f395c155140 0x7f395c140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.482+0000 7f396159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c0c8040 con 0x7f395c0adf50 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.482+0000 7f396159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c0c7d60 con 0x7f395c155140 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.482+0000 7f396159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c151510 con 0x7f395c0c9810 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 0x7f395c0b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 0x7f395c0b0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46142/0 (socket says 172.21.15.33:46142) 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 -- 172.21.15.33:0/1475518518 learned_addr learned my addr 172.21.15.33:0/1475518518 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395b7fe700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0c9810 0x7f395c0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f3960d9c700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f395c155140 0x7f395c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 -- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0c9810 msgr2=0x7f395c0c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0c9810 0x7f395c0c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 -- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f395c155140 msgr2=0x7f395c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f395c155140 0x7f395c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 -- 172.21.15.33:0/1475518518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954009ae0 con 0x7f395c0adf50 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f3960d9c700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f395c155140 0x7f395c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.483+0000 7f395bfff700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 0x7f395c0b0a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e930 tx=0x7f394c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.484+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c013680 con 0x7f395c0adf50 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.484+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c0137e0 con 0x7f395c0adf50 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.484+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c013b80 con 0x7f395c0adf50 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.484+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c1517f0 con 0x7f395c0adf50 2023-10-06T05:19:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.484+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c0c3c40 con 0x7f395c0adf50 2023-10-06T05:19:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.485+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c151e40 con 0x7f395c0adf50 2023-10-06T05:19:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.486+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f394c00c460 con 0x7f395c0adf50 2023-10-06T05:19:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.487+0000 7f3943fff700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3948066f30 0x7f39480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.487+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f394c084890 con 0x7f395c0adf50 2023-10-06T05:19:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.487+0000 7f395b7fe700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3948066f30 0x7f39480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.488+0000 7f395b7fe700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3948066f30 0x7f39480693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f395c0c2ab0 tx=0x7f3950008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:57.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.491+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f394c051950 con 0x7f395c0adf50 2023-10-06T05:19:57.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.741+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f395c04c380 con 0x7f395c0adf50 2023-10-06T05:19:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.741+0000 7f3943fff700 1 -- 172.21.15.33:0/1475518518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f394c054f70 con 0x7f395c0adf50 2023-10-06T05:19:57.744 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:57.744 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3948066f30 msgr2=0x7f39480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3948066f30 0x7f39480693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f395c0c2ab0 tx=0x7f3950008040 comp rx=0 tx=0).stop 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 msgr2=0x7f395c0b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 0x7f395c0b0a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e930 tx=0x7f394c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 shutdown_connections 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3948066f30 0x7f39480693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f395c155140 0x7f395c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c0adf50 0x7f395c0b0a00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 --2- 172.21.15.33:0/1475518518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f395c0c9810 0x7f395c0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 >> 172.21.15.33:0/1475518518 conn(0x7f395c06d030 msgr2=0x7f395c0c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 shutdown_connections 2023-10-06T05:19:57.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:57.744+0000 7f396159d700 1 -- 172.21.15.33:0/1475518518 wait complete. 2023-10-06T05:19:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:57.764 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:19:58.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:58.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.963+0000 7ffbc5d9d700 1 Processor -- start 2023-10-06T05:19:58.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.963+0000 7ffbc5d9d700 1 -- start start 2023-10-06T05:19:58.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.964+0000 7ffbc5d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffbc00ccfc0 0x7ffbc00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:58.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.964+0000 7ffbc5d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc00cbd80 con 0x7ffbc00cf3b0 2023-10-06T05:19:58.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc5d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc00af5f0 con 0x7ffbc00cc750 2023-10-06T05:19:58.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc5d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc00af8d0 con 0x7ffbc00ccfc0 2023-10-06T05:19:58.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc4d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffbc00ccfc0 0x7ffbc00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:58.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc4d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffbc00ccfc0 0x7ffbc00cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46156/0 (socket says 172.21.15.33:46156) 2023-10-06T05:19:58.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/136734708 learned_addr learned my addr 172.21.15.33:0/136734708 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:58.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbbffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffbc00cf3b0 0x7ffbc01594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55934/0 (socket says 172.21.15.33:55934) 2023-10-06T05:19:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/136734708 >> v1:172.21.15.167:6789/0 conn(0x7ffbc00cc750 legacy=0x7ffbc00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/136734708 >> v1:172.21.15.33:6789/0 conn(0x7ffbc00cf3b0 legacy=0x7ffbc01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/136734708 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00afbb0 con 0x7ffbc00ccfc0 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.965+0000 7ffbbf7fe700 1 -- 172.21.15.33:0/136734708 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899851621 0 0) 0x7ffbc00cbd80 con 0x7ffbc00cf3b0 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbc4d9b700 1 --2- 172.21.15.33:0/136734708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00ccfc0 0x7ffbc00cc210 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40099e0 tx=0x7ffbb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5cbeed268431fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbbf7fe700 1 -- 172.21.15.33:0/136734708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400b820 con 0x7ffbc00ccfc0 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbbf7fe700 1 -- 172.21.15.33:0/136734708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb400b980 con 0x7ffbc00ccfc0 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbbf7fe700 1 -- 172.21.15.33:0/136734708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400bd20 con 0x7ffbc00ccfc0 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/136734708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00ccfc0 msgr2=0x7ffbc00cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/136734708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00ccfc0 0x7ffbc00cc210 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40099e0 tx=0x7ffbb4009250 comp rx=0 tx=0).stop 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/136734708 shutdown_connections 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/136734708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00ccfc0 0x7ffbc00cc210 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.966+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/136734708 >> 172.21.15.33:0/136734708 conn(0x7ffbc006cff0 msgr2=0x7ffbc006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:58.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.967+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/136734708 shutdown_connections 2023-10-06T05:19:58.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.967+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/136734708 wait complete. 2023-10-06T05:19:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.968+0000 7ffbc5d9d700 1 Processor -- start 2023-10-06T05:19:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.968+0000 7ffbc5d9d700 1 -- start start 2023-10-06T05:19:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc5d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 0x7ffbc016df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbc00ccfc0 0x7ffbc0169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc5d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc00c5d40 0x7ffbc00c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc5d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00c66f0 con 0x7ffbc00cc750 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc5d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00b3350 con 0x7ffbc00c5d40 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc016e490 con 0x7ffbc00ccfc0 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc4d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 0x7ffbc016df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc4d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 0x7ffbc016df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46168/0 (socket says 172.21.15.33:46168) 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.969+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/3253672398 learned_addr learned my addr 172.21.15.33:0/3253672398 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbc00ccfc0 msgr2=0x7ffbc0169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc4d9b700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbc00ccfc0 0x7ffbc0169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc00c5d40 msgr2=0x7ffbc00c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc4d9b700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc00c5d40 0x7ffbc00c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc4d9b700 1 -- 172.21.15.33:0/3253672398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40094f0 con 0x7ffbc00cc750 2023-10-06T05:19:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc4d9b700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 0x7ffbc016df50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40094c0 tx=0x7ffbb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb401f070 con 0x7ffbc00cc750 2023-10-06T05:19:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb400bbc0 con 0x7ffbc00cc750 2023-10-06T05:19:58.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4025670 con 0x7ffbc00cc750 2023-10-06T05:19:58.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc00b35a0 con 0x7ffbc00cc750 2023-10-06T05:19:58.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.970+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc00b3b60 con 0x7ffbc00cc750 2023-10-06T05:19:58.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.971+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc004c380 con 0x7ffbc00cc750 2023-10-06T05:19:58.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.972+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbb4023070 con 0x7ffbc00cc750 2023-10-06T05:19:58.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.973+0000 7ffbbcff9700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:58.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.973+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ffbb4085c30 con 0x7ffbc00cc750 2023-10-06T05:19:58.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.974+0000 7ffbbffff700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:58.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.974+0000 7ffbbffff700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffbac000c00 tx=0x7ffbac007ac0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:58.977+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbb4052d20 con 0x7ffbc00cc750 2023-10-06T05:19:59.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.224+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc00b62d0 con 0x7ffbc00cc750 2023-10-06T05:19:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.224+0000 7ffbbcff9700 1 -- 172.21.15.33:0/3253672398 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ffbb4015090 con 0x7ffbc00cc750 2023-10-06T05:19:59.228 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:19:59.228 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:19:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffba4066f30 msgr2=0x7ffba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffbac000c00 tx=0x7ffbac007ac0 comp rx=0 tx=0).stop 2023-10-06T05:19:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 msgr2=0x7ffbc016df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 0x7ffbc016df50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40094c0 tx=0x7ffbb4017000 comp rx=0 tx=0).stop 2023-10-06T05:19:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 shutdown_connections 2023-10-06T05:19:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbc00c5d40 0x7ffbc00c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbc00cc750 0x7ffbc016df50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 --2- 172.21.15.33:0/3253672398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbc00ccfc0 0x7ffbc0169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 >> 172.21.15.33:0/3253672398 conn(0x7ffbc006cff0 msgr2=0x7ffbc014efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 shutdown_connections 2023-10-06T05:19:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:19:59.227+0000 7ffbc5d9d700 1 -- 172.21.15.33:0/3253672398 wait complete. 2023-10-06T05:19:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:19:59.247 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:00.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.446+0000 7fda0699d700 1 Processor -- start 2023-10-06T05:20:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.446+0000 7fda0699d700 1 -- start start 2023-10-06T05:20:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0699d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fda000c8b50 0x7fda000c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fda00155ec0 0x7fda000c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0699d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0006ab60 con 0x7fda00155af0 2023-10-06T05:20:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0699d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0006b7e0 con 0x7fda000c8b50 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0006ae40 con 0x7fda00155ec0 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0599b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fda00155af0 0x7fda0006a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58176/0 (socket says 172.21.15.33:58176) 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0599b700 1 -- 172.21.15.33:0/396909449 learned_addr learned my addr 172.21.15.33:0/396909449 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0519a700 1 --2- 172.21.15.33:0/396909449 >> v2:172.21.15.33:3300/0 conn(0x7fda000c8b50 0x7fda000c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda0619c700 1 --2- 172.21.15.33:0/396909449 >> v2:172.21.15.167:3300/0 conn(0x7fda00155ec0 0x7fda000c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda04999700 1 -- 172.21.15.33:0/396909449 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943403494 0 0) 0x7fda0006ab60 con 0x7fda00155af0 2023-10-06T05:20:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.448+0000 7fda04999700 1 -- 172.21.15.33:0/396909449 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ec003570 con 0x7fda00155af0 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda0519a700 1 -- 172.21.15.33:0/396909449 >> v2:172.21.15.167:3300/0 conn(0x7fda00155ec0 msgr2=0x7fda000c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda0519a700 1 --2- 172.21.15.33:0/396909449 >> v2:172.21.15.167:3300/0 conn(0x7fda00155ec0 0x7fda000c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda0519a700 1 -- 172.21.15.33:0/396909449 >> v1:172.21.15.33:6789/0 conn(0x7fda00155af0 legacy=0x7fda0006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda0519a700 1 -- 172.21.15.33:0/396909449 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0006b120 con 0x7fda000c8b50 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda0519a700 1 --2- 172.21.15.33:0/396909449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda000c8b50 0x7fda000c7210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fda0006fdf0 tx=0x7fd9f00080e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a989272930b207f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda04999700 1 -- 172.21.15.33:0/396909449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000a910 con 0x7fda000c8b50 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda04999700 1 -- 172.21.15.33:0/396909449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f000aa70 con 0x7fda000c8b50 2023-10-06T05:20:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.449+0000 7fda04999700 1 -- 172.21.15.33:0/396909449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000ae10 con 0x7fda000c8b50 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 -- 172.21.15.33:0/396909449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda000c8b50 msgr2=0x7fda000c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 --2- 172.21.15.33:0/396909449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda000c8b50 0x7fda000c7210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fda0006fdf0 tx=0x7fd9f00080e0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 -- 172.21.15.33:0/396909449 shutdown_connections 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 --2- 172.21.15.33:0/396909449 >> v2:172.21.15.167:3300/0 conn(0x7fda00155ec0 0x7fda000c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 --2- 172.21.15.33:0/396909449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda000c8b50 0x7fda000c7210 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 -- 172.21.15.33:0/396909449 >> 172.21.15.33:0/396909449 conn(0x7fda0006d030 msgr2=0x7fda0006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 -- 172.21.15.33:0/396909449 shutdown_connections 2023-10-06T05:20:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.450+0000 7fda0699d700 1 -- 172.21.15.33:0/396909449 wait complete. 2023-10-06T05:20:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.451+0000 7fda0699d700 1 Processor -- start 2023-10-06T05:20:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.451+0000 7fda0699d700 1 -- start start 2023-10-06T05:20:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.452+0000 7fda0699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda000c8b50 0x7fda00148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.452+0000 7fda0699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 0x7fda000ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.452+0000 7fda0699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda00155ec0 0x7fda00153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.452+0000 7fda0699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0016a510 con 0x7fda00155af0 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.452+0000 7fda0699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001491f0 con 0x7fda00155ec0 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.452+0000 7fda0699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda00169900 con 0x7fda000c8b50 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 0x7fda000ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 0x7fda000ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39840/0 (socket says 172.21.15.33:39840) 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 -- 172.21.15.33:0/726478228 learned_addr learned my addr 172.21.15.33:0/726478228 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0599b700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda000c8b50 0x7fda00148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0619c700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda00155ec0 0x7fda00153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 -- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda000c8b50 msgr2=0x7fda00148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda000c8b50 0x7fda00148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 -- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda00155ec0 msgr2=0x7fda00153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda00155ec0 0x7fda00153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 -- 172.21.15.33:0/726478228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f000d510 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.453+0000 7fda0519a700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 0x7fda000ca150 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fda000635d0 tx=0x7fd9f0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.454+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000abe0 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.454+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f000ad40 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.454+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda00169be0 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.454+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f001e680 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.454+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0016a1a0 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.455+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0016a360 con 0x7fda00155af0 2023-10-06T05:20:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.456+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9f001ea70 con 0x7fda00155af0 2023-10-06T05:20:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.456+0000 7fd9f5ffb700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9e4066f30 0x7fd9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.457+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd9f0085390 con 0x7fda00155af0 2023-10-06T05:20:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.457+0000 7fda0599b700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9e4066f30 0x7fd9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.458+0000 7fda0599b700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9e4066f30 0x7fd9e40693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0058d0 tx=0x7fd9fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.461+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f0052450 con 0x7fda00155af0 2023-10-06T05:20:00.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.707+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda0004c380 con 0x7fda00155af0 2023-10-06T05:20:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.707+0000 7fd9f5ffb700 1 -- 172.21.15.33:0/726478228 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd9f0055a70 con 0x7fda00155af0 2023-10-06T05:20:00.710 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:00.710 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:00.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.709+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9e4066f30 msgr2=0x7fd9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:00.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.709+0000 7fda0699d700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9e4066f30 0x7fd9e40693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0058d0 tx=0x7fd9fc009040 comp rx=0 tx=0).stop 2023-10-06T05:20:00.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.709+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 msgr2=0x7fda000ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:00.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.709+0000 7fda0699d700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 0x7fda000ca150 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fda000635d0 tx=0x7fd9f0019000 comp rx=0 tx=0).stop 2023-10-06T05:20:00.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 shutdown_connections 2023-10-06T05:20:00.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9e4066f30 0x7fd9e40693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda00155ec0 0x7fda00153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda000c8b50 0x7fda00148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 --2- 172.21.15.33:0/726478228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda00155af0 0x7fda000ca150 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:00.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 >> 172.21.15.33:0/726478228 conn(0x7fda0006d030 msgr2=0x7fda00154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:00.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 shutdown_connections 2023-10-06T05:20:00.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:00.710+0000 7fda0699d700 1 -- 172.21.15.33:0/726478228 wait complete. 2023-10-06T05:20:00.715 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:00.730 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:01.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.943+0000 7fab6359e700 1 Processor -- start 2023-10-06T05:20:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.943+0000 7fab6359e700 1 -- start start 2023-10-06T05:20:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab6359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab640cb710 0x7fab6406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab6359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab640c9cb0 con 0x7fab640c8050 2023-10-06T05:20:01.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab6359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab640ca270 con 0x7fab6413faa0 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab6359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6406b840 con 0x7fab640cb710 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab61d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab640c8050 0x7fab6406aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58192/0 (socket says 172.21.15.33:58192) 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab61d9b700 1 -- 172.21.15.33:0/127496834 learned_addr learned my addr 172.21.15.33:0/127496834 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab62d9d700 1 --2- 172.21.15.33:0/127496834 >> v2:172.21.15.33:3300/0 conn(0x7fab640cb710 0x7fab6406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab6159a700 1 -- 172.21.15.33:0/127496834 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728902915 0 0) 0x7fab640c9cb0 con 0x7fab640c8050 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab6159a700 1 -- 172.21.15.33:0/127496834 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4c003570 con 0x7fab640c8050 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab62d9d700 1 -- 172.21.15.33:0/127496834 >> v1:172.21.15.167:6789/0 conn(0x7fab6413faa0 legacy=0x7fab640c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab62d9d700 1 -- 172.21.15.33:0/127496834 >> v1:172.21.15.33:6789/0 conn(0x7fab640c8050 legacy=0x7fab6406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.945+0000 7fab62d9d700 1 -- 172.21.15.33:0/127496834 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6406bb20 con 0x7fab640cb710 2023-10-06T05:20:01.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.946+0000 7fab62d9d700 1 --2- 172.21.15.33:0/127496834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab6406b1e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009e60 tx=0x7fab5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca53ff4677ec3de8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.946+0000 7fab6159a700 1 -- 172.21.15.33:0/127496834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00e070 con 0x7fab640cb710 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.946+0000 7fab6159a700 1 -- 172.21.15.33:0/127496834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00bb60 con 0x7fab640cb710 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.946+0000 7fab6159a700 1 -- 172.21.15.33:0/127496834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c012430 con 0x7fab640cb710 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.946+0000 7fab6359e700 1 -- 172.21.15.33:0/127496834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 msgr2=0x7fab6406b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.946+0000 7fab6359e700 1 --2- 172.21.15.33:0/127496834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab6406b1e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009e60 tx=0x7fab5c009780 comp rx=0 tx=0).stop 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.947+0000 7fab6359e700 1 -- 172.21.15.33:0/127496834 shutdown_connections 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.947+0000 7fab6359e700 1 --2- 172.21.15.33:0/127496834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab6406b1e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.947+0000 7fab6359e700 1 -- 172.21.15.33:0/127496834 >> 172.21.15.33:0/127496834 conn(0x7fab6406d030 msgr2=0x7fab6406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.947+0000 7fab6359e700 1 -- 172.21.15.33:0/127496834 shutdown_connections 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.947+0000 7fab6359e700 1 -- 172.21.15.33:0/127496834 wait complete. 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.948+0000 7fab6359e700 1 Processor -- start 2023-10-06T05:20:01.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.948+0000 7fab6359e700 1 -- start start 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.949+0000 7fab6359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab640c8050 0x7fab641599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.949+0000 7fab6359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab640c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.949+0000 7fab6359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6413faa0 0x7fab640b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.949+0000 7fab6359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6413cda0 con 0x7fab640cb710 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.949+0000 7fab6359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab640c4fa0 con 0x7fab640c8050 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.949+0000 7fab6359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6413c810 con 0x7fab6413faa0 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab640c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab640c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39848/0 (socket says 172.21.15.33:39848) 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 -- 172.21.15.33:0/1240840416 learned_addr learned my addr 172.21.15.33:0/1240840416 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:01.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab6259c700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab640c8050 0x7fab641599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab62d9d700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6413faa0 0x7fab640b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 -- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6413faa0 msgr2=0x7fab640b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6413faa0 0x7fab640b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 -- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab640c8050 msgr2=0x7fab641599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab640c8050 0x7fab641599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 -- 172.21.15.33:0/1240840416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c009a20 con 0x7fab640cb710 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.950+0000 7fab61d9b700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab640c4a60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c7c0 tx=0x7fab58007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.951+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58005b30 con 0x7fab640cb710 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.951+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58005cb0 con 0x7fab640cb710 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.951+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab64158df0 con 0x7fab640cb710 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.951+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58007320 con 0x7fab640cb710 2023-10-06T05:20:01.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.951+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab64156ee0 con 0x7fab640cb710 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.952+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab640698c0 con 0x7fab640cb710 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.953+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab5801b3b0 con 0x7fab640cb710 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.954+0000 7fab527fc700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.954+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fab58083bd0 con 0x7fab640cb710 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.954+0000 7fab6259c700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.955+0000 7fab6259c700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab44066f30 0x7fab440693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fab640c7340 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:01.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:01.957+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab58050c90 con 0x7fab640cb710 2023-10-06T05:20:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.205+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6404c380 con 0x7fab640cb710 2023-10-06T05:20:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.205+0000 7fab527fc700 1 -- 172.21.15.33:0/1240840416 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fab58017020 con 0x7fab640cb710 2023-10-06T05:20:02.208 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:02.208 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab44066f30 msgr2=0x7fab440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab44066f30 0x7fab440693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fab640c7340 tx=0x7fab54009040 comp rx=0 tx=0).stop 2023-10-06T05:20:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 msgr2=0x7fab640c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab640c4a60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c7c0 tx=0x7fab58007e00 comp rx=0 tx=0).stop 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 shutdown_connections 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6413faa0 0x7fab640b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab640c8050 0x7fab641599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 --2- 172.21.15.33:0/1240840416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640cb710 0x7fab640c4a60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.207+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 >> 172.21.15.33:0/1240840416 conn(0x7fab6406d030 msgr2=0x7fab6413e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.208+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 shutdown_connections 2023-10-06T05:20:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:02.208+0000 7fab6359e700 1 -- 172.21.15.33:0/1240840416 wait complete. 2023-10-06T05:20:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:02.228 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:03.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.425+0000 7f6d7b59e700 1 Processor -- start 2023-10-06T05:20:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.425+0000 7f6d7b59e700 1 -- start start 2023-10-06T05:20:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.426+0000 7f6d7b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.426+0000 7f6d7b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d7c0c82a0 0x7f6d7c06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c0c65e0 con 0x7f6d7c0c6170 2023-10-06T05:20:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c06c820 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c06a160 con 0x7f6d7c159520 2023-10-06T05:20:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d7c0c82a0 0x7f6d7c06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d7c0c82a0 0x7f6d7c06a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39858/0 (socket says 172.21.15.33:39858) 2023-10-06T05:20:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 -- 172.21.15.33:0/855516581 learned_addr learned my addr 172.21.15.33:0/855516581 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7a59c700 1 --2- 172.21.15.33:0/855516581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 -- 172.21.15.33:0/855516581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 msgr2=0x7f6d7c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 --2- 172.21.15.33:0/855516581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 -- 172.21.15.33:0/855516581 >> v1:172.21.15.33:6789/0 conn(0x7f6d7c0c6170 legacy=0x7f6d7c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7ad9d700 1 -- 172.21.15.33:0/855516581 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0c8800 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.427+0000 7f6d7959a700 1 -- 172.21.15.33:0/855516581 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45671033 0 0) 0x7f6d7c0c65e0 con 0x7f6d7c0c6170 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7ad9d700 1 --2- 172.21.15.33:0/855516581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c06a660 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009f20 tx=0x7f6d74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9908f03ccf871127 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7959a700 1 -- 172.21.15.33:0/855516581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400e070 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7959a700 1 -- 172.21.15.33:0/855516581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7400b9d0 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7959a700 1 -- 172.21.15.33:0/855516581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d740092e0 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7b59e700 1 -- 172.21.15.33:0/855516581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 msgr2=0x7f6d7c06a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/855516581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c06a660 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009f20 tx=0x7f6d74009840 comp rx=0 tx=0).stop 2023-10-06T05:20:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7b59e700 1 -- 172.21.15.33:0/855516581 shutdown_connections 2023-10-06T05:20:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/855516581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c06a660 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/855516581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.428+0000 7f6d7b59e700 1 -- 172.21.15.33:0/855516581 >> 172.21.15.33:0/855516581 conn(0x7f6d7c0cc930 msgr2=0x7f6d7c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.429+0000 7f6d7b59e700 1 -- 172.21.15.33:0/855516581 shutdown_connections 2023-10-06T05:20:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.429+0000 7f6d7b59e700 1 -- 172.21.15.33:0/855516581 wait complete. 2023-10-06T05:20:03.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.430+0000 7f6d7b59e700 1 Processor -- start 2023-10-06T05:20:03.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.430+0000 7f6d7b59e700 1 -- start start 2023-10-06T05:20:03.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d7c0c6170 0x7f6d7c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:03.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:03.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c1528d0 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c14e300 con 0x7f6d7c0c6170 2023-10-06T05:20:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c151cc0 con 0x7f6d7c159520 2023-10-06T05:20:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d79d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d79d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c14ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39866/0 (socket says 172.21.15.33:39866) 2023-10-06T05:20:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d79d9b700 1 -- 172.21.15.33:0/539498860 learned_addr learned my addr 172.21.15.33:0/539498860 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.431+0000 7f6d7a59c700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d7c0c6170 0x7f6d7c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d7ad9d700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d79d9b700 1 -- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 msgr2=0x7f6d7c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d79d9b700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d79d9b700 1 -- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d7c0c6170 msgr2=0x7f6d7c15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d79d9b700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d7c0c6170 0x7f6d7c15ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d79d9b700 1 -- 172.21.15.33:0/539498860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74009ae0 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d7a59c700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d7c0c6170 0x7f6d7c15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d79d9b700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c14ddc0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c7e0 tx=0x7f6d70007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70012700 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d70012880 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70012c40 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c151fa0 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.432+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c152560 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.433+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c152a40 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.434+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d7001b400 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.435+0000 7f6d6a7fc700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d5c066f60 0x7f6d5c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:03.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.435+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6d70084320 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.436+0000 7f6d7a59c700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d5c066f60 0x7f6d5c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:03.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.436+0000 7f6d7a59c700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d5c066f60 0x7f6d5c069410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009ce0 tx=0x7f6d6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.439+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d70051360 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.687+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d7c04c380 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.688+0000 7f6d6a7fc700 1 -- 172.21.15.33:0/539498860 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f6d70054980 con 0x7f6d7c0c82a0 2023-10-06T05:20:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:03.690 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d5c066f60 msgr2=0x7f6d5c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d5c066f60 0x7f6d5c069410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009ce0 tx=0x7f6d6c008040 comp rx=0 tx=0).stop 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 msgr2=0x7f6d7c14ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c14ddc0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c7e0 tx=0x7f6d70007df0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 shutdown_connections 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d5c066f60 0x7f6d5c069410 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d7c159520 0x7f6d7c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d7c0c6170 0x7f6d7c15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 --2- 172.21.15.33:0/539498860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c0c82a0 0x7f6d7c14ddc0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:03.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 >> 172.21.15.33:0/539498860 conn(0x7f6d7c0cc930 msgr2=0x7f6d7c0c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:03.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 shutdown_connections 2023-10-06T05:20:03.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:03.690+0000 7f6d7b59e700 1 -- 172.21.15.33:0/539498860 wait complete. 2023-10-06T05:20:03.694 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:03.710 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:04.711 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:04.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 Processor -- start 2023-10-06T05:20:04.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 -- start start 2023-10-06T05:20:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba2413faa0 0x7fba2413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fba2406aad0 0x7fba2406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240c9cb0 con 0x7fba240c7940 2023-10-06T05:20:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240ca270 con 0x7fba2413faa0 2023-10-06T05:20:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.909+0000 7fba23fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2406b840 con 0x7fba2406aad0 2023-10-06T05:20:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba2413faa0 0x7fba2413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba2413faa0 0x7fba2413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39872/0 (socket says 172.21.15.33:39872) 2023-10-06T05:20:04.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 -- 172.21.15.33:0/41180414 learned_addr learned my addr 172.21.15.33:0/41180414 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba237fe700 1 --2- 172.21.15.33:0/41180414 >> v2:172.21.15.167:3300/0 conn(0x7fba2406aad0 0x7fba2406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 -- 172.21.15.33:0/41180414 >> v2:172.21.15.167:3300/0 conn(0x7fba2406aad0 msgr2=0x7fba2406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 --2- 172.21.15.33:0/41180414 >> v2:172.21.15.167:3300/0 conn(0x7fba2406aad0 0x7fba2406af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 -- 172.21.15.33:0/41180414 >> v1:172.21.15.167:6789/0 conn(0x7fba240c7940 legacy=0x7fba240c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 -- 172.21.15.33:0/41180414 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2406bb20 con 0x7fba2413faa0 2023-10-06T05:20:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba237fe700 1 --2- 172.21.15.33:0/41180414 >> v2:172.21.15.167:3300/0 conn(0x7fba2406aad0 0x7fba2406af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.910+0000 7fba22ffd700 1 --2- 172.21.15.33:0/41180414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba2413fe70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fba24156b90 tx=0x7fba14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e88dd913cf29611 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba21ffb700 1 -- 172.21.15.33:0/41180414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400b860 con 0x7fba2413faa0 2023-10-06T05:20:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba21ffb700 1 -- 172.21.15.33:0/41180414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400b9c0 con 0x7fba2413faa0 2023-10-06T05:20:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba21ffb700 1 -- 172.21.15.33:0/41180414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400bd60 con 0x7fba2413faa0 2023-10-06T05:20:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 -- 172.21.15.33:0/41180414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 msgr2=0x7fba2413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 --2- 172.21.15.33:0/41180414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba2413fe70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fba24156b90 tx=0x7fba14009250 comp rx=0 tx=0).stop 2023-10-06T05:20:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 -- 172.21.15.33:0/41180414 shutdown_connections 2023-10-06T05:20:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 --2- 172.21.15.33:0/41180414 >> v2:172.21.15.167:3300/0 conn(0x7fba2406aad0 0x7fba2406af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 --2- 172.21.15.33:0/41180414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba2413fe70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 -- 172.21.15.33:0/41180414 >> 172.21.15.33:0/41180414 conn(0x7fba2406d030 msgr2=0x7fba2406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 -- 172.21.15.33:0/41180414 shutdown_connections 2023-10-06T05:20:04.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.911+0000 7fba23fff700 1 -- 172.21.15.33:0/41180414 wait complete. 2023-10-06T05:20:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.913+0000 7fba23fff700 1 Processor -- start 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.913+0000 7fba23fff700 1 -- start start 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba23fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2406aad0 0x7fba2413af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba23fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba240c7940 0x7fba2413e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba23fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba240a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba23fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2413ebc0 con 0x7fba2413faa0 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba23fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240aeb10 con 0x7fba2406aad0 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba23fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2413e1f0 con 0x7fba240c7940 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.914+0000 7fba237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba240a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba240a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39874/0 (socket says 172.21.15.33:39874) 2023-10-06T05:20:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 -- 172.21.15.33:0/3172658452 learned_addr learned my addr 172.21.15.33:0/3172658452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba22ffd700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2406aad0 0x7fba2413af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba227fc700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba240c7940 0x7fba2413e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 -- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba240c7940 msgr2=0x7fba2413e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba240c7940 0x7fba2413e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 -- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2406aad0 msgr2=0x7fba2413af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2406aad0 0x7fba2413af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 -- 172.21.15.33:0/3172658452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140094f0 con 0x7fba2413faa0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba237fe700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba240a5480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00d1b0 tx=0x7fba1c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00dcf0 con 0x7fba2413faa0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c00f040 con 0x7fba2413faa0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.915+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2413d5e0 con 0x7fba2413faa0 2023-10-06T05:20:04.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.916+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c015670 con 0x7fba2413faa0 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.916+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2413dba0 con 0x7fba2413faa0 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.916+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2404c380 con 0x7fba2413faa0 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.917+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba1c01e3b0 con 0x7fba2413faa0 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.918+0000 7fba0affd700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba10066f30 0x7fba100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.918+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fba1c084ba0 con 0x7fba2413faa0 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.919+0000 7fba22ffd700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba10066f30 0x7fba100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.920+0000 7fba22ffd700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba10066f30 0x7fba100693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba24156b90 tx=0x7fba14017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:04.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:04.922+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba1c051be0 con 0x7fba2413faa0 2023-10-06T05:20:05.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.167+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba240c9f00 con 0x7fba2413faa0 2023-10-06T05:20:05.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.168+0000 7fba0affd700 1 -- 172.21.15.33:0/3172658452 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fba1c01a090 con 0x7fba2413faa0 2023-10-06T05:20:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:05.171 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba10066f30 msgr2=0x7fba100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba10066f30 0x7fba100693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba24156b90 tx=0x7fba14017000 comp rx=0 tx=0).stop 2023-10-06T05:20:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 msgr2=0x7fba240a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba240a5480 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00d1b0 tx=0x7fba1c009a50 comp rx=0 tx=0).stop 2023-10-06T05:20:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 shutdown_connections 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba10066f30 0x7fba100693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2413faa0 0x7fba240a5480 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2406aad0 0x7fba2413af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 --2- 172.21.15.33:0/3172658452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba240c7940 0x7fba2413e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.170+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 >> 172.21.15.33:0/3172658452 conn(0x7fba2406d030 msgr2=0x7fba240c2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.171+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 shutdown_connections 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:05.171+0000 7fba23fff700 1 -- 172.21.15.33:0/3172658452 wait complete. 2023-10-06T05:20:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:05.191 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:06.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:06.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.391+0000 7efff239d700 1 Processor -- start 2023-10-06T05:20:06.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.391+0000 7efff239d700 1 -- start start 2023-10-06T05:20:06.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff239d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7effec13faa0 0x7effec13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:06.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec0c9cb0 con 0x7effec06aad0 2023-10-06T05:20:06.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff239d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec0ca270 con 0x7effec0c7940 2023-10-06T05:20:06.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff239d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec06b640 con 0x7effec13faa0 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff139b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7effec13faa0 0x7effec13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff139b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7effec13faa0 0x7effec13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39884/0 (socket says 172.21.15.33:39884) 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff139b700 1 -- 172.21.15.33:0/644690363 learned_addr learned my addr 172.21.15.33:0/644690363 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff1b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7effec06aad0 0x7effec06af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58202/0 (socket says 172.21.15.33:58202) 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.393+0000 7efff139b700 1 -- 172.21.15.33:0/644690363 >> v1:172.21.15.167:6789/0 conn(0x7effec0c7940 legacy=0x7effec0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7efff139b700 1 -- 172.21.15.33:0/644690363 >> v1:172.21.15.33:6789/0 conn(0x7effec06aad0 legacy=0x7effec06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7efff139b700 1 -- 172.21.15.33:0/644690363 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec06b840 con 0x7effec13faa0 2023-10-06T05:20:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7effe3fff700 1 -- 172.21.15.33:0/644690363 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811879659 0 0) 0x7effec0c9cb0 con 0x7effec06aad0 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7efff139b700 1 --2- 172.21.15.33:0/644690363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec13fe70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7effe80099b0 tx=0x7effe8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade00690e620d875 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7effe3fff700 1 -- 172.21.15.33:0/644690363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800b860 con 0x7effec13faa0 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7effe3fff700 1 -- 172.21.15.33:0/644690363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe800b9c0 con 0x7effec13faa0 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.394+0000 7effe3fff700 1 -- 172.21.15.33:0/644690363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800bd60 con 0x7effec13faa0 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 -- 172.21.15.33:0/644690363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 msgr2=0x7effec13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 --2- 172.21.15.33:0/644690363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec13fe70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7effe80099b0 tx=0x7effe8009250 comp rx=0 tx=0).stop 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 -- 172.21.15.33:0/644690363 shutdown_connections 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 --2- 172.21.15.33:0/644690363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec13fe70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 -- 172.21.15.33:0/644690363 >> 172.21.15.33:0/644690363 conn(0x7effec06d030 msgr2=0x7effec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 -- 172.21.15.33:0/644690363 shutdown_connections 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.395+0000 7efff239d700 1 -- 172.21.15.33:0/644690363 wait complete. 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.396+0000 7efff239d700 1 Processor -- start 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.396+0000 7efff239d700 1 -- start start 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effec06aad0 0x7effec1657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec0c7940 0x7effec0c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec156c00 con 0x7effec13faa0 2023-10-06T05:20:06.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec1574a0 con 0x7effec0c7940 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec159b90 con 0x7effec06aad0 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff1b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.397+0000 7efff1b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec1591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39898/0 (socket says 172.21.15.33:39898) 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 -- 172.21.15.33:0/1444443282 learned_addr learned my addr 172.21.15.33:0/1444443282 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff139b700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effec06aad0 0x7effec1657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff0b9a700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec0c7940 0x7effec0c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 -- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effec06aad0 msgr2=0x7effec1657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effec06aad0 0x7effec1657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 -- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec0c7940 msgr2=0x7effec0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec0c7940 0x7effec0c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 -- 172.21.15.33:0/1444443282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80094f0 con 0x7effec13faa0 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff1b9c700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec1591b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7effe400cd50 tx=0x7effe4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4005df0 con 0x7effec13faa0 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe400a8b0 con 0x7effec13faa0 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400ac70 con 0x7effec13faa0 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.398+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec156e50 con 0x7effec13faa0 2023-10-06T05:20:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.399+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec0c5520 con 0x7effec13faa0 2023-10-06T05:20:06.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.400+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec04c380 con 0x7effec13faa0 2023-10-06T05:20:06.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.400+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effe4007300 con 0x7effec13faa0 2023-10-06T05:20:06.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.401+0000 7effe17fa700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7effd0066f30 0x7effd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:06.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.401+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7effe40847d0 con 0x7effec13faa0 2023-10-06T05:20:06.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.402+0000 7efff139b700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7effd0066f30 0x7effd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:06.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.403+0000 7efff139b700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7effd0066f30 0x7effd00693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7effe80099b0 tx=0x7effe8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:06.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.406+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effe4051890 con 0x7effec13faa0 2023-10-06T05:20:06.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.653+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effec06bb20 con 0x7effec13faa0 2023-10-06T05:20:06.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.653+0000 7effe17fa700 1 -- 172.21.15.33:0/1444443282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7effe4054eb0 con 0x7effec13faa0 2023-10-06T05:20:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:06.655 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7effd0066f30 msgr2=0x7effd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7effd0066f30 0x7effd00693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7effe80099b0 tx=0x7effe8017000 comp rx=0 tx=0).stop 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 msgr2=0x7effec1591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec1591b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7effe400cd50 tx=0x7effe4001b10 comp rx=0 tx=0).stop 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 shutdown_connections 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7effd0066f30 0x7effd00693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec13faa0 0x7effec1591b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7effec06aad0 0x7effec1657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 --2- 172.21.15.33:0/1444443282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec0c7940 0x7effec0c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:06.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 >> 172.21.15.33:0/1444443282 conn(0x7effec06d030 msgr2=0x7effec06e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:06.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 shutdown_connections 2023-10-06T05:20:06.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:06.656+0000 7efff239d700 1 -- 172.21.15.33:0/1444443282 wait complete. 2023-10-06T05:20:06.659 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:06.675 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:07.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.872+0000 7fd9ed79d700 1 Processor -- start 2023-10-06T05:20:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.872+0000 7fd9ed79d700 1 -- start start 2023-10-06T05:20:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ed79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd9e813faa0 0x7fd9e813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ed79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9e806aad0 0x7fd9e806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ed79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e80c9cb0 con 0x7fd9e80c7940 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ed79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e80ca270 con 0x7fd9e806aad0 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ed79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e806b840 con 0x7fd9e813faa0 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9e806aad0 0x7fd9e806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9e806aad0 0x7fd9e806af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39910/0 (socket says 172.21.15.33:39910) 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 -- 172.21.15.33:0/808395659 learned_addr learned my addr 172.21.15.33:0/808395659 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9e7fff700 1 --2- 172.21.15.33:0/808395659 >> v2:172.21.15.167:3300/0 conn(0x7fd9e813faa0 0x7fd9e813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 -- 172.21.15.33:0/808395659 >> v2:172.21.15.167:3300/0 conn(0x7fd9e813faa0 msgr2=0x7fd9e813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:07.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 --2- 172.21.15.33:0/808395659 >> v2:172.21.15.167:3300/0 conn(0x7fd9e813faa0 0x7fd9e813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 -- 172.21.15.33:0/808395659 >> v1:172.21.15.167:6789/0 conn(0x7fd9e80c7940 legacy=0x7fd9e80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.874+0000 7fd9ecf9c700 1 -- 172.21.15.33:0/808395659 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e806bb20 con 0x7fd9e806aad0 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9e7fff700 1 --2- 172.21.15.33:0/808395659 >> v2:172.21.15.167:3300/0 conn(0x7fd9e813faa0 0x7fd9e813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9ecf9c700 1 --2- 172.21.15.33:0/808395659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e806af30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd9e806fe50 tx=0x7fd9e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c0c722f9a13494 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9e6ffd700 1 -- 172.21.15.33:0/808395659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000e070 con 0x7fd9e806aad0 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9e6ffd700 1 -- 172.21.15.33:0/808395659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000ba10 con 0x7fd9e806aad0 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9e6ffd700 1 -- 172.21.15.33:0/808395659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e00092e0 con 0x7fd9e806aad0 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9ed79d700 1 -- 172.21.15.33:0/808395659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 msgr2=0x7fd9e806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.875+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/808395659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e806af30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd9e806fe50 tx=0x7fd9e0009840 comp rx=0 tx=0).stop 2023-10-06T05:20:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.876+0000 7fd9ed79d700 1 -- 172.21.15.33:0/808395659 shutdown_connections 2023-10-06T05:20:07.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.876+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/808395659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e806af30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.876+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/808395659 >> v2:172.21.15.167:3300/0 conn(0x7fd9e813faa0 0x7fd9e813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.876+0000 7fd9ed79d700 1 -- 172.21.15.33:0/808395659 >> 172.21.15.33:0/808395659 conn(0x7fd9e806d030 msgr2=0x7fd9e806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:07.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.876+0000 7fd9ed79d700 1 -- 172.21.15.33:0/808395659 shutdown_connections 2023-10-06T05:20:07.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.876+0000 7fd9ed79d700 1 -- 172.21.15.33:0/808395659 wait complete. 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.877+0000 7fd9ed79d700 1 Processor -- start 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.877+0000 7fd9ed79d700 1 -- start start 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.878+0000 7fd9ed79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.878+0000 7fd9ed79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e80c7940 0x7fd9e81584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.878+0000 7fd9ed79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e813faa0 0x7fd9e80c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.878+0000 7fd9ed79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8157410 con 0x7fd9e806aad0 2023-10-06T05:20:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.878+0000 7fd9ed79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e81576f0 con 0x7fd9e813faa0 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.878+0000 7fd9ed79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e81579d0 con 0x7fd9e80c7940 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e8156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39916/0 (socket says 172.21.15.33:39916) 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 -- 172.21.15.33:0/1865198144 learned_addr learned my addr 172.21.15.33:0/1865198144 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9ecf9c700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e813faa0 0x7fd9e80c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e77fe700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e80c7940 0x7fd9e81584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 -- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e80c7940 msgr2=0x7fd9e81584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e80c7940 0x7fd9e81584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 -- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e813faa0 msgr2=0x7fd9e80c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e813faa0 0x7fd9e80c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 -- 172.21.15.33:0/1865198144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e0009ae0 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e77fe700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e80c7940 0x7fd9e81584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9e7fff700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e8156be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c850 tx=0x7fd9d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.879+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011600 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.880+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8011760 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.880+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011b00 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.880+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e8157cb0 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.880+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e8157120 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.881+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e81581d0 con 0x7fd9e806aad0 2023-10-06T05:20:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.881+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9d80073b0 con 0x7fd9e806aad0 2023-10-06T05:20:07.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.882+0000 7fd9cffff700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9d4066f30 0x7fd9d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.882+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd9d8084760 con 0x7fd9e806aad0 2023-10-06T05:20:07.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.883+0000 7fd9e77fe700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9d4066f30 0x7fd9d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.883+0000 7fd9e77fe700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9d4066f30 0x7fd9d40693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd9e813f0e0 tx=0x7fd9dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:07.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:07.886+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9e81581d0 con 0x7fd9e806aad0 2023-10-06T05:20:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.132+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9e81581d0 con 0x7fd9e806aad0 2023-10-06T05:20:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.133+0000 7fd9cffff700 1 -- 172.21.15.33:0/1865198144 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd9e81581d0 con 0x7fd9e806aad0 2023-10-06T05:20:08.135 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:08.135 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9d4066f30 msgr2=0x7fd9d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9d4066f30 0x7fd9d40693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd9e813f0e0 tx=0x7fd9dc009040 comp rx=0 tx=0).stop 2023-10-06T05:20:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 msgr2=0x7fd9e8156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e8156be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c850 tx=0x7fd9d8008040 comp rx=0 tx=0).stop 2023-10-06T05:20:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 shutdown_connections 2023-10-06T05:20:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9d4066f30 0x7fd9d40693e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e813faa0 0x7fd9e80c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e806aad0 0x7fd9e8156be0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 --2- 172.21.15.33:0/1865198144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e80c7940 0x7fd9e81584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 >> 172.21.15.33:0/1865198144 conn(0x7fd9e806d030 msgr2=0x7fd9e815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 shutdown_connections 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:08.135+0000 7fd9ed79d700 1 -- 172.21.15.33:0/1865198144 wait complete. 2023-10-06T05:20:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:08.154 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:09.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.354+0000 7fa755f9d700 1 Processor -- start 2023-10-06T05:20:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.354+0000 7fa755f9d700 1 -- start start 2023-10-06T05:20:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.355+0000 7fa755f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa750159520 0x7fa7500c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.355+0000 7fa755f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 0x7fa7500c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.355+0000 7fa755f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7500c65e0 con 0x7fa7500c80d0 2023-10-06T05:20:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.355+0000 7fa755f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75006c820 con 0x7fa750159520 2023-10-06T05:20:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.355+0000 7fa755f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75006b670 con 0x7fa7500c6170 2023-10-06T05:20:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa74ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 0x7fa7500c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa754f9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa750159520 0x7fa7500c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa74ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 0x7fa7500c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:53678/0 (socket says 172.21.15.33:53678) 2023-10-06T05:20:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa75579c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa7500c80d0 0x7fa750067ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:60400/0 (socket says 172.21.15.33:60400) 2023-10-06T05:20:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa74ffff700 1 -- 172.21.15.33:0/3680531367 learned_addr learned my addr 172.21.15.33:0/3680531367 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa754f9b700 1 -- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 msgr2=0x7fa7500c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa754f9b700 1 --2- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 0x7fa7500c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa754f9b700 1 -- 172.21.15.33:0/3680531367 >> v1:172.21.15.167:6789/0 conn(0x7fa7500c80d0 legacy=0x7fa750067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa754f9b700 1 -- 172.21.15.33:0/3680531367 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75006a160 con 0x7fa750159520 2023-10-06T05:20:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.356+0000 7fa74ffff700 1 --2- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 0x7fa7500c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa754f9b700 1 --2- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750159520 0x7fa7500c5c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7440098e0 tx=0x7fa7440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ce0e33ead88340 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa74f7fe700 1 -- 172.21.15.33:0/3680531367 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74400b640 con 0x7fa750159520 2023-10-06T05:20:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa74f7fe700 1 -- 172.21.15.33:0/3680531367 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74400b7a0 con 0x7fa750159520 2023-10-06T05:20:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa74f7fe700 1 -- 172.21.15.33:0/3680531367 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74400bb40 con 0x7fa750159520 2023-10-06T05:20:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa755f9d700 1 -- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750159520 msgr2=0x7fa7500c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa755f9d700 1 --2- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750159520 0x7fa7500c5c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7440098e0 tx=0x7fa7440091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.357+0000 7fa755f9d700 1 -- 172.21.15.33:0/3680531367 shutdown_connections 2023-10-06T05:20:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.358+0000 7fa755f9d700 1 --2- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750159520 0x7fa7500c5c30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.358+0000 7fa755f9d700 1 --2- 172.21.15.33:0/3680531367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c6170 0x7fa7500c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.358+0000 7fa755f9d700 1 -- 172.21.15.33:0/3680531367 >> 172.21.15.33:0/3680531367 conn(0x7fa7500cc930 msgr2=0x7fa7500ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.358+0000 7fa755f9d700 1 -- 172.21.15.33:0/3680531367 shutdown_connections 2023-10-06T05:20:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.358+0000 7fa755f9d700 1 -- 172.21.15.33:0/3680531367 wait complete. 2023-10-06T05:20:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.359+0000 7fa755f9d700 1 Processor -- start 2023-10-06T05:20:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.359+0000 7fa755f9d700 1 -- start start 2023-10-06T05:20:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa755f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500c6170 0x7fa750156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa755f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c80d0 0x7fa7500abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa755f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 0x7fa75014a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa755f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75015e320 con 0x7fa750159520 2023-10-06T05:20:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa755f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500acf30 con 0x7fa7500c6170 2023-10-06T05:20:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa755f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75015eab0 con 0x7fa7500c80d0 2023-10-06T05:20:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa75579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 0x7fa75014a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa75579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 0x7fa75014a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39928/0 (socket says 172.21.15.33:39928) 2023-10-06T05:20:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.360+0000 7fa75579c700 1 -- 172.21.15.33:0/738084160 learned_addr learned my addr 172.21.15.33:0/738084160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa754f9b700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500c6170 0x7fa750156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa74ffff700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c80d0 0x7fa7500abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa75579c700 1 -- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c80d0 msgr2=0x7fa7500abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa75579c700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c80d0 0x7fa7500abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa75579c700 1 -- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500c6170 msgr2=0x7fa750156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa75579c700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500c6170 0x7fa750156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa75579c700 1 -- 172.21.15.33:0/738084160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0094f0 con 0x7fa750159520 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa75579c700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 0x7fa75014a300 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa74800cd00 tx=0x7fa748001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74800ce60 con 0x7fa750159520 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa748005df0 con 0x7fa750159520 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa744009420 con 0x7fa750159520 2023-10-06T05:20:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.361+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74800ab00 con 0x7fa750159520 2023-10-06T05:20:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.362+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7500ac870 con 0x7fa750159520 2023-10-06T05:20:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.362+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7500ad0c0 con 0x7fa750159520 2023-10-06T05:20:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.363+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7480070f0 con 0x7fa750159520 2023-10-06T05:20:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.365+0000 7fa74cff9700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa734066f30 0x7fa7340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.365+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa748084750 con 0x7fa750159520 2023-10-06T05:20:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.365+0000 7fa754f9b700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa734066f30 0x7fa7340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.366+0000 7fa754f9b700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa734066f30 0x7fa7340693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa7440093f0 tx=0x7fa744017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.369+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa748051810 con 0x7fa750159520 2023-10-06T05:20:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.617+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa75004c380 con 0x7fa750159520 2023-10-06T05:20:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.617+0000 7fa74cff9700 1 -- 172.21.15.33:0/738084160 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fa748082e60 con 0x7fa750159520 2023-10-06T05:20:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa734066f30 msgr2=0x7fa7340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa734066f30 0x7fa7340693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa7440093f0 tx=0x7fa744017000 comp rx=0 tx=0).stop 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 msgr2=0x7fa75014a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 0x7fa75014a300 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa74800cd00 tx=0x7fa748001b10 comp rx=0 tx=0).stop 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 shutdown_connections 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa734066f30 0x7fa7340693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa750159520 0x7fa75014a300 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7500c6170 0x7fa750156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 --2- 172.21.15.33:0/738084160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7500c80d0 0x7fa7500abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 >> 172.21.15.33:0/738084160 conn(0x7fa7500cc930 msgr2=0x7fa750158330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 shutdown_connections 2023-10-06T05:20:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:09.620+0000 7fa755f9d700 1 -- 172.21.15.33:0/738084160 wait complete. 2023-10-06T05:20:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:09.639 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:10.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.834+0000 7f613df9d700 1 Processor -- start 2023-10-06T05:20:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.834+0000 7f613df9d700 1 -- start start 2023-10-06T05:20:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.835+0000 7f613df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f613813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.835+0000 7f613df9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61380c7940 0x7f61380c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.835+0000 7f613df9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61380c9cb0 con 0x7f613806aad0 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.835+0000 7f613df9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613806b660 con 0x7f61380c7940 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.835+0000 7f613df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61380ca270 con 0x7f613813faa0 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61380c7940 0x7f61380c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f613d79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f613806aad0 0x7f613806af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59038/0 (socket says 172.21.15.33:59038) 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61380c7940 0x7f61380c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56236/0 (socket says 172.21.15.33:56236) 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f613d79c700 1 -- 172.21.15.33:0/605374660 learned_addr learned my addr 172.21.15.33:0/605374660 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f613cf9b700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f613813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 -- 172.21.15.33:0/605374660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 msgr2=0x7f613813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f613813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 -- 172.21.15.33:0/605374660 >> v1:172.21.15.33:6789/0 conn(0x7f613806aad0 legacy=0x7f613806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 -- 172.21.15.33:0/605374660 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613806b840 con 0x7f61380c7940 2023-10-06T05:20:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f61377fe700 1 -- 172.21.15.33:0/605374660 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916183583 0 0) 0x7f61380c9cb0 con 0x7f613806aad0 2023-10-06T05:20:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f613cf9b700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f613813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.836+0000 7f6137fff700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61380c7940 0x7f61380c7db0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f61240098e0 tx=0x7f61240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9146849b147728d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f61377fe700 1 -- 172.21.15.33:0/605374660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124009e80 con 0x7f61380c7940 2023-10-06T05:20:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f61377fe700 1 -- 172.21.15.33:0/605374660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612400b860 con 0x7f61380c7940 2023-10-06T05:20:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f61377fe700 1 -- 172.21.15.33:0/605374660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400bbe0 con 0x7f61380c7940 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 -- 172.21.15.33:0/605374660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61380c7940 msgr2=0x7f61380c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61380c7940 0x7f61380c7db0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f61240098e0 tx=0x7f61240091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 -- 172.21.15.33:0/605374660 shutdown_connections 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f613813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 --2- 172.21.15.33:0/605374660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61380c7940 0x7f61380c7db0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 -- 172.21.15.33:0/605374660 >> 172.21.15.33:0/605374660 conn(0x7f613806d030 msgr2=0x7f613806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 -- 172.21.15.33:0/605374660 shutdown_connections 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.837+0000 7f613df9d700 1 -- 172.21.15.33:0/605374660 wait complete. 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.839+0000 7f613df9d700 1 Processor -- start 2023-10-06T05:20:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.839+0000 7f613df9d700 1 -- start start 2023-10-06T05:20:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613df9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 0x7f61381360f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613df9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61380c7940 0x7f613816af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f6138155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613df9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613816bea0 con 0x7f613806aad0 2023-10-06T05:20:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613df9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6138155810 con 0x7f61380c7940 2023-10-06T05:20:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613816b8e0 con 0x7f613813faa0 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613cf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 0x7f61381360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613cf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 0x7f61381360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56240/0 (socket says 172.21.15.33:56240) 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613cf9b700 1 -- 172.21.15.33:0/3139525476 learned_addr learned my addr 172.21.15.33:0/3139525476 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613d79c700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f6138155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f6137fff700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61380c7940 0x7f613816af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613cf9b700 1 -- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 msgr2=0x7f6138155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.840+0000 7f613cf9b700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f6138155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f613cf9b700 1 -- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61380c7940 msgr2=0x7f613816af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f613cf9b700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61380c7940 0x7f613816af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f613cf9b700 1 -- 172.21.15.33:0/3139525476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6124009420 con 0x7f613806aad0 2023-10-06T05:20:10.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f6137fff700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61380c7940 0x7f613816af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:10.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f613cf9b700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 0x7f61381360f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f612c00d860 tx=0x7f612c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:10.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c0125f0 con 0x7f613806aad0 2023-10-06T05:20:10.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c012750 con 0x7f613806aad0 2023-10-06T05:20:10.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c012ad0 con 0x7f613806aad0 2023-10-06T05:20:10.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613816bbc0 con 0x7f613806aad0 2023-10-06T05:20:10.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.841+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6138136630 con 0x7f613806aad0 2023-10-06T05:20:10.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.842+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6138155af0 con 0x7f613806aad0 2023-10-06T05:20:10.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.843+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f612c014430 con 0x7f613806aad0 2023-10-06T05:20:10.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.844+0000 7f6134ff9700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f611c066f30 0x7f611c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:10.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.844+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f612c084bb0 con 0x7f613806aad0 2023-10-06T05:20:10.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.844+0000 7f6137fff700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f611c066f30 0x7f611c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:10.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.845+0000 7f6137fff700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f611c066f30 0x7f611c0693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f61380afca0 tx=0x7f6124017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:10.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:10.848+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f612c051bf0 con 0x7f613806aad0 2023-10-06T05:20:11.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.094+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61381552e0 con 0x7f613806aad0 2023-10-06T05:20:11.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.095+0000 7f6134ff9700 1 -- 172.21.15.33:0/3139525476 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f612c055210 con 0x7f613806aad0 2023-10-06T05:20:11.097 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:11.097 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:11.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f611c066f30 msgr2=0x7f611c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:11.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f611c066f30 0x7f611c0693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f61380afca0 tx=0x7f6124017000 comp rx=0 tx=0).stop 2023-10-06T05:20:11.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 msgr2=0x7f61381360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:11.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 0x7f61381360f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f612c00d860 tx=0x7f612c008040 comp rx=0 tx=0).stop 2023-10-06T05:20:11.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 shutdown_connections 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f611c066f30 0x7f611c0693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f613813faa0 0x7f6138155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f613806aad0 0x7f61381360f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 --2- 172.21.15.33:0/3139525476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61380c7940 0x7f613816af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.097+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 >> 172.21.15.33:0/3139525476 conn(0x7f613806d030 msgr2=0x7f61380c2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.098+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 shutdown_connections 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:11.098+0000 7f613df9d700 1 -- 172.21.15.33:0/3139525476 wait complete. 2023-10-06T05:20:11.100 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:11.117 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:12.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:12.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.317+0000 7f70a0b9d700 1 Processor -- start 2023-10-06T05:20:12.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.317+0000 7f70a0b9d700 1 -- start start 2023-10-06T05:20:12.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.318+0000 7f70a0b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f709c155140 0x7f709c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.318+0000 7f70a0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.318+0000 7f70a0b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0c8e40 con 0x7f709c0adf50 2023-10-06T05:20:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.318+0000 7f70a0b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c06a380 con 0x7f709c155140 2023-10-06T05:20:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.318+0000 7f70a0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c99e0 con 0x7f709c0699c0 2023-10-06T05:20:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54134/0 (socket says 172.21.15.33:54134) 2023-10-06T05:20:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709bfff700 1 -- 172.21.15.33:0/1336653528 learned_addr learned my addr 172.21.15.33:0/1336653528 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709b7fe700 1 --2- 172.21.15.33:0/1336653528 >> v2:172.21.15.167:3300/0 conn(0x7f709c155140 0x7f709c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709affd700 1 --1- 172.21.15.33:0/1336653528 >> v1:172.21.15.167:6789/0 conn(0x7f709c0adf50 0x7f709c0c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:42376/0 (socket says 172.21.15.33:42376) 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709b7fe700 1 -- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 msgr2=0x7f709c069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709b7fe700 1 --2- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709b7fe700 1 -- 172.21.15.33:0/1336653528 >> v1:172.21.15.167:6789/0 conn(0x7f709c0adf50 legacy=0x7f709c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709b7fe700 1 -- 172.21.15.33:0/1336653528 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0ae3f0 con 0x7f709c155140 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.319+0000 7f709bfff700 1 --2- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f709b7fe700 1 --2- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c158700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f708c009910 tx=0x7f708c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ef550b4fb6e3c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f709a7fc700 1 -- 172.21.15.33:0/1336653528 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00b640 con 0x7f709c155140 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f709a7fc700 1 -- 172.21.15.33:0/1336653528 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c00b7a0 con 0x7f709c155140 2023-10-06T05:20:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f709a7fc700 1 -- 172.21.15.33:0/1336653528 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00bb60 con 0x7f709c155140 2023-10-06T05:20:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f70a0b9d700 1 -- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 msgr2=0x7f709c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c158700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f708c009910 tx=0x7f708c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f70a0b9d700 1 -- 172.21.15.33:0/1336653528 shutdown_connections 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c158700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/1336653528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.320+0000 7f70a0b9d700 1 -- 172.21.15.33:0/1336653528 >> 172.21.15.33:0/1336653528 conn(0x7f709c06d030 msgr2=0x7f709c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.321+0000 7f70a0b9d700 1 -- 172.21.15.33:0/1336653528 shutdown_connections 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.321+0000 7f70a0b9d700 1 -- 172.21.15.33:0/1336653528 wait complete. 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.322+0000 7f70a0b9d700 1 Processor -- start 2023-10-06T05:20:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.322+0000 7f70a0b9d700 1 -- start start 2023-10-06T05:20:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f70a0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f70a0b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 0x7f709c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f70a0b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f70a0b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c1502d0 con 0x7f709c0adf50 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f70a0b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0cbcf0 con 0x7f709c155140 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f70a0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c9cb0 con 0x7f709c0699c0 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f709affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 0x7f709c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f709affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 0x7f709c0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56242/0 (socket says 172.21.15.33:56242) 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f709affd700 1 -- 172.21.15.33:0/234276204 learned_addr learned my addr 172.21.15.33:0/234276204 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f709b7fe700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.323+0000 7f709bfff700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f709affd700 1 -- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 msgr2=0x7f709c0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f709affd700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c0c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f709affd700 1 -- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 msgr2=0x7f709c0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f709affd700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f709affd700 1 -- 172.21.15.33:0/234276204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009ae0 con 0x7f709c0adf50 2023-10-06T05:20:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f709affd700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 0x7f709c0cb270 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f709000c7c0 tx=0x7f7090007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090005a70 con 0x7f709c0adf50 2023-10-06T05:20:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090005bd0 con 0x7f709c0adf50 2023-10-06T05:20:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900070d0 con 0x7f709c0adf50 2023-10-06T05:20:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c009420 con 0x7f709c0adf50 2023-10-06T05:20:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.324+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c169ef0 con 0x7f709c0adf50 2023-10-06T05:20:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.325+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c0b20f0 con 0x7f709c0adf50 2023-10-06T05:20:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.326+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7090007480 con 0x7f709c0adf50 2023-10-06T05:20:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.327+0000 7f70837fe700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7088066f30 0x7f70880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.327+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7090083b70 con 0x7f709c0adf50 2023-10-06T05:20:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.328+0000 7f709b7fe700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7088066f30 0x7f70880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.328+0000 7f709b7fe700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7088066f30 0x7f70880693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f708c0091e0 tx=0x7f708c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:12.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.331+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7090050bb0 con 0x7f709c0adf50 2023-10-06T05:20:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.578+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f709c04c380 con 0x7f709c0adf50 2023-10-06T05:20:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.579+0000 7f70837fe700 1 -- 172.21.15.33:0/234276204 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f7090017020 con 0x7f709c0adf50 2023-10-06T05:20:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:12.581 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7088066f30 msgr2=0x7f70880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7088066f30 0x7f70880693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f708c0091e0 tx=0x7f708c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 msgr2=0x7f709c0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 0x7f709c0cb270 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f709000c7c0 tx=0x7f7090007f70 comp rx=0 tx=0).stop 2023-10-06T05:20:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 shutdown_connections 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7088066f30 0x7f70880693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f709c155140 0x7f709c0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f709c0699c0 0x7f709c0c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 --2- 172.21.15.33:0/234276204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f709c0adf50 0x7f709c0cb270 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 >> 172.21.15.33:0/234276204 conn(0x7f709c06d030 msgr2=0x7f709c0c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 shutdown_connections 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:12.581+0000 7f70a0b9d700 1 -- 172.21.15.33:0/234276204 wait complete. 2023-10-06T05:20:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:12.600 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:13.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:13.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.800+0000 7fd9cdd9d700 1 Processor -- start 2023-10-06T05:20:13.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.800+0000 7fd9cdd9d700 1 -- start start 2023-10-06T05:20:13.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9cdd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c8153b10 0x7fd9c80c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:13.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9cdd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9c80c2610 0x7fd9c80c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9cdd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c815abd0 con 0x7fd9c80c6b30 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9cdd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c80c7510 con 0x7fd9c80c2610 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9cdd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c9b00 con 0x7fd9c8153b10 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9c7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd9c80c6b30 0x7fd9c80675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59046/0 (socket says 172.21.15.33:59046) 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9c7fff700 1 -- 172.21.15.33:0/3709165797 learned_addr learned my addr 172.21.15.33:0/3709165797 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9cd59c700 1 --2- 172.21.15.33:0/3709165797 >> v2:172.21.15.33:3300/0 conn(0x7fd9c80c2610 0x7fd9c80c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9ccd9b700 1 --2- 172.21.15.33:0/3709165797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c8153b10 0x7fd9c80c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.802+0000 7fd9c77fe700 1 -- 172.21.15.33:0/3709165797 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140372960 0 0) 0x7fd9c815abd0 con 0x7fd9c80c6b30 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9c77fe700 1 -- 172.21.15.33:0/3709165797 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b8003570 con 0x7fd9c80c6b30 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9cd59c700 1 -- 172.21.15.33:0/3709165797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c8153b10 msgr2=0x7fd9c80c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9cd59c700 1 --2- 172.21.15.33:0/3709165797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c8153b10 0x7fd9c80c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9cd59c700 1 -- 172.21.15.33:0/3709165797 >> v1:172.21.15.33:6789/0 conn(0x7fd9c80c6b30 legacy=0x7fd9c80675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9cd59c700 1 -- 172.21.15.33:0/3709165797 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c815a050 con 0x7fd9c80c2610 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9cd59c700 1 --2- 172.21.15.33:0/3709165797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c80c6fd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009e60 tx=0x7fd9c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4165053998e138cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9c77fe700 1 -- 172.21.15.33:0/3709165797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000e070 con 0x7fd9c80c2610 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9c77fe700 1 -- 172.21.15.33:0/3709165797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c000bab0 con 0x7fd9c80c2610 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.803+0000 7fd9c77fe700 1 -- 172.21.15.33:0/3709165797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000be70 con 0x7fd9c80c2610 2023-10-06T05:20:13.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3709165797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 msgr2=0x7fd9c80c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3709165797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c80c6fd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009e60 tx=0x7fd9c0009780 comp rx=0 tx=0).stop 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3709165797 shutdown_connections 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3709165797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c80c6fd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3709165797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c8153b10 0x7fd9c80c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3709165797 >> 172.21.15.33:0/3709165797 conn(0x7fd9c806c560 msgr2=0x7fd9c806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3709165797 shutdown_connections 2023-10-06T05:20:13.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.804+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3709165797 wait complete. 2023-10-06T05:20:13.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.806+0000 7fd9cdd9d700 1 Processor -- start 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.806+0000 7fd9cdd9d700 1 -- start start 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cdd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c813bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cdd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c80c6b30 0x7fd9c80af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cdd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9c8153b10 0x7fd9c8158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cdd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c813d3a0 con 0x7fd9c80c2610 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cdd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c813cd90 con 0x7fd9c8153b10 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cdd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80afde0 con 0x7fd9c80c6b30 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c813bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c813bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56262/0 (socket says 172.21.15.33:56262) 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 -- 172.21.15.33:0/3876903052 learned_addr learned my addr 172.21.15.33:0/3876903052 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9c7fff700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c80c6b30 0x7fd9c80af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:13.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9cd59c700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9c8153b10 0x7fd9c8158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:13.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 -- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c80c6b30 msgr2=0x7fd9c80af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:13.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c80c6b30 0x7fd9c80af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 -- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9c8153b10 msgr2=0x7fd9c8158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:13.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9c8153b10 0x7fd9c8158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 -- 172.21.15.33:0/3876903052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0009a20 con 0x7fd9c80c2610 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.807+0000 7fd9ccd9b700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c813bea0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00ec80 tx=0x7fd9bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.808+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00c9d0 con 0x7fd9c80c2610 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.808+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc00cb30 con 0x7fd9c80c2610 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.808+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c813d030 con 0x7fd9c80c2610 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.808+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc013470 con 0x7fd9c80c2610 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.808+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c813c3e0 con 0x7fd9c80c2610 2023-10-06T05:20:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.809+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c804c380 con 0x7fd9c80c2610 2023-10-06T05:20:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.810+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9bc013820 con 0x7fd9c80c2610 2023-10-06T05:20:13.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.811+0000 7fd9c4ff9700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9ac066f60 0x7fd9ac069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:13.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.811+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd9bc084a10 con 0x7fd9c80c2610 2023-10-06T05:20:13.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.812+0000 7fd9c7fff700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9ac066f60 0x7fd9ac069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.813+0000 7fd9c7fff700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9ac066f60 0x7fd9ac069410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd9c81533e0 tx=0x7fd9b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:13.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:13.816+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9bc051a50 con 0x7fd9c80c2610 2023-10-06T05:20:14.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.066+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9c80c9d00 con 0x7fd9c80c2610 2023-10-06T05:20:14.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.066+0000 7fd9c4ff9700 1 -- 172.21.15.33:0/3876903052 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd9bc055070 con 0x7fd9c80c2610 2023-10-06T05:20:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:14.068 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9ac066f60 msgr2=0x7fd9ac069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9ac066f60 0x7fd9ac069410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd9c81533e0 tx=0x7fd9b4009040 comp rx=0 tx=0).stop 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 msgr2=0x7fd9c813bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c813bea0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00ec80 tx=0x7fd9bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 shutdown_connections 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd9ac066f60 0x7fd9ac069410 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9c8153b10 0x7fd9c8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c80c2610 0x7fd9c813bea0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 --2- 172.21.15.33:0/3876903052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9c80c6b30 0x7fd9c80af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 >> 172.21.15.33:0/3876903052 conn(0x7fd9c806c560 msgr2=0x7fd9c80675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 shutdown_connections 2023-10-06T05:20:14.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:14.069+0000 7fd9cdd9d700 1 -- 172.21.15.33:0/3876903052 wait complete. 2023-10-06T05:20:14.072 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:14.088 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:15.089 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.286+0000 7f584a59d700 1 Processor -- start 2023-10-06T05:20:15.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.286+0000 7f584a59d700 1 -- start start 2023-10-06T05:20:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.287+0000 7f584a59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f584413faa0 0x7f584413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.287+0000 7f584a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f584406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:15.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.287+0000 7f584a59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58440c9cb0 con 0x7f58440c7940 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.287+0000 7f584a59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58440ca270 con 0x7f584413faa0 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.287+0000 7f584a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584406b840 con 0x7f584406aad0 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f584413faa0 0x7f584413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f584413faa0 0x7f584413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56268/0 (socket says 172.21.15.33:56268) 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 -- 172.21.15.33:0/3726850232 learned_addr learned my addr 172.21.15.33:0/3726850232 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f5848d9a700 1 --1- 172.21.15.33:0/3726850232 >> v1:172.21.15.33:6789/0 conn(0x7f58440c7940 0x7f58440c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59056/0 (socket says 172.21.15.33:59056) 2023-10-06T05:20:15.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f5849d9c700 1 --2- 172.21.15.33:0/3726850232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f584406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 -- 172.21.15.33:0/3726850232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 msgr2=0x7f584406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 --2- 172.21.15.33:0/3726850232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f584406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 -- 172.21.15.33:0/3726850232 >> v1:172.21.15.33:6789/0 conn(0x7f58440c7940 legacy=0x7f58440c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f584959b700 1 -- 172.21.15.33:0/3726850232 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584406bb20 con 0x7f584413faa0 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.288+0000 7f583bfff700 1 -- 172.21.15.33:0/3726850232 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274593274 0 0) 0x7f58440c9cb0 con 0x7f58440c7940 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584959b700 1 --2- 172.21.15.33:0/3726850232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f584413fe70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f58400099b0 tx=0x7f5840009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38667bdebf03a89d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f583bfff700 1 -- 172.21.15.33:0/3726850232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840009ea0 con 0x7f584413faa0 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f583bfff700 1 -- 172.21.15.33:0/3726850232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000b860 con 0x7f584413faa0 2023-10-06T05:20:15.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f583bfff700 1 -- 172.21.15.33:0/3726850232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000bc00 con 0x7f584413faa0 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584a59d700 1 -- 172.21.15.33:0/3726850232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 msgr2=0x7f584413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584a59d700 1 --2- 172.21.15.33:0/3726850232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f584413fe70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f58400099b0 tx=0x7f5840009250 comp rx=0 tx=0).stop 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584a59d700 1 -- 172.21.15.33:0/3726850232 shutdown_connections 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584a59d700 1 --2- 172.21.15.33:0/3726850232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f584413fe70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584a59d700 1 --2- 172.21.15.33:0/3726850232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f584406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.289+0000 7f584a59d700 1 -- 172.21.15.33:0/3726850232 >> 172.21.15.33:0/3726850232 conn(0x7f584406d030 msgr2=0x7f584406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.290+0000 7f584a59d700 1 -- 172.21.15.33:0/3726850232 shutdown_connections 2023-10-06T05:20:15.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.290+0000 7f584a59d700 1 -- 172.21.15.33:0/3726850232 wait complete. 2023-10-06T05:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.291+0000 7f584a59d700 1 Processor -- start 2023-10-06T05:20:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.291+0000 7f584a59d700 1 -- start start 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f58440b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584a59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58440c7940 0x7f5844158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584a59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f5844156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584a59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844159910 con 0x7f584413faa0 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584a59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58441572b0 con 0x7f58440c7940 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5844159390 con 0x7f584406aad0 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f5849d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f5844156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f5849d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f5844156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56270/0 (socket says 172.21.15.33:56270) 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f5849d9c700 1 -- 172.21.15.33:0/1336102651 learned_addr learned my addr 172.21.15.33:0/1336102651 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:15.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.292+0000 7f584959b700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f58440b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5848d9a700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58440c7940 0x7f5844158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5849d9c700 1 -- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 msgr2=0x7f58440b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5849d9c700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f58440b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5849d9c700 1 -- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58440c7940 msgr2=0x7f5844158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5849d9c700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58440c7940 0x7f5844158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5849d9c700 1 -- 172.21.15.33:0/1336102651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58400094f0 con 0x7f584413faa0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5848d9a700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58440c7940 0x7f5844158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f5849d9c700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f5844156360 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f583c009e90 tx=0x7f583c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c00cc20 con 0x7f584413faa0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00f040 con 0x7f584413faa0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c014480 con 0x7f584413faa0 2023-10-06T05:20:15.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5844159630 con 0x7f584413faa0 2023-10-06T05:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.293+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5844156f40 con 0x7f584413faa0 2023-10-06T05:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.294+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5844157420 con 0x7f584413faa0 2023-10-06T05:20:15.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.295+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f583c014830 con 0x7f584413faa0 2023-10-06T05:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.296+0000 7f58397fa700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5828066f30 0x7f58280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.296+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f583c0854e0 con 0x7f584413faa0 2023-10-06T05:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.297+0000 7f584959b700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5828066f30 0x7f58280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:15.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.298+0000 7f584959b700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5828066f30 0x7f58280693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58400099b0 tx=0x7f5840017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:15.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.301+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f583c055fb0 con 0x7f584413faa0 2023-10-06T05:20:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.553+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f584404c380 con 0x7f584413faa0 2023-10-06T05:20:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.553+0000 7f58397fa700 1 -- 172.21.15.33:0/1336102651 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f583c05b070 con 0x7f584413faa0 2023-10-06T05:20:15.555 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:15.555 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.555+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5828066f30 msgr2=0x7f58280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.555+0000 7f584a59d700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5828066f30 0x7f58280693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58400099b0 tx=0x7f5840017000 comp rx=0 tx=0).stop 2023-10-06T05:20:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 msgr2=0x7f5844156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f5844156360 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f583c009e90 tx=0x7f583c00b040 comp rx=0 tx=0).stop 2023-10-06T05:20:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 shutdown_connections 2023-10-06T05:20:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5828066f30 0x7f58280693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f584413faa0 0x7f5844156360 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f584406aad0 0x7f58440b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 --2- 172.21.15.33:0/1336102651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58440c7940 0x7f5844158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:15.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 >> 172.21.15.33:0/1336102651 conn(0x7f584406d030 msgr2=0x7f584415aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:15.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 shutdown_connections 2023-10-06T05:20:15.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:15.556+0000 7f584a59d700 1 -- 172.21.15.33:0/1336102651 wait complete. 2023-10-06T05:20:15.559 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:15.575 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:16.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:16.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.774+0000 7efc749fe700 1 Processor -- start 2023-10-06T05:20:16.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.774+0000 7efc749fe700 1 -- start start 2023-10-06T05:20:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.775+0000 7efc749fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc7006ab00 0x7efc700b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.775+0000 7efc749fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc7015e2d0 con 0x7efc700b2e30 2023-10-06T05:20:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.775+0000 7efc749fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc7006afa0 con 0x7efc700b0a40 2023-10-06T05:20:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.775+0000 7efc749fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7015d6c0 con 0x7efc7006ab00 2023-10-06T05:20:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6f59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efc700b2e30 0x7efc7015a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59058/0 (socket says 172.21.15.33:59058) 2023-10-06T05:20:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6f59e700 1 -- 172.21.15.33:0/3588372351 learned_addr learned my addr 172.21.15.33:0/3588372351 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6e59c700 1 --2- 172.21.15.33:0/3588372351 >> v2:172.21.15.167:3300/0 conn(0x7efc7006ab00 0x7efc700b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862265281 0 0) 0x7efc7015e2d0 con 0x7efc700b2e30 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc58003570 con 0x7efc700b2e30 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089837091 0 0) 0x7efc7006afa0 con 0x7efc700b0a40 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc7015e2d0 con 0x7efc700b0a40 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420570784 0 0) 0x7efc58003570 con 0x7efc700b2e30 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 >> v2:172.21.15.167:3300/0 conn(0x7efc7006ab00 msgr2=0x7efc700b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 --2- 172.21.15.33:0/3588372351 >> v2:172.21.15.167:3300/0 conn(0x7efc7006ab00 0x7efc700b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.776+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 >> v1:172.21.15.167:6789/0 conn(0x7efc700b0a40 legacy=0x7efc700b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7015d9a0 con 0x7efc700b2e30 2023-10-06T05:20:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7efc680017f0 con 0x7efc700b2e30 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 >> v1:172.21.15.33:6789/0 conn(0x7efc700b2e30 legacy=0x7efc7015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc58003bf0 0x7efc58003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58004810 0x7efc58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc58005230 0x7efc5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7015d6c0 con 0x7efc58005230 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5800c200 con 0x7efc58004810 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5800c3c0 con 0x7efc58003bf0 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6f59e700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc58005230 0x7efc5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6e59c700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58004810 0x7efc58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6ed9d700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc58003bf0 0x7efc58003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6f59e700 1 -- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc58003bf0 msgr2=0x7efc58003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6f59e700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc58003bf0 0x7efc58003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6f59e700 1 -- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58004810 msgr2=0x7efc58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6f59e700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58004810 0x7efc58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6f59e700 1 -- 172.21.15.33:0/3588372351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc58003570 con 0x7efc58005230 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.777+0000 7efc6ed9d700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc58003bf0 0x7efc58003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc6f59e700 1 -- 172.21.15.33:0/3588372351 reap_dead start 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc6f59e700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc58005230 0x7efc5800ba60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efc6800a8f0 tx=0x7efc68002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190fd29fd75b1d9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680070d0 con 0x7efc58005230 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc68007250 con 0x7efc58005230 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc6dd9b700 1 -- 172.21.15.33:0/3588372351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68007610 con 0x7efc58005230 2023-10-06T05:20:16.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc749fe700 1 -- 172.21.15.33:0/3588372351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc58005230 msgr2=0x7efc5800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.778+0000 7efc749fe700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc58005230 0x7efc5800ba60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efc6800a8f0 tx=0x7efc68002190 comp rx=0 tx=0).stop 2023-10-06T05:20:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.779+0000 7efc749fe700 1 -- 172.21.15.33:0/3588372351 shutdown_connections 2023-10-06T05:20:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.779+0000 7efc749fe700 1 --2- 172.21.15.33:0/3588372351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc58005230 0x7efc5800ba60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.779+0000 7efc749fe700 1 -- 172.21.15.33:0/3588372351 >> 172.21.15.33:0/3588372351 conn(0x7efc7006e0f0 msgr2=0x7efc700675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.779+0000 7efc749fe700 1 -- 172.21.15.33:0/3588372351 shutdown_connections 2023-10-06T05:20:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.779+0000 7efc749fe700 1 -- 172.21.15.33:0/3588372351 wait complete. 2023-10-06T05:20:16.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.780+0000 7efc749fe700 1 Processor -- start 2023-10-06T05:20:16.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.780+0000 7efc749fe700 1 -- start start 2023-10-06T05:20:16.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc749fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58005230 0x7efc701574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc749fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc700c3250 0x7efc700c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc749fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 0x7efc7015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc749fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc700c2f00 con 0x7efc700c1fd0 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc749fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc700c3ef0 con 0x7efc58005230 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc749fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc70157a10 con 0x7efc700c3250 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc6f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 0x7efc7015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc6f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 0x7efc7015c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56298/0 (socket says 172.21.15.33:56298) 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.781+0000 7efc6f59e700 1 -- 172.21.15.33:0/1979750623 learned_addr learned my addr 172.21.15.33:0/1979750623 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6f59e700 1 -- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc700c3250 msgr2=0x7efc700c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6ed9d700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58005230 0x7efc701574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6f59e700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc700c3250 0x7efc700c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6f59e700 1 -- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58005230 msgr2=0x7efc701574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6f59e700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58005230 0x7efc701574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6f59e700 1 -- 172.21.15.33:0/1979750623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc700c2940 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6ed9d700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58005230 0x7efc701574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc6f59e700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 0x7efc7015c0f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efc68000f80 tx=0x7efc68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68022070 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc680075d0 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6801d650 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc700c2c20 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.782+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc700c48c0 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.783+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc700c39d0 con 0x7efc700c1fd0 2023-10-06T05:20:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.784+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc6801da80 con 0x7efc700c1fd0 2023-10-06T05:20:16.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.785+0000 7efc56ffd700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc5c066f30 0x7efc5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:16.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.786+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7efc680866a0 con 0x7efc700c1fd0 2023-10-06T05:20:16.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.786+0000 7efc6ed9d700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc5c066f30 0x7efc5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:16.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.787+0000 7efc6ed9d700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc5c066f30 0x7efc5c0693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efc60000c00 tx=0x7efc60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:16.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:16.789+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc680536e0 con 0x7efc700c1fd0 2023-10-06T05:20:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.039+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc7006fcc0 con 0x7efc700c1fd0 2023-10-06T05:20:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.039+0000 7efc56ffd700 1 -- 172.21.15.33:0/1979750623 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7efc68056d00 con 0x7efc700c1fd0 2023-10-06T05:20:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:17.041 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc5c066f30 msgr2=0x7efc5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc5c066f30 0x7efc5c0693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efc60000c00 tx=0x7efc60009040 comp rx=0 tx=0).stop 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 msgr2=0x7efc7015c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 0x7efc7015c0f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efc68000f80 tx=0x7efc68016000 comp rx=0 tx=0).stop 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 shutdown_connections 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc5c066f30 0x7efc5c0693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc700c1fd0 0x7efc7015c0f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc58005230 0x7efc701574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 --2- 172.21.15.33:0/1979750623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc700c3250 0x7efc700c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 >> 172.21.15.33:0/1979750623 conn(0x7efc7006e0f0 msgr2=0x7efc700af340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 shutdown_connections 2023-10-06T05:20:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:17.042+0000 7efc749fe700 1 -- 172.21.15.33:0/1979750623 wait complete. 2023-10-06T05:20:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:17.062 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:18.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.260+0000 7f6d8b59e700 1 Processor -- start 2023-10-06T05:20:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.260+0000 7f6d8b59e700 1 -- start start 2023-10-06T05:20:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.261+0000 7f6d8b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d8c159530 0x7f6d8c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.261+0000 7f6d8b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c06e3e0 con 0x7f6d8c0b04e0 2023-10-06T05:20:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.261+0000 7f6d8b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c06f220 con 0x7f6d8c06df70 2023-10-06T05:20:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.261+0000 7f6d8b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0b1e00 con 0x7f6d8c159530 2023-10-06T05:20:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.261+0000 7f6d8ad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d8c0b04e0 0x7f6d8c15a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59074/0 (socket says 172.21.15.33:59074) 2023-10-06T05:20:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.261+0000 7f6d8ad9d700 1 -- 172.21.15.33:0/313528912 learned_addr learned my addr 172.21.15.33:0/313528912 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:18.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8a59c700 1 --2- 172.21.15.33:0/313528912 >> v2:172.21.15.167:3300/0 conn(0x7f6d8c159530 0x7f6d8c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327167632 0 0) 0x7f6d8c06e3e0 con 0x7f6d8c0b04e0 2023-10-06T05:20:18.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74003570 con 0x7f6d8c0b04e0 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503347662 0 0) 0x7f6d8c06f220 con 0x7f6d8c06df70 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d8c06e3e0 con 0x7f6d8c06df70 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281588775 0 0) 0x7f6d74003570 con 0x7f6d8c0b04e0 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8a59c700 1 -- 172.21.15.33:0/313528912 >> v1:172.21.15.167:6789/0 conn(0x7f6d8c06df70 legacy=0x7f6d8c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8a59c700 1 -- 172.21.15.33:0/313528912 >> v1:172.21.15.33:6789/0 conn(0x7f6d8c0b04e0 legacy=0x7f6d8c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.262+0000 7f6d8a59c700 1 -- 172.21.15.33:0/313528912 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c06e610 con 0x7f6d8c159530 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.263+0000 7f6d8a59c700 1 --2- 172.21.15.33:0/313528912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c159530 0x7f6d8c06da30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098e0 tx=0x7f6d7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62423e784d5959f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.263+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840017f0 con 0x7f6d8c159530 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.263+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c011420 con 0x7f6d8c159530 2023-10-06T05:20:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.263+0000 7f6d8959a700 1 -- 172.21.15.33:0/313528912 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0117c0 con 0x7f6d8c159530 2023-10-06T05:20:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.263+0000 7f6d8b59e700 1 -- 172.21.15.33:0/313528912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c159530 msgr2=0x7f6d8c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.263+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/313528912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c159530 0x7f6d8c06da30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098e0 tx=0x7f6d7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.264+0000 7f6d8b59e700 1 -- 172.21.15.33:0/313528912 shutdown_connections 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.264+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/313528912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c159530 0x7f6d8c06da30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.264+0000 7f6d8b59e700 1 -- 172.21.15.33:0/313528912 >> 172.21.15.33:0/313528912 conn(0x7f6d8c06c560 msgr2=0x7f6d8c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.264+0000 7f6d8b59e700 1 -- 172.21.15.33:0/313528912 shutdown_connections 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.264+0000 7f6d8b59e700 1 -- 172.21.15.33:0/313528912 wait complete. 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.265+0000 7f6d8b59e700 1 Processor -- start 2023-10-06T05:20:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.265+0000 7f6d8b59e700 1 -- start start 2023-10-06T05:20:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.266+0000 7f6d8b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c06df70 0x7f6d8c16a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.266+0000 7f6d8b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0b04e0 0x7f6d8c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.266+0000 7f6d8b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 0x7f6d8c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.266+0000 7f6d8b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1533b0 con 0x7f6d8c159530 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.266+0000 7f6d8b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c155630 con 0x7f6d8c06df70 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.266+0000 7f6d8b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c155370 con 0x7f6d8c0b04e0 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 0x7f6d8c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 0x7f6d8c14e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56314/0 (socket says 172.21.15.33:56314) 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 -- 172.21.15.33:0/2223904626 learned_addr learned my addr 172.21.15.33:0/2223904626 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8a59c700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c06df70 0x7f6d8c16a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d89d9b700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0b04e0 0x7f6d8c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 -- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0b04e0 msgr2=0x7f6d8c154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0b04e0 0x7f6d8c154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 -- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c06df70 msgr2=0x7f6d8c16a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c06df70 0x7f6d8c16a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 -- 172.21.15.33:0/2223904626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c009420 con 0x7f6d8c159530 2023-10-06T05:20:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.267+0000 7f6d8ad9d700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 0x7f6d8c14e580 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400cc80 tx=0x7f6d84007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.268+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400ce30 con 0x7f6d8c159530 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.268+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84004dc0 con 0x7f6d8c159530 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.268+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400ab70 con 0x7f6d8c159530 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.268+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c14eac0 con 0x7f6d8c159530 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.268+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c152d60 con 0x7f6d8c159530 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.269+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c04c380 con 0x7f6d8c159530 2023-10-06T05:20:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.269+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d84007150 con 0x7f6d8c159530 2023-10-06T05:20:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.271+0000 7f6d7a7fc700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d6c066f30 0x7f6d6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.271+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6d84083ec0 con 0x7f6d8c159530 2023-10-06T05:20:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.271+0000 7f6d8a59c700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d6c066f30 0x7f6d6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.272+0000 7f6d8a59c700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d6c066f30 0x7f6d6c0693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0093f0 tx=0x7f6d7c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:18.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.274+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d84050f80 con 0x7f6d8c159530 2023-10-06T05:20:18.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:18.406 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.524+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d8c157340 con 0x7f6d8c159530 2023-10-06T05:20:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.525+0000 7f6d7a7fc700 1 -- 172.21.15.33:0/2223904626 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f6d84018090 con 0x7f6d8c159530 2023-10-06T05:20:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:18.527 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.527+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d6c066f30 msgr2=0x7f6d6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.527+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d6c066f30 0x7f6d6c0693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0093f0 tx=0x7f6d7c016000 comp rx=0 tx=0).stop 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.527+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 msgr2=0x7f6d8c14e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.527+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 0x7f6d8c14e580 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400cc80 tx=0x7f6d84007d20 comp rx=0 tx=0).stop 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 shutdown_connections 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d6c066f30 0x7f6d6c0693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c159530 0x7f6d8c14e580 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c06df70 0x7f6d8c16a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 --2- 172.21.15.33:0/2223904626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0b04e0 0x7f6d8c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 >> 172.21.15.33:0/2223904626 conn(0x7f6d8c06c560 msgr2=0x7f6d8c155f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 shutdown_connections 2023-10-06T05:20:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:18.528+0000 7f6d8b59e700 1 -- 172.21.15.33:0/2223904626 wait complete. 2023-10-06T05:20:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:18.547 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:19.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.745+0000 7fd06e79d700 1 Processor -- start 2023-10-06T05:20:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.745+0000 7fd06e79d700 1 -- start start 2023-10-06T05:20:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 0x7fd0680c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06e79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0680c1600 0x7fd0680c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06e79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06815abd0 con 0x7fd068153b10 2023-10-06T05:20:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06e79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0680c74f0 con 0x7fd0680c1600 2023-10-06T05:20:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06815a050 con 0x7fd0680c6b30 2023-10-06T05:20:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06d79b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd068153b10 0x7fd0680c95c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59084/0 (socket says 172.21.15.33:59084) 2023-10-06T05:20:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06d79b700 1 -- 172.21.15.33:0/1826427267 learned_addr learned my addr 172.21.15.33:0/1826427267 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06df9c700 1 --2- 172.21.15.33:0/1826427267 >> v2:172.21.15.167:3300/0 conn(0x7fd0680c1600 0x7fd0680c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd06cf9a700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 0x7fd0680c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.747+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644155382 0 0) 0x7fd06815abd0 con 0x7fd068153b10 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd054003570 con 0x7fd068153b10 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132283891 0 0) 0x7fd054003570 con 0x7fd068153b10 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 msgr2=0x7fd0680c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 0x7fd0680c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 >> v2:172.21.15.167:3300/0 conn(0x7fd0680c1600 msgr2=0x7fd0680c1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 --2- 172.21.15.33:0/1826427267 >> v2:172.21.15.167:3300/0 conn(0x7fd0680c1600 0x7fd0680c1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06815a330 con 0x7fd068153b10 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fd064001470 con 0x7fd068153b10 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 >> v1:172.21.15.33:6789/0 conn(0x7fd068153b10 legacy=0x7fd0680c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 0x7fd054003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540040b0 0x7fd054004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd054004d50 0x7fd0540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680c74f0 con 0x7fd054004d50 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06815a050 con 0x7fd0540040b0 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05400bb60 con 0x7fd0680c6b30 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd06df9c700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd054004d50 0x7fd0540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd06d79b700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 0x7fd054003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.748+0000 7fd06cf9a700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540040b0 0x7fd054004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd06df9c700 1 -- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 msgr2=0x7fd054003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd06df9c700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0680c6b30 0x7fd054003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd06df9c700 1 -- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540040b0 msgr2=0x7fd054004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd06df9c700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0540040b0 0x7fd054004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd06df9c700 1 -- 172.21.15.33:0/1826427267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054003570 con 0x7fd054004d50 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd06df9c700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd054004d50 0x7fd0540051c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd0600094d0 tx=0x7fd06000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2132f054ac35b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000ea90 con 0x7fd054004d50 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06000f040 con 0x7fd054004d50 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.749+0000 7fd05ffff700 1 -- 172.21.15.33:0/1826427267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000ee20 con 0x7fd054004d50 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06e79d700 1 -- 172.21.15.33:0/1826427267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd054004d50 msgr2=0x7fd0540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06e79d700 1 --2- 172.21.15.33:0/1826427267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd054004d50 0x7fd0540051c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd0600094d0 tx=0x7fd06000b040 comp rx=0 tx=0).stop 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06df9c700 1 -- 172.21.15.33:0/1826427267 reap_dead start 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06e79d700 1 -- 172.21.15.33:0/1826427267 shutdown_connections 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06e79d700 1 -- 172.21.15.33:0/1826427267 >> 172.21.15.33:0/1826427267 conn(0x7fd06806c560 msgr2=0x7fd06806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06e79d700 1 -- 172.21.15.33:0/1826427267 shutdown_connections 2023-10-06T05:20:19.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.750+0000 7fd06e79d700 1 -- 172.21.15.33:0/1826427267 wait complete. 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.751+0000 7fd06e79d700 1 Processor -- start 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.752+0000 7fd06e79d700 1 -- start start 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.752+0000 7fd06e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd054004d50 0x7fd068138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.752+0000 7fd06e79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 0x7fd06813d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.752+0000 7fd06e79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06813cc60 0x7fd06813f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06e79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06813dd50 con 0x7fd068169a50 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06e79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680b0e90 con 0x7fd06813cc60 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd068139200 con 0x7fd054004d50 2023-10-06T05:20:19.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 0x7fd06813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 0x7fd06813d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56340/0 (socket says 172.21.15.33:56340) 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 -- 172.21.15.33:0/77563541 learned_addr learned my addr 172.21.15.33:0/77563541 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06d79b700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd054004d50 0x7fd068138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06df9c700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06813cc60 0x7fd06813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 -- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd054004d50 msgr2=0x7fd068138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd054004d50 0x7fd068138cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 -- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06813cc60 msgr2=0x7fd06813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06813cc60 0x7fd06813f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 -- 172.21.15.33:0/77563541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680b0560 con 0x7fd068169a50 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06d79b700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd054004d50 0x7fd068138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.753+0000 7fd06cf9a700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 0x7fd06813d500 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd058006ac0 tx=0x7fd0580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.754+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800fa10 con 0x7fd068169a50 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.754+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05800fb70 con 0x7fd068169a50 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.754+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0680b0840 con 0x7fd068169a50 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.754+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058014440 con 0x7fd068169a50 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.754+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06813da40 con 0x7fd068169a50 2023-10-06T05:20:19.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.755+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06804c380 con 0x7fd068169a50 2023-10-06T05:20:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.756+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd058014870 con 0x7fd068169a50 2023-10-06T05:20:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.756+0000 7fd05d7fa700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd04c066f30 0x7fd04c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.757+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd058018030 con 0x7fd068169a50 2023-10-06T05:20:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.757+0000 7fd06d79b700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd04c066f30 0x7fd04c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.758+0000 7fd06d79b700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd04c066f30 0x7fd04c0693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd064000e10 tx=0x7fd064009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:19.761+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0580522a0 con 0x7fd068169a50 2023-10-06T05:20:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.008+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0680c9d00 con 0x7fd068169a50 2023-10-06T05:20:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.009+0000 7fd05d7fa700 1 -- 172.21.15.33:0/77563541 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd0580558c0 con 0x7fd068169a50 2023-10-06T05:20:20.010 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:20.011 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd04c066f30 msgr2=0x7fd04c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd04c066f30 0x7fd04c0693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd064000e10 tx=0x7fd064009040 comp rx=0 tx=0).stop 2023-10-06T05:20:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 msgr2=0x7fd06813d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 0x7fd06813d500 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd058006ac0 tx=0x7fd0580091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 shutdown_connections 2023-10-06T05:20:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd04c066f30 0x7fd04c0693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06813cc60 0x7fd06813f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd054004d50 0x7fd068138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 --2- 172.21.15.33:0/77563541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd068169a50 0x7fd06813d500 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 >> 172.21.15.33:0/77563541 conn(0x7fd06806c560 msgr2=0x7fd0680cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 shutdown_connections 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:20.011+0000 7fd06e79d700 1 -- 172.21.15.33:0/77563541 wait complete. 2023-10-06T05:20:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:20.031 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:21.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.229+0000 7fb2ce59d700 1 Processor -- start 2023-10-06T05:20:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.229+0000 7fb2ce59d700 1 -- start start 2023-10-06T05:20:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.230+0000 7fb2ce59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb2c8159650 0x7fb2c80c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.230+0000 7fb2ce59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80c92d0 con 0x7fb2c8159a20 2023-10-06T05:20:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.230+0000 7fb2ce59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80c86c0 con 0x7fb2c80ca590 2023-10-06T05:20:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.230+0000 7fb2ce59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c80c89a0 con 0x7fb2c8159650 2023-10-06T05:20:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2ccd9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb2c8159a20 0x7fb2c80c9e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32958/0 (socket says 172.21.15.33:32958) 2023-10-06T05:20:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2ccd9a700 1 -- 172.21.15.33:0/2557140427 learned_addr learned my addr 172.21.15.33:0/2557140427 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2cd59b700 1 --2- 172.21.15.33:0/2557140427 >> v2:172.21.15.167:3300/0 conn(0x7fb2c8159650 0x7fb2c80c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589885314 0 0) 0x7fb2c80c92d0 con 0x7fb2c8159a20 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b4003570 con 0x7fb2c8159a20 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203471615 0 0) 0x7fb2c80c86c0 con 0x7fb2c80ca590 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c80c92d0 con 0x7fb2c80ca590 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2cd59b700 1 -- 172.21.15.33:0/2557140427 >> v1:172.21.15.167:6789/0 conn(0x7fb2c80ca590 legacy=0x7fb2c80a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2cd59b700 1 -- 172.21.15.33:0/2557140427 >> v1:172.21.15.33:6789/0 conn(0x7fb2c8159a20 legacy=0x7fb2c80c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.231+0000 7fb2cd59b700 1 -- 172.21.15.33:0/2557140427 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80c8c80 con 0x7fb2c8159650 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.232+0000 7fb2cd59b700 1 --2- 172.21.15.33:0/2557140427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80c8030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40098e0 tx=0x7fb2c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50288c11eba183e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.232+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4011630 con 0x7fb2c8159650 2023-10-06T05:20:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.232+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4011790 con 0x7fb2c8159650 2023-10-06T05:20:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.232+0000 7fb2bffff700 1 -- 172.21.15.33:0/2557140427 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4011b30 con 0x7fb2c8159650 2023-10-06T05:20:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2557140427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 msgr2=0x7fb2c80c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2557140427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80c8030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40098e0 tx=0x7fb2c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2557140427 shutdown_connections 2023-10-06T05:20:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2557140427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80c8030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2557140427 >> 172.21.15.33:0/2557140427 conn(0x7fb2c806d030 msgr2=0x7fb2c806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2557140427 shutdown_connections 2023-10-06T05:20:21.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.233+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2557140427 wait complete. 2023-10-06T05:20:21.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.234+0000 7fb2ce59d700 1 Processor -- start 2023-10-06T05:20:21.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.234+0000 7fb2ce59d700 1 -- start start 2023-10-06T05:20:21.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.235+0000 7fb2ce59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2c80ca590 0x7fb2c806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.235+0000 7fb2ce59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.235+0000 7fb2ce59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 0x7fb2c80abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.235+0000 7fb2ce59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80c51c0 con 0x7fb2c8159a20 2023-10-06T05:20:21.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.235+0000 7fb2ce59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80c4540 con 0x7fb2c8159650 2023-10-06T05:20:21.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.235+0000 7fb2ce59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80c4820 con 0x7fb2c80ca590 2023-10-06T05:20:21.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 0x7fb2c80abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 0x7fb2c80abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39474/0 (socket says 172.21.15.33:39474) 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 -- 172.21.15.33:0/2569329470 learned_addr learned my addr 172.21.15.33:0/2569329470 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cd59b700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2c80ca590 0x7fb2c806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2ccd9a700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 -- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2c80ca590 msgr2=0x7fb2c806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2c80ca590 0x7fb2c806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 -- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 msgr2=0x7fb2c80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 -- 172.21.15.33:0/2569329470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4009420 con 0x7fb2c8159a20 2023-10-06T05:20:21.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.236+0000 7fb2cdd9c700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 0x7fb2c80abac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c910 tx=0x7fb2c0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.237+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012630 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.237+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c00127b0 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.237+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c80c4b00 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.237+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012b70 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.237+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c806b610 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.238+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c804c380 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.239+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2c001b400 con 0x7fb2c8159a20 2023-10-06T05:20:21.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.239+0000 7fb2bd7fa700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2ac066f30 0x7fb2ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.240+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb2c0084930 con 0x7fb2c8159a20 2023-10-06T05:20:21.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.240+0000 7fb2cd59b700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2ac066f30 0x7fb2ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.241+0000 7fb2cd59b700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2ac066f30 0x7fb2ac0693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40093f0 tx=0x7fb2c4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.244+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2c00519f0 con 0x7fb2c8159a20 2023-10-06T05:20:21.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.495+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2c80c8f60 con 0x7fb2c8159a20 2023-10-06T05:20:21.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.496+0000 7fb2bd7fa700 1 -- 172.21.15.33:0/2569329470 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fb2c0016020 con 0x7fb2c8159a20 2023-10-06T05:20:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:21.498 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2ac066f30 msgr2=0x7fb2ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2ac066f30 0x7fb2ac0693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40093f0 tx=0x7fb2c4017000 comp rx=0 tx=0).stop 2023-10-06T05:20:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 msgr2=0x7fb2c80abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 0x7fb2c80abac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c910 tx=0x7fb2c0007f90 comp rx=0 tx=0).stop 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 shutdown_connections 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2ac066f30 0x7fb2ac0693e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2c8159a20 0x7fb2c80abac0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2c80ca590 0x7fb2c806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 --2- 172.21.15.33:0/2569329470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c8159650 0x7fb2c80b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.498+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 >> 172.21.15.33:0/2569329470 conn(0x7fb2c806d030 msgr2=0x7fb2c8153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.499+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 shutdown_connections 2023-10-06T05:20:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:21.499+0000 7fb2ce59d700 1 -- 172.21.15.33:0/2569329470 wait complete. 2023-10-06T05:20:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:21.518 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:22.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.716+0000 7f851eb9d700 1 Processor -- start 2023-10-06T05:20:22.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.716+0000 7f851eb9d700 1 -- start start 2023-10-06T05:20:22.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.717+0000 7f851eb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f851815a0a0 0x7f851815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.717+0000 7f851eb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851815e2d0 con 0x7f85180b0a40 2023-10-06T05:20:22.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.717+0000 7f851eb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851806afa0 con 0x7f851806ab00 2023-10-06T05:20:22.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.717+0000 7f851eb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851815d6c0 con 0x7f851815a0a0 2023-10-06T05:20:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.717+0000 7f851db9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f85180b0a40 0x7f85180b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32970/0 (socket says 172.21.15.33:32970) 2023-10-06T05:20:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851db9b700 1 -- 172.21.15.33:0/1876581236 learned_addr learned my addr 172.21.15.33:0/1876581236 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851e39c700 1 --2- 172.21.15.33:0/1876581236 >> v2:172.21.15.167:3300/0 conn(0x7f851815a0a0 0x7f851815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614214871 0 0) 0x7f851815e2d0 con 0x7f85180b0a40 2023-10-06T05:20:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003570 con 0x7f85180b0a40 2023-10-06T05:20:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714318528 0 0) 0x7f8504003570 con 0x7f85180b0a40 2023-10-06T05:20:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 >> v2:172.21.15.167:3300/0 conn(0x7f851815a0a0 msgr2=0x7f851815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 --2- 172.21.15.33:0/1876581236 >> v2:172.21.15.167:3300/0 conn(0x7f851815a0a0 0x7f851815a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 >> v1:172.21.15.167:6789/0 conn(0x7f851806ab00 legacy=0x7f85180b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851815d9a0 con 0x7f85180b0a40 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f8514001470 con 0x7f85180b0a40 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 >> v1:172.21.15.33:6789/0 conn(0x7f85180b0a40 legacy=0x7f85180b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8504003ab0 0x7f8504003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504004810 0x7f8504004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.718+0000 7f851cb99700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504005210 0x7f850400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851815d6c0 con 0x7f8504003ab0 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850400c1b0 con 0x7f8504005210 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400c370 con 0x7f8504004810 2023-10-06T05:20:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8504003ab0 0x7f8504003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851e39c700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504005210 0x7f850400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851d39a700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504004810 0x7f8504004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 -- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504004810 msgr2=0x7f8504004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504004810 0x7f8504004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 -- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504005210 msgr2=0x7f850400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504005210 0x7f850400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851e39c700 1 -- 172.21.15.33:0/1876581236 reap_dead start 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 -- 172.21.15.33:0/1876581236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504003570 con 0x7f8504003ab0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.719+0000 7f851db9b700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8504003ab0 0x7f8504003ec0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f851400c2d0 tx=0x7f8514001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af313b95d5bbbc41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400cb20 con 0x7f8504003ab0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400cca0 con 0x7f8504003ab0 2023-10-06T05:20:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851cb99700 1 -- 172.21.15.33:0/1876581236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514005b70 con 0x7f8504003ab0 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 -- 172.21.15.33:0/1876581236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8504003ab0 msgr2=0x7f8504003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8504003ab0 0x7f8504003ec0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f851400c2d0 tx=0x7f8514001d80 comp rx=0 tx=0).stop 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 -- 172.21.15.33:0/1876581236 shutdown_connections 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 --2- 172.21.15.33:0/1876581236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8504003ab0 0x7f8504003ec0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 -- 172.21.15.33:0/1876581236 >> 172.21.15.33:0/1876581236 conn(0x7f851806e0f0 msgr2=0x7f85180675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 -- 172.21.15.33:0/1876581236 shutdown_connections 2023-10-06T05:20:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.720+0000 7f851eb9d700 1 -- 172.21.15.33:0/1876581236 wait complete. 2023-10-06T05:20:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.722+0000 7f851eb9d700 1 Processor -- start 2023-10-06T05:20:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.722+0000 7f851eb9d700 1 -- start start 2023-10-06T05:20:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504003ab0 0x7f85180af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 0x7f85180ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518151710 0x7f85180abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180ad3a0 con 0x7f85181520e0 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518152650 con 0x7f8518151710 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180aca70 con 0x7f8504003ab0 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 0x7f85180ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 0x7f85180ab7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39484/0 (socket says 172.21.15.33:39484) 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 -- 172.21.15.33:0/4062449367 learned_addr learned my addr 172.21.15.33:0/4062449367 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851db9b700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504003ab0 0x7f85180af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851e39c700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518151710 0x7f85180abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 -- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504003ab0 msgr2=0x7f85180af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504003ab0 0x7f85180af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 -- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518151710 msgr2=0x7f85180abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518151710 0x7f85180abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.723+0000 7f851d39a700 1 -- 172.21.15.33:0/4062449367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180acd50 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.724+0000 7f851d39a700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 0x7f85180ab7e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f850800e590 tx=0x7f8508009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.724+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508012660 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.724+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080127c0 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.724+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180ad030 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.724+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508012b80 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.724+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85180ad830 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.725+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851806fcc0 con 0x7f85181520e0 2023-10-06T05:20:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.726+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8508013420 con 0x7f85181520e0 2023-10-06T05:20:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.727+0000 7f850dffb700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f84fc066f30 0x7f84fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.727+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f85080846e0 con 0x7f85181520e0 2023-10-06T05:20:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.727+0000 7f851db9b700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f84fc066f30 0x7f84fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.728+0000 7f851db9b700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f84fc066f30 0x7f84fc0693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8514007c00 tx=0x7f8514015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.731+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8508051720 con 0x7f85181520e0 2023-10-06T05:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.979+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f851804c380 con 0x7f85181520e0 2023-10-06T05:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.979+0000 7f850dffb700 1 -- 172.21.15.33:0/4062449367 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f8508054d40 con 0x7f85181520e0 2023-10-06T05:20:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:22.981 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f84fc066f30 msgr2=0x7f84fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f84fc066f30 0x7f84fc0693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8514007c00 tx=0x7f8514015000 comp rx=0 tx=0).stop 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 msgr2=0x7f85180ab7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 0x7f85180ab7e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f850800e590 tx=0x7f8508009040 comp rx=0 tx=0).stop 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 shutdown_connections 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f84fc066f30 0x7f84fc0693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518151710 0x7f85180abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8504003ab0 0x7f85180af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 --2- 172.21.15.33:0/4062449367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85181520e0 0x7f85180ab7e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 >> 172.21.15.33:0/4062449367 conn(0x7f851806e0f0 msgr2=0x7f851806b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 shutdown_connections 2023-10-06T05:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:22.982+0000 7f851eb9d700 1 -- 172.21.15.33:0/4062449367 wait complete. 2023-10-06T05:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:23.002 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:24.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:24.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.201+0000 7faba219d700 1 Processor -- start 2023-10-06T05:20:24.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.201+0000 7faba219d700 1 -- start start 2023-10-06T05:20:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba219d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fab9c0682b0 0x7fab9c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba219d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c06c770 con 0x7fab9c06dc10 2023-10-06T05:20:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba219d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c06a820 con 0x7fab9c1557a0 2023-10-06T05:20:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba219d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c06ab00 con 0x7fab9c0682b0 2023-10-06T05:20:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba199c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab9c06dc10 0x7fab9c155b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32974/0 (socket says 172.21.15.33:32974) 2023-10-06T05:20:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba199c700 1 -- 172.21.15.33:0/4152688983 learned_addr learned my addr 172.21.15.33:0/4152688983 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:24.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.203+0000 7faba099a700 1 --2- 172.21.15.33:0/4152688983 >> v2:172.21.15.167:3300/0 conn(0x7fab9c0682b0 0x7fab9c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951499206 0 0) 0x7fab9c06c770 con 0x7fab9c06dc10 2023-10-06T05:20:24.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab88003570 con 0x7fab9c06dc10 2023-10-06T05:20:24.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264936826 0 0) 0x7fab9c06a820 con 0x7fab9c1557a0 2023-10-06T05:20:24.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9c06c770 con 0x7fab9c1557a0 2023-10-06T05:20:24.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376240426 0 0) 0x7fab88003570 con 0x7fab9c06dc10 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 >> v2:172.21.15.167:3300/0 conn(0x7fab9c0682b0 msgr2=0x7fab9c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 --2- 172.21.15.33:0/4152688983 >> v2:172.21.15.167:3300/0 conn(0x7fab9c0682b0 0x7fab9c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 >> v1:172.21.15.167:6789/0 conn(0x7fab9c1557a0 legacy=0x7fab9c067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c06ade0 con 0x7fab9c06dc10 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fab940017f0 con 0x7fab9c06dc10 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 >> v1:172.21.15.33:6789/0 conn(0x7fab9c06dc10 legacy=0x7fab9c155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88003c40 0x7fab88004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88004810 0x7fab88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab88005210 0x7fab8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c06ab00 con 0x7fab88003c40 2023-10-06T05:20:24.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800c1b0 con 0x7fab88004810 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.204+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800c370 con 0x7fab88005210 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88003c40 0x7fab88004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba099a700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88004810 0x7fab88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba199c700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab88005210 0x7fab8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 -- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab88005210 msgr2=0x7fab8800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab88005210 0x7fab8800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 -- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88004810 msgr2=0x7fab88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88004810 0x7fab88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 -- 172.21.15.33:0/4152688983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88003570 con 0x7fab88003c40 2023-10-06T05:20:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba199c700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab88005210 0x7fab8800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba199c700 1 -- 172.21.15.33:0/4152688983 reap_dead start 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7faba119b700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88003c40 0x7fab88004030 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fab980019e0 tx=0x7fab98007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1433febaf261a96e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800c5d0 con 0x7fab88003c40 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9800c730 con 0x7fab88003c40 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.205+0000 7fab93fff700 1 -- 172.21.15.33:0/4152688983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800cad0 con 0x7fab88003c40 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 -- 172.21.15.33:0/4152688983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88003c40 msgr2=0x7fab88004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88003c40 0x7fab88004030 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fab980019e0 tx=0x7fab98007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 -- 172.21.15.33:0/4152688983 shutdown_connections 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 --2- 172.21.15.33:0/4152688983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88003c40 0x7fab88004030 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 -- 172.21.15.33:0/4152688983 >> 172.21.15.33:0/4152688983 conn(0x7fab9c0cc930 msgr2=0x7fab9c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 -- 172.21.15.33:0/4152688983 shutdown_connections 2023-10-06T05:20:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.206+0000 7faba219d700 1 -- 172.21.15.33:0/4152688983 wait complete. 2023-10-06T05:20:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.208+0000 7faba219d700 1 Processor -- start 2023-10-06T05:20:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.208+0000 7faba219d700 1 -- start start 2023-10-06T05:20:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88003c40 0x7fab9c0abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9c1582c0 0x7fab9c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 0x7fab9c16a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c158c70 con 0x7fab9c16a2d0 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c151840 con 0x7fab88003c40 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1523c0 con 0x7fab9c1582c0 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 0x7fab9c16a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 0x7fab9c16a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39496/0 (socket says 172.21.15.33:39496) 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 -- 172.21.15.33:0/4049713297 learned_addr learned my addr 172.21.15.33:0/4049713297 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba099a700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9c1582c0 0x7fab9c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba119b700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88003c40 0x7fab9c0abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 -- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9c1582c0 msgr2=0x7fab9c158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9c1582c0 0x7fab9c158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 -- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88003c40 msgr2=0x7fab9c0abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88003c40 0x7fab9c0abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.209+0000 7faba199c700 1 -- 172.21.15.33:0/4049713297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c152850 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.210+0000 7faba199c700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 0x7fab9c16a770 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fab9c06e080 tx=0x7fab940013e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.210+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94017070 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.210+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab94007ad0 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.210+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c151a90 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.210+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400ed40 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.210+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c152050 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.211+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c04c380 con 0x7fab9c16a2d0 2023-10-06T05:20:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.212+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab94013630 con 0x7fab9c16a2d0 2023-10-06T05:20:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.213+0000 7fab917fa700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab80066f30 0x7fab800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.213+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fab940843a0 con 0x7fab9c16a2d0 2023-10-06T05:20:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.213+0000 7faba119b700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab80066f30 0x7fab800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.214+0000 7faba119b700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab80066f30 0x7fab800693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0cb800 tx=0x7fab98015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.217+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab940513e0 con 0x7fab9c16a2d0 2023-10-06T05:20:24.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.466+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab9c152500 con 0x7fab9c16a2d0 2023-10-06T05:20:24.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.466+0000 7fab917fa700 1 -- 172.21.15.33:0/4049713297 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fab9400eea0 con 0x7fab9c16a2d0 2023-10-06T05:20:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:24.468 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab80066f30 msgr2=0x7fab800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab80066f30 0x7fab800693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0cb800 tx=0x7fab98015000 comp rx=0 tx=0).stop 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 msgr2=0x7fab9c16a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 0x7fab9c16a770 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fab9c06e080 tx=0x7fab940013e0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 shutdown_connections 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab80066f30 0x7fab800693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab9c16a2d0 0x7fab9c16a770 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab88003c40 0x7fab9c0abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 --2- 172.21.15.33:0/4049713297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab9c1582c0 0x7fab9c158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 >> 172.21.15.33:0/4049713297 conn(0x7fab9c0cc930 msgr2=0x7fab9c0ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 shutdown_connections 2023-10-06T05:20:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:24.469+0000 7faba219d700 1 -- 172.21.15.33:0/4049713297 wait complete. 2023-10-06T05:20:24.472 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:24.488 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:25.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:25.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.686+0000 7fc74a19d700 1 Processor -- start 2023-10-06T05:20:25.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.687+0000 7fc74a19d700 1 -- start start 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7440c8b50 0x7fc7440c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74a19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc744155ec0 0x7fc7440c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74a19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74406ab60 con 0x7fc744155af0 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74406b7e0 con 0x7fc7440c8b50 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74a19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74406ae40 con 0x7fc744155ec0 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7440c8b50 0x7fc7440c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7440c8b50 0x7fc7440c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39502/0 (socket says 172.21.15.33:39502) 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74899a700 1 -- 172.21.15.33:0/457978970 learned_addr learned my addr 172.21.15.33:0/457978970 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74999c700 1 --2- 172.21.15.33:0/457978970 >> v2:172.21.15.167:3300/0 conn(0x7fc744155ec0 0x7fc7440c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74899a700 1 -- 172.21.15.33:0/457978970 >> v2:172.21.15.167:3300/0 conn(0x7fc744155ec0 msgr2=0x7fc7440c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74899a700 1 --2- 172.21.15.33:0/457978970 >> v2:172.21.15.167:3300/0 conn(0x7fc744155ec0 0x7fc7440c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.688+0000 7fc74899a700 1 -- 172.21.15.33:0/457978970 >> v1:172.21.15.167:6789/0 conn(0x7fc744155af0 legacy=0x7fc74406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.689+0000 7fc74899a700 1 -- 172.21.15.33:0/457978970 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74406b120 con 0x7fc7440c8b50 2023-10-06T05:20:25.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.689+0000 7fc74899a700 1 --2- 172.21.15.33:0/457978970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7440c8b50 0x7fc7440c7210 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc74406fdf0 tx=0x7fc734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91936c5c8513091b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.689+0000 7fc73bfff700 1 -- 172.21.15.33:0/457978970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73400b820 con 0x7fc7440c8b50 2023-10-06T05:20:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.689+0000 7fc73bfff700 1 -- 172.21.15.33:0/457978970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73400b980 con 0x7fc7440c8b50 2023-10-06T05:20:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.689+0000 7fc73bfff700 1 -- 172.21.15.33:0/457978970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73400bd20 con 0x7fc7440c8b50 2023-10-06T05:20:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 -- 172.21.15.33:0/457978970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7440c8b50 msgr2=0x7fc7440c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 --2- 172.21.15.33:0/457978970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7440c8b50 0x7fc7440c7210 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc74406fdf0 tx=0x7fc734009250 comp rx=0 tx=0).stop 2023-10-06T05:20:25.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 -- 172.21.15.33:0/457978970 shutdown_connections 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 --2- 172.21.15.33:0/457978970 >> v2:172.21.15.167:3300/0 conn(0x7fc744155ec0 0x7fc7440c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 --2- 172.21.15.33:0/457978970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7440c8b50 0x7fc7440c7210 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 -- 172.21.15.33:0/457978970 >> 172.21.15.33:0/457978970 conn(0x7fc74406d030 msgr2=0x7fc74406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 -- 172.21.15.33:0/457978970 shutdown_connections 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.690+0000 7fc74a19d700 1 -- 172.21.15.33:0/457978970 wait complete. 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.691+0000 7fc74a19d700 1 Processor -- start 2023-10-06T05:20:25.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.691+0000 7fc74a19d700 1 -- start start 2023-10-06T05:20:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.692+0000 7fc74a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7440c8b50 0x7fc744152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.692+0000 7fc74a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 0x7fc7440ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.692+0000 7fc74a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744155ec0 0x7fc744153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440c5f70 con 0x7fc744155af0 2023-10-06T05:20:25.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441583c0 con 0x7fc744155ec0 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441577b0 con 0x7fc7440c8b50 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 0x7fc7440ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 0x7fc7440ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39516/0 (socket says 172.21.15.33:39516) 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 -- 172.21.15.33:0/826677175 learned_addr learned my addr 172.21.15.33:0/826677175 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74919b700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7440c8b50 0x7fc744152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74999c700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744155ec0 0x7fc744153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 -- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7440c8b50 msgr2=0x7fc744152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7440c8b50 0x7fc744152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 -- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744155ec0 msgr2=0x7fc744153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744155ec0 0x7fc744153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 -- 172.21.15.33:0/826677175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340094f0 con 0x7fc744155af0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74999c700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744155ec0 0x7fc744153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.693+0000 7fc74899a700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 0x7fc7440ab090 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc7340030c0 tx=0x7fc734016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.694+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734011e60 con 0x7fc744155af0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.694+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73401b680 con 0x7fc744155af0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.694+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73401ba20 con 0x7fc744155af0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.694+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc744157a90 con 0x7fc744155af0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.694+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc744158050 con 0x7fc744155af0 2023-10-06T05:20:25.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.695+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74406e7a0 con 0x7fc744155af0 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.696+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc73401c450 con 0x7fc744155af0 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.696+0000 7fc7397fa700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc728066f30 0x7fc7280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.697+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc734086320 con 0x7fc744155af0 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.697+0000 7fc74919b700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc728066f30 0x7fc7280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.698+0000 7fc74919b700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc728066f30 0x7fc7280693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc740003870 tx=0x7fc740009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:25.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.701+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7340533e0 con 0x7fc744155af0 2023-10-06T05:20:25.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.949+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc74404c380 con 0x7fc744155af0 2023-10-06T05:20:25.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.949+0000 7fc7397fa700 1 -- 172.21.15.33:0/826677175 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fc73402a090 con 0x7fc744155af0 2023-10-06T05:20:25.951 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:25.951 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc728066f30 msgr2=0x7fc7280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc728066f30 0x7fc7280693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc740003870 tx=0x7fc740009040 comp rx=0 tx=0).stop 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 msgr2=0x7fc7440ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 0x7fc7440ab090 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc7340030c0 tx=0x7fc734016000 comp rx=0 tx=0).stop 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 shutdown_connections 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc728066f30 0x7fc7280693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744155ec0 0x7fc744153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7440c8b50 0x7fc744152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 --2- 172.21.15.33:0/826677175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc744155af0 0x7fc7440ab090 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 >> 172.21.15.33:0/826677175 conn(0x7fc74406d030 msgr2=0x7fc7440c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 shutdown_connections 2023-10-06T05:20:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:25.952+0000 7fc74a19d700 1 -- 172.21.15.33:0/826677175 wait complete. 2023-10-06T05:20:25.955 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:25.973 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:26.974 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.171+0000 7f9bbb59e700 1 Processor -- start 2023-10-06T05:20:27.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.172+0000 7f9bbb59e700 1 -- start start 2023-10-06T05:20:27.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc13faa0 0x7f9bbc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9bbc0c7940 0x7f9bbc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bbc06aad0 0x7f9bbc06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0c9cb0 con 0x7f9bbc06aad0 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0ca270 con 0x7f9bbc0c7940 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc06b840 con 0x7f9bbc13faa0 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bbc06aad0 0x7f9bbc06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9bbc06aad0 0x7f9bbc06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39520/0 (socket says 172.21.15.33:39520) 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bbad9d700 1 -- 172.21.15.33:0/1892756121 learned_addr learned my addr 172.21.15.33:0/1892756121 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bba59c700 1 --2- 172.21.15.33:0/1892756121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc13faa0 0x7f9bbc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.173+0000 7f9bb9d9b700 1 --2- 172.21.15.33:0/1892756121 >> v2:172.21.15.167:3300/0 conn(0x7f9bbc0c7940 0x7f9bbc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bbad9d700 1 -- 172.21.15.33:0/1892756121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc13faa0 msgr2=0x7f9bbc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:27.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bbad9d700 1 --2- 172.21.15.33:0/1892756121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc13faa0 0x7f9bbc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bbad9d700 1 -- 172.21.15.33:0/1892756121 >> v2:172.21.15.167:3300/0 conn(0x7f9bbc0c7940 msgr2=0x7f9bbc0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bbad9d700 1 --2- 172.21.15.33:0/1892756121 >> v2:172.21.15.167:3300/0 conn(0x7f9bbc0c7940 0x7f9bbc0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bbad9d700 1 -- 172.21.15.33:0/1892756121 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc06bb20 con 0x7f9bbc06aad0 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bbad9d700 1 --2- 172.21.15.33:0/1892756121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc06aad0 0x7f9bbc06af50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0636b0 tx=0x7f9bb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca879bb96fe6505b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bb959a700 1 -- 172.21.15.33:0/1892756121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400e070 con 0x7f9bbc06aad0 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bb959a700 1 -- 172.21.15.33:0/1892756121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb400b970 con 0x7f9bbc06aad0 2023-10-06T05:20:27.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.174+0000 7f9bb959a700 1 -- 172.21.15.33:0/1892756121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40092e0 con 0x7f9bbc06aad0 2023-10-06T05:20:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 -- 172.21.15.33:0/1892756121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc06aad0 msgr2=0x7f9bbc06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/1892756121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc06aad0 0x7f9bbc06af50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0636b0 tx=0x7f9bb4009840 comp rx=0 tx=0).stop 2023-10-06T05:20:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 -- 172.21.15.33:0/1892756121 shutdown_connections 2023-10-06T05:20:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/1892756121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc06aad0 0x7f9bbc06af50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/1892756121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc13faa0 0x7f9bbc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/1892756121 >> v2:172.21.15.167:3300/0 conn(0x7f9bbc0c7940 0x7f9bbc0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 -- 172.21.15.33:0/1892756121 >> 172.21.15.33:0/1892756121 conn(0x7f9bbc06d030 msgr2=0x7f9bbc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:27.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 -- 172.21.15.33:0/1892756121 shutdown_connections 2023-10-06T05:20:27.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.175+0000 7f9bbb59e700 1 -- 172.21.15.33:0/1892756121 wait complete. 2023-10-06T05:20:27.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.176+0000 7f9bbb59e700 1 Processor -- start 2023-10-06T05:20:27.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.176+0000 7f9bbb59e700 1 -- start start 2023-10-06T05:20:27.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.177+0000 7f9bbb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc06aad0 0x7f9bbc1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.177+0000 7f9bbb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 0x7f9bbc136840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.177+0000 7f9bbb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bbc13faa0 0x7f9bbc0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.177+0000 7f9bbb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc137290 con 0x7f9bbc0c7940 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.177+0000 7f9bbb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc137570 con 0x7f9bbc13faa0 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.177+0000 7f9bbb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc137850 con 0x7f9bbc06aad0 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 0x7f9bbc136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 0x7f9bbc136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39528/0 (socket says 172.21.15.33:39528) 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 -- 172.21.15.33:0/667322952 learned_addr learned my addr 172.21.15.33:0/667322952 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 -- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc06aad0 msgr2=0x7f9bbc1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc06aad0 0x7f9bbc1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 -- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bbc13faa0 msgr2=0x7f9bbc0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bbc13faa0 0x7f9bbc0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 -- 172.21.15.33:0/667322952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4009ae0 con 0x7f9bbc0c7940 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bb9d9b700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 0x7f9bbc136840 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000ec80 tx=0x7f9bb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000c9d0 con 0x7f9bbc0c7940 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb000cb30 con 0x7f9bbc0c7940 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0013470 con 0x7f9bbc0c7940 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.178+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc137b30 con 0x7f9bbc0c7940 2023-10-06T05:20:27.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.179+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc155ef0 con 0x7f9bbc0c7940 2023-10-06T05:20:27.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.180+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc04c380 con 0x7f9bbc0c7940 2023-10-06T05:20:27.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.180+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bb0013820 con 0x7f9bbc0c7940 2023-10-06T05:20:27.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.181+0000 7f9baa7fc700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b9c066f30 0x7f9b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.181+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9bb0084af0 con 0x7f9bbc0c7940 2023-10-06T05:20:27.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.182+0000 7f9bba59c700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b9c066f30 0x7f9b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.182+0000 7f9bba59c700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b9c066f30 0x7f9b9c0693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009b10 tx=0x7f9bac0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:27.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.185+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bb0051b30 con 0x7f9bbc0c7940 2023-10-06T05:20:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.433+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bbc0c9f00 con 0x7f9bbc0c7940 2023-10-06T05:20:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.433+0000 7f9baa7fc700 1 -- 172.21.15.33:0/667322952 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f9bb0055150 con 0x7f9bbc0c7940 2023-10-06T05:20:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:27.435 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.435+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b9c066f30 msgr2=0x7f9b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.435+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b9c066f30 0x7f9b9c0693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009b10 tx=0x7f9bac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.435+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 msgr2=0x7f9bbc136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.435+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 0x7f9bbc136840 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000ec80 tx=0x7f9bb00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 shutdown_connections 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9b9c066f30 0x7f9b9c0693e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bbc13faa0 0x7f9bbc0cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bbc06aad0 0x7f9bbc1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 --2- 172.21.15.33:0/667322952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc0c7940 0x7f9bbc136840 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 >> 172.21.15.33:0/667322952 conn(0x7f9bbc06d030 msgr2=0x7f9bbc0c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 shutdown_connections 2023-10-06T05:20:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:27.436+0000 7f9bbb59e700 1 -- 172.21.15.33:0/667322952 wait complete. 2023-10-06T05:20:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:27.455 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:28.456 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.654+0000 7f241f59e700 1 Processor -- start 2023-10-06T05:20:28.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.654+0000 7f241f59e700 1 -- start start 2023-10-06T05:20:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.655+0000 7f241f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f24200c8050 0x7f242013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.655+0000 7f241f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.655+0000 7f241f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24200c9cb0 con 0x7f242013faa0 2023-10-06T05:20:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.655+0000 7f241f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24200ca270 con 0x7f24200c8050 2023-10-06T05:20:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.655+0000 7f241f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242006b840 con 0x7f242006aad0 2023-10-06T05:20:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f24200c8050 0x7f242013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f24200c8050 0x7f242013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39538/0 (socket says 172.21.15.33:39538) 2023-10-06T05:20:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 -- 172.21.15.33:0/3504613076 learned_addr learned my addr 172.21.15.33:0/3504613076 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:28.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241ed9d700 1 --2- 172.21.15.33:0/3504613076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 -- 172.21.15.33:0/3504613076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 msgr2=0x7f242006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:28.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 --2- 172.21.15.33:0/3504613076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242006af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 -- 172.21.15.33:0/3504613076 >> v1:172.21.15.167:6789/0 conn(0x7f242013faa0 legacy=0x7f24200c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.656+0000 7f241dd9b700 1 -- 172.21.15.33:0/3504613076 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242006bb20 con 0x7f24200c8050 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.657+0000 7f241dd9b700 1 --2- 172.21.15.33:0/3504613076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24200c8050 0x7f242013fe70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f241400bc50 tx=0x7f241400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d13e05a538deb9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.657+0000 7f241d59a700 1 -- 172.21.15.33:0/3504613076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400dbf0 con 0x7f24200c8050 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.657+0000 7f241d59a700 1 -- 172.21.15.33:0/3504613076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400dd50 con 0x7f24200c8050 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.657+0000 7f241d59a700 1 -- 172.21.15.33:0/3504613076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140196b0 con 0x7f24200c8050 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.657+0000 7f241f59e700 1 -- 172.21.15.33:0/3504613076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24200c8050 msgr2=0x7f242013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.657+0000 7f241f59e700 1 --2- 172.21.15.33:0/3504613076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24200c8050 0x7f242013fe70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f241400bc50 tx=0x7f241400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.658+0000 7f241f59e700 1 -- 172.21.15.33:0/3504613076 shutdown_connections 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.658+0000 7f241f59e700 1 --2- 172.21.15.33:0/3504613076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242006af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.658+0000 7f241f59e700 1 --2- 172.21.15.33:0/3504613076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24200c8050 0x7f242013fe70 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.658+0000 7f241f59e700 1 -- 172.21.15.33:0/3504613076 >> 172.21.15.33:0/3504613076 conn(0x7f242006d030 msgr2=0x7f242006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.658+0000 7f241f59e700 1 -- 172.21.15.33:0/3504613076 shutdown_connections 2023-10-06T05:20:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.658+0000 7f241f59e700 1 -- 172.21.15.33:0/3504613076 wait complete. 2023-10-06T05:20:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.659+0000 7f241f59e700 1 Processor -- start 2023-10-06T05:20:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.659+0000 7f241f59e700 1 -- start start 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200c8050 0x7f242013e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 0x7f242015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420156cf0 con 0x7f242013faa0 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242015daa0 con 0x7f24200c8050 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242015dd40 con 0x7f242006aad0 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.660+0000 7f241ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 0x7f242015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 0x7f242015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39542/0 (socket says 172.21.15.33:39542) 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 -- 172.21.15.33:0/2465843656 learned_addr learned my addr 172.21.15.33:0/2465843656 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241dd9b700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200c8050 0x7f242013e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241e59c700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 -- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 msgr2=0x7f242015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 -- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200c8050 msgr2=0x7f242013e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200c8050 0x7f242013e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 -- 172.21.15.33:0/2465843656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241400b790 con 0x7f242013faa0 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.661+0000 7f241ed9d700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 0x7f242015f2a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2418009e90 tx=0x7f241800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.662+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800cb70 con 0x7f242013faa0 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.662+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800f040 con 0x7f242013faa0 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.662+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418014480 con 0x7f242013faa0 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.662+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242015e020 con 0x7f242013faa0 2023-10-06T05:20:28.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.662+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242015e5e0 con 0x7f242013faa0 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.663+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2420156e60 con 0x7f242013faa0 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.664+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2418014830 con 0x7f242013faa0 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.665+0000 7f240e7fc700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2400066f30 0x7f24000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.665+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f2418085130 con 0x7f242013faa0 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.665+0000 7f241e59c700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2400066f30 0x7f24000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.666+0000 7f241e59c700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2400066f30 0x7f24000693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2410003950 tx=0x7f2410007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.669+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2418052170 con 0x7f242013faa0 2023-10-06T05:20:28.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.917+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f242004c380 con 0x7f242013faa0 2023-10-06T05:20:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.918+0000 7f240e7fc700 1 -- 172.21.15.33:0/2465843656 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f2418055790 con 0x7f242013faa0 2023-10-06T05:20:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:28.920 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2400066f30 msgr2=0x7f24000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2400066f30 0x7f24000693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2410003950 tx=0x7f2410007ec0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 msgr2=0x7f242015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 0x7f242015f2a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2418009e90 tx=0x7f241800b040 comp rx=0 tx=0).stop 2023-10-06T05:20:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 shutdown_connections 2023-10-06T05:20:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2400066f30 0x7f24000693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f242013faa0 0x7f242015f2a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242006aad0 0x7f242015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 --2- 172.21.15.33:0/2465843656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24200c8050 0x7f242013e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 >> 172.21.15.33:0/2465843656 conn(0x7f242006d030 msgr2=0x7f24200c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 shutdown_connections 2023-10-06T05:20:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:28.920+0000 7f241f59e700 1 -- 172.21.15.33:0/2465843656 wait complete. 2023-10-06T05:20:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:28.939 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:29.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.138+0000 7fdd0359e700 1 Processor -- start 2023-10-06T05:20:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.138+0000 7fdd0359e700 1 -- start start 2023-10-06T05:20:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.139+0000 7fdd0359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd040cf3b0 0x7fdd041594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.139+0000 7fdd0359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd040cbd80 con 0x7fdd040ccfc0 2023-10-06T05:20:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.139+0000 7fdd0359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd040af5f0 con 0x7fdd040af0a0 2023-10-06T05:20:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.139+0000 7fdd0359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040af8d0 con 0x7fdd040cf3b0 2023-10-06T05:20:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdd040ccfc0 0x7fdd040cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58248/0 (socket says 172.21.15.33:58248) 2023-10-06T05:20:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0259c700 1 -- 172.21.15.33:0/574675150 learned_addr learned my addr 172.21.15.33:0/574675150 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd01d9b700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd040cf3b0 0x7fdd041594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009386807 0 0) 0x7fdd040cbd80 con 0x7fdd040ccfc0 2023-10-06T05:20:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcec003570 con 0x7fdd040ccfc0 2023-10-06T05:20:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 986620040 0 0) 0x7fdcec003570 con 0x7fdd040ccfc0 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd040cf3b0 msgr2=0x7fdd041594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd040cf3b0 0x7fdd041594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 >> v1:172.21.15.167:6789/0 conn(0x7fdd040af0a0 legacy=0x7fdd040b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040afbb0 con 0x7fdd040ccfc0 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fdcf4001470 con 0x7fdd040ccfc0 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 >> v1:172.21.15.33:6789/0 conn(0x7fdd040ccfc0 legacy=0x7fdd040cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.140+0000 7fdd0159a700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdcec003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0159a700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcec004810 0x7fdcec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0159a700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcec0051c0 0x7fdcec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040af8d0 con 0x7fdcec003a90 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec00bfa0 con 0x7fdcec004810 2023-10-06T05:20:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec00c110 con 0x7fdcec0051c0 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdcec003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd01d9b700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcec004810 0x7fdcec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd02d9d700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcec0051c0 0x7fdcec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 -- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcec0051c0 msgr2=0x7fdcec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcec0051c0 0x7fdcec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 -- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcec004810 msgr2=0x7fdcec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcec004810 0x7fdcec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 -- 172.21.15.33:0/574675150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec003570 con 0x7fdcec003a90 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd02d9d700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcec0051c0 0x7fdcec00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.141+0000 7fdd0259c700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdcec003ea0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c3c0 tx=0x7fdcf4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798a86fd0520410c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400cc90 con 0x7fdcec003a90 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400cdf0 con 0x7fdcec003a90 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0159a700 1 -- 172.21.15.33:0/574675150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4005b50 con 0x7fdcec003a90 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0359e700 1 -- 172.21.15.33:0/574675150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 msgr2=0x7fdcec003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0359e700 1 --2- 172.21.15.33:0/574675150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdcec003ea0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c3c0 tx=0x7fdcf4001e50 comp rx=0 tx=0).stop 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd02d9d700 1 -- 172.21.15.33:0/574675150 reap_dead start 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0359e700 1 -- 172.21.15.33:0/574675150 shutdown_connections 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0359e700 1 -- 172.21.15.33:0/574675150 >> 172.21.15.33:0/574675150 conn(0x7fdd0406cff0 msgr2=0x7fdd0406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0359e700 1 -- 172.21.15.33:0/574675150 shutdown_connections 2023-10-06T05:20:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.142+0000 7fdd0359e700 1 -- 172.21.15.33:0/574675150 wait complete. 2023-10-06T05:20:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.144+0000 7fdd0359e700 1 Processor -- start 2023-10-06T05:20:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.144+0000 7fdd0359e700 1 -- start start 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdd0414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd0414c1a0 0x7fdd040c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd04148350 0x7fdd040c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0414c610 con 0x7fdcec003a90 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040ca430 con 0x7fdd04148350 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0414c810 con 0x7fdd0414c1a0 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdd0414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdd0414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53696/0 (socket says 172.21.15.33:53696) 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd0259c700 1 -- 172.21.15.33:0/156296341 learned_addr learned my addr 172.21.15.33:0/156296341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd01d9b700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd0414c1a0 0x7fdd040c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.145+0000 7fdd02d9d700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd04148350 0x7fdd040c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0259c700 1 -- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd0414c1a0 msgr2=0x7fdd040c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0259c700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd0414c1a0 0x7fdd040c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0259c700 1 -- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd04148350 msgr2=0x7fdd040c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0259c700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd04148350 0x7fdd040c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0259c700 1 -- 172.21.15.33:0/156296341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0414caf0 con 0x7fdcec003a90 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd01d9b700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd0414c1a0 0x7fdd040c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd02d9d700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd04148350 0x7fdd040c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0259c700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdd0414d8b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4001ed0 tx=0x7fdcf4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40070d0 con 0x7fdcec003a90 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4007230 con 0x7fdcec003a90 2023-10-06T05:20:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0414cdd0 con 0x7fdcec003a90 2023-10-06T05:20:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40075f0 con 0x7fdcec003a90 2023-10-06T05:20:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.146+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0414d390 con 0x7fdcec003a90 2023-10-06T05:20:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.147+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd040cd630 con 0x7fdcec003a90 2023-10-06T05:20:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.148+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcf4019030 con 0x7fdcec003a90 2023-10-06T05:20:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.149+0000 7fdcf27fc700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce4066f30 0x7fdce40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.149+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fdcf4085de0 con 0x7fdcec003a90 2023-10-06T05:20:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.150+0000 7fdd01d9b700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce4066f30 0x7fdce40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.150+0000 7fdd01d9b700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce4066f30 0x7fdce40693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006d60 tx=0x7fdcf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.154+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcf4052ea0 con 0x7fdcec003a90 2023-10-06T05:20:30.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.401+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd0414d700 con 0x7fdcec003a90 2023-10-06T05:20:30.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.402+0000 7fdcf27fc700 1 -- 172.21.15.33:0/156296341 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fdcf40564c0 con 0x7fdcec003a90 2023-10-06T05:20:30.406 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:30.407 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.404+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce4066f30 msgr2=0x7fdce40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.404+0000 7fdd0359e700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce4066f30 0x7fdce40693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006d60 tx=0x7fdcf8008040 comp rx=0 tx=0).stop 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.404+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 msgr2=0x7fdd0414d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.404+0000 7fdd0359e700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdd0414d8b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4001ed0 tx=0x7fdcf4015000 comp rx=0 tx=0).stop 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 shutdown_connections 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce4066f30 0x7fdce40693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd04148350 0x7fdd040c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcec003a90 0x7fdd0414d8b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 --2- 172.21.15.33:0/156296341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd0414c1a0 0x7fdd040c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 >> 172.21.15.33:0/156296341 conn(0x7fdd0406cff0 msgr2=0x7fdd04157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 shutdown_connections 2023-10-06T05:20:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:30.405+0000 7fdd0359e700 1 -- 172.21.15.33:0/156296341 wait complete. 2023-10-06T05:20:30.409 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:30.424 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:31.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:31.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.624+0000 7f1b2b39d700 1 Processor -- start 2023-10-06T05:20:31.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.624+0000 7f1b2b39d700 1 -- start start 2023-10-06T05:20:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.625+0000 7f1b2b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240ccfc0 0x7f1b240cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.625+0000 7f1b2b39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1b240cf3b0 0x7f1b241594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.625+0000 7f1b2b39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b240cc750 0x7f1b241599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.625+0000 7f1b2b39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240cbd80 con 0x7f1b240cc750 2023-10-06T05:20:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.625+0000 7f1b2b39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240af5f0 con 0x7f1b240cf3b0 2023-10-06T05:20:31.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.625+0000 7f1b2b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240af8d0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b240cc750 0x7f1b241599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b240cc750 0x7f1b241599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53700/0 (socket says 172.21.15.33:53700) 2023-10-06T05:20:31.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 -- 172.21.15.33:0/1793980983 learned_addr learned my addr 172.21.15.33:0/1793980983 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:31.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2a39b700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240ccfc0 0x7f1b240cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b29b9a700 1 --2- 172.21.15.33:0/1793980983 >> v2:172.21.15.167:3300/0 conn(0x7f1b240cf3b0 0x7f1b241594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 -- 172.21.15.33:0/1793980983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240ccfc0 msgr2=0x7f1b240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240ccfc0 0x7f1b240cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 -- 172.21.15.33:0/1793980983 >> v2:172.21.15.167:3300/0 conn(0x7f1b240cf3b0 msgr2=0x7f1b241594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 --2- 172.21.15.33:0/1793980983 >> v2:172.21.15.167:3300/0 conn(0x7f1b240cf3b0 0x7f1b241594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 -- 172.21.15.33:0/1793980983 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240afbb0 con 0x7f1b240cc750 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2a39b700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240ccfc0 0x7f1b240cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.626+0000 7f1b2ab9c700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240cc750 0x7f1b241599e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1b24071260 tx=0x7f1b1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc13ff9719b899f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b29399700 1 -- 172.21.15.33:0/1793980983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c00e070 con 0x7f1b240cc750 2023-10-06T05:20:31.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b29399700 1 -- 172.21.15.33:0/1793980983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c00ba20 con 0x7f1b240cc750 2023-10-06T05:20:31.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b29399700 1 -- 172.21.15.33:0/1793980983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0092e0 con 0x7f1b240cc750 2023-10-06T05:20:31.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 -- 172.21.15.33:0/1793980983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240cc750 msgr2=0x7f1b241599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240cc750 0x7f1b241599e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1b24071260 tx=0x7f1b1c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:31.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 -- 172.21.15.33:0/1793980983 shutdown_connections 2023-10-06T05:20:31.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240cc750 0x7f1b241599e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/1793980983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240ccfc0 0x7f1b240cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/1793980983 >> v2:172.21.15.167:3300/0 conn(0x7f1b240cf3b0 0x7f1b241594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 -- 172.21.15.33:0/1793980983 >> 172.21.15.33:0/1793980983 conn(0x7f1b2406cff0 msgr2=0x7f1b2406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:31.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 -- 172.21.15.33:0/1793980983 shutdown_connections 2023-10-06T05:20:31.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.627+0000 7f1b2b39d700 1 -- 172.21.15.33:0/1793980983 wait complete. 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.629+0000 7f1b2b39d700 1 Processor -- start 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.629+0000 7f1b2b39d700 1 -- start start 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240cc750 0x7f1b24153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 0x7f1b240b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240b41a0 0x7f1b240b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b4620 con 0x7f1b240ccfc0 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24169a30 con 0x7f1b240cc750 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240c9190 con 0x7f1b240b41a0 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b29b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 0x7f1b240b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b29b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 0x7f1b240b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53712/0 (socket says 172.21.15.33:53712) 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b29b9a700 1 -- 172.21.15.33:0/2872039473 learned_addr learned my addr 172.21.15.33:0/2872039473 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2a39b700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240cc750 0x7f1b24153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.630+0000 7f1b2ab9c700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240b41a0 0x7f1b240b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b29b9a700 1 -- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240b41a0 msgr2=0x7f1b240b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b29b9a700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240b41a0 0x7f1b240b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b29b9a700 1 -- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240cc750 msgr2=0x7f1b24153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b29b9a700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240cc750 0x7f1b24153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b29b9a700 1 -- 172.21.15.33:0/2872039473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c009ae0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b29b9a700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 0x7f1b240b3c60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400e900 tx=0x7f1b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14013680 con 0x7f1b240ccfc0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b140137e0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b240b4820 con 0x7f1b240ccfc0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14013ba0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.631+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b240b4de0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.633+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b240b4fd0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.633+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b140123d0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.634+0000 7f1b1a7fc700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.634+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1b140845d0 con 0x7f1b240ccfc0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.635+0000 7f1b2a39b700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.635+0000 7f1b2a39b700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1b08066f30 0x7f1b080693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b20006f00 tx=0x7f1b20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.639+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b14051610 con 0x7f1b240ccfc0 2023-10-06T05:20:31.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.884+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b2404c380 con 0x7f1b240ccfc0 2023-10-06T05:20:31.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.885+0000 7f1b1a7fc700 1 -- 172.21.15.33:0/2872039473 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f1b14054c30 con 0x7f1b240ccfc0 2023-10-06T05:20:31.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:31.887 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1b08066f30 msgr2=0x7f1b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1b08066f30 0x7f1b080693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b20006f00 tx=0x7f1b20008040 comp rx=0 tx=0).stop 2023-10-06T05:20:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 msgr2=0x7f1b240b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 0x7f1b240b3c60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400e900 tx=0x7f1b140091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 shutdown_connections 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b240b41a0 0x7f1b240b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b240cc750 0x7f1b24153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 --2- 172.21.15.33:0/2872039473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b240ccfc0 0x7f1b240b3c60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 >> 172.21.15.33:0/2872039473 conn(0x7f1b2406cff0 msgr2=0x7f1b24155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.887+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 shutdown_connections 2023-10-06T05:20:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:31.888+0000 7f1b2b39d700 1 -- 172.21.15.33:0/2872039473 wait complete. 2023-10-06T05:20:31.891 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:31.907 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:32.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.106+0000 7f727759e700 1 Processor -- start 2023-10-06T05:20:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.106+0000 7f727759e700 1 -- start start 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f727759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 0x7f727813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f727759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f727806aad0 0x7f727806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f727759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72780c9cb0 con 0x7f72780c7940 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f727759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72780ca270 con 0x7f727806aad0 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f727759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727806b840 con 0x7f727813faa0 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f7275d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f72780c7940 0x7f72780c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58252/0 (socket says 172.21.15.33:58252) 2023-10-06T05:20:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.107+0000 7f7275d9b700 1 -- 172.21.15.33:0/1505148750 learned_addr learned my addr 172.21.15.33:0/1505148750 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727659c700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 0x7f727813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f7276d9d700 1 --2- 172.21.15.33:0/1505148750 >> v2:172.21.15.167:3300/0 conn(0x7f727806aad0 0x7f727806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612785698 0 0) 0x7f72780c9cb0 con 0x7f72780c7940 2023-10-06T05:20:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7260003570 con 0x7f72780c7940 2023-10-06T05:20:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3529600805 0 0) 0x7f7260003570 con 0x7f72780c7940 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 msgr2=0x7f727813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 0x7f727813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 >> v2:172.21.15.167:3300/0 conn(0x7f727806aad0 msgr2=0x7f727806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 --2- 172.21.15.33:0/1505148750 >> v2:172.21.15.167:3300/0 conn(0x7f727806aad0 0x7f727806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727806bb20 con 0x7f72780c7940 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f726c001470 con 0x7f72780c7940 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f7276d9d700 1 --2- 172.21.15.33:0/1505148750 >> v2:172.21.15.167:3300/0 conn(0x7f727806aad0 0x7f727806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 >> v1:172.21.15.33:6789/0 conn(0x7f72780c7940 legacy=0x7f72780c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7260003a60 0x7f7260003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 0x7f7260004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260004d50 0x7f72600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780ca270 con 0x7f7260003a60 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727806b840 con 0x7f7260004d50 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.108+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726000dc70 con 0x7f727813faa0 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7260003a60 0x7f7260003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f7275d9b700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 0x7f7260004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f7276d9d700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260004d50 0x7f72600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 -- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 msgr2=0x7f7260004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f727813faa0 0x7f7260004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 -- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260004d50 msgr2=0x7f72600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260004d50 0x7f72600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 -- 172.21.15.33:0/1505148750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7260003570 con 0x7f7260003a60 2023-10-06T05:20:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.109+0000 7f727659c700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7260003a60 0x7f7260003e70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7268009c00 tx=0x7f72680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1793635e3c746ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800e6b0 con 0x7f7260003a60 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726800e810 con 0x7f7260003a60 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727559a700 1 -- 172.21.15.33:0/1505148750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800ebd0 con 0x7f7260003a60 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727759e700 1 -- 172.21.15.33:0/1505148750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7260003a60 msgr2=0x7f7260003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727759e700 1 --2- 172.21.15.33:0/1505148750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7260003a60 0x7f7260003e70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7268009c00 tx=0x7f72680091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f7276d9d700 1 -- 172.21.15.33:0/1505148750 reap_dead start 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727759e700 1 -- 172.21.15.33:0/1505148750 shutdown_connections 2023-10-06T05:20:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727759e700 1 -- 172.21.15.33:0/1505148750 >> 172.21.15.33:0/1505148750 conn(0x7f727806d030 msgr2=0x7f727806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727759e700 1 -- 172.21.15.33:0/1505148750 shutdown_connections 2023-10-06T05:20:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.110+0000 7f727759e700 1 -- 172.21.15.33:0/1505148750 wait complete. 2023-10-06T05:20:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.112+0000 7f727759e700 1 Processor -- start 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.112+0000 7f727759e700 1 -- start start 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.113+0000 7f727759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260003a60 0x7f72780a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.113+0000 7f727759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 0x7f72780a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.113+0000 7f727759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72780a7fc0 0x7f72780a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.113+0000 7f727759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727815d930 con 0x7f727815c5b0 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.113+0000 7f727759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727815cd20 con 0x7f7260003a60 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.113+0000 7f727759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727815d000 con 0x7f72780a7fc0 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 0x7f72780a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 0x7f72780a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53732/0 (socket says 172.21.15.33:53732) 2023-10-06T05:20:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 -- 172.21.15.33:0/3656110414 learned_addr learned my addr 172.21.15.33:0/3656110414 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f727659c700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260003a60 0x7f72780a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7276d9d700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72780a7fc0 0x7f72780a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 -- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72780a7fc0 msgr2=0x7f72780a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72780a7fc0 0x7f72780a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 -- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260003a60 msgr2=0x7f72780a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260003a60 0x7f72780a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 -- 172.21.15.33:0/3656110414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727815d2e0 con 0x7f727815c5b0 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7276d9d700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72780a7fc0 0x7f72780a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f7275d9b700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 0x7f72780a7a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f726c001050 tx=0x7f727806be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c018030 con 0x7f727815c5b0 2023-10-06T05:20:33.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c00f8a0 con 0x7f727815c5b0 2023-10-06T05:20:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.114+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727815d5c0 con 0x7f727815c5b0 2023-10-06T05:20:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.115+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00fc40 con 0x7f727815c5b0 2023-10-06T05:20:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.115+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727815ddc0 con 0x7f727815c5b0 2023-10-06T05:20:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.116+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727815dfe0 con 0x7f727815c5b0 2023-10-06T05:20:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.117+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f726c01b070 con 0x7f727815c5b0 2023-10-06T05:20:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.117+0000 7f72667fc700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7258066f30 0x7f72580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.117+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f726c019070 con 0x7f727815c5b0 2023-10-06T05:20:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.118+0000 7f727659c700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7258066f30 0x7f72580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.119+0000 7f727659c700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7258066f30 0x7f72580693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f72680091e0 tx=0x7f726801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.121+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f726c051820 con 0x7f727815c5b0 2023-10-06T05:20:33.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.369+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f727804c380 con 0x7f727815c5b0 2023-10-06T05:20:33.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.369+0000 7f72667fc700 1 -- 172.21.15.33:0/3656110414 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f726c054e40 con 0x7f727815c5b0 2023-10-06T05:20:33.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:33.371 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:33.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7258066f30 msgr2=0x7f72580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7258066f30 0x7f72580693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f72680091e0 tx=0x7f726801a000 comp rx=0 tx=0).stop 2023-10-06T05:20:33.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 msgr2=0x7f72780a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 0x7f72780a7a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f726c001050 tx=0x7f727806be00 comp rx=0 tx=0).stop 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 shutdown_connections 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7258066f30 0x7f72580693e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72780a7fc0 0x7f72780a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7260003a60 0x7f72780a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 --2- 172.21.15.33:0/3656110414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727815c5b0 0x7f72780a7a80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.371+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 >> 172.21.15.33:0/3656110414 conn(0x7f727806d030 msgr2=0x7f72781595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.372+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 shutdown_connections 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:33.372+0000 7f727759e700 1 -- 172.21.15.33:0/3656110414 wait complete. 2023-10-06T05:20:33.374 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:33.391 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:34.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.588+0000 7f5d2bfff700 1 Processor -- start 2023-10-06T05:20:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.589+0000 7f5d2bfff700 1 -- start start 2023-10-06T05:20:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d2c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c06ab60 con 0x7f5d2c0c7920 2023-10-06T05:20:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c06ae40 con 0x7f5d2c06a6a0 2023-10-06T05:20:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c06b7e0 con 0x7f5d2c155af0 2023-10-06T05:20:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5d2c0c7920 0x7f5d2c0c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58260/0 (socket says 172.21.15.33:58260) 2023-10-06T05:20:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2b7fe700 1 -- 172.21.15.33:0/3057394754 learned_addr learned my addr 172.21.15.33:0/3057394754 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.590+0000 7f5d2affd700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d2c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854069872 0 0) 0x7f5d2c06ab60 con 0x7f5d2c0c7920 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d14004680 con 0x7f5d2c0c7920 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877708642 0 0) 0x7f5d2c06ae40 con 0x7f5d2c06a6a0 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c06ab60 con 0x7f5d2c06a6a0 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 196000012 0 0) 0x7f5d14004680 con 0x7f5d2c0c7920 2023-10-06T05:20:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 msgr2=0x7f5d2c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d2c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 >> v1:172.21.15.167:6789/0 conn(0x7f5d2c06a6a0 legacy=0x7f5d2c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c06b120 con 0x7f5d2c0c7920 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f5d240017f0 con 0x7f5d2c0c7920 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 >> v1:172.21.15.33:6789/0 conn(0x7f5d2c0c7920 legacy=0x7f5d2c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d14004d30 0x7f5d14005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d14005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d14005e60 0x7f5d140062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c06b7e0 con 0x7f5d14004d30 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14003640 con 0x7f5d14005e60 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14003800 con 0x7f5d2c155af0 2023-10-06T05:20:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.591+0000 7f5d2affd700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d14004d30 0x7f5d14005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2b7fe700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d14005e60 0x7f5d140062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2a7fc700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d14005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2affd700 1 -- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 msgr2=0x7f5d14005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2affd700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d14005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2affd700 1 -- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d14005e60 msgr2=0x7f5d140062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2affd700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d14005e60 0x7f5d140062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2affd700 1 -- 172.21.15.33:0/3057394754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d14004680 con 0x7f5d14004d30 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2b7fe700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d14005e60 0x7f5d140062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2a7fc700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d2c155af0 0x7f5d14005920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d2affd700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d14004d30 0x7f5d14005120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009c00 tx=0x7f5d1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65511d44cb6e3378 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00e710 con 0x7f5d14004d30 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00e870 con 0x7f5d14004d30 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.592+0000 7f5d29ffb700 1 -- 172.21.15.33:0/3057394754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00ec30 con 0x7f5d14004d30 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2bfff700 1 -- 172.21.15.33:0/3057394754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d14004d30 msgr2=0x7f5d14005120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/3057394754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d14004d30 0x7f5d14005120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009c00 tx=0x7f5d1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2b7fe700 1 -- 172.21.15.33:0/3057394754 reap_dead start 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2bfff700 1 -- 172.21.15.33:0/3057394754 shutdown_connections 2023-10-06T05:20:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2bfff700 1 -- 172.21.15.33:0/3057394754 >> 172.21.15.33:0/3057394754 conn(0x7f5d2c06d030 msgr2=0x7f5d2c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2bfff700 1 -- 172.21.15.33:0/3057394754 shutdown_connections 2023-10-06T05:20:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.593+0000 7f5d2bfff700 1 -- 172.21.15.33:0/3057394754 wait complete. 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.594+0000 7f5d2bfff700 1 Processor -- start 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.595+0000 7f5d2bfff700 1 -- start start 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.595+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d14004d30 0x7f5d2c149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.595+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 0x7f5d2c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.595+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0af4f0 0x7f5d2c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.595+0000 7f5d2bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c158180 con 0x7f5d2c158460 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b0770 con 0x7f5d2c0af4f0 2023-10-06T05:20:34.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c157cf0 con 0x7f5d14004d30 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 0x7f5d2c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 0x7f5d2c0c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53742/0 (socket says 172.21.15.33:53742) 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 -- 172.21.15.33:0/2472082396 learned_addr learned my addr 172.21.15.33:0/2472082396 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2affd700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d14004d30 0x7f5d2c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2b7fe700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0af4f0 0x7f5d2c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 -- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d14004d30 msgr2=0x7f5d2c149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d14004d30 0x7f5d2c149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 -- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0af4f0 msgr2=0x7f5d2c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0af4f0 0x7f5d2c0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 -- 172.21.15.33:0/2472082396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0b01b0 con 0x7f5d2c158460 2023-10-06T05:20:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.596+0000 7f5d2a7fc700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 0x7f5d2c0c6f80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e810 tx=0x7f5d20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.597+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200026a0 con 0x7f5d2c158460 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.597+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d20002800 con 0x7f5d2c158460 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.597+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20002ba0 con 0x7f5d2c158460 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.597+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c0b0490 con 0x7f5d2c158460 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.597+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c0afeb0 con 0x7f5d2c158460 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.598+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c04c380 con 0x7f5d2c158460 2023-10-06T05:20:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.599+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d20014480 con 0x7f5d2c158460 2023-10-06T05:20:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.600+0000 7f5d12ffd700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d18066f30 0x7f5d180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.601+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5d20084680 con 0x7f5d2c158460 2023-10-06T05:20:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.601+0000 7f5d2affd700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d18066f30 0x7f5d180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.602+0000 7f5d2affd700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d18066f30 0x7f5d180693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0091e0 tx=0x7f5d1c01a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.605+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d2004d5a0 con 0x7f5d2c158460 2023-10-06T05:20:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.853+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d2c157e30 con 0x7f5d2c158460 2023-10-06T05:20:34.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.854+0000 7f5d12ffd700 1 -- 172.21.15.33:0/2472082396 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f5d20051770 con 0x7f5d2c158460 2023-10-06T05:20:34.855 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:34.855 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:19:45.853905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d18066f30 msgr2=0x7f5d180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d18066f30 0x7f5d180693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0091e0 tx=0x7f5d1c01a000 comp rx=0 tx=0).stop 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 msgr2=0x7f5d2c0c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 0x7f5d2c0c6f80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e810 tx=0x7f5d20009040 comp rx=0 tx=0).stop 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 shutdown_connections 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d18066f30 0x7f5d180693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2c0af4f0 0x7f5d2c0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d14004d30 0x7f5d2c149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 --2- 172.21.15.33:0/2472082396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c158460 0x7f5d2c0c6f80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 >> 172.21.15.33:0/2472082396 conn(0x7f5d2c06d030 msgr2=0x7f5d2c0c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 shutdown_connections 2023-10-06T05:20:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:34.856+0000 7f5d2bfff700 1 -- 172.21.15.33:0/2472082396 wait complete. 2023-10-06T05:20:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2023-10-06T05:20:34.875 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:20:35.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:36.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.074+0000 7fe33919d700 1 Processor -- start 2023-10-06T05:20:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.074+0000 7fe33919d700 1 -- start start 2023-10-06T05:20:36.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.075+0000 7fe33919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe334155af0 0x7fe33406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.075+0000 7fe33919d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe33406a6a0 0x7fe3340c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.075+0000 7fe33919d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33406ab60 con 0x7fe3340c8b50 2023-10-06T05:20:36.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.075+0000 7fe33919d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33406b7e0 con 0x7fe33406a6a0 2023-10-06T05:20:36.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.075+0000 7fe33919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33406ae40 con 0x7fe334155af0 2023-10-06T05:20:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe33899c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe3340c8b50 0x7fe3341598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58274/0 (socket says 172.21.15.33:58274) 2023-10-06T05:20:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe33899c700 1 -- 172.21.15.33:0/2078025934 learned_addr learned my addr 172.21.15.33:0/2078025934 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe333fff700 1 --2- 172.21.15.33:0/2078025934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe334155af0 0x7fe33406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe3337fe700 1 --2- 172.21.15.33:0/2078025934 >> v2:172.21.15.167:3300/0 conn(0x7fe33406a6a0 0x7fe3340c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe332ffd700 1 -- 172.21.15.33:0/2078025934 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55641585 0 0) 0x7fe33406ab60 con 0x7fe3340c8b50 2023-10-06T05:20:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe332ffd700 1 -- 172.21.15.33:0/2078025934 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe31c003570 con 0x7fe3340c8b50 2023-10-06T05:20:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe333fff700 1 -- 172.21.15.33:0/2078025934 >> v2:172.21.15.167:3300/0 conn(0x7fe33406a6a0 msgr2=0x7fe3340c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe333fff700 1 --2- 172.21.15.33:0/2078025934 >> v2:172.21.15.167:3300/0 conn(0x7fe33406a6a0 0x7fe3340c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe333fff700 1 -- 172.21.15.33:0/2078025934 >> v1:172.21.15.33:6789/0 conn(0x7fe3340c8b50 legacy=0x7fe3341598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.076+0000 7fe333fff700 1 -- 172.21.15.33:0/2078025934 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33406b120 con 0x7fe334155af0 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.077+0000 7fe333fff700 1 --2- 172.21.15.33:0/2078025934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe334155af0 0x7fe33406a160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe32400aa90 tx=0x7fe32400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ea2a0c2935ebd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.077+0000 7fe332ffd700 1 -- 172.21.15.33:0/2078025934 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32400ae70 con 0x7fe334155af0 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.077+0000 7fe332ffd700 1 -- 172.21.15.33:0/2078025934 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32400c970 con 0x7fe334155af0 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.077+0000 7fe332ffd700 1 -- 172.21.15.33:0/2078025934 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32400cd10 con 0x7fe334155af0 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.077+0000 7fe33919d700 1 -- 172.21.15.33:0/2078025934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe334155af0 msgr2=0x7fe33406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.077+0000 7fe33919d700 1 --2- 172.21.15.33:0/2078025934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe334155af0 0x7fe33406a160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe32400aa90 tx=0x7fe32400a330 comp rx=0 tx=0).stop 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.078+0000 7fe33919d700 1 -- 172.21.15.33:0/2078025934 shutdown_connections 2023-10-06T05:20:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.078+0000 7fe33919d700 1 --2- 172.21.15.33:0/2078025934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe334155af0 0x7fe33406a160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.078+0000 7fe33919d700 1 --2- 172.21.15.33:0/2078025934 >> v2:172.21.15.167:3300/0 conn(0x7fe33406a6a0 0x7fe3340c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.078+0000 7fe33919d700 1 -- 172.21.15.33:0/2078025934 >> 172.21.15.33:0/2078025934 conn(0x7fe33406d030 msgr2=0x7fe33406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.078+0000 7fe33919d700 1 -- 172.21.15.33:0/2078025934 shutdown_connections 2023-10-06T05:20:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.078+0000 7fe33919d700 1 -- 172.21.15.33:0/2078025934 wait complete. 2023-10-06T05:20:36.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.079+0000 7fe33919d700 1 Processor -- start 2023-10-06T05:20:36.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.079+0000 7fe33919d700 1 -- start start 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe33919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe33406a6a0 0x7fe3340b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe33919d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 0x7fe334151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe33919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe334155af0 0x7fe3340acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe33919d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341517d0 con 0x7fe3340c8b50 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe33919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340b1530 con 0x7fe334155af0 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe33919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340b0920 con 0x7fe33406a6a0 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe3337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 0x7fe334151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe3337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 0x7fe334151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53746/0 (socket says 172.21.15.33:53746) 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.080+0000 7fe3337fe700 1 -- 172.21.15.33:0/1886551946 learned_addr learned my addr 172.21.15.33:0/1886551946 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe333fff700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe33406a6a0 0x7fe3340b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe33899c700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe334155af0 0x7fe3340acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe3337fe700 1 -- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe33406a6a0 msgr2=0x7fe3340b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe3337fe700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe33406a6a0 0x7fe3340b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe3337fe700 1 -- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe334155af0 msgr2=0x7fe3340acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe3337fe700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe334155af0 0x7fe3340acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe3337fe700 1 -- 172.21.15.33:0/1886551946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32400a5d0 con 0x7fe3340c8b50 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe333fff700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe33406a6a0 0x7fe3340b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe33899c700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe334155af0 0x7fe3340acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe3337fe700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 0x7fe334151290 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe32800ec50 tx=0x7fe3280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32800c9d0 con 0x7fe3340c8b50 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32800cb30 con 0x7fe3340c8b50 2023-10-06T05:20:36.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328013470 con 0x7fe3340c8b50 2023-10-06T05:20:36.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.081+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3340b0c00 con 0x7fe3340c8b50 2023-10-06T05:20:36.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.082+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3340b11c0 con 0x7fe3340c8b50 2023-10-06T05:20:36.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.083+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340b16e0 con 0x7fe3340c8b50 2023-10-06T05:20:36.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.084+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe328013820 con 0x7fe3340c8b50 2023-10-06T05:20:36.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.085+0000 7fe31bfff700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe320066f30 0x7fe3200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.085+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe3280849f0 con 0x7fe3340c8b50 2023-10-06T05:20:36.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.085+0000 7fe333fff700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe320066f30 0x7fe3200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.086+0000 7fe333fff700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe320066f30 0x7fe3200693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe324008010 tx=0x7fe324019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.089+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3340b16e0 con 0x7fe3340c8b50 2023-10-06T05:20:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.336+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3340cd290 con 0x7fe3340c8b50 2023-10-06T05:20:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.337+0000 7fe31bfff700 1 -- 172.21.15.33:0/1886551946 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fe328051a80 con 0x7fe3340c8b50 2023-10-06T05:20:36.338 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:36.338 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:20:35.731516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe320066f30 msgr2=0x7fe3200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe320066f30 0x7fe3200693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe324008010 tx=0x7fe324019000 comp rx=0 tx=0).stop 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 msgr2=0x7fe334151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 0x7fe334151290 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe32800ec50 tx=0x7fe3280091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 shutdown_connections 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe320066f30 0x7fe3200693e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe334155af0 0x7fe3340acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe33406a6a0 0x7fe3340b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 --2- 172.21.15.33:0/1886551946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3340c8b50 0x7fe334151290 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 >> 172.21.15.33:0/1886551946 conn(0x7fe33406d030 msgr2=0x7fe3340c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 shutdown_connections 2023-10-06T05:20:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:36.339+0000 7fe33919d700 1 -- 172.21.15.33:0/1886551946 wait complete. 2023-10-06T05:20:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 48 2023-10-06T05:20:36.358 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2023-10-06T05:20:37.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.552+0000 7f0ba359e700 1 Processor -- start 2023-10-06T05:20:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.552+0000 7f0ba359e700 1 -- start start 2023-10-06T05:20:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.553+0000 7f0ba359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 0x7f0ba40c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.553+0000 7f0ba359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ba40c82a0 0x7f0ba406a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.553+0000 7f0ba359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba40c65e0 con 0x7f0ba40c6170 2023-10-06T05:20:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.553+0000 7f0ba359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba406c820 con 0x7f0ba40c82a0 2023-10-06T05:20:37.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.553+0000 7f0ba359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba406a160 con 0x7f0ba4159520 2023-10-06T05:20:37.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 0x7f0ba40c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba2d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ba40c82a0 0x7f0ba406a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 0x7f0ba40c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41778/0 (socket says 172.21.15.33:41778) 2023-10-06T05:20:37.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 -- 172.21.15.33:0/188231246 learned_addr learned my addr 172.21.15.33:0/188231246 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba1d9b700 1 --1- 172.21.15.33:0/188231246 >> v1:172.21.15.167:6789/0 conn(0x7f0ba40c6170 0x7f0ba40c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:60146/0 (socket says 172.21.15.33:60146) 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 -- 172.21.15.33:0/188231246 >> v2:172.21.15.167:3300/0 conn(0x7f0ba40c82a0 msgr2=0x7f0ba406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 --2- 172.21.15.33:0/188231246 >> v2:172.21.15.167:3300/0 conn(0x7f0ba40c82a0 0x7f0ba406a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 -- 172.21.15.33:0/188231246 >> v1:172.21.15.167:6789/0 conn(0x7f0ba40c6170 legacy=0x7f0ba40c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba159a700 1 -- 172.21.15.33:0/188231246 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410417456 0 0) 0x7f0ba40c65e0 con 0x7f0ba40c6170 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba259c700 1 -- 172.21.15.33:0/188231246 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40c8800 con 0x7f0ba4159520 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.554+0000 7f0ba2d9d700 1 --2- 172.21.15.33:0/188231246 >> v2:172.21.15.167:3300/0 conn(0x7f0ba40c82a0 0x7f0ba406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.555+0000 7f0ba259c700 1 --2- 172.21.15.33:0/188231246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 0x7f0ba40c5c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b940099b0 tx=0x7f0b94009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91811a5d1fd196ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.555+0000 7f0ba159a700 1 -- 172.21.15.33:0/188231246 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94009ea0 con 0x7f0ba4159520 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.555+0000 7f0ba159a700 1 -- 172.21.15.33:0/188231246 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9400b860 con 0x7f0ba4159520 2023-10-06T05:20:37.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.555+0000 7f0ba159a700 1 -- 172.21.15.33:0/188231246 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400bc00 con 0x7f0ba4159520 2023-10-06T05:20:37.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.555+0000 7f0ba359e700 1 -- 172.21.15.33:0/188231246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 msgr2=0x7f0ba40c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.555+0000 7f0ba359e700 1 --2- 172.21.15.33:0/188231246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 0x7f0ba40c5c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b940099b0 tx=0x7f0b94009250 comp rx=0 tx=0).stop 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.556+0000 7f0ba359e700 1 -- 172.21.15.33:0/188231246 shutdown_connections 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.556+0000 7f0ba359e700 1 --2- 172.21.15.33:0/188231246 >> v2:172.21.15.167:3300/0 conn(0x7f0ba40c82a0 0x7f0ba406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.556+0000 7f0ba359e700 1 --2- 172.21.15.33:0/188231246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba4159520 0x7f0ba40c5c30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.556+0000 7f0ba359e700 1 -- 172.21.15.33:0/188231246 >> 172.21.15.33:0/188231246 conn(0x7f0ba40cc930 msgr2=0x7f0ba40ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.556+0000 7f0ba359e700 1 -- 172.21.15.33:0/188231246 shutdown_connections 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.556+0000 7f0ba359e700 1 -- 172.21.15.33:0/188231246 wait complete. 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.557+0000 7f0ba359e700 1 Processor -- start 2023-10-06T05:20:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.557+0000 7f0ba359e700 1 -- start start 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba40c6170 0x7f0ba414d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 0x7f0ba4157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4159520 0x7f0ba41580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba415afd0 con 0x7f0ba40c82a0 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40ae130 con 0x7f0ba4159520 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40ae410 con 0x7f0ba40c6170 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 0x7f0ba4157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 0x7f0ba4157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53762/0 (socket says 172.21.15.33:53762) 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba2d9d700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4159520 0x7f0ba41580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba259c700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba40c6170 0x7f0ba414d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.558+0000 7f0ba1d9b700 1 -- 172.21.15.33:0/3656151158 learned_addr learned my addr 172.21.15.33:0/3656151158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba2d9d700 1 -- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba40c6170 msgr2=0x7f0ba414d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba2d9d700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba40c6170 0x7f0ba414d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba2d9d700 1 -- 172.21.15.33:0/3656151158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 msgr2=0x7f0ba4157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba2d9d700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 0x7f0ba4157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba2d9d700 1 -- 172.21.15.33:0/3656151158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c009a20 con 0x7f0ba4159520 2023-10-06T05:20:37.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba259c700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba40c6170 0x7f0ba414d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:37.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba1d9b700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 0x7f0ba4157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.559+0000 7f0ba2d9d700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4159520 0x7f0ba41580f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009f10 tx=0x7f0b9c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.560+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00cb70 con 0x7f0ba4159520 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.560+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c00f040 con 0x7f0ba4159520 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.560+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c014480 con 0x7f0ba4159520 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.560+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b940094f0 con 0x7f0ba4159520 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.560+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba40ae9d0 con 0x7f0ba4159520 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.561+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba415ada0 con 0x7f0ba4159520 2023-10-06T05:20:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.562+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b9c003020 con 0x7f0ba4159520 2023-10-06T05:20:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.563+0000 7f0b927fc700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b84066f30 0x7f0b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.563+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0b9c085830 con 0x7f0ba4159520 2023-10-06T05:20:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.564+0000 7f0ba259c700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b84066f30 0x7f0b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.564+0000 7f0ba259c700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b84066f30 0x7f0b840693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b940099b0 tx=0x7f0b94017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.567+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b9c0528a0 con 0x7f0ba4159520 2023-10-06T05:20:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.814+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ba415ada0 con 0x7f0ba4159520 2023-10-06T05:20:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.816+0000 7f0b927fc700 1 -- 172.21.15.33:0/3656151158 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f0ba415ada0 con 0x7f0ba4159520 2023-10-06T05:20:37.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:37.817 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:20:36.733515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.817+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b84066f30 msgr2=0x7f0b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.817+0000 7f0ba359e700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b84066f30 0x7f0b840693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b940099b0 tx=0x7f0b94017000 comp rx=0 tx=0).stop 2023-10-06T05:20:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.817+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4159520 msgr2=0x7f0ba41580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.817+0000 7f0ba359e700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4159520 0x7f0ba41580f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009f10 tx=0x7f0b9c00b040 comp rx=0 tx=0).stop 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 shutdown_connections 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b84066f30 0x7f0b840693e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4159520 0x7f0ba41580f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ba40c6170 0x7f0ba414d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 --2- 172.21.15.33:0/3656151158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ba40c82a0 0x7f0ba4157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 >> 172.21.15.33:0/3656151158 conn(0x7f0ba40cc930 msgr2=0x7f0ba4156f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 shutdown_connections 2023-10-06T05:20:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:37.818+0000 7f0ba359e700 1 -- 172.21.15.33:0/3656151158 wait complete. 2023-10-06T05:20:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2023-10-06T05:20:37.836 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:20:37.837 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 51.00569272041321s 2023-10-06T05:20:37.837 INFO:tasks.cephfs.test_client_recovery:Reached active... 2023-10-06T05:20:37.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:20:38.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.022+0000 7efc2959d700 1 Processor -- start 2023-10-06T05:20:38.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.022+0000 7efc2959d700 1 -- start start 2023-10-06T05:20:38.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.023+0000 7efc2959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 0x7efc240b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.023+0000 7efc2959d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc240b2e30 0x7efc2415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.023+0000 7efc2959d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2415e2d0 con 0x7efc240b0a40 2023-10-06T05:20:38.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.023+0000 7efc2959d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2406af70 con 0x7efc240b2e30 2023-10-06T05:20:38.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.023+0000 7efc2959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2415d6c0 con 0x7efc2406ab00 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 0x7efc240b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 0x7efc240b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41790/0 (socket says 172.21.15.33:41790) 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc23fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efc240b0a40 0x7efc240b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:60158/0 (socket says 172.21.15.33:60158) 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 -- 172.21.15.33:0/378400001 learned_addr learned my addr 172.21.15.33:0/378400001 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc28d9c700 1 --2- 172.21.15.33:0/378400001 >> v2:172.21.15.167:3300/0 conn(0x7efc240b2e30 0x7efc2415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc22ffd700 1 -- 172.21.15.33:0/378400001 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107742299 0 0) 0x7efc2415e2d0 con 0x7efc240b0a40 2023-10-06T05:20:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc22ffd700 1 -- 172.21.15.33:0/378400001 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc0c003570 con 0x7efc240b0a40 2023-10-06T05:20:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 -- 172.21.15.33:0/378400001 >> v2:172.21.15.167:3300/0 conn(0x7efc240b2e30 msgr2=0x7efc2415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 --2- 172.21.15.33:0/378400001 >> v2:172.21.15.167:3300/0 conn(0x7efc240b2e30 0x7efc2415a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 -- 172.21.15.33:0/378400001 >> v1:172.21.15.167:6789/0 conn(0x7efc240b0a40 legacy=0x7efc240b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc237fe700 1 -- 172.21.15.33:0/378400001 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2415d9a0 con 0x7efc2406ab00 2023-10-06T05:20:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.024+0000 7efc28d9c700 1 --2- 172.21.15.33:0/378400001 >> v2:172.21.15.167:3300/0 conn(0x7efc240b2e30 0x7efc2415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.025+0000 7efc237fe700 1 --2- 172.21.15.33:0/378400001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 0x7efc240b28f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc180099b0 tx=0x7efc18009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03409ebb021b637 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.025+0000 7efc22ffd700 1 -- 172.21.15.33:0/378400001 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800b680 con 0x7efc2406ab00 2023-10-06T05:20:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.025+0000 7efc22ffd700 1 -- 172.21.15.33:0/378400001 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800b7e0 con 0x7efc2406ab00 2023-10-06T05:20:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.025+0000 7efc22ffd700 1 -- 172.21.15.33:0/378400001 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800bb80 con 0x7efc2406ab00 2023-10-06T05:20:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 -- 172.21.15.33:0/378400001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 msgr2=0x7efc240b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 --2- 172.21.15.33:0/378400001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 0x7efc240b28f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc180099b0 tx=0x7efc18009250 comp rx=0 tx=0).stop 2023-10-06T05:20:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 -- 172.21.15.33:0/378400001 shutdown_connections 2023-10-06T05:20:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 --2- 172.21.15.33:0/378400001 >> v2:172.21.15.167:3300/0 conn(0x7efc240b2e30 0x7efc2415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 --2- 172.21.15.33:0/378400001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2406ab00 0x7efc240b28f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 -- 172.21.15.33:0/378400001 >> 172.21.15.33:0/378400001 conn(0x7efc2406e0f0 msgr2=0x7efc240675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 -- 172.21.15.33:0/378400001 shutdown_connections 2023-10-06T05:20:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.026+0000 7efc2959d700 1 -- 172.21.15.33:0/378400001 wait complete. 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.027+0000 7efc2959d700 1 Processor -- start 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.027+0000 7efc2959d700 1 -- start start 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.028+0000 7efc2959d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 0x7efc2415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.028+0000 7efc2959d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc240b0a40 0x7efc2416e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.028+0000 7efc2959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc240b2e30 0x7efc24150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.028+0000 7efc2959d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24151e40 con 0x7efc2406ab00 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.028+0000 7efc2959d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24152610 con 0x7efc240b0a40 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.028+0000 7efc2959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2416dde0 con 0x7efc240b2e30 2023-10-06T05:20:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 0x7efc2415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 0x7efc2415af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53764/0 (socket says 172.21.15.33:53764) 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 -- 172.21.15.33:0/1972679720 learned_addr learned my addr 172.21.15.33:0/1972679720 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc237fe700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc240b0a40 0x7efc2416e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc28d9c700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc240b2e30 0x7efc24150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 -- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc240b2e30 msgr2=0x7efc24150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc240b2e30 0x7efc24150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 -- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc240b0a40 msgr2=0x7efc2416e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc240b0a40 0x7efc2416e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 -- 172.21.15.33:0/1972679720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c009a20 con 0x7efc2406ab00 2023-10-06T05:20:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.029+0000 7efc23fff700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 0x7efc2415af90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c5c0 tx=0x7efc14007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.030+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14005a70 con 0x7efc2406ab00 2023-10-06T05:20:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.030+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14005bf0 con 0x7efc2406ab00 2023-10-06T05:20:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.030+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc140070d0 con 0x7efc2406ab00 2023-10-06T05:20:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.030+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc180094f0 con 0x7efc2406ab00 2023-10-06T05:20:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.030+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc24152330 con 0x7efc2406ab00 2023-10-06T05:20:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.031+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2406fcc0 con 0x7efc2406ab00 2023-10-06T05:20:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.032+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc140074c0 con 0x7efc2406ab00 2023-10-06T05:20:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.032+0000 7efc0bfff700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc10066f30 0x7efc100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.033+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efc14083bc0 con 0x7efc2406ab00 2023-10-06T05:20:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.033+0000 7efc237fe700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc10066f30 0x7efc100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.034+0000 7efc237fe700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc10066f30 0x7efc100693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc180099b0 tx=0x7efc18017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.037+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc14050c70 con 0x7efc2406ab00 2023-10-06T05:20:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.295+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efc2404c380 con 0x7efc2406ab00 2023-10-06T05:20:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.295+0000 7efc0bfff700 1 -- 172.21.15.33:0/1972679720 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7efc14017020 con 0x7efc2406ab00 2023-10-06T05:20:38.297 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:20:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc10066f30 msgr2=0x7efc100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc10066f30 0x7efc100693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc180099b0 tx=0x7efc18017000 comp rx=0 tx=0).stop 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 msgr2=0x7efc2415af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 0x7efc2415af90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c5c0 tx=0x7efc14007e00 comp rx=0 tx=0).stop 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 shutdown_connections 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc10066f30 0x7efc100693e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc240b2e30 0x7efc24150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2406ab00 0x7efc2415af90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 --2- 172.21.15.33:0/1972679720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc240b0a40 0x7efc2416e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.297+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 >> 172.21.15.33:0/1972679720 conn(0x7efc2406e0f0 msgr2=0x7efc240ab260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.298+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 shutdown_connections 2023-10-06T05:20:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:38.298+0000 7efc2959d700 1 -- 172.21.15.33:0/1972679720 wait complete. 2023-10-06T05:20:38.316 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:20:38.316 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:20:38.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:20:38.317 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:20:38.317 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:38.317 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:20:38.317 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:20:38.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:20:38.317 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:20:38.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:38.317 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.360 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:38.360 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:38.360 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:20:38.360 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41923sec preferred_lft 41923sec 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:38.364 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:20:38.462 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:20:38.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:38.466 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:20:38.504 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:20:38.504 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:20:38.504 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:20:38.504 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:20:38.505 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:20:38.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:38.506 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:20:38.506 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:20:38.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:20:38.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:20:38.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:20:38.506 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:20:38.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:38.701 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:20:38.738 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:20:38.738 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:20:38.739 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:20:38.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:38.739 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:20:38.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:38.840 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:38.877 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:20:38.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:38.877 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:38.953 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:20:38.953 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:20:38.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:20:39.083 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:20:39.083 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:20:39.083 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:20:39.099 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:20:39.100 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:20:39.100 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:20:39.100 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:20:39.100 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:20:39.121 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:20:39.121 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:20:39.121 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:20:39.121 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:20:39.122 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:20:39.122 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:20:39.122 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:20:39.122 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:20:39.126 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:20:39.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:39.127 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:39.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:39.187 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec ls subdir/childfile) 2023-10-06T05:20:39.225 INFO:teuthology.orchestra.run.smithi167.stdout:subdir/childfile 2023-10-06T05:20:39.226 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:39.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:39.313 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:39.398 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:39.398 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:20:39.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:39.398 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:39.486 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:20:39.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:39.486 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:20:39.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.718+0000 7f394db9d700 1 Processor -- start 2023-10-06T05:20:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.718+0000 7f394db9d700 1 -- start start 2023-10-06T05:20:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.719+0000 7f394db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f394806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.719+0000 7f394db9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39480c9cb0 con 0x7f39480c8050 2023-10-06T05:20:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.719+0000 7f394db9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39480ca270 con 0x7f394813faa0 2023-10-06T05:20:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.719+0000 7f394db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394806b840 con 0x7f39480cb710 2023-10-06T05:20:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.719+0000 7f3947fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f39480c8050 0x7f394806aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58282/0 (socket says 172.21.15.33:58282) 2023-10-06T05:20:39.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f3947fff700 1 -- 172.21.15.33:0/1150960257 learned_addr learned my addr 172.21.15.33:0/1150960257 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f394d39c700 1 --2- 172.21.15.33:0/1150960257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f394806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998716807 0 0) 0x7f39480c9cb0 con 0x7f39480c8050 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3938003570 con 0x7f39480c8050 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430681837 0 0) 0x7f39480ca270 con 0x7f394813faa0 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39480c9cb0 con 0x7f394813faa0 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f394d39c700 1 -- 172.21.15.33:0/1150960257 >> v1:172.21.15.167:6789/0 conn(0x7f394813faa0 legacy=0x7f39480c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f394d39c700 1 -- 172.21.15.33:0/1150960257 >> v1:172.21.15.33:6789/0 conn(0x7f39480c8050 legacy=0x7f394806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f394d39c700 1 -- 172.21.15.33:0/1150960257 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394806bb20 con 0x7f39480cb710 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.720+0000 7f394d39c700 1 --2- 172.21.15.33:0/1150960257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f394806b1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3940009e90 tx=0x7f3940009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86b18a67c96142b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000e070 con 0x7f39480cb710 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000bb80 con 0x7f39480cb710 2023-10-06T05:20:39.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f39477fe700 1 -- 172.21.15.33:0/1150960257 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940012430 con 0x7f39480cb710 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f394db9d700 1 -- 172.21.15.33:0/1150960257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 msgr2=0x7f394806b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f394db9d700 1 --2- 172.21.15.33:0/1150960257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f394806b1e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3940009e90 tx=0x7f3940009780 comp rx=0 tx=0).stop 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f394db9d700 1 -- 172.21.15.33:0/1150960257 shutdown_connections 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f394db9d700 1 --2- 172.21.15.33:0/1150960257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f394806b1e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f394db9d700 1 -- 172.21.15.33:0/1150960257 >> 172.21.15.33:0/1150960257 conn(0x7f394806d030 msgr2=0x7f394806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.721+0000 7f394db9d700 1 -- 172.21.15.33:0/1150960257 shutdown_connections 2023-10-06T05:20:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.722+0000 7f394db9d700 1 -- 172.21.15.33:0/1150960257 wait complete. 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.723+0000 7f394db9d700 1 Processor -- start 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.723+0000 7f394db9d700 1 -- start start 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 0x7f394813ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f39480cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394813faa0 0x7f39480cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394816b270 con 0x7f39480c8050 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394813f050 con 0x7f394813faa0 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394816ac60 con 0x7f39480cb710 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 0x7f394813ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 0x7f394813ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53766/0 (socket says 172.21.15.33:53766) 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394cb9b700 1 -- 172.21.15.33:0/50928335 learned_addr learned my addr 172.21.15.33:0/50928335 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f3947fff700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f39480cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.724+0000 7f394d39c700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394813faa0 0x7f39480cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394cb9b700 1 -- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 msgr2=0x7f39480cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394cb9b700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f39480cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394cb9b700 1 -- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394813faa0 msgr2=0x7f39480cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394cb9b700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394813faa0 0x7f39480cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394cb9b700 1 -- 172.21.15.33:0/50928335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940009a20 con 0x7f39480c8050 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394cb9b700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 0x7f394813ac70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c790 tx=0x7f393c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c010510 con 0x7f39480c8050 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c010670 con 0x7f39480c8050 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c010a10 con 0x7f39480c8050 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394813ed70 con 0x7f39480c8050 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.725+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394813b1b0 con 0x7f39480c8050 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.727+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394804c380 con 0x7f39480c8050 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.727+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f393c01b3b0 con 0x7f39480c8050 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.729+0000 7f3944ff9700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f392c066f30 0x7f392c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.729+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f393c083ff0 con 0x7f39480c8050 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.729+0000 7f3947fff700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f392c066f30 0x7f392c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.730+0000 7f3947fff700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f392c066f30 0x7f392c0693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f39480c9210 tx=0x7f3934009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.733+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f393c051040 con 0x7f39480c8050 2023-10-06T05:20:39.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.983+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39480c9f00 con 0x7f39480c8050 2023-10-06T05:20:39.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.983+0000 7f3944ff9700 1 -- 172.21.15.33:0/50928335 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f393c017090 con 0x7f39480c8050 2023-10-06T05:20:39.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:39.985 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:20:36.733515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:39.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f392c066f30 msgr2=0x7f392c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f392c066f30 0x7f392c0693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f39480c9210 tx=0x7f3934009040 comp rx=0 tx=0).stop 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 msgr2=0x7f394813ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 0x7f394813ac70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c790 tx=0x7f393c007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 shutdown_connections 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f392c066f30 0x7f392c0693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394813faa0 0x7f39480cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39480c8050 0x7f394813ac70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 --2- 172.21.15.33:0/50928335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39480cb710 0x7f39480cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 >> 172.21.15.33:0/50928335 conn(0x7f394806d030 msgr2=0x7f39480b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 shutdown_connections 2023-10-06T05:20:39.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:39.986+0000 7f394db9d700 1 -- 172.21.15.33:0/50928335 wait complete. 2023-10-06T05:20:39.989 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2023-10-06T05:20:40.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:40.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.189+0000 7f16fab9d700 1 Processor -- start 2023-10-06T05:20:40.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.190+0000 7f16fab9d700 1 -- start start 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16fab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16f40c8050 0x7f16f413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16fab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f40c9cb0 con 0x7f16f406aad0 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16fab9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f40ca270 con 0x7f16f413faa0 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16fab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f406b640 con 0x7f16f40c8050 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16f939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16f40c8050 0x7f16f413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16f939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16f40c8050 0x7f16f413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57320/0 (socket says 172.21.15.33:57320) 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16f939a700 1 -- 172.21.15.33:0/3061425292 learned_addr learned my addr 172.21.15.33:0/3061425292 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.191+0000 7f16fa39c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f16f406aad0 0x7f16f406af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43268/0 (socket says 172.21.15.33:43268) 2023-10-06T05:20:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f939a700 1 -- 172.21.15.33:0/3061425292 >> v1:172.21.15.167:6789/0 conn(0x7f16f413faa0 legacy=0x7f16f40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f939a700 1 -- 172.21.15.33:0/3061425292 >> v1:172.21.15.33:6789/0 conn(0x7f16f406aad0 legacy=0x7f16f406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f939a700 1 -- 172.21.15.33:0/3061425292 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f406b840 con 0x7f16f40c8050 2023-10-06T05:20:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f8b99700 1 -- 172.21.15.33:0/3061425292 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371153739 0 0) 0x7f16f40c9cb0 con 0x7f16f406aad0 2023-10-06T05:20:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f939a700 1 --2- 172.21.15.33:0/3061425292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f40c8050 0x7f16f413fe70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f16e40099b0 tx=0x7f16e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f78789cfe9d68f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f8b99700 1 -- 172.21.15.33:0/3061425292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400b860 con 0x7f16f40c8050 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f8b99700 1 -- 172.21.15.33:0/3061425292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e400b9c0 con 0x7f16f40c8050 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.192+0000 7f16f8b99700 1 -- 172.21.15.33:0/3061425292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400bd60 con 0x7f16f40c8050 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 -- 172.21.15.33:0/3061425292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f40c8050 msgr2=0x7f16f413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 --2- 172.21.15.33:0/3061425292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f40c8050 0x7f16f413fe70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f16e40099b0 tx=0x7f16e4009250 comp rx=0 tx=0).stop 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 -- 172.21.15.33:0/3061425292 shutdown_connections 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 --2- 172.21.15.33:0/3061425292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f40c8050 0x7f16f413fe70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 -- 172.21.15.33:0/3061425292 >> 172.21.15.33:0/3061425292 conn(0x7f16f406d030 msgr2=0x7f16f406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 -- 172.21.15.33:0/3061425292 shutdown_connections 2023-10-06T05:20:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.193+0000 7f16fab9d700 1 -- 172.21.15.33:0/3061425292 wait complete. 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.194+0000 7f16fab9d700 1 Processor -- start 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.194+0000 7f16fab9d700 1 -- start start 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.195+0000 7f16fab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 0x7f16f413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.195+0000 7f16fab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f40c8050 0x7f16f40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.195+0000 7f16fab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f413faa0 0x7f16f4169d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.195+0000 7f16fab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40a6990 con 0x7f16f406aad0 2023-10-06T05:20:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.195+0000 7f16fab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f413e610 con 0x7f16f413faa0 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.195+0000 7f16fab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40a5d80 con 0x7f16f40c8050 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 0x7f16f413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 0x7f16f413e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57334/0 (socket says 172.21.15.33:57334) 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 -- 172.21.15.33:0/1200892556 learned_addr learned my addr 172.21.15.33:0/1200892556 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16fa39c700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f413faa0 0x7f16f4169d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f939a700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f40c8050 0x7f16f40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 -- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f40c8050 msgr2=0x7f16f40a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f40c8050 0x7f16f40a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 -- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f413faa0 msgr2=0x7f16f4169d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f413faa0 0x7f16f4169d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 -- 172.21.15.33:0/1200892556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40094f0 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f939a700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f40c8050 0x7f16f40a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16fa39c700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f413faa0 0x7f16f4169d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16f9b9b700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 0x7f16f413e0d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16f000c4e0 tx=0x7f16f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0011590 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f0011710 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0011ad0 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.196+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f40a6060 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.197+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f40a6620 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.198+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f404c380 con 0x7f16f406aad0 2023-10-06T05:20:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.198+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16f001b3b0 con 0x7f16f406aad0 2023-10-06T05:20:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.199+0000 7f16e9ffb700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16d8066f30 0x7f16d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.199+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f16f0083c40 con 0x7f16f406aad0 2023-10-06T05:20:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.200+0000 7f16f939a700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16d8066f30 0x7f16d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.200+0000 7f16f939a700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16d8066f30 0x7f16d80693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16e40099b0 tx=0x7f16e4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.203+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16f0050cd0 con 0x7f16f406aad0 2023-10-06T05:20:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.407+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16f406bb20 con 0x7f16f406aad0 2023-10-06T05:20:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.408+0000 7f16e9ffb700 1 -- 172.21.15.33:0/1200892556 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17060 (secure 0 0 0) 0x7f16f0017070 con 0x7f16f406aad0 2023-10-06T05:20:40.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:40.410 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":78,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:34.725146+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:19:30.956591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:19:31.773352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3059565716":"2023-10-06T06:20:33.898361+0000","172.21.15.33:6835/2242593899":"2023-10-07T05:19:43.844349+0000","172.21.15.33:6834/2242593899":"2023-10-07T05:19:43.844349+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16d8066f30 msgr2=0x7f16d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16d8066f30 0x7f16d80693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16e40099b0 tx=0x7f16e4017000 comp rx=0 tx=0).stop 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 msgr2=0x7f16f413e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 0x7f16f413e0d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16f000c4e0 tx=0x7f16f000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 shutdown_connections 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16d8066f30 0x7f16d80693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f413faa0 0x7f16f4169d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f406aad0 0x7f16f413e0d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 --2- 172.21.15.33:0/1200892556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f40c8050 0x7f16f40a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 >> 172.21.15.33:0/1200892556 conn(0x7f16f406d030 msgr2=0x7f16f406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 shutdown_connections 2023-10-06T05:20:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.411+0000 7f16fab9d700 1 -- 172.21.15.33:0/1200892556 wait complete. 2023-10-06T05:20:40.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:19:30.956591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:19:31.773352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:20:40.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.618+0000 7f0e449fe700 1 Processor -- start 2023-10-06T05:20:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.618+0000 7f0e449fe700 1 -- start start 2023-10-06T05:20:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e449fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0e40154f20 0x7f0e400ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e449fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e401552f0 0x7f0e400cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e449fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e400b11e0 0x7f0e400b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e449fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4006bdf0 con 0x7f0e401552f0 2023-10-06T05:20:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e449fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e401586a0 con 0x7f0e40154f20 2023-10-06T05:20:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e449fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400b1b60 con 0x7f0e400b11e0 2023-10-06T05:20:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e3e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e401552f0 0x7f0e400cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e3e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e401552f0 0x7f0e400cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57350/0 (socket says 172.21.15.33:57350) 2023-10-06T05:20:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e3e59c700 1 -- 172.21.15.33:0/4024981565 learned_addr learned my addr 172.21.15.33:0/4024981565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e3f59e700 1 --2- 172.21.15.33:0/4024981565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e400b11e0 0x7f0e400b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.620+0000 7f0e3ed9d700 1 --2- 172.21.15.33:0/4024981565 >> v2:172.21.15.167:3300/0 conn(0x7f0e40154f20 0x7f0e400ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3e59c700 1 -- 172.21.15.33:0/4024981565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e400b11e0 msgr2=0x7f0e400b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3e59c700 1 --2- 172.21.15.33:0/4024981565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e400b11e0 0x7f0e400b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3e59c700 1 -- 172.21.15.33:0/4024981565 >> v2:172.21.15.167:3300/0 conn(0x7f0e40154f20 msgr2=0x7f0e400ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3e59c700 1 --2- 172.21.15.33:0/4024981565 >> v2:172.21.15.167:3300/0 conn(0x7f0e40154f20 0x7f0e400ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3e59c700 1 -- 172.21.15.33:0/4024981565 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40157ab0 con 0x7f0e401552f0 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3e59c700 1 --2- 172.21.15.33:0/4024981565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401552f0 0x7f0e400cc7e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0e400713f0 tx=0x7f0e34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9073dbbc8af159b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3dd9b700 1 -- 172.21.15.33:0/4024981565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400b640 con 0x7f0e401552f0 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3dd9b700 1 -- 172.21.15.33:0/4024981565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3400b7a0 con 0x7f0e401552f0 2023-10-06T05:20:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.621+0000 7f0e3dd9b700 1 -- 172.21.15.33:0/4024981565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400bb60 con 0x7f0e401552f0 2023-10-06T05:20:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 -- 172.21.15.33:0/4024981565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401552f0 msgr2=0x7f0e400cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4024981565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401552f0 0x7f0e400cc7e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0e400713f0 tx=0x7f0e34009250 comp rx=0 tx=0).stop 2023-10-06T05:20:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 -- 172.21.15.33:0/4024981565 shutdown_connections 2023-10-06T05:20:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4024981565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e400b11e0 0x7f0e400b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4024981565 >> v2:172.21.15.167:3300/0 conn(0x7f0e40154f20 0x7f0e400ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4024981565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401552f0 0x7f0e400cc7e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 -- 172.21.15.33:0/4024981565 >> 172.21.15.33:0/4024981565 conn(0x7f0e4006d030 msgr2=0x7f0e400cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 -- 172.21.15.33:0/4024981565 shutdown_connections 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.622+0000 7f0e449fe700 1 -- 172.21.15.33:0/4024981565 wait complete. 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.623+0000 7f0e449fe700 1 Processor -- start 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.623+0000 7f0e449fe700 1 -- start start 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.624+0000 7f0e449fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e400b11e0 0x7f0e40143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.624+0000 7f0e449fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e40154f20 0x7f0e400c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.624+0000 7f0e449fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 0x7f0e4006e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.624+0000 7f0e449fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4006eaa0 con 0x7f0e4006e0c0 2023-10-06T05:20:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.624+0000 7f0e449fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400c8920 con 0x7f0e400b11e0 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.624+0000 7f0e449fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400c7f50 con 0x7f0e40154f20 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 0x7f0e4006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 0x7f0e4006e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57360/0 (socket says 172.21.15.33:57360) 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 -- 172.21.15.33:0/4116327248 learned_addr learned my addr 172.21.15.33:0/4116327248 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3ed9d700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e400b11e0 0x7f0e40143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3e59c700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e40154f20 0x7f0e400c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 -- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e40154f20 msgr2=0x7f0e400c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e40154f20 0x7f0e400c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 -- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e400b11e0 msgr2=0x7f0e40143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e400b11e0 0x7f0e40143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 -- 172.21.15.33:0/4116327248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340094f0 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.625+0000 7f0e3f59e700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 0x7f0e4006e560 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009ef0 tx=0x7f0e3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.626+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800cb70 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.626+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3800f040 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.626+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38014480 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.626+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e400c7340 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.626+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e400c7900 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.627+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4004c380 con 0x7f0e4006e0c0 2023-10-06T05:20:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.628+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e38014830 con 0x7f0e4006e0c0 2023-10-06T05:20:40.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.629+0000 7f0e26ffd700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e2c066f30 0x7f0e2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.629+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0e380850c0 con 0x7f0e4006e0c0 2023-10-06T05:20:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.629+0000 7f0e3ed9d700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e2c066f30 0x7f0e2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.630+0000 7f0e3ed9d700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e2c066f30 0x7f0e2c0693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009db0 tx=0x7f0e300091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.633+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e380520b0 con 0x7f0e4006e0c0 2023-10-06T05:20:40.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.833+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e400ad8e0 con 0x7f0e4006e0c0 2023-10-06T05:20:40.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.834+0000 7f0e26ffd700 1 -- 172.21.15.33:0/4116327248 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e380556d0 con 0x7f0e4006e0c0 2023-10-06T05:20:40.836 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:20:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e2c066f30 msgr2=0x7f0e2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e2c066f30 0x7f0e2c0693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009db0 tx=0x7f0e300091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 msgr2=0x7f0e4006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 0x7f0e4006e560 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009ef0 tx=0x7f0e3800b040 comp rx=0 tx=0).stop 2023-10-06T05:20:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 shutdown_connections 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e2c066f30 0x7f0e2c0693e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e4006e0c0 0x7f0e4006e560 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e400b11e0 0x7f0e40143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 --2- 172.21.15.33:0/4116327248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e40154f20 0x7f0e400c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 >> 172.21.15.33:0/4116327248 conn(0x7f0e4006d030 msgr2=0x7f0e40154100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 shutdown_connections 2023-10-06T05:20:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:40.836+0000 7f0e449fe700 1 -- 172.21.15.33:0/4116327248 wait complete. 2023-10-06T05:20:40.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:41.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.044+0000 7f804f19d700 1 Processor -- start 2023-10-06T05:20:41.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.044+0000 7f804f19d700 1 -- start start 2023-10-06T05:20:41.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804806c7b0 0x7f8048067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80480aff90 0x7f80480a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804f19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804815d3a0 con 0x7f804806cb80 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80480b0470 con 0x7f80480aff90 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80480b0750 con 0x7f804806c7b0 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80480aff90 0x7f80480a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804e99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f804806cb80 0x7f804815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43282/0 (socket says 172.21.15.33:43282) 2023-10-06T05:20:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804e99c700 1 -- 172.21.15.33:0/4251084754 learned_addr learned my addr 172.21.15.33:0/4251084754 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:41.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.046+0000 7f804e19b700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804806c7b0 0x7f8048067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d99a700 1 -- 172.21.15.33:0/4251084754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804806c7b0 msgr2=0x7f8048067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d99a700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804806c7b0 0x7f8048067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d99a700 1 -- 172.21.15.33:0/4251084754 >> v1:172.21.15.33:6789/0 conn(0x7f804806cb80 legacy=0x7f804815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d99a700 1 -- 172.21.15.33:0/4251084754 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80480b0a30 con 0x7f80480aff90 2023-10-06T05:20:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804e19b700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804806c7b0 0x7f8048067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d99a700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80480aff90 0x7f80480a9890 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f80380098e0 tx=0x7f80380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907d934086a8f1e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d199700 1 -- 172.21.15.33:0/4251084754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038009e80 con 0x7f80480aff90 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d199700 1 -- 172.21.15.33:0/4251084754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800b860 con 0x7f80480aff90 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.047+0000 7f804d199700 1 -- 172.21.15.33:0/4251084754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800bbe0 con 0x7f80480aff90 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 -- 172.21.15.33:0/4251084754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80480aff90 msgr2=0x7f80480a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80480aff90 0x7f80480a9890 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f80380098e0 tx=0x7f80380091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 -- 172.21.15.33:0/4251084754 shutdown_connections 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804806c7b0 0x7f8048067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 --2- 172.21.15.33:0/4251084754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80480aff90 0x7f80480a9890 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 -- 172.21.15.33:0/4251084754 >> 172.21.15.33:0/4251084754 conn(0x7f804806d670 msgr2=0x7f804806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 -- 172.21.15.33:0/4251084754 shutdown_connections 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.048+0000 7f804f19d700 1 -- 172.21.15.33:0/4251084754 wait complete. 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.049+0000 7f804f19d700 1 Processor -- start 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.049+0000 7f804f19d700 1 -- start start 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.050+0000 7f804f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f804806c7b0 0x7f804814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.050+0000 7f804f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 0x7f804814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.050+0000 7f804f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80480aff90 0x7f8048169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.050+0000 7f804f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804816aea0 con 0x7f804806cb80 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.050+0000 7f804f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804814b690 con 0x7f804806c7b0 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.050+0000 7f804f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804816a8e0 con 0x7f80480aff90 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 0x7f804814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 0x7f804814bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57372/0 (socket says 172.21.15.33:57372) 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 -- 172.21.15.33:0/4204230964 learned_addr learned my addr 172.21.15.33:0/4204230964 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:41.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804e19b700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f804806c7b0 0x7f804814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804e99c700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80480aff90 0x7f8048169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 -- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80480aff90 msgr2=0x7f8048169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80480aff90 0x7f8048169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 -- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f804806c7b0 msgr2=0x7f804814a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f804806c7b0 0x7f804814a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 -- 172.21.15.33:0/4204230964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009420 con 0x7f804806cb80 2023-10-06T05:20:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.051+0000 7f804d99a700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 0x7f804814bb20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8038013000 tx=0x7f8038017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.052+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800bbe0 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.052+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8038011420 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.052+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380117c0 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.052+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804816abc0 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.052+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804814aeb0 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.053+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804814b800 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.054+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8038011bb0 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.054+0000 7f803e7fc700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f802c066f30 0x7f802c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.054+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8038085ea0 con 0x7f804806cb80 2023-10-06T05:20:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.055+0000 7f804e19b700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f802c066f30 0x7f802c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.056+0000 7f804e19b700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f802c066f30 0x7f802c0693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f804400ab80 tx=0x7f8044008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.059+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8038052e90 con 0x7f804806cb80 2023-10-06T05:20:41.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.263+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f804804c380 con 0x7f804806cb80 2023-10-06T05:20:41.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.263+0000 7f803e7fc700 1 -- 172.21.15.33:0/4204230964 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v78) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8038011e60 con 0x7f804806cb80 2023-10-06T05:20:41.265 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:20:41.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f802c066f30 msgr2=0x7f802c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f802c066f30 0x7f802c0693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f804400ab80 tx=0x7f8044008040 comp rx=0 tx=0).stop 2023-10-06T05:20:41.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 msgr2=0x7f804814bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 0x7f804814bb20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8038013000 tx=0x7f8038017000 comp rx=0 tx=0).stop 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 shutdown_connections 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f802c066f30 0x7f802c0693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80480aff90 0x7f8048169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f804806c7b0 0x7f804814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 --2- 172.21.15.33:0/4204230964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f804806cb80 0x7f804814bb20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 >> 172.21.15.33:0/4204230964 conn(0x7f804806d670 msgr2=0x7f80480acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 shutdown_connections 2023-10-06T05:20:41.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.266+0000 7f804f19d700 1 -- 172.21.15.33:0/4204230964 wait complete. 2023-10-06T05:20:41.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:20:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.473+0000 7f4e877ff700 1 Processor -- start 2023-10-06T05:20:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.474+0000 7f4e877ff700 1 -- start start 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e877ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e801566b0 0x7f4e800c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e877ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e800ae890 0x7f4e800c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e877ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e800af020 con 0x7f4e80156a80 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e877ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800c2a90 con 0x7f4e801566b0 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e877ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800c7300 con 0x7f4e800ae890 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e8559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e801566b0 0x7f4e800c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e8559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e801566b0 0x7f4e800c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57386/0 (socket says 172.21.15.33:57386) 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e8559b700 1 -- 172.21.15.33:0/1938216356 learned_addr learned my addr 172.21.15.33:0/1938216356 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.475+0000 7f4e85d9c700 1 --2- 172.21.15.33:0/1938216356 >> v2:172.21.15.167:3300/0 conn(0x7f4e800ae890 0x7f4e800c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e8559b700 1 -- 172.21.15.33:0/1938216356 >> v2:172.21.15.167:3300/0 conn(0x7f4e800ae890 msgr2=0x7f4e800c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e8559b700 1 --2- 172.21.15.33:0/1938216356 >> v2:172.21.15.167:3300/0 conn(0x7f4e800ae890 0x7f4e800c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e8559b700 1 -- 172.21.15.33:0/1938216356 >> v1:172.21.15.33:6789/0 conn(0x7f4e80156a80 legacy=0x7f4e800ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e8559b700 1 -- 172.21.15.33:0/1938216356 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800c12a0 con 0x7f4e801566b0 2023-10-06T05:20:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e85d9c700 1 --2- 172.21.15.33:0/1938216356 >> v2:172.21.15.167:3300/0 conn(0x7f4e800ae890 0x7f4e800c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e8559b700 1 --2- 172.21.15.33:0/1938216356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c1f20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4e800aed70 tx=0x7f4e7c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cbc49f5c84cda4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e77fff700 1 -- 172.21.15.33:0/1938216356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00dc50 con 0x7f4e801566b0 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e77fff700 1 -- 172.21.15.33:0/1938216356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c00ddb0 con 0x7f4e801566b0 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.476+0000 7f4e77fff700 1 -- 172.21.15.33:0/1938216356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0196d0 con 0x7f4e801566b0 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 -- 172.21.15.33:0/1938216356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 msgr2=0x7f4e800c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 --2- 172.21.15.33:0/1938216356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c1f20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4e800aed70 tx=0x7f4e7c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 -- 172.21.15.33:0/1938216356 shutdown_connections 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 --2- 172.21.15.33:0/1938216356 >> v2:172.21.15.167:3300/0 conn(0x7f4e800ae890 0x7f4e800c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 --2- 172.21.15.33:0/1938216356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c1f20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 -- 172.21.15.33:0/1938216356 >> 172.21.15.33:0/1938216356 conn(0x7f4e8006d030 msgr2=0x7f4e800cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 -- 172.21.15.33:0/1938216356 shutdown_connections 2023-10-06T05:20:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.477+0000 7f4e877ff700 1 -- 172.21.15.33:0/1938216356 wait complete. 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.478+0000 7f4e877ff700 1 Processor -- start 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.479+0000 7f4e877ff700 1 -- start start 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.479+0000 7f4e877ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e800ae890 0x7f4e80151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.479+0000 7f4e877ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e877ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e80156a80 0x7f4e801522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e877ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80169e80 con 0x7f4e801566b0 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e877ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8016a140 con 0x7f4e800ae890 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e877ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8016a420 con 0x7f4e80156a80 2023-10-06T05:20:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57394/0 (socket says 172.21.15.33:57394) 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 -- 172.21.15.33:0/4015420702 learned_addr learned my addr 172.21.15.33:0/4015420702 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e85d9c700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e80156a80 0x7f4e801522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e8559b700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e800ae890 0x7f4e80151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 -- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e80156a80 msgr2=0x7f4e801522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e80156a80 0x7f4e801522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 -- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e800ae890 msgr2=0x7f4e80151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e800ae890 0x7f4e80151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.480+0000 7f4e84d9a700 1 -- 172.21.15.33:0/4015420702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c00b790 con 0x7f4e801566b0 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.481+0000 7f4e84d9a700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c8eb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c590 tx=0x7f4e70007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.481+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011590 con 0x7f4e801566b0 2023-10-06T05:20:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.481+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e70011710 con 0x7f4e801566b0 2023-10-06T05:20:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.481+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011ad0 con 0x7f4e801566b0 2023-10-06T05:20:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.481+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8016a700 con 0x7f4e801566b0 2023-10-06T05:20:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.481+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8016ad50 con 0x7f4e801566b0 2023-10-06T05:20:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.482+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8016d480 con 0x7f4e801566b0 2023-10-06T05:20:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.483+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e7001a360 con 0x7f4e801566b0 2023-10-06T05:20:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.484+0000 7f4e757fa700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e64066f30 0x7f4e640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.484+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4e70083600 con 0x7f4e801566b0 2023-10-06T05:20:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.484+0000 7f4e8559b700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e64066f30 0x7f4e640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.485+0000 7f4e8559b700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e64066f30 0x7f4e640693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e800aed70 tx=0x7f4e7c01b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.488+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e8016d480 con 0x7f4e801566b0 2023-10-06T05:20:41.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.688+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e8016d480 con 0x7f4e801566b0 2023-10-06T05:20:41.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.689+0000 7f4e757fa700 1 -- 172.21.15.33:0/4015420702 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e8016d480 con 0x7f4e801566b0 2023-10-06T05:20:41.692 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:20:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e64066f30 msgr2=0x7f4e640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e64066f30 0x7f4e640693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e800aed70 tx=0x7f4e7c01b000 comp rx=0 tx=0).stop 2023-10-06T05:20:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 msgr2=0x7f4e800c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c8eb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c590 tx=0x7f4e70007f50 comp rx=0 tx=0).stop 2023-10-06T05:20:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 shutdown_connections 2023-10-06T05:20:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e64066f30 0x7f4e640693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e80156a80 0x7f4e801522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e800ae890 0x7f4e80151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 --2- 172.21.15.33:0/4015420702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e801566b0 0x7f4e800c8eb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 >> 172.21.15.33:0/4015420702 conn(0x7f4e8006d030 msgr2=0x7f4e8014bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 shutdown_connections 2023-10-06T05:20:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.691+0000 7f4e877ff700 1 -- 172.21.15.33:0/4015420702 wait complete. 2023-10-06T05:20:41.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:20:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.898+0000 7fcc52f9d700 1 Processor -- start 2023-10-06T05:20:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.898+0000 7fcc52f9d700 1 -- start start 2023-10-06T05:20:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc52f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc4c15acd0 0x7fcc4c0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc52f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc52f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c0b0cd0 con 0x7fcc4c159850 2023-10-06T05:20:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc52f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c15a840 con 0x7fcc4c15acd0 2023-10-06T05:20:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc52f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c159f10 con 0x7fcc4c06efa0 2023-10-06T05:20:41.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc51f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcc4c159850 0x7fcc4c06e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43294/0 (socket says 172.21.15.33:43294) 2023-10-06T05:20:41.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc5179a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc4c15acd0 0x7fcc4c0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc51f9b700 1 -- 172.21.15.33:0/2516137104 learned_addr learned my addr 172.21.15.33:0/2516137104 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:41.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc5279c700 1 --2- 172.21.15.33:0/2516137104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc50f99700 1 -- 172.21.15.33:0/2516137104 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569646286 0 0) 0x7fcc4c0b0cd0 con 0x7fcc4c159850 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc50f99700 1 -- 172.21.15.33:0/2516137104 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc38003570 con 0x7fcc4c159850 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc5179a700 1 -- 172.21.15.33:0/2516137104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c06efa0 msgr2=0x7fcc4c0ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc5179a700 1 --2- 172.21.15.33:0/2516137104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.900+0000 7fcc5179a700 1 -- 172.21.15.33:0/2516137104 >> v1:172.21.15.33:6789/0 conn(0x7fcc4c159850 legacy=0x7fcc4c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.901+0000 7fcc5179a700 1 -- 172.21.15.33:0/2516137104 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c15a1f0 con 0x7fcc4c15acd0 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.901+0000 7fcc5179a700 1 --2- 172.21.15.33:0/2516137104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c0abe90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0098e0 tx=0x7fcc3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878736cf5e241b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.901+0000 7fcc50f99700 1 -- 172.21.15.33:0/2516137104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c011630 con 0x7fcc4c15acd0 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.901+0000 7fcc50f99700 1 -- 172.21.15.33:0/2516137104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c011790 con 0x7fcc4c15acd0 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.901+0000 7fcc50f99700 1 -- 172.21.15.33:0/2516137104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c011b10 con 0x7fcc4c15acd0 2023-10-06T05:20:41.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 -- 172.21.15.33:0/2516137104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 msgr2=0x7fcc4c0abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/2516137104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c0abe90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0098e0 tx=0x7fcc3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 -- 172.21.15.33:0/2516137104 shutdown_connections 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/2516137104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/2516137104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c0abe90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 -- 172.21.15.33:0/2516137104 >> 172.21.15.33:0/2516137104 conn(0x7fcc4c06cd40 msgr2=0x7fcc4c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 -- 172.21.15.33:0/2516137104 shutdown_connections 2023-10-06T05:20:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.902+0000 7fcc52f9d700 1 -- 172.21.15.33:0/2516137104 wait complete. 2023-10-06T05:20:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.903+0000 7fcc52f9d700 1 Processor -- start 2023-10-06T05:20:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.903+0000 7fcc52f9d700 1 -- start start 2023-10-06T05:20:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc52f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc52f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c159850 0x7fcc4c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc52f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c14db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc52f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0cc570 con 0x7fcc4c15acd0 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc52f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c7b10 con 0x7fcc4c06efa0 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc52f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0cd800 con 0x7fcc4c159850 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.904+0000 7fcc5279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c14db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc51f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c14db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57414/0 (socket says 172.21.15.33:57414) 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5179a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c159850 0x7fcc4c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 -- 172.21.15.33:0/4054410184 learned_addr learned my addr 172.21.15.33:0/4054410184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 -- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c159850 msgr2=0x7fcc4c0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c159850 0x7fcc4c0ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 -- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4c06efa0 msgr2=0x7fcc4c0ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 -- 172.21.15.33:0/4054410184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c009420 con 0x7fcc4c15acd0 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.905+0000 7fcc5279c700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c14db40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009ec0 tx=0x7fcc4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.906+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4400cb70 con 0x7fcc4c15acd0 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.906+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4400f040 con 0x7fcc4c15acd0 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.906+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c0c74c0 con 0x7fcc4c15acd0 2023-10-06T05:20:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.906+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44014480 con 0x7fcc4c15acd0 2023-10-06T05:20:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.906+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c0acff0 con 0x7fcc4c15acd0 2023-10-06T05:20:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.907+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c0c7c80 con 0x7fcc4c15acd0 2023-10-06T05:20:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.908+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc440147f0 con 0x7fcc4c15acd0 2023-10-06T05:20:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.908+0000 7fcc41ffb700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcc30066f30 0x7fcc300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.908+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc440853e0 con 0x7fcc4c15acd0 2023-10-06T05:20:41.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.909+0000 7fcc51f9b700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcc30066f30 0x7fcc300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.909+0000 7fcc51f9b700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcc30066f30 0x7fcc300693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcc48007fd0 tx=0x7fcc48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:41.912+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc440523d0 con 0x7fcc4c15acd0 2023-10-06T05:20:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.115+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc4c04c380 con 0x7fcc4c15acd0 2023-10-06T05:20:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.116+0000 7fcc41ffb700 1 -- 172.21.15.33:0/4054410184 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc440559f0 con 0x7fcc4c15acd0 2023-10-06T05:20:42.117 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:20:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcc30066f30 msgr2=0x7fcc300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcc30066f30 0x7fcc300693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcc48007fd0 tx=0x7fcc48009040 comp rx=0 tx=0).stop 2023-10-06T05:20:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 msgr2=0x7fcc4c14db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c14db40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009ec0 tx=0x7fcc4400b040 comp rx=0 tx=0).stop 2023-10-06T05:20:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 shutdown_connections 2023-10-06T05:20:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcc30066f30 0x7fcc300693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc4c15acd0 0x7fcc4c14db40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4c06efa0 0x7fcc4c0ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 --2- 172.21.15.33:0/4054410184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc4c159850 0x7fcc4c0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 >> 172.21.15.33:0/4054410184 conn(0x7fcc4c06cd40 msgr2=0x7fcc4c1579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 shutdown_connections 2023-10-06T05:20:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.118+0000 7fcc52f9d700 1 -- 172.21.15.33:0/4054410184 wait complete. 2023-10-06T05:20:42.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:42.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.324+0000 7f9a7659d700 1 Processor -- start 2023-10-06T05:20:42.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.325+0000 7f9a7659d700 1 -- start start 2023-10-06T05:20:42.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a7659d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9a700c8050 0x7f9a7013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a7659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700c9cb0 con 0x7f9a7013faa0 2023-10-06T05:20:42.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a7659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700ca270 con 0x7f9a7006aad0 2023-10-06T05:20:42.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a7659d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7006b640 con 0x7f9a700c8050 2023-10-06T05:20:42.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a7559b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9a7013faa0 0x7f9a700c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43298/0 (socket says 172.21.15.33:43298) 2023-10-06T05:20:42.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a7559b700 1 -- 172.21.15.33:0/1198969174 learned_addr learned my addr 172.21.15.33:0/1198969174 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.326+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/1198969174 >> v2:172.21.15.167:3300/0 conn(0x7f9a700c8050 0x7f9a7013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238154848 0 0) 0x7f9a700c9cb0 con 0x7f9a7013faa0 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a5c003570 con 0x7f9a7013faa0 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19839670 0 0) 0x7f9a700ca270 con 0x7f9a7006aad0 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a700c9cb0 con 0x7f9a7006aad0 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2313041599 0 0) 0x7f9a5c003570 con 0x7f9a7013faa0 2023-10-06T05:20:42.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 >> v2:172.21.15.167:3300/0 conn(0x7f9a700c8050 msgr2=0x7f9a7013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 --2- 172.21.15.33:0/1198969174 >> v2:172.21.15.167:3300/0 conn(0x7f9a700c8050 0x7f9a7013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 >> v1:172.21.15.167:6789/0 conn(0x7f9a7006aad0 legacy=0x7f9a7006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7006b840 con 0x7f9a7013faa0 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9a6c001470 con 0x7f9a7013faa0 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 >> v1:172.21.15.33:6789/0 conn(0x7f9a7013faa0 legacy=0x7f9a700c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a5c003bf0 0x7f9a5c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5c004810 0x7f9a5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a5c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7006b640 con 0x7f9a5c0051c0 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c00c0b0 con 0x7f9a5c004810 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c00c270 con 0x7f9a5c003bf0 2023-10-06T05:20:42.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/1198969174 >> v2:172.21.15.167:3300/0 conn(0x7f9a700c8050 0x7f9a7013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.327+0000 7f9a75d9c700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a5c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a7559b700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a5c003bf0 0x7f9a5c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5c004810 0x7f9a5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 -- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a5c003bf0 msgr2=0x7f9a5c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a5c003bf0 0x7f9a5c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 -- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5c004810 msgr2=0x7f9a5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5c004810 0x7f9a5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 -- 172.21.15.33:0/1198969174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c003570 con 0x7f9a5c0051c0 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a5c004810 0x7f9a5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:42.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 -- 172.21.15.33:0/1198969174 reap_dead start 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a75d9c700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a5c00ba60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9a68002120 tx=0x7f9a680017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906385732a10cbe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800c980 con 0x7f9a5c0051c0 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6800cae0 con 0x7f9a5c0051c0 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.328+0000 7f9a67fff700 1 -- 172.21.15.33:0/1198969174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800ce80 con 0x7f9a5c0051c0 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 -- 172.21.15.33:0/1198969174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 msgr2=0x7f9a5c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a5c00ba60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9a68002120 tx=0x7f9a680017a0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 -- 172.21.15.33:0/1198969174 shutdown_connections 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 --2- 172.21.15.33:0/1198969174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a5c00ba60 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 -- 172.21.15.33:0/1198969174 >> 172.21.15.33:0/1198969174 conn(0x7f9a7006d030 msgr2=0x7f9a7006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 -- 172.21.15.33:0/1198969174 shutdown_connections 2023-10-06T05:20:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.329+0000 7f9a7659d700 1 -- 172.21.15.33:0/1198969174 wait complete. 2023-10-06T05:20:42.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.331+0000 7f9a7659d700 1 Processor -- start 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.331+0000 7f9a7659d700 1 -- start start 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a700a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a7015a0a0 0x7f9a7015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a70169570 0x7f9a701699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700a94a0 con 0x7f9a5c0051c0 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70159950 con 0x7f9a7015a0a0 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7015aa50 con 0x7f9a70169570 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a700a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a700a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57426/0 (socket says 172.21.15.33:57426) 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 -- 172.21.15.33:0/428022855 learned_addr learned my addr 172.21.15.33:0/428022855 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a7015a0a0 0x7f9a7015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 -- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a70169570 msgr2=0x7f9a701699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a70169570 0x7f9a701699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 -- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a7015a0a0 msgr2=0x7f9a7015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a7015a0a0 0x7f9a7015a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a7559b700 1 -- 172.21.15.33:0/428022855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7016a550 con 0x7f9a5c0051c0 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.332+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a7015a0a0 0x7f9a7015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.333+0000 7f9a7559b700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a700a8f60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c007390 tx=0x7f9a700c84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.333+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c018030 con 0x7f9a5c0051c0 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.333+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00f870 con 0x7f9a5c0051c0 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.333+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7016a270 con 0x7f9a5c0051c0 2023-10-06T05:20:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.333+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c007800 con 0x7f9a5c0051c0 2023-10-06T05:20:42.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.333+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7016ab90 con 0x7f9a5c0051c0 2023-10-06T05:20:42.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.334+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7004c380 con 0x7f9a5c0051c0 2023-10-06T05:20:42.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.335+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a6c01b070 con 0x7f9a5c0051c0 2023-10-06T05:20:42.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.336+0000 7f9a657fa700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a54066f30 0x7f9a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.336+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9a6c019070 con 0x7f9a5c0051c0 2023-10-06T05:20:42.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.336+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a54066f30 0x7f9a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.337+0000 7f9a74d9a700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a54066f30 0x7f9a540693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9a60006f00 tx=0x7f9a60008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.340+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a6c051690 con 0x7f9a5c0051c0 2023-10-06T05:20:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.587+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a7006bb20 con 0x7f9a5c0051c0 2023-10-06T05:20:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.588+0000 7f9a657fa700 1 -- 172.21.15.33:0/428022855 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f9a6c054cb0 con 0x7f9a5c0051c0 2023-10-06T05:20:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:42.590 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:20:36.733515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.590+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a54066f30 msgr2=0x7f9a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.590+0000 7f9a7659d700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a54066f30 0x7f9a540693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9a60006f00 tx=0x7f9a60008040 comp rx=0 tx=0).stop 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.590+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 msgr2=0x7f9a700a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.590+0000 7f9a7659d700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a700a8f60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c007390 tx=0x7f9a700c84c0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 shutdown_connections 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a54066f30 0x7f9a540693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a70169570 0x7f9a701699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a5c0051c0 0x7f9a700a8f60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 --2- 172.21.15.33:0/428022855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a7015a0a0 0x7f9a7015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 >> 172.21.15.33:0/428022855 conn(0x7f9a7006d030 msgr2=0x7f9a7013e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 shutdown_connections 2023-10-06T05:20:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.591+0000 7f9a7659d700 1 -- 172.21.15.33:0/428022855 wait complete. 2023-10-06T05:20:42.594 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2023-10-06T05:20:42.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:42.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.796+0000 7f4a1eb9d700 1 Processor -- start 2023-10-06T05:20:42.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.796+0000 7f4a1eb9d700 1 -- start start 2023-10-06T05:20:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.797+0000 7f4a1eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a18155f60 0x7f4a180c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.797+0000 7f4a1eb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1806ab60 con 0x7f4a18155af0 2023-10-06T05:20:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.797+0000 7f4a1eb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1806ae40 con 0x7f4a180c8b50 2023-10-06T05:20:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.797+0000 7f4a1eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1806b7e0 con 0x7f4a18155f60 2023-10-06T05:20:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.797+0000 7f4a1d39a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4a18155af0 0x7f4a1806a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43304/0 (socket says 172.21.15.33:43304) 2023-10-06T05:20:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.797+0000 7f4a1d39a700 1 -- 172.21.15.33:0/1352047614 learned_addr learned my addr 172.21.15.33:0/1352047614 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a18155f60 0x7f4a180c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981771519 0 0) 0x7f4a1806ab60 con 0x7f4a18155af0 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a04003570 con 0x7f4a18155af0 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978431831 0 0) 0x7f4a1806ae40 con 0x7f4a180c8b50 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a1806ab60 con 0x7f4a180c8b50 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556427792 0 0) 0x7f4a04003570 con 0x7f4a18155af0 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a18155f60 msgr2=0x7f4a180c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a18155f60 0x7f4a180c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 >> v1:172.21.15.167:6789/0 conn(0x7f4a180c8b50 legacy=0x7f4a1806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1806b120 con 0x7f4a18155af0 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f4a08001470 con 0x7f4a18155af0 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a18155f60 0x7f4a180c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 >> v1:172.21.15.33:6789/0 conn(0x7f4a18155af0 legacy=0x7f4a1806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a04003c20 0x7f4a04004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a04004810 0x7f4a04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a040051c0 0x7f4a0400dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1806b7e0 con 0x7f4a040051c0 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400e2c0 con 0x7f4a04004810 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.798+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400e480 con 0x7f4a04003c20 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a040051c0 0x7f4a0400dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1d39a700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a04004810 0x7f4a04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1db9b700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a04003c20 0x7f4a04004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 -- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a04003c20 msgr2=0x7f4a04004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a04003c20 0x7f4a04004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 -- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a04004810 msgr2=0x7f4a04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a04004810 0x7f4a04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 -- 172.21.15.33:0/1352047614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04003570 con 0x7f4a040051c0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.799+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a040051c0 0x7f4a0400dc70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4a18156330 tx=0x7f4a1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e79a8aef07da6ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000eb00 con 0x7f4a040051c0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1000f040 con 0x7f4a040051c0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1cb99700 1 -- 172.21.15.33:0/1352047614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000eeb0 con 0x7f4a040051c0 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/1352047614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a040051c0 msgr2=0x7f4a0400dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/1352047614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a040051c0 0x7f4a0400dc70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4a18156330 tx=0x7f4a1000b040 comp rx=0 tx=0).stop 2023-10-06T05:20:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1e39c700 1 -- 172.21.15.33:0/1352047614 reap_dead start 2023-10-06T05:20:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/1352047614 shutdown_connections 2023-10-06T05:20:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/1352047614 >> 172.21.15.33:0/1352047614 conn(0x7f4a1806d030 msgr2=0x7f4a1806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/1352047614 shutdown_connections 2023-10-06T05:20:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.800+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/1352047614 wait complete. 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.802+0000 7f4a1eb9d700 1 Processor -- start 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.802+0000 7f4a1eb9d700 1 -- start start 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a040051c0 0x7f4a18159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a180c2270 0x7f4a180c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 0x7f4a1816a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18150970 con 0x7f4a18169f00 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c2c20 con 0x7f4a040051c0 2023-10-06T05:20:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1816d550 con 0x7f4a180c2270 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1e39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 0x7f4a1816a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1e39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 0x7f4a1816a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57452/0 (socket says 172.21.15.33:57452) 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1e39c700 1 -- 172.21.15.33:0/427452369 learned_addr learned my addr 172.21.15.33:0/427452369 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1d39a700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a180c2270 0x7f4a180c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.803+0000 7f4a1db9b700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a040051c0 0x7f4a18159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1e39c700 1 -- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a180c2270 msgr2=0x7f4a180c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a180c2270 0x7f4a180c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1e39c700 1 -- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a040051c0 msgr2=0x7f4a18159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a040051c0 0x7f4a18159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1e39c700 1 -- 172.21.15.33:0/427452369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1816ddf0 con 0x7f4a18169f00 2023-10-06T05:20:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1e39c700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 0x7f4a1816a380 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e840 tx=0x7f4a10019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10007000 con 0x7f4a18169f00 2023-10-06T05:20:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1001f660 con 0x7f4a18169f00 2023-10-06T05:20:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1001f9e0 con 0x7f4a18169f00 2023-10-06T05:20:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1816acf0 con 0x7f4a18169f00 2023-10-06T05:20:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.804+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1816da80 con 0x7f4a18169f00 2023-10-06T05:20:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.805+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1804c380 con 0x7f4a18169f00 2023-10-06T05:20:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.806+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a1001e450 con 0x7f4a18169f00 2023-10-06T05:20:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.807+0000 7f4a0dffb700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49fc066f30 0x7f49fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.807+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4a10086fa0 con 0x7f4a18169f00 2023-10-06T05:20:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.808+0000 7f4a1db9b700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49fc066f30 0x7f49fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.808+0000 7f4a1db9b700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49fc066f30 0x7f49fc0693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007e40 tx=0x7f4a14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:42.811+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a10053f90 con 0x7f4a18169f00 2023-10-06T05:20:43.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.014+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a180c8fc0 con 0x7f4a18169f00 2023-10-06T05:20:43.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.015+0000 7f4a0dffb700 1 -- 172.21.15.33:0/427452369 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17060 (secure 0 0 0) 0x7f4a100575b0 con 0x7f4a18169f00 2023-10-06T05:20:43.017 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:43.017 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":78,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:34.725146+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:19:30.956591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:19:31.773352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3059565716":"2023-10-06T06:20:33.898361+0000","172.21.15.33:6835/2242593899":"2023-10-07T05:19:43.844349+0000","172.21.15.33:6834/2242593899":"2023-10-07T05:19:43.844349+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.017+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49fc066f30 msgr2=0x7f49fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.017+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49fc066f30 0x7f49fc0693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007e40 tx=0x7f4a14009040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.017+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 msgr2=0x7f4a1816a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.017+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 0x7f4a1816a380 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e840 tx=0x7f4a10019000 comp rx=0 tx=0).stop 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.017+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 shutdown_connections 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.017+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49fc066f30 0x7f49fc0693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.018+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a18169f00 0x7f4a1816a380 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.018+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a040051c0 0x7f4a18159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.018+0000 7f4a1eb9d700 1 --2- 172.21.15.33:0/427452369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a180c2270 0x7f4a180c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.018+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 >> 172.21.15.33:0/427452369 conn(0x7f4a1806d030 msgr2=0x7f4a1806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.018+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 shutdown_connections 2023-10-06T05:20:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.018+0000 7f4a1eb9d700 1 -- 172.21.15.33:0/427452369 wait complete. 2023-10-06T05:20:43.037 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:20:43.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.223+0000 7f87f339d700 1 Processor -- start 2023-10-06T05:20:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.223+0000 7f87f339d700 1 -- start start 2023-10-06T05:20:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.224+0000 7f87f339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 0x7f87ec0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f339d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec06ab60 con 0x7f87ec155af0 2023-10-06T05:20:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f339d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec06ae40 con 0x7f87ec0c8b50 2023-10-06T05:20:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec06b7e0 con 0x7f87ec155f60 2023-10-06T05:20:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f1b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87ec155af0 0x7f87ec06a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43320/0 (socket says 172.21.15.33:43320) 2023-10-06T05:20:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f1b9a700 1 -- 172.21.15.33:0/712071931 learned_addr learned my addr 172.21.15.33:0/712071931 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 0x7f87ec0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550521235 0 0) 0x7f87ec06ab60 con 0x7f87ec155af0 2023-10-06T05:20:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d8003570 con 0x7f87ec155af0 2023-10-06T05:20:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.225+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 734665678 0 0) 0x7f87d8003570 con 0x7f87ec155af0 2023-10-06T05:20:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 msgr2=0x7f87ec0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 0x7f87ec0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 >> v1:172.21.15.167:6789/0 conn(0x7f87ec0c8b50 legacy=0x7f87ec06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec06b120 con 0x7f87ec155af0 2023-10-06T05:20:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f87dc001470 con 0x7f87ec155af0 2023-10-06T05:20:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 >> v1:172.21.15.33:6789/0 conn(0x7f87ec155af0 legacy=0x7f87ec06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 0x7f87d8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d8003fd0 0x7f87d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d8004d50 0x7f87d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec06b7e0 con 0x7f87d8004d50 2023-10-06T05:20:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d800bb40 con 0x7f87d8003fd0 2023-10-06T05:20:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d800df10 con 0x7f87ec155f60 2023-10-06T05:20:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d8004d50 0x7f87d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f239b700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 0x7f87d8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f1b9a700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d8003fd0 0x7f87d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f2b9c700 1 -- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 msgr2=0x7f87d8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec155f60 0x7f87d8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f2b9c700 1 -- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d8003fd0 msgr2=0x7f87d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87d8003fd0 0x7f87d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.226+0000 7f87f2b9c700 1 -- 172.21.15.33:0/712071931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8003570 con 0x7f87d8004d50 2023-10-06T05:20:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.227+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d8004d50 0x7f87d80051c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f87e400e210 tx=0x7f87e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe6e113dded3eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.227+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400eac0 con 0x7f87d8004d50 2023-10-06T05:20:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.227+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e400f040 con 0x7f87d8004d50 2023-10-06T05:20:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.227+0000 7f87f1399700 1 -- 172.21.15.33:0/712071931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400ee70 con 0x7f87d8004d50 2023-10-06T05:20:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f339d700 1 -- 172.21.15.33:0/712071931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d8004d50 msgr2=0x7f87d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f339d700 1 --2- 172.21.15.33:0/712071931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d8004d50 0x7f87d80051c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f87e400e210 tx=0x7f87e400b040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f2b9c700 1 -- 172.21.15.33:0/712071931 reap_dead start 2023-10-06T05:20:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f339d700 1 -- 172.21.15.33:0/712071931 shutdown_connections 2023-10-06T05:20:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f339d700 1 -- 172.21.15.33:0/712071931 >> 172.21.15.33:0/712071931 conn(0x7f87ec06d030 msgr2=0x7f87ec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f339d700 1 -- 172.21.15.33:0/712071931 shutdown_connections 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.228+0000 7f87f339d700 1 -- 172.21.15.33:0/712071931 wait complete. 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.229+0000 7f87f339d700 1 Processor -- start 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.229+0000 7f87f339d700 1 -- start start 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ec167160 0x7f87ec16a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec1567e0 0x7f87ec1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 0x7f87ec0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec159060 con 0x7f87ec158be0 2023-10-06T05:20:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0c5fe0 con 0x7f87ec167160 2023-10-06T05:20:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec157940 con 0x7f87ec1567e0 2023-10-06T05:20:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 0x7f87ec0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 0x7f87ec0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57472/0 (socket says 172.21.15.33:57472) 2023-10-06T05:20:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.230+0000 7f87f2b9c700 1 -- 172.21.15.33:0/2825363790 learned_addr learned my addr 172.21.15.33:0/2825363790 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f2b9c700 1 -- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec1567e0 msgr2=0x7f87ec1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec1567e0 0x7f87ec1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f239b700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ec167160 0x7f87ec16a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f2b9c700 1 -- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ec167160 msgr2=0x7f87ec16a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ec167160 0x7f87ec16a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f2b9c700 1 -- 172.21.15.33:0/2825363790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec16ab40 con 0x7f87ec158be0 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f239b700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ec167160 0x7f87ec16a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f2b9c700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 0x7f87ec0c3190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f87d800bb10 tx=0x7f87e401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4027430 con 0x7f87ec158be0 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4022070 con 0x7f87ec158be0 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4027820 con 0x7f87ec158be0 2023-10-06T05:20:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.231+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec156d30 con 0x7f87ec158be0 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.232+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1572f0 con 0x7f87ec158be0 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.232+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec157ab0 con 0x7f87ec158be0 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.233+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87e4024030 con 0x7f87ec158be0 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.234+0000 7f87e27fc700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87d0066f30 0x7f87d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.234+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f87e4086130 con 0x7f87ec158be0 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.235+0000 7f87f239b700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87d0066f30 0x7f87d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.235+0000 7f87f239b700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87d0066f30 0x7f87d00693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000f60 tx=0x7f87e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.238+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87e4053120 con 0x7f87ec158be0 2023-10-06T05:20:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.487+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87ec157cd0 con 0x7f87ec158be0 2023-10-06T05:20:43.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.488+0000 7f87e27fc700 1 -- 172.21.15.33:0/2825363790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f87e4056740 con 0x7f87ec158be0 2023-10-06T05:20:43.489 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:43.489 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:20:36.733515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87d0066f30 msgr2=0x7f87d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87d0066f30 0x7f87d00693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000f60 tx=0x7f87e8009040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 msgr2=0x7f87ec0c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 0x7f87ec0c3190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f87d800bb10 tx=0x7f87e401a000 comp rx=0 tx=0).stop 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 shutdown_connections 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87d0066f30 0x7f87d00693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158be0 0x7f87ec0c3190 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87ec167160 0x7f87ec16a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 --2- 172.21.15.33:0/2825363790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87ec1567e0 0x7f87ec1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 >> 172.21.15.33:0/2825363790 conn(0x7f87ec06d030 msgr2=0x7f87ec0c6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 shutdown_connections 2023-10-06T05:20:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.490+0000 7f87f339d700 1 -- 172.21.15.33:0/2825363790 wait complete. 2023-10-06T05:20:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2023-10-06T05:20:43.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.695+0000 7fa93279d700 1 Processor -- start 2023-10-06T05:20:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.695+0000 7fa93279d700 1 -- start start 2023-10-06T05:20:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.696+0000 7fa93279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa92c0cf3b0 0x7fa92c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.696+0000 7fa93279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa92c0af0a0 0x7fa92c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.696+0000 7fa93279d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c0cbd80 con 0x7fa92c0ccfc0 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.696+0000 7fa93279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92c0af5f0 con 0x7fa92c0cf3b0 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.696+0000 7fa93279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92c0af8d0 con 0x7fa92c0af0a0 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa92c0cf3b0 0x7fa92c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa92c0cf3b0 0x7fa92c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57488/0 (socket says 172.21.15.33:57488) 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 -- 172.21.15.33:0/1703537810 learned_addr learned my addr 172.21.15.33:0/1703537810 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa93179b700 1 --1- 172.21.15.33:0/1703537810 >> v1:172.21.15.33:6789/0 conn(0x7fa92c0ccfc0 0x7fa92c0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43322/0 (socket says 172.21.15.33:43322) 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa931f9c700 1 --2- 172.21.15.33:0/1703537810 >> v2:172.21.15.167:3300/0 conn(0x7fa92c0af0a0 0x7fa92c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 -- 172.21.15.33:0/1703537810 >> v2:172.21.15.167:3300/0 conn(0x7fa92c0af0a0 msgr2=0x7fa92c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 --2- 172.21.15.33:0/1703537810 >> v2:172.21.15.167:3300/0 conn(0x7fa92c0af0a0 0x7fa92c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 -- 172.21.15.33:0/1703537810 >> v1:172.21.15.33:6789/0 conn(0x7fa92c0ccfc0 legacy=0x7fa92c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 -- 172.21.15.33:0/1703537810 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c0afbb0 con 0x7fa92c0cf3b0 2023-10-06T05:20:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.697+0000 7fa930f9a700 1 --2- 172.21.15.33:0/1703537810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c0cf3b0 0x7fa92c1594a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0099b0 tx=0x7fa91c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6694406383cf38ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa923fff700 1 -- 172.21.15.33:0/1703537810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00b820 con 0x7fa92c0cf3b0 2023-10-06T05:20:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa923fff700 1 -- 172.21.15.33:0/1703537810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c00b980 con 0x7fa92c0cf3b0 2023-10-06T05:20:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa923fff700 1 -- 172.21.15.33:0/1703537810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00bd20 con 0x7fa92c0cf3b0 2023-10-06T05:20:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 -- 172.21.15.33:0/1703537810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c0cf3b0 msgr2=0x7fa92c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 --2- 172.21.15.33:0/1703537810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c0cf3b0 0x7fa92c1594a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0099b0 tx=0x7fa91c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 -- 172.21.15.33:0/1703537810 shutdown_connections 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 --2- 172.21.15.33:0/1703537810 >> v2:172.21.15.167:3300/0 conn(0x7fa92c0af0a0 0x7fa92c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 --2- 172.21.15.33:0/1703537810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c0cf3b0 0x7fa92c1594a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 -- 172.21.15.33:0/1703537810 >> 172.21.15.33:0/1703537810 conn(0x7fa92c06cff0 msgr2=0x7fa92c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 -- 172.21.15.33:0/1703537810 shutdown_connections 2023-10-06T05:20:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.698+0000 7fa93279d700 1 -- 172.21.15.33:0/1703537810 wait complete. 2023-10-06T05:20:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.700+0000 7fa93279d700 1 Processor -- start 2023-10-06T05:20:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.700+0000 7fa93279d700 1 -- start start 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa92c0af0a0 0x7fa92c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa92c0ccfc0 0x7fa92c154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 0x7fa92c0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0c8540 con 0x7fa92c15ee40 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c15a1d0 con 0x7fa92c0ccfc0 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0c9090 con 0x7fa92c0af0a0 2023-10-06T05:20:43.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa931f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 0x7fa92c0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa931f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 0x7fa92c0c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57496/0 (socket says 172.21.15.33:57496) 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa931f9c700 1 -- 172.21.15.33:0/3188611115 learned_addr learned my addr 172.21.15.33:0/3188611115 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa930f9a700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa92c0ccfc0 0x7fa92c154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.701+0000 7fa93179b700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa92c0af0a0 0x7fa92c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa931f9c700 1 -- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa92c0af0a0 msgr2=0x7fa92c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa931f9c700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa92c0af0a0 0x7fa92c0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa931f9c700 1 -- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa92c0ccfc0 msgr2=0x7fa92c154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa931f9c700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa92c0ccfc0 0x7fa92c154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa931f9c700 1 -- 172.21.15.33:0/3188611115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0094f0 con 0x7fa92c15ee40 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa930f9a700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa92c0ccfc0 0x7fa92c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:43.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa931f9c700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 0x7fa92c0c6480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa924007f20 tx=0x7fa924009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400ebc0 con 0x7fa92c15ee40 2023-10-06T05:20:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92400ed20 con 0x7fa92c15ee40 2023-10-06T05:20:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924014670 con 0x7fa92c15ee40 2023-10-06T05:20:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c0c75b0 con 0x7fa92c15ee40 2023-10-06T05:20:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.702+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c0c7b70 con 0x7fa92c15ee40 2023-10-06T05:20:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.703+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92c0c7d00 con 0x7fa92c15ee40 2023-10-06T05:20:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.704+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa924014a20 con 0x7fa92c15ee40 2023-10-06T05:20:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.705+0000 7fa9217fa700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa910066f30 0x7fa9100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.705+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa924084df0 con 0x7fa92c15ee40 2023-10-06T05:20:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.706+0000 7fa93179b700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa910066f30 0x7fa9100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.706+0000 7fa93179b700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa910066f30 0x7fa9100693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa928004fd0 tx=0x7fa928007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.709+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa924051de0 con 0x7fa92c15ee40 2023-10-06T05:20:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.960+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92c04c380 con 0x7fa92c15ee40 2023-10-06T05:20:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.961+0000 7fa9217fa700 1 -- 172.21.15.33:0/3188611115 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fa924055400 con 0x7fa92c15ee40 2023-10-06T05:20:43.962 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:43.962 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:19:32.778469+0000","modified":"2023-10-06T05:20:36.733515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3981055252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3981055252},{"type":"v1","addr":"172.21.15.33:6835","nonce":3981055252}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa910066f30 msgr2=0x7fa9100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa910066f30 0x7fa9100693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa928004fd0 tx=0x7fa928007eb0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 msgr2=0x7fa92c0c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 0x7fa92c0c6480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa924007f20 tx=0x7fa924009840 comp rx=0 tx=0).stop 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 shutdown_connections 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa910066f30 0x7fa9100693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa92c15ee40 0x7fa92c0c6480 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa92c0af0a0 0x7fa92c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 --2- 172.21.15.33:0/3188611115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa92c0ccfc0 0x7fa92c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 >> 172.21.15.33:0/3188611115 conn(0x7fa92c06cff0 msgr2=0x7fa92c155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 shutdown_connections 2023-10-06T05:20:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:43.963+0000 7fa93279d700 1 -- 172.21.15.33:0/3188611115 wait complete. 2023-10-06T05:20:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2023-10-06T05:20:43.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.168+0000 7f09edf9d700 1 Processor -- start 2023-10-06T05:20:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.168+0000 7f09edf9d700 1 -- start start 2023-10-06T05:20:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09edf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09e806e140 0x7f09e815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09edf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f09e80afdd0 0x7f09e80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09edf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e806f220 con 0x7f09e8159530 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09edf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e80b1e00 con 0x7f09e806e140 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09edf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e806f6b0 con 0x7f09e80afdd0 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09e806e140 0x7f09e815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09e806e140 0x7f09e815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57498/0 (socket says 172.21.15.33:57498) 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 -- 172.21.15.33:0/1826779301 learned_addr learned my addr 172.21.15.33:0/1826779301 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 -- 172.21.15.33:0/1826779301 >> v2:172.21.15.167:3300/0 conn(0x7f09e80afdd0 msgr2=0x7f09e80b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09ed79c700 1 --2- 172.21.15.33:0/1826779301 >> v2:172.21.15.167:3300/0 conn(0x7f09e80afdd0 0x7f09e80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 --2- 172.21.15.33:0/1826779301 >> v2:172.21.15.167:3300/0 conn(0x7f09e80afdd0 0x7f09e80b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 -- 172.21.15.33:0/1826779301 >> v1:172.21.15.167:6789/0 conn(0x7f09e8159530 legacy=0x7f09e806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09e7fff700 1 -- 172.21.15.33:0/1826779301 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e815ae50 con 0x7f09e806e140 2023-10-06T05:20:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.170+0000 7f09ed79c700 1 --2- 172.21.15.33:0/1826779301 >> v2:172.21.15.167:3300/0 conn(0x7f09e80afdd0 0x7f09e80b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.171+0000 7f09e7fff700 1 --2- 172.21.15.33:0/1826779301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e815a910 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f09e8071970 tx=0x7f09d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2417ea78ad5179 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.171+0000 7f09e77fe700 1 -- 172.21.15.33:0/1826779301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400b7e0 con 0x7f09e806e140 2023-10-06T05:20:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.171+0000 7f09e77fe700 1 -- 172.21.15.33:0/1826779301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d400b940 con 0x7f09e806e140 2023-10-06T05:20:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.171+0000 7f09e77fe700 1 -- 172.21.15.33:0/1826779301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400bce0 con 0x7f09e806e140 2023-10-06T05:20:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.171+0000 7f09edf9d700 1 -- 172.21.15.33:0/1826779301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 msgr2=0x7f09e815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.171+0000 7f09edf9d700 1 --2- 172.21.15.33:0/1826779301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e815a910 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f09e8071970 tx=0x7f09d4009250 comp rx=0 tx=0).stop 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.172+0000 7f09edf9d700 1 -- 172.21.15.33:0/1826779301 shutdown_connections 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.172+0000 7f09edf9d700 1 --2- 172.21.15.33:0/1826779301 >> v2:172.21.15.167:3300/0 conn(0x7f09e80afdd0 0x7f09e80b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.172+0000 7f09edf9d700 1 --2- 172.21.15.33:0/1826779301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e815a910 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.172+0000 7f09edf9d700 1 -- 172.21.15.33:0/1826779301 >> 172.21.15.33:0/1826779301 conn(0x7f09e806c560 msgr2=0x7f09e806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.172+0000 7f09edf9d700 1 -- 172.21.15.33:0/1826779301 shutdown_connections 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.172+0000 7f09edf9d700 1 -- 172.21.15.33:0/1826779301 wait complete. 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.173+0000 7f09edf9d700 1 Processor -- start 2023-10-06T05:20:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.173+0000 7f09edf9d700 1 -- start start 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09edf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e816a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09edf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e80afdd0 0x7f09e80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09edf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09e8159530 0x7f09e815d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09edf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e816ca40 con 0x7f09e806e140 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09edf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80ced60 con 0x7f09e80afdd0 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09edf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80ce150 con 0x7f09e8159530 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09ecf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e816a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09ecf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e816a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57502/0 (socket says 172.21.15.33:57502) 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.174+0000 7f09ecf9b700 1 -- 172.21.15.33:0/3865710970 learned_addr learned my addr 172.21.15.33:0/3865710970 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09e7fff700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e80afdd0 0x7f09e80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ed79c700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09e8159530 0x7f09e815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ecf9b700 1 -- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09e8159530 msgr2=0x7f09e815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ecf9b700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09e8159530 0x7f09e815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ecf9b700 1 -- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e80afdd0 msgr2=0x7f09e80cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ecf9b700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e80afdd0 0x7f09e80cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ecf9b700 1 -- 172.21.15.33:0/3865710970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d40094f0 con 0x7f09e806e140 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09e7fff700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e80afdd0 0x7f09e80cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09ecf9b700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e816a1c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00b610 tx=0x7f09dc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00f510 con 0x7f09e806e140 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc00f670 con 0x7f09e806e140 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00fa30 con 0x7f09e806e140 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.175+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e80ce430 con 0x7f09e806e140 2023-10-06T05:20:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.176+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e80ce9f0 con 0x7f09e806e140 2023-10-06T05:20:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.176+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e804c380 con 0x7f09e806e140 2023-10-06T05:20:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.177+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09dc01a400 con 0x7f09e806e140 2023-10-06T05:20:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.179+0000 7f09e4ff9700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.179+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f09dc0842e0 con 0x7f09e806e140 2023-10-06T05:20:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.179+0000 7f09e7fff700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.180+0000 7f09e7fff700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09cc066f30 0x7f09cc0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09e8071970 tx=0x7f09d4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.183+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09dc051350 con 0x7f09e806e140 2023-10-06T05:20:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.392+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09e806b130 con 0x7f09e806e140 2023-10-06T05:20:44.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.393+0000 7f09e4ff9700 1 -- 172.21.15.33:0/3865710970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17060 (secure 0 0 0) 0x7f09dc054970 con 0x7f09e806e140 2023-10-06T05:20:44.395 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:44.396 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":78,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:34.725146+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:19:30.956591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:19:31.773352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3059565716":"2023-10-06T06:20:33.898361+0000","172.21.15.33:6835/2242593899":"2023-10-07T05:19:43.844349+0000","172.21.15.33:6834/2242593899":"2023-10-07T05:19:43.844349+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09cc066f30 msgr2=0x7f09cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09cc066f30 0x7f09cc0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09e8071970 tx=0x7f09d4016000 comp rx=0 tx=0).stop 2023-10-06T05:20:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 msgr2=0x7f09e816a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e816a1c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00b610 tx=0x7f09dc009120 comp rx=0 tx=0).stop 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 shutdown_connections 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09e8159530 0x7f09e815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09e806e140 0x7f09e816a1c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 --2- 172.21.15.33:0/3865710970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e80afdd0 0x7f09e80cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 >> 172.21.15.33:0/3865710970 conn(0x7f09e806c560 msgr2=0x7f09e8158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 shutdown_connections 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.395+0000 7f09edf9d700 1 -- 172.21.15.33:0/3865710970 wait complete. 2023-10-06T05:20:44.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:20:44.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.601+0000 7f4c9959d700 1 Processor -- start 2023-10-06T05:20:44.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.601+0000 7f4c9959d700 1 -- start start 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.602+0000 7f4c9959d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4c94159400 0x7f4c940cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.602+0000 7f4c9959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.602+0000 7f4c9959d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c940cc770 0x7f4c9406c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.602+0000 7f4c9959d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940d0ed0 con 0x7f4c940cc770 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.602+0000 7f4c9959d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940cb6d0 con 0x7f4c94159400 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.602+0000 7f4c9959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940cb9b0 con 0x7f4c941597d0 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c940cc770 0x7f4c9406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c940cc770 0x7f4c9406c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57506/0 (socket says 172.21.15.33:57506) 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 -- 172.21.15.33:0/1501006507 learned_addr learned my addr 172.21.15.33:0/1501006507 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:44.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c937fe700 1 --2- 172.21.15.33:0/1501006507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c93fff700 1 --2- 172.21.15.33:0/1501006507 >> v2:172.21.15.167:3300/0 conn(0x7f4c94159400 0x7f4c940cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 -- 172.21.15.33:0/1501006507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 msgr2=0x7f4c940cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 --2- 172.21.15.33:0/1501006507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 -- 172.21.15.33:0/1501006507 >> v2:172.21.15.167:3300/0 conn(0x7f4c94159400 msgr2=0x7f4c940cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 --2- 172.21.15.33:0/1501006507 >> v2:172.21.15.167:3300/0 conn(0x7f4c94159400 0x7f4c940cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 -- 172.21.15.33:0/1501006507 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940cbc90 con 0x7f4c940cc770 2023-10-06T05:20:44.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.603+0000 7f4c98d9c700 1 --2- 172.21.15.33:0/1501006507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c9406c560 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4c9406c030 tx=0x7f4c8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2585812aa9b0bf92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c92ffd700 1 -- 172.21.15.33:0/1501006507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00e070 con 0x7f4c940cc770 2023-10-06T05:20:44.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c92ffd700 1 -- 172.21.15.33:0/1501006507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00ba50 con 0x7f4c940cc770 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c92ffd700 1 -- 172.21.15.33:0/1501006507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0092e0 con 0x7f4c940cc770 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 -- 172.21.15.33:0/1501006507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 msgr2=0x7f4c9406c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 --2- 172.21.15.33:0/1501006507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c9406c560 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4c9406c030 tx=0x7f4c8c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 -- 172.21.15.33:0/1501006507 shutdown_connections 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 --2- 172.21.15.33:0/1501006507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c9406c560 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 --2- 172.21.15.33:0/1501006507 >> v2:172.21.15.167:3300/0 conn(0x7f4c94159400 0x7f4c940cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 --2- 172.21.15.33:0/1501006507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 -- 172.21.15.33:0/1501006507 >> 172.21.15.33:0/1501006507 conn(0x7f4c9406e4b0 msgr2=0x7f4c940d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 -- 172.21.15.33:0/1501006507 shutdown_connections 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.604+0000 7f4c9959d700 1 -- 172.21.15.33:0/1501006507 wait complete. 2023-10-06T05:20:44.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.606+0000 7f4c9959d700 1 Processor -- start 2023-10-06T05:20:44.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.606+0000 7f4c9959d700 1 -- start start 2023-10-06T05:20:44.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c9959d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c940b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c9959d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c94159400 0x7f4c940b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c9959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c9959d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b61e0 con 0x7f4c940cc770 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c9959d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9414e690 con 0x7f4c94159400 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c9959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940c8300 con 0x7f4c941597d0 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c940b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c940b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57512/0 (socket says 172.21.15.33:57512) 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c93fff700 1 -- 172.21.15.33:0/3254956452 learned_addr learned my addr 172.21.15.33:0/3254956452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c937fe700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c94159400 0x7f4c940b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.607+0000 7f4c98d9c700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c93fff700 1 -- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 msgr2=0x7f4c940b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c93fff700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c93fff700 1 -- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c94159400 msgr2=0x7f4c940b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c93fff700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c94159400 0x7f4c940b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c93fff700 1 -- 172.21.15.33:0/3254956452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009ae0 con 0x7f4c940cc770 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c98d9c700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c93fff700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c940b5af0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d8c0 tx=0x7f4c84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840125f0 con 0x7f4c940cc770 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84012750 con 0x7f4c940cc770 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84012af0 con 0x7f4c940cc770 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c940c85e0 con 0x7f4c940cc770 2023-10-06T05:20:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.608+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c940c8ba0 con 0x7f4c940cc770 2023-10-06T05:20:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.609+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c940c8f10 con 0x7f4c940cc770 2023-10-06T05:20:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.610+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c8401e460 con 0x7f4c940cc770 2023-10-06T05:20:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.611+0000 7f4c7bfff700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.611+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4c84084d50 con 0x7f4c940cc770 2023-10-06T05:20:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.612+0000 7f4c937fe700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.612+0000 7f4c937fe700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c80066f30 0x7f4c800693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4c94151a90 tx=0x7f4c88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.615+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c84051d40 con 0x7f4c940cc770 2023-10-06T05:20:44.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.866+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c940b2d40 con 0x7f4c940cc770 2023-10-06T05:20:44.868 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:20:44.867+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:20:44.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.871+0000 7f4c7bfff700 1 -- 172.21.15.33:0/3254956452 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4c84055360 con 0x7f4c940cc770 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c80066f30 msgr2=0x7f4c800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c80066f30 0x7f4c800693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4c94151a90 tx=0x7f4c88008040 comp rx=0 tx=0).stop 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 msgr2=0x7f4c940b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c940b5af0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d8c0 tx=0x7f4c84008040 comp rx=0 tx=0).stop 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 shutdown_connections 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c941597d0 0x7f4c940b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c940cc770 0x7f4c940b5af0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 --2- 172.21.15.33:0/3254956452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c94159400 0x7f4c940b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 >> 172.21.15.33:0/3254956452 conn(0x7f4c9406e4b0 msgr2=0x7f4c940cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 shutdown_connections 2023-10-06T05:20:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:44.874+0000 7f4c9959d700 1 -- 172.21.15.33:0/3254956452 wait complete. 2023-10-06T05:20:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:20:44.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:20:44.925 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:20:45.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.084+0000 7f5bfe19d700 1 Processor -- start 2023-10-06T05:20:45.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.085+0000 7f5bfe19d700 1 -- start start 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfe19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf815acd0 0x7f5bf80abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfe19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf80b0cd0 con 0x7f5bf806efa0 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfe19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf815a840 con 0x7f5bf8159850 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfe19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8159f10 con 0x7f5bf815acd0 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfd99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5bf806efa0 0x7f5bf80ac3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43336/0 (socket says 172.21.15.33:43336) 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfd99c700 1 -- 172.21.15.33:0/1692983471 learned_addr learned my addr 172.21.15.33:0/1692983471 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:45.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.086+0000 7f5bfc99a700 1 --2- 172.21.15.33:0/1692983471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf815acd0 0x7f5bf80abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480735901 0 0) 0x7f5bf80b0cd0 con 0x7f5bf806efa0 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be4003570 con 0x7f5bf806efa0 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557839073 0 0) 0x7f5bf815a840 con 0x7f5bf8159850 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf80b0cd0 con 0x7f5bf8159850 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5bfc99a700 1 -- 172.21.15.33:0/1692983471 >> v1:172.21.15.167:6789/0 conn(0x7f5bf8159850 legacy=0x7f5bf806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5bfc99a700 1 -- 172.21.15.33:0/1692983471 >> v1:172.21.15.33:6789/0 conn(0x7f5bf806efa0 legacy=0x7f5bf80ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5bfc99a700 1 -- 172.21.15.33:0/1692983471 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf815a1f0 con 0x7f5bf815acd0 2023-10-06T05:20:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.087+0000 7f5bfc99a700 1 --2- 172.21.15.33:0/1692983471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf815acd0 0x7f5bf80abe90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009910 tx=0x7f5be80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652b628971f88844 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8011630 con 0x7f5bf815acd0 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8011790 con 0x7f5bf815acd0 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5beffff700 1 -- 172.21.15.33:0/1692983471 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8011b30 con 0x7f5bf815acd0 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 -- 172.21.15.33:0/1692983471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf815acd0 msgr2=0x7f5bf80abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/1692983471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf815acd0 0x7f5bf80abe90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009910 tx=0x7f5be80091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 -- 172.21.15.33:0/1692983471 shutdown_connections 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/1692983471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf815acd0 0x7f5bf80abe90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 -- 172.21.15.33:0/1692983471 >> 172.21.15.33:0/1692983471 conn(0x7f5bf806cd40 msgr2=0x7f5bf80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 -- 172.21.15.33:0/1692983471 shutdown_connections 2023-10-06T05:20:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.088+0000 7f5bfe19d700 1 -- 172.21.15.33:0/1692983471 wait complete. 2023-10-06T05:20:45.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.090+0000 7f5bfe19d700 1 Processor -- start 2023-10-06T05:20:45.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.090+0000 7f5bfe19d700 1 -- start start 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfe19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf806efa0 0x7f5bf8157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfe19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8159850 0x7f5bf80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfe19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 0x7f5bf80c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfe19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80b2960 con 0x7f5bf815acd0 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfe19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80b2c40 con 0x7f5bf806efa0 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfe19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80b2f20 con 0x7f5bf8159850 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfd99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 0x7f5bf80c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfd99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 0x7f5bf80c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57530/0 (socket says 172.21.15.33:57530) 2023-10-06T05:20:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfd99c700 1 -- 172.21.15.33:0/2786991577 learned_addr learned my addr 172.21.15.33:0/2786991577 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfc99a700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8159850 0x7f5bf80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.091+0000 7f5bfd19b700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf806efa0 0x7f5bf8157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfd99c700 1 -- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8159850 msgr2=0x7f5bf80b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfd99c700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8159850 0x7f5bf80b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfd99c700 1 -- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf806efa0 msgr2=0x7f5bf8157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfd99c700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf806efa0 0x7f5bf8157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfd99c700 1 -- 172.21.15.33:0/2786991577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8009420 con 0x7f5bf815acd0 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfc99a700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8159850 0x7f5bf80b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfd99c700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 0x7f5bf80c8390 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000cc00 tx=0x7f5bf0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000a980 con 0x7f5bf815acd0 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf000ab00 con 0x7f5bf815acd0 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00070f0 con 0x7f5bf815acd0 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf80cc5b0 con 0x7f5bf815acd0 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.092+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf8158370 con 0x7f5bf815acd0 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.093+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf80ccad0 con 0x7f5bf815acd0 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.094+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bf00074a0 con 0x7f5bf815acd0 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.095+0000 7f5bed7fa700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bdc066f30 0x7f5bdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.095+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f5bf0084c30 con 0x7f5bf815acd0 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.096+0000 7f5bfd19b700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bdc066f30 0x7f5bdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.096+0000 7f5bfd19b700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bdc066f30 0x7f5bdc0693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80b4120 tx=0x7f5bf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.099+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bf80ccad0 con 0x7f5bf815acd0 2023-10-06T05:20:45.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.350+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5bf80ccad0 con 0x7f5bf815acd0 2023-10-06T05:20:45.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.872+0000 7f5bed7fa700 1 -- 172.21.15.33:0/2786991577 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5bf80ccad0 con 0x7f5bf815acd0 2023-10-06T05:20:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bdc066f30 msgr2=0x7f5bdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bdc066f30 0x7f5bdc0693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80b4120 tx=0x7f5bf4009040 comp rx=0 tx=0).stop 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 msgr2=0x7f5bf80c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 0x7f5bf80c8390 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000cc00 tx=0x7f5bf0001a80 comp rx=0 tx=0).stop 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 shutdown_connections 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bdc066f30 0x7f5bdc0693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bf815acd0 0x7f5bf80c8390 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf806efa0 0x7f5bf8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 --2- 172.21.15.33:0/2786991577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bf8159850 0x7f5bf80b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 >> 172.21.15.33:0/2786991577 conn(0x7f5bf806cd40 msgr2=0x7f5bf8069340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 shutdown_connections 2023-10-06T05:20:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:45.876+0000 7f5bfe19d700 1 -- 172.21.15.33:0/2786991577 wait complete. 2023-10-06T05:20:45.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:20:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.135+0000 7fe85d39d700 1 Processor -- start 2023-10-06T05:20:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.135+0000 7fe85d39d700 1 -- start start 2023-10-06T05:20:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.136+0000 7fe85d39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe85806dc00 0x7fe8581590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.136+0000 7fe85d39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe85815a740 0x7fe858159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.136+0000 7fe85d39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85815d280 con 0x7fe85806dfd0 2023-10-06T05:20:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.136+0000 7fe85d39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85815e290 con 0x7fe85806dc00 2023-10-06T05:20:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.136+0000 7fe85d39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85815ab90 con 0x7fe85815a740 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe85806dc00 0x7fe8581590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe85806dc00 0x7fe8581590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57532/0 (socket says 172.21.15.33:57532) 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 -- 172.21.15.33:0/3511135328 learned_addr learned my addr 172.21.15.33:0/3511135328 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe85cb9c700 1 --2- 172.21.15.33:0/3511135328 >> v2:172.21.15.167:3300/0 conn(0x7fe85815a740 0x7fe858159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 -- 172.21.15.33:0/3511135328 >> v2:172.21.15.167:3300/0 conn(0x7fe85815a740 msgr2=0x7fe858159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 --2- 172.21.15.33:0/3511135328 >> v2:172.21.15.167:3300/0 conn(0x7fe85815a740 0x7fe858159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 -- 172.21.15.33:0/3511135328 >> v1:172.21.15.167:6789/0 conn(0x7fe85806dfd0 legacy=0x7fe85815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 -- 172.21.15.33:0/3511135328 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85815de00 con 0x7fe85806dc00 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe85cb9c700 1 --2- 172.21.15.33:0/3511135328 >> v2:172.21.15.167:3300/0 conn(0x7fe85815a740 0x7fe858159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.137+0000 7fe857fff700 1 --2- 172.21.15.33:0/3511135328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85806dc00 0x7fe8581590e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe8580b29b0 tx=0x7fe848009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aa24069fa00ffa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe856ffd700 1 -- 172.21.15.33:0/3511135328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800b820 con 0x7fe85806dc00 2023-10-06T05:20:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe856ffd700 1 -- 172.21.15.33:0/3511135328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84800b980 con 0x7fe85806dc00 2023-10-06T05:20:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe856ffd700 1 -- 172.21.15.33:0/3511135328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800bd20 con 0x7fe85806dc00 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 -- 172.21.15.33:0/3511135328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85806dc00 msgr2=0x7fe8581590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 --2- 172.21.15.33:0/3511135328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85806dc00 0x7fe8581590e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe8580b29b0 tx=0x7fe848009250 comp rx=0 tx=0).stop 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 -- 172.21.15.33:0/3511135328 shutdown_connections 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 --2- 172.21.15.33:0/3511135328 >> v2:172.21.15.167:3300/0 conn(0x7fe85815a740 0x7fe858159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 --2- 172.21.15.33:0/3511135328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85806dc00 0x7fe8581590e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 -- 172.21.15.33:0/3511135328 >> 172.21.15.33:0/3511135328 conn(0x7fe8580d0550 msgr2=0x7fe8580d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 -- 172.21.15.33:0/3511135328 shutdown_connections 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.138+0000 7fe85d39d700 1 -- 172.21.15.33:0/3511135328 wait complete. 2023-10-06T05:20:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.140+0000 7fe85d39d700 1 Processor -- start 2023-10-06T05:20:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.140+0000 7fe85d39d700 1 -- start start 2023-10-06T05:20:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85d39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe85806dc00 0x7fe8580b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85d39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe85806dfd0 0x7fe8580c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85d39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 0x7fe85815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85d39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580b3830 con 0x7fe85815a740 2023-10-06T05:20:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85d39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580ca420 con 0x7fe85806dfd0 2023-10-06T05:20:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85d39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580ca180 con 0x7fe85806dc00 2023-10-06T05:20:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85cb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 0x7fe85815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85cb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 0x7fe85815b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57538/0 (socket says 172.21.15.33:57538) 2023-10-06T05:20:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85cb9c700 1 -- 172.21.15.33:0/2742797514 learned_addr learned my addr 172.21.15.33:0/2742797514 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe857fff700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe85806dc00 0x7fe8580b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85cb9c700 1 -- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe85806dc00 msgr2=0x7fe8580b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85cb9c700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe85806dc00 0x7fe8580b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe8577fe700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe85806dfd0 0x7fe8580c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.141+0000 7fe85cb9c700 1 -- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe85806dfd0 msgr2=0x7fe8580c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe85cb9c700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe85806dfd0 0x7fe8580c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe85cb9c700 1 -- 172.21.15.33:0/2742797514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480094f0 con 0x7fe85815a740 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe857fff700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe85806dc00 0x7fe8580b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe8577fe700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe85806dfd0 0x7fe8580c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:46.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe85cb9c700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 0x7fe85815b000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe85000a380 tx=0x7fe850007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000ae10 con 0x7fe85815a740 2023-10-06T05:20:46.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85000d650 con 0x7fe85815a740 2023-10-06T05:20:46.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000da10 con 0x7fe85815a740 2023-10-06T05:20:46.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8580b1620 con 0x7fe85815a740 2023-10-06T05:20:46.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.142+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580b1be0 con 0x7fe85815a740 2023-10-06T05:20:46.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.143+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe858063ff0 con 0x7fe85815a740 2023-10-06T05:20:46.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.144+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe85001e3b0 con 0x7fe85815a740 2023-10-06T05:20:46.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.145+0000 7fe83ffff700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe844066f30 0x7fe8440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.146+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe850085220 con 0x7fe85815a740 2023-10-06T05:20:46.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.146+0000 7fe857fff700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe844066f30 0x7fe8440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.147+0000 7fe857fff700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe844066f30 0x7fe8440693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8580b29b0 tx=0x7fe848016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.150+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8500521b0 con 0x7fe85815a740 2023-10-06T05:20:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.352+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe85806eb90 con 0x7fe85815a740 2023-10-06T05:20:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:46.996+0000 7fe83ffff700 1 -- 172.21.15.33:0/2742797514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe8500557d0 con 0x7fe85815a740 2023-10-06T05:20:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.002+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe844066f30 msgr2=0x7fe8440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.002+0000 7fe85d39d700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe844066f30 0x7fe8440693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8580b29b0 tx=0x7fe848016000 comp rx=0 tx=0).stop 2023-10-06T05:20:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.002+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 msgr2=0x7fe85815b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.002+0000 7fe85d39d700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 0x7fe85815b000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe85000a380 tx=0x7fe850007900 comp rx=0 tx=0).stop 2023-10-06T05:20:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.003+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 shutdown_connections 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.003+0000 7fe85d39d700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe844066f30 0x7fe8440693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.003+0000 7fe85d39d700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe85815a740 0x7fe85815b000 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.003+0000 7fe85d39d700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe85806dc00 0x7fe8580b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.003+0000 7fe85d39d700 1 --2- 172.21.15.33:0/2742797514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe85806dfd0 0x7fe8580c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.003+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 >> 172.21.15.33:0/2742797514 conn(0x7fe8580d0550 msgr2=0x7fe858158020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.004+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 shutdown_connections 2023-10-06T05:20:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.004+0000 7fe85d39d700 1 -- 172.21.15.33:0/2742797514 wait complete. 2023-10-06T05:20:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:20:47.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:20:47.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.221+0000 7fb1b159d700 1 Processor -- start 2023-10-06T05:20:47.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.221+0000 7fb1b159d700 1 -- start start 2023-10-06T05:20:47.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.222+0000 7fb1b159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.222+0000 7fb1b159d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1ac15acd0 0x7fb1ac158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.222+0000 7fb1b159d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac06b1f0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.222+0000 7fb1b159d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac06a3f0 con 0x7fb1ac15acd0 2023-10-06T05:20:47.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.222+0000 7fb1b159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac06a5e0 con 0x7fb1ac0ad010 2023-10-06T05:20:47.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1ab7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb1ac0ad3e0 0x7fb1ac15a5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43350/0 (socket says 172.21.15.33:43350) 2023-10-06T05:20:47.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/3556938395 learned_addr learned my addr 172.21.15.33:0/3556938395 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:47.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1abfff700 1 --2- 172.21.15.33:0/3556938395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1b0d9c700 1 --2- 172.21.15.33:0/3556938395 >> v2:172.21.15.167:3300/0 conn(0x7fb1ac15acd0 0x7fb1ac158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1aaffd700 1 -- 172.21.15.33:0/3556938395 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807197968 0 0) 0x7fb1ac06b1f0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1aaffd700 1 -- 172.21.15.33:0/3556938395 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb194003570 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1abfff700 1 -- 172.21.15.33:0/3556938395 >> v2:172.21.15.167:3300/0 conn(0x7fb1ac15acd0 msgr2=0x7fb1ac158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1abfff700 1 --2- 172.21.15.33:0/3556938395 >> v2:172.21.15.167:3300/0 conn(0x7fb1ac15acd0 0x7fb1ac158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1abfff700 1 -- 172.21.15.33:0/3556938395 >> v1:172.21.15.33:6789/0 conn(0x7fb1ac0ad3e0 legacy=0x7fb1ac15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.223+0000 7fb1abfff700 1 -- 172.21.15.33:0/3556938395 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac06a8c0 con 0x7fb1ac0ad010 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.224+0000 7fb1abfff700 1 --2- 172.21.15.33:0/3556938395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac06ea00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0098e0 tx=0x7fb19c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa91a8f2f7c91e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.224+0000 7fb1aaffd700 1 -- 172.21.15.33:0/3556938395 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011630 con 0x7fb1ac0ad010 2023-10-06T05:20:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.224+0000 7fb1aaffd700 1 -- 172.21.15.33:0/3556938395 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c011790 con 0x7fb1ac0ad010 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.224+0000 7fb1aaffd700 1 -- 172.21.15.33:0/3556938395 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011b10 con 0x7fb1ac0ad010 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.224+0000 7fb1b159d700 1 -- 172.21.15.33:0/3556938395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac0ad010 msgr2=0x7fb1ac06ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.224+0000 7fb1b159d700 1 --2- 172.21.15.33:0/3556938395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac06ea00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0098e0 tx=0x7fb19c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.225+0000 7fb1b159d700 1 -- 172.21.15.33:0/3556938395 shutdown_connections 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.225+0000 7fb1b159d700 1 --2- 172.21.15.33:0/3556938395 >> v2:172.21.15.167:3300/0 conn(0x7fb1ac15acd0 0x7fb1ac158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.225+0000 7fb1b159d700 1 --2- 172.21.15.33:0/3556938395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac06ea00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.225+0000 7fb1b159d700 1 -- 172.21.15.33:0/3556938395 >> 172.21.15.33:0/3556938395 conn(0x7fb1ac068090 msgr2=0x7fb1ac06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.225+0000 7fb1b159d700 1 -- 172.21.15.33:0/3556938395 shutdown_connections 2023-10-06T05:20:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.225+0000 7fb1b159d700 1 -- 172.21.15.33:0/3556938395 wait complete. 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.226+0000 7fb1b159d700 1 Processor -- start 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.226+0000 7fb1b159d700 1 -- start start 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1b159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1b159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 0x7fb1ac0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1b159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac15acd0 0x7fb1ac0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1b159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1690f0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1b159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0b12d0 con 0x7fb1ac0ad010 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1b159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac141ce0 con 0x7fb1ac15acd0 2023-10-06T05:20:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 0x7fb1ac0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 0x7fb1ac0c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57550/0 (socket says 172.21.15.33:57550) 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.227+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/1964078890 learned_addr learned my addr 172.21.15.33:0/1964078890 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1abfff700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1b0d9c700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac15acd0 0x7fb1ac0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac15acd0 msgr2=0x7fb1ac0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1ab7fe700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac15acd0 0x7fb1ac0c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1ac0ad010 msgr2=0x7fb1ac1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1ab7fe700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac1417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/1964078890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009420 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1abfff700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1ab7fe700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 0x7fb1ac0c8bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00011e0 tx=0x7fb1a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0005b30 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a0005c90 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00072e0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.228+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac0b09a0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.229+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac0b0f60 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.230+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac0b1480 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.231+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1a0007650 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.232+0000 7fb193fff700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb198066f30 0x7fb1980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.232+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fb1a0083a60 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.232+0000 7fb1abfff700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb198066f30 0x7fb1980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.233+0000 7fb1abfff700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb198066f30 0x7fb1980693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0091e0 tx=0x7fb19c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.236+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1a004b5a0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:47.440+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1ac063ff0 con 0x7fb1ac0ad3e0 2023-10-06T05:20:48.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.004+0000 7fb193fff700 1 -- 172.21.15.33:0/1964078890 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb1a0017020 con 0x7fb1ac0ad3e0 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb198066f30 msgr2=0x7fb1980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb198066f30 0x7fb1980693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0091e0 tx=0x7fb19c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 msgr2=0x7fb1ac0c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 0x7fb1ac0c8bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00011e0 tx=0x7fb1a0007df0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 shutdown_connections 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb198066f30 0x7fb1980693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1ac15acd0 0x7fb1ac0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 --2- 172.21.15.33:0/1964078890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1ac0ad3e0 0x7fb1ac0c8bf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.007+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 >> 172.21.15.33:0/1964078890 conn(0x7fb1ac068090 msgr2=0x7fb1ac159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.008+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 shutdown_connections 2023-10-06T05:20:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.008+0000 7fb1b159d700 1 -- 172.21.15.33:0/1964078890 wait complete. 2023-10-06T05:20:48.011 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:20:48.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:20:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.218+0000 7fd2e4d9d700 1 Processor -- start 2023-10-06T05:20:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.218+0000 7fd2e4d9d700 1 -- start start 2023-10-06T05:20:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.219+0000 7fd2e4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.219+0000 7fd2e4d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2e00aff90 0x7fd2e00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.219+0000 7fd2e4d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2e006cb80 0x7fd2e015d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.219+0000 7fd2e4d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e015d3a0 con 0x7fd2e006cb80 2023-10-06T05:20:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.219+0000 7fd2e4d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e00b0470 con 0x7fd2e00aff90 2023-10-06T05:20:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.219+0000 7fd2e4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00b0750 con 0x7fd2e006c7b0 2023-10-06T05:20:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2e006cb80 0x7fd2e015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2e006cb80 0x7fd2e015d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57558/0 (socket says 172.21.15.33:57558) 2023-10-06T05:20:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 -- 172.21.15.33:0/3429932206 learned_addr learned my addr 172.21.15.33:0/3429932206 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 -- 172.21.15.33:0/3429932206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 msgr2=0x7fd2e0067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 --2- 172.21.15.33:0/3429932206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 -- 172.21.15.33:0/3429932206 >> v2:172.21.15.167:3300/0 conn(0x7fd2e00aff90 msgr2=0x7fd2e00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 --2- 172.21.15.33:0/3429932206 >> v2:172.21.15.167:3300/0 conn(0x7fd2e00aff90 0x7fd2e00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 -- 172.21.15.33:0/3429932206 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00b0a30 con 0x7fd2e006cb80 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.220+0000 7fd2dffff700 1 --2- 172.21.15.33:0/3429932206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e006cb80 0x7fd2e015d830 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd2e006eb10 tx=0x7fd2d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c833d4e064edc83e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2de7fc700 1 -- 172.21.15.33:0/3429932206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800e070 con 0x7fd2e006cb80 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2de7fc700 1 -- 172.21.15.33:0/3429932206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d800ba50 con 0x7fd2e006cb80 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2de7fc700 1 -- 172.21.15.33:0/3429932206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80092e0 con 0x7fd2e006cb80 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/3429932206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e006cb80 msgr2=0x7fd2e015d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/3429932206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e006cb80 0x7fd2e015d830 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd2e006eb10 tx=0x7fd2d8009840 comp rx=0 tx=0).stop 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/3429932206 shutdown_connections 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/3429932206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e006cb80 0x7fd2e015d830 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/3429932206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e0067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/3429932206 >> v2:172.21.15.167:3300/0 conn(0x7fd2e00aff90 0x7fd2e00a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/3429932206 >> 172.21.15.33:0/3429932206 conn(0x7fd2e006d670 msgr2=0x7fd2e006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/3429932206 shutdown_connections 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.221+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/3429932206 wait complete. 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.223+0000 7fd2e4d9d700 1 Processor -- start 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.223+0000 7fd2e4d9d700 1 -- start start 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2e4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2e4d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2e006cb80 0x7fd2e00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2e4d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 0x7fd2e015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2e4d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e015ea80 con 0x7fd2e00aff90 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2e4d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e015f8a0 con 0x7fd2e006cb80 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2e4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e015cc20 con 0x7fd2e006c7b0 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 0x7fd2e015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 0x7fd2e015fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57564/0 (socket says 172.21.15.33:57564) 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2dffff700 1 -- 172.21.15.33:0/92935729 learned_addr learned my addr 172.21.15.33:0/92935729 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2df7fe700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.224+0000 7fd2deffd700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2e006cb80 0x7fd2e00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2dffff700 1 -- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 msgr2=0x7fd2e015c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2dffff700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e015c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2dffff700 1 -- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2e006cb80 msgr2=0x7fd2e00af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2dffff700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2e006cb80 0x7fd2e00af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2dffff700 1 -- 172.21.15.33:0/92935729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8009ae0 con 0x7fd2e00aff90 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2dffff700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 0x7fd2e015fd30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009fd0 tx=0x7fd2d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80092e0 con 0x7fd2e00aff90 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8008dc0 con 0x7fd2e00aff90 2023-10-06T05:20:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80266d0 con 0x7fd2e00aff90 2023-10-06T05:20:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e015ec90 con 0x7fd2e00aff90 2023-10-06T05:20:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.225+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e015f250 con 0x7fd2e00aff90 2023-10-06T05:20:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.226+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e015fa50 con 0x7fd2e00aff90 2023-10-06T05:20:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.227+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2d8003040 con 0x7fd2e00aff90 2023-10-06T05:20:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.228+0000 7fd2c77fe700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd2cc066f30 0x7fd2cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.228+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd2d8085f00 con 0x7fd2e00aff90 2023-10-06T05:20:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.229+0000 7fd2df7fe700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd2cc066f30 0x7fd2cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.229+0000 7fd2df7fe700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd2cc066f30 0x7fd2cc0693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009bd0 tx=0x7fd2d00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.232+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2e015fa50 con 0x7fd2e00aff90 2023-10-06T05:20:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.448+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2e015fa50 con 0x7fd2e00aff90 2023-10-06T05:20:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.452+0000 7fd2c77fe700 1 -- 172.21.15.33:0/92935729 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2e015fa50 con 0x7fd2e00aff90 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd2cc066f30 msgr2=0x7fd2cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd2cc066f30 0x7fd2cc0693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009bd0 tx=0x7fd2d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 msgr2=0x7fd2e015fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 0x7fd2e015fd30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009fd0 tx=0x7fd2d8018000 comp rx=0 tx=0).stop 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 shutdown_connections 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd2cc066f30 0x7fd2cc0693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e00aff90 0x7fd2e015fd30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2e006c7b0 0x7fd2e015c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 --2- 172.21.15.33:0/92935729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2e006cb80 0x7fd2e00af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 >> 172.21.15.33:0/92935729 conn(0x7fd2e006d670 msgr2=0x7fd2e01573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 shutdown_connections 2023-10-06T05:20:48.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.454+0000 7fd2e4d9d700 1 -- 172.21.15.33:0/92935729 wait complete. 2023-10-06T05:20:48.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync' 2023-10-06T05:20:48.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:48.523 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.662+0000 7f333259d700 1 Processor -- start 2023-10-06T05:20:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.662+0000 7f333259d700 1 -- start start 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f333259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c155af0 0x7f332c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f333259d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f332c0c7920 0x7f332c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f333259d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c06ab60 con 0x7f332c06a6a0 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f333259d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c06b7e0 con 0x7f332c0c7920 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f333259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c06ae40 con 0x7f332c155af0 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f3331d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f332c0c7920 0x7f332c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f3331d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f332c0c7920 0x7f332c0c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57568/0 (socket says 172.21.15.33:57568) 2023-10-06T05:20:48.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f3331d9c700 1 -- 172.21.15.33:0/1279479973 learned_addr learned my addr 172.21.15.33:0/1279479973 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:48.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.664+0000 7f333159b700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c155af0 0x7f332c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3331d9c700 1 -- 172.21.15.33:0/1279479973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c155af0 msgr2=0x7f332c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:48.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3331d9c700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c155af0 0x7f332c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3331d9c700 1 -- 172.21.15.33:0/1279479973 >> v1:172.21.15.167:6789/0 conn(0x7f332c06a6a0 legacy=0x7f332c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3331d9c700 1 -- 172.21.15.33:0/1279479973 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c06b120 con 0x7f332c0c7920 2023-10-06T05:20:48.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f333159b700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c155af0 0x7f332c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:48.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3331d9c700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c9e20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f332400afb0 tx=0x7f332400a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ddb8625d72bf90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3323fff700 1 -- 172.21.15.33:0/1279479973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332400a2e0 con 0x7f332c0c7920 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3323fff700 1 -- 172.21.15.33:0/1279479973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332400cb50 con 0x7f332c0c7920 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.665+0000 7f3323fff700 1 -- 172.21.15.33:0/1279479973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324008510 con 0x7f332c0c7920 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 -- 172.21.15.33:0/1279479973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 msgr2=0x7f332c0c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c9e20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f332400afb0 tx=0x7f332400a950 comp rx=0 tx=0).stop 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 -- 172.21.15.33:0/1279479973 shutdown_connections 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c9e20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 --2- 172.21.15.33:0/1279479973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c155af0 0x7f332c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 -- 172.21.15.33:0/1279479973 >> 172.21.15.33:0/1279479973 conn(0x7f332c06d030 msgr2=0x7f332c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 -- 172.21.15.33:0/1279479973 shutdown_connections 2023-10-06T05:20:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.666+0000 7f333259d700 1 -- 172.21.15.33:0/1279479973 wait complete. 2023-10-06T05:20:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.667+0000 7f333259d700 1 Processor -- start 2023-10-06T05:20:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.667+0000 7f333259d700 1 -- start start 2023-10-06T05:20:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.668+0000 7f333259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06a6a0 0x7f332c13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.668+0000 7f333259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.668+0000 7f333259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c155af0 0x7f332c0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.668+0000 7f333259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0c1790 con 0x7f332c0c7920 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.668+0000 7f333259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c151a60 con 0x7f332c155af0 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.668+0000 7f333259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0c19e0 con 0x7f332c06a6a0 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57584/0 (socket says 172.21.15.33:57584) 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 -- 172.21.15.33:0/3995403057 learned_addr learned my addr 172.21.15.33:0/3995403057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f333159b700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06a6a0 0x7f332c13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3331d9c700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c155af0 0x7f332c0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 -- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06a6a0 msgr2=0x7f332c13dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06a6a0 0x7f332c13dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 -- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c155af0 msgr2=0x7f332c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c155af0 0x7f332c0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 -- 172.21.15.33:0/3995403057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332400abf0 con 0x7f332c0c7920 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3331d9c700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c155af0 0x7f332c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f3330d9a700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c24c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f331c00c7c0 tx=0x7f331c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.669+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c011590 con 0x7f332c0c7920 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.670+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c011710 con 0x7f332c0c7920 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.670+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c011ad0 con 0x7f332c0c7920 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.670+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c0c1cc0 con 0x7f332c0c7920 2023-10-06T05:20:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.670+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c13e200 con 0x7f332c0c7920 2023-10-06T05:20:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.671+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c04c380 con 0x7f332c0c7920 2023-10-06T05:20:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.672+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f331c01a360 con 0x7f332c0c7920 2023-10-06T05:20:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.672+0000 7f33217fa700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3310066f30 0x7f33100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.673+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f331c083430 con 0x7f332c0c7920 2023-10-06T05:20:48.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.673+0000 7f333159b700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3310066f30 0x7f33100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.674+0000 7f333159b700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3310066f30 0x7f33100693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3328006f00 tx=0x7f3328008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.676+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f331c050720 con 0x7f332c0c7920 2023-10-06T05:20:48.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:48.965+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]} v 0) v1 -- 0x7f332c06b400 con 0x7f332c0c7920 2023-10-06T05:20:49.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.004+0000 7f33217fa700 1 -- 172.21.15.33:0/3995403057 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_create_fsync"]}]=0 v322) v1 ==== 149+0+0 (secure 0 0 0) 0x7f331c053d40 con 0x7f332c0c7920 2023-10-06T05:20:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3310066f30 msgr2=0x7f33100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3310066f30 0x7f33100693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3328006f00 tx=0x7f3328008040 comp rx=0 tx=0).stop 2023-10-06T05:20:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 msgr2=0x7f332c0c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c24c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f331c00c7c0 tx=0x7f331c007f50 comp rx=0 tx=0).stop 2023-10-06T05:20:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 shutdown_connections 2023-10-06T05:20:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3310066f30 0x7f33100693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f332c155af0 0x7f332c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f332c06a6a0 0x7f332c13dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 --2- 172.21.15.33:0/3995403057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c0c7920 0x7f332c0c24c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 >> 172.21.15.33:0/3995403057 conn(0x7f332c06d030 msgr2=0x7f332c06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 shutdown_connections 2023-10-06T05:20:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.007+0000 7f333259d700 1 -- 172.21.15.33:0/3995403057 wait complete. 2023-10-06T05:20:49.026 INFO:tasks.cephfs_test_runner:test_create_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:20:49.026 INFO:tasks.cephfs_test_runner:Starting test: test_dir_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:20:49.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync' 2023-10-06T05:20:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.252+0000 7f2e6f59e700 1 Processor -- start 2023-10-06T05:20:49.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.252+0000 7f2e6f59e700 1 -- start start 2023-10-06T05:20:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.253+0000 7f2e6f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e7013faa0 0x7f2e7013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.253+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e700c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.253+0000 7f2e6f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e700c9cb0 con 0x7f2e7006aad0 2023-10-06T05:20:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e7006b660 con 0x7f2e7013faa0 2023-10-06T05:20:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700ca270 con 0x7f2e700c7940 2023-10-06T05:20:49.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e7013faa0 0x7f2e7013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e7013faa0 0x7f2e7013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57588/0 (socket says 172.21.15.33:57588) 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 -- 172.21.15.33:0/3776394300 learned_addr learned my addr 172.21.15.33:0/3776394300 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 -- 172.21.15.33:0/3776394300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 msgr2=0x7f2e700c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 --2- 172.21.15.33:0/3776394300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e700c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 -- 172.21.15.33:0/3776394300 >> v1:172.21.15.167:6789/0 conn(0x7f2e7006aad0 legacy=0x7f2e7006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 -- 172.21.15.33:0/3776394300 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7006b840 con 0x7f2e7013faa0 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.254+0000 7f2e6e59c700 1 --2- 172.21.15.33:0/3776394300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e7013fe70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2e600099b0 tx=0x7f2e60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fd455703b3f06a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6d59a700 1 -- 172.21.15.33:0/3776394300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000b820 con 0x7f2e7013faa0 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6d59a700 1 -- 172.21.15.33:0/3776394300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6000b980 con 0x7f2e7013faa0 2023-10-06T05:20:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6d59a700 1 -- 172.21.15.33:0/3776394300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000bd20 con 0x7f2e7013faa0 2023-10-06T05:20:49.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 -- 172.21.15.33:0/3776394300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 msgr2=0x7f2e7013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/3776394300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e7013fe70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2e600099b0 tx=0x7f2e60009250 comp rx=0 tx=0).stop 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 -- 172.21.15.33:0/3776394300 shutdown_connections 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/3776394300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e7013fe70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/3776394300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e700c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 -- 172.21.15.33:0/3776394300 >> 172.21.15.33:0/3776394300 conn(0x7f2e7006d030 msgr2=0x7f2e7006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 -- 172.21.15.33:0/3776394300 shutdown_connections 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.255+0000 7f2e6f59e700 1 -- 172.21.15.33:0/3776394300 wait complete. 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.257+0000 7f2e6f59e700 1 Processor -- start 2023-10-06T05:20:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.257+0000 7f2e6f59e700 1 -- start start 2023-10-06T05:20:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7006aad0 0x7f2e70155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e7013a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e700c1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701370c0 con 0x7f2e7013faa0 2023-10-06T05:20:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70136b00 con 0x7f2e7006aad0 2023-10-06T05:20:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70136de0 con 0x7f2e700c7940 2023-10-06T05:20:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e700c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e700c1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57596/0 (socket says 172.21.15.33:57596) 2023-10-06T05:20:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6ed9d700 1 -- 172.21.15.33:0/972610680 learned_addr learned my addr 172.21.15.33:0/972610680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6e59c700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7006aad0 0x7f2e70155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.258+0000 7f2e6dd9b700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e7013a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6ed9d700 1 -- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 msgr2=0x7f2e7013a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6ed9d700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e7013a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6ed9d700 1 -- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7006aad0 msgr2=0x7f2e70155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6ed9d700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7006aad0 0x7f2e70155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6ed9d700 1 -- 172.21.15.33:0/972610680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e600094f0 con 0x7f2e7013faa0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6ed9d700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e700c1540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c930 tx=0x7f2e680026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68012630 con 0x7f2e7013faa0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680127d0 con 0x7f2e7013faa0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7013b110 con 0x7f2e7013faa0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.259+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68012b90 con 0x7f2e7013faa0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.260+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e701563b0 con 0x7f2e7013faa0 2023-10-06T05:20:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.260+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7004c380 con 0x7f2e7013faa0 2023-10-06T05:20:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.261+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e6801b400 con 0x7f2e7013faa0 2023-10-06T05:20:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.263+0000 7f2e5e7fc700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2e50066f30 0x7f2e500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.263+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2e68084720 con 0x7f2e7013faa0 2023-10-06T05:20:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.263+0000 7f2e6e59c700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2e50066f30 0x7f2e500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.264+0000 7f2e6e59c700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2e50066f30 0x7f2e500693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2e600099b0 tx=0x7f2e60016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.267+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e680554a0 con 0x7f2e7013faa0 2023-10-06T05:20:49.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:49.549+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync"]} v 0) v1 -- 0x7f2e7006bb20 con 0x7f2e7013faa0 2023-10-06T05:20:50.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.014+0000 7f2e5e7fc700 1 -- 172.21.15.33:0/972610680 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync"]}]=0 v323) v1 ==== 149+0+0 (secure 0 0 0) 0x7f2e68055030 con 0x7f2e7013faa0 2023-10-06T05:20:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.016+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2e50066f30 msgr2=0x7f2e500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.016+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2e50066f30 0x7f2e500693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2e600099b0 tx=0x7f2e60016000 comp rx=0 tx=0).stop 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.016+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 msgr2=0x7f2e700c1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.016+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e700c1540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c930 tx=0x7f2e680026e0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 shutdown_connections 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2e50066f30 0x7f2e500693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e7013faa0 0x7f2e700c1540 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7006aad0 0x7f2e70155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 --2- 172.21.15.33:0/972610680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e700c7940 0x7f2e7013a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 >> 172.21.15.33:0/972610680 conn(0x7f2e7006d030 msgr2=0x7f2e7013ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 shutdown_connections 2023-10-06T05:20:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.017+0000 7f2e6f59e700 1 -- 172.21.15.33:0/972610680 wait complete. 2023-10-06T05:20:50.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:20:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.228+0000 7f2a9b59e700 1 Processor -- start 2023-10-06T05:20:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.228+0000 7f2a9b59e700 1 -- start start 2023-10-06T05:20:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.229+0000 7f2a9b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.229+0000 7f2a9b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a9c070440 0x7f2a9c067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.229+0000 7f2a9b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c06ae10 con 0x7f2a9c068140 2023-10-06T05:20:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.229+0000 7f2a9b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c06a2d0 con 0x7f2a9c070440 2023-10-06T05:20:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.229+0000 7f2a9b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c154290 con 0x7f2a9c0b1f90 2023-10-06T05:20:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a9c070440 0x7f2a9c067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a9ad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2a9c068140 0x7f2a9c06c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54580/0 (socket says 172.21.15.33:54580) 2023-10-06T05:20:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a9c070440 0x7f2a9c067c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41550/0 (socket says 172.21.15.33:41550) 2023-10-06T05:20:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a9ad9d700 1 -- 172.21.15.33:0/3287776219 learned_addr learned my addr 172.21.15.33:0/3287776219 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a9a59c700 1 --2- 172.21.15.33:0/3287776219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a9959a700 1 -- 172.21.15.33:0/3287776219 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729297187 0 0) 0x7f2a9c06ae10 con 0x7f2a9c068140 2023-10-06T05:20:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a9959a700 1 -- 172.21.15.33:0/3287776219 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a84003570 con 0x7f2a9c068140 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 -- 172.21.15.33:0/3287776219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 msgr2=0x7f2a9c0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 --2- 172.21.15.33:0/3287776219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 -- 172.21.15.33:0/3287776219 >> v1:172.21.15.33:6789/0 conn(0x7f2a9c068140 legacy=0x7f2a9c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 -- 172.21.15.33:0/3287776219 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c068580 con 0x7f2a9c070440 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.230+0000 7f2a99d9b700 1 --2- 172.21.15.33:0/3287776219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c070440 0x7f2a9c067c00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2a900098e0 tx=0x7f2a900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa6d91187505deb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9959a700 1 -- 172.21.15.33:0/3287776219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90011630 con 0x7f2a9c070440 2023-10-06T05:20:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9959a700 1 -- 172.21.15.33:0/3287776219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a90011790 con 0x7f2a9c070440 2023-10-06T05:20:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9959a700 1 -- 172.21.15.33:0/3287776219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90011b30 con 0x7f2a9c070440 2023-10-06T05:20:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 -- 172.21.15.33:0/3287776219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c070440 msgr2=0x7f2a9c067c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/3287776219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c070440 0x7f2a9c067c00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2a900098e0 tx=0x7f2a900091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 -- 172.21.15.33:0/3287776219 shutdown_connections 2023-10-06T05:20:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/3287776219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/3287776219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c070440 0x7f2a9c067c00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 -- 172.21.15.33:0/3287776219 >> 172.21.15.33:0/3287776219 conn(0x7f2a9c06ea70 msgr2=0x7f2a9c06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 -- 172.21.15.33:0/3287776219 shutdown_connections 2023-10-06T05:20:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.231+0000 7f2a9b59e700 1 -- 172.21.15.33:0/3287776219 wait complete. 2023-10-06T05:20:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.233+0000 7f2a9b59e700 1 Processor -- start 2023-10-06T05:20:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.233+0000 7f2a9b59e700 1 -- start start 2023-10-06T05:20:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 0x7f2a9c15dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a9c070440 0x7f2a9c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c157a90 con 0x7f2a9c068140 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c159e20 con 0x7f2a9c070440 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c15e6d0 con 0x7f2a9c0b1f90 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 0x7f2a9c15dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 0x7f2a9c15dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41558/0 (socket says 172.21.15.33:41558) 2023-10-06T05:20:50.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 -- 172.21.15.33:0/515915422 learned_addr learned my addr 172.21.15.33:0/515915422 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 -- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 msgr2=0x7f2a9c169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 -- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a9c070440 msgr2=0x7f2a9c1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.234+0000 7f2a9a59c700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a9c070440 0x7f2a9c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a9a59c700 1 -- 172.21.15.33:0/515915422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a90009420 con 0x7f2a9c068140 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a9a59c700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 0x7f2a9c15dff0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00e930 tx=0x7f2a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c009510 con 0x7f2a9c068140 2023-10-06T05:20:50.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c00c850 con 0x7f2a9c068140 2023-10-06T05:20:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c15e9b0 con 0x7f2a9c068140 2023-10-06T05:20:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00cc10 con 0x7f2a9c068140 2023-10-06T05:20:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.235+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c15ef70 con 0x7f2a9c068140 2023-10-06T05:20:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.236+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c04c380 con 0x7f2a9c068140 2023-10-06T05:20:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.237+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a8c013470 con 0x7f2a9c068140 2023-10-06T05:20:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.239+0000 7f2a8a7fc700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.239+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2a8c084640 con 0x7f2a9c068140 2023-10-06T05:20:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.239+0000 7f2a99d9b700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.240+0000 7f2a99d9b700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a7c066f30 0x7f2a7c0693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c15ad70 tx=0x7f2a90017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.243+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a8c051930 con 0x7f2a9c068140 2023-10-06T05:20:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.446+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2a9c06a4e0 con 0x7f2a9c068140 2023-10-06T05:20:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.452+0000 7f2a8a7fc700 1 -- 172.21.15.33:0/515915422 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a8c054f50 con 0x7f2a9c068140 2023-10-06T05:20:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a7c066f30 msgr2=0x7f2a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a7c066f30 0x7f2a7c0693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c15ad70 tx=0x7f2a90017000 comp rx=0 tx=0).stop 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 msgr2=0x7f2a9c15dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 0x7f2a9c15dff0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00e930 tx=0x7f2a8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 shutdown_connections 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2a9c0b1f90 0x7f2a9c169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c068140 0x7f2a9c15dff0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 --2- 172.21.15.33:0/515915422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a9c070440 0x7f2a9c1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 >> 172.21.15.33:0/515915422 conn(0x7f2a9c06ea70 msgr2=0x7f2a9c0acf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 shutdown_connections 2023-10-06T05:20:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.455+0000 7f2a9b59e700 1 -- 172.21.15.33:0/515915422 wait complete. 2023-10-06T05:20:50.474 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:50.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:50.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:50.646 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:50.646 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:20:50.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:50.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.831+0000 7fcf50f9d700 1 Processor -- start 2023-10-06T05:20:50.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.831+0000 7fcf50f9d700 1 -- start start 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.832+0000 7fcf50f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf4c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.832+0000 7fcf50f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcf4c0c7920 0x7fcf4c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.832+0000 7fcf50f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c06ab60 con 0x7fcf4c06a6a0 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.832+0000 7fcf50f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c06b7e0 con 0x7fcf4c0c7920 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.832+0000 7fcf50f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c06ae40 con 0x7fcf4c155af0 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcf4c06a6a0 0x7fcf4c0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54582/0 (socket says 172.21.15.33:54582) 2023-10-06T05:20:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4affd700 1 -- 172.21.15.33:0/2488743824 learned_addr learned my addr 172.21.15.33:0/2488743824 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:50.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4bfff700 1 --2- 172.21.15.33:0/2488743824 >> v2:172.21.15.167:3300/0 conn(0x7fcf4c0c7920 0x7fcf4c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf4c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925332463 0 0) 0x7fcf4c06ab60 con 0x7fcf4c06a6a0 2023-10-06T05:20:50.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf34003570 con 0x7fcf4c06a6a0 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3949759555 0 0) 0x7fcf34003570 con 0x7fcf4c06a6a0 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 msgr2=0x7fcf4c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf4c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 >> v2:172.21.15.167:3300/0 conn(0x7fcf4c0c7920 msgr2=0x7fcf4c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 --2- 172.21.15.33:0/2488743824 >> v2:172.21.15.167:3300/0 conn(0x7fcf4c0c7920 0x7fcf4c0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c06b120 con 0x7fcf4c06a6a0 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fcf40001470 con 0x7fcf4c06a6a0 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 >> v1:172.21.15.33:6789/0 conn(0x7fcf4c06a6a0 legacy=0x7fcf4c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.833+0000 7fcf4a7fc700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf34003a60 0x7fcf34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4a7fc700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4a7fc700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf34004d50 0x7fcf340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c06b7e0 con 0x7fcf34003a60 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c06ae40 con 0x7fcf34004d50 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3400dc70 con 0x7fcf4c155af0 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf34003a60 0x7fcf34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4affd700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4bfff700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf34004d50 0x7fcf340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 -- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 msgr2=0x7fcf34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 -- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf34004d50 msgr2=0x7fcf340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf34004d50 0x7fcf340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 -- 172.21.15.33:0/2488743824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34003570 con 0x7fcf34003a60 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4bfff700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf34004d50 0x7fcf340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4affd700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf4c155af0 0x7fcf34004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.834+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf34003a60 0x7fcf34003e70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00acc0 tx=0x7fcf3c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff8168d7142b8e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00e720 con 0x7fcf34003a60 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00e880 con 0x7fcf34003a60 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf4a7fc700 1 -- 172.21.15.33:0/2488743824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00ea10 con 0x7fcf34003a60 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf50f9d700 1 -- 172.21.15.33:0/2488743824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf34003a60 msgr2=0x7fcf34003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/2488743824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf34003a60 0x7fcf34003e70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00acc0 tx=0x7fcf3c00a330 comp rx=0 tx=0).stop 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf4bfff700 1 -- 172.21.15.33:0/2488743824 reap_dead start 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf50f9d700 1 -- 172.21.15.33:0/2488743824 shutdown_connections 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf50f9d700 1 -- 172.21.15.33:0/2488743824 >> 172.21.15.33:0/2488743824 conn(0x7fcf4c06d030 msgr2=0x7fcf4c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf50f9d700 1 -- 172.21.15.33:0/2488743824 shutdown_connections 2023-10-06T05:20:50.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.835+0000 7fcf50f9d700 1 -- 172.21.15.33:0/2488743824 wait complete. 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.837+0000 7fcf50f9d700 1 Processor -- start 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.837+0000 7fcf50f9d700 1 -- start start 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf50f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf34003a60 0x7fcf4c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf50f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 0x7fcf4c0c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf50f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf4c0cac80 0x7fcf4c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf50f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0c42d0 con 0x7fcf4c0c4880 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf50f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c152100 con 0x7fcf4c0cac80 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf50f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0c5230 con 0x7fcf34003a60 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 0x7fcf4c0c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 0x7fcf4c0c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41576/0 (socket says 172.21.15.33:41576) 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf4affd700 1 -- 172.21.15.33:0/1514366419 learned_addr learned my addr 172.21.15.33:0/1514366419 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf34003a60 0x7fcf4c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.838+0000 7fcf4bfff700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf4c0cac80 0x7fcf4c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4affd700 1 -- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf34003a60 msgr2=0x7fcf4c151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4affd700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf34003a60 0x7fcf4c151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4affd700 1 -- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf4c0cac80 msgr2=0x7fcf4c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4affd700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf4c0cac80 0x7fcf4c0cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4affd700 1 -- 172.21.15.33:0/1514366419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0c3e40 con 0x7fcf4c0c4880 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4bfff700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf4c0cac80 0x7fcf4c0cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf4affd700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 0x7fcf4c0c4cf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000ebf0 tx=0x7fcf400014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000c7e0 con 0x7fcf4c0c4880 2023-10-06T05:20:50.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000c940 con 0x7fcf4c0c4880 2023-10-06T05:20:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000ccc0 con 0x7fcf4c0c4880 2023-10-06T05:20:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c0c3230 con 0x7fcf4c0c4880 2023-10-06T05:20:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.839+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c0c3870 con 0x7fcf4c0c4880 2023-10-06T05:20:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.840+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c0c3fd0 con 0x7fcf4c0c4880 2023-10-06T05:20:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.841+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf4001d460 con 0x7fcf4c0c4880 2023-10-06T05:20:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.842+0000 7fcf337fe700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf38066f30 0x7fcf380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.842+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fcf40084450 con 0x7fcf4c0c4880 2023-10-06T05:20:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.843+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf38066f30 0x7fcf380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.843+0000 7fcf4b7fe700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf38066f30 0x7fcf380693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00a370 tx=0x7fcf3c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:50.846+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf40051740 con 0x7fcf4c0c4880 2023-10-06T05:20:51.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.090+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf4c04c380 con 0x7fcf4c0c4880 2023-10-06T05:20:51.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.090+0000 7fcf337fe700 1 -- 172.21.15.33:0/1514366419 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fcf40054d60 con 0x7fcf4c0c4880 2023-10-06T05:20:51.092 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:51.092 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2119942467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2119942467},{"type":"v1","addr":"172.21.15.167:6837","nonce":2119942467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":5821,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3808325629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3808325629},{"type":"v1","addr":"172.21.15.33:6833","nonce":3808325629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5824,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1675916382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1675916382},{"type":"v1","addr":"172.21.15.167:6836","nonce":1675916382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/4221050867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":4221050867},{"type":"v1","addr":"172.21.15.33:6835","nonce":4221050867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf38066f30 msgr2=0x7fcf380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf38066f30 0x7fcf380693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00a370 tx=0x7fcf3c019000 comp rx=0 tx=0).stop 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 msgr2=0x7fcf4c0c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 0x7fcf4c0c4cf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000ebf0 tx=0x7fcf400014c0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 shutdown_connections 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf38066f30 0x7fcf380693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf4c0cac80 0x7fcf4c0cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf34003a60 0x7fcf4c151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 --2- 172.21.15.33:0/1514366419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf4c0c4880 0x7fcf4c0c4cf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 >> 172.21.15.33:0/1514366419 conn(0x7fcf4c06d030 msgr2=0x7fcf4c0c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:51.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 shutdown_connections 2023-10-06T05:20:51.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.093+0000 7fcf50f9d700 1 -- 172.21.15.33:0/1514366419 wait complete. 2023-10-06T05:20:51.096 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 51 2023-10-06T05:20:51.112 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:20:51.112 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:20:51.113 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:20:51.113 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:51.113 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:20:51.113 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:20:51.113 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:20:51.113 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:51.114 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:20:51.114 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:20:51.114 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:20:51.114 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:51.114 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:20:51.114 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:20:51.115 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:20:51.115 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:51.128 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:20:51.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:20:51.129 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:20:51.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:20:51.130 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:20:51.130 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:20:51.131 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:20:51.131 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:20:51.173 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:20:51.173 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:51.175 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:20:51.175 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:51.176 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:20:51.176 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:51.180 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:20:51.181 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:51.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:20:51.310 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:20:51.311 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:20:51.315 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:20:51.316 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:20:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.378+0000 7f5a20d9d700 1 Processor -- start 2023-10-06T05:20:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.379+0000 7f5a20d9d700 1 -- start start 2023-10-06T05:20:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a20d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c0af600 0x7f5a1c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a20d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c06ec40 con 0x7f5a1c1596c0 2023-10-06T05:20:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a20d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c0aff90 con 0x7f5a1c159a90 2023-10-06T05:20:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a20d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c06dd80 con 0x7f5a1c0af600 2023-10-06T05:20:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a1b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5a1c1596c0 0x7f5a1c06f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54608/0 (socket says 172.21.15.33:54608) 2023-10-06T05:20:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a1b7fe700 1 -- 172.21.15.33:0/1320219824 learned_addr learned my addr 172.21.15.33:0/1320219824 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a1bfff700 1 --2- 172.21.15.33:0/1320219824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c0af600 0x7f5a1c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.380+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948765509 0 0) 0x7f5a1c06ec40 con 0x7f5a1c1596c0 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a04003570 con 0x7f5a1c1596c0 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211729985 0 0) 0x7f5a1c0aff90 con 0x7f5a1c159a90 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1c06ec40 con 0x7f5a1c159a90 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1bfff700 1 -- 172.21.15.33:0/1320219824 >> v1:172.21.15.167:6789/0 conn(0x7f5a1c159a90 legacy=0x7f5a1c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1bfff700 1 -- 172.21.15.33:0/1320219824 >> v1:172.21.15.33:6789/0 conn(0x7f5a1c1596c0 legacy=0x7f5a1c06f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1bfff700 1 -- 172.21.15.33:0/1320219824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c06e030 con 0x7f5a1c0af600 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1bfff700 1 --2- 172.21.15.33:0/1320219824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c0af600 0x7f5a1c0afa50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009e90 tx=0x7f5a14009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9702812c6ed582ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400e070 con 0x7f5a1c0af600 2023-10-06T05:20:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1400bb80 con 0x7f5a1c0af600 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.381+0000 7f5a1a7fc700 1 -- 172.21.15.33:0/1320219824 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14012430 con 0x7f5a1c0af600 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 -- 172.21.15.33:0/1320219824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c0af600 msgr2=0x7f5a1c0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/1320219824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c0af600 0x7f5a1c0afa50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009e90 tx=0x7f5a14009780 comp rx=0 tx=0).stop 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 -- 172.21.15.33:0/1320219824 shutdown_connections 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/1320219824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c0af600 0x7f5a1c0afa50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 -- 172.21.15.33:0/1320219824 >> 172.21.15.33:0/1320219824 conn(0x7f5a1c087fb0 msgr2=0x7f5a1c0883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 -- 172.21.15.33:0/1320219824 shutdown_connections 2023-10-06T05:20:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.382+0000 7f5a20d9d700 1 -- 172.21.15.33:0/1320219824 wait complete. 2023-10-06T05:20:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.383+0000 7f5a20d9d700 1 Processor -- start 2023-10-06T05:20:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.383+0000 7f5a20d9d700 1 -- start start 2023-10-06T05:20:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.384+0000 7f5a20d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 0x7f5a1c15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.384+0000 7f5a20d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1c1596c0 0x7f5a1c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.384+0000 7f5a20d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c159a90 0x7f5a1c15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.384+0000 7f5a20d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0ca580 con 0x7f5a1c0af600 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.384+0000 7f5a20d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0cd890 con 0x7f5a1c1596c0 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.384+0000 7f5a20d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0ccc80 con 0x7f5a1c159a90 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 0x7f5a1c15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 0x7f5a1c15ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41620/0 (socket says 172.21.15.33:41620) 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 -- 172.21.15.33:0/413504004 learned_addr learned my addr 172.21.15.33:0/413504004 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1bfff700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c159a90 0x7f5a1c15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1affd700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1c1596c0 0x7f5a1c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 -- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c159a90 msgr2=0x7f5a1c15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c159a90 0x7f5a1c15a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 -- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1c1596c0 msgr2=0x7f5a1c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1c1596c0 0x7f5a1c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 -- 172.21.15.33:0/413504004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14009a20 con 0x7f5a1c0af600 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a1b7fe700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 0x7f5a1c15ebd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c7c0 tx=0x7f5a0c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c005b30 con 0x7f5a1c0af600 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c005c90 con 0x7f5a1c0af600 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.385+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007300 con 0x7f5a1c0af600 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.386+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c0ccf60 con 0x7f5a1c0af600 2023-10-06T05:20:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.386+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c0cd520 con 0x7f5a1c0af600 2023-10-06T05:20:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.387+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c0cd6e0 con 0x7f5a1c0af600 2023-10-06T05:20:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.388+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a0c007670 con 0x7f5a1c0af600 2023-10-06T05:20:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.389+0000 7f5a037fe700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a08066f30 0x7f5a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.389+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5a0c083970 con 0x7f5a1c0af600 2023-10-06T05:20:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.389+0000 7f5a1affd700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a08066f30 0x7f5a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.390+0000 7f5a1affd700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a08066f30 0x7f5a080693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5a10003ae0 tx=0x7f5a10007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.393+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a0c050c60 con 0x7f5a1c0af600 2023-10-06T05:20:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:51.614+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5a1c04c380 con 0x7f5a1c0af600 2023-10-06T05:20:52.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.024+0000 7f5a037fe700 1 -- 172.21.15.33:0/413504004 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5a0c017020 con 0x7f5a1c0af600 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a08066f30 msgr2=0x7f5a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a08066f30 0x7f5a080693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5a10003ae0 tx=0x7f5a10007d40 comp rx=0 tx=0).stop 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 msgr2=0x7f5a1c15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 0x7f5a1c15ebd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c7c0 tx=0x7f5a0c007ea0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 shutdown_connections 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a08066f30 0x7f5a080693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a1c159a90 0x7f5a1c15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a1c0af600 0x7f5a1c15ebd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 --2- 172.21.15.33:0/413504004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1c1596c0 0x7f5a1c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 >> 172.21.15.33:0/413504004 conn(0x7f5a1c087fb0 msgr2=0x7f5a1c1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 shutdown_connections 2023-10-06T05:20:52.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.028+0000 7f5a20d9d700 1 -- 172.21.15.33:0/413504004 wait complete. 2023-10-06T05:20:52.031 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:20:52.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:20:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.238+0000 7ff3ea19d700 1 Processor -- start 2023-10-06T05:20:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.238+0000 7ff3ea19d700 1 -- start start 2023-10-06T05:20:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3ea19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e4153b10 0x7ff3e40c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3ea19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3e40c6b30 0x7ff3e40c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3ea19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3e40675f0 0x7ff3e4067a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3ea19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e40c7510 con 0x7ff3e40c6b30 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3ea19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e415abd0 con 0x7ff3e40675f0 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3ea19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40c9b00 con 0x7ff3e4153b10 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3e899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3e40c6b30 0x7ff3e40c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3e899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3e40c6b30 0x7ff3e40c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41632/0 (socket says 172.21.15.33:41632) 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.239+0000 7ff3e899a700 1 -- 172.21.15.33:0/2172790227 learned_addr learned my addr 172.21.15.33:0/2172790227 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e919b700 1 --2- 172.21.15.33:0/2172790227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e4153b10 0x7ff3e40c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e999c700 1 --2- 172.21.15.33:0/2172790227 >> v2:172.21.15.167:3300/0 conn(0x7ff3e40675f0 0x7ff3e4067a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e899a700 1 -- 172.21.15.33:0/2172790227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e4153b10 msgr2=0x7ff3e40c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e899a700 1 --2- 172.21.15.33:0/2172790227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e4153b10 0x7ff3e40c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e899a700 1 -- 172.21.15.33:0/2172790227 >> v2:172.21.15.167:3300/0 conn(0x7ff3e40675f0 msgr2=0x7ff3e4067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e899a700 1 --2- 172.21.15.33:0/2172790227 >> v2:172.21.15.167:3300/0 conn(0x7ff3e40675f0 0x7ff3e4067a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e899a700 1 -- 172.21.15.33:0/2172790227 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e415a050 con 0x7ff3e40c6b30 2023-10-06T05:20:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3e899a700 1 --2- 172.21.15.33:0/2172790227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e40c6b30 0x7ff3e40c6fd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4065a70 tx=0x7ff3d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34273f8828dbae1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.240+0000 7ff3dbfff700 1 -- 172.21.15.33:0/2172790227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400b640 con 0x7ff3e40c6b30 2023-10-06T05:20:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3dbfff700 1 -- 172.21.15.33:0/2172790227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d400b7a0 con 0x7ff3e40c6b30 2023-10-06T05:20:52.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3dbfff700 1 -- 172.21.15.33:0/2172790227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400bb60 con 0x7ff3e40c6b30 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 -- 172.21.15.33:0/2172790227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e40c6b30 msgr2=0x7ff3e40c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/2172790227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e40c6b30 0x7ff3e40c6fd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4065a70 tx=0x7ff3d4009250 comp rx=0 tx=0).stop 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 -- 172.21.15.33:0/2172790227 shutdown_connections 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/2172790227 >> v2:172.21.15.167:3300/0 conn(0x7ff3e40675f0 0x7ff3e4067a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/2172790227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e4153b10 0x7ff3e40c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/2172790227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e40c6b30 0x7ff3e40c6fd0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 -- 172.21.15.33:0/2172790227 >> 172.21.15.33:0/2172790227 conn(0x7ff3e406c560 msgr2=0x7ff3e406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 -- 172.21.15.33:0/2172790227 shutdown_connections 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.241+0000 7ff3ea19d700 1 -- 172.21.15.33:0/2172790227 wait complete. 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.243+0000 7ff3ea19d700 1 Processor -- start 2023-10-06T05:20:52.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.243+0000 7ff3ea19d700 1 -- start start 2023-10-06T05:20:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3ea19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3e40675f0 0x7ff3e41591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3ea19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e40c6b30 0x7ff3e4157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3ea19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 0x7ff3e4158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3ea19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40c42c0 con 0x7ff3e4153b10 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3ea19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40c3640 con 0x7ff3e40675f0 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3ea19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40c3920 con 0x7ff3e40c6b30 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 0x7ff3e4158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 0x7ff3e4158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41644/0 (socket says 172.21.15.33:41644) 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 -- 172.21.15.33:0/430515835 learned_addr learned my addr 172.21.15.33:0/430515835 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e919b700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3e40675f0 0x7ff3e41591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e899a700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e40c6b30 0x7ff3e4157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 -- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e40c6b30 msgr2=0x7ff3e4157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e40c6b30 0x7ff3e4157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 -- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3e40675f0 msgr2=0x7ff3e41591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3e40675f0 0x7ff3e41591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e999c700 1 -- 172.21.15.33:0/430515835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40094f0 con 0x7ff3e4153b10 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e919b700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3e40675f0 0x7ff3e41591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.244+0000 7ff3e899a700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e40c6b30 0x7ff3e4157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.245+0000 7ff3e999c700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 0x7ff3e4158270 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009df0 tx=0x7ff3dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.245+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00cb50 con 0x7ff3e4153b10 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.245+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc00f040 con 0x7ff3e4153b10 2023-10-06T05:20:52.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.245+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc014480 con 0x7ff3e4153b10 2023-10-06T05:20:52.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.245+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e40c3c00 con 0x7ff3e4153b10 2023-10-06T05:20:52.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.245+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e41596f0 con 0x7ff3e4153b10 2023-10-06T05:20:52.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.246+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e40c4400 con 0x7ff3e4153b10 2023-10-06T05:20:52.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.247+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3dc014830 con 0x7ff3e4153b10 2023-10-06T05:20:52.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.248+0000 7ff3d97fa700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3c8066f30 0x7ff3c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.248+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff3dc084d60 con 0x7ff3e4153b10 2023-10-06T05:20:52.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.248+0000 7ff3e919b700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3c8066f30 0x7ff3c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.249+0000 7ff3e919b700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3c8066f30 0x7ff3c80693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009d80 tx=0x7ff3e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.252+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3dc052120 con 0x7ff3e4153b10 2023-10-06T05:20:52.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.534+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3e404c380 con 0x7ff3e4153b10 2023-10-06T05:20:52.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.534+0000 7ff3d97fa700 1 -- 172.21.15.33:0/430515835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff3dc055740 con 0x7ff3e4153b10 2023-10-06T05:20:52.537 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:52.537 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:20:52.537 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:20:52.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.537 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:20:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.544 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.546 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.547 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.550 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:20:52.551 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:20:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:20:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:20:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:20:52.557 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3c8066f30 msgr2=0x7ff3c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3c8066f30 0x7ff3c80693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009d80 tx=0x7ff3e0008040 comp rx=0 tx=0).stop 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 msgr2=0x7ff3e4158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 0x7ff3e4158270 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009df0 tx=0x7ff3dc00b040 comp rx=0 tx=0).stop 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 shutdown_connections 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3c8066f30 0x7ff3c80693e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3e4153b10 0x7ff3e4158270 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3e40675f0 0x7ff3e41591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 --2- 172.21.15.33:0/430515835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3e40c6b30 0x7ff3e4157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 >> 172.21.15.33:0/430515835 conn(0x7ff3e406c560 msgr2=0x7ff3e4153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 shutdown_connections 2023-10-06T05:20:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.537+0000 7ff3ea19d700 1 -- 172.21.15.33:0/430515835 wait complete. 2023-10-06T05:20:52.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.744+0000 7ff17f7ff700 1 Processor -- start 2023-10-06T05:20:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.744+0000 7ff17f7ff700 1 -- start start 2023-10-06T05:20:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17f7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff17813fb70 0x7ff17806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17f7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff1780c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17f7ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1780ca8d0 con 0x7ff1780ca350 2023-10-06T05:20:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17f7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780cabb0 con 0x7ff17813fb70 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17f7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17806ad00 con 0x7ff17806ba40 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff17813fb70 0x7ff17806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff17813fb70 0x7ff17806b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41650/0 (socket says 172.21.15.33:41650) 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 -- 172.21.15.33:0/2615840918 learned_addr learned my addr 172.21.15.33:0/2615840918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17cd9a700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff1780c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 -- 172.21.15.33:0/2615840918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 msgr2=0x7ff1780c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff1780c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 -- 172.21.15.33:0/2615840918 >> v1:172.21.15.167:6789/0 conn(0x7ff1780ca350 legacy=0x7ff1780af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17d59b700 1 -- 172.21.15.33:0/2615840918 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17806afe0 con 0x7ff17813fb70 2023-10-06T05:20:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.746+0000 7ff17cd9a700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff1780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.747+0000 7ff17d59b700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17813fb70 0x7ff17806b500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099b0 tx=0x7ff174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f11c1e31172761 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.747+0000 7ff16ffff700 1 -- 172.21.15.33:0/2615840918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400b860 con 0x7ff17813fb70 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.747+0000 7ff16ffff700 1 -- 172.21.15.33:0/2615840918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17400b9c0 con 0x7ff17813fb70 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.747+0000 7ff16ffff700 1 -- 172.21.15.33:0/2615840918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400bd60 con 0x7ff17813fb70 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.747+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2615840918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17813fb70 msgr2=0x7ff17806b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.747+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17813fb70 0x7ff17806b500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099b0 tx=0x7ff174009250 comp rx=0 tx=0).stop 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.748+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2615840918 shutdown_connections 2023-10-06T05:20:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.748+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17813fb70 0x7ff17806b500 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.748+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2615840918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff1780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.748+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2615840918 >> 172.21.15.33:0/2615840918 conn(0x7ff1780675f0 msgr2=0x7ff1780679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.748+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2615840918 shutdown_connections 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.748+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2615840918 wait complete. 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.749+0000 7ff17f7ff700 1 Processor -- start 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.749+0000 7ff17f7ff700 1 -- start start 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17f7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff178137bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17f7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 0x7ff1780abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17f7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17813fb70 0x7ff1780ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17f7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780b0cd0 con 0x7ff1780ca350 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17f7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780ae260 con 0x7ff17813fb70 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17f7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780ae540 con 0x7ff17806ba40 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17cd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 0x7ff1780abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17cd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 0x7ff1780abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41652/0 (socket says 172.21.15.33:41652) 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.750+0000 7ff17cd9a700 1 -- 172.21.15.33:0/2286855995 learned_addr learned my addr 172.21.15.33:0/2286855995 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17d59b700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff178137bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17dd9c700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17813fb70 0x7ff1780ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17cd9a700 1 -- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 msgr2=0x7ff178137bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17cd9a700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff178137bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17cd9a700 1 -- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17813fb70 msgr2=0x7ff1780ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17cd9a700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17813fb70 0x7ff1780ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17cd9a700 1 -- 172.21.15.33:0/2286855995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1740094f0 con 0x7ff1780ca350 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17dd9c700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17813fb70 0x7ff1780ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17cd9a700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 0x7ff1780abd40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e8a0 tx=0x7ff168008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168009370 con 0x7ff1780ca350 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff168013410 con 0x7ff1780ca350 2023-10-06T05:20:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1780ae820 con 0x7ff1780ca350 2023-10-06T05:20:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.751+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1680137d0 con 0x7ff1780ca350 2023-10-06T05:20:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.752+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1781380f0 con 0x7ff1780ca350 2023-10-06T05:20:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.752+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1780b07c0 con 0x7ff1780ca350 2023-10-06T05:20:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.753+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff168013bc0 con 0x7ff1780ca350 2023-10-06T05:20:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.754+0000 7ff16d7fa700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff15c066f30 0x7ff15c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.754+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff1680840c0 con 0x7ff1780ca350 2023-10-06T05:20:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.755+0000 7ff17d59b700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff15c066f30 0x7ff15c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.755+0000 7ff17d59b700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff15c066f30 0x7ff15c0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099b0 tx=0x7ff174017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.758+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1780b07c0 con 0x7ff1780ca350 2023-10-06T05:20:52.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.962+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1780b07c0 con 0x7ff1780ca350 2023-10-06T05:20:52.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.963+0000 7ff16d7fa700 1 -- 172.21.15.33:0/2286855995 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+13651 (secure 0 0 0) 0x7ff1780b07c0 con 0x7ff1780ca350 2023-10-06T05:20:52.966 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:52.966 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:52.030408+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:52.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff15c066f30 msgr2=0x7ff15c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff15c066f30 0x7ff15c0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099b0 tx=0x7ff174017000 comp rx=0 tx=0).stop 2023-10-06T05:20:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 msgr2=0x7ff1780abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 0x7ff1780abd40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e8a0 tx=0x7ff168008040 comp rx=0 tx=0).stop 2023-10-06T05:20:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 shutdown_connections 2023-10-06T05:20:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff15c066f30 0x7ff15c0693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17813fb70 0x7ff1780ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17806ba40 0x7ff178137bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 --2- 172.21.15.33:0/2286855995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1780ca350 0x7ff1780abd40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 >> 172.21.15.33:0/2286855995 conn(0x7ff1780675f0 msgr2=0x7ff1781383c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 shutdown_connections 2023-10-06T05:20:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:52.965+0000 7ff17f7ff700 1 -- 172.21.15.33:0/2286855995 wait complete. 2023-10-06T05:20:52.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:20:52.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.177+0000 7f34e339d700 1 Processor -- start 2023-10-06T05:20:53.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.177+0000 7f34e339d700 1 -- start start 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.178+0000 7f34e339d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f34dc0af910 0x7f34dc15a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.178+0000 7f34e339d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc0ce890 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.178+0000 7f34e339d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc15d890 con 0x7f34dc0af540 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.178+0000 7f34e339d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc0afd50 con 0x7f34dc0af910 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f34dc0cd5e0 0x7f34dc0ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54622/0 (socket says 172.21.15.33:54622) 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1b9a700 1 -- 172.21.15.33:0/2275908926 learned_addr learned my addr 172.21.15.33:0/2275908926 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e2b9c700 1 --2- 172.21.15.33:0/2275908926 >> v2:172.21.15.167:3300/0 conn(0x7f34dc0af910 0x7f34dc15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141635814 0 0) 0x7f34dc0ce890 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c8003570 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119589502 0 0) 0x7f34dc15d890 con 0x7f34dc0af540 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34dc0ce890 con 0x7f34dc0af540 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e2b9c700 1 -- 172.21.15.33:0/2275908926 >> v1:172.21.15.167:6789/0 conn(0x7f34dc0af540 legacy=0x7f34dc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e2b9c700 1 -- 172.21.15.33:0/2275908926 >> v1:172.21.15.33:6789/0 conn(0x7f34dc0cd5e0 legacy=0x7f34dc0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940193549 0 0) 0x7f34c8003570 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.179+0000 7f34e2b9c700 1 -- 172.21.15.33:0/2275908926 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc15db70 con 0x7f34dc0af910 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.180+0000 7f34e2b9c700 1 --2- 172.21.15.33:0/2275908926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc15a650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009e60 tx=0x7f34d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=349fe8378a0e3fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.180+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d400e070 con 0x7f34dc0af910 2023-10-06T05:20:53.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.180+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d400b7a0 con 0x7f34dc0af910 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.180+0000 7f34e1399700 1 -- 172.21.15.33:0/2275908926 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc001470 con 0x7f34dc0af910 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.180+0000 7f34e339d700 1 -- 172.21.15.33:0/2275908926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 msgr2=0x7f34dc15a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.180+0000 7f34e339d700 1 --2- 172.21.15.33:0/2275908926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc15a650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009e60 tx=0x7f34d4009780 comp rx=0 tx=0).stop 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.181+0000 7f34e339d700 1 -- 172.21.15.33:0/2275908926 shutdown_connections 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.181+0000 7f34e339d700 1 --2- 172.21.15.33:0/2275908926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc15a650 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.181+0000 7f34e339d700 1 -- 172.21.15.33:0/2275908926 >> 172.21.15.33:0/2275908926 conn(0x7f34dc06d9b0 msgr2=0x7f34dc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.181+0000 7f34e339d700 1 -- 172.21.15.33:0/2275908926 shutdown_connections 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.181+0000 7f34e339d700 1 -- 172.21.15.33:0/2275908926 wait complete. 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.182+0000 7f34e339d700 1 Processor -- start 2023-10-06T05:20:53.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.182+0000 7f34e339d700 1 -- start start 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0af540 0x7f34dc0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 0x7f34dc15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc155690 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc16a5c0 con 0x7f34dc0af910 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc1558e0 con 0x7f34dc0af540 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 0x7f34dc15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 0x7f34dc15ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41664/0 (socket says 172.21.15.33:41664) 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e2b9c700 1 -- 172.21.15.33:0/4257820571 learned_addr learned my addr 172.21.15.33:0/4257820571 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e1b9a700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.183+0000 7f34e239b700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0af540 0x7f34dc0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e2b9c700 1 -- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0af540 msgr2=0x7f34dc0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e2b9c700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0af540 0x7f34dc0b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e2b9c700 1 -- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 msgr2=0x7f34dc148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e2b9c700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e2b9c700 1 -- 172.21.15.33:0/4257820571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4009a20 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e2b9c700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 0x7f34dc15ac60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f34d40099f0 tx=0x7f34d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4012cb0 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4012e30 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d401e6a0 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.184+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc155bc0 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.185+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc149450 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.186+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc1560c0 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.187+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34d401ea50 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.187+0000 7f34d27fc700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f34c0066f60 0x7f34c0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.188+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f34d4086750 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.188+0000 7f34e239b700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f34c0066f60 0x7f34c0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.189+0000 7f34e239b700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f34c0066f60 0x7f34c0069410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f34d8007fd0 tx=0x7f34d8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.192+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34d4053bd0 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.396+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34dc04c380 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.396+0000 7f34d27fc700 1 -- 172.21.15.33:0/4257820571 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34d40571f0 con 0x7f34dc0cd5e0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:20:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f34c0066f60 msgr2=0x7f34c0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f34c0066f60 0x7f34c0069410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f34d8007fd0 tx=0x7f34d8007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 msgr2=0x7f34dc15ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 0x7f34dc15ac60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f34d40099f0 tx=0x7f34d4017000 comp rx=0 tx=0).stop 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 shutdown_connections 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f34c0066f60 0x7f34c0069410 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc0cd5e0 0x7f34dc15ac60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0af540 0x7f34dc0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 --2- 172.21.15.33:0/4257820571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0af910 0x7f34dc148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 >> 172.21.15.33:0/4257820571 conn(0x7f34dc06d9b0 msgr2=0x7f34dc0c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 shutdown_connections 2023-10-06T05:20:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.399+0000 7f34e339d700 1 -- 172.21.15.33:0/4257820571 wait complete. 2023-10-06T05:20:53.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.605+0000 7fc3aad9d700 1 Processor -- start 2023-10-06T05:20:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.606+0000 7fc3aad9d700 1 -- start start 2023-10-06T05:20:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.607+0000 7fc3aad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a40cc7e0 0x7fc3a40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.607+0000 7fc3aad9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40ad430 con 0x7fc3a40b01f0 2023-10-06T05:20:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.607+0000 7fc3aad9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40afe20 con 0x7fc3a4156370 2023-10-06T05:20:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.607+0000 7fc3aad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a406bc70 con 0x7fc3a40cc7e0 2023-10-06T05:20:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.607+0000 7fc3aa59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc3a40b01f0 0x7fc3a40ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54634/0 (socket says 172.21.15.33:54634) 2023-10-06T05:20:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.607+0000 7fc3aa59c700 1 -- 172.21.15.33:0/3744845869 learned_addr learned my addr 172.21.15.33:0/3744845869 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a959a700 1 --2- 172.21.15.33:0/3744845869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a40cc7e0 0x7fc3a40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374058718 0 0) 0x7fc3a40ad430 con 0x7fc3a40b01f0 2023-10-06T05:20:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc390003570 con 0x7fc3a40b01f0 2023-10-06T05:20:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189202150 0 0) 0x7fc3a40afe20 con 0x7fc3a4156370 2023-10-06T05:20:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a40ad430 con 0x7fc3a4156370 2023-10-06T05:20:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631257671 0 0) 0x7fc390003570 con 0x7fc3a40b01f0 2023-10-06T05:20:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a959a700 1 -- 172.21.15.33:0/3744845869 >> v1:172.21.15.167:6789/0 conn(0x7fc3a4156370 legacy=0x7fc3a40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a959a700 1 -- 172.21.15.33:0/3744845869 >> v1:172.21.15.33:6789/0 conn(0x7fc3a40b01f0 legacy=0x7fc3a40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a959a700 1 -- 172.21.15.33:0/3744845869 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40cccc0 con 0x7fc3a40cc7e0 2023-10-06T05:20:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.608+0000 7fc3a959a700 1 --2- 172.21.15.33:0/3744845869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a40cc7e0 0x7fc3a40af8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc394009910 tx=0x7fc3940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b0e0841804844d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0017f0 con 0x7fc3a40cc7e0 2023-10-06T05:20:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394011420 con 0x7fc3a40cc7e0 2023-10-06T05:20:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3a8d99700 1 -- 172.21.15.33:0/3744845869 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940117c0 con 0x7fc3a40cc7e0 2023-10-06T05:20:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 -- 172.21.15.33:0/3744845869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a40cc7e0 msgr2=0x7fc3a40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/3744845869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a40cc7e0 0x7fc3a40af8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc394009910 tx=0x7fc3940091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 -- 172.21.15.33:0/3744845869 shutdown_connections 2023-10-06T05:20:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/3744845869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a40cc7e0 0x7fc3a40af8e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 -- 172.21.15.33:0/3744845869 >> 172.21.15.33:0/3744845869 conn(0x7fc3a406d030 msgr2=0x7fc3a40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 -- 172.21.15.33:0/3744845869 shutdown_connections 2023-10-06T05:20:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.609+0000 7fc3aad9d700 1 -- 172.21.15.33:0/3744845869 wait complete. 2023-10-06T05:20:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.611+0000 7fc3aad9d700 1 Processor -- start 2023-10-06T05:20:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.611+0000 7fc3aad9d700 1 -- start start 2023-10-06T05:20:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 0x7fc3a4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3aad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3a40cc7e0 0x7fc3a41451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3aad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a4156370 0x7fc3a40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3aad9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40a62e0 con 0x7fc3a40b01f0 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3aad9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a4159240 con 0x7fc3a40cc7e0 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3aad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a4157d60 con 0x7fc3a4156370 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3a9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 0x7fc3a4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3a9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 0x7fc3a4158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41678/0 (socket says 172.21.15.33:41678) 2023-10-06T05:20:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.612+0000 7fc3a9d9b700 1 -- 172.21.15.33:0/225111381 learned_addr learned my addr 172.21.15.33:0/225111381 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3aa59c700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a4156370 0x7fc3a40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a959a700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3a40cc7e0 0x7fc3a41451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a9d9b700 1 -- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a4156370 msgr2=0x7fc3a40a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a9d9b700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a4156370 0x7fc3a40a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a9d9b700 1 -- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3a40cc7e0 msgr2=0x7fc3a41451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a9d9b700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3a40cc7e0 0x7fc3a41451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a9d9b700 1 -- 172.21.15.33:0/225111381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394009420 con 0x7fc3a40b01f0 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a959a700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3a40cc7e0 0x7fc3a41451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3aa59c700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a4156370 0x7fc3a40a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3a9d9b700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 0x7fc3a4158d00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c790 tx=0x7fc3a0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0010510 con 0x7fc3a40b01f0 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0010670 con 0x7fc3a40b01f0 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a41581f0 con 0x7fc3a40b01f0 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0010a10 con 0x7fc3a40b01f0 2023-10-06T05:20:53.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.613+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a4157790 con 0x7fc3a40b01f0 2023-10-06T05:20:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.614+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a4158040 con 0x7fc3a40b01f0 2023-10-06T05:20:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.615+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3a001b400 con 0x7fc3a40b01f0 2023-10-06T05:20:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.616+0000 7fc399ffb700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc388066f30 0x7fc3880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.616+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc3a00841b0 con 0x7fc3a40b01f0 2023-10-06T05:20:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.617+0000 7fc3a959a700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc388066f30 0x7fc3880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.618+0000 7fc3a959a700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc388066f30 0x7fc3880693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40ccbf0 tx=0x7fc394016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.621+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3a0051570 con 0x7fc3a40b01f0 2023-10-06T05:20:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.822+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3a404c380 con 0x7fc3a40b01f0 2023-10-06T05:20:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.822+0000 7fc399ffb700 1 -- 172.21.15.33:0/225111381 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v85) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc3a0054b90 con 0x7fc3a40b01f0 2023-10-06T05:20:53.825 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc388066f30 msgr2=0x7fc3880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc388066f30 0x7fc3880693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40ccbf0 tx=0x7fc394016000 comp rx=0 tx=0).stop 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 msgr2=0x7fc3a4158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 0x7fc3a4158d00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c790 tx=0x7fc3a0007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 shutdown_connections 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc388066f30 0x7fc3880693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3a4156370 0x7fc3a40a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3a40b01f0 0x7fc3a4158d00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 --2- 172.21.15.33:0/225111381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3a40cc7e0 0x7fc3a41451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 >> 172.21.15.33:0/225111381 conn(0x7fc3a406d030 msgr2=0x7fc3a4069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 shutdown_connections 2023-10-06T05:20:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:53.825+0000 7fc3aad9d700 1 -- 172.21.15.33:0/225111381 wait complete. 2023-10-06T05:20:53.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:20:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.030+0000 7f3f4299d700 1 Processor -- start 2023-10-06T05:20:54.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.030+0000 7f3f4299d700 1 -- start start 2023-10-06T05:20:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c159bb0 0x7f3f3c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4299d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0cbd80 con 0x7f3f3c0cf3b0 2023-10-06T05:20:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4299d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0af5f0 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0af8d0 con 0x7f3f3c159bb0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4119a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3f3c0cf3b0 0x7f3f3c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54640/0 (socket says 172.21.15.33:54640) 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4119a700 1 -- 172.21.15.33:0/3100060580 learned_addr learned my addr 172.21.15.33:0/3100060580 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.032+0000 7f3f4219c700 1 --2- 172.21.15.33:0/3100060580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c159bb0 0x7f3f3c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654046542 0 0) 0x7f3f3c0cbd80 con 0x7f3f3c0cf3b0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f28003570 con 0x7f3f3c0cf3b0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930770251 0 0) 0x7f3f3c0af5f0 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f3c0cbd80 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f4219c700 1 -- 172.21.15.33:0/3100060580 >> v1:172.21.15.167:6789/0 conn(0x7f3f3c0ccfc0 legacy=0x7f3f3c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f4219c700 1 -- 172.21.15.33:0/3100060580 >> v1:172.21.15.33:6789/0 conn(0x7f3f3c0cf3b0 legacy=0x7f3f3c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3673834953 0 0) 0x7f3f28003570 con 0x7f3f3c0cf3b0 2023-10-06T05:20:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f4219c700 1 -- 172.21.15.33:0/3100060580 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0afbb0 con 0x7f3f3c159bb0 2023-10-06T05:20:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f4219c700 1 --2- 172.21.15.33:0/3100060580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c159bb0 0x7f3f3c0b0460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009ec0 tx=0x7f3f34009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a200a5332a4d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.033+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3400e070 con 0x7f3f3c159bb0 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3400b870 con 0x7f3f3c159bb0 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f40999700 1 -- 172.21.15.33:0/3100060580 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c001470 con 0x7f3f3c159bb0 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 -- 172.21.15.33:0/3100060580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c159bb0 msgr2=0x7f3f3c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 --2- 172.21.15.33:0/3100060580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c159bb0 0x7f3f3c0b0460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009ec0 tx=0x7f3f34009780 comp rx=0 tx=0).stop 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 -- 172.21.15.33:0/3100060580 shutdown_connections 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 --2- 172.21.15.33:0/3100060580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c159bb0 0x7f3f3c0b0460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 -- 172.21.15.33:0/3100060580 >> 172.21.15.33:0/3100060580 conn(0x7f3f3c06cff0 msgr2=0x7f3f3c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 -- 172.21.15.33:0/3100060580 shutdown_connections 2023-10-06T05:20:54.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.034+0000 7f3f4299d700 1 -- 172.21.15.33:0/3100060580 wait complete. 2023-10-06T05:20:54.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.036+0000 7f3f4299d700 1 Processor -- start 2023-10-06T05:20:54.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.036+0000 7f3f4299d700 1 -- start start 2023-10-06T05:20:54.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 0x7f3f3c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f3c159bb0 0x7f3f3c0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c0c91f0 0x7f3f3c0c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0c9670 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c157220 con 0x7f3f3c159bb0 2023-10-06T05:20:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c14eb40 con 0x7f3f3c0c91f0 2023-10-06T05:20:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 0x7f3f3c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 0x7f3f3c14e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41680/0 (socket says 172.21.15.33:41680) 2023-10-06T05:20:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 -- 172.21.15.33:0/2283554739 learned_addr learned my addr 172.21.15.33:0/2283554739 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4219c700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c0c91f0 0x7f3f3c0c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4119a700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f3c159bb0 0x7f3f3c0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 -- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c0c91f0 msgr2=0x7f3f3c0c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c0c91f0 0x7f3f3c0c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 -- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f3c159bb0 msgr2=0x7f3f3c0c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f3c159bb0 0x7f3f3c0c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.037+0000 7f3f4199b700 1 -- 172.21.15.33:0/2283554739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f34009a20 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.038+0000 7f3f4199b700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 0x7f3f3c14e4b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c790 tx=0x7f3f38007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.038+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38010510 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.038+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38010670 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.038+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c0c8350 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.038+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38010a10 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.038+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c0c8910 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.039+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c04c380 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.040+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f3801b400 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.041+0000 7f3f31ffb700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f20066f30 0x7f3f200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.041+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3f38084190 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.041+0000 7f3f4119a700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f20066f30 0x7f3f200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.042+0000 7f3f4119a700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f20066f30 0x7f3f200693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c1589f0 tx=0x7f3f2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.045+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f38051550 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.250+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3f3c071060 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.251+0000 7f3f31ffb700 1 -- 172.21.15.33:0/2283554739 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3f3c071060 con 0x7f3f3c0ccfc0 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:20:54.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:20:54.255 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:20:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f20066f30 msgr2=0x7f3f200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f20066f30 0x7f3f200693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c1589f0 tx=0x7f3f2c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 msgr2=0x7f3f3c14e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 0x7f3f3c14e4b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c790 tx=0x7f3f38007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 shutdown_connections 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3f20066f30 0x7f3f200693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f3c0c91f0 0x7f3f3c0c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f3c0ccfc0 0x7f3f3c14e4b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 --2- 172.21.15.33:0/2283554739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f3c159bb0 0x7f3f3c0c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.253+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 >> 172.21.15.33:0/2283554739 conn(0x7f3f3c06cff0 msgr2=0x7f3f3c0c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.254+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 shutdown_connections 2023-10-06T05:20:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.254+0000 7f3f4299d700 1 -- 172.21.15.33:0/2283554739 wait complete. 2023-10-06T05:20:54.274 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:20:54.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:20:54.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.462+0000 7f5e9f59e700 1 Processor -- start 2023-10-06T05:20:54.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.462+0000 7f5e9f59e700 1 -- start start 2023-10-06T05:20:54.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ea0157320 0x7f5ea00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5ea01576f0 0x7f5ea00ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea00b0840 con 0x7f5ea00b13d0 2023-10-06T05:20:54.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea01585c0 con 0x7f5ea0157320 2023-10-06T05:20:54.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00b1ff0 con 0x7f5ea01576f0 2023-10-06T05:20:54.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ea0157320 0x7f5ea00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ea0157320 0x7f5ea00adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41696/0 (socket says 172.21.15.33:41696) 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9e59c700 1 -- 172.21.15.33:0/766043017 learned_addr learned my addr 172.21.15.33:0/766043017 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.463+0000 7f5e9dd9b700 1 --2- 172.21.15.33:0/766043017 >> v2:172.21.15.167:3300/0 conn(0x7f5ea01576f0 0x7f5ea00ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9e59c700 1 -- 172.21.15.33:0/766043017 >> v2:172.21.15.167:3300/0 conn(0x7f5ea01576f0 msgr2=0x7f5ea00ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9e59c700 1 --2- 172.21.15.33:0/766043017 >> v2:172.21.15.167:3300/0 conn(0x7f5ea01576f0 0x7f5ea00ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9e59c700 1 -- 172.21.15.33:0/766043017 >> v1:172.21.15.167:6789/0 conn(0x7f5ea00b13d0 legacy=0x7f5ea015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9e59c700 1 -- 172.21.15.33:0/766043017 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0157c90 con 0x7f5ea0157320 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9dd9b700 1 --2- 172.21.15.33:0/766043017 >> v2:172.21.15.167:3300/0 conn(0x7f5ea01576f0 0x7f5ea00ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9e59c700 1 --2- 172.21.15.33:0/766043017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea0157320 0x7f5ea00adf10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00646a0 tx=0x7f5e90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71a0608de9f74ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9d59a700 1 -- 172.21.15.33:0/766043017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000b8f0 con 0x7f5ea0157320 2023-10-06T05:20:54.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9d59a700 1 -- 172.21.15.33:0/766043017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9000ba50 con 0x7f5ea0157320 2023-10-06T05:20:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.464+0000 7f5e9d59a700 1 -- 172.21.15.33:0/766043017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000bdf0 con 0x7f5ea0157320 2023-10-06T05:20:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 -- 172.21.15.33:0/766043017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea0157320 msgr2=0x7f5ea00adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/766043017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea0157320 0x7f5ea00adf10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00646a0 tx=0x7f5e90009250 comp rx=0 tx=0).stop 2023-10-06T05:20:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 -- 172.21.15.33:0/766043017 shutdown_connections 2023-10-06T05:20:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/766043017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea0157320 0x7f5ea00adf10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/766043017 >> v2:172.21.15.167:3300/0 conn(0x7f5ea01576f0 0x7f5ea00ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 -- 172.21.15.33:0/766043017 >> 172.21.15.33:0/766043017 conn(0x7f5ea0068090 msgr2=0x7f5ea006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 -- 172.21.15.33:0/766043017 shutdown_connections 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.465+0000 7f5e9f59e700 1 -- 172.21.15.33:0/766043017 wait complete. 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.466+0000 7f5e9f59e700 1 Processor -- start 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 -- start start 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 0x7f5ea00c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ea0157320 0x7f5ea0167270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01576f0 0x7f5ea014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00a5480 con 0x7f5ea00b13d0 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00a5760 con 0x7f5ea01576f0 2023-10-06T05:20:54.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.467+0000 7f5e9f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01677b0 con 0x7f5ea0157320 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 0x7f5ea00c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 0x7f5ea00c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41702/0 (socket says 172.21.15.33:41702) 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 -- 172.21.15.33:0/4026730303 learned_addr learned my addr 172.21.15.33:0/4026730303 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 -- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ea0157320 msgr2=0x7f5ea0167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ea0157320 0x7f5ea0167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 -- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01576f0 msgr2=0x7f5ea014f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01576f0 0x7f5ea014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 -- 172.21.15.33:0/4026730303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900094f0 con 0x7f5ea00b13d0 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.468+0000 7f5e9e59c700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 0x7f5ea00c9520 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009290 tx=0x7f5e90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.469+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90006bc0 con 0x7f5ea00b13d0 2023-10-06T05:20:54.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.469+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9001f070 con 0x7f5ea00b13d0 2023-10-06T05:20:54.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.469+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9001a870 con 0x7f5ea00b13d0 2023-10-06T05:20:54.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.469+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea0167a40 con 0x7f5ea00b13d0 2023-10-06T05:20:54.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.469+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea0168000 con 0x7f5ea00b13d0 2023-10-06T05:20:54.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.470+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea01681f0 con 0x7f5ea00b13d0 2023-10-06T05:20:54.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.471+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e9001ac20 con 0x7f5ea00b13d0 2023-10-06T05:20:54.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.472+0000 7f5e8e7fc700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e80066f30 0x7f5e800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.472+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5e90021070 con 0x7f5ea00b13d0 2023-10-06T05:20:54.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.472+0000 7f5e9dd9b700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e80066f30 0x7f5e800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.473+0000 7f5e9dd9b700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e80066f30 0x7f5e800693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e94006fd0 tx=0x7f5e94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.476+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ea01681f0 con 0x7f5ea00b13d0 2023-10-06T05:20:54.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:54.678+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5ea01681f0 con 0x7f5ea00b13d0 2023-10-06T05:20:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.039+0000 7f5e8e7fc700 1 -- 172.21.15.33:0/4026730303 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5ea01681f0 con 0x7f5ea00b13d0 2023-10-06T05:20:55.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.044+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e80066f30 msgr2=0x7f5e800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.044+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e80066f30 0x7f5e800693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e94006fd0 tx=0x7f5e94008040 comp rx=0 tx=0).stop 2023-10-06T05:20:55.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.044+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 msgr2=0x7f5ea00c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.044+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 0x7f5ea00c9520 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009290 tx=0x7f5e90017000 comp rx=0 tx=0).stop 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 shutdown_connections 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e80066f30 0x7f5e800693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01576f0 0x7f5ea014f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea00b13d0 0x7f5ea00c9520 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 --2- 172.21.15.33:0/4026730303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ea0157320 0x7f5ea0167270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 >> 172.21.15.33:0/4026730303 conn(0x7f5ea0068090 msgr2=0x7f5ea0151760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 shutdown_connections 2023-10-06T05:20:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.045+0000 7f5e9f59e700 1 -- 172.21.15.33:0/4026730303 wait complete. 2023-10-06T05:20:55.049 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:20:55.065 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:20:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.256+0000 7f72eaf9d700 1 Processor -- start 2023-10-06T05:20:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.256+0000 7f72eaf9d700 1 -- start start 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.257+0000 7f72eaf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72e40af540 0x7f72e40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.257+0000 7f72eaf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f72e415a7d0 0x7f72e40cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.257+0000 7f72eaf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e415ddb0 con 0x7f72e40cced0 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.257+0000 7f72eaf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e40cba60 con 0x7f72e40af540 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.257+0000 7f72eaf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e415d1a0 con 0x7f72e415a7d0 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72e40af540 0x7f72e40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72e40af540 0x7f72e40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41716/0 (socket says 172.21.15.33:41716) 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 -- 172.21.15.33:0/2256046478 learned_addr learned my addr 172.21.15.33:0/2256046478 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 -- 172.21.15.33:0/2256046478 >> v2:172.21.15.167:3300/0 conn(0x7f72e415a7d0 msgr2=0x7f72e40cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 --2- 172.21.15.33:0/2256046478 >> v2:172.21.15.167:3300/0 conn(0x7f72e415a7d0 0x7f72e40cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 -- 172.21.15.33:0/2256046478 >> v1:172.21.15.167:6789/0 conn(0x7f72e40cced0 legacy=0x7f72e406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 -- 172.21.15.33:0/2256046478 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e415d480 con 0x7f72e40af540 2023-10-06T05:20:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.258+0000 7f72e9f9b700 1 --2- 172.21.15.33:0/2256046478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e40af540 0x7f72e40af910 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f72e4089600 tx=0x7f72e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6248a6ca165b4173 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72e8f99700 1 -- 172.21.15.33:0/2256046478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e000b820 con 0x7f72e40af540 2023-10-06T05:20:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72e8f99700 1 -- 172.21.15.33:0/2256046478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e000b980 con 0x7f72e40af540 2023-10-06T05:20:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72e8f99700 1 -- 172.21.15.33:0/2256046478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e000bd20 con 0x7f72e40af540 2023-10-06T05:20:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 -- 172.21.15.33:0/2256046478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e40af540 msgr2=0x7f72e40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/2256046478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e40af540 0x7f72e40af910 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f72e4089600 tx=0x7f72e0009250 comp rx=0 tx=0).stop 2023-10-06T05:20:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 -- 172.21.15.33:0/2256046478 shutdown_connections 2023-10-06T05:20:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/2256046478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e40af540 0x7f72e40af910 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/2256046478 >> v2:172.21.15.167:3300/0 conn(0x7f72e415a7d0 0x7f72e40cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 -- 172.21.15.33:0/2256046478 >> 172.21.15.33:0/2256046478 conn(0x7f72e406d9b0 msgr2=0x7f72e406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 -- 172.21.15.33:0/2256046478 shutdown_connections 2023-10-06T05:20:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.259+0000 7f72eaf9d700 1 -- 172.21.15.33:0/2256046478 wait complete. 2023-10-06T05:20:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.261+0000 7f72eaf9d700 1 Processor -- start 2023-10-06T05:20:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.261+0000 7f72eaf9d700 1 -- start start 2023-10-06T05:20:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72eaf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e40af540 0x7f72e416d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72eaf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e40cced0 0x7f72e415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72eaf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 0x7f72e415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72eaf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4158280 con 0x7f72e415a7d0 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72eaf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40b3270 con 0x7f72e40cced0 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72eaf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40b3ad0 con 0x7f72e40af540 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72ea79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 0x7f72e415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72ea79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 0x7f72e415c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41720/0 (socket says 172.21.15.33:41720) 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.262+0000 7f72ea79c700 1 -- 172.21.15.33:0/1255207199 learned_addr learned my addr 172.21.15.33:0/1255207199 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72e9f9b700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e40af540 0x7f72e416d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72e979a700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e40cced0 0x7f72e415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72ea79c700 1 -- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e40af540 msgr2=0x7f72e416d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72ea79c700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e40af540 0x7f72e416d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72ea79c700 1 -- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e40cced0 msgr2=0x7f72e415bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72ea79c700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e40cced0 0x7f72e415bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72ea79c700 1 -- 172.21.15.33:0/1255207199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00094f0 con 0x7f72e415a7d0 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72e979a700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e40cced0 0x7f72e415bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72ea79c700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 0x7f72e415c160 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c930 tx=0x7f72dc0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00ce70 con 0x7f72e415a7d0 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00a7e0 con 0x7f72e415a7d0 2023-10-06T05:20:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00aba0 con 0x7f72e415a7d0 2023-10-06T05:20:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e40b5d10 con 0x7f72e415a7d0 2023-10-06T05:20:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.263+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e40b3760 con 0x7f72e415a7d0 2023-10-06T05:20:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.264+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e404c380 con 0x7f72e415a7d0 2023-10-06T05:20:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.266+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72dc006ba0 con 0x7f72e415a7d0 2023-10-06T05:20:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.267+0000 7f72d9ffb700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.267+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f72dc084ec0 con 0x7f72e415a7d0 2023-10-06T05:20:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.268+0000 7f72e9f9b700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.268+0000 7f72e9f9b700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f72e4089600 tx=0x7f72e0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.271+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72dc052190 con 0x7f72e415a7d0 2023-10-06T05:20:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:55.471+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72e415d760 con 0x7f72e415a7d0 2023-10-06T05:20:56.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.047+0000 7f72d9ffb700 1 -- 172.21.15.33:0/1255207199 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7f72dc0557b0 con 0x7f72e415a7d0 2023-10-06T05:20:56.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.053+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c8066f30 msgr2=0x7f72c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.053+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f72e4089600 tx=0x7f72e0016000 comp rx=0 tx=0).stop 2023-10-06T05:20:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.053+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 msgr2=0x7f72e415c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.053+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 0x7f72e415c160 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c930 tx=0x7f72dc0026e0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.057+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 shutdown_connections 2023-10-06T05:20:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.057+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.057+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e415a7d0 0x7f72e415c160 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.057+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e40af540 0x7f72e416d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.057+0000 7f72eaf9d700 1 --2- 172.21.15.33:0/1255207199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e40cced0 0x7f72e415bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.057+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 >> 172.21.15.33:0/1255207199 conn(0x7f72e406d9b0 msgr2=0x7f72e4158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.063+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 shutdown_connections 2023-10-06T05:20:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.063+0000 7f72eaf9d700 1 -- 172.21.15.33:0/1255207199 wait complete. 2023-10-06T05:20:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:20:56.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:20:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.279+0000 7f199af9d700 1 Processor -- start 2023-10-06T05:20:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.279+0000 7f199af9d700 1 -- start start 2023-10-06T05:20:56.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.280+0000 7f199af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 0x7f1994067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.280+0000 7f199af9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19940ac020 con 0x7f19940b1f90 2023-10-06T05:20:56.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.280+0000 7f199af9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199406ae10 con 0x7f1994068140 2023-10-06T05:20:56.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.280+0000 7f199af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199406a2d0 con 0x7f19940abbd0 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1999f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f19940b1f90 0x7f1994070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54652/0 (socket says 172.21.15.33:54652) 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1999f9b700 1 -- 172.21.15.33:0/3164573440 learned_addr learned my addr 172.21.15.33:0/3164573440 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f199979a700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 0x7f1994067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453760504 0 0) 0x7f19940ac020 con 0x7f19940b1f90 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1980003570 con 0x7f19940b1f90 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211737865 0 0) 0x7f199406ae10 con 0x7f1994068140 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19940ac020 con 0x7f1994068140 2023-10-06T05:20:56.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2984814208 0 0) 0x7f1980003570 con 0x7f19940b1f90 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 msgr2=0x7f1994067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 0x7f1994067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 >> v1:172.21.15.167:6789/0 conn(0x7f1994068140 legacy=0x7f1994159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1994068580 con 0x7f19940b1f90 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1990001470 con 0x7f19940b1f90 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 >> v1:172.21.15.33:6789/0 conn(0x7f19940b1f90 legacy=0x7f1994070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.281+0000 7f1998f99700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980003c20 0x7f1980004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f1998f99700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980004810 0x7f1980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f1998f99700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 0x7f19800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199406a2d0 con 0x7f1980004810 2023-10-06T05:20:56.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000bb40 con 0x7f1980003c20 2023-10-06T05:20:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198000bd00 con 0x7f19940abbd0 2023-10-06T05:20:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199979a700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980004810 0x7f1980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f1999f9b700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980003c20 0x7f1980004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199a79c700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 0x7f19800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199979a700 1 -- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 msgr2=0x7f19800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199979a700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19940abbd0 0x7f19800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199979a700 1 -- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980003c20 msgr2=0x7f1980004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199979a700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980003c20 0x7f1980004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.282+0000 7f199979a700 1 -- 172.21.15.33:0/3164573440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980003570 con 0x7f1980004810 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199979a700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980004810 0x7f1980004c80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1984009c00 tx=0x7f19840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400c58ff551d7f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400e710 con 0x7f1980004810 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198400e870 con 0x7f1980004810 2023-10-06T05:20:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f1998f99700 1 -- 172.21.15.33:0/3164573440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400ec30 con 0x7f1980004810 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199af9d700 1 -- 172.21.15.33:0/3164573440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980004810 msgr2=0x7f1980004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199af9d700 1 --2- 172.21.15.33:0/3164573440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1980004810 0x7f1980004c80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1984009c00 tx=0x7f19840091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199a79c700 1 -- 172.21.15.33:0/3164573440 reap_dead start 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199af9d700 1 -- 172.21.15.33:0/3164573440 shutdown_connections 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199af9d700 1 -- 172.21.15.33:0/3164573440 >> 172.21.15.33:0/3164573440 conn(0x7f199406ea70 msgr2=0x7f199406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199af9d700 1 -- 172.21.15.33:0/3164573440 shutdown_connections 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.283+0000 7f199af9d700 1 -- 172.21.15.33:0/3164573440 wait complete. 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.285+0000 7f199af9d700 1 Processor -- start 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.285+0000 7f199af9d700 1 -- start start 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199af9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980004810 0x7f19940b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f199413ece0 0x7f199413f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199af9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 0x7f199416d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199af9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199416a590 con 0x7f199416d2e0 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199af9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994169ca0 con 0x7f1980004810 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1994169f40 con 0x7f199413ece0 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199a79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 0x7f199416d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199a79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 0x7f199416d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41742/0 (socket says 172.21.15.33:41742) 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199a79c700 1 -- 172.21.15.33:0/445415425 learned_addr learned my addr 172.21.15.33:0/445415425 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f1999f9b700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980004810 0x7f19940b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.286+0000 7f199979a700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f199413ece0 0x7f199413f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199a79c700 1 -- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f199413ece0 msgr2=0x7f199413f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199a79c700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f199413ece0 0x7f199413f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199a79c700 1 -- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980004810 msgr2=0x7f19940b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199a79c700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980004810 0x7f19940b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199a79c700 1 -- 172.21.15.33:0/445415425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199416a220 con 0x7f199416d2e0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199a79c700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 0x7f199416d780 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1994069b30 tx=0x7f198c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c004070 con 0x7f199416d2e0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c006db0 con 0x7f199416d2e0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00c7f0 con 0x7f199416d2e0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199413f690 con 0x7f199416d2e0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.287+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199416aa20 con 0x7f199416d2e0 2023-10-06T05:20:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.288+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199416a700 con 0x7f199416d2e0 2023-10-06T05:20:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.289+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f198c00cba0 con 0x7f199416d2e0 2023-10-06T05:20:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.290+0000 7f1989ffb700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1978066f30 0x7f19780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.290+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f198c084580 con 0x7f199416d2e0 2023-10-06T05:20:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.291+0000 7f1999f9b700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1978066f30 0x7f19780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.292+0000 7f1999f9b700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1978066f30 0x7f19780693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1990001200 tx=0x7f1990001e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.295+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f198c051650 con 0x7f199416d2e0 2023-10-06T05:20:56.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:56.544+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f199404c380 con 0x7f199416d2e0 2023-10-06T05:20:57.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.049+0000 7f1989ffb700 1 -- 172.21.15.33:0/445415425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f198c054c70 con 0x7f199416d2e0 2023-10-06T05:20:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1978066f30 msgr2=0x7f19780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1978066f30 0x7f19780693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1990001200 tx=0x7f1990001e10 comp rx=0 tx=0).stop 2023-10-06T05:20:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 msgr2=0x7f199416d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 0x7f199416d780 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1994069b30 tx=0x7f198c0017a0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 shutdown_connections 2023-10-06T05:20:57.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1978066f30 0x7f19780693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f199416d2e0 0x7f199416d780 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1980004810 0x7f19940b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 --2- 172.21.15.33:0/445415425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f199413ece0 0x7f199413f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.057+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 >> 172.21.15.33:0/445415425 conn(0x7f199406ea70 msgr2=0x7f19941534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.063+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 shutdown_connections 2023-10-06T05:20:57.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.063+0000 7f199af9d700 1 -- 172.21.15.33:0/445415425 wait complete. 2023-10-06T05:20:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T05:20:57.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:57.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.288+0000 7fac3ffff700 1 Processor -- start 2023-10-06T05:20:57.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.288+0000 7fac3ffff700 1 -- start start 2023-10-06T05:20:57.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.289+0000 7fac3ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.289+0000 7fac3ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac400cf3b0 0x7fac401594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.289+0000 7fac3ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac400cbd80 con 0x7fac400cc750 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.289+0000 7fac3ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac400af5f0 con 0x7fac400cf3b0 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.289+0000 7fac3ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400af8d0 con 0x7fac400ccfc0 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3e7fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac400cf3b0 0x7fac401594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:48044/0 (socket says 172.21.15.33:48044) 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 -- 172.21.15.33:0/4045476170 learned_addr learned my addr 172.21.15.33:0/4045476170 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:57.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3f7fe700 1 --1- 172.21.15.33:0/4045476170 >> v1:172.21.15.167:6789/0 conn(0x7fac400cc750 0x7fac400b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:36590/0 (socket says 172.21.15.33:36590) 2023-10-06T05:20:57.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 -- 172.21.15.33:0/4045476170 >> v2:172.21.15.167:3300/0 conn(0x7fac400cf3b0 msgr2=0x7fac401594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 --2- 172.21.15.33:0/4045476170 >> v2:172.21.15.167:3300/0 conn(0x7fac400cf3b0 0x7fac401594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 -- 172.21.15.33:0/4045476170 >> v1:172.21.15.167:6789/0 conn(0x7fac400cc750 legacy=0x7fac400b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3effd700 1 -- 172.21.15.33:0/4045476170 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400afbb0 con 0x7fac400ccfc0 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.290+0000 7fac3e7fc700 1 --2- 172.21.15.33:0/4045476170 >> v2:172.21.15.167:3300/0 conn(0x7fac400cf3b0 0x7fac401594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.291+0000 7fac3effd700 1 --2- 172.21.15.33:0/4045476170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400cc210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac300099b0 tx=0x7fac30009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b0826d55c3c512 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.291+0000 7fac3dffb700 1 -- 172.21.15.33:0/4045476170 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30009ea0 con 0x7fac400ccfc0 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.291+0000 7fac3dffb700 1 -- 172.21.15.33:0/4045476170 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3000b860 con 0x7fac400ccfc0 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.291+0000 7fac3dffb700 1 -- 172.21.15.33:0/4045476170 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000bc00 con 0x7fac400ccfc0 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 -- 172.21.15.33:0/4045476170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 msgr2=0x7fac400cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 --2- 172.21.15.33:0/4045476170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400cc210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac300099b0 tx=0x7fac30009250 comp rx=0 tx=0).stop 2023-10-06T05:20:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 -- 172.21.15.33:0/4045476170 shutdown_connections 2023-10-06T05:20:57.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 --2- 172.21.15.33:0/4045476170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400cc210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 --2- 172.21.15.33:0/4045476170 >> v2:172.21.15.167:3300/0 conn(0x7fac400cf3b0 0x7fac401594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 -- 172.21.15.33:0/4045476170 >> 172.21.15.33:0/4045476170 conn(0x7fac4006cff0 msgr2=0x7fac4006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 -- 172.21.15.33:0/4045476170 shutdown_connections 2023-10-06T05:20:57.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.292+0000 7fac3ffff700 1 -- 172.21.15.33:0/4045476170 wait complete. 2023-10-06T05:20:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.293+0000 7fac3ffff700 1 Processor -- start 2023-10-06T05:20:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.293+0000 7fac3ffff700 1 -- start start 2023-10-06T05:20:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.294+0000 7fac3ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac400cc750 0x7fac4014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.294+0000 7fac3ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.294+0000 7fac3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 0x7fac4014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.294+0000 7fac3ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac400c86b0 con 0x7fac40150380 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.294+0000 7fac3ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4014fb00 con 0x7fac400cc750 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.294+0000 7fac3ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400ca0c0 con 0x7fac400ccfc0 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 0x7fac4014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 0x7fac4014f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41756/0 (socket says 172.21.15.33:41756) 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 -- 172.21.15.33:0/3384309156 learned_addr learned my addr 172.21.15.33:0/3384309156 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3effd700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac400cc750 0x7fac4014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3e7fc700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 -- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 msgr2=0x7fac400c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 -- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac400cc750 msgr2=0x7fac4014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac400cc750 0x7fac4014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 -- 172.21.15.33:0/3384309156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac34009420 con 0x7fac40150380 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3effd700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac400cc750 0x7fac4014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.295+0000 7fac3f7fe700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 0x7fac4014f5c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cd60 tx=0x7fac38001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.296+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38005df0 con 0x7fac40150380 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.296+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3800a8c0 con 0x7fac40150380 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.296+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac300094f0 con 0x7fac40150380 2023-10-06T05:20:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.296+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3800ac80 con 0x7fac40150380 2023-10-06T05:20:57.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.296+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac400c8060 con 0x7fac40150380 2023-10-06T05:20:57.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.297+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac400c8860 con 0x7fac40150380 2023-10-06T05:20:57.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.298+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac38007300 con 0x7fac40150380 2023-10-06T05:20:57.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.298+0000 7fac26ffd700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac2c066f30 0x7fac2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.299+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fac38084860 con 0x7fac40150380 2023-10-06T05:20:57.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.299+0000 7fac3effd700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac2c066f30 0x7fac2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.300+0000 7fac3effd700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac2c066f30 0x7fac2c0693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fac300099b0 tx=0x7fac30017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.303+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac38051950 con 0x7fac40150380 2023-10-06T05:20:57.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.507+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac400af0a0 con 0x7fac40150380 2023-10-06T05:20:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.508+0000 7fac26ffd700 1 -- 172.21.15.33:0/3384309156 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16882 (secure 0 0 0) 0x7fac38054f70 con 0x7fac40150380 2023-10-06T05:20:57.511 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:57.512 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:56.546164+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:54.719267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:55.512486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac2c066f30 msgr2=0x7fac2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac2c066f30 0x7fac2c0693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fac300099b0 tx=0x7fac30017000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 msgr2=0x7fac4014f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 0x7fac4014f5c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cd60 tx=0x7fac38001ae0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 shutdown_connections 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac2c066f30 0x7fac2c0693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac40150380 0x7fac4014f5c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac400cc750 0x7fac4014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 --2- 172.21.15.33:0/3384309156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac400ccfc0 0x7fac400c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 >> 172.21.15.33:0/3384309156 conn(0x7fac4006cff0 msgr2=0x7fac400c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 shutdown_connections 2023-10-06T05:20:57.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.511+0000 7fac3ffff700 1 -- 172.21.15.33:0/3384309156 wait complete. 2023-10-06T05:20:57.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.716+0000 7f9997fff700 1 Processor -- start 2023-10-06T05:20:57.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.716+0000 7f9997fff700 1 -- start start 2023-10-06T05:20:57.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9997fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9998155af0 0x7f999806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9997fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999806ab60 con 0x7f999806a6a0 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9997fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999806ae40 con 0x7f99980c7920 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9997fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999806b7e0 con 0x7f9998155af0 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9996ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9998155af0 0x7f999806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9996ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9998155af0 0x7f999806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41770/0 (socket says 172.21.15.33:41770) 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f9996ffd700 1 -- 172.21.15.33:0/4261793403 learned_addr learned my addr 172.21.15.33:0/4261793403 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.717+0000 7f99967fc700 1 --1- 172.21.15.33:0/4261793403 >> v1:172.21.15.33:6789/0 conn(0x7f999806a6a0 0x7f99980c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54654/0 (socket says 172.21.15.33:54654) 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9996ffd700 1 -- 172.21.15.33:0/4261793403 >> v1:172.21.15.167:6789/0 conn(0x7f99980c7920 legacy=0x7f99980c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9996ffd700 1 -- 172.21.15.33:0/4261793403 >> v1:172.21.15.33:6789/0 conn(0x7f999806a6a0 legacy=0x7f99980c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9996ffd700 1 -- 172.21.15.33:0/4261793403 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999806b120 con 0x7f9998155af0 2023-10-06T05:20:57.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9995ffb700 1 -- 172.21.15.33:0/4261793403 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163407918 0 0) 0x7f999806ab60 con 0x7f999806a6a0 2023-10-06T05:20:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9996ffd700 1 --2- 172.21.15.33:0/4261793403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f999806a160 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f998800bc50 tx=0x7f998800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f653ae94868a433b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9995ffb700 1 -- 172.21.15.33:0/4261793403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998800be60 con 0x7f9998155af0 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9995ffb700 1 -- 172.21.15.33:0/4261793403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998800ddc0 con 0x7f9998155af0 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.718+0000 7f9995ffb700 1 -- 172.21.15.33:0/4261793403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880106a0 con 0x7f9998155af0 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 -- 172.21.15.33:0/4261793403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 msgr2=0x7f999806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 --2- 172.21.15.33:0/4261793403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f999806a160 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f998800bc50 tx=0x7f998800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 -- 172.21.15.33:0/4261793403 shutdown_connections 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 --2- 172.21.15.33:0/4261793403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f999806a160 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 -- 172.21.15.33:0/4261793403 >> 172.21.15.33:0/4261793403 conn(0x7f999806d030 msgr2=0x7f999806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 -- 172.21.15.33:0/4261793403 shutdown_connections 2023-10-06T05:20:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.719+0000 7f9997fff700 1 -- 172.21.15.33:0/4261793403 wait complete. 2023-10-06T05:20:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.720+0000 7f9997fff700 1 Processor -- start 2023-10-06T05:20:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.721+0000 7f9997fff700 1 -- start start 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9997fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999806a6a0 0x7f99980b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9997fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99980c7920 0x7f9998153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9997fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f99980c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9997fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980c4120 con 0x7f9998155af0 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9997fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980aff80 con 0x7f99980c7920 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9997fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99980b0de0 con 0x7f999806a6a0 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f99980c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9996ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999806a6a0 0x7f99980b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99967fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99980c7920 0x7f9998153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9996ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999806a6a0 0x7f99980b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:48062/0 (socket says 172.21.15.33:48062) 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99967fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99980c7920 0x7f9998153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:55724/0 (socket says 172.21.15.33:55724) 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f9996ffd700 1 -- 172.21.15.33:0/2867359301 learned_addr learned my addr 172.21.15.33:0/2867359301 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99977fe700 1 -- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999806a6a0 msgr2=0x7f99980b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99977fe700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999806a6a0 0x7f99980b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99977fe700 1 -- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99980c7920 msgr2=0x7f9998153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99977fe700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99980c7920 0x7f9998153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.722+0000 7f99977fe700 1 -- 172.21.15.33:0/2867359301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998800b790 con 0x7f9998155af0 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.723+0000 7f99977fe700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f99980c5500 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f999000c7e0 tx=0x7f9990002620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.723+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990012630 con 0x7f9998155af0 2023-10-06T05:20:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.723+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99900127b0 con 0x7f9998155af0 2023-10-06T05:20:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.723+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99980b01d0 con 0x7f9998155af0 2023-10-06T05:20:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.723+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990012b70 con 0x7f9998155af0 2023-10-06T05:20:57.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.723+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99980b0a70 con 0x7f9998155af0 2023-10-06T05:20:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.724+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99980b05d0 con 0x7f9998155af0 2023-10-06T05:20:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.725+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f999001a3b0 con 0x7f9998155af0 2023-10-06T05:20:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.726+0000 7f997effd700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9984066f30 0x7f99840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.726+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9990084220 con 0x7f9998155af0 2023-10-06T05:20:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.726+0000 7f9996ffd700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9984066f30 0x7f99840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.727+0000 7f9996ffd700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9984066f30 0x7f99840693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f998800bf90 tx=0x7f998801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.730+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9990051290 con 0x7f9998155af0 2023-10-06T05:20:57.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.934+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99980b0f70 con 0x7f9998155af0 2023-10-06T05:20:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.935+0000 7f997effd700 1 -- 172.21.15.33:0/2867359301 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16882 (secure 0 0 0) 0x7f99900548b0 con 0x7f9998155af0 2023-10-06T05:20:57.938 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:20:57.938 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:56.546164+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:54.719267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:55.512486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.937+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9984066f30 msgr2=0x7f99840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.937+0000 7f9997fff700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9984066f30 0x7f99840693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f998800bf90 tx=0x7f998801a000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.937+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 msgr2=0x7f99980c5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.937+0000 7f9997fff700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f99980c5500 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f999000c7e0 tx=0x7f9990002620 comp rx=0 tx=0).stop 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 shutdown_connections 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9984066f30 0x7f99840693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9998155af0 0x7f99980c5500 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f999806a6a0 0x7f99980b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 --2- 172.21.15.33:0/2867359301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99980c7920 0x7f9998153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 >> 172.21.15.33:0/2867359301 conn(0x7f999806d030 msgr2=0x7f99980c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 shutdown_connections 2023-10-06T05:20:57.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:57.938+0000 7f9997fff700 1 -- 172.21.15.33:0/2867359301 wait complete. 2023-10-06T05:20:57.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:20:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.148+0000 7fd19179d700 1 Processor -- start 2023-10-06T05:20:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.148+0000 7fd19179d700 1 -- start start 2023-10-06T05:20:58.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.149+0000 7fd19179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd18c156370 0x7fd18c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.149+0000 7fd19179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd18c0cc7e0 0x7fd18c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.149+0000 7fd19179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0ad430 con 0x7fd18c0b01f0 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.149+0000 7fd19179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0afe20 con 0x7fd18c156370 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.149+0000 7fd19179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c06bc70 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd18c156370 0x7fd18c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd18c156370 0x7fd18c0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41800/0 (socket says 172.21.15.33:41800) 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 -- 172.21.15.33:0/3144049727 learned_addr learned my addr 172.21.15.33:0/3144049727 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18b7fe700 1 --2- 172.21.15.33:0/3144049727 >> v2:172.21.15.167:3300/0 conn(0x7fd18c0cc7e0 0x7fd18c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 -- 172.21.15.33:0/3144049727 >> v2:172.21.15.167:3300/0 conn(0x7fd18c0cc7e0 msgr2=0x7fd18c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 --2- 172.21.15.33:0/3144049727 >> v2:172.21.15.167:3300/0 conn(0x7fd18c0cc7e0 0x7fd18c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 -- 172.21.15.33:0/3144049727 >> v1:172.21.15.167:6789/0 conn(0x7fd18c0b01f0 legacy=0x7fd18c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.150+0000 7fd18bfff700 1 -- 172.21.15.33:0/3144049727 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0c92f0 con 0x7fd18c156370 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd18bfff700 1 --2- 172.21.15.33:0/3144049727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c156370 0x7fd18c0af8e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ad3a0 tx=0x7fd17c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fb3e04f55e3ebc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd18affd700 1 -- 172.21.15.33:0/3144049727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00b860 con 0x7fd18c156370 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd18affd700 1 -- 172.21.15.33:0/3144049727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c00b9c0 con 0x7fd18c156370 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd18affd700 1 -- 172.21.15.33:0/3144049727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00bd60 con 0x7fd18c156370 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd19179d700 1 -- 172.21.15.33:0/3144049727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c156370 msgr2=0x7fd18c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd19179d700 1 --2- 172.21.15.33:0/3144049727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c156370 0x7fd18c0af8e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ad3a0 tx=0x7fd17c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd19179d700 1 -- 172.21.15.33:0/3144049727 shutdown_connections 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd19179d700 1 --2- 172.21.15.33:0/3144049727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c156370 0x7fd18c0af8e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd19179d700 1 --2- 172.21.15.33:0/3144049727 >> v2:172.21.15.167:3300/0 conn(0x7fd18c0cc7e0 0x7fd18c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.151+0000 7fd19179d700 1 -- 172.21.15.33:0/3144049727 >> 172.21.15.33:0/3144049727 conn(0x7fd18c06d030 msgr2=0x7fd18c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.152+0000 7fd19179d700 1 -- 172.21.15.33:0/3144049727 shutdown_connections 2023-10-06T05:20:58.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.152+0000 7fd19179d700 1 -- 172.21.15.33:0/3144049727 wait complete. 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.153+0000 7fd19179d700 1 Processor -- start 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.153+0000 7fd19179d700 1 -- start start 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd19179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd18c0b01f0 0x7fd18c167a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd19179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 0x7fd18c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd19179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd18c156370 0x7fd18c153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd19179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c1516f0 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd19179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c154410 con 0x7fd18c0b01f0 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd19179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c151940 con 0x7fd18c156370 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd18b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 0x7fd18c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd18b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 0x7fd18c06ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41806/0 (socket says 172.21.15.33:41806) 2023-10-06T05:20:58.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd18b7fe700 1 -- 172.21.15.33:0/3126372439 learned_addr learned my addr 172.21.15.33:0/3126372439 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:58.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd18bfff700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd18c0b01f0 0x7fd18c167a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.154+0000 7fd190f9c700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd18c156370 0x7fd18c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18b7fe700 1 -- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd18c156370 msgr2=0x7fd18c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18b7fe700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd18c156370 0x7fd18c153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18b7fe700 1 -- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd18c0b01f0 msgr2=0x7fd18c167a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18b7fe700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd18c0b01f0 0x7fd18c167a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18b7fe700 1 -- 172.21.15.33:0/3126372439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0094f0 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18bfff700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd18c0b01f0 0x7fd18c167a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:58.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd18b7fe700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 0x7fd18c06ac20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd18000d890 tx=0x7fd180008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800125f0 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180012770 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180012b30 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.155+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c151c20 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.156+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c167fa0 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.157+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c152140 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.158+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd18001e460 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.159+0000 7fd173fff700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd178066f30 0x7fd1780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.159+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd180084cb0 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.160+0000 7fd18bfff700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd178066f30 0x7fd1780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.161+0000 7fd18bfff700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd178066f30 0x7fd1780693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ad3a0 tx=0x7fd17c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.164+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd18c152140 con 0x7fd18c0cc7e0 2023-10-06T05:20:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:58.411+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd18c152140 con 0x7fd18c0cc7e0 2023-10-06T05:20:59.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.075+0000 7fd173fff700 1 -- 172.21.15.33:0/3126372439 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd18c152140 con 0x7fd18c0cc7e0 2023-10-06T05:20:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd178066f30 msgr2=0x7fd1780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd178066f30 0x7fd1780693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ad3a0 tx=0x7fd17c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 msgr2=0x7fd18c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 0x7fd18c06ac20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd18000d890 tx=0x7fd180008040 comp rx=0 tx=0).stop 2023-10-06T05:20:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 shutdown_connections 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd178066f30 0x7fd1780693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd18c156370 0x7fd18c153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd18c0b01f0 0x7fd18c167a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 --2- 172.21.15.33:0/3126372439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18c0cc7e0 0x7fd18c06ac20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 >> 172.21.15.33:0/3126372439 conn(0x7fd18c06d030 msgr2=0x7fd18c150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 shutdown_connections 2023-10-06T05:20:59.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.079+0000 7fd19179d700 1 -- 172.21.15.33:0/3126372439 wait complete. 2023-10-06T05:20:59.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:20:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.297+0000 7febf6b9d700 1 Processor -- start 2023-10-06T05:20:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.297+0000 7febf6b9d700 1 -- start start 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf6b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf0159850 0x7febf006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf6b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7febf015acd0 0x7febf006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf6b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf00b0cd0 con 0x7febf006d140 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf6b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf015a840 con 0x7febf0159850 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf6b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf0159f10 con 0x7febf015acd0 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf0159850 0x7febf006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7febf0159850 0x7febf006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41818/0 (socket says 172.21.15.33:41818) 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 -- 172.21.15.33:0/2119769051 learned_addr learned my addr 172.21.15.33:0/2119769051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf539a700 1 --2- 172.21.15.33:0/2119769051 >> v2:172.21.15.167:3300/0 conn(0x7febf015acd0 0x7febf006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 -- 172.21.15.33:0/2119769051 >> v2:172.21.15.167:3300/0 conn(0x7febf015acd0 msgr2=0x7febf006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 --2- 172.21.15.33:0/2119769051 >> v2:172.21.15.167:3300/0 conn(0x7febf015acd0 0x7febf006edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 -- 172.21.15.33:0/2119769051 >> v1:172.21.15.167:6789/0 conn(0x7febf006d140 legacy=0x7febf006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf5b9b700 1 -- 172.21.15.33:0/2119769051 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf015a1f0 con 0x7febf0159850 2023-10-06T05:20:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.299+0000 7febf539a700 1 --2- 172.21.15.33:0/2119769051 >> v2:172.21.15.167:3300/0 conn(0x7febf015acd0 0x7febf006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf5b9b700 1 --2- 172.21.15.33:0/2119769051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf0159850 0x7febf006e890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7febf006a730 tx=0x7febec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a68f971da070a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf4b99700 1 -- 172.21.15.33:0/2119769051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00b820 con 0x7febf0159850 2023-10-06T05:20:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf4b99700 1 -- 172.21.15.33:0/2119769051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00b980 con 0x7febf0159850 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf4b99700 1 -- 172.21.15.33:0/2119769051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00bd20 con 0x7febf0159850 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf6b9d700 1 -- 172.21.15.33:0/2119769051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf0159850 msgr2=0x7febf006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2119769051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf0159850 0x7febf006e890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7febf006a730 tx=0x7febec009250 comp rx=0 tx=0).stop 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf6b9d700 1 -- 172.21.15.33:0/2119769051 shutdown_connections 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2119769051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf0159850 0x7febf006e890 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2119769051 >> v2:172.21.15.167:3300/0 conn(0x7febf015acd0 0x7febf006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.300+0000 7febf6b9d700 1 -- 172.21.15.33:0/2119769051 >> 172.21.15.33:0/2119769051 conn(0x7febf006cd40 msgr2=0x7febf00ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.301+0000 7febf6b9d700 1 -- 172.21.15.33:0/2119769051 shutdown_connections 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.301+0000 7febf6b9d700 1 -- 172.21.15.33:0/2119769051 wait complete. 2023-10-06T05:20:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.302+0000 7febf6b9d700 1 Processor -- start 2023-10-06T05:20:59.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.302+0000 7febf6b9d700 1 -- start start 2023-10-06T05:20:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf006d140 0x7febf0157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf6b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf0159850 0x7febf015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf6b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 0x7febf00cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf6b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf015f8c0 con 0x7febf015acd0 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf6b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf015ef90 con 0x7febf0159850 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf015f270 con 0x7febf006d140 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 0x7febf00cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 0x7febf00cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41828/0 (socket says 172.21.15.33:41828) 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.303+0000 7febf639c700 1 -- 172.21.15.33:0/2681571802 learned_addr learned my addr 172.21.15.33:0/2681571802 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf539a700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf0159850 0x7febf015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf5b9b700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf006d140 0x7febf0157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf639c700 1 -- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf006d140 msgr2=0x7febf0157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf639c700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf006d140 0x7febf0157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf639c700 1 -- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf0159850 msgr2=0x7febf015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf639c700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf0159850 0x7febf015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf639c700 1 -- 172.21.15.33:0/2681571802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0094f0 con 0x7febf015acd0 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf639c700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 0x7febf00cad10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7febe800bb60 tx=0x7febe80097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800c7e0 con 0x7febf015acd0 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800c960 con 0x7febf015acd0 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800cd20 con 0x7febf015acd0 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.304+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf015f550 con 0x7febf015acd0 2023-10-06T05:20:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.305+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf0160290 con 0x7febf015acd0 2023-10-06T05:20:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.305+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf015fa30 con 0x7febf015acd0 2023-10-06T05:20:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.306+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febe8014620 con 0x7febf015acd0 2023-10-06T05:20:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.307+0000 7febe5ffb700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd4066f30 0x7febd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.307+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7febe80844d0 con 0x7febf015acd0 2023-10-06T05:20:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.308+0000 7febf5b9b700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd4066f30 0x7febd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.308+0000 7febf5b9b700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd4066f30 0x7febd40693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7febf006a730 tx=0x7febec016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.311+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febe804c5a0 con 0x7febf015acd0 2023-10-06T05:20:59.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:20:59.560+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7febf004c380 con 0x7febf015acd0 2023-10-06T05:21:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.082+0000 7febe5ffb700 1 -- 172.21.15.33:0/2681571802 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7febe80515e0 con 0x7febf015acd0 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd4066f30 msgr2=0x7febd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd4066f30 0x7febd40693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7febf006a730 tx=0x7febec016000 comp rx=0 tx=0).stop 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 msgr2=0x7febf00cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 0x7febf00cad10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7febe800bb60 tx=0x7febe80097f0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 shutdown_connections 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7febd4066f30 0x7febd40693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febf015acd0 0x7febf00cad10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7febf006d140 0x7febf0157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 --2- 172.21.15.33:0/2681571802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febf0159850 0x7febf015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 >> 172.21.15.33:0/2681571802 conn(0x7febf006cd40 msgr2=0x7febf0157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 shutdown_connections 2023-10-06T05:21:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.085+0000 7febf6b9d700 1 -- 172.21.15.33:0/2681571802 wait complete. 2023-10-06T05:21:00.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.296+0000 7f2cbb59e700 1 Processor -- start 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.296+0000 7f2cbb59e700 1 -- start start 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2cbc13faa0 0x7f2cbc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2cbc06aad0 0x7f2cbc06af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc0c9cb0 con 0x7f2cbc0c7940 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ca270 con 0x7f2cbc06aad0 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc06b840 con 0x7f2cbc13faa0 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2cbc06aad0 0x7f2cbc06af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2cbc06aad0 0x7f2cbc06af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43914/0 (socket says 172.21.15.33:43914) 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.297+0000 7f2cbad9d700 1 -- 172.21.15.33:0/2321227819 learned_addr learned my addr 172.21.15.33:0/2321227819 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cba59c700 1 --2- 172.21.15.33:0/2321227819 >> v2:172.21.15.167:3300/0 conn(0x7f2cbc13faa0 0x7f2cbc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cbad9d700 1 -- 172.21.15.33:0/2321227819 >> v2:172.21.15.167:3300/0 conn(0x7f2cbc13faa0 msgr2=0x7f2cbc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cbad9d700 1 --2- 172.21.15.33:0/2321227819 >> v2:172.21.15.167:3300/0 conn(0x7f2cbc13faa0 0x7f2cbc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cbad9d700 1 -- 172.21.15.33:0/2321227819 >> v1:172.21.15.167:6789/0 conn(0x7f2cbc0c7940 legacy=0x7f2cbc0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cbad9d700 1 -- 172.21.15.33:0/2321227819 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc06bb20 con 0x7f2cbc06aad0 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cb959a700 1 -- 172.21.15.33:0/2321227819 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465817151 0 0) 0x7f2cbc0c9cb0 con 0x7f2cbc0c7940 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cbad9d700 1 --2- 172.21.15.33:0/2321227819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc06af30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06fe50 tx=0x7f2cb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fe60ecdb22083e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.298+0000 7f2cb959a700 1 -- 172.21.15.33:0/2321227819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400e070 con 0x7f2cbc06aad0 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cb959a700 1 -- 172.21.15.33:0/2321227819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb400ba20 con 0x7f2cbc06aad0 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cb959a700 1 -- 172.21.15.33:0/2321227819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40092e0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 -- 172.21.15.33:0/2321227819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 msgr2=0x7f2cbc06af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/2321227819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc06af30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06fe50 tx=0x7f2cb4009840 comp rx=0 tx=0).stop 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 -- 172.21.15.33:0/2321227819 shutdown_connections 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/2321227819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc06af30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/2321227819 >> v2:172.21.15.167:3300/0 conn(0x7f2cbc13faa0 0x7f2cbc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 -- 172.21.15.33:0/2321227819 >> 172.21.15.33:0/2321227819 conn(0x7f2cbc06d030 msgr2=0x7f2cbc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 -- 172.21.15.33:0/2321227819 shutdown_connections 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.299+0000 7f2cbb59e700 1 -- 172.21.15.33:0/2321227819 wait complete. 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.301+0000 7f2cbb59e700 1 Processor -- start 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.301+0000 7f2cbb59e700 1 -- start start 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cbc0c7940 0x7f2cbc0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2cbc13faa0 0x7f2cbc13e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc16a0f0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c63b0 con 0x7f2cbc0c7940 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0af0e0 con 0x7f2cbc13faa0 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc169bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43922/0 (socket says 172.21.15.33:43922) 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 -- 172.21.15.33:0/1662865346 learned_addr learned my addr 172.21.15.33:0/1662865346 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cb9d9b700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cbc0c7940 0x7f2cbc0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbad9d700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2cbc13faa0 0x7f2cbc13e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 -- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2cbc13faa0 msgr2=0x7f2cbc13e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2cbc13faa0 0x7f2cbc13e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 -- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cbc0c7940 msgr2=0x7f2cbc0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cbc0c7940 0x7f2cbc0cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cba59c700 1 -- 172.21.15.33:0/1662865346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009ae0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cbad9d700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2cbc13faa0 0x7f2cbc13e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.302+0000 7f2cb9d9b700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cbc0c7940 0x7f2cbc0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.303+0000 7f2cba59c700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc169bb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e930 tx=0x7f2cac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.303+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00eed0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.303+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00c850 con 0x7f2cbc06aad0 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.303+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc0c5df0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.303+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00cbf0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.303+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc0c6180 con 0x7f2cbc06aad0 2023-10-06T05:21:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.304+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc0aebe0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.305+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cac013470 con 0x7f2cbc06aad0 2023-10-06T05:21:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.306+0000 7f2caa7fc700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c9c066f30 0x7f2c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.306+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2cac084910 con 0x7f2cbc06aad0 2023-10-06T05:21:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.306+0000 7f2cb9d9b700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c9c066f30 0x7f2c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.307+0000 7f2cb9d9b700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c9c066f30 0x7f2c9c0693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00011e0 tx=0x7f2cb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.310+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cac051980 con 0x7f2cbc06aad0 2023-10-06T05:21:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.557+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cbc0c6520 con 0x7f2cbc06aad0 2023-10-06T05:21:00.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.557+0000 7f2caa7fc700 1 -- 172.21.15.33:0/1662865346 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f2cac054fa0 con 0x7f2cbc06aad0 2023-10-06T05:21:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:00.560 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:00.075535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6681":{"gid":6681,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2184356465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2184356465},{"type":"v1","addr":"172.21.15.33:6835","nonce":2184356465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c9c066f30 msgr2=0x7f2c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c9c066f30 0x7f2c9c0693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00011e0 tx=0x7f2cb0009040 comp rx=0 tx=0).stop 2023-10-06T05:21:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 msgr2=0x7f2cbc169bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc169bb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e930 tx=0x7f2cac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 shutdown_connections 2023-10-06T05:21:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2c9c066f30 0x7f2c9c0693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2cbc13faa0 0x7f2cbc13e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cbc06aad0 0x7f2cbc169bb0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 --2- 172.21.15.33:0/1662865346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cbc0c7940 0x7f2cbc0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 >> 172.21.15.33:0/1662865346 conn(0x7f2cbc06d030 msgr2=0x7f2cbc0cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 shutdown_connections 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.560+0000 7f2cbb59e700 1 -- 172.21.15.33:0/1662865346 wait complete. 2023-10-06T05:21:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2023-10-06T05:21:00.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.765+0000 7f8b56d9d700 1 Processor -- start 2023-10-06T05:21:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.765+0000 7f8b56d9d700 1 -- start start 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b56d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b56d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b5006aad0 0x7f8b5006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b56d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500c9cb0 con 0x7f8b5013faa0 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b56d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500ca270 con 0x7f8b5006aad0 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b56d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5006b840 con 0x7f8b500c8050 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b5659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b5006aad0 0x7f8b5006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b5659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b5006aad0 0x7f8b5006af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43926/0 (socket says 172.21.15.33:43926) 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b5659c700 1 -- 172.21.15.33:0/2158873640 learned_addr learned my addr 172.21.15.33:0/2158873640 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b55d9b700 1 --1- 172.21.15.33:0/2158873640 >> v1:172.21.15.167:6789/0 conn(0x7f8b5013faa0 0x7f8b500c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33388/0 (socket says 172.21.15.33:33388) 2023-10-06T05:21:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.766+0000 7f8b5559a700 1 --2- 172.21.15.33:0/2158873640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b5659c700 1 -- 172.21.15.33:0/2158873640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 msgr2=0x7f8b5013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b5659c700 1 --2- 172.21.15.33:0/2158873640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b5659c700 1 -- 172.21.15.33:0/2158873640 >> v1:172.21.15.167:6789/0 conn(0x7f8b5013faa0 legacy=0x7f8b500c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b5659c700 1 -- 172.21.15.33:0/2158873640 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5006bb20 con 0x7f8b5006aad0 2023-10-06T05:21:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b54d99700 1 -- 172.21.15.33:0/2158873640 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312237835 0 0) 0x7f8b500c9cb0 con 0x7f8b5013faa0 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b5659c700 1 --2- 172.21.15.33:0/2158873640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b5006af50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009f20 tx=0x7f8b48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfba69666fffa0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b54d99700 1 -- 172.21.15.33:0/2158873640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800e070 con 0x7f8b5006aad0 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b54d99700 1 -- 172.21.15.33:0/2158873640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4800b9d0 con 0x7f8b5006aad0 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.767+0000 7f8b54d99700 1 -- 172.21.15.33:0/2158873640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b480092e0 con 0x7f8b5006aad0 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2158873640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 msgr2=0x7f8b5006af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2158873640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b5006af50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009f20 tx=0x7f8b48009840 comp rx=0 tx=0).stop 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2158873640 shutdown_connections 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2158873640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b5006af50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2158873640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2158873640 >> 172.21.15.33:0/2158873640 conn(0x7f8b5006d030 msgr2=0x7f8b5006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2158873640 shutdown_connections 2023-10-06T05:21:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.768+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2158873640 wait complete. 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.769+0000 7f8b56d9d700 1 Processor -- start 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 -- start start 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b50135ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5013faa0 0x7f8b5013dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5013efe0 con 0x7f8b5006aad0 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5013e3d0 con 0x7f8b5013faa0 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.770+0000 7f8b56d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5013e6b0 con 0x7f8b500c8050 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b50135ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b50135ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43928/0 (socket says 172.21.15.33:43928) 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 -- 172.21.15.33:0/2139933836 learned_addr learned my addr 172.21.15.33:0/2139933836 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b5659c700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5013faa0 0x7f8b5013dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b5559a700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 -- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 msgr2=0x7f8b5013d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 -- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5013faa0 msgr2=0x7f8b5013dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5013faa0 0x7f8b5013dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.771+0000 7f8b55d9b700 1 -- 172.21.15.33:0/2139933836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48009ae0 con 0x7f8b5006aad0 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.772+0000 7f8b55d9b700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b50135ba0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c5b0 tx=0x7f8b4c001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.772+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c005c10 con 0x7f8b5006aad0 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.772+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c005d70 con 0x7f8b5006aad0 2023-10-06T05:21:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.772+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c007300 con 0x7f8b5006aad0 2023-10-06T05:21:00.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.772+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5013e990 con 0x7f8b5006aad0 2023-10-06T05:21:00.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.772+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5013e0e0 con 0x7f8b5006aad0 2023-10-06T05:21:00.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.773+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b501475d0 con 0x7f8b5006aad0 2023-10-06T05:21:00.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.774+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b4c01b3b0 con 0x7f8b5006aad0 2023-10-06T05:21:00.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.775+0000 7f8b45ffb700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b34066f30 0x7f8b340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.775+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b4c083d30 con 0x7f8b5006aad0 2023-10-06T05:21:00.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.775+0000 7f8b5559a700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b34066f30 0x7f8b340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.776+0000 7f8b5559a700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b34066f30 0x7f8b340693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8b50156b90 tx=0x7f8b400091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.778+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b4c050e40 con 0x7f8b5006aad0 2023-10-06T05:21:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.982+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b5006c180 con 0x7f8b5006aad0 2023-10-06T05:21:00.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.983+0000 7f8b45ffb700 1 -- 172.21.15.33:0/2139933836 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f8b4c017070 con 0x7f8b5006aad0 2023-10-06T05:21:00.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:00.986 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:20:58.056905+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:54.719267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:55.512486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:00.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b34066f30 msgr2=0x7f8b340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b34066f30 0x7f8b340693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8b50156b90 tx=0x7f8b400091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 msgr2=0x7f8b50135ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b50135ba0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c5b0 tx=0x7f8b4c001040 comp rx=0 tx=0).stop 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 shutdown_connections 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b34066f30 0x7f8b340693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b5013faa0 0x7f8b5013dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b5006aad0 0x7f8b50135ba0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 --2- 172.21.15.33:0/2139933836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b500c8050 0x7f8b5013d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 >> 172.21.15.33:0/2139933836 conn(0x7f8b5006d030 msgr2=0x7f8b500c1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 shutdown_connections 2023-10-06T05:21:00.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:00.985+0000 7f8b56d9d700 1 -- 172.21.15.33:0/2139933836 wait complete. 2023-10-06T05:21:01.006 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:21:01.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.193+0000 7f5fcb7ff700 1 Processor -- start 2023-10-06T05:21:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.193+0000 7f5fcb7ff700 1 -- start start 2023-10-06T05:21:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fcb7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5fc4159630 0x7f5fc40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fcb7ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5fc40cea10 0x7f5fc40b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fcb7ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc40cda00 con 0x7f5fc40cd590 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fcb7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40cee90 con 0x7f5fc4159630 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fcb7ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40cfc80 con 0x7f5fc40cea10 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fc959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5fc4159630 0x7f5fc40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fc959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5fc4159630 0x7f5fc40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43942/0 (socket says 172.21.15.33:43942) 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.194+0000 7f5fc959b700 1 -- 172.21.15.33:0/2087622273 learned_addr learned my addr 172.21.15.33:0/2087622273 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fc959b700 1 -- 172.21.15.33:0/2087622273 >> v2:172.21.15.167:3300/0 conn(0x7f5fc40cea10 msgr2=0x7f5fc40b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fc959b700 1 --2- 172.21.15.33:0/2087622273 >> v2:172.21.15.167:3300/0 conn(0x7f5fc40cea10 0x7f5fc40b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fc959b700 1 -- 172.21.15.33:0/2087622273 >> v1:172.21.15.167:6789/0 conn(0x7f5fc40cd590 legacy=0x7f5fc40ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fc959b700 1 -- 172.21.15.33:0/2087622273 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40cf070 con 0x7f5fc4159630 2023-10-06T05:21:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fc959b700 1 --2- 172.21.15.33:0/2087622273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc40cd050 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40b00a0 tx=0x7f5fc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c775f293048d28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fbbfff700 1 -- 172.21.15.33:0/2087622273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000b820 con 0x7f5fc4159630 2023-10-06T05:21:01.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fbbfff700 1 -- 172.21.15.33:0/2087622273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000b980 con 0x7f5fc4159630 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.195+0000 7f5fbbfff700 1 -- 172.21.15.33:0/2087622273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000bd20 con 0x7f5fc4159630 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/2087622273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 msgr2=0x7f5fc40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/2087622273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc40cd050 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40b00a0 tx=0x7f5fc0009250 comp rx=0 tx=0).stop 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/2087622273 shutdown_connections 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/2087622273 >> v2:172.21.15.167:3300/0 conn(0x7f5fc40cea10 0x7f5fc40b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/2087622273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc40cd050 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/2087622273 >> 172.21.15.33:0/2087622273 conn(0x7f5fc406cff0 msgr2=0x7f5fc406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/2087622273 shutdown_connections 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.196+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/2087622273 wait complete. 2023-10-06T05:21:01.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.197+0000 7f5fcb7ff700 1 Processor -- start 2023-10-06T05:21:01.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.198+0000 7f5fcb7ff700 1 -- start start 2023-10-06T05:21:01.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.198+0000 7f5fcb7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc40cd590 0x7f5fc416aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.198+0000 7f5fcb7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc40cea10 0x7f5fc415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.198+0000 7f5fcb7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc4151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fcb7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc414f090 con 0x7f5fc4159630 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fcb7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41559a0 con 0x7f5fc40cea10 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fcb7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc4156240 con 0x7f5fc40cd590 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc4151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc4151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43950/0 (socket says 172.21.15.33:43950) 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 -- 172.21.15.33:0/3507309174 learned_addr learned my addr 172.21.15.33:0/3507309174 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc959b700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc40cd590 0x7f5fc416aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 -- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc40cd590 msgr2=0x7f5fc416aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc8d9a700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc40cea10 0x7f5fc415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc40cd590 0x7f5fc416aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 -- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc40cea10 msgr2=0x7f5fc415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc40cea10 0x7f5fc415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc9d9c700 1 -- 172.21.15.33:0/3507309174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00094f0 con 0x7f5fc4159630 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc959b700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc40cd590 0x7f5fc416aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.199+0000 7f5fc8d9a700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc40cea10 0x7f5fc415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.200+0000 7f5fc9d9c700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc4151760 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00bc60 tx=0x7f5fbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.200+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0092e0 con 0x7f5fc4159630 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.200+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc00c7b0 con 0x7f5fc4159630 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.200+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc415ebe0 con 0x7f5fc4159630 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.200+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00cb90 con 0x7f5fc4159630 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.200+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc4155bf0 con 0x7f5fc4159630 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.201+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc41563d0 con 0x7f5fc4159630 2023-10-06T05:21:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.202+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fbc012070 con 0x7f5fc4159630 2023-10-06T05:21:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.203+0000 7f5fb97fa700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5fa8066f30 0x7f5fa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.203+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5fbc0843b0 con 0x7f5fc4159630 2023-10-06T05:21:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.203+0000 7f5fc959b700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5fa8066f30 0x7f5fa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.204+0000 7f5fc959b700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5fa8066f30 0x7f5fa80693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40b00a0 tx=0x7f5fc0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.207+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fbc051420 con 0x7f5fc4159630 2023-10-06T05:21:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.406+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fc4064060 con 0x7f5fa8066f30 2023-10-06T05:21:01.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.417+0000 7f5fb97fa700 1 -- 172.21.15.33:0/3507309174 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307738 (secure 0 0 0) 0x7f5fc4064060 con 0x7f5fa8066f30 2023-10-06T05:21:01.424 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5fa8066f30 msgr2=0x7f5fa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5fa8066f30 0x7f5fa80693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40b00a0 tx=0x7f5fc0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 msgr2=0x7f5fc4151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc4151760 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00bc60 tx=0x7f5fbc009840 comp rx=0 tx=0).stop 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 shutdown_connections 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5fa8066f30 0x7f5fa80693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fc4159630 0x7f5fc4151760 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fc40cd590 0x7f5fc416aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 --2- 172.21.15.33:0/3507309174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc40cea10 0x7f5fc415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 >> 172.21.15.33:0/3507309174 conn(0x7f5fc406cff0 msgr2=0x7f5fc40b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 shutdown_connections 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.422+0000 7f5fcb7ff700 1 -- 172.21.15.33:0/3507309174 wait complete. 2023-10-06T05:21:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:21:01.443 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-10-06T05:20:59.913056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420584,"kb_used_data":9512,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752554136,"statfs":{"total":773094113280,"available":770615435264,"internally_reserved":0,"allocated":9740288,"data_stored":5025216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.874634"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047746+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054711+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047774+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061615+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059581+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059292+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:03.594147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048133+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:39.064497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052728+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052245+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:35.106422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053309+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.455963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051588+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:37.245927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.060583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:07.387943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056543+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.063672+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058649+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:58.648941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059036+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:10.851643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048750+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054442+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059967+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:18.831632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058550+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056058+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054085+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:24.396610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047445+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055287+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:59.686854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:04.727723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051706+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:10.346856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059449+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:08.100764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047908+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056488+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:17.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057141+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:34.761680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056494+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:42.458155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069083+0000","last_change":"2023-10-06T05:20:56.069083+0000","last_active":"2023-10-06T05:20:56.069083+0000","last_peered":"2023-10-06T05:20:56.069083+0000","last_clean":"2023-10-06T05:20:56.069083+0000","last_became_active":"2023-10-06T05:20:56.068775+0000","last_became_peered":"2023-10-06T05:20:56.068775+0000","last_unstale":"2023-10-06T05:20:56.069083+0000","last_undegraded":"2023-10-06T05:20:56.069083+0000","last_fullsized":"2023-10-06T05:20:56.069083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:43.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056509+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:23.982186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.061418+0000","last_change":"2023-10-06T05:20:56.061418+0000","last_active":"2023-10-06T05:20:56.061418+0000","last_peered":"2023-10-06T05:20:56.061418+0000","last_clean":"2023-10-06T05:20:56.061418+0000","last_became_active":"2023-10-06T05:20:56.060531+0000","last_became_peered":"2023-10-06T05:20:56.060531+0000","last_unstale":"2023-10-06T05:20:56.061418+0000","last_undegraded":"2023-10-06T05:20:56.061418+0000","last_fullsized":"2023-10-06T05:20:56.061418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:47.802684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048418+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063062+0000","last_change":"2023-10-06T05:20:56.063062+0000","last_active":"2023-10-06T05:20:56.063062+0000","last_peered":"2023-10-06T05:20:56.063062+0000","last_clean":"2023-10-06T05:20:56.063062+0000","last_became_active":"2023-10-06T05:20:56.062410+0000","last_became_peered":"2023-10-06T05:20:56.062410+0000","last_unstale":"2023-10-06T05:20:56.063062+0000","last_undegraded":"2023-10-06T05:20:56.063062+0000","last_fullsized":"2023-10-06T05:20:56.063062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:49.619842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057087+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:37.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062842+0000","last_change":"2023-10-06T05:20:56.062842+0000","last_active":"2023-10-06T05:20:56.062842+0000","last_peered":"2023-10-06T05:20:56.062842+0000","last_clean":"2023-10-06T05:20:56.062842+0000","last_became_active":"2023-10-06T05:20:56.062590+0000","last_became_peered":"2023-10-06T05:20:56.062590+0000","last_unstale":"2023-10-06T05:20:56.062842+0000","last_undegraded":"2023-10-06T05:20:56.062842+0000","last_fullsized":"2023-10-06T05:20:56.062842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:58.961459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057855+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.845343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063129+0000","last_change":"2023-10-06T05:20:56.063129+0000","last_active":"2023-10-06T05:20:56.063129+0000","last_peered":"2023-10-06T05:20:56.063129+0000","last_clean":"2023-10-06T05:20:56.063129+0000","last_became_active":"2023-10-06T05:20:56.062230+0000","last_became_peered":"2023-10-06T05:20:56.062230+0000","last_unstale":"2023-10-06T05:20:56.063129+0000","last_undegraded":"2023-10-06T05:20:56.063129+0000","last_fullsized":"2023-10-06T05:20:56.063129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:58.928023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066379+0000","last_change":"2023-10-06T05:20:56.066379+0000","last_active":"2023-10-06T05:20:56.066379+0000","last_peered":"2023-10-06T05:20:56.066379+0000","last_clean":"2023-10-06T05:20:56.066379+0000","last_became_active":"2023-10-06T05:20:56.064912+0000","last_became_peered":"2023-10-06T05:20:56.064912+0000","last_unstale":"2023-10-06T05:20:56.066379+0000","last_undegraded":"2023-10-06T05:20:56.066379+0000","last_fullsized":"2023-10-06T05:20:56.066379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:23.256874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058738+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:27.536202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063667+0000","last_change":"2023-10-06T05:20:56.063667+0000","last_active":"2023-10-06T05:20:56.063667+0000","last_peered":"2023-10-06T05:20:56.063667+0000","last_clean":"2023-10-06T05:20:56.063667+0000","last_became_active":"2023-10-06T05:20:56.063346+0000","last_became_peered":"2023-10-06T05:20:56.063346+0000","last_unstale":"2023-10-06T05:20:56.063667+0000","last_undegraded":"2023-10-06T05:20:56.063667+0000","last_fullsized":"2023-10-06T05:20:56.063667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058683+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:36.533849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.068185+0000","last_change":"2023-10-06T05:20:56.068185+0000","last_active":"2023-10-06T05:20:56.068185+0000","last_peered":"2023-10-06T05:20:56.068185+0000","last_clean":"2023-10-06T05:20:56.068185+0000","last_became_active":"2023-10-06T05:20:56.067889+0000","last_became_peered":"2023-10-06T05:20:56.067889+0000","last_unstale":"2023-10-06T05:20:56.068185+0000","last_undegraded":"2023-10-06T05:20:56.068185+0000","last_fullsized":"2023-10-06T05:20:56.068185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052435+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064663+0000","last_change":"2023-10-06T05:20:56.064663+0000","last_active":"2023-10-06T05:20:56.064663+0000","last_peered":"2023-10-06T05:20:56.064663+0000","last_clean":"2023-10-06T05:20:56.064663+0000","last_became_active":"2023-10-06T05:20:56.064382+0000","last_became_peered":"2023-10-06T05:20:56.064382+0000","last_unstale":"2023-10-06T05:20:56.064663+0000","last_undegraded":"2023-10-06T05:20:56.064663+0000","last_fullsized":"2023-10-06T05:20:56.064663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:07.091803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047994+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062871+0000","last_change":"2023-10-06T05:20:56.062871+0000","last_active":"2023-10-06T05:20:56.062871+0000","last_peered":"2023-10-06T05:20:56.062871+0000","last_clean":"2023-10-06T05:20:56.062871+0000","last_became_active":"2023-10-06T05:20:56.062606+0000","last_became_peered":"2023-10-06T05:20:56.062606+0000","last_unstale":"2023-10-06T05:20:56.062871+0000","last_undegraded":"2023-10-06T05:20:56.062871+0000","last_fullsized":"2023-10-06T05:20:56.062871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048278+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069433+0000","last_change":"2023-10-06T05:20:56.069433+0000","last_active":"2023-10-06T05:20:56.069433+0000","last_peered":"2023-10-06T05:20:56.069433+0000","last_clean":"2023-10-06T05:20:56.069433+0000","last_became_active":"2023-10-06T05:20:56.069152+0000","last_became_peered":"2023-10-06T05:20:56.069152+0000","last_unstale":"2023-10-06T05:20:56.069433+0000","last_undegraded":"2023-10-06T05:20:56.069433+0000","last_fullsized":"2023-10-06T05:20:56.069433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.728922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057259+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:12.235739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066398+0000","last_change":"2023-10-06T05:20:56.066398+0000","last_active":"2023-10-06T05:20:56.066398+0000","last_peered":"2023-10-06T05:20:56.066398+0000","last_clean":"2023-10-06T05:20:56.066398+0000","last_became_active":"2023-10-06T05:20:56.066051+0000","last_became_peered":"2023-10-06T05:20:56.066051+0000","last_unstale":"2023-10-06T05:20:56.066398+0000","last_undegraded":"2023-10-06T05:20:56.066398+0000","last_fullsized":"2023-10-06T05:20:56.066398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:26.538710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056633+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:48.550385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060068+0000","last_change":"2023-10-06T05:20:56.060068+0000","last_active":"2023-10-06T05:20:56.060068+0000","last_peered":"2023-10-06T05:20:56.060068+0000","last_clean":"2023-10-06T05:20:56.060068+0000","last_became_active":"2023-10-06T05:20:56.059765+0000","last_became_peered":"2023-10-06T05:20:56.059765+0000","last_unstale":"2023-10-06T05:20:56.060068+0000","last_undegraded":"2023-10-06T05:20:56.060068+0000","last_fullsized":"2023-10-06T05:20:56.060068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:56.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052031+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062607+0000","last_change":"2023-10-06T05:20:56.062607+0000","last_active":"2023-10-06T05:20:56.062607+0000","last_peered":"2023-10-06T05:20:56.062607+0000","last_clean":"2023-10-06T05:20:56.062607+0000","last_became_active":"2023-10-06T05:20:56.062344+0000","last_became_peered":"2023-10-06T05:20:56.062344+0000","last_unstale":"2023-10-06T05:20:56.062607+0000","last_undegraded":"2023-10-06T05:20:56.062607+0000","last_fullsized":"2023-10-06T05:20:56.062607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.897142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051968+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:14.557678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067504+0000","last_change":"2023-10-06T05:20:56.067504+0000","last_active":"2023-10-06T05:20:56.067504+0000","last_peered":"2023-10-06T05:20:56.067504+0000","last_clean":"2023-10-06T05:20:56.067504+0000","last_became_active":"2023-10-06T05:20:56.066985+0000","last_became_peered":"2023-10-06T05:20:56.066985+0000","last_unstale":"2023-10-06T05:20:56.067504+0000","last_undegraded":"2023-10-06T05:20:56.067504+0000","last_fullsized":"2023-10-06T05:20:56.067504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:43.665749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051195+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:39.400959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063906+0000","last_change":"2023-10-06T05:20:56.063906+0000","last_active":"2023-10-06T05:20:56.063906+0000","last_peered":"2023-10-06T05:20:56.063906+0000","last_clean":"2023-10-06T05:20:56.063906+0000","last_became_active":"2023-10-06T05:20:56.063625+0000","last_became_peered":"2023-10-06T05:20:56.063625+0000","last_unstale":"2023-10-06T05:20:56.063906+0000","last_undegraded":"2023-10-06T05:20:56.063906+0000","last_fullsized":"2023-10-06T05:20:56.063906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:07.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054747+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:55.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060664+0000","last_change":"2023-10-06T05:20:56.060664+0000","last_active":"2023-10-06T05:20:56.060664+0000","last_peered":"2023-10-06T05:20:56.060664+0000","last_clean":"2023-10-06T05:20:56.060664+0000","last_became_active":"2023-10-06T05:20:56.059179+0000","last_became_peered":"2023-10-06T05:20:56.059179+0000","last_unstale":"2023-10-06T05:20:56.060664+0000","last_undegraded":"2023-10-06T05:20:56.060664+0000","last_fullsized":"2023-10-06T05:20:56.060664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.078805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050160+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069531+0000","last_change":"2023-10-06T05:20:56.069531+0000","last_active":"2023-10-06T05:20:56.069531+0000","last_peered":"2023-10-06T05:20:56.069531+0000","last_clean":"2023-10-06T05:20:56.069531+0000","last_became_active":"2023-10-06T05:20:56.068428+0000","last_became_peered":"2023-10-06T05:20:56.068428+0000","last_unstale":"2023-10-06T05:20:56.069531+0000","last_undegraded":"2023-10-06T05:20:56.069531+0000","last_fullsized":"2023-10-06T05:20:56.069531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050790+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069310+0000","last_change":"2023-10-06T05:20:56.069310+0000","last_active":"2023-10-06T05:20:56.069310+0000","last_peered":"2023-10-06T05:20:56.069310+0000","last_clean":"2023-10-06T05:20:56.069310+0000","last_became_active":"2023-10-06T05:20:56.069031+0000","last_became_peered":"2023-10-06T05:20:56.069031+0000","last_unstale":"2023-10-06T05:20:56.069310+0000","last_undegraded":"2023-10-06T05:20:56.069310+0000","last_fullsized":"2023-10-06T05:20:56.069310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:58.677893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054813+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:44.878348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066147+0000","last_change":"2023-10-06T05:20:56.066147+0000","last_active":"2023-10-06T05:20:56.066147+0000","last_peered":"2023-10-06T05:20:56.066147+0000","last_clean":"2023-10-06T05:20:56.066147+0000","last_became_active":"2023-10-06T05:20:56.065618+0000","last_became_peered":"2023-10-06T05:20:56.065618+0000","last_unstale":"2023-10-06T05:20:56.066147+0000","last_undegraded":"2023-10-06T05:20:56.066147+0000","last_fullsized":"2023-10-06T05:20:56.066147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:33.934156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051253+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:59.973383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062131+0000","last_change":"2023-10-06T05:20:56.062131+0000","last_active":"2023-10-06T05:20:56.062131+0000","last_peered":"2023-10-06T05:20:56.062131+0000","last_clean":"2023-10-06T05:20:56.062131+0000","last_became_active":"2023-10-06T05:20:56.061821+0000","last_became_peered":"2023-10-06T05:20:56.061821+0000","last_unstale":"2023-10-06T05:20:56.062131+0000","last_undegraded":"2023-10-06T05:20:56.062131+0000","last_fullsized":"2023-10-06T05:20:56.062131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:33.997234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047924+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067790+0000","last_change":"2023-10-06T05:20:56.067790+0000","last_active":"2023-10-06T05:20:56.067790+0000","last_peered":"2023-10-06T05:20:56.067790+0000","last_clean":"2023-10-06T05:20:56.067790+0000","last_became_active":"2023-10-06T05:20:56.067491+0000","last_became_peered":"2023-10-06T05:20:56.067491+0000","last_unstale":"2023-10-06T05:20:56.067790+0000","last_undegraded":"2023-10-06T05:20:56.067790+0000","last_fullsized":"2023-10-06T05:20:56.067790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:06.378982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058270+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:05.605433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.065948+0000","last_change":"2023-10-06T05:20:56.065948+0000","last_active":"2023-10-06T05:20:56.065948+0000","last_peered":"2023-10-06T05:20:56.065948+0000","last_clean":"2023-10-06T05:20:56.065948+0000","last_became_active":"2023-10-06T05:20:56.065640+0000","last_became_peered":"2023-10-06T05:20:56.065640+0000","last_unstale":"2023-10-06T05:20:56.065948+0000","last_undegraded":"2023-10-06T05:20:56.065948+0000","last_fullsized":"2023-10-06T05:20:56.065948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:17.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.049531+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063353+0000","last_change":"2023-10-06T05:20:56.063353+0000","last_active":"2023-10-06T05:20:56.063353+0000","last_peered":"2023-10-06T05:20:56.063353+0000","last_clean":"2023-10-06T05:20:56.063353+0000","last_became_active":"2023-10-06T05:20:56.063086+0000","last_became_peered":"2023-10-06T05:20:56.063086+0000","last_unstale":"2023-10-06T05:20:56.063353+0000","last_undegraded":"2023-10-06T05:20:56.063353+0000","last_fullsized":"2023-10-06T05:20:56.063353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:59.973368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052028+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066899+0000","last_change":"2023-10-06T05:20:56.066899+0000","last_active":"2023-10-06T05:20:56.066899+0000","last_peered":"2023-10-06T05:20:56.066899+0000","last_clean":"2023-10-06T05:20:56.066899+0000","last_became_active":"2023-10-06T05:20:56.066645+0000","last_became_peered":"2023-10-06T05:20:56.066645+0000","last_unstale":"2023-10-06T05:20:56.066899+0000","last_undegraded":"2023-10-06T05:20:56.066899+0000","last_fullsized":"2023-10-06T05:20:56.066899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:38.055535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044210+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:20:55.044210+0000","last_peered":"2023-10-06T05:20:55.044210+0000","last_clean":"2023-10-06T05:20:55.044210+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:20:55.044210+0000","last_undegraded":"2023-10-06T05:20:55.044210+0000","last_fullsized":"2023-10-06T05:20:55.044210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053542+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064164+0000","last_change":"2023-10-06T05:20:56.064164+0000","last_active":"2023-10-06T05:20:56.064164+0000","last_peered":"2023-10-06T05:20:56.064164+0000","last_clean":"2023-10-06T05:20:56.064164+0000","last_became_active":"2023-10-06T05:20:56.063907+0000","last_became_peered":"2023-10-06T05:20:56.063907+0000","last_unstale":"2023-10-06T05:20:56.064164+0000","last_undegraded":"2023-10-06T05:20:56.064164+0000","last_fullsized":"2023-10-06T05:20:56.064164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:16.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047296+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:20:55.047296+0000","last_peered":"2023-10-06T05:20:55.047296+0000","last_clean":"2023-10-06T05:20:55.047296+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:20:55.047296+0000","last_undegraded":"2023-10-06T05:20:55.047296+0000","last_fullsized":"2023-10-06T05:20:55.047296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053376+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:12.313245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062639+0000","last_change":"2023-10-06T05:20:56.062639+0000","last_active":"2023-10-06T05:20:56.062639+0000","last_peered":"2023-10-06T05:20:56.062639+0000","last_clean":"2023-10-06T05:20:56.062639+0000","last_became_active":"2023-10-06T05:20:56.062307+0000","last_became_peered":"2023-10-06T05:20:56.062307+0000","last_unstale":"2023-10-06T05:20:56.062639+0000","last_undegraded":"2023-10-06T05:20:56.062639+0000","last_fullsized":"2023-10-06T05:20:56.062639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.040639+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:20:55.040639+0000","last_peered":"2023-10-06T05:20:55.040639+0000","last_clean":"2023-10-06T05:20:55.040639+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:20:55.040639+0000","last_undegraded":"2023-10-06T05:20:55.040639+0000","last_fullsized":"2023-10-06T05:20:55.040639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048052+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067383+0000","last_change":"2023-10-06T05:20:56.067383+0000","last_active":"2023-10-06T05:20:56.067383+0000","last_peered":"2023-10-06T05:20:56.067383+0000","last_clean":"2023-10-06T05:20:56.067383+0000","last_became_active":"2023-10-06T05:20:56.067105+0000","last_became_peered":"2023-10-06T05:20:56.067105+0000","last_unstale":"2023-10-06T05:20:56.067383+0000","last_undegraded":"2023-10-06T05:20:56.067383+0000","last_fullsized":"2023-10-06T05:20:56.067383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:58.649017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047360+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:20:55.047360+0000","last_peered":"2023-10-06T05:20:55.047360+0000","last_clean":"2023-10-06T05:20:55.047360+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:20:55.047360+0000","last_undegraded":"2023-10-06T05:20:55.047360+0000","last_fullsized":"2023-10-06T05:20:55.047360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044900+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:20:55.044900+0000","last_peered":"2023-10-06T05:20:55.044900+0000","last_clean":"2023-10-06T05:20:55.044900+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:20:55.044900+0000","last_undegraded":"2023-10-06T05:20:55.044900+0000","last_fullsized":"2023-10-06T05:20:55.044900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054988+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:06.913609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066067+0000","last_change":"2023-10-06T05:20:56.066067+0000","last_active":"2023-10-06T05:20:56.066067+0000","last_peered":"2023-10-06T05:20:56.066067+0000","last_clean":"2023-10-06T05:20:56.066067+0000","last_became_active":"2023-10-06T05:20:56.065696+0000","last_became_peered":"2023-10-06T05:20:56.065696+0000","last_unstale":"2023-10-06T05:20:56.066067+0000","last_undegraded":"2023-10-06T05:20:56.066067+0000","last_fullsized":"2023-10-06T05:20:56.066067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:03.468905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044461+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:20:55.044461+0000","last_peered":"2023-10-06T05:20:55.044461+0000","last_clean":"2023-10-06T05:20:55.044461+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:20:55.044461+0000","last_undegraded":"2023-10-06T05:20:55.044461+0000","last_fullsized":"2023-10-06T05:20:55.044461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053799+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:20.780861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063757+0000","last_change":"2023-10-06T05:20:56.063757+0000","last_active":"2023-10-06T05:20:56.063757+0000","last_peered":"2023-10-06T05:20:56.063757+0000","last_clean":"2023-10-06T05:20:56.063757+0000","last_became_active":"2023-10-06T05:20:56.063497+0000","last_became_peered":"2023-10-06T05:20:56.063497+0000","last_unstale":"2023-10-06T05:20:56.063757+0000","last_undegraded":"2023-10-06T05:20:56.063757+0000","last_fullsized":"2023-10-06T05:20:56.063757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:07.370987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044759+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:20:55.044759+0000","last_peered":"2023-10-06T05:20:55.044759+0000","last_clean":"2023-10-06T05:20:55.044759+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:20:55.044759+0000","last_undegraded":"2023-10-06T05:20:55.044759+0000","last_fullsized":"2023-10-06T05:20:55.044759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053788+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067795+0000","last_change":"2023-10-06T05:20:56.067795+0000","last_active":"2023-10-06T05:20:56.067795+0000","last_peered":"2023-10-06T05:20:56.067795+0000","last_clean":"2023-10-06T05:20:56.067795+0000","last_became_active":"2023-10-06T05:20:56.067515+0000","last_became_peered":"2023-10-06T05:20:56.067515+0000","last_unstale":"2023-10-06T05:20:56.067795+0000","last_undegraded":"2023-10-06T05:20:56.067795+0000","last_fullsized":"2023-10-06T05:20:56.067795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:17.822671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044296+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:20:55.044296+0000","last_peered":"2023-10-06T05:20:55.044296+0000","last_clean":"2023-10-06T05:20:55.044296+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:20:55.044296+0000","last_undegraded":"2023-10-06T05:20:55.044296+0000","last_fullsized":"2023-10-06T05:20:55.044296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051963+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:59.430802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066251+0000","last_change":"2023-10-06T05:20:56.066251+0000","last_active":"2023-10-06T05:20:56.066251+0000","last_peered":"2023-10-06T05:20:56.066251+0000","last_clean":"2023-10-06T05:20:56.066251+0000","last_became_active":"2023-10-06T05:20:56.065923+0000","last_became_peered":"2023-10-06T05:20:56.065923+0000","last_unstale":"2023-10-06T05:20:56.066251+0000","last_undegraded":"2023-10-06T05:20:56.066251+0000","last_fullsized":"2023-10-06T05:20:56.066251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:09.868032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.042122+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:20:55.042122+0000","last_peered":"2023-10-06T05:20:55.042122+0000","last_clean":"2023-10-06T05:20:55.042122+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:20:55.042122+0000","last_undegraded":"2023-10-06T05:20:55.042122+0000","last_fullsized":"2023-10-06T05:20:55.042122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99099999999999999,"5min":1.169,"15min":1.169},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.628},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.268,"15min":1.268},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.075,"15min":1.075},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.68799999999999994},"min":{"1min":0.57199999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.995,"5min":1.093,"15min":1.093},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":1.22,"15min":1.22},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0349999999999999,"15min":1.079},"last":0.624},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":1.117,"15min":1.202},"last":0.85299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.622},"min":{"1min":0.47199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":1.258,"15min":1.258},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.64500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97599999999999998,"5min":1.2050000000000001,"15min":1.407},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.92500000000000004,"5min":1.139,"15min":1.139},"last":0.747},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.24,"15min":1.3300000000000001},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.034},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.1599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.76200000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91500000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.67700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.95699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82899999999999996,"5min":0.97799999999999998,"15min":0.998},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.0760000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68999999999999995,"15min":0.77900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.77900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":0.97999999999999998,"15min":1.038},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73099999999999998,"15min":0.84399999999999997},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.442,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60099999999999998,"5min":0.73099999999999998,"15min":2.7349999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48499999999999999,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54300000000000004,"5min":0.71799999999999997,"15min":2.6520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53600000000000003,"15min":0.61099999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.85299999999999998,"15min":2.7639999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53400000000000003,"15min":0.61399999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.82699999999999996,"15min":2.8370000000000002},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.50700000000000001},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.125,"15min":1.2929999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.751,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93200000000000005,"5min":1.0289999999999999,"15min":1.349},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.82899999999999996},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":1.1419999999999999,"15min":1.669},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.86399999999999999},"min":{"1min":0.66800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91900000000000004,"5min":1.1779999999999999,"15min":1.6040000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88700000000000001},"min":{"1min":0.61499999999999999,"5min":0.503,"15min":0.503},"max":{"1min":0.96999999999999997,"5min":1.165,"15min":1.573},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":1.0780000000000001,"15min":1.528},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.75},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46200000000000002,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64600000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73199999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.62},{"interface":"front","average":{"1min":0.497,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.434}]},{"osd":3,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.879},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.75700000000000001},"min":{"1min":0.65200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0069999999999999,"5min":1.218,"15min":1.218},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.749},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.79400000000000004},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.05,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.63700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.46200000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48699999999999999,"15min":0.47399999999999998},"min":{"1min":0.42599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":1.0329999999999999,"5min":1.077,"15min":1.1379999999999999},"last":0.88},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.47499999999999998},"max":{"1min":0.98699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.495},"max":{"1min":1.04,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.319},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":0.79200000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.103,"5min":1.103,"15min":1.141},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52100000000000002},"max":{"1min":1.0609999999999999,"5min":1.1000000000000001,"15min":1.282},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.68100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.3520000000000001},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.70799999999999996,"5min":1.0720000000000001,"15min":2.0950000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":1.772},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.78100000000000003,"15min":1.913},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.66800000000000004,"5min":0.84399999999999997,"15min":2.0089999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55400000000000005,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.82399999999999995,"15min":1.954},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0469999999999999,"5min":1.2030000000000001,"15min":1.282},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.0669999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1399999999999999,"5min":1.2250000000000001,"15min":1.3009999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.1910000000000001,"15min":1.343},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3380000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82599999999999996,"15min":0.85999999999999999},"min":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56799999999999995},"max":{"1min":1.1140000000000001,"5min":1.206,"15min":1.3640000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.68700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1539999999999999,"5min":1.2090000000000001,"15min":1.214},"last":0.746}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.745,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76600000000000001,"15min":0.76000000000000001},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0269999999999999,"15min":1.169},"last":0.755}]},{"osd":1,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.042,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.117},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.002},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.196},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63500000000000001,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.8740000000000001},"last":0.5},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66100000000000003,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.375},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98799999999999999,"15min":1.004},"last":0.44},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95699999999999996,"5min":1.1639999999999999,"15min":1.9390000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71199999999999997,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93300000000000005,"5min":1.0209999999999999,"15min":1.466},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.83999999999999997},"min":{"1min":0.626,"5min":0.499,"15min":0.499},"max":{"1min":0.91800000000000004,"5min":1.1930000000000001,"15min":1.218},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90800000000000003,"5min":1.165,"15min":1.165},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98999999999999999,"5min":1.016,"15min":1.1890000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.69299999999999995},"min":{"1min":0.59899999999999998,"5min":0.45100000000000001,"15min":0.438},"max":{"1min":0.88200000000000001,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.038,"15min":1.038},"last":0.88700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":1.204,"15min":1.204},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.70699999999999996},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.995,"15min":1.0549999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.087},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.58499999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.97299999999999998,"5min":1.2769999999999999,"15min":1.278},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64900000000000002,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.996,"15min":1.294},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64400000000000002,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.141,"15min":1.141},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.876,"5min":0.999,"15min":1.254},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71899999999999997,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.042,"15min":1.337},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.55300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.248,"15min":1.248},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:01.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:01.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.667+0000 7f58c099d700 1 Processor -- start 2023-10-06T05:21:01.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.668+0000 7f58c099d700 1 -- start start 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58c099d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58bc13faa0 0x7f58bc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58c099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58c099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0c9cb0 con 0x7f58bc06aad0 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58c099d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc06b660 con 0x7f58bc13faa0 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58c099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0ca270 con 0x7f58bc0c7940 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58bb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58bc13faa0 0x7f58bc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58bbfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f58bc06aad0 0x7f58bc06af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56054/0 (socket says 172.21.15.33:56054) 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58bbfff700 1 -- 172.21.15.33:0/598091244 learned_addr learned my addr 172.21.15.33:0/598091244 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.669+0000 7f58baffd700 1 --2- 172.21.15.33:0/598091244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58bb7fe700 1 -- 172.21.15.33:0/598091244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 msgr2=0x7f58bc0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58bb7fe700 1 --2- 172.21.15.33:0/598091244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58bb7fe700 1 -- 172.21.15.33:0/598091244 >> v1:172.21.15.33:6789/0 conn(0x7f58bc06aad0 legacy=0x7f58bc06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58bb7fe700 1 -- 172.21.15.33:0/598091244 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc06b840 con 0x7f58bc13faa0 2023-10-06T05:21:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58ba7fc700 1 -- 172.21.15.33:0/598091244 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741574161 0 0) 0x7f58bc0c9cb0 con 0x7f58bc06aad0 2023-10-06T05:21:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58bb7fe700 1 --2- 172.21.15.33:0/598091244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc13fe70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00bb90 tx=0x7f58ac00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead2c41ebb4c0f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58ba7fc700 1 -- 172.21.15.33:0/598091244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00dba0 con 0x7f58bc13faa0 2023-10-06T05:21:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58ba7fc700 1 -- 172.21.15.33:0/598091244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00dd00 con 0x7f58bc13faa0 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.670+0000 7f58ba7fc700 1 -- 172.21.15.33:0/598091244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0196b0 con 0x7f58bc13faa0 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 -- 172.21.15.33:0/598091244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 msgr2=0x7f58bc13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 --2- 172.21.15.33:0/598091244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc13fe70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00bb90 tx=0x7f58ac00b430 comp rx=0 tx=0).stop 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 -- 172.21.15.33:0/598091244 shutdown_connections 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 --2- 172.21.15.33:0/598091244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc13fe70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 --2- 172.21.15.33:0/598091244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 -- 172.21.15.33:0/598091244 >> 172.21.15.33:0/598091244 conn(0x7f58bc06d030 msgr2=0x7f58bc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 -- 172.21.15.33:0/598091244 shutdown_connections 2023-10-06T05:21:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.671+0000 7f58c099d700 1 -- 172.21.15.33:0/598091244 wait complete. 2023-10-06T05:21:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.672+0000 7f58c099d700 1 Processor -- start 2023-10-06T05:21:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.672+0000 7f58c099d700 1 -- start start 2023-10-06T05:21:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.673+0000 7f58c099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58bc06aad0 0x7f58bc168750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.673+0000 7f58c099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.673+0000 7f58c099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc0c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.673+0000 7f58c099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0c5690 con 0x7f58bc13faa0 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.673+0000 7f58c099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0c47e0 con 0x7f58bc06aad0 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.673+0000 7f58c099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1567f0 con 0x7f58bc0c7940 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc0c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc0c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43966/0 (socket says 172.21.15.33:43966) 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 -- 172.21.15.33:0/2939997279 learned_addr learned my addr 172.21.15.33:0/2939997279 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bb7fe700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58bc06aad0 0x7f58bc168750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58baffd700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 -- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 msgr2=0x7f58bc0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 -- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58bc06aad0 msgr2=0x7f58bc168750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58bc06aad0 0x7f58bc168750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 -- 172.21.15.33:0/2939997279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac00b6d0 con 0x7f58bc13faa0 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58bbfff700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc0c42a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f58b400cc90 tx=0x7f58b4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.674+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b400cea0 con 0x7f58bc13faa0 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.675+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b400ab70 con 0x7f58bc13faa0 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.675+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40070f0 con 0x7f58bc13faa0 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.675+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc0c4a10 con 0x7f58bc13faa0 2023-10-06T05:21:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.675+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc0c4fd0 con 0x7f58bc13faa0 2023-10-06T05:21:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.676+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc0c57d0 con 0x7f58bc13faa0 2023-10-06T05:21:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.677+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58b40074a0 con 0x7f58bc13faa0 2023-10-06T05:21:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.678+0000 7f58a37fe700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a8066f30 0x7f58a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.678+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f58b4084bb0 con 0x7f58bc13faa0 2023-10-06T05:21:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.678+0000 7f58bb7fe700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a8066f30 0x7f58a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.679+0000 7f58bb7fe700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a8066f30 0x7f58a80693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00b6a0 tx=0x7f58ac01b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.682+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58bc0c57d0 con 0x7f58bc13faa0 2023-10-06T05:21:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.880+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58bc0c57d0 con 0x7f58a8066f30 2023-10-06T05:21:01.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.890+0000 7f58a37fe700 1 -- 172.21.15.33:0/2939997279 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307738 (secure 0 0 0) 0x7f58bc0c57d0 con 0x7f58a8066f30 2023-10-06T05:21:01.896 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a8066f30 msgr2=0x7f58a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a8066f30 0x7f58a80693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00b6a0 tx=0x7f58ac01b000 comp rx=0 tx=0).stop 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 msgr2=0x7f58bc0c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc0c42a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f58b400cc90 tx=0x7f58b4001a80 comp rx=0 tx=0).stop 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 shutdown_connections 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a8066f30 0x7f58a80693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58bc13faa0 0x7f58bc0c42a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58bc06aad0 0x7f58bc168750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 --2- 172.21.15.33:0/2939997279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58bc0c7940 0x7f58bc0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 >> 172.21.15.33:0/2939997279 conn(0x7f58bc06d030 msgr2=0x7f58bc0c1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 shutdown_connections 2023-10-06T05:21:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:01.894+0000 7f58c099d700 1 -- 172.21.15.33:0/2939997279 wait complete. 2023-10-06T05:21:01.901 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:21:01.915 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-10-06T05:20:59.913056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420584,"kb_used_data":9512,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752554136,"statfs":{"total":773094113280,"available":770615435264,"internally_reserved":0,"allocated":9740288,"data_stored":5025216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.874634"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047746+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054711+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047774+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061615+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059581+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059292+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:03.594147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048133+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:39.064497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052728+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052245+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:35.106422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053309+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.455963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051588+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:37.245927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.060583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:07.387943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056543+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.063672+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058649+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:58.648941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059036+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:10.851643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048750+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054442+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059967+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:18.831632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058550+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056058+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054085+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:24.396610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047445+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055287+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:59.686854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:04.727723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051706+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:10.346856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059449+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:08.100764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047908+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056488+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:17.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057141+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:34.761680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056494+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:42.458155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069083+0000","last_change":"2023-10-06T05:20:56.069083+0000","last_active":"2023-10-06T05:20:56.069083+0000","last_peered":"2023-10-06T05:20:56.069083+0000","last_clean":"2023-10-06T05:20:56.069083+0000","last_became_active":"2023-10-06T05:20:56.068775+0000","last_became_peered":"2023-10-06T05:20:56.068775+0000","last_unstale":"2023-10-06T05:20:56.069083+0000","last_undegraded":"2023-10-06T05:20:56.069083+0000","last_fullsized":"2023-10-06T05:20:56.069083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:43.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056509+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:23.982186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.061418+0000","last_change":"2023-10-06T05:20:56.061418+0000","last_active":"2023-10-06T05:20:56.061418+0000","last_peered":"2023-10-06T05:20:56.061418+0000","last_clean":"2023-10-06T05:20:56.061418+0000","last_became_active":"2023-10-06T05:20:56.060531+0000","last_became_peered":"2023-10-06T05:20:56.060531+0000","last_unstale":"2023-10-06T05:20:56.061418+0000","last_undegraded":"2023-10-06T05:20:56.061418+0000","last_fullsized":"2023-10-06T05:20:56.061418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:47.802684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048418+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063062+0000","last_change":"2023-10-06T05:20:56.063062+0000","last_active":"2023-10-06T05:20:56.063062+0000","last_peered":"2023-10-06T05:20:56.063062+0000","last_clean":"2023-10-06T05:20:56.063062+0000","last_became_active":"2023-10-06T05:20:56.062410+0000","last_became_peered":"2023-10-06T05:20:56.062410+0000","last_unstale":"2023-10-06T05:20:56.063062+0000","last_undegraded":"2023-10-06T05:20:56.063062+0000","last_fullsized":"2023-10-06T05:20:56.063062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:49.619842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057087+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:37.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062842+0000","last_change":"2023-10-06T05:20:56.062842+0000","last_active":"2023-10-06T05:20:56.062842+0000","last_peered":"2023-10-06T05:20:56.062842+0000","last_clean":"2023-10-06T05:20:56.062842+0000","last_became_active":"2023-10-06T05:20:56.062590+0000","last_became_peered":"2023-10-06T05:20:56.062590+0000","last_unstale":"2023-10-06T05:20:56.062842+0000","last_undegraded":"2023-10-06T05:20:56.062842+0000","last_fullsized":"2023-10-06T05:20:56.062842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:58.961459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057855+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.845343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063129+0000","last_change":"2023-10-06T05:20:56.063129+0000","last_active":"2023-10-06T05:20:56.063129+0000","last_peered":"2023-10-06T05:20:56.063129+0000","last_clean":"2023-10-06T05:20:56.063129+0000","last_became_active":"2023-10-06T05:20:56.062230+0000","last_became_peered":"2023-10-06T05:20:56.062230+0000","last_unstale":"2023-10-06T05:20:56.063129+0000","last_undegraded":"2023-10-06T05:20:56.063129+0000","last_fullsized":"2023-10-06T05:20:56.063129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:58.928023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066379+0000","last_change":"2023-10-06T05:20:56.066379+0000","last_active":"2023-10-06T05:20:56.066379+0000","last_peered":"2023-10-06T05:20:56.066379+0000","last_clean":"2023-10-06T05:20:56.066379+0000","last_became_active":"2023-10-06T05:20:56.064912+0000","last_became_peered":"2023-10-06T05:20:56.064912+0000","last_unstale":"2023-10-06T05:20:56.066379+0000","last_undegraded":"2023-10-06T05:20:56.066379+0000","last_fullsized":"2023-10-06T05:20:56.066379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:23.256874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058738+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:27.536202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063667+0000","last_change":"2023-10-06T05:20:56.063667+0000","last_active":"2023-10-06T05:20:56.063667+0000","last_peered":"2023-10-06T05:20:56.063667+0000","last_clean":"2023-10-06T05:20:56.063667+0000","last_became_active":"2023-10-06T05:20:56.063346+0000","last_became_peered":"2023-10-06T05:20:56.063346+0000","last_unstale":"2023-10-06T05:20:56.063667+0000","last_undegraded":"2023-10-06T05:20:56.063667+0000","last_fullsized":"2023-10-06T05:20:56.063667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058683+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:36.533849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.068185+0000","last_change":"2023-10-06T05:20:56.068185+0000","last_active":"2023-10-06T05:20:56.068185+0000","last_peered":"2023-10-06T05:20:56.068185+0000","last_clean":"2023-10-06T05:20:56.068185+0000","last_became_active":"2023-10-06T05:20:56.067889+0000","last_became_peered":"2023-10-06T05:20:56.067889+0000","last_unstale":"2023-10-06T05:20:56.068185+0000","last_undegraded":"2023-10-06T05:20:56.068185+0000","last_fullsized":"2023-10-06T05:20:56.068185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052435+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064663+0000","last_change":"2023-10-06T05:20:56.064663+0000","last_active":"2023-10-06T05:20:56.064663+0000","last_peered":"2023-10-06T05:20:56.064663+0000","last_clean":"2023-10-06T05:20:56.064663+0000","last_became_active":"2023-10-06T05:20:56.064382+0000","last_became_peered":"2023-10-06T05:20:56.064382+0000","last_unstale":"2023-10-06T05:20:56.064663+0000","last_undegraded":"2023-10-06T05:20:56.064663+0000","last_fullsized":"2023-10-06T05:20:56.064663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:07.091803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047994+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062871+0000","last_change":"2023-10-06T05:20:56.062871+0000","last_active":"2023-10-06T05:20:56.062871+0000","last_peered":"2023-10-06T05:20:56.062871+0000","last_clean":"2023-10-06T05:20:56.062871+0000","last_became_active":"2023-10-06T05:20:56.062606+0000","last_became_peered":"2023-10-06T05:20:56.062606+0000","last_unstale":"2023-10-06T05:20:56.062871+0000","last_undegraded":"2023-10-06T05:20:56.062871+0000","last_fullsized":"2023-10-06T05:20:56.062871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048278+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069433+0000","last_change":"2023-10-06T05:20:56.069433+0000","last_active":"2023-10-06T05:20:56.069433+0000","last_peered":"2023-10-06T05:20:56.069433+0000","last_clean":"2023-10-06T05:20:56.069433+0000","last_became_active":"2023-10-06T05:20:56.069152+0000","last_became_peered":"2023-10-06T05:20:56.069152+0000","last_unstale":"2023-10-06T05:20:56.069433+0000","last_undegraded":"2023-10-06T05:20:56.069433+0000","last_fullsized":"2023-10-06T05:20:56.069433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.728922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057259+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:12.235739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066398+0000","last_change":"2023-10-06T05:20:56.066398+0000","last_active":"2023-10-06T05:20:56.066398+0000","last_peered":"2023-10-06T05:20:56.066398+0000","last_clean":"2023-10-06T05:20:56.066398+0000","last_became_active":"2023-10-06T05:20:56.066051+0000","last_became_peered":"2023-10-06T05:20:56.066051+0000","last_unstale":"2023-10-06T05:20:56.066398+0000","last_undegraded":"2023-10-06T05:20:56.066398+0000","last_fullsized":"2023-10-06T05:20:56.066398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:26.538710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056633+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:48.550385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060068+0000","last_change":"2023-10-06T05:20:56.060068+0000","last_active":"2023-10-06T05:20:56.060068+0000","last_peered":"2023-10-06T05:20:56.060068+0000","last_clean":"2023-10-06T05:20:56.060068+0000","last_became_active":"2023-10-06T05:20:56.059765+0000","last_became_peered":"2023-10-06T05:20:56.059765+0000","last_unstale":"2023-10-06T05:20:56.060068+0000","last_undegraded":"2023-10-06T05:20:56.060068+0000","last_fullsized":"2023-10-06T05:20:56.060068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:56.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052031+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062607+0000","last_change":"2023-10-06T05:20:56.062607+0000","last_active":"2023-10-06T05:20:56.062607+0000","last_peered":"2023-10-06T05:20:56.062607+0000","last_clean":"2023-10-06T05:20:56.062607+0000","last_became_active":"2023-10-06T05:20:56.062344+0000","last_became_peered":"2023-10-06T05:20:56.062344+0000","last_unstale":"2023-10-06T05:20:56.062607+0000","last_undegraded":"2023-10-06T05:20:56.062607+0000","last_fullsized":"2023-10-06T05:20:56.062607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.897142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051968+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:14.557678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067504+0000","last_change":"2023-10-06T05:20:56.067504+0000","last_active":"2023-10-06T05:20:56.067504+0000","last_peered":"2023-10-06T05:20:56.067504+0000","last_clean":"2023-10-06T05:20:56.067504+0000","last_became_active":"2023-10-06T05:20:56.066985+0000","last_became_peered":"2023-10-06T05:20:56.066985+0000","last_unstale":"2023-10-06T05:20:56.067504+0000","last_undegraded":"2023-10-06T05:20:56.067504+0000","last_fullsized":"2023-10-06T05:20:56.067504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:43.665749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051195+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:39.400959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063906+0000","last_change":"2023-10-06T05:20:56.063906+0000","last_active":"2023-10-06T05:20:56.063906+0000","last_peered":"2023-10-06T05:20:56.063906+0000","last_clean":"2023-10-06T05:20:56.063906+0000","last_became_active":"2023-10-06T05:20:56.063625+0000","last_became_peered":"2023-10-06T05:20:56.063625+0000","last_unstale":"2023-10-06T05:20:56.063906+0000","last_undegraded":"2023-10-06T05:20:56.063906+0000","last_fullsized":"2023-10-06T05:20:56.063906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:07.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054747+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:55.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060664+0000","last_change":"2023-10-06T05:20:56.060664+0000","last_active":"2023-10-06T05:20:56.060664+0000","last_peered":"2023-10-06T05:20:56.060664+0000","last_clean":"2023-10-06T05:20:56.060664+0000","last_became_active":"2023-10-06T05:20:56.059179+0000","last_became_peered":"2023-10-06T05:20:56.059179+0000","last_unstale":"2023-10-06T05:20:56.060664+0000","last_undegraded":"2023-10-06T05:20:56.060664+0000","last_fullsized":"2023-10-06T05:20:56.060664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.078805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050160+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069531+0000","last_change":"2023-10-06T05:20:56.069531+0000","last_active":"2023-10-06T05:20:56.069531+0000","last_peered":"2023-10-06T05:20:56.069531+0000","last_clean":"2023-10-06T05:20:56.069531+0000","last_became_active":"2023-10-06T05:20:56.068428+0000","last_became_peered":"2023-10-06T05:20:56.068428+0000","last_unstale":"2023-10-06T05:20:56.069531+0000","last_undegraded":"2023-10-06T05:20:56.069531+0000","last_fullsized":"2023-10-06T05:20:56.069531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050790+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069310+0000","last_change":"2023-10-06T05:20:56.069310+0000","last_active":"2023-10-06T05:20:56.069310+0000","last_peered":"2023-10-06T05:20:56.069310+0000","last_clean":"2023-10-06T05:20:56.069310+0000","last_became_active":"2023-10-06T05:20:56.069031+0000","last_became_peered":"2023-10-06T05:20:56.069031+0000","last_unstale":"2023-10-06T05:20:56.069310+0000","last_undegraded":"2023-10-06T05:20:56.069310+0000","last_fullsized":"2023-10-06T05:20:56.069310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:58.677893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054813+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:44.878348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066147+0000","last_change":"2023-10-06T05:20:56.066147+0000","last_active":"2023-10-06T05:20:56.066147+0000","last_peered":"2023-10-06T05:20:56.066147+0000","last_clean":"2023-10-06T05:20:56.066147+0000","last_became_active":"2023-10-06T05:20:56.065618+0000","last_became_peered":"2023-10-06T05:20:56.065618+0000","last_unstale":"2023-10-06T05:20:56.066147+0000","last_undegraded":"2023-10-06T05:20:56.066147+0000","last_fullsized":"2023-10-06T05:20:56.066147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:33.934156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051253+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:59.973383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062131+0000","last_change":"2023-10-06T05:20:56.062131+0000","last_active":"2023-10-06T05:20:56.062131+0000","last_peered":"2023-10-06T05:20:56.062131+0000","last_clean":"2023-10-06T05:20:56.062131+0000","last_became_active":"2023-10-06T05:20:56.061821+0000","last_became_peered":"2023-10-06T05:20:56.061821+0000","last_unstale":"2023-10-06T05:20:56.062131+0000","last_undegraded":"2023-10-06T05:20:56.062131+0000","last_fullsized":"2023-10-06T05:20:56.062131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:33.997234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047924+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067790+0000","last_change":"2023-10-06T05:20:56.067790+0000","last_active":"2023-10-06T05:20:56.067790+0000","last_peered":"2023-10-06T05:20:56.067790+0000","last_clean":"2023-10-06T05:20:56.067790+0000","last_became_active":"2023-10-06T05:20:56.067491+0000","last_became_peered":"2023-10-06T05:20:56.067491+0000","last_unstale":"2023-10-06T05:20:56.067790+0000","last_undegraded":"2023-10-06T05:20:56.067790+0000","last_fullsized":"2023-10-06T05:20:56.067790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:06.378982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058270+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:05.605433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.065948+0000","last_change":"2023-10-06T05:20:56.065948+0000","last_active":"2023-10-06T05:20:56.065948+0000","last_peered":"2023-10-06T05:20:56.065948+0000","last_clean":"2023-10-06T05:20:56.065948+0000","last_became_active":"2023-10-06T05:20:56.065640+0000","last_became_peered":"2023-10-06T05:20:56.065640+0000","last_unstale":"2023-10-06T05:20:56.065948+0000","last_undegraded":"2023-10-06T05:20:56.065948+0000","last_fullsized":"2023-10-06T05:20:56.065948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:17.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.049531+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063353+0000","last_change":"2023-10-06T05:20:56.063353+0000","last_active":"2023-10-06T05:20:56.063353+0000","last_peered":"2023-10-06T05:20:56.063353+0000","last_clean":"2023-10-06T05:20:56.063353+0000","last_became_active":"2023-10-06T05:20:56.063086+0000","last_became_peered":"2023-10-06T05:20:56.063086+0000","last_unstale":"2023-10-06T05:20:56.063353+0000","last_undegraded":"2023-10-06T05:20:56.063353+0000","last_fullsized":"2023-10-06T05:20:56.063353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:59.973368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052028+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066899+0000","last_change":"2023-10-06T05:20:56.066899+0000","last_active":"2023-10-06T05:20:56.066899+0000","last_peered":"2023-10-06T05:20:56.066899+0000","last_clean":"2023-10-06T05:20:56.066899+0000","last_became_active":"2023-10-06T05:20:56.066645+0000","last_became_peered":"2023-10-06T05:20:56.066645+0000","last_unstale":"2023-10-06T05:20:56.066899+0000","last_undegraded":"2023-10-06T05:20:56.066899+0000","last_fullsized":"2023-10-06T05:20:56.066899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:38.055535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044210+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:20:55.044210+0000","last_peered":"2023-10-06T05:20:55.044210+0000","last_clean":"2023-10-06T05:20:55.044210+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:20:55.044210+0000","last_undegraded":"2023-10-06T05:20:55.044210+0000","last_fullsized":"2023-10-06T05:20:55.044210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053542+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064164+0000","last_change":"2023-10-06T05:20:56.064164+0000","last_active":"2023-10-06T05:20:56.064164+0000","last_peered":"2023-10-06T05:20:56.064164+0000","last_clean":"2023-10-06T05:20:56.064164+0000","last_became_active":"2023-10-06T05:20:56.063907+0000","last_became_peered":"2023-10-06T05:20:56.063907+0000","last_unstale":"2023-10-06T05:20:56.064164+0000","last_undegraded":"2023-10-06T05:20:56.064164+0000","last_fullsized":"2023-10-06T05:20:56.064164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:16.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047296+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:20:55.047296+0000","last_peered":"2023-10-06T05:20:55.047296+0000","last_clean":"2023-10-06T05:20:55.047296+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:20:55.047296+0000","last_undegraded":"2023-10-06T05:20:55.047296+0000","last_fullsized":"2023-10-06T05:20:55.047296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053376+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:12.313245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062639+0000","last_change":"2023-10-06T05:20:56.062639+0000","last_active":"2023-10-06T05:20:56.062639+0000","last_peered":"2023-10-06T05:20:56.062639+0000","last_clean":"2023-10-06T05:20:56.062639+0000","last_became_active":"2023-10-06T05:20:56.062307+0000","last_became_peered":"2023-10-06T05:20:56.062307+0000","last_unstale":"2023-10-06T05:20:56.062639+0000","last_undegraded":"2023-10-06T05:20:56.062639+0000","last_fullsized":"2023-10-06T05:20:56.062639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.040639+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:20:55.040639+0000","last_peered":"2023-10-06T05:20:55.040639+0000","last_clean":"2023-10-06T05:20:55.040639+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:20:55.040639+0000","last_undegraded":"2023-10-06T05:20:55.040639+0000","last_fullsized":"2023-10-06T05:20:55.040639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048052+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067383+0000","last_change":"2023-10-06T05:20:56.067383+0000","last_active":"2023-10-06T05:20:56.067383+0000","last_peered":"2023-10-06T05:20:56.067383+0000","last_clean":"2023-10-06T05:20:56.067383+0000","last_became_active":"2023-10-06T05:20:56.067105+0000","last_became_peered":"2023-10-06T05:20:56.067105+0000","last_unstale":"2023-10-06T05:20:56.067383+0000","last_undegraded":"2023-10-06T05:20:56.067383+0000","last_fullsized":"2023-10-06T05:20:56.067383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:58.649017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047360+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:20:55.047360+0000","last_peered":"2023-10-06T05:20:55.047360+0000","last_clean":"2023-10-06T05:20:55.047360+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:20:55.047360+0000","last_undegraded":"2023-10-06T05:20:55.047360+0000","last_fullsized":"2023-10-06T05:20:55.047360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044900+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:20:55.044900+0000","last_peered":"2023-10-06T05:20:55.044900+0000","last_clean":"2023-10-06T05:20:55.044900+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:20:55.044900+0000","last_undegraded":"2023-10-06T05:20:55.044900+0000","last_fullsized":"2023-10-06T05:20:55.044900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054988+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:06.913609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066067+0000","last_change":"2023-10-06T05:20:56.066067+0000","last_active":"2023-10-06T05:20:56.066067+0000","last_peered":"2023-10-06T05:20:56.066067+0000","last_clean":"2023-10-06T05:20:56.066067+0000","last_became_active":"2023-10-06T05:20:56.065696+0000","last_became_peered":"2023-10-06T05:20:56.065696+0000","last_unstale":"2023-10-06T05:20:56.066067+0000","last_undegraded":"2023-10-06T05:20:56.066067+0000","last_fullsized":"2023-10-06T05:20:56.066067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:03.468905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044461+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:20:55.044461+0000","last_peered":"2023-10-06T05:20:55.044461+0000","last_clean":"2023-10-06T05:20:55.044461+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:20:55.044461+0000","last_undegraded":"2023-10-06T05:20:55.044461+0000","last_fullsized":"2023-10-06T05:20:55.044461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053799+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:20.780861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063757+0000","last_change":"2023-10-06T05:20:56.063757+0000","last_active":"2023-10-06T05:20:56.063757+0000","last_peered":"2023-10-06T05:20:56.063757+0000","last_clean":"2023-10-06T05:20:56.063757+0000","last_became_active":"2023-10-06T05:20:56.063497+0000","last_became_peered":"2023-10-06T05:20:56.063497+0000","last_unstale":"2023-10-06T05:20:56.063757+0000","last_undegraded":"2023-10-06T05:20:56.063757+0000","last_fullsized":"2023-10-06T05:20:56.063757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:07.370987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044759+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:20:55.044759+0000","last_peered":"2023-10-06T05:20:55.044759+0000","last_clean":"2023-10-06T05:20:55.044759+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:20:55.044759+0000","last_undegraded":"2023-10-06T05:20:55.044759+0000","last_fullsized":"2023-10-06T05:20:55.044759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053788+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067795+0000","last_change":"2023-10-06T05:20:56.067795+0000","last_active":"2023-10-06T05:20:56.067795+0000","last_peered":"2023-10-06T05:20:56.067795+0000","last_clean":"2023-10-06T05:20:56.067795+0000","last_became_active":"2023-10-06T05:20:56.067515+0000","last_became_peered":"2023-10-06T05:20:56.067515+0000","last_unstale":"2023-10-06T05:20:56.067795+0000","last_undegraded":"2023-10-06T05:20:56.067795+0000","last_fullsized":"2023-10-06T05:20:56.067795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:17.822671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044296+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:20:55.044296+0000","last_peered":"2023-10-06T05:20:55.044296+0000","last_clean":"2023-10-06T05:20:55.044296+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:20:55.044296+0000","last_undegraded":"2023-10-06T05:20:55.044296+0000","last_fullsized":"2023-10-06T05:20:55.044296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051963+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:59.430802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066251+0000","last_change":"2023-10-06T05:20:56.066251+0000","last_active":"2023-10-06T05:20:56.066251+0000","last_peered":"2023-10-06T05:20:56.066251+0000","last_clean":"2023-10-06T05:20:56.066251+0000","last_became_active":"2023-10-06T05:20:56.065923+0000","last_became_peered":"2023-10-06T05:20:56.065923+0000","last_unstale":"2023-10-06T05:20:56.066251+0000","last_undegraded":"2023-10-06T05:20:56.066251+0000","last_fullsized":"2023-10-06T05:20:56.066251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:09.868032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.042122+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:20:55.042122+0000","last_peered":"2023-10-06T05:20:55.042122+0000","last_clean":"2023-10-06T05:20:55.042122+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:20:55.042122+0000","last_undegraded":"2023-10-06T05:20:55.042122+0000","last_fullsized":"2023-10-06T05:20:55.042122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99099999999999999,"5min":1.169,"15min":1.169},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.628},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.268,"15min":1.268},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.075,"15min":1.075},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.68799999999999994},"min":{"1min":0.57199999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.995,"5min":1.093,"15min":1.093},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":1.22,"15min":1.22},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0349999999999999,"15min":1.079},"last":0.624},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":1.117,"15min":1.202},"last":0.85299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.622},"min":{"1min":0.47199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":1.258,"15min":1.258},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.64500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97599999999999998,"5min":1.2050000000000001,"15min":1.407},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.92500000000000004,"5min":1.139,"15min":1.139},"last":0.747},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.24,"15min":1.3300000000000001},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.034},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.1599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.76200000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91500000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.67700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.95699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82899999999999996,"5min":0.97799999999999998,"15min":0.998},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.0760000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68999999999999995,"15min":0.77900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.77900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":0.97999999999999998,"15min":1.038},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73099999999999998,"15min":0.84399999999999997},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.442,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60099999999999998,"5min":0.73099999999999998,"15min":2.7349999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48499999999999999,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54300000000000004,"5min":0.71799999999999997,"15min":2.6520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53600000000000003,"15min":0.61099999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.85299999999999998,"15min":2.7639999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53400000000000003,"15min":0.61399999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.82699999999999996,"15min":2.8370000000000002},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.50700000000000001},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.125,"15min":1.2929999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.751,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93200000000000005,"5min":1.0289999999999999,"15min":1.349},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.82899999999999996},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":1.1419999999999999,"15min":1.669},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.86399999999999999},"min":{"1min":0.66800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91900000000000004,"5min":1.1779999999999999,"15min":1.6040000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88700000000000001},"min":{"1min":0.61499999999999999,"5min":0.503,"15min":0.503},"max":{"1min":0.96999999999999997,"5min":1.165,"15min":1.573},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":1.0780000000000001,"15min":1.528},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.75},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46200000000000002,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64600000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73199999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.62},{"interface":"front","average":{"1min":0.497,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.434}]},{"osd":3,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.879},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.75700000000000001},"min":{"1min":0.65200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0069999999999999,"5min":1.218,"15min":1.218},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.749},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.79400000000000004},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.05,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.63700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.46200000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48699999999999999,"15min":0.47399999999999998},"min":{"1min":0.42599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":1.0329999999999999,"5min":1.077,"15min":1.1379999999999999},"last":0.88},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.47499999999999998},"max":{"1min":0.98699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.495},"max":{"1min":1.04,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.319},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":0.79200000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.103,"5min":1.103,"15min":1.141},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52100000000000002},"max":{"1min":1.0609999999999999,"5min":1.1000000000000001,"15min":1.282},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.68100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.3520000000000001},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.70799999999999996,"5min":1.0720000000000001,"15min":2.0950000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":1.772},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.78100000000000003,"15min":1.913},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.66800000000000004,"5min":0.84399999999999997,"15min":2.0089999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55400000000000005,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.82399999999999995,"15min":1.954},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0469999999999999,"5min":1.2030000000000001,"15min":1.282},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.0669999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1399999999999999,"5min":1.2250000000000001,"15min":1.3009999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.1910000000000001,"15min":1.343},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3380000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82599999999999996,"15min":0.85999999999999999},"min":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56799999999999995},"max":{"1min":1.1140000000000001,"5min":1.206,"15min":1.3640000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.68700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1539999999999999,"5min":1.2090000000000001,"15min":1.214},"last":0.746}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.745,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76600000000000001,"15min":0.76000000000000001},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0269999999999999,"15min":1.169},"last":0.755}]},{"osd":1,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.042,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.117},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.002},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.196},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63500000000000001,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.8740000000000001},"last":0.5},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66100000000000003,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.375},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98799999999999999,"15min":1.004},"last":0.44},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95699999999999996,"5min":1.1639999999999999,"15min":1.9390000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71199999999999997,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93300000000000005,"5min":1.0209999999999999,"15min":1.466},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.83999999999999997},"min":{"1min":0.626,"5min":0.499,"15min":0.499},"max":{"1min":0.91800000000000004,"5min":1.1930000000000001,"15min":1.218},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90800000000000003,"5min":1.165,"15min":1.165},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98999999999999999,"5min":1.016,"15min":1.1890000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.69299999999999995},"min":{"1min":0.59899999999999998,"5min":0.45100000000000001,"15min":0.438},"max":{"1min":0.88200000000000001,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.038,"15min":1.038},"last":0.88700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":1.204,"15min":1.204},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.70699999999999996},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.995,"15min":1.0549999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.087},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.58499999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.97299999999999998,"5min":1.2769999999999999,"15min":1.278},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64900000000000002,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.996,"15min":1.294},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64400000000000002,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.141,"15min":1.141},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.876,"5min":0.999,"15min":1.254},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71899999999999997,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.042,"15min":1.337},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.55300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.248,"15min":1.248},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:01.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.141+0000 7f833dd9d700 1 Processor -- start 2023-10-06T05:21:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.141+0000 7f833dd9d700 1 -- start start 2023-10-06T05:21:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.142+0000 7f833dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f83380ccfc0 0x7f83380cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.142+0000 7f833dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380cc750 0x7f83380b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.142+0000 7f833dd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83380cbd80 con 0x7f83380cf3b0 2023-10-06T05:21:02.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.142+0000 7f833dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83380af5f0 con 0x7f83380ccfc0 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.142+0000 7f833dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83380af8d0 con 0x7f83380cc750 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f83380ccfc0 0x7f83380cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f83380ccfc0 0x7f83380cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43974/0 (socket says 172.21.15.33:43974) 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 -- 172.21.15.33:0/3513996672 learned_addr learned my addr 172.21.15.33:0/3513996672 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f8337fff700 1 --1- 172.21.15.33:0/3513996672 >> v1:172.21.15.33:6789/0 conn(0x7f83380cf3b0 0x7f83381594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56058/0 (socket says 172.21.15.33:56058) 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833d59c700 1 --2- 172.21.15.33:0/3513996672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380cc750 0x7f83380b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 -- 172.21.15.33:0/3513996672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380cc750 msgr2=0x7f83380b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 --2- 172.21.15.33:0/3513996672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380cc750 0x7f83380b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 -- 172.21.15.33:0/3513996672 >> v1:172.21.15.33:6789/0 conn(0x7f83380cf3b0 legacy=0x7f83381594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 -- 172.21.15.33:0/3513996672 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380afbb0 con 0x7f83380ccfc0 2023-10-06T05:21:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.143+0000 7f833cd9b700 1 --2- 172.21.15.33:0/3513996672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380ccfc0 0x7f83380cc210 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f832c0099e0 tx=0x7f832c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2051108a52e60a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f83377fe700 1 -- 172.21.15.33:0/3513996672 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00b820 con 0x7f83380ccfc0 2023-10-06T05:21:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f83377fe700 1 -- 172.21.15.33:0/3513996672 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c00b980 con 0x7f83380ccfc0 2023-10-06T05:21:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f83377fe700 1 -- 172.21.15.33:0/3513996672 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00bd20 con 0x7f83380ccfc0 2023-10-06T05:21:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 -- 172.21.15.33:0/3513996672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380ccfc0 msgr2=0x7f83380cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 --2- 172.21.15.33:0/3513996672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380ccfc0 0x7f83380cc210 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f832c0099e0 tx=0x7f832c009250 comp rx=0 tx=0).stop 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 -- 172.21.15.33:0/3513996672 shutdown_connections 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 --2- 172.21.15.33:0/3513996672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380ccfc0 0x7f83380cc210 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 --2- 172.21.15.33:0/3513996672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380cc750 0x7f83380b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 -- 172.21.15.33:0/3513996672 >> 172.21.15.33:0/3513996672 conn(0x7f833806cff0 msgr2=0x7f833806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 -- 172.21.15.33:0/3513996672 shutdown_connections 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.144+0000 7f833dd9d700 1 -- 172.21.15.33:0/3513996672 wait complete. 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.146+0000 7f833dd9d700 1 Processor -- start 2023-10-06T05:21:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.146+0000 7f833dd9d700 1 -- start start 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 0x7f83380b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83380ccfc0 0x7f83380b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380b6120 0x7f83380b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83380b3d20 con 0x7f83380cc750 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83380b1fe0 con 0x7f83380ccfc0 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83380b13d0 con 0x7f83380b6120 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 0x7f83380b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 0x7f83380b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43978/0 (socket says 172.21.15.33:43978) 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 -- 172.21.15.33:0/584155344 learned_addr learned my addr 172.21.15.33:0/584155344 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f8337fff700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83380ccfc0 0x7f83380b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833d59c700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380b6120 0x7f83380b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 -- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380b6120 msgr2=0x7f83380b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380b6120 0x7f83380b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 -- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83380ccfc0 msgr2=0x7f83380b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83380ccfc0 0x7f83380b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f833cd9b700 1 -- 172.21.15.33:0/584155344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0094f0 con 0x7f83380cc750 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.147+0000 7f8337fff700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83380ccfc0 0x7f83380b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.148+0000 7f833cd9b700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 0x7f83380b5be0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f833806be20 tx=0x7f832c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.148+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c011e70 con 0x7f83380cc750 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.148+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c024430 con 0x7f83380cc750 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.148+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0247f0 con 0x7f83380cc750 2023-10-06T05:21:02.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.148+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83380b16b0 con 0x7f83380cc750 2023-10-06T05:21:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.148+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83380b1c70 con 0x7f83380cc750 2023-10-06T05:21:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.149+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833804c380 con 0x7f83380cc750 2023-10-06T05:21:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.150+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f832c01a420 con 0x7f83380cc750 2023-10-06T05:21:02.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.151+0000 7f8334ff9700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f831c066f30 0x7f831c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.151+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f832c085bc0 con 0x7f83380cc750 2023-10-06T05:21:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.151+0000 7f8337fff700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f831c066f30 0x7f831c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.152+0000 7f8337fff700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f831c066f30 0x7f831c0693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8324007fd0 tx=0x7f8324007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.155+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f832c052cb0 con 0x7f83380cc750 2023-10-06T05:21:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.354+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8338159bb0 con 0x7f831c066f30 2023-10-06T05:21:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.364+0000 7f8334ff9700 1 -- 172.21.15.33:0/584155344 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307738 (secure 0 0 0) 0x7f8338159bb0 con 0x7f831c066f30 2023-10-06T05:21:02.370 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f831c066f30 msgr2=0x7f831c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f831c066f30 0x7f831c0693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8324007fd0 tx=0x7f8324007e10 comp rx=0 tx=0).stop 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 msgr2=0x7f83380b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 0x7f83380b5be0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f833806be20 tx=0x7f832c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 shutdown_connections 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f831c066f30 0x7f831c0693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83380b6120 0x7f83380b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83380cc750 0x7f83380b5be0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 --2- 172.21.15.33:0/584155344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83380ccfc0 0x7f83380b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 >> 172.21.15.33:0/584155344 conn(0x7f833806cff0 msgr2=0x7f83381588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 shutdown_connections 2023-10-06T05:21:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.368+0000 7f833dd9d700 1 -- 172.21.15.33:0/584155344 wait complete. 2023-10-06T05:21:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:21:02.389 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2023-10-06T05:21:01.913696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420584,"kb_used_data":9512,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752554136,"statfs":{"total":773094113280,"available":770615435264,"internally_reserved":0,"allocated":9740288,"data_stored":5025216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001814"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047746+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054711+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047774+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061615+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059581+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059292+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:03.594147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048133+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:39.064497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052728+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052245+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:35.106422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053309+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.455963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051588+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:37.245927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.060583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:07.387943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056543+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.063672+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058649+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:58.648941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059036+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:10.851643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048750+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054442+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059967+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:18.831632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058550+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056058+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054085+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:24.396610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047445+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055287+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:59.686854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:04.727723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051706+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:10.346856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059449+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:08.100764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047908+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056488+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:17.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057141+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:34.761680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056494+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:42.458155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069083+0000","last_change":"2023-10-06T05:20:56.069083+0000","last_active":"2023-10-06T05:20:56.069083+0000","last_peered":"2023-10-06T05:20:56.069083+0000","last_clean":"2023-10-06T05:20:56.069083+0000","last_became_active":"2023-10-06T05:20:56.068775+0000","last_became_peered":"2023-10-06T05:20:56.068775+0000","last_unstale":"2023-10-06T05:20:56.069083+0000","last_undegraded":"2023-10-06T05:20:56.069083+0000","last_fullsized":"2023-10-06T05:20:56.069083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:43.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056509+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:23.982186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.061418+0000","last_change":"2023-10-06T05:20:56.061418+0000","last_active":"2023-10-06T05:20:56.061418+0000","last_peered":"2023-10-06T05:20:56.061418+0000","last_clean":"2023-10-06T05:20:56.061418+0000","last_became_active":"2023-10-06T05:20:56.060531+0000","last_became_peered":"2023-10-06T05:20:56.060531+0000","last_unstale":"2023-10-06T05:20:56.061418+0000","last_undegraded":"2023-10-06T05:20:56.061418+0000","last_fullsized":"2023-10-06T05:20:56.061418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:47.802684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048418+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063062+0000","last_change":"2023-10-06T05:20:56.063062+0000","last_active":"2023-10-06T05:20:56.063062+0000","last_peered":"2023-10-06T05:20:56.063062+0000","last_clean":"2023-10-06T05:20:56.063062+0000","last_became_active":"2023-10-06T05:20:56.062410+0000","last_became_peered":"2023-10-06T05:20:56.062410+0000","last_unstale":"2023-10-06T05:20:56.063062+0000","last_undegraded":"2023-10-06T05:20:56.063062+0000","last_fullsized":"2023-10-06T05:20:56.063062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:49.619842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057087+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:37.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062842+0000","last_change":"2023-10-06T05:20:56.062842+0000","last_active":"2023-10-06T05:20:56.062842+0000","last_peered":"2023-10-06T05:20:56.062842+0000","last_clean":"2023-10-06T05:20:56.062842+0000","last_became_active":"2023-10-06T05:20:56.062590+0000","last_became_peered":"2023-10-06T05:20:56.062590+0000","last_unstale":"2023-10-06T05:20:56.062842+0000","last_undegraded":"2023-10-06T05:20:56.062842+0000","last_fullsized":"2023-10-06T05:20:56.062842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:58.961459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057855+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.845343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063129+0000","last_change":"2023-10-06T05:20:56.063129+0000","last_active":"2023-10-06T05:20:56.063129+0000","last_peered":"2023-10-06T05:20:56.063129+0000","last_clean":"2023-10-06T05:20:56.063129+0000","last_became_active":"2023-10-06T05:20:56.062230+0000","last_became_peered":"2023-10-06T05:20:56.062230+0000","last_unstale":"2023-10-06T05:20:56.063129+0000","last_undegraded":"2023-10-06T05:20:56.063129+0000","last_fullsized":"2023-10-06T05:20:56.063129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:58.928023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066379+0000","last_change":"2023-10-06T05:20:56.066379+0000","last_active":"2023-10-06T05:20:56.066379+0000","last_peered":"2023-10-06T05:20:56.066379+0000","last_clean":"2023-10-06T05:20:56.066379+0000","last_became_active":"2023-10-06T05:20:56.064912+0000","last_became_peered":"2023-10-06T05:20:56.064912+0000","last_unstale":"2023-10-06T05:20:56.066379+0000","last_undegraded":"2023-10-06T05:20:56.066379+0000","last_fullsized":"2023-10-06T05:20:56.066379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:23.256874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058738+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:27.536202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063667+0000","last_change":"2023-10-06T05:20:56.063667+0000","last_active":"2023-10-06T05:20:56.063667+0000","last_peered":"2023-10-06T05:20:56.063667+0000","last_clean":"2023-10-06T05:20:56.063667+0000","last_became_active":"2023-10-06T05:20:56.063346+0000","last_became_peered":"2023-10-06T05:20:56.063346+0000","last_unstale":"2023-10-06T05:20:56.063667+0000","last_undegraded":"2023-10-06T05:20:56.063667+0000","last_fullsized":"2023-10-06T05:20:56.063667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058683+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:36.533849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.068185+0000","last_change":"2023-10-06T05:20:56.068185+0000","last_active":"2023-10-06T05:20:56.068185+0000","last_peered":"2023-10-06T05:20:56.068185+0000","last_clean":"2023-10-06T05:20:56.068185+0000","last_became_active":"2023-10-06T05:20:56.067889+0000","last_became_peered":"2023-10-06T05:20:56.067889+0000","last_unstale":"2023-10-06T05:20:56.068185+0000","last_undegraded":"2023-10-06T05:20:56.068185+0000","last_fullsized":"2023-10-06T05:20:56.068185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052435+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064663+0000","last_change":"2023-10-06T05:20:56.064663+0000","last_active":"2023-10-06T05:20:56.064663+0000","last_peered":"2023-10-06T05:20:56.064663+0000","last_clean":"2023-10-06T05:20:56.064663+0000","last_became_active":"2023-10-06T05:20:56.064382+0000","last_became_peered":"2023-10-06T05:20:56.064382+0000","last_unstale":"2023-10-06T05:20:56.064663+0000","last_undegraded":"2023-10-06T05:20:56.064663+0000","last_fullsized":"2023-10-06T05:20:56.064663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:07.091803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047994+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062871+0000","last_change":"2023-10-06T05:20:56.062871+0000","last_active":"2023-10-06T05:20:56.062871+0000","last_peered":"2023-10-06T05:20:56.062871+0000","last_clean":"2023-10-06T05:20:56.062871+0000","last_became_active":"2023-10-06T05:20:56.062606+0000","last_became_peered":"2023-10-06T05:20:56.062606+0000","last_unstale":"2023-10-06T05:20:56.062871+0000","last_undegraded":"2023-10-06T05:20:56.062871+0000","last_fullsized":"2023-10-06T05:20:56.062871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048278+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069433+0000","last_change":"2023-10-06T05:20:56.069433+0000","last_active":"2023-10-06T05:20:56.069433+0000","last_peered":"2023-10-06T05:20:56.069433+0000","last_clean":"2023-10-06T05:20:56.069433+0000","last_became_active":"2023-10-06T05:20:56.069152+0000","last_became_peered":"2023-10-06T05:20:56.069152+0000","last_unstale":"2023-10-06T05:20:56.069433+0000","last_undegraded":"2023-10-06T05:20:56.069433+0000","last_fullsized":"2023-10-06T05:20:56.069433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.728922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057259+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:12.235739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066398+0000","last_change":"2023-10-06T05:20:56.066398+0000","last_active":"2023-10-06T05:20:56.066398+0000","last_peered":"2023-10-06T05:20:56.066398+0000","last_clean":"2023-10-06T05:20:56.066398+0000","last_became_active":"2023-10-06T05:20:56.066051+0000","last_became_peered":"2023-10-06T05:20:56.066051+0000","last_unstale":"2023-10-06T05:20:56.066398+0000","last_undegraded":"2023-10-06T05:20:56.066398+0000","last_fullsized":"2023-10-06T05:20:56.066398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:26.538710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056633+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:48.550385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060068+0000","last_change":"2023-10-06T05:20:56.060068+0000","last_active":"2023-10-06T05:20:56.060068+0000","last_peered":"2023-10-06T05:20:56.060068+0000","last_clean":"2023-10-06T05:20:56.060068+0000","last_became_active":"2023-10-06T05:20:56.059765+0000","last_became_peered":"2023-10-06T05:20:56.059765+0000","last_unstale":"2023-10-06T05:20:56.060068+0000","last_undegraded":"2023-10-06T05:20:56.060068+0000","last_fullsized":"2023-10-06T05:20:56.060068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:56.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052031+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062607+0000","last_change":"2023-10-06T05:20:56.062607+0000","last_active":"2023-10-06T05:20:56.062607+0000","last_peered":"2023-10-06T05:20:56.062607+0000","last_clean":"2023-10-06T05:20:56.062607+0000","last_became_active":"2023-10-06T05:20:56.062344+0000","last_became_peered":"2023-10-06T05:20:56.062344+0000","last_unstale":"2023-10-06T05:20:56.062607+0000","last_undegraded":"2023-10-06T05:20:56.062607+0000","last_fullsized":"2023-10-06T05:20:56.062607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.897142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051968+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:14.557678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067504+0000","last_change":"2023-10-06T05:20:56.067504+0000","last_active":"2023-10-06T05:20:56.067504+0000","last_peered":"2023-10-06T05:20:56.067504+0000","last_clean":"2023-10-06T05:20:56.067504+0000","last_became_active":"2023-10-06T05:20:56.066985+0000","last_became_peered":"2023-10-06T05:20:56.066985+0000","last_unstale":"2023-10-06T05:20:56.067504+0000","last_undegraded":"2023-10-06T05:20:56.067504+0000","last_fullsized":"2023-10-06T05:20:56.067504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:43.665749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051195+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:39.400959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063906+0000","last_change":"2023-10-06T05:20:56.063906+0000","last_active":"2023-10-06T05:20:56.063906+0000","last_peered":"2023-10-06T05:20:56.063906+0000","last_clean":"2023-10-06T05:20:56.063906+0000","last_became_active":"2023-10-06T05:20:56.063625+0000","last_became_peered":"2023-10-06T05:20:56.063625+0000","last_unstale":"2023-10-06T05:20:56.063906+0000","last_undegraded":"2023-10-06T05:20:56.063906+0000","last_fullsized":"2023-10-06T05:20:56.063906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:07.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054747+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:55.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060664+0000","last_change":"2023-10-06T05:20:56.060664+0000","last_active":"2023-10-06T05:20:56.060664+0000","last_peered":"2023-10-06T05:20:56.060664+0000","last_clean":"2023-10-06T05:20:56.060664+0000","last_became_active":"2023-10-06T05:20:56.059179+0000","last_became_peered":"2023-10-06T05:20:56.059179+0000","last_unstale":"2023-10-06T05:20:56.060664+0000","last_undegraded":"2023-10-06T05:20:56.060664+0000","last_fullsized":"2023-10-06T05:20:56.060664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.078805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050160+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069531+0000","last_change":"2023-10-06T05:20:56.069531+0000","last_active":"2023-10-06T05:20:56.069531+0000","last_peered":"2023-10-06T05:20:56.069531+0000","last_clean":"2023-10-06T05:20:56.069531+0000","last_became_active":"2023-10-06T05:20:56.068428+0000","last_became_peered":"2023-10-06T05:20:56.068428+0000","last_unstale":"2023-10-06T05:20:56.069531+0000","last_undegraded":"2023-10-06T05:20:56.069531+0000","last_fullsized":"2023-10-06T05:20:56.069531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050790+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069310+0000","last_change":"2023-10-06T05:20:56.069310+0000","last_active":"2023-10-06T05:20:56.069310+0000","last_peered":"2023-10-06T05:20:56.069310+0000","last_clean":"2023-10-06T05:20:56.069310+0000","last_became_active":"2023-10-06T05:20:56.069031+0000","last_became_peered":"2023-10-06T05:20:56.069031+0000","last_unstale":"2023-10-06T05:20:56.069310+0000","last_undegraded":"2023-10-06T05:20:56.069310+0000","last_fullsized":"2023-10-06T05:20:56.069310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:58.677893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054813+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:44.878348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066147+0000","last_change":"2023-10-06T05:20:56.066147+0000","last_active":"2023-10-06T05:20:56.066147+0000","last_peered":"2023-10-06T05:20:56.066147+0000","last_clean":"2023-10-06T05:20:56.066147+0000","last_became_active":"2023-10-06T05:20:56.065618+0000","last_became_peered":"2023-10-06T05:20:56.065618+0000","last_unstale":"2023-10-06T05:20:56.066147+0000","last_undegraded":"2023-10-06T05:20:56.066147+0000","last_fullsized":"2023-10-06T05:20:56.066147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:33.934156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051253+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:59.973383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062131+0000","last_change":"2023-10-06T05:20:56.062131+0000","last_active":"2023-10-06T05:20:56.062131+0000","last_peered":"2023-10-06T05:20:56.062131+0000","last_clean":"2023-10-06T05:20:56.062131+0000","last_became_active":"2023-10-06T05:20:56.061821+0000","last_became_peered":"2023-10-06T05:20:56.061821+0000","last_unstale":"2023-10-06T05:20:56.062131+0000","last_undegraded":"2023-10-06T05:20:56.062131+0000","last_fullsized":"2023-10-06T05:20:56.062131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:33.997234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047924+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067790+0000","last_change":"2023-10-06T05:20:56.067790+0000","last_active":"2023-10-06T05:20:56.067790+0000","last_peered":"2023-10-06T05:20:56.067790+0000","last_clean":"2023-10-06T05:20:56.067790+0000","last_became_active":"2023-10-06T05:20:56.067491+0000","last_became_peered":"2023-10-06T05:20:56.067491+0000","last_unstale":"2023-10-06T05:20:56.067790+0000","last_undegraded":"2023-10-06T05:20:56.067790+0000","last_fullsized":"2023-10-06T05:20:56.067790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:06.378982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058270+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:05.605433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.065948+0000","last_change":"2023-10-06T05:20:56.065948+0000","last_active":"2023-10-06T05:20:56.065948+0000","last_peered":"2023-10-06T05:20:56.065948+0000","last_clean":"2023-10-06T05:20:56.065948+0000","last_became_active":"2023-10-06T05:20:56.065640+0000","last_became_peered":"2023-10-06T05:20:56.065640+0000","last_unstale":"2023-10-06T05:20:56.065948+0000","last_undegraded":"2023-10-06T05:20:56.065948+0000","last_fullsized":"2023-10-06T05:20:56.065948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:17.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.049531+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063353+0000","last_change":"2023-10-06T05:20:56.063353+0000","last_active":"2023-10-06T05:20:56.063353+0000","last_peered":"2023-10-06T05:20:56.063353+0000","last_clean":"2023-10-06T05:20:56.063353+0000","last_became_active":"2023-10-06T05:20:56.063086+0000","last_became_peered":"2023-10-06T05:20:56.063086+0000","last_unstale":"2023-10-06T05:20:56.063353+0000","last_undegraded":"2023-10-06T05:20:56.063353+0000","last_fullsized":"2023-10-06T05:20:56.063353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:59.973368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052028+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066899+0000","last_change":"2023-10-06T05:20:56.066899+0000","last_active":"2023-10-06T05:20:56.066899+0000","last_peered":"2023-10-06T05:20:56.066899+0000","last_clean":"2023-10-06T05:20:56.066899+0000","last_became_active":"2023-10-06T05:20:56.066645+0000","last_became_peered":"2023-10-06T05:20:56.066645+0000","last_unstale":"2023-10-06T05:20:56.066899+0000","last_undegraded":"2023-10-06T05:20:56.066899+0000","last_fullsized":"2023-10-06T05:20:56.066899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:38.055535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044210+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:20:55.044210+0000","last_peered":"2023-10-06T05:20:55.044210+0000","last_clean":"2023-10-06T05:20:55.044210+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:20:55.044210+0000","last_undegraded":"2023-10-06T05:20:55.044210+0000","last_fullsized":"2023-10-06T05:20:55.044210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053542+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064164+0000","last_change":"2023-10-06T05:20:56.064164+0000","last_active":"2023-10-06T05:20:56.064164+0000","last_peered":"2023-10-06T05:20:56.064164+0000","last_clean":"2023-10-06T05:20:56.064164+0000","last_became_active":"2023-10-06T05:20:56.063907+0000","last_became_peered":"2023-10-06T05:20:56.063907+0000","last_unstale":"2023-10-06T05:20:56.064164+0000","last_undegraded":"2023-10-06T05:20:56.064164+0000","last_fullsized":"2023-10-06T05:20:56.064164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:16.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047296+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:20:55.047296+0000","last_peered":"2023-10-06T05:20:55.047296+0000","last_clean":"2023-10-06T05:20:55.047296+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:20:55.047296+0000","last_undegraded":"2023-10-06T05:20:55.047296+0000","last_fullsized":"2023-10-06T05:20:55.047296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053376+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:12.313245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062639+0000","last_change":"2023-10-06T05:20:56.062639+0000","last_active":"2023-10-06T05:20:56.062639+0000","last_peered":"2023-10-06T05:20:56.062639+0000","last_clean":"2023-10-06T05:20:56.062639+0000","last_became_active":"2023-10-06T05:20:56.062307+0000","last_became_peered":"2023-10-06T05:20:56.062307+0000","last_unstale":"2023-10-06T05:20:56.062639+0000","last_undegraded":"2023-10-06T05:20:56.062639+0000","last_fullsized":"2023-10-06T05:20:56.062639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.040639+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:20:55.040639+0000","last_peered":"2023-10-06T05:20:55.040639+0000","last_clean":"2023-10-06T05:20:55.040639+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:20:55.040639+0000","last_undegraded":"2023-10-06T05:20:55.040639+0000","last_fullsized":"2023-10-06T05:20:55.040639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048052+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067383+0000","last_change":"2023-10-06T05:20:56.067383+0000","last_active":"2023-10-06T05:20:56.067383+0000","last_peered":"2023-10-06T05:20:56.067383+0000","last_clean":"2023-10-06T05:20:56.067383+0000","last_became_active":"2023-10-06T05:20:56.067105+0000","last_became_peered":"2023-10-06T05:20:56.067105+0000","last_unstale":"2023-10-06T05:20:56.067383+0000","last_undegraded":"2023-10-06T05:20:56.067383+0000","last_fullsized":"2023-10-06T05:20:56.067383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:58.649017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047360+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:20:55.047360+0000","last_peered":"2023-10-06T05:20:55.047360+0000","last_clean":"2023-10-06T05:20:55.047360+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:20:55.047360+0000","last_undegraded":"2023-10-06T05:20:55.047360+0000","last_fullsized":"2023-10-06T05:20:55.047360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044900+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:20:55.044900+0000","last_peered":"2023-10-06T05:20:55.044900+0000","last_clean":"2023-10-06T05:20:55.044900+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:20:55.044900+0000","last_undegraded":"2023-10-06T05:20:55.044900+0000","last_fullsized":"2023-10-06T05:20:55.044900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054988+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:06.913609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066067+0000","last_change":"2023-10-06T05:20:56.066067+0000","last_active":"2023-10-06T05:20:56.066067+0000","last_peered":"2023-10-06T05:20:56.066067+0000","last_clean":"2023-10-06T05:20:56.066067+0000","last_became_active":"2023-10-06T05:20:56.065696+0000","last_became_peered":"2023-10-06T05:20:56.065696+0000","last_unstale":"2023-10-06T05:20:56.066067+0000","last_undegraded":"2023-10-06T05:20:56.066067+0000","last_fullsized":"2023-10-06T05:20:56.066067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:03.468905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044461+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:20:55.044461+0000","last_peered":"2023-10-06T05:20:55.044461+0000","last_clean":"2023-10-06T05:20:55.044461+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:20:55.044461+0000","last_undegraded":"2023-10-06T05:20:55.044461+0000","last_fullsized":"2023-10-06T05:20:55.044461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053799+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:20.780861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063757+0000","last_change":"2023-10-06T05:20:56.063757+0000","last_active":"2023-10-06T05:20:56.063757+0000","last_peered":"2023-10-06T05:20:56.063757+0000","last_clean":"2023-10-06T05:20:56.063757+0000","last_became_active":"2023-10-06T05:20:56.063497+0000","last_became_peered":"2023-10-06T05:20:56.063497+0000","last_unstale":"2023-10-06T05:20:56.063757+0000","last_undegraded":"2023-10-06T05:20:56.063757+0000","last_fullsized":"2023-10-06T05:20:56.063757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:07.370987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044759+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:20:55.044759+0000","last_peered":"2023-10-06T05:20:55.044759+0000","last_clean":"2023-10-06T05:20:55.044759+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:20:55.044759+0000","last_undegraded":"2023-10-06T05:20:55.044759+0000","last_fullsized":"2023-10-06T05:20:55.044759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053788+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067795+0000","last_change":"2023-10-06T05:20:56.067795+0000","last_active":"2023-10-06T05:20:56.067795+0000","last_peered":"2023-10-06T05:20:56.067795+0000","last_clean":"2023-10-06T05:20:56.067795+0000","last_became_active":"2023-10-06T05:20:56.067515+0000","last_became_peered":"2023-10-06T05:20:56.067515+0000","last_unstale":"2023-10-06T05:20:56.067795+0000","last_undegraded":"2023-10-06T05:20:56.067795+0000","last_fullsized":"2023-10-06T05:20:56.067795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:17.822671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044296+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:20:55.044296+0000","last_peered":"2023-10-06T05:20:55.044296+0000","last_clean":"2023-10-06T05:20:55.044296+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:20:55.044296+0000","last_undegraded":"2023-10-06T05:20:55.044296+0000","last_fullsized":"2023-10-06T05:20:55.044296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051963+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:59.430802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066251+0000","last_change":"2023-10-06T05:20:56.066251+0000","last_active":"2023-10-06T05:20:56.066251+0000","last_peered":"2023-10-06T05:20:56.066251+0000","last_clean":"2023-10-06T05:20:56.066251+0000","last_became_active":"2023-10-06T05:20:56.065923+0000","last_became_peered":"2023-10-06T05:20:56.065923+0000","last_unstale":"2023-10-06T05:20:56.066251+0000","last_undegraded":"2023-10-06T05:20:56.066251+0000","last_fullsized":"2023-10-06T05:20:56.066251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:09.868032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.042122+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:20:55.042122+0000","last_peered":"2023-10-06T05:20:55.042122+0000","last_clean":"2023-10-06T05:20:55.042122+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:20:55.042122+0000","last_undegraded":"2023-10-06T05:20:55.042122+0000","last_fullsized":"2023-10-06T05:20:55.042122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99099999999999999,"5min":1.169,"15min":1.169},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.628},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.268,"15min":1.268},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.075,"15min":1.075},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.68799999999999994},"min":{"1min":0.57199999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.995,"5min":1.093,"15min":1.093},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":1.22,"15min":1.22},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0349999999999999,"15min":1.079},"last":0.624},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":1.117,"15min":1.202},"last":0.85299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.622},"min":{"1min":0.47199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":1.258,"15min":1.258},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.64500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97599999999999998,"5min":1.2050000000000001,"15min":1.407},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.92500000000000004,"5min":1.139,"15min":1.139},"last":0.747},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.24,"15min":1.3300000000000001},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.034},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.1599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.76200000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91500000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.67700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.95699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82899999999999996,"5min":0.97799999999999998,"15min":0.998},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.0760000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68999999999999995,"15min":0.77900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.77900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":0.97999999999999998,"15min":1.038},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73099999999999998,"15min":0.84399999999999997},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.442,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60099999999999998,"5min":0.73099999999999998,"15min":2.7349999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48499999999999999,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54300000000000004,"5min":0.71799999999999997,"15min":2.6520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53600000000000003,"15min":0.61099999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.85299999999999998,"15min":2.7639999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53400000000000003,"15min":0.61399999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.82699999999999996,"15min":2.8370000000000002},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.50700000000000001},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.125,"15min":1.2929999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.751,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93200000000000005,"5min":1.0289999999999999,"15min":1.349},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.82899999999999996},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":1.1419999999999999,"15min":1.669},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.86399999999999999},"min":{"1min":0.66800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91900000000000004,"5min":1.1779999999999999,"15min":1.6040000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88700000000000001},"min":{"1min":0.61499999999999999,"5min":0.503,"15min":0.503},"max":{"1min":0.96999999999999997,"5min":1.165,"15min":1.573},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":1.0780000000000001,"15min":1.528},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.75},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46200000000000002,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64600000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73199999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.62},{"interface":"front","average":{"1min":0.497,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.434}]},{"osd":3,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.879},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.75700000000000001},"min":{"1min":0.65200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0069999999999999,"5min":1.218,"15min":1.218},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.749},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.79400000000000004},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.05,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.63700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.46200000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48699999999999999,"15min":0.47399999999999998},"min":{"1min":0.42599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":1.0329999999999999,"5min":1.077,"15min":1.1379999999999999},"last":0.88},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.47499999999999998},"max":{"1min":0.98699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.495},"max":{"1min":1.04,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.319},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":0.79200000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.103,"5min":1.103,"15min":1.141},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52100000000000002},"max":{"1min":1.0609999999999999,"5min":1.1000000000000001,"15min":1.282},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.68100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.3520000000000001},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.70799999999999996,"5min":1.0720000000000001,"15min":2.0950000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":1.772},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.78100000000000003,"15min":1.913},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.66800000000000004,"5min":0.84399999999999997,"15min":2.0089999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55400000000000005,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.82399999999999995,"15min":1.954},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0469999999999999,"5min":1.2030000000000001,"15min":1.282},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.0669999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1399999999999999,"5min":1.2250000000000001,"15min":1.3009999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.1910000000000001,"15min":1.343},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3380000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82599999999999996,"15min":0.85999999999999999},"min":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56799999999999995},"max":{"1min":1.1140000000000001,"5min":1.206,"15min":1.3640000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.68700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1539999999999999,"5min":1.2090000000000001,"15min":1.214},"last":0.746}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.745,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76600000000000001,"15min":0.76000000000000001},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0269999999999999,"15min":1.169},"last":0.755}]},{"osd":1,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.042,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.117},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.002},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.196},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63500000000000001,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.8740000000000001},"last":0.5},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66100000000000003,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.375},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98799999999999999,"15min":1.004},"last":0.44},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95699999999999996,"5min":1.1639999999999999,"15min":1.9390000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71199999999999997,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93300000000000005,"5min":1.0209999999999999,"15min":1.466},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.83999999999999997},"min":{"1min":0.626,"5min":0.499,"15min":0.499},"max":{"1min":0.91800000000000004,"5min":1.1930000000000001,"15min":1.218},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90800000000000003,"5min":1.165,"15min":1.165},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98999999999999999,"5min":1.016,"15min":1.1890000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.69299999999999995},"min":{"1min":0.59899999999999998,"5min":0.45100000000000001,"15min":0.438},"max":{"1min":0.88200000000000001,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.038,"15min":1.038},"last":0.88700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":1.204,"15min":1.204},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.70699999999999996},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.995,"15min":1.0549999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.087},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.58499999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.97299999999999998,"5min":1.2769999999999999,"15min":1.278},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64900000000000002,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.996,"15min":1.294},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64400000000000002,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.141,"15min":1.141},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.876,"5min":0.999,"15min":1.254},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71899999999999997,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.042,"15min":1.337},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.55300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.248,"15min":1.248},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:02.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2023-10-06T05:21:02.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.047746+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:02.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2023-10-06T05:21:02.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.054711+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:02.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2023-10-06T05:21:02.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.047774+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:02.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2023-10-06T05:21:02.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.061615+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2023-10-06T05:21:02.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.059581+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.059292+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:53:03.594147+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.048133+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.061078+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:40:39.064497+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.052728+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:24:29.341816+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2023-10-06T05:21:02.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.052245+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:29:35.106422+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:02.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2023-10-06T05:21:02.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053309+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:59:57.455963+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2023-10-06T05:21:02.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.051588+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:08:37.245927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:02.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2023-10-06T05:21:02.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.060583+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:35:07.387943+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2023-10-06T05:21:02.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.056543+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2023-10-06T05:21:02.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.063672+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.058649+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:29:58.648941+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.059036+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:22:10.851643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.048750+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2023-10-06T05:21:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.054442+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.059967+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:12:18.831632+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.058550+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.056058+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.054085+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:01:24.396610+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:02.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.047445+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.055287+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:38:59.686854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053078+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:18:04.727723+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.051706+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:58:10.346856+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2023-10-06T05:21:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.059449+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:38:08.100764+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.047908+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.055583+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.056488+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:29:17.590883+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2023-10-06T05:21:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.057141+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:33:34.761680+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.056494+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:12:42.458155+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.056509+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:49:23.982186+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.048418+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.057087+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:21:37.095202+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2023-10-06T05:21:02.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.057855+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:15:39.845343+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053583+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:22:58.928023+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.058738+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:31:27.536202+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.058683+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:00:36.533849+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2023-10-06T05:21:02.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.052435+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.047994+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.048278+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.057259+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:43:12.235739+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.056633+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:40:48.550385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:02.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.052031+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.051968+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:29:14.557678+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.051195+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:09:39.400959+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.054747+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:14:55.677080+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2023-10-06T05:21:02.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.050160+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.050790+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.054813+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:18:44.878348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.051253+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:08:59.973383+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.047924+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.058270+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:49:05.605433+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.049531+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.052028+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053542+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-10-06T05:21:02.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053376+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:26:12.313245+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:02.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-10-06T05:21:02.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.048052+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:02.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2023-10-06T05:21:02.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.054988+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:47:06.913609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2023-10-06T05:21:02.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053799+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:12:20.780861+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:02.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2023-10-06T05:21:02.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.053788+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:02.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2023-10-06T05:21:02.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:20:56.037124+0000', 'last_change': '2023-10-06T05:20:56.051963+0000', 'last_active': '2023-10-06T05:20:56.037124+0000', 'last_peered': '2023-10-06T05:20:56.037124+0000', 'last_clean': '2023-10-06T05:20:56.037124+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:20:56.037124+0000', 'last_undegraded': '2023-10-06T05:20:56.037124+0000', 'last_fullsized': '2023-10-06T05:20:56.037124+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'last_clean_scrub_stamp': '2023-10-06T05:20:56.037124+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:52:59.430802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:02.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T05:21:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.613+0000 7f811759e700 1 Processor -- start 2023-10-06T05:21:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.613+0000 7f811759e700 1 -- start start 2023-10-06T05:21:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.614+0000 7f811759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8118155af0 0x7f811806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.614+0000 7f811759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811806ab60 con 0x7f81180c8b50 2023-10-06T05:21:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811806b7e0 con 0x7f8118155af0 2023-10-06T05:21:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811806ae40 con 0x7f811806a6a0 2023-10-06T05:21:02.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8118155af0 0x7f811806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f8116d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81180c8b50 0x7f81181598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56074/0 (socket says 172.21.15.33:56074) 2023-10-06T05:21:02.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f8116d9d700 1 -- 172.21.15.33:0/2188412119 learned_addr learned my addr 172.21.15.33:0/2188412119 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f8115d9b700 1 --2- 172.21.15.33:0/2188412119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811659c700 1 -- 172.21.15.33:0/2188412119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 msgr2=0x7f81180c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811659c700 1 --2- 172.21.15.33:0/2188412119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811659c700 1 -- 172.21.15.33:0/2188412119 >> v1:172.21.15.33:6789/0 conn(0x7f81180c8b50 legacy=0x7f81181598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811659c700 1 -- 172.21.15.33:0/2188412119 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811806b120 con 0x7f8118155af0 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.615+0000 7f811559a700 1 -- 172.21.15.33:0/2188412119 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866533325 0 0) 0x7f811806ab60 con 0x7f81180c8b50 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.616+0000 7f811659c700 1 --2- 172.21.15.33:0/2188412119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8118155af0 0x7f811806a160 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f810800aa90 tx=0x7f810800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ffe7aa8388f6e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.616+0000 7f811559a700 1 -- 172.21.15.33:0/2188412119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800ca40 con 0x7f8118155af0 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.616+0000 7f811559a700 1 -- 172.21.15.33:0/2188412119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800cba0 con 0x7f8118155af0 2023-10-06T05:21:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.616+0000 7f811559a700 1 -- 172.21.15.33:0/2188412119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108018400 con 0x7f8118155af0 2023-10-06T05:21:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.616+0000 7f811759e700 1 -- 172.21.15.33:0/2188412119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8118155af0 msgr2=0x7f811806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.616+0000 7f811759e700 1 --2- 172.21.15.33:0/2188412119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8118155af0 0x7f811806a160 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f810800aa90 tx=0x7f810800a330 comp rx=0 tx=0).stop 2023-10-06T05:21:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.617+0000 7f811759e700 1 -- 172.21.15.33:0/2188412119 shutdown_connections 2023-10-06T05:21:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.617+0000 7f811759e700 1 --2- 172.21.15.33:0/2188412119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8118155af0 0x7f811806a160 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.617+0000 7f811759e700 1 --2- 172.21.15.33:0/2188412119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.617+0000 7f811759e700 1 -- 172.21.15.33:0/2188412119 >> 172.21.15.33:0/2188412119 conn(0x7f811806d030 msgr2=0x7f811806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.617+0000 7f811759e700 1 -- 172.21.15.33:0/2188412119 shutdown_connections 2023-10-06T05:21:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.617+0000 7f811759e700 1 -- 172.21.15.33:0/2188412119 wait complete. 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.618+0000 7f811759e700 1 Processor -- start 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.618+0000 7f811759e700 1 -- start start 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.619+0000 7f811759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.619+0000 7f811759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 0x7f811816a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.619+0000 7f811759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8118155af0 0x7f8118153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.619+0000 7f811759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81180ae200 con 0x7f81180c8b50 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.619+0000 7f811759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811816acb0 con 0x7f8118155af0 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.619+0000 7f811759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81180ac180 con 0x7f811806a6a0 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 0x7f811816a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 0x7f811816a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43994/0 (socket says 172.21.15.33:43994) 2023-10-06T05:21:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 -- 172.21.15.33:0/2921951310 learned_addr learned my addr 172.21.15.33:0/2921951310 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f811659c700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8116d9d700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8118155af0 0x7f8118153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 -- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 msgr2=0x7f81180abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 -- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8118155af0 msgr2=0x7f8118153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8118155af0 0x7f8118153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 -- 172.21.15.33:0/2921951310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810800a5d0 con 0x7f81180c8b50 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.620+0000 7f8115d9b700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 0x7f811816a770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d860 tx=0x7f810c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.621+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c0125f0 con 0x7f81180c8b50 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.621+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c012750 con 0x7f81180c8b50 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.621+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81180ac630 con 0x7f81180c8b50 2023-10-06T05:21:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.621+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c012ad0 con 0x7f81180c8b50 2023-10-06T05:21:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.621+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81180ae9f0 con 0x7f81180c8b50 2023-10-06T05:21:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.622+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8118069ae0 con 0x7f81180c8b50 2023-10-06T05:21:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.623+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f810c01e460 con 0x7f81180c8b50 2023-10-06T05:21:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.624+0000 7f81067fc700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.624+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f810c084d10 con 0x7f81180c8b50 2023-10-06T05:21:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.624+0000 7f811659c700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.625+0000 7f811659c700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f810800a5a0 tx=0x7f810801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.628+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f810c0558b0 con 0x7f81180c8b50 2023-10-06T05:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.912+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f81180ac7c0 con 0x7f81180c8b50 2023-10-06T05:21:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.912+0000 7f81067fc700 1 -- 172.21.15.33:0/2921951310 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1199 (secure 0 0 0) 0x7f810c055440 con 0x7f81180c8b50 2023-10-06T05:21:02.915 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:02.915 INFO:teuthology.orchestra.run.smithi033.stdout:{"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":424,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1696569251,"num_in_osds":8,"osd_in_since":1696569242,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2478678016,"bytes_avail":770615435264,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":58,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6681}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-06T05:20:29.904994+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T05:21:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 msgr2=0x7f80f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f810800a5a0 tx=0x7f810801a000 comp rx=0 tx=0).stop 2023-10-06T05:21:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 msgr2=0x7f811816a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 0x7f811816a770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d860 tx=0x7f810c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 shutdown_connections 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8118155af0 0x7f8118153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f811806a6a0 0x7f81180abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 --2- 172.21.15.33:0/2921951310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81180c8b50 0x7f811816a770 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 >> 172.21.15.33:0/2921951310 conn(0x7f811806d030 msgr2=0x7f81180cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.914+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 shutdown_connections 2023-10-06T05:21:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:02.915+0000 7f811759e700 1 -- 172.21.15.33:0/2921951310 wait complete. 2023-10-06T05:21:02.934 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T05:21:02.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:03.029 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:21:03.027+0000 7fd0b39f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.120+0000 7f41cb59e700 1 Processor -- start 2023-10-06T05:21:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.120+0000 7f41cb59e700 1 -- start start 2023-10-06T05:21:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41cc155af0 0x7f41cc06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41cc0c8b50 0x7f41cc0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc06b7e0 con 0x7f41cc155af0 2023-10-06T05:21:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc06ab60 con 0x7f41cc0c8b50 2023-10-06T05:21:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc06ae40 con 0x7f41cc06a6a0 2023-10-06T05:21:03.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41cc155af0 0x7f41cc06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41cc155af0 0x7f41cc06a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44010/0 (socket says 172.21.15.33:44010) 2023-10-06T05:21:03.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 -- 172.21.15.33:0/1973019057 learned_addr learned my addr 172.21.15.33:0/1973019057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:03.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41c9d9b700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41cad9d700 1 --2- 172.21.15.33:0/1973019057 >> v2:172.21.15.167:3300/0 conn(0x7f41cc0c8b50 0x7f41cc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 -- 172.21.15.33:0/1973019057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 msgr2=0x7f41cc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 -- 172.21.15.33:0/1973019057 >> v2:172.21.15.167:3300/0 conn(0x7f41cc0c8b50 msgr2=0x7f41cc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 --2- 172.21.15.33:0/1973019057 >> v2:172.21.15.167:3300/0 conn(0x7f41cc0c8b50 0x7f41cc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41ca59c700 1 -- 172.21.15.33:0/1973019057 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc06b120 con 0x7f41cc155af0 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.122+0000 7f41c9d9b700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.123+0000 7f41ca59c700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc155af0 0x7f41cc06a160 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0c66b0 tx=0x7f41bc00b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab01b2494cf5954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.123+0000 7f41c959a700 1 -- 172.21.15.33:0/1973019057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00da60 con 0x7f41cc155af0 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.123+0000 7f41c959a700 1 -- 172.21.15.33:0/1973019057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc00dbc0 con 0x7f41cc155af0 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.123+0000 7f41c959a700 1 -- 172.21.15.33:0/1973019057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc017410 con 0x7f41cc155af0 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.123+0000 7f41cb59e700 1 -- 172.21.15.33:0/1973019057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc155af0 msgr2=0x7f41cc06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.123+0000 7f41cb59e700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc155af0 0x7f41cc06a160 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0c66b0 tx=0x7f41bc00b4e0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 -- 172.21.15.33:0/1973019057 shutdown_connections 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 --2- 172.21.15.33:0/1973019057 >> v2:172.21.15.167:3300/0 conn(0x7f41cc0c8b50 0x7f41cc0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc155af0 0x7f41cc06a160 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 --2- 172.21.15.33:0/1973019057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 -- 172.21.15.33:0/1973019057 >> 172.21.15.33:0/1973019057 conn(0x7f41cc06d030 msgr2=0x7f41cc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 -- 172.21.15.33:0/1973019057 shutdown_connections 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.124+0000 7f41cb59e700 1 -- 172.21.15.33:0/1973019057 wait complete. 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.125+0000 7f41cb59e700 1 Processor -- start 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.125+0000 7f41cb59e700 1 -- start start 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41cb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41cb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 0x7f41cc0abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41cb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc155af0 0x7f41cc0ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41cb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0c3f40 con 0x7f41cc0c8b50 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41cb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0affa0 con 0x7f41cc155af0 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41cb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0b0280 con 0x7f41cc06a6a0 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41c9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 0x7f41cc0abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41c9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 0x7f41cc0abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44022/0 (socket says 172.21.15.33:44022) 2023-10-06T05:21:03.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.126+0000 7f41c9d9b700 1 -- 172.21.15.33:0/2039609291 learned_addr learned my addr 172.21.15.33:0/2039609291 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:03.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41c9d9b700 1 -- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 msgr2=0x7f41cc150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41c9d9b700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41c9d9b700 1 -- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc155af0 msgr2=0x7f41cc0ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41c9d9b700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc155af0 0x7f41cc0ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41c9d9b700 1 -- 172.21.15.33:0/2039609291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc00b780 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41c9d9b700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 0x7f41cc0abaf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f41c000e8a0 tx=0x7f41c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c000c430 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c000c590 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc0b0560 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c000c950 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.127+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc150ac0 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.128+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc0c4220 con 0x7f41cc0c8b50 2023-10-06T05:21:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.129+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41c000cd00 con 0x7f41cc0c8b50 2023-10-06T05:21:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.130+0000 7f41ba7fc700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41ac066f30 0x7f41ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.130+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41c00844e0 con 0x7f41cc0c8b50 2023-10-06T05:21:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.131+0000 7f41ca59c700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41ac066f30 0x7f41ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.131+0000 7f41ca59c700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41ac066f30 0x7f41ac0693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b700 tx=0x7f41bc019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.134+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41c0054fe0 con 0x7f41cc0c8b50 2023-10-06T05:21:03.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.332+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41cc0c3d10 con 0x7f41ac066f30 2023-10-06T05:21:03.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.342+0000 7f41ba7fc700 1 -- 172.21.15.33:0/2039609291 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307738 (secure 0 0 0) 0x7f41cc0c3d10 con 0x7f41ac066f30 2023-10-06T05:21:03.349 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:03.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41ac066f30 msgr2=0x7f41ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41ac066f30 0x7f41ac0693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b700 tx=0x7f41bc019000 comp rx=0 tx=0).stop 2023-10-06T05:21:03.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 msgr2=0x7f41cc0abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 0x7f41cc0abaf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f41c000e8a0 tx=0x7f41c0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 shutdown_connections 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41ac066f30 0x7f41ac0693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc155af0 0x7f41cc0ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc06a6a0 0x7f41cc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 --2- 172.21.15.33:0/2039609291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0c8b50 0x7f41cc0abaf0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 >> 172.21.15.33:0/2039609291 conn(0x7f41cc06d030 msgr2=0x7f41cc06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 shutdown_connections 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:03.346+0000 7f41cb59e700 1 -- 172.21.15.33:0/2039609291 wait complete. 2023-10-06T05:21:03.352 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:21:03.367 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2023-10-06T05:21:01.913696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420584,"kb_used_data":9512,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752554136,"statfs":{"total":773094113280,"available":770615435264,"internally_reserved":0,"allocated":9740288,"data_stored":5025216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001814"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047746+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054711+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047774+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061615+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059581+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059292+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:03.594147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048133+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.061078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:39.064497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052728+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052245+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:35.106422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053309+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.455963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051588+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:37.245927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.060583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:07.387943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056543+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.063672+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.150141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058649+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:58.648941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059036+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:10.851643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048750+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054442+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059967+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:18.831632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058550+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056058+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054085+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:24.396610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047445+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055287+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:59.686854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053078+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:04.727723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051706+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:10.346856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.059449+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:08.100764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047908+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:18.677411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.055583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:22.803656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056488+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:17.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057141+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:34.761680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056494+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:42.458155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069083+0000","last_change":"2023-10-06T05:20:56.069083+0000","last_active":"2023-10-06T05:20:56.069083+0000","last_peered":"2023-10-06T05:20:56.069083+0000","last_clean":"2023-10-06T05:20:56.069083+0000","last_became_active":"2023-10-06T05:20:56.068775+0000","last_became_peered":"2023-10-06T05:20:56.068775+0000","last_unstale":"2023-10-06T05:20:56.069083+0000","last_undegraded":"2023-10-06T05:20:56.069083+0000","last_fullsized":"2023-10-06T05:20:56.069083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:43.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056509+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:23.982186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.061418+0000","last_change":"2023-10-06T05:20:56.061418+0000","last_active":"2023-10-06T05:20:56.061418+0000","last_peered":"2023-10-06T05:20:56.061418+0000","last_clean":"2023-10-06T05:20:56.061418+0000","last_became_active":"2023-10-06T05:20:56.060531+0000","last_became_peered":"2023-10-06T05:20:56.060531+0000","last_unstale":"2023-10-06T05:20:56.061418+0000","last_undegraded":"2023-10-06T05:20:56.061418+0000","last_fullsized":"2023-10-06T05:20:56.061418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:47.802684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048418+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063062+0000","last_change":"2023-10-06T05:20:56.063062+0000","last_active":"2023-10-06T05:20:56.063062+0000","last_peered":"2023-10-06T05:20:56.063062+0000","last_clean":"2023-10-06T05:20:56.063062+0000","last_became_active":"2023-10-06T05:20:56.062410+0000","last_became_peered":"2023-10-06T05:20:56.062410+0000","last_unstale":"2023-10-06T05:20:56.063062+0000","last_undegraded":"2023-10-06T05:20:56.063062+0000","last_fullsized":"2023-10-06T05:20:56.063062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:49.619842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057087+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:37.095202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062842+0000","last_change":"2023-10-06T05:20:56.062842+0000","last_active":"2023-10-06T05:20:56.062842+0000","last_peered":"2023-10-06T05:20:56.062842+0000","last_clean":"2023-10-06T05:20:56.062842+0000","last_became_active":"2023-10-06T05:20:56.062590+0000","last_became_peered":"2023-10-06T05:20:56.062590+0000","last_unstale":"2023-10-06T05:20:56.062842+0000","last_undegraded":"2023-10-06T05:20:56.062842+0000","last_fullsized":"2023-10-06T05:20:56.062842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:58.961459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057855+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.845343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063129+0000","last_change":"2023-10-06T05:20:56.063129+0000","last_active":"2023-10-06T05:20:56.063129+0000","last_peered":"2023-10-06T05:20:56.063129+0000","last_clean":"2023-10-06T05:20:56.063129+0000","last_became_active":"2023-10-06T05:20:56.062230+0000","last_became_peered":"2023-10-06T05:20:56.062230+0000","last_unstale":"2023-10-06T05:20:56.063129+0000","last_undegraded":"2023-10-06T05:20:56.063129+0000","last_fullsized":"2023-10-06T05:20:56.063129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053583+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:58.928023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066379+0000","last_change":"2023-10-06T05:20:56.066379+0000","last_active":"2023-10-06T05:20:56.066379+0000","last_peered":"2023-10-06T05:20:56.066379+0000","last_clean":"2023-10-06T05:20:56.066379+0000","last_became_active":"2023-10-06T05:20:56.064912+0000","last_became_peered":"2023-10-06T05:20:56.064912+0000","last_unstale":"2023-10-06T05:20:56.066379+0000","last_undegraded":"2023-10-06T05:20:56.066379+0000","last_fullsized":"2023-10-06T05:20:56.066379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:23.256874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058738+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:27.536202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063667+0000","last_change":"2023-10-06T05:20:56.063667+0000","last_active":"2023-10-06T05:20:56.063667+0000","last_peered":"2023-10-06T05:20:56.063667+0000","last_clean":"2023-10-06T05:20:56.063667+0000","last_became_active":"2023-10-06T05:20:56.063346+0000","last_became_peered":"2023-10-06T05:20:56.063346+0000","last_unstale":"2023-10-06T05:20:56.063667+0000","last_undegraded":"2023-10-06T05:20:56.063667+0000","last_fullsized":"2023-10-06T05:20:56.063667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058683+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:36.533849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.068185+0000","last_change":"2023-10-06T05:20:56.068185+0000","last_active":"2023-10-06T05:20:56.068185+0000","last_peered":"2023-10-06T05:20:56.068185+0000","last_clean":"2023-10-06T05:20:56.068185+0000","last_became_active":"2023-10-06T05:20:56.067889+0000","last_became_peered":"2023-10-06T05:20:56.067889+0000","last_unstale":"2023-10-06T05:20:56.068185+0000","last_undegraded":"2023-10-06T05:20:56.068185+0000","last_fullsized":"2023-10-06T05:20:56.068185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052435+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064663+0000","last_change":"2023-10-06T05:20:56.064663+0000","last_active":"2023-10-06T05:20:56.064663+0000","last_peered":"2023-10-06T05:20:56.064663+0000","last_clean":"2023-10-06T05:20:56.064663+0000","last_became_active":"2023-10-06T05:20:56.064382+0000","last_became_peered":"2023-10-06T05:20:56.064382+0000","last_unstale":"2023-10-06T05:20:56.064663+0000","last_undegraded":"2023-10-06T05:20:56.064663+0000","last_fullsized":"2023-10-06T05:20:56.064663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:07.091803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047994+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062871+0000","last_change":"2023-10-06T05:20:56.062871+0000","last_active":"2023-10-06T05:20:56.062871+0000","last_peered":"2023-10-06T05:20:56.062871+0000","last_clean":"2023-10-06T05:20:56.062871+0000","last_became_active":"2023-10-06T05:20:56.062606+0000","last_became_peered":"2023-10-06T05:20:56.062606+0000","last_unstale":"2023-10-06T05:20:56.062871+0000","last_undegraded":"2023-10-06T05:20:56.062871+0000","last_fullsized":"2023-10-06T05:20:56.062871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048278+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:46.906103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069433+0000","last_change":"2023-10-06T05:20:56.069433+0000","last_active":"2023-10-06T05:20:56.069433+0000","last_peered":"2023-10-06T05:20:56.069433+0000","last_clean":"2023-10-06T05:20:56.069433+0000","last_became_active":"2023-10-06T05:20:56.069152+0000","last_became_peered":"2023-10-06T05:20:56.069152+0000","last_unstale":"2023-10-06T05:20:56.069433+0000","last_undegraded":"2023-10-06T05:20:56.069433+0000","last_fullsized":"2023-10-06T05:20:56.069433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.728922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.057259+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:12.235739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066398+0000","last_change":"2023-10-06T05:20:56.066398+0000","last_active":"2023-10-06T05:20:56.066398+0000","last_peered":"2023-10-06T05:20:56.066398+0000","last_clean":"2023-10-06T05:20:56.066398+0000","last_became_active":"2023-10-06T05:20:56.066051+0000","last_became_peered":"2023-10-06T05:20:56.066051+0000","last_unstale":"2023-10-06T05:20:56.066398+0000","last_undegraded":"2023-10-06T05:20:56.066398+0000","last_fullsized":"2023-10-06T05:20:56.066398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:26.538710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.056633+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:48.550385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060068+0000","last_change":"2023-10-06T05:20:56.060068+0000","last_active":"2023-10-06T05:20:56.060068+0000","last_peered":"2023-10-06T05:20:56.060068+0000","last_clean":"2023-10-06T05:20:56.060068+0000","last_became_active":"2023-10-06T05:20:56.059765+0000","last_became_peered":"2023-10-06T05:20:56.059765+0000","last_unstale":"2023-10-06T05:20:56.060068+0000","last_undegraded":"2023-10-06T05:20:56.060068+0000","last_fullsized":"2023-10-06T05:20:56.060068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:56.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052031+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062607+0000","last_change":"2023-10-06T05:20:56.062607+0000","last_active":"2023-10-06T05:20:56.062607+0000","last_peered":"2023-10-06T05:20:56.062607+0000","last_clean":"2023-10-06T05:20:56.062607+0000","last_became_active":"2023-10-06T05:20:56.062344+0000","last_became_peered":"2023-10-06T05:20:56.062344+0000","last_unstale":"2023-10-06T05:20:56.062607+0000","last_undegraded":"2023-10-06T05:20:56.062607+0000","last_fullsized":"2023-10-06T05:20:56.062607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.897142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051968+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:14.557678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067504+0000","last_change":"2023-10-06T05:20:56.067504+0000","last_active":"2023-10-06T05:20:56.067504+0000","last_peered":"2023-10-06T05:20:56.067504+0000","last_clean":"2023-10-06T05:20:56.067504+0000","last_became_active":"2023-10-06T05:20:56.066985+0000","last_became_peered":"2023-10-06T05:20:56.066985+0000","last_unstale":"2023-10-06T05:20:56.067504+0000","last_undegraded":"2023-10-06T05:20:56.067504+0000","last_fullsized":"2023-10-06T05:20:56.067504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:43.665749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051195+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:39.400959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063906+0000","last_change":"2023-10-06T05:20:56.063906+0000","last_active":"2023-10-06T05:20:56.063906+0000","last_peered":"2023-10-06T05:20:56.063906+0000","last_clean":"2023-10-06T05:20:56.063906+0000","last_became_active":"2023-10-06T05:20:56.063625+0000","last_became_peered":"2023-10-06T05:20:56.063625+0000","last_unstale":"2023-10-06T05:20:56.063906+0000","last_undegraded":"2023-10-06T05:20:56.063906+0000","last_fullsized":"2023-10-06T05:20:56.063906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:07.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054747+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:55.677080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.060664+0000","last_change":"2023-10-06T05:20:56.060664+0000","last_active":"2023-10-06T05:20:56.060664+0000","last_peered":"2023-10-06T05:20:56.060664+0000","last_clean":"2023-10-06T05:20:56.060664+0000","last_became_active":"2023-10-06T05:20:56.059179+0000","last_became_peered":"2023-10-06T05:20:56.059179+0000","last_unstale":"2023-10-06T05:20:56.060664+0000","last_undegraded":"2023-10-06T05:20:56.060664+0000","last_fullsized":"2023-10-06T05:20:56.060664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.078805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050160+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069531+0000","last_change":"2023-10-06T05:20:56.069531+0000","last_active":"2023-10-06T05:20:56.069531+0000","last_peered":"2023-10-06T05:20:56.069531+0000","last_clean":"2023-10-06T05:20:56.069531+0000","last_became_active":"2023-10-06T05:20:56.068428+0000","last_became_peered":"2023-10-06T05:20:56.068428+0000","last_unstale":"2023-10-06T05:20:56.069531+0000","last_undegraded":"2023-10-06T05:20:56.069531+0000","last_fullsized":"2023-10-06T05:20:56.069531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.050790+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.069310+0000","last_change":"2023-10-06T05:20:56.069310+0000","last_active":"2023-10-06T05:20:56.069310+0000","last_peered":"2023-10-06T05:20:56.069310+0000","last_clean":"2023-10-06T05:20:56.069310+0000","last_became_active":"2023-10-06T05:20:56.069031+0000","last_became_peered":"2023-10-06T05:20:56.069031+0000","last_unstale":"2023-10-06T05:20:56.069310+0000","last_undegraded":"2023-10-06T05:20:56.069310+0000","last_fullsized":"2023-10-06T05:20:56.069310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:58.677893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054813+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:44.878348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066147+0000","last_change":"2023-10-06T05:20:56.066147+0000","last_active":"2023-10-06T05:20:56.066147+0000","last_peered":"2023-10-06T05:20:56.066147+0000","last_clean":"2023-10-06T05:20:56.066147+0000","last_became_active":"2023-10-06T05:20:56.065618+0000","last_became_peered":"2023-10-06T05:20:56.065618+0000","last_unstale":"2023-10-06T05:20:56.066147+0000","last_undegraded":"2023-10-06T05:20:56.066147+0000","last_fullsized":"2023-10-06T05:20:56.066147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:33.934156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051253+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:59.973383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062131+0000","last_change":"2023-10-06T05:20:56.062131+0000","last_active":"2023-10-06T05:20:56.062131+0000","last_peered":"2023-10-06T05:20:56.062131+0000","last_clean":"2023-10-06T05:20:56.062131+0000","last_became_active":"2023-10-06T05:20:56.061821+0000","last_became_peered":"2023-10-06T05:20:56.061821+0000","last_unstale":"2023-10-06T05:20:56.062131+0000","last_undegraded":"2023-10-06T05:20:56.062131+0000","last_fullsized":"2023-10-06T05:20:56.062131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:33.997234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.047924+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:27.547672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067790+0000","last_change":"2023-10-06T05:20:56.067790+0000","last_active":"2023-10-06T05:20:56.067790+0000","last_peered":"2023-10-06T05:20:56.067790+0000","last_clean":"2023-10-06T05:20:56.067790+0000","last_became_active":"2023-10-06T05:20:56.067491+0000","last_became_peered":"2023-10-06T05:20:56.067491+0000","last_unstale":"2023-10-06T05:20:56.067790+0000","last_undegraded":"2023-10-06T05:20:56.067790+0000","last_fullsized":"2023-10-06T05:20:56.067790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:06.378982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.058270+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:05.605433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.065948+0000","last_change":"2023-10-06T05:20:56.065948+0000","last_active":"2023-10-06T05:20:56.065948+0000","last_peered":"2023-10-06T05:20:56.065948+0000","last_clean":"2023-10-06T05:20:56.065948+0000","last_became_active":"2023-10-06T05:20:56.065640+0000","last_became_peered":"2023-10-06T05:20:56.065640+0000","last_unstale":"2023-10-06T05:20:56.065948+0000","last_undegraded":"2023-10-06T05:20:56.065948+0000","last_fullsized":"2023-10-06T05:20:56.065948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:17.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.049531+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063353+0000","last_change":"2023-10-06T05:20:56.063353+0000","last_active":"2023-10-06T05:20:56.063353+0000","last_peered":"2023-10-06T05:20:56.063353+0000","last_clean":"2023-10-06T05:20:56.063353+0000","last_became_active":"2023-10-06T05:20:56.063086+0000","last_became_peered":"2023-10-06T05:20:56.063086+0000","last_unstale":"2023-10-06T05:20:56.063353+0000","last_undegraded":"2023-10-06T05:20:56.063353+0000","last_fullsized":"2023-10-06T05:20:56.063353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:59.973368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.052028+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066899+0000","last_change":"2023-10-06T05:20:56.066899+0000","last_active":"2023-10-06T05:20:56.066899+0000","last_peered":"2023-10-06T05:20:56.066899+0000","last_clean":"2023-10-06T05:20:56.066899+0000","last_became_active":"2023-10-06T05:20:56.066645+0000","last_became_peered":"2023-10-06T05:20:56.066645+0000","last_unstale":"2023-10-06T05:20:56.066899+0000","last_undegraded":"2023-10-06T05:20:56.066899+0000","last_fullsized":"2023-10-06T05:20:56.066899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:38.055535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044210+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:20:55.044210+0000","last_peered":"2023-10-06T05:20:55.044210+0000","last_clean":"2023-10-06T05:20:55.044210+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:20:55.044210+0000","last_undegraded":"2023-10-06T05:20:55.044210+0000","last_fullsized":"2023-10-06T05:20:55.044210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053542+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:38.910105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.064164+0000","last_change":"2023-10-06T05:20:56.064164+0000","last_active":"2023-10-06T05:20:56.064164+0000","last_peered":"2023-10-06T05:20:56.064164+0000","last_clean":"2023-10-06T05:20:56.064164+0000","last_became_active":"2023-10-06T05:20:56.063907+0000","last_became_peered":"2023-10-06T05:20:56.063907+0000","last_unstale":"2023-10-06T05:20:56.064164+0000","last_undegraded":"2023-10-06T05:20:56.064164+0000","last_fullsized":"2023-10-06T05:20:56.064164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:16.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047296+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:20:55.047296+0000","last_peered":"2023-10-06T05:20:55.047296+0000","last_clean":"2023-10-06T05:20:55.047296+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:20:55.047296+0000","last_undegraded":"2023-10-06T05:20:55.047296+0000","last_fullsized":"2023-10-06T05:20:55.047296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053376+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:12.313245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.062639+0000","last_change":"2023-10-06T05:20:56.062639+0000","last_active":"2023-10-06T05:20:56.062639+0000","last_peered":"2023-10-06T05:20:56.062639+0000","last_clean":"2023-10-06T05:20:56.062639+0000","last_became_active":"2023-10-06T05:20:56.062307+0000","last_became_peered":"2023-10-06T05:20:56.062307+0000","last_unstale":"2023-10-06T05:20:56.062639+0000","last_undegraded":"2023-10-06T05:20:56.062639+0000","last_fullsized":"2023-10-06T05:20:56.062639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.040639+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:20:55.040639+0000","last_peered":"2023-10-06T05:20:55.040639+0000","last_clean":"2023-10-06T05:20:55.040639+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:20:55.040639+0000","last_undegraded":"2023-10-06T05:20:55.040639+0000","last_fullsized":"2023-10-06T05:20:55.040639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.048052+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067383+0000","last_change":"2023-10-06T05:20:56.067383+0000","last_active":"2023-10-06T05:20:56.067383+0000","last_peered":"2023-10-06T05:20:56.067383+0000","last_clean":"2023-10-06T05:20:56.067383+0000","last_became_active":"2023-10-06T05:20:56.067105+0000","last_became_peered":"2023-10-06T05:20:56.067105+0000","last_unstale":"2023-10-06T05:20:56.067383+0000","last_undegraded":"2023-10-06T05:20:56.067383+0000","last_fullsized":"2023-10-06T05:20:56.067383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:58.649017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.047360+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:20:55.047360+0000","last_peered":"2023-10-06T05:20:55.047360+0000","last_clean":"2023-10-06T05:20:55.047360+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:20:55.047360+0000","last_undegraded":"2023-10-06T05:20:55.047360+0000","last_fullsized":"2023-10-06T05:20:55.047360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044900+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:20:55.044900+0000","last_peered":"2023-10-06T05:20:55.044900+0000","last_clean":"2023-10-06T05:20:55.044900+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:20:55.044900+0000","last_undegraded":"2023-10-06T05:20:55.044900+0000","last_fullsized":"2023-10-06T05:20:55.044900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.054988+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:06.913609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066067+0000","last_change":"2023-10-06T05:20:56.066067+0000","last_active":"2023-10-06T05:20:56.066067+0000","last_peered":"2023-10-06T05:20:56.066067+0000","last_clean":"2023-10-06T05:20:56.066067+0000","last_became_active":"2023-10-06T05:20:56.065696+0000","last_became_peered":"2023-10-06T05:20:56.065696+0000","last_unstale":"2023-10-06T05:20:56.066067+0000","last_undegraded":"2023-10-06T05:20:56.066067+0000","last_fullsized":"2023-10-06T05:20:56.066067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:03.468905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044461+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:20:55.044461+0000","last_peered":"2023-10-06T05:20:55.044461+0000","last_clean":"2023-10-06T05:20:55.044461+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:20:55.044461+0000","last_undegraded":"2023-10-06T05:20:55.044461+0000","last_fullsized":"2023-10-06T05:20:55.044461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053799+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:20.780861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.063757+0000","last_change":"2023-10-06T05:20:56.063757+0000","last_active":"2023-10-06T05:20:56.063757+0000","last_peered":"2023-10-06T05:20:56.063757+0000","last_clean":"2023-10-06T05:20:56.063757+0000","last_became_active":"2023-10-06T05:20:56.063497+0000","last_became_peered":"2023-10-06T05:20:56.063497+0000","last_unstale":"2023-10-06T05:20:56.063757+0000","last_undegraded":"2023-10-06T05:20:56.063757+0000","last_fullsized":"2023-10-06T05:20:56.063757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:07.370987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044759+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:20:55.044759+0000","last_peered":"2023-10-06T05:20:55.044759+0000","last_clean":"2023-10-06T05:20:55.044759+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:20:55.044759+0000","last_undegraded":"2023-10-06T05:20:55.044759+0000","last_fullsized":"2023-10-06T05:20:55.044759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.053788+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:22.590638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.067795+0000","last_change":"2023-10-06T05:20:56.067795+0000","last_active":"2023-10-06T05:20:56.067795+0000","last_peered":"2023-10-06T05:20:56.067795+0000","last_clean":"2023-10-06T05:20:56.067795+0000","last_became_active":"2023-10-06T05:20:56.067515+0000","last_became_peered":"2023-10-06T05:20:56.067515+0000","last_unstale":"2023-10-06T05:20:56.067795+0000","last_undegraded":"2023-10-06T05:20:56.067795+0000","last_fullsized":"2023-10-06T05:20:56.067795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:17.822671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.044296+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:20:55.044296+0000","last_peered":"2023-10-06T05:20:55.044296+0000","last_clean":"2023-10-06T05:20:55.044296+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:20:55.044296+0000","last_undegraded":"2023-10-06T05:20:55.044296+0000","last_fullsized":"2023-10-06T05:20:55.044296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:56.037124+0000","last_change":"2023-10-06T05:20:56.051963+0000","last_active":"2023-10-06T05:20:56.037124+0000","last_peered":"2023-10-06T05:20:56.037124+0000","last_clean":"2023-10-06T05:20:56.037124+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:56.037124+0000","last_undegraded":"2023-10-06T05:20:56.037124+0000","last_fullsized":"2023-10-06T05:20:56.037124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:59.430802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T05:20:56.066251+0000","last_change":"2023-10-06T05:20:56.066251+0000","last_active":"2023-10-06T05:20:56.066251+0000","last_peered":"2023-10-06T05:20:56.066251+0000","last_clean":"2023-10-06T05:20:56.066251+0000","last_became_active":"2023-10-06T05:20:56.065923+0000","last_became_peered":"2023-10-06T05:20:56.065923+0000","last_unstale":"2023-10-06T05:20:56.066251+0000","last_undegraded":"2023-10-06T05:20:56.066251+0000","last_fullsized":"2023-10-06T05:20:56.066251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:09.868032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T05:20:55.042122+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:20:55.042122+0000","last_peered":"2023-10-06T05:20:55.042122+0000","last_clean":"2023-10-06T05:20:55.042122+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:20:55.042122+0000","last_undegraded":"2023-10-06T05:20:55.042122+0000","last_fullsized":"2023-10-06T05:20:55.042122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99099999999999999,"5min":1.169,"15min":1.169},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.628},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.268,"15min":1.268},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.075,"15min":1.075},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.68799999999999994},"min":{"1min":0.57199999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.995,"5min":1.093,"15min":1.093},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":1.22,"15min":1.22},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0349999999999999,"15min":1.079},"last":0.624},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":1.117,"15min":1.202},"last":0.85299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.622},"min":{"1min":0.47199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":1.258,"15min":1.258},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.64500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97599999999999998,"5min":1.2050000000000001,"15min":1.407},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.92500000000000004,"5min":1.139,"15min":1.139},"last":0.747},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.24,"15min":1.3300000000000001},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.034},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.1599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.76200000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91500000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.67700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.95699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82899999999999996,"5min":0.97799999999999998,"15min":0.998},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.0760000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68999999999999995,"15min":0.77900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.755}]},{"osd":7,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.77900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":0.97999999999999998,"15min":1.038},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73099999999999998,"15min":0.84399999999999997},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.442,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60099999999999998,"5min":0.73099999999999998,"15min":2.7349999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48499999999999999,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54300000000000004,"5min":0.71799999999999997,"15min":2.6520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53600000000000003,"15min":0.61099999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.85299999999999998,"15min":2.7639999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53400000000000003,"15min":0.61399999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.82699999999999996,"15min":2.8370000000000002},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.50700000000000001},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.125,"15min":1.2929999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.751,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93200000000000005,"5min":1.0289999999999999,"15min":1.349},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.82899999999999996},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":1.1419999999999999,"15min":1.669},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.86399999999999999},"min":{"1min":0.66800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91900000000000004,"5min":1.1779999999999999,"15min":1.6040000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88700000000000001},"min":{"1min":0.61499999999999999,"5min":0.503,"15min":0.503},"max":{"1min":0.96999999999999997,"5min":1.165,"15min":1.573},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":1.0780000000000001,"15min":1.528},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.75},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1662976,"data_stored":1070928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46200000000000002,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64600000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73199999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.62},{"interface":"front","average":{"1min":0.497,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.434}]},{"osd":3,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.879},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.75700000000000001},"min":{"1min":0.65200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0069999999999999,"5min":1.218,"15min":1.218},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.749},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.79400000000000004},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.05,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.63700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.46200000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48699999999999999,"15min":0.47399999999999998},"min":{"1min":0.42599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":1.0329999999999999,"5min":1.077,"15min":1.1379999999999999},"last":0.88},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.47499999999999998},"max":{"1min":0.98699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.495},"max":{"1min":1.04,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.319},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":0.79200000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.103,"5min":1.103,"15min":1.141},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52100000000000002},"max":{"1min":1.0609999999999999,"5min":1.1000000000000001,"15min":1.282},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.68100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.3520000000000001},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.70799999999999996,"5min":1.0720000000000001,"15min":2.0950000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":1.772},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.78100000000000003,"15min":1.913},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.66800000000000004,"5min":0.84399999999999997,"15min":2.0089999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55400000000000005,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.82399999999999995,"15min":1.954},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0469999999999999,"5min":1.2030000000000001,"15min":1.282},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.0669999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1399999999999999,"5min":1.2250000000000001,"15min":1.3009999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.1910000000000001,"15min":1.343},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3380000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82599999999999996,"15min":0.85999999999999999},"min":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56799999999999995},"max":{"1min":1.1140000000000001,"5min":1.206,"15min":1.3640000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.68700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1539999999999999,"5min":1.2090000000000001,"15min":1.214},"last":0.746}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.745,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76600000000000001,"15min":0.76000000000000001},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0269999999999999,"15min":1.169},"last":0.755}]},{"osd":1,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.042,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.117},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.002},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.196},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63500000000000001,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.8740000000000001},"last":0.5},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66100000000000003,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.375},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98799999999999999,"15min":1.004},"last":0.44},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95699999999999996,"5min":1.1639999999999999,"15min":1.9390000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71199999999999997,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93300000000000005,"5min":1.0209999999999999,"15min":1.466},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.83999999999999997},"min":{"1min":0.626,"5min":0.499,"15min":0.499},"max":{"1min":0.91800000000000004,"5min":1.1930000000000001,"15min":1.218},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302428,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069412,"statfs":{"total":96636764160,"available":96327077888,"internally_reserved":0,"allocated":1069056,"data_stored":480560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90800000000000003,"5min":1.165,"15min":1.165},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98999999999999999,"5min":1.016,"15min":1.1890000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.69299999999999995},"min":{"1min":0.59899999999999998,"5min":0.45100000000000001,"15min":0.438},"max":{"1min":0.88200000000000001,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.038,"15min":1.038},"last":0.88700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":1.204,"15min":1.204},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.70699999999999996},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.995,"15min":1.0549999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.087},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.58499999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.97299999999999998,"5min":1.2769999999999999,"15min":1.278},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64900000000000002,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.996,"15min":1.294},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64400000000000002,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.141,"15min":1.141},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.876,"5min":0.999,"15min":1.254},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71899999999999997,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.042,"15min":1.337},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.55300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.248,"15min":1.248},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:06.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:06.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.571+0000 7f44a899d700 1 Processor -- start 2023-10-06T05:21:06.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.571+0000 7f44a899d700 1 -- start start 2023-10-06T05:21:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.572+0000 7f44a899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a413faa0 0x7f44a413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a899d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44a406aad0 0x7f44a406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a899d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40c9cb0 con 0x7f44a40c7940 2023-10-06T05:21:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a899d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40ca270 con 0x7f44a406aad0 2023-10-06T05:21:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a406b840 con 0x7f44a413faa0 2023-10-06T05:21:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44a406aad0 0x7f44a406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f44a406aad0 0x7f44a406af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44028/0 (socket says 172.21.15.33:44028) 2023-10-06T05:21:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 -- 172.21.15.33:0/596169941 learned_addr learned my addr 172.21.15.33:0/596169941 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a37fe700 1 --2- 172.21.15.33:0/596169941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a413faa0 0x7f44a413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 -- 172.21.15.33:0/596169941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a413faa0 msgr2=0x7f44a413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 --2- 172.21.15.33:0/596169941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a413faa0 0x7f44a413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 -- 172.21.15.33:0/596169941 >> v1:172.21.15.167:6789/0 conn(0x7f44a40c7940 legacy=0x7f44a40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.573+0000 7f44a3fff700 1 -- 172.21.15.33:0/596169941 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a406bb20 con 0x7f44a406aad0 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.574+0000 7f44a3fff700 1 --2- 172.21.15.33:0/596169941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a406aad0 0x7f44a406af50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f449c00d1a0 tx=0x7f449c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb44d3359c7cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.574+0000 7f44a27fc700 1 -- 172.21.15.33:0/596169941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c010070 con 0x7f44a406aad0 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.574+0000 7f44a27fc700 1 -- 172.21.15.33:0/596169941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c00b0d0 con 0x7f44a406aad0 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.574+0000 7f44a27fc700 1 -- 172.21.15.33:0/596169941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00b490 con 0x7f44a406aad0 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.574+0000 7f44a899d700 1 -- 172.21.15.33:0/596169941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a406aad0 msgr2=0x7f44a406af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.574+0000 7f44a899d700 1 --2- 172.21.15.33:0/596169941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a406aad0 0x7f44a406af50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f449c00d1a0 tx=0x7f449c00ba50 comp rx=0 tx=0).stop 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.575+0000 7f44a899d700 1 -- 172.21.15.33:0/596169941 shutdown_connections 2023-10-06T05:21:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.575+0000 7f44a899d700 1 --2- 172.21.15.33:0/596169941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a406aad0 0x7f44a406af50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.575+0000 7f44a899d700 1 --2- 172.21.15.33:0/596169941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a413faa0 0x7f44a413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.575+0000 7f44a899d700 1 -- 172.21.15.33:0/596169941 >> 172.21.15.33:0/596169941 conn(0x7f44a406d030 msgr2=0x7f44a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.575+0000 7f44a899d700 1 -- 172.21.15.33:0/596169941 shutdown_connections 2023-10-06T05:21:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.575+0000 7f44a899d700 1 -- 172.21.15.33:0/596169941 wait complete. 2023-10-06T05:21:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.576+0000 7f44a899d700 1 Processor -- start 2023-10-06T05:21:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.576+0000 7f44a899d700 1 -- start start 2023-10-06T05:21:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.577+0000 7f44a899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a406aad0 0x7f44a4156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.577+0000 7f44a899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44a40c7940 0x7f44a413a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.577+0000 7f44a899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 0x7f44a4153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.577+0000 7f44a899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a4154530 con 0x7f44a413faa0 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.577+0000 7f44a899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a4157270 con 0x7f44a40c7940 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.577+0000 7f44a899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a413f3d0 con 0x7f44a406aad0 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 0x7f44a4153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 0x7f44a4153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44044/0 (socket says 172.21.15.33:44044) 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 -- 172.21.15.33:0/2234758256 learned_addr learned my addr 172.21.15.33:0/2234758256 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a37fe700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a406aad0 0x7f44a4156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a2ffd700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44a40c7940 0x7f44a413a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 -- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a406aad0 msgr2=0x7f44a4156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a406aad0 0x7f44a4156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 -- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44a40c7940 msgr2=0x7f44a413a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44a40c7940 0x7f44a413a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 -- 172.21.15.33:0/2234758256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c00bcf0 con 0x7f44a413faa0 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a2ffd700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44a40c7940 0x7f44a413a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.578+0000 7f44a3fff700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 0x7f44a4153ff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f449c00d8f0 tx=0x7f449c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.579+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00b6f0 con 0x7f44a413faa0 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.579+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c0032f0 con 0x7f44a413faa0 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.579+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c01f880 con 0x7f44a413faa0 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.579+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a413ee10 con 0x7f44a413faa0 2023-10-06T05:21:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.579+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a41593d0 con 0x7f44a413faa0 2023-10-06T05:21:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.580+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a404c380 con 0x7f44a413faa0 2023-10-06T05:21:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.581+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f449c003040 con 0x7f44a413faa0 2023-10-06T05:21:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.582+0000 7f448b7fe700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4490066f30 0x7f44900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.582+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f449c088920 con 0x7f44a413faa0 2023-10-06T05:21:06.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.582+0000 7f44a37fe700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4490066f30 0x7f44900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.583+0000 7f44a37fe700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4490066f30 0x7f44900693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4494006fd0 tx=0x7f4494008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.587+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f449c055990 con 0x7f44a413faa0 2023-10-06T05:21:06.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.784+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44a40c9f00 con 0x7f4490066f30 2023-10-06T05:21:06.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.794+0000 7f448b7fe700 1 -- 172.21.15.33:0/2234758256 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310677 (secure 0 0 0) 0x7f44a40c9f00 con 0x7f4490066f30 2023-10-06T05:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:06.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4490066f30 msgr2=0x7f44900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4490066f30 0x7f44900693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4494006fd0 tx=0x7f4494008040 comp rx=0 tx=0).stop 2023-10-06T05:21:06.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 msgr2=0x7f44a4153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 0x7f44a4153ff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f449c00d8f0 tx=0x7f449c01a000 comp rx=0 tx=0).stop 2023-10-06T05:21:06.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 shutdown_connections 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4490066f30 0x7f44900693e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a413faa0 0x7f44a4153ff0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44a406aad0 0x7f44a4156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 --2- 172.21.15.33:0/2234758256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44a40c7940 0x7f44a413a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.798+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 >> 172.21.15.33:0/2234758256 conn(0x7f44a406d030 msgr2=0x7f44a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.799+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 shutdown_connections 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:06.799+0000 7f44a899d700 1 -- 172.21.15.33:0/2234758256 wait complete. 2023-10-06T05:21:06.807 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2023-10-06T05:21:05.914832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420840,"kb_used_data":9768,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752553880,"statfs":{"total":773094113280,"available":770615173120,"internally_reserved":0,"allocated":10002432,"data_stored":5139884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.867194"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.089165+0000","last_change":"2023-10-06T05:20:57.072076+0000","last_active":"2023-10-06T05:20:57.089165+0000","last_peered":"2023-10-06T05:20:57.089165+0000","last_clean":"2023-10-06T05:20:57.089165+0000","last_became_active":"2023-10-06T05:20:57.071588+0000","last_became_peered":"2023-10-06T05:20:57.071588+0000","last_unstale":"2023-10-06T05:20:57.089165+0000","last_undegraded":"2023-10-06T05:20:57.089165+0000","last_fullsized":"2023-10-06T05:20:57.089165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.072366+0000","last_change":"2023-10-06T05:20:57.069114+0000","last_active":"2023-10-06T05:20:57.072366+0000","last_peered":"2023-10-06T05:20:57.072366+0000","last_clean":"2023-10-06T05:20:57.072366+0000","last_became_active":"2023-10-06T05:20:57.068814+0000","last_became_peered":"2023-10-06T05:20:57.068814+0000","last_unstale":"2023-10-06T05:20:57.072366+0000","last_undegraded":"2023-10-06T05:20:57.072366+0000","last_fullsized":"2023-10-06T05:20:57.072366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080638+0000","last_change":"2023-10-06T05:20:57.069000+0000","last_active":"2023-10-06T05:20:57.080638+0000","last_peered":"2023-10-06T05:20:57.080638+0000","last_clean":"2023-10-06T05:20:57.080638+0000","last_became_active":"2023-10-06T05:20:57.067895+0000","last_became_peered":"2023-10-06T05:20:57.067895+0000","last_unstale":"2023-10-06T05:20:57.080638+0000","last_undegraded":"2023-10-06T05:20:57.080638+0000","last_fullsized":"2023-10-06T05:20:57.080638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092076+0000","last_change":"2023-10-06T05:20:57.087807+0000","last_active":"2023-10-06T05:20:57.092076+0000","last_peered":"2023-10-06T05:20:57.092076+0000","last_clean":"2023-10-06T05:20:57.092076+0000","last_became_active":"2023-10-06T05:20:57.087515+0000","last_became_peered":"2023-10-06T05:20:57.087515+0000","last_unstale":"2023-10-06T05:20:57.092076+0000","last_undegraded":"2023-10-06T05:20:57.092076+0000","last_fullsized":"2023-10-06T05:20:57.092076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.737884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077658+0000","last_change":"2023-10-06T05:20:57.069292+0000","last_active":"2023-10-06T05:20:57.077658+0000","last_peered":"2023-10-06T05:20:57.077658+0000","last_clean":"2023-10-06T05:20:57.077658+0000","last_became_active":"2023-10-06T05:20:57.069009+0000","last_became_peered":"2023-10-06T05:20:57.069009+0000","last_unstale":"2023-10-06T05:20:57.077658+0000","last_undegraded":"2023-10-06T05:20:57.077658+0000","last_fullsized":"2023-10-06T05:20:57.077658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:23.982186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092585+0000","last_change":"2023-10-06T05:20:57.089330+0000","last_active":"2023-10-06T05:20:57.092585+0000","last_peered":"2023-10-06T05:20:57.092585+0000","last_clean":"2023-10-06T05:20:57.092585+0000","last_became_active":"2023-10-06T05:20:57.089019+0000","last_became_peered":"2023-10-06T05:20:57.089019+0000","last_unstale":"2023-10-06T05:20:57.092585+0000","last_undegraded":"2023-10-06T05:20:57.092585+0000","last_fullsized":"2023-10-06T05:20:57.092585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:36.898819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077803+0000","last_change":"2023-10-06T05:20:57.073644+0000","last_active":"2023-10-06T05:20:57.077803+0000","last_peered":"2023-10-06T05:20:57.077803+0000","last_clean":"2023-10-06T05:20:57.077803+0000","last_became_active":"2023-10-06T05:20:57.072854+0000","last_became_peered":"2023-10-06T05:20:57.072854+0000","last_unstale":"2023-10-06T05:20:57.077803+0000","last_undegraded":"2023-10-06T05:20:57.077803+0000","last_fullsized":"2023-10-06T05:20:57.077803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.086413+0000","last_change":"2023-10-06T05:20:57.081710+0000","last_active":"2023-10-06T05:20:57.086413+0000","last_peered":"2023-10-06T05:20:57.086413+0000","last_clean":"2023-10-06T05:20:57.086413+0000","last_became_active":"2023-10-06T05:20:57.081195+0000","last_became_peered":"2023-10-06T05:20:57.081195+0000","last_unstale":"2023-10-06T05:20:57.086413+0000","last_undegraded":"2023-10-06T05:20:57.086413+0000","last_fullsized":"2023-10-06T05:20:57.086413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.066283+0000","last_change":"2023-10-06T05:20:57.062171+0000","last_active":"2023-10-06T05:20:57.066283+0000","last_peered":"2023-10-06T05:20:57.066283+0000","last_clean":"2023-10-06T05:20:57.066283+0000","last_became_active":"2023-10-06T05:20:57.061820+0000","last_became_peered":"2023-10-06T05:20:57.061820+0000","last_unstale":"2023-10-06T05:20:57.066283+0000","last_undegraded":"2023-10-06T05:20:57.066283+0000","last_fullsized":"2023-10-06T05:20:57.066283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:06.913609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.089367+0000","last_change":"2023-10-06T05:20:57.086647+0000","last_active":"2023-10-06T05:20:57.089367+0000","last_peered":"2023-10-06T05:20:57.089367+0000","last_clean":"2023-10-06T05:20:57.089367+0000","last_became_active":"2023-10-06T05:20:57.086318+0000","last_became_peered":"2023-10-06T05:20:57.086318+0000","last_unstale":"2023-10-06T05:20:57.089367+0000","last_undegraded":"2023-10-06T05:20:57.089367+0000","last_fullsized":"2023-10-06T05:20:57.089367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:59:08.614884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.078964+0000","last_change":"2023-10-06T05:20:57.075649+0000","last_active":"2023-10-06T05:20:57.078964+0000","last_peered":"2023-10-06T05:20:57.078964+0000","last_clean":"2023-10-06T05:20:57.078964+0000","last_became_active":"2023-10-06T05:20:57.075334+0000","last_became_peered":"2023-10-06T05:20:57.075334+0000","last_unstale":"2023-10-06T05:20:57.078964+0000","last_undegraded":"2023-10-06T05:20:57.078964+0000","last_fullsized":"2023-10-06T05:20:57.078964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:07.201913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.068361+0000","last_change":"2023-10-06T05:20:57.066623+0000","last_active":"2023-10-06T05:20:57.068361+0000","last_peered":"2023-10-06T05:20:57.068361+0000","last_clean":"2023-10-06T05:20:57.068361+0000","last_became_active":"2023-10-06T05:20:57.066299+0000","last_became_peered":"2023-10-06T05:20:57.066299+0000","last_unstale":"2023-10-06T05:20:57.068361+0000","last_undegraded":"2023-10-06T05:20:57.068361+0000","last_fullsized":"2023-10-06T05:20:57.068361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:10.316147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092117+0000","last_change":"2023-10-06T05:20:57.086537+0000","last_active":"2023-10-06T05:20:57.092117+0000","last_peered":"2023-10-06T05:20:57.092117+0000","last_clean":"2023-10-06T05:20:57.092117+0000","last_became_active":"2023-10-06T05:20:57.086214+0000","last_became_peered":"2023-10-06T05:20:57.086214+0000","last_unstale":"2023-10-06T05:20:57.092117+0000","last_undegraded":"2023-10-06T05:20:57.092117+0000","last_fullsized":"2023-10-06T05:20:57.092117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:00.982330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.090194+0000","last_change":"2023-10-06T05:20:57.089149+0000","last_active":"2023-10-06T05:20:57.090194+0000","last_peered":"2023-10-06T05:20:57.090194+0000","last_clean":"2023-10-06T05:20:57.090194+0000","last_became_active":"2023-10-06T05:20:57.088882+0000","last_became_peered":"2023-10-06T05:20:57.088882+0000","last_unstale":"2023-10-06T05:20:57.090194+0000","last_undegraded":"2023-10-06T05:20:57.090194+0000","last_fullsized":"2023-10-06T05:20:57.090194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.086869+0000","last_change":"2023-10-06T05:20:57.083794+0000","last_active":"2023-10-06T05:20:57.086869+0000","last_peered":"2023-10-06T05:20:57.086869+0000","last_clean":"2023-10-06T05:20:57.086869+0000","last_became_active":"2023-10-06T05:20:57.083486+0000","last_became_peered":"2023-10-06T05:20:57.083486+0000","last_unstale":"2023-10-06T05:20:57.086869+0000","last_undegraded":"2023-10-06T05:20:57.086869+0000","last_fullsized":"2023-10-06T05:20:57.086869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.087766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080002+0000","last_change":"2023-10-06T05:20:57.069890+0000","last_active":"2023-10-06T05:20:57.080002+0000","last_peered":"2023-10-06T05:20:57.080002+0000","last_clean":"2023-10-06T05:20:57.080002+0000","last_became_active":"2023-10-06T05:20:57.069606+0000","last_became_peered":"2023-10-06T05:20:57.069606+0000","last_unstale":"2023-10-06T05:20:57.080002+0000","last_undegraded":"2023-10-06T05:20:57.080002+0000","last_fullsized":"2023-10-06T05:20:57.080002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:51.282328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080372+0000","last_change":"2023-10-06T05:20:57.069825+0000","last_active":"2023-10-06T05:20:57.080372+0000","last_peered":"2023-10-06T05:20:57.080372+0000","last_clean":"2023-10-06T05:20:57.080372+0000","last_became_active":"2023-10-06T05:20:57.069248+0000","last_became_peered":"2023-10-06T05:20:57.069248+0000","last_unstale":"2023-10-06T05:20:57.080372+0000","last_undegraded":"2023-10-06T05:20:57.080372+0000","last_fullsized":"2023-10-06T05:20:57.080372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:25.329414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.070513+0000","last_change":"2023-10-06T05:20:57.066573+0000","last_active":"2023-10-06T05:20:57.070513+0000","last_peered":"2023-10-06T05:20:57.070513+0000","last_clean":"2023-10-06T05:20:57.070513+0000","last_became_active":"2023-10-06T05:20:57.066313+0000","last_became_peered":"2023-10-06T05:20:57.066313+0000","last_unstale":"2023-10-06T05:20:57.070513+0000","last_undegraded":"2023-10-06T05:20:57.070513+0000","last_fullsized":"2023-10-06T05:20:57.070513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:08.016162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.090184+0000","last_change":"2023-10-06T05:20:57.088656+0000","last_active":"2023-10-06T05:20:57.090184+0000","last_peered":"2023-10-06T05:20:57.090184+0000","last_clean":"2023-10-06T05:20:57.090184+0000","last_became_active":"2023-10-06T05:20:57.088361+0000","last_became_peered":"2023-10-06T05:20:57.088361+0000","last_unstale":"2023-10-06T05:20:57.090184+0000","last_undegraded":"2023-10-06T05:20:57.090184+0000","last_fullsized":"2023-10-06T05:20:57.090184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:00.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.094120+0000","last_change":"2023-10-06T05:20:57.090850+0000","last_active":"2023-10-06T05:20:57.094120+0000","last_peered":"2023-10-06T05:20:57.094120+0000","last_clean":"2023-10-06T05:20:57.094120+0000","last_became_active":"2023-10-06T05:20:57.090535+0000","last_became_peered":"2023-10-06T05:20:57.090535+0000","last_unstale":"2023-10-06T05:20:57.094120+0000","last_undegraded":"2023-10-06T05:20:57.094120+0000","last_fullsized":"2023-10-06T05:20:57.094120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:17.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.078305+0000","last_change":"2023-10-06T05:20:57.074373+0000","last_active":"2023-10-06T05:20:57.078305+0000","last_peered":"2023-10-06T05:20:57.078305+0000","last_clean":"2023-10-06T05:20:57.078305+0000","last_became_active":"2023-10-06T05:20:57.074080+0000","last_became_peered":"2023-10-06T05:20:57.074080+0000","last_unstale":"2023-10-06T05:20:57.078305+0000","last_undegraded":"2023-10-06T05:20:57.078305+0000","last_fullsized":"2023-10-06T05:20:57.078305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:45.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.089256+0000","last_change":"2023-10-06T05:20:57.088271+0000","last_active":"2023-10-06T05:20:57.089256+0000","last_peered":"2023-10-06T05:20:57.089256+0000","last_clean":"2023-10-06T05:20:57.089256+0000","last_became_active":"2023-10-06T05:20:57.087913+0000","last_became_peered":"2023-10-06T05:20:57.087913+0000","last_unstale":"2023-10-06T05:20:57.089256+0000","last_undegraded":"2023-10-06T05:20:57.089256+0000","last_fullsized":"2023-10-06T05:20:57.089256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.367162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.087082+0000","last_change":"2023-10-06T05:20:57.083560+0000","last_active":"2023-10-06T05:20:57.087082+0000","last_peered":"2023-10-06T05:20:57.087082+0000","last_clean":"2023-10-06T05:20:57.087082+0000","last_became_active":"2023-10-06T05:20:57.083262+0000","last_became_peered":"2023-10-06T05:20:57.083262+0000","last_unstale":"2023-10-06T05:20:57.087082+0000","last_undegraded":"2023-10-06T05:20:57.087082+0000","last_fullsized":"2023-10-06T05:20:57.087082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:21.384837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.070544+0000","last_change":"2023-10-06T05:20:57.067524+0000","last_active":"2023-10-06T05:20:57.070544+0000","last_peered":"2023-10-06T05:20:57.070544+0000","last_clean":"2023-10-06T05:20:57.070544+0000","last_became_active":"2023-10-06T05:20:57.067274+0000","last_became_peered":"2023-10-06T05:20:57.067274+0000","last_unstale":"2023-10-06T05:20:57.070544+0000","last_undegraded":"2023-10-06T05:20:57.070544+0000","last_fullsized":"2023-10-06T05:20:57.070544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.071400+0000","last_change":"2023-10-06T05:20:57.066824+0000","last_active":"2023-10-06T05:20:57.071400+0000","last_peered":"2023-10-06T05:20:57.071400+0000","last_clean":"2023-10-06T05:20:57.071400+0000","last_became_active":"2023-10-06T05:20:57.065581+0000","last_became_peered":"2023-10-06T05:20:57.065581+0000","last_unstale":"2023-10-06T05:20:57.071400+0000","last_undegraded":"2023-10-06T05:20:57.071400+0000","last_fullsized":"2023-10-06T05:20:57.071400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:48.811645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.086873+0000","last_change":"2023-10-06T05:20:57.083622+0000","last_active":"2023-10-06T05:20:57.086873+0000","last_peered":"2023-10-06T05:20:57.086873+0000","last_clean":"2023-10-06T05:20:57.086873+0000","last_became_active":"2023-10-06T05:20:57.083348+0000","last_became_peered":"2023-10-06T05:20:57.083348+0000","last_unstale":"2023-10-06T05:20:57.086873+0000","last_undegraded":"2023-10-06T05:20:57.086873+0000","last_fullsized":"2023-10-06T05:20:57.086873+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.168546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.060467+0000","last_change":"2023-10-06T05:20:57.058524+0000","last_active":"2023-10-06T05:20:57.060467+0000","last_peered":"2023-10-06T05:20:57.060467+0000","last_clean":"2023-10-06T05:20:57.060467+0000","last_became_active":"2023-10-06T05:20:57.058242+0000","last_became_peered":"2023-10-06T05:20:57.058242+0000","last_unstale":"2023-10-06T05:20:57.060467+0000","last_undegraded":"2023-10-06T05:20:57.060467+0000","last_fullsized":"2023-10-06T05:20:57.060467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:42.891949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.091900+0000","last_change":"2023-10-06T05:20:57.086400+0000","last_active":"2023-10-06T05:20:57.091900+0000","last_peered":"2023-10-06T05:20:57.091900+0000","last_clean":"2023-10-06T05:20:57.091900+0000","last_became_active":"2023-10-06T05:20:57.085701+0000","last_became_peered":"2023-10-06T05:20:57.085701+0000","last_unstale":"2023-10-06T05:20:57.091900+0000","last_undegraded":"2023-10-06T05:20:57.091900+0000","last_fullsized":"2023-10-06T05:20:57.091900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:59.686854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.073713+0000","last_change":"2023-10-06T05:20:57.066940+0000","last_active":"2023-10-06T05:20:57.073713+0000","last_peered":"2023-10-06T05:20:57.073713+0000","last_clean":"2023-10-06T05:20:57.073713+0000","last_became_active":"2023-10-06T05:20:57.066648+0000","last_became_peered":"2023-10-06T05:20:57.066648+0000","last_unstale":"2023-10-06T05:20:57.073713+0000","last_undegraded":"2023-10-06T05:20:57.073713+0000","last_fullsized":"2023-10-06T05:20:57.073713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.367162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.065859+0000","last_change":"2023-10-06T05:20:57.060884+0000","last_active":"2023-10-06T05:20:57.065859+0000","last_peered":"2023-10-06T05:20:57.065859+0000","last_clean":"2023-10-06T05:20:57.065859+0000","last_became_active":"2023-10-06T05:20:57.060534+0000","last_became_peered":"2023-10-06T05:20:57.060534+0000","last_unstale":"2023-10-06T05:20:57.065859+0000","last_undegraded":"2023-10-06T05:20:57.065859+0000","last_fullsized":"2023-10-06T05:20:57.065859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:19.366982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.086687+0000","last_change":"2023-10-06T05:20:57.084293+0000","last_active":"2023-10-06T05:20:57.086687+0000","last_peered":"2023-10-06T05:20:57.086687+0000","last_clean":"2023-10-06T05:20:57.086687+0000","last_became_active":"2023-10-06T05:20:57.083991+0000","last_became_peered":"2023-10-06T05:20:57.083991+0000","last_unstale":"2023-10-06T05:20:57.086687+0000","last_undegraded":"2023-10-06T05:20:57.086687+0000","last_fullsized":"2023-10-06T05:20:57.086687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080023+0000","last_change":"2023-10-06T05:20:57.070482+0000","last_active":"2023-10-06T05:20:57.080023+0000","last_peered":"2023-10-06T05:20:57.080023+0000","last_clean":"2023-10-06T05:20:57.080023+0000","last_became_active":"2023-10-06T05:20:57.070224+0000","last_became_peered":"2023-10-06T05:20:57.070224+0000","last_unstale":"2023-10-06T05:20:57.080023+0000","last_undegraded":"2023-10-06T05:20:57.080023+0000","last_fullsized":"2023-10-06T05:20:57.080023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:07.917711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.081351+0000","last_change":"2023-10-06T05:20:57.071082+0000","last_active":"2023-10-06T05:20:57.081351+0000","last_peered":"2023-10-06T05:20:57.081351+0000","last_clean":"2023-10-06T05:20:57.081351+0000","last_became_active":"2023-10-06T05:20:57.070797+0000","last_became_peered":"2023-10-06T05:20:57.070797+0000","last_unstale":"2023-10-06T05:20:57.081351+0000","last_undegraded":"2023-10-06T05:20:57.081351+0000","last_fullsized":"2023-10-06T05:20:57.081351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:19.771780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"88'4","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080951+0000","last_change":"2023-10-06T05:20:56.069083+0000","last_active":"2023-10-06T05:20:57.080951+0000","last_peered":"2023-10-06T05:20:57.080951+0000","last_clean":"2023-10-06T05:20:57.080951+0000","last_became_active":"2023-10-06T05:20:56.068775+0000","last_became_peered":"2023-10-06T05:20:56.068775+0000","last_unstale":"2023-10-06T05:20:57.080951+0000","last_undegraded":"2023-10-06T05:20:57.080951+0000","last_fullsized":"2023-10-06T05:20:57.080951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:43.079212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080069+0000","last_change":"2023-10-06T05:20:57.069168+0000","last_active":"2023-10-06T05:20:57.080069+0000","last_peered":"2023-10-06T05:20:57.080069+0000","last_clean":"2023-10-06T05:20:57.080069+0000","last_became_active":"2023-10-06T05:20:57.068775+0000","last_became_peered":"2023-10-06T05:20:57.068775+0000","last_unstale":"2023-10-06T05:20:57.080069+0000","last_undegraded":"2023-10-06T05:20:57.080069+0000","last_fullsized":"2023-10-06T05:20:57.080069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:02.552423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.078557+0000","last_change":"2023-10-06T05:20:56.061418+0000","last_active":"2023-10-06T05:20:57.078557+0000","last_peered":"2023-10-06T05:20:57.078557+0000","last_clean":"2023-10-06T05:20:57.078557+0000","last_became_active":"2023-10-06T05:20:56.060531+0000","last_became_peered":"2023-10-06T05:20:56.060531+0000","last_unstale":"2023-10-06T05:20:57.078557+0000","last_undegraded":"2023-10-06T05:20:57.078557+0000","last_fullsized":"2023-10-06T05:20:57.078557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:47.802684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.087548+0000","last_change":"2023-10-06T05:20:57.086346+0000","last_active":"2023-10-06T05:20:57.087548+0000","last_peered":"2023-10-06T05:20:57.087548+0000","last_clean":"2023-10-06T05:20:57.087548+0000","last_became_active":"2023-10-06T05:20:57.086123+0000","last_became_peered":"2023-10-06T05:20:57.086123+0000","last_unstale":"2023-10-06T05:20:57.087548+0000","last_undegraded":"2023-10-06T05:20:57.087548+0000","last_fullsized":"2023-10-06T05:20:57.087548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:27.519437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080553+0000","last_change":"2023-10-06T05:20:56.063062+0000","last_active":"2023-10-06T05:20:57.080553+0000","last_peered":"2023-10-06T05:20:57.080553+0000","last_clean":"2023-10-06T05:20:57.080553+0000","last_became_active":"2023-10-06T05:20:56.062410+0000","last_became_peered":"2023-10-06T05:20:56.062410+0000","last_unstale":"2023-10-06T05:20:57.080553+0000","last_undegraded":"2023-10-06T05:20:57.080553+0000","last_fullsized":"2023-10-06T05:20:57.080553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:49.619842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080392+0000","last_change":"2023-10-06T05:20:57.070130+0000","last_active":"2023-10-06T05:20:57.080392+0000","last_peered":"2023-10-06T05:20:57.080392+0000","last_clean":"2023-10-06T05:20:57.080392+0000","last_became_active":"2023-10-06T05:20:57.067350+0000","last_became_peered":"2023-10-06T05:20:57.067350+0000","last_unstale":"2023-10-06T05:20:57.080392+0000","last_undegraded":"2023-10-06T05:20:57.080392+0000","last_fullsized":"2023-10-06T05:20:57.080392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:36.586844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.087969+0000","last_change":"2023-10-06T05:20:56.062842+0000","last_active":"2023-10-06T05:20:57.087969+0000","last_peered":"2023-10-06T05:20:57.087969+0000","last_clean":"2023-10-06T05:20:57.087969+0000","last_became_active":"2023-10-06T05:20:56.062590+0000","last_became_peered":"2023-10-06T05:20:56.062590+0000","last_unstale":"2023-10-06T05:20:57.087969+0000","last_undegraded":"2023-10-06T05:20:57.087969+0000","last_fullsized":"2023-10-06T05:20:57.087969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:58.961459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.094164+0000","last_change":"2023-10-06T05:20:57.091039+0000","last_active":"2023-10-06T05:20:57.094164+0000","last_peered":"2023-10-06T05:20:57.094164+0000","last_clean":"2023-10-06T05:20:57.094164+0000","last_became_active":"2023-10-06T05:20:57.090511+0000","last_became_peered":"2023-10-06T05:20:57.090511+0000","last_unstale":"2023-10-06T05:20:57.094164+0000","last_undegraded":"2023-10-06T05:20:57.094164+0000","last_fullsized":"2023-10-06T05:20:57.094164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:44.878348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077557+0000","last_change":"2023-10-06T05:20:56.063129+0000","last_active":"2023-10-06T05:20:57.077557+0000","last_peered":"2023-10-06T05:20:57.077557+0000","last_clean":"2023-10-06T05:20:57.077557+0000","last_became_active":"2023-10-06T05:20:56.062230+0000","last_became_peered":"2023-10-06T05:20:56.062230+0000","last_unstale":"2023-10-06T05:20:57.077557+0000","last_undegraded":"2023-10-06T05:20:57.077557+0000","last_fullsized":"2023-10-06T05:20:57.077557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077132+0000","last_change":"2023-10-06T05:20:57.069780+0000","last_active":"2023-10-06T05:20:57.077132+0000","last_peered":"2023-10-06T05:20:57.077132+0000","last_clean":"2023-10-06T05:20:57.077132+0000","last_became_active":"2023-10-06T05:20:57.069501+0000","last_became_peered":"2023-10-06T05:20:57.069501+0000","last_unstale":"2023-10-06T05:20:57.077132+0000","last_undegraded":"2023-10-06T05:20:57.077132+0000","last_fullsized":"2023-10-06T05:20:57.077132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:02.864303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.084951+0000","last_change":"2023-10-06T05:20:56.066379+0000","last_active":"2023-10-06T05:20:57.084951+0000","last_peered":"2023-10-06T05:20:57.084951+0000","last_clean":"2023-10-06T05:20:57.084951+0000","last_became_active":"2023-10-06T05:20:56.064912+0000","last_became_peered":"2023-10-06T05:20:56.064912+0000","last_unstale":"2023-10-06T05:20:57.084951+0000","last_undegraded":"2023-10-06T05:20:57.084951+0000","last_fullsized":"2023-10-06T05:20:57.084951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:23.256874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092566+0000","last_change":"2023-10-06T05:20:57.088844+0000","last_active":"2023-10-06T05:20:57.092566+0000","last_peered":"2023-10-06T05:20:57.092566+0000","last_clean":"2023-10-06T05:20:57.092566+0000","last_became_active":"2023-10-06T05:20:57.088566+0000","last_became_peered":"2023-10-06T05:20:57.088566+0000","last_unstale":"2023-10-06T05:20:57.092566+0000","last_undegraded":"2023-10-06T05:20:57.092566+0000","last_fullsized":"2023-10-06T05:20:57.092566+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:57.144083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.053364+0000","last_change":"2023-10-06T05:20:56.063667+0000","last_active":"2023-10-06T05:20:57.053364+0000","last_peered":"2023-10-06T05:20:57.053364+0000","last_clean":"2023-10-06T05:20:57.053364+0000","last_became_active":"2023-10-06T05:20:56.063346+0000","last_became_peered":"2023-10-06T05:20:56.063346+0000","last_unstale":"2023-10-06T05:20:57.053364+0000","last_undegraded":"2023-10-06T05:20:57.053364+0000","last_fullsized":"2023-10-06T05:20:57.053364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092134+0000","last_change":"2023-10-06T05:20:57.087652+0000","last_active":"2023-10-06T05:20:57.092134+0000","last_peered":"2023-10-06T05:20:57.092134+0000","last_clean":"2023-10-06T05:20:57.092134+0000","last_became_active":"2023-10-06T05:20:57.087394+0000","last_became_peered":"2023-10-06T05:20:57.087394+0000","last_unstale":"2023-10-06T05:20:57.092134+0000","last_undegraded":"2023-10-06T05:20:57.092134+0000","last_fullsized":"2023-10-06T05:20:57.092134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:20.780861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.059461+0000","last_change":"2023-10-06T05:20:56.068185+0000","last_active":"2023-10-06T05:20:57.059461+0000","last_peered":"2023-10-06T05:20:57.059461+0000","last_clean":"2023-10-06T05:20:57.059461+0000","last_became_active":"2023-10-06T05:20:56.067889+0000","last_became_peered":"2023-10-06T05:20:56.067889+0000","last_unstale":"2023-10-06T05:20:57.059461+0000","last_undegraded":"2023-10-06T05:20:57.059461+0000","last_fullsized":"2023-10-06T05:20:57.059461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.066413+0000","last_change":"2023-10-06T05:20:57.063773+0000","last_active":"2023-10-06T05:20:57.066413+0000","last_peered":"2023-10-06T05:20:57.066413+0000","last_clean":"2023-10-06T05:20:57.066413+0000","last_became_active":"2023-10-06T05:20:57.062875+0000","last_became_peered":"2023-10-06T05:20:57.062875+0000","last_unstale":"2023-10-06T05:20:57.066413+0000","last_undegraded":"2023-10-06T05:20:57.066413+0000","last_fullsized":"2023-10-06T05:20:57.066413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:35.006196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.055231+0000","last_change":"2023-10-06T05:20:56.064663+0000","last_active":"2023-10-06T05:20:57.055231+0000","last_peered":"2023-10-06T05:20:57.055231+0000","last_clean":"2023-10-06T05:20:57.055231+0000","last_became_active":"2023-10-06T05:20:56.064382+0000","last_became_peered":"2023-10-06T05:20:56.064382+0000","last_unstale":"2023-10-06T05:20:57.055231+0000","last_undegraded":"2023-10-06T05:20:57.055231+0000","last_fullsized":"2023-10-06T05:20:57.055231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:07.091803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077870+0000","last_change":"2023-10-06T05:20:57.074325+0000","last_active":"2023-10-06T05:20:57.077870+0000","last_peered":"2023-10-06T05:20:57.077870+0000","last_clean":"2023-10-06T05:20:57.077870+0000","last_became_active":"2023-10-06T05:20:57.073871+0000","last_became_peered":"2023-10-06T05:20:57.073871+0000","last_unstale":"2023-10-06T05:20:57.077870+0000","last_undegraded":"2023-10-06T05:20:57.077870+0000","last_fullsized":"2023-10-06T05:20:57.077870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:44.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.054414+0000","last_change":"2023-10-06T05:20:56.062871+0000","last_active":"2023-10-06T05:20:57.054414+0000","last_peered":"2023-10-06T05:20:57.054414+0000","last_clean":"2023-10-06T05:20:57.054414+0000","last_became_active":"2023-10-06T05:20:56.062606+0000","last_became_peered":"2023-10-06T05:20:56.062606+0000","last_unstale":"2023-10-06T05:20:57.054414+0000","last_undegraded":"2023-10-06T05:20:57.054414+0000","last_fullsized":"2023-10-06T05:20:57.054414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.332854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080689+0000","last_change":"2023-10-06T05:20:57.068885+0000","last_active":"2023-10-06T05:20:57.080689+0000","last_peered":"2023-10-06T05:20:57.080689+0000","last_clean":"2023-10-06T05:20:57.080689+0000","last_became_active":"2023-10-06T05:20:57.068623+0000","last_became_peered":"2023-10-06T05:20:57.068623+0000","last_unstale":"2023-10-06T05:20:57.080689+0000","last_undegraded":"2023-10-06T05:20:57.080689+0000","last_fullsized":"2023-10-06T05:20:57.080689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.168546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"88'3","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080786+0000","last_change":"2023-10-06T05:20:56.069433+0000","last_active":"2023-10-06T05:20:57.080786+0000","last_peered":"2023-10-06T05:20:57.080786+0000","last_clean":"2023-10-06T05:20:57.080786+0000","last_became_active":"2023-10-06T05:20:56.069152+0000","last_became_peered":"2023-10-06T05:20:56.069152+0000","last_unstale":"2023-10-06T05:20:57.080786+0000","last_undegraded":"2023-10-06T05:20:57.080786+0000","last_fullsized":"2023-10-06T05:20:57.080786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:46.728922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.082509+0000","last_change":"2023-10-06T05:20:57.076623+0000","last_active":"2023-10-06T05:20:57.082509+0000","last_peered":"2023-10-06T05:20:57.082509+0000","last_clean":"2023-10-06T05:20:57.082509+0000","last_became_active":"2023-10-06T05:20:57.076325+0000","last_became_peered":"2023-10-06T05:20:57.076325+0000","last_unstale":"2023-10-06T05:20:57.082509+0000","last_undegraded":"2023-10-06T05:20:57.082509+0000","last_fullsized":"2023-10-06T05:20:57.082509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:13.105721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"88'6","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092427+0000","last_change":"2023-10-06T05:20:56.066398+0000","last_active":"2023-10-06T05:20:57.092427+0000","last_peered":"2023-10-06T05:20:57.092427+0000","last_clean":"2023-10-06T05:20:57.092427+0000","last_became_active":"2023-10-06T05:20:56.066051+0000","last_became_peered":"2023-10-06T05:20:56.066051+0000","last_unstale":"2023-10-06T05:20:57.092427+0000","last_undegraded":"2023-10-06T05:20:57.092427+0000","last_fullsized":"2023-10-06T05:20:57.092427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:26.538710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.085064+0000","last_change":"2023-10-06T05:20:57.081501+0000","last_active":"2023-10-06T05:20:57.085064+0000","last_peered":"2023-10-06T05:20:57.085064+0000","last_clean":"2023-10-06T05:20:57.085064+0000","last_became_active":"2023-10-06T05:20:57.078646+0000","last_became_peered":"2023-10-06T05:20:57.078646+0000","last_unstale":"2023-10-06T05:20:57.085064+0000","last_undegraded":"2023-10-06T05:20:57.085064+0000","last_fullsized":"2023-10-06T05:20:57.085064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:48.811645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080434+0000","last_change":"2023-10-06T05:20:56.060068+0000","last_active":"2023-10-06T05:20:57.080434+0000","last_peered":"2023-10-06T05:20:57.080434+0000","last_clean":"2023-10-06T05:20:57.080434+0000","last_became_active":"2023-10-06T05:20:56.059765+0000","last_became_peered":"2023-10-06T05:20:56.059765+0000","last_unstale":"2023-10-06T05:20:57.080434+0000","last_undegraded":"2023-10-06T05:20:57.080434+0000","last_fullsized":"2023-10-06T05:20:57.080434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:56.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077670+0000","last_change":"2023-10-06T05:20:57.069540+0000","last_active":"2023-10-06T05:20:57.077670+0000","last_peered":"2023-10-06T05:20:57.077670+0000","last_clean":"2023-10-06T05:20:57.077670+0000","last_became_active":"2023-10-06T05:20:57.069181+0000","last_became_peered":"2023-10-06T05:20:57.069181+0000","last_unstale":"2023-10-06T05:20:57.077670+0000","last_undegraded":"2023-10-06T05:20:57.077670+0000","last_fullsized":"2023-10-06T05:20:57.077670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:59.430802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.054797+0000","last_change":"2023-10-06T05:20:56.062607+0000","last_active":"2023-10-06T05:20:57.054797+0000","last_peered":"2023-10-06T05:20:57.054797+0000","last_clean":"2023-10-06T05:20:57.054797+0000","last_became_active":"2023-10-06T05:20:56.062344+0000","last_became_peered":"2023-10-06T05:20:56.062344+0000","last_unstale":"2023-10-06T05:20:57.054797+0000","last_undegraded":"2023-10-06T05:20:57.054797+0000","last_fullsized":"2023-10-06T05:20:57.054797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:45.897142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.081888+0000","last_change":"2023-10-06T05:20:57.079012+0000","last_active":"2023-10-06T05:20:57.081888+0000","last_peered":"2023-10-06T05:20:57.081888+0000","last_clean":"2023-10-06T05:20:57.081888+0000","last_became_active":"2023-10-06T05:20:57.078716+0000","last_became_peered":"2023-10-06T05:20:57.078716+0000","last_unstale":"2023-10-06T05:20:57.081888+0000","last_undegraded":"2023-10-06T05:20:57.081888+0000","last_fullsized":"2023-10-06T05:20:57.081888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:11.020411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.056176+0000","last_change":"2023-10-06T05:20:56.067504+0000","last_active":"2023-10-06T05:20:57.056176+0000","last_peered":"2023-10-06T05:20:57.056176+0000","last_clean":"2023-10-06T05:20:57.056176+0000","last_became_active":"2023-10-06T05:20:56.066985+0000","last_became_peered":"2023-10-06T05:20:56.066985+0000","last_unstale":"2023-10-06T05:20:57.056176+0000","last_undegraded":"2023-10-06T05:20:57.056176+0000","last_fullsized":"2023-10-06T05:20:57.056176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:43.665749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.064153+0000","last_change":"2023-10-06T05:20:57.062128+0000","last_active":"2023-10-06T05:20:57.064153+0000","last_peered":"2023-10-06T05:20:57.064153+0000","last_clean":"2023-10-06T05:20:57.064153+0000","last_became_active":"2023-10-06T05:20:57.061855+0000","last_became_peered":"2023-10-06T05:20:57.061855+0000","last_unstale":"2023-10-06T05:20:57.064153+0000","last_undegraded":"2023-10-06T05:20:57.064153+0000","last_fullsized":"2023-10-06T05:20:57.064153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:36.347846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"88'8","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.090258+0000","last_change":"2023-10-06T05:20:56.063906+0000","last_active":"2023-10-06T05:20:57.090258+0000","last_peered":"2023-10-06T05:20:57.090258+0000","last_clean":"2023-10-06T05:20:57.090258+0000","last_became_active":"2023-10-06T05:20:56.063625+0000","last_became_peered":"2023-10-06T05:20:56.063625+0000","last_unstale":"2023-10-06T05:20:57.090258+0000","last_undegraded":"2023-10-06T05:20:57.090258+0000","last_fullsized":"2023-10-06T05:20:57.090258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:07.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.072979+0000","last_change":"2023-10-06T05:20:57.068602+0000","last_active":"2023-10-06T05:20:57.072979+0000","last_peered":"2023-10-06T05:20:57.072979+0000","last_clean":"2023-10-06T05:20:57.072979+0000","last_became_active":"2023-10-06T05:20:57.068329+0000","last_became_peered":"2023-10-06T05:20:57.068329+0000","last_unstale":"2023-10-06T05:20:57.072979+0000","last_undegraded":"2023-10-06T05:20:57.072979+0000","last_fullsized":"2023-10-06T05:20:57.072979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:44.674711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.078676+0000","last_change":"2023-10-06T05:20:56.060664+0000","last_active":"2023-10-06T05:20:57.078676+0000","last_peered":"2023-10-06T05:20:57.078676+0000","last_clean":"2023-10-06T05:20:57.078676+0000","last_became_active":"2023-10-06T05:20:56.059179+0000","last_became_peered":"2023-10-06T05:20:56.059179+0000","last_unstale":"2023-10-06T05:20:57.078676+0000","last_undegraded":"2023-10-06T05:20:57.078676+0000","last_fullsized":"2023-10-06T05:20:57.078676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.078805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.080091+0000","last_change":"2023-10-06T05:20:57.071773+0000","last_active":"2023-10-06T05:20:57.080091+0000","last_peered":"2023-10-06T05:20:57.080091+0000","last_clean":"2023-10-06T05:20:57.080091+0000","last_became_active":"2023-10-06T05:20:57.071434+0000","last_became_peered":"2023-10-06T05:20:57.071434+0000","last_unstale":"2023-10-06T05:20:57.080091+0000","last_undegraded":"2023-10-06T05:20:57.080091+0000","last_fullsized":"2023-10-06T05:20:57.080091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:45.277443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T05:20:59.542462+0000","last_change":"2023-10-06T05:20:56.069531+0000","last_active":"2023-10-06T05:20:59.542462+0000","last_peered":"2023-10-06T05:20:59.542462+0000","last_clean":"2023-10-06T05:20:59.542462+0000","last_became_active":"2023-10-06T05:20:56.068428+0000","last_became_peered":"2023-10-06T05:20:56.068428+0000","last_unstale":"2023-10-06T05:20:59.542462+0000","last_undegraded":"2023-10-06T05:20:59.542462+0000","last_fullsized":"2023-10-06T05:20:59.542462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.079599+0000","last_change":"2023-10-06T05:20:57.069328+0000","last_active":"2023-10-06T05:20:57.079599+0000","last_peered":"2023-10-06T05:20:57.079599+0000","last_clean":"2023-10-06T05:20:57.079599+0000","last_became_active":"2023-10-06T05:20:57.067266+0000","last_became_peered":"2023-10-06T05:20:57.067266+0000","last_unstale":"2023-10-06T05:20:57.079599+0000","last_undegraded":"2023-10-06T05:20:57.079599+0000","last_fullsized":"2023-10-06T05:20:57.079599+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:45.920475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.056356+0000","last_change":"2023-10-06T05:20:56.069310+0000","last_active":"2023-10-06T05:20:57.056356+0000","last_peered":"2023-10-06T05:20:57.056356+0000","last_clean":"2023-10-06T05:20:57.056356+0000","last_became_active":"2023-10-06T05:20:56.069031+0000","last_became_peered":"2023-10-06T05:20:56.069031+0000","last_unstale":"2023-10-06T05:20:57.056356+0000","last_undegraded":"2023-10-06T05:20:57.056356+0000","last_fullsized":"2023-10-06T05:20:57.056356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:58.677893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.072395+0000","last_change":"2023-10-06T05:20:57.068222+0000","last_active":"2023-10-06T05:20:57.072395+0000","last_peered":"2023-10-06T05:20:57.072395+0000","last_clean":"2023-10-06T05:20:57.072395+0000","last_became_active":"2023-10-06T05:20:57.067939+0000","last_became_peered":"2023-10-06T05:20:57.067939+0000","last_unstale":"2023-10-06T05:20:57.072395+0000","last_undegraded":"2023-10-06T05:20:57.072395+0000","last_fullsized":"2023-10-06T05:20:57.072395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:47.737884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.051835+0000","last_change":"2023-10-06T05:20:56.066147+0000","last_active":"2023-10-06T05:20:57.051835+0000","last_peered":"2023-10-06T05:20:57.051835+0000","last_clean":"2023-10-06T05:20:57.051835+0000","last_became_active":"2023-10-06T05:20:56.065618+0000","last_became_peered":"2023-10-06T05:20:56.065618+0000","last_unstale":"2023-10-06T05:20:57.051835+0000","last_undegraded":"2023-10-06T05:20:57.051835+0000","last_fullsized":"2023-10-06T05:20:57.051835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:33.934156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.075138+0000","last_change":"2023-10-06T05:20:57.071605+0000","last_active":"2023-10-06T05:20:57.075138+0000","last_peered":"2023-10-06T05:20:57.075138+0000","last_clean":"2023-10-06T05:20:57.075138+0000","last_became_active":"2023-10-06T05:20:57.071072+0000","last_became_peered":"2023-10-06T05:20:57.071072+0000","last_unstale":"2023-10-06T05:20:57.075138+0000","last_undegraded":"2023-10-06T05:20:57.075138+0000","last_fullsized":"2023-10-06T05:20:57.075138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:41.213230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.057867+0000","last_change":"2023-10-06T05:20:56.062131+0000","last_active":"2023-10-06T05:20:57.057867+0000","last_peered":"2023-10-06T05:20:57.057867+0000","last_clean":"2023-10-06T05:20:57.057867+0000","last_became_active":"2023-10-06T05:20:56.061821+0000","last_became_peered":"2023-10-06T05:20:56.061821+0000","last_unstale":"2023-10-06T05:20:57.057867+0000","last_undegraded":"2023-10-06T05:20:57.057867+0000","last_fullsized":"2023-10-06T05:20:57.057867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:33.997234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.089901+0000","last_change":"2023-10-06T05:20:57.083318+0000","last_active":"2023-10-06T05:20:57.089901+0000","last_peered":"2023-10-06T05:20:57.089901+0000","last_clean":"2023-10-06T05:20:57.089901+0000","last_became_active":"2023-10-06T05:20:57.083082+0000","last_became_peered":"2023-10-06T05:20:57.083082+0000","last_unstale":"2023-10-06T05:20:57.089901+0000","last_undegraded":"2023-10-06T05:20:57.089901+0000","last_fullsized":"2023-10-06T05:20:57.089901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.367162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.055691+0000","last_change":"2023-10-06T05:20:56.067790+0000","last_active":"2023-10-06T05:20:57.055691+0000","last_peered":"2023-10-06T05:20:57.055691+0000","last_clean":"2023-10-06T05:20:57.055691+0000","last_became_active":"2023-10-06T05:20:56.067491+0000","last_became_peered":"2023-10-06T05:20:56.067491+0000","last_unstale":"2023-10-06T05:20:57.055691+0000","last_undegraded":"2023-10-06T05:20:57.055691+0000","last_fullsized":"2023-10-06T05:20:57.055691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:06.378982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.084664+0000","last_change":"2023-10-06T05:20:57.083476+0000","last_active":"2023-10-06T05:20:57.084664+0000","last_peered":"2023-10-06T05:20:57.084664+0000","last_clean":"2023-10-06T05:20:57.084664+0000","last_became_active":"2023-10-06T05:20:57.083203+0000","last_became_peered":"2023-10-06T05:20:57.083203+0000","last_unstale":"2023-10-06T05:20:57.084664+0000","last_undegraded":"2023-10-06T05:20:57.084664+0000","last_fullsized":"2023-10-06T05:20:57.084664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:24.265835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.052093+0000","last_change":"2023-10-06T05:20:56.065948+0000","last_active":"2023-10-06T05:20:57.052093+0000","last_peered":"2023-10-06T05:20:57.052093+0000","last_clean":"2023-10-06T05:20:57.052093+0000","last_became_active":"2023-10-06T05:20:56.065640+0000","last_became_peered":"2023-10-06T05:20:56.065640+0000","last_unstale":"2023-10-06T05:20:57.052093+0000","last_undegraded":"2023-10-06T05:20:57.052093+0000","last_fullsized":"2023-10-06T05:20:57.052093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:17.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.079804+0000","last_change":"2023-10-06T05:20:57.071567+0000","last_active":"2023-10-06T05:20:57.079804+0000","last_peered":"2023-10-06T05:20:57.079804+0000","last_clean":"2023-10-06T05:20:57.079804+0000","last_became_active":"2023-10-06T05:20:57.067185+0000","last_became_peered":"2023-10-06T05:20:57.067185+0000","last_unstale":"2023-10-06T05:20:57.079804+0000","last_undegraded":"2023-10-06T05:20:57.079804+0000","last_fullsized":"2023-10-06T05:20:57.079804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:47.367162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.059561+0000","last_change":"2023-10-06T05:20:56.063353+0000","last_active":"2023-10-06T05:20:57.059561+0000","last_peered":"2023-10-06T05:20:57.059561+0000","last_clean":"2023-10-06T05:20:57.059561+0000","last_became_active":"2023-10-06T05:20:56.063086+0000","last_became_peered":"2023-10-06T05:20:56.063086+0000","last_unstale":"2023-10-06T05:20:57.059561+0000","last_undegraded":"2023-10-06T05:20:57.059561+0000","last_fullsized":"2023-10-06T05:20:57.059561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:59.973368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.091653+0000","last_change":"2023-10-06T05:20:57.088812+0000","last_active":"2023-10-06T05:20:57.091653+0000","last_peered":"2023-10-06T05:20:57.091653+0000","last_clean":"2023-10-06T05:20:57.091653+0000","last_became_active":"2023-10-06T05:20:57.088510+0000","last_became_peered":"2023-10-06T05:20:57.088510+0000","last_unstale":"2023-10-06T05:20:57.091653+0000","last_undegraded":"2023-10-06T05:20:57.091653+0000","last_fullsized":"2023-10-06T05:20:57.091653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:19.366982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.089605+0000","last_change":"2023-10-06T05:20:56.066899+0000","last_active":"2023-10-06T05:20:57.089605+0000","last_peered":"2023-10-06T05:20:57.089605+0000","last_clean":"2023-10-06T05:20:57.089605+0000","last_became_active":"2023-10-06T05:20:56.066645+0000","last_became_peered":"2023-10-06T05:20:56.066645+0000","last_unstale":"2023-10-06T05:20:57.089605+0000","last_undegraded":"2023-10-06T05:20:57.089605+0000","last_fullsized":"2023-10-06T05:20:57.089605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:38.055535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.069261+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:20:57.069261+0000","last_peered":"2023-10-06T05:20:57.069261+0000","last_clean":"2023-10-06T05:20:57.069261+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:20:57.069261+0000","last_undegraded":"2023-10-06T05:20:57.069261+0000","last_fullsized":"2023-10-06T05:20:57.069261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.070033+0000","last_change":"2023-10-06T05:20:57.063825+0000","last_active":"2023-10-06T05:20:57.070033+0000","last_peered":"2023-10-06T05:20:57.070033+0000","last_clean":"2023-10-06T05:20:57.070033+0000","last_became_active":"2023-10-06T05:20:57.063529+0000","last_became_peered":"2023-10-06T05:20:57.063529+0000","last_unstale":"2023-10-06T05:20:57.070033+0000","last_undegraded":"2023-10-06T05:20:57.070033+0000","last_fullsized":"2023-10-06T05:20:57.070033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:42.458155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.092684+0000","last_change":"2023-10-06T05:20:56.064164+0000","last_active":"2023-10-06T05:20:57.092684+0000","last_peered":"2023-10-06T05:20:57.092684+0000","last_clean":"2023-10-06T05:20:57.092684+0000","last_became_active":"2023-10-06T05:20:56.063907+0000","last_became_peered":"2023-10-06T05:20:56.063907+0000","last_unstale":"2023-10-06T05:20:57.092684+0000","last_undegraded":"2023-10-06T05:20:57.092684+0000","last_fullsized":"2023-10-06T05:20:57.092684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:16.581921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.055594+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:20:57.055594+0000","last_peered":"2023-10-06T05:20:57.055594+0000","last_clean":"2023-10-06T05:20:57.055594+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:20:57.055594+0000","last_undegraded":"2023-10-06T05:20:57.055594+0000","last_fullsized":"2023-10-06T05:20:57.055594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.060265+0000","last_change":"2023-10-06T05:20:57.057312+0000","last_active":"2023-10-06T05:20:57.060265+0000","last_peered":"2023-10-06T05:20:57.060265+0000","last_clean":"2023-10-06T05:20:57.060265+0000","last_became_active":"2023-10-06T05:20:57.056965+0000","last_became_peered":"2023-10-06T05:20:57.056965+0000","last_unstale":"2023-10-06T05:20:57.060265+0000","last_undegraded":"2023-10-06T05:20:57.060265+0000","last_fullsized":"2023-10-06T05:20:57.060265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:09.986792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.078044+0000","last_change":"2023-10-06T05:20:56.062639+0000","last_active":"2023-10-06T05:20:57.078044+0000","last_peered":"2023-10-06T05:20:57.078044+0000","last_clean":"2023-10-06T05:20:57.078044+0000","last_became_active":"2023-10-06T05:20:56.062307+0000","last_became_peered":"2023-10-06T05:20:56.062307+0000","last_unstale":"2023-10-06T05:20:57.078044+0000","last_undegraded":"2023-10-06T05:20:57.078044+0000","last_fullsized":"2023-10-06T05:20:57.078044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:03.718762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.048780+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:20:57.048780+0000","last_peered":"2023-10-06T05:20:57.048780+0000","last_clean":"2023-10-06T05:20:57.048780+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:20:57.048780+0000","last_undegraded":"2023-10-06T05:20:57.048780+0000","last_fullsized":"2023-10-06T05:20:57.048780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077088+0000","last_change":"2023-10-06T05:20:57.073058+0000","last_active":"2023-10-06T05:20:57.077088+0000","last_peered":"2023-10-06T05:20:57.077088+0000","last_clean":"2023-10-06T05:20:57.077088+0000","last_became_active":"2023-10-06T05:20:57.072761+0000","last_became_peered":"2023-10-06T05:20:57.072761+0000","last_unstale":"2023-10-06T05:20:57.077088+0000","last_undegraded":"2023-10-06T05:20:57.077088+0000","last_fullsized":"2023-10-06T05:20:57.077088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:31.168546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.069166+0000","last_change":"2023-10-06T05:20:56.067383+0000","last_active":"2023-10-06T05:20:57.069166+0000","last_peered":"2023-10-06T05:20:57.069166+0000","last_clean":"2023-10-06T05:20:57.069166+0000","last_became_active":"2023-10-06T05:20:56.067105+0000","last_became_peered":"2023-10-06T05:20:56.067105+0000","last_unstale":"2023-10-06T05:20:57.069166+0000","last_undegraded":"2023-10-06T05:20:57.069166+0000","last_fullsized":"2023-10-06T05:20:57.069166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:58.649017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.054702+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:20:57.054702+0000","last_peered":"2023-10-06T05:20:57.054702+0000","last_clean":"2023-10-06T05:20:57.054702+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:20:57.054702+0000","last_undegraded":"2023-10-06T05:20:57.054702+0000","last_fullsized":"2023-10-06T05:20:57.054702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":223,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.059736+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:20:57.059736+0000","last_peered":"2023-10-06T05:20:57.059736+0000","last_clean":"2023-10-06T05:20:57.059736+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:20:57.059736+0000","last_undegraded":"2023-10-06T05:20:57.059736+0000","last_fullsized":"2023-10-06T05:20:57.059736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.081369+0000","last_change":"2023-10-06T05:20:57.072155+0000","last_active":"2023-10-06T05:20:57.081369+0000","last_peered":"2023-10-06T05:20:57.081369+0000","last_clean":"2023-10-06T05:20:57.081369+0000","last_became_active":"2023-10-06T05:20:57.071920+0000","last_became_peered":"2023-10-06T05:20:57.071920+0000","last_unstale":"2023-10-06T05:20:57.081369+0000","last_undegraded":"2023-10-06T05:20:57.081369+0000","last_fullsized":"2023-10-06T05:20:57.081369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:58.562901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.077348+0000","last_change":"2023-10-06T05:20:56.066067+0000","last_active":"2023-10-06T05:20:57.077348+0000","last_peered":"2023-10-06T05:20:57.077348+0000","last_clean":"2023-10-06T05:20:57.077348+0000","last_became_active":"2023-10-06T05:20:56.065696+0000","last_became_peered":"2023-10-06T05:20:56.065696+0000","last_unstale":"2023-10-06T05:20:57.077348+0000","last_undegraded":"2023-10-06T05:20:57.077348+0000","last_fullsized":"2023-10-06T05:20:57.077348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:03.468905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.059644+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:20:57.059644+0000","last_peered":"2023-10-06T05:20:57.059644+0000","last_clean":"2023-10-06T05:20:57.059644+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:20:57.059644+0000","last_undegraded":"2023-10-06T05:20:57.059644+0000","last_fullsized":"2023-10-06T05:20:57.059644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.067192+0000","last_change":"2023-10-06T05:20:57.064749+0000","last_active":"2023-10-06T05:20:57.067192+0000","last_peered":"2023-10-06T05:20:57.067192+0000","last_clean":"2023-10-06T05:20:57.067192+0000","last_became_active":"2023-10-06T05:20:57.064247+0000","last_became_peered":"2023-10-06T05:20:57.064247+0000","last_unstale":"2023-10-06T05:20:57.067192+0000","last_undegraded":"2023-10-06T05:20:57.067192+0000","last_fullsized":"2023-10-06T05:20:57.067192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:50.628803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.049010+0000","last_change":"2023-10-06T05:20:56.063757+0000","last_active":"2023-10-06T05:20:57.049010+0000","last_peered":"2023-10-06T05:20:57.049010+0000","last_clean":"2023-10-06T05:20:57.049010+0000","last_became_active":"2023-10-06T05:20:56.063497+0000","last_became_peered":"2023-10-06T05:20:56.063497+0000","last_unstale":"2023-10-06T05:20:57.049010+0000","last_undegraded":"2023-10-06T05:20:57.049010+0000","last_fullsized":"2023-10-06T05:20:57.049010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:07.370987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.060774+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:20:57.060774+0000","last_peered":"2023-10-06T05:20:57.060774+0000","last_clean":"2023-10-06T05:20:57.060774+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:20:57.060774+0000","last_undegraded":"2023-10-06T05:20:57.060774+0000","last_fullsized":"2023-10-06T05:20:57.060774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.090131+0000","last_change":"2023-10-06T05:20:57.088809+0000","last_active":"2023-10-06T05:20:57.090131+0000","last_peered":"2023-10-06T05:20:57.090131+0000","last_clean":"2023-10-06T05:20:57.090131+0000","last_became_active":"2023-10-06T05:20:57.088476+0000","last_became_peered":"2023-10-06T05:20:57.088476+0000","last_unstale":"2023-10-06T05:20:57.090131+0000","last_undegraded":"2023-10-06T05:20:57.090131+0000","last_fullsized":"2023-10-06T05:20:57.090131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:00.982330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.055902+0000","last_change":"2023-10-06T05:20:56.067795+0000","last_active":"2023-10-06T05:20:57.055902+0000","last_peered":"2023-10-06T05:20:57.055902+0000","last_clean":"2023-10-06T05:20:57.055902+0000","last_became_active":"2023-10-06T05:20:56.067515+0000","last_became_peered":"2023-10-06T05:20:56.067515+0000","last_unstale":"2023-10-06T05:20:57.055902+0000","last_undegraded":"2023-10-06T05:20:57.055902+0000","last_fullsized":"2023-10-06T05:20:57.055902+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:17.822671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.069755+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:20:57.069755+0000","last_peered":"2023-10-06T05:20:57.069755+0000","last_clean":"2023-10-06T05:20:57.069755+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:20:57.069755+0000","last_undegraded":"2023-10-06T05:20:57.069755+0000","last_fullsized":"2023-10-06T05:20:57.069755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.079833+0000","last_change":"2023-10-06T05:20:57.072106+0000","last_active":"2023-10-06T05:20:57.079833+0000","last_peered":"2023-10-06T05:20:57.079833+0000","last_clean":"2023-10-06T05:20:57.079833+0000","last_became_active":"2023-10-06T05:20:57.071808+0000","last_became_peered":"2023-10-06T05:20:57.071808+0000","last_unstale":"2023-10-06T05:20:57.079833+0000","last_undegraded":"2023-10-06T05:20:57.079833+0000","last_fullsized":"2023-10-06T05:20:57.079833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:56.037124+0000","last_clean_scrub_stamp":"2023-10-06T05:20:56.037124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:07.201913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.048721+0000","last_change":"2023-10-06T05:20:56.066251+0000","last_active":"2023-10-06T05:20:57.048721+0000","last_peered":"2023-10-06T05:20:57.048721+0000","last_clean":"2023-10-06T05:20:57.048721+0000","last_became_active":"2023-10-06T05:20:56.065923+0000","last_became_peered":"2023-10-06T05:20:56.065923+0000","last_unstale":"2023-10-06T05:20:57.048721+0000","last_undegraded":"2023-10-06T05:20:57.048721+0000","last_fullsized":"2023-10-06T05:20:57.048721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:55.028162+0000","last_clean_scrub_stamp":"2023-10-06T05:20:55.028162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:09.868032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T05:20:57.052416+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:20:57.052416+0000","last_peered":"2023-10-06T05:20:57.052416+0000","last_clean":"2023-10-06T05:20:57.052416+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:20:57.052416+0000","last_undegraded":"2023-10-06T05:20:57.052416+0000","last_fullsized":"2023-10-06T05:20:57.052416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303040,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1695744,"data_stored":1084741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99099999999999999,"5min":1.169,"15min":1.169},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.628},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":1.268,"15min":1.268},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.075,"15min":1.075},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.68799999999999994},"min":{"1min":0.57199999999999995,"5min":0.437,"15min":0.437},"max":{"1min":0.995,"5min":1.093,"15min":1.093},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.68700000000000006},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":1.22,"15min":1.22},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0349999999999999,"15min":1.079},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.57099999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":1.117,"15min":1.202},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.622},"min":{"1min":0.47199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.875,"5min":1.258,"15min":1.258},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0069999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.64500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97599999999999998,"5min":1.2050000000000001,"15min":1.407},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.92500000000000004,"5min":1.139,"15min":1.139},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.996,"5min":1.24,"15min":1.3300000000000001},"last":0.68000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":496429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.034},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70099999999999996,"15min":0.72499999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":1.008,"15min":1.2869999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.1599999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.76200000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91500000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.67700000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.94099999999999995,"15min":0.95699999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82899999999999996,"5min":0.97799999999999998,"15min":0.998},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":1.0760000000000001},"last":0.751},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.68999999999999995,"15min":0.77900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.77900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":0.97999999999999998,"15min":1.038},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73099999999999998,"15min":0.84399999999999997},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302456,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069384,"statfs":{"total":96636764160,"available":96327049216,"internally_reserved":0,"allocated":1097728,"data_stored":494387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.442,"15min":0.52500000000000002},"min":{"1min":0.30599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60099999999999998,"5min":0.73099999999999998,"15min":2.7349999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48499999999999999,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54300000000000004,"5min":0.71799999999999997,"15min":2.6520000000000001},"last":0.438},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53600000000000003,"15min":0.61099999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.372}]},{"osd":3,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.85299999999999998,"15min":2.7639999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53400000000000003,"15min":0.61399999999999999},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.82699999999999996,"15min":2.8370000000000002},"last":0.39000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.754,"15min":0.80000000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.50700000000000001},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":1.2010000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.125,"15min":1.2929999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.751,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93200000000000005,"5min":1.0289999999999999,"15min":1.349},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.82899999999999996},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":1.1419999999999999,"15min":1.669},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80200000000000005,"15min":0.86399999999999999},"min":{"1min":0.66800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91900000000000004,"5min":1.1779999999999999,"15min":1.6040000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.88700000000000001},"min":{"1min":0.61499999999999999,"5min":0.503,"15min":0.503},"max":{"1min":0.96999999999999997,"5min":1.165,"15min":1.573},"last":0.86499999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":1.0780000000000001,"15min":1.528},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91400000000000003,"5min":1.105,"15min":1.75},"last":0.84599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1712128,"data_stored":1086899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46200000000000002,"15min":0.442},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64600000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73199999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.37,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67000000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77300000000000002,"5min":0.879,"15min":0.879},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.75700000000000001},"min":{"1min":0.65200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77900000000000003,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0069999999999999,"5min":1.218,"15min":1.218},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.749}]},{"osd":6,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.79400000000000004},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.05,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.80300000000000005},"min":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.63700000000000001,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0149999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1093632,"data_stored":494317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.46200000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.42699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48699999999999999,"15min":0.47399999999999998},"min":{"1min":0.42599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.748,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.54500000000000004},"max":{"1min":1.0329999999999999,"5min":1.077,"15min":1.1379999999999999},"last":0.874},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.47499999999999998},"max":{"1min":0.98699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.495},"max":{"1min":1.04,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.319},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81399999999999995,"15min":0.79200000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.103,"5min":1.103,"15min":1.141},"last":0.90100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52100000000000002},"max":{"1min":1.0609999999999999,"5min":1.1000000000000001,"15min":1.282},"last":0.751},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.68100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.3520000000000001},"last":0.85499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1093632,"data_stored":494317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.70799999999999996,"5min":1.0720000000000001,"15min":2.0950000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":1.772},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.78100000000000003,"15min":1.913},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.66800000000000004,"5min":0.84399999999999997,"15min":2.0089999999999999},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55400000000000005,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.82399999999999995,"15min":1.954},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0469999999999999,"5min":1.2030000000000001,"15min":1.282},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.79900000000000004},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.0669999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.85599999999999998},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1399999999999999,"5min":1.2250000000000001,"15min":1.3009999999999999},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.1910000000000001,"15min":1.343},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.3109999999999999,"15min":1.3380000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82599999999999996,"15min":0.85999999999999999},"min":{"1min":0.63900000000000001,"5min":0.58199999999999996,"15min":0.56799999999999995},"max":{"1min":1.1140000000000001,"5min":1.206,"15min":1.3640000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.68700000000000006,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1539999999999999,"5min":1.2090000000000001,"15min":1.214},"last":0.875}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1101824,"data_stored":494477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.745,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93899999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76600000000000001,"15min":0.76000000000000001},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0269999999999999,"15min":1.169},"last":0.73399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.042,"15min":1.0569999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.117},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.002},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.749,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.196},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63500000000000001,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.8740000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66100000000000003,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.375},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98799999999999999,"15min":1.004},"last":0.879},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95699999999999996,"5min":1.1639999999999999,"15min":1.9390000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71199999999999997,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93300000000000005,"5min":1.0209999999999999,"15min":1.466},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.83999999999999997},"min":{"1min":0.626,"5min":0.499,"15min":0.499},"max":{"1min":0.91800000000000004,"5min":1.1930000000000001,"15min":1.218},"last":0.82999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302452,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069388,"statfs":{"total":96636764160,"available":96327053312,"internally_reserved":0,"allocated":1093632,"data_stored":494317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90800000000000003,"5min":1.165,"15min":1.165},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98999999999999999,"5min":1.016,"15min":1.1890000000000001},"last":0.81999999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.69299999999999995},"min":{"1min":0.59899999999999998,"5min":0.45100000000000001,"15min":0.438},"max":{"1min":0.88200000000000001,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.751},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.69399999999999995},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.038,"15min":1.038},"last":0.84599999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":1.204,"15min":1.204},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.70699999999999996},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.94899999999999995,"5min":0.995,"15min":1.0549999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.087},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.58499999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.97299999999999998,"5min":1.2769999999999999,"15min":1.278},"last":0.876}]},{"osd":4,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.621},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64900000000000002,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.996,"15min":1.294},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64400000000000002,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.141,"15min":1.141},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.876,"5min":0.999,"15min":1.254},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71899999999999997,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.042,"15min":1.337},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.55300000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.248,"15min":1.248},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:06.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:21:06.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:21:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.057+0000 7fa7d599d700 1 Processor -- start 2023-10-06T05:21:07.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.057+0000 7fa7d599d700 1 -- start start 2023-10-06T05:21:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d599d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7d00c7250 0x7fa7d00675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d599d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d006a3b0 con 0x7fa7d0159730 2023-10-06T05:21:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d599d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00687e0 con 0x7fa7d00c7250 2023-10-06T05:21:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d0068220 con 0x7fa7d006b850 2023-10-06T05:21:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d499b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa7d0159730 0x7fa7d00c6b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56088/0 (socket says 172.21.15.33:56088) 2023-10-06T05:21:07.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.058+0000 7fa7d499b700 1 -- 172.21.15.33:0/1233159186 learned_addr learned my addr 172.21.15.33:0/1233159186 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:07.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7d519c700 1 --2- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cffff700 1 --2- 172.21.15.33:0/1233159186 >> v2:172.21.15.167:3300/0 conn(0x7fa7d00c7250 0x7fa7d00675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cf7fe700 1 -- 172.21.15.33:0/1233159186 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774854513 0 0) 0x7fa7d006a3b0 con 0x7fa7d0159730 2023-10-06T05:21:07.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cf7fe700 1 -- 172.21.15.33:0/1233159186 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c0003570 con 0x7fa7d0159730 2023-10-06T05:21:07.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cffff700 1 -- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 msgr2=0x7fa7d0067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:07.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cffff700 1 --2- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0067b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cffff700 1 -- 172.21.15.33:0/1233159186 >> v1:172.21.15.33:6789/0 conn(0x7fa7d0159730 legacy=0x7fa7d00c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cffff700 1 -- 172.21.15.33:0/1233159186 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d0068500 con 0x7fa7d00c7250 2023-10-06T05:21:07.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cf7fe700 1 -- 172.21.15.33:0/1233159186 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1608988115 0 0) 0x7fa7c0003570 con 0x7fa7d0159730 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.059+0000 7fa7cffff700 1 --2- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d00c7250 0x7fa7d00675f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098e0 tx=0x7fa7bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57bed06681ce11bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7cf7fe700 1 -- 172.21.15.33:0/1233159186 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4001470 con 0x7fa7d00c7250 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7cf7fe700 1 -- 172.21.15.33:0/1233159186 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc011420 con 0x7fa7d00c7250 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7cf7fe700 1 -- 172.21.15.33:0/1233159186 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0117c0 con 0x7fa7d00c7250 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 -- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d00c7250 msgr2=0x7fa7d00675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d00c7250 0x7fa7d00675f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098e0 tx=0x7fa7bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 -- 172.21.15.33:0/1233159186 shutdown_connections 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1233159186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d00c7250 0x7fa7d00675f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 -- 172.21.15.33:0/1233159186 >> 172.21.15.33:0/1233159186 conn(0x7fa7d006f0e0 msgr2=0x7fa7d00cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 -- 172.21.15.33:0/1233159186 shutdown_connections 2023-10-06T05:21:07.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.060+0000 7fa7d599d700 1 -- 172.21.15.33:0/1233159186 wait complete. 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.062+0000 7fa7d599d700 1 Processor -- start 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.062+0000 7fa7d599d700 1 -- start start 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d599d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 0x7fa7d00cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d599d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d0159730 0x7fa7d00ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d599d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00cb9d0 con 0x7fa7d00c7250 2023-10-06T05:21:07.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d599d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00ca870 con 0x7fa7d0159730 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00cbbe0 con 0x7fa7d006b850 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 0x7fa7d00cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 0x7fa7d00cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44060/0 (socket says 172.21.15.33:44060) 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7cffff700 1 -- 172.21.15.33:0/1596946937 learned_addr learned my addr 172.21.15.33:0/1596946937 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.063+0000 7fa7d499b700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7d519c700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d0159730 0x7fa7d00ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7cffff700 1 -- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 msgr2=0x7fa7d0151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7cffff700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7cffff700 1 -- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d0159730 msgr2=0x7fa7d00ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7cffff700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d0159730 0x7fa7d00ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7cffff700 1 -- 172.21.15.33:0/1596946937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009420 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7cffff700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 0x7fa7d00cb490 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0093f0 tx=0x7fa7bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc01cc90 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc01cdf0 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d015b740 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc01a660 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.064+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d015bd00 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.065+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d004c380 con 0x7fa7d00c7250 2023-10-06T05:21:07.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.066+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7bc01aa50 con 0x7fa7d00c7250 2023-10-06T05:21:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.067+0000 7fa7ccff9700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.067+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fa7bc085d60 con 0x7fa7d00c7250 2023-10-06T05:21:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.067+0000 7fa7d499b700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.068+0000 7fa7d499b700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7b4066f30 0x7fa7b40693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa7d01527d0 tx=0x7fa7c4001120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.071+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7bc052e10 con 0x7fa7d00c7250 2023-10-06T05:21:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.353+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa7d00cbf70 con 0x7fa7d00c7250 2023-10-06T05:21:07.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.357+0000 7fa7ccff9700 1 -- 172.21.15.33:0/1596946937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa7bc0290e0 con 0x7fa7d00c7250 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7b4066f30 msgr2=0x7fa7b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7b4066f30 0x7fa7b40693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa7d01527d0 tx=0x7fa7c4001120 comp rx=0 tx=0).stop 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 msgr2=0x7fa7d00cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 0x7fa7d00cb490 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0093f0 tx=0x7fa7bc016000 comp rx=0 tx=0).stop 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 shutdown_connections 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7d0159730 0x7fa7d00ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7d006b850 0x7fa7d0151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 --2- 172.21.15.33:0/1596946937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7d00c7250 0x7fa7d00cb490 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 >> 172.21.15.33:0/1596946937 conn(0x7fa7d006f0e0 msgr2=0x7fa7d0157060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 shutdown_connections 2023-10-06T05:21:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.360+0000 7fa7d599d700 1 -- 172.21.15.33:0/1596946937 wait complete. 2023-10-06T05:21:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:21:07.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:07.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.565+0000 7ff1df39d700 1 Processor -- start 2023-10-06T05:21:07.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.565+0000 7ff1df39d700 1 -- start start 2023-10-06T05:21:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1df39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1d8159530 0x7ff1d806da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1df39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d806e3e0 con 0x7ff1d80b04e0 2023-10-06T05:21:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1df39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d806f220 con 0x7ff1d806df70 2023-10-06T05:21:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1df39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d80b1e00 con 0x7ff1d8159530 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1de39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1d8159530 0x7ff1d806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1deb9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1d80b04e0 0x7ff1d815a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56090/0 (socket says 172.21.15.33:56090) 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.566+0000 7ff1deb9c700 1 -- 172.21.15.33:0/2193042131 learned_addr learned my addr 172.21.15.33:0/2193042131 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1de39b700 1 -- 172.21.15.33:0/2193042131 >> v1:172.21.15.167:6789/0 conn(0x7ff1d806df70 legacy=0x7ff1d80afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1de39b700 1 -- 172.21.15.33:0/2193042131 >> v1:172.21.15.33:6789/0 conn(0x7ff1d80b04e0 legacy=0x7ff1d815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1de39b700 1 -- 172.21.15.33:0/2193042131 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d806e610 con 0x7ff1d8159530 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1dd399700 1 -- 172.21.15.33:0/2193042131 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057828699 0 0) 0x7ff1d806e3e0 con 0x7ff1d80b04e0 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1de39b700 1 --2- 172.21.15.33:0/2193042131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d8159530 0x7ff1d806da30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098e0 tx=0x7ff1d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfa5f77ff3d73d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1dd399700 1 -- 172.21.15.33:0/2193042131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4009e80 con 0x7ff1d8159530 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1dd399700 1 -- 172.21.15.33:0/2193042131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400b860 con 0x7ff1d8159530 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.567+0000 7ff1dd399700 1 -- 172.21.15.33:0/2193042131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400bbe0 con 0x7ff1d8159530 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 -- 172.21.15.33:0/2193042131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d8159530 msgr2=0x7ff1d806da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 --2- 172.21.15.33:0/2193042131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d8159530 0x7ff1d806da30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098e0 tx=0x7ff1d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 -- 172.21.15.33:0/2193042131 shutdown_connections 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 --2- 172.21.15.33:0/2193042131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d8159530 0x7ff1d806da30 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 -- 172.21.15.33:0/2193042131 >> 172.21.15.33:0/2193042131 conn(0x7ff1d806c560 msgr2=0x7ff1d806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:07.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 -- 172.21.15.33:0/2193042131 shutdown_connections 2023-10-06T05:21:07.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.568+0000 7ff1df39d700 1 -- 172.21.15.33:0/2193042131 wait complete. 2023-10-06T05:21:07.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.570+0000 7ff1df39d700 1 Processor -- start 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.570+0000 7ff1df39d700 1 -- start start 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1df39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1d806df70 0x7ff1d8153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1df39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 0x7ff1d8150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1df39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1d8159530 0x7ff1d814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1df39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8152010 con 0x7ff1d80b04e0 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1df39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8150ca0 con 0x7ff1d8159530 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1df39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8150f70 con 0x7ff1d806df70 2023-10-06T05:21:07.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 0x7ff1d8150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 0x7ff1d8150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44088/0 (socket says 172.21.15.33:44088) 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 -- 172.21.15.33:0/3682335957 learned_addr learned my addr 172.21.15.33:0/3682335957 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1deb9c700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1d8159530 0x7ff1d814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 -- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1d806df70 msgr2=0x7ff1d8153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1d806df70 0x7ff1d8153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 -- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1d8159530 msgr2=0x7ff1d814e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1d8159530 0x7ff1d814e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1ddb9a700 1 -- 172.21.15.33:0/3682335957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4009420 con 0x7ff1d80b04e0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.571+0000 7ff1deb9c700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1d8159530 0x7ff1d814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.572+0000 7ff1ddb9a700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 0x7ff1d8150760 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b5e0 tx=0x7ff1c8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.572+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800f510 con 0x7ff1d80b04e0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.572+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800f670 con 0x7ff1d80b04e0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.572+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800fa30 con 0x7ff1d80b04e0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.572+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d8151250 con 0x7ff1d80b04e0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.572+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d8151810 con 0x7ff1d80b04e0 2023-10-06T05:21:07.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.573+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d804c380 con 0x7ff1d80b04e0 2023-10-06T05:21:07.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.574+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1c801a400 con 0x7ff1d80b04e0 2023-10-06T05:21:07.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.575+0000 7ff1ce7fc700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1bc066f30 0x7ff1bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.575+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6174+0+0 (secure 0 0 0) 0x7ff1c80842a0 con 0x7ff1d80b04e0 2023-10-06T05:21:07.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.575+0000 7ff1de39b700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1bc066f30 0x7ff1bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.576+0000 7ff1de39b700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1bc066f30 0x7ff1bc0693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098e0 tx=0x7ff1d4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.579+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1c8051320 con 0x7ff1d80b04e0 2023-10-06T05:21:07.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.830+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1d806b130 con 0x7ff1d80b04e0 2023-10-06T05:21:07.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.831+0000 7ff1ce7fc700 1 -- 172.21.15.33:0/3682335957 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4605 (secure 0 0 0) 0x7ff1c8054940 con 0x7ff1d80b04e0 2023-10-06T05:21:07.834 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:07.834 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:00.075535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6681":{"gid":6681,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2184356465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2184356465},{"type":"v1","addr":"172.21.15.33:6835","nonce":2184356465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1bc066f30 msgr2=0x7ff1bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1bc066f30 0x7ff1bc0693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098e0 tx=0x7ff1d4016000 comp rx=0 tx=0).stop 2023-10-06T05:21:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 msgr2=0x7ff1d8150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 0x7ff1d8150760 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b5e0 tx=0x7ff1c8009120 comp rx=0 tx=0).stop 2023-10-06T05:21:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 shutdown_connections 2023-10-06T05:21:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1bc066f30 0x7ff1bc0693e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1d8159530 0x7ff1d814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1d806df70 0x7ff1d8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 --2- 172.21.15.33:0/3682335957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d80b04e0 0x7ff1d8150760 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 >> 172.21.15.33:0/3682335957 conn(0x7ff1d806c560 msgr2=0x7ff1d80c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 shutdown_connections 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:07.833+0000 7ff1df39d700 1 -- 172.21.15.33:0/3682335957 wait complete. 2023-10-06T05:21:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2023-10-06T05:21:08.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:20:56.546500+0000', 'modified': '2023-10-06T05:21:00.075535+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6681}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6681': {'gid': 6681, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2184356465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 2184356465}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2184356465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:08.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:21:08.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:21:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.447+0000 7fed0bfff700 1 Processor -- start 2023-10-06T05:21:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.447+0000 7fed0bfff700 1 -- start start 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.448+0000 7fed0bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fed0c1597d0 0x7fed0c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.448+0000 7fed0bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed0c0cc770 0x7fed0c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.448+0000 7fed0bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0c0d0ed0 con 0x7fed0c159400 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.448+0000 7fed0bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c0b1160 con 0x7fed0c1597d0 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.448+0000 7fed0bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c0cb6d0 con 0x7fed0c0cc770 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.448+0000 7fed0a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fed0c1597d0 0x7fed0c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fed0c1597d0 0x7fed0c06c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44092/0 (socket says 172.21.15.33:44092) 2023-10-06T05:21:08.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 -- 172.21.15.33:0/3363643253 learned_addr learned my addr 172.21.15.33:0/3363643253 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 -- 172.21.15.33:0/3363643253 >> v2:172.21.15.167:3300/0 conn(0x7fed0c0cc770 msgr2=0x7fed0c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/3363643253 >> v2:172.21.15.167:3300/0 conn(0x7fed0c0cc770 0x7fed0c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/3363643253 >> v2:172.21.15.167:3300/0 conn(0x7fed0c0cc770 0x7fed0c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 -- 172.21.15.33:0/3363643253 >> v1:172.21.15.167:6789/0 conn(0x7fed0c159400 legacy=0x7fed0c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 -- 172.21.15.33:0/3363643253 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c0cb9b0 con 0x7fed0c1597d0 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/3363643253 >> v2:172.21.15.167:3300/0 conn(0x7fed0c0cc770 0x7fed0c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.449+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/3363643253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c06c560 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0cfca0 tx=0x7fed00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cfb15b43cc80da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed09ffb700 1 -- 172.21.15.33:0/3363643253 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000b860 con 0x7fed0c1597d0 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed09ffb700 1 -- 172.21.15.33:0/3363643253 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0000b9c0 con 0x7fed0c1597d0 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed09ffb700 1 -- 172.21.15.33:0/3363643253 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000bd60 con 0x7fed0c1597d0 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 -- 172.21.15.33:0/3363643253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 msgr2=0x7fed0c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 --2- 172.21.15.33:0/3363643253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c06c560 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0cfca0 tx=0x7fed00009250 comp rx=0 tx=0).stop 2023-10-06T05:21:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 -- 172.21.15.33:0/3363643253 shutdown_connections 2023-10-06T05:21:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 --2- 172.21.15.33:0/3363643253 >> v2:172.21.15.167:3300/0 conn(0x7fed0c0cc770 0x7fed0c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 --2- 172.21.15.33:0/3363643253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c06c560 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 -- 172.21.15.33:0/3363643253 >> 172.21.15.33:0/3363643253 conn(0x7fed0c06e4b0 msgr2=0x7fed0c0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 -- 172.21.15.33:0/3363643253 shutdown_connections 2023-10-06T05:21:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.450+0000 7fed0bfff700 1 -- 172.21.15.33:0/3363643253 wait complete. 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.452+0000 7fed0bfff700 1 Processor -- start 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.452+0000 7fed0bfff700 1 -- start start 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed0c0cc770 0x7fed0c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 0x7fed0c13b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c13ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c13bf40 con 0x7fed0c1597d0 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0b4290 con 0x7fed0c159400 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0b4570 con 0x7fed0c0cc770 2023-10-06T05:21:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c13ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 0x7fed0c13b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0affd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed0c0cc770 0x7fed0c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 0x7fed0c13b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:51184/0 (socket says 172.21.15.33:51184) 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2605369701 learned_addr learned my addr 172.21.15.33:0/2605369701 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.453+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c13ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44100/0 (socket says 172.21.15.33:44100) 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed0c0cc770 msgr2=0x7fed0c154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed0c0cc770 0x7fed0c154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2605369701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 msgr2=0x7fed0c13ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c13ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2605369701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000094f0 con 0x7fed0c159400 2023-10-06T05:21:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c13ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 0x7fed0c13b4c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fed00006fd0 tx=0x7fed00017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.454+0000 7fecf2ffd700 1 -- 172.21.15.33:0/2605369701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00024440 con 0x7fed0c159400 2023-10-06T05:21:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.455+0000 7fecf2ffd700 1 -- 172.21.15.33:0/2605369701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed000245a0 con 0x7fed0c159400 2023-10-06T05:21:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.455+0000 7fecf2ffd700 1 -- 172.21.15.33:0/2605369701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00024960 con 0x7fed0c159400 2023-10-06T05:21:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.455+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c157870 con 0x7fed0c159400 2023-10-06T05:21:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.455+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c157e30 con 0x7fed0c159400 2023-10-06T05:21:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.457+0000 7fecf2ffd700 1 -- 172.21.15.33:0/2605369701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed00024d50 con 0x7fed0c159400 2023-10-06T05:21:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.458+0000 7fecf2ffd700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecf8066f30 0x7fecf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.458+0000 7fecf2ffd700 1 -- 172.21.15.33:0/2605369701 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fed00086240 con 0x7fed0c159400 2023-10-06T05:21:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.458+0000 7fed0affd700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecf8066f30 0x7fecf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.459+0000 7fed0affd700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecf8066f30 0x7fecf80693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fecfc003a80 tx=0x7fecfc007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.459+0000 7fed1ff9db80 1 Processor -- start 2023-10-06T05:21:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.459+0000 7fed1ff9db80 1 -- start start 2023-10-06T05:21:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.460+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92fe8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.460+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92feb8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.460+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92fe8b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.460+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92fe0fc80 con 0x55e92fed9bc0 2023-10-06T05:21:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.460+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92fde6a70 con 0x55e92fe8f950 2023-10-06T05:21:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.460+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e92fde4270 con 0x55e92feb8e60 2023-10-06T05:21:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92feb8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92fe8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92fe8fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:51190/0 (socket says 172.21.15.33:51190) 2023-10-06T05:21:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92feb8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44114/0 (socket says 172.21.15.33:44114) 2023-10-06T05:21:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0affd700 1 -- 172.21.15.33:0/2713410934 learned_addr learned my addr 172.21.15.33:0/2713410934 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92fe8b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 msgr2=0x55e92fe8b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92fe8b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 msgr2=0x55e92fe8fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92fe8fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 -- 172.21.15.33:0/2713410934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e92fc7a270 con 0x55e92fed9bc0 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92feb8920 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fed00084580 tx=0x7fed00084450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f8f25a283aa1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.461+0000 7fecebfff700 1 -- 172.21.15.33:0/2713410934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000568e0 con 0x55e92fed9bc0 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fecebfff700 1 -- 172.21.15.33:0/2713410934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0002b340 con 0x55e92fed9bc0 2023-10-06T05:21:08.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fecebfff700 1 -- 172.21.15.33:0/2713410934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00057680 con 0x55e92fed9bc0 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2713410934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 msgr2=0x55e92feb8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92feb8920 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fed00084580 tx=0x7fed00084450 comp rx=0 tx=0).stop 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2713410934 shutdown_connections 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92fe8b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92fe8fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2713410934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92feb8920 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2713410934 >> 172.21.15.33:0/2713410934 conn(0x55e92fe91270 msgr2=0x55e92fe91640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2713410934 shutdown_connections 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.462+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2713410934 wait complete. 2023-10-06T05:21:08.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.463+0000 7fed1ff9db80 1 -- start start 2023-10-06T05:21:08.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.463+0000 7fed1ff9db80 1 Processor -- start 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92feb5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92feb79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92fe861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92fde0060 con 0x55e92fed9bc0 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92fd8b890 con 0x55e92fe8f950 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e92fda6ea0 con 0x55e92feb8e60 2023-10-06T05:21:08.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.464+0000 7fed1ff9db80 10 client.0 _ll_register_callbacks cb 0x55e92fc7cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92fe861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92fe861a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44124/0 (socket says 172.21.15.33:44124) 2023-10-06T05:21:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 -- 172.21.15.33:0/3122702207 learned_addr learned my addr 172.21.15.33:0/3122702207 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92feb79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0affd700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92feb5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 -- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 msgr2=0x55e92feb79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92feb79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 -- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 msgr2=0x55e92feb5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92feb5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 -- 172.21.15.33:0/3122702207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c157e30 con 0x55e92fed9bc0 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92fe861a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fed0400c740 tx=0x7fed0400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be74c765cfcda609 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fece97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.465+0000 7fece97fa700 1 -- 172.21.15.33:0/3122702207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0401b070 con 0x55e92fed9bc0 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fece97fa700 1 -- 172.21.15.33:0/3122702207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fed0400b7e0 con 0x55e92fed9bc0 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e92fe84690 con 0x55e92fed9bc0 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fece97fa700 1 -- 172.21.15.33:0/3122702207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed04012050 con 0x55e92fed9bc0 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 10 client.6849 fetch_fsmap learned FSMap version 58 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55e92fc1cfe0 con 0x55e92fed9bc0 2023-10-06T05:21:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fece97fa700 1 -- 172.21.15.33:0/3122702207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e92fe84690 con 0x55e92fed9bc0 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fece97fa700 1 -- 172.21.15.33:0/3122702207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fed0401e020 con 0x55e92fed9bc0 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 10 client.6849 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 10 client.6849 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 20 client.6849 populate_metadata read hostname 'smithi033' 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92fed1ad0 0x55e92fed1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 4 client.6849 mds_command: new command op to 6681 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 --> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e92fe615c0 con 0x55e92fed1ad0 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.466+0000 7fed0affd700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92fed1ad0 0x55e92fed1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.467+0000 7fed0affd700 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92fed1ad0 0x55e92fed1ea0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131faa8e3b6e8df6 server_cookie=57eaa9347e6c2774 in_seq=0 out_seq=0 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.467+0000 7fece97fa700 10 client.6849 ms_handle_connect on v2:172.21.15.33:6833/2184356465 2023-10-06T05:21:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fece97fa700 1 -- 172.21.15.33:0/3122702207 <== mds.0 v2:172.21.15.33:6833/2184356465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e92fe615c0 con 0x55e92fed1ad0 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fece97fa700 10 client.6849 handle_command_reply: tid=0 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 client.6849 shutdown 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 msgr2=0x55e92fe861a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92fe861a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fed0400c740 tx=0x7fed0400f040 comp rx=0 tx=0).stop 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 shutdown_connections 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92fed1ad0 0x55e92fed1ea0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fed9bc0 0x55e92fe861a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fe8f950 0x55e92feb5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/3122702207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92feb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fece97fa700 0 client.6849 ms_handle_reset on v2:172.21.15.33:6833/2184356465 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 >> 172.21.15.33:0/3122702207 conn(0x55e92fe91270 msgr2=0x55e92fed54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 shutdown_connections 2023-10-06T05:21:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.468+0000 7fed1ff9db80 1 -- 172.21.15.33:0/3122702207 wait complete. 2023-10-06T05:21:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.469+0000 7fed1ff9db80 20 client.6849 trim_cache size 0 max 16384 2023-10-06T05:21:08.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.486+0000 7fed1ff9db80 1 Processor -- start 2023-10-06T05:21:08.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.486+0000 7fed1ff9db80 1 -- start start 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92febf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92febf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fed9bc0 0x55e92fee8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92fe8ec40 con 0x55e92fe8f950 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92fc62e30 con 0x55e92fed9bc0 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e92fed4100 con 0x55e92feb8e60 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed0affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92febf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed0affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92febf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44126/0 (socket says 172.21.15.33:44126) 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.487+0000 7fed0affd700 1 -- 172.21.15.33:0/2046991821 learned_addr learned my addr 172.21.15.33:0/2046991821 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0affd700 1 -- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 msgr2=0x55e92febf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92febf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fed9bc0 0x55e92fee8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0affd700 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92febf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0affd700 1 -- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fed9bc0 msgr2=0x55e92fee8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0affd700 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fed9bc0 0x55e92fee8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0affd700 1 -- 172.21.15.33:0/2046991821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e92fe8d9e0 con 0x55e92fe8f950 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fed9bc0 0x55e92fee8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fed0affd700 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92febf260 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fecfc007c80 tx=0x7fecfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf50bf999f6601f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fecf0ff9700 1 -- 172.21.15.33:0/2046991821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc028070 con 0x55e92fe8f950 2023-10-06T05:21:08.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fecf0ff9700 1 -- 172.21.15.33:0/2046991821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc023450 con 0x55e92fe8f950 2023-10-06T05:21:08.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.488+0000 7fecf0ff9700 1 -- 172.21.15.33:0/2046991821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0237d0 con 0x55e92fe8f950 2023-10-06T05:21:08.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2046991821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 msgr2=0x55e92febf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92febf260 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fecfc007c80 tx=0x7fecfc014000 comp rx=0 tx=0).stop 2023-10-06T05:21:08.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2046991821 shutdown_connections 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92fed9bc0 0x55e92fee8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92febf260 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/2046991821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92feb8e60 0x55e92febf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2046991821 >> 172.21.15.33:0/2046991821 conn(0x55e92fe91270 msgr2=0x55e92fe8fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2046991821 shutdown_connections 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.489+0000 7fed1ff9db80 1 -- 172.21.15.33:0/2046991821 wait complete. 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.490+0000 7fed1ff9db80 1 -- start start 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.490+0000 7fed1ff9db80 1 Processor -- start 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92feec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92feb8e60 0x55e92ff3aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92fed9bc0 0x55e92ff3af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92feea460 con 0x55e92fe8f950 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92feea0f0 con 0x55e92feb8e60 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e92fed7820 con 0x55e92fed9bc0 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed1ff9db80 10 client.0 _ll_register_callbacks cb 0x55e92fc7cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed0affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92feec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed0affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92feec8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44130/0 (socket says 172.21.15.33:44130) 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed0affd700 1 -- 172.21.15.33:0/1940618386 learned_addr learned my addr 172.21.15.33:0/1940618386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92feb8e60 0x55e92ff3aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.491+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92fed9bc0 0x55e92ff3af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0affd700 1 -- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92fed9bc0 msgr2=0x55e92ff3af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0affd700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92fed9bc0 0x55e92ff3af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0affd700 1 -- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92feb8e60 msgr2=0x55e92ff3aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0affd700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92feb8e60 0x55e92ff3aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0affd700 1 -- 172.21.15.33:0/1940618386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc019040 con 0x55e92fe8f950 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0b7fe700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92fed9bc0 0x55e92ff3af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed0affd700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92feec8f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fecfc007c50 tx=0x7fecfc025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9da281d3b9a3164 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fece97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fece97fa700 1 -- 172.21.15.33:0/1940618386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc031070 con 0x55e92fe8f950 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e92fe8d180 con 0x55e92fe8f950 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fece97fa700 1 -- 172.21.15.33:0/1940618386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fecfc01c040 con 0x55e92fe8f950 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fece97fa700 1 -- 172.21.15.33:0/1940618386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecfc023e20 con 0x55e92fe8f950 2023-10-06T05:21:08.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed1ff9db80 10 client.6855 fetch_fsmap learned FSMap version 58 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.492+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55e92fed1750 con 0x55e92fe8f950 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fece97fa700 1 -- 172.21.15.33:0/1940618386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e92fe8d180 con 0x55e92fe8f950 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fece97fa700 1 -- 172.21.15.33:0/1940618386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fecfc037070 con 0x55e92fe8f950 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed1ff9db80 10 client.6855 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed1ff9db80 10 client.6855 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed1ff9db80 20 client.6855 populate_metadata read hostname 'smithi033' 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92feea8f0 0x55e92feeacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed1ff9db80 4 client.6855 mds_command: new command op to 6681 tid=0[{"prefix": "status"}] 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 --> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fed0002b340 con 0x55e92feea8f0 2023-10-06T05:21:08.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.493+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92feea8f0 0x55e92feeacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fece97fa700 10 client.6855 ms_handle_connect on v2:172.21.15.33:6833/2184356465 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed0a7fc700 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92feea8f0 0x55e92feeacc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491adc52e0837a94 server_cookie=57fdb44f429c2e56 in_seq=0 out_seq=0 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fece97fa700 1 -- 172.21.15.33:0/1940618386 <== mds.0 v2:172.21.15.33:6833/2184356465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x7fed0002b340 con 0x55e92feea8f0 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fece97fa700 10 client.6855 handle_command_reply: tid=0 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 client.6855 shutdown 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 msgr2=0x55e92feec8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92feec8f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fecfc007c50 tx=0x7fecfc025000 comp rx=0 tx=0).stop 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 shutdown_connections 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:6833/2184356465,v1:172.21.15.33:6835/2184356465] conn(0x55e92feea8f0 0x55e92feeacc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55e92fed9bc0 0x55e92ff3af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e92fe8f950 0x55e92feec8f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fece97fa700 0 client.6855 ms_handle_reset on v2:172.21.15.33:6833/2184356465 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 --2- 172.21.15.33:0/1940618386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e92feb8e60 0x55e92ff3aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 >> 172.21.15.33:0/1940618386 conn(0x55e92fe91270 msgr2=0x55e92fe92b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 shutdown_connections 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.494+0000 7fed1ff9db80 1 -- 172.21.15.33:0/1940618386 wait complete. 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed1ff9db80 20 client.6855 trim_cache size 0 max 16384 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:21:08.503 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 6681, 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 11.449544216, 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 58, 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 91, 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 89, 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 17.179971962 2023-10-06T05:21:08.504 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:21:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecf8066f30 msgr2=0x7fecf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecf8066f30 0x7fecf80693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fecfc003a80 tx=0x7fecfc007ec0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 msgr2=0x7fed0c13b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 0x7fed0c13b4c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fed00006fd0 tx=0x7fed00017000 comp rx=0 tx=0).stop 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 shutdown_connections 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecf8066f30 0x7fecf80693e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed0c1597d0 0x7fed0c13ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed0c0cc770 0x7fed0c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 --2- 172.21.15.33:0/2605369701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed0c159400 0x7fed0c13b4c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 >> 172.21.15.33:0/2605369701 conn(0x7fed0c06e4b0 msgr2=0x7fed0c1584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 shutdown_connections 2023-10-06T05:21:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.495+0000 7fed0bfff700 1 -- 172.21.15.33:0/2605369701 wait complete. 2023-10-06T05:21:08.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:21:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.700+0000 7f739ffff700 1 Processor -- start 2023-10-06T05:21:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.700+0000 7f739ffff700 1 -- start start 2023-10-06T05:21:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.701+0000 7f739ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73a00c8050 0x7f73a013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.701+0000 7f739ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a00c9cb0 con 0x7f73a006aad0 2023-10-06T05:21:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.701+0000 7f739ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a00ca270 con 0x7f73a013faa0 2023-10-06T05:21:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.701+0000 7f739ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a006b640 con 0x7f73a00c8050 2023-10-06T05:21:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.701+0000 7f739f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73a006aad0 0x7f73a006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56100/0 (socket says 172.21.15.33:56100) 2023-10-06T05:21:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.701+0000 7f739f7fe700 1 -- 172.21.15.33:0/891990187 learned_addr learned my addr 172.21.15.33:0/891990187 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:08.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739e7fc700 1 --2- 172.21.15.33:0/891990187 >> v2:172.21.15.167:3300/0 conn(0x7f73a00c8050 0x7f73a013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265491080 0 0) 0x7f73a00c9cb0 con 0x7f73a006aad0 2023-10-06T05:21:08.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7388003570 con 0x7f73a006aad0 2023-10-06T05:21:08.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777193737 0 0) 0x7f73a00ca270 con 0x7f73a013faa0 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a00c9cb0 con 0x7f73a013faa0 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739e7fc700 1 -- 172.21.15.33:0/891990187 >> v1:172.21.15.167:6789/0 conn(0x7f73a013faa0 legacy=0x7f73a00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2870603017 0 0) 0x7f7388003570 con 0x7f73a006aad0 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739e7fc700 1 -- 172.21.15.33:0/891990187 >> v1:172.21.15.33:6789/0 conn(0x7f73a006aad0 legacy=0x7f73a006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.702+0000 7f739e7fc700 1 -- 172.21.15.33:0/891990187 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a006b840 con 0x7f73a00c8050 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739e7fc700 1 --2- 172.21.15.33:0/891990187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a00c8050 0x7f73a013fe70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f73940098e0 tx=0x7f73940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e425291f76ccbdc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980017f0 con 0x7f73a00c8050 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394011420 con 0x7f73a00c8050 2023-10-06T05:21:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739dffb700 1 -- 172.21.15.33:0/891990187 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940117c0 con 0x7f73a00c8050 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 -- 172.21.15.33:0/891990187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a00c8050 msgr2=0x7f73a013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 --2- 172.21.15.33:0/891990187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a00c8050 0x7f73a013fe70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f73940098e0 tx=0x7f73940091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 -- 172.21.15.33:0/891990187 shutdown_connections 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 --2- 172.21.15.33:0/891990187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a00c8050 0x7f73a013fe70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 -- 172.21.15.33:0/891990187 >> 172.21.15.33:0/891990187 conn(0x7f73a006d030 msgr2=0x7f73a006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 -- 172.21.15.33:0/891990187 shutdown_connections 2023-10-06T05:21:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.703+0000 7f739ffff700 1 -- 172.21.15.33:0/891990187 wait complete. 2023-10-06T05:21:08.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.705+0000 7f739ffff700 1 Processor -- start 2023-10-06T05:21:08.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.705+0000 7f739ffff700 1 -- start start 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a006aad0 0x7f73a0157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00c8050 0x7f73a0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 0x7f73a00b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0157b40 con 0x7f73a013faa0 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0156f30 con 0x7f73a006aad0 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0157210 con 0x7f73a00c8050 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 0x7f73a00b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 0x7f73a00b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44146/0 (socket says 172.21.15.33:44146) 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.706+0000 7f739f7fe700 1 -- 172.21.15.33:0/3956727976 learned_addr learned my addr 172.21.15.33:0/3956727976 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:08.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739f7fe700 1 -- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00c8050 msgr2=0x7f73a0155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739f7fe700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00c8050 0x7f73a0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739f7fe700 1 -- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a006aad0 msgr2=0x7f73a0157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739effd700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a006aad0 0x7f73a0157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739f7fe700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a006aad0 0x7f73a0157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739f7fe700 1 -- 172.21.15.33:0/3956727976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009420 con 0x7f73a013faa0 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739effd700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a006aad0 0x7f73a0157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739f7fe700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 0x7f73a00b07c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f739800cca0 tx=0x7f7398007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800ce50 con 0x7f73a013faa0 2023-10-06T05:21:08.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7398004dc0 con 0x7f73a013faa0 2023-10-06T05:21:08.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800ab70 con 0x7f73a013faa0 2023-10-06T05:21:08.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a01574f0 con 0x7f73a013faa0 2023-10-06T05:21:08.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.707+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a0156c00 con 0x7f73a013faa0 2023-10-06T05:21:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.708+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a013e690 con 0x7f73a013faa0 2023-10-06T05:21:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.709+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7398007150 con 0x7f73a013faa0 2023-10-06T05:21:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.710+0000 7f7386ffd700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f738c066f30 0x7f738c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.711+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f7398084050 con 0x7f73a013faa0 2023-10-06T05:21:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.711+0000 7f739effd700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f738c066f30 0x7f738c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.712+0000 7f739effd700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f738c066f30 0x7f738c0693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7390007fd0 tx=0x7f7390009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:08.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.714+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73980510d0 con 0x7f73a013faa0 2023-10-06T05:21:08.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.961+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f73a004c380 con 0x7f73a013faa0 2023-10-06T05:21:08.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.962+0000 7f7386ffd700 1 -- 172.21.15.33:0/3956727976 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7398018020 con 0x7f73a013faa0 2023-10-06T05:21:08.964 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f738c066f30 msgr2=0x7f738c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f738c066f30 0x7f738c0693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7390007fd0 tx=0x7f7390009040 comp rx=0 tx=0).stop 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 msgr2=0x7f73a00b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 0x7f73a00b07c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f739800cca0 tx=0x7f7398007d20 comp rx=0 tx=0).stop 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 shutdown_connections 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f738c066f30 0x7f738c0693e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a013faa0 0x7f73a00b07c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a006aad0 0x7f73a0157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 --2- 172.21.15.33:0/3956727976 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73a00c8050 0x7f73a0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 >> 172.21.15.33:0/3956727976 conn(0x7f73a006d030 msgr2=0x7f73a00c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 shutdown_connections 2023-10-06T05:21:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:08.964+0000 7f739ffff700 1 -- 172.21.15.33:0/3956727976 wait complete. 2023-10-06T05:21:08.984 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:21:08.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:21:08.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:21:08.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:21:08.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:08.985 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:21:08.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:21:08.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:21:08.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:08.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:08.985 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:21:09.022 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:21:09.023 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41888sec preferred_lft 41888sec 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:09.024 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:21:09.025 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:09.026 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:21:09.026 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:21:09.026 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:09.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:09.026 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:21:09.123 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:21:09.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:09.127 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:21:09.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:09.167 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:21:09.167 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:21:09.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:21:09.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:21:09.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:21:09.167 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:09.365 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:21:09.402 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:21:09.402 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:21:09.402 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:09.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:09.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:21:09.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:09.510 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:09.549 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:09.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:09.549 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:09.625 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:21:09.625 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:21:09.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:21:09.758 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:21:09.758 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:21:09.758 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:21:10.350 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:21:10.351 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:21:10.355 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:21:10.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:10.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:10.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:21:10.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.606+0000 7f10ff39d700 1 Processor -- start 2023-10-06T05:21:10.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.606+0000 7f10ff39d700 1 -- start start 2023-10-06T05:21:10.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10ff39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10ff39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f10f8155ec0 0x7f10f80c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10ff39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f806ab60 con 0x7f10f8155af0 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10ff39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f806b7e0 con 0x7f10f8155ec0 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10ff39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f806ae40 con 0x7f10f80c8b50 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10fe39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f10f8155af0 0x7f10f806a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49512/0 (socket says 172.21.15.33:49512) 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10fe39b700 1 -- 172.21.15.33:0/3097726103 learned_addr learned my addr 172.21.15.33:0/3097726103 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10fdb9a700 1 --2- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10feb9c700 1 --2- 172.21.15.33:0/3097726103 >> v2:172.21.15.167:3300/0 conn(0x7f10f8155ec0 0x7f10f80c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10fd399700 1 -- 172.21.15.33:0/3097726103 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623378127 0 0) 0x7f10f806ab60 con 0x7f10f8155af0 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.608+0000 7f10fd399700 1 -- 172.21.15.33:0/3097726103 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4003570 con 0x7f10f8155af0 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10feb9c700 1 -- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 msgr2=0x7f10f80c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10feb9c700 1 --2- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f80c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10feb9c700 1 -- 172.21.15.33:0/3097726103 >> v1:172.21.15.33:6789/0 conn(0x7f10f8155af0 legacy=0x7f10f806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10feb9c700 1 -- 172.21.15.33:0/3097726103 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f806b120 con 0x7f10f8155ec0 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10feb9c700 1 --2- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c7750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009e60 tx=0x7f10f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91023b568f04caca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10fd399700 1 -- 172.21.15.33:0/3097726103 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000e070 con 0x7f10f8155ec0 2023-10-06T05:21:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10fd399700 1 -- 172.21.15.33:0/3097726103 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f000bab0 con 0x7f10f8155ec0 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.609+0000 7f10fd399700 1 -- 172.21.15.33:0/3097726103 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000be70 con 0x7f10f8155ec0 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 -- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 msgr2=0x7f10f80c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 --2- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c7750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009e60 tx=0x7f10f0009780 comp rx=0 tx=0).stop 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 -- 172.21.15.33:0/3097726103 shutdown_connections 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 --2- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c7750 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 --2- 172.21.15.33:0/3097726103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f80c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 -- 172.21.15.33:0/3097726103 >> 172.21.15.33:0/3097726103 conn(0x7f10f806d030 msgr2=0x7f10f806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 -- 172.21.15.33:0/3097726103 shutdown_connections 2023-10-06T05:21:10.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.610+0000 7f10ff39d700 1 -- 172.21.15.33:0/3097726103 wait complete. 2023-10-06T05:21:10.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.611+0000 7f10ff39d700 1 Processor -- start 2023-10-06T05:21:10.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.612+0000 7f10ff39d700 1 -- start start 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.612+0000 7f10ff39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.612+0000 7f10ff39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 0x7f10f80c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.612+0000 7f10ff39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10ff39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80adce0 con 0x7f10f8155af0 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10ff39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8156b30 con 0x7f10f8155ec0 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10ff39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80ad160 con 0x7f10f80c8b50 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 0x7f10f80c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 0x7f10f80c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34130/0 (socket says 172.21.15.33:34130) 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 -- 172.21.15.33:0/2232512266 learned_addr learned my addr 172.21.15.33:0/2232512266 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fe39b700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10feb9c700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 -- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 msgr2=0x7f10f814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f814ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 -- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 msgr2=0x7f10f80c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 -- 172.21.15.33:0/2232512266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009a20 con 0x7f10f8155af0 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fe39b700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.613+0000 7f10fdb9a700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 0x7f10f80c4b90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f10e800e960 tx=0x7f10e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.614+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800c8e0 con 0x7f10f8155af0 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.614+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e800ca40 con 0x7f10f8155af0 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.614+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80ad440 con 0x7f10f8155af0 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.614+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800cde0 con 0x7f10f8155af0 2023-10-06T05:21:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.614+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80ada00 con 0x7f10f8155af0 2023-10-06T05:21:10.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.615+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f804c380 con 0x7f10f8155af0 2023-10-06T05:21:10.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.616+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10e8013680 con 0x7f10f8155af0 2023-10-06T05:21:10.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.617+0000 7f10ee7fc700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10dc066f60 0x7f10dc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.617+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f10e8084a00 con 0x7f10f8155af0 2023-10-06T05:21:10.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.617+0000 7f10fe39b700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10dc066f60 0x7f10dc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.618+0000 7f10fe39b700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10dc066f60 0x7f10dc069410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f10f4007fd0 tx=0x7f10f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.621+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10e8051a00 con 0x7f10f8155af0 2023-10-06T05:21:10.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.871+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f10f8156c70 con 0x7f10f8155af0 2023-10-06T05:21:10.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.871+0000 7f10ee7fc700 1 -- 172.21.15.33:0/2232512266 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f10e8055020 con 0x7f10f8155af0 2023-10-06T05:21:10.873 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:21:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10dc066f60 msgr2=0x7f10dc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10dc066f60 0x7f10dc069410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f10f4007fd0 tx=0x7f10f4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 msgr2=0x7f10f80c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 0x7f10f80c4b90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f10e800e960 tx=0x7f10e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 shutdown_connections 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10dc066f60 0x7f10dc069410 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f8155ec0 0x7f10f80c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f80c8b50 0x7f10f814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 --2- 172.21.15.33:0/2232512266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10f8155af0 0x7f10f80c4b90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 >> 172.21.15.33:0/2232512266 conn(0x7f10f806d030 msgr2=0x7f10f80b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 shutdown_connections 2023-10-06T05:21:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:10.874+0000 7f10ff39d700 1 -- 172.21.15.33:0/2232512266 wait complete. 2023-10-06T05:21:10.893 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:21:10.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:21:10.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:21:10.893 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:21:10.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:10.894 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:21:10.894 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:21:10.894 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:21:10.894 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:21:10.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:10.894 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41890sec preferred_lft 41890sec 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:21:10.937 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.938 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:10.939 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:21:10.939 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:21:10.939 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:10.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:10.939 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:21:11.039 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:21:11.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:11.043 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:21:11.082 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:21:11.083 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:21:11.083 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:21:11.083 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:21:11.083 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:21:11.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:11.083 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:21:11.083 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:21:11.083 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:21:11.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:21:11.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:21:11.084 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:21:11.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:11.278 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:21:11.313 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:21:11.314 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:21:11.314 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:21:11.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:11.314 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:21:11.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:11.414 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:11.450 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:21:11.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:11.451 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:11.525 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:21:11.526 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:21:11.526 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:21:11.655 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:21:11.655 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:21:11.656 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:21:11.672 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:21:11.672 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:21:11.672 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:21:11.672 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:21:11.672 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:21:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:21:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:21:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:21:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:21:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:21:11.693 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:21:11.694 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:21:11.694 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:21:11.698 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:21:11.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:11.699 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:11.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:21:11.981 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:21:11.981 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:21:11.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:21:12.189 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:21:12.190 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:21:12.190 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:21:12.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:12.191 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:12.273 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:21:12.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:12.273 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> print("Starting creation...") 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> start = time.time() 2023-10-06T05:21:12.314 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> os.mkdir(path) 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> dfd = os.open(path, os.O_DIRECTORY) 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> fd = open(os.path.join(path, "childfile"), "w") 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> print("Finished creation in {0}s".format(time.time() - start)) 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> print("Starting fsync...") 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> start = time.time() 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> if True: 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> os.fsync(dfd) 2023-10-06T05:21:12.315 DEBUG:teuthology.orchestra.run.smithi033:> else: 2023-10-06T05:21:12.316 DEBUG:teuthology.orchestra.run.smithi033:> os.fsync(fd) 2023-10-06T05:21:12.316 DEBUG:teuthology.orchestra.run.smithi033:> print("Finished fsync in {0}s".format(time.time() - start)) 2023-10-06T05:21:12.316 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:21:12.428 INFO:teuthology.orchestra.run.smithi033.stdout:Starting creation... 2023-10-06T05:21:12.428 INFO:teuthology.orchestra.run.smithi033.stdout:Finished creation in 0.0050160884857177734s 2023-10-06T05:21:12.428 INFO:teuthology.orchestra.run.smithi033.stdout:Starting fsync... 2023-10-06T05:21:12.428 INFO:teuthology.orchestra.run.smithi033.stdout:Finished fsync in 0.0018436908721923828s 2023-10-06T05:21:12.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:21:12.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.834+0000 7f55e7fff700 1 Processor -- start 2023-10-06T05:21:12.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.834+0000 7f55e7fff700 1 -- start start 2023-10-06T05:21:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f55e8140d20 0x7f55e80c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55e80c9210 0x7f55e80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e815bcb0 con 0x7f55e8069f00 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e80c9690 con 0x7f55e80c9210 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e81410f0 con 0x7f55e8140d20 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55e80c9210 0x7f55e80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55e80c9210 0x7f55e80699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34146/0 (socket says 172.21.15.33:34146) 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 -- 172.21.15.33:0/3131202162 learned_addr learned my addr 172.21.15.33:0/3131202162 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e77fe700 1 --1- 172.21.15.33:0/3131202162 >> v1:172.21.15.33:6789/0 conn(0x7f55e8069f00 0x7f55e80cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49514/0 (socket says 172.21.15.33:49514) 2023-10-06T05:21:12.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e6ffd700 1 --2- 172.21.15.33:0/3131202162 >> v2:172.21.15.167:3300/0 conn(0x7f55e8140d20 0x7f55e80c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 -- 172.21.15.33:0/3131202162 >> v2:172.21.15.167:3300/0 conn(0x7f55e8140d20 msgr2=0x7f55e80c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 --2- 172.21.15.33:0/3131202162 >> v2:172.21.15.167:3300/0 conn(0x7f55e8140d20 0x7f55e80c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 -- 172.21.15.33:0/3131202162 >> v1:172.21.15.33:6789/0 conn(0x7f55e8069f00 legacy=0x7f55e80cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e5ffb700 1 -- 172.21.15.33:0/3131202162 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545618474 0 0) 0x7f55e815bcb0 con 0x7f55e8069f00 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e67fc700 1 -- 172.21.15.33:0/3131202162 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e806a2f0 con 0x7f55e80c9210 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.836+0000 7f55e6ffd700 1 --2- 172.21.15.33:0/3131202162 >> v2:172.21.15.167:3300/0 conn(0x7f55e8140d20 0x7f55e80c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.837+0000 7f55e67fc700 1 --2- 172.21.15.33:0/3131202162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e80c9210 0x7f55e80699c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f55e806e150 tx=0x7f55dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc53762665994d1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.837+0000 7f55e5ffb700 1 -- 172.21.15.33:0/3131202162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00b860 con 0x7f55e80c9210 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.837+0000 7f55e5ffb700 1 -- 172.21.15.33:0/3131202162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc00b9c0 con 0x7f55e80c9210 2023-10-06T05:21:12.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.837+0000 7f55e5ffb700 1 -- 172.21.15.33:0/3131202162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00bd60 con 0x7f55e80c9210 2023-10-06T05:21:12.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 -- 172.21.15.33:0/3131202162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e80c9210 msgr2=0x7f55e80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 --2- 172.21.15.33:0/3131202162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e80c9210 0x7f55e80699c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f55e806e150 tx=0x7f55dc009250 comp rx=0 tx=0).stop 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 -- 172.21.15.33:0/3131202162 shutdown_connections 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 --2- 172.21.15.33:0/3131202162 >> v2:172.21.15.167:3300/0 conn(0x7f55e8140d20 0x7f55e80c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 --2- 172.21.15.33:0/3131202162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e80c9210 0x7f55e80699c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 -- 172.21.15.33:0/3131202162 >> 172.21.15.33:0/3131202162 conn(0x7f55e806c560 msgr2=0x7f55e806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 -- 172.21.15.33:0/3131202162 shutdown_connections 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.838+0000 7f55e7fff700 1 -- 172.21.15.33:0/3131202162 wait complete. 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.839+0000 7f55e7fff700 1 Processor -- start 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.839+0000 7f55e7fff700 1 -- start start 2023-10-06T05:21:12.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.840+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 0x7f55e816a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.840+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55e80c9210 0x7f55e80c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.840+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e8140d20 0x7f55e815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.840+0000 7f55e7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8169be0 con 0x7f55e8069f00 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.840+0000 7f55e7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c4470 con 0x7f55e8140d20 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.840+0000 7f55e7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c4750 con 0x7f55e80c9210 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 0x7f55e816a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 0x7f55e816a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34160/0 (socket says 172.21.15.33:34160) 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 -- 172.21.15.33:0/644415126 learned_addr learned my addr 172.21.15.33:0/644415126 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:12.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e67fc700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55e80c9210 0x7f55e80c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e77fe700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e8140d20 0x7f55e815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 -- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55e80c9210 msgr2=0x7f55e80c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55e80c9210 0x7f55e80c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 -- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e8140d20 msgr2=0x7f55e815a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e8140d20 0x7f55e815a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 -- 172.21.15.33:0/644415126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0094f0 con 0x7f55e8069f00 2023-10-06T05:21:12.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e67fc700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55e80c9210 0x7f55e80c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.841+0000 7f55e6ffd700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 0x7f55e816a070 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f55d800d860 tx=0x7f55d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.842+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80125f0 con 0x7f55e8069f00 2023-10-06T05:21:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.842+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8012750 con 0x7f55e8069f00 2023-10-06T05:21:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.842+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8012ad0 con 0x7f55e8069f00 2023-10-06T05:21:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.842+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e8169590 con 0x7f55e8069f00 2023-10-06T05:21:12.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.842+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e80c4010 con 0x7f55e8069f00 2023-10-06T05:21:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.843+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e804c380 con 0x7f55e8069f00 2023-10-06T05:21:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.844+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55d8014430 con 0x7f55e8069f00 2023-10-06T05:21:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.846+0000 7f55ceffd700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.846+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f55d8084bf0 con 0x7f55e8069f00 2023-10-06T05:21:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.846+0000 7f55e67fc700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.847+0000 7f55e67fc700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f55e806e150 tx=0x7f55dc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:12.850+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55d8051bf0 con 0x7f55e8069f00 2023-10-06T05:21:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.103+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55e80ca9f0 con 0x7f55e8069f00 2023-10-06T05:21:13.105 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:21:13.104+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:21:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.108+0000 7f55ceffd700 1 -- 172.21.15.33:0/644415126 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55d8055210 con 0x7f55e8069f00 2023-10-06T05:21:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f55d4066f30 msgr2=0x7f55d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f55e806e150 tx=0x7f55dc017000 comp rx=0 tx=0).stop 2023-10-06T05:21:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 msgr2=0x7f55e816a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 0x7f55e816a070 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f55d800d860 tx=0x7f55d8008040 comp rx=0 tx=0).stop 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 shutdown_connections 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e8140d20 0x7f55e815a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55e8069f00 0x7f55e816a070 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 --2- 172.21.15.33:0/644415126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55e80c9210 0x7f55e80c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.111+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 >> 172.21.15.33:0/644415126 conn(0x7f55e806c560 msgr2=0x7f55e80c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.112+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 shutdown_connections 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.112+0000 7f55e7fff700 1 -- 172.21.15.33:0/644415126 wait complete. 2023-10-06T05:21:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:21:13.115 INFO:tasks.ceph.mds.c.smithi033.stderr: -1091> 2023-10-06T05:21:03.027+0000 7fd0b39f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:13.131 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T05:21:13.131 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:13.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:13.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:21:13.152 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:21:13.202 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:21:13.202 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:21:13.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:13.203 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:21:13.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:21:13.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:13.320 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:21:13.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:21:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.586+0000 7f416679d700 1 Processor -- start 2023-10-06T05:21:13.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.586+0000 7f416679d700 1 -- start start 2023-10-06T05:21:13.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.587+0000 7f416679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f416006efa0 0x7f41600ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.587+0000 7f416679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41600b0cd0 con 0x7f4160159850 2023-10-06T05:21:13.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.587+0000 7f416679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416015a840 con 0x7f416015acd0 2023-10-06T05:21:13.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.587+0000 7f416679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4160159f10 con 0x7f416006efa0 2023-10-06T05:21:13.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f416579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4160159850 0x7f416006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49522/0 (socket says 172.21.15.33:49522) 2023-10-06T05:21:13.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f416579b700 1 -- 172.21.15.33:0/1299652272 learned_addr learned my addr 172.21.15.33:0/1299652272 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:13.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4165f9c700 1 --2- 172.21.15.33:0/1299652272 >> v2:172.21.15.33:3300/0 conn(0x7f416006efa0 0x7f41600ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4157fff700 1 -- 172.21.15.33:0/1299652272 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135779044 0 0) 0x7f41600b0cd0 con 0x7f4160159850 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4157fff700 1 -- 172.21.15.33:0/1299652272 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f414c003570 con 0x7f4160159850 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4165f9c700 1 -- 172.21.15.33:0/1299652272 >> v1:172.21.15.167:6789/0 conn(0x7f416015acd0 legacy=0x7f41600abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4165f9c700 1 -- 172.21.15.33:0/1299652272 >> v1:172.21.15.33:6789/0 conn(0x7f4160159850 legacy=0x7f416006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4165f9c700 1 -- 172.21.15.33:0/1299652272 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416015a1f0 con 0x7f416006efa0 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.588+0000 7f4165f9c700 1 --2- 172.21.15.33:0/1299652272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600ac5a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4158009e60 tx=0x7f4158009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce642f8ced57855 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f4157fff700 1 -- 172.21.15.33:0/1299652272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800e070 con 0x7f416006efa0 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f4157fff700 1 -- 172.21.15.33:0/1299652272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415800bb60 con 0x7f416006efa0 2023-10-06T05:21:13.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f4157fff700 1 -- 172.21.15.33:0/1299652272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158012430 con 0x7f416006efa0 2023-10-06T05:21:13.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 -- 172.21.15.33:0/1299652272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 msgr2=0x7f41600ac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 --2- 172.21.15.33:0/1299652272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600ac5a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4158009e60 tx=0x7f4158009780 comp rx=0 tx=0).stop 2023-10-06T05:21:13.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 -- 172.21.15.33:0/1299652272 shutdown_connections 2023-10-06T05:21:13.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 --2- 172.21.15.33:0/1299652272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600ac5a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 -- 172.21.15.33:0/1299652272 >> 172.21.15.33:0/1299652272 conn(0x7f416006cd40 msgr2=0x7f41600ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:13.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 -- 172.21.15.33:0/1299652272 shutdown_connections 2023-10-06T05:21:13.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.589+0000 7f416679d700 1 -- 172.21.15.33:0/1299652272 wait complete. 2023-10-06T05:21:13.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.591+0000 7f416679d700 1 Processor -- start 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.591+0000 7f416679d700 1 -- start start 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4160159850 0x7f416015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416015acd0 0x7f4160157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600cbc60 con 0x7f416006efa0 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600ca420 con 0x7f416015acd0 2023-10-06T05:21:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41600ca700 con 0x7f4160159850 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34186/0 (socket says 172.21.15.33:34186) 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416579b700 1 -- 172.21.15.33:0/2084050514 learned_addr learned my addr 172.21.15.33:0/2084050514 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f4165f9c700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416015acd0 0x7f4160157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f416579b700 1 -- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4160159850 msgr2=0x7f416015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.592+0000 7f4164f9a700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4160159850 0x7f416015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416579b700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4160159850 0x7f416015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416579b700 1 -- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416015acd0 msgr2=0x7f4160157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416579b700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416015acd0 0x7f4160157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416579b700 1 -- 172.21.15.33:0/2084050514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4158009a20 con 0x7f416006efa0 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f4165f9c700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416015acd0 0x7f4160157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:13.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f4164f9a700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4160159850 0x7f416015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416579b700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600cb720 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f415c00c7c0 tx=0x7f415c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c005b30 con 0x7f416006efa0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c005c90 con 0x7f416006efa0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c007340 con 0x7f416006efa0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41600c74c0 con 0x7f416006efa0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.593+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41600c7a80 con 0x7f416006efa0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.594+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41600c7ca0 con 0x7f416006efa0 2023-10-06T05:21:13.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.595+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f415c01b400 con 0x7f416006efa0 2023-10-06T05:21:13.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.596+0000 7f41557fa700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4144066f30 0x7f41440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.596+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f415c084310 con 0x7f416006efa0 2023-10-06T05:21:13.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.596+0000 7f4164f9a700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4144066f30 0x7f41440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.597+0000 7f4164f9a700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4144066f30 0x7f41440693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f41600ceec0 tx=0x7f4150009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.600+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f415c051330 con 0x7f416006efa0 2023-10-06T05:21:13.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:13.849+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f416004c380 con 0x7f416006efa0 2023-10-06T05:21:14.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.109+0000 7f41557fa700 1 -- 172.21.15.33:0/2084050514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v60) v1 ==== 162+0+0 (secure 0 0 0) 0x7f415c054950 con 0x7f416006efa0 2023-10-06T05:21:14.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4144066f30 msgr2=0x7f41440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4144066f30 0x7f41440693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f41600ceec0 tx=0x7f4150009040 comp rx=0 tx=0).stop 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 msgr2=0x7f41600cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600cb720 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f415c00c7c0 tx=0x7f415c007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 shutdown_connections 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4144066f30 0x7f41440693e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416015acd0 0x7f4160157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416006efa0 0x7f41600cb720 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 --2- 172.21.15.33:0/2084050514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4160159850 0x7f416015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 >> 172.21.15.33:0/2084050514 conn(0x7f416006cd40 msgr2=0x7f41600ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 shutdown_connections 2023-10-06T05:21:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.112+0000 7f416679d700 1 -- 172.21.15.33:0/2084050514 wait complete. 2023-10-06T05:21:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:21:14.134 INFO:tasks.cephfs.test_client_recovery:Waiting for reconnect... 2023-10-06T05:21:14.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.320+0000 7fe32f19d700 1 Processor -- start 2023-10-06T05:21:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.320+0000 7fe32f19d700 1 -- start start 2023-10-06T05:21:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3280c6340 0x7fe3280c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32f19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32806c820 con 0x7fe3280c80d0 2023-10-06T05:21:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32f19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32806a160 con 0x7fe328159520 2023-10-06T05:21:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3280c8800 con 0x7fe3280c6340 2023-10-06T05:21:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3280c6340 0x7fe3280c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32d99a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe3280c6340 0x7fe3280c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34188/0 (socket says 172.21.15.33:34188) 2023-10-06T05:21:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32d99a700 1 -- 172.21.15.33:0/3258855933 learned_addr learned my addr 172.21.15.33:0/3258855933 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32d99a700 1 -- 172.21.15.33:0/3258855933 >> v1:172.21.15.167:6789/0 conn(0x7fe328159520 legacy=0x7fe3280c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32d99a700 1 -- 172.21.15.33:0/3258855933 >> v1:172.21.15.33:6789/0 conn(0x7fe3280c80d0 legacy=0x7fe32806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.322+0000 7fe32d99a700 1 -- 172.21.15.33:0/3258855933 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3280c8ae0 con 0x7fe3280c6340 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.323+0000 7fe32d99a700 1 --2- 172.21.15.33:0/3258855933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c6340 0x7fe3280c7b90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe3180099b0 tx=0x7fe318009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a599e481b74c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.323+0000 7fe32d199700 1 -- 172.21.15.33:0/3258855933 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800b860 con 0x7fe3280c6340 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.323+0000 7fe32d199700 1 -- 172.21.15.33:0/3258855933 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31800b9c0 con 0x7fe3280c6340 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.323+0000 7fe32d199700 1 -- 172.21.15.33:0/3258855933 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800bd60 con 0x7fe3280c6340 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.323+0000 7fe32f19d700 1 -- 172.21.15.33:0/3258855933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c6340 msgr2=0x7fe3280c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.323+0000 7fe32f19d700 1 --2- 172.21.15.33:0/3258855933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c6340 0x7fe3280c7b90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe3180099b0 tx=0x7fe318009250 comp rx=0 tx=0).stop 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.324+0000 7fe32f19d700 1 -- 172.21.15.33:0/3258855933 shutdown_connections 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.324+0000 7fe32f19d700 1 --2- 172.21.15.33:0/3258855933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c6340 0x7fe3280c7b90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.324+0000 7fe32f19d700 1 -- 172.21.15.33:0/3258855933 >> 172.21.15.33:0/3258855933 conn(0x7fe3280cc930 msgr2=0x7fe3280ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.324+0000 7fe32f19d700 1 -- 172.21.15.33:0/3258855933 shutdown_connections 2023-10-06T05:21:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.324+0000 7fe32f19d700 1 -- 172.21.15.33:0/3258855933 wait complete. 2023-10-06T05:21:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.325+0000 7fe32f19d700 1 Processor -- start 2023-10-06T05:21:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.325+0000 7fe32f19d700 1 -- start start 2023-10-06T05:21:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3280c6340 0x7fe32814a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 0x7fe3280aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe328159520 0x7fe3280af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32815af00 con 0x7fe3280c80d0 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32816a5d0 con 0x7fe3280c6340 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281699c0 con 0x7fe328159520 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 0x7fe3280aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 0x7fe3280aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34190/0 (socket says 172.21.15.33:34190) 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32d99a700 1 -- 172.21.15.33:0/2473848465 learned_addr learned my addr 172.21.15.33:0/2473848465 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32e99c700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe328159520 0x7fe3280af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.326+0000 7fe32e19b700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3280c6340 0x7fe32814a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32d99a700 1 -- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe328159520 msgr2=0x7fe3280af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32d99a700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe328159520 0x7fe3280af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32d99a700 1 -- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3280c6340 msgr2=0x7fe32814a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32d99a700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3280c6340 0x7fe32814a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32d99a700 1 -- 172.21.15.33:0/2473848465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180094f0 con 0x7fe3280c80d0 2023-10-06T05:21:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32e99c700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe328159520 0x7fe3280af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:14.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32d99a700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 0x7fe3280aeb90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe3180094c0 tx=0x7fe318017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318011e10 con 0x7fe3280c80d0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31801a630 con 0x7fe3280c80d0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31801a9f0 con 0x7fe3280c80d0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe328169ca0 con 0x7fe3280c80d0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.327+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32816a260 con 0x7fe3280c80d0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.328+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32804c380 con 0x7fe3280c80d0 2023-10-06T05:21:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.329+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe31802d440 con 0x7fe3280c80d0 2023-10-06T05:21:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.330+0000 7fe31e7fc700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe30c066f30 0x7fe30c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.330+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fe318086450 con 0x7fe3280c80d0 2023-10-06T05:21:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.331+0000 7fe32e19b700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe30c066f30 0x7fe30c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.331+0000 7fe32e19b700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe30c066f30 0x7fe30c0693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe3240037c0 tx=0x7fe324009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.335+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3180533f0 con 0x7fe3280c80d0 2023-10-06T05:21:14.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.580+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe32816a710 con 0x7fe3280c80d0 2023-10-06T05:21:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.581+0000 7fe31e7fc700 1 -- 172.21.15.33:0/2473848465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fe318056a10 con 0x7fe3280c80d0 2023-10-06T05:21:14.582 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:14.582 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:14.110075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe30c066f30 msgr2=0x7fe30c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe30c066f30 0x7fe30c0693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe3240037c0 tx=0x7fe324009040 comp rx=0 tx=0).stop 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 msgr2=0x7fe3280aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 0x7fe3280aeb90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe3180094c0 tx=0x7fe318017000 comp rx=0 tx=0).stop 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 shutdown_connections 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe30c066f30 0x7fe30c0693e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe328159520 0x7fe3280af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3280c6340 0x7fe32814a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 --2- 172.21.15.33:0/2473848465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3280c80d0 0x7fe3280aeb90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 >> 172.21.15.33:0/2473848465 conn(0x7fe3280cc930 msgr2=0x7fe3281585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 shutdown_connections 2023-10-06T05:21:14.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:14.583+0000 7fe32f19d700 1 -- 172.21.15.33:0/2473848465 wait complete. 2023-10-06T05:21:14.586 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 61 2023-10-06T05:21:14.603 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2023-10-06T05:21:15.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.801+0000 7fcbe279d700 1 Processor -- start 2023-10-06T05:21:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.801+0000 7fcbe279d700 1 -- start start 2023-10-06T05:21:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.802+0000 7fcbe279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbdc0ccfc0 0x7fcbdc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.802+0000 7fcbe279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cf3b0 0x7fcbdc1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.802+0000 7fcbe279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbdc0cc750 0x7fcbdc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.802+0000 7fcbe279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc0cbd80 con 0x7fcbdc0cc750 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.802+0000 7fcbe279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc0af5f0 con 0x7fcbdc0ccfc0 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.802+0000 7fcbe279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0af8d0 con 0x7fcbdc0cf3b0 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbdc0cc750 0x7fcbdc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbdc0cc750 0x7fcbdc1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34194/0 (socket says 172.21.15.33:34194) 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/3982170143 learned_addr learned my addr 172.21.15.33:0/3982170143 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe0f9a700 1 --2- 172.21.15.33:0/3982170143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cf3b0 0x7fcbdc1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe179b700 1 --2- 172.21.15.33:0/3982170143 >> v2:172.21.15.167:3300/0 conn(0x7fcbdc0ccfc0 0x7fcbdc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/3982170143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cf3b0 msgr2=0x7fcbdc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 --2- 172.21.15.33:0/3982170143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cf3b0 0x7fcbdc1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/3982170143 >> v2:172.21.15.167:3300/0 conn(0x7fcbdc0ccfc0 msgr2=0x7fcbdc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 --2- 172.21.15.33:0/3982170143 >> v2:172.21.15.167:3300/0 conn(0x7fcbdc0ccfc0 0x7fcbdc0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/3982170143 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0afbb0 con 0x7fcbdc0cc750 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.803+0000 7fcbe1f9c700 1 --2- 172.21.15.33:0/3982170143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0cc750 0x7fcbdc1599e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc06d790 tx=0x7fcbd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ec39e4e5ec6266 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbd3fff700 1 -- 172.21.15.33:0/3982170143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400e070 con 0x7fcbdc0cc750 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbd3fff700 1 -- 172.21.15.33:0/3982170143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd400b970 con 0x7fcbdc0cc750 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbd3fff700 1 -- 172.21.15.33:0/3982170143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40092e0 con 0x7fcbdc0cc750 2023-10-06T05:21:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 -- 172.21.15.33:0/3982170143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0cc750 msgr2=0x7fcbdc1599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 --2- 172.21.15.33:0/3982170143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0cc750 0x7fcbdc1599e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc06d790 tx=0x7fcbd4009840 comp rx=0 tx=0).stop 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 -- 172.21.15.33:0/3982170143 shutdown_connections 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 --2- 172.21.15.33:0/3982170143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0cc750 0x7fcbdc1599e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 --2- 172.21.15.33:0/3982170143 >> v2:172.21.15.167:3300/0 conn(0x7fcbdc0ccfc0 0x7fcbdc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 --2- 172.21.15.33:0/3982170143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cf3b0 0x7fcbdc1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 -- 172.21.15.33:0/3982170143 >> 172.21.15.33:0/3982170143 conn(0x7fcbdc06cff0 msgr2=0x7fcbdc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 -- 172.21.15.33:0/3982170143 shutdown_connections 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.804+0000 7fcbe279d700 1 -- 172.21.15.33:0/3982170143 wait complete. 2023-10-06T05:21:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.806+0000 7fcbe279d700 1 Processor -- start 2023-10-06T05:21:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.806+0000 7fcbe279d700 1 -- start start 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cc750 0x7fcbdc153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbdc0ccfc0 0x7fcbdc0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 0x7fcbdc157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c75b0 con 0x7fcbdc0c9590 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c83d0 con 0x7fcbdc0ccfc0 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0c77c0 con 0x7fcbdc0cc750 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 0x7fcbdc157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe1f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 0x7fcbdc157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34202/0 (socket says 172.21.15.33:34202) 2023-10-06T05:21:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/1390966578 learned_addr learned my addr 172.21.15.33:0/1390966578 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe179b700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cc750 0x7fcbdc153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.807+0000 7fcbe0f9a700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbdc0ccfc0 0x7fcbdc0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cc750 msgr2=0x7fcbdc153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe1f9c700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cc750 0x7fcbdc153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbdc0ccfc0 msgr2=0x7fcbdc0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe1f9c700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbdc0ccfc0 0x7fcbdc0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe1f9c700 1 -- 172.21.15.33:0/1390966578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4009ae0 con 0x7fcbdc0c9590 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe1f9c700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 0x7fcbdc157c30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009880 tx=0x7fcbd4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4008b10 con 0x7fcbdc0c9590 2023-10-06T05:21:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd4008c70 con 0x7fcbdc0c9590 2023-10-06T05:21:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc0c7aa0 con 0x7fcbdc0c9590 2023-10-06T05:21:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4026460 con 0x7fcbdc0c9590 2023-10-06T05:21:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.808+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc0c8060 con 0x7fcbdc0c9590 2023-10-06T05:21:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.809+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc04c380 con 0x7fcbdc0c9590 2023-10-06T05:21:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.810+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbd4003040 con 0x7fcbdc0c9590 2023-10-06T05:21:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.811+0000 7fcbd17fa700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcbc0066f30 0x7fcbc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.811+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fcbd4086600 con 0x7fcbdc0c9590 2023-10-06T05:21:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.811+0000 7fcbe179b700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcbc0066f30 0x7fcbc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.812+0000 7fcbe179b700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcbc0066f30 0x7fcbc00693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc158a60 tx=0x7fcbd80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:15.815+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbd40535a0 con 0x7fcbdc0c9590 2023-10-06T05:21:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.061+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbdc0cb2e0 con 0x7fcbdc0c9590 2023-10-06T05:21:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.062+0000 7fcbd17fa700 1 -- 172.21.15.33:0/1390966578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fcbd4056bc0 con 0x7fcbdc0c9590 2023-10-06T05:21:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:16.063 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcbc0066f30 msgr2=0x7fcbc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcbc0066f30 0x7fcbc00693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc158a60 tx=0x7fcbd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 msgr2=0x7fcbdc157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 0x7fcbdc157c30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009880 tx=0x7fcbd4018000 comp rx=0 tx=0).stop 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 shutdown_connections 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcbc0066f30 0x7fcbc00693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbdc0c9590 0x7fcbdc157c30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbdc0cc750 0x7fcbdc153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 --2- 172.21.15.33:0/1390966578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbdc0ccfc0 0x7fcbdc0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 >> 172.21.15.33:0/1390966578 conn(0x7fcbdc06cff0 msgr2=0x7fcbdc155af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 shutdown_connections 2023-10-06T05:21:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.064+0000 7fcbe279d700 1 -- 172.21.15.33:0/1390966578 wait complete. 2023-10-06T05:21:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:16.084 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to up:reconnect 2023-10-06T05:21:16.084 DEBUG:tasks.cephfs.filesystem:reached state 'up:reconnect' in 1.9505841732025146s 2023-10-06T05:21:16.084 INFO:tasks.cephfs.test_client_recovery:Waiting for active... 2023-10-06T05:21:16.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:16.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.270+0000 7fb31a39d700 1 Processor -- start 2023-10-06T05:21:16.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.270+0000 7fb31a39d700 1 -- start start 2023-10-06T05:21:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb31a39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb31406ab00 0x7fb3140b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb31a39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb3140b2e30 0x7fb31415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb31a39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31415e2d0 con 0x7fb3140b0a40 2023-10-06T05:21:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb31a39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31406afa0 con 0x7fb31406ab00 2023-10-06T05:21:16.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb31a39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31415d6c0 con 0x7fb3140b2e30 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb31406ab00 0x7fb3140b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb31406ab00 0x7fb3140b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34214/0 (socket says 172.21.15.33:34214) 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 -- 172.21.15.33:0/1079715174 learned_addr learned my addr 172.21.15.33:0/1079715174 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 -- 172.21.15.33:0/1079715174 >> v2:172.21.15.167:3300/0 conn(0x7fb3140b2e30 msgr2=0x7fb31415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 --2- 172.21.15.33:0/1079715174 >> v2:172.21.15.167:3300/0 conn(0x7fb3140b2e30 0x7fb31415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 -- 172.21.15.33:0/1079715174 >> v1:172.21.15.167:6789/0 conn(0x7fb3140b0a40 legacy=0x7fb3140b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.272+0000 7fb318b9a700 1 -- 172.21.15.33:0/1079715174 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31415d9a0 con 0x7fb31406ab00 2023-10-06T05:21:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.273+0000 7fb318b9a700 1 --2- 172.21.15.33:0/1079715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb31406ab00 0x7fb3140b28f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb3140b0740 tx=0x7fb304009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13fc823dfde1001 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.273+0000 7fb30bfff700 1 -- 172.21.15.33:0/1079715174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400b820 con 0x7fb31406ab00 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.273+0000 7fb30bfff700 1 -- 172.21.15.33:0/1079715174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30400b980 con 0x7fb31406ab00 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.273+0000 7fb30bfff700 1 -- 172.21.15.33:0/1079715174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400bd40 con 0x7fb31406ab00 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.273+0000 7fb31a39d700 1 -- 172.21.15.33:0/1079715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb31406ab00 msgr2=0x7fb3140b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.273+0000 7fb31a39d700 1 --2- 172.21.15.33:0/1079715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb31406ab00 0x7fb3140b28f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb3140b0740 tx=0x7fb304009250 comp rx=0 tx=0).stop 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.274+0000 7fb31a39d700 1 -- 172.21.15.33:0/1079715174 shutdown_connections 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.274+0000 7fb31a39d700 1 --2- 172.21.15.33:0/1079715174 >> v2:172.21.15.167:3300/0 conn(0x7fb3140b2e30 0x7fb31415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.274+0000 7fb31a39d700 1 --2- 172.21.15.33:0/1079715174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb31406ab00 0x7fb3140b28f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.274+0000 7fb31a39d700 1 -- 172.21.15.33:0/1079715174 >> 172.21.15.33:0/1079715174 conn(0x7fb31406e0f0 msgr2=0x7fb3140675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.274+0000 7fb31a39d700 1 -- 172.21.15.33:0/1079715174 shutdown_connections 2023-10-06T05:21:16.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.274+0000 7fb31a39d700 1 -- 172.21.15.33:0/1079715174 wait complete. 2023-10-06T05:21:16.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.275+0000 7fb31a39d700 1 Processor -- start 2023-10-06T05:21:16.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.275+0000 7fb31a39d700 1 -- start start 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb31a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb31406ab00 0x7fb31416a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb31a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3140b0a40 0x7fb3140c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb31a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 0x7fb3140c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb31a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c39c0 con 0x7fb3140b2e30 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb31a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c2db0 con 0x7fb31406ab00 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb31a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c3090 con 0x7fb3140b0a40 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb319b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 0x7fb3140c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb319b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 0x7fb3140c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34216/0 (socket says 172.21.15.33:34216) 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.276+0000 7fb319b9c700 1 -- 172.21.15.33:0/372655077 learned_addr learned my addr 172.21.15.33:0/372655077 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb318b9a700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3140b0a40 0x7fb3140c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb31939b700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb31406ab00 0x7fb31416a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb319b9c700 1 -- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3140b0a40 msgr2=0x7fb3140c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb319b9c700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3140b0a40 0x7fb3140c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb319b9c700 1 -- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb31406ab00 msgr2=0x7fb31416a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb319b9c700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb31406ab00 0x7fb31416a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb319b9c700 1 -- 172.21.15.33:0/372655077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040094f0 con 0x7fb3140b2e30 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb319b9c700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 0x7fb3140c5700 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00bc30 tx=0x7fb30c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0092e0 con 0x7fb3140b2e30 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00c7a0 con 0x7fb3140b2e30 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3140c3370 con 0x7fb3140b2e30 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00cb80 con 0x7fb3140b2e30 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.277+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31416a850 con 0x7fb3140b2e30 2023-10-06T05:21:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.278+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3140ac570 con 0x7fb3140b2e30 2023-10-06T05:21:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.279+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb30c012070 con 0x7fb3140b2e30 2023-10-06T05:21:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.280+0000 7fb3097fa700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2f8066f30 0x7fb2f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.280+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb30c0844c0 con 0x7fb3140b2e30 2023-10-06T05:21:16.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.281+0000 7fb31939b700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2f8066f30 0x7fb2f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.281+0000 7fb31939b700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2f8066f30 0x7fb2f80693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb310006fd0 tx=0x7fb310008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:16.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.284+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb30c051460 con 0x7fb3140b2e30 2023-10-06T05:21:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.530+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31406fcc0 con 0x7fb3140b2e30 2023-10-06T05:21:16.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.531+0000 7fb3097fa700 1 -- 172.21.15.33:0/372655077 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fb30c054a80 con 0x7fb3140b2e30 2023-10-06T05:21:16.532 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:16.533 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2f8066f30 msgr2=0x7fb2f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2f8066f30 0x7fb2f80693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb310006fd0 tx=0x7fb310008040 comp rx=0 tx=0).stop 2023-10-06T05:21:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 msgr2=0x7fb3140c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 0x7fb3140c5700 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00bc30 tx=0x7fb30c009840 comp rx=0 tx=0).stop 2023-10-06T05:21:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 shutdown_connections 2023-10-06T05:21:16.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb2f8066f30 0x7fb2f80693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3140b2e30 0x7fb3140c5700 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb31406ab00 0x7fb31416a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 --2- 172.21.15.33:0/372655077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3140b0a40 0x7fb3140c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 >> 172.21.15.33:0/372655077 conn(0x7fb31406e0f0 msgr2=0x7fb3140abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 shutdown_connections 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:16.533+0000 7fb31a39d700 1 -- 172.21.15.33:0/372655077 wait complete. 2023-10-06T05:21:16.536 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:16.553 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:17.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:17.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.750+0000 7f744619d700 1 Processor -- start 2023-10-06T05:21:17.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.750+0000 7f744619d700 1 -- start start 2023-10-06T05:21:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.751+0000 7f744619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 0x7f74400b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.751+0000 7f744619d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f74400b2e30 0x7f744015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.751+0000 7f744619d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744015e2d0 con 0x7f74400b0a40 2023-10-06T05:21:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.751+0000 7f744619d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744006af70 con 0x7f74400b2e30 2023-10-06T05:21:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.751+0000 7f744619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744015d6c0 con 0x7f744006ab00 2023-10-06T05:21:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744499a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 0x7f74400b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744599c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f74400b2e30 0x7f744015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744499a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 0x7f74400b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35400/0 (socket says 172.21.15.33:35400) 2023-10-06T05:21:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744499a700 1 -- 172.21.15.33:0/1131064088 learned_addr learned my addr 172.21.15.33:0/1131064088 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744519b700 1 --1- 172.21.15.33:0/1131064088 >> v1:172.21.15.167:6789/0 conn(0x7f74400b0a40 0x7f74400b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:51322/0 (socket says 172.21.15.33:51322) 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744499a700 1 -- 172.21.15.33:0/1131064088 >> v2:172.21.15.167:3300/0 conn(0x7f74400b2e30 msgr2=0x7f744015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.752+0000 7f744499a700 1 --2- 172.21.15.33:0/1131064088 >> v2:172.21.15.167:3300/0 conn(0x7f74400b2e30 0x7f744015a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f744499a700 1 -- 172.21.15.33:0/1131064088 >> v1:172.21.15.167:6789/0 conn(0x7f74400b0a40 legacy=0x7f74400b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f744499a700 1 -- 172.21.15.33:0/1131064088 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744015d9a0 con 0x7f744006ab00 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f744599c700 1 --2- 172.21.15.33:0/1131064088 >> v2:172.21.15.167:3300/0 conn(0x7f74400b2e30 0x7f744015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f744499a700 1 --2- 172.21.15.33:0/1131064088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 0x7f74400b28f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f74300099b0 tx=0x7f7430009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdaae64b47430aa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f7437fff700 1 -- 172.21.15.33:0/1131064088 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430009ea0 con 0x7f744006ab00 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f7437fff700 1 -- 172.21.15.33:0/1131064088 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743000b860 con 0x7f744006ab00 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.753+0000 7f7437fff700 1 -- 172.21.15.33:0/1131064088 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743000bc00 con 0x7f744006ab00 2023-10-06T05:21:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 -- 172.21.15.33:0/1131064088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 msgr2=0x7f74400b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 --2- 172.21.15.33:0/1131064088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 0x7f74400b28f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f74300099b0 tx=0x7f7430009250 comp rx=0 tx=0).stop 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 -- 172.21.15.33:0/1131064088 shutdown_connections 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 --2- 172.21.15.33:0/1131064088 >> v2:172.21.15.167:3300/0 conn(0x7f74400b2e30 0x7f744015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 --2- 172.21.15.33:0/1131064088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f744006ab00 0x7f74400b28f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 -- 172.21.15.33:0/1131064088 >> 172.21.15.33:0/1131064088 conn(0x7f744006e0f0 msgr2=0x7f74400675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 -- 172.21.15.33:0/1131064088 shutdown_connections 2023-10-06T05:21:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.754+0000 7f744619d700 1 -- 172.21.15.33:0/1131064088 wait complete. 2023-10-06T05:21:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.755+0000 7f744619d700 1 Processor -- start 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.755+0000 7f744619d700 1 -- start start 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.756+0000 7f744619d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f744006ab00 0x7f7440146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.756+0000 7f744619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74400b0a40 0x7f74400a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.756+0000 7f744619d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 0x7f74400aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.756+0000 7f744619d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744015b270 con 0x7f74400b2e30 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.756+0000 7f744619d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744015b550 con 0x7f744006ab00 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.756+0000 7f744619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744015b830 con 0x7f74400b0a40 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 0x7f74400aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 0x7f74400aa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34218/0 (socket says 172.21.15.33:34218) 2023-10-06T05:21:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 -- 172.21.15.33:0/4039129193 learned_addr learned my addr 172.21.15.33:0/4039129193 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744499a700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74400b0a40 0x7f74400a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744519b700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f744006ab00 0x7f7440146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 -- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74400b0a40 msgr2=0x7f74400a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74400b0a40 0x7f74400a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 -- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f744006ab00 msgr2=0x7f7440146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f744006ab00 0x7f7440146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 -- 172.21.15.33:0/4039129193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438009a20 con 0x7f74400b2e30 2023-10-06T05:21:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.757+0000 7f744599c700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 0x7f74400aa3d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7438009fc0 tx=0x7f743800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.758+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800cbd0 con 0x7f74400b2e30 2023-10-06T05:21:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.758+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743800f040 con 0x7f74400b2e30 2023-10-06T05:21:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.758+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438014480 con 0x7f74400b2e30 2023-10-06T05:21:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.758+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74300094f0 con 0x7f74400b2e30 2023-10-06T05:21:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.758+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744015bdf0 con 0x7f74400b2e30 2023-10-06T05:21:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.759+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744006fcc0 con 0x7f74400b2e30 2023-10-06T05:21:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.760+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7438003020 con 0x7f74400b2e30 2023-10-06T05:21:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.760+0000 7f74357fa700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.760+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f7438085080 con 0x7f74400b2e30 2023-10-06T05:21:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.761+0000 7f744519b700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.762+0000 7f744519b700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f743c004fd0 tx=0x7f743c007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:17.765+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74380520c0 con 0x7f74400b2e30 2023-10-06T05:21:18.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.010+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f744015c010 con 0x7f74400b2e30 2023-10-06T05:21:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.010+0000 7f74357fa700 1 -- 172.21.15.33:0/4039129193 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f74380556e0 con 0x7f74400b2e30 2023-10-06T05:21:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:18.012 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 msgr2=0x7f74240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f743c004fd0 tx=0x7f743c007eb0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 msgr2=0x7f74400aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 0x7f74400aa3d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7438009fc0 tx=0x7f743800b040 comp rx=0 tx=0).stop 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 shutdown_connections 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74400b2e30 0x7f74400aa3d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f744006ab00 0x7f7440146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 --2- 172.21.15.33:0/4039129193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74400b0a40 0x7f74400a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 >> 172.21.15.33:0/4039129193 conn(0x7f744006e0f0 msgr2=0x7f74400b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 shutdown_connections 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:18.013+0000 7f744619d700 1 -- 172.21.15.33:0/4039129193 wait complete. 2023-10-06T05:21:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:18.032 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:18.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:18.638 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:19.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.229+0000 7f01bd59d700 1 Processor -- start 2023-10-06T05:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.229+0000 7f01bd59d700 1 -- start start 2023-10-06T05:21:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01bd59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01b80c8050 0x7f01b813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01bd59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01b806aad0 0x7f01b806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01bd59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b80c9cb0 con 0x7f01b813faa0 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01bd59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b80ca270 con 0x7f01b806aad0 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01bd59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b806b840 con 0x7f01b80c8050 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01bcd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01b806aad0 0x7f01b806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01b7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01b813faa0 0x7f01b80c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49532/0 (socket says 172.21.15.33:49532) 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01b7fff700 1 -- 172.21.15.33:0/2310234644 learned_addr learned my addr 172.21.15.33:0/2310234644 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.231+0000 7f01b77fe700 1 --2- 172.21.15.33:0/2310234644 >> v2:172.21.15.167:3300/0 conn(0x7f01b80c8050 0x7f01b813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01bcd9c700 1 -- 172.21.15.33:0/2310234644 >> v2:172.21.15.167:3300/0 conn(0x7f01b80c8050 msgr2=0x7f01b813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01bcd9c700 1 --2- 172.21.15.33:0/2310234644 >> v2:172.21.15.167:3300/0 conn(0x7f01b80c8050 0x7f01b813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01bcd9c700 1 -- 172.21.15.33:0/2310234644 >> v1:172.21.15.33:6789/0 conn(0x7f01b813faa0 legacy=0x7f01b80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01bcd9c700 1 -- 172.21.15.33:0/2310234644 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b806bb20 con 0x7f01b806aad0 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01b77fe700 1 --2- 172.21.15.33:0/2310234644 >> v2:172.21.15.167:3300/0 conn(0x7f01b80c8050 0x7f01b813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01bcd9c700 1 --2- 172.21.15.33:0/2310234644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b806aad0 0x7f01b806af30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f01b80cbef0 tx=0x7f01b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0312fe8b9bf2856 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01b6ffd700 1 -- 172.21.15.33:0/2310234644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b000e070 con 0x7f01b806aad0 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01b6ffd700 1 -- 172.21.15.33:0/2310234644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b000bc30 con 0x7f01b806aad0 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.232+0000 7f01b6ffd700 1 -- 172.21.15.33:0/2310234644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0012430 con 0x7f01b806aad0 2023-10-06T05:21:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 -- 172.21.15.33:0/2310234644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b806aad0 msgr2=0x7f01b806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 --2- 172.21.15.33:0/2310234644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b806aad0 0x7f01b806af30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f01b80cbef0 tx=0x7f01b0009780 comp rx=0 tx=0).stop 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 -- 172.21.15.33:0/2310234644 shutdown_connections 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 --2- 172.21.15.33:0/2310234644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b806aad0 0x7f01b806af30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 --2- 172.21.15.33:0/2310234644 >> v2:172.21.15.167:3300/0 conn(0x7f01b80c8050 0x7f01b813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 -- 172.21.15.33:0/2310234644 >> 172.21.15.33:0/2310234644 conn(0x7f01b806d030 msgr2=0x7f01b806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 -- 172.21.15.33:0/2310234644 shutdown_connections 2023-10-06T05:21:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.233+0000 7f01bd59d700 1 -- 172.21.15.33:0/2310234644 wait complete. 2023-10-06T05:21:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.234+0000 7f01bd59d700 1 Processor -- start 2023-10-06T05:21:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.234+0000 7f01bd59d700 1 -- start start 2023-10-06T05:21:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bd59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b806aad0 0x7f01b815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bd59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b80c8050 0x7f01b813e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bd59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 0x7f01b8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bd59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b80c3190 con 0x7f01b813faa0 2023-10-06T05:21:19.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bd59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b80c3d10 con 0x7f01b806aad0 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bd59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b80c33e0 con 0x7f01b80c8050 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.235+0000 7f01bcd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 0x7f01b8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 0x7f01b8158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34228/0 (socket says 172.21.15.33:34228) 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 -- 172.21.15.33:0/1714283732 learned_addr learned my addr 172.21.15.33:0/1714283732 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01b7fff700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b806aad0 0x7f01b815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01b77fe700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b80c8050 0x7f01b813e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 -- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b80c8050 msgr2=0x7f01b813e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b80c8050 0x7f01b813e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 -- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b806aad0 msgr2=0x7f01b815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b806aad0 0x7f01b815fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 -- 172.21.15.33:0/1714283732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b0009a20 con 0x7f01b813faa0 2023-10-06T05:21:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f01bcd9c700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 0x7f01b8158d30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f01b80719c0 tx=0x7f01b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b001e630 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.236+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b001e7d0 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.237+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b80c36c0 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.237+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b001eb90 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.237+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b8160430 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.238+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b80c3ea0 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.238+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01b0024070 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.239+0000 7f019ffff700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01a4066f30 0x7f01a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.239+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f01b0086fe0 con 0x7f01b813faa0 2023-10-06T05:21:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.240+0000 7f01b7fff700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01a4066f30 0x7f01a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.240+0000 7f01b7fff700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01a4066f30 0x7f01a40693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f01a8001780 tx=0x7f01a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.244+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01b0053f80 con 0x7f01b813faa0 2023-10-06T05:21:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.491+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01b804c380 con 0x7f01b813faa0 2023-10-06T05:21:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.492+0000 7f019ffff700 1 -- 172.21.15.33:0/1714283732 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f01b00575a0 con 0x7f01b813faa0 2023-10-06T05:21:19.493 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:19.493 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01a4066f30 msgr2=0x7f01a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01a4066f30 0x7f01a40693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f01a8001780 tx=0x7f01a8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 msgr2=0x7f01b8158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 0x7f01b8158d30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f01b80719c0 tx=0x7f01b0019000 comp rx=0 tx=0).stop 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 shutdown_connections 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01a4066f30 0x7f01a40693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b813faa0 0x7f01b8158d30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b806aad0 0x7f01b815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 --2- 172.21.15.33:0/1714283732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01b80c8050 0x7f01b813e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 >> 172.21.15.33:0/1714283732 conn(0x7f01b806d030 msgr2=0x7f01b815a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 shutdown_connections 2023-10-06T05:21:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:19.494+0000 7f01bd59d700 1 -- 172.21.15.33:0/1714283732 wait complete. 2023-10-06T05:21:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:19.513 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:20.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:20.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.710+0000 7f636e79d700 1 Processor -- start 2023-10-06T05:21:20.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.711+0000 7f636e79d700 1 -- start start 2023-10-06T05:21:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636e79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63680b0a40 0x7f636806ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f636815a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636e79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636815e2d0 con 0x7f63680b0e10 2023-10-06T05:21:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636e79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636815d6c0 con 0x7f63680b0a40 2023-10-06T05:21:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636815d9a0 con 0x7f636815a0a0 2023-10-06T05:21:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636cf9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f63680b0e10 0x7f63680b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54498/0 (socket says 172.21.15.33:54498) 2023-10-06T05:21:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636cf9a700 1 -- 172.21.15.33:0/1274476629 learned_addr learned my addr 172.21.15.33:0/1274476629 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636d79b700 1 --2- 172.21.15.33:0/1274476629 >> v2:172.21.15.167:3300/0 conn(0x7f63680b0a40 0x7f636806ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.712+0000 7f636df9c700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f636815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518484692 0 0) 0x7f636815e2d0 con 0x7f63680b0e10 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6354003570 con 0x7f63680b0e10 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3284754817 0 0) 0x7f6354003570 con 0x7f63680b0e10 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 msgr2=0x7f636815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f636815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 >> v2:172.21.15.167:3300/0 conn(0x7f63680b0a40 msgr2=0x7f636806ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 --2- 172.21.15.33:0/1274476629 >> v2:172.21.15.167:3300/0 conn(0x7f63680b0a40 0x7f636806ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636815dc80 con 0x7f63680b0e10 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f6358001470 con 0x7f63680b0e10 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 >> v1:172.21.15.33:6789/0 conn(0x7f63680b0e10 legacy=0x7f63680b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f6354003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6354003fe0 0x7f6354004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354004d50 0x7f63540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636815d6c0 con 0x7f6354003fe0 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636815d9a0 con 0x7f6354004d50 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635400ba90 con 0x7f636815a0a0 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.713+0000 7f636d79b700 1 --2- 172.21.15.33:0/1274476629 >> v2:172.21.15.167:3300/0 conn(0x7f63680b0a40 0x7f636806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6354003fe0 0x7f6354004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636df9c700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354004d50 0x7f63540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636d79b700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f6354003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 -- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 msgr2=0x7f6354003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f6354003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 -- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354004d50 msgr2=0x7f63540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354004d50 0x7f63540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 -- 172.21.15.33:0/1274476629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354003570 con 0x7f6354003fe0 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636d79b700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636815a0a0 0x7f6354003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f636cf9a700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6354003fe0 0x7f6354004810 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f635800c3c0 tx=0x7f6358001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1657802b3eddd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800cc30 con 0x7f6354003fe0 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800cdd0 con 0x7f6354003fe0 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.714+0000 7f635ffff700 1 -- 172.21.15.33:0/1274476629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358005b70 con 0x7f6354003fe0 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636e79d700 1 -- 172.21.15.33:0/1274476629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6354003fe0 msgr2=0x7f6354004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636e79d700 1 --2- 172.21.15.33:0/1274476629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6354003fe0 0x7f6354004810 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f635800c3c0 tx=0x7f6358001e90 comp rx=0 tx=0).stop 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636df9c700 1 -- 172.21.15.33:0/1274476629 reap_dead start 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636e79d700 1 -- 172.21.15.33:0/1274476629 shutdown_connections 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636e79d700 1 -- 172.21.15.33:0/1274476629 >> 172.21.15.33:0/1274476629 conn(0x7f636806e0f0 msgr2=0x7f63680675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636e79d700 1 -- 172.21.15.33:0/1274476629 shutdown_connections 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.715+0000 7f636e79d700 1 -- 172.21.15.33:0/1274476629 wait complete. 2023-10-06T05:21:20.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.716+0000 7f636e79d700 1 Processor -- start 2023-10-06T05:21:20.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.717+0000 7f636e79d700 1 -- start start 2023-10-06T05:21:20.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.717+0000 7f636e79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354003fe0 0x7f63680c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.717+0000 7f636e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63680aa860 0x7f63680aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.717+0000 7f636e79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 0x7f63680c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.717+0000 7f636e79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c5140 con 0x7f63680c4760 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636e79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680abfe0 con 0x7f6354003fe0 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c3fc0 con 0x7f63680aa860 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 0x7f63680c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 0x7f63680c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54580/0 (socket says 172.21.15.33:54580) 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 -- 172.21.15.33:0/1690489582 learned_addr learned my addr 172.21.15.33:0/1690489582 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636d79b700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354003fe0 0x7f63680c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636cf9a700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63680aa860 0x7f63680aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 -- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63680aa860 msgr2=0x7f63680aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63680aa860 0x7f63680aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 -- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354003fe0 msgr2=0x7f63680c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354003fe0 0x7f63680c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636df9c700 1 -- 172.21.15.33:0/1690489582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636816a280 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.718+0000 7f636cf9a700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63680aa860 0x7f63680aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.719+0000 7f636df9c700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 0x7f63680c4c00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f636806d380 tx=0x7f636000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.719+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000e070 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.719+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360011d00 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.719+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600166c0 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.719+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63680ab3d0 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.719+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63680ab990 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.720+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636806fcc0 con 0x7f63680c4760 2023-10-06T05:21:20.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.721+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6360016af0 con 0x7f63680c4760 2023-10-06T05:21:20.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.722+0000 7f635d7fa700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f634c066f30 0x7f634c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.722+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f6360085310 con 0x7f63680c4760 2023-10-06T05:21:20.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.722+0000 7f636d79b700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f634c066f30 0x7f634c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.723+0000 7f636d79b700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f634c066f30 0x7f634c0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63640093f0 tx=0x7f6364008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.726+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63600522b0 con 0x7f63680c4760 2023-10-06T05:21:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.972+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63680ac170 con 0x7f63680c4760 2023-10-06T05:21:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.972+0000 7f635d7fa700 1 -- 172.21.15.33:0/1690489582 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f63600558d0 con 0x7f63680c4760 2023-10-06T05:21:20.974 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:20.974 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f634c066f30 msgr2=0x7f634c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f634c066f30 0x7f634c0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63640093f0 tx=0x7f6364008040 comp rx=0 tx=0).stop 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 msgr2=0x7f63680c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 0x7f63680c4c00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f636806d380 tx=0x7f636000b040 comp rx=0 tx=0).stop 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 shutdown_connections 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f634c066f30 0x7f634c0693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63680c4760 0x7f63680c4c00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6354003fe0 0x7f63680c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 --2- 172.21.15.33:0/1690489582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63680aa860 0x7f63680aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 >> 172.21.15.33:0/1690489582 conn(0x7f636806e0f0 msgr2=0x7f63680cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 shutdown_connections 2023-10-06T05:21:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:20.975+0000 7f636e79d700 1 -- 172.21.15.33:0/1690489582 wait complete. 2023-10-06T05:21:20.978 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:20.995 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:21.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.193+0000 7fe69119d700 1 Processor -- start 2023-10-06T05:21:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.194+0000 7fe69119d700 1 -- start start 2023-10-06T05:21:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe68c0c7940 0x7fe68c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69119d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe68c06aad0 0x7fe68c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69119d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c0c9cb0 con 0x7fe68c06aad0 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c0ca270 con 0x7fe68c0c7940 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c06b840 con 0x7fe68c13faa0 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69099c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe68c06aad0 0x7fe68c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69099c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe68c06aad0 0x7fe68c06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54594/0 (socket says 172.21.15.33:54594) 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe69099c700 1 -- 172.21.15.33:0/3064718130 learned_addr learned my addr 172.21.15.33:0/3064718130 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe68bfff700 1 --2- 172.21.15.33:0/3064718130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.195+0000 7fe68b7fe700 1 --2- 172.21.15.33:0/3064718130 >> v2:172.21.15.167:3300/0 conn(0x7fe68c0c7940 0x7fe68c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe69099c700 1 -- 172.21.15.33:0/3064718130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 msgr2=0x7fe68c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe69099c700 1 --2- 172.21.15.33:0/3064718130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe69099c700 1 -- 172.21.15.33:0/3064718130 >> v2:172.21.15.167:3300/0 conn(0x7fe68c0c7940 msgr2=0x7fe68c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe69099c700 1 --2- 172.21.15.33:0/3064718130 >> v2:172.21.15.167:3300/0 conn(0x7fe68c0c7940 0x7fe68c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe69099c700 1 -- 172.21.15.33:0/3064718130 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c06bb20 con 0x7fe68c06aad0 2023-10-06T05:21:22.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe69099c700 1 --2- 172.21.15.33:0/3064718130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c06aad0 0x7fe68c06af50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0636b0 tx=0x7fe684009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3472fb137fb8473c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe68affd700 1 -- 172.21.15.33:0/3064718130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400e070 con 0x7fe68c06aad0 2023-10-06T05:21:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe68affd700 1 -- 172.21.15.33:0/3064718130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400b970 con 0x7fe68c06aad0 2023-10-06T05:21:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.196+0000 7fe68affd700 1 -- 172.21.15.33:0/3064718130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840092e0 con 0x7fe68c06aad0 2023-10-06T05:21:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 -- 172.21.15.33:0/3064718130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c06aad0 msgr2=0x7fe68c06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 --2- 172.21.15.33:0/3064718130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c06aad0 0x7fe68c06af50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0636b0 tx=0x7fe684009840 comp rx=0 tx=0).stop 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 -- 172.21.15.33:0/3064718130 shutdown_connections 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 --2- 172.21.15.33:0/3064718130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c06aad0 0x7fe68c06af50 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 --2- 172.21.15.33:0/3064718130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 --2- 172.21.15.33:0/3064718130 >> v2:172.21.15.167:3300/0 conn(0x7fe68c0c7940 0x7fe68c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 -- 172.21.15.33:0/3064718130 >> 172.21.15.33:0/3064718130 conn(0x7fe68c06d030 msgr2=0x7fe68c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 -- 172.21.15.33:0/3064718130 shutdown_connections 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.197+0000 7fe69119d700 1 -- 172.21.15.33:0/3064718130 wait complete. 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.198+0000 7fe69119d700 1 Processor -- start 2023-10-06T05:21:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.198+0000 7fe69119d700 1 -- start start 2023-10-06T05:21:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.199+0000 7fe69119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe68c06aad0 0x7fe68c13c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.199+0000 7fe69119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 0x7fe68c0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.199+0000 7fe69119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c0c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.199+0000 7fe69119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c156c00 con 0x7fe68c0c7940 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.199+0000 7fe69119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1574a0 con 0x7fe68c06aad0 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.199+0000 7fe69119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c13ceb0 con 0x7fe68c13faa0 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 0x7fe68c0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 0x7fe68c0c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54600/0 (socket says 172.21.15.33:54600) 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 -- 172.21.15.33:0/3268422440 learned_addr learned my addr 172.21.15.33:0/3268422440 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68bfff700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe68c06aad0 0x7fe68c13c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe69099c700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c0c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 -- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 msgr2=0x7fe68c0c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c0c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 -- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe68c06aad0 msgr2=0x7fe68c13c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe68c06aad0 0x7fe68c13c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 -- 172.21.15.33:0/3268422440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684009ae0 con 0x7fe68c0c7940 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe68b7fe700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 0x7fe68c0c60f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe68000ecb0 tx=0x7fe6800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.200+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68000c9d0 con 0x7fe68c0c7940 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.201+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000cb30 con 0x7fe68c0c7940 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.201+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c0c6a60 con 0x7fe68c0c7940 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.201+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68000ced0 con 0x7fe68c0c7940 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.201+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c157130 con 0x7fe68c0c7940 2023-10-06T05:21:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.202+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c04c380 con 0x7fe68c0c7940 2023-10-06T05:21:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.203+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe680013680 con 0x7fe68c0c7940 2023-10-06T05:21:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.203+0000 7fe673fff700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe678066f30 0x7fe6780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.204+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fe680084bc0 con 0x7fe68c0c7940 2023-10-06T05:21:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.204+0000 7fe68bfff700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe678066f30 0x7fe6780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.205+0000 7fe68bfff700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe678066f30 0x7fe6780693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009ca0 tx=0x7fe67c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.208+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe680051b60 con 0x7fe68c0c7940 2023-10-06T05:21:22.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.454+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe68c0cd200 con 0x7fe68c0c7940 2023-10-06T05:21:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.455+0000 7fe673fff700 1 -- 172.21.15.33:0/3268422440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fe680055180 con 0x7fe68c0c7940 2023-10-06T05:21:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:22.456 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe678066f30 msgr2=0x7fe6780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe678066f30 0x7fe6780693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009ca0 tx=0x7fe67c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 msgr2=0x7fe68c0c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 0x7fe68c0c60f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe68000ecb0 tx=0x7fe6800091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 shutdown_connections 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe678066f30 0x7fe6780693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe68c13faa0 0x7fe68c0c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe68c06aad0 0x7fe68c13c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 --2- 172.21.15.33:0/3268422440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68c0c7940 0x7fe68c0c60f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 >> 172.21.15.33:0/3268422440 conn(0x7fe68c06d030 msgr2=0x7fe68c0ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 shutdown_connections 2023-10-06T05:21:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:22.457+0000 7fe69119d700 1 -- 172.21.15.33:0/3268422440 wait complete. 2023-10-06T05:21:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:22.476 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:23.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:23.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.675+0000 7f80b1d9d700 1 Processor -- start 2023-10-06T05:21:23.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.675+0000 7f80b1d9d700 1 -- start start 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b1d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80ac159520 0x7f80ac0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b1d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac06c820 con 0x7f80ac0c82a0 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b1d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac06a160 con 0x7f80ac0c6170 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b1d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ac0c6610 con 0x7f80ac159520 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b0d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80ac159520 0x7f80ac0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b0d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80ac159520 0x7f80ac0c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54602/0 (socket says 172.21.15.33:54602) 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b159c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f80ac0c82a0 0x7f80ac06a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54506/0 (socket says 172.21.15.33:54506) 2023-10-06T05:21:23.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.676+0000 7f80b0d9b700 1 -- 172.21.15.33:0/575756784 learned_addr learned my addr 172.21.15.33:0/575756784 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80b0d9b700 1 -- 172.21.15.33:0/575756784 >> v1:172.21.15.167:6789/0 conn(0x7f80ac0c6170 legacy=0x7f80ac0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80b0d9b700 1 -- 172.21.15.33:0/575756784 >> v1:172.21.15.33:6789/0 conn(0x7f80ac0c82a0 legacy=0x7f80ac06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80ab7fe700 1 -- 172.21.15.33:0/575756784 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235833072 0 0) 0x7f80ac06c820 con 0x7f80ac0c82a0 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80b0d9b700 1 -- 172.21.15.33:0/575756784 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0c8800 con 0x7f80ac159520 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80b0d9b700 1 --2- 172.21.15.33:0/575756784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac0c5c30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f80a00099b0 tx=0x7f80a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdda43fa91e3e77d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80ab7fe700 1 -- 172.21.15.33:0/575756784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000b820 con 0x7f80ac159520 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80ab7fe700 1 -- 172.21.15.33:0/575756784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000b980 con 0x7f80ac159520 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.677+0000 7f80ab7fe700 1 -- 172.21.15.33:0/575756784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000bd20 con 0x7f80ac159520 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 -- 172.21.15.33:0/575756784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 msgr2=0x7f80ac0c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/575756784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac0c5c30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f80a00099b0 tx=0x7f80a0009250 comp rx=0 tx=0).stop 2023-10-06T05:21:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 -- 172.21.15.33:0/575756784 shutdown_connections 2023-10-06T05:21:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/575756784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac0c5c30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 -- 172.21.15.33:0/575756784 >> 172.21.15.33:0/575756784 conn(0x7f80ac0cc930 msgr2=0x7f80ac0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 -- 172.21.15.33:0/575756784 shutdown_connections 2023-10-06T05:21:23.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.678+0000 7f80b1d9d700 1 -- 172.21.15.33:0/575756784 wait complete. 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.679+0000 7f80b1d9d700 1 Processor -- start 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 -- start start 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80ac0c6170 0x7f80ac1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80ac0c82a0 0x7f80ac0a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac169960 con 0x7f80ac159520 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac157920 con 0x7f80ac0c6170 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.680+0000 7f80b1d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac15c430 con 0x7f80ac0c82a0 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac15f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54618/0 (socket says 172.21.15.33:54618) 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 -- 172.21.15.33:0/1651598563 learned_addr learned my addr 172.21.15.33:0/1651598563 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b0d9b700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80ac0c6170 0x7f80ac1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80abfff700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80ac0c82a0 0x7f80ac0a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 -- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80ac0c82a0 msgr2=0x7f80ac0a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80ac0c82a0 0x7f80ac0a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 -- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80ac0c6170 msgr2=0x7f80ac1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80ac0c6170 0x7f80ac1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 -- 172.21.15.33:0/1651598563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00094f0 con 0x7f80ac159520 2023-10-06T05:21:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b0d9b700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80ac0c6170 0x7f80ac1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.681+0000 7f80b159c700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac15f350 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f80a400cd50 tx=0x7f80a4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.682+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4005df0 con 0x7f80ac159520 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.682+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a400a960 con 0x7f80ac159520 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.682+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400ad20 con 0x7f80ac159520 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.682+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac15c710 con 0x7f80ac159520 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.682+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac169b90 con 0x7f80ac159520 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.683+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac04c380 con 0x7f80ac159520 2023-10-06T05:21:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.684+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80a4007300 con 0x7f80ac159520 2023-10-06T05:21:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.685+0000 7f80a8ff9700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8090066f30 0x7f80900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.685+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f80a40849d0 con 0x7f80ac159520 2023-10-06T05:21:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.685+0000 7f80b0d9b700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8090066f30 0x7f80900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.686+0000 7f80b0d9b700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8090066f30 0x7f80900693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80a00099b0 tx=0x7f80a0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.689+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80a4051a90 con 0x7f80ac159520 2023-10-06T05:21:23.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.940+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80ac0c8ae0 con 0x7f80ac159520 2023-10-06T05:21:23.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.940+0000 7f80a8ff9700 1 -- 172.21.15.33:0/1651598563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f80a40550b0 con 0x7f80ac159520 2023-10-06T05:21:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:23.942 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8090066f30 msgr2=0x7f80900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8090066f30 0x7f80900693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f80a00099b0 tx=0x7f80a0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 msgr2=0x7f80ac15f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac15f350 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f80a400cd50 tx=0x7f80a4001b10 comp rx=0 tx=0).stop 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 shutdown_connections 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8090066f30 0x7f80900693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac159520 0x7f80ac15f350 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80ac0c6170 0x7f80ac1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 --2- 172.21.15.33:0/1651598563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80ac0c82a0 0x7f80ac0a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 >> 172.21.15.33:0/1651598563 conn(0x7f80ac0cc930 msgr2=0x7f80ac158740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 shutdown_connections 2023-10-06T05:21:23.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:23.943+0000 7f80b1d9d700 1 -- 172.21.15.33:0/1651598563 wait complete. 2023-10-06T05:21:23.946 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:23.963 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:24.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.161+0000 7f68fad9d700 1 Processor -- start 2023-10-06T05:21:25.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.161+0000 7f68fad9d700 1 -- start start 2023-10-06T05:21:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.162+0000 7f68fad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68f40ccfc0 0x7f68f40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.162+0000 7f68fad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f40b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.162+0000 7f68fad9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f40cbd80 con 0x7f68f40cf3b0 2023-10-06T05:21:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.162+0000 7f68fad9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40af5f0 con 0x7f68f40ccfc0 2023-10-06T05:21:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.162+0000 7f68fad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40af8d0 con 0x7f68f40cc750 2023-10-06T05:21:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.162+0000 7f68f9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68f40ccfc0 0x7f68f40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68f40ccfc0 0x7f68f40cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54620/0 (socket says 172.21.15.33:54620) 2023-10-06T05:21:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 -- 172.21.15.33:0/1380674616 learned_addr learned my addr 172.21.15.33:0/1380674616 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68fa59c700 1 --2- 172.21.15.33:0/1380674616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f40b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 -- 172.21.15.33:0/1380674616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 msgr2=0x7f68f40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 --2- 172.21.15.33:0/1380674616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f40b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 -- 172.21.15.33:0/1380674616 >> v1:172.21.15.167:6789/0 conn(0x7f68f40cf3b0 legacy=0x7f68f41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 -- 172.21.15.33:0/1380674616 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40afbb0 con 0x7f68f40ccfc0 2023-10-06T05:21:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.163+0000 7f68f9d9b700 1 --2- 172.21.15.33:0/1380674616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ccfc0 0x7f68f40cc210 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f68f00099e0 tx=0x7f68f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975691208b4bfbac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68f8d99700 1 -- 172.21.15.33:0/1380674616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000b820 con 0x7f68f40ccfc0 2023-10-06T05:21:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68f8d99700 1 -- 172.21.15.33:0/1380674616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f000b980 con 0x7f68f40ccfc0 2023-10-06T05:21:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68f8d99700 1 -- 172.21.15.33:0/1380674616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000bd20 con 0x7f68f40ccfc0 2023-10-06T05:21:25.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 -- 172.21.15.33:0/1380674616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ccfc0 msgr2=0x7f68f40cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 --2- 172.21.15.33:0/1380674616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ccfc0 0x7f68f40cc210 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f68f00099e0 tx=0x7f68f0009250 comp rx=0 tx=0).stop 2023-10-06T05:21:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 -- 172.21.15.33:0/1380674616 shutdown_connections 2023-10-06T05:21:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 --2- 172.21.15.33:0/1380674616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ccfc0 0x7f68f40cc210 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 --2- 172.21.15.33:0/1380674616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f40b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 -- 172.21.15.33:0/1380674616 >> 172.21.15.33:0/1380674616 conn(0x7f68f406cff0 msgr2=0x7f68f406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 -- 172.21.15.33:0/1380674616 shutdown_connections 2023-10-06T05:21:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.164+0000 7f68fad9d700 1 -- 172.21.15.33:0/1380674616 wait complete. 2023-10-06T05:21:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.166+0000 7f68fad9d700 1 Processor -- start 2023-10-06T05:21:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.166+0000 7f68fad9d700 1 -- start start 2023-10-06T05:21:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f414e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f40ccfc0 0x7f68f40c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 0x7f68f414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fad9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f416d670 con 0x7f68f40ca0b0 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fad9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40c8440 con 0x7f68f40ccfc0 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f416e460 con 0x7f68f40cc750 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 0x7f68f414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 0x7f68f414ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54624/0 (socket says 172.21.15.33:54624) 2023-10-06T05:21:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68fa59c700 1 -- 172.21.15.33:0/3345721831 learned_addr learned my addr 172.21.15.33:0/3345721831 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68f959a700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f40ccfc0 0x7f68f40c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.167+0000 7f68f9d9b700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fa59c700 1 -- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 msgr2=0x7f68f414e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fa59c700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f414e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fa59c700 1 -- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f40ccfc0 msgr2=0x7f68f40c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fa59c700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f40ccfc0 0x7f68f40c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fa59c700 1 -- 172.21.15.33:0/3345721831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00094f0 con 0x7f68f40ca0b0 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68f9d9b700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fa59c700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 0x7f68f414ce80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009fa0 tx=0x7f68ec009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00ed20 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00ee80 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f416d850 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec014670 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.168+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f416de10 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.169+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f416e5d0 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.170+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68ec0149e0 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.172+0000 7f68e9ffb700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68d8066f30 0x7f68d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.172+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f68ec084f50 con 0x7f68f40ca0b0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.172+0000 7f68f9d9b700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68d8066f30 0x7f68d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.173+0000 7f68f9d9b700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68d8066f30 0x7f68d80693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68f00099e0 tx=0x7f68f0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.176+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68ec051ef0 con 0x7f68f40ca0b0 2023-10-06T05:21:25.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.422+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68f40b2e20 con 0x7f68f40ca0b0 2023-10-06T05:21:25.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.422+0000 7f68e9ffb700 1 -- 172.21.15.33:0/3345721831 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f68ec055510 con 0x7f68f40ca0b0 2023-10-06T05:21:25.424 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:25.424 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:25.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68d8066f30 msgr2=0x7f68d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68d8066f30 0x7f68d80693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68f00099e0 tx=0x7f68f0017000 comp rx=0 tx=0).stop 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 msgr2=0x7f68f414ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 0x7f68f414ce80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009fa0 tx=0x7f68ec009900 comp rx=0 tx=0).stop 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 shutdown_connections 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68d8066f30 0x7f68d80693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f40ca0b0 0x7f68f414ce80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f40cc750 0x7f68f414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 --2- 172.21.15.33:0/3345721831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f40ccfc0 0x7f68f40c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 >> 172.21.15.33:0/3345721831 conn(0x7f68f406cff0 msgr2=0x7f68f40b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 shutdown_connections 2023-10-06T05:21:25.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:25.425+0000 7f68fad9d700 1 -- 172.21.15.33:0/3345721831 wait complete. 2023-10-06T05:21:25.428 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:25.444 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:26.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:26.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.641+0000 7fbe57fff700 1 Processor -- start 2023-10-06T05:21:26.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.641+0000 7fbe57fff700 1 -- start start 2023-10-06T05:21:26.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe57fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe58155140 0x7fbe58158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580699c0 0x7fbe58069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe57fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580c8e40 con 0x7fbe580adf50 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe57fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5806a380 con 0x7fbe58155140 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe57fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c99e0 con 0x7fbe580699c0 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe567fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbe580adf50 0x7fbe580c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54508/0 (socket says 172.21.15.33:54508) 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe567fc700 1 -- 172.21.15.33:0/3609396443 learned_addr learned my addr 172.21.15.33:0/3609396443 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe56ffd700 1 --2- 172.21.15.33:0/3609396443 >> v2:172.21.15.167:3300/0 conn(0x7fbe58155140 0x7fbe58158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.643+0000 7fbe577fe700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580699c0 0x7fbe58069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe55ffb700 1 -- 172.21.15.33:0/3609396443 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986244235 0 0) 0x7fbe580c8e40 con 0x7fbe580adf50 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe55ffb700 1 -- 172.21.15.33:0/3609396443 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe40003570 con 0x7fbe580adf50 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe56ffd700 1 -- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580699c0 msgr2=0x7fbe58069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe56ffd700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580699c0 0x7fbe58069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe56ffd700 1 -- 172.21.15.33:0/3609396443 >> v1:172.21.15.33:6789/0 conn(0x7fbe580adf50 legacy=0x7fbe580c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe56ffd700 1 -- 172.21.15.33:0/3609396443 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580ae3f0 con 0x7fbe58155140 2023-10-06T05:21:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe577fe700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580699c0 0x7fbe58069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.644+0000 7fbe56ffd700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe58158700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009910 tx=0x7fbe480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11dcb30d1993ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe55ffb700 1 -- 172.21.15.33:0/3609396443 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011630 con 0x7fbe58155140 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe55ffb700 1 -- 172.21.15.33:0/3609396443 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48011790 con 0x7fbe58155140 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe55ffb700 1 -- 172.21.15.33:0/3609396443 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011b30 con 0x7fbe58155140 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 -- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 msgr2=0x7fbe58158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe58158700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009910 tx=0x7fbe480091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 -- 172.21.15.33:0/3609396443 shutdown_connections 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe58158700 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 --2- 172.21.15.33:0/3609396443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580699c0 0x7fbe58069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 -- 172.21.15.33:0/3609396443 >> 172.21.15.33:0/3609396443 conn(0x7fbe5806d030 msgr2=0x7fbe5806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 -- 172.21.15.33:0/3609396443 shutdown_connections 2023-10-06T05:21:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.645+0000 7fbe57fff700 1 -- 172.21.15.33:0/3609396443 wait complete. 2023-10-06T05:21:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.647+0000 7fbe57fff700 1 Processor -- start 2023-10-06T05:21:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.647+0000 7fbe57fff700 1 -- start start 2023-10-06T05:21:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 0x7fbe58170f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580adf50 0x7fbe5816f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe580b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe57fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5815cc60 con 0x7fbe580699c0 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe57fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580b16c0 con 0x7fbe58155140 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe57fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5815c330 con 0x7fbe580adf50 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 0x7fbe58170f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 0x7fbe58170f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54634/0 (socket says 172.21.15.33:54634) 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 -- 172.21.15.33:0/1426245626 learned_addr learned my addr 172.21.15.33:0/1426245626 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe577fe700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe580b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe567fc700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580adf50 0x7fbe5816f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 -- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580adf50 msgr2=0x7fbe5816f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580adf50 0x7fbe5816f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 -- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 msgr2=0x7fbe580b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe580b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.648+0000 7fbe56ffd700 1 -- 172.21.15.33:0/1426245626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50009a20 con 0x7fbe580699c0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.649+0000 7fbe56ffd700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 0x7fbe58170f50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbe58063610 tx=0x7fbe48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.649+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011b10 con 0x7fbe580699c0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.649+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48025400 con 0x7fbe580699c0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.649+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48025780 con 0x7fbe580699c0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.649+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe48009420 con 0x7fbe580699c0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.649+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5815c8f0 con 0x7fbe580699c0 2023-10-06T05:21:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.650+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe580c9820 con 0x7fbe580699c0 2023-10-06T05:21:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.651+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe4802e430 con 0x7fbe580699c0 2023-10-06T05:21:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.653+0000 7fbe3effd700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe44066f60 0x7fbe44069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.653+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fbe48085f50 con 0x7fbe580699c0 2023-10-06T05:21:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.653+0000 7fbe567fc700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe44066f60 0x7fbe44069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.654+0000 7fbe567fc700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe44066f60 0x7fbe44069410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbe5814f3a0 tx=0x7fbe4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.657+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe48052fb0 con 0x7fbe580699c0 2023-10-06T05:21:26.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.906+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe5804c380 con 0x7fbe580699c0 2023-10-06T05:21:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.906+0000 7fbe3effd700 1 -- 172.21.15.33:0/1426245626 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fbe480565d0 con 0x7fbe580699c0 2023-10-06T05:21:26.908 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:26.908 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe44066f60 msgr2=0x7fbe44069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe44066f60 0x7fbe44069410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbe5814f3a0 tx=0x7fbe4c009040 comp rx=0 tx=0).stop 2023-10-06T05:21:26.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 msgr2=0x7fbe58170f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 0x7fbe58170f50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbe58063610 tx=0x7fbe48017000 comp rx=0 tx=0).stop 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 shutdown_connections 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe44066f60 0x7fbe44069410 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58155140 0x7fbe580b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe580699c0 0x7fbe58170f50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 --2- 172.21.15.33:0/1426245626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe580adf50 0x7fbe5816f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.908+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 >> 172.21.15.33:0/1426245626 conn(0x7fbe5806d030 msgr2=0x7fbe5815ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.909+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 shutdown_connections 2023-10-06T05:21:26.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:26.909+0000 7fbe57fff700 1 -- 172.21.15.33:0/1426245626 wait complete. 2023-10-06T05:21:26.912 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:26.928 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:27.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.127+0000 7fb36539d700 1 Processor -- start 2023-10-06T05:21:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.127+0000 7fb36539d700 1 -- start start 2023-10-06T05:21:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb36539d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb36006ab00 0x7fb3600b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb36539d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36015e2d0 con 0x7fb3600b2e30 2023-10-06T05:21:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb36539d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36006afa0 con 0x7fb3600b0a40 2023-10-06T05:21:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb36539d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36015d6c0 con 0x7fb36006ab00 2023-10-06T05:21:28.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb364b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb3600b2e30 0x7fb36015a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54522/0 (socket says 172.21.15.33:54522) 2023-10-06T05:21:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb364b9c700 1 -- 172.21.15.33:0/3398662716 learned_addr learned my addr 172.21.15.33:0/3398662716 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/3398662716 >> v2:172.21.15.167:3300/0 conn(0x7fb36006ab00 0x7fb3600b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474678805 0 0) 0x7fb36015e2d0 con 0x7fb3600b2e30 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.129+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb348003570 con 0x7fb3600b2e30 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328164325 0 0) 0x7fb36006afa0 con 0x7fb3600b0a40 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb36015e2d0 con 0x7fb3600b0a40 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35f7fe700 1 -- 172.21.15.33:0/3398662716 >> v1:172.21.15.167:6789/0 conn(0x7fb3600b0a40 legacy=0x7fb3600b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35f7fe700 1 -- 172.21.15.33:0/3398662716 >> v1:172.21.15.33:6789/0 conn(0x7fb3600b2e30 legacy=0x7fb36015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35f7fe700 1 -- 172.21.15.33:0/3398662716 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36015d9a0 con 0x7fb36006ab00 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/3398662716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36006ab00 0x7fb3600b28f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb3540098e0 tx=0x7fb3540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e05968769afad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3580017f0 con 0x7fb36006ab00 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb354011420 con 0x7fb36006ab00 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.130+0000 7fb35effd700 1 -- 172.21.15.33:0/3398662716 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3540117c0 con 0x7fb36006ab00 2023-10-06T05:21:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 -- 172.21.15.33:0/3398662716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36006ab00 msgr2=0x7fb3600b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 --2- 172.21.15.33:0/3398662716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36006ab00 0x7fb3600b28f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb3540098e0 tx=0x7fb3540091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 -- 172.21.15.33:0/3398662716 shutdown_connections 2023-10-06T05:21:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 --2- 172.21.15.33:0/3398662716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36006ab00 0x7fb3600b28f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 -- 172.21.15.33:0/3398662716 >> 172.21.15.33:0/3398662716 conn(0x7fb36006e0f0 msgr2=0x7fb3600675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 -- 172.21.15.33:0/3398662716 shutdown_connections 2023-10-06T05:21:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.131+0000 7fb36539d700 1 -- 172.21.15.33:0/3398662716 wait complete. 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.133+0000 7fb36539d700 1 Processor -- start 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.133+0000 7fb36539d700 1 -- start start 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb36539d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 0x7fb360154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb36539d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3600b0a40 0x7fb36015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb36539d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3600b2e30 0x7fb3600aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb36539d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36016a130 con 0x7fb36006ab00 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb36539d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36016a320 con 0x7fb3600b0a40 2023-10-06T05:21:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb36539d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36016a600 con 0x7fb3600b2e30 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 0x7fb360154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 0x7fb360154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54644/0 (socket says 172.21.15.33:54644) 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 -- 172.21.15.33:0/3147656030 learned_addr learned my addr 172.21.15.33:0/3147656030 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3600b0a40 0x7fb36015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb364b9c700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3600b2e30 0x7fb3600aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 -- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3600b2e30 msgr2=0x7fb3600aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3600b2e30 0x7fb3600aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 -- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3600b0a40 msgr2=0x7fb36015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3600b0a40 0x7fb36015c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35ffff700 1 -- 172.21.15.33:0/3147656030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb354009420 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3600b0a40 0x7fb36015c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.134+0000 7fb364b9c700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3600b2e30 0x7fb3600aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.135+0000 7fb35ffff700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 0x7fb360154420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c790 tx=0x7fb350007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.135+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350010510 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.135+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36016a8e0 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.135+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350010670 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.135+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350010a10 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.135+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36016aea0 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.136+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36006fcc0 con 0x7fb36006ab00 2023-10-06T05:21:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.137+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb35001b400 con 0x7fb36006ab00 2023-10-06T05:21:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.138+0000 7fb347fff700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb34c066f30 0x7fb34c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.138+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb350084650 con 0x7fb36006ab00 2023-10-06T05:21:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.139+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb34c066f30 0x7fb34c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.140+0000 7fb35f7fe700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb34c066f30 0x7fb34c0693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3600cb3b0 tx=0x7fb354017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.143+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3500515f0 con 0x7fb36006ab00 2023-10-06T05:21:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.388+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb36016b0c0 con 0x7fb36006ab00 2023-10-06T05:21:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.389+0000 7fb347fff700 1 -- 172.21.15.33:0/3147656030 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fb350054c10 con 0x7fb36006ab00 2023-10-06T05:21:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:28.391 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.391+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb34c066f30 msgr2=0x7fb34c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.391+0000 7fb36539d700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb34c066f30 0x7fb34c0693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3600cb3b0 tx=0x7fb354017000 comp rx=0 tx=0).stop 2023-10-06T05:21:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.391+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 msgr2=0x7fb360154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.391+0000 7fb36539d700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 0x7fb360154420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c790 tx=0x7fb350007e10 comp rx=0 tx=0).stop 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.391+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 shutdown_connections 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb34c066f30 0x7fb34c0693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3600b2e30 0x7fb3600aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb36006ab00 0x7fb360154420 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 --2- 172.21.15.33:0/3147656030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3600b0a40 0x7fb36015c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 >> 172.21.15.33:0/3147656030 conn(0x7fb36006e0f0 msgr2=0x7fb3600675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 shutdown_connections 2023-10-06T05:21:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:28.392+0000 7fb36539d700 1 -- 172.21.15.33:0/3147656030 wait complete. 2023-10-06T05:21:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:28.411 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:29.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:30.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.290+0000 7f9d4bfff700 1 Processor -- start 2023-10-06T05:21:30.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.290+0000 7f9d4bfff700 1 -- start start 2023-10-06T05:21:30.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 0x7f9d4c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9d4c0adf50 0x7f9d4c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c0c8e40 con 0x7f9d4c0c9810 2023-10-06T05:21:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c0ae6c0 con 0x7f9d4c0adf50 2023-10-06T05:21:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0ae9a0 con 0x7f9d4c155140 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4affd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 0x7f9d4c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4affd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 0x7f9d4c158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:36258/0 (socket says 172.21.15.33:36258) 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4a7fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9d4c0adf50 0x7f9d4c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4affd700 1 -- 172.21.15.33:0/2387691414 learned_addr learned my addr 172.21.15.33:0/2387691414 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.292+0000 7f9d4b7fe700 1 --1- 172.21.15.33:0/2387691414 >> v1:172.21.15.167:6789/0 conn(0x7f9d4c0c9810 0x7f9d4c0699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47596/0 (socket says 172.21.15.33:47596) 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d4affd700 1 -- 172.21.15.33:0/2387691414 >> v2:172.21.15.167:3300/0 conn(0x7f9d4c0adf50 msgr2=0x7f9d4c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d4affd700 1 --2- 172.21.15.33:0/2387691414 >> v2:172.21.15.167:3300/0 conn(0x7f9d4c0adf50 0x7f9d4c0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d4affd700 1 -- 172.21.15.33:0/2387691414 >> v1:172.21.15.167:6789/0 conn(0x7f9d4c0c9810 legacy=0x7f9d4c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d4affd700 1 -- 172.21.15.33:0/2387691414 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c0c8ad0 con 0x7f9d4c155140 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d4a7fc700 1 --2- 172.21.15.33:0/2387691414 >> v2:172.21.15.167:3300/0 conn(0x7f9d4c0adf50 0x7f9d4c0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d4affd700 1 --2- 172.21.15.33:0/2387691414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 0x7f9d4c158700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0099b0 tx=0x7f9d3c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68636efc0e8abc2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d49ffb700 1 -- 172.21.15.33:0/2387691414 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c009ea0 con 0x7f9d4c155140 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d49ffb700 1 -- 172.21.15.33:0/2387691414 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00b860 con 0x7f9d4c155140 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.293+0000 7f9d49ffb700 1 -- 172.21.15.33:0/2387691414 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00bc00 con 0x7f9d4c155140 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 -- 172.21.15.33:0/2387691414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 msgr2=0x7f9d4c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/2387691414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 0x7f9d4c158700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0099b0 tx=0x7f9d3c009250 comp rx=0 tx=0).stop 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 -- 172.21.15.33:0/2387691414 shutdown_connections 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/2387691414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c155140 0x7f9d4c158700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/2387691414 >> v2:172.21.15.167:3300/0 conn(0x7f9d4c0adf50 0x7f9d4c0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 -- 172.21.15.33:0/2387691414 >> 172.21.15.33:0/2387691414 conn(0x7f9d4c06d030 msgr2=0x7f9d4c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 -- 172.21.15.33:0/2387691414 shutdown_connections 2023-10-06T05:21:30.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.294+0000 7f9d4bfff700 1 -- 172.21.15.33:0/2387691414 wait complete. 2023-10-06T05:21:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.296+0000 7f9d4bfff700 1 Processor -- start 2023-10-06T05:21:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.296+0000 7f9d4bfff700 1 -- start start 2023-10-06T05:21:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 0x7f9d4c14de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c0c9810 0x7f9d4c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d4c155140 0x7f9d4c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c153810 con 0x7f9d4c0adf50 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0c35a0 con 0x7f9d4c155140 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c153250 con 0x7f9d4c0c9810 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 0x7f9d4c14de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 0x7f9d4c14de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60378/0 (socket says 172.21.15.33:60378) 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 -- 172.21.15.33:0/1431456404 learned_addr learned my addr 172.21.15.33:0/1431456404 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4a7fc700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c0c9810 0x7f9d4c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4b7fe700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d4c155140 0x7f9d4c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 -- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c0c9810 msgr2=0x7f9d4c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c0c9810 0x7f9d4c152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 -- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d4c155140 msgr2=0x7f9d4c0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d4c155140 0x7f9d4c0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.297+0000 7f9d4affd700 1 -- 172.21.15.33:0/1431456404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40009420 con 0x7f9d4c0adf50 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.298+0000 7f9d4affd700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 0x7f9d4c14de30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c006fd0 tx=0x7f9d3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.298+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c000d10 con 0x7f9d4c0adf50 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.298+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00bbe0 con 0x7f9d4c0adf50 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.298+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c0094f0 con 0x7f9d4c0adf50 2023-10-06T05:21:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.298+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c01a6a0 con 0x7f9d4c0adf50 2023-10-06T05:21:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.298+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c0c3230 con 0x7f9d4c0adf50 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.299+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c04c380 con 0x7f9d4c0adf50 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.300+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d3c01aa50 con 0x7f9d4c0adf50 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.301+0000 7f9d32ffd700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9d38066f30 0x7f9d380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.301+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f9d3c0860f0 con 0x7f9d4c0adf50 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.301+0000 7f9d4a7fc700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9d38066f30 0x7f9d380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.302+0000 7f9d4a7fc700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9d38066f30 0x7f9d380693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0aaea0 tx=0x7f9d400091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.305+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d3c053090 con 0x7f9d4c0adf50 2023-10-06T05:21:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.552+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d4c06fa40 con 0x7f9d4c0adf50 2023-10-06T05:21:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.552+0000 7f9d32ffd700 1 -- 172.21.15.33:0/1431456404 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f9d3c0566b0 con 0x7f9d4c0adf50 2023-10-06T05:21:30.554 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:30.554 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9d38066f30 msgr2=0x7f9d380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9d38066f30 0x7f9d380693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0aaea0 tx=0x7f9d400091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 msgr2=0x7f9d4c14de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 0x7f9d4c14de30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c006fd0 tx=0x7f9d3c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 shutdown_connections 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9d38066f30 0x7f9d380693e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d4c155140 0x7f9d4c0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c0adf50 0x7f9d4c14de30 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 --2- 172.21.15.33:0/1431456404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d4c0c9810 0x7f9d4c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 >> 172.21.15.33:0/1431456404 conn(0x7f9d4c06d030 msgr2=0x7f9d4c14e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 shutdown_connections 2023-10-06T05:21:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:30.555+0000 7f9d4bfff700 1 -- 172.21.15.33:0/1431456404 wait complete. 2023-10-06T05:21:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:30.574 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:31.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:31.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.773+0000 7fd93b19d700 1 Processor -- start 2023-10-06T05:21:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.773+0000 7fd93b19d700 1 -- start start 2023-10-06T05:21:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd934155af0 0x7fd93406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd9340c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93b19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93406ab60 con 0x7fd9340c8b50 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93406b7e0 con 0x7fd934155af0 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93406ae40 con 0x7fd93406a6a0 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd934155af0 0x7fd93406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93a99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd9340c8b50 0x7fd9341598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40288/0 (socket says 172.21.15.33:40288) 2023-10-06T05:21:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.774+0000 7fd93a99c700 1 -- 172.21.15.33:0/2619529558 learned_addr learned my addr 172.21.15.33:0/2619529558 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93999a700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd9340c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93a19b700 1 -- 172.21.15.33:0/2619529558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 msgr2=0x7fd9340c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd939199700 1 -- 172.21.15.33:0/2619529558 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537536478 0 0) 0x7fd93406ab60 con 0x7fd9340c8b50 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93a19b700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd9340c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93a19b700 1 -- 172.21.15.33:0/2619529558 >> v1:172.21.15.33:6789/0 conn(0x7fd9340c8b50 legacy=0x7fd9341598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93a19b700 1 -- 172.21.15.33:0/2619529558 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93406b120 con 0x7fd934155af0 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93999a700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd9340c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd93a19b700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd934155af0 0x7fd93406a160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd93000aa90 tx=0x7fd93000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baae5a652eae340 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd939199700 1 -- 172.21.15.33:0/2619529558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000ca40 con 0x7fd934155af0 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.775+0000 7fd939199700 1 -- 172.21.15.33:0/2619529558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93000cba0 con 0x7fd934155af0 2023-10-06T05:21:31.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd939199700 1 -- 172.21.15.33:0/2619529558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930018400 con 0x7fd934155af0 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 -- 172.21.15.33:0/2619529558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd934155af0 msgr2=0x7fd93406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd934155af0 0x7fd93406a160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd93000aa90 tx=0x7fd93000a330 comp rx=0 tx=0).stop 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 -- 172.21.15.33:0/2619529558 shutdown_connections 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd934155af0 0x7fd93406a160 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 --2- 172.21.15.33:0/2619529558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd9340c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 -- 172.21.15.33:0/2619529558 >> 172.21.15.33:0/2619529558 conn(0x7fd93406d030 msgr2=0x7fd93406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 -- 172.21.15.33:0/2619529558 shutdown_connections 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.776+0000 7fd93b19d700 1 -- 172.21.15.33:0/2619529558 wait complete. 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.778+0000 7fd93b19d700 1 Processor -- start 2023-10-06T05:21:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.778+0000 7fd93b19d700 1 -- start start 2023-10-06T05:21:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd934150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 0x7fd9340c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd934155af0 0x7fd9340b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd934150b90 con 0x7fd9340c8b50 2023-10-06T05:21:31.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340c3eb0 con 0x7fd934155af0 2023-10-06T05:21:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340c32a0 con 0x7fd93406a6a0 2023-10-06T05:21:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 0x7fd9340c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 0x7fd9340c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60402/0 (socket says 172.21.15.33:60402) 2023-10-06T05:21:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 -- 172.21.15.33:0/365210140 learned_addr learned my addr 172.21.15.33:0/365210140 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93a19b700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd934150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93a99c700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd934155af0 0x7fd9340b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 -- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 msgr2=0x7fd934150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd934150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 -- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd934155af0 msgr2=0x7fd9340b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd934155af0 0x7fd9340b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.779+0000 7fd93999a700 1 -- 172.21.15.33:0/365210140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93000a5d0 con 0x7fd9340c8b50 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.780+0000 7fd93999a700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 0x7fd9340c53c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c880 tx=0x7fd924008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.780+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924011600 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.780+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924011760 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.780+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924011ae0 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.780+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9340c3580 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.780+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9340c3b40 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.781+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93404c380 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.782+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9240073d0 con 0x7fd9340c8b50 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.783+0000 7fd92a7fc700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd918066f30 0x7fd9180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.783+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fd924084900 con 0x7fd9340c8b50 2023-10-06T05:21:31.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.784+0000 7fd93a19b700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd918066f30 0x7fd9180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.785+0000 7fd93a19b700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd918066f30 0x7fd9180693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd93000a5a0 tx=0x7fd93001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:31.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:31.788+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9240518a0 con 0x7fd9340c8b50 2023-10-06T05:21:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.038+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd93406b400 con 0x7fd9340c8b50 2023-10-06T05:21:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.038+0000 7fd92a7fc700 1 -- 172.21.15.33:0/365210140 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd924054ec0 con 0x7fd9340c8b50 2023-10-06T05:21:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.040+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd918066f30 msgr2=0x7fd9180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd918066f30 0x7fd9180693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd93000a5a0 tx=0x7fd93001a000 comp rx=0 tx=0).stop 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 msgr2=0x7fd9340c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 0x7fd9340c53c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c880 tx=0x7fd924008040 comp rx=0 tx=0).stop 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 shutdown_connections 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd918066f30 0x7fd9180693e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd934155af0 0x7fd9340b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd93406a6a0 0x7fd934150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 --2- 172.21.15.33:0/365210140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9340c8b50 0x7fd9340c53c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 >> 172.21.15.33:0/365210140 conn(0x7fd93406d030 msgr2=0x7fd934154f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 shutdown_connections 2023-10-06T05:21:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:32.041+0000 7fd93b19d700 1 -- 172.21.15.33:0/365210140 wait complete. 2023-10-06T05:21:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:32.060 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:33.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.257+0000 7f6df979d700 1 Processor -- start 2023-10-06T05:21:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.257+0000 7f6df979d700 1 -- start start 2023-10-06T05:21:33.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 0x7f6df413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df979d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6df406aad0 0x7f6df406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df979d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df40c9cb0 con 0x7f6df413faa0 2023-10-06T05:21:33.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df979d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40ca270 con 0x7f6df406aad0 2023-10-06T05:21:33.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df406b840 con 0x7f6df40c8050 2023-10-06T05:21:33.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6df413faa0 0x7f6df40c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40292/0 (socket says 172.21.15.33:40292) 2023-10-06T05:21:33.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df3fff700 1 -- 172.21.15.33:0/1387035196 learned_addr learned my addr 172.21.15.33:0/1387035196 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df37fe700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 0x7f6df413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df8f9c700 1 --2- 172.21.15.33:0/1387035196 >> v2:172.21.15.167:3300/0 conn(0x7f6df406aad0 0x7f6df406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071063485 0 0) 0x7f6df40c9cb0 con 0x7f6df413faa0 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.259+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ddc003570 con 0x7f6df413faa0 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2376077585 0 0) 0x7f6ddc003570 con 0x7f6df413faa0 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 msgr2=0x7f6df413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 0x7f6df413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 >> v2:172.21.15.167:3300/0 conn(0x7f6df406aad0 msgr2=0x7f6df406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 --2- 172.21.15.33:0/1387035196 >> v2:172.21.15.167:3300/0 conn(0x7f6df406aad0 0x7f6df406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df406bb20 con 0x7f6df413faa0 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f6de4001470 con 0x7f6df413faa0 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 >> v1:172.21.15.33:6789/0 conn(0x7f6df413faa0 legacy=0x7f6df40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc003ae0 0x7f6ddc003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc004810 0x7f6ddc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 0x7f6ddc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40ca270 con 0x7f6ddc004810 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df406b840 con 0x7f6ddc003ae0 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc00bb10 con 0x7f6df40c8050 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df37fe700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc004810 0x7f6ddc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df3fff700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc003ae0 0x7f6ddc003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.260+0000 7f6df8f9c700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 0x7f6ddc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df37fe700 1 -- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 msgr2=0x7f6ddc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df37fe700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df40c8050 0x7f6ddc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df37fe700 1 -- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc003ae0 msgr2=0x7f6ddc003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df37fe700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc003ae0 0x7f6ddc003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df37fe700 1 -- 172.21.15.33:0/1387035196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc003570 con 0x7f6ddc004810 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df37fe700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc004810 0x7f6ddc004c80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009c00 tx=0x7f6de80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd99b91c785ce25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800e710 con 0x7f6ddc004810 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800e870 con 0x7f6ddc004810 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.261+0000 7f6df2ffd700 1 -- 172.21.15.33:0/1387035196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800ec30 con 0x7f6ddc004810 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df979d700 1 -- 172.21.15.33:0/1387035196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc004810 msgr2=0x7f6ddc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df979d700 1 --2- 172.21.15.33:0/1387035196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc004810 0x7f6ddc004c80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009c00 tx=0x7f6de80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df8f9c700 1 -- 172.21.15.33:0/1387035196 reap_dead start 2023-10-06T05:21:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df979d700 1 -- 172.21.15.33:0/1387035196 shutdown_connections 2023-10-06T05:21:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df979d700 1 -- 172.21.15.33:0/1387035196 >> 172.21.15.33:0/1387035196 conn(0x7f6df406d030 msgr2=0x7f6df406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df979d700 1 -- 172.21.15.33:0/1387035196 shutdown_connections 2023-10-06T05:21:33.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.262+0000 7f6df979d700 1 -- 172.21.15.33:0/1387035196 wait complete. 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.263+0000 7f6df979d700 1 Processor -- start 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.263+0000 7f6df979d700 1 -- start start 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc004810 0x7f6df413ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df4154890 0x7f6df4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 0x7f6df4128300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df41591f0 con 0x7f6df4153890 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4153db0 con 0x7f6ddc004810 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4154090 con 0x7f6df4154890 2023-10-06T05:21:33.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.264+0000 7f6df8f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 0x7f6df4128300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 0x7f6df4128300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60406/0 (socket says 172.21.15.33:60406) 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 -- 172.21.15.33:0/1365368244 learned_addr learned my addr 172.21.15.33:0/1365368244 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df3fff700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc004810 0x7f6df413ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df37fe700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df4154890 0x7f6df4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 -- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df4154890 msgr2=0x7f6df4153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df4154890 0x7f6df4153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 -- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc004810 msgr2=0x7f6df413ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc004810 0x7f6df413ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 -- 172.21.15.33:0/1365368244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df4154370 con 0x7f6df4153890 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df8f9c700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 0x7f6df4128300 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6df415a070 tx=0x7f6dec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00e040 con 0x7f6df4153890 2023-10-06T05:21:33.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec00f040 con 0x7f6df4153890 2023-10-06T05:21:33.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00fe20 con 0x7f6df4153890 2023-10-06T05:21:33.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.265+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df413f2a0 con 0x7f6df4153890 2023-10-06T05:21:33.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.266+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df4128a00 con 0x7f6df4153890 2023-10-06T05:21:33.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.266+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df404c380 con 0x7f6df4153890 2023-10-06T05:21:33.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.267+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dec015640 con 0x7f6df4153890 2023-10-06T05:21:33.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.268+0000 7f6ddbfff700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.268+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f6dec085100 con 0x7f6df4153890 2023-10-06T05:21:33.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.269+0000 7f6df3fff700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.269+0000 7f6df3fff700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6de4001660 tx=0x7f6de4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.272+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dec0520a0 con 0x7f6df4153890 2023-10-06T05:21:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.520+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df40c9f00 con 0x7f6df4153890 2023-10-06T05:21:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.521+0000 7f6ddbfff700 1 -- 172.21.15.33:0/1365368244 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f6dec0556c0 con 0x7f6df4153890 2023-10-06T05:21:33.523 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:33.523 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 msgr2=0x7f6de00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6de4001660 tx=0x7f6de4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 msgr2=0x7f6df4128300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 0x7f6df4128300 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6df415a070 tx=0x7f6dec00b040 comp rx=0 tx=0).stop 2023-10-06T05:21:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 shutdown_connections 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6df4153890 0x7f6df4128300 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ddc004810 0x7f6df413ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 --2- 172.21.15.33:0/1365368244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6df4154890 0x7f6df4153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.523+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 >> 172.21.15.33:0/1365368244 conn(0x7f6df406d030 msgr2=0x7f6df40c1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.524+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 shutdown_connections 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:33.524+0000 7f6df979d700 1 -- 172.21.15.33:0/1365368244 wait complete. 2023-10-06T05:21:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:33.542 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:34.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.739+0000 7f381cd9d700 1 Processor -- start 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.740+0000 7f381cd9d700 1 -- start start 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f381cd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f381806e140 0x7f381815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f381cd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38180afdd0 0x7f38180b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f381cd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381806f220 con 0x7f3818159530 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f381cd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38180b1e00 con 0x7f38180afdd0 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f381cd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381806f6b0 con 0x7f381806e140 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f3817fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38180afdd0 0x7f38180b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f3817fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38180afdd0 0x7f38180b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60412/0 (socket says 172.21.15.33:60412) 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f38177fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3818159530 0x7f381806da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40296/0 (socket says 172.21.15.33:40296) 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f3817fff700 1 -- 172.21.15.33:0/1228499468 learned_addr learned my addr 172.21.15.33:0/1228499468 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f3817fff700 1 -- 172.21.15.33:0/1228499468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f381806e140 msgr2=0x7f381815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.741+0000 7f3817fff700 1 --2- 172.21.15.33:0/1228499468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f381806e140 0x7f381815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f3817fff700 1 -- 172.21.15.33:0/1228499468 >> v1:172.21.15.33:6789/0 conn(0x7f3818159530 legacy=0x7f381806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f3817fff700 1 -- 172.21.15.33:0/1228499468 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381815ae50 con 0x7f38180afdd0 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f38167fc700 1 -- 172.21.15.33:0/1228499468 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549592898 0 0) 0x7f381806f220 con 0x7f3818159530 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f3817fff700 1 --2- 172.21.15.33:0/1228499468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38180afdd0 0x7f38180b0210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3810009f20 tx=0x7f3810009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3611df72c739c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f38167fc700 1 -- 172.21.15.33:0/1228499468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000e070 con 0x7f38180afdd0 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f38167fc700 1 -- 172.21.15.33:0/1228499468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381000ba80 con 0x7f38180afdd0 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.742+0000 7f38167fc700 1 -- 172.21.15.33:0/1228499468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100092e0 con 0x7f38180afdd0 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 -- 172.21.15.33:0/1228499468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38180afdd0 msgr2=0x7f38180b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1228499468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38180afdd0 0x7f38180b0210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3810009f20 tx=0x7f3810009840 comp rx=0 tx=0).stop 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 -- 172.21.15.33:0/1228499468 shutdown_connections 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1228499468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38180afdd0 0x7f38180b0210 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1228499468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f381806e140 0x7f381815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 -- 172.21.15.33:0/1228499468 >> 172.21.15.33:0/1228499468 conn(0x7f381806c560 msgr2=0x7f381806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 -- 172.21.15.33:0/1228499468 shutdown_connections 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.743+0000 7f381cd9d700 1 -- 172.21.15.33:0/1228499468 wait complete. 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.744+0000 7f381cd9d700 1 Processor -- start 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.744+0000 7f381cd9d700 1 -- start start 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f381cd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381806e140 0x7f3818143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f381cd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38180afdd0 0x7f38180a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f381cd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 0x7f38180a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f381cd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180ca630 con 0x7f3818159530 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f381cd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180ca910 con 0x7f381806e140 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f381cd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180cabf0 con 0x7f38180afdd0 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.745+0000 7f3817fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 0x7f38180a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 0x7f38180a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60420/0 (socket says 172.21.15.33:60420) 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 -- 172.21.15.33:0/1147011846 learned_addr learned my addr 172.21.15.33:0/1147011846 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f38177fe700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381806e140 0x7f3818143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3816ffd700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38180afdd0 0x7f38180a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 -- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38180afdd0 msgr2=0x7f38180a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38180afdd0 0x7f38180a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 -- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381806e140 msgr2=0x7f3818143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381806e140 0x7f3818143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 -- 172.21.15.33:0/1147011846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009ae0 con 0x7f3818159530 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f3817fff700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 0x7f38180a59c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f381000b6c0 tx=0x7f3810018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810026460 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38100265e0 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.746+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180caed0 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.747+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100269a0 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.747+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180c98c0 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.748+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381804c380 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.748+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3810003040 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.749+0000 7f37ff7fe700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3804066f30 0x7f38040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.749+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f38100865d0 con 0x7f3818159530 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.750+0000 7f38177fe700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3804066f30 0x7f38040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.750+0000 7f38177fe700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3804066f30 0x7f38040693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3808007fd0 tx=0x7f3808007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:34.754+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3810053620 con 0x7f3818159530 2023-10-06T05:21:35.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.000+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3818158d60 con 0x7f3818159530 2023-10-06T05:21:35.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.000+0000 7f37ff7fe700 1 -- 172.21.15.33:0/1147011846 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f3810056c40 con 0x7f3818159530 2023-10-06T05:21:35.002 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3804066f30 msgr2=0x7f38040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3804066f30 0x7f38040693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3808007fd0 tx=0x7f3808007e10 comp rx=0 tx=0).stop 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 msgr2=0x7f38180a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 0x7f38180a59c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f381000b6c0 tx=0x7f3810018000 comp rx=0 tx=0).stop 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 shutdown_connections 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3804066f30 0x7f38040693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3818159530 0x7f38180a59c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381806e140 0x7f3818143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 --2- 172.21.15.33:0/1147011846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38180afdd0 0x7f38180a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 >> 172.21.15.33:0/1147011846 conn(0x7f381806c560 msgr2=0x7f3818159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 shutdown_connections 2023-10-06T05:21:35.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:35.003+0000 7f381cd9d700 1 -- 172.21.15.33:0/1147011846 wait complete. 2023-10-06T05:21:35.006 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:35.022 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:36.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.222+0000 7fc46359e700 1 Processor -- start 2023-10-06T05:21:36.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.222+0000 7fc46359e700 1 -- start start 2023-10-06T05:21:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc46359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc464155af0 0x7fc46406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc46359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc46406a6a0 0x7fc4640c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc46359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46406ab60 con 0x7fc4640c8b50 2023-10-06T05:21:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc46359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46406b7e0 con 0x7fc46406a6a0 2023-10-06T05:21:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc46359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46406ae40 con 0x7fc464155af0 2023-10-06T05:21:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc461d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc46406a6a0 0x7fc4640c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.223+0000 7fc462d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc4640c8b50 0x7fc4641598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40302/0 (socket says 172.21.15.33:40302) 2023-10-06T05:21:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc462d9d700 1 -- 172.21.15.33:0/1802418366 learned_addr learned my addr 172.21.15.33:0/1802418366 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc46259c700 1 --2- 172.21.15.33:0/1802418366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc464155af0 0x7fc46406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc46159a700 1 -- 172.21.15.33:0/1802418366 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273914644 0 0) 0x7fc46406ab60 con 0x7fc4640c8b50 2023-10-06T05:21:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc461d9b700 1 -- 172.21.15.33:0/1802418366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc464155af0 msgr2=0x7fc46406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc461d9b700 1 --2- 172.21.15.33:0/1802418366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc464155af0 0x7fc46406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc461d9b700 1 -- 172.21.15.33:0/1802418366 >> v1:172.21.15.33:6789/0 conn(0x7fc4640c8b50 legacy=0x7fc4641598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc461d9b700 1 -- 172.21.15.33:0/1802418366 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46406b120 con 0x7fc46406a6a0 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.224+0000 7fc461d9b700 1 --2- 172.21.15.33:0/1802418366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46406a6a0 0x7fc4640c7210 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc45800aa90 tx=0x7fc45800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368dbb8ecbf31f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46159a700 1 -- 172.21.15.33:0/1802418366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800c970 con 0x7fc46406a6a0 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46159a700 1 -- 172.21.15.33:0/1802418366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45800cad0 con 0x7fc46406a6a0 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46159a700 1 -- 172.21.15.33:0/1802418366 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800ce70 con 0x7fc46406a6a0 2023-10-06T05:21:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 -- 172.21.15.33:0/1802418366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46406a6a0 msgr2=0x7fc4640c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 --2- 172.21.15.33:0/1802418366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46406a6a0 0x7fc4640c7210 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc45800aa90 tx=0x7fc45800a330 comp rx=0 tx=0).stop 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 -- 172.21.15.33:0/1802418366 shutdown_connections 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 --2- 172.21.15.33:0/1802418366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc464155af0 0x7fc46406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 --2- 172.21.15.33:0/1802418366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc46406a6a0 0x7fc4640c7210 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 -- 172.21.15.33:0/1802418366 >> 172.21.15.33:0/1802418366 conn(0x7fc46406d030 msgr2=0x7fc46406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 -- 172.21.15.33:0/1802418366 shutdown_connections 2023-10-06T05:21:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.225+0000 7fc46359e700 1 -- 172.21.15.33:0/1802418366 wait complete. 2023-10-06T05:21:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.227+0000 7fc46359e700 1 Processor -- start 2023-10-06T05:21:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.227+0000 7fc46359e700 1 -- start start 2023-10-06T05:21:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46406a6a0 0x7fc4640abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 0x7fc4640aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc464155af0 0x7fc4640c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640aa7e0 con 0x7fc4640c8b50 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464142aa0 con 0x7fc464155af0 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640aaa10 con 0x7fc46406a6a0 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc461d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 0x7fc4640aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc461d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 0x7fc4640aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60440/0 (socket says 172.21.15.33:60440) 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc461d9b700 1 -- 172.21.15.33:0/3394617494 learned_addr learned my addr 172.21.15.33:0/3394617494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc46259c700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46406a6a0 0x7fc4640abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.228+0000 7fc462d9d700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc464155af0 0x7fc4640c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc461d9b700 1 -- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46406a6a0 msgr2=0x7fc4640abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc461d9b700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46406a6a0 0x7fc4640abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc461d9b700 1 -- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc464155af0 msgr2=0x7fc4640c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc461d9b700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc464155af0 0x7fc4640c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc461d9b700 1 -- 172.21.15.33:0/3394617494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45800a5d0 con 0x7fc4640c8b50 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc462d9d700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc464155af0 0x7fc4640c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc461d9b700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 0x7fc4640aa2a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc45800a5a0 tx=0x7fc45801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458019e90 con 0x7fc4640c8b50 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc458010600 con 0x7fc4640c8b50 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580109c0 con 0x7fc4640c8b50 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4640aacf0 con 0x7fc4640c8b50 2023-10-06T05:21:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.229+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4640ab2b0 con 0x7fc4640c8b50 2023-10-06T05:21:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.230+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc464142d50 con 0x7fc4640c8b50 2023-10-06T05:21:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.231+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc458010d30 con 0x7fc4640c8b50 2023-10-06T05:21:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.233+0000 7fc4527fc700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc444066f30 0x7fc4440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.233+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fc458085ed0 con 0x7fc4640c8b50 2023-10-06T05:21:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.234+0000 7fc46259c700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc444066f30 0x7fc4440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.234+0000 7fc46259c700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc444066f30 0x7fc4440693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc454009ca0 tx=0x7fc454008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.237+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc458052e70 con 0x7fc4640c8b50 2023-10-06T05:21:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.482+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc46404c380 con 0x7fc4640c8b50 2023-10-06T05:21:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.483+0000 7fc4527fc700 1 -- 172.21.15.33:0/3394617494 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fc458056490 con 0x7fc4640c8b50 2023-10-06T05:21:36.484 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:36.485 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc444066f30 msgr2=0x7fc4440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc444066f30 0x7fc4440693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc454009ca0 tx=0x7fc454008040 comp rx=0 tx=0).stop 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 msgr2=0x7fc4640aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 0x7fc4640aa2a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc45800a5a0 tx=0x7fc45801a000 comp rx=0 tx=0).stop 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 shutdown_connections 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc444066f30 0x7fc4440693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc464155af0 0x7fc4640c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc46406a6a0 0x7fc4640abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 --2- 172.21.15.33:0/3394617494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4640c8b50 0x7fc4640aa2a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 >> 172.21.15.33:0/3394617494 conn(0x7fc46406d030 msgr2=0x7fc46406e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 shutdown_connections 2023-10-06T05:21:36.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:36.485+0000 7fc46359e700 1 -- 172.21.15.33:0/3394617494 wait complete. 2023-10-06T05:21:36.488 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:36.505 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:37.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.703+0000 7fd3cbfff700 1 Processor -- start 2023-10-06T05:21:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.703+0000 7fd3cbfff700 1 -- start start 2023-10-06T05:21:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.704+0000 7fd3cbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd3cc0cb710 0x7fd3cc06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.704+0000 7fd3cbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0c9cb0 con 0x7fd3cc0c8050 2023-10-06T05:21:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.704+0000 7fd3cbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0ca270 con 0x7fd3cc13faa0 2023-10-06T05:21:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.704+0000 7fd3cbfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc06b840 con 0x7fd3cc0cb710 2023-10-06T05:21:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3ca7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd3cc0c8050 0x7fd3cc06aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40308/0 (socket says 172.21.15.33:40308) 2023-10-06T05:21:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3ca7fc700 1 -- 172.21.15.33:0/971200118 learned_addr learned my addr 172.21.15.33:0/971200118 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:37.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3cb7fe700 1 --2- 172.21.15.33:0/971200118 >> v2:172.21.15.167:3300/0 conn(0x7fd3cc0cb710 0x7fd3cc06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816836497 0 0) 0x7fd3cc0c9cb0 con 0x7fd3cc0c8050 2023-10-06T05:21:37.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b4003570 con 0x7fd3cc0c8050 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139746217 0 0) 0x7fd3cc0ca270 con 0x7fd3cc13faa0 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.705+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3cc0c9cb0 con 0x7fd3cc13faa0 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530939633 0 0) 0x7fd3b4003570 con 0x7fd3cc0c8050 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 >> v2:172.21.15.167:3300/0 conn(0x7fd3cc0cb710 msgr2=0x7fd3cc06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 --2- 172.21.15.33:0/971200118 >> v2:172.21.15.167:3300/0 conn(0x7fd3cc0cb710 0x7fd3cc06b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 >> v1:172.21.15.167:6789/0 conn(0x7fd3cc13faa0 legacy=0x7fd3cc0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc06bb20 con 0x7fd3cc0c8050 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fd3c0001470 con 0x7fd3cc0c8050 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 --2- 172.21.15.33:0/971200118 >> v2:172.21.15.167:3300/0 conn(0x7fd3cc0cb710 0x7fd3cc06b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 >> v1:172.21.15.33:6789/0 conn(0x7fd3cc0c8050 legacy=0x7fd3cc06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b4003bf0 0x7fd3b4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b4004810 0x7fd3b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc06b840 con 0x7fd3b40051c0 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b400c0b0 con 0x7fd3b4003bf0 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b400e480 con 0x7fd3b4004810 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3caffd700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b4003bf0 0x7fd3b4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3ca7fc700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b4004810 0x7fd3b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 -- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b4004810 msgr2=0x7fd3b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3b4004810 0x7fd3b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 -- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b4003bf0 msgr2=0x7fd3b4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3b4003bf0 0x7fd3b4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.706+0000 7fd3cb7fe700 1 -- 172.21.15.33:0/971200118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b4003570 con 0x7fd3b40051c0 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3cb7fe700 1 -- 172.21.15.33:0/971200118 reap_dead start 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3cb7fe700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3b400ba60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400e3c0 tx=0x7fd3c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4709228735505f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4009c50 con 0x7fd3b40051c0 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c400f040 con 0x7fd3b40051c0 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3c9ffb700 1 -- 172.21.15.33:0/971200118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c400eec0 con 0x7fd3b40051c0 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3cbfff700 1 -- 172.21.15.33:0/971200118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 msgr2=0x7fd3b400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.707+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3b400ba60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400e3c0 tx=0x7fd3c400b040 comp rx=0 tx=0).stop 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.708+0000 7fd3cbfff700 1 -- 172.21.15.33:0/971200118 shutdown_connections 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.708+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/971200118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3b400ba60 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.708+0000 7fd3cbfff700 1 -- 172.21.15.33:0/971200118 >> 172.21.15.33:0/971200118 conn(0x7fd3cc06d030 msgr2=0x7fd3cc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.708+0000 7fd3cbfff700 1 -- 172.21.15.33:0/971200118 shutdown_connections 2023-10-06T05:21:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.708+0000 7fd3cbfff700 1 -- 172.21.15.33:0/971200118 wait complete. 2023-10-06T05:21:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.709+0000 7fd3cbfff700 1 Processor -- start 2023-10-06T05:21:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.709+0000 7fd3cbfff700 1 -- start start 2023-10-06T05:21:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3cc0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3cc0c36d0 0x7fd3cc156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157140 0x7fd3cc1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3cbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1561f0 con 0x7fd3b40051c0 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3cbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0c4860 con 0x7fd3cc157140 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3cbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0c3c50 con 0x7fd3cc0c36d0 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3cc0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3cc0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60462/0 (socket says 172.21.15.33:60462) 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.710+0000 7fd3caffd700 1 -- 172.21.15.33:0/1215548010 learned_addr learned my addr 172.21.15.33:0/1215548010 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3caffd700 1 -- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3cc0c36d0 msgr2=0x7fd3cc156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3caffd700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3cc0c36d0 0x7fd3cc156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3caffd700 1 -- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157140 msgr2=0x7fd3cc1575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3caffd700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157140 0x7fd3cc1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3caffd700 1 -- 172.21.15.33:0/1215548010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0c3f30 con 0x7fd3b40051c0 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3caffd700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3cc0c3190 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc002c00 tx=0x7fd3bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00fa50 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc00fbd0 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc006820 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc0c4210 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.711+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc13e0f0 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.712+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc0c46b0 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.713+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3bc006b90 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.714+0000 7fd3b2ffd700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3b8066f30 0x7fd3b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.714+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fd3bc084ca0 con 0x7fd3b40051c0 2023-10-06T05:21:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.715+0000 7fd3ca7fc700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3b8066f30 0x7fd3b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.715+0000 7fd3ca7fc700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3b8066f30 0x7fd3b80693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0001ed0 tx=0x7fd3cc0ca1e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.718+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3bc051c40 con 0x7fd3b40051c0 2023-10-06T05:21:37.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.966+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3cc04c380 con 0x7fd3b40051c0 2023-10-06T05:21:37.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.966+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1215548010 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd3bc055260 con 0x7fd3b40051c0 2023-10-06T05:21:37.968 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:37.968 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:37.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3b8066f30 msgr2=0x7fd3b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3b8066f30 0x7fd3b80693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0001ed0 tx=0x7fd3cc0ca1e0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 msgr2=0x7fd3cc0c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3cc0c3190 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc002c00 tx=0x7fd3bc007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 shutdown_connections 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd3b8066f30 0x7fd3b80693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3cc157140 0x7fd3cc1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40051c0 0x7fd3cc0c3190 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 --2- 172.21.15.33:0/1215548010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3cc0c36d0 0x7fd3cc156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 >> 172.21.15.33:0/1215548010 conn(0x7fd3cc06d030 msgr2=0x7fd3cc13e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 shutdown_connections 2023-10-06T05:21:37.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:37.969+0000 7fd3cbfff700 1 -- 172.21.15.33:0/1215548010 wait complete. 2023-10-06T05:21:37.972 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:37.989 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:38.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.187+0000 7fea9a19d700 1 Processor -- start 2023-10-06T05:21:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.187+0000 7fea9a19d700 1 -- start start 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.188+0000 7fea9a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 0x7fea9406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.188+0000 7fea9a19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940c9cb0 con 0x7fea940c8050 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.188+0000 7fea9a19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940ca270 con 0x7fea9413faa0 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.188+0000 7fea9a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9406b840 con 0x7fea940cb710 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.188+0000 7fea9899a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fea940c8050 0x7fea9406aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40316/0 (socket says 172.21.15.33:40316) 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.188+0000 7fea9899a700 1 -- 172.21.15.33:0/1795154795 learned_addr learned my addr 172.21.15.33:0/1795154795 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea9999c700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 0x7fea9406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220388319 0 0) 0x7fea940c9cb0 con 0x7fea940c8050 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea80003570 con 0x7fea940c8050 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627255818 0 0) 0x7fea940ca270 con 0x7fea9413faa0 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea940c9cb0 con 0x7fea9413faa0 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465884592 0 0) 0x7fea80003570 con 0x7fea940c8050 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 msgr2=0x7fea9406b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 0x7fea9406b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 >> v1:172.21.15.167:6789/0 conn(0x7fea9413faa0 legacy=0x7fea940c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9406bb20 con 0x7fea940c8050 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fea84001470 con 0x7fea940c8050 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 >> v1:172.21.15.33:6789/0 conn(0x7fea940c8050 legacy=0x7fea9406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea80003c20 0x7fea80004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 0x7fea80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea80004d50 0x7fea800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9406b840 con 0x7fea80004d50 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8000de00 con 0x7fea80003c20 2023-10-06T05:21:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.189+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8000df70 con 0x7fea940cb710 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea80004d50 0x7fea800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9919b700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea80003c20 0x7fea80004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9899a700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 0x7fea80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 -- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 msgr2=0x7fea80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea940cb710 0x7fea80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 -- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea80003c20 msgr2=0x7fea80004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea80003c20 0x7fea80004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 -- 172.21.15.33:0/1795154795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80003570 con 0x7fea80004d50 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.190+0000 7fea9999c700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea80004d50 0x7fea800051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0094d0 tx=0x7fea8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab78144f031d08e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00ea90 con 0x7fea80004d50 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c00f040 con 0x7fea80004d50 2023-10-06T05:21:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea8bfff700 1 -- 172.21.15.33:0/1795154795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00ee20 con 0x7fea80004d50 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9a19d700 1 -- 172.21.15.33:0/1795154795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea80004d50 msgr2=0x7fea800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9a19d700 1 --2- 172.21.15.33:0/1795154795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea80004d50 0x7fea800051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0094d0 tx=0x7fea8c00b040 comp rx=0 tx=0).stop 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9999c700 1 -- 172.21.15.33:0/1795154795 reap_dead start 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9a19d700 1 -- 172.21.15.33:0/1795154795 shutdown_connections 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9a19d700 1 -- 172.21.15.33:0/1795154795 >> 172.21.15.33:0/1795154795 conn(0x7fea9406d030 msgr2=0x7fea9406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9a19d700 1 -- 172.21.15.33:0/1795154795 shutdown_connections 2023-10-06T05:21:39.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.191+0000 7fea9a19d700 1 -- 172.21.15.33:0/1795154795 wait complete. 2023-10-06T05:21:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.193+0000 7fea9a19d700 1 Processor -- start 2023-10-06T05:21:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.193+0000 7fea9a19d700 1 -- start start 2023-10-06T05:21:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea80004d50 0x7fea940b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 0x7fea94166e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea94158bc0 0x7fea94159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea941668b0 con 0x7fea94166a20 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea94158730 con 0x7fea94158bc0 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea94157e40 con 0x7fea80004d50 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 0x7fea94166e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 0x7fea94166e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60482/0 (socket says 172.21.15.33:60482) 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 -- 172.21.15.33:0/4148624868 learned_addr learned my addr 172.21.15.33:0/4148624868 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9919b700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea80004d50 0x7fea940b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9999c700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea94158bc0 0x7fea94159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 -- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea80004d50 msgr2=0x7fea940b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea80004d50 0x7fea940b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 -- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea94158bc0 msgr2=0x7fea94159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea94158bc0 0x7fea94159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.194+0000 7fea9899a700 1 -- 172.21.15.33:0/4148624868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea941665d0 con 0x7fea94166a20 2023-10-06T05:21:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.195+0000 7fea9899a700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 0x7fea94166e90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fea840070e0 tx=0x7fea9406be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.195+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011590 con 0x7fea94166a20 2023-10-06T05:21:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.195+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea840116f0 con 0x7fea94166a20 2023-10-06T05:21:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.195+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011ab0 con 0x7fea94166a20 2023-10-06T05:21:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.195+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea941580e0 con 0x7fea94166a20 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.195+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea940c1420 con 0x7fea94166a20 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.196+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea940cbde0 con 0x7fea94166a20 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.197+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea8401d460 con 0x7fea94166a20 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.198+0000 7fea897fa700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea78066f30 0x7fea780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.198+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fea84084850 con 0x7fea94166a20 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.198+0000 7fea9919b700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea78066f30 0x7fea780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.199+0000 7fea9919b700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea78066f30 0x7fea780693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fea9413e820 tx=0x7fea90007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.202+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea84051890 con 0x7fea94166a20 2023-10-06T05:21:39.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.448+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea9404c380 con 0x7fea94166a20 2023-10-06T05:21:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.448+0000 7fea897fa700 1 -- 172.21.15.33:0/4148624868 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fea84054eb0 con 0x7fea94166a20 2023-10-06T05:21:39.450 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:39.450 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea78066f30 msgr2=0x7fea780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea78066f30 0x7fea780693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fea9413e820 tx=0x7fea90007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 msgr2=0x7fea94166e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 0x7fea94166e90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fea840070e0 tx=0x7fea9406be00 comp rx=0 tx=0).stop 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 shutdown_connections 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea78066f30 0x7fea780693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea94158bc0 0x7fea94159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea80004d50 0x7fea940b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 --2- 172.21.15.33:0/4148624868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea94166a20 0x7fea94166e90 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 >> 172.21.15.33:0/4148624868 conn(0x7fea9406d030 msgr2=0x7fea940c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 shutdown_connections 2023-10-06T05:21:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:39.451+0000 7fea9a19d700 1 -- 172.21.15.33:0/4148624868 wait complete. 2023-10-06T05:21:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:39.470 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:40.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:40.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.668+0000 7ff95edff700 1 Processor -- start 2023-10-06T05:21:40.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.668+0000 7ff95edff700 1 -- start start 2023-10-06T05:21:40.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95edff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff958159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95edff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff95815a290 0x7ff958067ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95edff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95815ae80 con 0x7ff95806dc50 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95edff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95806e020 con 0x7ff95815a290 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95edff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95815a6d0 con 0x7ff958070520 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95d39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff95815a290 0x7ff958067ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.669+0000 7ff95cb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff95806dc50 0x7ff95815dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34588/0 (socket says 172.21.15.33:34588) 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff95cb9b700 1 -- 172.21.15.33:0/2483965129 learned_addr learned my addr 172.21.15.33:0/2483965129 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:40.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff957fff700 1 --2- 172.21.15.33:0/2483965129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff958159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff9577fe700 1 -- 172.21.15.33:0/2483965129 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897190198 0 0) 0x7ff95815ae80 con 0x7ff95806dc50 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff9577fe700 1 -- 172.21.15.33:0/2483965129 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff948003570 con 0x7ff95806dc50 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff95d39c700 1 -- 172.21.15.33:0/2483965129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 msgr2=0x7ff958159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff95d39c700 1 --2- 172.21.15.33:0/2483965129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff958159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff95d39c700 1 -- 172.21.15.33:0/2483965129 >> v1:172.21.15.33:6789/0 conn(0x7ff95806dc50 legacy=0x7ff95815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff95d39c700 1 -- 172.21.15.33:0/2483965129 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95815d910 con 0x7ff95815a290 2023-10-06T05:21:40.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff95d39c700 1 --2- 172.21.15.33:0/2483965129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95815a290 0x7ff958067ce0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff950009e60 tx=0x7ff950009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d37ce96610004d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.670+0000 7ff9577fe700 1 -- 172.21.15.33:0/2483965129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95000e070 con 0x7ff95815a290 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff9577fe700 1 -- 172.21.15.33:0/2483965129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95000bb60 con 0x7ff95815a290 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff9577fe700 1 -- 172.21.15.33:0/2483965129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950012430 con 0x7ff95815a290 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 -- 172.21.15.33:0/2483965129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95815a290 msgr2=0x7ff958067ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 --2- 172.21.15.33:0/2483965129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95815a290 0x7ff958067ce0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff950009e60 tx=0x7ff950009780 comp rx=0 tx=0).stop 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 -- 172.21.15.33:0/2483965129 shutdown_connections 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 --2- 172.21.15.33:0/2483965129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95815a290 0x7ff958067ce0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 --2- 172.21.15.33:0/2483965129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff958159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 -- 172.21.15.33:0/2483965129 >> 172.21.15.33:0/2483965129 conn(0x7ff9580ccae0 msgr2=0x7ff9580ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 -- 172.21.15.33:0/2483965129 shutdown_connections 2023-10-06T05:21:40.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.671+0000 7ff95edff700 1 -- 172.21.15.33:0/2483965129 wait complete. 2023-10-06T05:21:40.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.673+0000 7ff95edff700 1 Processor -- start 2023-10-06T05:21:40.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.673+0000 7ff95edff700 1 -- start start 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95edff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 0x7ff9580cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95edff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff9580cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95edff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95815a290 0x7ff9580c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95edff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580b1e70 con 0x7ff95806dc50 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95edff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9581585e0 con 0x7ff95815a290 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95edff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580b1260 con 0x7ff958070520 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 0x7ff9580cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 0x7ff9580cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41640/0 (socket says 172.21.15.33:41640) 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 -- 172.21.15.33:0/3332912685 learned_addr learned my addr 172.21.15.33:0/3332912685 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:40.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff957fff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff9580cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95d39c700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95815a290 0x7ff9580c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 -- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 msgr2=0x7ff9580cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff9580cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 -- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95815a290 msgr2=0x7ff9580c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95815a290 0x7ff9580c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.674+0000 7ff95cb9b700 1 -- 172.21.15.33:0/3332912685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff950009a20 con 0x7ff95806dc50 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.675+0000 7ff95cb9b700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 0x7ff9580cb120 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c5c0 tx=0x7ff94c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.675+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c005b30 con 0x7ff95806dc50 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.675+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c005cb0 con 0x7ff95806dc50 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.675+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c007320 con 0x7ff95806dc50 2023-10-06T05:21:40.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.675+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9580b1540 con 0x7ff95806dc50 2023-10-06T05:21:40.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.675+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9580b1b00 con 0x7ff95806dc50 2023-10-06T05:21:40.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.676+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff958063ff0 con 0x7ff95806dc50 2023-10-06T05:21:40.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.677+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff94c01b3b0 con 0x7ff95806dc50 2023-10-06T05:21:40.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.678+0000 7ff954ff9700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff93c066f30 0x7ff93c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.678+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7ff94c083c30 con 0x7ff95806dc50 2023-10-06T05:21:40.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.679+0000 7ff957fff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff93c066f30 0x7ff93c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.679+0000 7ff957fff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff93c066f30 0x7ff93c0693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9580c3600 tx=0x7ff944008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.682+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff94c050bd0 con 0x7ff95806dc50 2023-10-06T05:21:40.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.931+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff958068c30 con 0x7ff95806dc50 2023-10-06T05:21:40.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.931+0000 7ff954ff9700 1 -- 172.21.15.33:0/3332912685 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ff94c017020 con 0x7ff95806dc50 2023-10-06T05:21:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:40.933 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:40.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff93c066f30 msgr2=0x7ff93c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff93c066f30 0x7ff93c0693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9580c3600 tx=0x7ff944008040 comp rx=0 tx=0).stop 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 msgr2=0x7ff9580cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 0x7ff9580cb120 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c5c0 tx=0x7ff94c007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 shutdown_connections 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff93c066f30 0x7ff93c0693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95815a290 0x7ff9580c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff95806dc50 0x7ff9580cb120 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 --2- 172.21.15.33:0/3332912685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff958070520 0x7ff9580cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 >> 172.21.15.33:0/3332912685 conn(0x7ff9580ccae0 msgr2=0x7ff9580a98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 shutdown_connections 2023-10-06T05:21:40.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:40.934+0000 7ff95edff700 1 -- 172.21.15.33:0/3332912685 wait complete. 2023-10-06T05:21:40.937 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:40.953 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:41.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.152+0000 7fe44a59d700 1 Processor -- start 2023-10-06T05:21:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.152+0000 7fe44a59d700 1 -- start start 2023-10-06T05:21:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.153+0000 7fe44a59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4440afdd0 0x7fe4440b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.153+0000 7fe44a59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44406f220 con 0x7fe444159530 2023-10-06T05:21:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.153+0000 7fe44a59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44406e610 con 0x7fe44406e140 2023-10-06T05:21:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.153+0000 7fe44a59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b1e00 con 0x7fe4440afdd0 2023-10-06T05:21:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe44959b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe444159530 0x7fe44406da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34594/0 (socket says 172.21.15.33:34594) 2023-10-06T05:21:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe44959b700 1 -- 172.21.15.33:0/1788283430 learned_addr learned my addr 172.21.15.33:0/1788283430 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe449d9c700 1 --2- 172.21.15.33:0/1788283430 >> v2:172.21.15.167:3300/0 conn(0x7fe4440afdd0 0x7fe4440b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584472499 0 0) 0x7fe44406f220 con 0x7fe444159530 2023-10-06T05:21:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe430003570 con 0x7fe444159530 2023-10-06T05:21:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189072146 0 0) 0x7fe430003570 con 0x7fe444159530 2023-10-06T05:21:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 >> v2:172.21.15.167:3300/0 conn(0x7fe4440afdd0 msgr2=0x7fe4440b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 --2- 172.21.15.33:0/1788283430 >> v2:172.21.15.167:3300/0 conn(0x7fe4440afdd0 0x7fe4440b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 >> v1:172.21.15.167:6789/0 conn(0x7fe44406e140 legacy=0x7fe44415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44406e8f0 con 0x7fe444159530 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe440001470 con 0x7fe444159530 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.154+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 >> v1:172.21.15.33:6789/0 conn(0x7fe444159530 legacy=0x7fe44406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe43bfff700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430003a60 0x7fe430003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe43bfff700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe430004810 0x7fe430004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe43bfff700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4300051c0 0x7fe43000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b1e00 con 0x7fe430003a60 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43000c0b0 con 0x7fe430004810 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43000c270 con 0x7fe4300051c0 2023-10-06T05:21:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe44959b700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430003a60 0x7fe430003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe448d9a700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe430004810 0x7fe430004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe449d9c700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4300051c0 0x7fe43000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe44959b700 1 -- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4300051c0 msgr2=0x7fe43000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe44959b700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4300051c0 0x7fe43000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe44959b700 1 -- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe430004810 msgr2=0x7fe430004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe44959b700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe430004810 0x7fe430004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe44959b700 1 -- 172.21.15.33:0/1788283430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe430003570 con 0x7fe430003a60 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.155+0000 7fe449d9c700 1 -- 172.21.15.33:0/1788283430 reap_dead start 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44959b700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430003a60 0x7fe430003e30 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c2d0 tx=0x7fe440001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308c5da3dc7eb281 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000cba0 con 0x7fe430003a60 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000cd00 con 0x7fe430003a60 2023-10-06T05:21:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe43bfff700 1 -- 172.21.15.33:0/1788283430 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440005b50 con 0x7fe430003a60 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44a59d700 1 -- 172.21.15.33:0/1788283430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430003a60 msgr2=0x7fe430003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44a59d700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430003a60 0x7fe430003e30 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c2d0 tx=0x7fe440001da0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44a59d700 1 -- 172.21.15.33:0/1788283430 shutdown_connections 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44a59d700 1 --2- 172.21.15.33:0/1788283430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe430003a60 0x7fe430003e30 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44a59d700 1 -- 172.21.15.33:0/1788283430 >> 172.21.15.33:0/1788283430 conn(0x7fe44406c560 msgr2=0x7fe44406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.156+0000 7fe44a59d700 1 -- 172.21.15.33:0/1788283430 shutdown_connections 2023-10-06T05:21:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.157+0000 7fe44a59d700 1 -- 172.21.15.33:0/1788283430 wait complete. 2023-10-06T05:21:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.158+0000 7fe44a59d700 1 Processor -- start 2023-10-06T05:21:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.158+0000 7fe44a59d700 1 -- start start 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe44a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe430003a60 0x7fe444167640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe44a59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 0x7fe44414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe44a59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe44415dac0 0x7fe44415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe44a59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44415e4a0 con 0x7fe44416a430 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe44a59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44416bbc0 con 0x7fe44415dac0 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe44a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44416afb0 con 0x7fe430003a60 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe448d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 0x7fe44414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe448d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 0x7fe44414bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41648/0 (socket says 172.21.15.33:41648) 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.159+0000 7fe448d9a700 1 -- 172.21.15.33:0/4272484445 learned_addr learned my addr 172.21.15.33:0/4272484445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe448d9a700 1 -- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe430003a60 msgr2=0x7fe444167640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe448d9a700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe430003a60 0x7fe444167640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe448d9a700 1 -- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe44415dac0 msgr2=0x7fe44415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe448d9a700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe44415dac0 0x7fe44415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe448d9a700 1 -- 172.21.15.33:0/4272484445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44416b290 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe448d9a700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 0x7fe44414bd10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe434002b90 tx=0x7fe434007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43400fa00 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43400fb80 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4340133f0 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44416b570 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.160+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe444159b30 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.161+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4440b23e0 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.162+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4340137a0 con 0x7fe44416a430 2023-10-06T05:21:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.163+0000 7fe4397fa700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe428066f30 0x7fe4280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.163+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fe434084fb0 con 0x7fe44416a430 2023-10-06T05:21:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.164+0000 7fe44959b700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe428066f30 0x7fe4280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.164+0000 7fe44959b700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe428066f30 0x7fe4280693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4440afa50 tx=0x7fe440015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.167+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe434051f50 con 0x7fe44416a430 2023-10-06T05:21:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.419+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe44404c380 con 0x7fe44416a430 2023-10-06T05:21:42.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.419+0000 7fe4397fa700 1 -- 172.21.15.33:0/4272484445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fe434055570 con 0x7fe44416a430 2023-10-06T05:21:42.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:42.421 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.421+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe428066f30 msgr2=0x7fe4280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.421+0000 7fe44a59d700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe428066f30 0x7fe4280693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4440afa50 tx=0x7fe440015000 comp rx=0 tx=0).stop 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.421+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 msgr2=0x7fe44414bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.421+0000 7fe44a59d700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 0x7fe44414bd10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe434002b90 tx=0x7fe434007d10 comp rx=0 tx=0).stop 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.421+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 shutdown_connections 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe428066f30 0x7fe4280693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe44415dac0 0x7fe44415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe430003a60 0x7fe444167640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 --2- 172.21.15.33:0/4272484445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe44416a430 0x7fe44414bd10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 >> 172.21.15.33:0/4272484445 conn(0x7fe44406c560 msgr2=0x7fe44414deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 shutdown_connections 2023-10-06T05:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:42.422+0000 7fe44a59d700 1 -- 172.21.15.33:0/4272484445 wait complete. 2023-10-06T05:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:42.441 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:43.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.640+0000 7f7cb499d700 1 Processor -- start 2023-10-06T05:21:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.641+0000 7f7cb499d700 1 -- start start 2023-10-06T05:21:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7cb499d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7cb00b0a40 0x7f7cb006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7cb499d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7cb015a0a0 0x7f7cb015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7cb499d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb015e2d0 con 0x7f7cb00b0e10 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7cb499d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb015d6c0 con 0x7f7cb015a0a0 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7cb499d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb015d9a0 con 0x7f7cb00b0a40 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7caffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7cb015a0a0 0x7f7cb015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7caffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7cb015a0a0 0x7f7cb015a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41658/0 (socket says 172.21.15.33:41658) 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.642+0000 7f7caffff700 1 -- 172.21.15.33:0/2144294219 learned_addr learned my addr 172.21.15.33:0/2144294219 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7caf7fe700 1 --2- 172.21.15.33:0/2144294219 >> v2:172.21.15.167:3300/0 conn(0x7f7cb00b0a40 0x7f7cb006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7caffff700 1 -- 172.21.15.33:0/2144294219 >> v2:172.21.15.167:3300/0 conn(0x7f7cb00b0a40 msgr2=0x7f7cb006ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7caffff700 1 --2- 172.21.15.33:0/2144294219 >> v2:172.21.15.167:3300/0 conn(0x7f7cb00b0a40 0x7f7cb006ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7caffff700 1 -- 172.21.15.33:0/2144294219 >> v1:172.21.15.167:6789/0 conn(0x7f7cb00b0e10 legacy=0x7f7cb00b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7caffff700 1 -- 172.21.15.33:0/2144294219 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb015dc80 con 0x7f7cb015a0a0 2023-10-06T05:21:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7caffff700 1 --2- 172.21.15.33:0/2144294219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015a4e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7cb006a230 tx=0x7f7ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2344024e016c56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7cae7fc700 1 -- 172.21.15.33:0/2144294219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800e070 con 0x7f7cb015a0a0 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7cae7fc700 1 -- 172.21.15.33:0/2144294219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca800ba50 con 0x7f7cb015a0a0 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.643+0000 7f7cae7fc700 1 -- 172.21.15.33:0/2144294219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80092e0 con 0x7f7cb015a0a0 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 -- 172.21.15.33:0/2144294219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 msgr2=0x7f7cb015a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 --2- 172.21.15.33:0/2144294219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015a4e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7cb006a230 tx=0x7f7ca8009840 comp rx=0 tx=0).stop 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 -- 172.21.15.33:0/2144294219 shutdown_connections 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 --2- 172.21.15.33:0/2144294219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015a4e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 --2- 172.21.15.33:0/2144294219 >> v2:172.21.15.167:3300/0 conn(0x7f7cb00b0a40 0x7f7cb006ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 -- 172.21.15.33:0/2144294219 >> 172.21.15.33:0/2144294219 conn(0x7f7cb006e0f0 msgr2=0x7f7cb00675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 -- 172.21.15.33:0/2144294219 shutdown_connections 2023-10-06T05:21:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.644+0000 7f7cb499d700 1 -- 172.21.15.33:0/2144294219 wait complete. 2023-10-06T05:21:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.645+0000 7f7cb499d700 1 Processor -- start 2023-10-06T05:21:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 -- start start 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb00b0a40 0x7f7cb00c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb00b0e10 0x7f7cb0156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c2d80 con 0x7f7cb015a0a0 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c2170 con 0x7f7cb00b0a40 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.646+0000 7f7cb499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c2450 con 0x7f7cb00b0e10 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41660/0 (socket says 172.21.15.33:41660) 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 -- 172.21.15.33:0/3591427863 learned_addr learned my addr 172.21.15.33:0/3591427863 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:43.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caf7fe700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb00b0a40 0x7f7cb00c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caeffd700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb00b0e10 0x7f7cb0156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 -- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb00b0e10 msgr2=0x7f7cb0156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb00b0e10 0x7f7cb0156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 -- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb00b0a40 msgr2=0x7f7cb00c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb00b0a40 0x7f7cb00c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 -- 172.21.15.33:0/3591427863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8009ae0 con 0x7f7cb015a0a0 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caf7fe700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb00b0a40 0x7f7cb00c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.647+0000 7f7caffff700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015c080 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00ce290 tx=0x7f7ca8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.648+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8008e40 con 0x7f7cb015a0a0 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.648+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca8025460 con 0x7f7cb015a0a0 2023-10-06T05:21:43.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.648+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8025820 con 0x7f7cb015a0a0 2023-10-06T05:21:43.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.648+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb00c2730 con 0x7f7cb015a0a0 2023-10-06T05:21:43.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.648+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb00c3750 con 0x7f7cb015a0a0 2023-10-06T05:21:43.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.649+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb006fcc0 con 0x7f7cb015a0a0 2023-10-06T05:21:43.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.650+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ca8003040 con 0x7f7cb015a0a0 2023-10-06T05:21:43.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.650+0000 7f7c977fe700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.650+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f7ca8086530 con 0x7f7cb015a0a0 2023-10-06T05:21:43.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.651+0000 7f7caf7fe700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.652+0000 7f7caf7fe700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.655+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ca80534d0 con 0x7f7cb015a0a0 2023-10-06T05:21:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.907+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cb004c380 con 0x7f7cb015a0a0 2023-10-06T05:21:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.907+0000 7f7c977fe700 1 -- 172.21.15.33:0/3591427863 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f7ca8056af0 con 0x7f7cb015a0a0 2023-10-06T05:21:43.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:43.910 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c9c066f30 msgr2=0x7f7c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 msgr2=0x7f7cb015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015c080 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00ce290 tx=0x7f7ca8018000 comp rx=0 tx=0).stop 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 shutdown_connections 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cb015a0a0 0x7f7cb015c080 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb00b0a40 0x7f7cb00c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 --2- 172.21.15.33:0/3591427863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb00b0e10 0x7f7cb0156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 >> 172.21.15.33:0/3591427863 conn(0x7f7cb006e0f0 msgr2=0x7f7cb00b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.909+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 shutdown_connections 2023-10-06T05:21:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:43.910+0000 7f7cb499d700 1 -- 172.21.15.33:0/3591427863 wait complete. 2023-10-06T05:21:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:43.929 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:44.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.133+0000 7f650359e700 1 Processor -- start 2023-10-06T05:21:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.133+0000 7f650359e700 1 -- start start 2023-10-06T05:21:45.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.134+0000 7f650359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f65040c8050 0x7f650413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.134+0000 7f650359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f650406aad0 0x7f650406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.134+0000 7f650359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040c9cb0 con 0x7f650413faa0 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.134+0000 7f650359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040ca270 con 0x7f65040c8050 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.134+0000 7f650359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650406b840 con 0x7f650406aad0 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f650259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f650413faa0 0x7f65040c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34608/0 (socket says 172.21.15.33:34608) 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f6501d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f65040c8050 0x7f650413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f650259c700 1 -- 172.21.15.33:0/2332457955 learned_addr learned my addr 172.21.15.33:0/2332457955 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f6502d9d700 1 --2- 172.21.15.33:0/2332457955 >> v2:172.21.15.167:3300/0 conn(0x7f650406aad0 0x7f650406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f6501d9b700 1 -- 172.21.15.33:0/2332457955 >> v2:172.21.15.167:3300/0 conn(0x7f650406aad0 msgr2=0x7f650406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f6501d9b700 1 --2- 172.21.15.33:0/2332457955 >> v2:172.21.15.167:3300/0 conn(0x7f650406aad0 0x7f650406af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f6501d9b700 1 -- 172.21.15.33:0/2332457955 >> v1:172.21.15.33:6789/0 conn(0x7f650413faa0 legacy=0x7f65040c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f6501d9b700 1 -- 172.21.15.33:0/2332457955 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650406bb20 con 0x7f65040c8050 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.135+0000 7f650159a700 1 -- 172.21.15.33:0/2332457955 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697671592 0 0) 0x7f65040c9cb0 con 0x7f650413faa0 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f6501d9b700 1 --2- 172.21.15.33:0/2332457955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65040c8050 0x7f650413fe70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6504156b90 tx=0x7f64f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5002b85d05223a8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650159a700 1 -- 172.21.15.33:0/2332457955 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8009e80 con 0x7f65040c8050 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650159a700 1 -- 172.21.15.33:0/2332457955 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f800b820 con 0x7f65040c8050 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650159a700 1 -- 172.21.15.33:0/2332457955 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800bba0 con 0x7f65040c8050 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 -- 172.21.15.33:0/2332457955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65040c8050 msgr2=0x7f650413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 --2- 172.21.15.33:0/2332457955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65040c8050 0x7f650413fe70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6504156b90 tx=0x7f64f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 -- 172.21.15.33:0/2332457955 shutdown_connections 2023-10-06T05:21:45.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 --2- 172.21.15.33:0/2332457955 >> v2:172.21.15.167:3300/0 conn(0x7f650406aad0 0x7f650406af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 --2- 172.21.15.33:0/2332457955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65040c8050 0x7f650413fe70 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 -- 172.21.15.33:0/2332457955 >> 172.21.15.33:0/2332457955 conn(0x7f650406d030 msgr2=0x7f650406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 -- 172.21.15.33:0/2332457955 shutdown_connections 2023-10-06T05:21:45.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.136+0000 7f650359e700 1 -- 172.21.15.33:0/2332457955 wait complete. 2023-10-06T05:21:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.138+0000 7f650359e700 1 Processor -- start 2023-10-06T05:21:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.138+0000 7f650359e700 1 -- start start 2023-10-06T05:21:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f650359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f650406aad0 0x7f6504156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f650359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65040c8050 0x7f65040b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f650359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 0x7f65040c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f650359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b0c30 con 0x7f650413faa0 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f650359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504156750 con 0x7f65040c8050 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f650359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b0020 con 0x7f650406aad0 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f6502d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 0x7f65040c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f6502d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 0x7f65040c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41672/0 (socket says 172.21.15.33:41672) 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.139+0000 7f6502d9d700 1 -- 172.21.15.33:0/129947202 learned_addr learned my addr 172.21.15.33:0/129947202 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f650259c700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f650406aad0 0x7f6504156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6501d9b700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65040c8050 0x7f65040b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6502d9d700 1 -- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f650406aad0 msgr2=0x7f6504156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6502d9d700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f650406aad0 0x7f6504156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6502d9d700 1 -- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65040c8050 msgr2=0x7f65040b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6502d9d700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65040c8050 0x7f65040b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6502d9d700 1 -- 172.21.15.33:0/129947202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8009420 con 0x7f650413faa0 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f650259c700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f650406aad0 0x7f6504156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6501d9b700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65040c8050 0x7f65040b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f6502d9d700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 0x7f65040c8ae0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009fa0 tx=0x7f64fc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00ed20 con 0x7f650413faa0 2023-10-06T05:21:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc00ee80 con 0x7f650413faa0 2023-10-06T05:21:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040b0300 con 0x7f650413faa0 2023-10-06T05:21:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc014670 con 0x7f650413faa0 2023-10-06T05:21:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.140+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040b08c0 con 0x7f650413faa0 2023-10-06T05:21:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.141+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65040b0da0 con 0x7f650413faa0 2023-10-06T05:21:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.142+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64fc0149e0 con 0x7f650413faa0 2023-10-06T05:21:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.143+0000 7f64f27fc700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64e4066f30 0x7f64e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.143+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f64fc084ff0 con 0x7f650413faa0 2023-10-06T05:21:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.144+0000 7f650259c700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64e4066f30 0x7f64e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.145+0000 7f650259c700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64e4066f30 0x7f64e40693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001810 tx=0x7f64f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.147+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64fc051f90 con 0x7f650413faa0 2023-10-06T05:21:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.393+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65040b0fc0 con 0x7f650413faa0 2023-10-06T05:21:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.394+0000 7f64f27fc700 1 -- 172.21.15.33:0/129947202 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f65040b0fc0 con 0x7f650413faa0 2023-10-06T05:21:45.396 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:45.396 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.396+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64e4066f30 msgr2=0x7f64e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.396+0000 7f650359e700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64e4066f30 0x7f64e40693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001810 tx=0x7f64f4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.396+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 msgr2=0x7f65040c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.396+0000 7f650359e700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 0x7f65040c8ae0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009fa0 tx=0x7f64fc009900 comp rx=0 tx=0).stop 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 shutdown_connections 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64e4066f30 0x7f64e40693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f650413faa0 0x7f65040c8ae0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f650406aad0 0x7f6504156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 --2- 172.21.15.33:0/129947202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65040c8050 0x7f65040b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 >> 172.21.15.33:0/129947202 conn(0x7f650406d030 msgr2=0x7f650406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 shutdown_connections 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:45.397+0000 7f650359e700 1 -- 172.21.15.33:0/129947202 wait complete. 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:45.416 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:46.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.614+0000 7f10bf59e700 1 Processor -- start 2023-10-06T05:21:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.614+0000 7f10bf59e700 1 -- start start 2023-10-06T05:21:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bf59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10c0147760 0x7f10c00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bf59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00cb5f0 con 0x7f10c00c7040 2023-10-06T05:21:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bf59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00ca9e0 con 0x7f10c0147390 2023-10-06T05:21:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bf59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00cacc0 con 0x7f10c0147760 2023-10-06T05:21:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10c0147760 0x7f10c00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bdd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f10c00c7040 0x7f10c00c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34614/0 (socket says 172.21.15.33:34614) 2023-10-06T05:21:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bdd9b700 1 -- 172.21.15.33:0/2469668626 learned_addr learned my addr 172.21.15.33:0/2469668626 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bed9d700 1 -- 172.21.15.33:0/2469668626 >> v1:172.21.15.167:6789/0 conn(0x7f10c0147390 legacy=0x7f10c00c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bed9d700 1 -- 172.21.15.33:0/2469668626 >> v1:172.21.15.33:6789/0 conn(0x7f10c00c7040 legacy=0x7f10c00c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bed9d700 1 -- 172.21.15.33:0/2469668626 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c00cafa0 con 0x7f10c0147760 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.616+0000 7f10bd59a700 1 -- 172.21.15.33:0/2469668626 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789925190 0 0) 0x7f10c00cb5f0 con 0x7f10c00c7040 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.617+0000 7f10bed9d700 1 --2- 172.21.15.33:0/2469668626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0147760 0x7f10c00ca3c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009e60 tx=0x7f10b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3564b3f7f760400 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.617+0000 7f10bd59a700 1 -- 172.21.15.33:0/2469668626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b800e070 con 0x7f10c0147760 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.617+0000 7f10bd59a700 1 -- 172.21.15.33:0/2469668626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b800bc30 con 0x7f10c0147760 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.617+0000 7f10bd59a700 1 -- 172.21.15.33:0/2469668626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8012430 con 0x7f10c0147760 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 -- 172.21.15.33:0/2469668626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0147760 msgr2=0x7f10c00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 --2- 172.21.15.33:0/2469668626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0147760 0x7f10c00ca3c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009e60 tx=0x7f10b8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 -- 172.21.15.33:0/2469668626 shutdown_connections 2023-10-06T05:21:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 --2- 172.21.15.33:0/2469668626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0147760 0x7f10c00ca3c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 -- 172.21.15.33:0/2469668626 >> 172.21.15.33:0/2469668626 conn(0x7f10c006d030 msgr2=0x7f10c006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 -- 172.21.15.33:0/2469668626 shutdown_connections 2023-10-06T05:21:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.618+0000 7f10bf59e700 1 -- 172.21.15.33:0/2469668626 wait complete. 2023-10-06T05:21:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.619+0000 7f10bf59e700 1 Processor -- start 2023-10-06T05:21:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.619+0000 7f10bf59e700 1 -- start start 2023-10-06T05:21:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.620+0000 7f10bf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 0x7f10c00c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.620+0000 7f10bf59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c0147390 0x7f10c00c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.620+0000 7f10bf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c0147760 0x7f10c015d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.620+0000 7f10bf59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00c6cf0 con 0x7f10c00c7040 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.620+0000 7f10bf59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00c5e50 con 0x7f10c0147390 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.620+0000 7f10bf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00c5480 con 0x7f10c0147760 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 0x7f10c00c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 0x7f10c00c4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41698/0 (socket says 172.21.15.33:41698) 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 -- 172.21.15.33:0/442422851 learned_addr learned my addr 172.21.15.33:0/442422851 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10bed9d700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c0147760 0x7f10c015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10bdd9b700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c0147390 0x7f10c00c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 -- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c0147760 msgr2=0x7f10c015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c0147760 0x7f10c015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 -- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c0147390 msgr2=0x7f10c00c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c0147390 0x7f10c00c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 -- 172.21.15.33:0/442422851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8009a20 con 0x7f10c00c7040 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.621+0000 7f10be59c700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 0x7f10c00c4160 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c5c0 tx=0x7f10b000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.622+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0011590 con 0x7f10c00c7040 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.622+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0011710 con 0x7f10c00c7040 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.622+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0011ad0 con 0x7f10c00c7040 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.622+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c00c4870 con 0x7f10c00c7040 2023-10-06T05:21:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.622+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c00c4e30 con 0x7f10c00c7040 2023-10-06T05:21:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.623+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c00c5760 con 0x7f10c00c7040 2023-10-06T05:21:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.624+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10b0012420 con 0x7f10c00c7040 2023-10-06T05:21:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.625+0000 7f10ae7fc700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10a0066f30 0x7f10a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.625+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f10b0083b30 con 0x7f10c00c7040 2023-10-06T05:21:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.625+0000 7f10bdd9b700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10a0066f30 0x7f10a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.626+0000 7f10bdd9b700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10a0066f30 0x7f10a00693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f10c0160550 tx=0x7f10b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.629+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10b004b5a0 con 0x7f10c00c7040 2023-10-06T05:21:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.879+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10c004c380 con 0x7f10c00c7040 2023-10-06T05:21:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.880+0000 7f10ae7fc700 1 -- 172.21.15.33:0/442422851 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f10b0017070 con 0x7f10c00c7040 2023-10-06T05:21:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:46.881 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10a0066f30 msgr2=0x7f10a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10a0066f30 0x7f10a00693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f10c0160550 tx=0x7f10b4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 msgr2=0x7f10c00c4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 0x7f10c00c4160 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c5c0 tx=0x7f10b000c980 comp rx=0 tx=0).stop 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 shutdown_connections 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f10a0066f30 0x7f10a00693e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c0147760 0x7f10c015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c00c7040 0x7f10c00c4160 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 --2- 172.21.15.33:0/442422851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c0147390 0x7f10c00c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 >> 172.21.15.33:0/442422851 conn(0x7f10c006d030 msgr2=0x7f10c006ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 shutdown_connections 2023-10-06T05:21:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:46.882+0000 7f10bf59e700 1 -- 172.21.15.33:0/442422851 wait complete. 2023-10-06T05:21:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:46.901 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:47.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.099+0000 7f338f59e700 1 Processor -- start 2023-10-06T05:21:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.099+0000 7f338f59e700 1 -- start start 2023-10-06T05:21:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.100+0000 7f338f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3390155af0 0x7f339006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.100+0000 7f338f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33900c7920 0x7f33900c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.100+0000 7f338f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339006ab60 con 0x7f339006a6a0 2023-10-06T05:21:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.100+0000 7f338f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339006b7e0 con 0x7f33900c7920 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.100+0000 7f338f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339006ae40 con 0x7f3390155af0 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33900c7920 0x7f33900c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f339006a6a0 0x7f33900c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34620/0 (socket says 172.21.15.33:34620) 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33900c7920 0x7f33900c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41706/0 (socket says 172.21.15.33:41706) 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338dd9b700 1 -- 172.21.15.33:0/2549989088 learned_addr learned my addr 172.21.15.33:0/2549989088 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338e59c700 1 --2- 172.21.15.33:0/2549989088 >> v2:172.21.15.167:3300/0 conn(0x7f3390155af0 0x7f339006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338ed9d700 1 -- 172.21.15.33:0/2549989088 >> v2:172.21.15.167:3300/0 conn(0x7f3390155af0 msgr2=0x7f339006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338ed9d700 1 --2- 172.21.15.33:0/2549989088 >> v2:172.21.15.167:3300/0 conn(0x7f3390155af0 0x7f339006a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338ed9d700 1 -- 172.21.15.33:0/2549989088 >> v1:172.21.15.33:6789/0 conn(0x7f339006a6a0 legacy=0x7f33900c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338ed9d700 1 -- 172.21.15.33:0/2549989088 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339006b120 con 0x7f33900c7920 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.101+0000 7f338d59a700 1 -- 172.21.15.33:0/2549989088 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308723434 0 0) 0x7f339006ab60 con 0x7f339006a6a0 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338ed9d700 1 --2- 172.21.15.33:0/2549989088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900c7920 0x7f33900c9e20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f33900697c0 tx=0x7f338800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3c43c44a1a8a89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338d59a700 1 -- 172.21.15.33:0/2549989088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388010070 con 0x7f33900c7920 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338d59a700 1 -- 172.21.15.33:0/2549989088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338800db00 con 0x7f33900c7920 2023-10-06T05:21:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338d59a700 1 -- 172.21.15.33:0/2549989088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800dea0 con 0x7f33900c7920 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338f59e700 1 -- 172.21.15.33:0/2549989088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900c7920 msgr2=0x7f33900c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338f59e700 1 --2- 172.21.15.33:0/2549989088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900c7920 0x7f33900c9e20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f33900697c0 tx=0x7f338800b990 comp rx=0 tx=0).stop 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338f59e700 1 -- 172.21.15.33:0/2549989088 shutdown_connections 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338f59e700 1 --2- 172.21.15.33:0/2549989088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900c7920 0x7f33900c9e20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338f59e700 1 --2- 172.21.15.33:0/2549989088 >> v2:172.21.15.167:3300/0 conn(0x7f3390155af0 0x7f339006a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.102+0000 7f338f59e700 1 -- 172.21.15.33:0/2549989088 >> 172.21.15.33:0/2549989088 conn(0x7f339006d030 msgr2=0x7f339006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.103+0000 7f338f59e700 1 -- 172.21.15.33:0/2549989088 shutdown_connections 2023-10-06T05:21:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.103+0000 7f338f59e700 1 -- 172.21.15.33:0/2549989088 wait complete. 2023-10-06T05:21:48.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.104+0000 7f338f59e700 1 Processor -- start 2023-10-06T05:21:48.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.104+0000 7f338f59e700 1 -- start start 2023-10-06T05:21:48.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f339006a6a0 0x7f33900b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33900c7920 0x7f339016ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 0x7f339016d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900c5430 con 0x7f3390155af0 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900c5f50 con 0x7f33900c7920 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33900c5620 con 0x7f339006a6a0 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 0x7f339016d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 0x7f339016d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41722/0 (socket says 172.21.15.33:41722) 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338ed9d700 1 -- 172.21.15.33:0/306707241 learned_addr learned my addr 172.21.15.33:0/306707241 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338e59c700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f339006a6a0 0x7f33900b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.105+0000 7f338dd9b700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33900c7920 0x7f339016ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338ed9d700 1 -- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f339006a6a0 msgr2=0x7f33900b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338ed9d700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f339006a6a0 0x7f33900b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338ed9d700 1 -- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33900c7920 msgr2=0x7f339016ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338ed9d700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33900c7920 0x7f339016ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338ed9d700 1 -- 172.21.15.33:0/306707241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338800bc30 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338e59c700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f339006a6a0 0x7f33900b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338ed9d700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 0x7f339016d370 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f338800b6e0 tx=0x7f338801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800a660 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338800a7e0 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33900c5900 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338800aba0 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.106+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33900b1bd0 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.107+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33900c60c0 con 0x7f3390155af0 2023-10-06T05:21:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.108+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3388003040 con 0x7f3390155af0 2023-10-06T05:21:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.109+0000 7f337e7fc700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3370066f30 0x7f33700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.109+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f3388086e00 con 0x7f3390155af0 2023-10-06T05:21:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.110+0000 7f338e59c700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3370066f30 0x7f33700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.111+0000 7f338e59c700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3370066f30 0x7f33700693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3380009cf0 tx=0x7f3380008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.113+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3388053e50 con 0x7f3390155af0 2023-10-06T05:21:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.360+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f339004c380 con 0x7f3390155af0 2023-10-06T05:21:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.361+0000 7f337e7fc700 1 -- 172.21.15.33:0/306707241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f338801a090 con 0x7f3390155af0 2023-10-06T05:21:48.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:48.364 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.363+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3370066f30 msgr2=0x7f33700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.363+0000 7f338f59e700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3370066f30 0x7f33700693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3380009cf0 tx=0x7f3380008040 comp rx=0 tx=0).stop 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.363+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 msgr2=0x7f339016d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.363+0000 7f338f59e700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 0x7f339016d370 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f338800b6e0 tx=0x7f338801b000 comp rx=0 tx=0).stop 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 shutdown_connections 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3370066f30 0x7f33700693e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3390155af0 0x7f339016d370 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f339006a6a0 0x7f33900b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 --2- 172.21.15.33:0/306707241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33900c7920 0x7f339016ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 >> 172.21.15.33:0/306707241 conn(0x7f339006d030 msgr2=0x7f3390159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 shutdown_connections 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:48.364+0000 7f338f59e700 1 -- 172.21.15.33:0/306707241 wait complete. 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:48.383 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:48.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:48.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:49.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:49.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.581+0000 7fc02159d700 1 Processor -- start 2023-10-06T05:21:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.581+0000 7fc02159d700 1 -- start start 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.582+0000 7fc02159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.582+0000 7fc02159d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc01c0cf3b0 0x7fc01c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.582+0000 7fc02159d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c0cbd80 con 0x7fc01c0cc750 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.582+0000 7fc02159d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c0af5f0 con 0x7fc01c0cf3b0 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.582+0000 7fc02159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0af8d0 con 0x7fc01c0ccfc0 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc01c0cf3b0 0x7fc01c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc01c0cf3b0 0x7fc01c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41736/0 (socket says 172.21.15.33:41736) 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 -- 172.21.15.33:0/3441998869 learned_addr learned my addr 172.21.15.33:0/3441998869 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01bfff700 1 --2- 172.21.15.33:0/3441998869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 -- 172.21.15.33:0/3441998869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 msgr2=0x7fc01c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 --2- 172.21.15.33:0/3441998869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 -- 172.21.15.33:0/3441998869 >> v1:172.21.15.33:6789/0 conn(0x7fc01c0cc750 legacy=0x7fc01c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 -- 172.21.15.33:0/3441998869 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c0afbb0 con 0x7fc01c0cf3b0 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01b7fe700 1 --2- 172.21.15.33:0/3441998869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cf3b0 0x7fc01c1594a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc0100099b0 tx=0x7fc010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e56ca4fceb8462 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.583+0000 7fc01affd700 1 -- 172.21.15.33:0/3441998869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000b860 con 0x7fc01c0cf3b0 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc01affd700 1 -- 172.21.15.33:0/3441998869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01000b9c0 con 0x7fc01c0cf3b0 2023-10-06T05:21:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc01affd700 1 -- 172.21.15.33:0/3441998869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000bd60 con 0x7fc01c0cf3b0 2023-10-06T05:21:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 -- 172.21.15.33:0/3441998869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cf3b0 msgr2=0x7fc01c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 --2- 172.21.15.33:0/3441998869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cf3b0 0x7fc01c1594a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc0100099b0 tx=0x7fc010009250 comp rx=0 tx=0).stop 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 -- 172.21.15.33:0/3441998869 shutdown_connections 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 --2- 172.21.15.33:0/3441998869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 --2- 172.21.15.33:0/3441998869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cf3b0 0x7fc01c1594a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 -- 172.21.15.33:0/3441998869 >> 172.21.15.33:0/3441998869 conn(0x7fc01c06cff0 msgr2=0x7fc01c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 -- 172.21.15.33:0/3441998869 shutdown_connections 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.584+0000 7fc02159d700 1 -- 172.21.15.33:0/3441998869 wait complete. 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.586+0000 7fc02159d700 1 Processor -- start 2023-10-06T05:21:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.586+0000 7fc02159d700 1 -- start start 2023-10-06T05:21:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc02159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 0x7fc01c16de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc02159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc02159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c0c7bd0 0x7fc01c13ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc02159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c16d9f0 con 0x7fc01c0cc750 2023-10-06T05:21:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc02159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c16e3c0 con 0x7fc01c0c7bd0 2023-10-06T05:21:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc02159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b5bc0 con 0x7fc01c0ccfc0 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 0x7fc01c16de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 0x7fc01c16de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41748/0 (socket says 172.21.15.33:41748) 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 -- 172.21.15.33:0/774903137 learned_addr learned my addr 172.21.15.33:0/774903137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01b7fe700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 -- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 msgr2=0x7fc01c0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 -- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c0c7bd0 msgr2=0x7fc01c13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc020d9c700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c0c7bd0 0x7fc01c13ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c0c7bd0 0x7fc01c13ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01bfff700 1 -- 172.21.15.33:0/774903137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100094f0 con 0x7fc01c0cc750 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc01b7fe700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.587+0000 7fc020d9c700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c0c7bd0 0x7fc01c13ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.588+0000 7fc01bfff700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 0x7fc01c16de80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00d890 tx=0x7fc00c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.588+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00b600 con 0x7fc01c0cc750 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.588+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c00b760 con 0x7fc01c0cc750 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.588+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00bae0 con 0x7fc01c0cc750 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.588+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c0b5ea0 con 0x7fc01c0cc750 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.588+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c0b57d0 con 0x7fc01c0cc750 2023-10-06T05:21:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.589+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c16d4e0 con 0x7fc01c0cc750 2023-10-06T05:21:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.590+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc00c01e460 con 0x7fc01c0cc750 2023-10-06T05:21:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.591+0000 7fc003fff700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc008066f30 0x7fc0080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.591+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fc00c084eb0 con 0x7fc01c0cc750 2023-10-06T05:21:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.592+0000 7fc01b7fe700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc008066f30 0x7fc0080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.592+0000 7fc01b7fe700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc008066f30 0x7fc0080693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0b0250 tx=0x7fc010017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.595+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc01c16d4e0 con 0x7fc01c0cc750 2023-10-06T05:21:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.843+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc01c16d4e0 con 0x7fc01c0cc750 2023-10-06T05:21:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.843+0000 7fc003fff700 1 -- 172.21.15.33:0/774903137 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fc01c16d4e0 con 0x7fc01c0cc750 2023-10-06T05:21:49.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:49.845 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc008066f30 msgr2=0x7fc0080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc008066f30 0x7fc0080693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0b0250 tx=0x7fc010017000 comp rx=0 tx=0).stop 2023-10-06T05:21:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 msgr2=0x7fc01c16de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 0x7fc01c16de80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00d890 tx=0x7fc00c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 shutdown_connections 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc008066f30 0x7fc0080693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c0c7bd0 0x7fc01c13ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc01c0cc750 0x7fc01c16de80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 --2- 172.21.15.33:0/774903137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc01c0ccfc0 0x7fc01c0b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.845+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 >> 172.21.15.33:0/774903137 conn(0x7fc01c06cff0 msgr2=0x7fc01c157560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.846+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 shutdown_connections 2023-10-06T05:21:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:49.846+0000 7fc02159d700 1 -- 172.21.15.33:0/774903137 wait complete. 2023-10-06T05:21:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:49.866 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:50.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.064+0000 7f7a6139d700 1 Processor -- start 2023-10-06T05:21:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.064+0000 7f7a6139d700 1 -- start start 2023-10-06T05:21:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a6139d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a5c13faa0 0x7f7a5c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a6139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c0c7940 0x7f7a5c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a6139d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a5c06aad0 0x7f7a5c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a6139d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c9cb0 con 0x7f7a5c06aad0 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a6139d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c0ca270 con 0x7f7a5c13faa0 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a6139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c06b840 con 0x7f7a5c0c7940 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a5c06aad0 0x7f7a5c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a5c06aad0 0x7f7a5c06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38390/0 (socket says 172.21.15.33:38390) 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 -- 172.21.15.33:0/1810735941 learned_addr learned my addr 172.21.15.33:0/1810735941 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a5b7fe700 1 --2- 172.21.15.33:0/1810735941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c0c7940 0x7f7a5c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a5bfff700 1 --2- 172.21.15.33:0/1810735941 >> v2:172.21.15.167:3300/0 conn(0x7f7a5c13faa0 0x7f7a5c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 -- 172.21.15.33:0/1810735941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c0c7940 msgr2=0x7f7a5c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 --2- 172.21.15.33:0/1810735941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c0c7940 0x7f7a5c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 -- 172.21.15.33:0/1810735941 >> v2:172.21.15.167:3300/0 conn(0x7f7a5c13faa0 msgr2=0x7f7a5c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 --2- 172.21.15.33:0/1810735941 >> v2:172.21.15.167:3300/0 conn(0x7f7a5c13faa0 0x7f7a5c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.066+0000 7f7a60b9c700 1 -- 172.21.15.33:0/1810735941 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c06bb20 con 0x7f7a5c06aad0 2023-10-06T05:21:51.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.067+0000 7f7a60b9c700 1 --2- 172.21.15.33:0/1810735941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c06af50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c06fe50 tx=0x7f7a5400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fa8b25a7a32d17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.067+0000 7f7a5affd700 1 -- 172.21.15.33:0/1810735941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54010070 con 0x7f7a5c06aad0 2023-10-06T05:21:51.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.067+0000 7f7a5affd700 1 -- 172.21.15.33:0/1810735941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5400de90 con 0x7f7a5c06aad0 2023-10-06T05:21:51.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.067+0000 7f7a5affd700 1 -- 172.21.15.33:0/1810735941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400b320 con 0x7f7a5c06aad0 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.067+0000 7f7a6139d700 1 -- 172.21.15.33:0/1810735941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c06aad0 msgr2=0x7f7a5c06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.067+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1810735941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c06af50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c06fe50 tx=0x7f7a5400ba50 comp rx=0 tx=0).stop 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 -- 172.21.15.33:0/1810735941 shutdown_connections 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1810735941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c06af50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1810735941 >> v2:172.21.15.167:3300/0 conn(0x7f7a5c13faa0 0x7f7a5c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1810735941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c0c7940 0x7f7a5c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 -- 172.21.15.33:0/1810735941 >> 172.21.15.33:0/1810735941 conn(0x7f7a5c06d030 msgr2=0x7f7a5c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 -- 172.21.15.33:0/1810735941 shutdown_connections 2023-10-06T05:21:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.068+0000 7f7a6139d700 1 -- 172.21.15.33:0/1810735941 wait complete. 2023-10-06T05:21:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.069+0000 7f7a6139d700 1 Processor -- start 2023-10-06T05:21:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.069+0000 7f7a6139d700 1 -- start start 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.070+0000 7f7a6139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c0c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.070+0000 7f7a6139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 0x7f7a5c13bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.070+0000 7f7a6139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c13faa0 0x7f7a5c13c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.070+0000 7f7a6139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c4b30 con 0x7f7a5c0c7940 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.070+0000 7f7a6139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c4290 con 0x7f7a5c06aad0 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.070+0000 7f7a6139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c4570 con 0x7f7a5c13faa0 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 0x7f7a5c13bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 0x7f7a5c13bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38402/0 (socket says 172.21.15.33:38402) 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 -- 172.21.15.33:0/1665055415 learned_addr learned my addr 172.21.15.33:0/1665055415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a60b9c700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c13faa0 0x7f7a5c13c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 -- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c13faa0 msgr2=0x7f7a5c13c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5bfff700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c0c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c13faa0 0x7f7a5c13c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 -- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a5c06aad0 msgr2=0x7f7a5c0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c0c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 -- 172.21.15.33:0/1665055415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5400bcf0 con 0x7f7a5c0c7940 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a60b9c700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c13faa0 0x7f7a5c13c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5bfff700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c0c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a5b7fe700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 0x7f7a5c13bef0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d890 tx=0x7f7a50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.071+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000b600 con 0x7f7a5c0c7940 2023-10-06T05:21:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.072+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000b760 con 0x7f7a5c0c7940 2023-10-06T05:21:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.072+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000bae0 con 0x7f7a5c0c7940 2023-10-06T05:21:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.072+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c0c4850 con 0x7f7a5c0c7940 2023-10-06T05:21:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.072+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c0c9780 con 0x7f7a5c0c7940 2023-10-06T05:21:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.073+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c158880 con 0x7f7a5c0c7940 2023-10-06T05:21:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.074+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a5001e460 con 0x7f7a5c0c7940 2023-10-06T05:21:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.075+0000 7f7a43fff700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a48066f30 0x7f7a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.075+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f7a50084eb0 con 0x7f7a5c0c7940 2023-10-06T05:21:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.075+0000 7f7a5bfff700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a48066f30 0x7f7a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.076+0000 7f7a5bfff700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a48066f30 0x7f7a480693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009990 tx=0x7f7a4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.079+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a5c158880 con 0x7f7a5c0c7940 2023-10-06T05:21:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.324+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a5c158880 con 0x7f7a5c0c7940 2023-10-06T05:21:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.325+0000 7f7a43fff700 1 -- 172.21.15.33:0/1665055415 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f7a5c158880 con 0x7f7a5c0c7940 2023-10-06T05:21:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:51.327 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a48066f30 msgr2=0x7f7a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a48066f30 0x7f7a480693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009990 tx=0x7f7a4c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 msgr2=0x7f7a5c13bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 0x7f7a5c13bef0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d890 tx=0x7f7a50008040 comp rx=0 tx=0).stop 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 shutdown_connections 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a48066f30 0x7f7a480693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a5c13faa0 0x7f7a5c13c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a5c06aad0 0x7f7a5c0c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 --2- 172.21.15.33:0/1665055415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a5c0c7940 0x7f7a5c13bef0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.327+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 >> 172.21.15.33:0/1665055415 conn(0x7f7a5c06d030 msgr2=0x7f7a5c15a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.328+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 shutdown_connections 2023-10-06T05:21:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:51.328+0000 7f7a6139d700 1 -- 172.21.15.33:0/1665055415 wait complete. 2023-10-06T05:21:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:51.347 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:52.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.545+0000 7f5b2a99d700 1 Processor -- start 2023-10-06T05:21:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.545+0000 7f5b2a99d700 1 -- start start 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.546+0000 7f5b2a99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b240ccfc0 0x7f5b240cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.546+0000 7f5b2a99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b240cf3b0 0x7f5b241594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.546+0000 7f5b2a99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240cbd80 con 0x7f5b240cc750 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.546+0000 7f5b2a99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240af5f0 con 0x7f5b240cf3b0 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.546+0000 7f5b2a99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240af8d0 con 0x7f5b240ccfc0 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b240cf3b0 0x7f5b241594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b240cf3b0 0x7f5b241594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38406/0 (socket says 172.21.15.33:38406) 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2a19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b240cc750 0x7f5b240b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55040/0 (socket says 172.21.15.33:55040) 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 -- 172.21.15.33:0/664007477 learned_addr learned my addr 172.21.15.33:0/664007477 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2999b700 1 --2- 172.21.15.33:0/664007477 >> v2:172.21.15.167:3300/0 conn(0x7f5b240ccfc0 0x7f5b240cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 -- 172.21.15.33:0/664007477 >> v2:172.21.15.167:3300/0 conn(0x7f5b240ccfc0 msgr2=0x7f5b240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 --2- 172.21.15.33:0/664007477 >> v2:172.21.15.167:3300/0 conn(0x7f5b240ccfc0 0x7f5b240cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 -- 172.21.15.33:0/664007477 >> v1:172.21.15.33:6789/0 conn(0x7f5b240cc750 legacy=0x7f5b240b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 -- 172.21.15.33:0/664007477 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240afbb0 con 0x7f5b240cf3b0 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b28999700 1 -- 172.21.15.33:0/664007477 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895350607 0 0) 0x7f5b240cbd80 con 0x7f5b240cc750 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.547+0000 7f5b2919a700 1 --2- 172.21.15.33:0/664007477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cf3b0 0x7f5b241594a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099b0 tx=0x7f5b14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba7e9e70b3e95a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b28999700 1 -- 172.21.15.33:0/664007477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14009ea0 con 0x7f5b240cf3b0 2023-10-06T05:21:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b28999700 1 -- 172.21.15.33:0/664007477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400b820 con 0x7f5b240cf3b0 2023-10-06T05:21:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b28999700 1 -- 172.21.15.33:0/664007477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400bbc0 con 0x7f5b240cf3b0 2023-10-06T05:21:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 -- 172.21.15.33:0/664007477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cf3b0 msgr2=0x7f5b241594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/664007477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cf3b0 0x7f5b241594a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099b0 tx=0x7f5b14009250 comp rx=0 tx=0).stop 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 -- 172.21.15.33:0/664007477 shutdown_connections 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/664007477 >> v2:172.21.15.167:3300/0 conn(0x7f5b240ccfc0 0x7f5b240cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/664007477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cf3b0 0x7f5b241594a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 -- 172.21.15.33:0/664007477 >> 172.21.15.33:0/664007477 conn(0x7f5b2406cff0 msgr2=0x7f5b2406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 -- 172.21.15.33:0/664007477 shutdown_connections 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.548+0000 7f5b2a99d700 1 -- 172.21.15.33:0/664007477 wait complete. 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.550+0000 7f5b2a99d700 1 Processor -- start 2023-10-06T05:21:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.550+0000 7f5b2a99d700 1 -- start start 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 0x7f5b240b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240ccfc0 0x7f5b240cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b240b5b20 0x7f5b240cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24169ea0 con 0x7f5b240cc750 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b4140 con 0x7f5b240b5b20 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b4420 con 0x7f5b240ccfc0 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 0x7f5b240b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 0x7f5b240b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38420/0 (socket says 172.21.15.33:38420) 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 -- 172.21.15.33:0/1168258561 learned_addr learned my addr 172.21.15.33:0/1168258561 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2a19c700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b240b5b20 0x7f5b240cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2919a700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240ccfc0 0x7f5b240cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 -- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240ccfc0 msgr2=0x7f5b240cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240ccfc0 0x7f5b240cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 -- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b240b5b20 msgr2=0x7f5b240cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b240b5b20 0x7f5b240cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.551+0000 7f5b2999b700 1 -- 172.21.15.33:0/1168258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140094f0 con 0x7f5b240cc750 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.552+0000 7f5b2999b700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 0x7f5b240b55e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009ef0 tx=0x7f5b2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.552+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000cc50 con 0x7f5b240cc750 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.552+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2000f040 con 0x7f5b240cc750 2023-10-06T05:21:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.552+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20014480 con 0x7f5b240cc750 2023-10-06T05:21:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.552+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b240b4700 con 0x7f5b240cc750 2023-10-06T05:21:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.552+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b24154390 con 0x7f5b240cc750 2023-10-06T05:21:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.553+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240b48f0 con 0x7f5b240cc750 2023-10-06T05:21:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.554+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b200147f0 con 0x7f5b240cc750 2023-10-06T05:21:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.555+0000 7f5b19ffb700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.555+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f5b200852b0 con 0x7f5b240cc750 2023-10-06T05:21:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.555+0000 7f5b2919a700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.556+0000 7f5b2919a700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099b0 tx=0x7f5b14017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.559+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b20052250 con 0x7f5b240cc750 2023-10-06T05:21:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.805+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b2404c380 con 0x7f5b240cc750 2023-10-06T05:21:52.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.805+0000 7f5b19ffb700 1 -- 172.21.15.33:0/1168258561 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f5b20055870 con 0x7f5b240cc750 2023-10-06T05:21:52.807 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:52.807 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 msgr2=0x7f5b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099b0 tx=0x7f5b14017000 comp rx=0 tx=0).stop 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 msgr2=0x7f5b240b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 0x7f5b240b55e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009ef0 tx=0x7f5b2000b040 comp rx=0 tx=0).stop 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 shutdown_connections 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b08066f30 0x7f5b080693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b240b5b20 0x7f5b240cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b240cc750 0x7f5b240b55e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 --2- 172.21.15.33:0/1168258561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b240ccfc0 0x7f5b240cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 >> 172.21.15.33:0/1168258561 conn(0x7f5b2406cff0 msgr2=0x7f5b24155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 shutdown_connections 2023-10-06T05:21:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:52.808+0000 7f5b2a99d700 1 -- 172.21.15.33:0/1168258561 wait complete. 2023-10-06T05:21:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:52.827 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:53.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.025+0000 7fca0659d700 1 Processor -- start 2023-10-06T05:21:54.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.025+0000 7fca0659d700 1 -- start start 2023-10-06T05:21:54.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0659d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca0013faa0 0x7fca0013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca000c7940 0x7fca000c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca000c9cb0 con 0x7fca0006aad0 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0659d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0006b660 con 0x7fca0013faa0 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca000ca270 con 0x7fca000c7940 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca0013faa0 0x7fca0013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca0013faa0 0x7fca0013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38428/0 (socket says 172.21.15.33:38428) 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca0559b700 1 -- 172.21.15.33:0/2251180640 learned_addr learned my addr 172.21.15.33:0/2251180640 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:54.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.027+0000 7fca04d9a700 1 --2- 172.21.15.33:0/2251180640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca000c7940 0x7fca000c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fca0559b700 1 -- 172.21.15.33:0/2251180640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca000c7940 msgr2=0x7fca000c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fca0559b700 1 --2- 172.21.15.33:0/2251180640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca000c7940 0x7fca000c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fca0559b700 1 -- 172.21.15.33:0/2251180640 >> v1:172.21.15.167:6789/0 conn(0x7fca0006aad0 legacy=0x7fca0006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fca0559b700 1 -- 172.21.15.33:0/2251180640 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0006b840 con 0x7fca0013faa0 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fca0559b700 1 --2- 172.21.15.33:0/2251180640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca0013faa0 0x7fca0013fe70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0099b0 tx=0x7fc9fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961186bfd90798fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fc9f7fff700 1 -- 172.21.15.33:0/2251180640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc009ea0 con 0x7fca0013faa0 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fc9f7fff700 1 -- 172.21.15.33:0/2251180640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc00b860 con 0x7fca0013faa0 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.028+0000 7fc9f7fff700 1 -- 172.21.15.33:0/2251180640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00bc00 con 0x7fca0013faa0 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 -- 172.21.15.33:0/2251180640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca0013faa0 msgr2=0x7fca0013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 --2- 172.21.15.33:0/2251180640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca0013faa0 0x7fca0013fe70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0099b0 tx=0x7fc9fc009250 comp rx=0 tx=0).stop 2023-10-06T05:21:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 -- 172.21.15.33:0/2251180640 shutdown_connections 2023-10-06T05:21:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 --2- 172.21.15.33:0/2251180640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca0013faa0 0x7fca0013fe70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 --2- 172.21.15.33:0/2251180640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca000c7940 0x7fca000c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 -- 172.21.15.33:0/2251180640 >> 172.21.15.33:0/2251180640 conn(0x7fca0006d030 msgr2=0x7fca0006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 -- 172.21.15.33:0/2251180640 shutdown_connections 2023-10-06T05:21:54.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.029+0000 7fca0659d700 1 -- 172.21.15.33:0/2251180640 wait complete. 2023-10-06T05:21:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.030+0000 7fca0659d700 1 Processor -- start 2023-10-06T05:21:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.030+0000 7fca0659d700 1 -- start start 2023-10-06T05:21:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca0659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0006aad0 0x7fca0015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca0659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 0x7fca00156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca0659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0013faa0 0x7fca001565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca0659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0013eae0 con 0x7fca000c7940 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca0659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0013ded0 con 0x7fca0006aad0 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca0659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0013e1b0 con 0x7fca0013faa0 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.031+0000 7fca04d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 0x7fca00156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 0x7fca00156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38436/0 (socket says 172.21.15.33:38436) 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 -- 172.21.15.33:0/3258127515 learned_addr learned my addr 172.21.15.33:0/3258127515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca0559b700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0006aad0 0x7fca0015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca05d9c700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0013faa0 0x7fca001565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 -- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0013faa0 msgr2=0x7fca001565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0013faa0 0x7fca001565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 -- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0006aad0 msgr2=0x7fca0015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0006aad0 0x7fca0015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 -- 172.21.15.33:0/3258127515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0094f0 con 0x7fca000c7940 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca05d9c700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0013faa0 0x7fca001565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca04d9a700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 0x7fca00156070 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000ec80 tx=0x7fc9f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000c9d0 con 0x7fca000c7940 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000cb30 con 0x7fca000c7940 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0013470 con 0x7fca000c7940 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.032+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0013e490 con 0x7fca000c7940 2023-10-06T05:21:54.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.033+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0013ef70 con 0x7fca000c7940 2023-10-06T05:21:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.034+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0013ec90 con 0x7fca000c7940 2023-10-06T05:21:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.034+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9f0013820 con 0x7fca000c7940 2023-10-06T05:21:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.036+0000 7fc9f57fa700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc9e4066f30 0x7fc9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.036+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fc9f0084b70 con 0x7fca000c7940 2023-10-06T05:21:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.036+0000 7fca0559b700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc9e4066f30 0x7fc9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.037+0000 7fca0559b700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc9e4066f30 0x7fc9e40693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0099b0 tx=0x7fc9fc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.040+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca0013ec90 con 0x7fca000c7940 2023-10-06T05:21:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.288+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca0004c380 con 0x7fca000c7940 2023-10-06T05:21:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.288+0000 7fc9f57fa700 1 -- 172.21.15.33:0/3258127515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fc9f0051b10 con 0x7fca000c7940 2023-10-06T05:21:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:54.290 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc9e4066f30 msgr2=0x7fc9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc9e4066f30 0x7fc9e40693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0099b0 tx=0x7fc9fc017000 comp rx=0 tx=0).stop 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 msgr2=0x7fca00156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 0x7fca00156070 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000ec80 tx=0x7fc9f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 shutdown_connections 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc9e4066f30 0x7fc9e40693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca0013faa0 0x7fca001565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca0006aad0 0x7fca0015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 --2- 172.21.15.33:0/3258127515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca000c7940 0x7fca00156070 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 >> 172.21.15.33:0/3258127515 conn(0x7fca0006d030 msgr2=0x7fca0006af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 shutdown_connections 2023-10-06T05:21:54.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:54.291+0000 7fca0659d700 1 -- 172.21.15.33:0/3258127515 wait complete. 2023-10-06T05:21:54.294 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:54.310 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:55.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.508+0000 7fa86759d700 1 Processor -- start 2023-10-06T05:21:55.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.508+0000 7fa86759d700 1 -- start start 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.509+0000 7fa86759d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8600cb710 0x7fa86006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.509+0000 7fa86759d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600c9cb0 con 0x7fa86013faa0 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.509+0000 7fa86759d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600ca270 con 0x7fa8600c8050 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.509+0000 7fa86759d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86006b840 con 0x7fa8600cb710 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa86659b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa86013faa0 0x7fa8600c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55042/0 (socket says 172.21.15.33:55042) 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa86659b700 1 -- 172.21.15.33:0/287430660 learned_addr learned my addr 172.21.15.33:0/287430660 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa866d9c700 1 --2- 172.21.15.33:0/287430660 >> v2:172.21.15.33:3300/0 conn(0x7fa8600cb710 0x7fa86006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa865599700 1 -- 172.21.15.33:0/287430660 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313510526 0 0) 0x7fa8600c9cb0 con 0x7fa86013faa0 2023-10-06T05:21:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa865599700 1 -- 172.21.15.33:0/287430660 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c003570 con 0x7fa86013faa0 2023-10-06T05:21:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa866d9c700 1 -- 172.21.15.33:0/287430660 >> v1:172.21.15.167:6789/0 conn(0x7fa8600c8050 legacy=0x7fa86006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa866d9c700 1 -- 172.21.15.33:0/287430660 >> v1:172.21.15.33:6789/0 conn(0x7fa86013faa0 legacy=0x7fa8600c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa866d9c700 1 -- 172.21.15.33:0/287430660 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86006bb20 con 0x7fa8600cb710 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.510+0000 7fa866d9c700 1 --2- 172.21.15.33:0/287430660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600cb710 0x7fa86006b1e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa858009e60 tx=0x7fa858009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b5dd2340bc86b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa865599700 1 -- 172.21.15.33:0/287430660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800e070 con 0x7fa8600cb710 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa865599700 1 -- 172.21.15.33:0/287430660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85800bb60 con 0x7fa8600cb710 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa865599700 1 -- 172.21.15.33:0/287430660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858012430 con 0x7fa8600cb710 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 -- 172.21.15.33:0/287430660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600cb710 msgr2=0x7fa86006b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 --2- 172.21.15.33:0/287430660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600cb710 0x7fa86006b1e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa858009e60 tx=0x7fa858009780 comp rx=0 tx=0).stop 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 -- 172.21.15.33:0/287430660 shutdown_connections 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 --2- 172.21.15.33:0/287430660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600cb710 0x7fa86006b1e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 -- 172.21.15.33:0/287430660 >> 172.21.15.33:0/287430660 conn(0x7fa86006d030 msgr2=0x7fa86006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 -- 172.21.15.33:0/287430660 shutdown_connections 2023-10-06T05:21:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.511+0000 7fa86759d700 1 -- 172.21.15.33:0/287430660 wait complete. 2023-10-06T05:21:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.513+0000 7fa86759d700 1 Processor -- start 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.513+0000 7fa86759d700 1 -- start start 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86759d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 0x7fa86013ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86759d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8600cb710 0x7fa86015d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86759d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86013faa0 0x7fa8601595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86759d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860158320 con 0x7fa8600c8050 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86759d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860159140 con 0x7fa8600cb710 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86759d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86013f360 con 0x7fa86013faa0 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 0x7fa86013ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 0x7fa86013ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38458/0 (socket says 172.21.15.33:38458) 2023-10-06T05:21:55.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.514+0000 7fa86659b700 1 -- 172.21.15.33:0/1363999714 learned_addr learned my addr 172.21.15.33:0/1363999714 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:55.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86659b700 1 -- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86013faa0 msgr2=0x7fa8601595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86659b700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86013faa0 0x7fa8601595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86659b700 1 -- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8600cb710 msgr2=0x7fa86015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86659b700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8600cb710 0x7fa86015d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86659b700 1 -- 172.21.15.33:0/1363999714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858009a20 con 0x7fa8600c8050 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86659b700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 0x7fa86013ee20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c590 tx=0x7fa85c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c005b30 con 0x7fa8600c8050 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c005c90 con 0x7fa8600c8050 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860158530 con 0x7fa8600c8050 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c007320 con 0x7fa8600c8050 2023-10-06T05:21:55.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.515+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa860158af0 con 0x7fa8600c8050 2023-10-06T05:21:55.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.516+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86004c380 con 0x7fa8600c8050 2023-10-06T05:21:55.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.517+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa85c01b3b0 con 0x7fa8600c8050 2023-10-06T05:21:55.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.518+0000 7fa8567fc700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa844066f30 0x7fa8440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.518+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fa85c083d30 con 0x7fa8600c8050 2023-10-06T05:21:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.518+0000 7fa865d9a700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa844066f30 0x7fa8440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.519+0000 7fa865d9a700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa844066f30 0x7fa8440693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa86006e430 tx=0x7fa850007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.522+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa85c050cd0 con 0x7fa8600c8050 2023-10-06T05:21:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.769+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8600c9f00 con 0x7fa8600c8050 2023-10-06T05:21:55.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.770+0000 7fa8567fc700 1 -- 172.21.15.33:0/1363999714 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fa85c017070 con 0x7fa8600c8050 2023-10-06T05:21:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:55.772 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa844066f30 msgr2=0x7fa8440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa844066f30 0x7fa8440693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa86006e430 tx=0x7fa850007b40 comp rx=0 tx=0).stop 2023-10-06T05:21:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 msgr2=0x7fa86013ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 0x7fa86013ee20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c590 tx=0x7fa85c007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 shutdown_connections 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa844066f30 0x7fa8440693e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa86013faa0 0x7fa8601595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8600c8050 0x7fa86013ee20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 --2- 172.21.15.33:0/1363999714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8600cb710 0x7fa86015d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 >> 172.21.15.33:0/1363999714 conn(0x7fa86006d030 msgr2=0x7fa8600b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 shutdown_connections 2023-10-06T05:21:55.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:55.772+0000 7fa86759d700 1 -- 172.21.15.33:0/1363999714 wait complete. 2023-10-06T05:21:55.777 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:55.791 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:56.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.992+0000 7f4c2939d700 1 Processor -- start 2023-10-06T05:21:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.992+0000 7f4c2939d700 1 -- start start 2023-10-06T05:21:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c2939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c2406a0d0 0x7f4c240adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c2939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c24158ba0 con 0x7f4c24155140 2023-10-06T05:21:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c2939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240ae490 con 0x7f4c24158700 2023-10-06T05:21:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c2939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240c8e40 con 0x7f4c2406a0d0 2023-10-06T05:21:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c28b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c2406a0d0 0x7f4c240adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c28b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c2406a0d0 0x7f4c240adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38472/0 (socket says 172.21.15.33:38472) 2023-10-06T05:21:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c28b9c700 1 -- 172.21.15.33:0/1517972451 learned_addr learned my addr 172.21.15.33:0/1517972451 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c28b9c700 1 -- 172.21.15.33:0/1517972451 >> v1:172.21.15.167:6789/0 conn(0x7f4c24158700 legacy=0x7f4c240699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c28b9c700 1 -- 172.21.15.33:0/1517972451 >> v1:172.21.15.33:6789/0 conn(0x7f4c24155140 legacy=0x7f4c240c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.994+0000 7f4c22ffd700 1 -- 172.21.15.33:0/1517972451 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544751653 0 0) 0x7f4c24158ba0 con 0x7f4c24155140 2023-10-06T05:21:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.995+0000 7f4c28b9c700 1 -- 172.21.15.33:0/1517972451 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240c99e0 con 0x7f4c2406a0d0 2023-10-06T05:21:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.995+0000 7f4c28b9c700 1 --2- 172.21.15.33:0/1517972451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2406a0d0 0x7f4c240adf50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009f50 tx=0x7f4c1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86817ef925a4aab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.995+0000 7f4c22ffd700 1 -- 172.21.15.33:0/1517972451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00e070 con 0x7f4c2406a0d0 2023-10-06T05:21:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.995+0000 7f4c22ffd700 1 -- 172.21.15.33:0/1517972451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c00ba30 con 0x7f4c2406a0d0 2023-10-06T05:21:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.995+0000 7f4c22ffd700 1 -- 172.21.15.33:0/1517972451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0092e0 con 0x7f4c2406a0d0 2023-10-06T05:21:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.995+0000 7f4c2939d700 1 -- 172.21.15.33:0/1517972451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2406a0d0 msgr2=0x7f4c240adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.996+0000 7f4c2939d700 1 --2- 172.21.15.33:0/1517972451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2406a0d0 0x7f4c240adf50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009f50 tx=0x7f4c1c009840 comp rx=0 tx=0).stop 2023-10-06T05:21:57.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.996+0000 7f4c2939d700 1 -- 172.21.15.33:0/1517972451 shutdown_connections 2023-10-06T05:21:57.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.996+0000 7f4c2939d700 1 --2- 172.21.15.33:0/1517972451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2406a0d0 0x7f4c240adf50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.996+0000 7f4c2939d700 1 -- 172.21.15.33:0/1517972451 >> 172.21.15.33:0/1517972451 conn(0x7f4c2406d030 msgr2=0x7f4c2406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.996+0000 7f4c2939d700 1 -- 172.21.15.33:0/1517972451 shutdown_connections 2023-10-06T05:21:57.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.996+0000 7f4c2939d700 1 -- 172.21.15.33:0/1517972451 wait complete. 2023-10-06T05:21:57.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.997+0000 7f4c2939d700 1 Processor -- start 2023-10-06T05:21:57.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.997+0000 7f4c2939d700 1 -- start start 2023-10-06T05:21:57.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c2939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2406a0d0 0x7f4c240b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c2939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c24155140 0x7f4c240aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c2939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 0x7f4c24152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c2939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240ca130 con 0x7f4c24158700 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c2939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240b20e0 con 0x7f4c24155140 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c2939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24153360 con 0x7f4c2406a0d0 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.998+0000 7f4c28b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 0x7f4c24152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 0x7f4c24152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38480/0 (socket says 172.21.15.33:38480) 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 -- 172.21.15.33:0/3699756181 learned_addr learned my addr 172.21.15.33:0/3699756181 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c23fff700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2406a0d0 0x7f4c240b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c237fe700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c24155140 0x7f4c240aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 -- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2406a0d0 msgr2=0x7f4c240b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2406a0d0 0x7f4c240b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 -- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c24155140 msgr2=0x7f4c240aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c24155140 0x7f4c240aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 -- 172.21.15.33:0/3699756181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009ae0 con 0x7f4c24158700 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c28b9c700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 0x7f4c24152e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4c24063610 tx=0x7f4c1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c012e60 con 0x7f4c24158700 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0092e0 con 0x7f4c24158700 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c240b1b20 con 0x7f4c24158700 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:56.999+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c01e660 con 0x7f4c24158700 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.000+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c240c9dc0 con 0x7f4c24158700 2023-10-06T05:21:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.001+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c240ca2e0 con 0x7f4c24158700 2023-10-06T05:21:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.001+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c1c01ea10 con 0x7f4c24158700 2023-10-06T05:21:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.002+0000 7f4c0bfff700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c10066f30 0x7f4c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.002+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f4c1c0864e0 con 0x7f4c24158700 2023-10-06T05:21:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.003+0000 7f4c23fff700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c10066f30 0x7f4c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.003+0000 7f4c23fff700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c10066f30 0x7f4c100693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4c140011e0 tx=0x7f4c14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.007+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c240ca2e0 con 0x7f4c24158700 2023-10-06T05:21:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.252+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c240ca2e0 con 0x7f4c24158700 2023-10-06T05:21:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.253+0000 7f4c0bfff700 1 -- 172.21.15.33:0/3699756181 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f4c240ca2e0 con 0x7f4c24158700 2023-10-06T05:21:57.255 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:57.255 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:57.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c10066f30 msgr2=0x7f4c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c10066f30 0x7f4c100693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4c140011e0 tx=0x7f4c14009040 comp rx=0 tx=0).stop 2023-10-06T05:21:57.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 msgr2=0x7f4c24152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 0x7f4c24152e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4c24063610 tx=0x7f4c1c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 shutdown_connections 2023-10-06T05:21:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c10066f30 0x7f4c100693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c24158700 0x7f4c24152e20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2406a0d0 0x7f4c240b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 --2- 172.21.15.33:0/3699756181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c24155140 0x7f4c240aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 >> 172.21.15.33:0/3699756181 conn(0x7f4c2406d030 msgr2=0x7f4c240cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 shutdown_connections 2023-10-06T05:21:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:57.255+0000 7f4c2939d700 1 -- 172.21.15.33:0/3699756181 wait complete. 2023-10-06T05:21:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:57.275 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:58.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:58.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.474+0000 7fb23699d700 1 Processor -- start 2023-10-06T05:21:58.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.474+0000 7fb23699d700 1 -- start start 2023-10-06T05:21:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.475+0000 7fb23699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb230155af0 0x7fb23006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.475+0000 7fb23699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb23006a6a0 0x7fb2300c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.475+0000 7fb23699d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23006ab60 con 0x7fb2300c8b50 2023-10-06T05:21:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.475+0000 7fb23699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23006b7e0 con 0x7fb23006a6a0 2023-10-06T05:21:58.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.475+0000 7fb23699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23006ae40 con 0x7fb230155af0 2023-10-06T05:21:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.476+0000 7fb23599b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb230155af0 0x7fb23006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.476+0000 7fb23619c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb2300c8b50 0x7fb2301598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:44494/0 (socket says 172.21.15.33:44494) 2023-10-06T05:21:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.476+0000 7fb23619c700 1 -- 172.21.15.33:0/911852915 learned_addr learned my addr 172.21.15.33:0/911852915 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.476+0000 7fb23519a700 1 --2- 172.21.15.33:0/911852915 >> v2:172.21.15.167:3300/0 conn(0x7fb23006a6a0 0x7fb2300c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb234999700 1 -- 172.21.15.33:0/911852915 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751509391 0 0) 0x7fb23006ab60 con 0x7fb2300c8b50 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb234999700 1 -- 172.21.15.33:0/911852915 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c003570 con 0x7fb2300c8b50 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb23599b700 1 -- 172.21.15.33:0/911852915 >> v2:172.21.15.167:3300/0 conn(0x7fb23006a6a0 msgr2=0x7fb2300c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb23599b700 1 --2- 172.21.15.33:0/911852915 >> v2:172.21.15.167:3300/0 conn(0x7fb23006a6a0 0x7fb2300c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb23599b700 1 -- 172.21.15.33:0/911852915 >> v1:172.21.15.167:6789/0 conn(0x7fb2300c8b50 legacy=0x7fb2301598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb23599b700 1 -- 172.21.15.33:0/911852915 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23006b120 con 0x7fb230155af0 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb23519a700 1 --2- 172.21.15.33:0/911852915 >> v2:172.21.15.167:3300/0 conn(0x7fb23006a6a0 0x7fb2300c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:58.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb23599b700 1 --2- 172.21.15.33:0/911852915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb230155af0 0x7fb23006a160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0098e0 tx=0x7fb22c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e09a8d78068040 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb234999700 1 -- 172.21.15.33:0/911852915 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c011630 con 0x7fb230155af0 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb234999700 1 -- 172.21.15.33:0/911852915 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c011790 con 0x7fb230155af0 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.477+0000 7fb234999700 1 -- 172.21.15.33:0/911852915 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c011b10 con 0x7fb230155af0 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 -- 172.21.15.33:0/911852915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb230155af0 msgr2=0x7fb23006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 --2- 172.21.15.33:0/911852915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb230155af0 0x7fb23006a160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0098e0 tx=0x7fb22c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 -- 172.21.15.33:0/911852915 shutdown_connections 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 --2- 172.21.15.33:0/911852915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb230155af0 0x7fb23006a160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 --2- 172.21.15.33:0/911852915 >> v2:172.21.15.167:3300/0 conn(0x7fb23006a6a0 0x7fb2300c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 -- 172.21.15.33:0/911852915 >> 172.21.15.33:0/911852915 conn(0x7fb23006d030 msgr2=0x7fb23006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 -- 172.21.15.33:0/911852915 shutdown_connections 2023-10-06T05:21:58.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.478+0000 7fb23699d700 1 -- 172.21.15.33:0/911852915 wait complete. 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.479+0000 7fb23699d700 1 Processor -- start 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.480+0000 7fb23699d700 1 -- start start 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.480+0000 7fb23699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb23006a6a0 0x7fb230143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.480+0000 7fb23699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 0x7fb230148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.480+0000 7fb23699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb230155af0 0x7fb2300c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300afcb0 con 0x7fb2300c8b50 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230149ce0 con 0x7fb230155af0 2023-10-06T05:21:58.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300af6a0 con 0x7fb23006a6a0 2023-10-06T05:21:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 0x7fb230148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 0x7fb230148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38484/0 (socket says 172.21.15.33:38484) 2023-10-06T05:21:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 -- 172.21.15.33:0/786492879 learned_addr learned my addr 172.21.15.33:0/786492879 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:58.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23599b700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb23006a6a0 0x7fb230143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23619c700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb230155af0 0x7fb2300c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 -- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb23006a6a0 msgr2=0x7fb230143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb23006a6a0 0x7fb230143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 -- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb230155af0 msgr2=0x7fb2300c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb230155af0 0x7fb2300c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 -- 172.21.15.33:0/786492879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220009420 con 0x7fb2300c8b50 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23599b700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb23006a6a0 0x7fb230143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23619c700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb230155af0 0x7fb2300c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.481+0000 7fb23519a700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 0x7fb230148950 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb22000ec80 tx=0x7fb2200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.482+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000c9d0 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.482+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22000cb30 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.482+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220013470 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.482+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c009420 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.482+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb230149720 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.483+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300afa80 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.484+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb220013820 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.485+0000 7fb225ffb700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb214066f30 0x7fb2140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.485+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb2200848a0 con 0x7fb2300c8b50 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.485+0000 7fb23599b700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb214066f30 0x7fb2140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.486+0000 7fb23599b700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb214066f30 0x7fb2140693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0098e0 tx=0x7fb22c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.489+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2300afa80 con 0x7fb2300c8b50 2023-10-06T05:21:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.736+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2300afa80 con 0x7fb2300c8b50 2023-10-06T05:21:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.737+0000 7fb225ffb700 1 -- 172.21.15.33:0/786492879 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fb2300afa80 con 0x7fb2300c8b50 2023-10-06T05:21:58.739 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:21:58.739 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb214066f30 msgr2=0x7fb2140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb214066f30 0x7fb2140693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0098e0 tx=0x7fb22c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 msgr2=0x7fb230148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 0x7fb230148950 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb22000ec80 tx=0x7fb2200091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 shutdown_connections 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb214066f30 0x7fb2140693e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb230155af0 0x7fb2300c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb23006a6a0 0x7fb230143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 --2- 172.21.15.33:0/786492879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2300c8b50 0x7fb230148950 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 >> 172.21.15.33:0/786492879 conn(0x7fb23006d030 msgr2=0x7fb2300c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 shutdown_connections 2023-10-06T05:21:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:58.739+0000 7fb23699d700 1 -- 172.21.15.33:0/786492879 wait complete. 2023-10-06T05:21:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:21:58.760 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:21:59.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.961+0000 7fadda39d700 1 Processor -- start 2023-10-06T05:21:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.961+0000 7fadda39d700 1 -- start start 2023-10-06T05:21:59.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadda39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fadd40ccfc0 0x7fadd40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadda39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40cf3b0 0x7fadd41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadda39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40cbd80 con 0x7fadd40cc750 2023-10-06T05:21:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadda39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd40af5f0 con 0x7fadd40ccfc0 2023-10-06T05:21:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadda39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40af8d0 con 0x7fadd40cf3b0 2023-10-06T05:21:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadd9b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fadd40cc750 0x7fadd40b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45206/0 (socket says 172.21.15.33:45206) 2023-10-06T05:21:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadd9b9c700 1 -- 172.21.15.33:0/1785407466 learned_addr learned my addr 172.21.15.33:0/1785407466 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:21:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadd8b9a700 1 --2- 172.21.15.33:0/1785407466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40cf3b0 0x7fadd41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.963+0000 7fadd939b700 1 --2- 172.21.15.33:0/1785407466 >> v2:172.21.15.167:3300/0 conn(0x7fadd40ccfc0 0x7fadd40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadd8b9a700 1 -- 172.21.15.33:0/1785407466 >> v2:172.21.15.167:3300/0 conn(0x7fadd40ccfc0 msgr2=0x7fadd40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadd8b9a700 1 --2- 172.21.15.33:0/1785407466 >> v2:172.21.15.167:3300/0 conn(0x7fadd40ccfc0 0x7fadd40cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadd8b9a700 1 -- 172.21.15.33:0/1785407466 >> v1:172.21.15.33:6789/0 conn(0x7fadd40cc750 legacy=0x7fadd40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadd8b9a700 1 -- 172.21.15.33:0/1785407466 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40afbb0 con 0x7fadd40cf3b0 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadd939b700 1 --2- 172.21.15.33:0/1785407466 >> v2:172.21.15.167:3300/0 conn(0x7fadd40ccfc0 0x7fadd40cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadd8b9a700 1 --2- 172.21.15.33:0/1785407466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40cf3b0 0x7fadd41594a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fadc40098e0 tx=0x7fadc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d2d247d28f098e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadcbfff700 1 -- 172.21.15.33:0/1785407466 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc400b640 con 0x7fadd40cf3b0 2023-10-06T05:21:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadcbfff700 1 -- 172.21.15.33:0/1785407466 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc400b7a0 con 0x7fadd40cf3b0 2023-10-06T05:21:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.964+0000 7fadcbfff700 1 -- 172.21.15.33:0/1785407466 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc400bb40 con 0x7fadd40cf3b0 2023-10-06T05:21:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 -- 172.21.15.33:0/1785407466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40cf3b0 msgr2=0x7fadd41594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 --2- 172.21.15.33:0/1785407466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40cf3b0 0x7fadd41594a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fadc40098e0 tx=0x7fadc40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 -- 172.21.15.33:0/1785407466 shutdown_connections 2023-10-06T05:21:59.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 --2- 172.21.15.33:0/1785407466 >> v2:172.21.15.167:3300/0 conn(0x7fadd40ccfc0 0x7fadd40cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 --2- 172.21.15.33:0/1785407466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40cf3b0 0x7fadd41594a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 -- 172.21.15.33:0/1785407466 >> 172.21.15.33:0/1785407466 conn(0x7fadd406cff0 msgr2=0x7fadd406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 -- 172.21.15.33:0/1785407466 shutdown_connections 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.965+0000 7fadda39d700 1 -- 172.21.15.33:0/1785407466 wait complete. 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.967+0000 7fadda39d700 1 Processor -- start 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.967+0000 7fadda39d700 1 -- start start 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadda39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 0x7fadd4153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadda39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd40ccfc0 0x7fadd40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadda39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40b5ab0 0x7fadd415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadda39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd41536a0 con 0x7fadd40cc750 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadda39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40b4640 con 0x7fadd40ccfc0 2023-10-06T05:21:59.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadda39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd416a7d0 con 0x7fadd40b5ab0 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 0x7fadd4153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 0x7fadd4153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55828/0 (socket says 172.21.15.33:55828) 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 -- 172.21.15.33:0/1805480797 learned_addr learned my addr 172.21.15.33:0/1805480797 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd9b9c700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40b5ab0 0x7fadd415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd8b9a700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd40ccfc0 0x7fadd40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 -- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40b5ab0 msgr2=0x7fadd415f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40b5ab0 0x7fadd415f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 -- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd40ccfc0 msgr2=0x7fadd40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd40ccfc0 0x7fadd40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.968+0000 7fadd939b700 1 -- 172.21.15.33:0/1805480797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0009420 con 0x7fadd40cc750 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.969+0000 7fadd939b700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 0x7fadd4153160 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fadd000ec80 tx=0x7fadd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.969+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000c920 con 0x7fadd40cc750 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.969+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000ca80 con 0x7fadd40cc750 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.969+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc4009420 con 0x7fadd40cc750 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.969+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000ce00 con 0x7fadd40cc750 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.969+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd416ad90 con 0x7fadd40cc750 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.970+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd40ccbb0 con 0x7fadd40cc750 2023-10-06T05:21:59.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.971+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadd0013680 con 0x7fadd40cc750 2023-10-06T05:21:59.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.972+0000 7fadc97fa700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadb8066f30 0x7fadb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.972+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fadd0084e90 con 0x7fadd40cc750 2023-10-06T05:21:59.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.972+0000 7fadd8b9a700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadb8066f30 0x7fadb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.973+0000 7fadd8b9a700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadb8066f30 0x7fadb80693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fadc40098e0 tx=0x7fadc4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:21:59.976+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadd0051eb0 con 0x7fadd40cc750 2023-10-06T05:22:00.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.224+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadd40d1230 con 0x7fadd40cc750 2023-10-06T05:22:00.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.224+0000 7fadc97fa700 1 -- 172.21.15.33:0/1805480797 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fadd0017090 con 0x7fadd40cc750 2023-10-06T05:22:00.227 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:00.227 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadb8066f30 msgr2=0x7fadb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadb8066f30 0x7fadb80693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fadc40098e0 tx=0x7fadc4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 msgr2=0x7fadd4153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 0x7fadd4153160 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fadd000ec80 tx=0x7fadd00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 shutdown_connections 2023-10-06T05:22:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fadb8066f30 0x7fadb80693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadd40b5ab0 0x7fadd415f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd40cc750 0x7fadd4153160 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 --2- 172.21.15.33:0/1805480797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd40ccfc0 0x7fadd40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 >> 172.21.15.33:0/1805480797 conn(0x7fadd406cff0 msgr2=0x7fadd4156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 shutdown_connections 2023-10-06T05:22:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:00.227+0000 7fadda39d700 1 -- 172.21.15.33:0/1805480797 wait complete. 2023-10-06T05:22:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:22:00.247 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:22:01.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.444+0000 7f6269b9d700 1 Processor -- start 2023-10-06T05:22:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.445+0000 7f6269b9d700 1 -- start start 2023-10-06T05:22:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6269b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6264155af0 0x7f626406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6269b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f626406a6a0 0x7f62640c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6269b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626406ab60 con 0x7f62640c8b50 2023-10-06T05:22:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6269b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626406b7e0 con 0x7f6264155af0 2023-10-06T05:22:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6269b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626406ae40 con 0x7f626406a6a0 2023-10-06T05:22:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6268b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6264155af0 0x7f626406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6268b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6264155af0 0x7f626406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55836/0 (socket says 172.21.15.33:55836) 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6268b9b700 1 -- 172.21.15.33:0/1200600037 learned_addr learned my addr 172.21.15.33:0/1200600037 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.446+0000 7f6263fff700 1 --2- 172.21.15.33:0/1200600037 >> v2:172.21.15.167:3300/0 conn(0x7f626406a6a0 0x7f62640c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f6268b9b700 1 -- 172.21.15.33:0/1200600037 >> v2:172.21.15.167:3300/0 conn(0x7f626406a6a0 msgr2=0x7f62640c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f6268b9b700 1 --2- 172.21.15.33:0/1200600037 >> v2:172.21.15.167:3300/0 conn(0x7f626406a6a0 0x7f62640c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f6268b9b700 1 -- 172.21.15.33:0/1200600037 >> v1:172.21.15.167:6789/0 conn(0x7f62640c8b50 legacy=0x7f62641598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f6268b9b700 1 -- 172.21.15.33:0/1200600037 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626406b120 con 0x7f6264155af0 2023-10-06T05:22:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f6268b9b700 1 --2- 172.21.15.33:0/1200600037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f626406a160 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f626406f250 tx=0x7f6258009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4ae656d586e3f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f62637fe700 1 -- 172.21.15.33:0/1200600037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625800b860 con 0x7f6264155af0 2023-10-06T05:22:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f62637fe700 1 -- 172.21.15.33:0/1200600037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625800b9c0 con 0x7f6264155af0 2023-10-06T05:22:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.447+0000 7f62637fe700 1 -- 172.21.15.33:0/1200600037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625800bd60 con 0x7f6264155af0 2023-10-06T05:22:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 -- 172.21.15.33:0/1200600037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 msgr2=0x7f626406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 --2- 172.21.15.33:0/1200600037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f626406a160 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f626406f250 tx=0x7f6258009250 comp rx=0 tx=0).stop 2023-10-06T05:22:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 -- 172.21.15.33:0/1200600037 shutdown_connections 2023-10-06T05:22:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 --2- 172.21.15.33:0/1200600037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f626406a160 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 --2- 172.21.15.33:0/1200600037 >> v2:172.21.15.167:3300/0 conn(0x7f626406a6a0 0x7f62640c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 -- 172.21.15.33:0/1200600037 >> 172.21.15.33:0/1200600037 conn(0x7f626406d030 msgr2=0x7f626406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 -- 172.21.15.33:0/1200600037 shutdown_connections 2023-10-06T05:22:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.448+0000 7f6269b9d700 1 -- 172.21.15.33:0/1200600037 wait complete. 2023-10-06T05:22:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.449+0000 7f6269b9d700 1 Processor -- start 2023-10-06T05:22:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 -- start start 2023-10-06T05:22:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f626406a6a0 0x7f62641446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62640c8b50 0x7f6264144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f6264145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c5bb0 con 0x7f6264155af0 2023-10-06T05:22:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c5e90 con 0x7f62640c8b50 2023-10-06T05:22:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.450+0000 7f6269b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c6170 con 0x7f626406a6a0 2023-10-06T05:22:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f6264145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f6264145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55840/0 (socket says 172.21.15.33:55840) 2023-10-06T05:22:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 -- 172.21.15.33:0/4023683445 learned_addr learned my addr 172.21.15.33:0/4023683445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f6263fff700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62640c8b50 0x7f6264144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f6268b9b700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f626406a6a0 0x7f62641446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 -- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f626406a6a0 msgr2=0x7f62641446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f626406a6a0 0x7f62641446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 -- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62640c8b50 msgr2=0x7f6264144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62640c8b50 0x7f6264144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.451+0000 7f626939c700 1 -- 172.21.15.33:0/4023683445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580094f0 con 0x7f6264155af0 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.452+0000 7f626939c700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f6264145150 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c910 tx=0x7f625c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.452+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c012630 con 0x7f6264155af0 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.452+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c012790 con 0x7f6264155af0 2023-10-06T05:22:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.452+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62640c6450 con 0x7f6264155af0 2023-10-06T05:22:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.452+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c012b50 con 0x7f6264155af0 2023-10-06T05:22:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.452+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6264145690 con 0x7f6264155af0 2023-10-06T05:22:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.453+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62640c6970 con 0x7f6264155af0 2023-10-06T05:22:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.454+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f625c01b450 con 0x7f6264155af0 2023-10-06T05:22:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.455+0000 7f6260ff9700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6248066f60 0x7f6248069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.455+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f625c0849e0 con 0x7f6264155af0 2023-10-06T05:22:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.456+0000 7f6268b9b700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6248066f60 0x7f6248069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.457+0000 7f6268b9b700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6248066f60 0x7f6248069410 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f626406f250 tx=0x7f6258017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.460+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62640c6970 con 0x7f6264155af0 2023-10-06T05:22:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.705+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62640c6970 con 0x7f6264155af0 2023-10-06T05:22:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.706+0000 7f6260ff9700 1 -- 172.21.15.33:0/4023683445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f62640c6970 con 0x7f6264155af0 2023-10-06T05:22:01.708 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:01.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6248066f60 msgr2=0x7f6248069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6248066f60 0x7f6248069410 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f626406f250 tx=0x7f6258017000 comp rx=0 tx=0).stop 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 msgr2=0x7f6264145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f6264145150 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c910 tx=0x7f625c0016f0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 shutdown_connections 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6248066f60 0x7f6248069410 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264155af0 0x7f6264145150 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f626406a6a0 0x7f62641446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 --2- 172.21.15.33:0/4023683445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62640c8b50 0x7f6264144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 >> 172.21.15.33:0/4023683445 conn(0x7f626406d030 msgr2=0x7f626406e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 shutdown_connections 2023-10-06T05:22:01.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:01.708+0000 7f6269b9d700 1 -- 172.21.15.33:0/4023683445 wait complete. 2023-10-06T05:22:01.712 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:22:01.728 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:22:02.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.925+0000 7f256359e700 1 Processor -- start 2023-10-06T05:22:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.925+0000 7f256359e700 1 -- start start 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 0x7f25640cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640cbd80 con 0x7f25640cc750 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640af5f0 con 0x7f25640cf3b0 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25640af8d0 con 0x7f25640ccfc0 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f2562d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f25640cc750 0x7f25640b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45222/0 (socket says 172.21.15.33:45222) 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f2562d9d700 1 -- 172.21.15.33:0/4200965714 learned_addr learned my addr 172.21.15.33:0/4200965714 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256259c700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 0x7f25640cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651133433 0 0) 0x7f25640cbd80 con 0x7f25640cc750 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.927+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f254c003570 con 0x7f25640cc750 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673977760 0 0) 0x7f25640af5f0 con 0x7f25640cf3b0 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25640cbd80 con 0x7f25640cf3b0 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2657831867 0 0) 0x7f254c003570 con 0x7f25640cc750 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 msgr2=0x7f25640cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 0x7f25640cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 >> v1:172.21.15.167:6789/0 conn(0x7f25640cf3b0 legacy=0x7f25641594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640afbb0 con 0x7f25640cc750 2023-10-06T05:22:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f255c0017f0 con 0x7f25640cc750 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 >> v1:172.21.15.33:6789/0 conn(0x7f25640cc750 legacy=0x7f25640b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 0x7f254c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c004180 0x7f254c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c004d50 0x7f254c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640af8d0 con 0x7f254c004d50 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c00bba0 con 0x7f254c004180 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c00bd60 con 0x7f25640ccfc0 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f2562d9d700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c004d50 0x7f254c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f256259c700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 0x7f254c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.928+0000 7f2561d9b700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c004180 0x7f254c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f2562d9d700 1 -- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 msgr2=0x7f254c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f2562d9d700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25640ccfc0 0x7f254c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f2562d9d700 1 -- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c004180 msgr2=0x7f254c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f2562d9d700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254c004180 0x7f254c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f2562d9d700 1 -- 172.21.15.33:0/4200965714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c003570 con 0x7f254c004d50 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f2562d9d700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c004d50 0x7f254c0051c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f255c00a8a0 tx=0x7f255c002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22877b194f669a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c0070d0 con 0x7f254c004d50 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c007230 con 0x7f254c004d50 2023-10-06T05:22:02.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.929+0000 7f256159a700 1 -- 172.21.15.33:0/4200965714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c0075f0 con 0x7f254c004d50 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f256359e700 1 -- 172.21.15.33:0/4200965714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c004d50 msgr2=0x7f254c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f256359e700 1 --2- 172.21.15.33:0/4200965714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c004d50 0x7f254c0051c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f255c00a8a0 tx=0x7f255c002190 comp rx=0 tx=0).stop 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f2562d9d700 1 -- 172.21.15.33:0/4200965714 reap_dead start 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f256359e700 1 -- 172.21.15.33:0/4200965714 shutdown_connections 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f256359e700 1 -- 172.21.15.33:0/4200965714 >> 172.21.15.33:0/4200965714 conn(0x7f256406cff0 msgr2=0x7f256406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f256359e700 1 -- 172.21.15.33:0/4200965714 shutdown_connections 2023-10-06T05:22:02.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.930+0000 7f256359e700 1 -- 172.21.15.33:0/4200965714 wait complete. 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.931+0000 7f256359e700 1 Processor -- start 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.931+0000 7f256359e700 1 -- start start 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f256359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c004d50 0x7f25640c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f256359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 0x7f25640b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f256359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2564159fe0 0x7f256415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f256359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256414ed10 con 0x7f256414ee70 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f256359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640c9ea0 con 0x7f2564159fe0 2023-10-06T05:22:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f256359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256415a970 con 0x7f254c004d50 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.932+0000 7f2561d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 0x7f25640b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 0x7f25640b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55862/0 (socket says 172.21.15.33:55862) 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 -- 172.21.15.33:0/3795833380 learned_addr learned my addr 172.21.15.33:0/3795833380 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f256259c700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c004d50 0x7f25640c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2562d9d700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2564159fe0 0x7f256415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 -- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c004d50 msgr2=0x7f25640c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c004d50 0x7f25640c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 -- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2564159fe0 msgr2=0x7f256415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2564159fe0 0x7f256415a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 -- 172.21.15.33:0/3795833380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640c9600 con 0x7f256414ee70 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f256259c700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c004d50 0x7f25640c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2562d9d700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2564159fe0 0x7f256415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f2561d9b700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 0x7f25640b5b70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2558002c30 tx=0x7f2558007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800fa50 con 0x7f256414ee70 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255800fbd0 con 0x7f256414ee70 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25640c98e0 con 0x7f256414ee70 2023-10-06T05:22:02.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.933+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558006820 con 0x7f256414ee70 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.934+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640b3da0 con 0x7f256414ee70 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.934+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2564071060 con 0x7f256414ee70 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.935+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2558006c50 con 0x7f256414ee70 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.936+0000 7f25527fc700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2544066f30 0x7f25440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.937+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f2558084ca0 con 0x7f256414ee70 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.937+0000 7f256259c700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2544066f30 0x7f25440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.938+0000 7f256259c700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2544066f30 0x7f25440693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25540093f0 tx=0x7f2554008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:02.940+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2564071060 con 0x7f256414ee70 2023-10-06T05:22:03.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.186+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2564071060 con 0x7f256414ee70 2023-10-06T05:22:03.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.187+0000 7f25527fc700 1 -- 172.21.15.33:0/3795833380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f2564071060 con 0x7f256414ee70 2023-10-06T05:22:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:03.190 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:21:15.111815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2544066f30 msgr2=0x7f25440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2544066f30 0x7f25440693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25540093f0 tx=0x7f2554008040 comp rx=0 tx=0).stop 2023-10-06T05:22:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 msgr2=0x7f25640b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 0x7f25640b5b70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2558002c30 tx=0x7f2558007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 shutdown_connections 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2544066f30 0x7f25440693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2564159fe0 0x7f256415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f254c004d50 0x7f25640c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 --2- 172.21.15.33:0/3795833380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256414ee70 0x7f25640b5b70 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 >> 172.21.15.33:0/3795833380 conn(0x7f256406cff0 msgr2=0x7f25640ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 shutdown_connections 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:03.189+0000 7f256359e700 1 -- 172.21.15.33:0/3795833380 wait complete. 2023-10-06T05:22:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 62 2023-10-06T05:22:03.429 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:22:04.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.625+0000 7fd2627ff700 1 Processor -- start 2023-10-06T05:22:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.626+0000 7fd2627ff700 1 -- start start 2023-10-06T05:22:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd2627ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd2627ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0ca8d0 con 0x7fd25c0af000 2023-10-06T05:22:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd2627ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0cabb0 con 0x7fd25c06ba40 2023-10-06T05:22:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd2627ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c06ad00 con 0x7fd25c13fb70 2023-10-06T05:22:04.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd260d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd25c0af000 0x7fd25c0c8c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45234/0 (socket says 172.21.15.33:45234) 2023-10-06T05:22:04.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd260d9c700 1 -- 172.21.15.33:0/374064921 learned_addr learned my addr 172.21.15.33:0/374064921 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:04.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.627+0000 7fd25bfff700 1 --2- 172.21.15.33:0/374064921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:04.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797148997 0 0) 0x7fd25c0ca8d0 con 0x7fd25c0af000 2023-10-06T05:22:04.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd244003570 con 0x7fd25c0af000 2023-10-06T05:22:04.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424768115 0 0) 0x7fd25c0cabb0 con 0x7fd25c06ba40 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd25c0ca8d0 con 0x7fd25c06ba40 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25bfff700 1 -- 172.21.15.33:0/374064921 >> v1:172.21.15.167:6789/0 conn(0x7fd25c06ba40 legacy=0x7fd25c0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25bfff700 1 -- 172.21.15.33:0/374064921 >> v1:172.21.15.33:6789/0 conn(0x7fd25c0af000 legacy=0x7fd25c0c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25bfff700 1 -- 172.21.15.33:0/374064921 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c06afe0 con 0x7fd25c13fb70 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.628+0000 7fd25bfff700 1 --2- 172.21.15.33:0/374064921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c06b500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009910 tx=0x7fd24c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c02cbdd6723a1b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c011630 con 0x7fd25c13fb70 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c011790 con 0x7fd25c13fb70 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd25affd700 1 -- 172.21.15.33:0/374064921 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c011b30 con 0x7fd25c13fb70 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 -- 172.21.15.33:0/374064921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 msgr2=0x7fd25c06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 --2- 172.21.15.33:0/374064921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c06b500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009910 tx=0x7fd24c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 -- 172.21.15.33:0/374064921 shutdown_connections 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 --2- 172.21.15.33:0/374064921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c06b500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 -- 172.21.15.33:0/374064921 >> 172.21.15.33:0/374064921 conn(0x7fd25c0675f0 msgr2=0x7fd25c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 -- 172.21.15.33:0/374064921 shutdown_connections 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.629+0000 7fd2627ff700 1 -- 172.21.15.33:0/374064921 wait complete. 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.631+0000 7fd2627ff700 1 Processor -- start 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.631+0000 7fd2627ff700 1 -- start start 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd2627ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd25c06ba40 0x7fd25c13dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd2627ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 0x7fd25c0add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd2627ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd2627ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c157fe0 con 0x7fd25c0af000 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd2627ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b0f90 con 0x7fd25c06ba40 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd2627ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0c3f00 con 0x7fd25c13fb70 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 0x7fd25c0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 0x7fd25c0add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55878/0 (socket says 172.21.15.33:55878) 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 -- 172.21.15.33:0/855684898 learned_addr learned my addr 172.21.15.33:0/855684898 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25bfff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd25c06ba40 0x7fd25c13dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd260d9c700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 -- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 msgr2=0x7fd25c0ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c0ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 -- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd25c06ba40 msgr2=0x7fd25c13dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd25c06ba40 0x7fd25c13dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.632+0000 7fd25b7fe700 1 -- 172.21.15.33:0/855684898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c009420 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd260d9c700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c0ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd25b7fe700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 0x7fd25c0add10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c7c0 tx=0x7fd250007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250010510 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250002e90 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500108e0 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0c41e0 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.633+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c157c00 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.634+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c063ff0 con 0x7fd25c0af000 2023-10-06T05:22:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.635+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd250010cd0 con 0x7fd25c0af000 2023-10-06T05:22:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.636+0000 7fd243fff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd248066f30 0x7fd2480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.636+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd250083f70 con 0x7fd25c0af000 2023-10-06T05:22:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.637+0000 7fd25bfff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd248066f30 0x7fd2480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.638+0000 7fd25bfff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd248066f30 0x7fd2480693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0091e0 tx=0x7fd24c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.641+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd250050ef0 con 0x7fd25c0af000 2023-10-06T05:22:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.891+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd25c158120 con 0x7fd25c0af000 2023-10-06T05:22:04.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.892+0000 7fd243fff700 1 -- 172.21.15.33:0/855684898 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fd250017020 con 0x7fd25c0af000 2023-10-06T05:22:04.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:04.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:22:04.211250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.894+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd248066f30 msgr2=0x7fd2480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.894+0000 7fd2627ff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd248066f30 0x7fd2480693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0091e0 tx=0x7fd24c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.894+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 msgr2=0x7fd25c0add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:04.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.894+0000 7fd2627ff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 0x7fd25c0add10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c7c0 tx=0x7fd250007de0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.894+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 shutdown_connections 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.894+0000 7fd2627ff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd248066f30 0x7fd2480693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.895+0000 7fd2627ff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd25c13fb70 0x7fd25c0ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.895+0000 7fd2627ff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd25c06ba40 0x7fd25c13dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.895+0000 7fd2627ff700 1 --2- 172.21.15.33:0/855684898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd25c0af000 0x7fd25c0add10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.895+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 >> 172.21.15.33:0/855684898 conn(0x7fd25c0675f0 msgr2=0x7fd25c13eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.895+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 shutdown_connections 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:04.895+0000 7fd2627ff700 1 -- 172.21.15.33:0/855684898 wait complete. 2023-10-06T05:22:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 63 2023-10-06T05:22:04.915 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2023-10-06T05:22:05.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:06.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.107+0000 7f23d979d700 1 Processor -- start 2023-10-06T05:22:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.107+0000 7f23d979d700 1 -- start start 2023-10-06T05:22:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.108+0000 7f23d979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40cf3b0 0x7f23d41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.108+0000 7f23d979d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40cbd80 con 0x7f23d40af0a0 2023-10-06T05:22:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40af5f0 con 0x7f23d40ccfc0 2023-10-06T05:22:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40af8d0 con 0x7f23d40cf3b0 2023-10-06T05:22:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d8f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f23d40af0a0 0x7f23d40b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45240/0 (socket says 172.21.15.33:45240) 2023-10-06T05:22:06.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d8f9c700 1 -- 172.21.15.33:0/976547734 learned_addr learned my addr 172.21.15.33:0/976547734 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:06.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d37fe700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40cf3b0 0x7f23d41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278925580 0 0) 0x7f23d40cbd80 con 0x7f23d40af0a0 2023-10-06T05:22:06.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23bc003570 con 0x7f23d40af0a0 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518657558 0 0) 0x7f23d40af5f0 con 0x7f23d40ccfc0 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d40cbd80 con 0x7f23d40ccfc0 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142371096 0 0) 0x7f23bc003570 con 0x7f23d40af0a0 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40cf3b0 msgr2=0x7f23d41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40cf3b0 0x7f23d41594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.109+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 >> v1:172.21.15.167:6789/0 conn(0x7f23d40ccfc0 legacy=0x7f23d40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40afbb0 con 0x7f23d40af0a0 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f23cc0017f0 con 0x7f23d40af0a0 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 >> v1:172.21.15.33:6789/0 conn(0x7f23d40af0a0 legacy=0x7f23d40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23bc003c20 0x7f23bc003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc004810 0x7f23bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23bc0051c0 0x7f23bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40af8d0 con 0x7f23bc0051c0 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc00bfd0 con 0x7f23bc004810 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc00c190 con 0x7f23bc003c20 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23bc0051c0 0x7f23bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d3fff700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23bc003c20 0x7f23bc003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d37fe700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc004810 0x7f23bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d8f9c700 1 -- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23bc003c20 msgr2=0x7f23bc003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23bc003c20 0x7f23bc003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d8f9c700 1 -- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc004810 msgr2=0x7f23bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc004810 0x7f23bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d8f9c700 1 -- 172.21.15.33:0/976547734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc003570 con 0x7f23bc0051c0 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.110+0000 7f23d3fff700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23bc003c20 0x7f23bc003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23bc0051c0 0x7f23bc00ba60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00a7f0 tx=0x7f23cc0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a30d48acee256c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0070d0 con 0x7f23bc0051c0 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc007250 con 0x7f23bc0051c0 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d2ffd700 1 -- 172.21.15.33:0/976547734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc007610 con 0x7f23bc0051c0 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d979d700 1 -- 172.21.15.33:0/976547734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23bc0051c0 msgr2=0x7f23bc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d979d700 1 --2- 172.21.15.33:0/976547734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23bc0051c0 0x7f23bc00ba60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00a7f0 tx=0x7f23cc0020c0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d8f9c700 1 -- 172.21.15.33:0/976547734 reap_dead start 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d979d700 1 -- 172.21.15.33:0/976547734 shutdown_connections 2023-10-06T05:22:06.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d979d700 1 -- 172.21.15.33:0/976547734 >> 172.21.15.33:0/976547734 conn(0x7f23d406cff0 msgr2=0x7f23d406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.111+0000 7f23d979d700 1 -- 172.21.15.33:0/976547734 shutdown_connections 2023-10-06T05:22:06.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.112+0000 7f23d979d700 1 -- 172.21.15.33:0/976547734 wait complete. 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.113+0000 7f23d979d700 1 Processor -- start 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.113+0000 7f23d979d700 1 -- start start 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc0051c0 0x7f23d40b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40c8d80 0x7f23d40c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 0x7f23d40c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c7af0 con 0x7f23d41431d0 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c88f0 con 0x7f23bc0051c0 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c7ce0 con 0x7f23d40c8d80 2023-10-06T05:22:06.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d8f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 0x7f23d40c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d8f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 0x7f23d40c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55898/0 (socket says 172.21.15.33:55898) 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.114+0000 7f23d8f9c700 1 -- 172.21.15.33:0/813379627 learned_addr learned my addr 172.21.15.33:0/813379627 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d37fe700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40c8d80 0x7f23d40c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d3fff700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc0051c0 0x7f23d40b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d8f9c700 1 -- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40c8d80 msgr2=0x7f23d40c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40c8d80 0x7f23d40c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d8f9c700 1 -- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc0051c0 msgr2=0x7f23d40b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc0051c0 0x7f23d40b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d8f9c700 1 -- 172.21.15.33:0/813379627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40c7fc0 con 0x7f23d41431d0 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d37fe700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40c8d80 0x7f23d40c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:06.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d3fff700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc0051c0 0x7f23d40b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d8f9c700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 0x7f23d40c9a40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f23cc016040 tx=0x7f23cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc004bb0 con 0x7f23d41431d0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc004d10 con 0x7f23d41431d0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d40c82a0 con 0x7f23d41431d0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc012650 con 0x7f23d41431d0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.115+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d4143640 con 0x7f23d41431d0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.116+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d404c380 con 0x7f23d41431d0 2023-10-06T05:22:06.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.117+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23cc01a070 con 0x7f23d41431d0 2023-10-06T05:22:06.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.118+0000 7f23bbfff700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23c0066f30 0x7f23c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.118+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f23cc085b40 con 0x7f23d41431d0 2023-10-06T05:22:06.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.119+0000 7f23d3fff700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23c0066f30 0x7f23c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.119+0000 7f23d3fff700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23c0066f30 0x7f23c00693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f23c4003c00 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.122+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23cc052ac0 con 0x7f23d41431d0 2023-10-06T05:22:06.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.375+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23d40c8a30 con 0x7f23d41431d0 2023-10-06T05:22:06.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.375+0000 7f23bbfff700 1 -- 172.21.15.33:0/813379627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f23cc025090 con 0x7f23d41431d0 2023-10-06T05:22:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:06.379 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:22:05.213347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23c0066f30 msgr2=0x7f23c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23c0066f30 0x7f23c00693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f23c4003c00 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 msgr2=0x7f23d40c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 0x7f23d40c9a40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f23cc016040 tx=0x7f23cc012000 comp rx=0 tx=0).stop 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 shutdown_connections 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23c0066f30 0x7f23c00693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d41431d0 0x7f23d40c9a40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc0051c0 0x7f23d40b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 --2- 172.21.15.33:0/813379627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d40c8d80 0x7f23d40c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 >> 172.21.15.33:0/813379627 conn(0x7f23d406cff0 msgr2=0x7f23d4156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 shutdown_connections 2023-10-06T05:22:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.378+0000 7f23d979d700 1 -- 172.21.15.33:0/813379627 wait complete. 2023-10-06T05:22:06.382 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 64 2023-10-06T05:22:06.397 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:22:06.398 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 50.31312847137451s 2023-10-06T05:22:06.398 INFO:tasks.cephfs.test_client_recovery:Reached active... 2023-10-06T05:22:06.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.581+0000 7f08e8b9d700 1 Processor -- start 2023-10-06T05:22:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.581+0000 7f08e8b9d700 1 -- start start 2023-10-06T05:22:06.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.582+0000 7f08e8b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f08e4159530 0x7f08e406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.582+0000 7f08e8b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f08e406df70 0x7f08e40afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.582+0000 7f08e8b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e4159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.582+0000 7f08e8b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e40b1e00 con 0x7f08e4159530 2023-10-06T05:22:06.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.582+0000 7f08e8b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e406f6b0 con 0x7f08e406df70 2023-10-06T05:22:06.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.582+0000 7f08e8b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e406e3e0 con 0x7f08e40b0310 2023-10-06T05:22:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f08e4159530 0x7f08e406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f08e4159530 0x7f08e406da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55900/0 (socket says 172.21.15.33:55900) 2023-10-06T05:22:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 -- 172.21.15.33:0/1303268794 learned_addr learned my addr 172.21.15.33:0/1303268794 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/1303268794 >> v2:172.21.15.167:3300/0 conn(0x7f08e406df70 0x7f08e40afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e3fff700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 -- 172.21.15.33:0/1303268794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 msgr2=0x7f08e4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e4159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 -- 172.21.15.33:0/1303268794 >> v2:172.21.15.167:3300/0 conn(0x7f08e406df70 msgr2=0x7f08e40afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 --2- 172.21.15.33:0/1303268794 >> v2:172.21.15.167:3300/0 conn(0x7f08e406df70 0x7f08e40afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 -- 172.21.15.33:0/1303268794 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e406f220 con 0x7f08e4159530 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/1303268794 >> v2:172.21.15.167:3300/0 conn(0x7f08e406df70 0x7f08e40afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e3fff700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.583+0000 7f08e37fe700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4159530 0x7f08e406da30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f08e40add90 tx=0x7f08d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ca65a20886a33a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e27fc700 1 -- 172.21.15.33:0/1303268794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400b820 con 0x7f08e4159530 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e27fc700 1 -- 172.21.15.33:0/1303268794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d400b980 con 0x7f08e4159530 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e27fc700 1 -- 172.21.15.33:0/1303268794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400bd20 con 0x7f08e4159530 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 -- 172.21.15.33:0/1303268794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4159530 msgr2=0x7f08e406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4159530 0x7f08e406da30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f08e40add90 tx=0x7f08d4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 -- 172.21.15.33:0/1303268794 shutdown_connections 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/1303268794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4159530 0x7f08e406da30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/1303268794 >> v2:172.21.15.167:3300/0 conn(0x7f08e406df70 0x7f08e40afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 -- 172.21.15.33:0/1303268794 >> 172.21.15.33:0/1303268794 conn(0x7f08e406c560 msgr2=0x7f08e406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.584+0000 7f08e8b9d700 1 -- 172.21.15.33:0/1303268794 shutdown_connections 2023-10-06T05:22:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.585+0000 7f08e8b9d700 1 -- 172.21.15.33:0/1303268794 wait complete. 2023-10-06T05:22:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.586+0000 7f08e8b9d700 1 Processor -- start 2023-10-06T05:22:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.586+0000 7f08e8b9d700 1 -- start start 2023-10-06T05:22:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e8b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 0x7f08e4147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e8b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e8b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4159530 0x7f08e40c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e8b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4148490 con 0x7f08e406df70 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e8b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40c7da0 con 0x7f08e4159530 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e8b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e415e5e0 con 0x7f08e40b0310 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 0x7f08e4147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 0x7f08e4147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55904/0 (socket says 172.21.15.33:55904) 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.587+0000 7f08e37fe700 1 -- 172.21.15.33:0/2569795957 learned_addr learned my addr 172.21.15.33:0/2569795957 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e3fff700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4159530 0x7f08e40c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e37fe700 1 -- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 msgr2=0x7f08e415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e37fe700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e37fe700 1 -- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4159530 msgr2=0x7f08e40c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e37fe700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4159530 0x7f08e40c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e37fe700 1 -- 172.21.15.33:0/2569795957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40094f0 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e3fff700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4159530 0x7f08e40c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e37fe700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 0x7f08e4147f50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f08d40094c0 tx=0x7f08d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d401a420 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d401a580 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d401a940 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40c7750 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.588+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40cbc60 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.589+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e40c75b0 con 0x7f08e406df70 2023-10-06T05:22:06.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.590+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08d402e450 con 0x7f08e406df70 2023-10-06T05:22:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.591+0000 7f08cb7fe700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d0066f30 0x7f08d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.591+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f08d4086210 con 0x7f08e406df70 2023-10-06T05:22:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.592+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d0066f30 0x7f08d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.592+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d0066f30 0x7f08d00693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006fd0 tx=0x7f08d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.595+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08e40c75b0 con 0x7f08e406df70 2023-10-06T05:22:06.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.847+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08e40c75b0 con 0x7f08e406df70 2023-10-06T05:22:06.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.847+0000 7f08cb7fe700 1 -- 172.21.15.33:0/2569795957 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f08e40c75b0 con 0x7f08e406df70 2023-10-06T05:22:06.850 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.849+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d0066f30 msgr2=0x7f08d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.849+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d0066f30 0x7f08d00693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006fd0 tx=0x7f08d8008040 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.849+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 msgr2=0x7f08e4147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.849+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 0x7f08e4147f50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f08d40094c0 tx=0x7f08d4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 shutdown_connections 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d0066f30 0x7f08d00693e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4159530 0x7f08e40c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e406df70 0x7f08e4147f50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 --2- 172.21.15.33:0/2569795957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08e40b0310 0x7f08e415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 >> 172.21.15.33:0/2569795957 conn(0x7f08e406c560 msgr2=0x7f08e4157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 shutdown_connections 2023-10-06T05:22:06.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:06.850+0000 7f08e8b9d700 1 -- 172.21.15.33:0/2569795957 wait complete. 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:06.870 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:06.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:06.871 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:22:06.913 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:06.913 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:06.913 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:06.913 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.913 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:06.914 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.914 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:06.914 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:06.914 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:22:06.914 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:06.915 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41834sec preferred_lft 41834sec 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:06.916 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.917 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:06.918 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:22:06.918 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:22:06.918 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:06.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:06.918 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:07.016 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:22:07.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:07.019 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:22:07.059 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:22:07.059 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:07.059 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:22:07.059 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:22:07.059 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:07.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:07.060 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:22:07.060 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:22:07.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:22:07.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:22:07.060 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:22:07.060 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:22:07.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:07.254 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:22:07.290 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:22:07.290 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:22:07.290 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:07.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:07.291 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:22:07.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:07.396 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:07.432 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:07.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:07.433 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:07.508 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:22:07.508 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:07.508 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:07.639 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:07.639 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:22:07.640 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:22:07.656 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:07.656 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:22:07.656 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:22:07.656 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:22:07.656 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:22:07.677 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:07.677 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:22:07.677 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:22:07.678 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:22:07.678 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:22:07.678 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:22:07.678 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:22:07.678 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:22:07.683 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:07.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:07.683 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:07.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:07.745 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec ls subdir/childfile) 2023-10-06T05:22:07.783 INFO:teuthology.orchestra.run.smithi167.stdout:subdir/childfile 2023-10-06T05:22:07.785 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:22:07.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:07.876 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:07.953 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:22:07.953 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:22:07.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:07.954 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:08.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:22:08.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:08.037 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:22:08.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.266+0000 7f42ca59d700 1 Processor -- start 2023-10-06T05:22:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.266+0000 7f42ca59d700 1 -- start start 2023-10-06T05:22:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42ca59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42c413faa0 0x7f42c413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42ca59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c406aad0 0x7f42c406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42ca59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c40c9cb0 con 0x7f42c40c7940 2023-10-06T05:22:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42ca59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40ca270 con 0x7f42c413faa0 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42ca59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c406b840 con 0x7f42c406aad0 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42c413faa0 0x7f42c413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42c413faa0 0x7f42c413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55910/0 (socket says 172.21.15.33:55910) 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 -- 172.21.15.33:0/225185873 learned_addr learned my addr 172.21.15.33:0/225185873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c8d9a700 1 --1- 172.21.15.33:0/225185873 >> v1:172.21.15.33:6789/0 conn(0x7f42c40c7940 0x7f42c40c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45256/0 (socket says 172.21.15.33:45256) 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c9d9c700 1 --2- 172.21.15.33:0/225185873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c406aad0 0x7f42c406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 -- 172.21.15.33:0/225185873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c406aad0 msgr2=0x7f42c406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 --2- 172.21.15.33:0/225185873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c406aad0 0x7f42c406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 -- 172.21.15.33:0/225185873 >> v1:172.21.15.33:6789/0 conn(0x7f42c40c7940 legacy=0x7f42c40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42c959b700 1 -- 172.21.15.33:0/225185873 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c406bb20 con 0x7f42c413faa0 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.268+0000 7f42bbfff700 1 -- 172.21.15.33:0/225185873 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592726373 0 0) 0x7f42c40c9cb0 con 0x7f42c40c7940 2023-10-06T05:22:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.269+0000 7f42c959b700 1 --2- 172.21.15.33:0/225185873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c413faa0 0x7f42c413fe70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f42c00099b0 tx=0x7f42c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc03424a03fc8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.269+0000 7f42bbfff700 1 -- 172.21.15.33:0/225185873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0009ea0 con 0x7f42c413faa0 2023-10-06T05:22:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.269+0000 7f42bbfff700 1 -- 172.21.15.33:0/225185873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c000b860 con 0x7f42c413faa0 2023-10-06T05:22:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.269+0000 7f42bbfff700 1 -- 172.21.15.33:0/225185873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000bc00 con 0x7f42c413faa0 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.269+0000 7f42ca59d700 1 -- 172.21.15.33:0/225185873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c413faa0 msgr2=0x7f42c413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.269+0000 7f42ca59d700 1 --2- 172.21.15.33:0/225185873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c413faa0 0x7f42c413fe70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f42c00099b0 tx=0x7f42c0009250 comp rx=0 tx=0).stop 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.270+0000 7f42ca59d700 1 -- 172.21.15.33:0/225185873 shutdown_connections 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.270+0000 7f42ca59d700 1 --2- 172.21.15.33:0/225185873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c413faa0 0x7f42c413fe70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.270+0000 7f42ca59d700 1 --2- 172.21.15.33:0/225185873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c406aad0 0x7f42c406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.270+0000 7f42ca59d700 1 -- 172.21.15.33:0/225185873 >> 172.21.15.33:0/225185873 conn(0x7f42c406d030 msgr2=0x7f42c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.270+0000 7f42ca59d700 1 -- 172.21.15.33:0/225185873 shutdown_connections 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.270+0000 7f42ca59d700 1 -- 172.21.15.33:0/225185873 wait complete. 2023-10-06T05:22:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.271+0000 7f42ca59d700 1 Processor -- start 2023-10-06T05:22:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.271+0000 7f42ca59d700 1 -- start start 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42ca59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 0x7f42c40b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42ca59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42c40c7940 0x7f42c413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42ca59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c413faa0 0x7f42c4158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42ca59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c416acb0 con 0x7f42c406aad0 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42ca59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40b1ab0 con 0x7f42c40c7940 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42ca59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c416a820 con 0x7f42c413faa0 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42c959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 0x7f42c40b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42c959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 0x7f42c40b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55914/0 (socket says 172.21.15.33:55914) 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.272+0000 7f42c959b700 1 -- 172.21.15.33:0/786686645 learned_addr learned my addr 172.21.15.33:0/786686645 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c8d9a700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42c40c7940 0x7f42c413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c9d9c700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c413faa0 0x7f42c4158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c959b700 1 -- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c413faa0 msgr2=0x7f42c4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c959b700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c413faa0 0x7f42c4158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c959b700 1 -- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42c40c7940 msgr2=0x7f42c413ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c959b700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42c40c7940 0x7f42c413ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c959b700 1 -- 172.21.15.33:0/786686645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00094f0 con 0x7f42c406aad0 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c9d9c700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c413faa0 0x7f42c4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42c959b700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 0x7f42c40b14d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f42c0006fd0 tx=0x7f42c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.273+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000bbe0 con 0x7f42c406aad0 2023-10-06T05:22:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.274+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0011420 con 0x7f42c406aad0 2023-10-06T05:22:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.274+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c4169ef0 con 0x7f42c406aad0 2023-10-06T05:22:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.274+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00117e0 con 0x7f42c406aad0 2023-10-06T05:22:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.274+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c416a4b0 con 0x7f42c406aad0 2023-10-06T05:22:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.275+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c416ab00 con 0x7f42c406aad0 2023-10-06T05:22:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.276+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42c0011b90 con 0x7f42c406aad0 2023-10-06T05:22:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.276+0000 7f42b97fa700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42a8066f30 0x7f42a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.277+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f42c0085ef0 con 0x7f42c406aad0 2023-10-06T05:22:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.277+0000 7f42c8d9a700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42a8066f30 0x7f42a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.277+0000 7f42c8d9a700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42a8066f30 0x7f42a80693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f42c415a040 tx=0x7f42b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.281+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42c0052f10 con 0x7f42c406aad0 2023-10-06T05:22:08.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.531+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42c404c380 con 0x7f42c406aad0 2023-10-06T05:22:08.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.531+0000 7f42b97fa700 1 -- 172.21.15.33:0/786686645 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f42c0056530 con 0x7f42c406aad0 2023-10-06T05:22:08.534 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:08.534 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:22:05.213347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:08.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42a8066f30 msgr2=0x7f42a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42a8066f30 0x7f42a80693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f42c415a040 tx=0x7f42b4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 msgr2=0x7f42c40b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 0x7f42c40b14d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f42c0006fd0 tx=0x7f42c0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 shutdown_connections 2023-10-06T05:22:08.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42a8066f30 0x7f42a80693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42c413faa0 0x7f42c4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42c406aad0 0x7f42c40b14d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 --2- 172.21.15.33:0/786686645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42c40c7940 0x7f42c413ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 >> 172.21.15.33:0/786686645 conn(0x7f42c406d030 msgr2=0x7f42c40cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 shutdown_connections 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.534+0000 7f42ca59d700 1 -- 172.21.15.33:0/786686645 wait complete. 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 64 2023-10-06T05:22:08.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.739+0000 7f05c259d700 1 Processor -- start 2023-10-06T05:22:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.739+0000 7f05c259d700 1 -- start start 2023-10-06T05:22:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.740+0000 7f05c259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 0x7f05bc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.740+0000 7f05c259d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f05bc0cf3b0 0x7f05bc1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.740+0000 7f05c259d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0cbd80 con 0x7f05bc0cc750 2023-10-06T05:22:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.740+0000 7f05c259d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc0af5f0 con 0x7f05bc0cf3b0 2023-10-06T05:22:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.740+0000 7f05c259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0af8d0 con 0x7f05bc0ccfc0 2023-10-06T05:22:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 0x7f05bc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 0x7f05bc0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:39510/0 (socket says 172.21.15.33:39510) 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 -- 172.21.15.33:0/2268419450 learned_addr learned my addr 172.21.15.33:0/2268419450 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c0d9a700 1 --2- 172.21.15.33:0/2268419450 >> v2:172.21.15.167:3300/0 conn(0x7f05bc0cf3b0 0x7f05bc1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 -- 172.21.15.33:0/2268419450 >> v2:172.21.15.167:3300/0 conn(0x7f05bc0cf3b0 msgr2=0x7f05bc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 --2- 172.21.15.33:0/2268419450 >> v2:172.21.15.167:3300/0 conn(0x7f05bc0cf3b0 0x7f05bc1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 -- 172.21.15.33:0/2268419450 >> v1:172.21.15.167:6789/0 conn(0x7f05bc0cc750 legacy=0x7f05bc0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c159b700 1 -- 172.21.15.33:0/2268419450 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0afbb0 con 0x7f05bc0ccfc0 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.741+0000 7f05c0d9a700 1 --2- 172.21.15.33:0/2268419450 >> v2:172.21.15.167:3300/0 conn(0x7f05bc0cf3b0 0x7f05bc1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.742+0000 7f05c159b700 1 --2- 172.21.15.33:0/2268419450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 0x7f05bc0cc210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05b80099b0 tx=0x7f05b8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3197bbe4fb03355d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.742+0000 7f05b3fff700 1 -- 172.21.15.33:0/2268419450 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8009ea0 con 0x7f05bc0ccfc0 2023-10-06T05:22:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.742+0000 7f05b3fff700 1 -- 172.21.15.33:0/2268419450 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b800b860 con 0x7f05bc0ccfc0 2023-10-06T05:22:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.742+0000 7f05b3fff700 1 -- 172.21.15.33:0/2268419450 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800bc00 con 0x7f05bc0ccfc0 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 -- 172.21.15.33:0/2268419450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 msgr2=0x7f05bc0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 --2- 172.21.15.33:0/2268419450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 0x7f05bc0cc210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05b80099b0 tx=0x7f05b8009250 comp rx=0 tx=0).stop 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 -- 172.21.15.33:0/2268419450 shutdown_connections 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 --2- 172.21.15.33:0/2268419450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0ccfc0 0x7f05bc0cc210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 --2- 172.21.15.33:0/2268419450 >> v2:172.21.15.167:3300/0 conn(0x7f05bc0cf3b0 0x7f05bc1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 -- 172.21.15.33:0/2268419450 >> 172.21.15.33:0/2268419450 conn(0x7f05bc06cff0 msgr2=0x7f05bc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 -- 172.21.15.33:0/2268419450 shutdown_connections 2023-10-06T05:22:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.743+0000 7f05c259d700 1 -- 172.21.15.33:0/2268419450 wait complete. 2023-10-06T05:22:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.745+0000 7f05c259d700 1 Processor -- start 2023-10-06T05:22:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.745+0000 7f05c259d700 1 -- start start 2023-10-06T05:22:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0cc750 0x7f05bc16a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05bc0ccfc0 0x7f05bc156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 0x7f05bc157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc154500 con 0x7f05bc141ac0 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b4250 con 0x7f05bc0ccfc0 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b60f0 con 0x7f05bc0cc750 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 0x7f05bc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c159b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0cc750 0x7f05bc16a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 0x7f05bc157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55916/0 (socket says 172.21.15.33:55916) 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 -- 172.21.15.33:0/3939938535 learned_addr learned my addr 172.21.15.33:0/3939938535 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c0d9a700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05bc0ccfc0 0x7f05bc156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 -- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0cc750 msgr2=0x7f05bc16a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0cc750 0x7f05bc16a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 -- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05bc0ccfc0 msgr2=0x7f05bc156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05bc0ccfc0 0x7f05bc156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.746+0000 7f05c1d9c700 1 -- 172.21.15.33:0/3939938535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009420 con 0x7f05bc141ac0 2023-10-06T05:22:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.747+0000 7f05c1d9c700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 0x7f05bc157370 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c910 tx=0x7f05b40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.747+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4012630 con 0x7f05bc141ac0 2023-10-06T05:22:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.747+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b4012790 con 0x7f05bc141ac0 2023-10-06T05:22:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.747+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b80094f0 con 0x7f05bc141ac0 2023-10-06T05:22:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.747+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4012b50 con 0x7f05bc141ac0 2023-10-06T05:22:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.747+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0b3ee0 con 0x7f05bc141ac0 2023-10-06T05:22:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.748+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc04c380 con 0x7f05bc141ac0 2023-10-06T05:22:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.749+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05b401b450 con 0x7f05bc141ac0 2023-10-06T05:22:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.750+0000 7f05b17fa700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05a0066f30 0x7f05a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.750+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f05b4084970 con 0x7f05bc141ac0 2023-10-06T05:22:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.750+0000 7f05c159b700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05a0066f30 0x7f05a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.751+0000 7f05c159b700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05a0066f30 0x7f05a00693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f05b80099b0 tx=0x7f05b8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.754+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05b40518f0 con 0x7f05bc141ac0 2023-10-06T05:22:08.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.956+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05bc0af0a0 con 0x7f05bc141ac0 2023-10-06T05:22:08.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.957+0000 7f05b17fa700 1 -- 172.21.15.33:0/3939938535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17253 (secure 0 0 0) 0x7f05b4054f10 con 0x7f05bc141ac0 2023-10-06T05:22:08.998 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:08.998 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":93,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:03.209427+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:54.719267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:55.512486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":5}]},{"pgid":"14.1a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/2184356465":"2023-10-07T05:21:13.104509+0000","172.21.15.33:0/590248546":"2023-10-06T06:22:03.159227+0000","172.21.15.33:6833/2184356465":"2023-10-07T05:21:13.104509+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:08.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05a0066f30 msgr2=0x7f05a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05a0066f30 0x7f05a00693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f05b80099b0 tx=0x7f05b8017000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 msgr2=0x7f05bc157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 0x7f05bc157370 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c910 tx=0x7f05b40016f0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 shutdown_connections 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f05a0066f30 0x7f05a00693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05bc141ac0 0x7f05bc157370 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05bc0cc750 0x7f05bc16a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 --2- 172.21.15.33:0/3939938535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05bc0ccfc0 0x7f05bc156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 >> 172.21.15.33:0/3939938535 conn(0x7f05bc06cff0 msgr2=0x7f05bc155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 shutdown_connections 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:08.959+0000 7f05c259d700 1 -- 172.21.15.33:0/3939938535 wait complete. 2023-10-06T05:22:09.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:20:54.719267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:20:55.512486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:22:09.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:22:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.200+0000 7f11ccdff700 1 Processor -- start 2023-10-06T05:22:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.200+0000 7f11ccdff700 1 -- start start 2023-10-06T05:22:09.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11ccdff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8069f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11ccdff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f11c806a460 0x7f11c80c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11ccdff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c80c75e0 con 0x7f11c8157740 2023-10-06T05:22:09.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11ccdff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c80c69d0 con 0x7f11c806a460 2023-10-06T05:22:09.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11ccdff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c6cb0 con 0x7f11c80ae910 2023-10-06T05:22:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11c659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11c8157740 0x7f11c80ae200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45272/0 (socket says 172.21.15.33:45272) 2023-10-06T05:22:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11c659c700 1 -- 172.21.15.33:0/2501760656 learned_addr learned my addr 172.21.15.33:0/2501760656 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:09.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/2501760656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11c6d9d700 1 --2- 172.21.15.33:0/2501760656 >> v2:172.21.15.167:3300/0 conn(0x7f11c806a460 0x7f11c80c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11c559a700 1 -- 172.21.15.33:0/2501760656 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070649136 0 0) 0x7f11c80c75e0 con 0x7f11c8157740 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.202+0000 7f11c559a700 1 -- 172.21.15.33:0/2501760656 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b0003570 con 0x7f11c8157740 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c5d9b700 1 -- 172.21.15.33:0/2501760656 >> v2:172.21.15.167:3300/0 conn(0x7f11c806a460 msgr2=0x7f11c80c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/2501760656 >> v2:172.21.15.167:3300/0 conn(0x7f11c806a460 0x7f11c80c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c5d9b700 1 -- 172.21.15.33:0/2501760656 >> v1:172.21.15.33:6789/0 conn(0x7f11c8157740 legacy=0x7f11c80ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c5d9b700 1 -- 172.21.15.33:0/2501760656 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80c6f90 con 0x7f11c80ae910 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/2501760656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8069f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0098e0 tx=0x7f11bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd23efd8d022d30d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c559a700 1 -- 172.21.15.33:0/2501760656 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc011630 con 0x7f11c80ae910 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c559a700 1 -- 172.21.15.33:0/2501760656 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc011790 con 0x7f11c80ae910 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.203+0000 7f11c559a700 1 -- 172.21.15.33:0/2501760656 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc011b30 con 0x7f11c80ae910 2023-10-06T05:22:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 -- 172.21.15.33:0/2501760656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 msgr2=0x7f11c8069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 --2- 172.21.15.33:0/2501760656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8069f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0098e0 tx=0x7f11bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 -- 172.21.15.33:0/2501760656 shutdown_connections 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 --2- 172.21.15.33:0/2501760656 >> v2:172.21.15.167:3300/0 conn(0x7f11c806a460 0x7f11c80c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 --2- 172.21.15.33:0/2501760656 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8069f20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 -- 172.21.15.33:0/2501760656 >> 172.21.15.33:0/2501760656 conn(0x7f11c806d030 msgr2=0x7f11c80cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 -- 172.21.15.33:0/2501760656 shutdown_connections 2023-10-06T05:22:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.204+0000 7f11ccdff700 1 -- 172.21.15.33:0/2501760656 wait complete. 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.205+0000 7f11ccdff700 1 Processor -- start 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.205+0000 7f11ccdff700 1 -- start start 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11ccdff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 0x7f11c80ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11ccdff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11ccdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11c8157740 0x7f11c816cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11ccdff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c1cd0 con 0x7f11c806a460 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11ccdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c1fb0 con 0x7f11c8157740 2023-10-06T05:22:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11ccdff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c2290 con 0x7f11c80ae910 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 0x7f11c80ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.206+0000 7f11c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 0x7f11c80ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55932/0 (socket says 172.21.15.33:55932) 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 -- 172.21.15.33:0/3509429706 learned_addr learned my addr 172.21.15.33:0/3509429706 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c6d9d700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11c8157740 0x7f11c816cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 -- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 msgr2=0x7f11c8156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 -- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11c8157740 msgr2=0x7f11c816cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11c8157740 0x7f11c816cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 -- 172.21.15.33:0/3509429706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc009420 con 0x7f11c806a460 2023-10-06T05:22:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c6d9d700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11c8157740 0x7f11c816cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11c659c700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 0x7f11c80ad700 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c590 tx=0x7f11b8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8005b30 con 0x7f11c806a460 2023-10-06T05:22:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8005cb0 con 0x7f11c806a460 2023-10-06T05:22:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.207+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8007320 con 0x7f11c806a460 2023-10-06T05:22:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.208+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c80c2570 con 0x7f11c806a460 2023-10-06T05:22:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.208+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c80adc40 con 0x7f11c806a460 2023-10-06T05:22:09.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.209+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c80c2790 con 0x7f11c806a460 2023-10-06T05:22:09.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.210+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11b801b3b0 con 0x7f11c806a460 2023-10-06T05:22:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.211+0000 7f11b67fc700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11a8066f30 0x7f11a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.211+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f11b8083de0 con 0x7f11c806a460 2023-10-06T05:22:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.211+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11a8066f30 0x7f11a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.212+0000 7f11c5d9b700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11a8066f30 0x7f11a80693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f11bc003080 tx=0x7f11bc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.214+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11b8050d60 con 0x7f11c806a460 2023-10-06T05:22:09.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.419+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11c80c1e10 con 0x7f11c806a460 2023-10-06T05:22:09.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.419+0000 7f11b67fc700 1 -- 172.21.15.33:0/3509429706 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11c80c1e10 con 0x7f11c806a460 2023-10-06T05:22:09.422 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:22:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.421+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11a8066f30 msgr2=0x7f11a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.421+0000 7f11ccdff700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11a8066f30 0x7f11a80693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f11bc003080 tx=0x7f11bc017000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.421+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 msgr2=0x7f11c80ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.421+0000 7f11ccdff700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 0x7f11c80ad700 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c590 tx=0x7f11b8007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 shutdown_connections 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11a8066f30 0x7f11a80693e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11c8157740 0x7f11c816cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c806a460 0x7f11c80ad700 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 --2- 172.21.15.33:0/3509429706 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11c80ae910 0x7f11c8156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 >> 172.21.15.33:0/3509429706 conn(0x7f11c806d030 msgr2=0x7f11c80af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 shutdown_connections 2023-10-06T05:22:09.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.422+0000 7f11ccdff700 1 -- 172.21.15.33:0/3509429706 wait complete. 2023-10-06T05:22:09.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:22:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.629+0000 7f877659d700 1 Processor -- start 2023-10-06T05:22:09.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.630+0000 7f877659d700 1 -- start start 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f877659d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87700cc7e0 0x7f87700af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f877659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700ad430 con 0x7f8770156370 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f877659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700afe20 con 0x7f87700b01f0 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f877659d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877006bc70 con 0x7f87700cc7e0 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f8774d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87700cc7e0 0x7f87700af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f8774d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87700cc7e0 0x7f87700af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55946/0 (socket says 172.21.15.33:55946) 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f8774d9a700 1 -- 172.21.15.33:0/2923483099 learned_addr learned my addr 172.21.15.33:0/2923483099 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:09.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.631+0000 7f877559b700 1 --1- 172.21.15.33:0/2923483099 >> v1:172.21.15.33:6789/0 conn(0x7f8770156370 0x7f87700c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45276/0 (socket says 172.21.15.33:45276) 2023-10-06T05:22:09.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8774d9a700 1 -- 172.21.15.33:0/2923483099 >> v1:172.21.15.167:6789/0 conn(0x7f87700b01f0 legacy=0x7f87700ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8774d9a700 1 -- 172.21.15.33:0/2923483099 >> v1:172.21.15.33:6789/0 conn(0x7f8770156370 legacy=0x7f87700c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8774d9a700 1 -- 172.21.15.33:0/2923483099 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700cccc0 con 0x7f87700cc7e0 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8774d9a700 1 --2- 172.21.15.33:0/2923483099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700af8e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f87600099b0 tx=0x7f8760009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0d701d6841dd67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8767fff700 1 -- 172.21.15.33:0/2923483099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000b860 con 0x7f87700cc7e0 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8767fff700 1 -- 172.21.15.33:0/2923483099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876000b9c0 con 0x7f87700cc7e0 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.632+0000 7f8767fff700 1 -- 172.21.15.33:0/2923483099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000bd60 con 0x7f87700cc7e0 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 -- 172.21.15.33:0/2923483099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 msgr2=0x7f87700af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 --2- 172.21.15.33:0/2923483099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700af8e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f87600099b0 tx=0x7f8760009250 comp rx=0 tx=0).stop 2023-10-06T05:22:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 -- 172.21.15.33:0/2923483099 shutdown_connections 2023-10-06T05:22:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 --2- 172.21.15.33:0/2923483099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700af8e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 -- 172.21.15.33:0/2923483099 >> 172.21.15.33:0/2923483099 conn(0x7f877006d030 msgr2=0x7f87700cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 -- 172.21.15.33:0/2923483099 shutdown_connections 2023-10-06T05:22:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.633+0000 7f877659d700 1 -- 172.21.15.33:0/2923483099 wait complete. 2023-10-06T05:22:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.634+0000 7f877659d700 1 Processor -- start 2023-10-06T05:22:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.635+0000 7f877659d700 1 -- start start 2023-10-06T05:22:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87700b01f0 0x7f8770151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700aa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770156370 0x7f87701695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87701580e0 con 0x7f87700cc7e0 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87701583c0 con 0x7f8770156370 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87701586a0 con 0x7f87700b01f0 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700aa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55956/0 (socket says 172.21.15.33:55956) 2023-10-06T05:22:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 -- 172.21.15.33:0/1172583103 learned_addr learned my addr 172.21.15.33:0/1172583103 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8775d9c700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770156370 0x7f87701695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877559b700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87700b01f0 0x7f8770151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 -- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87700b01f0 msgr2=0x7f8770151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87700b01f0 0x7f8770151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 -- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770156370 msgr2=0x7f87701695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770156370 0x7f87701695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f8774d9a700 1 -- 172.21.15.33:0/1172583103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600094f0 con 0x7f87700cc7e0 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.636+0000 7f877559b700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87700b01f0 0x7f8770151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.637+0000 7f8774d9a700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700aa270 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f877006d4d0 tx=0x7f8760017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.637+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760011ea0 con 0x7f87700cc7e0 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.637+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876001a630 con 0x7f87700cc7e0 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.637+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8770158980 con 0x7f87700cc7e0 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.637+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876001a9d0 con 0x7f87700cc7e0 2023-10-06T05:22:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.637+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8770154990 con 0x7f87700cc7e0 2023-10-06T05:22:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.638+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877004c380 con 0x7f87700cc7e0 2023-10-06T05:22:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.639+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f876002d440 con 0x7f87700cc7e0 2023-10-06T05:22:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.640+0000 7f87657fa700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8754066f30 0x7f87540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.640+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8760086330 con 0x7f87700cc7e0 2023-10-06T05:22:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.640+0000 7f877559b700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8754066f30 0x7f87540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.641+0000 7f877559b700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8754066f30 0x7f87540693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f876c007fd0 tx=0x7f876c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.644+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8760053350 con 0x7f87700cc7e0 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.847+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f877006f310 con 0x7f87700cc7e0 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.848+0000 7f87657fa700 1 -- 172.21.15.33:0/1172583103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v93) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8760056970 con 0x7f87700cc7e0 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8754066f30 msgr2=0x7f87540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8754066f30 0x7f87540693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f876c007fd0 tx=0x7f876c009040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 msgr2=0x7f87700aa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700aa270 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f877006d4d0 tx=0x7f8760017000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 shutdown_connections 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8754066f30 0x7f87540693e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770156370 0x7f87701695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87700b01f0 0x7f8770151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 --2- 172.21.15.33:0/1172583103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87700cc7e0 0x7f87700aa270 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 >> 172.21.15.33:0/1172583103 conn(0x7f877006d030 msgr2=0x7f87700af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 shutdown_connections 2023-10-06T05:22:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:09.850+0000 7f877659d700 1 -- 172.21.15.33:0/1172583103 wait complete. 2023-10-06T05:22:09.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:22:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.057+0000 7f9f5b59e700 1 Processor -- start 2023-10-06T05:22:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.058+0000 7f9f5b59e700 1 -- start start 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.059+0000 7f9f5b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c9490 0x7f9f5c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.059+0000 7f9f5b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0b0230 con 0x7f9f5c155a50 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.059+0000 7f9f5b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0b06b0 con 0x7f9f5c155ec0 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.059+0000 7f9f5b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c156300 con 0x7f9f5c0c9490 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.059+0000 7f9f59d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9f5c155a50 0x7f9f5c0af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50054/0 (socket says 172.21.15.33:50054) 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.059+0000 7f9f59d9b700 1 -- 172.21.15.33:0/4128095165 learned_addr learned my addr 172.21.15.33:0/4128095165 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5a59c700 1 --2- 172.21.15.33:0/4128095165 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c9490 0x7f9f5c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896191752 0 0) 0x7f9f5c0b0230 con 0x7f9f5c155a50 2023-10-06T05:22:10.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f44003570 con 0x7f9f5c155a50 2023-10-06T05:22:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 511881912 0 0) 0x7f9f44003570 con 0x7f9f5c155a50 2023-10-06T05:22:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c9490 msgr2=0x7f9f5c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 --2- 172.21.15.33:0/4128095165 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c9490 0x7f9f5c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 >> v1:172.21.15.167:6789/0 conn(0x7f9f5c155ec0 legacy=0x7f9f5c0afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0c99e0 con 0x7f9f5c155a50 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9f50001470 con 0x7f9f5c155a50 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 >> v1:172.21.15.33:6789/0 conn(0x7f9f5c155a50 legacy=0x7f9f5c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f44003a60 0x7f9f44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f44004810 0x7f9f44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c156300 con 0x7f9f44005210 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c1e0 con 0x7f9f44003a60 2023-10-06T05:22:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c3a0 con 0x7f9f44004810 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5ad9d700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.060+0000 7f9f5a59c700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f44003a60 0x7f9f44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f59d9b700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f44004810 0x7f9f44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 -- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f44004810 msgr2=0x7f9f44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f44004810 0x7f9f44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 -- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f44003a60 msgr2=0x7f9f44003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f44003a60 0x7f9f44003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 -- 172.21.15.33:0/4128095165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44003570 con 0x7f9f44005210 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 -- 172.21.15.33:0/4128095165 reap_dead start 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5ad9d700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f4400ba60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9f440057f0 tx=0x7f9f540016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da97055e776a5986 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540019e0 con 0x7f9f44005210 2023-10-06T05:22:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5400c8b0 con 0x7f9f44005210 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.061+0000 7f9f5959a700 1 -- 172.21.15.33:0/4128095165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400cc30 con 0x7f9f44005210 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 -- 172.21.15.33:0/4128095165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 msgr2=0x7f9f4400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f4400ba60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9f440057f0 tx=0x7f9f540016d0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 -- 172.21.15.33:0/4128095165 shutdown_connections 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/4128095165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f4400ba60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 -- 172.21.15.33:0/4128095165 >> 172.21.15.33:0/4128095165 conn(0x7f9f5c06d030 msgr2=0x7f9f5c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 -- 172.21.15.33:0/4128095165 shutdown_connections 2023-10-06T05:22:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.062+0000 7f9f5b59e700 1 -- 172.21.15.33:0/4128095165 wait complete. 2023-10-06T05:22:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.064+0000 7f9f5b59e700 1 Processor -- start 2023-10-06T05:22:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.064+0000 7f9f5b59e700 1 -- start start 2023-10-06T05:22:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f5c164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c4a70 0x7f9f5c0c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c167a10 0x7f9f5c0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c3ef0 con 0x7f9f44005210 2023-10-06T05:22:10.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c4ee0 con 0x7f9f5c167a10 2023-10-06T05:22:10.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c5150 con 0x7f9f5c0c4a70 2023-10-06T05:22:10.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f5c164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f5c164cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60976/0 (socket says 172.21.15.33:60976) 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 -- 172.21.15.33:0/1911100049 learned_addr learned my addr 172.21.15.33:0/1911100049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f59d9b700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c4a70 0x7f9f5c0c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5ad9d700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c167a10 0x7f9f5c0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 -- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c4a70 msgr2=0x7f9f5c0c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c4a70 0x7f9f5c0c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 -- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c167a10 msgr2=0x7f9f5c0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c167a10 0x7f9f5c0c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.065+0000 7f9f5a59c700 1 -- 172.21.15.33:0/1911100049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0c5430 con 0x7f9f44005210 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.066+0000 7f9f5a59c700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f5c164cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c011e50 tx=0x7f9f4c00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.066+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c016070 con 0x7f9f44005210 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.066+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c01b420 con 0x7f9f44005210 2023-10-06T05:22:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.066+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c01b7c0 con 0x7f9f44005210 2023-10-06T05:22:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.066+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c0c0f10 con 0x7f9f44005210 2023-10-06T05:22:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.066+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c0c14d0 con 0x7f9f44005210 2023-10-06T05:22:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.067+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c04c380 con 0x7f9f44005210 2023-10-06T05:22:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.068+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f4c01bb30 con 0x7f9f44005210 2023-10-06T05:22:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.070+0000 7f9f4a7fc700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.070+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9f4c087e70 con 0x7f9f44005210 2023-10-06T05:22:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.070+0000 7f9f59d9b700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.071+0000 7f9f59d9b700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001eb0 tx=0x7f9f50001da0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.074+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f4c054e70 con 0x7f9f44005210 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.277+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f5c0c9cc0 con 0x7f9f44005210 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.278+0000 7f9f4a7fc700 1 -- 172.21.15.33:0/1911100049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f4c058490 con 0x7f9f44005210 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f3c066f30 msgr2=0x7f9f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001eb0 tx=0x7f9f50001da0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 msgr2=0x7f9f5c164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f5c164cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c011e50 tx=0x7f9f4c00b430 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 shutdown_connections 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c167a10 0x7f9f5c0c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44005210 0x7f9f5c164cd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 --2- 172.21.15.33:0/1911100049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c4a70 0x7f9f5c0c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.280+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 >> 172.21.15.33:0/1911100049 conn(0x7f9f5c06d030 msgr2=0x7f9f5c0ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.281+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 shutdown_connections 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.281+0000 7f9f5b59e700 1 -- 172.21.15.33:0/1911100049 wait complete. 2023-10-06T05:22:10.301 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:22:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.488+0000 7f7000b9d700 1 Processor -- start 2023-10-06T05:22:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.489+0000 7f7000b9d700 1 -- start start 2023-10-06T05:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f7000b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc154f20 0x7f6ffc0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f7000b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ffc1552f0 0x7f6ffc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f7000b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc1586a0 con 0x7f6ffc0b11e0 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f7000b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc06bdf0 con 0x7f6ffc1552f0 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f7000b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc157ab0 con 0x7f6ffc154f20 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f6ffbfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6ffc0b11e0 0x7f6ffc15b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50060/0 (socket says 172.21.15.33:50060) 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f6ffbfff700 1 -- 172.21.15.33:0/2799263320 learned_addr learned my addr 172.21.15.33:0/2799263320 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f6ffaffd700 1 --2- 172.21.15.33:0/2799263320 >> v2:172.21.15.167:3300/0 conn(0x7f6ffc1552f0 0x7f6ffc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.490+0000 7f6ffb7fe700 1 --2- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc154f20 0x7f6ffc0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffa7fc700 1 -- 172.21.15.33:0/2799263320 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273612746 0 0) 0x7f6ffc1586a0 con 0x7f6ffc0b11e0 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffa7fc700 1 -- 172.21.15.33:0/2799263320 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe4003570 con 0x7f6ffc0b11e0 2023-10-06T05:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffaffd700 1 -- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc154f20 msgr2=0x7f6ffc0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffaffd700 1 --2- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc154f20 0x7f6ffc0ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffaffd700 1 -- 172.21.15.33:0/2799263320 >> v1:172.21.15.33:6789/0 conn(0x7f6ffc0b11e0 legacy=0x7f6ffc15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffaffd700 1 -- 172.21.15.33:0/2799263320 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc157d70 con 0x7f6ffc1552f0 2023-10-06T05:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.491+0000 7f6ffaffd700 1 --2- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1552f0 0x7f6ffc0cc7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00098e0 tx=0x7f6ff00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f64ef84d9f7abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f6ffa7fc700 1 -- 172.21.15.33:0/2799263320 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011630 con 0x7f6ffc1552f0 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f6ffa7fc700 1 -- 172.21.15.33:0/2799263320 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0011790 con 0x7f6ffc1552f0 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f6ffa7fc700 1 -- 172.21.15.33:0/2799263320 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011b30 con 0x7f6ffc1552f0 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 -- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1552f0 msgr2=0x7f6ffc0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 --2- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1552f0 0x7f6ffc0cc7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00098e0 tx=0x7f6ff00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 -- 172.21.15.33:0/2799263320 shutdown_connections 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 --2- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc154f20 0x7f6ffc0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 --2- 172.21.15.33:0/2799263320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1552f0 0x7f6ffc0cc7e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 -- 172.21.15.33:0/2799263320 >> 172.21.15.33:0/2799263320 conn(0x7f6ffc06d030 msgr2=0x7f6ffc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 -- 172.21.15.33:0/2799263320 shutdown_connections 2023-10-06T05:22:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.492+0000 7f7000b9d700 1 -- 172.21.15.33:0/2799263320 wait complete. 2023-10-06T05:22:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.494+0000 7f7000b9d700 1 Processor -- start 2023-10-06T05:22:10.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.494+0000 7f7000b9d700 1 -- start start 2023-10-06T05:22:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f7000b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc0b11e0 0x7f6ffc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f7000b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 0x7f6ffc0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f7000b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1520e0 0x7f6ffc152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f7000b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc15ba10 con 0x7f6ffc154f20 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f7000b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc152ac0 con 0x7f6ffc1520e0 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f7000b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc15bbf0 con 0x7f6ffc0b11e0 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f6ffaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 0x7f6ffc0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f6ffaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 0x7f6ffc0a60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60990/0 (socket says 172.21.15.33:60990) 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f6ffaffd700 1 -- 172.21.15.33:0/609254998 learned_addr learned my addr 172.21.15.33:0/609254998 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f6ffb7fe700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc0b11e0 0x7f6ffc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.495+0000 7f6ffbfff700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1520e0 0x7f6ffc152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffaffd700 1 -- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc0b11e0 msgr2=0x7f6ffc155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffaffd700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc0b11e0 0x7f6ffc155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffaffd700 1 -- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1520e0 msgr2=0x7f6ffc152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffaffd700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1520e0 0x7f6ffc152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffaffd700 1 -- 172.21.15.33:0/609254998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009420 con 0x7f6ffc154f20 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffbfff700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1520e0 0x7f6ffc152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6ffaffd700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 0x7f6ffc0a60e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc06d4d0 tx=0x7f6ff0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011b10 con 0x7f6ffc154f20 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0000cc0 con 0x7f6ffc154f20 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0025620 con 0x7f6ffc154f20 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc15bed0 con 0x7f6ffc154f20 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.496+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc15c490 con 0x7f6ffc154f20 2023-10-06T05:22:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.497+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc15c6b0 con 0x7f6ffc154f20 2023-10-06T05:22:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.498+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ff0025a10 con 0x7f6ffc154f20 2023-10-06T05:22:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.499+0000 7f6fe37fe700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fe8066f30 0x7f6fe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.500+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6ff0085fe0 con 0x7f6ffc154f20 2023-10-06T05:22:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.500+0000 7f6ffb7fe700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fe8066f30 0x7f6fe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.501+0000 7f6ffb7fe700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fe8066f30 0x7f6fe80693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009c80 tx=0x7f6fec0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.504+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ff0052fa0 con 0x7f6ffc154f20 2023-10-06T05:22:10.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.708+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ffc04c380 con 0x7f6ffc154f20 2023-10-06T05:22:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.708+0000 7f6fe37fe700 1 -- 172.21.15.33:0/609254998 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ff00565c0 con 0x7f6ffc154f20 2023-10-06T05:22:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:22:10.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fe8066f30 msgr2=0x7f6fe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fe8066f30 0x7f6fe80693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009c80 tx=0x7f6fec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 msgr2=0x7f6ffc0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 0x7f6ffc0a60e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc06d4d0 tx=0x7f6ff0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 shutdown_connections 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fe8066f30 0x7f6fe80693e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ffc1520e0 0x7f6ffc152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ffc0b11e0 0x7f6ffc155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 --2- 172.21.15.33:0/609254998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ffc154f20 0x7f6ffc0a60e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 >> 172.21.15.33:0/609254998 conn(0x7f6ffc06d030 msgr2=0x7f6ffc15d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 shutdown_connections 2023-10-06T05:22:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.711+0000 7f7000b9d700 1 -- 172.21.15.33:0/609254998 wait complete. 2023-10-06T05:22:10.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.916+0000 7f3309d9d700 1 Processor -- start 2023-10-06T05:22:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.916+0000 7f3309d9d700 1 -- start start 2023-10-06T05:22:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3309d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f330413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3309d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f330406aad0 0x7f330406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3309d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33040c9cb0 con 0x7f330413faa0 2023-10-06T05:22:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3309d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33040ca270 con 0x7f330406aad0 2023-10-06T05:22:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3309d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330406b840 con 0x7f33040c8050 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3303fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f330413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3303fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f330413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:52808/0 (socket says 172.21.15.33:52808) 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f3303fff700 1 -- 172.21.15.33:0/320063980 learned_addr learned my addr 172.21.15.33:0/320063980 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.918+0000 7f330959c700 1 --2- 172.21.15.33:0/320063980 >> v2:172.21.15.167:3300/0 conn(0x7f330406aad0 0x7f330406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f33037fe700 1 -- 172.21.15.33:0/320063980 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077600543 0 0) 0x7f33040c9cb0 con 0x7f330413faa0 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f33037fe700 1 -- 172.21.15.33:0/320063980 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f4003570 con 0x7f330413faa0 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f3303fff700 1 -- 172.21.15.33:0/320063980 >> v2:172.21.15.167:3300/0 conn(0x7f330406aad0 msgr2=0x7f330406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f3303fff700 1 --2- 172.21.15.33:0/320063980 >> v2:172.21.15.167:3300/0 conn(0x7f330406aad0 0x7f330406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f3303fff700 1 -- 172.21.15.33:0/320063980 >> v1:172.21.15.167:6789/0 conn(0x7f330413faa0 legacy=0x7f33040c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f3303fff700 1 -- 172.21.15.33:0/320063980 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330406bb20 con 0x7f33040c8050 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f330959c700 1 --2- 172.21.15.33:0/320063980 >> v2:172.21.15.167:3300/0 conn(0x7f330406aad0 0x7f330406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f3303fff700 1 --2- 172.21.15.33:0/320063980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f330413fe70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32f00099b0 tx=0x7f32f0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a34f23c8b45544c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f33037fe700 1 -- 172.21.15.33:0/320063980 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000f6b0 con 0x7f33040c8050 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f33037fe700 1 -- 172.21.15.33:0/320063980 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f000f810 con 0x7f33040c8050 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.919+0000 7f33037fe700 1 -- 172.21.15.33:0/320063980 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000fbb0 con 0x7f33040c8050 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 -- 172.21.15.33:0/320063980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 msgr2=0x7f330413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 --2- 172.21.15.33:0/320063980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f330413fe70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32f00099b0 tx=0x7f32f0009250 comp rx=0 tx=0).stop 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 -- 172.21.15.33:0/320063980 shutdown_connections 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 --2- 172.21.15.33:0/320063980 >> v2:172.21.15.167:3300/0 conn(0x7f330406aad0 0x7f330406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 --2- 172.21.15.33:0/320063980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f330413fe70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 -- 172.21.15.33:0/320063980 >> 172.21.15.33:0/320063980 conn(0x7f330406d030 msgr2=0x7f330406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 -- 172.21.15.33:0/320063980 shutdown_connections 2023-10-06T05:22:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.920+0000 7f3309d9d700 1 -- 172.21.15.33:0/320063980 wait complete. 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.922+0000 7f3309d9d700 1 Processor -- start 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.922+0000 7f3309d9d700 1 -- start start 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3309d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330406aad0 0x7f3304165cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3309d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f33040aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3309d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 0x7f3304155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3309d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33040c3190 con 0x7f330413faa0 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3309d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3304166300 con 0x7f330406aad0 2023-10-06T05:22:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3309d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33041665e0 con 0x7f33040c8050 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 0x7f3304155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 0x7f3304155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60996/0 (socket says 172.21.15.33:60996) 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 -- 172.21.15.33:0/2571166578 learned_addr learned my addr 172.21.15.33:0/2571166578 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3303fff700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f33040aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3308d9b700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330406aad0 0x7f3304165cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 -- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 msgr2=0x7f33040aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f33040aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 -- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330406aad0 msgr2=0x7f3304165cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330406aad0 0x7f3304165cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f330959c700 1 -- 172.21.15.33:0/2571166578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc009a20 con 0x7f330413faa0 2023-10-06T05:22:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.923+0000 7f3303fff700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f33040aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.924+0000 7f330959c700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 0x7f3304155cf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009fc0 tx=0x7f32fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.924+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00cd30 con 0x7f330413faa0 2023-10-06T05:22:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.924+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc00f040 con 0x7f330413faa0 2023-10-06T05:22:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.924+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0146d0 con 0x7f330413faa0 2023-10-06T05:22:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.924+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f00094f0 con 0x7f330413faa0 2023-10-06T05:22:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.924+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3304166ba0 con 0x7f330413faa0 2023-10-06T05:22:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.925+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330404c380 con 0x7f330413faa0 2023-10-06T05:22:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.926+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32fc003020 con 0x7f330413faa0 2023-10-06T05:22:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.927+0000 7f3300ff9700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32e8066f30 0x7f32e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.927+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f32fc0852b0 con 0x7f330413faa0 2023-10-06T05:22:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.927+0000 7f3308d9b700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32e8066f30 0x7f32e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.928+0000 7f3308d9b700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32e8066f30 0x7f32e80693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f32f80037e0 tx=0x7f32f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:10.931+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32fc052230 con 0x7f330413faa0 2023-10-06T05:22:11.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.180+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33040c9f00 con 0x7f330413faa0 2023-10-06T05:22:11.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.181+0000 7f3300ff9700 1 -- 172.21.15.33:0/2571166578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f32fc055850 con 0x7f330413faa0 2023-10-06T05:22:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:11.183 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:22:05.213347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32e8066f30 msgr2=0x7f32e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32e8066f30 0x7f32e80693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f32f80037e0 tx=0x7f32f8009040 comp rx=0 tx=0).stop 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 msgr2=0x7f3304155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 0x7f3304155cf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009fc0 tx=0x7f32fc00b040 comp rx=0 tx=0).stop 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 shutdown_connections 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f32e8066f30 0x7f32e80693e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f330413faa0 0x7f3304155cf0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330406aad0 0x7f3304165cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 --2- 172.21.15.33:0/2571166578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33040c8050 0x7f33040aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 >> 172.21.15.33:0/2571166578 conn(0x7f330406d030 msgr2=0x7f330406e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:11.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 shutdown_connections 2023-10-06T05:22:11.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.183+0000 7f3309d9d700 1 -- 172.21.15.33:0/2571166578 wait complete. 2023-10-06T05:22:11.187 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 64 2023-10-06T05:22:11.203 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.388+0000 7f4ab579d700 1 Processor -- start 2023-10-06T05:22:11.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.388+0000 7f4ab579d700 1 -- start start 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.389+0000 7f4ab579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.389+0000 7f4ab579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab00cbd80 con 0x7f4ab00ccfc0 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.389+0000 7f4ab579d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab00af5f0 con 0x7f4ab00cf3b0 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.389+0000 7f4ab579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00af8d0 con 0x7f4ab0159bb0 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.389+0000 7f4aaffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4ab00ccfc0 0x7f4ab00cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50068/0 (socket says 172.21.15.33:50068) 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.389+0000 7f4aaffff700 1 -- 172.21.15.33:0/1074045887 learned_addr learned my addr 172.21.15.33:0/1074045887 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4ab4f9c700 1 --2- 172.21.15.33:0/1074045887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568394630 0 0) 0x7f4ab00cbd80 con 0x7f4ab00ccfc0 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a98003570 con 0x7f4ab00ccfc0 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220079713 0 0) 0x7f4ab00af5f0 con 0x7f4ab00cf3b0 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab00cbd80 con 0x7f4ab00cf3b0 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/1074045887 >> v1:172.21.15.167:6789/0 conn(0x7f4ab00cf3b0 legacy=0x7f4ab01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/1074045887 >> v1:172.21.15.33:6789/0 conn(0x7f4ab00ccfc0 legacy=0x7f4ab00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3346561833 0 0) 0x7f4a98003570 con 0x7f4ab00ccfc0 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.390+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/1074045887 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00afbb0 con 0x7f4ab0159bb0 2023-10-06T05:22:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4ab4f9c700 1 --2- 172.21.15.33:0/1074045887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b0460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009ec0 tx=0x7f4aa8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39f7f84ca5f5502 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800e070 con 0x7f4ab0159bb0 2023-10-06T05:22:11.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa800b870 con 0x7f4ab0159bb0 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4aaeffd700 1 -- 172.21.15.33:0/1074045887 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0001470 con 0x7f4ab0159bb0 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4ab579d700 1 -- 172.21.15.33:0/1074045887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 msgr2=0x7f4ab00b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4ab579d700 1 --2- 172.21.15.33:0/1074045887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b0460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009ec0 tx=0x7f4aa8009780 comp rx=0 tx=0).stop 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4ab579d700 1 -- 172.21.15.33:0/1074045887 shutdown_connections 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4ab579d700 1 --2- 172.21.15.33:0/1074045887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b0460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.391+0000 7f4ab579d700 1 -- 172.21.15.33:0/1074045887 >> 172.21.15.33:0/1074045887 conn(0x7f4ab006cff0 msgr2=0x7f4ab006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.392+0000 7f4ab579d700 1 -- 172.21.15.33:0/1074045887 shutdown_connections 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.392+0000 7f4ab579d700 1 -- 172.21.15.33:0/1074045887 wait complete. 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.393+0000 7f4ab579d700 1 Processor -- start 2023-10-06T05:22:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.393+0000 7f4ab579d700 1 -- start start 2023-10-06T05:22:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab00ccfc0 0x7f4ab0144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 0x7f4ab0157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab0144740 con 0x7f4ab00c8a20 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab016dc90 con 0x7f4ab00ccfc0 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab016cda0 con 0x7f4ab0159bb0 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab4f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 0x7f4ab0157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab4f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 0x7f4ab0157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32780/0 (socket says 172.21.15.33:32780) 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/4268498713 learned_addr learned my addr 172.21.15.33:0/4268498713 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4aaffff700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab00ccfc0 0x7f4ab0144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.394+0000 7f4aaf7fe700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 msgr2=0x7f4ab00b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab4f9c700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab00ccfc0 msgr2=0x7f4ab0144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab4f9c700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab00ccfc0 0x7f4ab0144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab4f9c700 1 -- 172.21.15.33:0/4268498713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa8009a20 con 0x7f4ab00c8a20 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab4f9c700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 0x7f4ab0157e50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80097c0 tx=0x7f4aa8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800bee0 con 0x7f4ab00c8a20 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa801f410 con 0x7f4ab00c8a20 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa801f7f0 con 0x7f4ab00c8a20 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.395+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab016d080 con 0x7f4ab00c8a20 2023-10-06T05:22:11.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.396+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab016d640 con 0x7f4ab00c8a20 2023-10-06T05:22:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.397+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab004c380 con 0x7f4ab00c8a20 2023-10-06T05:22:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.397+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4aa801fbe0 con 0x7f4ab00c8a20 2023-10-06T05:22:11.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.399+0000 7f4a97fff700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a9c066f30 0x7f4a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.399+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4aa8086940 con 0x7f4ab00c8a20 2023-10-06T05:22:11.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.399+0000 7f4aaffff700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a9c066f30 0x7f4a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.400+0000 7f4aaffff700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a9c066f30 0x7f4a9c0693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00b5f30 tx=0x7f4aa0000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.403+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4aa80538c0 con 0x7f4ab00c8a20 2023-10-06T05:22:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.607+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ab00cc920 con 0x7f4ab00c8a20 2023-10-06T05:22:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.608+0000 7f4a97fff700 1 -- 172.21.15.33:0/4268498713 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17253 (secure 0 0 0) 0x7f4aa8056ee0 con 0x7f4ab00c8a20 2023-10-06T05:22:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:11.611 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":93,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:03.209427+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:54.719267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:55.512486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":5}]},{"pgid":"14.1a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/2184356465":"2023-10-07T05:21:13.104509+0000","172.21.15.33:0/590248546":"2023-10-06T06:22:03.159227+0000","172.21.15.33:6833/2184356465":"2023-10-07T05:21:13.104509+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a9c066f30 msgr2=0x7f4a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a9c066f30 0x7f4a9c0693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00b5f30 tx=0x7f4aa0000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 msgr2=0x7f4ab0157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 0x7f4ab0157e50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80097c0 tx=0x7f4aa8017000 comp rx=0 tx=0).stop 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 shutdown_connections 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a9c066f30 0x7f4a9c0693e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ab00c8a20 0x7f4ab0157e50 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab00ccfc0 0x7f4ab0144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 --2- 172.21.15.33:0/4268498713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab0159bb0 0x7f4ab00b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 >> 172.21.15.33:0/4268498713 conn(0x7f4ab006cff0 msgr2=0x7f4ab01524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 shutdown_connections 2023-10-06T05:22:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.611+0000 7f4ab579d700 1 -- 172.21.15.33:0/4268498713 wait complete. 2023-10-06T05:22:11.631 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:22:11.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.816+0000 7f5eebfff700 1 Processor -- start 2023-10-06T05:22:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.816+0000 7f5eebfff700 1 -- start start 2023-10-06T05:22:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eebfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 0x7f5eec13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eebfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec0c9cb0 con 0x7f5eec06aad0 2023-10-06T05:22:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eebfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec0ca270 con 0x7f5eec0c7940 2023-10-06T05:22:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eebfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec06b660 con 0x7f5eec13faa0 2023-10-06T05:22:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eeb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5eec06aad0 0x7f5eec06af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50076/0 (socket says 172.21.15.33:50076) 2023-10-06T05:22:11.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eeb7fe700 1 -- 172.21.15.33:0/2628699229 learned_addr learned my addr 172.21.15.33:0/2628699229 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.818+0000 7f5eeaffd700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 0x7f5eec13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972146481 0 0) 0x7f5eec0c9cb0 con 0x7f5eec06aad0 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed4003570 con 0x7f5eec06aad0 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012546726 0 0) 0x7f5eec0ca270 con 0x7f5eec0c7940 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eec0c9cb0 con 0x7f5eec0c7940 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880354604 0 0) 0x7f5ed4003570 con 0x7f5eec06aad0 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 msgr2=0x7f5eec13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 0x7f5eec13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 >> v1:172.21.15.167:6789/0 conn(0x7f5eec0c7940 legacy=0x7f5eec0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec06b840 con 0x7f5eec06aad0 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f5ee40017f0 con 0x7f5eec06aad0 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 >> v1:172.21.15.33:6789/0 conn(0x7f5eec06aad0 legacy=0x7f5eec06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed4003c20 0x7f5ed4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ed4004810 0x7f5ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 0x7f5ed4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec06b660 con 0x7f5ed4004810 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed400bbf0 con 0x7f5ed4003c20 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed400bdb0 con 0x7f5eec13faa0 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ed4004810 0x7f5ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.819+0000 7f5eeaffd700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed4003c20 0x7f5ed4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eeb7fe700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 0x7f5ed4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eea7fc700 1 -- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 msgr2=0x7f5ed4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5eec13faa0 0x7f5ed4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eea7fc700 1 -- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed4003c20 msgr2=0x7f5ed4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed4003c20 0x7f5ed4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eea7fc700 1 -- 172.21.15.33:0/2628699229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4003570 con 0x7f5ed4004810 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ed4004810 0x7f5ed4004c80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0001a10 tx=0x7f5ee0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0d21a4be049f27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000c5d0 con 0x7f5ed4004810 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000c730 con 0x7f5ed4004810 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.820+0000 7f5ee9ffb700 1 -- 172.21.15.33:0/2628699229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000cad0 con 0x7f5ed4004810 2023-10-06T05:22:11.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eebfff700 1 -- 172.21.15.33:0/2628699229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ed4004810 msgr2=0x7f5ed4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eebfff700 1 --2- 172.21.15.33:0/2628699229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ed4004810 0x7f5ed4004c80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0001a10 tx=0x7f5ee0007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eeb7fe700 1 -- 172.21.15.33:0/2628699229 reap_dead start 2023-10-06T05:22:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eebfff700 1 -- 172.21.15.33:0/2628699229 shutdown_connections 2023-10-06T05:22:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eebfff700 1 -- 172.21.15.33:0/2628699229 >> 172.21.15.33:0/2628699229 conn(0x7f5eec06d030 msgr2=0x7f5eec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eebfff700 1 -- 172.21.15.33:0/2628699229 shutdown_connections 2023-10-06T05:22:11.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.821+0000 7f5eebfff700 1 -- 172.21.15.33:0/2628699229 wait complete. 2023-10-06T05:22:11.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.822+0000 7f5eebfff700 1 Processor -- start 2023-10-06T05:22:11.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 -- start start 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ed4004810 0x7f5eec158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 0x7f5eec0c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eec13eb80 0x7f5eec0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13d390 con 0x7f5eec13e710 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13d670 con 0x7f5eec13eb80 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.823+0000 7f5eebfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13d950 con 0x7f5ed4004810 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 0x7f5eec0c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 0x7f5eec0c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32786/0 (socket says 172.21.15.33:32786) 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 -- 172.21.15.33:0/335640349 learned_addr learned my addr 172.21.15.33:0/335640349 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 -- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ed4004810 msgr2=0x7f5eec158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ed4004810 0x7f5eec158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eeb7fe700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eec13eb80 0x7f5eec0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 -- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eec13eb80 msgr2=0x7f5eec0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eec13eb80 0x7f5eec0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 -- 172.21.15.33:0/335640349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec13dc30 con 0x7f5eec13e710 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eeb7fe700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eec13eb80 0x7f5eec0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eea7fc700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 0x7f5eec0c4e00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c3c0 tx=0x7f5ee0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0001360 con 0x7f5eec13e710 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000ce40 con 0x7f5eec13e710 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000cad0 con 0x7f5eec13e710 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.824+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec13df10 con 0x7f5eec13e710 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.825+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec158f90 con 0x7f5eec13e710 2023-10-06T05:22:11.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.826+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec04c380 con 0x7f5eec13e710 2023-10-06T05:22:11.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.827+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ee0019070 con 0x7f5eec13e710 2023-10-06T05:22:11.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.828+0000 7f5ed2ffd700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ed8066f30 0x7f5ed80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.828+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5ee0085c20 con 0x7f5eec13e710 2023-10-06T05:22:11.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.828+0000 7f5eeaffd700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ed8066f30 0x7f5ed80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.829+0000 7f5eeaffd700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ed8066f30 0x7f5ed80693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5edc006f00 tx=0x7f5edc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:11.832+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ee0052ba0 con 0x7f5eec13e710 2023-10-06T05:22:12.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.080+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eec06bb20 con 0x7f5eec13e710 2023-10-06T05:22:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.080+0000 7f5ed2ffd700 1 -- 172.21.15.33:0/335640349 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f5ee0023020 con 0x7f5eec13e710 2023-10-06T05:22:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:12.083 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:22:05.213347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ed8066f30 msgr2=0x7f5ed80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ed8066f30 0x7f5ed80693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5edc006f00 tx=0x7f5edc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 msgr2=0x7f5eec0c4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 0x7f5eec0c4e00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c3c0 tx=0x7f5ee0015000 comp rx=0 tx=0).stop 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 shutdown_connections 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ed8066f30 0x7f5ed80693e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5eec13eb80 0x7f5eec0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ed4004810 0x7f5eec158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 --2- 172.21.15.33:0/335640349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eec13e710 0x7f5eec0c4e00 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 >> 172.21.15.33:0/335640349 conn(0x7f5eec06d030 msgr2=0x7f5eec0c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 shutdown_connections 2023-10-06T05:22:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.083+0000 7f5eebfff700 1 -- 172.21.15.33:0/335640349 wait complete. 2023-10-06T05:22:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 64 2023-10-06T05:22:12.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.287+0000 7f43da39d700 1 Processor -- start 2023-10-06T05:22:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.288+0000 7f43da39d700 1 -- start start 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.289+0000 7f43da39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d4158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.289+0000 7f43da39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f43d40adf50 0x7f43d40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.289+0000 7f43da39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d40c8e40 con 0x7f43d40c9810 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.289+0000 7f43da39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d40ae6c0 con 0x7f43d40adf50 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.289+0000 7f43da39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40ae9a0 con 0x7f43d4155140 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d9b9c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f43d40c9810 0x7f43d40699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55278/0 (socket says 172.21.15.33:55278) 2023-10-06T05:22:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d9b9c700 1 -- 172.21.15.33:0/4170714577 learned_addr learned my addr 172.21.15.33:0/4170714577 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:12.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.289+0000 7f43d939b700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d8b9a700 1 --2- 172.21.15.33:0/4170714577 >> v2:172.21.15.167:3300/0 conn(0x7f43d40adf50 0x7f43d40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43cbfff700 1 -- 172.21.15.33:0/4170714577 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475000188 0 0) 0x7f43d40c8e40 con 0x7f43d40c9810 2023-10-06T05:22:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43cbfff700 1 -- 172.21.15.33:0/4170714577 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c0003570 con 0x7f43d40c9810 2023-10-06T05:22:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d8b9a700 1 -- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 msgr2=0x7f43d4158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d8b9a700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d4158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d8b9a700 1 -- 172.21.15.33:0/4170714577 >> v1:172.21.15.167:6789/0 conn(0x7f43d40c9810 legacy=0x7f43d40699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d8b9a700 1 -- 172.21.15.33:0/4170714577 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40c8ad0 con 0x7f43d40adf50 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.290+0000 7f43d939b700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d4158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43d8b9a700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d40c92d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098e0 tx=0x7f43c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d76996bc4cab7570 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43cbfff700 1 -- 172.21.15.33:0/4170714577 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4011630 con 0x7f43d40adf50 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43cbfff700 1 -- 172.21.15.33:0/4170714577 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4011790 con 0x7f43d40adf50 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43cbfff700 1 -- 172.21.15.33:0/4170714577 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4011b30 con 0x7f43d40adf50 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43da39d700 1 -- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 msgr2=0x7f43d40c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43da39d700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d40c92d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098e0 tx=0x7f43c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43da39d700 1 -- 172.21.15.33:0/4170714577 shutdown_connections 2023-10-06T05:22:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43da39d700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d4158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43da39d700 1 --2- 172.21.15.33:0/4170714577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d40c92d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.291+0000 7f43da39d700 1 -- 172.21.15.33:0/4170714577 >> 172.21.15.33:0/4170714577 conn(0x7f43d406d030 msgr2=0x7f43d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.292+0000 7f43da39d700 1 -- 172.21.15.33:0/4170714577 shutdown_connections 2023-10-06T05:22:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.292+0000 7f43da39d700 1 -- 172.21.15.33:0/4170714577 wait complete. 2023-10-06T05:22:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.293+0000 7f43da39d700 1 Processor -- start 2023-10-06T05:22:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.293+0000 7f43da39d700 1 -- start start 2023-10-06T05:22:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43da39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d4150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43da39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 0x7f43d4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43da39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d40b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43da39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41507c0 con 0x7f43d40c9810 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43da39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41517a0 con 0x7f43d40adf50 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43da39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4151a80 con 0x7f43d4155140 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43d8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 0x7f43d4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43d8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 0x7f43d4155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32802/0 (socket says 172.21.15.33:32802) 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.294+0000 7f43d8b9a700 1 -- 172.21.15.33:0/3158499101 learned_addr learned my addr 172.21.15.33:0/3158499101 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d9b9c700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d40b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d939b700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d4150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d8b9a700 1 -- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 msgr2=0x7f43d40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d8b9a700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d40b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d8b9a700 1 -- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 msgr2=0x7f43d4150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d8b9a700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d4150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d8b9a700 1 -- 172.21.15.33:0/3158499101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009420 con 0x7f43d40c9810 2023-10-06T05:22:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43d8b9a700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 0x7f43d4155a50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f43d4063610 tx=0x7f43c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4011b10 con 0x7f43d40c9810 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40253f0 con 0x7f43d40c9810 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40257b0 con 0x7f43d40c9810 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.295+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c4009420 con 0x7f43d40c9810 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.296+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d4152040 con 0x7f43d40c9810 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.297+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d404c380 con 0x7f43d40c9810 2023-10-06T05:22:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.297+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43c4025ba0 con 0x7f43d40c9810 2023-10-06T05:22:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.298+0000 7f43c97fa700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b8066f30 0x7f43b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.298+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f43c4086150 con 0x7f43d40c9810 2023-10-06T05:22:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.299+0000 7f43d939b700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b8066f30 0x7f43b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.299+0000 7f43d939b700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b8066f30 0x7f43b80693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f43d00093f0 tx=0x7f43d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.303+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43c40530d0 con 0x7f43d40c9810 2023-10-06T05:22:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.549+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43d40c95f0 con 0x7f43d40c9810 2023-10-06T05:22:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.550+0000 7f43c97fa700 1 -- 172.21.15.33:0/3158499101 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f43c40566f0 con 0x7f43d40c9810 2023-10-06T05:22:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:12.553 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:56.546500+0000","modified":"2023-10-06T05:22:05.213347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6879":{"gid":6879,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":16,"addr":"172.21.15.33:6835/3442399006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3442399006},{"type":"v1","addr":"172.21.15.33:6835","nonce":3442399006}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b8066f30 msgr2=0x7f43b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b8066f30 0x7f43b80693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f43d00093f0 tx=0x7f43d0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 msgr2=0x7f43d4155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 0x7f43d4155a50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f43d4063610 tx=0x7f43c4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 shutdown_connections 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43b8066f30 0x7f43b80693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4155140 0x7f43d40b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d40adf50 0x7f43d4150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 --2- 172.21.15.33:0/3158499101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d40c9810 0x7f43d4155a50 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 >> 172.21.15.33:0/3158499101 conn(0x7f43d406d030 msgr2=0x7f43d40ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 shutdown_connections 2023-10-06T05:22:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.552+0000 7f43da39d700 1 -- 172.21.15.33:0/3158499101 wait complete. 2023-10-06T05:22:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 64 2023-10-06T05:22:12.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.757+0000 7f8c7579d700 1 Processor -- start 2023-10-06T05:22:12.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.757+0000 7f8c7579d700 1 -- start start 2023-10-06T05:22:12.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.758+0000 7f8c7579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c7920 0x7f8c70160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c7579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7006ab60 con 0x7f8c700c8b50 2023-10-06T05:22:12.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c7579d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7006ae40 con 0x7f8c70155af0 2023-10-06T05:22:12.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c7579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7006b7e0 con 0x7f8c700c7920 2023-10-06T05:22:12.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c6f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8c700c8b50 0x7f8c700c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50082/0 (socket says 172.21.15.33:50082) 2023-10-06T05:22:12.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c6f7fe700 1 -- 172.21.15.33:0/697096471 learned_addr learned my addr 172.21.15.33:0/697096471 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:12.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c74f9c700 1 --2- 172.21.15.33:0/697096471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c7920 0x7f8c70160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242430055 0 0) 0x7f8c7006ab60 con 0x7f8c700c8b50 2023-10-06T05:22:12.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c58004680 con 0x7f8c700c8b50 2023-10-06T05:22:12.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093376483 0 0) 0x7f8c7006ae40 con 0x7f8c70155af0 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.759+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c7006ab60 con 0x7f8c70155af0 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c74f9c700 1 -- 172.21.15.33:0/697096471 >> v1:172.21.15.167:6789/0 conn(0x7f8c70155af0 legacy=0x7f8c7006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3982891392 0 0) 0x7f8c58004680 con 0x7f8c700c8b50 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c74f9c700 1 -- 172.21.15.33:0/697096471 >> v1:172.21.15.33:6789/0 conn(0x7f8c700c8b50 legacy=0x7f8c700c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c74f9c700 1 -- 172.21.15.33:0/697096471 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7006b120 con 0x7f8c700c7920 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c74f9c700 1 --2- 172.21.15.33:0/697096471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c7920 0x7f8c70160c30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009e90 tx=0x7f8c68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6667a76ca4a51f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800e070 con 0x7f8c700c7920 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800b870 con 0x7f8c700c7920 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.760+0000 7f8c6effd700 1 -- 172.21.15.33:0/697096471 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64001470 con 0x7f8c700c7920 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 -- 172.21.15.33:0/697096471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c7920 msgr2=0x7f8c70160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 --2- 172.21.15.33:0/697096471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c7920 0x7f8c70160c30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009e90 tx=0x7f8c68009780 comp rx=0 tx=0).stop 2023-10-06T05:22:12.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 -- 172.21.15.33:0/697096471 shutdown_connections 2023-10-06T05:22:12.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 --2- 172.21.15.33:0/697096471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c7920 0x7f8c70160c30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 -- 172.21.15.33:0/697096471 >> 172.21.15.33:0/697096471 conn(0x7f8c7006d030 msgr2=0x7f8c7006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 -- 172.21.15.33:0/697096471 shutdown_connections 2023-10-06T05:22:12.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.761+0000 7f8c7579d700 1 -- 172.21.15.33:0/697096471 wait complete. 2023-10-06T05:22:12.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.762+0000 7f8c7579d700 1 Processor -- start 2023-10-06T05:22:12.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.762+0000 7f8c7579d700 1 -- start start 2023-10-06T05:22:12.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.763+0000 7f8c7579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c700c7920 0x7f8c70159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.763+0000 7f8c7579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c8b50 0x7f8c70153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.763+0000 7f8c7579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 0x7f8c700c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.763+0000 7f8c7579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70159660 con 0x7f8c70155af0 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.763+0000 7f8c7579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700af690 con 0x7f8c700c7920 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.763+0000 7f8c7579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700ac4d0 con 0x7f8c700c8b50 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 0x7f8c700c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 0x7f8c700c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32810/0 (socket says 172.21.15.33:32810) 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 -- 172.21.15.33:0/4026970191 learned_addr learned my addr 172.21.15.33:0/4026970191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c6f7fe700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c8b50 0x7f8c70153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c6ffff700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c700c7920 0x7f8c70159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 -- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c8b50 msgr2=0x7f8c70153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c8b50 0x7f8c70153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 -- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c700c7920 msgr2=0x7f8c70159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c700c7920 0x7f8c70159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 -- 172.21.15.33:0/4026970191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68009a20 con 0x7f8c70155af0 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c74f9c700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 0x7f8c700c52e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009de0 tx=0x7f8c68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.764+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68012cb0 con 0x7f8c70155af0 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.765+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68012e10 con 0x7f8c70155af0 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.765+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6801e680 con 0x7f8c70155af0 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.765+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c700ac7b0 con 0x7f8c70155af0 2023-10-06T05:22:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.765+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c700af2f0 con 0x7f8c70155af0 2023-10-06T05:22:12.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.766+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c700c9110 con 0x7f8c70155af0 2023-10-06T05:22:12.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.768+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c6801eab0 con 0x7f8c70155af0 2023-10-06T05:22:12.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.768+0000 7f8c57fff700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c5c066f30 0x7f8c5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.768+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8c680866e0 con 0x7f8c70155af0 2023-10-06T05:22:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.769+0000 7f8c6ffff700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c5c066f30 0x7f8c5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.769+0000 7f8c6ffff700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c5c066f30 0x7f8c5c0693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8c600037e0 tx=0x7f8c60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.773+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c68053660 con 0x7f8c70155af0 2023-10-06T05:22:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.980+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c700c92f0 con 0x7f8c70155af0 2023-10-06T05:22:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.981+0000 7f8c57fff700 1 -- 172.21.15.33:0/4026970191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17253 (secure 0 0 0) 0x7f8c68056c80 con 0x7f8c70155af0 2023-10-06T05:22:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:12.984 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":93,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:03.209427+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:54.719267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:55.512486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":5}]},{"pgid":"14.1a","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/2184356465":"2023-10-07T05:21:13.104509+0000","172.21.15.33:0/590248546":"2023-10-06T06:22:03.159227+0000","172.21.15.33:6833/2184356465":"2023-10-07T05:21:13.104509+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c5c066f30 msgr2=0x7f8c5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c5c066f30 0x7f8c5c0693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8c600037e0 tx=0x7f8c60009040 comp rx=0 tx=0).stop 2023-10-06T05:22:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 msgr2=0x7f8c700c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 0x7f8c700c52e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009de0 tx=0x7f8c68017000 comp rx=0 tx=0).stop 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 shutdown_connections 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c5c066f30 0x7f8c5c0693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c70155af0 0x7f8c700c52e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c700c7920 0x7f8c70159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 --2- 172.21.15.33:0/4026970191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c700c8b50 0x7f8c70153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 >> 172.21.15.33:0/4026970191 conn(0x7f8c7006d030 msgr2=0x7f8c700b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 shutdown_connections 2023-10-06T05:22:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:12.983+0000 7f8c7579d700 1 -- 172.21.15.33:0/4026970191 wait complete. 2023-10-06T05:22:13.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.190+0000 7f50ea59d700 1 Processor -- start 2023-10-06T05:22:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.190+0000 7f50ea59d700 1 -- start start 2023-10-06T05:22:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.190+0000 7f50ea59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50e4140d20 0x7f50e40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.190+0000 7f50ea59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e40c9210 0x7f50e40699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50ea59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e40c9660 con 0x7f50e4069f00 2023-10-06T05:22:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50ea59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e415bcb0 con 0x7f50e4140d20 2023-10-06T05:22:13.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50ea59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41410f0 con 0x7f50e40c9210 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50e8d9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e40c9210 0x7f50e40699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50e959b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50e4140d20 0x7f50e40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50e9d9c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f50e4069f00 0x7f50e40cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55292/0 (socket says 172.21.15.33:55292) 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.191+0000 7f50e9d9c700 1 -- 172.21.15.33:0/57466327 learned_addr learned my addr 172.21.15.33:0/57466327 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50dbfff700 1 -- 172.21.15.33:0/57466327 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84576191 0 0) 0x7f50e40c9660 con 0x7f50e4069f00 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50dbfff700 1 -- 172.21.15.33:0/57466327 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d0003570 con 0x7f50e4069f00 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50e959b700 1 -- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e40c9210 msgr2=0x7f50e40699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50e959b700 1 --2- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e40c9210 0x7f50e40699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50e959b700 1 -- 172.21.15.33:0/57466327 >> v1:172.21.15.167:6789/0 conn(0x7f50e4069f00 legacy=0x7f50e40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50e959b700 1 -- 172.21.15.33:0/57466327 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e406a300 con 0x7f50e4140d20 2023-10-06T05:22:13.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50e959b700 1 --2- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c8cd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f50e00098e0 tx=0x7f50e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61a0315d31ca2e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50dbfff700 1 -- 172.21.15.33:0/57466327 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0011630 con 0x7f50e4140d20 2023-10-06T05:22:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50dbfff700 1 -- 172.21.15.33:0/57466327 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0011790 con 0x7f50e4140d20 2023-10-06T05:22:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.192+0000 7f50dbfff700 1 -- 172.21.15.33:0/57466327 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0011b30 con 0x7f50e4140d20 2023-10-06T05:22:13.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 -- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 msgr2=0x7f50e40c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 --2- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c8cd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f50e00098e0 tx=0x7f50e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 -- 172.21.15.33:0/57466327 shutdown_connections 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 --2- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c8cd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 --2- 172.21.15.33:0/57466327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e40c9210 0x7f50e40699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 -- 172.21.15.33:0/57466327 >> 172.21.15.33:0/57466327 conn(0x7f50e406c560 msgr2=0x7f50e406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 -- 172.21.15.33:0/57466327 shutdown_connections 2023-10-06T05:22:13.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.193+0000 7f50ea59d700 1 -- 172.21.15.33:0/57466327 wait complete. 2023-10-06T05:22:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.195+0000 7f50ea59d700 1 Processor -- start 2023-10-06T05:22:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.195+0000 7f50ea59d700 1 -- start start 2023-10-06T05:22:13.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50ea59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e4069f00 0x7f50e40c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50ea59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 0x7f50e413f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50ea59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50ea59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4159230 con 0x7f50e40c9210 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50ea59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40c6430 con 0x7f50e4140d20 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50ea59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40c61e0 con 0x7f50e4069f00 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 0x7f50e413f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 0x7f50e413f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32824/0 (socket says 172.21.15.33:32824) 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 -- 172.21.15.33:0/4235153266 learned_addr learned my addr 172.21.15.33:0/4235153266 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e959b700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e4069f00 0x7f50e40c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e9d9c700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 -- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e4069f00 msgr2=0x7f50e40c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e4069f00 0x7f50e40c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 -- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 msgr2=0x7f50e40c1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.196+0000 7f50e8d9a700 1 -- 172.21.15.33:0/4235153266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009420 con 0x7f50e40c9210 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.197+0000 7f50e8d9a700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 0x7f50e413f030 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f50d400ecb0 tx=0x7f50d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.197+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400c850 con 0x7f50e40c9210 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.197+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d400c9b0 con 0x7f50e40c9210 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.197+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400cd50 con 0x7f50e40c9210 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.197+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e40c4750 con 0x7f50e40c9210 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.197+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e413f5c0 con 0x7f50e40c9210 2023-10-06T05:22:13.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.198+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e41593e0 con 0x7f50e40c9210 2023-10-06T05:22:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.199+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50d4013680 con 0x7f50e40c9210 2023-10-06T05:22:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.200+0000 7f50d97fa700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50c8066f30 0x7f50c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.200+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f50d4084a10 con 0x7f50e40c9210 2023-10-06T05:22:13.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.200+0000 7f50e959b700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50c8066f30 0x7f50c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.201+0000 7f50e959b700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50c8066f30 0x7f50c80693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f50e00091e0 tx=0x7f50e0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.205+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50e41593e0 con 0x7f50e40c9210 2023-10-06T05:22:13.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.452+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f50e41593e0 con 0x7f50e40c9210 2023-10-06T05:22:13.455 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:22:13.452+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:13.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.457+0000 7f50d97fa700 1 -- 172.21.15.33:0/4235153266 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7f50e41593e0 con 0x7f50e40c9210 2023-10-06T05:22:13.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50c8066f30 msgr2=0x7f50c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50c8066f30 0x7f50c80693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f50e00091e0 tx=0x7f50e0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 msgr2=0x7f50e413f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 0x7f50e413f030 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f50d400ecb0 tx=0x7f50d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 shutdown_connections 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50c8066f30 0x7f50c80693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e4140d20 0x7f50e40c1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.460+0000 7f50ea59d700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e4069f00 0x7f50e40c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.461+0000 7f50ea59d700 1 --2- 172.21.15.33:0/4235153266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e40c9210 0x7f50e413f030 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.461+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 >> 172.21.15.33:0/4235153266 conn(0x7f50e406c560 msgr2=0x7f50e40cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.461+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 shutdown_connections 2023-10-06T05:22:13.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.461+0000 7f50ea59d700 1 -- 172.21.15.33:0/4235153266 wait complete. 2023-10-06T05:22:13.465 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:13.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:22:13.523 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:22:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.680+0000 7fadff59e700 1 Processor -- start 2023-10-06T05:22:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.680+0000 7fadff59e700 1 -- start start 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadff59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fae000af540 0x7fae000af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadff59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae000cd5e0 0x7fae0015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadff59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0015d660 con 0x7fae0015a650 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadff59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae000ce890 con 0x7fae000cd5e0 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadff59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0015aac0 con 0x7fae000af540 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadfed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae000cd5e0 0x7fae0015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadfed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae000cd5e0 0x7fae0015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32852/0 (socket says 172.21.15.33:32852) 2023-10-06T05:22:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.681+0000 7fadfed9d700 1 -- 172.21.15.33:0/437970207 learned_addr learned my addr 172.21.15.33:0/437970207 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfe59c700 1 --2- 172.21.15.33:0/437970207 >> v2:172.21.15.167:3300/0 conn(0x7fae000af540 0x7fae000af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfed9d700 1 -- 172.21.15.33:0/437970207 >> v2:172.21.15.167:3300/0 conn(0x7fae000af540 msgr2=0x7fae000af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfed9d700 1 --2- 172.21.15.33:0/437970207 >> v2:172.21.15.167:3300/0 conn(0x7fae000af540 0x7fae000af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfed9d700 1 -- 172.21.15.33:0/437970207 >> v1:172.21.15.33:6789/0 conn(0x7fae0015a650 legacy=0x7fae000cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfed9d700 1 -- 172.21.15.33:0/437970207 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0015d890 con 0x7fae000cd5e0 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfd59a700 1 -- 172.21.15.33:0/437970207 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469953968 0 0) 0x7fae0015d660 con 0x7fae0015a650 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfe59c700 1 --2- 172.21.15.33:0/437970207 >> v2:172.21.15.167:3300/0 conn(0x7fae000af540 0x7fae000af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfed9d700 1 --2- 172.21.15.33:0/437970207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000cd5e0 0x7fae0015d120 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fae000c7530 tx=0x7fadf8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed75f42600361ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfd59a700 1 -- 172.21.15.33:0/437970207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf800e070 con 0x7fae000cd5e0 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfd59a700 1 -- 172.21.15.33:0/437970207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf800ba50 con 0x7fae000cd5e0 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.682+0000 7fadfd59a700 1 -- 172.21.15.33:0/437970207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80092e0 con 0x7fae000cd5e0 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 -- 172.21.15.33:0/437970207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000cd5e0 msgr2=0x7fae0015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 --2- 172.21.15.33:0/437970207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000cd5e0 0x7fae0015d120 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fae000c7530 tx=0x7fadf8009840 comp rx=0 tx=0).stop 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 -- 172.21.15.33:0/437970207 shutdown_connections 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 --2- 172.21.15.33:0/437970207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000cd5e0 0x7fae0015d120 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 --2- 172.21.15.33:0/437970207 >> v2:172.21.15.167:3300/0 conn(0x7fae000af540 0x7fae000af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 -- 172.21.15.33:0/437970207 >> 172.21.15.33:0/437970207 conn(0x7fae0006d9b0 msgr2=0x7fae0006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 -- 172.21.15.33:0/437970207 shutdown_connections 2023-10-06T05:22:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.683+0000 7fadff59e700 1 -- 172.21.15.33:0/437970207 wait complete. 2023-10-06T05:22:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.685+0000 7fadff59e700 1 Processor -- start 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.685+0000 7fadff59e700 1 -- start start 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadff59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae000af540 0x7fae000b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadff59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000cd5e0 0x7fae00156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadff59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 0x7fae000b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadff59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000b0df0 con 0x7fae0015a650 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadff59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000b4b40 con 0x7fae000cd5e0 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadff59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae000b01e0 con 0x7fae000af540 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 0x7fae000b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 0x7fae000b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32854/0 (socket says 172.21.15.33:32854) 2023-10-06T05:22:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 -- 172.21.15.33:0/1179379154 learned_addr learned my addr 172.21.15.33:0/1179379154 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfe59c700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae000af540 0x7fae000b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfdd9b700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000cd5e0 0x7fae00156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 -- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae000af540 msgr2=0x7fae000b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae000af540 0x7fae000b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 -- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000cd5e0 msgr2=0x7fae00156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000cd5e0 0x7fae00156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.686+0000 7fadfed9d700 1 -- 172.21.15.33:0/1179379154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8009ae0 con 0x7fae0015a650 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.687+0000 7fadfed9d700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 0x7fae000b1280 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fadf8012000 tx=0x7fadf8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.687+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8008d60 con 0x7fae0015a650 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.687+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf8025460 con 0x7fae0015a650 2023-10-06T05:22:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.687+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8025820 con 0x7fae0015a650 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.687+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae000b04c0 con 0x7fae0015a650 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.687+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae000b0a80 con 0x7fae0015a650 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.688+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae000b0f60 con 0x7fae0015a650 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.689+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadf8003040 con 0x7fae0015a650 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.690+0000 7fadee7fc700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fade0066f30 0x7fade00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.690+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fadf8085e80 con 0x7fae0015a650 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.691+0000 7fadfe59c700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fade0066f30 0x7fade00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.691+0000 7fadfe59c700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fade0066f30 0x7fade00693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fadf0006fd0 tx=0x7fadf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.694+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadf8052da0 con 0x7fae0015a650 2023-10-06T05:22:13.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:13.941+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fae0004c380 con 0x7fae0015a650 2023-10-06T05:22:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.459+0000 7fadee7fc700 1 -- 172.21.15.33:0/1179379154 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7fadf80563c0 con 0x7fae0015a650 2023-10-06T05:22:14.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fade0066f30 msgr2=0x7fade00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fade0066f30 0x7fade00693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fadf0006fd0 tx=0x7fadf0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 msgr2=0x7fae000b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 0x7fae000b1280 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fadf8012000 tx=0x7fadf8018000 comp rx=0 tx=0).stop 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 shutdown_connections 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fade0066f30 0x7fade00693e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae0015a650 0x7fae000b1280 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae000af540 0x7fae000b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 --2- 172.21.15.33:0/1179379154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae000cd5e0 0x7fae00156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 >> 172.21.15.33:0/1179379154 conn(0x7fae0006d9b0 msgr2=0x7fae000c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:14.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 shutdown_connections 2023-10-06T05:22:14.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.462+0000 7fadff59e700 1 -- 172.21.15.33:0/1179379154 wait complete. 2023-10-06T05:22:14.483 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:22:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.717+0000 7fd4703ff700 1 Processor -- start 2023-10-06T05:22:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.717+0000 7fd4703ff700 1 -- start start 2023-10-06T05:22:14.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd4703ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd46815ab90 0x7fd46806a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd4703ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd4680c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd4703ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4680c92c0 0x7fd4680cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd4703ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46806b840 con 0x7fd4680c92c0 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd4703ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4680af170 con 0x7fd46815ab90 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd4703ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680afa10 con 0x7fd4680c77f0 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd46e99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4680c92c0 0x7fd4680cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd46e99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4680c92c0 0x7fd4680cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32856/0 (socket says 172.21.15.33:32856) 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.718+0000 7fd46e99c700 1 -- 172.21.15.33:0/3137634058 learned_addr learned my addr 172.21.15.33:0/3137634058 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46e99c700 1 -- 172.21.15.33:0/3137634058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 msgr2=0x7fd4680c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46e99c700 1 --2- 172.21.15.33:0/3137634058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd4680c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46e99c700 1 -- 172.21.15.33:0/3137634058 >> v2:172.21.15.167:3300/0 conn(0x7fd46815ab90 msgr2=0x7fd46806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46e99c700 1 --2- 172.21.15.33:0/3137634058 >> v2:172.21.15.167:3300/0 conn(0x7fd46815ab90 0x7fd46806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46e99c700 1 -- 172.21.15.33:0/3137634058 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680c96c0 con 0x7fd4680c92c0 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46e99c700 1 --2- 172.21.15.33:0/3137634058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680cc320 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd4680cd420 tx=0x7fd460009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3901214d0e5ec507 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46d199700 1 -- 172.21.15.33:0/3137634058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000e070 con 0x7fd4680c92c0 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46d199700 1 -- 172.21.15.33:0/3137634058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46000ba50 con 0x7fd4680c92c0 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.719+0000 7fd46d199700 1 -- 172.21.15.33:0/3137634058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600092e0 con 0x7fd4680c92c0 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 -- 172.21.15.33:0/3137634058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 msgr2=0x7fd4680cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 --2- 172.21.15.33:0/3137634058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680cc320 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd4680cd420 tx=0x7fd460009840 comp rx=0 tx=0).stop 2023-10-06T05:22:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 -- 172.21.15.33:0/3137634058 shutdown_connections 2023-10-06T05:22:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 --2- 172.21.15.33:0/3137634058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680cc320 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 --2- 172.21.15.33:0/3137634058 >> v2:172.21.15.167:3300/0 conn(0x7fd46815ab90 0x7fd46806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 --2- 172.21.15.33:0/3137634058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd4680c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 -- 172.21.15.33:0/3137634058 >> 172.21.15.33:0/3137634058 conn(0x7fd46806e630 msgr2=0x7fd46806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 -- 172.21.15.33:0/3137634058 shutdown_connections 2023-10-06T05:22:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.720+0000 7fd4703ff700 1 -- 172.21.15.33:0/3137634058 wait complete. 2023-10-06T05:22:14.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.721+0000 7fd4703ff700 1 Processor -- start 2023-10-06T05:22:14.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 -- start start 2023-10-06T05:22:14.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd46814f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd46815ab90 0x7fd468169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46816ad70 con 0x7fd4680c92c0 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680c5730 con 0x7fd46815ab90 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.722+0000 7fd4703ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46816a240 con 0x7fd4680c77f0 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32860/0 (socket says 172.21.15.33:32860) 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 -- 172.21.15.33:0/2128413443 learned_addr learned my addr 172.21.15.33:0/2128413443 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46e19b700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd46814f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46e99c700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd46815ab90 0x7fd468169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 -- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 msgr2=0x7fd46814f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd46814f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 -- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd46815ab90 msgr2=0x7fd468169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd46815ab90 0x7fd468169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 -- 172.21.15.33:0/2128413443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009ae0 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46e99c700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd46815ab90 0x7fd468169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.723+0000 7fd46d99a700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680c4770 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd45800b810 tx=0x7fd458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.724+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45800f510 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.724+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45800f690 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.724+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45800fa50 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.724+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46816a4d0 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.724+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46816aa90 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.725+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4680c5500 con 0x7fd4680c92c0 2023-10-06T05:22:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.726+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd45801a400 con 0x7fd4680c92c0 2023-10-06T05:22:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.726+0000 7fd45e7fc700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd44c066f30 0x7fd44c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.727+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd458084410 con 0x7fd4680c92c0 2023-10-06T05:22:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.727+0000 7fd46e19b700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd44c066f30 0x7fd44c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.728+0000 7fd46e19b700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd44c066f30 0x7fd44c0693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd4680aebe0 tx=0x7fd4640090f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.731+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4580513b0 con 0x7fd4680c92c0 2023-10-06T05:22:14.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:14.937+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4680718b0 con 0x7fd4680c92c0 2023-10-06T05:22:15.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.794+0000 7fd45e7fc700 1 -- 172.21.15.33:0/2128413443 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4580549d0 con 0x7fd4680c92c0 2023-10-06T05:22:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd44c066f30 msgr2=0x7fd44c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd44c066f30 0x7fd44c0693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd4680aebe0 tx=0x7fd4640090f0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 msgr2=0x7fd4680c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680c4770 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd45800b810 tx=0x7fd458009250 comp rx=0 tx=0).stop 2023-10-06T05:22:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 shutdown_connections 2023-10-06T05:22:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd44c066f30 0x7fd44c0693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd46815ab90 0x7fd468169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4680c77f0 0x7fd46814f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 --2- 172.21.15.33:0/2128413443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4680c92c0 0x7fd4680c4770 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 >> 172.21.15.33:0/2128413443 conn(0x7fd46806e630 msgr2=0x7fd46806ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.800+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 shutdown_connections 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:15.801+0000 7fd4703ff700 1 -- 172.21.15.33:0/2128413443 wait complete. 2023-10-06T05:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:22:15.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:22:16.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.020+0000 7f9fae7ff700 1 Processor -- start 2023-10-06T05:22:16.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.021+0000 7f9fae7ff700 1 -- start start 2023-10-06T05:22:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fae7ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9fa8150ac0 0x7f9fa806be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fae7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fa80cc340 0x7f9fa80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fae7ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa806b080 con 0x7f9fa80ca1b0 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fae7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa80ccd00 con 0x7f9fa80cc340 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fae7ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa80cacc0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fa77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fa80cc340 0x7f9fa80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fa77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fa80cc340 0x7f9fa80cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32864/0 (socket says 172.21.15.33:32864) 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.022+0000 7f9fa77fe700 1 -- 172.21.15.33:0/2169454209 learned_addr learned my addr 172.21.15.33:0/2169454209 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa7fff700 1 --2- 172.21.15.33:0/2169454209 >> v2:172.21.15.167:3300/0 conn(0x7f9fa8150ac0 0x7f9fa806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa77fe700 1 -- 172.21.15.33:0/2169454209 >> v2:172.21.15.167:3300/0 conn(0x7f9fa8150ac0 msgr2=0x7f9fa806be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa77fe700 1 --2- 172.21.15.33:0/2169454209 >> v2:172.21.15.167:3300/0 conn(0x7f9fa8150ac0 0x7f9fa806be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa77fe700 1 -- 172.21.15.33:0/2169454209 >> v1:172.21.15.167:6789/0 conn(0x7f9fa80ca1b0 legacy=0x7f9fa80ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa77fe700 1 -- 172.21.15.33:0/2169454209 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa806b360 con 0x7f9fa80cc340 2023-10-06T05:22:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa77fe700 1 --2- 172.21.15.33:0/2169454209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa80cc340 0x7f9fa80cc7c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9fa806fd00 tx=0x7f9f9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e07c01b458d11e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa6ffd700 1 -- 172.21.15.33:0/2169454209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00b820 con 0x7f9fa80cc340 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa6ffd700 1 -- 172.21.15.33:0/2169454209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c00b980 con 0x7f9fa80cc340 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.023+0000 7f9fa6ffd700 1 -- 172.21.15.33:0/2169454209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00bd20 con 0x7f9fa80cc340 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 -- 172.21.15.33:0/2169454209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa80cc340 msgr2=0x7f9fa80cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/2169454209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa80cc340 0x7f9fa80cc7c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9fa806fd00 tx=0x7f9f9c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 -- 172.21.15.33:0/2169454209 shutdown_connections 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/2169454209 >> v2:172.21.15.167:3300/0 conn(0x7f9fa8150ac0 0x7f9fa806be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/2169454209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa80cc340 0x7f9fa80cc7c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 -- 172.21.15.33:0/2169454209 >> 172.21.15.33:0/2169454209 conn(0x7f9fa806e630 msgr2=0x7f9fa806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 -- 172.21.15.33:0/2169454209 shutdown_connections 2023-10-06T05:22:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.024+0000 7f9fae7ff700 1 -- 172.21.15.33:0/2169454209 wait complete. 2023-10-06T05:22:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.026+0000 7f9fae7ff700 1 Processor -- start 2023-10-06T05:22:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.026+0000 7f9fae7ff700 1 -- start start 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fae7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fa80ca1b0 0x7f9fa8146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fae7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fa80cc340 0x7f9fa80c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fae7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 0x7f9fa80c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fae7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80c4be0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fae7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8147100 con 0x7f9fa80ca1b0 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fae7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80b1d50 con 0x7f9fa80cc340 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 0x7f9fa80c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 0x7f9fa80c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32878/0 (socket says 172.21.15.33:32878) 2023-10-06T05:22:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 -- 172.21.15.33:0/525566826 learned_addr learned my addr 172.21.15.33:0/525566826 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fa7fff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fa80ca1b0 0x7f9fa8146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9fa77fe700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fa80cc340 0x7f9fa80c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 -- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fa80cc340 msgr2=0x7f9fa80c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fa80cc340 0x7f9fa80c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 -- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fa80ca1b0 msgr2=0x7f9fa8146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fa80ca1b0 0x7f9fa8146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.027+0000 7f9facd9c700 1 -- 172.21.15.33:0/525566826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0094f0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.028+0000 7f9facd9c700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 0x7f9fa80c39a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c840 tx=0x7f9fa0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.028+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0012630 con 0x7f9fa8150ac0 2023-10-06T05:22:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.028+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa00127b0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.028+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa80c3fd0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.028+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0012b70 con 0x7f9fa8150ac0 2023-10-06T05:22:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.028+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa80c4590 con 0x7f9fa8150ac0 2023-10-06T05:22:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.029+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa80c4d90 con 0x7f9fa8150ac0 2023-10-06T05:22:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.030+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fa001a3a0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.031+0000 7f9f8ffff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f94066f30 0x7f9f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.031+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f9fa00847a0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.031+0000 7f9fa7fff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f94066f30 0x7f9f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.032+0000 7f9fa7fff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f94066f30 0x7f9f940693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9f98006fd0 tx=0x7f9f98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.035+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fa0051880 con 0x7f9fa8150ac0 2023-10-06T05:22:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.237+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9fa80718b0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.799+0000 7f9f8ffff700 1 -- 172.21.15.33:0/525566826 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9fa0054ea0 con 0x7f9fa8150ac0 2023-10-06T05:22:16.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.806+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f94066f30 msgr2=0x7f9f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.806+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f94066f30 0x7f9f940693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9f98006fd0 tx=0x7f9f98008040 comp rx=0 tx=0).stop 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.806+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 msgr2=0x7f9fa80c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.806+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 0x7f9fa80c39a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c840 tx=0x7f9fa0001670 comp rx=0 tx=0).stop 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 shutdown_connections 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f94066f30 0x7f9f940693e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa8150ac0 0x7f9fa80c39a0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fa80ca1b0 0x7f9fa8146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 --2- 172.21.15.33:0/525566826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fa80cc340 0x7f9fa80c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 >> 172.21.15.33:0/525566826 conn(0x7f9fa806e630 msgr2=0x7f9fa80ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 shutdown_connections 2023-10-06T05:22:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:16.807+0000 7f9fae7ff700 1 -- 172.21.15.33:0/525566826 wait complete. 2023-10-06T05:22:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:22:16.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:22:17.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.033+0000 7f2da3fff700 1 Processor -- start 2023-10-06T05:22:17.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.034+0000 7f2da3fff700 1 -- start start 2023-10-06T05:22:17.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.035+0000 7f2da3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2da406cb80 0x7f2da415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.035+0000 7f2da3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da415d3a0 con 0x7f2da40aff90 2023-10-06T05:22:17.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.035+0000 7f2da3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da40b0470 con 0x7f2da406c7b0 2023-10-06T05:22:17.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.035+0000 7f2da3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da40686c0 con 0x7f2da406cb80 2023-10-06T05:22:17.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.035+0000 7f2da27fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2da40aff90 0x7f2da40a9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50130/0 (socket says 172.21.15.33:50130) 2023-10-06T05:22:17.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.035+0000 7f2da27fc700 1 -- 172.21.15.33:0/3251923153 learned_addr learned my addr 172.21.15.33:0/3251923153 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:17.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da37fe700 1 --2- 172.21.15.33:0/3251923153 >> v2:172.21.15.167:3300/0 conn(0x7f2da406cb80 0x7f2da415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175370208 0 0) 0x7f2da415d3a0 con 0x7f2da40aff90 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d8c003570 con 0x7f2da40aff90 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963788643 0 0) 0x7f2d8c003570 con 0x7f2da40aff90 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 >> v2:172.21.15.167:3300/0 conn(0x7f2da406cb80 msgr2=0x7f2da415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 --2- 172.21.15.33:0/3251923153 >> v2:172.21.15.167:3300/0 conn(0x7f2da406cb80 0x7f2da415d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 >> v1:172.21.15.167:6789/0 conn(0x7f2da406c7b0 legacy=0x7f2da4067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40b0750 con 0x7f2da40aff90 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f2d98001470 con 0x7f2da40aff90 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 >> v1:172.21.15.33:6789/0 conn(0x7f2da40aff90 legacy=0x7f2da40a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d8c003ab0 0x7f2d8c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c004810 0x7f2d8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d8c005210 0x7f2d8c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40686c0 con 0x7f2d8c003ab0 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c00c130 con 0x7f2d8c005210 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c00c2f0 con 0x7f2d8c004810 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.036+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d8c003ab0 0x7f2d8c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da37fe700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d8c005210 0x7f2d8c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da27fc700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c004810 0x7f2d8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da2ffd700 1 -- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c004810 msgr2=0x7f2d8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c004810 0x7f2d8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da2ffd700 1 -- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d8c005210 msgr2=0x7f2d8c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d8c005210 0x7f2d8c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da2ffd700 1 -- 172.21.15.33:0/3251923153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c003570 con 0x7f2d8c003ab0 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da37fe700 1 -- 172.21.15.33:0/3251923153 reap_dead start 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da27fc700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c004810 0x7f2d8c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d8c003ab0 0x7f2d8c003ec0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2d940018d0 tx=0x7f2d94007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2af7d39d3d259c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94001290 con 0x7f2d8c003ab0 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94012420 con 0x7f2d8c003ab0 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.037+0000 7f2da1ffb700 1 -- 172.21.15.33:0/3251923153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940127e0 con 0x7f2d8c003ab0 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 -- 172.21.15.33:0/3251923153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d8c003ab0 msgr2=0x7f2d8c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d8c003ab0 0x7f2d8c003ec0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2d940018d0 tx=0x7f2d94007d80 comp rx=0 tx=0).stop 2023-10-06T05:22:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 -- 172.21.15.33:0/3251923153 shutdown_connections 2023-10-06T05:22:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3251923153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d8c003ab0 0x7f2d8c003ec0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 -- 172.21.15.33:0/3251923153 >> 172.21.15.33:0/3251923153 conn(0x7f2da406d670 msgr2=0x7f2da406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 -- 172.21.15.33:0/3251923153 shutdown_connections 2023-10-06T05:22:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.038+0000 7f2da3fff700 1 -- 172.21.15.33:0/3251923153 wait complete. 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.039+0000 7f2da3fff700 1 Processor -- start 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.040+0000 7f2da3fff700 1 -- start start 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.040+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c003ab0 0x7f2da414c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.040+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 0x7f2da414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.040+0000 7f2da3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40ab5d0 0x7f2da40aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40abf80 con 0x7f2da414f1d0 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da414e590 con 0x7f2da40ab5d0 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da414b3d0 con 0x7f2d8c003ab0 2023-10-06T05:22:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 0x7f2da414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 0x7f2da414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32882/0 (socket says 172.21.15.33:32882) 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 -- 172.21.15.33:0/3621448100 learned_addr learned my addr 172.21.15.33:0/3621448100 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c003ab0 0x7f2da414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 -- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c003ab0 msgr2=0x7f2da414c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c003ab0 0x7f2da414c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 -- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40ab5d0 msgr2=0x7f2da40aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40ab5d0 0x7f2da40aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 -- 172.21.15.33:0/3621448100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da414b6b0 con 0x7f2da414f1d0 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c003ab0 0x7f2da414c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.041+0000 7f2da27fc700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 0x7f2da414e050 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2d98000fd0 tx=0x7f2d98001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.042+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012590 con 0x7f2da414f1d0 2023-10-06T05:22:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.042+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d980126f0 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.042+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012a70 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.042+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da414b990 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.042+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da414bf50 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.043+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da404c380 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.044+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d9801d460 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.045+0000 7f2d8affd700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d90066f60 0x7f2d90069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.045+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2d98084290 con 0x7f2da414f1d0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.045+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d90066f60 0x7f2d90069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.046+0000 7f2da2ffd700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d90066f60 0x7f2d90069410 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001060 tx=0x7f2d94015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.049+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d98051620 con 0x7f2da414f1d0 2023-10-06T05:22:17.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.253+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2da40b0a30 con 0x7f2da414f1d0 2023-10-06T05:22:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.257+0000 7f2d8affd700 1 -- 172.21.15.33:0/3621448100 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d98054c40 con 0x7f2da414f1d0 2023-10-06T05:22:17.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d90066f60 msgr2=0x7f2d90069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d90066f60 0x7f2d90069410 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001060 tx=0x7f2d94015000 comp rx=0 tx=0).stop 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 msgr2=0x7f2da414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 0x7f2da414e050 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2d98000fd0 tx=0x7f2d98001cc0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 shutdown_connections 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d90066f60 0x7f2d90069410 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40ab5d0 0x7f2da40aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d8c003ab0 0x7f2da414c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 --2- 172.21.15.33:0/3621448100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da414f1d0 0x7f2da414e050 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 >> 172.21.15.33:0/3621448100 conn(0x7f2da406d670 msgr2=0x7f2da4158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 shutdown_connections 2023-10-06T05:22:17.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.260+0000 7f2da3fff700 1 -- 172.21.15.33:0/3621448100 wait complete. 2023-10-06T05:22:17.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync' 2023-10-06T05:22:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.464+0000 7f27fbfff700 1 Processor -- start 2023-10-06T05:22:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.464+0000 7f27fbfff700 1 -- start start 2023-10-06T05:22:17.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc158700 0x7f27fc0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27fc0699c0 0x7f27fc069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27fc0ae490 con 0x7f27fc155140 2023-10-06T05:22:17.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fbfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc06a380 con 0x7f27fc0699c0 2023-10-06T05:22:17.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0c8e40 con 0x7f27fc158700 2023-10-06T05:22:17.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fa7fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc158700 0x7f27fc0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27faffd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f27fc155140 0x7f27fc0c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55332/0 (socket says 172.21.15.33:55332) 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27faffd700 1 -- 172.21.15.33:0/3329030087 learned_addr learned my addr 172.21.15.33:0/3329030087 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27fb7fe700 1 --2- 172.21.15.33:0/3329030087 >> v2:172.21.15.167:3300/0 conn(0x7f27fc0699c0 0x7f27fc069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.466+0000 7f27f9ffb700 1 -- 172.21.15.33:0/3329030087 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160693282 0 0) 0x7f27fc0ae490 con 0x7f27fc155140 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27f9ffb700 1 -- 172.21.15.33:0/3329030087 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e4003570 con 0x7f27fc155140 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27fa7fc700 1 -- 172.21.15.33:0/3329030087 >> v2:172.21.15.167:3300/0 conn(0x7f27fc0699c0 msgr2=0x7f27fc069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27fa7fc700 1 --2- 172.21.15.33:0/3329030087 >> v2:172.21.15.167:3300/0 conn(0x7f27fc0699c0 0x7f27fc069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27fa7fc700 1 -- 172.21.15.33:0/3329030087 >> v1:172.21.15.167:6789/0 conn(0x7f27fc155140 legacy=0x7f27fc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27fa7fc700 1 -- 172.21.15.33:0/3329030087 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc158b70 con 0x7f27fc158700 2023-10-06T05:22:17.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27fb7fe700 1 --2- 172.21.15.33:0/3329030087 >> v2:172.21.15.167:3300/0 conn(0x7f27fc0699c0 0x7f27fc069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27fa7fc700 1 --2- 172.21.15.33:0/3329030087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc158700 0x7f27fc0adf50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098e0 tx=0x7f27f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7efa8e76ee273b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27f9ffb700 1 -- 172.21.15.33:0/3329030087 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011630 con 0x7f27fc158700 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27f9ffb700 1 -- 172.21.15.33:0/3329030087 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0011790 con 0x7f27fc158700 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.467+0000 7f27f9ffb700 1 -- 172.21.15.33:0/3329030087 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011b30 con 0x7f27fc158700 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 -- 172.21.15.33:0/3329030087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc158700 msgr2=0x7f27fc0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 --2- 172.21.15.33:0/3329030087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc158700 0x7f27fc0adf50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098e0 tx=0x7f27f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 -- 172.21.15.33:0/3329030087 shutdown_connections 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 --2- 172.21.15.33:0/3329030087 >> v2:172.21.15.167:3300/0 conn(0x7f27fc0699c0 0x7f27fc069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 --2- 172.21.15.33:0/3329030087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc158700 0x7f27fc0adf50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 -- 172.21.15.33:0/3329030087 >> 172.21.15.33:0/3329030087 conn(0x7f27fc06d030 msgr2=0x7f27fc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 -- 172.21.15.33:0/3329030087 shutdown_connections 2023-10-06T05:22:17.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.468+0000 7f27fbfff700 1 -- 172.21.15.33:0/3329030087 wait complete. 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.469+0000 7f27fbfff700 1 Processor -- start 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.469+0000 7f27fbfff700 1 -- start start 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.470+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc0699c0 0x7f27fc147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.470+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 0x7f27fc1679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.470+0000 7f27fbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc158700 0x7f27fc0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.470+0000 7f27fbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc159100 con 0x7f27fc155140 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.470+0000 7f27fbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc158dd0 con 0x7f27fc158700 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.470+0000 7f27fbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc168320 con 0x7f27fc0699c0 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 0x7f27fc1679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 0x7f27fc1679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32892/0 (socket says 172.21.15.33:32892) 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 -- 172.21.15.33:0/2518860788 learned_addr learned my addr 172.21.15.33:0/2518860788 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:17.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27faffd700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc0699c0 0x7f27fc147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fb7fe700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc158700 0x7f27fc0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 -- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc0699c0 msgr2=0x7f27fc147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc0699c0 0x7f27fc147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 -- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc158700 msgr2=0x7f27fc0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc158700 0x7f27fc0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 -- 172.21.15.33:0/2518860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4009a20 con 0x7f27fc155140 2023-10-06T05:22:17.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.471+0000 7f27fa7fc700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 0x7f27fc1679b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f27fc068570 tx=0x7f27f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.472+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011b10 con 0x7f27fc155140 2023-10-06T05:22:17.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.472+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f001a430 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.472+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f001a7d0 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.472+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f0009420 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.472+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc1688e0 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.473+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc1592b0 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.474+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27f001abc0 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.475+0000 7f27e2ffd700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27e8066f30 0x7f27e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.475+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f27f0085ff0 con 0x7f27fc155140 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.475+0000 7f27faffd700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27e8066f30 0x7f27e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.476+0000 7f27faffd700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27e8066f30 0x7f27e80693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f27ec007fd0 tx=0x7f27ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.479+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27fc1592b0 con 0x7f27fc155140 2023-10-06T05:22:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.762+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync"]} v 0) v1 -- 0x7f27fc06bb80 con 0x7f27fc155140 2023-10-06T05:22:17.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.807+0000 7f27e2ffd700 1 -- 172.21.15.33:0/2518860788 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dir_fsync"]}]=0 v402) v1 ==== 146+0+0 (secure 0 0 0) 0x7f27f00533a0 con 0x7f27fc155140 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27e8066f30 msgr2=0x7f27e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27e8066f30 0x7f27e80693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f27ec007fd0 tx=0x7f27ec009040 comp rx=0 tx=0).stop 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 msgr2=0x7f27fc1679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 0x7f27fc1679b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f27fc068570 tx=0x7f27f0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 shutdown_connections 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27e8066f30 0x7f27e80693e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc158700 0x7f27fc0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc0699c0 0x7f27fc147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 --2- 172.21.15.33:0/2518860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc155140 0x7f27fc1679b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 >> 172.21.15.33:0/2518860788 conn(0x7f27fc06d030 msgr2=0x7f27fc153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 shutdown_connections 2023-10-06T05:22:17.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:17.810+0000 7f27fbfff700 1 -- 172.21.15.33:0/2518860788 wait complete. 2023-10-06T05:22:17.828 INFO:tasks.cephfs_test_runner:test_dir_fsync (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:22:17.829 INFO:tasks.cephfs_test_runner:Starting test: test_dont_mark_unresponsive_client_stale (tasks.cephfs.test_client_recovery.TestClientRecovery) Test that an unresponsive client holding caps is not marked stale or 2023-10-06T05:22:17.829 INFO:tasks.cephfs_test_runner:test_dont_mark_unresponsive_client_stale (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:22:17.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dont_mark_unresponsive_client_stale' 2023-10-06T05:22:18.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.015+0000 7fcedd99d700 1 Processor -- start 2023-10-06T05:22:18.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.015+0000 7fcedd99d700 1 -- start start 2023-10-06T05:22:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fced813faa0 0x7fced813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fced806aad0 0x7fced806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced80c9cb0 con 0x7fced80c7940 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced80ca270 con 0x7fced806aad0 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced806b840 con 0x7fced813faa0 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fced806aad0 0x7fced806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd19c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fced806aad0 0x7fced806af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32894/0 (socket says 172.21.15.33:32894) 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedd19c700 1 -- 172.21.15.33:0/2487684768 learned_addr learned my addr 172.21.15.33:0/2487684768 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fced7fff700 1 --1- 172.21.15.33:0/2487684768 >> v1:172.21.15.33:6789/0 conn(0x7fced80c7940 0x7fced80c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50138/0 (socket says 172.21.15.33:50138) 2023-10-06T05:22:18.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fcedd19c700 1 -- 172.21.15.33:0/2487684768 >> v2:172.21.15.167:3300/0 conn(0x7fced813faa0 msgr2=0x7fced813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.017+0000 7fcedc99b700 1 --2- 172.21.15.33:0/2487684768 >> v2:172.21.15.167:3300/0 conn(0x7fced813faa0 0x7fced813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fcedd19c700 1 --2- 172.21.15.33:0/2487684768 >> v2:172.21.15.167:3300/0 conn(0x7fced813faa0 0x7fced813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fcedd19c700 1 -- 172.21.15.33:0/2487684768 >> v1:172.21.15.33:6789/0 conn(0x7fced80c7940 legacy=0x7fced80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fcedd19c700 1 -- 172.21.15.33:0/2487684768 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced806bb20 con 0x7fced806aad0 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fcedc99b700 1 --2- 172.21.15.33:0/2487684768 >> v2:172.21.15.167:3300/0 conn(0x7fced813faa0 0x7fced813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fced77fe700 1 -- 172.21.15.33:0/2487684768 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952476912 0 0) 0x7fced80c9cb0 con 0x7fced80c7940 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fcedd19c700 1 --2- 172.21.15.33:0/2487684768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced806aad0 0x7fced806af30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fced806fe50 tx=0x7fced000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4b0a4bdff730c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fced77fe700 1 -- 172.21.15.33:0/2487684768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0010070 con 0x7fced806aad0 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fced77fe700 1 -- 172.21.15.33:0/2487684768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced000de90 con 0x7fced806aad0 2023-10-06T05:22:18.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.018+0000 7fced77fe700 1 -- 172.21.15.33:0/2487684768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000b320 con 0x7fced806aad0 2023-10-06T05:22:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 -- 172.21.15.33:0/2487684768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced806aad0 msgr2=0x7fced806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2487684768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced806aad0 0x7fced806af30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fced806fe50 tx=0x7fced000ba50 comp rx=0 tx=0).stop 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 -- 172.21.15.33:0/2487684768 shutdown_connections 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2487684768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced806aad0 0x7fced806af30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2487684768 >> v2:172.21.15.167:3300/0 conn(0x7fced813faa0 0x7fced813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 -- 172.21.15.33:0/2487684768 >> 172.21.15.33:0/2487684768 conn(0x7fced806d030 msgr2=0x7fced806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 -- 172.21.15.33:0/2487684768 shutdown_connections 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.019+0000 7fcedd99d700 1 -- 172.21.15.33:0/2487684768 wait complete. 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.020+0000 7fcedd99d700 1 Processor -- start 2023-10-06T05:22:18.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.020+0000 7fcedd99d700 1 -- start start 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.021+0000 7fcedd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced806aad0 0x7fced81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.021+0000 7fcedd99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fced80c7940 0x7fced80b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.021+0000 7fcedd99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 0x7fced80b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.021+0000 7fcedd99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced80c3b50 con 0x7fced813faa0 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.021+0000 7fcedd99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced80b1070 con 0x7fced80c7940 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.021+0000 7fcedd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced80c3d40 con 0x7fced806aad0 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 0x7fced80b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 0x7fced80b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32898/0 (socket says 172.21.15.33:32898) 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 -- 172.21.15.33:0/2232740433 learned_addr learned my addr 172.21.15.33:0/2232740433 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:18.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fced7fff700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fced80c7940 0x7fced80b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedc99b700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced806aad0 0x7fced81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 -- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced806aad0 msgr2=0x7fced81562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced806aad0 0x7fced81562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 -- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fced80c7940 msgr2=0x7fced80b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fced80c7940 0x7fced80b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 -- 172.21.15.33:0/2232740433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced000bcf0 con 0x7fced813faa0 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fced7fff700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fced80c7940 0x7fced80b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedc99b700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced806aad0 0x7fced81562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.022+0000 7fcedd19c700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 0x7fced80b1a20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fced80719c0 tx=0x7fced001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.023+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced001e440 con 0x7fced813faa0 2023-10-06T05:22:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.023+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced001e5c0 con 0x7fced813faa0 2023-10-06T05:22:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.023+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced001e980 con 0x7fced813faa0 2023-10-06T05:22:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.023+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced80c4020 con 0x7fced813faa0 2023-10-06T05:22:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.023+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced80c4660 con 0x7fced813faa0 2023-10-06T05:22:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.024+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced804c380 con 0x7fced813faa0 2023-10-06T05:22:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.025+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fced0003040 con 0x7fced813faa0 2023-10-06T05:22:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.025+0000 7fced4ff9700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcebc066f30 0x7fcebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.026+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fced00862e0 con 0x7fced813faa0 2023-10-06T05:22:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.026+0000 7fcedc99b700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcebc066f30 0x7fcebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.027+0000 7fcedc99b700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcebc066f30 0x7fcebc0693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcecc006fd0 tx=0x7fcecc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.030+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fced00535d0 con 0x7fced813faa0 2023-10-06T05:22:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.310+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dont_mark_unresponsive_client_stale"]} v 0) v1 -- 0x7fced80c41b0 con 0x7fced813faa0 2023-10-06T05:22:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.811+0000 7fced4ff9700 1 -- 172.21.15.33:0/2232740433 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dont_mark_unresponsive_client_stale"]}]=0 v403) v1 ==== 175+0+0 (secure 0 0 0) 0x7fced0056bf0 con 0x7fced813faa0 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcebc066f30 msgr2=0x7fcebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcebc066f30 0x7fcebc0693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcecc006fd0 tx=0x7fcecc008040 comp rx=0 tx=0).stop 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 msgr2=0x7fced80b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 0x7fced80b1a20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fced80719c0 tx=0x7fced001a000 comp rx=0 tx=0).stop 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 shutdown_connections 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcebc066f30 0x7fcebc0693e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced813faa0 0x7fced80b1a20 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fced806aad0 0x7fced81562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 --2- 172.21.15.33:0/2232740433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fced80c7940 0x7fced80b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 >> 172.21.15.33:0/2232740433 conn(0x7fced806d030 msgr2=0x7fced80c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 shutdown_connections 2023-10-06T05:22:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:18.814+0000 7fcedd99d700 1 -- 172.21.15.33:0/2232740433 wait complete. 2023-10-06T05:22:18.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:22:18.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:18.887 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:19.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.028+0000 7f5f7b59e700 1 Processor -- start 2023-10-06T05:22:19.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.028+0000 7f5f7b59e700 1 -- start start 2023-10-06T05:22:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f7b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f7c0aff90 0x7f5f7c0a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f7b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5f7c06cb80 0x7f5f7c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f7b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c15d3a0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f7b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c0686c0 con 0x7f5f7c0aff90 2023-10-06T05:22:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f7b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c0b0470 con 0x7f5f7c06cb80 2023-10-06T05:22:19.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f7c0aff90 0x7f5f7c0a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f7c0aff90 0x7f5f7c0a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32906/0 (socket says 172.21.15.33:32906) 2023-10-06T05:22:19.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 -- 172.21.15.33:0/3566293280 learned_addr learned my addr 172.21.15.33:0/3566293280 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:19.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f7ad9d700 1 --2- 172.21.15.33:0/3566293280 >> v2:172.21.15.167:3300/0 conn(0x7f5f7c06cb80 0x7f5f7c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 -- 172.21.15.33:0/3566293280 >> v2:172.21.15.167:3300/0 conn(0x7f5f7c06cb80 msgr2=0x7f5f7c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 --2- 172.21.15.33:0/3566293280 >> v2:172.21.15.167:3300/0 conn(0x7f5f7c06cb80 0x7f5f7c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 -- 172.21.15.33:0/3566293280 >> v1:172.21.15.167:6789/0 conn(0x7f5f7c06c7b0 legacy=0x7f5f7c067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.030+0000 7f5f79d9b700 1 -- 172.21.15.33:0/3566293280 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0b0750 con 0x7f5f7c0aff90 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.031+0000 7f5f79d9b700 1 --2- 172.21.15.33:0/3566293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c0a9890 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0b0ff0 tx=0x7f5f70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46971a6c3a343639 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.031+0000 7f5f7959a700 1 -- 172.21.15.33:0/3566293280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000b860 con 0x7f5f7c0aff90 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.031+0000 7f5f7959a700 1 -- 172.21.15.33:0/3566293280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000b9c0 con 0x7f5f7c0aff90 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.031+0000 7f5f7959a700 1 -- 172.21.15.33:0/3566293280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000bd60 con 0x7f5f7c0aff90 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.031+0000 7f5f7b59e700 1 -- 172.21.15.33:0/3566293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c0aff90 msgr2=0x7f5f7c0a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.031+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/3566293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c0a9890 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0b0ff0 tx=0x7f5f70009250 comp rx=0 tx=0).stop 2023-10-06T05:22:19.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.032+0000 7f5f7b59e700 1 -- 172.21.15.33:0/3566293280 shutdown_connections 2023-10-06T05:22:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.032+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/3566293280 >> v2:172.21.15.167:3300/0 conn(0x7f5f7c06cb80 0x7f5f7c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.032+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/3566293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c0a9890 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.032+0000 7f5f7b59e700 1 -- 172.21.15.33:0/3566293280 >> 172.21.15.33:0/3566293280 conn(0x7f5f7c06d670 msgr2=0x7f5f7c06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.032+0000 7f5f7b59e700 1 -- 172.21.15.33:0/3566293280 shutdown_connections 2023-10-06T05:22:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.032+0000 7f5f7b59e700 1 -- 172.21.15.33:0/3566293280 wait complete. 2023-10-06T05:22:19.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.033+0000 7f5f7b59e700 1 Processor -- start 2023-10-06T05:22:19.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.033+0000 7f5f7b59e700 1 -- start start 2023-10-06T05:22:19.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 0x7f5f7c151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f7c06cb80 0x7f5f7c13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c14e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c1510c0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c150230 con 0x7f5f7c0aff90 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c1504b0 con 0x7f5f7c06cb80 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 0x7f5f7c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 0x7f5f7c151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32918/0 (socket says 172.21.15.33:32918) 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.034+0000 7f5f7a59c700 1 -- 172.21.15.33:0/1842139434 learned_addr learned my addr 172.21.15.33:0/1842139434 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7ad9d700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c14e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f79d9b700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f7c06cb80 0x7f5f7c13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7a59c700 1 -- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f7c06cb80 msgr2=0x7f5f7c13fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7a59c700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f7c06cb80 0x7f5f7c13fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7a59c700 1 -- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7c0aff90 msgr2=0x7f5f7c14e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7a59c700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c14e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7a59c700 1 -- 172.21.15.33:0/1842139434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700094f0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f79d9b700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f7c06cb80 0x7f5f7c13fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7a59c700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 0x7f5f7c151550 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c7f0 tx=0x7f5f6c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c011590 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c011710 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c011ad0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.035+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c150790 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.036+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c150d50 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.036+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c1513a0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.037+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f6c01b3b0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.038+0000 7f5f6a7fc700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f5c066f30 0x7f5f5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.038+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5f6c083900 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.039+0000 7f5f79d9b700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f5c066f30 0x7f5f5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.040+0000 7f5f79d9b700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f5c066f30 0x7f5f5c0693e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0b0ff0 tx=0x7f5f70017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.043+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f6c050ca0 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.247+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5f7c0ce310 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.252+0000 7f5f6a7fc700 1 -- 172.21.15.33:0/1842139434 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f6c017070 con 0x7f5f7c06c7b0 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f5c066f30 msgr2=0x7f5f5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f5c066f30 0x7f5f5c0693e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0b0ff0 tx=0x7f5f70017000 comp rx=0 tx=0).stop 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 msgr2=0x7f5f7c151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 0x7f5f7c151550 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c7f0 tx=0x7f5f6c007f70 comp rx=0 tx=0).stop 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 shutdown_connections 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f5c066f30 0x7f5f5c0693e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7c0aff90 0x7f5f7c14e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f7c06c7b0 0x7f5f7c151550 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 --2- 172.21.15.33:0/1842139434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f7c06cb80 0x7f5f7c13fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 >> 172.21.15.33:0/1842139434 conn(0x7f5f7c06d670 msgr2=0x7f5f7c151e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 shutdown_connections 2023-10-06T05:22:19.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.254+0000 7f5f7b59e700 1 -- 172.21.15.33:0/1842139434 wait complete. 2023-10-06T05:22:19.274 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:22:19.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:19.359 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:19.441 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:22:19.442 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:22:19.442 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:19.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.628+0000 7ff5affff700 1 Processor -- start 2023-10-06T05:22:19.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.629+0000 7ff5affff700 1 -- start start 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5affff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5a813fb70 0x7ff5a806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5affff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff5a806ba40 0x7ff5a80c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5affff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80ca8d0 con 0x7ff5a80ca350 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5affff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80cabb0 con 0x7ff5a813fb70 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5affff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a806ad00 con 0x7ff5a806ba40 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5add9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5a813fb70 0x7ff5a806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5add9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5a813fb70 0x7ff5a806b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32934/0 (socket says 172.21.15.33:32934) 2023-10-06T05:22:19.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5add9b700 1 -- 172.21.15.33:0/2007228593 learned_addr learned my addr 172.21.15.33:0/2007228593 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:19.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.630+0000 7ff5ad59a700 1 --2- 172.21.15.33:0/2007228593 >> v2:172.21.15.167:3300/0 conn(0x7ff5a806ba40 0x7ff5a80c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5add9b700 1 -- 172.21.15.33:0/2007228593 >> v2:172.21.15.167:3300/0 conn(0x7ff5a806ba40 msgr2=0x7ff5a80c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5add9b700 1 --2- 172.21.15.33:0/2007228593 >> v2:172.21.15.167:3300/0 conn(0x7ff5a806ba40 0x7ff5a80c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5add9b700 1 -- 172.21.15.33:0/2007228593 >> v1:172.21.15.167:6789/0 conn(0x7ff5a80ca350 legacy=0x7ff5a80af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5add9b700 1 -- 172.21.15.33:0/2007228593 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a806afe0 con 0x7ff5a813fb70 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5add9b700 1 --2- 172.21.15.33:0/2007228593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a813fb70 0x7ff5a806b500 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80c6810 tx=0x7ff5a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4570bedac47c7eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5acd99700 1 -- 172.21.15.33:0/2007228593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400b860 con 0x7ff5a813fb70 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5acd99700 1 -- 172.21.15.33:0/2007228593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a400b9c0 con 0x7ff5a813fb70 2023-10-06T05:22:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.631+0000 7ff5acd99700 1 -- 172.21.15.33:0/2007228593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400bd60 con 0x7ff5a813fb70 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 -- 172.21.15.33:0/2007228593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a813fb70 msgr2=0x7ff5a806b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 --2- 172.21.15.33:0/2007228593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a813fb70 0x7ff5a806b500 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80c6810 tx=0x7ff5a4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 -- 172.21.15.33:0/2007228593 shutdown_connections 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 --2- 172.21.15.33:0/2007228593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a813fb70 0x7ff5a806b500 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 --2- 172.21.15.33:0/2007228593 >> v2:172.21.15.167:3300/0 conn(0x7ff5a806ba40 0x7ff5a80c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 -- 172.21.15.33:0/2007228593 >> 172.21.15.33:0/2007228593 conn(0x7ff5a80675f0 msgr2=0x7ff5a80679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 -- 172.21.15.33:0/2007228593 shutdown_connections 2023-10-06T05:22:19.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.632+0000 7ff5affff700 1 -- 172.21.15.33:0/2007228593 wait complete. 2023-10-06T05:22:19.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.633+0000 7ff5affff700 1 Processor -- start 2023-10-06T05:22:19.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.634+0000 7ff5affff700 1 -- start start 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.634+0000 7ff5affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 0x7ff5a80c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.634+0000 7ff5affff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5a80ca350 0x7ff5a81696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.634+0000 7ff5affff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a813fb70 0x7ff5a815fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5affff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b0880 con 0x7ff5a806ba40 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5affff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b0530 con 0x7ff5a813fb70 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5affff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8169f90 con 0x7ff5a80ca350 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 0x7ff5a80c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 0x7ff5a80c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32944/0 (socket says 172.21.15.33:32944) 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 -- 172.21.15.33:0/42204527 learned_addr learned my addr 172.21.15.33:0/42204527 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5ad59a700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5a80ca350 0x7ff5a81696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5ae59c700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a813fb70 0x7ff5a815fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 -- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5a80ca350 msgr2=0x7ff5a81696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5a80ca350 0x7ff5a81696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 -- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a813fb70 msgr2=0x7ff5a815fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a813fb70 0x7ff5a815fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 -- 172.21.15.33:0/42204527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40094f0 con 0x7ff5a806ba40 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5ae59c700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a813fb70 0x7ff5a815fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.635+0000 7ff5add9b700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 0x7ff5a80c4e20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40090b0 tx=0x7ff5a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.636+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4010470 con 0x7ff5a806ba40 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.636+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a40105f0 con 0x7ff5a806ba40 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.636+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a816a270 con 0x7ff5a806ba40 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.636+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40109b0 con 0x7ff5a806ba40 2023-10-06T05:22:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.636+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8169c30 con 0x7ff5a806ba40 2023-10-06T05:22:19.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.637+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a80b09c0 con 0x7ff5a806ba40 2023-10-06T05:22:19.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.638+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5a40233f0 con 0x7ff5a806ba40 2023-10-06T05:22:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.639+0000 7ff59dffb700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff58c066f30 0x7ff58c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.639+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff5a4014070 con 0x7ff5a806ba40 2023-10-06T05:22:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.639+0000 7ff5ad59a700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff58c066f30 0x7ff58c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.640+0000 7ff5ad59a700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff58c066f30 0x7ff58c0693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80c8800 tx=0x7ff598008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.643+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5a80b09c0 con 0x7ff5a806ba40 2023-10-06T05:22:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.895+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5a8063ff0 con 0x7ff5a806ba40 2023-10-06T05:22:19.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.895+0000 7ff59dffb700 1 -- 172.21.15.33:0/42204527 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ff5a40537e0 con 0x7ff5a806ba40 2023-10-06T05:22:19.897 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:19.897 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6313,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3370388380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3370388380},{"type":"v1","addr":"172.21.15.167:6836","nonce":3370388380}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3313926721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3313926721},{"type":"v1","addr":"172.21.15.167:6837","nonce":3313926721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3844303432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3844303432},{"type":"v1","addr":"172.21.15.33:6834","nonce":3844303432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7218,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3028772173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3028772173},{"type":"v1","addr":"172.21.15.33:6835","nonce":3028772173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[]} 2023-10-06T05:22:19.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff58c066f30 msgr2=0x7ff58c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff58c066f30 0x7ff58c0693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80c8800 tx=0x7ff598008040 comp rx=0 tx=0).stop 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 msgr2=0x7ff5a80c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 0x7ff5a80c4e20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40090b0 tx=0x7ff5a4019000 comp rx=0 tx=0).stop 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 shutdown_connections 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff58c066f30 0x7ff58c0693e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a813fb70 0x7ff5a815fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5a806ba40 0x7ff5a80c4e20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 --2- 172.21.15.33:0/42204527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5a80ca350 0x7ff5a81696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 >> 172.21.15.33:0/42204527 conn(0x7ff5a80675f0 msgr2=0x7ff5a80c1730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 shutdown_connections 2023-10-06T05:22:19.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:19.898+0000 7ff5affff700 1 -- 172.21.15.33:0/42204527 wait complete. 2023-10-06T05:22:19.901 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 66 2023-10-06T05:22:19.917 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:22:19.918 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:22:19.918 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:22:19.918 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:19.918 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:22:19.918 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:22:19.918 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:22:19.919 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:19.919 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:22:19.919 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:22:19.919 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:22:19.919 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:19.919 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:22:19.919 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:22:19.920 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:22:19.920 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:19.933 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:22:19.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:22:19.935 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:22:19.935 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:22:19.936 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:22:19.936 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:22:19.979 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:22:19.979 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:19.982 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:22:19.983 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:22:19.983 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:19.983 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:20.110 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:22:20.116 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:22:20.116 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:22:23.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T05:22:25.933 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:22:25.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:22:25.939 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:22:25.939 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:25.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:22:26.082 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:22:26.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.142+0000 7f0dd1f9d700 1 Processor -- start 2023-10-06T05:22:26.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.142+0000 7f0dd1f9d700 1 -- start start 2023-10-06T05:22:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd1f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dcc06c560 0x7f0dcc0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc1598a0 0x7f0dcc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd1f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc070810 con 0x7f0dcc0ca7a0 2023-10-06T05:22:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd1f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc06da40 con 0x7f0dcc06c560 2023-10-06T05:22:26.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc06dd20 con 0x7f0dcc1598a0 2023-10-06T05:22:26.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd0f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0dcc0ca7a0 0x7f0dcc159190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57302/0 (socket says 172.21.15.33:57302) 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd0f9b700 1 -- 172.21.15.33:0/3157913231 learned_addr learned my addr 172.21.15.33:0/3157913231 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dd179c700 1 --2- 172.21.15.33:0/3157913231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc1598a0 0x7f0dcc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dcbfff700 1 --2- 172.21.15.33:0/3157913231 >> v2:172.21.15.167:3300/0 conn(0x7f0dcc06c560 0x7f0dcc0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.144+0000 7f0dcb7fe700 1 -- 172.21.15.33:0/3157913231 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386653664 0 0) 0x7f0dcc070810 con 0x7f0dcc0ca7a0 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dcb7fe700 1 -- 172.21.15.33:0/3157913231 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dbc003570 con 0x7f0dcc0ca7a0 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dcb7fe700 1 -- 172.21.15.33:0/3157913231 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824449401 0 0) 0x7f0dbc003570 con 0x7f0dcc0ca7a0 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dd179c700 1 -- 172.21.15.33:0/3157913231 >> v2:172.21.15.167:3300/0 conn(0x7f0dcc06c560 msgr2=0x7f0dcc0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dd179c700 1 --2- 172.21.15.33:0/3157913231 >> v2:172.21.15.167:3300/0 conn(0x7f0dcc06c560 0x7f0dcc0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dd179c700 1 -- 172.21.15.33:0/3157913231 >> v1:172.21.15.33:6789/0 conn(0x7f0dcc0ca7a0 legacy=0x7f0dcc159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dd179c700 1 -- 172.21.15.33:0/3157913231 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0c7060 con 0x7f0dcc1598a0 2023-10-06T05:22:26.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dd179c700 1 --2- 172.21.15.33:0/3157913231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc1598a0 0x7f0dcc0c8080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009e60 tx=0x7f0dc4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41d9a9cb6b0d4ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.145+0000 7f0dcb7fe700 1 -- 172.21.15.33:0/3157913231 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400e070 con 0x7f0dcc1598a0 2023-10-06T05:22:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dcb7fe700 1 -- 172.21.15.33:0/3157913231 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc400b7a0 con 0x7f0dcc1598a0 2023-10-06T05:22:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dcb7fe700 1 -- 172.21.15.33:0/3157913231 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0001470 con 0x7f0dcc1598a0 2023-10-06T05:22:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3157913231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc1598a0 msgr2=0x7f0dcc0c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3157913231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc1598a0 0x7f0dcc0c8080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009e60 tx=0x7f0dc4009780 comp rx=0 tx=0).stop 2023-10-06T05:22:26.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3157913231 shutdown_connections 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3157913231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc1598a0 0x7f0dcc0c8080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3157913231 >> v2:172.21.15.167:3300/0 conn(0x7f0dcc06c560 0x7f0dcc0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3157913231 >> 172.21.15.33:0/3157913231 conn(0x7f0dcc06e650 msgr2=0x7f0dcc06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3157913231 shutdown_connections 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.146+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3157913231 wait complete. 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.148+0000 7f0dd1f9d700 1 Processor -- start 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.148+0000 7f0dd1f9d700 1 -- start start 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc06c560 0x7f0dcc14f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd1f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dcc0ca7a0 0x7f0dcc150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd1f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 0x7f0dcc150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd1f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b27e0 con 0x7f0dcc1598a0 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd1f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b2260 con 0x7f0dcc0ca7a0 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b2500 con 0x7f0dcc06c560 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd179c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 0x7f0dcc150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd179c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 0x7f0dcc150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49398/0 (socket says 172.21.15.33:49398) 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd179c700 1 -- 172.21.15.33:0/3855246144 learned_addr learned my addr 172.21.15.33:0/3855246144 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dcbfff700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dcc0ca7a0 0x7f0dcc150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd0f9b700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc06c560 0x7f0dcc14f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.149+0000 7f0dd179c700 1 -- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc06c560 msgr2=0x7f0dcc14f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd179c700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc06c560 0x7f0dcc14f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd179c700 1 -- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dcc0ca7a0 msgr2=0x7f0dcc150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd179c700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dcc0ca7a0 0x7f0dcc150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd179c700 1 -- 172.21.15.33:0/3855246144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4009a20 con 0x7f0dcc1598a0 2023-10-06T05:22:26.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd179c700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 0x7f0dcc150740 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc069d40 tx=0x7f0dc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4012bc0 con 0x7f0dcc1598a0 2023-10-06T05:22:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4012d60 con 0x7f0dcc1598a0 2023-10-06T05:22:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc401e680 con 0x7f0dcc1598a0 2023-10-06T05:22:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc151fd0 con 0x7f0dcc1598a0 2023-10-06T05:22:26.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.150+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc14f990 con 0x7f0dcc1598a0 2023-10-06T05:22:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.151+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc04c380 con 0x7f0dcc1598a0 2023-10-06T05:22:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.152+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0dc401e9f0 con 0x7f0dcc1598a0 2023-10-06T05:22:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.153+0000 7f0dc8ff9700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0db0066f30 0x7f0db00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.153+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0dc4086500 con 0x7f0dcc1598a0 2023-10-06T05:22:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.153+0000 7f0dd0f9b700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0db0066f30 0x7f0db00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.154+0000 7f0dd0f9b700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0db0066f30 0x7f0db00693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0c9aa0 tx=0x7f0dc0001aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.157+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dc40537f0 con 0x7f0dcc1598a0 2023-10-06T05:22:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:26.375+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0dcc0c7340 con 0x7f0dcc1598a0 2023-10-06T05:22:28.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.005+0000 7f0dc8ff9700 1 -- 172.21.15.33:0/3855246144 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0dc4056e10 con 0x7f0dcc1598a0 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.008+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0db0066f30 msgr2=0x7f0db00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0db0066f30 0x7f0db00693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0c9aa0 tx=0x7f0dc0001aa0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 msgr2=0x7f0dcc150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 0x7f0dcc150740 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc069d40 tx=0x7f0dc4018000 comp rx=0 tx=0).stop 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 shutdown_connections 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0db0066f30 0x7f0db00693e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dcc1598a0 0x7f0dcc150740 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dcc06c560 0x7f0dcc14f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 --2- 172.21.15.33:0/3855246144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dcc0ca7a0 0x7f0dcc150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 >> 172.21.15.33:0/3855246144 conn(0x7f0dcc06e650 msgr2=0x7f0dcc0c1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 shutdown_connections 2023-10-06T05:22:28.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.009+0000 7f0dd1f9d700 1 -- 172.21.15.33:0/3855246144 wait complete. 2023-10-06T05:22:28.012 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:22:28.033 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:22:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.224+0000 7fbfb3fff700 1 Processor -- start 2023-10-06T05:22:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.224+0000 7fbfb3fff700 1 -- start start 2023-10-06T05:22:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.225+0000 7fbfb3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbfb40c6340 0x7fbfb40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.225+0000 7fbfb3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb406c820 con 0x7fbfb4159520 2023-10-06T05:22:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.225+0000 7fbfb3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb406a160 con 0x7fbfb40c80d0 2023-10-06T05:22:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.225+0000 7fbfb3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb40c8800 con 0x7fbfb40c6340 2023-10-06T05:22:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.225+0000 7fbfb2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbfb4159520 0x7fbfb40c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57318/0 (socket says 172.21.15.33:57318) 2023-10-06T05:22:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.225+0000 7fbfb2ffd700 1 -- 172.21.15.33:0/1699944070 learned_addr learned my addr 172.21.15.33:0/1699944070 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb27fc700 1 --2- 172.21.15.33:0/1699944070 >> v2:172.21.15.167:3300/0 conn(0x7fbfb40c6340 0x7fbfb40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524175317 0 0) 0x7fbfb406c820 con 0x7fbfb4159520 2023-10-06T05:22:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf9c003570 con 0x7fbfb4159520 2023-10-06T05:22:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 942593909 0 0) 0x7fbf9c003570 con 0x7fbfb4159520 2023-10-06T05:22:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 >> v2:172.21.15.167:3300/0 conn(0x7fbfb40c6340 msgr2=0x7fbfb40c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 --2- 172.21.15.33:0/1699944070 >> v2:172.21.15.167:3300/0 conn(0x7fbfb40c6340 0x7fbfb40c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 >> v1:172.21.15.167:6789/0 conn(0x7fbfb40c80d0 legacy=0x7fbfb406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40c8ae0 con 0x7fbfb4159520 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fbfa4001470 con 0x7fbfb4159520 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 >> v1:172.21.15.33:6789/0 conn(0x7fbfb4159520 legacy=0x7fbfb40c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9c003a60 0x7fbf9c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c004810 0x7fbf9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0051c0 0x7fbf9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40c8800 con 0x7fbf9c0051c0 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c00c0b0 con 0x7fbf9c003a60 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.226+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c00c270 con 0x7fbf9c004810 2023-10-06T05:22:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9c003a60 0x7fbf9c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb37fe700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0051c0 0x7fbf9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb27fc700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c004810 0x7fbf9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 -- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c004810 msgr2=0x7fbf9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c004810 0x7fbf9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 -- 172.21.15.33:0/1699944070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0051c0 msgr2=0x7fbf9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0051c0 0x7fbf9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 -- 172.21.15.33:0/1699944070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c003570 con 0x7fbf9c003a60 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb37fe700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9c0051c0 0x7fbf9c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb37fe700 1 -- 172.21.15.33:0/1699944070 reap_dead start 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.227+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9c003a60 0x7fbf9c003e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400c2d0 tx=0x7fbfa4001da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e43adb7618b08c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa400cba0 con 0x7fbf9c003a60 2023-10-06T05:22:28.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa400cd20 con 0x7fbf9c003a60 2023-10-06T05:22:28.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb1ffb700 1 -- 172.21.15.33:0/1699944070 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4005b70 con 0x7fbf9c003a60 2023-10-06T05:22:28.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 -- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9c003a60 msgr2=0x7fbf9c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9c003a60 0x7fbf9c003e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400c2d0 tx=0x7fbfa4001da0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 -- 172.21.15.33:0/1699944070 shutdown_connections 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/1699944070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9c003a60 0x7fbf9c003e30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 -- 172.21.15.33:0/1699944070 >> 172.21.15.33:0/1699944070 conn(0x7fbfb40cc930 msgr2=0x7fbfb40ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 -- 172.21.15.33:0/1699944070 shutdown_connections 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.228+0000 7fbfb3fff700 1 -- 172.21.15.33:0/1699944070 wait complete. 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.230+0000 7fbfb3fff700 1 Processor -- start 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.230+0000 7fbfb3fff700 1 -- start start 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c003a60 0x7fbfb414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 0x7fbfb415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfb4169930 0x7fbfb4169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40aea40 con 0x7fbfb40adf30 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb415a1d0 con 0x7fbfb4169930 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb414fae0 con 0x7fbf9c003a60 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 0x7fbfb415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 0x7fbfb415d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49416/0 (socket says 172.21.15.33:49416) 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb27fc700 1 -- 172.21.15.33:0/2608495727 learned_addr learned my addr 172.21.15.33:0/2608495727 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c003a60 0x7fbfb414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.231+0000 7fbfb37fe700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfb4169930 0x7fbfb4169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb27fc700 1 -- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c003a60 msgr2=0x7fbfb414f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb27fc700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c003a60 0x7fbfb414f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb27fc700 1 -- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfb4169930 msgr2=0x7fbfb4169da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb27fc700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfb4169930 0x7fbfb4169da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb27fc700 1 -- 172.21.15.33:0/2608495727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb416a2e0 con 0x7fbfb40adf30 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb27fc700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 0x7fbfb415d160 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8011a80 tx=0x7fbfa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800f940 con 0x7fbfb40adf30 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800faa0 con 0x7fbfb40adf30 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb416aac0 con 0x7fbfb40adf30 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800fe40 con 0x7fbfb40adf30 2023-10-06T05:22:28.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.232+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb40ae760 con 0x7fbfb40adf30 2023-10-06T05:22:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.233+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb415a360 con 0x7fbfb40adf30 2023-10-06T05:22:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.234+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfa8014650 con 0x7fbfb40adf30 2023-10-06T05:22:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.235+0000 7fbf9affd700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfa0066f30 0x7fbfa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.236+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbfa8018030 con 0x7fbfb40adf30 2023-10-06T05:22:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.236+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfa0066f30 0x7fbfa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.237+0000 7fbfb2ffd700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfa0066f30 0x7fbfa00693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4007c20 tx=0x7fbfa4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.240+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfa8051f70 con 0x7fbfb40adf30 2023-10-06T05:22:28.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.521+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbfb404c380 con 0x7fbfb40adf30 2023-10-06T05:22:28.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.522+0000 7fbf9affd700 1 -- 172.21.15.33:0/2608495727 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbfa8022090 con 0x7fbfb40adf30 2023-10-06T05:22:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:28.523 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:22:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:22:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:22:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:22:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:22:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:22:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:22:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:22:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:22:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:22:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:22:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:22:28.539 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfa0066f30 msgr2=0x7fbfa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfa0066f30 0x7fbfa00693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4007c20 tx=0x7fbfa4015000 comp rx=0 tx=0).stop 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 msgr2=0x7fbfb415d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 0x7fbfb415d160 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8011a80 tx=0x7fbfa80091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 shutdown_connections 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfa0066f30 0x7fbfa00693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfb4169930 0x7fbfb4169da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf9c003a60 0x7fbfb414f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 --2- 172.21.15.33:0/2608495727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfb40adf30 0x7fbfb415d160 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 >> 172.21.15.33:0/2608495727 conn(0x7fbfb40cc930 msgr2=0x7fbfb406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 shutdown_connections 2023-10-06T05:22:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.524+0000 7fbfb3fff700 1 -- 172.21.15.33:0/2608495727 wait complete. 2023-10-06T05:22:28.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.728+0000 7f6642f9d700 1 Processor -- start 2023-10-06T05:22:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.728+0000 7f6642f9d700 1 -- start start 2023-10-06T05:22:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f6642f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f663c155140 0x7f663c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f6642f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f663c0adf50 0x7f663c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f6642f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c0c8e40 con 0x7f663c0c9810 2023-10-06T05:22:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f6642f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0ae3f0 con 0x7f663c0adf50 2023-10-06T05:22:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f6642f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0ae6c0 con 0x7f663c155140 2023-10-06T05:22:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f664179a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f663c0adf50 0x7f663c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f664179a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f663c0adf50 0x7f663c0c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49420/0 (socket says 172.21.15.33:49420) 2023-10-06T05:22:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f664179a700 1 -- 172.21.15.33:0/544174989 learned_addr learned my addr 172.21.15.33:0/544174989 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.730+0000 7f6641f9b700 1 --2- 172.21.15.33:0/544174989 >> v2:172.21.15.167:3300/0 conn(0x7f663c155140 0x7f663c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f664179a700 1 -- 172.21.15.33:0/544174989 >> v2:172.21.15.167:3300/0 conn(0x7f663c155140 msgr2=0x7f663c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f664179a700 1 --2- 172.21.15.33:0/544174989 >> v2:172.21.15.167:3300/0 conn(0x7f663c155140 0x7f663c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f664179a700 1 -- 172.21.15.33:0/544174989 >> v1:172.21.15.33:6789/0 conn(0x7f663c0c9810 legacy=0x7f663c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f664179a700 1 -- 172.21.15.33:0/544174989 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0ae9a0 con 0x7f663c0adf50 2023-10-06T05:22:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f664179a700 1 --2- 172.21.15.33:0/544174989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c0adf50 0x7f663c0c92d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f662c0099b0 tx=0x7f662c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a825f16c0584c6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f6640f99700 1 -- 172.21.15.33:0/544174989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c009ea0 con 0x7f663c0adf50 2023-10-06T05:22:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f6640f99700 1 -- 172.21.15.33:0/544174989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00b820 con 0x7f663c0adf50 2023-10-06T05:22:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.731+0000 7f6640f99700 1 -- 172.21.15.33:0/544174989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00bbc0 con 0x7f663c0adf50 2023-10-06T05:22:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 -- 172.21.15.33:0/544174989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c0adf50 msgr2=0x7f663c0c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 --2- 172.21.15.33:0/544174989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c0adf50 0x7f663c0c92d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f662c0099b0 tx=0x7f662c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 -- 172.21.15.33:0/544174989 shutdown_connections 2023-10-06T05:22:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 --2- 172.21.15.33:0/544174989 >> v2:172.21.15.167:3300/0 conn(0x7f663c155140 0x7f663c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 --2- 172.21.15.33:0/544174989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c0adf50 0x7f663c0c92d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 -- 172.21.15.33:0/544174989 >> 172.21.15.33:0/544174989 conn(0x7f663c06d030 msgr2=0x7f663c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 -- 172.21.15.33:0/544174989 shutdown_connections 2023-10-06T05:22:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.732+0000 7f6642f9d700 1 -- 172.21.15.33:0/544174989 wait complete. 2023-10-06T05:22:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.733+0000 7f6642f9d700 1 Processor -- start 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.733+0000 7f6642f9d700 1 -- start start 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.734+0000 7f6642f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f663c0adf50 0x7f663c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.734+0000 7f6642f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663c0c9810 0x7f663c15a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.734+0000 7f6642f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c155140 0x7f663c15a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.734+0000 7f6642f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0c5890 con 0x7f663c155140 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.734+0000 7f6642f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c159c30 con 0x7f663c0c9810 2023-10-06T05:22:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.734+0000 7f6642f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c159f10 con 0x7f663c0adf50 2023-10-06T05:22:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c155140 0x7f663c15a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f6641f9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f663c0adf50 0x7f663c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664179a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663c0c9810 0x7f663c15a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f6641f9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f663c0adf50 0x7f663c151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55620/0 (socket says 172.21.15.33:55620) 2023-10-06T05:22:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664179a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663c0c9810 0x7f663c15a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:47708/0 (socket says 172.21.15.33:47708) 2023-10-06T05:22:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f6641f9b700 1 -- 172.21.15.33:0/2444305287 learned_addr learned my addr 172.21.15.33:0/2444305287 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664279c700 1 -- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f663c0adf50 msgr2=0x7f663c151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664279c700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f663c0adf50 0x7f663c151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664279c700 1 -- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663c0c9810 msgr2=0x7f663c15a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664279c700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663c0c9810 0x7f663c15a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.735+0000 7f664279c700 1 -- 172.21.15.33:0/2444305287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c0094f0 con 0x7f663c155140 2023-10-06T05:22:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.736+0000 7f664279c700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c155140 0x7f663c15a8d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f663400c830 tx=0x7f6634001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.736+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634012630 con 0x7f663c155140 2023-10-06T05:22:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.736+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634012790 con 0x7f663c155140 2023-10-06T05:22:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.736+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c0c5240 con 0x7f663c155140 2023-10-06T05:22:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.736+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634012b50 con 0x7f663c155140 2023-10-06T05:22:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.736+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c152100 con 0x7f663c155140 2023-10-06T05:22:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.737+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c04c380 con 0x7f663c155140 2023-10-06T05:22:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.738+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f663401a3a0 con 0x7f663c155140 2023-10-06T05:22:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.739+0000 7f6631ffb700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6620066f30 0x7f66200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.739+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f66340844a0 con 0x7f663c155140 2023-10-06T05:22:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.739+0000 7f6641f9b700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6620066f30 0x7f66200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.740+0000 7f6641f9b700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6620066f30 0x7f66200693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f663c153ea0 tx=0x7f66380091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.743+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6634051860 con 0x7f663c155140 2023-10-06T05:22:28.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.949+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f663c06fa40 con 0x7f663c155140 2023-10-06T05:22:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.950+0000 7f6631ffb700 1 -- 172.21.15.33:0/2444305287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+13652 (secure 0 0 0) 0x7f6634054e80 con 0x7f663c155140 2023-10-06T05:22:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:28.952 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":101,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:28.001174+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6620066f30 msgr2=0x7f66200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6620066f30 0x7f66200693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f663c153ea0 tx=0x7f66380091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c155140 msgr2=0x7f663c15a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c155140 0x7f663c15a8d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f663400c830 tx=0x7f6634001690 comp rx=0 tx=0).stop 2023-10-06T05:22:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 shutdown_connections 2023-10-06T05:22:28.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6620066f30 0x7f66200693e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663c155140 0x7f663c15a8d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f663c0adf50 0x7f663c151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 --2- 172.21.15.33:0/2444305287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663c0c9810 0x7f663c15a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 >> 172.21.15.33:0/2444305287 conn(0x7f663c06d030 msgr2=0x7f663c0c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 shutdown_connections 2023-10-06T05:22:28.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:28.952+0000 7f6642f9d700 1 -- 172.21.15.33:0/2444305287 wait complete. 2023-10-06T05:22:28.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:22:28.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:22:29.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.160+0000 7f16c739d700 1 Processor -- start 2023-10-06T05:22:29.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.160+0000 7f16c739d700 1 -- start start 2023-10-06T05:22:29.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.161+0000 7f16c739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16c00cc7e0 0x7f16c00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00b01f0 0x7f16c00b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c739d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c00ad430 con 0x7f16c0156370 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00afe20 con 0x7f16c00cc7e0 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b0b70 con 0x7f16c00b01f0 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16c00cc7e0 0x7f16c00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c639b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f16c0156370 0x7f16c00c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57324/0 (socket says 172.21.15.33:57324) 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c639b700 1 -- 172.21.15.33:0/4167537821 learned_addr learned my addr 172.21.15.33:0/4167537821 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16c00cc7e0 0x7f16c00af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49426/0 (socket says 172.21.15.33:49426) 2023-10-06T05:22:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c6b9c700 1 --2- 172.21.15.33:0/4167537821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00b01f0 0x7f16c00b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5b9a700 1 -- 172.21.15.33:0/4167537821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00b01f0 msgr2=0x7f16c00b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5b9a700 1 --2- 172.21.15.33:0/4167537821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00b01f0 0x7f16c00b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5b9a700 1 -- 172.21.15.33:0/4167537821 >> v1:172.21.15.33:6789/0 conn(0x7f16c0156370 legacy=0x7f16c00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5b9a700 1 -- 172.21.15.33:0/4167537821 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c006bc70 con 0x7f16c00cc7e0 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.162+0000 7f16c5399700 1 -- 172.21.15.33:0/4167537821 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295213258 0 0) 0x7f16c00ad430 con 0x7f16c0156370 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.163+0000 7f16c5b9a700 1 --2- 172.21.15.33:0/4167537821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00cc7e0 0x7f16c00af8e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f16b00098e0 tx=0x7f16b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b3d2bdd4d3e6ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.163+0000 7f16c5399700 1 -- 172.21.15.33:0/4167537821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000b640 con 0x7f16c00cc7e0 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.163+0000 7f16c5399700 1 -- 172.21.15.33:0/4167537821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b000b7a0 con 0x7f16c00cc7e0 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.163+0000 7f16c5399700 1 -- 172.21.15.33:0/4167537821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000bb40 con 0x7f16c00cc7e0 2023-10-06T05:22:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.163+0000 7f16c739d700 1 -- 172.21.15.33:0/4167537821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00cc7e0 msgr2=0x7f16c00af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.163+0000 7f16c739d700 1 --2- 172.21.15.33:0/4167537821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00cc7e0 0x7f16c00af8e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f16b00098e0 tx=0x7f16b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.164+0000 7f16c739d700 1 -- 172.21.15.33:0/4167537821 shutdown_connections 2023-10-06T05:22:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.164+0000 7f16c739d700 1 --2- 172.21.15.33:0/4167537821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00b01f0 0x7f16c00b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.164+0000 7f16c739d700 1 --2- 172.21.15.33:0/4167537821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00cc7e0 0x7f16c00af8e0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.164+0000 7f16c739d700 1 -- 172.21.15.33:0/4167537821 >> 172.21.15.33:0/4167537821 conn(0x7f16c006d030 msgr2=0x7f16c00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.164+0000 7f16c739d700 1 -- 172.21.15.33:0/4167537821 shutdown_connections 2023-10-06T05:22:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.164+0000 7f16c739d700 1 -- 172.21.15.33:0/4167537821 wait complete. 2023-10-06T05:22:29.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.165+0000 7f16c739d700 1 Processor -- start 2023-10-06T05:22:29.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.165+0000 7f16c739d700 1 -- start start 2023-10-06T05:22:29.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 0x7f16c014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00cc7e0 0x7f16c00ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0156370 0x7f16c00ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00ab480 con 0x7f16c00b01f0 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00a55b0 con 0x7f16c0156370 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00aabe0 con 0x7f16c00cc7e0 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 0x7f16c014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c639b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 0x7f16c014ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49434/0 (socket says 172.21.15.33:49434) 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.166+0000 7f16c639b700 1 -- 172.21.15.33:0/4242419949 learned_addr learned my addr 172.21.15.33:0/4242419949 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c5b9a700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00cc7e0 0x7f16c00ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c6b9c700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0156370 0x7f16c00ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c639b700 1 -- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00cc7e0 msgr2=0x7f16c00ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c639b700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00cc7e0 0x7f16c00ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c639b700 1 -- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0156370 msgr2=0x7f16c00ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c639b700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0156370 0x7f16c00ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c639b700 1 -- 172.21.15.33:0/4242419949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0009420 con 0x7f16c00b01f0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c6b9c700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0156370 0x7f16c00ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c639b700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 0x7f16c014ad70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c7e0 tx=0x7f16bc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc005b30 con 0x7f16c00b01f0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc005c90 con 0x7f16c00b01f0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc007300 con 0x7f16c00b01f0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c00aaec0 con 0x7f16c00b01f0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.167+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c014b2b0 con 0x7f16c00b01f0 2023-10-06T05:22:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.168+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c004c380 con 0x7f16c00b01f0 2023-10-06T05:22:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.169+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16bc01b400 con 0x7f16c00b01f0 2023-10-06T05:22:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.170+0000 7f16b67fc700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f30 0x7f16a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.170+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f16bc083f60 con 0x7f16c00b01f0 2023-10-06T05:22:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.171+0000 7f16c5b9a700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f30 0x7f16a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.171+0000 7f16c5b9a700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f30 0x7f16a40693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f16b00093f0 tx=0x7f16b0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.175+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16bc0513a0 con 0x7f16c00b01f0 2023-10-06T05:22:29.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.374+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16c006f310 con 0x7f16c00b01f0 2023-10-06T05:22:29.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.375+0000 7f16b67fc700 1 -- 172.21.15.33:0/4242419949 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16bc0549c0 con 0x7f16c00b01f0 2023-10-06T05:22:29.376 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f30 msgr2=0x7f16a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f30 0x7f16a40693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f16b00093f0 tx=0x7f16b0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 msgr2=0x7f16c014ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 0x7f16c014ad70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c7e0 tx=0x7f16bc007df0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 shutdown_connections 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f30 0x7f16a40693e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0156370 0x7f16c00ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c00b01f0 0x7f16c014ad70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 --2- 172.21.15.33:0/4242419949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c00cc7e0 0x7f16c00ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 >> 172.21.15.33:0/4242419949 conn(0x7f16c006d030 msgr2=0x7f16c006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 shutdown_connections 2023-10-06T05:22:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.377+0000 7f16c739d700 1 -- 172.21.15.33:0/4242419949 wait complete. 2023-10-06T05:22:29.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:22:29.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.584+0000 7faf0179d700 1 Processor -- start 2023-10-06T05:22:29.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.584+0000 7faf0179d700 1 -- start start 2023-10-06T05:22:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf0179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faefc15acd0 0x7faefc0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf0179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 0x7faefc0ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf0179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0b0cd0 con 0x7faefc159850 2023-10-06T05:22:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf0179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc15a840 con 0x7faefc15acd0 2023-10-06T05:22:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf0179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc159f10 con 0x7faefc06efa0 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faefb7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faefc15acd0 0x7faefc0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf00f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 0x7faefc0ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf00f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 0x7faefc0ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55664/0 (socket says 172.21.15.33:55664) 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faf00f9c700 1 -- 172.21.15.33:0/1054456537 learned_addr learned my addr 172.21.15.33:0/1054456537 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faefb7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faefc15acd0 0x7faefc0abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:47740/0 (socket says 172.21.15.33:47740) 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.586+0000 7faefbfff700 1 --1- 172.21.15.33:0/1054456537 >> v1:172.21.15.167:6789/0 conn(0x7faefc159850 0x7faefc06e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57502/0 (socket says 172.21.15.33:57502) 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefb7fe700 1 -- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 msgr2=0x7faefc0ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:29.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefb7fe700 1 --2- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 0x7faefc0ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefb7fe700 1 -- 172.21.15.33:0/1054456537 >> v1:172.21.15.167:6789/0 conn(0x7faefc159850 legacy=0x7faefc06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefb7fe700 1 -- 172.21.15.33:0/1054456537 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc15a1f0 con 0x7faefc15acd0 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faf00f9c700 1 --2- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 0x7faefc0ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefb7fe700 1 --2- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc0abe90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faef00098e0 tx=0x7faef00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6019a2eae6945f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefaffd700 1 -- 172.21.15.33:0/1054456537 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000b640 con 0x7faefc15acd0 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefaffd700 1 -- 172.21.15.33:0/1054456537 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef000b7a0 con 0x7faefc15acd0 2023-10-06T05:22:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.587+0000 7faefaffd700 1 -- 172.21.15.33:0/1054456537 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000bb40 con 0x7faefc15acd0 2023-10-06T05:22:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 -- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 msgr2=0x7faefc0abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 --2- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc0abe90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faef00098e0 tx=0x7faef00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 -- 172.21.15.33:0/1054456537 shutdown_connections 2023-10-06T05:22:29.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 --2- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc06efa0 0x7faefc0ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 --2- 172.21.15.33:0/1054456537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc0abe90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 -- 172.21.15.33:0/1054456537 >> 172.21.15.33:0/1054456537 conn(0x7faefc06cd40 msgr2=0x7faefc0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 -- 172.21.15.33:0/1054456537 shutdown_connections 2023-10-06T05:22:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.588+0000 7faf0179d700 1 -- 172.21.15.33:0/1054456537 wait complete. 2023-10-06T05:22:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.589+0000 7faf0179d700 1 Processor -- start 2023-10-06T05:22:29.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.590+0000 7faf0179d700 1 -- start start 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.590+0000 7faf0179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 0x7faefc14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faf0179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc159850 0x7faefc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faf0179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faf0179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc15ead0 con 0x7faefc06efa0 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faf0179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0c86b0 con 0x7faefc15acd0 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faf0179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0cbbb0 con 0x7faefc159850 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 0x7faefc14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 0x7faefc14bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49448/0 (socket says 172.21.15.33:49448) 2023-10-06T05:22:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 -- 172.21.15.33:0/2391712688 learned_addr learned my addr 172.21.15.33:0/2391712688 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefb7fe700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc159850 0x7faefc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faf00f9c700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 -- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc159850 msgr2=0x7faefc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc159850 0x7faefc15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 -- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 msgr2=0x7faefc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc15dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.591+0000 7faefbfff700 1 -- 172.21.15.33:0/2391712688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009ae0 con 0x7faefc06efa0 2023-10-06T05:22:29.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.592+0000 7faefbfff700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 0x7faefc14bbe0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c7c0 tx=0x7faeec007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.592+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec005a70 con 0x7faefc06efa0 2023-10-06T05:22:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.592+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec005bd0 con 0x7faefc06efa0 2023-10-06T05:22:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.592+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0070d0 con 0x7faefc06efa0 2023-10-06T05:22:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.592+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef0009420 con 0x7faefc06efa0 2023-10-06T05:22:29.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.592+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc0cbf40 con 0x7faefc06efa0 2023-10-06T05:22:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.593+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc04c380 con 0x7faefc06efa0 2023-10-06T05:22:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.594+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faeec007480 con 0x7faefc06efa0 2023-10-06T05:22:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.595+0000 7faee3fff700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faee8066f30 0x7faee80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.595+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5206+0+0 (secure 0 0 0) 0x7faeec0837f0 con 0x7faefc06efa0 2023-10-06T05:22:29.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.595+0000 7faefb7fe700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faee8066f30 0x7faee80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.596+0000 7faefb7fe700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faee8066f30 0x7faee80693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faef00093f0 tx=0x7faef0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.599+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faeec050bb0 con 0x7faefc06efa0 2023-10-06T05:22:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.801+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faefc0cf150 con 0x7faefc06efa0 2023-10-06T05:22:29.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.801+0000 7faee3fff700 1 -- 172.21.15.33:0/2391712688 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7faeec0170a0 con 0x7faefc06efa0 2023-10-06T05:22:29.803 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faee8066f30 msgr2=0x7faee80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faee8066f30 0x7faee80693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faef00093f0 tx=0x7faef0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 msgr2=0x7faefc14bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 0x7faefc14bbe0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c7c0 tx=0x7faeec007f70 comp rx=0 tx=0).stop 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 shutdown_connections 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faee8066f30 0x7faee80693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faefc15acd0 0x7faefc15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faefc06efa0 0x7faefc14bbe0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 --2- 172.21.15.33:0/2391712688 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faefc159850 0x7faefc15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 >> 172.21.15.33:0/2391712688 conn(0x7faefc06cd40 msgr2=0x7faefc15bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 shutdown_connections 2023-10-06T05:22:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:29.804+0000 7faf0179d700 1 -- 172.21.15.33:0/2391712688 wait complete. 2023-10-06T05:22:29.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:22:30.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.008+0000 7f9cd3fff700 1 Processor -- start 2023-10-06T05:22:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.008+0000 7f9cd3fff700 1 -- start start 2023-10-06T05:22:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.009+0000 7f9cd3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cd4159530 0x7f9cd406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.009+0000 7f9cd3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9cd40b04e0 0x7f9cd415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.009+0000 7f9cd3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd406e3e0 con 0x7f9cd406df70 2023-10-06T05:22:30.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.009+0000 7f9cd3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd40b1e00 con 0x7f9cd4159530 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.009+0000 7f9cd3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd406f6b0 con 0x7f9cd40b04e0 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cd4159530 0x7f9cd406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cd4159530 0x7f9cd406da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44536/0 (socket says 172.21.15.33:44536) 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 -- 172.21.15.33:0/11747798 learned_addr learned my addr 172.21.15.33:0/11747798 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd37fe700 1 --2- 172.21.15.33:0/11747798 >> v2:172.21.15.167:3300/0 conn(0x7f9cd40b04e0 0x7f9cd415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 -- 172.21.15.33:0/11747798 >> v2:172.21.15.167:3300/0 conn(0x7f9cd40b04e0 msgr2=0x7f9cd415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 --2- 172.21.15.33:0/11747798 >> v2:172.21.15.167:3300/0 conn(0x7f9cd40b04e0 0x7f9cd415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 -- 172.21.15.33:0/11747798 >> v1:172.21.15.167:6789/0 conn(0x7f9cd406df70 legacy=0x7f9cd40afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.010+0000 7f9cd2ffd700 1 -- 172.21.15.33:0/11747798 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd406f220 con 0x7f9cd4159530 2023-10-06T05:22:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd2ffd700 1 --2- 172.21.15.33:0/11747798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd4159530 0x7f9cd406da30 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40cf3c0 tx=0x7f9cc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bec339150e5225 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd1ffb700 1 -- 172.21.15.33:0/11747798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400b820 con 0x7f9cd4159530 2023-10-06T05:22:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd1ffb700 1 -- 172.21.15.33:0/11747798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400b980 con 0x7f9cd4159530 2023-10-06T05:22:30.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd1ffb700 1 -- 172.21.15.33:0/11747798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400bd20 con 0x7f9cd4159530 2023-10-06T05:22:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd3fff700 1 -- 172.21.15.33:0/11747798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd4159530 msgr2=0x7f9cd406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/11747798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd4159530 0x7f9cd406da30 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40cf3c0 tx=0x7f9cc4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd3fff700 1 -- 172.21.15.33:0/11747798 shutdown_connections 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/11747798 >> v2:172.21.15.167:3300/0 conn(0x7f9cd40b04e0 0x7f9cd415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/11747798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd4159530 0x7f9cd406da30 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.011+0000 7f9cd3fff700 1 -- 172.21.15.33:0/11747798 >> 172.21.15.33:0/11747798 conn(0x7f9cd406c560 msgr2=0x7f9cd406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.012+0000 7f9cd3fff700 1 -- 172.21.15.33:0/11747798 shutdown_connections 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.012+0000 7f9cd3fff700 1 -- 172.21.15.33:0/11747798 wait complete. 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.013+0000 7f9cd3fff700 1 Processor -- start 2023-10-06T05:22:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.013+0000 7f9cd3fff700 1 -- start start 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd406df70 0x7f9cd41503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 0x7f9cd413feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd4159530 0x7f9cd415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd415d830 con 0x7f9cd40b04e0 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd41403f0 con 0x7f9cd406df70 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd416d4c0 con 0x7f9cd4159530 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 0x7f9cd413feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 0x7f9cd413feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44550/0 (socket says 172.21.15.33:44550) 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd27fc700 1 -- 172.21.15.33:0/2012765237 learned_addr learned my addr 172.21.15.33:0/2012765237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd2ffd700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd406df70 0x7f9cd41503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.014+0000 7f9cd37fe700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd4159530 0x7f9cd415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd27fc700 1 -- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd4159530 msgr2=0x7f9cd415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd27fc700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd4159530 0x7f9cd415cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd27fc700 1 -- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd406df70 msgr2=0x7f9cd41503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd27fc700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd406df70 0x7f9cd41503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd27fc700 1 -- 172.21.15.33:0/2012765237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc40094f0 con 0x7f9cd40b04e0 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd2ffd700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd406df70 0x7f9cd41503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.015+0000 7f9cd27fc700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 0x7f9cd413feb0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800c5c0 tx=0x7f9cc800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.017+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8011590 con 0x7f9cd40b04e0 2023-10-06T05:22:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.017+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8011710 con 0x7f9cd40b04e0 2023-10-06T05:22:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.017+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd415d550 con 0x7f9cd40b04e0 2023-10-06T05:22:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.017+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8011ad0 con 0x7f9cd40b04e0 2023-10-06T05:22:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.017+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd416d150 con 0x7f9cd40b04e0 2023-10-06T05:22:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.019+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd416d670 con 0x7f9cd40b04e0 2023-10-06T05:22:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.019+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9cc801b3b0 con 0x7f9cd40b04e0 2023-10-06T05:22:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.021+0000 7f9cbaffd700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.021+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9cc8083830 con 0x7f9cd40b04e0 2023-10-06T05:22:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.021+0000 7f9cd2ffd700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.022+0000 7f9cd2ffd700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cc0066f30 0x7f9cc00693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40cf3c0 tx=0x7f9cc4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.026+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cd416d670 con 0x7f9cd40b04e0 2023-10-06T05:22:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.232+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9cd416d630 con 0x7f9cd40b04e0 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.232+0000 7f9cbaffd700 1 -- 172.21.15.33:0/2012765237 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9cc8017020 con 0x7f9cd40b04e0 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:22:30.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:22:30.235 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:22:30.235 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:22:30.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:22:30.235 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:22:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.234+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cc0066f30 msgr2=0x7f9cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.234+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cc0066f30 0x7f9cc00693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40cf3c0 tx=0x7f9cc4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.234+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 msgr2=0x7f9cd413feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.234+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 0x7f9cd413feb0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800c5c0 tx=0x7f9cc800c8d0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 shutdown_connections 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd4159530 0x7f9cd415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd406df70 0x7f9cd41503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 --2- 172.21.15.33:0/2012765237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cd40b04e0 0x7f9cd413feb0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 >> 172.21.15.33:0/2012765237 conn(0x7f9cd406c560 msgr2=0x7f9cd40b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 shutdown_connections 2023-10-06T05:22:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.235+0000 7f9cd3fff700 1 -- 172.21.15.33:0/2012765237 wait complete. 2023-10-06T05:22:30.253 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:22:30.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:22:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.442+0000 7fe97ed9d700 1 Processor -- start 2023-10-06T05:22:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.442+0000 7fe97ed9d700 1 -- start start 2023-10-06T05:22:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97ed9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe9780ad830 0x7fe978159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97ed9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9780ad3a0 con 0x7fe97806ee20 2023-10-06T05:22:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97ed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9781576f0 con 0x7fe978157320 2023-10-06T05:22:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97ed9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780ac790 con 0x7fe9780ad830 2023-10-06T05:22:30.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe97806ee20 0x7fe9780aecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55898/0 (socket says 172.21.15.33:55898) 2023-10-06T05:22:30.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97e59c700 1 -- 172.21.15.33:0/646972399 learned_addr learned my addr 172.21.15.33:0/646972399 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:30.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.444+0000 7fe97d59a700 1 --2- 172.21.15.33:0/646972399 >> v2:172.21.15.167:3300/0 conn(0x7fe9780ad830 0x7fe978159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.445+0000 7fe97cd99700 1 -- 172.21.15.33:0/646972399 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801546616 0 0) 0x7fe9780ad3a0 con 0x7fe97806ee20 2023-10-06T05:22:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.445+0000 7fe97cd99700 1 -- 172.21.15.33:0/646972399 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe964003570 con 0x7fe97806ee20 2023-10-06T05:22:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.445+0000 7fe97d59a700 1 -- 172.21.15.33:0/646972399 >> v1:172.21.15.167:6789/0 conn(0x7fe978157320 legacy=0x7fe97806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.445+0000 7fe97d59a700 1 -- 172.21.15.33:0/646972399 >> v1:172.21.15.33:6789/0 conn(0x7fe97806ee20 legacy=0x7fe9780aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.445+0000 7fe97d59a700 1 -- 172.21.15.33:0/646972399 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780aca70 con 0x7fe9780ad830 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.445+0000 7fe97d59a700 1 --2- 172.21.15.33:0/646972399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe978159cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9680098e0 tx=0x7fe9680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc739fbde42dd98a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97cd99700 1 -- 172.21.15.33:0/646972399 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800b640 con 0x7fe9780ad830 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97cd99700 1 -- 172.21.15.33:0/646972399 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96800b7a0 con 0x7fe9780ad830 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97cd99700 1 -- 172.21.15.33:0/646972399 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800bb40 con 0x7fe9780ad830 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 -- 172.21.15.33:0/646972399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 msgr2=0x7fe978159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/646972399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe978159cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9680098e0 tx=0x7fe9680091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 -- 172.21.15.33:0/646972399 shutdown_connections 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/646972399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe978159cd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 -- 172.21.15.33:0/646972399 >> 172.21.15.33:0/646972399 conn(0x7fe97806cd40 msgr2=0x7fe9780ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 -- 172.21.15.33:0/646972399 shutdown_connections 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.446+0000 7fe97ed9d700 1 -- 172.21.15.33:0/646972399 wait complete. 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.448+0000 7fe97ed9d700 1 Processor -- start 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.448+0000 7fe97ed9d700 1 -- start start 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 0x7fe9781532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe9780b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978157320 0x7fe9780ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97ed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe978153820 con 0x7fe97806ee20 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97ed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780c8910 con 0x7fe9780ad830 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780b34a0 con 0x7fe978157320 2023-10-06T05:22:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 0x7fe9781532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 0x7fe9781532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44560/0 (socket says 172.21.15.33:44560) 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 -- 172.21.15.33:0/4144756594 learned_addr learned my addr 172.21.15.33:0/4144756594 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97d59a700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe9780b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97e59c700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978157320 0x7fe9780ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 -- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978157320 msgr2=0x7fe9780ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978157320 0x7fe9780ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 -- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 msgr2=0x7fe9780b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe9780b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97dd9b700 1 -- 172.21.15.33:0/4144756594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe968009420 con 0x7fe97806ee20 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.449+0000 7fe97d59a700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe9780b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.450+0000 7fe97dd9b700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 0x7fe9781532e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c4e0 tx=0x7fe97400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.450+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974011590 con 0x7fe97806ee20 2023-10-06T05:22:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.450+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe974011710 con 0x7fe97806ee20 2023-10-06T05:22:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.450+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9780b3780 con 0x7fe97806ee20 2023-10-06T05:22:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.450+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974011ad0 con 0x7fe97806ee20 2023-10-06T05:22:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.450+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9780b30e0 con 0x7fe97806ee20 2023-10-06T05:22:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.451+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe978063ff0 con 0x7fe97806ee20 2023-10-06T05:22:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.452+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe97401b3b0 con 0x7fe97806ee20 2023-10-06T05:22:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.453+0000 7fe96dffb700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe95c066f30 0x7fe95c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.453+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe974083920 con 0x7fe97806ee20 2023-10-06T05:22:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.453+0000 7fe97d59a700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe95c066f30 0x7fe95c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.454+0000 7fe97d59a700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe95c066f30 0x7fe95c0693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe9680093f0 tx=0x7fe968017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.457+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe974050ce0 con 0x7fe97806ee20 2023-10-06T05:22:30.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:30.658+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe9780acd50 con 0x7fe97806ee20 2023-10-06T05:22:30.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T05:22:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.028+0000 7fe96dffb700 1 -- 172.21.15.33:0/4144756594 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe974017070 con 0x7fe97806ee20 2023-10-06T05:22:31.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.032+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe95c066f30 msgr2=0x7fe95c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.032+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe95c066f30 0x7fe95c0693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe9680093f0 tx=0x7fe968017000 comp rx=0 tx=0).stop 2023-10-06T05:22:31.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.032+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 msgr2=0x7fe9781532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.032+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 0x7fe9781532e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c4e0 tx=0x7fe97400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.033+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 shutdown_connections 2023-10-06T05:22:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.033+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe95c066f30 0x7fe95c0693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.033+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe978157320 0x7fe9780ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.033+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe97806ee20 0x7fe9781532e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.033+0000 7fe97ed9d700 1 --2- 172.21.15.33:0/4144756594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9780ad830 0x7fe9780b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.033+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 >> 172.21.15.33:0/4144756594 conn(0x7fe97806cd40 msgr2=0x7fe978153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.034+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 shutdown_connections 2023-10-06T05:22:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.034+0000 7fe97ed9d700 1 -- 172.21.15.33:0/4144756594 wait complete. 2023-10-06T05:22:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:22:31.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:22:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.249+0000 7f15d2f9d700 1 Processor -- start 2023-10-06T05:22:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.249+0000 7f15d2f9d700 1 -- start start 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.250+0000 7f15d2f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15cc155bd0 0x7f15cc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.250+0000 7f15d2f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc06aab0 con 0x7f15cc0acb70 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.250+0000 7f15d2f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc069ea0 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.250+0000 7f15d2f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc06a180 con 0x7f15cc155bd0 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.250+0000 7f15d279c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f15cc0acb70 0x7f15cc160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55908/0 (socket says 172.21.15.33:55908) 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.250+0000 7f15d279c700 1 -- 172.21.15.33:0/2118457160 learned_addr learned my addr 172.21.15.33:0/2118457160 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d1f9b700 1 --2- 172.21.15.33:0/2118457160 >> v2:172.21.15.167:3300/0 conn(0x7f15cc155bd0 0x7f15cc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d0f99700 1 -- 172.21.15.33:0/2118457160 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848461637 0 0) 0x7f15cc06aab0 con 0x7f15cc0acb70 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d0f99700 1 -- 172.21.15.33:0/2118457160 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b8003570 con 0x7f15cc0acb70 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d1f9b700 1 -- 172.21.15.33:0/2118457160 >> v1:172.21.15.167:6789/0 conn(0x7f15cc0c8dc0 legacy=0x7f15cc0ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d0f99700 1 -- 172.21.15.33:0/2118457160 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181419805 0 0) 0x7f15cc069ea0 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d1f9b700 1 -- 172.21.15.33:0/2118457160 >> v1:172.21.15.33:6789/0 conn(0x7f15cc0acb70 legacy=0x7f15cc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.251+0000 7f15d1f9b700 1 -- 172.21.15.33:0/2118457160 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc06a460 con 0x7f15cc155bd0 2023-10-06T05:22:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d1f9b700 1 --2- 172.21.15.33:0/2118457160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc0cc0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f15c80098e0 tx=0x7f15c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684b59ab34ac1fa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d0f99700 1 -- 172.21.15.33:0/2118457160 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800b640 con 0x7f15cc155bd0 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d0f99700 1 -- 172.21.15.33:0/2118457160 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c800b7a0 con 0x7f15cc155bd0 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d0f99700 1 -- 172.21.15.33:0/2118457160 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800bb20 con 0x7f15cc155bd0 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d2f9d700 1 -- 172.21.15.33:0/2118457160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 msgr2=0x7f15cc0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/2118457160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc0cc0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f15c80098e0 tx=0x7f15c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d2f9d700 1 -- 172.21.15.33:0/2118457160 shutdown_connections 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/2118457160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc0cc0c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.252+0000 7f15d2f9d700 1 -- 172.21.15.33:0/2118457160 >> 172.21.15.33:0/2118457160 conn(0x7f15cc06c560 msgr2=0x7f15cc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.253+0000 7f15d2f9d700 1 -- 172.21.15.33:0/2118457160 shutdown_connections 2023-10-06T05:22:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.253+0000 7f15d2f9d700 1 -- 172.21.15.33:0/2118457160 wait complete. 2023-10-06T05:22:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.254+0000 7f15d2f9d700 1 Processor -- start 2023-10-06T05:22:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.254+0000 7f15d2f9d700 1 -- start start 2023-10-06T05:22:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15cc0acb70 0x7f15cc14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d2f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 0x7f15cc154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d2f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc1540c0 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc1543a0 con 0x7f15cc155bd0 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc154680 con 0x7f15cc0acb70 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 0x7f15cc154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 0x7f15cc154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44562/0 (socket says 172.21.15.33:44562) 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d179a700 1 -- 172.21.15.33:0/3541066970 learned_addr learned my addr 172.21.15.33:0/3541066970 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d1f9b700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15cc0acb70 0x7f15cc14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.255+0000 7f15d279c700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d179a700 1 -- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15cc0acb70 msgr2=0x7f15cc14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d179a700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15cc0acb70 0x7f15cc14b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d179a700 1 -- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 msgr2=0x7f15cc150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d179a700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d179a700 1 -- 172.21.15.33:0/3541066970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8009420 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d179a700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 0x7f15cc154e80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00c7e0 tx=0x7f15bc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc012700 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc012880 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc154960 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc012c40 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.256+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc14bd90 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.257+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc154ba0 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.259+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15bc025430 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.259+0000 7f15c1ffb700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15b0066f60 0x7f15b0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.260+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f15bc084a00 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.260+0000 7f15d1f9b700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15b0066f60 0x7f15b0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.261+0000 7f15d1f9b700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15b0066f60 0x7f15b0069410 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f15c80093f0 tx=0x7f15c8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.263+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15bc051cd0 con 0x7f15cc0c8dc0 2023-10-06T05:22:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:31.465+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f15cc154200 con 0x7f15cc0c8dc0 2023-10-06T05:22:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.037+0000 7f15c1ffb700 1 -- 172.21.15.33:0/3541066970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f15cc154200 con 0x7f15cc0c8dc0 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.043+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15b0066f60 msgr2=0x7f15b0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.043+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15b0066f60 0x7f15b0069410 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f15c80093f0 tx=0x7f15c8017000 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.043+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 msgr2=0x7f15cc154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.043+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 0x7f15cc154e80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00c7e0 tx=0x7f15bc007df0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.049+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 shutdown_connections 2023-10-06T05:22:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.049+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f15b0066f60 0x7f15b0069410 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.049+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15cc155bd0 0x7f15cc150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.049+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15cc0acb70 0x7f15cc14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.049+0000 7f15d2f9d700 1 --2- 172.21.15.33:0/3541066970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15cc0c8dc0 0x7f15cc154e80 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.049+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 >> 172.21.15.33:0/3541066970 conn(0x7f15cc06c560 msgr2=0x7f15cc0af470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.050+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 shutdown_connections 2023-10-06T05:22:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.051+0000 7f15d2f9d700 1 -- 172.21.15.33:0/3541066970 wait complete. 2023-10-06T05:22:32.060 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:22:32.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:22:32.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.272+0000 7fde05f9d700 1 Processor -- start 2023-10-06T05:22:32.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.272+0000 7fde05f9d700 1 -- start start 2023-10-06T05:22:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde05f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde0015acd0 0x7fde000abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde05f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fde0006efa0 0x7fde000ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde05f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000b0cd0 con 0x7fde00159850 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde05f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0015a840 con 0x7fde0015acd0 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde05f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde00159f10 con 0x7fde0006efa0 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fddfffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde0015acd0 0x7fde000abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde04f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fde00159850 0x7fde0006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55920/0 (socket says 172.21.15.33:55920) 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.273+0000 7fde04f9b700 1 -- 172.21.15.33:0/4161757970 learned_addr learned my addr 172.21.15.33:0/4161757970 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fde0579c700 1 --2- 172.21.15.33:0/4161757970 >> v2:172.21.15.167:3300/0 conn(0x7fde0006efa0 0x7fde000ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddfffff700 1 -- 172.21.15.33:0/4161757970 >> v2:172.21.15.167:3300/0 conn(0x7fde0006efa0 msgr2=0x7fde000ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddfffff700 1 --2- 172.21.15.33:0/4161757970 >> v2:172.21.15.167:3300/0 conn(0x7fde0006efa0 0x7fde000ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddfffff700 1 -- 172.21.15.33:0/4161757970 >> v1:172.21.15.33:6789/0 conn(0x7fde00159850 legacy=0x7fde0006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddff7fe700 1 -- 172.21.15.33:0/4161757970 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147122468 0 0) 0x7fde000b0cd0 con 0x7fde00159850 2023-10-06T05:22:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddfffff700 1 -- 172.21.15.33:0/4161757970 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0015a1f0 con 0x7fde0015acd0 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddfffff700 1 --2- 172.21.15.33:0/4161757970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0015acd0 0x7fde000abe90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fde0006afb0 tx=0x7fddec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca831e1fa26c907 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.274+0000 7fddff7fe700 1 -- 172.21.15.33:0/4161757970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00b6b0 con 0x7fde0015acd0 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fddff7fe700 1 -- 172.21.15.33:0/4161757970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00b810 con 0x7fde0015acd0 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fddff7fe700 1 -- 172.21.15.33:0/4161757970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00bbb0 con 0x7fde0015acd0 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 -- 172.21.15.33:0/4161757970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0015acd0 msgr2=0x7fde000abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 --2- 172.21.15.33:0/4161757970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0015acd0 0x7fde000abe90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fde0006afb0 tx=0x7fddec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 -- 172.21.15.33:0/4161757970 shutdown_connections 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 --2- 172.21.15.33:0/4161757970 >> v2:172.21.15.167:3300/0 conn(0x7fde0006efa0 0x7fde000ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 --2- 172.21.15.33:0/4161757970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0015acd0 0x7fde000abe90 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 -- 172.21.15.33:0/4161757970 >> 172.21.15.33:0/4161757970 conn(0x7fde0006cd40 msgr2=0x7fde000ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 -- 172.21.15.33:0/4161757970 shutdown_connections 2023-10-06T05:22:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.275+0000 7fde05f9d700 1 -- 172.21.15.33:0/4161757970 wait complete. 2023-10-06T05:22:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.277+0000 7fde05f9d700 1 Processor -- start 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.277+0000 7fde05f9d700 1 -- start start 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde05f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0006efa0 0x7fde0016a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde05f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 0x7fde000ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde05f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde0015acd0 0x7fde0016bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde05f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000abe50 con 0x7fde00159850 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde05f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000ac130 con 0x7fde0006efa0 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde05f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde00143b50 con 0x7fde0015acd0 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 0x7fde000ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 0x7fde000ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44570/0 (socket says 172.21.15.33:44570) 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 -- 172.21.15.33:0/3334572802 learned_addr learned my addr 172.21.15.33:0/3334572802 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde04f9b700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0006efa0 0x7fde0016a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fde0579c700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde0015acd0 0x7fde0016bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 -- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde0015acd0 msgr2=0x7fde0016bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde0015acd0 0x7fde0016bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 -- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0006efa0 msgr2=0x7fde0016a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0006efa0 0x7fde0016a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.278+0000 7fddfffff700 1 -- 172.21.15.33:0/3334572802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009420 con 0x7fde00159850 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.279+0000 7fddfffff700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 0x7fde000ccd40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fde00065080 tx=0x7fddec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.279+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec000d10 con 0x7fde00159850 2023-10-06T05:22:32.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.279+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00bb90 con 0x7fde00159850 2023-10-06T05:22:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.279+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec011650 con 0x7fde00159850 2023-10-06T05:22:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.279+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000cd690 con 0x7fde00159850 2023-10-06T05:22:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.279+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde000cdc50 con 0x7fde00159850 2023-10-06T05:22:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.280+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde000ac2a0 con 0x7fde00159850 2023-10-06T05:22:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.281+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddec011a80 con 0x7fde00159850 2023-10-06T05:22:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.282+0000 7fddfcff9700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde4066f30 0x7fdde40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.282+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fddec085e90 con 0x7fde00159850 2023-10-06T05:22:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.283+0000 7fde04f9b700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde4066f30 0x7fdde40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.283+0000 7fde04f9b700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde4066f30 0x7fdde40693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fde000b0df0 tx=0x7fddf4007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.287+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddec052f60 con 0x7fde00159850 2023-10-06T05:22:32.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:32.536+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fde0004c380 con 0x7fde00159850 2023-10-06T05:22:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.041+0000 7fddfcff9700 1 -- 172.21.15.33:0/3334572802 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7fddec056580 con 0x7fde00159850 2023-10-06T05:22:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.050+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde4066f30 msgr2=0x7fdde40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.050+0000 7fde05f9d700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde4066f30 0x7fdde40693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fde000b0df0 tx=0x7fddf4007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.050+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 msgr2=0x7fde000ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.050+0000 7fde05f9d700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 0x7fde000ccd40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fde00065080 tx=0x7fddec017000 comp rx=0 tx=0).stop 2023-10-06T05:22:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.052+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 shutdown_connections 2023-10-06T05:22:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.052+0000 7fde05f9d700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdde4066f30 0x7fdde40693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.052+0000 7fde05f9d700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde0015acd0 0x7fde0016bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.052+0000 7fde05f9d700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde0006efa0 0x7fde0016a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.052+0000 7fde05f9d700 1 --2- 172.21.15.33:0/3334572802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00159850 0x7fde000ccd40 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.052+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 >> 172.21.15.33:0/3334572802 conn(0x7fde0006cd40 msgr2=0x7fde000691b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.059+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 shutdown_connections 2023-10-06T05:22:33.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.060+0000 7fde05f9d700 1 -- 172.21.15.33:0/3334572802 wait complete. 2023-10-06T05:22:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 15 and data pool 16 2023-10-06T05:22:33.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.277+0000 7efd6ad9d700 1 Processor -- start 2023-10-06T05:22:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.277+0000 7efd6ad9d700 1 -- start start 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.278+0000 7efd6ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd641571f0 0x7efd640afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.278+0000 7efd6ad9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efd641575c0 0x7efd640b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.278+0000 7efd6ad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd640ccda0 0x7efd640cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.278+0000 7efd6ad9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640ad8c0 con 0x7efd641571f0 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.278+0000 7efd6ad9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640b1ae0 con 0x7efd641575c0 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.278+0000 7efd6ad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640b17b0 con 0x7efd640ccda0 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd641571f0 0x7efd640afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd641571f0 0x7efd640afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44582/0 (socket says 172.21.15.33:44582) 2023-10-06T05:22:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 -- 172.21.15.33:0/3845424794 learned_addr learned my addr 172.21.15.33:0/3845424794 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd6a59c700 1 --2- 172.21.15.33:0/3845424794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd640ccda0 0x7efd640cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd6959a700 1 --2- 172.21.15.33:0/3845424794 >> v2:172.21.15.167:3300/0 conn(0x7efd641575c0 0x7efd640b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 -- 172.21.15.33:0/3845424794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd640ccda0 msgr2=0x7efd640cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 --2- 172.21.15.33:0/3845424794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd640ccda0 0x7efd640cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 -- 172.21.15.33:0/3845424794 >> v2:172.21.15.167:3300/0 conn(0x7efd641575c0 msgr2=0x7efd640b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 --2- 172.21.15.33:0/3845424794 >> v2:172.21.15.167:3300/0 conn(0x7efd641575c0 0x7efd640b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 -- 172.21.15.33:0/3845424794 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640cd740 con 0x7efd641571f0 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd6959a700 1 --2- 172.21.15.33:0/3845424794 >> v2:172.21.15.167:3300/0 conn(0x7efd641575c0 0x7efd640b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.279+0000 7efd69d9b700 1 --2- 172.21.15.33:0/3845424794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640afdd0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efd640b1a50 tx=0x7efd60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19a8623895247d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd68d99700 1 -- 172.21.15.33:0/3845424794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60009e70 con 0x7efd641571f0 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd68d99700 1 -- 172.21.15.33:0/3845424794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6000b860 con 0x7efd641571f0 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd68d99700 1 -- 172.21.15.33:0/3845424794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6000bc00 con 0x7efd641571f0 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 -- 172.21.15.33:0/3845424794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 msgr2=0x7efd640afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/3845424794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640afdd0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efd640b1a50 tx=0x7efd60009250 comp rx=0 tx=0).stop 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 -- 172.21.15.33:0/3845424794 shutdown_connections 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/3845424794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd640ccda0 0x7efd640cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/3845424794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640afdd0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/3845424794 >> v2:172.21.15.167:3300/0 conn(0x7efd641575c0 0x7efd640b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 -- 172.21.15.33:0/3845424794 >> 172.21.15.33:0/3845424794 conn(0x7efd6406c560 msgr2=0x7efd6406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.280+0000 7efd6ad9d700 1 -- 172.21.15.33:0/3845424794 shutdown_connections 2023-10-06T05:22:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.281+0000 7efd6ad9d700 1 -- 172.21.15.33:0/3845424794 wait complete. 2023-10-06T05:22:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.282+0000 7efd6ad9d700 1 Processor -- start 2023-10-06T05:22:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.282+0000 7efd6ad9d700 1 -- start start 2023-10-06T05:22:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd640ccda0 0x7efd64152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6ad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd641575c0 0x7efd640ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6ad9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64159920 con 0x7efd641571f0 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6ad9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640c3450 con 0x7efd640ccda0 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6ad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd64158a30 con 0x7efd641575c0 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6959a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6959a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44592/0 (socket says 172.21.15.33:44592) 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6959a700 1 -- 172.21.15.33:0/2824371163 learned_addr learned my addr 172.21.15.33:0/2824371163 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd6a59c700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd641575c0 0x7efd640ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.283+0000 7efd69d9b700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd640ccda0 0x7efd64152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6959a700 1 -- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd641575c0 msgr2=0x7efd640ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6959a700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd641575c0 0x7efd640ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6959a700 1 -- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd640ccda0 msgr2=0x7efd64152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6959a700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd640ccda0 0x7efd64152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6959a700 1 -- 172.21.15.33:0/2824371163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600094f0 con 0x7efd641571f0 2023-10-06T05:22:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6959a700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640c9a30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d860 tx=0x7efd54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540125f0 con 0x7efd641571f0 2023-10-06T05:22:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54012750 con 0x7efd641571f0 2023-10-06T05:22:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54012af0 con 0x7efd641571f0 2023-10-06T05:22:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.284+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd64158d10 con 0x7efd641571f0 2023-10-06T05:22:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.285+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd641592d0 con 0x7efd641571f0 2023-10-06T05:22:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.285+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6404c380 con 0x7efd641571f0 2023-10-06T05:22:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.286+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd54013480 con 0x7efd641571f0 2023-10-06T05:22:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.287+0000 7efd59ffb700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd48066f30 0x7efd480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.287+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efd54084730 con 0x7efd641571f0 2023-10-06T05:22:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.288+0000 7efd69d9b700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd48066f30 0x7efd480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.289+0000 7efd69d9b700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd48066f30 0x7efd480693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efd640b1a50 tx=0x7efd60017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.292+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd54051840 con 0x7efd641571f0 2023-10-06T05:22:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.492+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd640cd9f0 con 0x7efd641571f0 2023-10-06T05:22:33.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.493+0000 7efd59ffb700 1 -- 172.21.15.33:0/2824371163 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16893 (secure 0 0 0) 0x7efd54054e60 con 0x7efd641571f0 2023-10-06T05:22:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:33.495 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":104,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:32.538073+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.698838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.505677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:33.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd48066f30 msgr2=0x7efd480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd48066f30 0x7efd480693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efd640b1a50 tx=0x7efd60017000 comp rx=0 tx=0).stop 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 msgr2=0x7efd640c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640c9a30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d860 tx=0x7efd54008040 comp rx=0 tx=0).stop 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 shutdown_connections 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd48066f30 0x7efd480693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd641575c0 0x7efd640ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd640ccda0 0x7efd64152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 --2- 172.21.15.33:0/2824371163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd641571f0 0x7efd640c9a30 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 >> 172.21.15.33:0/2824371163 conn(0x7efd6406c560 msgr2=0x7efd6415bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 shutdown_connections 2023-10-06T05:22:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.495+0000 7efd6ad9d700 1 -- 172.21.15.33:0/2824371163 wait complete. 2023-10-06T05:22:33.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:33.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.703+0000 7f70f9d9d700 1 Processor -- start 2023-10-06T05:22:33.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.703+0000 7f70f9d9d700 1 -- start start 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.704+0000 7f70f9d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f40ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.704+0000 7f70f9d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f70f41575c0 0x7f70f40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.704+0000 7f70f9d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f40b1ae0 con 0x7f70f41571f0 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.704+0000 7f70f9d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40ad8c0 con 0x7f70f41575c0 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.704+0000 7f70f9d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40b17b0 con 0x7f70f40b04e0 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f70f41575c0 0x7f70f40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f70f41575c0 0x7f70f40cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44606/0 (socket says 172.21.15.33:44606) 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 -- 172.21.15.33:0/2183947458 learned_addr learned my addr 172.21.15.33:0/2183947458 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70ebfff700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f40ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 -- 172.21.15.33:0/2183947458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 msgr2=0x7f70f40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f40ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 -- 172.21.15.33:0/2183947458 >> v1:172.21.15.167:6789/0 conn(0x7f70f41571f0 legacy=0x7f70f40afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 -- 172.21.15.33:0/2183947458 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40cd9f0 con 0x7f70f41575c0 2023-10-06T05:22:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70ebfff700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.705+0000 7f70f959c700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41575c0 0x7f70f40cd2e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009f20 tx=0x7f70ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fedefdcfc85cf49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70eb7fe700 1 -- 172.21.15.33:0/2183947458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00e070 con 0x7f70f41575c0 2023-10-06T05:22:33.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70eb7fe700 1 -- 172.21.15.33:0/2183947458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00b9d0 con 0x7f70f41575c0 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70eb7fe700 1 -- 172.21.15.33:0/2183947458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0092e0 con 0x7f70f41575c0 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 -- 172.21.15.33:0/2183947458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41575c0 msgr2=0x7f70f40cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41575c0 0x7f70f40cd2e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009f20 tx=0x7f70ec009840 comp rx=0 tx=0).stop 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 -- 172.21.15.33:0/2183947458 shutdown_connections 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41575c0 0x7f70f40cd2e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/2183947458 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 -- 172.21.15.33:0/2183947458 >> 172.21.15.33:0/2183947458 conn(0x7f70f406c560 msgr2=0x7f70f406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 -- 172.21.15.33:0/2183947458 shutdown_connections 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.706+0000 7f70f9d9d700 1 -- 172.21.15.33:0/2183947458 wait complete. 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.708+0000 7f70f9d9d700 1 Processor -- start 2023-10-06T05:22:33.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.708+0000 7f70f9d9d700 1 -- start start 2023-10-06T05:22:33.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f9d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f4141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f9d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 0x7f70f415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f9d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70f41575c0 0x7f70f4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f9d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f415c800 con 0x7f70f41571f0 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f9d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f415bbf0 con 0x7f70f41575c0 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f9d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f415bed0 con 0x7f70f40b04e0 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 0x7f70f415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 0x7f70f415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44612/0 (socket says 172.21.15.33:44612) 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70ebfff700 1 -- 172.21.15.33:0/1688899237 learned_addr learned my addr 172.21.15.33:0/1688899237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.709+0000 7f70f8d9b700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f4141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70f959c700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70f41575c0 0x7f70f4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70ebfff700 1 -- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 msgr2=0x7f70f4141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70ebfff700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f4141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70ebfff700 1 -- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70f41575c0 msgr2=0x7f70f4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70ebfff700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70f41575c0 0x7f70f4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70ebfff700 1 -- 172.21.15.33:0/1688899237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4009420 con 0x7f70f41571f0 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70ebfff700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 0x7f70f415b2c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f70e400ec80 tx=0x7f70e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400c9d0 con 0x7f70f41571f0 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e400cb30 con 0x7f70f41571f0 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400ceb0 con 0x7f70f41571f0 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec009ae0 con 0x7f70f41571f0 2023-10-06T05:22:33.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.710+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f415c490 con 0x7f70f41571f0 2023-10-06T05:22:33.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.711+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f404c380 con 0x7f70f41571f0 2023-10-06T05:22:33.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.712+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70e4013680 con 0x7f70f41571f0 2023-10-06T05:22:33.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.713+0000 7f70e8ff9700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f70e0066f30 0x7f70e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.713+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f70e4084ae0 con 0x7f70f41571f0 2023-10-06T05:22:33.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.714+0000 7f70f8d9b700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f70e0066f30 0x7f70e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.715+0000 7f70f8d9b700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f70e0066f30 0x7f70e00693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f70f0003860 tx=0x7f70f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.717+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70e4051bf0 con 0x7f70f41571f0 2023-10-06T05:22:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.923+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70f415c940 con 0x7f70f41571f0 2023-10-06T05:22:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.924+0000 7f70e8ff9700 1 -- 172.21.15.33:0/1688899237 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f70f415c940 con 0x7f70f41571f0 2023-10-06T05:22:33.927 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:33.927 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":104,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:32.538073+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.698838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.505677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.926+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f70e0066f30 msgr2=0x7f70e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.926+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f70e0066f30 0x7f70e00693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f70f0003860 tx=0x7f70f0009040 comp rx=0 tx=0).stop 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 msgr2=0x7f70f415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 0x7f70f415b2c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f70e400ec80 tx=0x7f70e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 shutdown_connections 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f70e0066f30 0x7f70e00693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70f41575c0 0x7f70f4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70f40b04e0 0x7f70f4141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 --2- 172.21.15.33:0/1688899237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f41571f0 0x7f70f415b2c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 >> 172.21.15.33:0/1688899237 conn(0x7f70f406c560 msgr2=0x7f70f41501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 shutdown_connections 2023-10-06T05:22:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:33.927+0000 7f70f9d9d700 1 -- 172.21.15.33:0/1688899237 wait complete. 2023-10-06T05:22:33.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:22:34.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.137+0000 7f66eed9d700 1 Processor -- start 2023-10-06T05:22:34.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.137+0000 7f66eed9d700 1 -- start start 2023-10-06T05:22:34.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66eed9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66e80abbd0 0x7f66e8067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66eed9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e806ae10 con 0x7f66e80b1f90 2023-10-06T05:22:34.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66eed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e8068580 con 0x7f66e8068140 2023-10-06T05:22:34.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66eed9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e80ac050 con 0x7f66e80abbd0 2023-10-06T05:22:34.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66edd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f66e80b1f90 0x7f66e8070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55930/0 (socket says 172.21.15.33:55930) 2023-10-06T05:22:34.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66edd9b700 1 -- 172.21.15.33:0/2301108346 learned_addr learned my addr 172.21.15.33:0/2301108346 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:34.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805359596 0 0) 0x7f66e806ae10 con 0x7f66e80b1f90 2023-10-06T05:22:34.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66ed59a700 1 --2- 172.21.15.33:0/2301108346 >> v2:172.21.15.167:3300/0 conn(0x7f66e80abbd0 0x7f66e8067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.139+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66d4003570 con 0x7f66e80b1f90 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095119900 0 0) 0x7f66d4003570 con 0x7f66e80b1f90 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 >> v2:172.21.15.167:3300/0 conn(0x7f66e80abbd0 msgr2=0x7f66e8067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 --2- 172.21.15.33:0/2301108346 >> v2:172.21.15.167:3300/0 conn(0x7f66e80abbd0 0x7f66e8067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 >> v1:172.21.15.167:6789/0 conn(0x7f66e8068140 legacy=0x7f66e8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e806a4e0 con 0x7f66e80b1f90 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f66e4001470 con 0x7f66e80b1f90 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 >> v1:172.21.15.33:6789/0 conn(0x7f66e80b1f90 legacy=0x7f66e8070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d4003a60 0x7f66d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d4004810 0x7f66d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d40051c0 0x7f66d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80ac050 con 0x7f66d40051c0 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d400c160 con 0x7f66d4004810 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d400c320 con 0x7f66d4003a60 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ee59c700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d40051c0 0x7f66d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66edd9b700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d4003a60 0x7f66d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.140+0000 7f66ed59a700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d4004810 0x7f66d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 -- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d4003a60 msgr2=0x7f66d4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d4003a60 0x7f66d4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 -- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d4004810 msgr2=0x7f66d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d4004810 0x7f66d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 -- 172.21.15.33:0/2301108346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d4003570 con 0x7f66d40051c0 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66edd9b700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d4003a60 0x7f66d4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:34.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ed59a700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d4004810 0x7f66d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 -- 172.21.15.33:0/2301108346 reap_dead start 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ee59c700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d40051c0 0x7f66d400ba60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001fc0 tx=0x7f66e0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c3fa35b3b6025 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0001950 con 0x7f66d40051c0 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e000c8b0 con 0x7f66d40051c0 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66ecd99700 1 -- 172.21.15.33:0/2301108346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000cc90 con 0x7f66d40051c0 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66eed9d700 1 -- 172.21.15.33:0/2301108346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d40051c0 msgr2=0x7f66d400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.141+0000 7f66eed9d700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d40051c0 0x7f66d400ba60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f66e0001fc0 tx=0x7f66e0001640 comp rx=0 tx=0).stop 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.142+0000 7f66eed9d700 1 -- 172.21.15.33:0/2301108346 shutdown_connections 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.142+0000 7f66eed9d700 1 --2- 172.21.15.33:0/2301108346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d40051c0 0x7f66d400ba60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.142+0000 7f66eed9d700 1 -- 172.21.15.33:0/2301108346 >> 172.21.15.33:0/2301108346 conn(0x7f66e806ea70 msgr2=0x7f66e806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.142+0000 7f66eed9d700 1 -- 172.21.15.33:0/2301108346 shutdown_connections 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.142+0000 7f66eed9d700 1 -- 172.21.15.33:0/2301108346 wait complete. 2023-10-06T05:22:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.143+0000 7f66eed9d700 1 Processor -- start 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.143+0000 7f66eed9d700 1 -- start start 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66eed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d40051c0 0x7f66e8145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66eed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 0x7f66e8157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66eed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e81559c0 0x7f66e8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66eed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8145bd0 con 0x7f66e81577c0 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66eed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8158170 con 0x7f66d40051c0 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66eed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e815d510 con 0x7f66e81559c0 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66ed59a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 0x7f66e8157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66ed59a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 0x7f66e8157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44636/0 (socket says 172.21.15.33:44636) 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.144+0000 7f66ed59a700 1 -- 172.21.15.33:0/899734588 learned_addr learned my addr 172.21.15.33:0/899734588 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ee59c700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e81559c0 0x7f66e8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66edd9b700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d40051c0 0x7f66e8145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ed59a700 1 -- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e81559c0 msgr2=0x7f66e8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ed59a700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e81559c0 0x7f66e8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ed59a700 1 -- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d40051c0 msgr2=0x7f66e8145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ed59a700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d40051c0 0x7f66e8145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ed59a700 1 -- 172.21.15.33:0/899734588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e81566f0 con 0x7f66e81577c0 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ee59c700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e81559c0 0x7f66e8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ed59a700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 0x7f66e8157c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f66d8011730 tx=0x7f66d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8015740 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d80158a0 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8015c40 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.145+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e81569d0 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.146+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e815d1a0 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.146+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e804c380 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.147+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66d8013410 con 0x7f66e81577c0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.148+0000 7f66ddffb700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66cc066f30 0x7f66cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.148+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f66d8084de0 con 0x7f66e81577c0 2023-10-06T05:22:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.149+0000 7f66edd9b700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66cc066f30 0x7f66cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.149+0000 7f66edd9b700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66cc066f30 0x7f66cc0693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f66e4001200 tx=0x7f66e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.153+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66d8051e50 con 0x7f66e81577c0 2023-10-06T05:22:34.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:34.404+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f66e815acc0 con 0x7f66e81577c0 2023-10-06T05:22:35.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.061+0000 7f66ddffb700 1 -- 172.21.15.33:0/899734588 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7f66d80220b0 con 0x7f66e81577c0 2023-10-06T05:22:35.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.064+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66cc066f30 msgr2=0x7f66cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.064+0000 7f66eed9d700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66cc066f30 0x7f66cc0693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f66e4001200 tx=0x7f66e4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:35.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.064+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 msgr2=0x7f66e8157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.064+0000 7f66eed9d700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 0x7f66e8157c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f66d8011730 tx=0x7f66d8008040 comp rx=0 tx=0).stop 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 shutdown_connections 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66cc066f30 0x7f66cc0693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66e81559c0 0x7f66e8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d40051c0 0x7f66e8145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 --2- 172.21.15.33:0/899734588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e81577c0 0x7f66e8157c30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 >> 172.21.15.33:0/899734588 conn(0x7f66e806ea70 msgr2=0x7f66e8153530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 shutdown_connections 2023-10-06T05:22:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.065+0000 7f66eed9d700 1 -- 172.21.15.33:0/899734588 wait complete. 2023-10-06T05:22:35.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:22:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.281+0000 7f995e39d700 1 Processor -- start 2023-10-06T05:22:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.282+0000 7f995e39d700 1 -- start start 2023-10-06T05:22:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995e39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9958155a50 0x7f9958155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995e39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99580afb20 0x7f99580caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995e39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99580b06b0 con 0x7f99580c9490 2023-10-06T05:22:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995e39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99580c99e0 con 0x7f99580afb20 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995e39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99580c9cc0 con 0x7f9958155a50 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995db9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99580afb20 0x7f99580caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995db9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99580afb20 0x7f99580caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44640/0 (socket says 172.21.15.33:44640) 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.283+0000 7f995db9c700 1 -- 172.21.15.33:0/193155850 learned_addr learned my addr 172.21.15.33:0/193155850 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f995d39b700 1 --2- 172.21.15.33:0/193155850 >> v2:172.21.15.167:3300/0 conn(0x7f9958155a50 0x7f9958155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f995db9c700 1 -- 172.21.15.33:0/193155850 >> v2:172.21.15.167:3300/0 conn(0x7f9958155a50 msgr2=0x7f9958155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f995db9c700 1 --2- 172.21.15.33:0/193155850 >> v2:172.21.15.167:3300/0 conn(0x7f9958155a50 0x7f9958155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f995db9c700 1 -- 172.21.15.33:0/193155850 >> v1:172.21.15.167:6789/0 conn(0x7f99580c9490 legacy=0x7f99580af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f995db9c700 1 -- 172.21.15.33:0/193155850 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580c9fa0 con 0x7f99580afb20 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f995db9c700 1 --2- 172.21.15.33:0/193155850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580afb20 0x7f99580caa80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f99580cd1e0 tx=0x7f995000a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0e44d30758e39e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f994ffff700 1 -- 172.21.15.33:0/193155850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000f070 con 0x7f99580afb20 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f994ffff700 1 -- 172.21.15.33:0/193155850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000a2e0 con 0x7f99580afb20 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.284+0000 7f994ffff700 1 -- 172.21.15.33:0/193155850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000cdc0 con 0x7f99580afb20 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 -- 172.21.15.33:0/193155850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580afb20 msgr2=0x7f99580caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 --2- 172.21.15.33:0/193155850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580afb20 0x7f99580caa80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f99580cd1e0 tx=0x7f995000a950 comp rx=0 tx=0).stop 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 -- 172.21.15.33:0/193155850 shutdown_connections 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 --2- 172.21.15.33:0/193155850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99580afb20 0x7f99580caa80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 --2- 172.21.15.33:0/193155850 >> v2:172.21.15.167:3300/0 conn(0x7f9958155a50 0x7f9958155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 -- 172.21.15.33:0/193155850 >> 172.21.15.33:0/193155850 conn(0x7f995806d030 msgr2=0x7f99580cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 -- 172.21.15.33:0/193155850 shutdown_connections 2023-10-06T05:22:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.285+0000 7f995e39d700 1 -- 172.21.15.33:0/193155850 wait complete. 2023-10-06T05:22:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.286+0000 7f995e39d700 1 Processor -- start 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.287+0000 7f995e39d700 1 -- start start 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.287+0000 7f995e39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99580afb20 0x7f995814ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995e39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580c9490 0x7f9958151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995e39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 0x7f9958158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995e39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9958169a50 con 0x7f9958155a50 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995e39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9958169d30 con 0x7f99580c9490 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995e39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9958151cf0 con 0x7f99580afb20 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 0x7f9958158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 0x7f9958158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44648/0 (socket says 172.21.15.33:44648) 2023-10-06T05:22:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 -- 172.21.15.33:0/1063173850 learned_addr learned my addr 172.21.15.33:0/1063173850 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:35.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995cb9a700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580c9490 0x7f9958151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995d39b700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99580afb20 0x7f995814ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 -- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99580afb20 msgr2=0x7f995814ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99580afb20 0x7f995814ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 -- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580c9490 msgr2=0x7f9958151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580c9490 0x7f9958151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995db9c700 1 -- 172.21.15.33:0/1063173850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995000abf0 con 0x7f9958155a50 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995d39b700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99580afb20 0x7f995814ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.288+0000 7f995cb9a700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580c9490 0x7f9958151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.289+0000 7f995db9c700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 0x7f9958158b10 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f995000abc0 tx=0x7f9950018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.289+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000a5f0 con 0x7f9958155a50 2023-10-06T05:22:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.289+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950008290 con 0x7f9958155a50 2023-10-06T05:22:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.289+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950013680 con 0x7f9958155a50 2023-10-06T05:22:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.289+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9958151fd0 con 0x7f9958155a50 2023-10-06T05:22:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.289+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995814f470 con 0x7f9958155a50 2023-10-06T05:22:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.290+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9958169ea0 con 0x7f9958155a50 2023-10-06T05:22:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.291+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f995001a070 con 0x7f9958155a50 2023-10-06T05:22:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.292+0000 7f994d7fa700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f993c066f30 0x7f993c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.292+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9950085d20 con 0x7f9958155a50 2023-10-06T05:22:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.292+0000 7f995d39b700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f993c066f30 0x7f993c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.293+0000 7f995d39b700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f993c066f30 0x7f993c0693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9954006fd0 tx=0x7f9954008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.296+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9950052d90 con 0x7f9958155a50 2023-10-06T05:22:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:35.544+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f995804c380 con 0x7f9958155a50 2023-10-06T05:22:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.068+0000 7f994d7fa700 1 -- 172.21.15.33:0/1063173850 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99500210c0 con 0x7f9958155a50 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f993c066f30 msgr2=0x7f993c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f993c066f30 0x7f993c0693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9954006fd0 tx=0x7f9954008040 comp rx=0 tx=0).stop 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 msgr2=0x7f9958158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 0x7f9958158b10 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f995000abc0 tx=0x7f9950018000 comp rx=0 tx=0).stop 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 shutdown_connections 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f993c066f30 0x7f993c0693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9958155a50 0x7f9958158b10 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99580afb20 0x7f995814ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 --2- 172.21.15.33:0/1063173850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99580c9490 0x7f9958151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 >> 172.21.15.33:0/1063173850 conn(0x7f995806d030 msgr2=0x7f99580c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 shutdown_connections 2023-10-06T05:22:36.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.071+0000 7f995e39d700 1 -- 172.21.15.33:0/1063173850 wait complete. 2023-10-06T05:22:36.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:36.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.279+0000 7f33cd99d700 1 Processor -- start 2023-10-06T05:22:36.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.279+0000 7f33cd99d700 1 -- start start 2023-10-06T05:22:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cd99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33c813faa0 0x7f33c813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 0x7f33c80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cd99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80c9cb0 con 0x7f33c806aad0 2023-10-06T05:22:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cd99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c806b660 con 0x7f33c813faa0 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80ca270 con 0x7f33c80c7940 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 0x7f33c80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cc99b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f33c813faa0 0x7f33c813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 0x7f33c80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:58666/0 (socket says 172.21.15.33:58666) 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33c7fff700 1 -- 172.21.15.33:0/3814844033 learned_addr learned my addr 172.21.15.33:0/3814844033 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.281+0000 7f33cd19c700 1 --1- 172.21.15.33:0/3814844033 >> v1:172.21.15.167:6789/0 conn(0x7f33c806aad0 0x7f33c806af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:38190/0 (socket says 172.21.15.33:38190) 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c7fff700 1 -- 172.21.15.33:0/3814844033 >> v2:172.21.15.167:3300/0 conn(0x7f33c813faa0 msgr2=0x7f33c813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c7fff700 1 --2- 172.21.15.33:0/3814844033 >> v2:172.21.15.167:3300/0 conn(0x7f33c813faa0 0x7f33c813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c7fff700 1 -- 172.21.15.33:0/3814844033 >> v1:172.21.15.167:6789/0 conn(0x7f33c806aad0 legacy=0x7f33c806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c7fff700 1 -- 172.21.15.33:0/3814844033 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c806b840 con 0x7f33c80c7940 2023-10-06T05:22:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c7fff700 1 --2- 172.21.15.33:0/3814844033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 0x7f33c80c7db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33b40099b0 tx=0x7f33b4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12518df7076f620 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c77fe700 1 -- 172.21.15.33:0/3814844033 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4009ea0 con 0x7f33c80c7940 2023-10-06T05:22:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c77fe700 1 -- 172.21.15.33:0/3814844033 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400b860 con 0x7f33c80c7940 2023-10-06T05:22:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.282+0000 7f33c77fe700 1 -- 172.21.15.33:0/3814844033 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400bc00 con 0x7f33c80c7940 2023-10-06T05:22:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 -- 172.21.15.33:0/3814844033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 msgr2=0x7f33c80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 --2- 172.21.15.33:0/3814844033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 0x7f33c80c7db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33b40099b0 tx=0x7f33b4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 -- 172.21.15.33:0/3814844033 shutdown_connections 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 --2- 172.21.15.33:0/3814844033 >> v2:172.21.15.167:3300/0 conn(0x7f33c813faa0 0x7f33c813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 --2- 172.21.15.33:0/3814844033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c80c7940 0x7f33c80c7db0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 -- 172.21.15.33:0/3814844033 >> 172.21.15.33:0/3814844033 conn(0x7f33c806d030 msgr2=0x7f33c806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 -- 172.21.15.33:0/3814844033 shutdown_connections 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.283+0000 7f33cd99d700 1 -- 172.21.15.33:0/3814844033 wait complete. 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.285+0000 7f33cd99d700 1 Processor -- start 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.285+0000 7f33cd99d700 1 -- start start 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c806aad0 0x7f33c816ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 0x7f33c80c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c813faa0 0x7f33c8158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80c6e90 con 0x7f33c80c7940 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80c6280 con 0x7f33c813faa0 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80c6560 con 0x7f33c806aad0 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 0x7f33c80c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 0x7f33c80c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44662/0 (socket says 172.21.15.33:44662) 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 -- 172.21.15.33:0/1313188624 learned_addr learned my addr 172.21.15.33:0/1313188624 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cd19c700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c813faa0 0x7f33c8158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33cc99b700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c806aad0 0x7f33c816ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 -- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c806aad0 msgr2=0x7f33c816ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c806aad0 0x7f33c816ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 -- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c813faa0 msgr2=0x7f33c8158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c813faa0 0x7f33c8158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.286+0000 7f33c7fff700 1 -- 172.21.15.33:0/1313188624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40094f0 con 0x7f33c80c7940 2023-10-06T05:22:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.287+0000 7f33c7fff700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 0x7f33c80c4df0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f33c80ca3b0 tx=0x7f33b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.287+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400bbe0 con 0x7f33c80c7940 2023-10-06T05:22:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.287+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b4011420 con 0x7f33c80c7940 2023-10-06T05:22:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.287+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40117c0 con 0x7f33c80c7940 2023-10-06T05:22:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.287+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c80c6840 con 0x7f33c80c7940 2023-10-06T05:22:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.287+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c80c5330 con 0x7f33c80c7940 2023-10-06T05:22:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.288+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c80c7040 con 0x7f33c80c7940 2023-10-06T05:22:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.289+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33b4011bb0 con 0x7f33c80c7940 2023-10-06T05:22:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.290+0000 7f33c4ff9700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f33ac066f30 0x7f33ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.290+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f33b40860c0 con 0x7f33c80c7940 2023-10-06T05:22:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.291+0000 7f33cc99b700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f33ac066f30 0x7f33ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.291+0000 7f33cc99b700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f33ac066f30 0x7f33ac0693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009ca0 tx=0x7f33bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.294+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33b4053130 con 0x7f33c80c7940 2023-10-06T05:22:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.539+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c804c380 con 0x7f33c80c7940 2023-10-06T05:22:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.540+0000 7f33c4ff9700 1 -- 172.21.15.33:0/1313188624 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f33b4011e60 con 0x7f33c80c7940 2023-10-06T05:22:36.541 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:36.541 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.538408+0000","modified":"2023-10-06T05:22:36.061558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2226881352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2226881352},{"type":"v1","addr":"172.21.15.33:6835","nonce":2226881352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:22:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f33ac066f30 msgr2=0x7f33ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f33ac066f30 0x7f33ac0693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009ca0 tx=0x7f33bc008040 comp rx=0 tx=0).stop 2023-10-06T05:22:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 msgr2=0x7f33c80c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 0x7f33c80c4df0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f33c80ca3b0 tx=0x7f33b4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 shutdown_connections 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f33ac066f30 0x7f33ac0693e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c813faa0 0x7f33c8158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33c806aad0 0x7f33c816ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 --2- 172.21.15.33:0/1313188624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33c80c7940 0x7f33c80c4df0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 >> 172.21.15.33:0/1313188624 conn(0x7f33c806d030 msgr2=0x7f33c806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 shutdown_connections 2023-10-06T05:22:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.542+0000 7f33cd99d700 1 -- 172.21.15.33:0/1313188624 wait complete. 2023-10-06T05:22:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 73 2023-10-06T05:22:36.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:36.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.747+0000 7ff45939d700 1 Processor -- start 2023-10-06T05:22:36.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.747+0000 7ff45939d700 1 -- start start 2023-10-06T05:22:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.748+0000 7ff45939d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff454155af0 0x7ff45406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.748+0000 7ff45939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4540c7920 0x7ff4540c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.748+0000 7ff45939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45406ab60 con 0x7ff45406a6a0 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.748+0000 7ff45939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45406b7e0 con 0x7ff4540c7920 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.748+0000 7ff45939d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45406ae40 con 0x7ff454155af0 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4540c7920 0x7ff4540c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4540c7920 0x7ff4540c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44668/0 (socket says 172.21.15.33:44668) 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 -- 172.21.15.33:0/2245043658 learned_addr learned my addr 172.21.15.33:0/2245043658 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff453fff700 1 --2- 172.21.15.33:0/2245043658 >> v2:172.21.15.167:3300/0 conn(0x7ff454155af0 0x7ff45406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 -- 172.21.15.33:0/2245043658 >> v2:172.21.15.167:3300/0 conn(0x7ff454155af0 msgr2=0x7ff45406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 --2- 172.21.15.33:0/2245043658 >> v2:172.21.15.167:3300/0 conn(0x7ff454155af0 0x7ff45406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 -- 172.21.15.33:0/2245043658 >> v1:172.21.15.167:6789/0 conn(0x7ff45406a6a0 legacy=0x7ff4540c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 -- 172.21.15.33:0/2245043658 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45406b120 con 0x7ff4540c7920 2023-10-06T05:22:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.749+0000 7ff458b9c700 1 --2- 172.21.15.33:0/2245043658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c7920 0x7ff4540c9e20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff4540697c0 tx=0x7ff44c00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0e688a99aa1d29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff452ffd700 1 -- 172.21.15.33:0/2245043658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00f070 con 0x7ff4540c7920 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff452ffd700 1 -- 172.21.15.33:0/2245043658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c00a2e0 con 0x7ff4540c7920 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff452ffd700 1 -- 172.21.15.33:0/2245043658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00cd50 con 0x7ff4540c7920 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 -- 172.21.15.33:0/2245043658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c7920 msgr2=0x7ff4540c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 --2- 172.21.15.33:0/2245043658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c7920 0x7ff4540c9e20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff4540697c0 tx=0x7ff44c00a950 comp rx=0 tx=0).stop 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 -- 172.21.15.33:0/2245043658 shutdown_connections 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 --2- 172.21.15.33:0/2245043658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c7920 0x7ff4540c9e20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 --2- 172.21.15.33:0/2245043658 >> v2:172.21.15.167:3300/0 conn(0x7ff454155af0 0x7ff45406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 -- 172.21.15.33:0/2245043658 >> 172.21.15.33:0/2245043658 conn(0x7ff45406d030 msgr2=0x7ff45406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 -- 172.21.15.33:0/2245043658 shutdown_connections 2023-10-06T05:22:36.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.750+0000 7ff45939d700 1 -- 172.21.15.33:0/2245043658 wait complete. 2023-10-06T05:22:36.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.752+0000 7ff45939d700 1 Processor -- start 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.752+0000 7ff45939d700 1 -- start start 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff45939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 0x7ff45414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff45939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4540c7920 0x7ff4540ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff45939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff454155af0 0x7ff4540c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff45939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540adc40 con 0x7ff45406a6a0 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff45939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540ad030 con 0x7ff4540c7920 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff45939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540ad310 con 0x7ff454155af0 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff453fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 0x7ff45414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff453fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 0x7ff45414fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44682/0 (socket says 172.21.15.33:44682) 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff453fff700 1 -- 172.21.15.33:0/2696742823 learned_addr learned my addr 172.21.15.33:0/2696742823 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff458b9c700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff454155af0 0x7ff4540c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.753+0000 7ff4537fe700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4540c7920 0x7ff4540ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff453fff700 1 -- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff454155af0 msgr2=0x7ff4540c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff453fff700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff454155af0 0x7ff4540c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff453fff700 1 -- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4540c7920 msgr2=0x7ff4540ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff453fff700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4540c7920 0x7ff4540ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff453fff700 1 -- 172.21.15.33:0/2696742823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c00abf0 con 0x7ff45406a6a0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff453fff700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 0x7ff45414fff0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff44400ec80 tx=0x7ff4440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400c850 con 0x7ff45406a6a0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400c9d0 con 0x7ff45406a6a0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400cd90 con 0x7ff45406a6a0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4540ad5f0 con 0x7ff45406a6a0 2023-10-06T05:22:36.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.754+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff454150530 con 0x7ff45406a6a0 2023-10-06T05:22:36.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.755+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4540c9560 con 0x7ff45406a6a0 2023-10-06T05:22:36.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.756+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff444013680 con 0x7ff45406a6a0 2023-10-06T05:22:36.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.757+0000 7ff43bfff700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff440067000 0x7ff4400694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.757+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff444084940 con 0x7ff45406a6a0 2023-10-06T05:22:36.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.758+0000 7ff4537fe700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff440067000 0x7ff4400694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.758+0000 7ff4537fe700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff440067000 0x7ff4400694b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff4540c91e0 tx=0x7ff448009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.761+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff444051a30 con 0x7ff45406a6a0 2023-10-06T05:22:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.967+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4540c8560 con 0x7ff45406a6a0 2023-10-06T05:22:36.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.968+0000 7ff43bfff700 1 -- 172.21.15.33:0/2696742823 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16880 (secure 0 0 0) 0x7ff444055050 con 0x7ff45406a6a0 2023-10-06T05:22:36.970 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:34.048391+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.698838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.505677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff440067000 msgr2=0x7ff4400694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff440067000 0x7ff4400694b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff4540c91e0 tx=0x7ff448009040 comp rx=0 tx=0).stop 2023-10-06T05:22:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 msgr2=0x7ff45414fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 0x7ff45414fff0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff44400ec80 tx=0x7ff4440091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 shutdown_connections 2023-10-06T05:22:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff440067000 0x7ff4400694b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff454155af0 0x7ff4540c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff45406a6a0 0x7ff45414fff0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 --2- 172.21.15.33:0/2696742823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4540c7920 0x7ff4540ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 >> 172.21.15.33:0/2696742823 conn(0x7ff45406d030 msgr2=0x7ff4540c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 shutdown_connections 2023-10-06T05:22:36.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:36.970+0000 7ff45939d700 1 -- 172.21.15.33:0/2696742823 wait complete. 2023-10-06T05:22:36.989 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:22:36.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:37.016 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:22:37.014+0000 7f5e101f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.176+0000 7f6a7e19d700 1 Processor -- start 2023-10-06T05:22:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.176+0000 7f6a7e19d700 1 -- start start 2023-10-06T05:22:37.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7e19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a7813faa0 0x7f6a7813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6a7806aad0 0x7f6a7806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7e19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a780c9cb0 con 0x7f6a780c7940 2023-10-06T05:22:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7e19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a780ca270 con 0x7f6a7813faa0 2023-10-06T05:22:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7806b840 con 0x7f6a7806aad0 2023-10-06T05:22:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7d19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a7813faa0 0x7f6a7813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7d19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a7813faa0 0x7f6a7813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44692/0 (socket says 172.21.15.33:44692) 2023-10-06T05:22:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7d19b700 1 -- 172.21.15.33:0/3959343469 learned_addr learned my addr 172.21.15.33:0/3959343469 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7d19b700 1 -- 172.21.15.33:0/3959343469 >> v2:172.21.15.167:3300/0 conn(0x7f6a7806aad0 msgr2=0x7f6a7806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7d99c700 1 --2- 172.21.15.33:0/3959343469 >> v2:172.21.15.167:3300/0 conn(0x7f6a7806aad0 0x7f6a7806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.178+0000 7f6a7d19b700 1 --2- 172.21.15.33:0/3959343469 >> v2:172.21.15.167:3300/0 conn(0x7f6a7806aad0 0x7f6a7806af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a7d19b700 1 -- 172.21.15.33:0/3959343469 >> v1:172.21.15.167:6789/0 conn(0x7f6a780c7940 legacy=0x7f6a780c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a7d19b700 1 -- 172.21.15.33:0/3959343469 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7806bb20 con 0x7f6a7813faa0 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a7d99c700 1 --2- 172.21.15.33:0/3959343469 >> v2:172.21.15.167:3300/0 conn(0x7f6a7806aad0 0x7f6a7806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a7d19b700 1 --2- 172.21.15.33:0/3959343469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a7813fe70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6a78156b90 tx=0x7f6a74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be02b72250832c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a6ffff700 1 -- 172.21.15.33:0/3959343469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400b820 con 0x7f6a7813faa0 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a6ffff700 1 -- 172.21.15.33:0/3959343469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7400b980 con 0x7f6a7813faa0 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.179+0000 7f6a6ffff700 1 -- 172.21.15.33:0/3959343469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400bd20 con 0x7f6a7813faa0 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 -- 172.21.15.33:0/3959343469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 msgr2=0x7f6a7813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/3959343469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a7813fe70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6a78156b90 tx=0x7f6a74009250 comp rx=0 tx=0).stop 2023-10-06T05:22:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 -- 172.21.15.33:0/3959343469 shutdown_connections 2023-10-06T05:22:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/3959343469 >> v2:172.21.15.167:3300/0 conn(0x7f6a7806aad0 0x7f6a7806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/3959343469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a7813fe70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 -- 172.21.15.33:0/3959343469 >> 172.21.15.33:0/3959343469 conn(0x7f6a7806d030 msgr2=0x7f6a7806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 -- 172.21.15.33:0/3959343469 shutdown_connections 2023-10-06T05:22:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.180+0000 7f6a7e19d700 1 -- 172.21.15.33:0/3959343469 wait complete. 2023-10-06T05:22:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.181+0000 7f6a7e19d700 1 Processor -- start 2023-10-06T05:22:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.181+0000 7f6a7e19d700 1 -- start start 2023-10-06T05:22:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.182+0000 7f6a7e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a7806aad0 0x7f6a780c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.182+0000 7f6a7e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a780c7940 0x7f6a78159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.182+0000 7f6a7e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a78156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.182+0000 7f6a7e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78156ed0 con 0x7f6a7813faa0 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.182+0000 7f6a7e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780c51c0 con 0x7f6a7806aad0 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.182+0000 7f6a7e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78159dc0 con 0x7f6a780c7940 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a78156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a78156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44704/0 (socket says 172.21.15.33:44704) 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 -- 172.21.15.33:0/660844259 learned_addr learned my addr 172.21.15.33:0/660844259 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7c99a700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a780c7940 0x7f6a78159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d19b700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a7806aad0 0x7f6a780c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 -- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a780c7940 msgr2=0x7f6a78159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a780c7940 0x7f6a78159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 -- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a7806aad0 msgr2=0x7f6a780c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a7806aad0 0x7f6a780c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 -- 172.21.15.33:0/660844259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740094f0 con 0x7f6a7813faa0 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.183+0000 7f6a7d99c700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a78156130 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000a380 tx=0x7f6a70007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.184+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70015440 con 0x7f6a7813faa0 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.184+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a700155a0 con 0x7f6a7813faa0 2023-10-06T05:22:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.184+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70015960 con 0x7f6a7813faa0 2023-10-06T05:22:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.184+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a78157120 con 0x7f6a7813faa0 2023-10-06T05:22:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.184+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a780c4e50 con 0x7f6a7813faa0 2023-10-06T05:22:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.185+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a780c5330 con 0x7f6a7813faa0 2023-10-06T05:22:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.186+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a7000d440 con 0x7f6a7813faa0 2023-10-06T05:22:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.187+0000 7f6a6d7fa700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a5c066f30 0x7f6a5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.187+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6a700851f0 con 0x7f6a7813faa0 2023-10-06T05:22:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.187+0000 7f6a7d19b700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a5c066f30 0x7f6a5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.188+0000 7f6a7d19b700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a5c066f30 0x7f6a5c0693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6a78156b90 tx=0x7f6a74016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.191+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a70052260 con 0x7f6a7813faa0 2023-10-06T05:22:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.388+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a7804c380 con 0x7f6a5c066f30 2023-10-06T05:22:37.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.398+0000 7f6a6d7fa700 1 -- 172.21.15.33:0/660844259 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300491 (secure 0 0 0) 0x7f6a7804c380 con 0x7f6a5c066f30 2023-10-06T05:22:37.403 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a5c066f30 msgr2=0x7f6a5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a5c066f30 0x7f6a5c0693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6a78156b90 tx=0x7f6a74016000 comp rx=0 tx=0).stop 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 msgr2=0x7f6a78156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a78156130 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000a380 tx=0x7f6a70007900 comp rx=0 tx=0).stop 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 shutdown_connections 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a5c066f30 0x7f6a5c0693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a7813faa0 0x7f6a78156130 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a7806aad0 0x7f6a780c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 --2- 172.21.15.33:0/660844259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a780c7940 0x7f6a78159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 >> 172.21.15.33:0/660844259 conn(0x7f6a7806d030 msgr2=0x7f6a7815a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 shutdown_connections 2023-10-06T05:22:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.402+0000 7f6a7e19d700 1 -- 172.21.15.33:0/660844259 wait complete. 2023-10-06T05:22:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:22:37.422 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-10-06T05:22:35.938529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422076,"kb_used_data":11004,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552644,"statfs":{"total":773094113280,"available":770613907456,"internally_reserved":0,"allocated":11268096,"data_stored":5765034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.910696"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029575+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:11.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.036032+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:12.637447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033384+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:26.064774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033234+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:48.955910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032834+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:48.915690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035751+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:02.462579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033610+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:04.671277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029553+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:54.730043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033128+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:48.915690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030652+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:57.874890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033988+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:03.338584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030063+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:11.133728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032489+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:11.824761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029588+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:40.124330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035512+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:27.689143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033036+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:46.719034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032383+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:21.340390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030015+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:22.564236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030128+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:23.566596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032753+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:09.916342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032976+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:47.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035891+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:42.741715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030126+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:23.838232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035417+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:32.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030882+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:06.505639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.005850+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:22:28.005850+0000","last_peered":"2023-10-06T05:22:28.005850+0000","last_clean":"2023-10-06T05:22:28.005850+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:22:28.005850+0000","last_undegraded":"2023-10-06T05:22:28.005850+0000","last_fullsized":"2023-10-06T05:22:28.005850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029947+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:58.635879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.733465+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:22:28.733465+0000","last_peered":"2023-10-06T05:22:28.733465+0000","last_clean":"2023-10-06T05:22:28.733465+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:22:28.733465+0000","last_undegraded":"2023-10-06T05:22:28.733465+0000","last_fullsized":"2023-10-06T05:22:28.733465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035675+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:51.938109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.834350+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:22:28.834350+0000","last_peered":"2023-10-06T05:22:28.834350+0000","last_clean":"2023-10-06T05:22:28.834350+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:22:28.834350+0000","last_undegraded":"2023-10-06T05:22:28.834350+0000","last_fullsized":"2023-10-06T05:22:28.834350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035540+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:13.199260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.733465+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:22:28.733465+0000","last_peered":"2023-10-06T05:22:28.733465+0000","last_clean":"2023-10-06T05:22:28.733465+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:22:28.733465+0000","last_undegraded":"2023-10-06T05:22:28.733465+0000","last_fullsized":"2023-10-06T05:22:28.733465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":233,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.012082+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:22:28.012082+0000","last_peered":"2023-10-06T05:22:28.012082+0000","last_clean":"2023-10-06T05:22:28.012082+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:22:28.012082+0000","last_undegraded":"2023-10-06T05:22:28.012082+0000","last_fullsized":"2023-10-06T05:22:28.012082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033456+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:04.671277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.012041+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:22:28.012041+0000","last_peered":"2023-10-06T05:22:28.012041+0000","last_clean":"2023-10-06T05:22:28.012041+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:22:28.012041+0000","last_undegraded":"2023-10-06T05:22:28.012041+0000","last_fullsized":"2023-10-06T05:22:28.012041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.036033+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:24.720616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.012029+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:22:28.012029+0000","last_peered":"2023-10-06T05:22:28.012029+0000","last_clean":"2023-10-06T05:22:28.012029+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:22:28.012029+0000","last_undegraded":"2023-10-06T05:22:28.012029+0000","last_fullsized":"2023-10-06T05:22:28.012029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035579+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:38.710580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.005855+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:22:28.005855+0000","last_peered":"2023-10-06T05:22:28.005855+0000","last_clean":"2023-10-06T05:22:28.005855+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:22:28.005855+0000","last_undegraded":"2023-10-06T05:22:28.005855+0000","last_fullsized":"2023-10-06T05:22:28.005855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.036385+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:47.677660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.834740+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:22:28.834740+0000","last_peered":"2023-10-06T05:22:28.834740+0000","last_clean":"2023-10-06T05:22:28.834740+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:22:28.834740+0000","last_undegraded":"2023-10-06T05:22:28.834740+0000","last_fullsized":"2023-10-06T05:22:28.834740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":1855488,"data_stored":1164175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.66200000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.755},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.501}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.745},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.80900000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":1.25},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.96099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":2.6240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303184,"kb_used_data":1800,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068656,"statfs":{"total":96636764160,"available":96326303744,"internally_reserved":0,"allocated":1843200,"data_stored":1158017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.63}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.753},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.752},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.628}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.254},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.30199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.772},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.78100000000000003,"15min":1.913},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73499999999999999,"5min":0.82399999999999995,"15min":1.954},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58499999999999996,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82599999999999996,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.78300000000000003},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94099999999999995,"5min":1.2030000000000001,"15min":1.282},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.52400000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1020000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.145,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80400000000000005,"15min":0.84899999999999998},"min":{"1min":0.55900000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1140000000000001,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.84399999999999997},"min":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.012,"5min":1.1910000000000001,"15min":1.343},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82299999999999995,"15min":0.83199999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.0149999999999999,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85499999999999998},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.56799999999999995},"max":{"1min":1.054,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.55600000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":1.2090000000000001,"15min":1.214},"last":0.80800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.86899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.753},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.622}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:37.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:37.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.648+0000 7fc83199d700 1 Processor -- start 2023-10-06T05:22:37.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.648+0000 7fc83199d700 1 -- start start 2023-10-06T05:22:37.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.649+0000 7fc83199d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc82c155140 0x7fc82c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.649+0000 7fc83199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0adf50 0x7fc82c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.649+0000 7fc83199d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82c0c8e40 con 0x7fc82c0c9810 2023-10-06T05:22:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.649+0000 7fc83199d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82c0ae6c0 con 0x7fc82c155140 2023-10-06T05:22:37.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.649+0000 7fc83199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0ae9a0 con 0x7fc82c0adf50 2023-10-06T05:22:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc83099b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc82c155140 0x7fc82c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0adf50 0x7fc82c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc83119c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc82c0c9810 0x7fc82c0699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:38222/0 (socket says 172.21.15.33:38222) 2023-10-06T05:22:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc83119c700 1 -- 172.21.15.33:0/3335250383 learned_addr learned my addr 172.21.15.33:0/3335250383 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82b7fe700 1 -- 172.21.15.33:0/3335250383 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134264919 0 0) 0x7fc82c0c8e40 con 0x7fc82c0c9810 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82b7fe700 1 -- 172.21.15.33:0/3335250383 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc81c003570 con 0x7fc82c0c9810 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82bfff700 1 -- 172.21.15.33:0/3335250383 >> v2:172.21.15.167:3300/0 conn(0x7fc82c155140 msgr2=0x7fc82c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82bfff700 1 --2- 172.21.15.33:0/3335250383 >> v2:172.21.15.167:3300/0 conn(0x7fc82c155140 0x7fc82c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82bfff700 1 -- 172.21.15.33:0/3335250383 >> v1:172.21.15.167:6789/0 conn(0x7fc82c0c9810 legacy=0x7fc82c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc82bfff700 1 -- 172.21.15.33:0/3335250383 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c0c8ad0 con 0x7fc82c0adf50 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.650+0000 7fc83099b700 1 --2- 172.21.15.33:0/3335250383 >> v2:172.21.15.167:3300/0 conn(0x7fc82c155140 0x7fc82c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.651+0000 7fc82bfff700 1 --2- 172.21.15.33:0/3335250383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0adf50 0x7fc82c0c92d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098e0 tx=0x7fc8180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f391bc047b5a72c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.651+0000 7fc82b7fe700 1 -- 172.21.15.33:0/3335250383 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818011630 con 0x7fc82c0adf50 2023-10-06T05:22:37.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.651+0000 7fc82b7fe700 1 -- 172.21.15.33:0/3335250383 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818011790 con 0x7fc82c0adf50 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.651+0000 7fc82b7fe700 1 -- 172.21.15.33:0/3335250383 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818011b30 con 0x7fc82c0adf50 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 -- 172.21.15.33:0/3335250383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0adf50 msgr2=0x7fc82c0c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 --2- 172.21.15.33:0/3335250383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0adf50 0x7fc82c0c92d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098e0 tx=0x7fc8180091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 -- 172.21.15.33:0/3335250383 shutdown_connections 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 --2- 172.21.15.33:0/3335250383 >> v2:172.21.15.167:3300/0 conn(0x7fc82c155140 0x7fc82c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 --2- 172.21.15.33:0/3335250383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0adf50 0x7fc82c0c92d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 -- 172.21.15.33:0/3335250383 >> 172.21.15.33:0/3335250383 conn(0x7fc82c06d030 msgr2=0x7fc82c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 -- 172.21.15.33:0/3335250383 shutdown_connections 2023-10-06T05:22:37.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.652+0000 7fc83199d700 1 -- 172.21.15.33:0/3335250383 wait complete. 2023-10-06T05:22:37.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.653+0000 7fc83199d700 1 Processor -- start 2023-10-06T05:22:37.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.653+0000 7fc83199d700 1 -- start start 2023-10-06T05:22:37.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.654+0000 7fc83199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c0adf50 0x7fc82c16c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.654+0000 7fc83199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0c9810 0x7fc82c16a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.654+0000 7fc83199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 0x7fc82c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.654+0000 7fc83199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0ca680 con 0x7fc82c155140 2023-10-06T05:22:37.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.654+0000 7fc83199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c15ba70 con 0x7fc82c0adf50 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.654+0000 7fc83199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0c9d50 con 0x7fc82c0c9810 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 0x7fc82c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 0x7fc82c158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44708/0 (socket says 172.21.15.33:44708) 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 -- 172.21.15.33:0/339598389 learned_addr learned my addr 172.21.15.33:0/339598389 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc82bfff700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0c9810 0x7fc82c16a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83099b700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c0adf50 0x7fc82c16c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 -- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0c9810 msgr2=0x7fc82c16a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0c9810 0x7fc82c16a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 -- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c0adf50 msgr2=0x7fc82c16c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c0adf50 0x7fc82c16c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 -- 172.21.15.33:0/339598389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820009420 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc82bfff700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0c9810 0x7fc82c16a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83099b700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c0adf50 0x7fc82c16c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.655+0000 7fc83119c700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 0x7fc82c158880 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc824001460 tx=0x7fc824001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.656+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82400a990 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.656+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82400aaf0 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.656+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82400ae70 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.656+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818009420 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.656+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82c0ca310 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.657+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82c06b330 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.658+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc824007300 con 0x7fc82c155140 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.658+0000 7fc828ff9700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc810066f30 0x7fc8100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.658+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc824084b90 con 0x7fc82c155140 2023-10-06T05:22:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.659+0000 7fc83099b700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc810066f30 0x7fc8100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.660+0000 7fc83099b700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc810066f30 0x7fc8100693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc8200093f0 tx=0x7fc820008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.662+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc824051c90 con 0x7fc82c155140 2023-10-06T05:22:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.863+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc82c06bac0 con 0x7fc810066f30 2023-10-06T05:22:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.872+0000 7fc828ff9700 1 -- 172.21.15.33:0/339598389 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300491 (secure 0 0 0) 0x7fc82c06bac0 con 0x7fc810066f30 2023-10-06T05:22:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc810066f30 msgr2=0x7fc8100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc810066f30 0x7fc8100693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc8200093f0 tx=0x7fc820008040 comp rx=0 tx=0).stop 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 msgr2=0x7fc82c158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 0x7fc82c158880 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc824001460 tx=0x7fc824001a80 comp rx=0 tx=0).stop 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 shutdown_connections 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc810066f30 0x7fc8100693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc82c155140 0x7fc82c158880 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c0adf50 0x7fc82c16c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 --2- 172.21.15.33:0/339598389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc82c0c9810 0x7fc82c16a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 >> 172.21.15.33:0/339598389 conn(0x7fc82c06d030 msgr2=0x7fc82c0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.876+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 shutdown_connections 2023-10-06T05:22:37.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:37.877+0000 7fc83199d700 1 -- 172.21.15.33:0/339598389 wait complete. 2023-10-06T05:22:37.881 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:22:37.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-10-06T05:22:35.938529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422076,"kb_used_data":11004,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552644,"statfs":{"total":773094113280,"available":770613907456,"internally_reserved":0,"allocated":11268096,"data_stored":5765034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.910696"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029575+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:11.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.036032+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:12.637447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033384+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:26.064774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033234+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:48.955910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032834+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:48.915690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035751+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:02.462579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033610+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:04.671277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029553+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:54.730043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033128+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:48.915690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030652+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:57.874890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033988+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:03.338584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030063+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:11.133728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032489+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:11.824761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029588+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:40.124330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035512+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:27.689143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033036+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:46.719034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032383+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:21.340390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030015+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:22.564236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030128+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:23.566596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032753+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:09.916342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.032976+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:47.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035891+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:42.741715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030126+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:23.838232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035417+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:32.728775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.030882+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:06.505639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.005850+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:22:28.005850+0000","last_peered":"2023-10-06T05:22:28.005850+0000","last_clean":"2023-10-06T05:22:28.005850+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:22:28.005850+0000","last_undegraded":"2023-10-06T05:22:28.005850+0000","last_fullsized":"2023-10-06T05:22:28.005850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.029947+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:58.635879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.733465+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:22:28.733465+0000","last_peered":"2023-10-06T05:22:28.733465+0000","last_clean":"2023-10-06T05:22:28.733465+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:22:28.733465+0000","last_undegraded":"2023-10-06T05:22:28.733465+0000","last_fullsized":"2023-10-06T05:22:28.733465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035675+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:51.938109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.834350+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:22:28.834350+0000","last_peered":"2023-10-06T05:22:28.834350+0000","last_clean":"2023-10-06T05:22:28.834350+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:22:28.834350+0000","last_undegraded":"2023-10-06T05:22:28.834350+0000","last_fullsized":"2023-10-06T05:22:28.834350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035540+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:13.199260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.733465+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:22:28.733465+0000","last_peered":"2023-10-06T05:22:28.733465+0000","last_clean":"2023-10-06T05:22:28.733465+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:22:28.733465+0000","last_undegraded":"2023-10-06T05:22:28.733465+0000","last_fullsized":"2023-10-06T05:22:28.733465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":233,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.012082+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:22:28.012082+0000","last_peered":"2023-10-06T05:22:28.012082+0000","last_clean":"2023-10-06T05:22:28.012082+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:22:28.012082+0000","last_undegraded":"2023-10-06T05:22:28.012082+0000","last_fullsized":"2023-10-06T05:22:28.012082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.033456+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:04.671277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.012041+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:22:28.012041+0000","last_peered":"2023-10-06T05:22:28.012041+0000","last_clean":"2023-10-06T05:22:28.012041+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:22:28.012041+0000","last_undegraded":"2023-10-06T05:22:28.012041+0000","last_fullsized":"2023-10-06T05:22:28.012041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.036033+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:24.720616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.012029+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:22:28.012029+0000","last_peered":"2023-10-06T05:22:28.012029+0000","last_clean":"2023-10-06T05:22:28.012029+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:22:28.012029+0000","last_undegraded":"2023-10-06T05:22:28.012029+0000","last_fullsized":"2023-10-06T05:22:28.012029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.035579+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:38.710580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.005855+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:22:28.005855+0000","last_peered":"2023-10-06T05:22:28.005855+0000","last_clean":"2023-10-06T05:22:28.005855+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:22:28.005855+0000","last_undegraded":"2023-10-06T05:22:28.005855+0000","last_fullsized":"2023-10-06T05:22:28.005855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:22:32.027088+0000","last_change":"2023-10-06T05:22:32.027088+0000","last_active":"2023-10-06T05:22:32.027088+0000","last_peered":"2023-10-06T05:22:32.027088+0000","last_clean":"2023-10-06T05:22:32.027088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:32.027088+0000","last_undegraded":"2023-10-06T05:22:32.027088+0000","last_fullsized":"2023-10-06T05:22:32.027088+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:22:31.020359+0000","last_change":"2023-10-06T05:22:31.036385+0000","last_active":"2023-10-06T05:22:31.020359+0000","last_peered":"2023-10-06T05:22:31.020359+0000","last_clean":"2023-10-06T05:22:31.020359+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:22:31.020359+0000","last_undegraded":"2023-10-06T05:22:31.020359+0000","last_fullsized":"2023-10-06T05:22:31.020359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:47.677660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-10-06T05:22:28.834740+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:22:28.834740+0000","last_peered":"2023-10-06T05:22:28.834740+0000","last_clean":"2023-10-06T05:22:28.834740+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:22:28.834740+0000","last_undegraded":"2023-10-06T05:22:28.834740+0000","last_fullsized":"2023-10-06T05:22:28.834740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":1855488,"data_stored":1164175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.66200000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.755},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.501}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.745},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.80900000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":1.25},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.96099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":2.6240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303184,"kb_used_data":1800,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068656,"statfs":{"total":96636764160,"available":96326303744,"internally_reserved":0,"allocated":1843200,"data_stored":1158017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.63}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.753},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.752},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.628}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.254},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.30199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":1.772},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.78100000000000003,"15min":1.913},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73499999999999999,"5min":0.82399999999999995,"15min":1.954},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58499999999999996,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82599999999999996,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.755,"15min":0.78300000000000003},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94099999999999995,"5min":1.2030000000000001,"15min":1.282},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.52400000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1020000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79900000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.145,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80400000000000005,"15min":0.84899999999999998},"min":{"1min":0.55900000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1140000000000001,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.84399999999999997},"min":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.012,"5min":1.1910000000000001,"15min":1.343},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82299999999999995,"15min":0.83199999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.0149999999999999,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85499999999999998},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.56799999999999995},"max":{"1min":1.054,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.55600000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0580000000000001,"5min":1.2090000000000001,"15min":1.214},"last":0.80800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.86899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":573807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.753},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.622}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:37.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:38.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.122+0000 7fba1e59d700 1 Processor -- start 2023-10-06T05:22:38.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.122+0000 7fba1e59d700 1 -- start start 2023-10-06T05:22:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.123+0000 7fba1e59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba180cf3b0 0x7fba181594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1e59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180cbd80 con 0x7fba180af0a0 2023-10-06T05:22:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1e59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180af5f0 con 0x7fba180ccfc0 2023-10-06T05:22:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1e59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba180af8d0 con 0x7fba180cf3b0 2023-10-06T05:22:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1cd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba180cf3b0 0x7fba181594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1cd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba180cf3b0 0x7fba181594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44718/0 (socket says 172.21.15.33:44718) 2023-10-06T05:22:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1cd9a700 1 -- 172.21.15.33:0/2155408632 learned_addr learned my addr 172.21.15.33:0/2155408632 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1dd9c700 1 --1- 172.21.15.33:0/2155408632 >> v1:172.21.15.33:6789/0 conn(0x7fba180af0a0 0x7fba180b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55946/0 (socket says 172.21.15.33:55946) 2023-10-06T05:22:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1cd9a700 1 -- 172.21.15.33:0/2155408632 >> v1:172.21.15.167:6789/0 conn(0x7fba180ccfc0 legacy=0x7fba180cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1cd9a700 1 -- 172.21.15.33:0/2155408632 >> v1:172.21.15.33:6789/0 conn(0x7fba180af0a0 legacy=0x7fba180b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.124+0000 7fba1cd9a700 1 -- 172.21.15.33:0/2155408632 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba180afbb0 con 0x7fba180cf3b0 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba1cd9a700 1 --2- 172.21.15.33:0/2155408632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180cf3b0 0x7fba181594a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fba080099e0 tx=0x7fba08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a29d0a0b60f091 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba0ffff700 1 -- 172.21.15.33:0/2155408632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800b820 con 0x7fba180cf3b0 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba0ffff700 1 -- 172.21.15.33:0/2155408632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800b980 con 0x7fba180cf3b0 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba0ffff700 1 -- 172.21.15.33:0/2155408632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800bd20 con 0x7fba180cf3b0 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba1e59d700 1 -- 172.21.15.33:0/2155408632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180cf3b0 msgr2=0x7fba181594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba1e59d700 1 --2- 172.21.15.33:0/2155408632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180cf3b0 0x7fba181594a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fba080099e0 tx=0x7fba08009250 comp rx=0 tx=0).stop 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba1e59d700 1 -- 172.21.15.33:0/2155408632 shutdown_connections 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.125+0000 7fba1e59d700 1 --2- 172.21.15.33:0/2155408632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180cf3b0 0x7fba181594a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.126+0000 7fba1e59d700 1 -- 172.21.15.33:0/2155408632 >> 172.21.15.33:0/2155408632 conn(0x7fba1806cff0 msgr2=0x7fba1806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.126+0000 7fba1e59d700 1 -- 172.21.15.33:0/2155408632 shutdown_connections 2023-10-06T05:22:38.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.126+0000 7fba1e59d700 1 -- 172.21.15.33:0/2155408632 wait complete. 2023-10-06T05:22:38.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.127+0000 7fba1e59d700 1 Processor -- start 2023-10-06T05:22:38.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.127+0000 7fba1e59d700 1 -- start start 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1e59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 0x7fba1814b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1e59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba180ccfc0 0x7fba180b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1e59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba180c9af0 0x7fba180b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1e59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1814b7b0 con 0x7fba180af0a0 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1e59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180c8a50 con 0x7fba180c9af0 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1e59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba180c8d30 con 0x7fba180ccfc0 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 0x7fba1814b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1d59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 0x7fba1814b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44724/0 (socket says 172.21.15.33:44724) 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.128+0000 7fba1d59b700 1 -- 172.21.15.33:0/715337899 learned_addr learned my addr 172.21.15.33:0/715337899 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1dd9c700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba180c9af0 0x7fba180b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1cd9a700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba180ccfc0 0x7fba180b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1d59b700 1 -- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba180ccfc0 msgr2=0x7fba180b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1d59b700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba180ccfc0 0x7fba180b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1d59b700 1 -- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba180c9af0 msgr2=0x7fba180b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1d59b700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba180c9af0 0x7fba180b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1d59b700 1 -- 172.21.15.33:0/715337899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080094f0 con 0x7fba180af0a0 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1dd9c700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba180c9af0 0x7fba180b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1d59b700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 0x7fba1814b270 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c590 tx=0x7fba1400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14011590 con 0x7fba180af0a0 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba140116f0 con 0x7fba180af0a0 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14011ab0 con 0x7fba180af0a0 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.129+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba180c9010 con 0x7fba180af0a0 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.130+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba180c95d0 con 0x7fba180af0a0 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.130+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1804c380 con 0x7fba180af0a0 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.131+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba1401b3b0 con 0x7fba180af0a0 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.132+0000 7fba0d7fa700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9fc066f30 0x7fb9fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.132+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba14083c50 con 0x7fba180af0a0 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.133+0000 7fba1cd9a700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9fc066f30 0x7fb9fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.134+0000 7fba1cd9a700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9fc066f30 0x7fb9fc0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fba080099e0 tx=0x7fba08017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.136+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba14050cc0 con 0x7fba180af0a0 2023-10-06T05:22:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.334+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba180cc920 con 0x7fb9fc066f30 2023-10-06T05:22:38.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.344+0000 7fba0d7fa700 1 -- 172.21.15.33:0/715337899 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311443 (secure 0 0 0) 0x7fba180cc920 con 0x7fb9fc066f30 2023-10-06T05:22:38.350 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:38.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9fc066f30 msgr2=0x7fb9fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9fc066f30 0x7fb9fc0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fba080099e0 tx=0x7fba08017000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 msgr2=0x7fba1814b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 0x7fba1814b270 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c590 tx=0x7fba1400c950 comp rx=0 tx=0).stop 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 shutdown_connections 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb9fc066f30 0x7fb9fc0693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba180c9af0 0x7fba180b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba180af0a0 0x7fba1814b270 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 --2- 172.21.15.33:0/715337899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba180ccfc0 0x7fba180b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 >> 172.21.15.33:0/715337899 conn(0x7fba1806cff0 msgr2=0x7fba18155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 shutdown_connections 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.349+0000 7fba1e59d700 1 -- 172.21.15.33:0/715337899 wait complete. 2023-10-06T05:22:38.353 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:22:38.368 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2023-10-06T05:22:37.939056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422440,"kb_used_data":11368,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552280,"statfs":{"total":773094113280,"available":770613534720,"internally_reserved":0,"allocated":11640832,"data_stored":5938788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001649"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077617+0000","last_change":"2023-10-06T05:22:33.071449+0000","last_active":"2023-10-06T05:22:33.077617+0000","last_peered":"2023-10-06T05:22:33.077617+0000","last_clean":"2023-10-06T05:22:33.077617+0000","last_became_active":"2023-10-06T05:22:33.070959+0000","last_became_peered":"2023-10-06T05:22:33.070959+0000","last_unstale":"2023-10-06T05:22:33.077617+0000","last_undegraded":"2023-10-06T05:22:33.077617+0000","last_fullsized":"2023-10-06T05:22:33.077617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:53.229898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065900+0000","last_change":"2023-10-06T05:22:33.058445+0000","last_active":"2023-10-06T05:22:33.065900+0000","last_peered":"2023-10-06T05:22:33.065900+0000","last_clean":"2023-10-06T05:22:33.065900+0000","last_became_active":"2023-10-06T05:22:33.058172+0000","last_became_peered":"2023-10-06T05:22:33.058172+0000","last_unstale":"2023-10-06T05:22:33.065900+0000","last_undegraded":"2023-10-06T05:22:33.065900+0000","last_fullsized":"2023-10-06T05:22:33.065900+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:22.347118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076927+0000","last_change":"2023-10-06T05:22:33.070804+0000","last_active":"2023-10-06T05:22:33.076927+0000","last_peered":"2023-10-06T05:22:33.076927+0000","last_clean":"2023-10-06T05:22:33.076927+0000","last_became_active":"2023-10-06T05:22:33.067750+0000","last_became_peered":"2023-10-06T05:22:33.067750+0000","last_unstale":"2023-10-06T05:22:33.076927+0000","last_undegraded":"2023-10-06T05:22:33.076927+0000","last_fullsized":"2023-10-06T05:22:33.076927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:17.036525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063340+0000","last_change":"2023-10-06T05:22:33.054655+0000","last_active":"2023-10-06T05:22:33.063340+0000","last_peered":"2023-10-06T05:22:33.063340+0000","last_clean":"2023-10-06T05:22:33.063340+0000","last_became_active":"2023-10-06T05:22:33.054393+0000","last_became_peered":"2023-10-06T05:22:33.054393+0000","last_unstale":"2023-10-06T05:22:33.063340+0000","last_undegraded":"2023-10-06T05:22:33.063340+0000","last_fullsized":"2023-10-06T05:22:33.063340+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:41.400666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062745+0000","last_change":"2023-10-06T05:22:33.057685+0000","last_active":"2023-10-06T05:22:33.062745+0000","last_peered":"2023-10-06T05:22:33.062745+0000","last_clean":"2023-10-06T05:22:33.062745+0000","last_became_active":"2023-10-06T05:22:33.057005+0000","last_became_peered":"2023-10-06T05:22:33.057005+0000","last_unstale":"2023-10-06T05:22:33.062745+0000","last_undegraded":"2023-10-06T05:22:33.062745+0000","last_fullsized":"2023-10-06T05:22:33.062745+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:09.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063336+0000","last_change":"2023-10-06T05:22:33.060400+0000","last_active":"2023-10-06T05:22:33.063336+0000","last_peered":"2023-10-06T05:22:33.063336+0000","last_clean":"2023-10-06T05:22:33.063336+0000","last_became_active":"2023-10-06T05:22:33.060142+0000","last_became_peered":"2023-10-06T05:22:33.060142+0000","last_unstale":"2023-10-06T05:22:33.063336+0000","last_undegraded":"2023-10-06T05:22:33.063336+0000","last_fullsized":"2023-10-06T05:22:33.063336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:12.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.058221+0000","last_change":"2023-10-06T05:22:33.054663+0000","last_active":"2023-10-06T05:22:33.058221+0000","last_peered":"2023-10-06T05:22:33.058221+0000","last_clean":"2023-10-06T05:22:33.058221+0000","last_became_active":"2023-10-06T05:22:33.054411+0000","last_became_peered":"2023-10-06T05:22:33.054411+0000","last_unstale":"2023-10-06T05:22:33.058221+0000","last_undegraded":"2023-10-06T05:22:33.058221+0000","last_fullsized":"2023-10-06T05:22:33.058221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:48.646638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062550+0000","last_change":"2023-10-06T05:22:33.056968+0000","last_active":"2023-10-06T05:22:33.062550+0000","last_peered":"2023-10-06T05:22:33.062550+0000","last_clean":"2023-10-06T05:22:33.062550+0000","last_became_active":"2023-10-06T05:22:33.056371+0000","last_became_peered":"2023-10-06T05:22:33.056371+0000","last_unstale":"2023-10-06T05:22:33.062550+0000","last_undegraded":"2023-10-06T05:22:33.062550+0000","last_fullsized":"2023-10-06T05:22:33.062550+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:54.838409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.056695+0000","last_change":"2023-10-06T05:22:33.053150+0000","last_active":"2023-10-06T05:22:33.056695+0000","last_peered":"2023-10-06T05:22:33.056695+0000","last_clean":"2023-10-06T05:22:33.056695+0000","last_became_active":"2023-10-06T05:22:33.050241+0000","last_became_peered":"2023-10-06T05:22:33.050241+0000","last_unstale":"2023-10-06T05:22:33.056695+0000","last_undegraded":"2023-10-06T05:22:33.056695+0000","last_fullsized":"2023-10-06T05:22:33.056695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:21.820269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068171+0000","last_change":"2023-10-06T05:22:33.064202+0000","last_active":"2023-10-06T05:22:33.068171+0000","last_peered":"2023-10-06T05:22:33.068171+0000","last_clean":"2023-10-06T05:22:33.068171+0000","last_became_active":"2023-10-06T05:22:33.063907+0000","last_became_peered":"2023-10-06T05:22:33.063907+0000","last_unstale":"2023-10-06T05:22:33.068171+0000","last_undegraded":"2023-10-06T05:22:33.068171+0000","last_fullsized":"2023-10-06T05:22:33.068171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:27.071503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.067334+0000","last_change":"2023-10-06T05:22:33.062952+0000","last_active":"2023-10-06T05:22:33.067334+0000","last_peered":"2023-10-06T05:22:33.067334+0000","last_clean":"2023-10-06T05:22:33.067334+0000","last_became_active":"2023-10-06T05:22:33.062656+0000","last_became_peered":"2023-10-06T05:22:33.062656+0000","last_unstale":"2023-10-06T05:22:33.067334+0000","last_undegraded":"2023-10-06T05:22:33.067334+0000","last_fullsized":"2023-10-06T05:22:33.067334+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:27.071503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.055073+0000","last_change":"2023-10-06T05:22:33.053452+0000","last_active":"2023-10-06T05:22:33.055073+0000","last_peered":"2023-10-06T05:22:33.055073+0000","last_clean":"2023-10-06T05:22:33.055073+0000","last_became_active":"2023-10-06T05:22:33.053201+0000","last_became_peered":"2023-10-06T05:22:33.053201+0000","last_unstale":"2023-10-06T05:22:33.055073+0000","last_undegraded":"2023-10-06T05:22:33.055073+0000","last_fullsized":"2023-10-06T05:22:33.055073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:53.779745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.070065+0000","last_change":"2023-10-06T05:22:33.060355+0000","last_active":"2023-10-06T05:22:33.070065+0000","last_peered":"2023-10-06T05:22:33.070065+0000","last_clean":"2023-10-06T05:22:33.070065+0000","last_became_active":"2023-10-06T05:22:33.060075+0000","last_became_peered":"2023-10-06T05:22:33.060075+0000","last_unstale":"2023-10-06T05:22:33.070065+0000","last_undegraded":"2023-10-06T05:22:33.070065+0000","last_fullsized":"2023-10-06T05:22:33.070065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:34.310804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059909+0000","last_change":"2023-10-06T05:22:33.051079+0000","last_active":"2023-10-06T05:22:33.059909+0000","last_peered":"2023-10-06T05:22:33.059909+0000","last_clean":"2023-10-06T05:22:33.059909+0000","last_became_active":"2023-10-06T05:22:33.050725+0000","last_became_peered":"2023-10-06T05:22:33.050725+0000","last_unstale":"2023-10-06T05:22:33.059909+0000","last_undegraded":"2023-10-06T05:22:33.059909+0000","last_fullsized":"2023-10-06T05:22:33.059909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:52.246335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.061496+0000","last_change":"2023-10-06T05:22:33.055509+0000","last_active":"2023-10-06T05:22:33.061496+0000","last_peered":"2023-10-06T05:22:33.061496+0000","last_clean":"2023-10-06T05:22:33.061496+0000","last_became_active":"2023-10-06T05:22:33.055264+0000","last_became_peered":"2023-10-06T05:22:33.055264+0000","last_unstale":"2023-10-06T05:22:33.061496+0000","last_undegraded":"2023-10-06T05:22:33.061496+0000","last_fullsized":"2023-10-06T05:22:33.061496+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:57.649299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.060751+0000","last_change":"2023-10-06T05:22:33.053802+0000","last_active":"2023-10-06T05:22:33.060751+0000","last_peered":"2023-10-06T05:22:33.060751+0000","last_clean":"2023-10-06T05:22:33.060751+0000","last_became_active":"2023-10-06T05:22:33.053548+0000","last_became_peered":"2023-10-06T05:22:33.053548+0000","last_unstale":"2023-10-06T05:22:33.060751+0000","last_undegraded":"2023-10-06T05:22:33.060751+0000","last_fullsized":"2023-10-06T05:22:33.060751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077597+0000","last_change":"2023-10-06T05:22:33.071425+0000","last_active":"2023-10-06T05:22:33.077597+0000","last_peered":"2023-10-06T05:22:33.077597+0000","last_clean":"2023-10-06T05:22:33.077597+0000","last_became_active":"2023-10-06T05:22:33.067947+0000","last_became_peered":"2023-10-06T05:22:33.067947+0000","last_unstale":"2023-10-06T05:22:33.077597+0000","last_undegraded":"2023-10-06T05:22:33.077597+0000","last_fullsized":"2023-10-06T05:22:33.077597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:04.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068326+0000","last_change":"2023-10-06T05:22:33.065607+0000","last_active":"2023-10-06T05:22:33.068326+0000","last_peered":"2023-10-06T05:22:33.068326+0000","last_clean":"2023-10-06T05:22:33.068326+0000","last_became_active":"2023-10-06T05:22:33.064624+0000","last_became_peered":"2023-10-06T05:22:33.064624+0000","last_unstale":"2023-10-06T05:22:33.068326+0000","last_undegraded":"2023-10-06T05:22:33.068326+0000","last_fullsized":"2023-10-06T05:22:33.068326+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:12.831489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076909+0000","last_change":"2023-10-06T05:22:33.070753+0000","last_active":"2023-10-06T05:22:33.076909+0000","last_peered":"2023-10-06T05:22:33.076909+0000","last_clean":"2023-10-06T05:22:33.076909+0000","last_became_active":"2023-10-06T05:22:33.067411+0000","last_became_peered":"2023-10-06T05:22:33.067411+0000","last_unstale":"2023-10-06T05:22:33.076909+0000","last_undegraded":"2023-10-06T05:22:33.076909+0000","last_fullsized":"2023-10-06T05:22:33.076909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:04.980573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064392+0000","last_change":"2023-10-06T05:22:33.057186+0000","last_active":"2023-10-06T05:22:33.064392+0000","last_peered":"2023-10-06T05:22:33.064392+0000","last_clean":"2023-10-06T05:22:33.064392+0000","last_became_active":"2023-10-06T05:22:33.056802+0000","last_became_peered":"2023-10-06T05:22:33.056802+0000","last_unstale":"2023-10-06T05:22:33.064392+0000","last_undegraded":"2023-10-06T05:22:33.064392+0000","last_fullsized":"2023-10-06T05:22:33.064392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:54.052349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063764+0000","last_change":"2023-10-06T05:22:33.060931+0000","last_active":"2023-10-06T05:22:33.063764+0000","last_peered":"2023-10-06T05:22:33.063764+0000","last_clean":"2023-10-06T05:22:33.063764+0000","last_became_active":"2023-10-06T05:22:33.060633+0000","last_became_peered":"2023-10-06T05:22:33.060633+0000","last_unstale":"2023-10-06T05:22:33.063764+0000","last_undegraded":"2023-10-06T05:22:33.063764+0000","last_fullsized":"2023-10-06T05:22:33.063764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:22.192166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064349+0000","last_change":"2023-10-06T05:22:33.061625+0000","last_active":"2023-10-06T05:22:33.064349+0000","last_peered":"2023-10-06T05:22:33.064349+0000","last_clean":"2023-10-06T05:22:33.064349+0000","last_became_active":"2023-10-06T05:22:33.061382+0000","last_became_peered":"2023-10-06T05:22:33.061382+0000","last_unstale":"2023-10-06T05:22:33.064349+0000","last_undegraded":"2023-10-06T05:22:33.064349+0000","last_fullsized":"2023-10-06T05:22:33.064349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:49.922418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077020+0000","last_change":"2023-10-06T05:22:33.071258+0000","last_active":"2023-10-06T05:22:33.077020+0000","last_peered":"2023-10-06T05:22:33.077020+0000","last_clean":"2023-10-06T05:22:33.077020+0000","last_became_active":"2023-10-06T05:22:33.070934+0000","last_became_peered":"2023-10-06T05:22:33.070934+0000","last_unstale":"2023-10-06T05:22:33.077020+0000","last_undegraded":"2023-10-06T05:22:33.077020+0000","last_fullsized":"2023-10-06T05:22:33.077020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:08.809909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078846+0000","last_change":"2023-10-06T05:22:33.073724+0000","last_active":"2023-10-06T05:22:33.078846+0000","last_peered":"2023-10-06T05:22:33.078846+0000","last_clean":"2023-10-06T05:22:33.078846+0000","last_became_active":"2023-10-06T05:22:33.073464+0000","last_became_peered":"2023-10-06T05:22:33.073464+0000","last_unstale":"2023-10-06T05:22:33.078846+0000","last_undegraded":"2023-10-06T05:22:33.078846+0000","last_fullsized":"2023-10-06T05:22:33.078846+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:47.725763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071620+0000","last_change":"2023-10-06T05:22:33.064542+0000","last_active":"2023-10-06T05:22:33.071620+0000","last_peered":"2023-10-06T05:22:33.071620+0000","last_clean":"2023-10-06T05:22:33.071620+0000","last_became_active":"2023-10-06T05:22:33.064136+0000","last_became_peered":"2023-10-06T05:22:33.064136+0000","last_unstale":"2023-10-06T05:22:33.071620+0000","last_undegraded":"2023-10-06T05:22:33.071620+0000","last_fullsized":"2023-10-06T05:22:33.071620+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:06.728920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064450+0000","last_change":"2023-10-06T05:22:33.056458+0000","last_active":"2023-10-06T05:22:33.064450+0000","last_peered":"2023-10-06T05:22:33.064450+0000","last_clean":"2023-10-06T05:22:33.064450+0000","last_became_active":"2023-10-06T05:22:33.056171+0000","last_became_peered":"2023-10-06T05:22:33.056171+0000","last_unstale":"2023-10-06T05:22:33.064450+0000","last_undegraded":"2023-10-06T05:22:33.064450+0000","last_fullsized":"2023-10-06T05:22:33.064450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:22.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059845+0000","last_change":"2023-10-06T05:22:33.055602+0000","last_active":"2023-10-06T05:22:33.059845+0000","last_peered":"2023-10-06T05:22:33.059845+0000","last_clean":"2023-10-06T05:22:33.059845+0000","last_became_active":"2023-10-06T05:22:33.054716+0000","last_became_peered":"2023-10-06T05:22:33.054716+0000","last_unstale":"2023-10-06T05:22:33.059845+0000","last_undegraded":"2023-10-06T05:22:33.059845+0000","last_fullsized":"2023-10-06T05:22:33.059845+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:03.942380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065045+0000","last_change":"2023-10-06T05:22:33.062809+0000","last_active":"2023-10-06T05:22:33.065045+0000","last_peered":"2023-10-06T05:22:33.065045+0000","last_clean":"2023-10-06T05:22:33.065045+0000","last_became_active":"2023-10-06T05:22:33.062546+0000","last_became_peered":"2023-10-06T05:22:33.062546+0000","last_unstale":"2023-10-06T05:22:33.065045+0000","last_undegraded":"2023-10-06T05:22:33.065045+0000","last_fullsized":"2023-10-06T05:22:33.065045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:49.962638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.052642+0000","last_change":"2023-10-06T05:22:33.050531+0000","last_active":"2023-10-06T05:22:33.052642+0000","last_peered":"2023-10-06T05:22:33.052642+0000","last_clean":"2023-10-06T05:22:33.052642+0000","last_became_active":"2023-10-06T05:22:33.050229+0000","last_became_peered":"2023-10-06T05:22:33.050229+0000","last_unstale":"2023-10-06T05:22:33.052642+0000","last_undegraded":"2023-10-06T05:22:33.052642+0000","last_fullsized":"2023-10-06T05:22:33.052642+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:40.575078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071086+0000","last_change":"2023-10-06T05:22:33.063546+0000","last_active":"2023-10-06T05:22:33.071086+0000","last_peered":"2023-10-06T05:22:33.071086+0000","last_clean":"2023-10-06T05:22:33.071086+0000","last_became_active":"2023-10-06T05:22:33.063267+0000","last_became_peered":"2023-10-06T05:22:33.063267+0000","last_unstale":"2023-10-06T05:22:33.071086+0000","last_undegraded":"2023-10-06T05:22:33.071086+0000","last_fullsized":"2023-10-06T05:22:33.071086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:14.467014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.075474+0000","last_change":"2023-10-06T05:22:33.073871+0000","last_active":"2023-10-06T05:22:33.075474+0000","last_peered":"2023-10-06T05:22:33.075474+0000","last_clean":"2023-10-06T05:22:33.075474+0000","last_became_active":"2023-10-06T05:22:33.073584+0000","last_became_peered":"2023-10-06T05:22:33.073584+0000","last_unstale":"2023-10-06T05:22:33.075474+0000","last_undegraded":"2023-10-06T05:22:33.075474+0000","last_fullsized":"2023-10-06T05:22:33.075474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:57.713424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062570+0000","last_change":"2023-10-06T05:22:33.057028+0000","last_active":"2023-10-06T05:22:33.062570+0000","last_peered":"2023-10-06T05:22:33.062570+0000","last_clean":"2023-10-06T05:22:33.062570+0000","last_became_active":"2023-10-06T05:22:33.056667+0000","last_became_peered":"2023-10-06T05:22:33.056667+0000","last_unstale":"2023-10-06T05:22:33.062570+0000","last_undegraded":"2023-10-06T05:22:33.062570+0000","last_fullsized":"2023-10-06T05:22:33.062570+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:00.470122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077074+0000","last_change":"2023-10-06T05:22:33.064036+0000","last_active":"2023-10-06T05:22:33.077074+0000","last_peered":"2023-10-06T05:22:33.077074+0000","last_clean":"2023-10-06T05:22:33.077074+0000","last_became_active":"2023-10-06T05:22:33.063620+0000","last_became_peered":"2023-10-06T05:22:33.063620+0000","last_unstale":"2023-10-06T05:22:33.077074+0000","last_undegraded":"2023-10-06T05:22:33.077074+0000","last_fullsized":"2023-10-06T05:22:33.077074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:38.128014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.070666+0000","last_change":"2023-10-06T05:22:32.053439+0000","last_active":"2023-10-06T05:22:33.070666+0000","last_peered":"2023-10-06T05:22:33.070666+0000","last_clean":"2023-10-06T05:22:33.070666+0000","last_became_active":"2023-10-06T05:22:32.053183+0000","last_became_peered":"2023-10-06T05:22:32.053183+0000","last_unstale":"2023-10-06T05:22:33.070666+0000","last_undegraded":"2023-10-06T05:22:33.070666+0000","last_fullsized":"2023-10-06T05:22:33.070666+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:23.173077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064771+0000","last_change":"2023-10-06T05:22:33.059313+0000","last_active":"2023-10-06T05:22:33.064771+0000","last_peered":"2023-10-06T05:22:33.064771+0000","last_clean":"2023-10-06T05:22:33.064771+0000","last_became_active":"2023-10-06T05:22:33.059065+0000","last_became_peered":"2023-10-06T05:22:33.059065+0000","last_unstale":"2023-10-06T05:22:33.064771+0000","last_undegraded":"2023-10-06T05:22:33.064771+0000","last_fullsized":"2023-10-06T05:22:33.064771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:29.790989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072412+0000","last_change":"2023-10-06T05:22:32.054436+0000","last_active":"2023-10-06T05:22:33.072412+0000","last_peered":"2023-10-06T05:22:33.072412+0000","last_clean":"2023-10-06T05:22:33.072412+0000","last_became_active":"2023-10-06T05:22:32.054147+0000","last_became_peered":"2023-10-06T05:22:32.054147+0000","last_unstale":"2023-10-06T05:22:33.072412+0000","last_undegraded":"2023-10-06T05:22:33.072412+0000","last_fullsized":"2023-10-06T05:22:33.072412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.372882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071641+0000","last_change":"2023-10-06T05:22:33.066515+0000","last_active":"2023-10-06T05:22:33.071641+0000","last_peered":"2023-10-06T05:22:33.071641+0000","last_clean":"2023-10-06T05:22:33.071641+0000","last_became_active":"2023-10-06T05:22:33.066223+0000","last_became_peered":"2023-10-06T05:22:33.066223+0000","last_unstale":"2023-10-06T05:22:33.071641+0000","last_undegraded":"2023-10-06T05:22:33.071641+0000","last_fullsized":"2023-10-06T05:22:33.071641+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:11.517044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071020+0000","last_change":"2023-10-06T05:22:32.062729+0000","last_active":"2023-10-06T05:22:33.071020+0000","last_peered":"2023-10-06T05:22:33.071020+0000","last_clean":"2023-10-06T05:22:33.071020+0000","last_became_active":"2023-10-06T05:22:32.062415+0000","last_became_peered":"2023-10-06T05:22:32.062415+0000","last_unstale":"2023-10-06T05:22:33.071020+0000","last_undegraded":"2023-10-06T05:22:33.071020+0000","last_fullsized":"2023-10-06T05:22:33.071020+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:54.464479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059421+0000","last_change":"2023-10-06T05:22:33.054407+0000","last_active":"2023-10-06T05:22:33.059421+0000","last_peered":"2023-10-06T05:22:33.059421+0000","last_clean":"2023-10-06T05:22:33.059421+0000","last_became_active":"2023-10-06T05:22:33.054115+0000","last_became_peered":"2023-10-06T05:22:33.054115+0000","last_unstale":"2023-10-06T05:22:33.059421+0000","last_undegraded":"2023-10-06T05:22:33.059421+0000","last_fullsized":"2023-10-06T05:22:33.059421+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:24.179806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076909+0000","last_change":"2023-10-06T05:22:32.054381+0000","last_active":"2023-10-06T05:22:33.076909+0000","last_peered":"2023-10-06T05:22:33.076909+0000","last_clean":"2023-10-06T05:22:33.076909+0000","last_became_active":"2023-10-06T05:22:32.053971+0000","last_became_peered":"2023-10-06T05:22:32.053971+0000","last_unstale":"2023-10-06T05:22:33.076909+0000","last_undegraded":"2023-10-06T05:22:33.076909+0000","last_fullsized":"2023-10-06T05:22:33.076909+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:41.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.069850+0000","last_change":"2023-10-06T05:22:33.059848+0000","last_active":"2023-10-06T05:22:33.069850+0000","last_peered":"2023-10-06T05:22:33.069850+0000","last_clean":"2023-10-06T05:22:33.069850+0000","last_became_active":"2023-10-06T05:22:33.059548+0000","last_became_peered":"2023-10-06T05:22:33.059548+0000","last_unstale":"2023-10-06T05:22:33.069850+0000","last_undegraded":"2023-10-06T05:22:33.069850+0000","last_fullsized":"2023-10-06T05:22:33.069850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:04.345313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.073162+0000","last_change":"2023-10-06T05:22:32.054712+0000","last_active":"2023-10-06T05:22:33.073162+0000","last_peered":"2023-10-06T05:22:33.073162+0000","last_clean":"2023-10-06T05:22:33.073162+0000","last_became_active":"2023-10-06T05:22:32.054389+0000","last_became_peered":"2023-10-06T05:22:32.054389+0000","last_unstale":"2023-10-06T05:22:33.073162+0000","last_undegraded":"2023-10-06T05:22:33.073162+0000","last_fullsized":"2023-10-06T05:22:33.073162+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:29.234061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076665+0000","last_change":"2023-10-06T05:22:33.062245+0000","last_active":"2023-10-06T05:22:33.076665+0000","last_peered":"2023-10-06T05:22:33.076665+0000","last_clean":"2023-10-06T05:22:33.076665+0000","last_became_active":"2023-10-06T05:22:33.061880+0000","last_became_peered":"2023-10-06T05:22:33.061880+0000","last_unstale":"2023-10-06T05:22:33.076665+0000","last_undegraded":"2023-10-06T05:22:33.076665+0000","last_fullsized":"2023-10-06T05:22:33.076665+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:05.582445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068878+0000","last_change":"2023-10-06T05:22:32.062015+0000","last_active":"2023-10-06T05:22:33.068878+0000","last_peered":"2023-10-06T05:22:33.068878+0000","last_clean":"2023-10-06T05:22:33.068878+0000","last_became_active":"2023-10-06T05:22:32.061659+0000","last_became_peered":"2023-10-06T05:22:32.061659+0000","last_unstale":"2023-10-06T05:22:33.068878+0000","last_undegraded":"2023-10-06T05:22:33.068878+0000","last_fullsized":"2023-10-06T05:22:33.068878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:27.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068348+0000","last_change":"2023-10-06T05:22:33.065794+0000","last_active":"2023-10-06T05:22:33.068348+0000","last_peered":"2023-10-06T05:22:33.068348+0000","last_clean":"2023-10-06T05:22:33.068348+0000","last_became_active":"2023-10-06T05:22:33.065328+0000","last_became_peered":"2023-10-06T05:22:33.065328+0000","last_unstale":"2023-10-06T05:22:33.068348+0000","last_undegraded":"2023-10-06T05:22:33.068348+0000","last_fullsized":"2023-10-06T05:22:33.068348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:10.923071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.047012+0000","last_change":"2023-10-06T05:22:32.057906+0000","last_active":"2023-10-06T05:22:33.047012+0000","last_peered":"2023-10-06T05:22:33.047012+0000","last_clean":"2023-10-06T05:22:33.047012+0000","last_became_active":"2023-10-06T05:22:32.057323+0000","last_became_peered":"2023-10-06T05:22:32.057323+0000","last_unstale":"2023-10-06T05:22:33.047012+0000","last_undegraded":"2023-10-06T05:22:33.047012+0000","last_fullsized":"2023-10-06T05:22:33.047012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:54.920418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076561+0000","last_change":"2023-10-06T05:22:33.074205+0000","last_active":"2023-10-06T05:22:33.076561+0000","last_peered":"2023-10-06T05:22:33.076561+0000","last_clean":"2023-10-06T05:22:33.076561+0000","last_became_active":"2023-10-06T05:22:33.073583+0000","last_became_peered":"2023-10-06T05:22:33.073583+0000","last_unstale":"2023-10-06T05:22:33.076561+0000","last_undegraded":"2023-10-06T05:22:33.076561+0000","last_fullsized":"2023-10-06T05:22:33.076561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:55.471207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.043562+0000","last_change":"2023-10-06T05:22:32.044400+0000","last_active":"2023-10-06T05:22:33.043562+0000","last_peered":"2023-10-06T05:22:33.043562+0000","last_clean":"2023-10-06T05:22:33.043562+0000","last_became_active":"2023-10-06T05:22:32.044113+0000","last_became_peered":"2023-10-06T05:22:32.044113+0000","last_unstale":"2023-10-06T05:22:33.043562+0000","last_undegraded":"2023-10-06T05:22:33.043562+0000","last_fullsized":"2023-10-06T05:22:33.043562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:01.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077091+0000","last_change":"2023-10-06T05:22:33.067562+0000","last_active":"2023-10-06T05:22:33.077091+0000","last_peered":"2023-10-06T05:22:33.077091+0000","last_clean":"2023-10-06T05:22:33.077091+0000","last_became_active":"2023-10-06T05:22:33.067224+0000","last_became_peered":"2023-10-06T05:22:33.067224+0000","last_unstale":"2023-10-06T05:22:33.077091+0000","last_undegraded":"2023-10-06T05:22:33.077091+0000","last_fullsized":"2023-10-06T05:22:33.077091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:45.268811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.048127+0000","last_change":"2023-10-06T05:22:32.056847+0000","last_active":"2023-10-06T05:22:33.048127+0000","last_peered":"2023-10-06T05:22:33.048127+0000","last_clean":"2023-10-06T05:22:33.048127+0000","last_became_active":"2023-10-06T05:22:32.056552+0000","last_became_peered":"2023-10-06T05:22:32.056552+0000","last_unstale":"2023-10-06T05:22:33.048127+0000","last_undegraded":"2023-10-06T05:22:33.048127+0000","last_fullsized":"2023-10-06T05:22:33.048127+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:42.984045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068280+0000","last_change":"2023-10-06T05:22:33.066298+0000","last_active":"2023-10-06T05:22:33.068280+0000","last_peered":"2023-10-06T05:22:33.068280+0000","last_clean":"2023-10-06T05:22:33.068280+0000","last_became_active":"2023-10-06T05:22:33.065366+0000","last_became_peered":"2023-10-06T05:22:33.065366+0000","last_unstale":"2023-10-06T05:22:33.068280+0000","last_undegraded":"2023-10-06T05:22:33.068280+0000","last_fullsized":"2023-10-06T05:22:33.068280+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:03.942380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.045045+0000","last_change":"2023-10-06T05:22:32.055847+0000","last_active":"2023-10-06T05:22:33.045045+0000","last_peered":"2023-10-06T05:22:33.045045+0000","last_clean":"2023-10-06T05:22:33.045045+0000","last_became_active":"2023-10-06T05:22:32.055526+0000","last_became_peered":"2023-10-06T05:22:32.055526+0000","last_unstale":"2023-10-06T05:22:33.045045+0000","last_undegraded":"2023-10-06T05:22:33.045045+0000","last_fullsized":"2023-10-06T05:22:33.045045+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:02.932701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065488+0000","last_change":"2023-10-06T05:22:33.060005+0000","last_active":"2023-10-06T05:22:33.065488+0000","last_peered":"2023-10-06T05:22:33.065488+0000","last_clean":"2023-10-06T05:22:33.065488+0000","last_became_active":"2023-10-06T05:22:33.059711+0000","last_became_peered":"2023-10-06T05:22:33.059711+0000","last_unstale":"2023-10-06T05:22:33.065488+0000","last_undegraded":"2023-10-06T05:22:33.065488+0000","last_fullsized":"2023-10-06T05:22:33.065488+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.884857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078002+0000","last_change":"2023-10-06T05:22:32.054607+0000","last_active":"2023-10-06T05:22:33.078002+0000","last_peered":"2023-10-06T05:22:33.078002+0000","last_clean":"2023-10-06T05:22:33.078002+0000","last_became_active":"2023-10-06T05:22:32.054320+0000","last_became_peered":"2023-10-06T05:22:32.054320+0000","last_unstale":"2023-10-06T05:22:33.078002+0000","last_undegraded":"2023-10-06T05:22:33.078002+0000","last_fullsized":"2023-10-06T05:22:33.078002+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:54.464479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068235+0000","last_change":"2023-10-06T05:22:33.064676+0000","last_active":"2023-10-06T05:22:33.068235+0000","last_peered":"2023-10-06T05:22:33.068235+0000","last_clean":"2023-10-06T05:22:33.068235+0000","last_became_active":"2023-10-06T05:22:33.064338+0000","last_became_peered":"2023-10-06T05:22:33.064338+0000","last_unstale":"2023-10-06T05:22:33.068235+0000","last_undegraded":"2023-10-06T05:22:33.068235+0000","last_fullsized":"2023-10-06T05:22:33.068235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.956570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.080125+0000","last_change":"2023-10-06T05:22:32.051465+0000","last_active":"2023-10-06T05:22:33.080125+0000","last_peered":"2023-10-06T05:22:33.080125+0000","last_clean":"2023-10-06T05:22:33.080125+0000","last_became_active":"2023-10-06T05:22:32.051099+0000","last_became_peered":"2023-10-06T05:22:32.051099+0000","last_unstale":"2023-10-06T05:22:33.080125+0000","last_undegraded":"2023-10-06T05:22:33.080125+0000","last_fullsized":"2023-10-06T05:22:33.080125+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:30.169712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.073248+0000","last_change":"2023-10-06T05:22:33.071230+0000","last_active":"2023-10-06T05:22:33.073248+0000","last_peered":"2023-10-06T05:22:33.073248+0000","last_clean":"2023-10-06T05:22:33.073248+0000","last_became_active":"2023-10-06T05:22:33.070965+0000","last_became_peered":"2023-10-06T05:22:33.070965+0000","last_unstale":"2023-10-06T05:22:33.073248+0000","last_undegraded":"2023-10-06T05:22:33.073248+0000","last_fullsized":"2023-10-06T05:22:33.073248+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.956570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.079345+0000","last_change":"2023-10-06T05:22:32.058056+0000","last_active":"2023-10-06T05:22:33.079345+0000","last_peered":"2023-10-06T05:22:33.079345+0000","last_clean":"2023-10-06T05:22:33.079345+0000","last_became_active":"2023-10-06T05:22:32.057684+0000","last_became_peered":"2023-10-06T05:22:32.057684+0000","last_unstale":"2023-10-06T05:22:33.079345+0000","last_undegraded":"2023-10-06T05:22:33.079345+0000","last_fullsized":"2023-10-06T05:22:33.079345+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:01.611932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063851+0000","last_change":"2023-10-06T05:22:33.058171+0000","last_active":"2023-10-06T05:22:33.063851+0000","last_peered":"2023-10-06T05:22:33.063851+0000","last_clean":"2023-10-06T05:22:33.063851+0000","last_became_active":"2023-10-06T05:22:33.057763+0000","last_became_peered":"2023-10-06T05:22:33.057763+0000","last_unstale":"2023-10-06T05:22:33.063851+0000","last_undegraded":"2023-10-06T05:22:33.063851+0000","last_fullsized":"2023-10-06T05:22:33.063851+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:11.672548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.043980+0000","last_change":"2023-10-06T05:22:32.051695+0000","last_active":"2023-10-06T05:22:33.043980+0000","last_peered":"2023-10-06T05:22:33.043980+0000","last_clean":"2023-10-06T05:22:33.043980+0000","last_became_active":"2023-10-06T05:22:32.051432+0000","last_became_peered":"2023-10-06T05:22:32.051432+0000","last_unstale":"2023-10-06T05:22:33.043980+0000","last_undegraded":"2023-10-06T05:22:33.043980+0000","last_fullsized":"2023-10-06T05:22:33.043980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:02.935652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063043+0000","last_change":"2023-10-06T05:22:33.056159+0000","last_active":"2023-10-06T05:22:33.063043+0000","last_peered":"2023-10-06T05:22:33.063043+0000","last_clean":"2023-10-06T05:22:33.063043+0000","last_became_active":"2023-10-06T05:22:33.055872+0000","last_became_peered":"2023-10-06T05:22:33.055872+0000","last_unstale":"2023-10-06T05:22:33.063043+0000","last_undegraded":"2023-10-06T05:22:33.063043+0000","last_fullsized":"2023-10-06T05:22:33.063043+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:38.181109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.039235+0000","last_change":"2023-10-06T05:22:32.059311+0000","last_active":"2023-10-06T05:22:33.039235+0000","last_peered":"2023-10-06T05:22:33.039235+0000","last_clean":"2023-10-06T05:22:33.039235+0000","last_became_active":"2023-10-06T05:22:32.059015+0000","last_became_peered":"2023-10-06T05:22:32.059015+0000","last_unstale":"2023-10-06T05:22:33.039235+0000","last_undegraded":"2023-10-06T05:22:33.039235+0000","last_fullsized":"2023-10-06T05:22:33.039235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:45.219115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077026+0000","last_change":"2023-10-06T05:22:33.071272+0000","last_active":"2023-10-06T05:22:33.077026+0000","last_peered":"2023-10-06T05:22:33.077026+0000","last_clean":"2023-10-06T05:22:33.077026+0000","last_became_active":"2023-10-06T05:22:33.067595+0000","last_became_peered":"2023-10-06T05:22:33.067595+0000","last_unstale":"2023-10-06T05:22:33.077026+0000","last_undegraded":"2023-10-06T05:22:33.077026+0000","last_fullsized":"2023-10-06T05:22:33.077026+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:55.283888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071807+0000","last_change":"2023-10-06T05:22:32.053722+0000","last_active":"2023-10-06T05:22:33.071807+0000","last_peered":"2023-10-06T05:22:33.071807+0000","last_clean":"2023-10-06T05:22:33.071807+0000","last_became_active":"2023-10-06T05:22:32.053444+0000","last_became_peered":"2023-10-06T05:22:32.053444+0000","last_unstale":"2023-10-06T05:22:33.071807+0000","last_undegraded":"2023-10-06T05:22:33.071807+0000","last_fullsized":"2023-10-06T05:22:33.071807+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:29.234061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077057+0000","last_change":"2023-10-06T05:22:33.063873+0000","last_active":"2023-10-06T05:22:33.077057+0000","last_peered":"2023-10-06T05:22:33.077057+0000","last_clean":"2023-10-06T05:22:33.077057+0000","last_became_active":"2023-10-06T05:22:33.063346+0000","last_became_peered":"2023-10-06T05:22:33.063346+0000","last_unstale":"2023-10-06T05:22:33.077057+0000","last_undegraded":"2023-10-06T05:22:33.077057+0000","last_fullsized":"2023-10-06T05:22:33.077057+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:58.347282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076815+0000","last_change":"2023-10-06T05:22:32.059484+0000","last_active":"2023-10-06T05:22:33.076815+0000","last_peered":"2023-10-06T05:22:33.076815+0000","last_clean":"2023-10-06T05:22:33.076815+0000","last_became_active":"2023-10-06T05:22:32.059228+0000","last_became_peered":"2023-10-06T05:22:32.059228+0000","last_unstale":"2023-10-06T05:22:33.076815+0000","last_undegraded":"2023-10-06T05:22:33.076815+0000","last_fullsized":"2023-10-06T05:22:33.076815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:27.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064424+0000","last_change":"2023-10-06T05:22:33.056125+0000","last_active":"2023-10-06T05:22:33.064424+0000","last_peered":"2023-10-06T05:22:33.064424+0000","last_clean":"2023-10-06T05:22:33.064424+0000","last_became_active":"2023-10-06T05:22:33.055849+0000","last_became_peered":"2023-10-06T05:22:33.055849+0000","last_unstale":"2023-10-06T05:22:33.064424+0000","last_undegraded":"2023-10-06T05:22:33.064424+0000","last_fullsized":"2023-10-06T05:22:33.064424+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:43.019236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T05:22:34.052903+0000","last_change":"2023-10-06T05:22:32.062155+0000","last_active":"2023-10-06T05:22:34.052903+0000","last_peered":"2023-10-06T05:22:34.052903+0000","last_clean":"2023-10-06T05:22:34.052903+0000","last_became_active":"2023-10-06T05:22:32.061860+0000","last_became_peered":"2023-10-06T05:22:32.061860+0000","last_unstale":"2023-10-06T05:22:34.052903+0000","last_undegraded":"2023-10-06T05:22:34.052903+0000","last_fullsized":"2023-10-06T05:22:34.052903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:54.479528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.069815+0000","last_change":"2023-10-06T05:22:33.059090+0000","last_active":"2023-10-06T05:22:33.069815+0000","last_peered":"2023-10-06T05:22:33.069815+0000","last_clean":"2023-10-06T05:22:33.069815+0000","last_became_active":"2023-10-06T05:22:33.058836+0000","last_became_peered":"2023-10-06T05:22:33.058836+0000","last_unstale":"2023-10-06T05:22:33.069815+0000","last_undegraded":"2023-10-06T05:22:33.069815+0000","last_fullsized":"2023-10-06T05:22:33.069815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:43.990773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.042006+0000","last_change":"2023-10-06T05:22:32.051860+0000","last_active":"2023-10-06T05:22:33.042006+0000","last_peered":"2023-10-06T05:22:33.042006+0000","last_clean":"2023-10-06T05:22:33.042006+0000","last_became_active":"2023-10-06T05:22:32.051570+0000","last_became_peered":"2023-10-06T05:22:32.051570+0000","last_unstale":"2023-10-06T05:22:33.042006+0000","last_undegraded":"2023-10-06T05:22:33.042006+0000","last_fullsized":"2023-10-06T05:22:33.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:23.584216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072285+0000","last_change":"2023-10-06T05:22:33.069678+0000","last_active":"2023-10-06T05:22:33.072285+0000","last_peered":"2023-10-06T05:22:33.072285+0000","last_clean":"2023-10-06T05:22:33.072285+0000","last_became_active":"2023-10-06T05:22:33.069419+0000","last_became_peered":"2023-10-06T05:22:33.069419+0000","last_unstale":"2023-10-06T05:22:33.072285+0000","last_undegraded":"2023-10-06T05:22:33.072285+0000","last_fullsized":"2023-10-06T05:22:33.072285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:47.725763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.053257+0000","last_change":"2023-10-06T05:22:32.054223+0000","last_active":"2023-10-06T05:22:33.053257+0000","last_peered":"2023-10-06T05:22:33.053257+0000","last_clean":"2023-10-06T05:22:33.053257+0000","last_became_active":"2023-10-06T05:22:32.053521+0000","last_became_peered":"2023-10-06T05:22:32.053521+0000","last_unstale":"2023-10-06T05:22:33.053257+0000","last_undegraded":"2023-10-06T05:22:33.053257+0000","last_fullsized":"2023-10-06T05:22:33.053257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:02.462579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063600+0000","last_change":"2023-10-06T05:22:33.060710+0000","last_active":"2023-10-06T05:22:33.063600+0000","last_peered":"2023-10-06T05:22:33.063600+0000","last_clean":"2023-10-06T05:22:33.063600+0000","last_became_active":"2023-10-06T05:22:33.060403+0000","last_became_peered":"2023-10-06T05:22:33.060403+0000","last_unstale":"2023-10-06T05:22:33.063600+0000","last_undegraded":"2023-10-06T05:22:33.063600+0000","last_fullsized":"2023-10-06T05:22:33.063600+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:24.590944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.045148+0000","last_change":"2023-10-06T05:22:32.056002+0000","last_active":"2023-10-06T05:22:33.045148+0000","last_peered":"2023-10-06T05:22:33.045148+0000","last_clean":"2023-10-06T05:22:33.045148+0000","last_became_active":"2023-10-06T05:22:32.055716+0000","last_became_peered":"2023-10-06T05:22:32.055716+0000","last_unstale":"2023-10-06T05:22:33.045148+0000","last_undegraded":"2023-10-06T05:22:33.045148+0000","last_fullsized":"2023-10-06T05:22:33.045148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:03.338584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064662+0000","last_change":"2023-10-06T05:22:33.058496+0000","last_active":"2023-10-06T05:22:33.064662+0000","last_peered":"2023-10-06T05:22:33.064662+0000","last_clean":"2023-10-06T05:22:33.064662+0000","last_became_active":"2023-10-06T05:22:33.058206+0000","last_became_peered":"2023-10-06T05:22:33.058206+0000","last_unstale":"2023-10-06T05:22:33.064662+0000","last_undegraded":"2023-10-06T05:22:33.064662+0000","last_fullsized":"2023-10-06T05:22:33.064662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:26.459752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040192+0000","last_change":"2023-10-06T05:22:32.059218+0000","last_active":"2023-10-06T05:22:33.040192+0000","last_peered":"2023-10-06T05:22:33.040192+0000","last_clean":"2023-10-06T05:22:33.040192+0000","last_became_active":"2023-10-06T05:22:32.058954+0000","last_became_peered":"2023-10-06T05:22:32.058954+0000","last_unstale":"2023-10-06T05:22:33.040192+0000","last_undegraded":"2023-10-06T05:22:33.040192+0000","last_fullsized":"2023-10-06T05:22:33.040192+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:46.497739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.067414+0000","last_change":"2023-10-06T05:22:33.063760+0000","last_active":"2023-10-06T05:22:33.067414+0000","last_peered":"2023-10-06T05:22:33.067414+0000","last_clean":"2023-10-06T05:22:33.067414+0000","last_became_active":"2023-10-06T05:22:33.063478+0000","last_became_peered":"2023-10-06T05:22:33.063478+0000","last_unstale":"2023-10-06T05:22:33.067414+0000","last_undegraded":"2023-10-06T05:22:33.067414+0000","last_fullsized":"2023-10-06T05:22:33.067414+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:02.999759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.041934+0000","last_change":"2023-10-06T05:22:32.052512+0000","last_active":"2023-10-06T05:22:33.041934+0000","last_peered":"2023-10-06T05:22:33.041934+0000","last_clean":"2023-10-06T05:22:33.041934+0000","last_became_active":"2023-10-06T05:22:32.052252+0000","last_became_peered":"2023-10-06T05:22:32.052252+0000","last_unstale":"2023-10-06T05:22:33.041934+0000","last_undegraded":"2023-10-06T05:22:33.041934+0000","last_fullsized":"2023-10-06T05:22:33.041934+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:28.784261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065508+0000","last_change":"2023-10-06T05:22:33.060430+0000","last_active":"2023-10-06T05:22:33.065508+0000","last_peered":"2023-10-06T05:22:33.065508+0000","last_clean":"2023-10-06T05:22:33.065508+0000","last_became_active":"2023-10-06T05:22:33.060098+0000","last_became_peered":"2023-10-06T05:22:33.060098+0000","last_unstale":"2023-10-06T05:22:33.065508+0000","last_undegraded":"2023-10-06T05:22:33.065508+0000","last_fullsized":"2023-10-06T05:22:33.065508+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:02.189479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040146+0000","last_change":"2023-10-06T05:22:32.058565+0000","last_active":"2023-10-06T05:22:33.040146+0000","last_peered":"2023-10-06T05:22:33.040146+0000","last_clean":"2023-10-06T05:22:33.040146+0000","last_became_active":"2023-10-06T05:22:32.058288+0000","last_became_peered":"2023-10-06T05:22:32.058288+0000","last_unstale":"2023-10-06T05:22:33.040146+0000","last_undegraded":"2023-10-06T05:22:33.040146+0000","last_fullsized":"2023-10-06T05:22:33.040146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:31.338180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063302+0000","last_change":"2023-10-06T05:22:33.058532+0000","last_active":"2023-10-06T05:22:33.063302+0000","last_peered":"2023-10-06T05:22:33.063302+0000","last_clean":"2023-10-06T05:22:33.063302+0000","last_became_active":"2023-10-06T05:22:33.058223+0000","last_became_peered":"2023-10-06T05:22:33.058223+0000","last_unstale":"2023-10-06T05:22:33.063302+0000","last_undegraded":"2023-10-06T05:22:33.063302+0000","last_fullsized":"2023-10-06T05:22:33.063302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:55.927147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.041783+0000","last_change":"2023-10-06T05:22:32.050247+0000","last_active":"2023-10-06T05:22:33.041783+0000","last_peered":"2023-10-06T05:22:33.041783+0000","last_clean":"2023-10-06T05:22:33.041783+0000","last_became_active":"2023-10-06T05:22:32.049943+0000","last_became_peered":"2023-10-06T05:22:32.049943+0000","last_unstale":"2023-10-06T05:22:33.041783+0000","last_undegraded":"2023-10-06T05:22:33.041783+0000","last_fullsized":"2023-10-06T05:22:33.041783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:25.453024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.053346+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:22:33.053346+0000","last_peered":"2023-10-06T05:22:33.053346+0000","last_clean":"2023-10-06T05:22:33.053346+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:22:33.053346+0000","last_undegraded":"2023-10-06T05:22:33.053346+0000","last_fullsized":"2023-10-06T05:22:33.053346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.075746+0000","last_change":"2023-10-06T05:22:33.073110+0000","last_active":"2023-10-06T05:22:33.075746+0000","last_peered":"2023-10-06T05:22:33.075746+0000","last_clean":"2023-10-06T05:22:33.075746+0000","last_became_active":"2023-10-06T05:22:33.072808+0000","last_became_peered":"2023-10-06T05:22:33.072808+0000","last_unstale":"2023-10-06T05:22:33.075746+0000","last_undegraded":"2023-10-06T05:22:33.075746+0000","last_fullsized":"2023-10-06T05:22:33.075746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:59.642608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072765+0000","last_change":"2023-10-06T05:22:32.057484+0000","last_active":"2023-10-06T05:22:33.072765+0000","last_peered":"2023-10-06T05:22:33.072765+0000","last_clean":"2023-10-06T05:22:33.072765+0000","last_became_active":"2023-10-06T05:22:32.056760+0000","last_became_peered":"2023-10-06T05:22:32.056760+0000","last_unstale":"2023-10-06T05:22:33.072765+0000","last_undegraded":"2023-10-06T05:22:33.072765+0000","last_fullsized":"2023-10-06T05:22:33.072765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.878129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.049516+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:22:33.049516+0000","last_peered":"2023-10-06T05:22:33.049516+0000","last_clean":"2023-10-06T05:22:33.049516+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:22:33.049516+0000","last_undegraded":"2023-10-06T05:22:33.049516+0000","last_fullsized":"2023-10-06T05:22:33.049516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068255+0000","last_change":"2023-10-06T05:22:33.065507+0000","last_active":"2023-10-06T05:22:33.068255+0000","last_peered":"2023-10-06T05:22:33.068255+0000","last_clean":"2023-10-06T05:22:33.068255+0000","last_became_active":"2023-10-06T05:22:33.065187+0000","last_became_peered":"2023-10-06T05:22:33.065187+0000","last_unstale":"2023-10-06T05:22:33.068255+0000","last_undegraded":"2023-10-06T05:22:33.068255+0000","last_fullsized":"2023-10-06T05:22:33.068255+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:02.189479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.049527+0000","last_change":"2023-10-06T05:22:32.061884+0000","last_active":"2023-10-06T05:22:33.049527+0000","last_peered":"2023-10-06T05:22:33.049527+0000","last_clean":"2023-10-06T05:22:33.049527+0000","last_became_active":"2023-10-06T05:22:32.061586+0000","last_became_peered":"2023-10-06T05:22:32.061586+0000","last_unstale":"2023-10-06T05:22:33.049527+0000","last_undegraded":"2023-10-06T05:22:33.049527+0000","last_fullsized":"2023-10-06T05:22:33.049527+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:41.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.039823+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:22:33.039823+0000","last_peered":"2023-10-06T05:22:33.039823+0000","last_clean":"2023-10-06T05:22:33.039823+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:22:33.039823+0000","last_undegraded":"2023-10-06T05:22:33.039823+0000","last_fullsized":"2023-10-06T05:22:33.039823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.074426+0000","last_change":"2023-10-06T05:22:33.073103+0000","last_active":"2023-10-06T05:22:33.074426+0000","last_peered":"2023-10-06T05:22:33.074426+0000","last_clean":"2023-10-06T05:22:33.074426+0000","last_became_active":"2023-10-06T05:22:33.072733+0000","last_became_peered":"2023-10-06T05:22:33.072733+0000","last_unstale":"2023-10-06T05:22:33.074426+0000","last_undegraded":"2023-10-06T05:22:33.074426+0000","last_fullsized":"2023-10-06T05:22:33.074426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:09.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.074507+0000","last_change":"2023-10-06T05:22:32.055034+0000","last_active":"2023-10-06T05:22:33.074507+0000","last_peered":"2023-10-06T05:22:33.074507+0000","last_clean":"2023-10-06T05:22:33.074507+0000","last_became_active":"2023-10-06T05:22:32.054712+0000","last_became_peered":"2023-10-06T05:22:32.054712+0000","last_unstale":"2023-10-06T05:22:33.074507+0000","last_undegraded":"2023-10-06T05:22:33.074507+0000","last_fullsized":"2023-10-06T05:22:33.074507+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:47.406781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.050186+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:22:33.050186+0000","last_peered":"2023-10-06T05:22:33.050186+0000","last_clean":"2023-10-06T05:22:33.050186+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:22:33.050186+0000","last_undegraded":"2023-10-06T05:22:33.050186+0000","last_fullsized":"2023-10-06T05:22:33.050186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044682+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:22:33.044682+0000","last_peered":"2023-10-06T05:22:33.044682+0000","last_clean":"2023-10-06T05:22:33.044682+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:22:33.044682+0000","last_undegraded":"2023-10-06T05:22:33.044682+0000","last_fullsized":"2023-10-06T05:22:33.044682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.066648+0000","last_change":"2023-10-06T05:22:33.063053+0000","last_active":"2023-10-06T05:22:33.066648+0000","last_peered":"2023-10-06T05:22:33.066648+0000","last_clean":"2023-10-06T05:22:33.066648+0000","last_became_active":"2023-10-06T05:22:33.062748+0000","last_became_peered":"2023-10-06T05:22:33.062748+0000","last_unstale":"2023-10-06T05:22:33.066648+0000","last_undegraded":"2023-10-06T05:22:33.066648+0000","last_fullsized":"2023-10-06T05:22:33.066648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:17.956567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071988+0000","last_change":"2023-10-06T05:22:32.055277+0000","last_active":"2023-10-06T05:22:33.071988+0000","last_peered":"2023-10-06T05:22:33.071988+0000","last_clean":"2023-10-06T05:22:33.071988+0000","last_became_active":"2023-10-06T05:22:32.055003+0000","last_became_peered":"2023-10-06T05:22:32.055003+0000","last_unstale":"2023-10-06T05:22:33.071988+0000","last_undegraded":"2023-10-06T05:22:33.071988+0000","last_fullsized":"2023-10-06T05:22:33.071988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:42.984045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044245+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:22:33.044245+0000","last_peered":"2023-10-06T05:22:33.044245+0000","last_clean":"2023-10-06T05:22:33.044245+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:22:33.044245+0000","last_undegraded":"2023-10-06T05:22:33.044245+0000","last_fullsized":"2023-10-06T05:22:33.044245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072363+0000","last_change":"2023-10-06T05:22:33.066198+0000","last_active":"2023-10-06T05:22:33.072363+0000","last_peered":"2023-10-06T05:22:33.072363+0000","last_clean":"2023-10-06T05:22:33.072363+0000","last_became_active":"2023-10-06T05:22:33.065914+0000","last_became_peered":"2023-10-06T05:22:33.065914+0000","last_unstale":"2023-10-06T05:22:33.072363+0000","last_undegraded":"2023-10-06T05:22:33.072363+0000","last_fullsized":"2023-10-06T05:22:33.072363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:06.769845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040040+0000","last_change":"2023-10-06T05:22:32.053709+0000","last_active":"2023-10-06T05:22:33.040040+0000","last_peered":"2023-10-06T05:22:33.040040+0000","last_clean":"2023-10-06T05:22:33.040040+0000","last_became_active":"2023-10-06T05:22:32.053115+0000","last_became_peered":"2023-10-06T05:22:32.053115+0000","last_unstale":"2023-10-06T05:22:33.040040+0000","last_undegraded":"2023-10-06T05:22:33.040040+0000","last_fullsized":"2023-10-06T05:22:33.040040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:24.907445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044708+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:22:33.044708+0000","last_peered":"2023-10-06T05:22:33.044708+0000","last_clean":"2023-10-06T05:22:33.044708+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:22:33.044708+0000","last_undegraded":"2023-10-06T05:22:33.044708+0000","last_fullsized":"2023-10-06T05:22:33.044708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077116+0000","last_change":"2023-10-06T05:22:33.067894+0000","last_active":"2023-10-06T05:22:33.077116+0000","last_peered":"2023-10-06T05:22:33.077116+0000","last_clean":"2023-10-06T05:22:33.077116+0000","last_became_active":"2023-10-06T05:22:33.067580+0000","last_became_peered":"2023-10-06T05:22:33.067580+0000","last_unstale":"2023-10-06T05:22:33.077116+0000","last_undegraded":"2023-10-06T05:22:33.077116+0000","last_fullsized":"2023-10-06T05:22:33.077116+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:31.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078053+0000","last_change":"2023-10-06T05:22:32.053701+0000","last_active":"2023-10-06T05:22:33.078053+0000","last_peered":"2023-10-06T05:22:33.078053+0000","last_clean":"2023-10-06T05:22:33.078053+0000","last_became_active":"2023-10-06T05:22:32.053435+0000","last_became_peered":"2023-10-06T05:22:32.053435+0000","last_unstale":"2023-10-06T05:22:33.078053+0000","last_undegraded":"2023-10-06T05:22:33.078053+0000","last_fullsized":"2023-10-06T05:22:33.078053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:27.622392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.052489+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:22:33.052489+0000","last_peered":"2023-10-06T05:22:33.052489+0000","last_clean":"2023-10-06T05:22:33.052489+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:22:33.052489+0000","last_undegraded":"2023-10-06T05:22:33.052489+0000","last_fullsized":"2023-10-06T05:22:33.052489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076755+0000","last_change":"2023-10-06T05:22:33.063092+0000","last_active":"2023-10-06T05:22:33.076755+0000","last_peered":"2023-10-06T05:22:33.076755+0000","last_clean":"2023-10-06T05:22:33.076755+0000","last_became_active":"2023-10-06T05:22:33.062195+0000","last_became_peered":"2023-10-06T05:22:33.062195+0000","last_unstale":"2023-10-06T05:22:33.076755+0000","last_undegraded":"2023-10-06T05:22:33.076755+0000","last_fullsized":"2023-10-06T05:22:33.076755+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:29.319953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040074+0000","last_change":"2023-10-06T05:22:32.051524+0000","last_active":"2023-10-06T05:22:33.040074+0000","last_peered":"2023-10-06T05:22:33.040074+0000","last_clean":"2023-10-06T05:22:33.040074+0000","last_became_active":"2023-10-06T05:22:32.050882+0000","last_became_peered":"2023-10-06T05:22:32.050882+0000","last_unstale":"2023-10-06T05:22:33.040074+0000","last_undegraded":"2023-10-06T05:22:33.040074+0000","last_fullsized":"2023-10-06T05:22:33.040074+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:08.923388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.042797+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:22:33.042797+0000","last_peered":"2023-10-06T05:22:33.042797+0000","last_clean":"2023-10-06T05:22:33.042797+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:22:33.042797+0000","last_undegraded":"2023-10-06T05:22:33.042797+0000","last_fullsized":"2023-10-06T05:22:33.042797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":1184708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.78100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302672,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069168,"statfs":{"total":96636764160,"available":96326828032,"internally_reserved":0,"allocated":1318912,"data_stored":596292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.621},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.377}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":0.746},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303248,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068592,"statfs":{"total":96636764160,"available":96326238208,"internally_reserved":0,"allocated":1908736,"data_stored":1186174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.501}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.629},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.753},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.78100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.627},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":0.61199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46999999999999997,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.505,"15min":0.5},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":1.772},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.913},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53900000000000003,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":1.954},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":0.96999999999999997,"5min":1.2030000000000001,"15min":1.282},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91500000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":0.98499999999999999,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.874},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.84199999999999997},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.83899999999999997},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0449999999999999,"5min":1.1910000000000001,"15min":1.343},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.84899999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.034,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":1.004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":0.998,"5min":1.2090000000000001,"15min":1.214},"last":0.95399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":594272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.755},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.84599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":594180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.63700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:38.377 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T05:22:38.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.593+0000 7fc1f5f9d700 1 Processor -- start 2023-10-06T05:22:38.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.593+0000 7fc1f5f9d700 1 -- start start 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f5f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1f0159530 0x7fc1f006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f5f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f006e3e0 con 0x7fc1f00b0310 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f5f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00b1e00 con 0x7fc1f0159530 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006f6b0 con 0x7fc1f006df70 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f4f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1f0159530 0x7fc1f006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f579c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc1f00b0310 0x7fc1f0159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55948/0 (socket says 172.21.15.33:55948) 2023-10-06T05:22:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.594+0000 7fc1f579c700 1 -- 172.21.15.33:0/880075981 learned_addr learned my addr 172.21.15.33:0/880075981 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1effff700 1 --2- 172.21.15.33:0/880075981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1f4f9b700 1 -- 172.21.15.33:0/880075981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 msgr2=0x7fc1f00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1f4f9b700 1 --2- 172.21.15.33:0/880075981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f00afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1f4f9b700 1 -- 172.21.15.33:0/880075981 >> v1:172.21.15.33:6789/0 conn(0x7fc1f00b0310 legacy=0x7fc1f0159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1f4f9b700 1 -- 172.21.15.33:0/880075981 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f006f220 con 0x7fc1f0159530 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1f4f9b700 1 --2- 172.21.15.33:0/880075981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f0159530 0x7fc1f006da30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098e0 tx=0x7fc1e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b10e35a974168a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1ef7fe700 1 -- 172.21.15.33:0/880075981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4009e80 con 0x7fc1f0159530 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.595+0000 7fc1ef7fe700 1 -- 172.21.15.33:0/880075981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e400b760 con 0x7fc1f0159530 2023-10-06T05:22:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1ef7fe700 1 -- 172.21.15.33:0/880075981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e400bae0 con 0x7fc1f0159530 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/880075981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f0159530 msgr2=0x7fc1f006da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/880075981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f0159530 0x7fc1f006da30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098e0 tx=0x7fc1e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/880075981 shutdown_connections 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/880075981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f0159530 0x7fc1f006da30 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/880075981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f00afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/880075981 >> 172.21.15.33:0/880075981 conn(0x7fc1f006c560 msgr2=0x7fc1f006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/880075981 shutdown_connections 2023-10-06T05:22:38.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.596+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/880075981 wait complete. 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.598+0000 7fc1f5f9d700 1 Processor -- start 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.598+0000 7fc1f5f9d700 1 -- start start 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f0151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 0x7fc1f0153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1f0159530 0x7fc1f014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01551e0 con 0x7fc1f00b0310 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01545d0 con 0x7fc1f0159530 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01548b0 con 0x7fc1f006df70 2023-10-06T05:22:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 0x7fc1f0153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 0x7fc1f0153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44746/0 (socket says 172.21.15.33:44746) 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 -- 172.21.15.33:0/3582870950 learned_addr learned my addr 172.21.15.33:0/3582870950 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f579c700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1f0159530 0x7fc1f014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1f4f9b700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f0151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 -- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 msgr2=0x7fc1f0151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f0151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 -- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1f0159530 msgr2=0x7fc1f014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1f0159530 0x7fc1f014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.599+0000 7fc1effff700 1 -- 172.21.15.33:0/3582870950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e4009420 con 0x7fc1f00b0310 2023-10-06T05:22:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.600+0000 7fc1effff700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 0x7fc1f0153da0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00d860 tx=0x7fc1dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.600+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc009610 con 0x7fc1f00b0310 2023-10-06T05:22:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.600+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc009770 con 0x7fc1f00b0310 2023-10-06T05:22:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.600+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc009b30 con 0x7fc1f00b0310 2023-10-06T05:22:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.600+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f0154b90 con 0x7fc1f00b0310 2023-10-06T05:22:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.600+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f01542e0 con 0x7fc1f00b0310 2023-10-06T05:22:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.601+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f0155390 con 0x7fc1f00b0310 2023-10-06T05:22:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.602+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1dc01e460 con 0x7fc1f00b0310 2023-10-06T05:22:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.603+0000 7fc1ecff9700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1d4066f30 0x7fc1d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.603+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1dc084cb0 con 0x7fc1f00b0310 2023-10-06T05:22:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.604+0000 7fc1f4f9b700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1d4066f30 0x7fc1d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.604+0000 7fc1f4f9b700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1d4066f30 0x7fc1d40693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40093f0 tx=0x7fc1e4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.607+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1f0155390 con 0x7fc1f00b0310 2023-10-06T05:22:38.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.893+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc1f0158dc0 con 0x7fc1f00b0310 2023-10-06T05:22:38.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.894+0000 7fc1ecff9700 1 -- 172.21.15.33:0/3582870950 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1019 (secure 0 0 0) 0x7fc1dc051dc0 con 0x7fc1f00b0310 2023-10-06T05:22:38.896 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:38.896 INFO:teuthology.orchestra.run.smithi033.stdout:{"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":520,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1696569251,"num_in_osds":8,"osd_in_since":1696569242,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592674,"bytes_used":2480578560,"bytes_avail":770613534720,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7299}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2023-10-06T05:22:13.933041+0000","services":{}},"progress_events":{}} 2023-10-06T05:22:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1d4066f30 msgr2=0x7fc1d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1d4066f30 0x7fc1d40693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40093f0 tx=0x7fc1e4016000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 msgr2=0x7fc1f0153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 0x7fc1f0153da0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00d860 tx=0x7fc1dc008040 comp rx=0 tx=0).stop 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 shutdown_connections 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1d4066f30 0x7fc1d40693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1f0159530 0x7fc1f014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1f006df70 0x7fc1f0151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 --2- 172.21.15.33:0/3582870950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f00b0310 0x7fc1f0153da0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 >> 172.21.15.33:0/3582870950 conn(0x7fc1f006c560 msgr2=0x7fc1f006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 shutdown_connections 2023-10-06T05:22:38.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:38.896+0000 7fc1f5f9d700 1 -- 172.21.15.33:0/3582870950 wait complete. 2023-10-06T05:22:38.916 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T05:22:38.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.102+0000 7f8a027ff700 1 Processor -- start 2023-10-06T05:22:39.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.103+0000 7f8a027ff700 1 -- start start 2023-10-06T05:22:39.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f8a027ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89fc13fb70 0x7f89fc06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f8a027ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89fc06ba40 0x7f89fc0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f8a027ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0ca8d0 con 0x7f89fc0ca350 2023-10-06T05:22:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f8a027ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc0cabb0 con 0x7f89fc06ba40 2023-10-06T05:22:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f8a027ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc06ad00 con 0x7f89fc13fb70 2023-10-06T05:22:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f89fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89fc06ba40 0x7f89fc0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f89fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89fc06ba40 0x7f89fc0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44762/0 (socket says 172.21.15.33:44762) 2023-10-06T05:22:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f89fb7fe700 1 -- 172.21.15.33:0/2467732045 learned_addr learned my addr 172.21.15.33:0/2467732045 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.104+0000 7f89fbfff700 1 --2- 172.21.15.33:0/2467732045 >> v2:172.21.15.167:3300/0 conn(0x7f89fc13fb70 0x7f89fc06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89fbfff700 1 -- 172.21.15.33:0/2467732045 >> v2:172.21.15.33:3300/0 conn(0x7f89fc06ba40 msgr2=0x7f89fc0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89fbfff700 1 --2- 172.21.15.33:0/2467732045 >> v2:172.21.15.33:3300/0 conn(0x7f89fc06ba40 0x7f89fc0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89fbfff700 1 -- 172.21.15.33:0/2467732045 >> v1:172.21.15.167:6789/0 conn(0x7f89fc0ca350 legacy=0x7f89fc0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89fbfff700 1 -- 172.21.15.33:0/2467732045 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc06afe0 con 0x7f89fc13fb70 2023-10-06T05:22:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89fbfff700 1 --2- 172.21.15.33:0/2467732045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc06b500 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06bf00 tx=0x7f89ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34eb8df7d5bdab45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89faffd700 1 -- 172.21.15.33:0/2467732045 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec011630 con 0x7f89fc13fb70 2023-10-06T05:22:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89faffd700 1 -- 172.21.15.33:0/2467732045 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec011790 con 0x7f89fc13fb70 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.105+0000 7f89faffd700 1 -- 172.21.15.33:0/2467732045 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec011b30 con 0x7f89fc13fb70 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 -- 172.21.15.33:0/2467732045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 msgr2=0x7f89fc06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 --2- 172.21.15.33:0/2467732045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc06b500 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06bf00 tx=0x7f89ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 -- 172.21.15.33:0/2467732045 shutdown_connections 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 --2- 172.21.15.33:0/2467732045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc06b500 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 --2- 172.21.15.33:0/2467732045 >> v2:172.21.15.33:3300/0 conn(0x7f89fc06ba40 0x7f89fc0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 -- 172.21.15.33:0/2467732045 >> 172.21.15.33:0/2467732045 conn(0x7f89fc0675f0 msgr2=0x7f89fc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 -- 172.21.15.33:0/2467732045 shutdown_connections 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.106+0000 7f8a027ff700 1 -- 172.21.15.33:0/2467732045 wait complete. 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.108+0000 7f8a027ff700 1 Processor -- start 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.108+0000 7f8a027ff700 1 -- start start 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a027ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89fc06ba40 0x7f89fc0aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a027ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 0x7f89fc13d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a027ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a027ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0afff0 con 0x7f89fc0ca350 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a027ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0c6120 con 0x7f89fc13fb70 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a027ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0af520 con 0x7f89fc06ba40 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 0x7f89fc13d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 0x7f89fc13d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44772/0 (socket says 172.21.15.33:44772) 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 -- 172.21.15.33:0/1854455388 learned_addr learned my addr 172.21.15.33:0/1854455388 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fbfff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89fc06ba40 0x7f89fc0aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a00d9c700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 -- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89fc06ba40 msgr2=0x7f89fc0aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89fc06ba40 0x7f89fc0aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 -- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 msgr2=0x7f89fc158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f89fb7fe700 1 -- 172.21.15.33:0/1854455388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec009420 con 0x7f89fc0ca350 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.109+0000 7f8a00d9c700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.110+0000 7f89fb7fe700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 0x7f89fc13d140 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f89f0010e60 tx=0x7f89f000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.110+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000edf0 con 0x7f89fc0ca350 2023-10-06T05:22:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.110+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f0011450 con 0x7f89fc0ca350 2023-10-06T05:22:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.110+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0011810 con 0x7f89fc0ca350 2023-10-06T05:22:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.110+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc0c63c0 con 0x7f89fc0ca350 2023-10-06T05:22:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.110+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc0af800 con 0x7f89fc0ca350 2023-10-06T05:22:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.111+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc0afe30 con 0x7f89fc0ca350 2023-10-06T05:22:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.113+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89f0011bc0 con 0x7f89fc0ca350 2023-10-06T05:22:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.113+0000 7f89e3fff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.114+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f89f0087360 con 0x7f89fc0ca350 2023-10-06T05:22:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.114+0000 7f89fbfff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.115+0000 7f89fbfff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0093f0 tx=0x7f89ec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.118+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f00543d0 con 0x7f89fc0ca350 2023-10-06T05:22:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.317+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89fc063ff0 con 0x7f89e8066f30 2023-10-06T05:22:39.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.327+0000 7f89e3fff700 1 -- 172.21.15.33:0/1854455388 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311443 (secure 0 0 0) 0x7f89fc063ff0 con 0x7f89e8066f30 2023-10-06T05:22:39.336 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89e8066f30 msgr2=0x7f89e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0093f0 tx=0x7f89ec017000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 msgr2=0x7f89fc13d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 0x7f89fc13d140 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f89f0010e60 tx=0x7f89f000b4f0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 shutdown_connections 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89fc13fb70 0x7f89fc158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89fc06ba40 0x7f89fc0aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 --2- 172.21.15.33:0/1854455388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89fc0ca350 0x7f89fc13d140 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 >> 172.21.15.33:0/1854455388 conn(0x7f89fc0675f0 msgr2=0x7f89fc13ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 shutdown_connections 2023-10-06T05:22:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:39.331+0000 7f8a027ff700 1 -- 172.21.15.33:0/1854455388 wait complete. 2023-10-06T05:22:39.341 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:22:39.354 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2023-10-06T05:22:37.939056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422440,"kb_used_data":11368,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552280,"statfs":{"total":773094113280,"available":770613534720,"internally_reserved":0,"allocated":11640832,"data_stored":5938788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001649"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077617+0000","last_change":"2023-10-06T05:22:33.071449+0000","last_active":"2023-10-06T05:22:33.077617+0000","last_peered":"2023-10-06T05:22:33.077617+0000","last_clean":"2023-10-06T05:22:33.077617+0000","last_became_active":"2023-10-06T05:22:33.070959+0000","last_became_peered":"2023-10-06T05:22:33.070959+0000","last_unstale":"2023-10-06T05:22:33.077617+0000","last_undegraded":"2023-10-06T05:22:33.077617+0000","last_fullsized":"2023-10-06T05:22:33.077617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:53.229898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065900+0000","last_change":"2023-10-06T05:22:33.058445+0000","last_active":"2023-10-06T05:22:33.065900+0000","last_peered":"2023-10-06T05:22:33.065900+0000","last_clean":"2023-10-06T05:22:33.065900+0000","last_became_active":"2023-10-06T05:22:33.058172+0000","last_became_peered":"2023-10-06T05:22:33.058172+0000","last_unstale":"2023-10-06T05:22:33.065900+0000","last_undegraded":"2023-10-06T05:22:33.065900+0000","last_fullsized":"2023-10-06T05:22:33.065900+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:22.347118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076927+0000","last_change":"2023-10-06T05:22:33.070804+0000","last_active":"2023-10-06T05:22:33.076927+0000","last_peered":"2023-10-06T05:22:33.076927+0000","last_clean":"2023-10-06T05:22:33.076927+0000","last_became_active":"2023-10-06T05:22:33.067750+0000","last_became_peered":"2023-10-06T05:22:33.067750+0000","last_unstale":"2023-10-06T05:22:33.076927+0000","last_undegraded":"2023-10-06T05:22:33.076927+0000","last_fullsized":"2023-10-06T05:22:33.076927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:17.036525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063340+0000","last_change":"2023-10-06T05:22:33.054655+0000","last_active":"2023-10-06T05:22:33.063340+0000","last_peered":"2023-10-06T05:22:33.063340+0000","last_clean":"2023-10-06T05:22:33.063340+0000","last_became_active":"2023-10-06T05:22:33.054393+0000","last_became_peered":"2023-10-06T05:22:33.054393+0000","last_unstale":"2023-10-06T05:22:33.063340+0000","last_undegraded":"2023-10-06T05:22:33.063340+0000","last_fullsized":"2023-10-06T05:22:33.063340+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:41.400666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062745+0000","last_change":"2023-10-06T05:22:33.057685+0000","last_active":"2023-10-06T05:22:33.062745+0000","last_peered":"2023-10-06T05:22:33.062745+0000","last_clean":"2023-10-06T05:22:33.062745+0000","last_became_active":"2023-10-06T05:22:33.057005+0000","last_became_peered":"2023-10-06T05:22:33.057005+0000","last_unstale":"2023-10-06T05:22:33.062745+0000","last_undegraded":"2023-10-06T05:22:33.062745+0000","last_fullsized":"2023-10-06T05:22:33.062745+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:09.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063336+0000","last_change":"2023-10-06T05:22:33.060400+0000","last_active":"2023-10-06T05:22:33.063336+0000","last_peered":"2023-10-06T05:22:33.063336+0000","last_clean":"2023-10-06T05:22:33.063336+0000","last_became_active":"2023-10-06T05:22:33.060142+0000","last_became_peered":"2023-10-06T05:22:33.060142+0000","last_unstale":"2023-10-06T05:22:33.063336+0000","last_undegraded":"2023-10-06T05:22:33.063336+0000","last_fullsized":"2023-10-06T05:22:33.063336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:12.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.058221+0000","last_change":"2023-10-06T05:22:33.054663+0000","last_active":"2023-10-06T05:22:33.058221+0000","last_peered":"2023-10-06T05:22:33.058221+0000","last_clean":"2023-10-06T05:22:33.058221+0000","last_became_active":"2023-10-06T05:22:33.054411+0000","last_became_peered":"2023-10-06T05:22:33.054411+0000","last_unstale":"2023-10-06T05:22:33.058221+0000","last_undegraded":"2023-10-06T05:22:33.058221+0000","last_fullsized":"2023-10-06T05:22:33.058221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:48.646638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062550+0000","last_change":"2023-10-06T05:22:33.056968+0000","last_active":"2023-10-06T05:22:33.062550+0000","last_peered":"2023-10-06T05:22:33.062550+0000","last_clean":"2023-10-06T05:22:33.062550+0000","last_became_active":"2023-10-06T05:22:33.056371+0000","last_became_peered":"2023-10-06T05:22:33.056371+0000","last_unstale":"2023-10-06T05:22:33.062550+0000","last_undegraded":"2023-10-06T05:22:33.062550+0000","last_fullsized":"2023-10-06T05:22:33.062550+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:54.838409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.056695+0000","last_change":"2023-10-06T05:22:33.053150+0000","last_active":"2023-10-06T05:22:33.056695+0000","last_peered":"2023-10-06T05:22:33.056695+0000","last_clean":"2023-10-06T05:22:33.056695+0000","last_became_active":"2023-10-06T05:22:33.050241+0000","last_became_peered":"2023-10-06T05:22:33.050241+0000","last_unstale":"2023-10-06T05:22:33.056695+0000","last_undegraded":"2023-10-06T05:22:33.056695+0000","last_fullsized":"2023-10-06T05:22:33.056695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:21.820269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068171+0000","last_change":"2023-10-06T05:22:33.064202+0000","last_active":"2023-10-06T05:22:33.068171+0000","last_peered":"2023-10-06T05:22:33.068171+0000","last_clean":"2023-10-06T05:22:33.068171+0000","last_became_active":"2023-10-06T05:22:33.063907+0000","last_became_peered":"2023-10-06T05:22:33.063907+0000","last_unstale":"2023-10-06T05:22:33.068171+0000","last_undegraded":"2023-10-06T05:22:33.068171+0000","last_fullsized":"2023-10-06T05:22:33.068171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:27.071503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.067334+0000","last_change":"2023-10-06T05:22:33.062952+0000","last_active":"2023-10-06T05:22:33.067334+0000","last_peered":"2023-10-06T05:22:33.067334+0000","last_clean":"2023-10-06T05:22:33.067334+0000","last_became_active":"2023-10-06T05:22:33.062656+0000","last_became_peered":"2023-10-06T05:22:33.062656+0000","last_unstale":"2023-10-06T05:22:33.067334+0000","last_undegraded":"2023-10-06T05:22:33.067334+0000","last_fullsized":"2023-10-06T05:22:33.067334+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:27.071503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.055073+0000","last_change":"2023-10-06T05:22:33.053452+0000","last_active":"2023-10-06T05:22:33.055073+0000","last_peered":"2023-10-06T05:22:33.055073+0000","last_clean":"2023-10-06T05:22:33.055073+0000","last_became_active":"2023-10-06T05:22:33.053201+0000","last_became_peered":"2023-10-06T05:22:33.053201+0000","last_unstale":"2023-10-06T05:22:33.055073+0000","last_undegraded":"2023-10-06T05:22:33.055073+0000","last_fullsized":"2023-10-06T05:22:33.055073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:53.779745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.070065+0000","last_change":"2023-10-06T05:22:33.060355+0000","last_active":"2023-10-06T05:22:33.070065+0000","last_peered":"2023-10-06T05:22:33.070065+0000","last_clean":"2023-10-06T05:22:33.070065+0000","last_became_active":"2023-10-06T05:22:33.060075+0000","last_became_peered":"2023-10-06T05:22:33.060075+0000","last_unstale":"2023-10-06T05:22:33.070065+0000","last_undegraded":"2023-10-06T05:22:33.070065+0000","last_fullsized":"2023-10-06T05:22:33.070065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:34.310804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059909+0000","last_change":"2023-10-06T05:22:33.051079+0000","last_active":"2023-10-06T05:22:33.059909+0000","last_peered":"2023-10-06T05:22:33.059909+0000","last_clean":"2023-10-06T05:22:33.059909+0000","last_became_active":"2023-10-06T05:22:33.050725+0000","last_became_peered":"2023-10-06T05:22:33.050725+0000","last_unstale":"2023-10-06T05:22:33.059909+0000","last_undegraded":"2023-10-06T05:22:33.059909+0000","last_fullsized":"2023-10-06T05:22:33.059909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:52.246335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.061496+0000","last_change":"2023-10-06T05:22:33.055509+0000","last_active":"2023-10-06T05:22:33.061496+0000","last_peered":"2023-10-06T05:22:33.061496+0000","last_clean":"2023-10-06T05:22:33.061496+0000","last_became_active":"2023-10-06T05:22:33.055264+0000","last_became_peered":"2023-10-06T05:22:33.055264+0000","last_unstale":"2023-10-06T05:22:33.061496+0000","last_undegraded":"2023-10-06T05:22:33.061496+0000","last_fullsized":"2023-10-06T05:22:33.061496+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:57.649299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.060751+0000","last_change":"2023-10-06T05:22:33.053802+0000","last_active":"2023-10-06T05:22:33.060751+0000","last_peered":"2023-10-06T05:22:33.060751+0000","last_clean":"2023-10-06T05:22:33.060751+0000","last_became_active":"2023-10-06T05:22:33.053548+0000","last_became_peered":"2023-10-06T05:22:33.053548+0000","last_unstale":"2023-10-06T05:22:33.060751+0000","last_undegraded":"2023-10-06T05:22:33.060751+0000","last_fullsized":"2023-10-06T05:22:33.060751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077597+0000","last_change":"2023-10-06T05:22:33.071425+0000","last_active":"2023-10-06T05:22:33.077597+0000","last_peered":"2023-10-06T05:22:33.077597+0000","last_clean":"2023-10-06T05:22:33.077597+0000","last_became_active":"2023-10-06T05:22:33.067947+0000","last_became_peered":"2023-10-06T05:22:33.067947+0000","last_unstale":"2023-10-06T05:22:33.077597+0000","last_undegraded":"2023-10-06T05:22:33.077597+0000","last_fullsized":"2023-10-06T05:22:33.077597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:04.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068326+0000","last_change":"2023-10-06T05:22:33.065607+0000","last_active":"2023-10-06T05:22:33.068326+0000","last_peered":"2023-10-06T05:22:33.068326+0000","last_clean":"2023-10-06T05:22:33.068326+0000","last_became_active":"2023-10-06T05:22:33.064624+0000","last_became_peered":"2023-10-06T05:22:33.064624+0000","last_unstale":"2023-10-06T05:22:33.068326+0000","last_undegraded":"2023-10-06T05:22:33.068326+0000","last_fullsized":"2023-10-06T05:22:33.068326+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:12.831489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076909+0000","last_change":"2023-10-06T05:22:33.070753+0000","last_active":"2023-10-06T05:22:33.076909+0000","last_peered":"2023-10-06T05:22:33.076909+0000","last_clean":"2023-10-06T05:22:33.076909+0000","last_became_active":"2023-10-06T05:22:33.067411+0000","last_became_peered":"2023-10-06T05:22:33.067411+0000","last_unstale":"2023-10-06T05:22:33.076909+0000","last_undegraded":"2023-10-06T05:22:33.076909+0000","last_fullsized":"2023-10-06T05:22:33.076909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:04.980573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064392+0000","last_change":"2023-10-06T05:22:33.057186+0000","last_active":"2023-10-06T05:22:33.064392+0000","last_peered":"2023-10-06T05:22:33.064392+0000","last_clean":"2023-10-06T05:22:33.064392+0000","last_became_active":"2023-10-06T05:22:33.056802+0000","last_became_peered":"2023-10-06T05:22:33.056802+0000","last_unstale":"2023-10-06T05:22:33.064392+0000","last_undegraded":"2023-10-06T05:22:33.064392+0000","last_fullsized":"2023-10-06T05:22:33.064392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:54.052349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063764+0000","last_change":"2023-10-06T05:22:33.060931+0000","last_active":"2023-10-06T05:22:33.063764+0000","last_peered":"2023-10-06T05:22:33.063764+0000","last_clean":"2023-10-06T05:22:33.063764+0000","last_became_active":"2023-10-06T05:22:33.060633+0000","last_became_peered":"2023-10-06T05:22:33.060633+0000","last_unstale":"2023-10-06T05:22:33.063764+0000","last_undegraded":"2023-10-06T05:22:33.063764+0000","last_fullsized":"2023-10-06T05:22:33.063764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:22.192166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064349+0000","last_change":"2023-10-06T05:22:33.061625+0000","last_active":"2023-10-06T05:22:33.064349+0000","last_peered":"2023-10-06T05:22:33.064349+0000","last_clean":"2023-10-06T05:22:33.064349+0000","last_became_active":"2023-10-06T05:22:33.061382+0000","last_became_peered":"2023-10-06T05:22:33.061382+0000","last_unstale":"2023-10-06T05:22:33.064349+0000","last_undegraded":"2023-10-06T05:22:33.064349+0000","last_fullsized":"2023-10-06T05:22:33.064349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:49.922418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077020+0000","last_change":"2023-10-06T05:22:33.071258+0000","last_active":"2023-10-06T05:22:33.077020+0000","last_peered":"2023-10-06T05:22:33.077020+0000","last_clean":"2023-10-06T05:22:33.077020+0000","last_became_active":"2023-10-06T05:22:33.070934+0000","last_became_peered":"2023-10-06T05:22:33.070934+0000","last_unstale":"2023-10-06T05:22:33.077020+0000","last_undegraded":"2023-10-06T05:22:33.077020+0000","last_fullsized":"2023-10-06T05:22:33.077020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:08.809909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078846+0000","last_change":"2023-10-06T05:22:33.073724+0000","last_active":"2023-10-06T05:22:33.078846+0000","last_peered":"2023-10-06T05:22:33.078846+0000","last_clean":"2023-10-06T05:22:33.078846+0000","last_became_active":"2023-10-06T05:22:33.073464+0000","last_became_peered":"2023-10-06T05:22:33.073464+0000","last_unstale":"2023-10-06T05:22:33.078846+0000","last_undegraded":"2023-10-06T05:22:33.078846+0000","last_fullsized":"2023-10-06T05:22:33.078846+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:47.725763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071620+0000","last_change":"2023-10-06T05:22:33.064542+0000","last_active":"2023-10-06T05:22:33.071620+0000","last_peered":"2023-10-06T05:22:33.071620+0000","last_clean":"2023-10-06T05:22:33.071620+0000","last_became_active":"2023-10-06T05:22:33.064136+0000","last_became_peered":"2023-10-06T05:22:33.064136+0000","last_unstale":"2023-10-06T05:22:33.071620+0000","last_undegraded":"2023-10-06T05:22:33.071620+0000","last_fullsized":"2023-10-06T05:22:33.071620+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:06.728920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064450+0000","last_change":"2023-10-06T05:22:33.056458+0000","last_active":"2023-10-06T05:22:33.064450+0000","last_peered":"2023-10-06T05:22:33.064450+0000","last_clean":"2023-10-06T05:22:33.064450+0000","last_became_active":"2023-10-06T05:22:33.056171+0000","last_became_peered":"2023-10-06T05:22:33.056171+0000","last_unstale":"2023-10-06T05:22:33.064450+0000","last_undegraded":"2023-10-06T05:22:33.064450+0000","last_fullsized":"2023-10-06T05:22:33.064450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:22.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059845+0000","last_change":"2023-10-06T05:22:33.055602+0000","last_active":"2023-10-06T05:22:33.059845+0000","last_peered":"2023-10-06T05:22:33.059845+0000","last_clean":"2023-10-06T05:22:33.059845+0000","last_became_active":"2023-10-06T05:22:33.054716+0000","last_became_peered":"2023-10-06T05:22:33.054716+0000","last_unstale":"2023-10-06T05:22:33.059845+0000","last_undegraded":"2023-10-06T05:22:33.059845+0000","last_fullsized":"2023-10-06T05:22:33.059845+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:03.942380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065045+0000","last_change":"2023-10-06T05:22:33.062809+0000","last_active":"2023-10-06T05:22:33.065045+0000","last_peered":"2023-10-06T05:22:33.065045+0000","last_clean":"2023-10-06T05:22:33.065045+0000","last_became_active":"2023-10-06T05:22:33.062546+0000","last_became_peered":"2023-10-06T05:22:33.062546+0000","last_unstale":"2023-10-06T05:22:33.065045+0000","last_undegraded":"2023-10-06T05:22:33.065045+0000","last_fullsized":"2023-10-06T05:22:33.065045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:49.962638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.052642+0000","last_change":"2023-10-06T05:22:33.050531+0000","last_active":"2023-10-06T05:22:33.052642+0000","last_peered":"2023-10-06T05:22:33.052642+0000","last_clean":"2023-10-06T05:22:33.052642+0000","last_became_active":"2023-10-06T05:22:33.050229+0000","last_became_peered":"2023-10-06T05:22:33.050229+0000","last_unstale":"2023-10-06T05:22:33.052642+0000","last_undegraded":"2023-10-06T05:22:33.052642+0000","last_fullsized":"2023-10-06T05:22:33.052642+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:40.575078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071086+0000","last_change":"2023-10-06T05:22:33.063546+0000","last_active":"2023-10-06T05:22:33.071086+0000","last_peered":"2023-10-06T05:22:33.071086+0000","last_clean":"2023-10-06T05:22:33.071086+0000","last_became_active":"2023-10-06T05:22:33.063267+0000","last_became_peered":"2023-10-06T05:22:33.063267+0000","last_unstale":"2023-10-06T05:22:33.071086+0000","last_undegraded":"2023-10-06T05:22:33.071086+0000","last_fullsized":"2023-10-06T05:22:33.071086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:14.467014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.075474+0000","last_change":"2023-10-06T05:22:33.073871+0000","last_active":"2023-10-06T05:22:33.075474+0000","last_peered":"2023-10-06T05:22:33.075474+0000","last_clean":"2023-10-06T05:22:33.075474+0000","last_became_active":"2023-10-06T05:22:33.073584+0000","last_became_peered":"2023-10-06T05:22:33.073584+0000","last_unstale":"2023-10-06T05:22:33.075474+0000","last_undegraded":"2023-10-06T05:22:33.075474+0000","last_fullsized":"2023-10-06T05:22:33.075474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:57.713424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062570+0000","last_change":"2023-10-06T05:22:33.057028+0000","last_active":"2023-10-06T05:22:33.062570+0000","last_peered":"2023-10-06T05:22:33.062570+0000","last_clean":"2023-10-06T05:22:33.062570+0000","last_became_active":"2023-10-06T05:22:33.056667+0000","last_became_peered":"2023-10-06T05:22:33.056667+0000","last_unstale":"2023-10-06T05:22:33.062570+0000","last_undegraded":"2023-10-06T05:22:33.062570+0000","last_fullsized":"2023-10-06T05:22:33.062570+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:00.470122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077074+0000","last_change":"2023-10-06T05:22:33.064036+0000","last_active":"2023-10-06T05:22:33.077074+0000","last_peered":"2023-10-06T05:22:33.077074+0000","last_clean":"2023-10-06T05:22:33.077074+0000","last_became_active":"2023-10-06T05:22:33.063620+0000","last_became_peered":"2023-10-06T05:22:33.063620+0000","last_unstale":"2023-10-06T05:22:33.077074+0000","last_undegraded":"2023-10-06T05:22:33.077074+0000","last_fullsized":"2023-10-06T05:22:33.077074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:38.128014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.070666+0000","last_change":"2023-10-06T05:22:32.053439+0000","last_active":"2023-10-06T05:22:33.070666+0000","last_peered":"2023-10-06T05:22:33.070666+0000","last_clean":"2023-10-06T05:22:33.070666+0000","last_became_active":"2023-10-06T05:22:32.053183+0000","last_became_peered":"2023-10-06T05:22:32.053183+0000","last_unstale":"2023-10-06T05:22:33.070666+0000","last_undegraded":"2023-10-06T05:22:33.070666+0000","last_fullsized":"2023-10-06T05:22:33.070666+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:23.173077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064771+0000","last_change":"2023-10-06T05:22:33.059313+0000","last_active":"2023-10-06T05:22:33.064771+0000","last_peered":"2023-10-06T05:22:33.064771+0000","last_clean":"2023-10-06T05:22:33.064771+0000","last_became_active":"2023-10-06T05:22:33.059065+0000","last_became_peered":"2023-10-06T05:22:33.059065+0000","last_unstale":"2023-10-06T05:22:33.064771+0000","last_undegraded":"2023-10-06T05:22:33.064771+0000","last_fullsized":"2023-10-06T05:22:33.064771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:29.790989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072412+0000","last_change":"2023-10-06T05:22:32.054436+0000","last_active":"2023-10-06T05:22:33.072412+0000","last_peered":"2023-10-06T05:22:33.072412+0000","last_clean":"2023-10-06T05:22:33.072412+0000","last_became_active":"2023-10-06T05:22:32.054147+0000","last_became_peered":"2023-10-06T05:22:32.054147+0000","last_unstale":"2023-10-06T05:22:33.072412+0000","last_undegraded":"2023-10-06T05:22:33.072412+0000","last_fullsized":"2023-10-06T05:22:33.072412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.372882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071641+0000","last_change":"2023-10-06T05:22:33.066515+0000","last_active":"2023-10-06T05:22:33.071641+0000","last_peered":"2023-10-06T05:22:33.071641+0000","last_clean":"2023-10-06T05:22:33.071641+0000","last_became_active":"2023-10-06T05:22:33.066223+0000","last_became_peered":"2023-10-06T05:22:33.066223+0000","last_unstale":"2023-10-06T05:22:33.071641+0000","last_undegraded":"2023-10-06T05:22:33.071641+0000","last_fullsized":"2023-10-06T05:22:33.071641+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:11.517044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071020+0000","last_change":"2023-10-06T05:22:32.062729+0000","last_active":"2023-10-06T05:22:33.071020+0000","last_peered":"2023-10-06T05:22:33.071020+0000","last_clean":"2023-10-06T05:22:33.071020+0000","last_became_active":"2023-10-06T05:22:32.062415+0000","last_became_peered":"2023-10-06T05:22:32.062415+0000","last_unstale":"2023-10-06T05:22:33.071020+0000","last_undegraded":"2023-10-06T05:22:33.071020+0000","last_fullsized":"2023-10-06T05:22:33.071020+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:54.464479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059421+0000","last_change":"2023-10-06T05:22:33.054407+0000","last_active":"2023-10-06T05:22:33.059421+0000","last_peered":"2023-10-06T05:22:33.059421+0000","last_clean":"2023-10-06T05:22:33.059421+0000","last_became_active":"2023-10-06T05:22:33.054115+0000","last_became_peered":"2023-10-06T05:22:33.054115+0000","last_unstale":"2023-10-06T05:22:33.059421+0000","last_undegraded":"2023-10-06T05:22:33.059421+0000","last_fullsized":"2023-10-06T05:22:33.059421+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:24.179806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076909+0000","last_change":"2023-10-06T05:22:32.054381+0000","last_active":"2023-10-06T05:22:33.076909+0000","last_peered":"2023-10-06T05:22:33.076909+0000","last_clean":"2023-10-06T05:22:33.076909+0000","last_became_active":"2023-10-06T05:22:32.053971+0000","last_became_peered":"2023-10-06T05:22:32.053971+0000","last_unstale":"2023-10-06T05:22:33.076909+0000","last_undegraded":"2023-10-06T05:22:33.076909+0000","last_fullsized":"2023-10-06T05:22:33.076909+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:41.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.069850+0000","last_change":"2023-10-06T05:22:33.059848+0000","last_active":"2023-10-06T05:22:33.069850+0000","last_peered":"2023-10-06T05:22:33.069850+0000","last_clean":"2023-10-06T05:22:33.069850+0000","last_became_active":"2023-10-06T05:22:33.059548+0000","last_became_peered":"2023-10-06T05:22:33.059548+0000","last_unstale":"2023-10-06T05:22:33.069850+0000","last_undegraded":"2023-10-06T05:22:33.069850+0000","last_fullsized":"2023-10-06T05:22:33.069850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:04.345313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.073162+0000","last_change":"2023-10-06T05:22:32.054712+0000","last_active":"2023-10-06T05:22:33.073162+0000","last_peered":"2023-10-06T05:22:33.073162+0000","last_clean":"2023-10-06T05:22:33.073162+0000","last_became_active":"2023-10-06T05:22:32.054389+0000","last_became_peered":"2023-10-06T05:22:32.054389+0000","last_unstale":"2023-10-06T05:22:33.073162+0000","last_undegraded":"2023-10-06T05:22:33.073162+0000","last_fullsized":"2023-10-06T05:22:33.073162+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:29.234061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076665+0000","last_change":"2023-10-06T05:22:33.062245+0000","last_active":"2023-10-06T05:22:33.076665+0000","last_peered":"2023-10-06T05:22:33.076665+0000","last_clean":"2023-10-06T05:22:33.076665+0000","last_became_active":"2023-10-06T05:22:33.061880+0000","last_became_peered":"2023-10-06T05:22:33.061880+0000","last_unstale":"2023-10-06T05:22:33.076665+0000","last_undegraded":"2023-10-06T05:22:33.076665+0000","last_fullsized":"2023-10-06T05:22:33.076665+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:05.582445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068878+0000","last_change":"2023-10-06T05:22:32.062015+0000","last_active":"2023-10-06T05:22:33.068878+0000","last_peered":"2023-10-06T05:22:33.068878+0000","last_clean":"2023-10-06T05:22:33.068878+0000","last_became_active":"2023-10-06T05:22:32.061659+0000","last_became_peered":"2023-10-06T05:22:32.061659+0000","last_unstale":"2023-10-06T05:22:33.068878+0000","last_undegraded":"2023-10-06T05:22:33.068878+0000","last_fullsized":"2023-10-06T05:22:33.068878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:27.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068348+0000","last_change":"2023-10-06T05:22:33.065794+0000","last_active":"2023-10-06T05:22:33.068348+0000","last_peered":"2023-10-06T05:22:33.068348+0000","last_clean":"2023-10-06T05:22:33.068348+0000","last_became_active":"2023-10-06T05:22:33.065328+0000","last_became_peered":"2023-10-06T05:22:33.065328+0000","last_unstale":"2023-10-06T05:22:33.068348+0000","last_undegraded":"2023-10-06T05:22:33.068348+0000","last_fullsized":"2023-10-06T05:22:33.068348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:10.923071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.047012+0000","last_change":"2023-10-06T05:22:32.057906+0000","last_active":"2023-10-06T05:22:33.047012+0000","last_peered":"2023-10-06T05:22:33.047012+0000","last_clean":"2023-10-06T05:22:33.047012+0000","last_became_active":"2023-10-06T05:22:32.057323+0000","last_became_peered":"2023-10-06T05:22:32.057323+0000","last_unstale":"2023-10-06T05:22:33.047012+0000","last_undegraded":"2023-10-06T05:22:33.047012+0000","last_fullsized":"2023-10-06T05:22:33.047012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:54.920418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076561+0000","last_change":"2023-10-06T05:22:33.074205+0000","last_active":"2023-10-06T05:22:33.076561+0000","last_peered":"2023-10-06T05:22:33.076561+0000","last_clean":"2023-10-06T05:22:33.076561+0000","last_became_active":"2023-10-06T05:22:33.073583+0000","last_became_peered":"2023-10-06T05:22:33.073583+0000","last_unstale":"2023-10-06T05:22:33.076561+0000","last_undegraded":"2023-10-06T05:22:33.076561+0000","last_fullsized":"2023-10-06T05:22:33.076561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:55.471207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.043562+0000","last_change":"2023-10-06T05:22:32.044400+0000","last_active":"2023-10-06T05:22:33.043562+0000","last_peered":"2023-10-06T05:22:33.043562+0000","last_clean":"2023-10-06T05:22:33.043562+0000","last_became_active":"2023-10-06T05:22:32.044113+0000","last_became_peered":"2023-10-06T05:22:32.044113+0000","last_unstale":"2023-10-06T05:22:33.043562+0000","last_undegraded":"2023-10-06T05:22:33.043562+0000","last_fullsized":"2023-10-06T05:22:33.043562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:01.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077091+0000","last_change":"2023-10-06T05:22:33.067562+0000","last_active":"2023-10-06T05:22:33.077091+0000","last_peered":"2023-10-06T05:22:33.077091+0000","last_clean":"2023-10-06T05:22:33.077091+0000","last_became_active":"2023-10-06T05:22:33.067224+0000","last_became_peered":"2023-10-06T05:22:33.067224+0000","last_unstale":"2023-10-06T05:22:33.077091+0000","last_undegraded":"2023-10-06T05:22:33.077091+0000","last_fullsized":"2023-10-06T05:22:33.077091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:45.268811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.048127+0000","last_change":"2023-10-06T05:22:32.056847+0000","last_active":"2023-10-06T05:22:33.048127+0000","last_peered":"2023-10-06T05:22:33.048127+0000","last_clean":"2023-10-06T05:22:33.048127+0000","last_became_active":"2023-10-06T05:22:32.056552+0000","last_became_peered":"2023-10-06T05:22:32.056552+0000","last_unstale":"2023-10-06T05:22:33.048127+0000","last_undegraded":"2023-10-06T05:22:33.048127+0000","last_fullsized":"2023-10-06T05:22:33.048127+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:42.984045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068280+0000","last_change":"2023-10-06T05:22:33.066298+0000","last_active":"2023-10-06T05:22:33.068280+0000","last_peered":"2023-10-06T05:22:33.068280+0000","last_clean":"2023-10-06T05:22:33.068280+0000","last_became_active":"2023-10-06T05:22:33.065366+0000","last_became_peered":"2023-10-06T05:22:33.065366+0000","last_unstale":"2023-10-06T05:22:33.068280+0000","last_undegraded":"2023-10-06T05:22:33.068280+0000","last_fullsized":"2023-10-06T05:22:33.068280+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:03.942380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.045045+0000","last_change":"2023-10-06T05:22:32.055847+0000","last_active":"2023-10-06T05:22:33.045045+0000","last_peered":"2023-10-06T05:22:33.045045+0000","last_clean":"2023-10-06T05:22:33.045045+0000","last_became_active":"2023-10-06T05:22:32.055526+0000","last_became_peered":"2023-10-06T05:22:32.055526+0000","last_unstale":"2023-10-06T05:22:33.045045+0000","last_undegraded":"2023-10-06T05:22:33.045045+0000","last_fullsized":"2023-10-06T05:22:33.045045+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:02.932701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065488+0000","last_change":"2023-10-06T05:22:33.060005+0000","last_active":"2023-10-06T05:22:33.065488+0000","last_peered":"2023-10-06T05:22:33.065488+0000","last_clean":"2023-10-06T05:22:33.065488+0000","last_became_active":"2023-10-06T05:22:33.059711+0000","last_became_peered":"2023-10-06T05:22:33.059711+0000","last_unstale":"2023-10-06T05:22:33.065488+0000","last_undegraded":"2023-10-06T05:22:33.065488+0000","last_fullsized":"2023-10-06T05:22:33.065488+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.884857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078002+0000","last_change":"2023-10-06T05:22:32.054607+0000","last_active":"2023-10-06T05:22:33.078002+0000","last_peered":"2023-10-06T05:22:33.078002+0000","last_clean":"2023-10-06T05:22:33.078002+0000","last_became_active":"2023-10-06T05:22:32.054320+0000","last_became_peered":"2023-10-06T05:22:32.054320+0000","last_unstale":"2023-10-06T05:22:33.078002+0000","last_undegraded":"2023-10-06T05:22:33.078002+0000","last_fullsized":"2023-10-06T05:22:33.078002+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:54.464479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068235+0000","last_change":"2023-10-06T05:22:33.064676+0000","last_active":"2023-10-06T05:22:33.068235+0000","last_peered":"2023-10-06T05:22:33.068235+0000","last_clean":"2023-10-06T05:22:33.068235+0000","last_became_active":"2023-10-06T05:22:33.064338+0000","last_became_peered":"2023-10-06T05:22:33.064338+0000","last_unstale":"2023-10-06T05:22:33.068235+0000","last_undegraded":"2023-10-06T05:22:33.068235+0000","last_fullsized":"2023-10-06T05:22:33.068235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.956570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.080125+0000","last_change":"2023-10-06T05:22:32.051465+0000","last_active":"2023-10-06T05:22:33.080125+0000","last_peered":"2023-10-06T05:22:33.080125+0000","last_clean":"2023-10-06T05:22:33.080125+0000","last_became_active":"2023-10-06T05:22:32.051099+0000","last_became_peered":"2023-10-06T05:22:32.051099+0000","last_unstale":"2023-10-06T05:22:33.080125+0000","last_undegraded":"2023-10-06T05:22:33.080125+0000","last_fullsized":"2023-10-06T05:22:33.080125+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:30.169712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.073248+0000","last_change":"2023-10-06T05:22:33.071230+0000","last_active":"2023-10-06T05:22:33.073248+0000","last_peered":"2023-10-06T05:22:33.073248+0000","last_clean":"2023-10-06T05:22:33.073248+0000","last_became_active":"2023-10-06T05:22:33.070965+0000","last_became_peered":"2023-10-06T05:22:33.070965+0000","last_unstale":"2023-10-06T05:22:33.073248+0000","last_undegraded":"2023-10-06T05:22:33.073248+0000","last_fullsized":"2023-10-06T05:22:33.073248+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.956570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.079345+0000","last_change":"2023-10-06T05:22:32.058056+0000","last_active":"2023-10-06T05:22:33.079345+0000","last_peered":"2023-10-06T05:22:33.079345+0000","last_clean":"2023-10-06T05:22:33.079345+0000","last_became_active":"2023-10-06T05:22:32.057684+0000","last_became_peered":"2023-10-06T05:22:32.057684+0000","last_unstale":"2023-10-06T05:22:33.079345+0000","last_undegraded":"2023-10-06T05:22:33.079345+0000","last_fullsized":"2023-10-06T05:22:33.079345+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:01.611932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063851+0000","last_change":"2023-10-06T05:22:33.058171+0000","last_active":"2023-10-06T05:22:33.063851+0000","last_peered":"2023-10-06T05:22:33.063851+0000","last_clean":"2023-10-06T05:22:33.063851+0000","last_became_active":"2023-10-06T05:22:33.057763+0000","last_became_peered":"2023-10-06T05:22:33.057763+0000","last_unstale":"2023-10-06T05:22:33.063851+0000","last_undegraded":"2023-10-06T05:22:33.063851+0000","last_fullsized":"2023-10-06T05:22:33.063851+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:11.672548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.043980+0000","last_change":"2023-10-06T05:22:32.051695+0000","last_active":"2023-10-06T05:22:33.043980+0000","last_peered":"2023-10-06T05:22:33.043980+0000","last_clean":"2023-10-06T05:22:33.043980+0000","last_became_active":"2023-10-06T05:22:32.051432+0000","last_became_peered":"2023-10-06T05:22:32.051432+0000","last_unstale":"2023-10-06T05:22:33.043980+0000","last_undegraded":"2023-10-06T05:22:33.043980+0000","last_fullsized":"2023-10-06T05:22:33.043980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:02.935652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063043+0000","last_change":"2023-10-06T05:22:33.056159+0000","last_active":"2023-10-06T05:22:33.063043+0000","last_peered":"2023-10-06T05:22:33.063043+0000","last_clean":"2023-10-06T05:22:33.063043+0000","last_became_active":"2023-10-06T05:22:33.055872+0000","last_became_peered":"2023-10-06T05:22:33.055872+0000","last_unstale":"2023-10-06T05:22:33.063043+0000","last_undegraded":"2023-10-06T05:22:33.063043+0000","last_fullsized":"2023-10-06T05:22:33.063043+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:38.181109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.039235+0000","last_change":"2023-10-06T05:22:32.059311+0000","last_active":"2023-10-06T05:22:33.039235+0000","last_peered":"2023-10-06T05:22:33.039235+0000","last_clean":"2023-10-06T05:22:33.039235+0000","last_became_active":"2023-10-06T05:22:32.059015+0000","last_became_peered":"2023-10-06T05:22:32.059015+0000","last_unstale":"2023-10-06T05:22:33.039235+0000","last_undegraded":"2023-10-06T05:22:33.039235+0000","last_fullsized":"2023-10-06T05:22:33.039235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:45.219115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077026+0000","last_change":"2023-10-06T05:22:33.071272+0000","last_active":"2023-10-06T05:22:33.077026+0000","last_peered":"2023-10-06T05:22:33.077026+0000","last_clean":"2023-10-06T05:22:33.077026+0000","last_became_active":"2023-10-06T05:22:33.067595+0000","last_became_peered":"2023-10-06T05:22:33.067595+0000","last_unstale":"2023-10-06T05:22:33.077026+0000","last_undegraded":"2023-10-06T05:22:33.077026+0000","last_fullsized":"2023-10-06T05:22:33.077026+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:55.283888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071807+0000","last_change":"2023-10-06T05:22:32.053722+0000","last_active":"2023-10-06T05:22:33.071807+0000","last_peered":"2023-10-06T05:22:33.071807+0000","last_clean":"2023-10-06T05:22:33.071807+0000","last_became_active":"2023-10-06T05:22:32.053444+0000","last_became_peered":"2023-10-06T05:22:32.053444+0000","last_unstale":"2023-10-06T05:22:33.071807+0000","last_undegraded":"2023-10-06T05:22:33.071807+0000","last_fullsized":"2023-10-06T05:22:33.071807+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:29.234061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077057+0000","last_change":"2023-10-06T05:22:33.063873+0000","last_active":"2023-10-06T05:22:33.077057+0000","last_peered":"2023-10-06T05:22:33.077057+0000","last_clean":"2023-10-06T05:22:33.077057+0000","last_became_active":"2023-10-06T05:22:33.063346+0000","last_became_peered":"2023-10-06T05:22:33.063346+0000","last_unstale":"2023-10-06T05:22:33.077057+0000","last_undegraded":"2023-10-06T05:22:33.077057+0000","last_fullsized":"2023-10-06T05:22:33.077057+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:58.347282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076815+0000","last_change":"2023-10-06T05:22:32.059484+0000","last_active":"2023-10-06T05:22:33.076815+0000","last_peered":"2023-10-06T05:22:33.076815+0000","last_clean":"2023-10-06T05:22:33.076815+0000","last_became_active":"2023-10-06T05:22:32.059228+0000","last_became_peered":"2023-10-06T05:22:32.059228+0000","last_unstale":"2023-10-06T05:22:33.076815+0000","last_undegraded":"2023-10-06T05:22:33.076815+0000","last_fullsized":"2023-10-06T05:22:33.076815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:27.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064424+0000","last_change":"2023-10-06T05:22:33.056125+0000","last_active":"2023-10-06T05:22:33.064424+0000","last_peered":"2023-10-06T05:22:33.064424+0000","last_clean":"2023-10-06T05:22:33.064424+0000","last_became_active":"2023-10-06T05:22:33.055849+0000","last_became_peered":"2023-10-06T05:22:33.055849+0000","last_unstale":"2023-10-06T05:22:33.064424+0000","last_undegraded":"2023-10-06T05:22:33.064424+0000","last_fullsized":"2023-10-06T05:22:33.064424+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:43.019236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T05:22:34.052903+0000","last_change":"2023-10-06T05:22:32.062155+0000","last_active":"2023-10-06T05:22:34.052903+0000","last_peered":"2023-10-06T05:22:34.052903+0000","last_clean":"2023-10-06T05:22:34.052903+0000","last_became_active":"2023-10-06T05:22:32.061860+0000","last_became_peered":"2023-10-06T05:22:32.061860+0000","last_unstale":"2023-10-06T05:22:34.052903+0000","last_undegraded":"2023-10-06T05:22:34.052903+0000","last_fullsized":"2023-10-06T05:22:34.052903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:54.479528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.069815+0000","last_change":"2023-10-06T05:22:33.059090+0000","last_active":"2023-10-06T05:22:33.069815+0000","last_peered":"2023-10-06T05:22:33.069815+0000","last_clean":"2023-10-06T05:22:33.069815+0000","last_became_active":"2023-10-06T05:22:33.058836+0000","last_became_peered":"2023-10-06T05:22:33.058836+0000","last_unstale":"2023-10-06T05:22:33.069815+0000","last_undegraded":"2023-10-06T05:22:33.069815+0000","last_fullsized":"2023-10-06T05:22:33.069815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:43.990773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.042006+0000","last_change":"2023-10-06T05:22:32.051860+0000","last_active":"2023-10-06T05:22:33.042006+0000","last_peered":"2023-10-06T05:22:33.042006+0000","last_clean":"2023-10-06T05:22:33.042006+0000","last_became_active":"2023-10-06T05:22:32.051570+0000","last_became_peered":"2023-10-06T05:22:32.051570+0000","last_unstale":"2023-10-06T05:22:33.042006+0000","last_undegraded":"2023-10-06T05:22:33.042006+0000","last_fullsized":"2023-10-06T05:22:33.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:23.584216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072285+0000","last_change":"2023-10-06T05:22:33.069678+0000","last_active":"2023-10-06T05:22:33.072285+0000","last_peered":"2023-10-06T05:22:33.072285+0000","last_clean":"2023-10-06T05:22:33.072285+0000","last_became_active":"2023-10-06T05:22:33.069419+0000","last_became_peered":"2023-10-06T05:22:33.069419+0000","last_unstale":"2023-10-06T05:22:33.072285+0000","last_undegraded":"2023-10-06T05:22:33.072285+0000","last_fullsized":"2023-10-06T05:22:33.072285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:47.725763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.053257+0000","last_change":"2023-10-06T05:22:32.054223+0000","last_active":"2023-10-06T05:22:33.053257+0000","last_peered":"2023-10-06T05:22:33.053257+0000","last_clean":"2023-10-06T05:22:33.053257+0000","last_became_active":"2023-10-06T05:22:32.053521+0000","last_became_peered":"2023-10-06T05:22:32.053521+0000","last_unstale":"2023-10-06T05:22:33.053257+0000","last_undegraded":"2023-10-06T05:22:33.053257+0000","last_fullsized":"2023-10-06T05:22:33.053257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:02.462579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063600+0000","last_change":"2023-10-06T05:22:33.060710+0000","last_active":"2023-10-06T05:22:33.063600+0000","last_peered":"2023-10-06T05:22:33.063600+0000","last_clean":"2023-10-06T05:22:33.063600+0000","last_became_active":"2023-10-06T05:22:33.060403+0000","last_became_peered":"2023-10-06T05:22:33.060403+0000","last_unstale":"2023-10-06T05:22:33.063600+0000","last_undegraded":"2023-10-06T05:22:33.063600+0000","last_fullsized":"2023-10-06T05:22:33.063600+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:24.590944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.045148+0000","last_change":"2023-10-06T05:22:32.056002+0000","last_active":"2023-10-06T05:22:33.045148+0000","last_peered":"2023-10-06T05:22:33.045148+0000","last_clean":"2023-10-06T05:22:33.045148+0000","last_became_active":"2023-10-06T05:22:32.055716+0000","last_became_peered":"2023-10-06T05:22:32.055716+0000","last_unstale":"2023-10-06T05:22:33.045148+0000","last_undegraded":"2023-10-06T05:22:33.045148+0000","last_fullsized":"2023-10-06T05:22:33.045148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:03.338584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064662+0000","last_change":"2023-10-06T05:22:33.058496+0000","last_active":"2023-10-06T05:22:33.064662+0000","last_peered":"2023-10-06T05:22:33.064662+0000","last_clean":"2023-10-06T05:22:33.064662+0000","last_became_active":"2023-10-06T05:22:33.058206+0000","last_became_peered":"2023-10-06T05:22:33.058206+0000","last_unstale":"2023-10-06T05:22:33.064662+0000","last_undegraded":"2023-10-06T05:22:33.064662+0000","last_fullsized":"2023-10-06T05:22:33.064662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:26.459752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040192+0000","last_change":"2023-10-06T05:22:32.059218+0000","last_active":"2023-10-06T05:22:33.040192+0000","last_peered":"2023-10-06T05:22:33.040192+0000","last_clean":"2023-10-06T05:22:33.040192+0000","last_became_active":"2023-10-06T05:22:32.058954+0000","last_became_peered":"2023-10-06T05:22:32.058954+0000","last_unstale":"2023-10-06T05:22:33.040192+0000","last_undegraded":"2023-10-06T05:22:33.040192+0000","last_fullsized":"2023-10-06T05:22:33.040192+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:46.497739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.067414+0000","last_change":"2023-10-06T05:22:33.063760+0000","last_active":"2023-10-06T05:22:33.067414+0000","last_peered":"2023-10-06T05:22:33.067414+0000","last_clean":"2023-10-06T05:22:33.067414+0000","last_became_active":"2023-10-06T05:22:33.063478+0000","last_became_peered":"2023-10-06T05:22:33.063478+0000","last_unstale":"2023-10-06T05:22:33.067414+0000","last_undegraded":"2023-10-06T05:22:33.067414+0000","last_fullsized":"2023-10-06T05:22:33.067414+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:02.999759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.041934+0000","last_change":"2023-10-06T05:22:32.052512+0000","last_active":"2023-10-06T05:22:33.041934+0000","last_peered":"2023-10-06T05:22:33.041934+0000","last_clean":"2023-10-06T05:22:33.041934+0000","last_became_active":"2023-10-06T05:22:32.052252+0000","last_became_peered":"2023-10-06T05:22:32.052252+0000","last_unstale":"2023-10-06T05:22:33.041934+0000","last_undegraded":"2023-10-06T05:22:33.041934+0000","last_fullsized":"2023-10-06T05:22:33.041934+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:28.784261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065508+0000","last_change":"2023-10-06T05:22:33.060430+0000","last_active":"2023-10-06T05:22:33.065508+0000","last_peered":"2023-10-06T05:22:33.065508+0000","last_clean":"2023-10-06T05:22:33.065508+0000","last_became_active":"2023-10-06T05:22:33.060098+0000","last_became_peered":"2023-10-06T05:22:33.060098+0000","last_unstale":"2023-10-06T05:22:33.065508+0000","last_undegraded":"2023-10-06T05:22:33.065508+0000","last_fullsized":"2023-10-06T05:22:33.065508+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:02.189479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040146+0000","last_change":"2023-10-06T05:22:32.058565+0000","last_active":"2023-10-06T05:22:33.040146+0000","last_peered":"2023-10-06T05:22:33.040146+0000","last_clean":"2023-10-06T05:22:33.040146+0000","last_became_active":"2023-10-06T05:22:32.058288+0000","last_became_peered":"2023-10-06T05:22:32.058288+0000","last_unstale":"2023-10-06T05:22:33.040146+0000","last_undegraded":"2023-10-06T05:22:33.040146+0000","last_fullsized":"2023-10-06T05:22:33.040146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:31.338180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063302+0000","last_change":"2023-10-06T05:22:33.058532+0000","last_active":"2023-10-06T05:22:33.063302+0000","last_peered":"2023-10-06T05:22:33.063302+0000","last_clean":"2023-10-06T05:22:33.063302+0000","last_became_active":"2023-10-06T05:22:33.058223+0000","last_became_peered":"2023-10-06T05:22:33.058223+0000","last_unstale":"2023-10-06T05:22:33.063302+0000","last_undegraded":"2023-10-06T05:22:33.063302+0000","last_fullsized":"2023-10-06T05:22:33.063302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:55.927147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.041783+0000","last_change":"2023-10-06T05:22:32.050247+0000","last_active":"2023-10-06T05:22:33.041783+0000","last_peered":"2023-10-06T05:22:33.041783+0000","last_clean":"2023-10-06T05:22:33.041783+0000","last_became_active":"2023-10-06T05:22:32.049943+0000","last_became_peered":"2023-10-06T05:22:32.049943+0000","last_unstale":"2023-10-06T05:22:33.041783+0000","last_undegraded":"2023-10-06T05:22:33.041783+0000","last_fullsized":"2023-10-06T05:22:33.041783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:25.453024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.053346+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:22:33.053346+0000","last_peered":"2023-10-06T05:22:33.053346+0000","last_clean":"2023-10-06T05:22:33.053346+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:22:33.053346+0000","last_undegraded":"2023-10-06T05:22:33.053346+0000","last_fullsized":"2023-10-06T05:22:33.053346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.075746+0000","last_change":"2023-10-06T05:22:33.073110+0000","last_active":"2023-10-06T05:22:33.075746+0000","last_peered":"2023-10-06T05:22:33.075746+0000","last_clean":"2023-10-06T05:22:33.075746+0000","last_became_active":"2023-10-06T05:22:33.072808+0000","last_became_peered":"2023-10-06T05:22:33.072808+0000","last_unstale":"2023-10-06T05:22:33.075746+0000","last_undegraded":"2023-10-06T05:22:33.075746+0000","last_fullsized":"2023-10-06T05:22:33.075746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:59.642608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072765+0000","last_change":"2023-10-06T05:22:32.057484+0000","last_active":"2023-10-06T05:22:33.072765+0000","last_peered":"2023-10-06T05:22:33.072765+0000","last_clean":"2023-10-06T05:22:33.072765+0000","last_became_active":"2023-10-06T05:22:32.056760+0000","last_became_peered":"2023-10-06T05:22:32.056760+0000","last_unstale":"2023-10-06T05:22:33.072765+0000","last_undegraded":"2023-10-06T05:22:33.072765+0000","last_fullsized":"2023-10-06T05:22:33.072765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.878129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.049516+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:22:33.049516+0000","last_peered":"2023-10-06T05:22:33.049516+0000","last_clean":"2023-10-06T05:22:33.049516+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:22:33.049516+0000","last_undegraded":"2023-10-06T05:22:33.049516+0000","last_fullsized":"2023-10-06T05:22:33.049516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068255+0000","last_change":"2023-10-06T05:22:33.065507+0000","last_active":"2023-10-06T05:22:33.068255+0000","last_peered":"2023-10-06T05:22:33.068255+0000","last_clean":"2023-10-06T05:22:33.068255+0000","last_became_active":"2023-10-06T05:22:33.065187+0000","last_became_peered":"2023-10-06T05:22:33.065187+0000","last_unstale":"2023-10-06T05:22:33.068255+0000","last_undegraded":"2023-10-06T05:22:33.068255+0000","last_fullsized":"2023-10-06T05:22:33.068255+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:02.189479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.049527+0000","last_change":"2023-10-06T05:22:32.061884+0000","last_active":"2023-10-06T05:22:33.049527+0000","last_peered":"2023-10-06T05:22:33.049527+0000","last_clean":"2023-10-06T05:22:33.049527+0000","last_became_active":"2023-10-06T05:22:32.061586+0000","last_became_peered":"2023-10-06T05:22:32.061586+0000","last_unstale":"2023-10-06T05:22:33.049527+0000","last_undegraded":"2023-10-06T05:22:33.049527+0000","last_fullsized":"2023-10-06T05:22:33.049527+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:41.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.039823+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:22:33.039823+0000","last_peered":"2023-10-06T05:22:33.039823+0000","last_clean":"2023-10-06T05:22:33.039823+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:22:33.039823+0000","last_undegraded":"2023-10-06T05:22:33.039823+0000","last_fullsized":"2023-10-06T05:22:33.039823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.074426+0000","last_change":"2023-10-06T05:22:33.073103+0000","last_active":"2023-10-06T05:22:33.074426+0000","last_peered":"2023-10-06T05:22:33.074426+0000","last_clean":"2023-10-06T05:22:33.074426+0000","last_became_active":"2023-10-06T05:22:33.072733+0000","last_became_peered":"2023-10-06T05:22:33.072733+0000","last_unstale":"2023-10-06T05:22:33.074426+0000","last_undegraded":"2023-10-06T05:22:33.074426+0000","last_fullsized":"2023-10-06T05:22:33.074426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:09.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.074507+0000","last_change":"2023-10-06T05:22:32.055034+0000","last_active":"2023-10-06T05:22:33.074507+0000","last_peered":"2023-10-06T05:22:33.074507+0000","last_clean":"2023-10-06T05:22:33.074507+0000","last_became_active":"2023-10-06T05:22:32.054712+0000","last_became_peered":"2023-10-06T05:22:32.054712+0000","last_unstale":"2023-10-06T05:22:33.074507+0000","last_undegraded":"2023-10-06T05:22:33.074507+0000","last_fullsized":"2023-10-06T05:22:33.074507+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:47.406781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.050186+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:22:33.050186+0000","last_peered":"2023-10-06T05:22:33.050186+0000","last_clean":"2023-10-06T05:22:33.050186+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:22:33.050186+0000","last_undegraded":"2023-10-06T05:22:33.050186+0000","last_fullsized":"2023-10-06T05:22:33.050186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044682+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:22:33.044682+0000","last_peered":"2023-10-06T05:22:33.044682+0000","last_clean":"2023-10-06T05:22:33.044682+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:22:33.044682+0000","last_undegraded":"2023-10-06T05:22:33.044682+0000","last_fullsized":"2023-10-06T05:22:33.044682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.066648+0000","last_change":"2023-10-06T05:22:33.063053+0000","last_active":"2023-10-06T05:22:33.066648+0000","last_peered":"2023-10-06T05:22:33.066648+0000","last_clean":"2023-10-06T05:22:33.066648+0000","last_became_active":"2023-10-06T05:22:33.062748+0000","last_became_peered":"2023-10-06T05:22:33.062748+0000","last_unstale":"2023-10-06T05:22:33.066648+0000","last_undegraded":"2023-10-06T05:22:33.066648+0000","last_fullsized":"2023-10-06T05:22:33.066648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:17.956567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071988+0000","last_change":"2023-10-06T05:22:32.055277+0000","last_active":"2023-10-06T05:22:33.071988+0000","last_peered":"2023-10-06T05:22:33.071988+0000","last_clean":"2023-10-06T05:22:33.071988+0000","last_became_active":"2023-10-06T05:22:32.055003+0000","last_became_peered":"2023-10-06T05:22:32.055003+0000","last_unstale":"2023-10-06T05:22:33.071988+0000","last_undegraded":"2023-10-06T05:22:33.071988+0000","last_fullsized":"2023-10-06T05:22:33.071988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:42.984045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044245+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:22:33.044245+0000","last_peered":"2023-10-06T05:22:33.044245+0000","last_clean":"2023-10-06T05:22:33.044245+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:22:33.044245+0000","last_undegraded":"2023-10-06T05:22:33.044245+0000","last_fullsized":"2023-10-06T05:22:33.044245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072363+0000","last_change":"2023-10-06T05:22:33.066198+0000","last_active":"2023-10-06T05:22:33.072363+0000","last_peered":"2023-10-06T05:22:33.072363+0000","last_clean":"2023-10-06T05:22:33.072363+0000","last_became_active":"2023-10-06T05:22:33.065914+0000","last_became_peered":"2023-10-06T05:22:33.065914+0000","last_unstale":"2023-10-06T05:22:33.072363+0000","last_undegraded":"2023-10-06T05:22:33.072363+0000","last_fullsized":"2023-10-06T05:22:33.072363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:06.769845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040040+0000","last_change":"2023-10-06T05:22:32.053709+0000","last_active":"2023-10-06T05:22:33.040040+0000","last_peered":"2023-10-06T05:22:33.040040+0000","last_clean":"2023-10-06T05:22:33.040040+0000","last_became_active":"2023-10-06T05:22:32.053115+0000","last_became_peered":"2023-10-06T05:22:32.053115+0000","last_unstale":"2023-10-06T05:22:33.040040+0000","last_undegraded":"2023-10-06T05:22:33.040040+0000","last_fullsized":"2023-10-06T05:22:33.040040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:24.907445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044708+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:22:33.044708+0000","last_peered":"2023-10-06T05:22:33.044708+0000","last_clean":"2023-10-06T05:22:33.044708+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:22:33.044708+0000","last_undegraded":"2023-10-06T05:22:33.044708+0000","last_fullsized":"2023-10-06T05:22:33.044708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077116+0000","last_change":"2023-10-06T05:22:33.067894+0000","last_active":"2023-10-06T05:22:33.077116+0000","last_peered":"2023-10-06T05:22:33.077116+0000","last_clean":"2023-10-06T05:22:33.077116+0000","last_became_active":"2023-10-06T05:22:33.067580+0000","last_became_peered":"2023-10-06T05:22:33.067580+0000","last_unstale":"2023-10-06T05:22:33.077116+0000","last_undegraded":"2023-10-06T05:22:33.077116+0000","last_fullsized":"2023-10-06T05:22:33.077116+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:31.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078053+0000","last_change":"2023-10-06T05:22:32.053701+0000","last_active":"2023-10-06T05:22:33.078053+0000","last_peered":"2023-10-06T05:22:33.078053+0000","last_clean":"2023-10-06T05:22:33.078053+0000","last_became_active":"2023-10-06T05:22:32.053435+0000","last_became_peered":"2023-10-06T05:22:32.053435+0000","last_unstale":"2023-10-06T05:22:33.078053+0000","last_undegraded":"2023-10-06T05:22:33.078053+0000","last_fullsized":"2023-10-06T05:22:33.078053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:27.622392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.052489+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:22:33.052489+0000","last_peered":"2023-10-06T05:22:33.052489+0000","last_clean":"2023-10-06T05:22:33.052489+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:22:33.052489+0000","last_undegraded":"2023-10-06T05:22:33.052489+0000","last_fullsized":"2023-10-06T05:22:33.052489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076755+0000","last_change":"2023-10-06T05:22:33.063092+0000","last_active":"2023-10-06T05:22:33.076755+0000","last_peered":"2023-10-06T05:22:33.076755+0000","last_clean":"2023-10-06T05:22:33.076755+0000","last_became_active":"2023-10-06T05:22:33.062195+0000","last_became_peered":"2023-10-06T05:22:33.062195+0000","last_unstale":"2023-10-06T05:22:33.076755+0000","last_undegraded":"2023-10-06T05:22:33.076755+0000","last_fullsized":"2023-10-06T05:22:33.076755+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:29.319953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040074+0000","last_change":"2023-10-06T05:22:32.051524+0000","last_active":"2023-10-06T05:22:33.040074+0000","last_peered":"2023-10-06T05:22:33.040074+0000","last_clean":"2023-10-06T05:22:33.040074+0000","last_became_active":"2023-10-06T05:22:32.050882+0000","last_became_peered":"2023-10-06T05:22:32.050882+0000","last_unstale":"2023-10-06T05:22:33.040074+0000","last_undegraded":"2023-10-06T05:22:33.040074+0000","last_fullsized":"2023-10-06T05:22:33.040074+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:08.923388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.042797+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:22:33.042797+0000","last_peered":"2023-10-06T05:22:33.042797+0000","last_clean":"2023-10-06T05:22:33.042797+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:22:33.042797+0000","last_undegraded":"2023-10-06T05:22:33.042797+0000","last_fullsized":"2023-10-06T05:22:33.042797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":1184708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.78100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302672,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069168,"statfs":{"total":96636764160,"available":96326828032,"internally_reserved":0,"allocated":1318912,"data_stored":596292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.621},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.377}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":0.746},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303248,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068592,"statfs":{"total":96636764160,"available":96326238208,"internally_reserved":0,"allocated":1908736,"data_stored":1186174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.501}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.629},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.753},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.78100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.627},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":0.61199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46999999999999997,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.505,"15min":0.5},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":1.772},"last":0.497}]},{"osd":1,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.913},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53900000000000003,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":1.954},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":0.96999999999999997,"5min":1.2030000000000001,"15min":1.282},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91500000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":0.98499999999999999,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.874},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.84199999999999997},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.83899999999999997},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0449999999999999,"5min":1.1910000000000001,"15min":1.343},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.84899999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.034,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":1.004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":0.998,"5min":1.2090000000000001,"15min":1.214},"last":0.95399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":594272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.755},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.84599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":594180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.63700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:42.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:42.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.576+0000 7f360979d700 1 Processor -- start 2023-10-06T05:22:42.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.576+0000 7f360979d700 1 -- start start 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f360979d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3604155af0 0x7f360406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f360979d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360406ab60 con 0x7f36040c7920 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f360979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360406ae40 con 0x7f360406a6a0 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f360979d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360406b7e0 con 0x7f3604155af0 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f3603fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3604155af0 0x7f360406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f3603fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3604155af0 0x7f360406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41164/0 (socket says 172.21.15.33:41164) 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f3603fff700 1 -- 172.21.15.33:0/3382004634 learned_addr learned my addr 172.21.15.33:0/3382004634 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:42.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.577+0000 7f3608f9c700 1 --1- 172.21.15.33:0/3382004634 >> v1:172.21.15.33:6789/0 conn(0x7f36040c7920 0x7f36040c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39322/0 (socket says 172.21.15.33:39322) 2023-10-06T05:22:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3603fff700 1 -- 172.21.15.33:0/3382004634 >> v1:172.21.15.167:6789/0 conn(0x7f360406a6a0 legacy=0x7f36040c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3603fff700 1 -- 172.21.15.33:0/3382004634 >> v1:172.21.15.33:6789/0 conn(0x7f36040c7920 legacy=0x7f36040c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3603fff700 1 -- 172.21.15.33:0/3382004634 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360406b120 con 0x7f3604155af0 2023-10-06T05:22:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3603fff700 1 --2- 172.21.15.33:0/3382004634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3604155af0 0x7f360406a160 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f35f40099b0 tx=0x7f35f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a63edc1e38daedc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3602ffd700 1 -- 172.21.15.33:0/3382004634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400b860 con 0x7f3604155af0 2023-10-06T05:22:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3602ffd700 1 -- 172.21.15.33:0/3382004634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400b9c0 con 0x7f3604155af0 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.578+0000 7f3602ffd700 1 -- 172.21.15.33:0/3382004634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400bd60 con 0x7f3604155af0 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 -- 172.21.15.33:0/3382004634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3604155af0 msgr2=0x7f360406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 --2- 172.21.15.33:0/3382004634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3604155af0 0x7f360406a160 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f35f40099b0 tx=0x7f35f4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 -- 172.21.15.33:0/3382004634 shutdown_connections 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 --2- 172.21.15.33:0/3382004634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3604155af0 0x7f360406a160 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 -- 172.21.15.33:0/3382004634 >> 172.21.15.33:0/3382004634 conn(0x7f360406d030 msgr2=0x7f360406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 -- 172.21.15.33:0/3382004634 shutdown_connections 2023-10-06T05:22:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.579+0000 7f360979d700 1 -- 172.21.15.33:0/3382004634 wait complete. 2023-10-06T05:22:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.580+0000 7f360979d700 1 Processor -- start 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.580+0000 7f360979d700 1 -- start start 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.581+0000 7f360979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360406a6a0 0x7f36040b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.581+0000 7f360979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 0x7f36040c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.581+0000 7f360979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3604155af0 0x7f36040cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.581+0000 7f360979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36040b02f0 con 0x7f36040c7920 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.581+0000 7f360979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604157330 con 0x7f360406a6a0 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.581+0000 7f360979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36040c3430 con 0x7f3604155af0 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 0x7f36040c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 0x7f36040c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41170/0 (socket says 172.21.15.33:41170) 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 -- 172.21.15.33:0/1775983760 learned_addr learned my addr 172.21.15.33:0/1775983760 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f3608f9c700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3604155af0 0x7f36040cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f3603fff700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360406a6a0 0x7f36040b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 -- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3604155af0 msgr2=0x7f36040cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3604155af0 0x7f36040cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 -- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360406a6a0 msgr2=0x7f36040b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360406a6a0 0x7f36040b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 -- 172.21.15.33:0/1775983760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f40094f0 con 0x7f36040c7920 2023-10-06T05:22:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f3603fff700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360406a6a0 0x7f36040b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f3608f9c700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3604155af0 0x7f36040cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.582+0000 7f36037fe700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 0x7f36040c49e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f35f800c7c0 tx=0x7f35f8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.583+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8011590 con 0x7f36040c7920 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.583+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f80116f0 con 0x7f36040c7920 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.583+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36040afd30 con 0x7f36040c7920 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.583+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8011ab0 con 0x7f36040c7920 2023-10-06T05:22:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.583+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3604156fc0 con 0x7f36040c7920 2023-10-06T05:22:42.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.584+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36041574e0 con 0x7f36040c7920 2023-10-06T05:22:42.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.585+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35f801b3b0 con 0x7f36040c7920 2023-10-06T05:22:42.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.586+0000 7f35ebfff700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f35f0066f30 0x7f35f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.586+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f35f8083c50 con 0x7f36040c7920 2023-10-06T05:22:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.586+0000 7f3603fff700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f35f0066f30 0x7f35f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.587+0000 7f3603fff700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f35f0066f30 0x7f35f00693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f35f40099b0 tx=0x7f35f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.590+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36041574e0 con 0x7f36040c7920 2023-10-06T05:22:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.788+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3604157150 con 0x7f35f0066f30 2023-10-06T05:22:42.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.798+0000 7f35ebfff700 1 -- 172.21.15.33:0/1775983760 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311318 (secure 0 0 0) 0x7f3604157150 con 0x7f35f0066f30 2023-10-06T05:22:42.804 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f35f0066f30 msgr2=0x7f35f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f35f0066f30 0x7f35f00693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f35f40099b0 tx=0x7f35f4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 msgr2=0x7f36040c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 0x7f36040c49e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f35f800c7c0 tx=0x7f35f8007f50 comp rx=0 tx=0).stop 2023-10-06T05:22:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 shutdown_connections 2023-10-06T05:22:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f35f0066f30 0x7f35f00693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3604155af0 0x7f36040cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360406a6a0 0x7f36040b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 --2- 172.21.15.33:0/1775983760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36040c7920 0x7f36040c49e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 >> 172.21.15.33:0/1775983760 conn(0x7f360406d030 msgr2=0x7f36040b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 shutdown_connections 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:42.802+0000 7f360979d700 1 -- 172.21.15.33:0/1775983760 wait complete. 2023-10-06T05:22:42.807 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:22:42.821 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-10-06T05:22:41.940152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422440,"kb_used_data":11368,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752552280,"statfs":{"total":773094113280,"available":770613534720,"internally_reserved":0,"allocated":11640832,"data_stored":5938788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.901942"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077617+0000","last_change":"2023-10-06T05:22:33.071449+0000","last_active":"2023-10-06T05:22:33.077617+0000","last_peered":"2023-10-06T05:22:33.077617+0000","last_clean":"2023-10-06T05:22:33.077617+0000","last_became_active":"2023-10-06T05:22:33.070959+0000","last_became_peered":"2023-10-06T05:22:33.070959+0000","last_unstale":"2023-10-06T05:22:33.077617+0000","last_undegraded":"2023-10-06T05:22:33.077617+0000","last_fullsized":"2023-10-06T05:22:33.077617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:53.229898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065900+0000","last_change":"2023-10-06T05:22:33.058445+0000","last_active":"2023-10-06T05:22:33.065900+0000","last_peered":"2023-10-06T05:22:33.065900+0000","last_clean":"2023-10-06T05:22:33.065900+0000","last_became_active":"2023-10-06T05:22:33.058172+0000","last_became_peered":"2023-10-06T05:22:33.058172+0000","last_unstale":"2023-10-06T05:22:33.065900+0000","last_undegraded":"2023-10-06T05:22:33.065900+0000","last_fullsized":"2023-10-06T05:22:33.065900+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:22.347118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076927+0000","last_change":"2023-10-06T05:22:33.070804+0000","last_active":"2023-10-06T05:22:33.076927+0000","last_peered":"2023-10-06T05:22:33.076927+0000","last_clean":"2023-10-06T05:22:33.076927+0000","last_became_active":"2023-10-06T05:22:33.067750+0000","last_became_peered":"2023-10-06T05:22:33.067750+0000","last_unstale":"2023-10-06T05:22:33.076927+0000","last_undegraded":"2023-10-06T05:22:33.076927+0000","last_fullsized":"2023-10-06T05:22:33.076927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:17.036525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063340+0000","last_change":"2023-10-06T05:22:33.054655+0000","last_active":"2023-10-06T05:22:33.063340+0000","last_peered":"2023-10-06T05:22:33.063340+0000","last_clean":"2023-10-06T05:22:33.063340+0000","last_became_active":"2023-10-06T05:22:33.054393+0000","last_became_peered":"2023-10-06T05:22:33.054393+0000","last_unstale":"2023-10-06T05:22:33.063340+0000","last_undegraded":"2023-10-06T05:22:33.063340+0000","last_fullsized":"2023-10-06T05:22:33.063340+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:41.400666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062745+0000","last_change":"2023-10-06T05:22:33.057685+0000","last_active":"2023-10-06T05:22:33.062745+0000","last_peered":"2023-10-06T05:22:33.062745+0000","last_clean":"2023-10-06T05:22:33.062745+0000","last_became_active":"2023-10-06T05:22:33.057005+0000","last_became_peered":"2023-10-06T05:22:33.057005+0000","last_unstale":"2023-10-06T05:22:33.062745+0000","last_undegraded":"2023-10-06T05:22:33.062745+0000","last_fullsized":"2023-10-06T05:22:33.062745+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:09.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063336+0000","last_change":"2023-10-06T05:22:33.060400+0000","last_active":"2023-10-06T05:22:33.063336+0000","last_peered":"2023-10-06T05:22:33.063336+0000","last_clean":"2023-10-06T05:22:33.063336+0000","last_became_active":"2023-10-06T05:22:33.060142+0000","last_became_peered":"2023-10-06T05:22:33.060142+0000","last_unstale":"2023-10-06T05:22:33.063336+0000","last_undegraded":"2023-10-06T05:22:33.063336+0000","last_fullsized":"2023-10-06T05:22:33.063336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:12.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.058221+0000","last_change":"2023-10-06T05:22:33.054663+0000","last_active":"2023-10-06T05:22:33.058221+0000","last_peered":"2023-10-06T05:22:33.058221+0000","last_clean":"2023-10-06T05:22:33.058221+0000","last_became_active":"2023-10-06T05:22:33.054411+0000","last_became_peered":"2023-10-06T05:22:33.054411+0000","last_unstale":"2023-10-06T05:22:33.058221+0000","last_undegraded":"2023-10-06T05:22:33.058221+0000","last_fullsized":"2023-10-06T05:22:33.058221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:48.646638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062550+0000","last_change":"2023-10-06T05:22:33.056968+0000","last_active":"2023-10-06T05:22:33.062550+0000","last_peered":"2023-10-06T05:22:33.062550+0000","last_clean":"2023-10-06T05:22:33.062550+0000","last_became_active":"2023-10-06T05:22:33.056371+0000","last_became_peered":"2023-10-06T05:22:33.056371+0000","last_unstale":"2023-10-06T05:22:33.062550+0000","last_undegraded":"2023-10-06T05:22:33.062550+0000","last_fullsized":"2023-10-06T05:22:33.062550+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:54.838409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.056695+0000","last_change":"2023-10-06T05:22:33.053150+0000","last_active":"2023-10-06T05:22:33.056695+0000","last_peered":"2023-10-06T05:22:33.056695+0000","last_clean":"2023-10-06T05:22:33.056695+0000","last_became_active":"2023-10-06T05:22:33.050241+0000","last_became_peered":"2023-10-06T05:22:33.050241+0000","last_unstale":"2023-10-06T05:22:33.056695+0000","last_undegraded":"2023-10-06T05:22:33.056695+0000","last_fullsized":"2023-10-06T05:22:33.056695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:21.820269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068171+0000","last_change":"2023-10-06T05:22:33.064202+0000","last_active":"2023-10-06T05:22:33.068171+0000","last_peered":"2023-10-06T05:22:33.068171+0000","last_clean":"2023-10-06T05:22:33.068171+0000","last_became_active":"2023-10-06T05:22:33.063907+0000","last_became_peered":"2023-10-06T05:22:33.063907+0000","last_unstale":"2023-10-06T05:22:33.068171+0000","last_undegraded":"2023-10-06T05:22:33.068171+0000","last_fullsized":"2023-10-06T05:22:33.068171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:27.071503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.067334+0000","last_change":"2023-10-06T05:22:33.062952+0000","last_active":"2023-10-06T05:22:33.067334+0000","last_peered":"2023-10-06T05:22:33.067334+0000","last_clean":"2023-10-06T05:22:33.067334+0000","last_became_active":"2023-10-06T05:22:33.062656+0000","last_became_peered":"2023-10-06T05:22:33.062656+0000","last_unstale":"2023-10-06T05:22:33.067334+0000","last_undegraded":"2023-10-06T05:22:33.067334+0000","last_fullsized":"2023-10-06T05:22:33.067334+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:27.071503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.055073+0000","last_change":"2023-10-06T05:22:33.053452+0000","last_active":"2023-10-06T05:22:33.055073+0000","last_peered":"2023-10-06T05:22:33.055073+0000","last_clean":"2023-10-06T05:22:33.055073+0000","last_became_active":"2023-10-06T05:22:33.053201+0000","last_became_peered":"2023-10-06T05:22:33.053201+0000","last_unstale":"2023-10-06T05:22:33.055073+0000","last_undegraded":"2023-10-06T05:22:33.055073+0000","last_fullsized":"2023-10-06T05:22:33.055073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:53.779745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.070065+0000","last_change":"2023-10-06T05:22:33.060355+0000","last_active":"2023-10-06T05:22:33.070065+0000","last_peered":"2023-10-06T05:22:33.070065+0000","last_clean":"2023-10-06T05:22:33.070065+0000","last_became_active":"2023-10-06T05:22:33.060075+0000","last_became_peered":"2023-10-06T05:22:33.060075+0000","last_unstale":"2023-10-06T05:22:33.070065+0000","last_undegraded":"2023-10-06T05:22:33.070065+0000","last_fullsized":"2023-10-06T05:22:33.070065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:34.310804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059909+0000","last_change":"2023-10-06T05:22:33.051079+0000","last_active":"2023-10-06T05:22:33.059909+0000","last_peered":"2023-10-06T05:22:33.059909+0000","last_clean":"2023-10-06T05:22:33.059909+0000","last_became_active":"2023-10-06T05:22:33.050725+0000","last_became_peered":"2023-10-06T05:22:33.050725+0000","last_unstale":"2023-10-06T05:22:33.059909+0000","last_undegraded":"2023-10-06T05:22:33.059909+0000","last_fullsized":"2023-10-06T05:22:33.059909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:52.246335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.061496+0000","last_change":"2023-10-06T05:22:33.055509+0000","last_active":"2023-10-06T05:22:33.061496+0000","last_peered":"2023-10-06T05:22:33.061496+0000","last_clean":"2023-10-06T05:22:33.061496+0000","last_became_active":"2023-10-06T05:22:33.055264+0000","last_became_peered":"2023-10-06T05:22:33.055264+0000","last_unstale":"2023-10-06T05:22:33.061496+0000","last_undegraded":"2023-10-06T05:22:33.061496+0000","last_fullsized":"2023-10-06T05:22:33.061496+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:57.649299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.060751+0000","last_change":"2023-10-06T05:22:33.053802+0000","last_active":"2023-10-06T05:22:33.060751+0000","last_peered":"2023-10-06T05:22:33.060751+0000","last_clean":"2023-10-06T05:22:33.060751+0000","last_became_active":"2023-10-06T05:22:33.053548+0000","last_became_peered":"2023-10-06T05:22:33.053548+0000","last_unstale":"2023-10-06T05:22:33.060751+0000","last_undegraded":"2023-10-06T05:22:33.060751+0000","last_fullsized":"2023-10-06T05:22:33.060751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:51.738850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077597+0000","last_change":"2023-10-06T05:22:33.071425+0000","last_active":"2023-10-06T05:22:33.077597+0000","last_peered":"2023-10-06T05:22:33.077597+0000","last_clean":"2023-10-06T05:22:33.077597+0000","last_became_active":"2023-10-06T05:22:33.067947+0000","last_became_peered":"2023-10-06T05:22:33.067947+0000","last_unstale":"2023-10-06T05:22:33.077597+0000","last_undegraded":"2023-10-06T05:22:33.077597+0000","last_fullsized":"2023-10-06T05:22:33.077597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:04.222046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068326+0000","last_change":"2023-10-06T05:22:33.065607+0000","last_active":"2023-10-06T05:22:33.068326+0000","last_peered":"2023-10-06T05:22:33.068326+0000","last_clean":"2023-10-06T05:22:33.068326+0000","last_became_active":"2023-10-06T05:22:33.064624+0000","last_became_peered":"2023-10-06T05:22:33.064624+0000","last_unstale":"2023-10-06T05:22:33.068326+0000","last_undegraded":"2023-10-06T05:22:33.068326+0000","last_fullsized":"2023-10-06T05:22:33.068326+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:12.831489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076909+0000","last_change":"2023-10-06T05:22:33.070753+0000","last_active":"2023-10-06T05:22:33.076909+0000","last_peered":"2023-10-06T05:22:33.076909+0000","last_clean":"2023-10-06T05:22:33.076909+0000","last_became_active":"2023-10-06T05:22:33.067411+0000","last_became_peered":"2023-10-06T05:22:33.067411+0000","last_unstale":"2023-10-06T05:22:33.076909+0000","last_undegraded":"2023-10-06T05:22:33.076909+0000","last_fullsized":"2023-10-06T05:22:33.076909+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:04.980573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064392+0000","last_change":"2023-10-06T05:22:33.057186+0000","last_active":"2023-10-06T05:22:33.064392+0000","last_peered":"2023-10-06T05:22:33.064392+0000","last_clean":"2023-10-06T05:22:33.064392+0000","last_became_active":"2023-10-06T05:22:33.056802+0000","last_became_peered":"2023-10-06T05:22:33.056802+0000","last_unstale":"2023-10-06T05:22:33.064392+0000","last_undegraded":"2023-10-06T05:22:33.064392+0000","last_fullsized":"2023-10-06T05:22:33.064392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:54.052349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063764+0000","last_change":"2023-10-06T05:22:33.060931+0000","last_active":"2023-10-06T05:22:33.063764+0000","last_peered":"2023-10-06T05:22:33.063764+0000","last_clean":"2023-10-06T05:22:33.063764+0000","last_became_active":"2023-10-06T05:22:33.060633+0000","last_became_peered":"2023-10-06T05:22:33.060633+0000","last_unstale":"2023-10-06T05:22:33.063764+0000","last_undegraded":"2023-10-06T05:22:33.063764+0000","last_fullsized":"2023-10-06T05:22:33.063764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:22.192166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064349+0000","last_change":"2023-10-06T05:22:33.061625+0000","last_active":"2023-10-06T05:22:33.064349+0000","last_peered":"2023-10-06T05:22:33.064349+0000","last_clean":"2023-10-06T05:22:33.064349+0000","last_became_active":"2023-10-06T05:22:33.061382+0000","last_became_peered":"2023-10-06T05:22:33.061382+0000","last_unstale":"2023-10-06T05:22:33.064349+0000","last_undegraded":"2023-10-06T05:22:33.064349+0000","last_fullsized":"2023-10-06T05:22:33.064349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:49.922418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077020+0000","last_change":"2023-10-06T05:22:33.071258+0000","last_active":"2023-10-06T05:22:33.077020+0000","last_peered":"2023-10-06T05:22:33.077020+0000","last_clean":"2023-10-06T05:22:33.077020+0000","last_became_active":"2023-10-06T05:22:33.070934+0000","last_became_peered":"2023-10-06T05:22:33.070934+0000","last_unstale":"2023-10-06T05:22:33.077020+0000","last_undegraded":"2023-10-06T05:22:33.077020+0000","last_fullsized":"2023-10-06T05:22:33.077020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:08.809909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078846+0000","last_change":"2023-10-06T05:22:33.073724+0000","last_active":"2023-10-06T05:22:33.078846+0000","last_peered":"2023-10-06T05:22:33.078846+0000","last_clean":"2023-10-06T05:22:33.078846+0000","last_became_active":"2023-10-06T05:22:33.073464+0000","last_became_peered":"2023-10-06T05:22:33.073464+0000","last_unstale":"2023-10-06T05:22:33.078846+0000","last_undegraded":"2023-10-06T05:22:33.078846+0000","last_fullsized":"2023-10-06T05:22:33.078846+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:47.725763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071620+0000","last_change":"2023-10-06T05:22:33.064542+0000","last_active":"2023-10-06T05:22:33.071620+0000","last_peered":"2023-10-06T05:22:33.071620+0000","last_clean":"2023-10-06T05:22:33.071620+0000","last_became_active":"2023-10-06T05:22:33.064136+0000","last_became_peered":"2023-10-06T05:22:33.064136+0000","last_unstale":"2023-10-06T05:22:33.071620+0000","last_undegraded":"2023-10-06T05:22:33.071620+0000","last_fullsized":"2023-10-06T05:22:33.071620+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:06.728920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064450+0000","last_change":"2023-10-06T05:22:33.056458+0000","last_active":"2023-10-06T05:22:33.064450+0000","last_peered":"2023-10-06T05:22:33.064450+0000","last_clean":"2023-10-06T05:22:33.064450+0000","last_became_active":"2023-10-06T05:22:33.056171+0000","last_became_peered":"2023-10-06T05:22:33.056171+0000","last_unstale":"2023-10-06T05:22:33.064450+0000","last_undegraded":"2023-10-06T05:22:33.064450+0000","last_fullsized":"2023-10-06T05:22:33.064450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:22.330423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059845+0000","last_change":"2023-10-06T05:22:33.055602+0000","last_active":"2023-10-06T05:22:33.059845+0000","last_peered":"2023-10-06T05:22:33.059845+0000","last_clean":"2023-10-06T05:22:33.059845+0000","last_became_active":"2023-10-06T05:22:33.054716+0000","last_became_peered":"2023-10-06T05:22:33.054716+0000","last_unstale":"2023-10-06T05:22:33.059845+0000","last_undegraded":"2023-10-06T05:22:33.059845+0000","last_fullsized":"2023-10-06T05:22:33.059845+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:03.942380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065045+0000","last_change":"2023-10-06T05:22:33.062809+0000","last_active":"2023-10-06T05:22:33.065045+0000","last_peered":"2023-10-06T05:22:33.065045+0000","last_clean":"2023-10-06T05:22:33.065045+0000","last_became_active":"2023-10-06T05:22:33.062546+0000","last_became_peered":"2023-10-06T05:22:33.062546+0000","last_unstale":"2023-10-06T05:22:33.065045+0000","last_undegraded":"2023-10-06T05:22:33.065045+0000","last_fullsized":"2023-10-06T05:22:33.065045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:49.962638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.052642+0000","last_change":"2023-10-06T05:22:33.050531+0000","last_active":"2023-10-06T05:22:33.052642+0000","last_peered":"2023-10-06T05:22:33.052642+0000","last_clean":"2023-10-06T05:22:33.052642+0000","last_became_active":"2023-10-06T05:22:33.050229+0000","last_became_peered":"2023-10-06T05:22:33.050229+0000","last_unstale":"2023-10-06T05:22:33.052642+0000","last_undegraded":"2023-10-06T05:22:33.052642+0000","last_fullsized":"2023-10-06T05:22:33.052642+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:40.575078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071086+0000","last_change":"2023-10-06T05:22:33.063546+0000","last_active":"2023-10-06T05:22:33.071086+0000","last_peered":"2023-10-06T05:22:33.071086+0000","last_clean":"2023-10-06T05:22:33.071086+0000","last_became_active":"2023-10-06T05:22:33.063267+0000","last_became_peered":"2023-10-06T05:22:33.063267+0000","last_unstale":"2023-10-06T05:22:33.071086+0000","last_undegraded":"2023-10-06T05:22:33.071086+0000","last_fullsized":"2023-10-06T05:22:33.071086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:14.467014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.075474+0000","last_change":"2023-10-06T05:22:33.073871+0000","last_active":"2023-10-06T05:22:33.075474+0000","last_peered":"2023-10-06T05:22:33.075474+0000","last_clean":"2023-10-06T05:22:33.075474+0000","last_became_active":"2023-10-06T05:22:33.073584+0000","last_became_peered":"2023-10-06T05:22:33.073584+0000","last_unstale":"2023-10-06T05:22:33.075474+0000","last_undegraded":"2023-10-06T05:22:33.075474+0000","last_fullsized":"2023-10-06T05:22:33.075474+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:57.713424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.062570+0000","last_change":"2023-10-06T05:22:33.057028+0000","last_active":"2023-10-06T05:22:33.062570+0000","last_peered":"2023-10-06T05:22:33.062570+0000","last_clean":"2023-10-06T05:22:33.062570+0000","last_became_active":"2023-10-06T05:22:33.056667+0000","last_became_peered":"2023-10-06T05:22:33.056667+0000","last_unstale":"2023-10-06T05:22:33.062570+0000","last_undegraded":"2023-10-06T05:22:33.062570+0000","last_fullsized":"2023-10-06T05:22:33.062570+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:00.470122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077074+0000","last_change":"2023-10-06T05:22:33.064036+0000","last_active":"2023-10-06T05:22:33.077074+0000","last_peered":"2023-10-06T05:22:33.077074+0000","last_clean":"2023-10-06T05:22:33.077074+0000","last_became_active":"2023-10-06T05:22:33.063620+0000","last_became_peered":"2023-10-06T05:22:33.063620+0000","last_unstale":"2023-10-06T05:22:33.077074+0000","last_undegraded":"2023-10-06T05:22:33.077074+0000","last_fullsized":"2023-10-06T05:22:33.077074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:38.128014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.070666+0000","last_change":"2023-10-06T05:22:32.053439+0000","last_active":"2023-10-06T05:22:33.070666+0000","last_peered":"2023-10-06T05:22:33.070666+0000","last_clean":"2023-10-06T05:22:33.070666+0000","last_became_active":"2023-10-06T05:22:32.053183+0000","last_became_peered":"2023-10-06T05:22:32.053183+0000","last_unstale":"2023-10-06T05:22:33.070666+0000","last_undegraded":"2023-10-06T05:22:33.070666+0000","last_fullsized":"2023-10-06T05:22:33.070666+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:23.173077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064771+0000","last_change":"2023-10-06T05:22:33.059313+0000","last_active":"2023-10-06T05:22:33.064771+0000","last_peered":"2023-10-06T05:22:33.064771+0000","last_clean":"2023-10-06T05:22:33.064771+0000","last_became_active":"2023-10-06T05:22:33.059065+0000","last_became_peered":"2023-10-06T05:22:33.059065+0000","last_unstale":"2023-10-06T05:22:33.064771+0000","last_undegraded":"2023-10-06T05:22:33.064771+0000","last_fullsized":"2023-10-06T05:22:33.064771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:29.790989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072412+0000","last_change":"2023-10-06T05:22:32.054436+0000","last_active":"2023-10-06T05:22:33.072412+0000","last_peered":"2023-10-06T05:22:33.072412+0000","last_clean":"2023-10-06T05:22:33.072412+0000","last_became_active":"2023-10-06T05:22:32.054147+0000","last_became_peered":"2023-10-06T05:22:32.054147+0000","last_unstale":"2023-10-06T05:22:33.072412+0000","last_undegraded":"2023-10-06T05:22:33.072412+0000","last_fullsized":"2023-10-06T05:22:33.072412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:10.372882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071641+0000","last_change":"2023-10-06T05:22:33.066515+0000","last_active":"2023-10-06T05:22:33.071641+0000","last_peered":"2023-10-06T05:22:33.071641+0000","last_clean":"2023-10-06T05:22:33.071641+0000","last_became_active":"2023-10-06T05:22:33.066223+0000","last_became_peered":"2023-10-06T05:22:33.066223+0000","last_unstale":"2023-10-06T05:22:33.071641+0000","last_undegraded":"2023-10-06T05:22:33.071641+0000","last_fullsized":"2023-10-06T05:22:33.071641+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:11.517044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071020+0000","last_change":"2023-10-06T05:22:32.062729+0000","last_active":"2023-10-06T05:22:33.071020+0000","last_peered":"2023-10-06T05:22:33.071020+0000","last_clean":"2023-10-06T05:22:33.071020+0000","last_became_active":"2023-10-06T05:22:32.062415+0000","last_became_peered":"2023-10-06T05:22:32.062415+0000","last_unstale":"2023-10-06T05:22:33.071020+0000","last_undegraded":"2023-10-06T05:22:33.071020+0000","last_fullsized":"2023-10-06T05:22:33.071020+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:54.464479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.059421+0000","last_change":"2023-10-06T05:22:33.054407+0000","last_active":"2023-10-06T05:22:33.059421+0000","last_peered":"2023-10-06T05:22:33.059421+0000","last_clean":"2023-10-06T05:22:33.059421+0000","last_became_active":"2023-10-06T05:22:33.054115+0000","last_became_peered":"2023-10-06T05:22:33.054115+0000","last_unstale":"2023-10-06T05:22:33.059421+0000","last_undegraded":"2023-10-06T05:22:33.059421+0000","last_fullsized":"2023-10-06T05:22:33.059421+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:24.179806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076909+0000","last_change":"2023-10-06T05:22:32.054381+0000","last_active":"2023-10-06T05:22:33.076909+0000","last_peered":"2023-10-06T05:22:33.076909+0000","last_clean":"2023-10-06T05:22:33.076909+0000","last_became_active":"2023-10-06T05:22:32.053971+0000","last_became_peered":"2023-10-06T05:22:32.053971+0000","last_unstale":"2023-10-06T05:22:33.076909+0000","last_undegraded":"2023-10-06T05:22:33.076909+0000","last_fullsized":"2023-10-06T05:22:33.076909+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:41.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.069850+0000","last_change":"2023-10-06T05:22:33.059848+0000","last_active":"2023-10-06T05:22:33.069850+0000","last_peered":"2023-10-06T05:22:33.069850+0000","last_clean":"2023-10-06T05:22:33.069850+0000","last_became_active":"2023-10-06T05:22:33.059548+0000","last_became_peered":"2023-10-06T05:22:33.059548+0000","last_unstale":"2023-10-06T05:22:33.069850+0000","last_undegraded":"2023-10-06T05:22:33.069850+0000","last_fullsized":"2023-10-06T05:22:33.069850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:04.345313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.073162+0000","last_change":"2023-10-06T05:22:32.054712+0000","last_active":"2023-10-06T05:22:33.073162+0000","last_peered":"2023-10-06T05:22:33.073162+0000","last_clean":"2023-10-06T05:22:33.073162+0000","last_became_active":"2023-10-06T05:22:32.054389+0000","last_became_peered":"2023-10-06T05:22:32.054389+0000","last_unstale":"2023-10-06T05:22:33.073162+0000","last_undegraded":"2023-10-06T05:22:33.073162+0000","last_fullsized":"2023-10-06T05:22:33.073162+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:29.234061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076665+0000","last_change":"2023-10-06T05:22:33.062245+0000","last_active":"2023-10-06T05:22:33.076665+0000","last_peered":"2023-10-06T05:22:33.076665+0000","last_clean":"2023-10-06T05:22:33.076665+0000","last_became_active":"2023-10-06T05:22:33.061880+0000","last_became_peered":"2023-10-06T05:22:33.061880+0000","last_unstale":"2023-10-06T05:22:33.076665+0000","last_undegraded":"2023-10-06T05:22:33.076665+0000","last_fullsized":"2023-10-06T05:22:33.076665+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:05.582445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068878+0000","last_change":"2023-10-06T05:22:32.062015+0000","last_active":"2023-10-06T05:22:33.068878+0000","last_peered":"2023-10-06T05:22:33.068878+0000","last_clean":"2023-10-06T05:22:33.068878+0000","last_became_active":"2023-10-06T05:22:32.061659+0000","last_became_peered":"2023-10-06T05:22:32.061659+0000","last_unstale":"2023-10-06T05:22:33.068878+0000","last_undegraded":"2023-10-06T05:22:33.068878+0000","last_fullsized":"2023-10-06T05:22:33.068878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:27.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068348+0000","last_change":"2023-10-06T05:22:33.065794+0000","last_active":"2023-10-06T05:22:33.068348+0000","last_peered":"2023-10-06T05:22:33.068348+0000","last_clean":"2023-10-06T05:22:33.068348+0000","last_became_active":"2023-10-06T05:22:33.065328+0000","last_became_peered":"2023-10-06T05:22:33.065328+0000","last_unstale":"2023-10-06T05:22:33.068348+0000","last_undegraded":"2023-10-06T05:22:33.068348+0000","last_fullsized":"2023-10-06T05:22:33.068348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:10.923071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.047012+0000","last_change":"2023-10-06T05:22:32.057906+0000","last_active":"2023-10-06T05:22:33.047012+0000","last_peered":"2023-10-06T05:22:33.047012+0000","last_clean":"2023-10-06T05:22:33.047012+0000","last_became_active":"2023-10-06T05:22:32.057323+0000","last_became_peered":"2023-10-06T05:22:32.057323+0000","last_unstale":"2023-10-06T05:22:33.047012+0000","last_undegraded":"2023-10-06T05:22:33.047012+0000","last_fullsized":"2023-10-06T05:22:33.047012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:54.920418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076561+0000","last_change":"2023-10-06T05:22:33.074205+0000","last_active":"2023-10-06T05:22:33.076561+0000","last_peered":"2023-10-06T05:22:33.076561+0000","last_clean":"2023-10-06T05:22:33.076561+0000","last_became_active":"2023-10-06T05:22:33.073583+0000","last_became_peered":"2023-10-06T05:22:33.073583+0000","last_unstale":"2023-10-06T05:22:33.076561+0000","last_undegraded":"2023-10-06T05:22:33.076561+0000","last_fullsized":"2023-10-06T05:22:33.076561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:55.471207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.043562+0000","last_change":"2023-10-06T05:22:32.044400+0000","last_active":"2023-10-06T05:22:33.043562+0000","last_peered":"2023-10-06T05:22:33.043562+0000","last_clean":"2023-10-06T05:22:33.043562+0000","last_became_active":"2023-10-06T05:22:32.044113+0000","last_became_peered":"2023-10-06T05:22:32.044113+0000","last_unstale":"2023-10-06T05:22:33.043562+0000","last_undegraded":"2023-10-06T05:22:33.043562+0000","last_fullsized":"2023-10-06T05:22:33.043562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:01.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077091+0000","last_change":"2023-10-06T05:22:33.067562+0000","last_active":"2023-10-06T05:22:33.077091+0000","last_peered":"2023-10-06T05:22:33.077091+0000","last_clean":"2023-10-06T05:22:33.077091+0000","last_became_active":"2023-10-06T05:22:33.067224+0000","last_became_peered":"2023-10-06T05:22:33.067224+0000","last_unstale":"2023-10-06T05:22:33.077091+0000","last_undegraded":"2023-10-06T05:22:33.077091+0000","last_fullsized":"2023-10-06T05:22:33.077091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:45.268811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.048127+0000","last_change":"2023-10-06T05:22:32.056847+0000","last_active":"2023-10-06T05:22:33.048127+0000","last_peered":"2023-10-06T05:22:33.048127+0000","last_clean":"2023-10-06T05:22:33.048127+0000","last_became_active":"2023-10-06T05:22:32.056552+0000","last_became_peered":"2023-10-06T05:22:32.056552+0000","last_unstale":"2023-10-06T05:22:33.048127+0000","last_undegraded":"2023-10-06T05:22:33.048127+0000","last_fullsized":"2023-10-06T05:22:33.048127+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:42.984045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068280+0000","last_change":"2023-10-06T05:22:33.066298+0000","last_active":"2023-10-06T05:22:33.068280+0000","last_peered":"2023-10-06T05:22:33.068280+0000","last_clean":"2023-10-06T05:22:33.068280+0000","last_became_active":"2023-10-06T05:22:33.065366+0000","last_became_peered":"2023-10-06T05:22:33.065366+0000","last_unstale":"2023-10-06T05:22:33.068280+0000","last_undegraded":"2023-10-06T05:22:33.068280+0000","last_fullsized":"2023-10-06T05:22:33.068280+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:03.942380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.045045+0000","last_change":"2023-10-06T05:22:32.055847+0000","last_active":"2023-10-06T05:22:33.045045+0000","last_peered":"2023-10-06T05:22:33.045045+0000","last_clean":"2023-10-06T05:22:33.045045+0000","last_became_active":"2023-10-06T05:22:32.055526+0000","last_became_peered":"2023-10-06T05:22:32.055526+0000","last_unstale":"2023-10-06T05:22:33.045045+0000","last_undegraded":"2023-10-06T05:22:33.045045+0000","last_fullsized":"2023-10-06T05:22:33.045045+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:02.932701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065488+0000","last_change":"2023-10-06T05:22:33.060005+0000","last_active":"2023-10-06T05:22:33.065488+0000","last_peered":"2023-10-06T05:22:33.065488+0000","last_clean":"2023-10-06T05:22:33.065488+0000","last_became_active":"2023-10-06T05:22:33.059711+0000","last_became_peered":"2023-10-06T05:22:33.059711+0000","last_unstale":"2023-10-06T05:22:33.065488+0000","last_undegraded":"2023-10-06T05:22:33.065488+0000","last_fullsized":"2023-10-06T05:22:33.065488+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:32.884857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078002+0000","last_change":"2023-10-06T05:22:32.054607+0000","last_active":"2023-10-06T05:22:33.078002+0000","last_peered":"2023-10-06T05:22:33.078002+0000","last_clean":"2023-10-06T05:22:33.078002+0000","last_became_active":"2023-10-06T05:22:32.054320+0000","last_became_peered":"2023-10-06T05:22:32.054320+0000","last_unstale":"2023-10-06T05:22:33.078002+0000","last_undegraded":"2023-10-06T05:22:33.078002+0000","last_fullsized":"2023-10-06T05:22:33.078002+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:54.464479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068235+0000","last_change":"2023-10-06T05:22:33.064676+0000","last_active":"2023-10-06T05:22:33.068235+0000","last_peered":"2023-10-06T05:22:33.068235+0000","last_clean":"2023-10-06T05:22:33.068235+0000","last_became_active":"2023-10-06T05:22:33.064338+0000","last_became_peered":"2023-10-06T05:22:33.064338+0000","last_unstale":"2023-10-06T05:22:33.068235+0000","last_undegraded":"2023-10-06T05:22:33.068235+0000","last_fullsized":"2023-10-06T05:22:33.068235+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.956570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.080125+0000","last_change":"2023-10-06T05:22:32.051465+0000","last_active":"2023-10-06T05:22:33.080125+0000","last_peered":"2023-10-06T05:22:33.080125+0000","last_clean":"2023-10-06T05:22:33.080125+0000","last_became_active":"2023-10-06T05:22:32.051099+0000","last_became_peered":"2023-10-06T05:22:32.051099+0000","last_unstale":"2023-10-06T05:22:33.080125+0000","last_undegraded":"2023-10-06T05:22:33.080125+0000","last_fullsized":"2023-10-06T05:22:33.080125+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:30.169712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.073248+0000","last_change":"2023-10-06T05:22:33.071230+0000","last_active":"2023-10-06T05:22:33.073248+0000","last_peered":"2023-10-06T05:22:33.073248+0000","last_clean":"2023-10-06T05:22:33.073248+0000","last_became_active":"2023-10-06T05:22:33.070965+0000","last_became_peered":"2023-10-06T05:22:33.070965+0000","last_unstale":"2023-10-06T05:22:33.073248+0000","last_undegraded":"2023-10-06T05:22:33.073248+0000","last_fullsized":"2023-10-06T05:22:33.073248+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.956570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.079345+0000","last_change":"2023-10-06T05:22:32.058056+0000","last_active":"2023-10-06T05:22:33.079345+0000","last_peered":"2023-10-06T05:22:33.079345+0000","last_clean":"2023-10-06T05:22:33.079345+0000","last_became_active":"2023-10-06T05:22:32.057684+0000","last_became_peered":"2023-10-06T05:22:32.057684+0000","last_unstale":"2023-10-06T05:22:33.079345+0000","last_undegraded":"2023-10-06T05:22:33.079345+0000","last_fullsized":"2023-10-06T05:22:33.079345+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:01.611932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063851+0000","last_change":"2023-10-06T05:22:33.058171+0000","last_active":"2023-10-06T05:22:33.063851+0000","last_peered":"2023-10-06T05:22:33.063851+0000","last_clean":"2023-10-06T05:22:33.063851+0000","last_became_active":"2023-10-06T05:22:33.057763+0000","last_became_peered":"2023-10-06T05:22:33.057763+0000","last_unstale":"2023-10-06T05:22:33.063851+0000","last_undegraded":"2023-10-06T05:22:33.063851+0000","last_fullsized":"2023-10-06T05:22:33.063851+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:11.672548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.043980+0000","last_change":"2023-10-06T05:22:32.051695+0000","last_active":"2023-10-06T05:22:33.043980+0000","last_peered":"2023-10-06T05:22:33.043980+0000","last_clean":"2023-10-06T05:22:33.043980+0000","last_became_active":"2023-10-06T05:22:32.051432+0000","last_became_peered":"2023-10-06T05:22:32.051432+0000","last_unstale":"2023-10-06T05:22:33.043980+0000","last_undegraded":"2023-10-06T05:22:33.043980+0000","last_fullsized":"2023-10-06T05:22:33.043980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:02.935652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063043+0000","last_change":"2023-10-06T05:22:33.056159+0000","last_active":"2023-10-06T05:22:33.063043+0000","last_peered":"2023-10-06T05:22:33.063043+0000","last_clean":"2023-10-06T05:22:33.063043+0000","last_became_active":"2023-10-06T05:22:33.055872+0000","last_became_peered":"2023-10-06T05:22:33.055872+0000","last_unstale":"2023-10-06T05:22:33.063043+0000","last_undegraded":"2023-10-06T05:22:33.063043+0000","last_fullsized":"2023-10-06T05:22:33.063043+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:38.181109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.039235+0000","last_change":"2023-10-06T05:22:32.059311+0000","last_active":"2023-10-06T05:22:33.039235+0000","last_peered":"2023-10-06T05:22:33.039235+0000","last_clean":"2023-10-06T05:22:33.039235+0000","last_became_active":"2023-10-06T05:22:32.059015+0000","last_became_peered":"2023-10-06T05:22:32.059015+0000","last_unstale":"2023-10-06T05:22:33.039235+0000","last_undegraded":"2023-10-06T05:22:33.039235+0000","last_fullsized":"2023-10-06T05:22:33.039235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:45.219115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077026+0000","last_change":"2023-10-06T05:22:33.071272+0000","last_active":"2023-10-06T05:22:33.077026+0000","last_peered":"2023-10-06T05:22:33.077026+0000","last_clean":"2023-10-06T05:22:33.077026+0000","last_became_active":"2023-10-06T05:22:33.067595+0000","last_became_peered":"2023-10-06T05:22:33.067595+0000","last_unstale":"2023-10-06T05:22:33.077026+0000","last_undegraded":"2023-10-06T05:22:33.077026+0000","last_fullsized":"2023-10-06T05:22:33.077026+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:55.283888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071807+0000","last_change":"2023-10-06T05:22:32.053722+0000","last_active":"2023-10-06T05:22:33.071807+0000","last_peered":"2023-10-06T05:22:33.071807+0000","last_clean":"2023-10-06T05:22:33.071807+0000","last_became_active":"2023-10-06T05:22:32.053444+0000","last_became_peered":"2023-10-06T05:22:32.053444+0000","last_unstale":"2023-10-06T05:22:33.071807+0000","last_undegraded":"2023-10-06T05:22:33.071807+0000","last_fullsized":"2023-10-06T05:22:33.071807+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:29.234061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077057+0000","last_change":"2023-10-06T05:22:33.063873+0000","last_active":"2023-10-06T05:22:33.077057+0000","last_peered":"2023-10-06T05:22:33.077057+0000","last_clean":"2023-10-06T05:22:33.077057+0000","last_became_active":"2023-10-06T05:22:33.063346+0000","last_became_peered":"2023-10-06T05:22:33.063346+0000","last_unstale":"2023-10-06T05:22:33.077057+0000","last_undegraded":"2023-10-06T05:22:33.077057+0000","last_fullsized":"2023-10-06T05:22:33.077057+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:58.347282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076815+0000","last_change":"2023-10-06T05:22:32.059484+0000","last_active":"2023-10-06T05:22:33.076815+0000","last_peered":"2023-10-06T05:22:33.076815+0000","last_clean":"2023-10-06T05:22:33.076815+0000","last_became_active":"2023-10-06T05:22:32.059228+0000","last_became_peered":"2023-10-06T05:22:32.059228+0000","last_unstale":"2023-10-06T05:22:33.076815+0000","last_undegraded":"2023-10-06T05:22:33.076815+0000","last_fullsized":"2023-10-06T05:22:33.076815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:27.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064424+0000","last_change":"2023-10-06T05:22:33.056125+0000","last_active":"2023-10-06T05:22:33.064424+0000","last_peered":"2023-10-06T05:22:33.064424+0000","last_clean":"2023-10-06T05:22:33.064424+0000","last_became_active":"2023-10-06T05:22:33.055849+0000","last_became_peered":"2023-10-06T05:22:33.055849+0000","last_unstale":"2023-10-06T05:22:33.064424+0000","last_undegraded":"2023-10-06T05:22:33.064424+0000","last_fullsized":"2023-10-06T05:22:33.064424+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:43.019236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T05:22:34.052903+0000","last_change":"2023-10-06T05:22:32.062155+0000","last_active":"2023-10-06T05:22:34.052903+0000","last_peered":"2023-10-06T05:22:34.052903+0000","last_clean":"2023-10-06T05:22:34.052903+0000","last_became_active":"2023-10-06T05:22:32.061860+0000","last_became_peered":"2023-10-06T05:22:32.061860+0000","last_unstale":"2023-10-06T05:22:34.052903+0000","last_undegraded":"2023-10-06T05:22:34.052903+0000","last_fullsized":"2023-10-06T05:22:34.052903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:54.479528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.069815+0000","last_change":"2023-10-06T05:22:33.059090+0000","last_active":"2023-10-06T05:22:33.069815+0000","last_peered":"2023-10-06T05:22:33.069815+0000","last_clean":"2023-10-06T05:22:33.069815+0000","last_became_active":"2023-10-06T05:22:33.058836+0000","last_became_peered":"2023-10-06T05:22:33.058836+0000","last_unstale":"2023-10-06T05:22:33.069815+0000","last_undegraded":"2023-10-06T05:22:33.069815+0000","last_fullsized":"2023-10-06T05:22:33.069815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:43.990773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.042006+0000","last_change":"2023-10-06T05:22:32.051860+0000","last_active":"2023-10-06T05:22:33.042006+0000","last_peered":"2023-10-06T05:22:33.042006+0000","last_clean":"2023-10-06T05:22:33.042006+0000","last_became_active":"2023-10-06T05:22:32.051570+0000","last_became_peered":"2023-10-06T05:22:32.051570+0000","last_unstale":"2023-10-06T05:22:33.042006+0000","last_undegraded":"2023-10-06T05:22:33.042006+0000","last_fullsized":"2023-10-06T05:22:33.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:23.584216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072285+0000","last_change":"2023-10-06T05:22:33.069678+0000","last_active":"2023-10-06T05:22:33.072285+0000","last_peered":"2023-10-06T05:22:33.072285+0000","last_clean":"2023-10-06T05:22:33.072285+0000","last_became_active":"2023-10-06T05:22:33.069419+0000","last_became_peered":"2023-10-06T05:22:33.069419+0000","last_unstale":"2023-10-06T05:22:33.072285+0000","last_undegraded":"2023-10-06T05:22:33.072285+0000","last_fullsized":"2023-10-06T05:22:33.072285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:47.725763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.053257+0000","last_change":"2023-10-06T05:22:32.054223+0000","last_active":"2023-10-06T05:22:33.053257+0000","last_peered":"2023-10-06T05:22:33.053257+0000","last_clean":"2023-10-06T05:22:33.053257+0000","last_became_active":"2023-10-06T05:22:32.053521+0000","last_became_peered":"2023-10-06T05:22:32.053521+0000","last_unstale":"2023-10-06T05:22:33.053257+0000","last_undegraded":"2023-10-06T05:22:33.053257+0000","last_fullsized":"2023-10-06T05:22:33.053257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:02.462579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063600+0000","last_change":"2023-10-06T05:22:33.060710+0000","last_active":"2023-10-06T05:22:33.063600+0000","last_peered":"2023-10-06T05:22:33.063600+0000","last_clean":"2023-10-06T05:22:33.063600+0000","last_became_active":"2023-10-06T05:22:33.060403+0000","last_became_peered":"2023-10-06T05:22:33.060403+0000","last_unstale":"2023-10-06T05:22:33.063600+0000","last_undegraded":"2023-10-06T05:22:33.063600+0000","last_fullsized":"2023-10-06T05:22:33.063600+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:24.590944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.045148+0000","last_change":"2023-10-06T05:22:32.056002+0000","last_active":"2023-10-06T05:22:33.045148+0000","last_peered":"2023-10-06T05:22:33.045148+0000","last_clean":"2023-10-06T05:22:33.045148+0000","last_became_active":"2023-10-06T05:22:32.055716+0000","last_became_peered":"2023-10-06T05:22:32.055716+0000","last_unstale":"2023-10-06T05:22:33.045148+0000","last_undegraded":"2023-10-06T05:22:33.045148+0000","last_fullsized":"2023-10-06T05:22:33.045148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:03.338584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.064662+0000","last_change":"2023-10-06T05:22:33.058496+0000","last_active":"2023-10-06T05:22:33.064662+0000","last_peered":"2023-10-06T05:22:33.064662+0000","last_clean":"2023-10-06T05:22:33.064662+0000","last_became_active":"2023-10-06T05:22:33.058206+0000","last_became_peered":"2023-10-06T05:22:33.058206+0000","last_unstale":"2023-10-06T05:22:33.064662+0000","last_undegraded":"2023-10-06T05:22:33.064662+0000","last_fullsized":"2023-10-06T05:22:33.064662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:26.459752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040192+0000","last_change":"2023-10-06T05:22:32.059218+0000","last_active":"2023-10-06T05:22:33.040192+0000","last_peered":"2023-10-06T05:22:33.040192+0000","last_clean":"2023-10-06T05:22:33.040192+0000","last_became_active":"2023-10-06T05:22:32.058954+0000","last_became_peered":"2023-10-06T05:22:32.058954+0000","last_unstale":"2023-10-06T05:22:33.040192+0000","last_undegraded":"2023-10-06T05:22:33.040192+0000","last_fullsized":"2023-10-06T05:22:33.040192+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:46.497739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.067414+0000","last_change":"2023-10-06T05:22:33.063760+0000","last_active":"2023-10-06T05:22:33.067414+0000","last_peered":"2023-10-06T05:22:33.067414+0000","last_clean":"2023-10-06T05:22:33.067414+0000","last_became_active":"2023-10-06T05:22:33.063478+0000","last_became_peered":"2023-10-06T05:22:33.063478+0000","last_unstale":"2023-10-06T05:22:33.067414+0000","last_undegraded":"2023-10-06T05:22:33.067414+0000","last_fullsized":"2023-10-06T05:22:33.067414+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:02.999759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.041934+0000","last_change":"2023-10-06T05:22:32.052512+0000","last_active":"2023-10-06T05:22:33.041934+0000","last_peered":"2023-10-06T05:22:33.041934+0000","last_clean":"2023-10-06T05:22:33.041934+0000","last_became_active":"2023-10-06T05:22:32.052252+0000","last_became_peered":"2023-10-06T05:22:32.052252+0000","last_unstale":"2023-10-06T05:22:33.041934+0000","last_undegraded":"2023-10-06T05:22:33.041934+0000","last_fullsized":"2023-10-06T05:22:33.041934+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:28.784261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.065508+0000","last_change":"2023-10-06T05:22:33.060430+0000","last_active":"2023-10-06T05:22:33.065508+0000","last_peered":"2023-10-06T05:22:33.065508+0000","last_clean":"2023-10-06T05:22:33.065508+0000","last_became_active":"2023-10-06T05:22:33.060098+0000","last_became_peered":"2023-10-06T05:22:33.060098+0000","last_unstale":"2023-10-06T05:22:33.065508+0000","last_undegraded":"2023-10-06T05:22:33.065508+0000","last_fullsized":"2023-10-06T05:22:33.065508+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:02.189479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040146+0000","last_change":"2023-10-06T05:22:32.058565+0000","last_active":"2023-10-06T05:22:33.040146+0000","last_peered":"2023-10-06T05:22:33.040146+0000","last_clean":"2023-10-06T05:22:33.040146+0000","last_became_active":"2023-10-06T05:22:32.058288+0000","last_became_peered":"2023-10-06T05:22:32.058288+0000","last_unstale":"2023-10-06T05:22:33.040146+0000","last_undegraded":"2023-10-06T05:22:33.040146+0000","last_fullsized":"2023-10-06T05:22:33.040146+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:31.338180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.063302+0000","last_change":"2023-10-06T05:22:33.058532+0000","last_active":"2023-10-06T05:22:33.063302+0000","last_peered":"2023-10-06T05:22:33.063302+0000","last_clean":"2023-10-06T05:22:33.063302+0000","last_became_active":"2023-10-06T05:22:33.058223+0000","last_became_peered":"2023-10-06T05:22:33.058223+0000","last_unstale":"2023-10-06T05:22:33.063302+0000","last_undegraded":"2023-10-06T05:22:33.063302+0000","last_fullsized":"2023-10-06T05:22:33.063302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:55.927147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.041783+0000","last_change":"2023-10-06T05:22:32.050247+0000","last_active":"2023-10-06T05:22:33.041783+0000","last_peered":"2023-10-06T05:22:33.041783+0000","last_clean":"2023-10-06T05:22:33.041783+0000","last_became_active":"2023-10-06T05:22:32.049943+0000","last_became_peered":"2023-10-06T05:22:32.049943+0000","last_unstale":"2023-10-06T05:22:33.041783+0000","last_undegraded":"2023-10-06T05:22:33.041783+0000","last_fullsized":"2023-10-06T05:22:33.041783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:25.453024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.053346+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:22:33.053346+0000","last_peered":"2023-10-06T05:22:33.053346+0000","last_clean":"2023-10-06T05:22:33.053346+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:22:33.053346+0000","last_undegraded":"2023-10-06T05:22:33.053346+0000","last_fullsized":"2023-10-06T05:22:33.053346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.075746+0000","last_change":"2023-10-06T05:22:33.073110+0000","last_active":"2023-10-06T05:22:33.075746+0000","last_peered":"2023-10-06T05:22:33.075746+0000","last_clean":"2023-10-06T05:22:33.075746+0000","last_became_active":"2023-10-06T05:22:33.072808+0000","last_became_peered":"2023-10-06T05:22:33.072808+0000","last_unstale":"2023-10-06T05:22:33.075746+0000","last_undegraded":"2023-10-06T05:22:33.075746+0000","last_fullsized":"2023-10-06T05:22:33.075746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:59.642608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072765+0000","last_change":"2023-10-06T05:22:32.057484+0000","last_active":"2023-10-06T05:22:33.072765+0000","last_peered":"2023-10-06T05:22:33.072765+0000","last_clean":"2023-10-06T05:22:33.072765+0000","last_became_active":"2023-10-06T05:22:32.056760+0000","last_became_peered":"2023-10-06T05:22:32.056760+0000","last_unstale":"2023-10-06T05:22:33.072765+0000","last_undegraded":"2023-10-06T05:22:33.072765+0000","last_fullsized":"2023-10-06T05:22:33.072765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:31.878129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.049516+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:22:33.049516+0000","last_peered":"2023-10-06T05:22:33.049516+0000","last_clean":"2023-10-06T05:22:33.049516+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:22:33.049516+0000","last_undegraded":"2023-10-06T05:22:33.049516+0000","last_fullsized":"2023-10-06T05:22:33.049516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.068255+0000","last_change":"2023-10-06T05:22:33.065507+0000","last_active":"2023-10-06T05:22:33.068255+0000","last_peered":"2023-10-06T05:22:33.068255+0000","last_clean":"2023-10-06T05:22:33.068255+0000","last_became_active":"2023-10-06T05:22:33.065187+0000","last_became_peered":"2023-10-06T05:22:33.065187+0000","last_unstale":"2023-10-06T05:22:33.068255+0000","last_undegraded":"2023-10-06T05:22:33.068255+0000","last_fullsized":"2023-10-06T05:22:33.068255+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:02.189479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.049527+0000","last_change":"2023-10-06T05:22:32.061884+0000","last_active":"2023-10-06T05:22:33.049527+0000","last_peered":"2023-10-06T05:22:33.049527+0000","last_clean":"2023-10-06T05:22:33.049527+0000","last_became_active":"2023-10-06T05:22:32.061586+0000","last_became_peered":"2023-10-06T05:22:32.061586+0000","last_unstale":"2023-10-06T05:22:33.049527+0000","last_undegraded":"2023-10-06T05:22:33.049527+0000","last_fullsized":"2023-10-06T05:22:33.049527+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:41.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.039823+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:22:33.039823+0000","last_peered":"2023-10-06T05:22:33.039823+0000","last_clean":"2023-10-06T05:22:33.039823+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:22:33.039823+0000","last_undegraded":"2023-10-06T05:22:33.039823+0000","last_fullsized":"2023-10-06T05:22:33.039823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.074426+0000","last_change":"2023-10-06T05:22:33.073103+0000","last_active":"2023-10-06T05:22:33.074426+0000","last_peered":"2023-10-06T05:22:33.074426+0000","last_clean":"2023-10-06T05:22:33.074426+0000","last_became_active":"2023-10-06T05:22:33.072733+0000","last_became_peered":"2023-10-06T05:22:33.072733+0000","last_unstale":"2023-10-06T05:22:33.074426+0000","last_undegraded":"2023-10-06T05:22:33.074426+0000","last_fullsized":"2023-10-06T05:22:33.074426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:09.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.074507+0000","last_change":"2023-10-06T05:22:32.055034+0000","last_active":"2023-10-06T05:22:33.074507+0000","last_peered":"2023-10-06T05:22:33.074507+0000","last_clean":"2023-10-06T05:22:33.074507+0000","last_became_active":"2023-10-06T05:22:32.054712+0000","last_became_peered":"2023-10-06T05:22:32.054712+0000","last_unstale":"2023-10-06T05:22:33.074507+0000","last_undegraded":"2023-10-06T05:22:33.074507+0000","last_fullsized":"2023-10-06T05:22:33.074507+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:47.406781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.050186+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:22:33.050186+0000","last_peered":"2023-10-06T05:22:33.050186+0000","last_clean":"2023-10-06T05:22:33.050186+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:22:33.050186+0000","last_undegraded":"2023-10-06T05:22:33.050186+0000","last_fullsized":"2023-10-06T05:22:33.050186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044682+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:22:33.044682+0000","last_peered":"2023-10-06T05:22:33.044682+0000","last_clean":"2023-10-06T05:22:33.044682+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:22:33.044682+0000","last_undegraded":"2023-10-06T05:22:33.044682+0000","last_fullsized":"2023-10-06T05:22:33.044682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.066648+0000","last_change":"2023-10-06T05:22:33.063053+0000","last_active":"2023-10-06T05:22:33.066648+0000","last_peered":"2023-10-06T05:22:33.066648+0000","last_clean":"2023-10-06T05:22:33.066648+0000","last_became_active":"2023-10-06T05:22:33.062748+0000","last_became_peered":"2023-10-06T05:22:33.062748+0000","last_unstale":"2023-10-06T05:22:33.066648+0000","last_undegraded":"2023-10-06T05:22:33.066648+0000","last_fullsized":"2023-10-06T05:22:33.066648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:17.956567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.071988+0000","last_change":"2023-10-06T05:22:32.055277+0000","last_active":"2023-10-06T05:22:33.071988+0000","last_peered":"2023-10-06T05:22:33.071988+0000","last_clean":"2023-10-06T05:22:33.071988+0000","last_became_active":"2023-10-06T05:22:32.055003+0000","last_became_peered":"2023-10-06T05:22:32.055003+0000","last_unstale":"2023-10-06T05:22:33.071988+0000","last_undegraded":"2023-10-06T05:22:33.071988+0000","last_fullsized":"2023-10-06T05:22:33.071988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:42.984045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044245+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:22:33.044245+0000","last_peered":"2023-10-06T05:22:33.044245+0000","last_clean":"2023-10-06T05:22:33.044245+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:22:33.044245+0000","last_undegraded":"2023-10-06T05:22:33.044245+0000","last_fullsized":"2023-10-06T05:22:33.044245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.072363+0000","last_change":"2023-10-06T05:22:33.066198+0000","last_active":"2023-10-06T05:22:33.072363+0000","last_peered":"2023-10-06T05:22:33.072363+0000","last_clean":"2023-10-06T05:22:33.072363+0000","last_became_active":"2023-10-06T05:22:33.065914+0000","last_became_peered":"2023-10-06T05:22:33.065914+0000","last_unstale":"2023-10-06T05:22:33.072363+0000","last_undegraded":"2023-10-06T05:22:33.072363+0000","last_fullsized":"2023-10-06T05:22:33.072363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:06.769845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040040+0000","last_change":"2023-10-06T05:22:32.053709+0000","last_active":"2023-10-06T05:22:33.040040+0000","last_peered":"2023-10-06T05:22:33.040040+0000","last_clean":"2023-10-06T05:22:33.040040+0000","last_became_active":"2023-10-06T05:22:32.053115+0000","last_became_peered":"2023-10-06T05:22:32.053115+0000","last_unstale":"2023-10-06T05:22:33.040040+0000","last_undegraded":"2023-10-06T05:22:33.040040+0000","last_fullsized":"2023-10-06T05:22:33.040040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:24.907445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.044708+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:22:33.044708+0000","last_peered":"2023-10-06T05:22:33.044708+0000","last_clean":"2023-10-06T05:22:33.044708+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:22:33.044708+0000","last_undegraded":"2023-10-06T05:22:33.044708+0000","last_fullsized":"2023-10-06T05:22:33.044708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.077116+0000","last_change":"2023-10-06T05:22:33.067894+0000","last_active":"2023-10-06T05:22:33.077116+0000","last_peered":"2023-10-06T05:22:33.077116+0000","last_clean":"2023-10-06T05:22:33.077116+0000","last_became_active":"2023-10-06T05:22:33.067580+0000","last_became_peered":"2023-10-06T05:22:33.067580+0000","last_unstale":"2023-10-06T05:22:33.077116+0000","last_undegraded":"2023-10-06T05:22:33.077116+0000","last_fullsized":"2023-10-06T05:22:33.077116+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:31.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.078053+0000","last_change":"2023-10-06T05:22:32.053701+0000","last_active":"2023-10-06T05:22:33.078053+0000","last_peered":"2023-10-06T05:22:33.078053+0000","last_clean":"2023-10-06T05:22:33.078053+0000","last_became_active":"2023-10-06T05:22:32.053435+0000","last_became_peered":"2023-10-06T05:22:32.053435+0000","last_unstale":"2023-10-06T05:22:33.078053+0000","last_undegraded":"2023-10-06T05:22:33.078053+0000","last_fullsized":"2023-10-06T05:22:33.078053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:27.622392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.052489+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:22:33.052489+0000","last_peered":"2023-10-06T05:22:33.052489+0000","last_clean":"2023-10-06T05:22:33.052489+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:22:33.052489+0000","last_undegraded":"2023-10-06T05:22:33.052489+0000","last_fullsized":"2023-10-06T05:22:33.052489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.076755+0000","last_change":"2023-10-06T05:22:33.063092+0000","last_active":"2023-10-06T05:22:33.076755+0000","last_peered":"2023-10-06T05:22:33.076755+0000","last_clean":"2023-10-06T05:22:33.076755+0000","last_became_active":"2023-10-06T05:22:33.062195+0000","last_became_peered":"2023-10-06T05:22:33.062195+0000","last_unstale":"2023-10-06T05:22:33.076755+0000","last_undegraded":"2023-10-06T05:22:33.076755+0000","last_fullsized":"2023-10-06T05:22:33.076755+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.027088+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.027088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:29.319953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.040074+0000","last_change":"2023-10-06T05:22:32.051524+0000","last_active":"2023-10-06T05:22:33.040074+0000","last_peered":"2023-10-06T05:22:33.040074+0000","last_clean":"2023-10-06T05:22:33.040074+0000","last_became_active":"2023-10-06T05:22:32.050882+0000","last_became_peered":"2023-10-06T05:22:32.050882+0000","last_unstale":"2023-10-06T05:22:33.040074+0000","last_undegraded":"2023-10-06T05:22:33.040074+0000","last_fullsized":"2023-10-06T05:22:33.040074+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.020359+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.020359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:08.923388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.042797+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:22:33.042797+0000","last_peered":"2023-10-06T05:22:33.042797+0000","last_clean":"2023-10-06T05:22:33.042797+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:22:33.042797+0000","last_undegraded":"2023-10-06T05:22:33.042797+0000","last_fullsized":"2023-10-06T05:22:33.042797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":1184708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.498},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.89500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.80900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302672,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069168,"statfs":{"total":96636764160,"available":96326828032,"internally_reserved":0,"allocated":1318912,"data_stored":596292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.621},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.443},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.499}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.96899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303248,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068592,"statfs":{"total":96636764160,"available":96326238208,"internally_reserved":0,"allocated":1908736,"data_stored":1186174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.438}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.505},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":1.032},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.99399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.95499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":0.999}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":594734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46999999999999997,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.505,"15min":0.5},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":1.772},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.913},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53900000000000003,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":1.954},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":0.96999999999999997,"5min":1.2030000000000001,"15min":1.282},"last":0.877},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91500000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":0.98499999999999999,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.84199999999999997},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.83899999999999997},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0449999999999999,"5min":1.1910000000000001,"15min":1.343},"last":1},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.84899999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.034,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":0.998,"5min":1.2090000000000001,"15min":1.214},"last":1.0329999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":594272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.621},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":594180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.748}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.54000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:42.829 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:22:42.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:22:43.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.055+0000 7fa433fff700 1 Processor -- start 2023-10-06T05:22:43.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.055+0000 7fa433fff700 1 -- start start 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.056+0000 7fa433fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa434156740 0x7fa4340cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.056+0000 7fa433fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43406b330 con 0x7fa434156370 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.056+0000 7fa433fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43406b610 con 0x7fa4340cc750 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.056+0000 7fa433fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43406beb0 con 0x7fa434156740 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa432ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa434156370 0x7fa4340aef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39328/0 (socket says 172.21.15.33:39328) 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa432ffd700 1 -- 172.21.15.33:0/2624266789 learned_addr learned my addr 172.21.15.33:0/2624266789 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa4327fc700 1 --2- 172.21.15.33:0/2624266789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa434156740 0x7fa4340cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa431ffb700 1 -- 172.21.15.33:0/2624266789 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734614430 0 0) 0x7fa43406b330 con 0x7fa434156370 2023-10-06T05:22:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa431ffb700 1 -- 172.21.15.33:0/2624266789 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41c003570 con 0x7fa434156370 2023-10-06T05:22:43.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa4327fc700 1 -- 172.21.15.33:0/2624266789 >> v1:172.21.15.167:6789/0 conn(0x7fa4340cc750 legacy=0x7fa4340c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa431ffb700 1 -- 172.21.15.33:0/2624266789 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740363360 0 0) 0x7fa43406b610 con 0x7fa4340cc750 2023-10-06T05:22:43.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa4327fc700 1 -- 172.21.15.33:0/2624266789 >> v1:172.21.15.33:6789/0 conn(0x7fa434156370 legacy=0x7fa4340aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.057+0000 7fa4327fc700 1 -- 172.21.15.33:0/2624266789 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43406b8f0 con 0x7fa434156740 2023-10-06T05:22:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.058+0000 7fa4327fc700 1 --2- 172.21.15.33:0/2624266789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa434156740 0x7fa4340cc210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa428009910 tx=0x7fa4280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41ef065c5f929f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.058+0000 7fa431ffb700 1 -- 172.21.15.33:0/2624266789 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428011630 con 0x7fa434156740 2023-10-06T05:22:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.058+0000 7fa431ffb700 1 -- 172.21.15.33:0/2624266789 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428011790 con 0x7fa434156740 2023-10-06T05:22:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.058+0000 7fa431ffb700 1 -- 172.21.15.33:0/2624266789 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428011b50 con 0x7fa434156740 2023-10-06T05:22:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.058+0000 7fa433fff700 1 -- 172.21.15.33:0/2624266789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa434156740 msgr2=0x7fa4340cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.058+0000 7fa433fff700 1 --2- 172.21.15.33:0/2624266789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa434156740 0x7fa4340cc210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa428009910 tx=0x7fa4280091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.059+0000 7fa433fff700 1 -- 172.21.15.33:0/2624266789 shutdown_connections 2023-10-06T05:22:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.059+0000 7fa433fff700 1 --2- 172.21.15.33:0/2624266789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa434156740 0x7fa4340cc210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.059+0000 7fa433fff700 1 -- 172.21.15.33:0/2624266789 >> 172.21.15.33:0/2624266789 conn(0x7fa43406c560 msgr2=0x7fa43406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.059+0000 7fa433fff700 1 -- 172.21.15.33:0/2624266789 shutdown_connections 2023-10-06T05:22:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.059+0000 7fa433fff700 1 -- 172.21.15.33:0/2624266789 wait complete. 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.060+0000 7fa433fff700 1 Processor -- start 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.060+0000 7fa433fff700 1 -- start start 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa433fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4340cc750 0x7fa4341495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa433fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 0x7fa43414fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa433fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa434156740 0x7fa4340b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa433fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434169f90 con 0x7fa434156370 2023-10-06T05:22:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa433fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43416a270 con 0x7fa434156740 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa433fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43416a550 con 0x7fa4340cc750 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa4327fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 0x7fa43414fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa4327fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 0x7fa43414fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41174/0 (socket says 172.21.15.33:41174) 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.061+0000 7fa4327fc700 1 -- 172.21.15.33:0/3323433635 learned_addr learned my addr 172.21.15.33:0/3323433635 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa432ffd700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4340cc750 0x7fa4341495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4337fe700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa434156740 0x7fa4340b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4327fc700 1 -- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4340cc750 msgr2=0x7fa4341495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4327fc700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4340cc750 0x7fa4341495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4327fc700 1 -- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa434156740 msgr2=0x7fa4340b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4327fc700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa434156740 0x7fa4340b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4327fc700 1 -- 172.21.15.33:0/3323433635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa428009420 con 0x7fa434156370 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4337fe700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa434156740 0x7fa4340b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa4327fc700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 0x7fa43414fb50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa428002000 tx=0x7fa428016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428011b10 con 0x7fa434156370 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42801a450 con 0x7fa434156370 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43416a830 con 0x7fa434156370 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.062+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42801a7f0 con 0x7fa434156370 2023-10-06T05:22:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.063+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa434149b00 con 0x7fa434156370 2023-10-06T05:22:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.064+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4340b1e20 con 0x7fa434156370 2023-10-06T05:22:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.064+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa42801ab60 con 0x7fa434156370 2023-10-06T05:22:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.065+0000 7fa41affd700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa420066f30 0x7fa4200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.065+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4280862e0 con 0x7fa434156370 2023-10-06T05:22:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.066+0000 7fa432ffd700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa420066f30 0x7fa4200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.066+0000 7fa432ffd700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa420066f30 0x7fa4200693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa424000fb0 tx=0x7fa424000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.069+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4280533f0 con 0x7fa434156370 2023-10-06T05:22:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.356+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa43404c380 con 0x7fa434156370 2023-10-06T05:22:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.360+0000 7fa41affd700 1 -- 172.21.15.33:0/3323433635 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa428056a10 con 0x7fa434156370 2023-10-06T05:22:43.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.362+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa420066f30 msgr2=0x7fa4200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.362+0000 7fa433fff700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa420066f30 0x7fa4200693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa424000fb0 tx=0x7fa424000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.362+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 msgr2=0x7fa43414fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.362+0000 7fa433fff700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 0x7fa43414fb50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa428002000 tx=0x7fa428016000 comp rx=0 tx=0).stop 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 shutdown_connections 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa420066f30 0x7fa4200693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa434156740 0x7fa4340b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4340cc750 0x7fa4341495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 --2- 172.21.15.33:0/3323433635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa434156370 0x7fa43414fb50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 >> 172.21.15.33:0/3323433635 conn(0x7fa43406c560 msgr2=0x7fa4340ac370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 shutdown_connections 2023-10-06T05:22:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.363+0000 7fa433fff700 1 -- 172.21.15.33:0/3323433635 wait complete. 2023-10-06T05:22:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:22:43.381 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.566+0000 7f772b19d700 1 Processor -- start 2023-10-06T05:22:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.567+0000 7f772b19d700 1 -- start start 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772b19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f772413faa0 0x7f772413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f772406aad0 0x7f772406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772b19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77240c9cb0 con 0x7f77240c7940 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77240ca270 con 0x7f772406aad0 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772b19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772406b840 con 0x7f772413faa0 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772a99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f772406aad0 0x7f772406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772a99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f772406aad0 0x7f772406af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41186/0 (socket says 172.21.15.33:41186) 2023-10-06T05:22:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772a99c700 1 -- 172.21.15.33:0/1305477271 learned_addr learned my addr 172.21.15.33:0/1305477271 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.568+0000 7f772a19b700 1 --2- 172.21.15.33:0/1305477271 >> v2:172.21.15.167:3300/0 conn(0x7f772413faa0 0x7f772413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f772a99c700 1 -- 172.21.15.33:0/1305477271 >> v2:172.21.15.167:3300/0 conn(0x7f772413faa0 msgr2=0x7f772413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f772a99c700 1 --2- 172.21.15.33:0/1305477271 >> v2:172.21.15.167:3300/0 conn(0x7f772413faa0 0x7f772413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f772a99c700 1 -- 172.21.15.33:0/1305477271 >> v1:172.21.15.167:6789/0 conn(0x7f77240c7940 legacy=0x7f77240c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f772a99c700 1 -- 172.21.15.33:0/1305477271 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772406bb20 con 0x7f772406aad0 2023-10-06T05:22:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f772a99c700 1 --2- 172.21.15.33:0/1305477271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f772406af30 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f772406fe50 tx=0x7f771c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cccdd346877212 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f7729199700 1 -- 172.21.15.33:0/1305477271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c00e070 con 0x7f772406aad0 2023-10-06T05:22:43.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f7729199700 1 -- 172.21.15.33:0/1305477271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c00ba20 con 0x7f772406aad0 2023-10-06T05:22:43.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.569+0000 7f7729199700 1 -- 172.21.15.33:0/1305477271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c0092e0 con 0x7f772406aad0 2023-10-06T05:22:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 -- 172.21.15.33:0/1305477271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 msgr2=0x7f772406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 --2- 172.21.15.33:0/1305477271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f772406af30 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f772406fe50 tx=0x7f771c009840 comp rx=0 tx=0).stop 2023-10-06T05:22:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 -- 172.21.15.33:0/1305477271 shutdown_connections 2023-10-06T05:22:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 --2- 172.21.15.33:0/1305477271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f772406af30 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 --2- 172.21.15.33:0/1305477271 >> v2:172.21.15.167:3300/0 conn(0x7f772413faa0 0x7f772413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 -- 172.21.15.33:0/1305477271 >> 172.21.15.33:0/1305477271 conn(0x7f772406d030 msgr2=0x7f772406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 -- 172.21.15.33:0/1305477271 shutdown_connections 2023-10-06T05:22:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.570+0000 7f772b19d700 1 -- 172.21.15.33:0/1305477271 wait complete. 2023-10-06T05:22:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.572+0000 7f772b19d700 1 Processor -- start 2023-10-06T05:22:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.572+0000 7f772b19d700 1 -- start start 2023-10-06T05:22:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f77240c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77240c7940 0x7f77240aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f772413faa0 0x7f77240c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240b03a0 con 0x7f772406aad0 2023-10-06T05:22:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240af4f0 con 0x7f77240c7940 2023-10-06T05:22:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77240af790 con 0x7f772413faa0 2023-10-06T05:22:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f77240c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f77240c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41188/0 (socket says 172.21.15.33:41188) 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 -- 172.21.15.33:0/2541631126 learned_addr learned my addr 172.21.15.33:0/2541631126 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772999a700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77240c7940 0x7f77240aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a99c700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f772413faa0 0x7f77240c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 -- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f772413faa0 msgr2=0x7f77240c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f772413faa0 0x7f77240c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 -- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77240c7940 msgr2=0x7f77240aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77240c7940 0x7f77240aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772a19b700 1 -- 172.21.15.33:0/2541631126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c009ae0 con 0x7f772406aad0 2023-10-06T05:22:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.573+0000 7f772999a700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77240c7940 0x7f77240aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.574+0000 7f772a19b700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f77240c33e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f772000e930 tx=0x7f77200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.574+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000eed0 con 0x7f772406aad0 2023-10-06T05:22:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.574+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000c850 con 0x7f772406aad0 2023-10-06T05:22:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.574+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000cbf0 con 0x7f772406aad0 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.574+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77240afa70 con 0x7f772406aad0 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.574+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77240b0030 con 0x7f772406aad0 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.575+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772404c380 con 0x7f772406aad0 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.576+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7720013470 con 0x7f772406aad0 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.577+0000 7f771a7fc700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7708066f30 0x7f77080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.577+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7720084960 con 0x7f772406aad0 2023-10-06T05:22:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.577+0000 7f772999a700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7708066f30 0x7f77080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.578+0000 7f772999a700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7708066f30 0x7f77080693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7714007fd0 tx=0x7f7714009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.581+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77200519d0 con 0x7f772406aad0 2023-10-06T05:22:43.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.830+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77240c9f00 con 0x7f772406aad0 2023-10-06T05:22:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.830+0000 7f771a7fc700 1 -- 172.21.15.33:0/2541631126 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f7720054ff0 con 0x7f772406aad0 2023-10-06T05:22:43.832 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:43.832 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.538408+0000","modified":"2023-10-06T05:22:36.061558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2226881352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2226881352},{"type":"v1","addr":"172.21.15.33:6835","nonce":2226881352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:22:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.832+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7708066f30 msgr2=0x7f77080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.832+0000 7f772b19d700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7708066f30 0x7f77080693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7714007fd0 tx=0x7f7714009040 comp rx=0 tx=0).stop 2023-10-06T05:22:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.832+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 msgr2=0x7f77240c33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.832+0000 7f772b19d700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f77240c33e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f772000e930 tx=0x7f77200091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 shutdown_connections 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7708066f30 0x7f77080693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f772413faa0 0x7f77240c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772406aad0 0x7f77240c33e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 --2- 172.21.15.33:0/2541631126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77240c7940 0x7f77240aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 >> 172.21.15.33:0/2541631126 conn(0x7f772406d030 msgr2=0x7f77240c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 shutdown_connections 2023-10-06T05:22:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:43.833+0000 7f772b19d700 1 -- 172.21.15.33:0/2541631126 wait complete. 2023-10-06T05:22:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 73 2023-10-06T05:22:44.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:22:32.538408+0000', 'modified': '2023-10-06T05:22:36.061558+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7299': {'gid': 7299, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2226881352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 2226881352}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2226881352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:44.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:22:44.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:22:44.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.442+0000 7fc18b59e700 1 Processor -- start 2023-10-06T05:22:44.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.442+0000 7fc18b59e700 1 -- start start 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.443+0000 7fc18b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 0x7fc18c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.443+0000 7fc18b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c0d0ed0 con 0x7fc18c159400 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.443+0000 7fc18b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c0cb6d0 con 0x7fc18c0cc770 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.443+0000 7fc18b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0cb9b0 con 0x7fc18c1597d0 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.443+0000 7fc18a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc18c159400 0x7fc18c0cab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39336/0 (socket says 172.21.15.33:39336) 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.443+0000 7fc18a59c700 1 -- 172.21.15.33:0/2857675506 learned_addr learned my addr 172.21.15.33:0/2857675506 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc189d9b700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 0x7fc18c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972067218 0 0) 0x7fc18c0d0ed0 con 0x7fc18c159400 2023-10-06T05:22:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc174003570 con 0x7fc18c159400 2023-10-06T05:22:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514760948 0 0) 0x7fc18c0cb6d0 con 0x7fc18c0cc770 2023-10-06T05:22:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc18c0d0ed0 con 0x7fc18c0cc770 2023-10-06T05:22:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175533308 0 0) 0x7fc174003570 con 0x7fc18c159400 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 msgr2=0x7fc18c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 0x7fc18c06c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 >> v1:172.21.15.167:6789/0 conn(0x7fc18c0cc770 legacy=0x7fc18c0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0cbc90 con 0x7fc18c159400 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc17c001470 con 0x7fc18c159400 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 >> v1:172.21.15.33:6789/0 conn(0x7fc18c159400 legacy=0x7fc18c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc174003c20 0x7fc174004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 0x7fc174004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.444+0000 7fc18959a700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174004d50 0x7fc1740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0cb9b0 con 0x7fc174003c20 2023-10-06T05:22:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17400bb70 con 0x7fc174004d50 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17400bd30 con 0x7fc18c1597d0 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc174003c20 0x7fc174004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc189d9b700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 0x7fc174004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174004d50 0x7fc1740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 -- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 msgr2=0x7fc174004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c1597d0 0x7fc174004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 -- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174004d50 msgr2=0x7fc1740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174004d50 0x7fc1740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 -- 172.21.15.33:0/2857675506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc174003570 con 0x7fc174003c20 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174004d50 0x7fc1740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.445+0000 7fc18a59c700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc174003c20 0x7fc174004010 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c2d0 tx=0x7fc17c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da61a4ee97a36a95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00cba0 con 0x7fc174003c20 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00cd00 con 0x7fc174003c20 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18959a700 1 -- 172.21.15.33:0/2857675506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005b70 con 0x7fc174003c20 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18b59e700 1 -- 172.21.15.33:0/2857675506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc174003c20 msgr2=0x7fc174004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18b59e700 1 --2- 172.21.15.33:0/2857675506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc174003c20 0x7fc174004010 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c2d0 tx=0x7fc17c001e10 comp rx=0 tx=0).stop 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18ad9d700 1 -- 172.21.15.33:0/2857675506 reap_dead start 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18b59e700 1 -- 172.21.15.33:0/2857675506 shutdown_connections 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18b59e700 1 -- 172.21.15.33:0/2857675506 >> 172.21.15.33:0/2857675506 conn(0x7fc18c06e4b0 msgr2=0x7fc18c0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18b59e700 1 -- 172.21.15.33:0/2857675506 shutdown_connections 2023-10-06T05:22:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.446+0000 7fc18b59e700 1 -- 172.21.15.33:0/2857675506 wait complete. 2023-10-06T05:22:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.448+0000 7fc18b59e700 1 Processor -- start 2023-10-06T05:22:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.448+0000 7fc18b59e700 1 -- start start 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc174003c20 0x7fc18c14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c9a50 0x7fc18c0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 0x7fc18c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0b20a0 con 0x7fc18c0b16c0 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c14de30 con 0x7fc18c0c9a50 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0ca500 con 0x7fc174003c20 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 0x7fc18c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 0x7fc18c0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41208/0 (socket says 172.21.15.33:41208) 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18ad9d700 1 -- 172.21.15.33:0/3044194058 learned_addr learned my addr 172.21.15.33:0/3044194058 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc18a59c700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc174003c20 0x7fc18c14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.449+0000 7fc189d9b700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c9a50 0x7fc18c0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18ad9d700 1 -- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc174003c20 msgr2=0x7fc18c14cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc174003c20 0x7fc18c14cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18ad9d700 1 -- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c9a50 msgr2=0x7fc18c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c9a50 0x7fc18c0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18ad9d700 1 -- 172.21.15.33:0/3044194058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0ca7e0 con 0x7fc18c0b16c0 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18a59c700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc174003c20 0x7fc18c14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 0x7fc18c0b1b60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc184001460 tx=0x7fc184009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc17a7fc700 1 -- 172.21.15.33:0/3044194058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840032e0 con 0x7fc18c0b16c0 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc17a7fc700 1 -- 172.21.15.33:0/3044194058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc184003440 con 0x7fc18c0b16c0 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc17a7fc700 1 -- 172.21.15.33:0/3044194058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184006df0 con 0x7fc18c0b16c0 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c14e080 con 0x7fc18c0b16c0 2023-10-06T05:22:44.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.450+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c150a10 con 0x7fc18c0b16c0 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.452+0000 7fc17a7fc700 1 -- 172.21.15.33:0/3044194058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1840018f0 con 0x7fc18c0b16c0 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.454+0000 7fc17a7fc700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc16c066f30 0x7fc16c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.454+0000 7fc17a7fc700 1 -- 172.21.15.33:0/3044194058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1840841e0 con 0x7fc18c0b16c0 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.454+0000 7fc18a59c700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc16c066f30 0x7fc16c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.455+0000 7fc18a59c700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc16c066f30 0x7fc16c0693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007c90 tx=0x7fc17c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.455+0000 7fc19f6f3b80 1 Processor -- start 2023-10-06T05:22:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.455+0000 7fc19f6f3b80 1 -- start start 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.456+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e18441d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.456+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1848bbc0 0x556e1846a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.456+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e1843d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.456+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e183c1c80 con 0x556e18441950 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.456+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e18398a70 con 0x556e1848bbc0 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.456+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e18396270 con 0x556e1846ae60 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e18441d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e18441d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41214/0 (socket says 172.21.15.33:41214) 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 -- 172.21.15.33:0/512124173 learned_addr learned my addr 172.21.15.33:0/512124173 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc189d9b700 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1848bbc0 0x556e1846a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e1843d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 -- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 msgr2=0x556e1843d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e1843d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 -- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1848bbc0 msgr2=0x556e1846a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1848bbc0 0x556e1846a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 -- 172.21.15.33:0/512124173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e1822c270 con 0x556e18441950 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc189d9b700 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1848bbc0 0x556e1846a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.457+0000 7fc18a59c700 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e18441d20 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c8c0 tx=0x7fc17c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b79353cf747f877 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc16b7fe700 1 -- 172.21.15.33:0/512124173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0070d0 con 0x556e18441950 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc16b7fe700 1 -- 172.21.15.33:0/512124173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c007230 con 0x556e18441950 2023-10-06T05:22:44.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc16b7fe700 1 -- 172.21.15.33:0/512124173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0075b0 con 0x556e18441950 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/512124173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 msgr2=0x556e18441d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e18441d20 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c8c0 tx=0x7fc17c021040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/512124173 shutdown_connections 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e1843d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e18441d20 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/512124173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1848bbc0 0x556e1846a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/512124173 >> 172.21.15.33:0/512124173 conn(0x556e18443270 msgr2=0x556e18443640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/512124173 shutdown_connections 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.458+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/512124173 wait complete. 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.459+0000 7fc19f6f3b80 1 -- start start 2023-10-06T05:22:44.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.459+0000 7fc19f6f3b80 1 Processor -- start 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e1847fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1846ae60 0x556e1849bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e184381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e18392060 con 0x556e18441950 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e1833d890 con 0x556e1846ae60 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e18358ea0 con 0x556e1848bbc0 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc19f6f3b80 10 client.0 _ll_register_callbacks cb 0x556e1822ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.460+0000 7fc18a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e1847fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e1847fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41224/0 (socket says 172.21.15.33:41224) 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 -- 172.21.15.33:0/1427214202 learned_addr learned my addr 172.21.15.33:0/1427214202 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:44.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc189d9b700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1846ae60 0x556e1849bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e184381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 -- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 msgr2=0x556e184381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e184381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 -- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1846ae60 msgr2=0x556e1849bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1846ae60 0x556e1849bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 -- 172.21.15.33:0/1427214202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc174003570 con 0x556e18441950 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc189d9b700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1846ae60 0x556e1849bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e184381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc18a59c700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e1847fae0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c9c0 tx=0x7fc17c02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7cf34e16a30449 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc168ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.461+0000 7fc168ff9700 1 -- 172.21.15.33:0/1427214202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c02b760 con 0x556e18441950 2023-10-06T05:22:44.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc168ff9700 1 -- 172.21.15.33:0/1427214202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc17c03fc90 con 0x556e18441950 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e18436690 con 0x556e18441950 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc168ff9700 1 -- 172.21.15.33:0/1427214202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc17c03fe50 con 0x556e18441950 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 10 client.7458 fetch_fsmap learned FSMap version 73 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556e181cefe0 con 0x556e18441950 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc168ff9700 1 -- 172.21.15.33:0/1427214202 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556e18436690 con 0x556e18441950 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc168ff9700 1 -- 172.21.15.33:0/1427214202 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc17c02bad0 con 0x556e18441950 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 10 client.7458 fetch_fsmap finished waiting for FSMap version 73 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 10 client.7458 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 20 client.7458 populate_metadata read hostname 'smithi033' 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e18469010 0x556e184693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 4 client.7458 mds_command: new command op to 7299 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:22:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.462+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 --> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e184135c0 con 0x556e18469010 2023-10-06T05:22:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.463+0000 7fc189d9b700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e18469010 0x556e184693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.463+0000 7fc189d9b700 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e18469010 0x556e184693e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d695b30c21e85211 server_cookie=d17a373297963375 in_seq=0 out_seq=0 2023-10-06T05:22:44.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.463+0000 7fc168ff9700 10 client.7458 ms_handle_connect on v2:172.21.15.33:6834/2226881352 2023-10-06T05:22:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc168ff9700 1 -- 172.21.15.33:0/1427214202 <== mds.0 v2:172.21.15.33:6834/2226881352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556e184135c0 con 0x556e18469010 2023-10-06T05:22:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc168ff9700 10 client.7458 handle_command_reply: tid=0 2023-10-06T05:22:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 client.7458 shutdown 2023-10-06T05:22:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 msgr2=0x556e1847fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e1847fae0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c9c0 tx=0x7fc17c02b000 comp rx=0 tx=0).stop 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 shutdown_connections 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e18469010 0x556e184693e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e184381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e18441950 0x556e1847fae0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/1427214202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e1846ae60 0x556e1849bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc168ff9700 0 client.7458 ms_handle_reset on v2:172.21.15.33:6834/2226881352 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 >> 172.21.15.33:0/1427214202 conn(0x556e18443270 msgr2=0x556e184874a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 shutdown_connections 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.464+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/1427214202 wait complete. 2023-10-06T05:22:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.465+0000 7fc19f6f3b80 20 client.7458 trim_cache size 0 max 16384 2023-10-06T05:22:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.482+0000 7fc19f6f3b80 1 Processor -- start 2023-10-06T05:22:44.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.482+0000 7fc19f6f3b80 1 -- start start 2023-10-06T05:22:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 0x556e18469920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e18496ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e18214e30 con 0x556e1846ae60 2023-10-06T05:22:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e1833d810 con 0x556e18441950 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e184210d0 con 0x556e1848bbc0 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 0x556e18469920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 0x556e18469920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41234/0 (socket says 172.21.15.33:41234) 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 -- 172.21.15.33:0/3504432811 learned_addr learned my addr 172.21.15.33:0/3504432811 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 -- 172.21.15.33:0/3504432811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 msgr2=0x556e18496ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e18496ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 -- 172.21.15.33:0/3504432811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 msgr2=0x556e184693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.483+0000 7fc189d9b700 1 -- 172.21.15.33:0/3504432811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e18443910 con 0x556e1846ae60 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc189d9b700 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 0x556e18469920 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc18c150ea0 tx=0x7fc180019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b4d0c045173d5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc16bfff700 1 -- 172.21.15.33:0/3504432811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18001c040 con 0x556e1846ae60 2023-10-06T05:22:44.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc16bfff700 1 -- 172.21.15.33:0/3504432811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc180025020 con 0x556e1846ae60 2023-10-06T05:22:44.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc16bfff700 1 -- 172.21.15.33:0/3504432811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800178c0 con 0x556e1846ae60 2023-10-06T05:22:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/3504432811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 msgr2=0x556e18469920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 0x556e18469920 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc18c150ea0 tx=0x7fc180019040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.484+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/3504432811 shutdown_connections 2023-10-06T05:22:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1848bbc0 0x556e18496ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/3504432811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1846ae60 0x556e18469920 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/3504432811 >> 172.21.15.33:0/3504432811 conn(0x556e18443270 msgr2=0x556e18471260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/3504432811 shutdown_connections 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/3504432811 wait complete. 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 -- start start 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.485+0000 7fc19f6f3b80 1 Processor -- start 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e18496f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 0x556e18497450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e184ed7f0 con 0x556e1848bbc0 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e18443de0 con 0x556e18441950 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e18496a80 con 0x556e1846ae60 2023-10-06T05:22:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.486+0000 7fc19f6f3b80 10 client.0 _ll_register_callbacks cb 0x556e1822ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 0x556e18497450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 0x556e18497450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41242/0 (socket says 172.21.15.33:41242) 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 -- 172.21.15.33:0/4246141532 learned_addr learned my addr 172.21.15.33:0/4246141532 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18a59c700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc189d9b700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e18496f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 -- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 msgr2=0x556e18496f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e18496f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 -- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 msgr2=0x556e184ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 -- 172.21.15.33:0/4246141532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1800099f0 con 0x556e1848bbc0 2023-10-06T05:22:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc18ad9d700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 0x556e18497450 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc184004290 tx=0x7fc184015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce940e54fa2507f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc168ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.487+0000 7fc168ff9700 1 -- 172.21.15.33:0/4246141532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18406f070 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc168ff9700 1 -- 172.21.15.33:0/4246141532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc184058160 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc17c007230 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc168ff9700 1 -- 172.21.15.33:0/4246141532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc184058ea0 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 10 client.7464 fetch_fsmap learned FSMap version 73 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556e1843daf0 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc168ff9700 1 -- 172.21.15.33:0/4246141532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c007230 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc168ff9700 1 -- 172.21.15.33:0/4246141532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc184082170 con 0x556e1848bbc0 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 10 client.7464 fetch_fsmap finished waiting for FSMap version 73 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 10 client.7464 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:22:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 20 client.7464 populate_metadata read hostname 'smithi033' 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e184a4ba0 0x556e184a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 4 client.7464 mds_command: new command op to 7299 tid=0[{"prefix": "status"}] 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 --> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556e18334e00 con 0x556e184a4ba0 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.488+0000 7fc18a59c700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e184a4ba0 0x556e184a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc18a59c700 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e184a4ba0 0x556e184a4f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac84136774590a1 server_cookie=94a6d90bd94e457e in_seq=0 out_seq=0 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc168ff9700 10 client.7464 ms_handle_connect on v2:172.21.15.33:6834/2226881352 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc168ff9700 1 -- 172.21.15.33:0/4246141532 <== mds.0 v2:172.21.15.33:6834/2226881352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556e18334e00 con 0x556e184a4ba0 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc168ff9700 10 client.7464 handle_command_reply: tid=0 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 client.7464 shutdown 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 msgr2=0x556e18497450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 0x556e18497450 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc184004290 tx=0x7fc184015040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 shutdown_connections 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:6834/2226881352,v1:172.21.15.33:6835/2226881352] conn(0x556e184a4ba0 0x556e184a4f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e1848bbc0 0x556e18497450 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e18441950 0x556e184ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 --2- 172.21.15.33:0/4246141532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556e1846ae60 0x556e18496f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 >> 172.21.15.33:0/4246141532 conn(0x556e18443270 msgr2=0x556e1846a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc168ff9700 0 client.7464 ms_handle_reset on v2:172.21.15.33:6834/2226881352 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 shutdown_connections 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.489+0000 7fc19f6f3b80 1 -- 172.21.15.33:0/4246141532 wait complete. 2023-10-06T05:22:44.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc19f6f3b80 20 client.7464 trim_cache size 0 max 16384 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 7299, 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 11.452714288999999, 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 73, 2023-10-06T05:22:44.497 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 105, 2023-10-06T05:22:44.498 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 105, 2023-10-06T05:22:44.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 18.402720321 2023-10-06T05:22:44.498 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc16c066f30 msgr2=0x7fc16c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc16c066f30 0x7fc16c0693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007c90 tx=0x7fc17c015000 comp rx=0 tx=0).stop 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 msgr2=0x7fc18c0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 0x7fc18c0b1b60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc184001460 tx=0x7fc184009040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 shutdown_connections 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc16c066f30 0x7fc16c0693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc18c0b16c0 0x7fc18c0b1b60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc174003c20 0x7fc18c14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 --2- 172.21.15.33:0/3044194058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c9a50 0x7fc18c0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 >> 172.21.15.33:0/3044194058 conn(0x7fc18c06e4b0 msgr2=0x7fc18c0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 shutdown_connections 2023-10-06T05:22:44.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.490+0000 7fc18b59e700 1 -- 172.21.15.33:0/3044194058 wait complete. 2023-10-06T05:22:44.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.694+0000 7faf5c9fe700 1 Processor -- start 2023-10-06T05:22:44.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.694+0000 7faf5c9fe700 1 -- start start 2023-10-06T05:22:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.695+0000 7faf5c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf580c82a0 0x7faf581641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.695+0000 7faf5c9fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5806c820 con 0x7faf58159520 2023-10-06T05:22:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.695+0000 7faf5c9fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5806b670 con 0x7faf580c6340 2023-10-06T05:22:44.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.695+0000 7faf5c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5806a160 con 0x7faf580c82a0 2023-10-06T05:22:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf56d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf58159520 0x7faf580c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39338/0 (socket says 172.21.15.33:39338) 2023-10-06T05:22:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf56d9d700 1 -- 172.21.15.33:0/3428433721 learned_addr learned my addr 172.21.15.33:0/3428433721 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf5759e700 1 --2- 172.21.15.33:0/3428433721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf580c82a0 0x7faf581641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153042121 0 0) 0x7faf5806c820 con 0x7faf58159520 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf40003570 con 0x7faf58159520 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946597077 0 0) 0x7faf5806b670 con 0x7faf580c6340 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf5806c820 con 0x7faf580c6340 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2903971137 0 0) 0x7faf40003570 con 0x7faf58159520 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.696+0000 7faf5759e700 1 -- 172.21.15.33:0/3428433721 >> v1:172.21.15.167:6789/0 conn(0x7faf580c6340 legacy=0x7faf580c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.697+0000 7faf5759e700 1 -- 172.21.15.33:0/3428433721 >> v1:172.21.15.33:6789/0 conn(0x7faf58159520 legacy=0x7faf580c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.697+0000 7faf5759e700 1 -- 172.21.15.33:0/3428433721 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580c8800 con 0x7faf580c82a0 2023-10-06T05:22:44.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.697+0000 7faf5759e700 1 --2- 172.21.15.33:0/3428433721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf580c82a0 0x7faf581641a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e90 tx=0x7faf50009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd23be8876f57b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.697+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000e070 con 0x7faf580c82a0 2023-10-06T05:22:44.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.697+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000b870 con 0x7faf580c82a0 2023-10-06T05:22:44.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.697+0000 7faf55d9b700 1 -- 172.21.15.33:0/3428433721 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48001470 con 0x7faf580c82a0 2023-10-06T05:22:44.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 -- 172.21.15.33:0/3428433721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf580c82a0 msgr2=0x7faf581641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/3428433721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf580c82a0 0x7faf581641a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e90 tx=0x7faf50009780 comp rx=0 tx=0).stop 2023-10-06T05:22:44.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 -- 172.21.15.33:0/3428433721 shutdown_connections 2023-10-06T05:22:44.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/3428433721 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf580c82a0 0x7faf581641a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 -- 172.21.15.33:0/3428433721 >> 172.21.15.33:0/3428433721 conn(0x7faf580cc930 msgr2=0x7faf580ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 -- 172.21.15.33:0/3428433721 shutdown_connections 2023-10-06T05:22:44.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.698+0000 7faf5c9fe700 1 -- 172.21.15.33:0/3428433721 wait complete. 2023-10-06T05:22:44.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.699+0000 7faf5c9fe700 1 Processor -- start 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 -- start start 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 0x7faf5815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf580c82a0 0x7faf581512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf58159520 0x7faf580b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5815a2c0 con 0x7faf580c6340 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58151820 con 0x7faf580c82a0 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.700+0000 7faf5c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5816a030 con 0x7faf58159520 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 0x7faf5815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 0x7faf5815a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41252/0 (socket says 172.21.15.33:41252) 2023-10-06T05:22:44.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 -- 172.21.15.33:0/2850686025 learned_addr learned my addr 172.21.15.33:0/2850686025 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf5659c700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf580c82a0 0x7faf581512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf5759e700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf58159520 0x7faf580b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 -- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf58159520 msgr2=0x7faf580b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf58159520 0x7faf580b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 -- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf580c82a0 msgr2=0x7faf581512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf580c82a0 0x7faf581512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 -- 172.21.15.33:0/2850686025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50009a20 con 0x7faf580c6340 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.701+0000 7faf56d9d700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 0x7faf5815a750 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c7e0 tx=0x7faf48001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.702+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48007360 con 0x7faf580c6340 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.702+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480074e0 con 0x7faf580c6340 2023-10-06T05:22:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.702+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48004850 con 0x7faf580c6340 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.702+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf58159990 con 0x7faf580c6340 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.702+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf58159f50 con 0x7faf580c6340 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.703+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5804c380 con 0x7faf580c6340 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.704+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf48004bc0 con 0x7faf580c6340 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.705+0000 7faf3effd700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf44066f30 0x7faf440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.705+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf48083960 con 0x7faf580c6340 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.705+0000 7faf5659c700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf44066f30 0x7faf440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.706+0000 7faf5659c700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf44066f30 0x7faf440693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faf58158880 tx=0x7faf4c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.709+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf48050a50 con 0x7faf580c6340 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.960+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faf580c8ae0 con 0x7faf580c6340 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.961+0000 7faf3effd700 1 -- 172.21.15.33:0/2850686025 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7faf48018020 con 0x7faf580c6340 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf44066f30 msgr2=0x7faf440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf44066f30 0x7faf440693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faf58158880 tx=0x7faf4c007d50 comp rx=0 tx=0).stop 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 msgr2=0x7faf5815a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 0x7faf5815a750 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c7e0 tx=0x7faf48001a30 comp rx=0 tx=0).stop 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 shutdown_connections 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf44066f30 0x7faf440693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf58159520 0x7faf580b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf580c6340 0x7faf5815a750 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 --2- 172.21.15.33:0/2850686025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf580c82a0 0x7faf581512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 >> 172.21.15.33:0/2850686025 conn(0x7faf580cc930 msgr2=0x7faf580cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 shutdown_connections 2023-10-06T05:22:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:44.963+0000 7faf5c9fe700 1 -- 172.21.15.33:0/2850686025 wait complete. 2023-10-06T05:22:44.982 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:44.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:44.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:22:44.982 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:22:44.983 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:44.983 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:44.983 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:44.983 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:44.983 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:44.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:44.983 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:22:45.020 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41792sec preferred_lft 41792sec 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:22:45.022 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:22:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:22:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:45.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:45.024 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:45.121 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:45.124 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:22:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:22:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:22:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:22:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:45.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:45.164 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:22:45.164 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:22:45.164 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:22:45.164 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:22:45.165 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:22:45.165 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:22:45.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:45.363 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:22:45.400 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:22:45.400 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:22:45.400 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:45.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:45.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:22:45.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:45.508 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:45.547 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:45.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:45.547 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:45.623 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:22:45.623 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:45.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:22:45.756 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:22:45.756 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:22:45.757 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:22:45.773 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:22:45.773 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:22:45.774 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:22:45.774 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:22:45.774 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:22:46.535 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:22:46.540 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:46.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:46.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:46.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:46.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.796+0000 7f4d3979d700 1 Processor -- start 2023-10-06T05:22:46.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.796+0000 7f4d3979d700 1 -- start start 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d3979d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4d34155140 0x7f4d34158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d3979d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d340adf50 0x7f4d340c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d3979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d340c8e40 con 0x7f4d340c9810 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d3979d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d340ae3f0 con 0x7f4d340adf50 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d3979d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d340ae6c0 con 0x7f4d34155140 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d340adf50 0x7f4d340c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d340adf50 0x7f4d340c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41266/0 (socket says 172.21.15.33:41266) 2023-10-06T05:22:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d337fe700 1 -- 172.21.15.33:0/1788986626 learned_addr learned my addr 172.21.15.33:0/1788986626 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d33fff700 1 --2- 172.21.15.33:0/1788986626 >> v2:172.21.15.167:3300/0 conn(0x7f4d34155140 0x7f4d34158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d337fe700 1 -- 172.21.15.33:0/1788986626 >> v2:172.21.15.167:3300/0 conn(0x7f4d34155140 msgr2=0x7f4d34158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.798+0000 7f4d337fe700 1 --2- 172.21.15.33:0/1788986626 >> v2:172.21.15.167:3300/0 conn(0x7f4d34155140 0x7f4d34158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d337fe700 1 -- 172.21.15.33:0/1788986626 >> v1:172.21.15.167:6789/0 conn(0x7f4d340c9810 legacy=0x7f4d340699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d337fe700 1 -- 172.21.15.33:0/1788986626 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340ae9a0 con 0x7f4d340adf50 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d33fff700 1 --2- 172.21.15.33:0/1788986626 >> v2:172.21.15.167:3300/0 conn(0x7f4d34155140 0x7f4d34158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d337fe700 1 --2- 172.21.15.33:0/1788986626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340adf50 0x7f4d340c92d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4d280099b0 tx=0x7f4d28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721fd8e649346370 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d32ffd700 1 -- 172.21.15.33:0/1788986626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800b7e0 con 0x7f4d340adf50 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d32ffd700 1 -- 172.21.15.33:0/1788986626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2800b940 con 0x7f4d340adf50 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.799+0000 7f4d32ffd700 1 -- 172.21.15.33:0/1788986626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800bce0 con 0x7f4d340adf50 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 -- 172.21.15.33:0/1788986626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340adf50 msgr2=0x7f4d340c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1788986626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340adf50 0x7f4d340c92d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4d280099b0 tx=0x7f4d28009250 comp rx=0 tx=0).stop 2023-10-06T05:22:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 -- 172.21.15.33:0/1788986626 shutdown_connections 2023-10-06T05:22:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1788986626 >> v2:172.21.15.167:3300/0 conn(0x7f4d34155140 0x7f4d34158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1788986626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340adf50 0x7f4d340c92d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 -- 172.21.15.33:0/1788986626 >> 172.21.15.33:0/1788986626 conn(0x7f4d3406d030 msgr2=0x7f4d3406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 -- 172.21.15.33:0/1788986626 shutdown_connections 2023-10-06T05:22:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.800+0000 7f4d3979d700 1 -- 172.21.15.33:0/1788986626 wait complete. 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.801+0000 7f4d3979d700 1 Processor -- start 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.801+0000 7f4d3979d700 1 -- start start 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d3979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340adf50 0x7f4d34150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d3979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d340c9810 0x7f4d340c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d3979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 0x7f4d34156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d3979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d34151430 con 0x7f4d34155140 2023-10-06T05:22:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d3979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d34159260 con 0x7f4d340c9810 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d3979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d34158c50 con 0x7f4d340adf50 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.802+0000 7f4d38f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 0x7f4d34156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 0x7f4d34156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41270/0 (socket says 172.21.15.33:41270) 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 -- 172.21.15.33:0/1448519473 learned_addr learned my addr 172.21.15.33:0/1448519473 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 -- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340adf50 msgr2=0x7f4d34150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340adf50 0x7f4d34150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d337fe700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d340c9810 0x7f4d340c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 -- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d340c9810 msgr2=0x7f4d340c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d340c9810 0x7f4d340c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 -- 172.21.15.33:0/1448519473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280094f0 con 0x7f4d34155140 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d337fe700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d340c9810 0x7f4d340c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d38f9c700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 0x7f4d34156c30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00c930 tx=0x7f4d2c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00ce70 con 0x7f4d34155140 2023-10-06T05:22:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c00a7e0 con 0x7f4d34155140 2023-10-06T05:22:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.803+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00aba0 con 0x7f4d34155140 2023-10-06T05:22:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.804+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d34156420 con 0x7f4d34155140 2023-10-06T05:22:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.804+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d34158ef0 con 0x7f4d34155140 2023-10-06T05:22:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.805+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d34156940 con 0x7f4d34155140 2023-10-06T05:22:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.806+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d2c012420 con 0x7f4d34155140 2023-10-06T05:22:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.807+0000 7f4d1bfff700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.807+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4d2c084b00 con 0x7f4d34155140 2023-10-06T05:22:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.807+0000 7f4d33fff700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.808+0000 7f4d33fff700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4d24006fd0 tx=0x7f4d24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:46.811+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d34156940 con 0x7f4d34155140 2023-10-06T05:22:47.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.064+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d34156940 con 0x7f4d34155140 2023-10-06T05:22:47.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.064+0000 7f4d1bfff700 1 -- 172.21.15.33:0/1448519473 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d34156940 con 0x7f4d34155140 2023-10-06T05:22:47.066 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 msgr2=0x7f4d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4d24006fd0 tx=0x7f4d24008040 comp rx=0 tx=0).stop 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 msgr2=0x7f4d34156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 0x7f4d34156c30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00c930 tx=0x7f4d2c0026e0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 shutdown_connections 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d34155140 0x7f4d34156c30 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340adf50 0x7f4d34150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 --2- 172.21.15.33:0/1448519473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d340c9810 0x7f4d340c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 >> 172.21.15.33:0/1448519473 conn(0x7f4d3406d030 msgr2=0x7f4d340b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 shutdown_connections 2023-10-06T05:22:47.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:47.067+0000 7f4d3979d700 1 -- 172.21.15.33:0/1448519473 wait complete. 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:47.086 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:47.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:47.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.087 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:22:47.128 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:47.129 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41794sec preferred_lft 41794sec 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:47.130 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:22:47.131 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:47.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.132 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:47.232 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:22:47.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.235 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:22:47.275 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:22:47.275 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:47.275 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:22:47.276 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:22:47.276 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:47.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.277 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:22:47.278 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:22:47.278 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:22:47.278 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:22:47.278 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:22:47.278 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:22:47.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.472 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:22:47.508 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:22:47.508 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:22:47.508 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:47.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.509 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:22:47.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:47.611 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:47.648 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:47.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:47.649 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:47.724 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:22:47.724 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:47.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:47.855 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:47.855 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:22:47.856 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:22:47.873 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:47.873 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:22:47.873 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:22:47.873 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:22:47.873 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:22:47.893 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:22:47.894 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:22:47.898 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:47.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:47.899 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:47.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:22:48.178 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:22:48.178 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:22:48.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:22:48.384 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:22:48.384 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:22:48.384 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:22:48.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:48.468 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:48.546 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:22:48.546 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:22:48.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:48.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:22:48.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:48.642 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:48.679 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:22:48.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.680 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:48.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:22:48.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:48.763 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:22:48.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.986+0000 7f420ffff700 1 Processor -- start 2023-10-06T05:22:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.986+0000 7f420ffff700 1 -- start start 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.987+0000 7f420ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42100c8b50 0x7f42100c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.987+0000 7f420ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421006ab60 con 0x7f4210155af0 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.987+0000 7f420ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421006ae40 con 0x7f4210155ec0 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.987+0000 7f420ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421006b7e0 con 0x7f42100c8b50 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.987+0000 7f420effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4210155af0 0x7f421006a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39354/0 (socket says 172.21.15.33:39354) 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.987+0000 7f420effd700 1 -- 172.21.15.33:0/3327470119 learned_addr learned my addr 172.21.15.33:0/3327470119 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420e7fc700 1 --2- 172.21.15.33:0/3327470119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42100c8b50 0x7f42100c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306542905 0 0) 0x7f421006ab60 con 0x7f4210155af0 2023-10-06T05:22:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f8003570 con 0x7f4210155af0 2023-10-06T05:22:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681893382 0 0) 0x7f421006ae40 con 0x7f4210155ec0 2023-10-06T05:22:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f421006ab60 con 0x7f4210155ec0 2023-10-06T05:22:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420e7fc700 1 -- 172.21.15.33:0/3327470119 >> v1:172.21.15.167:6789/0 conn(0x7f4210155ec0 legacy=0x7f42100c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420e7fc700 1 -- 172.21.15.33:0/3327470119 >> v1:172.21.15.33:6789/0 conn(0x7f4210155af0 legacy=0x7f421006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420e7fc700 1 -- 172.21.15.33:0/3327470119 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421006b120 con 0x7f42100c8b50 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.988+0000 7f420e7fc700 1 --2- 172.21.15.33:0/3327470119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42100c8b50 0x7f42100c7210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4204009910 tx=0x7f42040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6410cd6985fc192 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204011630 con 0x7f42100c8b50 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204011790 con 0x7f42100c8b50 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420dffb700 1 -- 172.21.15.33:0/3327470119 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204011b30 con 0x7f42100c8b50 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 -- 172.21.15.33:0/3327470119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42100c8b50 msgr2=0x7f42100c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 --2- 172.21.15.33:0/3327470119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42100c8b50 0x7f42100c7210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4204009910 tx=0x7f42040091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 -- 172.21.15.33:0/3327470119 shutdown_connections 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 --2- 172.21.15.33:0/3327470119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42100c8b50 0x7f42100c7210 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 -- 172.21.15.33:0/3327470119 >> 172.21.15.33:0/3327470119 conn(0x7f421006d030 msgr2=0x7f421006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 -- 172.21.15.33:0/3327470119 shutdown_connections 2023-10-06T05:22:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.989+0000 7f420ffff700 1 -- 172.21.15.33:0/3327470119 wait complete. 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.991+0000 7f420ffff700 1 Processor -- start 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.991+0000 7f420ffff700 1 -- start start 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 0x7f4210153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4210155af0 0x7f4210158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4210155ec0 0x7f4210156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4210169f40 con 0x7f42100c8b50 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421016a220 con 0x7f4210155af0 2023-10-06T05:22:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421016a500 con 0x7f4210155ec0 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 0x7f4210153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 0x7f4210153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41278/0 (socket says 172.21.15.33:41278) 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420effd700 1 -- 172.21.15.33:0/2584296930 learned_addr learned my addr 172.21.15.33:0/2584296930 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420f7fe700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4210155ec0 0x7f4210156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.992+0000 7f420e7fc700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4210155af0 0x7f4210158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420effd700 1 -- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4210155ec0 msgr2=0x7f4210156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420effd700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4210155ec0 0x7f4210156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420effd700 1 -- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4210155af0 msgr2=0x7f4210158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420effd700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4210155af0 0x7f4210158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420effd700 1 -- 172.21.15.33:0/2584296930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204009420 con 0x7f42100c8b50 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420e7fc700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4210155af0 0x7f4210158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420f7fe700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4210155ec0 0x7f4210156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420effd700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 0x7f4210153e80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f420000c790 tx=0x7f4200007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200005b30 con 0x7f42100c8b50 2023-10-06T05:22:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200005c90 con 0x7f42100c8b50 2023-10-06T05:22:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421016a7e0 con 0x7f42100c8b50 2023-10-06T05:22:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200007300 con 0x7f42100c8b50 2023-10-06T05:22:48.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.993+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42101543c0 con 0x7f42100c8b50 2023-10-06T05:22:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.994+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421004c380 con 0x7f42100c8b50 2023-10-06T05:22:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.995+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f420001b400 con 0x7f42100c8b50 2023-10-06T05:22:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.996+0000 7f41f6ffd700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41fc066f30 0x7f41fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.996+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4200084260 con 0x7f42100c8b50 2023-10-06T05:22:48.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.996+0000 7f420e7fc700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41fc066f30 0x7f41fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:48.997+0000 7f420e7fc700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41fc066f30 0x7f41fc0693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f42040091e0 tx=0x7f4204017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.001+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4200054e90 con 0x7f42100c8b50 2023-10-06T05:22:49.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.259+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f421006b400 con 0x7f42100c8b50 2023-10-06T05:22:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.260+0000 7f41f6ffd700 1 -- 172.21.15.33:0/2584296930 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f42000076f0 con 0x7f42100c8b50 2023-10-06T05:22:49.262 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:49.262 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.538408+0000","modified":"2023-10-06T05:22:36.061558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2226881352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2226881352},{"type":"v1","addr":"172.21.15.33:6835","nonce":2226881352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:22:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41fc066f30 msgr2=0x7f41fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41fc066f30 0x7f41fc0693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f42040091e0 tx=0x7f4204017000 comp rx=0 tx=0).stop 2023-10-06T05:22:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 msgr2=0x7f4210153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 0x7f4210153e80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f420000c790 tx=0x7f4200007ea0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 shutdown_connections 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41fc066f30 0x7f41fc0693e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4210155ec0 0x7f4210156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42100c8b50 0x7f4210153e80 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 --2- 172.21.15.33:0/2584296930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4210155af0 0x7f4210158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 >> 172.21.15.33:0/2584296930 conn(0x7f421006d030 msgr2=0x7f42100c14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 shutdown_connections 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.262+0000 7f420ffff700 1 -- 172.21.15.33:0/2584296930 wait complete. 2023-10-06T05:22:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 73 2023-10-06T05:22:49.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.467+0000 7f7f66d9d700 1 Processor -- start 2023-10-06T05:22:49.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.467+0000 7f7f66d9d700 1 -- start start 2023-10-06T05:22:49.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f66d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f60155af0 0x7f7f6006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f66d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6006ab60 con 0x7f7f6006a6a0 2023-10-06T05:22:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f66d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6006ae40 con 0x7f7f600c8b50 2023-10-06T05:22:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f66d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6006b7e0 con 0x7f7f60155af0 2023-10-06T05:22:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f6659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7f6006a6a0 0x7f7f600c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39370/0 (socket says 172.21.15.33:39370) 2023-10-06T05:22:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f6659c700 1 -- 172.21.15.33:0/2090275804 learned_addr learned my addr 172.21.15.33:0/2090275804 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.468+0000 7f7f6559a700 1 --2- 172.21.15.33:0/2090275804 >> v2:172.21.15.167:3300/0 conn(0x7f7f60155af0 0x7f7f6006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f64d99700 1 -- 172.21.15.33:0/2090275804 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503015373 0 0) 0x7f7f6006ab60 con 0x7f7f6006a6a0 2023-10-06T05:22:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f64d99700 1 -- 172.21.15.33:0/2090275804 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f4c003570 con 0x7f7f6006a6a0 2023-10-06T05:22:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f6559a700 1 -- 172.21.15.33:0/2090275804 >> v1:172.21.15.167:6789/0 conn(0x7f7f600c8b50 legacy=0x7f7f6006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f6559a700 1 -- 172.21.15.33:0/2090275804 >> v1:172.21.15.33:6789/0 conn(0x7f7f6006a6a0 legacy=0x7f7f600c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f6559a700 1 -- 172.21.15.33:0/2090275804 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6006b120 con 0x7f7f60155af0 2023-10-06T05:22:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f6559a700 1 --2- 172.21.15.33:0/2090275804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f60155af0 0x7f7f6006a160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7f500098e0 tx=0x7f7f500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b13b323487039d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.469+0000 7f7f64d99700 1 -- 172.21.15.33:0/2090275804 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000b640 con 0x7f7f60155af0 2023-10-06T05:22:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f64d99700 1 -- 172.21.15.33:0/2090275804 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5000b7a0 con 0x7f7f60155af0 2023-10-06T05:22:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f64d99700 1 -- 172.21.15.33:0/2090275804 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000bb20 con 0x7f7f60155af0 2023-10-06T05:22:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 -- 172.21.15.33:0/2090275804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f60155af0 msgr2=0x7f7f6006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/2090275804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f60155af0 0x7f7f6006a160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7f500098e0 tx=0x7f7f500091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 -- 172.21.15.33:0/2090275804 shutdown_connections 2023-10-06T05:22:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/2090275804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f60155af0 0x7f7f6006a160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 -- 172.21.15.33:0/2090275804 >> 172.21.15.33:0/2090275804 conn(0x7f7f6006d030 msgr2=0x7f7f6006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 -- 172.21.15.33:0/2090275804 shutdown_connections 2023-10-06T05:22:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.470+0000 7f7f66d9d700 1 -- 172.21.15.33:0/2090275804 wait complete. 2023-10-06T05:22:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.472+0000 7f7f66d9d700 1 Processor -- start 2023-10-06T05:22:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.472+0000 7f7f66d9d700 1 -- start start 2023-10-06T05:22:49.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f66d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f6006a6a0 0x7f7f60158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f66d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f600c8b50 0x7f7f60156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f66d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 0x7f7f6016a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f66d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60158d90 con 0x7f7f60155af0 2023-10-06T05:22:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f66d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60169da0 con 0x7f7f6006a6a0 2023-10-06T05:22:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f66d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60157400 con 0x7f7f600c8b50 2023-10-06T05:22:49.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 0x7f7f6016a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 0x7f7f6016a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41284/0 (socket says 172.21.15.33:41284) 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 -- 172.21.15.33:0/147456685 learned_addr learned my addr 172.21.15.33:0/147456685 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f65d9b700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f6006a6a0 0x7f7f60158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6559a700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f600c8b50 0x7f7f60156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 -- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f600c8b50 msgr2=0x7f7f60156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f600c8b50 0x7f7f60156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 -- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f6006a6a0 msgr2=0x7f7f60158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f6006a6a0 0x7f7f60158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.473+0000 7f7f6659c700 1 -- 172.21.15.33:0/147456685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f50009420 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.474+0000 7f7f6659c700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 0x7f7f6016a230 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800cc40 tx=0x7f7f58001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.474+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800a8b0 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.474+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5800aa10 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.474+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f600c2120 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.474+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800ad90 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.474+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f60159320 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.475+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f60169f10 con 0x7f7f60155af0 2023-10-06T05:22:49.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.476+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f58007300 con 0x7f7f60155af0 2023-10-06T05:22:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.477+0000 7f7f55ffb700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f44066f30 0x7f7f440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.477+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f58084ab0 con 0x7f7f60155af0 2023-10-06T05:22:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.477+0000 7f7f65d9b700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f44066f30 0x7f7f440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.478+0000 7f7f65d9b700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f44066f30 0x7f7f440693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0039e0 tx=0x7f7f5c007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.481+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f58051a90 con 0x7f7f60155af0 2023-10-06T05:22:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.685+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f6004c380 con 0x7f7f60155af0 2023-10-06T05:22:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.686+0000 7f7f55ffb700 1 -- 172.21.15.33:0/147456685 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f7f580550b0 con 0x7f7f60155af0 2023-10-06T05:22:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:49.688 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:34.048391+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.698838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.505677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:49.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f44066f30 msgr2=0x7f7f440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f44066f30 0x7f7f440693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0039e0 tx=0x7f7f5c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 msgr2=0x7f7f6016a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 0x7f7f6016a230 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800cc40 tx=0x7f7f58001a00 comp rx=0 tx=0).stop 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 shutdown_connections 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f44066f30 0x7f7f440693e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f60155af0 0x7f7f6016a230 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f6006a6a0 0x7f7f60158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 --2- 172.21.15.33:0/147456685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f600c8b50 0x7f7f60156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 >> 172.21.15.33:0/147456685 conn(0x7f7f6006d030 msgr2=0x7f7f600cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.688+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 shutdown_connections 2023-10-06T05:22:49.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.689+0000 7f7f66d9d700 1 -- 172.21.15.33:0/147456685 wait complete. 2023-10-06T05:22:49.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:22:30.698838+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:22:31.505677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:22:49.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:22:49.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.898+0000 7fdf851ff700 1 Processor -- start 2023-10-06T05:22:49.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.898+0000 7fdf851ff700 1 -- start start 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf851ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf800cabf0 0x7fdf800c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf851ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800d1120 con 0x7fdf8006d140 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf851ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800d1400 con 0x7fdf80159b10 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf851ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8006e920 con 0x7fdf800cabf0 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf7ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf800cabf0 0x7fdf800c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf7f59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdf8006d140 0x7fdf801634c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56406/0 (socket says 172.21.15.33:56406) 2023-10-06T05:22:49.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.899+0000 7fdf7f59e700 1 -- 172.21.15.33:0/201335280 learned_addr learned my addr 172.21.15.33:0/201335280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7ed9d700 1 -- 172.21.15.33:0/201335280 >> v1:172.21.15.167:6789/0 conn(0x7fdf80159b10 legacy=0x7fdf8006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7ed9d700 1 -- 172.21.15.33:0/201335280 >> v1:172.21.15.33:6789/0 conn(0x7fdf8006d140 legacy=0x7fdf801634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7ed9d700 1 -- 172.21.15.33:0/201335280 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8006ec00 con 0x7fdf800cabf0 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7dd9b700 1 -- 172.21.15.33:0/201335280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185420774 0 0) 0x7fdf800d1120 con 0x7fdf8006d140 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7ed9d700 1 --2- 172.21.15.33:0/201335280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf800cabf0 0x7fdf800c7660 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fdf700098d0 tx=0x7fdf70009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8183cc8fe47ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7dd9b700 1 -- 172.21.15.33:0/201335280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009e70 con 0x7fdf800cabf0 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7dd9b700 1 -- 172.21.15.33:0/201335280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70005c10 con 0x7fdf800cabf0 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.900+0000 7fdf7dd9b700 1 -- 172.21.15.33:0/201335280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000b650 con 0x7fdf800cabf0 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 -- 172.21.15.33:0/201335280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf800cabf0 msgr2=0x7fdf800c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 --2- 172.21.15.33:0/201335280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf800cabf0 0x7fdf800c7660 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fdf700098d0 tx=0x7fdf70009120 comp rx=0 tx=0).stop 2023-10-06T05:22:49.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 -- 172.21.15.33:0/201335280 shutdown_connections 2023-10-06T05:22:49.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 --2- 172.21.15.33:0/201335280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf800cabf0 0x7fdf800c7660 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 -- 172.21.15.33:0/201335280 >> 172.21.15.33:0/201335280 conn(0x7fdf8006cd40 msgr2=0x7fdf800d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 -- 172.21.15.33:0/201335280 shutdown_connections 2023-10-06T05:22:49.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.901+0000 7fdf851ff700 1 -- 172.21.15.33:0/201335280 wait complete. 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.903+0000 7fdf851ff700 1 Processor -- start 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.903+0000 7fdf851ff700 1 -- start start 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf851ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8006d140 0x7fdf800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf851ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf800cabf0 0x7fdf8015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf851ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 0x7fdf80156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf851ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8009a790 con 0x7fdf80159b10 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf851ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800b5720 con 0x7fdf8006d140 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf851ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800b5a00 con 0x7fdf800cabf0 2023-10-06T05:22:49.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 0x7fdf80156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 0x7fdf80156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47748/0 (socket says 172.21.15.33:47748) 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 -- 172.21.15.33:0/2916329853 learned_addr learned my addr 172.21.15.33:0/2916329853 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7ed9d700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8006d140 0x7fdf800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7e59c700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf800cabf0 0x7fdf8015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 -- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf800cabf0 msgr2=0x7fdf8015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf800cabf0 0x7fdf8015eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 -- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8006d140 msgr2=0x7fdf800b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8006d140 0x7fdf800b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.904+0000 7fdf7f59e700 1 -- 172.21.15.33:0/2916329853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700093c0 con 0x7fdf80159b10 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.905+0000 7fdf7f59e700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 0x7fdf80156890 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800cc90 tx=0x7fdf78001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.905+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800cea0 con 0x7fdf80159b10 2023-10-06T05:22:49.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.905+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800ab10 con 0x7fdf80159b10 2023-10-06T05:22:49.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.905+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8009a140 con 0x7fdf80159b10 2023-10-06T05:22:49.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.905+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780070f0 con 0x7fdf80159b10 2023-10-06T05:22:49.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.905+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf800b5360 con 0x7fdf80159b10 2023-10-06T05:22:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.906+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf800cc300 con 0x7fdf80159b10 2023-10-06T05:22:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.907+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf780074e0 con 0x7fdf80159b10 2023-10-06T05:22:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.908+0000 7fdf66ffd700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf6c066f30 0x7fdf6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.908+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdf78084b40 con 0x7fdf80159b10 2023-10-06T05:22:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.908+0000 7fdf7ed9d700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf6c066f30 0x7fdf6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.909+0000 7fdf7ed9d700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf6c066f30 0x7fdf6c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009390 tx=0x7fdf70016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:49.912+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf78051c30 con 0x7fdf80159b10 2023-10-06T05:22:50.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.115+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf8004c380 con 0x7fdf80159b10 2023-10-06T05:22:50.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.116+0000 7fdf66ffd700 1 -- 172.21.15.33:0/2916329853 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf78055250 con 0x7fdf80159b10 2023-10-06T05:22:50.118 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:22:50.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf6c066f30 msgr2=0x7fdf6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf6c066f30 0x7fdf6c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009390 tx=0x7fdf70016000 comp rx=0 tx=0).stop 2023-10-06T05:22:50.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 msgr2=0x7fdf80156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 0x7fdf80156890 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800cc90 tx=0x7fdf78001a00 comp rx=0 tx=0).stop 2023-10-06T05:22:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 shutdown_connections 2023-10-06T05:22:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf6c066f30 0x7fdf6c0693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80159b10 0x7fdf80156890 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8006d140 0x7fdf800b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 --2- 172.21.15.33:0/2916329853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf800cabf0 0x7fdf8015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 >> 172.21.15.33:0/2916329853 conn(0x7fdf8006cd40 msgr2=0x7fdf800c6090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 shutdown_connections 2023-10-06T05:22:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.118+0000 7fdf851ff700 1 -- 172.21.15.33:0/2916329853 wait complete. 2023-10-06T05:22:50.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:22:50.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.327+0000 7f3acf59e700 1 Processor -- start 2023-10-06T05:22:50.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.327+0000 7f3acf59e700 1 -- start start 2023-10-06T05:22:50.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acf59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ad00cd5e0 0x7f3ad00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acf59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3ad015a650 0x7f3ad015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acf59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad015d660 con 0x7f3ad00af540 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acf59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad00ce890 con 0x7f3ad00cd5e0 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acf59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad015aa60 con 0x7f3ad015a650 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3ace59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3ad00af540 0x7f3ad00cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56422/0 (socket says 172.21.15.33:56422) 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3ace59c700 1 -- 172.21.15.33:0/4155226536 learned_addr learned my addr 172.21.15.33:0/4155226536 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acdd9b700 1 --2- 172.21.15.33:0/4155226536 >> v2:172.21.15.33:3300/0 conn(0x7f3ad00cd5e0 0x7f3ad00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3aced9d700 1 --2- 172.21.15.33:0/4155226536 >> v2:172.21.15.167:3300/0 conn(0x7f3ad015a650 0x7f3ad015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acd59a700 1 -- 172.21.15.33:0/4155226536 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923645943 0 0) 0x7f3ad015d660 con 0x7f3ad00af540 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.329+0000 7f3acd59a700 1 -- 172.21.15.33:0/4155226536 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab8003570 con 0x7f3ad00af540 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acdd9b700 1 -- 172.21.15.33:0/4155226536 >> v2:172.21.15.167:3300/0 conn(0x7f3ad015a650 msgr2=0x7f3ad015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acdd9b700 1 --2- 172.21.15.33:0/4155226536 >> v2:172.21.15.167:3300/0 conn(0x7f3ad015a650 0x7f3ad015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acdd9b700 1 -- 172.21.15.33:0/4155226536 >> v1:172.21.15.33:6789/0 conn(0x7f3ad00af540 legacy=0x7f3ad00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acdd9b700 1 -- 172.21.15.33:0/4155226536 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad015d890 con 0x7f3ad00cd5e0 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acdd9b700 1 --2- 172.21.15.33:0/4155226536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad00cd5e0 0x7f3ad00af910 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00caa90 tx=0x7f3ac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dc891215e68d8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acd59a700 1 -- 172.21.15.33:0/4155226536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4011630 con 0x7f3ad00cd5e0 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acd59a700 1 -- 172.21.15.33:0/4155226536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac40117b0 con 0x7f3ad00cd5e0 2023-10-06T05:22:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.330+0000 7f3acd59a700 1 -- 172.21.15.33:0/4155226536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4011b70 con 0x7f3ad00cd5e0 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 -- 172.21.15.33:0/4155226536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad00cd5e0 msgr2=0x7f3ad00af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 --2- 172.21.15.33:0/4155226536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad00cd5e0 0x7f3ad00af910 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00caa90 tx=0x7f3ac40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 -- 172.21.15.33:0/4155226536 shutdown_connections 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 --2- 172.21.15.33:0/4155226536 >> v2:172.21.15.167:3300/0 conn(0x7f3ad015a650 0x7f3ad015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 --2- 172.21.15.33:0/4155226536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad00cd5e0 0x7f3ad00af910 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 -- 172.21.15.33:0/4155226536 >> 172.21.15.33:0/4155226536 conn(0x7f3ad006d9b0 msgr2=0x7f3ad006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 -- 172.21.15.33:0/4155226536 shutdown_connections 2023-10-06T05:22:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.331+0000 7f3acf59e700 1 -- 172.21.15.33:0/4155226536 wait complete. 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.332+0000 7f3acf59e700 1 Processor -- start 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.332+0000 7f3acf59e700 1 -- start start 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3acf59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ad00af540 0x7f3ad00ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3acf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ad00cd5e0 0x7f3ad0145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3acf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 0x7f3ad00b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3acf59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00c9d30 con 0x7f3ad015a650 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3acf59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01464a0 con 0x7f3ad00af540 2023-10-06T05:22:50.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3acf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00c8e40 con 0x7f3ad00cd5e0 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.333+0000 7f3aced9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 0x7f3ad00b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 0x7f3ad00b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47764/0 (socket says 172.21.15.33:47764) 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 -- 172.21.15.33:0/891096101 learned_addr learned my addr 172.21.15.33:0/891096101 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3ace59c700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ad00af540 0x7f3ad00ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3acdd9b700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ad00cd5e0 0x7f3ad0145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 -- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ad00cd5e0 msgr2=0x7f3ad0145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ad00cd5e0 0x7f3ad0145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 -- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ad00af540 msgr2=0x7f3ad00ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ad00af540 0x7f3ad00ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 -- 172.21.15.33:0/891096101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4009420 con 0x7f3ad015a650 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3aced9d700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 0x7f3ad00b4950 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009e90 tx=0x7f3ac800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:50.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.334+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800cc20 con 0x7f3ad015a650 2023-10-06T05:22:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.335+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac800f040 con 0x7f3ad015a650 2023-10-06T05:22:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.335+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad00c9120 con 0x7f3ad015a650 2023-10-06T05:22:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.335+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8014480 con 0x7f3ad015a650 2023-10-06T05:22:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.335+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad00c96e0 con 0x7f3ad015a650 2023-10-06T05:22:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.336+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad00c9ea0 con 0x7f3ad015a650 2023-10-06T05:22:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.336+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ac8014830 con 0x7f3ad015a650 2023-10-06T05:22:50.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.338+0000 7f3abe7fc700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3ab0066f30 0x7f3ab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.338+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ac80851c0 con 0x7f3ad015a650 2023-10-06T05:22:50.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.338+0000 7f3ace59c700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3ab0066f30 0x7f3ab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.339+0000 7f3ace59c700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3ab0066f30 0x7f3ab00693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00058d0 tx=0x7f3ac0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:50.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.341+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ac8055cc0 con 0x7f3ad015a650 2023-10-06T05:22:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.540+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ad004c380 con 0x7f3ad015a650 2023-10-06T05:22:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.541+0000 7f3abe7fc700 1 -- 172.21.15.33:0/891096101 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v105) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ac8055850 con 0x7f3ad015a650 2023-10-06T05:22:50.543 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:22:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.543+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3ab0066f30 msgr2=0x7f3ab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.543+0000 7f3acf59e700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3ab0066f30 0x7f3ab00693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00058d0 tx=0x7f3ac0009040 comp rx=0 tx=0).stop 2023-10-06T05:22:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.543+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 msgr2=0x7f3ad00b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.543+0000 7f3acf59e700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 0x7f3ad00b4950 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009e90 tx=0x7f3ac800b040 comp rx=0 tx=0).stop 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 shutdown_connections 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3ab0066f30 0x7f3ab00693e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ad015a650 0x7f3ad00b4950 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ad00af540 0x7f3ad00ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 --2- 172.21.15.33:0/891096101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ad00cd5e0 0x7f3ad0145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 >> 172.21.15.33:0/891096101 conn(0x7f3ad006d9b0 msgr2=0x7f3ad00cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 shutdown_connections 2023-10-06T05:22:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.544+0000 7f3acf59e700 1 -- 172.21.15.33:0/891096101 wait complete. 2023-10-06T05:22:50.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:22:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.750+0000 7fde2b39d700 1 Processor -- start 2023-10-06T05:22:50.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.750+0000 7fde2b39d700 1 -- start start 2023-10-06T05:22:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2b39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde24155a50 0x7fde24155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde240caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240b06b0 con 0x7fde240c9490 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2b39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde240c99e0 con 0x7fde24155a50 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde240c9cc0 con 0x7fde240afb20 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2a39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde24155a50 0x7fde24155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde29b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fde240c9490 0x7fde240af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56426/0 (socket says 172.21.15.33:56426) 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2a39b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fde24155a50 0x7fde24155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47768/0 (socket says 172.21.15.33:47768) 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde29b9a700 1 -- 172.21.15.33:0/3990244152 learned_addr learned my addr 172.21.15.33:0/3990244152 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.752+0000 7fde2ab9c700 1 --2- 172.21.15.33:0/3990244152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde240caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde2a39b700 1 -- 172.21.15.33:0/3990244152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 msgr2=0x7fde240caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde2a39b700 1 --2- 172.21.15.33:0/3990244152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde240caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde2a39b700 1 -- 172.21.15.33:0/3990244152 >> v1:172.21.15.33:6789/0 conn(0x7fde240c9490 legacy=0x7fde240af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde2a39b700 1 -- 172.21.15.33:0/3990244152 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240c9fa0 con 0x7fde24155a50 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde2a39b700 1 --2- 172.21.15.33:0/3990244152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde24155a50 0x7fde24155e20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fde200098e0 tx=0x7fde200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5d257d7d1dc62d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde29399700 1 -- 172.21.15.33:0/3990244152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2000b640 con 0x7fde24155a50 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde29399700 1 -- 172.21.15.33:0/3990244152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2000b7a0 con 0x7fde24155a50 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.753+0000 7fde29399700 1 -- 172.21.15.33:0/3990244152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2000bb40 con 0x7fde24155a50 2023-10-06T05:22:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 -- 172.21.15.33:0/3990244152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde24155a50 msgr2=0x7fde24155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 --2- 172.21.15.33:0/3990244152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde24155a50 0x7fde24155e20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fde200098e0 tx=0x7fde200091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 -- 172.21.15.33:0/3990244152 shutdown_connections 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 --2- 172.21.15.33:0/3990244152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde24155a50 0x7fde24155e20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 --2- 172.21.15.33:0/3990244152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde240caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 -- 172.21.15.33:0/3990244152 >> 172.21.15.33:0/3990244152 conn(0x7fde2406d030 msgr2=0x7fde240cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 -- 172.21.15.33:0/3990244152 shutdown_connections 2023-10-06T05:22:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.754+0000 7fde2b39d700 1 -- 172.21.15.33:0/3990244152 wait complete. 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.755+0000 7fde2b39d700 1 Processor -- start 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.755+0000 7fde2b39d700 1 -- start start 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.756+0000 7fde2b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde2414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.756+0000 7fde2b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 0x7fde24151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.756+0000 7fde2b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde24155a50 0x7fde240c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.756+0000 7fde2b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde24156470 con 0x7fde240c9490 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.756+0000 7fde2b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde24156a30 con 0x7fde24155a50 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.756+0000 7fde2b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde241527e0 con 0x7fde240afb20 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 0x7fde24151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 0x7fde24151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47776/0 (socket says 172.21.15.33:47776) 2023-10-06T05:22:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 -- 172.21.15.33:0/2011822915 learned_addr learned my addr 172.21.15.33:0/2011822915 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde2a39b700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde2414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde2ab9c700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde24155a50 0x7fde240c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 -- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 msgr2=0x7fde2414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde2414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 -- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde24155a50 msgr2=0x7fde240c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde24155a50 0x7fde240c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 -- 172.21.15.33:0/2011822915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde20009420 con 0x7fde240c9490 2023-10-06T05:22:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.757+0000 7fde29b9a700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 0x7fde24151d20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fde1400c7c0 tx=0x7fde14001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.758+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140062e0 con 0x7fde240c9490 2023-10-06T05:22:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.758+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14006440 con 0x7fde240c9490 2023-10-06T05:22:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.758+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14004830 con 0x7fde240c9490 2023-10-06T05:22:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.758+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde24152ac0 con 0x7fde240c9490 2023-10-06T05:22:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.758+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2414b100 con 0x7fde240c9490 2023-10-06T05:22:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.759+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde24156bc0 con 0x7fde240c9490 2023-10-06T05:22:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.760+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde14004ba0 con 0x7fde240c9490 2023-10-06T05:22:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.761+0000 7fde1a7fc700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.761+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde14083950 con 0x7fde240c9490 2023-10-06T05:22:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.761+0000 7fde2a39b700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.762+0000 7fde2a39b700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde08066f30 0x7fde080693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fde200093f0 tx=0x7fde20017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:50.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.765+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde1404b5a0 con 0x7fde240c9490 2023-10-06T05:22:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.966+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde2404c380 con 0x7fde240c9490 2023-10-06T05:22:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.967+0000 7fde1a7fc700 1 -- 172.21.15.33:0/2011822915 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde140170a0 con 0x7fde240c9490 2023-10-06T05:22:50.968 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:22:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde08066f30 msgr2=0x7fde080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde08066f30 0x7fde080693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fde200093f0 tx=0x7fde20017000 comp rx=0 tx=0).stop 2023-10-06T05:22:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 msgr2=0x7fde24151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 0x7fde24151d20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fde1400c7c0 tx=0x7fde14001120 comp rx=0 tx=0).stop 2023-10-06T05:22:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 shutdown_connections 2023-10-06T05:22:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde24155a50 0x7fde240c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde240afb20 0x7fde2414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 --2- 172.21.15.33:0/2011822915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde240c9490 0x7fde24151d20 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 >> 172.21.15.33:0/2011822915 conn(0x7fde2406d030 msgr2=0x7fde2406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 shutdown_connections 2023-10-06T05:22:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:50.969+0000 7fde2b39d700 1 -- 172.21.15.33:0/2011822915 wait complete. 2023-10-06T05:22:50.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:22:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.178+0000 7f18f759e700 1 Processor -- start 2023-10-06T05:22:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.178+0000 7f18f759e700 1 -- start start 2023-10-06T05:22:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.179+0000 7f18f759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f18f815acd0 0x7f18f80abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.179+0000 7f18f759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80b0cd0 con 0x7f18f8159850 2023-10-06T05:22:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.179+0000 7f18f759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f815a840 con 0x7f18f806efa0 2023-10-06T05:22:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.179+0000 7f18f759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f8159f10 con 0x7f18f815acd0 2023-10-06T05:22:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.179+0000 7f18f659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f18f8159850 0x7f18f806e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56428/0 (socket says 172.21.15.33:56428) 2023-10-06T05:22:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.179+0000 7f18f659c700 1 -- 172.21.15.33:0/3247852118 learned_addr learned my addr 172.21.15.33:0/3247852118 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:51.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/3247852118 >> v2:172.21.15.167:3300/0 conn(0x7f18f815acd0 0x7f18f80abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002266943 0 0) 0x7f18f80b0cd0 con 0x7f18f8159850 2023-10-06T05:22:51.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e0003570 con 0x7f18f8159850 2023-10-06T05:22:51.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851657214 0 0) 0x7f18e0003570 con 0x7f18f8159850 2023-10-06T05:22:51.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 >> v2:172.21.15.167:3300/0 conn(0x7f18f815acd0 msgr2=0x7f18f80abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 --2- 172.21.15.33:0/3247852118 >> v2:172.21.15.167:3300/0 conn(0x7f18f815acd0 0x7f18f80abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 >> v1:172.21.15.167:6789/0 conn(0x7f18f806efa0 legacy=0x7f18f80ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f815a1f0 con 0x7f18f8159850 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f18e8001470 con 0x7f18f8159850 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 >> v1:172.21.15.33:6789/0 conn(0x7f18f8159850 legacy=0x7f18f806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0003ab0 0x7f18e0003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0004810 0x7f18e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18e0005210 0x7f18e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8159f10 con 0x7f18e0004810 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e000c1b0 con 0x7f18e0003ab0 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.180+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e000c370 con 0x7f18e0005210 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0004810 0x7f18e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f659c700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0003ab0 0x7f18e0003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f6d9d700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18e0005210 0x7f18e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 -- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18e0005210 msgr2=0x7f18e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18e0005210 0x7f18e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 -- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0003ab0 msgr2=0x7f18e0003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0003ab0 0x7f18e0003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 -- 172.21.15.33:0/3247852118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0003570 con 0x7f18e0004810 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f6d9d700 1 -- 172.21.15.33:0/3247852118 reap_dead start 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.181+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0004810 0x7f18e0004c80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009c00 tx=0x7f18ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1efbc2c7c9ae441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00e710 con 0x7f18e0004810 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00e870 con 0x7f18e0004810 2023-10-06T05:22:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f559a700 1 -- 172.21.15.33:0/3247852118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00ec30 con 0x7f18e0004810 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 -- 172.21.15.33:0/3247852118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0004810 msgr2=0x7f18e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0004810 0x7f18e0004c80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009c00 tx=0x7f18ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 -- 172.21.15.33:0/3247852118 shutdown_connections 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 --2- 172.21.15.33:0/3247852118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0004810 0x7f18e0004c80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 -- 172.21.15.33:0/3247852118 >> 172.21.15.33:0/3247852118 conn(0x7f18f806cd40 msgr2=0x7f18f80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 -- 172.21.15.33:0/3247852118 shutdown_connections 2023-10-06T05:22:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.182+0000 7f18f759e700 1 -- 172.21.15.33:0/3247852118 wait complete. 2023-10-06T05:22:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.184+0000 7f18f759e700 1 Processor -- start 2023-10-06T05:22:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.184+0000 7f18f759e700 1 -- start start 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0004810 0x7f18f80cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 0x7f18f80accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f8151f60 0x7f18f8156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80ad230 con 0x7f18f80cad10 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80ac860 con 0x7f18e0004810 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80cb210 con 0x7f18f8151f60 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 0x7f18f80accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 0x7f18f80accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47798/0 (socket says 172.21.15.33:47798) 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 -- 172.21.15.33:0/1089113200 learned_addr learned my addr 172.21.15.33:0/1089113200 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f6d9d700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f8151f60 0x7f18f8156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f659c700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0004810 0x7f18f80cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 -- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f8151f60 msgr2=0x7f18f8156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f8151f60 0x7f18f8156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 -- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0004810 msgr2=0x7f18f80cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0004810 0x7f18f80cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.185+0000 7f18f5d9b700 1 -- 172.21.15.33:0/1089113200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f80abf30 con 0x7f18f80cad10 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.186+0000 7f18f5d9b700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 0x7f18f80accf0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e4b0 tx=0x7f18ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.186+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec022070 con 0x7f18f80cad10 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.186+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00e9e0 con 0x7f18f80cad10 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.186+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f80ac210 con 0x7f18f80cad10 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.186+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00e710 con 0x7f18f80cad10 2023-10-06T05:22:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.186+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f80cb710 con 0x7f18f80cad10 2023-10-06T05:22:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.187+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f80ac9d0 con 0x7f18f80cad10 2023-10-06T05:22:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.188+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18ec027680 con 0x7f18f80cad10 2023-10-06T05:22:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.189+0000 7f18e67fc700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18d8066f30 0x7f18d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.189+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f18ec086160 con 0x7f18f80cad10 2023-10-06T05:22:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.190+0000 7f18f659c700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18d8066f30 0x7f18d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.190+0000 7f18f659c700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18d8066f30 0x7f18d80693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f18e8001c10 tx=0x7f18e8001d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.193+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18ec053280 con 0x7f18f80cad10 2023-10-06T05:22:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.396+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18f804c380 con 0x7f18f80cad10 2023-10-06T05:22:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.396+0000 7f18e67fc700 1 -- 172.21.15.33:0/1089113200 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18ec0568a0 con 0x7f18f80cad10 2023-10-06T05:22:51.398 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18d8066f30 msgr2=0x7f18d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18d8066f30 0x7f18d80693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f18e8001c10 tx=0x7f18e8001d80 comp rx=0 tx=0).stop 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 msgr2=0x7f18f80accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 0x7f18f80accf0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e4b0 tx=0x7f18ec01a000 comp rx=0 tx=0).stop 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 shutdown_connections 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18d8066f30 0x7f18d80693e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18f8151f60 0x7f18f8156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e0004810 0x7f18f80cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 --2- 172.21.15.33:0/1089113200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f80cad10 0x7f18f80accf0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 >> 172.21.15.33:0/1089113200 conn(0x7f18f806cd40 msgr2=0x7f18f815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 shutdown_connections 2023-10-06T05:22:51.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.399+0000 7f18f759e700 1 -- 172.21.15.33:0/1089113200 wait complete. 2023-10-06T05:22:51.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:51.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.604+0000 7fa0c719d700 1 Processor -- start 2023-10-06T05:22:51.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.604+0000 7fa0c719d700 1 -- start start 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa0c00cb710 0x7fa0c006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c00c9cb0 con 0x7fa0c00c8050 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c00ca270 con 0x7fa0c013faa0 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c006b840 con 0x7fa0c00cb710 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa0c00c8050 0x7fa0c006aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56434/0 (socket says 172.21.15.33:56434) 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c599a700 1 -- 172.21.15.33:0/3024246777 learned_addr learned my addr 172.21.15.33:0/3024246777 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.605+0000 7fa0c699c700 1 --2- 172.21.15.33:0/3024246777 >> v2:172.21.15.33:3300/0 conn(0x7fa0c00cb710 0x7fa0c006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c5199700 1 -- 172.21.15.33:0/3024246777 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971620190 0 0) 0x7fa0c00c9cb0 con 0x7fa0c00c8050 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c5199700 1 -- 172.21.15.33:0/3024246777 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ac003570 con 0x7fa0c00c8050 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c699c700 1 -- 172.21.15.33:0/3024246777 >> v1:172.21.15.167:6789/0 conn(0x7fa0c013faa0 legacy=0x7fa0c00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c699c700 1 -- 172.21.15.33:0/3024246777 >> v1:172.21.15.33:6789/0 conn(0x7fa0c00c8050 legacy=0x7fa0c006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c699c700 1 -- 172.21.15.33:0/3024246777 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c006bb20 con 0x7fa0c00cb710 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c699c700 1 --2- 172.21.15.33:0/3024246777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c006b1e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334c8cc5802a7982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.606+0000 7fa0c5199700 1 -- 172.21.15.33:0/3024246777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800e070 con 0x7fa0c00cb710 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c5199700 1 -- 172.21.15.33:0/3024246777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800bb60 con 0x7fa0c00cb710 2023-10-06T05:22:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c5199700 1 -- 172.21.15.33:0/3024246777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8012430 con 0x7fa0c00cb710 2023-10-06T05:22:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 -- 172.21.15.33:0/3024246777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 msgr2=0x7fa0c006b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 --2- 172.21.15.33:0/3024246777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c006b1e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b8009780 comp rx=0 tx=0).stop 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 -- 172.21.15.33:0/3024246777 shutdown_connections 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 --2- 172.21.15.33:0/3024246777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c006b1e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 -- 172.21.15.33:0/3024246777 >> 172.21.15.33:0/3024246777 conn(0x7fa0c006d030 msgr2=0x7fa0c006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 -- 172.21.15.33:0/3024246777 shutdown_connections 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.607+0000 7fa0c719d700 1 -- 172.21.15.33:0/3024246777 wait complete. 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.609+0000 7fa0c719d700 1 Processor -- start 2023-10-06T05:22:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.609+0000 7fa0c719d700 1 -- start start 2023-10-06T05:22:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c00c8050 0x7fa0c01594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c013c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0c013faa0 0x7fa0c0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c00c51a0 con 0x7fa0c00cb710 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c00c3630 con 0x7fa0c00c8050 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c00c4be0 con 0x7fa0c013faa0 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c013c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c013c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47806/0 (socket says 172.21.15.33:47806) 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 -- 172.21.15.33:0/617472905 learned_addr learned my addr 172.21.15.33:0/617472905 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c699c700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0c013faa0 0x7fa0c0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c619b700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c00c8050 0x7fa0c01594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 -- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0c013faa0 msgr2=0x7fa0c0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0c013faa0 0x7fa0c0157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 -- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c00c8050 msgr2=0x7fa0c01594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c00c8050 0x7fa0c01594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c599a700 1 -- 172.21.15.33:0/617472905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009a20 con 0x7fa0c00cb710 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.610+0000 7fa0c619b700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c00c8050 0x7fa0c01594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.611+0000 7fa0c599a700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c013c690 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7c0 tx=0x7fa0b0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.611+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0005b30 con 0x7fa0c00cb710 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.611+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0005cb0 con 0x7fa0c00cb710 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.611+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0007320 con 0x7fa0c00cb710 2023-10-06T05:22:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.611+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c00c4ec0 con 0x7fa0c00cb710 2023-10-06T05:22:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.611+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c0159a20 con 0x7fa0c00cb710 2023-10-06T05:22:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.612+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c00c37a0 con 0x7fa0c00cb710 2023-10-06T05:22:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.613+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0b001b3b0 con 0x7fa0c00cb710 2023-10-06T05:22:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.614+0000 7fa0b67fc700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0a4066f30 0x7fa0a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.614+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa0b0083b90 con 0x7fa0c00cb710 2023-10-06T05:22:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.614+0000 7fa0c619b700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0a4066f30 0x7fa0a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.615+0000 7fa0c619b700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0a4066f30 0x7fa0a40693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc007fd0 tx=0x7fa0bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.618+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b0050ca0 con 0x7fa0c00cb710 2023-10-06T05:22:51.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.864+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0c004c380 con 0x7fa0c00cb710 2023-10-06T05:22:51.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.864+0000 7fa0b67fc700 1 -- 172.21.15.33:0/617472905 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fa0b0017020 con 0x7fa0c00cb710 2023-10-06T05:22:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:51.866 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.538408+0000","modified":"2023-10-06T05:22:36.061558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2226881352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2226881352},{"type":"v1","addr":"172.21.15.33:6835","nonce":2226881352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0a4066f30 msgr2=0x7fa0a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0a4066f30 0x7fa0a40693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc007fd0 tx=0x7fa0bc009040 comp rx=0 tx=0).stop 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 msgr2=0x7fa0c013c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c013c690 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7c0 tx=0x7fa0b0007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 shutdown_connections 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa0a4066f30 0x7fa0a40693e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0c013faa0 0x7fa0c0157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c00c8050 0x7fa0c01594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 --2- 172.21.15.33:0/617472905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c00cb710 0x7fa0c013c690 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 >> 172.21.15.33:0/617472905 conn(0x7fa0c006d030 msgr2=0x7fa0c00c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 shutdown_connections 2023-10-06T05:22:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:51.867+0000 7fa0c719d700 1 -- 172.21.15.33:0/617472905 wait complete. 2023-10-06T05:22:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 73 2023-10-06T05:22:51.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.072+0000 7fbac09fe700 1 Processor -- start 2023-10-06T05:22:52.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.073+0000 7fbac09fe700 1 -- start start 2023-10-06T05:22:52.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbac09fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbac09fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbabc0adf50 0x7fbabc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbac09fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbabc0c8e40 con 0x7fbabc0c9810 2023-10-06T05:22:52.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbac09fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbabc0ae6c0 con 0x7fbabc0adf50 2023-10-06T05:22:52.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbac09fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0ae9a0 con 0x7fbabc155140 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbaba59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbabc0adf50 0x7fbabc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbaba59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbabc0adf50 0x7fbabc0c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47812/0 (socket says 172.21.15.33:47812) 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbaba59c700 1 -- 172.21.15.33:0/2707504697 learned_addr learned my addr 172.21.15.33:0/2707504697 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.074+0000 7fbabad9d700 1 --2- 172.21.15.33:0/2707504697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbaba59c700 1 -- 172.21.15.33:0/2707504697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 msgr2=0x7fbabc158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbaba59c700 1 --2- 172.21.15.33:0/2707504697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbaba59c700 1 -- 172.21.15.33:0/2707504697 >> v1:172.21.15.167:6789/0 conn(0x7fbabc0c9810 legacy=0x7fbabc0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbaba59c700 1 -- 172.21.15.33:0/2707504697 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0c8ad0 con 0x7fbabc0adf50 2023-10-06T05:22:52.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbaba59c700 1 --2- 172.21.15.33:0/2707504697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0adf50 0x7fbabc0c92d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbab00099b0 tx=0x7fbab0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd640b6870c59bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbab9d9b700 1 -- 172.21.15.33:0/2707504697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab000b860 con 0x7fbabc0adf50 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbab9d9b700 1 -- 172.21.15.33:0/2707504697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab000b9c0 con 0x7fbabc0adf50 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.075+0000 7fbab9d9b700 1 -- 172.21.15.33:0/2707504697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab000bd60 con 0x7fbabc0adf50 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 -- 172.21.15.33:0/2707504697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0adf50 msgr2=0x7fbabc0c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 --2- 172.21.15.33:0/2707504697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0adf50 0x7fbabc0c92d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbab00099b0 tx=0x7fbab0009250 comp rx=0 tx=0).stop 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 -- 172.21.15.33:0/2707504697 shutdown_connections 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 --2- 172.21.15.33:0/2707504697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 --2- 172.21.15.33:0/2707504697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0adf50 0x7fbabc0c92d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 -- 172.21.15.33:0/2707504697 >> 172.21.15.33:0/2707504697 conn(0x7fbabc06d030 msgr2=0x7fbabc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 -- 172.21.15.33:0/2707504697 shutdown_connections 2023-10-06T05:22:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.076+0000 7fbac09fe700 1 -- 172.21.15.33:0/2707504697 wait complete. 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.077+0000 7fbac09fe700 1 Processor -- start 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.078+0000 7fbac09fe700 1 -- start start 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbac09fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc0adf50 0x7fbabc15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbac09fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 0x7fbabc15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbac09fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbac09fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc152210 con 0x7fbabc0c9810 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbac09fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc151600 con 0x7fbabc0adf50 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbac09fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1518e0 con 0x7fbabc155140 2023-10-06T05:22:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 0x7fbabc15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 0x7fbabc15b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47820/0 (socket says 172.21.15.33:47820) 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 -- 172.21.15.33:0/832405215 learned_addr learned my addr 172.21.15.33:0/832405215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbabad9d700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc0adf50 0x7fbabc15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbabb59e700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 -- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 msgr2=0x7fbabc0ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc0ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 -- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc0adf50 msgr2=0x7fbabc15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc0adf50 0x7fbabc15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.079+0000 7fbaba59c700 1 -- 172.21.15.33:0/832405215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00094f0 con 0x7fbabc0c9810 2023-10-06T05:22:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.080+0000 7fbaba59c700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 0x7fbabc15b230 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbabc063610 tx=0x7fbab0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.080+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0024440 con 0x7fbabc0c9810 2023-10-06T05:22:52.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.080+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab00245c0 con 0x7fbabc0c9810 2023-10-06T05:22:52.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.080+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0024980 con 0x7fbabc0c9810 2023-10-06T05:22:52.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.080+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc151bc0 con 0x7fbabc0c9810 2023-10-06T05:22:52.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.080+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc1546d0 con 0x7fbabc0c9810 2023-10-06T05:22:52.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.081+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc04c380 con 0x7fbabc0c9810 2023-10-06T05:22:52.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.082+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbab001a420 con 0x7fbabc0c9810 2023-10-06T05:22:52.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.083+0000 7fbaa2ffd700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbaa8066f30 0x7fbaa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.083+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbab0086440 con 0x7fbabc0c9810 2023-10-06T05:22:52.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.083+0000 7fbabad9d700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbaa8066f30 0x7fbaa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.084+0000 7fbabad9d700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbaa8066f30 0x7fbaa80693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006fd0 tx=0x7fbaac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.087+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbab00534b0 con 0x7fbabc0c9810 2023-10-06T05:22:52.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.288+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbabc15c030 con 0x7fbabc0c9810 2023-10-06T05:22:52.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.289+0000 7fbaa2ffd700 1 -- 172.21.15.33:0/832405215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fbab0056ad0 con 0x7fbabc0c9810 2023-10-06T05:22:52.291 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:52.291 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:34.048391+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.698838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.505677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbaa8066f30 msgr2=0x7fbaa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbaa8066f30 0x7fbaa80693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006fd0 tx=0x7fbaac008040 comp rx=0 tx=0).stop 2023-10-06T05:22:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 msgr2=0x7fbabc15b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 0x7fbabc15b230 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbabc063610 tx=0x7fbab0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 shutdown_connections 2023-10-06T05:22:52.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbaa8066f30 0x7fbaa80693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc155140 0x7fbabc0ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc0adf50 0x7fbabc15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 --2- 172.21.15.33:0/832405215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbabc0c9810 0x7fbabc15b230 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 >> 172.21.15.33:0/832405215 conn(0x7fbabc06d030 msgr2=0x7fbabc0c26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 shutdown_connections 2023-10-06T05:22:52.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.292+0000 7fbac09fe700 1 -- 172.21.15.33:0/832405215 wait complete. 2023-10-06T05:22:52.312 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:22:52.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.497+0000 7fa29ffff700 1 Processor -- start 2023-10-06T05:22:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.497+0000 7fa29ffff700 1 -- start start 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa2a00c8050 0x7fa2a013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a00c9cb0 con 0x7fa2a006aad0 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a00ca270 con 0x7fa2a013faa0 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a006b640 con 0x7fa2a00c8050 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa2a006aad0 0x7fa2a006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56448/0 (socket says 172.21.15.33:56448) 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29f7fe700 1 -- 172.21.15.33:0/1452857348 learned_addr learned my addr 172.21.15.33:0/1452857348 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.498+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/1452857348 >> v2:172.21.15.167:3300/0 conn(0x7fa2a00c8050 0x7fa2a013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.499+0000 7fa29dffb700 1 -- 172.21.15.33:0/1452857348 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018257892 0 0) 0x7fa2a00c9cb0 con 0x7fa2a006aad0 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.499+0000 7fa29dffb700 1 -- 172.21.15.33:0/1452857348 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa288003570 con 0x7fa2a006aad0 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.499+0000 7fa29e7fc700 1 -- 172.21.15.33:0/1452857348 >> v1:172.21.15.167:6789/0 conn(0x7fa2a013faa0 legacy=0x7fa2a00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.499+0000 7fa29e7fc700 1 -- 172.21.15.33:0/1452857348 >> v1:172.21.15.33:6789/0 conn(0x7fa2a006aad0 legacy=0x7fa2a006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.499+0000 7fa29e7fc700 1 -- 172.21.15.33:0/1452857348 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a006b840 con 0x7fa2a00c8050 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.499+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/1452857348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a00c8050 0x7fa2a013fe70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa2940098e0 tx=0x7fa2940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2d713f0416fef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29dffb700 1 -- 172.21.15.33:0/1452857348 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400b640 con 0x7fa2a00c8050 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29dffb700 1 -- 172.21.15.33:0/1452857348 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400b7a0 con 0x7fa2a00c8050 2023-10-06T05:22:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29dffb700 1 -- 172.21.15.33:0/1452857348 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400bb40 con 0x7fa2a00c8050 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 -- 172.21.15.33:0/1452857348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a00c8050 msgr2=0x7fa2a013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 --2- 172.21.15.33:0/1452857348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a00c8050 0x7fa2a013fe70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa2940098e0 tx=0x7fa2940091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 -- 172.21.15.33:0/1452857348 shutdown_connections 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 --2- 172.21.15.33:0/1452857348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a00c8050 0x7fa2a013fe70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 -- 172.21.15.33:0/1452857348 >> 172.21.15.33:0/1452857348 conn(0x7fa2a006d030 msgr2=0x7fa2a006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 -- 172.21.15.33:0/1452857348 shutdown_connections 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.500+0000 7fa29ffff700 1 -- 172.21.15.33:0/1452857348 wait complete. 2023-10-06T05:22:52.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.502+0000 7fa29ffff700 1 Processor -- start 2023-10-06T05:22:52.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.502+0000 7fa29ffff700 1 -- start start 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a006aad0 0x7fa2a00b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a00c8050 0x7fa2a00c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 0x7fa2a0169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01567d0 con 0x7fa2a013faa0 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00afec0 con 0x7fa2a006aad0 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00b01a0 con 0x7fa2a00c8050 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 0x7fa2a0169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 0x7fa2a0169ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47822/0 (socket says 172.21.15.33:47822) 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 -- 172.21.15.33:0/2207078187 learned_addr learned my addr 172.21.15.33:0/2207078187 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a00c8050 0x7fa2a00c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29effd700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a006aad0 0x7fa2a00b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 -- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a00c8050 msgr2=0x7fa2a00c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a00c8050 0x7fa2a00c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 -- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a006aad0 msgr2=0x7fa2a00b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a006aad0 0x7fa2a00b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29f7fe700 1 -- 172.21.15.33:0/2207078187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294009420 con 0x7fa2a013faa0 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29e7fc700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a00c8050 0x7fa2a00c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.503+0000 7fa29effd700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a006aad0 0x7fa2a00b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.504+0000 7fa29f7fe700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 0x7fa2a0169ab0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa298001460 tx=0x7fa298001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.504+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800ce80 con 0x7fa2a013faa0 2023-10-06T05:22:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.504+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a00b0480 con 0x7fa2a013faa0 2023-10-06T05:22:52.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.504+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29800ab70 con 0x7fa2a013faa0 2023-10-06T05:22:52.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.504+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2980070f0 con 0x7fa2a013faa0 2023-10-06T05:22:52.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.504+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a00b1980 con 0x7fa2a013faa0 2023-10-06T05:22:52.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.505+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a004c380 con 0x7fa2a013faa0 2023-10-06T05:22:52.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.506+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa298007520 con 0x7fa2a013faa0 2023-10-06T05:22:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.507+0000 7fa286ffd700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.507+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa298084bb0 con 0x7fa2a013faa0 2023-10-06T05:22:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.507+0000 7fa29effd700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.508+0000 7fa29effd700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00aa620 tx=0x7fa290009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.511+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa298051cb0 con 0x7fa2a013faa0 2023-10-06T05:22:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.758+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2a006bb20 con 0x7fa2a013faa0 2023-10-06T05:22:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.758+0000 7fa286ffd700 1 -- 172.21.15.33:0/2207078187 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fa2980552d0 con 0x7fa2a013faa0 2023-10-06T05:22:52.760 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:52.760 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.538408+0000","modified":"2023-10-06T05:22:36.061558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2226881352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2226881352},{"type":"v1","addr":"172.21.15.33:6835","nonce":2226881352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:22:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.760+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 msgr2=0x7fa28c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.760+0000 7fa29ffff700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00aa620 tx=0x7fa290009040 comp rx=0 tx=0).stop 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 msgr2=0x7fa2a0169ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 0x7fa2a0169ab0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa298001460 tx=0x7fa298001ae0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 shutdown_connections 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a013faa0 0x7fa2a0169ab0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a006aad0 0x7fa2a00b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 --2- 172.21.15.33:0/2207078187 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2a00c8050 0x7fa2a00c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 >> 172.21.15.33:0/2207078187 conn(0x7fa2a006d030 msgr2=0x7fa2a006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 shutdown_connections 2023-10-06T05:22:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.761+0000 7fa29ffff700 1 -- 172.21.15.33:0/2207078187 wait complete. 2023-10-06T05:22:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 73 2023-10-06T05:22:52.780 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:52.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.966+0000 7f4c8d59d700 1 Processor -- start 2023-10-06T05:22:52.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.966+0000 7f4c8d59d700 1 -- start start 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8d59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0a40 0x7f4c8806ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8d59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c8815a0a0 0x7f4c8815a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8d59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8815e2d0 con 0x7f4c880b0e10 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8d59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8815d6c0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8d59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8815d9a0 con 0x7f4c880b0a40 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8cd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c8815a0a0 0x7f4c8815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8cd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c8815a0a0 0x7f4c8815a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47834/0 (socket says 172.21.15.33:47834) 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/1372772858 learned_addr learned my addr 172.21.15.33:0/1372772858 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.967+0000 7f4c877fe700 1 --1- 172.21.15.33:0/1372772858 >> v1:172.21.15.33:6789/0 conn(0x7f4c880b0e10 0x7f4c880b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56464/0 (socket says 172.21.15.33:56464) 2023-10-06T05:22:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c87fff700 1 --2- 172.21.15.33:0/1372772858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0a40 0x7f4c8806ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/1372772858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0a40 msgr2=0x7f4c8806ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c8cd9c700 1 --2- 172.21.15.33:0/1372772858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0a40 0x7f4c8806ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/1372772858 >> v1:172.21.15.33:6789/0 conn(0x7f4c880b0e10 legacy=0x7f4c880b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/1372772858 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8815dc80 con 0x7f4c8815a0a0 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c86ffd700 1 -- 172.21.15.33:0/1372772858 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657455165 0 0) 0x7f4c8815e2d0 con 0x7f4c880b0e10 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c8cd9c700 1 --2- 172.21.15.33:0/1372772858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c8815a520 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009f20 tx=0x7f4c80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccfaf9ddae75f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c86ffd700 1 -- 172.21.15.33:0/1372772858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000e070 con 0x7f4c8815a0a0 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c86ffd700 1 -- 172.21.15.33:0/1372772858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000b9d0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.968+0000 7f4c86ffd700 1 -- 172.21.15.33:0/1372772858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800092e0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 -- 172.21.15.33:0/1372772858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 msgr2=0x7f4c8815a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/1372772858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c8815a520 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009f20 tx=0x7f4c80009840 comp rx=0 tx=0).stop 2023-10-06T05:22:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 -- 172.21.15.33:0/1372772858 shutdown_connections 2023-10-06T05:22:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/1372772858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c8815a520 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/1372772858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0a40 0x7f4c8806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 -- 172.21.15.33:0/1372772858 >> 172.21.15.33:0/1372772858 conn(0x7f4c8806e0f0 msgr2=0x7f4c880675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 -- 172.21.15.33:0/1372772858 shutdown_connections 2023-10-06T05:22:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.969+0000 7f4c8d59d700 1 -- 172.21.15.33:0/1372772858 wait complete. 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.970+0000 7f4c8d59d700 1 Processor -- start 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.971+0000 7f4c8d59d700 1 -- start start 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.971+0000 7f4c8d59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880b0a40 0x7f4c88169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.971+0000 7f4c8d59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0e10 0x7f4c880ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8d59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c880c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8d59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880ac6b0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8d59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880c3380 con 0x7f4c880b0a40 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8d59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88169cb0 con 0x7f4c880b0e10 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c880c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c880c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47842/0 (socket says 172.21.15.33:47842) 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/2391509463 learned_addr learned my addr 172.21.15.33:0/2391509463 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c87fff700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880b0a40 0x7f4c88169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c877fe700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0e10 0x7f4c880ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0e10 msgr2=0x7f4c880ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0e10 0x7f4c880ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880b0a40 msgr2=0x7f4c88169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880b0a40 0x7f4c88169770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c8cd9c700 1 -- 172.21.15.33:0/2391509463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009ae0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.972+0000 7f4c877fe700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0e10 0x7f4c880ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.973+0000 7f4c8cd9c700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c880c4f10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000b610 tx=0x7f4c80018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.973+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80008b80 con 0x7f4c8815a0a0 2023-10-06T05:22:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.973+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80008d00 con 0x7f4c8815a0a0 2023-10-06T05:22:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.973+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800256b0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.973+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880ac0f0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.973+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c880c3010 con 0x7f4c8815a0a0 2023-10-06T05:22:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.974+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8806fcc0 con 0x7f4c8815a0a0 2023-10-06T05:22:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.975+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c80003040 con 0x7f4c8815a0a0 2023-10-06T05:22:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.976+0000 7f4c6ffff700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c74066f30 0x7f4c740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.976+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c80086290 con 0x7f4c8815a0a0 2023-10-06T05:22:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.976+0000 7f4c87fff700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c74066f30 0x7f4c740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.977+0000 7f4c87fff700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c74066f30 0x7f4c740693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009db0 tx=0x7f4c780091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:52.980+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c80053300 con 0x7f4c8815a0a0 2023-10-06T05:22:53.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.227+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c880c3510 con 0x7f4c8815a0a0 2023-10-06T05:22:53.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.227+0000 7f4c6ffff700 1 -- 172.21.15.33:0/2391509463 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f4c80056920 con 0x7f4c8815a0a0 2023-10-06T05:22:53.229 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:53.230 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.538408+0000","modified":"2023-10-06T05:22:36.061558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7299":{"gid":7299,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2226881352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2226881352},{"type":"v1","addr":"172.21.15.33:6835","nonce":2226881352}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:22:53.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.229+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c74066f30 msgr2=0x7f4c740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.229+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c74066f30 0x7f4c740693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009db0 tx=0x7f4c780091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.229+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 msgr2=0x7f4c880c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.229+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c880c4f10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000b610 tx=0x7f4c80018000 comp rx=0 tx=0).stop 2023-10-06T05:22:53.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 shutdown_connections 2023-10-06T05:22:53.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c74066f30 0x7f4c740693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c8815a0a0 0x7f4c880c4f10 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880b0a40 0x7f4c88169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 --2- 172.21.15.33:0/2391509463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b0e10 0x7f4c880ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 >> 172.21.15.33:0/2391509463 conn(0x7f4c8806e0f0 msgr2=0x7f4c88158120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 shutdown_connections 2023-10-06T05:22:53.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.230+0000 7f4c8d59d700 1 -- 172.21.15.33:0/2391509463 wait complete. 2023-10-06T05:22:53.235 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 73 2023-10-06T05:22:53.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.434+0000 7f30ff59e700 1 Processor -- start 2023-10-06T05:22:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.435+0000 7f30ff59e700 1 -- start start 2023-10-06T05:22:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30ff59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31000c8050 0x7f310013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30ff59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31000c9cb0 con 0x7f310006aad0 2023-10-06T05:22:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30ff59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31000ca270 con 0x7f310013faa0 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30ff59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310006b640 con 0x7f31000c8050 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30fdd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31000c8050 0x7f310013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30fdd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31000c8050 0x7f310013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47858/0 (socket says 172.21.15.33:47858) 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.436+0000 7f30fdd9b700 1 -- 172.21.15.33:0/2732125764 learned_addr learned my addr 172.21.15.33:0/2732125764 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fdd9b700 1 -- 172.21.15.33:0/2732125764 >> v1:172.21.15.167:6789/0 conn(0x7f310013faa0 legacy=0x7f31000c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fdd9b700 1 -- 172.21.15.33:0/2732125764 >> v1:172.21.15.33:6789/0 conn(0x7f310006aad0 legacy=0x7f310006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fdd9b700 1 -- 172.21.15.33:0/2732125764 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310006b840 con 0x7f31000c8050 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fd59a700 1 -- 172.21.15.33:0/2732125764 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502795770 0 0) 0x7f31000c9cb0 con 0x7f310006aad0 2023-10-06T05:22:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fdd9b700 1 --2- 172.21.15.33:0/2732125764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8050 0x7f310013fe70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f30f40099b0 tx=0x7f30f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f866ca3ae73676fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fd59a700 1 -- 172.21.15.33:0/2732125764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400b860 con 0x7f31000c8050 2023-10-06T05:22:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fd59a700 1 -- 172.21.15.33:0/2732125764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f400b9c0 con 0x7f31000c8050 2023-10-06T05:22:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.437+0000 7f30fd59a700 1 -- 172.21.15.33:0/2732125764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400bd60 con 0x7f31000c8050 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 -- 172.21.15.33:0/2732125764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8050 msgr2=0x7f310013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 --2- 172.21.15.33:0/2732125764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8050 0x7f310013fe70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f30f40099b0 tx=0x7f30f4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 -- 172.21.15.33:0/2732125764 shutdown_connections 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 --2- 172.21.15.33:0/2732125764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8050 0x7f310013fe70 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 -- 172.21.15.33:0/2732125764 >> 172.21.15.33:0/2732125764 conn(0x7f310006d030 msgr2=0x7f310006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 -- 172.21.15.33:0/2732125764 shutdown_connections 2023-10-06T05:22:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.438+0000 7f30ff59e700 1 -- 172.21.15.33:0/2732125764 wait complete. 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.439+0000 7f30ff59e700 1 Processor -- start 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.439+0000 7f30ff59e700 1 -- start start 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.440+0000 7f30ff59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f310006aad0 0x7f31000adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.440+0000 7f30ff59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c8050 0x7f31000c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.440+0000 7f30ff59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 0x7f310013df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.440+0000 7f30ff59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31000c6950 con 0x7f310013faa0 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.440+0000 7f30ff59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31000c47b0 con 0x7f31000c8050 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.440+0000 7f30ff59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31000ae150 con 0x7f310006aad0 2023-10-06T05:22:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 0x7f310013df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 0x7f310013df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47860/0 (socket says 172.21.15.33:47860) 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 -- 172.21.15.33:0/1196972282 learned_addr learned my addr 172.21.15.33:0/1196972282 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fe59c700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f310006aad0 0x7f31000adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fdd9b700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c8050 0x7f31000c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 -- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f310006aad0 msgr2=0x7f31000adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f310006aad0 0x7f31000adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 -- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c8050 msgr2=0x7f31000c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c8050 0x7f31000c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 -- 172.21.15.33:0/1196972282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40094f0 con 0x7f310013faa0 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.441+0000 7f30fed9d700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 0x7f310013df60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f30f8001560 tx=0x7f30f8001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.442+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800a780 con 0x7f310013faa0 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.442+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f800a8e0 con 0x7f310013faa0 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.442+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800aca0 con 0x7f310013faa0 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.442+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31000ae3f0 con 0x7f310013faa0 2023-10-06T05:22:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.442+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31000c4160 con 0x7f310013faa0 2023-10-06T05:22:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.443+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31000c4940 con 0x7f310013faa0 2023-10-06T05:22:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.444+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30f8012630 con 0x7f310013faa0 2023-10-06T05:22:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.445+0000 7f30ee7fc700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e0066f30 0x7f30e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.445+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f30f8084a80 con 0x7f310013faa0 2023-10-06T05:22:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.446+0000 7f30fe59c700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e0066f30 0x7f30e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.446+0000 7f30fe59c700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e0066f30 0x7f30e00693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f30f00037c0 tx=0x7f30f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.449+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30f8051af0 con 0x7f310013faa0 2023-10-06T05:22:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.654+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f310004c380 con 0x7f310013faa0 2023-10-06T05:22:53.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.655+0000 7f30ee7fc700 1 -- 172.21.15.33:0/1196972282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f30f8055110 con 0x7f310013faa0 2023-10-06T05:22:53.657 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:22:53.657 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:22:34.048391+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.698838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.505677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:53.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.657+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e0066f30 msgr2=0x7f30e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e0066f30 0x7f30e00693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f30f00037c0 tx=0x7f30f0009040 comp rx=0 tx=0).stop 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 msgr2=0x7f310013df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 0x7f310013df60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f30f8001560 tx=0x7f30f8001770 comp rx=0 tx=0).stop 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 shutdown_connections 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e0066f30 0x7f30e00693e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310013faa0 0x7f310013df60 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f310006aad0 0x7f31000adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 --2- 172.21.15.33:0/1196972282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c8050 0x7f31000c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 >> 172.21.15.33:0/1196972282 conn(0x7f310006d030 msgr2=0x7f310006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 shutdown_connections 2023-10-06T05:22:53.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.658+0000 7f30ff59e700 1 -- 172.21.15.33:0/1196972282 wait complete. 2023-10-06T05:22:53.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:53.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.866+0000 7fa1e1f9d700 1 Processor -- start 2023-10-06T05:22:53.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.866+0000 7fa1e1f9d700 1 -- start start 2023-10-06T05:22:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1e1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1e1f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0b03a0 con 0x7fa1dc06c560 2023-10-06T05:22:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1e1f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0b0f40 con 0x7fa1dc1592e0 2023-10-06T05:22:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1e1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc06ccd0 con 0x7fa1dc1596b0 2023-10-06T05:22:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1e179c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa1dc06c560 0x7fa1dc0aeca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56476/0 (socket says 172.21.15.33:56476) 2023-10-06T05:22:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1e179c700 1 -- 172.21.15.33:0/569804153 learned_addr learned my addr 172.21.15.33:0/569804153 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/569804153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688561512 0 0) 0x7fa1dc0b03a0 con 0x7fa1dc06c560 2023-10-06T05:22:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.868+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1cc003570 con 0x7fa1dc06c560 2023-10-06T05:22:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933339034 0 0) 0x7fa1dc0b0f40 con 0x7fa1dc1592e0 2023-10-06T05:22:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1dc0b03a0 con 0x7fa1dc1592e0 2023-10-06T05:22:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1dbfff700 1 -- 172.21.15.33:0/569804153 >> v1:172.21.15.167:6789/0 conn(0x7fa1dc1592e0 legacy=0x7fa1dc0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1dbfff700 1 -- 172.21.15.33:0/569804153 >> v1:172.21.15.33:6789/0 conn(0x7fa1dc06c560 legacy=0x7fa1dc0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1dbfff700 1 -- 172.21.15.33:0/569804153 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc06cfb0 con 0x7fa1dc1596b0 2023-10-06T05:22:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/569804153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0cdd30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009910 tx=0x7fa1c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b51b075b306bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8011630 con 0x7fa1dc1596b0 2023-10-06T05:22:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c8011790 con 0x7fa1dc1596b0 2023-10-06T05:22:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.869+0000 7fa1db7fe700 1 -- 172.21.15.33:0/569804153 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8011b30 con 0x7fa1dc1596b0 2023-10-06T05:22:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/569804153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 msgr2=0x7fa1dc0cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/569804153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0cdd30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009910 tx=0x7fa1c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/569804153 shutdown_connections 2023-10-06T05:22:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/569804153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0cdd30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/569804153 >> 172.21.15.33:0/569804153 conn(0x7fa1dc0675f0 msgr2=0x7fa1dc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/569804153 shutdown_connections 2023-10-06T05:22:53.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.870+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/569804153 wait complete. 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.872+0000 7fa1e1f9d700 1 Processor -- start 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.872+0000 7fa1e1f9d700 1 -- start start 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e1f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 0x7fa1dc151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e1f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1dc1592e0 0x7fa1dc169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e1f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cb290 con 0x7fa1dc06c560 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e1f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc155f60 con 0x7fa1dc1592e0 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc156240 con 0x7fa1dc1596b0 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 0x7fa1dc151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 0x7fa1dc151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47868/0 (socket says 172.21.15.33:47868) 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 -- 172.21.15.33:0/176095975 learned_addr learned my addr 172.21.15.33:0/176095975 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1dc1592e0 0x7fa1dc169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e179c700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 -- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 msgr2=0x7fa1dc0ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 -- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1dc1592e0 msgr2=0x7fa1dc169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1dc1592e0 0x7fa1dc169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1e0f9b700 1 -- 172.21.15.33:0/176095975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c8009420 con 0x7fa1dc06c560 2023-10-06T05:22:53.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.873+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1dc1592e0 0x7fa1dc169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.874+0000 7fa1e0f9b700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 0x7fa1dc151d80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c790 tx=0x7fa1d0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.874+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0010510 con 0x7fa1dc06c560 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.874+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0010670 con 0x7fa1dc06c560 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.874+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc156520 con 0x7fa1dc06c560 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.874+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0010a10 con 0x7fa1dc06c560 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.874+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc1522c0 con 0x7fa1dc06c560 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.875+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc0657b0 con 0x7fa1dc06c560 2023-10-06T05:22:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.876+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1d001b400 con 0x7fa1dc06c560 2023-10-06T05:22:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.877+0000 7fa1d8ff9700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1c0066f30 0x7fa1c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.877+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1d00844e0 con 0x7fa1dc06c560 2023-10-06T05:22:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.878+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1c0066f30 0x7fa1c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.879+0000 7fa1dbfff700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1c0066f30 0x7fa1c00693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009860 tx=0x7fa1c8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:53.882+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1d0051550 con 0x7fa1dc06c560 2023-10-06T05:22:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.132+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa1dc04bba0 con 0x7fa1dc06c560 2023-10-06T05:22:54.134 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:22:54.133+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:54.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.138+0000 7fa1d8ff9700 1 -- 172.21.15.33:0/176095975 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa1d0054b70 con 0x7fa1dc06c560 2023-10-06T05:22:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1c0066f30 msgr2=0x7fa1c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1c0066f30 0x7fa1c00693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009860 tx=0x7fa1c8017000 comp rx=0 tx=0).stop 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 msgr2=0x7fa1dc151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 0x7fa1dc151d80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c790 tx=0x7fa1d0007e10 comp rx=0 tx=0).stop 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 shutdown_connections 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1c0066f30 0x7fa1c00693e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1dc1596b0 0x7fa1dc0ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1dc06c560 0x7fa1dc151d80 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 --2- 172.21.15.33:0/176095975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1dc1592e0 0x7fa1dc169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 >> 172.21.15.33:0/176095975 conn(0x7fa1dc0675f0 msgr2=0x7fa1dc153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 shutdown_connections 2023-10-06T05:22:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.141+0000 7fa1e1f9d700 1 -- 172.21.15.33:0/176095975 wait complete. 2023-10-06T05:22:54.144 INFO:tasks.ceph.mds.a.smithi033.stderr: -799> 2023-10-06T05:22:37.014+0000 7f5e101f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:54.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:22:54.180 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:22:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.350+0000 7f845759e700 1 Processor -- start 2023-10-06T05:22:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.350+0000 7f845759e700 1 -- start start 2023-10-06T05:22:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.351+0000 7f845759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8458159850 0x7f845806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.351+0000 7f845759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f845806d140 0x7f845806edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.351+0000 7f845759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84580b0cd0 con 0x7f845815acd0 2023-10-06T05:22:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.351+0000 7f845759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845815a840 con 0x7f8458159850 2023-10-06T05:22:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.351+0000 7f845759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8458159f10 con 0x7f845806d140 2023-10-06T05:22:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f8455d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f845815acd0 0x7f845806d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56480/0 (socket says 172.21.15.33:56480) 2023-10-06T05:22:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f8455d9b700 1 -- 172.21.15.33:0/400538011 learned_addr learned my addr 172.21.15.33:0/400538011 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f8456d9d700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f845806d140 0x7f845806edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845659c700 1 --2- 172.21.15.33:0/400538011 >> v2:172.21.15.167:3300/0 conn(0x7f8458159850 0x7f845806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845559a700 1 -- 172.21.15.33:0/400538011 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826810350 0 0) 0x7f84580b0cd0 con 0x7f845815acd0 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845559a700 1 -- 172.21.15.33:0/400538011 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8440003570 con 0x7f845815acd0 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845659c700 1 -- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f845806d140 msgr2=0x7f845806edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845659c700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f845806d140 0x7f845806edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845659c700 1 -- 172.21.15.33:0/400538011 >> v1:172.21.15.33:6789/0 conn(0x7f845815acd0 legacy=0x7f845806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f845659c700 1 -- 172.21.15.33:0/400538011 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845815a1f0 con 0x7f8458159850 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.352+0000 7f8456d9d700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f845806d140 0x7f845806edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.353+0000 7f845659c700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8458159850 0x7f845806e890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84480098e0 tx=0x7f84480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99522b3ddde3a2bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.353+0000 7f845559a700 1 -- 172.21.15.33:0/400538011 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448011630 con 0x7f8458159850 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.353+0000 7f845559a700 1 -- 172.21.15.33:0/400538011 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8448011790 con 0x7f8458159850 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.353+0000 7f845559a700 1 -- 172.21.15.33:0/400538011 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448011b30 con 0x7f8458159850 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 -- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8458159850 msgr2=0x7f845806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8458159850 0x7f845806e890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84480098e0 tx=0x7f84480091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 -- 172.21.15.33:0/400538011 shutdown_connections 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8458159850 0x7f845806e890 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 --2- 172.21.15.33:0/400538011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f845806d140 0x7f845806edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 -- 172.21.15.33:0/400538011 >> 172.21.15.33:0/400538011 conn(0x7f845806cd40 msgr2=0x7f84580ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 -- 172.21.15.33:0/400538011 shutdown_connections 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.354+0000 7f845759e700 1 -- 172.21.15.33:0/400538011 wait complete. 2023-10-06T05:22:54.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.355+0000 7f845759e700 1 Processor -- start 2023-10-06T05:22:54.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.355+0000 7f845759e700 1 -- start start 2023-10-06T05:22:54.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f845759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f845806d140 0x7f845815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f845759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8458159850 0x7f84580caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f845759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 0x7f84580cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f845759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b28f0 con 0x7f845815acd0 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f845759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845815dbc0 con 0x7f845806d140 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f845759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580cbd20 con 0x7f8458159850 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f8456d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 0x7f84580cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f8456d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 0x7f84580cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47888/0 (socket says 172.21.15.33:47888) 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.356+0000 7f8456d9d700 1 -- 172.21.15.33:0/589566408 learned_addr learned my addr 172.21.15.33:0/589566408 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f845659c700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f845806d140 0x7f845815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8455d9b700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8458159850 0x7f84580caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8456d9d700 1 -- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8458159850 msgr2=0x7f84580caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8456d9d700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8458159850 0x7f84580caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8456d9d700 1 -- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f845806d140 msgr2=0x7f845815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8456d9d700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f845806d140 0x7f845815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8456d9d700 1 -- 172.21.15.33:0/589566408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450009a20 con 0x7f845815acd0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f8456d9d700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 0x7f84580cb4b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8450009f60 tx=0x7f845000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000cb70 con 0x7f845815acd0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845000f040 con 0x7f845815acd0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450014480 con 0x7f845815acd0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.357+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8448009420 con 0x7f845815acd0 2023-10-06T05:22:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.358+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84580b2580 con 0x7f845815acd0 2023-10-06T05:22:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.359+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84580b2aa0 con 0x7f845815acd0 2023-10-06T05:22:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.360+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84500147f0 con 0x7f845815acd0 2023-10-06T05:22:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.361+0000 7f84467fc700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8438066f30 0x7f84380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.361+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8450085370 con 0x7f845815acd0 2023-10-06T05:22:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.361+0000 7f845659c700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8438066f30 0x7f84380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.362+0000 7f845659c700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8438066f30 0x7f84380693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f84480091e0 tx=0x7f8448017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:54.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.365+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84580b2aa0 con 0x7f845815acd0 2023-10-06T05:22:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:54.613+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f84580b2a30 con 0x7f845815acd0 2023-10-06T05:22:55.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.139+0000 7f84467fc700 1 -- 172.21.15.33:0/589566408 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8450052430 con 0x7f845815acd0 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8438066f30 msgr2=0x7f84380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8438066f30 0x7f84380693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f84480091e0 tx=0x7f8448017000 comp rx=0 tx=0).stop 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 msgr2=0x7f84580cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 0x7f84580cb4b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8450009f60 tx=0x7f845000b040 comp rx=0 tx=0).stop 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 shutdown_connections 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8438066f30 0x7f84380693e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f845815acd0 0x7f84580cb4b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f845806d140 0x7f845815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 --2- 172.21.15.33:0/589566408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8458159850 0x7f84580caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 >> 172.21.15.33:0/589566408 conn(0x7f845806cd40 msgr2=0x7f84580aae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 shutdown_connections 2023-10-06T05:22:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.142+0000 7f845759e700 1 -- 172.21.15.33:0/589566408 wait complete. 2023-10-06T05:22:55.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:22:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.397+0000 7fc20eb9d700 1 Processor -- start 2023-10-06T05:22:55.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.397+0000 7fc20eb9d700 1 -- start start 2023-10-06T05:22:55.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.398+0000 7fc20eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 0x7fc20806c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.398+0000 7fc20eb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2080b0520 0x7fc20815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.398+0000 7fc20eb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080b1fe0 con 0x7fc2080b2da0 2023-10-06T05:22:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.398+0000 7fc20eb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080b22c0 con 0x7fc2080b0520 2023-10-06T05:22:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.398+0000 7fc20eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080b25a0 con 0x7fc2080b0150 2023-10-06T05:22:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2080b0520 0x7fc20815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20db9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 0x7fc20806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20db9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 0x7fc20806c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:36322/0 (socket says 172.21.15.33:36322) 2023-10-06T05:22:55.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2080b0520 0x7fc20815a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:42244/0 (socket says 172.21.15.33:42244) 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20db9b700 1 -- 172.21.15.33:0/1891218834 learned_addr learned my addr 172.21.15.33:0/1891218834 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 -- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 msgr2=0x7fc20806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 --2- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 0x7fc20806c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 -- 172.21.15.33:0/1891218834 >> v1:172.21.15.167:6789/0 conn(0x7fc2080b2da0 legacy=0x7fc20815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 -- 172.21.15.33:0/1891218834 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080b2880 con 0x7fc2080b0520 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20db9b700 1 --2- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 0x7fc20806c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.399+0000 7fc20e39c700 1 --2- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 0x7fc20815a7e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc200009e60 tx=0x7fc200009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8070c7235682c868 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20cb99700 1 -- 172.21.15.33:0/1891218834 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20000e070 con 0x7fc2080b0520 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20cb99700 1 -- 172.21.15.33:0/1891218834 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20000bb60 con 0x7fc2080b0520 2023-10-06T05:22:55.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20cb99700 1 -- 172.21.15.33:0/1891218834 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200012430 con 0x7fc2080b0520 2023-10-06T05:22:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20eb9d700 1 -- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 msgr2=0x7fc20815a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 0x7fc20815a7e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc200009e60 tx=0x7fc200009780 comp rx=0 tx=0).stop 2023-10-06T05:22:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20eb9d700 1 -- 172.21.15.33:0/1891218834 shutdown_connections 2023-10-06T05:22:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 0x7fc20815a7e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/1891218834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b0150 0x7fc20806c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.400+0000 7fc20eb9d700 1 -- 172.21.15.33:0/1891218834 >> 172.21.15.33:0/1891218834 conn(0x7fc20806d670 msgr2=0x7fc20806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.401+0000 7fc20eb9d700 1 -- 172.21.15.33:0/1891218834 shutdown_connections 2023-10-06T05:22:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.401+0000 7fc20eb9d700 1 -- 172.21.15.33:0/1891218834 wait complete. 2023-10-06T05:22:55.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.402+0000 7fc20eb9d700 1 Processor -- start 2023-10-06T05:22:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.402+0000 7fc20eb9d700 1 -- start start 2023-10-06T05:22:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 0x7fc2080990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 0x7fc20816d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b2da0 0x7fc2080cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080cda30 con 0x7fc2080b0150 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208153130 con 0x7fc2080b0520 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080996a0 con 0x7fc2080b2da0 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 0x7fc2080990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 0x7fc2080990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47900/0 (socket says 172.21.15.33:47900) 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.403+0000 7fc20db9b700 1 -- 172.21.15.33:0/4020632990 learned_addr learned my addr 172.21.15.33:0/4020632990 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20db9b700 1 -- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b2da0 msgr2=0x7fc2080cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20db9b700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b2da0 0x7fc2080cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20db9b700 1 -- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 msgr2=0x7fc20816d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20db9b700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 0x7fc20816d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20db9b700 1 -- 172.21.15.33:0/4020632990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040094f0 con 0x7fc2080b0150 2023-10-06T05:22:55.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20db9b700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 0x7fc2080990c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ec50 tx=0x7fc204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20400c9d0 con 0x7fc2080b0150 2023-10-06T05:22:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20400cb30 con 0x7fc2080b0150 2023-10-06T05:22:55.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204013470 con 0x7fc2080b0150 2023-10-06T05:22:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc200009a20 con 0x7fc2080b0150 2023-10-06T05:22:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.404+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2080caeb0 con 0x7fc2080b0150 2023-10-06T05:22:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.406+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20804c380 con 0x7fc2080b0150 2023-10-06T05:22:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.406+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc204013860 con 0x7fc2080b0150 2023-10-06T05:22:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.407+0000 7fc1fdffb700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1ec066f30 0x7fc1ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:55.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.407+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc204084ed0 con 0x7fc2080b0150 2023-10-06T05:22:55.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.408+0000 7fc20d39a700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1ec066f30 0x7fc1ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:55.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.408+0000 7fc20d39a700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1ec066f30 0x7fc1ec0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc2080983d0 tx=0x7fc1f8007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:55.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.411+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc204051f90 con 0x7fc2080b0150 2023-10-06T05:22:55.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:55.616+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc2080b1c90 con 0x7fc2080b0150 2023-10-06T05:22:56.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.141+0000 7fc1fdffb700 1 -- 172.21.15.33:0/4020632990 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc2040555b0 con 0x7fc2080b0150 2023-10-06T05:22:56.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1ec066f30 msgr2=0x7fc1ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1ec066f30 0x7fc1ec0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc2080983d0 tx=0x7fc1f8007b40 comp rx=0 tx=0).stop 2023-10-06T05:22:56.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 msgr2=0x7fc2080990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 0x7fc2080990c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ec50 tx=0x7fc204009250 comp rx=0 tx=0).stop 2023-10-06T05:22:56.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 shutdown_connections 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1ec066f30 0x7fc1ec0693e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2080b2da0 0x7fc2080cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080b0150 0x7fc2080990c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 --2- 172.21.15.33:0/4020632990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2080b0520 0x7fc20816d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.144+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 >> 172.21.15.33:0/4020632990 conn(0x7fc20806d670 msgr2=0x7fc2080acde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.145+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 shutdown_connections 2023-10-06T05:22:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.145+0000 7fc20eb9d700 1 -- 172.21.15.33:0/4020632990 wait complete. 2023-10-06T05:22:56.148 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:22:56.166 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:22:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.363+0000 7fbdd539d700 1 Processor -- start 2023-10-06T05:22:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.364+0000 7fbdd539d700 1 -- start start 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd539d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd00c9720 0x7fbdd00ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd539d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdd015dc50 0x7fbdd00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd539d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd00afc40 con 0x7fbdd006c560 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd539d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd015d7c0 con 0x7fbdd015dc50 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd539d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd0159bc0 con 0x7fbdd00c9720 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdcf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdd015dc50 0x7fbdd00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd4b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbdd006c560 0x7fbdd006ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56494/0 (socket says 172.21.15.33:56494) 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdd4b9c700 1 -- 172.21.15.33:0/813720005 learned_addr learned my addr 172.21.15.33:0/813720005 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.365+0000 7fbdcffff700 1 --2- 172.21.15.33:0/813720005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd00c9720 0x7fbdd00ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/813720005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd00c9720 msgr2=0x7fbdd00ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdcf7fe700 1 --2- 172.21.15.33:0/813720005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd00c9720 0x7fbdd00ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/813720005 >> v1:172.21.15.33:6789/0 conn(0x7fbdd006c560 legacy=0x7fbdd006ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/813720005 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00af680 con 0x7fbdd015dc50 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdceffd700 1 -- 172.21.15.33:0/813720005 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456290413 0 0) 0x7fbdd00afc40 con 0x7fbdd006c560 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdcf7fe700 1 --2- 172.21.15.33:0/813720005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd015dc50 0x7fbdd00aeda0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40098e0 tx=0x7fbdc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91590a9b142cc270 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdceffd700 1 -- 172.21.15.33:0/813720005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4009e80 con 0x7fbdd015dc50 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdceffd700 1 -- 172.21.15.33:0/813720005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc400b860 con 0x7fbdd015dc50 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.366+0000 7fbdceffd700 1 -- 172.21.15.33:0/813720005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400bbe0 con 0x7fbdd015dc50 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 -- 172.21.15.33:0/813720005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd015dc50 msgr2=0x7fbdd00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 --2- 172.21.15.33:0/813720005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd015dc50 0x7fbdd00aeda0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40098e0 tx=0x7fbdc40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 -- 172.21.15.33:0/813720005 shutdown_connections 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 --2- 172.21.15.33:0/813720005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd00c9720 0x7fbdd00ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 --2- 172.21.15.33:0/813720005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd015dc50 0x7fbdd00aeda0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 -- 172.21.15.33:0/813720005 >> 172.21.15.33:0/813720005 conn(0x7fbdd00675f0 msgr2=0x7fbdd00679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 -- 172.21.15.33:0/813720005 shutdown_connections 2023-10-06T05:22:56.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.367+0000 7fbdd539d700 1 -- 172.21.15.33:0/813720005 wait complete. 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.368+0000 7fbdd539d700 1 Processor -- start 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.369+0000 7fbdd539d700 1 -- start start 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.369+0000 7fbdd539d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdd006c560 0x7fbdd0171fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.369+0000 7fbdd539d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 0x7fbdd016d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.369+0000 7fbdd539d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd015dc50 0x7fbdd00b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.369+0000 7fbdd539d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd015a4a0 con 0x7fbdd00c9720 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdd539d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd014a080 con 0x7fbdd006c560 2023-10-06T05:22:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdd539d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd015c780 con 0x7fbdd015dc50 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 0x7fbdd016d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 0x7fbdd016d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47914/0 (socket says 172.21.15.33:47914) 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/93111473 learned_addr learned my addr 172.21.15.33:0/93111473 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcffff700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdd006c560 0x7fbdd0171fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdd4b9c700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd015dc50 0x7fbdd00b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd015dc50 msgr2=0x7fbdd00b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd015dc50 0x7fbdd00b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdd006c560 msgr2=0x7fbdd0171fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdd006c560 0x7fbdd0171fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdcf7fe700 1 -- 172.21.15.33:0/93111473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4009420 con 0x7fbdd00c9720 2023-10-06T05:22:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.370+0000 7fbdd4b9c700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd015dc50 0x7fbdd00b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:56.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.371+0000 7fbdcf7fe700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 0x7fbdd016d700 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbdd006ece0 tx=0x7fbdc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.371+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400b9d0 con 0x7fbdd00c9720 2023-10-06T05:22:56.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.371+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc400bb50 con 0x7fbdd00c9720 2023-10-06T05:22:56.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.371+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4025670 con 0x7fbdd00c9720 2023-10-06T05:22:56.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.371+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd015ca60 con 0x7fbdd00c9720 2023-10-06T05:22:56.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.371+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd01530f0 con 0x7fbdd00c9720 2023-10-06T05:22:56.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.372+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd0153340 con 0x7fbdd00c9720 2023-10-06T05:22:56.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.373+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdc4023070 con 0x7fbdd00c9720 2023-10-06T05:22:56.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.374+0000 7fbdb7fff700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdbc066f30 0x7fbdbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.374+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbdc4085b40 con 0x7fbdd00c9720 2023-10-06T05:22:56.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.374+0000 7fbdcffff700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdbc066f30 0x7fbdbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.375+0000 7fbdcffff700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdbc066f30 0x7fbdbc0693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006fd0 tx=0x7fbdc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.378+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdc4052d10 con 0x7fbdd00c9720 2023-10-06T05:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:56.579+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbdd0063ff0 con 0x7fbdd00c9720 2023-10-06T05:22:57.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.147+0000 7fbdb7fff700 1 -- 172.21.15.33:0/93111473 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbdc4056330 con 0x7fbdd00c9720 2023-10-06T05:22:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdbc066f30 msgr2=0x7fbdbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdbc066f30 0x7fbdbc0693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006fd0 tx=0x7fbdc0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 msgr2=0x7fbdd016d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 0x7fbdd016d700 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbdd006ece0 tx=0x7fbdc4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 shutdown_connections 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdbc066f30 0x7fbdbc0693e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdd015dc50 0x7fbdd00b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdd006c560 0x7fbdd0171fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 --2- 172.21.15.33:0/93111473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdd00c9720 0x7fbdd016d700 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.152+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 >> 172.21.15.33:0/93111473 conn(0x7fbdd00675f0 msgr2=0x7fbdd0157920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.153+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 shutdown_connections 2023-10-06T05:22:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.153+0000 7fbdd539d700 1 -- 172.21.15.33:0/93111473 wait complete. 2023-10-06T05:22:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:22:57.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:22:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.374+0000 7f1ab6d9d700 1 Processor -- start 2023-10-06T05:22:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.374+0000 7f1ab6d9d700 1 -- start start 2023-10-06T05:22:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.375+0000 7f1ab6d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ab00af910 0x7f1ab015a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.375+0000 7f1ab6d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab00ce890 con 0x7f1ab00cd5e0 2023-10-06T05:22:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.375+0000 7f1ab6d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab015d890 con 0x7f1ab00af540 2023-10-06T05:22:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.375+0000 7f1ab6d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab00afd50 con 0x7f1ab00af910 2023-10-06T05:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab559a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1ab00cd5e0 0x7f1ab00ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56500/0 (socket says 172.21.15.33:56500) 2023-10-06T05:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab559a700 1 -- 172.21.15.33:0/2769213127 learned_addr learned my addr 172.21.15.33:0/2769213127 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab659c700 1 --2- 172.21.15.33:0/2769213127 >> v2:172.21.15.167:3300/0 conn(0x7f1ab00af910 0x7f1ab015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34084188 0 0) 0x7f1ab00ce890 con 0x7f1ab00cd5e0 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a9c003570 con 0x7f1ab00cd5e0 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1501845991 0 0) 0x7f1a9c003570 con 0x7f1ab00cd5e0 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 >> v2:172.21.15.167:3300/0 conn(0x7f1ab00af910 msgr2=0x7f1ab015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 --2- 172.21.15.33:0/2769213127 >> v2:172.21.15.167:3300/0 conn(0x7f1ab00af910 0x7f1ab015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 >> v1:172.21.15.167:6789/0 conn(0x7f1ab00af540 legacy=0x7f1ab00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab015db70 con 0x7f1ab00cd5e0 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.376+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1aa0001470 con 0x7f1ab00cd5e0 2023-10-06T05:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 >> v1:172.21.15.33:6789/0 conn(0x7f1ab00cd5e0 legacy=0x7f1ab00ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c003a60 0x7f1a9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9c004810 0x7f1a9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9c0051c0 0x7f1a9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00afd50 con 0x7f1a9c004810 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c00c0d0 con 0x7f1a9c003a60 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c00c290 con 0x7f1a9c0051c0 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab559a700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9c004810 0x7f1a9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab5d9b700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c003a60 0x7f1a9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab659c700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9c0051c0 0x7f1a9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab559a700 1 -- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9c0051c0 msgr2=0x7f1a9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab559a700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a9c0051c0 0x7f1a9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab559a700 1 -- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c003a60 msgr2=0x7f1a9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab559a700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c003a60 0x7f1a9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab559a700 1 -- 172.21.15.33:0/2769213127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c003570 con 0x7f1a9c004810 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.377+0000 7f1ab659c700 1 -- 172.21.15.33:0/2769213127 reap_dead start 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab559a700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9c004810 0x7f1a9c004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c390 tx=0x7f1ab015da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c6addf8e685776 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa000cc60 con 0x7f1a9c004810 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa000cdc0 con 0x7f1a9c004810 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab4d99700 1 -- 172.21.15.33:0/2769213127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0005b70 con 0x7f1a9c004810 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/2769213127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9c004810 msgr2=0x7f1a9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9c004810 0x7f1a9c004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c390 tx=0x7f1ab015da60 comp rx=0 tx=0).stop 2023-10-06T05:22:57.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.378+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/2769213127 shutdown_connections 2023-10-06T05:22:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.379+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/2769213127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a9c004810 0x7f1a9c004c80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.379+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/2769213127 >> 172.21.15.33:0/2769213127 conn(0x7f1ab006d9b0 msgr2=0x7f1ab006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.379+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/2769213127 shutdown_connections 2023-10-06T05:22:57.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.379+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/2769213127 wait complete. 2023-10-06T05:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.380+0000 7f1ab6d9d700 1 Processor -- start 2023-10-06T05:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.380+0000 7f1ab6d9d700 1 -- start start 2023-10-06T05:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab6d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c004810 0x7f1ab00b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab6d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ab0156470 0x7f1ab0151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 0x7f1ab015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab6d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00b3930 con 0x7f1ab00b49d0 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab6d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00b3c10 con 0x7f1a9c004810 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab6d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00b3ef0 con 0x7f1ab0156470 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 0x7f1ab015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 0x7f1ab015ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47938/0 (socket says 172.21.15.33:47938) 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.381+0000 7f1ab659c700 1 -- 172.21.15.33:0/1887644160 learned_addr learned my addr 172.21.15.33:0/1887644160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab5d9b700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c004810 0x7f1ab00b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab559a700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ab0156470 0x7f1ab0151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab659c700 1 -- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ab0156470 msgr2=0x7f1ab0151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab659c700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ab0156470 0x7f1ab0151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab659c700 1 -- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c004810 msgr2=0x7f1ab00b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab659c700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c004810 0x7f1ab00b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab659c700 1 -- 172.21.15.33:0/1887644160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab00b41d0 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab559a700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ab0156470 0x7f1ab0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab659c700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 0x7f1ab015ab10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00cce00 tx=0x7f1aa800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800e040 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa800f040 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800fe20 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.382+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab00b44b0 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.383+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab01515b0 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.383+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab0065890 con 0x7f1ab00b49d0 2023-10-06T05:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.384+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1aa8015640 con 0x7f1ab00b49d0 2023-10-06T05:22:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.385+0000 7f1aa5ffb700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a94066f30 0x7f1a940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.385+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1aa8084d30 con 0x7f1ab00b49d0 2023-10-06T05:22:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.386+0000 7f1ab5d9b700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a94066f30 0x7f1a940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.386+0000 7f1ab5d9b700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a94066f30 0x7f1a940693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1aac003c30 tx=0x7f1aac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.390+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1aa80520a0 con 0x7f1ab00b49d0 2023-10-06T05:22:57.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.592+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1ab00b46a0 con 0x7f1ab00b49d0 2023-10-06T05:22:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.597+0000 7f1aa5ffb700 1 -- 172.21.15.33:0/1887644160 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1aa80556c0 con 0x7f1ab00b49d0 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a94066f30 msgr2=0x7f1a940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a94066f30 0x7f1a940693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1aac003c30 tx=0x7f1aac009040 comp rx=0 tx=0).stop 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 msgr2=0x7f1ab015ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 0x7f1ab015ab10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00cce00 tx=0x7f1aa800b040 comp rx=0 tx=0).stop 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 shutdown_connections 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1a94066f30 0x7f1a940693e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab00b49d0 0x7f1ab015ab10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a9c004810 0x7f1ab00b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 --2- 172.21.15.33:0/1887644160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ab0156470 0x7f1ab0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 >> 172.21.15.33:0/1887644160 conn(0x7f1ab006d9b0 msgr2=0x7f1ab015be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 shutdown_connections 2023-10-06T05:22:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.599+0000 7f1ab6d9d700 1 -- 172.21.15.33:0/1887644160 wait complete. 2023-10-06T05:22:57.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dont_mark_unresponsive_client_stale' 2023-10-06T05:22:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.804+0000 7fa23599d700 1 Processor -- start 2023-10-06T05:22:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.804+0000 7fa23599d700 1 -- start start 2023-10-06T05:22:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.805+0000 7fa23599d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 0x7fa23006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.805+0000 7fa23599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa230155af0 0x7fa23006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.805+0000 7fa23599d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23006ab60 con 0x7fa23006a6a0 2023-10-06T05:22:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa23599d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23006b7e0 con 0x7fa2300c8b50 2023-10-06T05:22:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa23599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23006ae40 con 0x7fa230155af0 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa23499b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 0x7fa23006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa22ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa230155af0 0x7fa23006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa23499b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 0x7fa23006e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:42290/0 (socket says 172.21.15.33:42290) 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa23499b700 1 -- 172.21.15.33:0/1478603564 learned_addr learned my addr 172.21.15.33:0/1478603564 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.806+0000 7fa23519c700 1 --1- 172.21.15.33:0/1478603564 >> v1:172.21.15.167:6789/0 conn(0x7fa23006a6a0 0x7fa2300c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:44710/0 (socket says 172.21.15.33:44710) 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22ffff700 1 -- 172.21.15.33:0/1478603564 >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 msgr2=0x7fa23006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22ffff700 1 --2- 172.21.15.33:0/1478603564 >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 0x7fa23006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22ffff700 1 -- 172.21.15.33:0/1478603564 >> v1:172.21.15.167:6789/0 conn(0x7fa23006a6a0 legacy=0x7fa2300c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22ffff700 1 -- 172.21.15.33:0/1478603564 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23006b120 con 0x7fa230155af0 2023-10-06T05:22:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa23499b700 1 --2- 172.21.15.33:0/1478603564 >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 0x7fa23006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22f7fe700 1 -- 172.21.15.33:0/1478603564 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849344264 0 0) 0x7fa23006ab60 con 0x7fa23006a6a0 2023-10-06T05:22:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22ffff700 1 --2- 172.21.15.33:0/1478603564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa230155af0 0x7fa23006a160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0098e0 tx=0x7fa21c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d1d32e0fb95351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22f7fe700 1 -- 172.21.15.33:0/1478603564 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00b640 con 0x7fa230155af0 2023-10-06T05:22:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22f7fe700 1 -- 172.21.15.33:0/1478603564 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c00b7a0 con 0x7fa230155af0 2023-10-06T05:22:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.807+0000 7fa22f7fe700 1 -- 172.21.15.33:0/1478603564 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00bb40 con 0x7fa230155af0 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 -- 172.21.15.33:0/1478603564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa230155af0 msgr2=0x7fa23006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 --2- 172.21.15.33:0/1478603564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa230155af0 0x7fa23006a160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0098e0 tx=0x7fa21c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 -- 172.21.15.33:0/1478603564 shutdown_connections 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 --2- 172.21.15.33:0/1478603564 >> v2:172.21.15.167:3300/0 conn(0x7fa2300c8b50 0x7fa23006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 --2- 172.21.15.33:0/1478603564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa230155af0 0x7fa23006a160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 -- 172.21.15.33:0/1478603564 >> 172.21.15.33:0/1478603564 conn(0x7fa23006d030 msgr2=0x7fa23006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 -- 172.21.15.33:0/1478603564 shutdown_connections 2023-10-06T05:22:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.808+0000 7fa23599d700 1 -- 172.21.15.33:0/1478603564 wait complete. 2023-10-06T05:22:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.809+0000 7fa23599d700 1 Processor -- start 2023-10-06T05:22:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.810+0000 7fa23599d700 1 -- start start 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.810+0000 7fa23599d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23006a6a0 0x7fa230169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.810+0000 7fa23599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2300c8b50 0x7fa230158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23599d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 0x7fa230158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23599d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300c5ad0 con 0x7fa230155af0 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23599d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300c5230 con 0x7fa23006a6a0 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300c5510 con 0x7fa2300c8b50 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 0x7fa230158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 0x7fa230158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47946/0 (socket says 172.21.15.33:47946) 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 -- 172.21.15.33:0/765001033 learned_addr learned my addr 172.21.15.33:0/765001033 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23499b700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23006a6a0 0x7fa230169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa22ffff700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2300c8b50 0x7fa230158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 -- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2300c8b50 msgr2=0x7fa230158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2300c8b50 0x7fa230158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 -- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23006a6a0 msgr2=0x7fa230169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23006a6a0 0x7fa230169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.811+0000 7fa23519c700 1 -- 172.21.15.33:0/765001033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240094f0 con 0x7fa230155af0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.812+0000 7fa23519c700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 0x7fa230158e40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa22800ccc0 tx=0x7fa228001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.812+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228005df0 con 0x7fa230155af0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.812+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22800a690 con 0x7fa230155af0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.812+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800aa50 con 0x7fa230155af0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.812+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c009420 con 0x7fa230155af0 2023-10-06T05:22:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.812+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23016a960 con 0x7fa230155af0 2023-10-06T05:22:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.813+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23016ab50 con 0x7fa230155af0 2023-10-06T05:22:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.814+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2280070f0 con 0x7fa230155af0 2023-10-06T05:22:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.815+0000 7fa22cff9700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa214066f30 0x7fa2140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.815+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa2280843e0 con 0x7fa230155af0 2023-10-06T05:22:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.815+0000 7fa23499b700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa214066f30 0x7fa2140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.816+0000 7fa23499b700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa214066f30 0x7fa2140693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa224006d20 tx=0x7fa224009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:57.819+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa228051880 con 0x7fa230155af0 2023-10-06T05:22:58.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.107+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dont_mark_unresponsive_client_stale"]} v 0) v1 -- 0x7fa23004c380 con 0x7fa230155af0 2023-10-06T05:22:58.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.146+0000 7fa22cff9700 1 -- 172.21.15.33:0/765001033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_dont_mark_unresponsive_client_stale"]}]=0 v439) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa228054ea0 con 0x7fa230155af0 2023-10-06T05:22:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa214066f30 msgr2=0x7fa2140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa214066f30 0x7fa2140693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa224006d20 tx=0x7fa224009250 comp rx=0 tx=0).stop 2023-10-06T05:22:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 msgr2=0x7fa230158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 0x7fa230158e40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa22800ccc0 tx=0x7fa228001a40 comp rx=0 tx=0).stop 2023-10-06T05:22:58.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 shutdown_connections 2023-10-06T05:22:58.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa214066f30 0x7fa2140693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230155af0 0x7fa230158e40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa23006a6a0 0x7fa230169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 --2- 172.21.15.33:0/765001033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2300c8b50 0x7fa230158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 >> 172.21.15.33:0/765001033 conn(0x7fa23006d030 msgr2=0x7fa2300c7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 shutdown_connections 2023-10-06T05:22:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.149+0000 7fa23599d700 1 -- 172.21.15.33:0/765001033 wait complete. 2023-10-06T05:22:58.168 INFO:tasks.cephfs_test_runner:Test that an unresponsive client holding caps is not marked stale or ... skipped 'Require FUSE client to handle signal STOP/CONT' 2023-10-06T05:22:58.168 INFO:tasks.cephfs_test_runner:Starting test: test_evicted_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:22:58.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_evicted_caps' 2023-10-06T05:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.354+0000 7f59e679d700 1 Processor -- start 2023-10-06T05:22:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.354+0000 7f59e679d700 1 -- start start 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59e00ccfc0 0x7f59e00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cf3b0 0x7f59e01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00cbd80 con 0x7f59e00cc750 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00af5f0 con 0x7f59e00ccfc0 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00af8d0 con 0x7f59e00cf3b0 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e579b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59e00ccfc0 0x7f59e00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e5f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f59e00cc750 0x7f59e00b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56506/0 (socket says 172.21.15.33:56506) 2023-10-06T05:22:58.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.355+0000 7f59e5f9c700 1 -- 172.21.15.33:0/2522606718 learned_addr learned my addr 172.21.15.33:0/2522606718 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59e4f9a700 1 --2- 172.21.15.33:0/2522606718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cf3b0 0x7f59e01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59d7fff700 1 -- 172.21.15.33:0/2522606718 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408241652 0 0) 0x7f59e00cbd80 con 0x7f59e00cc750 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59e579b700 1 -- 172.21.15.33:0/2522606718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cf3b0 msgr2=0x7f59e01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59e579b700 1 --2- 172.21.15.33:0/2522606718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cf3b0 0x7f59e01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59e579b700 1 -- 172.21.15.33:0/2522606718 >> v1:172.21.15.33:6789/0 conn(0x7f59e00cc750 legacy=0x7f59e00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59e579b700 1 -- 172.21.15.33:0/2522606718 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e00afbb0 con 0x7f59e00ccfc0 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59e579b700 1 --2- 172.21.15.33:0/2522606718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00cc210 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0098e0 tx=0x7f59dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a493db6dbc092c28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59d7fff700 1 -- 172.21.15.33:0/2522606718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc009e80 con 0x7f59e00ccfc0 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59d7fff700 1 -- 172.21.15.33:0/2522606718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59dc00b860 con 0x7f59e00ccfc0 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.356+0000 7f59d7fff700 1 -- 172.21.15.33:0/2522606718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00bbe0 con 0x7f59e00ccfc0 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 -- 172.21.15.33:0/2522606718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 msgr2=0x7f59e00cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 --2- 172.21.15.33:0/2522606718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00cc210 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0098e0 tx=0x7f59dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 -- 172.21.15.33:0/2522606718 shutdown_connections 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 --2- 172.21.15.33:0/2522606718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00cc210 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 --2- 172.21.15.33:0/2522606718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cf3b0 0x7f59e01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 -- 172.21.15.33:0/2522606718 >> 172.21.15.33:0/2522606718 conn(0x7f59e006cff0 msgr2=0x7f59e006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 -- 172.21.15.33:0/2522606718 shutdown_connections 2023-10-06T05:22:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.357+0000 7f59e679d700 1 -- 172.21.15.33:0/2522606718 wait complete. 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.358+0000 7f59e679d700 1 Processor -- start 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.359+0000 7f59e679d700 1 -- start start 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.359+0000 7f59e679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cc750 0x7f59e01471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.359+0000 7f59e679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e00b53e0 0x7f59e00c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00b32a0 con 0x7f59e00ccfc0 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01476e0 con 0x7f59e00b53e0 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0154b50 con 0x7f59e00cc750 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47960/0 (socket says 172.21.15.33:47960) 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 -- 172.21.15.33:0/793907757 learned_addr learned my addr 172.21.15.33:0/793907757 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e579b700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cc750 0x7f59e01471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e5f9c700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e00b53e0 0x7f59e00c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 -- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cc750 msgr2=0x7f59e01471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cc750 0x7f59e01471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 -- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e00b53e0 msgr2=0x7f59e00c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e00b53e0 0x7f59e00c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 -- 172.21.15.33:0/793907757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc009420 con 0x7f59e00ccfc0 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e579b700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cc750 0x7f59e01471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.360+0000 7f59e4f9a700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00b4ea0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d890 tx=0x7f59d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.361+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0018070 con 0x7f59e00ccfc0 2023-10-06T05:22:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.361+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d000b690 con 0x7f59e00ccfc0 2023-10-06T05:22:58.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.361+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d000ba30 con 0x7f59e00ccfc0 2023-10-06T05:22:58.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.361+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e00b34b0 con 0x7f59e00ccfc0 2023-10-06T05:22:58.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.361+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e00b3a70 con 0x7f59e00ccfc0 2023-10-06T05:22:58.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.362+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e004c380 con 0x7f59e00ccfc0 2023-10-06T05:22:58.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.363+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59d0014430 con 0x7f59e00ccfc0 2023-10-06T05:22:58.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.364+0000 7f59d57fa700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c4066f30 0x7f59c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:58.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.364+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f59d0084770 con 0x7f59e00ccfc0 2023-10-06T05:22:58.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.365+0000 7f59e579b700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c4066f30 0x7f59c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:58.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.366+0000 7f59e579b700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c4066f30 0x7f59c40693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0091e0 tx=0x7f59dc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:58.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.368+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59d0051b90 con 0x7f59e00ccfc0 2023-10-06T05:22:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:58.648+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_evicted_caps"]} v 0) v1 -- 0x7f59e00af0a0 con 0x7f59e00ccfc0 2023-10-06T05:22:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.148+0000 7f59d57fa700 1 -- 172.21.15.33:0/793907757 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_evicted_caps"]}]=0 v440) v1 ==== 152+0+0 (secure 0 0 0) 0x7f59d00551b0 con 0x7f59e00ccfc0 2023-10-06T05:22:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c4066f30 msgr2=0x7f59c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c4066f30 0x7f59c40693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0091e0 tx=0x7f59dc017000 comp rx=0 tx=0).stop 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 msgr2=0x7f59e00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00b4ea0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d890 tx=0x7f59d0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 shutdown_connections 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c4066f30 0x7f59c40693e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59e00b53e0 0x7f59e00c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59e00cc750 0x7f59e01471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 --2- 172.21.15.33:0/793907757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59e00ccfc0 0x7f59e00b4ea0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 >> 172.21.15.33:0/793907757 conn(0x7f59e006cff0 msgr2=0x7f59e0156b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 shutdown_connections 2023-10-06T05:22:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.151+0000 7f59e679d700 1 -- 172.21.15.33:0/793907757 wait complete. 2023-10-06T05:22:59.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:22:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.363+0000 7f365939d700 1 Processor -- start 2023-10-06T05:22:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.363+0000 7f365939d700 1 -- start start 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.364+0000 7f365939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f365415acd0 0x7f36540abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.364+0000 7f365939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f36540ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.364+0000 7f365939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36540b0cd0 con 0x7f3654159850 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.364+0000 7f365939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365415a840 con 0x7f365415acd0 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.364+0000 7f365939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3654159f10 con 0x7f365406efa0 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f365415acd0 0x7f36540abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f365415acd0 0x7f36540abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47972/0 (socket says 172.21.15.33:47972) 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 -- 172.21.15.33:0/2334400017 learned_addr learned my addr 172.21.15.33:0/2334400017 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f3658b9c700 1 --2- 172.21.15.33:0/2334400017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f36540ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 -- 172.21.15.33:0/2334400017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 msgr2=0x7f36540ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:59.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 --2- 172.21.15.33:0/2334400017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f36540ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 -- 172.21.15.33:0/2334400017 >> v1:172.21.15.167:6789/0 conn(0x7f3654159850 legacy=0x7f365406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 -- 172.21.15.33:0/2334400017 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365415a1f0 con 0x7f365415acd0 2023-10-06T05:22:59.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.365+0000 7f36537fe700 1 --2- 172.21.15.33:0/2334400017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f365415acd0 0x7f36540abe90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f36480099b0 tx=0x7f3648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212e68d9d71240b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f3652ffd700 1 -- 172.21.15.33:0/2334400017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364800b860 con 0x7f365415acd0 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f3652ffd700 1 -- 172.21.15.33:0/2334400017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364800b9c0 con 0x7f365415acd0 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f3652ffd700 1 -- 172.21.15.33:0/2334400017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364800bd60 con 0x7f365415acd0 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 -- 172.21.15.33:0/2334400017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f365415acd0 msgr2=0x7f36540abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 --2- 172.21.15.33:0/2334400017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f365415acd0 0x7f36540abe90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f36480099b0 tx=0x7f3648009250 comp rx=0 tx=0).stop 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 -- 172.21.15.33:0/2334400017 shutdown_connections 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 --2- 172.21.15.33:0/2334400017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f36540ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 --2- 172.21.15.33:0/2334400017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f365415acd0 0x7f36540abe90 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 -- 172.21.15.33:0/2334400017 >> 172.21.15.33:0/2334400017 conn(0x7f365406cd40 msgr2=0x7f36540ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 -- 172.21.15.33:0/2334400017 shutdown_connections 2023-10-06T05:22:59.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.366+0000 7f365939d700 1 -- 172.21.15.33:0/2334400017 wait complete. 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.368+0000 7f365939d700 1 Processor -- start 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.368+0000 7f365939d700 1 -- start start 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f365939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f3654156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f365939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 0x7f36540ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f365939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f365415acd0 0x7f36540ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f365939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540ccff0 con 0x7f3654159850 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f365939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540b1270 con 0x7f365415acd0 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f365939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36541574d0 con 0x7f365406efa0 2023-10-06T05:22:59.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f36537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 0x7f36540ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f36537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 0x7f36540ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47982/0 (socket says 172.21.15.33:47982) 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f36537fe700 1 -- 172.21.15.33:0/2320361041 learned_addr learned my addr 172.21.15.33:0/2320361041 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f3658b9c700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f365415acd0 0x7f36540ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.369+0000 7f3653fff700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f3654156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f36537fe700 1 -- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 msgr2=0x7f3654156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f36537fe700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f3654156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f36537fe700 1 -- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f365415acd0 msgr2=0x7f36540ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f36537fe700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f365415acd0 0x7f36540ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f36537fe700 1 -- 172.21.15.33:0/2320361041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480094f0 con 0x7f3654159850 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f36537fe700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 0x7f36540ad110 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f3648006fd0 tx=0x7f3648017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648024440 con 0x7f3654159850 2023-10-06T05:22:59.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36480245a0 con 0x7f3654159850 2023-10-06T05:22:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648024940 con 0x7f3654159850 2023-10-06T05:22:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36540cd720 con 0x7f3654159850 2023-10-06T05:22:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.370+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36540cca30 con 0x7f3654159850 2023-10-06T05:22:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.371+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365404c380 con 0x7f3654159850 2023-10-06T05:22:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.372+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f364801a420 con 0x7f3654159850 2023-10-06T05:22:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.373+0000 7f363bfff700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3640066f30 0x7f36400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.373+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f36480861a0 con 0x7f3654159850 2023-10-06T05:22:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.373+0000 7f3653fff700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3640066f30 0x7f36400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.374+0000 7f3653fff700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3640066f30 0x7f36400693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3644003860 tx=0x7f3644009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.378+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3648053590 con 0x7f3654159850 2023-10-06T05:22:59.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.580+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3654064750 con 0x7f3654159850 2023-10-06T05:22:59.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.584+0000 7f363bfff700 1 -- 172.21.15.33:0/2320361041 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3648056bb0 con 0x7f3654159850 2023-10-06T05:22:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.586+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3640066f30 msgr2=0x7f36400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.586+0000 7f365939d700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3640066f30 0x7f36400693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3644003860 tx=0x7f3644009040 comp rx=0 tx=0).stop 2023-10-06T05:22:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.586+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 msgr2=0x7f36540ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.586+0000 7f365939d700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 0x7f36540ad110 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f3648006fd0 tx=0x7f3648017000 comp rx=0 tx=0).stop 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 shutdown_connections 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3640066f30 0x7f36400693e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f365415acd0 0x7f36540ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f365406efa0 0x7f3654156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 --2- 172.21.15.33:0/2320361041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3654159850 0x7f36540ad110 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 >> 172.21.15.33:0/2320361041 conn(0x7f365406cd40 msgr2=0x7f36541587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 shutdown_connections 2023-10-06T05:22:59.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.587+0000 7f365939d700 1 -- 172.21.15.33:0/2320361041 wait complete. 2023-10-06T05:22:59.606 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:22:59.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:59.684 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:59.770 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:22:59.771 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:22:59.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:59.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.956+0000 7f127219d700 1 Processor -- start 2023-10-06T05:22:59.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.956+0000 7f127219d700 1 -- start start 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127219d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f126c0ccfc0 0x7f126c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127219d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f126c0cf3b0 0x7f126c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127219d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126c0cbd80 con 0x7f126c0cc750 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127219d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c0af5f0 con 0x7f126c0ccfc0 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127219d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c0af8d0 con 0x7f126c0cf3b0 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f126c0ccfc0 0x7f126c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127119b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f126c0ccfc0 0x7f126c0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35460/0 (socket says 172.21.15.33:35460) 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127119b700 1 -- 172.21.15.33:0/3572228692 learned_addr learned my addr 172.21.15.33:0/3572228692 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.957+0000 7f127199c700 1 --1- 172.21.15.33:0/3572228692 >> v1:172.21.15.33:6789/0 conn(0x7f126c0cc750 0x7f126c0b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33824/0 (socket says 172.21.15.33:33824) 2023-10-06T05:22:59.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f127099a700 1 --2- 172.21.15.33:0/3572228692 >> v2:172.21.15.167:3300/0 conn(0x7f126c0cf3b0 0x7f126c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f127119b700 1 -- 172.21.15.33:0/3572228692 >> v2:172.21.15.167:3300/0 conn(0x7f126c0cf3b0 msgr2=0x7f126c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f127119b700 1 --2- 172.21.15.33:0/3572228692 >> v2:172.21.15.167:3300/0 conn(0x7f126c0cf3b0 0x7f126c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f127119b700 1 -- 172.21.15.33:0/3572228692 >> v1:172.21.15.33:6789/0 conn(0x7f126c0cc750 legacy=0x7f126c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f127119b700 1 -- 172.21.15.33:0/3572228692 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c0afbb0 con 0x7f126c0ccfc0 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f1263fff700 1 -- 172.21.15.33:0/3572228692 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978677538 0 0) 0x7f126c0cbd80 con 0x7f126c0cc750 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f127119b700 1 --2- 172.21.15.33:0/3572228692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0ccfc0 0x7f126c0cc210 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f12680099b0 tx=0x7f1268009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c94cc15a7aae93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f1263fff700 1 -- 172.21.15.33:0/3572228692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800b820 con 0x7f126c0ccfc0 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f1263fff700 1 -- 172.21.15.33:0/3572228692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126800b980 con 0x7f126c0ccfc0 2023-10-06T05:22:59.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.958+0000 7f1263fff700 1 -- 172.21.15.33:0/3572228692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800bd20 con 0x7f126c0ccfc0 2023-10-06T05:22:59.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 -- 172.21.15.33:0/3572228692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0ccfc0 msgr2=0x7f126c0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 --2- 172.21.15.33:0/3572228692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0ccfc0 0x7f126c0cc210 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f12680099b0 tx=0x7f1268009250 comp rx=0 tx=0).stop 2023-10-06T05:22:59.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 -- 172.21.15.33:0/3572228692 shutdown_connections 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 --2- 172.21.15.33:0/3572228692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0ccfc0 0x7f126c0cc210 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 --2- 172.21.15.33:0/3572228692 >> v2:172.21.15.167:3300/0 conn(0x7f126c0cf3b0 0x7f126c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 -- 172.21.15.33:0/3572228692 >> 172.21.15.33:0/3572228692 conn(0x7f126c06cff0 msgr2=0x7f126c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 -- 172.21.15.33:0/3572228692 shutdown_connections 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.959+0000 7f127219d700 1 -- 172.21.15.33:0/3572228692 wait complete. 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.961+0000 7f127219d700 1 Processor -- start 2023-10-06T05:22:59.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.961+0000 7f127219d700 1 -- start start 2023-10-06T05:22:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 0x7f126c14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126c0ccfc0 0x7f126c0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f126c0c9060 0x7f126c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0ca1f0 con 0x7f126c0cc750 2023-10-06T05:22:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0c95e0 con 0x7f126c0ccfc0 2023-10-06T05:22:59.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0c98c0 con 0x7f126c0c9060 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 0x7f126c14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 0x7f126c14f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35466/0 (socket says 172.21.15.33:35466) 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 -- 172.21.15.33:0/2613377851 learned_addr learned my addr 172.21.15.33:0/2613377851 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127099a700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126c0ccfc0 0x7f126c0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127199c700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f126c0c9060 0x7f126c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 -- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f126c0c9060 msgr2=0x7f126c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f126c0c9060 0x7f126c154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 -- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126c0ccfc0 msgr2=0x7f126c0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126c0ccfc0 0x7f126c0c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127119b700 1 -- 172.21.15.33:0/2613377851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680094f0 con 0x7f126c0cc750 2023-10-06T05:22:59.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.962+0000 7f127199c700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f126c0c9060 0x7f126c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.963+0000 7f127119b700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 0x7f126c14f430 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f12680094c0 tx=0x7f1268017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.963+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800bd00 con 0x7f126c0cc750 2023-10-06T05:22:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.963+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1268026450 con 0x7f126c0cc750 2023-10-06T05:22:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.963+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680267d0 con 0x7f126c0cc750 2023-10-06T05:22:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.963+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c0c9ba0 con 0x7f126c0cc750 2023-10-06T05:22:59.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.963+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c0b34d0 con 0x7f126c0cc750 2023-10-06T05:22:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.964+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c04c380 con 0x7f126c0cc750 2023-10-06T05:22:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.965+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f126801a420 con 0x7f126c0cc750 2023-10-06T05:22:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.966+0000 7f12617fa700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1250066f30 0x7f12500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.966+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1268085f50 con 0x7f126c0cc750 2023-10-06T05:22:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.966+0000 7f127099a700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1250066f30 0x7f12500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.967+0000 7f127099a700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1250066f30 0x7f12500693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f126c0b3be0 tx=0x7f125c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:22:59.971+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1268053340 con 0x7f126c0cc750 2023-10-06T05:23:00.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.273+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f126c0af0a0 con 0x7f126c0cc750 2023-10-06T05:23:00.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.274+0000 7f12617fa700 1 -- 172.21.15.33:0/2613377851 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f126802bde0 con 0x7f126c0cc750 2023-10-06T05:23:00.276 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:00.276 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/212570262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":212570262},{"type":"v1","addr":"172.21.15.167:6836","nonce":212570262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2298424163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2298424163},{"type":"v1","addr":"172.21.15.167:6837","nonce":2298424163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2652705209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2652705209},{"type":"v1","addr":"172.21.15.33:6833","nonce":2652705209}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7575,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2224464430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2224464430},{"type":"v1","addr":"172.21.15.33:6835","nonce":2224464430}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1250066f30 msgr2=0x7f12500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1250066f30 0x7f12500693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f126c0b3be0 tx=0x7f125c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 msgr2=0x7f126c14f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 0x7f126c14f430 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f12680094c0 tx=0x7f1268017000 comp rx=0 tx=0).stop 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 shutdown_connections 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1250066f30 0x7f12500693e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f126c0c9060 0x7f126c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f126c0cc750 0x7f126c14f430 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 --2- 172.21.15.33:0/2613377851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126c0ccfc0 0x7f126c0c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 >> 172.21.15.33:0/2613377851 conn(0x7f126c06cff0 msgr2=0x7f126c14fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 shutdown_connections 2023-10-06T05:23:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.276+0000 7f127219d700 1 -- 172.21.15.33:0/2613377851 wait complete. 2023-10-06T05:23:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 75 2023-10-06T05:23:00.296 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:23:00.296 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:23:00.296 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:23:00.297 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:00.297 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:23:00.297 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:23:00.297 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:23:00.297 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:00.297 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:23:00.298 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:23:00.298 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:23:00.298 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:00.298 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:23:00.298 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:23:00.298 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:23:00.298 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:00.312 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:23:00.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:23:00.314 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:23:00.314 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:23:00.315 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:23:00.315 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:23:00.316 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:23:00.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:23:00.359 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:23:00.359 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:00.360 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:23:00.360 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:00.362 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:23:00.362 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:00.364 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:23:00.364 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:00.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:23:00.496 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:23:00.497 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:23:00.499 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:23:00.499 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:23:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.558+0000 7eff5a59d700 1 Processor -- start 2023-10-06T05:23:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.559+0000 7eff5a59d700 1 -- start start 2023-10-06T05:23:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff5a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54159e10 0x7eff54157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff5a59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5406bd10 con 0x7eff54156880 2023-10-06T05:23:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff5a59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5406a820 con 0x7eff540c8090 2023-10-06T05:23:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff5a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff540cc290 con 0x7eff54159e10 2023-10-06T05:23:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff5959b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7eff54156880 0x7eff540c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33840/0 (socket says 172.21.15.33:33840) 2023-10-06T05:23:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff5959b700 1 -- 172.21.15.33:0/707457233 learned_addr learned my addr 172.21.15.33:0/707457233 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.560+0000 7eff59d9c700 1 --2- 172.21.15.33:0/707457233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54159e10 0x7eff54157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043099596 0 0) 0x7eff5406bd10 con 0x7eff54156880 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff40003570 con 0x7eff54156880 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816328136 0 0) 0x7eff5406a820 con 0x7eff540c8090 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff5406bd10 con 0x7eff540c8090 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff59d9c700 1 -- 172.21.15.33:0/707457233 >> v1:172.21.15.167:6789/0 conn(0x7eff540c8090 legacy=0x7eff54156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff59d9c700 1 -- 172.21.15.33:0/707457233 >> v1:172.21.15.33:6789/0 conn(0x7eff54156880 legacy=0x7eff540c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff59d9c700 1 -- 172.21.15.33:0/707457233 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5415a2f0 con 0x7eff54159e10 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff59d9c700 1 --2- 172.21.15.33:0/707457233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54159e10 0x7eff54157360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009ec0 tx=0x7eff4c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33b370ddc4d7d35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00e070 con 0x7eff54159e10 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c00bb40 con 0x7eff54159e10 2023-10-06T05:23:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.561+0000 7eff4bfff700 1 -- 172.21.15.33:0/707457233 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c012430 con 0x7eff54159e10 2023-10-06T05:23:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 -- 172.21.15.33:0/707457233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54159e10 msgr2=0x7eff54157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 --2- 172.21.15.33:0/707457233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54159e10 0x7eff54157360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009ec0 tx=0x7eff4c009780 comp rx=0 tx=0).stop 2023-10-06T05:23:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 -- 172.21.15.33:0/707457233 shutdown_connections 2023-10-06T05:23:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 --2- 172.21.15.33:0/707457233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54159e10 0x7eff54157360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 -- 172.21.15.33:0/707457233 >> 172.21.15.33:0/707457233 conn(0x7eff5406c560 msgr2=0x7eff5406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 -- 172.21.15.33:0/707457233 shutdown_connections 2023-10-06T05:23:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.562+0000 7eff5a59d700 1 -- 172.21.15.33:0/707457233 wait complete. 2023-10-06T05:23:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.563+0000 7eff5a59d700 1 Processor -- start 2023-10-06T05:23:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.564+0000 7eff5a59d700 1 -- start start 2023-10-06T05:23:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.564+0000 7eff5a59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff540c8090 0x7eff5415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.564+0000 7eff5a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54156880 0x7eff5415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.564+0000 7eff5a59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 0x7eff540b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff5a59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5416aa40 con 0x7eff54159e10 2023-10-06T05:23:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff5a59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5415b440 con 0x7eff540c8090 2023-10-06T05:23:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff5a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5415c330 con 0x7eff54156880 2023-10-06T05:23:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 0x7eff540b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 0x7eff540b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35496/0 (socket says 172.21.15.33:35496) 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 -- 172.21.15.33:0/4132012968 learned_addr learned my addr 172.21.15.33:0/4132012968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff5959b700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff540c8090 0x7eff5415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff58d9a700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54156880 0x7eff5415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 -- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54156880 msgr2=0x7eff5415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54156880 0x7eff5415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 -- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff540c8090 msgr2=0x7eff5415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff540c8090 0x7eff5415af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.565+0000 7eff59d9c700 1 -- 172.21.15.33:0/4132012968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c009a20 con 0x7eff54159e10 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.566+0000 7eff59d9c700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 0x7eff540b0650 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7eff540636f0 tx=0x7eff4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.566+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c012e00 con 0x7eff54159e10 2023-10-06T05:23:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.566+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c01b630 con 0x7eff54159e10 2023-10-06T05:23:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.566+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff54169ec0 con 0x7eff54159e10 2023-10-06T05:23:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.566+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c01b9f0 con 0x7eff54159e10 2023-10-06T05:23:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.566+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5416a480 con 0x7eff54159e10 2023-10-06T05:23:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.567+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff540af100 con 0x7eff54159e10 2023-10-06T05:23:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.568+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff4c01c400 con 0x7eff54159e10 2023-10-06T05:23:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.569+0000 7eff497fa700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7eff38066f30 0x7eff380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.569+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5292+0+0 (secure 0 0 0) 0x7eff4c085ef0 con 0x7eff54159e10 2023-10-06T05:23:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.569+0000 7eff5959b700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7eff38066f30 0x7eff380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.570+0000 7eff5959b700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7eff38066f30 0x7eff380693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7eff500058d0 tx=0x7eff50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.573+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff540af100 con 0x7eff54159e10 2023-10-06T05:23:00.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:00.795+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7eff540af100 con 0x7eff54159e10 2023-10-06T05:23:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.611+0000 7eff497fa700 1 -- 172.21.15.33:0/4132012968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7eff540af100 con 0x7eff54159e10 2023-10-06T05:23:01.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7eff38066f30 msgr2=0x7eff380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7eff38066f30 0x7eff380693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7eff500058d0 tx=0x7eff50009040 comp rx=0 tx=0).stop 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 msgr2=0x7eff540b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 0x7eff540b0650 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7eff540636f0 tx=0x7eff4c019000 comp rx=0 tx=0).stop 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 shutdown_connections 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7eff38066f30 0x7eff380693e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff54159e10 0x7eff540b0650 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff540c8090 0x7eff5415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 --2- 172.21.15.33:0/4132012968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff54156880 0x7eff5415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 >> 172.21.15.33:0/4132012968 conn(0x7eff5406c560 msgr2=0x7eff541503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 shutdown_connections 2023-10-06T05:23:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.614+0000 7eff5a59d700 1 -- 172.21.15.33:0/4132012968 wait complete. 2023-10-06T05:23:01.618 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:23:01.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:23:01.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.825+0000 7f3c5bfff700 1 Processor -- start 2023-10-06T05:23:01.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.825+0000 7f3c5bfff700 1 -- start start 2023-10-06T05:23:01.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c159520 0x7f3c5c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:01.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c0c65e0 con 0x7f3c5c0c6170 2023-10-06T05:23:01.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c06c820 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c06a160 con 0x7f3c5c159520 2023-10-06T05:23:01.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5a7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c5c0c6170 0x7f3c5c0c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33848/0 (socket says 172.21.15.33:33848) 2023-10-06T05:23:01.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5a7fc700 1 -- 172.21.15.33:0/3315256808 learned_addr learned my addr 172.21.15.33:0/3315256808 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c5affd700 1 --2- 172.21.15.33:0/3315256808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c159520 0x7f3c5c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.827+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548089593 0 0) 0x7f3c5c0c65e0 con 0x7f3c5c0c6170 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c44003570 con 0x7f3c5c0c6170 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690708379 0 0) 0x7f3c5c06c820 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c5c0c65e0 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4041845684 0 0) 0x7f3c44003570 con 0x7f3c5c0c6170 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c5affd700 1 -- 172.21.15.33:0/3315256808 >> v1:172.21.15.167:6789/0 conn(0x7f3c5c0c82a0 legacy=0x7f3c5c06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c5affd700 1 -- 172.21.15.33:0/3315256808 >> v1:172.21.15.33:6789/0 conn(0x7f3c5c0c6170 legacy=0x7f3c5c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c5affd700 1 -- 172.21.15.33:0/3315256808 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c0c8800 con 0x7f3c5c159520 2023-10-06T05:23:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.828+0000 7f3c5affd700 1 --2- 172.21.15.33:0/3315256808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c159520 0x7f3c5c0c5c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009910 tx=0x7f3c4c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c40cf6274e9e3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50001470 con 0x7f3c5c159520 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c011420 con 0x7f3c5c159520 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c59ffb700 1 -- 172.21.15.33:0/3315256808 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0117c0 con 0x7f3c5c159520 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 -- 172.21.15.33:0/3315256808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c159520 msgr2=0x7f3c5c0c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/3315256808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c159520 0x7f3c5c0c5c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009910 tx=0x7f3c4c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 -- 172.21.15.33:0/3315256808 shutdown_connections 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/3315256808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c159520 0x7f3c5c0c5c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 -- 172.21.15.33:0/3315256808 >> 172.21.15.33:0/3315256808 conn(0x7f3c5c0cc930 msgr2=0x7f3c5c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:01.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 -- 172.21.15.33:0/3315256808 shutdown_connections 2023-10-06T05:23:01.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.829+0000 7f3c5bfff700 1 -- 172.21.15.33:0/3315256808 wait complete. 2023-10-06T05:23:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.831+0000 7f3c5bfff700 1 Processor -- start 2023-10-06T05:23:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.831+0000 7f3c5bfff700 1 -- start start 2023-10-06T05:23:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c0c6170 0x7f3c5c0abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 0x7f3c5c1483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c5c159520 0x7f3c5c157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c0ac180 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c0ac390 con 0x7f3c5c159520 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c0ac670 con 0x7f3c5c0c6170 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 0x7f3c5c1483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 0x7f3c5c1483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35502/0 (socket says 172.21.15.33:35502) 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5a7fc700 1 -- 172.21.15.33:0/502591875 learned_addr learned my addr 172.21.15.33:0/502591875 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5affd700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c0c6170 0x7f3c5c0abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.832+0000 7f3c5b7fe700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c5c159520 0x7f3c5c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5a7fc700 1 -- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c0c6170 msgr2=0x7f3c5c0abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5a7fc700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c0c6170 0x7f3c5c0abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5a7fc700 1 -- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c5c159520 msgr2=0x7f3c5c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5a7fc700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c5c159520 0x7f3c5c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5a7fc700 1 -- 172.21.15.33:0/502591875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009420 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5b7fe700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c5c159520 0x7f3c5c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5a7fc700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 0x7f3c5c1483a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000cb70 tx=0x7f3c50001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50001d00 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50007360 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50004830 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c0ac950 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.833+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c1488e0 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.834+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c04c380 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.835+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c50004be0 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.837+0000 7f3c42ffd700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c48066f30 0x7f3c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:01.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.837+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3c50083b80 con 0x7f3c5c0c82a0 2023-10-06T05:23:01.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.837+0000 7f3c5affd700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c48066f30 0x7f3c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:01.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.838+0000 7f3c5affd700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c48066f30 0x7f3c480693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0030c0 tx=0x7f3c4c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:01.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:01.841+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c50050fc0 con 0x7f3c5c0c82a0 2023-10-06T05:23:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.123+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3c5c0c8ae0 con 0x7f3c5c0c82a0 2023-10-06T05:23:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.123+0000 7f3c42ffd700 1 -- 172.21.15.33:0/502591875 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3c50018020 con 0x7f3c5c0c82a0 2023-10-06T05:23:02.126 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.131 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:23:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.135 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:23:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:23:02.138 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:23:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:23:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:02.142 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:23:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:23:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c48066f30 msgr2=0x7f3c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c48066f30 0x7f3c480693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0030c0 tx=0x7f3c4c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 msgr2=0x7f3c5c1483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 0x7f3c5c1483a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000cb70 tx=0x7f3c50001ab0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 shutdown_connections 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c48066f30 0x7f3c480693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c5c159520 0x7f3c5c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c5c0c6170 0x7f3c5c0abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 --2- 172.21.15.33:0/502591875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0c82a0 0x7f3c5c1483a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 >> 172.21.15.33:0/502591875 conn(0x7f3c5c0cc930 msgr2=0x7f3c5c158660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 shutdown_connections 2023-10-06T05:23:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.126+0000 7f3c5bfff700 1 -- 172.21.15.33:0/502591875 wait complete. 2023-10-06T05:23:02.146 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.331+0000 7f46b659d700 1 Processor -- start 2023-10-06T05:23:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.332+0000 7f46b659d700 1 -- start start 2023-10-06T05:23:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b659d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46b00cb710 0x7f46b006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b00c9cb0 con 0x7f46b013faa0 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b00ca270 con 0x7f46b00c8050 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b659d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b006b840 con 0x7f46b00cb710 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b5d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46b00cb710 0x7f46b006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b5d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46b00cb710 0x7f46b006b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35504/0 (socket says 172.21.15.33:35504) 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b5d9c700 1 -- 172.21.15.33:0/1244912634 learned_addr learned my addr 172.21.15.33:0/1244912634 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.333+0000 7f46b559b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f46b013faa0 0x7f46b00c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33864/0 (socket says 172.21.15.33:33864) 2023-10-06T05:23:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46a7fff700 1 -- 172.21.15.33:0/1244912634 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92645402 0 0) 0x7f46b00c9cb0 con 0x7f46b013faa0 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46b5d9c700 1 -- 172.21.15.33:0/1244912634 >> v1:172.21.15.167:6789/0 conn(0x7f46b00c8050 legacy=0x7f46b006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46b5d9c700 1 -- 172.21.15.33:0/1244912634 >> v1:172.21.15.33:6789/0 conn(0x7f46b013faa0 legacy=0x7f46b00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46b5d9c700 1 -- 172.21.15.33:0/1244912634 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b006bb20 con 0x7f46b00cb710 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46b5d9c700 1 --2- 172.21.15.33:0/1244912634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b00cb710 0x7f46b006b1e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009f20 tx=0x7f46a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e4a947c4d5c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46a7fff700 1 -- 172.21.15.33:0/1244912634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a800e070 con 0x7f46b00cb710 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46a7fff700 1 -- 172.21.15.33:0/1244912634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a800ba80 con 0x7f46b00cb710 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.334+0000 7f46a7fff700 1 -- 172.21.15.33:0/1244912634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80092e0 con 0x7f46b00cb710 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 -- 172.21.15.33:0/1244912634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b00cb710 msgr2=0x7f46b006b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 --2- 172.21.15.33:0/1244912634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b00cb710 0x7f46b006b1e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009f20 tx=0x7f46a8009840 comp rx=0 tx=0).stop 2023-10-06T05:23:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 -- 172.21.15.33:0/1244912634 shutdown_connections 2023-10-06T05:23:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 --2- 172.21.15.33:0/1244912634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b00cb710 0x7f46b006b1e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 -- 172.21.15.33:0/1244912634 >> 172.21.15.33:0/1244912634 conn(0x7f46b006d030 msgr2=0x7f46b006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 -- 172.21.15.33:0/1244912634 shutdown_connections 2023-10-06T05:23:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.335+0000 7f46b659d700 1 -- 172.21.15.33:0/1244912634 wait complete. 2023-10-06T05:23:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.336+0000 7f46b659d700 1 Processor -- start 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.336+0000 7f46b659d700 1 -- start start 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.337+0000 7f46b659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b00c8050 0x7f46b00aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.337+0000 7f46b659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b00cb710 0x7f46b00c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.337+0000 7f46b659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 0x7f46b00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.337+0000 7f46b659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00b0230 con 0x7f46b013faa0 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.337+0000 7f46b659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c44e0 con 0x7f46b00cb710 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.337+0000 7f46b659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00af620 con 0x7f46b00c8050 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 0x7f46b00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 0x7f46b00b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35520/0 (socket says 172.21.15.33:35520) 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 -- 172.21.15.33:0/874083468 learned_addr learned my addr 172.21.15.33:0/874083468 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b4d9a700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b00cb710 0x7f46b00c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b559b700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b00c8050 0x7f46b00aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 -- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b00c8050 msgr2=0x7f46b00aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b00c8050 0x7f46b00aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 -- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b00cb710 msgr2=0x7f46b00c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b00cb710 0x7f46b00c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 -- 172.21.15.33:0/874083468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a8009ae0 con 0x7f46b013faa0 2023-10-06T05:23:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.338+0000 7f46b5d9c700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 0x7f46b00b06c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f46b00719c0 tx=0x7f46a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.339+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8007d10 con 0x7f46b013faa0 2023-10-06T05:23:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.339+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a8007e90 con 0x7f46b013faa0 2023-10-06T05:23:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.339+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b00af900 con 0x7f46b013faa0 2023-10-06T05:23:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.339+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80266b0 con 0x7f46b013faa0 2023-10-06T05:23:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.339+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b00afec0 con 0x7f46b013faa0 2023-10-06T05:23:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.340+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b00b03e0 con 0x7f46b013faa0 2023-10-06T05:23:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.341+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46a8026a20 con 0x7f46b013faa0 2023-10-06T05:23:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.342+0000 7f46a57fa700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4694066f30 0x7f46940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.342+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f46a8086370 con 0x7f46b013faa0 2023-10-06T05:23:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.342+0000 7f46b559b700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4694066f30 0x7f46940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.343+0000 7f46b559b700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4694066f30 0x7f46940693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f46ac000c00 tx=0x7f46ac007ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.346+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46b00b03e0 con 0x7f46b013faa0 2023-10-06T05:23:02.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.550+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46b004c380 con 0x7f46b013faa0 2023-10-06T05:23:02.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.550+0000 7f46a57fa700 1 -- 172.21.15.33:0/874083468 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f46a80537d0 con 0x7f46b013faa0 2023-10-06T05:23:02.553 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:02.553 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":113,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:01.608348+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4694066f30 msgr2=0x7f46940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4694066f30 0x7f46940693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f46ac000c00 tx=0x7f46ac007ed0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 msgr2=0x7f46b00b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 0x7f46b00b06c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f46b00719c0 tx=0x7f46a8018000 comp rx=0 tx=0).stop 2023-10-06T05:23:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 shutdown_connections 2023-10-06T05:23:02.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4694066f30 0x7f46940693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b013faa0 0x7f46b00b06c0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b00c8050 0x7f46b00aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 --2- 172.21.15.33:0/874083468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b00cb710 0x7f46b00c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 >> 172.21.15.33:0/874083468 conn(0x7f46b006d030 msgr2=0x7f46b00c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 shutdown_connections 2023-10-06T05:23:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.553+0000 7f46b659d700 1 -- 172.21.15.33:0/874083468 wait complete. 2023-10-06T05:23:02.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:23:02.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:23:02.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.763+0000 7f208af9d700 1 Processor -- start 2023-10-06T05:23:02.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.763+0000 7f208af9d700 1 -- start start 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.764+0000 7f208af9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f20840af540 0x7f20840af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.764+0000 7f208af9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f20840cd5e0 0x7f208415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.764+0000 7f208af9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208415d660 con 0x7f208415a650 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.764+0000 7f208af9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20840ce890 con 0x7f20840cd5e0 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.764+0000 7f208af9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208415aac0 con 0x7f20840af540 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208a79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f20840cd5e0 0x7f208415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208979a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f208415a650 0x7f20840cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33878/0 (socket says 172.21.15.33:33878) 2023-10-06T05:23:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208979a700 1 -- 172.21.15.33:0/2784971790 learned_addr learned my addr 172.21.15.33:0/2784971790 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f2089f9b700 1 --2- 172.21.15.33:0/2784971790 >> v2:172.21.15.167:3300/0 conn(0x7f20840af540 0x7f20840af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208a79c700 1 -- 172.21.15.33:0/2784971790 >> v2:172.21.15.167:3300/0 conn(0x7f20840af540 msgr2=0x7f20840af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208a79c700 1 --2- 172.21.15.33:0/2784971790 >> v2:172.21.15.167:3300/0 conn(0x7f20840af540 0x7f20840af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208a79c700 1 -- 172.21.15.33:0/2784971790 >> v1:172.21.15.33:6789/0 conn(0x7f208415a650 legacy=0x7f20840cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f208a79c700 1 -- 172.21.15.33:0/2784971790 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208415d890 con 0x7f20840cd5e0 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.765+0000 7f2088f99700 1 -- 172.21.15.33:0/2784971790 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582409656 0 0) 0x7f208415d660 con 0x7f208415a650 2023-10-06T05:23:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208a79c700 1 --2- 172.21.15.33:0/2784971790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208415d120 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f20840c7530 tx=0x7f207c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e64369330e5b260 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f2088f99700 1 -- 172.21.15.33:0/2784971790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00e070 con 0x7f20840cd5e0 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f2088f99700 1 -- 172.21.15.33:0/2784971790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c00bb60 con 0x7f20840cd5e0 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f2088f99700 1 -- 172.21.15.33:0/2784971790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c012430 con 0x7f20840cd5e0 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208af9d700 1 -- 172.21.15.33:0/2784971790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 msgr2=0x7f208415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208af9d700 1 --2- 172.21.15.33:0/2784971790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208415d120 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f20840c7530 tx=0x7f207c009780 comp rx=0 tx=0).stop 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208af9d700 1 -- 172.21.15.33:0/2784971790 shutdown_connections 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208af9d700 1 --2- 172.21.15.33:0/2784971790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208415d120 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208af9d700 1 --2- 172.21.15.33:0/2784971790 >> v2:172.21.15.167:3300/0 conn(0x7f20840af540 0x7f20840af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.766+0000 7f208af9d700 1 -- 172.21.15.33:0/2784971790 >> 172.21.15.33:0/2784971790 conn(0x7f208406d9b0 msgr2=0x7f208406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.767+0000 7f208af9d700 1 -- 172.21.15.33:0/2784971790 shutdown_connections 2023-10-06T05:23:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.767+0000 7f208af9d700 1 -- 172.21.15.33:0/2784971790 wait complete. 2023-10-06T05:23:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.768+0000 7f208af9d700 1 Processor -- start 2023-10-06T05:23:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.768+0000 7f208af9d700 1 -- start start 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20840af540 0x7f20840b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208af9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208416d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208af9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f208415a650 0x7f208415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208af9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840c7580 con 0x7f20840cd5e0 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208af9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208416d730 con 0x7f208415a650 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208416da00 con 0x7f20840af540 2023-10-06T05:23:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208416d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208416d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35548/0 (socket says 172.21.15.33:35548) 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.769+0000 7f208979a700 1 -- 172.21.15.33:0/2953584322 learned_addr learned my addr 172.21.15.33:0/2953584322 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f2089f9b700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20840af540 0x7f20840b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208979a700 1 -- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20840af540 msgr2=0x7f20840b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208979a700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20840af540 0x7f20840b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208a79c700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f208415a650 0x7f208415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208979a700 1 -- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f208415a650 msgr2=0x7f208415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208979a700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f208415a650 0x7f208415c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208979a700 1 -- 172.21.15.33:0/2953584322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c009a20 con 0x7f20840cd5e0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f2089f9b700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20840af540 0x7f20840b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208a79c700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f208415a650 0x7f208415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208979a700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208416d1f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f207400c7e0 tx=0x7f2074007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074005b30 con 0x7f20840cd5e0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2074005c90 con 0x7f20840cd5e0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208416dce0 con 0x7f20840cd5e0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.770+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074007300 con 0x7f20840cd5e0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.771+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20840c77d0 con 0x7f20840cd5e0 2023-10-06T05:23:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.772+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20840c79f0 con 0x7f20840cd5e0 2023-10-06T05:23:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.773+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f207401b400 con 0x7f20840cd5e0 2023-10-06T05:23:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.773+0000 7f2079ffb700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2068066f30 0x7f20680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.773+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2074083ec0 con 0x7f20840cd5e0 2023-10-06T05:23:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.774+0000 7f2089f9b700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2068066f30 0x7f20680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.775+0000 7f2089f9b700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2068066f30 0x7f20680693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2080006f00 tx=0x7f2080008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.778+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2074051300 con 0x7f20840cd5e0 2023-10-06T05:23:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.979+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f208404c380 con 0x7f20840cd5e0 2023-10-06T05:23:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.980+0000 7f2079ffb700 1 -- 172.21.15.33:0/2953584322 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2074054920 con 0x7f20840cd5e0 2023-10-06T05:23:02.982 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2068066f30 msgr2=0x7f20680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2068066f30 0x7f20680693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2080006f00 tx=0x7f2080008040 comp rx=0 tx=0).stop 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 msgr2=0x7f208416d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208416d1f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f207400c7e0 tx=0x7f2074007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 shutdown_connections 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2068066f30 0x7f20680693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f208415a650 0x7f208415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20840af540 0x7f20840b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 --2- 172.21.15.33:0/2953584322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20840cd5e0 0x7f208416d1f0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 >> 172.21.15.33:0/2953584322 conn(0x7f208406d9b0 msgr2=0x7f2084158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 shutdown_connections 2023-10-06T05:23:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:02.982+0000 7f208af9d700 1 -- 172.21.15.33:0/2953584322 wait complete. 2023-10-06T05:23:03.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:23:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.190+0000 7f765db9d700 1 Processor -- start 2023-10-06T05:23:03.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.190+0000 7f765db9d700 1 -- start start 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.191+0000 7f765db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.191+0000 7f765db9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76580ce890 con 0x7f765815a650 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.191+0000 7f765db9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765815aad0 con 0x7f76580af540 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.191+0000 7f765db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765815d890 con 0x7f76580cd5e0 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f765d39c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f765815a650 0x7f76580ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33880/0 (socket says 172.21.15.33:33880) 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f765d39c700 1 -- 172.21.15.33:0/2431923301 learned_addr learned my addr 172.21.15.33:0/2431923301 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f7657fff700 1 --2- 172.21.15.33:0/2431923301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059287296 0 0) 0x7f76580ce890 con 0x7f765815a650 2023-10-06T05:23:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7648003570 con 0x7f765815a650 2023-10-06T05:23:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165225312 0 0) 0x7f765815aad0 con 0x7f76580af540 2023-10-06T05:23:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76580ce890 con 0x7f76580af540 2023-10-06T05:23:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f7657fff700 1 -- 172.21.15.33:0/2431923301 >> v1:172.21.15.167:6789/0 conn(0x7f76580af540 legacy=0x7f76580cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f7657fff700 1 -- 172.21.15.33:0/2431923301 >> v1:172.21.15.33:6789/0 conn(0x7f765815a650 legacy=0x7f76580ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f7657fff700 1 -- 172.21.15.33:0/2431923301 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765815db70 con 0x7f76580cd5e0 2023-10-06T05:23:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.192+0000 7f7657fff700 1 --2- 172.21.15.33:0/2431923301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580af910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7644009910 tx=0x7f76440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa55899a2258e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644011630 con 0x7f76580cd5e0 2023-10-06T05:23:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644011790 con 0x7f76580cd5e0 2023-10-06T05:23:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f76577fe700 1 -- 172.21.15.33:0/2431923301 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644011b30 con 0x7f76580cd5e0 2023-10-06T05:23:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f765db9d700 1 -- 172.21.15.33:0/2431923301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 msgr2=0x7f76580af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f765db9d700 1 --2- 172.21.15.33:0/2431923301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580af910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7644009910 tx=0x7f76440091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f765db9d700 1 -- 172.21.15.33:0/2431923301 shutdown_connections 2023-10-06T05:23:03.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f765db9d700 1 --2- 172.21.15.33:0/2431923301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580af910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f765db9d700 1 -- 172.21.15.33:0/2431923301 >> 172.21.15.33:0/2431923301 conn(0x7f765806d9b0 msgr2=0x7f765806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:03.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.193+0000 7f765db9d700 1 -- 172.21.15.33:0/2431923301 shutdown_connections 2023-10-06T05:23:03.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.194+0000 7f765db9d700 1 -- 172.21.15.33:0/2431923301 wait complete. 2023-10-06T05:23:03.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.195+0000 7f765db9d700 1 Processor -- start 2023-10-06T05:23:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.195+0000 7f765db9d700 1 -- start start 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76580af540 0x7f7658148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 0x7f7658158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765816d070 con 0x7f765815a650 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765816d310 con 0x7f76580af540 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765816d5f0 con 0x7f76580cd5e0 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 0x7f7658158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 0x7f7658158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35550/0 (socket says 172.21.15.33:35550) 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.196+0000 7f765d39c700 1 -- 172.21.15.33:0/2397615071 learned_addr learned my addr 172.21.15.33:0/2397615071 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f7657fff700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765d39c700 1 -- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 msgr2=0x7f76580c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765cb9b700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76580af540 0x7f7658148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765d39c700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765d39c700 1 -- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76580af540 msgr2=0x7f7658148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765d39c700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76580af540 0x7f7658148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765d39c700 1 -- 172.21.15.33:0/2397615071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644009420 con 0x7f765815a650 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f7657fff700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765cb9b700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76580af540 0x7f7658148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765d39c700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 0x7f7658158b70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f765000cc80 tx=0x7f7650001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000ce90 con 0x7f765815a650 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000ab70 con 0x7f765815a650 2023-10-06T05:23:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500070f0 con 0x7f765815a650 2023-10-06T05:23:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765816d8d0 con 0x7f765815a650 2023-10-06T05:23:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.197+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658157590 con 0x7f765815a650 2023-10-06T05:23:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.198+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765816ddf0 con 0x7f765815a650 2023-10-06T05:23:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.199+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76500074a0 con 0x7f765815a650 2023-10-06T05:23:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.200+0000 7f7654ff9700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f763c066f30 0x7f763c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.200+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f76500848b0 con 0x7f765815a650 2023-10-06T05:23:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.201+0000 7f765cb9b700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f763c066f30 0x7f763c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.201+0000 7f765cb9b700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f763c066f30 0x7f763c0693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f764c007fd0 tx=0x7f764c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.204+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f765816ddf0 con 0x7f765815a650 2023-10-06T05:23:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.405+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f765816ddf0 con 0x7f765815a650 2023-10-06T05:23:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.406+0000 7f7654ff9700 1 -- 172.21.15.33:0/2397615071 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f765816ddf0 con 0x7f765815a650 2023-10-06T05:23:03.409 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:23:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f763c066f30 msgr2=0x7f763c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f763c066f30 0x7f763c0693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f764c007fd0 tx=0x7f764c009040 comp rx=0 tx=0).stop 2023-10-06T05:23:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 msgr2=0x7f7658158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 0x7f7658158b70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f765000cc80 tx=0x7f7650001ae0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 shutdown_connections 2023-10-06T05:23:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f763c066f30 0x7f763c0693e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765815a650 0x7f7658158b70 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76580af540 0x7f7658148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 --2- 172.21.15.33:0/2397615071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580cd5e0 0x7f76580c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 >> 172.21.15.33:0/2397615071 conn(0x7f765806d9b0 msgr2=0x7f76580c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 shutdown_connections 2023-10-06T05:23:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.408+0000 7f765db9d700 1 -- 172.21.15.33:0/2397615071 wait complete. 2023-10-06T05:23:03.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:23:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.612+0000 7fe7beb9d700 1 Processor -- start 2023-10-06T05:23:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.612+0000 7fe7beb9d700 1 -- start start 2023-10-06T05:23:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.613+0000 7fe7beb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe7b80b0a40 0x7fe7b806ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.613+0000 7fe7beb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b815e2d0 con 0x7fe7b80b0e10 2023-10-06T05:23:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7beb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b815d6c0 con 0x7fe7b815a0a0 2023-10-06T05:23:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7beb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b815d9a0 con 0x7fe7b80b0a40 2023-10-06T05:23:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7bd39a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7b80b0e10 0x7fe7b80b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33894/0 (socket says 172.21.15.33:33894) 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7bd39a700 1 -- 172.21.15.33:0/3111175294 learned_addr learned my addr 172.21.15.33:0/3111175294 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731106232 0 0) 0x7fe7b815e2d0 con 0x7fe7b80b0e10 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a4003570 con 0x7fe7b80b0e10 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.614+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/3111175294 >> v2:172.21.15.167:3300/0 conn(0x7fe7b80b0a40 0x7fe7b806ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651324375 0 0) 0x7fe7a4003570 con 0x7fe7b80b0e10 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 >> v2:172.21.15.167:3300/0 conn(0x7fe7b80b0a40 msgr2=0x7fe7b806ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 --2- 172.21.15.33:0/3111175294 >> v2:172.21.15.167:3300/0 conn(0x7fe7b80b0a40 0x7fe7b806ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 >> v1:172.21.15.167:6789/0 conn(0x7fe7b815a0a0 legacy=0x7fe7b815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b815dc80 con 0x7fe7b80b0e10 2023-10-06T05:23:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe7a8001470 con 0x7fe7b80b0e10 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 >> v1:172.21.15.33:6789/0 conn(0x7fe7b80b0e10 legacy=0x7fe7b80b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a4003a60 0x7fe7a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7a4004810 0x7fe7a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a40051c0 0x7fe7a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b815d9a0 con 0x7fe7a4003a60 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a400c160 con 0x7fe7a4004810 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a400c320 con 0x7fe7a40051c0 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a4003a60 0x7fe7a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bd39a700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7a4004810 0x7fe7a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7be39c700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a40051c0 0x7fe7a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bdb9b700 1 -- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a40051c0 msgr2=0x7fe7a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a40051c0 0x7fe7a400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bdb9b700 1 -- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7a4004810 msgr2=0x7fe7a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7a4004810 0x7fe7a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.615+0000 7fe7bdb9b700 1 -- 172.21.15.33:0/3111175294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4003570 con 0x7fe7a4003a60 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7be39c700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a40051c0 0x7fe7a400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7be39c700 1 -- 172.21.15.33:0/3111175294 reap_dead start 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a4003a60 0x7fe7a4003e70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009c00 tx=0x7fe7b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c805f0075b8e54af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400e710 con 0x7fe7a4003a60 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b400e870 con 0x7fe7a4003a60 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7bcb99700 1 -- 172.21.15.33:0/3111175294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400ec30 con 0x7fe7a4003a60 2023-10-06T05:23:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7beb9d700 1 -- 172.21.15.33:0/3111175294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a4003a60 msgr2=0x7fe7a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.616+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a4003a60 0x7fe7a4003e70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009c00 tx=0x7fe7b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.617+0000 7fe7beb9d700 1 -- 172.21.15.33:0/3111175294 shutdown_connections 2023-10-06T05:23:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.617+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/3111175294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7a4003a60 0x7fe7a4003e70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.617+0000 7fe7beb9d700 1 -- 172.21.15.33:0/3111175294 >> 172.21.15.33:0/3111175294 conn(0x7fe7b806e0f0 msgr2=0x7fe7b80675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.617+0000 7fe7beb9d700 1 -- 172.21.15.33:0/3111175294 shutdown_connections 2023-10-06T05:23:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.617+0000 7fe7beb9d700 1 -- 172.21.15.33:0/3111175294 wait complete. 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.618+0000 7fe7beb9d700 1 Processor -- start 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.618+0000 7fe7beb9d700 1 -- start start 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7beb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a4003a60 0x7fe7b8169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7beb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b816aee0 0x7fe7b8151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7beb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 0x7fe7b80ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7beb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8151d30 con 0x7fe7b814e910 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7beb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b816aa50 con 0x7fe7b816aee0 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7beb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b816a170 con 0x7fe7a4003a60 2023-10-06T05:23:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7be39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 0x7fe7b80ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7be39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 0x7fe7b80ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35560/0 (socket says 172.21.15.33:35560) 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7be39c700 1 -- 172.21.15.33:0/201900817 learned_addr learned my addr 172.21.15.33:0/201900817 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7bd39a700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b816aee0 0x7fe7b8151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.619+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a4003a60 0x7fe7b8169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7be39c700 1 -- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a4003a60 msgr2=0x7fe7b8169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7be39c700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a4003a60 0x7fe7b8169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7be39c700 1 -- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b816aee0 msgr2=0x7fe7b8151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7be39c700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b816aee0 0x7fe7b8151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7be39c700 1 -- 172.21.15.33:0/201900817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b8151a50 con 0x7fe7b814e910 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7bd39a700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b816aee0 0x7fe7b8151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a4003a60 0x7fe7b8169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7be39c700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 0x7fe7b80ade50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80696b0 tx=0x7fe7b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0017070 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0006ba0 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b816a400 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000b070 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.620+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b8151660 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.621+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b806fcc0 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.622+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7b0003030 con 0x7fe7b814e910 2023-10-06T05:23:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.624+0000 7fe7adffb700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe79c066f30 0x7fe79c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.624+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe7b0083bd0 con 0x7fe7b814e910 2023-10-06T05:23:03.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.624+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe79c066f30 0x7fe79c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.625+0000 7fe7bdb9b700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe79c066f30 0x7fe79c0693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40091e0 tx=0x7fe7b401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.628+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7b0050f90 con 0x7fe7b814e910 2023-10-06T05:23:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.833+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe7b816abc0 con 0x7fe7b814e910 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.833+0000 7fe7adffb700 1 -- 172.21.15.33:0/201900817 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe7b00545b0 con 0x7fe7b814e910 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:23:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:23:03.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:03.837 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:23:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe79c066f30 msgr2=0x7fe79c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe79c066f30 0x7fe79c0693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40091e0 tx=0x7fe7b401a000 comp rx=0 tx=0).stop 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 msgr2=0x7fe7b80ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 0x7fe7b80ade50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80696b0 tx=0x7fe7b0009040 comp rx=0 tx=0).stop 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 shutdown_connections 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe79c066f30 0x7fe79c0693e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b814e910 0x7fe7b80ade50 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7a4003a60 0x7fe7b8169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.835+0000 7fe7beb9d700 1 --2- 172.21.15.33:0/201900817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b816aee0 0x7fe7b8151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.836+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 >> 172.21.15.33:0/201900817 conn(0x7fe7b806e0f0 msgr2=0x7fe7b806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.836+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 shutdown_connections 2023-10-06T05:23:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:03.836+0000 7fe7beb9d700 1 -- 172.21.15.33:0/201900817 wait complete. 2023-10-06T05:23:03.856 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:23:03.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:23:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.045+0000 7fab29b9d700 1 Processor -- start 2023-10-06T05:23:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.045+0000 7fab29b9d700 1 -- start start 2023-10-06T05:23:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.046+0000 7fab29b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fab24159400 0x7fab240a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.046+0000 7fab29b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab2406dbf0 0x7fab24067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.046+0000 7fab29b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240a9dd0 con 0x7fab241597d0 2023-10-06T05:23:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.046+0000 7fab29b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2406d760 con 0x7fab2406dbf0 2023-10-06T05:23:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.046+0000 7fab29b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240aa8c0 con 0x7fab24159400 2023-10-06T05:23:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab2406dbf0 0x7fab24067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab2406dbf0 0x7fab24067b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35566/0 (socket says 172.21.15.33:35566) 2023-10-06T05:23:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 -- 172.21.15.33:0/389185128 learned_addr learned my addr 172.21.15.33:0/389185128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab28b9b700 1 --2- 172.21.15.33:0/389185128 >> v2:172.21.15.167:3300/0 conn(0x7fab24159400 0x7fab240a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 -- 172.21.15.33:0/389185128 >> v2:172.21.15.167:3300/0 conn(0x7fab24159400 msgr2=0x7fab240a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 --2- 172.21.15.33:0/389185128 >> v2:172.21.15.167:3300/0 conn(0x7fab24159400 0x7fab240a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 -- 172.21.15.33:0/389185128 >> v1:172.21.15.167:6789/0 conn(0x7fab241597d0 legacy=0x7fab2406a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab23fff700 1 -- 172.21.15.33:0/389185128 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240aa5e0 con 0x7fab2406dbf0 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.047+0000 7fab28b9b700 1 --2- 172.21.15.33:0/389185128 >> v2:172.21.15.167:3300/0 conn(0x7fab24159400 0x7fab240a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab23fff700 1 --2- 172.21.15.33:0/389185128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2406dbf0 0x7fab24067b50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fab100099b0 tx=0x7fab10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f647e209f97643a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab237fe700 1 -- 172.21.15.33:0/389185128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000b820 con 0x7fab2406dbf0 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab237fe700 1 -- 172.21.15.33:0/389185128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1000b980 con 0x7fab2406dbf0 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab237fe700 1 -- 172.21.15.33:0/389185128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000bd20 con 0x7fab2406dbf0 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab29b9d700 1 -- 172.21.15.33:0/389185128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2406dbf0 msgr2=0x7fab24067b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab29b9d700 1 --2- 172.21.15.33:0/389185128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2406dbf0 0x7fab24067b50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fab100099b0 tx=0x7fab10009250 comp rx=0 tx=0).stop 2023-10-06T05:23:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab29b9d700 1 -- 172.21.15.33:0/389185128 shutdown_connections 2023-10-06T05:23:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab29b9d700 1 --2- 172.21.15.33:0/389185128 >> v2:172.21.15.167:3300/0 conn(0x7fab24159400 0x7fab240a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab29b9d700 1 --2- 172.21.15.33:0/389185128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2406dbf0 0x7fab24067b50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.048+0000 7fab29b9d700 1 -- 172.21.15.33:0/389185128 >> 172.21.15.33:0/389185128 conn(0x7fab2406ebf0 msgr2=0x7fab2406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.049+0000 7fab29b9d700 1 -- 172.21.15.33:0/389185128 shutdown_connections 2023-10-06T05:23:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.049+0000 7fab29b9d700 1 -- 172.21.15.33:0/389185128 wait complete. 2023-10-06T05:23:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.050+0000 7fab29b9d700 1 Processor -- start 2023-10-06T05:23:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.050+0000 7fab29b9d700 1 -- start start 2023-10-06T05:23:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab29b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2406dbf0 0x7fab240acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab29b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab24159400 0x7fab2414fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab29b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 0x7fab24150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab29b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24155f40 con 0x7fab241597d0 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab29b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240ae300 con 0x7fab2406dbf0 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab29b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab24153770 con 0x7fab24159400 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab2939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 0x7fab24150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab2939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 0x7fab24150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35578/0 (socket says 172.21.15.33:35578) 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.051+0000 7fab2939c700 1 -- 172.21.15.33:0/3799197681 learned_addr learned my addr 172.21.15.33:0/3799197681 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab28b9b700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2406dbf0 0x7fab240acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab23fff700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab24159400 0x7fab2414fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab2939c700 1 -- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab24159400 msgr2=0x7fab2414fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab2939c700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab24159400 0x7fab2414fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab2939c700 1 -- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2406dbf0 msgr2=0x7fab240acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab2939c700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2406dbf0 0x7fab240acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab2939c700 1 -- 172.21.15.33:0/3799197681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100094f0 con 0x7fab241597d0 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab2939c700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 0x7fab24150070 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c8e0 tx=0x7fab1c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00ce80 con 0x7fab241597d0 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c00a7e0 con 0x7fab241597d0 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00ab80 con 0x7fab241597d0 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.052+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240ad6f0 con 0x7fab241597d0 2023-10-06T05:23:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.053+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab240adcb0 con 0x7fab241597d0 2023-10-06T05:23:04.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.053+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab241538e0 con 0x7fab241597d0 2023-10-06T05:23:04.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.054+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab1c012420 con 0x7fab241597d0 2023-10-06T05:23:04.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.055+0000 7fab20ff9700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab08066f30 0x7fab080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:04.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.055+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fab1c084550 con 0x7fab241597d0 2023-10-06T05:23:04.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.056+0000 7fab28b9b700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab08066f30 0x7fab080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.056+0000 7fab28b9b700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab08066f30 0x7fab080693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fab18006f00 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.060+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab241538e0 con 0x7fab241597d0 2023-10-06T05:23:04.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:04.265+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fab240a9dd0 con 0x7fab241597d0 2023-10-06T05:23:05.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.174+0000 7fab20ff9700 1 -- 172.21.15.33:0/3799197681 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7fab240a9dd0 con 0x7fab241597d0 2023-10-06T05:23:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab08066f30 msgr2=0x7fab080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab08066f30 0x7fab080693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fab18006f00 tx=0x7fab18008040 comp rx=0 tx=0).stop 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 msgr2=0x7fab24150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 0x7fab24150070 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c8e0 tx=0x7fab1c001710 comp rx=0 tx=0).stop 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 shutdown_connections 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab08066f30 0x7fab080693e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab241597d0 0x7fab24150070 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2406dbf0 0x7fab240acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 --2- 172.21.15.33:0/3799197681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab24159400 0x7fab2414fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.180+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 >> 172.21.15.33:0/3799197681 conn(0x7fab2406ebf0 msgr2=0x7fab2414ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.181+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 shutdown_connections 2023-10-06T05:23:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.181+0000 7fab29b9d700 1 -- 172.21.15.33:0/3799197681 wait complete. 2023-10-06T05:23:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:23:05.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:23:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.402+0000 7f4b87fff700 1 Processor -- start 2023-10-06T05:23:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.402+0000 7f4b87fff700 1 -- start start 2023-10-06T05:23:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b87fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b88155bd0 0x7f4b880c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b87fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b880ac460 0x7f4b880ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b87fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8806aab0 con 0x7f4b8806af40 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b87fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880ace20 con 0x7f4b88155bd0 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b87fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b88069ea0 con 0x7f4b880ac460 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b86ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b88155bd0 0x7f4b880c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b86ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b88155bd0 0x7f4b880c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35584/0 (socket says 172.21.15.33:35584) 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.403+0000 7f4b86ffd700 1 -- 172.21.15.33:0/4107521435 learned_addr learned my addr 172.21.15.33:0/4107521435 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.404+0000 7f4b867fc700 1 --2- 172.21.15.33:0/4107521435 >> v2:172.21.15.167:3300/0 conn(0x7f4b880ac460 0x7f4b880ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.404+0000 7f4b86ffd700 1 -- 172.21.15.33:0/4107521435 >> v2:172.21.15.167:3300/0 conn(0x7f4b880ac460 msgr2=0x7f4b880ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.404+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/4107521435 >> v2:172.21.15.167:3300/0 conn(0x7f4b880ac460 0x7f4b880ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.404+0000 7f4b86ffd700 1 -- 172.21.15.33:0/4107521435 >> v1:172.21.15.167:6789/0 conn(0x7f4b8806af40 legacy=0x7f4b8806b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.404+0000 7f4b86ffd700 1 -- 172.21.15.33:0/4107521435 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8806a180 con 0x7f4b88155bd0 2023-10-06T05:23:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.404+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/4107521435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b88155bd0 0x7f4b880c8dc0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4b88069460 tx=0x7f4b7800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9b4e783e0ec484 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b85ffb700 1 -- 172.21.15.33:0/4107521435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800dc50 con 0x7f4b88155bd0 2023-10-06T05:23:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b85ffb700 1 -- 172.21.15.33:0/4107521435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7800ddb0 con 0x7f4b88155bd0 2023-10-06T05:23:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b85ffb700 1 -- 172.21.15.33:0/4107521435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780196d0 con 0x7f4b88155bd0 2023-10-06T05:23:05.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 -- 172.21.15.33:0/4107521435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b88155bd0 msgr2=0x7f4b880c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 --2- 172.21.15.33:0/4107521435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b88155bd0 0x7f4b880c8dc0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4b88069460 tx=0x7f4b7800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 -- 172.21.15.33:0/4107521435 shutdown_connections 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 --2- 172.21.15.33:0/4107521435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b88155bd0 0x7f4b880c8dc0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 --2- 172.21.15.33:0/4107521435 >> v2:172.21.15.167:3300/0 conn(0x7f4b880ac460 0x7f4b880ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 -- 172.21.15.33:0/4107521435 >> 172.21.15.33:0/4107521435 conn(0x7f4b8806c560 msgr2=0x7f4b8806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 -- 172.21.15.33:0/4107521435 shutdown_connections 2023-10-06T05:23:05.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.405+0000 7f4b87fff700 1 -- 172.21.15.33:0/4107521435 wait complete. 2023-10-06T05:23:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.407+0000 7f4b87fff700 1 Processor -- start 2023-10-06T05:23:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.407+0000 7f4b87fff700 1 -- start start 2023-10-06T05:23:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806af40 0x7f4b88147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 0x7f4b88158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b88155bd0 0x7f4b88158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b87fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c8da0 con 0x7f4b880ac460 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b87fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88147d70 con 0x7f4b88155bd0 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b87fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c8500 con 0x7f4b8806af40 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 0x7f4b88158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 0x7f4b88158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35588/0 (socket says 172.21.15.33:35588) 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 -- 172.21.15.33:0/2876802552 learned_addr learned my addr 172.21.15.33:0/2876802552 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806af40 0x7f4b88147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b877fe700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b88155bd0 0x7f4b88158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 -- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806af40 msgr2=0x7f4b88147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806af40 0x7f4b88147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 -- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b88155bd0 msgr2=0x7f4b88158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b88155bd0 0x7f4b88158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b867fc700 1 -- 172.21.15.33:0/2876802552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7800b790 con 0x7f4b880ac460 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.408+0000 7f4b877fe700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b88155bd0 0x7f4b88158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.409+0000 7f4b867fc700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 0x7f4b88158640 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00d890 tx=0x7f4b7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.409+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0125f0 con 0x7f4b880ac460 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.409+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c012770 con 0x7f4b880ac460 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.409+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c012b30 con 0x7f4b880ac460 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.409+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b880c87e0 con 0x7f4b880ac460 2023-10-06T05:23:05.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.409+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881477d0 con 0x7f4b880ac460 2023-10-06T05:23:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.410+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b88147f00 con 0x7f4b880ac460 2023-10-06T05:23:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.411+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b7c01e460 con 0x7f4b880ac460 2023-10-06T05:23:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.412+0000 7f4b6effd700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.412+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4b7c084b60 con 0x7f4b880ac460 2023-10-06T05:23:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.413+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.414+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b88069460 tx=0x7f4b7801b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.417+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b7c051e50 con 0x7f4b880ac460 2023-10-06T05:23:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:05.619+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b88147b40 con 0x7f4b880ac460 2023-10-06T05:23:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.180+0000 7f4b6effd700 1 -- 172.21.15.33:0/2876802552 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b7c055470 con 0x7f4b880ac460 2023-10-06T05:23:06.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.187+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 msgr2=0x7f4b740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.187+0000 7f4b87fff700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b88069460 tx=0x7f4b7801b000 comp rx=0 tx=0).stop 2023-10-06T05:23:06.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.187+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 msgr2=0x7f4b88158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.187+0000 7f4b87fff700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 0x7f4b88158640 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00d890 tx=0x7f4b7c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.193+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 shutdown_connections 2023-10-06T05:23:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.193+0000 7f4b87fff700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.193+0000 7f4b87fff700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b88155bd0 0x7f4b88158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.193+0000 7f4b87fff700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806af40 0x7f4b88147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.193+0000 7f4b87fff700 1 --2- 172.21.15.33:0/2876802552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880ac460 0x7f4b88158640 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.194+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 >> 172.21.15.33:0/2876802552 conn(0x7f4b8806c560 msgr2=0x7f4b880cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:06.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.200+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 shutdown_connections 2023-10-06T05:23:06.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.201+0000 7f4b87fff700 1 -- 172.21.15.33:0/2876802552 wait complete. 2023-10-06T05:23:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:23:06.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:23:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.417+0000 7f881199d700 1 Processor -- start 2023-10-06T05:23:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.417+0000 7f881199d700 1 -- start start 2023-10-06T05:23:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.418+0000 7f881199d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f880c156370 0x7f880c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.418+0000 7f881199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0b01f0 0x7f880c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.418+0000 7f881199d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880c0ad430 con 0x7f880c0cc7e0 2023-10-06T05:23:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.418+0000 7f881199d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880c0afe20 con 0x7f880c156370 2023-10-06T05:23:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.418+0000 7f881199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0b0b70 con 0x7f880c0b01f0 2023-10-06T05:23:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881119c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0b01f0 0x7f880c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f880bfff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f880c0cc7e0 0x7f880c0c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:38532/0 (socket says 172.21.15.33:38532) 2023-10-06T05:23:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881099b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f880c156370 0x7f880c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f880bfff700 1 -- 172.21.15.33:0/517888753 learned_addr learned my addr 172.21.15.33:0/517888753 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f880b7fe700 1 -- 172.21.15.33:0/517888753 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123983379 0 0) 0x7f880c0ad430 con 0x7f880c0cc7e0 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881099b700 1 -- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0b01f0 msgr2=0x7f880c0b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881099b700 1 --2- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0b01f0 0x7f880c0b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881099b700 1 -- 172.21.15.33:0/517888753 >> v1:172.21.15.167:6789/0 conn(0x7f880c0cc7e0 legacy=0x7f880c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881099b700 1 -- 172.21.15.33:0/517888753 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c06bc70 con 0x7f880c156370 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.419+0000 7f881099b700 1 --2- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c0af8e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f88000098e0 tx=0x7f88000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796351e8e9a3fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f880b7fe700 1 -- 172.21.15.33:0/517888753 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880000b640 con 0x7f880c156370 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f880b7fe700 1 -- 172.21.15.33:0/517888753 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880000b7a0 con 0x7f880c156370 2023-10-06T05:23:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f880b7fe700 1 -- 172.21.15.33:0/517888753 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880000bb40 con 0x7f880c156370 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 -- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 msgr2=0x7f880c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 --2- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c0af8e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f88000098e0 tx=0x7f88000091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 -- 172.21.15.33:0/517888753 shutdown_connections 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 --2- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c0af8e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 --2- 172.21.15.33:0/517888753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0b01f0 0x7f880c0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 -- 172.21.15.33:0/517888753 >> 172.21.15.33:0/517888753 conn(0x7f880c06d030 msgr2=0x7f880c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.420+0000 7f881199d700 1 -- 172.21.15.33:0/517888753 shutdown_connections 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.421+0000 7f881199d700 1 -- 172.21.15.33:0/517888753 wait complete. 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.422+0000 7f881199d700 1 Processor -- start 2023-10-06T05:23:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.422+0000 7f881199d700 1 -- start start 2023-10-06T05:23:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 0x7f880c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0cc7e0 0x7f880c0aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c158850 con 0x7f880c0b01f0 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0c9740 con 0x7f880c156370 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c159ca0 con 0x7f880c0cc7e0 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 0x7f880c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 0x7f880c0cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35598/0 (socket says 172.21.15.33:35598) 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881099b700 1 -- 172.21.15.33:0/3827914789 learned_addr learned my addr 172.21.15.33:0/3827914789 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f881119c700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.423+0000 7f880bfff700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0cc7e0 0x7f880c0aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881099b700 1 -- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0cc7e0 msgr2=0x7f880c0aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881099b700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0cc7e0 0x7f880c0aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881099b700 1 -- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 msgr2=0x7f880c146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881099b700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881099b700 1 -- 172.21.15.33:0/3827914789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800009420 con 0x7f880c0b01f0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881099b700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 0x7f880c0cb730 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8800009f80 tx=0x7f8800017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880000b910 con 0x7f880c0b01f0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880000ba70 con 0x7f880c0b01f0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c158a40 con 0x7f880c0b01f0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800024670 con 0x7f880c0b01f0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.424+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c159000 con 0x7f880c0b01f0 2023-10-06T05:23:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.425+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c0c9510 con 0x7f880c0b01f0 2023-10-06T05:23:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.427+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8800011420 con 0x7f880c0b01f0 2023-10-06T05:23:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.427+0000 7f8808ff9700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87f0066f30 0x7f87f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.427+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f88000862e0 con 0x7f880c0b01f0 2023-10-06T05:23:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.428+0000 7f880bfff700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87f0066f30 0x7f87f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.429+0000 7f880bfff700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87f0066f30 0x7f87f00693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f880c14c4f0 tx=0x7f87f8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:06.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.431+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8800053460 con 0x7f880c0b01f0 2023-10-06T05:23:06.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:06.681+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f880c04c380 con 0x7f880c0b01f0 2023-10-06T05:23:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.182+0000 7f8808ff9700 1 -- 172.21.15.33:0/3827914789 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8800056a80 con 0x7f880c0b01f0 2023-10-06T05:23:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.194+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87f0066f30 msgr2=0x7f87f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.194+0000 7f881199d700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87f0066f30 0x7f87f00693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f880c14c4f0 tx=0x7f87f8007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.194+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 msgr2=0x7f880c0cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.194+0000 7f881199d700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 0x7f880c0cb730 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8800009f80 tx=0x7f8800017000 comp rx=0 tx=0).stop 2023-10-06T05:23:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.196+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 shutdown_connections 2023-10-06T05:23:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.196+0000 7f881199d700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f87f0066f30 0x7f87f00693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.196+0000 7f881199d700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c156370 0x7f880c146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.196+0000 7f881199d700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f880c0b01f0 0x7f880c0cb730 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.196+0000 7f881199d700 1 --2- 172.21.15.33:0/3827914789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f880c0cc7e0 0x7f880c0aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.196+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 >> 172.21.15.33:0/3827914789 conn(0x7f880c06d030 msgr2=0x7f880c0ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.199+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 shutdown_connections 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.200+0000 7f881199d700 1 -- 172.21.15.33:0/3827914789 wait complete. 2023-10-06T05:23:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T05:23:07.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:07.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.952+0000 7f0ac3fff700 1 Processor -- start 2023-10-06T05:23:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.952+0000 7f0ac3fff700 1 -- start start 2023-10-06T05:23:08.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ac40b0a40 0x7f0ac406ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ac40b0e10 0x7f0ac40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac415e2d0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac415d6c0 con 0x7f0ac40b0a40 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac415d9a0 con 0x7f0ac40b0e10 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ac40b0a40 0x7f0ac406ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ac40b0a40 0x7f0ac406ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35612/0 (socket says 172.21.15.33:35612) 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0ac40b2e30 0x7f0ac415a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33902/0 (socket says 172.21.15.33:33902) 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac2ffd700 1 -- 172.21.15.33:0/3975839181 learned_addr learned my addr 172.21.15.33:0/3975839181 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.954+0000 7f0ac27fc700 1 --2- 172.21.15.33:0/3975839181 >> v2:172.21.15.167:3300/0 conn(0x7f0ac40b0e10 0x7f0ac40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac2ffd700 1 -- 172.21.15.33:0/3975839181 >> v2:172.21.15.167:3300/0 conn(0x7f0ac40b0e10 msgr2=0x7f0ac40b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac2ffd700 1 --2- 172.21.15.33:0/3975839181 >> v2:172.21.15.167:3300/0 conn(0x7f0ac40b0e10 0x7f0ac40b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac2ffd700 1 -- 172.21.15.33:0/3975839181 >> v1:172.21.15.33:6789/0 conn(0x7f0ac40b2e30 legacy=0x7f0ac415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac2ffd700 1 -- 172.21.15.33:0/3975839181 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac415dc80 con 0x7f0ac40b0a40 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac1ffb700 1 -- 172.21.15.33:0/3975839181 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581109387 0 0) 0x7f0ac415e2d0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac2ffd700 1 --2- 172.21.15.33:0/3975839181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b0a40 0x7f0ac406ab00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4065020 tx=0x7f0ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357a5f0293fe7860 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac1ffb700 1 -- 172.21.15.33:0/3975839181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400b860 con 0x7f0ac40b0a40 2023-10-06T05:23:08.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac1ffb700 1 -- 172.21.15.33:0/3975839181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab400b9c0 con 0x7f0ac40b0a40 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.955+0000 7f0ac1ffb700 1 -- 172.21.15.33:0/3975839181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400bd80 con 0x7f0ac40b0a40 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 -- 172.21.15.33:0/3975839181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b0a40 msgr2=0x7f0ac406ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/3975839181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b0a40 0x7f0ac406ab00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4065020 tx=0x7f0ab4009250 comp rx=0 tx=0).stop 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 -- 172.21.15.33:0/3975839181 shutdown_connections 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/3975839181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b0a40 0x7f0ac406ab00 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/3975839181 >> v2:172.21.15.167:3300/0 conn(0x7f0ac40b0e10 0x7f0ac40b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 -- 172.21.15.33:0/3975839181 >> 172.21.15.33:0/3975839181 conn(0x7f0ac406e0f0 msgr2=0x7f0ac40675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 -- 172.21.15.33:0/3975839181 shutdown_connections 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.956+0000 7f0ac3fff700 1 -- 172.21.15.33:0/3975839181 wait complete. 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.957+0000 7f0ac3fff700 1 Processor -- start 2023-10-06T05:23:08.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.958+0000 7f0ac3fff700 1 -- start start 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.958+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac40b0a40 0x7f0ac416a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.958+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac40b0e10 0x7f0ac40c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.958+0000 7f0ac3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 0x7f0ac415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac416a750 con 0x7f0ac40b2e30 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac416b200 con 0x7f0ac40b0a40 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac416b5f0 con 0x7f0ac40b0e10 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 0x7f0ac415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 0x7f0ac415c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35626/0 (socket says 172.21.15.33:35626) 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 -- 172.21.15.33:0/570064305 learned_addr learned my addr 172.21.15.33:0/570064305 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac2ffd700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac40b0a40 0x7f0ac416a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac27fc700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac40b0e10 0x7f0ac40c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 -- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac40b0e10 msgr2=0x7f0ac40c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac40b0e10 0x7f0ac40c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 -- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac40b0a40 msgr2=0x7f0ac416a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac40b0a40 0x7f0ac416a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 -- 172.21.15.33:0/570064305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40094f0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac2ffd700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac40b0a40 0x7f0ac416a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.959+0000 7f0ac37fe700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 0x7f0ac415c1f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00cd80 tx=0x7f0abc001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.960+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc005df0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.960+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc00a920 con 0x7f0ac40b2e30 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.960+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00ace0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.960+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac416a960 con 0x7f0ac40b2e30 2023-10-06T05:23:08.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.960+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac416af20 con 0x7f0ac40b2e30 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.961+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac406fcc0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.962+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0abc007300 con 0x7f0ac40b2e30 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.963+0000 7f0aaaffd700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ab0066f30 0x7f0ab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.963+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0abc0848c0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.964+0000 7f0ac2ffd700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ab0066f30 0x7f0ab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.964+0000 7f0ac2ffd700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ab0066f30 0x7f0ab00693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4065020 tx=0x7f0ab4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:07.967+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0abc0519b0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.169+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac404c380 con 0x7f0ac40b2e30 2023-10-06T05:23:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.170+0000 7f0aaaffd700 1 -- 172.21.15.33:0/570064305 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f0abc054fd0 con 0x7f0ac40b2e30 2023-10-06T05:23:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:08.238 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":116,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:06.683218+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:04.306347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:05.660589+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ab0066f30 msgr2=0x7f0ab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ab0066f30 0x7f0ab00693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4065020 tx=0x7f0ab4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 msgr2=0x7f0ac415c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 0x7f0ac415c1f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00cd80 tx=0x7f0abc001b10 comp rx=0 tx=0).stop 2023-10-06T05:23:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 shutdown_connections 2023-10-06T05:23:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ab0066f30 0x7f0ab00693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac40b2e30 0x7f0ac415c1f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac40b0a40 0x7f0ac416a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 --2- 172.21.15.33:0/570064305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac40b0e10 0x7f0ac40c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 >> 172.21.15.33:0/570064305 conn(0x7f0ac406e0f0 msgr2=0x7f0ac40abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 shutdown_connections 2023-10-06T05:23:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.173+0000 7f0ac3fff700 1 -- 172.21.15.33:0/570064305 wait complete. 2023-10-06T05:23:08.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:08.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.533+0000 7f9cf0f9d700 1 Processor -- start 2023-10-06T05:23:08.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.533+0000 7f9cf0f9d700 1 -- start start 2023-10-06T05:23:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9cf0f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec155140 0x7f9cec158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9cf0f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cec0adf50 0x7f9cec0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9cf0f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0c8e40 con 0x7f9cec0c9810 2023-10-06T05:23:08.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9cf0f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0ae6c0 con 0x7f9cec0adf50 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9cf0f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0ae9a0 con 0x7f9cec155140 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cec0adf50 0x7f9cec0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cec0adf50 0x7f9cec0c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35628/0 (socket says 172.21.15.33:35628) 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 -- 172.21.15.33:0/453012412 learned_addr learned my addr 172.21.15.33:0/453012412 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 -- 172.21.15.33:0/453012412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec155140 msgr2=0x7f9cec158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceb7fe700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec155140 0x7f9cec158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec155140 0x7f9cec158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 -- 172.21.15.33:0/453012412 >> v1:172.21.15.167:6789/0 conn(0x7f9cec0c9810 legacy=0x7f9cec0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceaffd700 1 -- 172.21.15.33:0/453012412 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0c8ad0 con 0x7f9cec0adf50 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.535+0000 7f9ceb7fe700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec155140 0x7f9cec158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9ceaffd700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec0adf50 0x7f9cec0c92d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00099b0 tx=0x7f9ce0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44210e5d1be90a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cea7fc700 1 -- 172.21.15.33:0/453012412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000b820 con 0x7f9cec0adf50 2023-10-06T05:23:08.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cea7fc700 1 -- 172.21.15.33:0/453012412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000b980 con 0x7f9cec0adf50 2023-10-06T05:23:08.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cea7fc700 1 -- 172.21.15.33:0/453012412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000bd20 con 0x7f9cec0adf50 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/453012412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec0adf50 msgr2=0x7f9cec0c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec0adf50 0x7f9cec0c92d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00099b0 tx=0x7f9ce0009250 comp rx=0 tx=0).stop 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/453012412 shutdown_connections 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec155140 0x7f9cec158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/453012412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec0adf50 0x7f9cec0c92d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.536+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/453012412 >> 172.21.15.33:0/453012412 conn(0x7f9cec06d030 msgr2=0x7f9cec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.537+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/453012412 shutdown_connections 2023-10-06T05:23:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.537+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/453012412 wait complete. 2023-10-06T05:23:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.538+0000 7f9cf0f9d700 1 Processor -- start 2023-10-06T05:23:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.538+0000 7f9cf0f9d700 1 -- start start 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cf0f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec0adf50 0x7f9cec152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cf0f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cec0c9810 0x7f9cec155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cf0f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 0x7f9cec150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cf0f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec169cd0 con 0x7f9cec155140 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cf0f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec156410 con 0x7f9cec0c9810 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cf0f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec1505a0 con 0x7f9cec0adf50 2023-10-06T05:23:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 0x7f9cec150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 0x7f9cec150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35642/0 (socket says 172.21.15.33:35642) 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.539+0000 7f9cebfff700 1 -- 172.21.15.33:0/3709050520 learned_addr learned my addr 172.21.15.33:0/3709050520 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9ceb7fe700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec0adf50 0x7f9cec152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9ceaffd700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cec0c9810 0x7f9cec155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cebfff700 1 -- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec0adf50 msgr2=0x7f9cec152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cebfff700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec0adf50 0x7f9cec152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cebfff700 1 -- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cec0c9810 msgr2=0x7f9cec155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cebfff700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cec0c9810 0x7f9cec155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cebfff700 1 -- 172.21.15.33:0/3709050520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00094f0 con 0x7f9cec155140 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9ceaffd700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cec0c9810 0x7f9cec155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cebfff700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 0x7f9cec150060 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400c860 tx=0x7f9ce4002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4012630 con 0x7f9cec155140 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce40127b0 con 0x7f9cec155140 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4012b70 con 0x7f9cec155140 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec156690 con 0x7f9cec155140 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.540+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec156c50 con 0x7f9cec155140 2023-10-06T05:23:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.541+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec04c380 con 0x7f9cec155140 2023-10-06T05:23:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.542+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ce401b400 con 0x7f9cec155140 2023-10-06T05:23:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.543+0000 7f9cd37fe700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cd8066f30 0x7f9cd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.543+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ce4084aa0 con 0x7f9cec155140 2023-10-06T05:23:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.544+0000 7f9ceb7fe700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cd8066f30 0x7f9cd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.544+0000 7f9ceb7fe700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cd8066f30 0x7f9cd80693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006fd0 tx=0x7f9cdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.548+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ce4051bc0 con 0x7f9cec155140 2023-10-06T05:23:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.750+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cec06fa40 con 0x7f9cec155140 2023-10-06T05:23:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.751+0000 7f9cd37fe700 1 -- 172.21.15.33:0/3709050520 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f9ce40551e0 con 0x7f9cec155140 2023-10-06T05:23:08.753 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:08.754 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":117,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:08.189738+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:04.306347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:05.660589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cd8066f30 msgr2=0x7f9cd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cd8066f30 0x7f9cd80693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006fd0 tx=0x7f9cdc008040 comp rx=0 tx=0).stop 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 msgr2=0x7f9cec150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 0x7f9cec150060 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400c860 tx=0x7f9ce4002680 comp rx=0 tx=0).stop 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 shutdown_connections 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9cd8066f30 0x7f9cd80693e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cec155140 0x7f9cec150060 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cec0adf50 0x7f9cec152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 --2- 172.21.15.33:0/3709050520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cec0c9810 0x7f9cec155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 >> 172.21.15.33:0/3709050520 conn(0x7f9cec06d030 msgr2=0x7f9cec150ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 shutdown_connections 2023-10-06T05:23:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.753+0000 7f9cf0f9d700 1 -- 172.21.15.33:0/3709050520 wait complete. 2023-10-06T05:23:08.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:23:08.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.961+0000 7fb08239d700 1 Processor -- start 2023-10-06T05:23:08.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.961+0000 7fb08239d700 1 -- start start 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb08239d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb07c0cc7e0 0x7fb07c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb08239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c0ad430 con 0x7fb07c0b01f0 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb08239d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c0afe20 con 0x7fb07c156370 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb08239d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c06bc70 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb080b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb07c0cc7e0 0x7fb07c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb080b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb07c0cc7e0 0x7fb07c0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35658/0 (socket says 172.21.15.33:35658) 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb080b9a700 1 -- 172.21.15.33:0/2192703087 learned_addr learned my addr 172.21.15.33:0/2192703087 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb081b9c700 1 --1- 172.21.15.33:0/2192703087 >> v1:172.21.15.33:6789/0 conn(0x7fb07c0b01f0 0x7fb07c0ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33910/0 (socket says 172.21.15.33:33910) 2023-10-06T05:23:08.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb080b9a700 1 -- 172.21.15.33:0/2192703087 >> v1:172.21.15.167:6789/0 conn(0x7fb07c156370 legacy=0x7fb07c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb080b9a700 1 -- 172.21.15.33:0/2192703087 >> v1:172.21.15.33:6789/0 conn(0x7fb07c0b01f0 legacy=0x7fb07c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.963+0000 7fb080b9a700 1 -- 172.21.15.33:0/2192703087 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c0cccc0 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.964+0000 7fb080b9a700 1 --2- 172.21.15.33:0/2192703087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c0af8e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0099b0 tx=0x7fb06c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae48a2818c3bafb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.964+0000 7fb073fff700 1 -- 172.21.15.33:0/2192703087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c00b860 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.964+0000 7fb073fff700 1 -- 172.21.15.33:0/2192703087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c00b9c0 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.964+0000 7fb073fff700 1 -- 172.21.15.33:0/2192703087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c00bd60 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.964+0000 7fb08239d700 1 -- 172.21.15.33:0/2192703087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 msgr2=0x7fb07c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.964+0000 7fb08239d700 1 --2- 172.21.15.33:0/2192703087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c0af8e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0099b0 tx=0x7fb06c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.965+0000 7fb08239d700 1 -- 172.21.15.33:0/2192703087 shutdown_connections 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.965+0000 7fb08239d700 1 --2- 172.21.15.33:0/2192703087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c0af8e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.965+0000 7fb08239d700 1 -- 172.21.15.33:0/2192703087 >> 172.21.15.33:0/2192703087 conn(0x7fb07c06d030 msgr2=0x7fb07c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.965+0000 7fb08239d700 1 -- 172.21.15.33:0/2192703087 shutdown_connections 2023-10-06T05:23:08.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.965+0000 7fb08239d700 1 -- 172.21.15.33:0/2192703087 wait complete. 2023-10-06T05:23:08.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.966+0000 7fb08239d700 1 Processor -- start 2023-10-06T05:23:08.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.966+0000 7fb08239d700 1 -- start start 2023-10-06T05:23:08.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.967+0000 7fb08239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb07c0b01f0 0x7fb07c169e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.967+0000 7fb08239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.967+0000 7fb08239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb07c156370 0x7fb07c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.967+0000 7fb08239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c154890 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.967+0000 7fb08239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c16a7a0 con 0x7fb07c156370 2023-10-06T05:23:08.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.967+0000 7fb08239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c16aa80 con 0x7fb07c0b01f0 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c14ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35674/0 (socket says 172.21.15.33:35674) 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 -- 172.21.15.33:0/4192327162 learned_addr learned my addr 172.21.15.33:0/4192327162 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 -- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb07c0b01f0 msgr2=0x7fb07c169e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb07c0b01f0 0x7fb07c169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 -- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb07c156370 msgr2=0x7fb07c0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb07c156370 0x7fb07c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 -- 172.21.15.33:0/4192327162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0094f0 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb080b9a700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c14ed00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb07c06d4d0 tx=0x7fb06c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c011ea0 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c01a630 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.968+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c01a9d0 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.969+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c16ad60 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.969+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c154420 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.970+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c154640 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.970+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb06c02d440 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.971+0000 7fb0717fa700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb060066f30 0x7fb0600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.971+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb06c0862e0 con 0x7fb07c0cc7e0 2023-10-06T05:23:08.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.972+0000 7fb08139b700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb060066f30 0x7fb0600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.973+0000 7fb08139b700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb060066f30 0x7fb0600693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb07c15aa50 tx=0x7fb078007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:08.976+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb06c053350 con 0x7fb07c0cc7e0 2023-10-06T05:23:09.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:09.224+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb07c04c380 con 0x7fb07c0cc7e0 2023-10-06T05:23:10.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.203+0000 7fb0717fa700 1 -- 172.21.15.33:0/4192327162 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb06c056970 con 0x7fb07c0cc7e0 2023-10-06T05:23:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.206+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb060066f30 msgr2=0x7fb0600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.206+0000 7fb08239d700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb060066f30 0x7fb0600693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb07c15aa50 tx=0x7fb078007b40 comp rx=0 tx=0).stop 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.206+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 msgr2=0x7fb07c14ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.206+0000 7fb08239d700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c14ed00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb07c06d4d0 tx=0x7fb06c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 shutdown_connections 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb060066f30 0x7fb0600693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb07c156370 0x7fb07c0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb07c0b01f0 0x7fb07c169e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 --2- 172.21.15.33:0/4192327162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb07c0cc7e0 0x7fb07c14ed00 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 >> 172.21.15.33:0/4192327162 conn(0x7fb07c06d030 msgr2=0x7fb07c0ac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 shutdown_connections 2023-10-06T05:23:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.207+0000 7fb08239d700 1 -- 172.21.15.33:0/4192327162 wait complete. 2023-10-06T05:23:10.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:23:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.422+0000 7f00bc99d700 1 Processor -- start 2023-10-06T05:23:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.422+0000 7f00bc99d700 1 -- start start 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.423+0000 7f00bc99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.423+0000 7f00bc99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00b80cc7e0 0x7f00b80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00bc99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b80ad430 con 0x7f00b80b01f0 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00bc99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b80afe20 con 0x7f00b80cc7e0 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00bc99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b806bc70 con 0x7f00b8156370 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00b80cc7e0 0x7f00b80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00b80cc7e0 0x7f00b80c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51640/0 (socket says 172.21.15.33:51640) 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 -- 172.21.15.33:0/3686551653 learned_addr learned my addr 172.21.15.33:0/3686551653 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:10.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b77fe700 1 --2- 172.21.15.33:0/3686551653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 -- 172.21.15.33:0/3686551653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 msgr2=0x7f00b80af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 --2- 172.21.15.33:0/3686551653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b80af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 -- 172.21.15.33:0/3686551653 >> v1:172.21.15.167:6789/0 conn(0x7f00b80b01f0 legacy=0x7f00b80ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.424+0000 7f00b6ffd700 1 -- 172.21.15.33:0/3686551653 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80c92f0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00b6ffd700 1 --2- 172.21.15.33:0/3686551653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b80c8db0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0099b0 tx=0x7f00ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc482e609c9d842 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00b67fc700 1 -- 172.21.15.33:0/3686551653 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00b860 con 0x7f00b80cc7e0 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00b67fc700 1 -- 172.21.15.33:0/3686551653 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00b9c0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00b67fc700 1 -- 172.21.15.33:0/3686551653 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00bd60 con 0x7f00b80cc7e0 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00bc99d700 1 -- 172.21.15.33:0/3686551653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 msgr2=0x7f00b80c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00bc99d700 1 --2- 172.21.15.33:0/3686551653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b80c8db0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0099b0 tx=0x7f00ac009250 comp rx=0 tx=0).stop 2023-10-06T05:23:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00bc99d700 1 -- 172.21.15.33:0/3686551653 shutdown_connections 2023-10-06T05:23:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00bc99d700 1 --2- 172.21.15.33:0/3686551653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b80af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00bc99d700 1 --2- 172.21.15.33:0/3686551653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b80c8db0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.425+0000 7f00bc99d700 1 -- 172.21.15.33:0/3686551653 >> 172.21.15.33:0/3686551653 conn(0x7f00b806d030 msgr2=0x7f00b80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.426+0000 7f00bc99d700 1 -- 172.21.15.33:0/3686551653 shutdown_connections 2023-10-06T05:23:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.426+0000 7f00bc99d700 1 -- 172.21.15.33:0/3686551653 wait complete. 2023-10-06T05:23:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.427+0000 7f00bc99d700 1 Processor -- start 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.427+0000 7f00bc99d700 1 -- start start 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00bc99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00b80b01f0 0x7f00b8151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00bc99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b8150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00bc99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b81549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00bc99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b81578b0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00bc99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b81596b0 con 0x7f00b80b01f0 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00bc99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b8158e10 con 0x7f00b8156370 2023-10-06T05:23:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b8150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b8150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51642/0 (socket says 172.21.15.33:51642) 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00b6ffd700 1 -- 172.21.15.33:0/616073047 learned_addr learned my addr 172.21.15.33:0/616073047 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00b77fe700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00b80b01f0 0x7f00b8151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.428+0000 7f00b7fff700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b81549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b6ffd700 1 -- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 msgr2=0x7f00b81549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b6ffd700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b81549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b6ffd700 1 -- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00b80b01f0 msgr2=0x7f00b8151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b6ffd700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00b80b01f0 0x7f00b8151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b6ffd700 1 -- 172.21.15.33:0/616073047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0094f0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b77fe700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00b80b01f0 0x7f00b8151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00b6ffd700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b8150e00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f00b806d4d0 tx=0x7f00ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac024440 con 0x7f00b80cc7e0 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac0245c0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac024980 con 0x7f00b80cc7e0 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b81590f0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.429+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b8158aa0 con 0x7f00b80cc7e0 2023-10-06T05:23:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.430+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b804c380 con 0x7f00b80cc7e0 2023-10-06T05:23:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.431+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00ac01a420 con 0x7f00b80cc7e0 2023-10-06T05:23:10.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.432+0000 7f009f7fe700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00a4066f30 0x7f00a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.432+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f00ac086500 con 0x7f00b80cc7e0 2023-10-06T05:23:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.433+0000 7f00b77fe700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00a4066f30 0x7f00a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.433+0000 7f00b77fe700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00a4066f30 0x7f00a40693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.436+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00ac053570 con 0x7f00b80cc7e0 2023-10-06T05:23:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:10.685+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00b806f310 con 0x7f00b80cc7e0 2023-10-06T05:23:11.162 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:23:11.160+0000 7f87f9ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.205+0000 7f009f7fe700 1 -- 172.21.15.33:0/616073047 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00ac056b90 con 0x7f00b80cc7e0 2023-10-06T05:23:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00a4066f30 msgr2=0x7f00a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00a4066f30 0x7f00a40693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a8008040 comp rx=0 tx=0).stop 2023-10-06T05:23:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 msgr2=0x7f00b8150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b8150e00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f00b806d4d0 tx=0x7f00ac017000 comp rx=0 tx=0).stop 2023-10-06T05:23:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 shutdown_connections 2023-10-06T05:23:11.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00a4066f30 0x7f00a40693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f00b8156370 0x7f00b81549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00b80b01f0 0x7f00b8151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 --2- 172.21.15.33:0/616073047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00b80cc7e0 0x7f00b8150e00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.208+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 >> 172.21.15.33:0/616073047 conn(0x7f00b806d030 msgr2=0x7f00b80cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.209+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 shutdown_connections 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.209+0000 7f00bc99d700 1 -- 172.21.15.33:0/616073047 wait complete. 2023-10-06T05:23:11.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:11.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.419+0000 7fea28f9d700 1 Processor -- start 2023-10-06T05:23:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.419+0000 7fea28f9d700 1 -- start start 2023-10-06T05:23:11.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.420+0000 7fea28f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 0x7fea2413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.420+0000 7fea28f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea240c9cb0 con 0x7fea240c7940 2023-10-06T05:23:11.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.420+0000 7fea28f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea240ca270 con 0x7fea2406aad0 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.420+0000 7fea28f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2406b660 con 0x7fea2413faa0 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea22ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fea240c7940 0x7fea240c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40348/0 (socket says 172.21.15.33:40348) 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea22ffd700 1 -- 172.21.15.33:0/4257165779 learned_addr learned my addr 172.21.15.33:0/4257165779 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea237fe700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 0x7fea2413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952416157 0 0) 0x7fea240c9cb0 con 0x7fea240c7940 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea0c003570 con 0x7fea240c7940 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 46278783 0 0) 0x7fea0c003570 con 0x7fea240c7940 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 msgr2=0x7fea2413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 0x7fea2413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 >> v1:172.21.15.167:6789/0 conn(0x7fea2406aad0 legacy=0x7fea2406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2406b840 con 0x7fea240c7940 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fea18001470 con 0x7fea240c7940 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.421+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 >> v1:172.21.15.33:6789/0 conn(0x7fea240c7940 legacy=0x7fea240c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea227fc700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 0x7fea0c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea227fc700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea227fc700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea0c004d50 0x7fea0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2406b660 con 0x7fea0c003fd0 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c010070 con 0x7fea0c004d50 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c0101b0 con 0x7fea2413faa0 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea237fe700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 0x7fea0c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea23fff700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea0c004d50 0x7fea0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 -- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 msgr2=0x7fea0c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea2413faa0 0x7fea0c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 -- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea0c004d50 msgr2=0x7fea0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea0c004d50 0x7fea0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 -- 172.21.15.33:0/4257165779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c003570 con 0x7fea0c003fd0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.422+0000 7fea22ffd700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea0c004810 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c2a0 tx=0x7fea240ca1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2314fde1e56b41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800cb70 con 0x7fea0c003fd0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800cd10 con 0x7fea0c003fd0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea227fc700 1 -- 172.21.15.33:0/4257165779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18005b70 con 0x7fea0c003fd0 2023-10-06T05:23:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea28f9d700 1 -- 172.21.15.33:0/4257165779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 msgr2=0x7fea0c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea28f9d700 1 --2- 172.21.15.33:0/4257165779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea0c004810 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c2a0 tx=0x7fea240ca1e0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea23fff700 1 -- 172.21.15.33:0/4257165779 reap_dead start 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea28f9d700 1 -- 172.21.15.33:0/4257165779 shutdown_connections 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea28f9d700 1 -- 172.21.15.33:0/4257165779 >> 172.21.15.33:0/4257165779 conn(0x7fea2406d030 msgr2=0x7fea2406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea28f9d700 1 -- 172.21.15.33:0/4257165779 shutdown_connections 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.423+0000 7fea28f9d700 1 -- 172.21.15.33:0/4257165779 wait complete. 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.425+0000 7fea28f9d700 1 Processor -- start 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.425+0000 7fea28f9d700 1 -- start start 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea28f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea2412fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea28f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea24130390 0x7fea24168c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea28f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea24169190 0x7fea2412ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea28f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea24169610 con 0x7fea0c003fd0 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea28f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2412ea70 con 0x7fea24130390 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea28f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2412de60 con 0x7fea24169190 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea2412fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea2412fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51658/0 (socket says 172.21.15.33:51658) 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 -- 172.21.15.33:0/1744075955 learned_addr learned my addr 172.21.15.33:0/1744075955 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 -- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea24169190 msgr2=0x7fea2412ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea24169190 0x7fea2412ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 -- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea24130390 msgr2=0x7fea24168c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea24130390 0x7fea24168c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.426+0000 7fea237fe700 1 -- 172.21.15.33:0/1744075955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2412e140 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.427+0000 7fea237fe700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea2412fe50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fea14011d40 tx=0x7fea140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.427+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1400fac0 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.427+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1400fc20 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.427+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14014440 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.427+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2412e420 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.427+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea240c6560 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.428+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2404c380 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.429+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea14014a80 con 0x7fea0c003fd0 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.430+0000 7fea0b7fe700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea10066f30 0x7fea100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.430+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fea14018030 con 0x7fea0c003fd0 2023-10-06T05:23:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.431+0000 7fea22ffd700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea10066f30 0x7fea100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.431+0000 7fea22ffd700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea10066f30 0x7fea100693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c2a0 tx=0x7fea18015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.434+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea14052520 con 0x7fea0c003fd0 2023-10-06T05:23:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.681+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea2406bb20 con 0x7fea0c003fd0 2023-10-06T05:23:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.682+0000 7fea0b7fe700 1 -- 172.21.15.33:0/1744075955 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fea1405b030 con 0x7fea0c003fd0 2023-10-06T05:23:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:11.685 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:11.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.684+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea10066f30 msgr2=0x7fea100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.684+0000 7fea28f9d700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea10066f30 0x7fea100693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c2a0 tx=0x7fea18015000 comp rx=0 tx=0).stop 2023-10-06T05:23:11.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.684+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 msgr2=0x7fea2412fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.684+0000 7fea28f9d700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea2412fe50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fea14011d40 tx=0x7fea140091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.684+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 shutdown_connections 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea10066f30 0x7fea100693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea24169190 0x7fea2412ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea0c003fd0 0x7fea2412fe50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 --2- 172.21.15.33:0/1744075955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea24130390 0x7fea24168c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 >> 172.21.15.33:0/1744075955 conn(0x7fea2406d030 msgr2=0x7fea240c17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 shutdown_connections 2023-10-06T05:23:11.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.685+0000 7fea28f9d700 1 -- 172.21.15.33:0/1744075955 wait complete. 2023-10-06T05:23:11.689 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:11.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:11.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.891+0000 7f659af9d700 1 Processor -- start 2023-10-06T05:23:11.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.891+0000 7f659af9d700 1 -- start start 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659af9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f659413faa0 0x7f659413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940c7940 0x7f65940c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659af9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940c9cb0 con 0x7f659406aad0 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659af9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659406b660 con 0x7f659413faa0 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940ca270 con 0x7f65940c7940 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659a79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f659406aad0 0x7f659406af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40362/0 (socket says 172.21.15.33:40362) 2023-10-06T05:23:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.892+0000 7f659a79c700 1 -- 172.21.15.33:0/3691434016 learned_addr learned my addr 172.21.15.33:0/3691434016 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3691434016 >> v2:172.21.15.167:3300/0 conn(0x7f659413faa0 0x7f659413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f659979a700 1 --2- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940c7940 0x7f65940c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6598f99700 1 -- 172.21.15.33:0/3691434016 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531598490 0 0) 0x7f65940c9cb0 con 0x7f659406aad0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6598f99700 1 -- 172.21.15.33:0/3691434016 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6580003570 con 0x7f659406aad0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6599f9b700 1 -- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940c7940 msgr2=0x7f65940c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940c7940 0x7f65940c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6599f9b700 1 -- 172.21.15.33:0/3691434016 >> v1:172.21.15.33:6789/0 conn(0x7f659406aad0 legacy=0x7f659406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.893+0000 7f6599f9b700 1 -- 172.21.15.33:0/3691434016 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659406b840 con 0x7f659413faa0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.894+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659413faa0 0x7f659413fe70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f65900098e0 tx=0x7f65900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff20c8ed868a702 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.894+0000 7f6598f99700 1 -- 172.21.15.33:0/3691434016 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590011630 con 0x7f659413faa0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.894+0000 7f6598f99700 1 -- 172.21.15.33:0/3691434016 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590011790 con 0x7f659413faa0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.894+0000 7f6598f99700 1 -- 172.21.15.33:0/3691434016 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590011b10 con 0x7f659413faa0 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.894+0000 7f659af9d700 1 -- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659413faa0 msgr2=0x7f659413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.894+0000 7f659af9d700 1 --2- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659413faa0 0x7f659413fe70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f65900098e0 tx=0x7f65900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.895+0000 7f659af9d700 1 -- 172.21.15.33:0/3691434016 shutdown_connections 2023-10-06T05:23:11.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.895+0000 7f659af9d700 1 --2- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659413faa0 0x7f659413fe70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.895+0000 7f659af9d700 1 --2- 172.21.15.33:0/3691434016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940c7940 0x7f65940c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.895+0000 7f659af9d700 1 -- 172.21.15.33:0/3691434016 >> 172.21.15.33:0/3691434016 conn(0x7f659406d030 msgr2=0x7f659406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.895+0000 7f659af9d700 1 -- 172.21.15.33:0/3691434016 shutdown_connections 2023-10-06T05:23:11.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.895+0000 7f659af9d700 1 -- 172.21.15.33:0/3691434016 wait complete. 2023-10-06T05:23:11.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.896+0000 7f659af9d700 1 Processor -- start 2023-10-06T05:23:11.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.896+0000 7f659af9d700 1 -- start start 2023-10-06T05:23:11.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659af9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659406aad0 0x7f659413e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659af9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 0x7f6594136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f659413faa0 0x7f659413cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659af9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594137250 con 0x7f65940c7940 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659af9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659413d330 con 0x7f659406aad0 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659413d5f0 con 0x7f659413faa0 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 0x7f6594136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659979a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 0x7f6594136d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51666/0 (socket says 172.21.15.33:51666) 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.897+0000 7f659979a700 1 -- 172.21.15.33:0/3570321114 learned_addr learned my addr 172.21.15.33:0/3570321114 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659a79c700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f659413faa0 0x7f659413cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659406aad0 0x7f659413e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659979a700 1 -- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f659413faa0 msgr2=0x7f659413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659979a700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f659413faa0 0x7f659413cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659979a700 1 -- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659406aad0 msgr2=0x7f659413e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659979a700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659406aad0 0x7f659413e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659979a700 1 -- 172.21.15.33:0/3570321114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009420 con 0x7f65940c7940 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659406aad0 0x7f659413e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659a79c700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f659413faa0 0x7f659413cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659979a700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 0x7f6594136d10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f658400ec80 tx=0x7f65840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658400c9d0 con 0x7f65940c7940 2023-10-06T05:23:11.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658400cb30 con 0x7f65940c7940 2023-10-06T05:23:11.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584013470 con 0x7f65940c7940 2023-10-06T05:23:11.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.898+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659413d8d0 con 0x7f65940c7940 2023-10-06T05:23:11.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.899+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659413de90 con 0x7f65940c7940 2023-10-06T05:23:11.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.900+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659404c380 con 0x7f65940c7940 2023-10-06T05:23:11.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.901+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6584013820 con 0x7f65940c7940 2023-10-06T05:23:11.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.901+0000 7f6589ffb700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.902+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6584084a20 con 0x7f65940c7940 2023-10-06T05:23:11.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.902+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.903+0000 7f6599f9b700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f65900091e0 tx=0x7f6590017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:11.905+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6584051b30 con 0x7f65940c7940 2023-10-06T05:23:12.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.109+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f659406bb20 con 0x7f65940c7940 2023-10-06T05:23:12.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.110+0000 7f6589ffb700 1 -- 172.21.15.33:0/3570321114 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f6584055150 con 0x7f65940c7940 2023-10-06T05:23:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:12.114 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":117,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:08.189738+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:04.306347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:05.660589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:12.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 msgr2=0x7f65780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f65900091e0 tx=0x7f6590017000 comp rx=0 tx=0).stop 2023-10-06T05:23:12.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 msgr2=0x7f6594136d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 0x7f6594136d10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f658400ec80 tx=0x7f65840091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 shutdown_connections 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f659413faa0 0x7f659413cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659406aad0 0x7f659413e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 --2- 172.21.15.33:0/3570321114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940c7940 0x7f6594136d10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.112+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 >> 172.21.15.33:0/3570321114 conn(0x7f659406d030 msgr2=0x7f65940ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.113+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 shutdown_connections 2023-10-06T05:23:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.113+0000 7f659af9d700 1 -- 172.21.15.33:0/3570321114 wait complete. 2023-10-06T05:23:12.133 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:23:12.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:23:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.318+0000 7f915679d700 1 Processor -- start 2023-10-06T05:23:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.318+0000 7f915679d700 1 -- start start 2023-10-06T05:23:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.319+0000 7f915679d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f91500cf3b0 0x7f91501594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.319+0000 7f915679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91500cbd80 con 0x7f91500af0a0 2023-10-06T05:23:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.319+0000 7f915679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91500af5f0 con 0x7f91500ccfc0 2023-10-06T05:23:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.319+0000 7f915679d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91500af8d0 con 0x7f91500cf3b0 2023-10-06T05:23:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9155f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f91500af0a0 0x7f91500b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40368/0 (socket says 172.21.15.33:40368) 2023-10-06T05:23:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9155f9c700 1 -- 172.21.15.33:0/4127320207 learned_addr learned my addr 172.21.15.33:0/4127320207 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9154f9a700 1 --2- 172.21.15.33:0/4127320207 >> v2:172.21.15.167:3300/0 conn(0x7f91500cf3b0 0x7f91501594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620337786 0 0) 0x7f91500cbd80 con 0x7f91500af0a0 2023-10-06T05:23:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f913c003570 con 0x7f91500af0a0 2023-10-06T05:23:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654397780 0 0) 0x7f913c003570 con 0x7f91500af0a0 2023-10-06T05:23:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 >> v2:172.21.15.167:3300/0 conn(0x7f91500cf3b0 msgr2=0x7f91501594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 --2- 172.21.15.33:0/4127320207 >> v2:172.21.15.167:3300/0 conn(0x7f91500cf3b0 0x7f91501594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 >> v1:172.21.15.167:6789/0 conn(0x7f91500ccfc0 legacy=0x7f91500cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500afbb0 con 0x7f91500af0a0 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f91480017f0 con 0x7f91500af0a0 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 >> v1:172.21.15.33:6789/0 conn(0x7f91500af0a0 legacy=0x7f91500b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c003a90 0x7f913c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.320+0000 7f9147fff700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f913c004810 0x7f913c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9147fff700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f913c0051c0 0x7f913c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91500af8d0 con 0x7f913c0051c0 2023-10-06T05:23:12.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c00c130 con 0x7f913c003a90 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c00c2f0 con 0x7f913c004810 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f913c0051c0 0x7f913c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9154f9a700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f913c004810 0x7f913c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f915579b700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c003a90 0x7f913c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 -- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f913c004810 msgr2=0x7f913c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f913c004810 0x7f913c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 -- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c003a90 msgr2=0x7f913c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c003a90 0x7f913c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 -- 172.21.15.33:0/4127320207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c003570 con 0x7f913c0051c0 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9154f9a700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f913c004810 0x7f913c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.321+0000 7f9155f9c700 1 -- 172.21.15.33:0/4127320207 reap_dead start 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f9155f9c700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f913c0051c0 0x7f913c00ba60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f91500af520 tx=0x7f91480021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc609bf6e3bf48d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480070d0 con 0x7f913c0051c0 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9148007230 con 0x7f913c0051c0 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f9147fff700 1 -- 172.21.15.33:0/4127320207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480075f0 con 0x7f913c0051c0 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f915679d700 1 -- 172.21.15.33:0/4127320207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f913c0051c0 msgr2=0x7f913c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f915679d700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f913c0051c0 0x7f913c00ba60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f91500af520 tx=0x7f91480021f0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f915679d700 1 -- 172.21.15.33:0/4127320207 shutdown_connections 2023-10-06T05:23:12.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f915679d700 1 --2- 172.21.15.33:0/4127320207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f913c0051c0 0x7f913c00ba60 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.322+0000 7f915679d700 1 -- 172.21.15.33:0/4127320207 >> 172.21.15.33:0/4127320207 conn(0x7f915006cff0 msgr2=0x7f915006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.323+0000 7f915679d700 1 -- 172.21.15.33:0/4127320207 shutdown_connections 2023-10-06T05:23:12.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.323+0000 7f915679d700 1 -- 172.21.15.33:0/4127320207 wait complete. 2023-10-06T05:23:12.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.324+0000 7f915679d700 1 Processor -- start 2023-10-06T05:23:12.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.324+0000 7f915679d700 1 -- start start 2023-10-06T05:23:12.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c0051c0 0x7f9150144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 0x7f91500b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91500b5b90 0x7f91500c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91500c7f80 con 0x7f91500ca160 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150144790 con 0x7f913c0051c0 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91500b6010 con 0x7f91500b5b90 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f9154f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 0x7f91500b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f9154f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 0x7f91500b5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51674/0 (socket says 172.21.15.33:51674) 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f9154f9a700 1 -- 172.21.15.33:0/3747449962 learned_addr learned my addr 172.21.15.33:0/3747449962 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.325+0000 7f915579b700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c0051c0 0x7f9150144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9155f9c700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91500b5b90 0x7f91500c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9154f9a700 1 -- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91500b5b90 msgr2=0x7f91500c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9154f9a700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91500b5b90 0x7f91500c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9154f9a700 1 -- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c0051c0 msgr2=0x7f9150144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9154f9a700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c0051c0 0x7f9150144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9154f9a700 1 -- 172.21.15.33:0/3747449962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500c7650 con 0x7f91500ca160 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f9154f9a700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 0x7f91500b5650 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f91400116b0 tx=0x7f9140008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140015740 con 0x7f91500ca160 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91400158a0 con 0x7f91500ca160 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140015c80 con 0x7f91500ca160 2023-10-06T05:23:12.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91500c7930 con 0x7f91500ca160 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.326+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91500c6a60 con 0x7f91500ca160 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.327+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91500c80f0 con 0x7f91500ca160 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.328+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9140013620 con 0x7f91500ca160 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.329+0000 7f91457fa700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9134066f30 0x7f91340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.329+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9140084e10 con 0x7f91500ca160 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.330+0000 7f915579b700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9134066f30 0x7f91340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.330+0000 7f915579b700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9134066f30 0x7f91340693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f914c003ad0 tx=0x7f914c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.333+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9140088030 con 0x7f91500ca160 2023-10-06T05:23:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.531+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f915004c380 con 0x7f9134066f30 2023-10-06T05:23:12.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.541+0000 7f91457fa700 1 -- 172.21.15.33:0/3747449962 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311422 (secure 0 0 0) 0x7f915004c380 con 0x7f9134066f30 2023-10-06T05:23:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9134066f30 msgr2=0x7f91340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9134066f30 0x7f91340693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f914c003ad0 tx=0x7f914c009040 comp rx=0 tx=0).stop 2023-10-06T05:23:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 msgr2=0x7f91500b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 0x7f91500b5650 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f91400116b0 tx=0x7f9140008040 comp rx=0 tx=0).stop 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 shutdown_connections 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9134066f30 0x7f91340693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91500b5b90 0x7f91500c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913c0051c0 0x7f9150144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 --2- 172.21.15.33:0/3747449962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91500ca160 0x7f91500b5650 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 >> 172.21.15.33:0/3747449962 conn(0x7f915006cff0 msgr2=0x7f91500cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 shutdown_connections 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.545+0000 7f915679d700 1 -- 172.21.15.33:0/3747449962 wait complete. 2023-10-06T05:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:23:12.565 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2023-10-06T05:23:11.947969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423656,"kb_used_data":12584,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551064,"statfs":{"total":773094113280,"available":770612289536,"internally_reserved":0,"allocated":12886016,"data_stored":6519644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001427"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.201337+0000","last_change":"2023-10-06T05:23:07.198281+0000","last_active":"2023-10-06T05:23:07.201337+0000","last_peered":"2023-10-06T05:23:07.201337+0000","last_clean":"2023-10-06T05:23:07.201337+0000","last_became_active":"2023-10-06T05:23:07.197957+0000","last_became_peered":"2023-10-06T05:23:07.197957+0000","last_unstale":"2023-10-06T05:23:07.201337+0000","last_undegraded":"2023-10-06T05:23:07.201337+0000","last_fullsized":"2023-10-06T05:23:07.201337+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:41.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222033+0000","last_change":"2023-10-06T05:23:07.221005+0000","last_active":"2023-10-06T05:23:07.222033+0000","last_peered":"2023-10-06T05:23:07.222033+0000","last_clean":"2023-10-06T05:23:07.222033+0000","last_became_active":"2023-10-06T05:23:07.219844+0000","last_became_peered":"2023-10-06T05:23:07.219844+0000","last_unstale":"2023-10-06T05:23:07.222033+0000","last_undegraded":"2023-10-06T05:23:07.222033+0000","last_fullsized":"2023-10-06T05:23:07.222033+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:01.003406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224468+0000","last_change":"2023-10-06T05:23:07.223793+0000","last_active":"2023-10-06T05:23:07.224468+0000","last_peered":"2023-10-06T05:23:07.224468+0000","last_clean":"2023-10-06T05:23:07.224468+0000","last_became_active":"2023-10-06T05:23:07.223463+0000","last_became_peered":"2023-10-06T05:23:07.223463+0000","last_unstale":"2023-10-06T05:23:07.224468+0000","last_undegraded":"2023-10-06T05:23:07.224468+0000","last_fullsized":"2023-10-06T05:23:07.224468+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:48.160316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222469+0000","last_change":"2023-10-06T05:23:07.217469+0000","last_active":"2023-10-06T05:23:07.222469+0000","last_peered":"2023-10-06T05:23:07.222469+0000","last_clean":"2023-10-06T05:23:07.222469+0000","last_became_active":"2023-10-06T05:23:07.217200+0000","last_became_peered":"2023-10-06T05:23:07.217200+0000","last_unstale":"2023-10-06T05:23:07.222469+0000","last_undegraded":"2023-10-06T05:23:07.222469+0000","last_fullsized":"2023-10-06T05:23:07.222469+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:20.355274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214582+0000","last_change":"2023-10-06T05:23:07.203067+0000","last_active":"2023-10-06T05:23:07.214582+0000","last_peered":"2023-10-06T05:23:07.214582+0000","last_clean":"2023-10-06T05:23:07.214582+0000","last_became_active":"2023-10-06T05:23:07.202684+0000","last_became_peered":"2023-10-06T05:23:07.202684+0000","last_unstale":"2023-10-06T05:23:07.214582+0000","last_undegraded":"2023-10-06T05:23:07.214582+0000","last_fullsized":"2023-10-06T05:23:07.214582+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:48.160316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220934+0000","last_change":"2023-10-06T05:23:07.215777+0000","last_active":"2023-10-06T05:23:07.220934+0000","last_peered":"2023-10-06T05:23:07.220934+0000","last_clean":"2023-10-06T05:23:07.220934+0000","last_became_active":"2023-10-06T05:23:07.215501+0000","last_became_peered":"2023-10-06T05:23:07.215501+0000","last_unstale":"2023-10-06T05:23:07.220934+0000","last_undegraded":"2023-10-06T05:23:07.220934+0000","last_fullsized":"2023-10-06T05:23:07.220934+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:15.494697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220890+0000","last_change":"2023-10-06T05:23:07.212624+0000","last_active":"2023-10-06T05:23:07.220890+0000","last_peered":"2023-10-06T05:23:07.220890+0000","last_clean":"2023-10-06T05:23:07.220890+0000","last_became_active":"2023-10-06T05:23:07.212396+0000","last_became_peered":"2023-10-06T05:23:07.212396+0000","last_unstale":"2023-10-06T05:23:07.220890+0000","last_undegraded":"2023-10-06T05:23:07.220890+0000","last_fullsized":"2023-10-06T05:23:07.220890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:32.049620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221131+0000","last_change":"2023-10-06T05:23:07.213372+0000","last_active":"2023-10-06T05:23:07.221131+0000","last_peered":"2023-10-06T05:23:07.221131+0000","last_clean":"2023-10-06T05:23:07.221131+0000","last_became_active":"2023-10-06T05:23:07.213125+0000","last_became_peered":"2023-10-06T05:23:07.213125+0000","last_unstale":"2023-10-06T05:23:07.221131+0000","last_undegraded":"2023-10-06T05:23:07.221131+0000","last_fullsized":"2023-10-06T05:23:07.221131+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:28.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222543+0000","last_change":"2023-10-06T05:23:07.218468+0000","last_active":"2023-10-06T05:23:07.222543+0000","last_peered":"2023-10-06T05:23:07.222543+0000","last_clean":"2023-10-06T05:23:07.222543+0000","last_became_active":"2023-10-06T05:23:07.218187+0000","last_became_peered":"2023-10-06T05:23:07.218187+0000","last_unstale":"2023-10-06T05:23:07.222543+0000","last_undegraded":"2023-10-06T05:23:07.222543+0000","last_fullsized":"2023-10-06T05:23:07.222543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:12.847514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214408+0000","last_change":"2023-10-06T05:23:07.205065+0000","last_active":"2023-10-06T05:23:07.214408+0000","last_peered":"2023-10-06T05:23:07.214408+0000","last_clean":"2023-10-06T05:23:07.214408+0000","last_became_active":"2023-10-06T05:23:07.203629+0000","last_became_peered":"2023-10-06T05:23:07.203629+0000","last_unstale":"2023-10-06T05:23:07.214408+0000","last_undegraded":"2023-10-06T05:23:07.214408+0000","last_fullsized":"2023-10-06T05:23:07.214408+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:58.966299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214634+0000","last_change":"2023-10-06T05:23:07.207345+0000","last_active":"2023-10-06T05:23:07.214634+0000","last_peered":"2023-10-06T05:23:07.214634+0000","last_clean":"2023-10-06T05:23:07.214634+0000","last_became_active":"2023-10-06T05:23:07.207072+0000","last_became_peered":"2023-10-06T05:23:07.207072+0000","last_unstale":"2023-10-06T05:23:07.214634+0000","last_undegraded":"2023-10-06T05:23:07.214634+0000","last_fullsized":"2023-10-06T05:23:07.214634+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:26.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221018+0000","last_change":"2023-10-06T05:23:07.216031+0000","last_active":"2023-10-06T05:23:07.221018+0000","last_peered":"2023-10-06T05:23:07.221018+0000","last_clean":"2023-10-06T05:23:07.221018+0000","last_became_active":"2023-10-06T05:23:07.215742+0000","last_became_peered":"2023-10-06T05:23:07.215742+0000","last_unstale":"2023-10-06T05:23:07.221018+0000","last_undegraded":"2023-10-06T05:23:07.221018+0000","last_fullsized":"2023-10-06T05:23:07.221018+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:50.930486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218010+0000","last_change":"2023-10-06T05:23:07.209257+0000","last_active":"2023-10-06T05:23:07.218010+0000","last_peered":"2023-10-06T05:23:07.218010+0000","last_clean":"2023-10-06T05:23:07.218010+0000","last_became_active":"2023-10-06T05:23:07.208919+0000","last_became_peered":"2023-10-06T05:23:07.208919+0000","last_unstale":"2023-10-06T05:23:07.218010+0000","last_undegraded":"2023-10-06T05:23:07.218010+0000","last_fullsized":"2023-10-06T05:23:07.218010+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:51.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224779+0000","last_change":"2023-10-06T05:23:07.223790+0000","last_active":"2023-10-06T05:23:07.224779+0000","last_peered":"2023-10-06T05:23:07.224779+0000","last_clean":"2023-10-06T05:23:07.224779+0000","last_became_active":"2023-10-06T05:23:07.223406+0000","last_became_peered":"2023-10-06T05:23:07.223406+0000","last_unstale":"2023-10-06T05:23:07.224779+0000","last_undegraded":"2023-10-06T05:23:07.224779+0000","last_fullsized":"2023-10-06T05:23:07.224779+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:24.685795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214658+0000","last_change":"2023-10-06T05:23:07.206656+0000","last_active":"2023-10-06T05:23:07.214658+0000","last_peered":"2023-10-06T05:23:07.214658+0000","last_clean":"2023-10-06T05:23:07.214658+0000","last_became_active":"2023-10-06T05:23:07.206381+0000","last_became_peered":"2023-10-06T05:23:07.206381+0000","last_unstale":"2023-10-06T05:23:07.214658+0000","last_undegraded":"2023-10-06T05:23:07.214658+0000","last_fullsized":"2023-10-06T05:23:07.214658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:20.355274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.208346+0000","last_change":"2023-10-06T05:23:07.204953+0000","last_active":"2023-10-06T05:23:07.208346+0000","last_peered":"2023-10-06T05:23:07.208346+0000","last_clean":"2023-10-06T05:23:07.208346+0000","last_became_active":"2023-10-06T05:23:07.204643+0000","last_became_peered":"2023-10-06T05:23:07.204643+0000","last_unstale":"2023-10-06T05:23:07.208346+0000","last_undegraded":"2023-10-06T05:23:07.208346+0000","last_fullsized":"2023-10-06T05:23:07.208346+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:55.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214553+0000","last_change":"2023-10-06T05:23:07.207642+0000","last_active":"2023-10-06T05:23:07.214553+0000","last_peered":"2023-10-06T05:23:07.214553+0000","last_clean":"2023-10-06T05:23:07.214553+0000","last_became_active":"2023-10-06T05:23:07.207390+0000","last_became_peered":"2023-10-06T05:23:07.207390+0000","last_unstale":"2023-10-06T05:23:07.214553+0000","last_undegraded":"2023-10-06T05:23:07.214553+0000","last_fullsized":"2023-10-06T05:23:07.214553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:06.654520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221201+0000","last_change":"2023-10-06T05:23:07.219544+0000","last_active":"2023-10-06T05:23:07.221201+0000","last_peered":"2023-10-06T05:23:07.221201+0000","last_clean":"2023-10-06T05:23:07.221201+0000","last_became_active":"2023-10-06T05:23:07.219223+0000","last_became_peered":"2023-10-06T05:23:07.219223+0000","last_unstale":"2023-10-06T05:23:07.221201+0000","last_undegraded":"2023-10-06T05:23:07.221201+0000","last_fullsized":"2023-10-06T05:23:07.221201+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:35.452400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.205371+0000","last_change":"2023-10-06T05:23:07.199697+0000","last_active":"2023-10-06T05:23:07.205371+0000","last_peered":"2023-10-06T05:23:07.205371+0000","last_clean":"2023-10-06T05:23:07.205371+0000","last_became_active":"2023-10-06T05:23:07.199439+0000","last_became_peered":"2023-10-06T05:23:07.199439+0000","last_unstale":"2023-10-06T05:23:07.205371+0000","last_undegraded":"2023-10-06T05:23:07.205371+0000","last_fullsized":"2023-10-06T05:23:07.205371+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:09.989368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.203884+0000","last_change":"2023-10-06T05:23:07.197182+0000","last_active":"2023-10-06T05:23:07.203884+0000","last_peered":"2023-10-06T05:23:07.203884+0000","last_clean":"2023-10-06T05:23:07.203884+0000","last_became_active":"2023-10-06T05:23:07.196841+0000","last_became_peered":"2023-10-06T05:23:07.196841+0000","last_unstale":"2023-10-06T05:23:07.203884+0000","last_undegraded":"2023-10-06T05:23:07.203884+0000","last_fullsized":"2023-10-06T05:23:07.203884+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:37.144809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.216659+0000","last_change":"2023-10-06T05:23:07.210237+0000","last_active":"2023-10-06T05:23:07.216659+0000","last_peered":"2023-10-06T05:23:07.216659+0000","last_clean":"2023-10-06T05:23:07.216659+0000","last_became_active":"2023-10-06T05:23:07.209159+0000","last_became_peered":"2023-10-06T05:23:07.209159+0000","last_unstale":"2023-10-06T05:23:07.216659+0000","last_undegraded":"2023-10-06T05:23:07.216659+0000","last_fullsized":"2023-10-06T05:23:07.216659+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:05.463087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221844+0000","last_change":"2023-10-06T05:23:07.214878+0000","last_active":"2023-10-06T05:23:07.221844+0000","last_peered":"2023-10-06T05:23:07.221844+0000","last_clean":"2023-10-06T05:23:07.221844+0000","last_became_active":"2023-10-06T05:23:07.214605+0000","last_became_peered":"2023-10-06T05:23:07.214605+0000","last_unstale":"2023-10-06T05:23:07.221844+0000","last_undegraded":"2023-10-06T05:23:07.221844+0000","last_fullsized":"2023-10-06T05:23:07.221844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:27.252085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222495+0000","last_change":"2023-10-06T05:23:07.218164+0000","last_active":"2023-10-06T05:23:07.222495+0000","last_peered":"2023-10-06T05:23:07.222495+0000","last_clean":"2023-10-06T05:23:07.222495+0000","last_became_active":"2023-10-06T05:23:07.217857+0000","last_became_peered":"2023-10-06T05:23:07.217857+0000","last_unstale":"2023-10-06T05:23:07.222495+0000","last_undegraded":"2023-10-06T05:23:07.222495+0000","last_fullsized":"2023-10-06T05:23:07.222495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:35.452400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210368+0000","last_change":"2023-10-06T05:23:07.202649+0000","last_active":"2023-10-06T05:23:07.210368+0000","last_peered":"2023-10-06T05:23:07.210368+0000","last_clean":"2023-10-06T05:23:07.210368+0000","last_became_active":"2023-10-06T05:23:07.202341+0000","last_became_peered":"2023-10-06T05:23:07.202341+0000","last_unstale":"2023-10-06T05:23:07.210368+0000","last_undegraded":"2023-10-06T05:23:07.210368+0000","last_fullsized":"2023-10-06T05:23:07.210368+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:20.422846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204198+0000","last_change":"2023-10-06T05:23:07.198523+0000","last_active":"2023-10-06T05:23:07.204198+0000","last_peered":"2023-10-06T05:23:07.204198+0000","last_clean":"2023-10-06T05:23:07.204198+0000","last_became_active":"2023-10-06T05:23:07.197618+0000","last_became_peered":"2023-10-06T05:23:07.197618+0000","last_unstale":"2023-10-06T05:23:07.204198+0000","last_undegraded":"2023-10-06T05:23:07.204198+0000","last_fullsized":"2023-10-06T05:23:07.204198+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:51.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.203937+0000","last_change":"2023-10-06T05:23:07.198118+0000","last_active":"2023-10-06T05:23:07.203937+0000","last_peered":"2023-10-06T05:23:07.203937+0000","last_clean":"2023-10-06T05:23:07.203937+0000","last_became_active":"2023-10-06T05:23:07.197868+0000","last_became_peered":"2023-10-06T05:23:07.197868+0000","last_unstale":"2023-10-06T05:23:07.203937+0000","last_undegraded":"2023-10-06T05:23:07.203937+0000","last_fullsized":"2023-10-06T05:23:07.203937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:56.475473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.205391+0000","last_change":"2023-10-06T05:23:07.200369+0000","last_active":"2023-10-06T05:23:07.205391+0000","last_peered":"2023-10-06T05:23:07.205391+0000","last_clean":"2023-10-06T05:23:07.205391+0000","last_became_active":"2023-10-06T05:23:07.200088+0000","last_became_peered":"2023-10-06T05:23:07.200088+0000","last_unstale":"2023-10-06T05:23:07.205391+0000","last_undegraded":"2023-10-06T05:23:07.205391+0000","last_fullsized":"2023-10-06T05:23:07.205391+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.144319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210924+0000","last_change":"2023-10-06T05:23:07.206937+0000","last_active":"2023-10-06T05:23:07.210924+0000","last_peered":"2023-10-06T05:23:07.210924+0000","last_clean":"2023-10-06T05:23:07.210924+0000","last_became_active":"2023-10-06T05:23:07.206648+0000","last_became_peered":"2023-10-06T05:23:07.206648+0000","last_unstale":"2023-10-06T05:23:07.210924+0000","last_undegraded":"2023-10-06T05:23:07.210924+0000","last_fullsized":"2023-10-06T05:23:07.210924+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:56.975166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210006+0000","last_change":"2023-10-06T05:23:07.204709+0000","last_active":"2023-10-06T05:23:07.210006+0000","last_peered":"2023-10-06T05:23:07.210006+0000","last_clean":"2023-10-06T05:23:07.210006+0000","last_became_active":"2023-10-06T05:23:07.204414+0000","last_became_peered":"2023-10-06T05:23:07.204414+0000","last_unstale":"2023-10-06T05:23:07.210006+0000","last_undegraded":"2023-10-06T05:23:07.210006+0000","last_fullsized":"2023-10-06T05:23:07.210006+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:17.164286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.223471+0000","last_change":"2023-10-06T05:23:07.221989+0000","last_active":"2023-10-06T05:23:07.223471+0000","last_peered":"2023-10-06T05:23:07.223471+0000","last_clean":"2023-10-06T05:23:07.223471+0000","last_became_active":"2023-10-06T05:23:07.221668+0000","last_became_peered":"2023-10-06T05:23:07.221668+0000","last_unstale":"2023-10-06T05:23:07.223471+0000","last_undegraded":"2023-10-06T05:23:07.223471+0000","last_fullsized":"2023-10-06T05:23:07.223471+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:05.463087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.212831+0000","last_change":"2023-10-06T05:23:07.205279+0000","last_active":"2023-10-06T05:23:07.212831+0000","last_peered":"2023-10-06T05:23:07.212831+0000","last_clean":"2023-10-06T05:23:07.212831+0000","last_became_active":"2023-10-06T05:23:07.204901+0000","last_became_peered":"2023-10-06T05:23:07.204901+0000","last_unstale":"2023-10-06T05:23:07.212831+0000","last_undegraded":"2023-10-06T05:23:07.212831+0000","last_fullsized":"2023-10-06T05:23:07.212831+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:55.149048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210389+0000","last_change":"2023-10-06T05:23:07.203599+0000","last_active":"2023-10-06T05:23:07.210389+0000","last_peered":"2023-10-06T05:23:07.210389+0000","last_clean":"2023-10-06T05:23:07.210389+0000","last_became_active":"2023-10-06T05:23:07.203332+0000","last_became_peered":"2023-10-06T05:23:07.203332+0000","last_unstale":"2023-10-06T05:23:07.210389+0000","last_undegraded":"2023-10-06T05:23:07.210389+0000","last_fullsized":"2023-10-06T05:23:07.210389+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:58.901456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214598+0000","last_change":"2023-10-06T05:23:07.202910+0000","last_active":"2023-10-06T05:23:07.214598+0000","last_peered":"2023-10-06T05:23:07.214598+0000","last_clean":"2023-10-06T05:23:07.214598+0000","last_became_active":"2023-10-06T05:23:07.202639+0000","last_became_peered":"2023-10-06T05:23:07.202639+0000","last_unstale":"2023-10-06T05:23:07.214598+0000","last_undegraded":"2023-10-06T05:23:07.214598+0000","last_fullsized":"2023-10-06T05:23:07.214598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.373703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"116'4","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218297+0000","last_change":"2023-10-06T05:23:06.198817+0000","last_active":"2023-10-06T05:23:07.218297+0000","last_peered":"2023-10-06T05:23:07.218297+0000","last_clean":"2023-10-06T05:23:07.218297+0000","last_became_active":"2023-10-06T05:23:06.198525+0000","last_became_peered":"2023-10-06T05:23:06.198525+0000","last_unstale":"2023-10-06T05:23:07.218297+0000","last_undegraded":"2023-10-06T05:23:07.218297+0000","last_fullsized":"2023-10-06T05:23:07.218297+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:24.694784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210777+0000","last_change":"2023-10-06T05:23:07.208244+0000","last_active":"2023-10-06T05:23:07.210777+0000","last_peered":"2023-10-06T05:23:07.210777+0000","last_clean":"2023-10-06T05:23:07.210777+0000","last_became_active":"2023-10-06T05:23:07.207983+0000","last_became_peered":"2023-10-06T05:23:07.207983+0000","last_unstale":"2023-10-06T05:23:07.210777+0000","last_undegraded":"2023-10-06T05:23:07.210777+0000","last_fullsized":"2023-10-06T05:23:07.210777+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:03.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214440+0000","last_change":"2023-10-06T05:23:06.190681+0000","last_active":"2023-10-06T05:23:07.214440+0000","last_peered":"2023-10-06T05:23:07.214440+0000","last_clean":"2023-10-06T05:23:07.214440+0000","last_became_active":"2023-10-06T05:23:06.190403+0000","last_became_peered":"2023-10-06T05:23:06.190403+0000","last_unstale":"2023-10-06T05:23:07.214440+0000","last_undegraded":"2023-10-06T05:23:07.214440+0000","last_fullsized":"2023-10-06T05:23:07.214440+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:37.354403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.205064+0000","last_change":"2023-10-06T05:23:07.199681+0000","last_active":"2023-10-06T05:23:07.205064+0000","last_peered":"2023-10-06T05:23:07.205064+0000","last_clean":"2023-10-06T05:23:07.205064+0000","last_became_active":"2023-10-06T05:23:07.199411+0000","last_became_peered":"2023-10-06T05:23:07.199411+0000","last_unstale":"2023-10-06T05:23:07.205064+0000","last_undegraded":"2023-10-06T05:23:07.205064+0000","last_fullsized":"2023-10-06T05:23:07.205064+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:58.966299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218395+0000","last_change":"2023-10-06T05:23:06.200524+0000","last_active":"2023-10-06T05:23:07.218395+0000","last_peered":"2023-10-06T05:23:07.218395+0000","last_clean":"2023-10-06T05:23:07.218395+0000","last_became_active":"2023-10-06T05:23:06.200263+0000","last_became_peered":"2023-10-06T05:23:06.200263+0000","last_unstale":"2023-10-06T05:23:07.218395+0000","last_undegraded":"2023-10-06T05:23:07.218395+0000","last_fullsized":"2023-10-06T05:23:07.218395+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.361010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.211865+0000","last_change":"2023-10-06T05:23:07.207032+0000","last_active":"2023-10-06T05:23:07.211865+0000","last_peered":"2023-10-06T05:23:07.211865+0000","last_clean":"2023-10-06T05:23:07.211865+0000","last_became_active":"2023-10-06T05:23:07.206732+0000","last_became_peered":"2023-10-06T05:23:07.206732+0000","last_unstale":"2023-10-06T05:23:07.211865+0000","last_undegraded":"2023-10-06T05:23:07.211865+0000","last_fullsized":"2023-10-06T05:23:07.211865+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:26.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218541+0000","last_change":"2023-10-06T05:23:06.199909+0000","last_active":"2023-10-06T05:23:07.218541+0000","last_peered":"2023-10-06T05:23:07.218541+0000","last_clean":"2023-10-06T05:23:07.218541+0000","last_became_active":"2023-10-06T05:23:06.199574+0000","last_became_peered":"2023-10-06T05:23:06.199574+0000","last_unstale":"2023-10-06T05:23:07.218541+0000","last_undegraded":"2023-10-06T05:23:07.218541+0000","last_fullsized":"2023-10-06T05:23:07.218541+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:28.416245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214690+0000","last_change":"2023-10-06T05:23:07.204595+0000","last_active":"2023-10-06T05:23:07.214690+0000","last_peered":"2023-10-06T05:23:07.214690+0000","last_clean":"2023-10-06T05:23:07.214690+0000","last_became_active":"2023-10-06T05:23:07.204313+0000","last_became_peered":"2023-10-06T05:23:07.204313+0000","last_unstale":"2023-10-06T05:23:07.214690+0000","last_undegraded":"2023-10-06T05:23:07.214690+0000","last_fullsized":"2023-10-06T05:23:07.214690+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:43.760144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.219062+0000","last_change":"2023-10-06T05:23:06.199624+0000","last_active":"2023-10-06T05:23:07.219062+0000","last_peered":"2023-10-06T05:23:07.219062+0000","last_clean":"2023-10-06T05:23:07.219062+0000","last_became_active":"2023-10-06T05:23:06.199367+0000","last_became_peered":"2023-10-06T05:23:06.199367+0000","last_unstale":"2023-10-06T05:23:07.219062+0000","last_undegraded":"2023-10-06T05:23:07.219062+0000","last_fullsized":"2023-10-06T05:23:07.219062+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:53.211927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214531+0000","last_change":"2023-10-06T05:23:07.202632+0000","last_active":"2023-10-06T05:23:07.214531+0000","last_peered":"2023-10-06T05:23:07.214531+0000","last_clean":"2023-10-06T05:23:07.214531+0000","last_became_active":"2023-10-06T05:23:07.202340+0000","last_became_peered":"2023-10-06T05:23:07.202340+0000","last_unstale":"2023-10-06T05:23:07.214531+0000","last_undegraded":"2023-10-06T05:23:07.214531+0000","last_fullsized":"2023-10-06T05:23:07.214531+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:05.463087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224697+0000","last_change":"2023-10-06T05:23:06.200316+0000","last_active":"2023-10-06T05:23:07.224697+0000","last_peered":"2023-10-06T05:23:07.224697+0000","last_clean":"2023-10-06T05:23:07.224697+0000","last_became_active":"2023-10-06T05:23:06.200053+0000","last_became_peered":"2023-10-06T05:23:06.200053+0000","last_unstale":"2023-10-06T05:23:07.224697+0000","last_undegraded":"2023-10-06T05:23:07.224697+0000","last_fullsized":"2023-10-06T05:23:07.224697+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:15.368204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221190+0000","last_change":"2023-10-06T05:23:07.216861+0000","last_active":"2023-10-06T05:23:07.221190+0000","last_peered":"2023-10-06T05:23:07.221190+0000","last_clean":"2023-10-06T05:23:07.221190+0000","last_became_active":"2023-10-06T05:23:07.216589+0000","last_became_peered":"2023-10-06T05:23:07.216589+0000","last_unstale":"2023-10-06T05:23:07.221190+0000","last_undegraded":"2023-10-06T05:23:07.221190+0000","last_fullsized":"2023-10-06T05:23:07.221190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:41.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.188799+0000","last_change":"2023-10-06T05:23:06.200972+0000","last_active":"2023-10-06T05:23:07.188799+0000","last_peered":"2023-10-06T05:23:07.188799+0000","last_clean":"2023-10-06T05:23:07.188799+0000","last_became_active":"2023-10-06T05:23:06.200703+0000","last_became_peered":"2023-10-06T05:23:06.200703+0000","last_unstale":"2023-10-06T05:23:07.188799+0000","last_undegraded":"2023-10-06T05:23:07.188799+0000","last_fullsized":"2023-10-06T05:23:07.188799+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:02.452310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220867+0000","last_change":"2023-10-06T05:23:07.219132+0000","last_active":"2023-10-06T05:23:07.220867+0000","last_peered":"2023-10-06T05:23:07.220867+0000","last_clean":"2023-10-06T05:23:07.220867+0000","last_became_active":"2023-10-06T05:23:07.218899+0000","last_became_peered":"2023-10-06T05:23:07.218899+0000","last_unstale":"2023-10-06T05:23:07.220867+0000","last_undegraded":"2023-10-06T05:23:07.220867+0000","last_fullsized":"2023-10-06T05:23:07.220867+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:54.224620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.190488+0000","last_change":"2023-10-06T05:23:06.201230+0000","last_active":"2023-10-06T05:23:07.190488+0000","last_peered":"2023-10-06T05:23:07.190488+0000","last_clean":"2023-10-06T05:23:07.190488+0000","last_became_active":"2023-10-06T05:23:06.200890+0000","last_became_peered":"2023-10-06T05:23:06.200890+0000","last_unstale":"2023-10-06T05:23:07.190488+0000","last_undegraded":"2023-10-06T05:23:07.190488+0000","last_fullsized":"2023-10-06T05:23:07.190488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:50.911792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204680+0000","last_change":"2023-10-06T05:23:07.200537+0000","last_active":"2023-10-06T05:23:07.204680+0000","last_peered":"2023-10-06T05:23:07.204680+0000","last_clean":"2023-10-06T05:23:07.204680+0000","last_became_active":"2023-10-06T05:23:07.200039+0000","last_became_peered":"2023-10-06T05:23:07.200039+0000","last_unstale":"2023-10-06T05:23:07.204680+0000","last_undegraded":"2023-10-06T05:23:07.204680+0000","last_fullsized":"2023-10-06T05:23:07.204680+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:54.224620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.217842+0000","last_change":"2023-10-06T05:23:06.196899+0000","last_active":"2023-10-06T05:23:07.217842+0000","last_peered":"2023-10-06T05:23:07.217842+0000","last_clean":"2023-10-06T05:23:07.217842+0000","last_became_active":"2023-10-06T05:23:06.196553+0000","last_became_peered":"2023-10-06T05:23:06.196553+0000","last_unstale":"2023-10-06T05:23:07.217842+0000","last_undegraded":"2023-10-06T05:23:07.217842+0000","last_fullsized":"2023-10-06T05:23:07.217842+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:41.942266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214480+0000","last_change":"2023-10-06T05:23:07.204031+0000","last_active":"2023-10-06T05:23:07.214480+0000","last_peered":"2023-10-06T05:23:07.214480+0000","last_clean":"2023-10-06T05:23:07.214480+0000","last_became_active":"2023-10-06T05:23:07.203769+0000","last_became_peered":"2023-10-06T05:23:07.203769+0000","last_unstale":"2023-10-06T05:23:07.214480+0000","last_undegraded":"2023-10-06T05:23:07.214480+0000","last_fullsized":"2023-10-06T05:23:07.214480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:24.685795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.209208+0000","last_change":"2023-10-06T05:23:06.202660+0000","last_active":"2023-10-06T05:23:07.209208+0000","last_peered":"2023-10-06T05:23:07.209208+0000","last_clean":"2023-10-06T05:23:07.209208+0000","last_became_active":"2023-10-06T05:23:06.202082+0000","last_became_peered":"2023-10-06T05:23:06.202082+0000","last_unstale":"2023-10-06T05:23:07.209208+0000","last_undegraded":"2023-10-06T05:23:07.209208+0000","last_fullsized":"2023-10-06T05:23:07.209208+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:14.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.207107+0000","last_change":"2023-10-06T05:23:07.201677+0000","last_active":"2023-10-06T05:23:07.207107+0000","last_peered":"2023-10-06T05:23:07.207107+0000","last_clean":"2023-10-06T05:23:07.207107+0000","last_became_active":"2023-10-06T05:23:07.201418+0000","last_became_peered":"2023-10-06T05:23:07.201418+0000","last_unstale":"2023-10-06T05:23:07.207107+0000","last_undegraded":"2023-10-06T05:23:07.207107+0000","last_fullsized":"2023-10-06T05:23:07.207107+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:56.704239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218052+0000","last_change":"2023-10-06T05:23:06.203653+0000","last_active":"2023-10-06T05:23:07.218052+0000","last_peered":"2023-10-06T05:23:07.218052+0000","last_clean":"2023-10-06T05:23:07.218052+0000","last_became_active":"2023-10-06T05:23:06.203334+0000","last_became_peered":"2023-10-06T05:23:06.203334+0000","last_unstale":"2023-10-06T05:23:07.218052+0000","last_undegraded":"2023-10-06T05:23:07.218052+0000","last_fullsized":"2023-10-06T05:23:07.218052+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:53.211927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204218+0000","last_change":"2023-10-06T05:23:07.200406+0000","last_active":"2023-10-06T05:23:07.204218+0000","last_peered":"2023-10-06T05:23:07.204218+0000","last_clean":"2023-10-06T05:23:07.204218+0000","last_became_active":"2023-10-06T05:23:07.199845+0000","last_became_peered":"2023-10-06T05:23:07.199845+0000","last_unstale":"2023-10-06T05:23:07.204218+0000","last_undegraded":"2023-10-06T05:23:07.204218+0000","last_fullsized":"2023-10-06T05:23:07.204218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:13.685691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.195000+0000","last_change":"2023-10-06T05:23:06.202829+0000","last_active":"2023-10-06T05:23:07.195000+0000","last_peered":"2023-10-06T05:23:07.195000+0000","last_clean":"2023-10-06T05:23:07.195000+0000","last_became_active":"2023-10-06T05:23:06.202182+0000","last_became_peered":"2023-10-06T05:23:06.202182+0000","last_unstale":"2023-10-06T05:23:07.195000+0000","last_undegraded":"2023-10-06T05:23:07.195000+0000","last_fullsized":"2023-10-06T05:23:07.195000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:05.641827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.207063+0000","last_change":"2023-10-06T05:23:07.201685+0000","last_active":"2023-10-06T05:23:07.207063+0000","last_peered":"2023-10-06T05:23:07.207063+0000","last_clean":"2023-10-06T05:23:07.207063+0000","last_became_active":"2023-10-06T05:23:07.201460+0000","last_became_peered":"2023-10-06T05:23:07.201460+0000","last_unstale":"2023-10-06T05:23:07.207063+0000","last_undegraded":"2023-10-06T05:23:07.207063+0000","last_fullsized":"2023-10-06T05:23:07.207063+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:54.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222480+0000","last_change":"2023-10-06T05:23:06.197993+0000","last_active":"2023-10-06T05:23:07.222480+0000","last_peered":"2023-10-06T05:23:07.222480+0000","last_clean":"2023-10-06T05:23:07.222480+0000","last_became_active":"2023-10-06T05:23:06.197634+0000","last_became_peered":"2023-10-06T05:23:06.197634+0000","last_unstale":"2023-10-06T05:23:07.222480+0000","last_undegraded":"2023-10-06T05:23:07.222480+0000","last_fullsized":"2023-10-06T05:23:07.222480+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:21.132679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220964+0000","last_change":"2023-10-06T05:23:07.211979+0000","last_active":"2023-10-06T05:23:07.220964+0000","last_peered":"2023-10-06T05:23:07.220964+0000","last_clean":"2023-10-06T05:23:07.220964+0000","last_became_active":"2023-10-06T05:23:07.211621+0000","last_became_peered":"2023-10-06T05:23:07.211621+0000","last_unstale":"2023-10-06T05:23:07.220964+0000","last_undegraded":"2023-10-06T05:23:07.220964+0000","last_fullsized":"2023-10-06T05:23:07.220964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:26.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.195326+0000","last_change":"2023-10-06T05:23:06.202339+0000","last_active":"2023-10-06T05:23:07.195326+0000","last_peered":"2023-10-06T05:23:07.195326+0000","last_clean":"2023-10-06T05:23:07.195326+0000","last_became_active":"2023-10-06T05:23:06.201873+0000","last_became_peered":"2023-10-06T05:23:06.201873+0000","last_unstale":"2023-10-06T05:23:07.195326+0000","last_undegraded":"2023-10-06T05:23:07.195326+0000","last_fullsized":"2023-10-06T05:23:07.195326+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.953606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221053+0000","last_change":"2023-10-06T05:23:07.212696+0000","last_active":"2023-10-06T05:23:07.221053+0000","last_peered":"2023-10-06T05:23:07.221053+0000","last_clean":"2023-10-06T05:23:07.221053+0000","last_became_active":"2023-10-06T05:23:07.212373+0000","last_became_peered":"2023-10-06T05:23:07.212373+0000","last_unstale":"2023-10-06T05:23:07.221053+0000","last_undegraded":"2023-10-06T05:23:07.221053+0000","last_fullsized":"2023-10-06T05:23:07.221053+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:20.355274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.202778+0000","last_change":"2023-10-06T05:23:06.204978+0000","last_active":"2023-10-06T05:23:07.202778+0000","last_peered":"2023-10-06T05:23:07.202778+0000","last_clean":"2023-10-06T05:23:07.202778+0000","last_became_active":"2023-10-06T05:23:06.204599+0000","last_became_peered":"2023-10-06T05:23:06.204599+0000","last_unstale":"2023-10-06T05:23:07.202778+0000","last_undegraded":"2023-10-06T05:23:07.202778+0000","last_fullsized":"2023-10-06T05:23:07.202778+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:02.282810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204523+0000","last_change":"2023-10-06T05:23:07.199940+0000","last_active":"2023-10-06T05:23:07.204523+0000","last_peered":"2023-10-06T05:23:07.204523+0000","last_clean":"2023-10-06T05:23:07.204523+0000","last_became_active":"2023-10-06T05:23:07.199633+0000","last_became_peered":"2023-10-06T05:23:07.199633+0000","last_unstale":"2023-10-06T05:23:07.204523+0000","last_undegraded":"2023-10-06T05:23:07.204523+0000","last_fullsized":"2023-10-06T05:23:07.204523+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.373703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193486+0000","last_change":"2023-10-06T05:23:06.196929+0000","last_active":"2023-10-06T05:23:07.193486+0000","last_peered":"2023-10-06T05:23:07.193486+0000","last_clean":"2023-10-06T05:23:07.193486+0000","last_became_active":"2023-10-06T05:23:06.196631+0000","last_became_peered":"2023-10-06T05:23:06.196631+0000","last_unstale":"2023-10-06T05:23:07.193486+0000","last_undegraded":"2023-10-06T05:23:07.193486+0000","last_fullsized":"2023-10-06T05:23:07.193486+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.136355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.207027+0000","last_change":"2023-10-06T05:23:07.202684+0000","last_active":"2023-10-06T05:23:07.207027+0000","last_peered":"2023-10-06T05:23:07.207027+0000","last_clean":"2023-10-06T05:23:07.207027+0000","last_became_active":"2023-10-06T05:23:07.202304+0000","last_became_peered":"2023-10-06T05:23:07.202304+0000","last_unstale":"2023-10-06T05:23:07.207027+0000","last_undegraded":"2023-10-06T05:23:07.207027+0000","last_fullsized":"2023-10-06T05:23:07.207027+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:50.391306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220924+0000","last_change":"2023-10-06T05:23:06.201976+0000","last_active":"2023-10-06T05:23:07.220924+0000","last_peered":"2023-10-06T05:23:07.220924+0000","last_clean":"2023-10-06T05:23:07.220924+0000","last_became_active":"2023-10-06T05:23:06.201627+0000","last_became_peered":"2023-10-06T05:23:06.201627+0000","last_unstale":"2023-10-06T05:23:07.220924+0000","last_undegraded":"2023-10-06T05:23:07.220924+0000","last_fullsized":"2023-10-06T05:23:07.220924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:12.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214564+0000","last_change":"2023-10-06T05:23:07.209042+0000","last_active":"2023-10-06T05:23:07.214564+0000","last_peered":"2023-10-06T05:23:07.214564+0000","last_clean":"2023-10-06T05:23:07.214564+0000","last_became_active":"2023-10-06T05:23:07.208724+0000","last_became_peered":"2023-10-06T05:23:07.208724+0000","last_unstale":"2023-10-06T05:23:07.214564+0000","last_undegraded":"2023-10-06T05:23:07.214564+0000","last_fullsized":"2023-10-06T05:23:07.214564+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:58.901456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"117'1","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T05:23:08.205911+0000","last_change":"2023-10-06T05:23:06.197983+0000","last_active":"2023-10-06T05:23:08.205911+0000","last_peered":"2023-10-06T05:23:08.205911+0000","last_clean":"2023-10-06T05:23:08.205911+0000","last_became_active":"2023-10-06T05:23:06.197524+0000","last_became_peered":"2023-10-06T05:23:06.197524+0000","last_unstale":"2023-10-06T05:23:08.205911+0000","last_undegraded":"2023-10-06T05:23:08.205911+0000","last_fullsized":"2023-10-06T05:23:08.205911+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:54.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.217713+0000","last_change":"2023-10-06T05:23:07.213731+0000","last_active":"2023-10-06T05:23:07.217713+0000","last_peered":"2023-10-06T05:23:07.217713+0000","last_clean":"2023-10-06T05:23:07.217713+0000","last_became_active":"2023-10-06T05:23:07.213396+0000","last_became_peered":"2023-10-06T05:23:07.213396+0000","last_unstale":"2023-10-06T05:23:07.217713+0000","last_undegraded":"2023-10-06T05:23:07.217713+0000","last_fullsized":"2023-10-06T05:23:07.217713+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:24.685795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.186252+0000","last_change":"2023-10-06T05:23:06.197886+0000","last_active":"2023-10-06T05:23:07.186252+0000","last_peered":"2023-10-06T05:23:07.186252+0000","last_clean":"2023-10-06T05:23:07.186252+0000","last_became_active":"2023-10-06T05:23:06.197587+0000","last_became_peered":"2023-10-06T05:23:06.197587+0000","last_unstale":"2023-10-06T05:23:07.186252+0000","last_undegraded":"2023-10-06T05:23:07.186252+0000","last_fullsized":"2023-10-06T05:23:07.186252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:08.996129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204508+0000","last_change":"2023-10-06T05:23:07.199511+0000","last_active":"2023-10-06T05:23:07.204508+0000","last_peered":"2023-10-06T05:23:07.204508+0000","last_clean":"2023-10-06T05:23:07.204508+0000","last_became_active":"2023-10-06T05:23:07.199195+0000","last_became_peered":"2023-10-06T05:23:07.199195+0000","last_unstale":"2023-10-06T05:23:07.204508+0000","last_undegraded":"2023-10-06T05:23:07.204508+0000","last_fullsized":"2023-10-06T05:23:07.204508+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.662094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193611+0000","last_change":"2023-10-06T05:23:06.198697+0000","last_active":"2023-10-06T05:23:07.193611+0000","last_peered":"2023-10-06T05:23:07.193611+0000","last_clean":"2023-10-06T05:23:07.193611+0000","last_became_active":"2023-10-06T05:23:06.198427+0000","last_became_peered":"2023-10-06T05:23:06.198427+0000","last_unstale":"2023-10-06T05:23:07.193611+0000","last_undegraded":"2023-10-06T05:23:07.193611+0000","last_fullsized":"2023-10-06T05:23:07.193611+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:41.918067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220737+0000","last_change":"2023-10-06T05:23:07.211299+0000","last_active":"2023-10-06T05:23:07.220737+0000","last_peered":"2023-10-06T05:23:07.220737+0000","last_clean":"2023-10-06T05:23:07.220737+0000","last_became_active":"2023-10-06T05:23:07.211041+0000","last_became_peered":"2023-10-06T05:23:07.211041+0000","last_unstale":"2023-10-06T05:23:07.220737+0000","last_undegraded":"2023-10-06T05:23:07.220737+0000","last_fullsized":"2023-10-06T05:23:07.220737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:27.432526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.191635+0000","last_change":"2023-10-06T05:23:06.193432+0000","last_active":"2023-10-06T05:23:07.191635+0000","last_peered":"2023-10-06T05:23:07.191635+0000","last_clean":"2023-10-06T05:23:07.191635+0000","last_became_active":"2023-10-06T05:23:06.193147+0000","last_became_peered":"2023-10-06T05:23:06.193147+0000","last_unstale":"2023-10-06T05:23:07.191635+0000","last_undegraded":"2023-10-06T05:23:07.191635+0000","last_fullsized":"2023-10-06T05:23:07.191635+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:05.641827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220937+0000","last_change":"2023-10-06T05:23:07.211294+0000","last_active":"2023-10-06T05:23:07.220937+0000","last_peered":"2023-10-06T05:23:07.220937+0000","last_clean":"2023-10-06T05:23:07.220937+0000","last_became_active":"2023-10-06T05:23:07.211003+0000","last_became_peered":"2023-10-06T05:23:07.211003+0000","last_unstale":"2023-10-06T05:23:07.220937+0000","last_undegraded":"2023-10-06T05:23:07.220937+0000","last_fullsized":"2023-10-06T05:23:07.220937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:12.847514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.192719+0000","last_change":"2023-10-06T05:23:06.195605+0000","last_active":"2023-10-06T05:23:07.192719+0000","last_peered":"2023-10-06T05:23:07.192719+0000","last_clean":"2023-10-06T05:23:07.192719+0000","last_became_active":"2023-10-06T05:23:06.195140+0000","last_became_peered":"2023-10-06T05:23:06.195140+0000","last_unstale":"2023-10-06T05:23:07.192719+0000","last_undegraded":"2023-10-06T05:23:07.192719+0000","last_fullsized":"2023-10-06T05:23:07.192719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.953606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.211392+0000","last_change":"2023-10-06T05:23:07.206761+0000","last_active":"2023-10-06T05:23:07.211392+0000","last_peered":"2023-10-06T05:23:07.211392+0000","last_clean":"2023-10-06T05:23:07.211392+0000","last_became_active":"2023-10-06T05:23:07.206481+0000","last_became_peered":"2023-10-06T05:23:07.206481+0000","last_unstale":"2023-10-06T05:23:07.211392+0000","last_undegraded":"2023-10-06T05:23:07.211392+0000","last_fullsized":"2023-10-06T05:23:07.211392+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:27.252085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.191764+0000","last_change":"2023-10-06T05:23:06.198789+0000","last_active":"2023-10-06T05:23:07.191764+0000","last_peered":"2023-10-06T05:23:07.191764+0000","last_clean":"2023-10-06T05:23:07.191764+0000","last_became_active":"2023-10-06T05:23:06.198517+0000","last_became_peered":"2023-10-06T05:23:06.198517+0000","last_unstale":"2023-10-06T05:23:07.191764+0000","last_undegraded":"2023-10-06T05:23:07.191764+0000","last_fullsized":"2023-10-06T05:23:07.191764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:40.583487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218000+0000","last_change":"2023-10-06T05:23:07.205550+0000","last_active":"2023-10-06T05:23:07.218000+0000","last_peered":"2023-10-06T05:23:07.218000+0000","last_clean":"2023-10-06T05:23:07.218000+0000","last_became_active":"2023-10-06T05:23:07.205250+0000","last_became_peered":"2023-10-06T05:23:07.205250+0000","last_unstale":"2023-10-06T05:23:07.218000+0000","last_undegraded":"2023-10-06T05:23:07.218000+0000","last_fullsized":"2023-10-06T05:23:07.218000+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:55.149048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193601+0000","last_change":"2023-10-06T05:23:06.201667+0000","last_active":"2023-10-06T05:23:07.193601+0000","last_peered":"2023-10-06T05:23:07.193601+0000","last_clean":"2023-10-06T05:23:07.193601+0000","last_became_active":"2023-10-06T05:23:06.201206+0000","last_became_peered":"2023-10-06T05:23:06.201206+0000","last_unstale":"2023-10-06T05:23:07.193601+0000","last_undegraded":"2023-10-06T05:23:07.193601+0000","last_fullsized":"2023-10-06T05:23:07.193601+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:50.043245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.200427+0000","last_change":"2023-10-06T05:23:07.197380+0000","last_active":"2023-10-06T05:23:07.200427+0000","last_peered":"2023-10-06T05:23:07.200427+0000","last_clean":"2023-10-06T05:23:07.200427+0000","last_became_active":"2023-10-06T05:23:07.197118+0000","last_became_peered":"2023-10-06T05:23:07.197118+0000","last_unstale":"2023-10-06T05:23:07.200427+0000","last_undegraded":"2023-10-06T05:23:07.200427+0000","last_fullsized":"2023-10-06T05:23:07.200427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:35.023054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.185652+0000","last_change":"2023-10-06T05:23:06.197865+0000","last_active":"2023-10-06T05:23:07.185652+0000","last_peered":"2023-10-06T05:23:07.185652+0000","last_clean":"2023-10-06T05:23:07.185652+0000","last_became_active":"2023-10-06T05:23:06.196939+0000","last_became_peered":"2023-10-06T05:23:06.196939+0000","last_unstale":"2023-10-06T05:23:07.185652+0000","last_undegraded":"2023-10-06T05:23:07.185652+0000","last_fullsized":"2023-10-06T05:23:07.185652+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:23.663009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193887+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:23:07.193887+0000","last_peered":"2023-10-06T05:23:07.193887+0000","last_clean":"2023-10-06T05:23:07.193887+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:23:07.193887+0000","last_undegraded":"2023-10-06T05:23:07.193887+0000","last_fullsized":"2023-10-06T05:23:07.193887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.209964+0000","last_change":"2023-10-06T05:23:07.205864+0000","last_active":"2023-10-06T05:23:07.209964+0000","last_peered":"2023-10-06T05:23:07.209964+0000","last_clean":"2023-10-06T05:23:07.209964+0000","last_became_active":"2023-10-06T05:23:07.203827+0000","last_became_peered":"2023-10-06T05:23:07.203827+0000","last_unstale":"2023-10-06T05:23:07.209964+0000","last_undegraded":"2023-10-06T05:23:07.209964+0000","last_fullsized":"2023-10-06T05:23:07.209964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:28.197399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.202650+0000","last_change":"2023-10-06T05:23:06.205054+0000","last_active":"2023-10-06T05:23:07.202650+0000","last_peered":"2023-10-06T05:23:07.202650+0000","last_clean":"2023-10-06T05:23:07.202650+0000","last_became_active":"2023-10-06T05:23:06.204764+0000","last_became_peered":"2023-10-06T05:23:06.204764+0000","last_unstale":"2023-10-06T05:23:07.202650+0000","last_undegraded":"2023-10-06T05:23:07.202650+0000","last_fullsized":"2023-10-06T05:23:07.202650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:57.888763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.188902+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:23:07.188902+0000","last_peered":"2023-10-06T05:23:07.188902+0000","last_clean":"2023-10-06T05:23:07.188902+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:23:07.188902+0000","last_undegraded":"2023-10-06T05:23:07.188902+0000","last_fullsized":"2023-10-06T05:23:07.188902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214525+0000","last_change":"2023-10-06T05:23:07.204932+0000","last_active":"2023-10-06T05:23:07.214525+0000","last_peered":"2023-10-06T05:23:07.214525+0000","last_clean":"2023-10-06T05:23:07.214525+0000","last_became_active":"2023-10-06T05:23:07.204639+0000","last_became_peered":"2023-10-06T05:23:07.204639+0000","last_unstale":"2023-10-06T05:23:07.214525+0000","last_undegraded":"2023-10-06T05:23:07.214525+0000","last_fullsized":"2023-10-06T05:23:07.214525+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:20.422846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.208037+0000","last_change":"2023-10-06T05:23:06.204342+0000","last_active":"2023-10-06T05:23:07.208037+0000","last_peered":"2023-10-06T05:23:07.208037+0000","last_clean":"2023-10-06T05:23:07.208037+0000","last_became_active":"2023-10-06T05:23:06.204076+0000","last_became_peered":"2023-10-06T05:23:06.204076+0000","last_unstale":"2023-10-06T05:23:07.208037+0000","last_undegraded":"2023-10-06T05:23:07.208037+0000","last_fullsized":"2023-10-06T05:23:07.208037+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:39.385198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.183474+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:23:07.183474+0000","last_peered":"2023-10-06T05:23:07.183474+0000","last_clean":"2023-10-06T05:23:07.183474+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:23:07.183474+0000","last_undegraded":"2023-10-06T05:23:07.183474+0000","last_fullsized":"2023-10-06T05:23:07.183474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.216678+0000","last_change":"2023-10-06T05:23:07.210747+0000","last_active":"2023-10-06T05:23:07.216678+0000","last_peered":"2023-10-06T05:23:07.216678+0000","last_clean":"2023-10-06T05:23:07.216678+0000","last_became_active":"2023-10-06T05:23:07.209550+0000","last_became_peered":"2023-10-06T05:23:07.209550+0000","last_unstale":"2023-10-06T05:23:07.216678+0000","last_undegraded":"2023-10-06T05:23:07.216678+0000","last_fullsized":"2023-10-06T05:23:07.216678+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:41.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"116'8","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218208+0000","last_change":"2023-10-06T05:23:06.205826+0000","last_active":"2023-10-06T05:23:07.218208+0000","last_peered":"2023-10-06T05:23:07.218208+0000","last_clean":"2023-10-06T05:23:07.218208+0000","last_became_active":"2023-10-06T05:23:06.205485+0000","last_became_peered":"2023-10-06T05:23:06.205485+0000","last_unstale":"2023-10-06T05:23:07.218208+0000","last_undegraded":"2023-10-06T05:23:07.218208+0000","last_fullsized":"2023-10-06T05:23:07.218208+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:42.747451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.188874+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:23:07.188874+0000","last_peered":"2023-10-06T05:23:07.188874+0000","last_clean":"2023-10-06T05:23:07.188874+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:23:07.188874+0000","last_undegraded":"2023-10-06T05:23:07.188874+0000","last_fullsized":"2023-10-06T05:23:07.188874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193876+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:23:07.193876+0000","last_peered":"2023-10-06T05:23:07.193876+0000","last_clean":"2023-10-06T05:23:07.193876+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:23:07.193876+0000","last_undegraded":"2023-10-06T05:23:07.193876+0000","last_fullsized":"2023-10-06T05:23:07.193876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221626+0000","last_change":"2023-10-06T05:23:07.215719+0000","last_active":"2023-10-06T05:23:07.221626+0000","last_peered":"2023-10-06T05:23:07.221626+0000","last_clean":"2023-10-06T05:23:07.221626+0000","last_became_active":"2023-10-06T05:23:07.215173+0000","last_became_peered":"2023-10-06T05:23:07.215173+0000","last_unstale":"2023-10-06T05:23:07.221626+0000","last_undegraded":"2023-10-06T05:23:07.221626+0000","last_fullsized":"2023-10-06T05:23:07.221626+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:12.273065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218435+0000","last_change":"2023-10-06T05:23:06.206088+0000","last_active":"2023-10-06T05:23:07.218435+0000","last_peered":"2023-10-06T05:23:07.218435+0000","last_clean":"2023-10-06T05:23:07.218435+0000","last_became_active":"2023-10-06T05:23:06.205802+0000","last_became_peered":"2023-10-06T05:23:06.205802+0000","last_unstale":"2023-10-06T05:23:07.218435+0000","last_undegraded":"2023-10-06T05:23:07.218435+0000","last_fullsized":"2023-10-06T05:23:07.218435+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:19.410153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193688+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:23:07.193688+0000","last_peered":"2023-10-06T05:23:07.193688+0000","last_clean":"2023-10-06T05:23:07.193688+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:23:07.193688+0000","last_undegraded":"2023-10-06T05:23:07.193688+0000","last_fullsized":"2023-10-06T05:23:07.193688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.213613+0000","last_change":"2023-10-06T05:23:07.205270+0000","last_active":"2023-10-06T05:23:07.213613+0000","last_peered":"2023-10-06T05:23:07.213613+0000","last_clean":"2023-10-06T05:23:07.213613+0000","last_became_active":"2023-10-06T05:23:07.204939+0000","last_became_peered":"2023-10-06T05:23:07.204939+0000","last_unstale":"2023-10-06T05:23:07.213613+0000","last_undegraded":"2023-10-06T05:23:07.213613+0000","last_fullsized":"2023-10-06T05:23:07.213613+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:03.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.189000+0000","last_change":"2023-10-06T05:23:06.199786+0000","last_active":"2023-10-06T05:23:07.189000+0000","last_peered":"2023-10-06T05:23:07.189000+0000","last_clean":"2023-10-06T05:23:07.189000+0000","last_became_active":"2023-10-06T05:23:06.199507+0000","last_became_peered":"2023-10-06T05:23:06.199507+0000","last_unstale":"2023-10-06T05:23:07.189000+0000","last_undegraded":"2023-10-06T05:23:07.189000+0000","last_fullsized":"2023-10-06T05:23:07.189000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:26.362255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.195303+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:23:07.195303+0000","last_peered":"2023-10-06T05:23:07.195303+0000","last_clean":"2023-10-06T05:23:07.195303+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:23:07.195303+0000","last_undegraded":"2023-10-06T05:23:07.195303+0000","last_fullsized":"2023-10-06T05:23:07.195303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224769+0000","last_change":"2023-10-06T05:23:07.223597+0000","last_active":"2023-10-06T05:23:07.224769+0000","last_peered":"2023-10-06T05:23:07.224769+0000","last_clean":"2023-10-06T05:23:07.224769+0000","last_became_active":"2023-10-06T05:23:07.223305+0000","last_became_peered":"2023-10-06T05:23:07.223305+0000","last_unstale":"2023-10-06T05:23:07.224769+0000","last_undegraded":"2023-10-06T05:23:07.224769+0000","last_fullsized":"2023-10-06T05:23:07.224769+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:27.252085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"116'3","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222704+0000","last_change":"2023-10-06T05:23:06.197088+0000","last_active":"2023-10-06T05:23:07.222704+0000","last_peered":"2023-10-06T05:23:07.222704+0000","last_clean":"2023-10-06T05:23:07.222704+0000","last_became_active":"2023-10-06T05:23:06.196775+0000","last_became_peered":"2023-10-06T05:23:06.196775+0000","last_unstale":"2023-10-06T05:23:07.222704+0000","last_undegraded":"2023-10-06T05:23:07.222704+0000","last_fullsized":"2023-10-06T05:23:07.222704+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:28.416245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.191645+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:23:07.191645+0000","last_peered":"2023-10-06T05:23:07.191645+0000","last_clean":"2023-10-06T05:23:07.191645+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:23:07.191645+0000","last_undegraded":"2023-10-06T05:23:07.191645+0000","last_fullsized":"2023-10-06T05:23:07.191645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.212853+0000","last_change":"2023-10-06T05:23:07.207631+0000","last_active":"2023-10-06T05:23:07.212853+0000","last_peered":"2023-10-06T05:23:07.212853+0000","last_clean":"2023-10-06T05:23:07.212853+0000","last_became_active":"2023-10-06T05:23:07.207359+0000","last_became_peered":"2023-10-06T05:23:07.207359+0000","last_unstale":"2023-10-06T05:23:07.212853+0000","last_undegraded":"2023-10-06T05:23:07.212853+0000","last_fullsized":"2023-10-06T05:23:07.212853+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:06.654520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"116'6","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.228798+0000","last_change":"2023-10-06T05:23:06.203784+0000","last_active":"2023-10-06T05:23:07.228798+0000","last_peered":"2023-10-06T05:23:07.228798+0000","last_clean":"2023-10-06T05:23:07.228798+0000","last_became_active":"2023-10-06T05:23:06.203511+0000","last_became_peered":"2023-10-06T05:23:06.203511+0000","last_unstale":"2023-10-06T05:23:07.228798+0000","last_undegraded":"2023-10-06T05:23:07.228798+0000","last_fullsized":"2023-10-06T05:23:07.228798+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:12.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":104,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.185882+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:23:07.185882+0000","last_peered":"2023-10-06T05:23:07.185882+0000","last_clean":"2023-10-06T05:23:07.185882+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:23:07.185882+0000","last_undegraded":"2023-10-06T05:23:07.185882+0000","last_fullsized":"2023-10-06T05:23:07.185882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303400,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068440,"statfs":{"total":96636764160,"available":96326082560,"internally_reserved":0,"allocated":2064384,"data_stored":1257903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.874},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.627},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.0369999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.93799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":667341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.748}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":666809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.374},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.5},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.86899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":1.141}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.97299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303388,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068452,"statfs":{"total":96636764160,"available":96326094848,"internally_reserved":0,"allocated":2052096,"data_stored":1257177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.624},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302804,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069036,"statfs":{"total":96636764160,"available":96326692864,"internally_reserved":0,"allocated":1454080,"data_stored":666787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.628}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.81499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.753},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":1.006}]}]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302820,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069020,"statfs":{"total":96636764160,"available":96326676480,"internally_reserved":0,"allocated":1470464,"data_stored":668357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46999999999999997,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.505,"15min":0.5},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":1.772},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.913},"last":0.378},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53900000000000003,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":1.954},"last":0.44},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":0.96999999999999997,"5min":1.2030000000000001,"15min":1.282},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91500000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":0.98499999999999999,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.84199999999999997},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.83899999999999997},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0449999999999999,"5min":1.1910000000000001,"15min":1.343},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.84899999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.034,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":0.998,"5min":1.2090000000000001,"15min":1.214},"last":0.81000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302816,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069024,"statfs":{"total":96636764160,"available":96326680576,"internally_reserved":0,"allocated":1466368,"data_stored":668323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.503},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.70899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302812,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069028,"statfs":{"total":96636764160,"available":96326684672,"internally_reserved":0,"allocated":1462272,"data_stored":666947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:23:12.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:23:12.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.789+0000 7f6cd49fe700 1 Processor -- start 2023-10-06T05:23:12.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.789+0000 7f6cd49fe700 1 -- start start 2023-10-06T05:23:12.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cd49fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6cd0155af0 0x7f6cd006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cd49fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd006ab60 con 0x7f6cd00c7920 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cd49fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd006ae40 con 0x7f6cd006a6a0 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cd49fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd006b7e0 con 0x7f6cd0155af0 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cced9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6cd0155af0 0x7f6cd006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cced9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6cd0155af0 0x7f6cd006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51678/0 (socket says 172.21.15.33:51678) 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6cced9d700 1 -- 172.21.15.33:0/1212657386 learned_addr learned my addr 172.21.15.33:0/1212657386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:12.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.791+0000 7f6ccf59e700 1 --1- 172.21.15.33:0/1212657386 >> v1:172.21.15.33:6789/0 conn(0x7f6cd00c7920 0x7f6cd00c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40370/0 (socket says 172.21.15.33:40370) 2023-10-06T05:23:12.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6cced9d700 1 -- 172.21.15.33:0/1212657386 >> v1:172.21.15.167:6789/0 conn(0x7f6cd006a6a0 legacy=0x7f6cd00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6cced9d700 1 -- 172.21.15.33:0/1212657386 >> v1:172.21.15.33:6789/0 conn(0x7f6cd00c7920 legacy=0x7f6cd00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6cced9d700 1 -- 172.21.15.33:0/1212657386 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd006b120 con 0x7f6cd0155af0 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6cced9d700 1 --2- 172.21.15.33:0/1212657386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd006a160 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00099b0 tx=0x7f6cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cc9652c8104608 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6ccdd9b700 1 -- 172.21.15.33:0/1212657386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000b860 con 0x7f6cd0155af0 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6ccdd9b700 1 -- 172.21.15.33:0/1212657386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc000b9c0 con 0x7f6cd0155af0 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.792+0000 7f6ccdd9b700 1 -- 172.21.15.33:0/1212657386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000bd60 con 0x7f6cd0155af0 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 -- 172.21.15.33:0/1212657386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 msgr2=0x7f6cd006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/1212657386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd006a160 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00099b0 tx=0x7f6cc0009250 comp rx=0 tx=0).stop 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 -- 172.21.15.33:0/1212657386 shutdown_connections 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/1212657386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd006a160 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 -- 172.21.15.33:0/1212657386 >> 172.21.15.33:0/1212657386 conn(0x7f6cd006d030 msgr2=0x7f6cd006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 -- 172.21.15.33:0/1212657386 shutdown_connections 2023-10-06T05:23:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.793+0000 7f6cd49fe700 1 -- 172.21.15.33:0/1212657386 wait complete. 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.794+0000 7f6cd49fe700 1 Processor -- start 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.794+0000 7f6cd49fe700 1 -- start start 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6cd49fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd006a6a0 0x7f6cd016b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6cd49fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd00c7920 0x7f6cd0168290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6cd49fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd00c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6cd49fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00c5c80 con 0x7f6cd0155af0 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6cd49fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00c5f20 con 0x7f6cd006a6a0 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6cd49fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd016ad40 con 0x7f6cd00c7920 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6ccf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd00c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6ccf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd00c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51688/0 (socket says 172.21.15.33:51688) 2023-10-06T05:23:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.795+0000 7f6ccf59e700 1 -- 172.21.15.33:0/2228760731 learned_addr learned my addr 172.21.15.33:0/2228760731 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6cced9d700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd006a6a0 0x7f6cd016b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6cce59c700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd00c7920 0x7f6cd0168290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6ccf59e700 1 -- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd00c7920 msgr2=0x7f6cd0168290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6ccf59e700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd00c7920 0x7f6cd0168290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6ccf59e700 1 -- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd006a6a0 msgr2=0x7f6cd016b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6ccf59e700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd006a6a0 0x7f6cd016b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6ccf59e700 1 -- 172.21.15.33:0/2228760731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc00094f0 con 0x7f6cd0155af0 2023-10-06T05:23:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6ccf59e700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd00c5740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800cd30 tx=0x7f6cc8001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8005df0 con 0x7f6cd0155af0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc800a960 con 0x7f6cd0155af0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800ad20 con 0x7f6cd0155af0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.796+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd016b020 con 0x7f6cd0155af0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.797+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd00c6200 con 0x7f6cd0155af0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.798+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd004c380 con 0x7f6cd0155af0 2023-10-06T05:23:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.798+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cc8007300 con 0x7f6cd0155af0 2023-10-06T05:23:12.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.799+0000 7f6cb6ffd700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6cbc066f30 0x7f6cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.800+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6cc80847b0 con 0x7f6cd0155af0 2023-10-06T05:23:12.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.800+0000 7f6cced9d700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6cbc066f30 0x7f6cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.801+0000 7f6cced9d700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6cbc066f30 0x7f6cbc0693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00099b0 tx=0x7f6cc0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:12.803+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cc8051940 con 0x7f6cd0155af0 2023-10-06T05:23:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.001+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cd006b400 con 0x7f6cbc066f30 2023-10-06T05:23:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.011+0000 7f6cb6ffd700 1 -- 172.21.15.33:0/2228760731 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311422 (secure 0 0 0) 0x7f6cd006b400 con 0x7f6cbc066f30 2023-10-06T05:23:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6cbc066f30 msgr2=0x7f6cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6cbc066f30 0x7f6cbc0693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00099b0 tx=0x7f6cc0017000 comp rx=0 tx=0).stop 2023-10-06T05:23:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 msgr2=0x7f6cd00c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd00c5740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800cd30 tx=0x7f6cc8001ab0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 shutdown_connections 2023-10-06T05:23:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6cbc066f30 0x7f6cbc0693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cd0155af0 0x7f6cd00c5740 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd006a6a0 0x7f6cd016b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 --2- 172.21.15.33:0/2228760731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cd00c7920 0x7f6cd0168290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 >> 172.21.15.33:0/2228760731 conn(0x7f6cd006d030 msgr2=0x7f6cd0159100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 shutdown_connections 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.015+0000 7f6cd49fe700 1 -- 172.21.15.33:0/2228760731 wait complete. 2023-10-06T05:23:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:23:13.035 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2023-10-06T05:23:11.947969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423656,"kb_used_data":12584,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551064,"statfs":{"total":773094113280,"available":770612289536,"internally_reserved":0,"allocated":12886016,"data_stored":6519644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001427"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.201337+0000","last_change":"2023-10-06T05:23:07.198281+0000","last_active":"2023-10-06T05:23:07.201337+0000","last_peered":"2023-10-06T05:23:07.201337+0000","last_clean":"2023-10-06T05:23:07.201337+0000","last_became_active":"2023-10-06T05:23:07.197957+0000","last_became_peered":"2023-10-06T05:23:07.197957+0000","last_unstale":"2023-10-06T05:23:07.201337+0000","last_undegraded":"2023-10-06T05:23:07.201337+0000","last_fullsized":"2023-10-06T05:23:07.201337+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:41.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222033+0000","last_change":"2023-10-06T05:23:07.221005+0000","last_active":"2023-10-06T05:23:07.222033+0000","last_peered":"2023-10-06T05:23:07.222033+0000","last_clean":"2023-10-06T05:23:07.222033+0000","last_became_active":"2023-10-06T05:23:07.219844+0000","last_became_peered":"2023-10-06T05:23:07.219844+0000","last_unstale":"2023-10-06T05:23:07.222033+0000","last_undegraded":"2023-10-06T05:23:07.222033+0000","last_fullsized":"2023-10-06T05:23:07.222033+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:01.003406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224468+0000","last_change":"2023-10-06T05:23:07.223793+0000","last_active":"2023-10-06T05:23:07.224468+0000","last_peered":"2023-10-06T05:23:07.224468+0000","last_clean":"2023-10-06T05:23:07.224468+0000","last_became_active":"2023-10-06T05:23:07.223463+0000","last_became_peered":"2023-10-06T05:23:07.223463+0000","last_unstale":"2023-10-06T05:23:07.224468+0000","last_undegraded":"2023-10-06T05:23:07.224468+0000","last_fullsized":"2023-10-06T05:23:07.224468+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:48.160316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222469+0000","last_change":"2023-10-06T05:23:07.217469+0000","last_active":"2023-10-06T05:23:07.222469+0000","last_peered":"2023-10-06T05:23:07.222469+0000","last_clean":"2023-10-06T05:23:07.222469+0000","last_became_active":"2023-10-06T05:23:07.217200+0000","last_became_peered":"2023-10-06T05:23:07.217200+0000","last_unstale":"2023-10-06T05:23:07.222469+0000","last_undegraded":"2023-10-06T05:23:07.222469+0000","last_fullsized":"2023-10-06T05:23:07.222469+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:20.355274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214582+0000","last_change":"2023-10-06T05:23:07.203067+0000","last_active":"2023-10-06T05:23:07.214582+0000","last_peered":"2023-10-06T05:23:07.214582+0000","last_clean":"2023-10-06T05:23:07.214582+0000","last_became_active":"2023-10-06T05:23:07.202684+0000","last_became_peered":"2023-10-06T05:23:07.202684+0000","last_unstale":"2023-10-06T05:23:07.214582+0000","last_undegraded":"2023-10-06T05:23:07.214582+0000","last_fullsized":"2023-10-06T05:23:07.214582+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:48.160316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220934+0000","last_change":"2023-10-06T05:23:07.215777+0000","last_active":"2023-10-06T05:23:07.220934+0000","last_peered":"2023-10-06T05:23:07.220934+0000","last_clean":"2023-10-06T05:23:07.220934+0000","last_became_active":"2023-10-06T05:23:07.215501+0000","last_became_peered":"2023-10-06T05:23:07.215501+0000","last_unstale":"2023-10-06T05:23:07.220934+0000","last_undegraded":"2023-10-06T05:23:07.220934+0000","last_fullsized":"2023-10-06T05:23:07.220934+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:15.494697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220890+0000","last_change":"2023-10-06T05:23:07.212624+0000","last_active":"2023-10-06T05:23:07.220890+0000","last_peered":"2023-10-06T05:23:07.220890+0000","last_clean":"2023-10-06T05:23:07.220890+0000","last_became_active":"2023-10-06T05:23:07.212396+0000","last_became_peered":"2023-10-06T05:23:07.212396+0000","last_unstale":"2023-10-06T05:23:07.220890+0000","last_undegraded":"2023-10-06T05:23:07.220890+0000","last_fullsized":"2023-10-06T05:23:07.220890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:32.049620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221131+0000","last_change":"2023-10-06T05:23:07.213372+0000","last_active":"2023-10-06T05:23:07.221131+0000","last_peered":"2023-10-06T05:23:07.221131+0000","last_clean":"2023-10-06T05:23:07.221131+0000","last_became_active":"2023-10-06T05:23:07.213125+0000","last_became_peered":"2023-10-06T05:23:07.213125+0000","last_unstale":"2023-10-06T05:23:07.221131+0000","last_undegraded":"2023-10-06T05:23:07.221131+0000","last_fullsized":"2023-10-06T05:23:07.221131+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:28.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222543+0000","last_change":"2023-10-06T05:23:07.218468+0000","last_active":"2023-10-06T05:23:07.222543+0000","last_peered":"2023-10-06T05:23:07.222543+0000","last_clean":"2023-10-06T05:23:07.222543+0000","last_became_active":"2023-10-06T05:23:07.218187+0000","last_became_peered":"2023-10-06T05:23:07.218187+0000","last_unstale":"2023-10-06T05:23:07.222543+0000","last_undegraded":"2023-10-06T05:23:07.222543+0000","last_fullsized":"2023-10-06T05:23:07.222543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:12.847514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214408+0000","last_change":"2023-10-06T05:23:07.205065+0000","last_active":"2023-10-06T05:23:07.214408+0000","last_peered":"2023-10-06T05:23:07.214408+0000","last_clean":"2023-10-06T05:23:07.214408+0000","last_became_active":"2023-10-06T05:23:07.203629+0000","last_became_peered":"2023-10-06T05:23:07.203629+0000","last_unstale":"2023-10-06T05:23:07.214408+0000","last_undegraded":"2023-10-06T05:23:07.214408+0000","last_fullsized":"2023-10-06T05:23:07.214408+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:58.966299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214634+0000","last_change":"2023-10-06T05:23:07.207345+0000","last_active":"2023-10-06T05:23:07.214634+0000","last_peered":"2023-10-06T05:23:07.214634+0000","last_clean":"2023-10-06T05:23:07.214634+0000","last_became_active":"2023-10-06T05:23:07.207072+0000","last_became_peered":"2023-10-06T05:23:07.207072+0000","last_unstale":"2023-10-06T05:23:07.214634+0000","last_undegraded":"2023-10-06T05:23:07.214634+0000","last_fullsized":"2023-10-06T05:23:07.214634+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:26.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221018+0000","last_change":"2023-10-06T05:23:07.216031+0000","last_active":"2023-10-06T05:23:07.221018+0000","last_peered":"2023-10-06T05:23:07.221018+0000","last_clean":"2023-10-06T05:23:07.221018+0000","last_became_active":"2023-10-06T05:23:07.215742+0000","last_became_peered":"2023-10-06T05:23:07.215742+0000","last_unstale":"2023-10-06T05:23:07.221018+0000","last_undegraded":"2023-10-06T05:23:07.221018+0000","last_fullsized":"2023-10-06T05:23:07.221018+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:50.930486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218010+0000","last_change":"2023-10-06T05:23:07.209257+0000","last_active":"2023-10-06T05:23:07.218010+0000","last_peered":"2023-10-06T05:23:07.218010+0000","last_clean":"2023-10-06T05:23:07.218010+0000","last_became_active":"2023-10-06T05:23:07.208919+0000","last_became_peered":"2023-10-06T05:23:07.208919+0000","last_unstale":"2023-10-06T05:23:07.218010+0000","last_undegraded":"2023-10-06T05:23:07.218010+0000","last_fullsized":"2023-10-06T05:23:07.218010+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:51.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224779+0000","last_change":"2023-10-06T05:23:07.223790+0000","last_active":"2023-10-06T05:23:07.224779+0000","last_peered":"2023-10-06T05:23:07.224779+0000","last_clean":"2023-10-06T05:23:07.224779+0000","last_became_active":"2023-10-06T05:23:07.223406+0000","last_became_peered":"2023-10-06T05:23:07.223406+0000","last_unstale":"2023-10-06T05:23:07.224779+0000","last_undegraded":"2023-10-06T05:23:07.224779+0000","last_fullsized":"2023-10-06T05:23:07.224779+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:24.685795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214658+0000","last_change":"2023-10-06T05:23:07.206656+0000","last_active":"2023-10-06T05:23:07.214658+0000","last_peered":"2023-10-06T05:23:07.214658+0000","last_clean":"2023-10-06T05:23:07.214658+0000","last_became_active":"2023-10-06T05:23:07.206381+0000","last_became_peered":"2023-10-06T05:23:07.206381+0000","last_unstale":"2023-10-06T05:23:07.214658+0000","last_undegraded":"2023-10-06T05:23:07.214658+0000","last_fullsized":"2023-10-06T05:23:07.214658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:20.355274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.208346+0000","last_change":"2023-10-06T05:23:07.204953+0000","last_active":"2023-10-06T05:23:07.208346+0000","last_peered":"2023-10-06T05:23:07.208346+0000","last_clean":"2023-10-06T05:23:07.208346+0000","last_became_active":"2023-10-06T05:23:07.204643+0000","last_became_peered":"2023-10-06T05:23:07.204643+0000","last_unstale":"2023-10-06T05:23:07.208346+0000","last_undegraded":"2023-10-06T05:23:07.208346+0000","last_fullsized":"2023-10-06T05:23:07.208346+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:55.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214553+0000","last_change":"2023-10-06T05:23:07.207642+0000","last_active":"2023-10-06T05:23:07.214553+0000","last_peered":"2023-10-06T05:23:07.214553+0000","last_clean":"2023-10-06T05:23:07.214553+0000","last_became_active":"2023-10-06T05:23:07.207390+0000","last_became_peered":"2023-10-06T05:23:07.207390+0000","last_unstale":"2023-10-06T05:23:07.214553+0000","last_undegraded":"2023-10-06T05:23:07.214553+0000","last_fullsized":"2023-10-06T05:23:07.214553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:06.654520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221201+0000","last_change":"2023-10-06T05:23:07.219544+0000","last_active":"2023-10-06T05:23:07.221201+0000","last_peered":"2023-10-06T05:23:07.221201+0000","last_clean":"2023-10-06T05:23:07.221201+0000","last_became_active":"2023-10-06T05:23:07.219223+0000","last_became_peered":"2023-10-06T05:23:07.219223+0000","last_unstale":"2023-10-06T05:23:07.221201+0000","last_undegraded":"2023-10-06T05:23:07.221201+0000","last_fullsized":"2023-10-06T05:23:07.221201+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:35.452400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.205371+0000","last_change":"2023-10-06T05:23:07.199697+0000","last_active":"2023-10-06T05:23:07.205371+0000","last_peered":"2023-10-06T05:23:07.205371+0000","last_clean":"2023-10-06T05:23:07.205371+0000","last_became_active":"2023-10-06T05:23:07.199439+0000","last_became_peered":"2023-10-06T05:23:07.199439+0000","last_unstale":"2023-10-06T05:23:07.205371+0000","last_undegraded":"2023-10-06T05:23:07.205371+0000","last_fullsized":"2023-10-06T05:23:07.205371+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:09.989368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.203884+0000","last_change":"2023-10-06T05:23:07.197182+0000","last_active":"2023-10-06T05:23:07.203884+0000","last_peered":"2023-10-06T05:23:07.203884+0000","last_clean":"2023-10-06T05:23:07.203884+0000","last_became_active":"2023-10-06T05:23:07.196841+0000","last_became_peered":"2023-10-06T05:23:07.196841+0000","last_unstale":"2023-10-06T05:23:07.203884+0000","last_undegraded":"2023-10-06T05:23:07.203884+0000","last_fullsized":"2023-10-06T05:23:07.203884+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:37.144809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.216659+0000","last_change":"2023-10-06T05:23:07.210237+0000","last_active":"2023-10-06T05:23:07.216659+0000","last_peered":"2023-10-06T05:23:07.216659+0000","last_clean":"2023-10-06T05:23:07.216659+0000","last_became_active":"2023-10-06T05:23:07.209159+0000","last_became_peered":"2023-10-06T05:23:07.209159+0000","last_unstale":"2023-10-06T05:23:07.216659+0000","last_undegraded":"2023-10-06T05:23:07.216659+0000","last_fullsized":"2023-10-06T05:23:07.216659+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:05.463087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221844+0000","last_change":"2023-10-06T05:23:07.214878+0000","last_active":"2023-10-06T05:23:07.221844+0000","last_peered":"2023-10-06T05:23:07.221844+0000","last_clean":"2023-10-06T05:23:07.221844+0000","last_became_active":"2023-10-06T05:23:07.214605+0000","last_became_peered":"2023-10-06T05:23:07.214605+0000","last_unstale":"2023-10-06T05:23:07.221844+0000","last_undegraded":"2023-10-06T05:23:07.221844+0000","last_fullsized":"2023-10-06T05:23:07.221844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:27.252085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222495+0000","last_change":"2023-10-06T05:23:07.218164+0000","last_active":"2023-10-06T05:23:07.222495+0000","last_peered":"2023-10-06T05:23:07.222495+0000","last_clean":"2023-10-06T05:23:07.222495+0000","last_became_active":"2023-10-06T05:23:07.217857+0000","last_became_peered":"2023-10-06T05:23:07.217857+0000","last_unstale":"2023-10-06T05:23:07.222495+0000","last_undegraded":"2023-10-06T05:23:07.222495+0000","last_fullsized":"2023-10-06T05:23:07.222495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:35.452400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210368+0000","last_change":"2023-10-06T05:23:07.202649+0000","last_active":"2023-10-06T05:23:07.210368+0000","last_peered":"2023-10-06T05:23:07.210368+0000","last_clean":"2023-10-06T05:23:07.210368+0000","last_became_active":"2023-10-06T05:23:07.202341+0000","last_became_peered":"2023-10-06T05:23:07.202341+0000","last_unstale":"2023-10-06T05:23:07.210368+0000","last_undegraded":"2023-10-06T05:23:07.210368+0000","last_fullsized":"2023-10-06T05:23:07.210368+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:20.422846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204198+0000","last_change":"2023-10-06T05:23:07.198523+0000","last_active":"2023-10-06T05:23:07.204198+0000","last_peered":"2023-10-06T05:23:07.204198+0000","last_clean":"2023-10-06T05:23:07.204198+0000","last_became_active":"2023-10-06T05:23:07.197618+0000","last_became_peered":"2023-10-06T05:23:07.197618+0000","last_unstale":"2023-10-06T05:23:07.204198+0000","last_undegraded":"2023-10-06T05:23:07.204198+0000","last_fullsized":"2023-10-06T05:23:07.204198+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:51.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.203937+0000","last_change":"2023-10-06T05:23:07.198118+0000","last_active":"2023-10-06T05:23:07.203937+0000","last_peered":"2023-10-06T05:23:07.203937+0000","last_clean":"2023-10-06T05:23:07.203937+0000","last_became_active":"2023-10-06T05:23:07.197868+0000","last_became_peered":"2023-10-06T05:23:07.197868+0000","last_unstale":"2023-10-06T05:23:07.203937+0000","last_undegraded":"2023-10-06T05:23:07.203937+0000","last_fullsized":"2023-10-06T05:23:07.203937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:56.475473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.205391+0000","last_change":"2023-10-06T05:23:07.200369+0000","last_active":"2023-10-06T05:23:07.205391+0000","last_peered":"2023-10-06T05:23:07.205391+0000","last_clean":"2023-10-06T05:23:07.205391+0000","last_became_active":"2023-10-06T05:23:07.200088+0000","last_became_peered":"2023-10-06T05:23:07.200088+0000","last_unstale":"2023-10-06T05:23:07.205391+0000","last_undegraded":"2023-10-06T05:23:07.205391+0000","last_fullsized":"2023-10-06T05:23:07.205391+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:31.144319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210924+0000","last_change":"2023-10-06T05:23:07.206937+0000","last_active":"2023-10-06T05:23:07.210924+0000","last_peered":"2023-10-06T05:23:07.210924+0000","last_clean":"2023-10-06T05:23:07.210924+0000","last_became_active":"2023-10-06T05:23:07.206648+0000","last_became_peered":"2023-10-06T05:23:07.206648+0000","last_unstale":"2023-10-06T05:23:07.210924+0000","last_undegraded":"2023-10-06T05:23:07.210924+0000","last_fullsized":"2023-10-06T05:23:07.210924+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:56.975166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210006+0000","last_change":"2023-10-06T05:23:07.204709+0000","last_active":"2023-10-06T05:23:07.210006+0000","last_peered":"2023-10-06T05:23:07.210006+0000","last_clean":"2023-10-06T05:23:07.210006+0000","last_became_active":"2023-10-06T05:23:07.204414+0000","last_became_peered":"2023-10-06T05:23:07.204414+0000","last_unstale":"2023-10-06T05:23:07.210006+0000","last_undegraded":"2023-10-06T05:23:07.210006+0000","last_fullsized":"2023-10-06T05:23:07.210006+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:17.164286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.223471+0000","last_change":"2023-10-06T05:23:07.221989+0000","last_active":"2023-10-06T05:23:07.223471+0000","last_peered":"2023-10-06T05:23:07.223471+0000","last_clean":"2023-10-06T05:23:07.223471+0000","last_became_active":"2023-10-06T05:23:07.221668+0000","last_became_peered":"2023-10-06T05:23:07.221668+0000","last_unstale":"2023-10-06T05:23:07.223471+0000","last_undegraded":"2023-10-06T05:23:07.223471+0000","last_fullsized":"2023-10-06T05:23:07.223471+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:05.463087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.212831+0000","last_change":"2023-10-06T05:23:07.205279+0000","last_active":"2023-10-06T05:23:07.212831+0000","last_peered":"2023-10-06T05:23:07.212831+0000","last_clean":"2023-10-06T05:23:07.212831+0000","last_became_active":"2023-10-06T05:23:07.204901+0000","last_became_peered":"2023-10-06T05:23:07.204901+0000","last_unstale":"2023-10-06T05:23:07.212831+0000","last_undegraded":"2023-10-06T05:23:07.212831+0000","last_fullsized":"2023-10-06T05:23:07.212831+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:55.149048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210389+0000","last_change":"2023-10-06T05:23:07.203599+0000","last_active":"2023-10-06T05:23:07.210389+0000","last_peered":"2023-10-06T05:23:07.210389+0000","last_clean":"2023-10-06T05:23:07.210389+0000","last_became_active":"2023-10-06T05:23:07.203332+0000","last_became_peered":"2023-10-06T05:23:07.203332+0000","last_unstale":"2023-10-06T05:23:07.210389+0000","last_undegraded":"2023-10-06T05:23:07.210389+0000","last_fullsized":"2023-10-06T05:23:07.210389+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:58.901456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214598+0000","last_change":"2023-10-06T05:23:07.202910+0000","last_active":"2023-10-06T05:23:07.214598+0000","last_peered":"2023-10-06T05:23:07.214598+0000","last_clean":"2023-10-06T05:23:07.214598+0000","last_became_active":"2023-10-06T05:23:07.202639+0000","last_became_peered":"2023-10-06T05:23:07.202639+0000","last_unstale":"2023-10-06T05:23:07.214598+0000","last_undegraded":"2023-10-06T05:23:07.214598+0000","last_fullsized":"2023-10-06T05:23:07.214598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.373703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"116'4","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218297+0000","last_change":"2023-10-06T05:23:06.198817+0000","last_active":"2023-10-06T05:23:07.218297+0000","last_peered":"2023-10-06T05:23:07.218297+0000","last_clean":"2023-10-06T05:23:07.218297+0000","last_became_active":"2023-10-06T05:23:06.198525+0000","last_became_peered":"2023-10-06T05:23:06.198525+0000","last_unstale":"2023-10-06T05:23:07.218297+0000","last_undegraded":"2023-10-06T05:23:07.218297+0000","last_fullsized":"2023-10-06T05:23:07.218297+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:24.694784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.210777+0000","last_change":"2023-10-06T05:23:07.208244+0000","last_active":"2023-10-06T05:23:07.210777+0000","last_peered":"2023-10-06T05:23:07.210777+0000","last_clean":"2023-10-06T05:23:07.210777+0000","last_became_active":"2023-10-06T05:23:07.207983+0000","last_became_peered":"2023-10-06T05:23:07.207983+0000","last_unstale":"2023-10-06T05:23:07.210777+0000","last_undegraded":"2023-10-06T05:23:07.210777+0000","last_fullsized":"2023-10-06T05:23:07.210777+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:03.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214440+0000","last_change":"2023-10-06T05:23:06.190681+0000","last_active":"2023-10-06T05:23:07.214440+0000","last_peered":"2023-10-06T05:23:07.214440+0000","last_clean":"2023-10-06T05:23:07.214440+0000","last_became_active":"2023-10-06T05:23:06.190403+0000","last_became_peered":"2023-10-06T05:23:06.190403+0000","last_unstale":"2023-10-06T05:23:07.214440+0000","last_undegraded":"2023-10-06T05:23:07.214440+0000","last_fullsized":"2023-10-06T05:23:07.214440+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:37.354403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.205064+0000","last_change":"2023-10-06T05:23:07.199681+0000","last_active":"2023-10-06T05:23:07.205064+0000","last_peered":"2023-10-06T05:23:07.205064+0000","last_clean":"2023-10-06T05:23:07.205064+0000","last_became_active":"2023-10-06T05:23:07.199411+0000","last_became_peered":"2023-10-06T05:23:07.199411+0000","last_unstale":"2023-10-06T05:23:07.205064+0000","last_undegraded":"2023-10-06T05:23:07.205064+0000","last_fullsized":"2023-10-06T05:23:07.205064+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:58.966299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218395+0000","last_change":"2023-10-06T05:23:06.200524+0000","last_active":"2023-10-06T05:23:07.218395+0000","last_peered":"2023-10-06T05:23:07.218395+0000","last_clean":"2023-10-06T05:23:07.218395+0000","last_became_active":"2023-10-06T05:23:06.200263+0000","last_became_peered":"2023-10-06T05:23:06.200263+0000","last_unstale":"2023-10-06T05:23:07.218395+0000","last_undegraded":"2023-10-06T05:23:07.218395+0000","last_fullsized":"2023-10-06T05:23:07.218395+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.361010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.211865+0000","last_change":"2023-10-06T05:23:07.207032+0000","last_active":"2023-10-06T05:23:07.211865+0000","last_peered":"2023-10-06T05:23:07.211865+0000","last_clean":"2023-10-06T05:23:07.211865+0000","last_became_active":"2023-10-06T05:23:07.206732+0000","last_became_peered":"2023-10-06T05:23:07.206732+0000","last_unstale":"2023-10-06T05:23:07.211865+0000","last_undegraded":"2023-10-06T05:23:07.211865+0000","last_fullsized":"2023-10-06T05:23:07.211865+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:26.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218541+0000","last_change":"2023-10-06T05:23:06.199909+0000","last_active":"2023-10-06T05:23:07.218541+0000","last_peered":"2023-10-06T05:23:07.218541+0000","last_clean":"2023-10-06T05:23:07.218541+0000","last_became_active":"2023-10-06T05:23:06.199574+0000","last_became_peered":"2023-10-06T05:23:06.199574+0000","last_unstale":"2023-10-06T05:23:07.218541+0000","last_undegraded":"2023-10-06T05:23:07.218541+0000","last_fullsized":"2023-10-06T05:23:07.218541+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:28.416245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214690+0000","last_change":"2023-10-06T05:23:07.204595+0000","last_active":"2023-10-06T05:23:07.214690+0000","last_peered":"2023-10-06T05:23:07.214690+0000","last_clean":"2023-10-06T05:23:07.214690+0000","last_became_active":"2023-10-06T05:23:07.204313+0000","last_became_peered":"2023-10-06T05:23:07.204313+0000","last_unstale":"2023-10-06T05:23:07.214690+0000","last_undegraded":"2023-10-06T05:23:07.214690+0000","last_fullsized":"2023-10-06T05:23:07.214690+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:43.760144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.219062+0000","last_change":"2023-10-06T05:23:06.199624+0000","last_active":"2023-10-06T05:23:07.219062+0000","last_peered":"2023-10-06T05:23:07.219062+0000","last_clean":"2023-10-06T05:23:07.219062+0000","last_became_active":"2023-10-06T05:23:06.199367+0000","last_became_peered":"2023-10-06T05:23:06.199367+0000","last_unstale":"2023-10-06T05:23:07.219062+0000","last_undegraded":"2023-10-06T05:23:07.219062+0000","last_fullsized":"2023-10-06T05:23:07.219062+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:53.211927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214531+0000","last_change":"2023-10-06T05:23:07.202632+0000","last_active":"2023-10-06T05:23:07.214531+0000","last_peered":"2023-10-06T05:23:07.214531+0000","last_clean":"2023-10-06T05:23:07.214531+0000","last_became_active":"2023-10-06T05:23:07.202340+0000","last_became_peered":"2023-10-06T05:23:07.202340+0000","last_unstale":"2023-10-06T05:23:07.214531+0000","last_undegraded":"2023-10-06T05:23:07.214531+0000","last_fullsized":"2023-10-06T05:23:07.214531+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:05.463087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224697+0000","last_change":"2023-10-06T05:23:06.200316+0000","last_active":"2023-10-06T05:23:07.224697+0000","last_peered":"2023-10-06T05:23:07.224697+0000","last_clean":"2023-10-06T05:23:07.224697+0000","last_became_active":"2023-10-06T05:23:06.200053+0000","last_became_peered":"2023-10-06T05:23:06.200053+0000","last_unstale":"2023-10-06T05:23:07.224697+0000","last_undegraded":"2023-10-06T05:23:07.224697+0000","last_fullsized":"2023-10-06T05:23:07.224697+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:15.368204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221190+0000","last_change":"2023-10-06T05:23:07.216861+0000","last_active":"2023-10-06T05:23:07.221190+0000","last_peered":"2023-10-06T05:23:07.221190+0000","last_clean":"2023-10-06T05:23:07.221190+0000","last_became_active":"2023-10-06T05:23:07.216589+0000","last_became_peered":"2023-10-06T05:23:07.216589+0000","last_unstale":"2023-10-06T05:23:07.221190+0000","last_undegraded":"2023-10-06T05:23:07.221190+0000","last_fullsized":"2023-10-06T05:23:07.221190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:41.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.188799+0000","last_change":"2023-10-06T05:23:06.200972+0000","last_active":"2023-10-06T05:23:07.188799+0000","last_peered":"2023-10-06T05:23:07.188799+0000","last_clean":"2023-10-06T05:23:07.188799+0000","last_became_active":"2023-10-06T05:23:06.200703+0000","last_became_peered":"2023-10-06T05:23:06.200703+0000","last_unstale":"2023-10-06T05:23:07.188799+0000","last_undegraded":"2023-10-06T05:23:07.188799+0000","last_fullsized":"2023-10-06T05:23:07.188799+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:02.452310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220867+0000","last_change":"2023-10-06T05:23:07.219132+0000","last_active":"2023-10-06T05:23:07.220867+0000","last_peered":"2023-10-06T05:23:07.220867+0000","last_clean":"2023-10-06T05:23:07.220867+0000","last_became_active":"2023-10-06T05:23:07.218899+0000","last_became_peered":"2023-10-06T05:23:07.218899+0000","last_unstale":"2023-10-06T05:23:07.220867+0000","last_undegraded":"2023-10-06T05:23:07.220867+0000","last_fullsized":"2023-10-06T05:23:07.220867+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:54.224620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.190488+0000","last_change":"2023-10-06T05:23:06.201230+0000","last_active":"2023-10-06T05:23:07.190488+0000","last_peered":"2023-10-06T05:23:07.190488+0000","last_clean":"2023-10-06T05:23:07.190488+0000","last_became_active":"2023-10-06T05:23:06.200890+0000","last_became_peered":"2023-10-06T05:23:06.200890+0000","last_unstale":"2023-10-06T05:23:07.190488+0000","last_undegraded":"2023-10-06T05:23:07.190488+0000","last_fullsized":"2023-10-06T05:23:07.190488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:50.911792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204680+0000","last_change":"2023-10-06T05:23:07.200537+0000","last_active":"2023-10-06T05:23:07.204680+0000","last_peered":"2023-10-06T05:23:07.204680+0000","last_clean":"2023-10-06T05:23:07.204680+0000","last_became_active":"2023-10-06T05:23:07.200039+0000","last_became_peered":"2023-10-06T05:23:07.200039+0000","last_unstale":"2023-10-06T05:23:07.204680+0000","last_undegraded":"2023-10-06T05:23:07.204680+0000","last_fullsized":"2023-10-06T05:23:07.204680+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:54.224620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.217842+0000","last_change":"2023-10-06T05:23:06.196899+0000","last_active":"2023-10-06T05:23:07.217842+0000","last_peered":"2023-10-06T05:23:07.217842+0000","last_clean":"2023-10-06T05:23:07.217842+0000","last_became_active":"2023-10-06T05:23:06.196553+0000","last_became_peered":"2023-10-06T05:23:06.196553+0000","last_unstale":"2023-10-06T05:23:07.217842+0000","last_undegraded":"2023-10-06T05:23:07.217842+0000","last_fullsized":"2023-10-06T05:23:07.217842+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:41.942266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214480+0000","last_change":"2023-10-06T05:23:07.204031+0000","last_active":"2023-10-06T05:23:07.214480+0000","last_peered":"2023-10-06T05:23:07.214480+0000","last_clean":"2023-10-06T05:23:07.214480+0000","last_became_active":"2023-10-06T05:23:07.203769+0000","last_became_peered":"2023-10-06T05:23:07.203769+0000","last_unstale":"2023-10-06T05:23:07.214480+0000","last_undegraded":"2023-10-06T05:23:07.214480+0000","last_fullsized":"2023-10-06T05:23:07.214480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:24.685795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.209208+0000","last_change":"2023-10-06T05:23:06.202660+0000","last_active":"2023-10-06T05:23:07.209208+0000","last_peered":"2023-10-06T05:23:07.209208+0000","last_clean":"2023-10-06T05:23:07.209208+0000","last_became_active":"2023-10-06T05:23:06.202082+0000","last_became_peered":"2023-10-06T05:23:06.202082+0000","last_unstale":"2023-10-06T05:23:07.209208+0000","last_undegraded":"2023-10-06T05:23:07.209208+0000","last_fullsized":"2023-10-06T05:23:07.209208+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:14.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.207107+0000","last_change":"2023-10-06T05:23:07.201677+0000","last_active":"2023-10-06T05:23:07.207107+0000","last_peered":"2023-10-06T05:23:07.207107+0000","last_clean":"2023-10-06T05:23:07.207107+0000","last_became_active":"2023-10-06T05:23:07.201418+0000","last_became_peered":"2023-10-06T05:23:07.201418+0000","last_unstale":"2023-10-06T05:23:07.207107+0000","last_undegraded":"2023-10-06T05:23:07.207107+0000","last_fullsized":"2023-10-06T05:23:07.207107+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:56.704239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218052+0000","last_change":"2023-10-06T05:23:06.203653+0000","last_active":"2023-10-06T05:23:07.218052+0000","last_peered":"2023-10-06T05:23:07.218052+0000","last_clean":"2023-10-06T05:23:07.218052+0000","last_became_active":"2023-10-06T05:23:06.203334+0000","last_became_peered":"2023-10-06T05:23:06.203334+0000","last_unstale":"2023-10-06T05:23:07.218052+0000","last_undegraded":"2023-10-06T05:23:07.218052+0000","last_fullsized":"2023-10-06T05:23:07.218052+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:53.211927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204218+0000","last_change":"2023-10-06T05:23:07.200406+0000","last_active":"2023-10-06T05:23:07.204218+0000","last_peered":"2023-10-06T05:23:07.204218+0000","last_clean":"2023-10-06T05:23:07.204218+0000","last_became_active":"2023-10-06T05:23:07.199845+0000","last_became_peered":"2023-10-06T05:23:07.199845+0000","last_unstale":"2023-10-06T05:23:07.204218+0000","last_undegraded":"2023-10-06T05:23:07.204218+0000","last_fullsized":"2023-10-06T05:23:07.204218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:13.685691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.195000+0000","last_change":"2023-10-06T05:23:06.202829+0000","last_active":"2023-10-06T05:23:07.195000+0000","last_peered":"2023-10-06T05:23:07.195000+0000","last_clean":"2023-10-06T05:23:07.195000+0000","last_became_active":"2023-10-06T05:23:06.202182+0000","last_became_peered":"2023-10-06T05:23:06.202182+0000","last_unstale":"2023-10-06T05:23:07.195000+0000","last_undegraded":"2023-10-06T05:23:07.195000+0000","last_fullsized":"2023-10-06T05:23:07.195000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:05.641827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.207063+0000","last_change":"2023-10-06T05:23:07.201685+0000","last_active":"2023-10-06T05:23:07.207063+0000","last_peered":"2023-10-06T05:23:07.207063+0000","last_clean":"2023-10-06T05:23:07.207063+0000","last_became_active":"2023-10-06T05:23:07.201460+0000","last_became_peered":"2023-10-06T05:23:07.201460+0000","last_unstale":"2023-10-06T05:23:07.207063+0000","last_undegraded":"2023-10-06T05:23:07.207063+0000","last_fullsized":"2023-10-06T05:23:07.207063+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:54.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222480+0000","last_change":"2023-10-06T05:23:06.197993+0000","last_active":"2023-10-06T05:23:07.222480+0000","last_peered":"2023-10-06T05:23:07.222480+0000","last_clean":"2023-10-06T05:23:07.222480+0000","last_became_active":"2023-10-06T05:23:06.197634+0000","last_became_peered":"2023-10-06T05:23:06.197634+0000","last_unstale":"2023-10-06T05:23:07.222480+0000","last_undegraded":"2023-10-06T05:23:07.222480+0000","last_fullsized":"2023-10-06T05:23:07.222480+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:21.132679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220964+0000","last_change":"2023-10-06T05:23:07.211979+0000","last_active":"2023-10-06T05:23:07.220964+0000","last_peered":"2023-10-06T05:23:07.220964+0000","last_clean":"2023-10-06T05:23:07.220964+0000","last_became_active":"2023-10-06T05:23:07.211621+0000","last_became_peered":"2023-10-06T05:23:07.211621+0000","last_unstale":"2023-10-06T05:23:07.220964+0000","last_undegraded":"2023-10-06T05:23:07.220964+0000","last_fullsized":"2023-10-06T05:23:07.220964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:26.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.195326+0000","last_change":"2023-10-06T05:23:06.202339+0000","last_active":"2023-10-06T05:23:07.195326+0000","last_peered":"2023-10-06T05:23:07.195326+0000","last_clean":"2023-10-06T05:23:07.195326+0000","last_became_active":"2023-10-06T05:23:06.201873+0000","last_became_peered":"2023-10-06T05:23:06.201873+0000","last_unstale":"2023-10-06T05:23:07.195326+0000","last_undegraded":"2023-10-06T05:23:07.195326+0000","last_fullsized":"2023-10-06T05:23:07.195326+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.953606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221053+0000","last_change":"2023-10-06T05:23:07.212696+0000","last_active":"2023-10-06T05:23:07.221053+0000","last_peered":"2023-10-06T05:23:07.221053+0000","last_clean":"2023-10-06T05:23:07.221053+0000","last_became_active":"2023-10-06T05:23:07.212373+0000","last_became_peered":"2023-10-06T05:23:07.212373+0000","last_unstale":"2023-10-06T05:23:07.221053+0000","last_undegraded":"2023-10-06T05:23:07.221053+0000","last_fullsized":"2023-10-06T05:23:07.221053+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:20.355274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.202778+0000","last_change":"2023-10-06T05:23:06.204978+0000","last_active":"2023-10-06T05:23:07.202778+0000","last_peered":"2023-10-06T05:23:07.202778+0000","last_clean":"2023-10-06T05:23:07.202778+0000","last_became_active":"2023-10-06T05:23:06.204599+0000","last_became_peered":"2023-10-06T05:23:06.204599+0000","last_unstale":"2023-10-06T05:23:07.202778+0000","last_undegraded":"2023-10-06T05:23:07.202778+0000","last_fullsized":"2023-10-06T05:23:07.202778+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:02.282810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204523+0000","last_change":"2023-10-06T05:23:07.199940+0000","last_active":"2023-10-06T05:23:07.204523+0000","last_peered":"2023-10-06T05:23:07.204523+0000","last_clean":"2023-10-06T05:23:07.204523+0000","last_became_active":"2023-10-06T05:23:07.199633+0000","last_became_peered":"2023-10-06T05:23:07.199633+0000","last_unstale":"2023-10-06T05:23:07.204523+0000","last_undegraded":"2023-10-06T05:23:07.204523+0000","last_fullsized":"2023-10-06T05:23:07.204523+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.373703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193486+0000","last_change":"2023-10-06T05:23:06.196929+0000","last_active":"2023-10-06T05:23:07.193486+0000","last_peered":"2023-10-06T05:23:07.193486+0000","last_clean":"2023-10-06T05:23:07.193486+0000","last_became_active":"2023-10-06T05:23:06.196631+0000","last_became_peered":"2023-10-06T05:23:06.196631+0000","last_unstale":"2023-10-06T05:23:07.193486+0000","last_undegraded":"2023-10-06T05:23:07.193486+0000","last_fullsized":"2023-10-06T05:23:07.193486+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.136355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.207027+0000","last_change":"2023-10-06T05:23:07.202684+0000","last_active":"2023-10-06T05:23:07.207027+0000","last_peered":"2023-10-06T05:23:07.207027+0000","last_clean":"2023-10-06T05:23:07.207027+0000","last_became_active":"2023-10-06T05:23:07.202304+0000","last_became_peered":"2023-10-06T05:23:07.202304+0000","last_unstale":"2023-10-06T05:23:07.207027+0000","last_undegraded":"2023-10-06T05:23:07.207027+0000","last_fullsized":"2023-10-06T05:23:07.207027+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:50.391306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220924+0000","last_change":"2023-10-06T05:23:06.201976+0000","last_active":"2023-10-06T05:23:07.220924+0000","last_peered":"2023-10-06T05:23:07.220924+0000","last_clean":"2023-10-06T05:23:07.220924+0000","last_became_active":"2023-10-06T05:23:06.201627+0000","last_became_peered":"2023-10-06T05:23:06.201627+0000","last_unstale":"2023-10-06T05:23:07.220924+0000","last_undegraded":"2023-10-06T05:23:07.220924+0000","last_fullsized":"2023-10-06T05:23:07.220924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:12.215526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214564+0000","last_change":"2023-10-06T05:23:07.209042+0000","last_active":"2023-10-06T05:23:07.214564+0000","last_peered":"2023-10-06T05:23:07.214564+0000","last_clean":"2023-10-06T05:23:07.214564+0000","last_became_active":"2023-10-06T05:23:07.208724+0000","last_became_peered":"2023-10-06T05:23:07.208724+0000","last_unstale":"2023-10-06T05:23:07.214564+0000","last_undegraded":"2023-10-06T05:23:07.214564+0000","last_fullsized":"2023-10-06T05:23:07.214564+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:58.901456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"117'1","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T05:23:08.205911+0000","last_change":"2023-10-06T05:23:06.197983+0000","last_active":"2023-10-06T05:23:08.205911+0000","last_peered":"2023-10-06T05:23:08.205911+0000","last_clean":"2023-10-06T05:23:08.205911+0000","last_became_active":"2023-10-06T05:23:06.197524+0000","last_became_peered":"2023-10-06T05:23:06.197524+0000","last_unstale":"2023-10-06T05:23:08.205911+0000","last_undegraded":"2023-10-06T05:23:08.205911+0000","last_fullsized":"2023-10-06T05:23:08.205911+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:54.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.217713+0000","last_change":"2023-10-06T05:23:07.213731+0000","last_active":"2023-10-06T05:23:07.217713+0000","last_peered":"2023-10-06T05:23:07.217713+0000","last_clean":"2023-10-06T05:23:07.217713+0000","last_became_active":"2023-10-06T05:23:07.213396+0000","last_became_peered":"2023-10-06T05:23:07.213396+0000","last_unstale":"2023-10-06T05:23:07.217713+0000","last_undegraded":"2023-10-06T05:23:07.217713+0000","last_fullsized":"2023-10-06T05:23:07.217713+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:24.685795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.186252+0000","last_change":"2023-10-06T05:23:06.197886+0000","last_active":"2023-10-06T05:23:07.186252+0000","last_peered":"2023-10-06T05:23:07.186252+0000","last_clean":"2023-10-06T05:23:07.186252+0000","last_became_active":"2023-10-06T05:23:06.197587+0000","last_became_peered":"2023-10-06T05:23:06.197587+0000","last_unstale":"2023-10-06T05:23:07.186252+0000","last_undegraded":"2023-10-06T05:23:07.186252+0000","last_fullsized":"2023-10-06T05:23:07.186252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:08.996129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.204508+0000","last_change":"2023-10-06T05:23:07.199511+0000","last_active":"2023-10-06T05:23:07.204508+0000","last_peered":"2023-10-06T05:23:07.204508+0000","last_clean":"2023-10-06T05:23:07.204508+0000","last_became_active":"2023-10-06T05:23:07.199195+0000","last_became_peered":"2023-10-06T05:23:07.199195+0000","last_unstale":"2023-10-06T05:23:07.204508+0000","last_undegraded":"2023-10-06T05:23:07.204508+0000","last_fullsized":"2023-10-06T05:23:07.204508+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.662094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193611+0000","last_change":"2023-10-06T05:23:06.198697+0000","last_active":"2023-10-06T05:23:07.193611+0000","last_peered":"2023-10-06T05:23:07.193611+0000","last_clean":"2023-10-06T05:23:07.193611+0000","last_became_active":"2023-10-06T05:23:06.198427+0000","last_became_peered":"2023-10-06T05:23:06.198427+0000","last_unstale":"2023-10-06T05:23:07.193611+0000","last_undegraded":"2023-10-06T05:23:07.193611+0000","last_fullsized":"2023-10-06T05:23:07.193611+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:41.918067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220737+0000","last_change":"2023-10-06T05:23:07.211299+0000","last_active":"2023-10-06T05:23:07.220737+0000","last_peered":"2023-10-06T05:23:07.220737+0000","last_clean":"2023-10-06T05:23:07.220737+0000","last_became_active":"2023-10-06T05:23:07.211041+0000","last_became_peered":"2023-10-06T05:23:07.211041+0000","last_unstale":"2023-10-06T05:23:07.220737+0000","last_undegraded":"2023-10-06T05:23:07.220737+0000","last_fullsized":"2023-10-06T05:23:07.220737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:27.432526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.191635+0000","last_change":"2023-10-06T05:23:06.193432+0000","last_active":"2023-10-06T05:23:07.191635+0000","last_peered":"2023-10-06T05:23:07.191635+0000","last_clean":"2023-10-06T05:23:07.191635+0000","last_became_active":"2023-10-06T05:23:06.193147+0000","last_became_peered":"2023-10-06T05:23:06.193147+0000","last_unstale":"2023-10-06T05:23:07.191635+0000","last_undegraded":"2023-10-06T05:23:07.191635+0000","last_fullsized":"2023-10-06T05:23:07.191635+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:05.641827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.220937+0000","last_change":"2023-10-06T05:23:07.211294+0000","last_active":"2023-10-06T05:23:07.220937+0000","last_peered":"2023-10-06T05:23:07.220937+0000","last_clean":"2023-10-06T05:23:07.220937+0000","last_became_active":"2023-10-06T05:23:07.211003+0000","last_became_peered":"2023-10-06T05:23:07.211003+0000","last_unstale":"2023-10-06T05:23:07.220937+0000","last_undegraded":"2023-10-06T05:23:07.220937+0000","last_fullsized":"2023-10-06T05:23:07.220937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:12.847514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.192719+0000","last_change":"2023-10-06T05:23:06.195605+0000","last_active":"2023-10-06T05:23:07.192719+0000","last_peered":"2023-10-06T05:23:07.192719+0000","last_clean":"2023-10-06T05:23:07.192719+0000","last_became_active":"2023-10-06T05:23:06.195140+0000","last_became_peered":"2023-10-06T05:23:06.195140+0000","last_unstale":"2023-10-06T05:23:07.192719+0000","last_undegraded":"2023-10-06T05:23:07.192719+0000","last_fullsized":"2023-10-06T05:23:07.192719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.953606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.211392+0000","last_change":"2023-10-06T05:23:07.206761+0000","last_active":"2023-10-06T05:23:07.211392+0000","last_peered":"2023-10-06T05:23:07.211392+0000","last_clean":"2023-10-06T05:23:07.211392+0000","last_became_active":"2023-10-06T05:23:07.206481+0000","last_became_peered":"2023-10-06T05:23:07.206481+0000","last_unstale":"2023-10-06T05:23:07.211392+0000","last_undegraded":"2023-10-06T05:23:07.211392+0000","last_fullsized":"2023-10-06T05:23:07.211392+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:27.252085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.191764+0000","last_change":"2023-10-06T05:23:06.198789+0000","last_active":"2023-10-06T05:23:07.191764+0000","last_peered":"2023-10-06T05:23:07.191764+0000","last_clean":"2023-10-06T05:23:07.191764+0000","last_became_active":"2023-10-06T05:23:06.198517+0000","last_became_peered":"2023-10-06T05:23:06.198517+0000","last_unstale":"2023-10-06T05:23:07.191764+0000","last_undegraded":"2023-10-06T05:23:07.191764+0000","last_fullsized":"2023-10-06T05:23:07.191764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:40.583487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218000+0000","last_change":"2023-10-06T05:23:07.205550+0000","last_active":"2023-10-06T05:23:07.218000+0000","last_peered":"2023-10-06T05:23:07.218000+0000","last_clean":"2023-10-06T05:23:07.218000+0000","last_became_active":"2023-10-06T05:23:07.205250+0000","last_became_peered":"2023-10-06T05:23:07.205250+0000","last_unstale":"2023-10-06T05:23:07.218000+0000","last_undegraded":"2023-10-06T05:23:07.218000+0000","last_fullsized":"2023-10-06T05:23:07.218000+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:55.149048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193601+0000","last_change":"2023-10-06T05:23:06.201667+0000","last_active":"2023-10-06T05:23:07.193601+0000","last_peered":"2023-10-06T05:23:07.193601+0000","last_clean":"2023-10-06T05:23:07.193601+0000","last_became_active":"2023-10-06T05:23:06.201206+0000","last_became_peered":"2023-10-06T05:23:06.201206+0000","last_unstale":"2023-10-06T05:23:07.193601+0000","last_undegraded":"2023-10-06T05:23:07.193601+0000","last_fullsized":"2023-10-06T05:23:07.193601+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:50.043245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.200427+0000","last_change":"2023-10-06T05:23:07.197380+0000","last_active":"2023-10-06T05:23:07.200427+0000","last_peered":"2023-10-06T05:23:07.200427+0000","last_clean":"2023-10-06T05:23:07.200427+0000","last_became_active":"2023-10-06T05:23:07.197118+0000","last_became_peered":"2023-10-06T05:23:07.197118+0000","last_unstale":"2023-10-06T05:23:07.200427+0000","last_undegraded":"2023-10-06T05:23:07.200427+0000","last_fullsized":"2023-10-06T05:23:07.200427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:35.023054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.185652+0000","last_change":"2023-10-06T05:23:06.197865+0000","last_active":"2023-10-06T05:23:07.185652+0000","last_peered":"2023-10-06T05:23:07.185652+0000","last_clean":"2023-10-06T05:23:07.185652+0000","last_became_active":"2023-10-06T05:23:06.196939+0000","last_became_peered":"2023-10-06T05:23:06.196939+0000","last_unstale":"2023-10-06T05:23:07.185652+0000","last_undegraded":"2023-10-06T05:23:07.185652+0000","last_fullsized":"2023-10-06T05:23:07.185652+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:23.663009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193887+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:23:07.193887+0000","last_peered":"2023-10-06T05:23:07.193887+0000","last_clean":"2023-10-06T05:23:07.193887+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:23:07.193887+0000","last_undegraded":"2023-10-06T05:23:07.193887+0000","last_fullsized":"2023-10-06T05:23:07.193887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.209964+0000","last_change":"2023-10-06T05:23:07.205864+0000","last_active":"2023-10-06T05:23:07.209964+0000","last_peered":"2023-10-06T05:23:07.209964+0000","last_clean":"2023-10-06T05:23:07.209964+0000","last_became_active":"2023-10-06T05:23:07.203827+0000","last_became_peered":"2023-10-06T05:23:07.203827+0000","last_unstale":"2023-10-06T05:23:07.209964+0000","last_undegraded":"2023-10-06T05:23:07.209964+0000","last_fullsized":"2023-10-06T05:23:07.209964+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:28.197399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.202650+0000","last_change":"2023-10-06T05:23:06.205054+0000","last_active":"2023-10-06T05:23:07.202650+0000","last_peered":"2023-10-06T05:23:07.202650+0000","last_clean":"2023-10-06T05:23:07.202650+0000","last_became_active":"2023-10-06T05:23:06.204764+0000","last_became_peered":"2023-10-06T05:23:06.204764+0000","last_unstale":"2023-10-06T05:23:07.202650+0000","last_undegraded":"2023-10-06T05:23:07.202650+0000","last_fullsized":"2023-10-06T05:23:07.202650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:57.888763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.188902+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:23:07.188902+0000","last_peered":"2023-10-06T05:23:07.188902+0000","last_clean":"2023-10-06T05:23:07.188902+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:23:07.188902+0000","last_undegraded":"2023-10-06T05:23:07.188902+0000","last_fullsized":"2023-10-06T05:23:07.188902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.214525+0000","last_change":"2023-10-06T05:23:07.204932+0000","last_active":"2023-10-06T05:23:07.214525+0000","last_peered":"2023-10-06T05:23:07.214525+0000","last_clean":"2023-10-06T05:23:07.214525+0000","last_became_active":"2023-10-06T05:23:07.204639+0000","last_became_peered":"2023-10-06T05:23:07.204639+0000","last_unstale":"2023-10-06T05:23:07.214525+0000","last_undegraded":"2023-10-06T05:23:07.214525+0000","last_fullsized":"2023-10-06T05:23:07.214525+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:20.422846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.208037+0000","last_change":"2023-10-06T05:23:06.204342+0000","last_active":"2023-10-06T05:23:07.208037+0000","last_peered":"2023-10-06T05:23:07.208037+0000","last_clean":"2023-10-06T05:23:07.208037+0000","last_became_active":"2023-10-06T05:23:06.204076+0000","last_became_peered":"2023-10-06T05:23:06.204076+0000","last_unstale":"2023-10-06T05:23:07.208037+0000","last_undegraded":"2023-10-06T05:23:07.208037+0000","last_fullsized":"2023-10-06T05:23:07.208037+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:39.385198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.183474+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:23:07.183474+0000","last_peered":"2023-10-06T05:23:07.183474+0000","last_clean":"2023-10-06T05:23:07.183474+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:23:07.183474+0000","last_undegraded":"2023-10-06T05:23:07.183474+0000","last_fullsized":"2023-10-06T05:23:07.183474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.216678+0000","last_change":"2023-10-06T05:23:07.210747+0000","last_active":"2023-10-06T05:23:07.216678+0000","last_peered":"2023-10-06T05:23:07.216678+0000","last_clean":"2023-10-06T05:23:07.216678+0000","last_became_active":"2023-10-06T05:23:07.209550+0000","last_became_peered":"2023-10-06T05:23:07.209550+0000","last_unstale":"2023-10-06T05:23:07.216678+0000","last_undegraded":"2023-10-06T05:23:07.216678+0000","last_fullsized":"2023-10-06T05:23:07.216678+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:41.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"116'8","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218208+0000","last_change":"2023-10-06T05:23:06.205826+0000","last_active":"2023-10-06T05:23:07.218208+0000","last_peered":"2023-10-06T05:23:07.218208+0000","last_clean":"2023-10-06T05:23:07.218208+0000","last_became_active":"2023-10-06T05:23:06.205485+0000","last_became_peered":"2023-10-06T05:23:06.205485+0000","last_unstale":"2023-10-06T05:23:07.218208+0000","last_undegraded":"2023-10-06T05:23:07.218208+0000","last_fullsized":"2023-10-06T05:23:07.218208+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:42.747451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":90,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.188874+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:23:07.188874+0000","last_peered":"2023-10-06T05:23:07.188874+0000","last_clean":"2023-10-06T05:23:07.188874+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:23:07.188874+0000","last_undegraded":"2023-10-06T05:23:07.188874+0000","last_fullsized":"2023-10-06T05:23:07.188874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193876+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:23:07.193876+0000","last_peered":"2023-10-06T05:23:07.193876+0000","last_clean":"2023-10-06T05:23:07.193876+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:23:07.193876+0000","last_undegraded":"2023-10-06T05:23:07.193876+0000","last_fullsized":"2023-10-06T05:23:07.193876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.221626+0000","last_change":"2023-10-06T05:23:07.215719+0000","last_active":"2023-10-06T05:23:07.221626+0000","last_peered":"2023-10-06T05:23:07.221626+0000","last_clean":"2023-10-06T05:23:07.221626+0000","last_became_active":"2023-10-06T05:23:07.215173+0000","last_became_peered":"2023-10-06T05:23:07.215173+0000","last_unstale":"2023-10-06T05:23:07.221626+0000","last_undegraded":"2023-10-06T05:23:07.221626+0000","last_fullsized":"2023-10-06T05:23:07.221626+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:12.273065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.218435+0000","last_change":"2023-10-06T05:23:06.206088+0000","last_active":"2023-10-06T05:23:07.218435+0000","last_peered":"2023-10-06T05:23:07.218435+0000","last_clean":"2023-10-06T05:23:07.218435+0000","last_became_active":"2023-10-06T05:23:06.205802+0000","last_became_peered":"2023-10-06T05:23:06.205802+0000","last_unstale":"2023-10-06T05:23:07.218435+0000","last_undegraded":"2023-10-06T05:23:07.218435+0000","last_fullsized":"2023-10-06T05:23:07.218435+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:19.410153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.193688+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:23:07.193688+0000","last_peered":"2023-10-06T05:23:07.193688+0000","last_clean":"2023-10-06T05:23:07.193688+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:23:07.193688+0000","last_undegraded":"2023-10-06T05:23:07.193688+0000","last_fullsized":"2023-10-06T05:23:07.193688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.213613+0000","last_change":"2023-10-06T05:23:07.205270+0000","last_active":"2023-10-06T05:23:07.213613+0000","last_peered":"2023-10-06T05:23:07.213613+0000","last_clean":"2023-10-06T05:23:07.213613+0000","last_became_active":"2023-10-06T05:23:07.204939+0000","last_became_peered":"2023-10-06T05:23:07.204939+0000","last_unstale":"2023-10-06T05:23:07.213613+0000","last_undegraded":"2023-10-06T05:23:07.213613+0000","last_fullsized":"2023-10-06T05:23:07.213613+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:03.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.189000+0000","last_change":"2023-10-06T05:23:06.199786+0000","last_active":"2023-10-06T05:23:07.189000+0000","last_peered":"2023-10-06T05:23:07.189000+0000","last_clean":"2023-10-06T05:23:07.189000+0000","last_became_active":"2023-10-06T05:23:06.199507+0000","last_became_peered":"2023-10-06T05:23:06.199507+0000","last_unstale":"2023-10-06T05:23:07.189000+0000","last_undegraded":"2023-10-06T05:23:07.189000+0000","last_fullsized":"2023-10-06T05:23:07.189000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:26.362255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.195303+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:23:07.195303+0000","last_peered":"2023-10-06T05:23:07.195303+0000","last_clean":"2023-10-06T05:23:07.195303+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:23:07.195303+0000","last_undegraded":"2023-10-06T05:23:07.195303+0000","last_fullsized":"2023-10-06T05:23:07.195303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.224769+0000","last_change":"2023-10-06T05:23:07.223597+0000","last_active":"2023-10-06T05:23:07.224769+0000","last_peered":"2023-10-06T05:23:07.224769+0000","last_clean":"2023-10-06T05:23:07.224769+0000","last_became_active":"2023-10-06T05:23:07.223305+0000","last_became_peered":"2023-10-06T05:23:07.223305+0000","last_unstale":"2023-10-06T05:23:07.224769+0000","last_undegraded":"2023-10-06T05:23:07.224769+0000","last_fullsized":"2023-10-06T05:23:07.224769+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:27.252085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"116'3","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.222704+0000","last_change":"2023-10-06T05:23:06.197088+0000","last_active":"2023-10-06T05:23:07.222704+0000","last_peered":"2023-10-06T05:23:07.222704+0000","last_clean":"2023-10-06T05:23:07.222704+0000","last_became_active":"2023-10-06T05:23:06.196775+0000","last_became_peered":"2023-10-06T05:23:06.196775+0000","last_unstale":"2023-10-06T05:23:07.222704+0000","last_undegraded":"2023-10-06T05:23:07.222704+0000","last_fullsized":"2023-10-06T05:23:07.222704+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:28.416245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.191645+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:23:07.191645+0000","last_peered":"2023-10-06T05:23:07.191645+0000","last_clean":"2023-10-06T05:23:07.191645+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:23:07.191645+0000","last_undegraded":"2023-10-06T05:23:07.191645+0000","last_fullsized":"2023-10-06T05:23:07.191645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.212853+0000","last_change":"2023-10-06T05:23:07.207631+0000","last_active":"2023-10-06T05:23:07.212853+0000","last_peered":"2023-10-06T05:23:07.212853+0000","last_clean":"2023-10-06T05:23:07.212853+0000","last_became_active":"2023-10-06T05:23:07.207359+0000","last_became_peered":"2023-10-06T05:23:07.207359+0000","last_unstale":"2023-10-06T05:23:07.212853+0000","last_undegraded":"2023-10-06T05:23:07.212853+0000","last_fullsized":"2023-10-06T05:23:07.212853+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:06.172138+0000","last_clean_scrub_stamp":"2023-10-06T05:23:06.172138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:06.654520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"116'6","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.228798+0000","last_change":"2023-10-06T05:23:06.203784+0000","last_active":"2023-10-06T05:23:07.228798+0000","last_peered":"2023-10-06T05:23:07.228798+0000","last_clean":"2023-10-06T05:23:07.228798+0000","last_became_active":"2023-10-06T05:23:06.203511+0000","last_became_peered":"2023-10-06T05:23:06.203511+0000","last_unstale":"2023-10-06T05:23:07.228798+0000","last_undegraded":"2023-10-06T05:23:07.228798+0000","last_fullsized":"2023-10-06T05:23:07.228798+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:05.159445+0000","last_clean_scrub_stamp":"2023-10-06T05:23:05.159445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:12.672998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":104,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-06T05:23:07.185882+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:23:07.185882+0000","last_peered":"2023-10-06T05:23:07.185882+0000","last_clean":"2023-10-06T05:23:07.185882+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:23:07.185882+0000","last_undegraded":"2023-10-06T05:23:07.185882+0000","last_fullsized":"2023-10-06T05:23:07.185882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303400,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068440,"statfs":{"total":96636764160,"available":96326082560,"internally_reserved":0,"allocated":2064384,"data_stored":1257903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.65600000000000003},"min":{"1min":0.432,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.874},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.746,"15min":0.69699999999999995},"min":{"1min":0.58299999999999996,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.98499999999999999,"5min":1.169,"15min":1.169},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93600000000000005,"5min":1.268,"15min":1.268},"last":0.627},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.075},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":1.093,"15min":1.093},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83499999999999996,"5min":1.22,"15min":1.22},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.85399999999999998,"5min":1.0349999999999999,"15min":1.079},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0169999999999999,"5min":1.117,"15min":1.202},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.84599999999999997,"5min":1.258,"15min":1.258},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86899999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.0369999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.438,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90800000000000003,"5min":1.054,"15min":1.054},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"min":{"1min":0.58699999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88600000000000001,"5min":1.139,"15min":1.139},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71799999999999997,"15min":0.74399999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.3300000000000001},"last":0.93799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":667341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1,"15min":1},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":1.05,"15min":1.1599999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71499999999999997,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.93200000000000005,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.89100000000000001,"15min":0.95699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":0.97799999999999998,"15min":0.998},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.748}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.76300000000000001},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73299999999999998,"15min":0.82199999999999995},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.996,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":666809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.45700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.374},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.49399999999999999,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.69999999999999996,"5min":0.74299999999999999,"15min":2.8050000000000002},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46300000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.5},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56699999999999995,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":2.7639999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":2.8370000000000002},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.79100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75600000000000001,"15min":0.81399999999999995},"min":{"1min":0.503,"5min":0.49099999999999999,"15min":0.45200000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.349},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.669},"last":0.86899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.52600000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.6040000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.573},"last":1.141}]},{"osd":7,"last update":"Fri Oct 6 05:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.84799999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.52200000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.528},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.85899999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.97299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303388,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068452,"statfs":{"total":96636764160,"available":96326094848,"internally_reserved":0,"allocated":2052096,"data_stored":1257177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.58099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.499,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68700000000000006,"5min":0.879,"15min":0.879},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.58299999999999996,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.624},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.218,"15min":1.218},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77700000000000002,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 05:22:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.075,"5min":1.25,"15min":1.25},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.79700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.018,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302804,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069036,"statfs":{"total":96636764160,"available":96326692864,"internally_reserved":0,"allocated":1454080,"data_stored":666787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.44600000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.73599999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.50700000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":0.70499999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47599999999999998,"15min":0.46899999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.57799999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.628}]},{"osd":4,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79800000000000004,"15min":0.76600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.89600000000000002,"5min":1.081,"15min":1.1379999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.53000000000000003},"max":{"1min":0.995,"5min":1.161,"15min":1.319},"last":0.81499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1399999999999999,"15min":1.5289999999999999},"last":0.753},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84699999999999998,"5min":1.103,"15min":1.141},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.212,"15min":1.282},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":0.94199999999999995,"5min":1.143,"15min":1.3520000000000001},"last":1.006}]}]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302820,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069020,"statfs":{"total":96636764160,"available":96326676480,"internally_reserved":0,"allocated":1470464,"data_stored":668357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46999999999999997,"15min":0.50900000000000001},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.63100000000000001,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.505,"15min":0.5},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":1.772},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":1.913},"last":0.378},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53900000000000003,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.34999999999999998,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.82399999999999995,"15min":1.954},"last":0.44},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":2.0539999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":0.96999999999999997,"5min":1.2030000000000001,"15min":1.282},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91500000000000004,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":0.98499999999999999,"5min":1.1539999999999999,"15min":1.1930000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.84199999999999997},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.83899999999999997},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0449999999999999,"5min":1.1910000000000001,"15min":1.343},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61299999999999999,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2390000000000001,"15min":1.3380000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82699999999999996,"15min":0.84899999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.034,"5min":1.1759999999999999,"15min":1.3640000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":0.998,"5min":1.2090000000000001,"15min":1.214},"last":0.81000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302816,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069024,"statfs":{"total":96636764160,"available":96326680576,"internally_reserved":0,"allocated":1466368,"data_stored":668323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.70799999999999996},"min":{"1min":0.46800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.073},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0269999999999999,"15min":1.169},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70299999999999996,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.83499999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.433,"15min":0.432},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.117},"last":0.503},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":1.002},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.90200000000000002,"5min":1.0069999999999999,"15min":1.0780000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.49199999999999999,"15min":0.44600000000000001},"max":{"1min":0.94499999999999995,"5min":0.996,"15min":1.196},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.442,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8740000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.751},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.375},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.746},"min":{"1min":0.53000000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":1.004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.747,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.80600000000000005},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.466},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79100000000000004,"15min":0.82999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97499999999999998,"5min":1.1930000000000001,"15min":1.218},"last":0.70899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302812,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069028,"statfs":{"total":96636764160,"available":96326684672,"internally_reserved":0,"allocated":1462272,"data_stored":666947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":1.165,"15min":1.165},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.53600000000000003,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":1.016,"15min":1.1890000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72999999999999998,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92800000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":1.048,"15min":1.048},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":0.89600000000000002,"5min":1.204,"15min":1.204},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":0.997,"15min":1.0549999999999999},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73899999999999999,"15min":0.71099999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.087},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74199999999999999,"15min":0.72899999999999998},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.433},"max":{"1min":0.91800000000000004,"5min":1.2769999999999999,"15min":1.278},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.1379999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66500000000000004,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.85899999999999999,"5min":1.1539999999999999,"15min":1.294},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":1.141,"15min":1.141},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.254},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90600000000000003,"5min":1.0049999999999999,"15min":1.337},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":1.248,"15min":1.248},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:23:13.043 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:23:13.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:23:13.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.268+0000 7f90acd9d700 1 Processor -- start 2023-10-06T05:23:13.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.269+0000 7f90acd9d700 1 -- start start 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90acd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a815a7d0 0x7f90a80af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90acd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90a80cced0 0x7f90a80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90acd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a815ddb0 con 0x7f90a80af540 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90acd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a80cd890 con 0x7f90a80cced0 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90acd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a815d1a0 con 0x7f90a815a7d0 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90a7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90a80cced0 0x7f90a80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90a7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90a80cced0 0x7f90a80cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51700/0 (socket says 172.21.15.33:51700) 2023-10-06T05:23:13.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.270+0000 7f90a7fff700 1 -- 172.21.15.33:0/1918710784 learned_addr learned my addr 172.21.15.33:0/1918710784 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:13.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a6ffd700 1 --2- 172.21.15.33:0/1918710784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a815a7d0 0x7f90a80af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a7fff700 1 -- 172.21.15.33:0/1918710784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a815a7d0 msgr2=0x7f90a80af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a7fff700 1 --2- 172.21.15.33:0/1918710784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a815a7d0 0x7f90a80af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a7fff700 1 -- 172.21.15.33:0/1918710784 >> v1:172.21.15.167:6789/0 conn(0x7f90a80af540 legacy=0x7f90a80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a7fff700 1 -- 172.21.15.33:0/1918710784 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a815d480 con 0x7f90a80cced0 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a7fff700 1 --2- 172.21.15.33:0/1918710784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a80cced0 0x7f90a80cd350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f20 tx=0x7f90a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2e85e651f809c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a67fc700 1 -- 172.21.15.33:0/1918710784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000e070 con 0x7f90a80cced0 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a67fc700 1 -- 172.21.15.33:0/1918710784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000ba80 con 0x7f90a80cced0 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.271+0000 7f90a67fc700 1 -- 172.21.15.33:0/1918710784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00092e0 con 0x7f90a80cced0 2023-10-06T05:23:13.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 -- 172.21.15.33:0/1918710784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a80cced0 msgr2=0x7f90a80cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1918710784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a80cced0 0x7f90a80cd350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f20 tx=0x7f90a0009840 comp rx=0 tx=0).stop 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 -- 172.21.15.33:0/1918710784 shutdown_connections 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1918710784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a80cced0 0x7f90a80cd350 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1918710784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a815a7d0 0x7f90a80af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 -- 172.21.15.33:0/1918710784 >> 172.21.15.33:0/1918710784 conn(0x7f90a806d9b0 msgr2=0x7f90a806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 -- 172.21.15.33:0/1918710784 shutdown_connections 2023-10-06T05:23:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.272+0000 7f90acd9d700 1 -- 172.21.15.33:0/1918710784 wait complete. 2023-10-06T05:23:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.274+0000 7f90acd9d700 1 Processor -- start 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.274+0000 7f90acd9d700 1 -- start start 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90acd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90a80af540 0x7f90a80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90acd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a80cced0 0x7f90a80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90acd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 0x7f90a8169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90acd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c7580 con 0x7f90a815a7d0 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90acd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c83e0 con 0x7f90a80af540 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90acd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c77d0 con 0x7f90a80cced0 2023-10-06T05:23:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 0x7f90a8169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 0x7f90a8169520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51712/0 (socket says 172.21.15.33:51712) 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 -- 172.21.15.33:0/1261327932 learned_addr learned my addr 172.21.15.33:0/1261327932 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a6ffd700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a80cced0 0x7f90a80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a77fe700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90a80af540 0x7f90a80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 -- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a80cced0 msgr2=0x7f90a80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a80cced0 0x7f90a80c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 -- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90a80af540 msgr2=0x7f90a80c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90a80af540 0x7f90a80c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a7fff700 1 -- 172.21.15.33:0/1261327932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009ae0 con 0x7f90a815a7d0 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.275+0000 7f90a77fe700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90a80af540 0x7f90a80c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.276+0000 7f90a7fff700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 0x7f90a8169520 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009430 tx=0x7f90a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.276+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00032f0 con 0x7f90a815a7d0 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.276+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0008dc0 con 0x7f90a815a7d0 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.276+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a80c7ab0 con 0x7f90a815a7d0 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.276+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0026670 con 0x7f90a815a7d0 2023-10-06T05:23:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.276+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a80c8070 con 0x7f90a815a7d0 2023-10-06T05:23:13.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.277+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a80c8570 con 0x7f90a815a7d0 2023-10-06T05:23:13.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.278+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90a0003040 con 0x7f90a815a7d0 2023-10-06T05:23:13.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.279+0000 7f908f7fe700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9094066f30 0x7f90940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.279+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f90a00867b0 con 0x7f90a815a7d0 2023-10-06T05:23:13.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.279+0000 7f90a77fe700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9094066f30 0x7f90940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.280+0000 7f90a77fe700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9094066f30 0x7f90940693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9098007fd0 tx=0x7f9098009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.283+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90a0053820 con 0x7f90a815a7d0 2023-10-06T05:23:13.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.566+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f90a815d760 con 0x7f90a815a7d0 2023-10-06T05:23:13.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.570+0000 7f908f7fe700 1 -- 172.21.15.33:0/1261327932 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f90a0008f30 con 0x7f90a815a7d0 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9094066f30 msgr2=0x7f90940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9094066f30 0x7f90940693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9098007fd0 tx=0x7f9098009040 comp rx=0 tx=0).stop 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 msgr2=0x7f90a8169520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 0x7f90a8169520 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009430 tx=0x7f90a0018000 comp rx=0 tx=0).stop 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 shutdown_connections 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9094066f30 0x7f90940693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90a815a7d0 0x7f90a8169520 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90a80af540 0x7f90a80c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 --2- 172.21.15.33:0/1261327932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90a80cced0 0x7f90a80c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 >> 172.21.15.33:0/1261327932 conn(0x7f90a806d9b0 msgr2=0x7f90a8158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 shutdown_connections 2023-10-06T05:23:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.573+0000 7f90acd9d700 1 -- 172.21.15.33:0/1261327932 wait complete. 2023-10-06T05:23:13.577 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:23:13.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:13.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.778+0000 7fb1c899d700 1 Processor -- start 2023-10-06T05:23:13.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.779+0000 7fb1c899d700 1 -- start start 2023-10-06T05:23:13.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.780+0000 7fb1c899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1c413faa0 0x7fb1c413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.780+0000 7fb1c899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 0x7fb1c40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.780+0000 7fb1c899d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40c9cb0 con 0x7fb1c406aad0 2023-10-06T05:23:13.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.780+0000 7fb1c899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c406b660 con 0x7fb1c413faa0 2023-10-06T05:23:13.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.780+0000 7fb1c899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40ca270 con 0x7fb1c40c7940 2023-10-06T05:23:13.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.780+0000 7fb1c2ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 0x7fb1c40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c37fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1c413faa0 0x7fb1c413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c3fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb1c406aad0 0x7fb1c406af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55516/0 (socket says 172.21.15.33:55516) 2023-10-06T05:23:13.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c3fff700 1 -- 172.21.15.33:0/518441998 learned_addr learned my addr 172.21.15.33:0/518441998 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:13.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c2ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 0x7fb1c40c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:34362/0 (socket says 172.21.15.33:34362) 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/518441998 >> v2:172.21.15.167:3300/0 conn(0x7fb1c413faa0 msgr2=0x7fb1c413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c2ffd700 1 --2- 172.21.15.33:0/518441998 >> v2:172.21.15.167:3300/0 conn(0x7fb1c413faa0 0x7fb1c413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/518441998 >> v1:172.21.15.167:6789/0 conn(0x7fb1c406aad0 legacy=0x7fb1c406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c27fc700 1 -- 172.21.15.33:0/518441998 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400217922 0 0) 0x7fb1c40c9cb0 con 0x7fb1c406aad0 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/518441998 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c406b840 con 0x7fb1c40c7940 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.781+0000 7fb1c2ffd700 1 --2- 172.21.15.33:0/518441998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 0x7fb1c40c7db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098e0 tx=0x7fb1b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5032c4af297f58b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c27fc700 1 -- 172.21.15.33:0/518441998 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800b640 con 0x7fb1c40c7940 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c27fc700 1 -- 172.21.15.33:0/518441998 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b800b7a0 con 0x7fb1c40c7940 2023-10-06T05:23:13.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c27fc700 1 -- 172.21.15.33:0/518441998 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800bb40 con 0x7fb1c40c7940 2023-10-06T05:23:13.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 -- 172.21.15.33:0/518441998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 msgr2=0x7fb1c40c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 --2- 172.21.15.33:0/518441998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 0x7fb1c40c7db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098e0 tx=0x7fb1b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 -- 172.21.15.33:0/518441998 shutdown_connections 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 --2- 172.21.15.33:0/518441998 >> v2:172.21.15.167:3300/0 conn(0x7fb1c413faa0 0x7fb1c413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 --2- 172.21.15.33:0/518441998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c40c7940 0x7fb1c40c7db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 -- 172.21.15.33:0/518441998 >> 172.21.15.33:0/518441998 conn(0x7fb1c406d030 msgr2=0x7fb1c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 -- 172.21.15.33:0/518441998 shutdown_connections 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.782+0000 7fb1c899d700 1 -- 172.21.15.33:0/518441998 wait complete. 2023-10-06T05:23:13.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.784+0000 7fb1c899d700 1 Processor -- start 2023-10-06T05:23:13.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.784+0000 7fb1c899d700 1 -- start start 2023-10-06T05:23:13.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c406aad0 0x7fb1c413cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 0x7fb1c4158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1c413faa0 0x7fb1c40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c416a660 con 0x7fb1c40c7940 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4157c70 con 0x7fb1c413faa0 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4157f10 con 0x7fb1c406aad0 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 0x7fb1c4158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 0x7fb1c4158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51722/0 (socket says 172.21.15.33:51722) 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.785+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/762217587 learned_addr learned my addr 172.21.15.33:0/762217587 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c37fe700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c406aad0 0x7fb1c413cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c3fff700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1c413faa0 0x7fb1c40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c406aad0 msgr2=0x7fb1c413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c2ffd700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c406aad0 0x7fb1c413cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1c413faa0 msgr2=0x7fb1c40c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c2ffd700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1c413faa0 0x7fb1c40c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c2ffd700 1 -- 172.21.15.33:0/762217587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8009420 con 0x7fb1c40c7940 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c2ffd700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 0x7fb1c4158cd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009f80 tx=0x7fb1b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8000d10 con 0x7fb1c40c7940 2023-10-06T05:23:13.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b800bb20 con 0x7fb1c40c7940 2023-10-06T05:23:13.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c41581f0 con 0x7fb1c40c7940 2023-10-06T05:23:13.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.786+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8011670 con 0x7fb1c40c7940 2023-10-06T05:23:13.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.787+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c41587b0 con 0x7fb1c40c7940 2023-10-06T05:23:13.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.787+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c41589a0 con 0x7fb1c40c7940 2023-10-06T05:23:13.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.788+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1b8011aa0 con 0x7fb1c40c7940 2023-10-06T05:23:13.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.789+0000 7fb1ab7fe700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1b0066f30 0x7fb1b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.789+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb1b80861d0 con 0x7fb1c40c7940 2023-10-06T05:23:13.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.790+0000 7fb1c37fe700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1b0066f30 0x7fb1b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.790+0000 7fb1c37fe700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1b0066f30 0x7fb1b00693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb1c406efd0 tx=0x7fb1b40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:13.794+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1b8053240 con 0x7fb1c40c7940 2023-10-06T05:23:14.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.045+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c404c380 con 0x7fb1c40c7940 2023-10-06T05:23:14.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.045+0000 7fb1ab7fe700 1 -- 172.21.15.33:0/762217587 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fb1b8056860 con 0x7fb1c40c7940 2023-10-06T05:23:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:14.048 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:14.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1b0066f30 msgr2=0x7fb1b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1b0066f30 0x7fb1b00693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb1c406efd0 tx=0x7fb1b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 msgr2=0x7fb1c4158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 0x7fb1c4158cd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009f80 tx=0x7fb1b8017000 comp rx=0 tx=0).stop 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 shutdown_connections 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1b0066f30 0x7fb1b00693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1c413faa0 0x7fb1c40c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1c406aad0 0x7fb1c413cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 --2- 172.21.15.33:0/762217587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c40c7940 0x7fb1c4158cd0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 >> 172.21.15.33:0/762217587 conn(0x7fb1c406d030 msgr2=0x7fb1c413ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 shutdown_connections 2023-10-06T05:23:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.048+0000 7fb1c899d700 1 -- 172.21.15.33:0/762217587 wait complete. 2023-10-06T05:23:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:14.470 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:23:06.683552+0000', 'modified': '2023-10-06T05:23:11.203927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7644': {'gid': 7644, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6834/723044599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 723044599}, {'type': 'v1', 'addr': '172.21.15.33:6834', 'nonce': 723044599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:23:14.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:23:14.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:23:14.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.659+0000 7f90a139d700 1 Processor -- start 2023-10-06T05:23:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.659+0000 7f90a139d700 1 -- start start 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a139d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f909c1557f0 0x7f909c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c155bc0 0x7f909c0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c06a410 con 0x7f909c157150 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a139d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0cccd0 con 0x7f909c1557f0 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c7500 con 0x7f909c155bc0 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a0b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f909c157150 0x7f909c0c8b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40378/0 (socket says 172.21.15.33:40378) 2023-10-06T05:23:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.660+0000 7f90a0b9c700 1 -- 172.21.15.33:0/1704352 learned_addr learned my addr 172.21.15.33:0/1704352 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:14.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909b7fe700 1 --2- 172.21.15.33:0/1704352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c155bc0 0x7f909c0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909bfff700 1 --2- 172.21.15.33:0/1704352 >> v2:172.21.15.167:3300/0 conn(0x7f909c1557f0 0x7f909c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909affd700 1 -- 172.21.15.33:0/1704352 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573614942 0 0) 0x7f909c06a410 con 0x7f909c157150 2023-10-06T05:23:14.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909affd700 1 -- 172.21.15.33:0/1704352 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9084003570 con 0x7f909c157150 2023-10-06T05:23:14.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909b7fe700 1 -- 172.21.15.33:0/1704352 >> v2:172.21.15.167:3300/0 conn(0x7f909c1557f0 msgr2=0x7f909c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909b7fe700 1 --2- 172.21.15.33:0/1704352 >> v2:172.21.15.167:3300/0 conn(0x7f909c1557f0 0x7f909c0c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909b7fe700 1 -- 172.21.15.33:0/1704352 >> v1:172.21.15.33:6789/0 conn(0x7f909c157150 legacy=0x7f909c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909b7fe700 1 -- 172.21.15.33:0/1704352 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0c77e0 con 0x7f909c155bc0 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909bfff700 1 --2- 172.21.15.33:0/1704352 >> v2:172.21.15.167:3300/0 conn(0x7f909c1557f0 0x7f909c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.661+0000 7f909b7fe700 1 --2- 172.21.15.33:0/1704352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c155bc0 0x7f909c0c6ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90900098e0 tx=0x7f90900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1708dc9431989e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f909affd700 1 -- 172.21.15.33:0/1704352 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011630 con 0x7f909c155bc0 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f909affd700 1 -- 172.21.15.33:0/1704352 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090011790 con 0x7f909c155bc0 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f909affd700 1 -- 172.21.15.33:0/1704352 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011b30 con 0x7f909c155bc0 2023-10-06T05:23:14.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 -- 172.21.15.33:0/1704352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c155bc0 msgr2=0x7f909c0c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 --2- 172.21.15.33:0/1704352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c155bc0 0x7f909c0c6ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90900098e0 tx=0x7f90900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 -- 172.21.15.33:0/1704352 shutdown_connections 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 --2- 172.21.15.33:0/1704352 >> v2:172.21.15.167:3300/0 conn(0x7f909c1557f0 0x7f909c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 --2- 172.21.15.33:0/1704352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c155bc0 0x7f909c0c6ea0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 -- 172.21.15.33:0/1704352 >> 172.21.15.33:0/1704352 conn(0x7f909c06c560 msgr2=0x7f909c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 -- 172.21.15.33:0/1704352 shutdown_connections 2023-10-06T05:23:14.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.662+0000 7f90a139d700 1 -- 172.21.15.33:0/1704352 wait complete. 2023-10-06T05:23:14.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.664+0000 7f90a139d700 1 Processor -- start 2023-10-06T05:23:14.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.664+0000 7f90a139d700 1 -- start start 2023-10-06T05:23:14.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f909c1557f0 0x7f909c149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 0x7f909c169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c157150 0x7f909c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c16a520 con 0x7f909c155bc0 2023-10-06T05:23:14.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c152a60 con 0x7f909c1557f0 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c151e50 con 0x7f909c157150 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f909b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 0x7f909c169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f909b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 0x7f909c169fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51738/0 (socket says 172.21.15.33:51738) 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f909b7fe700 1 -- 172.21.15.33:0/1377923980 learned_addr learned my addr 172.21.15.33:0/1377923980 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c157150 0x7f909c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.665+0000 7f909bfff700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f909c1557f0 0x7f909c149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909b7fe700 1 -- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c157150 msgr2=0x7f909c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909b7fe700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c157150 0x7f909c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909b7fe700 1 -- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f909c1557f0 msgr2=0x7f909c149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909b7fe700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f909c1557f0 0x7f909c149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909b7fe700 1 -- 172.21.15.33:0/1377923980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c00a5d0 con 0x7f909c155bc0 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909bfff700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f909c1557f0 0x7f909c149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c157150 0x7f909c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f909b7fe700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 0x7f909c169fe0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f909c0c7640 tx=0x7f9090017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f9083fff700 1 -- 172.21.15.33:0/1377923980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011b10 con 0x7f909c155bc0 2023-10-06T05:23:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f9083fff700 1 -- 172.21.15.33:0/1377923980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90900253f0 con 0x7f909c155bc0 2023-10-06T05:23:14.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9090009420 con 0x7f909c155bc0 2023-10-06T05:23:14.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f9083fff700 1 -- 172.21.15.33:0/1377923980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900257b0 con 0x7f909c155bc0 2023-10-06T05:23:14.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.666+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c152410 con 0x7f909c155bc0 2023-10-06T05:23:14.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.669+0000 7f9083fff700 1 -- 172.21.15.33:0/1377923980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9090025ba0 con 0x7f909c155bc0 2023-10-06T05:23:14.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.669+0000 7f9083fff700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9088066f30 0x7f90880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.670+0000 7f9083fff700 1 -- 172.21.15.33:0/1377923980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9090086060 con 0x7f909c155bc0 2023-10-06T05:23:14.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.670+0000 7f909bfff700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9088066f30 0x7f90880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.671+0000 7f909bfff700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9088066f30 0x7f90880693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f909c0cbae0 tx=0x7f908c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.671+0000 7f90b0e17b80 1 Processor -- start 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.671+0000 7f90b0e17b80 1 -- start start 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.672+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.672+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a76920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.672+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6a49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.672+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e69cdc80 con 0x55b9e6a76e60 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.672+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e69a4a70 con 0x55b9e6a4d950 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.672+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9e69a2270 con 0x55b9e6a97bc0 2023-10-06T05:23:14.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6a49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6a49990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51740/0 (socket says 172.21.15.33:51740) 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 -- 172.21.15.33:0/3323747227 learned_addr learned my addr 172.21.15.33:0/3323747227 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f909b7fe700 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a76920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f909bfff700 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 -- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 msgr2=0x55b9e6a76920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a76920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 -- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 msgr2=0x55b9e6a4dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 -- 172.21.15.33:0/3323747227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e6838270 con 0x55b9e6a76e60 2023-10-06T05:23:14.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6a49990 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f90940021f0 tx=0x7f9094009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0549c718486b37e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90817fa700 1 -- 172.21.15.33:0/3323747227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400ed70 con 0x55b9e6a76e60 2023-10-06T05:23:14.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90817fa700 1 -- 172.21.15.33:0/3323747227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9094014480 con 0x55b9e6a76e60 2023-10-06T05:23:14.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.673+0000 7f90817fa700 1 -- 172.21.15.33:0/3323747227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094014800 con 0x55b9e6a76e60 2023-10-06T05:23:14.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 -- 172.21.15.33:0/3323747227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 msgr2=0x55b9e6a49990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6a49990 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f90940021f0 tx=0x7f9094009040 comp rx=0 tx=0).stop 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 -- 172.21.15.33:0/3323747227 shutdown_connections 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6a49990 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/3323747227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a76920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 -- 172.21.15.33:0/3323747227 >> 172.21.15.33:0/3323747227 conn(0x55b9e6a4f270 msgr2=0x55b9e6a4f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 -- 172.21.15.33:0/3323747227 shutdown_connections 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.674+0000 7f90b0e17b80 1 -- 172.21.15.33:0/3323747227 wait complete. 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.675+0000 7f90b0e17b80 1 -- start start 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.675+0000 7f90b0e17b80 1 Processor -- start 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 0x55b9e6a8c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6a8e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a8e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e699e060 con 0x55b9e6a4d950 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6949890 con 0x55b9e6a76e60 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6964ea0 con 0x55b9e6a97bc0 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f90b0e17b80 10 client.0 _ll_register_callbacks cb 0x55b9e683af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f909bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 0x55b9e6a8c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f909bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 0x55b9e6a8c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51742/0 (socket says 172.21.15.33:51742) 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.676+0000 7f909bfff700 1 -- 172.21.15.33:0/2759343349 learned_addr learned my addr 172.21.15.33:0/2759343349 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909b7fe700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6a8e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a8e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909bfff700 1 -- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 msgr2=0x55b9e6a8e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909bfff700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a8e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909bfff700 1 -- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 msgr2=0x55b9e6a8e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909bfff700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6a8e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909bfff700 1 -- 172.21.15.33:0/2759343349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909400f040 con 0x55b9e6a4d950 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a8e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f909bfff700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 0x55b9e6a8c750 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f908c013de0 tx=0x7f908c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa93ed38214707a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f907a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f907a7fc700 1 -- 172.21.15.33:0/2759343349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c012bf0 con 0x55b9e6a4d950 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f907a7fc700 1 -- 172.21.15.33:0/2759343349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f908c012d50 con 0x55b9e6a4d950 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.677+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9e6a42690 con 0x55b9e6a4d950 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f907a7fc700 1 -- 172.21.15.33:0/2759343349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908c021630 con 0x55b9e6a4d950 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 10 client.7785 fetch_fsmap learned FSMap version 81 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55b9e67dafe0 con 0x55b9e6a4d950 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f907a7fc700 1 -- 172.21.15.33:0/2759343349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9e6a42690 con 0x55b9e6a4d950 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f907a7fc700 1 -- 172.21.15.33:0/2759343349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f908c02a430 con 0x55b9e6a4d950 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 10 client.7785 fetch_fsmap finished waiting for FSMap version 81 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 10 client.7785 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 20 client.7785 populate_metadata read hostname 'smithi033' 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a96810 0x55b9e6a96be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 4 client.7785 mds_command: new command op to 7644 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 --> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9e6a1f5c0 con 0x55b9e6a96810 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.678+0000 7f909b7fe700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a96810 0x55b9e6a96be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.679+0000 7f909b7fe700 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a96810 0x55b9e6a96be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad69e5e9516ced5 server_cookie=4c16d2c1ac734514 in_seq=0 out_seq=0 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.679+0000 7f907a7fc700 10 client.7785 ms_handle_connect on v2:172.21.15.33:6832/723044599 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.679+0000 7f907a7fc700 1 -- 172.21.15.33:0/2759343349 <== mds.0 v2:172.21.15.33:6832/723044599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b9e6a1f5c0 con 0x55b9e6a96810 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.679+0000 7f907a7fc700 10 client.7785 handle_command_reply: tid=0 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.679+0000 7f90b0e17b80 1 client.7785 shutdown 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 msgr2=0x55b9e6a8c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 0x55b9e6a8c750 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f908c013de0 tx=0x7f908c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 shutdown_connections 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a96810 0x55b9e6a96be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6a8e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a4d950 0x55b9e6a8c750 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f907a7fc700 0 client.7785 ms_handle_reset on v2:172.21.15.33:6832/723044599 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/2759343349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6a8e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 >> 172.21.15.33:0/2759343349 conn(0x55b9e6a4f270 msgr2=0x55b9e6a4e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 shutdown_connections 2023-10-06T05:23:14.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 1 -- 172.21.15.33:0/2759343349 wait complete. 2023-10-06T05:23:14.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.680+0000 7f90b0e17b80 20 client.7785 trim_cache size 0 max 16384 2023-10-06T05:23:14.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 Processor -- start 2023-10-06T05:23:14.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 -- start start 2023-10-06T05:23:14.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a4d950 0x55b9e6a445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6aa2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 0x55b9e6a96be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6820e30 con 0x55b9e6a97bc0 2023-10-06T05:23:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6a7d300 con 0x55b9e6a76e60 2023-10-06T05:23:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.698+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6aad1d0 con 0x55b9e6a4d950 2023-10-06T05:23:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 0x55b9e6a96be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 0x55b9e6a96be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51758/0 (socket says 172.21.15.33:51758) 2023-10-06T05:23:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 -- 172.21.15.33:0/1624907191 learned_addr learned my addr 172.21.15.33:0/1624907191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 -- 172.21.15.33:0/1624907191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a4d950 msgr2=0x55b9e6a445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a4d950 0x55b9e6a445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 -- 172.21.15.33:0/1624907191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 msgr2=0x55b9e6aa2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6aa2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 -- 172.21.15.33:0/1624907191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e696c780 con 0x55b9e6a97bc0 2023-10-06T05:23:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f90a0b9c700 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 0x55b9e6a96be0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9094000f80 tx=0x7f9094017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbbcfd633a744d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f9081ffb700 1 -- 172.21.15.33:0/1624907191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094001de0 con 0x55b9e6a97bc0 2023-10-06T05:23:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f9081ffb700 1 -- 172.21.15.33:0/1624907191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400ed70 con 0x55b9e6a97bc0 2023-10-06T05:23:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.699+0000 7f9081ffb700 1 -- 172.21.15.33:0/1624907191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400eee0 con 0x55b9e6a97bc0 2023-10-06T05:23:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 -- 172.21.15.33:0/1624907191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 msgr2=0x55b9e6a96be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 0x55b9e6a96be0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9094000f80 tx=0x7f9094017000 comp rx=0 tx=0).stop 2023-10-06T05:23:14.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 -- 172.21.15.33:0/1624907191 shutdown_connections 2023-10-06T05:23:14.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a97bc0 0x55b9e6a96be0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a4d950 0x55b9e6a445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/1624907191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a76e60 0x55b9e6aa2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 -- 172.21.15.33:0/1624907191 >> 172.21.15.33:0/1624907191 conn(0x55b9e6a4f270 msgr2=0x55b9e6a7d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 -- 172.21.15.33:0/1624907191 shutdown_connections 2023-10-06T05:23:14.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.700+0000 7f90b0e17b80 1 -- 172.21.15.33:0/1624907191 wait complete. 2023-10-06T05:23:14.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.701+0000 7f90b0e17b80 1 -- start start 2023-10-06T05:23:14.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.701+0000 7f90b0e17b80 1 Processor -- start 2023-10-06T05:23:14.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6af85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6af8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6a4ddc0 con 0x55b9e6a76e60 2023-10-06T05:23:14.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6a4e0a0 con 0x55b9e6a4d950 2023-10-06T05:23:14.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9e6a4e380 con 0x55b9e6a97bc0 2023-10-06T05:23:14.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f90b0e17b80 10 client.0 _ll_register_callbacks cb 0x55b9e683af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f909b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6af85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f909b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6af85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51768/0 (socket says 172.21.15.33:51768) 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.702+0000 7f909b7fe700 1 -- 172.21.15.33:0/126593016 learned_addr learned my addr 172.21.15.33:0/126593016 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f909b7fe700 1 -- 172.21.15.33:0/126593016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 msgr2=0x55b9e6af8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f909b7fe700 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6af8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f909b7fe700 1 -- 172.21.15.33:0/126593016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 msgr2=0x55b9e6a4e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f909b7fe700 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f909b7fe700 1 -- 172.21.15.33:0/126593016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9e6838270 con 0x55b9e6a76e60 2023-10-06T05:23:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f909b7fe700 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6af85e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9090056a90 tx=0x7f9090059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c1db4ddb43b030 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f907a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:23:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f907a7fc700 1 -- 172.21.15.33:0/126593016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090059fe0 con 0x55b9e6a76e60 2023-10-06T05:23:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9094014480 con 0x55b9e6a76e60 2023-10-06T05:23:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f907a7fc700 1 -- 172.21.15.33:0/126593016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f909005dd30 con 0x55b9e6a76e60 2023-10-06T05:23:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f907a7fc700 1 -- 172.21.15.33:0/126593016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9090055130 con 0x55b9e6a76e60 2023-10-06T05:23:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f90b0e17b80 10 client.7791 fetch_fsmap learned FSMap version 81 2023-10-06T05:23:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55b9e6953470 con 0x55b9e6a76e60 2023-10-06T05:23:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.703+0000 7f907a7fc700 1 -- 172.21.15.33:0/126593016 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9094014480 con 0x55b9e6a76e60 2023-10-06T05:23:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f907a7fc700 1 -- 172.21.15.33:0/126593016 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9090056ff0 con 0x55b9e6a76e60 2023-10-06T05:23:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f90b0e17b80 10 client.7791 fetch_fsmap finished waiting for FSMap version 81 2023-10-06T05:23:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f90b0e17b80 10 client.7791 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:23:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f90b0e17b80 20 client.7791 populate_metadata read hostname 'smithi033' 2023-10-06T05:23:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a49ba0 0x55b9e6a49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f90b0e17b80 4 client.7791 mds_command: new command op to 7644 tid=0[{"prefix": "status"}] 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 --> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b9e6940e00 con 0x55b9e6a49ba0 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f909bfff700 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a49ba0 0x55b9e6a49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f907a7fc700 10 client.7791 ms_handle_connect on v2:172.21.15.33:6832/723044599 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.704+0000 7f909bfff700 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a49ba0 0x55b9e6a49f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287a014ea1f03902 server_cookie=129025a2a0ed5d2a in_seq=0 out_seq=0 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f907a7fc700 1 -- 172.21.15.33:0/126593016 <== mds.0 v2:172.21.15.33:6832/723044599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55b9e6940e00 con 0x55b9e6a49ba0 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f907a7fc700 10 client.7791 handle_command_reply: tid=0 2023-10-06T05:23:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 client.7791 shutdown 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 msgr2=0x55b9e6af85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6af85e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9090056a90 tx=0x7f9090059040 comp rx=0 tx=0).stop 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 shutdown_connections 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:6832/723044599,v1:172.21.15.33:6834/723044599] conn(0x55b9e6a49ba0 0x55b9e6a49f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b9e6a97bc0 0x55b9e6af8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9e6a4d950 0x55b9e6a4e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 --2- 172.21.15.33:0/126593016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9e6a76e60 0x55b9e6af85e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f907a7fc700 0 client.7791 ms_handle_reset on v2:172.21.15.33:6832/723044599 2023-10-06T05:23:14.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 >> 172.21.15.33:0/126593016 conn(0x55b9e6a4f270 msgr2=0x55b9e6a4fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 shutdown_connections 2023-10-06T05:23:14.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 1 -- 172.21.15.33:0/126593016 wait complete. 2023-10-06T05:23:14.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.705+0000 7f90b0e17b80 20 client.7791 trim_cache size 0 max 16384 2023-10-06T05:23:14.722 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:23:14.722 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:23:14.722 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:23:14.722 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 7644, 2023-10-06T05:23:14.722 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:23:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:23:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:23:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 7.52497449, 2023-10-06T05:23:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 81, 2023-10-06T05:23:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 117, 2023-10-06T05:23:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 117, 2023-10-06T05:23:14.724 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 14.206759842 2023-10-06T05:23:14.724 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:23:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9088066f30 msgr2=0x7f90880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9088066f30 0x7f90880693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f909c0cbae0 tx=0x7f908c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 msgr2=0x7f909c169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 0x7f909c169fe0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f909c0c7640 tx=0x7f9090017000 comp rx=0 tx=0).stop 2023-10-06T05:23:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 shutdown_connections 2023-10-06T05:23:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9088066f30 0x7f90880693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f909c157150 0x7f909c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f909c1557f0 0x7f909c149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 --2- 172.21.15.33:0/1377923980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f909c155bc0 0x7f909c169fe0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 >> 172.21.15.33:0/1377923980 conn(0x7f909c06c560 msgr2=0x7f909c0b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 shutdown_connections 2023-10-06T05:23:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.706+0000 7f90a139d700 1 -- 172.21.15.33:0/1377923980 wait complete. 2023-10-06T05:23:14.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:14.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.913+0000 7f273a19d700 1 Processor -- start 2023-10-06T05:23:14.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.913+0000 7f273a19d700 1 -- start start 2023-10-06T05:23:14.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.914+0000 7f273a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27340ccfc0 0x7f27340cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.914+0000 7f273a19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27340cf3b0 0x7f27341594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.914+0000 7f273a19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27340cbd80 con 0x7f27340cc750 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.914+0000 7f273a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27340af5f0 con 0x7f27340ccfc0 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.914+0000 7f273a19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27340af8d0 con 0x7f27340cf3b0 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27340ccfc0 0x7f27340cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273999c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f27340cc750 0x7f27340b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40388/0 (socket says 172.21.15.33:40388) 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27340ccfc0 0x7f27340cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51782/0 (socket says 172.21.15.33:51782) 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273999c700 1 -- 172.21.15.33:0/3330952135 learned_addr learned my addr 172.21.15.33:0/3330952135 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273899a700 1 --2- 172.21.15.33:0/3330952135 >> v2:172.21.15.167:3300/0 conn(0x7f27340cf3b0 0x7f27341594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 -- 172.21.15.33:0/3330952135 >> v2:172.21.15.167:3300/0 conn(0x7f27340cf3b0 msgr2=0x7f27341594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 --2- 172.21.15.33:0/3330952135 >> v2:172.21.15.167:3300/0 conn(0x7f27340cf3b0 0x7f27341594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 -- 172.21.15.33:0/3330952135 >> v1:172.21.15.33:6789/0 conn(0x7f27340cc750 legacy=0x7f27340b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 -- 172.21.15.33:0/3330952135 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27340afbb0 con 0x7f27340ccfc0 2023-10-06T05:23:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273899a700 1 --2- 172.21.15.33:0/3330952135 >> v2:172.21.15.167:3300/0 conn(0x7f27340cf3b0 0x7f27341594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.915+0000 7f273919b700 1 --2- 172.21.15.33:0/3330952135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340cc210 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f27300098e0 tx=0x7f27300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f48cae6ab10ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f272bfff700 1 -- 172.21.15.33:0/3330952135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730009e80 con 0x7f27340ccfc0 2023-10-06T05:23:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f272bfff700 1 -- 172.21.15.33:0/3330952135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273000b820 con 0x7f27340ccfc0 2023-10-06T05:23:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f272bfff700 1 -- 172.21.15.33:0/3330952135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000bba0 con 0x7f27340ccfc0 2023-10-06T05:23:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 -- 172.21.15.33:0/3330952135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 msgr2=0x7f27340cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 --2- 172.21.15.33:0/3330952135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340cc210 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f27300098e0 tx=0x7f27300091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 -- 172.21.15.33:0/3330952135 shutdown_connections 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 --2- 172.21.15.33:0/3330952135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340cc210 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 --2- 172.21.15.33:0/3330952135 >> v2:172.21.15.167:3300/0 conn(0x7f27340cf3b0 0x7f27341594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 -- 172.21.15.33:0/3330952135 >> 172.21.15.33:0/3330952135 conn(0x7f273406cff0 msgr2=0x7f273406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 -- 172.21.15.33:0/3330952135 shutdown_connections 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.916+0000 7f273a19d700 1 -- 172.21.15.33:0/3330952135 wait complete. 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.918+0000 7f273a19d700 1 Processor -- start 2023-10-06T05:23:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.918+0000 7f273a19d700 1 -- start start 2023-10-06T05:23:14.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340cc750 0x7f2734146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27340b4a10 0x7f27340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27340c79c0 con 0x7f27340ccfc0 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2734146740 con 0x7f27340cc750 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27340b18c0 con 0x7f27340b4a10 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51788/0 (socket says 172.21.15.33:51788) 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273899a700 1 -- 172.21.15.33:0/4066628722 learned_addr learned my addr 172.21.15.33:0/4066628722 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273999c700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27340b4a10 0x7f27340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.919+0000 7f273919b700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340cc750 0x7f2734146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273899a700 1 -- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27340b4a10 msgr2=0x7f27340b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273899a700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27340b4a10 0x7f27340b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273899a700 1 -- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340cc750 msgr2=0x7f2734146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273899a700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340cc750 0x7f2734146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273899a700 1 -- 172.21.15.33:0/4066628722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2730009420 con 0x7f27340ccfc0 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273919b700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340cc750 0x7f2734146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273899a700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340c9d00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f272400d860 tx=0x7f2724008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240125f0 con 0x7f27340ccfc0 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2724012750 con 0x7f27340ccfc0 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724012af0 con 0x7f27340ccfc0 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27340b1ba0 con 0x7f27340ccfc0 2023-10-06T05:23:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.920+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27340c7650 con 0x7f27340ccfc0 2023-10-06T05:23:14.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.921+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273404c380 con 0x7f27340ccfc0 2023-10-06T05:23:14.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.922+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2724014430 con 0x7f27340ccfc0 2023-10-06T05:23:14.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.923+0000 7f27297fa700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2718066f30 0x7f27180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.923+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f27240849a0 con 0x7f27340ccfc0 2023-10-06T05:23:14.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.924+0000 7f273919b700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2718066f30 0x7f27180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.924+0000 7f273919b700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2718066f30 0x7f27180693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27300093f0 tx=0x7f2730017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:14.927+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2724051ab0 con 0x7f27340ccfc0 2023-10-06T05:23:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.173+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f27340c7b00 con 0x7f27340ccfc0 2023-10-06T05:23:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.174+0000 7f27297fa700 1 -- 172.21.15.33:0/4066628722 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7f27240550d0 con 0x7f27340ccfc0 2023-10-06T05:23:15.176 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2718066f30 msgr2=0x7f27180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2718066f30 0x7f27180693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27300093f0 tx=0x7f2730017000 comp rx=0 tx=0).stop 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 msgr2=0x7f27340c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340c9d00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f272400d860 tx=0x7f2724008040 comp rx=0 tx=0).stop 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 shutdown_connections 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2718066f30 0x7f27180693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27340b4a10 0x7f27340b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340cc750 0x7f2734146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 --2- 172.21.15.33:0/4066628722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27340ccfc0 0x7f27340c9d00 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 >> 172.21.15.33:0/4066628722 conn(0x7f273406cff0 msgr2=0x7f2734158700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 shutdown_connections 2023-10-06T05:23:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:15.176+0000 7f273a19d700 1 -- 172.21.15.33:0/4066628722 wait complete. 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:23:15.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:15.197 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:15.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:15.197 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:23:15.237 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41762sec preferred_lft 41762sec 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:15.241 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:15.339 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:23:15.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:15.342 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:15.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:15.383 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:23:15.383 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:23:15.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:23:15.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:23:15.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:23:15.383 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:23:15.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:15.581 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:23:15.618 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:23:15.618 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:23:15.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:15.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:15.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:23:15.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:15.721 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:15.758 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:15.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:15.758 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:15.834 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:23:15.835 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:15.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:23:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:23:15.983 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:15.983 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:23:15.983 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:23:15.983 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:23:15.983 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:23:16.003 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:16.003 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:23:16.004 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:23:16.004 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:23:16.004 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:23:16.004 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:23:16.004 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:23:16.004 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:23:16.008 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:23:16.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:16.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:16.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.255+0000 7fb65f59e700 1 Processor -- start 2023-10-06T05:23:16.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.255+0000 7fb65f59e700 1 -- start start 2023-10-06T05:23:16.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb66006ab00 0x7fb6600b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb6600b2e30 0x7fb66015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66015e2d0 con 0x7fb6600b0a40 2023-10-06T05:23:16.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66006af70 con 0x7fb6600b2e30 2023-10-06T05:23:16.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66015d6c0 con 0x7fb66006ab00 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65e59c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb6600b0a40 0x7fb6600b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55530/0 (socket says 172.21.15.33:55530) 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65e59c700 1 -- 172.21.15.33:0/1161942048 learned_addr learned my addr 172.21.15.33:0/1161942048 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65dd9b700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb66006ab00 0x7fb6600b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.257+0000 7fb65ed9d700 1 --2- 172.21.15.33:0/1161942048 >> v2:172.21.15.167:3300/0 conn(0x7fb6600b2e30 0x7fb66015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65d59a700 1 -- 172.21.15.33:0/1161942048 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571983093 0 0) 0x7fb66015e2d0 con 0x7fb6600b0a40 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65d59a700 1 -- 172.21.15.33:0/1161942048 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb648003570 con 0x7fb6600b0a40 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65ed9d700 1 -- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb66006ab00 msgr2=0x7fb6600b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65ed9d700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb66006ab00 0x7fb6600b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65ed9d700 1 -- 172.21.15.33:0/1161942048 >> v1:172.21.15.167:6789/0 conn(0x7fb6600b0a40 legacy=0x7fb6600b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65ed9d700 1 -- 172.21.15.33:0/1161942048 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66015d9a0 con 0x7fb6600b2e30 2023-10-06T05:23:16.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65dd9b700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb66006ab00 0x7fb6600b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:16.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65ed9d700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b2e30 0x7fb66015a0a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e60 tx=0x7fb658009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57a5a5df19ff6b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.258+0000 7fb65d59a700 1 -- 172.21.15.33:0/1161942048 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65800e070 con 0x7fb6600b2e30 2023-10-06T05:23:16.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65d59a700 1 -- 172.21.15.33:0/1161942048 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65800bab0 con 0x7fb6600b2e30 2023-10-06T05:23:16.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65d59a700 1 -- 172.21.15.33:0/1161942048 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65800be70 con 0x7fb6600b2e30 2023-10-06T05:23:16.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 -- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b2e30 msgr2=0x7fb66015a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b2e30 0x7fb66015a0a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e60 tx=0x7fb658009780 comp rx=0 tx=0).stop 2023-10-06T05:23:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 -- 172.21.15.33:0/1161942048 shutdown_connections 2023-10-06T05:23:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b2e30 0x7fb66015a0a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 --2- 172.21.15.33:0/1161942048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb66006ab00 0x7fb6600b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 -- 172.21.15.33:0/1161942048 >> 172.21.15.33:0/1161942048 conn(0x7fb66006e0f0 msgr2=0x7fb6600675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 -- 172.21.15.33:0/1161942048 shutdown_connections 2023-10-06T05:23:16.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.259+0000 7fb65f59e700 1 -- 172.21.15.33:0/1161942048 wait complete. 2023-10-06T05:23:16.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.261+0000 7fb65f59e700 1 Processor -- start 2023-10-06T05:23:16.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.261+0000 7fb65f59e700 1 -- start start 2023-10-06T05:23:16.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 0x7fb6600c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b0a40 0x7fb660169d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6600b2e30 0x7fb6600ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66016ad90 con 0x7fb66006ab00 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66016b460 con 0x7fb6600b0a40 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66016a7d0 con 0x7fb6600b2e30 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 0x7fb6600c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 0x7fb6600c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51796/0 (socket says 172.21.15.33:51796) 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 -- 172.21.15.33:0/4038264940 learned_addr learned my addr 172.21.15.33:0/4038264940 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65ed9d700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6600b2e30 0x7fb6600ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65dd9b700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b0a40 0x7fb660169d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 -- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6600b2e30 msgr2=0x7fb6600ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6600b2e30 0x7fb6600ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 -- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b0a40 msgr2=0x7fb660169d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b0a40 0x7fb660169d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65e59c700 1 -- 172.21.15.33:0/4038264940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654009420 con 0x7fb66006ab00 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.262+0000 7fb65ed9d700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6600b2e30 0x7fb6600ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.263+0000 7fb65e59c700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 0x7fb6600c5550 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c590 tx=0x7fb650007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.263+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650005b30 con 0x7fb66006ab00 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.263+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650005c90 con 0x7fb66006ab00 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.263+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb658009a20 con 0x7fb66006ab00 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.263+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650007320 con 0x7fb66006ab00 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.263+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66016b0f0 con 0x7fb66006ab00 2023-10-06T05:23:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.264+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66006fcc0 con 0x7fb66006ab00 2023-10-06T05:23:16.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.265+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb65001b3b0 con 0x7fb66006ab00 2023-10-06T05:23:16.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.266+0000 7fb64e7fc700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb640066f30 0x7fb6400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.266+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb650083b10 con 0x7fb66006ab00 2023-10-06T05:23:16.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.267+0000 7fb65dd9b700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb640066f30 0x7fb6400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.267+0000 7fb65dd9b700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb640066f30 0x7fb6400693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb6600b08d0 tx=0x7fb6540091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.271+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb650050b80 con 0x7fb66006ab00 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.516+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb66004c380 con 0x7fb66006ab00 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.517+0000 7fb64e7fc700 1 -- 172.21.15.33:0/4038264940 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb650017020 con 0x7fb66006ab00 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb640066f30 msgr2=0x7fb6400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb640066f30 0x7fb6400693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb6600b08d0 tx=0x7fb6540091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 msgr2=0x7fb6600c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 0x7fb6600c5550 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c590 tx=0x7fb650007e00 comp rx=0 tx=0).stop 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 shutdown_connections 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb640066f30 0x7fb6400693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6600b2e30 0x7fb6600ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb66006ab00 0x7fb6600c5550 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 --2- 172.21.15.33:0/4038264940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6600b0a40 0x7fb660169d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 >> 172.21.15.33:0/4038264940 conn(0x7fb66006e0f0 msgr2=0x7fb6600af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 shutdown_connections 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:16.520+0000 7fb65f59e700 1 -- 172.21.15.33:0/4038264940 wait complete. 2023-10-06T05:23:16.539 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:23:16.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:16.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:23:16.540 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:23:16.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:16.540 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:23:16.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:23:16.540 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:16.540 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:16.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.541 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:23:16.586 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:16.587 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:16.587 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:16.587 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.588 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:16.588 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.589 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:16.589 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:16.589 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:23:16.589 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:16.590 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:16.590 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:23:16.590 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:16.590 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:16.590 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41764sec preferred_lft 41764sec 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:16.591 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:23:16.592 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.593 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:16.593 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:23:16.593 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:23:16.593 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:16.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.594 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:16.690 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:23:16.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.693 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:23:16.732 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:23:16.732 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:16.732 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:23:16.732 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:23:16.732 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:16.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.734 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:23:16.734 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:23:16.734 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:23:16.734 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:23:16.734 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:23:16.734 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:23:16.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.930 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:23:16.966 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:23:16.966 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:23:16.966 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:16.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:16.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:23:17.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:17.068 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:17.103 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:17.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:17.104 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:17.179 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:23:17.179 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:17.179 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:23:17.310 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:23:17.310 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:23:17.310 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:23:17.326 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:23:17.326 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:23:17.326 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:23:17.326 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:23:17.327 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:23:17.347 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:23:17.347 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:23:17.347 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:23:17.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:23:17.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:23:17.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:23:17.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:23:17.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:23:17.351 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:23:17.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:17.352 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:17.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:23:17.628 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:23:17.629 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:23:17.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:23:17.833 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:23:17.834 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:23:17.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.021+0000 7fa69cf9d700 1 Processor -- start 2023-10-06T05:23:18.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.021+0000 7fa69cf9d700 1 -- start start 2023-10-06T05:23:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.022+0000 7fa69cf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa69806a0d0 0x7fa6980adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa69cf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa698158ba0 con 0x7fa698158700 2023-10-06T05:23:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa69cf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6980ae490 con 0x7fa698155140 2023-10-06T05:23:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa69cf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6980c8e40 con 0x7fa69806a0d0 2023-10-06T05:23:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa696ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa698158700 0x7fa6980699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40390/0 (socket says 172.21.15.33:40390) 2023-10-06T05:23:18.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa696ffd700 1 -- 172.21.15.33:0/444663948 learned_addr learned my addr 172.21.15.33:0/444663948 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa697fff700 1 --2- 172.21.15.33:0/444663948 >> v2:172.21.15.167:3300/0 conn(0x7fa69806a0d0 0x7fa6980adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404410299 0 0) 0x7fa698158ba0 con 0x7fa698158700 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.023+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa680003570 con 0x7fa698158700 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404352287 0 0) 0x7fa680003570 con 0x7fa698158700 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 >> v2:172.21.15.167:3300/0 conn(0x7fa69806a0d0 msgr2=0x7fa6980adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 --2- 172.21.15.33:0/444663948 >> v2:172.21.15.167:3300/0 conn(0x7fa69806a0d0 0x7fa6980adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 >> v1:172.21.15.167:6789/0 conn(0x7fa698155140 legacy=0x7fa6980c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980c99e0 con 0x7fa698158700 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa68c001470 con 0x7fa698158700 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 >> v1:172.21.15.33:6789/0 conn(0x7fa698158700 legacy=0x7fa6980699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa680003a60 0x7fa680003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa680004810 0x7fa680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6800051c0 0x7fa68000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980c8e40 con 0x7fa6800051c0 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68000c0b0 con 0x7fa680003a60 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68000c270 con 0x7fa680004810 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa697fff700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6800051c0 0x7fa68000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa6977fe700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa680003a60 0x7fa680003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa696ffd700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa680004810 0x7fa680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa697fff700 1 -- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa680004810 msgr2=0x7fa680004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa697fff700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa680004810 0x7fa680004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa697fff700 1 -- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa680003a60 msgr2=0x7fa680003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa697fff700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa680003a60 0x7fa680003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.024+0000 7fa697fff700 1 -- 172.21.15.33:0/444663948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680003570 con 0x7fa6800051c0 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa697fff700 1 -- 172.21.15.33:0/444663948 reap_dead start 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa697fff700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6800051c0 0x7fa68000ba60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa69806a4d0 tx=0x7fa69000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d41bb69b7e7dab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000ea60 con 0x7fa6800051c0 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69000f040 con 0x7fa6800051c0 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa6967fc700 1 -- 172.21.15.33:0/444663948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000ee30 con 0x7fa6800051c0 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa69cf9d700 1 -- 172.21.15.33:0/444663948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6800051c0 msgr2=0x7fa68000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.025+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6800051c0 0x7fa68000ba60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa69806a4d0 tx=0x7fa69000b040 comp rx=0 tx=0).stop 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.026+0000 7fa69cf9d700 1 -- 172.21.15.33:0/444663948 shutdown_connections 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.026+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/444663948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6800051c0 0x7fa68000ba60 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.026+0000 7fa69cf9d700 1 -- 172.21.15.33:0/444663948 >> 172.21.15.33:0/444663948 conn(0x7fa69806d030 msgr2=0x7fa69806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:18.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.026+0000 7fa69cf9d700 1 -- 172.21.15.33:0/444663948 shutdown_connections 2023-10-06T05:23:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.026+0000 7fa69cf9d700 1 -- 172.21.15.33:0/444663948 wait complete. 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.027+0000 7fa69cf9d700 1 Processor -- start 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.027+0000 7fa69cf9d700 1 -- start start 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa69cf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6800051c0 0x7fa69816a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa69cf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 0x7fa6980adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa69cf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6980c4770 0x7fa69814a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa69cf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980ae4c0 con 0x7fa69815aa30 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa69cf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980cb3c0 con 0x7fa6980c4770 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa69cf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69815aea0 con 0x7fa6800051c0 2023-10-06T05:23:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa696ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 0x7fa6980adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa696ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 0x7fa6980adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51828/0 (socket says 172.21.15.33:51828) 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.028+0000 7fa696ffd700 1 -- 172.21.15.33:0/2915946362 learned_addr learned my addr 172.21.15.33:0/2915946362 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa697fff700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6980c4770 0x7fa69814a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa6977fe700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6800051c0 0x7fa69816a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa696ffd700 1 -- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6800051c0 msgr2=0x7fa69816a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa696ffd700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6800051c0 0x7fa69816a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa696ffd700 1 -- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6980c4770 msgr2=0x7fa69814a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa696ffd700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6980c4770 0x7fa69814a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa696ffd700 1 -- 172.21.15.33:0/2915946362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980c9cb0 con 0x7fa69815aa30 2023-10-06T05:23:18.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa6977fe700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6800051c0 0x7fa69816a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa696ffd700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 0x7fa6980adf80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa68c001c30 tx=0x7fa68c001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c018030 con 0x7fa69815aa30 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c00f970 con 0x7fa69815aa30 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00fd10 con 0x7fa69815aa30 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69816a7b0 con 0x7fa69815aa30 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.029+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69816ad70 con 0x7fa69815aa30 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.030+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6980c9ed0 con 0x7fa69815aa30 2023-10-06T05:23:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.031+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa68c01b070 con 0x7fa69815aa30 2023-10-06T05:23:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.032+0000 7fa67f7fe700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa684066f30 0x7fa6840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.032+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa68c019070 con 0x7fa69815aa30 2023-10-06T05:23:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.033+0000 7fa6977fe700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa684066f30 0x7fa6840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.033+0000 7fa6977fe700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa684066f30 0x7fa6840693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa688000c00 tx=0x7fa688009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.036+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6980c9ed0 con 0x7fa69815aa30 2023-10-06T05:23:18.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.283+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa69804c380 con 0x7fa69815aa30 2023-10-06T05:23:18.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.284+0000 7fa67f7fe700 1 -- 172.21.15.33:0/2915946362 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fa68c0517f0 con 0x7fa69815aa30 2023-10-06T05:23:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:18.286 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:18.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.286+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa684066f30 msgr2=0x7fa6840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.286+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa684066f30 0x7fa6840693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa688000c00 tx=0x7fa688009040 comp rx=0 tx=0).stop 2023-10-06T05:23:18.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.286+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 msgr2=0x7fa6980adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.286+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 0x7fa6980adf80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa68c001c30 tx=0x7fa68c001da0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 shutdown_connections 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa684066f30 0x7fa6840693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6980c4770 0x7fa69814a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6800051c0 0x7fa69816a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 --2- 172.21.15.33:0/2915946362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa69815aa30 0x7fa6980adf80 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 >> 172.21.15.33:0/2915946362 conn(0x7fa69806d030 msgr2=0x7fa6980c6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 shutdown_connections 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:18.287+0000 7fa69cf9d700 1 -- 172.21.15.33:0/2915946362 wait complete. 2023-10-06T05:23:18.291 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:18.306 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:23:18.306 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:23:18.306 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:23:18.307 DEBUG:teuthology.orchestra.run.smithi033:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2023-10-06T05:23:18.307 DEBUG:teuthology.orchestra.run.smithi033:> f.write("content") 2023-10-06T05:23:18.307 DEBUG:teuthology.orchestra.run.smithi033:> f.flush() 2023-10-06T05:23:18.307 DEBUG:teuthology.orchestra.run.smithi033:> while True: 2023-10-06T05:23:18.308 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(1) 2023-10-06T05:23:18.308 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:23:18.312 DEBUG:teuthology.orchestra.run.smithi033:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/background_file 2023-10-06T05:23:18.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2023-10-06T05:23:19.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:19.117 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:19.352 DEBUG:teuthology.orchestra.run.smithi033:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/background_file 2023-10-06T05:23:19.391 INFO:teuthology.orchestra.run.smithi033.stdout:7 2023-10-06T05:23:19.391 INFO:tasks.cephfs.mount:File background_file became visible with size 7 from 0 after 1s 2023-10-06T05:23:19.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:19.392 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:23:19.506 INFO:teuthology.orchestra.run.smithi033.stdout:client7803 2023-10-06T05:23:19.507 DEBUG:teuthology.orchestra.run.smithi167:> stat /home/ubuntu/cephtest/mnt.1/background_file 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi167.stdout: File: /home/ubuntu/cephtest/mnt.1/background_file 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 1 IO Block: 4194304 regular file 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 2bh/43d Inode: 1099511627776 Links: 1 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0664/-rw-rw-r--) Uid: ( 1000/ ubuntu) Gid: ( 1275/ ubuntu) 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:23:18.413704179 +0000 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:23:18.421704053 +0000 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:23:18.421704053 +0000 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:23:19.553 INFO:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2023-10-06T05:23:19.553 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:19.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> import os 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> import time 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:23:24.626 DEBUG:teuthology.orchestra.run.smithi167:> while True: 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> os.write(fd, b'"'"'content'"'"') 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> time.sleep(1) 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> if not False: 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> break 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> os.close(fd) 2023-10-06T05:23:24.627 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:23:29.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.932+0000 7f8028d9d700 1 Processor -- start 2023-10-06T05:23:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.932+0000 7f8028d9d700 1 -- start start 2023-10-06T05:23:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.933+0000 7f8028d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80240cf3b0 0x7f80241594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.933+0000 7f8028d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80240cbd80 con 0x7f80240af0a0 2023-10-06T05:23:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.933+0000 7f8028d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80240af5f0 con 0x7f80240ccfc0 2023-10-06T05:23:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.933+0000 7f8028d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80240af8d0 con 0x7f80240cf3b0 2023-10-06T05:23:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.933+0000 7f8023fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f80240af0a0 0x7f80240b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47788/0 (socket says 172.21.15.33:47788) 2023-10-06T05:23:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.933+0000 7f8023fff700 1 -- 172.21.15.33:0/305859379 learned_addr learned my addr 172.21.15.33:0/305859379 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f8022ffd700 1 --2- 172.21.15.33:0/305859379 >> v2:172.21.15.167:3300/0 conn(0x7f80240cf3b0 0x7f80241594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736773333 0 0) 0x7f80240cbd80 con 0x7f80240af0a0 2023-10-06T05:23:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f800c003570 con 0x7f80240af0a0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511172700 0 0) 0x7f80240af5f0 con 0x7f80240ccfc0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80240cbd80 con 0x7f80240ccfc0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f8022ffd700 1 -- 172.21.15.33:0/305859379 >> v1:172.21.15.167:6789/0 conn(0x7f80240ccfc0 legacy=0x7f80240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f8022ffd700 1 -- 172.21.15.33:0/305859379 >> v1:172.21.15.33:6789/0 conn(0x7f80240af0a0 legacy=0x7f80240b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.934+0000 7f8022ffd700 1 -- 172.21.15.33:0/305859379 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240afbb0 con 0x7f80240cf3b0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f8022ffd700 1 --2- 172.21.15.33:0/305859379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240cf3b0 0x7f80241594a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8018009910 tx=0x7f80180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed428f2da4acf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018011630 con 0x7f80240cf3b0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8018011790 con 0x7f80240cf3b0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f80227fc700 1 -- 172.21.15.33:0/305859379 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018011b30 con 0x7f80240cf3b0 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f8028d9d700 1 -- 172.21.15.33:0/305859379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240cf3b0 msgr2=0x7f80241594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f8028d9d700 1 --2- 172.21.15.33:0/305859379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240cf3b0 0x7f80241594a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8018009910 tx=0x7f80180091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f8028d9d700 1 -- 172.21.15.33:0/305859379 shutdown_connections 2023-10-06T05:23:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f8028d9d700 1 --2- 172.21.15.33:0/305859379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240cf3b0 0x7f80241594a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.935+0000 7f8028d9d700 1 -- 172.21.15.33:0/305859379 >> 172.21.15.33:0/305859379 conn(0x7f802406cff0 msgr2=0x7f802406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.936+0000 7f8028d9d700 1 -- 172.21.15.33:0/305859379 shutdown_connections 2023-10-06T05:23:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.936+0000 7f8028d9d700 1 -- 172.21.15.33:0/305859379 wait complete. 2023-10-06T05:23:29.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.937+0000 7f8028d9d700 1 Processor -- start 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.937+0000 7f8028d9d700 1 -- start start 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8028d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80240af0a0 0x7f80240b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8028d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 0x7f8024153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8028d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240c9ee0 0x7f80240ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8028d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024169900 con 0x7f80240ccfc0 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8028d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240b3bd0 con 0x7f80240c9ee0 2023-10-06T05:23:29.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8028d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80240b3eb0 con 0x7f80240af0a0 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8022ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 0x7f8024153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8022ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 0x7f8024153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56072/0 (socket says 172.21.15.33:56072) 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.938+0000 7f8022ffd700 1 -- 172.21.15.33:0/126184579 learned_addr learned my addr 172.21.15.33:0/126184579 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8023fff700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240c9ee0 0x7f80240ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f80237fe700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80240af0a0 0x7f80240b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8022ffd700 1 -- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80240af0a0 msgr2=0x7f80240b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8022ffd700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80240af0a0 0x7f80240b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8022ffd700 1 -- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240c9ee0 msgr2=0x7f80240ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8022ffd700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240c9ee0 0x7f80240ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8022ffd700 1 -- 172.21.15.33:0/126184579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018009420 con 0x7f80240ccfc0 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f8022ffd700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 0x7f8024153fb0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8018013000 tx=0x7f8018017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.939+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801801f070 con 0x7f80240ccfc0 2023-10-06T05:23:29.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.940+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8018000bc0 con 0x7f80240ccfc0 2023-10-06T05:23:29.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.940+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80240b4190 con 0x7f80240ccfc0 2023-10-06T05:23:29.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.940+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018025420 con 0x7f80240ccfc0 2023-10-06T05:23:29.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.940+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80240b3850 con 0x7f80240ccfc0 2023-10-06T05:23:29.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.941+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80240b46b0 con 0x7f80240ccfc0 2023-10-06T05:23:29.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.942+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8018023070 con 0x7f80240ccfc0 2023-10-06T05:23:29.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.943+0000 7f800b7fe700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8010066f30 0x7f80100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.943+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8018085a20 con 0x7f80240ccfc0 2023-10-06T05:23:29.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.943+0000 7f80237fe700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8010066f30 0x7f80100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.944+0000 7f80237fe700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8010066f30 0x7f80100693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8014003870 tx=0x7f8014009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:29.947+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80240b46b0 con 0x7f80240ccfc0 2023-10-06T05:23:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.192+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80240b46b0 con 0x7f80240ccfc0 2023-10-06T05:23:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.193+0000 7f800b7fe700 1 -- 172.21.15.33:0/126184579 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f80240b46b0 con 0x7f80240ccfc0 2023-10-06T05:23:30.194 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:30.195 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8010066f30 msgr2=0x7f80100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8010066f30 0x7f80100693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8014003870 tx=0x7f8014009040 comp rx=0 tx=0).stop 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 msgr2=0x7f8024153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 0x7f8024153fb0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8018013000 tx=0x7f8018017000 comp rx=0 tx=0).stop 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 shutdown_connections 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8010066f30 0x7f80100693e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80240c9ee0 0x7f80240ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80240af0a0 0x7f80240b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 --2- 172.21.15.33:0/126184579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80240ccfc0 0x7f8024153fb0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 >> 172.21.15.33:0/126184579 conn(0x7f802406cff0 msgr2=0x7f8024155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 shutdown_connections 2023-10-06T05:23:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:30.195+0000 7f8028d9d700 1 -- 172.21.15.33:0/126184579 wait complete. 2023-10-06T05:23:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:30.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session evict 7803 2023-10-06T05:23:31.046 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:23:32.135 INFO:tasks.cephfs.test_client_recovery:cap_waiter waited 1.089029312133789s 2023-10-06T05:23:32.135 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:32.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:32.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:23:32.221 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:23:32.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:32.344 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:32.433 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:23:32.434 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:23:32.434 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:23:32.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:32.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:33.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:23:33.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:33.488 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:33.529 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:23:33.529 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:23:33.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:33.529 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:33.617 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:23:33.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:33.617 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:23:33.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.849+0000 7fa3c539d700 1 Processor -- start 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.849+0000 7fa3c539d700 1 -- start start 2023-10-06T05:23:33.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.850+0000 7fa3c539d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa3c00c8050 0x7fa3c013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3c539d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c00c9cb0 con 0x7fa3c013faa0 2023-10-06T05:23:33.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3c539d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c00ca270 con 0x7fa3c006aad0 2023-10-06T05:23:33.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3c539d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c006b640 con 0x7fa3c00c8050 2023-10-06T05:23:33.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3bffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa3c013faa0 0x7fa3c00c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47798/0 (socket says 172.21.15.33:47798) 2023-10-06T05:23:33.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3bffff700 1 -- 172.21.15.33:0/3030573885 learned_addr learned my addr 172.21.15.33:0/3030573885 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/3030573885 >> v2:172.21.15.167:3300/0 conn(0x7fa3c00c8050 0x7fa3c013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34663728 0 0) 0x7fa3c00c9cb0 con 0x7fa3c013faa0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.851+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a8003570 con 0x7fa3c013faa0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618074705 0 0) 0x7fa3c00ca270 con 0x7fa3c006aad0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c00c9cb0 con 0x7fa3c006aad0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 491172559 0 0) 0x7fa3a8003570 con 0x7fa3c013faa0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 >> v2:172.21.15.167:3300/0 conn(0x7fa3c00c8050 msgr2=0x7fa3c013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 --2- 172.21.15.33:0/3030573885 >> v2:172.21.15.167:3300/0 conn(0x7fa3c00c8050 0x7fa3c013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 >> v1:172.21.15.167:6789/0 conn(0x7fa3c006aad0 legacy=0x7fa3c006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c006b840 con 0x7fa3c013faa0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa3b0001470 con 0x7fa3c013faa0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 >> v1:172.21.15.33:6789/0 conn(0x7fa3c013faa0 legacy=0x7fa3c00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a8003c40 0x7fa3a8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a8005210 0x7fa3a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c006b640 con 0x7fa3a8004810 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800c130 con 0x7fa3a8003c40 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800c2c0 con 0x7fa3a8005210 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3bffff700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a8003c40 0x7fa3a8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.852+0000 7fa3c4b9c700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a8005210 0x7fa3a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3bf7fe700 1 -- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a8005210 msgr2=0x7fa3a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a8005210 0x7fa3a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3bf7fe700 1 -- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a8003c40 msgr2=0x7fa3a8004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3a8003c40 0x7fa3a8004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3bf7fe700 1 -- 172.21.15.33:0/3030573885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8003570 con 0x7fa3a8004810 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3c4b9c700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3a8005210 0x7fa3a800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3c4b9c700 1 -- 172.21.15.33:0/3030573885 reap_dead start 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3a8004c80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009c30 tx=0x7fa3b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eac472a6639afad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40095b0 con 0x7fa3a8004810 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b400e710 con 0x7fa3a8004810 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.853+0000 7fa3beffd700 1 -- 172.21.15.33:0/3030573885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400ead0 con 0x7fa3a8004810 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 -- 172.21.15.33:0/3030573885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 msgr2=0x7fa3a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3a8004c80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009c30 tx=0x7fa3b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 -- 172.21.15.33:0/3030573885 shutdown_connections 2023-10-06T05:23:33.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 --2- 172.21.15.33:0/3030573885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3a8004c80 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 -- 172.21.15.33:0/3030573885 >> 172.21.15.33:0/3030573885 conn(0x7fa3c006d030 msgr2=0x7fa3c006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 -- 172.21.15.33:0/3030573885 shutdown_connections 2023-10-06T05:23:33.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.854+0000 7fa3c539d700 1 -- 172.21.15.33:0/3030573885 wait complete. 2023-10-06T05:23:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.855+0000 7fa3c539d700 1 Processor -- start 2023-10-06T05:23:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 -- start start 2023-10-06T05:23:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3c00c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3c0156c00 0x7fa3c0139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c0157070 0x7fa3c013a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00c5b80 con 0x7fa3a8004810 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c013dde0 con 0x7fa3c0157070 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.856+0000 7fa3c539d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00c58a0 con 0x7fa3c0156c00 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3c00c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3c00c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56090/0 (socket says 172.21.15.33:56090) 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 -- 172.21.15.33:0/2791874632 learned_addr learned my addr 172.21.15.33:0/2791874632 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 -- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3c0156c00 msgr2=0x7fa3c0139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3c0156c00 0x7fa3c0139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 -- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c0157070 msgr2=0x7fa3c013a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c0157070 0x7fa3c013a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 -- 172.21.15.33:0/2791874632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c013d790 con 0x7fa3a8004810 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3bffff700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3c00c47e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0007390 tx=0x7fa3c00c84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000f8d0 con 0x7fa3a8004810 2023-10-06T05:23:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000fa50 con 0x7fa3a8004810 2023-10-06T05:23:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.857+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000fe10 con 0x7fa3a8004810 2023-10-06T05:23:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.858+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c013da70 con 0x7fa3a8004810 2023-10-06T05:23:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.858+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c013e270 con 0x7fa3a8004810 2023-10-06T05:23:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.859+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c013e0c0 con 0x7fa3a8004810 2023-10-06T05:23:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.860+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3b0013600 con 0x7fa3a8004810 2023-10-06T05:23:33.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.861+0000 7fa3a7fff700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3ac066fb0 0x7fa3ac069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.861+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3b0084920 con 0x7fa3a8004810 2023-10-06T05:23:33.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.862+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3ac066fb0 0x7fa3ac069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.862+0000 7fa3bf7fe700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3ac066fb0 0x7fa3ac069460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00c2ec0 tx=0x7fa3b401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:33.865+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3b0051960 con 0x7fa3a8004810 2023-10-06T05:23:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.114+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c004c380 con 0x7fa3a8004810 2023-10-06T05:23:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.115+0000 7fa3a7fff700 1 -- 172.21.15.33:0/2791874632 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fa3b0054f80 con 0x7fa3a8004810 2023-10-06T05:23:34.116 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:34.117 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3ac066fb0 msgr2=0x7fa3ac069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3ac066fb0 0x7fa3ac069460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00c2ec0 tx=0x7fa3b401a000 comp rx=0 tx=0).stop 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 msgr2=0x7fa3c00c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3c00c47e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0007390 tx=0x7fa3c00c84c0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 shutdown_connections 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3ac066fb0 0x7fa3ac069460 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c0157070 0x7fa3c013a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a8004810 0x7fa3c00c47e0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 --2- 172.21.15.33:0/2791874632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3c0156c00 0x7fa3c0139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 >> 172.21.15.33:0/2791874632 conn(0x7fa3c006d030 msgr2=0x7fa3c0155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 shutdown_connections 2023-10-06T05:23:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.117+0000 7fa3c539d700 1 -- 172.21.15.33:0/2791874632 wait complete. 2023-10-06T05:23:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:34.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.322+0000 7fda46f9d700 1 Processor -- start 2023-10-06T05:23:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.322+0000 7fda46f9d700 1 -- start start 2023-10-06T05:23:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.323+0000 7fda46f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fda40159530 0x7fda4006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.323+0000 7fda46f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4006df70 0x7fda400afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.323+0000 7fda46f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4006e3e0 con 0x7fda400b0310 2023-10-06T05:23:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.323+0000 7fda46f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda400b1e00 con 0x7fda40159530 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.323+0000 7fda46f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4006f6b0 con 0x7fda4006df70 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda45f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fda40159530 0x7fda4006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda4679c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fda400b0310 0x7fda40159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47800/0 (socket says 172.21.15.33:47800) 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda4679c700 1 -- 172.21.15.33:0/2713585027 learned_addr learned my addr 172.21.15.33:0/2713585027 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda4579a700 1 --2- 172.21.15.33:0/2713585027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4006df70 0x7fda400afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda44f99700 1 -- 172.21.15.33:0/2713585027 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400081794 0 0) 0x7fda4006e3e0 con 0x7fda400b0310 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda45f9b700 1 -- 172.21.15.33:0/2713585027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4006df70 msgr2=0x7fda400afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda45f9b700 1 --2- 172.21.15.33:0/2713585027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4006df70 0x7fda400afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda45f9b700 1 -- 172.21.15.33:0/2713585027 >> v1:172.21.15.33:6789/0 conn(0x7fda400b0310 legacy=0x7fda40159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.324+0000 7fda45f9b700 1 -- 172.21.15.33:0/2713585027 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4006f220 con 0x7fda40159530 2023-10-06T05:23:34.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda45f9b700 1 --2- 172.21.15.33:0/2713585027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda40159530 0x7fda4006da30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0098e0 tx=0x7fda3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6aaeb43a3ec46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda44f99700 1 -- 172.21.15.33:0/2713585027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c009e80 con 0x7fda40159530 2023-10-06T05:23:34.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda44f99700 1 -- 172.21.15.33:0/2713585027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c00b860 con 0x7fda40159530 2023-10-06T05:23:34.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda44f99700 1 -- 172.21.15.33:0/2713585027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00bbe0 con 0x7fda40159530 2023-10-06T05:23:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda46f9d700 1 -- 172.21.15.33:0/2713585027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda40159530 msgr2=0x7fda4006da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda46f9d700 1 --2- 172.21.15.33:0/2713585027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda40159530 0x7fda4006da30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0098e0 tx=0x7fda3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda46f9d700 1 -- 172.21.15.33:0/2713585027 shutdown_connections 2023-10-06T05:23:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda46f9d700 1 --2- 172.21.15.33:0/2713585027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda40159530 0x7fda4006da30 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda46f9d700 1 --2- 172.21.15.33:0/2713585027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4006df70 0x7fda400afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.325+0000 7fda46f9d700 1 -- 172.21.15.33:0/2713585027 >> 172.21.15.33:0/2713585027 conn(0x7fda4006c560 msgr2=0x7fda4006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.326+0000 7fda46f9d700 1 -- 172.21.15.33:0/2713585027 shutdown_connections 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.326+0000 7fda46f9d700 1 -- 172.21.15.33:0/2713585027 wait complete. 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.327+0000 7fda46f9d700 1 Processor -- start 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.327+0000 7fda46f9d700 1 -- start start 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda46f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 0x7fda4016cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda46f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda400b0310 0x7fda40150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda46f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda40159530 0x7fda4015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda46f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40153e00 con 0x7fda4006df70 2023-10-06T05:23:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda46f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4016d160 con 0x7fda400b0310 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda46f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda401531f0 con 0x7fda40159530 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda45f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 0x7fda4016cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda45f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 0x7fda4016cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56118/0 (socket says 172.21.15.33:56118) 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda45f9b700 1 -- 172.21.15.33:0/1160216241 learned_addr learned my addr 172.21.15.33:0/1160216241 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda4679c700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda40159530 0x7fda4015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.328+0000 7fda4579a700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda400b0310 0x7fda40150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda45f9b700 1 -- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda40159530 msgr2=0x7fda4015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda45f9b700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda40159530 0x7fda4015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda45f9b700 1 -- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda400b0310 msgr2=0x7fda40150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda45f9b700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda400b0310 0x7fda40150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda45f9b700 1 -- 172.21.15.33:0/1160216241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009420 con 0x7fda4006df70 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda4679c700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda40159530 0x7fda4015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda45f9b700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 0x7fda4016cc20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fda4006b870 tx=0x7fda3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00bbe0 con 0x7fda4006df70 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c011420 con 0x7fda4006df70 2023-10-06T05:23:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda401534d0 con 0x7fda4006df70 2023-10-06T05:23:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011800 con 0x7fda4006df70 2023-10-06T05:23:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.329+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda40153a90 con 0x7fda4006df70 2023-10-06T05:23:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.330+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4004c380 con 0x7fda4006df70 2023-10-06T05:23:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.331+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda3c011bf0 con 0x7fda4006df70 2023-10-06T05:23:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.332+0000 7fda35ffb700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda24066f30 0x7fda240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.332+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda3c0860e0 con 0x7fda4006df70 2023-10-06T05:23:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.333+0000 7fda4579a700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda24066f30 0x7fda240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.333+0000 7fda4579a700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda24066f30 0x7fda240693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda400b4270 tx=0x7fda30008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.336+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda3c053120 con 0x7fda4006df70 2023-10-06T05:23:34.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.540+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda4006dd50 con 0x7fda4006df70 2023-10-06T05:23:34.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.541+0000 7fda35ffb700 1 -- 172.21.15.33:0/1160216241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16939 (secure 0 0 0) 0x7fda3c056740 con 0x7fda4006df70 2023-10-06T05:23:34.543 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:34.543 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":118,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:31.012006+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:04.306347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:05.660589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/588756950":"2023-10-06T06:23:30.410369+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda24066f30 msgr2=0x7fda240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda24066f30 0x7fda240693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda400b4270 tx=0x7fda30008040 comp rx=0 tx=0).stop 2023-10-06T05:23:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 msgr2=0x7fda4016cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 0x7fda4016cc20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fda4006b870 tx=0x7fda3c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 shutdown_connections 2023-10-06T05:23:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda24066f30 0x7fda240693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda40159530 0x7fda4015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4006df70 0x7fda4016cc20 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 --2- 172.21.15.33:0/1160216241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda400b0310 0x7fda40150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 >> 172.21.15.33:0/1160216241 conn(0x7fda4006c560 msgr2=0x7fda40155980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 shutdown_connections 2023-10-06T05:23:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.543+0000 7fda46f9d700 1 -- 172.21.15.33:0/1160216241 wait complete. 2023-10-06T05:23:34.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:23:04.306347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:23:05.660589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:23:34.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:23:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.751+0000 7fd03ffff700 1 Processor -- start 2023-10-06T05:23:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.751+0000 7fd03ffff700 1 -- start start 2023-10-06T05:23:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0400cc7e0 0x7fd0400af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400ad430 con 0x7fd0400b01f0 2023-10-06T05:23:34.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400afe20 con 0x7fd040156370 2023-10-06T05:23:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04006bc70 con 0x7fd0400cc7e0 2023-10-06T05:23:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0400cc7e0 0x7fd0400af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd0400b01f0 0x7fd0400ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47804/0 (socket says 172.21.15.33:47804) 2023-10-06T05:23:34.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03f7fe700 1 -- 172.21.15.33:0/2221320085 learned_addr learned my addr 172.21.15.33:0/2221320085 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03e7fc700 1 -- 172.21.15.33:0/2221320085 >> v1:172.21.15.167:6789/0 conn(0x7fd040156370 legacy=0x7fd0400c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03e7fc700 1 -- 172.21.15.33:0/2221320085 >> v1:172.21.15.33:6789/0 conn(0x7fd0400b01f0 legacy=0x7fd0400ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.753+0000 7fd03e7fc700 1 -- 172.21.15.33:0/2221320085 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400cccc0 con 0x7fd0400cc7e0 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.754+0000 7fd03e7fc700 1 --2- 172.21.15.33:0/2221320085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400cc7e0 0x7fd0400af8e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd0340098e0 tx=0x7fd0340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260e64105d5dc6f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.754+0000 7fd03dffb700 1 -- 172.21.15.33:0/2221320085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034009e80 con 0x7fd0400cc7e0 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.754+0000 7fd03dffb700 1 -- 172.21.15.33:0/2221320085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03400b860 con 0x7fd0400cc7e0 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.754+0000 7fd03dffb700 1 -- 172.21.15.33:0/2221320085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400bbe0 con 0x7fd0400cc7e0 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.754+0000 7fd03ffff700 1 -- 172.21.15.33:0/2221320085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400cc7e0 msgr2=0x7fd0400af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.754+0000 7fd03ffff700 1 --2- 172.21.15.33:0/2221320085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400cc7e0 0x7fd0400af8e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd0340098e0 tx=0x7fd0340091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.755+0000 7fd03ffff700 1 -- 172.21.15.33:0/2221320085 shutdown_connections 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.755+0000 7fd03ffff700 1 --2- 172.21.15.33:0/2221320085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0400cc7e0 0x7fd0400af8e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.755+0000 7fd03ffff700 1 -- 172.21.15.33:0/2221320085 >> 172.21.15.33:0/2221320085 conn(0x7fd04006d030 msgr2=0x7fd0400cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.755+0000 7fd03ffff700 1 -- 172.21.15.33:0/2221320085 shutdown_connections 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.755+0000 7fd03ffff700 1 -- 172.21.15.33:0/2221320085 wait complete. 2023-10-06T05:23:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.756+0000 7fd03ffff700 1 Processor -- start 2023-10-06T05:23:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.756+0000 7fd03ffff700 1 -- start start 2023-10-06T05:23:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.757+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0400b01f0 0x7fd040142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.757+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0400cc7e0 0x7fd0400c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.757+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 0x7fd0400c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.757+0000 7fd03ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040142e50 con 0x7fd040156370 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.757+0000 7fd03ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040150550 con 0x7fd0400b01f0 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.757+0000 7fd03ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd040158500 con 0x7fd0400cc7e0 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 0x7fd0400c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 0x7fd0400c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56128/0 (socket says 172.21.15.33:56128) 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 -- 172.21.15.33:0/505703069 learned_addr learned my addr 172.21.15.33:0/505703069 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03effd700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0400b01f0 0x7fd040142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03e7fc700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0400cc7e0 0x7fd0400c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 -- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0400cc7e0 msgr2=0x7fd0400c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0400cc7e0 0x7fd0400c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 -- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0400b01f0 msgr2=0x7fd040142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0400b01f0 0x7fd040142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 -- 172.21.15.33:0/505703069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034009420 con 0x7fd040156370 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.758+0000 7fd03f7fe700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 0x7fd0400c9f40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd03800cc70 tx=0x7fd038001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.759+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800a900 con 0x7fd040156370 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.759+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03800aa60 con 0x7fd040156370 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.759+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0401587e0 con 0x7fd040156370 2023-10-06T05:23:34.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.759+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800ae20 con 0x7fd040156370 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.759+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd040158e20 con 0x7fd040156370 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.760+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04004c380 con 0x7fd040156370 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.761+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd038007300 con 0x7fd040156370 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.762+0000 7fd026ffd700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd02c066f30 0x7fd02c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.762+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd038084a00 con 0x7fd040156370 2023-10-06T05:23:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.762+0000 7fd03effd700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd02c066f30 0x7fd02c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.763+0000 7fd03effd700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd02c066f30 0x7fd02c0693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0400ae7e0 tx=0x7fd030009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.766+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd038051b60 con 0x7fd040156370 2023-10-06T05:23:34.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.968+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd040155b50 con 0x7fd040156370 2023-10-06T05:23:34.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.968+0000 7fd026ffd700 1 -- 172.21.15.33:0/505703069 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd038055180 con 0x7fd040156370 2023-10-06T05:23:34.970 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:23:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd02c066f30 msgr2=0x7fd02c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd02c066f30 0x7fd02c0693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0400ae7e0 tx=0x7fd030009040 comp rx=0 tx=0).stop 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 msgr2=0x7fd0400c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 0x7fd0400c9f40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd03800cc70 tx=0x7fd038001a50 comp rx=0 tx=0).stop 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 shutdown_connections 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd02c066f30 0x7fd02c0693e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd040156370 0x7fd0400c9f40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0400b01f0 0x7fd040142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 --2- 172.21.15.33:0/505703069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0400cc7e0 0x7fd0400c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 >> 172.21.15.33:0/505703069 conn(0x7fd04006d030 msgr2=0x7fd0400cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 shutdown_connections 2023-10-06T05:23:34.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:34.971+0000 7fd03ffff700 1 -- 172.21.15.33:0/505703069 wait complete. 2023-10-06T05:23:34.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:23:35.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.179+0000 7ffa49d9d700 1 Processor -- start 2023-10-06T05:23:35.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.179+0000 7ffa49d9d700 1 -- start start 2023-10-06T05:23:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.180+0000 7ffa49d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa44159850 0x7ffa4406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.180+0000 7ffa49d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffa4415acd0 0x7ffa4406edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.180+0000 7ffa49d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa440b0cd0 con 0x7ffa4406d140 2023-10-06T05:23:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.180+0000 7ffa49d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4415a840 con 0x7ffa44159850 2023-10-06T05:23:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.180+0000 7ffa49d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa44159f10 con 0x7ffa4415acd0 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.180+0000 7ffa48d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa44159850 0x7ffa4406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa44159850 0x7ffa4406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56130/0 (socket says 172.21.15.33:56130) 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 -- 172.21.15.33:0/489001849 learned_addr learned my addr 172.21.15.33:0/489001849 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 -- 172.21.15.33:0/489001849 >> v2:172.21.15.167:3300/0 conn(0x7ffa4415acd0 msgr2=0x7ffa4406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa43fff700 1 --2- 172.21.15.33:0/489001849 >> v2:172.21.15.167:3300/0 conn(0x7ffa4415acd0 0x7ffa4406edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 --2- 172.21.15.33:0/489001849 >> v2:172.21.15.167:3300/0 conn(0x7ffa4415acd0 0x7ffa4406edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 -- 172.21.15.33:0/489001849 >> v1:172.21.15.167:6789/0 conn(0x7ffa4406d140 legacy=0x7ffa4406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 -- 172.21.15.33:0/489001849 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4415a1f0 con 0x7ffa44159850 2023-10-06T05:23:35.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa43fff700 1 --2- 172.21.15.33:0/489001849 >> v2:172.21.15.167:3300/0 conn(0x7ffa4415acd0 0x7ffa4406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa48d9b700 1 --2- 172.21.15.33:0/489001849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4406e890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ffa4406a730 tx=0x7ffa38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a5b0fcb97a170e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.181+0000 7ffa437fe700 1 -- 172.21.15.33:0/489001849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800b820 con 0x7ffa44159850 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa437fe700 1 -- 172.21.15.33:0/489001849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3800b980 con 0x7ffa44159850 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa437fe700 1 -- 172.21.15.33:0/489001849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800bd20 con 0x7ffa44159850 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 -- 172.21.15.33:0/489001849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 msgr2=0x7ffa4406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/489001849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4406e890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ffa4406a730 tx=0x7ffa38009250 comp rx=0 tx=0).stop 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 -- 172.21.15.33:0/489001849 shutdown_connections 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/489001849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4406e890 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/489001849 >> v2:172.21.15.167:3300/0 conn(0x7ffa4415acd0 0x7ffa4406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 -- 172.21.15.33:0/489001849 >> 172.21.15.33:0/489001849 conn(0x7ffa4406cd40 msgr2=0x7ffa440ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 -- 172.21.15.33:0/489001849 shutdown_connections 2023-10-06T05:23:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.182+0000 7ffa49d9d700 1 -- 172.21.15.33:0/489001849 wait complete. 2023-10-06T05:23:35.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.184+0000 7ffa49d9d700 1 Processor -- start 2023-10-06T05:23:35.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.184+0000 7ffa49d9d700 1 -- start start 2023-10-06T05:23:35.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa49d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4406d140 0x7ffa44157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa49d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa49d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4415acd0 0x7ffa440cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa49d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415f8c0 con 0x7ffa44159850 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa49d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415ef90 con 0x7ffa4415acd0 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa49d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415f270 con 0x7ffa4406d140 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4415fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56140/0 (socket says 172.21.15.33:56140) 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 -- 172.21.15.33:0/2338904933 learned_addr learned my addr 172.21.15.33:0/2338904933 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa48d9b700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4406d140 0x7ffa44157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa4959c700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4415acd0 0x7ffa440cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 -- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4406d140 msgr2=0x7ffa44157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4406d140 0x7ffa44157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 -- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4415acd0 msgr2=0x7ffa440cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4415acd0 0x7ffa440cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa43fff700 1 -- 172.21.15.33:0/2338904933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380094f0 con 0x7ffa44159850 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.185+0000 7ffa4959c700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4415acd0 0x7ffa440cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.186+0000 7ffa43fff700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4415fd50 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000c830 tx=0x7ffa30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.186+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011600 con 0x7ffa44159850 2023-10-06T05:23:35.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.186+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa30011760 con 0x7ffa44159850 2023-10-06T05:23:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.186+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011b20 con 0x7ffa44159850 2023-10-06T05:23:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.186+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4415f550 con 0x7ffa44159850 2023-10-06T05:23:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.186+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa44160290 con 0x7ffa44159850 2023-10-06T05:23:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.187+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4415fa30 con 0x7ffa44159850 2023-10-06T05:23:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.188+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa3000a3e0 con 0x7ffa44159850 2023-10-06T05:23:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.189+0000 7ffa40ff9700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa28066f30 0x7ffa280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.189+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa300847d0 con 0x7ffa44159850 2023-10-06T05:23:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.190+0000 7ffa48d9b700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa28066f30 0x7ffa280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.191+0000 7ffa48d9b700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa28066f30 0x7ffa280693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ffa4406a730 tx=0x7ffa38016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.193+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa30051810 con 0x7ffa44159850 2023-10-06T05:23:35.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.401+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa4404c380 con 0x7ffa44159850 2023-10-06T05:23:35.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.402+0000 7ffa40ff9700 1 -- 172.21.15.33:0/2338904933 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa30054e30 con 0x7ffa44159850 2023-10-06T05:23:35.404 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:23:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa28066f30 msgr2=0x7ffa280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa28066f30 0x7ffa280693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ffa4406a730 tx=0x7ffa38016000 comp rx=0 tx=0).stop 2023-10-06T05:23:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 msgr2=0x7ffa4415fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4415fd50 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000c830 tx=0x7ffa30008040 comp rx=0 tx=0).stop 2023-10-06T05:23:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 shutdown_connections 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa28066f30 0x7ffa280693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4415acd0 0x7ffa440cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4406d140 0x7ffa44157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 --2- 172.21.15.33:0/2338904933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa44159850 0x7ffa4415fd50 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 >> 172.21.15.33:0/2338904933 conn(0x7ffa4406cd40 msgr2=0x7ffa44157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 shutdown_connections 2023-10-06T05:23:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.404+0000 7ffa49d9d700 1 -- 172.21.15.33:0/2338904933 wait complete. 2023-10-06T05:23:35.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:23:35.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.612+0000 7fa407fff700 1 Processor -- start 2023-10-06T05:23:35.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.612+0000 7fa407fff700 1 -- start start 2023-10-06T05:23:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.613+0000 7fa407fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080ad460 0x7fa4080ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.613+0000 7fa407fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4080aa7d0 con 0x7fa4081595d0 2023-10-06T05:23:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.613+0000 7fa407fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4080a9bc0 con 0x7fa4080ac180 2023-10-06T05:23:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.613+0000 7fa407fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4080a9ea0 con 0x7fa4080ad460 2023-10-06T05:23:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa406ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa4081595d0 0x7fa4080aac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47810/0 (socket says 172.21.15.33:47810) 2023-10-06T05:23:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa406ffd700 1 -- 172.21.15.33:0/4282529628 learned_addr learned my addr 172.21.15.33:0/4282529628 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa4077fe700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080ad460 0x7fa4080ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731392332 0 0) 0x7fa4080aa7d0 con 0x7fa4081595d0 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f0003570 con 0x7fa4081595d0 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131935091 0 0) 0x7fa4080a9bc0 con 0x7fa4080ac180 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4080aa7d0 con 0x7fa4080ac180 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893053749 0 0) 0x7fa3f0003570 con 0x7fa4081595d0 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080ad460 msgr2=0x7fa4080ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.614+0000 7fa405ffb700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080ad460 0x7fa4080ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 >> v1:172.21.15.167:6789/0 conn(0x7fa4080ac180 legacy=0x7fa4080acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080aa180 con 0x7fa4081595d0 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa3f8001470 con 0x7fa4081595d0 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 >> v1:172.21.15.33:6789/0 conn(0x7fa4081595d0 legacy=0x7fa4080aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3f0003c20 0x7fa3f0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0004810 0x7fa3f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f00051c0 0x7fa3f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4080a9ea0 con 0x7fa3f0004810 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f000c130 con 0x7fa3f00051c0 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f000c2c0 con 0x7fa3f0003c20 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa4077fe700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080ad460 0x7fa4080ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa4067fc700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0004810 0x7fa3f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa406ffd700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3f0003c20 0x7fa3f0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa4077fe700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f00051c0 0x7fa3f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa406ffd700 1 -- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f00051c0 msgr2=0x7fa3f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa406ffd700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f00051c0 0x7fa3f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa406ffd700 1 -- 172.21.15.33:0/4282529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0004810 msgr2=0x7fa3f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa406ffd700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0004810 0x7fa3f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa406ffd700 1 -- 172.21.15.33:0/4282529628 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0003570 con 0x7fa3f0003c20 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.615+0000 7fa4067fc700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0004810 0x7fa3f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.616+0000 7fa406ffd700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3f0003c20 0x7fa3f0004010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c210 tx=0x7fa3f8001d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02b8bda152ab318 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.616+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800cc10 con 0x7fa3f0003c20 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.616+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f800cd70 con 0x7fa3f0003c20 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.616+0000 7fa405ffb700 1 -- 172.21.15.33:0/4282529628 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8005b70 con 0x7fa3f0003c20 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.616+0000 7fa407fff700 1 -- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3f0003c20 msgr2=0x7fa3f0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.616+0000 7fa407fff700 1 --2- 172.21.15.33:0/4282529628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3f0003c20 0x7fa3f0004010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c210 tx=0x7fa3f8001d40 comp rx=0 tx=0).stop 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.617+0000 7fa4077fe700 1 -- 172.21.15.33:0/4282529628 reap_dead start 2023-10-06T05:23:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.617+0000 7fa407fff700 1 -- 172.21.15.33:0/4282529628 shutdown_connections 2023-10-06T05:23:35.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.617+0000 7fa407fff700 1 -- 172.21.15.33:0/4282529628 >> 172.21.15.33:0/4282529628 conn(0x7fa40806d030 msgr2=0x7fa4080cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.617+0000 7fa407fff700 1 -- 172.21.15.33:0/4282529628 shutdown_connections 2023-10-06T05:23:35.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.617+0000 7fa407fff700 1 -- 172.21.15.33:0/4282529628 wait complete. 2023-10-06T05:23:35.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.618+0000 7fa407fff700 1 Processor -- start 2023-10-06T05:23:35.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.618+0000 7fa407fff700 1 -- start start 2023-10-06T05:23:35.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa407fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 0x7fa4081549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa407fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080cc550 0x7fa40815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa407fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40815c9f0 0x7fa4080c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa407fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40815d430 con 0x7fa3f0003c20 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa407fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40816a3a0 con 0x7fa40815c9f0 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa407fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40815ce70 con 0x7fa4080cc550 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa406ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 0x7fa4081549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa406ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 0x7fa4081549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56150/0 (socket says 172.21.15.33:56150) 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa406ffd700 1 -- 172.21.15.33:0/2904030335 learned_addr learned my addr 172.21.15.33:0/2904030335 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa4077fe700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40815c9f0 0x7fa4080c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.619+0000 7fa4067fc700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080cc550 0x7fa40815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa406ffd700 1 -- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080cc550 msgr2=0x7fa40815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa406ffd700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080cc550 0x7fa40815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa406ffd700 1 -- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40815c9f0 msgr2=0x7fa4080c17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa406ffd700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40815c9f0 0x7fa4080c17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa406ffd700 1 -- 172.21.15.33:0/2904030335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40815d150 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa4067fc700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080cc550 0x7fa40815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa406ffd700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 0x7fa4081549b0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c800 tx=0x7fa3f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800cee0 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8007a30 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8007550 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.620+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40816a030 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.621+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4080c1d00 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.621+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa408169e90 con 0x7fa3f0003c20 2023-10-06T05:23:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.622+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3f8019030 con 0x7fa3f0003c20 2023-10-06T05:23:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.623+0000 7fa3eeffd700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3f4066f30 0x7fa3f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.624+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3f8085c00 con 0x7fa3f0003c20 2023-10-06T05:23:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.624+0000 7fa4067fc700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3f4066f30 0x7fa3f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.625+0000 7fa4067fc700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3f4066f30 0x7fa3f40693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0010e0 tx=0x7fa3fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.628+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa408169e90 con 0x7fa3f0003c20 2023-10-06T05:23:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.828+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa408169e90 con 0x7fa3f0003c20 2023-10-06T05:23:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.829+0000 7fa3eeffd700 1 -- 172.21.15.33:0/2904030335 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa408169e90 con 0x7fa3f0003c20 2023-10-06T05:23:35.831 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3f4066f30 msgr2=0x7fa3f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3f4066f30 0x7fa3f40693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0010e0 tx=0x7fa3fc009040 comp rx=0 tx=0).stop 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 msgr2=0x7fa4081549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 0x7fa4081549b0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c800 tx=0x7fa3f8015000 comp rx=0 tx=0).stop 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 shutdown_connections 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3f4066f30 0x7fa3f40693e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa40815c9f0 0x7fa4080c17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f0003c20 0x7fa4081549b0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 --2- 172.21.15.33:0/2904030335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4080cc550 0x7fa40815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 >> 172.21.15.33:0/2904030335 conn(0x7fa40806d030 msgr2=0x7fa4080b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 shutdown_connections 2023-10-06T05:23:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:35.831+0000 7fa407fff700 1 -- 172.21.15.33:0/2904030335 wait complete. 2023-10-06T05:23:35.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:23:36.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.041+0000 7fb53dd9d700 1 Processor -- start 2023-10-06T05:23:36.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.041+0000 7fb53dd9d700 1 -- start start 2023-10-06T05:23:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb538155a50 0x7fb538155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5380afb20 0x7fb5380caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53dd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5380b06b0 con 0x7fb5380c9490 2023-10-06T05:23:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380c99e0 con 0x7fb5380afb20 2023-10-06T05:23:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380c9cc0 con 0x7fb538155a50 2023-10-06T05:23:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5380afb20 0x7fb5380caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5380afb20 0x7fb5380caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56160/0 (socket says 172.21.15.33:56160) 2023-10-06T05:23:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53d59c700 1 -- 172.21.15.33:0/3634052288 learned_addr learned my addr 172.21.15.33:0/3634052288 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53d59c700 1 -- 172.21.15.33:0/3634052288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb538155a50 msgr2=0x7fb538155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.043+0000 7fb53cd9b700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb538155a50 0x7fb538155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb53d59c700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb538155a50 0x7fb538155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb53d59c700 1 -- 172.21.15.33:0/3634052288 >> v1:172.21.15.167:6789/0 conn(0x7fb5380c9490 legacy=0x7fb5380af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb53d59c700 1 -- 172.21.15.33:0/3634052288 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380c9fa0 con 0x7fb5380afb20 2023-10-06T05:23:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb53cd9b700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb538155a50 0x7fb538155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb53d59c700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb5380caa80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb530009f50 tx=0x7fb530009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba0b19ad75657db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb5377fe700 1 -- 172.21.15.33:0/3634052288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53000e070 con 0x7fb5380afb20 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb5377fe700 1 -- 172.21.15.33:0/3634052288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53000ba70 con 0x7fb5380afb20 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.044+0000 7fb5377fe700 1 -- 172.21.15.33:0/3634052288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300092e0 con 0x7fb5380afb20 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 -- 172.21.15.33:0/3634052288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 msgr2=0x7fb5380caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb5380caa80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb530009f50 tx=0x7fb530009840 comp rx=0 tx=0).stop 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 -- 172.21.15.33:0/3634052288 shutdown_connections 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb5380caa80 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/3634052288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb538155a50 0x7fb538155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 -- 172.21.15.33:0/3634052288 >> 172.21.15.33:0/3634052288 conn(0x7fb53806d030 msgr2=0x7fb5380cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 -- 172.21.15.33:0/3634052288 shutdown_connections 2023-10-06T05:23:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.045+0000 7fb53dd9d700 1 -- 172.21.15.33:0/3634052288 wait complete. 2023-10-06T05:23:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.046+0000 7fb53dd9d700 1 Processor -- start 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.046+0000 7fb53dd9d700 1 -- start start 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.047+0000 7fb53dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb53814b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.047+0000 7fb53dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5380c9490 0x7fb538149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.047+0000 7fb53dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb538155a50 0x7fb53814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.047+0000 7fb53dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53814c590 con 0x7fb5380afb20 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.047+0000 7fb53dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53814b980 con 0x7fb538155a50 2023-10-06T05:23:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.047+0000 7fb53dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53814bc60 con 0x7fb5380c9490 2023-10-06T05:23:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb53814b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb53814b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56164/0 (socket says 172.21.15.33:56164) 2023-10-06T05:23:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 -- 172.21.15.33:0/4086650873 learned_addr learned my addr 172.21.15.33:0/4086650873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb537fff700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5380c9490 0x7fb538149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53d59c700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb538155a50 0x7fb53814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 -- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5380c9490 msgr2=0x7fb538149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5380c9490 0x7fb538149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 -- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb538155a50 msgr2=0x7fb53814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb538155a50 0x7fb53814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 -- 172.21.15.33:0/4086650873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530009ae0 con 0x7fb5380afb20 2023-10-06T05:23:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.048+0000 7fb53cd9b700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb53814b100 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00c830 tx=0x7fb52c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.049+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c011600 con 0x7fb5380afb20 2023-10-06T05:23:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.049+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c011760 con 0x7fb5380afb20 2023-10-06T05:23:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.049+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c011b20 con 0x7fb5380afb20 2023-10-06T05:23:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.049+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53814bf40 con 0x7fb5380afb20 2023-10-06T05:23:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.049+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53814b640 con 0x7fb5380afb20 2023-10-06T05:23:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.050+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53804c380 con 0x7fb5380afb20 2023-10-06T05:23:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.051+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb52c018360 con 0x7fb5380afb20 2023-10-06T05:23:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.052+0000 7fb534ff9700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb51c066f30 0x7fb51c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.052+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb52c0846f0 con 0x7fb5380afb20 2023-10-06T05:23:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.052+0000 7fb537fff700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb51c066f30 0x7fb51c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.053+0000 7fb537fff700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb51c066f30 0x7fb51c0693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb5381561e0 tx=0x7fb524007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.056+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb52c051730 con 0x7fb5380afb20 2023-10-06T05:23:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.260+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb53806a630 con 0x7fb5380afb20 2023-10-06T05:23:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.261+0000 7fb534ff9700 1 -- 172.21.15.33:0/4086650873 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb52c054d50 con 0x7fb5380afb20 2023-10-06T05:23:36.262 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:23:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb51c066f30 msgr2=0x7fb51c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb51c066f30 0x7fb51c0693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb5381561e0 tx=0x7fb524007ec0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 msgr2=0x7fb53814b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb53814b100 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00c830 tx=0x7fb52c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 shutdown_connections 2023-10-06T05:23:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb51c066f30 0x7fb51c0693e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb538155a50 0x7fb53814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5380afb20 0x7fb53814b100 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 --2- 172.21.15.33:0/4086650873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5380c9490 0x7fb538149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 >> 172.21.15.33:0/4086650873 conn(0x7fb53806d030 msgr2=0x7fb53806ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 shutdown_connections 2023-10-06T05:23:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.263+0000 7fb53dd9d700 1 -- 172.21.15.33:0/4086650873 wait complete. 2023-10-06T05:23:36.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:36.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.466+0000 7fb4c339d700 1 Processor -- start 2023-10-06T05:23:36.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.467+0000 7fb4c339d700 1 -- start start 2023-10-06T05:23:36.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc13faa0 0x7fb4bc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c339d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4bc0c7940 0x7fb4bc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c339d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0c9cb0 con 0x7fb4bc06aad0 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c339d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc06b660 con 0x7fb4bc0c7940 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0ca270 con 0x7fb4bc13faa0 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c1b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4bc0c7940 0x7fb4bc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c2b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb4bc06aad0 0x7fb4bc06af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47814/0 (socket says 172.21.15.33:47814) 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c2b9c700 1 -- 172.21.15.33:0/2401775217 learned_addr learned my addr 172.21.15.33:0/2401775217 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.468+0000 7fb4c239b700 1 --2- 172.21.15.33:0/2401775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc13faa0 0x7fb4bc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1b9a700 1 -- 172.21.15.33:0/2401775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc13faa0 msgr2=0x7fb4bc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1b9a700 1 --2- 172.21.15.33:0/2401775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc13faa0 0x7fb4bc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1b9a700 1 -- 172.21.15.33:0/2401775217 >> v1:172.21.15.33:6789/0 conn(0x7fb4bc06aad0 legacy=0x7fb4bc06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1b9a700 1 -- 172.21.15.33:0/2401775217 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc06b840 con 0x7fb4bc0c7940 2023-10-06T05:23:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1b9a700 1 --2- 172.21.15.33:0/2401775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0c7940 0x7fb4bc0c7db0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0098e0 tx=0x7fb4ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62b9b78392d0682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1399700 1 -- 172.21.15.33:0/2401775217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00b640 con 0x7fb4bc0c7940 2023-10-06T05:23:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1399700 1 -- 172.21.15.33:0/2401775217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac00b7a0 con 0x7fb4bc0c7940 2023-10-06T05:23:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.469+0000 7fb4c1399700 1 -- 172.21.15.33:0/2401775217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00bb40 con 0x7fb4bc0c7940 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 -- 172.21.15.33:0/2401775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0c7940 msgr2=0x7fb4bc0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2401775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0c7940 0x7fb4bc0c7db0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0098e0 tx=0x7fb4ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 -- 172.21.15.33:0/2401775217 shutdown_connections 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2401775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc13faa0 0x7fb4bc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2401775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0c7940 0x7fb4bc0c7db0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 -- 172.21.15.33:0/2401775217 >> 172.21.15.33:0/2401775217 conn(0x7fb4bc06d030 msgr2=0x7fb4bc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 -- 172.21.15.33:0/2401775217 shutdown_connections 2023-10-06T05:23:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.470+0000 7fb4c339d700 1 -- 172.21.15.33:0/2401775217 wait complete. 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.471+0000 7fb4c339d700 1 Processor -- start 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.471+0000 7fb4c339d700 1 -- start start 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 0x7fb4bc15fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0c7940 0x7fb4bc13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc13faa0 0x7fb4bc13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc1608c0 con 0x7fb4bc06aad0 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc13c720 con 0x7fb4bc13faa0 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc15da70 con 0x7fb4bc0c7940 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.472+0000 7fb4c239b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 0x7fb4bc15fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 0x7fb4bc15fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56180/0 (socket says 172.21.15.33:56180) 2023-10-06T05:23:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 -- 172.21.15.33:0/2941348884 learned_addr learned my addr 172.21.15.33:0/2941348884 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c2b9c700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc13faa0 0x7fb4bc13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c1b9a700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0c7940 0x7fb4bc13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 -- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0c7940 msgr2=0x7fb4bc13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0c7940 0x7fb4bc13c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 -- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc13faa0 msgr2=0x7fb4bc13cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc13faa0 0x7fb4bc13cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 -- 172.21.15.33:0/2941348884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac009420 con 0x7fb4bc06aad0 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c1b9a700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0c7940 0x7fb4bc13c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4c239b700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 0x7fb4bc15fc70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800e930 tx=0x7fb4b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.473+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800eed0 con 0x7fb4bc06aad0 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.474+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b800c7f0 con 0x7fb4bc06aad0 2023-10-06T05:23:36.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.474+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800cb90 con 0x7fb4bc06aad0 2023-10-06T05:23:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.474+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc160300 con 0x7fb4bc06aad0 2023-10-06T05:23:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.474+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc15d700 con 0x7fb4bc06aad0 2023-10-06T05:23:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.475+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc15d560 con 0x7fb4bc06aad0 2023-10-06T05:23:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.476+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4b8013470 con 0x7fb4bc06aad0 2023-10-06T05:23:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.476+0000 7fb4b27fc700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a0066f60 0x7fb4a0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.477+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4b80848d0 con 0x7fb4bc06aad0 2023-10-06T05:23:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.477+0000 7fb4c1b9a700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a0066f60 0x7fb4a0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.478+0000 7fb4c1b9a700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a0066f60 0x7fb4a0069410 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc06b600 tx=0x7fb4ac017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.481+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4bc15d560 con 0x7fb4bc06aad0 2023-10-06T05:23:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.727+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4bc15d560 con 0x7fb4bc06aad0 2023-10-06T05:23:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.728+0000 7fb4b27fc700 1 -- 172.21.15.33:0/2941348884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fb4bc15d560 con 0x7fb4bc06aad0 2023-10-06T05:23:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:36.730 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:36.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.730+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a0066f60 msgr2=0x7fb4a0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.730+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a0066f60 0x7fb4a0069410 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc06b600 tx=0x7fb4ac017000 comp rx=0 tx=0).stop 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.730+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 msgr2=0x7fb4bc15fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.730+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 0x7fb4bc15fc70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800e930 tx=0x7fb4b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 shutdown_connections 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a0066f60 0x7fb4a0069410 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc13faa0 0x7fb4bc13cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc06aad0 0x7fb4bc15fc70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 --2- 172.21.15.33:0/2941348884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0c7940 0x7fb4bc13c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 >> 172.21.15.33:0/2941348884 conn(0x7fb4bc06d030 msgr2=0x7fb4bc0c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 shutdown_connections 2023-10-06T05:23:36.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.731+0000 7fb4c339d700 1 -- 172.21.15.33:0/2941348884 wait complete. 2023-10-06T05:23:36.734 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:36.750 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.938+0000 7feb9a39d700 1 Processor -- start 2023-10-06T05:23:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.938+0000 7feb9a39d700 1 -- start start 2023-10-06T05:23:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.939+0000 7feb9a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 0x7feb940b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.939+0000 7feb9a39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940cbd80 con 0x7feb940ccfc0 2023-10-06T05:23:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.939+0000 7feb9a39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940af5f0 con 0x7feb940cf3b0 2023-10-06T05:23:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.939+0000 7feb9a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb940af8d0 con 0x7feb94159bb0 2023-10-06T05:23:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.939+0000 7feb9939b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7feb940ccfc0 0x7feb940cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47818/0 (socket says 172.21.15.33:47818) 2023-10-06T05:23:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb9939b700 1 -- 172.21.15.33:0/3824759946 learned_addr learned my addr 172.21.15.33:0/3824759946 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366854536 0 0) 0x7feb940cbd80 con 0x7feb940ccfc0 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb80003570 con 0x7feb940ccfc0 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb99b9c700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 0x7feb940b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3896579498 0 0) 0x7feb80003570 con 0x7feb940ccfc0 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 msgr2=0x7feb940b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 0x7feb940b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 >> v1:172.21.15.167:6789/0 conn(0x7feb940cf3b0 legacy=0x7feb941594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940afbb0 con 0x7feb940ccfc0 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7feb90001470 con 0x7feb940ccfc0 2023-10-06T05:23:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 >> v1:172.21.15.33:6789/0 conn(0x7feb940ccfc0 legacy=0x7feb940cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 0x7feb80003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb80004110 0x7feb80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb800052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940af8d0 con 0x7feb80004e70 2023-10-06T05:23:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8000bcc0 con 0x7feb80004110 2023-10-06T05:23:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.940+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8000be80 con 0x7feb94159bb0 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb800052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb9939b700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 0x7feb80003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb98b9a700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb80004110 0x7feb80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 -- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 msgr2=0x7feb80003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb94159bb0 0x7feb80003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 -- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb80004110 msgr2=0x7feb80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb80004110 0x7feb80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 -- 172.21.15.33:0/3824759946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb80003570 con 0x7feb80004e70 2023-10-06T05:23:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb99b9c700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb800052e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7feb8c007440 tx=0x7feb8c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14edb80cb1e631e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.941+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c009a40 con 0x7feb80004e70 2023-10-06T05:23:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c007cf0 con 0x7feb80004e70 2023-10-06T05:23:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb8bfff700 1 -- 172.21.15.33:0/3824759946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c014630 con 0x7feb80004e70 2023-10-06T05:23:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb9a39d700 1 -- 172.21.15.33:0/3824759946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 msgr2=0x7feb800052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb9a39d700 1 --2- 172.21.15.33:0/3824759946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb800052e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7feb8c007440 tx=0x7feb8c009710 comp rx=0 tx=0).stop 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb99b9c700 1 -- 172.21.15.33:0/3824759946 reap_dead start 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb9a39d700 1 -- 172.21.15.33:0/3824759946 shutdown_connections 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb9a39d700 1 -- 172.21.15.33:0/3824759946 >> 172.21.15.33:0/3824759946 conn(0x7feb9406cff0 msgr2=0x7feb9406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb9a39d700 1 -- 172.21.15.33:0/3824759946 shutdown_connections 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.942+0000 7feb9a39d700 1 -- 172.21.15.33:0/3824759946 wait complete. 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.944+0000 7feb9a39d700 1 Processor -- start 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.944+0000 7feb9a39d700 1 -- start start 2023-10-06T05:23:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb94154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb940c9df0 0x7feb940b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb940b29b0 0x7feb94169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940b43d0 con 0x7feb80004e70 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940b58b0 con 0x7feb940c9df0 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb940b2e20 con 0x7feb940b29b0 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb94154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb94154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56186/0 (socket says 172.21.15.33:56186) 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 -- 172.21.15.33:0/4256115141 learned_addr learned my addr 172.21.15.33:0/4256115141 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb98b9a700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb940c9df0 0x7feb940b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb99b9c700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb940b29b0 0x7feb94169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 -- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb940b29b0 msgr2=0x7feb94169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb940b29b0 0x7feb94169aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 -- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb940c9df0 msgr2=0x7feb940b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb940c9df0 0x7feb940b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.945+0000 7feb9939b700 1 -- 172.21.15.33:0/4256115141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940b5e30 con 0x7feb80004e70 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.946+0000 7feb9939b700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb94154000 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7feb90001050 tx=0x7feb90001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.946+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000f8d0 con 0x7feb80004e70 2023-10-06T05:23:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.946+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9000fa30 con 0x7feb80004e70 2023-10-06T05:23:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.946+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000fdb0 con 0x7feb80004e70 2023-10-06T05:23:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.946+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb940b3e10 con 0x7feb80004e70 2023-10-06T05:23:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.946+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb940b5ac0 con 0x7feb80004e70 2023-10-06T05:23:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.947+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb940b5c50 con 0x7feb80004e70 2023-10-06T05:23:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.948+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb90013600 con 0x7feb80004e70 2023-10-06T05:23:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.949+0000 7feb897fa700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb78066f30 0x7feb780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.949+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb900848a0 con 0x7feb80004e70 2023-10-06T05:23:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.949+0000 7feb98b9a700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb78066f30 0x7feb780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.950+0000 7feb98b9a700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb78066f30 0x7feb780693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feb94153fb0 tx=0x7feb84007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:36.953+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb90051980 con 0x7feb80004e70 2023-10-06T05:23:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.158+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb940b5fc0 con 0x7feb80004e70 2023-10-06T05:23:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.159+0000 7feb897fa700 1 -- 172.21.15.33:0/4256115141 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16939 (secure 0 0 0) 0x7feb90054fa0 con 0x7feb80004e70 2023-10-06T05:23:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:37.162 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":118,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:31.012006+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:04.306347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:05.660589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/588756950":"2023-10-06T06:23:30.410369+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb78066f30 msgr2=0x7feb780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb78066f30 0x7feb780693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feb94153fb0 tx=0x7feb84007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 msgr2=0x7feb94154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb94154000 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7feb90001050 tx=0x7feb90001d80 comp rx=0 tx=0).stop 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 shutdown_connections 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb78066f30 0x7feb780693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb940b29b0 0x7feb94169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb80004e70 0x7feb94154000 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 --2- 172.21.15.33:0/4256115141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb940c9df0 0x7feb940b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 >> 172.21.15.33:0/4256115141 conn(0x7feb9406cff0 msgr2=0x7feb940cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.161+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 shutdown_connections 2023-10-06T05:23:37.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.162+0000 7feb9a39d700 1 -- 172.21.15.33:0/4256115141 wait complete. 2023-10-06T05:23:37.181 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:23:37.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.366+0000 7fbe7c99d700 1 Processor -- start 2023-10-06T05:23:37.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.367+0000 7fbe7c99d700 1 -- start start 2023-10-06T05:23:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe7c99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe780ccfc0 0x7fbe780cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe7c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780cf3b0 0x7fbe781594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe7c99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780cbd80 con 0x7fbe780cc750 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe7c99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780af5f0 con 0x7fbe780ccfc0 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe7c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780af8d0 con 0x7fbe780cf3b0 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe77fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbe780cc750 0x7fbe780b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47832/0 (socket says 172.21.15.33:47832) 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe77fff700 1 -- 172.21.15.33:0/105528388 learned_addr learned my addr 172.21.15.33:0/105528388 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe777fe700 1 --2- 172.21.15.33:0/105528388 >> v2:172.21.15.167:3300/0 conn(0x7fbe780ccfc0 0x7fbe780cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.368+0000 7fbe76ffd700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780cf3b0 0x7fbe781594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951479734 0 0) 0x7fbe780cbd80 con 0x7fbe780cc750 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe60003570 con 0x7fbe780cc750 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1839322122 0 0) 0x7fbe60003570 con 0x7fbe780cc750 2023-10-06T05:23:37.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780cf3b0 msgr2=0x7fbe781594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780cf3b0 0x7fbe781594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 >> v2:172.21.15.167:3300/0 conn(0x7fbe780ccfc0 msgr2=0x7fbe780cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 --2- 172.21.15.33:0/105528388 >> v2:172.21.15.167:3300/0 conn(0x7fbe780ccfc0 0x7fbe780cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780afbb0 con 0x7fbe780cc750 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fbe700017f0 con 0x7fbe780cc750 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 >> v1:172.21.15.33:6789/0 conn(0x7fbe780cc750 legacy=0x7fbe780b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe60003ae0 0x7fbe60003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60004810 0x7fbe60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe600051c0 0x7fbe6000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780af5f0 con 0x7fbe60003ae0 2023-10-06T05:23:37.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780af8d0 con 0x7fbe60004810 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6000bfa0 con 0x7fbe600051c0 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.369+0000 7fbe777fe700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe60003ae0 0x7fbe60003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe77fff700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe600051c0 0x7fbe6000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe76ffd700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60004810 0x7fbe60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe777fe700 1 -- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe600051c0 msgr2=0x7fbe6000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe777fe700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe600051c0 0x7fbe6000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe777fe700 1 -- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60004810 msgr2=0x7fbe60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe777fe700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60004810 0x7fbe60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe777fe700 1 -- 172.21.15.33:0/105528388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe60003570 con 0x7fbe60003ae0 2023-10-06T05:23:37.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe777fe700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe60003ae0 0x7fbe60003ef0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009c00 tx=0x7fbe680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd4bd30a8508a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800e710 con 0x7fbe60003ae0 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6800e870 con 0x7fbe60003ae0 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.370+0000 7fbe767fc700 1 -- 172.21.15.33:0/105528388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800ec30 con 0x7fbe60003ae0 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe7c99d700 1 -- 172.21.15.33:0/105528388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe60003ae0 msgr2=0x7fbe60003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/105528388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe60003ae0 0x7fbe60003ef0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009c00 tx=0x7fbe680091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe77fff700 1 -- 172.21.15.33:0/105528388 reap_dead start 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe7c99d700 1 -- 172.21.15.33:0/105528388 shutdown_connections 2023-10-06T05:23:37.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe7c99d700 1 -- 172.21.15.33:0/105528388 >> 172.21.15.33:0/105528388 conn(0x7fbe7806cff0 msgr2=0x7fbe7806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe7c99d700 1 -- 172.21.15.33:0/105528388 shutdown_connections 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.371+0000 7fbe7c99d700 1 -- 172.21.15.33:0/105528388 wait complete. 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.372+0000 7fbe7c99d700 1 Processor -- start 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.373+0000 7fbe7c99d700 1 -- start start 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.373+0000 7fbe7c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60003ae0 0x7fbe781502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.373+0000 7fbe7c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780c7f90 0x7fbe78169040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.373+0000 7fbe7c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 0x7fbe780c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe7c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780c75b0 con 0x7fbe780c8400 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe7c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780c7b00 con 0x7fbe60003ae0 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe7c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78169b30 con 0x7fbe780c7f90 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 0x7fbe780c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 0x7fbe780c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56206/0 (socket says 172.21.15.33:56206) 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 -- 172.21.15.33:0/127580420 learned_addr learned my addr 172.21.15.33:0/127580420 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe777fe700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60003ae0 0x7fbe781502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 -- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780c7f90 msgr2=0x7fbe78169040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780c7f90 0x7fbe78169040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 -- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60003ae0 msgr2=0x7fbe781502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60003ae0 0x7fbe781502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 -- 172.21.15.33:0/127580420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78169e10 con 0x7fbe780c8400 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe777fe700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60003ae0 0x7fbe781502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.374+0000 7fbe77fff700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 0x7fbe780c64a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbe700013d0 tx=0x7fbe70002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.375+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000ad20 con 0x7fbe780c8400 2023-10-06T05:23:37.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.375+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe700133f0 con 0x7fbe780c8400 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.375+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000aea0 con 0x7fbe780c8400 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.375+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe780c7790 con 0x7fbe780c8400 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.375+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe78153cf0 con 0x7fbe780c8400 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.376+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78071060 con 0x7fbe780c8400 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.377+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe700139c0 con 0x7fbe780c8400 2023-10-06T05:23:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.377+0000 7fbe5f7fe700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe64066f30 0x7fbe640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.378+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe70085260 con 0x7fbe780c8400 2023-10-06T05:23:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.378+0000 7fbe777fe700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe64066f30 0x7fbe640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.379+0000 7fbe777fe700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe64066f30 0x7fbe640693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbe680091e0 tx=0x7fbe6801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:37.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.381+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe78071060 con 0x7fbe780c8400 2023-10-06T05:23:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.627+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe78071060 con 0x7fbe780c8400 2023-10-06T05:23:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.628+0000 7fbe5f7fe700 1 -- 172.21.15.33:0/127580420 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fbe78071060 con 0x7fbe780c8400 2023-10-06T05:23:37.629 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:37.629 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:37.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe64066f30 msgr2=0x7fbe640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe64066f30 0x7fbe640693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbe680091e0 tx=0x7fbe6801a000 comp rx=0 tx=0).stop 2023-10-06T05:23:37.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 msgr2=0x7fbe780c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 0x7fbe780c64a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbe700013d0 tx=0x7fbe70002140 comp rx=0 tx=0).stop 2023-10-06T05:23:37.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 shutdown_connections 2023-10-06T05:23:37.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbe64066f30 0x7fbe640693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe780c8400 0x7fbe780c64a0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe60003ae0 0x7fbe781502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 --2- 172.21.15.33:0/127580420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe780c7f90 0x7fbe78169040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 >> 172.21.15.33:0/127580420 conn(0x7fbe7806cff0 msgr2=0x7fbe78156c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:37.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 shutdown_connections 2023-10-06T05:23:37.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.630+0000 7fbe7c99d700 1 -- 172.21.15.33:0/127580420 wait complete. 2023-10-06T05:23:37.634 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:37.650 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.836+0000 7f795759e700 1 Processor -- start 2023-10-06T05:23:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.836+0000 7f795759e700 1 -- start start 2023-10-06T05:23:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79580b1840 0x7f7958156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580c8960 con 0x7f79580ac770 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79580c8680 con 0x7f79580b1840 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79581560d0 con 0x7f79580acb40 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f7955d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795659c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f79580ac770 0x7f79580c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:43720/0 (socket says 172.21.15.33:43720) 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f795659c700 1 -- 172.21.15.33:0/2444952822 learned_addr learned my addr 172.21.15.33:0/2444952822 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.838+0000 7f7956d9d700 1 --2- 172.21.15.33:0/2444952822 >> v2:172.21.15.167:3300/0 conn(0x7f79580b1840 0x7f7958156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f795559a700 1 -- 172.21.15.33:0/2444952822 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262194578 0 0) 0x7f79580c8960 con 0x7f79580ac770 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f795559a700 1 -- 172.21.15.33:0/2444952822 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7940003570 con 0x7f79580ac770 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f7955d9b700 1 -- 172.21.15.33:0/2444952822 >> v2:172.21.15.167:3300/0 conn(0x7f79580b1840 msgr2=0x7f7958156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f7955d9b700 1 --2- 172.21.15.33:0/2444952822 >> v2:172.21.15.167:3300/0 conn(0x7f79580b1840 0x7f7958156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f7955d9b700 1 -- 172.21.15.33:0/2444952822 >> v1:172.21.15.167:6789/0 conn(0x7f79580ac770 legacy=0x7f79580c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f7955d9b700 1 -- 172.21.15.33:0/2444952822 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79581563b0 con 0x7f79580acb40 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f7955d9b700 1 --2- 172.21.15.33:0/2444952822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b1300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098e0 tx=0x7f794c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cce7d4fbdec5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f795559a700 1 -- 172.21.15.33:0/2444952822 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c011630 con 0x7f79580acb40 2023-10-06T05:23:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f795559a700 1 -- 172.21.15.33:0/2444952822 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c011790 con 0x7f79580acb40 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.839+0000 7f795559a700 1 -- 172.21.15.33:0/2444952822 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c011b30 con 0x7f79580acb40 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 -- 172.21.15.33:0/2444952822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 msgr2=0x7f79580b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 --2- 172.21.15.33:0/2444952822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b1300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098e0 tx=0x7f794c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 -- 172.21.15.33:0/2444952822 shutdown_connections 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 --2- 172.21.15.33:0/2444952822 >> v2:172.21.15.167:3300/0 conn(0x7f79580b1840 0x7f7958156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 --2- 172.21.15.33:0/2444952822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b1300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 -- 172.21.15.33:0/2444952822 >> 172.21.15.33:0/2444952822 conn(0x7f795806c560 msgr2=0x7f795806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 -- 172.21.15.33:0/2444952822 shutdown_connections 2023-10-06T05:23:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.840+0000 7f795759e700 1 -- 172.21.15.33:0/2444952822 wait complete. 2023-10-06T05:23:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.841+0000 7f795759e700 1 Processor -- start 2023-10-06T05:23:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.841+0000 7f795759e700 1 -- start start 2023-10-06T05:23:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.842+0000 7f795759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 0x7f795814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.842+0000 7f795759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.842+0000 7f795759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79580b1840 0x7f7958146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.842+0000 7f795759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795814c610 con 0x7f79580ac770 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.842+0000 7f795759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795814e940 con 0x7f79580b1840 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.842+0000 7f795759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795814ec20 con 0x7f79580acb40 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 0x7f795814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 0x7f795814b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56214/0 (socket says 172.21.15.33:56214) 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 -- 172.21.15.33:0/2712083958 learned_addr learned my addr 172.21.15.33:0/2712083958 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f7956d9d700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79580b1840 0x7f7958146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f7955d9b700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 -- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 msgr2=0x7f79580b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 -- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79580b1840 msgr2=0x7f7958146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79580b1840 0x7f7958146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 -- 172.21.15.33:0/2712083958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c009420 con 0x7f79580ac770 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f7956d9d700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79580b1840 0x7f7958146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f7955d9b700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.843+0000 7f795659c700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 0x7f795814b2e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7c0 tx=0x7f7948007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.844+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948005b30 con 0x7f79580ac770 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.844+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7948005c90 con 0x7f79580ac770 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.844+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948007340 con 0x7f79580ac770 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.844+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795814ef00 con 0x7f79580ac770 2023-10-06T05:23:37.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.844+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795814b820 con 0x7f79580ac770 2023-10-06T05:23:37.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.845+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795814f4f0 con 0x7f79580ac770 2023-10-06T05:23:37.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.846+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f794801b400 con 0x7f79580ac770 2023-10-06T05:23:37.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.847+0000 7f79467fc700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7938066f30 0x7f79380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:37.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.847+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7948084270 con 0x7f79580ac770 2023-10-06T05:23:37.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.847+0000 7f7955d9b700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7938066f30 0x7f79380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:37.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.848+0000 7f7955d9b700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7938066f30 0x7f79380693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f794c0091e0 tx=0x7f794c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:37.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:37.851+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7948051330 con 0x7f79580ac770 2023-10-06T05:23:38.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.099+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f795804c380 con 0x7f79580ac770 2023-10-06T05:23:38.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.099+0000 7f79467fc700 1 -- 172.21.15.33:0/2712083958 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f7948054950 con 0x7f79580ac770 2023-10-06T05:23:38.101 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:38.101 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:06.683552+0000","modified":"2023-10-06T05:23:11.203927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/723044599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":723044599},{"type":"v1","addr":"172.21.15.33:6834","nonce":723044599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7938066f30 msgr2=0x7f79380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7938066f30 0x7f79380693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f794c0091e0 tx=0x7f794c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 msgr2=0x7f795814b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 0x7f795814b2e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7c0 tx=0x7f7948007e00 comp rx=0 tx=0).stop 2023-10-06T05:23:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 shutdown_connections 2023-10-06T05:23:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7938066f30 0x7f79380693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79580b1840 0x7f7958146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79580ac770 0x7f795814b2e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 --2- 172.21.15.33:0/2712083958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79580acb40 0x7f79580b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 >> 172.21.15.33:0/2712083958 conn(0x7f795806c560 msgr2=0x7f795806a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 shutdown_connections 2023-10-06T05:23:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.102+0000 7f795759e700 1 -- 172.21.15.33:0/2712083958 wait complete. 2023-10-06T05:23:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2023-10-06T05:23:38.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:38.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.308+0000 7f767b39d700 1 Processor -- start 2023-10-06T05:23:38.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.308+0000 7f767b39d700 1 -- start start 2023-10-06T05:23:38.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767b39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f76740c7920 0x7f7674160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767406ab60 con 0x7f7674155af0 2023-10-06T05:23:38.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767b39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767406ae40 con 0x7f76740c8b50 2023-10-06T05:23:38.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767b39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767406b7e0 con 0x7f76740c7920 2023-10-06T05:23:38.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767a39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7674155af0 0x7f767406a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47836/0 (socket says 172.21.15.33:47836) 2023-10-06T05:23:38.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767a39b700 1 -- 172.21.15.33:0/2683491776 learned_addr learned my addr 172.21.15.33:0/2683491776 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f767ab9c700 1 --2- 172.21.15.33:0/2683491776 >> v2:172.21.15.167:3300/0 conn(0x7f76740c7920 0x7f7674160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235894905 0 0) 0x7f767406ab60 con 0x7f7674155af0 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.310+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7660003570 con 0x7f7674155af0 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808991527 0 0) 0x7f767406ae40 con 0x7f76740c8b50 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f767406ab60 con 0x7f76740c8b50 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2100424552 0 0) 0x7f7660003570 con 0x7f7674155af0 2023-10-06T05:23:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 >> v2:172.21.15.167:3300/0 conn(0x7f76740c7920 msgr2=0x7f7674160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 --2- 172.21.15.33:0/2683491776 >> v2:172.21.15.167:3300/0 conn(0x7f76740c7920 0x7f7674160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 >> v1:172.21.15.167:6789/0 conn(0x7f76740c8b50 legacy=0x7f76740c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767406b120 con 0x7f7674155af0 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f7670001470 con 0x7f7674155af0 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 >> v1:172.21.15.33:6789/0 conn(0x7f7674155af0 legacy=0x7f767406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660003c40 0x7f7660004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660004810 0x7f7660004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7660005210 0x7f766000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767406b7e0 con 0x7f7660004810 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766000c1b0 con 0x7f7660003c40 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766000c370 con 0x7f7660005210 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f7679b9a700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660004810 0x7f7660004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f767a39b700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660003c40 0x7f7660004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.311+0000 7f767ab9c700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7660005210 0x7f766000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679b9a700 1 -- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7660005210 msgr2=0x7f766000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679b9a700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7660005210 0x7f766000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679b9a700 1 -- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660003c40 msgr2=0x7f7660004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679b9a700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660003c40 0x7f7660004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679b9a700 1 -- 172.21.15.33:0/2683491776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660003570 con 0x7f7660004810 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f767ab9c700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7660005210 0x7f766000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f767ab9c700 1 -- 172.21.15.33:0/2683491776 reap_dead start 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679b9a700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660004810 0x7f7660004c80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f76640019e0 tx=0x7f7664007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39dad14bb28de91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400c5d0 con 0x7f7660004810 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400c730 con 0x7f7660004810 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.312+0000 7f7679399700 1 -- 172.21.15.33:0/2683491776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400caf0 con 0x7f7660004810 2023-10-06T05:23:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 -- 172.21.15.33:0/2683491776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660004810 msgr2=0x7f7660004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660004810 0x7f7660004c80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f76640019e0 tx=0x7f7664007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 -- 172.21.15.33:0/2683491776 shutdown_connections 2023-10-06T05:23:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 --2- 172.21.15.33:0/2683491776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660004810 0x7f7660004c80 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 -- 172.21.15.33:0/2683491776 >> 172.21.15.33:0/2683491776 conn(0x7f767406d030 msgr2=0x7f767406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 -- 172.21.15.33:0/2683491776 shutdown_connections 2023-10-06T05:23:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.313+0000 7f767b39d700 1 -- 172.21.15.33:0/2683491776 wait complete. 2023-10-06T05:23:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.314+0000 7f767b39d700 1 Processor -- start 2023-10-06T05:23:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.315+0000 7f767b39d700 1 -- start start 2023-10-06T05:23:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.315+0000 7f767b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660004810 0x7f7674169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.315+0000 7f767b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f767414b9b0 0x7f76740c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.315+0000 7f767b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 0x7f767414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.315+0000 7f767b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76740ad340 con 0x7f76740c4f60 2023-10-06T05:23:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.315+0000 7f767b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76740c5500 con 0x7f7660004810 2023-10-06T05:23:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76740c57e0 con 0x7f767414b9b0 2023-10-06T05:23:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 0x7f767414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 0x7f767414c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56224/0 (socket says 172.21.15.33:56224) 2023-10-06T05:23:38.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 -- 172.21.15.33:0/1552554834 learned_addr learned my addr 172.21.15.33:0/1552554834 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:38.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767a39b700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660004810 0x7f7674169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f7679b9a700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f767414b9b0 0x7f76740c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 -- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f767414b9b0 msgr2=0x7f76740c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f767414b9b0 0x7f76740c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 -- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660004810 msgr2=0x7f7674169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660004810 0x7f7674169770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 -- 172.21.15.33:0/1552554834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76740accf0 con 0x7f76740c4f60 2023-10-06T05:23:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.316+0000 7f767ab9c700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 0x7f767414c160 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f767406da40 tx=0x7f766c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.317+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c01a070 con 0x7f76740c4f60 2023-10-06T05:23:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.317+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c010050 con 0x7f76740c4f60 2023-10-06T05:23:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.317+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c007b40 con 0x7f76740c4f60 2023-10-06T05:23:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.317+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76740acfd0 con 0x7f76740c4f60 2023-10-06T05:23:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.317+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767414cec0 con 0x7f76740c4f60 2023-10-06T05:23:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.318+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767414d0b0 con 0x7f76740c4f60 2023-10-06T05:23:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.319+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f766c003020 con 0x7f76740c4f60 2023-10-06T05:23:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.320+0000 7f766a7fc700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7658066f30 0x7f76580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.320+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f766c084960 con 0x7f76740c4f60 2023-10-06T05:23:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.320+0000 7f767a39b700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7658066f30 0x7f76580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.321+0000 7f767a39b700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7658066f30 0x7f76580693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7670001200 tx=0x7f7670001d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.324+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f767414d0b0 con 0x7f76740c4f60 2023-10-06T05:23:38.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.526+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f767414d0b0 con 0x7f76740c4f60 2023-10-06T05:23:38.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.527+0000 7f766a7fc700 1 -- 172.21.15.33:0/1552554834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16939 (secure 0 0 0) 0x7f767414d0b0 con 0x7f76740c4f60 2023-10-06T05:23:38.529 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:38.529 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":118,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:31.012006+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:04.306347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:05.660589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/588756950":"2023-10-06T06:23:30.410369+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:38.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.529+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7658066f30 msgr2=0x7f76580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.529+0000 7f767b39d700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7658066f30 0x7f76580693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7670001200 tx=0x7f7670001d20 comp rx=0 tx=0).stop 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.529+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 msgr2=0x7f767414c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.529+0000 7f767b39d700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 0x7f767414c160 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f767406da40 tx=0x7f766c00c040 comp rx=0 tx=0).stop 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 shutdown_connections 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7658066f30 0x7f76580693e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76740c4f60 0x7f767414c160 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7660004810 0x7f7674169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 --2- 172.21.15.33:0/1552554834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f767414b9b0 0x7f76740c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 >> 172.21.15.33:0/1552554834 conn(0x7f767406d030 msgr2=0x7f76740cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 shutdown_connections 2023-10-06T05:23:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.530+0000 7f767b39d700 1 -- 172.21.15.33:0/1552554834 wait complete. 2023-10-06T05:23:38.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:38.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.735+0000 7f117e79d700 1 Processor -- start 2023-10-06T05:23:38.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.735+0000 7f117e79d700 1 -- start start 2023-10-06T05:23:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.736+0000 7f117e79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1178159400 0x7f11780cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.736+0000 7f117e79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11781597d0 0x7f11780cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.736+0000 7f117e79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11780d0ed0 con 0x7f11780cc770 2023-10-06T05:23:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.736+0000 7f117e79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11780b1160 con 0x7f11781597d0 2023-10-06T05:23:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.736+0000 7f117e79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11780cb6d0 con 0x7f1178159400 2023-10-06T05:23:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.736+0000 7f117df9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11780cc770 0x7f117806c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47846/0 (socket says 172.21.15.33:47846) 2023-10-06T05:23:38.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117df9c700 1 -- 172.21.15.33:0/4092687895 learned_addr learned my addr 172.21.15.33:0/4092687895 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:38.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117cf9a700 1 --2- 172.21.15.33:0/4092687895 >> v2:172.21.15.33:3300/0 conn(0x7f11781597d0 0x7f11780cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117d79b700 1 --2- 172.21.15.33:0/4092687895 >> v2:172.21.15.167:3300/0 conn(0x7f1178159400 0x7f11780cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f116ffff700 1 -- 172.21.15.33:0/4092687895 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090342965 0 0) 0x7f11780d0ed0 con 0x7f11780cc770 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f116ffff700 1 -- 172.21.15.33:0/4092687895 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1164003570 con 0x7f11780cc770 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117cf9a700 1 -- 172.21.15.33:0/4092687895 >> v2:172.21.15.167:3300/0 conn(0x7f1178159400 msgr2=0x7f11780cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117cf9a700 1 --2- 172.21.15.33:0/4092687895 >> v2:172.21.15.167:3300/0 conn(0x7f1178159400 0x7f11780cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117cf9a700 1 -- 172.21.15.33:0/4092687895 >> v1:172.21.15.33:6789/0 conn(0x7f11780cc770 legacy=0x7f117806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117cf9a700 1 -- 172.21.15.33:0/4092687895 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11780cb9b0 con 0x7f11781597d0 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.737+0000 7f117cf9a700 1 --2- 172.21.15.33:0/4092687895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11781597d0 0x7f11780cb0a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f117806c030 tx=0x7f11680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f2a601e31a1c77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f116ffff700 1 -- 172.21.15.33:0/4092687895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011630 con 0x7f11781597d0 2023-10-06T05:23:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f116ffff700 1 -- 172.21.15.33:0/4092687895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11680117b0 con 0x7f11781597d0 2023-10-06T05:23:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f116ffff700 1 -- 172.21.15.33:0/4092687895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011b30 con 0x7f11781597d0 2023-10-06T05:23:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 -- 172.21.15.33:0/4092687895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11781597d0 msgr2=0x7f11780cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 --2- 172.21.15.33:0/4092687895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11781597d0 0x7f11780cb0a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f117806c030 tx=0x7f11680091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 -- 172.21.15.33:0/4092687895 shutdown_connections 2023-10-06T05:23:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 --2- 172.21.15.33:0/4092687895 >> v2:172.21.15.167:3300/0 conn(0x7f1178159400 0x7f11780cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 --2- 172.21.15.33:0/4092687895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11781597d0 0x7f11780cb0a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 -- 172.21.15.33:0/4092687895 >> 172.21.15.33:0/4092687895 conn(0x7f117806e4b0 msgr2=0x7f11780d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 -- 172.21.15.33:0/4092687895 shutdown_connections 2023-10-06T05:23:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.738+0000 7f117e79d700 1 -- 172.21.15.33:0/4092687895 wait complete. 2023-10-06T05:23:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.740+0000 7f117e79d700 1 Processor -- start 2023-10-06T05:23:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.740+0000 7f117e79d700 1 -- start start 2023-10-06T05:23:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117e79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11780cc770 0x7f117814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117e79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 0x7f1178140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11781597d0 0x7f11780c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117e79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117814df20 con 0x7f1178159400 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117e79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117816de60 con 0x7f11780cc770 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117814f720 con 0x7f11781597d0 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117cf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 0x7f1178140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117cf9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 0x7f1178140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56254/0 (socket says 172.21.15.33:56254) 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117cf9a700 1 -- 172.21.15.33:0/1074482158 learned_addr learned my addr 172.21.15.33:0/1074482158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117d79b700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11780cc770 0x7f117814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.741+0000 7f117df9c700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11781597d0 0x7f11780c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117cf9a700 1 -- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11781597d0 msgr2=0x7f11780c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117cf9a700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11781597d0 0x7f11780c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117cf9a700 1 -- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11780cc770 msgr2=0x7f117814d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117cf9a700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11780cc770 0x7f117814d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117cf9a700 1 -- 172.21.15.33:0/1074482158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168009420 con 0x7f1178159400 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117d79b700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11780cc770 0x7f117814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117cf9a700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 0x7f1178140ee0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1168013000 tx=0x7f1168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011b30 con 0x7f1178159400 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168000d10 con 0x7f1178159400 2023-10-06T05:23:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116801a660 con 0x7f1178159400 2023-10-06T05:23:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117814fa00 con 0x7f1178159400 2023-10-06T05:23:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.742+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1178141420 con 0x7f1178159400 2023-10-06T05:23:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.744+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117814ff00 con 0x7f1178159400 2023-10-06T05:23:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.744+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f116801aa10 con 0x7f1178159400 2023-10-06T05:23:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.745+0000 7f116d7fa700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f115c066f30 0x7f115c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.745+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1168086290 con 0x7f1178159400 2023-10-06T05:23:38.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.746+0000 7f117d79b700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f115c066f30 0x7f115c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.746+0000 7f117d79b700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f115c066f30 0x7f115c0693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1174009cf0 tx=0x7f1174008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.749+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1168053370 con 0x7f1178159400 2023-10-06T05:23:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:38.998+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f117804c380 con 0x7f1178159400 2023-10-06T05:23:39.001 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:23:38.999+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.003+0000 7f116d7fa700 1 -- 172.21.15.33:0/1074482158 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1168056990 con 0x7f1178159400 2023-10-06T05:23:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f115c066f30 msgr2=0x7f115c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f115c066f30 0x7f115c0693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1174009cf0 tx=0x7f1174008040 comp rx=0 tx=0).stop 2023-10-06T05:23:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 msgr2=0x7f1178140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 0x7f1178140ee0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1168013000 tx=0x7f1168017000 comp rx=0 tx=0).stop 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 shutdown_connections 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f115c066f30 0x7f115c0693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11781597d0 0x7f11780c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11780cc770 0x7f117814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 --2- 172.21.15.33:0/1074482158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1178159400 0x7f1178140ee0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 >> 172.21.15.33:0/1074482158 conn(0x7f117806e4b0 msgr2=0x7f1178153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.006+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 shutdown_connections 2023-10-06T05:23:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.007+0000 7f117e79d700 1 -- 172.21.15.33:0/1074482158 wait complete. 2023-10-06T05:23:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:39.012 INFO:tasks.ceph.mds.a.smithi033.stderr: -2815> 2023-10-06T05:23:11.160+0000 7f87f9ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:39.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:23:39.054 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:23:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.216+0000 7fcb9b59e700 1 Processor -- start 2023-10-06T05:23:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.217+0000 7fcb9b59e700 1 -- start start 2023-10-06T05:23:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c0cd5e0 0x7fcb9c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcb9c15a650 0x7fcb9c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9c15d660 con 0x7fcb9c0af540 2023-10-06T05:23:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c0ce890 con 0x7fcb9c15a650 2023-10-06T05:23:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c15aad0 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcb9c0af540 0x7fcb9c0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47870/0 (socket says 172.21.15.33:47870) 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9a59c700 1 -- 172.21.15.33:0/4246640131 learned_addr learned my addr 172.21.15.33:0/4246640131 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb99d9b700 1 --2- 172.21.15.33:0/4246640131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c0cd5e0 0x7fcb9c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.218+0000 7fcb9ad9d700 1 --2- 172.21.15.33:0/4246640131 >> v2:172.21.15.167:3300/0 conn(0x7fcb9c15a650 0x7fcb9c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.219+0000 7fcb9959a700 1 -- 172.21.15.33:0/4246640131 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894166892 0 0) 0x7fcb9c15d660 con 0x7fcb9c0af540 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.219+0000 7fcb99d9b700 1 -- 172.21.15.33:0/4246640131 >> v2:172.21.15.167:3300/0 conn(0x7fcb9c15a650 msgr2=0x7fcb9c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.219+0000 7fcb99d9b700 1 --2- 172.21.15.33:0/4246640131 >> v2:172.21.15.167:3300/0 conn(0x7fcb9c15a650 0x7fcb9c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.219+0000 7fcb99d9b700 1 -- 172.21.15.33:0/4246640131 >> v1:172.21.15.33:6789/0 conn(0x7fcb9c0af540 legacy=0x7fcb9c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.219+0000 7fcb99d9b700 1 -- 172.21.15.33:0/4246640131 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c15d890 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.219+0000 7fcb99d9b700 1 --2- 172.21.15.33:0/4246640131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c0cd5e0 0x7fcb9c0af910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb900098e0 tx=0x7fcb900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234238ae94265568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9959a700 1 -- 172.21.15.33:0/4246640131 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000b640 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9959a700 1 -- 172.21.15.33:0/4246640131 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9000b7a0 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9959a700 1 -- 172.21.15.33:0/4246640131 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000bb40 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 -- 172.21.15.33:0/4246640131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c0cd5e0 msgr2=0x7fcb9c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/4246640131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c0cd5e0 0x7fcb9c0af910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb900098e0 tx=0x7fcb900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 -- 172.21.15.33:0/4246640131 shutdown_connections 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/4246640131 >> v2:172.21.15.167:3300/0 conn(0x7fcb9c15a650 0x7fcb9c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/4246640131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c0cd5e0 0x7fcb9c0af910 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 -- 172.21.15.33:0/4246640131 >> 172.21.15.33:0/4246640131 conn(0x7fcb9c06d9b0 msgr2=0x7fcb9c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 -- 172.21.15.33:0/4246640131 shutdown_connections 2023-10-06T05:23:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.220+0000 7fcb9b59e700 1 -- 172.21.15.33:0/4246640131 wait complete. 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.222+0000 7fcb9b59e700 1 Processor -- start 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.222+0000 7fcb9b59e700 1 -- start start 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c0af540 0x7fcb9c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 0x7fcb9c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c15a650 0x7fcb9c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0aee60 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0b3af0 con 0x7fcb9c0af540 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1579d0 con 0x7fcb9c15a650 2023-10-06T05:23:39.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 0x7fcb9c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 0x7fcb9c15bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56264/0 (socket says 172.21.15.33:56264) 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 -- 172.21.15.33:0/2927770215 learned_addr learned my addr 172.21.15.33:0/2927770215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9a59c700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c0af540 0x7fcb9c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb9ad9d700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c15a650 0x7fcb9c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 -- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c15a650 msgr2=0x7fcb9c15c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c15a650 0x7fcb9c15c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 -- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c0af540 msgr2=0x7fcb9c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.223+0000 7fcb99d9b700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c0af540 0x7fcb9c157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb99d9b700 1 -- 172.21.15.33:0/2927770215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90009420 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb9a59c700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c0af540 0x7fcb9c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb99d9b700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 0x7fcb9c15bdc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c15ac10 tx=0x7fcb90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000bb20 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90011420 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900117a0 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c0aeb80 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.224+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c0b3780 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.225+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c04c380 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.226+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb90011b50 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.227+0000 7fcb8a7fc700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcb7c066f30 0x7fcb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.227+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb90085df0 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.227+0000 7fcb9a59c700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcb7c066f30 0x7fcb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.228+0000 7fcb9a59c700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcb7c066f30 0x7fcb7c0693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b2180 tx=0x7fcb8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.232+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb90052e80 con 0x7fcb9c0cd5e0 2023-10-06T05:23:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:39.479+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb9c15db70 con 0x7fcb9c0cd5e0 2023-10-06T05:23:40.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.006+0000 7fcb8a7fc700 1 -- 172.21.15.33:0/2927770215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb900564a0 con 0x7fcb9c0cd5e0 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcb7c066f30 msgr2=0x7fcb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcb7c066f30 0x7fcb7c0693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b2180 tx=0x7fcb8c009040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 msgr2=0x7fcb9c15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 0x7fcb9c15bdc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c15ac10 tx=0x7fcb90017000 comp rx=0 tx=0).stop 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 shutdown_connections 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcb7c066f30 0x7fcb7c0693e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb9c15a650 0x7fcb9c15c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb9c0af540 0x7fcb9c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 --2- 172.21.15.33:0/2927770215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb9c0cd5e0 0x7fcb9c15bdc0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 >> 172.21.15.33:0/2927770215 conn(0x7fcb9c06d9b0 msgr2=0x7fcb9c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 shutdown_connections 2023-10-06T05:23:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.009+0000 7fcb9b59e700 1 -- 172.21.15.33:0/2927770215 wait complete. 2023-10-06T05:23:40.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:23:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.263+0000 7f38a179d700 1 Processor -- start 2023-10-06T05:23:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.263+0000 7f38a179d700 1 -- start start 2023-10-06T05:23:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f38a179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f389c0ad830 0x7f389c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f38a179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c06ee20 0x7f389c0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f38a179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c0ad3a0 con 0x7f389c157320 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f38a179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389c0ac580 con 0x7f389c0ad830 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f38a179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0afcc0 con 0x7f389c06ee20 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f389b7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f389c0ad830 0x7f389c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f38a0f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c06ee20 0x7f389c0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f389bfff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f389c157320 0x7f389c06e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:52876/0 (socket says 172.21.15.33:52876) 2023-10-06T05:23:40.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.265+0000 7f389bfff700 1 -- 172.21.15.33:0/1870350344 learned_addr learned my addr 172.21.15.33:0/1870350344 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389affd700 1 -- 172.21.15.33:0/1870350344 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730058739 0 0) 0x7f389c0ad3a0 con 0x7f389c157320 2023-10-06T05:23:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389affd700 1 -- 172.21.15.33:0/1870350344 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3884003570 con 0x7f389c157320 2023-10-06T05:23:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389b7fe700 1 -- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c06ee20 msgr2=0x7f389c0ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389b7fe700 1 --2- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c06ee20 0x7f389c0ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389b7fe700 1 -- 172.21.15.33:0/1870350344 >> v1:172.21.15.167:6789/0 conn(0x7f389c157320 legacy=0x7f389c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389b7fe700 1 -- 172.21.15.33:0/1870350344 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c1576f0 con 0x7f389c0ad830 2023-10-06T05:23:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f38a0f9c700 1 --2- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c06ee20 0x7f389c0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389b7fe700 1 --2- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c0ad830 0x7f389c159cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f38900098e0 tx=0x7f38900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45bda3a3b38f45e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389affd700 1 -- 172.21.15.33:0/1870350344 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890011630 con 0x7f389c0ad830 2023-10-06T05:23:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389affd700 1 -- 172.21.15.33:0/1870350344 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890011790 con 0x7f389c0ad830 2023-10-06T05:23:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.266+0000 7f389affd700 1 -- 172.21.15.33:0/1870350344 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890011b30 con 0x7f389c0ad830 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 -- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c0ad830 msgr2=0x7f389c159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 --2- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c0ad830 0x7f389c159cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f38900098e0 tx=0x7f38900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 -- 172.21.15.33:0/1870350344 shutdown_connections 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 --2- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c06ee20 0x7f389c0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 --2- 172.21.15.33:0/1870350344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c0ad830 0x7f389c159cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 -- 172.21.15.33:0/1870350344 >> 172.21.15.33:0/1870350344 conn(0x7f389c06cd40 msgr2=0x7f389c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 -- 172.21.15.33:0/1870350344 shutdown_connections 2023-10-06T05:23:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.267+0000 7f38a179d700 1 -- 172.21.15.33:0/1870350344 wait complete. 2023-10-06T05:23:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.269+0000 7f38a179d700 1 Processor -- start 2023-10-06T05:23:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.269+0000 7f38a179d700 1 -- start start 2023-10-06T05:23:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 0x7f389c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c0ad830 0x7f389c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c157320 0x7f389c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c9070 con 0x7f389c06ee20 2023-10-06T05:23:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c9910 con 0x7f389c157320 2023-10-06T05:23:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c8370 con 0x7f389c0ad830 2023-10-06T05:23:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 0x7f389c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 0x7f389c0c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43418/0 (socket says 172.21.15.33:43418) 2023-10-06T05:23:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 -- 172.21.15.33:0/659361344 learned_addr learned my addr 172.21.15.33:0/659361344 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389b7fe700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c0ad830 0x7f389c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a0f9c700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c157320 0x7f389c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 -- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c0ad830 msgr2=0x7f389c14b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c0ad830 0x7f389c14b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 -- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c157320 msgr2=0x7f389c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c157320 0x7f389c1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f389bfff700 1 -- 172.21.15.33:0/659361344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3894009a20 con 0x7f389c06ee20 2023-10-06T05:23:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.270+0000 7f38a0f9c700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c157320 0x7f389c1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.271+0000 7f389bfff700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 0x7f389c0c9da0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c5c0 tx=0x7f388c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.271+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c005b30 con 0x7f389c06ee20 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.271+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c005cb0 con 0x7f389c06ee20 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.271+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c007320 con 0x7f389c06ee20 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.271+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3890009420 con 0x7f389c06ee20 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.271+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c0c95a0 con 0x7f389c06ee20 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.272+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c0c9bf0 con 0x7f389c06ee20 2023-10-06T05:23:40.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.273+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f388c01b3b0 con 0x7f389c06ee20 2023-10-06T05:23:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.274+0000 7f3883fff700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3888066f30 0x7f38880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.274+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f388c083ca0 con 0x7f389c06ee20 2023-10-06T05:23:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.274+0000 7f389b7fe700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3888066f30 0x7f38880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.275+0000 7f389b7fe700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3888066f30 0x7f38880693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f38900091e0 tx=0x7f3890017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.278+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f388c050c90 con 0x7f389c06ee20 2023-10-06T05:23:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:40.481+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f389c063ff0 con 0x7f389c06ee20 2023-10-06T05:23:41.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.008+0000 7f3883fff700 1 -- 172.21.15.33:0/659361344 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7f388c017020 con 0x7f389c06ee20 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.012+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3888066f30 msgr2=0x7f38880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.012+0000 7f38a179d700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3888066f30 0x7f38880693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f38900091e0 tx=0x7f3890017000 comp rx=0 tx=0).stop 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.012+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 msgr2=0x7f389c0c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.012+0000 7f38a179d700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 0x7f389c0c9da0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c5c0 tx=0x7f388c007e00 comp rx=0 tx=0).stop 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 shutdown_connections 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3888066f30 0x7f38880693e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f389c157320 0x7f389c1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c06ee20 0x7f389c0c9da0 secure :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c5c0 tx=0x7f388c007e00 comp rx=0 tx=0).stop 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 --2- 172.21.15.33:0/659361344 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389c0ad830 0x7f389c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 >> 172.21.15.33:0/659361344 conn(0x7f389c06cd40 msgr2=0x7f389c069090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:41.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 shutdown_connections 2023-10-06T05:23:41.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.013+0000 7f38a179d700 1 -- 172.21.15.33:0/659361344 wait complete. 2023-10-06T05:23:41.017 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:23:41.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:23:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.228+0000 7f68f579d700 1 Processor -- start 2023-10-06T05:23:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.228+0000 7f68f579d700 1 -- start start 2023-10-06T05:23:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f579d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68f006dc00 0x7f68f01590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 0x7f68f0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f015d280 con 0x7f68f015a030 2023-10-06T05:23:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f579d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f015e290 con 0x7f68f006dc00 2023-10-06T05:23:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f015ab90 con 0x7f68f006dfd0 2023-10-06T05:23:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f4f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f68f015a030 0x7f68f015a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53534/0 (socket says 172.21.15.33:53534) 2023-10-06T05:23:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.229+0000 7f68f4f9c700 1 -- 172.21.15.33:0/3715910924 learned_addr learned my addr 172.21.15.33:0/3715910924 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68effff700 1 --2- 172.21.15.33:0/3715910924 >> v2:172.21.15.167:3300/0 conn(0x7f68f006dc00 0x7f68f01590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68ef7fe700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 0x7f68f0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50918073 0 0) 0x7f68f015d280 con 0x7f68f015a030 2023-10-06T05:23:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d8003570 con 0x7f68f015a030 2023-10-06T05:23:41.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1680261760 0 0) 0x7f68d8003570 con 0x7f68f015a030 2023-10-06T05:23:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 msgr2=0x7f68f0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:41.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 0x7f68f0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 >> v2:172.21.15.167:3300/0 conn(0x7f68f006dc00 msgr2=0x7f68f01590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 --2- 172.21.15.33:0/3715910924 >> v2:172.21.15.167:3300/0 conn(0x7f68f006dc00 0x7f68f01590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f015de00 con 0x7f68f015a030 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f68e80017f0 con 0x7f68f015a030 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 >> v1:172.21.15.33:6789/0 conn(0x7f68f015a030 legacy=0x7f68f015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 0x7f68d8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004010 0x7f68d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d8004d50 0x7f68d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f015ab90 con 0x7f68d8004010 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d800ba60 con 0x7f68d8004d50 2023-10-06T05:23:41.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.230+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d800bbf0 con 0x7f68f006dfd0 2023-10-06T05:23:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004010 0x7f68d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68effff700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 0x7f68d8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68f4f9c700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d8004d50 0x7f68d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 -- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 msgr2=0x7f68d8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:41.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68f006dfd0 0x7f68d8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 -- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d8004d50 msgr2=0x7f68d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68d8004d50 0x7f68d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 -- 172.21.15.33:0/3715910924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8003570 con 0x7f68d8004010 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.231+0000 7f68ef7fe700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004010 0x7f68d8004810 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009c00 tx=0x7f68e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619b34efa2f58818 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400e710 con 0x7f68d8004010 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400e870 con 0x7f68d8004010 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68eeffd700 1 -- 172.21.15.33:0/3715910924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400ec30 con 0x7f68d8004010 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f579d700 1 -- 172.21.15.33:0/3715910924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004010 msgr2=0x7f68d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f579d700 1 --2- 172.21.15.33:0/3715910924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68d8004010 0x7f68d8004810 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009c00 tx=0x7f68e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f4f9c700 1 -- 172.21.15.33:0/3715910924 reap_dead start 2023-10-06T05:23:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f579d700 1 -- 172.21.15.33:0/3715910924 shutdown_connections 2023-10-06T05:23:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f579d700 1 -- 172.21.15.33:0/3715910924 >> 172.21.15.33:0/3715910924 conn(0x7f68f00d0550 msgr2=0x7f68f00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f579d700 1 -- 172.21.15.33:0/3715910924 shutdown_connections 2023-10-06T05:23:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.232+0000 7f68f579d700 1 -- 172.21.15.33:0/3715910924 wait complete. 2023-10-06T05:23:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.234+0000 7f68f579d700 1 Processor -- start 2023-10-06T05:23:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.234+0000 7f68f579d700 1 -- start start 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d8004010 0x7f68f014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f00b14c0 0x7f68f00c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 0x7f68f00b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00c88c0 con 0x7f68f00c7b40 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00b1030 con 0x7f68f00b14c0 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00c8080 con 0x7f68d8004010 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f4f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 0x7f68f00b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f4f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 0x7f68f00b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43432/0 (socket says 172.21.15.33:43432) 2023-10-06T05:23:41.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68f4f9c700 1 -- 172.21.15.33:0/4149413110 learned_addr learned my addr 172.21.15.33:0/4149413110 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68effff700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d8004010 0x7f68f014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.235+0000 7f68ef7fe700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f00b14c0 0x7f68f00c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f4f9c700 1 -- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d8004010 msgr2=0x7f68f014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f4f9c700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d8004010 0x7f68f014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f4f9c700 1 -- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f00b14c0 msgr2=0x7f68f00c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f4f9c700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f00b14c0 0x7f68f00c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f4f9c700 1 -- 172.21.15.33:0/4149413110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00b0700 con 0x7f68f00c7b40 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f4f9c700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 0x7f68f00b1c30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f68f015ad90 tx=0x7f68e80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e800ac30 con 0x7f68f00c7b40 2023-10-06T05:23:41.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e800adb0 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f00b09e0 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8013640 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.236+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f00ca300 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.237+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f0063ff0 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.238+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68e8013a30 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.239+0000 7f68d7fff700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68dc066f30 0x7f68dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.239+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f68e8084c80 con 0x7f68f00c7b40 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.240+0000 7f68effff700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68dc066f30 0x7f68dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.241+0000 7f68effff700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68dc066f30 0x7f68dc0693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006f00 tx=0x7f68e00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:41.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.244+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68e8051ec0 con 0x7f68f00c7b40 2023-10-06T05:23:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:41.446+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68f015d4d0 con 0x7f68f00c7b40 2023-10-06T05:23:42.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.017+0000 7f68d7fff700 1 -- 172.21.15.33:0/4149413110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7f68e80554e0 con 0x7f68f00c7b40 2023-10-06T05:23:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.023+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68dc066f30 msgr2=0x7f68dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.023+0000 7f68f579d700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68dc066f30 0x7f68dc0693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006f00 tx=0x7f68e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.023+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 msgr2=0x7f68f00b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.023+0000 7f68f579d700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 0x7f68f00b1c30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f68f015ad90 tx=0x7f68e80021d0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.023+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 shutdown_connections 2023-10-06T05:23:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.023+0000 7f68f579d700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f68dc066f30 0x7f68dc0693e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.024+0000 7f68f579d700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68f00c7b40 0x7f68f00b1c30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.024+0000 7f68f579d700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68d8004010 0x7f68f014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.024+0000 7f68f579d700 1 --2- 172.21.15.33:0/4149413110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68f00b14c0 0x7f68f00c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.024+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 >> 172.21.15.33:0/4149413110 conn(0x7f68f00d0550 msgr2=0x7f68f014cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.024+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 shutdown_connections 2023-10-06T05:23:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.024+0000 7f68f579d700 1 -- 172.21.15.33:0/4149413110 wait complete. 2023-10-06T05:23:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:23:42.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:23:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.238+0000 7fb5e259d700 1 Processor -- start 2023-10-06T05:23:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.239+0000 7fb5e259d700 1 -- start start 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 0x7fb5dc155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e259d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb5dc0afb20 0x7fb5dc0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e259d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc0b06b0 con 0x7fb5dc0c9490 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e259d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5dc0c99e0 con 0x7fb5dc0afb20 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0c9cc0 con 0x7fb5dc155a50 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e0d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb5dc0c9490 0x7fb5dc0af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53544/0 (socket says 172.21.15.33:53544) 2023-10-06T05:23:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.240+0000 7fb5e0d9a700 1 -- 172.21.15.33:0/325203172 learned_addr learned my addr 172.21.15.33:0/325203172 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007863437 0 0) 0x7fb5dc0b06b0 con 0x7fb5dc0c9490 2023-10-06T05:23:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5e159b700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 0x7fb5dc155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c8003570 con 0x7fb5dc0c9490 2023-10-06T05:23:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5e1d9c700 1 --2- 172.21.15.33:0/325203172 >> v2:172.21.15.167:3300/0 conn(0x7fb5dc0afb20 0x7fb5dc0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2201046083 0 0) 0x7fb5c8003570 con 0x7fb5dc0c9490 2023-10-06T05:23:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 msgr2=0x7fb5dc155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 0x7fb5dc155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 >> v2:172.21.15.167:3300/0 conn(0x7fb5dc0afb20 msgr2=0x7fb5dc0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 --2- 172.21.15.33:0/325203172 >> v2:172.21.15.167:3300/0 conn(0x7fb5dc0afb20 0x7fb5dc0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc0c9fa0 con 0x7fb5dc0c9490 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb5cc001470 con 0x7fb5dc0c9490 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5e1d9c700 1 --2- 172.21.15.33:0/325203172 >> v2:172.21.15.167:3300/0 conn(0x7fb5dc0afb20 0x7fb5dc0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 >> v1:172.21.15.33:6789/0 conn(0x7fb5dc0c9490 legacy=0x7fb5dc0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 0x7fb5c8003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c8004d50 0x7fb5c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0c99e0 con 0x7fb5c80040b0 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0c9cc0 con 0x7fb5c8004d50 2023-10-06T05:23:42.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c800de30 con 0x7fb5dc155a50 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.241+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e159b700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 0x7fb5c8003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e1d9c700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c8004d50 0x7fb5c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e0d9a700 1 -- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 msgr2=0x7fb5c8003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc155a50 0x7fb5c8003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e0d9a700 1 -- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c8004d50 msgr2=0x7fb5c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c8004d50 0x7fb5c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.242+0000 7fb5e0d9a700 1 -- 172.21.15.33:0/325203172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8003570 con 0x7fb5c80040b0 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.243+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5c8004810 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc007bf0 tx=0x7fb5cc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f2b9ce2b402580 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.243+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00cb70 con 0x7fb5c80040b0 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.243+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00ccd0 con 0x7fb5c80040b0 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.243+0000 7fb5d3fff700 1 -- 172.21.15.33:0/325203172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00ce80 con 0x7fb5c80040b0 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e259d700 1 -- 172.21.15.33:0/325203172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 msgr2=0x7fb5c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e259d700 1 --2- 172.21.15.33:0/325203172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5c8004810 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc007bf0 tx=0x7fb5cc001d80 comp rx=0 tx=0).stop 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e1d9c700 1 -- 172.21.15.33:0/325203172 reap_dead start 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e259d700 1 -- 172.21.15.33:0/325203172 shutdown_connections 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e259d700 1 -- 172.21.15.33:0/325203172 >> 172.21.15.33:0/325203172 conn(0x7fb5dc06d030 msgr2=0x7fb5dc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e259d700 1 -- 172.21.15.33:0/325203172 shutdown_connections 2023-10-06T05:23:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.244+0000 7fb5e259d700 1 -- 172.21.15.33:0/325203172 wait complete. 2023-10-06T05:23:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.246+0000 7fb5e259d700 1 Processor -- start 2023-10-06T05:23:42.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.246+0000 7fb5e259d700 1 -- start start 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.247+0000 7fb5e259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5dc133e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.247+0000 7fb5e259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc1488a0 0x7fb5dc153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc149730 0x7fb5dc149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc14a0f0 con 0x7fb5c80040b0 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc15ae70 con 0x7fb5dc149730 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc153ca0 con 0x7fb5dc1488a0 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5dc133e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5dc133e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43452/0 (socket says 172.21.15.33:43452) 2023-10-06T05:23:42.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 -- 172.21.15.33:0/2247762898 learned_addr learned my addr 172.21.15.33:0/2247762898 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e1d9c700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc149730 0x7fb5dc149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc1488a0 0x7fb5dc153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 -- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc1488a0 msgr2=0x7fb5dc153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc1488a0 0x7fb5dc153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 -- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc149730 msgr2=0x7fb5dc149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc149730 0x7fb5dc149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.248+0000 7fb5e159b700 1 -- 172.21.15.33:0/2247762898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc153f80 con 0x7fb5c80040b0 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.249+0000 7fb5e159b700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5dc133e20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8011d70 tx=0x7fb5d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.249+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800f990 con 0x7fb5c80040b0 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.249+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800faf0 con 0x7fb5c80040b0 2023-10-06T05:23:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.249+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800fe70 con 0x7fb5c80040b0 2023-10-06T05:23:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.250+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc154260 con 0x7fb5c80040b0 2023-10-06T05:23:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.250+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc154540 con 0x7fb5c80040b0 2023-10-06T05:23:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.251+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5dc04c380 con 0x7fb5c80040b0 2023-10-06T05:23:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.253+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5d8014650 con 0x7fb5c80040b0 2023-10-06T05:23:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.254+0000 7fb5d17fa700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5c0067000 0x7fb5c00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.254+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb5d8018030 con 0x7fb5c80040b0 2023-10-06T05:23:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.254+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5c0067000 0x7fb5c00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.255+0000 7fb5e0d9a700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5c0067000 0x7fb5c00694b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc148cf0 tx=0x7fb5cc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.258+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5d8052040 con 0x7fb5c80040b0 2023-10-06T05:23:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.463+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5dc0ca280 con 0x7fb5c80040b0 2023-10-06T05:23:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.467+0000 7fb5d17fa700 1 -- 172.21.15.33:0/2247762898 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5d8055660 con 0x7fb5c80040b0 2023-10-06T05:23:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.469+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5c0067000 msgr2=0x7fb5c00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.469+0000 7fb5e259d700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5c0067000 0x7fb5c00694b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc148cf0 tx=0x7fb5cc016000 comp rx=0 tx=0).stop 2023-10-06T05:23:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.469+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 msgr2=0x7fb5dc133e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.469+0000 7fb5e259d700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5dc133e20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8011d70 tx=0x7fb5d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 shutdown_connections 2023-10-06T05:23:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5c0067000 0x7fb5c00694b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5dc149730 0x7fb5dc149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80040b0 0x7fb5dc133e20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 --2- 172.21.15.33:0/2247762898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5dc1488a0 0x7fb5dc153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 >> 172.21.15.33:0/2247762898 conn(0x7fb5dc06d030 msgr2=0x7fb5dc06a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 shutdown_connections 2023-10-06T05:23:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.470+0000 7fb5e259d700 1 -- 172.21.15.33:0/2247762898 wait complete. 2023-10-06T05:23:42.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_evicted_caps' 2023-10-06T05:23:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.674+0000 7fc203fff700 1 Processor -- start 2023-10-06T05:23:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.674+0000 7fc203fff700 1 -- start start 2023-10-06T05:23:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc203fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2040c8050 0x7fc20413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc203fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2040c9cb0 con 0x7fc20406aad0 2023-10-06T05:23:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc203fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2040ca270 con 0x7fc20413faa0 2023-10-06T05:23:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc203fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20406b660 con 0x7fc2040c8050 2023-10-06T05:23:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc2037fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc20406aad0 0x7fc20406af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53552/0 (socket says 172.21.15.33:53552) 2023-10-06T05:23:42.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc2037fe700 1 -- 172.21.15.33:0/2678344611 learned_addr learned my addr 172.21.15.33:0/2678344611 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.675+0000 7fc2027fc700 1 --2- 172.21.15.33:0/2678344611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2040c8050 0x7fc20413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896920713 0 0) 0x7fc2040c9cb0 con 0x7fc20406aad0 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ec003570 con 0x7fc20406aad0 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304817773 0 0) 0x7fc2040ca270 con 0x7fc20413faa0 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2040c9cb0 con 0x7fc20413faa0 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc2027fc700 1 -- 172.21.15.33:0/2678344611 >> v1:172.21.15.167:6789/0 conn(0x7fc20413faa0 legacy=0x7fc2040c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc2027fc700 1 -- 172.21.15.33:0/2678344611 >> v1:172.21.15.33:6789/0 conn(0x7fc20406aad0 legacy=0x7fc20406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc2027fc700 1 -- 172.21.15.33:0/2678344611 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20406b840 con 0x7fc2040c8050 2023-10-06T05:23:42.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.676+0000 7fc2027fc700 1 --2- 172.21.15.33:0/2678344611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2040c8050 0x7fc20413fe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009910 tx=0x7fc1f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5dba8bbd8da769c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8011630 con 0x7fc2040c8050 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f8011790 con 0x7fc2040c8050 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc201ffb700 1 -- 172.21.15.33:0/2678344611 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8011b30 con 0x7fc2040c8050 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 -- 172.21.15.33:0/2678344611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2040c8050 msgr2=0x7fc20413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 --2- 172.21.15.33:0/2678344611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2040c8050 0x7fc20413fe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009910 tx=0x7fc1f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 -- 172.21.15.33:0/2678344611 shutdown_connections 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 --2- 172.21.15.33:0/2678344611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2040c8050 0x7fc20413fe70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 -- 172.21.15.33:0/2678344611 >> 172.21.15.33:0/2678344611 conn(0x7fc20406d030 msgr2=0x7fc20406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 -- 172.21.15.33:0/2678344611 shutdown_connections 2023-10-06T05:23:42.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.677+0000 7fc203fff700 1 -- 172.21.15.33:0/2678344611 wait complete. 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.679+0000 7fc203fff700 1 Processor -- start 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.679+0000 7fc203fff700 1 -- start start 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc203fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc20406aad0 0x7fc204155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc203fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2040c8050 0x7fc20413e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc203fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 0x7fc20413ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc203fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040c3ed0 con 0x7fc20413faa0 2023-10-06T05:23:42.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc203fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040c32c0 con 0x7fc2040c8050 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc203fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040c35a0 con 0x7fc20406aad0 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 0x7fc20413ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 0x7fc20413ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43464/0 (socket says 172.21.15.33:43464) 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 -- 172.21.15.33:0/1024714707 learned_addr learned my addr 172.21.15.33:0/1024714707 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc202ffd700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc20406aad0 0x7fc204155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2027fc700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2040c8050 0x7fc20413e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 -- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc20406aad0 msgr2=0x7fc204155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc20406aad0 0x7fc204155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 -- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2040c8050 msgr2=0x7fc20413e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2040c8050 0x7fc20413e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.680+0000 7fc2037fe700 1 -- 172.21.15.33:0/1024714707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f8009420 con 0x7fc20413faa0 2023-10-06T05:23:42.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.681+0000 7fc2037fe700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 0x7fc20413ef00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00cc60 tx=0x7fc1fc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.681+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00a8b0 con 0x7fc20413faa0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.681+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc00aa30 con 0x7fc20413faa0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.681+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2040c3880 con 0x7fc20413faa0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.681+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00adf0 con 0x7fc20413faa0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.681+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc204156340 con 0x7fc20413faa0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.682+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2040c4040 con 0x7fc20413faa0 2023-10-06T05:23:42.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.683+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1fc007300 con 0x7fc20413faa0 2023-10-06T05:23:42.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.684+0000 7fc1eaffd700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1f0066f30 0x7fc1f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.684+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc1fc0846f0 con 0x7fc20413faa0 2023-10-06T05:23:42.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.685+0000 7fc202ffd700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1f0066f30 0x7fc1f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.685+0000 7fc202ffd700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1f0066f30 0x7fc1f00693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4007fd0 tx=0x7fc1f4007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.689+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1fc0555e0 con 0x7fc20413faa0 2023-10-06T05:23:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:42.971+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_evicted_caps"]} v 0) v1 -- 0x7fc20404c380 con 0x7fc20413faa0 2023-10-06T05:23:43.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.015+0000 7fc1eaffd700 1 -- 172.21.15.33:0/1024714707 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_evicted_caps"]}]=0 v476) v1 ==== 149+0+0 (secure 0 0 0) 0x7fc1fc055170 con 0x7fc20413faa0 2023-10-06T05:23:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.017+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1f0066f30 msgr2=0x7fc1f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.017+0000 7fc203fff700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1f0066f30 0x7fc1f00693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4007fd0 tx=0x7fc1f4007d50 comp rx=0 tx=0).stop 2023-10-06T05:23:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.017+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 msgr2=0x7fc20413ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.017+0000 7fc203fff700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 0x7fc20413ef00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00cc60 tx=0x7fc1fc001ae0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.017+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 shutdown_connections 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.017+0000 7fc203fff700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc1f0066f30 0x7fc1f00693e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.018+0000 7fc203fff700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20413faa0 0x7fc20413ef00 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.018+0000 7fc203fff700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc20406aad0 0x7fc204155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.018+0000 7fc203fff700 1 --2- 172.21.15.33:0/1024714707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2040c8050 0x7fc20413e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.018+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 >> 172.21.15.33:0/1024714707 conn(0x7fc20406d030 msgr2=0x7fc2040cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.018+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 shutdown_connections 2023-10-06T05:23:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.018+0000 7fc203fff700 1 -- 172.21.15.33:0/1024714707 wait complete. 2023-10-06T05:23:43.037 INFO:tasks.cephfs_test_runner:test_evicted_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:23:43.037 INFO:tasks.cephfs_test_runner:Starting test: test_filelock (tasks.cephfs.test_client_recovery.TestClientRecovery) Check that file lock doesn't get lost after an MDS restart 2023-10-06T05:23:43.037 INFO:tasks.cephfs_test_runner:test_filelock (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:23:43.038 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock' 2023-10-06T05:23:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.223+0000 7f631759e700 1 Processor -- start 2023-10-06T05:23:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.224+0000 7f631759e700 1 -- start start 2023-10-06T05:23:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f631759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f631815a0a0 0x7f631815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f631759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631815e2d0 con 0x7f63180b0a40 2023-10-06T05:23:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f631759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631806afa0 con 0x7f631806ab00 2023-10-06T05:23:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f631759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631815d6c0 con 0x7f631815a0a0 2023-10-06T05:23:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f6316d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f631815a0a0 0x7f631815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f631659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f63180b0a40 0x7f63180b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53558/0 (socket says 172.21.15.33:53558) 2023-10-06T05:23:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.225+0000 7f631659c700 1 -- 172.21.15.33:0/960695494 learned_addr learned my addr 172.21.15.33:0/960695494 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f6316d9d700 1 -- 172.21.15.33:0/960695494 >> v1:172.21.15.167:6789/0 conn(0x7f631806ab00 legacy=0x7f63180b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f6316d9d700 1 -- 172.21.15.33:0/960695494 >> v1:172.21.15.33:6789/0 conn(0x7f63180b0a40 legacy=0x7f63180b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f6316d9d700 1 -- 172.21.15.33:0/960695494 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631815d9a0 con 0x7f631815a0a0 2023-10-06T05:23:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f631559a700 1 -- 172.21.15.33:0/960695494 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123412826 0 0) 0x7f631815e2d0 con 0x7f63180b0a40 2023-10-06T05:23:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f6316d9d700 1 --2- 172.21.15.33:0/960695494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f631815a4e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6310009e60 tx=0x7f6310009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7631aea696bd711c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f631559a700 1 -- 172.21.15.33:0/960695494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631000e070 con 0x7f631815a0a0 2023-10-06T05:23:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f631559a700 1 -- 172.21.15.33:0/960695494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631000bbf0 con 0x7f631815a0a0 2023-10-06T05:23:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.226+0000 7f631559a700 1 -- 172.21.15.33:0/960695494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310012430 con 0x7f631815a0a0 2023-10-06T05:23:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 -- 172.21.15.33:0/960695494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 msgr2=0x7f631815a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 --2- 172.21.15.33:0/960695494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f631815a4e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6310009e60 tx=0x7f6310009780 comp rx=0 tx=0).stop 2023-10-06T05:23:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 -- 172.21.15.33:0/960695494 shutdown_connections 2023-10-06T05:23:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 --2- 172.21.15.33:0/960695494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f631815a4e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 -- 172.21.15.33:0/960695494 >> 172.21.15.33:0/960695494 conn(0x7f631806e0f0 msgr2=0x7f63180675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 -- 172.21.15.33:0/960695494 shutdown_connections 2023-10-06T05:23:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.227+0000 7f631759e700 1 -- 172.21.15.33:0/960695494 wait complete. 2023-10-06T05:23:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.228+0000 7f631759e700 1 Processor -- start 2023-10-06T05:23:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.229+0000 7f631759e700 1 -- start start 2023-10-06T05:23:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631806ab00 0x7f63181565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63180b0a40 0x7f63180adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f63180ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318169a90 con 0x7f631815a0a0 2023-10-06T05:23:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631816a890 con 0x7f631806ab00 2023-10-06T05:23:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6318156af0 con 0x7f63180b0a40 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f63180ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f63180ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43486/0 (socket says 172.21.15.33:43486) 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 -- 172.21.15.33:0/1067642974 learned_addr learned my addr 172.21.15.33:0/1067642974 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f631659c700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631806ab00 0x7f63181565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6315d9b700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63180b0a40 0x7f63180adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 -- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63180b0a40 msgr2=0x7f63180adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63180b0a40 0x7f63180adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 -- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631806ab00 msgr2=0x7f63181565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631806ab00 0x7f63181565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.230+0000 7f6316d9d700 1 -- 172.21.15.33:0/1067642974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310009a20 con 0x7f631815a0a0 2023-10-06T05:23:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.231+0000 7f6316d9d700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f63180ae230 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6310012c00 tx=0x7f6310018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.231+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631001d710 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.231+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631001d870 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.231+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631001dc30 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.231+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6318169c80 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.231+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631816a240 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.232+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631806fcc0 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.233+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6310024070 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.234+0000 7f63067fc700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f62f8066f30 0x7f62f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.234+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6310086ff0 con 0x7f631815a0a0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.234+0000 7f631659c700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f62f8066f30 0x7f62f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.235+0000 7f631659c700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f62f8066f30 0x7f62f80693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f630800c8f0 tx=0x7f6308009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.238+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63100b85a0 con 0x7f631815a0a0 2023-10-06T05:23:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:43.523+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock"]} v 0) v1 -- 0x7f631804c380 con 0x7f631815a0a0 2023-10-06T05:23:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.017+0000 7f63067fc700 1 -- 172.21.15.33:0/1067642974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock"]}]=0 v477) v1 ==== 148+0+0 (secure 0 0 0) 0x7f6310054350 con 0x7f631815a0a0 2023-10-06T05:23:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f62f8066f30 msgr2=0x7f62f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f62f8066f30 0x7f62f80693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f630800c8f0 tx=0x7f6308009040 comp rx=0 tx=0).stop 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 msgr2=0x7f63180ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f63180ae230 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6310012c00 tx=0x7f6310018000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 shutdown_connections 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f62f8066f30 0x7f62f80693e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f631815a0a0 0x7f63180ae230 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f631806ab00 0x7f63181565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 --2- 172.21.15.33:0/1067642974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63180b0a40 0x7f63180adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 >> 172.21.15.33:0/1067642974 conn(0x7f631806e0f0 msgr2=0x7f631806b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 shutdown_connections 2023-10-06T05:23:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.020+0000 7f631759e700 1 -- 172.21.15.33:0/1067642974 wait complete. 2023-10-06T05:23:44.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:23:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.232+0000 7efc1b59d700 1 Processor -- start 2023-10-06T05:23:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.233+0000 7efc1b59d700 1 -- start start 2023-10-06T05:23:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1b59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc14156370 0x7efc140af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1b59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140ad430 con 0x7efc140b01f0 2023-10-06T05:23:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1b59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140afe20 con 0x7efc140cc7e0 2023-10-06T05:23:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1b59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1406bc70 con 0x7efc14156370 2023-10-06T05:23:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1a59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc14156370 0x7efc140af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1ad9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efc140b01f0 0x7efc140ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53564/0 (socket says 172.21.15.33:53564) 2023-10-06T05:23:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1ad9c700 1 -- 172.21.15.33:0/3865498250 learned_addr learned my addr 172.21.15.33:0/3865498250 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.234+0000 7efc1a59b700 1 -- 172.21.15.33:0/3865498250 >> v1:172.21.15.167:6789/0 conn(0x7efc140cc7e0 legacy=0x7efc140c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc1a59b700 1 -- 172.21.15.33:0/3865498250 >> v1:172.21.15.33:6789/0 conn(0x7efc140b01f0 legacy=0x7efc140ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc1a59b700 1 -- 172.21.15.33:0/3865498250 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140cccc0 con 0x7efc14156370 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc19599700 1 -- 172.21.15.33:0/3865498250 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545316182 0 0) 0x7efc140ad430 con 0x7efc140b01f0 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc1a59b700 1 --2- 172.21.15.33:0/3865498250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc140af8e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7efc100098e0 tx=0x7efc100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de845daa0e4b9a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc19599700 1 -- 172.21.15.33:0/3865498250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10009e80 con 0x7efc14156370 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc19599700 1 -- 172.21.15.33:0/3865498250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1000b820 con 0x7efc14156370 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.235+0000 7efc19599700 1 -- 172.21.15.33:0/3865498250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000bba0 con 0x7efc14156370 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 -- 172.21.15.33:0/3865498250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 msgr2=0x7efc140af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 --2- 172.21.15.33:0/3865498250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc140af8e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7efc100098e0 tx=0x7efc100091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 -- 172.21.15.33:0/3865498250 shutdown_connections 2023-10-06T05:23:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 --2- 172.21.15.33:0/3865498250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc140af8e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 -- 172.21.15.33:0/3865498250 >> 172.21.15.33:0/3865498250 conn(0x7efc1406d030 msgr2=0x7efc140cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 -- 172.21.15.33:0/3865498250 shutdown_connections 2023-10-06T05:23:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.236+0000 7efc1b59d700 1 -- 172.21.15.33:0/3865498250 wait complete. 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.237+0000 7efc1b59d700 1 Processor -- start 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.237+0000 7efc1b59d700 1 -- start start 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc140b01f0 0x7efc14158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc140cc7e0 0x7efc1414a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc14146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc14144680 con 0x7efc14156370 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc14159600 con 0x7efc140cc7e0 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc14143de0 con 0x7efc140b01f0 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc14146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc14146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43504/0 (socket says 172.21.15.33:43504) 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.238+0000 7efc1ad9c700 1 -- 172.21.15.33:0/695631283 learned_addr learned my addr 172.21.15.33:0/695631283 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1a59b700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc140b01f0 0x7efc14158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc19d9a700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc140cc7e0 0x7efc1414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1ad9c700 1 -- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc140b01f0 msgr2=0x7efc14158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1ad9c700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc140b01f0 0x7efc14158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1ad9c700 1 -- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc140cc7e0 msgr2=0x7efc1414a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1ad9c700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc140cc7e0 0x7efc1414a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1ad9c700 1 -- 172.21.15.33:0/695631283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10009420 con 0x7efc14156370 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1ad9c700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc14146700 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00cc90 tx=0x7efc0c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00a8b0 con 0x7efc14156370 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c00aa10 con 0x7efc14156370 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00add0 con 0x7efc14156370 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.239+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc141440c0 con 0x7efc14156370 2023-10-06T05:23:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.240+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc141592d0 con 0x7efc14156370 2023-10-06T05:23:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.240+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1404c380 con 0x7efc14156370 2023-10-06T05:23:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.241+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc0c007300 con 0x7efc14156370 2023-10-06T05:23:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.242+0000 7efc0a7fc700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efbf8066f30 0x7efbf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.242+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efc0c084740 con 0x7efc14156370 2023-10-06T05:23:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.243+0000 7efc1a59b700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efbf8066f30 0x7efbf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.244+0000 7efc1a59b700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efbf8066f30 0x7efbf80693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc100030c0 tx=0x7efc10016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.247+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc0c051b00 con 0x7efc14156370 2023-10-06T05:23:44.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.449+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efc1406f310 con 0x7efc14156370 2023-10-06T05:23:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.453+0000 7efc0a7fc700 1 -- 172.21.15.33:0/695631283 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7efc0c055120 con 0x7efc14156370 2023-10-06T05:23:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efbf8066f30 msgr2=0x7efbf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efbf8066f30 0x7efbf80693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc100030c0 tx=0x7efc10016000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 msgr2=0x7efc14146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc14146700 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00cc90 tx=0x7efc0c001a80 comp rx=0 tx=0).stop 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 shutdown_connections 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efbf8066f30 0x7efbf80693e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc14156370 0x7efc14146700 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc140b01f0 0x7efc14158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 --2- 172.21.15.33:0/695631283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc140cc7e0 0x7efc1414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 >> 172.21.15.33:0/695631283 conn(0x7efc1406d030 msgr2=0x7efc14159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 shutdown_connections 2023-10-06T05:23:44.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.455+0000 7efc1b59d700 1 -- 172.21.15.33:0/695631283 wait complete. 2023-10-06T05:23:44.474 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:23:44.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:44.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:44.646 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:23:44.649 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:23:44.650 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.836+0000 7f46ab59e700 1 Processor -- start 2023-10-06T05:23:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.837+0000 7f46ab59e700 1 -- start start 2023-10-06T05:23:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46ab59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac155af0 0x7f46ac06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46ab59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46ac06a6a0 0x7f46ac0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46ab59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ac06ab60 con 0x7f46ac0c8b50 2023-10-06T05:23:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46ab59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ac06b7e0 con 0x7f46ac06a6a0 2023-10-06T05:23:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46ab59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac06ae40 con 0x7f46ac155af0 2023-10-06T05:23:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46a9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46ac06a6a0 0x7f46ac0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46a9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46ac06a6a0 0x7f46ac0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43512/0 (socket says 172.21.15.33:43512) 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46a9d9b700 1 -- 172.21.15.33:0/1954853260 learned_addr learned my addr 172.21.15.33:0/1954853260 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.838+0000 7f46aa59c700 1 --2- 172.21.15.33:0/1954853260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac155af0 0x7f46ac06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a9d9b700 1 -- 172.21.15.33:0/1954853260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac155af0 msgr2=0x7f46ac06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a9d9b700 1 --2- 172.21.15.33:0/1954853260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac155af0 0x7f46ac06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a9d9b700 1 -- 172.21.15.33:0/1954853260 >> v1:172.21.15.167:6789/0 conn(0x7f46ac0c8b50 legacy=0x7f46ac1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a9d9b700 1 -- 172.21.15.33:0/1954853260 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac06b120 con 0x7f46ac06a6a0 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a9d9b700 1 --2- 172.21.15.33:0/1954853260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac06a6a0 0x7f46ac0c7210 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f46a00099b0 tx=0x7f46a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe78ae94503c4b96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a959a700 1 -- 172.21.15.33:0/1954853260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000b860 con 0x7f46ac06a6a0 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a959a700 1 -- 172.21.15.33:0/1954853260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a000b9c0 con 0x7f46ac06a6a0 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.839+0000 7f46a959a700 1 -- 172.21.15.33:0/1954853260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000bd60 con 0x7f46ac06a6a0 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 -- 172.21.15.33:0/1954853260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac06a6a0 msgr2=0x7f46ac0c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 --2- 172.21.15.33:0/1954853260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac06a6a0 0x7f46ac0c7210 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f46a00099b0 tx=0x7f46a0009250 comp rx=0 tx=0).stop 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 -- 172.21.15.33:0/1954853260 shutdown_connections 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 --2- 172.21.15.33:0/1954853260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac155af0 0x7f46ac06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 --2- 172.21.15.33:0/1954853260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac06a6a0 0x7f46ac0c7210 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 -- 172.21.15.33:0/1954853260 >> 172.21.15.33:0/1954853260 conn(0x7f46ac06d030 msgr2=0x7f46ac06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 -- 172.21.15.33:0/1954853260 shutdown_connections 2023-10-06T05:23:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.840+0000 7f46ab59e700 1 -- 172.21.15.33:0/1954853260 wait complete. 2023-10-06T05:23:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.841+0000 7f46ab59e700 1 Processor -- start 2023-10-06T05:23:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.842+0000 7f46ab59e700 1 -- start start 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.842+0000 7f46ab59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac06a6a0 0x7f46ac1698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46ab59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46ac0c8b50 0x7f46ac0c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46ab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 0x7f46ac0a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46ab59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0acac0 con 0x7f46ac155af0 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46ab59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0ad360 con 0x7f46ac0c8b50 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46ab59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0afe30 con 0x7f46ac06a6a0 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 0x7f46ac0a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 0x7f46ac0a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43522/0 (socket says 172.21.15.33:43522) 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 -- 172.21.15.33:0/3410814889 learned_addr learned my addr 172.21.15.33:0/3410814889 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46a9d9b700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46ac0c8b50 0x7f46ac0c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aa59c700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac06a6a0 0x7f46ac1698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 -- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac06a6a0 msgr2=0x7f46ac1698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac06a6a0 0x7f46ac1698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 -- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46ac0c8b50 msgr2=0x7f46ac0c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46ac0c8b50 0x7f46ac0c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.843+0000 7f46aad9d700 1 -- 172.21.15.33:0/3410814889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00094f0 con 0x7f46ac155af0 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.844+0000 7f46aad9d700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 0x7f46ac0a9cd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f46a400c940 tx=0x7f46a40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.844+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4012630 con 0x7f46ac155af0 2023-10-06T05:23:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.844+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a4012790 con 0x7f46ac155af0 2023-10-06T05:23:44.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.844+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4012b50 con 0x7f46ac155af0 2023-10-06T05:23:44.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.844+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac0acd10 con 0x7f46ac155af0 2023-10-06T05:23:44.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.844+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac169e30 con 0x7f46ac155af0 2023-10-06T05:23:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.845+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac069a60 con 0x7f46ac155af0 2023-10-06T05:23:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.846+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46a401b450 con 0x7f46ac155af0 2023-10-06T05:23:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.847+0000 7f469a7fc700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f468c066f30 0x7f468c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.847+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f46a40844c0 con 0x7f46ac155af0 2023-10-06T05:23:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.847+0000 7f46aa59c700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f468c066f30 0x7f468c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.848+0000 7f46aa59c700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f468c066f30 0x7f468c0693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f469c006fd0 tx=0x7f469c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:44.851+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46a4055330 con 0x7f46ac155af0 2023-10-06T05:23:45.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.101+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46ac04c380 con 0x7f46ac155af0 2023-10-06T05:23:45.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.101+0000 7f469a7fc700 1 -- 172.21.15.33:0/3410814889 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f46a4016de0 con 0x7f46ac155af0 2023-10-06T05:23:45.103 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:45.103 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/386045330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":386045330},{"type":"v1","addr":"172.21.15.167:6836","nonce":386045330}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2940832954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2940832954},{"type":"v1","addr":"172.21.15.167:6837","nonce":2940832954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1022836897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1022836897},{"type":"v1","addr":"172.21.15.33:6835","nonce":1022836897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":7905,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/546713306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":546713306},{"type":"v1","addr":"172.21.15.33:6834","nonce":546713306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[]} 2023-10-06T05:23:45.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.103+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f468c066f30 msgr2=0x7f468c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.103+0000 7f46ab59e700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f468c066f30 0x7f468c0693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f469c006fd0 tx=0x7f469c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:45.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.103+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 msgr2=0x7f46ac0a9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.103+0000 7f46ab59e700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 0x7f46ac0a9cd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f46a400c940 tx=0x7f46a40016f0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 shutdown_connections 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f468c066f30 0x7f468c0693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ac155af0 0x7f46ac0a9cd0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46ac06a6a0 0x7f46ac1698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 --2- 172.21.15.33:0/3410814889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46ac0c8b50 0x7f46ac0c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 >> 172.21.15.33:0/3410814889 conn(0x7f46ac06d030 msgr2=0x7f46ac06eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 shutdown_connections 2023-10-06T05:23:45.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.104+0000 7f46ab59e700 1 -- 172.21.15.33:0/3410814889 wait complete. 2023-10-06T05:23:45.107 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 83 2023-10-06T05:23:45.123 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:23:45.123 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:23:45.123 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:23:45.123 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:45.124 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:23:45.124 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:23:45.124 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:23:45.124 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:45.124 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:23:45.124 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:23:45.125 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:23:45.125 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:45.125 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:23:45.125 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:23:45.125 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:23:45.125 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:45.137 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:23:45.137 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:23:45.138 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:23:45.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:23:45.140 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:23:45.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:23:45.141 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:23:45.141 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:23:45.182 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:23:45.182 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:45.185 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:23:45.185 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:45.185 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:23:45.186 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:45.187 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:23:45.188 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:45.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:23:45.319 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:23:45.320 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:23:45.323 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:23:45.324 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:23:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.384+0000 7f24b719d700 1 Processor -- start 2023-10-06T05:23:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.384+0000 7f24b719d700 1 -- start start 2023-10-06T05:23:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.385+0000 7f24b719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f24b0159400 0x7f24b00cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.385+0000 7f24b719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f24b006cc70 0x7f24b00cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.385+0000 7f24b719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b00d0ed0 con 0x7f24b01597d0 2023-10-06T05:23:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.385+0000 7f24b719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b00b1160 con 0x7f24b006cc70 2023-10-06T05:23:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.385+0000 7f24b719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b00cb6d0 con 0x7f24b0159400 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f24b006cc70 0x7f24b00cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f24b006cc70 0x7f24b00cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43568/0 (socket says 172.21.15.33:43568) 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 -- 172.21.15.33:0/1158865908 learned_addr learned my addr 172.21.15.33:0/1158865908 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b619b700 1 --2- 172.21.15.33:0/1158865908 >> v2:172.21.15.167:3300/0 conn(0x7f24b0159400 0x7f24b00cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 -- 172.21.15.33:0/1158865908 >> v2:172.21.15.167:3300/0 conn(0x7f24b0159400 msgr2=0x7f24b00cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 --2- 172.21.15.33:0/1158865908 >> v2:172.21.15.167:3300/0 conn(0x7f24b0159400 0x7f24b00cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 -- 172.21.15.33:0/1158865908 >> v1:172.21.15.167:6789/0 conn(0x7f24b01597d0 legacy=0x7f24b006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 -- 172.21.15.33:0/1158865908 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00cb9b0 con 0x7f24b006cc70 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b619b700 1 --2- 172.21.15.33:0/1158865908 >> v2:172.21.15.167:3300/0 conn(0x7f24b0159400 0x7f24b00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.386+0000 7f24b699c700 1 --2- 172.21.15.33:0/1158865908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b006cc70 0x7f24b00cb0a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f24b006c030 tx=0x7f24a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3615234a7366c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b5199700 1 -- 172.21.15.33:0/1158865908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800e070 con 0x7f24b006cc70 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b5199700 1 -- 172.21.15.33:0/1158865908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a800ba10 con 0x7f24b006cc70 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b5199700 1 -- 172.21.15.33:0/1158865908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80092e0 con 0x7f24b006cc70 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 -- 172.21.15.33:0/1158865908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b006cc70 msgr2=0x7f24b00cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 --2- 172.21.15.33:0/1158865908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b006cc70 0x7f24b00cb0a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f24b006c030 tx=0x7f24a8009840 comp rx=0 tx=0).stop 2023-10-06T05:23:45.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 -- 172.21.15.33:0/1158865908 shutdown_connections 2023-10-06T05:23:45.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 --2- 172.21.15.33:0/1158865908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b006cc70 0x7f24b00cb0a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 --2- 172.21.15.33:0/1158865908 >> v2:172.21.15.167:3300/0 conn(0x7f24b0159400 0x7f24b00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 -- 172.21.15.33:0/1158865908 >> 172.21.15.33:0/1158865908 conn(0x7f24b006e4b0 msgr2=0x7f24b00d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 -- 172.21.15.33:0/1158865908 shutdown_connections 2023-10-06T05:23:45.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.387+0000 7f24b719d700 1 -- 172.21.15.33:0/1158865908 wait complete. 2023-10-06T05:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.389+0000 7f24b719d700 1 Processor -- start 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.389+0000 7f24b719d700 1 -- start start 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b006cc70 0x7f24b0152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 0x7f24b014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b01597d0 0x7f24b014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b014c930 con 0x7f24b0159400 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01577c0 con 0x7f24b01597d0 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b0157aa0 con 0x7f24b006cc70 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 0x7f24b014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 0x7f24b014beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43570/0 (socket says 172.21.15.33:43570) 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.390+0000 7f24b599a700 1 -- 172.21.15.33:0/561669639 learned_addr learned my addr 172.21.15.33:0/561669639 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b699c700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b01597d0 0x7f24b014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b619b700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b006cc70 0x7f24b0152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b599a700 1 -- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b006cc70 msgr2=0x7f24b0152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b599a700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b006cc70 0x7f24b0152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b599a700 1 -- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b01597d0 msgr2=0x7f24b014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b599a700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b01597d0 0x7f24b014c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b599a700 1 -- 172.21.15.33:0/561669639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8009ae0 con 0x7f24b0159400 2023-10-06T05:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b699c700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b01597d0 0x7f24b014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b619b700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b006cc70 0x7f24b0152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b599a700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 0x7f24b014beb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f24a000b5e0 tx=0x7f24a00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a000f510 con 0x7f24b0159400 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a000f690 con 0x7f24b0159400 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a000fa50 con 0x7f24b0159400 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b0157d80 con 0x7f24b0159400 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.391+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b00b3370 con 0x7f24b0159400 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.392+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b004c380 con 0x7f24b0159400 2023-10-06T05:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.393+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24a001a400 con 0x7f24b0159400 2023-10-06T05:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.394+0000 7f24a67fc700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.394+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f24a0084090 con 0x7f24b0159400 2023-10-06T05:23:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.395+0000 7f24b619b700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.395+0000 7f24b619b700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f24ac008b20 tx=0x7f24ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.398+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24a0054e50 con 0x7f24b0159400 2023-10-06T05:23:45.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:45.616+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24b00cbc90 con 0x7f24b0159400 2023-10-06T05:23:46.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.031+0000 7f24a67fc700 1 -- 172.21.15.33:0/561669639 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24a00549e0 con 0x7f24b0159400 2023-10-06T05:23:46.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 msgr2=0x7f24940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f24ac008b20 tx=0x7f24ac008040 comp rx=0 tx=0).stop 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 msgr2=0x7f24b014beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 0x7f24b014beb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f24a000b5e0 tx=0x7f24a00090f0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 shutdown_connections 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2494066f30 0x7f24940693e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b01597d0 0x7f24b014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24b006cc70 0x7f24b0152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 --2- 172.21.15.33:0/561669639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0159400 0x7f24b014beb0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 >> 172.21.15.33:0/561669639 conn(0x7f24b006e4b0 msgr2=0x7f24b00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 shutdown_connections 2023-10-06T05:23:46.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.035+0000 7f24b719d700 1 -- 172.21.15.33:0/561669639 wait complete. 2023-10-06T05:23:46.038 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:23:46.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:23:46.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.244+0000 7f8921f9d700 1 Processor -- start 2023-10-06T05:23:46.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.244+0000 7f8921f9d700 1 -- start start 2023-10-06T05:23:46.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8921f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f891c13faa0 0x7f891c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8921f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f891c06aad0 0x7f891c06af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8921f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c0c9cb0 con 0x7f891c0c7940 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8921f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0ca270 con 0x7f891c06aad0 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8921f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c06b840 con 0x7f891c13faa0 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f892179c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f891c06aad0 0x7f891c06af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8920f9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f891c13faa0 0x7f891c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8920f9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f891c13faa0 0x7f891c13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:38586/0 (socket says 172.21.15.33:38586) 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f8920f9b700 1 -- 172.21.15.33:0/2919858874 learned_addr learned my addr 172.21.15.33:0/2919858874 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f892179c700 1 --2- 172.21.15.33:0/2919858874 >> v2:172.21.15.33:3300/0 conn(0x7f891c06aad0 0x7f891c06af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43586/0 (socket says 172.21.15.33:43586) 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f892179c700 1 -- 172.21.15.33:0/2919858874 >> v2:172.21.15.167:3300/0 conn(0x7f891c13faa0 msgr2=0x7f891c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f892179c700 1 --2- 172.21.15.33:0/2919858874 >> v2:172.21.15.167:3300/0 conn(0x7f891c13faa0 0x7f891c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f892179c700 1 -- 172.21.15.33:0/2919858874 >> v1:172.21.15.167:6789/0 conn(0x7f891c0c7940 legacy=0x7f891c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.246+0000 7f891b7fe700 1 -- 172.21.15.33:0/2919858874 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781737876 0 0) 0x7f891c0c9cb0 con 0x7f891c0c7940 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f892179c700 1 -- 172.21.15.33:0/2919858874 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c06bb20 con 0x7f891c06aad0 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f892179c700 1 --2- 172.21.15.33:0/2919858874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c06aad0 0x7f891c06af30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f891c06fe50 tx=0x7f8914009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a30d8a53133216 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f891b7fe700 1 -- 172.21.15.33:0/2919858874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400e070 con 0x7f891c06aad0 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f891b7fe700 1 -- 172.21.15.33:0/2919858874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891400bb60 con 0x7f891c06aad0 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.247+0000 7f891b7fe700 1 -- 172.21.15.33:0/2919858874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914012430 con 0x7f891c06aad0 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 -- 172.21.15.33:0/2919858874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c06aad0 msgr2=0x7f891c06af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 --2- 172.21.15.33:0/2919858874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c06aad0 0x7f891c06af30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f891c06fe50 tx=0x7f8914009780 comp rx=0 tx=0).stop 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 -- 172.21.15.33:0/2919858874 shutdown_connections 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 --2- 172.21.15.33:0/2919858874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c06aad0 0x7f891c06af30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 --2- 172.21.15.33:0/2919858874 >> v2:172.21.15.167:3300/0 conn(0x7f891c13faa0 0x7f891c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 -- 172.21.15.33:0/2919858874 >> 172.21.15.33:0/2919858874 conn(0x7f891c06d030 msgr2=0x7f891c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 -- 172.21.15.33:0/2919858874 shutdown_connections 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.248+0000 7f8921f9d700 1 -- 172.21.15.33:0/2919858874 wait complete. 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.249+0000 7f8921f9d700 1 Processor -- start 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.249+0000 7f8921f9d700 1 -- start start 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.250+0000 7f8921f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f891c06aad0 0x7f891c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.250+0000 7f8921f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 0x7f891c0c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.250+0000 7f8921f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f891c13faa0 0x7f891c0c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.250+0000 7f8921f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c13cd60 con 0x7f891c0c7940 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.250+0000 7f8921f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0c3de0 con 0x7f891c13faa0 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.250+0000 7f8921f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0c40c0 con 0x7f891c06aad0 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 0x7f891c0c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 0x7f891c0c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43590/0 (socket says 172.21.15.33:43590) 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 -- 172.21.15.33:0/3168925171 learned_addr learned my addr 172.21.15.33:0/3168925171 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f892179c700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f891c13faa0 0x7f891c0c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f8920f9b700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f891c06aad0 0x7f891c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 -- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f891c06aad0 msgr2=0x7f891c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f891c06aad0 0x7f891c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 -- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f891c13faa0 msgr2=0x7f891c0c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f891c13faa0 0x7f891c0c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 -- 172.21.15.33:0/3168925171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8914009a20 con 0x7f891c0c7940 2023-10-06T05:23:46.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f891bfff700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 0x7f891c0c1a20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f890800c7e0 tx=0x7f8908007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.251+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908012700 con 0x7f891c0c7940 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.252+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8908012880 con 0x7f891c0c7940 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.252+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908012c40 con 0x7f891c0c7940 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.252+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0c43a0 con 0x7f891c0c7940 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.252+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c156440 con 0x7f891c0c7940 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.253+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c13cb30 con 0x7f891c0c7940 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.254+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f890801b400 con 0x7f891c0c7940 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.254+0000 7f8918ff9700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8900066f30 0x7f89000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.255+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8908083fb0 con 0x7f891c0c7940 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.255+0000 7f8920f9b700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8900066f30 0x7f89000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.255+0000 7f8920f9b700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8900066f30 0x7f89000693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8910006fd0 tx=0x7f891000b4f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.259+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f891c13cb30 con 0x7f891c0c7940 2023-10-06T05:23:46.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.545+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f891c13cb30 con 0x7f891c0c7940 2023-10-06T05:23:46.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.545+0000 7f8918ff9700 1 -- 172.21.15.33:0/3168925171 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f891c13cb30 con 0x7f891c0c7940 2023-10-06T05:23:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:46.547 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:23:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:23:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:23:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:23:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:23:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:23:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:23:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:23:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:23:46.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:23:46.561 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.562 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:23:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.564 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:23:46.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:46.566 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:23:46.566 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:23:46.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8900066f30 msgr2=0x7f89000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8900066f30 0x7f89000693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8910006fd0 tx=0x7f891000b4f0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 msgr2=0x7f891c0c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 0x7f891c0c1a20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f890800c7e0 tx=0x7f8908007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 shutdown_connections 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8900066f30 0x7f89000693e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f891c13faa0 0x7f891c0c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f891c06aad0 0x7f891c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 --2- 172.21.15.33:0/3168925171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f891c0c7940 0x7f891c0c1a20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 >> 172.21.15.33:0/3168925171 conn(0x7f891c06d030 msgr2=0x7f891c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 shutdown_connections 2023-10-06T05:23:46.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.548+0000 7f8921f9d700 1 -- 172.21.15.33:0/3168925171 wait complete. 2023-10-06T05:23:46.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.754+0000 7fb1fb59e700 1 Processor -- start 2023-10-06T05:23:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.755+0000 7fb1fb59e700 1 -- start start 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc0c82a0 0x7fb1fc1641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1fb59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc06c820 con 0x7fb1fc0c6340 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1fb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc06b670 con 0x7fb1fc159520 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc06a160 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1f9d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb1fc0c6340 0x7fb1fc0c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53578/0 (socket says 172.21.15.33:53578) 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1f9d9b700 1 -- 172.21.15.33:0/2573361451 learned_addr learned my addr 172.21.15.33:0/2573361451 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.756+0000 7fb1fad9d700 1 --2- 172.21.15.33:0/2573361451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc0c82a0 0x7fb1fc1641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992197978 0 0) 0x7fb1fc06c820 con 0x7fb1fc0c6340 2023-10-06T05:23:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e4003570 con 0x7fb1fc0c6340 2023-10-06T05:23:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973834659 0 0) 0x7fb1fc06b670 con 0x7fb1fc159520 2023-10-06T05:23:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc06c820 con 0x7fb1fc159520 2023-10-06T05:23:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1fad9d700 1 -- 172.21.15.33:0/2573361451 >> v1:172.21.15.167:6789/0 conn(0x7fb1fc159520 legacy=0x7fb1fc0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1fad9d700 1 -- 172.21.15.33:0/2573361451 >> v1:172.21.15.33:6789/0 conn(0x7fb1fc0c6340 legacy=0x7fb1fc0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1fad9d700 1 -- 172.21.15.33:0/2573361451 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc0c8800 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1fad9d700 1 --2- 172.21.15.33:0/2573361451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc0c82a0 0x7fb1fc1641a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e90 tx=0x7fb1f4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7977d223712f76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.757+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400e070 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f400bb80 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1f959a700 1 -- 172.21.15.33:0/2573361451 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4012430 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2573361451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc0c82a0 msgr2=0x7fb1fc1641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2573361451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc0c82a0 0x7fb1fc1641a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e90 tx=0x7fb1f4009780 comp rx=0 tx=0).stop 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2573361451 shutdown_connections 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2573361451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc0c82a0 0x7fb1fc1641a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2573361451 >> 172.21.15.33:0/2573361451 conn(0x7fb1fc0cc930 msgr2=0x7fb1fc0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2573361451 shutdown_connections 2023-10-06T05:23:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.758+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2573361451 wait complete. 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.760+0000 7fb1fb59e700 1 Processor -- start 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.760+0000 7fb1fb59e700 1 -- start start 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1fc0c6340 0x7fb1fc169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 0x7fb1fc0aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc159520 0x7fb1fc0af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc15b970 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc15bc50 con 0x7fb1fc0c6340 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc15bf30 con 0x7fb1fc159520 2023-10-06T05:23:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 0x7fb1fc0aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 0x7fb1fc0aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43600/0 (socket says 172.21.15.33:43600) 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 -- 172.21.15.33:0/2544389143 learned_addr learned my addr 172.21.15.33:0/2544389143 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fad9d700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc159520 0x7fb1fc0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1fa59c700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1fc0c6340 0x7fb1fc169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 -- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc159520 msgr2=0x7fb1fc0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc159520 0x7fb1fc0af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 -- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1fc0c6340 msgr2=0x7fb1fc169b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1fc0c6340 0x7fb1fc169b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.761+0000 7fb1f9d9b700 1 -- 172.21.15.33:0/2544389143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4009a20 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.762+0000 7fb1f9d9b700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 0x7fb1fc0aeb00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c790 tx=0x7fb1f0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.762+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0005b30 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.762+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0005c90 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.762+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc15c210 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.762+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0007300 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.762+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc15ac10 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.763+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc0ac350 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.764+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1f001b3b0 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.765+0000 7fb1ea7fc700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1dc066f30 0x7fb1dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.765+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb1f0083910 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.765+0000 7fb1fa59c700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1dc066f30 0x7fb1dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.766+0000 7fb1fa59c700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1dc066f30 0x7fb1dc0693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ac2e0 tx=0x7fb1ec007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.769+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1fc0ac350 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.972+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1fc0ac350 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.973+0000 7fb1ea7fc700 1 -- 172.21.15.33:0/2544389143 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fb1fc0ac350 con 0x7fb1fc0c82a0 2023-10-06T05:23:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:46.975 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":125,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:46.037178+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1dc066f30 msgr2=0x7fb1dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1dc066f30 0x7fb1dc0693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ac2e0 tx=0x7fb1ec007d50 comp rx=0 tx=0).stop 2023-10-06T05:23:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 msgr2=0x7fb1fc0aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 0x7fb1fc0aeb00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c790 tx=0x7fb1f0007e40 comp rx=0 tx=0).stop 2023-10-06T05:23:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 shutdown_connections 2023-10-06T05:23:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1dc066f30 0x7fb1dc0693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1fc159520 0x7fb1fc0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1fc0c6340 0x7fb1fc169b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 --2- 172.21.15.33:0/2544389143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1fc0c82a0 0x7fb1fc0aeb00 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 >> 172.21.15.33:0/2544389143 conn(0x7fb1fc0cc930 msgr2=0x7fb1fc0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.975+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 shutdown_connections 2023-10-06T05:23:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:46.976+0000 7fb1fb59e700 1 -- 172.21.15.33:0/2544389143 wait complete. 2023-10-06T05:23:46.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:23:46.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:23:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.187+0000 7f6f789fe700 1 Processor -- start 2023-10-06T05:23:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.187+0000 7f6f789fe700 1 -- start start 2023-10-06T05:23:47.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f789fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f74155a50 0x7f6f74155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f789fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f740afb20 0x7f6f740caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f789fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f740b06b0 con 0x7f6f740c9490 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f789fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f740c99e0 con 0x7f6f740afb20 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f789fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c9cc0 con 0x7f6f74155a50 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f7359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f740afb20 0x7f6f740caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f7359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f740afb20 0x7f6f740caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43614/0 (socket says 172.21.15.33:43614) 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.188+0000 7f6f7359e700 1 -- 172.21.15.33:0/4135200808 learned_addr learned my addr 172.21.15.33:0/4135200808 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.189+0000 7f6f72d9d700 1 --2- 172.21.15.33:0/4135200808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f74155a50 0x7f6f74155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.189+0000 7f6f7359e700 1 -- 172.21.15.33:0/4135200808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f74155a50 msgr2=0x7f6f74155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.189+0000 7f6f7359e700 1 --2- 172.21.15.33:0/4135200808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f74155a50 0x7f6f74155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.189+0000 7f6f7359e700 1 -- 172.21.15.33:0/4135200808 >> v1:172.21.15.167:6789/0 conn(0x7f6f740c9490 legacy=0x7f6f740af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.189+0000 7f6f7359e700 1 -- 172.21.15.33:0/4135200808 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740c9fa0 con 0x7f6f740afb20 2023-10-06T05:23:47.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.189+0000 7f6f7359e700 1 --2- 172.21.15.33:0/4135200808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f740afb20 0x7f6f740caa80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009f50 tx=0x7f6f6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f468e4ec5394a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f71d9b700 1 -- 172.21.15.33:0/4135200808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0092e0 con 0x7f6f740afb20 2023-10-06T05:23:47.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f71d9b700 1 -- 172.21.15.33:0/4135200808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00b9c0 con 0x7f6f740afb20 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f71d9b700 1 -- 172.21.15.33:0/4135200808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00bd80 con 0x7f6f740afb20 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 -- 172.21.15.33:0/4135200808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f740afb20 msgr2=0x7f6f740caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 --2- 172.21.15.33:0/4135200808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f740afb20 0x7f6f740caa80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009f50 tx=0x7f6f6c009840 comp rx=0 tx=0).stop 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 -- 172.21.15.33:0/4135200808 shutdown_connections 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 --2- 172.21.15.33:0/4135200808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f740afb20 0x7f6f740caa80 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 --2- 172.21.15.33:0/4135200808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f74155a50 0x7f6f74155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 -- 172.21.15.33:0/4135200808 >> 172.21.15.33:0/4135200808 conn(0x7f6f7406d030 msgr2=0x7f6f740cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 -- 172.21.15.33:0/4135200808 shutdown_connections 2023-10-06T05:23:47.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.190+0000 7f6f789fe700 1 -- 172.21.15.33:0/4135200808 wait complete. 2023-10-06T05:23:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.192+0000 7f6f789fe700 1 Processor -- start 2023-10-06T05:23:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.192+0000 7f6f789fe700 1 -- start start 2023-10-06T05:23:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f789fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 0x7f6f74152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f789fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f740c9490 0x7f6f74169090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f789fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f74155a50 0x7f6f740c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f789fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74148690 con 0x7f6f74155a50 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f789fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7414a9c0 con 0x7f6f740afb20 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f789fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c6360 con 0x7f6f740c9490 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f72d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 0x7f6f74152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f7259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f740c9490 0x7f6f74169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f7359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f74155a50 0x7f6f740c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f72d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 0x7f6f74152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:38622/0 (socket says 172.21.15.33:38622) 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.193+0000 7f6f72d9d700 1 -- 172.21.15.33:0/1247182228 learned_addr learned my addr 172.21.15.33:0/1247182228 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f72d9d700 1 -- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f740c9490 msgr2=0x7f6f74169090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f72d9d700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f740c9490 0x7f6f74169090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f72d9d700 1 -- 172.21.15.33:0/1247182228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f74155a50 msgr2=0x7f6f740c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f72d9d700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f74155a50 0x7f6f740c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f72d9d700 1 -- 172.21.15.33:0/1247182228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009ae0 con 0x7f6f740afb20 2023-10-06T05:23:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f7359e700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f74155a50 0x7f6f740c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f7259c700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f740c9490 0x7f6f74169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:47.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f72d9d700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 0x7f6f74152ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400ec80 tx=0x7f6f640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400c850 con 0x7f6f740afb20 2023-10-06T05:23:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400c9b0 con 0x7f6f740afb20 2023-10-06T05:23:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400cd50 con 0x7f6f740afb20 2023-10-06T05:23:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f74149db0 con 0x7f6f740afb20 2023-10-06T05:23:47.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.194+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7414a370 con 0x7f6f740afb20 2023-10-06T05:23:47.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.195+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7404c380 con 0x7f6f740afb20 2023-10-06T05:23:47.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.197+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f64013680 con 0x7f6f740afb20 2023-10-06T05:23:47.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.198+0000 7f6f5affd700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f60066f30 0x7f6f600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.198+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6f64084f10 con 0x7f6f740afb20 2023-10-06T05:23:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.198+0000 7f6f7259c700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f60066f30 0x7f6f600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.199+0000 7f6f7259c700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f60066f30 0x7f6f600693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6f74151390 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.202+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f640522d0 con 0x7f6f740afb20 2023-10-06T05:23:47.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.405+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f7406fb00 con 0x7f6f740afb20 2023-10-06T05:23:47.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.406+0000 7f6f5affd700 1 -- 172.21.15.33:0/1247182228 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f640558f0 con 0x7f6f740afb20 2023-10-06T05:23:47.408 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:23:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f60066f30 msgr2=0x7f6f600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f60066f30 0x7f6f600693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6f74151390 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2023-10-06T05:23:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 msgr2=0x7f6f74152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 0x7f6f74152ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400ec80 tx=0x7f6f640091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 shutdown_connections 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f60066f30 0x7f6f600693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f74155a50 0x7f6f740c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f740afb20 0x7f6f74152ea0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 --2- 172.21.15.33:0/1247182228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f740c9490 0x7f6f74169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 >> 172.21.15.33:0/1247182228 conn(0x7f6f7406d030 msgr2=0x7f6f740ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 shutdown_connections 2023-10-06T05:23:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.408+0000 7f6f789fe700 1 -- 172.21.15.33:0/1247182228 wait complete. 2023-10-06T05:23:47.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:23:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.615+0000 7f855679d700 1 Processor -- start 2023-10-06T05:23:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.615+0000 7f855679d700 1 -- start start 2023-10-06T05:23:47.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.616+0000 7f855679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85500ad460 0x7f85500ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.616+0000 7f855679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500aa7d0 con 0x7f85501595d0 2023-10-06T05:23:47.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.616+0000 7f855679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500a9bc0 con 0x7f85500ac180 2023-10-06T05:23:47.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f855679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500a9ea0 con 0x7f85500ad460 2023-10-06T05:23:47.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f855579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f85501595d0 0x7f85500aac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53586/0 (socket says 172.21.15.33:53586) 2023-10-06T05:23:47.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f855579b700 1 -- 172.21.15.33:0/1222635883 learned_addr learned my addr 172.21.15.33:0/1222635883 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:47.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8555f9c700 1 --2- 172.21.15.33:0/1222635883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85500ad460 0x7f85500ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8547fff700 1 -- 172.21.15.33:0/1222635883 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838594066 0 0) 0x7f85500aa7d0 con 0x7f85501595d0 2023-10-06T05:23:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8547fff700 1 -- 172.21.15.33:0/1222635883 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f853c003570 con 0x7f85501595d0 2023-10-06T05:23:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8555f9c700 1 -- 172.21.15.33:0/1222635883 >> v1:172.21.15.167:6789/0 conn(0x7f85500ac180 legacy=0x7f85500acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8547fff700 1 -- 172.21.15.33:0/1222635883 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421507585 0 0) 0x7f85500a9bc0 con 0x7f85500ac180 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8555f9c700 1 -- 172.21.15.33:0/1222635883 >> v1:172.21.15.33:6789/0 conn(0x7f85501595d0 legacy=0x7f85500aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.617+0000 7f8555f9c700 1 -- 172.21.15.33:0/1222635883 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500aa180 con 0x7f85500ad460 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.618+0000 7f8555f9c700 1 --2- 172.21.15.33:0/1222635883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85500ad460 0x7f85500ac5f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8548009e90 tx=0x7f8548009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6a95e16182688b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.618+0000 7f8547fff700 1 -- 172.21.15.33:0/1222635883 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800e070 con 0x7f85500ad460 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.618+0000 7f8547fff700 1 -- 172.21.15.33:0/1222635883 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854800bc30 con 0x7f85500ad460 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.618+0000 7f8547fff700 1 -- 172.21.15.33:0/1222635883 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548012430 con 0x7f85500ad460 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 -- 172.21.15.33:0/1222635883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85500ad460 msgr2=0x7f85500ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 --2- 172.21.15.33:0/1222635883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85500ad460 0x7f85500ac5f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8548009e90 tx=0x7f8548009780 comp rx=0 tx=0).stop 2023-10-06T05:23:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 -- 172.21.15.33:0/1222635883 shutdown_connections 2023-10-06T05:23:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 --2- 172.21.15.33:0/1222635883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85500ad460 0x7f85500ac5f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 -- 172.21.15.33:0/1222635883 >> 172.21.15.33:0/1222635883 conn(0x7f855006d030 msgr2=0x7f85500cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 -- 172.21.15.33:0/1222635883 shutdown_connections 2023-10-06T05:23:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.619+0000 7f855679d700 1 -- 172.21.15.33:0/1222635883 wait complete. 2023-10-06T05:23:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.621+0000 7f855679d700 1 Processor -- start 2023-10-06T05:23:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.621+0000 7f855679d700 1 -- start start 2023-10-06T05:23:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 0x7f85501543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85500ad460 0x7f855015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85501595d0 0x7f8550156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855015f580 con 0x7f85500ac180 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855015f860 con 0x7f85500ad460 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855015ec10 con 0x7f85501595d0 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 0x7f85501543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 0x7f85501543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43632/0 (socket says 172.21.15.33:43632) 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 -- 172.21.15.33:0/1648820805 learned_addr learned my addr 172.21.15.33:0/1648820805 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f8554f9a700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85500ad460 0x7f855015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f8555f9c700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85501595d0 0x7f8550156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 -- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85501595d0 msgr2=0x7f8550156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85501595d0 0x7f8550156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 -- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85500ad460 msgr2=0x7f855015e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85500ad460 0x7f855015e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.622+0000 7f855579b700 1 -- 172.21.15.33:0/1648820805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548009a20 con 0x7f85500ac180 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.623+0000 7f855579b700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 0x7f85501543e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f854c0011e0 tx=0x7f854c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.623+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c005b30 con 0x7f85500ac180 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.623+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c005c90 con 0x7f85500ac180 2023-10-06T05:23:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.623+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c007320 con 0x7f85500ac180 2023-10-06T05:23:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.623+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855015eef0 con 0x7f85500ac180 2023-10-06T05:23:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.623+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855015e920 con 0x7f85500ac180 2023-10-06T05:23:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.624+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85500a96d0 con 0x7f85500ac180 2023-10-06T05:23:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.625+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f854c007690 con 0x7f85500ac180 2023-10-06T05:23:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.626+0000 7f85457fa700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8534066f30 0x7f85340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.626+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f854c0838e0 con 0x7f85500ac180 2023-10-06T05:23:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.626+0000 7f8554f9a700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8534066f30 0x7f85340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.627+0000 7f8554f9a700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8534066f30 0x7f85340693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8550154920 tx=0x7f8540009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.630+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f854c050ca0 con 0x7f85500ac180 2023-10-06T05:23:47.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.831+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f855004c380 con 0x7f85500ac180 2023-10-06T05:23:47.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.831+0000 7f85457fa700 1 -- 172.21.15.33:0/1648820805 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v125) v1 ==== 92+0+10 (secure 0 0 0) 0x7f854c017020 con 0x7f85500ac180 2023-10-06T05:23:47.833 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:23:47.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.833+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8534066f30 msgr2=0x7f85340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.833+0000 7f855679d700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8534066f30 0x7f85340693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8550154920 tx=0x7f8540009040 comp rx=0 tx=0).stop 2023-10-06T05:23:47.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.833+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 msgr2=0x7f85501543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.833+0000 7f855679d700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 0x7f85501543e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f854c0011e0 tx=0x7f854c007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 shutdown_connections 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8534066f30 0x7f85340693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85501595d0 0x7f8550156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85500ac180 0x7f85501543e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 --2- 172.21.15.33:0/1648820805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85500ad460 0x7f855015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 >> 172.21.15.33:0/1648820805 conn(0x7f855006d030 msgr2=0x7f85500cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 shutdown_connections 2023-10-06T05:23:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:47.834+0000 7f855679d700 1 -- 172.21.15.33:0/1648820805 wait complete. 2023-10-06T05:23:47.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:23:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.040+0000 7fa1d499d700 1 Processor -- start 2023-10-06T05:23:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.040+0000 7fa1d499d700 1 -- start start 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1d499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00ccfc0 0x7fa1d00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1d499d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa1d00cf3b0 0x7fa1d01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1d499d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa1d00cc750 0x7fa1d01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1d499d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d00cbd80 con 0x7fa1d00cf3b0 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1d499d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d00af5f0 con 0x7fa1d00cc750 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1d499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d00af8d0 con 0x7fa1d00ccfc0 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1ceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa1d00cf3b0 0x7fa1d01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1ceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa1d00cf3b0 0x7fa1d01594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43646/0 (socket says 172.21.15.33:43646) 2023-10-06T05:23:48.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.041+0000 7fa1ceffd700 1 -- 172.21.15.33:0/1204734071 learned_addr learned my addr 172.21.15.33:0/1204734071 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1cffff700 1 --2- 172.21.15.33:0/1204734071 >> v2:172.21.15.167:3300/0 conn(0x7fa1d00cc750 0x7fa1d01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1cf7fe700 1 --2- 172.21.15.33:0/1204734071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00ccfc0 0x7fa1d00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ceffd700 1 -- 172.21.15.33:0/1204734071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00ccfc0 msgr2=0x7fa1d00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ceffd700 1 --2- 172.21.15.33:0/1204734071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00ccfc0 0x7fa1d00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ceffd700 1 -- 172.21.15.33:0/1204734071 >> v2:172.21.15.167:3300/0 conn(0x7fa1d00cc750 msgr2=0x7fa1d01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ceffd700 1 --2- 172.21.15.33:0/1204734071 >> v2:172.21.15.167:3300/0 conn(0x7fa1d00cc750 0x7fa1d01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ceffd700 1 -- 172.21.15.33:0/1204734071 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d00afbb0 con 0x7fa1d00cf3b0 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1cffff700 1 --2- 172.21.15.33:0/1204734071 >> v2:172.21.15.167:3300/0 conn(0x7fa1d00cc750 0x7fa1d01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ceffd700 1 --2- 172.21.15.33:0/1204734071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cf3b0 0x7fa1d01594a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00cdb60 tx=0x7fa1c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466482b947f3c347 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ce7fc700 1 -- 172.21.15.33:0/1204734071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400b860 con 0x7fa1d00cf3b0 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ce7fc700 1 -- 172.21.15.33:0/1204734071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c400b9c0 con 0x7fa1d00cf3b0 2023-10-06T05:23:48.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.042+0000 7fa1ce7fc700 1 -- 172.21.15.33:0/1204734071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400bd60 con 0x7fa1d00cf3b0 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 -- 172.21.15.33:0/1204734071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cf3b0 msgr2=0x7fa1d01594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 --2- 172.21.15.33:0/1204734071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cf3b0 0x7fa1d01594a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00cdb60 tx=0x7fa1c4009250 comp rx=0 tx=0).stop 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 -- 172.21.15.33:0/1204734071 shutdown_connections 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 --2- 172.21.15.33:0/1204734071 >> v2:172.21.15.167:3300/0 conn(0x7fa1d00cc750 0x7fa1d01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 --2- 172.21.15.33:0/1204734071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00ccfc0 0x7fa1d00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 --2- 172.21.15.33:0/1204734071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cf3b0 0x7fa1d01594a0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 -- 172.21.15.33:0/1204734071 >> 172.21.15.33:0/1204734071 conn(0x7fa1d006cff0 msgr2=0x7fa1d006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 -- 172.21.15.33:0/1204734071 shutdown_connections 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.043+0000 7fa1d499d700 1 -- 172.21.15.33:0/1204734071 wait complete. 2023-10-06T05:23:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.045+0000 7fa1d499d700 1 Processor -- start 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.045+0000 7fa1d499d700 1 -- start start 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1d499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 0x7fa1d016cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1d499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d00ccfc0 0x7fa1d00c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1d499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00c65d0 0x7fa1d0169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1d499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d016ace0 con 0x7fa1d00cc750 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1d499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01548d0 con 0x7fa1d00ccfc0 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1d499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d00c6a70 con 0x7fa1d00c65d0 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 0x7fa1d016cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 0x7fa1d016cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43652/0 (socket says 172.21.15.33:43652) 2023-10-06T05:23:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 -- 172.21.15.33:0/2885481662 learned_addr learned my addr 172.21.15.33:0/2885481662 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1ceffd700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d00ccfc0 0x7fa1d00c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cffff700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00c65d0 0x7fa1d0169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 -- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00c65d0 msgr2=0x7fa1d0169c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00c65d0 0x7fa1d0169c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 -- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d00ccfc0 msgr2=0x7fa1d00c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d00ccfc0 0x7fa1d00c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 -- 172.21.15.33:0/2885481662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c40094f0 con 0x7fa1d00cc750 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.046+0000 7fa1cf7fe700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 0x7fa1d016cc10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d890 tx=0x7fa1c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.047+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00125f0 con 0x7fa1d00cc750 2023-10-06T05:23:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.047+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c0012770 con 0x7fa1d00cc750 2023-10-06T05:23:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.047+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0012b30 con 0x7fa1d00cc750 2023-10-06T05:23:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.047+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d016daa0 con 0x7fa1d00cc750 2023-10-06T05:23:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.047+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d016aa00 con 0x7fa1d00cc750 2023-10-06T05:23:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.048+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d004c380 con 0x7fa1d00cc750 2023-10-06T05:23:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.049+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1c001e460 con 0x7fa1d00cc750 2023-10-06T05:23:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.049+0000 7fa1b77fe700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1bc066f30 0x7fa1bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.050+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa1c00849b0 con 0x7fa1d00cc750 2023-10-06T05:23:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.050+0000 7fa1ceffd700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1bc066f30 0x7fa1bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.051+0000 7fa1ceffd700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1bc066f30 0x7fa1bc0693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00b0250 tx=0x7fa1c4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.054+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1c0051d70 con 0x7fa1d00cc750 2023-10-06T05:23:48.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.254+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa1d00d1310 con 0x7fa1d00cc750 2023-10-06T05:23:48.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.255+0000 7fa1b77fe700 1 -- 172.21.15.33:0/2885481662 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa1c0055390 con 0x7fa1d00cc750 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:23:48.257 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:23:48.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:23:48.258 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:23:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.257+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1bc066f30 msgr2=0x7fa1bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.257+0000 7fa1d499d700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1bc066f30 0x7fa1bc0693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00b0250 tx=0x7fa1c4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.257+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 msgr2=0x7fa1d016cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.257+0000 7fa1d499d700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 0x7fa1d016cc10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d890 tx=0x7fa1c0008040 comp rx=0 tx=0).stop 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 shutdown_connections 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa1bc066f30 0x7fa1bc0693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1d00c65d0 0x7fa1d0169c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d00cc750 0x7fa1d016cc10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 --2- 172.21.15.33:0/2885481662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1d00ccfc0 0x7fa1d00c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 >> 172.21.15.33:0/2885481662 conn(0x7fa1d006cff0 msgr2=0x7fa1d01561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 shutdown_connections 2023-10-06T05:23:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.258+0000 7fa1d499d700 1 -- 172.21.15.33:0/2885481662 wait complete. 2023-10-06T05:23:48.276 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:23:48.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:23:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.466+0000 7f25f259d700 1 Processor -- start 2023-10-06T05:23:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.466+0000 7f25f259d700 1 -- start start 2023-10-06T05:23:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f259d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25ec06c560 0x7f25ec06c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f259d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec0afc40 con 0x7f25ec0c9720 2023-10-06T05:23:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f259d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec15d7c0 con 0x7f25ec0ad520 2023-10-06T05:23:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f259d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec0ad9c0 con 0x7f25ec06c560 2023-10-06T05:23:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f159b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f25ec0c9720 0x7f25ec15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53592/0 (socket says 172.21.15.33:53592) 2023-10-06T05:23:48.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f159b700 1 -- 172.21.15.33:0/4164975093 learned_addr learned my addr 172.21.15.33:0/4164975093 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25f1d9c700 1 --2- 172.21.15.33:0/4164975093 >> v2:172.21.15.167:3300/0 conn(0x7f25ec06c560 0x7f25ec06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.468+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974340786 0 0) 0x7f25ec0afc40 con 0x7f25ec0c9720 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d8003570 con 0x7f25ec0c9720 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433283040 0 0) 0x7f25ec15d7c0 con 0x7f25ec0ad520 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec0afc40 con 0x7f25ec0ad520 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25f1d9c700 1 -- 172.21.15.33:0/4164975093 >> v1:172.21.15.167:6789/0 conn(0x7f25ec0ad520 legacy=0x7f25ec0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25f1d9c700 1 -- 172.21.15.33:0/4164975093 >> v1:172.21.15.33:6789/0 conn(0x7f25ec0c9720 legacy=0x7f25ec15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3629169065 0 0) 0x7f25d8003570 con 0x7f25ec0c9720 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25f1d9c700 1 -- 172.21.15.33:0/4164975093 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec159bc0 con 0x7f25ec06c560 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25f1d9c700 1 --2- 172.21.15.33:0/4164975093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec06c990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009e60 tx=0x7f25e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a23b987964ef89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.469+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400e070 con 0x7f25ec06c560 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e400bab0 con 0x7f25ec06c560 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25e3fff700 1 -- 172.21.15.33:0/4164975093 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400be70 con 0x7f25ec06c560 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 -- 172.21.15.33:0/4164975093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 msgr2=0x7f25ec06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 --2- 172.21.15.33:0/4164975093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec06c990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009e60 tx=0x7f25e4009780 comp rx=0 tx=0).stop 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 -- 172.21.15.33:0/4164975093 shutdown_connections 2023-10-06T05:23:48.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 --2- 172.21.15.33:0/4164975093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec06c990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 -- 172.21.15.33:0/4164975093 >> 172.21.15.33:0/4164975093 conn(0x7f25ec0675f0 msgr2=0x7f25ec0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 -- 172.21.15.33:0/4164975093 shutdown_connections 2023-10-06T05:23:48.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.470+0000 7f25f259d700 1 -- 172.21.15.33:0/4164975093 wait complete. 2023-10-06T05:23:48.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.472+0000 7f25f259d700 1 Processor -- start 2023-10-06T05:23:48.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.472+0000 7f25f259d700 1 -- start start 2023-10-06T05:23:48.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec1693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 0x7f25ec16a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25ec0c9720 0x7f25ec15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec159d90 con 0x7f25ec0ad520 2023-10-06T05:23:48.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec155760 con 0x7f25ec06c560 2023-10-06T05:23:48.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec16ab30 con 0x7f25ec0c9720 2023-10-06T05:23:48.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 0x7f25ec16a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 0x7f25ec16a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43668/0 (socket says 172.21.15.33:43668) 2023-10-06T05:23:48.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 -- 172.21.15.33:0/3846822306 learned_addr learned my addr 172.21.15.33:0/3846822306 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f159b700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec1693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f1d9c700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25ec0c9720 0x7f25ec15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 -- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25ec0c9720 msgr2=0x7f25ec15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25ec0c9720 0x7f25ec15b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 -- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 msgr2=0x7f25ec1693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec1693e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f0d9a700 1 -- 172.21.15.33:0/3846822306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e4009a20 con 0x7f25ec0ad520 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.473+0000 7f25f159b700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec1693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.474+0000 7f25f0d9a700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 0x7f25ec16a5f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00c7c0 tx=0x7f25dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.474+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc010510 con 0x7f25ec0ad520 2023-10-06T05:23:48.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.474+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc010670 con 0x7f25ec0ad520 2023-10-06T05:23:48.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.474+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec1699e0 con 0x7f25ec0ad520 2023-10-06T05:23:48.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.474+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc010a10 con 0x7f25ec0ad520 2023-10-06T05:23:48.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.474+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec169fa0 con 0x7f25ec0ad520 2023-10-06T05:23:48.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.475+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec159f40 con 0x7f25ec0ad520 2023-10-06T05:23:48.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.476+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25dc01b3b0 con 0x7f25ec0ad520 2023-10-06T05:23:48.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.477+0000 7f25e17fa700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25d0066f30 0x7f25d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.477+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f25dc083c60 con 0x7f25ec0ad520 2023-10-06T05:23:48.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.477+0000 7f25f159b700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25d0066f30 0x7f25d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.478+0000 7f25f159b700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25d0066f30 0x7f25d00693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f25e80011e0 tx=0x7f25e8007af0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.481+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25ec159f40 con 0x7f25ec0ad520 2023-10-06T05:23:48.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:48.684+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f25ec159f40 con 0x7f25ec0ad520 2023-10-06T05:23:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.049+0000 7f25e17fa700 1 -- 172.21.15.33:0/3846822306 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) v1 ==== 138+0+0 (secure 0 0 0) 0x7f25ec159f40 con 0x7f25ec0ad520 2023-10-06T05:23:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.053+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25d0066f30 msgr2=0x7f25d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.053+0000 7f25f259d700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25d0066f30 0x7f25d00693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f25e80011e0 tx=0x7f25e8007af0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.053+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 msgr2=0x7f25ec16a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.053+0000 7f25f259d700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 0x7f25ec16a5f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00c7c0 tx=0x7f25dc007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 shutdown_connections 2023-10-06T05:23:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f25d0066f30 0x7f25d00693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25ec0c9720 0x7f25ec15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25ec06c560 0x7f25ec1693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 --2- 172.21.15.33:0/3846822306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25ec0ad520 0x7f25ec16a5f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 >> 172.21.15.33:0/3846822306 conn(0x7f25ec0675f0 msgr2=0x7f25ec06b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 shutdown_connections 2023-10-06T05:23:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.054+0000 7f25f259d700 1 -- 172.21.15.33:0/3846822306 wait complete. 2023-10-06T05:23:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:23:49.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:23:49.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:49.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.269+0000 7f73ef5ff700 1 Processor -- start 2023-10-06T05:23:49.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.270+0000 7f73ef5ff700 1 -- start start 2023-10-06T05:23:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73ef5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e806a630 0x7f73e8162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73ef5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e80c75e0 con 0x7f73e80ae910 2023-10-06T05:23:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73ef5ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e80c8a80 con 0x7f73e8157740 2023-10-06T05:23:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73ef5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80c69d0 con 0x7f73e806a630 2023-10-06T05:23:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73ecb9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73e80ae910 0x7f73e8069f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53598/0 (socket says 172.21.15.33:53598) 2023-10-06T05:23:49.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73ecb9a700 1 -- 172.21.15.33:0/37031064 learned_addr learned my addr 172.21.15.33:0/37031064 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.271+0000 7f73edb9c700 1 --2- 172.21.15.33:0/37031064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e806a630 0x7f73e8162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460681300 0 0) 0x7f73e80c75e0 con 0x7f73e80ae910 2023-10-06T05:23:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d4003570 con 0x7f73e80ae910 2023-10-06T05:23:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160089348 0 0) 0x7f73e80c8a80 con 0x7f73e8157740 2023-10-06T05:23:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e80c75e0 con 0x7f73e8157740 2023-10-06T05:23:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73edb9c700 1 -- 172.21.15.33:0/37031064 >> v1:172.21.15.167:6789/0 conn(0x7f73e8157740 legacy=0x7f73e80ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73edb9c700 1 -- 172.21.15.33:0/37031064 >> v1:172.21.15.33:6789/0 conn(0x7f73e80ae910 legacy=0x7f73e8069f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3039584413 0 0) 0x7f73d4003570 con 0x7f73e80ae910 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73edb9c700 1 -- 172.21.15.33:0/37031064 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80c6cb0 con 0x7f73e806a630 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.272+0000 7f73edb9c700 1 --2- 172.21.15.33:0/37031064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e806a630 0x7f73e8162510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009e90 tx=0x7f73e0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92985f40d8cc2db4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.273+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000e070 con 0x7f73e806a630 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.273+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e000b870 con 0x7f73e806a630 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.273+0000 7f73dffff700 1 -- 172.21.15.33:0/37031064 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8001470 con 0x7f73e806a630 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.273+0000 7f73ef5ff700 1 -- 172.21.15.33:0/37031064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e806a630 msgr2=0x7f73e8162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.273+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/37031064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e806a630 0x7f73e8162510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009e90 tx=0x7f73e0009780 comp rx=0 tx=0).stop 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.274+0000 7f73ef5ff700 1 -- 172.21.15.33:0/37031064 shutdown_connections 2023-10-06T05:23:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.274+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/37031064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e806a630 0x7f73e8162510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.274+0000 7f73ef5ff700 1 -- 172.21.15.33:0/37031064 >> 172.21.15.33:0/37031064 conn(0x7f73e806d030 msgr2=0x7f73e80cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.274+0000 7f73ef5ff700 1 -- 172.21.15.33:0/37031064 shutdown_connections 2023-10-06T05:23:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.274+0000 7f73ef5ff700 1 -- 172.21.15.33:0/37031064 wait complete. 2023-10-06T05:23:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.275+0000 7f73ef5ff700 1 Processor -- start 2023-10-06T05:23:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.275+0000 7f73ef5ff700 1 -- start start 2023-10-06T05:23:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ef5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 0x7f73e80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ef5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e80ae910 0x7f73e81544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ef5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e8157740 0x7f73e80c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ef5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8155190 con 0x7f73e806a630 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ef5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80b0dd0 con 0x7f73e80ae910 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ef5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80a59c0 con 0x7f73e8157740 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.276+0000 7f73ed39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 0x7f73e80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73ed39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 0x7f73e80a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43684/0 (socket says 172.21.15.33:43684) 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73ed39b700 1 -- 172.21.15.33:0/2768835660 learned_addr learned my addr 172.21.15.33:0/2768835660 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e8157740 0x7f73e80c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73ecb9a700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e80ae910 0x7f73e81544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 -- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e80ae910 msgr2=0x7f73e81544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e80ae910 0x7f73e81544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 -- 172.21.15.33:0/2768835660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 msgr2=0x7f73e80a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 0x7f73e80a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 -- 172.21.15.33:0/2768835660 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e0009a20 con 0x7f73e8157740 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73ed39b700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 0x7f73e80a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.277+0000 7f73edb9c700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e8157740 0x7f73e80c4870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73e0014000 tx=0x7f73e0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.278+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0012b50 con 0x7f73e8157740 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.278+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0012cb0 con 0x7f73e8157740 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.278+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e80b04a0 con 0x7f73e8157740 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.278+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e001e6a0 con 0x7f73e8157740 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.278+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e80b0a60 con 0x7f73e8157740 2023-10-06T05:23:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.279+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e80b0f80 con 0x7f73e8157740 2023-10-06T05:23:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.280+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73e001ea90 con 0x7f73e8157740 2023-10-06T05:23:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.281+0000 7f73dd7fa700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73cc066f30 0x7f73cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.281+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f73e0086230 con 0x7f73e8157740 2023-10-06T05:23:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.281+0000 7f73ed39b700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73cc066f30 0x7f73cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.282+0000 7f73ed39b700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73cc066f30 0x7f73cc0693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007fd0 tx=0x7f73e4007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.287+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73e80b0f80 con 0x7f73e8157740 2023-10-06T05:23:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:49.499+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f73e80b0f80 con 0x7f73e8157740 2023-10-06T05:23:50.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.060+0000 7f73dd7fa700 1 -- 172.21.15.33:0/2768835660 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) v1 ==== 170+0+0 (secure 0 0 0) 0x7f73e80b0f80 con 0x7f73e8157740 2023-10-06T05:23:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.066+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73cc066f30 msgr2=0x7f73cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.066+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73cc066f30 0x7f73cc0693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007fd0 tx=0x7f73e4007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.066+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e8157740 msgr2=0x7f73e80c4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.066+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e8157740 0x7f73e80c4870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73e0014000 tx=0x7f73e0017000 comp rx=0 tx=0).stop 2023-10-06T05:23:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.068+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 shutdown_connections 2023-10-06T05:23:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.068+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73cc066f30 0x7f73cc0693e0 secure :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007fd0 tx=0x7f73e4007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.068+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73e8157740 0x7f73e80c4870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.068+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73e806a630 0x7f73e80a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.068+0000 7f73ef5ff700 1 --2- 172.21.15.33:0/2768835660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73e80ae910 0x7f73e81544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.068+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 >> 172.21.15.33:0/2768835660 conn(0x7f73e806d030 msgr2=0x7f73e80c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.072+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 shutdown_connections 2023-10-06T05:23:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.074+0000 7f73ef5ff700 1 -- 172.21.15.33:0/2768835660 wait complete. 2023-10-06T05:23:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:23:50.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:23:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.292+0000 7f2845f9d700 1 Processor -- start 2023-10-06T05:23:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.293+0000 7f2845f9d700 1 -- start start 2023-10-06T05:23:50.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f2845f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28400aff90 0x7f28400a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f2845f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284015d3a0 con 0x7f284006c7b0 2023-10-06T05:23:50.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f2845f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28400b0470 con 0x7f284006cb80 2023-10-06T05:23:50.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f2845f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28400686c0 con 0x7f28400aff90 2023-10-06T05:23:50.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f2844f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f284006c7b0 0x7f2840067fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52330/0 (socket says 172.21.15.33:52330) 2023-10-06T05:23:50.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f2844f9b700 1 -- 172.21.15.33:0/275028153 learned_addr learned my addr 172.21.15.33:0/275028153 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:50.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.294+0000 7f283ffff700 1 --2- 172.21.15.33:0/275028153 >> v2:172.21.15.33:3300/0 conn(0x7f28400aff90 0x7f28400a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283ffff700 1 -- 172.21.15.33:0/275028153 >> v1:172.21.15.167:6789/0 conn(0x7f284006cb80 legacy=0x7f284015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283f7fe700 1 -- 172.21.15.33:0/275028153 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901720395 0 0) 0x7f284015d3a0 con 0x7f284006c7b0 2023-10-06T05:23:50.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283ffff700 1 -- 172.21.15.33:0/275028153 >> v1:172.21.15.33:6789/0 conn(0x7f284006c7b0 legacy=0x7f2840067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283ffff700 1 -- 172.21.15.33:0/275028153 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400b0750 con 0x7f28400aff90 2023-10-06T05:23:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283ffff700 1 --2- 172.21.15.33:0/275028153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28400aff90 0x7f28400a9890 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098e0 tx=0x7f282c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf1ac4cf534b4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283f7fe700 1 -- 172.21.15.33:0/275028153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c009e80 con 0x7f28400aff90 2023-10-06T05:23:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283f7fe700 1 -- 172.21.15.33:0/275028153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c00b860 con 0x7f28400aff90 2023-10-06T05:23:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.295+0000 7f283f7fe700 1 -- 172.21.15.33:0/275028153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00bbe0 con 0x7f28400aff90 2023-10-06T05:23:50.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 -- 172.21.15.33:0/275028153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28400aff90 msgr2=0x7f28400a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 --2- 172.21.15.33:0/275028153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28400aff90 0x7f28400a9890 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098e0 tx=0x7f282c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 -- 172.21.15.33:0/275028153 shutdown_connections 2023-10-06T05:23:50.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 --2- 172.21.15.33:0/275028153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28400aff90 0x7f28400a9890 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 -- 172.21.15.33:0/275028153 >> 172.21.15.33:0/275028153 conn(0x7f284006d670 msgr2=0x7f284006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:50.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 -- 172.21.15.33:0/275028153 shutdown_connections 2023-10-06T05:23:50.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.296+0000 7f2845f9d700 1 -- 172.21.15.33:0/275028153 wait complete. 2023-10-06T05:23:50.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.298+0000 7f2845f9d700 1 Processor -- start 2023-10-06T05:23:50.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.298+0000 7f2845f9d700 1 -- start start 2023-10-06T05:23:50.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2845f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 0x7f284013c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2845f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f284006cb80 0x7f2840150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2845f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28400aff90 0x7f2840151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2845f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284013ca90 con 0x7f284006c7b0 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2845f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2840151bb0 con 0x7f28400aff90 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2845f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2840139ba0 con 0x7f284006cb80 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 0x7f284013c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 0x7f284013c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56184/0 (socket says 172.21.15.33:56184) 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 -- 172.21.15.33:0/1297545428 learned_addr learned my addr 172.21.15.33:0/1297545428 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f283ffff700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f284006cb80 0x7f2840150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f284579c700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28400aff90 0x7f2840151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 -- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f284006cb80 msgr2=0x7f2840150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f284006cb80 0x7f2840150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 -- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28400aff90 msgr2=0x7f2840151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28400aff90 0x7f2840151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.299+0000 7f2844f9b700 1 -- 172.21.15.33:0/1297545428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009420 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.300+0000 7f2844f9b700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 0x7f284013c550 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f283400c810 tx=0x7f2834007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.300+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834007300 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.300+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28340074a0 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.300+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834005a40 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.300+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2840139e80 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.300+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284013a440 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.301+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284013a660 con 0x7f284006c7b0 2023-10-06T05:23:50.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.302+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f283401b400 con 0x7f284006c7b0 2023-10-06T05:23:50.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.303+0000 7f283cff9700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2824066f30 0x7f28240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.303+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f2834084170 con 0x7f284006c7b0 2023-10-06T05:23:50.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.304+0000 7f283ffff700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2824066f30 0x7f28240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.305+0000 7f283ffff700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2824066f30 0x7f28240693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f28400b03a0 tx=0x7f282c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.308+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28340512c0 con 0x7f284006c7b0 2023-10-06T05:23:50.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:50.562+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f284004c380 con 0x7f284006c7b0 2023-10-06T05:23:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.070+0000 7f283cff9700 1 -- 172.21.15.33:0/1297545428 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7f28340548e0 con 0x7f284006c7b0 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.076+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2824066f30 msgr2=0x7f28240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.076+0000 7f2845f9d700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2824066f30 0x7f28240693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f28400b03a0 tx=0x7f282c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.077+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 msgr2=0x7f284013c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.077+0000 7f2845f9d700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 0x7f284013c550 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f283400c810 tx=0x7f2834007c00 comp rx=0 tx=0).stop 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.081+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 shutdown_connections 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.081+0000 7f2845f9d700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2824066f30 0x7f28240693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.081+0000 7f2845f9d700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28400aff90 0x7f2840151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.081+0000 7f2845f9d700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284006c7b0 0x7f284013c550 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.081+0000 7f2845f9d700 1 --2- 172.21.15.33:0/1297545428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f284006cb80 0x7f2840150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.081+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 >> 172.21.15.33:0/1297545428 conn(0x7f284006d670 msgr2=0x7f28400ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.082+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 shutdown_connections 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.082+0000 7f2845f9d700 1 -- 172.21.15.33:0/1297545428 wait complete. 2023-10-06T05:23:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T05:23:51.109 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.294+0000 7faa2b59e700 1 Processor -- start 2023-10-06T05:23:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.295+0000 7faa2b59e700 1 -- start start 2023-10-06T05:23:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa2b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faa2c155140 0x7faa2c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa2b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0c8e40 con 0x7faa2c0adf50 2023-10-06T05:23:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa2b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0c99e0 con 0x7faa2c0699c0 2023-10-06T05:23:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa2b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c0ae3f0 con 0x7faa2c155140 2023-10-06T05:23:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa29d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faa2c0adf50 0x7faa2c0c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52342/0 (socket says 172.21.15.33:52342) 2023-10-06T05:23:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa29d9b700 1 -- 172.21.15.33:0/4094681239 learned_addr learned my addr 172.21.15.33:0/4094681239 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.296+0000 7faa2a59c700 1 --2- 172.21.15.33:0/4094681239 >> v2:172.21.15.167:3300/0 conn(0x7faa2c155140 0x7faa2c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448176442 0 0) 0x7faa2c0c8e40 con 0x7faa2c0adf50 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa14003570 con 0x7faa2c0adf50 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196101605 0 0) 0x7faa2c0c99e0 con 0x7faa2c0699c0 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa2c0c8e40 con 0x7faa2c0699c0 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2a59c700 1 -- 172.21.15.33:0/4094681239 >> v1:172.21.15.167:6789/0 conn(0x7faa2c0699c0 legacy=0x7faa2c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487577803 0 0) 0x7faa14003570 con 0x7faa2c0adf50 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2a59c700 1 -- 172.21.15.33:0/4094681239 >> v1:172.21.15.33:6789/0 conn(0x7faa2c0adf50 legacy=0x7faa2c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2a59c700 1 -- 172.21.15.33:0/4094681239 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0ae6c0 con 0x7faa2c155140 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.297+0000 7faa2a59c700 1 --2- 172.21.15.33:0/4094681239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c158700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009910 tx=0x7faa1c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2aae32477d5050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20001470 con 0x7faa2c155140 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c011420 con 0x7faa2c155140 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2959a700 1 -- 172.21.15.33:0/4094681239 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0117c0 con 0x7faa2c155140 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 -- 172.21.15.33:0/4094681239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 msgr2=0x7faa2c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 --2- 172.21.15.33:0/4094681239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c158700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009910 tx=0x7faa1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 -- 172.21.15.33:0/4094681239 shutdown_connections 2023-10-06T05:23:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 --2- 172.21.15.33:0/4094681239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c158700 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 -- 172.21.15.33:0/4094681239 >> 172.21.15.33:0/4094681239 conn(0x7faa2c06d030 msgr2=0x7faa2c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 -- 172.21.15.33:0/4094681239 shutdown_connections 2023-10-06T05:23:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.298+0000 7faa2b59e700 1 -- 172.21.15.33:0/4094681239 wait complete. 2023-10-06T05:23:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.300+0000 7faa2b59e700 1 Processor -- start 2023-10-06T05:23:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.300+0000 7faa2b59e700 1 -- start start 2023-10-06T05:23:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2c0699c0 0x7faa2c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 0x7faa2c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c16d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0b0db0 con 0x7faa2c0adf50 2023-10-06T05:23:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c150730 con 0x7faa2c155140 2023-10-06T05:23:51.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c16dc80 con 0x7faa2c0699c0 2023-10-06T05:23:51.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa29d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 0x7faa2c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa29d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 0x7faa2c1501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56190/0 (socket says 172.21.15.33:56190) 2023-10-06T05:23:51.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa29d9b700 1 -- 172.21.15.33:0/1221691234 learned_addr learned my addr 172.21.15.33:0/1221691234 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:51.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2a59c700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2c0699c0 0x7faa2c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.301+0000 7faa2ad9d700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c16d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa29d9b700 1 -- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2c0699c0 msgr2=0x7faa2c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa29d9b700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2c0699c0 0x7faa2c15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa29d9b700 1 -- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 msgr2=0x7faa2c16d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa29d9b700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c16d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa29d9b700 1 -- 172.21.15.33:0/1221691234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009420 con 0x7faa2c0adf50 2023-10-06T05:23:51.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa29d9b700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 0x7faa2c1501f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c960 tx=0x7faa20001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20007360 con 0x7faa2c0adf50 2023-10-06T05:23:51.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa200074e0 con 0x7faa2c0adf50 2023-10-06T05:23:51.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20004830 con 0x7faa2c0adf50 2023-10-06T05:23:51.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c16df60 con 0x7faa2c0adf50 2023-10-06T05:23:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.302+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0b0a40 con 0x7faa2c0adf50 2023-10-06T05:23:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.303+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c0b08a0 con 0x7faa2c0adf50 2023-10-06T05:23:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.304+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa20004c00 con 0x7faa2c0adf50 2023-10-06T05:23:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.305+0000 7faa1a7fc700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa0c066f30 0x7faa0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.306+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa20083a30 con 0x7faa2c0adf50 2023-10-06T05:23:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.306+0000 7faa2a59c700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa0c066f30 0x7faa0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.307+0000 7faa2a59c700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa0c066f30 0x7faa0c0693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0093f0 tx=0x7faa1c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.309+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa2c0b08a0 con 0x7faa2c0adf50 2023-10-06T05:23:51.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.510+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa2c0b08a0 con 0x7faa2c0adf50 2023-10-06T05:23:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.511+0000 7faa1a7fc700 1 -- 172.21.15.33:0/1221691234 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16893 (secure 0 0 0) 0x7faa2c0b08a0 con 0x7faa2c0adf50 2023-10-06T05:23:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:51.514 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":128,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:50.563479+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:48.724864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:49.540442+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:51.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.513+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa0c066f30 msgr2=0x7faa0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.513+0000 7faa2b59e700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa0c066f30 0x7faa0c0693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0093f0 tx=0x7faa1c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:51.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.513+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 msgr2=0x7faa2c1501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.513+0000 7faa2b59e700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 0x7faa2c1501f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c960 tx=0x7faa20001b80 comp rx=0 tx=0).stop 2023-10-06T05:23:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 shutdown_connections 2023-10-06T05:23:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa0c066f30 0x7faa0c0693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa2c155140 0x7faa2c16d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa2c0699c0 0x7faa2c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 --2- 172.21.15.33:0/1221691234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa2c0adf50 0x7faa2c1501f0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 >> 172.21.15.33:0/1221691234 conn(0x7faa2c06d030 msgr2=0x7faa2c150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 shutdown_connections 2023-10-06T05:23:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.514+0000 7faa2b59e700 1 -- 172.21.15.33:0/1221691234 wait complete. 2023-10-06T05:23:51.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:51.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.718+0000 7f4ce359e700 1 Processor -- start 2023-10-06T05:23:51.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.718+0000 7f4ce359e700 1 -- start start 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.719+0000 7f4ce359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ce413faa0 0x7f4ce413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.719+0000 7f4ce359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.719+0000 7f4ce359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ce406aad0 0x7f4ce406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.719+0000 7f4ce359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce40c9cb0 con 0x7f4ce413faa0 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.719+0000 7f4ce359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce40ca270 con 0x7f4ce406aad0 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.719+0000 7f4ce359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce406b840 con 0x7f4ce40c7940 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ce413faa0 0x7f4ce413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ce413faa0 0x7f4ce413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56202/0 (socket says 172.21.15.33:56202) 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 -- 172.21.15.33:0/2024217258 learned_addr learned my addr 172.21.15.33:0/2024217258 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:51.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce1d9b700 1 --2- 172.21.15.33:0/2024217258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce2d9d700 1 --2- 172.21.15.33:0/2024217258 >> v2:172.21.15.167:3300/0 conn(0x7f4ce406aad0 0x7f4ce406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 -- 172.21.15.33:0/2024217258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 msgr2=0x7f4ce40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 --2- 172.21.15.33:0/2024217258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce40c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 -- 172.21.15.33:0/2024217258 >> v2:172.21.15.167:3300/0 conn(0x7f4ce406aad0 msgr2=0x7f4ce406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 --2- 172.21.15.33:0/2024217258 >> v2:172.21.15.167:3300/0 conn(0x7f4ce406aad0 0x7f4ce406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 -- 172.21.15.33:0/2024217258 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce406bb20 con 0x7f4ce413faa0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.720+0000 7f4ce259c700 1 --2- 172.21.15.33:0/2024217258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce413fe70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40cc0b0 tx=0x7f4cd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc7eab37ce51b3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce159a700 1 -- 172.21.15.33:0/2024217258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400b860 con 0x7f4ce413faa0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce159a700 1 -- 172.21.15.33:0/2024217258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd400b9c0 con 0x7f4ce413faa0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce159a700 1 -- 172.21.15.33:0/2024217258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400bd60 con 0x7f4ce413faa0 2023-10-06T05:23:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 -- 172.21.15.33:0/2024217258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 msgr2=0x7f4ce413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 --2- 172.21.15.33:0/2024217258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce413fe70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40cc0b0 tx=0x7f4cd4009250 comp rx=0 tx=0).stop 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 -- 172.21.15.33:0/2024217258 shutdown_connections 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 --2- 172.21.15.33:0/2024217258 >> v2:172.21.15.167:3300/0 conn(0x7f4ce406aad0 0x7f4ce406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 --2- 172.21.15.33:0/2024217258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce413fe70 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 --2- 172.21.15.33:0/2024217258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 -- 172.21.15.33:0/2024217258 >> 172.21.15.33:0/2024217258 conn(0x7f4ce406d030 msgr2=0x7f4ce406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 -- 172.21.15.33:0/2024217258 shutdown_connections 2023-10-06T05:23:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.721+0000 7f4ce359e700 1 -- 172.21.15.33:0/2024217258 wait complete. 2023-10-06T05:23:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.723+0000 7f4ce359e700 1 Processor -- start 2023-10-06T05:23:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.723+0000 7f4ce359e700 1 -- start start 2023-10-06T05:23:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce406aad0 0x7f4ce4158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce412d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce412dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce412e1c0 con 0x7f4ce413faa0 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40b0670 con 0x7f4ce406aad0 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40afa60 con 0x7f4ce40c7940 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce412dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce412dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56216/0 (socket says 172.21.15.33:56216) 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 -- 172.21.15.33:0/4059183842 learned_addr learned my addr 172.21.15.33:0/4059183842 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce259c700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce406aad0 0x7f4ce4158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce1d9b700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce412d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 -- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 msgr2=0x7f4ce412d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce412d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 -- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce406aad0 msgr2=0x7f4ce4158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce406aad0 0x7f4ce4158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce2d9d700 1 -- 172.21.15.33:0/4059183842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40094f0 con 0x7f4ce413faa0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.724+0000 7f4ce259c700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce406aad0 0x7f4ce4158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.725+0000 7f4ce2d9d700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce412dc80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fa0 tx=0x7f4cdc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.725+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc00ec70 con 0x7f4ce413faa0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.725+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc00edd0 con 0x7f4ce413faa0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.725+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc014670 con 0x7f4ce413faa0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.725+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce40afd40 con 0x7f4ce413faa0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.725+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce40b0300 con 0x7f4ce413faa0 2023-10-06T05:23:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.726+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce40b0820 con 0x7f4ce413faa0 2023-10-06T05:23:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.727+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cdc0149e0 con 0x7f4ce413faa0 2023-10-06T05:23:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.728+0000 7f4cd27fc700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4cc4066f30 0x7f4cc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.728+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4cdc084d50 con 0x7f4ce413faa0 2023-10-06T05:23:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.729+0000 7f4ce259c700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4cc4066f30 0x7f4cc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.729+0000 7f4ce259c700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4cc4066f30 0x7f4cc40693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40cc0b0 tx=0x7f4cd4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.732+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cdc088030 con 0x7f4ce413faa0 2023-10-06T05:23:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.936+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce404c380 con 0x7f4ce413faa0 2023-10-06T05:23:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.937+0000 7f4cd27fc700 1 -- 172.21.15.33:0/4059183842 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f4cdc051e60 con 0x7f4ce413faa0 2023-10-06T05:23:51.939 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:51.939 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":128,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:50.563479+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:48.724864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:49.540442+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4cc4066f30 msgr2=0x7f4cc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4cc4066f30 0x7f4cc40693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40cc0b0 tx=0x7f4cd4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 msgr2=0x7f4ce412dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce412dc80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fa0 tx=0x7f4cdc009900 comp rx=0 tx=0).stop 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 shutdown_connections 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4cc4066f30 0x7f4cc40693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce413faa0 0x7f4ce412dc80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce406aad0 0x7f4ce4158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 --2- 172.21.15.33:0/4059183842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ce40c7940 0x7f4ce412d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.939+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 >> 172.21.15.33:0/4059183842 conn(0x7f4ce406d030 msgr2=0x7f4ce40c1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.940+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 shutdown_connections 2023-10-06T05:23:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:51.940+0000 7f4ce359e700 1 -- 172.21.15.33:0/4059183842 wait complete. 2023-10-06T05:23:51.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:23:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.154+0000 7f4c8f59d700 1 Processor -- start 2023-10-06T05:23:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.154+0000 7f4c8f59d700 1 -- start start 2023-10-06T05:23:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.155+0000 7f4c8f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4c880af910 0x7f4c8815a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.155+0000 7f4c8f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c880ce890 con 0x7f4c880af540 2023-10-06T05:23:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.155+0000 7f4c8f59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8815d890 con 0x7f4c880cd5e0 2023-10-06T05:23:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.155+0000 7f4c8f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c880afd50 con 0x7f4c880af910 2023-10-06T05:23:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4c880af540 0x7f4c880cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52346/0 (socket says 172.21.15.33:52346) 2023-10-06T05:23:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8e59b700 1 -- 172.21.15.33:0/113793346 learned_addr learned my addr 172.21.15.33:0/113793346 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8ed9c700 1 --2- 172.21.15.33:0/113793346 >> v2:172.21.15.167:3300/0 conn(0x7f4c880af910 0x7f4c8815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600820361 0 0) 0x7f4c880ce890 con 0x7f4c880af540 2023-10-06T05:23:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c74003570 con 0x7f4c880af540 2023-10-06T05:23:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029032805 0 0) 0x7f4c74003570 con 0x7f4c880af540 2023-10-06T05:23:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 >> v2:172.21.15.167:3300/0 conn(0x7f4c880af910 msgr2=0x7f4c8815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 --2- 172.21.15.33:0/113793346 >> v2:172.21.15.167:3300/0 conn(0x7f4c880af910 0x7f4c8815a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 >> v1:172.21.15.167:6789/0 conn(0x7f4c880cd5e0 legacy=0x7f4c880ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8815db70 con 0x7f4c880af540 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f4c84001470 con 0x7f4c880af540 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 >> v1:172.21.15.33:6789/0 conn(0x7f4c880af540 legacy=0x7f4c880cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c74004810 0x7f4c74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.156+0000 7f4c8d599700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740051c0 0x7f4c7400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880afd50 con 0x7f4c74003a60 2023-10-06T05:23:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7400c0d0 con 0x7f4c740051c0 2023-10-06T05:23:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7400c290 con 0x7f4c74004810 2023-10-06T05:23:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8ed9c700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740051c0 0x7f4c7400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8dd9a700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c74004810 0x7f4c74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 -- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c74004810 msgr2=0x7f4c74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c74004810 0x7f4c74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 -- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740051c0 msgr2=0x7f4c7400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740051c0 0x7f4c7400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 -- 172.21.15.33:0/113793346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74003570 con 0x7f4c74003a60 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8ed9c700 1 -- 172.21.15.33:0/113793346 reap_dead start 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.157+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c74003e70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c390 tx=0x7f4c8815da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd8c8471b26e9cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400cbe0 con 0x7f4c74003a60 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8400cd40 con 0x7f4c74003a60 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8d599700 1 -- 172.21.15.33:0/113793346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84005b70 con 0x7f4c74003a60 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 -- 172.21.15.33:0/113793346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 msgr2=0x7f4c74003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c74003e70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c390 tx=0x7f4c8815da60 comp rx=0 tx=0).stop 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 -- 172.21.15.33:0/113793346 shutdown_connections 2023-10-06T05:23:52.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/113793346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c74003e70 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 -- 172.21.15.33:0/113793346 >> 172.21.15.33:0/113793346 conn(0x7f4c8806d9b0 msgr2=0x7f4c8806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:52.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 -- 172.21.15.33:0/113793346 shutdown_connections 2023-10-06T05:23:52.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.158+0000 7f4c8f59d700 1 -- 172.21.15.33:0/113793346 wait complete. 2023-10-06T05:23:52.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.160+0000 7f4c8f59d700 1 Processor -- start 2023-10-06T05:23:52.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.160+0000 7f4c8f59d700 1 -- start start 2023-10-06T05:23:52.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c88144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b5af0 0x7f4c880b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880c5c10 0x7f4c880c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880c65b0 con 0x7f4c74003a60 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880c8790 con 0x7f4c880c5c10 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880c7b80 con 0x7f4c880b5af0 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c88144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c88144b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56222/0 (socket says 172.21.15.33:56222) 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 -- 172.21.15.33:0/551118918 learned_addr learned my addr 172.21.15.33:0/551118918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8ed9c700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880c5c10 0x7f4c880c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8dd9a700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b5af0 0x7f4c880b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 -- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b5af0 msgr2=0x7f4c880b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b5af0 0x7f4c880b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 -- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880c5c10 msgr2=0x7f4c880c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880c5c10 0x7f4c880c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.161+0000 7f4c8e59b700 1 -- 172.21.15.33:0/551118918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880c7e60 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.162+0000 7f4c8e59b700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c88144b80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400c0a0 tx=0x7f4c84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.162+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84022070 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.162+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8400ceb0 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.162+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400cbe0 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.162+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880c8140 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.162+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c880b3430 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.163+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c880c8a70 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.164+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c84007550 con 0x7f4c74003a60 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.165+0000 7f4c7e7fc700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c6c066f30 0x7f4c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.166+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c84085330 con 0x7f4c74003a60 2023-10-06T05:23:52.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.166+0000 7f4c8dd9a700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c6c066f30 0x7f4c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.167+0000 7f4c8dd9a700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c6c066f30 0x7f4c6c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4c8816d110 tx=0x7f4c78009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.170+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c840523a0 con 0x7f4c74003a60 2023-10-06T05:23:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:52.420+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c8804c380 con 0x7f4c74003a60 2023-10-06T05:23:53.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.077+0000 7f4c7e7fc700 1 -- 172.21.15.33:0/551118918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c840559c0 con 0x7f4c74003a60 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c6c066f30 msgr2=0x7f4c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c6c066f30 0x7f4c6c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4c8816d110 tx=0x7f4c78009040 comp rx=0 tx=0).stop 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 msgr2=0x7f4c88144b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c88144b80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400c0a0 tx=0x7f4c84015000 comp rx=0 tx=0).stop 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 shutdown_connections 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c6c066f30 0x7f4c6c0693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c880c5c10 0x7f4c880c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c74003a60 0x7f4c88144b80 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 --2- 172.21.15.33:0/551118918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c880b5af0 0x7f4c880b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 >> 172.21.15.33:0/551118918 conn(0x7f4c8806d9b0 msgr2=0x7f4c88156c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:53.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 shutdown_connections 2023-10-06T05:23:53.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.080+0000 7f4c8f59d700 1 -- 172.21.15.33:0/551118918 wait complete. 2023-10-06T05:23:53.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:23:53.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.295+0000 7f91dd99d700 1 Processor -- start 2023-10-06T05:23:53.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.295+0000 7f91dd99d700 1 -- start start 2023-10-06T05:23:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.296+0000 7f91dd99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91d8154f20 0x7f91d80ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.296+0000 7f91dd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d80b18f0 0x7f91d80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.296+0000 7f91dd99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d81586a0 con 0x7f91d81552f0 2023-10-06T05:23:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.296+0000 7f91dd99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d806bdf0 con 0x7f91d8154f20 2023-10-06T05:23:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.296+0000 7f91dd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8157ab0 con 0x7f91d80b18f0 2023-10-06T05:23:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.296+0000 7f91dc99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91d8154f20 0x7f91d80ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91d8154f20 0x7f91d80ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56234/0 (socket says 172.21.15.33:56234) 2023-10-06T05:23:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 -- 172.21.15.33:0/751959258 learned_addr learned my addr 172.21.15.33:0/751959258 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 -- 172.21.15.33:0/751959258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d80b18f0 msgr2=0x7f91d80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dd19c700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d80b18f0 0x7f91d80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d80b18f0 0x7f91d80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 -- 172.21.15.33:0/751959258 >> v1:172.21.15.167:6789/0 conn(0x7f91d81552f0 legacy=0x7f91d80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 -- 172.21.15.33:0/751959258 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8157d70 con 0x7f91d8154f20 2023-10-06T05:23:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dd19c700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d80b18f0 0x7f91d80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.297+0000 7f91dc99b700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d8154f20 0x7f91d80ae720 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0099b0 tx=0x7f91cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89915ab1256f22d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91d77fe700 1 -- 172.21.15.33:0/751959258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00b820 con 0x7f91d8154f20 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91d77fe700 1 -- 172.21.15.33:0/751959258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc00b980 con 0x7f91d8154f20 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91d77fe700 1 -- 172.21.15.33:0/751959258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00bd20 con 0x7f91d8154f20 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 -- 172.21.15.33:0/751959258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d8154f20 msgr2=0x7f91d80ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d8154f20 0x7f91d80ae720 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0099b0 tx=0x7f91cc009250 comp rx=0 tx=0).stop 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 -- 172.21.15.33:0/751959258 shutdown_connections 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d8154f20 0x7f91d80ae720 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 --2- 172.21.15.33:0/751959258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d80b18f0 0x7f91d80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 -- 172.21.15.33:0/751959258 >> 172.21.15.33:0/751959258 conn(0x7f91d806d030 msgr2=0x7f91d80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 -- 172.21.15.33:0/751959258 shutdown_connections 2023-10-06T05:23:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.298+0000 7f91dd99d700 1 -- 172.21.15.33:0/751959258 wait complete. 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.300+0000 7f91dd99d700 1 Processor -- start 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.300+0000 7f91dd99d700 1 -- start start 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d80b18f0 0x7f91d8169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d8154f20 0x7f91d816b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 0x7f91d80b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80b0c00 con 0x7f91d80b0220 2023-10-06T05:23:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8169fb0 con 0x7f91d80b18f0 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d816b060 con 0x7f91d8154f20 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 0x7f91d80b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 0x7f91d80b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56240/0 (socket says 172.21.15.33:56240) 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 -- 172.21.15.33:0/996673973 learned_addr learned my addr 172.21.15.33:0/996673973 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dc99b700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d80b18f0 0x7f91d8169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91d7fff700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d8154f20 0x7f91d816b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 -- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d8154f20 msgr2=0x7f91d816b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d8154f20 0x7f91d816b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 -- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d80b18f0 msgr2=0x7f91d8169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d80b18f0 0x7f91d8169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.301+0000 7f91dd19c700 1 -- 172.21.15.33:0/996673973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0094f0 con 0x7f91d80b0220 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91dc99b700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d80b18f0 0x7f91d8169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91d7fff700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d8154f20 0x7f91d816b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91dd19c700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 0x7f91d80b06c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f91d000cef0 tx=0x7f91d0007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000ae00 con 0x7f91d80b0220 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d0016430 con 0x7f91d80b0220 2023-10-06T05:23:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00167f0 con 0x7f91d80b0220 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80c6b40 con 0x7f91d80b0220 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.302+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80c7100 con 0x7f91d80b0220 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.303+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d804c380 con 0x7f91d80b0220 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.304+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91d0003020 con 0x7f91d80b0220 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.305+0000 7f91d4ff9700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f91bc066f30 0x7f91bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.305+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91d0012070 con 0x7f91d80b0220 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.305+0000 7f91dc99b700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f91bc066f30 0x7f91bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.306+0000 7f91dc99b700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f91bc066f30 0x7f91bc0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0099b0 tx=0x7f91cc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.309+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91d0052640 con 0x7f91d80b0220 2023-10-06T05:23:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:53.556+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f91d806f370 con 0x7f91d80b0220 2023-10-06T05:23:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.084+0000 7f91d4ff9700 1 -- 172.21.15.33:0/996673973 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91d0055c60 con 0x7f91d80b0220 2023-10-06T05:23:54.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f91bc066f30 msgr2=0x7f91bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f91bc066f30 0x7f91bc0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0099b0 tx=0x7f91cc016000 comp rx=0 tx=0).stop 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 msgr2=0x7f91d80b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 0x7f91d80b06c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f91d000cef0 tx=0x7f91d0007820 comp rx=0 tx=0).stop 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 shutdown_connections 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f91bc066f30 0x7f91bc0693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d80b0220 0x7f91d80b06c0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91d80b18f0 0x7f91d8169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 --2- 172.21.15.33:0/996673973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91d8154f20 0x7f91d816b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 >> 172.21.15.33:0/996673973 conn(0x7f91d806d030 msgr2=0x7f91d8069d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 shutdown_connections 2023-10-06T05:23:54.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.087+0000 7f91dd99d700 1 -- 172.21.15.33:0/996673973 wait complete. 2023-10-06T05:23:54.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:54.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.296+0000 7f105759e700 1 Processor -- start 2023-10-06T05:23:54.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.296+0000 7f105759e700 1 -- start start 2023-10-06T05:23:54.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10580ccfc0 0x7f10580cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580cf3b0 0x7f10581594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f10580cc750 0x7f10581599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10580cbd80 con 0x7f10580ccfc0 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10580af5f0 con 0x7f10580cc750 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580af8d0 con 0x7f10580cf3b0 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.297+0000 7f105659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10580ccfc0 0x7f10580cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10580ccfc0 0x7f10580cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56258/0 (socket says 172.21.15.33:56258) 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 -- 172.21.15.33:0/621692157 learned_addr learned my addr 172.21.15.33:0/621692157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f1056d9d700 1 --2- 172.21.15.33:0/621692157 >> v2:172.21.15.167:3300/0 conn(0x7f10580cc750 0x7f10581599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f1055d9b700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580cf3b0 0x7f10581594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 -- 172.21.15.33:0/621692157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580cf3b0 msgr2=0x7f10581594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580cf3b0 0x7f10581594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 -- 172.21.15.33:0/621692157 >> v2:172.21.15.167:3300/0 conn(0x7f10580cc750 msgr2=0x7f10581599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 --2- 172.21.15.33:0/621692157 >> v2:172.21.15.167:3300/0 conn(0x7f10580cc750 0x7f10581599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 -- 172.21.15.33:0/621692157 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580afbb0 con 0x7f10580ccfc0 2023-10-06T05:23:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f1055d9b700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580cf3b0 0x7f10581594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f1056d9d700 1 --2- 172.21.15.33:0/621692157 >> v2:172.21.15.167:3300/0 conn(0x7f10580cc750 0x7f10581599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.298+0000 7f105659c700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f10580cc210 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f10580cdb60 tx=0x7f1048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a36dc61a6a33d77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105559a700 1 -- 172.21.15.33:0/621692157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800b860 con 0x7f10580ccfc0 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105559a700 1 -- 172.21.15.33:0/621692157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104800b9c0 con 0x7f10580ccfc0 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105559a700 1 -- 172.21.15.33:0/621692157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800bd60 con 0x7f10580ccfc0 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 -- 172.21.15.33:0/621692157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 msgr2=0x7f10580cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f10580cc210 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f10580cdb60 tx=0x7f1048009250 comp rx=0 tx=0).stop 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 -- 172.21.15.33:0/621692157 shutdown_connections 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 --2- 172.21.15.33:0/621692157 >> v2:172.21.15.167:3300/0 conn(0x7f10580cc750 0x7f10581599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f10580cc210 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 --2- 172.21.15.33:0/621692157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580cf3b0 0x7f10581594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 -- 172.21.15.33:0/621692157 >> 172.21.15.33:0/621692157 conn(0x7f105806cff0 msgr2=0x7f105806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 -- 172.21.15.33:0/621692157 shutdown_connections 2023-10-06T05:23:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.299+0000 7f105759e700 1 -- 172.21.15.33:0/621692157 wait complete. 2023-10-06T05:23:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.301+0000 7f105759e700 1 Processor -- start 2023-10-06T05:23:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.301+0000 7f105759e700 1 -- start start 2023-10-06T05:23:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10580cc750 0x7f105816a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f105815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580c75b0 0x7f10580c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105816aa40 con 0x7f10580ccfc0 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10580c7b70 con 0x7f10580cc750 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580c7e50 con 0x7f10580c75b0 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f105815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f105815ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56262/0 (socket says 172.21.15.33:56262) 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 -- 172.21.15.33:0/37872324 learned_addr learned my addr 172.21.15.33:0/37872324 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f105659c700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10580cc750 0x7f105816a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1056d9d700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580c75b0 0x7f10580c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 -- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580c75b0 msgr2=0x7f10580c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580c75b0 0x7f10580c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 -- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10580cc750 msgr2=0x7f105816a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10580cc750 0x7f105816a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.302+0000 7f1055d9b700 1 -- 172.21.15.33:0/37872324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480094f0 con 0x7f10580ccfc0 2023-10-06T05:23:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.303+0000 7f1055d9b700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f105815ea90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f104c00d860 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.303+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0125f0 con 0x7f10580ccfc0 2023-10-06T05:23:54.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.303+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c012750 con 0x7f10580ccfc0 2023-10-06T05:23:54.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.303+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c012b10 con 0x7f10580ccfc0 2023-10-06T05:23:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.303+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10580c8130 con 0x7f10580ccfc0 2023-10-06T05:23:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.303+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105815efd0 con 0x7f10580ccfc0 2023-10-06T05:23:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.304+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105815f1f0 con 0x7f10580ccfc0 2023-10-06T05:23:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.305+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f104c013480 con 0x7f10580ccfc0 2023-10-06T05:23:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.306+0000 7f10467fc700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.306+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f104c084950 con 0x7f10580ccfc0 2023-10-06T05:23:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.306+0000 7f105659c700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.307+0000 7f105659c700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1038066f30 0x7f10380693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f10580cdb60 tx=0x7f1048017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.310+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f105815f1f0 con 0x7f10580ccfc0 2023-10-06T05:23:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.559+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10580c8650 con 0x7f10580ccfc0 2023-10-06T05:23:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.560+0000 7f10467fc700 1 -- 172.21.15.33:0/37872324 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f10580c8650 con 0x7f10580ccfc0 2023-10-06T05:23:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:54.562 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1831041283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1831041283},{"type":"v1","addr":"172.21.15.33:6834","nonce":1831041283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/606556444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":606556444},{"type":"v1","addr":"172.21.15.167:6836","nonce":606556444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3370703117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3370703117},{"type":"v1","addr":"172.21.15.167:6837","nonce":3370703117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:23:54.077277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/117155768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":117155768},{"type":"v1","addr":"172.21.15.33:6835","nonce":117155768}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1038066f30 msgr2=0x7f10380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1038066f30 0x7f10380693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f10580cdb60 tx=0x7f1048017000 comp rx=0 tx=0).stop 2023-10-06T05:23:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 msgr2=0x7f105815ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f105815ea90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f104c00d860 tx=0x7f104c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 shutdown_connections 2023-10-06T05:23:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10580c75b0 0x7f10580c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10580cc750 0x7f105816a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 --2- 172.21.15.33:0/37872324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10580ccfc0 0x7f105815ea90 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 >> 172.21.15.33:0/37872324 conn(0x7f105806cff0 msgr2=0x7f10580b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 shutdown_connections 2023-10-06T05:23:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.563+0000 7f105759e700 1 -- 172.21.15.33:0/37872324 wait complete. 2023-10-06T05:23:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 90 2023-10-06T05:23:54.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.767+0000 7efc2519d700 1 Processor -- start 2023-10-06T05:23:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.768+0000 7efc2519d700 1 -- start start 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2519d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc20155af0 0x7efc2006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc200c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2519d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2006ab60 con 0x7efc200c8b50 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2519d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2006b7e0 con 0x7efc20155af0 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2006ae40 con 0x7efc2006a6a0 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc1ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc20155af0 0x7efc2006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2499c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efc200c8b50 0x7efc201598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52354/0 (socket says 172.21.15.33:52354) 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc2499c700 1 -- 172.21.15.33:0/128586694 learned_addr learned my addr 172.21.15.33:0/128586694 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.769+0000 7efc1f7fe700 1 --2- 172.21.15.33:0/128586694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc200c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1ffff700 1 -- 172.21.15.33:0/128586694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 msgr2=0x7efc200c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1ffff700 1 --2- 172.21.15.33:0/128586694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc200c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1ffff700 1 -- 172.21.15.33:0/128586694 >> v1:172.21.15.33:6789/0 conn(0x7efc200c8b50 legacy=0x7efc201598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1ffff700 1 -- 172.21.15.33:0/128586694 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2006b120 con 0x7efc20155af0 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1effd700 1 -- 172.21.15.33:0/128586694 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850045830 0 0) 0x7efc2006ab60 con 0x7efc200c8b50 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1ffff700 1 --2- 172.21.15.33:0/128586694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc2006a160 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7efc100098e0 tx=0x7efc100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144e9d8619557137 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1effd700 1 -- 172.21.15.33:0/128586694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000b640 con 0x7efc20155af0 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1effd700 1 -- 172.21.15.33:0/128586694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1000b7a0 con 0x7efc20155af0 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.770+0000 7efc1effd700 1 -- 172.21.15.33:0/128586694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000bb40 con 0x7efc20155af0 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 -- 172.21.15.33:0/128586694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 msgr2=0x7efc2006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 --2- 172.21.15.33:0/128586694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc2006a160 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7efc100098e0 tx=0x7efc100091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 -- 172.21.15.33:0/128586694 shutdown_connections 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 --2- 172.21.15.33:0/128586694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc2006a160 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 --2- 172.21.15.33:0/128586694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc200c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 -- 172.21.15.33:0/128586694 >> 172.21.15.33:0/128586694 conn(0x7efc2006d030 msgr2=0x7efc2006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 -- 172.21.15.33:0/128586694 shutdown_connections 2023-10-06T05:23:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.771+0000 7efc2519d700 1 -- 172.21.15.33:0/128586694 wait complete. 2023-10-06T05:23:54.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.773+0000 7efc2519d700 1 Processor -- start 2023-10-06T05:23:54.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.773+0000 7efc2519d700 1 -- start start 2023-10-06T05:23:54.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc2014ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2519d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc200c8b50 0x7efc200c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2519d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc200c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2519d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2014b780 con 0x7efc20155af0 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2519d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2014b910 con 0x7efc200c8b50 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc200c3b00 con 0x7efc2006a6a0 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc200c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc200c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56284/0 (socket says 172.21.15.33:56284) 2023-10-06T05:23:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc2499c700 1 -- 172.21.15.33:0/1228011765 learned_addr learned my addr 172.21.15.33:0/1228011765 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc1ffff700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc2014ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.774+0000 7efc1f7fe700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc200c8b50 0x7efc200c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2499c700 1 -- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 msgr2=0x7efc2014ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2499c700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc2014ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2499c700 1 -- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc200c8b50 msgr2=0x7efc200c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2499c700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc200c8b50 0x7efc200c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2499c700 1 -- 172.21.15.33:0/1228011765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10009420 con 0x7efc20155af0 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc1f7fe700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc200c8b50 0x7efc200c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2499c700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc200c9cb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7efc1800ccc0 tx=0x7efc18001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800a930 con 0x7efc20155af0 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800aa90 con 0x7efc20155af0 2023-10-06T05:23:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc200c3de0 con 0x7efc20155af0 2023-10-06T05:23:54.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800ae30 con 0x7efc20155af0 2023-10-06T05:23:54.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.775+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2014b2d0 con 0x7efc20155af0 2023-10-06T05:23:54.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.776+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc200c9a50 con 0x7efc20155af0 2023-10-06T05:23:54.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.777+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc18007300 con 0x7efc20155af0 2023-10-06T05:23:54.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.778+0000 7efc07fff700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.778+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efc18084ac0 con 0x7efc20155af0 2023-10-06T05:23:54.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.779+0000 7efc1ffff700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.779+0000 7efc1ffff700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc0c066f30 0x7efc0c0693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc100093f0 tx=0x7efc10017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.782+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc18051bb0 con 0x7efc20155af0 2023-10-06T05:23:54.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.987+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc2014ba80 con 0x7efc20155af0 2023-10-06T05:23:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.988+0000 7efc07fff700 1 -- 172.21.15.33:0/1228011765 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16880 (secure 0 0 0) 0x7efc180551d0 con 0x7efc20155af0 2023-10-06T05:23:54.989 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:54.990 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":129,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:23:52.053251+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:48.724864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:49.540442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc0c066f30 msgr2=0x7efc0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc0c066f30 0x7efc0c0693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc100093f0 tx=0x7efc10017000 comp rx=0 tx=0).stop 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 msgr2=0x7efc200c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc200c9cb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7efc1800ccc0 tx=0x7efc18001a20 comp rx=0 tx=0).stop 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 shutdown_connections 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20155af0 0x7efc200c9cb0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc2006a6a0 0x7efc2014ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 --2- 172.21.15.33:0/1228011765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc200c8b50 0x7efc200c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 >> 172.21.15.33:0/1228011765 conn(0x7efc2006d030 msgr2=0x7efc2014f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 shutdown_connections 2023-10-06T05:23:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:54.990+0000 7efc2519d700 1 -- 172.21.15.33:0/1228011765 wait complete. 2023-10-06T05:23:55.009 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:23:55.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:23:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.196+0000 7fdfbcd9d700 1 Processor -- start 2023-10-06T05:23:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.196+0000 7fdfbcd9d700 1 -- start start 2023-10-06T05:23:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.197+0000 7fdfbcd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb80cf3b0 0x7fdfb81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.197+0000 7fdfbcd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfb80af0a0 0x7fdfb81599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.197+0000 7fdfbcd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb80cbd80 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.197+0000 7fdfbcd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb80af5f0 con 0x7fdfb80af0a0 2023-10-06T05:23:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.197+0000 7fdfbcd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80af8d0 con 0x7fdfb80cf3b0 2023-10-06T05:23:55.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfb80af0a0 0x7fdfb81599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfb80af0a0 0x7fdfb81599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56292/0 (socket says 172.21.15.33:56292) 2023-10-06T05:23:55.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 -- 172.21.15.33:0/1783598791 learned_addr learned my addr 172.21.15.33:0/1783598791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:55.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb6ffd700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb80cf3b0 0x7fdfb81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 -- 172.21.15.33:0/1783598791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb80cf3b0 msgr2=0x7fdfb81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb80cf3b0 0x7fdfb81594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 -- 172.21.15.33:0/1783598791 >> v1:172.21.15.167:6789/0 conn(0x7fdfb80ccfc0 legacy=0x7fdfb80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 -- 172.21.15.33:0/1783598791 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80afbb0 con 0x7fdfb80af0a0 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb6ffd700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb80cf3b0 0x7fdfb81594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.198+0000 7fdfb7fff700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80af0a0 0x7fdfb81599e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009f20 tx=0x7fdfb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12631d5c88ad26c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfb67fc700 1 -- 172.21.15.33:0/1783598791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000e070 con 0x7fdfb80af0a0 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfb67fc700 1 -- 172.21.15.33:0/1783598791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb000ba40 con 0x7fdfb80af0a0 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfb67fc700 1 -- 172.21.15.33:0/1783598791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00092e0 con 0x7fdfb80af0a0 2023-10-06T05:23:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1783598791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80af0a0 msgr2=0x7fdfb81599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80af0a0 0x7fdfb81599e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009f20 tx=0x7fdfb0009840 comp rx=0 tx=0).stop 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1783598791 shutdown_connections 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80af0a0 0x7fdfb81599e0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1783598791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb80cf3b0 0x7fdfb81594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1783598791 >> 172.21.15.33:0/1783598791 conn(0x7fdfb806cff0 msgr2=0x7fdfb806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1783598791 shutdown_connections 2023-10-06T05:23:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.199+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1783598791 wait complete. 2023-10-06T05:23:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.201+0000 7fdfbcd9d700 1 Processor -- start 2023-10-06T05:23:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.201+0000 7fdfbcd9d700 1 -- start start 2023-10-06T05:23:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfbcd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb80af0a0 0x7fdfb8169600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfbcd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 0x7fdfb8169b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfbcd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb816a080 0x7fdfb80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfbcd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb816a520 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfbcd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80c9c40 con 0x7fdfb80af0a0 2023-10-06T05:23:55.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfbcd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80c9030 con 0x7fdfb816a080 2023-10-06T05:23:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 0x7fdfb8169b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 0x7fdfb8169b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56304/0 (socket says 172.21.15.33:56304) 2023-10-06T05:23:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/1206353719 learned_addr learned my addr 172.21.15.33:0/1206353719 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb816a080 msgr2=0x7fdfb80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb816a080 0x7fdfb80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb80af0a0 msgr2=0x7fdfb8169600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.202+0000 7fdfb6ffd700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb80af0a0 0x7fdfb8169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/1206353719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0009ae0 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdfb6ffd700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 0x7fdfb8169b40 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c880 tx=0x7fdfac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac011600 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac011760 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac011b00 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb80c9310 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.203+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb80c98d0 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.204+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb804c380 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.205+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfac00a3e0 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.206+0000 7fdf9f7fe700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfa4066f30 0x7fdfa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.206+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdfac084400 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.206+0000 7fdfb77fe700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfa4066f30 0x7fdfa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.207+0000 7fdfb77fe700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfa4066f30 0x7fdfa40693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80b6310 tx=0x7fdfa8006c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.210+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfac051510 con 0x7fdfb80ccfc0 2023-10-06T05:23:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.406+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfb80c9db0 con 0x7fdfa4066f30 2023-10-06T05:23:55.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.416+0000 7fdf9f7fe700 1 -- 172.21.15.33:0/1206353719 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311265 (secure 0 0 0) 0x7fdfb80c9db0 con 0x7fdfa4066f30 2023-10-06T05:23:55.424 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:55.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfa4066f30 msgr2=0x7fdfa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfa4066f30 0x7fdfa40693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80b6310 tx=0x7fdfa8006c60 comp rx=0 tx=0).stop 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 msgr2=0x7fdfb8169b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 0x7fdfb8169b40 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c880 tx=0x7fdfac008040 comp rx=0 tx=0).stop 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 shutdown_connections 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfa4066f30 0x7fdfa40693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfb816a080 0x7fdfb80b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfb80af0a0 0x7fdfb8169600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 --2- 172.21.15.33:0/1206353719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ccfc0 0x7fdfb8169b40 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 >> 172.21.15.33:0/1206353719 conn(0x7fdfb806cff0 msgr2=0x7fdfb80b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 shutdown_connections 2023-10-06T05:23:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.420+0000 7fdfbcd9d700 1 -- 172.21.15.33:0/1206353719 wait complete. 2023-10-06T05:23:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:23:55.439 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-10-06T05:23:53.958983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424740,"kb_used_data":13668,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549980,"statfs":{"total":773094113280,"available":770611179520,"internally_reserved":0,"allocated":13996032,"data_stored":7045590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.908201"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.064867+0000","last_change":"2023-10-06T05:23:51.064867+0000","last_active":"2023-10-06T05:23:51.064867+0000","last_peered":"2023-10-06T05:23:51.064867+0000","last_clean":"2023-10-06T05:23:51.064867+0000","last_became_active":"2023-10-06T05:23:51.064599+0000","last_became_peered":"2023-10-06T05:23:51.064599+0000","last_unstale":"2023-10-06T05:23:51.064867+0000","last_undegraded":"2023-10-06T05:23:51.064867+0000","last_fullsized":"2023-10-06T05:23:51.064867+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:00.257938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.082904+0000","last_change":"2023-10-06T05:23:51.082904+0000","last_active":"2023-10-06T05:23:51.082904+0000","last_peered":"2023-10-06T05:23:51.082904+0000","last_clean":"2023-10-06T05:23:51.082904+0000","last_became_active":"2023-10-06T05:23:51.082566+0000","last_became_peered":"2023-10-06T05:23:51.082566+0000","last_unstale":"2023-10-06T05:23:51.082904+0000","last_undegraded":"2023-10-06T05:23:51.082904+0000","last_fullsized":"2023-10-06T05:23:51.082904+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:12.272018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075293+0000","last_change":"2023-10-06T05:23:51.075293+0000","last_active":"2023-10-06T05:23:51.075293+0000","last_peered":"2023-10-06T05:23:51.075293+0000","last_clean":"2023-10-06T05:23:51.075293+0000","last_became_active":"2023-10-06T05:23:51.074980+0000","last_became_peered":"2023-10-06T05:23:51.074980+0000","last_unstale":"2023-10-06T05:23:51.075293+0000","last_undegraded":"2023-10-06T05:23:51.075293+0000","last_fullsized":"2023-10-06T05:23:51.075293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:35.066197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.074485+0000","last_change":"2023-10-06T05:23:51.074485+0000","last_active":"2023-10-06T05:23:51.074485+0000","last_peered":"2023-10-06T05:23:51.074485+0000","last_clean":"2023-10-06T05:23:51.074485+0000","last_became_active":"2023-10-06T05:23:51.074169+0000","last_became_peered":"2023-10-06T05:23:51.074169+0000","last_unstale":"2023-10-06T05:23:51.074485+0000","last_undegraded":"2023-10-06T05:23:51.074485+0000","last_fullsized":"2023-10-06T05:23:51.074485+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:51.473460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080677+0000","last_change":"2023-10-06T05:23:51.080677+0000","last_active":"2023-10-06T05:23:51.080677+0000","last_peered":"2023-10-06T05:23:51.080677+0000","last_clean":"2023-10-06T05:23:51.080677+0000","last_became_active":"2023-10-06T05:23:51.080411+0000","last_became_peered":"2023-10-06T05:23:51.080411+0000","last_unstale":"2023-10-06T05:23:51.080677+0000","last_undegraded":"2023-10-06T05:23:51.080677+0000","last_fullsized":"2023-10-06T05:23:51.080677+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:52.175784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.088652+0000","last_change":"2023-10-06T05:23:51.088652+0000","last_active":"2023-10-06T05:23:51.088652+0000","last_peered":"2023-10-06T05:23:51.088652+0000","last_clean":"2023-10-06T05:23:51.088652+0000","last_became_active":"2023-10-06T05:23:51.087719+0000","last_became_peered":"2023-10-06T05:23:51.087719+0000","last_unstale":"2023-10-06T05:23:51.088652+0000","last_undegraded":"2023-10-06T05:23:51.088652+0000","last_fullsized":"2023-10-06T05:23:51.088652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:23.466596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.085954+0000","last_change":"2023-10-06T05:23:51.085954+0000","last_active":"2023-10-06T05:23:51.085954+0000","last_peered":"2023-10-06T05:23:51.085954+0000","last_clean":"2023-10-06T05:23:51.085954+0000","last_became_active":"2023-10-06T05:23:51.085630+0000","last_became_peered":"2023-10-06T05:23:51.085630+0000","last_unstale":"2023-10-06T05:23:51.085954+0000","last_undegraded":"2023-10-06T05:23:51.085954+0000","last_fullsized":"2023-10-06T05:23:51.085954+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:56.357935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075721+0000","last_change":"2023-10-06T05:23:51.075721+0000","last_active":"2023-10-06T05:23:51.075721+0000","last_peered":"2023-10-06T05:23:51.075721+0000","last_clean":"2023-10-06T05:23:51.075721+0000","last_became_active":"2023-10-06T05:23:51.075424+0000","last_became_peered":"2023-10-06T05:23:51.075424+0000","last_unstale":"2023-10-06T05:23:51.075721+0000","last_undegraded":"2023-10-06T05:23:51.075721+0000","last_fullsized":"2023-10-06T05:23:51.075721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:23.592163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077765+0000","last_change":"2023-10-06T05:23:51.077765+0000","last_active":"2023-10-06T05:23:51.077765+0000","last_peered":"2023-10-06T05:23:51.077765+0000","last_clean":"2023-10-06T05:23:51.077765+0000","last_became_active":"2023-10-06T05:23:51.077480+0000","last_became_peered":"2023-10-06T05:23:51.077480+0000","last_unstale":"2023-10-06T05:23:51.077765+0000","last_undegraded":"2023-10-06T05:23:51.077765+0000","last_fullsized":"2023-10-06T05:23:51.077765+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079325+0000","last_change":"2023-10-06T05:23:51.079325+0000","last_active":"2023-10-06T05:23:51.079325+0000","last_peered":"2023-10-06T05:23:51.079325+0000","last_clean":"2023-10-06T05:23:51.079325+0000","last_became_active":"2023-10-06T05:23:51.079016+0000","last_became_peered":"2023-10-06T05:23:51.079016+0000","last_unstale":"2023-10-06T05:23:51.079325+0000","last_undegraded":"2023-10-06T05:23:51.079325+0000","last_fullsized":"2023-10-06T05:23:51.079325+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:22.620204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.083218+0000","last_change":"2023-10-06T05:23:51.083218+0000","last_active":"2023-10-06T05:23:51.083218+0000","last_peered":"2023-10-06T05:23:51.083218+0000","last_clean":"2023-10-06T05:23:51.083218+0000","last_became_active":"2023-10-06T05:23:51.082952+0000","last_became_peered":"2023-10-06T05:23:51.082952+0000","last_unstale":"2023-10-06T05:23:51.083218+0000","last_undegraded":"2023-10-06T05:23:51.083218+0000","last_fullsized":"2023-10-06T05:23:51.083218+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:46.425293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.067315+0000","last_change":"2023-10-06T05:23:51.067315+0000","last_active":"2023-10-06T05:23:51.067315+0000","last_peered":"2023-10-06T05:23:51.067315+0000","last_clean":"2023-10-06T05:23:51.067315+0000","last_became_active":"2023-10-06T05:23:51.067063+0000","last_became_peered":"2023-10-06T05:23:51.067063+0000","last_unstale":"2023-10-06T05:23:51.067315+0000","last_undegraded":"2023-10-06T05:23:51.067315+0000","last_fullsized":"2023-10-06T05:23:51.067315+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:44.414574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.087872+0000","last_change":"2023-10-06T05:23:51.087872+0000","last_active":"2023-10-06T05:23:51.087872+0000","last_peered":"2023-10-06T05:23:51.087872+0000","last_clean":"2023-10-06T05:23:51.087872+0000","last_became_active":"2023-10-06T05:23:51.087519+0000","last_became_peered":"2023-10-06T05:23:51.087519+0000","last_unstale":"2023-10-06T05:23:51.087872+0000","last_undegraded":"2023-10-06T05:23:51.087872+0000","last_fullsized":"2023-10-06T05:23:51.087872+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:35.419217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077651+0000","last_change":"2023-10-06T05:23:51.077651+0000","last_active":"2023-10-06T05:23:51.077651+0000","last_peered":"2023-10-06T05:23:51.077651+0000","last_clean":"2023-10-06T05:23:51.077651+0000","last_became_active":"2023-10-06T05:23:51.077383+0000","last_became_peered":"2023-10-06T05:23:51.077383+0000","last_unstale":"2023-10-06T05:23:51.077651+0000","last_undegraded":"2023-10-06T05:23:51.077651+0000","last_fullsized":"2023-10-06T05:23:51.077651+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:56.362092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.078292+0000","last_change":"2023-10-06T05:23:51.078292+0000","last_active":"2023-10-06T05:23:51.078292+0000","last_peered":"2023-10-06T05:23:51.078292+0000","last_clean":"2023-10-06T05:23:51.078292+0000","last_became_active":"2023-10-06T05:23:51.078025+0000","last_became_peered":"2023-10-06T05:23:51.078025+0000","last_unstale":"2023-10-06T05:23:51.078292+0000","last_undegraded":"2023-10-06T05:23:51.078292+0000","last_fullsized":"2023-10-06T05:23:51.078292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:31.168313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.066356+0000","last_change":"2023-10-06T05:23:51.066356+0000","last_active":"2023-10-06T05:23:51.066356+0000","last_peered":"2023-10-06T05:23:51.066356+0000","last_clean":"2023-10-06T05:23:51.066356+0000","last_became_active":"2023-10-06T05:23:51.066097+0000","last_became_peered":"2023-10-06T05:23:51.066097+0000","last_unstale":"2023-10-06T05:23:51.066356+0000","last_undegraded":"2023-10-06T05:23:51.066356+0000","last_fullsized":"2023-10-06T05:23:51.066356+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:25.508597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.066171+0000","last_change":"2023-10-06T05:23:51.066171+0000","last_active":"2023-10-06T05:23:51.066171+0000","last_peered":"2023-10-06T05:23:51.066171+0000","last_clean":"2023-10-06T05:23:51.066171+0000","last_became_active":"2023-10-06T05:23:51.065904+0000","last_became_peered":"2023-10-06T05:23:51.065904+0000","last_unstale":"2023-10-06T05:23:51.066171+0000","last_undegraded":"2023-10-06T05:23:51.066171+0000","last_fullsized":"2023-10-06T05:23:51.066171+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:40.750249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.073566+0000","last_change":"2023-10-06T05:23:51.073566+0000","last_active":"2023-10-06T05:23:51.073566+0000","last_peered":"2023-10-06T05:23:51.073566+0000","last_clean":"2023-10-06T05:23:51.073566+0000","last_became_active":"2023-10-06T05:23:51.073283+0000","last_became_peered":"2023-10-06T05:23:51.073283+0000","last_unstale":"2023-10-06T05:23:51.073566+0000","last_undegraded":"2023-10-06T05:23:51.073566+0000","last_fullsized":"2023-10-06T05:23:51.073566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.073404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.087657+0000","last_change":"2023-10-06T05:23:51.087657+0000","last_active":"2023-10-06T05:23:51.087657+0000","last_peered":"2023-10-06T05:23:51.087657+0000","last_clean":"2023-10-06T05:23:51.087657+0000","last_became_active":"2023-10-06T05:23:51.087283+0000","last_became_peered":"2023-10-06T05:23:51.087283+0000","last_unstale":"2023-10-06T05:23:51.087657+0000","last_undegraded":"2023-10-06T05:23:51.087657+0000","last_fullsized":"2023-10-06T05:23:51.087657+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:03.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076591+0000","last_change":"2023-10-06T05:23:51.076591+0000","last_active":"2023-10-06T05:23:51.076591+0000","last_peered":"2023-10-06T05:23:51.076591+0000","last_clean":"2023-10-06T05:23:51.076591+0000","last_became_active":"2023-10-06T05:23:51.076247+0000","last_became_peered":"2023-10-06T05:23:51.076247+0000","last_unstale":"2023-10-06T05:23:51.076591+0000","last_undegraded":"2023-10-06T05:23:51.076591+0000","last_fullsized":"2023-10-06T05:23:51.076591+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:53.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077776+0000","last_change":"2023-10-06T05:23:51.077776+0000","last_active":"2023-10-06T05:23:51.077776+0000","last_peered":"2023-10-06T05:23:51.077776+0000","last_clean":"2023-10-06T05:23:51.077776+0000","last_became_active":"2023-10-06T05:23:51.077524+0000","last_became_peered":"2023-10-06T05:23:51.077524+0000","last_unstale":"2023-10-06T05:23:51.077776+0000","last_undegraded":"2023-10-06T05:23:51.077776+0000","last_fullsized":"2023-10-06T05:23:51.077776+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:13.365095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075684+0000","last_change":"2023-10-06T05:23:51.075684+0000","last_active":"2023-10-06T05:23:51.075684+0000","last_peered":"2023-10-06T05:23:51.075684+0000","last_clean":"2023-10-06T05:23:51.075684+0000","last_became_active":"2023-10-06T05:23:51.075411+0000","last_became_peered":"2023-10-06T05:23:51.075411+0000","last_unstale":"2023-10-06T05:23:51.075684+0000","last_undegraded":"2023-10-06T05:23:51.075684+0000","last_fullsized":"2023-10-06T05:23:51.075684+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:31.650696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079748+0000","last_change":"2023-10-06T05:23:51.079748+0000","last_active":"2023-10-06T05:23:51.079748+0000","last_peered":"2023-10-06T05:23:51.079748+0000","last_clean":"2023-10-06T05:23:51.079748+0000","last_became_active":"2023-10-06T05:23:51.078573+0000","last_became_peered":"2023-10-06T05:23:51.078573+0000","last_unstale":"2023-10-06T05:23:51.079748+0000","last_undegraded":"2023-10-06T05:23:51.079748+0000","last_fullsized":"2023-10-06T05:23:51.079748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:19.355231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075535+0000","last_change":"2023-10-06T05:23:51.075535+0000","last_active":"2023-10-06T05:23:51.075535+0000","last_peered":"2023-10-06T05:23:51.075535+0000","last_clean":"2023-10-06T05:23:51.075535+0000","last_became_active":"2023-10-06T05:23:51.075264+0000","last_became_peered":"2023-10-06T05:23:51.075264+0000","last_unstale":"2023-10-06T05:23:51.075535+0000","last_undegraded":"2023-10-06T05:23:51.075535+0000","last_fullsized":"2023-10-06T05:23:51.075535+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:14.269604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.081084+0000","last_change":"2023-10-06T05:23:51.081084+0000","last_active":"2023-10-06T05:23:51.081084+0000","last_peered":"2023-10-06T05:23:51.081084+0000","last_clean":"2023-10-06T05:23:51.081084+0000","last_became_active":"2023-10-06T05:23:51.080683+0000","last_became_peered":"2023-10-06T05:23:51.080683+0000","last_unstale":"2023-10-06T05:23:51.081084+0000","last_undegraded":"2023-10-06T05:23:51.081084+0000","last_fullsized":"2023-10-06T05:23:51.081084+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:40.852207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080407+0000","last_change":"2023-10-06T05:23:51.080407+0000","last_active":"2023-10-06T05:23:51.080407+0000","last_peered":"2023-10-06T05:23:51.080407+0000","last_clean":"2023-10-06T05:23:51.080407+0000","last_became_active":"2023-10-06T05:23:51.080092+0000","last_became_peered":"2023-10-06T05:23:51.080092+0000","last_unstale":"2023-10-06T05:23:51.080407+0000","last_undegraded":"2023-10-06T05:23:51.080407+0000","last_fullsized":"2023-10-06T05:23:51.080407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:34.807526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080782+0000","last_change":"2023-10-06T05:23:51.080782+0000","last_active":"2023-10-06T05:23:51.080782+0000","last_peered":"2023-10-06T05:23:51.080782+0000","last_clean":"2023-10-06T05:23:51.080782+0000","last_became_active":"2023-10-06T05:23:51.080515+0000","last_became_peered":"2023-10-06T05:23:51.080515+0000","last_unstale":"2023-10-06T05:23:51.080782+0000","last_undegraded":"2023-10-06T05:23:51.080782+0000","last_fullsized":"2023-10-06T05:23:51.080782+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:30.675809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.074824+0000","last_change":"2023-10-06T05:23:51.074824+0000","last_active":"2023-10-06T05:23:51.074824+0000","last_peered":"2023-10-06T05:23:51.074824+0000","last_clean":"2023-10-06T05:23:51.074824+0000","last_became_active":"2023-10-06T05:23:51.074491+0000","last_became_peered":"2023-10-06T05:23:51.074491+0000","last_unstale":"2023-10-06T05:23:51.074824+0000","last_undegraded":"2023-10-06T05:23:51.074824+0000","last_fullsized":"2023-10-06T05:23:51.074824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:22.215131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076563+0000","last_change":"2023-10-06T05:23:51.076563+0000","last_active":"2023-10-06T05:23:51.076563+0000","last_peered":"2023-10-06T05:23:51.076563+0000","last_clean":"2023-10-06T05:23:51.076563+0000","last_became_active":"2023-10-06T05:23:51.075590+0000","last_became_peered":"2023-10-06T05:23:51.075590+0000","last_unstale":"2023-10-06T05:23:51.076563+0000","last_undegraded":"2023-10-06T05:23:51.076563+0000","last_fullsized":"2023-10-06T05:23:51.076563+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:38.572213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075703+0000","last_change":"2023-10-06T05:23:51.075703+0000","last_active":"2023-10-06T05:23:51.075703+0000","last_peered":"2023-10-06T05:23:51.075703+0000","last_clean":"2023-10-06T05:23:51.075703+0000","last_became_active":"2023-10-06T05:23:51.075193+0000","last_became_peered":"2023-10-06T05:23:51.075193+0000","last_unstale":"2023-10-06T05:23:51.075703+0000","last_undegraded":"2023-10-06T05:23:51.075703+0000","last_fullsized":"2023-10-06T05:23:51.075703+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:34.268347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.078954+0000","last_change":"2023-10-06T05:23:51.078954+0000","last_active":"2023-10-06T05:23:51.078954+0000","last_peered":"2023-10-06T05:23:51.078954+0000","last_clean":"2023-10-06T05:23:51.078954+0000","last_became_active":"2023-10-06T05:23:51.078673+0000","last_became_peered":"2023-10-06T05:23:51.078673+0000","last_unstale":"2023-10-06T05:23:51.078954+0000","last_undegraded":"2023-10-06T05:23:51.078954+0000","last_fullsized":"2023-10-06T05:23:51.078954+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.067708+0000","last_change":"2023-10-06T05:23:51.067708+0000","last_active":"2023-10-06T05:23:51.067708+0000","last_peered":"2023-10-06T05:23:51.067708+0000","last_clean":"2023-10-06T05:23:51.067708+0000","last_became_active":"2023-10-06T05:23:51.067368+0000","last_became_peered":"2023-10-06T05:23:51.067368+0000","last_unstale":"2023-10-06T05:23:51.067708+0000","last_undegraded":"2023-10-06T05:23:51.067708+0000","last_fullsized":"2023-10-06T05:23:51.067708+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:45.971146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079361+0000","last_change":"2023-10-06T05:23:51.079361+0000","last_active":"2023-10-06T05:23:51.079361+0000","last_peered":"2023-10-06T05:23:51.079361+0000","last_clean":"2023-10-06T05:23:51.079361+0000","last_became_active":"2023-10-06T05:23:51.079083+0000","last_became_peered":"2023-10-06T05:23:51.079083+0000","last_unstale":"2023-10-06T05:23:51.079361+0000","last_undegraded":"2023-10-06T05:23:51.079361+0000","last_fullsized":"2023-10-06T05:23:51.079361+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:25.508597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.100493+0000","last_change":"2023-10-06T05:23:50.075471+0000","last_active":"2023-10-06T05:23:51.100493+0000","last_peered":"2023-10-06T05:23:51.100493+0000","last_clean":"2023-10-06T05:23:51.100493+0000","last_became_active":"2023-10-06T05:23:50.075089+0000","last_became_peered":"2023-10-06T05:23:50.075089+0000","last_unstale":"2023-10-06T05:23:51.100493+0000","last_undegraded":"2023-10-06T05:23:51.100493+0000","last_fullsized":"2023-10-06T05:23:51.100493+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:59.250058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.074895+0000","last_change":"2023-10-06T05:23:51.074895+0000","last_active":"2023-10-06T05:23:51.074895+0000","last_peered":"2023-10-06T05:23:51.074895+0000","last_clean":"2023-10-06T05:23:51.074895+0000","last_became_active":"2023-10-06T05:23:51.074589+0000","last_became_peered":"2023-10-06T05:23:51.074589+0000","last_unstale":"2023-10-06T05:23:51.074895+0000","last_undegraded":"2023-10-06T05:23:51.074895+0000","last_fullsized":"2023-10-06T05:23:51.074895+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:01.362870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.064714+0000","last_change":"2023-10-06T05:23:50.083940+0000","last_active":"2023-10-06T05:23:51.064714+0000","last_peered":"2023-10-06T05:23:51.064714+0000","last_clean":"2023-10-06T05:23:51.064714+0000","last_became_active":"2023-10-06T05:23:50.083651+0000","last_became_peered":"2023-10-06T05:23:50.083651+0000","last_unstale":"2023-10-06T05:23:51.064714+0000","last_undegraded":"2023-10-06T05:23:51.064714+0000","last_fullsized":"2023-10-06T05:23:51.064714+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:01.198348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077168+0000","last_change":"2023-10-06T05:23:51.077168+0000","last_active":"2023-10-06T05:23:51.077168+0000","last_peered":"2023-10-06T05:23:51.077168+0000","last_clean":"2023-10-06T05:23:51.077168+0000","last_became_active":"2023-10-06T05:23:51.076910+0000","last_became_peered":"2023-10-06T05:23:51.076910+0000","last_unstale":"2023-10-06T05:23:51.077168+0000","last_undegraded":"2023-10-06T05:23:51.077168+0000","last_fullsized":"2023-10-06T05:23:51.077168+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:07.210197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.095445+0000","last_change":"2023-10-06T05:23:50.079705+0000","last_active":"2023-10-06T05:23:51.095445+0000","last_peered":"2023-10-06T05:23:51.095445+0000","last_clean":"2023-10-06T05:23:51.095445+0000","last_became_active":"2023-10-06T05:23:50.079452+0000","last_became_peered":"2023-10-06T05:23:50.079452+0000","last_unstale":"2023-10-06T05:23:51.095445+0000","last_undegraded":"2023-10-06T05:23:51.095445+0000","last_fullsized":"2023-10-06T05:23:51.095445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:24.500717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077039+0000","last_change":"2023-10-06T05:23:51.077039+0000","last_active":"2023-10-06T05:23:51.077039+0000","last_peered":"2023-10-06T05:23:51.077039+0000","last_clean":"2023-10-06T05:23:51.077039+0000","last_became_active":"2023-10-06T05:23:51.076797+0000","last_became_peered":"2023-10-06T05:23:51.076797+0000","last_unstale":"2023-10-06T05:23:51.077039+0000","last_undegraded":"2023-10-06T05:23:51.077039+0000","last_fullsized":"2023-10-06T05:23:51.077039+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:57.105260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.095715+0000","last_change":"2023-10-06T05:23:50.083731+0000","last_active":"2023-10-06T05:23:51.095715+0000","last_peered":"2023-10-06T05:23:51.095715+0000","last_clean":"2023-10-06T05:23:51.095715+0000","last_became_active":"2023-10-06T05:23:50.083461+0000","last_became_peered":"2023-10-06T05:23:50.083461+0000","last_unstale":"2023-10-06T05:23:51.095715+0000","last_undegraded":"2023-10-06T05:23:51.095715+0000","last_fullsized":"2023-10-06T05:23:51.095715+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:15.738326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079616+0000","last_change":"2023-10-06T05:23:51.079616+0000","last_active":"2023-10-06T05:23:51.079616+0000","last_peered":"2023-10-06T05:23:51.079616+0000","last_clean":"2023-10-06T05:23:51.079616+0000","last_became_active":"2023-10-06T05:23:51.079321+0000","last_became_peered":"2023-10-06T05:23:51.079321+0000","last_unstale":"2023-10-06T05:23:51.079616+0000","last_undegraded":"2023-10-06T05:23:51.079616+0000","last_fullsized":"2023-10-06T05:23:51.079616+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:51.386080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.073933+0000","last_change":"2023-10-06T05:23:50.081102+0000","last_active":"2023-10-06T05:23:51.073933+0000","last_peered":"2023-10-06T05:23:51.073933+0000","last_clean":"2023-10-06T05:23:51.073933+0000","last_became_active":"2023-10-06T05:23:50.080748+0000","last_became_peered":"2023-10-06T05:23:50.080748+0000","last_unstale":"2023-10-06T05:23:51.073933+0000","last_undegraded":"2023-10-06T05:23:51.073933+0000","last_fullsized":"2023-10-06T05:23:51.073933+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:10.164174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.087909+0000","last_change":"2023-10-06T05:23:51.087909+0000","last_active":"2023-10-06T05:23:51.087909+0000","last_peered":"2023-10-06T05:23:51.087909+0000","last_clean":"2023-10-06T05:23:51.087909+0000","last_became_active":"2023-10-06T05:23:51.087616+0000","last_became_peered":"2023-10-06T05:23:51.087616+0000","last_unstale":"2023-10-06T05:23:51.087909+0000","last_undegraded":"2023-10-06T05:23:51.087909+0000","last_fullsized":"2023-10-06T05:23:51.087909+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:07.278307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.061552+0000","last_change":"2023-10-06T05:23:50.081063+0000","last_active":"2023-10-06T05:23:51.061552+0000","last_peered":"2023-10-06T05:23:51.061552+0000","last_clean":"2023-10-06T05:23:51.061552+0000","last_became_active":"2023-10-06T05:23:50.080819+0000","last_became_peered":"2023-10-06T05:23:50.080819+0000","last_unstale":"2023-10-06T05:23:51.061552+0000","last_undegraded":"2023-10-06T05:23:51.061552+0000","last_fullsized":"2023-10-06T05:23:51.061552+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:10.164174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.078129+0000","last_change":"2023-10-06T05:23:51.078129+0000","last_active":"2023-10-06T05:23:51.078129+0000","last_peered":"2023-10-06T05:23:51.078129+0000","last_clean":"2023-10-06T05:23:51.078129+0000","last_became_active":"2023-10-06T05:23:51.077885+0000","last_became_peered":"2023-10-06T05:23:51.077885+0000","last_unstale":"2023-10-06T05:23:51.078129+0000","last_undegraded":"2023-10-06T05:23:51.078129+0000","last_fullsized":"2023-10-06T05:23:51.078129+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:13.687528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.082921+0000","last_change":"2023-10-06T05:23:50.077384+0000","last_active":"2023-10-06T05:23:50.082921+0000","last_peered":"2023-10-06T05:23:50.082921+0000","last_clean":"2023-10-06T05:23:50.082921+0000","last_became_active":"2023-10-06T05:23:50.077080+0000","last_became_peered":"2023-10-06T05:23:50.077080+0000","last_unstale":"2023-10-06T05:23:50.082921+0000","last_undegraded":"2023-10-06T05:23:50.082921+0000","last_fullsized":"2023-10-06T05:23:50.082921+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:03.576359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.083004+0000","last_change":"2023-10-06T05:23:51.083004+0000","last_active":"2023-10-06T05:23:51.083004+0000","last_peered":"2023-10-06T05:23:51.083004+0000","last_clean":"2023-10-06T05:23:51.083004+0000","last_became_active":"2023-10-06T05:23:51.082726+0000","last_became_peered":"2023-10-06T05:23:51.082726+0000","last_unstale":"2023-10-06T05:23:51.083004+0000","last_undegraded":"2023-10-06T05:23:51.083004+0000","last_fullsized":"2023-10-06T05:23:51.083004+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:11.377016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079439+0000","last_change":"2023-10-06T05:23:50.072076+0000","last_active":"2023-10-06T05:23:50.079439+0000","last_peered":"2023-10-06T05:23:50.079439+0000","last_clean":"2023-10-06T05:23:50.079439+0000","last_became_active":"2023-10-06T05:23:50.071797+0000","last_became_peered":"2023-10-06T05:23:50.071797+0000","last_unstale":"2023-10-06T05:23:50.079439+0000","last_undegraded":"2023-10-06T05:23:50.079439+0000","last_fullsized":"2023-10-06T05:23:50.079439+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:05.014533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076750+0000","last_change":"2023-10-06T05:23:51.076750+0000","last_active":"2023-10-06T05:23:51.076750+0000","last_peered":"2023-10-06T05:23:51.076750+0000","last_clean":"2023-10-06T05:23:51.076750+0000","last_became_active":"2023-10-06T05:23:51.075611+0000","last_became_peered":"2023-10-06T05:23:51.075611+0000","last_unstale":"2023-10-06T05:23:51.076750+0000","last_undegraded":"2023-10-06T05:23:51.076750+0000","last_fullsized":"2023-10-06T05:23:51.076750+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:14.296341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.068172+0000","last_change":"2023-10-06T05:23:50.076841+0000","last_active":"2023-10-06T05:23:51.068172+0000","last_peered":"2023-10-06T05:23:51.068172+0000","last_clean":"2023-10-06T05:23:51.068172+0000","last_became_active":"2023-10-06T05:23:50.076338+0000","last_became_peered":"2023-10-06T05:23:50.076338+0000","last_unstale":"2023-10-06T05:23:51.068172+0000","last_undegraded":"2023-10-06T05:23:51.068172+0000","last_fullsized":"2023-10-06T05:23:51.068172+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:33.938775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076341+0000","last_change":"2023-10-06T05:23:51.076341+0000","last_active":"2023-10-06T05:23:51.076341+0000","last_peered":"2023-10-06T05:23:51.076341+0000","last_clean":"2023-10-06T05:23:51.076341+0000","last_became_active":"2023-10-06T05:23:51.075961+0000","last_became_peered":"2023-10-06T05:23:51.075961+0000","last_unstale":"2023-10-06T05:23:51.076341+0000","last_undegraded":"2023-10-06T05:23:51.076341+0000","last_fullsized":"2023-10-06T05:23:51.076341+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:27.051904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.095843+0000","last_change":"2023-10-06T05:23:50.076095+0000","last_active":"2023-10-06T05:23:51.095843+0000","last_peered":"2023-10-06T05:23:51.095843+0000","last_clean":"2023-10-06T05:23:51.095843+0000","last_became_active":"2023-10-06T05:23:50.075785+0000","last_became_peered":"2023-10-06T05:23:50.075785+0000","last_unstale":"2023-10-06T05:23:51.095843+0000","last_undegraded":"2023-10-06T05:23:51.095843+0000","last_fullsized":"2023-10-06T05:23:51.095843+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:56.097379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079327+0000","last_change":"2023-10-06T05:23:51.079327+0000","last_active":"2023-10-06T05:23:51.079327+0000","last_peered":"2023-10-06T05:23:51.079327+0000","last_clean":"2023-10-06T05:23:51.079327+0000","last_became_active":"2023-10-06T05:23:51.079073+0000","last_became_peered":"2023-10-06T05:23:51.079073+0000","last_unstale":"2023-10-06T05:23:51.079327+0000","last_undegraded":"2023-10-06T05:23:51.079327+0000","last_fullsized":"2023-10-06T05:23:51.079327+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:25.543598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.060721+0000","last_change":"2023-10-06T05:23:50.076933+0000","last_active":"2023-10-06T05:23:51.060721+0000","last_peered":"2023-10-06T05:23:51.060721+0000","last_clean":"2023-10-06T05:23:51.060721+0000","last_became_active":"2023-10-06T05:23:50.076658+0000","last_became_peered":"2023-10-06T05:23:50.076658+0000","last_unstale":"2023-10-06T05:23:51.060721+0000","last_undegraded":"2023-10-06T05:23:51.060721+0000","last_fullsized":"2023-10-06T05:23:51.060721+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:33.938775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.085864+0000","last_change":"2023-10-06T05:23:51.085864+0000","last_active":"2023-10-06T05:23:51.085864+0000","last_peered":"2023-10-06T05:23:51.085864+0000","last_clean":"2023-10-06T05:23:51.085864+0000","last_became_active":"2023-10-06T05:23:51.085578+0000","last_became_peered":"2023-10-06T05:23:51.085578+0000","last_unstale":"2023-10-06T05:23:51.085864+0000","last_undegraded":"2023-10-06T05:23:51.085864+0000","last_fullsized":"2023-10-06T05:23:51.085864+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:38.621529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.080513+0000","last_change":"2023-10-06T05:23:50.076213+0000","last_active":"2023-10-06T05:23:50.080513+0000","last_peered":"2023-10-06T05:23:50.080513+0000","last_clean":"2023-10-06T05:23:50.080513+0000","last_became_active":"2023-10-06T05:23:50.075892+0000","last_became_peered":"2023-10-06T05:23:50.075892+0000","last_unstale":"2023-10-06T05:23:50.080513+0000","last_undegraded":"2023-10-06T05:23:50.080513+0000","last_fullsized":"2023-10-06T05:23:50.080513+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:12.357215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.068889+0000","last_change":"2023-10-06T05:23:51.068889+0000","last_active":"2023-10-06T05:23:51.068889+0000","last_peered":"2023-10-06T05:23:51.068889+0000","last_clean":"2023-10-06T05:23:51.068889+0000","last_became_active":"2023-10-06T05:23:51.068660+0000","last_became_peered":"2023-10-06T05:23:51.068660+0000","last_unstale":"2023-10-06T05:23:51.068889+0000","last_undegraded":"2023-10-06T05:23:51.068889+0000","last_fullsized":"2023-10-06T05:23:51.068889+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:07.278307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.100733+0000","last_change":"2023-10-06T05:23:50.084943+0000","last_active":"2023-10-06T05:23:51.100733+0000","last_peered":"2023-10-06T05:23:51.100733+0000","last_clean":"2023-10-06T05:23:51.100733+0000","last_became_active":"2023-10-06T05:23:50.084658+0000","last_became_peered":"2023-10-06T05:23:50.084658+0000","last_unstale":"2023-10-06T05:23:51.100733+0000","last_undegraded":"2023-10-06T05:23:51.100733+0000","last_fullsized":"2023-10-06T05:23:51.100733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:06.202316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076463+0000","last_change":"2023-10-06T05:23:51.076463+0000","last_active":"2023-10-06T05:23:51.076463+0000","last_peered":"2023-10-06T05:23:51.076463+0000","last_clean":"2023-10-06T05:23:51.076463+0000","last_became_active":"2023-10-06T05:23:51.076198+0000","last_became_peered":"2023-10-06T05:23:51.076198+0000","last_unstale":"2023-10-06T05:23:51.076463+0000","last_undegraded":"2023-10-06T05:23:51.076463+0000","last_fullsized":"2023-10-06T05:23:51.076463+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:08.568729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083593+0000","last_change":"2023-10-06T05:23:50.081273+0000","last_active":"2023-10-06T05:23:50.083593+0000","last_peered":"2023-10-06T05:23:50.083593+0000","last_clean":"2023-10-06T05:23:50.083593+0000","last_became_active":"2023-10-06T05:23:50.080998+0000","last_became_peered":"2023-10-06T05:23:50.080998+0000","last_unstale":"2023-10-06T05:23:50.083593+0000","last_undegraded":"2023-10-06T05:23:50.083593+0000","last_fullsized":"2023-10-06T05:23:50.083593+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:07.554956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.082242+0000","last_change":"2023-10-06T05:23:51.082242+0000","last_active":"2023-10-06T05:23:51.082242+0000","last_peered":"2023-10-06T05:23:51.082242+0000","last_clean":"2023-10-06T05:23:51.082242+0000","last_became_active":"2023-10-06T05:23:51.081961+0000","last_became_peered":"2023-10-06T05:23:51.081961+0000","last_unstale":"2023-10-06T05:23:51.082242+0000","last_undegraded":"2023-10-06T05:23:51.082242+0000","last_fullsized":"2023-10-06T05:23:51.082242+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:58.865079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083557+0000","last_change":"2023-10-06T05:23:50.077858+0000","last_active":"2023-10-06T05:23:50.083557+0000","last_peered":"2023-10-06T05:23:50.083557+0000","last_clean":"2023-10-06T05:23:50.083557+0000","last_became_active":"2023-10-06T05:23:50.077581+0000","last_became_peered":"2023-10-06T05:23:50.077581+0000","last_unstale":"2023-10-06T05:23:50.083557+0000","last_undegraded":"2023-10-06T05:23:50.083557+0000","last_fullsized":"2023-10-06T05:23:50.083557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:51.219335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.083788+0000","last_change":"2023-10-06T05:23:51.083788+0000","last_active":"2023-10-06T05:23:51.083788+0000","last_peered":"2023-10-06T05:23:51.083788+0000","last_clean":"2023-10-06T05:23:51.083788+0000","last_became_active":"2023-10-06T05:23:51.083549+0000","last_became_peered":"2023-10-06T05:23:51.083549+0000","last_unstale":"2023-10-06T05:23:51.083788+0000","last_undegraded":"2023-10-06T05:23:51.083788+0000","last_fullsized":"2023-10-06T05:23:51.083788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:20.047358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079684+0000","last_change":"2023-10-06T05:23:50.075840+0000","last_active":"2023-10-06T05:23:50.079684+0000","last_peered":"2023-10-06T05:23:50.079684+0000","last_clean":"2023-10-06T05:23:50.079684+0000","last_became_active":"2023-10-06T05:23:50.075571+0000","last_became_peered":"2023-10-06T05:23:50.075571+0000","last_unstale":"2023-10-06T05:23:50.079684+0000","last_undegraded":"2023-10-06T05:23:50.079684+0000","last_fullsized":"2023-10-06T05:23:50.079684+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:19.886288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077345+0000","last_change":"2023-10-06T05:23:51.077345+0000","last_active":"2023-10-06T05:23:51.077345+0000","last_peered":"2023-10-06T05:23:51.077345+0000","last_clean":"2023-10-06T05:23:51.077345+0000","last_became_active":"2023-10-06T05:23:51.076973+0000","last_became_peered":"2023-10-06T05:23:51.076973+0000","last_unstale":"2023-10-06T05:23:51.077345+0000","last_undegraded":"2023-10-06T05:23:51.077345+0000","last_fullsized":"2023-10-06T05:23:51.077345+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:13.400120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.096315+0000","last_change":"2023-10-06T05:23:50.079113+0000","last_active":"2023-10-06T05:23:51.096315+0000","last_peered":"2023-10-06T05:23:51.096315+0000","last_clean":"2023-10-06T05:23:51.096315+0000","last_became_active":"2023-10-06T05:23:50.078834+0000","last_became_peered":"2023-10-06T05:23:50.078834+0000","last_unstale":"2023-10-06T05:23:51.096315+0000","last_undegraded":"2023-10-06T05:23:51.096315+0000","last_fullsized":"2023-10-06T05:23:51.096315+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:48.332248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079234+0000","last_change":"2023-10-06T05:23:51.079234+0000","last_active":"2023-10-06T05:23:51.079234+0000","last_peered":"2023-10-06T05:23:51.079234+0000","last_clean":"2023-10-06T05:23:51.079234+0000","last_became_active":"2023-10-06T05:23:51.078368+0000","last_became_peered":"2023-10-06T05:23:51.078368+0000","last_unstale":"2023-10-06T05:23:51.079234+0000","last_undegraded":"2023-10-06T05:23:51.079234+0000","last_fullsized":"2023-10-06T05:23:51.079234+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:11.309566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.087223+0000","last_change":"2023-10-06T05:23:50.084353+0000","last_active":"2023-10-06T05:23:50.087223+0000","last_peered":"2023-10-06T05:23:50.087223+0000","last_clean":"2023-10-06T05:23:50.087223+0000","last_became_active":"2023-10-06T05:23:50.084044+0000","last_became_peered":"2023-10-06T05:23:50.084044+0000","last_unstale":"2023-10-06T05:23:50.087223+0000","last_undegraded":"2023-10-06T05:23:50.087223+0000","last_fullsized":"2023-10-06T05:23:50.087223+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:51.219335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.086357+0000","last_change":"2023-10-06T05:23:51.086357+0000","last_active":"2023-10-06T05:23:51.086357+0000","last_peered":"2023-10-06T05:23:51.086357+0000","last_clean":"2023-10-06T05:23:51.086357+0000","last_became_active":"2023-10-06T05:23:51.085974+0000","last_became_peered":"2023-10-06T05:23:51.085974+0000","last_unstale":"2023-10-06T05:23:51.086357+0000","last_undegraded":"2023-10-06T05:23:51.086357+0000","last_fullsized":"2023-10-06T05:23:51.086357+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:26.967879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083783+0000","last_change":"2023-10-06T05:23:50.080671+0000","last_active":"2023-10-06T05:23:50.083783+0000","last_peered":"2023-10-06T05:23:50.083783+0000","last_clean":"2023-10-06T05:23:50.083783+0000","last_became_active":"2023-10-06T05:23:50.080347+0000","last_became_peered":"2023-10-06T05:23:50.080347+0000","last_unstale":"2023-10-06T05:23:50.083783+0000","last_undegraded":"2023-10-06T05:23:50.083783+0000","last_fullsized":"2023-10-06T05:23:50.083783+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:12.357215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079756+0000","last_change":"2023-10-06T05:23:51.079756+0000","last_active":"2023-10-06T05:23:51.079756+0000","last_peered":"2023-10-06T05:23:51.079756+0000","last_clean":"2023-10-06T05:23:51.079756+0000","last_became_active":"2023-10-06T05:23:51.079524+0000","last_became_peered":"2023-10-06T05:23:51.079524+0000","last_unstale":"2023-10-06T05:23:51.079756+0000","last_undegraded":"2023-10-06T05:23:51.079756+0000","last_fullsized":"2023-10-06T05:23:51.079756+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.190926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079644+0000","last_change":"2023-10-06T05:23:50.073916+0000","last_active":"2023-10-06T05:23:50.079644+0000","last_peered":"2023-10-06T05:23:50.079644+0000","last_clean":"2023-10-06T05:23:50.079644+0000","last_became_active":"2023-10-06T05:23:50.073627+0000","last_became_peered":"2023-10-06T05:23:50.073627+0000","last_unstale":"2023-10-06T05:23:50.079644+0000","last_undegraded":"2023-10-06T05:23:50.079644+0000","last_fullsized":"2023-10-06T05:23:50.079644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:55.310330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.070656+0000","last_change":"2023-10-06T05:23:51.070656+0000","last_active":"2023-10-06T05:23:51.070656+0000","last_peered":"2023-10-06T05:23:51.070656+0000","last_clean":"2023-10-06T05:23:51.070656+0000","last_became_active":"2023-10-06T05:23:51.070406+0000","last_became_peered":"2023-10-06T05:23:51.070406+0000","last_unstale":"2023-10-06T05:23:51.070656+0000","last_undegraded":"2023-10-06T05:23:51.070656+0000","last_fullsized":"2023-10-06T05:23:51.070656+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:52.227215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.082484+0000","last_change":"2023-10-06T05:23:50.078002+0000","last_active":"2023-10-06T05:23:50.082484+0000","last_peered":"2023-10-06T05:23:50.082484+0000","last_clean":"2023-10-06T05:23:50.082484+0000","last_became_active":"2023-10-06T05:23:50.077713+0000","last_became_peered":"2023-10-06T05:23:50.077713+0000","last_unstale":"2023-10-06T05:23:50.082484+0000","last_undegraded":"2023-10-06T05:23:50.082484+0000","last_fullsized":"2023-10-06T05:23:50.082484+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:56.520298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.071541+0000","last_change":"2023-10-06T05:23:51.071541+0000","last_active":"2023-10-06T05:23:51.071541+0000","last_peered":"2023-10-06T05:23:51.071541+0000","last_clean":"2023-10-06T05:23:51.071541+0000","last_became_active":"2023-10-06T05:23:51.071271+0000","last_became_peered":"2023-10-06T05:23:51.071271+0000","last_unstale":"2023-10-06T05:23:51.071541+0000","last_undegraded":"2023-10-06T05:23:51.071541+0000","last_fullsized":"2023-10-06T05:23:51.071541+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:02.206229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.086015+0000","last_change":"2023-10-06T05:23:50.081614+0000","last_active":"2023-10-06T05:23:50.086015+0000","last_peered":"2023-10-06T05:23:50.086015+0000","last_clean":"2023-10-06T05:23:50.086015+0000","last_became_active":"2023-10-06T05:23:50.081260+0000","last_became_peered":"2023-10-06T05:23:50.081260+0000","last_unstale":"2023-10-06T05:23:50.086015+0000","last_undegraded":"2023-10-06T05:23:50.086015+0000","last_fullsized":"2023-10-06T05:23:50.086015+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:31.519158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080754+0000","last_change":"2023-10-06T05:23:51.080754+0000","last_active":"2023-10-06T05:23:51.080754+0000","last_peered":"2023-10-06T05:23:51.080754+0000","last_clean":"2023-10-06T05:23:51.080754+0000","last_became_active":"2023-10-06T05:23:51.080491+0000","last_became_peered":"2023-10-06T05:23:51.080491+0000","last_unstale":"2023-10-06T05:23:51.080754+0000","last_undegraded":"2023-10-06T05:23:51.080754+0000","last_fullsized":"2023-10-06T05:23:51.080754+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:32.527038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.074384+0000","last_change":"2023-10-06T05:23:50.067624+0000","last_active":"2023-10-06T05:23:50.074384+0000","last_peered":"2023-10-06T05:23:50.074384+0000","last_clean":"2023-10-06T05:23:50.074384+0000","last_became_active":"2023-10-06T05:23:50.067352+0000","last_became_peered":"2023-10-06T05:23:50.067352+0000","last_unstale":"2023-10-06T05:23:50.074384+0000","last_undegraded":"2023-10-06T05:23:50.074384+0000","last_fullsized":"2023-10-06T05:23:50.074384+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:49.328707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.086722+0000","last_change":"2023-10-06T05:23:51.086722+0000","last_active":"2023-10-06T05:23:51.086722+0000","last_peered":"2023-10-06T05:23:51.086722+0000","last_clean":"2023-10-06T05:23:51.086722+0000","last_became_active":"2023-10-06T05:23:51.086364+0000","last_became_peered":"2023-10-06T05:23:51.086364+0000","last_unstale":"2023-10-06T05:23:51.086722+0000","last_undegraded":"2023-10-06T05:23:51.086722+0000","last_fullsized":"2023-10-06T05:23:51.086722+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:12:41.375195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.080196+0000","last_change":"2023-10-06T05:23:50.076663+0000","last_active":"2023-10-06T05:23:50.080196+0000","last_peered":"2023-10-06T05:23:50.080196+0000","last_clean":"2023-10-06T05:23:50.080196+0000","last_became_active":"2023-10-06T05:23:50.076397+0000","last_became_peered":"2023-10-06T05:23:50.076397+0000","last_unstale":"2023-10-06T05:23:50.080196+0000","last_undegraded":"2023-10-06T05:23:50.080196+0000","last_fullsized":"2023-10-06T05:23:50.080196+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:51.219335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075473+0000","last_change":"2023-10-06T05:23:51.075473+0000","last_active":"2023-10-06T05:23:51.075473+0000","last_peered":"2023-10-06T05:23:51.075473+0000","last_clean":"2023-10-06T05:23:51.075473+0000","last_became_active":"2023-10-06T05:23:51.074881+0000","last_became_peered":"2023-10-06T05:23:51.074881+0000","last_unstale":"2023-10-06T05:23:51.075473+0000","last_undegraded":"2023-10-06T05:23:51.075473+0000","last_fullsized":"2023-10-06T05:23:51.075473+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:48.949235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.100978+0000","last_change":"2023-10-06T05:23:50.080697+0000","last_active":"2023-10-06T05:23:51.100978+0000","last_peered":"2023-10-06T05:23:51.100978+0000","last_clean":"2023-10-06T05:23:51.100978+0000","last_became_active":"2023-10-06T05:23:50.080428+0000","last_became_peered":"2023-10-06T05:23:50.080428+0000","last_unstale":"2023-10-06T05:23:51.100978+0000","last_undegraded":"2023-10-06T05:23:51.100978+0000","last_fullsized":"2023-10-06T05:23:51.100978+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:50.378199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.059613+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:23:50.059613+0000","last_peered":"2023-10-06T05:23:50.059613+0000","last_clean":"2023-10-06T05:23:50.059613+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:23:50.059613+0000","last_undegraded":"2023-10-06T05:23:50.059613+0000","last_fullsized":"2023-10-06T05:23:50.059613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.063875+0000","last_change":"2023-10-06T05:23:51.063875+0000","last_active":"2023-10-06T05:23:51.063875+0000","last_peered":"2023-10-06T05:23:51.063875+0000","last_clean":"2023-10-06T05:23:51.063875+0000","last_became_active":"2023-10-06T05:23:51.063591+0000","last_became_peered":"2023-10-06T05:23:51.063591+0000","last_unstale":"2023-10-06T05:23:51.063875+0000","last_undegraded":"2023-10-06T05:23:51.063875+0000","last_fullsized":"2023-10-06T05:23:51.063875+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:36.494944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.090343+0000","last_change":"2023-10-06T05:23:50.074371+0000","last_active":"2023-10-06T05:23:51.090343+0000","last_peered":"2023-10-06T05:23:51.090343+0000","last_clean":"2023-10-06T05:23:51.090343+0000","last_became_active":"2023-10-06T05:23:50.074094+0000","last_became_peered":"2023-10-06T05:23:50.074094+0000","last_unstale":"2023-10-06T05:23:51.090343+0000","last_undegraded":"2023-10-06T05:23:51.090343+0000","last_fullsized":"2023-10-06T05:23:51.090343+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:08.957287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.071967+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:23:50.071967+0000","last_peered":"2023-10-06T05:23:50.071967+0000","last_clean":"2023-10-06T05:23:50.071967+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:23:50.071967+0000","last_undegraded":"2023-10-06T05:23:50.071967+0000","last_fullsized":"2023-10-06T05:23:50.071967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.071164+0000","last_change":"2023-10-06T05:23:51.071164+0000","last_active":"2023-10-06T05:23:51.071164+0000","last_peered":"2023-10-06T05:23:51.071164+0000","last_clean":"2023-10-06T05:23:51.071164+0000","last_became_active":"2023-10-06T05:23:51.070896+0000","last_became_peered":"2023-10-06T05:23:51.070896+0000","last_unstale":"2023-10-06T05:23:51.071164+0000","last_undegraded":"2023-10-06T05:23:51.071164+0000","last_fullsized":"2023-10-06T05:23:51.071164+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:44.228394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"128'8","reported_seq":17,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.096701+0000","last_change":"2023-10-06T05:23:50.076026+0000","last_active":"2023-10-06T05:23:51.096701+0000","last_peered":"2023-10-06T05:23:51.096701+0000","last_clean":"2023-10-06T05:23:51.096701+0000","last_became_active":"2023-10-06T05:23:50.074645+0000","last_became_peered":"2023-10-06T05:23:50.074645+0000","last_unstale":"2023-10-06T05:23:51.096701+0000","last_undegraded":"2023-10-06T05:23:51.096701+0000","last_fullsized":"2023-10-06T05:23:51.096701+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:49.912009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.062886+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:23:50.062886+0000","last_peered":"2023-10-06T05:23:50.062886+0000","last_clean":"2023-10-06T05:23:50.062886+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:23:50.062886+0000","last_undegraded":"2023-10-06T05:23:50.062886+0000","last_fullsized":"2023-10-06T05:23:50.062886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080160+0000","last_change":"2023-10-06T05:23:51.080160+0000","last_active":"2023-10-06T05:23:51.080160+0000","last_peered":"2023-10-06T05:23:51.080160+0000","last_clean":"2023-10-06T05:23:51.080160+0000","last_became_active":"2023-10-06T05:23:51.079785+0000","last_became_peered":"2023-10-06T05:23:51.079785+0000","last_unstale":"2023-10-06T05:23:51.080160+0000","last_undegraded":"2023-10-06T05:23:51.080160+0000","last_fullsized":"2023-10-06T05:23:51.080160+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:38.890312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083529+0000","last_change":"2023-10-06T05:23:50.079038+0000","last_active":"2023-10-06T05:23:50.083529+0000","last_peered":"2023-10-06T05:23:50.083529+0000","last_clean":"2023-10-06T05:23:50.083529+0000","last_became_active":"2023-10-06T05:23:50.078702+0000","last_became_peered":"2023-10-06T05:23:50.078702+0000","last_unstale":"2023-10-06T05:23:50.083529+0000","last_undegraded":"2023-10-06T05:23:50.083529+0000","last_fullsized":"2023-10-06T05:23:50.083529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:13.261724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.071907+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:23:50.071907+0000","last_peered":"2023-10-06T05:23:50.071907+0000","last_clean":"2023-10-06T05:23:50.071907+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:23:50.071907+0000","last_undegraded":"2023-10-06T05:23:50.071907+0000","last_fullsized":"2023-10-06T05:23:50.071907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":254,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.068744+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:23:50.068744+0000","last_peered":"2023-10-06T05:23:50.068744+0000","last_clean":"2023-10-06T05:23:50.068744+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:23:50.068744+0000","last_undegraded":"2023-10-06T05:23:50.068744+0000","last_fullsized":"2023-10-06T05:23:50.068744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.085985+0000","last_change":"2023-10-06T05:23:51.085985+0000","last_active":"2023-10-06T05:23:51.085985+0000","last_peered":"2023-10-06T05:23:51.085985+0000","last_clean":"2023-10-06T05:23:51.085985+0000","last_became_active":"2023-10-06T05:23:51.085462+0000","last_became_peered":"2023-10-06T05:23:51.085462+0000","last_unstale":"2023-10-06T05:23:51.085985+0000","last_undegraded":"2023-10-06T05:23:51.085985+0000","last_fullsized":"2023-10-06T05:23:51.085985+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:44.128897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079389+0000","last_change":"2023-10-06T05:23:50.073654+0000","last_active":"2023-10-06T05:23:50.079389+0000","last_peered":"2023-10-06T05:23:50.079389+0000","last_clean":"2023-10-06T05:23:50.079389+0000","last_became_active":"2023-10-06T05:23:50.073048+0000","last_became_peered":"2023-10-06T05:23:50.073048+0000","last_unstale":"2023-10-06T05:23:50.079389+0000","last_undegraded":"2023-10-06T05:23:50.079389+0000","last_fullsized":"2023-10-06T05:23:50.079389+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:34.793646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.068644+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:23:50.068644+0000","last_peered":"2023-10-06T05:23:50.068644+0000","last_clean":"2023-10-06T05:23:50.068644+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:23:50.068644+0000","last_undegraded":"2023-10-06T05:23:50.068644+0000","last_fullsized":"2023-10-06T05:23:50.068644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076425+0000","last_change":"2023-10-06T05:23:51.076425+0000","last_active":"2023-10-06T05:23:51.076425+0000","last_peered":"2023-10-06T05:23:51.076425+0000","last_clean":"2023-10-06T05:23:51.076425+0000","last_became_active":"2023-10-06T05:23:51.076165+0000","last_became_peered":"2023-10-06T05:23:51.076165+0000","last_unstale":"2023-10-06T05:23:51.076425+0000","last_undegraded":"2023-10-06T05:23:51.076425+0000","last_fullsized":"2023-10-06T05:23:51.076425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:47.157000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.096116+0000","last_change":"2023-10-06T05:23:50.078141+0000","last_active":"2023-10-06T05:23:51.096116+0000","last_peered":"2023-10-06T05:23:51.096116+0000","last_clean":"2023-10-06T05:23:51.096116+0000","last_became_active":"2023-10-06T05:23:50.077555+0000","last_became_peered":"2023-10-06T05:23:50.077555+0000","last_unstale":"2023-10-06T05:23:51.096116+0000","last_undegraded":"2023-10-06T05:23:51.096116+0000","last_fullsized":"2023-10-06T05:23:51.096116+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:22.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.070094+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:23:50.070094+0000","last_peered":"2023-10-06T05:23:50.070094+0000","last_clean":"2023-10-06T05:23:50.070094+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:23:50.070094+0000","last_undegraded":"2023-10-06T05:23:50.070094+0000","last_fullsized":"2023-10-06T05:23:50.070094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080364+0000","last_change":"2023-10-06T05:23:51.080364+0000","last_active":"2023-10-06T05:23:51.080364+0000","last_peered":"2023-10-06T05:23:51.080364+0000","last_clean":"2023-10-06T05:23:51.080364+0000","last_became_active":"2023-10-06T05:23:51.080101+0000","last_became_peered":"2023-10-06T05:23:51.080101+0000","last_unstale":"2023-10-06T05:23:51.080364+0000","last_undegraded":"2023-10-06T05:23:51.080364+0000","last_fullsized":"2023-10-06T05:23:51.080364+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.160526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.082696+0000","last_change":"2023-10-06T05:23:50.079386+0000","last_active":"2023-10-06T05:23:50.082696+0000","last_peered":"2023-10-06T05:23:50.082696+0000","last_clean":"2023-10-06T05:23:50.082696+0000","last_became_active":"2023-10-06T05:23:50.079087+0000","last_became_peered":"2023-10-06T05:23:50.079087+0000","last_unstale":"2023-10-06T05:23:50.082696+0000","last_undegraded":"2023-10-06T05:23:50.082696+0000","last_fullsized":"2023-10-06T05:23:50.082696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:09.677549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.059191+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:23:50.059191+0000","last_peered":"2023-10-06T05:23:50.059191+0000","last_clean":"2023-10-06T05:23:50.059191+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:23:50.059191+0000","last_undegraded":"2023-10-06T05:23:50.059191+0000","last_fullsized":"2023-10-06T05:23:50.059191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.068006+0000","last_change":"2023-10-06T05:23:51.068006+0000","last_active":"2023-10-06T05:23:51.068006+0000","last_peered":"2023-10-06T05:23:51.068006+0000","last_clean":"2023-10-06T05:23:51.068006+0000","last_became_active":"2023-10-06T05:23:51.067720+0000","last_became_peered":"2023-10-06T05:23:51.067720+0000","last_unstale":"2023-10-06T05:23:51.068006+0000","last_undegraded":"2023-10-06T05:23:51.068006+0000","last_fullsized":"2023-10-06T05:23:51.068006+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:51.386080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.078146+0000","last_change":"2023-10-06T05:23:50.075215+0000","last_active":"2023-10-06T05:23:50.078146+0000","last_peered":"2023-10-06T05:23:50.078146+0000","last_clean":"2023-10-06T05:23:50.078146+0000","last_became_active":"2023-10-06T05:23:50.074909+0000","last_became_peered":"2023-10-06T05:23:50.074909+0000","last_unstale":"2023-10-06T05:23:50.078146+0000","last_undegraded":"2023-10-06T05:23:50.078146+0000","last_fullsized":"2023-10-06T05:23:50.078146+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:03.224435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.070081+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:23:50.070081+0000","last_peered":"2023-10-06T05:23:50.070081+0000","last_clean":"2023-10-06T05:23:50.070081+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:23:50.070081+0000","last_undegraded":"2023-10-06T05:23:50.070081+0000","last_fullsized":"2023-10-06T05:23:50.070081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303540,"kb_used_data":2156,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068300,"statfs":{"total":96636764160,"available":96325939200,"internally_reserved":0,"allocated":2207744,"data_stored":1325626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70099999999999996,"15min":0.65800000000000003},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.92600000000000005,"5min":1.169,"15min":1.169},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.64700000000000002},"min":{"1min":0.51300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.268,"15min":1.268},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.075,"15min":1.075},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":1.093,"15min":1.093},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.69699999999999995},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93400000000000005,"5min":1.22,"15min":1.22},"last":0.69799999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.079},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46600000000000003},"max":{"1min":0.96199999999999997,"5min":1.117,"15min":1.202},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.623},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":1.258,"15min":1.258},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.73099999999999998},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69999999999999996,"15min":0.69199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.054,"15min":1.054},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.95599999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94499999999999995,"5min":1.139,"15min":1.139},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.74399999999999999},"min":{"1min":0.45100000000000001,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.24,"15min":1.3300000000000001},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":734276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.52000000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":1.0209999999999999,"15min":1.034},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.70499999999999996},"min":{"1min":0.49399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69699999999999995,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.72499999999999998},"min":{"1min":0.54000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.755,"5min":1.05,"15min":1.1599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.90600000000000003,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92000000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.78500000000000003},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.65900000000000003},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.95699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89100000000000001,"5min":0.90500000000000003,"15min":0.998},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.71799999999999997},"min":{"1min":0.434,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.751},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93600000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.754},"min":{"1min":0.501,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.81000000000000005},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302932,"kb_used_data":1548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068908,"statfs":{"total":96636764160,"available":96326561792,"internally_reserved":0,"allocated":1585152,"data_stored":726516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44900000000000001,"15min":0.502},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.438},{"interface":"front","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.54600000000000004},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":2.8050000000000002},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82799999999999996,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.63400000000000001},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.88,"5min":0.88,"15min":2.7639999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":2.8370000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75,"15min":0.78700000000000003},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.1080000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.0660000000000001,"5min":1.25,"15min":1.349},"last":0.877},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":1.016,"15min":1.669},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81000000000000005,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.86399999999999999},"min":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.141,"5min":1.3280000000000001,"15min":1.573},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.528},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.85199999999999998},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":0.97299999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303532,"kb_used_data":2148,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068308,"statfs":{"total":96636764160,"available":96325947392,"internally_reserved":0,"allocated":2199552,"data_stored":1324056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.61299999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.81499999999999995},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67200000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84499999999999997,"5min":0.879,"15min":0.879},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.76300000000000001},"min":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.1120000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.753},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.125,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.59199999999999997,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.218,"15min":1.218},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.089,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.111,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.80600000000000005},"min":{"1min":0.61499999999999999,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0780000000000001,"5min":1.25,"15min":1.25},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":734220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.497},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67000000000000004,"5min":0.77200000000000002,"15min":0.82599999999999996},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.85299999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46999999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75800000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.88100000000000001},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.60199999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97999999999999998,"5min":1.081,"15min":1.1379999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.77400000000000002},"min":{"1min":0.54000000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80200000000000005,"15min":0.80500000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.53000000000000003},"max":{"1min":1.147,"5min":1.161,"15min":1.319},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79400000000000004,"15min":0.81499999999999995},"min":{"1min":0.57599999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.628,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.103,"15min":1.141},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.627,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":1.125,"5min":1.212,"15min":1.282},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81100000000000005,"15min":0.82599999999999996},"min":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.143,"15min":1.3520000000000001},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":733632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46100000000000002,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.64500000000000002,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.495,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.67500000000000004,"5min":0.77100000000000002,"15min":1.772},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.60099999999999998,"5min":0.79400000000000004,"15min":1.913},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.954},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.60899999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.044,"5min":1.0640000000000001,"15min":1.282},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91100000000000003,"5min":1.216,"15min":1.2609999999999999},"last":0.78500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.028,"5min":1.145,"15min":1.1930000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79100000000000004,"15min":0.83299999999999996},"min":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1.0880000000000001,"15min":1.343},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":0.98199999999999998,"5min":1.1779999999999999,"15min":1.3380000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.3640000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":1.085,"5min":1.1539999999999999,"15min":1.214},"last":0.81399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":733632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72799999999999998,"15min":0.71199999999999997},"min":{"1min":0.57499999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98699999999999999,"5min":1.0249999999999999,"15min":1.073},"last":0.438},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.169},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.47199999999999998,"15min":0.432},"max":{"1min":0.93000000000000005,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69999999999999996,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":0.91300000000000003,"5min":1.016,"15min":1.117},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57999999999999996,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":0.96999999999999997,"15min":1.002},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.55300000000000005,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":0.98799999999999999,"15min":1.0780000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.196},"last":0.88}]},{"osd":5,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.433},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.375},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":1.004},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.50900000000000001,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.996,"15min":1.466},"last":1.05},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79400000000000004,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98699999999999999,"5min":1.018,"15min":1.218},"last":0.91400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":733632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.67900000000000005},"min":{"1min":0.57699999999999996,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":1.165,"15min":1.165},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":1.1890000000000001},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83199999999999996,"5min":0.92800000000000005,"15min":0.98299999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.048,"15min":1.048},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.86899999999999999,"5min":1.204,"15min":1.204},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.495},"max":{"1min":0.79600000000000004,"5min":0.997,"15min":1.0549999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.71499999999999997},"min":{"1min":0.55900000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92700000000000005,"5min":1.002,"15min":1.087},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.495,"15min":0.433},"max":{"1min":0.90000000000000002,"5min":1.2769999999999999,"15min":1.278},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.65700000000000003},"min":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.90000000000000002,"5min":1.1539999999999999,"15min":1.294},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":1.141,"15min":1.141},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0429999999999999,"15min":1.254},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.337},"last":0.751},{"interface":"front","average":{"1min":0.75,"5min":0.74299999999999999,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.248,"15min":1.248},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:23:55.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:23:55.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.664+0000 7fc8cf59e700 1 Processor -- start 2023-10-06T05:23:55.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.664+0000 7fc8cf59e700 1 -- start start 2023-10-06T05:23:55.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8cf59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc8d00ccfc0 0x7fc8d00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8cf59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc8d00cf3b0 0x7fc8d01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8cf59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00cbd80 con 0x7fc8d00cc750 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8cf59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00af5f0 con 0x7fc8d00cf3b0 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8cf59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00af8d0 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8cdd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc8d00cf3b0 0x7fc8d01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8ced9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc8d00cc750 0x7fc8d00b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52358/0 (socket says 172.21.15.33:52358) 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.665+0000 7fc8ced9d700 1 -- 172.21.15.33:0/2545676876 learned_addr learned my addr 172.21.15.33:0/2545676876 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8ce59c700 1 --2- 172.21.15.33:0/2545676876 >> v2:172.21.15.167:3300/0 conn(0x7fc8d00ccfc0 0x7fc8d00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/2545676876 >> v2:172.21.15.167:3300/0 conn(0x7fc8d00ccfc0 msgr2=0x7fc8d00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8cdd9b700 1 --2- 172.21.15.33:0/2545676876 >> v2:172.21.15.167:3300/0 conn(0x7fc8d00ccfc0 0x7fc8d00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/2545676876 >> v1:172.21.15.33:6789/0 conn(0x7fc8d00cc750 legacy=0x7fc8d00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/2545676876 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00afbb0 con 0x7fc8d00cf3b0 2023-10-06T05:23:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8cdd9b700 1 --2- 172.21.15.33:0/2545676876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00cf3b0 0x7fc8d01594a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40098e0 tx=0x7fc8c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd00c426069af40b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.666+0000 7fc8cd59a700 1 -- 172.21.15.33:0/2545676876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4011630 con 0x7fc8d00cf3b0 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cd59a700 1 -- 172.21.15.33:0/2545676876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c4011790 con 0x7fc8d00cf3b0 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cd59a700 1 -- 172.21.15.33:0/2545676876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4011b30 con 0x7fc8d00cf3b0 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 -- 172.21.15.33:0/2545676876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00cf3b0 msgr2=0x7fc8d01594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/2545676876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00cf3b0 0x7fc8d01594a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40098e0 tx=0x7fc8c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 -- 172.21.15.33:0/2545676876 shutdown_connections 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/2545676876 >> v2:172.21.15.167:3300/0 conn(0x7fc8d00ccfc0 0x7fc8d00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/2545676876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00cf3b0 0x7fc8d01594a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 -- 172.21.15.33:0/2545676876 >> 172.21.15.33:0/2545676876 conn(0x7fc8d006cff0 msgr2=0x7fc8d006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 -- 172.21.15.33:0/2545676876 shutdown_connections 2023-10-06T05:23:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.667+0000 7fc8cf59e700 1 -- 172.21.15.33:0/2545676876 wait complete. 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.669+0000 7fc8cf59e700 1 Processor -- start 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.669+0000 7fc8cf59e700 1 -- start start 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8d00cc750 0x7fc8d013ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 0x7fc8d016bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cf59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8d016c1c0 0x7fc8d00c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cf59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00b3440 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cf59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d016cf50 con 0x7fc8d016c1c0 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d016c6b0 con 0x7fc8d00cc750 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 0x7fc8d016bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 0x7fc8d016bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56318/0 (socket says 172.21.15.33:56318) 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/1680979563 learned_addr learned my addr 172.21.15.33:0/1680979563 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8ce59c700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8d00cc750 0x7fc8d013ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.670+0000 7fc8ced9d700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8d016c1c0 0x7fc8d00c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8d00cc750 msgr2=0x7fc8d013ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cdd9b700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8d00cc750 0x7fc8d013ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8d016c1c0 msgr2=0x7fc8d00c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cdd9b700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8d016c1c0 0x7fc8d00c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cdd9b700 1 -- 172.21.15.33:0/1680979563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009420 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8ced9d700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8d016c1c0 0x7fc8d00c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cdd9b700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 0x7fc8d016bc80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40091e0 tx=0x7fc8c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c401bb60 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c401bce0 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d016c990 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4025670 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.671+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d016af20 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.672+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d004c380 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.673+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8c4023070 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.675+0000 7fc8be7fc700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc8b0066f30 0x7fc8b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.675+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc8c4085b40 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.676+0000 7fc8ce59c700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc8b0066f30 0x7fc8b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.676+0000 7fc8ce59c700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc8b0066f30 0x7fc8b00693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b1970 tx=0x7fc8c00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.679+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8c4052c70 con 0x7fc8d00ccfc0 2023-10-06T05:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.877+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8d00b3580 con 0x7fc8b0066f30 2023-10-06T05:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.887+0000 7fc8be7fc700 1 -- 172.21.15.33:0/1680979563 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311265 (secure 0 0 0) 0x7fc8d00b3580 con 0x7fc8b0066f30 2023-10-06T05:23:55.963 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc8b0066f30 msgr2=0x7fc8b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc8b0066f30 0x7fc8b00693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b1970 tx=0x7fc8c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 msgr2=0x7fc8d016bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 0x7fc8d016bc80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40091e0 tx=0x7fc8c4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 shutdown_connections 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc8b0066f30 0x7fc8b00693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8d016c1c0 0x7fc8d00c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8d00cc750 0x7fc8d013ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 --2- 172.21.15.33:0/1680979563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d00ccfc0 0x7fc8d016bc80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 >> 172.21.15.33:0/1680979563 conn(0x7fc8d006cff0 msgr2=0x7fc8d00ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 shutdown_connections 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:55.891+0000 7fc8cf59e700 1 -- 172.21.15.33:0/1680979563 wait complete. 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-10-06T05:23:53.958983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424740,"kb_used_data":13668,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549980,"statfs":{"total":773094113280,"available":770611179520,"internally_reserved":0,"allocated":13996032,"data_stored":7045590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.908201"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.064867+0000","last_change":"2023-10-06T05:23:51.064867+0000","last_active":"2023-10-06T05:23:51.064867+0000","last_peered":"2023-10-06T05:23:51.064867+0000","last_clean":"2023-10-06T05:23:51.064867+0000","last_became_active":"2023-10-06T05:23:51.064599+0000","last_became_peered":"2023-10-06T05:23:51.064599+0000","last_unstale":"2023-10-06T05:23:51.064867+0000","last_undegraded":"2023-10-06T05:23:51.064867+0000","last_fullsized":"2023-10-06T05:23:51.064867+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:00.257938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.082904+0000","last_change":"2023-10-06T05:23:51.082904+0000","last_active":"2023-10-06T05:23:51.082904+0000","last_peered":"2023-10-06T05:23:51.082904+0000","last_clean":"2023-10-06T05:23:51.082904+0000","last_became_active":"2023-10-06T05:23:51.082566+0000","last_became_peered":"2023-10-06T05:23:51.082566+0000","last_unstale":"2023-10-06T05:23:51.082904+0000","last_undegraded":"2023-10-06T05:23:51.082904+0000","last_fullsized":"2023-10-06T05:23:51.082904+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:12.272018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075293+0000","last_change":"2023-10-06T05:23:51.075293+0000","last_active":"2023-10-06T05:23:51.075293+0000","last_peered":"2023-10-06T05:23:51.075293+0000","last_clean":"2023-10-06T05:23:51.075293+0000","last_became_active":"2023-10-06T05:23:51.074980+0000","last_became_peered":"2023-10-06T05:23:51.074980+0000","last_unstale":"2023-10-06T05:23:51.075293+0000","last_undegraded":"2023-10-06T05:23:51.075293+0000","last_fullsized":"2023-10-06T05:23:51.075293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:35.066197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.074485+0000","last_change":"2023-10-06T05:23:51.074485+0000","last_active":"2023-10-06T05:23:51.074485+0000","last_peered":"2023-10-06T05:23:51.074485+0000","last_clean":"2023-10-06T05:23:51.074485+0000","last_became_active":"2023-10-06T05:23:51.074169+0000","last_became_peered":"2023-10-06T05:23:51.074169+0000","last_unstale":"2023-10-06T05:23:51.074485+0000","last_undegraded":"2023-10-06T05:23:51.074485+0000","last_fullsized":"2023-10-06T05:23:51.074485+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:51.473460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080677+0000","last_change":"2023-10-06T05:23:51.080677+0000","last_active":"2023-10-06T05:23:51.080677+0000","last_peered":"2023-10-06T05:23:51.080677+0000","last_clean":"2023-10-06T05:23:51.080677+0000","last_became_active":"2023-10-06T05:23:51.080411+0000","last_became_peered":"2023-10-06T05:23:51.080411+0000","last_unstale":"2023-10-06T05:23:51.080677+0000","last_undegraded":"2023-10-06T05:23:51.080677+0000","last_fullsized":"2023-10-06T05:23:51.080677+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:52.175784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.088652+0000","last_change":"2023-10-06T05:23:51.088652+0000","last_active":"2023-10-06T05:23:51.088652+0000","last_peered":"2023-10-06T05:23:51.088652+0000","last_clean":"2023-10-06T05:23:51.088652+0000","last_became_active":"2023-10-06T05:23:51.087719+0000","last_became_peered":"2023-10-06T05:23:51.087719+0000","last_unstale":"2023-10-06T05:23:51.088652+0000","last_undegraded":"2023-10-06T05:23:51.088652+0000","last_fullsized":"2023-10-06T05:23:51.088652+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:23.466596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.085954+0000","last_change":"2023-10-06T05:23:51.085954+0000","last_active":"2023-10-06T05:23:51.085954+0000","last_peered":"2023-10-06T05:23:51.085954+0000","last_clean":"2023-10-06T05:23:51.085954+0000","last_became_active":"2023-10-06T05:23:51.085630+0000","last_became_peered":"2023-10-06T05:23:51.085630+0000","last_unstale":"2023-10-06T05:23:51.085954+0000","last_undegraded":"2023-10-06T05:23:51.085954+0000","last_fullsized":"2023-10-06T05:23:51.085954+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:56.357935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075721+0000","last_change":"2023-10-06T05:23:51.075721+0000","last_active":"2023-10-06T05:23:51.075721+0000","last_peered":"2023-10-06T05:23:51.075721+0000","last_clean":"2023-10-06T05:23:51.075721+0000","last_became_active":"2023-10-06T05:23:51.075424+0000","last_became_peered":"2023-10-06T05:23:51.075424+0000","last_unstale":"2023-10-06T05:23:51.075721+0000","last_undegraded":"2023-10-06T05:23:51.075721+0000","last_fullsized":"2023-10-06T05:23:51.075721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:23.592163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077765+0000","last_change":"2023-10-06T05:23:51.077765+0000","last_active":"2023-10-06T05:23:51.077765+0000","last_peered":"2023-10-06T05:23:51.077765+0000","last_clean":"2023-10-06T05:23:51.077765+0000","last_became_active":"2023-10-06T05:23:51.077480+0000","last_became_peered":"2023-10-06T05:23:51.077480+0000","last_unstale":"2023-10-06T05:23:51.077765+0000","last_undegraded":"2023-10-06T05:23:51.077765+0000","last_fullsized":"2023-10-06T05:23:51.077765+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079325+0000","last_change":"2023-10-06T05:23:51.079325+0000","last_active":"2023-10-06T05:23:51.079325+0000","last_peered":"2023-10-06T05:23:51.079325+0000","last_clean":"2023-10-06T05:23:51.079325+0000","last_became_active":"2023-10-06T05:23:51.079016+0000","last_became_peered":"2023-10-06T05:23:51.079016+0000","last_unstale":"2023-10-06T05:23:51.079325+0000","last_undegraded":"2023-10-06T05:23:51.079325+0000","last_fullsized":"2023-10-06T05:23:51.079325+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:22.620204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.083218+0000","last_change":"2023-10-06T05:23:51.083218+0000","last_active":"2023-10-06T05:23:51.083218+0000","last_peered":"2023-10-06T05:23:51.083218+0000","last_clean":"2023-10-06T05:23:51.083218+0000","last_became_active":"2023-10-06T05:23:51.082952+0000","last_became_peered":"2023-10-06T05:23:51.082952+0000","last_unstale":"2023-10-06T05:23:51.083218+0000","last_undegraded":"2023-10-06T05:23:51.083218+0000","last_fullsized":"2023-10-06T05:23:51.083218+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:46.425293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.067315+0000","last_change":"2023-10-06T05:23:51.067315+0000","last_active":"2023-10-06T05:23:51.067315+0000","last_peered":"2023-10-06T05:23:51.067315+0000","last_clean":"2023-10-06T05:23:51.067315+0000","last_became_active":"2023-10-06T05:23:51.067063+0000","last_became_peered":"2023-10-06T05:23:51.067063+0000","last_unstale":"2023-10-06T05:23:51.067315+0000","last_undegraded":"2023-10-06T05:23:51.067315+0000","last_fullsized":"2023-10-06T05:23:51.067315+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:44.414574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.087872+0000","last_change":"2023-10-06T05:23:51.087872+0000","last_active":"2023-10-06T05:23:51.087872+0000","last_peered":"2023-10-06T05:23:51.087872+0000","last_clean":"2023-10-06T05:23:51.087872+0000","last_became_active":"2023-10-06T05:23:51.087519+0000","last_became_peered":"2023-10-06T05:23:51.087519+0000","last_unstale":"2023-10-06T05:23:51.087872+0000","last_undegraded":"2023-10-06T05:23:51.087872+0000","last_fullsized":"2023-10-06T05:23:51.087872+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:35.419217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077651+0000","last_change":"2023-10-06T05:23:51.077651+0000","last_active":"2023-10-06T05:23:51.077651+0000","last_peered":"2023-10-06T05:23:51.077651+0000","last_clean":"2023-10-06T05:23:51.077651+0000","last_became_active":"2023-10-06T05:23:51.077383+0000","last_became_peered":"2023-10-06T05:23:51.077383+0000","last_unstale":"2023-10-06T05:23:51.077651+0000","last_undegraded":"2023-10-06T05:23:51.077651+0000","last_fullsized":"2023-10-06T05:23:51.077651+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:56.362092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.078292+0000","last_change":"2023-10-06T05:23:51.078292+0000","last_active":"2023-10-06T05:23:51.078292+0000","last_peered":"2023-10-06T05:23:51.078292+0000","last_clean":"2023-10-06T05:23:51.078292+0000","last_became_active":"2023-10-06T05:23:51.078025+0000","last_became_peered":"2023-10-06T05:23:51.078025+0000","last_unstale":"2023-10-06T05:23:51.078292+0000","last_undegraded":"2023-10-06T05:23:51.078292+0000","last_fullsized":"2023-10-06T05:23:51.078292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:31.168313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.066356+0000","last_change":"2023-10-06T05:23:51.066356+0000","last_active":"2023-10-06T05:23:51.066356+0000","last_peered":"2023-10-06T05:23:51.066356+0000","last_clean":"2023-10-06T05:23:51.066356+0000","last_became_active":"2023-10-06T05:23:51.066097+0000","last_became_peered":"2023-10-06T05:23:51.066097+0000","last_unstale":"2023-10-06T05:23:51.066356+0000","last_undegraded":"2023-10-06T05:23:51.066356+0000","last_fullsized":"2023-10-06T05:23:51.066356+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:25.508597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.066171+0000","last_change":"2023-10-06T05:23:51.066171+0000","last_active":"2023-10-06T05:23:51.066171+0000","last_peered":"2023-10-06T05:23:51.066171+0000","last_clean":"2023-10-06T05:23:51.066171+0000","last_became_active":"2023-10-06T05:23:51.065904+0000","last_became_peered":"2023-10-06T05:23:51.065904+0000","last_unstale":"2023-10-06T05:23:51.066171+0000","last_undegraded":"2023-10-06T05:23:51.066171+0000","last_fullsized":"2023-10-06T05:23:51.066171+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:40.750249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.073566+0000","last_change":"2023-10-06T05:23:51.073566+0000","last_active":"2023-10-06T05:23:51.073566+0000","last_peered":"2023-10-06T05:23:51.073566+0000","last_clean":"2023-10-06T05:23:51.073566+0000","last_became_active":"2023-10-06T05:23:51.073283+0000","last_became_peered":"2023-10-06T05:23:51.073283+0000","last_unstale":"2023-10-06T05:23:51.073566+0000","last_undegraded":"2023-10-06T05:23:51.073566+0000","last_fullsized":"2023-10-06T05:23:51.073566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:15.073404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.087657+0000","last_change":"2023-10-06T05:23:51.087657+0000","last_active":"2023-10-06T05:23:51.087657+0000","last_peered":"2023-10-06T05:23:51.087657+0000","last_clean":"2023-10-06T05:23:51.087657+0000","last_became_active":"2023-10-06T05:23:51.087283+0000","last_became_peered":"2023-10-06T05:23:51.087283+0000","last_unstale":"2023-10-06T05:23:51.087657+0000","last_undegraded":"2023-10-06T05:23:51.087657+0000","last_fullsized":"2023-10-06T05:23:51.087657+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:03.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076591+0000","last_change":"2023-10-06T05:23:51.076591+0000","last_active":"2023-10-06T05:23:51.076591+0000","last_peered":"2023-10-06T05:23:51.076591+0000","last_clean":"2023-10-06T05:23:51.076591+0000","last_became_active":"2023-10-06T05:23:51.076247+0000","last_became_peered":"2023-10-06T05:23:51.076247+0000","last_unstale":"2023-10-06T05:23:51.076591+0000","last_undegraded":"2023-10-06T05:23:51.076591+0000","last_fullsized":"2023-10-06T05:23:51.076591+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:53.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077776+0000","last_change":"2023-10-06T05:23:51.077776+0000","last_active":"2023-10-06T05:23:51.077776+0000","last_peered":"2023-10-06T05:23:51.077776+0000","last_clean":"2023-10-06T05:23:51.077776+0000","last_became_active":"2023-10-06T05:23:51.077524+0000","last_became_peered":"2023-10-06T05:23:51.077524+0000","last_unstale":"2023-10-06T05:23:51.077776+0000","last_undegraded":"2023-10-06T05:23:51.077776+0000","last_fullsized":"2023-10-06T05:23:51.077776+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:13.365095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075684+0000","last_change":"2023-10-06T05:23:51.075684+0000","last_active":"2023-10-06T05:23:51.075684+0000","last_peered":"2023-10-06T05:23:51.075684+0000","last_clean":"2023-10-06T05:23:51.075684+0000","last_became_active":"2023-10-06T05:23:51.075411+0000","last_became_peered":"2023-10-06T05:23:51.075411+0000","last_unstale":"2023-10-06T05:23:51.075684+0000","last_undegraded":"2023-10-06T05:23:51.075684+0000","last_fullsized":"2023-10-06T05:23:51.075684+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:31.650696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079748+0000","last_change":"2023-10-06T05:23:51.079748+0000","last_active":"2023-10-06T05:23:51.079748+0000","last_peered":"2023-10-06T05:23:51.079748+0000","last_clean":"2023-10-06T05:23:51.079748+0000","last_became_active":"2023-10-06T05:23:51.078573+0000","last_became_peered":"2023-10-06T05:23:51.078573+0000","last_unstale":"2023-10-06T05:23:51.079748+0000","last_undegraded":"2023-10-06T05:23:51.079748+0000","last_fullsized":"2023-10-06T05:23:51.079748+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:19.355231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075535+0000","last_change":"2023-10-06T05:23:51.075535+0000","last_active":"2023-10-06T05:23:51.075535+0000","last_peered":"2023-10-06T05:23:51.075535+0000","last_clean":"2023-10-06T05:23:51.075535+0000","last_became_active":"2023-10-06T05:23:51.075264+0000","last_became_peered":"2023-10-06T05:23:51.075264+0000","last_unstale":"2023-10-06T05:23:51.075535+0000","last_undegraded":"2023-10-06T05:23:51.075535+0000","last_fullsized":"2023-10-06T05:23:51.075535+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:14.269604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.081084+0000","last_change":"2023-10-06T05:23:51.081084+0000","last_active":"2023-10-06T05:23:51.081084+0000","last_peered":"2023-10-06T05:23:51.081084+0000","last_clean":"2023-10-06T05:23:51.081084+0000","last_became_active":"2023-10-06T05:23:51.080683+0000","last_became_peered":"2023-10-06T05:23:51.080683+0000","last_unstale":"2023-10-06T05:23:51.081084+0000","last_undegraded":"2023-10-06T05:23:51.081084+0000","last_fullsized":"2023-10-06T05:23:51.081084+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:40.852207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080407+0000","last_change":"2023-10-06T05:23:51.080407+0000","last_active":"2023-10-06T05:23:51.080407+0000","last_peered":"2023-10-06T05:23:51.080407+0000","last_clean":"2023-10-06T05:23:51.080407+0000","last_became_active":"2023-10-06T05:23:51.080092+0000","last_became_peered":"2023-10-06T05:23:51.080092+0000","last_unstale":"2023-10-06T05:23:51.080407+0000","last_undegraded":"2023-10-06T05:23:51.080407+0000","last_fullsized":"2023-10-06T05:23:51.080407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:34.807526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080782+0000","last_change":"2023-10-06T05:23:51.080782+0000","last_active":"2023-10-06T05:23:51.080782+0000","last_peered":"2023-10-06T05:23:51.080782+0000","last_clean":"2023-10-06T05:23:51.080782+0000","last_became_active":"2023-10-06T05:23:51.080515+0000","last_became_peered":"2023-10-06T05:23:51.080515+0000","last_unstale":"2023-10-06T05:23:51.080782+0000","last_undegraded":"2023-10-06T05:23:51.080782+0000","last_fullsized":"2023-10-06T05:23:51.080782+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:30.675809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.074824+0000","last_change":"2023-10-06T05:23:51.074824+0000","last_active":"2023-10-06T05:23:51.074824+0000","last_peered":"2023-10-06T05:23:51.074824+0000","last_clean":"2023-10-06T05:23:51.074824+0000","last_became_active":"2023-10-06T05:23:51.074491+0000","last_became_peered":"2023-10-06T05:23:51.074491+0000","last_unstale":"2023-10-06T05:23:51.074824+0000","last_undegraded":"2023-10-06T05:23:51.074824+0000","last_fullsized":"2023-10-06T05:23:51.074824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:22.215131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076563+0000","last_change":"2023-10-06T05:23:51.076563+0000","last_active":"2023-10-06T05:23:51.076563+0000","last_peered":"2023-10-06T05:23:51.076563+0000","last_clean":"2023-10-06T05:23:51.076563+0000","last_became_active":"2023-10-06T05:23:51.075590+0000","last_became_peered":"2023-10-06T05:23:51.075590+0000","last_unstale":"2023-10-06T05:23:51.076563+0000","last_undegraded":"2023-10-06T05:23:51.076563+0000","last_fullsized":"2023-10-06T05:23:51.076563+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:38.572213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075703+0000","last_change":"2023-10-06T05:23:51.075703+0000","last_active":"2023-10-06T05:23:51.075703+0000","last_peered":"2023-10-06T05:23:51.075703+0000","last_clean":"2023-10-06T05:23:51.075703+0000","last_became_active":"2023-10-06T05:23:51.075193+0000","last_became_peered":"2023-10-06T05:23:51.075193+0000","last_unstale":"2023-10-06T05:23:51.075703+0000","last_undegraded":"2023-10-06T05:23:51.075703+0000","last_fullsized":"2023-10-06T05:23:51.075703+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:34.268347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.078954+0000","last_change":"2023-10-06T05:23:51.078954+0000","last_active":"2023-10-06T05:23:51.078954+0000","last_peered":"2023-10-06T05:23:51.078954+0000","last_clean":"2023-10-06T05:23:51.078954+0000","last_became_active":"2023-10-06T05:23:51.078673+0000","last_became_peered":"2023-10-06T05:23:51.078673+0000","last_unstale":"2023-10-06T05:23:51.078954+0000","last_undegraded":"2023-10-06T05:23:51.078954+0000","last_fullsized":"2023-10-06T05:23:51.078954+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.067708+0000","last_change":"2023-10-06T05:23:51.067708+0000","last_active":"2023-10-06T05:23:51.067708+0000","last_peered":"2023-10-06T05:23:51.067708+0000","last_clean":"2023-10-06T05:23:51.067708+0000","last_became_active":"2023-10-06T05:23:51.067368+0000","last_became_peered":"2023-10-06T05:23:51.067368+0000","last_unstale":"2023-10-06T05:23:51.067708+0000","last_undegraded":"2023-10-06T05:23:51.067708+0000","last_fullsized":"2023-10-06T05:23:51.067708+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:45.971146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079361+0000","last_change":"2023-10-06T05:23:51.079361+0000","last_active":"2023-10-06T05:23:51.079361+0000","last_peered":"2023-10-06T05:23:51.079361+0000","last_clean":"2023-10-06T05:23:51.079361+0000","last_became_active":"2023-10-06T05:23:51.079083+0000","last_became_peered":"2023-10-06T05:23:51.079083+0000","last_unstale":"2023-10-06T05:23:51.079361+0000","last_undegraded":"2023-10-06T05:23:51.079361+0000","last_fullsized":"2023-10-06T05:23:51.079361+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:25.508597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.100493+0000","last_change":"2023-10-06T05:23:50.075471+0000","last_active":"2023-10-06T05:23:51.100493+0000","last_peered":"2023-10-06T05:23:51.100493+0000","last_clean":"2023-10-06T05:23:51.100493+0000","last_became_active":"2023-10-06T05:23:50.075089+0000","last_became_peered":"2023-10-06T05:23:50.075089+0000","last_unstale":"2023-10-06T05:23:51.100493+0000","last_undegraded":"2023-10-06T05:23:51.100493+0000","last_fullsized":"2023-10-06T05:23:51.100493+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:59.250058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.074895+0000","last_change":"2023-10-06T05:23:51.074895+0000","last_active":"2023-10-06T05:23:51.074895+0000","last_peered":"2023-10-06T05:23:51.074895+0000","last_clean":"2023-10-06T05:23:51.074895+0000","last_became_active":"2023-10-06T05:23:51.074589+0000","last_became_peered":"2023-10-06T05:23:51.074589+0000","last_unstale":"2023-10-06T05:23:51.074895+0000","last_undegraded":"2023-10-06T05:23:51.074895+0000","last_fullsized":"2023-10-06T05:23:51.074895+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:01.362870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.064714+0000","last_change":"2023-10-06T05:23:50.083940+0000","last_active":"2023-10-06T05:23:51.064714+0000","last_peered":"2023-10-06T05:23:51.064714+0000","last_clean":"2023-10-06T05:23:51.064714+0000","last_became_active":"2023-10-06T05:23:50.083651+0000","last_became_peered":"2023-10-06T05:23:50.083651+0000","last_unstale":"2023-10-06T05:23:51.064714+0000","last_undegraded":"2023-10-06T05:23:51.064714+0000","last_fullsized":"2023-10-06T05:23:51.064714+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:01.198348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077168+0000","last_change":"2023-10-06T05:23:51.077168+0000","last_active":"2023-10-06T05:23:51.077168+0000","last_peered":"2023-10-06T05:23:51.077168+0000","last_clean":"2023-10-06T05:23:51.077168+0000","last_became_active":"2023-10-06T05:23:51.076910+0000","last_became_peered":"2023-10-06T05:23:51.076910+0000","last_unstale":"2023-10-06T05:23:51.077168+0000","last_undegraded":"2023-10-06T05:23:51.077168+0000","last_fullsized":"2023-10-06T05:23:51.077168+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:07.210197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.095445+0000","last_change":"2023-10-06T05:23:50.079705+0000","last_active":"2023-10-06T05:23:51.095445+0000","last_peered":"2023-10-06T05:23:51.095445+0000","last_clean":"2023-10-06T05:23:51.095445+0000","last_became_active":"2023-10-06T05:23:50.079452+0000","last_became_peered":"2023-10-06T05:23:50.079452+0000","last_unstale":"2023-10-06T05:23:51.095445+0000","last_undegraded":"2023-10-06T05:23:51.095445+0000","last_fullsized":"2023-10-06T05:23:51.095445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:24.500717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077039+0000","last_change":"2023-10-06T05:23:51.077039+0000","last_active":"2023-10-06T05:23:51.077039+0000","last_peered":"2023-10-06T05:23:51.077039+0000","last_clean":"2023-10-06T05:23:51.077039+0000","last_became_active":"2023-10-06T05:23:51.076797+0000","last_became_peered":"2023-10-06T05:23:51.076797+0000","last_unstale":"2023-10-06T05:23:51.077039+0000","last_undegraded":"2023-10-06T05:23:51.077039+0000","last_fullsized":"2023-10-06T05:23:51.077039+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:57.105260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.095715+0000","last_change":"2023-10-06T05:23:50.083731+0000","last_active":"2023-10-06T05:23:51.095715+0000","last_peered":"2023-10-06T05:23:51.095715+0000","last_clean":"2023-10-06T05:23:51.095715+0000","last_became_active":"2023-10-06T05:23:50.083461+0000","last_became_peered":"2023-10-06T05:23:50.083461+0000","last_unstale":"2023-10-06T05:23:51.095715+0000","last_undegraded":"2023-10-06T05:23:51.095715+0000","last_fullsized":"2023-10-06T05:23:51.095715+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:15.738326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079616+0000","last_change":"2023-10-06T05:23:51.079616+0000","last_active":"2023-10-06T05:23:51.079616+0000","last_peered":"2023-10-06T05:23:51.079616+0000","last_clean":"2023-10-06T05:23:51.079616+0000","last_became_active":"2023-10-06T05:23:51.079321+0000","last_became_peered":"2023-10-06T05:23:51.079321+0000","last_unstale":"2023-10-06T05:23:51.079616+0000","last_undegraded":"2023-10-06T05:23:51.079616+0000","last_fullsized":"2023-10-06T05:23:51.079616+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:51.386080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.073933+0000","last_change":"2023-10-06T05:23:50.081102+0000","last_active":"2023-10-06T05:23:51.073933+0000","last_peered":"2023-10-06T05:23:51.073933+0000","last_clean":"2023-10-06T05:23:51.073933+0000","last_became_active":"2023-10-06T05:23:50.080748+0000","last_became_peered":"2023-10-06T05:23:50.080748+0000","last_unstale":"2023-10-06T05:23:51.073933+0000","last_undegraded":"2023-10-06T05:23:51.073933+0000","last_fullsized":"2023-10-06T05:23:51.073933+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:10.164174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.087909+0000","last_change":"2023-10-06T05:23:51.087909+0000","last_active":"2023-10-06T05:23:51.087909+0000","last_peered":"2023-10-06T05:23:51.087909+0000","last_clean":"2023-10-06T05:23:51.087909+0000","last_became_active":"2023-10-06T05:23:51.087616+0000","last_became_peered":"2023-10-06T05:23:51.087616+0000","last_unstale":"2023-10-06T05:23:51.087909+0000","last_undegraded":"2023-10-06T05:23:51.087909+0000","last_fullsized":"2023-10-06T05:23:51.087909+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:07.278307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.061552+0000","last_change":"2023-10-06T05:23:50.081063+0000","last_active":"2023-10-06T05:23:51.061552+0000","last_peered":"2023-10-06T05:23:51.061552+0000","last_clean":"2023-10-06T05:23:51.061552+0000","last_became_active":"2023-10-06T05:23:50.080819+0000","last_became_peered":"2023-10-06T05:23:50.080819+0000","last_unstale":"2023-10-06T05:23:51.061552+0000","last_undegraded":"2023-10-06T05:23:51.061552+0000","last_fullsized":"2023-10-06T05:23:51.061552+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:10.164174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.078129+0000","last_change":"2023-10-06T05:23:51.078129+0000","last_active":"2023-10-06T05:23:51.078129+0000","last_peered":"2023-10-06T05:23:51.078129+0000","last_clean":"2023-10-06T05:23:51.078129+0000","last_became_active":"2023-10-06T05:23:51.077885+0000","last_became_peered":"2023-10-06T05:23:51.077885+0000","last_unstale":"2023-10-06T05:23:51.078129+0000","last_undegraded":"2023-10-06T05:23:51.078129+0000","last_fullsized":"2023-10-06T05:23:51.078129+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:13.687528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.082921+0000","last_change":"2023-10-06T05:23:50.077384+0000","last_active":"2023-10-06T05:23:50.082921+0000","last_peered":"2023-10-06T05:23:50.082921+0000","last_clean":"2023-10-06T05:23:50.082921+0000","last_became_active":"2023-10-06T05:23:50.077080+0000","last_became_peered":"2023-10-06T05:23:50.077080+0000","last_unstale":"2023-10-06T05:23:50.082921+0000","last_undegraded":"2023-10-06T05:23:50.082921+0000","last_fullsized":"2023-10-06T05:23:50.082921+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:03.576359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.083004+0000","last_change":"2023-10-06T05:23:51.083004+0000","last_active":"2023-10-06T05:23:51.083004+0000","last_peered":"2023-10-06T05:23:51.083004+0000","last_clean":"2023-10-06T05:23:51.083004+0000","last_became_active":"2023-10-06T05:23:51.082726+0000","last_became_peered":"2023-10-06T05:23:51.082726+0000","last_unstale":"2023-10-06T05:23:51.083004+0000","last_undegraded":"2023-10-06T05:23:51.083004+0000","last_fullsized":"2023-10-06T05:23:51.083004+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:11.377016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079439+0000","last_change":"2023-10-06T05:23:50.072076+0000","last_active":"2023-10-06T05:23:50.079439+0000","last_peered":"2023-10-06T05:23:50.079439+0000","last_clean":"2023-10-06T05:23:50.079439+0000","last_became_active":"2023-10-06T05:23:50.071797+0000","last_became_peered":"2023-10-06T05:23:50.071797+0000","last_unstale":"2023-10-06T05:23:50.079439+0000","last_undegraded":"2023-10-06T05:23:50.079439+0000","last_fullsized":"2023-10-06T05:23:50.079439+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:05.014533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076750+0000","last_change":"2023-10-06T05:23:51.076750+0000","last_active":"2023-10-06T05:23:51.076750+0000","last_peered":"2023-10-06T05:23:51.076750+0000","last_clean":"2023-10-06T05:23:51.076750+0000","last_became_active":"2023-10-06T05:23:51.075611+0000","last_became_peered":"2023-10-06T05:23:51.075611+0000","last_unstale":"2023-10-06T05:23:51.076750+0000","last_undegraded":"2023-10-06T05:23:51.076750+0000","last_fullsized":"2023-10-06T05:23:51.076750+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:14.296341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.068172+0000","last_change":"2023-10-06T05:23:50.076841+0000","last_active":"2023-10-06T05:23:51.068172+0000","last_peered":"2023-10-06T05:23:51.068172+0000","last_clean":"2023-10-06T05:23:51.068172+0000","last_became_active":"2023-10-06T05:23:50.076338+0000","last_became_peered":"2023-10-06T05:23:50.076338+0000","last_unstale":"2023-10-06T05:23:51.068172+0000","last_undegraded":"2023-10-06T05:23:51.068172+0000","last_fullsized":"2023-10-06T05:23:51.068172+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:33.938775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076341+0000","last_change":"2023-10-06T05:23:51.076341+0000","last_active":"2023-10-06T05:23:51.076341+0000","last_peered":"2023-10-06T05:23:51.076341+0000","last_clean":"2023-10-06T05:23:51.076341+0000","last_became_active":"2023-10-06T05:23:51.075961+0000","last_became_peered":"2023-10-06T05:23:51.075961+0000","last_unstale":"2023-10-06T05:23:51.076341+0000","last_undegraded":"2023-10-06T05:23:51.076341+0000","last_fullsized":"2023-10-06T05:23:51.076341+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:27.051904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.095843+0000","last_change":"2023-10-06T05:23:50.076095+0000","last_active":"2023-10-06T05:23:51.095843+0000","last_peered":"2023-10-06T05:23:51.095843+0000","last_clean":"2023-10-06T05:23:51.095843+0000","last_became_active":"2023-10-06T05:23:50.075785+0000","last_became_peered":"2023-10-06T05:23:50.075785+0000","last_unstale":"2023-10-06T05:23:51.095843+0000","last_undegraded":"2023-10-06T05:23:51.095843+0000","last_fullsized":"2023-10-06T05:23:51.095843+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:56.097379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079327+0000","last_change":"2023-10-06T05:23:51.079327+0000","last_active":"2023-10-06T05:23:51.079327+0000","last_peered":"2023-10-06T05:23:51.079327+0000","last_clean":"2023-10-06T05:23:51.079327+0000","last_became_active":"2023-10-06T05:23:51.079073+0000","last_became_peered":"2023-10-06T05:23:51.079073+0000","last_unstale":"2023-10-06T05:23:51.079327+0000","last_undegraded":"2023-10-06T05:23:51.079327+0000","last_fullsized":"2023-10-06T05:23:51.079327+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:25.543598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"128'1","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.060721+0000","last_change":"2023-10-06T05:23:50.076933+0000","last_active":"2023-10-06T05:23:51.060721+0000","last_peered":"2023-10-06T05:23:51.060721+0000","last_clean":"2023-10-06T05:23:51.060721+0000","last_became_active":"2023-10-06T05:23:50.076658+0000","last_became_peered":"2023-10-06T05:23:50.076658+0000","last_unstale":"2023-10-06T05:23:51.060721+0000","last_undegraded":"2023-10-06T05:23:51.060721+0000","last_fullsized":"2023-10-06T05:23:51.060721+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:33.938775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.085864+0000","last_change":"2023-10-06T05:23:51.085864+0000","last_active":"2023-10-06T05:23:51.085864+0000","last_peered":"2023-10-06T05:23:51.085864+0000","last_clean":"2023-10-06T05:23:51.085864+0000","last_became_active":"2023-10-06T05:23:51.085578+0000","last_became_peered":"2023-10-06T05:23:51.085578+0000","last_unstale":"2023-10-06T05:23:51.085864+0000","last_undegraded":"2023-10-06T05:23:51.085864+0000","last_fullsized":"2023-10-06T05:23:51.085864+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:38.621529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.080513+0000","last_change":"2023-10-06T05:23:50.076213+0000","last_active":"2023-10-06T05:23:50.080513+0000","last_peered":"2023-10-06T05:23:50.080513+0000","last_clean":"2023-10-06T05:23:50.080513+0000","last_became_active":"2023-10-06T05:23:50.075892+0000","last_became_peered":"2023-10-06T05:23:50.075892+0000","last_unstale":"2023-10-06T05:23:50.080513+0000","last_undegraded":"2023-10-06T05:23:50.080513+0000","last_fullsized":"2023-10-06T05:23:50.080513+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:12.357215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.068889+0000","last_change":"2023-10-06T05:23:51.068889+0000","last_active":"2023-10-06T05:23:51.068889+0000","last_peered":"2023-10-06T05:23:51.068889+0000","last_clean":"2023-10-06T05:23:51.068889+0000","last_became_active":"2023-10-06T05:23:51.068660+0000","last_became_peered":"2023-10-06T05:23:51.068660+0000","last_unstale":"2023-10-06T05:23:51.068889+0000","last_undegraded":"2023-10-06T05:23:51.068889+0000","last_fullsized":"2023-10-06T05:23:51.068889+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:07.278307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.100733+0000","last_change":"2023-10-06T05:23:50.084943+0000","last_active":"2023-10-06T05:23:51.100733+0000","last_peered":"2023-10-06T05:23:51.100733+0000","last_clean":"2023-10-06T05:23:51.100733+0000","last_became_active":"2023-10-06T05:23:50.084658+0000","last_became_peered":"2023-10-06T05:23:50.084658+0000","last_unstale":"2023-10-06T05:23:51.100733+0000","last_undegraded":"2023-10-06T05:23:51.100733+0000","last_fullsized":"2023-10-06T05:23:51.100733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:06.202316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076463+0000","last_change":"2023-10-06T05:23:51.076463+0000","last_active":"2023-10-06T05:23:51.076463+0000","last_peered":"2023-10-06T05:23:51.076463+0000","last_clean":"2023-10-06T05:23:51.076463+0000","last_became_active":"2023-10-06T05:23:51.076198+0000","last_became_peered":"2023-10-06T05:23:51.076198+0000","last_unstale":"2023-10-06T05:23:51.076463+0000","last_undegraded":"2023-10-06T05:23:51.076463+0000","last_fullsized":"2023-10-06T05:23:51.076463+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:08.568729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083593+0000","last_change":"2023-10-06T05:23:50.081273+0000","last_active":"2023-10-06T05:23:50.083593+0000","last_peered":"2023-10-06T05:23:50.083593+0000","last_clean":"2023-10-06T05:23:50.083593+0000","last_became_active":"2023-10-06T05:23:50.080998+0000","last_became_peered":"2023-10-06T05:23:50.080998+0000","last_unstale":"2023-10-06T05:23:50.083593+0000","last_undegraded":"2023-10-06T05:23:50.083593+0000","last_fullsized":"2023-10-06T05:23:50.083593+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:07.554956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.082242+0000","last_change":"2023-10-06T05:23:51.082242+0000","last_active":"2023-10-06T05:23:51.082242+0000","last_peered":"2023-10-06T05:23:51.082242+0000","last_clean":"2023-10-06T05:23:51.082242+0000","last_became_active":"2023-10-06T05:23:51.081961+0000","last_became_peered":"2023-10-06T05:23:51.081961+0000","last_unstale":"2023-10-06T05:23:51.082242+0000","last_undegraded":"2023-10-06T05:23:51.082242+0000","last_fullsized":"2023-10-06T05:23:51.082242+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:58.865079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083557+0000","last_change":"2023-10-06T05:23:50.077858+0000","last_active":"2023-10-06T05:23:50.083557+0000","last_peered":"2023-10-06T05:23:50.083557+0000","last_clean":"2023-10-06T05:23:50.083557+0000","last_became_active":"2023-10-06T05:23:50.077581+0000","last_became_peered":"2023-10-06T05:23:50.077581+0000","last_unstale":"2023-10-06T05:23:50.083557+0000","last_undegraded":"2023-10-06T05:23:50.083557+0000","last_fullsized":"2023-10-06T05:23:50.083557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:51.219335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.083788+0000","last_change":"2023-10-06T05:23:51.083788+0000","last_active":"2023-10-06T05:23:51.083788+0000","last_peered":"2023-10-06T05:23:51.083788+0000","last_clean":"2023-10-06T05:23:51.083788+0000","last_became_active":"2023-10-06T05:23:51.083549+0000","last_became_peered":"2023-10-06T05:23:51.083549+0000","last_unstale":"2023-10-06T05:23:51.083788+0000","last_undegraded":"2023-10-06T05:23:51.083788+0000","last_fullsized":"2023-10-06T05:23:51.083788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:20.047358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079684+0000","last_change":"2023-10-06T05:23:50.075840+0000","last_active":"2023-10-06T05:23:50.079684+0000","last_peered":"2023-10-06T05:23:50.079684+0000","last_clean":"2023-10-06T05:23:50.079684+0000","last_became_active":"2023-10-06T05:23:50.075571+0000","last_became_peered":"2023-10-06T05:23:50.075571+0000","last_unstale":"2023-10-06T05:23:50.079684+0000","last_undegraded":"2023-10-06T05:23:50.079684+0000","last_fullsized":"2023-10-06T05:23:50.079684+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:19.886288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.077345+0000","last_change":"2023-10-06T05:23:51.077345+0000","last_active":"2023-10-06T05:23:51.077345+0000","last_peered":"2023-10-06T05:23:51.077345+0000","last_clean":"2023-10-06T05:23:51.077345+0000","last_became_active":"2023-10-06T05:23:51.076973+0000","last_became_peered":"2023-10-06T05:23:51.076973+0000","last_unstale":"2023-10-06T05:23:51.077345+0000","last_undegraded":"2023-10-06T05:23:51.077345+0000","last_fullsized":"2023-10-06T05:23:51.077345+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:13.400120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.096315+0000","last_change":"2023-10-06T05:23:50.079113+0000","last_active":"2023-10-06T05:23:51.096315+0000","last_peered":"2023-10-06T05:23:51.096315+0000","last_clean":"2023-10-06T05:23:51.096315+0000","last_became_active":"2023-10-06T05:23:50.078834+0000","last_became_peered":"2023-10-06T05:23:50.078834+0000","last_unstale":"2023-10-06T05:23:51.096315+0000","last_undegraded":"2023-10-06T05:23:51.096315+0000","last_fullsized":"2023-10-06T05:23:51.096315+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:48.332248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079234+0000","last_change":"2023-10-06T05:23:51.079234+0000","last_active":"2023-10-06T05:23:51.079234+0000","last_peered":"2023-10-06T05:23:51.079234+0000","last_clean":"2023-10-06T05:23:51.079234+0000","last_became_active":"2023-10-06T05:23:51.078368+0000","last_became_peered":"2023-10-06T05:23:51.078368+0000","last_unstale":"2023-10-06T05:23:51.079234+0000","last_undegraded":"2023-10-06T05:23:51.079234+0000","last_fullsized":"2023-10-06T05:23:51.079234+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:11.309566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.087223+0000","last_change":"2023-10-06T05:23:50.084353+0000","last_active":"2023-10-06T05:23:50.087223+0000","last_peered":"2023-10-06T05:23:50.087223+0000","last_clean":"2023-10-06T05:23:50.087223+0000","last_became_active":"2023-10-06T05:23:50.084044+0000","last_became_peered":"2023-10-06T05:23:50.084044+0000","last_unstale":"2023-10-06T05:23:50.087223+0000","last_undegraded":"2023-10-06T05:23:50.087223+0000","last_fullsized":"2023-10-06T05:23:50.087223+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:51.219335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.086357+0000","last_change":"2023-10-06T05:23:51.086357+0000","last_active":"2023-10-06T05:23:51.086357+0000","last_peered":"2023-10-06T05:23:51.086357+0000","last_clean":"2023-10-06T05:23:51.086357+0000","last_became_active":"2023-10-06T05:23:51.085974+0000","last_became_peered":"2023-10-06T05:23:51.085974+0000","last_unstale":"2023-10-06T05:23:51.086357+0000","last_undegraded":"2023-10-06T05:23:51.086357+0000","last_fullsized":"2023-10-06T05:23:51.086357+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:26.967879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083783+0000","last_change":"2023-10-06T05:23:50.080671+0000","last_active":"2023-10-06T05:23:50.083783+0000","last_peered":"2023-10-06T05:23:50.083783+0000","last_clean":"2023-10-06T05:23:50.083783+0000","last_became_active":"2023-10-06T05:23:50.080347+0000","last_became_peered":"2023-10-06T05:23:50.080347+0000","last_unstale":"2023-10-06T05:23:50.083783+0000","last_undegraded":"2023-10-06T05:23:50.083783+0000","last_fullsized":"2023-10-06T05:23:50.083783+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:12.357215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.079756+0000","last_change":"2023-10-06T05:23:51.079756+0000","last_active":"2023-10-06T05:23:51.079756+0000","last_peered":"2023-10-06T05:23:51.079756+0000","last_clean":"2023-10-06T05:23:51.079756+0000","last_became_active":"2023-10-06T05:23:51.079524+0000","last_became_peered":"2023-10-06T05:23:51.079524+0000","last_unstale":"2023-10-06T05:23:51.079756+0000","last_undegraded":"2023-10-06T05:23:51.079756+0000","last_fullsized":"2023-10-06T05:23:51.079756+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.190926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079644+0000","last_change":"2023-10-06T05:23:50.073916+0000","last_active":"2023-10-06T05:23:50.079644+0000","last_peered":"2023-10-06T05:23:50.079644+0000","last_clean":"2023-10-06T05:23:50.079644+0000","last_became_active":"2023-10-06T05:23:50.073627+0000","last_became_peered":"2023-10-06T05:23:50.073627+0000","last_unstale":"2023-10-06T05:23:50.079644+0000","last_undegraded":"2023-10-06T05:23:50.079644+0000","last_fullsized":"2023-10-06T05:23:50.079644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:55.310330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.070656+0000","last_change":"2023-10-06T05:23:51.070656+0000","last_active":"2023-10-06T05:23:51.070656+0000","last_peered":"2023-10-06T05:23:51.070656+0000","last_clean":"2023-10-06T05:23:51.070656+0000","last_became_active":"2023-10-06T05:23:51.070406+0000","last_became_peered":"2023-10-06T05:23:51.070406+0000","last_unstale":"2023-10-06T05:23:51.070656+0000","last_undegraded":"2023-10-06T05:23:51.070656+0000","last_fullsized":"2023-10-06T05:23:51.070656+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:52.227215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.082484+0000","last_change":"2023-10-06T05:23:50.078002+0000","last_active":"2023-10-06T05:23:50.082484+0000","last_peered":"2023-10-06T05:23:50.082484+0000","last_clean":"2023-10-06T05:23:50.082484+0000","last_became_active":"2023-10-06T05:23:50.077713+0000","last_became_peered":"2023-10-06T05:23:50.077713+0000","last_unstale":"2023-10-06T05:23:50.082484+0000","last_undegraded":"2023-10-06T05:23:50.082484+0000","last_fullsized":"2023-10-06T05:23:50.082484+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:56.520298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.071541+0000","last_change":"2023-10-06T05:23:51.071541+0000","last_active":"2023-10-06T05:23:51.071541+0000","last_peered":"2023-10-06T05:23:51.071541+0000","last_clean":"2023-10-06T05:23:51.071541+0000","last_became_active":"2023-10-06T05:23:51.071271+0000","last_became_peered":"2023-10-06T05:23:51.071271+0000","last_unstale":"2023-10-06T05:23:51.071541+0000","last_undegraded":"2023-10-06T05:23:51.071541+0000","last_fullsized":"2023-10-06T05:23:51.071541+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:02.206229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.086015+0000","last_change":"2023-10-06T05:23:50.081614+0000","last_active":"2023-10-06T05:23:50.086015+0000","last_peered":"2023-10-06T05:23:50.086015+0000","last_clean":"2023-10-06T05:23:50.086015+0000","last_became_active":"2023-10-06T05:23:50.081260+0000","last_became_peered":"2023-10-06T05:23:50.081260+0000","last_unstale":"2023-10-06T05:23:50.086015+0000","last_undegraded":"2023-10-06T05:23:50.086015+0000","last_fullsized":"2023-10-06T05:23:50.086015+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:31.519158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080754+0000","last_change":"2023-10-06T05:23:51.080754+0000","last_active":"2023-10-06T05:23:51.080754+0000","last_peered":"2023-10-06T05:23:51.080754+0000","last_clean":"2023-10-06T05:23:51.080754+0000","last_became_active":"2023-10-06T05:23:51.080491+0000","last_became_peered":"2023-10-06T05:23:51.080491+0000","last_unstale":"2023-10-06T05:23:51.080754+0000","last_undegraded":"2023-10-06T05:23:51.080754+0000","last_fullsized":"2023-10-06T05:23:51.080754+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:32.527038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.074384+0000","last_change":"2023-10-06T05:23:50.067624+0000","last_active":"2023-10-06T05:23:50.074384+0000","last_peered":"2023-10-06T05:23:50.074384+0000","last_clean":"2023-10-06T05:23:50.074384+0000","last_became_active":"2023-10-06T05:23:50.067352+0000","last_became_peered":"2023-10-06T05:23:50.067352+0000","last_unstale":"2023-10-06T05:23:50.074384+0000","last_undegraded":"2023-10-06T05:23:50.074384+0000","last_fullsized":"2023-10-06T05:23:50.074384+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:49.328707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.086722+0000","last_change":"2023-10-06T05:23:51.086722+0000","last_active":"2023-10-06T05:23:51.086722+0000","last_peered":"2023-10-06T05:23:51.086722+0000","last_clean":"2023-10-06T05:23:51.086722+0000","last_became_active":"2023-10-06T05:23:51.086364+0000","last_became_peered":"2023-10-06T05:23:51.086364+0000","last_unstale":"2023-10-06T05:23:51.086722+0000","last_undegraded":"2023-10-06T05:23:51.086722+0000","last_fullsized":"2023-10-06T05:23:51.086722+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:12:41.375195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.080196+0000","last_change":"2023-10-06T05:23:50.076663+0000","last_active":"2023-10-06T05:23:50.080196+0000","last_peered":"2023-10-06T05:23:50.080196+0000","last_clean":"2023-10-06T05:23:50.080196+0000","last_became_active":"2023-10-06T05:23:50.076397+0000","last_became_peered":"2023-10-06T05:23:50.076397+0000","last_unstale":"2023-10-06T05:23:50.080196+0000","last_undegraded":"2023-10-06T05:23:50.080196+0000","last_fullsized":"2023-10-06T05:23:50.080196+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:51.219335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.075473+0000","last_change":"2023-10-06T05:23:51.075473+0000","last_active":"2023-10-06T05:23:51.075473+0000","last_peered":"2023-10-06T05:23:51.075473+0000","last_clean":"2023-10-06T05:23:51.075473+0000","last_became_active":"2023-10-06T05:23:51.074881+0000","last_became_peered":"2023-10-06T05:23:51.074881+0000","last_unstale":"2023-10-06T05:23:51.075473+0000","last_undegraded":"2023-10-06T05:23:51.075473+0000","last_fullsized":"2023-10-06T05:23:51.075473+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:48.949235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.100978+0000","last_change":"2023-10-06T05:23:50.080697+0000","last_active":"2023-10-06T05:23:51.100978+0000","last_peered":"2023-10-06T05:23:51.100978+0000","last_clean":"2023-10-06T05:23:51.100978+0000","last_became_active":"2023-10-06T05:23:50.080428+0000","last_became_peered":"2023-10-06T05:23:50.080428+0000","last_unstale":"2023-10-06T05:23:51.100978+0000","last_undegraded":"2023-10-06T05:23:51.100978+0000","last_fullsized":"2023-10-06T05:23:51.100978+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:50.378199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.059613+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:23:50.059613+0000","last_peered":"2023-10-06T05:23:50.059613+0000","last_clean":"2023-10-06T05:23:50.059613+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:23:50.059613+0000","last_undegraded":"2023-10-06T05:23:50.059613+0000","last_fullsized":"2023-10-06T05:23:50.059613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.063875+0000","last_change":"2023-10-06T05:23:51.063875+0000","last_active":"2023-10-06T05:23:51.063875+0000","last_peered":"2023-10-06T05:23:51.063875+0000","last_clean":"2023-10-06T05:23:51.063875+0000","last_became_active":"2023-10-06T05:23:51.063591+0000","last_became_peered":"2023-10-06T05:23:51.063591+0000","last_unstale":"2023-10-06T05:23:51.063875+0000","last_undegraded":"2023-10-06T05:23:51.063875+0000","last_fullsized":"2023-10-06T05:23:51.063875+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:36.494944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.090343+0000","last_change":"2023-10-06T05:23:50.074371+0000","last_active":"2023-10-06T05:23:51.090343+0000","last_peered":"2023-10-06T05:23:51.090343+0000","last_clean":"2023-10-06T05:23:51.090343+0000","last_became_active":"2023-10-06T05:23:50.074094+0000","last_became_peered":"2023-10-06T05:23:50.074094+0000","last_unstale":"2023-10-06T05:23:51.090343+0000","last_undegraded":"2023-10-06T05:23:51.090343+0000","last_fullsized":"2023-10-06T05:23:51.090343+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:08.957287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.071967+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:23:50.071967+0000","last_peered":"2023-10-06T05:23:50.071967+0000","last_clean":"2023-10-06T05:23:50.071967+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:23:50.071967+0000","last_undegraded":"2023-10-06T05:23:50.071967+0000","last_fullsized":"2023-10-06T05:23:50.071967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.071164+0000","last_change":"2023-10-06T05:23:51.071164+0000","last_active":"2023-10-06T05:23:51.071164+0000","last_peered":"2023-10-06T05:23:51.071164+0000","last_clean":"2023-10-06T05:23:51.071164+0000","last_became_active":"2023-10-06T05:23:51.070896+0000","last_became_peered":"2023-10-06T05:23:51.070896+0000","last_unstale":"2023-10-06T05:23:51.071164+0000","last_undegraded":"2023-10-06T05:23:51.071164+0000","last_fullsized":"2023-10-06T05:23:51.071164+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:44.228394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"128'8","reported_seq":17,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.096701+0000","last_change":"2023-10-06T05:23:50.076026+0000","last_active":"2023-10-06T05:23:51.096701+0000","last_peered":"2023-10-06T05:23:51.096701+0000","last_clean":"2023-10-06T05:23:51.096701+0000","last_became_active":"2023-10-06T05:23:50.074645+0000","last_became_peered":"2023-10-06T05:23:50.074645+0000","last_unstale":"2023-10-06T05:23:51.096701+0000","last_undegraded":"2023-10-06T05:23:51.096701+0000","last_fullsized":"2023-10-06T05:23:51.096701+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:49.912009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.062886+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:23:50.062886+0000","last_peered":"2023-10-06T05:23:50.062886+0000","last_clean":"2023-10-06T05:23:50.062886+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:23:50.062886+0000","last_undegraded":"2023-10-06T05:23:50.062886+0000","last_fullsized":"2023-10-06T05:23:50.062886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080160+0000","last_change":"2023-10-06T05:23:51.080160+0000","last_active":"2023-10-06T05:23:51.080160+0000","last_peered":"2023-10-06T05:23:51.080160+0000","last_clean":"2023-10-06T05:23:51.080160+0000","last_became_active":"2023-10-06T05:23:51.079785+0000","last_became_peered":"2023-10-06T05:23:51.079785+0000","last_unstale":"2023-10-06T05:23:51.080160+0000","last_undegraded":"2023-10-06T05:23:51.080160+0000","last_fullsized":"2023-10-06T05:23:51.080160+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:38.890312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.083529+0000","last_change":"2023-10-06T05:23:50.079038+0000","last_active":"2023-10-06T05:23:50.083529+0000","last_peered":"2023-10-06T05:23:50.083529+0000","last_clean":"2023-10-06T05:23:50.083529+0000","last_became_active":"2023-10-06T05:23:50.078702+0000","last_became_peered":"2023-10-06T05:23:50.078702+0000","last_unstale":"2023-10-06T05:23:50.083529+0000","last_undegraded":"2023-10-06T05:23:50.083529+0000","last_fullsized":"2023-10-06T05:23:50.083529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:13.261724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":99,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.071907+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:23:50.071907+0000","last_peered":"2023-10-06T05:23:50.071907+0000","last_clean":"2023-10-06T05:23:50.071907+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:23:50.071907+0000","last_undegraded":"2023-10-06T05:23:50.071907+0000","last_fullsized":"2023-10-06T05:23:50.071907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":254,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.068744+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:23:50.068744+0000","last_peered":"2023-10-06T05:23:50.068744+0000","last_clean":"2023-10-06T05:23:50.068744+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:23:50.068744+0000","last_undegraded":"2023-10-06T05:23:50.068744+0000","last_fullsized":"2023-10-06T05:23:50.068744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.085985+0000","last_change":"2023-10-06T05:23:51.085985+0000","last_active":"2023-10-06T05:23:51.085985+0000","last_peered":"2023-10-06T05:23:51.085985+0000","last_clean":"2023-10-06T05:23:51.085985+0000","last_became_active":"2023-10-06T05:23:51.085462+0000","last_became_peered":"2023-10-06T05:23:51.085462+0000","last_unstale":"2023-10-06T05:23:51.085985+0000","last_undegraded":"2023-10-06T05:23:51.085985+0000","last_fullsized":"2023-10-06T05:23:51.085985+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:44.128897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.079389+0000","last_change":"2023-10-06T05:23:50.073654+0000","last_active":"2023-10-06T05:23:50.079389+0000","last_peered":"2023-10-06T05:23:50.079389+0000","last_clean":"2023-10-06T05:23:50.079389+0000","last_became_active":"2023-10-06T05:23:50.073048+0000","last_became_peered":"2023-10-06T05:23:50.073048+0000","last_unstale":"2023-10-06T05:23:50.079389+0000","last_undegraded":"2023-10-06T05:23:50.079389+0000","last_fullsized":"2023-10-06T05:23:50.079389+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:34.793646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.068644+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:23:50.068644+0000","last_peered":"2023-10-06T05:23:50.068644+0000","last_clean":"2023-10-06T05:23:50.068644+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:23:50.068644+0000","last_undegraded":"2023-10-06T05:23:50.068644+0000","last_fullsized":"2023-10-06T05:23:50.068644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.076425+0000","last_change":"2023-10-06T05:23:51.076425+0000","last_active":"2023-10-06T05:23:51.076425+0000","last_peered":"2023-10-06T05:23:51.076425+0000","last_clean":"2023-10-06T05:23:51.076425+0000","last_became_active":"2023-10-06T05:23:51.076165+0000","last_became_peered":"2023-10-06T05:23:51.076165+0000","last_unstale":"2023-10-06T05:23:51.076425+0000","last_undegraded":"2023-10-06T05:23:51.076425+0000","last_fullsized":"2023-10-06T05:23:51.076425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:47.157000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.096116+0000","last_change":"2023-10-06T05:23:50.078141+0000","last_active":"2023-10-06T05:23:51.096116+0000","last_peered":"2023-10-06T05:23:51.096116+0000","last_clean":"2023-10-06T05:23:51.096116+0000","last_became_active":"2023-10-06T05:23:50.077555+0000","last_became_peered":"2023-10-06T05:23:50.077555+0000","last_unstale":"2023-10-06T05:23:51.096116+0000","last_undegraded":"2023-10-06T05:23:51.096116+0000","last_fullsized":"2023-10-06T05:23:51.096116+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:22.458716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.070094+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:23:50.070094+0000","last_peered":"2023-10-06T05:23:50.070094+0000","last_clean":"2023-10-06T05:23:50.070094+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:23:50.070094+0000","last_undegraded":"2023-10-06T05:23:50.070094+0000","last_fullsized":"2023-10-06T05:23:50.070094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.080364+0000","last_change":"2023-10-06T05:23:51.080364+0000","last_active":"2023-10-06T05:23:51.080364+0000","last_peered":"2023-10-06T05:23:51.080364+0000","last_clean":"2023-10-06T05:23:51.080364+0000","last_became_active":"2023-10-06T05:23:51.080101+0000","last_became_peered":"2023-10-06T05:23:51.080101+0000","last_unstale":"2023-10-06T05:23:51.080364+0000","last_undegraded":"2023-10-06T05:23:51.080364+0000","last_fullsized":"2023-10-06T05:23:51.080364+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.160526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.082696+0000","last_change":"2023-10-06T05:23:50.079386+0000","last_active":"2023-10-06T05:23:50.082696+0000","last_peered":"2023-10-06T05:23:50.082696+0000","last_clean":"2023-10-06T05:23:50.082696+0000","last_became_active":"2023-10-06T05:23:50.079087+0000","last_became_peered":"2023-10-06T05:23:50.079087+0000","last_unstale":"2023-10-06T05:23:50.082696+0000","last_undegraded":"2023-10-06T05:23:50.082696+0000","last_fullsized":"2023-10-06T05:23:50.082696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:09.677549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":102,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.059191+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:23:50.059191+0000","last_peered":"2023-10-06T05:23:50.059191+0000","last_clean":"2023-10-06T05:23:50.059191+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:23:50.059191+0000","last_undegraded":"2023-10-06T05:23:50.059191+0000","last_fullsized":"2023-10-06T05:23:50.059191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-06T05:23:51.068006+0000","last_change":"2023-10-06T05:23:51.068006+0000","last_active":"2023-10-06T05:23:51.068006+0000","last_peered":"2023-10-06T05:23:51.068006+0000","last_clean":"2023-10-06T05:23:51.068006+0000","last_became_active":"2023-10-06T05:23:51.067720+0000","last_became_peered":"2023-10-06T05:23:51.067720+0000","last_unstale":"2023-10-06T05:23:51.068006+0000","last_undegraded":"2023-10-06T05:23:51.068006+0000","last_fullsized":"2023-10-06T05:23:51.068006+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:50.049179+0000","last_clean_scrub_stamp":"2023-10-06T05:23:50.049179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:51.386080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.078146+0000","last_change":"2023-10-06T05:23:50.075215+0000","last_active":"2023-10-06T05:23:50.078146+0000","last_peered":"2023-10-06T05:23:50.078146+0000","last_clean":"2023-10-06T05:23:50.078146+0000","last_became_active":"2023-10-06T05:23:50.074909+0000","last_became_peered":"2023-10-06T05:23:50.074909+0000","last_unstale":"2023-10-06T05:23:50.078146+0000","last_undegraded":"2023-10-06T05:23:50.078146+0000","last_fullsized":"2023-10-06T05:23:50.078146+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:49.041298+0000","last_clean_scrub_stamp":"2023-10-06T05:23:49.041298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:03.224435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:23:50.070081+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:23:50.070081+0000","last_peered":"2023-10-06T05:23:50.070081+0000","last_clean":"2023-10-06T05:23:50.070081+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:23:50.070081+0000","last_undegraded":"2023-10-06T05:23:50.070081+0000","last_fullsized":"2023-10-06T05:23:50.070081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303540,"kb_used_data":2156,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068300,"statfs":{"total":96636764160,"available":96325939200,"internally_reserved":0,"allocated":2207744,"data_stored":1325626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70099999999999996,"15min":0.65800000000000003},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.92600000000000005,"5min":1.169,"15min":1.169},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.64700000000000002},"min":{"1min":0.51300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.268,"15min":1.268},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.075,"15min":1.075},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":1.093,"15min":1.093},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.69699999999999995},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93400000000000005,"5min":1.22,"15min":1.22},"last":0.69799999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.079},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46600000000000003},"max":{"1min":0.96199999999999997,"5min":1.117,"15min":1.202},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.623},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":1.258,"15min":1.258},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.73099999999999998},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69999999999999996,"15min":0.69199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.054,"15min":1.054},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.95599999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94499999999999995,"5min":1.139,"15min":1.139},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.74399999999999999},"min":{"1min":0.45100000000000001,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.24,"15min":1.3300000000000001},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":734276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.52000000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":1.0209999999999999,"15min":1.034},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.70499999999999996},"min":{"1min":0.49399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69699999999999995,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.72499999999999998},"min":{"1min":0.54000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.755,"5min":1.05,"15min":1.1599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.90600000000000003,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92000000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.78500000000000003},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.65900000000000003},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.95699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89100000000000001,"5min":0.90500000000000003,"15min":0.998},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.71799999999999997},"min":{"1min":0.434,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.751},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93600000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.754},"min":{"1min":0.501,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.81000000000000005},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302932,"kb_used_data":1548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068908,"statfs":{"total":96636764160,"available":96326561792,"internally_reserved":0,"allocated":1585152,"data_stored":726516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44900000000000001,"15min":0.502},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.438},{"interface":"front","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.54600000000000004},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":2.8050000000000002},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82799999999999996,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.63400000000000001},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.88,"5min":0.88,"15min":2.7639999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":2.8370000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75,"15min":0.78700000000000003},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.1080000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.0660000000000001,"5min":1.25,"15min":1.349},"last":0.877},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":1.016,"15min":1.669},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81000000000000005,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.86399999999999999},"min":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.141,"5min":1.3280000000000001,"15min":1.573},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.528},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.85199999999999998},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":0.97299999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303532,"kb_used_data":2148,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068308,"statfs":{"total":96636764160,"available":96325947392,"internally_reserved":0,"allocated":2199552,"data_stored":1324056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.61299999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.81499999999999995},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67200000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84499999999999997,"5min":0.879,"15min":0.879},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.76300000000000001},"min":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.1120000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.753},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.125,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.59199999999999997,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.218,"15min":1.218},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.089,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.111,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.80600000000000005},"min":{"1min":0.61499999999999999,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0780000000000001,"5min":1.25,"15min":1.25},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":734220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.497},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67000000000000004,"5min":0.77200000000000002,"15min":0.82599999999999996},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.85299999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46999999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75800000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.88100000000000001},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78800000000000003,"15min":0.76900000000000002},"min":{"1min":0.60199999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97999999999999998,"5min":1.081,"15min":1.1379999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.77400000000000002},"min":{"1min":0.54000000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80200000000000005,"15min":0.80500000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.53000000000000003},"max":{"1min":1.147,"5min":1.161,"15min":1.319},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79400000000000004,"15min":0.81499999999999995},"min":{"1min":0.57599999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.628,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.103,"15min":1.141},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.627,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":1.125,"5min":1.212,"15min":1.282},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81100000000000005,"15min":0.82599999999999996},"min":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.143,"15min":1.3520000000000001},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":733632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46100000000000002,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.64500000000000002,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.495,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.67500000000000004,"5min":0.77100000000000002,"15min":1.772},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.60099999999999998,"5min":0.79400000000000004,"15min":1.913},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.954},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.60899999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.044,"5min":1.0640000000000001,"15min":1.282},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91100000000000003,"5min":1.216,"15min":1.2609999999999999},"last":0.78500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.028,"5min":1.145,"15min":1.1930000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79100000000000004,"15min":0.83299999999999996},"min":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1.0880000000000001,"15min":1.343},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":0.98199999999999998,"5min":1.1779999999999999,"15min":1.3380000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.3640000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":1.085,"5min":1.1539999999999999,"15min":1.214},"last":0.81399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":733632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72799999999999998,"15min":0.71199999999999997},"min":{"1min":0.57499999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98699999999999999,"5min":1.0249999999999999,"15min":1.073},"last":0.438},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.169},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.47199999999999998,"15min":0.432},"max":{"1min":0.93000000000000005,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69999999999999996,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":0.91300000000000003,"5min":1.016,"15min":1.117},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57999999999999996,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":0.96999999999999997,"15min":1.002},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.55300000000000005,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":0.98799999999999999,"15min":1.0780000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.196},"last":0.88}]},{"osd":5,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.433},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.375},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":1.004},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.50900000000000001,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.996,"15min":1.466},"last":1.05},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79400000000000004,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98699999999999999,"5min":1.018,"15min":1.218},"last":0.91400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":733632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.67900000000000005},"min":{"1min":0.57699999999999996,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":1.165,"15min":1.165},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":1.1890000000000001},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83199999999999996,"5min":0.92800000000000005,"15min":0.98299999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.048,"15min":1.048},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.86899999999999999,"5min":1.204,"15min":1.204},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.495},"max":{"1min":0.79600000000000004,"5min":0.997,"15min":1.0549999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.71499999999999997},"min":{"1min":0.55900000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92700000000000005,"5min":1.002,"15min":1.087},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.495,"15min":0.433},"max":{"1min":0.90000000000000002,"5min":1.2769999999999999,"15min":1.278},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.65700000000000003},"min":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.90000000000000002,"5min":1.1539999999999999,"15min":1.294},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":1.141,"15min":1.141},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0429999999999999,"15min":1.254},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.337},"last":0.751},{"interface":"front","average":{"1min":0.75,"5min":0.74299999999999999,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.248,"15min":1.248},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:23:56.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:23:56.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:23:56.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.209+0000 7f399a39d700 1 Processor -- start 2023-10-06T05:23:56.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.209+0000 7f399a39d700 1 -- start start 2023-10-06T05:23:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f399a39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39941594c0 0x7f399406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f399a39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399415dbe0 con 0x7f3994159890 2023-10-06T05:23:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f399a39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39940c9e30 con 0x7f39940c95c0 2023-10-06T05:23:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f399a39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940ca110 con 0x7f39941594c0 2023-10-06T05:23:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f3999b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3994159890 0x7f39940cb780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52372/0 (socket says 172.21.15.33:52372) 2023-10-06T05:23:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f3999b9c700 1 -- 172.21.15.33:0/2523911061 learned_addr learned my addr 172.21.15.33:0/2523911061 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f399939b700 1 --2- 172.21.15.33:0/2523911061 >> v2:172.21.15.167:3300/0 conn(0x7f39941594c0 0x7f399406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853051726 0 0) 0x7f399415dbe0 con 0x7f3994159890 2023-10-06T05:23:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.211+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3980003570 con 0x7f3994159890 2023-10-06T05:23:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4134603041 0 0) 0x7f3980003570 con 0x7f3994159890 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 >> v2:172.21.15.167:3300/0 conn(0x7f39941594c0 msgr2=0x7f399406a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 --2- 172.21.15.33:0/2523911061 >> v2:172.21.15.167:3300/0 conn(0x7f39941594c0 0x7f399406a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 >> v1:172.21.15.167:6789/0 conn(0x7f39940c95c0 legacy=0x7f399406a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399415de30 con 0x7f3994159890 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f398c0017f0 con 0x7f3994159890 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 >> v1:172.21.15.33:6789/0 conn(0x7f3994159890 legacy=0x7f39940cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3980003a60 0x7f3980003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3980004810 0x7f3980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39800051c0 0x7f398000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940ca110 con 0x7f39800051c0 2023-10-06T05:23:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398000c160 con 0x7f3980004810 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398000c320 con 0x7f3980003a60 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f3999b9c700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39800051c0 0x7f398000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f3998b9a700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3980004810 0x7f3980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.212+0000 7f399939b700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3980003a60 0x7f3980003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 -- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3980003a60 msgr2=0x7f3980003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3980003a60 0x7f3980003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 -- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3980004810 msgr2=0x7f3980004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3980004810 0x7f3980004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 -- 172.21.15.33:0/2523911061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3980003570 con 0x7f39800051c0 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3998b9a700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3980004810 0x7f3980004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:56.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 -- 172.21.15.33:0/2523911061 reap_dead start 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f3999b9c700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39800051c0 0x7f398000ba60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3994159c90 tx=0x7f398c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c8df692e6612e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0070d0 con 0x7f39800051c0 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c007230 con 0x7f39800051c0 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.213+0000 7f398bfff700 1 -- 172.21.15.33:0/2523911061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0075d0 con 0x7f39800051c0 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 -- 172.21.15.33:0/2523911061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39800051c0 msgr2=0x7f398000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39800051c0 0x7f398000ba60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3994159c90 tx=0x7f398c0021d0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 -- 172.21.15.33:0/2523911061 shutdown_connections 2023-10-06T05:23:56.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 --2- 172.21.15.33:0/2523911061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39800051c0 0x7f398000ba60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 -- 172.21.15.33:0/2523911061 >> 172.21.15.33:0/2523911061 conn(0x7f39940cd3b0 msgr2=0x7f39940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 -- 172.21.15.33:0/2523911061 shutdown_connections 2023-10-06T05:23:56.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.214+0000 7f399a39d700 1 -- 172.21.15.33:0/2523911061 wait complete. 2023-10-06T05:23:56.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.215+0000 7f399a39d700 1 Processor -- start 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.215+0000 7f399a39d700 1 -- start start 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f399a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39800051c0 0x7f399416a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f399a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 0x7f39940aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f399a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399415be10 0x7f399415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f399a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399415c7f0 con 0x7f39940c85b0 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f399a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399415b430 con 0x7f399415be10 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f399a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399415afa0 con 0x7f39800051c0 2023-10-06T05:23:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f3998b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 0x7f39940aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f3998b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 0x7f39940aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56330/0 (socket says 172.21.15.33:56330) 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.216+0000 7f3998b9a700 1 -- 172.21.15.33:0/368716924 learned_addr learned my addr 172.21.15.33:0/368716924 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3998b9a700 1 -- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39800051c0 msgr2=0x7f399416a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3998b9a700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39800051c0 0x7f399416a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3998b9a700 1 -- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399415be10 msgr2=0x7f399415c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3999b9c700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399415be10 0x7f399415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3998b9a700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399415be10 0x7f399415c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3998b9a700 1 -- 172.21.15.33:0/368716924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399416ac10 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3999b9c700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399415be10 0x7f399415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f3998b9a700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 0x7f39940aed70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f398400e7b0 tx=0x7f3984009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984010490 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984010040 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399415a0e0 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.217+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984010840 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.218+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399415a670 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.218+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399404c380 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.219+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3984010bf0 con 0x7f39940c85b0 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.220+0000 7f39897fa700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3978066f30 0x7f39780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.220+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3984083af0 con 0x7f39940c85b0 2023-10-06T05:23:56.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.221+0000 7f399939b700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3978066f30 0x7f39780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.221+0000 7f399939b700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3978066f30 0x7f39780693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3990009170 tx=0x7f39900091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.224+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f398404d5a0 con 0x7f39940c85b0 2023-10-06T05:23:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.506+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f399415b150 con 0x7f39940c85b0 2023-10-06T05:23:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.510+0000 7f39897fa700 1 -- 172.21.15.33:0/368716924 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f399415b150 con 0x7f39940c85b0 2023-10-06T05:23:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3978066f30 msgr2=0x7f39780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3978066f30 0x7f39780693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3990009170 tx=0x7f39900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 msgr2=0x7f39940aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 0x7f39940aed70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f398400e7b0 tx=0x7f3984009040 comp rx=0 tx=0).stop 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 shutdown_connections 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3978066f30 0x7f39780693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399415be10 0x7f399415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39800051c0 0x7f399416a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 --2- 172.21.15.33:0/368716924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39940c85b0 0x7f39940aed70 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 >> 172.21.15.33:0/368716924 conn(0x7f39940cd3b0 msgr2=0x7f3994153580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.512+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 shutdown_connections 2023-10-06T05:23:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.513+0000 7f399a39d700 1 -- 172.21.15.33:0/368716924 wait complete. 2023-10-06T05:23:56.517 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:23:56.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:56.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.716+0000 7fc260b9d700 1 Processor -- start 2023-10-06T05:23:56.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.716+0000 7fc260b9d700 1 -- start start 2023-10-06T05:23:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.717+0000 7fc260b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc25c0ccfc0 0x7fc25c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.717+0000 7fc260b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0cbd80 con 0x7fc25c0cf3b0 2023-10-06T05:23:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.717+0000 7fc260b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0af5f0 con 0x7fc25c0cc750 2023-10-06T05:23:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.717+0000 7fc260b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c0af8d0 con 0x7fc25c0ccfc0 2023-10-06T05:23:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc25c0cf3b0 0x7fc25c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52374/0 (socket says 172.21.15.33:52374) 2023-10-06T05:23:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25affd700 1 -- 172.21.15.33:0/1945654570 learned_addr learned my addr 172.21.15.33:0/1945654570 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25b7fe700 1 --2- 172.21.15.33:0/1945654570 >> v2:172.21.15.167:3300/0 conn(0x7fc25c0ccfc0 0x7fc25c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695093586 0 0) 0x7fc25c0cbd80 con 0x7fc25c0cf3b0 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc244003570 con 0x7fc25c0cf3b0 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1640331580 0 0) 0x7fc244003570 con 0x7fc25c0cf3b0 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 >> v2:172.21.15.167:3300/0 conn(0x7fc25c0ccfc0 msgr2=0x7fc25c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 --2- 172.21.15.33:0/1945654570 >> v2:172.21.15.167:3300/0 conn(0x7fc25c0ccfc0 0x7fc25c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 >> v1:172.21.15.167:6789/0 conn(0x7fc25c0cc750 legacy=0x7fc25c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0afbb0 con 0x7fc25c0cf3b0 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc250001470 con 0x7fc25c0cf3b0 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 >> v1:172.21.15.33:6789/0 conn(0x7fc25c0cf3b0 legacy=0x7fc25c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244003ab0 0x7fc244003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.718+0000 7fc25a7fc700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244004810 0x7fc244004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25a7fc700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc244005210 0x7fc24400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0af8d0 con 0x7fc244005210 2023-10-06T05:23:56.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24400c1b0 con 0x7fc244004810 2023-10-06T05:23:56.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24400c370 con 0x7fc244003ab0 2023-10-06T05:23:56.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc244005210 0x7fc24400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25b7fe700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244003ab0 0x7fc244003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25affd700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244004810 0x7fc244004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 -- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244003ab0 msgr2=0x7fc244003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244003ab0 0x7fc244003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 -- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244004810 msgr2=0x7fc244004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc244004810 0x7fc244004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 -- 172.21.15.33:0/1945654570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc244003570 con 0x7fc244005210 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.719+0000 7fc25bfff700 1 -- 172.21.15.33:0/1945654570 reap_dead start 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc25bfff700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc244005210 0x7fc24400ba60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc254001fc0 tx=0x7fc254001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415d133705bdfc8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254001950 con 0x7fc244005210 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25400c8b0 con 0x7fc244005210 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc25a7fc700 1 -- 172.21.15.33:0/1945654570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400cc90 con 0x7fc244005210 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 -- 172.21.15.33:0/1945654570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc244005210 msgr2=0x7fc24400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc244005210 0x7fc24400ba60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc254001fc0 tx=0x7fc254001640 comp rx=0 tx=0).stop 2023-10-06T05:23:56.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 -- 172.21.15.33:0/1945654570 shutdown_connections 2023-10-06T05:23:56.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1945654570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc244005210 0x7fc24400ba60 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 -- 172.21.15.33:0/1945654570 >> 172.21.15.33:0/1945654570 conn(0x7fc25c06cff0 msgr2=0x7fc25c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 -- 172.21.15.33:0/1945654570 shutdown_connections 2023-10-06T05:23:56.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.720+0000 7fc260b9d700 1 -- 172.21.15.33:0/1945654570 wait complete. 2023-10-06T05:23:56.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.722+0000 7fc260b9d700 1 Processor -- start 2023-10-06T05:23:56.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.722+0000 7fc260b9d700 1 -- start start 2023-10-06T05:23:56.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc260b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244005210 0x7fc25c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc260b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 0x7fc25c16a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc260b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc25c1532f0 0x7fc25c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc260b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c169e00 con 0x7fc25c15ee80 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc260b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0c9ca0 con 0x7fc25c1532f0 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc260b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0c9f80 con 0x7fc244005210 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 0x7fc25c16a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 0x7fc25c16a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56338/0 (socket says 172.21.15.33:56338) 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 -- 172.21.15.33:0/1527842057 learned_addr learned my addr 172.21.15.33:0/1527842057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25b7fe700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244005210 0x7fc25c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25bfff700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc25c1532f0 0x7fc25c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 -- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244005210 msgr2=0x7fc25c153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244005210 0x7fc25c153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 -- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc25c1532f0 msgr2=0x7fc25c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.723+0000 7fc25affd700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc25c1532f0 0x7fc25c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc25affd700 1 -- 172.21.15.33:0/1527842057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0ca260 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc25b7fe700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244005210 0x7fc25c153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc25affd700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 0x7fc25c16a290 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc2500070e0 tx=0x7fc25c0cffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000f5b0 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25000f730 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c169a90 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000faf0 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.724+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c153760 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.725+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c04c380 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.726+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc25001d460 con 0x7fc25c15ee80 2023-10-06T05:23:56.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.727+0000 7fc2437fe700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc248066f30 0x7fc2480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.727+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc2500847c0 con 0x7fc25c15ee80 2023-10-06T05:23:56.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.727+0000 7fc25b7fe700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc248066f30 0x7fc2480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.728+0000 7fc25b7fe700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc248066f30 0x7fc2480693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006f00 tx=0x7fc24c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.731+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc250051830 con 0x7fc25c15ee80 2023-10-06T05:23:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.980+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc25c0af0a0 con 0x7fc25c15ee80 2023-10-06T05:23:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.980+0000 7fc2437fe700 1 -- 172.21.15.33:0/1527842057 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fc25000fea0 con 0x7fc25c15ee80 2023-10-06T05:23:56.982 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:23:56.982 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1831041283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1831041283},{"type":"v1","addr":"172.21.15.33:6834","nonce":1831041283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/606556444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":606556444},{"type":"v1","addr":"172.21.15.167:6836","nonce":606556444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7426,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3370703117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3370703117},{"type":"v1","addr":"172.21.15.167:6837","nonce":3370703117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:23:54.077277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7971":{"gid":7971,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/117155768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":117155768},{"type":"v1","addr":"172.21.15.33:6835","nonce":117155768}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.982+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc248066f30 msgr2=0x7fc2480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.982+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc248066f30 0x7fc2480693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006f00 tx=0x7fc24c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.982+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 msgr2=0x7fc25c16a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.982+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 0x7fc25c16a290 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc2500070e0 tx=0x7fc25c0cffc0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 shutdown_connections 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc248066f30 0x7fc2480693e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc25c1532f0 0x7fc25c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc244005210 0x7fc25c153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 --2- 172.21.15.33:0/1527842057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc25c15ee80 0x7fc25c16a290 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 >> 172.21.15.33:0/1527842057 conn(0x7fc25c06cff0 msgr2=0x7fc25c157180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 shutdown_connections 2023-10-06T05:23:56.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:56.983+0000 7fc260b9d700 1 -- 172.21.15.33:0/1527842057 wait complete. 2023-10-06T05:23:56.986 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 90 2023-10-06T05:23:57.039 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:23:57.038+0000 7f9fbd3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:57.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:23:50.563828+0000', 'modified': '2023-10-06T05:23:54.077277+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7971}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7971': {'gid': 7971, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/117155768', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 117155768}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 117155768}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:23:57.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:23:57.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:23:57.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.598+0000 7f006159d700 1 Processor -- start 2023-10-06T05:23:57.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.598+0000 7f006159d700 1 -- start start 2023-10-06T05:23:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f006159d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f005c0c7070 0x7f005c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f006159d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c06a410 con 0x7f005c155bc0 2023-10-06T05:23:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f006159d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c0cccd0 con 0x7f005c1557f0 2023-10-06T05:23:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f006159d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c0c7500 con 0x7f005c0c7070 2023-10-06T05:23:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f005b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f005c0c7070 0x7f005c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f0060d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f005c155bc0 0x7f005c06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52384/0 (socket says 172.21.15.33:52384) 2023-10-06T05:23:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f0060d9c700 1 -- 172.21.15.33:0/178110686 learned_addr learned my addr 172.21.15.33:0/178110686 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:23:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f005affd700 1 -- 172.21.15.33:0/178110686 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692201943 0 0) 0x7f005c06a410 con 0x7f005c155bc0 2023-10-06T05:23:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f005affd700 1 -- 172.21.15.33:0/178110686 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0044003570 con 0x7f005c155bc0 2023-10-06T05:23:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f005b7fe700 1 -- 172.21.15.33:0/178110686 >> v1:172.21.15.167:6789/0 conn(0x7f005c1557f0 legacy=0x7f005c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.600+0000 7f005b7fe700 1 -- 172.21.15.33:0/178110686 >> v1:172.21.15.33:6789/0 conn(0x7f005c155bc0 legacy=0x7f005c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f005b7fe700 1 -- 172.21.15.33:0/178110686 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0c77e0 con 0x7f005c0c7070 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f005b7fe700 1 --2- 172.21.15.33:0/178110686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c0c8b10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f005000aa90 tx=0x7f005000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f397edef28c31bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f005affd700 1 -- 172.21.15.33:0/178110686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005000c970 con 0x7f005c0c7070 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f005affd700 1 -- 172.21.15.33:0/178110686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005000cad0 con 0x7f005c0c7070 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f005affd700 1 -- 172.21.15.33:0/178110686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005000ce70 con 0x7f005c0c7070 2023-10-06T05:23:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f006159d700 1 -- 172.21.15.33:0/178110686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 msgr2=0x7f005c0c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.601+0000 7f006159d700 1 --2- 172.21.15.33:0/178110686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c0c8b10 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f005000aa90 tx=0x7f005000a330 comp rx=0 tx=0).stop 2023-10-06T05:23:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.602+0000 7f006159d700 1 -- 172.21.15.33:0/178110686 shutdown_connections 2023-10-06T05:23:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.602+0000 7f006159d700 1 --2- 172.21.15.33:0/178110686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c0c8b10 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.602+0000 7f006159d700 1 -- 172.21.15.33:0/178110686 >> 172.21.15.33:0/178110686 conn(0x7f005c06c560 msgr2=0x7f005c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.602+0000 7f006159d700 1 -- 172.21.15.33:0/178110686 shutdown_connections 2023-10-06T05:23:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.602+0000 7f006159d700 1 -- 172.21.15.33:0/178110686 wait complete. 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.603+0000 7f006159d700 1 Processor -- start 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.603+0000 7f006159d700 1 -- start start 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f006159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f006159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f005c1557f0 0x7f005c0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f006159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f005c155bc0 0x7f005c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f006159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c14c160 con 0x7f005c0c7070 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f006159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c14e440 con 0x7f005c1557f0 2023-10-06T05:23:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f006159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c14e720 con 0x7f005c155bc0 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f005bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f005bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c14a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56360/0 (socket says 172.21.15.33:56360) 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.604+0000 7f005bfff700 1 -- 172.21.15.33:0/3132189102 learned_addr learned my addr 172.21.15.33:0/3132189102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005b7fe700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f005c1557f0 0x7f005c0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f0060d9c700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f005c155bc0 0x7f005c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005bfff700 1 -- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f005c155bc0 msgr2=0x7f005c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005bfff700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f005c155bc0 0x7f005c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005bfff700 1 -- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f005c1557f0 msgr2=0x7f005c0af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005bfff700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f005c1557f0 0x7f005c0af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005bfff700 1 -- 172.21.15.33:0/3132189102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005000a5d0 con 0x7f005c0c7070 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f0060d9c700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f005c155bc0 0x7f005c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f005bfff700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c14a5e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f004c00c590 tx=0x7f004c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f0043fff700 1 -- 172.21.15.33:0/3132189102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c011590 con 0x7f005c0c7070 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f0043fff700 1 -- 172.21.15.33:0/3132189102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c011710 con 0x7f005c0c7070 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f0043fff700 1 -- 172.21.15.33:0/3132189102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c011ad0 con 0x7f005c0c7070 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.605+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c14ea00 con 0x7f005c0c7070 2023-10-06T05:23:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.606+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c14ab20 con 0x7f005c0c7070 2023-10-06T05:23:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.608+0000 7f0043fff700 1 -- 172.21.15.33:0/3132189102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f004c01b3b0 con 0x7f005c0c7070 2023-10-06T05:23:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.609+0000 7f0043fff700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0048066f30 0x7f00480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.609+0000 7f0043fff700 1 -- 172.21.15.33:0/3132189102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f004c083c90 con 0x7f005c0c7070 2023-10-06T05:23:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.610+0000 7f005b7fe700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0048066f30 0x7f00480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.611+0000 7f005b7fe700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0048066f30 0x7f00480693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f005000a5a0 tx=0x7f005001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.611+0000 7f0070e48b80 1 Processor -- start 2023-10-06T05:23:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.611+0000 7f0070e48b80 1 -- start start 2023-10-06T05:23:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab478d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab4784990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0070e48b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab4708c80 con 0x55fab47b6270 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab46dfa70 con 0x55fab478cd60 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab46dd270 con 0x55fab47d6fd0 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0060d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab4784990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0060d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab4784990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56364/0 (socket says 172.21.15.33:56364) 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.612+0000 7f0060d9c700 1 -- 172.21.15.33:0/1425423128 learned_addr learned my addr 172.21.15.33:0/1425423128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f005bfff700 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab478d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f005b7fe700 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f0060d9c700 1 -- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 msgr2=0x55fab47b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f0060d9c700 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f0060d9c700 1 -- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 msgr2=0x55fab478d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f0060d9c700 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab478d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f0060d9c700 1 -- 172.21.15.33:0/1425423128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab4573270 con 0x55fab47b6270 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f005bfff700 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab478d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f0060d9c700 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab4784990 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f00540021a0 tx=0x7f0054009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413a0fdf90e93e9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f00417fa700 1 -- 172.21.15.33:0/1425423128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005400ed20 con 0x55fab47b6270 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f00417fa700 1 -- 172.21.15.33:0/1425423128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005400ee80 con 0x55fab47b6270 2023-10-06T05:23:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.613+0000 7f00417fa700 1 -- 172.21.15.33:0/1425423128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054014690 con 0x55fab47b6270 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 -- 172.21.15.33:0/1425423128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 msgr2=0x55fab4784990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab4784990 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f00540021a0 tx=0x7f0054009040 comp rx=0 tx=0).stop 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 -- 172.21.15.33:0/1425423128 shutdown_connections 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab4784990 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab478d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1425423128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 -- 172.21.15.33:0/1425423128 >> 172.21.15.33:0/1425423128 conn(0x55fab478e680 msgr2=0x55fab478ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 -- 172.21.15.33:0/1425423128 shutdown_connections 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.614+0000 7f0070e48b80 1 -- 172.21.15.33:0/1425423128 wait complete. 2023-10-06T05:23:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.615+0000 7f0070e48b80 1 -- start start 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.615+0000 7f0070e48b80 1 Processor -- start 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab47a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab46d9060 con 0x55fab47b6270 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab4684890 con 0x55fab478cd60 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab469fea0 con 0x55fab47d6fd0 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0070e48b80 10 client.0 _ll_register_callbacks cb 0x55fab4575f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f005b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f005b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56376/0 (socket says 172.21.15.33:56376) 2023-10-06T05:23:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f005b7fe700 1 -- 172.21.15.33:0/254832183 learned_addr learned my addr 172.21.15.33:0/254832183 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f0060d9c700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.616+0000 7f005bfff700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab47a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005b7fe700 1 -- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 msgr2=0x55fab47d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005b7fe700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47d4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005b7fe700 1 -- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 msgr2=0x55fab47a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005b7fe700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab47a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005b7fe700 1 -- 172.21.15.33:0/254832183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005400f040 con 0x55fab47b6270 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005bfff700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab47a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f005b7fe700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47b4c50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f005000a1c0 tx=0x7f0050008050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53bf90d73cdf29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f003a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f003a7fc700 1 -- 172.21.15.33:0/254832183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050024710 con 0x55fab47b6270 2023-10-06T05:23:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f003a7fc700 1 -- 172.21.15.33:0/254832183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0050024870 con 0x55fab47b6270 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab475a5c0 con 0x55fab47b6270 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.617+0000 7f003a7fc700 1 -- 172.21.15.33:0/254832183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0050024c60 con 0x55fab47b6270 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 10 client.8115 fetch_fsmap learned FSMap version 90 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55fab4515fe0 con 0x55fab47b6270 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f003a7fc700 1 -- 172.21.15.33:0/254832183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fab475a5c0 con 0x55fab47b6270 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f003a7fc700 1 -- 172.21.15.33:0/254832183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0050030470 con 0x55fab47b6270 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 10 client.8115 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 10 client.8115 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 20 client.8115 populate_metadata read hostname 'smithi033' 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47e6c90 0x55fab47e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 4 client.8115 mds_command: new command op to 7971 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:23:57.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 --> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fab467be00 con 0x55fab47e6c90 2023-10-06T05:23:57.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.618+0000 7f005bfff700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47e6c90 0x55fab47e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.619+0000 7f005bfff700 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47e6c90 0x55fab47e7060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d90ad7c34d75ad server_cookie=82b162ba0cce095e in_seq=0 out_seq=0 2023-10-06T05:23:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.619+0000 7f003a7fc700 10 client.8115 ms_handle_connect on v2:172.21.15.33:6833/117155768 2023-10-06T05:23:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.619+0000 7f003a7fc700 1 -- 172.21.15.33:0/254832183 <== mds.0 v2:172.21.15.33:6833/117155768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55fab467be00 con 0x55fab47e6c90 2023-10-06T05:23:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.619+0000 7f003a7fc700 10 client.8115 handle_command_reply: tid=0 2023-10-06T05:23:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.619+0000 7f0070e48b80 1 client.8115 shutdown 2023-10-06T05:23:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 msgr2=0x55fab47b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47b4c50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f005000a1c0 tx=0x7f0050008050 comp rx=0 tx=0).stop 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 shutdown_connections 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47d6fd0 0x55fab47d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab47a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47e6c90 0x55fab47e7060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 --2- 172.21.15.33:0/254832183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47b4c50 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f003a7fc700 0 client.8115 ms_handle_reset on v2:172.21.15.33:6833/117155768 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 >> 172.21.15.33:0/254832183 conn(0x55fab478e680 msgr2=0x55fab477f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 shutdown_connections 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 1 -- 172.21.15.33:0/254832183 wait complete. 2023-10-06T05:23:57.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.620+0000 7f0070e48b80 20 client.8115 trim_cache size 0 max 16384 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.638+0000 7f0070e48b80 1 Processor -- start 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.638+0000 7f0070e48b80 1 -- start start 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab477f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47b6270 0x55fab477f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 0x55fab47ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0070e48b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab478c050 con 0x55fab47d6fd0 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab455be30 con 0x55fab478cd60 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab47cb5f0 con 0x55fab47b6270 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 0x55fab47ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 0x55fab47ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56392/0 (socket says 172.21.15.33:56392) 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 -- 172.21.15.33:0/1745796792 learned_addr learned my addr 172.21.15.33:0/1745796792 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f005b7fe700 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47b6270 0x55fab477f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f005bfff700 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab477f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 -- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47b6270 msgr2=0x55fab477f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47b6270 0x55fab477f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 -- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 msgr2=0x55fab477f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab477f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.639+0000 7f0060d9c700 1 -- 172.21.15.33:0/1745796792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab46a7780 con 0x55fab47d6fd0 2023-10-06T05:23:57.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0060d9c700 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 0x55fab47ce1a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0054002170 tx=0x7f0054017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6429e8dde6485783 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0041ffb700 1 -- 172.21.15.33:0/1745796792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054001840 con 0x55fab47d6fd0 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0041ffb700 1 -- 172.21.15.33:0/1745796792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0054001d90 con 0x55fab47d6fd0 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0041ffb700 1 -- 172.21.15.33:0/1745796792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00540146b0 con 0x55fab47d6fd0 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 -- 172.21.15.33:0/1745796792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 msgr2=0x55fab47ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 0x55fab47ce1a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0054002170 tx=0x7f0054017000 comp rx=0 tx=0).stop 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 -- 172.21.15.33:0/1745796792 shutdown_connections 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47d6fd0 0x55fab47ce1a0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab478cd60 0x55fab477f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 --2- 172.21.15.33:0/1745796792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab47b6270 0x55fab477f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 -- 172.21.15.33:0/1745796792 >> 172.21.15.33:0/1745796792 conn(0x55fab478e680 msgr2=0x55fab478f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 -- 172.21.15.33:0/1745796792 shutdown_connections 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.640+0000 7f0070e48b80 1 -- 172.21.15.33:0/1745796792 wait complete. 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.641+0000 7f0070e48b80 1 -- start start 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.641+0000 7f0070e48b80 1 Processor -- start 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab478cd60 0x55fab47e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab47d6fd0 0x55fab47d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab47cef30 con 0x55fab47b6270 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab483ca10 con 0x55fab47d6fd0 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab47a5440 con 0x55fab478cd60 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.642+0000 7f0070e48b80 10 client.0 _ll_register_callbacks cb 0x55fab4575f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47e5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56404/0 (socket says 172.21.15.33:56404) 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 -- 172.21.15.33:0/2088607117 learned_addr learned my addr 172.21.15.33:0/2088607117 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005bfff700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab478cd60 0x55fab47e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f0060d9c700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab47d6fd0 0x55fab47d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 -- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab478cd60 msgr2=0x55fab47e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab478cd60 0x55fab47e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 -- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab47d6fd0 msgr2=0x55fab47d7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab47d6fd0 0x55fab47d7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 -- 172.21.15.33:0/2088607117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fab4573270 con 0x55fab47b6270 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f0060d9c700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab47d6fd0 0x55fab47d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f005b7fe700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47e5c90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0050006b90 tx=0x7f0050027070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3b7020f392ae85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.643+0000 7f003a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f003a7fc700 1 -- 172.21.15.33:0/2088607117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050019620 con 0x55fab47b6270 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f003a7fc700 1 -- 172.21.15.33:0/2088607117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0050030470 con 0x55fab47b6270 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab477d690 con 0x55fab47b6270 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f003a7fc700 1 -- 172.21.15.33:0/2088607117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0050041600 con 0x55fab47b6270 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 10 client.8121 fetch_fsmap learned FSMap version 90 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55fab468e470 con 0x55fab47b6270 2023-10-06T05:23:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f003a7fc700 1 -- 172.21.15.33:0/2088607117 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fab477d690 con 0x55fab47b6270 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f003a7fc700 1 -- 172.21.15.33:0/2088607117 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f00500419f0 con 0x55fab47b6270 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 10 client.8121 fetch_fsmap finished waiting for FSMap version 90 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 10 client.8121 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 20 client.8121 populate_metadata read hostname 'smithi033' 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47ee1d0 0x55fab47d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 4 client.8121 mds_command: new command op to 7971 tid=0[{"prefix": "status"}] 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 --> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fab467bb20 con 0x55fab47ee1d0 2023-10-06T05:23:57.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.644+0000 7f005bfff700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47ee1d0 0x55fab47d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f005bfff700 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47ee1d0 0x55fab47d7d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2e97a907162118 server_cookie=feeb18bffda33f1e in_seq=0 out_seq=0 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f003a7fc700 10 client.8121 ms_handle_connect on v2:172.21.15.33:6833/117155768 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f003a7fc700 1 -- 172.21.15.33:0/2088607117 <== mds.0 v2:172.21.15.33:6833/117155768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fab467bb20 con 0x55fab47ee1d0 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f003a7fc700 10 client.8121 handle_command_reply: tid=0 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 client.8121 shutdown 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 msgr2=0x55fab47e5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47e5c90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0050006b90 tx=0x7f0050027070 comp rx=0 tx=0).stop 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 shutdown_connections 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fab47d6fd0 0x55fab47d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55fab478cd60 0x55fab47e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:6833/117155768,v1:172.21.15.33:6835/117155768] conn(0x55fab47ee1d0 0x55fab47d7d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 --2- 172.21.15.33:0/2088607117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fab47b6270 0x55fab47e5c90 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 >> 172.21.15.33:0/2088607117 conn(0x55fab478e680 msgr2=0x55fab477f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.645+0000 7f003a7fc700 0 client.8121 ms_handle_reset on v2:172.21.15.33:6833/117155768 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 shutdown_connections 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f0070e48b80 1 -- 172.21.15.33:0/2088607117 wait complete. 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f0070e48b80 20 client.8121 trim_cache size 0 max 16384 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:23:57.653 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 7971, 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5770797820000002, 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 90, 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 129, 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 129, 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.321781453 2023-10-06T05:23:57.654 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0048066f30 msgr2=0x7f00480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0048066f30 0x7f00480693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f005000a5a0 tx=0x7f005001a000 comp rx=0 tx=0).stop 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 msgr2=0x7f005c14a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c14a5e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f004c00c590 tx=0x7f004c00c8a0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 shutdown_connections 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0048066f30 0x7f00480693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f005c155bc0 0x7f005c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c0c7070 0x7f005c14a5e0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 --2- 172.21.15.33:0/3132189102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f005c1557f0 0x7f005c0af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.646+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 >> 172.21.15.33:0/3132189102 conn(0x7f005c06c560 msgr2=0x7f005c0c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.647+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 shutdown_connections 2023-10-06T05:23:57.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.647+0000 7f006159d700 1 -- 172.21.15.33:0/3132189102 wait complete. 2023-10-06T05:23:57.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.851+0000 7faf6eb9d700 1 Processor -- start 2023-10-06T05:23:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.851+0000 7faf6eb9d700 1 -- start start 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf680c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6eb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf681665b0 0x7faf680cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6eb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf680cca20 con 0x7faf680c69c0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6eb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680cb620 con 0x7faf681665b0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf680ca580 con 0x7faf681661e0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf681665b0 0x7faf680cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf681665b0 0x7faf680cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56420/0 (socket says 172.21.15.33:56420) 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6d39a700 1 -- 172.21.15.33:0/2402973457 learned_addr learned my addr 172.21.15.33:0/2402973457 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.854+0000 7faf6cb99700 1 -- 172.21.15.33:0/2402973457 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209520277 0 0) 0x7faf680cca20 con 0x7faf680c69c0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.853+0000 7faf6db9b700 1 --2- 172.21.15.33:0/2402973457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf680c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.870+0000 7faf6cb99700 1 -- 172.21.15.33:0/2402973457 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf54003550 con 0x7faf680c69c0 2023-10-06T05:23:57.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.870+0000 7faf6d39a700 1 -- 172.21.15.33:0/2402973457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 msgr2=0x7faf680c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.870+0000 7faf6d39a700 1 --2- 172.21.15.33:0/2402973457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf680c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.870+0000 7faf6d39a700 1 -- 172.21.15.33:0/2402973457 >> v1:172.21.15.167:6789/0 conn(0x7faf680c69c0 legacy=0x7faf680c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.870+0000 7faf6d39a700 1 -- 172.21.15.33:0/2402973457 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf680ca860 con 0x7faf681665b0 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.871+0000 7faf6d39a700 1 --2- 172.21.15.33:0/2402973457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681665b0 0x7faf680cc4e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7faf58007460 tx=0x7faf580072c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95d713665f46895 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.871+0000 7faf6cb99700 1 -- 172.21.15.33:0/2402973457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800e070 con 0x7faf681665b0 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.871+0000 7faf6cb99700 1 -- 172.21.15.33:0/2402973457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf580093b0 con 0x7faf681665b0 2023-10-06T05:23:57.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.871+0000 7faf6cb99700 1 -- 172.21.15.33:0/2402973457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf580076c0 con 0x7faf681665b0 2023-10-06T05:23:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.871+0000 7faf6eb9d700 1 -- 172.21.15.33:0/2402973457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681665b0 msgr2=0x7faf680cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.871+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/2402973457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681665b0 0x7faf680cc4e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7faf58007460 tx=0x7faf580072c0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.872+0000 7faf6eb9d700 1 -- 172.21.15.33:0/2402973457 shutdown_connections 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.872+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/2402973457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf680c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.872+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/2402973457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf681665b0 0x7faf680cc4e0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.872+0000 7faf6eb9d700 1 -- 172.21.15.33:0/2402973457 >> 172.21.15.33:0/2402973457 conn(0x7faf6806c560 msgr2=0x7faf6806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.872+0000 7faf6eb9d700 1 -- 172.21.15.33:0/2402973457 shutdown_connections 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.872+0000 7faf6eb9d700 1 -- 172.21.15.33:0/2402973457 wait complete. 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.873+0000 7faf6eb9d700 1 Processor -- start 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.873+0000 7faf6eb9d700 1 -- start start 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.874+0000 7faf6eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 0x7faf68160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.874+0000 7faf6eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf6816b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.874+0000 7faf6eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681665b0 0x7faf680c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.874+0000 7faf6eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680c3190 con 0x7faf680c69c0 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.874+0000 7faf6eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf680afec0 con 0x7faf681665b0 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.874+0000 7faf6eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf680b01a0 con 0x7faf681661e0 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 0x7faf68160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 0x7faf68160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56426/0 (socket says 172.21.15.33:56426) 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 -- 172.21.15.33:0/1484742660 learned_addr learned my addr 172.21.15.33:0/1484742660 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 -- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 msgr2=0x7faf6816b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf6816b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 -- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681665b0 msgr2=0x7faf680c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681665b0 0x7faf680c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 -- 172.21.15.33:0/1484742660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58009590 con 0x7faf680c69c0 2023-10-06T05:23:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6db9b700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 0x7faf68160d50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7faf64012d60 tx=0x7faf6400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400d2e0 con 0x7faf680c69c0 2023-10-06T05:23:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64013040 con 0x7faf680c69c0 2023-10-06T05:23:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64010b70 con 0x7faf680c69c0 2023-10-06T05:23:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.875+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf680b0480 con 0x7faf680c69c0 2023-10-06T05:23:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.876+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68161290 con 0x7faf680c69c0 2023-10-06T05:23:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.877+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6804c380 con 0x7faf680c69c0 2023-10-06T05:23:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.878+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf64018480 con 0x7faf680c69c0 2023-10-06T05:23:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.879+0000 7faf5dffb700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.879+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf640888b0 con 0x7faf680c69c0 2023-10-06T05:23:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.879+0000 7faf6d39a700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.880+0000 7faf6d39a700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faf58007460 tx=0x7faf58017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:57.883+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf64055920 con 0x7faf680c69c0 2023-10-06T05:23:58.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.137+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faf6806db50 con 0x7faf680c69c0 2023-10-06T05:23:58.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.138+0000 7faf5dffb700 1 -- 172.21.15.33:0/1484742660 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7faf64058f40 con 0x7faf680c69c0 2023-10-06T05:23:58.139 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 msgr2=0x7faf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faf58007460 tx=0x7faf58017000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 msgr2=0x7faf68160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 0x7faf68160d50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7faf64012d60 tx=0x7faf6400f040 comp rx=0 tx=0).stop 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 shutdown_connections 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf4c066f30 0x7faf4c0693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681665b0 0x7faf680c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680c69c0 0x7faf68160d50 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 --2- 172.21.15.33:0/1484742660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf681661e0 0x7faf6816b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 >> 172.21.15.33:0/1484742660 conn(0x7faf6806c560 msgr2=0x7faf68162ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 shutdown_connections 2023-10-06T05:23:58.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:58.140+0000 7faf6eb9d700 1 -- 172.21.15.33:0/1484742660 wait complete. 2023-10-06T05:23:58.159 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:58.160 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:58.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.161 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:23:58.198 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:58.198 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:58.198 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:58.198 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:23:58.199 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41719sec preferred_lft 41719sec 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:23:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:58.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.202 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:58.299 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:23:58.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.302 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:23:58.341 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:23:58.341 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:58.341 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:23:58.341 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:23:58.341 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:58.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.342 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:23:58.343 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:23:58.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:23:58.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:23:58.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:23:58.343 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:23:58.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.541 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:23:58.577 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:23:58.577 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:23:58.578 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:58.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:23:58.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:58.680 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:58.717 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:58.717 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:58.717 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:58.793 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:58.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:58.926 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:58.926 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:23:58.926 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:23:58.943 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:58.943 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:23:58.943 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:23:58.943 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:23:58.943 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:23:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:23:58.965 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:23:58.968 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:23:58.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:58.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:59.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:59.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.218+0000 7f73beb9d700 1 Processor -- start 2023-10-06T05:23:59.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.219+0000 7f73beb9d700 1 -- start start 2023-10-06T05:23:59.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.220+0000 7f73beb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 0x7f73b806ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.220+0000 7f73beb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 0x7f73b815a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.220+0000 7f73beb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b815e2d0 con 0x7f73b80b0e10 2023-10-06T05:23:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.220+0000 7f73beb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b815d6c0 con 0x7f73b80b0a40 2023-10-06T05:23:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.220+0000 7f73beb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b815d9a0 con 0x7f73b815a0a0 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73bdb9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 0x7f73b806ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 0x7f73b815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 0x7f73b815a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:53008/0 (socket says 172.21.15.33:53008) 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73bdb9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 0x7f73b806ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:60566/0 (socket says 172.21.15.33:60566) 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 -- 172.21.15.33:0/2203793724 learned_addr learned my addr 172.21.15.33:0/2203793724 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 -- 172.21.15.33:0/2203793724 >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 msgr2=0x7f73b806ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 --2- 172.21.15.33:0/2203793724 >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 0x7f73b806ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 -- 172.21.15.33:0/2203793724 >> v1:172.21.15.167:6789/0 conn(0x7f73b80b0e10 legacy=0x7f73b80b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 -- 172.21.15.33:0/2203793724 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b815dc80 con 0x7f73b815a0a0 2023-10-06T05:23:59.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73bdb9b700 1 --2- 172.21.15.33:0/2203793724 >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 0x7f73b806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:59.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.221+0000 7f73be39c700 1 --2- 172.21.15.33:0/2203793724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 0x7f73b815a520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009f20 tx=0x7f73b0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e95c173794866e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73bcb99700 1 -- 172.21.15.33:0/2203793724 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000e070 con 0x7f73b815a0a0 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73bcb99700 1 -- 172.21.15.33:0/2203793724 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b000b9d0 con 0x7f73b815a0a0 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73bcb99700 1 -- 172.21.15.33:0/2203793724 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b00092e0 con 0x7f73b815a0a0 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73beb9d700 1 -- 172.21.15.33:0/2203793724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 msgr2=0x7f73b815a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73beb9d700 1 --2- 172.21.15.33:0/2203793724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 0x7f73b815a520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009f20 tx=0x7f73b0009840 comp rx=0 tx=0).stop 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73beb9d700 1 -- 172.21.15.33:0/2203793724 shutdown_connections 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73beb9d700 1 --2- 172.21.15.33:0/2203793724 >> v2:172.21.15.167:3300/0 conn(0x7f73b80b0a40 0x7f73b806ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73beb9d700 1 --2- 172.21.15.33:0/2203793724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b815a0a0 0x7f73b815a520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.222+0000 7f73beb9d700 1 -- 172.21.15.33:0/2203793724 >> 172.21.15.33:0/2203793724 conn(0x7f73b806e0f0 msgr2=0x7f73b80675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.223+0000 7f73beb9d700 1 -- 172.21.15.33:0/2203793724 shutdown_connections 2023-10-06T05:23:59.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.223+0000 7f73beb9d700 1 -- 172.21.15.33:0/2203793724 wait complete. 2023-10-06T05:23:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.224+0000 7f73beb9d700 1 Processor -- start 2023-10-06T05:23:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.224+0000 7f73beb9d700 1 -- start start 2023-10-06T05:23:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73beb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b80b0a40 0x7f73b80ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73beb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 0x7f73b815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73beb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73b815a0a0 0x7f73b81549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73beb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b816a0d0 con 0x7f73b80b0e10 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73beb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b816a390 con 0x7f73b815a0a0 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73beb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b816a670 con 0x7f73b80b0a40 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73bd39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 0x7f73b815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73bd39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 0x7f73b815c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56436/0 (socket says 172.21.15.33:56436) 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73bd39a700 1 -- 172.21.15.33:0/3832672008 learned_addr learned my addr 172.21.15.33:0/3832672008 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73be39c700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73b815a0a0 0x7f73b81549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.225+0000 7f73bdb9b700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b80b0a40 0x7f73b80ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73bd39a700 1 -- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b80b0a40 msgr2=0x7f73b80ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73bd39a700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b80b0a40 0x7f73b80ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73bd39a700 1 -- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73b815a0a0 msgr2=0x7f73b81549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73bd39a700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73b815a0a0 0x7f73b81549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73bd39a700 1 -- 172.21.15.33:0/3832672008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b4009420 con 0x7f73b80b0e10 2023-10-06T05:23:59.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73bd39a700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 0x7f73b815c810 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c590 tx=0x7f73a800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8011590 con 0x7f73b80b0e10 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8011710 con 0x7f73b80b0e10 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8011ad0 con 0x7f73b80b0e10 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b0009ae0 con 0x7f73b80b0e10 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.226+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b816ac30 con 0x7f73b80b0e10 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.227+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b80acc40 con 0x7f73b80b0e10 2023-10-06T05:23:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.228+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73a801b3b0 con 0x7f73b80b0e10 2023-10-06T05:23:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.229+0000 7f73adffb700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f739c066f30 0x7f739c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.229+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f73a8083b40 con 0x7f73b80b0e10 2023-10-06T05:23:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.229+0000 7f73bdb9b700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f739c066f30 0x7f739c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.230+0000 7f73bdb9b700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f739c066f30 0x7f739c0693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009f40 tx=0x7f73b40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.233+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73b80acc40 con 0x7f73b80b0e10 2023-10-06T05:23:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.481+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f73b80acc40 con 0x7f73b80b0e10 2023-10-06T05:23:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.481+0000 7f73adffb700 1 -- 172.21.15.33:0/3832672008 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f73b80acc40 con 0x7f73b80b0e10 2023-10-06T05:23:59.483 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f739c066f30 msgr2=0x7f739c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f739c066f30 0x7f739c0693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009f40 tx=0x7f73b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 msgr2=0x7f73b815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 0x7f73b815c810 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c590 tx=0x7f73a800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 shutdown_connections 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f739c066f30 0x7f739c0693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73b815a0a0 0x7f73b81549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73b80b0a40 0x7f73b80ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 --2- 172.21.15.33:0/3832672008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80b0e10 0x7f73b815c810 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 >> 172.21.15.33:0/3832672008 conn(0x7f73b806e0f0 msgr2=0x7f73b8158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 shutdown_connections 2023-10-06T05:23:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:23:59.484+0000 7f73beb9d700 1 -- 172.21.15.33:0/3832672008 wait complete. 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:23:59.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:23:59.504 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:59.504 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:59.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:59.504 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:23:59.545 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:59.545 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41721sec preferred_lft 41721sec 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:59.547 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:59.548 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:59.549 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:59.648 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:23:59.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:59.651 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:23:59.691 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:23:59.691 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:59.691 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:23:59.691 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:23:59.691 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:59.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:59.692 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:23:59.692 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:23:59.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:23:59.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:23:59.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:23:59.692 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:23:59.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:59.887 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:23:59.923 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:23:59.923 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:23:59.923 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:59.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:59.924 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:24:00.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:00.025 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:00.061 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:00.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:00.062 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:00.138 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:24:00.138 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:00.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:00.269 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:00.269 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:24:00.269 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:24:00.285 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:00.286 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:24:00.286 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:24:00.286 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:00.286 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:00.307 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:00.307 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:24:00.308 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:24:00.308 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:00.308 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:00.308 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:24:00.308 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:24:00.308 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:24:00.311 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:00.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:00.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:00.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:24:00.592 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:24:00.592 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:24:00.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:24:00.797 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:24:00.798 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:24:00.798 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q fuse --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:24:00.844 INFO:teuthology.orchestra.run.smithi033.stdout:2.9.7-12.el8 2023-10-06T05:24:00.845 INFO:teuthology.packaging:The installed version of fuse is 2.9.7-12.el8 2023-10-06T05:24:00.845 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q fuse --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:24:00.892 INFO:teuthology.orchestra.run.smithi167.stdout:2.9.7-12.el8 2023-10-06T05:24:00.892 INFO:teuthology.packaging:The installed version of fuse is 2.9.7-12.el8 2023-10-06T05:24:00.893 INFO:tasks.cephfs.test_client_recovery:flock locks are available 2023-10-06T05:24:00.893 INFO:tasks.cephfs.mount:lock_background file background_file 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> import fcntl 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> import struct 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> f1 = open("/home/ubuntu/cephtest/mnt.0/background_file-1", '"'"'w'"'"') 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> fcntl.flock(f1, fcntl.LOCK_EX | fcntl.LOCK_NB) 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> f2 = open("/home/ubuntu/cephtest/mnt.0/background_file-2", '"'"'w'"'"') 2023-10-06T05:24:00.893 DEBUG:teuthology.orchestra.run.smithi033:> lockdata = struct.pack('"'"'hhllhh'"'"', fcntl.F_WRLCK, 0, 0, 0, 0, 0) 2023-10-06T05:24:00.894 DEBUG:teuthology.orchestra.run.smithi033:> fcntl.fcntl(f2, fcntl.F_SETLK, lockdata) 2023-10-06T05:24:00.894 DEBUG:teuthology.orchestra.run.smithi033:> while True: 2023-10-06T05:24:00.894 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(1) 2023-10-06T05:24:00.894 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:24:00.898 DEBUG:teuthology.orchestra.run.smithi167:> stat /home/ubuntu/cephtest/mnt.1/background_file-2 2023-10-06T05:24:00.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:00.936 INFO:teuthology.orchestra.run.smithi167.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.1/background_file-2': No such file or directory 2023-10-06T05:24:01.936 DEBUG:teuthology.orchestra.run.smithi167:> stat /home/ubuntu/cephtest/mnt.1/background_file-2 2023-10-06T05:24:01.986 INFO:teuthology.orchestra.run.smithi167.stdout: File: /home/ubuntu/cephtest/mnt.1/background_file-2 2023-10-06T05:24:01.986 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 0 Blocks: 0 IO Block: 4194304 regular empty file 2023-10-06T05:24:01.986 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 2bh/43d Inode: 1099511627777 Links: 1 2023-10-06T05:24:01.986 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0664/-rw-rw-r--) Uid: ( 1000/ ubuntu) Gid: ( 1275/ ubuntu) 2023-10-06T05:24:01.986 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-06T05:24:01.986 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:24:01.003031064 +0000 2023-10-06T05:24:01.987 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:24:01.003031064 +0000 2023-10-06T05:24:01.987 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:24:01.003031064 +0000 2023-10-06T05:24:01.987 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:24:01.987 INFO:tasks.cephfs.mount:File background_file-2 became visible from 1 after 1s 2023-10-06T05:24:01.987 INFO:tasks.cephfs.mount:check lock on file background_file 2023-10-06T05:24:01.987 DEBUG:teuthology.orchestra.run.smithi167:> python3 -c ' 2023-10-06T05:24:01.987 DEBUG:teuthology.orchestra.run.smithi167:> import fcntl 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> import errno 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> import struct 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> f1 = open("/home/ubuntu/cephtest/mnt.1/background_file-1", '"'"'r'"'"') 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.flock(f1, fcntl.LOCK_EX | fcntl.LOCK_NB) 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> if e.errno == errno.EAGAIN: 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> else: 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("flock on file /home/ubuntu/cephtest/mnt.1/background_file-1 not found") 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> f2 = open("/home/ubuntu/cephtest/mnt.1/background_file-2", '"'"'r'"'"') 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:24:01.988 DEBUG:teuthology.orchestra.run.smithi167:> lockdata = struct.pack('"'"'hhllhh'"'"', fcntl.F_WRLCK, 0, 0, 0, 0, 0) 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.fcntl(f2, fcntl.F_SETLK, lockdata) 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> if e.errno == errno.EAGAIN: 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> else: 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("posix lock on file /home/ubuntu/cephtest/mnt.1/background_file-2 not found") 2023-10-06T05:24:01.989 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:24:02.061 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:24:02.062 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:02.062 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:24:02.062 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:02.062 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:24:02.062 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:02.063 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:24:02.063 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:02.075 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:24:02.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2023-10-06T05:24:02.076 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:24:02.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2023-10-06T05:24:02.078 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:24:02.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2023-10-06T05:24:02.090 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:24:02.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2023-10-06T05:24:02.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.332+0000 7f5dc759e700 1 Processor -- start 2023-10-06T05:24:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.332+0000 7f5dc759e700 1 -- start start 2023-10-06T05:24:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7fa31d79d700 1 Processor -- start 2023-10-06T05:24:02.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7fa31d79d700 1 -- start start 2023-10-06T05:24:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7f5dc759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5dc8159400 0x7f5dc80cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7f5dc759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5dc806cc70 0x7f5dc80cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7f5dc759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc80d0ed0 con 0x7f5dc81597d0 2023-10-06T05:24:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7f5dc759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc80b1160 con 0x7f5dc8159400 2023-10-06T05:24:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.333+0000 7f5dc759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc80cb6d0 con 0x7f5dc806cc70 2023-10-06T05:24:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.334+0000 7f5dc659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5dc8159400 0x7f5dc80cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.334+0000 7f5dc659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5dc8159400 0x7f5dc80cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51046/0 (socket says 172.21.15.33:51046) 2023-10-06T05:24:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.334+0000 7f5dc659c700 1 -- 172.21.15.33:0/4013488991 learned_addr learned my addr 172.21.15.33:0/4013488991 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.334+0000 7f5dc6d9d700 1 --2- 172.21.15.33:0/4013488991 >> v2:172.21.15.167:3300/0 conn(0x7f5dc806cc70 0x7f5dc80cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.334+0000 7fa31d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 0x7fa3180cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc659c700 1 -- 172.21.15.33:0/4013488991 >> v2:172.21.15.167:3300/0 conn(0x7f5dc806cc70 msgr2=0x7f5dc80cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.334+0000 7fa31d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 0x7fa3180cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc659c700 1 --2- 172.21.15.33:0/4013488991 >> v2:172.21.15.167:3300/0 conn(0x7f5dc806cc70 0x7f5dc80cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc659c700 1 -- 172.21.15.33:0/4013488991 >> v1:172.21.15.167:6789/0 conn(0x7f5dc81597d0 legacy=0x7f5dc806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc659c700 1 -- 172.21.15.33:0/4013488991 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80cb9b0 con 0x7f5dc8159400 2023-10-06T05:24:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa3177fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 0x7fa3180cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa317fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 0x7fa3180cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc659c700 1 --2- 172.21.15.33:0/4013488991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8159400 0x7f5dc80cab60 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80cfca0 tx=0x7f5db8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee823f97984dd0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa317fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 0x7fa3180cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35636/0 (socket says 172.21.15.33:35636) 2023-10-06T05:24:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa3177fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 0x7fa3180cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51052/0 (socket says 172.21.15.33:51052) 2023-10-06T05:24:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa317fff700 1 -- 172.21.15.33:0/709999631 learned_addr learned my addr 172.21.15.33:0/709999631 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3180d0ed0 con 0x7fa3180cc770 2023-10-06T05:24:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3180b1160 con 0x7fa3181597d0 2023-10-06T05:24:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180cb6d0 con 0x7fa318159400 2023-10-06T05:24:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc559a700 1 -- 172.21.15.33:0/4013488991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8009e70 con 0x7f5dc8159400 2023-10-06T05:24:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc559a700 1 -- 172.21.15.33:0/4013488991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db800b860 con 0x7f5dc8159400 2023-10-06T05:24:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.335+0000 7f5dc559a700 1 -- 172.21.15.33:0/4013488991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800bc00 con 0x7f5dc8159400 2023-10-06T05:24:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa317fff700 1 -- 172.21.15.33:0/709999631 >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 msgr2=0x7fa3180cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa317fff700 1 --2- 172.21.15.33:0/709999631 >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 0x7fa3180cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa317fff700 1 -- 172.21.15.33:0/709999631 >> v1:172.21.15.167:6789/0 conn(0x7fa3180cc770 legacy=0x7fa31806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7f5dc759e700 1 -- 172.21.15.33:0/4013488991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8159400 msgr2=0x7f5dc80cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7f5dc759e700 1 --2- 172.21.15.33:0/4013488991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8159400 0x7f5dc80cab60 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80cfca0 tx=0x7f5db8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa316ffd700 1 -- 172.21.15.33:0/709999631 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181337742 0 0) 0x7fa3180d0ed0 con 0x7fa3180cc770 2023-10-06T05:24:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa317fff700 1 -- 172.21.15.33:0/709999631 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180cb9b0 con 0x7fa318159400 2023-10-06T05:24:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa3177fe700 1 --2- 172.21.15.33:0/709999631 >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 0x7fa3180cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7f5dc759e700 1 -- 172.21.15.33:0/4013488991 shutdown_connections 2023-10-06T05:24:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7f5dc759e700 1 --2- 172.21.15.33:0/4013488991 >> v2:172.21.15.167:3300/0 conn(0x7f5dc806cc70 0x7f5dc80cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.336+0000 7fa317fff700 1 --2- 172.21.15.33:0/709999631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 0x7fa3180cab60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa3080099b0 tx=0x7fa308009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56daba8b08eb69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa316ffd700 1 -- 172.21.15.33:0/709999631 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308009ea0 con 0x7fa318159400 2023-10-06T05:24:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa316ffd700 1 -- 172.21.15.33:0/709999631 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30800b860 con 0x7fa318159400 2023-10-06T05:24:02.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa316ffd700 1 -- 172.21.15.33:0/709999631 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800bbe0 con 0x7fa318159400 2023-10-06T05:24:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7f5dc759e700 1 --2- 172.21.15.33:0/4013488991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8159400 0x7f5dc80cab60 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7f5dc759e700 1 -- 172.21.15.33:0/4013488991 >> 172.21.15.33:0/4013488991 conn(0x7f5dc806e4b0 msgr2=0x7f5dc80d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7f5dc759e700 1 -- 172.21.15.33:0/4013488991 shutdown_connections 2023-10-06T05:24:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 msgr2=0x7fa3180cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa31d79d700 1 --2- 172.21.15.33:0/709999631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 0x7fa3180cab60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa3080099b0 tx=0x7fa308009250 comp rx=0 tx=0).stop 2023-10-06T05:24:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 shutdown_connections 2023-10-06T05:24:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.338+0000 7f5dc759e700 1 -- 172.21.15.33:0/4013488991 wait complete. 2023-10-06T05:24:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa31d79d700 1 --2- 172.21.15.33:0/709999631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa318159400 0x7fa3180cab60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa31d79d700 1 --2- 172.21.15.33:0/709999631 >> v2:172.21.15.33:3300/0 conn(0x7fa3181597d0 0x7fa3180cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.337+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 >> 172.21.15.33:0/709999631 conn(0x7fa31806e4b0 msgr2=0x7fa3180d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.338+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 shutdown_connections 2023-10-06T05:24:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.338+0000 7fa31d79d700 1 -- 172.21.15.33:0/709999631 wait complete. 2023-10-06T05:24:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.339+0000 7fa31d79d700 1 Processor -- start 2023-10-06T05:24:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.339+0000 7fa31d79d700 1 -- start start 2023-10-06T05:24:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.340+0000 7f5dc759e700 1 Processor -- start 2023-10-06T05:24:02.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.340+0000 7f61ed79d700 1 Processor -- start 2023-10-06T05:24:02.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.340+0000 7f61ed79d700 1 -- start start 2023-10-06T05:24:02.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.340+0000 7f5dc759e700 1 -- start start 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.340+0000 7fa31d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3180cc770 0x7fa3181698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.340+0000 7fa31d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 0x7fa31815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa31d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3181597d0 0x7fa318152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa31d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180ca0c0 con 0x7fa318159400 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa31d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31815a860 con 0x7fa3181597d0 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa31d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa318159f30 con 0x7fa3180cc770 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa3177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 0x7fa31815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa317fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3180cc770 0x7fa3181698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa3177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 0x7fa31815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51056/0 (socket says 172.21.15.33:51056) 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa3177fe700 1 -- 172.21.15.33:0/824755020 learned_addr learned my addr 172.21.15.33:0/824755020 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa31cf9c700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3181597d0 0x7fa318152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa317fff700 1 -- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3181597d0 msgr2=0x7fa318152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa317fff700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3181597d0 0x7fa318152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.341+0000 7fa317fff700 1 -- 172.21.15.33:0/824755020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 msgr2=0x7fa31815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7fa317fff700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 0x7fa31815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7fa317fff700 1 -- 172.21.15.33:0/824755020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009420 con 0x7fa3180cc770 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7fa3177fe700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 0x7fa31815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61ed79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f61e8159400 0x7f61e80cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7fa317fff700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3180cc770 0x7fa3181698d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cb5e0 tx=0x7fa308017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f5dc759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 0x7f5dc80c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61ed79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e80d0ed0 con 0x7f61e81597d0 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61ed79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e80cb6d0 con 0x7f61e806cc70 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61ed79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e80cb9b0 con 0x7f61e8159400 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61e77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f61e81597d0 0x7f61e806c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34758/0 (socket says 172.21.15.33:34758) 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61e77fe700 1 -- 172.21.15.33:0/2592689757 learned_addr learned my addr 172.21.15.33:0/2592689757 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.342+0000 7f61e7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f61e8159400 0x7f61e80cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f5dc759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc8159400 0x7f5dc815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308000d10 con 0x7fa3180cc770 2023-10-06T05:24:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30800b9d0 con 0x7fa3180cc770 2023-10-06T05:24:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308025650 con 0x7fa3180cc770 2023-10-06T05:24:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f5dc659c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 0x7f5dc80c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f5dc659c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 0x7f5dc80c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35658/0 (socket says 172.21.15.33:35658) 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f5dc759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81597d0 0x7f5dc815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f61e6ffd700 1 -- 172.21.15.33:0/2592689757 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343651102 0 0) 0x7f61e80cb6d0 con 0x7f61e806cc70 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f5dc5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc8159400 0x7f5dc815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f5dc5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc8159400 0x7f5dc815db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:54674/0 (socket says 172.21.15.33:54674) 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f61e7fff700 1 -- 172.21.15.33:0/2592689757 >> v1:172.21.15.167:6789/0 conn(0x7f61e806cc70 legacy=0x7f61e80b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f61e7fff700 1 -- 172.21.15.33:0/2592689757 >> v1:172.21.15.33:6789/0 conn(0x7f61e81597d0 legacy=0x7f61e806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7f61e7fff700 1 -- 172.21.15.33:0/2592689757 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80cbc90 con 0x7f61e8159400 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.343+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3080094f0 con 0x7fa3180cc770 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31815a4f0 con 0x7fa3180cc770 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f61e7fff700 1 --2- 172.21.15.33:0/2592689757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e80cab60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f61d80098e0 tx=0x7f61d80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=178a62ba9e865335 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8169b20 con 0x7f5dc81597d0 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc659c700 1 -- 172.21.15.33:0/2369769565 learned_addr learned my addr 172.21.15.33:0/2369769565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f61e6ffd700 1 -- 172.21.15.33:0/2592689757 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8009e80 con 0x7f61e8159400 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc6d9d700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81597d0 0x7f5dc815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc816acb0 con 0x7f5dc8159400 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8169d10 con 0x7f5dc806cc70 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f61e6ffd700 1 -- 172.21.15.33:0/2592689757 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d800b860 con 0x7f61e8159400 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f61e6ffd700 1 -- 172.21.15.33:0/2592689757 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d800bc00 con 0x7f61e8159400 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc659c700 1 -- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc8159400 msgr2=0x7f5dc815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.344+0000 7f5dc659c700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc8159400 0x7f5dc815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5dc659c700 1 -- 172.21.15.33:0/2369769565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81597d0 msgr2=0x7f5dc815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5dc659c700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81597d0 0x7f5dc815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 -- 172.21.15.33:0/2592689757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 msgr2=0x7f61e80cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 --2- 172.21.15.33:0/2592689757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e80cab60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f61d80098e0 tx=0x7f61d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5dc659c700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80094f0 con 0x7f5dc806cc70 2023-10-06T05:24:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31804c380 con 0x7fa3180cc770 2023-10-06T05:24:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 -- 172.21.15.33:0/2592689757 shutdown_connections 2023-10-06T05:24:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 --2- 172.21.15.33:0/2592689757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e80cab60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 -- 172.21.15.33:0/2592689757 >> 172.21.15.33:0/2592689757 conn(0x7f61e806e4b0 msgr2=0x7f61e80d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5dc6d9d700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81597d0 0x7f5dc815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5dc659c700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 0x7f5dc80c64a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8065970 tx=0x7f5db8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 -- 172.21.15.33:0/2592689757 shutdown_connections 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800bbe0 con 0x7f5dc806cc70 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f61ed79d700 1 -- 172.21.15.33:0/2592689757 wait complete. 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db8011420 con 0x7f5dc806cc70 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.345+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80117a0 con 0x7f5dc806cc70 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.346+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc8169ff0 con 0x7f5dc806cc70 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.346+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc816a5b0 con 0x7f5dc806cc70 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.347+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa308023070 con 0x7fa3180cc770 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.347+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc804c380 con 0x7f5dc806cc70 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.347+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5db8011b90 con 0x7f5dc806cc70 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.347+0000 7fa2fffff700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa304066f30 0x7fa3040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.348+0000 7f61ed79d700 1 Processor -- start 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.348+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa308085760 con 0x7fa3180cc770 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.348+0000 7f61ed79d700 1 -- start start 2023-10-06T05:24:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.348+0000 7fa3177fe700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa304066f30 0x7fa3040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7fa3177fe700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa304066f30 0x7fa3040693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa31815bf90 tx=0x7fa30c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f5db67fc700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5da8066f30 0x7f5da80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5db8085b70 con 0x7f5dc806cc70 2023-10-06T05:24:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5da8066f30 0x7f5da80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61ed79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 0x7f61e8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61ed79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61ed79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e81597d0 0x7f61e814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61ed79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e814d5c0 con 0x7f61e806cc70 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61ed79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8155370 con 0x7f61e8159400 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61ed79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8155650 con 0x7f61e81597d0 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 0x7f61e8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 0x7f61e8154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51068/0 (socket says 172.21.15.33:51068) 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61e7fff700 1 -- 172.21.15.33:0/4033165119 learned_addr learned my addr 172.21.15.33:0/4033165119 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f5dc5d9b700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5da8066f30 0x7f5da80693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80b4340 tx=0x7f5dbc007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.349+0000 7f61e77fe700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61ecf9c700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e81597d0 0x7f61e814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61e7fff700 1 -- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e81597d0 msgr2=0x7f61e814d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61e7fff700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e81597d0 0x7f61e814d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61e7fff700 1 -- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 msgr2=0x7f61e814f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61e7fff700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e814f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61e7fff700 1 -- 172.21.15.33:0/4033165119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d8009420 con 0x7f61e806cc70 2023-10-06T05:24:02.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61e7fff700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 0x7f61e8154d00 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f61e8065970 tx=0x7f61d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8026450 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.350+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e814f130 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.351+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e80caa60 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.351+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d80267c0 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.351+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8026930 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.353+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e804c380 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.353+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61d8011420 con 0x7f61e806cc70 2023-10-06T05:24:02.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.353+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa308052870 con 0x7fa3180cc770 2023-10-06T05:24:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.354+0000 7f61cffff700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61d4066f30 0x7f61d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.354+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61d80863e0 con 0x7f61e806cc70 2023-10-06T05:24:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.355+0000 7f61e77fe700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61d4066f30 0x7f61d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.355+0000 7f61e77fe700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61d4066f30 0x7f61d40693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f61e8158b70 tx=0x7f61dc000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.358+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5db8052c90 con 0x7f5dc806cc70 2023-10-06T05:24:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.362+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61d8053500 con 0x7f61e806cc70 2023-10-06T05:24:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.390+0000 7fd0e4f9d700 1 Processor -- start 2023-10-06T05:24:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.390+0000 7fd0e4f9d700 1 -- start start 2023-10-06T05:24:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0e4f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e006c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0e4f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0e00cc770 0x7fd0e006caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0e4f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e00d0ed0 con 0x7fd0e0159400 2023-10-06T05:24:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0e4f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e00b1160 con 0x7fd0e00cc770 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0e4f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00cb6d0 con 0x7fd0e01597d0 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0df7fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd0e0159400 0x7fd0e00cab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:52492/0 (socket says 172.21.15.33:52492) 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0df7fe700 1 -- 172.21.15.33:0/2593392469 learned_addr learned my addr 172.21.15.33:0/2593392469 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0dffff700 1 --2- 172.21.15.33:0/2593392469 >> v2:172.21.15.33:3300/0 conn(0x7fd0e00cc770 0x7fd0e006caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.392+0000 7fd0deffd700 1 --2- 172.21.15.33:0/2593392469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0de7fc700 1 -- 172.21.15.33:0/2593392469 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071890199 0 0) 0x7fd0e00d0ed0 con 0x7fd0e0159400 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0de7fc700 1 -- 172.21.15.33:0/2593392469 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c8003570 con 0x7fd0e0159400 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0deffd700 1 -- 172.21.15.33:0/2593392469 >> v2:172.21.15.33:3300/0 conn(0x7fd0e00cc770 msgr2=0x7fd0e006caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0deffd700 1 --2- 172.21.15.33:0/2593392469 >> v2:172.21.15.33:3300/0 conn(0x7fd0e00cc770 0x7fd0e006caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0deffd700 1 -- 172.21.15.33:0/2593392469 >> v1:172.21.15.167:6789/0 conn(0x7fd0e0159400 legacy=0x7fd0e00cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0deffd700 1 -- 172.21.15.33:0/2593392469 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e00cb9b0 con 0x7fd0e01597d0 2023-10-06T05:24:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.393+0000 7fd0deffd700 1 --2- 172.21.15.33:0/2593392469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e006c560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40098e0 tx=0x7fd0d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d347f194e5c9a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.394+0000 7fd0de7fc700 1 -- 172.21.15.33:0/2593392469 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011630 con 0x7fd0e01597d0 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.394+0000 7fd0de7fc700 1 -- 172.21.15.33:0/2593392469 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d4011790 con 0x7fd0e01597d0 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.394+0000 7fd0de7fc700 1 -- 172.21.15.33:0/2593392469 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011b30 con 0x7fd0e01597d0 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.394+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/2593392469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 msgr2=0x7fd0e006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.394+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/2593392469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e006c560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40098e0 tx=0x7fd0d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.395+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/2593392469 shutdown_connections 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.395+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/2593392469 >> v2:172.21.15.33:3300/0 conn(0x7fd0e00cc770 0x7fd0e006caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.395+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/2593392469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e006c560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.395+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/2593392469 >> 172.21.15.33:0/2593392469 conn(0x7fd0e006e4b0 msgr2=0x7fd0e00d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.395+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/2593392469 shutdown_connections 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.395+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/2593392469 wait complete. 2023-10-06T05:24:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.397+0000 7fd0e4f9d700 1 Processor -- start 2023-10-06T05:24:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.397+0000 7fd0e4f9d700 1 -- start start 2023-10-06T05:24:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.399+0000 7fd0e4f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 0x7fd0e0152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.399+0000 7fd0e4f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e0159400 0x7fd0e0158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.399+0000 7fd0e4f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.399+0000 7fd0e4f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e016aae0 con 0x7fd0e00cc770 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.399+0000 7fd0e4f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0169ed0 con 0x7fd0e0159400 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.399+0000 7fd0e4f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e016a1b0 con 0x7fd0e01597d0 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 0x7fd0e0152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 0x7fd0e0152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51088/0 (socket says 172.21.15.33:51088) 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 -- 172.21.15.33:0/3351930993 learned_addr learned my addr 172.21.15.33:0/3351930993 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0dffff700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0deffd700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e0159400 0x7fd0e0158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 -- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 msgr2=0x7fd0e015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 -- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e0159400 msgr2=0x7fd0e0158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e0159400 0x7fd0e0158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.400+0000 7fd0df7fe700 1 -- 172.21.15.33:0/3351930993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d4009420 con 0x7fd0e00cc770 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.401+0000 7fd0df7fe700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 0x7fd0e0152490 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c880 tx=0x7fd0d0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.401+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0007150 con 0x7fd0e00cc770 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.401+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d00072b0 con 0x7fd0e00cc770 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.401+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e016a490 con 0x7fd0e00cc770 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.401+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0007630 con 0x7fd0e00cc770 2023-10-06T05:24:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.401+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e0169b20 con 0x7fd0e00cc770 2023-10-06T05:24:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.402+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e004c380 con 0x7fd0e00cc770 2023-10-06T05:24:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.404+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0d0004a40 con 0x7fd0e00cc770 2023-10-06T05:24:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.405+0000 7fd0c77fe700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0cc066f30 0x7fd0cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.405+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0d0083a30 con 0x7fd0e00cc770 2023-10-06T05:24:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.406+0000 7fd0deffd700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0cc066f30 0x7fd0cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.407+0000 7fd0deffd700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0cc066f30 0x7fd0cc0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40091e0 tx=0x7fd0d4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.414+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0d004c5a0 con 0x7fd0e00cc770 2023-10-06T05:24:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.649+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f5dc80cbc90 con 0x7f5dc806cc70 2023-10-06T05:24:02.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.654+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fa3180cbc90 con 0x7fa3180cc770 2023-10-06T05:24:02.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.654+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f61e80cbf70 con 0x7f61e806cc70 2023-10-06T05:24:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:02.700+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fd0e00cbc90 con 0x7fd0e00cc770 2023-10-06T05:24:03.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.033+0000 7f5db67fc700 1 -- 172.21.15.33:0/2369769565 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6512 v91) v1 ==== 95+0+0 (secure 0 0 0) 0x7f5db8015090 con 0x7f5dc806cc70 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5da8066f30 msgr2=0x7f5da80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5da8066f30 0x7f5da80693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80b4340 tx=0x7f5dbc007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 msgr2=0x7f5dc80c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 0x7f5dc80c64a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8065970 tx=0x7f5db8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 shutdown_connections 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5da8066f30 0x7f5da80693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81597d0 0x7f5dc815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc806cc70 0x7f5dc80c64a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 --2- 172.21.15.33:0/2369769565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc8159400 0x7f5dc815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 >> 172.21.15.33:0/2369769565 conn(0x7f5dc806e4b0 msgr2=0x7f5dc80cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 shutdown_connections 2023-10-06T05:24:03.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.035+0000 7f5dc759e700 1 -- 172.21.15.33:0/2369769565 wait complete. 2023-10-06T05:24:03.040 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 6512 2023-10-06T05:24:03.045 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:24:03.043+0000 7f50273ec700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.053+0000 7f61cffff700 1 -- 172.21.15.33:0/4033165119 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6518 v92) v1 ==== 95+0+0 (secure 0 0 0) 0x7f61d8056b20 con 0x7f61e806cc70 2023-10-06T05:24:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.053+0000 7fd0c77fe700 1 -- 172.21.15.33:0/3351930993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7971 v92) v1 ==== 95+0+0 (secure 0 0 0) 0x7fd0d0018070 con 0x7fd0e00cc770 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0cc066f30 msgr2=0x7fd0cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0cc066f30 0x7fd0cc0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40091e0 tx=0x7fd0d4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 msgr2=0x7fd0e0152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 0x7fd0e0152490 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c880 tx=0x7fd0d0007e40 comp rx=0 tx=0).stop 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61d4066f30 msgr2=0x7f61d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61d4066f30 0x7f61d40693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f61e8158b70 tx=0x7f61dc000bc0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 msgr2=0x7f61e8154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 0x7f61e8154d00 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f61e8065970 tx=0x7f61d8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 shutdown_connections 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0cc066f30 0x7fd0cc0693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e01597d0 0x7fd0e015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00cc770 0x7fd0e0152490 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 --2- 172.21.15.33:0/3351930993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e0159400 0x7fd0e0158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 >> 172.21.15.33:0/3351930993 conn(0x7fd0e006e4b0 msgr2=0x7fd0e0152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 shutdown_connections 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61d4066f30 0x7f61d40693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61e81597d0 0x7f61e814d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61e806cc70 0x7f61e8154d00 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 --2- 172.21.15.33:0/4033165119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e8159400 0x7f61e814f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.056+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 >> 172.21.15.33:0/4033165119 conn(0x7f61e806e4b0 msgr2=0x7f61e80ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.057+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 shutdown_connections 2023-10-06T05:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.057+0000 7fd0e4f9d700 1 -- 172.21.15.33:0/3351930993 wait complete. 2023-10-06T05:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.057+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 shutdown_connections 2023-10-06T05:24:03.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.057+0000 7f61ed79d700 1 -- 172.21.15.33:0/4033165119 wait complete. 2023-10-06T05:24:03.066 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:24:03.066 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:24:03.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.058+0000 7fa2fffff700 1 -- 172.21.15.33:0/824755020 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 7426 v92) v1 ==== 95+0+0 (secure 0 0 0) 0x7fa308055e90 con 0x7fa3180cc770 2023-10-06T05:24:03.067 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 7971 2023-10-06T05:24:03.067 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 6518 2023-10-06T05:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa304066f30 msgr2=0x7fa3040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa304066f30 0x7fa3040693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa31815bf90 tx=0x7fa30c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3180cc770 msgr2=0x7fa3181698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3180cc770 0x7fa3181698d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3180cb5e0 tx=0x7fa308017000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 shutdown_connections 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa304066f30 0x7fa3040693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3181597d0 0x7fa318152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3180cc770 0x7fa3181698d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 --2- 172.21.15.33:0/824755020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa318159400 0x7fa31815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 >> 172.21.15.33:0/824755020 conn(0x7fa31806e4b0 msgr2=0x7fa3180b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 shutdown_connections 2023-10-06T05:24:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.062+0000 7fa31d79d700 1 -- 172.21.15.33:0/824755020 wait complete. 2023-10-06T05:24:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 7426 2023-10-06T05:24:03.076 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:24:03.076 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:03.083 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:24:03.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:24:03.084 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:24:03.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:24:03.089 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:24:03.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:24:03.090 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:24:03.090 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:03.094 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:24:03.094 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:03.133 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:24:03.133 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:03.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:03.213 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:24:03.230 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:24:03.231 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:24:03.273 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:24:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.330+0000 7ff3da99d700 1 Processor -- start 2023-10-06T05:24:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.330+0000 7ff3da99d700 1 -- start start 2023-10-06T05:24:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.331+0000 7ff3da99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3d40c6340 0x7ff3d40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.331+0000 7ff3da99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d406c820 con 0x7ff3d40c80d0 2023-10-06T05:24:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.331+0000 7ff3da99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d406a160 con 0x7ff3d4159520 2023-10-06T05:24:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.331+0000 7ff3da99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d40c8800 con 0x7ff3d40c6340 2023-10-06T05:24:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3d40c6340 0x7ff3d40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3d40c6340 0x7ff3d40c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51132/0 (socket says 172.21.15.33:51132) 2023-10-06T05:24:03.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3da19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff3d40c80d0 0x7ff3d406a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34784/0 (socket says 172.21.15.33:34784) 2023-10-06T05:24:03.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 -- 172.21.15.33:0/2571568558 learned_addr learned my addr 172.21.15.33:0/2571568558 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 -- 172.21.15.33:0/2571568558 >> v1:172.21.15.167:6789/0 conn(0x7ff3d4159520 legacy=0x7ff3d40c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 -- 172.21.15.33:0/2571568558 >> v1:172.21.15.33:6789/0 conn(0x7ff3d40c80d0 legacy=0x7ff3d406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d8999700 1 -- 172.21.15.33:0/2571568558 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984392092 0 0) 0x7ff3d406c820 con 0x7ff3d40c80d0 2023-10-06T05:24:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 -- 172.21.15.33:0/2571568558 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40c8ae0 con 0x7ff3d40c6340 2023-10-06T05:24:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.332+0000 7ff3d919a700 1 --2- 172.21.15.33:0/2571568558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d40c6340 0x7ff3d40c7b90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40099b0 tx=0x7ff3c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9859008214ff1364 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3d8999700 1 -- 172.21.15.33:0/2571568558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400b860 con 0x7ff3d40c6340 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3d8999700 1 -- 172.21.15.33:0/2571568558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c400b9c0 con 0x7ff3d40c6340 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3d8999700 1 -- 172.21.15.33:0/2571568558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400bd60 con 0x7ff3d40c6340 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 -- 172.21.15.33:0/2571568558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d40c6340 msgr2=0x7ff3d40c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2571568558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d40c6340 0x7ff3d40c7b90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40099b0 tx=0x7ff3c4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 -- 172.21.15.33:0/2571568558 shutdown_connections 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2571568558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d40c6340 0x7ff3d40c7b90 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 -- 172.21.15.33:0/2571568558 >> 172.21.15.33:0/2571568558 conn(0x7ff3d40cc930 msgr2=0x7ff3d40ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 -- 172.21.15.33:0/2571568558 shutdown_connections 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.333+0000 7ff3da99d700 1 -- 172.21.15.33:0/2571568558 wait complete. 2023-10-06T05:24:03.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.335+0000 7ff3da99d700 1 Processor -- start 2023-10-06T05:24:03.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.335+0000 7ff3da99d700 1 -- start start 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3d40c6340 0x7ff3d41565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d40c80d0 0x7ff3d415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 0x7ff3d40aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d415bb20 con 0x7ff3d4159520 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4156b30 con 0x7ff3d40c80d0 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d415af10 con 0x7ff3d40c6340 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 0x7ff3d40aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 0x7ff3d40aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51142/0 (socket says 172.21.15.33:51142) 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 -- 172.21.15.33:0/2750013998 learned_addr learned my addr 172.21.15.33:0/2750013998 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3d999b700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3d40c6340 0x7ff3d41565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3d919a700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d40c80d0 0x7ff3d415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 -- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3d40c6340 msgr2=0x7ff3d41565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3d40c6340 0x7ff3d41565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 -- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d40c80d0 msgr2=0x7ff3d415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d40c80d0 0x7ff3d415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3da19c700 1 -- 172.21.15.33:0/2750013998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40094f0 con 0x7ff3d4159520 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.336+0000 7ff3d919a700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d40c80d0 0x7ff3d415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.337+0000 7ff3da19c700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 0x7ff3d40aec10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00cd50 tx=0x7ff3cc001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.337+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc005df0 con 0x7ff3d4159520 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.337+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc00a960 con 0x7ff3d4159520 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.337+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00ad20 con 0x7ff3d4159520 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.337+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d415b1f0 con 0x7ff3d4159520 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.337+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d415b7b0 con 0x7ff3d4159520 2023-10-06T05:24:03.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.338+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d415bc90 con 0x7ff3d4159520 2023-10-06T05:24:03.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.339+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3cc007300 con 0x7ff3d4159520 2023-10-06T05:24:03.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.340+0000 7ff3c9ffb700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3b8066f30 0x7ff3b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.341+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff3cc084970 con 0x7ff3d4159520 2023-10-06T05:24:03.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.341+0000 7ff3d999b700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3b8066f30 0x7ff3b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.341+0000 7ff3d999b700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3b8066f30 0x7ff3b80693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00037c0 tx=0x7ff3d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.345+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3cc051a30 con 0x7ff3d4159520 2023-10-06T05:24:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.593+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3d404c380 con 0x7ff3d4159520 2023-10-06T05:24:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.594+0000 7ff3c9ffb700 1 -- 172.21.15.33:0/2750013998 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+1565 (secure 0 0 0) 0x7ff3cc055050 con 0x7ff3d4159520 2023-10-06T05:24:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:03.597 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:03.045110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3b8066f30 msgr2=0x7ff3b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3b8066f30 0x7ff3b80693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00037c0 tx=0x7ff3d0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 msgr2=0x7ff3d40aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 0x7ff3d40aec10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00cd50 tx=0x7ff3cc001b10 comp rx=0 tx=0).stop 2023-10-06T05:24:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 shutdown_connections 2023-10-06T05:24:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3b8066f30 0x7ff3b80693e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4159520 0x7ff3d40aec10 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3d40c6340 0x7ff3d41565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 --2- 172.21.15.33:0/2750013998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d40c80d0 0x7ff3d415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 >> 172.21.15.33:0/2750013998 conn(0x7ff3d40cc930 msgr2=0x7ff3d40c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 shutdown_connections 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:03.596+0000 7ff3da99d700 1 -- 172.21.15.33:0/2750013998 wait complete. 2023-10-06T05:24:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 92 2023-10-06T05:24:03.616 DEBUG:tasks.cephfs.filesystem:mapped states [] to None 2023-10-06T05:24:04.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.818+0000 7f3cd2b9d700 1 Processor -- start 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.818+0000 7f3cd2b9d700 1 -- start start 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.819+0000 7f3cd2b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3ccc15a0a0 0x7f3ccc15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.819+0000 7f3cd2b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc15e2d0 con 0x7f3ccc06ab00 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd2b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc06afa0 con 0x7f3ccc0b0a40 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd2b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc15d6c0 con 0x7f3ccc15a0a0 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd139a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3ccc06ab00 0x7f3ccc0b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34796/0 (socket says 172.21.15.33:34796) 2023-10-06T05:24:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd139a700 1 -- 172.21.15.33:0/2789019426 learned_addr learned my addr 172.21.15.33:0/2789019426 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd239c700 1 --2- 172.21.15.33:0/2789019426 >> v2:172.21.15.167:3300/0 conn(0x7f3ccc15a0a0 0x7f3ccc15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004890453 0 0) 0x7f3ccc15e2d0 con 0x7f3ccc06ab00 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb8003570 con 0x7f3ccc06ab00 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261363521 0 0) 0x7f3ccc06afa0 con 0x7f3ccc0b0a40 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ccc15e2d0 con 0x7f3ccc0b0a40 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd239c700 1 -- 172.21.15.33:0/2789019426 >> v1:172.21.15.167:6789/0 conn(0x7f3ccc0b0a40 legacy=0x7f3ccc0b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd239c700 1 -- 172.21.15.33:0/2789019426 >> v1:172.21.15.33:6789/0 conn(0x7f3ccc06ab00 legacy=0x7f3ccc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.820+0000 7f3cd239c700 1 -- 172.21.15.33:0/2789019426 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc15d9a0 con 0x7f3ccc15a0a0 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.821+0000 7f3cd239c700 1 --2- 172.21.15.33:0/2789019426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc15a4e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009e60 tx=0x7f3cc4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa824ab3f60c642 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.821+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400e070 con 0x7f3ccc15a0a0 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.821+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc400bab0 con 0x7f3ccc15a0a0 2023-10-06T05:24:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.821+0000 7f3cd0b99700 1 -- 172.21.15.33:0/2789019426 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400be70 con 0x7f3ccc15a0a0 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/2789019426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 msgr2=0x7f3ccc15a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/2789019426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc15a4e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009e60 tx=0x7f3cc4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/2789019426 shutdown_connections 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/2789019426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc15a4e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/2789019426 >> 172.21.15.33:0/2789019426 conn(0x7f3ccc06e0f0 msgr2=0x7f3ccc0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/2789019426 shutdown_connections 2023-10-06T05:24:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.822+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/2789019426 wait complete. 2023-10-06T05:24:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.823+0000 7f3cd2b9d700 1 Processor -- start 2023-10-06T05:24:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.823+0000 7f3cd2b9d700 1 -- start start 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd2b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 0x7f3ccc16a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd2b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ccc0b0a40 0x7f3ccc16b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd2b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd2b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc16d130 con 0x7f3ccc06ab00 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd2b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc16be10 con 0x7f3ccc15a0a0 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd2b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0cbd00 con 0x7f3ccc0b0a40 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.824+0000 7f3cd1b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 0x7f3ccc16a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 0x7f3ccc16a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51158/0 (socket says 172.21.15.33:51158) 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 -- 172.21.15.33:0/4082906465 learned_addr learned my addr 172.21.15.33:0/4082906465 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd139a700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ccc0b0a40 0x7f3ccc16b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd239c700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 -- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ccc0b0a40 msgr2=0x7f3ccc16b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ccc0b0a40 0x7f3ccc16b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 -- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 msgr2=0x7f3ccc0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 -- 172.21.15.33:0/4082906465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009a20 con 0x7f3ccc06ab00 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd139a700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ccc0b0a40 0x7f3ccc16b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd239c700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd1b9b700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 0x7f3ccc16a920 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c790 tx=0x7f3cc8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8010510 con 0x7f3ccc06ab00 2023-10-06T05:24:04.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8010670 con 0x7f3ccc06ab00 2023-10-06T05:24:04.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8010a10 con 0x7f3ccc06ab00 2023-10-06T05:24:04.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.825+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc16c090 con 0x7f3ccc06ab00 2023-10-06T05:24:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.826+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc16c650 con 0x7f3ccc06ab00 2023-10-06T05:24:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.826+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc06fcc0 con 0x7f3ccc06ab00 2023-10-06T05:24:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.827+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cc801b400 con 0x7f3ccc06ab00 2023-10-06T05:24:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.828+0000 7f3cc1ffb700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cb0066f30 0x7f3cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.828+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3cc8084530 con 0x7f3ccc06ab00 2023-10-06T05:24:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.829+0000 7f3cd139a700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cb0066f30 0x7f3cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.829+0000 7f3cd139a700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cb0066f30 0x7f3cb00693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc007fd0 tx=0x7f3cbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:04.832+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cc8051570 con 0x7f3ccc06ab00 2023-10-06T05:24:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.080+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ccc04c380 con 0x7f3ccc06ab00 2023-10-06T05:24:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.080+0000 7f3cc1ffb700 1 -- 172.21.15.33:0/4082906465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f3cc8054b90 con 0x7f3ccc06ab00 2023-10-06T05:24:05.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:05.084 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:04.779402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cb0066f30 msgr2=0x7f3cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cb0066f30 0x7f3cb00693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc007fd0 tx=0x7f3cbc009040 comp rx=0 tx=0).stop 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 msgr2=0x7f3ccc16a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 0x7f3ccc16a920 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c790 tx=0x7f3cc8007e10 comp rx=0 tx=0).stop 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 shutdown_connections 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cb0066f30 0x7f3cb00693e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ccc15a0a0 0x7f3ccc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ccc06ab00 0x7f3ccc16a920 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 --2- 172.21.15.33:0/4082906465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ccc0b0a40 0x7f3ccc16b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 >> 172.21.15.33:0/4082906465 conn(0x7f3ccc06e0f0 msgr2=0x7f3ccc158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:05.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 shutdown_connections 2023-10-06T05:24:05.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:05.083+0000 7f3cd2b9d700 1 -- 172.21.15.33:0/4082906465 wait complete. 2023-10-06T05:24:05.087 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 94 2023-10-06T05:24:05.103 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:24:06.104 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.301+0000 7f5aeb59e700 1 Processor -- start 2023-10-06T05:24:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.301+0000 7f5aeb59e700 1 -- start start 2023-10-06T05:24:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aeb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aeb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5aec0cc750 0x7f5aec0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aeb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0cbd80 con 0x7f5aec0cf3b0 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aeb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec0af5f0 con 0x7f5aec0cc750 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aeb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0af8d0 con 0x7f5aec0ccfc0 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aead9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5aec0cc750 0x7f5aec0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aead9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5aec0cc750 0x7f5aec0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51160/0 (socket says 172.21.15.33:51160) 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.302+0000 7f5aead9d700 1 -- 172.21.15.33:0/559765152 learned_addr learned my addr 172.21.15.33:0/559765152 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5aea59c700 1 --2- 172.21.15.33:0/559765152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5aead9d700 1 -- 172.21.15.33:0/559765152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 msgr2=0x7f5aec0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5aead9d700 1 --2- 172.21.15.33:0/559765152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5aead9d700 1 -- 172.21.15.33:0/559765152 >> v1:172.21.15.167:6789/0 conn(0x7f5aec0cf3b0 legacy=0x7f5aec1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5aead9d700 1 -- 172.21.15.33:0/559765152 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0afbb0 con 0x7f5aec0cc750 2023-10-06T05:24:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5aead9d700 1 --2- 172.21.15.33:0/559765152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec0cc750 0x7f5aec0b0460 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009f20 tx=0x7f5ae4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02636a9f908939c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5ae959a700 1 -- 172.21.15.33:0/559765152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400e070 con 0x7f5aec0cc750 2023-10-06T05:24:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5ae959a700 1 -- 172.21.15.33:0/559765152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae400b9d0 con 0x7f5aec0cc750 2023-10-06T05:24:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.303+0000 7f5ae959a700 1 -- 172.21.15.33:0/559765152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40092e0 con 0x7f5aec0cc750 2023-10-06T05:24:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 -- 172.21.15.33:0/559765152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec0cc750 msgr2=0x7f5aec0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/559765152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec0cc750 0x7f5aec0b0460 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009f20 tx=0x7f5ae4009840 comp rx=0 tx=0).stop 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 -- 172.21.15.33:0/559765152 shutdown_connections 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/559765152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec0cc750 0x7f5aec0b0460 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/559765152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 -- 172.21.15.33:0/559765152 >> 172.21.15.33:0/559765152 conn(0x7f5aec06cff0 msgr2=0x7f5aec06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 -- 172.21.15.33:0/559765152 shutdown_connections 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.304+0000 7f5aeb59e700 1 -- 172.21.15.33:0/559765152 wait complete. 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.306+0000 7f5aeb59e700 1 Processor -- start 2023-10-06T05:24:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.306+0000 7f5aeb59e700 1 -- start start 2023-10-06T05:24:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aeb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec0cc750 0x7f5aec14f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aeb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aeb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 0x7f5aec15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aeb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b3c80 con 0x7f5aec158950 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aeb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0c7990 con 0x7f5aec0cc750 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aeb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0c7c70 con 0x7f5aec0ccfc0 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 0x7f5aec15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 0x7f5aec15f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51162/0 (socket says 172.21.15.33:51162) 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 -- 172.21.15.33:0/3691209150 learned_addr learned my addr 172.21.15.33:0/3691209150 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aea59c700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec0cc750 0x7f5aec14f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5ae9d9b700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 -- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 msgr2=0x7f5aec158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 -- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec0cc750 msgr2=0x7f5aec14f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec0cc750 0x7f5aec14f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5aead9d700 1 -- 172.21.15.33:0/3691209150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009ae0 con 0x7f5aec158950 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.307+0000 7f5ae9d9b700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.308+0000 7f5aead9d700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 0x7f5aec15f210 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5aec06be20 tx=0x7f5ae4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.308+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4008b80 con 0x7f5aec158950 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.308+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4008ce0 con 0x7f5aec158950 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.308+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40266b0 con 0x7f5aec158950 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.308+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec0c7f50 con 0x7f5aec158950 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.308+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec0c75b0 con 0x7f5aec158950 2023-10-06T05:24:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.309+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec0b3e80 con 0x7f5aec158950 2023-10-06T05:24:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.310+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ae4003040 con 0x7f5aec158950 2023-10-06T05:24:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.311+0000 7f5ada7fc700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5acc066f30 0x7f5acc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.311+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5ae4086060 con 0x7f5aec158950 2023-10-06T05:24:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.312+0000 7f5aea59c700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5acc066f30 0x7f5acc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.312+0000 7f5aea59c700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5acc066f30 0x7f5acc0693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009db0 tx=0x7f5adc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.315+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5aec0b3e80 con 0x7f5aec158950 2023-10-06T05:24:06.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.563+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aec04c380 con 0x7f5aec158950 2023-10-06T05:24:06.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.563+0000 7f5ada7fc700 1 -- 172.21.15.33:0/3691209150 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f5ae4053120 con 0x7f5aec158950 2023-10-06T05:24:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:06.570 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:05.786297+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.565+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5acc066f30 msgr2=0x7f5acc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.565+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5acc066f30 0x7f5acc0693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009db0 tx=0x7f5adc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.565+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 msgr2=0x7f5aec15f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.565+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 0x7f5aec15f210 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5aec06be20 tx=0x7f5ae4018000 comp rx=0 tx=0).stop 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 shutdown_connections 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5acc066f30 0x7f5acc0693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec158950 0x7f5aec15f210 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec0cc750 0x7f5aec14f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 --2- 172.21.15.33:0/3691209150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0ccfc0 0x7f5aec158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 >> 172.21.15.33:0/3691209150 conn(0x7f5aec06cff0 msgr2=0x7f5aec0cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:06.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 shutdown_connections 2023-10-06T05:24:06.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:06.566+0000 7f5aeb59e700 1 -- 172.21.15.33:0/3691209150 wait complete. 2023-10-06T05:24:06.573 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 95 2023-10-06T05:24:06.585 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2023-10-06T05:24:07.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.782+0000 7fce0e99d700 1 Processor -- start 2023-10-06T05:24:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.782+0000 7fce0e99d700 1 -- start start 2023-10-06T05:24:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.783+0000 7fce0e99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce0806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.783+0000 7fce0e99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fce0806a6a0 0x7fce080c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.783+0000 7fce0e99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0806ab60 con 0x7fce080c8b50 2023-10-06T05:24:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.783+0000 7fce0e99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0806b7e0 con 0x7fce0806a6a0 2023-10-06T05:24:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.783+0000 7fce0e99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0806ae40 con 0x7fce08155af0 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.784+0000 7fce0d19a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fce0806a6a0 0x7fce080c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.784+0000 7fce0d99b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce0806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.784+0000 7fce0e19c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fce080c8b50 0x7fce081598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:52534/0 (socket says 172.21.15.33:52534) 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.784+0000 7fce0e19c700 1 -- 172.21.15.33:0/2389570452 learned_addr learned my addr 172.21.15.33:0/2389570452 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0c999700 1 -- 172.21.15.33:0/2389570452 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324906901 0 0) 0x7fce0806ab60 con 0x7fce080c8b50 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0c999700 1 -- 172.21.15.33:0/2389570452 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf4003570 con 0x7fce080c8b50 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0d99b700 1 -- 172.21.15.33:0/2389570452 >> v2:172.21.15.167:3300/0 conn(0x7fce0806a6a0 msgr2=0x7fce080c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0d99b700 1 --2- 172.21.15.33:0/2389570452 >> v2:172.21.15.167:3300/0 conn(0x7fce0806a6a0 0x7fce080c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0d99b700 1 -- 172.21.15.33:0/2389570452 >> v1:172.21.15.167:6789/0 conn(0x7fce080c8b50 legacy=0x7fce081598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0d99b700 1 -- 172.21.15.33:0/2389570452 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0806b120 con 0x7fce08155af0 2023-10-06T05:24:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0d19a700 1 --2- 172.21.15.33:0/2389570452 >> v2:172.21.15.167:3300/0 conn(0x7fce0806a6a0 0x7fce080c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0d99b700 1 --2- 172.21.15.33:0/2389570452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce0806a160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce040098e0 tx=0x7fce040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d88062dc49319b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0c999700 1 -- 172.21.15.33:0/2389570452 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04011630 con 0x7fce08155af0 2023-10-06T05:24:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.785+0000 7fce0c999700 1 -- 172.21.15.33:0/2389570452 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04011790 con 0x7fce08155af0 2023-10-06T05:24:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0c999700 1 -- 172.21.15.33:0/2389570452 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04011b10 con 0x7fce08155af0 2023-10-06T05:24:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 -- 172.21.15.33:0/2389570452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 msgr2=0x7fce0806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2389570452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce0806a160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce040098e0 tx=0x7fce040091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 -- 172.21.15.33:0/2389570452 shutdown_connections 2023-10-06T05:24:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2389570452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce0806a160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2389570452 >> v2:172.21.15.167:3300/0 conn(0x7fce0806a6a0 0x7fce080c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 -- 172.21.15.33:0/2389570452 >> 172.21.15.33:0/2389570452 conn(0x7fce0806d030 msgr2=0x7fce0806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 -- 172.21.15.33:0/2389570452 shutdown_connections 2023-10-06T05:24:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.786+0000 7fce0e99d700 1 -- 172.21.15.33:0/2389570452 wait complete. 2023-10-06T05:24:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.787+0000 7fce0e99d700 1 Processor -- start 2023-10-06T05:24:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.788+0000 7fce0e99d700 1 -- start start 2023-10-06T05:24:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.788+0000 7fce0e99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 0x7fce08143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0e99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce080c8b50 0x7fce08148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0e99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce080c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0e99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce080afcb0 con 0x7fce0806a6a0 2023-10-06T05:24:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0e99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce08149ce0 con 0x7fce080c8b50 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0e99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce080af6a0 con 0x7fce08155af0 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 0x7fce08143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 0x7fce08143e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51172/0 (socket says 172.21.15.33:51172) 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 -- 172.21.15.33:0/2839574592 learned_addr learned my addr 172.21.15.33:0/2839574592 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d19a700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce080c8b50 0x7fce08148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0e19c700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce080c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 -- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 msgr2=0x7fce080c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce080c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 -- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce080c8b50 msgr2=0x7fce08148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce080c8b50 0x7fce08148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.789+0000 7fce0d99b700 1 -- 172.21.15.33:0/2839574592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8009420 con 0x7fce0806a6a0 2023-10-06T05:24:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.790+0000 7fce0d99b700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 0x7fce08143e70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fce0806da70 tx=0x7fce04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.790+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04011900 con 0x7fce0806a6a0 2023-10-06T05:24:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.790+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04011a60 con 0x7fce0806a6a0 2023-10-06T05:24:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.790+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce04009420 con 0x7fce0806a6a0 2023-10-06T05:24:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.790+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0401a680 con 0x7fce0806a6a0 2023-10-06T05:24:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.790+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce08149720 con 0x7fce0806a6a0 2023-10-06T05:24:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.791+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0804c380 con 0x7fce0806a6a0 2023-10-06T05:24:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.792+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce0401aa70 con 0x7fce0806a6a0 2023-10-06T05:24:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.793+0000 7fcdfdffb700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcdec066f30 0x7fcdec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.793+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fce040862c0 con 0x7fce0806a6a0 2023-10-06T05:24:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.793+0000 7fce0d19a700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcdec066f30 0x7fcdec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.794+0000 7fce0d19a700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcdec066f30 0x7fcdec0693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce0814a8e0 tx=0x7fcdf80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:07.797+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce04053300 con 0x7fce0806a6a0 2023-10-06T05:24:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.047+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce0806fb40 con 0x7fce0806a6a0 2023-10-06T05:24:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.047+0000 7fcdfdffb700 1 -- 172.21.15.33:0/2839574592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fce04056920 con 0x7fce0806a6a0 2023-10-06T05:24:08.050 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:08.050 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":96,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:06.792632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:08.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcdec066f30 msgr2=0x7fcdec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcdec066f30 0x7fcdec0693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce0814a8e0 tx=0x7fcdf80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 msgr2=0x7fce08143e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 0x7fce08143e70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fce0806da70 tx=0x7fce04017000 comp rx=0 tx=0).stop 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 shutdown_connections 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcdec066f30 0x7fcdec0693e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce08155af0 0x7fce080c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0806a6a0 0x7fce08143e70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 --2- 172.21.15.33:0/2839574592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce080c8b50 0x7fce08148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 >> 172.21.15.33:0/2839574592 conn(0x7fce0806d030 msgr2=0x7fce080c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 shutdown_connections 2023-10-06T05:24:08.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.050+0000 7fce0e99d700 1 -- 172.21.15.33:0/2839574592 wait complete. 2023-10-06T05:24:08.054 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 96 2023-10-06T05:24:08.070 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:24:08.070 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 4.9361982345581055s 2023-10-06T05:24:08.070 INFO:tasks.cephfs.mount:check lock on file background_file 2023-10-06T05:24:08.070 DEBUG:teuthology.orchestra.run.smithi167:> python3 -c ' 2023-10-06T05:24:08.070 DEBUG:teuthology.orchestra.run.smithi167:> import fcntl 2023-10-06T05:24:08.070 DEBUG:teuthology.orchestra.run.smithi167:> import errno 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> import struct 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> f1 = open("/home/ubuntu/cephtest/mnt.1/background_file-1", '"'"'r'"'"') 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.flock(f1, fcntl.LOCK_EX | fcntl.LOCK_NB) 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> if e.errno == errno.EAGAIN: 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> else: 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("flock on file /home/ubuntu/cephtest/mnt.1/background_file-1 not found") 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> f2 = open("/home/ubuntu/cephtest/mnt.1/background_file-2", '"'"'r'"'"') 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:24:08.071 DEBUG:teuthology.orchestra.run.smithi167:> lockdata = struct.pack('"'"'hhllhh'"'"', fcntl.F_WRLCK, 0, 0, 0, 0, 0) 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.fcntl(f2, fcntl.F_SETLK, lockdata) 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> if e.errno == errno.EAGAIN: 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> else: 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("posix lock on file /home/ubuntu/cephtest/mnt.1/background_file-2 not found") 2023-10-06T05:24:08.072 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:24:08.156 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:24:08.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:08.280 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:08.367 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:24:08.368 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:24:08.368 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:24:08.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:08.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:08.451 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:24:08.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:08.452 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:08.489 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:24:08.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:08.489 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:08.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:24:08.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:08.574 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:24:08.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.795+0000 7f1d5499d700 1 Processor -- start 2023-10-06T05:24:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.795+0000 7f1d5499d700 1 -- start start 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d5499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d500c8b50 0x7f1d500c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d5499d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5006ab60 con 0x7f1d50155ec0 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d5499d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5006ae40 con 0x7f1d50155af0 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d5499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5006b7e0 con 0x7f1d500c8b50 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d4ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1d50155ec0 0x7f1d500c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34808/0 (socket says 172.21.15.33:34808) 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d4ffff700 1 -- 172.21.15.33:0/272303600 learned_addr learned my addr 172.21.15.33:0/272303600 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.796+0000 7f1d4effd700 1 --2- 172.21.15.33:0/272303600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d500c8b50 0x7f1d500c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589167631 0 0) 0x7f1d5006ab60 con 0x7f1d50155ec0 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d38003570 con 0x7f1d50155ec0 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138155002 0 0) 0x7f1d5006ae40 con 0x7f1d50155af0 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5006ab60 con 0x7f1d50155af0 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4effd700 1 -- 172.21.15.33:0/272303600 >> v1:172.21.15.167:6789/0 conn(0x7f1d50155af0 legacy=0x7f1d5006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4effd700 1 -- 172.21.15.33:0/272303600 >> v1:172.21.15.33:6789/0 conn(0x7f1d50155ec0 legacy=0x7f1d500c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4effd700 1 -- 172.21.15.33:0/272303600 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5006b120 con 0x7f1d500c8b50 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.797+0000 7f1d4effd700 1 --2- 172.21.15.33:0/272303600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d500c8b50 0x7f1d500c7210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009910 tx=0x7f1d440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7659cd302a31de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44011630 con 0x7f1d500c8b50 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44011790 con 0x7f1d500c8b50 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d4e7fc700 1 -- 172.21.15.33:0/272303600 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44011b30 con 0x7f1d500c8b50 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 -- 172.21.15.33:0/272303600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d500c8b50 msgr2=0x7f1d500c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 --2- 172.21.15.33:0/272303600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d500c8b50 0x7f1d500c7210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009910 tx=0x7f1d440091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 -- 172.21.15.33:0/272303600 shutdown_connections 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 --2- 172.21.15.33:0/272303600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d500c8b50 0x7f1d500c7210 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 -- 172.21.15.33:0/272303600 >> 172.21.15.33:0/272303600 conn(0x7f1d5006d030 msgr2=0x7f1d5006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 -- 172.21.15.33:0/272303600 shutdown_connections 2023-10-06T05:24:08.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.798+0000 7f1d5499d700 1 -- 172.21.15.33:0/272303600 wait complete. 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.800+0000 7f1d5499d700 1 Processor -- start 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.800+0000 7f1d5499d700 1 -- start start 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d5499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d500c8b50 0x7f1d50158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d5499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 0x7f1d501589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d5499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d50155ec0 0x7f1d500aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d5499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d500b12d0 con 0x7f1d50155ec0 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d5499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50159670 con 0x7f1d500c8b50 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d5499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50142f20 con 0x7f1d50155af0 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d4ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d50155ec0 0x7f1d500aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d4effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 0x7f1d501589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d4effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 0x7f1d501589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35802/0 (socket says 172.21.15.33:35802) 2023-10-06T05:24:08.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d500c8b50 0x7f1d50158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.801+0000 7f1d4effd700 1 -- 172.21.15.33:0/1315160979 learned_addr learned my addr 172.21.15.33:0/1315160979 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4effd700 1 -- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d500c8b50 msgr2=0x7f1d50158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4effd700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d500c8b50 0x7f1d50158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4effd700 1 -- 172.21.15.33:0/1315160979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d50155ec0 msgr2=0x7f1d500aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4effd700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d50155ec0 0x7f1d500aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4effd700 1 -- 172.21.15.33:0/1315160979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009420 con 0x7f1d50155af0 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4ffff700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d50155ec0 0x7f1d500aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d4effd700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 0x7f1d501589c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d500635d0 tx=0x7f1d44017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4401f070 con 0x7f1d50155af0 2023-10-06T05:24:08.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.802+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44002f70 con 0x7f1d50155af0 2023-10-06T05:24:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.803+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44011b30 con 0x7f1d50155af0 2023-10-06T05:24:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.803+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d501590b0 con 0x7f1d50155af0 2023-10-06T05:24:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.803+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d50142bb0 con 0x7f1d50155af0 2023-10-06T05:24:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.804+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d50143090 con 0x7f1d50155af0 2023-10-06T05:24:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.805+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d440193b0 con 0x7f1d50155af0 2023-10-06T05:24:08.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.805+0000 7f1d377fe700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d3c066f30 0x7f1d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.805+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1d44085940 con 0x7f1d50155af0 2023-10-06T05:24:08.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.806+0000 7f1d4f7fe700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d3c066f30 0x7f1d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.806+0000 7f1d4f7fe700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d3c066f30 0x7f1d3c0693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1d40007fd0 tx=0x7f1d40007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:08.811+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d44052980 con 0x7f1d50155af0 2023-10-06T05:24:09.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.059+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d5004c380 con 0x7f1d50155af0 2023-10-06T05:24:09.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.060+0000 7f1d377fe700 1 -- 172.21.15.33:0/1315160979 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f1d44028020 con 0x7f1d50155af0 2023-10-06T05:24:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:09.063 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":96,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:06.792632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d3c066f30 msgr2=0x7f1d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d3c066f30 0x7f1d3c0693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1d40007fd0 tx=0x7f1d40007d50 comp rx=0 tx=0).stop 2023-10-06T05:24:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 msgr2=0x7f1d501589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 0x7f1d501589c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d500635d0 tx=0x7f1d44017000 comp rx=0 tx=0).stop 2023-10-06T05:24:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 shutdown_connections 2023-10-06T05:24:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d3c066f30 0x7f1d3c0693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d50155ec0 0x7f1d500aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d500c8b50 0x7f1d50158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 --2- 172.21.15.33:0/1315160979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d50155af0 0x7f1d501589c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 >> 172.21.15.33:0/1315160979 conn(0x7f1d5006d030 msgr2=0x7f1d500cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 shutdown_connections 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.062+0000 7f1d5499d700 1 -- 172.21.15.33:0/1315160979 wait complete. 2023-10-06T05:24:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 96 2023-10-06T05:24:09.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:09.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.266+0000 7f89d359e700 1 Processor -- start 2023-10-06T05:24:09.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.266+0000 7f89d359e700 1 -- start start 2023-10-06T05:24:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89d40c8050 0x7f89d413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d406aad0 0x7f89d406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d40c9cb0 con 0x7f89d413faa0 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d40ca270 con 0x7f89d40c8050 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d406b840 con 0x7f89d406aad0 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f89d413faa0 0x7f89d40c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34818/0 (socket says 172.21.15.33:34818) 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d259c700 1 -- 172.21.15.33:0/1244794350 learned_addr learned my addr 172.21.15.33:0/1244794350 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d1d9b700 1 --2- 172.21.15.33:0/1244794350 >> v2:172.21.15.167:3300/0 conn(0x7f89d40c8050 0x7f89d413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d2d9d700 1 --2- 172.21.15.33:0/1244794350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d406aad0 0x7f89d406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d159a700 1 -- 172.21.15.33:0/1244794350 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641765308 0 0) 0x7f89d40c9cb0 con 0x7f89d413faa0 2023-10-06T05:24:09.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.268+0000 7f89d159a700 1 -- 172.21.15.33:0/1244794350 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89bc003570 con 0x7f89d413faa0 2023-10-06T05:24:09.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d2d9d700 1 -- 172.21.15.33:0/1244794350 >> v2:172.21.15.167:3300/0 conn(0x7f89d40c8050 msgr2=0x7f89d413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:09.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d2d9d700 1 --2- 172.21.15.33:0/1244794350 >> v2:172.21.15.167:3300/0 conn(0x7f89d40c8050 0x7f89d413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d2d9d700 1 -- 172.21.15.33:0/1244794350 >> v1:172.21.15.33:6789/0 conn(0x7f89d413faa0 legacy=0x7f89d40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d2d9d700 1 -- 172.21.15.33:0/1244794350 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d406bb20 con 0x7f89d406aad0 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d2d9d700 1 --2- 172.21.15.33:0/1244794350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d406aad0 0x7f89d406af50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009e60 tx=0x7f89cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2090f23715d8cd9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d159a700 1 -- 172.21.15.33:0/1244794350 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc00e070 con 0x7f89d406aad0 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d159a700 1 -- 172.21.15.33:0/1244794350 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc00bab0 con 0x7f89d406aad0 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.269+0000 7f89d159a700 1 -- 172.21.15.33:0/1244794350 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc00be70 con 0x7f89d406aad0 2023-10-06T05:24:09.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 -- 172.21.15.33:0/1244794350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d406aad0 msgr2=0x7f89d406af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 --2- 172.21.15.33:0/1244794350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d406aad0 0x7f89d406af50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009e60 tx=0x7f89cc009780 comp rx=0 tx=0).stop 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 -- 172.21.15.33:0/1244794350 shutdown_connections 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 --2- 172.21.15.33:0/1244794350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d406aad0 0x7f89d406af50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 --2- 172.21.15.33:0/1244794350 >> v2:172.21.15.167:3300/0 conn(0x7f89d40c8050 0x7f89d413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 -- 172.21.15.33:0/1244794350 >> 172.21.15.33:0/1244794350 conn(0x7f89d406d030 msgr2=0x7f89d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 -- 172.21.15.33:0/1244794350 shutdown_connections 2023-10-06T05:24:09.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.270+0000 7f89d359e700 1 -- 172.21.15.33:0/1244794350 wait complete. 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.271+0000 7f89d359e700 1 Processor -- start 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.272+0000 7f89d359e700 1 -- start start 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.272+0000 7f89d359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d406aad0 0x7f89d413d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.272+0000 7f89d359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d40c8050 0x7f89d40c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 0x7f89d40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d416a0e0 con 0x7f89d413faa0 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41697f0 con 0x7f89d406aad0 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4169a90 con 0x7f89d40c8050 2023-10-06T05:24:09.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 0x7f89d40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 0x7f89d40c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51192/0 (socket says 172.21.15.33:51192) 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d406aad0 0x7f89d413d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 -- 172.21.15.33:0/2882467953 learned_addr learned my addr 172.21.15.33:0/2882467953 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d1d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d40c8050 0x7f89d40c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 -- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d40c8050 msgr2=0x7f89d40c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d40c8050 0x7f89d40c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 -- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d406aad0 msgr2=0x7f89d413d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d406aad0 0x7f89d413d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.273+0000 7f89d2d9d700 1 -- 172.21.15.33:0/2882467953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc009a20 con 0x7f89d413faa0 2023-10-06T05:24:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.274+0000 7f89d2d9d700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 0x7f89d40c8b50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f89d406b980 tx=0x7f89cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.274+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc012e50 con 0x7f89d413faa0 2023-10-06T05:24:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.274+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc027470 con 0x7f89d413faa0 2023-10-06T05:24:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.274+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d4169d70 con 0x7f89d413faa0 2023-10-06T05:24:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.274+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc027830 con 0x7f89d413faa0 2023-10-06T05:24:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.274+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d4158f60 con 0x7f89d413faa0 2023-10-06T05:24:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.275+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d40cb610 con 0x7f89d413faa0 2023-10-06T05:24:09.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.276+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89cc024070 con 0x7f89d413faa0 2023-10-06T05:24:09.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.277+0000 7f89c27fc700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89b4066f30 0x7f89b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.277+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f89cc086e20 con 0x7f89d413faa0 2023-10-06T05:24:09.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.277+0000 7f89d259c700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89b4066f30 0x7f89b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.278+0000 7f89d259c700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89b4066f30 0x7f89b40693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89c4007fd0 tx=0x7f89c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.281+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89cc053e60 con 0x7f89d413faa0 2023-10-06T05:24:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.483+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89d404c380 con 0x7f89d413faa0 2023-10-06T05:24:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.484+0000 7f89c27fc700 1 -- 172.21.15.33:0/2882467953 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17007 (secure 0 0 0) 0x7f89cc057480 con 0x7f89d413faa0 2023-10-06T05:24:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:09.487 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":133,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:03.044950+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:48.724864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:49.540442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/117155768":"2023-10-07T05:24:03.044929+0000","172.21.15.33:6833/117155768":"2023-10-07T05:24:03.044929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89b4066f30 msgr2=0x7f89b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89b4066f30 0x7f89b40693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89c4007fd0 tx=0x7f89c4009040 comp rx=0 tx=0).stop 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 msgr2=0x7f89d40c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 0x7f89d40c8b50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f89d406b980 tx=0x7f89cc019000 comp rx=0 tx=0).stop 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 shutdown_connections 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f89b4066f30 0x7f89b40693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89d413faa0 0x7f89d40c8b50 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89d406aad0 0x7f89d413d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 --2- 172.21.15.33:0/2882467953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89d40c8050 0x7f89d40c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 >> 172.21.15.33:0/2882467953 conn(0x7f89d406d030 msgr2=0x7f89d40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 shutdown_connections 2023-10-06T05:24:09.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.486+0000 7f89d359e700 1 -- 172.21.15.33:0/2882467953 wait complete. 2023-10-06T05:24:09.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:23:48.724864+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:23:49.540442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:24:09.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:09.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.695+0000 7fdd3219d700 1 Processor -- start 2023-10-06T05:24:09.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.695+0000 7fdd3219d700 1 -- start start 2023-10-06T05:24:09.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3219d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdd2c0b1f90 0x7fdd2c0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3219d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd2c070440 0x7fdd2c067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3219d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c06ae10 con 0x7fdd2c068140 2023-10-06T05:24:09.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3219d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c06a2d0 con 0x7fdd2c070440 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3219d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c154290 con 0x7fdd2c0b1f90 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd2c070440 0x7fdd2c067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd2c070440 0x7fdd2c067c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51204/0 (socket says 172.21.15.33:51204) 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3099a700 1 -- 172.21.15.33:0/894542425 learned_addr learned my addr 172.21.15.33:0/894542425 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.697+0000 7fdd3119b700 1 --2- 172.21.15.33:0/894542425 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c0b1f90 0x7fdd2c0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd3099a700 1 -- 172.21.15.33:0/894542425 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c0b1f90 msgr2=0x7fdd2c0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd3099a700 1 --2- 172.21.15.33:0/894542425 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c0b1f90 0x7fdd2c0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd3099a700 1 -- 172.21.15.33:0/894542425 >> v1:172.21.15.33:6789/0 conn(0x7fdd2c068140 legacy=0x7fdd2c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd3099a700 1 -- 172.21.15.33:0/894542425 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c068580 con 0x7fdd2c070440 2023-10-06T05:24:09.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd3099a700 1 --2- 172.21.15.33:0/894542425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c070440 0x7fdd2c067c00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c069b60 tx=0x7fdd1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2f352f19dec5df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd23fff700 1 -- 172.21.15.33:0/894542425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c009e70 con 0x7fdd2c070440 2023-10-06T05:24:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd23fff700 1 -- 172.21.15.33:0/894542425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c00b860 con 0x7fdd2c070440 2023-10-06T05:24:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.698+0000 7fdd23fff700 1 -- 172.21.15.33:0/894542425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00bc00 con 0x7fdd2c070440 2023-10-06T05:24:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 -- 172.21.15.33:0/894542425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c070440 msgr2=0x7fdd2c067c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 --2- 172.21.15.33:0/894542425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c070440 0x7fdd2c067c00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c069b60 tx=0x7fdd1c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:09.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 -- 172.21.15.33:0/894542425 shutdown_connections 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 --2- 172.21.15.33:0/894542425 >> v2:172.21.15.167:3300/0 conn(0x7fdd2c0b1f90 0x7fdd2c0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 --2- 172.21.15.33:0/894542425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c070440 0x7fdd2c067c00 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 -- 172.21.15.33:0/894542425 >> 172.21.15.33:0/894542425 conn(0x7fdd2c06ea70 msgr2=0x7fdd2c06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 -- 172.21.15.33:0/894542425 shutdown_connections 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.699+0000 7fdd3219d700 1 -- 172.21.15.33:0/894542425 wait complete. 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.700+0000 7fdd3219d700 1 Processor -- start 2023-10-06T05:24:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.700+0000 7fdd3219d700 1 -- start start 2023-10-06T05:24:09.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.701+0000 7fdd3219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 0x7fdd2c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.701+0000 7fdd3219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c070440 0x7fdd2c157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.701+0000 7fdd3219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c0b1f90 0x7fdd2c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.701+0000 7fdd3219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c15dd30 con 0x7fdd2c068140 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.701+0000 7fdd3219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c15df40 con 0x7fdd2c070440 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.701+0000 7fdd3219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c15e220 con 0x7fdd2c0b1f90 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 0x7fdd2c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 0x7fdd2c158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51214/0 (socket says 172.21.15.33:51214) 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 -- 172.21.15.33:0/3753888835 learned_addr learned my addr 172.21.15.33:0/3753888835 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3099a700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c070440 0x7fdd2c157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3199c700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c0b1f90 0x7fdd2c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 -- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c0b1f90 msgr2=0x7fdd2c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c0b1f90 0x7fdd2c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 -- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c070440 msgr2=0x7fdd2c157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c070440 0x7fdd2c157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 -- 172.21.15.33:0/3753888835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0094f0 con 0x7fdd2c068140 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3199c700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c0b1f90 0x7fdd2c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd3119b700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 0x7fdd2c158460 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800e930 tx=0x7fdd280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28013680 con 0x7fdd2c068140 2023-10-06T05:24:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd280137e0 con 0x7fdd2c068140 2023-10-06T05:24:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.702+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28013b80 con 0x7fdd2c068140 2023-10-06T05:24:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.703+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c157980 con 0x7fdd2c068140 2023-10-06T05:24:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.703+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c157f40 con 0x7fdd2c068140 2023-10-06T05:24:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.704+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c158130 con 0x7fdd2c068140 2023-10-06T05:24:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.704+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd2800c460 con 0x7fdd2c068140 2023-10-06T05:24:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.705+0000 7fdd217fa700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdd10066f30 0x7fdd100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.705+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fdd28084850 con 0x7fdd2c068140 2023-10-06T05:24:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.706+0000 7fdd3099a700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdd10066f30 0x7fdd100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.707+0000 7fdd3099a700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdd10066f30 0x7fdd100693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c069b60 tx=0x7fdd1c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.710+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd28051910 con 0x7fdd2c068140 2023-10-06T05:24:09.781 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:24:09.778+0000 7fe4bbbf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:09.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.913+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd2c04c380 con 0x7fdd2c068140 2023-10-06T05:24:09.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.914+0000 7fdd217fa700 1 -- 172.21.15.33:0/3753888835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd28054f30 con 0x7fdd2c068140 2023-10-06T05:24:09.916 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdd10066f30 msgr2=0x7fdd100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdd10066f30 0x7fdd100693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c069b60 tx=0x7fdd1c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 msgr2=0x7fdd2c158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 0x7fdd2c158460 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800e930 tx=0x7fdd280091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 shutdown_connections 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdd10066f30 0x7fdd100693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd2c0b1f90 0x7fdd2c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd2c068140 0x7fdd2c158460 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 --2- 172.21.15.33:0/3753888835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2c070440 0x7fdd2c157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 >> 172.21.15.33:0/3753888835 conn(0x7fdd2c06ea70 msgr2=0x7fdd2c06d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 shutdown_connections 2023-10-06T05:24:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:09.916+0000 7fdd3219d700 1 -- 172.21.15.33:0/3753888835 wait complete. 2023-10-06T05:24:09.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:10.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.124+0000 7f03a579d700 1 Processor -- start 2023-10-06T05:24:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.124+0000 7f03a579d700 1 -- start start 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.125+0000 7f03a579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a0156370 0x7f03a00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.125+0000 7f03a579d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03a00b01f0 0x7f03a00b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.125+0000 7f03a579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a00ad430 con 0x7f03a00cc7e0 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.125+0000 7f03a579d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00afe20 con 0x7f03a00b01f0 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.125+0000 7f03a579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00b0b70 con 0x7f03a0156370 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03a00b01f0 0x7f03a00b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03a00b01f0 0x7f03a00b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34598/0 (socket says 172.21.15.33:34598) 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 -- 172.21.15.33:0/213775488 learned_addr learned my addr 172.21.15.33:0/213775488 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f039ffff700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a0156370 0x7f03a00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 -- 172.21.15.33:0/213775488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a0156370 msgr2=0x7f03a00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a0156370 0x7f03a00af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 -- 172.21.15.33:0/213775488 >> v1:172.21.15.33:6789/0 conn(0x7f03a00cc7e0 legacy=0x7f03a00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 -- 172.21.15.33:0/213775488 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a006bc70 con 0x7f03a00b01f0 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f039ffff700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a0156370 0x7f03a00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.126+0000 7f03a4f9c700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a00b0630 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0398009f20 tx=0x7f0398009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe570de6914b10cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f039effd700 1 -- 172.21.15.33:0/213775488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800e070 con 0x7f03a00b01f0 2023-10-06T05:24:10.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f039effd700 1 -- 172.21.15.33:0/213775488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039800ba80 con 0x7f03a00b01f0 2023-10-06T05:24:10.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f039effd700 1 -- 172.21.15.33:0/213775488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980092e0 con 0x7f03a00b01f0 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 -- 172.21.15.33:0/213775488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 msgr2=0x7f03a00b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a00b0630 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0398009f20 tx=0x7f0398009840 comp rx=0 tx=0).stop 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 -- 172.21.15.33:0/213775488 shutdown_connections 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a00b0630 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 --2- 172.21.15.33:0/213775488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a0156370 0x7f03a00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 -- 172.21.15.33:0/213775488 >> 172.21.15.33:0/213775488 conn(0x7f03a006d030 msgr2=0x7f03a00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 -- 172.21.15.33:0/213775488 shutdown_connections 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.127+0000 7f03a579d700 1 -- 172.21.15.33:0/213775488 wait complete. 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.129+0000 7f03a579d700 1 Processor -- start 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.129+0000 7f03a579d700 1 -- start start 2023-10-06T05:24:10.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a0151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a00cc7e0 0x7f03a0158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a0156370 0x7f03a00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a01699d0 con 0x7f03a00b01f0 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0169f50 con 0x7f03a0156370 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0151b80 con 0x7f03a00cc7e0 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f039ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a0151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f039ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a0151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34608/0 (socket says 172.21.15.33:34608) 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f039ffff700 1 -- 172.21.15.33:0/4099902661 learned_addr learned my addr 172.21.15.33:0/4099902661 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f039f7fe700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a00cc7e0 0x7f03a0158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.130+0000 7f03a4f9c700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a0156370 0x7f03a00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039ffff700 1 -- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a00cc7e0 msgr2=0x7f03a0158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039ffff700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a00cc7e0 0x7f03a0158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039ffff700 1 -- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a0156370 msgr2=0x7f03a00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039ffff700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a0156370 0x7f03a00a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039ffff700 1 -- 172.21.15.33:0/4099902661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398009ae0 con 0x7f03a00b01f0 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039f7fe700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a00cc7e0 0x7f03a0158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f03a4f9c700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a0156370 0x7f03a00a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f039ffff700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a0151640 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f039000e900 tx=0x7f0390008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000eea0 con 0x7f03a00b01f0 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000c5e0 con 0x7f03a00b01f0 2023-10-06T05:24:10.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000c980 con 0x7f03a00b01f0 2023-10-06T05:24:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a01591a0 con 0x7f03a00b01f0 2023-10-06T05:24:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.131+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a0159440 con 0x7f03a00b01f0 2023-10-06T05:24:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.132+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a0159660 con 0x7f03a00b01f0 2023-10-06T05:24:10.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.133+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f039000cd30 con 0x7f03a00b01f0 2023-10-06T05:24:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.134+0000 7f0387fff700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f038c066f30 0x7f038c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.135+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f03900845b0 con 0x7f03a00b01f0 2023-10-06T05:24:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.135+0000 7f039f7fe700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f038c066f30 0x7f038c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.136+0000 7f039f7fe700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f038c066f30 0x7f038c0693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0394007fd0 tx=0x7f0394009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.139+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0390055080 con 0x7f03a00b01f0 2023-10-06T05:24:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.342+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03a004c380 con 0x7f03a00b01f0 2023-10-06T05:24:10.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.342+0000 7f0387fff700 1 -- 172.21.15.33:0/4099902661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0390054c10 con 0x7f03a00b01f0 2023-10-06T05:24:10.345 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:24:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.344+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f038c066f30 msgr2=0x7f038c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.344+0000 7f03a579d700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f038c066f30 0x7f038c0693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0394007fd0 tx=0x7f0394009040 comp rx=0 tx=0).stop 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.344+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 msgr2=0x7f03a0151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.344+0000 7f03a579d700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a0151640 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f039000e900 tx=0x7f0390008040 comp rx=0 tx=0).stop 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 shutdown_connections 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f038c066f30 0x7f038c0693e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03a0156370 0x7f03a00a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03a00b01f0 0x7f03a0151640 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 --2- 172.21.15.33:0/4099902661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03a00cc7e0 0x7f03a0158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 >> 172.21.15.33:0/4099902661 conn(0x7f03a006d030 msgr2=0x7f03a00ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 shutdown_connections 2023-10-06T05:24:10.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.345+0000 7f03a579d700 1 -- 172.21.15.33:0/4099902661 wait complete. 2023-10-06T05:24:10.365 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:24:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.552+0000 7f721d79d700 1 Processor -- start 2023-10-06T05:24:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.552+0000 7f721d79d700 1 -- start start 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f721d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7218156370 0x7f72180af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f721d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72180cc7e0 0x7f72180c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f721d79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72180ad430 con 0x7f72180b01f0 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f721d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72180afe20 con 0x7f7218156370 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f721d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721806bc70 con 0x7f72180cc7e0 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7218156370 0x7f72180af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7218156370 0x7f72180af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34620/0 (socket says 172.21.15.33:34620) 2023-10-06T05:24:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 -- 172.21.15.33:0/1979210791 learned_addr learned my addr 172.21.15.33:0/1979210791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f72177fe700 1 --2- 172.21.15.33:0/1979210791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72180cc7e0 0x7f72180c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 -- 172.21.15.33:0/1979210791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72180cc7e0 msgr2=0x7f72180c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 --2- 172.21.15.33:0/1979210791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72180cc7e0 0x7f72180c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 -- 172.21.15.33:0/1979210791 >> v1:172.21.15.167:6789/0 conn(0x7f72180b01f0 legacy=0x7f72180ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.554+0000 7f7217fff700 1 -- 172.21.15.33:0/1979210791 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180c92f0 con 0x7f7218156370 2023-10-06T05:24:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.555+0000 7f7217fff700 1 --2- 172.21.15.33:0/1979210791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7218156370 0x7f72180af8e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f720800bc50 tx=0x7f720800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5915b8db5de123ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.555+0000 7f7216ffd700 1 -- 172.21.15.33:0/1979210791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720800be60 con 0x7f7218156370 2023-10-06T05:24:10.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.555+0000 7f7216ffd700 1 -- 172.21.15.33:0/1979210791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720800de00 con 0x7f7218156370 2023-10-06T05:24:10.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.555+0000 7f7216ffd700 1 -- 172.21.15.33:0/1979210791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72080196b0 con 0x7f7218156370 2023-10-06T05:24:10.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 -- 172.21.15.33:0/1979210791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7218156370 msgr2=0x7f72180af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 --2- 172.21.15.33:0/1979210791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7218156370 0x7f72180af8e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f720800bc50 tx=0x7f720800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 -- 172.21.15.33:0/1979210791 shutdown_connections 2023-10-06T05:24:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 --2- 172.21.15.33:0/1979210791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7218156370 0x7f72180af8e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 --2- 172.21.15.33:0/1979210791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72180cc7e0 0x7f72180c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 -- 172.21.15.33:0/1979210791 >> 172.21.15.33:0/1979210791 conn(0x7f721806d030 msgr2=0x7f72180cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 -- 172.21.15.33:0/1979210791 shutdown_connections 2023-10-06T05:24:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.556+0000 7f721d79d700 1 -- 172.21.15.33:0/1979210791 wait complete. 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.557+0000 7f721d79d700 1 Processor -- start 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.557+0000 7f721d79d700 1 -- start start 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f721d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72180b01f0 0x7f72181589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f721d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 0x7f7218150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f721d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7218156370 0x7f72180ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f721d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721816d3f0 con 0x7f72180cc7e0 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f721d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72181529e0 con 0x7f72180b01f0 2023-10-06T05:24:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f721d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7218152cc0 con 0x7f7218156370 2023-10-06T05:24:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.558+0000 7f72177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 0x7f7218150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 0x7f7218150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34634/0 (socket says 172.21.15.33:34634) 2023-10-06T05:24:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 -- 172.21.15.33:0/4014480923 learned_addr learned my addr 172.21.15.33:0/4014480923 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 -- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7218156370 msgr2=0x7f72180ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7218156370 0x7f72180ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 -- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72180b01f0 msgr2=0x7f72181589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72180b01f0 0x7f72181589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 -- 172.21.15.33:0/4014480923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720800b790 con 0x7f72180cc7e0 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f72177fe700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 0x7f7218150e00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f720c00d890 tx=0x7f720c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0125f0 con 0x7f72180cc7e0 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c012770 con 0x7f72180cc7e0 2023-10-06T05:24:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c012b30 con 0x7f72180cc7e0 2023-10-06T05:24:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.559+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7218152fa0 con 0x7f72180cc7e0 2023-10-06T05:24:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.560+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7218153280 con 0x7f72180cc7e0 2023-10-06T05:24:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.561+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721804c380 con 0x7f72180cc7e0 2023-10-06T05:24:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.562+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f720c01e460 con 0x7f72180cc7e0 2023-10-06T05:24:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.563+0000 7f71fffff700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7204066f30 0x7f72040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.563+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f720c084d30 con 0x7f72180cc7e0 2023-10-06T05:24:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.563+0000 7f7217fff700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7204066f30 0x7f72040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.564+0000 7f7217fff700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7204066f30 0x7f72040693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f720800bc50 tx=0x7f720801b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.567+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f720c051d70 con 0x7f72180cc7e0 2023-10-06T05:24:10.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.771+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f721816d530 con 0x7f72180cc7e0 2023-10-06T05:24:10.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.771+0000 7f71fffff700 1 -- 172.21.15.33:0/4014480923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f720c055390 con 0x7f72180cc7e0 2023-10-06T05:24:10.774 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:24:10.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.773+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7204066f30 msgr2=0x7f72040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.773+0000 7f721d79d700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7204066f30 0x7f72040693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f720800bc50 tx=0x7f720801b000 comp rx=0 tx=0).stop 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 msgr2=0x7f7218150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 0x7f7218150e00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f720c00d890 tx=0x7f720c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 shutdown_connections 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7204066f30 0x7f72040693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7218156370 0x7f72180ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72180b01f0 0x7f72181589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 --2- 172.21.15.33:0/4014480923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72180cc7e0 0x7f7218150e00 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 >> 172.21.15.33:0/4014480923 conn(0x7f721806d030 msgr2=0x7f72180abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 shutdown_connections 2023-10-06T05:24:10.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.774+0000 7f721d79d700 1 -- 172.21.15.33:0/4014480923 wait complete. 2023-10-06T05:24:10.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:24:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.983+0000 7f85a0d9d700 1 Processor -- start 2023-10-06T05:24:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.983+0000 7f85a0d9d700 1 -- start start 2023-10-06T05:24:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.984+0000 7f85a0d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f859c0af540 0x7f859c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.984+0000 7f85a0d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f859c15a650 0x7f859c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.984+0000 7f85a0d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c0cced0 0x7f859c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.984+0000 7f85a0d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c0cd890 con 0x7f859c0af540 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.984+0000 7f85a0d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c15d660 con 0x7f859c15a650 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.984+0000 7f85a0d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0ce890 con 0x7f859c0cced0 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f859c0af540 0x7f859c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f859c0af540 0x7f859c0af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34648/0 (socket says 172.21.15.33:34648) 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 -- 172.21.15.33:0/3122260386 learned_addr learned my addr 172.21.15.33:0/3122260386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 -- 172.21.15.33:0/3122260386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c0cced0 msgr2=0x7f859c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 --2- 172.21.15.33:0/3122260386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c0cced0 0x7f859c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 -- 172.21.15.33:0/3122260386 >> v2:172.21.15.167:3300/0 conn(0x7f859c15a650 msgr2=0x7f859c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 --2- 172.21.15.33:0/3122260386 >> v2:172.21.15.167:3300/0 conn(0x7f859c15a650 0x7f859c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 -- 172.21.15.33:0/3122260386 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c15aac0 con 0x7f859c0af540 2023-10-06T05:24:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.985+0000 7f859b7fe700 1 --2- 172.21.15.33:0/3122260386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c0af910 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f859c0893f0 tx=0x7f858c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b97d7d8ad8e664 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f859a7fc700 1 -- 172.21.15.33:0/3122260386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c00b820 con 0x7f859c0af540 2023-10-06T05:24:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f859a7fc700 1 -- 172.21.15.33:0/3122260386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c00b980 con 0x7f859c0af540 2023-10-06T05:24:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f859a7fc700 1 -- 172.21.15.33:0/3122260386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c00bd20 con 0x7f859c0af540 2023-10-06T05:24:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3122260386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 msgr2=0x7f859c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3122260386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c0af910 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f859c0893f0 tx=0x7f858c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3122260386 shutdown_connections 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3122260386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c0cced0 0x7f859c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3122260386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c0af910 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3122260386 >> v2:172.21.15.167:3300/0 conn(0x7f859c15a650 0x7f859c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3122260386 >> 172.21.15.33:0/3122260386 conn(0x7f859c06d9b0 msgr2=0x7f859c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3122260386 shutdown_connections 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.986+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3122260386 wait complete. 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.988+0000 7f85a0d9d700 1 Processor -- start 2023-10-06T05:24:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.988+0000 7f85a0d9d700 1 -- start start 2023-10-06T05:24:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f85a0d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f85a0d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859c0cced0 0x7f859c0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f85a0d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c15a650 0x7f859c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f85a0d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c16ae70 con 0x7f859c0af540 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f85a0d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c157740 con 0x7f859c0cced0 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f85a0d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0b4780 con 0x7f859c15a650 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34662/0 (socket says 172.21.15.33:34662) 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 -- 172.21.15.33:0/3438770829 learned_addr learned my addr 172.21.15.33:0/3438770829 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859affd700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859c0cced0 0x7f859c0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859bfff700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c15a650 0x7f859c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 -- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c15a650 msgr2=0x7f859c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c15a650 0x7f859c0d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 -- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859c0cced0 msgr2=0x7f859c0b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859c0cced0 0x7f859c0b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.989+0000 7f859b7fe700 1 -- 172.21.15.33:0/3438770829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0094f0 con 0x7f859c0af540 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.990+0000 7f859b7fe700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c157200 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f858c009290 tx=0x7f858c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.990+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c026480 con 0x7f859c0af540 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.990+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c01f070 con 0x7f859c0af540 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.990+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c026850 con 0x7f859c0af540 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.990+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c16a260 con 0x7f859c0af540 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.990+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c16a820 con 0x7f859c0af540 2023-10-06T05:24:10.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.991+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c16d640 con 0x7f859c0af540 2023-10-06T05:24:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.992+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f858c026bc0 con 0x7f859c0af540 2023-10-06T05:24:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.993+0000 7f85837fe700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8588066f30 0x7f85880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.993+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f858c021070 con 0x7f859c0af540 2023-10-06T05:24:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.993+0000 7f859affd700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8588066f30 0x7f85880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.994+0000 7f859affd700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8588066f30 0x7f85880693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f859c0c6500 tx=0x7f8590008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:10.997+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f858c0536b0 con 0x7f859c0af540 2023-10-06T05:24:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.197+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f859c04c380 con 0x7f859c0af540 2023-10-06T05:24:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.197+0000 7f85837fe700 1 -- 172.21.15.33:0/3438770829 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7f858c056cd0 con 0x7f859c0af540 2023-10-06T05:24:11.200 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:24:11.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8588066f30 msgr2=0x7f85880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8588066f30 0x7f85880693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f859c0c6500 tx=0x7f8590008040 comp rx=0 tx=0).stop 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 msgr2=0x7f859c157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c157200 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f858c009290 tx=0x7f858c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 shutdown_connections 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8588066f30 0x7f85880693e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f859c15a650 0x7f859c0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f859c0af540 0x7f859c157200 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 --2- 172.21.15.33:0/3438770829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f859c0cced0 0x7f859c0b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 >> 172.21.15.33:0/3438770829 conn(0x7f859c06d9b0 msgr2=0x7f859c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 shutdown_connections 2023-10-06T05:24:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.200+0000 7f85a0d9d700 1 -- 172.21.15.33:0/3438770829 wait complete. 2023-10-06T05:24:11.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.406+0000 7fe27a79d700 1 Processor -- start 2023-10-06T05:24:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.406+0000 7fe27a79d700 1 -- start start 2023-10-06T05:24:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.407+0000 7fe27a79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2740cf3b0 0x7fe2741594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.407+0000 7fe27a79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740af0a0 0x7fe2741599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.407+0000 7fe27a79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740cbd80 con 0x7fe2740ccfc0 2023-10-06T05:24:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.407+0000 7fe27a79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2740af5f0 con 0x7fe2740cf3b0 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.407+0000 7fe27a79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740af8d0 con 0x7fe2740af0a0 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2740cf3b0 0x7fe2741594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2740cf3b0 0x7fe2741594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34670/0 (socket says 172.21.15.33:34670) 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 -- 172.21.15.33:0/2377597679 learned_addr learned my addr 172.21.15.33:0/2377597679 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe279f9c700 1 --2- 172.21.15.33:0/2377597679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740af0a0 0x7fe2741599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 -- 172.21.15.33:0/2377597679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740af0a0 msgr2=0x7fe2741599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 --2- 172.21.15.33:0/2377597679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740af0a0 0x7fe2741599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 -- 172.21.15.33:0/2377597679 >> v1:172.21.15.33:6789/0 conn(0x7fe2740ccfc0 legacy=0x7fe2740cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 -- 172.21.15.33:0/2377597679 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740afbb0 con 0x7fe2740cf3b0 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.408+0000 7fe278f9a700 1 --2- 172.21.15.33:0/2377597679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2740cf3b0 0x7fe2741594a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe2640099e0 tx=0x7fe264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaace7a0d085d8bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe26bfff700 1 -- 172.21.15.33:0/2377597679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26400b820 con 0x7fe2740cf3b0 2023-10-06T05:24:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe26bfff700 1 -- 172.21.15.33:0/2377597679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26400b980 con 0x7fe2740cf3b0 2023-10-06T05:24:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe26bfff700 1 -- 172.21.15.33:0/2377597679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26400bd20 con 0x7fe2740cf3b0 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 -- 172.21.15.33:0/2377597679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2740cf3b0 msgr2=0x7fe2741594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 --2- 172.21.15.33:0/2377597679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2740cf3b0 0x7fe2741594a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe2640099e0 tx=0x7fe264009250 comp rx=0 tx=0).stop 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 -- 172.21.15.33:0/2377597679 shutdown_connections 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 --2- 172.21.15.33:0/2377597679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740af0a0 0x7fe2741599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 --2- 172.21.15.33:0/2377597679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2740cf3b0 0x7fe2741594a0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 -- 172.21.15.33:0/2377597679 >> 172.21.15.33:0/2377597679 conn(0x7fe27406cff0 msgr2=0x7fe27406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 -- 172.21.15.33:0/2377597679 shutdown_connections 2023-10-06T05:24:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.409+0000 7fe27a79d700 1 -- 172.21.15.33:0/2377597679 wait complete. 2023-10-06T05:24:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.411+0000 7fe27a79d700 1 Processor -- start 2023-10-06T05:24:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.411+0000 7fe27a79d700 1 -- start start 2023-10-06T05:24:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27a79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2740af0a0 0x7fe274148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27a79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740ccfc0 0x7fe2740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27a79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 0x7fe2740b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27a79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740c75b0 con 0x7fe274169bc0 2023-10-06T05:24:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27a79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740c7b70 con 0x7fe2740af0a0 2023-10-06T05:24:11.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27a79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740b3fd0 con 0x7fe2740ccfc0 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe279f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 0x7fe2740b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe279f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 0x7fe2740b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34684/0 (socket says 172.21.15.33:34684) 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe279f9c700 1 -- 172.21.15.33:0/3773829393 learned_addr learned my addr 172.21.15.33:0/3773829393 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe278f9a700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740ccfc0 0x7fe2740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe27979b700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2740af0a0 0x7fe274148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.412+0000 7fe279f9c700 1 -- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740ccfc0 msgr2=0x7fe2740b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe279f9c700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740ccfc0 0x7fe2740b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe279f9c700 1 -- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2740af0a0 msgr2=0x7fe274148120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe279f9c700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2740af0a0 0x7fe274148120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe279f9c700 1 -- 172.21.15.33:0/3773829393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640094f0 con 0x7fe274169bc0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe278f9a700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740ccfc0 0x7fe2740b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe279f9c700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 0x7fe2740b5360 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe26c007f20 tx=0x7fe26c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00ec40 con 0x7fe274169bc0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c00eda0 con 0x7fe274169bc0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2740b42b0 con 0x7fe274169bc0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c014670 con 0x7fe274169bc0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.413+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe274148660 con 0x7fe274169bc0 2023-10-06T05:24:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.414+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2740c7d00 con 0x7fe274169bc0 2023-10-06T05:24:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.415+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe26c014aa0 con 0x7fe274169bc0 2023-10-06T05:24:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.416+0000 7fe2697fa700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe258066f30 0x7fe2580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.416+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe26c084dd0 con 0x7fe274169bc0 2023-10-06T05:24:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.416+0000 7fe27979b700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe258066f30 0x7fe2580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.417+0000 7fe27979b700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe258066f30 0x7fe2580693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe270003860 tx=0x7fe270009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.420+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe26c051e10 con 0x7fe274169bc0 2023-10-06T05:24:11.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.667+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe27404c380 con 0x7fe274169bc0 2023-10-06T05:24:11.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.668+0000 7fe2697fa700 1 -- 172.21.15.33:0/3773829393 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fe26c055430 con 0x7fe274169bc0 2023-10-06T05:24:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:11.670 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":96,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:06.792632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe258066f30 msgr2=0x7fe2580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe258066f30 0x7fe2580693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe270003860 tx=0x7fe270009040 comp rx=0 tx=0).stop 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 msgr2=0x7fe2740b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 0x7fe2740b5360 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe26c007f20 tx=0x7fe26c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 shutdown_connections 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe258066f30 0x7fe2580693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe274169bc0 0x7fe2740b5360 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2740af0a0 0x7fe274148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 --2- 172.21.15.33:0/3773829393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2740ccfc0 0x7fe2740b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 >> 172.21.15.33:0/3773829393 conn(0x7fe27406cff0 msgr2=0x7fe274155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 shutdown_connections 2023-10-06T05:24:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.670+0000 7fe27a79d700 1 -- 172.21.15.33:0/3773829393 wait complete. 2023-10-06T05:24:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 96 2023-10-06T05:24:11.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:11.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.875+0000 7f9a2099d700 1 Processor -- start 2023-10-06T05:24:11.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.875+0000 7f9a2099d700 1 -- start start 2023-10-06T05:24:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.876+0000 7f9a2099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c0ccfc0 0x7f9a1c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.876+0000 7f9a2099d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9a1c0cc750 0x7f9a1c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.876+0000 7f9a2099d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c0cbd80 con 0x7f9a1c0cf3b0 2023-10-06T05:24:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.876+0000 7f9a2099d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c0af5f0 con 0x7f9a1c0cc750 2023-10-06T05:24:11.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.876+0000 7f9a2099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0af8d0 con 0x7f9a1c0ccfc0 2023-10-06T05:24:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9a1c0cc750 0x7f9a1c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1b7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c0ccfc0 0x7f9a1c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1affd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9a1c0cf3b0 0x7f9a1c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:50766/0 (socket says 172.21.15.33:50766) 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1affd700 1 -- 172.21.15.33:0/2771605805 learned_addr learned my addr 172.21.15.33:0/2771605805 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1bfff700 1 -- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c0ccfc0 msgr2=0x7f9a1c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1bfff700 1 --2- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c0ccfc0 0x7f9a1c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1bfff700 1 -- 172.21.15.33:0/2771605805 >> v1:172.21.15.167:6789/0 conn(0x7f9a1c0cf3b0 legacy=0x7f9a1c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1bfff700 1 -- 172.21.15.33:0/2771605805 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0afbb0 con 0x7f9a1c0cc750 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1b7fe700 1 --2- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c0ccfc0 0x7f9a1c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.877+0000 7f9a1a7fc700 1 -- 172.21.15.33:0/2771605805 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920910545 0 0) 0x7f9a1c0cbd80 con 0x7f9a1c0cf3b0 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.878+0000 7f9a1bfff700 1 --2- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c0b0460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009e60 tx=0x7f9a14009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcd780bcdd5533d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.878+0000 7f9a1a7fc700 1 -- 172.21.15.33:0/2771605805 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400e070 con 0x7f9a1c0cc750 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.878+0000 7f9a1a7fc700 1 -- 172.21.15.33:0/2771605805 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1400bb60 con 0x7f9a1c0cc750 2023-10-06T05:24:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.878+0000 7f9a1a7fc700 1 -- 172.21.15.33:0/2771605805 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14012430 con 0x7f9a1c0cc750 2023-10-06T05:24:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.878+0000 7f9a2099d700 1 -- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cc750 msgr2=0x7f9a1c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.878+0000 7f9a2099d700 1 --2- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c0b0460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009e60 tx=0x7f9a14009780 comp rx=0 tx=0).stop 2023-10-06T05:24:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.879+0000 7f9a2099d700 1 -- 172.21.15.33:0/2771605805 shutdown_connections 2023-10-06T05:24:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.879+0000 7f9a2099d700 1 --2- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c0b0460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.879+0000 7f9a2099d700 1 --2- 172.21.15.33:0/2771605805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c0ccfc0 0x7f9a1c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.879+0000 7f9a2099d700 1 -- 172.21.15.33:0/2771605805 >> 172.21.15.33:0/2771605805 conn(0x7f9a1c06cff0 msgr2=0x7f9a1c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.879+0000 7f9a2099d700 1 -- 172.21.15.33:0/2771605805 shutdown_connections 2023-10-06T05:24:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.879+0000 7f9a2099d700 1 -- 172.21.15.33:0/2771605805 wait complete. 2023-10-06T05:24:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.880+0000 7f9a2099d700 1 Processor -- start 2023-10-06T05:24:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.880+0000 7f9a2099d700 1 -- start start 2023-10-06T05:24:11.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a2099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a2099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0ccfc0 0x7f9a1c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a2099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c16d850 0x7f9a1c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a2099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c153fd0 con 0x7f9a1c0cc750 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a2099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c153cf0 con 0x7f9a1c0ccfc0 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a2099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c16cd70 con 0x7f9a1c16d850 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34690/0 (socket says 172.21.15.33:34690) 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.881+0000 7f9a1b7fe700 1 -- 172.21.15.33:0/4285115035 learned_addr learned my addr 172.21.15.33:0/4285115035 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1bfff700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c16d850 0x7f9a1c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1affd700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0ccfc0 0x7f9a1c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1b7fe700 1 -- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c16d850 msgr2=0x7f9a1c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1b7fe700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c16d850 0x7f9a1c0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1b7fe700 1 -- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0ccfc0 msgr2=0x7f9a1c153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1b7fe700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0ccfc0 0x7f9a1c153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1b7fe700 1 -- 172.21.15.33:0/4285115035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009420 con 0x7f9a1c0cc750 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a1b7fe700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c154190 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ec80 tx=0x7f9a0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00c9d0 con 0x7f9a1c0cc750 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00cb30 con 0x7f9a1c0cc750 2023-10-06T05:24:11.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a14009a20 con 0x7f9a1c0cc750 2023-10-06T05:24:11.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00ceb0 con 0x7f9a1c0cc750 2023-10-06T05:24:11.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.882+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c16d330 con 0x7f9a1c0cc750 2023-10-06T05:24:11.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.883+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c04c380 con 0x7f9a1c0cc750 2023-10-06T05:24:11.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.884+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a0c013680 con 0x7f9a1c0cc750 2023-10-06T05:24:11.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.885+0000 7f9a037fe700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a08066f30 0x7f9a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.885+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9a0c084bc0 con 0x7f9a1c0cc750 2023-10-06T05:24:11.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.886+0000 7f9a1affd700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a08066f30 0x7f9a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.887+0000 7f9a1affd700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a08066f30 0x7f9a080693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0b4730 tx=0x7f9a10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:11.889+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a0c051c00 con 0x7f9a1c0cc750 2023-10-06T05:24:12.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.092+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a1c0b07a0 con 0x7f9a1c0cc750 2023-10-06T05:24:12.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.093+0000 7f9a037fe700 1 -- 172.21.15.33:0/4285115035 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17007 (secure 0 0 0) 0x7f9a0c055220 con 0x7f9a1c0cc750 2023-10-06T05:24:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:12.096 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":133,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:03.044950+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:48.724864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:49.540442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/117155768":"2023-10-07T05:24:03.044929+0000","172.21.15.33:6833/117155768":"2023-10-07T05:24:03.044929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a08066f30 msgr2=0x7f9a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a08066f30 0x7f9a080693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0b4730 tx=0x7f9a10009040 comp rx=0 tx=0).stop 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 msgr2=0x7f9a1c154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c154190 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00ec80 tx=0x7f9a0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 shutdown_connections 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a08066f30 0x7f9a080693e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a1c16d850 0x7f9a1c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a1c0cc750 0x7f9a1c154190 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 --2- 172.21.15.33:0/4285115035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0ccfc0 0x7f9a1c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 >> 172.21.15.33:0/4285115035 conn(0x7f9a1c06cff0 msgr2=0x7f9a1c0c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 shutdown_connections 2023-10-06T05:24:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.096+0000 7f9a2099d700 1 -- 172.21.15.33:0/4285115035 wait complete. 2023-10-06T05:24:12.116 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:24:12.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:12.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.300+0000 7f949d19d700 1 Processor -- start 2023-10-06T05:24:12.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.300+0000 7f949d19d700 1 -- start start 2023-10-06T05:24:12.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f949d19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9498155af0 0x7f949806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f949d19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f949806a6a0 0x7f94980c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f949d19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949806ab60 con 0x7f94980c8b50 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f949d19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949806b7e0 con 0x7f9498155af0 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f949d19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949806ae40 con 0x7f949806a6a0 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9498155af0 0x7f949806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9498155af0 0x7f949806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34706/0 (socket says 172.21.15.33:34706) 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 -- 172.21.15.33:0/1263532029 learned_addr learned my addr 172.21.15.33:0/1263532029 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 -- 172.21.15.33:0/1263532029 >> v2:172.21.15.167:3300/0 conn(0x7f949806a6a0 msgr2=0x7f94980c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 --2- 172.21.15.33:0/1263532029 >> v2:172.21.15.167:3300/0 conn(0x7f949806a6a0 0x7f94980c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 -- 172.21.15.33:0/1263532029 >> v1:172.21.15.33:6789/0 conn(0x7f94980c8b50 legacy=0x7f94981598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.302+0000 7f9497fff700 1 -- 172.21.15.33:0/1263532029 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949806b120 con 0x7f9498155af0 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f9497fff700 1 --2- 172.21.15.33:0/1263532029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9498155af0 0x7f949806a160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f949806f250 tx=0x7f948800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42b25b088c4ec1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f9496ffd700 1 -- 172.21.15.33:0/1263532029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800dc50 con 0x7f9498155af0 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f9496ffd700 1 -- 172.21.15.33:0/1263532029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948800ddb0 con 0x7f9498155af0 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f9496ffd700 1 -- 172.21.15.33:0/1263532029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880196d0 con 0x7f9498155af0 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f949d19d700 1 -- 172.21.15.33:0/1263532029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9498155af0 msgr2=0x7f949806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f949d19d700 1 --2- 172.21.15.33:0/1263532029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9498155af0 0x7f949806a160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f949806f250 tx=0x7f948800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f949d19d700 1 -- 172.21.15.33:0/1263532029 shutdown_connections 2023-10-06T05:24:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f949d19d700 1 --2- 172.21.15.33:0/1263532029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9498155af0 0x7f949806a160 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f949d19d700 1 --2- 172.21.15.33:0/1263532029 >> v2:172.21.15.167:3300/0 conn(0x7f949806a6a0 0x7f94980c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.303+0000 7f949d19d700 1 -- 172.21.15.33:0/1263532029 >> 172.21.15.33:0/1263532029 conn(0x7f949806d030 msgr2=0x7f949806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.304+0000 7f949d19d700 1 -- 172.21.15.33:0/1263532029 shutdown_connections 2023-10-06T05:24:12.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.304+0000 7f949d19d700 1 -- 172.21.15.33:0/1263532029 wait complete. 2023-10-06T05:24:12.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.305+0000 7f949d19d700 1 Processor -- start 2023-10-06T05:24:12.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.305+0000 7f949d19d700 1 -- start start 2023-10-06T05:24:12.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949d19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 0x7f94980c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949d19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94980c8b50 0x7f94980b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949d19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9498155af0 0x7f94980afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949d19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94980b04e0 con 0x7f949806a6a0 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949d19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94980b07c0 con 0x7f94980c8b50 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949d19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94980b0aa0 con 0x7f9498155af0 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f9497fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 0x7f94980c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f9497fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 0x7f94980c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34720/0 (socket says 172.21.15.33:34720) 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f9497fff700 1 -- 172.21.15.33:0/3456317179 learned_addr learned my addr 172.21.15.33:0/3456317179 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f949c99c700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9498155af0 0x7f94980afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.306+0000 7f94977fe700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94980c8b50 0x7f94980b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f9497fff700 1 -- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9498155af0 msgr2=0x7f94980afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f9497fff700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9498155af0 0x7f94980afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f9497fff700 1 -- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94980c8b50 msgr2=0x7f94980b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f9497fff700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94980c8b50 0x7f94980b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f9497fff700 1 -- 172.21.15.33:0/3456317179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948800b790 con 0x7f949806a6a0 2023-10-06T05:24:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f94977fe700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94980c8b50 0x7f94980b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f949c99c700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9498155af0 0x7f94980afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f9497fff700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 0x7f94980c7310 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f94980635d0 tx=0x7f948801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880107d0 con 0x7f949806a6a0 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488010930 con 0x7f949806a6a0 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488010cd0 con 0x7f949806a6a0 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94980b0d80 con 0x7f949806a6a0 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.307+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94980c3ac0 con 0x7f949806a6a0 2023-10-06T05:24:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.308+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949804c380 con 0x7f949806a6a0 2023-10-06T05:24:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.309+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f94880183e0 con 0x7f949806a6a0 2023-10-06T05:24:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.310+0000 7f947ffff700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9484066f60 0x7f9484069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.310+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9488014070 con 0x7f949806a6a0 2023-10-06T05:24:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.310+0000 7f94977fe700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9484066f60 0x7f9484069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.311+0000 7f94977fe700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9484066f60 0x7f9484069410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f94980b1fb0 tx=0x7f948c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.314+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9488055960 con 0x7f949806a6a0 2023-10-06T05:24:12.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.559+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f949806b400 con 0x7f949806a6a0 2023-10-06T05:24:12.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.559+0000 7f947ffff700 1 -- 172.21.15.33:0/3456317179 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f9488028090 con 0x7f949806a6a0 2023-10-06T05:24:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:12.563 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":96,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:06.792632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9484066f60 msgr2=0x7f9484069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9484066f60 0x7f9484069410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f94980b1fb0 tx=0x7f948c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 msgr2=0x7f94980c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 0x7f94980c7310 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f94980635d0 tx=0x7f948801b000 comp rx=0 tx=0).stop 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 shutdown_connections 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9484066f60 0x7f9484069410 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9498155af0 0x7f94980afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f949806a6a0 0x7f94980c7310 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 --2- 172.21.15.33:0/3456317179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94980c8b50 0x7f94980b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 >> 172.21.15.33:0/3456317179 conn(0x7f949806d030 msgr2=0x7f94980c1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 shutdown_connections 2023-10-06T05:24:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.562+0000 7f949d19d700 1 -- 172.21.15.33:0/3456317179 wait complete. 2023-10-06T05:24:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 96 2023-10-06T05:24:12.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.766+0000 7ff02759e700 1 Processor -- start 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.766+0000 7ff02759e700 1 -- start start 2023-10-06T05:24:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff02813faa0 0x7ff02813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0280c7940 0x7ff0280c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0280c9cb0 con 0x7ff02806aad0 2023-10-06T05:24:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02806b660 con 0x7ff02813faa0 2023-10-06T05:24:12.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280ca270 con 0x7ff0280c7940 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff02813faa0 0x7ff02813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff02813faa0 0x7ff02813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34726/0 (socket says 172.21.15.33:34726) 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff02659c700 1 -- 172.21.15.33:0/1763171295 learned_addr learned my addr 172.21.15.33:0/1763171295 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.768+0000 7ff025d9b700 1 --2- 172.21.15.33:0/1763171295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0280c7940 0x7ff0280c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02659c700 1 -- 172.21.15.33:0/1763171295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0280c7940 msgr2=0x7ff0280c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02659c700 1 --2- 172.21.15.33:0/1763171295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0280c7940 0x7ff0280c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02659c700 1 -- 172.21.15.33:0/1763171295 >> v1:172.21.15.167:6789/0 conn(0x7ff02806aad0 legacy=0x7ff02806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02659c700 1 -- 172.21.15.33:0/1763171295 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02806b840 con 0x7ff02813faa0 2023-10-06T05:24:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02659c700 1 --2- 172.21.15.33:0/1763171295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff02813fe70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ff0180099b0 tx=0x7ff018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f824f78ca7a6e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02559a700 1 -- 172.21.15.33:0/1763171295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01800b860 con 0x7ff02813faa0 2023-10-06T05:24:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02559a700 1 -- 172.21.15.33:0/1763171295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01800b9c0 con 0x7ff02813faa0 2023-10-06T05:24:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.769+0000 7ff02559a700 1 -- 172.21.15.33:0/1763171295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01800bd60 con 0x7ff02813faa0 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 -- 172.21.15.33:0/1763171295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 msgr2=0x7ff02813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 --2- 172.21.15.33:0/1763171295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff02813fe70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ff0180099b0 tx=0x7ff018009250 comp rx=0 tx=0).stop 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 -- 172.21.15.33:0/1763171295 shutdown_connections 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 --2- 172.21.15.33:0/1763171295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff02813fe70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 --2- 172.21.15.33:0/1763171295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0280c7940 0x7ff0280c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 -- 172.21.15.33:0/1763171295 >> 172.21.15.33:0/1763171295 conn(0x7ff02806d030 msgr2=0x7ff02806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 -- 172.21.15.33:0/1763171295 shutdown_connections 2023-10-06T05:24:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.770+0000 7ff02759e700 1 -- 172.21.15.33:0/1763171295 wait complete. 2023-10-06T05:24:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.771+0000 7ff02759e700 1 Processor -- start 2023-10-06T05:24:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 -- start start 2023-10-06T05:24:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff02806aad0 0x7ff02815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0280c7940 0x7ff028156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff0281565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02813eae0 con 0x7ff02813faa0 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02813ded0 con 0x7ff0280c7940 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.772+0000 7ff02759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02813e1b0 con 0x7ff02806aad0 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff0281565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff0281565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34742/0 (socket says 172.21.15.33:34742) 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 -- 172.21.15.33:0/1857647378 learned_addr learned my addr 172.21.15.33:0/1857647378 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff025d9b700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0280c7940 0x7ff028156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff02659c700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff02806aad0 0x7ff02815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 -- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff02806aad0 msgr2=0x7ff02815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff02806aad0 0x7ff02815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 -- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0280c7940 msgr2=0x7ff028156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0280c7940 0x7ff028156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff026d9d700 1 -- 172.21.15.33:0/1857647378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0180094f0 con 0x7ff02813faa0 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.773+0000 7ff025d9b700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0280c7940 0x7ff028156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.774+0000 7ff026d9d700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff0281565b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c8e0 tx=0x7ff020001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.774+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020012630 con 0x7ff02813faa0 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.774+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff020012790 con 0x7ff02813faa0 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.774+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02813e490 con 0x7ff02813faa0 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.774+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020012b50 con 0x7ff02813faa0 2023-10-06T05:24:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.774+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02813ef70 con 0x7ff02813faa0 2023-10-06T05:24:12.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.775+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02813ec90 con 0x7ff02813faa0 2023-10-06T05:24:12.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.776+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff02000a3b0 con 0x7ff02813faa0 2023-10-06T05:24:12.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.777+0000 7ff0167fc700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff008066f30 0x7ff0080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.777+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff0200849e0 con 0x7ff02813faa0 2023-10-06T05:24:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.777+0000 7ff02659c700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff008066f30 0x7ff0080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.778+0000 7ff02659c700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff008066f30 0x7ff0080693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0180099b0 tx=0x7ff018017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:12.781+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff02813ec90 con 0x7ff02813faa0 2023-10-06T05:24:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.028+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff02804c380 con 0x7ff02813faa0 2023-10-06T05:24:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.028+0000 7ff0167fc700 1 -- 172.21.15.33:0/1857647378 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff020051a20 con 0x7ff02813faa0 2023-10-06T05:24:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:13.031 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":96,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:50.563828+0000","modified":"2023-10-06T05:24:06.792632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6835/2422185514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2422185514},{"type":"v1","addr":"172.21.15.33:6835","nonce":2422185514}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff008066f30 msgr2=0x7ff0080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff008066f30 0x7ff0080693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0180099b0 tx=0x7ff018017000 comp rx=0 tx=0).stop 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 msgr2=0x7ff0281565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff0281565b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c8e0 tx=0x7ff020001710 comp rx=0 tx=0).stop 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 shutdown_connections 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff008066f30 0x7ff0080693e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff02813faa0 0x7ff0281565b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff02806aad0 0x7ff02815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 --2- 172.21.15.33:0/1857647378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0280c7940 0x7ff028156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 >> 172.21.15.33:0/1857647378 conn(0x7ff02806d030 msgr2=0x7ff02806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 shutdown_connections 2023-10-06T05:24:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.031+0000 7ff02759e700 1 -- 172.21.15.33:0/1857647378 wait complete. 2023-10-06T05:24:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 96 2023-10-06T05:24:13.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:13.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.236+0000 7fdab939d700 1 Processor -- start 2023-10-06T05:24:13.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.237+0000 7fdab939d700 1 -- start start 2023-10-06T05:24:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdab4155af0 0x7fdab406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab939d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdab40c7920 0x7fdab40c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab406ab60 con 0x7fdab406a6a0 2023-10-06T05:24:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab406b7e0 con 0x7fdab4155af0 2023-10-06T05:24:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab939d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab406ae40 con 0x7fdab40c7920 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdab4155af0 0x7fdab406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdab4155af0 0x7fdab406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34746/0 (socket says 172.21.15.33:34746) 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab3fff700 1 -- 172.21.15.33:0/3533683316 learned_addr learned my addr 172.21.15.33:0/3533683316 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.238+0000 7fdab8b9c700 1 --2- 172.21.15.33:0/3533683316 >> v2:172.21.15.167:3300/0 conn(0x7fdab40c7920 0x7fdab40c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab3fff700 1 -- 172.21.15.33:0/3533683316 >> v2:172.21.15.167:3300/0 conn(0x7fdab40c7920 msgr2=0x7fdab40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab3fff700 1 --2- 172.21.15.33:0/3533683316 >> v2:172.21.15.167:3300/0 conn(0x7fdab40c7920 0x7fdab40c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab3fff700 1 -- 172.21.15.33:0/3533683316 >> v1:172.21.15.167:6789/0 conn(0x7fdab406a6a0 legacy=0x7fdab40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab3fff700 1 -- 172.21.15.33:0/3533683316 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab406b120 con 0x7fdab4155af0 2023-10-06T05:24:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab3fff700 1 --2- 172.21.15.33:0/3533683316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab406a160 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdab4070fb0 tx=0x7fdaa400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9202e15bae295843 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab2ffd700 1 -- 172.21.15.33:0/3533683316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400ca00 con 0x7fdab4155af0 2023-10-06T05:24:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab2ffd700 1 -- 172.21.15.33:0/3533683316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa400cb60 con 0x7fdab4155af0 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.239+0000 7fdab2ffd700 1 -- 172.21.15.33:0/3533683316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40183f0 con 0x7fdab4155af0 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 -- 172.21.15.33:0/3533683316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 msgr2=0x7fdab406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 --2- 172.21.15.33:0/3533683316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab406a160 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdab4070fb0 tx=0x7fdaa400a3f0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 -- 172.21.15.33:0/3533683316 shutdown_connections 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 --2- 172.21.15.33:0/3533683316 >> v2:172.21.15.167:3300/0 conn(0x7fdab40c7920 0x7fdab40c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 --2- 172.21.15.33:0/3533683316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab406a160 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 -- 172.21.15.33:0/3533683316 >> 172.21.15.33:0/3533683316 conn(0x7fdab406d030 msgr2=0x7fdab406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 -- 172.21.15.33:0/3533683316 shutdown_connections 2023-10-06T05:24:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.240+0000 7fdab939d700 1 -- 172.21.15.33:0/3533683316 wait complete. 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.241+0000 7fdab939d700 1 Processor -- start 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 -- start start 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdab406a6a0 0x7fdab4146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdab40c7920 0x7fdab4157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40ae7a0 con 0x7fdab4155af0 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4147300 con 0x7fdab40c7920 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.242+0000 7fdab939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40aca00 con 0x7fdab406a6a0 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab41584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34754/0 (socket says 172.21.15.33:34754) 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 -- 172.21.15.33:0/2121201640 learned_addr learned my addr 172.21.15.33:0/2121201640 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab37fe700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdab40c7920 0x7fdab4157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab3fff700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdab406a6a0 0x7fdab4146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 -- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdab406a6a0 msgr2=0x7fdab4146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdab406a6a0 0x7fdab4146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 -- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdab40c7920 msgr2=0x7fdab4157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdab40c7920 0x7fdab4157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 -- 172.21.15.33:0/2121201640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa400a690 con 0x7fdab4155af0 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.243+0000 7fdab8b9c700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab41584a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdaac007f20 tx=0x7fdaac0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.244+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00eca0 con 0x7fdab4155af0 2023-10-06T05:24:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.244+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac00ee20 con 0x7fdab4155af0 2023-10-06T05:24:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.244+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac014670 con 0x7fdab4155af0 2023-10-06T05:24:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.244+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab40acce0 con 0x7fdab4155af0 2023-10-06T05:24:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.244+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab40ad2a0 con 0x7fdab4155af0 2023-10-06T05:24:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.245+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab40682b0 con 0x7fdab4155af0 2023-10-06T05:24:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.246+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdaac014a60 con 0x7fdab4155af0 2023-10-06T05:24:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.246+0000 7fda9bfff700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaa0066f30 0x7fdaa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.247+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fdaac084e90 con 0x7fdab4155af0 2023-10-06T05:24:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.247+0000 7fdab3fff700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaa0066f30 0x7fdaa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.248+0000 7fdab3fff700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaa0066f30 0x7fdaa00693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdab4070fb0 tx=0x7fdaa4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.250+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdaac051ed0 con 0x7fdab4155af0 2023-10-06T05:24:13.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.455+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdab404c380 con 0x7fdab4155af0 2023-10-06T05:24:13.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.456+0000 7fda9bfff700 1 -- 172.21.15.33:0/2121201640 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17007 (secure 0 0 0) 0x7fdaac0554f0 con 0x7fdab4155af0 2023-10-06T05:24:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:13.459 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":133,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:03.044950+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:48.724864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:49.540442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/117155768":"2023-10-07T05:24:03.044929+0000","172.21.15.33:6833/117155768":"2023-10-07T05:24:03.044929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaa0066f30 msgr2=0x7fdaa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaa0066f30 0x7fdaa00693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdab4070fb0 tx=0x7fdaa4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 msgr2=0x7fdab41584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab41584a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdaac007f20 tx=0x7fdaac0098a0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 shutdown_connections 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaa0066f30 0x7fdaa00693e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdab4155af0 0x7fdab41584a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdab406a6a0 0x7fdab4146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 --2- 172.21.15.33:0/2121201640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdab40c7920 0x7fdab4157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 >> 172.21.15.33:0/2121201640 conn(0x7fdab406d030 msgr2=0x7fdab40c13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 shutdown_connections 2023-10-06T05:24:13.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.458+0000 7fdab939d700 1 -- 172.21.15.33:0/2121201640 wait complete. 2023-10-06T05:24:13.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.665+0000 7f2d7279d700 1 Processor -- start 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.666+0000 7f2d7279d700 1 -- start start 2023-10-06T05:24:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.667+0000 7f2d7279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2d6c06a0d0 0x7f2d6c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.667+0000 7f2d7279d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c15bcb0 con 0x7f2d6c140d20 2023-10-06T05:24:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.667+0000 7f2d7279d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c1410f0 con 0x7f2d6c0c93e0 2023-10-06T05:24:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.667+0000 7f2d7279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c15b0a0 con 0x7f2d6c06a0d0 2023-10-06T05:24:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.667+0000 7f2d7179b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2d6c140d20 0x7f2d6c0c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60474/0 (socket says 172.21.15.33:60474) 2023-10-06T05:24:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.667+0000 7f2d7179b700 1 -- 172.21.15.33:0/516980723 learned_addr learned my addr 172.21.15.33:0/516980723 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:13.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d71f9c700 1 --2- 172.21.15.33:0/516980723 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c06a0d0 0x7f2d6c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951759066 0 0) 0x7f2d6c15bcb0 con 0x7f2d6c140d20 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003570 con 0x7f2d6c140d20 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655700368 0 0) 0x7f2d6c1410f0 con 0x7f2d6c0c93e0 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c15bcb0 con 0x7f2d6c0c93e0 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143573713 0 0) 0x7f2d58003570 con 0x7f2d6c140d20 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c06a0d0 msgr2=0x7f2d6c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 --2- 172.21.15.33:0/516980723 >> v2:172.21.15.167:3300/0 conn(0x7f2d6c06a0d0 0x7f2d6c1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 >> v1:172.21.15.167:6789/0 conn(0x7f2d6c0c93e0 legacy=0x7f2d6c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c15b380 con 0x7f2d6c140d20 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f2d68001470 con 0x7f2d6c140d20 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 >> v1:172.21.15.33:6789/0 conn(0x7f2d6c140d20 legacy=0x7f2d6c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d58003bf0 0x7f2d58003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d580051c0 0x7f2d5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c15b0a0 con 0x7f2d580051c0 2023-10-06T05:24:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800c0b0 con 0x7f2d58003bf0 2023-10-06T05:24:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.668+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800c270 con 0x7f2d58004810 2023-10-06T05:24:13.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d7179b700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d58003bf0 0x7f2d58003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d580051c0 0x7f2d5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d70f9a700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 -- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d58004810 msgr2=0x7f2d58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 -- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d58003bf0 msgr2=0x7f2d58003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d58003bf0 0x7f2d58003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 -- 172.21.15.33:0/516980723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58003570 con 0x7f2d580051c0 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 -- 172.21.15.33:0/516980723 reap_dead start 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d71f9c700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d580051c0 0x7f2d5800ba60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c06a4d0 tx=0x7f2d6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b837f19a60d0b050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.669+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400ea90 con 0x7f2d580051c0 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400f040 con 0x7f2d580051c0 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d63fff700 1 -- 172.21.15.33:0/516980723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400ee40 con 0x7f2d580051c0 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 -- 172.21.15.33:0/516980723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d580051c0 msgr2=0x7f2d5800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d580051c0 0x7f2d5800ba60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c06a4d0 tx=0x7f2d6400b040 comp rx=0 tx=0).stop 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 -- 172.21.15.33:0/516980723 shutdown_connections 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 --2- 172.21.15.33:0/516980723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d580051c0 0x7f2d5800ba60 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 -- 172.21.15.33:0/516980723 >> 172.21.15.33:0/516980723 conn(0x7f2d6c06c560 msgr2=0x7f2d6c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 -- 172.21.15.33:0/516980723 shutdown_connections 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.670+0000 7f2d7279d700 1 -- 172.21.15.33:0/516980723 wait complete. 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.672+0000 7f2d7279d700 1 Processor -- start 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.672+0000 7f2d7279d700 1 -- start start 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d580051c0 0x7f2d6c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 0x7f2d6c13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c1589f0 0x7f2d6c13ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c4100 con 0x7f2d6c158580 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c4970 con 0x7f2d580051c0 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c16acd0 con 0x7f2d6c1589f0 2023-10-06T05:24:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d70f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 0x7f2d6c13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d70f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 0x7f2d6c13e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34762/0 (socket says 172.21.15.33:34762) 2023-10-06T05:24:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d70f9a700 1 -- 172.21.15.33:0/33137613 learned_addr learned my addr 172.21.15.33:0/33137613 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d7179b700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d580051c0 0x7f2d6c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.673+0000 7f2d71f9c700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c1589f0 0x7f2d6c13ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d70f9a700 1 -- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c1589f0 msgr2=0x7f2d6c13ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d70f9a700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c1589f0 0x7f2d6c13ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d70f9a700 1 -- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d580051c0 msgr2=0x7f2d6c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d70f9a700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d580051c0 0x7f2d6c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d70f9a700 1 -- 172.21.15.33:0/33137613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0c4310 con 0x7f2d6c158580 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d70f9a700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 0x7f2d6c13e9f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c002b80 tx=0x7f2d5c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00f8d0 con 0x7f2d6c158580 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c00fa50 con 0x7f2d6c158580 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00fe10 con 0x7f2d6c158580 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0c45f0 con 0x7f2d6c158580 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.674+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c16a930 con 0x7f2d6c158580 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.675+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0c4b00 con 0x7f2d6c158580 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.676+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d5c006a30 con 0x7f2d6c158580 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.677+0000 7f2d617fa700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d50066f30 0x7f2d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.677+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f2d5c084a80 con 0x7f2d6c158580 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.677+0000 7f2d7179b700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d50066f30 0x7f2d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.678+0000 7f2d7179b700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d50066f30 0x7f2d500693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2d68001200 tx=0x7f2d68001e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.681+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d5c051ac0 con 0x7f2d6c158580 2023-10-06T05:24:13.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.934+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2d6c04c380 con 0x7f2d6c158580 2023-10-06T05:24:13.937 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:24:13.935+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:13.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.939+0000 7f2d617fa700 1 -- 172.21.15.33:0/33137613 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2d5c0550e0 con 0x7f2d6c158580 2023-10-06T05:24:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d50066f30 msgr2=0x7f2d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d50066f30 0x7f2d500693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2d68001200 tx=0x7f2d68001e10 comp rx=0 tx=0).stop 2023-10-06T05:24:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 msgr2=0x7f2d6c13e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 0x7f2d6c13e9f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c002b80 tx=0x7f2d5c007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 shutdown_connections 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d50066f30 0x7f2d500693e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6c1589f0 0x7f2d6c13ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d580051c0 0x7f2d6c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 --2- 172.21.15.33:0/33137613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c158580 0x7f2d6c13e9f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 >> 172.21.15.33:0/33137613 conn(0x7f2d6c06c560 msgr2=0x7f2d6c13fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 shutdown_connections 2023-10-06T05:24:13.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:13.942+0000 7f2d7279d700 1 -- 172.21.15.33:0/33137613 wait complete. 2023-10-06T05:24:13.947 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:13.952 INFO:tasks.ceph.mds.c.smithi033.stderr: -116> 2023-10-06T05:24:09.778+0000 7fe4bbbf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:13.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:24:13.987 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:24:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.153+0000 7f0f5ab9d700 1 Processor -- start 2023-10-06T05:24:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.154+0000 7f0f5ab9d700 1 -- start start 2023-10-06T05:24:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5ab9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0f54156370 0x7f0f540af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5ab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f540b01f0 0x7f0f540b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5ab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f540ad430 con 0x7f0f540cc7e0 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5ab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f540afe20 con 0x7f0f540b01f0 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5ab9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f540b0b70 con 0x7f0f54156370 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5a39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f540b01f0 0x7f0f540b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5939a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0f540cc7e0 0x7f0f540c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60492/0 (socket says 172.21.15.33:60492) 2023-10-06T05:24:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.155+0000 7f0f5939a700 1 -- 172.21.15.33:0/197565905 learned_addr learned my addr 172.21.15.33:0/197565905 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f59b9b700 1 --2- 172.21.15.33:0/197565905 >> v2:172.21.15.167:3300/0 conn(0x7f0f54156370 0x7f0f540af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f58b99700 1 -- 172.21.15.33:0/197565905 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052896654 0 0) 0x7f0f540ad430 con 0x7f0f540cc7e0 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f58b99700 1 -- 172.21.15.33:0/197565905 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f40003570 con 0x7f0f540cc7e0 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f5a39c700 1 -- 172.21.15.33:0/197565905 >> v2:172.21.15.167:3300/0 conn(0x7f0f54156370 msgr2=0x7f0f540af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f5a39c700 1 --2- 172.21.15.33:0/197565905 >> v2:172.21.15.167:3300/0 conn(0x7f0f54156370 0x7f0f540af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f5a39c700 1 -- 172.21.15.33:0/197565905 >> v1:172.21.15.33:6789/0 conn(0x7f0f540cc7e0 legacy=0x7f0f540c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f5a39c700 1 -- 172.21.15.33:0/197565905 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5406bc70 con 0x7f0f540b01f0 2023-10-06T05:24:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f5a39c700 1 --2- 172.21.15.33:0/197565905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540b01f0 0x7f0f540b0630 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0f540b21a0 tx=0x7f0f4c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfef9c47187220a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f58b99700 1 -- 172.21.15.33:0/197565905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00e070 con 0x7f0f540b01f0 2023-10-06T05:24:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f58b99700 1 -- 172.21.15.33:0/197565905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c00bb60 con 0x7f0f540b01f0 2023-10-06T05:24:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.156+0000 7f0f58b99700 1 -- 172.21.15.33:0/197565905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c012430 con 0x7f0f540b01f0 2023-10-06T05:24:14.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/197565905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540b01f0 msgr2=0x7f0f540b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/197565905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540b01f0 0x7f0f540b0630 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0f540b21a0 tx=0x7f0f4c009780 comp rx=0 tx=0).stop 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/197565905 shutdown_connections 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/197565905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540b01f0 0x7f0f540b0630 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/197565905 >> v2:172.21.15.167:3300/0 conn(0x7f0f54156370 0x7f0f540af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/197565905 >> 172.21.15.33:0/197565905 conn(0x7f0f5406d030 msgr2=0x7f0f540cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/197565905 shutdown_connections 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.157+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/197565905 wait complete. 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.158+0000 7f0f5ab9d700 1 Processor -- start 2023-10-06T05:24:14.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.159+0000 7f0f5ab9d700 1 -- start start 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.159+0000 7f0f5ab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f540b01f0 0x7f0f541439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.159+0000 7f0f5ab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 0x7f0f540ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.159+0000 7f0f5ab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f54156370 0x7f0f540ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.159+0000 7f0f5ab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f540ab4b0 con 0x7f0f540cc7e0 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5ab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f540aa8a0 con 0x7f0f540b01f0 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5ab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f540aab80 con 0x7f0f54156370 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 0x7f0f540ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 0x7f0f540ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34792/0 (socket says 172.21.15.33:34792) 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 -- 172.21.15.33:0/1125821995 learned_addr learned my addr 172.21.15.33:0/1125821995 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f59b9b700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f540b01f0 0x7f0f541439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5a39c700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f54156370 0x7f0f540ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 -- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f54156370 msgr2=0x7f0f540ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f54156370 0x7f0f540ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 -- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f540b01f0 msgr2=0x7f0f541439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f540b01f0 0x7f0f541439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 -- 172.21.15.33:0/1125821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c009a20 con 0x7f0f540cc7e0 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.160+0000 7f0f5939a700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 0x7f0f540ac040 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c7c0 tx=0x7f0f44007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:14.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.161+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44005b30 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.161+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f44005c90 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.161+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44007340 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.161+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f540aae60 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.161+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f54143f00 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.162+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5404c380 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.163+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f4401b400 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.163+0000 7f0f49ffb700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f38066f30 0x7f0f380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.164+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f0f44084290 con 0x7f0f540cc7e0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.164+0000 7f0f59b9b700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f38066f30 0x7f0f380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.165+0000 7f0f59b9b700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f38066f30 0x7f0f380693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.168+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f44051370 con 0x7f0f540cc7e0 2023-10-06T05:24:14.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.417+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0f5406f310 con 0x7f0f540cc7e0 2023-10-06T05:24:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.945+0000 7f0f49ffb700 1 -- 172.21.15.33:0/1125821995 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0f44054990 con 0x7f0f540cc7e0 2023-10-06T05:24:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f38066f30 msgr2=0x7f0f380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f38066f30 0x7f0f380693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50008040 comp rx=0 tx=0).stop 2023-10-06T05:24:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 msgr2=0x7f0f540ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 0x7f0f540ac040 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c7c0 tx=0x7f0f44007e00 comp rx=0 tx=0).stop 2023-10-06T05:24:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 shutdown_connections 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f38066f30 0x7f0f380693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f54156370 0x7f0f540ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f540b01f0 0x7f0f541439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 --2- 172.21.15.33:0/1125821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f540cc7e0 0x7f0f540ac040 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.948+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 >> 172.21.15.33:0/1125821995 conn(0x7f0f5406d030 msgr2=0x7f0f541552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.949+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 shutdown_connections 2023-10-06T05:24:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:14.949+0000 7f0f5ab9d700 1 -- 172.21.15.33:0/1125821995 wait complete. 2023-10-06T05:24:14.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:24:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.207+0000 7f38a519d700 1 Processor -- start 2023-10-06T05:24:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.208+0000 7f38a519d700 1 -- start start 2023-10-06T05:24:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f38a519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a0157320 0x7f38a00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f38a519d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38a00b1ae0 0x7f38a00ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f38a519d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00b0840 con 0x7f38a01576f0 2023-10-06T05:24:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f38a519d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a01585c0 con 0x7f38a00b1ae0 2023-10-06T05:24:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f38a519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00b1ff0 con 0x7f38a0157320 2023-10-06T05:24:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f38a499c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38a00b1ae0 0x7f38a00ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f389f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f38a01576f0 0x7f38a00b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60504/0 (socket says 172.21.15.33:60504) 2023-10-06T05:24:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.209+0000 7f389f7fe700 1 -- 172.21.15.33:0/1050859648 learned_addr learned my addr 172.21.15.33:0/1050859648 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f38a499c700 1 -- 172.21.15.33:0/1050859648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a0157320 msgr2=0x7f38a00adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f389ffff700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a0157320 0x7f38a00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f389effd700 1 -- 172.21.15.33:0/1050859648 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440165897 0 0) 0x7f38a00b0840 con 0x7f38a01576f0 2023-10-06T05:24:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f38a499c700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a0157320 0x7f38a00adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f38a499c700 1 -- 172.21.15.33:0/1050859648 >> v1:172.21.15.33:6789/0 conn(0x7f38a01576f0 legacy=0x7f38a00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f38a499c700 1 -- 172.21.15.33:0/1050859648 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0157c90 con 0x7f38a00b1ae0 2023-10-06T05:24:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f389ffff700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a0157320 0x7f38a00adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f38a499c700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00b1ae0 0x7f38a00ae450 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3898009e60 tx=0x7f3898009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495051914af618b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f389effd700 1 -- 172.21.15.33:0/1050859648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800e070 con 0x7f38a00b1ae0 2023-10-06T05:24:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f389effd700 1 -- 172.21.15.33:0/1050859648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389800bcc0 con 0x7f38a00b1ae0 2023-10-06T05:24:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.210+0000 7f389effd700 1 -- 172.21.15.33:0/1050859648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898012680 con 0x7f38a00b1ae0 2023-10-06T05:24:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 -- 172.21.15.33:0/1050859648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00b1ae0 msgr2=0x7f38a00ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00b1ae0 0x7f38a00ae450 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3898009e60 tx=0x7f3898009780 comp rx=0 tx=0).stop 2023-10-06T05:24:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 -- 172.21.15.33:0/1050859648 shutdown_connections 2023-10-06T05:24:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00b1ae0 0x7f38a00ae450 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 --2- 172.21.15.33:0/1050859648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a0157320 0x7f38a00adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 -- 172.21.15.33:0/1050859648 >> 172.21.15.33:0/1050859648 conn(0x7f38a0068090 msgr2=0x7f38a006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 -- 172.21.15.33:0/1050859648 shutdown_connections 2023-10-06T05:24:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.211+0000 7f38a519d700 1 -- 172.21.15.33:0/1050859648 wait complete. 2023-10-06T05:24:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.212+0000 7f38a519d700 1 Processor -- start 2023-10-06T05:24:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.212+0000 7f38a519d700 1 -- start start 2023-10-06T05:24:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.213+0000 7f38a519d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a00b1ae0 0x7f38a0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.213+0000 7f38a519d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 0x7f38a0151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.213+0000 7f38a519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a01576f0 0x7f38a0152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.213+0000 7f38a519d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0152860 con 0x7f38a0157320 2023-10-06T05:24:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.213+0000 7f38a519d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00ad280 con 0x7f38a00b1ae0 2023-10-06T05:24:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.213+0000 7f38a519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00ac670 con 0x7f38a01576f0 2023-10-06T05:24:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 0x7f38a0151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 0x7f38a0151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34812/0 (socket says 172.21.15.33:34812) 2023-10-06T05:24:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 -- 172.21.15.33:0/2332016961 learned_addr learned my addr 172.21.15.33:0/2332016961 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389ffff700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a00b1ae0 0x7f38a0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f38a499c700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a01576f0 0x7f38a0152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 -- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a01576f0 msgr2=0x7f38a0152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a01576f0 0x7f38a0152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 -- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a00b1ae0 msgr2=0x7f38a0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a00b1ae0 0x7f38a0155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 -- 172.21.15.33:0/2332016961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898009a20 con 0x7f38a0157320 2023-10-06T05:24:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.214+0000 7f389f7fe700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 0x7f38a0151de0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f389400c5b0 tx=0x7f3894001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.215+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894005b30 con 0x7f38a0157320 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.215+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3894005c90 con 0x7f38a0157320 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.215+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894007300 con 0x7f38a0157320 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.215+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00ac950 con 0x7f38a0157320 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.215+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00acf10 con 0x7f38a0157320 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.216+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a00ad3f0 con 0x7f38a0157320 2023-10-06T05:24:15.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.217+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f389401b3b0 con 0x7f38a0157320 2023-10-06T05:24:15.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.218+0000 7f3887fff700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f388c066f30 0x7f388c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.218+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f3894083d10 con 0x7f38a0157320 2023-10-06T05:24:15.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.218+0000 7f389ffff700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f388c066f30 0x7f388c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.219+0000 7f389ffff700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f388c066f30 0x7f388c0693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f389000ac00 tx=0x7f3890006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:15.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.222+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3894089020 con 0x7f38a0157320 2023-10-06T05:24:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.426+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38a0063ff0 con 0x7f38a0157320 2023-10-06T05:24:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.948+0000 7f3887fff700 1 -- 172.21.15.33:0/2332016961 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38940170d0 con 0x7f38a0157320 2023-10-06T05:24:15.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f388c066f30 msgr2=0x7f388c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f388c066f30 0x7f388c0693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f389000ac00 tx=0x7f3890006f90 comp rx=0 tx=0).stop 2023-10-06T05:24:15.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 msgr2=0x7f38a0151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 0x7f38a0151de0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f389400c5b0 tx=0x7f3894001070 comp rx=0 tx=0).stop 2023-10-06T05:24:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 shutdown_connections 2023-10-06T05:24:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f388c066f30 0x7f388c0693e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38a01576f0 0x7f38a0152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a00b1ae0 0x7f38a0155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 --2- 172.21.15.33:0/2332016961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a0157320 0x7f38a0151de0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.952+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 >> 172.21.15.33:0/2332016961 conn(0x7f38a0068090 msgr2=0x7f38a00cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.953+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 shutdown_connections 2023-10-06T05:24:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:15.953+0000 7f38a519d700 1 -- 172.21.15.33:0/2332016961 wait complete. 2023-10-06T05:24:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:24:15.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:24:16.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.173+0000 7f9822d9d700 1 Processor -- start 2023-10-06T05:24:16.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.173+0000 7f9822d9d700 1 -- start start 2023-10-06T05:24:16.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.174+0000 7f9822d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f981c1597f0 0x7f981c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.174+0000 7f9822d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f981c06dfd0 0x7f981c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.174+0000 7f9822d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c15d280 con 0x7f981c06dc00 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.174+0000 7f9822d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c15e290 con 0x7f981c1597f0 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.174+0000 7f9822d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c15de00 con 0x7f981c06dfd0 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f981c1597f0 0x7f981c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f981c1597f0 0x7f981c15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34822/0 (socket says 172.21.15.33:34822) 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 -- 172.21.15.33:0/1138680499 learned_addr learned my addr 172.21.15.33:0/1138680499 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982259c700 1 --2- 172.21.15.33:0/1138680499 >> v2:172.21.15.167:3300/0 conn(0x7f981c06dfd0 0x7f981c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 -- 172.21.15.33:0/1138680499 >> v2:172.21.15.167:3300/0 conn(0x7f981c06dfd0 msgr2=0x7f981c15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 --2- 172.21.15.33:0/1138680499 >> v2:172.21.15.167:3300/0 conn(0x7f981c06dfd0 0x7f981c15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 -- 172.21.15.33:0/1138680499 >> v1:172.21.15.167:6789/0 conn(0x7f981c06dc00 legacy=0x7f981c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 -- 172.21.15.33:0/1138680499 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c15d4d0 con 0x7f981c1597f0 2023-10-06T05:24:16.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982259c700 1 --2- 172.21.15.33:0/1138680499 >> v2:172.21.15.167:3300/0 conn(0x7f981c06dfd0 0x7f981c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:16.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f982159a700 1 --2- 172.21.15.33:0/1138680499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c1597f0 0x7f981c15a030 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b29b0 tx=0x7f980c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d489f0da09f4a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.175+0000 7f9820d99700 1 -- 172.21.15.33:0/1138680499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c00b860 con 0x7f981c1597f0 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9820d99700 1 -- 172.21.15.33:0/1138680499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c00b9c0 con 0x7f981c1597f0 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9820d99700 1 -- 172.21.15.33:0/1138680499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c00bd60 con 0x7f981c1597f0 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 -- 172.21.15.33:0/1138680499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c1597f0 msgr2=0x7f981c15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 --2- 172.21.15.33:0/1138680499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c1597f0 0x7f981c15a030 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b29b0 tx=0x7f980c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 -- 172.21.15.33:0/1138680499 shutdown_connections 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 --2- 172.21.15.33:0/1138680499 >> v2:172.21.15.167:3300/0 conn(0x7f981c06dfd0 0x7f981c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 --2- 172.21.15.33:0/1138680499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c1597f0 0x7f981c15a030 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 -- 172.21.15.33:0/1138680499 >> 172.21.15.33:0/1138680499 conn(0x7f981c0d0550 msgr2=0x7f981c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:16.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 -- 172.21.15.33:0/1138680499 shutdown_connections 2023-10-06T05:24:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.176+0000 7f9822d9d700 1 -- 172.21.15.33:0/1138680499 wait complete. 2023-10-06T05:24:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.178+0000 7f9822d9d700 1 Processor -- start 2023-10-06T05:24:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.178+0000 7f9822d9d700 1 -- start start 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9822d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 0x7f981c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9822d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f981c06dfd0 0x7f981c169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9822d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f981c1597f0 0x7f981c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9822d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c16d4d0 con 0x7f981c06dc00 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9822d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c150280 con 0x7f981c1597f0 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9822d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c16d130 con 0x7f981c06dfd0 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 0x7f981c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 0x7f981c14fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34836/0 (socket says 172.21.15.33:34836) 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 -- 172.21.15.33:0/2174193343 learned_addr learned my addr 172.21.15.33:0/2174193343 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f982259c700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f981c1597f0 0x7f981c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f982159a700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f981c06dfd0 0x7f981c169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 -- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f981c06dfd0 msgr2=0x7f981c169af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f981c06dfd0 0x7f981c169af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 -- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f981c1597f0 msgr2=0x7f981c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f981c1597f0 0x7f981c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.179+0000 7f9821d9b700 1 -- 172.21.15.33:0/2174193343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c0094f0 con 0x7f981c06dc00 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.180+0000 7f9821d9b700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 0x7f981c14fd40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f981800c590 tx=0x7f981800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.180+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818011590 con 0x7f981c06dc00 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.180+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9818011710 con 0x7f981c06dc00 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.180+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c16a360 con 0x7f981c06dc00 2023-10-06T05:24:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.180+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818011ad0 con 0x7f981c06dc00 2023-10-06T05:24:16.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.180+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c16a920 con 0x7f981c06dc00 2023-10-06T05:24:16.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.181+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c16d740 con 0x7f981c06dc00 2023-10-06T05:24:16.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.182+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f981801b3b0 con 0x7f981c06dc00 2023-10-06T05:24:16.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.183+0000 7f9811ffb700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9800066f30 0x7f98000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.183+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f9818083a70 con 0x7f981c06dc00 2023-10-06T05:24:16.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.183+0000 7f982159a700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9800066f30 0x7f98000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.184+0000 7f982159a700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9800066f30 0x7f98000693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b29b0 tx=0x7f980c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.187+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9818050c10 con 0x7f981c06dc00 2023-10-06T05:24:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.390+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f981c063ff0 con 0x7f981c06dc00 2023-10-06T05:24:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.952+0000 7f9811ffb700 1 -- 172.21.15.33:0/2174193343 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9818017070 con 0x7f981c06dc00 2023-10-06T05:24:16.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.958+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9800066f30 msgr2=0x7f98000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.958+0000 7f9822d9d700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9800066f30 0x7f98000693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f981c0b29b0 tx=0x7f980c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:16.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 msgr2=0x7f981c14fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 0x7f981c14fd40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f981800c590 tx=0x7f981800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 shutdown_connections 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9800066f30 0x7f98000693e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f981c1597f0 0x7f981c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c06dc00 0x7f981c14fd40 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 --2- 172.21.15.33:0/2174193343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f981c06dfd0 0x7f981c169af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.959+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 >> 172.21.15.33:0/2174193343 conn(0x7f981c0d0550 msgr2=0x7f981c0c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.960+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 shutdown_connections 2023-10-06T05:24:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:16.960+0000 7f9822d9d700 1 -- 172.21.15.33:0/2174193343 wait complete. 2023-10-06T05:24:16.964 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:24:16.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:24:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.176+0000 7fa0ba39d700 1 Processor -- start 2023-10-06T05:24:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.176+0000 7fa0ba39d700 1 -- start start 2023-10-06T05:24:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.177+0000 7fa0ba39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0b40af540 0x7fa0b40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.177+0000 7fa0ba39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0b415a650 0x7fa0b415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.177+0000 7fa0ba39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b415d660 con 0x7fa0b40cced0 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.177+0000 7fa0ba39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b40ce890 con 0x7fa0b40af540 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.177+0000 7fa0ba39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b415aac0 con 0x7fa0b415a650 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0b9b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa0b40cced0 0x7fa0b406c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60516/0 (socket says 172.21.15.33:60516) 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0b9b9c700 1 -- 172.21.15.33:0/164399944 learned_addr learned my addr 172.21.15.33:0/164399944 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0b939b700 1 --2- 172.21.15.33:0/164399944 >> v2:172.21.15.167:3300/0 conn(0x7fa0b40af540 0x7fa0b40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0b8b9a700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0b415a650 0x7fa0b415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169554656 0 0) 0x7fa0b415d660 con 0x7fa0b40cced0 2023-10-06T05:24:17.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a0003570 con 0x7fa0b40cced0 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318629093 0 0) 0x7fa0a0003570 con 0x7fa0b40cced0 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0b415a650 msgr2=0x7fa0b415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0b415a650 0x7fa0b415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 >> v2:172.21.15.167:3300/0 conn(0x7fa0b40af540 msgr2=0x7fa0b40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 --2- 172.21.15.33:0/164399944 >> v2:172.21.15.167:3300/0 conn(0x7fa0b40af540 0x7fa0b40af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b415d890 con 0x7fa0b40cced0 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa0ac0017f0 con 0x7fa0b40cced0 2023-10-06T05:24:17.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 >> v1:172.21.15.33:6789/0 conn(0x7fa0b40cced0 legacy=0x7fa0b406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.178+0000 7fa0abfff700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0a0003ae0 0x7fa0a0003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0abfff700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a0004810 0x7fa0a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0abfff700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a00051c0 0x7fa0a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40ce890 con 0x7fa0a00051c0 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b415aac0 con 0x7fa0a0003ae0 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a000bfd0 con 0x7fa0a0004810 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b8b9a700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0b415a650 0x7fa0b415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b939b700 1 --2- 172.21.15.33:0/164399944 >> v2:172.21.15.167:3300/0 conn(0x7fa0b40af540 0x7fa0b40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b9b9c700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a00051c0 0x7fa0a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b8b9a700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a0004810 0x7fa0a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b939b700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0a0003ae0 0x7fa0a0003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b9b9c700 1 -- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a0004810 msgr2=0x7fa0a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b9b9c700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a0004810 0x7fa0a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b9b9c700 1 -- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0a0003ae0 msgr2=0x7fa0a0003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b9b9c700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0a0003ae0 0x7fa0a0003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b9b9c700 1 -- 172.21.15.33:0/164399944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0003570 con 0x7fa0a00051c0 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.179+0000 7fa0b939b700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0a0003ae0 0x7fa0a0003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0b9b9c700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a00051c0 0x7fa0a000ba60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00a890 tx=0x7fa0b40ce800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e61594f18fd0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0070f0 con 0x7fa0a00051c0 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac007250 con 0x7fa0a00051c0 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0abfff700 1 -- 172.21.15.33:0/164399944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac007610 con 0x7fa0a00051c0 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0ba39d700 1 -- 172.21.15.33:0/164399944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a00051c0 msgr2=0x7fa0a000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/164399944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0a00051c0 0x7fa0a000ba60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00a890 tx=0x7fa0b40ce800 comp rx=0 tx=0).stop 2023-10-06T05:24:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0b9b9c700 1 -- 172.21.15.33:0/164399944 reap_dead start 2023-10-06T05:24:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0ba39d700 1 -- 172.21.15.33:0/164399944 shutdown_connections 2023-10-06T05:24:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0ba39d700 1 -- 172.21.15.33:0/164399944 >> 172.21.15.33:0/164399944 conn(0x7fa0b406d9b0 msgr2=0x7fa0b406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.180+0000 7fa0ba39d700 1 -- 172.21.15.33:0/164399944 shutdown_connections 2023-10-06T05:24:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.181+0000 7fa0ba39d700 1 -- 172.21.15.33:0/164399944 wait complete. 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.182+0000 7fa0ba39d700 1 Processor -- start 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.182+0000 7fa0ba39d700 1 -- start start 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0ba39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a00051c0 0x7fa0b40b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0ba39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 0x7fa0b40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0ba39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0b4146fd0 0x7fa0b4158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0ba39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b38d0 con 0x7fa0b4155750 2023-10-06T05:24:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0ba39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4155bc0 con 0x7fa0b4146fd0 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0ba39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b4dd0 con 0x7fa0a00051c0 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0b8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 0x7fa0b40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0b8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 0x7fa0b40b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34860/0 (socket says 172.21.15.33:34860) 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0b8b9a700 1 -- 172.21.15.33:0/4078844761 learned_addr learned my addr 172.21.15.33:0/4078844761 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0b9b9c700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0b4146fd0 0x7fa0b4158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.183+0000 7fa0b939b700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a00051c0 0x7fa0b40b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0b8b9a700 1 -- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a00051c0 msgr2=0x7fa0b40b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0b8b9a700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a00051c0 0x7fa0b40b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0b8b9a700 1 -- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0b4146fd0 msgr2=0x7fa0b4158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0b8b9a700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0b4146fd0 0x7fa0b4158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0b8b9a700 1 -- 172.21.15.33:0/4078844761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4155de0 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0b8b9a700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 0x7fa0b40b3390 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4011d80 tx=0x7fa0a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400fa10 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a400fb70 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a401d480 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b4156f20 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.184+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b40b5090 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.185+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b40b5220 con 0x7fa0b4155750 2023-10-06T05:24:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.186+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0a401d8b0 con 0x7fa0b4155750 2023-10-06T05:24:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.187+0000 7fa0a97fa700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa098066f30 0x7fa0980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.187+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fa0a4018070 con 0x7fa0b4155750 2023-10-06T05:24:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.187+0000 7fa0b939b700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa098066f30 0x7fa0980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.188+0000 7fa0b939b700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa098066f30 0x7fa0980693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00093f0 tx=0x7fa0b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.191+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0a40524c0 con 0x7fa0b4155750 2023-10-06T05:24:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.394+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0b404c380 con 0x7fa0b4155750 2023-10-06T05:24:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.398+0000 7fa0a97fa700 1 -- 172.21.15.33:0/4078844761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0a4055ae0 con 0x7fa0b4155750 2023-10-06T05:24:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa098066f30 msgr2=0x7fa0980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa098066f30 0x7fa0980693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00093f0 tx=0x7fa0b0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 msgr2=0x7fa0b40b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 0x7fa0b40b3390 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4011d80 tx=0x7fa0a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 shutdown_connections 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa098066f30 0x7fa0980693e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0b4146fd0 0x7fa0b4158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0a00051c0 0x7fa0b40b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 --2- 172.21.15.33:0/4078844761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0b4155750 0x7fa0b40b3390 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.400+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 >> 172.21.15.33:0/4078844761 conn(0x7fa0b406d9b0 msgr2=0x7fa0b40c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.401+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 shutdown_connections 2023-10-06T05:24:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.401+0000 7fa0ba39d700 1 -- 172.21.15.33:0/4078844761 wait complete. 2023-10-06T05:24:17.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock' 2023-10-06T05:24:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.605+0000 7efcd339d700 1 Processor -- start 2023-10-06T05:24:17.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.605+0000 7efcd339d700 1 -- start start 2023-10-06T05:24:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.606+0000 7efcd339d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 0x7efccc15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.606+0000 7efcd339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 0x7efccc0b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.606+0000 7efcd339d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efccc06f220 con 0x7efccc159530 2023-10-06T05:24:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.606+0000 7efcd339d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efccc0b1e00 con 0x7efccc06e140 2023-10-06T05:24:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.606+0000 7efcd339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc06f6b0 con 0x7efccc0afdd0 2023-10-06T05:24:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd2b9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 0x7efccc0b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd1b9a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 0x7efccc15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd1b9a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 0x7efccc15a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:55464/0 (socket says 172.21.15.33:55464) 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd2b9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 0x7efccc0b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55108/0 (socket says 172.21.15.33:55108) 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd1b9a700 1 -- 172.21.15.33:0/815031726 learned_addr learned my addr 172.21.15.33:0/815031726 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd2b9c700 1 -- 172.21.15.33:0/815031726 >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 msgr2=0x7efccc15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.607+0000 7efcd2b9c700 1 --2- 172.21.15.33:0/815031726 >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 0x7efccc15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd2b9c700 1 -- 172.21.15.33:0/815031726 >> v1:172.21.15.167:6789/0 conn(0x7efccc159530 legacy=0x7efccc06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd2b9c700 1 -- 172.21.15.33:0/815031726 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc15ae50 con 0x7efccc0afdd0 2023-10-06T05:24:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd1399700 1 -- 172.21.15.33:0/815031726 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573662611 0 0) 0x7efccc06f220 con 0x7efccc159530 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd1b9a700 1 --2- 172.21.15.33:0/815031726 >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 0x7efccc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd2b9c700 1 --2- 172.21.15.33:0/815031726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 0x7efccc0b01f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009e60 tx=0x7efcc4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f13a273c7d01c20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd1399700 1 -- 172.21.15.33:0/815031726 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc400e070 con 0x7efccc0afdd0 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd1399700 1 -- 172.21.15.33:0/815031726 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc400bb60 con 0x7efccc0afdd0 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.608+0000 7efcd1399700 1 -- 172.21.15.33:0/815031726 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4012430 con 0x7efccc0afdd0 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 -- 172.21.15.33:0/815031726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 msgr2=0x7efccc0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 --2- 172.21.15.33:0/815031726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 0x7efccc0b01f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009e60 tx=0x7efcc4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 -- 172.21.15.33:0/815031726 shutdown_connections 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 --2- 172.21.15.33:0/815031726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc0afdd0 0x7efccc0b01f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 --2- 172.21.15.33:0/815031726 >> v2:172.21.15.167:3300/0 conn(0x7efccc06e140 0x7efccc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 -- 172.21.15.33:0/815031726 >> 172.21.15.33:0/815031726 conn(0x7efccc06c560 msgr2=0x7efccc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 -- 172.21.15.33:0/815031726 shutdown_connections 2023-10-06T05:24:17.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.609+0000 7efcd339d700 1 -- 172.21.15.33:0/815031726 wait complete. 2023-10-06T05:24:17.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.610+0000 7efcd339d700 1 Processor -- start 2023-10-06T05:24:17.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.611+0000 7efcd339d700 1 -- start start 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.611+0000 7efcd339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc06e140 0x7efccc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.611+0000 7efcd339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc0afdd0 0x7efccc0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.611+0000 7efcd339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 0x7efccc0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0ca160 con 0x7efccc159530 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc148490 con 0x7efccc0afdd0 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0c8cf0 con 0x7efccc06e140 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 0x7efccc0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 0x7efccc0ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34864/0 (socket says 172.21.15.33:34864) 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 -- 172.21.15.33:0/2888214184 learned_addr learned my addr 172.21.15.33:0/2888214184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd1b9a700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc0afdd0 0x7efccc0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd239b700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc06e140 0x7efccc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 -- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc06e140 msgr2=0x7efccc147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc06e140 0x7efccc147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 -- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc0afdd0 msgr2=0x7efccc0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc0afdd0 0x7efccc0cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd2b9c700 1 -- 172.21.15.33:0/2888214184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0094f0 con 0x7efccc159530 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.612+0000 7efcd1b9a700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc0afdd0 0x7efccc0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.613+0000 7efcd2b9c700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 0x7efccc0ccb10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efcc4012a50 tx=0x7efcc4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.613+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4012e60 con 0x7efccc159530 2023-10-06T05:24:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.613+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc401d630 con 0x7efccc159530 2023-10-06T05:24:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.613+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc4009a20 con 0x7efccc159530 2023-10-06T05:24:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.613+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc401d9f0 con 0x7efccc159530 2023-10-06T05:24:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.613+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efccc0c92b0 con 0x7efccc159530 2023-10-06T05:24:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.614+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efccc04c380 con 0x7efccc159530 2023-10-06T05:24:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.615+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcc4024070 con 0x7efccc159530 2023-10-06T05:24:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.616+0000 7efcc27fc700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efcb0066f30 0x7efcb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.616+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5378+0+0 (secure 0 0 0) 0x7efcc4086bd0 con 0x7efccc159530 2023-10-06T05:24:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.616+0000 7efcd239b700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efcb0066f30 0x7efcb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.617+0000 7efcd239b700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efcb0066f30 0x7efcb00693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efcc80011e0 tx=0x7efcc8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.620+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcc4053f80 con 0x7efccc159530 2023-10-06T05:24:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.899+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock"]} v 0) v1 -- 0x7efccc0b0730 con 0x7efccc159530 2023-10-06T05:24:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.959+0000 7efcc27fc700 1 -- 172.21.15.33:0/2888214184 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock"]}]=0 v509) v1 ==== 145+0+0 (secure 0 0 0) 0x7efcc40575a0 con 0x7efccc159530 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efcb0066f30 msgr2=0x7efcb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efcb0066f30 0x7efcb00693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efcc80011e0 tx=0x7efcc8009040 comp rx=0 tx=0).stop 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 msgr2=0x7efccc0ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 0x7efccc0ccb10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efcc4012a50 tx=0x7efcc4019000 comp rx=0 tx=0).stop 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 shutdown_connections 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efcb0066f30 0x7efcb00693e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efccc159530 0x7efccc0ccb10 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efccc06e140 0x7efccc147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 --2- 172.21.15.33:0/2888214184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc0afdd0 0x7efccc0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 >> 172.21.15.33:0/2888214184 conn(0x7efccc06c560 msgr2=0x7efccc15b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 shutdown_connections 2023-10-06T05:24:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:17.961+0000 7efcd339d700 1 -- 172.21.15.33:0/2888214184 wait complete. 2023-10-06T05:24:17.981 INFO:tasks.cephfs_test_runner:Check that file lock doesn't get lost after an MDS restart ... ok 2023-10-06T05:24:17.981 INFO:tasks.cephfs_test_runner:Starting test: test_filelock_eviction (tasks.cephfs.test_client_recovery.TestClientRecovery) Check that file lock held by evicted client is given to 2023-10-06T05:24:17.981 INFO:tasks.cephfs_test_runner:test_filelock_eviction (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:24:17.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock_eviction' 2023-10-06T05:24:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.165+0000 7f136c9fe700 1 Processor -- start 2023-10-06T05:24:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.165+0000 7f136c9fe700 1 -- start start 2023-10-06T05:24:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136c9fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1368159650 0x7f13680c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136c9fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1368159a20 0x7f13680c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136c9fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13680c92d0 con 0x7f13680c9760 2023-10-06T05:24:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136c9fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13680c7bc0 con 0x7f1368159a20 2023-10-06T05:24:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136c9fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13680c86c0 con 0x7f1368159650 2023-10-06T05:24:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1368159a20 0x7f13680c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1368159a20 0x7f13680c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34866/0 (socket says 172.21.15.33:34866) 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f136659c700 1 -- 172.21.15.33:0/1573147307 learned_addr learned my addr 172.21.15.33:0/1573147307 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.167+0000 7f1366d9d700 1 --2- 172.21.15.33:0/1573147307 >> v2:172.21.15.167:3300/0 conn(0x7f1368159650 0x7f13680c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f136659c700 1 -- 172.21.15.33:0/1573147307 >> v2:172.21.15.167:3300/0 conn(0x7f1368159650 msgr2=0x7f13680c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f136659c700 1 --2- 172.21.15.33:0/1573147307 >> v2:172.21.15.167:3300/0 conn(0x7f1368159650 0x7f13680c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f136659c700 1 -- 172.21.15.33:0/1573147307 >> v1:172.21.15.167:6789/0 conn(0x7f13680c9760 legacy=0x7f13680c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f136659c700 1 -- 172.21.15.33:0/1573147307 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13680c89a0 con 0x7f1368159a20 2023-10-06T05:24:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f136659c700 1 --2- 172.21.15.33:0/1573147307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1368159a20 0x7f13680c7680 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f136806d4f0 tx=0x7f135c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cad41d092c46f7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f1365d9b700 1 -- 172.21.15.33:0/1573147307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c00b820 con 0x7f1368159a20 2023-10-06T05:24:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f1365d9b700 1 -- 172.21.15.33:0/1573147307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c00b980 con 0x7f1368159a20 2023-10-06T05:24:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.168+0000 7f1365d9b700 1 -- 172.21.15.33:0/1573147307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c00bd20 con 0x7f1368159a20 2023-10-06T05:24:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 -- 172.21.15.33:0/1573147307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1368159a20 msgr2=0x7f13680c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 --2- 172.21.15.33:0/1573147307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1368159a20 0x7f13680c7680 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f136806d4f0 tx=0x7f135c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 -- 172.21.15.33:0/1573147307 shutdown_connections 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 --2- 172.21.15.33:0/1573147307 >> v2:172.21.15.167:3300/0 conn(0x7f1368159650 0x7f13680c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 --2- 172.21.15.33:0/1573147307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1368159a20 0x7f13680c7680 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 -- 172.21.15.33:0/1573147307 >> 172.21.15.33:0/1573147307 conn(0x7f136806d030 msgr2=0x7f136806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 -- 172.21.15.33:0/1573147307 shutdown_connections 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.169+0000 7f136c9fe700 1 -- 172.21.15.33:0/1573147307 wait complete. 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.170+0000 7f136c9fe700 1 Processor -- start 2023-10-06T05:24:18.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.170+0000 7f136c9fe700 1 -- start start 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.171+0000 7f136c9fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 0x7f13680b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.171+0000 7f136c9fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1368159650 0x7f136816a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.171+0000 7f136c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1368159a20 0x7f136816cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.171+0000 7f136c9fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13680b0fb0 con 0x7f13680c9760 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.171+0000 7f136c9fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13680b1290 con 0x7f1368159650 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.171+0000 7f136c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13680b1570 con 0x7f1368159a20 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 0x7f13680b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 0x7f13680b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34870/0 (socket says 172.21.15.33:34870) 2023-10-06T05:24:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 -- 172.21.15.33:0/3074990464 learned_addr learned my addr 172.21.15.33:0/3074990464 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f136659c700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1368159650 0x7f136816a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 -- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1368159a20 msgr2=0x7f136816cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1368159a20 0x7f136816cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 -- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1368159650 msgr2=0x7f136816a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1368159650 0x7f136816a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 -- 172.21.15.33:0/3074990464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c0094f0 con 0x7f13680c9760 2023-10-06T05:24:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f136659c700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1368159650 0x7f136816a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f1366d9d700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 0x7f13680b0910 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f135800d890 tx=0x7f1358008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800b600 con 0x7f13680c9760 2023-10-06T05:24:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135800b760 con 0x7f13680c9760 2023-10-06T05:24:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13680b1850 con 0x7f13680c9760 2023-10-06T05:24:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.172+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800bae0 con 0x7f13680c9760 2023-10-06T05:24:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.173+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136816ae70 con 0x7f13680c9760 2023-10-06T05:24:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.173+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13680b1cf0 con 0x7f13680c9760 2023-10-06T05:24:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.174+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f135801e460 con 0x7f13680c9760 2023-10-06T05:24:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.175+0000 7f134effd700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1354066f30 0x7f13540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.175+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f1358084b40 con 0x7f13680c9760 2023-10-06T05:24:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.176+0000 7f136659c700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1354066f30 0x7f13540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.177+0000 7f136659c700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1354066f30 0x7f13540693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f136806d4f0 tx=0x7f135c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.179+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1358051e50 con 0x7f13680c9760 2023-10-06T05:24:18.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.472+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock_eviction"]} v 0) v1 -- 0x7f136804c380 con 0x7f13680c9760 2023-10-06T05:24:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.964+0000 7f134effd700 1 -- 172.21.15.33:0/3074990464 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock_eviction"]}]=0 v510) v1 ==== 157+0+0 (secure 0 0 0) 0x7f1358055470 con 0x7f13680c9760 2023-10-06T05:24:18.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1354066f30 msgr2=0x7f13540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1354066f30 0x7f13540693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f136806d4f0 tx=0x7f135c016000 comp rx=0 tx=0).stop 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 msgr2=0x7f13680b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 0x7f13680b0910 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f135800d890 tx=0x7f1358008040 comp rx=0 tx=0).stop 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 shutdown_connections 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1354066f30 0x7f13540693e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1368159a20 0x7f136816cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13680c9760 0x7f13680b0910 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 --2- 172.21.15.33:0/3074990464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1368159650 0x7f136816a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 >> 172.21.15.33:0/3074990464 conn(0x7f136806d030 msgr2=0x7f13680c14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 shutdown_connections 2023-10-06T05:24:18.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:18.967+0000 7f136c9fe700 1 -- 172.21.15.33:0/3074990464 wait complete. 2023-10-06T05:24:18.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:24:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.178+0000 7f4fa939d700 1 Processor -- start 2023-10-06T05:24:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.178+0000 7f4fa939d700 1 -- start start 2023-10-06T05:24:19.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa939d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4fa40af540 0x7f4fa40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4fa415a650 0x7f4fa415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40cced0 0x7f4fa40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa40cd890 con 0x7f4fa415a650 2023-10-06T05:24:19.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa939d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa415d660 con 0x7f4fa40af540 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40ce890 con 0x7f4fa40cced0 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4fa415a650 0x7f4fa415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4fa415a650 0x7f4fa415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34872/0 (socket says 172.21.15.33:34872) 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 -- 172.21.15.33:0/4065252387 learned_addr learned my addr 172.21.15.33:0/4065252387 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 -- 172.21.15.33:0/4065252387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40cced0 msgr2=0x7f4fa40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 --2- 172.21.15.33:0/4065252387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40cced0 0x7f4fa40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 -- 172.21.15.33:0/4065252387 >> v2:172.21.15.167:3300/0 conn(0x7f4fa40af540 msgr2=0x7f4fa40af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 --2- 172.21.15.33:0/4065252387 >> v2:172.21.15.167:3300/0 conn(0x7f4fa40af540 0x7f4fa40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.180+0000 7f4fa37fe700 1 -- 172.21.15.33:0/4065252387 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa415aac0 con 0x7f4fa415a650 2023-10-06T05:24:19.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa37fe700 1 --2- 172.21.15.33:0/4065252387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa415a650 0x7f4fa415d120 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40c7530 tx=0x7f4f98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea214c95b7d46602 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa2ffd700 1 -- 172.21.15.33:0/4065252387 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800b820 con 0x7f4fa415a650 2023-10-06T05:24:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa2ffd700 1 -- 172.21.15.33:0/4065252387 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9800b980 con 0x7f4fa415a650 2023-10-06T05:24:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa2ffd700 1 -- 172.21.15.33:0/4065252387 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800bd20 con 0x7f4fa415a650 2023-10-06T05:24:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 -- 172.21.15.33:0/4065252387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa415a650 msgr2=0x7f4fa415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 --2- 172.21.15.33:0/4065252387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa415a650 0x7f4fa415d120 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40c7530 tx=0x7f4f98009250 comp rx=0 tx=0).stop 2023-10-06T05:24:19.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 -- 172.21.15.33:0/4065252387 shutdown_connections 2023-10-06T05:24:19.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 --2- 172.21.15.33:0/4065252387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40cced0 0x7f4fa40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 --2- 172.21.15.33:0/4065252387 >> v2:172.21.15.167:3300/0 conn(0x7f4fa40af540 0x7f4fa40af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 --2- 172.21.15.33:0/4065252387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa415a650 0x7f4fa415d120 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.181+0000 7f4fa939d700 1 -- 172.21.15.33:0/4065252387 >> 172.21.15.33:0/4065252387 conn(0x7f4fa406d9b0 msgr2=0x7f4fa406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.182+0000 7f4fa939d700 1 -- 172.21.15.33:0/4065252387 shutdown_connections 2023-10-06T05:24:19.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.182+0000 7f4fa939d700 1 -- 172.21.15.33:0/4065252387 wait complete. 2023-10-06T05:24:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.183+0000 7f4fa939d700 1 Processor -- start 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.183+0000 7f4fa939d700 1 -- start start 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40af540 0x7f4fa414be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 0x7f4fa40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa415a650 0x7f4fa4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40b4240 con 0x7f4fa40cced0 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40b3630 con 0x7f4fa415a650 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40b3910 con 0x7f4fa40af540 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 0x7f4fa40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 0x7f4fa40c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34878/0 (socket says 172.21.15.33:34878) 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.184+0000 7f4fa37fe700 1 -- 172.21.15.33:0/1370997230 learned_addr learned my addr 172.21.15.33:0/1370997230 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa37fe700 1 -- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40af540 msgr2=0x7f4fa414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa37fe700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40af540 0x7f4fa414be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa37fe700 1 -- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa415a650 msgr2=0x7f4fa4158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa37fe700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa415a650 0x7f4fa4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa37fe700 1 -- 172.21.15.33:0/1370997230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980094f0 con 0x7f4fa40cced0 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa37fe700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 0x7f4fa40c9e60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4f980094c0 tx=0x7f4f98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9801a420 con 0x7f4fa40cced0 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9801a580 con 0x7f4fa40cced0 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9801a940 con 0x7f4fa40cced0 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa40b3bf0 con 0x7f4fa40cced0 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.185+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa40b3270 con 0x7f4fa40cced0 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.186+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa404c380 con 0x7f4fa40cced0 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.187+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f9801ad30 con 0x7f4fa40cced0 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.189+0000 7f4f8bfff700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f90066f30 0x7f4f900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.189+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f4f98085fd0 con 0x7f4fa40cced0 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.189+0000 7f4fa3fff700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f90066f30 0x7f4f900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.190+0000 7f4fa3fff700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f90066f30 0x7f4f900693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40b6320 tx=0x7f4f94006d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.194+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f980532e0 con 0x7f4fa40cced0 2023-10-06T05:24:19.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:19.355 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.397+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4fa40b4380 con 0x7f4fa40cced0 2023-10-06T05:24:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.402+0000 7f4f8bfff700 1 -- 172.21.15.33:0/1370997230 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fa40b4380 con 0x7f4fa40cced0 2023-10-06T05:24:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f90066f30 msgr2=0x7f4f900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f90066f30 0x7f4f900693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40b6320 tx=0x7f4f94006d10 comp rx=0 tx=0).stop 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 msgr2=0x7f4fa40c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 0x7f4fa40c9e60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4f980094c0 tx=0x7f4f98017000 comp rx=0 tx=0).stop 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 shutdown_connections 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f90066f30 0x7f4f900693e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa415a650 0x7f4fa4158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fa40af540 0x7f4fa414be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 --2- 172.21.15.33:0/1370997230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa40cced0 0x7f4fa40c9e60 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.404+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 >> 172.21.15.33:0/1370997230 conn(0x7f4fa406d9b0 msgr2=0x7f4fa40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.405+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 shutdown_connections 2023-10-06T05:24:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.405+0000 7f4fa939d700 1 -- 172.21.15.33:0/1370997230 wait complete. 2023-10-06T05:24:19.426 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:24:19.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:19.504 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:19.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:24:19.585 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:24:19.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:19.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.771+0000 7fe12939d700 1 Processor -- start 2023-10-06T05:24:19.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.771+0000 7fe12939d700 1 -- start start 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe12939d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1240cf3b0 0x7fe1241594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe12939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1240af0a0 0x7fe1241599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe12939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1240cbd80 con 0x7fe1240ccfc0 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe12939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240af5f0 con 0x7fe1240af0a0 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe12939d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240af8d0 con 0x7fe1240cf3b0 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe123fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe1240ccfc0 0x7fe1240cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60526/0 (socket says 172.21.15.33:60526) 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe123fff700 1 -- 172.21.15.33:0/3854257638 learned_addr learned my addr 172.21.15.33:0/3854257638 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe128b9c700 1 --2- 172.21.15.33:0/3854257638 >> v2:172.21.15.33:3300/0 conn(0x7fe1240af0a0 0x7fe1241599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe1237fe700 1 --2- 172.21.15.33:0/3854257638 >> v2:172.21.15.167:3300/0 conn(0x7fe1240cf3b0 0x7fe1241594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe122ffd700 1 -- 172.21.15.33:0/3854257638 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004059844 0 0) 0x7fe1240cbd80 con 0x7fe1240ccfc0 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe122ffd700 1 -- 172.21.15.33:0/3854257638 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c003570 con 0x7fe1240ccfc0 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe128b9c700 1 -- 172.21.15.33:0/3854257638 >> v2:172.21.15.167:3300/0 conn(0x7fe1240cf3b0 msgr2=0x7fe1241594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe128b9c700 1 --2- 172.21.15.33:0/3854257638 >> v2:172.21.15.167:3300/0 conn(0x7fe1240cf3b0 0x7fe1241594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe128b9c700 1 -- 172.21.15.33:0/3854257638 >> v1:172.21.15.33:6789/0 conn(0x7fe1240ccfc0 legacy=0x7fe1240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe128b9c700 1 -- 172.21.15.33:0/3854257638 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1240afbb0 con 0x7fe1240af0a0 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.772+0000 7fe128b9c700 1 --2- 172.21.15.33:0/3854257638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240af0a0 0x7fe1241599e0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009e60 tx=0x7fe11c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155c24f6538d2a51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe122ffd700 1 -- 172.21.15.33:0/3854257638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00e070 con 0x7fe1240af0a0 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe122ffd700 1 -- 172.21.15.33:0/3854257638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c00bb20 con 0x7fe1240af0a0 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe122ffd700 1 -- 172.21.15.33:0/3854257638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c012430 con 0x7fe1240af0a0 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 -- 172.21.15.33:0/3854257638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240af0a0 msgr2=0x7fe1241599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 --2- 172.21.15.33:0/3854257638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240af0a0 0x7fe1241599e0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009e60 tx=0x7fe11c009780 comp rx=0 tx=0).stop 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 -- 172.21.15.33:0/3854257638 shutdown_connections 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 --2- 172.21.15.33:0/3854257638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240af0a0 0x7fe1241599e0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 --2- 172.21.15.33:0/3854257638 >> v2:172.21.15.167:3300/0 conn(0x7fe1240cf3b0 0x7fe1241594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 -- 172.21.15.33:0/3854257638 >> 172.21.15.33:0/3854257638 conn(0x7fe12406cff0 msgr2=0x7fe12406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 -- 172.21.15.33:0/3854257638 shutdown_connections 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.773+0000 7fe12939d700 1 -- 172.21.15.33:0/3854257638 wait complete. 2023-10-06T05:24:19.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.775+0000 7fe12939d700 1 Processor -- start 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.775+0000 7fe12939d700 1 -- start start 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe12939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1240af0a0 0x7fe12416a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe12939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 0x7fe1240c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe12939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b2430 0x7fe1240b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe12939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240b2df0 con 0x7fe1240ccfc0 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe12939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12416adb0 con 0x7fe1240af0a0 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe12939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240b4ce0 con 0x7fe1240b2430 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 0x7fe1240c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 0x7fe1240c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34902/0 (socket says 172.21.15.33:34902) 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe1237fe700 1 -- 172.21.15.33:0/2883301554 learned_addr learned my addr 172.21.15.33:0/2883301554 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe128b9c700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b2430 0x7fe1240b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.776+0000 7fe123fff700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1240af0a0 0x7fe12416a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe1237fe700 1 -- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b2430 msgr2=0x7fe1240b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe1237fe700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b2430 0x7fe1240b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe1237fe700 1 -- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1240af0a0 msgr2=0x7fe12416a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe1237fe700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1240af0a0 0x7fe12416a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe1237fe700 1 -- 172.21.15.33:0/2883301554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c009a20 con 0x7fe1240ccfc0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe1237fe700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 0x7fe1240c9d10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe11800d890 tx=0x7fe118008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1180125f0 con 0x7fe1240ccfc0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118012750 con 0x7fe1240ccfc0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1240b3660 con 0x7fe1240ccfc0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118012b10 con 0x7fe1240ccfc0 2023-10-06T05:24:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.777+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1240b3c20 con 0x7fe1240ccfc0 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.779+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12404c380 con 0x7fe1240ccfc0 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.779+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe11801e460 con 0x7fe1240ccfc0 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.780+0000 7fe10bfff700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe110066f30 0x7fe1100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.780+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fe1180849b0 con 0x7fe1240ccfc0 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.781+0000 7fe123fff700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe110066f30 0x7fe1100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.781+0000 7fe123fff700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe110066f30 0x7fe1100693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe12415a650 tx=0x7fe114009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:19.785+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe118051cc0 con 0x7fe1240ccfc0 2023-10-06T05:24:20.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.034+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1240b62d0 con 0x7fe1240ccfc0 2023-10-06T05:24:20.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.035+0000 7fe10bfff700 1 -- 172.21.15.33:0/2883301554 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fe1180552e0 con 0x7fe1240ccfc0 2023-10-06T05:24:20.038 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:20.038 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6725,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3889543432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3889543432},{"type":"v1","addr":"172.21.15.33:6835","nonce":3889543432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1068003767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1068003767},{"type":"v1","addr":"172.21.15.167:6837","nonce":1068003767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/692553220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":692553220},{"type":"v1","addr":"172.21.15.33:6833","nonce":692553220}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":8175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/946297362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":946297362},{"type":"v1","addr":"172.21.15.167:6836","nonce":946297362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-10-06T05:24:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe110066f30 msgr2=0x7fe1100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe110066f30 0x7fe1100693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe12415a650 tx=0x7fe114009040 comp rx=0 tx=0).stop 2023-10-06T05:24:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 msgr2=0x7fe1240c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 0x7fe1240c9d10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe11800d890 tx=0x7fe118008040 comp rx=0 tx=0).stop 2023-10-06T05:24:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 shutdown_connections 2023-10-06T05:24:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe110066f30 0x7fe1100693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1240b2430 0x7fe1240b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1240af0a0 0x7fe12416a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 --2- 172.21.15.33:0/2883301554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1240ccfc0 0x7fe1240c9d10 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 >> 172.21.15.33:0/2883301554 conn(0x7fe12406cff0 msgr2=0x7fe1240c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 shutdown_connections 2023-10-06T05:24:20.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.037+0000 7fe12939d700 1 -- 172.21.15.33:0/2883301554 wait complete. 2023-10-06T05:24:20.043 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 98 2023-10-06T05:24:20.058 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:24:20.058 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:24:20.059 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:24:20.059 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:20.059 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:24:20.059 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:24:20.059 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:24:20.060 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:20.060 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:24:20.060 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:24:20.060 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:24:20.060 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:20.060 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:24:20.060 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:24:20.061 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:24:20.061 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:20.072 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:24:20.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:24:20.073 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:24:20.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:24:20.074 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:24:20.074 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:24:20.075 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:24:20.076 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:24:20.117 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:24:20.118 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:20.119 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:24:20.119 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:20.120 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:24:20.120 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:20.122 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:24:20.123 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:20.123 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:24:20.253 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:24:20.254 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:24:20.256 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:24:20.256 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:24:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.322+0000 7fd2b0d9d700 1 Processor -- start 2023-10-06T05:24:20.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.322+0000 7fd2b0d9d700 1 -- start start 2023-10-06T05:24:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.323+0000 7fd2b0d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2ac06cc70 0x7fd2ac1644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.323+0000 7fd2b0d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0b1160 con 0x7fd2ac159400 2023-10-06T05:24:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.323+0000 7fd2b0d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0d0ed0 con 0x7fd2ac1597d0 2023-10-06T05:24:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.323+0000 7fd2b0d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac0cb6d0 con 0x7fd2ac06cc70 2023-10-06T05:24:20.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2abfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2ac06cc70 0x7fd2ac1644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2ab7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd2ac159400 0x7fd2ac0cab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43092/0 (socket says 172.21.15.33:43092) 2023-10-06T05:24:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2ab7fe700 1 -- 172.21.15.33:0/794257779 learned_addr learned my addr 172.21.15.33:0/794257779 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2abfff700 1 -- 172.21.15.33:0/794257779 >> v1:172.21.15.167:6789/0 conn(0x7fd2ac1597d0 legacy=0x7fd2ac06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2aa7fc700 1 -- 172.21.15.33:0/794257779 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750463017 0 0) 0x7fd2ac0d0ed0 con 0x7fd2ac1597d0 2023-10-06T05:24:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2abfff700 1 -- 172.21.15.33:0/794257779 >> v1:172.21.15.33:6789/0 conn(0x7fd2ac159400 legacy=0x7fd2ac0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2abfff700 1 -- 172.21.15.33:0/794257779 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0cb9b0 con 0x7fd2ac06cc70 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.324+0000 7fd2abfff700 1 --2- 172.21.15.33:0/794257779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac06cc70 0x7fd2ac1644d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009e60 tx=0x7fd2a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bec599fe4312352 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2aa7fc700 1 -- 172.21.15.33:0/794257779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400e070 con 0x7fd2ac06cc70 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2aa7fc700 1 -- 172.21.15.33:0/794257779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400bc30 con 0x7fd2ac06cc70 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2aa7fc700 1 -- 172.21.15.33:0/794257779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4012430 con 0x7fd2ac06cc70 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/794257779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac06cc70 msgr2=0x7fd2ac1644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/794257779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac06cc70 0x7fd2ac1644d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009e60 tx=0x7fd2a4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/794257779 shutdown_connections 2023-10-06T05:24:20.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/794257779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac06cc70 0x7fd2ac1644d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/794257779 >> 172.21.15.33:0/794257779 conn(0x7fd2ac06e4b0 msgr2=0x7fd2ac0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/794257779 shutdown_connections 2023-10-06T05:24:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.325+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/794257779 wait complete. 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.327+0000 7fd2b0d9d700 1 Processor -- start 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.327+0000 7fd2b0d9d700 1 -- start start 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2b0d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2ac06cc70 0x7fd2ac0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2b0d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac159400 0x7fd2ac0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2b0d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 0x7fd2ac152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2b0d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0c76e0 con 0x7fd2ac1597d0 2023-10-06T05:24:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2b0d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0b3800 con 0x7fd2ac159400 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2b0d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0b3aa0 con 0x7fd2ac06cc70 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 0x7fd2ac152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 0x7fd2ac152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37272/0 (socket says 172.21.15.33:37272) 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 -- 172.21.15.33:0/382655811 learned_addr learned my addr 172.21.15.33:0/382655811 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2ab7fe700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2ac06cc70 0x7fd2ac0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2aaffd700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac159400 0x7fd2ac0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 -- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2ac06cc70 msgr2=0x7fd2ac0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2ac06cc70 0x7fd2ac0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 -- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac159400 msgr2=0x7fd2ac0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac159400 0x7fd2ac0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.328+0000 7fd2abfff700 1 -- 172.21.15.33:0/382655811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4009a20 con 0x7fd2ac1597d0 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.329+0000 7fd2abfff700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 0x7fd2ac152780 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4013000 tx=0x7fd2a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.329+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a401b420 con 0x7fd2ac1597d0 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.329+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a401b580 con 0x7fd2ac1597d0 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.329+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a401b920 con 0x7fd2ac1597d0 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.329+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac0b3d80 con 0x7fd2ac1597d0 2023-10-06T05:24:20.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.329+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac0b43c0 con 0x7fd2ac1597d0 2023-10-06T05:24:20.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.330+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac04c380 con 0x7fd2ac1597d0 2023-10-06T05:24:20.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.331+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2a401bc90 con 0x7fd2ac1597d0 2023-10-06T05:24:20.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.333+0000 7fd2937fe700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd298066f30 0x7fd2980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.333+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fd2a40862e0 con 0x7fd2ac1597d0 2023-10-06T05:24:20.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.333+0000 7fd2ab7fe700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd298066f30 0x7fd2980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.334+0000 7fd2ab7fe700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd298066f30 0x7fd2980693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd29c007130 tx=0x7fd29c007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.337+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2a40535f0 con 0x7fd2ac1597d0 2023-10-06T05:24:20.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.554+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd2ac0cbc90 con 0x7fd2ac1597d0 2023-10-06T05:24:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.976+0000 7fd2937fe700 1 -- 172.21.15.33:0/382655811 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd2a4056c10 con 0x7fd2ac1597d0 2023-10-06T05:24:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd298066f30 msgr2=0x7fd2980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd298066f30 0x7fd2980693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd29c007130 tx=0x7fd29c007eb0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 msgr2=0x7fd2ac152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 0x7fd2ac152780 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4013000 tx=0x7fd2a4019000 comp rx=0 tx=0).stop 2023-10-06T05:24:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 shutdown_connections 2023-10-06T05:24:20.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd298066f30 0x7fd2980693e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2ac1597d0 0x7fd2ac152780 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2ac06cc70 0x7fd2ac0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 --2- 172.21.15.33:0/382655811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac159400 0x7fd2ac0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 >> 172.21.15.33:0/382655811 conn(0x7fd2ac06e4b0 msgr2=0x7fd2ac0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 shutdown_connections 2023-10-06T05:24:20.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:20.979+0000 7fd2b0d9d700 1 -- 172.21.15.33:0/382655811 wait complete. 2023-10-06T05:24:20.983 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:24:21.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:24:21.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.186+0000 7fe73d79d700 1 Processor -- start 2023-10-06T05:24:21.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.187+0000 7fe73d79d700 1 -- start start 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe73d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe73d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7380adf50 0x7fe7380c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe73d79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7380c8e40 con 0x7fe7380c9810 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe73d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7380ae6c0 con 0x7fe7380adf50 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe73d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380ae9a0 con 0x7fe738155140 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe7377fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7380adf50 0x7fe7380c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe7377fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7380adf50 0x7fe7380c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37276/0 (socket says 172.21.15.33:37276) 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe73cf9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7380c9810 0x7fe7380699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43098/0 (socket says 172.21.15.33:43098) 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe7377fe700 1 -- 172.21.15.33:0/2362062230 learned_addr learned my addr 172.21.15.33:0/2362062230 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.188+0000 7fe737fff700 1 --2- 172.21.15.33:0/2362062230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe7377fe700 1 -- 172.21.15.33:0/2362062230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 msgr2=0x7fe738158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe7377fe700 1 --2- 172.21.15.33:0/2362062230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe7377fe700 1 -- 172.21.15.33:0/2362062230 >> v1:172.21.15.33:6789/0 conn(0x7fe7380c9810 legacy=0x7fe7380699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe7377fe700 1 -- 172.21.15.33:0/2362062230 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380c8ad0 con 0x7fe7380adf50 2023-10-06T05:24:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe736ffd700 1 -- 172.21.15.33:0/2362062230 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203545746 0 0) 0x7fe7380c8e40 con 0x7fe7380c9810 2023-10-06T05:24:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe7377fe700 1 --2- 172.21.15.33:0/2362062230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380adf50 0x7fe7380c92d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0099b0 tx=0x7fe72c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8e3c4907fa8ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe736ffd700 1 -- 172.21.15.33:0/2362062230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00b860 con 0x7fe7380adf50 2023-10-06T05:24:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe736ffd700 1 -- 172.21.15.33:0/2362062230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c00b9c0 con 0x7fe7380adf50 2023-10-06T05:24:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.189+0000 7fe736ffd700 1 -- 172.21.15.33:0/2362062230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00bd60 con 0x7fe7380adf50 2023-10-06T05:24:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 -- 172.21.15.33:0/2362062230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380adf50 msgr2=0x7fe7380c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 --2- 172.21.15.33:0/2362062230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380adf50 0x7fe7380c92d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0099b0 tx=0x7fe72c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 -- 172.21.15.33:0/2362062230 shutdown_connections 2023-10-06T05:24:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 --2- 172.21.15.33:0/2362062230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 --2- 172.21.15.33:0/2362062230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380adf50 0x7fe7380c92d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 -- 172.21.15.33:0/2362062230 >> 172.21.15.33:0/2362062230 conn(0x7fe73806d030 msgr2=0x7fe73806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 -- 172.21.15.33:0/2362062230 shutdown_connections 2023-10-06T05:24:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.190+0000 7fe73d79d700 1 -- 172.21.15.33:0/2362062230 wait complete. 2023-10-06T05:24:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.191+0000 7fe73d79d700 1 Processor -- start 2023-10-06T05:24:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.192+0000 7fe73d79d700 1 -- start start 2023-10-06T05:24:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.192+0000 7fe73d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7380adf50 0x7fe7381580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe73d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 0x7fe7380c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe73d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe73d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe738159d60 con 0x7fe7380c9810 2023-10-06T05:24:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe73d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380c5fc0 con 0x7fe7380adf50 2023-10-06T05:24:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe73d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380c6290 con 0x7fe738155140 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 0x7fe7380c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 0x7fe7380c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37292/0 (socket says 172.21.15.33:37292) 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 -- 172.21.15.33:0/4039887691 learned_addr learned my addr 172.21.15.33:0/4039887691 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe73cf9c700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe737fff700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7380adf50 0x7fe7381580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 -- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 msgr2=0x7fe738152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 -- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7380adf50 msgr2=0x7fe7381580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7380adf50 0x7fe7381580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.193+0000 7fe7377fe700 1 -- 172.21.15.33:0/4039887691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0094f0 con 0x7fe7380c9810 2023-10-06T05:24:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.194+0000 7fe7377fe700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 0x7fe7380c5a80 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe738063610 tx=0x7fe72c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.194+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c011d70 con 0x7fe7380c9810 2023-10-06T05:24:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.194+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c024440 con 0x7fe7380c9810 2023-10-06T05:24:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.194+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c024800 con 0x7fe7380c9810 2023-10-06T05:24:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.194+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7380c6570 con 0x7fe7380c9810 2023-10-06T05:24:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.194+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe738153430 con 0x7fe7380c9810 2023-10-06T05:24:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.195+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73806baf0 con 0x7fe7380c9810 2023-10-06T05:24:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.196+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe72c01a420 con 0x7fe7380c9810 2023-10-06T05:24:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.197+0000 7fe71ffff700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe724066f30 0x7fe7240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.197+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe72c086300 con 0x7fe7380c9810 2023-10-06T05:24:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.197+0000 7fe737fff700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe724066f30 0x7fe7240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.198+0000 7fe737fff700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe724066f30 0x7fe7240693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe728006fd0 tx=0x7fe728008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.201+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe72c0536c0 con 0x7fe7380c9810 2023-10-06T05:24:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.485+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe73804c380 con 0x7fe7380c9810 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.486+0000 7fe71ffff700 1 -- 172.21.15.33:0/4039887691 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe72c056ce0 con 0x7fe7380c9810 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:24:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:24:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:24:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:24:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:24:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:24:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:24:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:24:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:24:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:24:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:24:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:24:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:24:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.503 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.504 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:24:21.504 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:24:21.504 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:24:21.505 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:24:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.507 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:24:21.507 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:24:21.507 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.508 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.508 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:24:21.509 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:24:21.509 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.509 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.509 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.510 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:24:21.510 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:24:21.510 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.511 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:24:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:24:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:24:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:24:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:24:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:21.518 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:24:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:24:21.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.488+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe724066f30 msgr2=0x7fe7240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.488+0000 7fe73d79d700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe724066f30 0x7fe7240693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe728006fd0 tx=0x7fe728008040 comp rx=0 tx=0).stop 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.488+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 msgr2=0x7fe7380c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.488+0000 7fe73d79d700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 0x7fe7380c5a80 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe738063610 tx=0x7fe72c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 shutdown_connections 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe724066f30 0x7fe7240693e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe738155140 0x7fe738152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7380adf50 0x7fe7381580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 --2- 172.21.15.33:0/4039887691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7380c9810 0x7fe7380c5a80 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 >> 172.21.15.33:0/4039887691 conn(0x7fe73806d030 msgr2=0x7fe7380afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 shutdown_connections 2023-10-06T05:24:21.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.489+0000 7fe73d79d700 1 -- 172.21.15.33:0/4039887691 wait complete. 2023-10-06T05:24:21.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.706+0000 7f6d8ffff700 1 Processor -- start 2023-10-06T05:24:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.706+0000 7f6d8ffff700 1 -- start start 2023-10-06T05:24:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.707+0000 7f6d8ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d900cf3b0 0x7f6d901594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.707+0000 7f6d8ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900cbd80 con 0x7f6d900af0a0 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.707+0000 7f6d8ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900af5f0 con 0x7f6d900ccfc0 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.707+0000 7f6d8ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900af8d0 con 0x7f6d900cf3b0 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.707+0000 7f6d8e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d900cf3b0 0x7f6d901594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d900cf3b0 0x7f6d901594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37302/0 (socket says 172.21.15.33:37302) 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8e7fc700 1 -- 172.21.15.33:0/2791064177 learned_addr learned my addr 172.21.15.33:0/2791064177 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d900af0a0 0x7f6d900b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43114/0 (socket says 172.21.15.33:43114) 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8e7fc700 1 -- 172.21.15.33:0/2791064177 >> v1:172.21.15.167:6789/0 conn(0x7f6d900ccfc0 legacy=0x7f6d900cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8e7fc700 1 -- 172.21.15.33:0/2791064177 >> v1:172.21.15.33:6789/0 conn(0x7f6d900af0a0 legacy=0x7f6d900b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8e7fc700 1 -- 172.21.15.33:0/2791064177 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900afbb0 con 0x7f6d900cf3b0 2023-10-06T05:24:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8e7fc700 1 --2- 172.21.15.33:0/2791064177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900cf3b0 0x7f6d901594a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6d840099e0 tx=0x7f6d84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded9e7f06cb09219 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.708+0000 7f6d8dffb700 1 -- 172.21.15.33:0/2791064177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400b820 con 0x7f6d900cf3b0 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8dffb700 1 -- 172.21.15.33:0/2791064177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8400b980 con 0x7f6d900cf3b0 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8dffb700 1 -- 172.21.15.33:0/2791064177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400bd20 con 0x7f6d900cf3b0 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 -- 172.21.15.33:0/2791064177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900cf3b0 msgr2=0x7f6d901594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/2791064177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900cf3b0 0x7f6d901594a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6d840099e0 tx=0x7f6d84009250 comp rx=0 tx=0).stop 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 -- 172.21.15.33:0/2791064177 shutdown_connections 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/2791064177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900cf3b0 0x7f6d901594a0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 -- 172.21.15.33:0/2791064177 >> 172.21.15.33:0/2791064177 conn(0x7f6d9006cff0 msgr2=0x7f6d9006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 -- 172.21.15.33:0/2791064177 shutdown_connections 2023-10-06T05:24:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.709+0000 7f6d8ffff700 1 -- 172.21.15.33:0/2791064177 wait complete. 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.711+0000 7f6d8ffff700 1 Processor -- start 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.711+0000 7f6d8ffff700 1 -- start start 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d900af0a0 0x7f6d9014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d900ccfc0 0x7f6d900b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 0x7f6d900b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9014b7b0 con 0x7f6d900c9af0 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900c8a50 con 0x7f6d900af0a0 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900c8d30 con 0x7f6d900ccfc0 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 0x7f6d900b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 0x7f6d900b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37310/0 (socket says 172.21.15.33:37310) 2023-10-06T05:24:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 -- 172.21.15.33:0/1421282812 learned_addr learned my addr 172.21.15.33:0/1421282812 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8e7fc700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d900ccfc0 0x7f6d900b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8effd700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d900af0a0 0x7f6d9014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 -- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d900ccfc0 msgr2=0x7f6d900b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d900ccfc0 0x7f6d900b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 -- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d900af0a0 msgr2=0x7f6d9014b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d900af0a0 0x7f6d9014b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.712+0000 7f6d8f7fe700 1 -- 172.21.15.33:0/1421282812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840094f0 con 0x7f6d900c9af0 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.713+0000 7f6d8f7fe700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 0x7f6d900b13f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800cc70 tx=0x7f6d880019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.713+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88005df0 con 0x7f6d900c9af0 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.713+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800a5c0 con 0x7f6d900c9af0 2023-10-06T05:24:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.713+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800a980 con 0x7f6d900c9af0 2023-10-06T05:24:21.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.713+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900c9010 con 0x7f6d900c9af0 2023-10-06T05:24:21.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.713+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900c95d0 con 0x7f6d900c9af0 2023-10-06T05:24:21.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.714+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9004c380 con 0x7f6d900c9af0 2023-10-06T05:24:21.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.715+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d8800acf0 con 0x7f6d900c9af0 2023-10-06T05:24:21.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.716+0000 7f6d76ffd700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d7c066f30 0x7f6d7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.716+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6d88084300 con 0x7f6d900c9af0 2023-10-06T05:24:21.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.717+0000 7f6d8effd700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d7c066f30 0x7f6d7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.717+0000 7f6d8effd700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d7c066f30 0x7f6d7c0693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6d900b56a0 tx=0x7f6d80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.720+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d880517f0 con 0x7f6d900c9af0 2023-10-06T05:24:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.923+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d900cc920 con 0x7f6d900c9af0 2023-10-06T05:24:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.924+0000 7f6d76ffd700 1 -- 172.21.15.33:0/1421282812 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f6d88054e10 con 0x7f6d900c9af0 2023-10-06T05:24:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:21.925 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":140,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:20.983247+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d7c066f30 msgr2=0x7f6d7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d7c066f30 0x7f6d7c0693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6d900b56a0 tx=0x7f6d80009040 comp rx=0 tx=0).stop 2023-10-06T05:24:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 msgr2=0x7f6d900b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 0x7f6d900b13f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800cc70 tx=0x7f6d880019f0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 shutdown_connections 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d7c066f30 0x7f6d7c0693e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d900c9af0 0x7f6d900b13f0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d900af0a0 0x7f6d9014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 --2- 172.21.15.33:0/1421282812 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d900ccfc0 0x7f6d900b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 >> 172.21.15.33:0/1421282812 conn(0x7f6d9006cff0 msgr2=0x7f6d90155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 shutdown_connections 2023-10-06T05:24:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:21.926+0000 7f6d8ffff700 1 -- 172.21.15.33:0/1421282812 wait complete. 2023-10-06T05:24:21.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:24:21.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.140+0000 7fd54ef9d700 1 Processor -- start 2023-10-06T05:24:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.140+0000 7fd54ef9d700 1 -- start start 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.141+0000 7fd54ef9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd548156370 0x7fd5480af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54ef9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5480cc7e0 0x7fd5480c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54ef9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5480ad430 con 0x7fd5480b01f0 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54ef9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5480afe20 con 0x7fd5480cc7e0 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54ef9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54806bc70 con 0x7fd548156370 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5480cc7e0 0x7fd5480c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5480cc7e0 0x7fd5480c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37312/0 (socket says 172.21.15.33:37312) 2023-10-06T05:24:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 -- 172.21.15.33:0/3919805000 learned_addr learned my addr 172.21.15.33:0/3919805000 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54e79c700 1 --1- 172.21.15.33:0/3919805000 >> v1:172.21.15.33:6789/0 conn(0x7fd5480b01f0 0x7fd5480ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43116/0 (socket says 172.21.15.33:43116) 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54df9b700 1 --2- 172.21.15.33:0/3919805000 >> v2:172.21.15.167:3300/0 conn(0x7fd548156370 0x7fd5480af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 -- 172.21.15.33:0/3919805000 >> v2:172.21.15.167:3300/0 conn(0x7fd548156370 msgr2=0x7fd5480af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 --2- 172.21.15.33:0/3919805000 >> v2:172.21.15.167:3300/0 conn(0x7fd548156370 0x7fd5480af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 -- 172.21.15.33:0/3919805000 >> v1:172.21.15.33:6789/0 conn(0x7fd5480b01f0 legacy=0x7fd5480ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54d79a700 1 -- 172.21.15.33:0/3919805000 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480c92f0 con 0x7fd5480cc7e0 2023-10-06T05:24:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.142+0000 7fd54cf99700 1 -- 172.21.15.33:0/3919805000 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650428968 0 0) 0x7fd5480ad430 con 0x7fd5480b01f0 2023-10-06T05:24:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.143+0000 7fd54d79a700 1 --2- 172.21.15.33:0/3919805000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5480cc7e0 0x7fd5480c8db0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd5480b21a0 tx=0x7fd538009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa99ac2b1389989e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.143+0000 7fd54cf99700 1 -- 172.21.15.33:0/3919805000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538009e70 con 0x7fd5480cc7e0 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.143+0000 7fd54cf99700 1 -- 172.21.15.33:0/3919805000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53800b860 con 0x7fd5480cc7e0 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.143+0000 7fd54cf99700 1 -- 172.21.15.33:0/3919805000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800bc00 con 0x7fd5480cc7e0 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.143+0000 7fd54ef9d700 1 -- 172.21.15.33:0/3919805000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5480cc7e0 msgr2=0x7fd5480c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.143+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/3919805000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5480cc7e0 0x7fd5480c8db0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd5480b21a0 tx=0x7fd538009250 comp rx=0 tx=0).stop 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.144+0000 7fd54ef9d700 1 -- 172.21.15.33:0/3919805000 shutdown_connections 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.144+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/3919805000 >> v2:172.21.15.167:3300/0 conn(0x7fd548156370 0x7fd5480af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.144+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/3919805000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5480cc7e0 0x7fd5480c8db0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.144+0000 7fd54ef9d700 1 -- 172.21.15.33:0/3919805000 >> 172.21.15.33:0/3919805000 conn(0x7fd54806d030 msgr2=0x7fd5480cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.144+0000 7fd54ef9d700 1 -- 172.21.15.33:0/3919805000 shutdown_connections 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.144+0000 7fd54ef9d700 1 -- 172.21.15.33:0/3919805000 wait complete. 2023-10-06T05:24:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.145+0000 7fd54ef9d700 1 Processor -- start 2023-10-06T05:24:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.145+0000 7fd54ef9d700 1 -- start start 2023-10-06T05:24:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5480b01f0 0x7fd548154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54ef9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5480cc7e0 0x7fd54815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54ef9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 0x7fd54806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54ef9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54815cb20 con 0x7fd548156370 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54ef9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54815ce00 con 0x7fd5480cc7e0 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54815d0e0 con 0x7fd5480b01f0 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54e79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 0x7fd54806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54e79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 0x7fd54806e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37318/0 (socket says 172.21.15.33:37318) 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.146+0000 7fd54e79c700 1 -- 172.21.15.33:0/2783328639 learned_addr learned my addr 172.21.15.33:0/2783328639 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54d79a700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5480cc7e0 0x7fd54815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54df9b700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5480b01f0 0x7fd548154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54e79c700 1 -- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5480b01f0 msgr2=0x7fd548154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54e79c700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5480b01f0 0x7fd548154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54e79c700 1 -- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5480cc7e0 msgr2=0x7fd54815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54e79c700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5480cc7e0 0x7fd54815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54e79c700 1 -- 172.21.15.33:0/2783328639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380094f0 con 0x7fd548156370 2023-10-06T05:24:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54df9b700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5480b01f0 0x7fd548154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54d79a700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5480cc7e0 0x7fd54815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54e79c700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 0x7fd54806e0c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd54000cd50 tx=0x7fd540001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540005df0 con 0x7fd548156370 2023-10-06T05:24:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54000a960 con 0x7fd548156370 2023-10-06T05:24:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000ad20 con 0x7fd548156370 2023-10-06T05:24:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.147+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54815d3c0 con 0x7fd548156370 2023-10-06T05:24:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.148+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54815c7b0 con 0x7fd548156370 2023-10-06T05:24:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.149+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54815d8e0 con 0x7fd548156370 2023-10-06T05:24:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.149+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd540007300 con 0x7fd548156370 2023-10-06T05:24:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.150+0000 7fd53dffb700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd52c066f30 0x7fd52c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.150+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd5400845f0 con 0x7fd548156370 2023-10-06T05:24:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.151+0000 7fd54df9b700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd52c066f30 0x7fd52c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.152+0000 7fd54df9b700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd52c066f30 0x7fd52c0693e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd544006f00 tx=0x7fd544008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.154+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd54815d8e0 con 0x7fd548156370 2023-10-06T05:24:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.357+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd54815d8e0 con 0x7fd548156370 2023-10-06T05:24:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.357+0000 7fd53dffb700 1 -- 172.21.15.33:0/2783328639 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd54815d8e0 con 0x7fd548156370 2023-10-06T05:24:22.359 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd52c066f30 msgr2=0x7fd52c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd52c066f30 0x7fd52c0693e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd544006f00 tx=0x7fd544008040 comp rx=0 tx=0).stop 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 msgr2=0x7fd54806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 0x7fd54806e0c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd54000cd50 tx=0x7fd540001b10 comp rx=0 tx=0).stop 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 shutdown_connections 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd52c066f30 0x7fd52c0693e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd548156370 0x7fd54806e0c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5480b01f0 0x7fd548154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 --2- 172.21.15.33:0/2783328639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5480cc7e0 0x7fd54815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 >> 172.21.15.33:0/2783328639 conn(0x7fd54806d030 msgr2=0x7fd5480cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 shutdown_connections 2023-10-06T05:24:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.360+0000 7fd54ef9d700 1 -- 172.21.15.33:0/2783328639 wait complete. 2023-10-06T05:24:22.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:22.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.568+0000 7f36d4d9d700 1 Processor -- start 2023-10-06T05:24:22.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.568+0000 7f36d4d9d700 1 -- start start 2023-10-06T05:24:22.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36d4d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36d00cc7e0 0x7f36d00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36d4d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36d00b01f0 0x7f36d00b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36d4d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d00ad430 con 0x7f36d0156370 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36d4d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d00afe20 con 0x7f36d00cc7e0 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36d4d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d00b0b70 con 0x7f36d00b01f0 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36ceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36d00cc7e0 0x7f36d00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36ceffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36d00cc7e0 0x7f36d00af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37320/0 (socket says 172.21.15.33:37320) 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36ceffd700 1 -- 172.21.15.33:0/3600929293 learned_addr learned my addr 172.21.15.33:0/3600929293 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36cf7fe700 1 --1- 172.21.15.33:0/3600929293 >> v1:172.21.15.33:6789/0 conn(0x7f36d0156370 0x7f36d00c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43118/0 (socket says 172.21.15.33:43118) 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.570+0000 7f36cffff700 1 --2- 172.21.15.33:0/3600929293 >> v2:172.21.15.167:3300/0 conn(0x7f36d00b01f0 0x7f36d00b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ceffd700 1 -- 172.21.15.33:0/3600929293 >> v2:172.21.15.167:3300/0 conn(0x7f36d00b01f0 msgr2=0x7f36d00b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ceffd700 1 --2- 172.21.15.33:0/3600929293 >> v2:172.21.15.167:3300/0 conn(0x7f36d00b01f0 0x7f36d00b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ceffd700 1 -- 172.21.15.33:0/3600929293 >> v1:172.21.15.33:6789/0 conn(0x7f36d0156370 legacy=0x7f36d00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ceffd700 1 -- 172.21.15.33:0/3600929293 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d006bc70 con 0x7f36d00cc7e0 2023-10-06T05:24:22.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ce7fc700 1 -- 172.21.15.33:0/3600929293 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620997025 0 0) 0x7f36d00ad430 con 0x7f36d0156370 2023-10-06T05:24:22.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ceffd700 1 --2- 172.21.15.33:0/3600929293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d00cc7e0 0x7f36d00af8e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f36d00b13b0 tx=0x7f36c400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377b0bd1009e199f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ce7fc700 1 -- 172.21.15.33:0/3600929293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c400dba0 con 0x7f36d00cc7e0 2023-10-06T05:24:22.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ce7fc700 1 -- 172.21.15.33:0/3600929293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c400dd00 con 0x7f36d00cc7e0 2023-10-06T05:24:22.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.571+0000 7f36ce7fc700 1 -- 172.21.15.33:0/3600929293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c40196d0 con 0x7f36d00cc7e0 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 -- 172.21.15.33:0/3600929293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d00cc7e0 msgr2=0x7f36d00af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/3600929293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d00cc7e0 0x7f36d00af8e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f36d00b13b0 tx=0x7f36c400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 -- 172.21.15.33:0/3600929293 shutdown_connections 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/3600929293 >> v2:172.21.15.167:3300/0 conn(0x7f36d00b01f0 0x7f36d00b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/3600929293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d00cc7e0 0x7f36d00af8e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 -- 172.21.15.33:0/3600929293 >> 172.21.15.33:0/3600929293 conn(0x7f36d006d030 msgr2=0x7f36d00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 -- 172.21.15.33:0/3600929293 shutdown_connections 2023-10-06T05:24:22.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.572+0000 7f36d4d9d700 1 -- 172.21.15.33:0/3600929293 wait complete. 2023-10-06T05:24:22.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.573+0000 7f36d4d9d700 1 Processor -- start 2023-10-06T05:24:22.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.573+0000 7f36d4d9d700 1 -- start start 2023-10-06T05:24:22.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36d4d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36d00b01f0 0x7f36d0150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36d4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36d00cc7e0 0x7f36d0152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36d4d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 0x7f36d0169b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36d4d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d01505b0 con 0x7f36d0156370 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36d4d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0157a80 con 0x7f36d00b01f0 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36d4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0152cd0 con 0x7f36d00cc7e0 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 0x7f36d0169b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 0x7f36d0169b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37332/0 (socket says 172.21.15.33:37332) 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.574+0000 7f36cffff700 1 -- 172.21.15.33:0/177693515 learned_addr learned my addr 172.21.15.33:0/177693515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36cffff700 1 -- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36d00cc7e0 msgr2=0x7f36d0152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36cffff700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36d00cc7e0 0x7f36d0152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36cffff700 1 -- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36d00b01f0 msgr2=0x7f36d0150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36cffff700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36d00b01f0 0x7f36d0150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36cffff700 1 -- 172.21.15.33:0/177693515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c400b790 con 0x7f36d0156370 2023-10-06T05:24:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36cffff700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 0x7f36d0169b10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009df0 tx=0x7f36c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800cb50 con 0x7f36d0156370 2023-10-06T05:24:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c800f040 con 0x7f36d0156370 2023-10-06T05:24:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d0155630 con 0x7f36d0156370 2023-10-06T05:24:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.575+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8014480 con 0x7f36d0156370 2023-10-06T05:24:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.576+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d0158050 con 0x7f36d0156370 2023-10-06T05:24:22.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.576+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d004c380 con 0x7f36d0156370 2023-10-06T05:24:22.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.577+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36c8014870 con 0x7f36d0156370 2023-10-06T05:24:22.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.578+0000 7f36b77fe700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36bc066f30 0x7f36bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.578+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f36c8084c40 con 0x7f36d0156370 2023-10-06T05:24:22.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.579+0000 7f36cf7fe700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36bc066f30 0x7f36bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.579+0000 7f36cf7fe700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36bc066f30 0x7f36bc0693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f36c00011e0 tx=0x7f36c00079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.583+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36c80520a0 con 0x7f36d0156370 2023-10-06T05:24:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.783+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36d006f310 con 0x7f36d0156370 2023-10-06T05:24:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.783+0000 7f36b77fe700 1 -- 172.21.15.33:0/177693515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v140) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36c8018020 con 0x7f36d0156370 2023-10-06T05:24:22.785 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:24:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.785+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36bc066f30 msgr2=0x7f36bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.785+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36bc066f30 0x7f36bc0693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f36c00011e0 tx=0x7f36c00079b0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.785+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 msgr2=0x7f36d0169b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.785+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 0x7f36d0169b10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009df0 tx=0x7f36c800b040 comp rx=0 tx=0).stop 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 shutdown_connections 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36bc066f30 0x7f36bc0693e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36d0156370 0x7f36d0169b10 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36d00b01f0 0x7f36d0150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 --2- 172.21.15.33:0/177693515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36d00cc7e0 0x7f36d0152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 >> 172.21.15.33:0/177693515 conn(0x7f36d006d030 msgr2=0x7f36d0151e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 shutdown_connections 2023-10-06T05:24:22.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.786+0000 7f36d4d9d700 1 -- 172.21.15.33:0/177693515 wait complete. 2023-10-06T05:24:22.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:24:22.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.990+0000 7fc218b9d700 1 Processor -- start 2023-10-06T05:24:22.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.991+0000 7fc218b9d700 1 -- start start 2023-10-06T05:24:22.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc218b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc2140cf3b0 0x7fc2141594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc218b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2140af0a0 0x7fc2141599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc218b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140cbd80 con 0x7fc2140ccfc0 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc218b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2140af5f0 con 0x7fc2140af0a0 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc218b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2140af8d0 con 0x7fc2140cf3b0 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc213fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2140af0a0 0x7fc2141599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc213fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2140af0a0 0x7fc2141599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37334/0 (socket says 172.21.15.33:37334) 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc213fff700 1 -- 172.21.15.33:0/2691368739 learned_addr learned my addr 172.21.15.33:0/2691368739 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:22.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.992+0000 7fc212ffd700 1 --2- 172.21.15.33:0/2691368739 >> v2:172.21.15.167:3300/0 conn(0x7fc2140cf3b0 0x7fc2141594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc213fff700 1 -- 172.21.15.33:0/2691368739 >> v2:172.21.15.167:3300/0 conn(0x7fc2140cf3b0 msgr2=0x7fc2141594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc213fff700 1 --2- 172.21.15.33:0/2691368739 >> v2:172.21.15.167:3300/0 conn(0x7fc2140cf3b0 0x7fc2141594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc213fff700 1 -- 172.21.15.33:0/2691368739 >> v1:172.21.15.167:6789/0 conn(0x7fc2140ccfc0 legacy=0x7fc2140cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc213fff700 1 -- 172.21.15.33:0/2691368739 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140afbb0 con 0x7fc2140af0a0 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc213fff700 1 --2- 172.21.15.33:0/2691368739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2141599e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009f20 tx=0x7fc20c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1ee389bb5e74df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc2127fc700 1 -- 172.21.15.33:0/2691368739 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00e070 con 0x7fc2140af0a0 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc2127fc700 1 -- 172.21.15.33:0/2691368739 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00ba40 con 0x7fc2140af0a0 2023-10-06T05:24:22.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.993+0000 7fc2127fc700 1 -- 172.21.15.33:0/2691368739 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0092e0 con 0x7fc2140af0a0 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 -- 172.21.15.33:0/2691368739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 msgr2=0x7fc2141599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 --2- 172.21.15.33:0/2691368739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2141599e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009f20 tx=0x7fc20c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 -- 172.21.15.33:0/2691368739 shutdown_connections 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 --2- 172.21.15.33:0/2691368739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2141599e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 --2- 172.21.15.33:0/2691368739 >> v2:172.21.15.167:3300/0 conn(0x7fc2140cf3b0 0x7fc2141594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 -- 172.21.15.33:0/2691368739 >> 172.21.15.33:0/2691368739 conn(0x7fc21406cff0 msgr2=0x7fc21406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 -- 172.21.15.33:0/2691368739 shutdown_connections 2023-10-06T05:24:22.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.994+0000 7fc218b9d700 1 -- 172.21.15.33:0/2691368739 wait complete. 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.995+0000 7fc218b9d700 1 Processor -- start 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 -- start start 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2140b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2140ccfc0 0x7fc2140c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2140c92c0 0x7fc214153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214169680 con 0x7fc2140af0a0 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140b4500 con 0x7fc2140c92c0 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.996+0000 7fc218b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140b47e0 con 0x7fc2140ccfc0 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2140b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2140b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37336/0 (socket says 172.21.15.33:37336) 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 -- 172.21.15.33:0/1286902723 learned_addr learned my addr 172.21.15.33:0/1286902723 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc213fff700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2140c92c0 0x7fc214153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc212ffd700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2140ccfc0 0x7fc2140c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 -- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2140ccfc0 msgr2=0x7fc2140c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2140ccfc0 0x7fc2140c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 -- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2140c92c0 msgr2=0x7fc214153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2140c92c0 0x7fc214153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc2137fe700 1 -- 172.21.15.33:0/1286902723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c009ae0 con 0x7fc2140af0a0 2023-10-06T05:24:23.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc212ffd700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2140ccfc0 0x7fc2140c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.997+0000 7fc213fff700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2140c92c0 0x7fc214153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.998+0000 7fc2137fe700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2140b3cb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c4e0 tx=0x7fc20400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.998+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204011590 con 0x7fc2140af0a0 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.998+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc204011710 con 0x7fc2140af0a0 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.998+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204011ad0 con 0x7fc2140af0a0 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.998+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2140b4ac0 con 0x7fc2140af0a0 2023-10-06T05:24:23.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.998+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2140b41f0 con 0x7fc2140af0a0 2023-10-06T05:24:23.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:22.999+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc214159f20 con 0x7fc2140af0a0 2023-10-06T05:24:23.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.000+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc20401b3b0 con 0x7fc2140af0a0 2023-10-06T05:24:23.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.001+0000 7fc1fb7fe700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc200066f30 0x7fc2000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.001+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc2040838a0 con 0x7fc2140af0a0 2023-10-06T05:24:23.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.001+0000 7fc212ffd700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc200066f30 0x7fc2000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.002+0000 7fc212ffd700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc200066f30 0x7fc2000693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc214155e40 tx=0x7fc208008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.005+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc204050c60 con 0x7fc2140af0a0 2023-10-06T05:24:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.210+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc21404c380 con 0x7fc2140af0a0 2023-10-06T05:24:23.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.210+0000 7fc1fb7fe700 1 -- 172.21.15.33:0/1286902723 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v140) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc204017070 con 0x7fc2140af0a0 2023-10-06T05:24:23.212 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:23.212 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:23.213 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:24:23.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc200066f30 msgr2=0x7fc2000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc200066f30 0x7fc2000693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc214155e40 tx=0x7fc208008040 comp rx=0 tx=0).stop 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 msgr2=0x7fc2140b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2140b3cb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c4e0 tx=0x7fc20400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 shutdown_connections 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc200066f30 0x7fc2000693e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2140c92c0 0x7fc214153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2140af0a0 0x7fc2140b3cb0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 --2- 172.21.15.33:0/1286902723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2140ccfc0 0x7fc2140c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 >> 172.21.15.33:0/1286902723 conn(0x7fc21406cff0 msgr2=0x7fc2141570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 shutdown_connections 2023-10-06T05:24:23.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.213+0000 7fc218b9d700 1 -- 172.21.15.33:0/1286902723 wait complete. 2023-10-06T05:24:23.232 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:24:23.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:24:23.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.422+0000 7fb4c619d700 1 Processor -- start 2023-10-06T05:24:23.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.423+0000 7fb4c619d700 1 -- start start 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4c00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c619d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00a9dd0 con 0x7fb4c0159400 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c619d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c006d760 con 0x7fb4c006dbf0 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00aa8c0 con 0x7fb4c0068260 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c519b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb4c0159400 0x7fb4c0067b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43128/0 (socket says 172.21.15.33:43128) 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c519b700 1 -- 172.21.15.33:0/1725105226 learned_addr learned my addr 172.21.15.33:0/1725105226 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.424+0000 7fb4c499a700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4c00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328682494 0 0) 0x7fb4bc001110 con 0x7fb4c0159400 2023-10-06T05:24:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ac003570 con 0x7fb4c0159400 2023-10-06T05:24:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083971631 0 0) 0x7fb4ac003570 con 0x7fb4c0159400 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 msgr2=0x7fb4c00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4c00a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 >> v1:172.21.15.167:6789/0 conn(0x7fb4c006dbf0 legacy=0x7fb4c00af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00aa5e0 con 0x7fb4c0159400 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb4bc001600 con 0x7fb4c0159400 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 >> v1:172.21.15.33:6789/0 conn(0x7fb4c0159400 legacy=0x7fb4c0067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac003a90 0x7fb4ac003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac004d50 0x7fb4ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00aa8c0 con 0x7fb4ac003a90 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00bb10 con 0x7fb4ac004d50 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00bcd0 con 0x7fb4c0068260 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.425+0000 7fb4c519b700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac003a90 0x7fb4ac003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c599c700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac004d50 0x7fb4ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c499a700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c519b700 1 -- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 msgr2=0x7fb4ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c519b700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c519b700 1 -- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac004d50 msgr2=0x7fb4ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c519b700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac004d50 0x7fb4ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c519b700 1 -- 172.21.15.33:0/1725105226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac003570 con 0x7fb4ac003a90 2023-10-06T05:24:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c499a700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c0068260 0x7fb4ac004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4c519b700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac003a90 0x7fb4ac003e60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00c440 tx=0x7fb4bc0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c4dbb1fd555779 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00cd70 con 0x7fb4ac003a90 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc005920 con 0x7fb4ac003a90 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.426+0000 7fb4b7fff700 1 -- 172.21.15.33:0/1725105226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc005ce0 con 0x7fb4ac003a90 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c619d700 1 -- 172.21.15.33:0/1725105226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac003a90 msgr2=0x7fb4ac003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c619d700 1 --2- 172.21.15.33:0/1725105226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac003a90 0x7fb4ac003e60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00c440 tx=0x7fb4bc0079d0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c599c700 1 -- 172.21.15.33:0/1725105226 reap_dead start 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c619d700 1 -- 172.21.15.33:0/1725105226 shutdown_connections 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c619d700 1 -- 172.21.15.33:0/1725105226 >> 172.21.15.33:0/1725105226 conn(0x7fb4c006ebf0 msgr2=0x7fb4c006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c619d700 1 -- 172.21.15.33:0/1725105226 shutdown_connections 2023-10-06T05:24:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.427+0000 7fb4c619d700 1 -- 172.21.15.33:0/1725105226 wait complete. 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.428+0000 7fb4c619d700 1 Processor -- start 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.428+0000 7fb4c619d700 1 -- start start 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.429+0000 7fb4c619d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac003a90 0x7fb4c00af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.429+0000 7fb4c619d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 0x7fb4c0155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.429+0000 7fb4c619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c00aca20 0x7fb4c00acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.429+0000 7fb4c619d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00ad400 con 0x7fb4c00ca980 2023-10-06T05:24:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.429+0000 7fb4c619d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c016da70 con 0x7fb4ac003a90 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.429+0000 7fb4c619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c015d480 con 0x7fb4c00aca20 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 0x7fb4c0155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 0x7fb4c0155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37356/0 (socket says 172.21.15.33:37356) 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 -- 172.21.15.33:0/3866344738 learned_addr learned my addr 172.21.15.33:0/3866344738 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c519b700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac003a90 0x7fb4c00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c599c700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c00aca20 0x7fb4c00acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 -- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c00aca20 msgr2=0x7fb4c00acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c00aca20 0x7fb4c00acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 -- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac003a90 msgr2=0x7fb4c00af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac003a90 0x7fb4c00af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 -- 172.21.15.33:0/3866344738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c015d760 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c519b700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac003a90 0x7fb4c00af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.430+0000 7fb4c499a700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 0x7fb4c0155090 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00119c0 tx=0x7fb4b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.431+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0009600 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.431+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b000f8e0 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.431+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c00cb1b0 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.431+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000fca0 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.431+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c00cb770 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.432+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c015d8f0 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.433+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4b0015670 con 0x7fb4c00ca980 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.433+0000 7fb4b57fa700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a4066f30 0x7fb4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.433+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb4b0084b10 con 0x7fb4c00ca980 2023-10-06T05:24:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.434+0000 7fb4c519b700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a4066f30 0x7fb4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.435+0000 7fb4c519b700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a4066f30 0x7fb4a40693e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cbd10 tx=0x7fb4bc015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.437+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4b0051ed0 con 0x7fb4c00ca980 2023-10-06T05:24:23.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.639+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb4c0063ff0 con 0x7fb4c00ca980 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.991+0000 7fb4b57fa700 1 -- 172.21.15.33:0/3866344738 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v141) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb4b00554f0 con 0x7fb4c00ca980 2023-10-06T05:24:23.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a4066f30 msgr2=0x7fb4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a4066f30 0x7fb4a40693e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cbd10 tx=0x7fb4bc015000 comp rx=0 tx=0).stop 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 msgr2=0x7fb4c0155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 0x7fb4c0155090 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00119c0 tx=0x7fb4b0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 shutdown_connections 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4a4066f30 0x7fb4a40693e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4c00aca20 0x7fb4c00acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4ac003a90 0x7fb4c00af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 --2- 172.21.15.33:0/3866344738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4c00ca980 0x7fb4c0155090 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.996+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 >> 172.21.15.33:0/3866344738 conn(0x7fb4c006ebf0 msgr2=0x7fb4c00b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.997+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 shutdown_connections 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:23.997+0000 7fb4c619d700 1 -- 172.21.15.33:0/3866344738 wait complete. 2023-10-06T05:24:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:24:24.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.212+0000 7f97ad99d700 1 Processor -- start 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.212+0000 7f97ad99d700 1 -- start start 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.213+0000 7f97ad99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97a80cc600 0x7f97a80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ad99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97a806c560 0x7f97a806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ad99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a806cee0 con 0x7f97a80ccf30 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ad99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a8159d10 con 0x7f97a80cc600 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ad99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a80b0350 con 0x7f97a806c560 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97a80cc600 0x7f97a80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97a80cc600 0x7f97a80cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37366/0 (socket says 172.21.15.33:37366) 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 -- 172.21.15.33:0/760891792 learned_addr learned my addr 172.21.15.33:0/760891792 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ad19c700 1 --2- 172.21.15.33:0/760891792 >> v2:172.21.15.167:3300/0 conn(0x7f97a806c560 0x7f97a806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 -- 172.21.15.33:0/760891792 >> v2:172.21.15.167:3300/0 conn(0x7f97a806c560 msgr2=0x7f97a806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 --2- 172.21.15.33:0/760891792 >> v2:172.21.15.167:3300/0 conn(0x7f97a806c560 0x7f97a806c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 -- 172.21.15.33:0/760891792 >> v1:172.21.15.167:6789/0 conn(0x7f97a80ccf30 legacy=0x7f97a8067e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.214+0000 7f97ac99b700 1 -- 172.21.15.33:0/760891792 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80b0630 con 0x7f97a80cc600 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.215+0000 7f97ac99b700 1 --2- 172.21.15.33:0/760891792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80cc600 0x7f97a80cc9d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f97a80b10c0 tx=0x7f979c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c3e8c9e8c4173a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.215+0000 7f97a77fe700 1 -- 172.21.15.33:0/760891792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00b860 con 0x7f97a80cc600 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.215+0000 7f97a77fe700 1 -- 172.21.15.33:0/760891792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00b9c0 con 0x7f97a80cc600 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.215+0000 7f97a77fe700 1 -- 172.21.15.33:0/760891792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00bd60 con 0x7f97a80cc600 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.215+0000 7f97ad99d700 1 -- 172.21.15.33:0/760891792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80cc600 msgr2=0x7f97a80cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.215+0000 7f97ad99d700 1 --2- 172.21.15.33:0/760891792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80cc600 0x7f97a80cc9d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f97a80b10c0 tx=0x7f979c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.216+0000 7f97ad99d700 1 -- 172.21.15.33:0/760891792 shutdown_connections 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.216+0000 7f97ad99d700 1 --2- 172.21.15.33:0/760891792 >> v2:172.21.15.167:3300/0 conn(0x7f97a806c560 0x7f97a806c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.216+0000 7f97ad99d700 1 --2- 172.21.15.33:0/760891792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80cc600 0x7f97a80cc9d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.216+0000 7f97ad99d700 1 -- 172.21.15.33:0/760891792 >> 172.21.15.33:0/760891792 conn(0x7f97a806d670 msgr2=0x7f97a806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.216+0000 7f97ad99d700 1 -- 172.21.15.33:0/760891792 shutdown_connections 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.216+0000 7f97ad99d700 1 -- 172.21.15.33:0/760891792 wait complete. 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.217+0000 7f97ad99d700 1 Processor -- start 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.217+0000 7f97ad99d700 1 -- start start 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a806c560 0x7f97a816aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a80cc600 0x7f97a8159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 0x7f97a80abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80aee40 con 0x7f97a80ccf30 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a816b000 con 0x7f97a80cc600 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80ace70 con 0x7f97a806c560 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.218+0000 7f97ad19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 0x7f97a80abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 0x7f97a80abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37380/0 (socket says 172.21.15.33:37380) 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 -- 172.21.15.33:0/201139735 learned_addr learned my addr 172.21.15.33:0/201139735 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ac99b700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a806c560 0x7f97a816aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97a7fff700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a80cc600 0x7f97a8159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 -- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a806c560 msgr2=0x7f97a816aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a806c560 0x7f97a816aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 -- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a80cc600 msgr2=0x7f97a8159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a80cc600 0x7f97a8159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 -- 172.21.15.33:0/201139735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0094f0 con 0x7f97a80ccf30 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97a7fff700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a80cc600 0x7f97a8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad19c700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 0x7f97a80abad0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009fd0 tx=0x7f97a0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000eca0 con 0x7f97a80ccf30 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000ee00 con 0x7f97a80ccf30 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.219+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a816db60 con 0x7f97a80ccf30 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.220+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0014670 con 0x7f97a80ccf30 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.220+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a816e120 con 0x7f97a80ccf30 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.221+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a804c380 con 0x7f97a80ccf30 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.222+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97a00149e0 con 0x7f97a80ccf30 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.223+0000 7f97a4ff9700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f978c066f30 0x7f978c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.223+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f97a0084c80 con 0x7f97a80ccf30 2023-10-06T05:24:24.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.224+0000 7f97ac99b700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f978c066f30 0x7f978c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.224+0000 7f97ac99b700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f978c066f30 0x7f978c0693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f97a80b10c0 tx=0x7f979c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.228+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97a0051ed0 con 0x7f97a80ccf30 2023-10-06T05:24:24.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:24.432+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f97a815c2d0 con 0x7f97a80ccf30 2023-10-06T05:24:25.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.000+0000 7f97a4ff9700 1 -- 172.21.15.33:0/201139735 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v142) v1 ==== 170+0+0 (secure 0 0 0) 0x7f97a00554f0 con 0x7f97a80ccf30 2023-10-06T05:24:25.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.011+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f978c066f30 msgr2=0x7f978c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.011+0000 7f97ad99d700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f978c066f30 0x7f978c0693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f97a80b10c0 tx=0x7f979c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:25.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.011+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 msgr2=0x7f97a80abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.011+0000 7f97ad99d700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 0x7f97a80abad0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009fd0 tx=0x7f97a0009900 comp rx=0 tx=0).stop 2023-10-06T05:24:25.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 shutdown_connections 2023-10-06T05:24:25.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f978c066f30 0x7f978c0693e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97a80ccf30 0x7f97a80abad0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97a806c560 0x7f97a816aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 --2- 172.21.15.33:0/201139735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a80cc600 0x7f97a8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 >> 172.21.15.33:0/201139735 conn(0x7f97a806d670 msgr2=0x7f97a8158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.012+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 shutdown_connections 2023-10-06T05:24:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.014+0000 7f97ad99d700 1 -- 172.21.15.33:0/201139735 wait complete. 2023-10-06T05:24:25.021 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:24:25.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:24:25.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.231+0000 7f7c5f39d700 1 Processor -- start 2023-10-06T05:24:25.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.231+0000 7f7c5f39d700 1 -- start start 2023-10-06T05:24:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.232+0000 7f7c5f39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7c5806c7b0 0x7f7c58067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.232+0000 7f7c5f39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c580aff90 0x7f7c580a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.232+0000 7f7c5f39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c5806cb80 0x7f7c5815d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5f39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5815d3a0 con 0x7f7c5806cb80 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5f39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c580b0470 con 0x7f7c5806c7b0 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5f39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580b0750 con 0x7f7c580aff90 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c5806cb80 0x7f7c5815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c5806cb80 0x7f7c5815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37392/0 (socket says 172.21.15.33:37392) 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 -- 172.21.15.33:0/2402432906 learned_addr learned my addr 172.21.15.33:0/2402432906 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5db9a700 1 --2- 172.21.15.33:0/2402432906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c580aff90 0x7f7c580a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5e39b700 1 --2- 172.21.15.33:0/2402432906 >> v2:172.21.15.167:3300/0 conn(0x7f7c5806c7b0 0x7f7c58067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 -- 172.21.15.33:0/2402432906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c580aff90 msgr2=0x7f7c580a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 --2- 172.21.15.33:0/2402432906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c580aff90 0x7f7c580a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 -- 172.21.15.33:0/2402432906 >> v2:172.21.15.167:3300/0 conn(0x7f7c5806c7b0 msgr2=0x7f7c58067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 --2- 172.21.15.33:0/2402432906 >> v2:172.21.15.167:3300/0 conn(0x7f7c5806c7b0 0x7f7c58067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.233+0000 7f7c5eb9c700 1 -- 172.21.15.33:0/2402432906 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580b0a30 con 0x7f7c5806cb80 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.234+0000 7f7c5eb9c700 1 --2- 172.21.15.33:0/2402432906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806cb80 0x7f7c5815d830 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7c580afc10 tx=0x7f7c50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f97f701e56e387 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.234+0000 7f7c5d399700 1 -- 172.21.15.33:0/2402432906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000e070 con 0x7f7c5806cb80 2023-10-06T05:24:25.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.234+0000 7f7c5d399700 1 -- 172.21.15.33:0/2402432906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5000ba50 con 0x7f7c5806cb80 2023-10-06T05:24:25.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.234+0000 7f7c5d399700 1 -- 172.21.15.33:0/2402432906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500092e0 con 0x7f7c5806cb80 2023-10-06T05:24:25.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.234+0000 7f7c5f39d700 1 -- 172.21.15.33:0/2402432906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806cb80 msgr2=0x7f7c5815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.234+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/2402432906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806cb80 0x7f7c5815d830 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7c580afc10 tx=0x7f7c50009840 comp rx=0 tx=0).stop 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 -- 172.21.15.33:0/2402432906 shutdown_connections 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/2402432906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806cb80 0x7f7c5815d830 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/2402432906 >> v2:172.21.15.167:3300/0 conn(0x7f7c5806c7b0 0x7f7c58067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/2402432906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c580aff90 0x7f7c580a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 -- 172.21.15.33:0/2402432906 >> 172.21.15.33:0/2402432906 conn(0x7f7c5806d670 msgr2=0x7f7c5806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 -- 172.21.15.33:0/2402432906 shutdown_connections 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.235+0000 7f7c5f39d700 1 -- 172.21.15.33:0/2402432906 wait complete. 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.236+0000 7f7c5f39d700 1 Processor -- start 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.236+0000 7f7c5f39d700 1 -- start start 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5f39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 0x7f7c58152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5f39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5806cb80 0x7f7c58158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5f39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c580aff90 0x7f7c58159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5f39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58159a60 con 0x7f7c5806c7b0 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5f39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58159d30 con 0x7f7c580aff90 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5f39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5815a010 con 0x7f7c5806cb80 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 0x7f7c58152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 0x7f7c58152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37406/0 (socket says 172.21.15.33:37406) 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.237+0000 7f7c5e39b700 1 -- 172.21.15.33:0/3652531930 learned_addr learned my addr 172.21.15.33:0/3652531930 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5db9a700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5806cb80 0x7f7c58158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5eb9c700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c580aff90 0x7f7c58159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5e39b700 1 -- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5806cb80 msgr2=0x7f7c58158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5e39b700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5806cb80 0x7f7c58158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5e39b700 1 -- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c580aff90 msgr2=0x7f7c58159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5e39b700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c580aff90 0x7f7c58159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5e39b700 1 -- 172.21.15.33:0/3652531930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009ae0 con 0x7f7c5806c7b0 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5e39b700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 0x7f7c58152b60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400d810 tx=0x7f7c54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540125f0 con 0x7f7c5806c7b0 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c54012750 con 0x7f7c5806c7b0 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54012b10 con 0x7f7c5806c7b0 2023-10-06T05:24:25.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.238+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5815a2f0 con 0x7f7c5806c7b0 2023-10-06T05:24:25.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.239+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5815a8b0 con 0x7f7c5806c7b0 2023-10-06T05:24:25.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.239+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5804c380 con 0x7f7c5806c7b0 2023-10-06T05:24:25.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.241+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c5401e460 con 0x7f7c5806c7b0 2023-10-06T05:24:25.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.241+0000 7f7c4e7fc700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c3c066f60 0x7f7c3c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.242+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f7c54084d30 con 0x7f7c5806c7b0 2023-10-06T05:24:25.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.242+0000 7f7c5db9a700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c3c066f60 0x7f7c3c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.243+0000 7f7c5db9a700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c3c066f60 0x7f7c3c069410 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7c580b1d90 tx=0x7f7c48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.246+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c54051e00 con 0x7f7c5806c7b0 2023-10-06T05:24:25.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:25.494+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7c5806efb0 con 0x7f7c5806c7b0 2023-10-06T05:24:26.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.007+0000 7f7c4e7fc700 1 -- 172.21.15.33:0/3652531930 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v101) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7c54055420 con 0x7f7c5806c7b0 2023-10-06T05:24:26.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.015+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c3c066f60 msgr2=0x7f7c3c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.015+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c3c066f60 0x7f7c3c069410 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7c580b1d90 tx=0x7f7c48008040 comp rx=0 tx=0).stop 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.015+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 msgr2=0x7f7c58152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.015+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 0x7f7c58152b60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400d810 tx=0x7f7c54008040 comp rx=0 tx=0).stop 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.016+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 shutdown_connections 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.016+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7c3c066f60 0x7f7c3c069410 secure :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7c580b1d90 tx=0x7f7c48008040 comp rx=0 tx=0).stop 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.016+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c580aff90 0x7f7c58159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.016+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c5806c7b0 0x7f7c58152b60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.016+0000 7f7c5f39d700 1 --2- 172.21.15.33:0/3652531930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c5806cb80 0x7f7c58158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.016+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 >> 172.21.15.33:0/3652531930 conn(0x7f7c5806d670 msgr2=0x7f7c580ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.017+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 shutdown_connections 2023-10-06T05:24:26.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.020+0000 7f7c5f39d700 1 -- 172.21.15.33:0/3652531930 wait complete. 2023-10-06T05:24:26.025 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 21 and data pool 22 2023-10-06T05:24:26.054 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:26.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.242+0000 7fee0979d700 1 Processor -- start 2023-10-06T05:24:26.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.242+0000 7fee0979d700 1 -- start start 2023-10-06T05:24:26.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee0979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0a40 0x7fee0406ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee0979d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fee040b0e10 0x7fee040b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee0979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0415e2d0 con 0x7fee040b2e30 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee0979d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0415d6c0 con 0x7fee040b0e10 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee0979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0415d9a0 con 0x7fee040b0a40 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fee040b0e10 0x7fee040b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fee040b0e10 0x7fee040b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37414/0 (socket says 172.21.15.33:37414) 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 -- 172.21.15.33:0/160392040 learned_addr learned my addr 172.21.15.33:0/160392040 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:26.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee03fff700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0a40 0x7fee0406ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 -- 172.21.15.33:0/160392040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0a40 msgr2=0x7fee0406ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:26.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0a40 0x7fee0406ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 -- 172.21.15.33:0/160392040 >> v1:172.21.15.167:6789/0 conn(0x7fee040b2e30 legacy=0x7fee0415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.244+0000 7fee037fe700 1 -- 172.21.15.33:0/160392040 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0415dc80 con 0x7fee040b0e10 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.245+0000 7fee03fff700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0a40 0x7fee0406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.245+0000 7fee037fe700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0e10 0x7fee040b28f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ae136d70467031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.245+0000 7fee02ffd700 1 -- 172.21.15.33:0/160392040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800b860 con 0x7fee040b0e10 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.245+0000 7fee02ffd700 1 -- 172.21.15.33:0/160392040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf800b9c0 con 0x7fee040b0e10 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.245+0000 7fee02ffd700 1 -- 172.21.15.33:0/160392040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800bd60 con 0x7fee040b0e10 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 -- 172.21.15.33:0/160392040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0e10 msgr2=0x7fee040b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0e10 0x7fee040b28f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 -- 172.21.15.33:0/160392040 shutdown_connections 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0a40 0x7fee0406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 --2- 172.21.15.33:0/160392040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0e10 0x7fee040b28f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 -- 172.21.15.33:0/160392040 >> 172.21.15.33:0/160392040 conn(0x7fee0406e0f0 msgr2=0x7fee040675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 -- 172.21.15.33:0/160392040 shutdown_connections 2023-10-06T05:24:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.246+0000 7fee0979d700 1 -- 172.21.15.33:0/160392040 wait complete. 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.247+0000 7fee0979d700 1 Processor -- start 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.247+0000 7fee0979d700 1 -- start start 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.248+0000 7fee0979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 0x7fee04142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.248+0000 7fee0979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0e10 0x7fee04155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.248+0000 7fee0979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee040b2e30 0x7fee04156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.248+0000 7fee0979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0415c780 con 0x7fee040b0a40 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.248+0000 7fee0979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee04142f70 con 0x7fee040b2e30 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.248+0000 7fee0979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0415bb70 con 0x7fee040b0e10 2023-10-06T05:24:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 0x7fee04142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 0x7fee04142a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37428/0 (socket says 172.21.15.33:37428) 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 -- 172.21.15.33:0/2708558341 learned_addr learned my addr 172.21.15.33:0/2708558341 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee037fe700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0e10 0x7fee04155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee08f9c700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee040b2e30 0x7fee04156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 -- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0e10 msgr2=0x7fee04155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0e10 0x7fee04155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 -- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee040b2e30 msgr2=0x7fee04156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee040b2e30 0x7fee04156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 -- 172.21.15.33:0/2708558341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80094f0 con 0x7fee040b0a40 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee037fe700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0e10 0x7fee04155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee08f9c700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee040b2e30 0x7fee04156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.249+0000 7fee03fff700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 0x7fee04142a30 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e900 tx=0x7fedf4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.250+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf400eea0 con 0x7fee040b0a40 2023-10-06T05:24:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.250+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf400c670 con 0x7fee040b0a40 2023-10-06T05:24:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.250+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0415be50 con 0x7fee040b0a40 2023-10-06T05:24:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.250+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf400ca10 con 0x7fee040b0a40 2023-10-06T05:24:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.250+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0415c410 con 0x7fee040b0a40 2023-10-06T05:24:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.251+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0406fcc0 con 0x7fee040b0a40 2023-10-06T05:24:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.252+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedf4013410 con 0x7fee040b0a40 2023-10-06T05:24:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.253+0000 7fedebfff700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fedf0066f30 0x7fedf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.253+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fedf4084660 con 0x7fee040b0a40 2023-10-06T05:24:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.253+0000 7fee037fe700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fedf0066f30 0x7fedf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.254+0000 7fee037fe700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fedf0066f30 0x7fedf00693e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.257+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedf4055160 con 0x7fee040b0a40 2023-10-06T05:24:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.463+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee0404c380 con 0x7fee040b0a40 2023-10-06T05:24:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.464+0000 7fedebfff700 1 -- 172.21.15.33:0/2708558341 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fedf4054cf0 con 0x7fee040b0a40 2023-10-06T05:24:26.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:26.468 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":143,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:25.495173+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:23.679709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:24.472340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fedf0066f30 msgr2=0x7fedf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fedf0066f30 0x7fedf00693e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 msgr2=0x7fee04142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 0x7fee04142a30 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e900 tx=0x7fedf4008040 comp rx=0 tx=0).stop 2023-10-06T05:24:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 shutdown_connections 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fedf0066f30 0x7fedf00693e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee040b2e30 0x7fee04156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee040b0a40 0x7fee04142a30 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 --2- 172.21.15.33:0/2708558341 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee040b0e10 0x7fee04155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 >> 172.21.15.33:0/2708558341 conn(0x7fee0406e0f0 msgr2=0x7fee0406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 shutdown_connections 2023-10-06T05:24:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.467+0000 7fee0979d700 1 -- 172.21.15.33:0/2708558341 wait complete. 2023-10-06T05:24:26.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:26.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.671+0000 7fdb17fff700 1 Processor -- start 2023-10-06T05:24:26.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.671+0000 7fdb17fff700 1 -- start start 2023-10-06T05:24:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb17fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb1813faa0 0x7fdb1813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb17fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1806aad0 0x7fdb1806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb17fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180c9cb0 con 0x7fdb180c7940 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb17fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb180ca270 con 0x7fdb1813faa0 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb17fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1806b840 con 0x7fdb1806aad0 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb167fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb180c7940 0x7fdb180c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43132/0 (socket says 172.21.15.33:43132) 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.672+0000 7fdb167fc700 1 -- 172.21.15.33:0/215611852 learned_addr learned my addr 172.21.15.33:0/215611852 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb177fe700 1 --2- 172.21.15.33:0/215611852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1806aad0 0x7fdb1806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb16ffd700 1 --2- 172.21.15.33:0/215611852 >> v2:172.21.15.167:3300/0 conn(0x7fdb1813faa0 0x7fdb1813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb15ffb700 1 -- 172.21.15.33:0/215611852 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781440162 0 0) 0x7fdb180c9cb0 con 0x7fdb180c7940 2023-10-06T05:24:26.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb15ffb700 1 -- 172.21.15.33:0/215611852 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb00003570 con 0x7fdb180c7940 2023-10-06T05:24:26.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb15ffb700 1 -- 172.21.15.33:0/215611852 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 879911079 0 0) 0x7fdb00003570 con 0x7fdb180c7940 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb177fe700 1 -- 172.21.15.33:0/215611852 >> v2:172.21.15.167:3300/0 conn(0x7fdb1813faa0 msgr2=0x7fdb1813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb177fe700 1 --2- 172.21.15.33:0/215611852 >> v2:172.21.15.167:3300/0 conn(0x7fdb1813faa0 0x7fdb1813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb177fe700 1 -- 172.21.15.33:0/215611852 >> v1:172.21.15.33:6789/0 conn(0x7fdb180c7940 legacy=0x7fdb180c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb177fe700 1 -- 172.21.15.33:0/215611852 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1806bb20 con 0x7fdb1806aad0 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.673+0000 7fdb16ffd700 1 --2- 172.21.15.33:0/215611852 >> v2:172.21.15.167:3300/0 conn(0x7fdb1813faa0 0x7fdb1813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.674+0000 7fdb177fe700 1 --2- 172.21.15.33:0/215611852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1806aad0 0x7fdb1806af50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009e60 tx=0x7fdb10009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76464779a6e5972f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.674+0000 7fdb15ffb700 1 -- 172.21.15.33:0/215611852 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000e070 con 0x7fdb1806aad0 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.674+0000 7fdb15ffb700 1 -- 172.21.15.33:0/215611852 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1000b7a0 con 0x7fdb1806aad0 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.674+0000 7fdb15ffb700 1 -- 172.21.15.33:0/215611852 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c001470 con 0x7fdb1806aad0 2023-10-06T05:24:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.674+0000 7fdb17fff700 1 -- 172.21.15.33:0/215611852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1806aad0 msgr2=0x7fdb1806af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.674+0000 7fdb17fff700 1 --2- 172.21.15.33:0/215611852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1806aad0 0x7fdb1806af50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009e60 tx=0x7fdb10009780 comp rx=0 tx=0).stop 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.675+0000 7fdb17fff700 1 -- 172.21.15.33:0/215611852 shutdown_connections 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.675+0000 7fdb17fff700 1 --2- 172.21.15.33:0/215611852 >> v2:172.21.15.167:3300/0 conn(0x7fdb1813faa0 0x7fdb1813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.675+0000 7fdb17fff700 1 --2- 172.21.15.33:0/215611852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1806aad0 0x7fdb1806af50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.675+0000 7fdb17fff700 1 -- 172.21.15.33:0/215611852 >> 172.21.15.33:0/215611852 conn(0x7fdb1806d030 msgr2=0x7fdb1806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.675+0000 7fdb17fff700 1 -- 172.21.15.33:0/215611852 shutdown_connections 2023-10-06T05:24:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.675+0000 7fdb17fff700 1 -- 172.21.15.33:0/215611852 wait complete. 2023-10-06T05:24:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.676+0000 7fdb17fff700 1 Processor -- start 2023-10-06T05:24:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.676+0000 7fdb17fff700 1 -- start start 2023-10-06T05:24:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb17fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1806aad0 0x7fdb18169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb17fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb180c7940 0x7fdb18155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb17fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 0x7fdb180c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb17fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1816a390 con 0x7fdb1813faa0 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb17fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813d180 con 0x7fdb1806aad0 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb17fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813c570 con 0x7fdb180c7940 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 0x7fdb180c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 0x7fdb180c0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37438/0 (socket says 172.21.15.33:37438) 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.677+0000 7fdb177fe700 1 -- 172.21.15.33:0/2312106810 learned_addr learned my addr 172.21.15.33:0/2312106810 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb167fc700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb180c7940 0x7fdb18155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb16ffd700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1806aad0 0x7fdb18169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb177fe700 1 -- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb180c7940 msgr2=0x7fdb18155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb177fe700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb180c7940 0x7fdb18155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb177fe700 1 -- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1806aad0 msgr2=0x7fdb18169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb177fe700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1806aad0 0x7fdb18169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb177fe700 1 -- 172.21.15.33:0/2312106810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08009420 con 0x7fdb1813faa0 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb16ffd700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1806aad0 0x7fdb18169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb167fc700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb180c7940 0x7fdb18155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb177fe700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 0x7fdb180c0f30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fdb1806b980 tx=0x7fdb10018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10012d20 con 0x7fdb1813faa0 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1001d430 con 0x7fdb1813faa0 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1001d7f0 con 0x7fdb1813faa0 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.678+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb10009a20 con 0x7fdb1813faa0 2023-10-06T05:24:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.679+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1813cb30 con 0x7fdb1813faa0 2023-10-06T05:24:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.680+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1804c380 con 0x7fdb1813faa0 2023-10-06T05:24:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.681+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb10024070 con 0x7fdb1813faa0 2023-10-06T05:24:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.682+0000 7fdafeffd700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb04066f30 0x7fdb040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.682+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb10086a00 con 0x7fdb1813faa0 2023-10-06T05:24:26.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.682+0000 7fdb16ffd700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb04066f30 0x7fdb040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.683+0000 7fdb16ffd700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb04066f30 0x7fdb040693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdb080093f0 tx=0x7fdb08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.686+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb10053a70 con 0x7fdb1813faa0 2023-10-06T05:24:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.887+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb180c9f00 con 0x7fdb1813faa0 2023-10-06T05:24:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.888+0000 7fdafeffd700 1 -- 172.21.15.33:0/2312106810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fdb10057090 con 0x7fdb1813faa0 2023-10-06T05:24:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:26.890 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":143,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:25.495173+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:23.679709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:24.472340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:26.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb04066f30 msgr2=0x7fdb040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb04066f30 0x7fdb040693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdb080093f0 tx=0x7fdb08008040 comp rx=0 tx=0).stop 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 msgr2=0x7fdb180c0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 0x7fdb180c0f30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fdb1806b980 tx=0x7fdb10018000 comp rx=0 tx=0).stop 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 shutdown_connections 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb04066f30 0x7fdb040693e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb1813faa0 0x7fdb180c0f30 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1806aad0 0x7fdb18169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 --2- 172.21.15.33:0/2312106810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb180c7940 0x7fdb18155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 >> 172.21.15.33:0/2312106810 conn(0x7fdb1806d030 msgr2=0x7fdb1806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 shutdown_connections 2023-10-06T05:24:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:26.891+0000 7fdb17fff700 1 -- 172.21.15.33:0/2312106810 wait complete. 2023-10-06T05:24:26.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:24:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.101+0000 7fd5e8f9d700 1 Processor -- start 2023-10-06T05:24:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.101+0000 7fd5e8f9d700 1 -- start start 2023-10-06T05:24:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.102+0000 7fd5e8f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5e40cc7e0 0x7fd5e40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.102+0000 7fd5e8f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e40ad430 con 0x7fd5e4156370 2023-10-06T05:24:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.102+0000 7fd5e8f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e40afe20 con 0x7fd5e40b01f0 2023-10-06T05:24:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.102+0000 7fd5e8f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e406bc70 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5e40cc7e0 0x7fd5e40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5e40cc7e0 0x7fd5e40af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37442/0 (socket says 172.21.15.33:37442) 2023-10-06T05:24:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd5e4156370 0x7fd5e40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43136/0 (socket says 172.21.15.33:43136) 2023-10-06T05:24:27.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/2015257560 learned_addr learned my addr 172.21.15.33:0/2015257560 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/2015257560 >> v1:172.21.15.167:6789/0 conn(0x7fd5e40b01f0 legacy=0x7fd5e40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/2015257560 >> v1:172.21.15.33:6789/0 conn(0x7fd5e4156370 legacy=0x7fd5e40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/2015257560 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40cccc0 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e27fc700 1 -- 172.21.15.33:0/2015257560 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915150670 0 0) 0x7fd5e40ad430 con 0x7fd5e4156370 2023-10-06T05:24:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.103+0000 7fd5e2ffd700 1 --2- 172.21.15.33:0/2015257560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e40af8e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80099b0 tx=0x7fd5d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d54ec8aef9a2fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e27fc700 1 -- 172.21.15.33:0/2015257560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800b820 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e27fc700 1 -- 172.21.15.33:0/2015257560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800b980 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e27fc700 1 -- 172.21.15.33:0/2015257560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800bd20 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/2015257560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 msgr2=0x7fd5e40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/2015257560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e40af8e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80099b0 tx=0x7fd5d8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/2015257560 shutdown_connections 2023-10-06T05:24:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/2015257560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e40af8e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/2015257560 >> 172.21.15.33:0/2015257560 conn(0x7fd5e406d030 msgr2=0x7fd5e40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/2015257560 shutdown_connections 2023-10-06T05:24:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.104+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/2015257560 wait complete. 2023-10-06T05:24:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.106+0000 7fd5e8f9d700 1 Processor -- start 2023-10-06T05:24:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.106+0000 7fd5e8f9d700 1 -- start start 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5e40b01f0 0x7fd5e4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e8f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e4150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e8f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5e4156370 0x7fd5e41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e8f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4154700 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e8f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e416ad60 con 0x7fd5e4156370 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4169d50 con 0x7fd5e40b01f0 2023-10-06T05:24:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e4150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e4150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37452/0 (socket says 172.21.15.33:37452) 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/898863352 learned_addr learned my addr 172.21.15.33:0/898863352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e3fff700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5e4156370 0x7fd5e41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e37fe700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5e40b01f0 0x7fd5e4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5e40b01f0 msgr2=0x7fd5e4154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5e40b01f0 0x7fd5e4154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5e4156370 msgr2=0x7fd5e41541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5e4156370 0x7fd5e41541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.107+0000 7fd5e2ffd700 1 -- 172.21.15.33:0/898863352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80094f0 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.108+0000 7fd5e2ffd700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e4150ff0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd5e406d4d0 tx=0x7fd5d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.108+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d801c450 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.108+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d801c5d0 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.108+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e416a8d0 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.108+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d801c990 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.108+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e416a280 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.109+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e416aa60 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.110+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5d801cd00 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.111+0000 7fd5cb7fe700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5d0066f30 0x7fd5d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.111+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5d80868d0 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.112+0000 7fd5e37fe700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5d0066f30 0x7fd5d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.112+0000 7fd5e37fe700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5d0066f30 0x7fd5d00693e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ab480 tx=0x7fd5d4007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.114+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5d80539c0 con 0x7fd5e40cc7e0 2023-10-06T05:24:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:27.361+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd5e404c380 con 0x7fd5e40cc7e0 2023-10-06T05:24:28.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.026+0000 7fd5cb7fe700 1 -- 172.21.15.33:0/898863352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd5d8056fe0 con 0x7fd5e40cc7e0 2023-10-06T05:24:28.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5d0066f30 msgr2=0x7fd5d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5d0066f30 0x7fd5d00693e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ab480 tx=0x7fd5d4007b60 comp rx=0 tx=0).stop 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 msgr2=0x7fd5e4150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e4150ff0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd5e406d4d0 tx=0x7fd5d8016000 comp rx=0 tx=0).stop 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 shutdown_connections 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5d0066f30 0x7fd5d00693e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5e4156370 0x7fd5e41541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5e40b01f0 0x7fd5e4154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 --2- 172.21.15.33:0/898863352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e40cc7e0 0x7fd5e4150ff0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 >> 172.21.15.33:0/898863352 conn(0x7fd5e406d030 msgr2=0x7fd5e406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 shutdown_connections 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.030+0000 7fd5e8f9d700 1 -- 172.21.15.33:0/898863352 wait complete. 2023-10-06T05:24:28.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:24:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.246+0000 7f040939d700 1 Processor -- start 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.246+0000 7f040939d700 1 -- start start 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f040939d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04040abbd0 0x7f0404067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f040939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040406ae10 con 0x7f04040b1f90 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f040939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0404068580 con 0x7f0404068140 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f040939d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040ac050 con 0x7f04040abbd0 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f0403fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f04040b1f90 0x7f0404070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43152/0 (socket says 172.21.15.33:43152) 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f0403fff700 1 -- 172.21.15.33:0/2262688701 learned_addr learned my addr 172.21.15.33:0/2262688701 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164501247 0 0) 0x7f040406ae10 con 0x7f04040b1f90 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f04037fe700 1 --2- 172.21.15.33:0/2262688701 >> v2:172.21.15.167:3300/0 conn(0x7f04040abbd0 0x7f0404067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.248+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03ec003570 con 0x7f04040b1f90 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134907296 0 0) 0x7f03ec003570 con 0x7f04040b1f90 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 >> v2:172.21.15.167:3300/0 conn(0x7f04040abbd0 msgr2=0x7f0404067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 --2- 172.21.15.33:0/2262688701 >> v2:172.21.15.167:3300/0 conn(0x7f04040abbd0 0x7f0404067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 >> v1:172.21.15.167:6789/0 conn(0x7f0404068140 legacy=0x7f0404159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040406a4e0 con 0x7f04040b1f90 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f03f4001470 con 0x7f04040b1f90 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 >> v1:172.21.15.33:6789/0 conn(0x7f04040b1f90 legacy=0x7f0404070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec003a60 0x7f03ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03ec004810 0x7f03ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0051c0 0x7f03ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040ac050 con 0x7f03ec004810 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec00c160 con 0x7f03ec0051c0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec00c320 con 0x7f03ec003a60 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f04037fe700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03ec004810 0x7f03ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0403fff700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec003a60 0x7f03ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0408b9c700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0051c0 0x7f03ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f04037fe700 1 -- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec003a60 msgr2=0x7f03ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f04037fe700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec003a60 0x7f03ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f04037fe700 1 -- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0051c0 msgr2=0x7f03ec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f04037fe700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ec0051c0 0x7f03ec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f04037fe700 1 -- 172.21.15.33:0/2262688701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec003570 con 0x7f03ec004810 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0408b9c700 1 -- 172.21.15.33:0/2262688701 reap_dead start 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.249+0000 7f0403fff700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec003a60 0x7f03ec003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.250+0000 7f04037fe700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03ec004810 0x7f03ec004c80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009c00 tx=0x7f03f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c216b565b966b76a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.250+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800e710 con 0x7f03ec004810 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.250+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f800e870 con 0x7f03ec004810 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.250+0000 7f0402ffd700 1 -- 172.21.15.33:0/2262688701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800ec30 con 0x7f03ec004810 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.250+0000 7f040939d700 1 -- 172.21.15.33:0/2262688701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03ec004810 msgr2=0x7f03ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.250+0000 7f040939d700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03ec004810 0x7f03ec004c80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009c00 tx=0x7f03f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.251+0000 7f040939d700 1 -- 172.21.15.33:0/2262688701 shutdown_connections 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.251+0000 7f040939d700 1 --2- 172.21.15.33:0/2262688701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03ec004810 0x7f03ec004c80 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.251+0000 7f040939d700 1 -- 172.21.15.33:0/2262688701 >> 172.21.15.33:0/2262688701 conn(0x7f040406ea70 msgr2=0x7f040406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.251+0000 7f040939d700 1 -- 172.21.15.33:0/2262688701 shutdown_connections 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.251+0000 7f040939d700 1 -- 172.21.15.33:0/2262688701 wait complete. 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.252+0000 7f040939d700 1 Processor -- start 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.252+0000 7f040939d700 1 -- start start 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f040939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec004810 0x7f040415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f040939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 0x7f0404151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f040939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0404150a70 0x7f04040ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f040939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040415b600 con 0x7f0404158310 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f040939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404158830 con 0x7f0404150a70 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f040939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040415afb0 con 0x7f03ec004810 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f04037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 0x7f0404151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f04037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 0x7f0404151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37474/0 (socket says 172.21.15.33:37474) 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.253+0000 7f04037fe700 1 -- 172.21.15.33:0/2985244902 learned_addr learned my addr 172.21.15.33:0/2985244902 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f0408b9c700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0404150a70 0x7f04040ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f0403fff700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec004810 0x7f040415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f04037fe700 1 -- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec004810 msgr2=0x7f040415ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f04037fe700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec004810 0x7f040415ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f04037fe700 1 -- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0404150a70 msgr2=0x7f04040ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f04037fe700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0404150a70 0x7f04040ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f04037fe700 1 -- 172.21.15.33:0/2985244902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040415b290 con 0x7f0404158310 2023-10-06T05:24:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f04037fe700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 0x7f0404151240 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c130 tx=0x7f03f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.254+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8022070 con 0x7f0404158310 2023-10-06T05:24:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.255+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f800e9e0 con 0x7f0404158310 2023-10-06T05:24:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.255+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f801c450 con 0x7f0404158310 2023-10-06T05:24:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.255+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040415b600 con 0x7f0404158310 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.255+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0404151980 con 0x7f0404158310 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.256+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040415b790 con 0x7f0404158310 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.257+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03f801c8a0 con 0x7f0404158310 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.257+0000 7f03ebfff700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f03f0066f30 0x7f03f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.257+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f03f8085db0 con 0x7f0404158310 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.258+0000 7f0403fff700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f03f0066f30 0x7f03f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.258+0000 7f0403fff700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f03f0066f30 0x7f03f00693e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f4007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.262+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03f8052ee0 con 0x7f0404158310 2023-10-06T05:24:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:28.510+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f040404c380 con 0x7f0404158310 2023-10-06T05:24:29.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.027+0000 7f03ebfff700 1 -- 172.21.15.33:0/2985244902 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) v1 ==== 122+0+0 (secure 0 0 0) 0x7f03f8056500 con 0x7f0404158310 2023-10-06T05:24:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f03f0066f30 msgr2=0x7f03f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f03f0066f30 0x7f03f00693e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f4007040 comp rx=0 tx=0).stop 2023-10-06T05:24:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 msgr2=0x7f0404151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 0x7f0404151240 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c130 tx=0x7f03f801a000 comp rx=0 tx=0).stop 2023-10-06T05:24:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 shutdown_connections 2023-10-06T05:24:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f03f0066f30 0x7f03f00693e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0404150a70 0x7f04040ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03ec004810 0x7f040415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 --2- 172.21.15.33:0/2985244902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0404158310 0x7f0404151240 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 >> 172.21.15.33:0/2985244902 conn(0x7f040406ea70 msgr2=0x7f0404153530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 shutdown_connections 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.030+0000 7f040939d700 1 -- 172.21.15.33:0/2985244902 wait complete. 2023-10-06T05:24:29.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.239+0000 7f7f93fff700 1 Processor -- start 2023-10-06T05:24:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.239+0000 7f7f93fff700 1 -- start start 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f93fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f940c8050 0x7f7f9413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f93fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f9406aad0 0x7f7f9406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f93fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940c9cb0 con 0x7f7f9413faa0 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f93fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940ca270 con 0x7f7f9406aad0 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f93fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9406b840 con 0x7f7f940c8050 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f937fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f9406aad0 0x7f7f9406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f937fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f9406aad0 0x7f7f9406af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37484/0 (socket says 172.21.15.33:37484) 2023-10-06T05:24:29.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.240+0000 7f7f937fe700 1 -- 172.21.15.33:0/1768310822 learned_addr learned my addr 172.21.15.33:0/1768310822 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f927fc700 1 --2- 172.21.15.33:0/1768310822 >> v2:172.21.15.167:3300/0 conn(0x7f7f940c8050 0x7f7f9413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f937fe700 1 -- 172.21.15.33:0/1768310822 >> v2:172.21.15.167:3300/0 conn(0x7f7f940c8050 msgr2=0x7f7f9413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f937fe700 1 --2- 172.21.15.33:0/1768310822 >> v2:172.21.15.167:3300/0 conn(0x7f7f940c8050 0x7f7f9413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f937fe700 1 -- 172.21.15.33:0/1768310822 >> v1:172.21.15.167:6789/0 conn(0x7f7f9413faa0 legacy=0x7f7f940c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f937fe700 1 -- 172.21.15.33:0/1768310822 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9406bb20 con 0x7f7f9406aad0 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f937fe700 1 --2- 172.21.15.33:0/1768310822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f9406af30 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7f940cbef0 tx=0x7f7f8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba922c7dbb0822fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f91ffb700 1 -- 172.21.15.33:0/1768310822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00e070 con 0x7f7f9406aad0 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f91ffb700 1 -- 172.21.15.33:0/1768310822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00ba40 con 0x7f7f9406aad0 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.241+0000 7f7f91ffb700 1 -- 172.21.15.33:0/1768310822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0092e0 con 0x7f7f9406aad0 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 -- 172.21.15.33:0/1768310822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 msgr2=0x7f7f9406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 --2- 172.21.15.33:0/1768310822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f9406af30 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7f940cbef0 tx=0x7f7f8c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 -- 172.21.15.33:0/1768310822 shutdown_connections 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 --2- 172.21.15.33:0/1768310822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f9406af30 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 --2- 172.21.15.33:0/1768310822 >> v2:172.21.15.167:3300/0 conn(0x7f7f940c8050 0x7f7f9413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 -- 172.21.15.33:0/1768310822 >> 172.21.15.33:0/1768310822 conn(0x7f7f9406d030 msgr2=0x7f7f9406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 -- 172.21.15.33:0/1768310822 shutdown_connections 2023-10-06T05:24:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.242+0000 7f7f93fff700 1 -- 172.21.15.33:0/1768310822 wait complete. 2023-10-06T05:24:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.244+0000 7f7f93fff700 1 Processor -- start 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.244+0000 7f7f93fff700 1 -- start start 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f94159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f940c8050 0x7f7f94156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f9413faa0 0x7f7f941564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f93fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94157120 con 0x7f7f9406aad0 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f93fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94156a20 con 0x7f7f940c8050 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f93fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94157350 con 0x7f7f9413faa0 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f94159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f94159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37498/0 (socket says 172.21.15.33:37498) 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f92ffd700 1 -- 172.21.15.33:0/969420745 learned_addr learned my addr 172.21.15.33:0/969420745 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f937fe700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f9413faa0 0x7f7f941564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f927fc700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f940c8050 0x7f7f94156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f92ffd700 1 -- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f9413faa0 msgr2=0x7f7f941564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.245+0000 7f7f92ffd700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f9413faa0 0x7f7f941564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f92ffd700 1 -- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f940c8050 msgr2=0x7f7f94156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f92ffd700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f940c8050 0x7f7f94156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f92ffd700 1 -- 172.21.15.33:0/969420745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009ae0 con 0x7f7f9406aad0 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f92ffd700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f94159530 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400c4e0 tx=0x7f7f8400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400cea0 con 0x7f7f9406aad0 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84011590 con 0x7f7f9406aad0 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84011930 con 0x7f7f9406aad0 2023-10-06T05:24:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f94157630 con 0x7f7f9406aad0 2023-10-06T05:24:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.246+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f94157bf0 con 0x7f7f9406aad0 2023-10-06T05:24:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.247+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9404bba0 con 0x7f7f9406aad0 2023-10-06T05:24:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.248+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f84011d20 con 0x7f7f9406aad0 2023-10-06T05:24:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.249+0000 7f7f7affd700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f80066f30 0x7f7f800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.249+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f84083a40 con 0x7f7f9406aad0 2023-10-06T05:24:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.250+0000 7f7f927fc700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f80066f30 0x7f7f800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.250+0000 7f7f927fc700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f80066f30 0x7f7f800693e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7f940c5290 tx=0x7f7f88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.254+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f84050ab0 con 0x7f7f9406aad0 2023-10-06T05:24:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.500+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f9404c380 con 0x7f7f9406aad0 2023-10-06T05:24:29.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.500+0000 7f7f7affd700 1 -- 172.21.15.33:0/969420745 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7f7f84017070 con 0x7f7f9406aad0 2023-10-06T05:24:29.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f80066f30 msgr2=0x7f7f800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f80066f30 0x7f7f800693e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7f940c5290 tx=0x7f7f88008040 comp rx=0 tx=0).stop 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 msgr2=0x7f7f94159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f94159530 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400c4e0 tx=0x7f7f8400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 shutdown_connections 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7f80066f30 0x7f7f800693e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f9413faa0 0x7f7f941564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f9406aad0 0x7f7f94159530 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 --2- 172.21.15.33:0/969420745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f940c8050 0x7f7f94156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 >> 172.21.15.33:0/969420745 conn(0x7f7f9406d030 msgr2=0x7f7f940cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 shutdown_connections 2023-10-06T05:24:29.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.503+0000 7f7f93fff700 1 -- 172.21.15.33:0/969420745 wait complete. 2023-10-06T05:24:29.506 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:29.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.708+0000 7f978359d700 1 Processor -- start 2023-10-06T05:24:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.708+0000 7f978359d700 1 -- start start 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978359d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f977c155af0 0x7f977c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978359d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f977c06a6a0 0x7f977c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978359d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c06b7e0 con 0x7f977c155af0 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978359d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c06ab60 con 0x7f977c06a6a0 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c06ae40 con 0x7f977c0c8b50 2023-10-06T05:24:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f977c155af0 0x7f977c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f977c155af0 0x7f977c06a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37510/0 (socket says 172.21.15.33:37510) 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.709+0000 7f978259b700 1 -- 172.21.15.33:0/1155216619 learned_addr learned my addr 172.21.15.33:0/1155216619 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f9782d9c700 1 --2- 172.21.15.33:0/1155216619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f9781d9a700 1 --2- 172.21.15.33:0/1155216619 >> v2:172.21.15.167:3300/0 conn(0x7f977c06a6a0 0x7f977c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f978259b700 1 -- 172.21.15.33:0/1155216619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 msgr2=0x7f977c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f978259b700 1 --2- 172.21.15.33:0/1155216619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f978259b700 1 -- 172.21.15.33:0/1155216619 >> v2:172.21.15.167:3300/0 conn(0x7f977c06a6a0 msgr2=0x7f977c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f978259b700 1 --2- 172.21.15.33:0/1155216619 >> v2:172.21.15.167:3300/0 conn(0x7f977c06a6a0 0x7f977c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f978259b700 1 -- 172.21.15.33:0/1155216619 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c06b120 con 0x7f977c155af0 2023-10-06T05:24:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f978259b700 1 --2- 172.21.15.33:0/1155216619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c155af0 0x7f977c06a160 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f977c06f250 tx=0x7f9778009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f799f57cdb23bd3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.710+0000 7f9781599700 1 -- 172.21.15.33:0/1155216619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977800b640 con 0x7f977c155af0 2023-10-06T05:24:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f9781599700 1 -- 172.21.15.33:0/1155216619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977800b7a0 con 0x7f977c155af0 2023-10-06T05:24:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f9781599700 1 -- 172.21.15.33:0/1155216619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977800bb60 con 0x7f977c155af0 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 -- 172.21.15.33:0/1155216619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c155af0 msgr2=0x7f977c06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 --2- 172.21.15.33:0/1155216619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c155af0 0x7f977c06a160 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f977c06f250 tx=0x7f9778009250 comp rx=0 tx=0).stop 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 -- 172.21.15.33:0/1155216619 shutdown_connections 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 --2- 172.21.15.33:0/1155216619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 --2- 172.21.15.33:0/1155216619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c155af0 0x7f977c06a160 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 --2- 172.21.15.33:0/1155216619 >> v2:172.21.15.167:3300/0 conn(0x7f977c06a6a0 0x7f977c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 -- 172.21.15.33:0/1155216619 >> 172.21.15.33:0/1155216619 conn(0x7f977c06d030 msgr2=0x7f977c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 -- 172.21.15.33:0/1155216619 shutdown_connections 2023-10-06T05:24:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.711+0000 7f978359d700 1 -- 172.21.15.33:0/1155216619 wait complete. 2023-10-06T05:24:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.713+0000 7f978359d700 1 Processor -- start 2023-10-06T05:24:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.713+0000 7f978359d700 1 -- start start 2023-10-06T05:24:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 0x7f977c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978359d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f977c155af0 0x7f977c1468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0c34d0 con 0x7f977c06a6a0 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978359d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0acac0 con 0x7f977c155af0 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0abeb0 con 0x7f977c0c8b50 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 0x7f977c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 0x7f977c0c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37518/0 (socket says 172.21.15.33:37518) 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 -- 172.21.15.33:0/3987483782 learned_addr learned my addr 172.21.15.33:0/3987483782 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 -- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 msgr2=0x7f977c158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 -- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f977c155af0 msgr2=0x7f977c1468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f977c155af0 0x7f977c1468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.714+0000 7f978259b700 1 -- 172.21.15.33:0/3987483782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780094f0 con 0x7f977c06a6a0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.715+0000 7f978259b700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 0x7f977c0c5dc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f977c0635d0 tx=0x7f9778017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.715+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778000d10 con 0x7f977c06a6a0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.715+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977800bb20 con 0x7f977c06a6a0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.715+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c0ac190 con 0x7f977c06a6a0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.715+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778011630 con 0x7f977c06a6a0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.715+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c0ac750 con 0x7f977c06a6a0 2023-10-06T05:24:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.716+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c06bc40 con 0x7f977c06a6a0 2023-10-06T05:24:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.717+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97780119a0 con 0x7f977c06a6a0 2023-10-06T05:24:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.718+0000 7f97727fc700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9760066f30 0x7f97600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.718+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9778085a60 con 0x7f977c06a6a0 2023-10-06T05:24:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.718+0000 7f9781d9a700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9760066f30 0x7f97600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.719+0000 7f9781d9a700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9760066f30 0x7f97600693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f977c159530 tx=0x7f976c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.722+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9778052c00 con 0x7f977c06a6a0 2023-10-06T05:24:29.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.927+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977c04c380 con 0x7f977c06a6a0 2023-10-06T05:24:29.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.928+0000 7f97727fc700 1 -- 172.21.15.33:0/3987483782 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f9778015090 con 0x7f977c06a6a0 2023-10-06T05:24:29.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:29.930 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":144,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:27.011934+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:23.679709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:24.472340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9760066f30 msgr2=0x7f97600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9760066f30 0x7f97600693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f977c159530 tx=0x7f976c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 msgr2=0x7f977c0c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 0x7f977c0c5dc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f977c0635d0 tx=0x7f9778017000 comp rx=0 tx=0).stop 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 shutdown_connections 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9760066f30 0x7f97600693e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f977c155af0 0x7f977c1468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f977c06a6a0 0x7f977c0c5dc0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 --2- 172.21.15.33:0/3987483782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f977c0c8b50 0x7f977c158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 >> 172.21.15.33:0/3987483782 conn(0x7f977c06d030 msgr2=0x7f977c0c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 shutdown_connections 2023-10-06T05:24:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:29.930+0000 7f978359d700 1 -- 172.21.15.33:0/3987483782 wait complete. 2023-10-06T05:24:29.949 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:24:29.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:24:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.137+0000 7f0d4359e700 1 Processor -- start 2023-10-06T05:24:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.137+0000 7f0d4359e700 1 -- start start 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d440ccfc0 0x7f0d440cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0d440cf3b0 0x7f0d441594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d441599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d440cbd80 con 0x7f0d440ccfc0 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d440af5f0 con 0x7f0d440cf3b0 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440af8d0 con 0x7f0d440cc750 2023-10-06T05:24:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d440ccfc0 0x7f0d440cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d440ccfc0 0x7f0d440cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35438/0 (socket says 172.21.15.33:35438) 2023-10-06T05:24:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.138+0000 7f0d4259c700 1 -- 172.21.15.33:0/1056640326 learned_addr learned my addr 172.21.15.33:0/1056640326 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d41d9b700 1 --2- 172.21.15.33:0/1056640326 >> v2:172.21.15.167:3300/0 conn(0x7f0d440cf3b0 0x7f0d441594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d42d9d700 1 --2- 172.21.15.33:0/1056640326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d441599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4259c700 1 -- 172.21.15.33:0/1056640326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 msgr2=0x7f0d441599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4259c700 1 --2- 172.21.15.33:0/1056640326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d441599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4259c700 1 -- 172.21.15.33:0/1056640326 >> v2:172.21.15.167:3300/0 conn(0x7f0d440cf3b0 msgr2=0x7f0d441594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4259c700 1 --2- 172.21.15.33:0/1056640326 >> v2:172.21.15.167:3300/0 conn(0x7f0d440cf3b0 0x7f0d441594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4259c700 1 -- 172.21.15.33:0/1056640326 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d440afbb0 con 0x7f0d440ccfc0 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4259c700 1 --2- 172.21.15.33:0/1056640326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440ccfc0 0x7f0d440cc210 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0d340099b0 tx=0x7f0d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336e64315b2f02dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.139+0000 7f0d4159a700 1 -- 172.21.15.33:0/1056640326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400b640 con 0x7f0d440ccfc0 2023-10-06T05:24:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4159a700 1 -- 172.21.15.33:0/1056640326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3400b7a0 con 0x7f0d440ccfc0 2023-10-06T05:24:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4159a700 1 -- 172.21.15.33:0/1056640326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400bb40 con 0x7f0d440ccfc0 2023-10-06T05:24:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 -- 172.21.15.33:0/1056640326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440ccfc0 msgr2=0x7f0d440cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 --2- 172.21.15.33:0/1056640326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440ccfc0 0x7f0d440cc210 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f0d340099b0 tx=0x7f0d34009250 comp rx=0 tx=0).stop 2023-10-06T05:24:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 -- 172.21.15.33:0/1056640326 shutdown_connections 2023-10-06T05:24:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 --2- 172.21.15.33:0/1056640326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d441599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 --2- 172.21.15.33:0/1056640326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440ccfc0 0x7f0d440cc210 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 --2- 172.21.15.33:0/1056640326 >> v2:172.21.15.167:3300/0 conn(0x7f0d440cf3b0 0x7f0d441594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 -- 172.21.15.33:0/1056640326 >> 172.21.15.33:0/1056640326 conn(0x7f0d4406cff0 msgr2=0x7f0d4406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 -- 172.21.15.33:0/1056640326 shutdown_connections 2023-10-06T05:24:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.140+0000 7f0d4359e700 1 -- 172.21.15.33:0/1056640326 wait complete. 2023-10-06T05:24:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.142+0000 7f0d4359e700 1 Processor -- start 2023-10-06T05:24:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.142+0000 7f0d4359e700 1 -- start start 2023-10-06T05:24:30.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d440cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d440ccfc0 0x7f0d440b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 0x7f0d440c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440c9480 con 0x7f0d440b41c0 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440b4900 con 0x7f0d440ccfc0 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d440b4be0 con 0x7f0d440cc750 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 0x7f0d440c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 0x7f0d440c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35448/0 (socket says 172.21.15.33:35448) 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 -- 172.21.15.33:0/844004965 learned_addr learned my addr 172.21.15.33:0/844004965 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:30.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d4259c700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d440cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d41d9b700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d440ccfc0 0x7f0d440b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 -- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 msgr2=0x7f0d440cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d440cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 -- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d440ccfc0 msgr2=0x7f0d440b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d440ccfc0 0x7f0d440b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.143+0000 7f0d42d9d700 1 -- 172.21.15.33:0/844004965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340094f0 con 0x7f0d440b41c0 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.144+0000 7f0d42d9d700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 0x7f0d440c75b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009e90 tx=0x7f0d3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.144+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00cc20 con 0x7f0d440b41c0 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.144+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c00f040 con 0x7f0d440b41c0 2023-10-06T05:24:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.144+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c014480 con 0x7f0d440b41c0 2023-10-06T05:24:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.144+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d440b4ec0 con 0x7f0d440b41c0 2023-10-06T05:24:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.144+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d440b4620 con 0x7f0d440b41c0 2023-10-06T05:24:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.145+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4404c380 con 0x7f0d440b41c0 2023-10-06T05:24:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.146+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d3c0147f0 con 0x7f0d440b41c0 2023-10-06T05:24:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.147+0000 7f0d327fc700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d24066f30 0x7f0d240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.147+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d3c085460 con 0x7f0d440b41c0 2023-10-06T05:24:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.148+0000 7f0d4259c700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d24066f30 0x7f0d240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.148+0000 7f0d4259c700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d24066f30 0x7f0d240693e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0d340099b0 tx=0x7f0d34017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.151+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d3c055f60 con 0x7f0d440b41c0 2023-10-06T05:24:30.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.347+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d44157810 con 0x7f0d24066f30 2023-10-06T05:24:30.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.358+0000 7f0d327fc700 1 -- 172.21.15.33:0/844004965 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311293 (secure 0 0 0) 0x7f0d44157810 con 0x7f0d24066f30 2023-10-06T05:24:30.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d24066f30 msgr2=0x7f0d240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d24066f30 0x7f0d240693e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0d340099b0 tx=0x7f0d34017000 comp rx=0 tx=0).stop 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 msgr2=0x7f0d440c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 0x7f0d440c75b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009e90 tx=0x7f0d3c00b040 comp rx=0 tx=0).stop 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 shutdown_connections 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d24066f30 0x7f0d240693e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d440b41c0 0x7f0d440c75b0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d440cc750 0x7f0d440cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 --2- 172.21.15.33:0/844004965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d440ccfc0 0x7f0d440b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 >> 172.21.15.33:0/844004965 conn(0x7f0d4406cff0 msgr2=0x7f0d440c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 shutdown_connections 2023-10-06T05:24:30.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.362+0000 7f0d4359e700 1 -- 172.21.15.33:0/844004965 wait complete. 2023-10-06T05:24:30.367 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:24:30.381 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2023-10-06T05:24:29.967211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426244,"kb_used_data":15172,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548476,"statfs":{"total":773094113280,"available":770609639424,"internally_reserved":0,"allocated":15536128,"data_stored":7785370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.974683"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.030065+0000","last_change":"2023-10-06T05:24:26.030065+0000","last_active":"2023-10-06T05:24:26.030065+0000","last_peered":"2023-10-06T05:24:26.030065+0000","last_clean":"2023-10-06T05:24:26.030065+0000","last_became_active":"2023-10-06T05:24:26.029754+0000","last_became_peered":"2023-10-06T05:24:26.029754+0000","last_unstale":"2023-10-06T05:24:26.030065+0000","last_undegraded":"2023-10-06T05:24:26.030065+0000","last_fullsized":"2023-10-06T05:24:26.030065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:23.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.019316+0000","last_change":"2023-10-06T05:24:26.019316+0000","last_active":"2023-10-06T05:24:26.019316+0000","last_peered":"2023-10-06T05:24:26.019316+0000","last_clean":"2023-10-06T05:24:26.019316+0000","last_became_active":"2023-10-06T05:24:26.019131+0000","last_became_peered":"2023-10-06T05:24:26.019131+0000","last_unstale":"2023-10-06T05:24:26.019316+0000","last_undegraded":"2023-10-06T05:24:26.019316+0000","last_fullsized":"2023-10-06T05:24:26.019316+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:33.272368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029308+0000","last_change":"2023-10-06T05:24:26.029308+0000","last_active":"2023-10-06T05:24:26.029308+0000","last_peered":"2023-10-06T05:24:26.029308+0000","last_clean":"2023-10-06T05:24:26.029308+0000","last_became_active":"2023-10-06T05:24:26.028982+0000","last_became_peered":"2023-10-06T05:24:26.028982+0000","last_unstale":"2023-10-06T05:24:26.029308+0000","last_undegraded":"2023-10-06T05:24:26.029308+0000","last_fullsized":"2023-10-06T05:24:26.029308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:12.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.020981+0000","last_change":"2023-10-06T05:24:26.020981+0000","last_active":"2023-10-06T05:24:26.020981+0000","last_peered":"2023-10-06T05:24:26.020981+0000","last_clean":"2023-10-06T05:24:26.020981+0000","last_became_active":"2023-10-06T05:24:26.020673+0000","last_became_peered":"2023-10-06T05:24:26.020673+0000","last_unstale":"2023-10-06T05:24:26.020981+0000","last_undegraded":"2023-10-06T05:24:26.020981+0000","last_fullsized":"2023-10-06T05:24:26.020981+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:54.205522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036648+0000","last_change":"2023-10-06T05:24:26.036648+0000","last_active":"2023-10-06T05:24:26.036648+0000","last_peered":"2023-10-06T05:24:26.036648+0000","last_clean":"2023-10-06T05:24:26.036648+0000","last_became_active":"2023-10-06T05:24:26.036171+0000","last_became_peered":"2023-10-06T05:24:26.036171+0000","last_unstale":"2023-10-06T05:24:26.036648+0000","last_undegraded":"2023-10-06T05:24:26.036648+0000","last_fullsized":"2023-10-06T05:24:26.036648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:52.160651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039175+0000","last_change":"2023-10-06T05:24:26.039175+0000","last_active":"2023-10-06T05:24:26.039175+0000","last_peered":"2023-10-06T05:24:26.039175+0000","last_clean":"2023-10-06T05:24:26.039175+0000","last_became_active":"2023-10-06T05:24:26.038917+0000","last_became_peered":"2023-10-06T05:24:26.038917+0000","last_unstale":"2023-10-06T05:24:26.039175+0000","last_undegraded":"2023-10-06T05:24:26.039175+0000","last_fullsized":"2023-10-06T05:24:26.039175+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:45.824772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.030820+0000","last_change":"2023-10-06T05:24:26.030820+0000","last_active":"2023-10-06T05:24:26.030820+0000","last_peered":"2023-10-06T05:24:26.030820+0000","last_clean":"2023-10-06T05:24:26.030820+0000","last_became_active":"2023-10-06T05:24:26.030516+0000","last_became_peered":"2023-10-06T05:24:26.030516+0000","last_unstale":"2023-10-06T05:24:26.030820+0000","last_undegraded":"2023-10-06T05:24:26.030820+0000","last_fullsized":"2023-10-06T05:24:26.030820+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:40.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025104+0000","last_change":"2023-10-06T05:24:26.025104+0000","last_active":"2023-10-06T05:24:26.025104+0000","last_peered":"2023-10-06T05:24:26.025104+0000","last_clean":"2023-10-06T05:24:26.025104+0000","last_became_active":"2023-10-06T05:24:26.024810+0000","last_became_peered":"2023-10-06T05:24:26.024810+0000","last_unstale":"2023-10-06T05:24:26.025104+0000","last_undegraded":"2023-10-06T05:24:26.025104+0000","last_fullsized":"2023-10-06T05:24:26.025104+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:42.251029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033411+0000","last_change":"2023-10-06T05:24:26.033411+0000","last_active":"2023-10-06T05:24:26.033411+0000","last_peered":"2023-10-06T05:24:26.033411+0000","last_clean":"2023-10-06T05:24:26.033411+0000","last_became_active":"2023-10-06T05:24:26.030165+0000","last_became_peered":"2023-10-06T05:24:26.030165+0000","last_unstale":"2023-10-06T05:24:26.033411+0000","last_undegraded":"2023-10-06T05:24:26.033411+0000","last_fullsized":"2023-10-06T05:24:26.033411+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:54.852065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033969+0000","last_change":"2023-10-06T05:24:26.033969+0000","last_active":"2023-10-06T05:24:26.033969+0000","last_peered":"2023-10-06T05:24:26.033969+0000","last_clean":"2023-10-06T05:24:26.033969+0000","last_became_active":"2023-10-06T05:24:26.033221+0000","last_became_peered":"2023-10-06T05:24:26.033221+0000","last_unstale":"2023-10-06T05:24:26.033969+0000","last_undegraded":"2023-10-06T05:24:26.033969+0000","last_fullsized":"2023-10-06T05:24:26.033969+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:05.486519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.023631+0000","last_change":"2023-10-06T05:24:26.023631+0000","last_active":"2023-10-06T05:24:26.023631+0000","last_peered":"2023-10-06T05:24:26.023631+0000","last_clean":"2023-10-06T05:24:26.023631+0000","last_became_active":"2023-10-06T05:24:26.023188+0000","last_became_peered":"2023-10-06T05:24:26.023188+0000","last_unstale":"2023-10-06T05:24:26.023631+0000","last_undegraded":"2023-10-06T05:24:26.023631+0000","last_fullsized":"2023-10-06T05:24:26.023631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:48.339847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.040460+0000","last_change":"2023-10-06T05:24:26.040460+0000","last_active":"2023-10-06T05:24:26.040460+0000","last_peered":"2023-10-06T05:24:26.040460+0000","last_clean":"2023-10-06T05:24:26.040460+0000","last_became_active":"2023-10-06T05:24:26.040210+0000","last_became_peered":"2023-10-06T05:24:26.040210+0000","last_unstale":"2023-10-06T05:24:26.040460+0000","last_undegraded":"2023-10-06T05:24:26.040460+0000","last_fullsized":"2023-10-06T05:24:26.040460+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:11.434671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.037292+0000","last_change":"2023-10-06T05:24:26.037292+0000","last_active":"2023-10-06T05:24:26.037292+0000","last_peered":"2023-10-06T05:24:26.037292+0000","last_clean":"2023-10-06T05:24:26.037292+0000","last_became_active":"2023-10-06T05:24:26.036377+0000","last_became_peered":"2023-10-06T05:24:26.036377+0000","last_unstale":"2023-10-06T05:24:26.037292+0000","last_undegraded":"2023-10-06T05:24:26.037292+0000","last_fullsized":"2023-10-06T05:24:26.037292+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:00.692340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.035166+0000","last_change":"2023-10-06T05:24:26.035166+0000","last_active":"2023-10-06T05:24:26.035166+0000","last_peered":"2023-10-06T05:24:26.035166+0000","last_clean":"2023-10-06T05:24:26.035166+0000","last_became_active":"2023-10-06T05:24:26.034892+0000","last_became_peered":"2023-10-06T05:24:26.034892+0000","last_unstale":"2023-10-06T05:24:26.035166+0000","last_undegraded":"2023-10-06T05:24:26.035166+0000","last_fullsized":"2023-10-06T05:24:26.035166+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:48.744050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025545+0000","last_change":"2023-10-06T05:24:26.025545+0000","last_active":"2023-10-06T05:24:26.025545+0000","last_peered":"2023-10-06T05:24:26.025545+0000","last_clean":"2023-10-06T05:24:26.025545+0000","last_became_active":"2023-10-06T05:24:26.014072+0000","last_became_peered":"2023-10-06T05:24:26.014072+0000","last_unstale":"2023-10-06T05:24:26.025545+0000","last_undegraded":"2023-10-06T05:24:26.025545+0000","last_fullsized":"2023-10-06T05:24:26.025545+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:42.251029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.034588+0000","last_change":"2023-10-06T05:24:26.034588+0000","last_active":"2023-10-06T05:24:26.034588+0000","last_peered":"2023-10-06T05:24:26.034588+0000","last_clean":"2023-10-06T05:24:26.034588+0000","last_became_active":"2023-10-06T05:24:26.034286+0000","last_became_peered":"2023-10-06T05:24:26.034286+0000","last_unstale":"2023-10-06T05:24:26.034588+0000","last_undegraded":"2023-10-06T05:24:26.034588+0000","last_fullsized":"2023-10-06T05:24:26.034588+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:12.738848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.015833+0000","last_change":"2023-10-06T05:24:26.015833+0000","last_active":"2023-10-06T05:24:26.015833+0000","last_peered":"2023-10-06T05:24:26.015833+0000","last_clean":"2023-10-06T05:24:26.015833+0000","last_became_active":"2023-10-06T05:24:26.015568+0000","last_became_peered":"2023-10-06T05:24:26.015568+0000","last_unstale":"2023-10-06T05:24:26.015833+0000","last_undegraded":"2023-10-06T05:24:26.015833+0000","last_fullsized":"2023-10-06T05:24:26.015833+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:04.408452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024224+0000","last_change":"2023-10-06T05:24:26.024224+0000","last_active":"2023-10-06T05:24:26.024224+0000","last_peered":"2023-10-06T05:24:26.024224+0000","last_clean":"2023-10-06T05:24:26.024224+0000","last_became_active":"2023-10-06T05:24:26.023934+0000","last_became_peered":"2023-10-06T05:24:26.023934+0000","last_unstale":"2023-10-06T05:24:26.024224+0000","last_undegraded":"2023-10-06T05:24:26.024224+0000","last_fullsized":"2023-10-06T05:24:26.024224+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:56.282142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029407+0000","last_change":"2023-10-06T05:24:26.029407+0000","last_active":"2023-10-06T05:24:26.029407+0000","last_peered":"2023-10-06T05:24:26.029407+0000","last_clean":"2023-10-06T05:24:26.029407+0000","last_became_active":"2023-10-06T05:24:26.029023+0000","last_became_peered":"2023-10-06T05:24:26.029023+0000","last_unstale":"2023-10-06T05:24:26.029407+0000","last_undegraded":"2023-10-06T05:24:26.029407+0000","last_fullsized":"2023-10-06T05:24:26.029407+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:11.684203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.020916+0000","last_change":"2023-10-06T05:24:26.020916+0000","last_active":"2023-10-06T05:24:26.020916+0000","last_peered":"2023-10-06T05:24:26.020916+0000","last_clean":"2023-10-06T05:24:26.020916+0000","last_became_active":"2023-10-06T05:24:26.020641+0000","last_became_peered":"2023-10-06T05:24:26.020641+0000","last_unstale":"2023-10-06T05:24:26.020916+0000","last_undegraded":"2023-10-06T05:24:26.020916+0000","last_fullsized":"2023-10-06T05:24:26.020916+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:46.027779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.037535+0000","last_change":"2023-10-06T05:24:26.037535+0000","last_active":"2023-10-06T05:24:26.037535+0000","last_peered":"2023-10-06T05:24:26.037535+0000","last_clean":"2023-10-06T05:24:26.037535+0000","last_became_active":"2023-10-06T05:24:26.036898+0000","last_became_peered":"2023-10-06T05:24:26.036898+0000","last_unstale":"2023-10-06T05:24:26.037535+0000","last_undegraded":"2023-10-06T05:24:26.037535+0000","last_fullsized":"2023-10-06T05:24:26.037535+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:47.028934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033254+0000","last_change":"2023-10-06T05:24:26.033254+0000","last_active":"2023-10-06T05:24:26.033254+0000","last_peered":"2023-10-06T05:24:26.033254+0000","last_clean":"2023-10-06T05:24:26.033254+0000","last_became_active":"2023-10-06T05:24:26.031191+0000","last_became_peered":"2023-10-06T05:24:26.031191+0000","last_unstale":"2023-10-06T05:24:26.033254+0000","last_undegraded":"2023-10-06T05:24:26.033254+0000","last_fullsized":"2023-10-06T05:24:26.033254+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:07.984962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024977+0000","last_change":"2023-10-06T05:24:26.024977+0000","last_active":"2023-10-06T05:24:26.024977+0000","last_peered":"2023-10-06T05:24:26.024977+0000","last_clean":"2023-10-06T05:24:26.024977+0000","last_became_active":"2023-10-06T05:24:26.024676+0000","last_became_peered":"2023-10-06T05:24:26.024676+0000","last_unstale":"2023-10-06T05:24:26.024977+0000","last_undegraded":"2023-10-06T05:24:26.024977+0000","last_fullsized":"2023-10-06T05:24:26.024977+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:33.272368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.032208+0000","last_change":"2023-10-06T05:24:26.032208+0000","last_active":"2023-10-06T05:24:26.032208+0000","last_peered":"2023-10-06T05:24:26.032208+0000","last_clean":"2023-10-06T05:24:26.032208+0000","last_became_active":"2023-10-06T05:24:26.031940+0000","last_became_peered":"2023-10-06T05:24:26.031940+0000","last_unstale":"2023-10-06T05:24:26.032208+0000","last_undegraded":"2023-10-06T05:24:26.032208+0000","last_fullsized":"2023-10-06T05:24:26.032208+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:27.115511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.026801+0000","last_change":"2023-10-06T05:24:26.026801+0000","last_active":"2023-10-06T05:24:26.026801+0000","last_peered":"2023-10-06T05:24:26.026801+0000","last_clean":"2023-10-06T05:24:26.026801+0000","last_became_active":"2023-10-06T05:24:26.026488+0000","last_became_peered":"2023-10-06T05:24:26.026488+0000","last_unstale":"2023-10-06T05:24:26.026801+0000","last_undegraded":"2023-10-06T05:24:26.026801+0000","last_fullsized":"2023-10-06T05:24:26.026801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:46.521669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025065+0000","last_change":"2023-10-06T05:24:26.025065+0000","last_active":"2023-10-06T05:24:26.025065+0000","last_peered":"2023-10-06T05:24:26.025065+0000","last_clean":"2023-10-06T05:24:26.025065+0000","last_became_active":"2023-10-06T05:24:26.024792+0000","last_became_peered":"2023-10-06T05:24:26.024792+0000","last_unstale":"2023-10-06T05:24:26.025065+0000","last_undegraded":"2023-10-06T05:24:26.025065+0000","last_fullsized":"2023-10-06T05:24:26.025065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:07.102122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036635+0000","last_change":"2023-10-06T05:24:26.036635+0000","last_active":"2023-10-06T05:24:26.036635+0000","last_peered":"2023-10-06T05:24:26.036635+0000","last_clean":"2023-10-06T05:24:26.036635+0000","last_became_active":"2023-10-06T05:24:26.036298+0000","last_became_peered":"2023-10-06T05:24:26.036298+0000","last_unstale":"2023-10-06T05:24:26.036635+0000","last_undegraded":"2023-10-06T05:24:26.036635+0000","last_fullsized":"2023-10-06T05:24:26.036635+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:51.249505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028856+0000","last_change":"2023-10-06T05:24:26.028856+0000","last_active":"2023-10-06T05:24:26.028856+0000","last_peered":"2023-10-06T05:24:26.028856+0000","last_clean":"2023-10-06T05:24:26.028856+0000","last_became_active":"2023-10-06T05:24:26.026938+0000","last_became_peered":"2023-10-06T05:24:26.026938+0000","last_unstale":"2023-10-06T05:24:26.028856+0000","last_undegraded":"2023-10-06T05:24:26.028856+0000","last_fullsized":"2023-10-06T05:24:26.028856+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:19.168120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039686+0000","last_change":"2023-10-06T05:24:26.039686+0000","last_active":"2023-10-06T05:24:26.039686+0000","last_peered":"2023-10-06T05:24:26.039686+0000","last_clean":"2023-10-06T05:24:26.039686+0000","last_became_active":"2023-10-06T05:24:26.039411+0000","last_became_peered":"2023-10-06T05:24:26.039411+0000","last_unstale":"2023-10-06T05:24:26.039686+0000","last_undegraded":"2023-10-06T05:24:26.039686+0000","last_fullsized":"2023-10-06T05:24:26.039686+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:06.625908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.019299+0000","last_change":"2023-10-06T05:24:26.019299+0000","last_active":"2023-10-06T05:24:26.019299+0000","last_peered":"2023-10-06T05:24:26.019299+0000","last_clean":"2023-10-06T05:24:26.019299+0000","last_became_active":"2023-10-06T05:24:26.019033+0000","last_became_peered":"2023-10-06T05:24:26.019033+0000","last_unstale":"2023-10-06T05:24:26.019299+0000","last_undegraded":"2023-10-06T05:24:26.019299+0000","last_fullsized":"2023-10-06T05:24:26.019299+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:27.986222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.035236+0000","last_change":"2023-10-06T05:24:26.035236+0000","last_active":"2023-10-06T05:24:26.035236+0000","last_peered":"2023-10-06T05:24:26.035236+0000","last_clean":"2023-10-06T05:24:26.035236+0000","last_became_active":"2023-10-06T05:24:26.034977+0000","last_became_peered":"2023-10-06T05:24:26.034977+0000","last_unstale":"2023-10-06T05:24:26.035236+0000","last_undegraded":"2023-10-06T05:24:26.035236+0000","last_fullsized":"2023-10-06T05:24:26.035236+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:53.238696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025712+0000","last_change":"2023-10-06T05:24:26.025712+0000","last_active":"2023-10-06T05:24:26.025712+0000","last_peered":"2023-10-06T05:24:26.025712+0000","last_clean":"2023-10-06T05:24:26.025712+0000","last_became_active":"2023-10-06T05:24:26.025453+0000","last_became_peered":"2023-10-06T05:24:26.025453+0000","last_unstale":"2023-10-06T05:24:26.025712+0000","last_undegraded":"2023-10-06T05:24:26.025712+0000","last_fullsized":"2023-10-06T05:24:26.025712+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:52.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025531+0000","last_change":"2023-10-06T05:24:26.025531+0000","last_active":"2023-10-06T05:24:26.025531+0000","last_peered":"2023-10-06T05:24:26.025531+0000","last_clean":"2023-10-06T05:24:26.025531+0000","last_became_active":"2023-10-06T05:24:26.025246+0000","last_became_peered":"2023-10-06T05:24:26.025246+0000","last_unstale":"2023-10-06T05:24:26.025531+0000","last_undegraded":"2023-10-06T05:24:26.025531+0000","last_fullsized":"2023-10-06T05:24:26.025531+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:06.625908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"143'4","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.042143+0000","last_change":"2023-10-06T05:24:25.013400+0000","last_active":"2023-10-06T05:24:26.042143+0000","last_peered":"2023-10-06T05:24:26.042143+0000","last_clean":"2023-10-06T05:24:26.042143+0000","last_became_active":"2023-10-06T05:24:25.013121+0000","last_became_peered":"2023-10-06T05:24:25.013121+0000","last_unstale":"2023-10-06T05:24:26.042143+0000","last_undegraded":"2023-10-06T05:24:26.042143+0000","last_fullsized":"2023-10-06T05:24:26.042143+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:45.512677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.018801+0000","last_change":"2023-10-06T05:24:26.018801+0000","last_active":"2023-10-06T05:24:26.018801+0000","last_peered":"2023-10-06T05:24:26.018801+0000","last_clean":"2023-10-06T05:24:26.018801+0000","last_became_active":"2023-10-06T05:24:26.018487+0000","last_became_peered":"2023-10-06T05:24:26.018487+0000","last_unstale":"2023-10-06T05:24:26.018801+0000","last_undegraded":"2023-10-06T05:24:26.018801+0000","last_fullsized":"2023-10-06T05:24:26.018801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:35.985599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.013277+0000","last_change":"2023-10-06T05:24:25.024469+0000","last_active":"2023-10-06T05:24:26.013277+0000","last_peered":"2023-10-06T05:24:26.013277+0000","last_clean":"2023-10-06T05:24:26.013277+0000","last_became_active":"2023-10-06T05:24:25.024125+0000","last_became_peered":"2023-10-06T05:24:25.024125+0000","last_unstale":"2023-10-06T05:24:26.013277+0000","last_undegraded":"2023-10-06T05:24:26.013277+0000","last_fullsized":"2023-10-06T05:24:26.013277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:34.976606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.014014+0000","last_change":"2023-10-06T05:24:26.014014+0000","last_active":"2023-10-06T05:24:26.014014+0000","last_peered":"2023-10-06T05:24:26.014014+0000","last_clean":"2023-10-06T05:24:26.014014+0000","last_became_active":"2023-10-06T05:24:26.013723+0000","last_became_peered":"2023-10-06T05:24:26.013723+0000","last_unstale":"2023-10-06T05:24:26.014014+0000","last_undegraded":"2023-10-06T05:24:26.014014+0000","last_fullsized":"2023-10-06T05:24:26.014014+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:35.985599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039517+0000","last_change":"2023-10-06T05:24:25.024540+0000","last_active":"2023-10-06T05:24:26.039517+0000","last_peered":"2023-10-06T05:24:26.039517+0000","last_clean":"2023-10-06T05:24:26.039517+0000","last_became_active":"2023-10-06T05:24:25.024219+0000","last_became_peered":"2023-10-06T05:24:25.024219+0000","last_unstale":"2023-10-06T05:24:26.039517+0000","last_undegraded":"2023-10-06T05:24:26.039517+0000","last_fullsized":"2023-10-06T05:24:26.039517+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:51.375221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.031842+0000","last_change":"2023-10-06T05:24:26.031842+0000","last_active":"2023-10-06T05:24:26.031842+0000","last_peered":"2023-10-06T05:24:26.031842+0000","last_clean":"2023-10-06T05:24:26.031842+0000","last_became_active":"2023-10-06T05:24:26.031563+0000","last_became_peered":"2023-10-06T05:24:26.031563+0000","last_unstale":"2023-10-06T05:24:26.031842+0000","last_undegraded":"2023-10-06T05:24:26.031842+0000","last_fullsized":"2023-10-06T05:24:26.031842+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:52.160651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029336+0000","last_change":"2023-10-06T05:24:25.019593+0000","last_active":"2023-10-06T05:24:26.029336+0000","last_peered":"2023-10-06T05:24:26.029336+0000","last_clean":"2023-10-06T05:24:26.029336+0000","last_became_active":"2023-10-06T05:24:25.019304+0000","last_became_peered":"2023-10-06T05:24:25.019304+0000","last_unstale":"2023-10-06T05:24:26.029336+0000","last_undegraded":"2023-10-06T05:24:26.029336+0000","last_fullsized":"2023-10-06T05:24:26.029336+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:39.487652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.020429+0000","last_change":"2023-10-06T05:24:26.020429+0000","last_active":"2023-10-06T05:24:26.020429+0000","last_peered":"2023-10-06T05:24:26.020429+0000","last_clean":"2023-10-06T05:24:26.020429+0000","last_became_active":"2023-10-06T05:24:26.020094+0000","last_became_peered":"2023-10-06T05:24:26.020094+0000","last_unstale":"2023-10-06T05:24:26.020429+0000","last_undegraded":"2023-10-06T05:24:26.020429+0000","last_fullsized":"2023-10-06T05:24:26.020429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:50.880104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.008587+0000","last_change":"2023-10-06T05:24:25.021114+0000","last_active":"2023-10-06T05:24:26.008587+0000","last_peered":"2023-10-06T05:24:26.008587+0000","last_clean":"2023-10-06T05:24:26.008587+0000","last_became_active":"2023-10-06T05:24:25.020763+0000","last_became_peered":"2023-10-06T05:24:25.020763+0000","last_unstale":"2023-10-06T05:24:26.008587+0000","last_undegraded":"2023-10-06T05:24:26.008587+0000","last_fullsized":"2023-10-06T05:24:26.008587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:44.414631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029187+0000","last_change":"2023-10-06T05:24:26.029187+0000","last_active":"2023-10-06T05:24:26.029187+0000","last_peered":"2023-10-06T05:24:26.029187+0000","last_clean":"2023-10-06T05:24:26.029187+0000","last_became_active":"2023-10-06T05:24:26.028726+0000","last_became_peered":"2023-10-06T05:24:26.028726+0000","last_unstale":"2023-10-06T05:24:26.029187+0000","last_undegraded":"2023-10-06T05:24:26.029187+0000","last_fullsized":"2023-10-06T05:24:26.029187+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.022041+0000","last_change":"2023-10-06T05:24:25.018287+0000","last_active":"2023-10-06T05:24:26.022041+0000","last_peered":"2023-10-06T05:24:26.022041+0000","last_clean":"2023-10-06T05:24:26.022041+0000","last_became_active":"2023-10-06T05:24:25.018010+0000","last_became_peered":"2023-10-06T05:24:25.018010+0000","last_unstale":"2023-10-06T05:24:26.022041+0000","last_undegraded":"2023-10-06T05:24:26.022041+0000","last_fullsized":"2023-10-06T05:24:26.022041+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:58.475188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025051+0000","last_change":"2023-10-06T05:24:26.025051+0000","last_active":"2023-10-06T05:24:26.025051+0000","last_peered":"2023-10-06T05:24:26.025051+0000","last_clean":"2023-10-06T05:24:26.025051+0000","last_became_active":"2023-10-06T05:24:26.024353+0000","last_became_peered":"2023-10-06T05:24:26.024353+0000","last_unstale":"2023-10-06T05:24:26.025051+0000","last_undegraded":"2023-10-06T05:24:26.025051+0000","last_fullsized":"2023-10-06T05:24:26.025051+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:46.521669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029602+0000","last_change":"2023-10-06T05:24:25.025798+0000","last_active":"2023-10-06T05:24:25.029602+0000","last_peered":"2023-10-06T05:24:25.029602+0000","last_clean":"2023-10-06T05:24:25.029602+0000","last_became_active":"2023-10-06T05:24:25.025370+0000","last_became_peered":"2023-10-06T05:24:25.025370+0000","last_unstale":"2023-10-06T05:24:25.029602+0000","last_undegraded":"2023-10-06T05:24:25.029602+0000","last_fullsized":"2023-10-06T05:24:25.029602+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:27.439082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.021285+0000","last_change":"2023-10-06T05:24:26.021285+0000","last_active":"2023-10-06T05:24:26.021285+0000","last_peered":"2023-10-06T05:24:26.021285+0000","last_clean":"2023-10-06T05:24:26.021285+0000","last_became_active":"2023-10-06T05:24:26.021039+0000","last_became_peered":"2023-10-06T05:24:26.021039+0000","last_unstale":"2023-10-06T05:24:26.021285+0000","last_undegraded":"2023-10-06T05:24:26.021285+0000","last_fullsized":"2023-10-06T05:24:26.021285+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:04.330883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.021993+0000","last_change":"2023-10-06T05:24:25.020326+0000","last_active":"2023-10-06T05:24:25.021993+0000","last_peered":"2023-10-06T05:24:25.021993+0000","last_clean":"2023-10-06T05:24:25.021993+0000","last_became_active":"2023-10-06T05:24:25.020068+0000","last_became_peered":"2023-10-06T05:24:25.020068+0000","last_unstale":"2023-10-06T05:24:25.021993+0000","last_undegraded":"2023-10-06T05:24:25.021993+0000","last_fullsized":"2023-10-06T05:24:25.021993+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:03.321890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.040460+0000","last_change":"2023-10-06T05:24:26.040460+0000","last_active":"2023-10-06T05:24:26.040460+0000","last_peered":"2023-10-06T05:24:26.040460+0000","last_clean":"2023-10-06T05:24:26.040460+0000","last_became_active":"2023-10-06T05:24:26.040114+0000","last_became_peered":"2023-10-06T05:24:26.040114+0000","last_unstale":"2023-10-06T05:24:26.040460+0000","last_undegraded":"2023-10-06T05:24:26.040460+0000","last_fullsized":"2023-10-06T05:24:26.040460+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:51.249505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.011094+0000","last_change":"2023-10-06T05:24:25.023229+0000","last_active":"2023-10-06T05:24:26.011094+0000","last_peered":"2023-10-06T05:24:26.011094+0000","last_clean":"2023-10-06T05:24:26.011094+0000","last_became_active":"2023-10-06T05:24:25.022820+0000","last_became_peered":"2023-10-06T05:24:25.022820+0000","last_unstale":"2023-10-06T05:24:26.011094+0000","last_undegraded":"2023-10-06T05:24:26.011094+0000","last_fullsized":"2023-10-06T05:24:26.011094+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:09.349951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029035+0000","last_change":"2023-10-06T05:24:26.029035+0000","last_active":"2023-10-06T05:24:26.029035+0000","last_peered":"2023-10-06T05:24:26.029035+0000","last_clean":"2023-10-06T05:24:26.029035+0000","last_became_active":"2023-10-06T05:24:26.028759+0000","last_became_peered":"2023-10-06T05:24:26.028759+0000","last_unstale":"2023-10-06T05:24:26.029035+0000","last_undegraded":"2023-10-06T05:24:26.029035+0000","last_fullsized":"2023-10-06T05:24:26.029035+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:53.238696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.032742+0000","last_change":"2023-10-06T05:24:25.023013+0000","last_active":"2023-10-06T05:24:26.032742+0000","last_peered":"2023-10-06T05:24:26.032742+0000","last_clean":"2023-10-06T05:24:26.032742+0000","last_became_active":"2023-10-06T05:24:25.022243+0000","last_became_peered":"2023-10-06T05:24:25.022243+0000","last_unstale":"2023-10-06T05:24:26.032742+0000","last_undegraded":"2023-10-06T05:24:26.032742+0000","last_fullsized":"2023-10-06T05:24:26.032742+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:37.782847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.041720+0000","last_change":"2023-10-06T05:24:26.041720+0000","last_active":"2023-10-06T05:24:26.041720+0000","last_peered":"2023-10-06T05:24:26.041720+0000","last_clean":"2023-10-06T05:24:26.041720+0000","last_became_active":"2023-10-06T05:24:26.041385+0000","last_became_peered":"2023-10-06T05:24:26.041385+0000","last_unstale":"2023-10-06T05:24:26.041720+0000","last_undegraded":"2023-10-06T05:24:26.041720+0000","last_fullsized":"2023-10-06T05:24:26.041720+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:18.157493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.007376+0000","last_change":"2023-10-06T05:24:25.020411+0000","last_active":"2023-10-06T05:24:26.007376+0000","last_peered":"2023-10-06T05:24:26.007376+0000","last_clean":"2023-10-06T05:24:26.007376+0000","last_became_active":"2023-10-06T05:24:25.020162+0000","last_became_peered":"2023-10-06T05:24:25.020162+0000","last_unstale":"2023-10-06T05:24:26.007376+0000","last_undegraded":"2023-10-06T05:24:26.007376+0000","last_fullsized":"2023-10-06T05:24:26.007376+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:09.742919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033544+0000","last_change":"2023-10-06T05:24:26.033544+0000","last_active":"2023-10-06T05:24:26.033544+0000","last_peered":"2023-10-06T05:24:26.033544+0000","last_clean":"2023-10-06T05:24:26.033544+0000","last_became_active":"2023-10-06T05:24:26.031554+0000","last_became_peered":"2023-10-06T05:24:26.031554+0000","last_unstale":"2023-10-06T05:24:26.033544+0000","last_undegraded":"2023-10-06T05:24:26.033544+0000","last_fullsized":"2023-10-06T05:24:26.033544+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:37.201579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029146+0000","last_change":"2023-10-06T05:24:25.027334+0000","last_active":"2023-10-06T05:24:25.029146+0000","last_peered":"2023-10-06T05:24:25.029146+0000","last_clean":"2023-10-06T05:24:25.029146+0000","last_became_active":"2023-10-06T05:24:25.027034+0000","last_became_peered":"2023-10-06T05:24:25.027034+0000","last_unstale":"2023-10-06T05:24:25.029146+0000","last_undegraded":"2023-10-06T05:24:25.029146+0000","last_fullsized":"2023-10-06T05:24:25.029146+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:18.059631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025280+0000","last_change":"2023-10-06T05:24:26.025280+0000","last_active":"2023-10-06T05:24:26.025280+0000","last_peered":"2023-10-06T05:24:26.025280+0000","last_clean":"2023-10-06T05:24:26.025280+0000","last_became_active":"2023-10-06T05:24:26.024992+0000","last_became_peered":"2023-10-06T05:24:26.024992+0000","last_unstale":"2023-10-06T05:24:26.025280+0000","last_undegraded":"2023-10-06T05:24:26.025280+0000","last_fullsized":"2023-10-06T05:24:26.025280+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:12.738848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.031325+0000","last_change":"2023-10-06T05:24:25.021450+0000","last_active":"2023-10-06T05:24:26.031325+0000","last_peered":"2023-10-06T05:24:26.031325+0000","last_clean":"2023-10-06T05:24:26.031325+0000","last_became_active":"2023-10-06T05:24:25.021072+0000","last_became_peered":"2023-10-06T05:24:25.021072+0000","last_unstale":"2023-10-06T05:24:26.031325+0000","last_undegraded":"2023-10-06T05:24:26.031325+0000","last_fullsized":"2023-10-06T05:24:26.031325+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:00.985124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028000+0000","last_change":"2023-10-06T05:24:26.028000+0000","last_active":"2023-10-06T05:24:26.028000+0000","last_peered":"2023-10-06T05:24:26.028000+0000","last_clean":"2023-10-06T05:24:26.028000+0000","last_became_active":"2023-10-06T05:24:26.027736+0000","last_became_peered":"2023-10-06T05:24:26.027736+0000","last_unstale":"2023-10-06T05:24:26.028000+0000","last_undegraded":"2023-10-06T05:24:26.028000+0000","last_fullsized":"2023-10-06T05:24:26.028000+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:38.791840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029445+0000","last_change":"2023-10-06T05:24:25.025497+0000","last_active":"2023-10-06T05:24:25.029445+0000","last_peered":"2023-10-06T05:24:25.029445+0000","last_clean":"2023-10-06T05:24:25.029445+0000","last_became_active":"2023-10-06T05:24:25.025185+0000","last_became_peered":"2023-10-06T05:24:25.025185+0000","last_unstale":"2023-10-06T05:24:25.029445+0000","last_undegraded":"2023-10-06T05:24:25.029445+0000","last_fullsized":"2023-10-06T05:24:25.029445+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:33.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033727+0000","last_change":"2023-10-06T05:24:26.033727+0000","last_active":"2023-10-06T05:24:26.033727+0000","last_peered":"2023-10-06T05:24:26.033727+0000","last_clean":"2023-10-06T05:24:26.033727+0000","last_became_active":"2023-10-06T05:24:26.033465+0000","last_became_peered":"2023-10-06T05:24:26.033465+0000","last_unstale":"2023-10-06T05:24:26.033727+0000","last_undegraded":"2023-10-06T05:24:26.033727+0000","last_fullsized":"2023-10-06T05:24:26.033727+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:45.824772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.023401+0000","last_change":"2023-10-06T05:24:25.021480+0000","last_active":"2023-10-06T05:24:25.023401+0000","last_peered":"2023-10-06T05:24:25.023401+0000","last_clean":"2023-10-06T05:24:25.023401+0000","last_became_active":"2023-10-06T05:24:25.021150+0000","last_became_peered":"2023-10-06T05:24:25.021150+0000","last_unstale":"2023-10-06T05:24:25.023401+0000","last_undegraded":"2023-10-06T05:24:25.023401+0000","last_fullsized":"2023-10-06T05:24:25.023401+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:53.843072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.018797+0000","last_change":"2023-10-06T05:24:26.018797+0000","last_active":"2023-10-06T05:24:26.018797+0000","last_peered":"2023-10-06T05:24:26.018797+0000","last_clean":"2023-10-06T05:24:26.018797+0000","last_became_active":"2023-10-06T05:24:26.018469+0000","last_became_peered":"2023-10-06T05:24:26.018469+0000","last_unstale":"2023-10-06T05:24:26.018797+0000","last_undegraded":"2023-10-06T05:24:26.018797+0000","last_fullsized":"2023-10-06T05:24:26.018797+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:03.789342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029404+0000","last_change":"2023-10-06T05:24:25.025727+0000","last_active":"2023-10-06T05:24:25.029404+0000","last_peered":"2023-10-06T05:24:25.029404+0000","last_clean":"2023-10-06T05:24:25.029404+0000","last_became_active":"2023-10-06T05:24:25.025416+0000","last_became_peered":"2023-10-06T05:24:25.025416+0000","last_unstale":"2023-10-06T05:24:25.029404+0000","last_undegraded":"2023-10-06T05:24:25.029404+0000","last_fullsized":"2023-10-06T05:24:25.029404+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:09.112325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039366+0000","last_change":"2023-10-06T05:24:26.039366+0000","last_active":"2023-10-06T05:24:26.039366+0000","last_peered":"2023-10-06T05:24:26.039366+0000","last_clean":"2023-10-06T05:24:26.039366+0000","last_became_active":"2023-10-06T05:24:26.038821+0000","last_became_peered":"2023-10-06T05:24:26.038821+0000","last_unstale":"2023-10-06T05:24:26.039366+0000","last_undegraded":"2023-10-06T05:24:26.039366+0000","last_fullsized":"2023-10-06T05:24:26.039366+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:11.684203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028866+0000","last_change":"2023-10-06T05:24:25.029383+0000","last_active":"2023-10-06T05:24:26.028866+0000","last_peered":"2023-10-06T05:24:26.028866+0000","last_clean":"2023-10-06T05:24:26.028866+0000","last_became_active":"2023-10-06T05:24:25.029098+0000","last_became_peered":"2023-10-06T05:24:25.029098+0000","last_unstale":"2023-10-06T05:24:26.028866+0000","last_undegraded":"2023-10-06T05:24:26.028866+0000","last_fullsized":"2023-10-06T05:24:26.028866+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:37.528748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039826+0000","last_change":"2023-10-06T05:24:26.039826+0000","last_active":"2023-10-06T05:24:26.039826+0000","last_peered":"2023-10-06T05:24:26.039826+0000","last_clean":"2023-10-06T05:24:26.039826+0000","last_became_active":"2023-10-06T05:24:26.039562+0000","last_became_peered":"2023-10-06T05:24:26.039562+0000","last_unstale":"2023-10-06T05:24:26.039826+0000","last_undegraded":"2023-10-06T05:24:26.039826+0000","last_fullsized":"2023-10-06T05:24:26.039826+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:37.951417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029773+0000","last_change":"2023-10-06T05:24:25.027434+0000","last_active":"2023-10-06T05:24:25.029773+0000","last_peered":"2023-10-06T05:24:25.029773+0000","last_clean":"2023-10-06T05:24:25.029773+0000","last_became_active":"2023-10-06T05:24:25.026797+0000","last_became_peered":"2023-10-06T05:24:25.026797+0000","last_unstale":"2023-10-06T05:24:25.029773+0000","last_undegraded":"2023-10-06T05:24:25.029773+0000","last_fullsized":"2023-10-06T05:24:25.029773+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:56.176190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.030547+0000","last_change":"2023-10-06T05:24:26.030547+0000","last_active":"2023-10-06T05:24:26.030547+0000","last_peered":"2023-10-06T05:24:26.030547+0000","last_clean":"2023-10-06T05:24:26.030547+0000","last_became_active":"2023-10-06T05:24:26.029799+0000","last_became_peered":"2023-10-06T05:24:26.029799+0000","last_unstale":"2023-10-06T05:24:26.030547+0000","last_undegraded":"2023-10-06T05:24:26.030547+0000","last_fullsized":"2023-10-06T05:24:26.030547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:59.484181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.024714+0000","last_change":"2023-10-06T05:24:25.020809+0000","last_active":"2023-10-06T05:24:25.024714+0000","last_peered":"2023-10-06T05:24:25.024714+0000","last_clean":"2023-10-06T05:24:25.024714+0000","last_became_active":"2023-10-06T05:24:25.020494+0000","last_became_peered":"2023-10-06T05:24:25.020494+0000","last_unstale":"2023-10-06T05:24:25.024714+0000","last_undegraded":"2023-10-06T05:24:25.024714+0000","last_fullsized":"2023-10-06T05:24:25.024714+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.852222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036404+0000","last_change":"2023-10-06T05:24:26.036404+0000","last_active":"2023-10-06T05:24:26.036404+0000","last_peered":"2023-10-06T05:24:26.036404+0000","last_clean":"2023-10-06T05:24:26.036404+0000","last_became_active":"2023-10-06T05:24:26.036121+0000","last_became_peered":"2023-10-06T05:24:26.036121+0000","last_unstale":"2023-10-06T05:24:26.036404+0000","last_undegraded":"2023-10-06T05:24:26.036404+0000","last_fullsized":"2023-10-06T05:24:26.036404+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:46.407284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.023798+0000","last_change":"2023-10-06T05:24:25.022622+0000","last_active":"2023-10-06T05:24:25.023798+0000","last_peered":"2023-10-06T05:24:25.023798+0000","last_clean":"2023-10-06T05:24:25.023798+0000","last_became_active":"2023-10-06T05:24:25.022333+0000","last_became_peered":"2023-10-06T05:24:25.022333+0000","last_unstale":"2023-10-06T05:24:25.023798+0000","last_undegraded":"2023-10-06T05:24:25.023798+0000","last_fullsized":"2023-10-06T05:24:25.023798+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:39.566765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.014888+0000","last_change":"2023-10-06T05:24:26.014888+0000","last_active":"2023-10-06T05:24:26.014888+0000","last_peered":"2023-10-06T05:24:26.014888+0000","last_clean":"2023-10-06T05:24:26.014888+0000","last_became_active":"2023-10-06T05:24:26.014609+0000","last_became_peered":"2023-10-06T05:24:26.014609+0000","last_unstale":"2023-10-06T05:24:26.014888+0000","last_undegraded":"2023-10-06T05:24:26.014888+0000","last_fullsized":"2023-10-06T05:24:26.014888+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:38.791840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.024724+0000","last_change":"2023-10-06T05:24:25.022738+0000","last_active":"2023-10-06T05:24:25.024724+0000","last_peered":"2023-10-06T05:24:25.024724+0000","last_clean":"2023-10-06T05:24:25.024724+0000","last_became_active":"2023-10-06T05:24:25.022469+0000","last_became_peered":"2023-10-06T05:24:25.022469+0000","last_unstale":"2023-10-06T05:24:25.024724+0000","last_undegraded":"2023-10-06T05:24:25.024724+0000","last_fullsized":"2023-10-06T05:24:25.024724+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:02.933521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.035450+0000","last_change":"2023-10-06T05:24:26.035450+0000","last_active":"2023-10-06T05:24:26.035450+0000","last_peered":"2023-10-06T05:24:26.035450+0000","last_clean":"2023-10-06T05:24:26.035450+0000","last_became_active":"2023-10-06T05:24:26.034570+0000","last_became_peered":"2023-10-06T05:24:26.034570+0000","last_unstale":"2023-10-06T05:24:26.035450+0000","last_undegraded":"2023-10-06T05:24:26.035450+0000","last_fullsized":"2023-10-06T05:24:26.035450+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:47.407150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.018482+0000","last_change":"2023-10-06T05:24:25.015760+0000","last_active":"2023-10-06T05:24:25.018482+0000","last_peered":"2023-10-06T05:24:25.018482+0000","last_clean":"2023-10-06T05:24:25.018482+0000","last_became_active":"2023-10-06T05:24:25.015434+0000","last_became_peered":"2023-10-06T05:24:25.015434+0000","last_unstale":"2023-10-06T05:24:25.018482+0000","last_undegraded":"2023-10-06T05:24:25.018482+0000","last_fullsized":"2023-10-06T05:24:25.018482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:14.680983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025301+0000","last_change":"2023-10-06T05:24:26.025301+0000","last_active":"2023-10-06T05:24:26.025301+0000","last_peered":"2023-10-06T05:24:26.025301+0000","last_clean":"2023-10-06T05:24:26.025301+0000","last_became_active":"2023-10-06T05:24:26.024689+0000","last_became_peered":"2023-10-06T05:24:26.024689+0000","last_unstale":"2023-10-06T05:24:26.025301+0000","last_undegraded":"2023-10-06T05:24:26.025301+0000","last_fullsized":"2023-10-06T05:24:26.025301+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:32.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.022344+0000","last_change":"2023-10-06T05:24:25.021014+0000","last_active":"2023-10-06T05:24:25.022344+0000","last_peered":"2023-10-06T05:24:25.022344+0000","last_clean":"2023-10-06T05:24:25.022344+0000","last_became_active":"2023-10-06T05:24:25.020675+0000","last_became_peered":"2023-10-06T05:24:25.020675+0000","last_unstale":"2023-10-06T05:24:25.022344+0000","last_undegraded":"2023-10-06T05:24:25.022344+0000","last_fullsized":"2023-10-06T05:24:25.022344+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:15.549580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039626+0000","last_change":"2023-10-06T05:24:26.039626+0000","last_active":"2023-10-06T05:24:26.039626+0000","last_peered":"2023-10-06T05:24:26.039626+0000","last_clean":"2023-10-06T05:24:26.039626+0000","last_became_active":"2023-10-06T05:24:26.039339+0000","last_became_peered":"2023-10-06T05:24:26.039339+0000","last_unstale":"2023-10-06T05:24:26.039626+0000","last_undegraded":"2023-10-06T05:24:26.039626+0000","last_fullsized":"2023-10-06T05:24:26.039626+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:36.901929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.026397+0000","last_change":"2023-10-06T05:24:25.023175+0000","last_active":"2023-10-06T05:24:25.026397+0000","last_peered":"2023-10-06T05:24:25.026397+0000","last_clean":"2023-10-06T05:24:25.026397+0000","last_became_active":"2023-10-06T05:24:25.022581+0000","last_became_peered":"2023-10-06T05:24:25.022581+0000","last_unstale":"2023-10-06T05:24:25.026397+0000","last_undegraded":"2023-10-06T05:24:25.026397+0000","last_fullsized":"2023-10-06T05:24:25.026397+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:02.780349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024025+0000","last_change":"2023-10-06T05:24:26.024025+0000","last_active":"2023-10-06T05:24:26.024025+0000","last_peered":"2023-10-06T05:24:26.024025+0000","last_clean":"2023-10-06T05:24:26.024025+0000","last_became_active":"2023-10-06T05:24:26.023694+0000","last_became_peered":"2023-10-06T05:24:26.023694+0000","last_unstale":"2023-10-06T05:24:26.024025+0000","last_undegraded":"2023-10-06T05:24:26.024025+0000","last_fullsized":"2023-10-06T05:24:26.024025+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:03.789342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036867+0000","last_change":"2023-10-06T05:24:25.016833+0000","last_active":"2023-10-06T05:24:26.036867+0000","last_peered":"2023-10-06T05:24:26.036867+0000","last_clean":"2023-10-06T05:24:26.036867+0000","last_became_active":"2023-10-06T05:24:25.016189+0000","last_became_peered":"2023-10-06T05:24:25.016189+0000","last_unstale":"2023-10-06T05:24:26.036867+0000","last_undegraded":"2023-10-06T05:24:26.036867+0000","last_fullsized":"2023-10-06T05:24:26.036867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:36.192586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":114,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.006321+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:24:25.006321+0000","last_peered":"2023-10-06T05:24:25.006321+0000","last_clean":"2023-10-06T05:24:25.006321+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:24:25.006321+0000","last_undegraded":"2023-10-06T05:24:25.006321+0000","last_fullsized":"2023-10-06T05:24:25.006321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.023636+0000","last_change":"2023-10-06T05:24:26.023636+0000","last_active":"2023-10-06T05:24:26.023636+0000","last_peered":"2023-10-06T05:24:26.023636+0000","last_clean":"2023-10-06T05:24:26.023636+0000","last_became_active":"2023-10-06T05:24:26.023074+0000","last_became_peered":"2023-10-06T05:24:26.023074+0000","last_unstale":"2023-10-06T05:24:26.023636+0000","last_undegraded":"2023-10-06T05:24:26.023636+0000","last_fullsized":"2023-10-06T05:24:26.023636+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:48.627255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"143'3","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.034151+0000","last_change":"2023-10-06T05:24:25.011596+0000","last_active":"2023-10-06T05:24:26.034151+0000","last_peered":"2023-10-06T05:24:26.034151+0000","last_clean":"2023-10-06T05:24:26.034151+0000","last_became_active":"2023-10-06T05:24:25.011318+0000","last_became_peered":"2023-10-06T05:24:25.011318+0000","last_unstale":"2023-10-06T05:24:26.034151+0000","last_undegraded":"2023-10-06T05:24:26.034151+0000","last_fullsized":"2023-10-06T05:24:26.034151+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:57.397330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.012714+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:24:25.012714+0000","last_peered":"2023-10-06T05:24:25.012714+0000","last_clean":"2023-10-06T05:24:25.012714+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:24:25.012714+0000","last_undegraded":"2023-10-06T05:24:25.012714+0000","last_fullsized":"2023-10-06T05:24:25.012714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.022575+0000","last_change":"2023-10-06T05:24:26.022575+0000","last_active":"2023-10-06T05:24:26.022575+0000","last_peered":"2023-10-06T05:24:26.022575+0000","last_clean":"2023-10-06T05:24:26.022575+0000","last_became_active":"2023-10-06T05:24:26.022302+0000","last_became_peered":"2023-10-06T05:24:26.022302+0000","last_unstale":"2023-10-06T05:24:26.022575+0000","last_undegraded":"2023-10-06T05:24:26.022575+0000","last_fullsized":"2023-10-06T05:24:26.022575+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:50.013131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.027081+0000","last_change":"2023-10-06T05:24:25.023856+0000","last_active":"2023-10-06T05:24:25.027081+0000","last_peered":"2023-10-06T05:24:25.027081+0000","last_clean":"2023-10-06T05:24:25.027081+0000","last_became_active":"2023-10-06T05:24:25.023528+0000","last_became_peered":"2023-10-06T05:24:25.023528+0000","last_unstale":"2023-10-06T05:24:25.027081+0000","last_undegraded":"2023-10-06T05:24:25.027081+0000","last_fullsized":"2023-10-06T05:24:25.027081+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:43.540230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":110,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.005543+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:24:25.005543+0000","last_peered":"2023-10-06T05:24:25.005543+0000","last_clean":"2023-10-06T05:24:25.005543+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:24:25.005543+0000","last_undegraded":"2023-10-06T05:24:25.005543+0000","last_fullsized":"2023-10-06T05:24:25.005543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.032891+0000","last_change":"2023-10-06T05:24:26.032891+0000","last_active":"2023-10-06T05:24:26.032891+0000","last_peered":"2023-10-06T05:24:26.032891+0000","last_clean":"2023-10-06T05:24:26.032891+0000","last_became_active":"2023-10-06T05:24:26.032631+0000","last_became_peered":"2023-10-06T05:24:26.032631+0000","last_unstale":"2023-10-06T05:24:26.032891+0000","last_undegraded":"2023-10-06T05:24:26.032891+0000","last_fullsized":"2023-10-06T05:24:26.032891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:32.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"143'6","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.034232+0000","last_change":"2023-10-06T05:24:25.027090+0000","last_active":"2023-10-06T05:24:26.034232+0000","last_peered":"2023-10-06T05:24:26.034232+0000","last_clean":"2023-10-06T05:24:26.034232+0000","last_became_active":"2023-10-06T05:24:25.026814+0000","last_became_peered":"2023-10-06T05:24:25.026814+0000","last_unstale":"2023-10-06T05:24:26.034232+0000","last_undegraded":"2023-10-06T05:24:26.034232+0000","last_fullsized":"2023-10-06T05:24:26.034232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:34.147695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.016039+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:24:25.016039+0000","last_peered":"2023-10-06T05:24:25.016039+0000","last_clean":"2023-10-06T05:24:25.016039+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:24:25.016039+0000","last_undegraded":"2023-10-06T05:24:25.016039+0000","last_fullsized":"2023-10-06T05:24:25.016039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":264,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.018658+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:24:25.018658+0000","last_peered":"2023-10-06T05:24:25.018658+0000","last_clean":"2023-10-06T05:24:25.018658+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:24:25.018658+0000","last_undegraded":"2023-10-06T05:24:25.018658+0000","last_fullsized":"2023-10-06T05:24:25.018658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.023340+0000","last_change":"2023-10-06T05:24:26.023340+0000","last_active":"2023-10-06T05:24:26.023340+0000","last_peered":"2023-10-06T05:24:26.023340+0000","last_clean":"2023-10-06T05:24:26.023340+0000","last_became_active":"2023-10-06T05:24:26.023008+0000","last_became_peered":"2023-10-06T05:24:26.023008+0000","last_unstale":"2023-10-06T05:24:26.023340+0000","last_undegraded":"2023-10-06T05:24:26.023340+0000","last_fullsized":"2023-10-06T05:24:26.023340+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:12.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.026280+0000","last_change":"2023-10-06T05:24:25.022787+0000","last_active":"2023-10-06T05:24:25.026280+0000","last_peered":"2023-10-06T05:24:25.026280+0000","last_clean":"2023-10-06T05:24:25.026280+0000","last_became_active":"2023-10-06T05:24:25.022520+0000","last_became_peered":"2023-10-06T05:24:25.022520+0000","last_unstale":"2023-10-06T05:24:25.026280+0000","last_undegraded":"2023-10-06T05:24:25.026280+0000","last_fullsized":"2023-10-06T05:24:25.026280+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.017500+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:24:25.017500+0000","last_peered":"2023-10-06T05:24:25.017500+0000","last_clean":"2023-10-06T05:24:25.017500+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:24:25.017500+0000","last_undegraded":"2023-10-06T05:24:25.017500+0000","last_fullsized":"2023-10-06T05:24:25.017500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024618+0000","last_change":"2023-10-06T05:24:26.024618+0000","last_active":"2023-10-06T05:24:26.024618+0000","last_peered":"2023-10-06T05:24:26.024618+0000","last_clean":"2023-10-06T05:24:26.024618+0000","last_became_active":"2023-10-06T05:24:26.024324+0000","last_became_peered":"2023-10-06T05:24:26.024324+0000","last_unstale":"2023-10-06T05:24:26.024618+0000","last_undegraded":"2023-10-06T05:24:26.024618+0000","last_fullsized":"2023-10-06T05:24:26.024618+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:52.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.031339+0000","last_change":"2023-10-06T05:24:25.030188+0000","last_active":"2023-10-06T05:24:25.031339+0000","last_peered":"2023-10-06T05:24:25.031339+0000","last_clean":"2023-10-06T05:24:25.031339+0000","last_became_active":"2023-10-06T05:24:25.029889+0000","last_became_peered":"2023-10-06T05:24:25.029889+0000","last_unstale":"2023-10-06T05:24:25.031339+0000","last_undegraded":"2023-10-06T05:24:25.031339+0000","last_fullsized":"2023-10-06T05:24:25.031339+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:30.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.013220+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:24:25.013220+0000","last_peered":"2023-10-06T05:24:25.013220+0000","last_clean":"2023-10-06T05:24:25.013220+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:24:25.013220+0000","last_undegraded":"2023-10-06T05:24:25.013220+0000","last_fullsized":"2023-10-06T05:24:25.013220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028606+0000","last_change":"2023-10-06T05:24:26.028606+0000","last_active":"2023-10-06T05:24:26.028606+0000","last_peered":"2023-10-06T05:24:26.028606+0000","last_clean":"2023-10-06T05:24:26.028606+0000","last_became_active":"2023-10-06T05:24:26.028237+0000","last_became_peered":"2023-10-06T05:24:26.028237+0000","last_unstale":"2023-10-06T05:24:26.028606+0000","last_undegraded":"2023-10-06T05:24:26.028606+0000","last_fullsized":"2023-10-06T05:24:26.028606+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:21.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.019388+0000","last_change":"2023-10-06T05:24:25.016644+0000","last_active":"2023-10-06T05:24:25.019388+0000","last_peered":"2023-10-06T05:24:25.019388+0000","last_clean":"2023-10-06T05:24:25.019388+0000","last_became_active":"2023-10-06T05:24:25.016328+0000","last_became_peered":"2023-10-06T05:24:25.016328+0000","last_unstale":"2023-10-06T05:24:25.019388+0000","last_undegraded":"2023-10-06T05:24:25.019388+0000","last_fullsized":"2023-10-06T05:24:25.019388+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:00.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.004721+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:24:25.004721+0000","last_peered":"2023-10-06T05:24:25.004721+0000","last_clean":"2023-10-06T05:24:25.004721+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:24:25.004721+0000","last_undegraded":"2023-10-06T05:24:25.004721+0000","last_fullsized":"2023-10-06T05:24:25.004721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.040178+0000","last_change":"2023-10-06T05:24:26.040178+0000","last_active":"2023-10-06T05:24:26.040178+0000","last_peered":"2023-10-06T05:24:26.040178+0000","last_clean":"2023-10-06T05:24:26.040178+0000","last_became_active":"2023-10-06T05:24:26.039879+0000","last_became_peered":"2023-10-06T05:24:26.039879+0000","last_unstale":"2023-10-06T05:24:26.040178+0000","last_undegraded":"2023-10-06T05:24:26.040178+0000","last_fullsized":"2023-10-06T05:24:26.040178+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:04.330883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"143'8","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.042793+0000","last_change":"2023-10-06T05:24:25.027860+0000","last_active":"2023-10-06T05:24:26.042793+0000","last_peered":"2023-10-06T05:24:26.042793+0000","last_clean":"2023-10-06T05:24:26.042793+0000","last_became_active":"2023-10-06T05:24:25.027258+0000","last_became_peered":"2023-10-06T05:24:25.027258+0000","last_unstale":"2023-10-06T05:24:26.042793+0000","last_undegraded":"2023-10-06T05:24:26.042793+0000","last_fullsized":"2023-10-06T05:24:26.042793+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:04.868067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.014238+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:24:25.014238+0000","last_peered":"2023-10-06T05:24:25.014238+0000","last_clean":"2023-10-06T05:24:25.014238+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:24:25.014238+0000","last_undegraded":"2023-10-06T05:24:25.014238+0000","last_fullsized":"2023-10-06T05:24:25.014238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303716,"kb_used_data":2332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068124,"statfs":{"total":96636764160,"available":96325758976,"internally_reserved":0,"allocated":2387968,"data_stored":1416559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70099999999999996,"15min":0.65800000000000003},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.92600000000000005,"5min":1.169,"15min":1.169},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.64700000000000002},"min":{"1min":0.51300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.268,"15min":1.268},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.075,"15min":1.075},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":1.093,"15min":1.093},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.69699999999999995},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93400000000000005,"5min":1.22,"15min":1.22},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46600000000000003},"max":{"1min":0.96199999999999997,"5min":1.117,"15min":1.202},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.623},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":1.258,"15min":1.258},"last":0.378},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.73099999999999998},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69999999999999996,"15min":0.69199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.054,"15min":1.054},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.95599999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94499999999999995,"5min":1.139,"15min":1.139},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.74399999999999999},"min":{"1min":0.45100000000000001,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.24,"15min":1.3300000000000001},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303136,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068704,"statfs":{"total":96636764160,"available":96326352896,"internally_reserved":0,"allocated":1794048,"data_stored":826239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.52000000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":1.0209999999999999,"15min":1.034},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.70499999999999996},"min":{"1min":0.49399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.59099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69699999999999995,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.72499999999999998},"min":{"1min":0.54000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.755,"5min":1.05,"15min":1.1599999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.90600000000000003,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92000000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.78500000000000003},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.65900000000000003},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.95699999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89100000000000001,"5min":0.90500000000000003,"15min":0.998},"last":0.498}]},{"osd":5,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.71799999999999997},"min":{"1min":0.434,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.751},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93600000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.749}]},{"osd":7,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.754},"min":{"1min":0.501,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.81000000000000005},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.76200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303136,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068704,"statfs":{"total":96636764160,"available":96326352896,"internally_reserved":0,"allocated":1794048,"data_stored":826239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44900000000000001,"15min":0.502},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.54600000000000004},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":2.8050000000000002},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82799999999999996,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.63400000000000001},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.88,"5min":0.88,"15min":2.7639999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":2.8370000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75,"15min":0.78700000000000003},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.1080000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.0660000000000001,"5min":1.25,"15min":1.349},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":1.016,"15min":1.669},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81000000000000005,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.86399999999999999},"min":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.141,"5min":1.3280000000000001,"15min":1.573},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.528},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.85199999999999998},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":0.97299999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303700,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068140,"statfs":{"total":96636764160,"available":96325775360,"internally_reserved":0,"allocated":2371584,"data_stored":1409421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.61299999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.81499999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.502}]},{"osd":3,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67200000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84499999999999997,"5min":0.879,"15min":0.879},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.76300000000000001},"min":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.1120000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.125,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.59199999999999997,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.218,"15min":1.218},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.089,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.625},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.111,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.80600000000000005},"min":{"1min":0.61499999999999999,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0780000000000001,"5min":1.25,"15min":1.25},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":826281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":1.03,"5min":1.056,"15min":1.056},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.30599999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.94399999999999995,"5min":1.073,"15min":1.073},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.626}]},{"osd":4,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.109,"5min":1.109,"15min":1.1379999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.59799999999999998,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.86399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55800000000000005,"15min":0.53000000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.319},"last":0.88}]},{"osd":6,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77800000000000002,"15min":0.81200000000000006},"min":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.175,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.141},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.748},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.3520000000000001},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":826757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46100000000000002,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.64500000000000002,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.495,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.67500000000000004,"5min":0.77100000000000002,"15min":1.772},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.60099999999999998,"5min":0.79400000000000004,"15min":1.913},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.954},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.60899999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.044,"5min":1.0640000000000001,"15min":1.282},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91100000000000003,"5min":1.216,"15min":1.2609999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.028,"5min":1.145,"15min":1.1930000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.628}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79100000000000004,"15min":0.83299999999999996},"min":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1.0880000000000001,"15min":1.343},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":0.98199999999999998,"5min":1.1779999999999999,"15min":1.3380000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.3640000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":1.085,"5min":1.1539999999999999,"15min":1.214},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303144,"kb_used_data":1760,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068696,"statfs":{"total":96636764160,"available":96326344704,"internally_reserved":0,"allocated":1802240,"data_stored":827705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72799999999999998,"15min":0.71199999999999997},"min":{"1min":0.57499999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98699999999999999,"5min":1.0249999999999999,"15min":1.073},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.169},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.47199999999999998,"15min":0.432},"max":{"1min":0.93000000000000005,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69999999999999996,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":0.91300000000000003,"5min":1.016,"15min":1.117},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57999999999999996,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":0.96999999999999997,"15min":1.002},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.55300000000000005,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":0.98799999999999999,"15min":1.0780000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.196},"last":0.73399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.433},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.375},"last":0.879}]},{"osd":6,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":1.004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.50900000000000001,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.996,"15min":1.466},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79400000000000004,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98699999999999999,"5min":1.018,"15min":1.218},"last":0.81299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303132,"kb_used_data":1748,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068708,"statfs":{"total":96636764160,"available":96326356992,"internally_reserved":0,"allocated":1789952,"data_stored":826169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.67900000000000005},"min":{"1min":0.57699999999999996,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":1.165,"15min":1.165},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":1.1890000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83199999999999996,"5min":0.92800000000000005,"15min":0.98299999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.048,"15min":1.048},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.86899999999999999,"5min":1.204,"15min":1.204},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.495},"max":{"1min":0.79600000000000004,"5min":0.997,"15min":1.0549999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.71499999999999997},"min":{"1min":0.55900000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92700000000000005,"5min":1.002,"15min":1.087},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.495,"15min":0.433},"max":{"1min":0.90000000000000002,"5min":1.2769999999999999,"15min":1.278},"last":0.72499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.65700000000000003},"min":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.90000000000000002,"5min":1.1539999999999999,"15min":1.294},"last":0.748}]},{"osd":6,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":1.141,"15min":1.141},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0429999999999999,"15min":1.254},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.337},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.74299999999999999,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.248,"15min":1.248},"last":0.81200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:24:30.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:24:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.607+0000 7f4a30d9d700 1 Processor -- start 2023-10-06T05:24:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.607+0000 7f4a30d9d700 1 -- start start 2023-10-06T05:24:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a30d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a2c0ccfc0 0x7f4a2c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a30d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 0x7f4a2c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a30d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2c0cbd80 con 0x7f4a2c0cc750 2023-10-06T05:24:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a30d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2c0af5f0 con 0x7f4a2c0ccfc0 2023-10-06T05:24:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a30d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0af8d0 con 0x7f4a2c0cf3b0 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a2affd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 0x7f4a2c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a2b7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a2c0ccfc0 0x7f4a2c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a2affd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 0x7f4a2c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:47234/0 (socket says 172.21.15.33:47234) 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a2affd700 1 -- 172.21.15.33:0/1017915719 learned_addr learned my addr 172.21.15.33:0/1017915719 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.609+0000 7f4a2bfff700 1 --1- 172.21.15.33:0/1017915719 >> v1:172.21.15.167:6789/0 conn(0x7f4a2c0cc750 0x7f4a2c0b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:44376/0 (socket says 172.21.15.33:44376) 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2affd700 1 -- 172.21.15.33:0/1017915719 >> v2:172.21.15.167:3300/0 conn(0x7f4a2c0ccfc0 msgr2=0x7f4a2c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2affd700 1 --2- 172.21.15.33:0/1017915719 >> v2:172.21.15.167:3300/0 conn(0x7f4a2c0ccfc0 0x7f4a2c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2affd700 1 -- 172.21.15.33:0/1017915719 >> v1:172.21.15.167:6789/0 conn(0x7f4a2c0cc750 legacy=0x7f4a2c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2affd700 1 -- 172.21.15.33:0/1017915719 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0afbb0 con 0x7f4a2c0cf3b0 2023-10-06T05:24:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2affd700 1 --2- 172.21.15.33:0/1017915719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 0x7f4a2c1594a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4a200099b0 tx=0x7f4a20009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8543346fe9a995 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2a7fc700 1 -- 172.21.15.33:0/1017915719 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20009ea0 con 0x7f4a2c0cf3b0 2023-10-06T05:24:30.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2a7fc700 1 -- 172.21.15.33:0/1017915719 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000b860 con 0x7f4a2c0cf3b0 2023-10-06T05:24:30.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.610+0000 7f4a2a7fc700 1 -- 172.21.15.33:0/1017915719 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000bc00 con 0x7f4a2c0cf3b0 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 -- 172.21.15.33:0/1017915719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 msgr2=0x7f4a2c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/1017915719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 0x7f4a2c1594a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4a200099b0 tx=0x7f4a20009250 comp rx=0 tx=0).stop 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 -- 172.21.15.33:0/1017915719 shutdown_connections 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/1017915719 >> v2:172.21.15.167:3300/0 conn(0x7f4a2c0ccfc0 0x7f4a2c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/1017915719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0cf3b0 0x7f4a2c1594a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 -- 172.21.15.33:0/1017915719 >> 172.21.15.33:0/1017915719 conn(0x7f4a2c06cff0 msgr2=0x7f4a2c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 -- 172.21.15.33:0/1017915719 shutdown_connections 2023-10-06T05:24:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.611+0000 7f4a30d9d700 1 -- 172.21.15.33:0/1017915719 wait complete. 2023-10-06T05:24:30.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.612+0000 7f4a30d9d700 1 Processor -- start 2023-10-06T05:24:30.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 -- start start 2023-10-06T05:24:30.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 0x7f4a2c0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0ccfc0 0x7f4a2c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c0b32a0 0x7f4a2c142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0c7f80 con 0x7f4a2c0cc750 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c14fb00 con 0x7f4a2c0b32a0 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.613+0000 7f4a30d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0c9070 con 0x7f4a2c0ccfc0 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 0x7f4a2c0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 0x7f4a2c0c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35454/0 (socket says 172.21.15.33:35454) 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 -- 172.21.15.33:0/434606796 learned_addr learned my addr 172.21.15.33:0/434606796 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2bfff700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c0b32a0 0x7f4a2c142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2affd700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0ccfc0 0x7f4a2c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 -- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0ccfc0 msgr2=0x7f4a2c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0ccfc0 0x7f4a2c0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 -- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c0b32a0 msgr2=0x7f4a2c142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c0b32a0 0x7f4a2c142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 -- 172.21.15.33:0/434606796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200094f0 con 0x7f4a2c0cc750 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.614+0000 7f4a2b7fe700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 0x7f4a2c0c8990 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00ec80 tx=0x7f4a1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.615+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00c9d0 con 0x7f4a2c0cc750 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.615+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c00cb30 con 0x7f4a2c0cc750 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.615+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2c0c7650 con 0x7f4a2c0cc750 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.615+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c013470 con 0x7f4a2c0cc750 2023-10-06T05:24:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.615+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2c0c7c10 con 0x7f4a2c0cc750 2023-10-06T05:24:30.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.616+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c04c380 con 0x7f4a2c0cc750 2023-10-06T05:24:30.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.617+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a1c013820 con 0x7f4a2c0cc750 2023-10-06T05:24:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.617+0000 7f4a137fe700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.618+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4a1c084b40 con 0x7f4a2c0cc750 2023-10-06T05:24:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.618+0000 7f4a2affd700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.619+0000 7f4a2affd700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a18066f30 0x7f4a180693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b0250 tx=0x7f4a20017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.622+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a1c051bb0 con 0x7f4a2c0cc750 2023-10-06T05:24:30.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.819+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a2c0c80c0 con 0x7f4a18066f30 2023-10-06T05:24:30.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.828+0000 7f4a137fe700 1 -- 172.21.15.33:0/434606796 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311293 (secure 0 0 0) 0x7f4a2000b9d0 con 0x7f4a18066f30 2023-10-06T05:24:30.834 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:30.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.832+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a18066f30 msgr2=0x7f4a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.832+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a18066f30 0x7f4a180693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b0250 tx=0x7f4a20017000 comp rx=0 tx=0).stop 2023-10-06T05:24:30.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.832+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 msgr2=0x7f4a2c0c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 0x7f4a2c0c8990 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00ec80 tx=0x7f4a1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 shutdown_connections 2023-10-06T05:24:30.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c0b32a0 0x7f4a2c142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0cc750 0x7f4a2c0c8990 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 --2- 172.21.15.33:0/434606796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c0ccfc0 0x7f4a2c0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 >> 172.21.15.33:0/434606796 conn(0x7f4a2c06cff0 msgr2=0x7f4a2c0cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 shutdown_connections 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:30.833+0000 7f4a30d9d700 1 -- 172.21.15.33:0/434606796 wait complete. 2023-10-06T05:24:30.838 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:24:30.852 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2023-10-06T05:24:29.967211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426244,"kb_used_data":15172,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548476,"statfs":{"total":773094113280,"available":770609639424,"internally_reserved":0,"allocated":15536128,"data_stored":7785370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.974683"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.030065+0000","last_change":"2023-10-06T05:24:26.030065+0000","last_active":"2023-10-06T05:24:26.030065+0000","last_peered":"2023-10-06T05:24:26.030065+0000","last_clean":"2023-10-06T05:24:26.030065+0000","last_became_active":"2023-10-06T05:24:26.029754+0000","last_became_peered":"2023-10-06T05:24:26.029754+0000","last_unstale":"2023-10-06T05:24:26.030065+0000","last_undegraded":"2023-10-06T05:24:26.030065+0000","last_fullsized":"2023-10-06T05:24:26.030065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:23.647324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.019316+0000","last_change":"2023-10-06T05:24:26.019316+0000","last_active":"2023-10-06T05:24:26.019316+0000","last_peered":"2023-10-06T05:24:26.019316+0000","last_clean":"2023-10-06T05:24:26.019316+0000","last_became_active":"2023-10-06T05:24:26.019131+0000","last_became_peered":"2023-10-06T05:24:26.019131+0000","last_unstale":"2023-10-06T05:24:26.019316+0000","last_undegraded":"2023-10-06T05:24:26.019316+0000","last_fullsized":"2023-10-06T05:24:26.019316+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:33.272368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029308+0000","last_change":"2023-10-06T05:24:26.029308+0000","last_active":"2023-10-06T05:24:26.029308+0000","last_peered":"2023-10-06T05:24:26.029308+0000","last_clean":"2023-10-06T05:24:26.029308+0000","last_became_active":"2023-10-06T05:24:26.028982+0000","last_became_peered":"2023-10-06T05:24:26.028982+0000","last_unstale":"2023-10-06T05:24:26.029308+0000","last_undegraded":"2023-10-06T05:24:26.029308+0000","last_fullsized":"2023-10-06T05:24:26.029308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:12.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.020981+0000","last_change":"2023-10-06T05:24:26.020981+0000","last_active":"2023-10-06T05:24:26.020981+0000","last_peered":"2023-10-06T05:24:26.020981+0000","last_clean":"2023-10-06T05:24:26.020981+0000","last_became_active":"2023-10-06T05:24:26.020673+0000","last_became_peered":"2023-10-06T05:24:26.020673+0000","last_unstale":"2023-10-06T05:24:26.020981+0000","last_undegraded":"2023-10-06T05:24:26.020981+0000","last_fullsized":"2023-10-06T05:24:26.020981+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:54.205522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036648+0000","last_change":"2023-10-06T05:24:26.036648+0000","last_active":"2023-10-06T05:24:26.036648+0000","last_peered":"2023-10-06T05:24:26.036648+0000","last_clean":"2023-10-06T05:24:26.036648+0000","last_became_active":"2023-10-06T05:24:26.036171+0000","last_became_peered":"2023-10-06T05:24:26.036171+0000","last_unstale":"2023-10-06T05:24:26.036648+0000","last_undegraded":"2023-10-06T05:24:26.036648+0000","last_fullsized":"2023-10-06T05:24:26.036648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:52.160651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039175+0000","last_change":"2023-10-06T05:24:26.039175+0000","last_active":"2023-10-06T05:24:26.039175+0000","last_peered":"2023-10-06T05:24:26.039175+0000","last_clean":"2023-10-06T05:24:26.039175+0000","last_became_active":"2023-10-06T05:24:26.038917+0000","last_became_peered":"2023-10-06T05:24:26.038917+0000","last_unstale":"2023-10-06T05:24:26.039175+0000","last_undegraded":"2023-10-06T05:24:26.039175+0000","last_fullsized":"2023-10-06T05:24:26.039175+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:45.824772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.030820+0000","last_change":"2023-10-06T05:24:26.030820+0000","last_active":"2023-10-06T05:24:26.030820+0000","last_peered":"2023-10-06T05:24:26.030820+0000","last_clean":"2023-10-06T05:24:26.030820+0000","last_became_active":"2023-10-06T05:24:26.030516+0000","last_became_peered":"2023-10-06T05:24:26.030516+0000","last_unstale":"2023-10-06T05:24:26.030820+0000","last_undegraded":"2023-10-06T05:24:26.030820+0000","last_fullsized":"2023-10-06T05:24:26.030820+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:40.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025104+0000","last_change":"2023-10-06T05:24:26.025104+0000","last_active":"2023-10-06T05:24:26.025104+0000","last_peered":"2023-10-06T05:24:26.025104+0000","last_clean":"2023-10-06T05:24:26.025104+0000","last_became_active":"2023-10-06T05:24:26.024810+0000","last_became_peered":"2023-10-06T05:24:26.024810+0000","last_unstale":"2023-10-06T05:24:26.025104+0000","last_undegraded":"2023-10-06T05:24:26.025104+0000","last_fullsized":"2023-10-06T05:24:26.025104+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:42.251029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033411+0000","last_change":"2023-10-06T05:24:26.033411+0000","last_active":"2023-10-06T05:24:26.033411+0000","last_peered":"2023-10-06T05:24:26.033411+0000","last_clean":"2023-10-06T05:24:26.033411+0000","last_became_active":"2023-10-06T05:24:26.030165+0000","last_became_peered":"2023-10-06T05:24:26.030165+0000","last_unstale":"2023-10-06T05:24:26.033411+0000","last_undegraded":"2023-10-06T05:24:26.033411+0000","last_fullsized":"2023-10-06T05:24:26.033411+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:54.852065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033969+0000","last_change":"2023-10-06T05:24:26.033969+0000","last_active":"2023-10-06T05:24:26.033969+0000","last_peered":"2023-10-06T05:24:26.033969+0000","last_clean":"2023-10-06T05:24:26.033969+0000","last_became_active":"2023-10-06T05:24:26.033221+0000","last_became_peered":"2023-10-06T05:24:26.033221+0000","last_unstale":"2023-10-06T05:24:26.033969+0000","last_undegraded":"2023-10-06T05:24:26.033969+0000","last_fullsized":"2023-10-06T05:24:26.033969+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:05.486519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.023631+0000","last_change":"2023-10-06T05:24:26.023631+0000","last_active":"2023-10-06T05:24:26.023631+0000","last_peered":"2023-10-06T05:24:26.023631+0000","last_clean":"2023-10-06T05:24:26.023631+0000","last_became_active":"2023-10-06T05:24:26.023188+0000","last_became_peered":"2023-10-06T05:24:26.023188+0000","last_unstale":"2023-10-06T05:24:26.023631+0000","last_undegraded":"2023-10-06T05:24:26.023631+0000","last_fullsized":"2023-10-06T05:24:26.023631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:48.339847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.040460+0000","last_change":"2023-10-06T05:24:26.040460+0000","last_active":"2023-10-06T05:24:26.040460+0000","last_peered":"2023-10-06T05:24:26.040460+0000","last_clean":"2023-10-06T05:24:26.040460+0000","last_became_active":"2023-10-06T05:24:26.040210+0000","last_became_peered":"2023-10-06T05:24:26.040210+0000","last_unstale":"2023-10-06T05:24:26.040460+0000","last_undegraded":"2023-10-06T05:24:26.040460+0000","last_fullsized":"2023-10-06T05:24:26.040460+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:11.434671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.037292+0000","last_change":"2023-10-06T05:24:26.037292+0000","last_active":"2023-10-06T05:24:26.037292+0000","last_peered":"2023-10-06T05:24:26.037292+0000","last_clean":"2023-10-06T05:24:26.037292+0000","last_became_active":"2023-10-06T05:24:26.036377+0000","last_became_peered":"2023-10-06T05:24:26.036377+0000","last_unstale":"2023-10-06T05:24:26.037292+0000","last_undegraded":"2023-10-06T05:24:26.037292+0000","last_fullsized":"2023-10-06T05:24:26.037292+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:00.692340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.035166+0000","last_change":"2023-10-06T05:24:26.035166+0000","last_active":"2023-10-06T05:24:26.035166+0000","last_peered":"2023-10-06T05:24:26.035166+0000","last_clean":"2023-10-06T05:24:26.035166+0000","last_became_active":"2023-10-06T05:24:26.034892+0000","last_became_peered":"2023-10-06T05:24:26.034892+0000","last_unstale":"2023-10-06T05:24:26.035166+0000","last_undegraded":"2023-10-06T05:24:26.035166+0000","last_fullsized":"2023-10-06T05:24:26.035166+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:48.744050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025545+0000","last_change":"2023-10-06T05:24:26.025545+0000","last_active":"2023-10-06T05:24:26.025545+0000","last_peered":"2023-10-06T05:24:26.025545+0000","last_clean":"2023-10-06T05:24:26.025545+0000","last_became_active":"2023-10-06T05:24:26.014072+0000","last_became_peered":"2023-10-06T05:24:26.014072+0000","last_unstale":"2023-10-06T05:24:26.025545+0000","last_undegraded":"2023-10-06T05:24:26.025545+0000","last_fullsized":"2023-10-06T05:24:26.025545+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:42.251029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.034588+0000","last_change":"2023-10-06T05:24:26.034588+0000","last_active":"2023-10-06T05:24:26.034588+0000","last_peered":"2023-10-06T05:24:26.034588+0000","last_clean":"2023-10-06T05:24:26.034588+0000","last_became_active":"2023-10-06T05:24:26.034286+0000","last_became_peered":"2023-10-06T05:24:26.034286+0000","last_unstale":"2023-10-06T05:24:26.034588+0000","last_undegraded":"2023-10-06T05:24:26.034588+0000","last_fullsized":"2023-10-06T05:24:26.034588+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:12.738848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.015833+0000","last_change":"2023-10-06T05:24:26.015833+0000","last_active":"2023-10-06T05:24:26.015833+0000","last_peered":"2023-10-06T05:24:26.015833+0000","last_clean":"2023-10-06T05:24:26.015833+0000","last_became_active":"2023-10-06T05:24:26.015568+0000","last_became_peered":"2023-10-06T05:24:26.015568+0000","last_unstale":"2023-10-06T05:24:26.015833+0000","last_undegraded":"2023-10-06T05:24:26.015833+0000","last_fullsized":"2023-10-06T05:24:26.015833+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:04.408452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024224+0000","last_change":"2023-10-06T05:24:26.024224+0000","last_active":"2023-10-06T05:24:26.024224+0000","last_peered":"2023-10-06T05:24:26.024224+0000","last_clean":"2023-10-06T05:24:26.024224+0000","last_became_active":"2023-10-06T05:24:26.023934+0000","last_became_peered":"2023-10-06T05:24:26.023934+0000","last_unstale":"2023-10-06T05:24:26.024224+0000","last_undegraded":"2023-10-06T05:24:26.024224+0000","last_fullsized":"2023-10-06T05:24:26.024224+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:56.282142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029407+0000","last_change":"2023-10-06T05:24:26.029407+0000","last_active":"2023-10-06T05:24:26.029407+0000","last_peered":"2023-10-06T05:24:26.029407+0000","last_clean":"2023-10-06T05:24:26.029407+0000","last_became_active":"2023-10-06T05:24:26.029023+0000","last_became_peered":"2023-10-06T05:24:26.029023+0000","last_unstale":"2023-10-06T05:24:26.029407+0000","last_undegraded":"2023-10-06T05:24:26.029407+0000","last_fullsized":"2023-10-06T05:24:26.029407+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:11.684203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.020916+0000","last_change":"2023-10-06T05:24:26.020916+0000","last_active":"2023-10-06T05:24:26.020916+0000","last_peered":"2023-10-06T05:24:26.020916+0000","last_clean":"2023-10-06T05:24:26.020916+0000","last_became_active":"2023-10-06T05:24:26.020641+0000","last_became_peered":"2023-10-06T05:24:26.020641+0000","last_unstale":"2023-10-06T05:24:26.020916+0000","last_undegraded":"2023-10-06T05:24:26.020916+0000","last_fullsized":"2023-10-06T05:24:26.020916+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:46.027779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.037535+0000","last_change":"2023-10-06T05:24:26.037535+0000","last_active":"2023-10-06T05:24:26.037535+0000","last_peered":"2023-10-06T05:24:26.037535+0000","last_clean":"2023-10-06T05:24:26.037535+0000","last_became_active":"2023-10-06T05:24:26.036898+0000","last_became_peered":"2023-10-06T05:24:26.036898+0000","last_unstale":"2023-10-06T05:24:26.037535+0000","last_undegraded":"2023-10-06T05:24:26.037535+0000","last_fullsized":"2023-10-06T05:24:26.037535+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:47.028934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033254+0000","last_change":"2023-10-06T05:24:26.033254+0000","last_active":"2023-10-06T05:24:26.033254+0000","last_peered":"2023-10-06T05:24:26.033254+0000","last_clean":"2023-10-06T05:24:26.033254+0000","last_became_active":"2023-10-06T05:24:26.031191+0000","last_became_peered":"2023-10-06T05:24:26.031191+0000","last_unstale":"2023-10-06T05:24:26.033254+0000","last_undegraded":"2023-10-06T05:24:26.033254+0000","last_fullsized":"2023-10-06T05:24:26.033254+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:07.984962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024977+0000","last_change":"2023-10-06T05:24:26.024977+0000","last_active":"2023-10-06T05:24:26.024977+0000","last_peered":"2023-10-06T05:24:26.024977+0000","last_clean":"2023-10-06T05:24:26.024977+0000","last_became_active":"2023-10-06T05:24:26.024676+0000","last_became_peered":"2023-10-06T05:24:26.024676+0000","last_unstale":"2023-10-06T05:24:26.024977+0000","last_undegraded":"2023-10-06T05:24:26.024977+0000","last_fullsized":"2023-10-06T05:24:26.024977+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:33.272368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.032208+0000","last_change":"2023-10-06T05:24:26.032208+0000","last_active":"2023-10-06T05:24:26.032208+0000","last_peered":"2023-10-06T05:24:26.032208+0000","last_clean":"2023-10-06T05:24:26.032208+0000","last_became_active":"2023-10-06T05:24:26.031940+0000","last_became_peered":"2023-10-06T05:24:26.031940+0000","last_unstale":"2023-10-06T05:24:26.032208+0000","last_undegraded":"2023-10-06T05:24:26.032208+0000","last_fullsized":"2023-10-06T05:24:26.032208+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:27.115511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.026801+0000","last_change":"2023-10-06T05:24:26.026801+0000","last_active":"2023-10-06T05:24:26.026801+0000","last_peered":"2023-10-06T05:24:26.026801+0000","last_clean":"2023-10-06T05:24:26.026801+0000","last_became_active":"2023-10-06T05:24:26.026488+0000","last_became_peered":"2023-10-06T05:24:26.026488+0000","last_unstale":"2023-10-06T05:24:26.026801+0000","last_undegraded":"2023-10-06T05:24:26.026801+0000","last_fullsized":"2023-10-06T05:24:26.026801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:46.521669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025065+0000","last_change":"2023-10-06T05:24:26.025065+0000","last_active":"2023-10-06T05:24:26.025065+0000","last_peered":"2023-10-06T05:24:26.025065+0000","last_clean":"2023-10-06T05:24:26.025065+0000","last_became_active":"2023-10-06T05:24:26.024792+0000","last_became_peered":"2023-10-06T05:24:26.024792+0000","last_unstale":"2023-10-06T05:24:26.025065+0000","last_undegraded":"2023-10-06T05:24:26.025065+0000","last_fullsized":"2023-10-06T05:24:26.025065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:07.102122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036635+0000","last_change":"2023-10-06T05:24:26.036635+0000","last_active":"2023-10-06T05:24:26.036635+0000","last_peered":"2023-10-06T05:24:26.036635+0000","last_clean":"2023-10-06T05:24:26.036635+0000","last_became_active":"2023-10-06T05:24:26.036298+0000","last_became_peered":"2023-10-06T05:24:26.036298+0000","last_unstale":"2023-10-06T05:24:26.036635+0000","last_undegraded":"2023-10-06T05:24:26.036635+0000","last_fullsized":"2023-10-06T05:24:26.036635+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:51.249505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028856+0000","last_change":"2023-10-06T05:24:26.028856+0000","last_active":"2023-10-06T05:24:26.028856+0000","last_peered":"2023-10-06T05:24:26.028856+0000","last_clean":"2023-10-06T05:24:26.028856+0000","last_became_active":"2023-10-06T05:24:26.026938+0000","last_became_peered":"2023-10-06T05:24:26.026938+0000","last_unstale":"2023-10-06T05:24:26.028856+0000","last_undegraded":"2023-10-06T05:24:26.028856+0000","last_fullsized":"2023-10-06T05:24:26.028856+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:19.168120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039686+0000","last_change":"2023-10-06T05:24:26.039686+0000","last_active":"2023-10-06T05:24:26.039686+0000","last_peered":"2023-10-06T05:24:26.039686+0000","last_clean":"2023-10-06T05:24:26.039686+0000","last_became_active":"2023-10-06T05:24:26.039411+0000","last_became_peered":"2023-10-06T05:24:26.039411+0000","last_unstale":"2023-10-06T05:24:26.039686+0000","last_undegraded":"2023-10-06T05:24:26.039686+0000","last_fullsized":"2023-10-06T05:24:26.039686+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:06.625908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.019299+0000","last_change":"2023-10-06T05:24:26.019299+0000","last_active":"2023-10-06T05:24:26.019299+0000","last_peered":"2023-10-06T05:24:26.019299+0000","last_clean":"2023-10-06T05:24:26.019299+0000","last_became_active":"2023-10-06T05:24:26.019033+0000","last_became_peered":"2023-10-06T05:24:26.019033+0000","last_unstale":"2023-10-06T05:24:26.019299+0000","last_undegraded":"2023-10-06T05:24:26.019299+0000","last_fullsized":"2023-10-06T05:24:26.019299+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:27.986222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.035236+0000","last_change":"2023-10-06T05:24:26.035236+0000","last_active":"2023-10-06T05:24:26.035236+0000","last_peered":"2023-10-06T05:24:26.035236+0000","last_clean":"2023-10-06T05:24:26.035236+0000","last_became_active":"2023-10-06T05:24:26.034977+0000","last_became_peered":"2023-10-06T05:24:26.034977+0000","last_unstale":"2023-10-06T05:24:26.035236+0000","last_undegraded":"2023-10-06T05:24:26.035236+0000","last_fullsized":"2023-10-06T05:24:26.035236+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:53.238696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025712+0000","last_change":"2023-10-06T05:24:26.025712+0000","last_active":"2023-10-06T05:24:26.025712+0000","last_peered":"2023-10-06T05:24:26.025712+0000","last_clean":"2023-10-06T05:24:26.025712+0000","last_became_active":"2023-10-06T05:24:26.025453+0000","last_became_peered":"2023-10-06T05:24:26.025453+0000","last_unstale":"2023-10-06T05:24:26.025712+0000","last_undegraded":"2023-10-06T05:24:26.025712+0000","last_fullsized":"2023-10-06T05:24:26.025712+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:52.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025531+0000","last_change":"2023-10-06T05:24:26.025531+0000","last_active":"2023-10-06T05:24:26.025531+0000","last_peered":"2023-10-06T05:24:26.025531+0000","last_clean":"2023-10-06T05:24:26.025531+0000","last_became_active":"2023-10-06T05:24:26.025246+0000","last_became_peered":"2023-10-06T05:24:26.025246+0000","last_unstale":"2023-10-06T05:24:26.025531+0000","last_undegraded":"2023-10-06T05:24:26.025531+0000","last_fullsized":"2023-10-06T05:24:26.025531+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:06.625908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"143'4","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.042143+0000","last_change":"2023-10-06T05:24:25.013400+0000","last_active":"2023-10-06T05:24:26.042143+0000","last_peered":"2023-10-06T05:24:26.042143+0000","last_clean":"2023-10-06T05:24:26.042143+0000","last_became_active":"2023-10-06T05:24:25.013121+0000","last_became_peered":"2023-10-06T05:24:25.013121+0000","last_unstale":"2023-10-06T05:24:26.042143+0000","last_undegraded":"2023-10-06T05:24:26.042143+0000","last_fullsized":"2023-10-06T05:24:26.042143+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:45.512677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.018801+0000","last_change":"2023-10-06T05:24:26.018801+0000","last_active":"2023-10-06T05:24:26.018801+0000","last_peered":"2023-10-06T05:24:26.018801+0000","last_clean":"2023-10-06T05:24:26.018801+0000","last_became_active":"2023-10-06T05:24:26.018487+0000","last_became_peered":"2023-10-06T05:24:26.018487+0000","last_unstale":"2023-10-06T05:24:26.018801+0000","last_undegraded":"2023-10-06T05:24:26.018801+0000","last_fullsized":"2023-10-06T05:24:26.018801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:35.985599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.013277+0000","last_change":"2023-10-06T05:24:25.024469+0000","last_active":"2023-10-06T05:24:26.013277+0000","last_peered":"2023-10-06T05:24:26.013277+0000","last_clean":"2023-10-06T05:24:26.013277+0000","last_became_active":"2023-10-06T05:24:25.024125+0000","last_became_peered":"2023-10-06T05:24:25.024125+0000","last_unstale":"2023-10-06T05:24:26.013277+0000","last_undegraded":"2023-10-06T05:24:26.013277+0000","last_fullsized":"2023-10-06T05:24:26.013277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:34.976606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.014014+0000","last_change":"2023-10-06T05:24:26.014014+0000","last_active":"2023-10-06T05:24:26.014014+0000","last_peered":"2023-10-06T05:24:26.014014+0000","last_clean":"2023-10-06T05:24:26.014014+0000","last_became_active":"2023-10-06T05:24:26.013723+0000","last_became_peered":"2023-10-06T05:24:26.013723+0000","last_unstale":"2023-10-06T05:24:26.014014+0000","last_undegraded":"2023-10-06T05:24:26.014014+0000","last_fullsized":"2023-10-06T05:24:26.014014+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:35.985599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039517+0000","last_change":"2023-10-06T05:24:25.024540+0000","last_active":"2023-10-06T05:24:26.039517+0000","last_peered":"2023-10-06T05:24:26.039517+0000","last_clean":"2023-10-06T05:24:26.039517+0000","last_became_active":"2023-10-06T05:24:25.024219+0000","last_became_peered":"2023-10-06T05:24:25.024219+0000","last_unstale":"2023-10-06T05:24:26.039517+0000","last_undegraded":"2023-10-06T05:24:26.039517+0000","last_fullsized":"2023-10-06T05:24:26.039517+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:51.375221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.031842+0000","last_change":"2023-10-06T05:24:26.031842+0000","last_active":"2023-10-06T05:24:26.031842+0000","last_peered":"2023-10-06T05:24:26.031842+0000","last_clean":"2023-10-06T05:24:26.031842+0000","last_became_active":"2023-10-06T05:24:26.031563+0000","last_became_peered":"2023-10-06T05:24:26.031563+0000","last_unstale":"2023-10-06T05:24:26.031842+0000","last_undegraded":"2023-10-06T05:24:26.031842+0000","last_fullsized":"2023-10-06T05:24:26.031842+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:52.160651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029336+0000","last_change":"2023-10-06T05:24:25.019593+0000","last_active":"2023-10-06T05:24:26.029336+0000","last_peered":"2023-10-06T05:24:26.029336+0000","last_clean":"2023-10-06T05:24:26.029336+0000","last_became_active":"2023-10-06T05:24:25.019304+0000","last_became_peered":"2023-10-06T05:24:25.019304+0000","last_unstale":"2023-10-06T05:24:26.029336+0000","last_undegraded":"2023-10-06T05:24:26.029336+0000","last_fullsized":"2023-10-06T05:24:26.029336+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:39.487652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.020429+0000","last_change":"2023-10-06T05:24:26.020429+0000","last_active":"2023-10-06T05:24:26.020429+0000","last_peered":"2023-10-06T05:24:26.020429+0000","last_clean":"2023-10-06T05:24:26.020429+0000","last_became_active":"2023-10-06T05:24:26.020094+0000","last_became_peered":"2023-10-06T05:24:26.020094+0000","last_unstale":"2023-10-06T05:24:26.020429+0000","last_undegraded":"2023-10-06T05:24:26.020429+0000","last_fullsized":"2023-10-06T05:24:26.020429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:50.880104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.008587+0000","last_change":"2023-10-06T05:24:25.021114+0000","last_active":"2023-10-06T05:24:26.008587+0000","last_peered":"2023-10-06T05:24:26.008587+0000","last_clean":"2023-10-06T05:24:26.008587+0000","last_became_active":"2023-10-06T05:24:25.020763+0000","last_became_peered":"2023-10-06T05:24:25.020763+0000","last_unstale":"2023-10-06T05:24:26.008587+0000","last_undegraded":"2023-10-06T05:24:26.008587+0000","last_fullsized":"2023-10-06T05:24:26.008587+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:44.414631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029187+0000","last_change":"2023-10-06T05:24:26.029187+0000","last_active":"2023-10-06T05:24:26.029187+0000","last_peered":"2023-10-06T05:24:26.029187+0000","last_clean":"2023-10-06T05:24:26.029187+0000","last_became_active":"2023-10-06T05:24:26.028726+0000","last_became_peered":"2023-10-06T05:24:26.028726+0000","last_unstale":"2023-10-06T05:24:26.029187+0000","last_undegraded":"2023-10-06T05:24:26.029187+0000","last_fullsized":"2023-10-06T05:24:26.029187+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:57.559931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.022041+0000","last_change":"2023-10-06T05:24:25.018287+0000","last_active":"2023-10-06T05:24:26.022041+0000","last_peered":"2023-10-06T05:24:26.022041+0000","last_clean":"2023-10-06T05:24:26.022041+0000","last_became_active":"2023-10-06T05:24:25.018010+0000","last_became_peered":"2023-10-06T05:24:25.018010+0000","last_unstale":"2023-10-06T05:24:26.022041+0000","last_undegraded":"2023-10-06T05:24:26.022041+0000","last_fullsized":"2023-10-06T05:24:26.022041+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:58.475188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025051+0000","last_change":"2023-10-06T05:24:26.025051+0000","last_active":"2023-10-06T05:24:26.025051+0000","last_peered":"2023-10-06T05:24:26.025051+0000","last_clean":"2023-10-06T05:24:26.025051+0000","last_became_active":"2023-10-06T05:24:26.024353+0000","last_became_peered":"2023-10-06T05:24:26.024353+0000","last_unstale":"2023-10-06T05:24:26.025051+0000","last_undegraded":"2023-10-06T05:24:26.025051+0000","last_fullsized":"2023-10-06T05:24:26.025051+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:46.521669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029602+0000","last_change":"2023-10-06T05:24:25.025798+0000","last_active":"2023-10-06T05:24:25.029602+0000","last_peered":"2023-10-06T05:24:25.029602+0000","last_clean":"2023-10-06T05:24:25.029602+0000","last_became_active":"2023-10-06T05:24:25.025370+0000","last_became_peered":"2023-10-06T05:24:25.025370+0000","last_unstale":"2023-10-06T05:24:25.029602+0000","last_undegraded":"2023-10-06T05:24:25.029602+0000","last_fullsized":"2023-10-06T05:24:25.029602+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:27.439082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.021285+0000","last_change":"2023-10-06T05:24:26.021285+0000","last_active":"2023-10-06T05:24:26.021285+0000","last_peered":"2023-10-06T05:24:26.021285+0000","last_clean":"2023-10-06T05:24:26.021285+0000","last_became_active":"2023-10-06T05:24:26.021039+0000","last_became_peered":"2023-10-06T05:24:26.021039+0000","last_unstale":"2023-10-06T05:24:26.021285+0000","last_undegraded":"2023-10-06T05:24:26.021285+0000","last_fullsized":"2023-10-06T05:24:26.021285+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:04.330883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.021993+0000","last_change":"2023-10-06T05:24:25.020326+0000","last_active":"2023-10-06T05:24:25.021993+0000","last_peered":"2023-10-06T05:24:25.021993+0000","last_clean":"2023-10-06T05:24:25.021993+0000","last_became_active":"2023-10-06T05:24:25.020068+0000","last_became_peered":"2023-10-06T05:24:25.020068+0000","last_unstale":"2023-10-06T05:24:25.021993+0000","last_undegraded":"2023-10-06T05:24:25.021993+0000","last_fullsized":"2023-10-06T05:24:25.021993+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:03.321890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.040460+0000","last_change":"2023-10-06T05:24:26.040460+0000","last_active":"2023-10-06T05:24:26.040460+0000","last_peered":"2023-10-06T05:24:26.040460+0000","last_clean":"2023-10-06T05:24:26.040460+0000","last_became_active":"2023-10-06T05:24:26.040114+0000","last_became_peered":"2023-10-06T05:24:26.040114+0000","last_unstale":"2023-10-06T05:24:26.040460+0000","last_undegraded":"2023-10-06T05:24:26.040460+0000","last_fullsized":"2023-10-06T05:24:26.040460+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:51.249505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.011094+0000","last_change":"2023-10-06T05:24:25.023229+0000","last_active":"2023-10-06T05:24:26.011094+0000","last_peered":"2023-10-06T05:24:26.011094+0000","last_clean":"2023-10-06T05:24:26.011094+0000","last_became_active":"2023-10-06T05:24:25.022820+0000","last_became_peered":"2023-10-06T05:24:25.022820+0000","last_unstale":"2023-10-06T05:24:26.011094+0000","last_undegraded":"2023-10-06T05:24:26.011094+0000","last_fullsized":"2023-10-06T05:24:26.011094+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:09.349951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.029035+0000","last_change":"2023-10-06T05:24:26.029035+0000","last_active":"2023-10-06T05:24:26.029035+0000","last_peered":"2023-10-06T05:24:26.029035+0000","last_clean":"2023-10-06T05:24:26.029035+0000","last_became_active":"2023-10-06T05:24:26.028759+0000","last_became_peered":"2023-10-06T05:24:26.028759+0000","last_unstale":"2023-10-06T05:24:26.029035+0000","last_undegraded":"2023-10-06T05:24:26.029035+0000","last_fullsized":"2023-10-06T05:24:26.029035+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:53.238696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.032742+0000","last_change":"2023-10-06T05:24:25.023013+0000","last_active":"2023-10-06T05:24:26.032742+0000","last_peered":"2023-10-06T05:24:26.032742+0000","last_clean":"2023-10-06T05:24:26.032742+0000","last_became_active":"2023-10-06T05:24:25.022243+0000","last_became_peered":"2023-10-06T05:24:25.022243+0000","last_unstale":"2023-10-06T05:24:26.032742+0000","last_undegraded":"2023-10-06T05:24:26.032742+0000","last_fullsized":"2023-10-06T05:24:26.032742+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:37.782847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.041720+0000","last_change":"2023-10-06T05:24:26.041720+0000","last_active":"2023-10-06T05:24:26.041720+0000","last_peered":"2023-10-06T05:24:26.041720+0000","last_clean":"2023-10-06T05:24:26.041720+0000","last_became_active":"2023-10-06T05:24:26.041385+0000","last_became_peered":"2023-10-06T05:24:26.041385+0000","last_unstale":"2023-10-06T05:24:26.041720+0000","last_undegraded":"2023-10-06T05:24:26.041720+0000","last_fullsized":"2023-10-06T05:24:26.041720+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:18.157493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"143'1","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.007376+0000","last_change":"2023-10-06T05:24:25.020411+0000","last_active":"2023-10-06T05:24:26.007376+0000","last_peered":"2023-10-06T05:24:26.007376+0000","last_clean":"2023-10-06T05:24:26.007376+0000","last_became_active":"2023-10-06T05:24:25.020162+0000","last_became_peered":"2023-10-06T05:24:25.020162+0000","last_unstale":"2023-10-06T05:24:26.007376+0000","last_undegraded":"2023-10-06T05:24:26.007376+0000","last_fullsized":"2023-10-06T05:24:26.007376+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:09.742919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033544+0000","last_change":"2023-10-06T05:24:26.033544+0000","last_active":"2023-10-06T05:24:26.033544+0000","last_peered":"2023-10-06T05:24:26.033544+0000","last_clean":"2023-10-06T05:24:26.033544+0000","last_became_active":"2023-10-06T05:24:26.031554+0000","last_became_peered":"2023-10-06T05:24:26.031554+0000","last_unstale":"2023-10-06T05:24:26.033544+0000","last_undegraded":"2023-10-06T05:24:26.033544+0000","last_fullsized":"2023-10-06T05:24:26.033544+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:37.201579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029146+0000","last_change":"2023-10-06T05:24:25.027334+0000","last_active":"2023-10-06T05:24:25.029146+0000","last_peered":"2023-10-06T05:24:25.029146+0000","last_clean":"2023-10-06T05:24:25.029146+0000","last_became_active":"2023-10-06T05:24:25.027034+0000","last_became_peered":"2023-10-06T05:24:25.027034+0000","last_unstale":"2023-10-06T05:24:25.029146+0000","last_undegraded":"2023-10-06T05:24:25.029146+0000","last_fullsized":"2023-10-06T05:24:25.029146+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:18.059631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025280+0000","last_change":"2023-10-06T05:24:26.025280+0000","last_active":"2023-10-06T05:24:26.025280+0000","last_peered":"2023-10-06T05:24:26.025280+0000","last_clean":"2023-10-06T05:24:26.025280+0000","last_became_active":"2023-10-06T05:24:26.024992+0000","last_became_peered":"2023-10-06T05:24:26.024992+0000","last_unstale":"2023-10-06T05:24:26.025280+0000","last_undegraded":"2023-10-06T05:24:26.025280+0000","last_fullsized":"2023-10-06T05:24:26.025280+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:12.738848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.031325+0000","last_change":"2023-10-06T05:24:25.021450+0000","last_active":"2023-10-06T05:24:26.031325+0000","last_peered":"2023-10-06T05:24:26.031325+0000","last_clean":"2023-10-06T05:24:26.031325+0000","last_became_active":"2023-10-06T05:24:25.021072+0000","last_became_peered":"2023-10-06T05:24:25.021072+0000","last_unstale":"2023-10-06T05:24:26.031325+0000","last_undegraded":"2023-10-06T05:24:26.031325+0000","last_fullsized":"2023-10-06T05:24:26.031325+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:00.985124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028000+0000","last_change":"2023-10-06T05:24:26.028000+0000","last_active":"2023-10-06T05:24:26.028000+0000","last_peered":"2023-10-06T05:24:26.028000+0000","last_clean":"2023-10-06T05:24:26.028000+0000","last_became_active":"2023-10-06T05:24:26.027736+0000","last_became_peered":"2023-10-06T05:24:26.027736+0000","last_unstale":"2023-10-06T05:24:26.028000+0000","last_undegraded":"2023-10-06T05:24:26.028000+0000","last_fullsized":"2023-10-06T05:24:26.028000+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:38.791840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029445+0000","last_change":"2023-10-06T05:24:25.025497+0000","last_active":"2023-10-06T05:24:25.029445+0000","last_peered":"2023-10-06T05:24:25.029445+0000","last_clean":"2023-10-06T05:24:25.029445+0000","last_became_active":"2023-10-06T05:24:25.025185+0000","last_became_peered":"2023-10-06T05:24:25.025185+0000","last_unstale":"2023-10-06T05:24:25.029445+0000","last_undegraded":"2023-10-06T05:24:25.029445+0000","last_fullsized":"2023-10-06T05:24:25.029445+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:33.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.033727+0000","last_change":"2023-10-06T05:24:26.033727+0000","last_active":"2023-10-06T05:24:26.033727+0000","last_peered":"2023-10-06T05:24:26.033727+0000","last_clean":"2023-10-06T05:24:26.033727+0000","last_became_active":"2023-10-06T05:24:26.033465+0000","last_became_peered":"2023-10-06T05:24:26.033465+0000","last_unstale":"2023-10-06T05:24:26.033727+0000","last_undegraded":"2023-10-06T05:24:26.033727+0000","last_fullsized":"2023-10-06T05:24:26.033727+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:45.824772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.023401+0000","last_change":"2023-10-06T05:24:25.021480+0000","last_active":"2023-10-06T05:24:25.023401+0000","last_peered":"2023-10-06T05:24:25.023401+0000","last_clean":"2023-10-06T05:24:25.023401+0000","last_became_active":"2023-10-06T05:24:25.021150+0000","last_became_peered":"2023-10-06T05:24:25.021150+0000","last_unstale":"2023-10-06T05:24:25.023401+0000","last_undegraded":"2023-10-06T05:24:25.023401+0000","last_fullsized":"2023-10-06T05:24:25.023401+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:53.843072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.018797+0000","last_change":"2023-10-06T05:24:26.018797+0000","last_active":"2023-10-06T05:24:26.018797+0000","last_peered":"2023-10-06T05:24:26.018797+0000","last_clean":"2023-10-06T05:24:26.018797+0000","last_became_active":"2023-10-06T05:24:26.018469+0000","last_became_peered":"2023-10-06T05:24:26.018469+0000","last_unstale":"2023-10-06T05:24:26.018797+0000","last_undegraded":"2023-10-06T05:24:26.018797+0000","last_fullsized":"2023-10-06T05:24:26.018797+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:03.789342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029404+0000","last_change":"2023-10-06T05:24:25.025727+0000","last_active":"2023-10-06T05:24:25.029404+0000","last_peered":"2023-10-06T05:24:25.029404+0000","last_clean":"2023-10-06T05:24:25.029404+0000","last_became_active":"2023-10-06T05:24:25.025416+0000","last_became_peered":"2023-10-06T05:24:25.025416+0000","last_unstale":"2023-10-06T05:24:25.029404+0000","last_undegraded":"2023-10-06T05:24:25.029404+0000","last_fullsized":"2023-10-06T05:24:25.029404+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:09.112325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039366+0000","last_change":"2023-10-06T05:24:26.039366+0000","last_active":"2023-10-06T05:24:26.039366+0000","last_peered":"2023-10-06T05:24:26.039366+0000","last_clean":"2023-10-06T05:24:26.039366+0000","last_became_active":"2023-10-06T05:24:26.038821+0000","last_became_peered":"2023-10-06T05:24:26.038821+0000","last_unstale":"2023-10-06T05:24:26.039366+0000","last_undegraded":"2023-10-06T05:24:26.039366+0000","last_fullsized":"2023-10-06T05:24:26.039366+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:11.684203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028866+0000","last_change":"2023-10-06T05:24:25.029383+0000","last_active":"2023-10-06T05:24:26.028866+0000","last_peered":"2023-10-06T05:24:26.028866+0000","last_clean":"2023-10-06T05:24:26.028866+0000","last_became_active":"2023-10-06T05:24:25.029098+0000","last_became_peered":"2023-10-06T05:24:25.029098+0000","last_unstale":"2023-10-06T05:24:26.028866+0000","last_undegraded":"2023-10-06T05:24:26.028866+0000","last_fullsized":"2023-10-06T05:24:26.028866+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:37.528748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039826+0000","last_change":"2023-10-06T05:24:26.039826+0000","last_active":"2023-10-06T05:24:26.039826+0000","last_peered":"2023-10-06T05:24:26.039826+0000","last_clean":"2023-10-06T05:24:26.039826+0000","last_became_active":"2023-10-06T05:24:26.039562+0000","last_became_peered":"2023-10-06T05:24:26.039562+0000","last_unstale":"2023-10-06T05:24:26.039826+0000","last_undegraded":"2023-10-06T05:24:26.039826+0000","last_fullsized":"2023-10-06T05:24:26.039826+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:37.951417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.029773+0000","last_change":"2023-10-06T05:24:25.027434+0000","last_active":"2023-10-06T05:24:25.029773+0000","last_peered":"2023-10-06T05:24:25.029773+0000","last_clean":"2023-10-06T05:24:25.029773+0000","last_became_active":"2023-10-06T05:24:25.026797+0000","last_became_peered":"2023-10-06T05:24:25.026797+0000","last_unstale":"2023-10-06T05:24:25.029773+0000","last_undegraded":"2023-10-06T05:24:25.029773+0000","last_fullsized":"2023-10-06T05:24:25.029773+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:56.176190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.030547+0000","last_change":"2023-10-06T05:24:26.030547+0000","last_active":"2023-10-06T05:24:26.030547+0000","last_peered":"2023-10-06T05:24:26.030547+0000","last_clean":"2023-10-06T05:24:26.030547+0000","last_became_active":"2023-10-06T05:24:26.029799+0000","last_became_peered":"2023-10-06T05:24:26.029799+0000","last_unstale":"2023-10-06T05:24:26.030547+0000","last_undegraded":"2023-10-06T05:24:26.030547+0000","last_fullsized":"2023-10-06T05:24:26.030547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:59.484181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.024714+0000","last_change":"2023-10-06T05:24:25.020809+0000","last_active":"2023-10-06T05:24:25.024714+0000","last_peered":"2023-10-06T05:24:25.024714+0000","last_clean":"2023-10-06T05:24:25.024714+0000","last_became_active":"2023-10-06T05:24:25.020494+0000","last_became_peered":"2023-10-06T05:24:25.020494+0000","last_unstale":"2023-10-06T05:24:25.024714+0000","last_undegraded":"2023-10-06T05:24:25.024714+0000","last_fullsized":"2023-10-06T05:24:25.024714+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.852222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036404+0000","last_change":"2023-10-06T05:24:26.036404+0000","last_active":"2023-10-06T05:24:26.036404+0000","last_peered":"2023-10-06T05:24:26.036404+0000","last_clean":"2023-10-06T05:24:26.036404+0000","last_became_active":"2023-10-06T05:24:26.036121+0000","last_became_peered":"2023-10-06T05:24:26.036121+0000","last_unstale":"2023-10-06T05:24:26.036404+0000","last_undegraded":"2023-10-06T05:24:26.036404+0000","last_fullsized":"2023-10-06T05:24:26.036404+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:46.407284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.023798+0000","last_change":"2023-10-06T05:24:25.022622+0000","last_active":"2023-10-06T05:24:25.023798+0000","last_peered":"2023-10-06T05:24:25.023798+0000","last_clean":"2023-10-06T05:24:25.023798+0000","last_became_active":"2023-10-06T05:24:25.022333+0000","last_became_peered":"2023-10-06T05:24:25.022333+0000","last_unstale":"2023-10-06T05:24:25.023798+0000","last_undegraded":"2023-10-06T05:24:25.023798+0000","last_fullsized":"2023-10-06T05:24:25.023798+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:01:39.566765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.014888+0000","last_change":"2023-10-06T05:24:26.014888+0000","last_active":"2023-10-06T05:24:26.014888+0000","last_peered":"2023-10-06T05:24:26.014888+0000","last_clean":"2023-10-06T05:24:26.014888+0000","last_became_active":"2023-10-06T05:24:26.014609+0000","last_became_peered":"2023-10-06T05:24:26.014609+0000","last_unstale":"2023-10-06T05:24:26.014888+0000","last_undegraded":"2023-10-06T05:24:26.014888+0000","last_fullsized":"2023-10-06T05:24:26.014888+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:38.791840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.024724+0000","last_change":"2023-10-06T05:24:25.022738+0000","last_active":"2023-10-06T05:24:25.024724+0000","last_peered":"2023-10-06T05:24:25.024724+0000","last_clean":"2023-10-06T05:24:25.024724+0000","last_became_active":"2023-10-06T05:24:25.022469+0000","last_became_peered":"2023-10-06T05:24:25.022469+0000","last_unstale":"2023-10-06T05:24:25.024724+0000","last_undegraded":"2023-10-06T05:24:25.024724+0000","last_fullsized":"2023-10-06T05:24:25.024724+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:02.933521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.035450+0000","last_change":"2023-10-06T05:24:26.035450+0000","last_active":"2023-10-06T05:24:26.035450+0000","last_peered":"2023-10-06T05:24:26.035450+0000","last_clean":"2023-10-06T05:24:26.035450+0000","last_became_active":"2023-10-06T05:24:26.034570+0000","last_became_peered":"2023-10-06T05:24:26.034570+0000","last_unstale":"2023-10-06T05:24:26.035450+0000","last_undegraded":"2023-10-06T05:24:26.035450+0000","last_fullsized":"2023-10-06T05:24:26.035450+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:47.407150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.018482+0000","last_change":"2023-10-06T05:24:25.015760+0000","last_active":"2023-10-06T05:24:25.018482+0000","last_peered":"2023-10-06T05:24:25.018482+0000","last_clean":"2023-10-06T05:24:25.018482+0000","last_became_active":"2023-10-06T05:24:25.015434+0000","last_became_peered":"2023-10-06T05:24:25.015434+0000","last_unstale":"2023-10-06T05:24:25.018482+0000","last_undegraded":"2023-10-06T05:24:25.018482+0000","last_fullsized":"2023-10-06T05:24:25.018482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:14.680983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.025301+0000","last_change":"2023-10-06T05:24:26.025301+0000","last_active":"2023-10-06T05:24:26.025301+0000","last_peered":"2023-10-06T05:24:26.025301+0000","last_clean":"2023-10-06T05:24:26.025301+0000","last_became_active":"2023-10-06T05:24:26.024689+0000","last_became_peered":"2023-10-06T05:24:26.024689+0000","last_unstale":"2023-10-06T05:24:26.025301+0000","last_undegraded":"2023-10-06T05:24:26.025301+0000","last_fullsized":"2023-10-06T05:24:26.025301+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:32.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.022344+0000","last_change":"2023-10-06T05:24:25.021014+0000","last_active":"2023-10-06T05:24:25.022344+0000","last_peered":"2023-10-06T05:24:25.022344+0000","last_clean":"2023-10-06T05:24:25.022344+0000","last_became_active":"2023-10-06T05:24:25.020675+0000","last_became_peered":"2023-10-06T05:24:25.020675+0000","last_unstale":"2023-10-06T05:24:25.022344+0000","last_undegraded":"2023-10-06T05:24:25.022344+0000","last_fullsized":"2023-10-06T05:24:25.022344+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:15.549580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.039626+0000","last_change":"2023-10-06T05:24:26.039626+0000","last_active":"2023-10-06T05:24:26.039626+0000","last_peered":"2023-10-06T05:24:26.039626+0000","last_clean":"2023-10-06T05:24:26.039626+0000","last_became_active":"2023-10-06T05:24:26.039339+0000","last_became_peered":"2023-10-06T05:24:26.039339+0000","last_unstale":"2023-10-06T05:24:26.039626+0000","last_undegraded":"2023-10-06T05:24:26.039626+0000","last_fullsized":"2023-10-06T05:24:26.039626+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:36.901929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.026397+0000","last_change":"2023-10-06T05:24:25.023175+0000","last_active":"2023-10-06T05:24:25.026397+0000","last_peered":"2023-10-06T05:24:25.026397+0000","last_clean":"2023-10-06T05:24:25.026397+0000","last_became_active":"2023-10-06T05:24:25.022581+0000","last_became_peered":"2023-10-06T05:24:25.022581+0000","last_unstale":"2023-10-06T05:24:25.026397+0000","last_undegraded":"2023-10-06T05:24:25.026397+0000","last_fullsized":"2023-10-06T05:24:25.026397+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:02.780349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024025+0000","last_change":"2023-10-06T05:24:26.024025+0000","last_active":"2023-10-06T05:24:26.024025+0000","last_peered":"2023-10-06T05:24:26.024025+0000","last_clean":"2023-10-06T05:24:26.024025+0000","last_became_active":"2023-10-06T05:24:26.023694+0000","last_became_peered":"2023-10-06T05:24:26.023694+0000","last_unstale":"2023-10-06T05:24:26.024025+0000","last_undegraded":"2023-10-06T05:24:26.024025+0000","last_fullsized":"2023-10-06T05:24:26.024025+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:03.789342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"143'2","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.036867+0000","last_change":"2023-10-06T05:24:25.016833+0000","last_active":"2023-10-06T05:24:26.036867+0000","last_peered":"2023-10-06T05:24:26.036867+0000","last_clean":"2023-10-06T05:24:26.036867+0000","last_became_active":"2023-10-06T05:24:25.016189+0000","last_became_peered":"2023-10-06T05:24:25.016189+0000","last_unstale":"2023-10-06T05:24:26.036867+0000","last_undegraded":"2023-10-06T05:24:26.036867+0000","last_fullsized":"2023-10-06T05:24:26.036867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:36.192586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":114,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.006321+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:24:25.006321+0000","last_peered":"2023-10-06T05:24:25.006321+0000","last_clean":"2023-10-06T05:24:25.006321+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:24:25.006321+0000","last_undegraded":"2023-10-06T05:24:25.006321+0000","last_fullsized":"2023-10-06T05:24:25.006321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.023636+0000","last_change":"2023-10-06T05:24:26.023636+0000","last_active":"2023-10-06T05:24:26.023636+0000","last_peered":"2023-10-06T05:24:26.023636+0000","last_clean":"2023-10-06T05:24:26.023636+0000","last_became_active":"2023-10-06T05:24:26.023074+0000","last_became_peered":"2023-10-06T05:24:26.023074+0000","last_unstale":"2023-10-06T05:24:26.023636+0000","last_undegraded":"2023-10-06T05:24:26.023636+0000","last_fullsized":"2023-10-06T05:24:26.023636+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:48.627255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"143'3","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.034151+0000","last_change":"2023-10-06T05:24:25.011596+0000","last_active":"2023-10-06T05:24:26.034151+0000","last_peered":"2023-10-06T05:24:26.034151+0000","last_clean":"2023-10-06T05:24:26.034151+0000","last_became_active":"2023-10-06T05:24:25.011318+0000","last_became_peered":"2023-10-06T05:24:25.011318+0000","last_unstale":"2023-10-06T05:24:26.034151+0000","last_undegraded":"2023-10-06T05:24:26.034151+0000","last_fullsized":"2023-10-06T05:24:26.034151+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:57.397330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.012714+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:24:25.012714+0000","last_peered":"2023-10-06T05:24:25.012714+0000","last_clean":"2023-10-06T05:24:25.012714+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:24:25.012714+0000","last_undegraded":"2023-10-06T05:24:25.012714+0000","last_fullsized":"2023-10-06T05:24:25.012714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.022575+0000","last_change":"2023-10-06T05:24:26.022575+0000","last_active":"2023-10-06T05:24:26.022575+0000","last_peered":"2023-10-06T05:24:26.022575+0000","last_clean":"2023-10-06T05:24:26.022575+0000","last_became_active":"2023-10-06T05:24:26.022302+0000","last_became_peered":"2023-10-06T05:24:26.022302+0000","last_unstale":"2023-10-06T05:24:26.022575+0000","last_undegraded":"2023-10-06T05:24:26.022575+0000","last_fullsized":"2023-10-06T05:24:26.022575+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:50.013131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.027081+0000","last_change":"2023-10-06T05:24:25.023856+0000","last_active":"2023-10-06T05:24:25.027081+0000","last_peered":"2023-10-06T05:24:25.027081+0000","last_clean":"2023-10-06T05:24:25.027081+0000","last_became_active":"2023-10-06T05:24:25.023528+0000","last_became_peered":"2023-10-06T05:24:25.023528+0000","last_unstale":"2023-10-06T05:24:25.027081+0000","last_undegraded":"2023-10-06T05:24:25.027081+0000","last_fullsized":"2023-10-06T05:24:25.027081+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:43.540230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":110,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.005543+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:24:25.005543+0000","last_peered":"2023-10-06T05:24:25.005543+0000","last_clean":"2023-10-06T05:24:25.005543+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:24:25.005543+0000","last_undegraded":"2023-10-06T05:24:25.005543+0000","last_fullsized":"2023-10-06T05:24:25.005543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.032891+0000","last_change":"2023-10-06T05:24:26.032891+0000","last_active":"2023-10-06T05:24:26.032891+0000","last_peered":"2023-10-06T05:24:26.032891+0000","last_clean":"2023-10-06T05:24:26.032891+0000","last_became_active":"2023-10-06T05:24:26.032631+0000","last_became_peered":"2023-10-06T05:24:26.032631+0000","last_unstale":"2023-10-06T05:24:26.032891+0000","last_undegraded":"2023-10-06T05:24:26.032891+0000","last_fullsized":"2023-10-06T05:24:26.032891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:32.823896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"143'6","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.034232+0000","last_change":"2023-10-06T05:24:25.027090+0000","last_active":"2023-10-06T05:24:26.034232+0000","last_peered":"2023-10-06T05:24:26.034232+0000","last_clean":"2023-10-06T05:24:26.034232+0000","last_became_active":"2023-10-06T05:24:25.026814+0000","last_became_peered":"2023-10-06T05:24:25.026814+0000","last_unstale":"2023-10-06T05:24:26.034232+0000","last_undegraded":"2023-10-06T05:24:26.034232+0000","last_fullsized":"2023-10-06T05:24:26.034232+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:34.147695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.016039+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:24:25.016039+0000","last_peered":"2023-10-06T05:24:25.016039+0000","last_clean":"2023-10-06T05:24:25.016039+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:24:25.016039+0000","last_undegraded":"2023-10-06T05:24:25.016039+0000","last_fullsized":"2023-10-06T05:24:25.016039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":264,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.018658+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:24:25.018658+0000","last_peered":"2023-10-06T05:24:25.018658+0000","last_clean":"2023-10-06T05:24:25.018658+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:24:25.018658+0000","last_undegraded":"2023-10-06T05:24:25.018658+0000","last_fullsized":"2023-10-06T05:24:25.018658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.023340+0000","last_change":"2023-10-06T05:24:26.023340+0000","last_active":"2023-10-06T05:24:26.023340+0000","last_peered":"2023-10-06T05:24:26.023340+0000","last_clean":"2023-10-06T05:24:26.023340+0000","last_became_active":"2023-10-06T05:24:26.023008+0000","last_became_peered":"2023-10-06T05:24:26.023008+0000","last_unstale":"2023-10-06T05:24:26.023340+0000","last_undegraded":"2023-10-06T05:24:26.023340+0000","last_fullsized":"2023-10-06T05:24:26.023340+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:12.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.026280+0000","last_change":"2023-10-06T05:24:25.022787+0000","last_active":"2023-10-06T05:24:25.026280+0000","last_peered":"2023-10-06T05:24:25.026280+0000","last_clean":"2023-10-06T05:24:25.026280+0000","last_became_active":"2023-10-06T05:24:25.022520+0000","last_became_peered":"2023-10-06T05:24:25.022520+0000","last_unstale":"2023-10-06T05:24:25.026280+0000","last_undegraded":"2023-10-06T05:24:25.026280+0000","last_fullsized":"2023-10-06T05:24:25.026280+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:41.242037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.017500+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:24:25.017500+0000","last_peered":"2023-10-06T05:24:25.017500+0000","last_clean":"2023-10-06T05:24:25.017500+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:24:25.017500+0000","last_undegraded":"2023-10-06T05:24:25.017500+0000","last_fullsized":"2023-10-06T05:24:25.017500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.024618+0000","last_change":"2023-10-06T05:24:26.024618+0000","last_active":"2023-10-06T05:24:26.024618+0000","last_peered":"2023-10-06T05:24:26.024618+0000","last_clean":"2023-10-06T05:24:26.024618+0000","last_became_active":"2023-10-06T05:24:26.024324+0000","last_became_peered":"2023-10-06T05:24:26.024324+0000","last_unstale":"2023-10-06T05:24:26.024618+0000","last_undegraded":"2023-10-06T05:24:26.024618+0000","last_fullsized":"2023-10-06T05:24:26.024618+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:52.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.031339+0000","last_change":"2023-10-06T05:24:25.030188+0000","last_active":"2023-10-06T05:24:25.031339+0000","last_peered":"2023-10-06T05:24:25.031339+0000","last_clean":"2023-10-06T05:24:25.031339+0000","last_became_active":"2023-10-06T05:24:25.029889+0000","last_became_peered":"2023-10-06T05:24:25.029889+0000","last_unstale":"2023-10-06T05:24:25.031339+0000","last_undegraded":"2023-10-06T05:24:25.031339+0000","last_fullsized":"2023-10-06T05:24:25.031339+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:30.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.013220+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:24:25.013220+0000","last_peered":"2023-10-06T05:24:25.013220+0000","last_clean":"2023-10-06T05:24:25.013220+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:24:25.013220+0000","last_undegraded":"2023-10-06T05:24:25.013220+0000","last_fullsized":"2023-10-06T05:24:25.013220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.028606+0000","last_change":"2023-10-06T05:24:26.028606+0000","last_active":"2023-10-06T05:24:26.028606+0000","last_peered":"2023-10-06T05:24:26.028606+0000","last_clean":"2023-10-06T05:24:26.028606+0000","last_became_active":"2023-10-06T05:24:26.028237+0000","last_became_peered":"2023-10-06T05:24:26.028237+0000","last_unstale":"2023-10-06T05:24:26.028606+0000","last_undegraded":"2023-10-06T05:24:26.028606+0000","last_fullsized":"2023-10-06T05:24:26.028606+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:21.204638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.019388+0000","last_change":"2023-10-06T05:24:25.016644+0000","last_active":"2023-10-06T05:24:25.019388+0000","last_peered":"2023-10-06T05:24:25.019388+0000","last_clean":"2023-10-06T05:24:25.019388+0000","last_became_active":"2023-10-06T05:24:25.016328+0000","last_became_peered":"2023-10-06T05:24:25.016328+0000","last_unstale":"2023-10-06T05:24:25.019388+0000","last_undegraded":"2023-10-06T05:24:25.019388+0000","last_fullsized":"2023-10-06T05:24:25.019388+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:00.898613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.004721+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:24:25.004721+0000","last_peered":"2023-10-06T05:24:25.004721+0000","last_clean":"2023-10-06T05:24:25.004721+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:24:25.004721+0000","last_undegraded":"2023-10-06T05:24:25.004721+0000","last_fullsized":"2023-10-06T05:24:25.004721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":8,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.040178+0000","last_change":"2023-10-06T05:24:26.040178+0000","last_active":"2023-10-06T05:24:26.040178+0000","last_peered":"2023-10-06T05:24:26.040178+0000","last_clean":"2023-10-06T05:24:26.040178+0000","last_became_active":"2023-10-06T05:24:26.039879+0000","last_became_peered":"2023-10-06T05:24:26.039879+0000","last_unstale":"2023-10-06T05:24:26.040178+0000","last_undegraded":"2023-10-06T05:24:26.040178+0000","last_fullsized":"2023-10-06T05:24:26.040178+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:24.988905+0000","last_clean_scrub_stamp":"2023-10-06T05:24:24.988905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:04.330883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"143'8","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-06T05:24:26.042793+0000","last_change":"2023-10-06T05:24:25.027860+0000","last_active":"2023-10-06T05:24:26.042793+0000","last_peered":"2023-10-06T05:24:26.042793+0000","last_clean":"2023-10-06T05:24:26.042793+0000","last_became_active":"2023-10-06T05:24:25.027258+0000","last_became_peered":"2023-10-06T05:24:25.027258+0000","last_unstale":"2023-10-06T05:24:26.042793+0000","last_undegraded":"2023-10-06T05:24:26.042793+0000","last_fullsized":"2023-10-06T05:24:26.042793+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:23.979913+0000","last_clean_scrub_stamp":"2023-10-06T05:24:23.979913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:04.868067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:24:25.014238+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:24:25.014238+0000","last_peered":"2023-10-06T05:24:25.014238+0000","last_clean":"2023-10-06T05:24:25.014238+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:24:25.014238+0000","last_undegraded":"2023-10-06T05:24:25.014238+0000","last_fullsized":"2023-10-06T05:24:25.014238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303716,"kb_used_data":2332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068124,"statfs":{"total":96636764160,"available":96325758976,"internally_reserved":0,"allocated":2387968,"data_stored":1416559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70099999999999996,"15min":0.65800000000000003},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":0.92600000000000005,"5min":1.169,"15min":1.169},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.64700000000000002},"min":{"1min":0.51300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":1.268,"15min":1.268},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.075,"15min":1.075},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.45700000000000002,"5min":0.45300000000000001,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":1.093,"15min":1.093},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.69699999999999995},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93400000000000005,"5min":1.22,"15min":1.22},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.49399999999999999,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46600000000000003},"max":{"1min":0.96199999999999997,"5min":1.117,"15min":1.202},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.623},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.83599999999999997,"5min":1.258,"15min":1.258},"last":0.378},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.73099999999999998},"min":{"1min":0.52600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69999999999999996,"15min":0.69199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.054,"15min":1.054},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.95599999999999996,"5min":1.2050000000000001,"15min":1.407},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94499999999999995,"5min":1.139,"15min":1.139},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.74399999999999999},"min":{"1min":0.45100000000000001,"5min":0.442,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.24,"15min":1.3300000000000001},"last":0.48699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303136,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068704,"statfs":{"total":96636764160,"available":96326352896,"internally_reserved":0,"allocated":1794048,"data_stored":826239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.52000000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":1.0209999999999999,"15min":1.034},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.70499999999999996},"min":{"1min":0.49399999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.59099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69699999999999995,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.72499999999999998},"min":{"1min":0.54000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70599999999999996,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.755,"5min":1.05,"15min":1.1599999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.90600000000000003,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92000000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.78500000000000003},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.65900000000000003},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.95699999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63500000000000001,"15min":0.68500000000000005},"min":{"1min":0.443,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89100000000000001,"5min":0.90500000000000003,"15min":0.998},"last":0.498}]},{"osd":5,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.71799999999999997},"min":{"1min":0.434,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.751},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93600000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.749}]},{"osd":7,"last update":"Fri Oct 6 05:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.754},"min":{"1min":0.501,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.81000000000000005},"min":{"1min":0.53700000000000003,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.76200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303136,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068704,"statfs":{"total":96636764160,"available":96326352896,"internally_reserved":0,"allocated":1794048,"data_stored":826239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44900000000000001,"15min":0.502},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.54600000000000004},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":2.8050000000000002},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.83899999999999997,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82799999999999996,"5min":0.89500000000000002,"15min":2.7010000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.63400000000000001},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.88,"5min":0.88,"15min":2.7639999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":2.8370000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75,"15min":0.78700000000000003},"min":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":1.1080000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.0660000000000001,"5min":1.25,"15min":1.349},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78100000000000003,"15min":0.81499999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":1.016,"15min":1.669},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81000000000000005,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.86399999999999999},"min":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.503},"max":{"1min":1.141,"5min":1.3280000000000001,"15min":1.573},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.528},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.85199999999999998},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":0.97299999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303700,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068140,"statfs":{"total":96636764160,"available":96325775360,"internally_reserved":0,"allocated":2371584,"data_stored":1409421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.61299999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.502,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.81499999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.502}]},{"osd":3,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67200000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84499999999999997,"5min":0.879,"15min":0.879},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.76300000000000001},"min":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.1120000000000001,"5min":1.115,"15min":1.1160000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.125,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.59199999999999997,"5min":0.56499999999999995,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.218,"15min":1.218},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.089,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.52000000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.625},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.111,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.80600000000000005},"min":{"1min":0.61499999999999999,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0780000000000001,"5min":1.25,"15min":1.25},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.256,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":826281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":1.03,"5min":1.056,"15min":1.056},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.30599999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.94399999999999995,"5min":1.073,"15min":1.073},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.626}]},{"osd":4,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.109,"5min":1.109,"15min":1.1379999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.59799999999999998,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.86399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55800000000000005,"15min":0.53000000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.319},"last":0.88}]},{"osd":6,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77800000000000002,"15min":0.81200000000000006},"min":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.175,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.141},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.748},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.3520000000000001},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303140,"kb_used_data":1756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068700,"statfs":{"total":96636764160,"available":96326348800,"internally_reserved":0,"allocated":1798144,"data_stored":826757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46100000000000002,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.254},"max":{"1min":0.64500000000000002,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.495,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.29699999999999999},"max":{"1min":0.67500000000000004,"5min":0.77100000000000002,"15min":1.772},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.60099999999999998,"5min":0.79400000000000004,"15min":1.913},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":1.954},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.60899999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.044,"5min":1.0640000000000001,"15min":1.282},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":0.91100000000000003,"5min":1.216,"15min":1.2609999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.028,"5min":1.145,"15min":1.1930000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.51500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1399999999999999,"15min":1.3009999999999999},"last":0.628}]},{"osd":6,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79100000000000004,"15min":0.83299999999999996},"min":{"1min":0.60599999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1.0880000000000001,"15min":1.343},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"max":{"1min":0.98199999999999998,"5min":1.1779999999999999,"15min":1.3380000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:23:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.3640000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.53000000000000003},"max":{"1min":1.085,"5min":1.1539999999999999,"15min":1.214},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303144,"kb_used_data":1760,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068696,"statfs":{"total":96636764160,"available":96326344704,"internally_reserved":0,"allocated":1802240,"data_stored":827705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72799999999999998,"15min":0.71199999999999997},"min":{"1min":0.57499999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98699999999999999,"5min":1.0249999999999999,"15min":1.073},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.59499999999999997,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.169},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71399999999999997},"min":{"1min":0.51800000000000002,"5min":0.47199999999999998,"15min":0.432},"max":{"1min":0.93000000000000005,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69999999999999996,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.47199999999999998,"15min":0.42699999999999999},"max":{"1min":0.91300000000000003,"5min":1.016,"15min":1.117},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57999999999999996,"5min":0.498,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":0.96999999999999997,"15min":1.002},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.55300000000000005,"5min":0.496,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":0.98799999999999999,"15min":1.0780000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.75700000000000001,"15min":0.75800000000000001},"min":{"1min":0.55100000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.196},"last":0.73399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.433},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.375},"last":0.879}]},{"osd":6,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.96899999999999997,"15min":1.004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":0.95699999999999996,"15min":1.9390000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76000000000000001,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.50900000000000001,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.996,"15min":1.466},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79400000000000004,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98699999999999999,"5min":1.018,"15min":1.218},"last":0.81299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303132,"kb_used_data":1748,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068708,"statfs":{"total":96636764160,"available":96326356992,"internally_reserved":0,"allocated":1789952,"data_stored":826169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.67900000000000005},"min":{"1min":0.57699999999999996,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":1.165,"15min":1.165},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":1.1890000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83199999999999996,"5min":0.92800000000000005,"15min":0.98299999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.048,"15min":1.048},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.68600000000000005},"min":{"1min":0.54300000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.86899999999999999,"5min":1.204,"15min":1.204},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.495},"max":{"1min":0.79600000000000004,"5min":0.997,"15min":1.0549999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.71499999999999997},"min":{"1min":0.55900000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92700000000000005,"5min":1.002,"15min":1.087},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.495,"15min":0.433},"max":{"1min":0.90000000000000002,"5min":1.2769999999999999,"15min":1.278},"last":0.72499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63400000000000001,"15min":0.65700000000000003},"min":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.372},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.90000000000000002,"5min":1.1539999999999999,"15min":1.294},"last":0.748}]},{"osd":6,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.89200000000000002,"5min":1.141,"15min":1.141},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.0429999999999999,"15min":1.254},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.76900000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.0049999999999999,"15min":1.337},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.74299999999999999,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.248,"15min":1.248},"last":0.81200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:24:30.862 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:24:30.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.085+0000 7f1477fff700 1 Processor -- start 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.085+0000 7f1477fff700 1 -- start start 2023-10-06T05:24:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1477fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 0x7f147806a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1477fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1478159890 0x7f14780cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1477fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147815dbe0 con 0x7f14780c95c0 2023-10-06T05:24:31.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1477fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14780c9e30 con 0x7f1478159890 2023-10-06T05:24:31.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1477fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780ca110 con 0x7f14781594c0 2023-10-06T05:24:31.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f14767fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f14780c95c0 0x7f147806a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46482/0 (socket says 172.21.15.33:46482) 2023-10-06T05:24:31.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f14767fc700 1 -- 172.21.15.33:0/3716543885 learned_addr learned my addr 172.21.15.33:0/3716543885 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1476ffd700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 0x7f147806a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f14777fe700 1 --2- 172.21.15.33:0/3716543885 >> v2:172.21.15.167:3300/0 conn(0x7f1478159890 0x7f14780cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835988185 0 0) 0x7f147815dbe0 con 0x7f14780c95c0 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.087+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1460003570 con 0x7f14780c95c0 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043338043 0 0) 0x7f1460003570 con 0x7f14780c95c0 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 msgr2=0x7f147806a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 0x7f147806a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 >> v2:172.21.15.167:3300/0 conn(0x7f1478159890 msgr2=0x7f14780cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 --2- 172.21.15.33:0/3716543885 >> v2:172.21.15.167:3300/0 conn(0x7f1478159890 0x7f14780cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147815de30 con 0x7f14780c95c0 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f146c001470 con 0x7f14780c95c0 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 >> v1:172.21.15.33:6789/0 conn(0x7f14780c95c0 legacy=0x7f147806a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1460003ae0 0x7f1460003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 0x7f1460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460004d50 0x7f14600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c9e30 con 0x7f1460003ae0 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780ca110 con 0x7f1460004d50 2023-10-06T05:24:31.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146000bb10 con 0x7f14781594c0 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f1476ffd700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1460003ae0 0x7f1460003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f14767fc700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 0x7f1460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.088+0000 7f14777fe700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460004d50 0x7f14600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1476ffd700 1 -- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 msgr2=0x7f1460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1476ffd700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14781594c0 0x7f1460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1476ffd700 1 -- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460004d50 msgr2=0x7f14600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1476ffd700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460004d50 0x7f14600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1476ffd700 1 -- 172.21.15.33:0/3716543885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1460003570 con 0x7f1460003ae0 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f14777fe700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460004d50 0x7f14600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:31.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1476ffd700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1460003ae0 0x7f1460003ef0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1468009c30 tx=0x7f14680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25a7ff89da88f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800e710 con 0x7f1460003ae0 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146800e870 con 0x7f1460003ae0 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1475ffb700 1 -- 172.21.15.33:0/3716543885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800ec30 con 0x7f1460003ae0 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1477fff700 1 -- 172.21.15.33:0/3716543885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1460003ae0 msgr2=0x7f1460003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.089+0000 7f1477fff700 1 --2- 172.21.15.33:0/3716543885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1460003ae0 0x7f1460003ef0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1468009c30 tx=0x7f14680091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.090+0000 7f14777fe700 1 -- 172.21.15.33:0/3716543885 reap_dead start 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.090+0000 7f1477fff700 1 -- 172.21.15.33:0/3716543885 shutdown_connections 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.090+0000 7f1477fff700 1 -- 172.21.15.33:0/3716543885 >> 172.21.15.33:0/3716543885 conn(0x7f14780cd3b0 msgr2=0x7f14780b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.090+0000 7f1477fff700 1 -- 172.21.15.33:0/3716543885 shutdown_connections 2023-10-06T05:24:31.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.090+0000 7f1477fff700 1 -- 172.21.15.33:0/3716543885 wait complete. 2023-10-06T05:24:31.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.091+0000 7f1477fff700 1 Processor -- start 2023-10-06T05:24:31.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.091+0000 7f1477fff700 1 -- start start 2023-10-06T05:24:31.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f1477fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460003ae0 0x7f147814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f1477fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14780c7150 0x7f14780b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f1477fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 0x7f14780c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f1477fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c7a30 con 0x7f14780c5b30 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f1477fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c75c0 con 0x7f1460003ae0 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f1477fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147814eb10 con 0x7f14780c7150 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.092+0000 7f14777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 0x7f14780c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 0x7f14780c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35482/0 (socket says 172.21.15.33:35482) 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 -- 172.21.15.33:0/2932919306 learned_addr learned my addr 172.21.15.33:0/2932919306 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14767fc700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14780c7150 0x7f14780b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f1476ffd700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460003ae0 0x7f147814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 -- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14780c7150 msgr2=0x7f14780b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14780c7150 0x7f14780b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 -- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460003ae0 msgr2=0x7f147814ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460003ae0 0x7f147814ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 -- 172.21.15.33:0/2932919306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780c7c40 con 0x7f14780c5b30 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14767fc700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14780c7150 0x7f14780b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f14777fe700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 0x7f14780c5fb0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f14780cb2c0 tx=0x7f147000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147001b070 con 0x7f14780c5b30 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147000f040 con 0x7f14780c5b30 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000e050 con 0x7f14780c5b30 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.093+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14780c7f20 con 0x7f14780c5b30 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.094+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14780c84e0 con 0x7f14780c5b30 2023-10-06T05:24:31.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.095+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14780c8700 con 0x7f14780c5b30 2023-10-06T05:24:31.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.096+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1470003020 con 0x7f14780c5b30 2023-10-06T05:24:31.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.097+0000 7f145effd700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1464066f60 0x7f1464069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.097+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1470084b00 con 0x7f14780c5b30 2023-10-06T05:24:31.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.097+0000 7f1476ffd700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1464066f60 0x7f1464069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.098+0000 7f1476ffd700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1464066f60 0x7f1464069410 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f14680099c0 tx=0x7f146801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.100+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1470086e60 con 0x7f14780c5b30 2023-10-06T05:24:31.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.383+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f14780c7850 con 0x7f14780c5b30 2023-10-06T05:24:31.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.388+0000 7f145effd700 1 -- 172.21.15.33:0/2932919306 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1470051b70 con 0x7f14780c5b30 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1464066f60 msgr2=0x7f1464069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1464066f60 0x7f1464069410 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f14680099c0 tx=0x7f146801a000 comp rx=0 tx=0).stop 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 msgr2=0x7f14780c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 0x7f14780c5fb0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f14780cb2c0 tx=0x7f147000b040 comp rx=0 tx=0).stop 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 shutdown_connections 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1464066f60 0x7f1464069410 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14780c5b30 0x7f14780c5fb0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1460003ae0 0x7f147814ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 --2- 172.21.15.33:0/2932919306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14780c7150 0x7f14780b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 >> 172.21.15.33:0/2932919306 conn(0x7f14780cd3b0 msgr2=0x7f1478153830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 shutdown_connections 2023-10-06T05:24:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.390+0000 7f1477fff700 1 -- 172.21.15.33:0/2932919306 wait complete. 2023-10-06T05:24:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:24:31.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.594+0000 7fc0def9d700 1 Processor -- start 2023-10-06T05:24:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.594+0000 7fc0def9d700 1 -- start start 2023-10-06T05:24:31.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.595+0000 7fc0def9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc0d80c8050 0x7fc0d813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.595+0000 7fc0def9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d80c9cb0 con 0x7fc0d813faa0 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.595+0000 7fc0def9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d80ca270 con 0x7fc0d806aad0 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.595+0000 7fc0def9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d806b640 con 0x7fc0d80c8050 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0ddf9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc0d813faa0 0x7fc0d80c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46486/0 (socket says 172.21.15.33:46486) 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/296888578 learned_addr learned my addr 172.21.15.33:0/296888578 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dd79a700 1 --2- 172.21.15.33:0/296888578 >> v2:172.21.15.167:3300/0 conn(0x7fc0d80c8050 0x7fc0d813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124302621 0 0) 0x7fc0d80c9cb0 con 0x7fc0d813faa0 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c4003570 con 0x7fc0d813faa0 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3405929163 0 0) 0x7fc0c4003570 con 0x7fc0d813faa0 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 >> v2:172.21.15.167:3300/0 conn(0x7fc0d80c8050 msgr2=0x7fc0d813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 --2- 172.21.15.33:0/296888578 >> v2:172.21.15.167:3300/0 conn(0x7fc0d80c8050 0x7fc0d813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 >> v1:172.21.15.167:6789/0 conn(0x7fc0d806aad0 legacy=0x7fc0d806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d806b840 con 0x7fc0d813faa0 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.596+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc0d4001470 con 0x7fc0d813faa0 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 >> v1:172.21.15.33:6789/0 conn(0x7fc0d813faa0 legacy=0x7fc0d80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0c40051c0 0x7fc0c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d806b640 con 0x7fc0c4003a60 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c400c050 con 0x7fc0c4004810 2023-10-06T05:24:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c400c210 con 0x7fc0c40051c0 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dd79a700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0de79c700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0c40051c0 0x7fc0c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0c40051c0 msgr2=0x7fc0c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0c40051c0 0x7fc0c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4004810 msgr2=0x7fc0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/296888578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4003570 con 0x7fc0c4003a60 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0dd79a700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.597+0000 7fc0de79c700 1 -- 172.21.15.33:0/296888578 reap_dead start 2023-10-06T05:24:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0c4003e30 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c390 tx=0x7fc0d80c84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbbf3c832178357 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400cc60 con 0x7fc0c4003a60 2023-10-06T05:24:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d400cdc0 con 0x7fc0c4003a60 2023-10-06T05:24:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0dcf99700 1 -- 172.21.15.33:0/296888578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4005b70 con 0x7fc0c4003a60 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0def9d700 1 -- 172.21.15.33:0/296888578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 msgr2=0x7fc0c4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0def9d700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0c4003e30 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c390 tx=0x7fc0d80c84c0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0def9d700 1 -- 172.21.15.33:0/296888578 shutdown_connections 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0def9d700 1 --2- 172.21.15.33:0/296888578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0c4003e30 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.598+0000 7fc0def9d700 1 -- 172.21.15.33:0/296888578 >> 172.21.15.33:0/296888578 conn(0x7fc0d806d030 msgr2=0x7fc0d806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.599+0000 7fc0def9d700 1 -- 172.21.15.33:0/296888578 shutdown_connections 2023-10-06T05:24:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.599+0000 7fc0def9d700 1 -- 172.21.15.33:0/296888578 wait complete. 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.600+0000 7fc0def9d700 1 Processor -- start 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.600+0000 7fc0def9d700 1 -- start start 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0def9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0d80c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0def9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d80c4630 0x7fc0d81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0def9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d813ce90 0x7fc0d813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0def9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80c3560 con 0x7fc0c4003a60 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0def9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80c59b0 con 0x7fc0d813ce90 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0def9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d813d830 con 0x7fc0d80c4630 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0ddf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0d80c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0ddf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0d80c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35492/0 (socket says 172.21.15.33:35492) 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.601+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/2366705910 learned_addr learned my addr 172.21.15.33:0/2366705910 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0dd79a700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d80c4630 0x7fc0d81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0de79c700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d813ce90 0x7fc0d813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d80c4630 msgr2=0x7fc0d81595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d80c4630 0x7fc0d81595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d813ce90 msgr2=0x7fc0d813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d813ce90 0x7fc0d813d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0ddf9b700 1 -- 172.21.15.33:0/2366705910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d80c4da0 con 0x7fc0c4003a60 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0ddf9b700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0d80c4080 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4004b60 tx=0x7fc0d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4022070 con 0x7fc0c4003a60 2023-10-06T05:24:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d40072e0 con 0x7fc0c4003a60 2023-10-06T05:24:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d80c5080 con 0x7fc0c4003a60 2023-10-06T05:24:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400cc60 con 0x7fc0c4003a60 2023-10-06T05:24:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.602+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d80c5640 con 0x7fc0c4003a60 2023-10-06T05:24:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.603+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d804c380 con 0x7fc0c4003a60 2023-10-06T05:24:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.604+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0d401e430 con 0x7fc0c4003a60 2023-10-06T05:24:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.606+0000 7fc0cdffb700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0bc066f30 0x7fc0bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.606+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0d4085ac0 con 0x7fc0c4003a60 2023-10-06T05:24:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.606+0000 7fc0dd79a700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0bc066f30 0x7fc0bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.607+0000 7fc0dd79a700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0bc066f30 0x7fc0bc0693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80093f0 tx=0x7fc0c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.610+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0d4052b30 con 0x7fc0c4003a60 2023-10-06T05:24:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.859+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0d806bb20 con 0x7fc0c4003a60 2023-10-06T05:24:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.860+0000 7fc0cdffb700 1 -- 172.21.15.33:0/2366705910 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7fc0d4056150 con 0x7fc0c4003a60 2023-10-06T05:24:31.861 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:31.861 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0bc066f30 msgr2=0x7fc0bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0bc066f30 0x7fc0bc0693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80093f0 tx=0x7fc0c8008040 comp rx=0 tx=0).stop 2023-10-06T05:24:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 msgr2=0x7fc0d80c4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0d80c4080 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4004b60 tx=0x7fc0d4015000 comp rx=0 tx=0).stop 2023-10-06T05:24:31.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 shutdown_connections 2023-10-06T05:24:31.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0bc066f30 0x7fc0bc0693e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d813ce90 0x7fc0d813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0c4003a60 0x7fc0d80c4080 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 --2- 172.21.15.33:0/2366705910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d80c4630 0x7fc0d81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 >> 172.21.15.33:0/2366705910 conn(0x7fc0d806d030 msgr2=0x7fc0d806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 shutdown_connections 2023-10-06T05:24:31.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:31.862+0000 7fc0def9d700 1 -- 172.21.15.33:0/2366705910 wait complete. 2023-10-06T05:24:31.866 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:31.985 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:24:31.984+0000 7fd1e0bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:32.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:24:25.495496+0000', 'modified': '2023-10-06T05:24:29.025589+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8373': {'gid': 8373, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/1921586494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 1921586494}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 1921586494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:24:32.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:24:32.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:24:32.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.475+0000 7fd4edd9d700 1 Processor -- start 2023-10-06T05:24:32.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.475+0000 7fd4edd9d700 1 -- start start 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4edd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4e80c7070 0x7fd4e80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4edd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e8155bc0 0x7fd4e80c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4edd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e806a410 con 0x7fd4e81557f0 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4edd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e80cccd0 con 0x7fd4e80c7070 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4edd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c7500 con 0x7fd4e8155bc0 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4e7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4e80c7070 0x7fd4e80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4e7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4e80c7070 0x7fd4e80c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35506/0 (socket says 172.21.15.33:35506) 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4e7fff700 1 -- 172.21.15.33:0/3172272694 learned_addr learned my addr 172.21.15.33:0/3172272694 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.476+0000 7fd4ecd9b700 1 --1- 172.21.15.33:0/3172272694 >> v1:172.21.15.33:6789/0 conn(0x7fd4e81557f0 0x7fd4e80c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46496/0 (socket says 172.21.15.33:46496) 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e7fff700 1 -- 172.21.15.33:0/3172272694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e8155bc0 msgr2=0x7fd4e80c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e8155bc0 0x7fd4e80c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e8155bc0 0x7fd4e80c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e7fff700 1 -- 172.21.15.33:0/3172272694 >> v1:172.21.15.33:6789/0 conn(0x7fd4e81557f0 legacy=0x7fd4e80c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e7fff700 1 -- 172.21.15.33:0/3172272694 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80c77e0 con 0x7fd4e80c7070 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e8155bc0 0x7fd4e80c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e77fe700 1 -- 172.21.15.33:0/3172272694 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208395654 0 0) 0x7fd4e806a410 con 0x7fd4e81557f0 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e80c7070 0x7fd4e80c8b10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40099b0 tx=0x7fd4d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860f6f992693e248 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e77fe700 1 -- 172.21.15.33:0/3172272694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400b860 con 0x7fd4e80c7070 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e77fe700 1 -- 172.21.15.33:0/3172272694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d400b9c0 con 0x7fd4e80c7070 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.477+0000 7fd4e77fe700 1 -- 172.21.15.33:0/3172272694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400bd60 con 0x7fd4e80c7070 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 -- 172.21.15.33:0/3172272694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e80c7070 msgr2=0x7fd4e80c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e80c7070 0x7fd4e80c8b10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40099b0 tx=0x7fd4d4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 -- 172.21.15.33:0/3172272694 shutdown_connections 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e8155bc0 0x7fd4e80c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/3172272694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e80c7070 0x7fd4e80c8b10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 -- 172.21.15.33:0/3172272694 >> 172.21.15.33:0/3172272694 conn(0x7fd4e806c560 msgr2=0x7fd4e806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 -- 172.21.15.33:0/3172272694 shutdown_connections 2023-10-06T05:24:32.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.478+0000 7fd4edd9d700 1 -- 172.21.15.33:0/3172272694 wait complete. 2023-10-06T05:24:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.479+0000 7fd4edd9d700 1 Processor -- start 2023-10-06T05:24:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.480+0000 7fd4edd9d700 1 -- start start 2023-10-06T05:24:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.480+0000 7fd4edd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e80c7070 0x7fd4e80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.480+0000 7fd4edd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 0x7fd4e80c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4edd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4e8155bc0 0x7fd4e80c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4edd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c46b0 con 0x7fd4e81557f0 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4edd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c4990 con 0x7fd4e8155bc0 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4edd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c4c70 con 0x7fd4e80c7070 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 0x7fd4e80c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 0x7fd4e80c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35514/0 (socket says 172.21.15.33:35514) 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 -- 172.21.15.33:0/2518498540 learned_addr learned my addr 172.21.15.33:0/2518498540 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e80c7070 0x7fd4e80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4e8155bc0 0x7fd4e80c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 -- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e80c7070 msgr2=0x7fd4e80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e80c7070 0x7fd4e80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 -- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4e8155bc0 msgr2=0x7fd4e80c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4e8155bc0 0x7fd4e80c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 -- 172.21.15.33:0/2518498540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40094f0 con 0x7fd4e81557f0 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e80c7070 0x7fd4e80c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4e8155bc0 0x7fd4e80c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.481+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 0x7fd4e80c2120 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400c010 tx=0x7fd4d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.482+0000 7fd4e4ff9700 1 -- 172.21.15.33:0/2518498540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400bd40 con 0x7fd4e81557f0 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.482+0000 7fd4e4ff9700 1 -- 172.21.15.33:0/2518498540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d4020440 con 0x7fd4e81557f0 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.482+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e8169210 con 0x7fd4e81557f0 2023-10-06T05:24:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.482+0000 7fd4e4ff9700 1 -- 172.21.15.33:0/2518498540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4020800 con 0x7fd4e81557f0 2023-10-06T05:24:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.482+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e80c39f0 con 0x7fd4e81557f0 2023-10-06T05:24:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.485+0000 7fd4e4ff9700 1 -- 172.21.15.33:0/2518498540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4d4020bb0 con 0x7fd4e81557f0 2023-10-06T05:24:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.485+0000 7fd4e4ff9700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4cc066f30 0x7fd4cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.486+0000 7fd4e4ff9700 1 -- 172.21.15.33:0/2518498540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4d4013070 con 0x7fd4e81557f0 2023-10-06T05:24:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.486+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4cc066f30 0x7fd4cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.487+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4cc066f30 0x7fd4cc0693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001840 tx=0x7fd4dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.487+0000 7fd4fd71db80 1 Processor -- start 2023-10-06T05:24:32.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.487+0000 7fd4fd71db80 1 -- start start 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a95bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a984920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a957990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a8dbc80 con 0x55659a95b950 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a8b2a70 con 0x55659a9a5bc0 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55659a8b0270 con 0x55659a984e60 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4ecd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a95bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4ecd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a95bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35530/0 (socket says 172.21.15.33:35530) 2023-10-06T05:24:32.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.488+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/1787282695 learned_addr learned my addr 172.21.15.33:0/1787282695 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a957990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a984920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 msgr2=0x55659a957990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a957990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 msgr2=0x55659a984920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a984920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/1787282695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55659a746270 con 0x55659a95b950 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a984920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a957990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a95bd20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005ca0 tx=0x7fd4dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9484c3fddeae054f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4d1ffb700 1 -- 172.21.15.33:0/1787282695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005e00 con 0x55659a95b950 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4d1ffb700 1 -- 172.21.15.33:0/1787282695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00d920 con 0x55659a95b950 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.489+0000 7fd4d1ffb700 1 -- 172.21.15.33:0/1787282695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00dca0 con 0x55659a95b950 2023-10-06T05:24:32.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1787282695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 msgr2=0x55659a95bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a95bd20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005ca0 tx=0x7fd4dc012000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1787282695 shutdown_connections 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a957990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a95bd20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1787282695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a984920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1787282695 >> 172.21.15.33:0/1787282695 conn(0x55659a95d270 msgr2=0x55659a95d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1787282695 shutdown_connections 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.490+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1787282695 wait complete. 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.491+0000 7fd4fd71db80 1 -- start start 2023-10-06T05:24:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.491+0000 7fd4fd71db80 1 Processor -- start 2023-10-06T05:24:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a99fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659a9521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a8ac060 con 0x55659a9a5bc0 2023-10-06T05:24:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a857890 con 0x55659a95b950 2023-10-06T05:24:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55659a872ea0 con 0x55659a984e60 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4fd71db80 10 client.0 _ll_register_callbacks cb 0x55659a748f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4ed59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659a9521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.492+0000 7fd4ed59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659a9521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35534/0 (socket says 172.21.15.33:35534) 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 -- 172.21.15.33:0/1832827872 learned_addr learned my addr 172.21.15.33:0/1832827872 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a99fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 -- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 msgr2=0x55659a99fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a99fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 -- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 msgr2=0x55659a9814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 -- 172.21.15.33:0/1832827872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc005860 con 0x55659a9a5bc0 2023-10-06T05:24:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659a9521a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000ea50 tx=0x7fd4e0007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6173078c0855a12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/1832827872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000ca70 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/1832827872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4e000cbd0 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.493+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55659a950690 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/1832827872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4e0016410 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 10 client.8517 fetch_fsmap learned FSMap version 105 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55659a6e8fe0 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/1832827872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55659a950690 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/1832827872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 105) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd4e0016570 con 0x55659a9a5bc0 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 10 client.8517 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T05:24:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 10 client.8517 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 20 client.8517 populate_metadata read hostname 'smithi033' 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9835b0 0x55659a983980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 4 client.8517 mds_command: new command op to 8373 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 --> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55659a92d5c0 con 0x55659a9835b0 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.494+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9835b0 0x55659a983980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.495+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9835b0 0x55659a983980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f586589e3832874d server_cookie=af6c482c9e3e637e in_seq=0 out_seq=0 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.495+0000 7fd4c6ffd700 10 client.8517 ms_handle_connect on v2:172.21.15.33:6833/1921586494 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/1832827872 <== mds.0 v2:172.21.15.33:6833/1921586494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55659a92d5c0 con 0x55659a9835b0 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4c6ffd700 10 client.8517 handle_command_reply: tid=0 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 client.8517 shutdown 2023-10-06T05:24:32.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 msgr2=0x55659a9521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659a9521a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000ea50 tx=0x7fd4e0007a80 comp rx=0 tx=0).stop 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 shutdown_connections 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9835b0 0x55659a983980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659a9521a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4c6ffd700 0 client.8517 ms_handle_reset on v2:172.21.15.33:6833/1921586494 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/1832827872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a99fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 >> 172.21.15.33:0/1832827872 conn(0x55659a95d270 msgr2=0x55659a9a14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 shutdown_connections 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 1 -- 172.21.15.33:0/1832827872 wait complete. 2023-10-06T05:24:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.496+0000 7fd4fd71db80 20 client.8517 trim_cache size 0 max 16384 2023-10-06T05:24:32.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 Processor -- start 2023-10-06T05:24:32.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 -- start start 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a98ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a999ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a99f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a95ac40 con 0x55659a95b950 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a72ee30 con 0x55659a9a5bc0 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.514+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55659a9a3910 con 0x55659a984e60 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a98ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a98ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35544/0 (socket says 172.21.15.33:35544) 2023-10-06T05:24:32.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/894464067 learned_addr learned my addr 172.21.15.33:0/894464067 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a999ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a99f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 msgr2=0x55659a999ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a999ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 msgr2=0x55659a99f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a99f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 -- 172.21.15.33:0/894464067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55659a99bee0 con 0x55659a95b950 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a99f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.515+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a98ba30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0016f0 tx=0x7fd4dc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320b7177525fa9ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4d27fc700 1 -- 172.21.15.33:0/894464067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00da90 con 0x55659a95b950 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4d27fc700 1 -- 172.21.15.33:0/894464067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc018030 con 0x55659a95b950 2023-10-06T05:24:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4d27fc700 1 -- 172.21.15.33:0/894464067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00dc10 con 0x55659a95b950 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 -- 172.21.15.33:0/894464067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 msgr2=0x55659a98ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a98ba30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0016f0 tx=0x7fd4dc01c000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 -- 172.21.15.33:0/894464067 shutdown_connections 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a9a5bc0 0x55659a99f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a95b950 0x55659a98ba30 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/894464067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659a999ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 -- 172.21.15.33:0/894464067 >> 172.21.15.33:0/894464067 conn(0x55659a95d270 msgr2=0x55659a959710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 -- 172.21.15.33:0/894464067 shutdown_connections 2023-10-06T05:24:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.516+0000 7fd4fd71db80 1 -- 172.21.15.33:0/894464067 wait complete. 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.517+0000 7fd4fd71db80 1 -- start start 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.517+0000 7fd4fd71db80 1 Processor -- start 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659aa07950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659aa07e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a9991d0 con 0x55659a9a5bc0 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659a9b0ee0 con 0x55659a95b950 2023-10-06T05:24:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55659a9988a0 con 0x55659a984e60 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4fd71db80 10 client.0 _ll_register_callbacks cb 0x55659a748f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4ed59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659aa07e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4ed59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659aa07e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35560/0 (socket says 172.21.15.33:35560) 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.518+0000 7fd4ed59c700 1 -- 172.21.15.33:0/2680440854 learned_addr learned my addr 172.21.15.33:0/2680440854 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4e7fff700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659aa07950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ed59c700 1 -- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 msgr2=0x55659aa07950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659aa07950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ed59c700 1 -- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 msgr2=0x55659a9b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ed59c700 1 -- 172.21.15.33:0/2680440854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55659a746270 con 0x55659a9a5bc0 2023-10-06T05:24:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4ed59c700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659aa07e90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00070b0 tx=0x7fd4e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c335c6e6f5761e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/2680440854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000c650 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/2680440854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4e0016ad0 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.519+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4dc00d920 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/2680440854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4e0016410 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 10 client.8523 fetch_fsmap learned FSMap version 105 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55659a87a780 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/2680440854 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00d920 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/2680440854 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 105) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd4e0016570 con 0x55659a9a5bc0 2023-10-06T05:24:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 10 client.8523 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 10 client.8523 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 20 client.8523 populate_metadata read hostname 'smithi033' 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9bf270 0x55659a9bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 4 client.8523 mds_command: new command op to 8373 tid=0[{"prefix": "status"}] 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 --> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55659a84ee00 con 0x55659a9bf270 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.520+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9bf270 0x55659a9bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4c6ffd700 10 client.8523 ms_handle_connect on v2:172.21.15.33:6833/1921586494 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4ecd9b700 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9bf270 0x55659a9bf640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee40a2aff214f8b server_cookie=828236efdbc2c124 in_seq=0 out_seq=0 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4c6ffd700 1 -- 172.21.15.33:0/2680440854 <== mds.0 v2:172.21.15.33:6833/1921586494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55659a84ee00 con 0x55659a9bf270 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4c6ffd700 10 client.8523 handle_command_reply: tid=0 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 client.8523 shutdown 2023-10-06T05:24:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 msgr2=0x55659aa07e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659aa07e90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00070b0 tx=0x7fd4e001a000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 shutdown_connections 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:6833/1921586494,v1:172.21.15.33:6835/1921586494] conn(0x55659a9bf270 0x55659a9bf640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55659a9a5bc0 0x55659aa07e90 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55659a95b950 0x55659a9b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 --2- 172.21.15.33:0/2680440854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55659a984e60 0x55659aa07950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4c6ffd700 0 client.8523 ms_handle_reset on v2:172.21.15.33:6833/1921586494 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 >> 172.21.15.33:0/2680440854 conn(0x55659a95d270 msgr2=0x55659a999ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 shutdown_connections 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.521+0000 7fd4fd71db80 1 -- 172.21.15.33:0/2680440854 wait complete. 2023-10-06T05:24:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4fd71db80 20 client.8523 trim_cache size 0 max 16384 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 8373, 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5199804759999997, 2023-10-06T05:24:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 105, 2023-10-06T05:24:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 144, 2023-10-06T05:24:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 144, 2023-10-06T05:24:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.261508258999999 2023-10-06T05:24:32.530 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4cc066f30 msgr2=0x7fd4cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4cc066f30 0x7fd4cc0693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001840 tx=0x7fd4dc009040 comp rx=0 tx=0).stop 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 msgr2=0x7fd4e80c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 0x7fd4e80c2120 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400c010 tx=0x7fd4d4019000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 shutdown_connections 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4cc066f30 0x7fd4cc0693e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4e8155bc0 0x7fd4e80c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4e80c7070 0x7fd4e80c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 --2- 172.21.15.33:0/2518498540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4e81557f0 0x7fd4e80c2120 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 >> 172.21.15.33:0/2518498540 conn(0x7fd4e806c560 msgr2=0x7fd4e80ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 shutdown_connections 2023-10-06T05:24:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.522+0000 7fd4edd9d700 1 -- 172.21.15.33:0/2518498540 wait complete. 2023-10-06T05:24:32.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.727+0000 7fc001f9d700 1 Processor -- start 2023-10-06T05:24:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.728+0000 7fc001f9d700 1 -- start start 2023-10-06T05:24:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fc001f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc0c6340 0x7fbffc0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fc001f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc06c820 con 0x7fbffc0c80d0 2023-10-06T05:24:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fc001f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc06a160 con 0x7fbffc159520 2023-10-06T05:24:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fc001f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0c8800 con 0x7fbffc0c6340 2023-10-06T05:24:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fc00179c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbffc0c80d0 0x7fbffc06a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46502/0 (socket says 172.21.15.33:46502) 2023-10-06T05:24:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fc00179c700 1 -- 172.21.15.33:0/431045701 learned_addr learned my addr 172.21.15.33:0/431045701 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.729+0000 7fbffbfff700 1 --2- 172.21.15.33:0/431045701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc0c6340 0x7fbffc0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810051807 0 0) 0x7fbffc06c820 con 0x7fbffc0c80d0 2023-10-06T05:24:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfec003570 con 0x7fbffc0c80d0 2023-10-06T05:24:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917571962 0 0) 0x7fbffc06a160 con 0x7fbffc159520 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbffc06c820 con 0x7fbffc159520 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffbfff700 1 -- 172.21.15.33:0/431045701 >> v1:172.21.15.167:6789/0 conn(0x7fbffc159520 legacy=0x7fbffc0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffbfff700 1 -- 172.21.15.33:0/431045701 >> v1:172.21.15.33:6789/0 conn(0x7fbffc0c80d0 legacy=0x7fbffc06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919628345 0 0) 0x7fbfec003570 con 0x7fbffc0c80d0 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffbfff700 1 -- 172.21.15.33:0/431045701 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0c8ae0 con 0x7fbffc0c6340 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.730+0000 7fbffbfff700 1 --2- 172.21.15.33:0/431045701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc0c6340 0x7fbffc0c7b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009910 tx=0x7fbfe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f358fc3860cd9d95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff40017f0 con 0x7fbffc0c6340 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe8011420 con 0x7fbffc0c6340 2023-10-06T05:24:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fbffb7fe700 1 -- 172.21.15.33:0/431045701 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80117c0 con 0x7fbffc0c6340 2023-10-06T05:24:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 -- 172.21.15.33:0/431045701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc0c6340 msgr2=0x7fbffc0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 --2- 172.21.15.33:0/431045701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc0c6340 0x7fbffc0c7b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009910 tx=0x7fbfe80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 -- 172.21.15.33:0/431045701 shutdown_connections 2023-10-06T05:24:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 --2- 172.21.15.33:0/431045701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc0c6340 0x7fbffc0c7b90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 -- 172.21.15.33:0/431045701 >> 172.21.15.33:0/431045701 conn(0x7fbffc0cc930 msgr2=0x7fbffc0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 -- 172.21.15.33:0/431045701 shutdown_connections 2023-10-06T05:24:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.731+0000 7fc001f9d700 1 -- 172.21.15.33:0/431045701 wait complete. 2023-10-06T05:24:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.733+0000 7fc001f9d700 1 Processor -- start 2023-10-06T05:24:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.733+0000 7fc001f9d700 1 -- start start 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc001f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 0x7fbffc15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc001f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc0c80d0 0x7fbffc1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc001f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc159520 0x7fbffc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc001f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc153d30 con 0x7fbffc0c6340 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc001f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc154010 con 0x7fbffc0c80d0 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc001f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1542f0 con 0x7fbffc159520 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 0x7fbffc15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 0x7fbffc15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35570/0 (socket says 172.21.15.33:35570) 2023-10-06T05:24:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 -- 172.21.15.33:0/2412823476 learned_addr learned my addr 172.21.15.33:0/2412823476 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 -- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc159520 msgr2=0x7fbffc159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc00179c700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc159520 0x7fbffc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fbffbfff700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc0c80d0 0x7fbffc1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc159520 0x7fbffc159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 -- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc0c80d0 msgr2=0x7fbffc1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc0c80d0 0x7fbffc1532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc000f9b700 1 -- 172.21.15.33:0/2412823476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8009420 con 0x7fbffc0c6340 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fc00179c700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc159520 0x7fbffc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.734+0000 7fbffbfff700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc0c80d0 0x7fbffc1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.735+0000 7fc000f9b700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 0x7fbffc15bf10 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c790 tx=0x7fbff0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.735+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0010510 con 0x7fbffc0c6340 2023-10-06T05:24:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.735+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0010670 con 0x7fbffc0c6340 2023-10-06T05:24:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.735+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc1545d0 con 0x7fbffc0c6340 2023-10-06T05:24:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.735+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0010a10 con 0x7fbffc0c6340 2023-10-06T05:24:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.735+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc14f8a0 con 0x7fbffc0c6340 2023-10-06T05:24:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.736+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc154af0 con 0x7fbffc0c6340 2023-10-06T05:24:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.737+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbff001b400 con 0x7fbffc0c6340 2023-10-06T05:24:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.738+0000 7fbff8ff9700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfe0066f60 0x7fbfe0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.738+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbff0084440 con 0x7fbffc0c6340 2023-10-06T05:24:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.738+0000 7fbffbfff700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfe0066f60 0x7fbfe0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.739+0000 7fbffbfff700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfe0066f60 0x7fbfe0069410 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009860 tx=0x7fbfe8016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.742+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbffc154af0 con 0x7fbffc0c6340 2023-10-06T05:24:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.996+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbffc154af0 con 0x7fbffc0c6340 2023-10-06T05:24:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.996+0000 7fbff8ff9700 1 -- 172.21.15.33:0/2412823476 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbffc154af0 con 0x7fbffc0c6340 2023-10-06T05:24:32.998 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.998+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfe0066f60 msgr2=0x7fbfe0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.998+0000 7fc001f9d700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfe0066f60 0x7fbfe0069410 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009860 tx=0x7fbfe8016000 comp rx=0 tx=0).stop 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.998+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 msgr2=0x7fbffc15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.998+0000 7fc001f9d700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 0x7fbffc15bf10 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c790 tx=0x7fbff0007e10 comp rx=0 tx=0).stop 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 shutdown_connections 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbfe0066f60 0x7fbfe0069410 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbffc159520 0x7fbffc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbffc0c6340 0x7fbffc15bf10 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 --2- 172.21.15.33:0/2412823476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc0c80d0 0x7fbffc1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 >> 172.21.15.33:0/2412823476 conn(0x7fbffc0cc930 msgr2=0x7fbffc156dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 shutdown_connections 2023-10-06T05:24:33.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:32.999+0000 7fc001f9d700 1 -- 172.21.15.33:0/2412823476 wait complete. 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:33.018 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:33.019 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:33.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.019 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:24:33.056 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:33.057 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41684sec preferred_lft 41684sec 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:33.058 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:24:33.060 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:24:33.060 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:33.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.060 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:33.156 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:24:33.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.160 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:24:33.198 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:24:33.198 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:33.198 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:24:33.198 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:24:33.198 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:33.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.199 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:24:33.200 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:24:33.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:24:33.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:24:33.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:24:33.200 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:24:33.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.400 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:24:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:24:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:24:33.437 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:33.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:24:33.541 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:33.542 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:33.579 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:33.579 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:33.579 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:33.655 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:24:33.655 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:33.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:33.788 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:33.788 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:24:33.789 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:24:33.805 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:33.805 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:24:33.805 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:24:33.805 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:33.806 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:33.826 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:33.826 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:24:33.826 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:24:33.826 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:33.827 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:33.827 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:24:33.827 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:24:33.827 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:24:33.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:33.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:33.831 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:33.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:34.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.082+0000 7ff50af9d700 1 Processor -- start 2023-10-06T05:24:34.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.082+0000 7ff50af9d700 1 -- start start 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff504155af0 0x7ff50406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50af9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5040c7920 0x7ff5040c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50af9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50406ab60 con 0x7ff50406a6a0 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50af9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50406b7e0 con 0x7ff5040c7920 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50406ae40 con 0x7ff504155af0 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50a79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5040c7920 0x7ff5040c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50a79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5040c7920 0x7ff5040c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35586/0 (socket says 172.21.15.33:35586) 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50a79c700 1 -- 172.21.15.33:0/3077417098 learned_addr learned my addr 172.21.15.33:0/3077417098 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff509f9b700 1 --2- 172.21.15.33:0/3077417098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff504155af0 0x7ff50406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50a79c700 1 -- 172.21.15.33:0/3077417098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff504155af0 msgr2=0x7ff50406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50a79c700 1 --2- 172.21.15.33:0/3077417098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff504155af0 0x7ff50406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.084+0000 7ff50a79c700 1 -- 172.21.15.33:0/3077417098 >> v1:172.21.15.167:6789/0 conn(0x7ff50406a6a0 legacy=0x7ff5040c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff50a79c700 1 -- 172.21.15.33:0/3077417098 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50406b120 con 0x7ff5040c7920 2023-10-06T05:24:34.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff50a79c700 1 --2- 172.21.15.33:0/3077417098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5040c7920 0x7ff5040c9e20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009f20 tx=0x7ff4fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd4c6c4613958bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff508f99700 1 -- 172.21.15.33:0/3077417098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00e070 con 0x7ff5040c7920 2023-10-06T05:24:34.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff508f99700 1 -- 172.21.15.33:0/3077417098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00ba80 con 0x7ff5040c7920 2023-10-06T05:24:34.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff508f99700 1 -- 172.21.15.33:0/3077417098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0092e0 con 0x7ff5040c7920 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff50af9d700 1 -- 172.21.15.33:0/3077417098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5040c7920 msgr2=0x7ff5040c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.085+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3077417098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5040c7920 0x7ff5040c9e20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009f20 tx=0x7ff4fc009840 comp rx=0 tx=0).stop 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.086+0000 7ff50af9d700 1 -- 172.21.15.33:0/3077417098 shutdown_connections 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.086+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3077417098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5040c7920 0x7ff5040c9e20 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.086+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3077417098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff504155af0 0x7ff50406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.086+0000 7ff50af9d700 1 -- 172.21.15.33:0/3077417098 >> 172.21.15.33:0/3077417098 conn(0x7ff50406d030 msgr2=0x7ff50406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.086+0000 7ff50af9d700 1 -- 172.21.15.33:0/3077417098 shutdown_connections 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.086+0000 7ff50af9d700 1 -- 172.21.15.33:0/3077417098 wait complete. 2023-10-06T05:24:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.087+0000 7ff50af9d700 1 Processor -- start 2023-10-06T05:24:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.087+0000 7ff50af9d700 1 -- start start 2023-10-06T05:24:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.088+0000 7ff50af9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50406a6a0 0x7ff5040c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.088+0000 7ff50af9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5040c7920 0x7ff5040c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.088+0000 7ff50af9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 0x7ff5040c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.088+0000 7ff50af9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040c3970 con 0x7ff504155af0 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.088+0000 7ff50af9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040ac5c0 con 0x7ff5040c7920 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.088+0000 7ff50af9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040c36d0 con 0x7ff50406a6a0 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 0x7ff5040c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 0x7ff5040c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35594/0 (socket says 172.21.15.33:35594) 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 -- 172.21.15.33:0/3904021263 learned_addr learned my addr 172.21.15.33:0/3904021263 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff509f9b700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50406a6a0 0x7ff5040c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50979a700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5040c7920 0x7ff5040c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 -- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50406a6a0 msgr2=0x7ff5040c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50406a6a0 0x7ff5040c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 -- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5040c7920 msgr2=0x7ff5040c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5040c7920 0x7ff5040c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 -- 172.21.15.33:0/3904021263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc009ae0 con 0x7ff504155af0 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff50a79c700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 0x7ff5040c23f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff5040635d0 tx=0x7ff4fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0032f0 con 0x7ff504155af0 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.089+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc008e80 con 0x7ff504155af0 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.090+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5040abc90 con 0x7ff504155af0 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.090+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc026690 con 0x7ff504155af0 2023-10-06T05:24:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.090+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5040ac250 con 0x7ff504155af0 2023-10-06T05:24:34.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.091+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50404c380 con 0x7ff504155af0 2023-10-06T05:24:34.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.092+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4fc003040 con 0x7ff504155af0 2023-10-06T05:24:34.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.093+0000 7ff4f9ffb700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4e8066f30 0x7ff4e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.093+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff4fc0866c0 con 0x7ff504155af0 2023-10-06T05:24:34.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.093+0000 7ff509f9b700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4e8066f30 0x7ff4e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.094+0000 7ff509f9b700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4e8066f30 0x7ff4e80693e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff500006fd0 tx=0x7ff500008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.097+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4fc053730 con 0x7ff504155af0 2023-10-06T05:24:34.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.346+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff50406b400 con 0x7ff504155af0 2023-10-06T05:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.346+0000 7ff4f9ffb700 1 -- 172.21.15.33:0/3904021263 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4fc056d50 con 0x7ff504155af0 2023-10-06T05:24:34.348 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:34.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4e8066f30 msgr2=0x7ff4e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4e8066f30 0x7ff4e80693e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff500006fd0 tx=0x7ff500008040 comp rx=0 tx=0).stop 2023-10-06T05:24:34.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 msgr2=0x7ff5040c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 0x7ff5040c23f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff5040635d0 tx=0x7ff4fc018000 comp rx=0 tx=0).stop 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 shutdown_connections 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4e8066f30 0x7ff4e80693e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff504155af0 0x7ff5040c23f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50406a6a0 0x7ff5040c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 --2- 172.21.15.33:0/3904021263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5040c7920 0x7ff5040c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 >> 172.21.15.33:0/3904021263 conn(0x7ff50406d030 msgr2=0x7ff5040ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.348+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 shutdown_connections 2023-10-06T05:24:34.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:34.349+0000 7ff50af9d700 1 -- 172.21.15.33:0/3904021263 wait complete. 2023-10-06T05:24:34.367 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:34.367 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:34.367 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:24:34.368 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:24:34.368 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:34.368 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:34.368 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:34.368 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:34.368 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:34.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.369 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:24:34.410 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:34.410 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:34.410 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:34.411 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41687sec preferred_lft 41687sec 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:24:34.412 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:34.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.413 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:34.515 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:24:34.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.518 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:24:34.557 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:24:34.557 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:34.557 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:24:34.557 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:24:34.557 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:34.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.558 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:24:34.558 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:24:34.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:24:34.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:24:34.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:24:34.559 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:24:34.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.753 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:24:34.789 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:24:34.790 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:24:34.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:34.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.791 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:24:34.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:34.892 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:34.928 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:34.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:34.929 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:35.004 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:24:35.004 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:35.004 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:35.134 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:35.134 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:24:35.134 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:24:35.150 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:35.150 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:24:35.150 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:24:35.150 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:35.150 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:35.173 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:24:35.173 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:24:35.174 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:24:35.174 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:35.174 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:35.174 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:24:35.174 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:24:35.174 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:24:35.179 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:35.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:35.179 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:35.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:24:35.458 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:24:35.458 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:24:35.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:24:35.662 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:24:35.662 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:24:35.662 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q fuse --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:24:35.708 INFO:teuthology.orchestra.run.smithi033.stdout:2.9.7-12.el8 2023-10-06T05:24:35.708 INFO:teuthology.packaging:The installed version of fuse is 2.9.7-12.el8 2023-10-06T05:24:35.709 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q fuse --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:24:35.755 INFO:teuthology.orchestra.run.smithi167.stdout:2.9.7-12.el8 2023-10-06T05:24:35.755 INFO:teuthology.packaging:The installed version of fuse is 2.9.7-12.el8 2023-10-06T05:24:35.755 INFO:tasks.cephfs.test_client_recovery:flock locks are available 2023-10-06T05:24:35.755 INFO:tasks.cephfs.mount:lock_background file background_file 2023-10-06T05:24:35.755 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> import fcntl 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> import struct 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> f1 = open("/home/ubuntu/cephtest/mnt.0/background_file-1", '"'"'w'"'"') 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> fcntl.flock(f1, fcntl.LOCK_EX | fcntl.LOCK_NB) 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> f2 = open("/home/ubuntu/cephtest/mnt.0/background_file-2", '"'"'w'"'"') 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> lockdata = struct.pack('"'"'hhllhh'"'"', fcntl.F_WRLCK, 0, 0, 0, 0, 0) 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> fcntl.fcntl(f2, fcntl.F_SETLK, lockdata) 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> while True: 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(1) 2023-10-06T05:24:35.756 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:24:35.761 DEBUG:teuthology.orchestra.run.smithi167:> stat /home/ubuntu/cephtest/mnt.1/background_file-2 2023-10-06T05:24:35.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:35.832 INFO:teuthology.orchestra.run.smithi167.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.1/background_file-2': No such file or directory 2023-10-06T05:24:36.833 DEBUG:teuthology.orchestra.run.smithi167:> stat /home/ubuntu/cephtest/mnt.1/background_file-2 2023-10-06T05:24:36.880 INFO:teuthology.orchestra.run.smithi167.stdout: File: /home/ubuntu/cephtest/mnt.1/background_file-2 2023-10-06T05:24:36.880 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 0 Blocks: 0 IO Block: 4194304 regular empty file 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 2bh/43d Inode: 1099511627777 Links: 1 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0664/-rw-rw-r--) Uid: ( 1000/ ubuntu) Gid: ( 1275/ ubuntu) 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 05:24:35.865479911 +0000 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 05:24:35.865479911 +0000 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 05:24:35.865479911 +0000 2023-10-06T05:24:36.881 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T05:24:36.881 INFO:tasks.cephfs.mount:File background_file-2 became visible from 1 after 1s 2023-10-06T05:24:36.881 INFO:tasks.cephfs.mount:check lock on file background_file 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> python3 -c ' 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> import fcntl 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> import errno 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> import struct 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> f1 = open("/home/ubuntu/cephtest/mnt.1/background_file-1", '"'"'r'"'"') 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.flock(f1, fcntl.LOCK_EX | fcntl.LOCK_NB) 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> if e.errno == errno.EAGAIN: 2023-10-06T05:24:36.882 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> else: 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("flock on file /home/ubuntu/cephtest/mnt.1/background_file-1 not found") 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> f2 = open("/home/ubuntu/cephtest/mnt.1/background_file-2", '"'"'r'"'"') 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> lockdata = struct.pack('"'"'hhllhh'"'"', fcntl.F_WRLCK, 0, 0, 0, 0, 0) 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.fcntl(f2, fcntl.F_SETLK, lockdata) 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> if e.errno == errno.EAGAIN: 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> else: 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("posix lock on file /home/ubuntu/cephtest/mnt.1/background_file-2 not found") 2023-10-06T05:24:36.883 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:24:36.992 INFO:tasks.cephfs.mount:lock_and_release file background_file 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> import time 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> import fcntl 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> import struct 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> f1 = open("/home/ubuntu/cephtest/mnt.1/background_file-1", '"'"'w'"'"') 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.flock(f1, fcntl.LOCK_EX) 2023-10-06T05:24:36.992 DEBUG:teuthology.orchestra.run.smithi167:> f2 = open("/home/ubuntu/cephtest/mnt.1/background_file-2", '"'"'w'"'"') 2023-10-06T05:24:36.993 DEBUG:teuthology.orchestra.run.smithi167:> lockdata = struct.pack('"'"'hhllhh'"'"', fcntl.F_WRLCK, 0, 0, 0, 0, 0) 2023-10-06T05:24:36.993 DEBUG:teuthology.orchestra.run.smithi167:> fcntl.fcntl(f2, fcntl.F_SETLK, lockdata) 2023-10-06T05:24:36.993 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:24:39.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:39.200 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:24:39.280 INFO:teuthology.orchestra.run.smithi033.stdout:client8532 2023-10-06T05:24:39.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.469+0000 7f9f0359e700 1 Processor -- start 2023-10-06T05:24:39.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.469+0000 7f9f0359e700 1 -- start start 2023-10-06T05:24:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f0359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9f040ccfc0 0x7f9f040cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f0359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9f040cf3b0 0x7f9f041594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f0359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f040cc750 0x7f9f041599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f0359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f040cbd80 con 0x7f9f040cf3b0 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f0359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f040af5f0 con 0x7f9f040ccfc0 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f0359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040af8d0 con 0x7f9f040cc750 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.470+0000 7f9f01d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9f040cf3b0 0x7f9f041594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9f040cf3b0 0x7f9f041594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35598/0 (socket says 172.21.15.33:35598) 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 -- 172.21.15.33:0/461526844 learned_addr learned my addr 172.21.15.33:0/461526844 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f0259c700 1 --2- 172.21.15.33:0/461526844 >> v2:172.21.15.167:3300/0 conn(0x7f9f040ccfc0 0x7f9f040cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f02d9d700 1 --2- 172.21.15.33:0/461526844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f040cc750 0x7f9f041599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 -- 172.21.15.33:0/461526844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f040cc750 msgr2=0x7f9f041599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 --2- 172.21.15.33:0/461526844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f040cc750 0x7f9f041599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 -- 172.21.15.33:0/461526844 >> v2:172.21.15.167:3300/0 conn(0x7f9f040ccfc0 msgr2=0x7f9f040cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:39.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 --2- 172.21.15.33:0/461526844 >> v2:172.21.15.167:3300/0 conn(0x7f9f040ccfc0 0x7f9f040cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 -- 172.21.15.33:0/461526844 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040afbb0 con 0x7f9f040cf3b0 2023-10-06T05:24:39.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f0259c700 1 --2- 172.21.15.33:0/461526844 >> v2:172.21.15.167:3300/0 conn(0x7f9f040ccfc0 0x7f9f040cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.471+0000 7f9f01d9b700 1 --2- 172.21.15.33:0/461526844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cf3b0 0x7f9f041594a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80099b0 tx=0x7f9ef8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4040e995cb43c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0159a700 1 -- 172.21.15.33:0/461526844 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800b820 con 0x7f9f040cf3b0 2023-10-06T05:24:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0159a700 1 -- 172.21.15.33:0/461526844 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef800b980 con 0x7f9f040cf3b0 2023-10-06T05:24:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0159a700 1 -- 172.21.15.33:0/461526844 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800bd20 con 0x7f9f040cf3b0 2023-10-06T05:24:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 -- 172.21.15.33:0/461526844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cf3b0 msgr2=0x7f9f041594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 --2- 172.21.15.33:0/461526844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cf3b0 0x7f9f041594a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80099b0 tx=0x7f9ef8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 -- 172.21.15.33:0/461526844 shutdown_connections 2023-10-06T05:24:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 --2- 172.21.15.33:0/461526844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f040cc750 0x7f9f041599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 --2- 172.21.15.33:0/461526844 >> v2:172.21.15.167:3300/0 conn(0x7f9f040ccfc0 0x7f9f040cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 --2- 172.21.15.33:0/461526844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cf3b0 0x7f9f041594a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 -- 172.21.15.33:0/461526844 >> 172.21.15.33:0/461526844 conn(0x7f9f0406cff0 msgr2=0x7f9f0406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 -- 172.21.15.33:0/461526844 shutdown_connections 2023-10-06T05:24:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.472+0000 7f9f0359e700 1 -- 172.21.15.33:0/461526844 wait complete. 2023-10-06T05:24:39.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.474+0000 7f9f0359e700 1 Processor -- start 2023-10-06T05:24:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.474+0000 7f9f0359e700 1 -- start start 2023-10-06T05:24:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 0x7f9f040ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f040ccfc0 0x7f9f041699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f04145050 0x7f9f04158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040b32a0 con 0x7f9f040cc750 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040b3580 con 0x7f9f040ccfc0 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0416a280 con 0x7f9f04145050 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 0x7f9f040ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 0x7f9f040ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35602/0 (socket says 172.21.15.33:35602) 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 -- 172.21.15.33:0/1296871214 learned_addr learned my addr 172.21.15.33:0/1296871214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f01d9b700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f040ccfc0 0x7f9f041699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f02d9d700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f04145050 0x7f9f04158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 -- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f04145050 msgr2=0x7f9f04158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f04145050 0x7f9f04158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 -- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f040ccfc0 msgr2=0x7f9f041699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.475+0000 7f9f0259c700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f040ccfc0 0x7f9f041699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9f0259c700 1 -- 172.21.15.33:0/1296871214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef80094f0 con 0x7f9f040cc750 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9f02d9d700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f04145050 0x7f9f04158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9f0259c700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 0x7f9f040ca060 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e8d0 tx=0x7f9ef4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4013410 con 0x7f9f040cc750 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4013570 con 0x7f9f040cc750 2023-10-06T05:24:39.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4013910 con 0x7f9f040cc750 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0416a560 con 0x7f9f040cc750 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.476+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0416ac00 con 0x7f9f040cc750 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.477+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0404c380 con 0x7f9f040cc750 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.478+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ef4013d00 con 0x7f9f040cc750 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.479+0000 7f9ef27fc700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ee4066f30 0x7f9ee40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.479+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ef40844a0 con 0x7f9f040cc750 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.480+0000 7f9f01d9b700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ee4066f30 0x7f9ee40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.480+0000 7f9f01d9b700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ee4066f30 0x7f9ee40693e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80099b0 tx=0x7f9ef8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.484+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ef4051510 con 0x7f9f040cc750 2023-10-06T05:24:39.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.732+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f040c5a90 con 0x7f9f040cc750 2023-10-06T05:24:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.732+0000 7f9ef27fc700 1 -- 172.21.15.33:0/1296871214 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7f9ef4054b30 con 0x7f9f040cc750 2023-10-06T05:24:39.734 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:39.734 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ee4066f30 msgr2=0x7f9ee40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ee4066f30 0x7f9ee40693e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80099b0 tx=0x7f9ef8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 msgr2=0x7f9f040ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 0x7f9f040ca060 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e8d0 tx=0x7f9ef4008040 comp rx=0 tx=0).stop 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 shutdown_connections 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ee4066f30 0x7f9ee40693e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f04145050 0x7f9f04158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f040cc750 0x7f9f040ca060 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 --2- 172.21.15.33:0/1296871214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f040ccfc0 0x7f9f041699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 >> 172.21.15.33:0/1296871214 conn(0x7f9f0406cff0 msgr2=0x7f9f040b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 shutdown_connections 2023-10-06T05:24:39.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:39.735+0000 7f9f0359e700 1 -- 172.21.15.33:0/1296871214 wait complete. 2023-10-06T05:24:39.738 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:39.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session evict 8532 2023-10-06T05:24:40.857 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:24:40.886 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:41.238 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T05:24:41.238 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:41.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:41.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:24:41.311 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:24:41.311 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:24:41.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:41.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:24:41.390 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:24:41.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:41.390 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:41.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:41.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.653+0000 7f990a59d700 1 Processor -- start 2023-10-06T05:24:41.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.654+0000 7f990a59d700 1 -- start start 2023-10-06T05:24:41.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f990a59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9904155af0 0x7f990406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f990a59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f990406a6a0 0x7f99040c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f990a59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990406ab60 con 0x7f99040c8b50 2023-10-06T05:24:41.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f990a59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990406b7e0 con 0x7f990406a6a0 2023-10-06T05:24:41.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f990a59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990406ae40 con 0x7f9904155af0 2023-10-06T05:24:41.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f9908d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f990406a6a0 0x7f99040c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f9908d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f990406a6a0 0x7f99040c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52970/0 (socket says 172.21.15.33:52970) 2023-10-06T05:24:41.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.655+0000 7f9908d9a700 1 -- 172.21.15.33:0/3212950598 learned_addr learned my addr 172.21.15.33:0/3212950598 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f990959b700 1 --2- 172.21.15.33:0/3212950598 >> v2:172.21.15.167:3300/0 conn(0x7f9904155af0 0x7f990406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f9908d9a700 1 -- 172.21.15.33:0/3212950598 >> v2:172.21.15.167:3300/0 conn(0x7f9904155af0 msgr2=0x7f990406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f9908d9a700 1 --2- 172.21.15.33:0/3212950598 >> v2:172.21.15.167:3300/0 conn(0x7f9904155af0 0x7f990406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f9908d9a700 1 -- 172.21.15.33:0/3212950598 >> v1:172.21.15.167:6789/0 conn(0x7f99040c8b50 legacy=0x7f99041598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f9908d9a700 1 -- 172.21.15.33:0/3212950598 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990406b120 con 0x7f990406a6a0 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f9908d9a700 1 --2- 172.21.15.33:0/3212950598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990406a6a0 0x7f99040c7210 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f99040697c0 tx=0x7f98f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6fa8030f42d9fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f98fbfff700 1 -- 172.21.15.33:0/3212950598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400b860 con 0x7f990406a6a0 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f98fbfff700 1 -- 172.21.15.33:0/3212950598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400b9c0 con 0x7f990406a6a0 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.656+0000 7f98fbfff700 1 -- 172.21.15.33:0/3212950598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400bd60 con 0x7f990406a6a0 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 -- 172.21.15.33:0/3212950598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990406a6a0 msgr2=0x7f99040c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 --2- 172.21.15.33:0/3212950598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990406a6a0 0x7f99040c7210 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f99040697c0 tx=0x7f98f4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 -- 172.21.15.33:0/3212950598 shutdown_connections 2023-10-06T05:24:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 --2- 172.21.15.33:0/3212950598 >> v2:172.21.15.167:3300/0 conn(0x7f9904155af0 0x7f990406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 --2- 172.21.15.33:0/3212950598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990406a6a0 0x7f99040c7210 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 -- 172.21.15.33:0/3212950598 >> 172.21.15.33:0/3212950598 conn(0x7f990406d030 msgr2=0x7f990406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 -- 172.21.15.33:0/3212950598 shutdown_connections 2023-10-06T05:24:41.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.657+0000 7f990a59d700 1 -- 172.21.15.33:0/3212950598 wait complete. 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.658+0000 7f990a59d700 1 Processor -- start 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.659+0000 7f990a59d700 1 -- start start 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.659+0000 7f990a59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990406a6a0 0x7f9904146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f990a59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 0x7f9904157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f990a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9904155af0 0x7f99041584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f990a59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990416a020 con 0x7f99040c8b50 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f990a59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9904158ad0 con 0x7f990406a6a0 2023-10-06T05:24:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f990a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9904158db0 con 0x7f9904155af0 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 0x7f9904157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 0x7f9904157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52986/0 (socket says 172.21.15.33:52986) 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 -- 172.21.15.33:0/4089396917 learned_addr learned my addr 172.21.15.33:0/4089396917 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f990959b700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990406a6a0 0x7f9904146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9909d9c700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9904155af0 0x7f99041584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 -- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9904155af0 msgr2=0x7f99041584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9904155af0 0x7f99041584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 -- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990406a6a0 msgr2=0x7f9904146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990406a6a0 0x7f9904146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.660+0000 7f9908d9a700 1 -- 172.21.15.33:0/4089396917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40094f0 con 0x7f99040c8b50 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.661+0000 7f9908d9a700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 0x7f9904157f60 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f99040635d0 tx=0x7f98f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.661+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4024440 con 0x7f99040c8b50 2023-10-06T05:24:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.661+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f40245a0 con 0x7f99040c8b50 2023-10-06T05:24:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.661+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9904159090 con 0x7f99040c8b50 2023-10-06T05:24:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.661+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4024960 con 0x7f99040c8b50 2023-10-06T05:24:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.661+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99040a6120 con 0x7f99040c8b50 2023-10-06T05:24:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.662+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99041595b0 con 0x7f99040c8b50 2023-10-06T05:24:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.663+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98f401a420 con 0x7f99040c8b50 2023-10-06T05:24:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.664+0000 7f98f97fa700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98e8066f30 0x7f98e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.664+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98f40864b0 con 0x7f99040c8b50 2023-10-06T05:24:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.664+0000 7f990959b700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98e8066f30 0x7f98e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.665+0000 7f990959b700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98e8066f30 0x7f98e80693e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9900006fd0 tx=0x7f9900008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.668+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99041595b0 con 0x7f99040c8b50 2023-10-06T05:24:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.922+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99041595b0 con 0x7f99040c8b50 2023-10-06T05:24:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.923+0000 7f98f97fa700 1 -- 172.21.15.33:0/4089396917 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f99041595b0 con 0x7f99040c8b50 2023-10-06T05:24:41.925 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:41.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98e8066f30 msgr2=0x7f98e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98e8066f30 0x7f98e80693e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9900006fd0 tx=0x7f9900008040 comp rx=0 tx=0).stop 2023-10-06T05:24:41.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 msgr2=0x7f9904157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 0x7f9904157f60 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f99040635d0 tx=0x7f98f4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 shutdown_connections 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98e8066f30 0x7f98e80693e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9904155af0 0x7f99041584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990406a6a0 0x7f9904146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.925+0000 7f990a59d700 1 --2- 172.21.15.33:0/4089396917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99040c8b50 0x7f9904157f60 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.926+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 >> 172.21.15.33:0/4089396917 conn(0x7f990406d030 msgr2=0x7f99040c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.926+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 shutdown_connections 2023-10-06T05:24:41.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:41.926+0000 7f990a59d700 1 -- 172.21.15.33:0/4089396917 wait complete. 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:41.945 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:41.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:41.946 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:24:41.982 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41675sec preferred_lft 41675sec 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:24:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:41.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:41.986 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:42.085 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:24:42.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:42.088 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:42.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:42.128 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:24:42.128 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:24:42.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:24:42.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:24:42.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:24:42.129 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:24:42.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:42.327 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:24:42.364 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:24:42.364 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:24:42.364 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:42.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:42.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:24:42.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:42.473 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:42.510 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:42.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:42.510 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:42.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:24:42.586 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:42.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:42.719 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:42.719 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:24:42.719 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:24:42.735 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:42.736 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:24:42.736 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:24:42.736 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:42.736 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:24:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:24:43.469 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:43.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:43.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:43.536 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:24:43.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:43.623 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:43.712 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:24:43.713 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:24:43.713 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:24:43.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:43.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:43.795 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:24:43.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:43.795 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:43.832 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:24:43.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:43.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:43.924 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:24:43.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:43.925 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:24:43.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:44.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.148+0000 7fbbbf39d700 1 Processor -- start 2023-10-06T05:24:44.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.149+0000 7fbbbf39d700 1 -- start start 2023-10-06T05:24:44.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbf39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbf39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb806c820 con 0x7fbbb8159520 2023-10-06T05:24:44.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbf39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb806a160 con 0x7fbbb80c80d0 2023-10-06T05:24:44.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbf39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c8800 con 0x7fbbb80c6340 2023-10-06T05:24:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbe39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbbb8159520 0x7fbbb80c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38182/0 (socket says 172.21.15.33:38182) 2023-10-06T05:24:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbe39b700 1 -- 172.21.15.33:0/2665021254 learned_addr learned my addr 172.21.15.33:0/2665021254 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.150+0000 7fbbbdb9a700 1 --2- 172.21.15.33:0/2665021254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126193775 0 0) 0x7fbbb806c820 con 0x7fbbb8159520 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba4003570 con 0x7fbbb8159520 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442358683 0 0) 0x7fbbb806a160 con 0x7fbbb80c80d0 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb806c820 con 0x7fbbb80c80d0 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983189950 0 0) 0x7fbba4003570 con 0x7fbbb8159520 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2665021254 >> v1:172.21.15.167:6789/0 conn(0x7fbbb80c80d0 legacy=0x7fbbb806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2665021254 >> v1:172.21.15.33:6789/0 conn(0x7fbbb8159520 legacy=0x7fbbb80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2665021254 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb80c8ae0 con 0x7fbbb80c6340 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.151+0000 7fbbbdb9a700 1 --2- 172.21.15.33:0/2665021254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80c7b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009910 tx=0x7fbba80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7ecfd6eb03790e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4001470 con 0x7fbbb80c6340 2023-10-06T05:24:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8011420 con 0x7fbbb80c6340 2023-10-06T05:24:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbd399700 1 -- 172.21.15.33:0/2665021254 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80117c0 con 0x7fbbb80c6340 2023-10-06T05:24:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2665021254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 msgr2=0x7fbbb80c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2665021254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80c7b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009910 tx=0x7fbba80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2665021254 shutdown_connections 2023-10-06T05:24:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2665021254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80c7b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2665021254 >> 172.21.15.33:0/2665021254 conn(0x7fbbb80cc930 msgr2=0x7fbbb80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2665021254 shutdown_connections 2023-10-06T05:24:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.152+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2665021254 wait complete. 2023-10-06T05:24:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.154+0000 7fbbbf39d700 1 Processor -- start 2023-10-06T05:24:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.154+0000 7fbbbf39d700 1 -- start start 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbf39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbf39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 0x7fbbb8147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbf39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb8159520 0x7fbbb80ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbf39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80abc40 con 0x7fbbb80c80d0 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbf39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80ac4e0 con 0x7fbbb8159520 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbf39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb815be60 con 0x7fbbb80c6340 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbdb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 0x7fbbb8147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbdb9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 0x7fbbb8147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53000/0 (socket says 172.21.15.33:53000) 2023-10-06T05:24:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2090839352 learned_addr learned my addr 172.21.15.33:0/2090839352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbe39b700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.155+0000 7fbbbeb9c700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb8159520 0x7fbbb80ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 msgr2=0x7fbbb80aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbdb9a700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb8159520 msgr2=0x7fbbb80ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbdb9a700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb8159520 0x7fbbb80ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbdb9a700 1 -- 172.21.15.33:0/2090839352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8009420 con 0x7fbbb80c80d0 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbdb9a700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 0x7fbbb8147e60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbba80030c0 tx=0x7fbba8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba801cc50 con 0x7fbbb80c80d0 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba801cdb0 con 0x7fbbb80c80d0 2023-10-06T05:24:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba801a6a0 con 0x7fbbb80c80d0 2023-10-06T05:24:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb80abe90 con 0x7fbbb80c80d0 2023-10-06T05:24:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.156+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb80ab1c0 con 0x7fbbb80c80d0 2023-10-06T05:24:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.157+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb80ac670 con 0x7fbbb80c80d0 2023-10-06T05:24:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.158+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbba801aa90 con 0x7fbbb80c80d0 2023-10-06T05:24:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.159+0000 7fbbae7fc700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbb9c066f30 0x7fbb9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.159+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbba80865c0 con 0x7fbbb80c80d0 2023-10-06T05:24:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.160+0000 7fbbbe39b700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbb9c066f30 0x7fbb9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.160+0000 7fbbbe39b700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbb9c066f30 0x7fbb9c0693e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40015a0 tx=0x7fbbb4000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.164+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbba8053600 con 0x7fbbb80c80d0 2023-10-06T05:24:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.407+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbb804c380 con 0x7fbbb80c80d0 2023-10-06T05:24:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.408+0000 7fbbae7fc700 1 -- 172.21.15.33:0/2090839352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7fbba8056c20 con 0x7fbbb80c80d0 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:44.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.410+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbb9c066f30 msgr2=0x7fbb9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.410+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbb9c066f30 0x7fbb9c0693e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40015a0 tx=0x7fbbb4000bc0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.410+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 msgr2=0x7fbbb8147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.410+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 0x7fbbb8147e60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbba80030c0 tx=0x7fbba8016000 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 shutdown_connections 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbb9c066f30 0x7fbb9c0693e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb8159520 0x7fbbb80ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbb80c6340 0x7fbbb80aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 --2- 172.21.15.33:0/2090839352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbb80c80d0 0x7fbbb8147e60 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 >> 172.21.15.33:0/2090839352 conn(0x7fbbb80cc930 msgr2=0x7fbbb80b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 shutdown_connections 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.411+0000 7fbbbf39d700 1 -- 172.21.15.33:0/2090839352 wait complete. 2023-10-06T05:24:44.414 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:44.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.615+0000 7f1146d9d700 1 Processor -- start 2023-10-06T05:24:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.615+0000 7f1146d9d700 1 -- start start 2023-10-06T05:24:44.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f1146d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1140159520 0x7f11400c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f1146d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c82a0 0x7f114006a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f1146d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114006c820 con 0x7f11400c6170 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f1146d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11400c6610 con 0x7f1140159520 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f1146d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114006a160 con 0x7f11400c82a0 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f114559a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11400c6170 0x7f11400c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38188/0 (socket says 172.21.15.33:38188) 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f114559a700 1 -- 172.21.15.33:0/1230611603 learned_addr learned my addr 172.21.15.33:0/1230611603 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f1145d9b700 1 --2- 172.21.15.33:0/1230611603 >> v2:172.21.15.167:3300/0 conn(0x7f1140159520 0x7f11400c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.617+0000 7f114659c700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c82a0 0x7f114006a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1144d99700 1 -- 172.21.15.33:0/1230611603 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953560756 0 0) 0x7f114006c820 con 0x7f11400c6170 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1145d9b700 1 -- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c82a0 msgr2=0x7f114006a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1145d9b700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c82a0 0x7f114006a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1145d9b700 1 -- 172.21.15.33:0/1230611603 >> v1:172.21.15.33:6789/0 conn(0x7f11400c6170 legacy=0x7f11400c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1145d9b700 1 -- 172.21.15.33:0/1230611603 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400c8800 con 0x7f1140159520 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f114659c700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c82a0 0x7f114006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1145d9b700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1140159520 0x7f11400c5c30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f113c0098e0 tx=0x7f113c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5af98595fe2e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1144d99700 1 -- 172.21.15.33:0/1230611603 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00b640 con 0x7f1140159520 2023-10-06T05:24:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1144d99700 1 -- 172.21.15.33:0/1230611603 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c00b7a0 con 0x7f1140159520 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.618+0000 7f1144d99700 1 -- 172.21.15.33:0/1230611603 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00bb40 con 0x7f1140159520 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 -- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1140159520 msgr2=0x7f11400c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1140159520 0x7f11400c5c30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f113c0098e0 tx=0x7f113c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 -- 172.21.15.33:0/1230611603 shutdown_connections 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1140159520 0x7f11400c5c30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 --2- 172.21.15.33:0/1230611603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c82a0 0x7f114006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 -- 172.21.15.33:0/1230611603 >> 172.21.15.33:0/1230611603 conn(0x7f11400cc930 msgr2=0x7f11400ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 -- 172.21.15.33:0/1230611603 shutdown_connections 2023-10-06T05:24:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.619+0000 7f1146d9d700 1 -- 172.21.15.33:0/1230611603 wait complete. 2023-10-06T05:24:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.620+0000 7f1146d9d700 1 Processor -- start 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.620+0000 7f1146d9d700 1 -- start start 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.621+0000 7f1146d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c6170 0x7f1140149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.621+0000 7f1146d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11400c82a0 0x7f1140145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.621+0000 7f1146d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 0x7f1140146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.621+0000 7f1146d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140148620 con 0x7f1140159520 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.621+0000 7f1146d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401569e0 con 0x7f11400c82a0 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.621+0000 7f1146d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1140147a10 con 0x7f11400c6170 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 0x7f1140146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 0x7f1140146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53006/0 (socket says 172.21.15.33:53006) 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 -- 172.21.15.33:0/532853799 learned_addr learned my addr 172.21.15.33:0/532853799 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f1145d9b700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c6170 0x7f1140149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114559a700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11400c82a0 0x7f1140145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 -- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c6170 msgr2=0x7f1140149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c6170 0x7f1140149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 -- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11400c82a0 msgr2=0x7f1140145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11400c82a0 0x7f1140145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 -- 172.21.15.33:0/532853799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009a20 con 0x7f1140159520 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114559a700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11400c82a0 0x7f1140145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.622+0000 7f114659c700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 0x7f1140146f70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1138009fc0 tx=0x7f113800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.623+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800cc80 con 0x7f1140159520 2023-10-06T05:24:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.623+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113800f040 con 0x7f1140159520 2023-10-06T05:24:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.623+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138014480 con 0x7f1140159520 2023-10-06T05:24:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.623+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c009420 con 0x7f1140159520 2023-10-06T05:24:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.623+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1140147fd0 con 0x7f1140159520 2023-10-06T05:24:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.624+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114004c380 con 0x7f1140159520 2023-10-06T05:24:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.625+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1138003020 con 0x7f1140159520 2023-10-06T05:24:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.625+0000 7f1135ffb700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1124066f30 0x7f11240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.626+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11380851f0 con 0x7f1140159520 2023-10-06T05:24:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.626+0000 7f1145d9b700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1124066f30 0x7f11240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.627+0000 7f1145d9b700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1124066f30 0x7f11240693e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f113c0093f0 tx=0x7f113c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.630+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1138052230 con 0x7f1140159520 2023-10-06T05:24:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.833+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11400c8ae0 con 0x7f1140159520 2023-10-06T05:24:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.834+0000 7f1135ffb700 1 -- 172.21.15.33:0/532853799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f1138055850 con 0x7f1140159520 2023-10-06T05:24:44.836 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:44.836 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":145,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:40.783691+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:23.679709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:24.472340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2321415227":"2023-10-06T06:24:39.948943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1124066f30 msgr2=0x7f11240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1124066f30 0x7f11240693e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f113c0093f0 tx=0x7f113c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 msgr2=0x7f1140146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 0x7f1140146f70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1138009fc0 tx=0x7f113800b040 comp rx=0 tx=0).stop 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 shutdown_connections 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1124066f30 0x7f11240693e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1140159520 0x7f1140146f70 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11400c6170 0x7f1140149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 --2- 172.21.15.33:0/532853799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11400c82a0 0x7f1140145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 >> 172.21.15.33:0/532853799 conn(0x7f11400cc930 msgr2=0x7f11401578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 shutdown_connections 2023-10-06T05:24:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:44.837+0000 7f1146d9d700 1 -- 172.21.15.33:0/532853799 wait complete. 2023-10-06T05:24:44.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:24:23.679709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:24:24.472340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:24:44.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.045+0000 7f196ffff700 1 Processor -- start 2023-10-06T05:24:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.045+0000 7f196ffff700 1 -- start start 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1970162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19700c7080 con 0x7f1970140960 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19700c7360 con 0x7f197006b980 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197006aff0 con 0x7f197006f230 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1970140960 0x7f19700c6870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38198/0 (socket says 172.21.15.33:38198) 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196effd700 1 -- 172.21.15.33:0/1295597125 learned_addr learned my addr 172.21.15.33:0/1295597125 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196f7fe700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1970162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.047+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270302034 0 0) 0x7f19700c7080 con 0x7f1970140960 2023-10-06T05:24:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1958003570 con 0x7f1970140960 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465746815 0 0) 0x7f19700c7360 con 0x7f197006b980 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19700c7080 con 0x7f197006b980 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530598347 0 0) 0x7f1958003570 con 0x7f1970140960 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 msgr2=0x7f1970162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1970162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 >> v1:172.21.15.167:6789/0 conn(0x7f197006b980 legacy=0x7f19701578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700c7640 con 0x7f1970140960 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1960001470 con 0x7f1970140960 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196f7fe700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1970162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 >> v1:172.21.15.33:6789/0 conn(0x7f1970140960 legacy=0x7f19700c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1958003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f1958004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1958004d50 0x7f19580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197006aff0 con 0x7f19580041b0 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195800bbf0 con 0x7f1958004d50 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195800bdb0 con 0x7f197006f230 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.048+0000 7f196e7fc700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f1958004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196effd700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1958003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196f7fe700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1958004d50 0x7f19580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196e7fc700 1 -- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 msgr2=0x7f1958003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196e7fc700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197006f230 0x7f1958003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196e7fc700 1 -- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1958004d50 msgr2=0x7f19580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196e7fc700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1958004d50 0x7f19580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196e7fc700 1 -- 172.21.15.33:0/1295597125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009a20 con 0x7f19580041b0 2023-10-06T05:24:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196f7fe700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1958004d50 0x7f19580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196e7fc700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f1958004810 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1964001a10 tx=0x7f1964007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a003c51831ac69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400c5d0 con 0x7f19580041b0 2023-10-06T05:24:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196400c730 con 0x7f19580041b0 2023-10-06T05:24:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.049+0000 7f196dffb700 1 -- 172.21.15.33:0/1295597125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400cad0 con 0x7f19580041b0 2023-10-06T05:24:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196ffff700 1 -- 172.21.15.33:0/1295597125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 msgr2=0x7f1958004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196ffff700 1 --2- 172.21.15.33:0/1295597125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f1958004810 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1964001a10 tx=0x7f1964007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196f7fe700 1 -- 172.21.15.33:0/1295597125 reap_dead start 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196ffff700 1 -- 172.21.15.33:0/1295597125 shutdown_connections 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196ffff700 1 -- 172.21.15.33:0/1295597125 >> 172.21.15.33:0/1295597125 conn(0x7f197006cfe0 msgr2=0x7f19700cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196ffff700 1 -- 172.21.15.33:0/1295597125 shutdown_connections 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.050+0000 7f196ffff700 1 -- 172.21.15.33:0/1295597125 wait complete. 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.051+0000 7f196ffff700 1 Processor -- start 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.052+0000 7f196ffff700 1 -- start start 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.052+0000 7f196ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f19700ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.052+0000 7f196ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19700b02a0 0x7f19700c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19700c5b30 0x7f19700b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700aefb0 con 0x7f19580041b0 2023-10-06T05:24:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700afe10 con 0x7f19700c5b30 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700af200 con 0x7f19700b02a0 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f19700ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f19700ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53014/0 (socket says 172.21.15.33:53014) 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 -- 172.21.15.33:0/3984853438 learned_addr learned my addr 172.21.15.33:0/3984853438 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 -- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19700b02a0 msgr2=0x7f19700c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19700b02a0 0x7f19700c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 -- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19700c5b30 msgr2=0x7f19700b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19700c5b30 0x7f19700b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 -- 172.21.15.33:0/3984853438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700af4e0 con 0x7f19580041b0 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f196effd700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f19700ae0a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f196000e870 tx=0x7f19600010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.053+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960012590 con 0x7f19580041b0 2023-10-06T05:24:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.054+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960012710 con 0x7f19580041b0 2023-10-06T05:24:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.054+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960012ad0 con 0x7f19580041b0 2023-10-06T05:24:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.054+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700af7c0 con 0x7f19580041b0 2023-10-06T05:24:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.054+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19700b15d0 con 0x7f19580041b0 2023-10-06T05:24:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.055+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19700c5f70 con 0x7f19580041b0 2023-10-06T05:24:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.055+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f196001d460 con 0x7f19580041b0 2023-10-06T05:24:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.057+0000 7f1956ffd700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f195c066f30 0x7f195c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.057+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1960084690 con 0x7f19580041b0 2023-10-06T05:24:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.057+0000 7f196e7fc700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f195c066f30 0x7f195c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.058+0000 7f196e7fc700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f195c066f30 0x7f195c0693e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f19700b1de0 tx=0x7f1964015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.061+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19600516d0 con 0x7f19580041b0 2023-10-06T05:24:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.262+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19700afc60 con 0x7f19580041b0 2023-10-06T05:24:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.263+0000 7f1956ffd700 1 -- 172.21.15.33:0/3984853438 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1960054cf0 con 0x7f19580041b0 2023-10-06T05:24:45.264 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:24:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f195c066f30 msgr2=0x7f195c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f195c066f30 0x7f195c0693e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f19700b1de0 tx=0x7f1964015000 comp rx=0 tx=0).stop 2023-10-06T05:24:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 msgr2=0x7f19700ae0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f19700ae0a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f196000e870 tx=0x7f19600010a0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 shutdown_connections 2023-10-06T05:24:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f195c066f30 0x7f195c0693e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19700c5b30 0x7f19700b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19580041b0 0x7f19700ae0a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 --2- 172.21.15.33:0/3984853438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19700b02a0 0x7f19700c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 >> 172.21.15.33:0/3984853438 conn(0x7f197006cfe0 msgr2=0x7f197006e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 shutdown_connections 2023-10-06T05:24:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.265+0000 7f196ffff700 1 -- 172.21.15.33:0/3984853438 wait complete. 2023-10-06T05:24:45.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:45.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.473+0000 7f77b519d700 1 Processor -- start 2023-10-06T05:24:45.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.473+0000 7f77b519d700 1 -- start start 2023-10-06T05:24:45.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b519d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f77b00cc7e0 0x7f77b00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b519d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b00ad430 con 0x7f77b00b01f0 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b519d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b00afe20 con 0x7f77b00cc7e0 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b006bc70 con 0x7f77b0156370 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b499c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f77b00b01f0 0x7f77b00ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38206/0 (socket says 172.21.15.33:38206) 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77b499c700 1 -- 172.21.15.33:0/3076938194 learned_addr learned my addr 172.21.15.33:0/3076938194 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77af7fe700 1 --2- 172.21.15.33:0/3076938194 >> v2:172.21.15.167:3300/0 conn(0x7f77b00cc7e0 0x7f77b00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.475+0000 7f77affff700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77aeffd700 1 -- 172.21.15.33:0/3076938194 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502075896 0 0) 0x7f77b00ad430 con 0x7f77b00b01f0 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77aeffd700 1 -- 172.21.15.33:0/3076938194 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7798003570 con 0x7f77b00b01f0 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77af7fe700 1 -- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 msgr2=0x7f77b00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77af7fe700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b00af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77af7fe700 1 -- 172.21.15.33:0/3076938194 >> v1:172.21.15.33:6789/0 conn(0x7f77b00b01f0 legacy=0x7f77b00ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77af7fe700 1 -- 172.21.15.33:0/3076938194 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b00c92f0 con 0x7f77b00cc7e0 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77affff700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:45.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77af7fe700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b00c8db0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f77a40098e0 tx=0x7f77a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1106b44eda34def4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77aeffd700 1 -- 172.21.15.33:0/3076938194 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4011630 con 0x7f77b00cc7e0 2023-10-06T05:24:45.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77aeffd700 1 -- 172.21.15.33:0/3076938194 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4011790 con 0x7f77b00cc7e0 2023-10-06T05:24:45.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.476+0000 7f77aeffd700 1 -- 172.21.15.33:0/3076938194 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4011b10 con 0x7f77b00cc7e0 2023-10-06T05:24:45.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 -- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 msgr2=0x7f77b00c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b00c8db0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f77a40098e0 tx=0x7f77a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 -- 172.21.15.33:0/3076938194 shutdown_connections 2023-10-06T05:24:45.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 --2- 172.21.15.33:0/3076938194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b00c8db0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 -- 172.21.15.33:0/3076938194 >> 172.21.15.33:0/3076938194 conn(0x7f77b006d030 msgr2=0x7f77b00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 -- 172.21.15.33:0/3076938194 shutdown_connections 2023-10-06T05:24:45.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.477+0000 7f77b519d700 1 -- 172.21.15.33:0/3076938194 wait complete. 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.478+0000 7f77b519d700 1 Processor -- start 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 -- start start 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 0x7f77b0148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b01578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b0146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0158110 con 0x7f77b00b01f0 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01583f0 con 0x7f77b00cc7e0 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.479+0000 7f77b519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01586d0 con 0x7f77b0156370 2023-10-06T05:24:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 0x7f77b0148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 0x7f77b0148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53022/0 (socket says 172.21.15.33:53022) 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 -- 172.21.15.33:0/2496098410 learned_addr learned my addr 172.21.15.33:0/2496098410 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77af7fe700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b01578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77b499c700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b0146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 -- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 msgr2=0x7f77b0146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b0146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 -- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 msgr2=0x7f77b01578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b01578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 -- 172.21.15.33:0/2496098410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009420 con 0x7f77b00b01f0 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.480+0000 7f77affff700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 0x7f77b0148110 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f77a000ec80 tx=0x7f77a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.481+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000c9d0 con 0x7f77b00b01f0 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.481+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a000cb30 con 0x7f77b00b01f0 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.481+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a4009420 con 0x7f77b00b01f0 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.481+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000ceb0 con 0x7f77b00b01f0 2023-10-06T05:24:45.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.481+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b0158c90 con 0x7f77b00b01f0 2023-10-06T05:24:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.482+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b004c380 con 0x7f77b00b01f0 2023-10-06T05:24:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.483+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77a0013680 con 0x7f77b00b01f0 2023-10-06T05:24:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.483+0000 7f7797fff700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f779c066f30 0x7f779c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.484+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f77a0084b00 con 0x7f77b00b01f0 2023-10-06T05:24:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.484+0000 7f77af7fe700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f779c066f30 0x7f779c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.485+0000 7f77af7fe700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f779c066f30 0x7f779c0693e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f77b00ccbf0 tx=0x7f77a4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.488+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77a0086e60 con 0x7f77b00b01f0 2023-10-06T05:24:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.691+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77b006f310 con 0x7f77b00b01f0 2023-10-06T05:24:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.691+0000 7f7797fff700 1 -- 172.21.15.33:0/2496098410 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v145) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77a0051b40 con 0x7f77b00b01f0 2023-10-06T05:24:45.693 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f779c066f30 msgr2=0x7f779c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f779c066f30 0x7f779c0693e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f77b00ccbf0 tx=0x7f77a4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 msgr2=0x7f77b0148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 0x7f77b0148110 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f77a000ec80 tx=0x7f77a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 shutdown_connections 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f779c066f30 0x7f779c0693e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77b0156370 0x7f77b0146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77b00b01f0 0x7f77b0148110 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 --2- 172.21.15.33:0/2496098410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77b00cc7e0 0x7f77b01578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 >> 172.21.15.33:0/2496098410 conn(0x7f77b006d030 msgr2=0x7f77b0069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 shutdown_connections 2023-10-06T05:24:45.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.694+0000 7f77b519d700 1 -- 172.21.15.33:0/2496098410 wait complete. 2023-10-06T05:24:45.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:24:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.902+0000 7f991139d700 1 Processor -- start 2023-10-06T05:24:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.902+0000 7f991139d700 1 -- start start 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f991139d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f990c159760 0x7f990c0aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f991139d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f990c159b30 0x7f990c0aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f991139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c0af6a0 0x7f990c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f991139d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c0aacf0 con 0x7f990c159b30 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f991139d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c0aaf20 con 0x7f990c159760 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f991139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0ab200 con 0x7f990c0af6a0 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f990b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f990c159b30 0x7f990c0aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f990b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f990c159b30 0x7f990c0aa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53038/0 (socket says 172.21.15.33:53038) 2023-10-06T05:24:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.903+0000 7f990b7fe700 1 -- 172.21.15.33:0/1064891025 learned_addr learned my addr 172.21.15.33:0/1064891025 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990bfff700 1 --2- 172.21.15.33:0/1064891025 >> v2:172.21.15.167:3300/0 conn(0x7f990c159760 0x7f990c0aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f9910b9c700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c0af6a0 0x7f990c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990b7fe700 1 -- 172.21.15.33:0/1064891025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c0af6a0 msgr2=0x7f990c0afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990b7fe700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c0af6a0 0x7f990c0afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990b7fe700 1 -- 172.21.15.33:0/1064891025 >> v2:172.21.15.167:3300/0 conn(0x7f990c159760 msgr2=0x7f990c0aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990b7fe700 1 --2- 172.21.15.33:0/1064891025 >> v2:172.21.15.167:3300/0 conn(0x7f990c159760 0x7f990c0aa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990b7fe700 1 -- 172.21.15.33:0/1064891025 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0ab4e0 con 0x7f990c159b30 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990bfff700 1 --2- 172.21.15.33:0/1064891025 >> v2:172.21.15.167:3300/0 conn(0x7f990c159760 0x7f990c0aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f9910b9c700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c0af6a0 0x7f990c0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990b7fe700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c159b30 0x7f990c0aa7b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f990c0af0f0 tx=0x7f9900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316aabbbdeb78e55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990affd700 1 -- 172.21.15.33:0/1064891025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000b860 con 0x7f990c159b30 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990affd700 1 -- 172.21.15.33:0/1064891025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990000b9c0 con 0x7f990c159b30 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.904+0000 7f990affd700 1 -- 172.21.15.33:0/1064891025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000bd60 con 0x7f990c159b30 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 -- 172.21.15.33:0/1064891025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c159b30 msgr2=0x7f990c0aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c159b30 0x7f990c0aa7b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f990c0af0f0 tx=0x7f9900009250 comp rx=0 tx=0).stop 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 -- 172.21.15.33:0/1064891025 shutdown_connections 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c0af6a0 0x7f990c0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 --2- 172.21.15.33:0/1064891025 >> v2:172.21.15.167:3300/0 conn(0x7f990c159760 0x7f990c0aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 --2- 172.21.15.33:0/1064891025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c159b30 0x7f990c0aa7b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 -- 172.21.15.33:0/1064891025 >> 172.21.15.33:0/1064891025 conn(0x7f990c0cd000 msgr2=0x7f990c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 -- 172.21.15.33:0/1064891025 shutdown_connections 2023-10-06T05:24:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.905+0000 7f991139d700 1 -- 172.21.15.33:0/1064891025 wait complete. 2023-10-06T05:24:45.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.906+0000 7f991139d700 1 Processor -- start 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.907+0000 7f991139d700 1 -- start start 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.907+0000 7f991139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 0x7f990c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.907+0000 7f991139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c159760 0x7f990c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.907+0000 7f991139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c159b30 0x7f990c169360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f991139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c15a710 con 0x7f990c0af6a0 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f991139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c155930 con 0x7f990c159b30 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f991139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c15c850 con 0x7f990c159760 2023-10-06T05:24:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 0x7f990c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 0x7f990c1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53044/0 (socket says 172.21.15.33:53044) 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 -- 172.21.15.33:0/3299637709 learned_addr learned my addr 172.21.15.33:0/3299637709 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 -- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c159760 msgr2=0x7f990c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c159760 0x7f990c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 -- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c159b30 msgr2=0x7f990c169360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c159b30 0x7f990c169360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 -- 172.21.15.33:0/3299637709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000094f0 con 0x7f990c0af6a0 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.908+0000 7f990bfff700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 0x7f990c1553f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00c830 tx=0x7f98fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.909+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc011600 con 0x7f990c0af6a0 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.909+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc011760 con 0x7f990c0af6a0 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.909+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc011b20 con 0x7f990c0af6a0 2023-10-06T05:24:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.909+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c15cb30 con 0x7f990c0af6a0 2023-10-06T05:24:45.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.909+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c15a3a0 con 0x7f990c0af6a0 2023-10-06T05:24:45.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.910+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c04c380 con 0x7f990c0af6a0 2023-10-06T05:24:45.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.911+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98fc00a3e0 con 0x7f990c0af6a0 2023-10-06T05:24:45.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.912+0000 7f98f3fff700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f8066f30 0x7f98f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.912+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98fc084c70 con 0x7f990c0af6a0 2023-10-06T05:24:45.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.912+0000 7f990b7fe700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f8066f30 0x7f98f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.913+0000 7f990b7fe700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f8066f30 0x7f98f80693e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f990c0af0f0 tx=0x7f9900017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:45.916+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98fc051cb0 con 0x7f990c0af6a0 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.119+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f990c06e080 con 0x7f990c0af6a0 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.119+0000 7f98f3fff700 1 -- 172.21.15.33:0/3299637709 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98fc015020 con 0x7f990c0af6a0 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:24:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.121+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f8066f30 msgr2=0x7f98f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.121+0000 7f991139d700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f8066f30 0x7f98f80693e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f990c0af0f0 tx=0x7f9900017000 comp rx=0 tx=0).stop 2023-10-06T05:24:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.121+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 msgr2=0x7f990c1553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.121+0000 7f991139d700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 0x7f990c1553f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00c830 tx=0x7f98fc008040 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 shutdown_connections 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98f8066f30 0x7f98f80693e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c159b30 0x7f990c169360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f990c0af6a0 0x7f990c1553f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 --2- 172.21.15.33:0/3299637709 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990c159760 0x7f990c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 >> 172.21.15.33:0/3299637709 conn(0x7f990c0cd000 msgr2=0x7f990c155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 shutdown_connections 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.122+0000 7f991139d700 1 -- 172.21.15.33:0/3299637709 wait complete. 2023-10-06T05:24:46.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:24:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.329+0000 7f8e26b9d700 1 Processor -- start 2023-10-06T05:24:46.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.329+0000 7f8e26b9d700 1 -- start start 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.330+0000 7f8e26b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8e200af540 0x7f8e200af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.330+0000 7f8e26b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 0x7f8e2015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.330+0000 7f8e26b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2015d660 con 0x7f8e2015a650 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.330+0000 7f8e26b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e200ce890 con 0x7f8e200af540 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.330+0000 7f8e26b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2015aac0 con 0x7f8e200cd5e0 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e2539a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8e2015a650 0x7f8e200cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38216/0 (socket says 172.21.15.33:38216) 2023-10-06T05:24:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e2539a700 1 -- 172.21.15.33:0/2531683028 learned_addr learned my addr 172.21.15.33:0/2531683028 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2531683028 >> v2:172.21.15.167:3300/0 conn(0x7f8e200af540 0x7f8e200af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e2639c700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 0x7f8e2015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023363269 0 0) 0x7f8e2015d660 con 0x7f8e2015a650 2023-10-06T05:24:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e0c003570 con 0x7f8e2015a650 2023-10-06T05:24:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 392429622 0 0) 0x7f8e0c003570 con 0x7f8e2015a650 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 msgr2=0x7f8e2015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 0x7f8e2015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 >> v2:172.21.15.167:3300/0 conn(0x7f8e200af540 msgr2=0x7f8e200af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 --2- 172.21.15.33:0/2531683028 >> v2:172.21.15.167:3300/0 conn(0x7f8e200af540 0x7f8e200af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2015d890 con 0x7f8e2015a650 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f8e10001470 con 0x7f8e2015a650 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 >> v1:172.21.15.33:6789/0 conn(0x7f8e2015a650 legacy=0x7f8e200cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 0x7f8e0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e0c004d50 0x7f8e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200ce890 con 0x7f8e0c003a60 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.331+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2015aac0 con 0x7f8e0c004d50 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c00ba60 con 0x7f8e200cd5e0 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e2539a700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 0x7f8e0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e2639c700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e0c004d50 0x7f8e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 msgr2=0x7f8e0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200cd5e0 0x7f8e0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e0c004d50 msgr2=0x7f8e0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e0c004d50 0x7f8e0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2531683028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c003570 con 0x7f8e0c003a60 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.332+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e0c003e70 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009c00 tx=0x7f8e1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551d0cedf804e4b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00e6b0 con 0x7f8e0c003a60 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c00e810 con 0x7f8e0c003a60 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e24b99700 1 -- 172.21.15.33:0/2531683028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00ebd0 con 0x7f8e0c003a60 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2531683028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 msgr2=0x7f8e0c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2531683028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e0c003e70 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009c00 tx=0x7f8e1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e2639c700 1 -- 172.21.15.33:0/2531683028 reap_dead start 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2531683028 shutdown_connections 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2531683028 >> 172.21.15.33:0/2531683028 conn(0x7f8e2006d9b0 msgr2=0x7f8e2006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2531683028 shutdown_connections 2023-10-06T05:24:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.333+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2531683028 wait complete. 2023-10-06T05:24:46.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.335+0000 7f8e26b9d700 1 Processor -- start 2023-10-06T05:24:46.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.335+0000 7f8e26b9d700 1 -- start start 2023-10-06T05:24:46.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e26b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e20156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e26b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e200b5020 0x7f8e20157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e26b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200c8b10 0x7f8e200c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e26b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200ca490 con 0x7f8e0c003a60 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e26b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200c9880 con 0x7f8e200b5020 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e26b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200c9b60 con 0x7f8e200c8b10 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e25b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e20156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e25b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e20156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53066/0 (socket says 172.21.15.33:53066) 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2646846066 learned_addr learned my addr 172.21.15.33:0/2646846066 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e2539a700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e200b5020 0x7f8e20157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e2639c700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200c8b10 0x7f8e200c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.336+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200c8b10 msgr2=0x7f8e200c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200c8b10 0x7f8e200c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e200b5020 msgr2=0x7f8e20157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e200b5020 0x7f8e20157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e25b9b700 1 -- 172.21.15.33:0/2646846066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e200c9e40 con 0x7f8e0c003a60 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e25b9b700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e20156d20 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00e4b0 tx=0x7f8e1c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c022070 con 0x7f8e0c003a60 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c00e980 con 0x7f8e0c003a60 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e200ca120 con 0x7f8e0c003a60 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c027470 con 0x7f8e0c003a60 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.337+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e200cec20 con 0x7f8e0c003a60 2023-10-06T05:24:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.338+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e200ca770 con 0x7f8e0c003a60 2023-10-06T05:24:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.339+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e1c01c450 con 0x7f8e0c003a60 2023-10-06T05:24:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.340+0000 7f8e15ffb700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e04066f30 0x7f8e040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.340+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e1c086300 con 0x7f8e0c003a60 2023-10-06T05:24:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.341+0000 7f8e2539a700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e04066f30 0x7f8e040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.341+0000 7f8e2539a700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e04066f30 0x7f8e040693e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e10000e10 tx=0x7f8e10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.344+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e1c0533c0 con 0x7f8e0c003a60 2023-10-06T05:24:46.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.547+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e2004c380 con 0x7f8e0c003a60 2023-10-06T05:24:46.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.548+0000 7f8e15ffb700 1 -- 172.21.15.33:0/2646846066 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e1c056960 con 0x7f8e0c003a60 2023-10-06T05:24:46.550 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:24:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e04066f30 msgr2=0x7f8e040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e04066f30 0x7f8e040693e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e10000e10 tx=0x7f8e10009040 comp rx=0 tx=0).stop 2023-10-06T05:24:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 msgr2=0x7f8e20156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e20156d20 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00e4b0 tx=0x7f8e1c01a000 comp rx=0 tx=0).stop 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 shutdown_connections 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e04066f30 0x7f8e040693e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e200c8b10 0x7f8e200c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c003a60 0x7f8e20156d20 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 --2- 172.21.15.33:0/2646846066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e200b5020 0x7f8e20157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 >> 172.21.15.33:0/2646846066 conn(0x7f8e2006d9b0 msgr2=0x7f8e2015d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 shutdown_connections 2023-10-06T05:24:46.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.550+0000 7f8e26b9d700 1 -- 172.21.15.33:0/2646846066 wait complete. 2023-10-06T05:24:46.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.753+0000 7fd4d659d700 1 Processor -- start 2023-10-06T05:24:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.754+0000 7fd4d659d700 1 -- start start 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d659d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd4d0158700 0x7fd4d00adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d659d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd4d00699c0 0x7fd4d0069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d0158ba0 con 0x7fd4d0155140 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d659d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00c8e40 con 0x7fd4d00699c0 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d659d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d006a340 con 0x7fd4d0158700 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d559b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd4d0155140 0x7fd4d00c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38224/0 (socket says 172.21.15.33:38224) 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d559b700 1 -- 172.21.15.33:0/126202371 learned_addr learned my addr 172.21.15.33:0/126202371 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d5d9c700 1 --2- 172.21.15.33:0/126202371 >> v2:172.21.15.33:3300/0 conn(0x7fd4d00699c0 0x7fd4d0069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.755+0000 7fd4d4d9a700 1 --2- 172.21.15.33:0/126202371 >> v2:172.21.15.167:3300/0 conn(0x7fd4d0158700 0x7fd4d00adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4c7fff700 1 -- 172.21.15.33:0/126202371 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871099565 0 0) 0x7fd4d0158ba0 con 0x7fd4d0155140 2023-10-06T05:24:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4c7fff700 1 -- 172.21.15.33:0/126202371 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4bc003570 con 0x7fd4d0155140 2023-10-06T05:24:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4d5d9c700 1 -- 172.21.15.33:0/126202371 >> v2:172.21.15.167:3300/0 conn(0x7fd4d0158700 msgr2=0x7fd4d00adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4d5d9c700 1 --2- 172.21.15.33:0/126202371 >> v2:172.21.15.167:3300/0 conn(0x7fd4d0158700 0x7fd4d00adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4d5d9c700 1 -- 172.21.15.33:0/126202371 >> v1:172.21.15.33:6789/0 conn(0x7fd4d0155140 legacy=0x7fd4d00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4d5d9c700 1 -- 172.21.15.33:0/126202371 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00ae490 con 0x7fd4d00699c0 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4d5d9c700 1 --2- 172.21.15.33:0/126202371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d00699c0 0x7fd4d0069e00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e60 tx=0x7fd4c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c901ff2020247dec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4c7fff700 1 -- 172.21.15.33:0/126202371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800e070 con 0x7fd4d00699c0 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4c7fff700 1 -- 172.21.15.33:0/126202371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c800ba70 con 0x7fd4d00699c0 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.756+0000 7fd4c7fff700 1 -- 172.21.15.33:0/126202371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800be30 con 0x7fd4d00699c0 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 -- 172.21.15.33:0/126202371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d00699c0 msgr2=0x7fd4d0069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 --2- 172.21.15.33:0/126202371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d00699c0 0x7fd4d0069e00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e60 tx=0x7fd4c8009780 comp rx=0 tx=0).stop 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 -- 172.21.15.33:0/126202371 shutdown_connections 2023-10-06T05:24:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 --2- 172.21.15.33:0/126202371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d00699c0 0x7fd4d0069e00 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 --2- 172.21.15.33:0/126202371 >> v2:172.21.15.167:3300/0 conn(0x7fd4d0158700 0x7fd4d00adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 -- 172.21.15.33:0/126202371 >> 172.21.15.33:0/126202371 conn(0x7fd4d006d030 msgr2=0x7fd4d006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 -- 172.21.15.33:0/126202371 shutdown_connections 2023-10-06T05:24:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.757+0000 7fd4d659d700 1 -- 172.21.15.33:0/126202371 wait complete. 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.758+0000 7fd4d659d700 1 Processor -- start 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.759+0000 7fd4d659d700 1 -- start start 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.759+0000 7fd4d659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4d00699c0 0x7fd4d00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 0x7fd4d01578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4d0158700 0x7fd4d0153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00af6a0 con 0x7fd4d0155140 2023-10-06T05:24:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0158160 con 0x7fd4d0158700 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0158440 con 0x7fd4d00699c0 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 0x7fd4d01578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 0x7fd4d01578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53072/0 (socket says 172.21.15.33:53072) 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 -- 172.21.15.33:0/812528214 learned_addr learned my addr 172.21.15.33:0/812528214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d559b700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4d00699c0 0x7fd4d00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d5d9c700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4d0158700 0x7fd4d0153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 -- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4d00699c0 msgr2=0x7fd4d00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4d00699c0 0x7fd4d00b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 -- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4d0158700 msgr2=0x7fd4d0153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4d0158700 0x7fd4d0153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d4d9a700 1 -- 172.21.15.33:0/812528214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c8009a20 con 0x7fd4d0155140 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.760+0000 7fd4d5d9c700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4d0158700 0x7fd4d0153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.761+0000 7fd4d4d9a700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 0x7fd4d01578e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000e930 tx=0x7fd4c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.761+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0013680 con 0x7fd4d0155140 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.761+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c00137e0 con 0x7fd4d0155140 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.761+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d00af050 con 0x7fd4d0155140 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.761+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0013b80 con 0x7fd4d0155140 2023-10-06T05:24:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.761+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d0157e20 con 0x7fd4d0155140 2023-10-06T05:24:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.762+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d004c380 con 0x7fd4d0155140 2023-10-06T05:24:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.763+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4c000c460 con 0x7fd4d0155140 2023-10-06T05:24:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.764+0000 7fd4c57fa700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4b4066f30 0x7fd4b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.764+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4c00848d0 con 0x7fd4d0155140 2023-10-06T05:24:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.764+0000 7fd4d559b700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4b4066f30 0x7fd4b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.765+0000 7fd4d559b700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4b4066f30 0x7fd4b40693e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc007fd0 tx=0x7fd4cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:46.768+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4c0051910 con 0x7fd4d0155140 2023-10-06T05:24:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.018+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4d006e740 con 0x7fd4d0155140 2023-10-06T05:24:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.019+0000 7fd4c57fa700 1 -- 172.21.15.33:0/812528214 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7fd4c0054f30 con 0x7fd4d0155140 2023-10-06T05:24:47.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:47.020 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4b4066f30 msgr2=0x7fd4b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4b4066f30 0x7fd4b40693e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc007fd0 tx=0x7fd4cc009040 comp rx=0 tx=0).stop 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 msgr2=0x7fd4d01578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 0x7fd4d01578e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000e930 tx=0x7fd4c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 shutdown_connections 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd4b4066f30 0x7fd4b40693e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4d0158700 0x7fd4d0153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd4d00699c0 0x7fd4d00b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 --2- 172.21.15.33:0/812528214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0155140 0x7fd4d01578e0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 >> 172.21.15.33:0/812528214 conn(0x7fd4d006d030 msgr2=0x7fd4d015bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 shutdown_connections 2023-10-06T05:24:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.021+0000 7fd4d659d700 1 -- 172.21.15.33:0/812528214 wait complete. 2023-10-06T05:24:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:47.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.227+0000 7f5bc7fff700 1 Processor -- start 2023-10-06T05:24:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.227+0000 7f5bc7fff700 1 -- start start 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.228+0000 7f5bc7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5bc80ccfc0 0x7f5bc80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.228+0000 7f5bc7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5bc80cf3b0 0x7f5bc81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.228+0000 7f5bc7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc80cbd80 con 0x7f5bc80cc750 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.228+0000 7f5bc7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80af5f0 con 0x7f5bc80cf3b0 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.228+0000 7f5bc7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80af8d0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.228+0000 7f5bc67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5bc80cf3b0 0x7f5bc81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5bc80cf3b0 0x7f5bc81594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53088/0 (socket says 172.21.15.33:53088) 2023-10-06T05:24:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 -- 172.21.15.33:0/812706725 learned_addr learned my addr 172.21.15.33:0/812706725 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc6ffd700 1 --2- 172.21.15.33:0/812706725 >> v2:172.21.15.167:3300/0 conn(0x7f5bc80ccfc0 0x7f5bc80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 -- 172.21.15.33:0/812706725 >> v2:172.21.15.167:3300/0 conn(0x7f5bc80ccfc0 msgr2=0x7f5bc80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 --2- 172.21.15.33:0/812706725 >> v2:172.21.15.167:3300/0 conn(0x7f5bc80ccfc0 0x7f5bc80cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 -- 172.21.15.33:0/812706725 >> v1:172.21.15.167:6789/0 conn(0x7f5bc80cc750 legacy=0x7f5bc80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 -- 172.21.15.33:0/812706725 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80afbb0 con 0x7f5bc80cf3b0 2023-10-06T05:24:47.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.229+0000 7f5bc67fc700 1 --2- 172.21.15.33:0/812706725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80cf3b0 0x7f5bc81594a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0099b0 tx=0x7f5bbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf59796e190376e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc5ffb700 1 -- 172.21.15.33:0/812706725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00b820 con 0x7f5bc80cf3b0 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc5ffb700 1 -- 172.21.15.33:0/812706725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc00b980 con 0x7f5bc80cf3b0 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc5ffb700 1 -- 172.21.15.33:0/812706725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00bd20 con 0x7f5bc80cf3b0 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 -- 172.21.15.33:0/812706725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80cf3b0 msgr2=0x7f5bc81594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/812706725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80cf3b0 0x7f5bc81594a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0099b0 tx=0x7f5bbc009250 comp rx=0 tx=0).stop 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 -- 172.21.15.33:0/812706725 shutdown_connections 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/812706725 >> v2:172.21.15.167:3300/0 conn(0x7f5bc80ccfc0 0x7f5bc80cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/812706725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80cf3b0 0x7f5bc81594a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 -- 172.21.15.33:0/812706725 >> 172.21.15.33:0/812706725 conn(0x7f5bc806cff0 msgr2=0x7f5bc806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 -- 172.21.15.33:0/812706725 shutdown_connections 2023-10-06T05:24:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.230+0000 7f5bc7fff700 1 -- 172.21.15.33:0/812706725 wait complete. 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.232+0000 7f5bc7fff700 1 Processor -- start 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.232+0000 7f5bc7fff700 1 -- start start 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc80cc750 0x7f5bc81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 0x7f5bc80b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bc80b5d50 0x7f5bc814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8171ba0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8172720 con 0x7f5bc80cc750 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80c8a80 con 0x7f5bc80b5d50 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 0x7f5bc80b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 0x7f5bc80b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53100/0 (socket says 172.21.15.33:53100) 2023-10-06T05:24:47.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc67fc700 1 -- 172.21.15.33:0/2657816053 learned_addr learned my addr 172.21.15.33:0/2657816053 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:47.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc6ffd700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc80cc750 0x7f5bc81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.233+0000 7f5bc77fe700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bc80b5d50 0x7f5bc814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc67fc700 1 -- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bc80b5d50 msgr2=0x7f5bc814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc67fc700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bc80b5d50 0x7f5bc814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc67fc700 1 -- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc80cc750 msgr2=0x7f5bc81570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc67fc700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc80cc750 0x7f5bc81570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc67fc700 1 -- 172.21.15.33:0/2657816053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0094f0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc67fc700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 0x7f5bc80b5810 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5bc806be20 tx=0x7f5bbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00bd00 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc024440 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0247e0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8171df0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.234+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc81723b0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.235+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc80d0020 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.236+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bbc02d440 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.237+0000 7f5baeffd700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bb4066f30 0x7f5bb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.237+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5bbc085fd0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.238+0000 7f5bc6ffd700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bb4066f30 0x7f5bb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.239+0000 7f5bc6ffd700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bb4066f30 0x7f5bb40693e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006fd0 tx=0x7f5bb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.241+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bbc053010 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.444+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bc80af0a0 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.445+0000 7f5baeffd700 1 -- 172.21.15.33:0/2657816053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f5bbc056630 con 0x7f5bc80ccfc0 2023-10-06T05:24:47.447 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:47.447 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":145,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:40.783691+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:23.679709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:24.472340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2321415227":"2023-10-06T06:24:39.948943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bb4066f30 msgr2=0x7f5bb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bb4066f30 0x7f5bb40693e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006fd0 tx=0x7f5bb8008040 comp rx=0 tx=0).stop 2023-10-06T05:24:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 msgr2=0x7f5bc80b5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 0x7f5bc80b5810 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5bc806be20 tx=0x7f5bbc017000 comp rx=0 tx=0).stop 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 shutdown_connections 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bb4066f30 0x7f5bb40693e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bc80b5d50 0x7f5bc814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bc80cc750 0x7f5bc81570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 --2- 172.21.15.33:0/2657816053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bc80ccfc0 0x7f5bc80b5810 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 >> 172.21.15.33:0/2657816053 conn(0x7f5bc806cff0 msgr2=0x7f5bc814fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.447+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 shutdown_connections 2023-10-06T05:24:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.448+0000 7f5bc7fff700 1 -- 172.21.15.33:0/2657816053 wait complete. 2023-10-06T05:24:47.467 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:24:47.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:47.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.653+0000 7f817719d700 1 Processor -- start 2023-10-06T05:24:47.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.654+0000 7f817719d700 1 -- start start 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8170159530 0x7f817006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817006e3e0 con 0x7f81700b0310 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81700b1e00 con 0x7f8170159530 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817006f6b0 con 0x7f817006df70 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817699c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81700b0310 0x7f8170159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38228/0 (socket says 172.21.15.33:38228) 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817699c700 1 -- 172.21.15.33:0/2187975165 learned_addr learned my addr 172.21.15.33:0/2187975165 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817599a700 1 --2- 172.21.15.33:0/2187975165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.655+0000 7f817619b700 1 --2- 172.21.15.33:0/2187975165 >> v2:172.21.15.167:3300/0 conn(0x7f8170159530 0x7f817006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f8175199700 1 -- 172.21.15.33:0/2187975165 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628465436 0 0) 0x7f817006e3e0 con 0x7f81700b0310 2023-10-06T05:24:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f8175199700 1 -- 172.21.15.33:0/2187975165 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815c003570 con 0x7f81700b0310 2023-10-06T05:24:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f817599a700 1 -- 172.21.15.33:0/2187975165 >> v2:172.21.15.167:3300/0 conn(0x7f8170159530 msgr2=0x7f817006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f817599a700 1 --2- 172.21.15.33:0/2187975165 >> v2:172.21.15.167:3300/0 conn(0x7f8170159530 0x7f817006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f817599a700 1 -- 172.21.15.33:0/2187975165 >> v1:172.21.15.33:6789/0 conn(0x7f81700b0310 legacy=0x7f8170159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f817599a700 1 -- 172.21.15.33:0/2187975165 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817006f220 con 0x7f817006df70 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.656+0000 7f817599a700 1 --2- 172.21.15.33:0/2187975165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700afdd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f81600098e0 tx=0x7f81600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9565e263d24f01b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f8175199700 1 -- 172.21.15.33:0/2187975165 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160011630 con 0x7f817006df70 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f8175199700 1 -- 172.21.15.33:0/2187975165 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160011790 con 0x7f817006df70 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f8175199700 1 -- 172.21.15.33:0/2187975165 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160011b30 con 0x7f817006df70 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 -- 172.21.15.33:0/2187975165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 msgr2=0x7f81700afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 --2- 172.21.15.33:0/2187975165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700afdd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f81600098e0 tx=0x7f81600091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 -- 172.21.15.33:0/2187975165 shutdown_connections 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 --2- 172.21.15.33:0/2187975165 >> v2:172.21.15.167:3300/0 conn(0x7f8170159530 0x7f817006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 --2- 172.21.15.33:0/2187975165 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700afdd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 -- 172.21.15.33:0/2187975165 >> 172.21.15.33:0/2187975165 conn(0x7f817006c560 msgr2=0x7f817006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 -- 172.21.15.33:0/2187975165 shutdown_connections 2023-10-06T05:24:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.657+0000 7f817719d700 1 -- 172.21.15.33:0/2187975165 wait complete. 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.659+0000 7f817719d700 1 Processor -- start 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.659+0000 7f817719d700 1 -- start start 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 0x7f81700c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8170159530 0x7f81700cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81700c9f00 con 0x7f81700b0310 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81700ccb10 con 0x7f8170159530 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81700cbf00 con 0x7f817006df70 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 0x7f81700c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 0x7f81700c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53116/0 (socket says 172.21.15.33:53116) 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 -- 172.21.15.33:0/2119312320 learned_addr learned my addr 172.21.15.33:0/2119312320 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817699c700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8170159530 0x7f81700cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817619b700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 -- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 msgr2=0x7f81700cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 -- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8170159530 msgr2=0x7f81700cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.660+0000 7f817599a700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8170159530 0x7f81700cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f817599a700 1 -- 172.21.15.33:0/2119312320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160009420 con 0x7f81700b0310 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f817599a700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 0x7f81700c99c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f81700719a0 tx=0x7f8160017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160000d10 con 0x7f81700b0310 2023-10-06T05:24:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160011b10 con 0x7f81700b0310 2023-10-06T05:24:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81700cc1e0 con 0x7f81700b0310 2023-10-06T05:24:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160025660 con 0x7f81700b0310 2023-10-06T05:24:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.661+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81700cc7a0 con 0x7f81700b0310 2023-10-06T05:24:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.662+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817004c380 con 0x7f81700b0310 2023-10-06T05:24:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.663+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8160025a50 con 0x7f81700b0310 2023-10-06T05:24:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.664+0000 7f81667fc700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8154066f30 0x7f81540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.664+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8160085ec0 con 0x7f81700b0310 2023-10-06T05:24:47.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.665+0000 7f817619b700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8154066f30 0x7f81540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.665+0000 7f817619b700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8154066f30 0x7f81540693e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f816c009d20 tx=0x7f816c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.668+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8160052f00 con 0x7f81700b0310 2023-10-06T05:24:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.914+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81700b3a80 con 0x7f81700b0310 2023-10-06T05:24:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.914+0000 7f81667fc700 1 -- 172.21.15.33:0/2119312320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7f8160056520 con 0x7f81700b0310 2023-10-06T05:24:47.916 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:47.916 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8154066f30 msgr2=0x7f81540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8154066f30 0x7f81540693e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f816c009d20 tx=0x7f816c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 msgr2=0x7f81700c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 0x7f81700c99c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f81700719a0 tx=0x7f8160017000 comp rx=0 tx=0).stop 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 shutdown_connections 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8154066f30 0x7f81540693e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8170159530 0x7f81700cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f817006df70 0x7f81700cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 --2- 172.21.15.33:0/2119312320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81700b0310 0x7f81700c99c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 >> 172.21.15.33:0/2119312320 conn(0x7f817006c560 msgr2=0x7f81700ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 shutdown_connections 2023-10-06T05:24:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:47.917+0000 7f817719d700 1 -- 172.21.15.33:0/2119312320 wait complete. 2023-10-06T05:24:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:47.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:48.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.122+0000 7f57caf9d700 1 Processor -- start 2023-10-06T05:24:48.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.122+0000 7f57caf9d700 1 -- start start 2023-10-06T05:24:48.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57caf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f57c40cb710 0x7f57c406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57caf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c40c9cb0 con 0x7f57c40c8050 2023-10-06T05:24:48.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57caf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c40ca270 con 0x7f57c413faa0 2023-10-06T05:24:48.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57caf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c406b840 con 0x7f57c40cb710 2023-10-06T05:24:48.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57c979a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f57c40c8050 0x7f57c406aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38236/0 (socket says 172.21.15.33:38236) 2023-10-06T05:24:48.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57c979a700 1 -- 172.21.15.33:0/593994465 learned_addr learned my addr 172.21.15.33:0/593994465 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57ca79c700 1 --2- 172.21.15.33:0/593994465 >> v2:172.21.15.167:3300/0 conn(0x7f57c40cb710 0x7f57c406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.124+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190015297 0 0) 0x7f57c40c9cb0 con 0x7f57c40c8050 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b0003570 con 0x7f57c40c8050 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258447525 0 0) 0x7f57b0003570 con 0x7f57c40c8050 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 >> v2:172.21.15.167:3300/0 conn(0x7f57c40cb710 msgr2=0x7f57c406b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 --2- 172.21.15.33:0/593994465 >> v2:172.21.15.167:3300/0 conn(0x7f57c40cb710 0x7f57c406b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 >> v1:172.21.15.167:6789/0 conn(0x7f57c413faa0 legacy=0x7f57c40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c406bb20 con 0x7f57c40c8050 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f57b4001470 con 0x7f57c40c8050 2023-10-06T05:24:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 >> v1:172.21.15.33:6789/0 conn(0x7f57c40c8050 legacy=0x7f57c406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57b0003a60 0x7f57b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57b0004810 0x7f57b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c406b840 con 0x7f57b00051c0 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b000c0b0 con 0x7f57b0003a60 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b000c270 con 0x7f57b0004810 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57ca79c700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c9f9b700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57b0003a60 0x7f57b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.125+0000 7f57c979a700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57b0004810 0x7f57b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 -- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57b0004810 msgr2=0x7f57b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57b0004810 0x7f57b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 -- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57b0003a60 msgr2=0x7f57b0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57b0003a60 0x7f57b0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 -- 172.21.15.33:0/593994465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0003570 con 0x7f57b00051c0 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 -- 172.21.15.33:0/593994465 reap_dead start 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57ca79c700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57b000ba60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009fc0 tx=0x7f57bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8b799cf5904221 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00ea90 con 0x7f57b00051c0 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00f040 con 0x7f57b00051c0 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.126+0000 7f57c8f99700 1 -- 172.21.15.33:0/593994465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00ee40 con 0x7f57b00051c0 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 -- 172.21.15.33:0/593994465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 msgr2=0x7f57b000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57b000ba60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009fc0 tx=0x7f57bc00b040 comp rx=0 tx=0).stop 2023-10-06T05:24:48.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 -- 172.21.15.33:0/593994465 shutdown_connections 2023-10-06T05:24:48.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 --2- 172.21.15.33:0/593994465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57b000ba60 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 -- 172.21.15.33:0/593994465 >> 172.21.15.33:0/593994465 conn(0x7f57c406d030 msgr2=0x7f57c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 -- 172.21.15.33:0/593994465 shutdown_connections 2023-10-06T05:24:48.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.127+0000 7f57caf9d700 1 -- 172.21.15.33:0/593994465 wait complete. 2023-10-06T05:24:48.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.128+0000 7f57caf9d700 1 Processor -- start 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.128+0000 7f57caf9d700 1 -- start start 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57caf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57c40c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57caf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c40c3190 0x7f57c416a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57caf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57c40c5320 0x7f57c413e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57caf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c416a970 con 0x7f57b00051c0 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57caf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40c3690 con 0x7f57c40c3190 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57caf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40c3970 con 0x7f57c40c5320 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57c9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57c40c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57c9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57c40c4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53126/0 (socket says 172.21.15.33:53126) 2023-10-06T05:24:48.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.129+0000 7f57c9f9b700 1 -- 172.21.15.33:0/1662821403 learned_addr learned my addr 172.21.15.33:0/1662821403 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57c9f9b700 1 -- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57c40c5320 msgr2=0x7f57c413e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57c9f9b700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57c40c5320 0x7f57c413e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57c9f9b700 1 -- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c40c3190 msgr2=0x7f57c416a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57c9f9b700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c40c3190 0x7f57c416a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57c9f9b700 1 -- 172.21.15.33:0/1662821403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40c3c50 con 0x7f57b00051c0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57c9f9b700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57c40c4730 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f57c0002b00 tx=0x7f57c0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000f9a0 con 0x7f57b00051c0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c000fb40 con 0x7f57b00051c0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c40c3f30 con 0x7f57b00051c0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.130+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00133f0 con 0x7f57b00051c0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.131+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c4158e40 con 0x7f57b00051c0 2023-10-06T05:24:48.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.131+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c40c4450 con 0x7f57b00051c0 2023-10-06T05:24:48.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.132+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57c0013760 con 0x7f57b00051c0 2023-10-06T05:24:48.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.133+0000 7f57b9ffb700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f57a8066f30 0x7f57a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.133+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57c0084820 con 0x7f57b00051c0 2023-10-06T05:24:48.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.134+0000 7f57c979a700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f57a8066f30 0x7f57a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.134+0000 7f57c979a700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f57a8066f30 0x7f57a80693e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001200 tx=0x7f57c40ca1e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.137+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57c40c4450 con 0x7f57b00051c0 2023-10-06T05:24:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.383+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57c40c4450 con 0x7f57b00051c0 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.384+0000 7f57b9ffb700 1 -- 172.21.15.33:0/1662821403 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4612 (secure 0 0 0) 0x7f57c40c4450 con 0x7f57b00051c0 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:25.495496+0000","modified":"2023-10-06T05:24:29.025589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1921586494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1921586494},{"type":"v1","addr":"172.21.15.33:6835","nonce":1921586494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f57a8066f30 msgr2=0x7f57a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f57a8066f30 0x7f57a80693e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001200 tx=0x7f57c40ca1e0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 msgr2=0x7f57c40c4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57c40c4730 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f57c0002b00 tx=0x7f57c0007d10 comp rx=0 tx=0).stop 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 shutdown_connections 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f57a8066f30 0x7f57a80693e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57c40c5320 0x7f57c413e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b00051c0 0x7f57c40c4730 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 --2- 172.21.15.33:0/1662821403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c40c3190 0x7f57c416a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 >> 172.21.15.33:0/1662821403 conn(0x7f57c406d030 msgr2=0x7f57c4159450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 shutdown_connections 2023-10-06T05:24:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.386+0000 7f57caf9d700 1 -- 172.21.15.33:0/1662821403 wait complete. 2023-10-06T05:24:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 105 2023-10-06T05:24:48.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:48.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.590+0000 7f38ba99d700 1 Processor -- start 2023-10-06T05:24:48.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.591+0000 7f38ba99d700 1 -- start start 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38ba99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f38b4155af0 0x7f38b406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38ba99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38b406a6a0 0x7f38b40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38ba99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b406ab60 con 0x7f38b40c8b50 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38ba99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b406b7e0 con 0x7f38b406a6a0 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38ba99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b406ae40 con 0x7f38b4155af0 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38b919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38b406a6a0 0x7f38b40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38b919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38b406a6a0 0x7f38b40c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53132/0 (socket says 172.21.15.33:53132) 2023-10-06T05:24:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.592+0000 7f38b919a700 1 -- 172.21.15.33:0/619590899 learned_addr learned my addr 172.21.15.33:0/619590899 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b999b700 1 --2- 172.21.15.33:0/619590899 >> v2:172.21.15.167:3300/0 conn(0x7f38b4155af0 0x7f38b406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b919a700 1 -- 172.21.15.33:0/619590899 >> v2:172.21.15.167:3300/0 conn(0x7f38b4155af0 msgr2=0x7f38b406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b919a700 1 --2- 172.21.15.33:0/619590899 >> v2:172.21.15.167:3300/0 conn(0x7f38b4155af0 0x7f38b406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b919a700 1 -- 172.21.15.33:0/619590899 >> v1:172.21.15.167:6789/0 conn(0x7f38b40c8b50 legacy=0x7f38b41598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b919a700 1 -- 172.21.15.33:0/619590899 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b406b120 con 0x7f38b406a6a0 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b919a700 1 --2- 172.21.15.33:0/619590899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b406a6a0 0x7f38b40c7210 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f38b40697c0 tx=0x7f38a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a437344bbae92c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b8999700 1 -- 172.21.15.33:0/619590899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a400b820 con 0x7f38b406a6a0 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b8999700 1 -- 172.21.15.33:0/619590899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a400b980 con 0x7f38b406a6a0 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.593+0000 7f38b8999700 1 -- 172.21.15.33:0/619590899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a400bd20 con 0x7f38b406a6a0 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 -- 172.21.15.33:0/619590899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b406a6a0 msgr2=0x7f38b40c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 --2- 172.21.15.33:0/619590899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b406a6a0 0x7f38b40c7210 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f38b40697c0 tx=0x7f38a4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 -- 172.21.15.33:0/619590899 shutdown_connections 2023-10-06T05:24:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 --2- 172.21.15.33:0/619590899 >> v2:172.21.15.167:3300/0 conn(0x7f38b4155af0 0x7f38b406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 --2- 172.21.15.33:0/619590899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b406a6a0 0x7f38b40c7210 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 -- 172.21.15.33:0/619590899 >> 172.21.15.33:0/619590899 conn(0x7f38b406d030 msgr2=0x7f38b406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 -- 172.21.15.33:0/619590899 shutdown_connections 2023-10-06T05:24:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.594+0000 7f38ba99d700 1 -- 172.21.15.33:0/619590899 wait complete. 2023-10-06T05:24:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.595+0000 7f38ba99d700 1 Processor -- start 2023-10-06T05:24:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.595+0000 7f38ba99d700 1 -- start start 2023-10-06T05:24:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.596+0000 7f38ba99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38b406a6a0 0x7f38b414bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.596+0000 7f38ba99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 0x7f38b4141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.596+0000 7f38ba99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b4155af0 0x7f38b414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.596+0000 7f38ba99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40af970 con 0x7f38b40c8b50 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.596+0000 7f38ba99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40b0490 con 0x7f38b4155af0 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.596+0000 7f38ba99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b414c1a0 con 0x7f38b406a6a0 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 0x7f38b4141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 0x7f38b4141000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53148/0 (socket says 172.21.15.33:53148) 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 -- 172.21.15.33:0/621762629 learned_addr learned my addr 172.21.15.33:0/621762629 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 -- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38b406a6a0 msgr2=0x7f38b414bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38b406a6a0 0x7f38b414bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 -- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b4155af0 msgr2=0x7f38b414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b4155af0 0x7f38b414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 -- 172.21.15.33:0/621762629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40094f0 con 0x7f38b40c8b50 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38b919a700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 0x7f38b4141000 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009290 tx=0x7f38a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4011e20 con 0x7f38b40c8b50 2023-10-06T05:24:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a401b680 con 0x7f38b40c8b50 2023-10-06T05:24:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a401ba40 con 0x7f38b40c8b50 2023-10-06T05:24:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.597+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b40afb60 con 0x7f38b40c8b50 2023-10-06T05:24:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.598+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b40b0120 con 0x7f38b40c8b50 2023-10-06T05:24:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.598+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b404c380 con 0x7f38b40c8b50 2023-10-06T05:24:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.599+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38a401c450 con 0x7f38b40c8b50 2023-10-06T05:24:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.601+0000 7f38a9ffb700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3898066f30 0x7f38980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.601+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38a4086450 con 0x7f38b40c8b50 2023-10-06T05:24:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.601+0000 7f38b999b700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3898066f30 0x7f38980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.602+0000 7f38b999b700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3898066f30 0x7f38980693e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38b40aecb0 tx=0x7f38b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.605+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38a4053490 con 0x7f38b40c8b50 2023-10-06T05:24:48.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.809+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38b406b400 con 0x7f38b40c8b50 2023-10-06T05:24:48.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.810+0000 7f38a9ffb700 1 -- 172.21.15.33:0/621762629 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f38a4056ab0 con 0x7f38b40c8b50 2023-10-06T05:24:48.812 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":145,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:40.783691+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:23.679709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:24.472340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2321415227":"2023-10-06T06:24:39.948943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3898066f30 msgr2=0x7f38980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3898066f30 0x7f38980693e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38b40aecb0 tx=0x7f38b0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 msgr2=0x7f38b4141000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 0x7f38b4141000 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009290 tx=0x7f38a4016000 comp rx=0 tx=0).stop 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 shutdown_connections 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3898066f30 0x7f38980693e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b4155af0 0x7f38b414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38b406a6a0 0x7f38b414bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 --2- 172.21.15.33:0/621762629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b40c8b50 0x7f38b4141000 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.812+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 >> 172.21.15.33:0/621762629 conn(0x7f38b406d030 msgr2=0x7f38b40c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.813+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 shutdown_connections 2023-10-06T05:24:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:48.813+0000 7f38ba99d700 1 -- 172.21.15.33:0/621762629 wait complete. 2023-10-06T05:24:48.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.019+0000 7fd088f9d700 1 Processor -- start 2023-10-06T05:24:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.019+0000 7fd088f9d700 1 -- start start 2023-10-06T05:24:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd088f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0840c7070 0x7fd0840cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd088f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0841663c0 0x7fd0840cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd088f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0840caca0 con 0x7fd084165ff0 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd088f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0840ccc00 con 0x7fd0840c7070 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd088f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0840ca090 con 0x7fd0841663c0 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd082ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0840c7070 0x7fd0840cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd082ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0840c7070 0x7fd0840cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53164/0 (socket says 172.21.15.33:53164) 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.020+0000 7fd082ffd700 1 -- 172.21.15.33:0/3442537068 learned_addr learned my addr 172.21.15.33:0/3442537068 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.021+0000 7fd0827fc700 1 -- 172.21.15.33:0/3442537068 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572179875 0 0) 0x7fd0840caca0 con 0x7fd084165ff0 2023-10-06T05:24:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.021+0000 7fd083fff700 1 --2- 172.21.15.33:0/3442537068 >> v2:172.21.15.167:3300/0 conn(0x7fd0841663c0 0x7fd0840cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.046+0000 7fd0827fc700 1 -- 172.21.15.33:0/3442537068 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c003550 con 0x7fd084165ff0 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd082ffd700 1 -- 172.21.15.33:0/3442537068 >> v2:172.21.15.167:3300/0 conn(0x7fd0841663c0 msgr2=0x7fd0840cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd082ffd700 1 --2- 172.21.15.33:0/3442537068 >> v2:172.21.15.167:3300/0 conn(0x7fd0841663c0 0x7fd0840cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd082ffd700 1 -- 172.21.15.33:0/3442537068 >> v1:172.21.15.33:6789/0 conn(0x7fd084165ff0 legacy=0x7fd0840c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd0827fc700 1 -- 172.21.15.33:0/3442537068 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760649369 0 0) 0x7fd06c003550 con 0x7fd084165ff0 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd082ffd700 1 -- 172.21.15.33:0/3442537068 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840ca370 con 0x7fd0840c7070 2023-10-06T05:24:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd082ffd700 1 --2- 172.21.15.33:0/3442537068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0840c7070 0x7fd0840cc180 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd0840ccfa0 tx=0x7fd078009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dd75cc334bc3eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd0827fc700 1 -- 172.21.15.33:0/3442537068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800e070 con 0x7fd0840c7070 2023-10-06T05:24:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd0827fc700 1 -- 172.21.15.33:0/3442537068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0780090d0 con 0x7fd0840c7070 2023-10-06T05:24:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.047+0000 7fd0827fc700 1 -- 172.21.15.33:0/3442537068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074001470 con 0x7fd0840c7070 2023-10-06T05:24:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 -- 172.21.15.33:0/3442537068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0840c7070 msgr2=0x7fd0840cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3442537068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0840c7070 0x7fd0840cc180 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd0840ccfa0 tx=0x7fd078009b70 comp rx=0 tx=0).stop 2023-10-06T05:24:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 -- 172.21.15.33:0/3442537068 shutdown_connections 2023-10-06T05:24:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3442537068 >> v2:172.21.15.167:3300/0 conn(0x7fd0841663c0 0x7fd0840cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3442537068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0840c7070 0x7fd0840cc180 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 -- 172.21.15.33:0/3442537068 >> 172.21.15.33:0/3442537068 conn(0x7fd08406c560 msgr2=0x7fd08406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 -- 172.21.15.33:0/3442537068 shutdown_connections 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.048+0000 7fd088f9d700 1 -- 172.21.15.33:0/3442537068 wait complete. 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.049+0000 7fd088f9d700 1 Processor -- start 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.050+0000 7fd088f9d700 1 -- start start 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.050+0000 7fd088f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0840c7070 0x7fd084164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.050+0000 7fd088f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd084165ff0 0x7fd0840c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.050+0000 7fd088f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 0x7fd084151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd088f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840af020 con 0x7fd0841663c0 2023-10-06T05:24:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd088f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840af5e0 con 0x7fd0840c7070 2023-10-06T05:24:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd088f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd084164bc0 con 0x7fd084165ff0 2023-10-06T05:24:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 0x7fd084151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 0x7fd084151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53166/0 (socket says 172.21.15.33:53166) 2023-10-06T05:24:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 -- 172.21.15.33:0/3321962481 learned_addr learned my addr 172.21.15.33:0/3321962481 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd082ffd700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd084165ff0 0x7fd0840c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd0837fe700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0840c7070 0x7fd084164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 -- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd084165ff0 msgr2=0x7fd0840c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd084165ff0 0x7fd0840c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 -- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0840c7070 msgr2=0x7fd084164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0840c7070 0x7fd084164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 -- 172.21.15.33:0/3321962481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009e10 con 0x7fd0841663c0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.051+0000 7fd083fff700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 0x7fd084151600 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00ccf0 tx=0x7fd07c00e2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.052+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c007980 con 0x7fd0841663c0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.052+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c014040 con 0x7fd0841663c0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.052+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c007d50 con 0x7fd0841663c0 2023-10-06T05:24:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.052+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd084164ea0 con 0x7fd0841663c0 2023-10-06T05:24:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.052+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0841647b0 con 0x7fd0841663c0 2023-10-06T05:24:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.053+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08404c380 con 0x7fd0841663c0 2023-10-06T05:24:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.054+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd07c018600 con 0x7fd0841663c0 2023-10-06T05:24:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.055+0000 7fd06b7fe700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd070066f30 0x7fd0700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.055+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd07c088900 con 0x7fd0841663c0 2023-10-06T05:24:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.055+0000 7fd0837fe700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd070066f30 0x7fd0700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.056+0000 7fd0837fe700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd070066f30 0x7fd0700693e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd0740011e0 tx=0x7fd07400a3d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.059+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd07c055940 con 0x7fd0841663c0 2023-10-06T05:24:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.304+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0840ca650 con 0x7fd0841663c0 2023-10-06T05:24:49.306 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:24:49.304+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:49.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.309+0000 7fd06b7fe700 1 -- 172.21.15.33:0/3321962481 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd07c058f60 con 0x7fd0841663c0 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd070066f30 msgr2=0x7fd0700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd070066f30 0x7fd0700693e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd0740011e0 tx=0x7fd07400a3d0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 msgr2=0x7fd084151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 0x7fd084151600 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00ccf0 tx=0x7fd07c00e2e0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 shutdown_connections 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd070066f30 0x7fd0700693e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0841663c0 0x7fd084151600 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0840c7070 0x7fd084164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 --2- 172.21.15.33:0/3321962481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd084165ff0 0x7fd0840c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 >> 172.21.15.33:0/3321962481 conn(0x7fd08406c560 msgr2=0x7fd084161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 shutdown_connections 2023-10-06T05:24:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.313+0000 7fd088f9d700 1 -- 172.21.15.33:0/3321962481 wait complete. 2023-10-06T05:24:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:49.326 INFO:tasks.ceph.mds.c.smithi033.stderr: -6035> 2023-10-06T05:24:31.984+0000 7fd1e0bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:49.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:24:49.380 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:24:49.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:49.462 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.532+0000 7f3740fff700 1 Processor -- start 2023-10-06T05:24:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.532+0000 7f3740fff700 1 -- start start 2023-10-06T05:24:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.533+0000 7f3740fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f373c0aeeb0 0x7f373c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.533+0000 7f3740fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f373c0aa2a0 0x7f373c163520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.533+0000 7f3740fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c06d780 con 0x7f373c06a420 2023-10-06T05:24:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.533+0000 7f3740fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c0ac1c0 con 0x7f373c0aa2a0 2023-10-06T05:24:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.533+0000 7f3740fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c06d9d0 con 0x7f373c0aeeb0 2023-10-06T05:24:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f373c0aa2a0 0x7f373c163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f373c0aa2a0 0x7f373c163520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53190/0 (socket says 172.21.15.33:53190) 2023-10-06T05:24:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 -- 172.21.15.33:0/175103537 learned_addr learned my addr 172.21.15.33:0/175103537 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f3739d9b700 1 --2- 172.21.15.33:0/175103537 >> v2:172.21.15.167:3300/0 conn(0x7f373c0aeeb0 0x7f373c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 -- 172.21.15.33:0/175103537 >> v2:172.21.15.167:3300/0 conn(0x7f373c0aeeb0 msgr2=0x7f373c15a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 --2- 172.21.15.33:0/175103537 >> v2:172.21.15.167:3300/0 conn(0x7f373c0aeeb0 0x7f373c15a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 -- 172.21.15.33:0/175103537 >> v1:172.21.15.167:6789/0 conn(0x7f373c06a420 legacy=0x7f373c06a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.534+0000 7f373ad9d700 1 -- 172.21.15.33:0/175103537 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c06dcb0 con 0x7f373c0aa2a0 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f373ad9d700 1 --2- 172.21.15.33:0/175103537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aa2a0 0x7f373c163520 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f373c06a3b0 tx=0x7f3734009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d018e7db65dbbde5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f373959a700 1 -- 172.21.15.33:0/175103537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373400e070 con 0x7f373c0aa2a0 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f373959a700 1 -- 172.21.15.33:0/175103537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373400ba50 con 0x7f373c0aa2a0 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f373959a700 1 -- 172.21.15.33:0/175103537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340092e0 con 0x7f373c0aa2a0 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f3740fff700 1 -- 172.21.15.33:0/175103537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aa2a0 msgr2=0x7f373c163520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f3740fff700 1 --2- 172.21.15.33:0/175103537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aa2a0 0x7f373c163520 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f373c06a3b0 tx=0x7f3734009840 comp rx=0 tx=0).stop 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f3740fff700 1 -- 172.21.15.33:0/175103537 shutdown_connections 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f3740fff700 1 --2- 172.21.15.33:0/175103537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aa2a0 0x7f373c163520 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f3740fff700 1 --2- 172.21.15.33:0/175103537 >> v2:172.21.15.167:3300/0 conn(0x7f373c0aeeb0 0x7f373c15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.535+0000 7f3740fff700 1 -- 172.21.15.33:0/175103537 >> 172.21.15.33:0/175103537 conn(0x7f373c06f7b0 msgr2=0x7f373c0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.536+0000 7f3740fff700 1 -- 172.21.15.33:0/175103537 shutdown_connections 2023-10-06T05:24:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.536+0000 7f3740fff700 1 -- 172.21.15.33:0/175103537 wait complete. 2023-10-06T05:24:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.537+0000 7f3740fff700 1 Processor -- start 2023-10-06T05:24:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.537+0000 7f3740fff700 1 -- start start 2023-10-06T05:24:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f3740fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c06a420 0x7f373c16c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f3740fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c0aa2a0 0x7f373c169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f3740fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 0x7f373c16a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f3740fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b13d0 con 0x7f373c0aeeb0 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f3740fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c14d0f0 con 0x7f373c06a420 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f3740fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c14d6e0 con 0x7f373c0aa2a0 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f373ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 0x7f373c16a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f373ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 0x7f373c16a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53202/0 (socket says 172.21.15.33:53202) 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f373ad9d700 1 -- 172.21.15.33:0/2010735549 learned_addr learned my addr 172.21.15.33:0/2010735549 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f3739d9b700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c0aa2a0 0x7f373c169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.538+0000 7f373a59c700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c06a420 0x7f373c16c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f373ad9d700 1 -- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c0aa2a0 msgr2=0x7f373c169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f373ad9d700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c0aa2a0 0x7f373c169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f373ad9d700 1 -- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c06a420 msgr2=0x7f373c16c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f373ad9d700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c06a420 0x7f373c16c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f373ad9d700 1 -- 172.21.15.33:0/2010735549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3734009ae0 con 0x7f373c0aeeb0 2023-10-06T05:24:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f373ad9d700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 0x7f373c16a190 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f3734009fd0 tx=0x7f3734018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340094e0 con 0x7f373c0aeeb0 2023-10-06T05:24:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734008e40 con 0x7f373c0aeeb0 2023-10-06T05:24:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c0b07c0 con 0x7f373c0aeeb0 2023-10-06T05:24:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340256d0 con 0x7f373c0aeeb0 2023-10-06T05:24:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.539+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0b0d80 con 0x7f373c0aeeb0 2023-10-06T05:24:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.540+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c0b1580 con 0x7f373c0aeeb0 2023-10-06T05:24:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.541+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3734003040 con 0x7f373c0aeeb0 2023-10-06T05:24:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.542+0000 7f372a7fc700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f371c066f30 0x7f371c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.543+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3734086490 con 0x7f373c0aeeb0 2023-10-06T05:24:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.543+0000 7f373a59c700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f371c066f30 0x7f371c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.544+0000 7f373a59c700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f371c066f30 0x7f371c0693e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f372c0037c0 tx=0x7f372c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.546+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f373c0b1580 con 0x7f373c0aeeb0 2023-10-06T05:24:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:49.797+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f373c04c380 con 0x7f373c0aeeb0 2023-10-06T05:24:50.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.310+0000 7f372a7fc700 1 -- 172.21.15.33:0/2010735549 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3734053480 con 0x7f373c0aeeb0 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f371c066f30 msgr2=0x7f371c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f371c066f30 0x7f371c0693e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f372c0037c0 tx=0x7f372c009040 comp rx=0 tx=0).stop 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 msgr2=0x7f373c16a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 0x7f373c16a190 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f3734009fd0 tx=0x7f3734018000 comp rx=0 tx=0).stop 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 shutdown_connections 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f371c066f30 0x7f371c0693e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c0aeeb0 0x7f373c16a190 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f373c06a420 0x7f373c16c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 --2- 172.21.15.33:0/2010735549 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f373c0aa2a0 0x7f373c169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 >> 172.21.15.33:0/2010735549 conn(0x7f373c06f7b0 msgr2=0x7f373c14a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 shutdown_connections 2023-10-06T05:24:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.313+0000 7f3740fff700 1 -- 172.21.15.33:0/2010735549 wait complete. 2023-10-06T05:24:50.334 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:24:50.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.526+0000 7f22af39d700 1 Processor -- start 2023-10-06T05:24:50.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.526+0000 7f22af39d700 1 -- start start 2023-10-06T05:24:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.527+0000 7f22af39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 0x7f22a81590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.527+0000 7f22af39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a815d280 con 0x7f22a815a740 2023-10-06T05:24:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.527+0000 7f22af39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a815de00 con 0x7f22a806dfd0 2023-10-06T05:24:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.527+0000 7f22af39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a815e290 con 0x7f22a806dc00 2023-10-06T05:24:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22aeb9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f22a815a740 0x7f22a80cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40600/0 (socket says 172.21.15.33:40600) 2023-10-06T05:24:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22aeb9c700 1 -- 172.21.15.33:0/2091728910 learned_addr learned my addr 172.21.15.33:0/2091728910 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:50.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ae39b700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 0x7f22a81590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368147296 0 0) 0x7f22a815d280 con 0x7f22a815a740 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2294003570 con 0x7f22a815a740 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3733490267 0 0) 0x7f2294003570 con 0x7f22a815a740 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 msgr2=0x7f22a81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 0x7f22a81590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 >> v1:172.21.15.167:6789/0 conn(0x7f22a806dfd0 legacy=0x7f22a815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a815d4d0 con 0x7f22a815a740 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.528+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f22a00017f0 con 0x7f22a815a740 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 >> v1:172.21.15.33:6789/0 conn(0x7f22a815a740 legacy=0x7f22a80cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 0x7f2294003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f2294004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2294004d50 0x7f22940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229400bb70 con 0x7f2294003fd0 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229400bce0 con 0x7f2294004d50 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229400bea0 con 0x7f22a806dc00 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22adb9a700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f2294004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22ae39b700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 0x7f2294003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22aeb9c700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2294004d50 0x7f22940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22adb9a700 1 -- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 msgr2=0x7f2294003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22adb9a700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a806dc00 0x7f2294003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22adb9a700 1 -- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2294004d50 msgr2=0x7f22940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22adb9a700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2294004d50 0x7f22940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.529+0000 7f22adb9a700 1 -- 172.21.15.33:0/2091728910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2294003570 con 0x7f2294003fd0 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22adb9a700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f2294004810 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f22980019a0 tx=0x7f2298007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b06571f2e6abb64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800c5d0 con 0x7f2294003fd0 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229800c750 con 0x7f2294003fd0 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22ad399700 1 -- 172.21.15.33:0/2091728910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800cb10 con 0x7f2294003fd0 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22af39d700 1 -- 172.21.15.33:0/2091728910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 msgr2=0x7f2294004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22af39d700 1 --2- 172.21.15.33:0/2091728910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f2294004810 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f22980019a0 tx=0x7f2298007d10 comp rx=0 tx=0).stop 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22aeb9c700 1 -- 172.21.15.33:0/2091728910 reap_dead start 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22af39d700 1 -- 172.21.15.33:0/2091728910 shutdown_connections 2023-10-06T05:24:50.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22af39d700 1 -- 172.21.15.33:0/2091728910 >> 172.21.15.33:0/2091728910 conn(0x7f22a80d0550 msgr2=0x7f22a80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.530+0000 7f22af39d700 1 -- 172.21.15.33:0/2091728910 shutdown_connections 2023-10-06T05:24:50.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.531+0000 7f22af39d700 1 -- 172.21.15.33:0/2091728910 wait complete. 2023-10-06T05:24:50.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.532+0000 7f22af39d700 1 Processor -- start 2023-10-06T05:24:50.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.532+0000 7f22af39d700 1 -- start start 2023-10-06T05:24:50.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22af39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f22a8155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22af39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a80c87d0 0x7f22a80b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22af39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a80b5dc0 0x7f22a815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22af39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a816aa60 con 0x7f2294003fd0 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22af39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c7730 con 0x7f22a80b5dc0 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22af39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c7a10 con 0x7f22a80c87d0 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22ae39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f22a8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22ae39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f22a8155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51506/0 (socket says 172.21.15.33:51506) 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22ae39b700 1 -- 172.21.15.33:0/1446452894 learned_addr learned my addr 172.21.15.33:0/1446452894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22adb9a700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a80c87d0 0x7f22a80b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.533+0000 7f22aeb9c700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a80b5dc0 0x7f22a815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22ae39b700 1 -- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a80c87d0 msgr2=0x7f22a80b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22ae39b700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a80c87d0 0x7f22a80b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22ae39b700 1 -- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a80b5dc0 msgr2=0x7f22a815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22ae39b700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a80b5dc0 0x7f22a815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22ae39b700 1 -- 172.21.15.33:0/1446452894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80c7cf0 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22adb9a700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a80c87d0 0x7f22a80b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22ae39b700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f22a8155490 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f22a4011d40 tx=0x7f22a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a400fac0 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a400fc20 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4014440 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a80c7fd0 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.534+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81559d0 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.535+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a80c8620 con 0x7f2294003fd0 2023-10-06T05:24:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.536+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22a4014a80 con 0x7f2294003fd0 2023-10-06T05:24:50.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.537+0000 7f229e7fc700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f228c066f30 0x7f228c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.538+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f22a4018030 con 0x7f2294003fd0 2023-10-06T05:24:50.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.538+0000 7f22adb9a700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f228c066f30 0x7f228c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.539+0000 7f22adb9a700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f228c066f30 0x7f228c0693e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2298008fd0 tx=0x7f2298015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.542+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22a40524a0 con 0x7f2294003fd0 2023-10-06T05:24:50.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:50.744+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f22a80b6190 con 0x7f2294003fd0 2023-10-06T05:24:51.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.313+0000 7f229e7fc700 1 -- 172.21.15.33:0/1446452894 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f22a4055ac0 con 0x7f2294003fd0 2023-10-06T05:24:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f228c066f30 msgr2=0x7f228c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f228c066f30 0x7f228c0693e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2298008fd0 tx=0x7f2298015000 comp rx=0 tx=0).stop 2023-10-06T05:24:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 msgr2=0x7f22a8155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f22a8155490 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f22a4011d40 tx=0x7f22a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 shutdown_connections 2023-10-06T05:24:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f228c066f30 0x7f228c0693e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a80b5dc0 0x7f22a815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2294003fd0 0x7f22a8155490 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 --2- 172.21.15.33:0/1446452894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22a80c87d0 0x7f22a80b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 >> 172.21.15.33:0/1446452894 conn(0x7f22a80d0550 msgr2=0x7f22a80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 shutdown_connections 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.317+0000 7f22af39d700 1 -- 172.21.15.33:0/1446452894 wait complete. 2023-10-06T05:24:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:24:51.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:24:51.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.535+0000 7f0b5e19d700 1 Processor -- start 2023-10-06T05:24:51.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.535+0000 7f0b5e19d700 1 -- start start 2023-10-06T05:24:51.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.536+0000 7f0b5e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0b58157320 0x7f0b580adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.536+0000 7f0b5e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 0x7f0b580ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.536+0000 7f0b5e19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b580b0840 con 0x7f0b581576f0 2023-10-06T05:24:51.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.536+0000 7f0b5e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b581585c0 con 0x7f0b58157320 2023-10-06T05:24:51.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.536+0000 7f0b5e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580b1ff0 con 0x7f0b580b1ae0 2023-10-06T05:24:51.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b5c99a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0b581576f0 0x7f0b580b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40616/0 (socket says 172.21.15.33:40616) 2023-10-06T05:24:51.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b5c99a700 1 -- 172.21.15.33:0/2881422536 learned_addr learned my addr 172.21.15.33:0/2881422536 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:51.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b5d19b700 1 --2- 172.21.15.33:0/2881422536 >> v2:172.21.15.167:3300/0 conn(0x7f0b58157320 0x7f0b580adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551762048 0 0) 0x7f0b580b0840 con 0x7f0b581576f0 2023-10-06T05:24:51.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b5d99c700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 0x7f0b580ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b44003570 con 0x7f0b581576f0 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977726758 0 0) 0x7f0b44003570 con 0x7f0b581576f0 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 msgr2=0x7f0b580ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 0x7f0b580ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 >> v2:172.21.15.167:3300/0 conn(0x7f0b58157320 msgr2=0x7f0b580adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 --2- 172.21.15.33:0/2881422536 >> v2:172.21.15.167:3300/0 conn(0x7f0b58157320 0x7f0b580adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58157c90 con 0x7f0b581576f0 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.537+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f0b48001470 con 0x7f0b581576f0 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 >> v1:172.21.15.33:6789/0 conn(0x7f0b581576f0 legacy=0x7f0b580b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44003a60 0x7f0b44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44004810 0x7f0b44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 0x7f0b440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b581585c0 con 0x7f0b44004810 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580b1ff0 con 0x7f0b44003a60 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4400ba60 con 0x7f0b580b1ae0 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44004810 0x7f0b44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5d19b700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44003a60 0x7f0b44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5d99c700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 0x7f0b440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5c99a700 1 -- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 msgr2=0x7f0b440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580b1ae0 0x7f0b440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5c99a700 1 -- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44003a60 msgr2=0x7f0b44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44003a60 0x7f0b44003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.538+0000 7f0b5c99a700 1 -- 172.21.15.33:0/2881422536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44003570 con 0x7f0b44004810 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44004810 0x7f0b44004c80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c3f0 tx=0x7f0b58158530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbfa4c336b92d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800cc80 con 0x7f0b44004810 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4800cde0 con 0x7f0b44004810 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b4ffff700 1 -- 172.21.15.33:0/2881422536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48005b70 con 0x7f0b44004810 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b5e19d700 1 -- 172.21.15.33:0/2881422536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44004810 msgr2=0x7f0b44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/2881422536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44004810 0x7f0b44004c80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c3f0 tx=0x7f0b58158530 comp rx=0 tx=0).stop 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b5d99c700 1 -- 172.21.15.33:0/2881422536 reap_dead start 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b5e19d700 1 -- 172.21.15.33:0/2881422536 shutdown_connections 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.539+0000 7f0b5e19d700 1 -- 172.21.15.33:0/2881422536 >> 172.21.15.33:0/2881422536 conn(0x7f0b58068090 msgr2=0x7f0b5806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.540+0000 7f0b5e19d700 1 -- 172.21.15.33:0/2881422536 shutdown_connections 2023-10-06T05:24:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.540+0000 7f0b5e19d700 1 -- 172.21.15.33:0/2881422536 wait complete. 2023-10-06T05:24:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.541+0000 7f0b5e19d700 1 Processor -- start 2023-10-06T05:24:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.541+0000 7f0b5e19d700 1 -- start start 2023-10-06T05:24:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44004810 0x7f0b581543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 0x7f0b5814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580acf90 0x7f0b580ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580acb00 con 0x7f0b580aa320 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5814e6d0 con 0x7f0b44004810 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580aaa70 con 0x7f0b580acf90 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 0x7f0b5814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5c99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 0x7f0b5814e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51520/0 (socket says 172.21.15.33:51520) 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.542+0000 7f0b5c99a700 1 -- 172.21.15.33:0/773587815 learned_addr learned my addr 172.21.15.33:0/773587815 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5d19b700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44004810 0x7f0b581543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5d99c700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580acf90 0x7f0b580ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5c99a700 1 -- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580acf90 msgr2=0x7f0b580ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580acf90 0x7f0b580ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5c99a700 1 -- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44004810 msgr2=0x7f0b581543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44004810 0x7f0b581543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5c99a700 1 -- 172.21.15.33:0/773587815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580ac1d0 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5d99c700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580acf90 0x7f0b580ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5c99a700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 0x7f0b5814e190 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007d10 tx=0x7f0b48015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800cc80 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b480072e0 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480076a0 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.543+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b580ac4b0 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.544+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b580aa790 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.545+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b58070840 con 0x7f0b580aa320 2023-10-06T05:24:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.545+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b48019030 con 0x7f0b580aa320 2023-10-06T05:24:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.546+0000 7f0b4d7fa700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b3c066f30 0x7f0b3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.546+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f0b48085a10 con 0x7f0b580aa320 2023-10-06T05:24:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.547+0000 7f0b5d19b700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b3c066f30 0x7f0b3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.547+0000 7f0b5d19b700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b3c066f30 0x7f0b3c0693e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0b54006f00 tx=0x7f0b540091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.550+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b48052bb0 con 0x7f0b580aa320 2023-10-06T05:24:51.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:51.755+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b58063ff0 con 0x7f0b580aa320 2023-10-06T05:24:52.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.320+0000 7f0b4d7fa700 1 -- 172.21.15.33:0/773587815 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0b480561d0 con 0x7f0b580aa320 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b3c066f30 msgr2=0x7f0b3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b3c066f30 0x7f0b3c0693e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0b54006f00 tx=0x7f0b540091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 msgr2=0x7f0b5814e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 0x7f0b5814e190 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007d10 tx=0x7f0b48015000 comp rx=0 tx=0).stop 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 shutdown_connections 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0b3c066f30 0x7f0b3c0693e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b580acf90 0x7f0b580ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b44004810 0x7f0b581543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 --2- 172.21.15.33:0/773587815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b580aa320 0x7f0b5814e190 secure :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007d10 tx=0x7f0b48015000 comp rx=0 tx=0).stop 2023-10-06T05:24:52.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 >> 172.21.15.33:0/773587815 conn(0x7f0b58068090 msgr2=0x7f0b580b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 shutdown_connections 2023-10-06T05:24:52.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.324+0000 7f0b5e19d700 1 -- 172.21.15.33:0/773587815 wait complete. 2023-10-06T05:24:52.327 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:24:52.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:24:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.544+0000 7f6c4359e700 1 Processor -- start 2023-10-06T05:24:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.545+0000 7f6c4359e700 1 -- start start 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c44156370 0x7f6c440af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c4359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c440cc7e0 0x7f6c440c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c4359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c440ad430 con 0x7f6c440b01f0 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c4359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440afe20 con 0x7f6c440cc7e0 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c4359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4406bc70 con 0x7f6c44156370 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c41d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c440cc7e0 0x7f6c440c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c41d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c440cc7e0 0x7f6c440c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51532/0 (socket says 172.21.15.33:51532) 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c41d9b700 1 -- 172.21.15.33:0/4137308330 learned_addr learned my addr 172.21.15.33:0/4137308330 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.546+0000 7f6c4259c700 1 --2- 172.21.15.33:0/4137308330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c44156370 0x7f6c440af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c41d9b700 1 -- 172.21.15.33:0/4137308330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c44156370 msgr2=0x7f6c440af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c41d9b700 1 --2- 172.21.15.33:0/4137308330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c44156370 0x7f6c440af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c41d9b700 1 -- 172.21.15.33:0/4137308330 >> v1:172.21.15.167:6789/0 conn(0x7f6c440b01f0 legacy=0x7f6c440ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c41d9b700 1 -- 172.21.15.33:0/4137308330 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440c92f0 con 0x7f6c440cc7e0 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c41d9b700 1 --2- 172.21.15.33:0/4137308330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c440c8db0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6c380099b0 tx=0x7f6c38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe767fcdb1ab32b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c4159a700 1 -- 172.21.15.33:0/4137308330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800b860 con 0x7f6c440cc7e0 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c4159a700 1 -- 172.21.15.33:0/4137308330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3800b9c0 con 0x7f6c440cc7e0 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.547+0000 7f6c4159a700 1 -- 172.21.15.33:0/4137308330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800bd60 con 0x7f6c440cc7e0 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 -- 172.21.15.33:0/4137308330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 msgr2=0x7f6c440c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 --2- 172.21.15.33:0/4137308330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c440c8db0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f6c380099b0 tx=0x7f6c38009250 comp rx=0 tx=0).stop 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 -- 172.21.15.33:0/4137308330 shutdown_connections 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 --2- 172.21.15.33:0/4137308330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c44156370 0x7f6c440af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 --2- 172.21.15.33:0/4137308330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c440c8db0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 -- 172.21.15.33:0/4137308330 >> 172.21.15.33:0/4137308330 conn(0x7f6c4406d030 msgr2=0x7f6c440cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 -- 172.21.15.33:0/4137308330 shutdown_connections 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.548+0000 7f6c4359e700 1 -- 172.21.15.33:0/4137308330 wait complete. 2023-10-06T05:24:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.549+0000 7f6c4359e700 1 Processor -- start 2023-10-06T05:24:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.550+0000 7f6c4359e700 1 -- start start 2023-10-06T05:24:52.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.550+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c440b01f0 0x7f6c440aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c44157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c44156370 0x7f6c44169a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c4359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44157850 con 0x7f6c440cc7e0 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c4359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44156bd0 con 0x7f6c44156370 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c4359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44156eb0 con 0x7f6c440b01f0 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c44157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c44157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51546/0 (socket says 172.21.15.33:51546) 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 -- 172.21.15.33:0/1428081527 learned_addr learned my addr 172.21.15.33:0/1428081527 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c4259c700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c440b01f0 0x7f6c440aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c42d9d700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c44156370 0x7f6c44169a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 -- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c440b01f0 msgr2=0x7f6c440aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c440b01f0 0x7f6c440aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 -- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c44156370 msgr2=0x7f6c44169a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c44156370 0x7f6c44169a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c41d9b700 1 -- 172.21.15.33:0/1428081527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380094f0 con 0x7f6c440cc7e0 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.551+0000 7f6c42d9d700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c44156370 0x7f6c44169a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.552+0000 7f6c41d9b700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c44157c70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009290 tx=0x7f6c38019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.552+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38019e60 con 0x7f6c440cc7e0 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.552+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c38010470 con 0x7f6c440cc7e0 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.552+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38010830 con 0x7f6c440cc7e0 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.552+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c44157190 con 0x7f6c440cc7e0 2023-10-06T05:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.552+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c440aa980 con 0x7f6c440cc7e0 2023-10-06T05:24:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.553+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c44156ff0 con 0x7f6c440cc7e0 2023-10-06T05:24:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.554+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c3802c430 con 0x7f6c440cc7e0 2023-10-06T05:24:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.555+0000 7f6c327fc700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.555+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c38014070 con 0x7f6c440cc7e0 2023-10-06T05:24:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.555+0000 7f6c4259c700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.556+0000 7f6c4259c700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c24066f30 0x7f6c240693e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6c34006fd0 tx=0x7f6c34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.559+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c44156ff0 con 0x7f6c440cc7e0 2023-10-06T05:24:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.762+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6c44157990 con 0x7f6c440cc7e0 2023-10-06T05:24:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.766+0000 7f6c327fc700 1 -- 172.21.15.33:0/1428081527 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c44157990 con 0x7f6c440cc7e0 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c24066f30 msgr2=0x7f6c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c24066f30 0x7f6c240693e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6c34006fd0 tx=0x7f6c34008040 comp rx=0 tx=0).stop 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 msgr2=0x7f6c44157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c44157c70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009290 tx=0x7f6c38019000 comp rx=0 tx=0).stop 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 shutdown_connections 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c44156370 0x7f6c44169a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c440b01f0 0x7f6c440aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 --2- 172.21.15.33:0/1428081527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c440cc7e0 0x7f6c44157c70 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 >> 172.21.15.33:0/1428081527 conn(0x7f6c4406d030 msgr2=0x7f6c44151060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 shutdown_connections 2023-10-06T05:24:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.768+0000 7f6c4359e700 1 -- 172.21.15.33:0/1428081527 wait complete. 2023-10-06T05:24:52.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock_eviction' 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.972+0000 7f6c2c9fe700 1 Processor -- start 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.972+0000 7f6c2c9fe700 1 -- start start 2023-10-06T05:24:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2c9fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c28153b10 0x7f6c280c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2c9fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c280c2610 0x7f6c280c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2c9fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2815abd0 con 0x7f6c280c6b30 2023-10-06T05:24:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2c9fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c280c74f0 con 0x7f6c280c2610 2023-10-06T05:24:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2c9fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c280c9b00 con 0x7f6c28153b10 2023-10-06T05:24:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c280c6b30 0x7f6c280675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40626/0 (socket says 172.21.15.33:40626) 2023-10-06T05:24:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2659c700 1 -- 172.21.15.33:0/3305781461 learned_addr learned my addr 172.21.15.33:0/3305781461 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c2759e700 1 --2- 172.21.15.33:0/3305781461 >> v2:172.21.15.33:3300/0 conn(0x7f6c280c2610 0x7f6c280c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c26d9d700 1 --2- 172.21.15.33:0/3305781461 >> v2:172.21.15.167:3300/0 conn(0x7f6c28153b10 0x7f6c280c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c25d9b700 1 -- 172.21.15.33:0/3305781461 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204905914 0 0) 0x7f6c2815abd0 con 0x7f6c280c6b30 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.974+0000 7f6c25d9b700 1 -- 172.21.15.33:0/3305781461 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c10003570 con 0x7f6c280c6b30 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c2759e700 1 -- 172.21.15.33:0/3305781461 >> v2:172.21.15.167:3300/0 conn(0x7f6c28153b10 msgr2=0x7f6c280c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c2759e700 1 --2- 172.21.15.33:0/3305781461 >> v2:172.21.15.167:3300/0 conn(0x7f6c28153b10 0x7f6c280c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c2759e700 1 -- 172.21.15.33:0/3305781461 >> v1:172.21.15.33:6789/0 conn(0x7f6c280c6b30 legacy=0x7f6c280675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c2759e700 1 -- 172.21.15.33:0/3305781461 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2815a050 con 0x7f6c280c2610 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c2759e700 1 --2- 172.21.15.33:0/3305781461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c2610 0x7f6c280c6fb0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f6c280c9ca0 tx=0x7f6c20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c91663f895c6896 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c25d9b700 1 -- 172.21.15.33:0/3305781461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20016070 con 0x7f6c280c2610 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c25d9b700 1 -- 172.21.15.33:0/3305781461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000ba80 con 0x7f6c280c2610 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.975+0000 7f6c25d9b700 1 -- 172.21.15.33:0/3305781461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000e070 con 0x7f6c280c2610 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/3305781461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c2610 msgr2=0x7f6c280c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/3305781461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c2610 0x7f6c280c6fb0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f6c280c9ca0 tx=0x7f6c20009780 comp rx=0 tx=0).stop 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/3305781461 shutdown_connections 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/3305781461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c2610 0x7f6c280c6fb0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/3305781461 >> v2:172.21.15.167:3300/0 conn(0x7f6c28153b10 0x7f6c280c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/3305781461 >> 172.21.15.33:0/3305781461 conn(0x7f6c2806c560 msgr2=0x7f6c2806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/3305781461 shutdown_connections 2023-10-06T05:24:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.976+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/3305781461 wait complete. 2023-10-06T05:24:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.977+0000 7f6c2c9fe700 1 Processor -- start 2023-10-06T05:24:52.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.977+0000 7f6c2c9fe700 1 -- start start 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.978+0000 7f6c2c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c280c2610 0x7f6c2815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.978+0000 7f6c2c9fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 0x7f6c2813c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.978+0000 7f6c2c9fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c28153b10 0x7f6c280cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.978+0000 7f6c2c9fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2813fc10 con 0x7f6c280c6b30 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.978+0000 7f6c2c9fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2813e280 con 0x7f6c28153b10 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.978+0000 7f6c2c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280b0fa0 con 0x7f6c280c2610 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 0x7f6c2813c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 0x7f6c2813c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51564/0 (socket says 172.21.15.33:51564) 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 -- 172.21.15.33:0/1885509559 learned_addr learned my addr 172.21.15.33:0/1885509559 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2759e700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c28153b10 0x7f6c280cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c26d9d700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c280c2610 0x7f6c2815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 -- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c280c2610 msgr2=0x7f6c2815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c280c2610 0x7f6c2815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 -- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c28153b10 msgr2=0x7f6c280cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c28153b10 0x7f6c280cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 -- 172.21.15.33:0/1885509559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20009a20 con 0x7f6c280c6b30 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c2659c700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 0x7f6c2813c040 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c7f0 tx=0x7f6c1c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.979+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c005b30 con 0x7f6c280c6b30 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.980+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c005c90 con 0x7f6c280c6b30 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.980+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c007340 con 0x7f6c280c6b30 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.980+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c280b0670 con 0x7f6c280c6b30 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.980+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280b0c30 con 0x7f6c280c6b30 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.981+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c280b1130 con 0x7f6c280c6b30 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.982+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c1c01b400 con 0x7f6c280c6b30 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.983+0000 7f6c0effd700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c14066f30 0x7f6c140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.983+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c1c083f00 con 0x7f6c280c6b30 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.983+0000 7f6c26d9d700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c14066f30 0x7f6c140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.984+0000 7f6c26d9d700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c14066f30 0x7f6c140693e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009ca0 tx=0x7f6c18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:52.987+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c1c051370 con 0x7f6c280c6b30 2023-10-06T05:24:53.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.270+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock_eviction"]} v 0) v1 -- 0x7f6c2804c380 con 0x7f6c280c6b30 2023-10-06T05:24:53.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.319+0000 7f6c0effd700 1 -- 172.21.15.33:0/1885509559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_filelock_eviction"]}]=0 v540) v1 ==== 154+0+0 (secure 0 0 0) 0x7f6c1c0076f0 con 0x7f6c280c6b30 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c14066f30 msgr2=0x7f6c140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c14066f30 0x7f6c140693e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009ca0 tx=0x7f6c18008040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 msgr2=0x7f6c2813c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 0x7f6c2813c040 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c7f0 tx=0x7f6c1c007e00 comp rx=0 tx=0).stop 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 shutdown_connections 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c14066f30 0x7f6c140693e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c28153b10 0x7f6c280cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c280c2610 0x7f6c2815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 --2- 172.21.15.33:0/1885509559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c280c6b30 0x7f6c2813c040 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 >> 172.21.15.33:0/1885509559 conn(0x7f6c2806c560 msgr2=0x7f6c280675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 shutdown_connections 2023-10-06T05:24:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.322+0000 7f6c2c9fe700 1 -- 172.21.15.33:0/1885509559 wait complete. 2023-10-06T05:24:53.342 INFO:tasks.cephfs_test_runner:Check that file lock held by evicted client is given to ... ok 2023-10-06T05:24:53.342 INFO:tasks.cephfs_test_runner:Starting test: test_mount_after_evicted_client (tasks.cephfs.test_client_recovery.TestClientRecovery) Test if a new mount of same fs works after client eviction. 2023-10-06T05:24:53.342 INFO:tasks.cephfs_test_runner:test_mount_after_evicted_client (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:24:53.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_mount_after_evicted_client' 2023-10-06T05:24:53.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.532+0000 7fabdcb9d700 1 Processor -- start 2023-10-06T05:24:53.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.532+0000 7fabdcb9d700 1 -- start start 2023-10-06T05:24:53.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabdcb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd813faa0 0x7fabd813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabdcb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fabd80c7940 0x7fabd80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabdcb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80c9cb0 con 0x7fabd806aad0 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabdcb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd806b660 con 0x7fabd80c7940 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabdcb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80ca270 con 0x7fabd813faa0 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fabd80c7940 0x7fabd80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fabd806aad0 0x7fabd806af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40632/0 (socket says 172.21.15.33:40632) 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd7fff700 1 -- 172.21.15.33:0/898514683 learned_addr learned my addr 172.21.15.33:0/898514683 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd77fe700 1 --2- 172.21.15.33:0/898514683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd813faa0 0x7fabd813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd6ffd700 1 -- 172.21.15.33:0/898514683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd813faa0 msgr2=0x7fabd813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd6ffd700 1 --2- 172.21.15.33:0/898514683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd813faa0 0x7fabd813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd6ffd700 1 -- 172.21.15.33:0/898514683 >> v1:172.21.15.33:6789/0 conn(0x7fabd806aad0 legacy=0x7fabd806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.535+0000 7fabd6ffd700 1 -- 172.21.15.33:0/898514683 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd806b840 con 0x7fabd80c7940 2023-10-06T05:24:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.534+0000 7fabd67fc700 1 -- 172.21.15.33:0/898514683 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792598136 0 0) 0x7fabd80c9cb0 con 0x7fabd806aad0 2023-10-06T05:24:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.535+0000 7fabd6ffd700 1 --2- 172.21.15.33:0/898514683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd80c7db0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0098e0 tx=0x7fabcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e396af9be5851ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.535+0000 7fabd67fc700 1 -- 172.21.15.33:0/898514683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc009e80 con 0x7fabd80c7940 2023-10-06T05:24:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.535+0000 7fabd67fc700 1 -- 172.21.15.33:0/898514683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00b860 con 0x7fabd80c7940 2023-10-06T05:24:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.535+0000 7fabd67fc700 1 -- 172.21.15.33:0/898514683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00bbe0 con 0x7fabd80c7940 2023-10-06T05:24:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 -- 172.21.15.33:0/898514683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 msgr2=0x7fabd80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/898514683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd80c7db0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0098e0 tx=0x7fabcc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 -- 172.21.15.33:0/898514683 shutdown_connections 2023-10-06T05:24:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/898514683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd813faa0 0x7fabd813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/898514683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd80c7db0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 -- 172.21.15.33:0/898514683 >> 172.21.15.33:0/898514683 conn(0x7fabd806d030 msgr2=0x7fabd806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 -- 172.21.15.33:0/898514683 shutdown_connections 2023-10-06T05:24:53.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.536+0000 7fabdcb9d700 1 -- 172.21.15.33:0/898514683 wait complete. 2023-10-06T05:24:53.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.537+0000 7fabdcb9d700 1 Processor -- start 2023-10-06T05:24:53.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.537+0000 7fabdcb9d700 1 -- start start 2023-10-06T05:24:53.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.538+0000 7fabdcb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd806aad0 0x7fabd8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.538+0000 7fabdcb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd812ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.538+0000 7fabdcb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd813faa0 0x7fabd8158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.538+0000 7fabdcb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81579a0 con 0x7fabd80c7940 2023-10-06T05:24:53.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.538+0000 7fabdcb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8157c80 con 0x7fabd813faa0 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.538+0000 7fabdcb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8157f60 con 0x7fabd806aad0 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd812ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd812ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51580/0 (socket says 172.21.15.33:51580) 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 -- 172.21.15.33:0/2220439049 learned_addr learned my addr 172.21.15.33:0/2220439049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd77fe700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd806aad0 0x7fabd8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd7fff700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd813faa0 0x7fabd8158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 -- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd806aad0 msgr2=0x7fabd8159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd806aad0 0x7fabd8159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 -- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd813faa0 msgr2=0x7fabd8158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd813faa0 0x7fabd8158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 -- 172.21.15.33:0/2220439049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc009420 con 0x7fabd80c7940 2023-10-06T05:24:53.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabd6ffd700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd812ea60 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabd80719c0 tx=0x7fabcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.539+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00bbe0 con 0x7fabd80c7940 2023-10-06T05:24:53.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.540+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc011420 con 0x7fabd80c7940 2023-10-06T05:24:53.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.540+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0117e0 con 0x7fabd80c7940 2023-10-06T05:24:53.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.540+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd8158240 con 0x7fabd80c7940 2023-10-06T05:24:53.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.540+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd8159c70 con 0x7fabd80c7940 2023-10-06T05:24:53.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.541+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd804c380 con 0x7fabd80c7940 2023-10-06T05:24:53.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.541+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabcc011bd0 con 0x7fabd80c7940 2023-10-06T05:24:53.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.543+0000 7fabbf7fe700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc4066f30 0x7fabc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.543+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fabcc085e40 con 0x7fabd80c7940 2023-10-06T05:24:53.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.543+0000 7fabd77fe700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc4066f30 0x7fabc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.544+0000 7fabd77fe700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc4066f30 0x7fabc40693e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.547+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabcc053220 con 0x7fabd80c7940 2023-10-06T05:24:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:53.830+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_mount_after_evicted_client"]} v 0) v1 -- 0x7fabd8159e90 con 0x7fabd80c7940 2023-10-06T05:24:54.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.325+0000 7fabbf7fe700 1 -- 172.21.15.33:0/2220439049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_mount_after_evicted_client"]}]=0 v541) v1 ==== 166+0+0 (secure 0 0 0) 0x7fabcc056840 con 0x7fabd80c7940 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc4066f30 msgr2=0x7fabc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc4066f30 0x7fabc40693e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8008040 comp rx=0 tx=0).stop 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 msgr2=0x7fabd812ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd812ea60 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabd80719c0 tx=0x7fabcc017000 comp rx=0 tx=0).stop 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 shutdown_connections 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fabc4066f30 0x7fabc40693e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabd813faa0 0x7fabd8158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabd806aad0 0x7fabd8159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 --2- 172.21.15.33:0/2220439049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabd80c7940 0x7fabd812ea60 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 >> 172.21.15.33:0/2220439049 conn(0x7fabd806d030 msgr2=0x7fabd80c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 shutdown_connections 2023-10-06T05:24:54.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.328+0000 7fabdcb9d700 1 -- 172.21.15.33:0/2220439049 wait complete. 2023-10-06T05:24:54.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:24:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.540+0000 7f7b2759e700 1 Processor -- start 2023-10-06T05:24:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.540+0000 7f7b2759e700 1 -- start start 2023-10-06T05:24:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b28156370 0x7f7b280af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b280cc7e0 0x7f7b280c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7b280b01f0 0x7f7b280b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b280ad430 con 0x7f7b28156370 2023-10-06T05:24:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b280afe20 con 0x7f7b280b01f0 2023-10-06T05:24:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b0b70 con 0x7f7b280cc7e0 2023-10-06T05:24:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b28156370 0x7f7b280af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7b28156370 0x7f7b280af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51586/0 (socket says 172.21.15.33:51586) 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b2659c700 1 -- 172.21.15.33:0/2711084197 learned_addr learned my addr 172.21.15.33:0/2711084197 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b25d9b700 1 --2- 172.21.15.33:0/2711084197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b280cc7e0 0x7f7b280c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.542+0000 7f7b26d9d700 1 --2- 172.21.15.33:0/2711084197 >> v2:172.21.15.167:3300/0 conn(0x7f7b280b01f0 0x7f7b280b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2659c700 1 -- 172.21.15.33:0/2711084197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b280cc7e0 msgr2=0x7f7b280c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2659c700 1 --2- 172.21.15.33:0/2711084197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b280cc7e0 0x7f7b280c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2659c700 1 -- 172.21.15.33:0/2711084197 >> v2:172.21.15.167:3300/0 conn(0x7f7b280b01f0 msgr2=0x7f7b280b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2659c700 1 --2- 172.21.15.33:0/2711084197 >> v2:172.21.15.167:3300/0 conn(0x7f7b280b01f0 0x7f7b280b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2659c700 1 -- 172.21.15.33:0/2711084197 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2806bc70 con 0x7f7b28156370 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2659c700 1 --2- 172.21.15.33:0/2711084197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b28156370 0x7f7b280af8e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f7b28069810 tx=0x7f7b1800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a71acc19044ba46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2559a700 1 -- 172.21.15.33:0/2711084197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800dc50 con 0x7f7b28156370 2023-10-06T05:24:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2559a700 1 -- 172.21.15.33:0/2711084197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1800ddb0 con 0x7f7b28156370 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.543+0000 7f7b2559a700 1 -- 172.21.15.33:0/2711084197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b180196d0 con 0x7f7b28156370 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 -- 172.21.15.33:0/2711084197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b28156370 msgr2=0x7f7b280af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 --2- 172.21.15.33:0/2711084197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b28156370 0x7f7b280af8e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f7b28069810 tx=0x7f7b1800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 -- 172.21.15.33:0/2711084197 shutdown_connections 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 --2- 172.21.15.33:0/2711084197 >> v2:172.21.15.167:3300/0 conn(0x7f7b280b01f0 0x7f7b280b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 --2- 172.21.15.33:0/2711084197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b28156370 0x7f7b280af8e0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 --2- 172.21.15.33:0/2711084197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b280cc7e0 0x7f7b280c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 -- 172.21.15.33:0/2711084197 >> 172.21.15.33:0/2711084197 conn(0x7f7b2806d030 msgr2=0x7f7b280cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 -- 172.21.15.33:0/2711084197 shutdown_connections 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.544+0000 7f7b2759e700 1 -- 172.21.15.33:0/2711084197 wait complete. 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.545+0000 7f7b2759e700 1 Processor -- start 2023-10-06T05:24:54.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.546+0000 7f7b2759e700 1 -- start start 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.546+0000 7f7b2759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 0x7f7b28150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b280cc7e0 0x7f7b28154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b28156370 0x7f7b28154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280a5f80 con 0x7f7b280b01f0 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281662f0 con 0x7f7b280cc7e0 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281665d0 con 0x7f7b28156370 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 0x7f7b28150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 0x7f7b28150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51598/0 (socket says 172.21.15.33:51598) 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 -- 172.21.15.33:0/77246263 learned_addr learned my addr 172.21.15.33:0/77246263 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b25d9b700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b280cc7e0 0x7f7b28154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b26d9d700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b28156370 0x7f7b28154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 -- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b28156370 msgr2=0x7f7b28154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b28156370 0x7f7b28154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 -- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b280cc7e0 msgr2=0x7f7b28154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b280cc7e0 0x7f7b28154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b2659c700 1 -- 172.21.15.33:0/77246263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1800b790 con 0x7f7b280b01f0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.547+0000 7f7b25d9b700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b280cc7e0 0x7f7b28154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.548+0000 7f7b2659c700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 0x7f7b28150ca0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b760 tx=0x7f7b1801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.548+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18010470 con 0x7f7b280b01f0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.548+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b180105d0 con 0x7f7b280b01f0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.548+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b281668b0 con 0x7f7b280b01f0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.548+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18010990 con 0x7f7b280b01f0 2023-10-06T05:24:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.548+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b28166e70 con 0x7f7b280b01f0 2023-10-06T05:24:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.549+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2804c380 con 0x7f7b280b01f0 2023-10-06T05:24:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.550+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b18022030 con 0x7f7b280b01f0 2023-10-06T05:24:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.552+0000 7f7b167fc700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7b08066f30 0x7f7b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.552+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7b18014070 con 0x7f7b280b01f0 2023-10-06T05:24:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.552+0000 7f7b25d9b700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7b08066f30 0x7f7b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.553+0000 7f7b25d9b700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7b08066f30 0x7f7b080693e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7b280acf40 tx=0x7f7b1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.556+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b18055980 con 0x7f7b280b01f0 2023-10-06T05:24:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.757+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b2806f310 con 0x7f7b280b01f0 2023-10-06T05:24:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.761+0000 7f7b167fc700 1 -- 172.21.15.33:0/77246263 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b18058f60 con 0x7f7b280b01f0 2023-10-06T05:24:54.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7b08066f30 msgr2=0x7f7b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7b08066f30 0x7f7b080693e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7b280acf40 tx=0x7f7b1c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:54.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 msgr2=0x7f7b28150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 0x7f7b28150ca0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b760 tx=0x7f7b1801b000 comp rx=0 tx=0).stop 2023-10-06T05:24:54.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 shutdown_connections 2023-10-06T05:24:54.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7b08066f30 0x7f7b080693e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7b28156370 0x7f7b28154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b280b01f0 0x7f7b28150ca0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 --2- 172.21.15.33:0/77246263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b280cc7e0 0x7f7b28154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 >> 172.21.15.33:0/77246263 conn(0x7f7b2806d030 msgr2=0x7f7b280cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 shutdown_connections 2023-10-06T05:24:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:54.764+0000 7f7b2759e700 1 -- 172.21.15.33:0/77246263 wait complete. 2023-10-06T05:24:54.784 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:24:54.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:54.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:54.954 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:24:54.954 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:24:54.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.140+0000 7f23e4d9d700 1 Processor -- start 2023-10-06T05:24:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.141+0000 7f23e4d9d700 1 -- start start 2023-10-06T05:24:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23e4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23e4d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23e00c7940 0x7f23e00c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23e4d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00c9cb0 con 0x7f23e006aad0 2023-10-06T05:24:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23e4d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e006b660 con 0x7f23e00c7940 2023-10-06T05:24:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23e4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00ca270 con 0x7f23e013faa0 2023-10-06T05:24:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23deffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23e00c7940 0x7f23e00c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23deffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23e00c7940 0x7f23e00c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51612/0 (socket says 172.21.15.33:51612) 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23deffd700 1 -- 172.21.15.33:0/2618572196 learned_addr learned my addr 172.21.15.33:0/2618572196 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23df7fe700 1 --2- 172.21.15.33:0/2618572196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23deffd700 1 -- 172.21.15.33:0/2618572196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 msgr2=0x7f23e013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23deffd700 1 --2- 172.21.15.33:0/2618572196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.142+0000 7f23deffd700 1 -- 172.21.15.33:0/2618572196 >> v1:172.21.15.167:6789/0 conn(0x7f23e006aad0 legacy=0x7f23e006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.143+0000 7f23deffd700 1 -- 172.21.15.33:0/2618572196 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e006b840 con 0x7f23e00c7940 2023-10-06T05:24:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.143+0000 7f23deffd700 1 --2- 172.21.15.33:0/2618572196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e00c7db0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f23d40099b0 tx=0x7f23d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76aee70bcf08f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.143+0000 7f23de7fc700 1 -- 172.21.15.33:0/2618572196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d400b860 con 0x7f23e00c7940 2023-10-06T05:24:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.143+0000 7f23de7fc700 1 -- 172.21.15.33:0/2618572196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d400b9c0 con 0x7f23e00c7940 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.143+0000 7f23de7fc700 1 -- 172.21.15.33:0/2618572196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d400bd60 con 0x7f23e00c7940 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2618572196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 msgr2=0x7f23e00c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2618572196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e00c7db0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f23d40099b0 tx=0x7f23d4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2618572196 shutdown_connections 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2618572196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2618572196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e00c7db0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2618572196 >> 172.21.15.33:0/2618572196 conn(0x7f23e006d030 msgr2=0x7f23e006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2618572196 shutdown_connections 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.144+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2618572196 wait complete. 2023-10-06T05:24:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.145+0000 7f23e4d9d700 1 Processor -- start 2023-10-06T05:24:55.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.145+0000 7f23e4d9d700 1 -- start start 2023-10-06T05:24:55.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23e4d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e006aad0 0x7f23e01393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23e4d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e0156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23e4d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e0140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23e4d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e01564e0 con 0x7f23e00c7940 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23e4d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0158ca0 con 0x7f23e006aad0 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23e4d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0139930 con 0x7f23e013faa0 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.146+0000 7f23deffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e0156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e0156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51614/0 (socket says 172.21.15.33:51614) 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 -- 172.21.15.33:0/2938822772 learned_addr learned my addr 172.21.15.33:0/2938822772 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23dffff700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e0140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 -- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 msgr2=0x7f23e0140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23df7fe700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e006aad0 0x7f23e01393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e0140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 -- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e006aad0 msgr2=0x7f23e01393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e006aad0 0x7f23e01393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 -- 172.21.15.33:0/2938822772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40094f0 con 0x7f23e00c7940 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23dffff700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e0140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23df7fe700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e006aad0 0x7f23e01393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23deffd700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e0156e50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f23e00719c0 tx=0x7f23d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4024440 con 0x7f23e00c7940 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d40245c0 con 0x7f23e00c7940 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.147+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e013a430 con 0x7f23e00c7940 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.148+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4024980 con 0x7f23e00c7940 2023-10-06T05:24:55.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.148+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e0139e70 con 0x7f23e00c7940 2023-10-06T05:24:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.148+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e0158a70 con 0x7f23e00c7940 2023-10-06T05:24:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.150+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23d401a420 con 0x7f23e00c7940 2023-10-06T05:24:55.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.150+0000 7f23c77fe700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23cc066f30 0x7f23cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.150+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f23d40861a0 con 0x7f23e00c7940 2023-10-06T05:24:55.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.151+0000 7f23df7fe700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23cc066f30 0x7f23cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.152+0000 7f23df7fe700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23cc066f30 0x7f23cc0693e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009990 tx=0x7f23d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.154+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23e0158a70 con 0x7f23e00c7940 2023-10-06T05:24:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.403+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23e0158a70 con 0x7f23e00c7940 2023-10-06T05:24:55.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.404+0000 7f23c77fe700 1 -- 172.21.15.33:0/2938822772 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3539 (secure 0 0 0) 0x7f23d4015df0 con 0x7f23e00c7940 2023-10-06T05:24:55.405 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:55.406 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/4055078134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4055078134},{"type":"v1","addr":"172.21.15.33:6834","nonce":4055078134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/4095505213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":4095505213},{"type":"v1","addr":"172.21.15.167:6836","nonce":4095505213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3485925688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3485925688},{"type":"v1","addr":"172.21.15.167:6837","nonce":3485925688}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":8637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3200724149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3200724149},{"type":"v1","addr":"172.21.15.33:6835","nonce":3200724149}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[]} 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23cc066f30 msgr2=0x7f23cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23cc066f30 0x7f23cc0693e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009990 tx=0x7f23d0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 msgr2=0x7f23e0156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e0156e50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f23e00719c0 tx=0x7f23d4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 shutdown_connections 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23cc066f30 0x7f23cc0693e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e013faa0 0x7f23e0140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e006aad0 0x7f23e01393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 --2- 172.21.15.33:0/2938822772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00c7940 0x7f23e0156e50 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 >> 172.21.15.33:0/2938822772 conn(0x7f23e006d030 msgr2=0x7f23e00b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:55.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 shutdown_connections 2023-10-06T05:24:55.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.406+0000 7f23e4d9d700 1 -- 172.21.15.33:0/2938822772 wait complete. 2023-10-06T05:24:55.409 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 107 2023-10-06T05:24:55.425 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:24:55.426 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:24:55.426 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:24:55.426 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:55.426 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:24:55.426 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:24:55.426 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:24:55.427 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:55.427 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:24:55.427 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:24:55.427 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:24:55.427 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:55.427 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:24:55.427 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:24:55.428 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:24:55.428 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:55.440 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:24:55.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:24:55.441 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:24:55.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:24:55.443 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:24:55.443 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:24:55.444 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:24:55.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:24:55.483 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:24:55.483 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:55.486 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:24:55.486 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:55.488 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:24:55.488 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:55.491 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:24:55.491 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:55.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:24:55.622 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:24:55.623 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:24:55.630 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:24:55.630 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:24:55.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.689+0000 7f764b59d700 1 Processor -- start 2023-10-06T05:24:55.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.689+0000 7f764b59d700 1 -- start start 2023-10-06T05:24:55.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.690+0000 7f764b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 0x7f764406a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.690+0000 7f764b59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f76440b2480 0x7f764406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764b59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764415b6c0 con 0x7f764415b100 2023-10-06T05:24:55.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764b59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764415b9a0 con 0x7f76440b2480 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764415bc80 con 0x7f76440c7210 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 0x7f764406a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f7649d9a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f76440b2480 0x7f764406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 0x7f764406a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:53172/0 (socket says 172.21.15.33:53172) 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 -- 172.21.15.33:0/4137757681 learned_addr learned my addr 172.21.15.33:0/4137757681 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 -- 172.21.15.33:0/4137757681 >> v2:172.21.15.167:3300/0 conn(0x7f76440b2480 msgr2=0x7f764406a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 --2- 172.21.15.33:0/4137757681 >> v2:172.21.15.167:3300/0 conn(0x7f76440b2480 0x7f764406a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 -- 172.21.15.33:0/4137757681 >> v1:172.21.15.167:6789/0 conn(0x7f764415b100 legacy=0x7f7644156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.691+0000 7f764a59b700 1 -- 172.21.15.33:0/4137757681 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764415bf60 con 0x7f76440c7210 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.692+0000 7f764a59b700 1 --2- 172.21.15.33:0/4137757681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 0x7f764406a260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76400099b0 tx=0x7f7640009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc88f9e0b7797eaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.692+0000 7f7649599700 1 -- 172.21.15.33:0/4137757681 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640009ea0 con 0x7f76440c7210 2023-10-06T05:24:55.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.692+0000 7f7649599700 1 -- 172.21.15.33:0/4137757681 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000b860 con 0x7f76440c7210 2023-10-06T05:24:55.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.692+0000 7f7649599700 1 -- 172.21.15.33:0/4137757681 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000bc00 con 0x7f76440c7210 2023-10-06T05:24:55.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.692+0000 7f764b59d700 1 -- 172.21.15.33:0/4137757681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 msgr2=0x7f764406a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.692+0000 7f764b59d700 1 --2- 172.21.15.33:0/4137757681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 0x7f764406a260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76400099b0 tx=0x7f7640009250 comp rx=0 tx=0).stop 2023-10-06T05:24:55.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.693+0000 7f764b59d700 1 -- 172.21.15.33:0/4137757681 shutdown_connections 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.693+0000 7f764b59d700 1 --2- 172.21.15.33:0/4137757681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440c7210 0x7f764406a260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.693+0000 7f764b59d700 1 --2- 172.21.15.33:0/4137757681 >> v2:172.21.15.167:3300/0 conn(0x7f76440b2480 0x7f764406a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.693+0000 7f764b59d700 1 -- 172.21.15.33:0/4137757681 >> 172.21.15.33:0/4137757681 conn(0x7f764406e650 msgr2=0x7f764406ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.693+0000 7f764b59d700 1 -- 172.21.15.33:0/4137757681 shutdown_connections 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.693+0000 7f764b59d700 1 -- 172.21.15.33:0/4137757681 wait complete. 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.694+0000 7f764b59d700 1 Processor -- start 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.694+0000 7f764b59d700 1 -- start start 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440b2480 0x7f7644158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76440c7210 0x7f7644148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 0x7f76440c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c59e0 con 0x7f764415b100 2023-10-06T05:24:55.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c4dd0 con 0x7f76440c7210 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c50b0 con 0x7f76440b2480 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.695+0000 7f764ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 0x7f76440c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 0x7f76440c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51626/0 (socket says 172.21.15.33:51626) 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 -- 172.21.15.33:0/966716714 learned_addr learned my addr 172.21.15.33:0/966716714 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f7649d9a700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76440c7210 0x7f7644148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764a59b700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440b2480 0x7f7644158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 -- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440b2480 msgr2=0x7f7644158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440b2480 0x7f7644158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 -- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76440c7210 msgr2=0x7f7644148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76440c7210 0x7f7644148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 -- 172.21.15.33:0/966716714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400094f0 con 0x7f764415b100 2023-10-06T05:24:55.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.696+0000 7f764ad9c700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 0x7f76440c2600 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f763c00c840 tx=0x7f763c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.697+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c012630 con 0x7f764415b100 2023-10-06T05:24:55.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.697+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0127b0 con 0x7f764415b100 2023-10-06T05:24:55.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.697+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76440c5390 con 0x7f764415b100 2023-10-06T05:24:55.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.697+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c012b70 con 0x7f764415b100 2023-10-06T05:24:55.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.697+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7644148aa0 con 0x7f764415b100 2023-10-06T05:24:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.698+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7644069e60 con 0x7f764415b100 2023-10-06T05:24:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.699+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f763c01a3a0 con 0x7f764415b100 2023-10-06T05:24:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.699+0000 7f763a7fc700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7628066f30 0x7f76280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.700+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f763c084540 con 0x7f764415b100 2023-10-06T05:24:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.700+0000 7f764a59b700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7628066f30 0x7f76280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.700+0000 7f764a59b700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7628066f30 0x7f76280693e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f76400099b0 tx=0x7f7640017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.704+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f763c051880 con 0x7f764415b100 2023-10-06T05:24:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:55.920+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f764404c380 con 0x7f764415b100 2023-10-06T05:24:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.337+0000 7f763a7fc700 1 -- 172.21.15.33:0/966716714 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) v1 ==== 97+0+0 (secure 0 0 0) 0x7f763c054ea0 con 0x7f764415b100 2023-10-06T05:24:56.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7628066f30 msgr2=0x7f76280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7628066f30 0x7f76280693e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f76400099b0 tx=0x7f7640017000 comp rx=0 tx=0).stop 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 msgr2=0x7f76440c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 0x7f76440c2600 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f763c00c840 tx=0x7f763c001670 comp rx=0 tx=0).stop 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 shutdown_connections 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7628066f30 0x7f76280693e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764415b100 0x7f76440c2600 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76440b2480 0x7f7644158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 --2- 172.21.15.33:0/966716714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76440c7210 0x7f7644148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 >> 172.21.15.33:0/966716714 conn(0x7f764406e650 msgr2=0x7f7644148ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 shutdown_connections 2023-10-06T05:24:56.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.340+0000 7f764b59d700 1 -- 172.21.15.33:0/966716714 wait complete. 2023-10-06T05:24:56.343 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:24:56.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:24:56.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.550+0000 7f6cc239d700 1 Processor -- start 2023-10-06T05:24:56.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.550+0000 7f6cc239d700 1 -- start start 2023-10-06T05:24:56.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.551+0000 7f6cc239d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6cbc0c8050 0x7f6cbc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.551+0000 7f6cc239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0c9cb0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.551+0000 7f6cc239d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0ca270 con 0x7f6cbc13faa0 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.551+0000 7f6cc239d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc06b640 con 0x7f6cbc0c8050 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.551+0000 7f6cc1b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6cbc06aad0 0x7f6cbc06af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40636/0 (socket says 172.21.15.33:40636) 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.551+0000 7f6cc1b9c700 1 -- 172.21.15.33:0/2674044903 learned_addr learned my addr 172.21.15.33:0/2674044903 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cc0b9a700 1 --2- 172.21.15.33:0/2674044903 >> v2:172.21.15.167:3300/0 conn(0x7f6cbc0c8050 0x7f6cbc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cb3fff700 1 -- 172.21.15.33:0/2674044903 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633034246 0 0) 0x7f6cbc0c9cb0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cb3fff700 1 -- 172.21.15.33:0/2674044903 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca8003570 con 0x7f6cbc06aad0 2023-10-06T05:24:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cc0b9a700 1 -- 172.21.15.33:0/2674044903 >> v1:172.21.15.167:6789/0 conn(0x7f6cbc13faa0 legacy=0x7f6cbc0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cc0b9a700 1 -- 172.21.15.33:0/2674044903 >> v1:172.21.15.33:6789/0 conn(0x7f6cbc06aad0 legacy=0x7f6cbc06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cb3fff700 1 -- 172.21.15.33:0/2674044903 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677600567 0 0) 0x7f6ca8003570 con 0x7f6cbc06aad0 2023-10-06T05:24:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.552+0000 7f6cc0b9a700 1 -- 172.21.15.33:0/2674044903 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc06b840 con 0x7f6cbc0c8050 2023-10-06T05:24:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cc0b9a700 1 --2- 172.21.15.33:0/2674044903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc0c8050 0x7f6cbc13fe70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0098e0 tx=0x7f6cac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5530b8e68ee09e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cb3fff700 1 -- 172.21.15.33:0/2674044903 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40017f0 con 0x7f6cbc0c8050 2023-10-06T05:24:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cb3fff700 1 -- 172.21.15.33:0/2674044903 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac011420 con 0x7f6cbc0c8050 2023-10-06T05:24:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cb3fff700 1 -- 172.21.15.33:0/2674044903 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0117c0 con 0x7f6cbc0c8050 2023-10-06T05:24:56.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cc239d700 1 -- 172.21.15.33:0/2674044903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc0c8050 msgr2=0x7f6cbc13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2674044903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc0c8050 0x7f6cbc13fe70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0098e0 tx=0x7f6cac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cc239d700 1 -- 172.21.15.33:0/2674044903 shutdown_connections 2023-10-06T05:24:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2674044903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc0c8050 0x7f6cbc13fe70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.553+0000 7f6cc239d700 1 -- 172.21.15.33:0/2674044903 >> 172.21.15.33:0/2674044903 conn(0x7f6cbc06d030 msgr2=0x7f6cbc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.554+0000 7f6cc239d700 1 -- 172.21.15.33:0/2674044903 shutdown_connections 2023-10-06T05:24:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.554+0000 7f6cc239d700 1 -- 172.21.15.33:0/2674044903 wait complete. 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.555+0000 7f6cc239d700 1 Processor -- start 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.555+0000 7f6cc239d700 1 -- start start 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 0x7f6cbc0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cbc0c8050 0x7f6cbc0ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc13faa0 0x7f6cbc16d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0b0850 con 0x7f6cbc06aad0 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0af9c0 con 0x7f6cbc13faa0 2023-10-06T05:24:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc156a00 con 0x7f6cbc0c8050 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 0x7f6cbc0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 0x7f6cbc0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51636/0 (socket says 172.21.15.33:51636) 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc139b700 1 -- 172.21.15.33:0/2017020051 learned_addr learned my addr 172.21.15.33:0/2017020051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.556+0000 7f6cc1b9c700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc13faa0 0x7f6cbc16d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc0b9a700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cbc0c8050 0x7f6cbc0ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc139b700 1 -- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cbc0c8050 msgr2=0x7f6cbc0ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc139b700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cbc0c8050 0x7f6cbc0ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc139b700 1 -- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc13faa0 msgr2=0x7f6cbc16d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc139b700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc13faa0 0x7f6cbc16d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc139b700 1 -- 172.21.15.33:0/2017020051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac009420 con 0x7f6cbc06aad0 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc139b700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 0x7f6cbc0c3190 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c4e0 tx=0x7f6cb800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:56.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8011590 con 0x7f6cbc06aad0 2023-10-06T05:24:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8011710 con 0x7f6cbc06aad0 2023-10-06T05:24:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8011ad0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.557+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc0c36d0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.558+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0afeb0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.558+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc04c380 con 0x7f6cbc06aad0 2023-10-06T05:24:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.559+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cb801b3b0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.561+0000 7f6cb17fa700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ca0066f30 0x7f6ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.561+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6cb80837a0 con 0x7f6cbc06aad0 2023-10-06T05:24:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.561+0000 7f6cc0b9a700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ca0066f30 0x7f6ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.562+0000 7f6cc0b9a700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ca0066f30 0x7f6ca00693e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0091e0 tx=0x7f6cac016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.565+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cb8050b60 con 0x7f6cbc06aad0 2023-10-06T05:24:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.843+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6cbc06bb20 con 0x7f6cbc06aad0 2023-10-06T05:24:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.844+0000 7f6cb17fa700 1 -- 172.21.15.33:0/2017020051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6cb8017020 con 0x7f6cbc06aad0 2023-10-06T05:24:56.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:56.846 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:24:56.849 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:24:56.853 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.854 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:24:56.856 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.857 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:24:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:24:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:24:56.860 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:56.861 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:24:56.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:56.862 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:24:56.862 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ca0066f30 msgr2=0x7f6ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ca0066f30 0x7f6ca00693e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0091e0 tx=0x7f6cac016000 comp rx=0 tx=0).stop 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 msgr2=0x7f6cbc0c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 0x7f6cbc0c3190 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c4e0 tx=0x7f6cb800c7f0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 shutdown_connections 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ca0066f30 0x7f6ca00693e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cbc13faa0 0x7f6cbc16d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cbc06aad0 0x7f6cbc0c3190 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 --2- 172.21.15.33:0/2017020051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6cbc0c8050 0x7f6cbc0ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 >> 172.21.15.33:0/2017020051 conn(0x7f6cbc06d030 msgr2=0x7f6cbc13eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 shutdown_connections 2023-10-06T05:24:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:56.846+0000 7f6cc239d700 1 -- 172.21.15.33:0/2017020051 wait complete. 2023-10-06T05:24:56.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:57.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.050+0000 7fb28639d700 1 Processor -- start 2023-10-06T05:24:57.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.051+0000 7fb28639d700 1 -- start start 2023-10-06T05:24:57.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb28639d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb28006ab00 0x7fb2800b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb28639d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28015e2d0 con 0x7fb2800b2e30 2023-10-06T05:24:57.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb28639d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28006afa0 con 0x7fb2800b0a40 2023-10-06T05:24:57.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb28639d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28015d6c0 con 0x7fb28006ab00 2023-10-06T05:24:57.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb285b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb2800b2e30 0x7fb28015a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40652/0 (socket says 172.21.15.33:40652) 2023-10-06T05:24:57.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb285b9c700 1 -- 172.21.15.33:0/2543891264 learned_addr learned my addr 172.21.15.33:0/2543891264 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.052+0000 7fb284b9a700 1 --2- 172.21.15.33:0/2543891264 >> v2:172.21.15.167:3300/0 conn(0x7fb28006ab00 0x7fb2800b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493954139 0 0) 0x7fb28015e2d0 con 0x7fb2800b2e30 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26c003570 con 0x7fb2800b2e30 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819645090 0 0) 0x7fb28006afa0 con 0x7fb2800b0a40 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb28015e2d0 con 0x7fb2800b0a40 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3888846742 0 0) 0x7fb26c003570 con 0x7fb2800b2e30 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 >> v2:172.21.15.167:3300/0 conn(0x7fb28006ab00 msgr2=0x7fb2800b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:57.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 --2- 172.21.15.33:0/2543891264 >> v2:172.21.15.167:3300/0 conn(0x7fb28006ab00 0x7fb2800b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 >> v1:172.21.15.167:6789/0 conn(0x7fb2800b0a40 legacy=0x7fb2800b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28015d9a0 con 0x7fb2800b2e30 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb2780017f0 con 0x7fb2800b2e30 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 >> v1:172.21.15.33:6789/0 conn(0x7fb2800b2e30 legacy=0x7fb28015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb26c003c40 0x7fb26c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb26c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26c005280 0x7fb26c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28015d6c0 con 0x7fb26c004810 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c00c250 con 0x7fb26c005280 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c00c410 con 0x7fb26c003c40 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb284b9a700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb26c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.053+0000 7fb285b9c700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26c005280 0x7fb26c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb28539b700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb26c003c40 0x7fb26c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb284b9a700 1 -- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb26c003c40 msgr2=0x7fb26c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb284b9a700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb26c003c40 0x7fb26c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb284b9a700 1 -- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26c005280 msgr2=0x7fb26c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb284b9a700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26c005280 0x7fb26c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb284b9a700 1 -- 172.21.15.33:0/2543891264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c003570 con 0x7fb26c004810 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb285b9c700 1 -- 172.21.15.33:0/2543891264 reap_dead start 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb28539b700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb26c003c40 0x7fb26c004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb284b9a700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb26c004c80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb270009c00 tx=0x7fb2700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2453ae9ad28a30b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000e710 con 0x7fb26c004810 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27000e870 con 0x7fb26c004810 2023-10-06T05:24:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.054+0000 7fb277fff700 1 -- 172.21.15.33:0/2543891264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000ec30 con 0x7fb26c004810 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 -- 172.21.15.33:0/2543891264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 msgr2=0x7fb26c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb26c004c80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb270009c00 tx=0x7fb2700091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 -- 172.21.15.33:0/2543891264 shutdown_connections 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 --2- 172.21.15.33:0/2543891264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb26c004c80 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 -- 172.21.15.33:0/2543891264 >> 172.21.15.33:0/2543891264 conn(0x7fb28006e0f0 msgr2=0x7fb2800675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 -- 172.21.15.33:0/2543891264 shutdown_connections 2023-10-06T05:24:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.055+0000 7fb28639d700 1 -- 172.21.15.33:0/2543891264 wait complete. 2023-10-06T05:24:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.056+0000 7fb28639d700 1 Processor -- start 2023-10-06T05:24:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 -- start start 2023-10-06T05:24:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb280147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28015c880 0x7fb28015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb28016a610 0x7fb2800c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28015c3f0 con 0x7fb26c004810 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28016aa90 con 0x7fb28015c880 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.057+0000 7fb28639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28015b7e0 con 0x7fb28016a610 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb280147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb280147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51656/0 (socket says 172.21.15.33:51656) 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 -- 172.21.15.33:0/3929402225 learned_addr learned my addr 172.21.15.33:0/3929402225 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb284b9a700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28015c880 0x7fb28015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb285b9c700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb28016a610 0x7fb2800c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 -- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb28016a610 msgr2=0x7fb2800c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb28016a610 0x7fb2800c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 -- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28015c880 msgr2=0x7fb28015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28015c880 0x7fb28015b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 -- 172.21.15.33:0/3929402225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28015bac0 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb285b9c700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb28016a610 0x7fb2800c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.058+0000 7fb28539b700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb280147e10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e5e0 tx=0x7fb27c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.059+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c012660 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.059+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c0127c0 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.059+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c012b60 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.059+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28015bda0 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.059+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2800c4bc0 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.060+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28006fcc0 con 0x7fb26c004810 2023-10-06T05:24:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.061+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb27c01d460 con 0x7fb26c004810 2023-10-06T05:24:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.061+0000 7fb2757fa700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb264066f30 0x7fb2640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.062+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb27c0846b0 con 0x7fb26c004810 2023-10-06T05:24:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.062+0000 7fb284b9a700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb264066f30 0x7fb2640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.063+0000 7fb284b9a700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb264066f30 0x7fb2640693e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb2700091e0 tx=0x7fb27001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.065+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb27c051a70 con 0x7fb26c004810 2023-10-06T05:24:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.267+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb28004c380 con 0x7fb26c004810 2023-10-06T05:24:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.268+0000 7fb2757fa700 1 -- 172.21.15.33:0/3929402225 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fb27c055090 con 0x7fb26c004810 2023-10-06T05:24:57.270 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:57.270 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:24:56.343797+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb264066f30 msgr2=0x7fb2640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb264066f30 0x7fb2640693e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb2700091e0 tx=0x7fb27001a000 comp rx=0 tx=0).stop 2023-10-06T05:24:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 msgr2=0x7fb280147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb280147e10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e5e0 tx=0x7fb27c009040 comp rx=0 tx=0).stop 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 shutdown_connections 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb264066f30 0x7fb2640693e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb28016a610 0x7fb2800c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb26c004810 0x7fb280147e10 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 --2- 172.21.15.33:0/3929402225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28015c880 0x7fb28015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.270+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 >> 172.21.15.33:0/3929402225 conn(0x7fb28006e0f0 msgr2=0x7fb2800abb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.271+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 shutdown_connections 2023-10-06T05:24:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.271+0000 7fb28639d700 1 -- 172.21.15.33:0/3929402225 wait complete. 2023-10-06T05:24:57.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:24:57.290 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.480+0000 7fa04cb9d700 1 Processor -- start 2023-10-06T05:24:57.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.480+0000 7fa04cb9d700 1 -- start start 2023-10-06T05:24:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.481+0000 7fa04cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480cc7e0 0x7fa0480af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.481+0000 7fa04cb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0480ad430 con 0x7fa048156370 2023-10-06T05:24:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.481+0000 7fa04cb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0480afe20 con 0x7fa0480b01f0 2023-10-06T05:24:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.481+0000 7fa04cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04806bc70 con 0x7fa0480cc7e0 2023-10-06T05:24:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0477fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa048156370 0x7fa0480c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40654/0 (socket says 172.21.15.33:40654) 2023-10-06T05:24:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0477fe700 1 -- 172.21.15.33:0/1109864550 learned_addr learned my addr 172.21.15.33:0/1109864550 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa046ffd700 1 --2- 172.21.15.33:0/1109864550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480cc7e0 0x7fa0480af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810446873 0 0) 0x7fa0480ad430 con 0x7fa048156370 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa030003570 con 0x7fa048156370 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936990107 0 0) 0x7fa0480afe20 con 0x7fa0480b01f0 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0480ad430 con 0x7fa0480b01f0 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732661081 0 0) 0x7fa030003570 con 0x7fa048156370 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.482+0000 7fa046ffd700 1 -- 172.21.15.33:0/1109864550 >> v1:172.21.15.167:6789/0 conn(0x7fa0480b01f0 legacy=0x7fa0480ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.483+0000 7fa046ffd700 1 -- 172.21.15.33:0/1109864550 >> v1:172.21.15.33:6789/0 conn(0x7fa048156370 legacy=0x7fa0480c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.483+0000 7fa046ffd700 1 -- 172.21.15.33:0/1109864550 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480cccc0 con 0x7fa0480cc7e0 2023-10-06T05:24:57.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.483+0000 7fa046ffd700 1 --2- 172.21.15.33:0/1109864550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480cc7e0 0x7fa0480af8e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009910 tx=0x7fa03c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=effbab97a791e897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.483+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038001470 con 0x7fa0480cc7e0 2023-10-06T05:24:57.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.483+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c011420 con 0x7fa0480cc7e0 2023-10-06T05:24:57.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.483+0000 7fa0467fc700 1 -- 172.21.15.33:0/1109864550 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0117c0 con 0x7fa0480cc7e0 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 -- 172.21.15.33:0/1109864550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480cc7e0 msgr2=0x7fa0480af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/1109864550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480cc7e0 0x7fa0480af8e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009910 tx=0x7fa03c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 -- 172.21.15.33:0/1109864550 shutdown_connections 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/1109864550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480cc7e0 0x7fa0480af8e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 -- 172.21.15.33:0/1109864550 >> 172.21.15.33:0/1109864550 conn(0x7fa04806d030 msgr2=0x7fa0480cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 -- 172.21.15.33:0/1109864550 shutdown_connections 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.484+0000 7fa04cb9d700 1 -- 172.21.15.33:0/1109864550 wait complete. 2023-10-06T05:24:57.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.485+0000 7fa04cb9d700 1 Processor -- start 2023-10-06T05:24:57.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.485+0000 7fa04cb9d700 1 -- start start 2023-10-06T05:24:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.486+0000 7fa04cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480b01f0 0x7fa04816b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.486+0000 7fa04cb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 0x7fa0481510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.486+0000 7fa04cb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048156370 0x7fa0481526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.486+0000 7fa04cb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04816b5a0 con 0x7fa0480cc7e0 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.486+0000 7fa04cb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048154880 con 0x7fa048156370 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.486+0000 7fa04cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa048154b60 con 0x7fa0480b01f0 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 0x7fa0481510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 0x7fa0481510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51664/0 (socket says 172.21.15.33:51664) 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 -- 172.21.15.33:0/681012284 learned_addr learned my addr 172.21.15.33:0/681012284 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa047fff700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048156370 0x7fa0481526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa0477fe700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480b01f0 0x7fa04816b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 -- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480b01f0 msgr2=0x7fa04816b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480b01f0 0x7fa04816b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 -- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048156370 msgr2=0x7fa0481526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048156370 0x7fa0481526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 -- 172.21.15.33:0/681012284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c009420 con 0x7fa0480cc7e0 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.487+0000 7fa046ffd700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 0x7fa0481510f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0091e0 tx=0x7fa03c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.488+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c01cc90 con 0x7fa0480cc7e0 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.488+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c01cdf0 con 0x7fa0480cc7e0 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.488+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c01a6a0 con 0x7fa0480cc7e0 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.488+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa048154e40 con 0x7fa0480cc7e0 2023-10-06T05:24:57.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.488+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa048155400 con 0x7fa0480cc7e0 2023-10-06T05:24:57.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.489+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04804bb40 con 0x7fa0480cc7e0 2023-10-06T05:24:57.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.489+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa03c01aa90 con 0x7fa0480cc7e0 2023-10-06T05:24:57.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.491+0000 7fa02f7fe700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa034066f30 0x7fa0340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.491+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa03c086290 con 0x7fa0480cc7e0 2023-10-06T05:24:57.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.491+0000 7fa0477fe700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa034066f30 0x7fa0340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.492+0000 7fa0477fe700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa034066f30 0x7fa0340693e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa0380011e0 tx=0x7fa0380015c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.496+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa03c053650 con 0x7fa0480cc7e0 2023-10-06T05:24:57.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.696+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0481555f0 con 0x7fa0480cc7e0 2023-10-06T05:24:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.696+0000 7fa02f7fe700 1 -- 172.21.15.33:0/681012284 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa03c056c70 con 0x7fa0480cc7e0 2023-10-06T05:24:57.698 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:24:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa034066f30 msgr2=0x7fa0340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa034066f30 0x7fa0340693e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa0380011e0 tx=0x7fa0380015c0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 msgr2=0x7fa0481510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 0x7fa0481510f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0091e0 tx=0x7fa03c016000 comp rx=0 tx=0).stop 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 shutdown_connections 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa034066f30 0x7fa0340693e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048156370 0x7fa0481526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0480b01f0 0x7fa04816b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.698+0000 7fa04cb9d700 1 --2- 172.21.15.33:0/681012284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0480cc7e0 0x7fa0481510f0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.699+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 >> 172.21.15.33:0/681012284 conn(0x7fa04806d030 msgr2=0x7fa04806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.699+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 shutdown_connections 2023-10-06T05:24:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.699+0000 7fa04cb9d700 1 -- 172.21.15.33:0/681012284 wait complete. 2023-10-06T05:24:57.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.906+0000 7fe71cb9d700 1 Processor -- start 2023-10-06T05:24:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.906+0000 7fe71cb9d700 1 -- start start 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe71cb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7181552f0 0x7fe7180ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe71cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7180b18f0 0x7fe7180cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe71cb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7181586a0 con 0x7fe718154f20 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe71cb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71806bdf0 con 0x7fe7181552f0 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe71cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe718157ab0 con 0x7fe7180b18f0 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe716ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe7181552f0 0x7fe7180ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.907+0000 7fe7177fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe718154f20 0x7fe7180b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40658/0 (socket says 172.21.15.33:40658) 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe7177fe700 1 -- 172.21.15.33:0/761256200 learned_addr learned my addr 172.21.15.33:0/761256200 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe717fff700 1 --2- 172.21.15.33:0/761256200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7180b18f0 0x7fe7180cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe716ffd700 1 -- 172.21.15.33:0/761256200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7180b18f0 msgr2=0x7fe7180cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe716ffd700 1 --2- 172.21.15.33:0/761256200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7180b18f0 0x7fe7180cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe716ffd700 1 -- 172.21.15.33:0/761256200 >> v1:172.21.15.33:6789/0 conn(0x7fe718154f20 legacy=0x7fe7180b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe716ffd700 1 -- 172.21.15.33:0/761256200 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718157d70 con 0x7fe7181552f0 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe7167fc700 1 -- 172.21.15.33:0/761256200 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397876522 0 0) 0x7fe7181586a0 con 0x7fe718154f20 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe716ffd700 1 --2- 172.21.15.33:0/761256200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7181552f0 0x7fe7180ae720 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0098e0 tx=0x7fe70c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d6eed6245aecff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe7167fc700 1 -- 172.21.15.33:0/761256200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c009e80 con 0x7fe7181552f0 2023-10-06T05:24:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.908+0000 7fe7167fc700 1 -- 172.21.15.33:0/761256200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c00b860 con 0x7fe7181552f0 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe7167fc700 1 -- 172.21.15.33:0/761256200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00bbe0 con 0x7fe7181552f0 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 -- 172.21.15.33:0/761256200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7181552f0 msgr2=0x7fe7180ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/761256200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7181552f0 0x7fe7180ae720 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0098e0 tx=0x7fe70c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 -- 172.21.15.33:0/761256200 shutdown_connections 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/761256200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7180b18f0 0x7fe7180cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/761256200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7181552f0 0x7fe7180ae720 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 -- 172.21.15.33:0/761256200 >> 172.21.15.33:0/761256200 conn(0x7fe71806d030 msgr2=0x7fe7180cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 -- 172.21.15.33:0/761256200 shutdown_connections 2023-10-06T05:24:57.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.909+0000 7fe71cb9d700 1 -- 172.21.15.33:0/761256200 wait complete. 2023-10-06T05:24:57.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.911+0000 7fe71cb9d700 1 Processor -- start 2023-10-06T05:24:57.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.911+0000 7fe71cb9d700 1 -- start start 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe71cb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 0x7fe7180aac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe71cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe718154f20 0x7fe71806b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe71cb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe71806b890 0x7fe7180af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe71cb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7180aa780 con 0x7fe7180b18f0 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe71cb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe718151650 con 0x7fe71806b890 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe71cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7180a9890 con 0x7fe718154f20 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 0x7fe7180aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 0x7fe7180aac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51678/0 (socket says 172.21.15.33:51678) 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 -- 172.21.15.33:0/4214024579 learned_addr learned my addr 172.21.15.33:0/4214024579 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe716ffd700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe718154f20 0x7fe71806b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe717fff700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe71806b890 0x7fe7180af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 -- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe718154f20 msgr2=0x7fe71806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe718154f20 0x7fe71806b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 -- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe71806b890 msgr2=0x7fe7180af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe71806b890 0x7fe7180af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.912+0000 7fe7177fe700 1 -- 172.21.15.33:0/4214024579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009420 con 0x7fe7180b18f0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.913+0000 7fe7177fe700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 0x7fe7180aac10 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c810 tx=0x7fe708007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.913+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708005b30 con 0x7fe7180b18f0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.913+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708005c90 con 0x7fe7180b18f0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.913+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7180a9b70 con 0x7fe7180b18f0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.913+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708007300 con 0x7fe7180b18f0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.913+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7180aa130 con 0x7fe7180b18f0 2023-10-06T05:24:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.914+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7180c9b40 con 0x7fe7180b18f0 2023-10-06T05:24:57.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.915+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe70801b3b0 con 0x7fe7180b18f0 2023-10-06T05:24:57.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.916+0000 7fe6ff7fe700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe704066f30 0x7fe7040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.916+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe708083980 con 0x7fe7180b18f0 2023-10-06T05:24:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.916+0000 7fe716ffd700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe704066f30 0x7fe7040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.917+0000 7fe716ffd700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe704066f30 0x7fe7040693e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0091e0 tx=0x7fe70c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:57.920+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe708050d40 con 0x7fe7180b18f0 2023-10-06T05:24:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.123+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe71804c380 con 0x7fe7180b18f0 2023-10-06T05:24:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.124+0000 7fe6ff7fe700 1 -- 172.21.15.33:0/4214024579 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v152) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe708017020 con 0x7fe7180b18f0 2023-10-06T05:24:58.126 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:24:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe704066f30 msgr2=0x7fe7040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe704066f30 0x7fe7040693e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0091e0 tx=0x7fe70c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 msgr2=0x7fe7180aac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 0x7fe7180aac10 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c810 tx=0x7fe708007df0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 shutdown_connections 2023-10-06T05:24:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe704066f30 0x7fe7040693e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe71806b890 0x7fe7180af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7180b18f0 0x7fe7180aac10 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 --2- 172.21.15.33:0/4214024579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe718154f20 0x7fe71806b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 >> 172.21.15.33:0/4214024579 conn(0x7fe71806d030 msgr2=0x7fe718152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 shutdown_connections 2023-10-06T05:24:58.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.126+0000 7fe71cb9d700 1 -- 172.21.15.33:0/4214024579 wait complete. 2023-10-06T05:24:58.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:24:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.329+0000 7f538a99d700 1 Processor -- start 2023-10-06T05:24:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.330+0000 7f538a99d700 1 -- start start 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538a99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53840c82a0 0x7f53841641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538a99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538406c820 con 0x7f53840c6340 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538a99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538406b670 con 0x7f5384159520 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538a99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538406a160 con 0x7f53840c82a0 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538919a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f53840c6340 0x7f53840c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40670/0 (socket says 172.21.15.33:40670) 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538919a700 1 -- 172.21.15.33:0/4045134440 learned_addr learned my addr 172.21.15.33:0/4045134440 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.331+0000 7f538a19c700 1 --2- 172.21.15.33:0/4045134440 >> v2:172.21.15.33:3300/0 conn(0x7f53840c82a0 0x7f53841641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f5388999700 1 -- 172.21.15.33:0/4045134440 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085876230 0 0) 0x7f538406c820 con 0x7f53840c6340 2023-10-06T05:24:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f5388999700 1 -- 172.21.15.33:0/4045134440 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5370003570 con 0x7f53840c6340 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f538a19c700 1 -- 172.21.15.33:0/4045134440 >> v1:172.21.15.167:6789/0 conn(0x7f5384159520 legacy=0x7f53840c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f538a19c700 1 -- 172.21.15.33:0/4045134440 >> v1:172.21.15.33:6789/0 conn(0x7f53840c6340 legacy=0x7f53840c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f538a19c700 1 -- 172.21.15.33:0/4045134440 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53840c8800 con 0x7f53840c82a0 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f538a19c700 1 --2- 172.21.15.33:0/4045134440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c82a0 0x7f53841641a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f537c009e60 tx=0x7f537c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddfca00470f165a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f5388999700 1 -- 172.21.15.33:0/4045134440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c00e070 con 0x7f53840c82a0 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f5388999700 1 -- 172.21.15.33:0/4045134440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c00bb60 con 0x7f53840c82a0 2023-10-06T05:24:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.332+0000 7f5388999700 1 -- 172.21.15.33:0/4045134440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c012430 con 0x7f53840c82a0 2023-10-06T05:24:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 -- 172.21.15.33:0/4045134440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c82a0 msgr2=0x7f53841641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 --2- 172.21.15.33:0/4045134440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c82a0 0x7f53841641a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f537c009e60 tx=0x7f537c009780 comp rx=0 tx=0).stop 2023-10-06T05:24:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 -- 172.21.15.33:0/4045134440 shutdown_connections 2023-10-06T05:24:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 --2- 172.21.15.33:0/4045134440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c82a0 0x7f53841641a0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 -- 172.21.15.33:0/4045134440 >> 172.21.15.33:0/4045134440 conn(0x7f53840cc930 msgr2=0x7f53840ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 -- 172.21.15.33:0/4045134440 shutdown_connections 2023-10-06T05:24:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.333+0000 7f538a99d700 1 -- 172.21.15.33:0/4045134440 wait complete. 2023-10-06T05:24:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.334+0000 7f538a99d700 1 Processor -- start 2023-10-06T05:24:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.334+0000 7f538a99d700 1 -- start start 2023-10-06T05:24:58.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.335+0000 7f538a99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 0x7f5384150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.335+0000 7f538a99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53840c82a0 0x7f5384152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.335+0000 7f538a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5384159520 0x7f538415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.335+0000 7f538a99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538415d0c0 con 0x7f53840c6340 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.335+0000 7f538a99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538415d3a0 con 0x7f53840c82a0 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.335+0000 7f538a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538415d680 con 0x7f5384159520 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 0x7f5384150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 0x7f5384150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51692/0 (socket says 172.21.15.33:51692) 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 -- 172.21.15.33:0/1747155607 learned_addr learned my addr 172.21.15.33:0/1747155607 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538919a700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53840c82a0 0x7f5384152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538a19c700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5384159520 0x7f538415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 -- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5384159520 msgr2=0x7f538415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5384159520 0x7f538415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 -- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53840c82a0 msgr2=0x7f5384152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53840c82a0 0x7f5384152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.336+0000 7f538999b700 1 -- 172.21.15.33:0/1747155607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c009a20 con 0x7f53840c6340 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.337+0000 7f538999b700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 0x7f5384150e10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f538000c4e0 tx=0x7f538000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.337+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380011590 con 0x7f53840c6340 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.337+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380011710 con 0x7f53840c6340 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.337+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380011ad0 con 0x7f53840c6340 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.337+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538415d960 con 0x7f53840c6340 2023-10-06T05:24:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.337+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538415df20 con 0x7f53840c6340 2023-10-06T05:24:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.338+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538404c380 con 0x7f53840c6340 2023-10-06T05:24:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.339+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f538001b3b0 con 0x7f53840c6340 2023-10-06T05:24:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.340+0000 7f5379ffb700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5368066f30 0x7f53680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.340+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f53800837a0 con 0x7f53840c6340 2023-10-06T05:24:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.340+0000 7f538919a700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5368066f30 0x7f53680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.341+0000 7f538919a700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5368066f30 0x7f53680693e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5384154f10 tx=0x7f5374009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.346+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5380050b60 con 0x7f53840c6340 2023-10-06T05:24:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.554+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f53840c8ae0 con 0x7f53840c6340 2023-10-06T05:24:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.554+0000 7f5379ffb700 1 -- 172.21.15.33:0/1747155607 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5380017020 con 0x7f53840c6340 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:24:58.556 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:24:58.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:24:58.557 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:24:58.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5368066f30 msgr2=0x7f53680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5368066f30 0x7f53680693e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5384154f10 tx=0x7f5374009040 comp rx=0 tx=0).stop 2023-10-06T05:24:58.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 msgr2=0x7f5384150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 0x7f5384150e10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f538000c4e0 tx=0x7f538000c7f0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 shutdown_connections 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5368066f30 0x7f53680693e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5384159520 0x7f538415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53840c6340 0x7f5384150e10 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 --2- 172.21.15.33:0/1747155607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53840c82a0 0x7f5384152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.556+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 >> 172.21.15.33:0/1747155607 conn(0x7f53840cc930 msgr2=0x7f53841578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.557+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 shutdown_connections 2023-10-06T05:24:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.557+0000 7f538a99d700 1 -- 172.21.15.33:0/1747155607 wait complete. 2023-10-06T05:24:58.576 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:24:58.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:24:58.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.765+0000 7f9925d9d700 1 Processor -- start 2023-10-06T05:24:58.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.765+0000 7f9925d9d700 1 -- start start 2023-10-06T05:24:58.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.766+0000 7f9925d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99201597f0 0x7f992015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.766+0000 7f9925d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f992015a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.766+0000 7f9925d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992015d280 con 0x7f992006dc00 2023-10-06T05:24:58.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.766+0000 7f9925d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992015e290 con 0x7f99201597f0 2023-10-06T05:24:58.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.766+0000 7f9925d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992015de00 con 0x7f992006dfd0 2023-10-06T05:24:58.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f9924d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f992006dc00 0x7f99201590e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40686/0 (socket says 172.21.15.33:40686) 2023-10-06T05:24:58.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f9924d9b700 1 -- 172.21.15.33:0/2172268906 learned_addr learned my addr 172.21.15.33:0/2172268906 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:24:58.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f991ffff700 1 --2- 172.21.15.33:0/2172268906 >> v2:172.21.15.33:3300/0 conn(0x7f99201597f0 0x7f992015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f992559c700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f992015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f991ffff700 1 -- 172.21.15.33:0/2172268906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 msgr2=0x7f992015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f991ffff700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f992015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f991f7fe700 1 -- 172.21.15.33:0/2172268906 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166645214 0 0) 0x7f992015d280 con 0x7f992006dc00 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f991ffff700 1 -- 172.21.15.33:0/2172268906 >> v1:172.21.15.33:6789/0 conn(0x7f992006dc00 legacy=0x7f99201590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f991ffff700 1 -- 172.21.15.33:0/2172268906 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992015d4d0 con 0x7f99201597f0 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.767+0000 7f992559c700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f992015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f991ffff700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99201597f0 0x7f992015a030 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098e0 tx=0x7f990c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcc8cc33d2bdf01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f991f7fe700 1 -- 172.21.15.33:0/2172268906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c009e80 con 0x7f99201597f0 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f991f7fe700 1 -- 172.21.15.33:0/2172268906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00b860 con 0x7f99201597f0 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f991f7fe700 1 -- 172.21.15.33:0/2172268906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00bbe0 con 0x7f99201597f0 2023-10-06T05:24:58.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f9925d9d700 1 -- 172.21.15.33:0/2172268906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99201597f0 msgr2=0x7f992015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f9925d9d700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99201597f0 0x7f992015a030 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098e0 tx=0x7f990c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f9925d9d700 1 -- 172.21.15.33:0/2172268906 shutdown_connections 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f9925d9d700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f992015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f9925d9d700 1 --2- 172.21.15.33:0/2172268906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99201597f0 0x7f992015a030 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.768+0000 7f9925d9d700 1 -- 172.21.15.33:0/2172268906 >> 172.21.15.33:0/2172268906 conn(0x7f99200d0550 msgr2=0x7f99200d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.769+0000 7f9925d9d700 1 -- 172.21.15.33:0/2172268906 shutdown_connections 2023-10-06T05:24:58.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.769+0000 7f9925d9d700 1 -- 172.21.15.33:0/2172268906 wait complete. 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.770+0000 7f9925d9d700 1 Processor -- start 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.770+0000 7f9925d9d700 1 -- start start 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9925d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 0x7f992015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9925d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f99200c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9925d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99201597f0 0x7f99200c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9925d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992016aad0 con 0x7f992006dc00 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9925d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920169ec0 con 0x7f99201597f0 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9925d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992016a1a0 con 0x7f992006dfd0 2023-10-06T05:24:58.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9924d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 0x7f992015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9924d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 0x7f992015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51702/0 (socket says 172.21.15.33:51702) 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.771+0000 7f9924d9b700 1 -- 172.21.15.33:0/645799454 learned_addr learned my addr 172.21.15.33:0/645799454 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f992559c700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99201597f0 0x7f99200c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9924d9b700 1 -- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 msgr2=0x7f99200c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f991ffff700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f99200c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9924d9b700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f99200c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9924d9b700 1 -- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99201597f0 msgr2=0x7f99200c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9924d9b700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99201597f0 0x7f99200c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9924d9b700 1 -- 172.21.15.33:0/645799454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c009420 con 0x7f992006dc00 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f992559c700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99201597f0 0x7f99200c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f991ffff700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f99200c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9924d9b700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 0x7f992015c190 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f991400c7e0 tx=0x7f9914007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914005b30 con 0x7f992006dc00 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914005c90 con 0x7f992006dc00 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914007300 con 0x7f992006dc00 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992016a480 con 0x7f992006dc00 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.772+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9920169ae0 con 0x7f992006dc00 2023-10-06T05:24:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.773+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920063ff0 con 0x7f992006dc00 2023-10-06T05:24:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.774+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f991401b400 con 0x7f992006dc00 2023-10-06T05:24:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.775+0000 7f991cff9700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9904066f30 0x7f99040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.775+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9914083ec0 con 0x7f992006dc00 2023-10-06T05:24:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.776+0000 7f991ffff700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9904066f30 0x7f99040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.777+0000 7f991ffff700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9904066f30 0x7f99040693e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f992014c930 tx=0x7f990c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.779+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9914051300 con 0x7f992006dc00 2023-10-06T05:24:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:58.983+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f992006eb90 con 0x7f992006dc00 2023-10-06T05:24:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.791+0000 7f991cff9700 1 -- 172.21.15.33:0/645799454 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9914054920 con 0x7f992006dc00 2023-10-06T05:24:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9904066f30 msgr2=0x7f99040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9904066f30 0x7f99040693e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f992014c930 tx=0x7f990c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 msgr2=0x7f992015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 0x7f992015c190 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f991400c7e0 tx=0x7f9914007df0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 shutdown_connections 2023-10-06T05:24:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9904066f30 0x7f99040693e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99201597f0 0x7f99200c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f992006dc00 0x7f992015c190 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 --2- 172.21.15.33:0/645799454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f992006dfd0 0x7f99200c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.796+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 >> 172.21.15.33:0/645799454 conn(0x7f99200d0550 msgr2=0x7f992014cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.798+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 shutdown_connections 2023-10-06T05:24:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:24:59.799+0000 7f9925d9d700 1 -- 172.21.15.33:0/645799454 wait complete. 2023-10-06T05:24:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:24:59.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:25:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.022+0000 7fa969b9d700 1 Processor -- start 2023-10-06T05:25:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.022+0000 7fa969b9d700 1 -- start start 2023-10-06T05:25:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.023+0000 7fa969b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa9640af540 0x7fa9640af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.023+0000 7fa969b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96415ddb0 con 0x7fa96415a7d0 2023-10-06T05:25:00.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.023+0000 7fa969b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96415d1a0 con 0x7fa9640cced0 2023-10-06T05:25:00.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.023+0000 7fa969b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96415d480 con 0x7fa9640af540 2023-10-06T05:25:00.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.023+0000 7fa963fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa96415a7d0 0x7fa9640cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46644/0 (socket says 172.21.15.33:46644) 2023-10-06T05:25:00.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa963fff700 1 -- 172.21.15.33:0/3949231291 learned_addr learned my addr 172.21.15.33:0/3949231291 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:00.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa968b9b700 1 --2- 172.21.15.33:0/3949231291 >> v2:172.21.15.167:3300/0 conn(0x7fa9640af540 0x7fa9640af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa9637fe700 1 -- 172.21.15.33:0/3949231291 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915073303 0 0) 0x7fa96415ddb0 con 0x7fa96415a7d0 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa9637fe700 1 -- 172.21.15.33:0/3949231291 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa954003570 con 0x7fa96415a7d0 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa968b9b700 1 -- 172.21.15.33:0/3949231291 >> v1:172.21.15.167:6789/0 conn(0x7fa9640cced0 legacy=0x7fa9640ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa9637fe700 1 -- 172.21.15.33:0/3949231291 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779662161 0 0) 0x7fa96415d1a0 con 0x7fa9640cced0 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa968b9b700 1 -- 172.21.15.33:0/3949231291 >> v1:172.21.15.33:6789/0 conn(0x7fa96415a7d0 legacy=0x7fa9640cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.024+0000 7fa968b9b700 1 -- 172.21.15.33:0/3949231291 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96415d760 con 0x7fa9640af540 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.025+0000 7fa968b9b700 1 --2- 172.21.15.33:0/3949231291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9640af540 0x7fa9640af910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa9580098e0 tx=0x7fa9580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755bfd9b7f5fa213 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.025+0000 7fa9637fe700 1 -- 172.21.15.33:0/3949231291 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800b640 con 0x7fa9640af540 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.025+0000 7fa9637fe700 1 -- 172.21.15.33:0/3949231291 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95800b7a0 con 0x7fa9640af540 2023-10-06T05:25:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.025+0000 7fa9637fe700 1 -- 172.21.15.33:0/3949231291 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800bb40 con 0x7fa9640af540 2023-10-06T05:25:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.025+0000 7fa969b9d700 1 -- 172.21.15.33:0/3949231291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9640af540 msgr2=0x7fa9640af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.025+0000 7fa969b9d700 1 --2- 172.21.15.33:0/3949231291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9640af540 0x7fa9640af910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa9580098e0 tx=0x7fa9580091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.026+0000 7fa969b9d700 1 -- 172.21.15.33:0/3949231291 shutdown_connections 2023-10-06T05:25:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.026+0000 7fa969b9d700 1 --2- 172.21.15.33:0/3949231291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9640af540 0x7fa9640af910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.026+0000 7fa969b9d700 1 -- 172.21.15.33:0/3949231291 >> 172.21.15.33:0/3949231291 conn(0x7fa96406d9b0 msgr2=0x7fa96406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.026+0000 7fa969b9d700 1 -- 172.21.15.33:0/3949231291 shutdown_connections 2023-10-06T05:25:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.026+0000 7fa969b9d700 1 -- 172.21.15.33:0/3949231291 wait complete. 2023-10-06T05:25:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.027+0000 7fa969b9d700 1 Processor -- start 2023-10-06T05:25:00.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.027+0000 7fa969b9d700 1 -- start start 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa969b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9640af540 0x7fa9640c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa969b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 0x7fa9640b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa969b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa96415a7d0 0x7fa964158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa969b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640c7b30 con 0x7fa9640cced0 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa969b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640c8430 con 0x7fa96415a7d0 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa969b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640b3f60 con 0x7fa9640af540 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa963fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 0x7fa9640b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa963fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 0x7fa9640b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56068/0 (socket says 172.21.15.33:56068) 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.028+0000 7fa963fff700 1 -- 172.21.15.33:0/78041152 learned_addr learned my addr 172.21.15.33:0/78041152 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:00.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa96939c700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa96415a7d0 0x7fa964158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa968b9b700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9640af540 0x7fa9640c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa963fff700 1 -- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9640af540 msgr2=0x7fa9640c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa963fff700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9640af540 0x7fa9640c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa963fff700 1 -- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa96415a7d0 msgr2=0x7fa964158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa963fff700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa96415a7d0 0x7fa964158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa963fff700 1 -- 172.21.15.33:0/78041152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009420 con 0x7fa9640cced0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa963fff700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 0x7fa9640b12b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa9500011e0 tx=0x7fa950007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950005b30 con 0x7fa9640cced0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9640b4240 con 0x7fa9640cced0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.029+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa950005c90 con 0x7fa9640cced0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.030+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950005e00 con 0x7fa9640cced0 2023-10-06T05:25:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.030+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9640b17f0 con 0x7fa9640cced0 2023-10-06T05:25:00.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.031+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96404c380 con 0x7fa9640cced0 2023-10-06T05:25:00.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.032+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9500074f0 con 0x7fa9640cced0 2023-10-06T05:25:00.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.033+0000 7fa960ff9700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa948066f30 0x7fa9480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.033+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fa950083840 con 0x7fa9640cced0 2023-10-06T05:25:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.033+0000 7fa968b9b700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa948066f30 0x7fa9480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.034+0000 7fa968b9b700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa948066f30 0x7fa9480693e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa9580093f0 tx=0x7fa958017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.038+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa950050b40 con 0x7fa9640cced0 2023-10-06T05:25:00.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.237+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa96415da40 con 0x7fa9640cced0 2023-10-06T05:25:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.802+0000 7fa960ff9700 1 -- 172.21.15.33:0/78041152 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa9500170d0 con 0x7fa9640cced0 2023-10-06T05:25:00.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.810+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa948066f30 msgr2=0x7fa9480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.810+0000 7fa969b9d700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa948066f30 0x7fa9480693e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa9580093f0 tx=0x7fa958017000 comp rx=0 tx=0).stop 2023-10-06T05:25:00.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.811+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 msgr2=0x7fa9640b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.811+0000 7fa969b9d700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 0x7fa9640b12b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa9500011e0 tx=0x7fa950007df0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.813+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 shutdown_connections 2023-10-06T05:25:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.813+0000 7fa969b9d700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa948066f30 0x7fa9480693e0 secure :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa9580093f0 tx=0x7fa958017000 comp rx=0 tx=0).stop 2023-10-06T05:25:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.813+0000 7fa969b9d700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa96415a7d0 0x7fa964158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.813+0000 7fa969b9d700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9640af540 0x7fa9640c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.813+0000 7fa969b9d700 1 --2- 172.21.15.33:0/78041152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9640cced0 0x7fa9640b12b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.813+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 >> 172.21.15.33:0/78041152 conn(0x7fa96406d9b0 msgr2=0x7fa9640c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.819+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 shutdown_connections 2023-10-06T05:25:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:00.822+0000 7fa969b9d700 1 -- 172.21.15.33:0/78041152 wait complete. 2023-10-06T05:25:00.827 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:25:00.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:25:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.037+0000 7f776679d700 1 Processor -- start 2023-10-06T05:25:01.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.037+0000 7f776679d700 1 -- start start 2023-10-06T05:25:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.038+0000 7f776679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f77600c9490 0x7f77600cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.038+0000 7f776679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7760155a50 0x7f7760155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.038+0000 7f776679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600b06b0 con 0x7f77600af410 2023-10-06T05:25:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.038+0000 7f776679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600c99e0 con 0x7f77600c9490 2023-10-06T05:25:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.038+0000 7f776679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600c9cc0 con 0x7f7760155a50 2023-10-06T05:25:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f776579b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f77600c9490 0x7f77600cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f7765f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f77600af410 0x7f77600af870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46652/0 (socket says 172.21.15.33:46652) 2023-10-06T05:25:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f7765f9c700 1 -- 172.21.15.33:0/464882069 learned_addr learned my addr 172.21.15.33:0/464882069 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:01.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f776579b700 1 -- 172.21.15.33:0/464882069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7760155a50 msgr2=0x7f7760155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:01.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f7764f9a700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7760155a50 0x7f7760155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:01.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f776579b700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7760155a50 0x7f7760155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f776579b700 1 -- 172.21.15.33:0/464882069 >> v1:172.21.15.33:6789/0 conn(0x7f77600af410 legacy=0x7f77600af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f7757fff700 1 -- 172.21.15.33:0/464882069 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50828388 0 0) 0x7f77600b06b0 con 0x7f77600af410 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f776579b700 1 -- 172.21.15.33:0/464882069 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77600c9fa0 con 0x7f77600c9490 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f7764f9a700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7760155a50 0x7f7760155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.039+0000 7f776579b700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77600c9490 0x7f77600cc7e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098e0 tx=0x7f775c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c8c2065f51bd46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f7757fff700 1 -- 172.21.15.33:0/464882069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c009e80 con 0x7f77600c9490 2023-10-06T05:25:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f7757fff700 1 -- 172.21.15.33:0/464882069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c00b860 con 0x7f77600c9490 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f7757fff700 1 -- 172.21.15.33:0/464882069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c00bbe0 con 0x7f77600c9490 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 -- 172.21.15.33:0/464882069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77600c9490 msgr2=0x7f77600cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77600c9490 0x7f77600cc7e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f775c0098e0 tx=0x7f775c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 -- 172.21.15.33:0/464882069 shutdown_connections 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77600c9490 0x7f77600cc7e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 --2- 172.21.15.33:0/464882069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7760155a50 0x7f7760155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 -- 172.21.15.33:0/464882069 >> 172.21.15.33:0/464882069 conn(0x7f776006d030 msgr2=0x7f77600cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 -- 172.21.15.33:0/464882069 shutdown_connections 2023-10-06T05:25:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.040+0000 7f776679d700 1 -- 172.21.15.33:0/464882069 wait complete. 2023-10-06T05:25:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.042+0000 7f776679d700 1 Processor -- start 2023-10-06T05:25:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.042+0000 7f776679d700 1 -- start start 2023-10-06T05:25:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77600af410 0x7f776014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77600c9490 0x7f77600c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 0x7f77600c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600c82c0 con 0x7f7760155a50 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600c7fe0 con 0x7f77600af410 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7760153050 con 0x7f77600c9490 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f7765f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 0x7f77600c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f7765f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 0x7f77600c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56094/0 (socket says 172.21.15.33:56094) 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f7765f9c700 1 -- 172.21.15.33:0/2782947813 learned_addr learned my addr 172.21.15.33:0/2782947813 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f7764f9a700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77600c9490 0x7f77600c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.043+0000 7f776579b700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77600af410 0x7f776014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f7765f9c700 1 -- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77600c9490 msgr2=0x7f77600c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f7765f9c700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77600c9490 0x7f77600c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f7765f9c700 1 -- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77600af410 msgr2=0x7f776014fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f7765f9c700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77600af410 0x7f776014fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f7765f9c700 1 -- 172.21.15.33:0/2782947813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c009420 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f7765f9c700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 0x7f77600c5b30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f775800cc90 tx=0x7f7758001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800a8b0 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775800aa10 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800add0 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7760153330 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.044+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77600c56e0 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.046+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776004c380 con 0x7f7760155a50 2023-10-06T05:25:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.048+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7758007300 con 0x7f7760155a50 2023-10-06T05:25:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.048+0000 7f77557fa700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7744066f30 0x7f77440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.049+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f77580854e0 con 0x7f7760155a50 2023-10-06T05:25:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.049+0000 7f776579b700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7744066f30 0x7f77440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.050+0000 7f776579b700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7744066f30 0x7f77440693e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f775c0091e0 tx=0x7f775c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.052+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77580526b0 con 0x7f7760155a50 2023-10-06T05:25:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.297+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f77600ca280 con 0x7f7760155a50 2023-10-06T05:25:01.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.802+0000 7f77557fa700 1 -- 172.21.15.33:0/2782947813 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v110) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7758055cd0 con 0x7f7760155a50 2023-10-06T05:25:01.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.808+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7744066f30 msgr2=0x7f77440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.808+0000 7f776679d700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7744066f30 0x7f77440693e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f775c0091e0 tx=0x7f775c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:01.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.809+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 msgr2=0x7f77600c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.809+0000 7f776679d700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 0x7f77600c5b30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f775800cc90 tx=0x7f7758001a80 comp rx=0 tx=0).stop 2023-10-06T05:25:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.811+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 shutdown_connections 2023-10-06T05:25:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.811+0000 7f776679d700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7744066f30 0x7f77440693e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.811+0000 7f776679d700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7760155a50 0x7f77600c5b30 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.811+0000 7f776679d700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77600af410 0x7f776014fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.811+0000 7f776679d700 1 --2- 172.21.15.33:0/2782947813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77600c9490 0x7f77600c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.811+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 >> 172.21.15.33:0/2782947813 conn(0x7f776006d030 msgr2=0x7f7760154120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:01.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.813+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 shutdown_connections 2023-10-06T05:25:01.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:01.819+0000 7f776679d700 1 -- 172.21.15.33:0/2782947813 wait complete. 2023-10-06T05:25:01.829 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 23 and data pool 24 2023-10-06T05:25:01.854 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:02.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.042+0000 7fc5e759e700 1 Processor -- start 2023-10-06T05:25:02.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.042+0000 7fc5e759e700 1 -- start start 2023-10-06T05:25:02.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.043+0000 7fc5e759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc5e80ccfc0 0x7fc5e80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.043+0000 7fc5e759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e80cbd80 con 0x7fc5e80cf3b0 2023-10-06T05:25:02.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.043+0000 7fc5e759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e80af5f0 con 0x7fc5e80cc750 2023-10-06T05:25:02.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.043+0000 7fc5e759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e80af8d0 con 0x7fc5e80ccfc0 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e5d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc5e80cf3b0 0x7fc5e81594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46666/0 (socket says 172.21.15.33:46666) 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e5d9b700 1 -- 172.21.15.33:0/2493579914 learned_addr learned my addr 172.21.15.33:0/2493579914 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e659c700 1 --2- 172.21.15.33:0/2493579914 >> v2:172.21.15.33:3300/0 conn(0x7fc5e80ccfc0 0x7fc5e80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e559a700 1 -- 172.21.15.33:0/2493579914 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197129099 0 0) 0x7fc5e80cbd80 con 0x7fc5e80cf3b0 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e659c700 1 -- 172.21.15.33:0/2493579914 >> v1:172.21.15.167:6789/0 conn(0x7fc5e80cc750 legacy=0x7fc5e80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e659c700 1 -- 172.21.15.33:0/2493579914 >> v1:172.21.15.33:6789/0 conn(0x7fc5e80cf3b0 legacy=0x7fc5e81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e659c700 1 -- 172.21.15.33:0/2493579914 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80afbb0 con 0x7fc5e80ccfc0 2023-10-06T05:25:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.044+0000 7fc5e659c700 1 --2- 172.21.15.33:0/2493579914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80ccfc0 0x7fc5e80cc210 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009910 tx=0x7fc5d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d2379ab938b7bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e559a700 1 -- 172.21.15.33:0/2493579914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800b820 con 0x7fc5e80ccfc0 2023-10-06T05:25:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e559a700 1 -- 172.21.15.33:0/2493579914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d800b980 con 0x7fc5e80ccfc0 2023-10-06T05:25:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e559a700 1 -- 172.21.15.33:0/2493579914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800bd20 con 0x7fc5e80ccfc0 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 -- 172.21.15.33:0/2493579914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80ccfc0 msgr2=0x7fc5e80cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 --2- 172.21.15.33:0/2493579914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80ccfc0 0x7fc5e80cc210 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009910 tx=0x7fc5d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 -- 172.21.15.33:0/2493579914 shutdown_connections 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 --2- 172.21.15.33:0/2493579914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80ccfc0 0x7fc5e80cc210 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 -- 172.21.15.33:0/2493579914 >> 172.21.15.33:0/2493579914 conn(0x7fc5e806cff0 msgr2=0x7fc5e806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 -- 172.21.15.33:0/2493579914 shutdown_connections 2023-10-06T05:25:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.045+0000 7fc5e759e700 1 -- 172.21.15.33:0/2493579914 wait complete. 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.047+0000 7fc5e759e700 1 Processor -- start 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.047+0000 7fc5e759e700 1 -- start start 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e80cc750 0x7fc5e816df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5e80ccfc0 0x7fc5e8169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 0x7fc5e80c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e80c66f0 con 0x7fc5e80c5d40 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e80b3350 con 0x7fc5e80cc750 2023-10-06T05:25:02.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e816e490 con 0x7fc5e80ccfc0 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 0x7fc5e80c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 0x7fc5e80c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56106/0 (socket says 172.21.15.33:56106) 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e6d9d700 1 -- 172.21.15.33:0/3300232567 learned_addr learned my addr 172.21.15.33:0/3300232567 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e5d9b700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5e80ccfc0 0x7fc5e8169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.048+0000 7fc5e659c700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e80cc750 0x7fc5e816df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e6d9d700 1 -- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5e80ccfc0 msgr2=0x7fc5e8169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e6d9d700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5e80ccfc0 0x7fc5e8169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e6d9d700 1 -- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e80cc750 msgr2=0x7fc5e816df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e6d9d700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e80cc750 0x7fc5e816df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e6d9d700 1 -- 172.21.15.33:0/3300232567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8009420 con 0x7fc5e80c5d40 2023-10-06T05:25:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e6d9d700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 0x7fc5e80c61b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c8e0 tx=0x7fc5e0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0012630 con 0x7fc5e80c5d40 2023-10-06T05:25:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e0012790 con 0x7fc5e80c5d40 2023-10-06T05:25:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0012b50 con 0x7fc5e80c5d40 2023-10-06T05:25:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e80b35a0 con 0x7fc5e80c5d40 2023-10-06T05:25:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.049+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e80b3b60 con 0x7fc5e80c5d40 2023-10-06T05:25:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.050+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e804c380 con 0x7fc5e80c5d40 2023-10-06T05:25:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.051+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5e001a3a0 con 0x7fc5e80c5d40 2023-10-06T05:25:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.052+0000 7fc5d67fc700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc5c8066f30 0x7fc5c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.052+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc5e0084870 con 0x7fc5e80c5d40 2023-10-06T05:25:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.052+0000 7fc5e659c700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc5c8066f30 0x7fc5c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.053+0000 7fc5e659c700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc5c8066f30 0x7fc5c80693e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80093f0 tx=0x7fc5d8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.056+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5e0055370 con 0x7fc5e80c5d40 2023-10-06T05:25:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.261+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5e80b62d0 con 0x7fc5e80c5d40 2023-10-06T05:25:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.262+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3300232567 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fc5e0054f00 con 0x7fc5e80c5d40 2023-10-06T05:25:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:02.264 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":155,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:25:01.298469+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:59.024220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:00.278297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc5c8066f30 msgr2=0x7fc5c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc5c8066f30 0x7fc5c80693e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80093f0 tx=0x7fc5d8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 msgr2=0x7fc5e80c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 0x7fc5e80c61b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c8e0 tx=0x7fc5e0001710 comp rx=0 tx=0).stop 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 shutdown_connections 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc5c8066f30 0x7fc5c80693e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5e80c5d40 0x7fc5e80c61b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e80cc750 0x7fc5e816df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 --2- 172.21.15.33:0/3300232567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5e80ccfc0 0x7fc5e8169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 >> 172.21.15.33:0/3300232567 conn(0x7fc5e806cff0 msgr2=0x7fc5e814efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 shutdown_connections 2023-10-06T05:25:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.264+0000 7fc5e759e700 1 -- 172.21.15.33:0/3300232567 wait complete. 2023-10-06T05:25:02.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.470+0000 7f8021dff700 1 Processor -- start 2023-10-06T05:25:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.470+0000 7f8021dff700 1 -- start start 2023-10-06T05:25:02.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f8021dff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f801c13fb70 0x7f801c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f8021dff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c06ba40 0x7f801c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f8021dff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f801c0ca350 0x7f801c0af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f8021dff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c0ca8d0 con 0x7f801c0ca350 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f8021dff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c0cabb0 con 0x7f801c13fb70 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f8021dff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c06ad00 con 0x7f801c06ba40 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f801c0ca350 0x7f801c0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f801c0ca350 0x7f801c0af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56108/0 (socket says 172.21.15.33:56108) 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 -- 172.21.15.33:0/330118451 learned_addr learned my addr 172.21.15.33:0/330118451 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801affd700 1 --2- 172.21.15.33:0/330118451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c06ba40 0x7f801c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801b7fe700 1 --2- 172.21.15.33:0/330118451 >> v2:172.21.15.167:3300/0 conn(0x7f801c13fb70 0x7f801c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 -- 172.21.15.33:0/330118451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c06ba40 msgr2=0x7f801c0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 --2- 172.21.15.33:0/330118451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c06ba40 0x7f801c0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 -- 172.21.15.33:0/330118451 >> v2:172.21.15.167:3300/0 conn(0x7f801c13fb70 msgr2=0x7f801c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.472+0000 7f801bfff700 1 --2- 172.21.15.33:0/330118451 >> v2:172.21.15.167:3300/0 conn(0x7f801c13fb70 0x7f801c06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.473+0000 7f801bfff700 1 -- 172.21.15.33:0/330118451 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c06afe0 con 0x7f801c0ca350 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.473+0000 7f801b7fe700 1 --2- 172.21.15.33:0/330118451 >> v2:172.21.15.167:3300/0 conn(0x7f801c13fb70 0x7f801c06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.473+0000 7f801bfff700 1 --2- 172.21.15.33:0/330118451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c0ca350 0x7f801c0af000 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f801c06bf00 tx=0x7f8014009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe99a1643602ebd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.473+0000 7f801a7fc700 1 -- 172.21.15.33:0/330118451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400e070 con 0x7f801c0ca350 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.473+0000 7f801a7fc700 1 -- 172.21.15.33:0/330118451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400ba20 con 0x7f801c0ca350 2023-10-06T05:25:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.473+0000 7f801a7fc700 1 -- 172.21.15.33:0/330118451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140092e0 con 0x7f801c0ca350 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 -- 172.21.15.33:0/330118451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c0ca350 msgr2=0x7f801c0af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 --2- 172.21.15.33:0/330118451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c0ca350 0x7f801c0af000 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f801c06bf00 tx=0x7f8014009840 comp rx=0 tx=0).stop 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 -- 172.21.15.33:0/330118451 shutdown_connections 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 --2- 172.21.15.33:0/330118451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c0ca350 0x7f801c0af000 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 --2- 172.21.15.33:0/330118451 >> v2:172.21.15.167:3300/0 conn(0x7f801c13fb70 0x7f801c06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 --2- 172.21.15.33:0/330118451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c06ba40 0x7f801c0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 -- 172.21.15.33:0/330118451 >> 172.21.15.33:0/330118451 conn(0x7f801c0675f0 msgr2=0x7f801c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 -- 172.21.15.33:0/330118451 shutdown_connections 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.474+0000 7f8021dff700 1 -- 172.21.15.33:0/330118451 wait complete. 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.475+0000 7f8021dff700 1 Processor -- start 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.475+0000 7f8021dff700 1 -- start start 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f8021dff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 0x7f801c0c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f8021dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c0ca350 0x7f801c13eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f8021dff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c13fb70 0x7f801c0aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f8021dff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c167960 con 0x7f801c06ba40 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f8021dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c52e0 con 0x7f801c0ca350 2023-10-06T05:25:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f8021dff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c55c0 con 0x7f801c13fb70 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 0x7f801c0c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.476+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 0x7f801c0c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56110/0 (socket says 172.21.15.33:56110) 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 -- 172.21.15.33:0/2569035680 learned_addr learned my addr 172.21.15.33:0/2569035680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801bfff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c13fb70 0x7f801c0aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801affd700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c0ca350 0x7f801c13eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 -- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c13fb70 msgr2=0x7f801c0aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c13fb70 0x7f801c0aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 -- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c0ca350 msgr2=0x7f801c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c0ca350 0x7f801c13eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 -- 172.21.15.33:0/2569035680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014009ae0 con 0x7f801c06ba40 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801affd700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c0ca350 0x7f801c13eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f801b7fe700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 0x7f801c0c4a40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f800c00d860 tx=0x7f800c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c0125f0 con 0x7f801c06ba40 2023-10-06T05:25:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c012750 con 0x7f801c06ba40 2023-10-06T05:25:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.477+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c012af0 con 0x7f801c06ba40 2023-10-06T05:25:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.478+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c0c58a0 con 0x7f801c06ba40 2023-10-06T05:25:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.478+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c0c4f80 con 0x7f801c06ba40 2023-10-06T05:25:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.479+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c063ff0 con 0x7f801c06ba40 2023-10-06T05:25:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.480+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f800c014430 con 0x7f801c06ba40 2023-10-06T05:25:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.481+0000 7f80037fe700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8008066f30 0x7f80080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.481+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f800c084ad0 con 0x7f801c06ba40 2023-10-06T05:25:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.482+0000 7f801affd700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8008066f30 0x7f80080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.482+0000 7f801affd700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8008066f30 0x7f80080693e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f801c0aeac0 tx=0x7f8010008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.485+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f800c051b40 con 0x7f801c06ba40 2023-10-06T05:25:02.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.689+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f801c167c40 con 0x7f801c06ba40 2023-10-06T05:25:02.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.690+0000 7f80037fe700 1 -- 172.21.15.33:0/2569035680 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f800c055160 con 0x7f801c06ba40 2023-10-06T05:25:02.692 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:02.693 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":155,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:25:01.298469+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:59.024220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:00.278297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:02.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8008066f30 msgr2=0x7f80080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8008066f30 0x7f80080693e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f801c0aeac0 tx=0x7f8010008040 comp rx=0 tx=0).stop 2023-10-06T05:25:02.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 msgr2=0x7f801c0c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 0x7f801c0c4a40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f800c00d860 tx=0x7f800c008040 comp rx=0 tx=0).stop 2023-10-06T05:25:02.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 shutdown_connections 2023-10-06T05:25:02.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8008066f30 0x7f80080693e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f801c13fb70 0x7f801c0aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f801c06ba40 0x7f801c0c4a40 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 --2- 172.21.15.33:0/2569035680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c0ca350 0x7f801c13eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 >> 172.21.15.33:0/2569035680 conn(0x7f801c0675f0 msgr2=0x7f801c0cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:02.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 shutdown_connections 2023-10-06T05:25:02.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.693+0000 7f8021dff700 1 -- 172.21.15.33:0/2569035680 wait complete. 2023-10-06T05:25:02.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:25:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.905+0000 7f496559d700 1 Processor -- start 2023-10-06T05:25:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.906+0000 7f496559d700 1 -- start start 2023-10-06T05:25:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f496559d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49600cd5e0 0x7f49600af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f496559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f496015a650 0x7f496015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f496559d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496015d660 con 0x7f49600af540 2023-10-06T05:25:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f496559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600ce890 con 0x7f496015a650 2023-10-06T05:25:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f496559d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496015aa60 con 0x7f49600cd5e0 2023-10-06T05:25:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f4964d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f496015a650 0x7f496015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f495f7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49600cd5e0 0x7f49600af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f4964d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f496015a650 0x7f496015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56112/0 (socket says 172.21.15.33:56112) 2023-10-06T05:25:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.907+0000 7f4964d9c700 1 -- 172.21.15.33:0/1729444068 learned_addr learned my addr 172.21.15.33:0/1729444068 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.908+0000 7f4964d9c700 1 -- 172.21.15.33:0/1729444068 >> v2:172.21.15.167:3300/0 conn(0x7f49600cd5e0 msgr2=0x7f49600af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.908+0000 7f4964d9c700 1 --2- 172.21.15.33:0/1729444068 >> v2:172.21.15.167:3300/0 conn(0x7f49600cd5e0 0x7f49600af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.908+0000 7f4964d9c700 1 -- 172.21.15.33:0/1729444068 >> v1:172.21.15.167:6789/0 conn(0x7f49600af540 legacy=0x7f49600cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.908+0000 7f4964d9c700 1 -- 172.21.15.33:0/1729444068 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496015d890 con 0x7f496015a650 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.908+0000 7f4964d9c700 1 --2- 172.21.15.33:0/1729444068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f496015a650 0x7f496015d120 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f49600893f0 tx=0x7f4958009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1734e9a9c8dde299 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f495effd700 1 -- 172.21.15.33:0/1729444068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800e070 con 0x7f496015a650 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f495effd700 1 -- 172.21.15.33:0/1729444068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495800ba20 con 0x7f496015a650 2023-10-06T05:25:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f495effd700 1 -- 172.21.15.33:0/1729444068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580092e0 con 0x7f496015a650 2023-10-06T05:25:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 -- 172.21.15.33:0/1729444068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f496015a650 msgr2=0x7f496015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 --2- 172.21.15.33:0/1729444068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f496015a650 0x7f496015d120 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f49600893f0 tx=0x7f4958009840 comp rx=0 tx=0).stop 2023-10-06T05:25:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 -- 172.21.15.33:0/1729444068 shutdown_connections 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 --2- 172.21.15.33:0/1729444068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f496015a650 0x7f496015d120 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 --2- 172.21.15.33:0/1729444068 >> v2:172.21.15.167:3300/0 conn(0x7f49600cd5e0 0x7f49600af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 -- 172.21.15.33:0/1729444068 >> 172.21.15.33:0/1729444068 conn(0x7f496006d9b0 msgr2=0x7f496006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 -- 172.21.15.33:0/1729444068 shutdown_connections 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.909+0000 7f496559d700 1 -- 172.21.15.33:0/1729444068 wait complete. 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.911+0000 7f496559d700 1 Processor -- start 2023-10-06T05:25:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.911+0000 7f496559d700 1 -- start start 2023-10-06T05:25:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f496559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 0x7f49600c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f496559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49600cd5e0 0x7f4960157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f496559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f496015a650 0x7f49600c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f496559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c95b0 con 0x7f49600af540 2023-10-06T05:25:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f496559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c7fd0 con 0x7f49600cd5e0 2023-10-06T05:25:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f496559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c82b0 con 0x7f496015a650 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f495ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 0x7f49600c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f495ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 0x7f49600c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56126/0 (socket says 172.21.15.33:56126) 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f495ffff700 1 -- 172.21.15.33:0/2232419543 learned_addr learned my addr 172.21.15.33:0/2232419543 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f495f7fe700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49600cd5e0 0x7f4960157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.912+0000 7f4964d9c700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f496015a650 0x7f49600c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f495ffff700 1 -- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f496015a650 msgr2=0x7f49600c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f495ffff700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f496015a650 0x7f49600c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f495ffff700 1 -- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49600cd5e0 msgr2=0x7f4960157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f495ffff700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49600cd5e0 0x7f4960157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f495ffff700 1 -- 172.21.15.33:0/2232419543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4958009ae0 con 0x7f49600af540 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f495ffff700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 0x7f49600c9070 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f495000c590 tx=0x7f4950007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950011590 con 0x7f49600af540 2023-10-06T05:25:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4950011710 con 0x7f49600af540 2023-10-06T05:25:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950011ad0 con 0x7f49600af540 2023-10-06T05:25:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49600c8590 con 0x7f49600af540 2023-10-06T05:25:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.913+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49600c8b50 con 0x7f49600af540 2023-10-06T05:25:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.914+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496004c380 con 0x7f49600af540 2023-10-06T05:25:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.915+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4950000f90 con 0x7f49600af540 2023-10-06T05:25:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.917+0000 7f4947fff700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f494c066f30 0x7f494c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.917+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4950083860 con 0x7f49600af540 2023-10-06T05:25:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.917+0000 7f495f7fe700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f494c066f30 0x7f494c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:02.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.918+0000 7f495f7fe700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f494c066f30 0x7f494c0693e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f49600b28a0 tx=0x7f49540091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:02.921+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f495004b5a0 con 0x7f49600af540 2023-10-06T05:25:03.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.173+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f496015db70 con 0x7f49600af540 2023-10-06T05:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.830+0000 7f4947fff700 1 -- 172.21.15.33:0/2232419543 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v113) v1 ==== 118+0+0 (secure 0 0 0) 0x7f49500170b0 con 0x7f49600af540 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f494c066f30 msgr2=0x7f494c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f494c066f30 0x7f494c0693e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f49600b28a0 tx=0x7f49540091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 msgr2=0x7f49600c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 0x7f49600c9070 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f495000c590 tx=0x7f4950007f30 comp rx=0 tx=0).stop 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 shutdown_connections 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f494c066f30 0x7f494c0693e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f496015a650 0x7f49600c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49600af540 0x7f49600c9070 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 --2- 172.21.15.33:0/2232419543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49600cd5e0 0x7f4960157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 >> 172.21.15.33:0/2232419543 conn(0x7f496006d9b0 msgr2=0x7f4960158270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 shutdown_connections 2023-10-06T05:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:03.833+0000 7f496559d700 1 -- 172.21.15.33:0/2232419543 wait complete. 2023-10-06T05:25:03.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:25:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.047+0000 7f267f19d700 1 Processor -- start 2023-10-06T05:25:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.047+0000 7f267f19d700 1 -- start start 2023-10-06T05:25:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267f19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f26780b1f90 0x7f26780abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2678070440 0x7f2678067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2678068140 0x7f267806c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267806a2d0 con 0x7f2678068140 2023-10-06T05:25:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267f19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2678154290 con 0x7f26780b1f90 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267806ae10 con 0x7f2678070440 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267e99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2678068140 0x7f267806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267e99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2678068140 0x7f267806c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56128/0 (socket says 172.21.15.33:56128) 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267e99c700 1 -- 172.21.15.33:0/475981336 learned_addr learned my addr 172.21.15.33:0/475981336 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267e19b700 1 --2- 172.21.15.33:0/475981336 >> v2:172.21.15.167:3300/0 conn(0x7f26780b1f90 0x7f26780abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.049+0000 7f267d99a700 1 --2- 172.21.15.33:0/475981336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2678070440 0x7f2678067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267e99c700 1 -- 172.21.15.33:0/475981336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2678070440 msgr2=0x7f2678067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267e99c700 1 --2- 172.21.15.33:0/475981336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2678070440 0x7f2678067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267e99c700 1 -- 172.21.15.33:0/475981336 >> v2:172.21.15.167:3300/0 conn(0x7f26780b1f90 msgr2=0x7f26780abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267e99c700 1 --2- 172.21.15.33:0/475981336 >> v2:172.21.15.167:3300/0 conn(0x7f26780b1f90 0x7f26780abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267e99c700 1 -- 172.21.15.33:0/475981336 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678068580 con 0x7f2678068140 2023-10-06T05:25:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267e99c700 1 --2- 172.21.15.33:0/475981336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678068140 0x7f267806c560 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2678069b60 tx=0x7f2670009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9ab5d179c69954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267d199700 1 -- 172.21.15.33:0/475981336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000e070 con 0x7f2678068140 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267d199700 1 -- 172.21.15.33:0/475981336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267000b970 con 0x7f2678068140 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.050+0000 7f267d199700 1 -- 172.21.15.33:0/475981336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700092e0 con 0x7f2678068140 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 -- 172.21.15.33:0/475981336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678068140 msgr2=0x7f267806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 --2- 172.21.15.33:0/475981336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678068140 0x7f267806c560 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2678069b60 tx=0x7f2670009840 comp rx=0 tx=0).stop 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 -- 172.21.15.33:0/475981336 shutdown_connections 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 --2- 172.21.15.33:0/475981336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678068140 0x7f267806c560 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 --2- 172.21.15.33:0/475981336 >> v2:172.21.15.167:3300/0 conn(0x7f26780b1f90 0x7f26780abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 --2- 172.21.15.33:0/475981336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2678070440 0x7f2678067c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 -- 172.21.15.33:0/475981336 >> 172.21.15.33:0/475981336 conn(0x7f267806ea70 msgr2=0x7f267806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 -- 172.21.15.33:0/475981336 shutdown_connections 2023-10-06T05:25:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.051+0000 7f267f19d700 1 -- 172.21.15.33:0/475981336 wait complete. 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.052+0000 7f267f19d700 1 Processor -- start 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.052+0000 7f267f19d700 1 -- start start 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.053+0000 7f267f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2678068140 0x7f267815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.053+0000 7f267f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 0x7f267814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.053+0000 7f267f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26780b1f90 0x7f267814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.053+0000 7f267f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267815d630 con 0x7f2678070440 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.053+0000 7f267f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267815ed70 con 0x7f2678068140 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.053+0000 7f267f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267815cc60 con 0x7f26780b1f90 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 0x7f267814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 0x7f267814dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56142/0 (socket says 172.21.15.33:56142) 2023-10-06T05:25:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 -- 172.21.15.33:0/4096767985 learned_addr learned my addr 172.21.15.33:0/4096767985 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267e19b700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2678068140 0x7f267815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267e99c700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26780b1f90 0x7f267814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 -- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26780b1f90 msgr2=0x7f267814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26780b1f90 0x7f267814a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 -- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2678068140 msgr2=0x7f267815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2678068140 0x7f267815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 -- 172.21.15.33:0/4096767985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670009ae0 con 0x7f2678070440 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267e19b700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2678068140 0x7f267815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f267d99a700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 0x7f267814dd70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f266800ecb0 tx=0x7f26680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800c920 con 0x7f2678070440 2023-10-06T05:25:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266800ca80 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.054+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800ce40 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.055+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267814e2b0 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.055+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267815c610 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.056+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267804c380 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.057+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2668013680 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.057+0000 7f266e7fc700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f265c066f30 0x7f265c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.057+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26680849a0 con 0x7f2678070440 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.058+0000 7f267e19b700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f265c066f30 0x7f265c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.058+0000 7f267e19b700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f265c066f30 0x7f265c0693e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ce0 tx=0x7f2674008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.062+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2668051a10 con 0x7f2678070440 2023-10-06T05:25:04.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.307+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f267815cda0 con 0x7f2678070440 2023-10-06T05:25:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.836+0000 7f266e7fc700 1 -- 172.21.15.33:0/4096767985 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v114) v1 ==== 122+0+0 (secure 0 0 0) 0x7f26680170c0 con 0x7f2678070440 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f265c066f30 msgr2=0x7f265c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f265c066f30 0x7f265c0693e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ce0 tx=0x7f2674008040 comp rx=0 tx=0).stop 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 msgr2=0x7f267814dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 0x7f267814dd70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f266800ecb0 tx=0x7f26680091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 shutdown_connections 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f265c066f30 0x7f265c0693e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26780b1f90 0x7f267814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2678068140 0x7f267815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 --2- 172.21.15.33:0/4096767985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2678070440 0x7f267814dd70 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 >> 172.21.15.33:0/4096767985 conn(0x7f267806ea70 msgr2=0x7f26780acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 shutdown_connections 2023-10-06T05:25:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:04.839+0000 7f267f19d700 1 -- 172.21.15.33:0/4096767985 wait complete. 2023-10-06T05:25:04.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.047+0000 7f1c7359e700 1 Processor -- start 2023-10-06T05:25:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.047+0000 7f1c7359e700 1 -- start start 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c7406e140 0x7f1c7415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7406f220 con 0x7f1c74159530 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740b1e00 con 0x7f1c7406e140 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7406f6b0 con 0x7f1c740afdd0 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c71d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c7406e140 0x7f1c7415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1c74159530 0x7f1c7406da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46678/0 (socket says 172.21.15.33:46678) 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.048+0000 7f1c7259c700 1 -- 172.21.15.33:0/1331942381 learned_addr learned my addr 172.21.15.33:0/1331942381 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c72d9d700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c71d9b700 1 -- 172.21.15.33:0/1331942381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 msgr2=0x7f1c740b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c7159a700 1 -- 172.21.15.33:0/1331942381 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118291650 0 0) 0x7f1c7406f220 con 0x7f1c74159530 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c71d9b700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c71d9b700 1 -- 172.21.15.33:0/1331942381 >> v1:172.21.15.33:6789/0 conn(0x7f1c74159530 legacy=0x7f1c7406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c71d9b700 1 -- 172.21.15.33:0/1331942381 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7415ae50 con 0x7f1c7406e140 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c72d9d700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c71d9b700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7406e140 0x7f1c7415a910 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1c680098e0 tx=0x7f1c680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c998438cb097cb63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c7159a700 1 -- 172.21.15.33:0/1331942381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68009e80 con 0x7f1c7406e140 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c7159a700 1 -- 172.21.15.33:0/1331942381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6800b860 con 0x7f1c7406e140 2023-10-06T05:25:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.049+0000 7f1c7159a700 1 -- 172.21.15.33:0/1331942381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800bbe0 con 0x7f1c7406e140 2023-10-06T05:25:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 -- 172.21.15.33:0/1331942381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7406e140 msgr2=0x7f1c7415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7406e140 0x7f1c7415a910 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1c680098e0 tx=0x7f1c680091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 -- 172.21.15.33:0/1331942381 shutdown_connections 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 --2- 172.21.15.33:0/1331942381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c7406e140 0x7f1c7415a910 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 -- 172.21.15.33:0/1331942381 >> 172.21.15.33:0/1331942381 conn(0x7f1c7406c560 msgr2=0x7f1c7406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 -- 172.21.15.33:0/1331942381 shutdown_connections 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.050+0000 7f1c7359e700 1 -- 172.21.15.33:0/1331942381 wait complete. 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.052+0000 7f1c7359e700 1 Processor -- start 2023-10-06T05:25:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.052+0000 7f1c7359e700 1 -- start start 2023-10-06T05:25:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7406e140 0x7f1c74151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 0x7f1c7415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c81c0 con 0x7f1c74159530 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c74151750 con 0x7f1c7406e140 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c75b0 con 0x7f1c740afdd0 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c72d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 0x7f1c7415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c72d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 0x7f1c7415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56164/0 (socket says 172.21.15.33:56164) 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c72d9d700 1 -- 172.21.15.33:0/777926367 learned_addr learned my addr 172.21.15.33:0/777926367 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c71d9b700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.053+0000 7f1c7259c700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7406e140 0x7f1c74151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c72d9d700 1 -- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 msgr2=0x7f1c740c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c72d9d700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c72d9d700 1 -- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7406e140 msgr2=0x7f1c74151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c72d9d700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7406e140 0x7f1c74151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c72d9d700 1 -- 172.21.15.33:0/777926367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009420 con 0x7f1c74159530 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c71d9b700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c7259c700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7406e140 0x7f1c74151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c72d9d700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 0x7f1c7415b010 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00d180 tx=0x7f1c6c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00dcc0 con 0x7f1c74159530 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00f040 con 0x7f1c74159530 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c015690 con 0x7f1c74159530 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740c7890 con 0x7f1c74159530 2023-10-06T05:25:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.054+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740c7e50 con 0x7f1c74159530 2023-10-06T05:25:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.055+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7404c380 con 0x7f1c74159530 2023-10-06T05:25:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.056+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c6c01e3b0 con 0x7f1c74159530 2023-10-06T05:25:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.057+0000 7f1c627fc700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c54066f30 0x7f1c540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.058+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1c6c084b50 con 0x7f1c74159530 2023-10-06T05:25:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.058+0000 7f1c7259c700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c54066f30 0x7f1c540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.059+0000 7f1c7259c700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c54066f30 0x7f1c540693e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1c64007fd0 tx=0x7f1c64007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.061+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c6c051c60 con 0x7f1c74159530 2023-10-06T05:25:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.311+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c740c8300 con 0x7f1c74159530 2023-10-06T05:25:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.311+0000 7f1c627fc700 1 -- 172.21.15.33:0/777926367 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f1c6c01a090 con 0x7f1c74159530 2023-10-06T05:25:05.313 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:05.313 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c54066f30 msgr2=0x7f1c540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c54066f30 0x7f1c540693e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1c64007fd0 tx=0x7f1c64007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 msgr2=0x7f1c7415b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 0x7f1c7415b010 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00d180 tx=0x7f1c6c009a50 comp rx=0 tx=0).stop 2023-10-06T05:25:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 shutdown_connections 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c54066f30 0x7f1c540693e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c74159530 0x7f1c7415b010 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c7406e140 0x7f1c74151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 --2- 172.21.15.33:0/777926367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c740afdd0 0x7f1c740c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 >> 172.21.15.33:0/777926367 conn(0x7f1c7406c560 msgr2=0x7f1c7415c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 shutdown_connections 2023-10-06T05:25:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.314+0000 7f1c7359e700 1 -- 172.21.15.33:0/777926367 wait complete. 2023-10-06T05:25:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:05.333 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.518+0000 7f8c3c9fe700 1 Processor -- start 2023-10-06T05:25:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.518+0000 7f8c3c9fe700 1 -- start start 2023-10-06T05:25:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c3c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c380cb710 0x7f8c3806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c3c9fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c380c9cb0 con 0x7f8c3813faa0 2023-10-06T05:25:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c3c9fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c380ca270 con 0x7f8c380c8050 2023-10-06T05:25:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c3c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3806b840 con 0x7f8c380cb710 2023-10-06T05:25:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c36d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8c3813faa0 0x7f8c380c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46682/0 (socket says 172.21.15.33:46682) 2023-10-06T05:25:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c36d9d700 1 -- 172.21.15.33:0/3097468742 learned_addr learned my addr 172.21.15.33:0/3097468742 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c3759e700 1 --2- 172.21.15.33:0/3097468742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c380cb710 0x7f8c3806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.520+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457875421 0 0) 0x7f8c380c9cb0 con 0x7f8c3813faa0 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c20003570 con 0x7f8c3813faa0 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91138096 0 0) 0x7f8c380ca270 con 0x7f8c380c8050 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c380c9cb0 con 0x7f8c380c8050 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3301662657 0 0) 0x7f8c20003570 con 0x7f8c3813faa0 2023-10-06T05:25:05.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c3759e700 1 -- 172.21.15.33:0/3097468742 >> v1:172.21.15.167:6789/0 conn(0x7f8c380c8050 legacy=0x7f8c3806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c3759e700 1 -- 172.21.15.33:0/3097468742 >> v1:172.21.15.33:6789/0 conn(0x7f8c3813faa0 legacy=0x7f8c380c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c3759e700 1 -- 172.21.15.33:0/3097468742 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3806bb20 con 0x7f8c380cb710 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c3759e700 1 --2- 172.21.15.33:0/3097468742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c380cb710 0x7f8c3806b1e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009e90 tx=0x7f8c30009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27bb6a1e63a7519 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.521+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000e070 con 0x7f8c380cb710 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3000b870 con 0x7f8c380cb710 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c35d9b700 1 -- 172.21.15.33:0/3097468742 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28001470 con 0x7f8c380cb710 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/3097468742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c380cb710 msgr2=0x7f8c3806b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/3097468742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c380cb710 0x7f8c3806b1e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009e90 tx=0x7f8c30009780 comp rx=0 tx=0).stop 2023-10-06T05:25:05.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/3097468742 shutdown_connections 2023-10-06T05:25:05.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/3097468742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c380cb710 0x7f8c3806b1e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/3097468742 >> 172.21.15.33:0/3097468742 conn(0x7f8c3806d030 msgr2=0x7f8c3806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/3097468742 shutdown_connections 2023-10-06T05:25:05.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.522+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/3097468742 wait complete. 2023-10-06T05:25:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.524+0000 7f8c3c9fe700 1 Processor -- start 2023-10-06T05:25:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.524+0000 7f8c3c9fe700 1 -- start start 2023-10-06T05:25:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3c9fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 0x7f8c38159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3c9fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c380cb710 0x7f8c3813c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3c9fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c3813faa0 0x7f8c3813c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3c9fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c38159200 con 0x7f8c380c8050 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3c9fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c381585f0 con 0x7f8c380cb710 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3c9fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c381588d0 con 0x7f8c3813faa0 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 0x7f8c38159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 0x7f8c38159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56174/0 (socket says 172.21.15.33:56174) 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 -- 172.21.15.33:0/4115687792 learned_addr learned my addr 172.21.15.33:0/4115687792 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3759e700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c3813faa0 0x7f8c3813c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c3659c700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c380cb710 0x7f8c3813c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 -- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c3813faa0 msgr2=0x7f8c3813c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c3813faa0 0x7f8c3813c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 -- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c380cb710 msgr2=0x7f8c3813c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c380cb710 0x7f8c3813c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.525+0000 7f8c36d9d700 1 -- 172.21.15.33:0/4115687792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009a20 con 0x7f8c380c8050 2023-10-06T05:25:05.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.526+0000 7f8c36d9d700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 0x7f8c38159690 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cb90 tx=0x7f8c28001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.526+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28007150 con 0x7f8c380c8050 2023-10-06T05:25:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.526+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c280072d0 con 0x7f8c380c8050 2023-10-06T05:25:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.526+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28007690 con 0x7f8c380c8050 2023-10-06T05:25:05.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.526+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c38158bb0 con 0x7f8c380c8050 2023-10-06T05:25:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.526+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c38159bd0 con 0x7f8c380c8050 2023-10-06T05:25:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.527+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c381594e0 con 0x7f8c380c8050 2023-10-06T05:25:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.528+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c28004a40 con 0x7f8c380c8050 2023-10-06T05:25:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.529+0000 7f8c1effd700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c24066f30 0x7f8c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.529+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8c28083a00 con 0x7f8c380c8050 2023-10-06T05:25:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.530+0000 7f8c3659c700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c24066f30 0x7f8c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.531+0000 7f8c3659c700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c24066f30 0x7f8c240693e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8c380c8b90 tx=0x7f8c2c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.533+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c28050b90 con 0x7f8c380c8050 2023-10-06T05:25:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.738+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c3804c380 con 0x7f8c380c8050 2023-10-06T05:25:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.739+0000 7f8c1effd700 1 -- 172.21.15.33:0/4115687792 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f8c28018070 con 0x7f8c380c8050 2023-10-06T05:25:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:05.741 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":156,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:25:02.799539+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:59.024220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:00.278297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c24066f30 msgr2=0x7f8c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c24066f30 0x7f8c240693e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8c380c8b90 tx=0x7f8c2c007d50 comp rx=0 tx=0).stop 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 msgr2=0x7f8c38159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 0x7f8c38159690 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cb90 tx=0x7f8c28001ab0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 shutdown_connections 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c24066f30 0x7f8c240693e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c3813faa0 0x7f8c3813c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380c8050 0x7f8c38159690 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 --2- 172.21.15.33:0/4115687792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c380cb710 0x7f8c3813c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 >> 172.21.15.33:0/4115687792 conn(0x7f8c3806d030 msgr2=0x7f8c3806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 shutdown_connections 2023-10-06T05:25:05.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.741+0000 7f8c3c9fe700 1 -- 172.21.15.33:0/4115687792 wait complete. 2023-10-06T05:25:05.760 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:25:05.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:25:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.947+0000 7ff778d9d700 1 Processor -- start 2023-10-06T05:25:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.947+0000 7ff778d9d700 1 -- start start 2023-10-06T05:25:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff778d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff774158700 0x7ff7740adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff778d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7740699c0 0x7ff774069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff778d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff774158ba0 con 0x7ff774155140 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff778d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740c8e40 con 0x7ff7740699c0 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff778d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77406a340 con 0x7ff774158700 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7740699c0 0x7ff774069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7740699c0 0x7ff774069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56176/0 (socket says 172.21.15.33:56176) 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 -- 172.21.15.33:0/286297388 learned_addr learned my addr 172.21.15.33:0/286297388 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff772ffd700 1 --2- 172.21.15.33:0/286297388 >> v2:172.21.15.167:3300/0 conn(0x7ff774158700 0x7ff7740adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 -- 172.21.15.33:0/286297388 >> v2:172.21.15.167:3300/0 conn(0x7ff774158700 msgr2=0x7ff7740adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 --2- 172.21.15.33:0/286297388 >> v2:172.21.15.167:3300/0 conn(0x7ff774158700 0x7ff7740adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 -- 172.21.15.33:0/286297388 >> v1:172.21.15.167:6789/0 conn(0x7ff774155140 legacy=0x7ff7740c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.949+0000 7ff773fff700 1 -- 172.21.15.33:0/286297388 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740ae490 con 0x7ff7740699c0 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.950+0000 7ff773fff700 1 --2- 172.21.15.33:0/286297388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7740699c0 0x7ff774069e00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009f20 tx=0x7ff76c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310ce8a7c8cd855 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.950+0000 7ff7727fc700 1 -- 172.21.15.33:0/286297388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00e070 con 0x7ff7740699c0 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.950+0000 7ff7727fc700 1 -- 172.21.15.33:0/286297388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00b990 con 0x7ff7740699c0 2023-10-06T05:25:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.950+0000 7ff7727fc700 1 -- 172.21.15.33:0/286297388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0092e0 con 0x7ff7740699c0 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.950+0000 7ff778d9d700 1 -- 172.21.15.33:0/286297388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7740699c0 msgr2=0x7ff774069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.950+0000 7ff778d9d700 1 --2- 172.21.15.33:0/286297388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7740699c0 0x7ff774069e00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009f20 tx=0x7ff76c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.951+0000 7ff778d9d700 1 -- 172.21.15.33:0/286297388 shutdown_connections 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.951+0000 7ff778d9d700 1 --2- 172.21.15.33:0/286297388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7740699c0 0x7ff774069e00 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.951+0000 7ff778d9d700 1 --2- 172.21.15.33:0/286297388 >> v2:172.21.15.167:3300/0 conn(0x7ff774158700 0x7ff7740adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.951+0000 7ff778d9d700 1 -- 172.21.15.33:0/286297388 >> 172.21.15.33:0/286297388 conn(0x7ff77406d030 msgr2=0x7ff77406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.951+0000 7ff778d9d700 1 -- 172.21.15.33:0/286297388 shutdown_connections 2023-10-06T05:25:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.951+0000 7ff778d9d700 1 -- 172.21.15.33:0/286297388 wait complete. 2023-10-06T05:25:05.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.952+0000 7ff778d9d700 1 Processor -- start 2023-10-06T05:25:05.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.952+0000 7ff778d9d700 1 -- start start 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff778d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7740699c0 0x7ff7740b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff778d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff774155140 0x7ff77414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff778d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 0x7ff774156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff778d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740af3c0 con 0x7ff774158700 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff778d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741502a0 con 0x7ff7740699c0 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff778d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b1db0 con 0x7ff774155140 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff773fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 0x7ff774156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff773fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 0x7ff774156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56188/0 (socket says 172.21.15.33:56188) 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.953+0000 7ff773fff700 1 -- 172.21.15.33:0/3044155680 learned_addr learned my addr 172.21.15.33:0/3044155680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:05.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff772ffd700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff774155140 0x7ff77414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff7737fe700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7740699c0 0x7ff7740b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff773fff700 1 -- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff774155140 msgr2=0x7ff77414f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff773fff700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff774155140 0x7ff77414f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff773fff700 1 -- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7740699c0 msgr2=0x7ff7740b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff773fff700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7740699c0 0x7ff7740b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff773fff700 1 -- 172.21.15.33:0/3044155680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009ae0 con 0x7ff774158700 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff7737fe700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7740699c0 0x7ff7740b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff773fff700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 0x7ff774156e70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009880 tx=0x7ff76c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c008c50 con 0x7ff774158700 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c008db0 con 0x7ff774158700 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c01d670 con 0x7ff774158700 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.954+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77415e700 con 0x7ff774158700 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.955+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77415ecc0 con 0x7ff774158700 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.956+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7740af570 con 0x7ff774158700 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.956+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff76c003040 con 0x7ff774158700 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.958+0000 7ff75b7fe700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.958+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff76c0862b0 con 0x7ff774158700 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.958+0000 7ff7737fe700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.959+0000 7ff7737fe700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff760066f30 0x7ff7600693e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff764007fd0 tx=0x7ff764009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:05.962+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7740af570 con 0x7ff774158700 2023-10-06T05:25:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.160+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff77404c380 con 0x7ff760066f30 2023-10-06T05:25:06.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.170+0000 7ff75b7fe700 1 -- 172.21.15.33:0/3044155680 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311704 (secure 0 0 0) 0x7ff77404c380 con 0x7ff760066f30 2023-10-06T05:25:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff760066f30 msgr2=0x7ff7600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff760066f30 0x7ff7600693e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff764007fd0 tx=0x7ff764009040 comp rx=0 tx=0).stop 2023-10-06T05:25:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 msgr2=0x7ff774156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 0x7ff774156e70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009880 tx=0x7ff76c018000 comp rx=0 tx=0).stop 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 shutdown_connections 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff760066f30 0x7ff7600693e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff774158700 0x7ff774156e70 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7740699c0 0x7ff7740b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 --2- 172.21.15.33:0/3044155680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff774155140 0x7ff77414f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 >> 172.21.15.33:0/3044155680 conn(0x7ff77406d030 msgr2=0x7ff7740c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 shutdown_connections 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.174+0000 7ff778d9d700 1 -- 172.21.15.33:0/3044155680 wait complete. 2023-10-06T05:25:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:25:06.193 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2023-10-06T05:25:05.976378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427360,"kb_used_data":16288,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547360,"statfs":{"total":773094113280,"available":770608496640,"internally_reserved":0,"allocated":16678912,"data_stored":8335280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001427"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834132+0000","last_change":"2023-10-06T05:25:01.834132+0000","last_active":"2023-10-06T05:25:01.834132+0000","last_peered":"2023-10-06T05:25:01.834132+0000","last_clean":"2023-10-06T05:25:01.834132+0000","last_became_active":"2023-10-06T05:25:01.833486+0000","last_became_peered":"2023-10-06T05:25:01.833486+0000","last_unstale":"2023-10-06T05:25:01.834132+0000","last_undegraded":"2023-10-06T05:25:01.834132+0000","last_fullsized":"2023-10-06T05:25:01.834132+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:12.366555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.820469+0000","last_change":"2023-10-06T05:25:01.820469+0000","last_active":"2023-10-06T05:25:01.820469+0000","last_peered":"2023-10-06T05:25:01.820469+0000","last_clean":"2023-10-06T05:25:01.820469+0000","last_became_active":"2023-10-06T05:25:01.820195+0000","last_became_peered":"2023-10-06T05:25:01.820195+0000","last_unstale":"2023-10-06T05:25:01.820469+0000","last_undegraded":"2023-10-06T05:25:01.820469+0000","last_fullsized":"2023-10-06T05:25:01.820469+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.763659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.832423+0000","last_change":"2023-10-06T05:25:01.832423+0000","last_active":"2023-10-06T05:25:01.832423+0000","last_peered":"2023-10-06T05:25:01.832423+0000","last_clean":"2023-10-06T05:25:01.832423+0000","last_became_active":"2023-10-06T05:25:01.832160+0000","last_became_peered":"2023-10-06T05:25:01.832160+0000","last_unstale":"2023-10-06T05:25:01.832423+0000","last_undegraded":"2023-10-06T05:25:01.832423+0000","last_fullsized":"2023-10-06T05:25:01.832423+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.592621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.837154+0000","last_change":"2023-10-06T05:25:01.837154+0000","last_active":"2023-10-06T05:25:01.837154+0000","last_peered":"2023-10-06T05:25:01.837154+0000","last_clean":"2023-10-06T05:25:01.837154+0000","last_became_active":"2023-10-06T05:25:01.836534+0000","last_became_peered":"2023-10-06T05:25:01.836534+0000","last_unstale":"2023-10-06T05:25:01.837154+0000","last_undegraded":"2023-10-06T05:25:01.837154+0000","last_fullsized":"2023-10-06T05:25:01.837154+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:20.181901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838304+0000","last_change":"2023-10-06T05:25:01.838304+0000","last_active":"2023-10-06T05:25:01.838304+0000","last_peered":"2023-10-06T05:25:01.838304+0000","last_clean":"2023-10-06T05:25:01.838304+0000","last_became_active":"2023-10-06T05:25:01.837981+0000","last_became_peered":"2023-10-06T05:25:01.837981+0000","last_unstale":"2023-10-06T05:25:01.838304+0000","last_undegraded":"2023-10-06T05:25:01.838304+0000","last_fullsized":"2023-10-06T05:25:01.838304+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:29.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.832795+0000","last_change":"2023-10-06T05:25:01.832795+0000","last_active":"2023-10-06T05:25:01.832795+0000","last_peered":"2023-10-06T05:25:01.832795+0000","last_clean":"2023-10-06T05:25:01.832795+0000","last_became_active":"2023-10-06T05:25:01.832522+0000","last_became_peered":"2023-10-06T05:25:01.832522+0000","last_unstale":"2023-10-06T05:25:01.832795+0000","last_undegraded":"2023-10-06T05:25:01.832795+0000","last_fullsized":"2023-10-06T05:25:01.832795+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:48.311249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821925+0000","last_change":"2023-10-06T05:25:01.821925+0000","last_active":"2023-10-06T05:25:01.821925+0000","last_peered":"2023-10-06T05:25:01.821925+0000","last_clean":"2023-10-06T05:25:01.821925+0000","last_became_active":"2023-10-06T05:25:01.820403+0000","last_became_peered":"2023-10-06T05:25:01.820403+0000","last_unstale":"2023-10-06T05:25:01.821925+0000","last_undegraded":"2023-10-06T05:25:01.821925+0000","last_fullsized":"2023-10-06T05:25:01.821925+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:47.211965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.839233+0000","last_change":"2023-10-06T05:25:01.839233+0000","last_active":"2023-10-06T05:25:01.839233+0000","last_peered":"2023-10-06T05:25:01.839233+0000","last_clean":"2023-10-06T05:25:01.839233+0000","last_became_active":"2023-10-06T05:25:01.838933+0000","last_became_peered":"2023-10-06T05:25:01.838933+0000","last_unstale":"2023-10-06T05:25:01.839233+0000","last_undegraded":"2023-10-06T05:25:01.839233+0000","last_fullsized":"2023-10-06T05:25:01.839233+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.763659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.819748+0000","last_change":"2023-10-06T05:25:01.819748+0000","last_active":"2023-10-06T05:25:01.819748+0000","last_peered":"2023-10-06T05:25:01.819748+0000","last_clean":"2023-10-06T05:25:01.819748+0000","last_became_active":"2023-10-06T05:25:01.819397+0000","last_became_peered":"2023-10-06T05:25:01.819397+0000","last_unstale":"2023-10-06T05:25:01.819748+0000","last_undegraded":"2023-10-06T05:25:01.819748+0000","last_fullsized":"2023-10-06T05:25:01.819748+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.397656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.837972+0000","last_change":"2023-10-06T05:25:01.837972+0000","last_active":"2023-10-06T05:25:01.837972+0000","last_peered":"2023-10-06T05:25:01.837972+0000","last_clean":"2023-10-06T05:25:01.837972+0000","last_became_active":"2023-10-06T05:25:01.837643+0000","last_became_peered":"2023-10-06T05:25:01.837643+0000","last_unstale":"2023-10-06T05:25:01.837972+0000","last_undegraded":"2023-10-06T05:25:01.837972+0000","last_fullsized":"2023-10-06T05:25:01.837972+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:00.796847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.813228+0000","last_change":"2023-10-06T05:25:01.813228+0000","last_active":"2023-10-06T05:25:01.813228+0000","last_peered":"2023-10-06T05:25:01.813228+0000","last_clean":"2023-10-06T05:25:01.813228+0000","last_became_active":"2023-10-06T05:25:01.812941+0000","last_became_peered":"2023-10-06T05:25:01.812941+0000","last_unstale":"2023-10-06T05:25:01.813228+0000","last_undegraded":"2023-10-06T05:25:01.813228+0000","last_fullsized":"2023-10-06T05:25:01.813228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:57.102129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834956+0000","last_change":"2023-10-06T05:25:01.834956+0000","last_active":"2023-10-06T05:25:01.834956+0000","last_peered":"2023-10-06T05:25:01.834956+0000","last_clean":"2023-10-06T05:25:01.834956+0000","last_became_active":"2023-10-06T05:25:01.834642+0000","last_became_peered":"2023-10-06T05:25:01.834642+0000","last_unstale":"2023-10-06T05:25:01.834956+0000","last_undegraded":"2023-10-06T05:25:01.834956+0000","last_fullsized":"2023-10-06T05:25:01.834956+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:45.989290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826122+0000","last_change":"2023-10-06T05:25:01.826122+0000","last_active":"2023-10-06T05:25:01.826122+0000","last_peered":"2023-10-06T05:25:01.826122+0000","last_clean":"2023-10-06T05:25:01.826122+0000","last_became_active":"2023-10-06T05:25:01.825869+0000","last_became_peered":"2023-10-06T05:25:01.825869+0000","last_unstale":"2023-10-06T05:25:01.826122+0000","last_undegraded":"2023-10-06T05:25:01.826122+0000","last_fullsized":"2023-10-06T05:25:01.826122+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:29.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826307+0000","last_change":"2023-10-06T05:25:01.826307+0000","last_active":"2023-10-06T05:25:01.826307+0000","last_peered":"2023-10-06T05:25:01.826307+0000","last_clean":"2023-10-06T05:25:01.826307+0000","last_became_active":"2023-10-06T05:25:01.826041+0000","last_became_peered":"2023-10-06T05:25:01.826041+0000","last_unstale":"2023-10-06T05:25:01.826307+0000","last_undegraded":"2023-10-06T05:25:01.826307+0000","last_fullsized":"2023-10-06T05:25:01.826307+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:48.637292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816093+0000","last_change":"2023-10-06T05:25:01.816093+0000","last_active":"2023-10-06T05:25:01.816093+0000","last_peered":"2023-10-06T05:25:01.816093+0000","last_clean":"2023-10-06T05:25:01.816093+0000","last_became_active":"2023-10-06T05:25:01.815829+0000","last_became_peered":"2023-10-06T05:25:01.815829+0000","last_unstale":"2023-10-06T05:25:01.816093+0000","last_undegraded":"2023-10-06T05:25:01.816093+0000","last_fullsized":"2023-10-06T05:25:01.816093+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.592621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.836955+0000","last_change":"2023-10-06T05:25:01.836955+0000","last_active":"2023-10-06T05:25:01.836955+0000","last_peered":"2023-10-06T05:25:01.836955+0000","last_clean":"2023-10-06T05:25:01.836955+0000","last_became_active":"2023-10-06T05:25:01.836533+0000","last_became_peered":"2023-10-06T05:25:01.836533+0000","last_unstale":"2023-10-06T05:25:01.836955+0000","last_undegraded":"2023-10-06T05:25:01.836955+0000","last_fullsized":"2023-10-06T05:25:01.836955+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:38.444330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816888+0000","last_change":"2023-10-06T05:25:01.816888+0000","last_active":"2023-10-06T05:25:01.816888+0000","last_peered":"2023-10-06T05:25:01.816888+0000","last_clean":"2023-10-06T05:25:01.816888+0000","last_became_active":"2023-10-06T05:25:01.816557+0000","last_became_peered":"2023-10-06T05:25:01.816557+0000","last_unstale":"2023-10-06T05:25:01.816888+0000","last_undegraded":"2023-10-06T05:25:01.816888+0000","last_fullsized":"2023-10-06T05:25:01.816888+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:04.378686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814763+0000","last_change":"2023-10-06T05:25:01.814763+0000","last_active":"2023-10-06T05:25:01.814763+0000","last_peered":"2023-10-06T05:25:01.814763+0000","last_clean":"2023-10-06T05:25:01.814763+0000","last_became_active":"2023-10-06T05:25:01.814509+0000","last_became_peered":"2023-10-06T05:25:01.814509+0000","last_unstale":"2023-10-06T05:25:01.814763+0000","last_undegraded":"2023-10-06T05:25:01.814763+0000","last_fullsized":"2023-10-06T05:25:01.814763+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:37.794373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816780+0000","last_change":"2023-10-06T05:25:01.816780+0000","last_active":"2023-10-06T05:25:01.816780+0000","last_peered":"2023-10-06T05:25:01.816780+0000","last_clean":"2023-10-06T05:25:01.816780+0000","last_became_active":"2023-10-06T05:25:01.816520+0000","last_became_peered":"2023-10-06T05:25:01.816520+0000","last_unstale":"2023-10-06T05:25:01.816780+0000","last_undegraded":"2023-10-06T05:25:01.816780+0000","last_fullsized":"2023-10-06T05:25:01.816780+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:57.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.817395+0000","last_change":"2023-10-06T05:25:01.817395+0000","last_active":"2023-10-06T05:25:01.817395+0000","last_peered":"2023-10-06T05:25:01.817395+0000","last_clean":"2023-10-06T05:25:01.817395+0000","last_became_active":"2023-10-06T05:25:01.817088+0000","last_became_peered":"2023-10-06T05:25:01.817088+0000","last_unstale":"2023-10-06T05:25:01.817395+0000","last_undegraded":"2023-10-06T05:25:01.817395+0000","last_fullsized":"2023-10-06T05:25:01.817395+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:27.049761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834960+0000","last_change":"2023-10-06T05:25:01.834960+0000","last_active":"2023-10-06T05:25:01.834960+0000","last_peered":"2023-10-06T05:25:01.834960+0000","last_clean":"2023-10-06T05:25:01.834960+0000","last_became_active":"2023-10-06T05:25:01.834678+0000","last_became_peered":"2023-10-06T05:25:01.834678+0000","last_unstale":"2023-10-06T05:25:01.834960+0000","last_undegraded":"2023-10-06T05:25:01.834960+0000","last_fullsized":"2023-10-06T05:25:01.834960+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.822032+0000","last_change":"2023-10-06T05:25:01.822032+0000","last_active":"2023-10-06T05:25:01.822032+0000","last_peered":"2023-10-06T05:25:01.822032+0000","last_clean":"2023-10-06T05:25:01.822032+0000","last_became_active":"2023-10-06T05:25:01.821293+0000","last_became_peered":"2023-10-06T05:25:01.821293+0000","last_unstale":"2023-10-06T05:25:01.822032+0000","last_undegraded":"2023-10-06T05:25:01.822032+0000","last_fullsized":"2023-10-06T05:25:01.822032+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:14.297512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814454+0000","last_change":"2023-10-06T05:25:01.814454+0000","last_active":"2023-10-06T05:25:01.814454+0000","last_peered":"2023-10-06T05:25:01.814454+0000","last_clean":"2023-10-06T05:25:01.814454+0000","last_became_active":"2023-10-06T05:25:01.814159+0000","last_became_peered":"2023-10-06T05:25:01.814159+0000","last_unstale":"2023-10-06T05:25:01.814454+0000","last_undegraded":"2023-10-06T05:25:01.814454+0000","last_fullsized":"2023-10-06T05:25:01.814454+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.832524+0000","last_change":"2023-10-06T05:25:01.832524+0000","last_active":"2023-10-06T05:25:01.832524+0000","last_peered":"2023-10-06T05:25:01.832524+0000","last_clean":"2023-10-06T05:25:01.832524+0000","last_became_active":"2023-10-06T05:25:01.832240+0000","last_became_peered":"2023-10-06T05:25:01.832240+0000","last_unstale":"2023-10-06T05:25:01.832524+0000","last_undegraded":"2023-10-06T05:25:01.832524+0000","last_fullsized":"2023-10-06T05:25:01.832524+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:32.752707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.835593+0000","last_change":"2023-10-06T05:25:01.835593+0000","last_active":"2023-10-06T05:25:01.835593+0000","last_peered":"2023-10-06T05:25:01.835593+0000","last_clean":"2023-10-06T05:25:01.835593+0000","last_became_active":"2023-10-06T05:25:01.833807+0000","last_became_peered":"2023-10-06T05:25:01.833807+0000","last_unstale":"2023-10-06T05:25:01.835593+0000","last_undegraded":"2023-10-06T05:25:01.835593+0000","last_fullsized":"2023-10-06T05:25:01.835593+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:48.553439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834793+0000","last_change":"2023-10-06T05:25:01.834793+0000","last_active":"2023-10-06T05:25:01.834793+0000","last_peered":"2023-10-06T05:25:01.834793+0000","last_clean":"2023-10-06T05:25:01.834793+0000","last_became_active":"2023-10-06T05:25:01.834511+0000","last_became_peered":"2023-10-06T05:25:01.834511+0000","last_unstale":"2023-10-06T05:25:01.834793+0000","last_undegraded":"2023-10-06T05:25:01.834793+0000","last_fullsized":"2023-10-06T05:25:01.834793+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:12.877182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.822293+0000","last_change":"2023-10-06T05:25:01.822293+0000","last_active":"2023-10-06T05:25:01.822293+0000","last_peered":"2023-10-06T05:25:01.822293+0000","last_clean":"2023-10-06T05:25:01.822293+0000","last_became_active":"2023-10-06T05:25:01.821873+0000","last_became_peered":"2023-10-06T05:25:01.821873+0000","last_unstale":"2023-10-06T05:25:01.822293+0000","last_undegraded":"2023-10-06T05:25:01.822293+0000","last_fullsized":"2023-10-06T05:25:01.822293+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:52.164405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838644+0000","last_change":"2023-10-06T05:25:01.838644+0000","last_active":"2023-10-06T05:25:01.838644+0000","last_peered":"2023-10-06T05:25:01.838644+0000","last_clean":"2023-10-06T05:25:01.838644+0000","last_became_active":"2023-10-06T05:25:01.838115+0000","last_became_peered":"2023-10-06T05:25:01.838115+0000","last_unstale":"2023-10-06T05:25:01.838644+0000","last_undegraded":"2023-10-06T05:25:01.838644+0000","last_fullsized":"2023-10-06T05:25:01.838644+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:48.637292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821843+0000","last_change":"2023-10-06T05:25:01.821843+0000","last_active":"2023-10-06T05:25:01.821843+0000","last_peered":"2023-10-06T05:25:01.821843+0000","last_clean":"2023-10-06T05:25:01.821843+0000","last_became_active":"2023-10-06T05:25:01.821574+0000","last_became_peered":"2023-10-06T05:25:01.821574+0000","last_unstale":"2023-10-06T05:25:01.821843+0000","last_undegraded":"2023-10-06T05:25:01.821843+0000","last_fullsized":"2023-10-06T05:25:01.821843+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:35.691266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814201+0000","last_change":"2023-10-06T05:25:01.814201+0000","last_active":"2023-10-06T05:25:01.814201+0000","last_peered":"2023-10-06T05:25:01.814201+0000","last_clean":"2023-10-06T05:25:01.814201+0000","last_became_active":"2023-10-06T05:25:01.813935+0000","last_became_peered":"2023-10-06T05:25:01.813935+0000","last_unstale":"2023-10-06T05:25:01.814201+0000","last_undegraded":"2023-10-06T05:25:01.814201+0000","last_fullsized":"2023-10-06T05:25:01.814201+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:22.207540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.815565+0000","last_change":"2023-10-06T05:25:01.815565+0000","last_active":"2023-10-06T05:25:01.815565+0000","last_peered":"2023-10-06T05:25:01.815565+0000","last_clean":"2023-10-06T05:25:01.815565+0000","last_became_active":"2023-10-06T05:25:01.813980+0000","last_became_peered":"2023-10-06T05:25:01.813980+0000","last_unstale":"2023-10-06T05:25:01.815565+0000","last_undegraded":"2023-10-06T05:25:01.815565+0000","last_fullsized":"2023-10-06T05:25:01.815565+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:22.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.824093+0000","last_change":"2023-10-06T05:25:01.824093+0000","last_active":"2023-10-06T05:25:01.824093+0000","last_peered":"2023-10-06T05:25:01.824093+0000","last_clean":"2023-10-06T05:25:01.824093+0000","last_became_active":"2023-10-06T05:25:01.822277+0000","last_became_peered":"2023-10-06T05:25:01.822277+0000","last_unstale":"2023-10-06T05:25:01.824093+0000","last_undegraded":"2023-10-06T05:25:01.824093+0000","last_fullsized":"2023-10-06T05:25:01.824093+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:30.652321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.829024+0000","last_change":"2023-10-06T05:25:01.829024+0000","last_active":"2023-10-06T05:25:01.829024+0000","last_peered":"2023-10-06T05:25:01.829024+0000","last_clean":"2023-10-06T05:25:01.829024+0000","last_became_active":"2023-10-06T05:25:01.826969+0000","last_became_peered":"2023-10-06T05:25:01.826969+0000","last_unstale":"2023-10-06T05:25:01.829024+0000","last_undegraded":"2023-10-06T05:25:01.829024+0000","last_fullsized":"2023-10-06T05:25:01.829024+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:22.178512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"155'4","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838299+0000","last_change":"2023-10-06T05:25:00.815910+0000","last_active":"2023-10-06T05:25:01.838299+0000","last_peered":"2023-10-06T05:25:01.838299+0000","last_clean":"2023-10-06T05:25:01.838299+0000","last_became_active":"2023-10-06T05:25:00.815626+0000","last_became_peered":"2023-10-06T05:25:00.815626+0000","last_unstale":"2023-10-06T05:25:01.838299+0000","last_undegraded":"2023-10-06T05:25:01.838299+0000","last_fullsized":"2023-10-06T05:25:01.838299+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:51.743551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.827284+0000","last_change":"2023-10-06T05:25:01.827284+0000","last_active":"2023-10-06T05:25:01.827284+0000","last_peered":"2023-10-06T05:25:01.827284+0000","last_clean":"2023-10-06T05:25:01.827284+0000","last_became_active":"2023-10-06T05:25:01.826969+0000","last_became_peered":"2023-10-06T05:25:01.826969+0000","last_unstale":"2023-10-06T05:25:01.827284+0000","last_undegraded":"2023-10-06T05:25:01.827284+0000","last_fullsized":"2023-10-06T05:25:01.827284+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:32.921424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.811503+0000","last_change":"2023-10-06T05:25:00.825691+0000","last_active":"2023-10-06T05:25:01.811503+0000","last_peered":"2023-10-06T05:25:01.811503+0000","last_clean":"2023-10-06T05:25:01.811503+0000","last_became_active":"2023-10-06T05:25:00.825390+0000","last_became_peered":"2023-10-06T05:25:00.825390+0000","last_unstale":"2023-10-06T05:25:01.811503+0000","last_undegraded":"2023-10-06T05:25:01.811503+0000","last_fullsized":"2023-10-06T05:25:01.811503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:30.943459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826808+0000","last_change":"2023-10-06T05:25:01.826808+0000","last_active":"2023-10-06T05:25:01.826808+0000","last_peered":"2023-10-06T05:25:01.826808+0000","last_clean":"2023-10-06T05:25:01.826808+0000","last_became_active":"2023-10-06T05:25:01.826470+0000","last_became_peered":"2023-10-06T05:25:01.826470+0000","last_unstale":"2023-10-06T05:25:01.826808+0000","last_undegraded":"2023-10-06T05:25:01.826808+0000","last_fullsized":"2023-10-06T05:25:01.826808+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:48.553439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.829029+0000","last_change":"2023-10-06T05:25:00.815175+0000","last_active":"2023-10-06T05:25:01.829029+0000","last_peered":"2023-10-06T05:25:01.829029+0000","last_clean":"2023-10-06T05:25:01.829029+0000","last_became_active":"2023-10-06T05:25:00.814854+0000","last_became_peered":"2023-10-06T05:25:00.814854+0000","last_unstale":"2023-10-06T05:25:01.829029+0000","last_undegraded":"2023-10-06T05:25:01.829029+0000","last_fullsized":"2023-10-06T05:25:01.829029+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:02.337673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826509+0000","last_change":"2023-10-06T05:25:01.826509+0000","last_active":"2023-10-06T05:25:01.826509+0000","last_peered":"2023-10-06T05:25:01.826509+0000","last_clean":"2023-10-06T05:25:01.826509+0000","last_became_active":"2023-10-06T05:25:01.826235+0000","last_became_peered":"2023-10-06T05:25:01.826235+0000","last_unstale":"2023-10-06T05:25:01.826509+0000","last_undegraded":"2023-10-06T05:25:01.826509+0000","last_fullsized":"2023-10-06T05:25:01.826509+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:00.796847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.830531+0000","last_change":"2023-10-06T05:25:00.817182+0000","last_active":"2023-10-06T05:25:01.830531+0000","last_peered":"2023-10-06T05:25:01.830531+0000","last_clean":"2023-10-06T05:25:01.830531+0000","last_became_active":"2023-10-06T05:25:00.816919+0000","last_became_peered":"2023-10-06T05:25:00.816919+0000","last_unstale":"2023-10-06T05:25:01.830531+0000","last_undegraded":"2023-10-06T05:25:01.830531+0000","last_fullsized":"2023-10-06T05:25:01.830531+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:34.281255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838572+0000","last_change":"2023-10-06T05:25:01.838572+0000","last_active":"2023-10-06T05:25:01.838572+0000","last_peered":"2023-10-06T05:25:01.838572+0000","last_clean":"2023-10-06T05:25:01.838572+0000","last_became_active":"2023-10-06T05:25:01.838307+0000","last_became_peered":"2023-10-06T05:25:01.838307+0000","last_unstale":"2023-10-06T05:25:01.838572+0000","last_undegraded":"2023-10-06T05:25:01.838572+0000","last_fullsized":"2023-10-06T05:25:01.838572+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:05.773514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.806899+0000","last_change":"2023-10-06T05:25:00.828191+0000","last_active":"2023-10-06T05:25:01.806899+0000","last_peered":"2023-10-06T05:25:01.806899+0000","last_clean":"2023-10-06T05:25:01.806899+0000","last_became_active":"2023-10-06T05:25:00.827814+0000","last_became_peered":"2023-10-06T05:25:00.827814+0000","last_unstale":"2023-10-06T05:25:01.806899+0000","last_undegraded":"2023-10-06T05:25:01.806899+0000","last_fullsized":"2023-10-06T05:25:01.806899+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:20.003764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.823667+0000","last_change":"2023-10-06T05:25:01.823667+0000","last_active":"2023-10-06T05:25:01.823667+0000","last_peered":"2023-10-06T05:25:01.823667+0000","last_clean":"2023-10-06T05:25:01.823667+0000","last_became_active":"2023-10-06T05:25:01.823288+0000","last_became_peered":"2023-10-06T05:25:01.823288+0000","last_unstale":"2023-10-06T05:25:01.823667+0000","last_undegraded":"2023-10-06T05:25:01.823667+0000","last_fullsized":"2023-10-06T05:25:01.823667+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:41.286775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.808647+0000","last_change":"2023-10-06T05:25:00.818938+0000","last_active":"2023-10-06T05:25:01.808647+0000","last_peered":"2023-10-06T05:25:01.808647+0000","last_clean":"2023-10-06T05:25:01.808647+0000","last_became_active":"2023-10-06T05:25:00.818656+0000","last_became_peered":"2023-10-06T05:25:00.818656+0000","last_unstale":"2023-10-06T05:25:01.808647+0000","last_undegraded":"2023-10-06T05:25:01.808647+0000","last_fullsized":"2023-10-06T05:25:01.808647+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:38.985887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.815814+0000","last_change":"2023-10-06T05:25:01.815814+0000","last_active":"2023-10-06T05:25:01.815814+0000","last_peered":"2023-10-06T05:25:01.815814+0000","last_clean":"2023-10-06T05:25:01.815814+0000","last_became_active":"2023-10-06T05:25:01.815545+0000","last_became_peered":"2023-10-06T05:25:01.815545+0000","last_unstale":"2023-10-06T05:25:01.815814+0000","last_undegraded":"2023-10-06T05:25:01.815814+0000","last_fullsized":"2023-10-06T05:25:01.815814+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:20.984857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.817671+0000","last_change":"2023-10-06T05:25:00.815937+0000","last_active":"2023-10-06T05:25:00.817671+0000","last_peered":"2023-10-06T05:25:00.817671+0000","last_clean":"2023-10-06T05:25:00.817671+0000","last_became_active":"2023-10-06T05:25:00.815495+0000","last_became_peered":"2023-10-06T05:25:00.815495+0000","last_unstale":"2023-10-06T05:25:00.817671+0000","last_undegraded":"2023-10-06T05:25:00.817671+0000","last_fullsized":"2023-10-06T05:25:00.817671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:11.699003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834362+0000","last_change":"2023-10-06T05:25:01.834362+0000","last_active":"2023-10-06T05:25:01.834362+0000","last_peered":"2023-10-06T05:25:01.834362+0000","last_clean":"2023-10-06T05:25:01.834362+0000","last_became_active":"2023-10-06T05:25:01.834099+0000","last_became_peered":"2023-10-06T05:25:01.834099+0000","last_unstale":"2023-10-06T05:25:01.834362+0000","last_undegraded":"2023-10-06T05:25:01.834362+0000","last_fullsized":"2023-10-06T05:25:01.834362+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:57.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.824555+0000","last_change":"2023-10-06T05:25:00.822837+0000","last_active":"2023-10-06T05:25:00.824555+0000","last_peered":"2023-10-06T05:25:00.824555+0000","last_clean":"2023-10-06T05:25:00.824555+0000","last_became_active":"2023-10-06T05:25:00.822499+0000","last_became_peered":"2023-10-06T05:25:00.822499+0000","last_unstale":"2023-10-06T05:25:00.824555+0000","last_undegraded":"2023-10-06T05:25:00.824555+0000","last_fullsized":"2023-10-06T05:25:00.824555+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:25.772262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.811643+0000","last_change":"2023-10-06T05:25:01.811643+0000","last_active":"2023-10-06T05:25:01.811643+0000","last_peered":"2023-10-06T05:25:01.811643+0000","last_clean":"2023-10-06T05:25:01.811643+0000","last_became_active":"2023-10-06T05:25:01.811346+0000","last_became_peered":"2023-10-06T05:25:01.811346+0000","last_unstale":"2023-10-06T05:25:01.811643+0000","last_undegraded":"2023-10-06T05:25:01.811643+0000","last_fullsized":"2023-10-06T05:25:01.811643+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.170230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.818940+0000","last_change":"2023-10-06T05:25:00.815505+0000","last_active":"2023-10-06T05:25:01.818940+0000","last_peered":"2023-10-06T05:25:01.818940+0000","last_clean":"2023-10-06T05:25:01.818940+0000","last_became_active":"2023-10-06T05:25:00.815222+0000","last_became_peered":"2023-10-06T05:25:00.815222+0000","last_unstale":"2023-10-06T05:25:01.818940+0000","last_undegraded":"2023-10-06T05:25:01.818940+0000","last_fullsized":"2023-10-06T05:25:01.818940+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:22.167048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.813428+0000","last_change":"2023-10-06T05:25:01.813428+0000","last_active":"2023-10-06T05:25:01.813428+0000","last_peered":"2023-10-06T05:25:01.813428+0000","last_clean":"2023-10-06T05:25:01.813428+0000","last_became_active":"2023-10-06T05:25:01.812521+0000","last_became_peered":"2023-10-06T05:25:01.812521+0000","last_unstale":"2023-10-06T05:25:01.813428+0000","last_undegraded":"2023-10-06T05:25:01.813428+0000","last_fullsized":"2023-10-06T05:25:01.813428+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:10.956943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.835570+0000","last_change":"2023-10-06T05:25:00.823266+0000","last_active":"2023-10-06T05:25:01.835570+0000","last_peered":"2023-10-06T05:25:01.835570+0000","last_clean":"2023-10-06T05:25:01.835570+0000","last_became_active":"2023-10-06T05:25:00.822966+0000","last_became_peered":"2023-10-06T05:25:00.822966+0000","last_unstale":"2023-10-06T05:25:01.835570+0000","last_undegraded":"2023-10-06T05:25:01.835570+0000","last_fullsized":"2023-10-06T05:25:01.835570+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:57.016157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834154+0000","last_change":"2023-10-06T05:25:01.834154+0000","last_active":"2023-10-06T05:25:01.834154+0000","last_peered":"2023-10-06T05:25:01.834154+0000","last_clean":"2023-10-06T05:25:01.834154+0000","last_became_active":"2023-10-06T05:25:01.833889+0000","last_became_peered":"2023-10-06T05:25:01.833889+0000","last_unstale":"2023-10-06T05:25:01.834154+0000","last_undegraded":"2023-10-06T05:25:01.834154+0000","last_fullsized":"2023-10-06T05:25:01.834154+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:35.045088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.807757+0000","last_change":"2023-10-06T05:25:00.817258+0000","last_active":"2023-10-06T05:25:01.807757+0000","last_peered":"2023-10-06T05:25:01.807757+0000","last_clean":"2023-10-06T05:25:01.807757+0000","last_became_active":"2023-10-06T05:25:00.816968+0000","last_became_peered":"2023-10-06T05:25:00.816968+0000","last_unstale":"2023-10-06T05:25:01.807757+0000","last_undegraded":"2023-10-06T05:25:01.807757+0000","last_fullsized":"2023-10-06T05:25:01.807757+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:47.308067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821158+0000","last_change":"2023-10-06T05:25:01.821158+0000","last_active":"2023-10-06T05:25:01.821158+0000","last_peered":"2023-10-06T05:25:01.821158+0000","last_clean":"2023-10-06T05:25:01.821158+0000","last_became_active":"2023-10-06T05:25:01.820877+0000","last_became_peered":"2023-10-06T05:25:01.820877+0000","last_unstale":"2023-10-06T05:25:01.821158+0000","last_undegraded":"2023-10-06T05:25:01.821158+0000","last_fullsized":"2023-10-06T05:25:01.821158+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:19.350475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.833036+0000","last_change":"2023-10-06T05:25:00.828055+0000","last_active":"2023-10-06T05:25:00.833036+0000","last_peered":"2023-10-06T05:25:00.833036+0000","last_clean":"2023-10-06T05:25:00.833036+0000","last_became_active":"2023-10-06T05:25:00.827813+0000","last_became_peered":"2023-10-06T05:25:00.827813+0000","last_unstale":"2023-10-06T05:25:00.833036+0000","last_undegraded":"2023-10-06T05:25:00.833036+0000","last_fullsized":"2023-10-06T05:25:00.833036+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:34.041906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834596+0000","last_change":"2023-10-06T05:25:01.834596+0000","last_active":"2023-10-06T05:25:01.834596+0000","last_peered":"2023-10-06T05:25:01.834596+0000","last_clean":"2023-10-06T05:25:01.834596+0000","last_became_active":"2023-10-06T05:25:01.834341+0000","last_became_peered":"2023-10-06T05:25:01.834341+0000","last_unstale":"2023-10-06T05:25:01.834596+0000","last_undegraded":"2023-10-06T05:25:01.834596+0000","last_fullsized":"2023-10-06T05:25:01.834596+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:09.879405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.841160+0000","last_change":"2023-10-06T05:25:00.818787+0000","last_active":"2023-10-06T05:25:01.841160+0000","last_peered":"2023-10-06T05:25:01.841160+0000","last_clean":"2023-10-06T05:25:01.841160+0000","last_became_active":"2023-10-06T05:25:00.818471+0000","last_became_peered":"2023-10-06T05:25:00.818471+0000","last_unstale":"2023-10-06T05:25:01.841160+0000","last_undegraded":"2023-10-06T05:25:01.841160+0000","last_fullsized":"2023-10-06T05:25:01.841160+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:21.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.812488+0000","last_change":"2023-10-06T05:25:01.812488+0000","last_active":"2023-10-06T05:25:01.812488+0000","last_peered":"2023-10-06T05:25:01.812488+0000","last_clean":"2023-10-06T05:25:01.812488+0000","last_became_active":"2023-10-06T05:25:01.812203+0000","last_became_peered":"2023-10-06T05:25:01.812203+0000","last_unstale":"2023-10-06T05:25:01.812488+0000","last_undegraded":"2023-10-06T05:25:01.812488+0000","last_fullsized":"2023-10-06T05:25:01.812488+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:59.858227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.816083+0000","last_change":"2023-10-06T05:25:00.814570+0000","last_active":"2023-10-06T05:25:00.816083+0000","last_peered":"2023-10-06T05:25:00.816083+0000","last_clean":"2023-10-06T05:25:00.816083+0000","last_became_active":"2023-10-06T05:25:00.814313+0000","last_became_peered":"2023-10-06T05:25:00.814313+0000","last_unstale":"2023-10-06T05:25:00.816083+0000","last_undegraded":"2023-10-06T05:25:00.816083+0000","last_fullsized":"2023-10-06T05:25:00.816083+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:36.972771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.817919+0000","last_change":"2023-10-06T05:25:01.817919+0000","last_active":"2023-10-06T05:25:01.817919+0000","last_peered":"2023-10-06T05:25:01.817919+0000","last_clean":"2023-10-06T05:25:01.817919+0000","last_became_active":"2023-10-06T05:25:01.817542+0000","last_became_peered":"2023-10-06T05:25:01.817542+0000","last_unstale":"2023-10-06T05:25:01.817919+0000","last_undegraded":"2023-10-06T05:25:01.817919+0000","last_fullsized":"2023-10-06T05:25:01.817919+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:20.600837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.812614+0000","last_change":"2023-10-06T05:25:00.808172+0000","last_active":"2023-10-06T05:25:00.812614+0000","last_peered":"2023-10-06T05:25:00.812614+0000","last_clean":"2023-10-06T05:25:00.812614+0000","last_became_active":"2023-10-06T05:25:00.807870+0000","last_became_peered":"2023-10-06T05:25:00.807870+0000","last_unstale":"2023-10-06T05:25:00.812614+0000","last_undegraded":"2023-10-06T05:25:00.812614+0000","last_fullsized":"2023-10-06T05:25:00.812614+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:39.127957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826245+0000","last_change":"2023-10-06T05:25:01.826245+0000","last_active":"2023-10-06T05:25:01.826245+0000","last_peered":"2023-10-06T05:25:01.826245+0000","last_clean":"2023-10-06T05:25:01.826245+0000","last_became_active":"2023-10-06T05:25:01.825943+0000","last_became_peered":"2023-10-06T05:25:01.825943+0000","last_unstale":"2023-10-06T05:25:01.826245+0000","last_undegraded":"2023-10-06T05:25:01.826245+0000","last_fullsized":"2023-10-06T05:25:01.826245+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:31.946641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.823434+0000","last_change":"2023-10-06T05:25:00.819383+0000","last_active":"2023-10-06T05:25:00.823434+0000","last_peered":"2023-10-06T05:25:00.823434+0000","last_clean":"2023-10-06T05:25:00.823434+0000","last_became_active":"2023-10-06T05:25:00.818908+0000","last_became_peered":"2023-10-06T05:25:00.818908+0000","last_unstale":"2023-10-06T05:25:00.823434+0000","last_undegraded":"2023-10-06T05:25:00.823434+0000","last_fullsized":"2023-10-06T05:25:00.823434+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814185+0000","last_change":"2023-10-06T05:25:01.814185+0000","last_active":"2023-10-06T05:25:01.814185+0000","last_peered":"2023-10-06T05:25:01.814185+0000","last_clean":"2023-10-06T05:25:01.814185+0000","last_became_active":"2023-10-06T05:25:01.813906+0000","last_became_peered":"2023-10-06T05:25:01.813906+0000","last_unstale":"2023-10-06T05:25:01.814185+0000","last_undegraded":"2023-10-06T05:25:01.814185+0000","last_fullsized":"2023-10-06T05:25:01.814185+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:05.736590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.840787+0000","last_change":"2023-10-06T05:25:00.816147+0000","last_active":"2023-10-06T05:25:01.840787+0000","last_peered":"2023-10-06T05:25:01.840787+0000","last_clean":"2023-10-06T05:25:01.840787+0000","last_became_active":"2023-10-06T05:25:00.815880+0000","last_became_peered":"2023-10-06T05:25:00.815880+0000","last_unstale":"2023-10-06T05:25:01.840787+0000","last_undegraded":"2023-10-06T05:25:01.840787+0000","last_fullsized":"2023-10-06T05:25:01.840787+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:42.360638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.820375+0000","last_change":"2023-10-06T05:25:01.820375+0000","last_active":"2023-10-06T05:25:01.820375+0000","last_peered":"2023-10-06T05:25:01.820375+0000","last_clean":"2023-10-06T05:25:01.820375+0000","last_became_active":"2023-10-06T05:25:01.820064+0000","last_became_peered":"2023-10-06T05:25:01.820064+0000","last_unstale":"2023-10-06T05:25:01.820375+0000","last_undegraded":"2023-10-06T05:25:01.820375+0000","last_fullsized":"2023-10-06T05:25:01.820375+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:28.586702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.826706+0000","last_change":"2023-10-06T05:25:00.824176+0000","last_active":"2023-10-06T05:25:00.826706+0000","last_peered":"2023-10-06T05:25:00.826706+0000","last_clean":"2023-10-06T05:25:00.826706+0000","last_became_active":"2023-10-06T05:25:00.823933+0000","last_became_peered":"2023-10-06T05:25:00.823933+0000","last_unstale":"2023-10-06T05:25:00.826706+0000","last_undegraded":"2023-10-06T05:25:00.826706+0000","last_fullsized":"2023-10-06T05:25:00.826706+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:04.770332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816436+0000","last_change":"2023-10-06T05:25:01.816436+0000","last_active":"2023-10-06T05:25:01.816436+0000","last_peered":"2023-10-06T05:25:01.816436+0000","last_clean":"2023-10-06T05:25:01.816436+0000","last_became_active":"2023-10-06T05:25:01.816165+0000","last_became_peered":"2023-10-06T05:25:01.816165+0000","last_unstale":"2023-10-06T05:25:01.816436+0000","last_undegraded":"2023-10-06T05:25:01.816436+0000","last_fullsized":"2023-10-06T05:25:01.816436+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:11.785855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.824308+0000","last_change":"2023-10-06T05:25:00.820365+0000","last_active":"2023-10-06T05:25:00.824308+0000","last_peered":"2023-10-06T05:25:00.824308+0000","last_clean":"2023-10-06T05:25:00.824308+0000","last_became_active":"2023-10-06T05:25:00.820096+0000","last_became_peered":"2023-10-06T05:25:00.820096+0000","last_unstale":"2023-10-06T05:25:00.824308+0000","last_undegraded":"2023-10-06T05:25:00.824308+0000","last_fullsized":"2023-10-06T05:25:00.824308+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:27.583520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821126+0000","last_change":"2023-10-06T05:25:01.821126+0000","last_active":"2023-10-06T05:25:01.821126+0000","last_peered":"2023-10-06T05:25:01.821126+0000","last_clean":"2023-10-06T05:25:01.821126+0000","last_became_active":"2023-10-06T05:25:01.820778+0000","last_became_peered":"2023-10-06T05:25:01.820778+0000","last_unstale":"2023-10-06T05:25:01.821126+0000","last_undegraded":"2023-10-06T05:25:01.821126+0000","last_fullsized":"2023-10-06T05:25:01.821126+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:19.649700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.821539+0000","last_change":"2023-10-06T05:25:00.820643+0000","last_active":"2023-10-06T05:25:00.821539+0000","last_peered":"2023-10-06T05:25:00.821539+0000","last_clean":"2023-10-06T05:25:00.821539+0000","last_became_active":"2023-10-06T05:25:00.820368+0000","last_became_peered":"2023-10-06T05:25:00.820368+0000","last_unstale":"2023-10-06T05:25:00.821539+0000","last_undegraded":"2023-10-06T05:25:00.821539+0000","last_fullsized":"2023-10-06T05:25:00.821539+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:58.444398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838250+0000","last_change":"2023-10-06T05:25:01.838250+0000","last_active":"2023-10-06T05:25:01.838250+0000","last_peered":"2023-10-06T05:25:01.838250+0000","last_clean":"2023-10-06T05:25:01.838250+0000","last_became_active":"2023-10-06T05:25:01.837885+0000","last_became_peered":"2023-10-06T05:25:01.837885+0000","last_unstale":"2023-10-06T05:25:01.838250+0000","last_undegraded":"2023-10-06T05:25:01.838250+0000","last_fullsized":"2023-10-06T05:25:01.838250+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:04.638103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.823697+0000","last_change":"2023-10-06T05:25:00.820603+0000","last_active":"2023-10-06T05:25:00.823697+0000","last_peered":"2023-10-06T05:25:00.823697+0000","last_clean":"2023-10-06T05:25:00.823697+0000","last_became_active":"2023-10-06T05:25:00.820279+0000","last_became_peered":"2023-10-06T05:25:00.820279+0000","last_unstale":"2023-10-06T05:25:00.823697+0000","last_undegraded":"2023-10-06T05:25:00.823697+0000","last_fullsized":"2023-10-06T05:25:00.823697+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:23.596756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.827266+0000","last_change":"2023-10-06T05:25:01.827266+0000","last_active":"2023-10-06T05:25:01.827266+0000","last_peered":"2023-10-06T05:25:01.827266+0000","last_clean":"2023-10-06T05:25:01.827266+0000","last_became_active":"2023-10-06T05:25:01.826904+0000","last_became_peered":"2023-10-06T05:25:01.826904+0000","last_unstale":"2023-10-06T05:25:01.827266+0000","last_undegraded":"2023-10-06T05:25:01.827266+0000","last_fullsized":"2023-10-06T05:25:01.827266+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:39.989069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.819610+0000","last_change":"2023-10-06T05:25:00.812525+0000","last_active":"2023-10-06T05:25:00.819610+0000","last_peered":"2023-10-06T05:25:00.819610+0000","last_clean":"2023-10-06T05:25:00.819610+0000","last_became_active":"2023-10-06T05:25:00.812231+0000","last_became_peered":"2023-10-06T05:25:00.812231+0000","last_unstale":"2023-10-06T05:25:00.819610+0000","last_undegraded":"2023-10-06T05:25:00.819610+0000","last_fullsized":"2023-10-06T05:25:00.819610+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:59.793665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.839929+0000","last_change":"2023-10-06T05:25:01.839929+0000","last_active":"2023-10-06T05:25:01.839929+0000","last_peered":"2023-10-06T05:25:01.839929+0000","last_clean":"2023-10-06T05:25:01.839929+0000","last_became_active":"2023-10-06T05:25:01.839624+0000","last_became_peered":"2023-10-06T05:25:01.839624+0000","last_unstale":"2023-10-06T05:25:01.839929+0000","last_undegraded":"2023-10-06T05:25:01.839929+0000","last_fullsized":"2023-10-06T05:25:01.839929+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:48.477374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.820819+0000","last_change":"2023-10-06T05:25:00.819347+0000","last_active":"2023-10-06T05:25:00.820819+0000","last_peered":"2023-10-06T05:25:00.820819+0000","last_clean":"2023-10-06T05:25:00.820819+0000","last_became_active":"2023-10-06T05:25:00.819011+0000","last_became_peered":"2023-10-06T05:25:00.819011+0000","last_unstale":"2023-10-06T05:25:00.820819+0000","last_undegraded":"2023-10-06T05:25:00.820819+0000","last_fullsized":"2023-10-06T05:25:00.820819+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:32.185150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.818528+0000","last_change":"2023-10-06T05:25:01.818528+0000","last_active":"2023-10-06T05:25:01.818528+0000","last_peered":"2023-10-06T05:25:01.818528+0000","last_clean":"2023-10-06T05:25:01.818528+0000","last_became_active":"2023-10-06T05:25:01.818253+0000","last_became_peered":"2023-10-06T05:25:01.818253+0000","last_unstale":"2023-10-06T05:25:01.818528+0000","last_undegraded":"2023-10-06T05:25:01.818528+0000","last_fullsized":"2023-10-06T05:25:01.818528+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.625028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.821671+0000","last_change":"2023-10-06T05:25:00.818496+0000","last_active":"2023-10-06T05:25:00.821671+0000","last_peered":"2023-10-06T05:25:00.821671+0000","last_clean":"2023-10-06T05:25:00.821671+0000","last_became_active":"2023-10-06T05:25:00.817743+0000","last_became_peered":"2023-10-06T05:25:00.817743+0000","last_unstale":"2023-10-06T05:25:00.821671+0000","last_undegraded":"2023-10-06T05:25:00.821671+0000","last_fullsized":"2023-10-06T05:25:00.821671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:59.793665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.820459+0000","last_change":"2023-10-06T05:25:01.820459+0000","last_active":"2023-10-06T05:25:01.820459+0000","last_peered":"2023-10-06T05:25:01.820459+0000","last_clean":"2023-10-06T05:25:01.820459+0000","last_became_active":"2023-10-06T05:25:01.819428+0000","last_became_peered":"2023-10-06T05:25:01.819428+0000","last_unstale":"2023-10-06T05:25:01.820459+0000","last_undegraded":"2023-10-06T05:25:01.820459+0000","last_fullsized":"2023-10-06T05:25:01.820459+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:58.019339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.819195+0000","last_change":"2023-10-06T05:25:00.817255+0000","last_active":"2023-10-06T05:25:00.819195+0000","last_peered":"2023-10-06T05:25:00.819195+0000","last_clean":"2023-10-06T05:25:00.819195+0000","last_became_active":"2023-10-06T05:25:00.815987+0000","last_became_peered":"2023-10-06T05:25:00.815987+0000","last_unstale":"2023-10-06T05:25:00.819195+0000","last_undegraded":"2023-10-06T05:25:00.819195+0000","last_fullsized":"2023-10-06T05:25:00.819195+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:05.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.814847+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:25:00.814847+0000","last_peered":"2023-10-06T05:25:00.814847+0000","last_clean":"2023-10-06T05:25:00.814847+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:25:00.814847+0000","last_undegraded":"2023-10-06T05:25:00.814847+0000","last_fullsized":"2023-10-06T05:25:00.814847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826847+0000","last_change":"2023-10-06T05:25:01.826847+0000","last_active":"2023-10-06T05:25:01.826847+0000","last_peered":"2023-10-06T05:25:01.826847+0000","last_clean":"2023-10-06T05:25:01.826847+0000","last_became_active":"2023-10-06T05:25:01.825519+0000","last_became_peered":"2023-10-06T05:25:01.825519+0000","last_unstale":"2023-10-06T05:25:01.826847+0000","last_undegraded":"2023-10-06T05:25:01.826847+0000","last_fullsized":"2023-10-06T05:25:01.826847+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:43.363820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.823052+0000","last_change":"2023-10-06T05:25:00.818514+0000","last_active":"2023-10-06T05:25:00.823052+0000","last_peered":"2023-10-06T05:25:00.823052+0000","last_clean":"2023-10-06T05:25:00.823052+0000","last_became_active":"2023-10-06T05:25:00.818262+0000","last_became_peered":"2023-10-06T05:25:00.818262+0000","last_unstale":"2023-10-06T05:25:00.823052+0000","last_undegraded":"2023-10-06T05:25:00.823052+0000","last_fullsized":"2023-10-06T05:25:00.823052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:46.208783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":122,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.811591+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:25:00.811591+0000","last_peered":"2023-10-06T05:25:00.811591+0000","last_clean":"2023-10-06T05:25:00.811591+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:25:00.811591+0000","last_undegraded":"2023-10-06T05:25:00.811591+0000","last_fullsized":"2023-10-06T05:25:00.811591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.818502+0000","last_change":"2023-10-06T05:25:01.818502+0000","last_active":"2023-10-06T05:25:01.818502+0000","last_peered":"2023-10-06T05:25:01.818502+0000","last_clean":"2023-10-06T05:25:01.818502+0000","last_became_active":"2023-10-06T05:25:01.818227+0000","last_became_peered":"2023-10-06T05:25:01.818227+0000","last_unstale":"2023-10-06T05:25:01.818502+0000","last_undegraded":"2023-10-06T05:25:01.818502+0000","last_fullsized":"2023-10-06T05:25:01.818502+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:36.492596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"155'6","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.831540+0000","last_change":"2023-10-06T05:25:00.819108+0000","last_active":"2023-10-06T05:25:01.831540+0000","last_peered":"2023-10-06T05:25:01.831540+0000","last_clean":"2023-10-06T05:25:01.831540+0000","last_became_active":"2023-10-06T05:25:00.818855+0000","last_became_peered":"2023-10-06T05:25:00.818855+0000","last_unstale":"2023-10-06T05:25:01.831540+0000","last_undegraded":"2023-10-06T05:25:01.831540+0000","last_fullsized":"2023-10-06T05:25:01.831540+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:27.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.800628+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:25:00.800628+0000","last_peered":"2023-10-06T05:25:00.800628+0000","last_clean":"2023-10-06T05:25:00.800628+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:25:00.800628+0000","last_undegraded":"2023-10-06T05:25:00.800628+0000","last_fullsized":"2023-10-06T05:25:00.800628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.825798+0000","last_change":"2023-10-06T05:25:01.825798+0000","last_active":"2023-10-06T05:25:01.825798+0000","last_peered":"2023-10-06T05:25:01.825798+0000","last_clean":"2023-10-06T05:25:01.825798+0000","last_became_active":"2023-10-06T05:25:01.825409+0000","last_became_peered":"2023-10-06T05:25:01.825409+0000","last_unstale":"2023-10-06T05:25:01.825798+0000","last_undegraded":"2023-10-06T05:25:01.825798+0000","last_fullsized":"2023-10-06T05:25:01.825798+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:04.378686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.826739+0000","last_change":"2023-10-06T05:25:00.824174+0000","last_active":"2023-10-06T05:25:00.826739+0000","last_peered":"2023-10-06T05:25:00.826739+0000","last_clean":"2023-10-06T05:25:00.826739+0000","last_became_active":"2023-10-06T05:25:00.823902+0000","last_became_peered":"2023-10-06T05:25:00.823902+0000","last_unstale":"2023-10-06T05:25:00.826739+0000","last_undegraded":"2023-10-06T05:25:00.826739+0000","last_fullsized":"2023-10-06T05:25:00.826739+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:48.760477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.811025+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:25:00.811025+0000","last_peered":"2023-10-06T05:25:00.811025+0000","last_clean":"2023-10-06T05:25:00.811025+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:25:00.811025+0000","last_undegraded":"2023-10-06T05:25:00.811025+0000","last_fullsized":"2023-10-06T05:25:00.811025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":273,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.810655+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:25:00.810655+0000","last_peered":"2023-10-06T05:25:00.810655+0000","last_clean":"2023-10-06T05:25:00.810655+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:25:00.810655+0000","last_undegraded":"2023-10-06T05:25:00.810655+0000","last_fullsized":"2023-10-06T05:25:00.810655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.831413+0000","last_change":"2023-10-06T05:25:01.831413+0000","last_active":"2023-10-06T05:25:01.831413+0000","last_peered":"2023-10-06T05:25:01.831413+0000","last_clean":"2023-10-06T05:25:01.831413+0000","last_became_active":"2023-10-06T05:25:01.831131+0000","last_became_peered":"2023-10-06T05:25:01.831131+0000","last_unstale":"2023-10-06T05:25:01.831413+0000","last_undegraded":"2023-10-06T05:25:01.831413+0000","last_fullsized":"2023-10-06T05:25:01.831413+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:51.482718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"155'3","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.833578+0000","last_change":"2023-10-06T05:25:00.816811+0000","last_active":"2023-10-06T05:25:01.833578+0000","last_peered":"2023-10-06T05:25:01.833578+0000","last_clean":"2023-10-06T05:25:01.833578+0000","last_became_active":"2023-10-06T05:25:00.816551+0000","last_became_peered":"2023-10-06T05:25:00.816551+0000","last_unstale":"2023-10-06T05:25:01.833578+0000","last_undegraded":"2023-10-06T05:25:01.833578+0000","last_fullsized":"2023-10-06T05:25:01.833578+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:04.733408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.810566+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:25:00.810566+0000","last_peered":"2023-10-06T05:25:00.810566+0000","last_clean":"2023-10-06T05:25:00.810566+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:25:00.810566+0000","last_undegraded":"2023-10-06T05:25:00.810566+0000","last_fullsized":"2023-10-06T05:25:00.810566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834886+0000","last_change":"2023-10-06T05:25:01.834886+0000","last_active":"2023-10-06T05:25:01.834886+0000","last_peered":"2023-10-06T05:25:01.834886+0000","last_clean":"2023-10-06T05:25:01.834886+0000","last_became_active":"2023-10-06T05:25:01.834610+0000","last_became_peered":"2023-10-06T05:25:01.834610+0000","last_unstale":"2023-10-06T05:25:01.834886+0000","last_undegraded":"2023-10-06T05:25:01.834886+0000","last_fullsized":"2023-10-06T05:25:01.834886+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:22.829190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"155'8","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.840382+0000","last_change":"2023-10-06T05:25:00.826903+0000","last_active":"2023-10-06T05:25:01.840382+0000","last_peered":"2023-10-06T05:25:01.840382+0000","last_clean":"2023-10-06T05:25:01.840382+0000","last_became_active":"2023-10-06T05:25:00.826602+0000","last_became_peered":"2023-10-06T05:25:00.826602+0000","last_unstale":"2023-10-06T05:25:01.840382+0000","last_undegraded":"2023-10-06T05:25:01.840382+0000","last_fullsized":"2023-10-06T05:25:01.840382+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:49.487935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":120,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.813085+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:25:00.813085+0000","last_peered":"2023-10-06T05:25:00.813085+0000","last_clean":"2023-10-06T05:25:00.813085+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:25:00.813085+0000","last_undegraded":"2023-10-06T05:25:00.813085+0000","last_fullsized":"2023-10-06T05:25:00.813085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821952+0000","last_change":"2023-10-06T05:25:01.821952+0000","last_active":"2023-10-06T05:25:01.821952+0000","last_peered":"2023-10-06T05:25:01.821952+0000","last_clean":"2023-10-06T05:25:01.821952+0000","last_became_active":"2023-10-06T05:25:01.821672+0000","last_became_peered":"2023-10-06T05:25:01.821672+0000","last_unstale":"2023-10-06T05:25:01.821952+0000","last_undegraded":"2023-10-06T05:25:01.821952+0000","last_fullsized":"2023-10-06T05:25:01.821952+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:27.960907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838582+0000","last_change":"2023-10-06T05:25:00.816635+0000","last_active":"2023-10-06T05:25:01.838582+0000","last_peered":"2023-10-06T05:25:01.838582+0000","last_clean":"2023-10-06T05:25:01.838582+0000","last_became_active":"2023-10-06T05:25:00.816385+0000","last_became_peered":"2023-10-06T05:25:00.816385+0000","last_unstale":"2023-10-06T05:25:01.838582+0000","last_undegraded":"2023-10-06T05:25:01.838582+0000","last_fullsized":"2023-10-06T05:25:01.838582+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:44.986107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.820280+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:25:00.820280+0000","last_peered":"2023-10-06T05:25:00.820280+0000","last_clean":"2023-10-06T05:25:00.820280+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:25:00.820280+0000","last_undegraded":"2023-10-06T05:25:00.820280+0000","last_fullsized":"2023-10-06T05:25:00.820280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.813218+0000","last_change":"2023-10-06T05:25:01.813218+0000","last_active":"2023-10-06T05:25:01.813218+0000","last_peered":"2023-10-06T05:25:01.813218+0000","last_clean":"2023-10-06T05:25:01.813218+0000","last_became_active":"2023-10-06T05:25:01.812899+0000","last_became_peered":"2023-10-06T05:25:01.812899+0000","last_unstale":"2023-10-06T05:25:01.813218+0000","last_undegraded":"2023-10-06T05:25:01.813218+0000","last_fullsized":"2023-10-06T05:25:01.813218+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:22.178512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.811557+0000","last_change":"2023-10-06T05:25:00.809201+0000","last_active":"2023-10-06T05:25:00.811557+0000","last_peered":"2023-10-06T05:25:00.811557+0000","last_clean":"2023-10-06T05:25:00.811557+0000","last_became_active":"2023-10-06T05:25:00.808935+0000","last_became_peered":"2023-10-06T05:25:00.808935+0000","last_unstale":"2023-10-06T05:25:00.811557+0000","last_undegraded":"2023-10-06T05:25:00.811557+0000","last_fullsized":"2023-10-06T05:25:00.811557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:31.407052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.804722+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:25:00.804722+0000","last_peered":"2023-10-06T05:25:00.804722+0000","last_clean":"2023-10-06T05:25:00.804722+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:25:00.804722+0000","last_undegraded":"2023-10-06T05:25:00.804722+0000","last_fullsized":"2023-10-06T05:25:00.804722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303864,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067976,"statfs":{"total":96636764160,"available":96325607424,"internally_reserved":0,"allocated":2539520,"data_stored":1490023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95999999999999996,"15min":1.1850000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.169},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":1.268},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.96099999999999997,"15min":1.075},"last":0.496}]},{"osd":2,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.0780000000000001,"15min":1.093},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.495,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.22},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.079},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.51900000000000002,"15min":0.46600000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.202},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.258},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.39400000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.755,"15min":0.748},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.441},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.407},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.47799999999999998,"5min":0.45100000000000001,"15min":0.38600000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3300000000000001},"last":0.60099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":899655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.68999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.84799999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65500000000000003,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.995,"5min":0.995,"15min":1.2869999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.83199999999999996,"5min":1,"15min":1},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69299999999999995,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.876,"5min":1.05,"15min":1.1599999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68899999999999995,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.745}]},{"osd":3,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92700000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.82499999999999996,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.65200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.95699999999999996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64400000000000002,"15min":0.68000000000000005},"min":{"1min":0.441,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.873,"5min":0.89100000000000001,"15min":0.998},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66200000000000003,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93000000000000005,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.747},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.96499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96199999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73499999999999999,"15min":0.80000000000000004},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.96899999999999997,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.84799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.497},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47899999999999998,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.67100000000000004,"5min":0.77000000000000002,"15min":2.8050000000000002},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.45800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.78800000000000003,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52500000000000002,"15min":0.58099999999999996},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86299999999999999,"5min":0.88200000000000001,"15min":2.7010000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.87,"5min":0.88,"15min":2.7639999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.57799999999999996},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":2.8370000000000002},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.59299999999999997,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.79900000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.95099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77400000000000002,"15min":0.80300000000000005},"min":{"1min":0.58899999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.145,"5min":1.25,"15min":1.349},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.669},"last":0.97199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81100000000000005,"15min":0.84399999999999997},"min":{"1min":0.57399999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.169,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.81000000000000005,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.503},"max":{"1min":1.204,"5min":1.3280000000000001,"15min":1.573},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.79700000000000004,"15min":0.83999999999999997},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.528},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.84499999999999997},"min":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":1.3049999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.73299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2527232,"data_stored":1482907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.63600000000000001,"5min":0.68100000000000005,"15min":0.79400000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.497},"min":{"1min":0.34599999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51100000000000001,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.65500000000000003,"5min":0.78700000000000003,"15min":0.84199999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":0.70999999999999996,"15min":0.84999999999999998},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72299999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.70799999999999996,"5min":0.84499999999999997,"15min":0.879},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.98699999999999999,"5min":1.1120000000000001,"15min":1.1160000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.125,"15min":1.2310000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":0.997,"5min":1.159,"15min":1.218},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92800000000000005,"5min":1.089,"15min":1.1970000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":0.97499999999999998,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.59199999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99399999999999999,"5min":1.111,"15min":1.2629999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80500000000000005},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.25},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.04,"5min":1.256,"15min":1.2789999999999999},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":1.03,"5min":1.056,"15min":1.056},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.30599999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.94399999999999995,"5min":1.073,"15min":1.073},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.109,"5min":1.109,"15min":1.1379999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.59799999999999998,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55800000000000005,"15min":0.53000000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.319},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77800000000000002,"15min":0.81200000000000006},"min":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.175,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.141},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.3520000000000001},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45600000000000002,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.254},"max":{"1min":0.70199999999999996,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.47999999999999998,"15min":0.49399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.72699999999999998,"5min":0.77100000000000002,"15min":1.772},"last":0.439}]},{"osd":1,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.58399999999999996,"5min":0.79400000000000004,"15min":1.913},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.66400000000000003,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.63,"5min":0.79700000000000004,"15min":1.954},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.77100000000000002},"min":{"1min":0.496,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.282},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":1.1839999999999999,"5min":1.216,"15min":1.2609999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.51100000000000001},"max":{"1min":1.0840000000000001,"5min":1.145,"15min":1.1930000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.82999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.3009999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.53900000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.343},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78500000000000003,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":1.1779999999999999,"15min":1.3380000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.83599999999999997},"min":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.3640000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.83499999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.214},"last":0.63300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.073},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.46100000000000002},"max":{"1min":0.88900000000000001,"5min":1.0089999999999999,"15min":1.169},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.41199999999999998},"max":{"1min":0.83299999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.68200000000000005,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42699999999999999},"max":{"1min":0.81399999999999995,"5min":1.016,"15min":1.117},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.002},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47099999999999997},"max":{"1min":0.94999999999999996,"5min":0.97599999999999998,"15min":1.0780000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":1.018,"15min":1.196},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66400000000000003,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70299999999999996,"15min":0.73299999999999998},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.95999999999999996,"5min":0.999,"15min":1.375},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70199999999999996,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.74099999999999999,"15min":0.77600000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.9390000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.752,"15min":0.78800000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.466},"last":0.872},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.81899999999999995},"min":{"1min":0.501,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":0.98699999999999999,"15min":1.218},"last":0.89300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.93899999999999995,"15min":1.165},"last":0.624},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":0.98999999999999999,"15min":1.1890000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.92800000000000005,"15min":0.98299999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.877,"5min":1.048,"15min":1.048},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79700000000000004,"5min":0.93600000000000005,"15min":1.204},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90300000000000002,"5min":0.95199999999999996,"15min":1.087},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72199999999999998,"15min":0.72699999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":0.85399999999999998,"5min":0.99299999999999999,"15min":1.278},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.372},"max":{"1min":0.93600000000000005,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.1539999999999999,"15min":1.294},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":0.96899999999999997,"15min":1.141},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73699999999999999,"15min":0.74199999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":1.0429999999999999,"15min":1.254},"last":0.874}]},{"osd":7,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.337},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.76500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.0609999999999999,"15min":1.248},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:25:06.202 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:25:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.417+0000 7f9eccf9d700 1 Processor -- start 2023-10-06T05:25:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.417+0000 7f9eccf9d700 1 -- start start 2023-10-06T05:25:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9eccf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ec80c7040 0x7f9ec80cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9eccf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9eccf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec80cb5f0 con 0x7f9ec8147390 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9eccf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec80ca9e0 con 0x7f9ec80c7040 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9eccf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cacc0 con 0x7f9ec8147760 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9ec77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ec8147390 0x7f9ec80c6930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46694/0 (socket says 172.21.15.33:46694) 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9ec77fe700 1 -- 172.21.15.33:0/3589715511 learned_addr learned my addr 172.21.15.33:0/3589715511 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9ec6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ec80c7040 0x7f9ec80cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9ec7fff700 1 --2- 172.21.15.33:0/3589715511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.419+0000 7f9ec67fc700 1 -- 172.21.15.33:0/3589715511 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797874265 0 0) 0x7f9ec80cb5f0 con 0x7f9ec8147390 2023-10-06T05:25:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec67fc700 1 -- 172.21.15.33:0/3589715511 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb0003570 con 0x7f9ec8147390 2023-10-06T05:25:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec6ffd700 1 -- 172.21.15.33:0/3589715511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 msgr2=0x7f9ec80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec6ffd700 1 --2- 172.21.15.33:0/3589715511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec80c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec6ffd700 1 -- 172.21.15.33:0/3589715511 >> v1:172.21.15.33:6789/0 conn(0x7f9ec8147390 legacy=0x7f9ec80c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec6ffd700 1 -- 172.21.15.33:0/3589715511 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80cafa0 con 0x7f9ec80c7040 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec6ffd700 1 --2- 172.21.15.33:0/3589715511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec80cba80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0098e0 tx=0x7f9ebc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a2afd9170639ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec67fc700 1 -- 172.21.15.33:0/3589715511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011630 con 0x7f9ec80c7040 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec67fc700 1 -- 172.21.15.33:0/3589715511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc011790 con 0x7f9ec80c7040 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.420+0000 7f9ec67fc700 1 -- 172.21.15.33:0/3589715511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011b30 con 0x7f9ec80c7040 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 -- 172.21.15.33:0/3589715511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 msgr2=0x7f9ec80cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/3589715511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec80cba80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0098e0 tx=0x7f9ebc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 -- 172.21.15.33:0/3589715511 shutdown_connections 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/3589715511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/3589715511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec80cba80 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 -- 172.21.15.33:0/3589715511 >> 172.21.15.33:0/3589715511 conn(0x7f9ec806d030 msgr2=0x7f9ec806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 -- 172.21.15.33:0/3589715511 shutdown_connections 2023-10-06T05:25:06.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.421+0000 7f9eccf9d700 1 -- 172.21.15.33:0/3589715511 wait complete. 2023-10-06T05:25:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.422+0000 7f9eccf9d700 1 Processor -- start 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.422+0000 7f9eccf9d700 1 -- start start 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9eccf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec816d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9eccf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec8147390 0x7f9ec8160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9eccf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec815e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9eccf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80c5950 con 0x7f9ec80c7040 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9eccf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec81618d0 con 0x7f9ec8147390 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9eccf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8161bb0 con 0x7f9ec8147760 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.423+0000 7f9ec77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec816d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec816d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56208/0 (socket says 172.21.15.33:56208) 2023-10-06T05:25:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 -- 172.21.15.33:0/1209915869 learned_addr learned my addr 172.21.15.33:0/1209915869 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec7fff700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec6ffd700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec8147390 0x7f9ec8160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 -- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 msgr2=0x7f9ec815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 -- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec8147390 msgr2=0x7f9ec8160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec8147390 0x7f9ec8160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 -- 172.21.15.33:0/1209915869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc009420 con 0x7f9ec80c7040 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9ec77fe700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec816d1d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c7c0 tx=0x7f9eb8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8005b30 con 0x7f9ec80c7040 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8005c90 con 0x7f9ec80c7040 2023-10-06T05:25:06.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8007340 con 0x7f9ec80c7040 2023-10-06T05:25:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.424+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec8161e90 con 0x7f9ec80c7040 2023-10-06T05:25:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.425+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec816d710 con 0x7f9ec80c7040 2023-10-06T05:25:06.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.426+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec80c5da0 con 0x7f9ec80c7040 2023-10-06T05:25:06.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.427+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9eb801b400 con 0x7f9ec80c7040 2023-10-06T05:25:06.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.427+0000 7f9eaf7fe700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9eb4066f30 0x7f9eb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.427+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9eb80841d0 con 0x7f9ec80c7040 2023-10-06T05:25:06.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.428+0000 7f9ec6ffd700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9eb4066f30 0x7f9eb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.429+0000 7f9ec6ffd700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9eb4066f30 0x7f9eb40693e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0091e0 tx=0x7f9ebc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.431+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9eb80512c0 con 0x7f9ec80c7040 2023-10-06T05:25:06.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.629+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ec815fb90 con 0x7f9eb4066f30 2023-10-06T05:25:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.638+0000 7f9eaf7fe700 1 -- 172.21.15.33:0/1209915869 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311704 (secure 0 0 0) 0x7f9ec815fb90 con 0x7f9eb4066f30 2023-10-06T05:25:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9eb4066f30 msgr2=0x7f9eb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9eb4066f30 0x7f9eb40693e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0091e0 tx=0x7f9ebc017000 comp rx=0 tx=0).stop 2023-10-06T05:25:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 msgr2=0x7f9ec816d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec816d1d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c7c0 tx=0x7f9eb8007e00 comp rx=0 tx=0).stop 2023-10-06T05:25:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 shutdown_connections 2023-10-06T05:25:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9eb4066f30 0x7f9eb40693e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ec8147760 0x7f9ec815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec80c7040 0x7f9ec816d1d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 --2- 172.21.15.33:0/1209915869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec8147390 0x7f9ec8160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.642+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 >> 172.21.15.33:0/1209915869 conn(0x7f9ec806d030 msgr2=0x7f9ec806aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.643+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 shutdown_connections 2023-10-06T05:25:06.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.643+0000 7f9eccf9d700 1 -- 172.21.15.33:0/1209915869 wait complete. 2023-10-06T05:25:06.649 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:25:06.663 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2023-10-06T05:25:05.976378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427360,"kb_used_data":16288,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547360,"statfs":{"total":773094113280,"available":770608496640,"internally_reserved":0,"allocated":16678912,"data_stored":8335280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001427"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834132+0000","last_change":"2023-10-06T05:25:01.834132+0000","last_active":"2023-10-06T05:25:01.834132+0000","last_peered":"2023-10-06T05:25:01.834132+0000","last_clean":"2023-10-06T05:25:01.834132+0000","last_became_active":"2023-10-06T05:25:01.833486+0000","last_became_peered":"2023-10-06T05:25:01.833486+0000","last_unstale":"2023-10-06T05:25:01.834132+0000","last_undegraded":"2023-10-06T05:25:01.834132+0000","last_fullsized":"2023-10-06T05:25:01.834132+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:12.366555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.820469+0000","last_change":"2023-10-06T05:25:01.820469+0000","last_active":"2023-10-06T05:25:01.820469+0000","last_peered":"2023-10-06T05:25:01.820469+0000","last_clean":"2023-10-06T05:25:01.820469+0000","last_became_active":"2023-10-06T05:25:01.820195+0000","last_became_peered":"2023-10-06T05:25:01.820195+0000","last_unstale":"2023-10-06T05:25:01.820469+0000","last_undegraded":"2023-10-06T05:25:01.820469+0000","last_fullsized":"2023-10-06T05:25:01.820469+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.763659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.832423+0000","last_change":"2023-10-06T05:25:01.832423+0000","last_active":"2023-10-06T05:25:01.832423+0000","last_peered":"2023-10-06T05:25:01.832423+0000","last_clean":"2023-10-06T05:25:01.832423+0000","last_became_active":"2023-10-06T05:25:01.832160+0000","last_became_peered":"2023-10-06T05:25:01.832160+0000","last_unstale":"2023-10-06T05:25:01.832423+0000","last_undegraded":"2023-10-06T05:25:01.832423+0000","last_fullsized":"2023-10-06T05:25:01.832423+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.592621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.837154+0000","last_change":"2023-10-06T05:25:01.837154+0000","last_active":"2023-10-06T05:25:01.837154+0000","last_peered":"2023-10-06T05:25:01.837154+0000","last_clean":"2023-10-06T05:25:01.837154+0000","last_became_active":"2023-10-06T05:25:01.836534+0000","last_became_peered":"2023-10-06T05:25:01.836534+0000","last_unstale":"2023-10-06T05:25:01.837154+0000","last_undegraded":"2023-10-06T05:25:01.837154+0000","last_fullsized":"2023-10-06T05:25:01.837154+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:20.181901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838304+0000","last_change":"2023-10-06T05:25:01.838304+0000","last_active":"2023-10-06T05:25:01.838304+0000","last_peered":"2023-10-06T05:25:01.838304+0000","last_clean":"2023-10-06T05:25:01.838304+0000","last_became_active":"2023-10-06T05:25:01.837981+0000","last_became_peered":"2023-10-06T05:25:01.837981+0000","last_unstale":"2023-10-06T05:25:01.838304+0000","last_undegraded":"2023-10-06T05:25:01.838304+0000","last_fullsized":"2023-10-06T05:25:01.838304+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:29.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.832795+0000","last_change":"2023-10-06T05:25:01.832795+0000","last_active":"2023-10-06T05:25:01.832795+0000","last_peered":"2023-10-06T05:25:01.832795+0000","last_clean":"2023-10-06T05:25:01.832795+0000","last_became_active":"2023-10-06T05:25:01.832522+0000","last_became_peered":"2023-10-06T05:25:01.832522+0000","last_unstale":"2023-10-06T05:25:01.832795+0000","last_undegraded":"2023-10-06T05:25:01.832795+0000","last_fullsized":"2023-10-06T05:25:01.832795+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:48.311249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821925+0000","last_change":"2023-10-06T05:25:01.821925+0000","last_active":"2023-10-06T05:25:01.821925+0000","last_peered":"2023-10-06T05:25:01.821925+0000","last_clean":"2023-10-06T05:25:01.821925+0000","last_became_active":"2023-10-06T05:25:01.820403+0000","last_became_peered":"2023-10-06T05:25:01.820403+0000","last_unstale":"2023-10-06T05:25:01.821925+0000","last_undegraded":"2023-10-06T05:25:01.821925+0000","last_fullsized":"2023-10-06T05:25:01.821925+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:47.211965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.839233+0000","last_change":"2023-10-06T05:25:01.839233+0000","last_active":"2023-10-06T05:25:01.839233+0000","last_peered":"2023-10-06T05:25:01.839233+0000","last_clean":"2023-10-06T05:25:01.839233+0000","last_became_active":"2023-10-06T05:25:01.838933+0000","last_became_peered":"2023-10-06T05:25:01.838933+0000","last_unstale":"2023-10-06T05:25:01.839233+0000","last_undegraded":"2023-10-06T05:25:01.839233+0000","last_fullsized":"2023-10-06T05:25:01.839233+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.763659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.819748+0000","last_change":"2023-10-06T05:25:01.819748+0000","last_active":"2023-10-06T05:25:01.819748+0000","last_peered":"2023-10-06T05:25:01.819748+0000","last_clean":"2023-10-06T05:25:01.819748+0000","last_became_active":"2023-10-06T05:25:01.819397+0000","last_became_peered":"2023-10-06T05:25:01.819397+0000","last_unstale":"2023-10-06T05:25:01.819748+0000","last_undegraded":"2023-10-06T05:25:01.819748+0000","last_fullsized":"2023-10-06T05:25:01.819748+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.397656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.837972+0000","last_change":"2023-10-06T05:25:01.837972+0000","last_active":"2023-10-06T05:25:01.837972+0000","last_peered":"2023-10-06T05:25:01.837972+0000","last_clean":"2023-10-06T05:25:01.837972+0000","last_became_active":"2023-10-06T05:25:01.837643+0000","last_became_peered":"2023-10-06T05:25:01.837643+0000","last_unstale":"2023-10-06T05:25:01.837972+0000","last_undegraded":"2023-10-06T05:25:01.837972+0000","last_fullsized":"2023-10-06T05:25:01.837972+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:00.796847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.813228+0000","last_change":"2023-10-06T05:25:01.813228+0000","last_active":"2023-10-06T05:25:01.813228+0000","last_peered":"2023-10-06T05:25:01.813228+0000","last_clean":"2023-10-06T05:25:01.813228+0000","last_became_active":"2023-10-06T05:25:01.812941+0000","last_became_peered":"2023-10-06T05:25:01.812941+0000","last_unstale":"2023-10-06T05:25:01.813228+0000","last_undegraded":"2023-10-06T05:25:01.813228+0000","last_fullsized":"2023-10-06T05:25:01.813228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:57.102129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834956+0000","last_change":"2023-10-06T05:25:01.834956+0000","last_active":"2023-10-06T05:25:01.834956+0000","last_peered":"2023-10-06T05:25:01.834956+0000","last_clean":"2023-10-06T05:25:01.834956+0000","last_became_active":"2023-10-06T05:25:01.834642+0000","last_became_peered":"2023-10-06T05:25:01.834642+0000","last_unstale":"2023-10-06T05:25:01.834956+0000","last_undegraded":"2023-10-06T05:25:01.834956+0000","last_fullsized":"2023-10-06T05:25:01.834956+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:45.989290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826122+0000","last_change":"2023-10-06T05:25:01.826122+0000","last_active":"2023-10-06T05:25:01.826122+0000","last_peered":"2023-10-06T05:25:01.826122+0000","last_clean":"2023-10-06T05:25:01.826122+0000","last_became_active":"2023-10-06T05:25:01.825869+0000","last_became_peered":"2023-10-06T05:25:01.825869+0000","last_unstale":"2023-10-06T05:25:01.826122+0000","last_undegraded":"2023-10-06T05:25:01.826122+0000","last_fullsized":"2023-10-06T05:25:01.826122+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:29.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826307+0000","last_change":"2023-10-06T05:25:01.826307+0000","last_active":"2023-10-06T05:25:01.826307+0000","last_peered":"2023-10-06T05:25:01.826307+0000","last_clean":"2023-10-06T05:25:01.826307+0000","last_became_active":"2023-10-06T05:25:01.826041+0000","last_became_peered":"2023-10-06T05:25:01.826041+0000","last_unstale":"2023-10-06T05:25:01.826307+0000","last_undegraded":"2023-10-06T05:25:01.826307+0000","last_fullsized":"2023-10-06T05:25:01.826307+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:48.637292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816093+0000","last_change":"2023-10-06T05:25:01.816093+0000","last_active":"2023-10-06T05:25:01.816093+0000","last_peered":"2023-10-06T05:25:01.816093+0000","last_clean":"2023-10-06T05:25:01.816093+0000","last_became_active":"2023-10-06T05:25:01.815829+0000","last_became_peered":"2023-10-06T05:25:01.815829+0000","last_unstale":"2023-10-06T05:25:01.816093+0000","last_undegraded":"2023-10-06T05:25:01.816093+0000","last_fullsized":"2023-10-06T05:25:01.816093+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:00.592621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.836955+0000","last_change":"2023-10-06T05:25:01.836955+0000","last_active":"2023-10-06T05:25:01.836955+0000","last_peered":"2023-10-06T05:25:01.836955+0000","last_clean":"2023-10-06T05:25:01.836955+0000","last_became_active":"2023-10-06T05:25:01.836533+0000","last_became_peered":"2023-10-06T05:25:01.836533+0000","last_unstale":"2023-10-06T05:25:01.836955+0000","last_undegraded":"2023-10-06T05:25:01.836955+0000","last_fullsized":"2023-10-06T05:25:01.836955+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:38.444330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816888+0000","last_change":"2023-10-06T05:25:01.816888+0000","last_active":"2023-10-06T05:25:01.816888+0000","last_peered":"2023-10-06T05:25:01.816888+0000","last_clean":"2023-10-06T05:25:01.816888+0000","last_became_active":"2023-10-06T05:25:01.816557+0000","last_became_peered":"2023-10-06T05:25:01.816557+0000","last_unstale":"2023-10-06T05:25:01.816888+0000","last_undegraded":"2023-10-06T05:25:01.816888+0000","last_fullsized":"2023-10-06T05:25:01.816888+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:04.378686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814763+0000","last_change":"2023-10-06T05:25:01.814763+0000","last_active":"2023-10-06T05:25:01.814763+0000","last_peered":"2023-10-06T05:25:01.814763+0000","last_clean":"2023-10-06T05:25:01.814763+0000","last_became_active":"2023-10-06T05:25:01.814509+0000","last_became_peered":"2023-10-06T05:25:01.814509+0000","last_unstale":"2023-10-06T05:25:01.814763+0000","last_undegraded":"2023-10-06T05:25:01.814763+0000","last_fullsized":"2023-10-06T05:25:01.814763+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:37.794373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816780+0000","last_change":"2023-10-06T05:25:01.816780+0000","last_active":"2023-10-06T05:25:01.816780+0000","last_peered":"2023-10-06T05:25:01.816780+0000","last_clean":"2023-10-06T05:25:01.816780+0000","last_became_active":"2023-10-06T05:25:01.816520+0000","last_became_peered":"2023-10-06T05:25:01.816520+0000","last_unstale":"2023-10-06T05:25:01.816780+0000","last_undegraded":"2023-10-06T05:25:01.816780+0000","last_fullsized":"2023-10-06T05:25:01.816780+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:57.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.817395+0000","last_change":"2023-10-06T05:25:01.817395+0000","last_active":"2023-10-06T05:25:01.817395+0000","last_peered":"2023-10-06T05:25:01.817395+0000","last_clean":"2023-10-06T05:25:01.817395+0000","last_became_active":"2023-10-06T05:25:01.817088+0000","last_became_peered":"2023-10-06T05:25:01.817088+0000","last_unstale":"2023-10-06T05:25:01.817395+0000","last_undegraded":"2023-10-06T05:25:01.817395+0000","last_fullsized":"2023-10-06T05:25:01.817395+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:27.049761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834960+0000","last_change":"2023-10-06T05:25:01.834960+0000","last_active":"2023-10-06T05:25:01.834960+0000","last_peered":"2023-10-06T05:25:01.834960+0000","last_clean":"2023-10-06T05:25:01.834960+0000","last_became_active":"2023-10-06T05:25:01.834678+0000","last_became_peered":"2023-10-06T05:25:01.834678+0000","last_unstale":"2023-10-06T05:25:01.834960+0000","last_undegraded":"2023-10-06T05:25:01.834960+0000","last_fullsized":"2023-10-06T05:25:01.834960+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.822032+0000","last_change":"2023-10-06T05:25:01.822032+0000","last_active":"2023-10-06T05:25:01.822032+0000","last_peered":"2023-10-06T05:25:01.822032+0000","last_clean":"2023-10-06T05:25:01.822032+0000","last_became_active":"2023-10-06T05:25:01.821293+0000","last_became_peered":"2023-10-06T05:25:01.821293+0000","last_unstale":"2023-10-06T05:25:01.822032+0000","last_undegraded":"2023-10-06T05:25:01.822032+0000","last_fullsized":"2023-10-06T05:25:01.822032+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:14.297512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814454+0000","last_change":"2023-10-06T05:25:01.814454+0000","last_active":"2023-10-06T05:25:01.814454+0000","last_peered":"2023-10-06T05:25:01.814454+0000","last_clean":"2023-10-06T05:25:01.814454+0000","last_became_active":"2023-10-06T05:25:01.814159+0000","last_became_peered":"2023-10-06T05:25:01.814159+0000","last_unstale":"2023-10-06T05:25:01.814454+0000","last_undegraded":"2023-10-06T05:25:01.814454+0000","last_fullsized":"2023-10-06T05:25:01.814454+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:48.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.832524+0000","last_change":"2023-10-06T05:25:01.832524+0000","last_active":"2023-10-06T05:25:01.832524+0000","last_peered":"2023-10-06T05:25:01.832524+0000","last_clean":"2023-10-06T05:25:01.832524+0000","last_became_active":"2023-10-06T05:25:01.832240+0000","last_became_peered":"2023-10-06T05:25:01.832240+0000","last_unstale":"2023-10-06T05:25:01.832524+0000","last_undegraded":"2023-10-06T05:25:01.832524+0000","last_fullsized":"2023-10-06T05:25:01.832524+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:32.752707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.835593+0000","last_change":"2023-10-06T05:25:01.835593+0000","last_active":"2023-10-06T05:25:01.835593+0000","last_peered":"2023-10-06T05:25:01.835593+0000","last_clean":"2023-10-06T05:25:01.835593+0000","last_became_active":"2023-10-06T05:25:01.833807+0000","last_became_peered":"2023-10-06T05:25:01.833807+0000","last_unstale":"2023-10-06T05:25:01.835593+0000","last_undegraded":"2023-10-06T05:25:01.835593+0000","last_fullsized":"2023-10-06T05:25:01.835593+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:48.553439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834793+0000","last_change":"2023-10-06T05:25:01.834793+0000","last_active":"2023-10-06T05:25:01.834793+0000","last_peered":"2023-10-06T05:25:01.834793+0000","last_clean":"2023-10-06T05:25:01.834793+0000","last_became_active":"2023-10-06T05:25:01.834511+0000","last_became_peered":"2023-10-06T05:25:01.834511+0000","last_unstale":"2023-10-06T05:25:01.834793+0000","last_undegraded":"2023-10-06T05:25:01.834793+0000","last_fullsized":"2023-10-06T05:25:01.834793+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:12.877182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.822293+0000","last_change":"2023-10-06T05:25:01.822293+0000","last_active":"2023-10-06T05:25:01.822293+0000","last_peered":"2023-10-06T05:25:01.822293+0000","last_clean":"2023-10-06T05:25:01.822293+0000","last_became_active":"2023-10-06T05:25:01.821873+0000","last_became_peered":"2023-10-06T05:25:01.821873+0000","last_unstale":"2023-10-06T05:25:01.822293+0000","last_undegraded":"2023-10-06T05:25:01.822293+0000","last_fullsized":"2023-10-06T05:25:01.822293+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:52.164405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838644+0000","last_change":"2023-10-06T05:25:01.838644+0000","last_active":"2023-10-06T05:25:01.838644+0000","last_peered":"2023-10-06T05:25:01.838644+0000","last_clean":"2023-10-06T05:25:01.838644+0000","last_became_active":"2023-10-06T05:25:01.838115+0000","last_became_peered":"2023-10-06T05:25:01.838115+0000","last_unstale":"2023-10-06T05:25:01.838644+0000","last_undegraded":"2023-10-06T05:25:01.838644+0000","last_fullsized":"2023-10-06T05:25:01.838644+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:48.637292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821843+0000","last_change":"2023-10-06T05:25:01.821843+0000","last_active":"2023-10-06T05:25:01.821843+0000","last_peered":"2023-10-06T05:25:01.821843+0000","last_clean":"2023-10-06T05:25:01.821843+0000","last_became_active":"2023-10-06T05:25:01.821574+0000","last_became_peered":"2023-10-06T05:25:01.821574+0000","last_unstale":"2023-10-06T05:25:01.821843+0000","last_undegraded":"2023-10-06T05:25:01.821843+0000","last_fullsized":"2023-10-06T05:25:01.821843+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:35.691266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814201+0000","last_change":"2023-10-06T05:25:01.814201+0000","last_active":"2023-10-06T05:25:01.814201+0000","last_peered":"2023-10-06T05:25:01.814201+0000","last_clean":"2023-10-06T05:25:01.814201+0000","last_became_active":"2023-10-06T05:25:01.813935+0000","last_became_peered":"2023-10-06T05:25:01.813935+0000","last_unstale":"2023-10-06T05:25:01.814201+0000","last_undegraded":"2023-10-06T05:25:01.814201+0000","last_fullsized":"2023-10-06T05:25:01.814201+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:22.207540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.815565+0000","last_change":"2023-10-06T05:25:01.815565+0000","last_active":"2023-10-06T05:25:01.815565+0000","last_peered":"2023-10-06T05:25:01.815565+0000","last_clean":"2023-10-06T05:25:01.815565+0000","last_became_active":"2023-10-06T05:25:01.813980+0000","last_became_peered":"2023-10-06T05:25:01.813980+0000","last_unstale":"2023-10-06T05:25:01.815565+0000","last_undegraded":"2023-10-06T05:25:01.815565+0000","last_fullsized":"2023-10-06T05:25:01.815565+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:22.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.824093+0000","last_change":"2023-10-06T05:25:01.824093+0000","last_active":"2023-10-06T05:25:01.824093+0000","last_peered":"2023-10-06T05:25:01.824093+0000","last_clean":"2023-10-06T05:25:01.824093+0000","last_became_active":"2023-10-06T05:25:01.822277+0000","last_became_peered":"2023-10-06T05:25:01.822277+0000","last_unstale":"2023-10-06T05:25:01.824093+0000","last_undegraded":"2023-10-06T05:25:01.824093+0000","last_fullsized":"2023-10-06T05:25:01.824093+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:30.652321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.829024+0000","last_change":"2023-10-06T05:25:01.829024+0000","last_active":"2023-10-06T05:25:01.829024+0000","last_peered":"2023-10-06T05:25:01.829024+0000","last_clean":"2023-10-06T05:25:01.829024+0000","last_became_active":"2023-10-06T05:25:01.826969+0000","last_became_peered":"2023-10-06T05:25:01.826969+0000","last_unstale":"2023-10-06T05:25:01.829024+0000","last_undegraded":"2023-10-06T05:25:01.829024+0000","last_fullsized":"2023-10-06T05:25:01.829024+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:22.178512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"155'4","reported_seq":13,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838299+0000","last_change":"2023-10-06T05:25:00.815910+0000","last_active":"2023-10-06T05:25:01.838299+0000","last_peered":"2023-10-06T05:25:01.838299+0000","last_clean":"2023-10-06T05:25:01.838299+0000","last_became_active":"2023-10-06T05:25:00.815626+0000","last_became_peered":"2023-10-06T05:25:00.815626+0000","last_unstale":"2023-10-06T05:25:01.838299+0000","last_undegraded":"2023-10-06T05:25:01.838299+0000","last_fullsized":"2023-10-06T05:25:01.838299+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:51.743551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.827284+0000","last_change":"2023-10-06T05:25:01.827284+0000","last_active":"2023-10-06T05:25:01.827284+0000","last_peered":"2023-10-06T05:25:01.827284+0000","last_clean":"2023-10-06T05:25:01.827284+0000","last_became_active":"2023-10-06T05:25:01.826969+0000","last_became_peered":"2023-10-06T05:25:01.826969+0000","last_unstale":"2023-10-06T05:25:01.827284+0000","last_undegraded":"2023-10-06T05:25:01.827284+0000","last_fullsized":"2023-10-06T05:25:01.827284+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:32.921424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.811503+0000","last_change":"2023-10-06T05:25:00.825691+0000","last_active":"2023-10-06T05:25:01.811503+0000","last_peered":"2023-10-06T05:25:01.811503+0000","last_clean":"2023-10-06T05:25:01.811503+0000","last_became_active":"2023-10-06T05:25:00.825390+0000","last_became_peered":"2023-10-06T05:25:00.825390+0000","last_unstale":"2023-10-06T05:25:01.811503+0000","last_undegraded":"2023-10-06T05:25:01.811503+0000","last_fullsized":"2023-10-06T05:25:01.811503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:30.943459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826808+0000","last_change":"2023-10-06T05:25:01.826808+0000","last_active":"2023-10-06T05:25:01.826808+0000","last_peered":"2023-10-06T05:25:01.826808+0000","last_clean":"2023-10-06T05:25:01.826808+0000","last_became_active":"2023-10-06T05:25:01.826470+0000","last_became_peered":"2023-10-06T05:25:01.826470+0000","last_unstale":"2023-10-06T05:25:01.826808+0000","last_undegraded":"2023-10-06T05:25:01.826808+0000","last_fullsized":"2023-10-06T05:25:01.826808+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:48.553439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.829029+0000","last_change":"2023-10-06T05:25:00.815175+0000","last_active":"2023-10-06T05:25:01.829029+0000","last_peered":"2023-10-06T05:25:01.829029+0000","last_clean":"2023-10-06T05:25:01.829029+0000","last_became_active":"2023-10-06T05:25:00.814854+0000","last_became_peered":"2023-10-06T05:25:00.814854+0000","last_unstale":"2023-10-06T05:25:01.829029+0000","last_undegraded":"2023-10-06T05:25:01.829029+0000","last_fullsized":"2023-10-06T05:25:01.829029+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:02.337673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826509+0000","last_change":"2023-10-06T05:25:01.826509+0000","last_active":"2023-10-06T05:25:01.826509+0000","last_peered":"2023-10-06T05:25:01.826509+0000","last_clean":"2023-10-06T05:25:01.826509+0000","last_became_active":"2023-10-06T05:25:01.826235+0000","last_became_peered":"2023-10-06T05:25:01.826235+0000","last_unstale":"2023-10-06T05:25:01.826509+0000","last_undegraded":"2023-10-06T05:25:01.826509+0000","last_fullsized":"2023-10-06T05:25:01.826509+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:00.796847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.830531+0000","last_change":"2023-10-06T05:25:00.817182+0000","last_active":"2023-10-06T05:25:01.830531+0000","last_peered":"2023-10-06T05:25:01.830531+0000","last_clean":"2023-10-06T05:25:01.830531+0000","last_became_active":"2023-10-06T05:25:00.816919+0000","last_became_peered":"2023-10-06T05:25:00.816919+0000","last_unstale":"2023-10-06T05:25:01.830531+0000","last_undegraded":"2023-10-06T05:25:01.830531+0000","last_fullsized":"2023-10-06T05:25:01.830531+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:34.281255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838572+0000","last_change":"2023-10-06T05:25:01.838572+0000","last_active":"2023-10-06T05:25:01.838572+0000","last_peered":"2023-10-06T05:25:01.838572+0000","last_clean":"2023-10-06T05:25:01.838572+0000","last_became_active":"2023-10-06T05:25:01.838307+0000","last_became_peered":"2023-10-06T05:25:01.838307+0000","last_unstale":"2023-10-06T05:25:01.838572+0000","last_undegraded":"2023-10-06T05:25:01.838572+0000","last_fullsized":"2023-10-06T05:25:01.838572+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:05.773514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.806899+0000","last_change":"2023-10-06T05:25:00.828191+0000","last_active":"2023-10-06T05:25:01.806899+0000","last_peered":"2023-10-06T05:25:01.806899+0000","last_clean":"2023-10-06T05:25:01.806899+0000","last_became_active":"2023-10-06T05:25:00.827814+0000","last_became_peered":"2023-10-06T05:25:00.827814+0000","last_unstale":"2023-10-06T05:25:01.806899+0000","last_undegraded":"2023-10-06T05:25:01.806899+0000","last_fullsized":"2023-10-06T05:25:01.806899+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:20.003764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.823667+0000","last_change":"2023-10-06T05:25:01.823667+0000","last_active":"2023-10-06T05:25:01.823667+0000","last_peered":"2023-10-06T05:25:01.823667+0000","last_clean":"2023-10-06T05:25:01.823667+0000","last_became_active":"2023-10-06T05:25:01.823288+0000","last_became_peered":"2023-10-06T05:25:01.823288+0000","last_unstale":"2023-10-06T05:25:01.823667+0000","last_undegraded":"2023-10-06T05:25:01.823667+0000","last_fullsized":"2023-10-06T05:25:01.823667+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:41.286775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.808647+0000","last_change":"2023-10-06T05:25:00.818938+0000","last_active":"2023-10-06T05:25:01.808647+0000","last_peered":"2023-10-06T05:25:01.808647+0000","last_clean":"2023-10-06T05:25:01.808647+0000","last_became_active":"2023-10-06T05:25:00.818656+0000","last_became_peered":"2023-10-06T05:25:00.818656+0000","last_unstale":"2023-10-06T05:25:01.808647+0000","last_undegraded":"2023-10-06T05:25:01.808647+0000","last_fullsized":"2023-10-06T05:25:01.808647+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:38.985887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.815814+0000","last_change":"2023-10-06T05:25:01.815814+0000","last_active":"2023-10-06T05:25:01.815814+0000","last_peered":"2023-10-06T05:25:01.815814+0000","last_clean":"2023-10-06T05:25:01.815814+0000","last_became_active":"2023-10-06T05:25:01.815545+0000","last_became_peered":"2023-10-06T05:25:01.815545+0000","last_unstale":"2023-10-06T05:25:01.815814+0000","last_undegraded":"2023-10-06T05:25:01.815814+0000","last_fullsized":"2023-10-06T05:25:01.815814+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:20.984857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.817671+0000","last_change":"2023-10-06T05:25:00.815937+0000","last_active":"2023-10-06T05:25:00.817671+0000","last_peered":"2023-10-06T05:25:00.817671+0000","last_clean":"2023-10-06T05:25:00.817671+0000","last_became_active":"2023-10-06T05:25:00.815495+0000","last_became_peered":"2023-10-06T05:25:00.815495+0000","last_unstale":"2023-10-06T05:25:00.817671+0000","last_undegraded":"2023-10-06T05:25:00.817671+0000","last_fullsized":"2023-10-06T05:25:00.817671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:11.699003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834362+0000","last_change":"2023-10-06T05:25:01.834362+0000","last_active":"2023-10-06T05:25:01.834362+0000","last_peered":"2023-10-06T05:25:01.834362+0000","last_clean":"2023-10-06T05:25:01.834362+0000","last_became_active":"2023-10-06T05:25:01.834099+0000","last_became_peered":"2023-10-06T05:25:01.834099+0000","last_unstale":"2023-10-06T05:25:01.834362+0000","last_undegraded":"2023-10-06T05:25:01.834362+0000","last_fullsized":"2023-10-06T05:25:01.834362+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:57.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.824555+0000","last_change":"2023-10-06T05:25:00.822837+0000","last_active":"2023-10-06T05:25:00.824555+0000","last_peered":"2023-10-06T05:25:00.824555+0000","last_clean":"2023-10-06T05:25:00.824555+0000","last_became_active":"2023-10-06T05:25:00.822499+0000","last_became_peered":"2023-10-06T05:25:00.822499+0000","last_unstale":"2023-10-06T05:25:00.824555+0000","last_undegraded":"2023-10-06T05:25:00.824555+0000","last_fullsized":"2023-10-06T05:25:00.824555+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:25.772262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.811643+0000","last_change":"2023-10-06T05:25:01.811643+0000","last_active":"2023-10-06T05:25:01.811643+0000","last_peered":"2023-10-06T05:25:01.811643+0000","last_clean":"2023-10-06T05:25:01.811643+0000","last_became_active":"2023-10-06T05:25:01.811346+0000","last_became_peered":"2023-10-06T05:25:01.811346+0000","last_unstale":"2023-10-06T05:25:01.811643+0000","last_undegraded":"2023-10-06T05:25:01.811643+0000","last_fullsized":"2023-10-06T05:25:01.811643+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:23.170230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.818940+0000","last_change":"2023-10-06T05:25:00.815505+0000","last_active":"2023-10-06T05:25:01.818940+0000","last_peered":"2023-10-06T05:25:01.818940+0000","last_clean":"2023-10-06T05:25:01.818940+0000","last_became_active":"2023-10-06T05:25:00.815222+0000","last_became_peered":"2023-10-06T05:25:00.815222+0000","last_unstale":"2023-10-06T05:25:01.818940+0000","last_undegraded":"2023-10-06T05:25:01.818940+0000","last_fullsized":"2023-10-06T05:25:01.818940+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:22.167048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.813428+0000","last_change":"2023-10-06T05:25:01.813428+0000","last_active":"2023-10-06T05:25:01.813428+0000","last_peered":"2023-10-06T05:25:01.813428+0000","last_clean":"2023-10-06T05:25:01.813428+0000","last_became_active":"2023-10-06T05:25:01.812521+0000","last_became_peered":"2023-10-06T05:25:01.812521+0000","last_unstale":"2023-10-06T05:25:01.813428+0000","last_undegraded":"2023-10-06T05:25:01.813428+0000","last_fullsized":"2023-10-06T05:25:01.813428+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:10.956943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.835570+0000","last_change":"2023-10-06T05:25:00.823266+0000","last_active":"2023-10-06T05:25:01.835570+0000","last_peered":"2023-10-06T05:25:01.835570+0000","last_clean":"2023-10-06T05:25:01.835570+0000","last_became_active":"2023-10-06T05:25:00.822966+0000","last_became_peered":"2023-10-06T05:25:00.822966+0000","last_unstale":"2023-10-06T05:25:01.835570+0000","last_undegraded":"2023-10-06T05:25:01.835570+0000","last_fullsized":"2023-10-06T05:25:01.835570+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:57.016157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834154+0000","last_change":"2023-10-06T05:25:01.834154+0000","last_active":"2023-10-06T05:25:01.834154+0000","last_peered":"2023-10-06T05:25:01.834154+0000","last_clean":"2023-10-06T05:25:01.834154+0000","last_became_active":"2023-10-06T05:25:01.833889+0000","last_became_peered":"2023-10-06T05:25:01.833889+0000","last_unstale":"2023-10-06T05:25:01.834154+0000","last_undegraded":"2023-10-06T05:25:01.834154+0000","last_fullsized":"2023-10-06T05:25:01.834154+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:35.045088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"155'1","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.807757+0000","last_change":"2023-10-06T05:25:00.817258+0000","last_active":"2023-10-06T05:25:01.807757+0000","last_peered":"2023-10-06T05:25:01.807757+0000","last_clean":"2023-10-06T05:25:01.807757+0000","last_became_active":"2023-10-06T05:25:00.816968+0000","last_became_peered":"2023-10-06T05:25:00.816968+0000","last_unstale":"2023-10-06T05:25:01.807757+0000","last_undegraded":"2023-10-06T05:25:01.807757+0000","last_fullsized":"2023-10-06T05:25:01.807757+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:47.308067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821158+0000","last_change":"2023-10-06T05:25:01.821158+0000","last_active":"2023-10-06T05:25:01.821158+0000","last_peered":"2023-10-06T05:25:01.821158+0000","last_clean":"2023-10-06T05:25:01.821158+0000","last_became_active":"2023-10-06T05:25:01.820877+0000","last_became_peered":"2023-10-06T05:25:01.820877+0000","last_unstale":"2023-10-06T05:25:01.821158+0000","last_undegraded":"2023-10-06T05:25:01.821158+0000","last_fullsized":"2023-10-06T05:25:01.821158+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:19.350475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.833036+0000","last_change":"2023-10-06T05:25:00.828055+0000","last_active":"2023-10-06T05:25:00.833036+0000","last_peered":"2023-10-06T05:25:00.833036+0000","last_clean":"2023-10-06T05:25:00.833036+0000","last_became_active":"2023-10-06T05:25:00.827813+0000","last_became_peered":"2023-10-06T05:25:00.827813+0000","last_unstale":"2023-10-06T05:25:00.833036+0000","last_undegraded":"2023-10-06T05:25:00.833036+0000","last_fullsized":"2023-10-06T05:25:00.833036+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:34.041906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834596+0000","last_change":"2023-10-06T05:25:01.834596+0000","last_active":"2023-10-06T05:25:01.834596+0000","last_peered":"2023-10-06T05:25:01.834596+0000","last_clean":"2023-10-06T05:25:01.834596+0000","last_became_active":"2023-10-06T05:25:01.834341+0000","last_became_peered":"2023-10-06T05:25:01.834341+0000","last_unstale":"2023-10-06T05:25:01.834596+0000","last_undegraded":"2023-10-06T05:25:01.834596+0000","last_fullsized":"2023-10-06T05:25:01.834596+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:09.879405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.841160+0000","last_change":"2023-10-06T05:25:00.818787+0000","last_active":"2023-10-06T05:25:01.841160+0000","last_peered":"2023-10-06T05:25:01.841160+0000","last_clean":"2023-10-06T05:25:01.841160+0000","last_became_active":"2023-10-06T05:25:00.818471+0000","last_became_peered":"2023-10-06T05:25:00.818471+0000","last_unstale":"2023-10-06T05:25:01.841160+0000","last_undegraded":"2023-10-06T05:25:01.841160+0000","last_fullsized":"2023-10-06T05:25:01.841160+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:21.175330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.812488+0000","last_change":"2023-10-06T05:25:01.812488+0000","last_active":"2023-10-06T05:25:01.812488+0000","last_peered":"2023-10-06T05:25:01.812488+0000","last_clean":"2023-10-06T05:25:01.812488+0000","last_became_active":"2023-10-06T05:25:01.812203+0000","last_became_peered":"2023-10-06T05:25:01.812203+0000","last_unstale":"2023-10-06T05:25:01.812488+0000","last_undegraded":"2023-10-06T05:25:01.812488+0000","last_fullsized":"2023-10-06T05:25:01.812488+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:59.858227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.816083+0000","last_change":"2023-10-06T05:25:00.814570+0000","last_active":"2023-10-06T05:25:00.816083+0000","last_peered":"2023-10-06T05:25:00.816083+0000","last_clean":"2023-10-06T05:25:00.816083+0000","last_became_active":"2023-10-06T05:25:00.814313+0000","last_became_peered":"2023-10-06T05:25:00.814313+0000","last_unstale":"2023-10-06T05:25:00.816083+0000","last_undegraded":"2023-10-06T05:25:00.816083+0000","last_fullsized":"2023-10-06T05:25:00.816083+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:36.972771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.817919+0000","last_change":"2023-10-06T05:25:01.817919+0000","last_active":"2023-10-06T05:25:01.817919+0000","last_peered":"2023-10-06T05:25:01.817919+0000","last_clean":"2023-10-06T05:25:01.817919+0000","last_became_active":"2023-10-06T05:25:01.817542+0000","last_became_peered":"2023-10-06T05:25:01.817542+0000","last_unstale":"2023-10-06T05:25:01.817919+0000","last_undegraded":"2023-10-06T05:25:01.817919+0000","last_fullsized":"2023-10-06T05:25:01.817919+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:20.600837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.812614+0000","last_change":"2023-10-06T05:25:00.808172+0000","last_active":"2023-10-06T05:25:00.812614+0000","last_peered":"2023-10-06T05:25:00.812614+0000","last_clean":"2023-10-06T05:25:00.812614+0000","last_became_active":"2023-10-06T05:25:00.807870+0000","last_became_peered":"2023-10-06T05:25:00.807870+0000","last_unstale":"2023-10-06T05:25:00.812614+0000","last_undegraded":"2023-10-06T05:25:00.812614+0000","last_fullsized":"2023-10-06T05:25:00.812614+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:39.127957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826245+0000","last_change":"2023-10-06T05:25:01.826245+0000","last_active":"2023-10-06T05:25:01.826245+0000","last_peered":"2023-10-06T05:25:01.826245+0000","last_clean":"2023-10-06T05:25:01.826245+0000","last_became_active":"2023-10-06T05:25:01.825943+0000","last_became_peered":"2023-10-06T05:25:01.825943+0000","last_unstale":"2023-10-06T05:25:01.826245+0000","last_undegraded":"2023-10-06T05:25:01.826245+0000","last_fullsized":"2023-10-06T05:25:01.826245+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:31.946641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.823434+0000","last_change":"2023-10-06T05:25:00.819383+0000","last_active":"2023-10-06T05:25:00.823434+0000","last_peered":"2023-10-06T05:25:00.823434+0000","last_clean":"2023-10-06T05:25:00.823434+0000","last_became_active":"2023-10-06T05:25:00.818908+0000","last_became_peered":"2023-10-06T05:25:00.818908+0000","last_unstale":"2023-10-06T05:25:00.823434+0000","last_undegraded":"2023-10-06T05:25:00.823434+0000","last_fullsized":"2023-10-06T05:25:00.823434+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.814185+0000","last_change":"2023-10-06T05:25:01.814185+0000","last_active":"2023-10-06T05:25:01.814185+0000","last_peered":"2023-10-06T05:25:01.814185+0000","last_clean":"2023-10-06T05:25:01.814185+0000","last_became_active":"2023-10-06T05:25:01.813906+0000","last_became_peered":"2023-10-06T05:25:01.813906+0000","last_unstale":"2023-10-06T05:25:01.814185+0000","last_undegraded":"2023-10-06T05:25:01.814185+0000","last_fullsized":"2023-10-06T05:25:01.814185+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:05.736590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.840787+0000","last_change":"2023-10-06T05:25:00.816147+0000","last_active":"2023-10-06T05:25:01.840787+0000","last_peered":"2023-10-06T05:25:01.840787+0000","last_clean":"2023-10-06T05:25:01.840787+0000","last_became_active":"2023-10-06T05:25:00.815880+0000","last_became_peered":"2023-10-06T05:25:00.815880+0000","last_unstale":"2023-10-06T05:25:01.840787+0000","last_undegraded":"2023-10-06T05:25:01.840787+0000","last_fullsized":"2023-10-06T05:25:01.840787+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:42.360638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.820375+0000","last_change":"2023-10-06T05:25:01.820375+0000","last_active":"2023-10-06T05:25:01.820375+0000","last_peered":"2023-10-06T05:25:01.820375+0000","last_clean":"2023-10-06T05:25:01.820375+0000","last_became_active":"2023-10-06T05:25:01.820064+0000","last_became_peered":"2023-10-06T05:25:01.820064+0000","last_unstale":"2023-10-06T05:25:01.820375+0000","last_undegraded":"2023-10-06T05:25:01.820375+0000","last_fullsized":"2023-10-06T05:25:01.820375+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:28.586702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.826706+0000","last_change":"2023-10-06T05:25:00.824176+0000","last_active":"2023-10-06T05:25:00.826706+0000","last_peered":"2023-10-06T05:25:00.826706+0000","last_clean":"2023-10-06T05:25:00.826706+0000","last_became_active":"2023-10-06T05:25:00.823933+0000","last_became_peered":"2023-10-06T05:25:00.823933+0000","last_unstale":"2023-10-06T05:25:00.826706+0000","last_undegraded":"2023-10-06T05:25:00.826706+0000","last_fullsized":"2023-10-06T05:25:00.826706+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:04.770332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.816436+0000","last_change":"2023-10-06T05:25:01.816436+0000","last_active":"2023-10-06T05:25:01.816436+0000","last_peered":"2023-10-06T05:25:01.816436+0000","last_clean":"2023-10-06T05:25:01.816436+0000","last_became_active":"2023-10-06T05:25:01.816165+0000","last_became_peered":"2023-10-06T05:25:01.816165+0000","last_unstale":"2023-10-06T05:25:01.816436+0000","last_undegraded":"2023-10-06T05:25:01.816436+0000","last_fullsized":"2023-10-06T05:25:01.816436+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:11.785855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.824308+0000","last_change":"2023-10-06T05:25:00.820365+0000","last_active":"2023-10-06T05:25:00.824308+0000","last_peered":"2023-10-06T05:25:00.824308+0000","last_clean":"2023-10-06T05:25:00.824308+0000","last_became_active":"2023-10-06T05:25:00.820096+0000","last_became_peered":"2023-10-06T05:25:00.820096+0000","last_unstale":"2023-10-06T05:25:00.824308+0000","last_undegraded":"2023-10-06T05:25:00.824308+0000","last_fullsized":"2023-10-06T05:25:00.824308+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:27.583520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821126+0000","last_change":"2023-10-06T05:25:01.821126+0000","last_active":"2023-10-06T05:25:01.821126+0000","last_peered":"2023-10-06T05:25:01.821126+0000","last_clean":"2023-10-06T05:25:01.821126+0000","last_became_active":"2023-10-06T05:25:01.820778+0000","last_became_peered":"2023-10-06T05:25:01.820778+0000","last_unstale":"2023-10-06T05:25:01.821126+0000","last_undegraded":"2023-10-06T05:25:01.821126+0000","last_fullsized":"2023-10-06T05:25:01.821126+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:19.649700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.821539+0000","last_change":"2023-10-06T05:25:00.820643+0000","last_active":"2023-10-06T05:25:00.821539+0000","last_peered":"2023-10-06T05:25:00.821539+0000","last_clean":"2023-10-06T05:25:00.821539+0000","last_became_active":"2023-10-06T05:25:00.820368+0000","last_became_peered":"2023-10-06T05:25:00.820368+0000","last_unstale":"2023-10-06T05:25:00.821539+0000","last_undegraded":"2023-10-06T05:25:00.821539+0000","last_fullsized":"2023-10-06T05:25:00.821539+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:58.444398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838250+0000","last_change":"2023-10-06T05:25:01.838250+0000","last_active":"2023-10-06T05:25:01.838250+0000","last_peered":"2023-10-06T05:25:01.838250+0000","last_clean":"2023-10-06T05:25:01.838250+0000","last_became_active":"2023-10-06T05:25:01.837885+0000","last_became_peered":"2023-10-06T05:25:01.837885+0000","last_unstale":"2023-10-06T05:25:01.838250+0000","last_undegraded":"2023-10-06T05:25:01.838250+0000","last_fullsized":"2023-10-06T05:25:01.838250+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:04.638103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.823697+0000","last_change":"2023-10-06T05:25:00.820603+0000","last_active":"2023-10-06T05:25:00.823697+0000","last_peered":"2023-10-06T05:25:00.823697+0000","last_clean":"2023-10-06T05:25:00.823697+0000","last_became_active":"2023-10-06T05:25:00.820279+0000","last_became_peered":"2023-10-06T05:25:00.820279+0000","last_unstale":"2023-10-06T05:25:00.823697+0000","last_undegraded":"2023-10-06T05:25:00.823697+0000","last_fullsized":"2023-10-06T05:25:00.823697+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:23.596756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.827266+0000","last_change":"2023-10-06T05:25:01.827266+0000","last_active":"2023-10-06T05:25:01.827266+0000","last_peered":"2023-10-06T05:25:01.827266+0000","last_clean":"2023-10-06T05:25:01.827266+0000","last_became_active":"2023-10-06T05:25:01.826904+0000","last_became_peered":"2023-10-06T05:25:01.826904+0000","last_unstale":"2023-10-06T05:25:01.827266+0000","last_undegraded":"2023-10-06T05:25:01.827266+0000","last_fullsized":"2023-10-06T05:25:01.827266+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:39.989069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.819610+0000","last_change":"2023-10-06T05:25:00.812525+0000","last_active":"2023-10-06T05:25:00.819610+0000","last_peered":"2023-10-06T05:25:00.819610+0000","last_clean":"2023-10-06T05:25:00.819610+0000","last_became_active":"2023-10-06T05:25:00.812231+0000","last_became_peered":"2023-10-06T05:25:00.812231+0000","last_unstale":"2023-10-06T05:25:00.819610+0000","last_undegraded":"2023-10-06T05:25:00.819610+0000","last_fullsized":"2023-10-06T05:25:00.819610+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:59.793665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.839929+0000","last_change":"2023-10-06T05:25:01.839929+0000","last_active":"2023-10-06T05:25:01.839929+0000","last_peered":"2023-10-06T05:25:01.839929+0000","last_clean":"2023-10-06T05:25:01.839929+0000","last_became_active":"2023-10-06T05:25:01.839624+0000","last_became_peered":"2023-10-06T05:25:01.839624+0000","last_unstale":"2023-10-06T05:25:01.839929+0000","last_undegraded":"2023-10-06T05:25:01.839929+0000","last_fullsized":"2023-10-06T05:25:01.839929+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:48.477374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.820819+0000","last_change":"2023-10-06T05:25:00.819347+0000","last_active":"2023-10-06T05:25:00.820819+0000","last_peered":"2023-10-06T05:25:00.820819+0000","last_clean":"2023-10-06T05:25:00.820819+0000","last_became_active":"2023-10-06T05:25:00.819011+0000","last_became_peered":"2023-10-06T05:25:00.819011+0000","last_unstale":"2023-10-06T05:25:00.820819+0000","last_undegraded":"2023-10-06T05:25:00.820819+0000","last_fullsized":"2023-10-06T05:25:00.820819+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:32.185150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.818528+0000","last_change":"2023-10-06T05:25:01.818528+0000","last_active":"2023-10-06T05:25:01.818528+0000","last_peered":"2023-10-06T05:25:01.818528+0000","last_clean":"2023-10-06T05:25:01.818528+0000","last_became_active":"2023-10-06T05:25:01.818253+0000","last_became_peered":"2023-10-06T05:25:01.818253+0000","last_unstale":"2023-10-06T05:25:01.818528+0000","last_undegraded":"2023-10-06T05:25:01.818528+0000","last_fullsized":"2023-10-06T05:25:01.818528+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.625028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.821671+0000","last_change":"2023-10-06T05:25:00.818496+0000","last_active":"2023-10-06T05:25:00.821671+0000","last_peered":"2023-10-06T05:25:00.821671+0000","last_clean":"2023-10-06T05:25:00.821671+0000","last_became_active":"2023-10-06T05:25:00.817743+0000","last_became_peered":"2023-10-06T05:25:00.817743+0000","last_unstale":"2023-10-06T05:25:00.821671+0000","last_undegraded":"2023-10-06T05:25:00.821671+0000","last_fullsized":"2023-10-06T05:25:00.821671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:59.793665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.820459+0000","last_change":"2023-10-06T05:25:01.820459+0000","last_active":"2023-10-06T05:25:01.820459+0000","last_peered":"2023-10-06T05:25:01.820459+0000","last_clean":"2023-10-06T05:25:01.820459+0000","last_became_active":"2023-10-06T05:25:01.819428+0000","last_became_peered":"2023-10-06T05:25:01.819428+0000","last_unstale":"2023-10-06T05:25:01.820459+0000","last_undegraded":"2023-10-06T05:25:01.820459+0000","last_fullsized":"2023-10-06T05:25:01.820459+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:58.019339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.819195+0000","last_change":"2023-10-06T05:25:00.817255+0000","last_active":"2023-10-06T05:25:00.819195+0000","last_peered":"2023-10-06T05:25:00.819195+0000","last_clean":"2023-10-06T05:25:00.819195+0000","last_became_active":"2023-10-06T05:25:00.815987+0000","last_became_peered":"2023-10-06T05:25:00.815987+0000","last_unstale":"2023-10-06T05:25:00.819195+0000","last_undegraded":"2023-10-06T05:25:00.819195+0000","last_fullsized":"2023-10-06T05:25:00.819195+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:05.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.814847+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:25:00.814847+0000","last_peered":"2023-10-06T05:25:00.814847+0000","last_clean":"2023-10-06T05:25:00.814847+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:25:00.814847+0000","last_undegraded":"2023-10-06T05:25:00.814847+0000","last_fullsized":"2023-10-06T05:25:00.814847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.826847+0000","last_change":"2023-10-06T05:25:01.826847+0000","last_active":"2023-10-06T05:25:01.826847+0000","last_peered":"2023-10-06T05:25:01.826847+0000","last_clean":"2023-10-06T05:25:01.826847+0000","last_became_active":"2023-10-06T05:25:01.825519+0000","last_became_peered":"2023-10-06T05:25:01.825519+0000","last_unstale":"2023-10-06T05:25:01.826847+0000","last_undegraded":"2023-10-06T05:25:01.826847+0000","last_fullsized":"2023-10-06T05:25:01.826847+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:43.363820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.823052+0000","last_change":"2023-10-06T05:25:00.818514+0000","last_active":"2023-10-06T05:25:00.823052+0000","last_peered":"2023-10-06T05:25:00.823052+0000","last_clean":"2023-10-06T05:25:00.823052+0000","last_became_active":"2023-10-06T05:25:00.818262+0000","last_became_peered":"2023-10-06T05:25:00.818262+0000","last_unstale":"2023-10-06T05:25:00.823052+0000","last_undegraded":"2023-10-06T05:25:00.823052+0000","last_fullsized":"2023-10-06T05:25:00.823052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:46.208783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":122,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.811591+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:25:00.811591+0000","last_peered":"2023-10-06T05:25:00.811591+0000","last_clean":"2023-10-06T05:25:00.811591+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:25:00.811591+0000","last_undegraded":"2023-10-06T05:25:00.811591+0000","last_fullsized":"2023-10-06T05:25:00.811591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.818502+0000","last_change":"2023-10-06T05:25:01.818502+0000","last_active":"2023-10-06T05:25:01.818502+0000","last_peered":"2023-10-06T05:25:01.818502+0000","last_clean":"2023-10-06T05:25:01.818502+0000","last_became_active":"2023-10-06T05:25:01.818227+0000","last_became_peered":"2023-10-06T05:25:01.818227+0000","last_unstale":"2023-10-06T05:25:01.818502+0000","last_undegraded":"2023-10-06T05:25:01.818502+0000","last_fullsized":"2023-10-06T05:25:01.818502+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:36.492596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"155'6","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.831540+0000","last_change":"2023-10-06T05:25:00.819108+0000","last_active":"2023-10-06T05:25:01.831540+0000","last_peered":"2023-10-06T05:25:01.831540+0000","last_clean":"2023-10-06T05:25:01.831540+0000","last_became_active":"2023-10-06T05:25:00.818855+0000","last_became_peered":"2023-10-06T05:25:00.818855+0000","last_unstale":"2023-10-06T05:25:01.831540+0000","last_undegraded":"2023-10-06T05:25:01.831540+0000","last_fullsized":"2023-10-06T05:25:01.831540+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:27.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.800628+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:25:00.800628+0000","last_peered":"2023-10-06T05:25:00.800628+0000","last_clean":"2023-10-06T05:25:00.800628+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:25:00.800628+0000","last_undegraded":"2023-10-06T05:25:00.800628+0000","last_fullsized":"2023-10-06T05:25:00.800628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.825798+0000","last_change":"2023-10-06T05:25:01.825798+0000","last_active":"2023-10-06T05:25:01.825798+0000","last_peered":"2023-10-06T05:25:01.825798+0000","last_clean":"2023-10-06T05:25:01.825798+0000","last_became_active":"2023-10-06T05:25:01.825409+0000","last_became_peered":"2023-10-06T05:25:01.825409+0000","last_unstale":"2023-10-06T05:25:01.825798+0000","last_undegraded":"2023-10-06T05:25:01.825798+0000","last_fullsized":"2023-10-06T05:25:01.825798+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:04.378686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.826739+0000","last_change":"2023-10-06T05:25:00.824174+0000","last_active":"2023-10-06T05:25:00.826739+0000","last_peered":"2023-10-06T05:25:00.826739+0000","last_clean":"2023-10-06T05:25:00.826739+0000","last_became_active":"2023-10-06T05:25:00.823902+0000","last_became_peered":"2023-10-06T05:25:00.823902+0000","last_unstale":"2023-10-06T05:25:00.826739+0000","last_undegraded":"2023-10-06T05:25:00.826739+0000","last_fullsized":"2023-10-06T05:25:00.826739+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:48.760477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.811025+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:25:00.811025+0000","last_peered":"2023-10-06T05:25:00.811025+0000","last_clean":"2023-10-06T05:25:00.811025+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:25:00.811025+0000","last_undegraded":"2023-10-06T05:25:00.811025+0000","last_fullsized":"2023-10-06T05:25:00.811025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":273,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.810655+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:25:00.810655+0000","last_peered":"2023-10-06T05:25:00.810655+0000","last_clean":"2023-10-06T05:25:00.810655+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:25:00.810655+0000","last_undegraded":"2023-10-06T05:25:00.810655+0000","last_fullsized":"2023-10-06T05:25:00.810655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.831413+0000","last_change":"2023-10-06T05:25:01.831413+0000","last_active":"2023-10-06T05:25:01.831413+0000","last_peered":"2023-10-06T05:25:01.831413+0000","last_clean":"2023-10-06T05:25:01.831413+0000","last_became_active":"2023-10-06T05:25:01.831131+0000","last_became_peered":"2023-10-06T05:25:01.831131+0000","last_unstale":"2023-10-06T05:25:01.831413+0000","last_undegraded":"2023-10-06T05:25:01.831413+0000","last_fullsized":"2023-10-06T05:25:01.831413+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:51.482718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"155'3","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.833578+0000","last_change":"2023-10-06T05:25:00.816811+0000","last_active":"2023-10-06T05:25:01.833578+0000","last_peered":"2023-10-06T05:25:01.833578+0000","last_clean":"2023-10-06T05:25:01.833578+0000","last_became_active":"2023-10-06T05:25:00.816551+0000","last_became_peered":"2023-10-06T05:25:00.816551+0000","last_unstale":"2023-10-06T05:25:01.833578+0000","last_undegraded":"2023-10-06T05:25:01.833578+0000","last_fullsized":"2023-10-06T05:25:01.833578+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:04.733408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.810566+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:25:00.810566+0000","last_peered":"2023-10-06T05:25:00.810566+0000","last_clean":"2023-10-06T05:25:00.810566+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:25:00.810566+0000","last_undegraded":"2023-10-06T05:25:00.810566+0000","last_fullsized":"2023-10-06T05:25:00.810566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.834886+0000","last_change":"2023-10-06T05:25:01.834886+0000","last_active":"2023-10-06T05:25:01.834886+0000","last_peered":"2023-10-06T05:25:01.834886+0000","last_clean":"2023-10-06T05:25:01.834886+0000","last_became_active":"2023-10-06T05:25:01.834610+0000","last_became_peered":"2023-10-06T05:25:01.834610+0000","last_unstale":"2023-10-06T05:25:01.834886+0000","last_undegraded":"2023-10-06T05:25:01.834886+0000","last_fullsized":"2023-10-06T05:25:01.834886+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:22.829190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"155'8","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.840382+0000","last_change":"2023-10-06T05:25:00.826903+0000","last_active":"2023-10-06T05:25:01.840382+0000","last_peered":"2023-10-06T05:25:01.840382+0000","last_clean":"2023-10-06T05:25:01.840382+0000","last_became_active":"2023-10-06T05:25:00.826602+0000","last_became_peered":"2023-10-06T05:25:00.826602+0000","last_unstale":"2023-10-06T05:25:01.840382+0000","last_undegraded":"2023-10-06T05:25:01.840382+0000","last_fullsized":"2023-10-06T05:25:01.840382+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:49.487935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":120,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.813085+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:25:00.813085+0000","last_peered":"2023-10-06T05:25:00.813085+0000","last_clean":"2023-10-06T05:25:00.813085+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:25:00.813085+0000","last_undegraded":"2023-10-06T05:25:00.813085+0000","last_fullsized":"2023-10-06T05:25:00.813085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.821952+0000","last_change":"2023-10-06T05:25:01.821952+0000","last_active":"2023-10-06T05:25:01.821952+0000","last_peered":"2023-10-06T05:25:01.821952+0000","last_clean":"2023-10-06T05:25:01.821952+0000","last_became_active":"2023-10-06T05:25:01.821672+0000","last_became_peered":"2023-10-06T05:25:01.821672+0000","last_unstale":"2023-10-06T05:25:01.821952+0000","last_undegraded":"2023-10-06T05:25:01.821952+0000","last_fullsized":"2023-10-06T05:25:01.821952+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:27.960907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"155'2","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.838582+0000","last_change":"2023-10-06T05:25:00.816635+0000","last_active":"2023-10-06T05:25:01.838582+0000","last_peered":"2023-10-06T05:25:01.838582+0000","last_clean":"2023-10-06T05:25:01.838582+0000","last_became_active":"2023-10-06T05:25:00.816385+0000","last_became_peered":"2023-10-06T05:25:00.816385+0000","last_unstale":"2023-10-06T05:25:01.838582+0000","last_undegraded":"2023-10-06T05:25:01.838582+0000","last_fullsized":"2023-10-06T05:25:01.838582+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:44.986107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.820280+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:25:00.820280+0000","last_peered":"2023-10-06T05:25:00.820280+0000","last_clean":"2023-10-06T05:25:00.820280+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:25:00.820280+0000","last_undegraded":"2023-10-06T05:25:00.820280+0000","last_fullsized":"2023-10-06T05:25:00.820280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":8,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-06T05:25:01.813218+0000","last_change":"2023-10-06T05:25:01.813218+0000","last_active":"2023-10-06T05:25:01.813218+0000","last_peered":"2023-10-06T05:25:01.813218+0000","last_clean":"2023-10-06T05:25:01.813218+0000","last_became_active":"2023-10-06T05:25:01.812899+0000","last_became_peered":"2023-10-06T05:25:01.812899+0000","last_unstale":"2023-10-06T05:25:01.813218+0000","last_undegraded":"2023-10-06T05:25:01.813218+0000","last_fullsized":"2023-10-06T05:25:01.813218+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:00.789161+0000","last_clean_scrub_stamp":"2023-10-06T05:25:00.789161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:22.178512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.811557+0000","last_change":"2023-10-06T05:25:00.809201+0000","last_active":"2023-10-06T05:25:00.811557+0000","last_peered":"2023-10-06T05:25:00.811557+0000","last_clean":"2023-10-06T05:25:00.811557+0000","last_became_active":"2023-10-06T05:25:00.808935+0000","last_became_peered":"2023-10-06T05:25:00.808935+0000","last_unstale":"2023-10-06T05:25:00.811557+0000","last_undegraded":"2023-10-06T05:25:00.811557+0000","last_fullsized":"2023-10-06T05:25:00.811557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:59.785979+0000","last_clean_scrub_stamp":"2023-10-06T05:24:59.785979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:31.407052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T05:25:00.804722+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:25:00.804722+0000","last_peered":"2023-10-06T05:25:00.804722+0000","last_clean":"2023-10-06T05:25:00.804722+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:25:00.804722+0000","last_undegraded":"2023-10-06T05:25:00.804722+0000","last_fullsized":"2023-10-06T05:25:00.804722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303864,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067976,"statfs":{"total":96636764160,"available":96325607424,"internally_reserved":0,"allocated":2539520,"data_stored":1490023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95999999999999996,"15min":1.1850000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.42999999999999999,"15min":0.41099999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.169},"last":0.45100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":1.268},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.96099999999999997,"15min":1.075},"last":0.496}]},{"osd":2,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83999999999999997,"5min":1.0780000000000001,"15min":1.093},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.495,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.22},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.079},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.56599999999999995,"5min":0.51900000000000002,"15min":0.46600000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.202},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.258},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.39400000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.755,"15min":0.748},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.441},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.407},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.47799999999999998,"5min":0.45100000000000001,"15min":0.38600000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3300000000000001},"last":0.60099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":899655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.68999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.84799999999999998,"5min":1.0209999999999999,"15min":1.034},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65500000000000003,"15min":0.69899999999999995},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.995,"5min":0.995,"15min":1.2869999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.83199999999999996,"5min":1,"15min":1},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.98299999999999998,"15min":0.98899999999999999},"last":0.93700000000000006}]},{"osd":2,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69299999999999995,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.876,"5min":1.05,"15min":1.1599999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68899999999999995,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.745}]},{"osd":3,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92700000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.82499999999999996,"5min":1.0860000000000001,"15min":1.1020000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.65200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.95699999999999996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64400000000000002,"15min":0.68000000000000005},"min":{"1min":0.441,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.873,"5min":0.89100000000000001,"15min":0.998},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66200000000000003,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93000000000000005,"5min":0.95399999999999996,"15min":1.0760000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.747},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.96499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.96199999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73499999999999999,"15min":0.80000000000000004},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.436},"max":{"1min":0.96899999999999997,"5min":1.0349999999999999,"15min":1.1359999999999999},"last":0.84799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.497},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47899999999999998,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.67100000000000004,"5min":0.77000000000000002,"15min":2.8050000000000002},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.45800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.78800000000000003,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52500000000000002,"15min":0.58099999999999996},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86299999999999999,"5min":0.88200000000000001,"15min":2.7010000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.87,"5min":0.88,"15min":2.7639999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.57799999999999996},"min":{"1min":0.40000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":2.8370000000000002},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.59299999999999997,"5min":0.54400000000000004,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.79900000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.0469999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.95099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77400000000000002,"15min":0.80300000000000005},"min":{"1min":0.58899999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.145,"5min":1.25,"15min":1.349},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.669},"last":0.97199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81100000000000005,"15min":0.84399999999999997},"min":{"1min":0.57399999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.169,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.81000000000000005,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.503},"max":{"1min":1.204,"5min":1.3280000000000001,"15min":1.573},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.79700000000000004,"15min":0.83999999999999997},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.528},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.84499999999999997},"min":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":1.3049999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.73299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2527232,"data_stored":1482907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.63600000000000001,"5min":0.68100000000000005,"15min":0.79400000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.497},"min":{"1min":0.34599999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51100000000000001,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.65500000000000003,"5min":0.78700000000000003,"15min":0.84199999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70799999999999996,"5min":0.70999999999999996,"15min":0.84999999999999998},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72299999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.70799999999999996,"5min":0.84499999999999997,"15min":0.879},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.98699999999999999,"5min":1.1120000000000001,"15min":1.1160000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94499999999999995,"5min":1.125,"15min":1.2310000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":0.997,"5min":1.159,"15min":1.218},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92800000000000005,"5min":1.089,"15min":1.1970000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":0.97499999999999998,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.59199999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99399999999999999,"5min":1.111,"15min":1.2629999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80500000000000005},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0489999999999999,"5min":1.0780000000000001,"15min":1.25},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.04,"5min":1.256,"15min":1.2789999999999999},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":1.03,"5min":1.056,"15min":1.056},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.30599999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.94399999999999995,"5min":1.073,"15min":1.073},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69699999999999995,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.109,"5min":1.109,"15min":1.1379999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.59799999999999998,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.50700000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.58299999999999996,"5min":0.55800000000000005,"15min":0.53000000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.319},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77800000000000002,"15min":0.81200000000000006},"min":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.175,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.141},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.64100000000000001,"5min":0.56200000000000006,"15min":0.54900000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.3520000000000001},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45600000000000002,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.254},"max":{"1min":0.70199999999999996,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.47999999999999998,"15min":0.49399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.72699999999999998,"5min":0.77100000000000002,"15min":1.772},"last":0.439}]},{"osd":1,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.29099999999999998},"max":{"1min":0.58399999999999996,"5min":0.79400000000000004,"15min":1.913},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.66400000000000003,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.63,"5min":0.79700000000000004,"15min":1.954},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.77100000000000002},"min":{"1min":0.496,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.282},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":1.1839999999999999,"5min":1.216,"15min":1.2609999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.51100000000000001},"max":{"1min":1.0840000000000001,"5min":1.145,"15min":1.1930000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.82999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.3009999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.53900000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.343},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78500000000000003,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0649999999999999,"5min":1.1779999999999999,"15min":1.3380000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.83599999999999997},"min":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.194,"5min":1.194,"15min":1.3640000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.83499999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.214},"last":0.63300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.073},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.46100000000000002},"max":{"1min":0.88900000000000001,"5min":1.0089999999999999,"15min":1.169},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.41199999999999998},"max":{"1min":0.83299999999999996,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.71099999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.68200000000000005,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42699999999999999},"max":{"1min":0.81399999999999995,"5min":1.016,"15min":1.117},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.002},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.47099999999999997},"max":{"1min":0.94999999999999996,"5min":0.97599999999999998,"15min":1.0780000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":1.018,"15min":1.196},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66400000000000003,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70299999999999996,"15min":0.73299999999999998},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.95999999999999996,"5min":0.999,"15min":1.375},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70199999999999996,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.74099999999999999,"15min":0.77600000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.9390000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.752,"15min":0.78800000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.466},"last":0.872},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.81899999999999995},"min":{"1min":0.501,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":0.98699999999999999,"15min":1.218},"last":0.89300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":892539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":0.93899999999999995,"15min":1.165},"last":0.624},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":0.98999999999999999,"15min":1.1890000000000001},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.92800000000000005,"15min":0.98299999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.877,"5min":1.048,"15min":1.048},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79700000000000004,"5min":0.93600000000000005,"15min":1.204},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.746}]},{"osd":3,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90300000000000002,"5min":0.95199999999999996,"15min":1.087},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72199999999999998,"15min":0.72699999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":0.85399999999999998,"5min":0.99299999999999999,"15min":1.278},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.372},"max":{"1min":0.93600000000000005,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.1539999999999999,"15min":1.294},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.89900000000000002,"5min":0.96899999999999997,"15min":1.141},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73699999999999999,"15min":0.74199999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":1.0429999999999999,"15min":1.254},"last":0.874}]},{"osd":7,"last update":"Fri Oct 6 05:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.337},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.76500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.0609999999999999,"15min":1.248},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:25:06.670 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:25:06.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:25:06.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.895+0000 7f4983fff700 1 Processor -- start 2023-10-06T05:25:06.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.896+0000 7f4983fff700 1 -- start start 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4983fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4984159730 0x7f49840c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4983fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49840c7080 0x7f49840675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4983fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498406a3b0 con 0x7f4984067b30 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4983fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840687e0 con 0x7f4984159730 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4983fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49840c74d0 con 0x7f49840c7080 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4982ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4984159730 0x7f49840c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4982ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4984159730 0x7f49840c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56212/0 (socket says 172.21.15.33:56212) 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f4982ffd700 1 -- 172.21.15.33:0/3812627474 learned_addr learned my addr 172.21.15.33:0/3812627474 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:06.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.897+0000 7f49827fc700 1 --2- 172.21.15.33:0/3812627474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49840c7080 0x7f49840675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4982ffd700 1 -- 172.21.15.33:0/3812627474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49840c7080 msgr2=0x7f49840675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4982ffd700 1 --2- 172.21.15.33:0/3812627474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49840c7080 0x7f49840675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4982ffd700 1 -- 172.21.15.33:0/3812627474 >> v1:172.21.15.167:6789/0 conn(0x7f4984067b30 legacy=0x7f498406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4982ffd700 1 -- 172.21.15.33:0/3812627474 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984067f90 con 0x7f4984159730 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4982ffd700 1 --2- 172.21.15.33:0/3812627474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f49840c6b40 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f49740099b0 tx=0x7f4974009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c503cf1de95f1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4981ffb700 1 -- 172.21.15.33:0/3812627474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400b860 con 0x7f4984159730 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4981ffb700 1 -- 172.21.15.33:0/3812627474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400b9c0 con 0x7f4984159730 2023-10-06T05:25:06.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.898+0000 7f4981ffb700 1 -- 172.21.15.33:0/3812627474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400bd60 con 0x7f4984159730 2023-10-06T05:25:06.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 -- 172.21.15.33:0/3812627474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 msgr2=0x7f49840c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 --2- 172.21.15.33:0/3812627474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f49840c6b40 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f49740099b0 tx=0x7f4974009250 comp rx=0 tx=0).stop 2023-10-06T05:25:06.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 -- 172.21.15.33:0/3812627474 shutdown_connections 2023-10-06T05:25:06.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 --2- 172.21.15.33:0/3812627474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f49840c6b40 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 --2- 172.21.15.33:0/3812627474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49840c7080 0x7f49840675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 -- 172.21.15.33:0/3812627474 >> 172.21.15.33:0/3812627474 conn(0x7f498406f0e0 msgr2=0x7f49840cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 -- 172.21.15.33:0/3812627474 shutdown_connections 2023-10-06T05:25:06.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.899+0000 7f4983fff700 1 -- 172.21.15.33:0/3812627474 wait complete. 2023-10-06T05:25:06.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.900+0000 7f4983fff700 1 Processor -- start 2023-10-06T05:25:06.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.901+0000 7f4983fff700 1 -- start start 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.901+0000 7f4983fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4984067b30 0x7f4984157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.901+0000 7f4983fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49840c7080 0x7f49841534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.901+0000 7f4983fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f498415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.901+0000 7f4983fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498415ce00 con 0x7f4984159730 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.901+0000 7f4983fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984157340 con 0x7f49840c7080 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f4983fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498415cb20 con 0x7f4984067b30 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f498415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f498415be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56224/0 (socket says 172.21.15.33:56224) 2023-10-06T05:25:06.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 -- 172.21.15.33:0/3365806603 learned_addr learned my addr 172.21.15.33:0/3365806603 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49827fc700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49840c7080 0x7f49841534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f4982ffd700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4984067b30 0x7f4984157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 -- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4984067b30 msgr2=0x7f4984157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4984067b30 0x7f4984157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 -- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49840c7080 msgr2=0x7f49841534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49840c7080 0x7f49841534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.902+0000 7f49837fe700 1 -- 172.21.15.33:0/3365806603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740094f0 con 0x7f4984159730 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.903+0000 7f49837fe700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f498415be60 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f497c00c840 tx=0x7f497c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.903+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c012630 con 0x7f4984159730 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.903+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c012790 con 0x7f4984159730 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.903+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c012b50 con 0x7f4984159730 2023-10-06T05:25:06.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.903+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49840abf50 con 0x7f4984159730 2023-10-06T05:25:06.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.903+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49840ac510 con 0x7f4984159730 2023-10-06T05:25:06.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.904+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498404c380 con 0x7f4984159730 2023-10-06T05:25:06.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.905+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f497c003030 con 0x7f4984159730 2023-10-06T05:25:06.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.907+0000 7f496affd700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4970066f30 0x7f49700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.907+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f497c084520 con 0x7f4984159730 2023-10-06T05:25:06.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.907+0000 7f4982ffd700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4970066f30 0x7f49700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.908+0000 7f4982ffd700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4970066f30 0x7f49700693e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f49740099b0 tx=0x7f4974017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:06.910+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f497c051590 con 0x7f4984159730 2023-10-06T05:25:07.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.195+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f49840abc00 con 0x7f4984159730 2023-10-06T05:25:07.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.199+0000 7f496affd700 1 -- 172.21.15.33:0/3365806603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f497c054bb0 con 0x7f4984159730 2023-10-06T05:25:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4970066f30 msgr2=0x7f49700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4970066f30 0x7f49700693e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f49740099b0 tx=0x7f4974017000 comp rx=0 tx=0).stop 2023-10-06T05:25:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 msgr2=0x7f498415be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f498415be60 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f497c00c840 tx=0x7f497c001670 comp rx=0 tx=0).stop 2023-10-06T05:25:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 shutdown_connections 2023-10-06T05:25:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4970066f30 0x7f49700693e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4984159730 0x7f498415be60 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4984067b30 0x7f4984157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 --2- 172.21.15.33:0/3365806603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49840c7080 0x7f49841534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.201+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 >> 172.21.15.33:0/3365806603 conn(0x7f498406f0e0 msgr2=0x7f4984153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.202+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 shutdown_connections 2023-10-06T05:25:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.202+0000 7f4983fff700 1 -- 172.21.15.33:0/3365806603 wait complete. 2023-10-06T05:25:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:25:07.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:07.348 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:25:07.346+0000 7f6a7ebf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:07.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.406+0000 7f27f619d700 1 Processor -- start 2023-10-06T05:25:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.406+0000 7f27f619d700 1 -- start start 2023-10-06T05:25:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.407+0000 7f27f619d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27f00cf3b0 0x7f27f01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.407+0000 7f27f619d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f27f00af0a0 0x7f27f01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.407+0000 7f27f619d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f00cbd80 con 0x7f27f00ccfc0 2023-10-06T05:25:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.407+0000 7f27f619d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f00af5f0 con 0x7f27f00cf3b0 2023-10-06T05:25:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.407+0000 7f27f619d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f00af8d0 con 0x7f27f00af0a0 2023-10-06T05:25:07.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.407+0000 7f27f499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27f00cf3b0 0x7f27f01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27f00cf3b0 0x7f27f01594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56232/0 (socket says 172.21.15.33:56232) 2023-10-06T05:25:07.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 -- 172.21.15.33:0/2553611510 learned_addr learned my addr 172.21.15.33:0/2553611510 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:07.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f599c700 1 --2- 172.21.15.33:0/2553611510 >> v2:172.21.15.167:3300/0 conn(0x7f27f00af0a0 0x7f27f01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 -- 172.21.15.33:0/2553611510 >> v2:172.21.15.167:3300/0 conn(0x7f27f00af0a0 msgr2=0x7f27f01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 --2- 172.21.15.33:0/2553611510 >> v2:172.21.15.167:3300/0 conn(0x7f27f00af0a0 0x7f27f01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 -- 172.21.15.33:0/2553611510 >> v1:172.21.15.167:6789/0 conn(0x7f27f00ccfc0 legacy=0x7f27f00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 -- 172.21.15.33:0/2553611510 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f00afbb0 con 0x7f27f00cf3b0 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.408+0000 7f27f499a700 1 --2- 172.21.15.33:0/2553611510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00cf3b0 0x7f27f01594a0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f27e00099b0 tx=0x7f27e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0069cfa24f4ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27e7fff700 1 -- 172.21.15.33:0/2553611510 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0009ea0 con 0x7f27f00cf3b0 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27e7fff700 1 -- 172.21.15.33:0/2553611510 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e000b820 con 0x7f27f00cf3b0 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27e7fff700 1 -- 172.21.15.33:0/2553611510 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e000bbc0 con 0x7f27f00cf3b0 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 -- 172.21.15.33:0/2553611510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00cf3b0 msgr2=0x7f27f01594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 --2- 172.21.15.33:0/2553611510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00cf3b0 0x7f27f01594a0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f27e00099b0 tx=0x7f27e0009250 comp rx=0 tx=0).stop 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 -- 172.21.15.33:0/2553611510 shutdown_connections 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 --2- 172.21.15.33:0/2553611510 >> v2:172.21.15.167:3300/0 conn(0x7f27f00af0a0 0x7f27f01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 --2- 172.21.15.33:0/2553611510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00cf3b0 0x7f27f01594a0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 -- 172.21.15.33:0/2553611510 >> 172.21.15.33:0/2553611510 conn(0x7f27f006cff0 msgr2=0x7f27f006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 -- 172.21.15.33:0/2553611510 shutdown_connections 2023-10-06T05:25:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.409+0000 7f27f619d700 1 -- 172.21.15.33:0/2553611510 wait complete. 2023-10-06T05:25:07.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.411+0000 7f27f619d700 1 Processor -- start 2023-10-06T05:25:07.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.411+0000 7f27f619d700 1 -- start start 2023-10-06T05:25:07.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f619d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f00af0a0 0x7f27f014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27f00ccfc0 0x7f27f0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f619d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 0x7f27f0156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f619d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f014ec20 con 0x7f27f00c9050 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f619d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00ca1d0 con 0x7f27f00af0a0 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00c95c0 con 0x7f27f00ccfc0 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 0x7f27f0156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 0x7f27f0156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56242/0 (socket says 172.21.15.33:56242) 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 -- 172.21.15.33:0/3141418051 learned_addr learned my addr 172.21.15.33:0/3141418051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f499a700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27f00ccfc0 0x7f27f0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f519b700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f00af0a0 0x7f27f014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 -- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27f00ccfc0 msgr2=0x7f27f0155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27f00ccfc0 0x7f27f0155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 -- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f00af0a0 msgr2=0x7f27f014e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f00af0a0 0x7f27f014e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f599c700 1 -- 172.21.15.33:0/3141418051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00094f0 con 0x7f27f00c9050 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.412+0000 7f27f519b700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f00af0a0 0x7f27f014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.413+0000 7f27f599c700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 0x7f27f0156360 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009da0 tx=0x7f27e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.413+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e800cb50 con 0x7f27f00c9050 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.413+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e800f040 con 0x7f27f00c9050 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.413+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f00c98a0 con 0x7f27f00c9050 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.413+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8014480 con 0x7f27f00c9050 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.413+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f00c9e60 con 0x7f27f00c9050 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.414+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f00ca360 con 0x7f27f00c9050 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.415+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27e8014830 con 0x7f27f00c9050 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.416+0000 7f27e57fa700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27d4066f30 0x7f27d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.416+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f27e80851c0 con 0x7f27f00c9050 2023-10-06T05:25:07.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.417+0000 7f27f519b700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27d4066f30 0x7f27d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.417+0000 7f27f519b700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27d4066f30 0x7f27d40693e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f27ec000c00 tx=0x7f27ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.420+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27e80522d0 con 0x7f27f00c9050 2023-10-06T05:25:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.667+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27f004c380 con 0x7f27f00c9050 2023-10-06T05:25:07.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.668+0000 7f27e57fa700 1 -- 172.21.15.33:0/3141418051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f27e80558f0 con 0x7f27f00c9050 2023-10-06T05:25:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:07.670 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27d4066f30 msgr2=0x7f27d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27d4066f30 0x7f27d40693e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f27ec000c00 tx=0x7f27ec009040 comp rx=0 tx=0).stop 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 msgr2=0x7f27f0156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 0x7f27f0156360 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009da0 tx=0x7f27e800b040 comp rx=0 tx=0).stop 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 shutdown_connections 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f27d4066f30 0x7f27d40693e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27f00c9050 0x7f27f0156360 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f00af0a0 0x7f27f014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 --2- 172.21.15.33:0/3141418051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27f00ccfc0 0x7f27f0155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 >> 172.21.15.33:0/3141418051 conn(0x7f27f006cff0 msgr2=0x7f27f00c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 shutdown_connections 2023-10-06T05:25:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:07.670+0000 7f27f619d700 1 -- 172.21.15.33:0/3141418051 wait complete. 2023-10-06T05:25:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:08.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:25:01.298802+0000', 'modified': '2023-10-06T05:25:04.829925+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8059}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8059': {'gid': 8059, 'name': 'b', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/582175774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 582175774}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 582175774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:08.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:25:08.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:25:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.283+0000 7f869359e700 1 Processor -- start 2023-10-06T05:25:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.283+0000 7f869359e700 1 -- start start 2023-10-06T05:25:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f869359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86940c7070 0x7f86940c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f869359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8694155bc0 0x7f86940c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f869359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869406a410 con 0x7f86941557f0 2023-10-06T05:25:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f869359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940cccd0 con 0x7f8694155bc0 2023-10-06T05:25:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f869359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c7500 con 0x7f86940c7070 2023-10-06T05:25:08.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f8692d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8694155bc0 0x7f86940c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f8691d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86940c7070 0x7f86940c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f8692d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8694155bc0 0x7f86940c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:52004/0 (socket says 172.21.15.33:52004) 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.284+0000 7f8692d9d700 1 -- 172.21.15.33:0/3392320690 learned_addr learned my addr 172.21.15.33:0/3392320690 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f869159a700 1 -- 172.21.15.33:0/3392320690 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536755308 0 0) 0x7f869406a410 con 0x7f86941557f0 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f869159a700 1 -- 172.21.15.33:0/3392320690 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867c003570 con 0x7f86941557f0 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f8692d9d700 1 -- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86940c7070 msgr2=0x7f86940c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f8692d9d700 1 --2- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86940c7070 0x7f86940c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f8692d9d700 1 -- 172.21.15.33:0/3392320690 >> v1:172.21.15.167:6789/0 conn(0x7f86941557f0 legacy=0x7f86940c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f8692d9d700 1 -- 172.21.15.33:0/3392320690 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940c77e0 con 0x7f8694155bc0 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86940c7070 0x7f86940c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.285+0000 7f8692d9d700 1 --2- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f86940c9050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f868c009f20 tx=0x7f868c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17c3f79c2683874 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869159a700 1 -- 172.21.15.33:0/3392320690 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00e070 con 0x7f8694155bc0 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869159a700 1 -- 172.21.15.33:0/3392320690 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00b970 con 0x7f8694155bc0 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869159a700 1 -- 172.21.15.33:0/3392320690 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0092e0 con 0x7f8694155bc0 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 -- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 msgr2=0x7f86940c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 --2- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f86940c9050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f868c009f20 tx=0x7f868c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 -- 172.21.15.33:0/3392320690 shutdown_connections 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 --2- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f86940c9050 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 --2- 172.21.15.33:0/3392320690 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86940c7070 0x7f86940c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 -- 172.21.15.33:0/3392320690 >> 172.21.15.33:0/3392320690 conn(0x7f869406c560 msgr2=0x7f869406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 -- 172.21.15.33:0/3392320690 shutdown_connections 2023-10-06T05:25:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.286+0000 7f869359e700 1 -- 172.21.15.33:0/3392320690 wait complete. 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.288+0000 7f869359e700 1 Processor -- start 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.288+0000 7f869359e700 1 -- start start 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 0x7f86940ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86941557f0 0x7f86941648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f8694145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694149900 con 0x7f86940c7070 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c3b80 con 0x7f8694155bc0 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694149620 con 0x7f86941557f0 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 0x7f86940ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 0x7f86940ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56258/0 (socket says 172.21.15.33:56258) 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f869259c700 1 -- 172.21.15.33:0/3055215324 learned_addr learned my addr 172.21.15.33:0/3055215324 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f8692d9d700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f8694145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.289+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86941557f0 0x7f86941648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869259c700 1 -- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86941557f0 msgr2=0x7f86941648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869259c700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86941557f0 0x7f86941648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869259c700 1 -- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 msgr2=0x7f8694145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869259c700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f8694145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869259c700 1 -- 172.21.15.33:0/3055215324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688009420 con 0x7f86940c7070 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869259c700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 0x7f86940ae130 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f868400c790 tx=0x7f8684007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f86827fc700 1 -- 172.21.15.33:0/3055215324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684010510 con 0x7f86940c7070 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f86827fc700 1 -- 172.21.15.33:0/3055215324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684010670 con 0x7f86940c7070 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f86827fc700 1 -- 172.21.15.33:0/3055215324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684010a10 con 0x7f86940c7070 2023-10-06T05:25:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c009ae0 con 0x7f86940c7070 2023-10-06T05:25:08.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.290+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940c4050 con 0x7f86940c7070 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.292+0000 7f86827fc700 1 -- 172.21.15.33:0/3055215324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f868401b3b0 con 0x7f86940c7070 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.293+0000 7f86827fc700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8674066f30 0x7f86740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.294+0000 7f86827fc700 1 -- 172.21.15.33:0/3055215324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8684083fd0 con 0x7f86940c7070 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.294+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8674066f30 0x7f86740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.295+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8674066f30 0x7f86740693e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8694148c90 tx=0x7f86880091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.295+0000 7f86a760eb80 1 Processor -- start 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.295+0000 7f86a760eb80 1 -- start start 2023-10-06T05:25:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.296+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 0x5608a52f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.296+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a5340fd0 0x5608a531fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.296+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52ee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.296+0000 7f86a760eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5272c80 con 0x5608a52f6d60 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.296+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5249a70 con 0x5608a5340fd0 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.296+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a5247270 con 0x5608a5320270 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 0x5608a52f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 0x5608a52f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56268/0 (socket says 172.21.15.33:56268) 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 -- 172.21.15.33:0/1003485745 learned_addr learned my addr 172.21.15.33:0/1003485745 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f8692d9d700 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f8691d9b700 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a5340fd0 0x5608a531fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 -- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 msgr2=0x5608a52ee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52ee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 -- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a5340fd0 msgr2=0x5608a531fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a5340fd0 0x5608a531fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 -- 172.21.15.33:0/1003485745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a50dd270 con 0x5608a52f6d60 2023-10-06T05:25:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f869259c700 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 0x5608a52f7130 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f868400a040 tx=0x7f8684082250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e16de7ce68ec2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f86737fe700 1 -- 172.21.15.33:0/1003485745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684057f50 con 0x5608a52f6d60 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f86737fe700 1 -- 172.21.15.33:0/1003485745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868405bca0 con 0x5608a52f6d60 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.297+0000 7f86737fe700 1 -- 172.21.15.33:0/1003485745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684010e00 con 0x5608a52f6d60 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 -- 172.21.15.33:0/1003485745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 msgr2=0x5608a52f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 0x5608a52f7130 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f868400a040 tx=0x7f8684082250 comp rx=0 tx=0).stop 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 -- 172.21.15.33:0/1003485745 shutdown_connections 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a52f6d60 0x5608a52f7130 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1003485745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a5340fd0 0x5608a531fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 -- 172.21.15.33:0/1003485745 >> 172.21.15.33:0/1003485745 conn(0x5608a52f8680 msgr2=0x5608a52f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 -- 172.21.15.33:0/1003485745 shutdown_connections 2023-10-06T05:25:08.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.298+0000 7f86a760eb80 1 -- 172.21.15.33:0/1003485745 wait complete. 2023-10-06T05:25:08.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.299+0000 7f86a760eb80 1 -- start start 2023-10-06T05:25:08.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.299+0000 7f86a760eb80 1 Processor -- start 2023-10-06T05:25:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a5337600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 0x5608a53384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5340fd0 0x5608a533a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5243060 con 0x5608a5320270 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a51ee890 con 0x5608a52f6d60 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a5209ea0 con 0x5608a5340fd0 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f86a760eb80 10 client.0 _ll_register_callbacks cb 0x5608a50dff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f8691d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 0x5608a53384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f8691d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 0x5608a53384a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56284/0 (socket says 172.21.15.33:56284) 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.300+0000 7f8691d9b700 1 -- 172.21.15.33:0/3644986349 learned_addr learned my addr 172.21.15.33:0/3644986349 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8691d9b700 1 -- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5340fd0 msgr2=0x5608a533a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5340fd0 0x5608a533a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f869259c700 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a5337600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8691d9b700 1 -- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 msgr2=0x5608a5337600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a5337600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8691d9b700 1 -- 172.21.15.33:0/3644986349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940c4050 con 0x5608a5320270 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8691d9b700 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 0x5608a53384a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8688014b70 tx=0x7f8688016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a8a9dbbdb079aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8670ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8670ff9700 1 -- 172.21.15.33:0/3644986349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688013a40 con 0x5608a5320270 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8670ff9700 1 -- 172.21.15.33:0/3644986349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8688013ba0 con 0x5608a5320270 2023-10-06T05:25:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a52e7690 con 0x5608a5320270 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.301+0000 7f8670ff9700 1 -- 172.21.15.33:0/3644986349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8688005a00 con 0x5608a5320270 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 10 client.8850 fetch_fsmap learned FSMap version 114 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x5608a507ffe0 con 0x5608a5320270 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f8670ff9700 1 -- 172.21.15.33:0/3644986349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8688030030 con 0x5608a5320270 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f8670ff9700 1 -- 172.21.15.33:0/3644986349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 114) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f868802b440 con 0x5608a5320270 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 10 client.8850 fetch_fsmap finished waiting for FSMap version 114 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 10 client.8850 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 20 client.8850 populate_metadata read hostname 'smithi033' 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a533ab50 0x5608a531e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 4 client.8850 mds_command: new command op to 8059 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.302+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 --> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608a52c45c0 con 0x5608a533ab50 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.303+0000 7f869259c700 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a533ab50 0x5608a531e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.303+0000 7f869259c700 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a533ab50 0x5608a531e700 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3130a7a79ddbe2ac server_cookie=91c240bf094ad544 in_seq=0 out_seq=0 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.303+0000 7f8670ff9700 10 client.8850 ms_handle_connect on v2:172.21.15.167:6834/582175774 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.304+0000 7f8670ff9700 1 -- 172.21.15.33:0/3644986349 <== mds.0 v2:172.21.15.167:6834/582175774 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5608a52c45c0 con 0x5608a533ab50 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.304+0000 7f8670ff9700 10 client.8850 handle_command_reply: tid=0 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.304+0000 7f86a760eb80 1 client.8850 shutdown 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 msgr2=0x5608a53384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 0x5608a53384a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8688014b70 tx=0x7f8688016000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 shutdown_connections 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5340fd0 0x5608a533a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a5337600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a533ab50 0x5608a531e700 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 --2- 172.21.15.33:0/3644986349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5320270 0x5608a53384a0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 >> 172.21.15.33:0/3644986349 conn(0x5608a52f8680 msgr2=0x5608a533c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f8670ff9700 0 client.8850 ms_handle_reset on v2:172.21.15.167:6834/582175774 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 shutdown_connections 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 1 -- 172.21.15.33:0/3644986349 wait complete. 2023-10-06T05:25:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.305+0000 7f86a760eb80 20 client.8850 trim_cache size 0 max 16384 2023-10-06T05:25:08.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.322+0000 7f86a760eb80 1 Processor -- start 2023-10-06T05:25:08.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.322+0000 7f86a760eb80 1 -- start start 2023-10-06T05:25:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.323+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a52f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.323+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.323+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a5326670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.323+0000 7f86a760eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a50c5e30 con 0x5608a5340fd0 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.323+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a533e6b0 con 0x5608a52f6d60 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.323+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a5334d30 con 0x5608a5320270 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a5326670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a5326670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56298/0 (socket says 172.21.15.33:56298) 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 -- 172.21.15.33:0/641159160 learned_addr learned my addr 172.21.15.33:0/641159160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8691d9b700 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f869259c700 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a52f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 -- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 msgr2=0x5608a52e9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52e9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 -- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 msgr2=0x5608a52f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a52f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 -- 172.21.15.33:0/641159160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a5211780 con 0x5608a5340fd0 2023-10-06T05:25:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8691d9b700 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.324+0000 7f8692d9d700 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a5326670 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f86940636f0 tx=0x7f868c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faf48abdadd750f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f8673fff700 1 -- 172.21.15.33:0/641159160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c024070 con 0x5608a5340fd0 2023-10-06T05:25:08.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f8673fff700 1 -- 172.21.15.33:0/641159160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c0092e0 con 0x5608a5340fd0 2023-10-06T05:25:08.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f8673fff700 1 -- 172.21.15.33:0/641159160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c029460 con 0x5608a5340fd0 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 -- 172.21.15.33:0/641159160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 msgr2=0x5608a5326670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a5326670 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f86940636f0 tx=0x7f868c018000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 -- 172.21.15.33:0/641159160 shutdown_connections 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a5326670 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a52f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 --2- 172.21.15.33:0/641159160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a52e9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 -- 172.21.15.33:0/641159160 >> 172.21.15.33:0/641159160 conn(0x5608a52f8680 msgr2=0x5608a5337600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 -- 172.21.15.33:0/641159160 shutdown_connections 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.325+0000 7f86a760eb80 1 -- 172.21.15.33:0/641159160 wait complete. 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.326+0000 7f86a760eb80 1 -- start start 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.326+0000 7f86a760eb80 1 Processor -- start 2023-10-06T05:25:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a539cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a534f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a534f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a53a4560 con 0x5608a5340fd0 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a52ef330 con 0x5608a52f6d60 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608a53383b0 con 0x5608a5320270 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f86a760eb80 10 client.0 _ll_register_callbacks cb 0x7f8688014440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f8692d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a534f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f8692d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a534f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56304/0 (socket says 172.21.15.33:56304) 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f8692d9d700 1 -- 172.21.15.33:0/1867556931 learned_addr learned my addr 172.21.15.33:0/1867556931 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f869259c700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a539cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.327+0000 7f8691d9b700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a534f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8692d9d700 1 -- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 msgr2=0x5608a534f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8692d9d700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a534f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8692d9d700 1 -- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 msgr2=0x5608a539cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8692d9d700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a539cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8692d9d700 1 -- 172.21.15.33:0/1867556931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940c77e0 con 0x5608a5340fd0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8692d9d700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a534f950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f868c020000 tx=0x7f868c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea212954a6226c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8670ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8670ff9700 1 -- 172.21.15.33:0/1867556931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c02f070 con 0x5608a5340fd0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f8670ff9700 1 -- 172.21.15.33:0/1867556931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f868c02ace0 con 0x5608a5340fd0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.328+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f868405bca0 con 0x5608a5340fd0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f8670ff9700 1 -- 172.21.15.33:0/1867556931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f868c02aea0 con 0x5608a5340fd0 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 10 client.8856 fetch_fsmap learned FSMap version 114 2023-10-06T05:25:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x5608a51f8470 con 0x5608a5340fd0 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f8670ff9700 1 -- 172.21.15.33:0/1867556931 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868405bca0 con 0x5608a5340fd0 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f8670ff9700 1 -- 172.21.15.33:0/1867556931 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 114) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f868c00b050 con 0x5608a5340fd0 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 10 client.8856 fetch_fsmap finished waiting for FSMap version 114 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 10 client.8856 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 20 client.8856 populate_metadata read hostname 'smithi033' 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a53a34a0 0x5608a53a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 4 client.8856 mds_command: new command op to 8059 tid=0[{"prefix": "status"}] 2023-10-06T05:25:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 --> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5608a51e5e00 con 0x5608a53a34a0 2023-10-06T05:25:08.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.329+0000 7f869259c700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a53a34a0 0x5608a53a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f869259c700 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a53a34a0 0x5608a53a3870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4017e1ccbfb17cf server_cookie=7a57ebfa5b2d2ba6 in_seq=0 out_seq=0 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f8670ff9700 10 client.8856 ms_handle_connect on v2:172.21.15.167:6834/582175774 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f8670ff9700 1 -- 172.21.15.33:0/1867556931 <== mds.0 v2:172.21.15.167:6834/582175774 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x5608a51e5e00 con 0x5608a53a34a0 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f8670ff9700 10 client.8856 handle_command_reply: tid=0 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 client.8856 shutdown 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 msgr2=0x5608a534f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a534f950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f868c020000 tx=0x7f868c024000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 shutdown_connections 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:6834/582175774,v1:172.21.15.167:6837/582175774] conn(0x5608a53a34a0 0x5608a53a3870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608a5340fd0 0x5608a534f950 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608a52f6d60 0x5608a539cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f8670ff9700 0 client.8856 ms_handle_reset on v2:172.21.15.167:6834/582175774 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 --2- 172.21.15.33:0/1867556931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5608a5320270 0x5608a534f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 >> 172.21.15.33:0/1867556931 conn(0x5608a52f8680 msgr2=0x5608a533c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.330+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 shutdown_connections 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f86a760eb80 1 -- 172.21.15.33:0/1867556931 wait complete. 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f86a760eb80 20 client.8856 trim_cache size 0 max 16384 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:25:08.338 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 8059, 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.530897811, 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 114, 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 156, 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 156, 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.69880071 2023-10-06T05:25:08.339 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:25:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8674066f30 msgr2=0x7f86740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8674066f30 0x7f86740693e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8694148c90 tx=0x7f86880091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 msgr2=0x7f86940ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 0x7f86940ae130 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f868400c790 tx=0x7f8684007e10 comp rx=0 tx=0).stop 2023-10-06T05:25:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 shutdown_connections 2023-10-06T05:25:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8674066f30 0x7f86740693e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8694155bc0 0x7f8694145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86940c7070 0x7f86940ae130 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 --2- 172.21.15.33:0/3055215324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86941557f0 0x7f86941648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 >> 172.21.15.33:0/3055215324 conn(0x7f869406c560 msgr2=0x7f86940c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 shutdown_connections 2023-10-06T05:25:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.331+0000 7f869359e700 1 -- 172.21.15.33:0/3055215324 wait complete. 2023-10-06T05:25:08.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.536+0000 7f97faf9d700 1 Processor -- start 2023-10-06T05:25:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.536+0000 7f97faf9d700 1 -- start start 2023-10-06T05:25:08.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97faf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97f406a0d0 0x7f97f40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97faf9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f4158ba0 con 0x7f97f4158700 2023-10-06T05:25:08.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97faf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f40ae490 con 0x7f97f4155140 2023-10-06T05:25:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97faf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f40c8e40 con 0x7f97f406a0d0 2023-10-06T05:25:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97f979a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97f4158700 0x7f97f40699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46702/0 (socket says 172.21.15.33:46702) 2023-10-06T05:25:08.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97f979a700 1 -- 172.21.15.33:0/3233381713 learned_addr learned my addr 172.21.15.33:0/3233381713 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97fa79c700 1 --2- 172.21.15.33:0/3233381713 >> v2:172.21.15.167:3300/0 conn(0x7f97f406a0d0 0x7f97f40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290973870 0 0) 0x7f97f4158ba0 con 0x7f97f4158700 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.538+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e0003570 con 0x7f97f4158700 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2183563482 0 0) 0x7f97e0003570 con 0x7f97f4158700 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 >> v2:172.21.15.167:3300/0 conn(0x7f97f406a0d0 msgr2=0x7f97f40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 --2- 172.21.15.33:0/3233381713 >> v2:172.21.15.167:3300/0 conn(0x7f97f406a0d0 0x7f97f40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 >> v1:172.21.15.167:6789/0 conn(0x7f97f4155140 legacy=0x7f97f40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f40c99e0 con 0x7f97f4158700 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f97e4001470 con 0x7f97f4158700 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 >> v1:172.21.15.33:6789/0 conn(0x7f97f4158700 legacy=0x7f97f40699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e0003a60 0x7f97e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e00051c0 0x7f97e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40c8e40 con 0x7f97e0004810 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00104e0 con 0x7f97e00051c0 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00106a0 con 0x7f97e0003a60 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f979a700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97f9f9b700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e0003a60 0x7f97e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.539+0000 7f97fa79c700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e00051c0 0x7f97e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f979a700 1 -- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e0003a60 msgr2=0x7f97e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f979a700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e0003a60 0x7f97e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f979a700 1 -- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e00051c0 msgr2=0x7f97e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f979a700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e00051c0 0x7f97e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f979a700 1 -- 172.21.15.33:0/3233381713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0003570 con 0x7f97e0004810 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97fa79c700 1 -- 172.21.15.33:0/3233381713 reap_dead start 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f979a700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97e0004c80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c3c0 tx=0x7f97e4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129342d9b9c735b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e400cc90 con 0x7f97e0004810 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e400cdf0 con 0x7f97e0004810 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.540+0000 7f97f8f99700 1 -- 172.21.15.33:0/3233381713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4005b50 con 0x7f97e0004810 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 -- 172.21.15.33:0/3233381713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 msgr2=0x7f97e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97e0004c80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c3c0 tx=0x7f97e4001e50 comp rx=0 tx=0).stop 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 -- 172.21.15.33:0/3233381713 shutdown_connections 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3233381713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97e0004c80 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 -- 172.21.15.33:0/3233381713 >> 172.21.15.33:0/3233381713 conn(0x7f97f406d030 msgr2=0x7f97f406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 -- 172.21.15.33:0/3233381713 shutdown_connections 2023-10-06T05:25:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.541+0000 7f97faf9d700 1 -- 172.21.15.33:0/3233381713 wait complete. 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.542+0000 7f97faf9d700 1 Processor -- start 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.542+0000 7f97faf9d700 1 -- start start 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97faf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97f4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97faf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f41528c0 0x7f97f4151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97faf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f4152d30 0x7f97f4151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97faf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4153f70 con 0x7f97e0004810 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97faf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4150880 con 0x7f97f4152d30 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97faf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41532f0 con 0x7f97f41528c0 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.543+0000 7f97f9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97f4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97f4150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56328/0 (socket says 172.21.15.33:56328) 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 -- 172.21.15.33:0/3301614339 learned_addr learned my addr 172.21.15.33:0/3301614339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 -- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f41528c0 msgr2=0x7f97f4151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97fa79c700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f4152d30 0x7f97f4151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f979a700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f41528c0 0x7f97f4151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f41528c0 0x7f97f4151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 -- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f4152d30 msgr2=0x7f97f4151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f4152d30 0x7f97f4151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 -- 172.21.15.33:0/3301614339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f41535d0 con 0x7f97e0004810 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97fa79c700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f4152d30 0x7f97f4151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97f9f9b700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97f4150340 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f97f0002950 tx=0x7f97f0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000f8d0 con 0x7f97e0004810 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000fa30 con 0x7f97e0004810 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f41538b0 con 0x7f97e0004810 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.544+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000fdd0 con 0x7f97e0004810 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.545+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f40c59e0 con 0x7f97e0004810 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.545+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f404c380 con 0x7f97e0004810 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.546+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97f0013600 con 0x7f97e0004810 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.547+0000 7f97e9ffb700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97d8066f30 0x7f97d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.547+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f97f0084bf0 con 0x7f97e0004810 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.548+0000 7f97f979a700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97d8066f30 0x7f97d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.549+0000 7f97f979a700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97d8066f30 0x7f97d80693e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f97f4168b80 tx=0x7f97e4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.551+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97f0051c60 con 0x7f97e0004810 2023-10-06T05:25:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.804+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f97f415af00 con 0x7f97e0004810 2023-10-06T05:25:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.804+0000 7f97e9ffb700 1 -- 172.21.15.33:0/3301614339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) v1 ==== 53+0+73 (secure 0 0 0) 0x7f97f0055280 con 0x7f97e0004810 2023-10-06T05:25:08.807 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:08.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97d8066f30 msgr2=0x7f97d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97d8066f30 0x7f97d80693e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f97f4168b80 tx=0x7f97e4015000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 msgr2=0x7f97f4150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97f4150340 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f97f0002950 tx=0x7f97f0007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 shutdown_connections 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97d8066f30 0x7f97d80693e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f4152d30 0x7f97f4151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97e0004810 0x7f97f4150340 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 --2- 172.21.15.33:0/3301614339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f41528c0 0x7f97f4151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 >> 172.21.15.33:0/3301614339 conn(0x7f97f406d030 msgr2=0x7f97f4150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 shutdown_connections 2023-10-06T05:25:08.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:08.807+0000 7f97faf9d700 1 -- 172.21.15.33:0/3301614339 wait complete. 2023-10-06T05:25:08.830 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:08.831 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:08.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:08.832 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:25:08.870 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41648sec preferred_lft 41648sec 2023-10-06T05:25:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:08.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:08.873 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:08.970 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:25:08.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:08.973 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:25:09.012 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:25:09.013 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:09.013 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:25:09.013 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:25:09.013 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:09.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:09.013 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:25:09.013 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:25:09.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:25:09.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:25:09.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:25:09.014 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:25:09.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:09.211 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:25:09.248 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:25:09.248 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:25:09.248 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:09.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:09.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:25:09.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:09.351 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:09.387 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:09.387 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:09.387 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:09.463 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:25:09.463 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:09.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:09.595 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:09.595 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:25:09.596 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:25:09.611 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:09.612 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:25:09.612 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:25:09.612 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:25:09.612 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:25:09.633 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:09.633 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:25:09.633 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:25:09.634 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:25:09.634 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:25:09.634 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:25:09.634 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:25:09.634 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:25:09.638 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:25:09.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:09.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:09.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:09.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.888+0000 7fc01f59e700 1 Processor -- start 2023-10-06T05:25:09.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.888+0000 7fc01f59e700 1 -- start start 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc02006e140 0x7fc02015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02006f220 con 0x7fc020159530 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02015ae50 con 0x7fc0200afdd0 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0200b1e00 con 0x7fc02006e140 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc020159530 0x7fc02006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43400/0 (socket says 172.21.15.33:43400) 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01e59c700 1 -- 172.21.15.33:0/508729090 learned_addr learned my addr 172.21.15.33:0/508729090 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:09.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.889+0000 7fc01dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc02006e140 0x7fc02015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01d59a700 1 -- 172.21.15.33:0/508729090 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891030494 0 0) 0x7fc02006f220 con 0x7fc020159530 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01d59a700 1 -- 172.21.15.33:0/508729090 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc008003570 con 0x7fc020159530 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01dd9b700 1 -- 172.21.15.33:0/508729090 >> v1:172.21.15.167:6789/0 conn(0x7fc0200afdd0 legacy=0x7fc0200b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01dd9b700 1 -- 172.21.15.33:0/508729090 >> v1:172.21.15.33:6789/0 conn(0x7fc020159530 legacy=0x7fc02006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01dd9b700 1 -- 172.21.15.33:0/508729090 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02006e610 con 0x7fc02006e140 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01dd9b700 1 --2- 172.21.15.33:0/508729090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc02006e140 0x7fc02015a910 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fc0140098e0 tx=0x7fc0140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8d3c8f06d213de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.890+0000 7fc01d59a700 1 -- 172.21.15.33:0/508729090 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400b640 con 0x7fc02006e140 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01d59a700 1 -- 172.21.15.33:0/508729090 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01400b7a0 con 0x7fc02006e140 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01d59a700 1 -- 172.21.15.33:0/508729090 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400bb40 con 0x7fc02006e140 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 -- 172.21.15.33:0/508729090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc02006e140 msgr2=0x7fc02015a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 --2- 172.21.15.33:0/508729090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc02006e140 0x7fc02015a910 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fc0140098e0 tx=0x7fc0140091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 -- 172.21.15.33:0/508729090 shutdown_connections 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 --2- 172.21.15.33:0/508729090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc02006e140 0x7fc02015a910 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 -- 172.21.15.33:0/508729090 >> 172.21.15.33:0/508729090 conn(0x7fc02006c560 msgr2=0x7fc02006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 -- 172.21.15.33:0/508729090 shutdown_connections 2023-10-06T05:25:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.891+0000 7fc01f59e700 1 -- 172.21.15.33:0/508729090 wait complete. 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.893+0000 7fc01f59e700 1 Processor -- start 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.893+0000 7fc01f59e700 1 -- start start 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02006e140 0x7fc0200aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 0x7fc0200cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020159530 0x7fc0200c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200cb820 con 0x7fc0200afdd0 2023-10-06T05:25:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200cbb00 con 0x7fc020159530 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200cbde0 con 0x7fc02006e140 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 0x7fc0200cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 0x7fc0200cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60430/0 (socket says 172.21.15.33:60430) 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 -- 172.21.15.33:0/293275120 learned_addr learned my addr 172.21.15.33:0/293275120 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01e59c700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02006e140 0x7fc0200aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01ed9d700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020159530 0x7fc0200c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 -- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02006e140 msgr2=0x7fc0200aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02006e140 0x7fc0200aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 -- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020159530 msgr2=0x7fc0200c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020159530 0x7fc0200c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.894+0000 7fc01dd9b700 1 -- 172.21.15.33:0/293275120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014009420 con 0x7fc0200afdd0 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.895+0000 7fc01dd9b700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 0x7fc0200cea00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc02006b870 tx=0x7fc014017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.895+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400bb20 con 0x7fc0200afdd0 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.895+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc014011420 con 0x7fc0200afdd0 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.895+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200cc0c0 con 0x7fc0200afdd0 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.895+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140117a0 con 0x7fc0200afdd0 2023-10-06T05:25:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.895+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0200cc8a0 con 0x7fc0200afdd0 2023-10-06T05:25:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.896+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0200cc560 con 0x7fc0200afdd0 2023-10-06T05:25:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.897+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc014011b50 con 0x7fc0200afdd0 2023-10-06T05:25:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.898+0000 7fc00e7fc700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc000066f30 0x7fc0000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.898+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc014085e30 con 0x7fc0200afdd0 2023-10-06T05:25:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.898+0000 7fc01e59c700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc000066f30 0x7fc0000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.899+0000 7fc01e59c700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc000066f30 0x7fc0000693e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc0100058d0 tx=0x7fc010009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:09.903+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc014052ea0 con 0x7fc0200afdd0 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.150+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc02004c380 con 0x7fc0200afdd0 2023-10-06T05:25:10.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.150+0000 7fc00e7fc700 1 -- 172.21.15.33:0/293275120 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc0140564c0 con 0x7fc0200afdd0 2023-10-06T05:25:10.153 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc000066f30 msgr2=0x7fc0000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc000066f30 0x7fc0000693e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc0100058d0 tx=0x7fc010009040 comp rx=0 tx=0).stop 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 msgr2=0x7fc0200cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 0x7fc0200cea00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc02006b870 tx=0x7fc014017000 comp rx=0 tx=0).stop 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 shutdown_connections 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc000066f30 0x7fc0000693e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020159530 0x7fc0200c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02006e140 0x7fc0200aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 --2- 172.21.15.33:0/293275120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200afdd0 0x7fc0200cea00 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 >> 172.21.15.33:0/293275120 conn(0x7fc02006c560 msgr2=0x7fc02006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 shutdown_connections 2023-10-06T05:25:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:10.152+0000 7fc01f59e700 1 -- 172.21.15.33:0/293275120 wait complete. 2023-10-06T05:25:10.172 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:25:10.172 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:10.173 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:10.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.173 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:25:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:10.216 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:10.217 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:10.217 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.217 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:10.217 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:10.217 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:10.218 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:25:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:10.218 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:10.218 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:25:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:10.219 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:10.219 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:10.219 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:25:10.219 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41651sec preferred_lft 41651sec 2023-10-06T05:25:10.219 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:25:10.220 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:10.220 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:10.220 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:10.220 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.221 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:25:10.221 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:10.221 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:25:10.221 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:25:10.221 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:10.222 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:25:10.222 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:25:10.222 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:10.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.223 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:10.319 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:25:10.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.323 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:25:10.362 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:25:10.362 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:10.362 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:25:10.362 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:25:10.362 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:10.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.363 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:25:10.363 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:25:10.363 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:25:10.363 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:25:10.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:25:10.364 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:25:10.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.559 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:25:10.595 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:25:10.595 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:25:10.595 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:10.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.596 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:25:10.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:10.698 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:10.734 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:10.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:10.735 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:10.809 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:25:10.810 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:10.810 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:25:10.939 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:25:10.940 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:25:10.940 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:25:10.956 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:25:10.956 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:25:10.956 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:25:10.956 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:25:10.956 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:25:10.976 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:25:10.976 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:25:10.976 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:25:10.977 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:25:10.977 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:25:10.977 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:25:10.977 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:25:10.977 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:25:10.980 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:25:10.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.981 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:11.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:25:11.258 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:25:11.258 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:25:11.258 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:25:11.463 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:25:11.463 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:25:11.463 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:25:11.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:11.464 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:11.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:25:11.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:11.545 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:25:11.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:11.582 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:25:11.654 INFO:teuthology.orchestra.run.smithi033.stdout:client7082 2023-10-06T05:25:11.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.839+0000 7f97d6f9d700 1 Processor -- start 2023-10-06T05:25:11.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.839+0000 7f97d6f9d700 1 -- start start 2023-10-06T05:25:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.840+0000 7f97d6f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d013faa0 0x7f97d013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d6f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97d00c7940 0x7f97d00c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d6f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d00c9cb0 con 0x7f97d006aad0 2023-10-06T05:25:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d6f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d006b660 con 0x7f97d00c7940 2023-10-06T05:25:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d6f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00ca270 con 0x7f97d013faa0 2023-10-06T05:25:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d579a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97d00c7940 0x7f97d00c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d679c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97d006aad0 0x7f97d006af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43414/0 (socket says 172.21.15.33:43414) 2023-10-06T05:25:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d679c700 1 -- 172.21.15.33:0/1205217842 learned_addr learned my addr 172.21.15.33:0/1205217842 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d5f9b700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d013faa0 0x7f97d013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d579a700 1 -- 172.21.15.33:0/1205217842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d013faa0 msgr2=0x7f97d013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d579a700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d013faa0 0x7f97d013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d579a700 1 -- 172.21.15.33:0/1205217842 >> v1:172.21.15.33:6789/0 conn(0x7f97d006aad0 legacy=0x7f97d006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d579a700 1 -- 172.21.15.33:0/1205217842 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d006b840 con 0x7f97d00c7940 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d5f9b700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d013faa0 0x7f97d013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.841+0000 7f97d4f99700 1 -- 172.21.15.33:0/1205217842 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861265966 0 0) 0x7f97d00c9cb0 con 0x7f97d006aad0 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d579a700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7940 0x7f97d00c7db0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f97c00098e0 tx=0x7f97c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d06f470532d904 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d4f99700 1 -- 172.21.15.33:0/1205217842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0009e80 con 0x7f97d00c7940 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d4f99700 1 -- 172.21.15.33:0/1205217842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c000b860 con 0x7f97d00c7940 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d4f99700 1 -- 172.21.15.33:0/1205217842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c000bbe0 con 0x7f97d00c7940 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d6f9d700 1 -- 172.21.15.33:0/1205217842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7940 msgr2=0x7f97d00c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7940 0x7f97d00c7db0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f97c00098e0 tx=0x7f97c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d6f9d700 1 -- 172.21.15.33:0/1205217842 shutdown_connections 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d013faa0 0x7f97d013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/1205217842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7940 0x7f97d00c7db0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.842+0000 7f97d6f9d700 1 -- 172.21.15.33:0/1205217842 >> 172.21.15.33:0/1205217842 conn(0x7f97d006d030 msgr2=0x7f97d006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.843+0000 7f97d6f9d700 1 -- 172.21.15.33:0/1205217842 shutdown_connections 2023-10-06T05:25:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.843+0000 7f97d6f9d700 1 -- 172.21.15.33:0/1205217842 wait complete. 2023-10-06T05:25:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.844+0000 7f97d6f9d700 1 Processor -- start 2023-10-06T05:25:11.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.844+0000 7f97d6f9d700 1 -- start start 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d6f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 0x7f97d013b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d6f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d00c7940 0x7f97d016cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d6f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d013faa0 0x7f97d016d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d6f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d013c8b0 con 0x7f97d006aad0 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d6f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d013bcc0 con 0x7f97d013faa0 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d6f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d013bf10 con 0x7f97d00c7940 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d5f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 0x7f97d013b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d5f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 0x7f97d013b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60452/0 (socket says 172.21.15.33:60452) 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.845+0000 7f97d5f9b700 1 -- 172.21.15.33:0/2188639647 learned_addr learned my addr 172.21.15.33:0/2188639647 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:11.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d579a700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d00c7940 0x7f97d016cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d5f9b700 1 -- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d00c7940 msgr2=0x7f97d016cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d5f9b700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d00c7940 0x7f97d016cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d5f9b700 1 -- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d013faa0 msgr2=0x7f97d016d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d5f9b700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d013faa0 0x7f97d016d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d5f9b700 1 -- 172.21.15.33:0/2188639647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0009420 con 0x7f97d006aad0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d5f9b700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 0x7f97d013b330 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c850 tx=0x7f97cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc011600 con 0x7f97d006aad0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc011760 con 0x7f97d006aad0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d013c1f0 con 0x7f97d006aad0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc011b00 con 0x7f97d006aad0 2023-10-06T05:25:11.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.846+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d013b870 con 0x7f97d006aad0 2023-10-06T05:25:11.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.847+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d004c380 con 0x7f97d006aad0 2023-10-06T05:25:11.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.848+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97cc0073b0 con 0x7f97d006aad0 2023-10-06T05:25:11.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.849+0000 7f97c5ffb700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97b4066f30 0x7f97b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.849+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f97cc084730 con 0x7f97d006aad0 2023-10-06T05:25:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.850+0000 7f97d579a700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97b4066f30 0x7f97b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.850+0000 7f97d579a700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97b4066f30 0x7f97b40693e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f97d00c2ed0 tx=0x7f97c0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:11.853+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97cc055230 con 0x7f97d006aad0 2023-10-06T05:25:12.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.105+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97d013cb20 con 0x7f97d006aad0 2023-10-06T05:25:12.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.105+0000 7f97c5ffb700 1 -- 172.21.15.33:0/2188639647 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f97cc054dc0 con 0x7f97d006aad0 2023-10-06T05:25:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:12.108 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97b4066f30 msgr2=0x7f97b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97b4066f30 0x7f97b40693e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f97d00c2ed0 tx=0x7f97c0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 msgr2=0x7f97d013b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 0x7f97d013b330 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c850 tx=0x7f97cc008040 comp rx=0 tx=0).stop 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 shutdown_connections 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f97b4066f30 0x7f97b40693e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d013faa0 0x7f97d016d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d006aad0 0x7f97d013b330 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 --2- 172.21.15.33:0/2188639647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97d00c7940 0x7f97d016cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 >> 172.21.15.33:0/2188639647 conn(0x7f97d006d030 msgr2=0x7f97d006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 shutdown_connections 2023-10-06T05:25:12.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:12.108+0000 7f97d6f9d700 1 -- 172.21.15.33:0/2188639647 wait complete. 2023-10-06T05:25:12.112 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:12.128 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session evict 7082 2023-10-06T05:25:13.078 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:25:13.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.273+0000 7f1f64d9d700 1 Processor -- start 2023-10-06T05:25:13.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.273+0000 7f1f64d9d700 1 -- start start 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.274+0000 7f1f64d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f600ccfc0 0x7f1f600cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.274+0000 7f1f64d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cf3b0 0x7f1f601594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f64d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f600cc750 0x7f1f601599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f64d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600cbd80 con 0x7f1f600ccfc0 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f64d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600af5f0 con 0x7f1f600cc750 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f64d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600af8d0 con 0x7f1f600cf3b0 2023-10-06T05:25:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f600ccfc0 0x7f1f600cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f600ccfc0 0x7f1f600cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60468/0 (socket says 172.21.15.33:60468) 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 -- 172.21.15.33:0/1778635156 learned_addr learned my addr 172.21.15.33:0/1778635156 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5ffff700 1 --2- 172.21.15.33:0/1778635156 >> v2:172.21.15.167:3300/0 conn(0x7f1f600cc750 0x7f1f601599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5effd700 1 --2- 172.21.15.33:0/1778635156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cf3b0 0x7f1f601594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 -- 172.21.15.33:0/1778635156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cf3b0 msgr2=0x7f1f601594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 --2- 172.21.15.33:0/1778635156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cf3b0 0x7f1f601594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 -- 172.21.15.33:0/1778635156 >> v2:172.21.15.167:3300/0 conn(0x7f1f600cc750 msgr2=0x7f1f601599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 --2- 172.21.15.33:0/1778635156 >> v2:172.21.15.167:3300/0 conn(0x7f1f600cc750 0x7f1f601599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.275+0000 7f1f5f7fe700 1 -- 172.21.15.33:0/1778635156 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600afbb0 con 0x7f1f600ccfc0 2023-10-06T05:25:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f5f7fe700 1 --2- 172.21.15.33:0/1778635156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600ccfc0 0x7f1f600cc210 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cdb60 tx=0x7f1f50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bb43ad19c4caae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f5e7fc700 1 -- 172.21.15.33:0/1778635156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000b860 con 0x7f1f600ccfc0 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f5e7fc700 1 -- 172.21.15.33:0/1778635156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000b9c0 con 0x7f1f600ccfc0 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f5e7fc700 1 -- 172.21.15.33:0/1778635156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000bd60 con 0x7f1f600ccfc0 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f64d9d700 1 -- 172.21.15.33:0/1778635156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600ccfc0 msgr2=0x7f1f600cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/1778635156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600ccfc0 0x7f1f600cc210 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cdb60 tx=0x7f1f50009250 comp rx=0 tx=0).stop 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f64d9d700 1 -- 172.21.15.33:0/1778635156 shutdown_connections 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.276+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/1778635156 >> v2:172.21.15.167:3300/0 conn(0x7f1f600cc750 0x7f1f601599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.277+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/1778635156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600ccfc0 0x7f1f600cc210 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.277+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/1778635156 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cf3b0 0x7f1f601594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.277+0000 7f1f64d9d700 1 -- 172.21.15.33:0/1778635156 >> 172.21.15.33:0/1778635156 conn(0x7f1f6006cff0 msgr2=0x7f1f6006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.277+0000 7f1f64d9d700 1 -- 172.21.15.33:0/1778635156 shutdown_connections 2023-10-06T05:25:13.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.277+0000 7f1f64d9d700 1 -- 172.21.15.33:0/1778635156 wait complete. 2023-10-06T05:25:13.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.278+0000 7f1f64d9d700 1 Processor -- start 2023-10-06T05:25:13.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.278+0000 7f1f64d9d700 1 -- start start 2023-10-06T05:25:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f64d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cc750 0x7f1f601482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f64d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f600ccfc0 0x7f1f600c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f64d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 0x7f1f600c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f64d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600c9b70 con 0x7f1f600c7af0 2023-10-06T05:25:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f64d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f601542b0 con 0x7f1f600ccfc0 2023-10-06T05:25:13.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f64d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600c8050 con 0x7f1f600cc750 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 0x7f1f600c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 0x7f1f600c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60478/0 (socket says 172.21.15.33:60478) 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.279+0000 7f1f5ffff700 1 -- 172.21.15.33:0/3972795834 learned_addr learned my addr 172.21.15.33:0/3972795834 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5effd700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f600ccfc0 0x7f1f600c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5f7fe700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cc750 0x7f1f601482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5ffff700 1 -- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cc750 msgr2=0x7f1f601482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5ffff700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cc750 0x7f1f601482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5ffff700 1 -- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f600ccfc0 msgr2=0x7f1f600c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5ffff700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f600ccfc0 0x7f1f600c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5ffff700 1 -- 172.21.15.33:0/3972795834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500094f0 con 0x7f1f600c7af0 2023-10-06T05:25:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f5ffff700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 0x7f1f600c5b10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f1f58007f30 tx=0x7f1f580098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800ecf0 con 0x7f1f600c7af0 2023-10-06T05:25:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5800ee50 con 0x7f1f600c7af0 2023-10-06T05:25:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f600c9520 con 0x7f1f600c7af0 2023-10-06T05:25:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58014670 con 0x7f1f600c7af0 2023-10-06T05:25:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.280+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f600c8330 con 0x7f1f600c7af0 2023-10-06T05:25:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.282+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f600c99c0 con 0x7f1f600c7af0 2023-10-06T05:25:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.282+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f580149e0 con 0x7f1f600c7af0 2023-10-06T05:25:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.283+0000 7f1f477fe700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f4c066f30 0x7f1f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.283+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f58084e50 con 0x7f1f600c7af0 2023-10-06T05:25:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.284+0000 7f1f5f7fe700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f4c066f30 0x7f1f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.284+0000 7f1f5f7fe700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f4c066f30 0x7f1f4c0693e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cdb60 tx=0x7f1f50017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.287+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f580559c0 con 0x7f1f600c7af0 2023-10-06T05:25:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.536+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f6004c380 con 0x7f1f600c7af0 2023-10-06T05:25:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.537+0000 7f1f477fe700 1 -- 172.21.15.33:0/3972795834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1f58009a20 con 0x7f1f600c7af0 2023-10-06T05:25:13.539 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:13.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f4c066f30 msgr2=0x7f1f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f4c066f30 0x7f1f4c0693e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cdb60 tx=0x7f1f50017000 comp rx=0 tx=0).stop 2023-10-06T05:25:13.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 msgr2=0x7f1f600c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 0x7f1f600c5b10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f1f58007f30 tx=0x7f1f580098e0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 shutdown_connections 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f4c066f30 0x7f1f4c0693e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f600c7af0 0x7f1f600c5b10 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f600cc750 0x7f1f601482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 --2- 172.21.15.33:0/3972795834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f600ccfc0 0x7f1f600c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 >> 172.21.15.33:0/3972795834 conn(0x7f1f6006cff0 msgr2=0x7f1f6006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 shutdown_connections 2023-10-06T05:25:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:13.539+0000 7f1f64d9d700 1 -- 172.21.15.33:0/3972795834 wait complete. 2023-10-06T05:25:13.559 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:25:13.559 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:13.559 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:25:13.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:25:13.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:13.560 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:25:13.560 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:25:13.560 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:13.560 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:13.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:13.560 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:25:13.597 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:25:13.598 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41644sec preferred_lft 41644sec 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:25:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:13.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:13.601 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:13.697 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:25:13.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:13.700 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:25:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:25:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:25:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:25:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:13.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:13.740 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:25:13.740 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:25:13.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:25:13.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:25:13.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:25:13.740 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:25:13.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:13.947 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:25:13.984 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:25:13.985 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:25:13.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:13.985 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-10-06T05:25:14.061 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 1 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.2) 2023-10-06T05:25:14.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.063 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:25:14.063 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:25:14.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:25:14.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 2 2023-10-06T05:25:14.063 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:25:14.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr 2023-10-06T05:25:14.294 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether a6:1c:13:80:50:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.2/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::a41c:13ff:fe80:50a2/64 scope link 2023-10-06T05:25:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:25:14.359 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:e5:0e:f4:e3:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.360 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c8e5:eff:fef4:e3fd/64 scope link 2023-10-06T05:25:14.361 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr 2023-10-06T05:25:14.424 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether a6:1c:13:80:50:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.2/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:25:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.426 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::a41c:13ff:fe80:50a2/64 scope link 2023-10-06T05:25:14.426 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr 2023-10-06T05:25:14.490 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:14.490 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:14.490 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:14.490 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether a6:1c:13:80:50:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.2/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::a41c:13ff:fe80:50a2/64 scope link 2023-10-06T05:25:14.491 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.556 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:14.557 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:e5:0e:f4:e3:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:25:14.557 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:25:14.557 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.557 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c8e5:eff:fef4:e3fd/64 scope link 2023-10-06T05:25:14.557 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:14.559 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.3/16 2023-10-06T05:25:14.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.559 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:25:14.559 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:25:14.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.2 2023-10-06T05:25:14.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.3/16 brd 192.168.255.255 dev veth0 2023-10-06T05:25:14.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T05:25:14.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T05:25:14.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T05:25:14.560 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:25:14.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:14.795 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:25:14.795 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:25:14.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.2 up 2023-10-06T05:25:14.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.2 master ceph-brx 2023-10-06T05:25:14.795 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:25:14.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:14.955 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:14.992 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:14.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:14.992 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:15.068 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:25:15.069 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:15.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:15.201 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:15.202 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:25:15.202 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:25:15.217 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:15.218 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:25:15.218 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:25:15.218 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:25:15.218 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:25:15.239 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:25:15.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:25:15.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:15.243 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:15.305 INFO:tasks.cephfs.mount:Creating file a 2023-10-06T05:25:15.306 DEBUG:teuthology.orchestra.run.smithi033:> touch /home/ubuntu/cephtest/mnt.1/a 2023-10-06T05:25:15.345 INFO:tasks.cephfs.mount:Creating file b 2023-10-06T05:25:15.346 DEBUG:teuthology.orchestra.run.smithi033:> touch /home/ubuntu/cephtest/mnt.1/b 2023-10-06T05:25:15.423 INFO:tasks.cephfs.mount:Creating file c 2023-10-06T05:25:15.423 DEBUG:teuthology.orchestra.run.smithi033:> touch /home/ubuntu/cephtest/mnt.1/c 2023-10-06T05:25:15.501 INFO:tasks.cephfs.mount:Checking file a 2023-10-06T05:25:15.502 DEBUG:teuthology.orchestra.run.smithi033:> ls /home/ubuntu/cephtest/mnt.1/a 2023-10-06T05:25:15.578 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.1/a 2023-10-06T05:25:15.578 INFO:tasks.cephfs.mount:Checking file b 2023-10-06T05:25:15.578 DEBUG:teuthology.orchestra.run.smithi033:> ls /home/ubuntu/cephtest/mnt.1/b 2023-10-06T05:25:15.655 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.1/b 2023-10-06T05:25:15.656 INFO:tasks.cephfs.mount:Checking file c 2023-10-06T05:25:15.656 DEBUG:teuthology.orchestra.run.smithi033:> ls /home/ubuntu/cephtest/mnt.1/c 2023-10-06T05:25:15.732 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.1/c 2023-10-06T05:25:15.733 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:25:15.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:15.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:15.873 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:25:15.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:15.873 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:25:15.916 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:25:15.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:16.036 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:16.124 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:25:16.124 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:25:16.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:16.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:19.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:19.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:49.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:49.671 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:52.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:25:52.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:52.287 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:52.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.558+0000 7fc7df59e700 1 Processor -- start 2023-10-06T05:25:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.558+0000 7fc7df59e700 1 -- start start 2023-10-06T05:25:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.559+0000 7fc7df59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e0155af0 0x7fc7e006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.559+0000 7fc7df59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7e006a6a0 0x7fc7e00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.559+0000 7fc7df59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e006ab60 con 0x7fc7e00c8b50 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.559+0000 7fc7df59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e006b7e0 con 0x7fc7e006a6a0 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.559+0000 7fc7df59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e006f070 con 0x7fc7e0155af0 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ded9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc7e00c8b50 0x7fc7e006e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60190/0 (socket says 172.21.15.33:60190) 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ded9d700 1 -- 172.21.15.33:0/3367352482 learned_addr learned my addr 172.21.15.33:0/3367352482 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ddd9b700 1 --2- 172.21.15.33:0/3367352482 >> v2:172.21.15.167:3300/0 conn(0x7fc7e006a6a0 0x7fc7e00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7de59c700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e0155af0 0x7fc7e006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7dd59a700 1 -- 172.21.15.33:0/3367352482 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605044716 0 0) 0x7fc7e006ab60 con 0x7fc7e00c8b50 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7dd59a700 1 -- 172.21.15.33:0/3367352482 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c8003570 con 0x7fc7e00c8b50 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ddd9b700 1 -- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e0155af0 msgr2=0x7fc7e006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ddd9b700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e0155af0 0x7fc7e006a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ddd9b700 1 -- 172.21.15.33:0/3367352482 >> v1:172.21.15.33:6789/0 conn(0x7fc7e00c8b50 legacy=0x7fc7e006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7ddd9b700 1 -- 172.21.15.33:0/3367352482 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e006ae40 con 0x7fc7e006a6a0 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.560+0000 7fc7de59c700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e0155af0 0x7fc7e006a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7ddd9b700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e00c7210 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40098e0 tx=0x7fc7d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d3ad9e010b6515 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7dd59a700 1 -- 172.21.15.33:0/3367352482 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4011630 con 0x7fc7e006a6a0 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7dd59a700 1 -- 172.21.15.33:0/3367352482 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d4011790 con 0x7fc7e006a6a0 2023-10-06T05:25:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7dd59a700 1 -- 172.21.15.33:0/3367352482 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4011b30 con 0x7fc7e006a6a0 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7df59e700 1 -- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 msgr2=0x7fc7e00c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7df59e700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e00c7210 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40098e0 tx=0x7fc7d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7df59e700 1 -- 172.21.15.33:0/3367352482 shutdown_connections 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7df59e700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e0155af0 0x7fc7e006a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7df59e700 1 --2- 172.21.15.33:0/3367352482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e00c7210 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.561+0000 7fc7df59e700 1 -- 172.21.15.33:0/3367352482 >> 172.21.15.33:0/3367352482 conn(0x7fc7e006d030 msgr2=0x7fc7e006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.562+0000 7fc7df59e700 1 -- 172.21.15.33:0/3367352482 shutdown_connections 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.562+0000 7fc7df59e700 1 -- 172.21.15.33:0/3367352482 wait complete. 2023-10-06T05:25:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.563+0000 7fc7df59e700 1 Processor -- start 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.563+0000 7fc7df59e700 1 -- start start 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7df59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e0169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7df59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e00c8b50 0x7fc7e016c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7df59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 0x7fc7e00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7df59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0169d80 con 0x7fc7e0155af0 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7df59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e00b0960 con 0x7fc7e006a6a0 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7df59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e00ace10 con 0x7fc7e00c8b50 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7ded9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 0x7fc7e00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7ded9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 0x7fc7e00a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42138/0 (socket says 172.21.15.33:42138) 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7ded9d700 1 -- 172.21.15.33:0/2230357103 learned_addr learned my addr 172.21.15.33:0/2230357103 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7ddd9b700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e00c8b50 0x7fc7e016c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.564+0000 7fc7de59c700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e0169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ded9d700 1 -- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e00c8b50 msgr2=0x7fc7e016c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ded9d700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e00c8b50 0x7fc7e016c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ded9d700 1 -- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 msgr2=0x7fc7e0169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ded9d700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e0169840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ded9d700 1 -- 172.21.15.33:0/2230357103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d0009420 con 0x7fc7e0155af0 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ded9d700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 0x7fc7e00a9890 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800cc40 tx=0x7fc7d8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800a8b0 con 0x7fc7e0155af0 2023-10-06T05:25:52.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d800aa10 con 0x7fc7e0155af0 2023-10-06T05:25:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800ad90 con 0x7fc7e0155af0 2023-10-06T05:25:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.565+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d4009420 con 0x7fc7e0155af0 2023-10-06T05:25:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.566+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e00ad0b0 con 0x7fc7e0155af0 2023-10-06T05:25:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.567+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e00cba80 con 0x7fc7e0155af0 2023-10-06T05:25:52.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.568+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7d8007300 con 0x7fc7e0155af0 2023-10-06T05:25:52.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.569+0000 7fc7ce7fc700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7c0067000 0x7fc7c00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.569+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7d8084960 con 0x7fc7e0155af0 2023-10-06T05:25:52.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.570+0000 7fc7de59c700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7c0067000 0x7fc7c00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.570+0000 7fc7de59c700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7c0067000 0x7fc7c00694b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00b0c90 tx=0x7fc7d00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.574+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7d8051aa0 con 0x7fc7e0155af0 2023-10-06T05:25:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.820+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7e004c3e0 con 0x7fc7e0155af0 2023-10-06T05:25:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.821+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/2230357103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fc7d80550c0 con 0x7fc7e0155af0 2023-10-06T05:25:52.823 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:52.823 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7c0067000 msgr2=0x7fc7c00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7c0067000 0x7fc7c00694b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00b0c90 tx=0x7fc7d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 msgr2=0x7fc7e00a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 0x7fc7e00a9890 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800cc40 tx=0x7fc7d8001a00 comp rx=0 tx=0).stop 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 shutdown_connections 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7c0067000 0x7fc7c00694b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7e0155af0 0x7fc7e00a9890 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e006a6a0 0x7fc7e0169840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 --2- 172.21.15.33:0/2230357103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7e00c8b50 0x7fc7e016c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 >> 172.21.15.33:0/2230357103 conn(0x7fc7e006d030 msgr2=0x7fc7e00c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 shutdown_connections 2023-10-06T05:25:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:52.823+0000 7fc7df59e700 1 -- 172.21.15.33:0/2230357103 wait complete. 2023-10-06T05:25:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:52.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.028+0000 7f0c05b9d700 1 Processor -- start 2023-10-06T05:25:53.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.028+0000 7f0c05b9d700 1 -- start start 2023-10-06T05:25:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c05b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c0013faa0 0x7f0c0013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c05b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c000c9cb0 con 0x7f0c000c7940 2023-10-06T05:25:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c05b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c000ca270 con 0x7f0c0006aad0 2023-10-06T05:25:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c05b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0006b640 con 0x7f0c0013faa0 2023-10-06T05:25:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0bfffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c000c7940 0x7f0c000c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60196/0 (socket says 172.21.15.33:60196) 2023-10-06T05:25:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c04b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c0013faa0 0x7f0c0013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0bfffff700 1 -- 172.21.15.33:0/381822696 learned_addr learned my addr 172.21.15.33:0/381822696 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0bff7fe700 1 -- 172.21.15.33:0/381822696 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969615008 0 0) 0x7f0c000c9cb0 con 0x7f0c000c7940 2023-10-06T05:25:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c04b9b700 1 -- 172.21.15.33:0/381822696 >> v1:172.21.15.167:6789/0 conn(0x7f0c0006aad0 legacy=0x7f0c0006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c04b9b700 1 -- 172.21.15.33:0/381822696 >> v1:172.21.15.33:6789/0 conn(0x7f0c000c7940 legacy=0x7f0c000c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.030+0000 7f0c04b9b700 1 -- 172.21.15.33:0/381822696 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0006b840 con 0x7f0c0013faa0 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.031+0000 7f0c04b9b700 1 --2- 172.21.15.33:0/381822696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c0013fe70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40098e0 tx=0x7f0bf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69b20950a866228 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.031+0000 7f0bff7fe700 1 -- 172.21.15.33:0/381822696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4009e80 con 0x7f0c0013faa0 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.031+0000 7f0bff7fe700 1 -- 172.21.15.33:0/381822696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400b860 con 0x7f0c0013faa0 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.031+0000 7f0bff7fe700 1 -- 172.21.15.33:0/381822696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400bbe0 con 0x7f0c0013faa0 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.031+0000 7f0c05b9d700 1 -- 172.21.15.33:0/381822696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 msgr2=0x7f0c0013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.031+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/381822696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c0013fe70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40098e0 tx=0x7f0bf40091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.032+0000 7f0c05b9d700 1 -- 172.21.15.33:0/381822696 shutdown_connections 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.032+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/381822696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c0013fe70 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.032+0000 7f0c05b9d700 1 -- 172.21.15.33:0/381822696 >> 172.21.15.33:0/381822696 conn(0x7f0c0006d030 msgr2=0x7f0c0006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.032+0000 7f0c05b9d700 1 -- 172.21.15.33:0/381822696 shutdown_connections 2023-10-06T05:25:53.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.032+0000 7f0c05b9d700 1 -- 172.21.15.33:0/381822696 wait complete. 2023-10-06T05:25:53.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.033+0000 7f0c05b9d700 1 Processor -- start 2023-10-06T05:25:53.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.033+0000 7f0c05b9d700 1 -- start start 2023-10-06T05:25:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c05b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0006aad0 0x7f0c0012f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c05b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c000c7940 0x7f0c000ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c05b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c000cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c05b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0013e4b0 con 0x7f0c0013faa0 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c05b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000c3f10 con 0x7f0c0006aad0 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c05b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000c41f0 con 0x7f0c000c7940 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.034+0000 7f0c0539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c000cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c000cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42154/0 (socket says 172.21.15.33:42154) 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 -- 172.21.15.33:0/4076732269 learned_addr learned my addr 172.21.15.33:0/4076732269 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c04b9b700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0006aad0 0x7f0c0012f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0bfffff700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c000c7940 0x7f0c000ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 -- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c000c7940 msgr2=0x7f0c000ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c000c7940 0x7f0c000ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 -- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0006aad0 msgr2=0x7f0c0012f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0006aad0 0x7f0c0012f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 -- 172.21.15.33:0/4076732269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4009420 con 0x7f0c0013faa0 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c0539c700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c000cc020 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c8e0 tx=0x7f0bf8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800ce80 con 0x7f0c0013faa0 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf800a7e0 con 0x7f0c0013faa0 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800ab80 con 0x7f0c0013faa0 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.035+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c000c44d0 con 0x7f0c0013faa0 2023-10-06T05:25:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.036+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c000c4a90 con 0x7f0c0013faa0 2023-10-06T05:25:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.037+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c000c4cb0 con 0x7f0c0013faa0 2023-10-06T05:25:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.038+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0bf8012420 con 0x7f0c0013faa0 2023-10-06T05:25:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.038+0000 7f0bfcff9700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0be4066f30 0x7f0be40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.039+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bf80849b0 con 0x7f0c0013faa0 2023-10-06T05:25:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.039+0000 7f0c04b9b700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0be4066f30 0x7f0be40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.040+0000 7f0c04b9b700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0be4066f30 0x7f0be40693e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40093f0 tx=0x7f0bf4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.043+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bf8051a90 con 0x7f0c0013faa0 2023-10-06T05:25:53.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.245+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c0004c380 con 0x7f0c0013faa0 2023-10-06T05:25:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.246+0000 7f0bfcff9700 1 -- 172.21.15.33:0/4076732269 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f0bf80550b0 con 0x7f0c0013faa0 2023-10-06T05:25:53.248 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:53.249 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":157,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:25:13.046893+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:59.024220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:00.278297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1400837691":"2023-10-06T06:25:12.317114+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0be4066f30 msgr2=0x7f0be40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0be4066f30 0x7f0be40693e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40093f0 tx=0x7f0bf4017000 comp rx=0 tx=0).stop 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 msgr2=0x7f0c000cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c000cc020 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c8e0 tx=0x7f0bf8001710 comp rx=0 tx=0).stop 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 shutdown_connections 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0be4066f30 0x7f0be40693e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0013faa0 0x7f0c000cc020 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0006aad0 0x7f0c0012f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 --2- 172.21.15.33:0/4076732269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c000c7940 0x7f0c000ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 >> 172.21.15.33:0/4076732269 conn(0x7f0c0006d030 msgr2=0x7f0c000c1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 shutdown_connections 2023-10-06T05:25:53.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.249+0000 7f0c05b9d700 1 -- 172.21.15.33:0/4076732269 wait complete. 2023-10-06T05:25:53.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:24:59.024220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:25:00.278297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:25:53.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:25:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.457+0000 7fc25719d700 1 Processor -- start 2023-10-06T05:25:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.457+0000 7fc25719d700 1 -- start start 2023-10-06T05:25:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc250155a50 0x7fc250155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2500afb20 0x7fc2500caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500b06b0 con 0x7fc2500c9490 2023-10-06T05:25:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500c99e0 con 0x7fc250155a50 2023-10-06T05:25:53.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500c9cc0 con 0x7fc2500afb20 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25619b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc250155a50 0x7fc250155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25699c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2500afb20 0x7fc2500caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25619b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc250155a50 0x7fc250155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:57102/0 (socket says 172.21.15.33:57102) 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.459+0000 7fc25619b700 1 -- 172.21.15.33:0/403040004 learned_addr learned my addr 172.21.15.33:0/403040004 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc25619b700 1 -- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2500afb20 msgr2=0x7fc2500caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc25619b700 1 --2- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2500afb20 0x7fc2500caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc25619b700 1 -- 172.21.15.33:0/403040004 >> v1:172.21.15.167:6789/0 conn(0x7fc2500c9490 legacy=0x7fc2500af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc25619b700 1 -- 172.21.15.33:0/403040004 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500c9fa0 con 0x7fc250155a50 2023-10-06T05:25:53.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc25619b700 1 --2- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250155a50 0x7fc250155e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00ab50 tx=0x7fc24c00a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=386e145ea43f3da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc255199700 1 -- 172.21.15.33:0/403040004 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00c970 con 0x7fc250155a50 2023-10-06T05:25:53.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc255199700 1 -- 172.21.15.33:0/403040004 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c00cad0 con 0x7fc250155a50 2023-10-06T05:25:53.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.460+0000 7fc255199700 1 -- 172.21.15.33:0/403040004 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00ce70 con 0x7fc250155a50 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 -- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250155a50 msgr2=0x7fc250155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 --2- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250155a50 0x7fc250155e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00ab50 tx=0x7fc24c00a3f0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 -- 172.21.15.33:0/403040004 shutdown_connections 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 --2- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250155a50 0x7fc250155e20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 --2- 172.21.15.33:0/403040004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2500afb20 0x7fc2500caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 -- 172.21.15.33:0/403040004 >> 172.21.15.33:0/403040004 conn(0x7fc25006d030 msgr2=0x7fc2500cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 -- 172.21.15.33:0/403040004 shutdown_connections 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.461+0000 7fc25719d700 1 -- 172.21.15.33:0/403040004 wait complete. 2023-10-06T05:25:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.462+0000 7fc25719d700 1 Processor -- start 2023-10-06T05:25:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.463+0000 7fc25719d700 1 -- start start 2023-10-06T05:25:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.463+0000 7fc25719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2500afb20 0x7fc250160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 0x7fc250169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250155a50 0x7fc2500c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501567e0 con 0x7fc2500c9490 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25016a030 con 0x7fc2500afb20 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25016a310 con 0x7fc250155a50 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 0x7fc250169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 0x7fc250169770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42162/0 (socket says 172.21.15.33:42162) 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 -- 172.21.15.33:0/3224302760 learned_addr learned my addr 172.21.15.33:0/3224302760 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25699c700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250155a50 0x7fc2500c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25619b700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2500afb20 0x7fc250160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 -- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250155a50 msgr2=0x7fc2500c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250155a50 0x7fc2500c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 -- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2500afb20 msgr2=0x7fc250160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2500afb20 0x7fc250160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 -- 172.21.15.33:0/3224302760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c00a690 con 0x7fc2500c9490 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.464+0000 7fc25599a700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 0x7fc250169770 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c790 tx=0x7fc240007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.465+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240011590 con 0x7fc2500c9490 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.465+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc240011710 con 0x7fc2500c9490 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.465+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240011ad0 con 0x7fc2500c9490 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.465+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25016a5f0 con 0x7fc2500c9490 2023-10-06T05:25:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.465+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc250169cb0 con 0x7fc2500c9490 2023-10-06T05:25:53.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.466+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250156950 con 0x7fc2500c9490 2023-10-06T05:25:53.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.467+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc24001b3b0 con 0x7fc2500c9490 2023-10-06T05:25:53.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.467+0000 7fc2467fc700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc234066f30 0x7fc2340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.468+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc240083b20 con 0x7fc2500c9490 2023-10-06T05:25:53.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.468+0000 7fc25619b700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc234066f30 0x7fc2340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.469+0000 7fc25619b700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc234066f30 0x7fc2340693e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00ab50 tx=0x7fc24c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.472+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc240050b60 con 0x7fc2500c9490 2023-10-06T05:25:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.671+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc25004c380 con 0x7fc2500c9490 2023-10-06T05:25:53.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.671+0000 7fc2467fc700 1 -- 172.21.15.33:0/3224302760 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc240017070 con 0x7fc2500c9490 2023-10-06T05:25:53.673 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:25:53.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc234066f30 msgr2=0x7fc2340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc234066f30 0x7fc2340693e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00ab50 tx=0x7fc24c019000 comp rx=0 tx=0).stop 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 msgr2=0x7fc250169770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 0x7fc250169770 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c790 tx=0x7fc240007f40 comp rx=0 tx=0).stop 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 shutdown_connections 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc234066f30 0x7fc2340693e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250155a50 0x7fc2500c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2500afb20 0x7fc250160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 --2- 172.21.15.33:0/3224302760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2500c9490 0x7fc250169770 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 >> 172.21.15.33:0/3224302760 conn(0x7fc25006d030 msgr2=0x7fc2500cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 shutdown_connections 2023-10-06T05:25:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.674+0000 7fc25719d700 1 -- 172.21.15.33:0/3224302760 wait complete. 2023-10-06T05:25:53.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:25:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.884+0000 7f18b959d700 1 Processor -- start 2023-10-06T05:25:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.884+0000 7f18b959d700 1 -- start start 2023-10-06T05:25:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b959d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f18b40b0900 0x7f18b40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b959d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b40ad430 con 0x7f18b4156370 2023-10-06T05:25:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b959d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b40afe20 con 0x7f18b40cc7e0 2023-10-06T05:25:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b959d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b406bc70 con 0x7f18b40b0900 2023-10-06T05:25:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f18b4156370 0x7f18b40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60208/0 (socket says 172.21.15.33:60208) 2023-10-06T05:25:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b3fff700 1 -- 172.21.15.33:0/146565831 learned_addr learned my addr 172.21.15.33:0/146565831 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b8d9c700 1 --2- 172.21.15.33:0/146565831 >> v2:172.21.15.33:3300/0 conn(0x7f18b40b0900 0x7f18b40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b2ffd700 1 -- 172.21.15.33:0/146565831 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994176479 0 0) 0x7f18b40ad430 con 0x7f18b4156370 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.886+0000 7f18b2ffd700 1 -- 172.21.15.33:0/146565831 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c003570 con 0x7f18b4156370 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b8d9c700 1 -- 172.21.15.33:0/146565831 >> v1:172.21.15.167:6789/0 conn(0x7f18b40cc7e0 legacy=0x7f18b40b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b8d9c700 1 -- 172.21.15.33:0/146565831 >> v1:172.21.15.33:6789/0 conn(0x7f18b4156370 legacy=0x7f18b40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b8d9c700 1 -- 172.21.15.33:0/146565831 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40cccc0 con 0x7f18b40b0900 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b8d9c700 1 --2- 172.21.15.33:0/146565831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40b0900 0x7f18b40af8e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009e60 tx=0x7f18ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b953427f2ed6a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b2ffd700 1 -- 172.21.15.33:0/146565831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00e070 con 0x7f18b40b0900 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b2ffd700 1 -- 172.21.15.33:0/146565831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac00bb60 con 0x7f18b40b0900 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.887+0000 7f18b2ffd700 1 -- 172.21.15.33:0/146565831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac012430 con 0x7f18b40b0900 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 -- 172.21.15.33:0/146565831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40b0900 msgr2=0x7f18b40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 --2- 172.21.15.33:0/146565831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40b0900 0x7f18b40af8e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009e60 tx=0x7f18ac009780 comp rx=0 tx=0).stop 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 -- 172.21.15.33:0/146565831 shutdown_connections 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 --2- 172.21.15.33:0/146565831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40b0900 0x7f18b40af8e0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 -- 172.21.15.33:0/146565831 >> 172.21.15.33:0/146565831 conn(0x7f18b406d030 msgr2=0x7f18b40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 -- 172.21.15.33:0/146565831 shutdown_connections 2023-10-06T05:25:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.888+0000 7f18b959d700 1 -- 172.21.15.33:0/146565831 wait complete. 2023-10-06T05:25:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.889+0000 7f18b959d700 1 Processor -- start 2023-10-06T05:25:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.889+0000 7f18b959d700 1 -- start start 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.890+0000 7f18b959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b40b0900 0x7f18b414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.890+0000 7f18b959d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 0x7f18b40c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.890+0000 7f18b959d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18b4156370 0x7f18b40ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.890+0000 7f18b959d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41576c0 con 0x7f18b40cc7e0 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.890+0000 7f18b959d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4156ab0 con 0x7f18b4156370 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.890+0000 7f18b959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4156d90 con 0x7f18b40b0900 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 0x7f18b40c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 0x7f18b40c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42186/0 (socket says 172.21.15.33:42186) 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 -- 172.21.15.33:0/809737229 learned_addr learned my addr 172.21.15.33:0/809737229 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b3fff700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b40b0900 0x7f18b414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 -- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b40b0900 msgr2=0x7f18b414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b40b0900 0x7f18b414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b8d9c700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18b4156370 0x7f18b40ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 -- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18b4156370 msgr2=0x7f18b40ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18b4156370 0x7f18b40ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 -- 172.21.15.33:0/809737229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac009a20 con 0x7f18b40cc7e0 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b3fff700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b40b0900 0x7f18b414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b8d9c700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18b4156370 0x7f18b40ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.891+0000 7f18b37fe700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 0x7f18b40c2c80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c590 tx=0x7f18a800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.892+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8011590 con 0x7f18b40cc7e0 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.892+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80116f0 con 0x7f18b40cc7e0 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.892+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8011ab0 con 0x7f18b40cc7e0 2023-10-06T05:25:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.892+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b4157070 con 0x7f18b40cc7e0 2023-10-06T05:25:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.892+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b4156740 con 0x7f18b40cc7e0 2023-10-06T05:25:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.893+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b404c380 con 0x7f18b40cc7e0 2023-10-06T05:25:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.894+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18a801b3b0 con 0x7f18b40cc7e0 2023-10-06T05:25:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.895+0000 7f189bfff700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.895+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18a8083cc0 con 0x7f18b40cc7e0 2023-10-06T05:25:53.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.896+0000 7f18b3fff700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.896+0000 7f18b3fff700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18a0066f30 0x7f18a00693e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f18a40058d0 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:53.899+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18a808a080 con 0x7f18b40cc7e0 2023-10-06T05:25:54.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.103+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f18b406f310 con 0x7f18b40cc7e0 2023-10-06T05:25:54.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.104+0000 7f189bfff700 1 -- 172.21.15.33:0/809737229 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v157) v1 ==== 92+0+10 (secure 0 0 0) 0x7f18a8017020 con 0x7f18b40cc7e0 2023-10-06T05:25:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:25:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18a0066f30 msgr2=0x7f18a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18a0066f30 0x7f18a00693e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f18a40058d0 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2023-10-06T05:25:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 msgr2=0x7f18b40c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 0x7f18b40c2c80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c590 tx=0x7f18a800c950 comp rx=0 tx=0).stop 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 shutdown_connections 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18b4156370 0x7f18b40ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18b40b0900 0x7f18b414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 --2- 172.21.15.33:0/809737229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b40cc7e0 0x7f18b40c2c80 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 >> 172.21.15.33:0/809737229 conn(0x7f18b406d030 msgr2=0x7f18b4069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 shutdown_connections 2023-10-06T05:25:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.106+0000 7f18b959d700 1 -- 172.21.15.33:0/809737229 wait complete. 2023-10-06T05:25:54.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:25:54.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.314+0000 7ff302b9d700 1 Processor -- start 2023-10-06T05:25:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.314+0000 7ff302b9d700 1 -- start start 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.315+0000 7ff302b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.315+0000 7ff302b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff2fc0cd5e0 0x7ff2fc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.315+0000 7ff302b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2fc15d660 con 0x7ff2fc15a650 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.315+0000 7ff302b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc0ce890 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.315+0000 7ff302b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc15aac0 con 0x7ff2fc0af540 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30139a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff2fc15a650 0x7ff2fc0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60220/0 (socket says 172.21.15.33:60220) 2023-10-06T05:25:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30139a700 1 -- 172.21.15.33:0/2515518359 learned_addr learned my addr 172.21.15.33:0/2515518359 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30239c700 1 --2- 172.21.15.33:0/2515518359 >> v2:172.21.15.167:3300/0 conn(0x7ff2fc0cd5e0 0x7ff2fc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff301b9b700 1 --2- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff300b99700 1 -- 172.21.15.33:0/2515518359 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333181087 0 0) 0x7ff2fc15d660 con 0x7ff2fc15a650 2023-10-06T05:25:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff300b99700 1 -- 172.21.15.33:0/2515518359 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e8003570 con 0x7ff2fc15a650 2023-10-06T05:25:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30239c700 1 -- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 msgr2=0x7ff2fc0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30239c700 1 --2- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30239c700 1 -- 172.21.15.33:0/2515518359 >> v1:172.21.15.33:6789/0 conn(0x7ff2fc15a650 legacy=0x7ff2fc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.316+0000 7ff30239c700 1 -- 172.21.15.33:0/2515518359 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc15d890 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.317+0000 7ff30239c700 1 --2- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc15d120 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009e60 tx=0x7ff2f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6032d1c8d00be860 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.317+0000 7ff300b99700 1 -- 172.21.15.33:0/2515518359 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400e070 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.317+0000 7ff300b99700 1 -- 172.21.15.33:0/2515518359 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f400bab0 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.317+0000 7ff300b99700 1 -- 172.21.15.33:0/2515518359 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400be70 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.317+0000 7ff302b9d700 1 -- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc0cd5e0 msgr2=0x7ff2fc15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.317+0000 7ff302b9d700 1 --2- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc15d120 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009e60 tx=0x7ff2f4009780 comp rx=0 tx=0).stop 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.318+0000 7ff302b9d700 1 -- 172.21.15.33:0/2515518359 shutdown_connections 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.318+0000 7ff302b9d700 1 --2- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc15d120 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.318+0000 7ff302b9d700 1 --2- 172.21.15.33:0/2515518359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.318+0000 7ff302b9d700 1 -- 172.21.15.33:0/2515518359 >> 172.21.15.33:0/2515518359 conn(0x7ff2fc06d9b0 msgr2=0x7ff2fc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.318+0000 7ff302b9d700 1 -- 172.21.15.33:0/2515518359 shutdown_connections 2023-10-06T05:25:54.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.318+0000 7ff302b9d700 1 -- 172.21.15.33:0/2515518359 wait complete. 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.319+0000 7ff302b9d700 1 Processor -- start 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.319+0000 7ff302b9d700 1 -- start start 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff302b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff302b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc0c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff302b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc15a650 0x7ff2fc0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff302b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0b4d50 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff302b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0b4140 con 0x7ff2fc15a650 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff302b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0b4420 con 0x7ff2fc0af540 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff30139a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc0c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff30139a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc0c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42188/0 (socket says 172.21.15.33:42188) 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.320+0000 7ff30139a700 1 -- 172.21.15.33:0/1253406838 learned_addr learned my addr 172.21.15.33:0/1253406838 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30239c700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc15a650 0x7ff2fc0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff301b9b700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30139a700 1 -- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 msgr2=0x7ff2fc14cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30139a700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc14cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30139a700 1 -- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc15a650 msgr2=0x7ff2fc0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30139a700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc15a650 0x7ff2fc0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30139a700 1 -- 172.21.15.33:0/1253406838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009a20 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff30139a700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc0c93d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c7c0 tx=0x7ff2ec007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec005b30 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec005c90 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec007340 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.321+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc0b4700 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.322+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc14d490 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.322+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc0b4f00 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.323+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2ec01b400 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.324+0000 7ff2f1ffb700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2e0066f30 0x7ff2e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.324+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff2ec084200 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.325+0000 7ff301b9b700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2e0066f30 0x7ff2e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.325+0000 7ff301b9b700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2e0066f30 0x7ff2e00693e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ca0 tx=0x7ff2f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.328+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2fc0b4f00 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.528+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2fc04c380 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.528+0000 7ff2f1ffb700 1 -- 172.21.15.33:0/1253406838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2ec0512c0 con 0x7ff2fc0cd5e0 2023-10-06T05:25:54.530 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:25:54.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2e0066f30 msgr2=0x7ff2e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2e0066f30 0x7ff2e00693e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ca0 tx=0x7ff2f8008040 comp rx=0 tx=0).stop 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 msgr2=0x7ff2fc0c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc0c93d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c7c0 tx=0x7ff2ec007e00 comp rx=0 tx=0).stop 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 shutdown_connections 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2e0066f30 0x7ff2e00693e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2fc15a650 0x7ff2fc0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2fc0af540 0x7ff2fc14cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 --2- 172.21.15.33:0/1253406838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0cd5e0 0x7ff2fc0c93d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 >> 172.21.15.33:0/1253406838 conn(0x7ff2fc06d9b0 msgr2=0x7ff2fc158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 shutdown_connections 2023-10-06T05:25:54.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.531+0000 7ff302b9d700 1 -- 172.21.15.33:0/1253406838 wait complete. 2023-10-06T05:25:54.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:25:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.737+0000 7fd52759e700 1 Processor -- start 2023-10-06T05:25:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.737+0000 7fd52759e700 1 -- start start 2023-10-06T05:25:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.738+0000 7fd52759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5280af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.738+0000 7fd52759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5280cd5e0 0x7fd52815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.738+0000 7fd52759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52815d660 con 0x7fd52815a650 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.738+0000 7fd52759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5280ce890 con 0x7fd5280cd5e0 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.738+0000 7fd52759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52815aac0 con 0x7fd5280af540 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5280cd5e0 0x7fd52815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5280cd5e0 0x7fd52815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42190/0 (socket says 172.21.15.33:42190) 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 -- 172.21.15.33:0/3017144474 learned_addr learned my addr 172.21.15.33:0/3017144474 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd52659c700 1 --2- 172.21.15.33:0/3017144474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5280af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 -- 172.21.15.33:0/3017144474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 msgr2=0x7fd5280af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 --2- 172.21.15.33:0/3017144474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5280af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd52559a700 1 -- 172.21.15.33:0/3017144474 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605095998 0 0) 0x7fd52815d660 con 0x7fd52815a650 2023-10-06T05:25:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 -- 172.21.15.33:0/3017144474 >> v1:172.21.15.167:6789/0 conn(0x7fd52815a650 legacy=0x7fd5280cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.739+0000 7fd526d9d700 1 -- 172.21.15.33:0/3017144474 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52815d890 con 0x7fd5280cd5e0 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd526d9d700 1 --2- 172.21.15.33:0/3017144474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5280cd5e0 0x7fd52815d120 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd520009f20 tx=0x7fd520009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30ef851cb693719 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52559a700 1 -- 172.21.15.33:0/3017144474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000e070 con 0x7fd5280cd5e0 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52559a700 1 -- 172.21.15.33:0/3017144474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52000b9d0 con 0x7fd5280cd5e0 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52559a700 1 -- 172.21.15.33:0/3017144474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5200092e0 con 0x7fd5280cd5e0 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 -- 172.21.15.33:0/3017144474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5280cd5e0 msgr2=0x7fd52815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 --2- 172.21.15.33:0/3017144474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5280cd5e0 0x7fd52815d120 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd520009f20 tx=0x7fd520009840 comp rx=0 tx=0).stop 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 -- 172.21.15.33:0/3017144474 shutdown_connections 2023-10-06T05:25:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 --2- 172.21.15.33:0/3017144474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5280cd5e0 0x7fd52815d120 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 --2- 172.21.15.33:0/3017144474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5280af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 -- 172.21.15.33:0/3017144474 >> 172.21.15.33:0/3017144474 conn(0x7fd52806d9b0 msgr2=0x7fd52806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 -- 172.21.15.33:0/3017144474 shutdown_connections 2023-10-06T05:25:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.740+0000 7fd52759e700 1 -- 172.21.15.33:0/3017144474 wait complete. 2023-10-06T05:25:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.742+0000 7fd52759e700 1 Processor -- start 2023-10-06T05:25:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.742+0000 7fd52759e700 1 -- start start 2023-10-06T05:25:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd52759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5281590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd52759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5280cd5e0 0x7fd528169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd52759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 0x7fd5280c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd52759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280c7c10 con 0x7fd52815a650 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd52759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528159630 con 0x7fd5280cd5e0 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd52759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280b37f0 con 0x7fd5280af540 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd526d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 0x7fd5280c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd526d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 0x7fd5280c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42196/0 (socket says 172.21.15.33:42196) 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.743+0000 7fd526d9d700 1 -- 172.21.15.33:0/2233611822 learned_addr learned my addr 172.21.15.33:0/2233611822 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd52659c700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5281590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd525d9b700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5280cd5e0 0x7fd528169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd526d9d700 1 -- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 msgr2=0x7fd5281590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd526d9d700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5281590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd526d9d700 1 -- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5280cd5e0 msgr2=0x7fd528169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd526d9d700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5280cd5e0 0x7fd528169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd526d9d700 1 -- 172.21.15.33:0/2233611822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520009ae0 con 0x7fd52815a650 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd526d9d700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 0x7fd5280c8800 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd520012000 tx=0x7fd520018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520008c70 con 0x7fd52815a650 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520008df0 con 0x7fd52815a650 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5200266b0 con 0x7fd52815a650 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5280b3ad0 con 0x7fd52815a650 2023-10-06T05:25:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.744+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5280b4090 con 0x7fd52815a650 2023-10-06T05:25:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.745+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52804c380 con 0x7fd52815a650 2023-10-06T05:25:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.746+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd520003040 con 0x7fd52815a650 2023-10-06T05:25:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.748+0000 7fd5167fc700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd508066f30 0x7fd5080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.748+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd520086270 con 0x7fd52815a650 2023-10-06T05:25:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.748+0000 7fd52659c700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd508066f30 0x7fd5080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.749+0000 7fd52659c700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd508066f30 0x7fd5080693e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd518009ca0 tx=0x7fd518008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.752+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd520053360 con 0x7fd52815a650 2023-10-06T05:25:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.954+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd52815db70 con 0x7fd52815a650 2023-10-06T05:25:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.955+0000 7fd5167fc700 1 -- 172.21.15.33:0/2233611822 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd520056980 con 0x7fd52815a650 2023-10-06T05:25:54.956 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:25:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd508066f30 msgr2=0x7fd5080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd508066f30 0x7fd5080693e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd518009ca0 tx=0x7fd518008040 comp rx=0 tx=0).stop 2023-10-06T05:25:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 msgr2=0x7fd5280c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 0x7fd5280c8800 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd520012000 tx=0x7fd520018000 comp rx=0 tx=0).stop 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 shutdown_connections 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd508066f30 0x7fd5080693e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd52815a650 0x7fd5280c8800 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5280af540 0x7fd5281590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 --2- 172.21.15.33:0/2233611822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5280cd5e0 0x7fd528169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 >> 172.21.15.33:0/2233611822 conn(0x7fd52806d9b0 msgr2=0x7fd528159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 shutdown_connections 2023-10-06T05:25:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:54.957+0000 7fd52759e700 1 -- 172.21.15.33:0/2233611822 wait complete. 2023-10-06T05:25:54.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:55.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.161+0000 7faf55d9d700 1 Processor -- start 2023-10-06T05:25:55.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.161+0000 7faf55d9d700 1 -- start start 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf55d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 0x7faf5013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf55d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 0x7faf5006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf55d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500c9cb0 con 0x7faf500c7940 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf55d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500ca270 con 0x7faf5013faa0 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf55d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5006b840 con 0x7faf5006aad0 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf54d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 0x7faf5013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf5559c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 0x7faf5006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf54d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 0x7faf5013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:57164/0 (socket says 172.21.15.33:57164) 2023-10-06T05:25:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf54d9b700 1 -- 172.21.15.33:0/2909562494 learned_addr learned my addr 172.21.15.33:0/2909562494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.163+0000 7faf5559c700 1 --2- 172.21.15.33:0/2909562494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 0x7faf5006af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:43878/0 (socket says 172.21.15.33:43878) 2023-10-06T05:25:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf5559c700 1 -- 172.21.15.33:0/2909562494 >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 msgr2=0x7faf5013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf5559c700 1 --2- 172.21.15.33:0/2909562494 >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 0x7faf5013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf5559c700 1 -- 172.21.15.33:0/2909562494 >> v1:172.21.15.167:6789/0 conn(0x7faf500c7940 legacy=0x7faf500c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf5559c700 1 -- 172.21.15.33:0/2909562494 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5006bb20 con 0x7faf5006aad0 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf54d9b700 1 --2- 172.21.15.33:0/2909562494 >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 0x7faf5013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf5559c700 1 --2- 172.21.15.33:0/2909562494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 0x7faf5006af50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faf48009e60 tx=0x7faf48009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a65c2e45453562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf4f7fe700 1 -- 172.21.15.33:0/2909562494 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800e070 con 0x7faf5006aad0 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf4f7fe700 1 -- 172.21.15.33:0/2909562494 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4800bb60 con 0x7faf5006aad0 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.164+0000 7faf4f7fe700 1 -- 172.21.15.33:0/2909562494 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48012430 con 0x7faf5006aad0 2023-10-06T05:25:55.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 -- 172.21.15.33:0/2909562494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 msgr2=0x7faf5006af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 --2- 172.21.15.33:0/2909562494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 0x7faf5006af50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faf48009e60 tx=0x7faf48009780 comp rx=0 tx=0).stop 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 -- 172.21.15.33:0/2909562494 shutdown_connections 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 --2- 172.21.15.33:0/2909562494 >> v2:172.21.15.167:3300/0 conn(0x7faf5013faa0 0x7faf5013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 --2- 172.21.15.33:0/2909562494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf5006aad0 0x7faf5006af50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 -- 172.21.15.33:0/2909562494 >> 172.21.15.33:0/2909562494 conn(0x7faf5006d030 msgr2=0x7faf5006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 -- 172.21.15.33:0/2909562494 shutdown_connections 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.165+0000 7faf55d9d700 1 -- 172.21.15.33:0/2909562494 wait complete. 2023-10-06T05:25:55.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.166+0000 7faf55d9d700 1 Processor -- start 2023-10-06T05:25:55.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.167+0000 7faf55d9d700 1 -- start start 2023-10-06T05:25:55.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.167+0000 7faf55d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf5006aad0 0x7faf50155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.167+0000 7faf55d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf500c7940 0x7faf500b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.167+0000 7faf55d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 0x7faf50158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf55d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50156420 con 0x7faf5013faa0 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf55d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500b0790 con 0x7faf5006aad0 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf55d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf501579b0 con 0x7faf500c7940 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 0x7faf50158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 0x7faf50158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42200/0 (socket says 172.21.15.33:42200) 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 -- 172.21.15.33:0/876472849 learned_addr learned my addr 172.21.15.33:0/876472849 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:55.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf4ffff700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf500c7940 0x7faf500b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf54d9b700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf5006aad0 0x7faf50155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 -- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf500c7940 msgr2=0x7faf500b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf500c7940 0x7faf500b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 -- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf5006aad0 msgr2=0x7faf50155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf5006aad0 0x7faf50155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.168+0000 7faf5559c700 1 -- 172.21.15.33:0/876472849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440094f0 con 0x7faf5013faa0 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.169+0000 7faf5559c700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 0x7faf50158a50 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7faf5006b980 tx=0x7faf48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.169+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48012e80 con 0x7faf5013faa0 2023-10-06T05:25:55.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.169+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4801d630 con 0x7faf5013faa0 2023-10-06T05:25:55.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.169+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf48009a20 con 0x7faf5013faa0 2023-10-06T05:25:55.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.169+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4801d9f0 con 0x7faf5013faa0 2023-10-06T05:25:55.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.169+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf50157d40 con 0x7faf5013faa0 2023-10-06T05:25:55.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.170+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5004c380 con 0x7faf5013faa0 2023-10-06T05:25:55.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.171+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf48024070 con 0x7faf5013faa0 2023-10-06T05:25:55.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.172+0000 7faf4cff9700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf34066f30 0x7faf340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.172+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faf48086fe0 con 0x7faf5013faa0 2023-10-06T05:25:55.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.172+0000 7faf54d9b700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf34066f30 0x7faf340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.173+0000 7faf54d9b700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf34066f30 0x7faf340693e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faf44006fd0 tx=0x7faf44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.176+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf480540d0 con 0x7faf5013faa0 2023-10-06T05:25:55.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.424+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf5006fc10 con 0x7faf5013faa0 2023-10-06T05:25:55.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.425+0000 7faf4cff9700 1 -- 172.21.15.33:0/876472849 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7faf480576f0 con 0x7faf5013faa0 2023-10-06T05:25:55.427 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:55.427 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf34066f30 msgr2=0x7faf340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf34066f30 0x7faf340693e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faf44006fd0 tx=0x7faf44008040 comp rx=0 tx=0).stop 2023-10-06T05:25:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 msgr2=0x7faf50158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 0x7faf50158a50 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7faf5006b980 tx=0x7faf48019000 comp rx=0 tx=0).stop 2023-10-06T05:25:55.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 shutdown_connections 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf34066f30 0x7faf340693e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf5013faa0 0x7faf50158a50 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf5006aad0 0x7faf50155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 --2- 172.21.15.33:0/876472849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf500c7940 0x7faf500b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 >> 172.21.15.33:0/876472849 conn(0x7faf5006d030 msgr2=0x7faf500b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 shutdown_connections 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.427+0000 7faf55d9d700 1 -- 172.21.15.33:0/876472849 wait complete. 2023-10-06T05:25:55.430 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:55.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:55.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.631+0000 7f98d7fff700 1 Processor -- start 2023-10-06T05:25:55.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.631+0000 7f98d7fff700 1 -- start start 2023-10-06T05:25:55.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f98d80c6340 0x7f98d80c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d806c820 con 0x7f98d8159520 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d806a160 con 0x7f98d80c6340 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80c8800 con 0x7f98d80c80d0 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d6ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f98d8159520 0x7f98d80c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60236/0 (socket says 172.21.15.33:60236) 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d6ffd700 1 -- 172.21.15.33:0/727574164 learned_addr learned my addr 172.21.15.33:0/727574164 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d67fc700 1 --2- 172.21.15.33:0/727574164 >> v2:172.21.15.167:3300/0 conn(0x7f98d80c6340 0x7f98d80c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d77fe700 1 --2- 172.21.15.33:0/727574164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d5ffb700 1 -- 172.21.15.33:0/727574164 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045641146 0 0) 0x7f98d806c820 con 0x7f98d8159520 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.633+0000 7f98d5ffb700 1 -- 172.21.15.33:0/727574164 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c0003570 con 0x7f98d8159520 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d77fe700 1 -- 172.21.15.33:0/727574164 >> v2:172.21.15.167:3300/0 conn(0x7f98d80c6340 msgr2=0x7f98d80c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d77fe700 1 --2- 172.21.15.33:0/727574164 >> v2:172.21.15.167:3300/0 conn(0x7f98d80c6340 0x7f98d80c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d77fe700 1 -- 172.21.15.33:0/727574164 >> v1:172.21.15.33:6789/0 conn(0x7f98d8159520 legacy=0x7f98d80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d77fe700 1 -- 172.21.15.33:0/727574164 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80c8ae0 con 0x7f98d80c80d0 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d67fc700 1 --2- 172.21.15.33:0/727574164 >> v2:172.21.15.167:3300/0 conn(0x7f98d80c6340 0x7f98d80c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d77fe700 1 --2- 172.21.15.33:0/727574164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d806a660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009e60 tx=0x7f98d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1e451c8a1ef2c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d5ffb700 1 -- 172.21.15.33:0/727574164 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000e070 con 0x7f98d80c80d0 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d5ffb700 1 -- 172.21.15.33:0/727574164 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d000bab0 con 0x7f98d80c80d0 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.634+0000 7f98d5ffb700 1 -- 172.21.15.33:0/727574164 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000be70 con 0x7f98d80c80d0 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 -- 172.21.15.33:0/727574164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 msgr2=0x7f98d806a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 --2- 172.21.15.33:0/727574164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d806a660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009e60 tx=0x7f98d0009780 comp rx=0 tx=0).stop 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 -- 172.21.15.33:0/727574164 shutdown_connections 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 --2- 172.21.15.33:0/727574164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d806a660 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 --2- 172.21.15.33:0/727574164 >> v2:172.21.15.167:3300/0 conn(0x7f98d80c6340 0x7f98d80c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 -- 172.21.15.33:0/727574164 >> 172.21.15.33:0/727574164 conn(0x7f98d80cc930 msgr2=0x7f98d80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 -- 172.21.15.33:0/727574164 shutdown_connections 2023-10-06T05:25:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.635+0000 7f98d7fff700 1 -- 172.21.15.33:0/727574164 wait complete. 2023-10-06T05:25:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.636+0000 7f98d7fff700 1 Processor -- start 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.636+0000 7f98d7fff700 1 -- start start 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 0x7f98d816caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d80b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d8159520 0x7f98d8153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8169a40 con 0x7f98d80c6340 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d816da00 con 0x7f98d8159520 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8169710 con 0x7f98d80c80d0 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.637+0000 7f98d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 0x7f98d816caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 0x7f98d816caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42212/0 (socket says 172.21.15.33:42212) 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 -- 172.21.15.33:0/3837838386 learned_addr learned my addr 172.21.15.33:0/3837838386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d67fc700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d80b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d77fe700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d8159520 0x7f98d8153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 -- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 msgr2=0x7f98d80b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d80b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 -- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d8159520 msgr2=0x7f98d8153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d8159520 0x7f98d8153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 -- 172.21.15.33:0/3837838386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009420 con 0x7f98d80c6340 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d67fc700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d80b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d77fe700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d8159520 0x7f98d8153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98d6ffd700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 0x7f98d816caf0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c590 tx=0x7f98c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8005b30 con 0x7f98d80c6340 2023-10-06T05:25:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.638+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8005cb0 con 0x7f98d80c6340 2023-10-06T05:25:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.639+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8007320 con 0x7f98d80c6340 2023-10-06T05:25:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.639+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d0009a20 con 0x7f98d80c6340 2023-10-06T05:25:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.639+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d816d720 con 0x7f98d80c6340 2023-10-06T05:25:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.640+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d8169570 con 0x7f98d80c6340 2023-10-06T05:25:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.640+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98c801b3b0 con 0x7f98d80c6340 2023-10-06T05:25:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.642+0000 7f98beffd700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98c4066f30 0x7f98c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.642+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98c8083d80 con 0x7f98d80c6340 2023-10-06T05:25:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.642+0000 7f98d67fc700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98c4066f30 0x7f98c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.643+0000 7f98d67fc700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98c4066f30 0x7f98c40693e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0093f0 tx=0x7f98cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.645+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98d8169570 con 0x7f98d80c6340 2023-10-06T05:25:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.848+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98d8169570 con 0x7f98d80c6340 2023-10-06T05:25:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.849+0000 7f98beffd700 1 -- 172.21.15.33:0/3837838386 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f98d8169570 con 0x7f98d80c6340 2023-10-06T05:25:55.851 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:55.851 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":157,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:25:13.046893+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:59.024220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:00.278297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1400837691":"2023-10-06T06:25:12.317114+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98c4066f30 msgr2=0x7f98c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98c4066f30 0x7f98c40693e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0093f0 tx=0x7f98cc008040 comp rx=0 tx=0).stop 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 msgr2=0x7f98d816caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 0x7f98d816caf0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c590 tx=0x7f98c8007e00 comp rx=0 tx=0).stop 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 shutdown_connections 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98c4066f30 0x7f98c40693e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98d8159520 0x7f98d8153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d80c6340 0x7f98d816caf0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 --2- 172.21.15.33:0/3837838386 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98d80c80d0 0x7f98d80b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 >> 172.21.15.33:0/3837838386 conn(0x7f98d80cc930 msgr2=0x7f98d806ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 shutdown_connections 2023-10-06T05:25:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:55.852+0000 7f98d7fff700 1 -- 172.21.15.33:0/3837838386 wait complete. 2023-10-06T05:25:55.872 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:25:55.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:56.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.058+0000 7fc7bbfff700 1 Processor -- start 2023-10-06T05:25:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.058+0000 7fc7bbfff700 1 -- start start 2023-10-06T05:25:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.059+0000 7fc7bbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc0c8b50 0x7fc7bc06e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.059+0000 7fc7bbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc06ab60 con 0x7fc7bc155af0 2023-10-06T05:25:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.059+0000 7fc7bbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc06ae40 con 0x7fc7bc155f60 2023-10-06T05:25:56.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.059+0000 7fc7bbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc06b7e0 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.059+0000 7fc7ba7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc7bc155af0 0x7fc7bc06a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60252/0 (socket says 172.21.15.33:60252) 2023-10-06T05:25:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.059+0000 7fc7ba7fc700 1 -- 172.21.15.33:0/3562472815 learned_addr learned my addr 172.21.15.33:0/3562472815 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7baffd700 1 --2- 172.21.15.33:0/3562472815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc0c8b50 0x7fc7bc06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7b9ffb700 1 -- 172.21.15.33:0/3562472815 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092214906 0 0) 0x7fc7bc06ab60 con 0x7fc7bc155af0 2023-10-06T05:25:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7b9ffb700 1 -- 172.21.15.33:0/3562472815 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a4003570 con 0x7fc7bc155af0 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7baffd700 1 -- 172.21.15.33:0/3562472815 >> v1:172.21.15.167:6789/0 conn(0x7fc7bc155f60 legacy=0x7fc7bc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7baffd700 1 -- 172.21.15.33:0/3562472815 >> v1:172.21.15.33:6789/0 conn(0x7fc7bc155af0 legacy=0x7fc7bc06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7b9ffb700 1 -- 172.21.15.33:0/3562472815 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131620043 0 0) 0x7fc7bc06ae40 con 0x7fc7bc155f60 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.060+0000 7fc7baffd700 1 -- 172.21.15.33:0/3562472815 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc06b120 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.061+0000 7fc7baffd700 1 --2- 172.21.15.33:0/3562472815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc0c8b50 0x7fc7bc06e960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009910 tx=0x7fc7ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f61684aca88e54c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.061+0000 7fc7b9ffb700 1 -- 172.21.15.33:0/3562472815 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00f6b0 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.061+0000 7fc7b9ffb700 1 -- 172.21.15.33:0/3562472815 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00f810 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.061+0000 7fc7b9ffb700 1 -- 172.21.15.33:0/3562472815 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00fbd0 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 -- 172.21.15.33:0/3562472815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc0c8b50 msgr2=0x7fc7bc06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/3562472815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc0c8b50 0x7fc7bc06e960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009910 tx=0x7fc7ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 -- 172.21.15.33:0/3562472815 shutdown_connections 2023-10-06T05:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/3562472815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc0c8b50 0x7fc7bc06e960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 -- 172.21.15.33:0/3562472815 >> 172.21.15.33:0/3562472815 conn(0x7fc7bc06d030 msgr2=0x7fc7bc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 -- 172.21.15.33:0/3562472815 shutdown_connections 2023-10-06T05:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.062+0000 7fc7bbfff700 1 -- 172.21.15.33:0/3562472815 wait complete. 2023-10-06T05:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.063+0000 7fc7bbfff700 1 Processor -- start 2023-10-06T05:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.063+0000 7fc7bbfff700 1 -- start start 2023-10-06T05:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.064+0000 7fc7bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 0x7fc7bc0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.064+0000 7fc7bbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7bc155af0 0x7fc7bc0c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.064+0000 7fc7bbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc155f60 0x7fc7bc0c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.064+0000 7fc7bbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc1669f0 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.064+0000 7fc7bbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0c71b0 con 0x7fc7bc155af0 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.064+0000 7fc7bbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0c6880 con 0x7fc7bc155f60 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 0x7fc7bc0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 0x7fc7bc0c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42228/0 (socket says 172.21.15.33:42228) 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 -- 172.21.15.33:0/1169868515 learned_addr learned my addr 172.21.15.33:0/1169868515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7ba7fc700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7bc155af0 0x7fc7bc0c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7bb7fe700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc155f60 0x7fc7bc0c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 -- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc155f60 msgr2=0x7fc7bc0c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc155f60 0x7fc7bc0c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 -- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7bc155af0 msgr2=0x7fc7bc0c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7bc155af0 0x7fc7bc0c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 -- 172.21.15.33:0/1169868515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac009420 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.065+0000 7fc7baffd700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 0x7fc7bc0c5760 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0635d0 tx=0x7fc7ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.066+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac01f070 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.066+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00f980 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.066+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac010470 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.066+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc0c6b60 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.066+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc0c7640 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.067+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc04c380 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.068+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7ac023070 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.069+0000 7fc7a2ffd700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7a8066fb0 0x7fc7a8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.069+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7ac014070 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.070+0000 7fc7ba7fc700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7a8066fb0 0x7fc7a8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.070+0000 7fc7ba7fc700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7a8066fb0 0x7fc7a8069460 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00058d0 tx=0x7fc7b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.073+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7ac054890 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.324+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7bc06b400 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.324+0000 7fc7a2ffd700 1 -- 172.21.15.33:0/1169868515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fc7ac057eb0 con 0x7fc7bc0c8b50 2023-10-06T05:25:56.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:56.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7a8066fb0 msgr2=0x7fc7a8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7a8066fb0 0x7fc7a8069460 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00058d0 tx=0x7fc7b0009040 comp rx=0 tx=0).stop 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 msgr2=0x7fc7bc0c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 0x7fc7bc0c5760 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0635d0 tx=0x7fc7ac019000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 shutdown_connections 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7a8066fb0 0x7fc7a8069460 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7bc155f60 0x7fc7bc0c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7bc0c8b50 0x7fc7bc0c5760 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 --2- 172.21.15.33:0/1169868515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7bc155af0 0x7fc7bc0c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 >> 172.21.15.33:0/1169868515 conn(0x7fc7bc06d030 msgr2=0x7fc7bc0cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 shutdown_connections 2023-10-06T05:25:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.327+0000 7fc7bbfff700 1 -- 172.21.15.33:0/1169868515 wait complete. 2023-10-06T05:25:56.330 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:56.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.531+0000 7f0eee79d700 1 Processor -- start 2023-10-06T05:25:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.532+0000 7f0eee79d700 1 -- start start 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eee79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ee80cf3b0 0x7f0ee81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eee79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee80cbd80 con 0x7f0ee80ccfc0 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eee79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee80af5f0 con 0x7f0ee80af0a0 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eee79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee80af8d0 con 0x7f0ee80cf3b0 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eed79b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0ee80ccfc0 0x7f0ee80cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60258/0 (socket says 172.21.15.33:60258) 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eed79b700 1 -- 172.21.15.33:0/2232382542 learned_addr learned my addr 172.21.15.33:0/2232382542 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:56.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.533+0000 7f0eecf9a700 1 --2- 172.21.15.33:0/2232382542 >> v2:172.21.15.33:3300/0 conn(0x7f0ee80cf3b0 0x7f0ee81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0edffff700 1 -- 172.21.15.33:0/2232382542 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845678681 0 0) 0x7f0ee80cbd80 con 0x7f0ee80ccfc0 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0edffff700 1 -- 172.21.15.33:0/2232382542 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed4003570 con 0x7f0ee80ccfc0 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0eecf9a700 1 -- 172.21.15.33:0/2232382542 >> v1:172.21.15.167:6789/0 conn(0x7f0ee80af0a0 legacy=0x7f0ee80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0eecf9a700 1 -- 172.21.15.33:0/2232382542 >> v1:172.21.15.33:6789/0 conn(0x7f0ee80ccfc0 legacy=0x7f0ee80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0eecf9a700 1 -- 172.21.15.33:0/2232382542 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee80afbb0 con 0x7f0ee80cf3b0 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0eecf9a700 1 --2- 172.21.15.33:0/2232382542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80cf3b0 0x7f0ee81594a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009910 tx=0x7f0ed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3efbe7d11ff96b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0edffff700 1 -- 172.21.15.33:0/2232382542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800b640 con 0x7f0ee80cf3b0 2023-10-06T05:25:56.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0edffff700 1 -- 172.21.15.33:0/2232382542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed800b7a0 con 0x7f0ee80cf3b0 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.534+0000 7f0edffff700 1 -- 172.21.15.33:0/2232382542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800bb60 con 0x7f0ee80cf3b0 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 -- 172.21.15.33:0/2232382542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80cf3b0 msgr2=0x7f0ee81594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 --2- 172.21.15.33:0/2232382542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80cf3b0 0x7f0ee81594a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009910 tx=0x7f0ed80091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 -- 172.21.15.33:0/2232382542 shutdown_connections 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 --2- 172.21.15.33:0/2232382542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80cf3b0 0x7f0ee81594a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 -- 172.21.15.33:0/2232382542 >> 172.21.15.33:0/2232382542 conn(0x7f0ee806cff0 msgr2=0x7f0ee806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 -- 172.21.15.33:0/2232382542 shutdown_connections 2023-10-06T05:25:56.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.535+0000 7f0eee79d700 1 -- 172.21.15.33:0/2232382542 wait complete. 2023-10-06T05:25:56.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.536+0000 7f0eee79d700 1 Processor -- start 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.537+0000 7f0eee79d700 1 -- start start 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.537+0000 7f0eee79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 0x7f0ee8152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.537+0000 7f0eee79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee80ccfc0 0x7f0ee81512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.537+0000 7f0eee79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee81517f0 0x7f0ee8153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eee79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8152510 con 0x7f0ee80af0a0 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eee79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee814bb30 con 0x7f0ee80ccfc0 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eee79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8151c70 con 0x7f0ee81517f0 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 0x7f0ee8152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 0x7f0ee8152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42256/0 (socket says 172.21.15.33:42256) 2023-10-06T05:25:56.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 -- 172.21.15.33:0/1382252183 learned_addr learned my addr 172.21.15.33:0/1382252183 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eecf9a700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee80ccfc0 0x7f0ee81512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eedf9c700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee81517f0 0x7f0ee8153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 -- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee81517f0 msgr2=0x7f0ee8153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee81517f0 0x7f0ee8153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 -- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee80ccfc0 msgr2=0x7f0ee81512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee80ccfc0 0x7f0ee81512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 -- 172.21.15.33:0/1382252183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8009420 con 0x7f0ee80af0a0 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eecf9a700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee80ccfc0 0x7f0ee81512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.538+0000 7f0eed79b700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 0x7f0ee8152c90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400c7c0 tx=0x7f0ee4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.539+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4005b30 con 0x7f0ee80af0a0 2023-10-06T05:25:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.539+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee4005c90 con 0x7f0ee80af0a0 2023-10-06T05:25:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.539+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4007340 con 0x7f0ee80af0a0 2023-10-06T05:25:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.539+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8151f50 con 0x7f0ee80af0a0 2023-10-06T05:25:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.539+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee81531d0 con 0x7f0ee80af0a0 2023-10-06T05:25:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.540+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee814b900 con 0x7f0ee80af0a0 2023-10-06T05:25:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.541+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ee401b400 con 0x7f0ee80af0a0 2023-10-06T05:25:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.541+0000 7f0edd7fa700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ecc066f30 0x7f0ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.542+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ee4084220 con 0x7f0ee80af0a0 2023-10-06T05:25:56.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.542+0000 7f0eecf9a700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ecc066f30 0x7f0ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.542+0000 7f0eecf9a700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ecc066f30 0x7f0ecc0693e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b0250 tx=0x7f0ed8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.545+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ee814b900 con 0x7f0ee80af0a0 2023-10-06T05:25:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.792+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ee814b900 con 0x7f0ee80af0a0 2023-10-06T05:25:56.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.793+0000 7f0edd7fa700 1 -- 172.21.15.33:0/1382252183 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f0ee814b900 con 0x7f0ee80af0a0 2023-10-06T05:25:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:56.795 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":114,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:01.298802+0000","modified":"2023-10-06T05:25:04.829925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8059":{"gid":8059,"name":"b","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/582175774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":582175774},{"type":"v1","addr":"172.21.15.167:6837","nonce":582175774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ecc066f30 msgr2=0x7f0ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ecc066f30 0x7f0ecc0693e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b0250 tx=0x7f0ed8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 msgr2=0x7f0ee8152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 0x7f0ee8152c90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400c7c0 tx=0x7f0ee4007e00 comp rx=0 tx=0).stop 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 shutdown_connections 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ecc066f30 0x7f0ecc0693e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ee81517f0 0x7f0ee8153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ee80af0a0 0x7f0ee8152c90 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 --2- 172.21.15.33:0/1382252183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ee80ccfc0 0x7f0ee81512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 >> 172.21.15.33:0/1382252183 conn(0x7f0ee806cff0 msgr2=0x7f0ee806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 shutdown_connections 2023-10-06T05:25:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:56.795+0000 7f0eee79d700 1 -- 172.21.15.33:0/1382252183 wait complete. 2023-10-06T05:25:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 114 2023-10-06T05:25:56.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.003+0000 7fe53519d700 1 Processor -- start 2023-10-06T05:25:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.003+0000 7fe53519d700 1 -- start start 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 0x7fe53006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53519d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 0x7fe5300c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53519d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53006ab60 con 0x7fe53006a6a0 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53519d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53006b7e0 con 0x7fe5300c7920 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53006ae40 con 0x7fe530155af0 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe52ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 0x7fe53006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53499c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 0x7fe5300c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53499c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 0x7fe5300c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:57206/0 (socket says 172.21.15.33:57206) 2023-10-06T05:25:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe52ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 0x7fe53006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:43924/0 (socket says 172.21.15.33:43924) 2023-10-06T05:25:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.005+0000 7fe53499c700 1 -- 172.21.15.33:0/3144630136 learned_addr learned my addr 172.21.15.33:0/3144630136 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52ffff700 1 -- 172.21.15.33:0/3144630136 >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 msgr2=0x7fe5300c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52ffff700 1 --2- 172.21.15.33:0/3144630136 >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 0x7fe5300c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52ffff700 1 -- 172.21.15.33:0/3144630136 >> v1:172.21.15.167:6789/0 conn(0x7fe53006a6a0 legacy=0x7fe5300c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52ffff700 1 -- 172.21.15.33:0/3144630136 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53006b120 con 0x7fe530155af0 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe53499c700 1 --2- 172.21.15.33:0/3144630136 >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 0x7fe5300c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52ffff700 1 --2- 172.21.15.33:0/3144630136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 0x7fe53006a160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098e0 tx=0x7fe5200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec1372463156517 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52effd700 1 -- 172.21.15.33:0/3144630136 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000b640 con 0x7fe530155af0 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52effd700 1 -- 172.21.15.33:0/3144630136 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52000b7a0 con 0x7fe530155af0 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.006+0000 7fe52effd700 1 -- 172.21.15.33:0/3144630136 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000bb40 con 0x7fe530155af0 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 -- 172.21.15.33:0/3144630136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 msgr2=0x7fe53006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 --2- 172.21.15.33:0/3144630136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 0x7fe53006a160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098e0 tx=0x7fe5200091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 -- 172.21.15.33:0/3144630136 shutdown_connections 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 --2- 172.21.15.33:0/3144630136 >> v2:172.21.15.167:3300/0 conn(0x7fe5300c7920 0x7fe5300c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 --2- 172.21.15.33:0/3144630136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe530155af0 0x7fe53006a160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 -- 172.21.15.33:0/3144630136 >> 172.21.15.33:0/3144630136 conn(0x7fe53006d030 msgr2=0x7fe53006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 -- 172.21.15.33:0/3144630136 shutdown_connections 2023-10-06T05:25:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.007+0000 7fe53519d700 1 -- 172.21.15.33:0/3144630136 wait complete. 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.008+0000 7fe53519d700 1 Processor -- start 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 -- start start 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe53006a6a0 0x7fe530155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5300c7920 0x7fe5300b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 0x7fe5300cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530165650 con 0x7fe530155af0 2023-10-06T05:25:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530165910 con 0x7fe5300c7920 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.009+0000 7fe53519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe530165bf0 con 0x7fe53006a6a0 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 0x7fe5300cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 0x7fe5300cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42260/0 (socket says 172.21.15.33:42260) 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 -- 172.21.15.33:0/1685415214 learned_addr learned my addr 172.21.15.33:0/1685415214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe52ffff700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe53006a6a0 0x7fe530155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe52f7fe700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5300c7920 0x7fe5300b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 -- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe53006a6a0 msgr2=0x7fe530155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe53006a6a0 0x7fe530155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 -- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5300c7920 msgr2=0x7fe5300b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5300c7920 0x7fe5300b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe53499c700 1 -- 172.21.15.33:0/1685415214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52800bcf0 con 0x7fe530155af0 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.010+0000 7fe52f7fe700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5300c7920 0x7fe5300b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.011+0000 7fe53499c700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 0x7fe5300cac80 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe528010870 tx=0x7fe52800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.011+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52801b070 con 0x7fe530155af0 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.011+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe528011040 con 0x7fe530155af0 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.011+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe520009420 con 0x7fe530155af0 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.011+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528015800 con 0x7fe530155af0 2023-10-06T05:25:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.011+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5301661b0 con 0x7fe530155af0 2023-10-06T05:25:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.012+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5300c84d0 con 0x7fe530155af0 2023-10-06T05:25:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.013+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe528003020 con 0x7fe530155af0 2023-10-06T05:25:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.014+0000 7fe517fff700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe51c066f30 0x7fe51c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.014+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe528086f50 con 0x7fe530155af0 2023-10-06T05:25:57.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.014+0000 7fe52ffff700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe51c066f30 0x7fe51c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.015+0000 7fe52ffff700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe51c066f30 0x7fe51c0693e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe5200093f0 tx=0x7fe520017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.018+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5280505a0 con 0x7fe530155af0 2023-10-06T05:25:57.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.222+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe53004c380 con 0x7fe530155af0 2023-10-06T05:25:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.223+0000 7fe517fff700 1 -- 172.21.15.33:0/1685415214 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16941 (secure 0 0 0) 0x7fe528053f90 con 0x7fe530155af0 2023-10-06T05:25:57.225 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:25:57.225 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":157,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:25:13.046893+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:59.024220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:00.278297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1400837691":"2023-10-06T06:25:12.317114+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe51c066f30 msgr2=0x7fe51c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe51c066f30 0x7fe51c0693e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe5200093f0 tx=0x7fe520017000 comp rx=0 tx=0).stop 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 msgr2=0x7fe5300cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 0x7fe5300cac80 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe528010870 tx=0x7fe52800ba50 comp rx=0 tx=0).stop 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 shutdown_connections 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe51c066f30 0x7fe51c0693e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe530155af0 0x7fe5300cac80 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe53006a6a0 0x7fe530155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 --2- 172.21.15.33:0/1685415214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5300c7920 0x7fe5300b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 >> 172.21.15.33:0/1685415214 conn(0x7fe53006d030 msgr2=0x7fe5300cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 shutdown_connections 2023-10-06T05:25:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.226+0000 7fe53519d700 1 -- 172.21.15.33:0/1685415214 wait complete. 2023-10-06T05:25:57.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.431+0000 7f657e59d700 1 Processor -- start 2023-10-06T05:25:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.431+0000 7f657e59d700 1 -- start start 2023-10-06T05:25:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657e59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f6578156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657e59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6578159e10 0x7f6578157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657e59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657806a820 con 0x7f6578156880 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657e59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65780cc290 con 0x7f6578159e10 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657e59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657806bd10 con 0x7f65780c8090 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657d59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6578156880 0x7f65780c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60274/0 (socket says 172.21.15.33:60274) 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657d59b700 1 -- 172.21.15.33:0/3529623369 learned_addr learned my addr 172.21.15.33:0/3529623369 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657dd9c700 1 --2- 172.21.15.33:0/3529623369 >> v2:172.21.15.33:3300/0 conn(0x7f6578159e10 0x7f6578157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f657cd9a700 1 --2- 172.21.15.33:0/3529623369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f6578156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f656ffff700 1 -- 172.21.15.33:0/3529623369 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039363108 0 0) 0x7f657806a820 con 0x7f6578156880 2023-10-06T05:25:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.433+0000 7f656ffff700 1 -- 172.21.15.33:0/3529623369 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6564003570 con 0x7f6578156880 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f657dd9c700 1 -- 172.21.15.33:0/3529623369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 msgr2=0x7f6578156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f657dd9c700 1 --2- 172.21.15.33:0/3529623369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f6578156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f657dd9c700 1 -- 172.21.15.33:0/3529623369 >> v1:172.21.15.33:6789/0 conn(0x7f6578156880 legacy=0x7f65780c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f657dd9c700 1 -- 172.21.15.33:0/3529623369 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657815a2f0 con 0x7f6578159e10 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f657dd9c700 1 --2- 172.21.15.33:0/3529623369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f6578157190 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f6570009e60 tx=0x7f6570009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa013f28e7e931e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f656ffff700 1 -- 172.21.15.33:0/3529623369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000e070 con 0x7f6578159e10 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f656ffff700 1 -- 172.21.15.33:0/3529623369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657000bab0 con 0x7f6578159e10 2023-10-06T05:25:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.434+0000 7f656ffff700 1 -- 172.21.15.33:0/3529623369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000be70 con 0x7f6578159e10 2023-10-06T05:25:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 -- 172.21.15.33:0/3529623369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 msgr2=0x7f6578157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 --2- 172.21.15.33:0/3529623369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f6578157190 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f6570009e60 tx=0x7f6570009780 comp rx=0 tx=0).stop 2023-10-06T05:25:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 -- 172.21.15.33:0/3529623369 shutdown_connections 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 --2- 172.21.15.33:0/3529623369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f6578157190 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 --2- 172.21.15.33:0/3529623369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f6578156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 -- 172.21.15.33:0/3529623369 >> 172.21.15.33:0/3529623369 conn(0x7f657806c560 msgr2=0x7f657806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 -- 172.21.15.33:0/3529623369 shutdown_connections 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.435+0000 7f657e59d700 1 -- 172.21.15.33:0/3529623369 wait complete. 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.436+0000 7f657e59d700 1 Processor -- start 2023-10-06T05:25:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.436+0000 7f657e59d700 1 -- start start 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.437+0000 7f657e59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f65780b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.437+0000 7f657e59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6578156880 0x7f657816cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.437+0000 7f657e59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f657816d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.437+0000 7f657e59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6578149100 con 0x7f6578159e10 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.437+0000 7f657e59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657814ab20 con 0x7f6578156880 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.437+0000 7f657e59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657814ae00 con 0x7f65780c8090 2023-10-06T05:25:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f657816d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f657816d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42288/0 (socket says 172.21.15.33:42288) 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 -- 172.21.15.33:0/3063883050 learned_addr learned my addr 172.21.15.33:0/3063883050 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657cd9a700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6578156880 0x7f657816cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657d59b700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f65780b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 -- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 msgr2=0x7f65780b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f65780b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 -- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6578156880 msgr2=0x7f657816cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6578156880 0x7f657816cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 -- 172.21.15.33:0/3063883050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009a20 con 0x7f6578159e10 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.438+0000 7f657dd9c700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f657816d0b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f65780636f0 tx=0x7f6570019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.439+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570027470 con 0x7f6578159e10 2023-10-06T05:25:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.439+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65700275f0 con 0x7f6578159e10 2023-10-06T05:25:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.439+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700279b0 con 0x7f6578159e10 2023-10-06T05:25:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.439+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657814b0e0 con 0x7f6578159e10 2023-10-06T05:25:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.439+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657814b6a0 con 0x7f6578159e10 2023-10-06T05:25:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.440+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657814b8e0 con 0x7f6578159e10 2023-10-06T05:25:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.441+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6570024070 con 0x7f6578159e10 2023-10-06T05:25:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.441+0000 7f656d7fa700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f655c066f30 0x7f655c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.441+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6570087120 con 0x7f6578159e10 2023-10-06T05:25:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.442+0000 7f657d59b700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f655c066f30 0x7f655c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.443+0000 7f657d59b700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f655c066f30 0x7f655c0693e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f657806a2c0 tx=0x7f6574009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.446+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f657814b8e0 con 0x7f6578159e10 2023-10-06T05:25:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.693+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f657814b8e0 con 0x7f6578159e10 2023-10-06T05:25:57.695 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:25:57.694+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.698+0000 7f656d7fa700 1 -- 172.21.15.33:0/3063883050 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) v1 ==== 161+0+0 (secure 0 0 0) 0x7f657814b8e0 con 0x7f6578159e10 2023-10-06T05:25:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f655c066f30 msgr2=0x7f655c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f655c066f30 0x7f655c0693e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f657806a2c0 tx=0x7f6574009040 comp rx=0 tx=0).stop 2023-10-06T05:25:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 msgr2=0x7f657816d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f657816d0b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f65780636f0 tx=0x7f6570019000 comp rx=0 tx=0).stop 2023-10-06T05:25:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 shutdown_connections 2023-10-06T05:25:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f655c066f30 0x7f655c0693e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6578159e10 0x7f657816d0b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65780c8090 0x7f65780b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 --2- 172.21.15.33:0/3063883050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6578156880 0x7f657816cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 >> 172.21.15.33:0/3063883050 conn(0x7f657806c560 msgr2=0x7f65780c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 shutdown_connections 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.701+0000 7f657e59d700 1 -- 172.21.15.33:0/3063883050 wait complete. 2023-10-06T05:25:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:57.709 INFO:tasks.ceph.mds.b.smithi167.stderr: -4016> 2023-10-06T05:25:07.346+0000 7f6a7ebf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:57.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:25:57.755 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:25:57.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.913+0000 7fc62759e700 1 Processor -- start 2023-10-06T05:25:57.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.913+0000 7fc62759e700 1 -- start start 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc62759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc628156370 0x7fc6280af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc62759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc6280cc7e0 0x7fc6280c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc62759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6280ad430 con 0x7fc6280b01f0 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc62759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6280afe20 con 0x7fc6280cc7e0 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc62759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62806bc70 con 0x7fc628156370 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc626d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc6280b01f0 0x7fc6280ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60276/0 (socket says 172.21.15.33:60276) 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc626d9d700 1 -- 172.21.15.33:0/218893669 learned_addr learned my addr 172.21.15.33:0/218893669 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc625d9b700 1 --2- 172.21.15.33:0/218893669 >> v2:172.21.15.33:3300/0 conn(0x7fc6280cc7e0 0x7fc6280c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.915+0000 7fc62659c700 1 --2- 172.21.15.33:0/218893669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc628156370 0x7fc6280af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc62559a700 1 -- 172.21.15.33:0/218893669 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22890156 0 0) 0x7fc6280ad430 con 0x7fc6280b01f0 2023-10-06T05:25:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc62559a700 1 -- 172.21.15.33:0/218893669 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610003570 con 0x7fc6280b01f0 2023-10-06T05:25:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc625d9b700 1 -- 172.21.15.33:0/218893669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc628156370 msgr2=0x7fc6280af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc625d9b700 1 --2- 172.21.15.33:0/218893669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc628156370 0x7fc6280af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc625d9b700 1 -- 172.21.15.33:0/218893669 >> v1:172.21.15.33:6789/0 conn(0x7fc6280b01f0 legacy=0x7fc6280ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc625d9b700 1 -- 172.21.15.33:0/218893669 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280c92f0 con 0x7fc6280cc7e0 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc625d9b700 1 --2- 172.21.15.33:0/218893669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6280cc7e0 0x7fc6280c8db0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098e0 tx=0x7fc61c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9a1092a70784a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc62559a700 1 -- 172.21.15.33:0/218893669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011630 con 0x7fc6280cc7e0 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc62559a700 1 -- 172.21.15.33:0/218893669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c011790 con 0x7fc6280cc7e0 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.916+0000 7fc62559a700 1 -- 172.21.15.33:0/218893669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011b30 con 0x7fc6280cc7e0 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 -- 172.21.15.33:0/218893669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6280cc7e0 msgr2=0x7fc6280c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 --2- 172.21.15.33:0/218893669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6280cc7e0 0x7fc6280c8db0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098e0 tx=0x7fc61c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 -- 172.21.15.33:0/218893669 shutdown_connections 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 --2- 172.21.15.33:0/218893669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc628156370 0x7fc6280af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 --2- 172.21.15.33:0/218893669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6280cc7e0 0x7fc6280c8db0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 -- 172.21.15.33:0/218893669 >> 172.21.15.33:0/218893669 conn(0x7fc62806d030 msgr2=0x7fc6280cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 -- 172.21.15.33:0/218893669 shutdown_connections 2023-10-06T05:25:57.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.917+0000 7fc62759e700 1 -- 172.21.15.33:0/218893669 wait complete. 2023-10-06T05:25:57.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.918+0000 7fc62759e700 1 Processor -- start 2023-10-06T05:25:57.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.918+0000 7fc62759e700 1 -- start start 2023-10-06T05:25:57.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.919+0000 7fc62759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6280b01f0 0x7fc628158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.919+0000 7fc62759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6280cc7e0 0x7fc628152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.919+0000 7fc62759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 0x7fc628151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.919+0000 7fc62759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628152c40 con 0x7fc628156370 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.919+0000 7fc62759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628153a80 con 0x7fc6280b01f0 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.919+0000 7fc62759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc628152e70 con 0x7fc6280cc7e0 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 0x7fc628151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 0x7fc628151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42310/0 (socket says 172.21.15.33:42310) 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 -- 172.21.15.33:0/2151694223 learned_addr learned my addr 172.21.15.33:0/2151694223 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc62659c700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6280b01f0 0x7fc628158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc625d9b700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6280cc7e0 0x7fc628152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 -- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6280cc7e0 msgr2=0x7fc628152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6280cc7e0 0x7fc628152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 -- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6280b01f0 msgr2=0x7fc628158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6280b01f0 0x7fc628158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 -- 172.21.15.33:0/2151694223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c009420 con 0x7fc628156370 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc626d9d700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 0x7fc628151a20 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc620001460 tx=0x7fc620001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.920+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62000a890 con 0x7fc628156370 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.921+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62000a9f0 con 0x7fc628156370 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.921+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628153150 con 0x7fc628156370 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.921+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62000ad70 con 0x7fc628156370 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.921+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc628153710 con 0x7fc628156370 2023-10-06T05:25:57.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.922+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc628153c10 con 0x7fc628156370 2023-10-06T05:25:57.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.922+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc620007300 con 0x7fc628156370 2023-10-06T05:25:57.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.923+0000 7fc6167fc700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc608066f30 0x7fc6080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.923+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc620084c30 con 0x7fc628156370 2023-10-06T05:25:57.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.924+0000 7fc62659c700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc608066f30 0x7fc6080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.924+0000 7fc62659c700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc608066f30 0x7fc6080693e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc618009d10 tx=0x7fc6180091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:57.927+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc620051c10 con 0x7fc628156370 2023-10-06T05:25:58.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.174+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc62804c380 con 0x7fc628156370 2023-10-06T05:25:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.700+0000 7fc6167fc700 1 -- 172.21.15.33:0/2151694223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc620055230 con 0x7fc628156370 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc608066f30 msgr2=0x7fc6080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc608066f30 0x7fc6080693e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc618009d10 tx=0x7fc6180091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 msgr2=0x7fc628151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 0x7fc628151a20 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc620001460 tx=0x7fc620001ae0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 shutdown_connections 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc608066f30 0x7fc6080693e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc628156370 0x7fc628151a20 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6280b01f0 0x7fc628158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 --2- 172.21.15.33:0/2151694223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6280cc7e0 0x7fc628152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 >> 172.21.15.33:0/2151694223 conn(0x7fc62806d030 msgr2=0x7fc6280cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 shutdown_connections 2023-10-06T05:25:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.703+0000 7fc62759e700 1 -- 172.21.15.33:0/2151694223 wait complete. 2023-10-06T05:25:58.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:25:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.919+0000 7fc0ed99d700 1 Processor -- start 2023-10-06T05:25:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.919+0000 7fc0ed99d700 1 -- start start 2023-10-06T05:25:58.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.920+0000 7fc0ed99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e81597f0 0x7fc0e815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.920+0000 7fc0ed99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e815d280 con 0x7fc0e806dc00 2023-10-06T05:25:58.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.920+0000 7fc0ed99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e815de00 con 0x7fc0e806dfd0 2023-10-06T05:25:58.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.920+0000 7fc0ed99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e815e290 con 0x7fc0e81597f0 2023-10-06T05:25:58.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.920+0000 7fc0ec99b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc0e806dc00 0x7fc0e81590e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60282/0 (socket says 172.21.15.33:60282) 2023-10-06T05:25:58.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.920+0000 7fc0ec99b700 1 -- 172.21.15.33:0/2646428260 learned_addr learned my addr 172.21.15.33:0/2646428260 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e7fff700 1 --2- 172.21.15.33:0/2646428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e81597f0 0x7fc0e815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e77fe700 1 -- 172.21.15.33:0/2646428260 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283246243 0 0) 0x7fc0e815d280 con 0x7fc0e806dc00 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e77fe700 1 -- 172.21.15.33:0/2646428260 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d8003570 con 0x7fc0e806dc00 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e77fe700 1 -- 172.21.15.33:0/2646428260 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245350175 0 0) 0x7fc0e815de00 con 0x7fc0e806dfd0 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e7fff700 1 -- 172.21.15.33:0/2646428260 >> v1:172.21.15.167:6789/0 conn(0x7fc0e806dfd0 legacy=0x7fc0e815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e7fff700 1 -- 172.21.15.33:0/2646428260 >> v1:172.21.15.33:6789/0 conn(0x7fc0e806dc00 legacy=0x7fc0e81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e7fff700 1 -- 172.21.15.33:0/2646428260 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e815d4d0 con 0x7fc0e81597f0 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.921+0000 7fc0e7fff700 1 --2- 172.21.15.33:0/2646428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e81597f0 0x7fc0e815a030 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009910 tx=0x7fc0d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e852638950c3a10a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0e77fe700 1 -- 172.21.15.33:0/2646428260 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc001470 con 0x7fc0e81597f0 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0e77fe700 1 -- 172.21.15.33:0/2646428260 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4011420 con 0x7fc0e81597f0 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0e77fe700 1 -- 172.21.15.33:0/2646428260 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40117c0 con 0x7fc0e81597f0 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 -- 172.21.15.33:0/2646428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e81597f0 msgr2=0x7fc0e815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/2646428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e81597f0 0x7fc0e815a030 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009910 tx=0x7fc0d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 -- 172.21.15.33:0/2646428260 shutdown_connections 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/2646428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e81597f0 0x7fc0e815a030 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 -- 172.21.15.33:0/2646428260 >> 172.21.15.33:0/2646428260 conn(0x7fc0e80d0550 msgr2=0x7fc0e80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 -- 172.21.15.33:0/2646428260 shutdown_connections 2023-10-06T05:25:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.922+0000 7fc0ed99d700 1 -- 172.21.15.33:0/2646428260 wait complete. 2023-10-06T05:25:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.924+0000 7fc0ed99d700 1 Processor -- start 2023-10-06T05:25:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.924+0000 7fc0ed99d700 1 -- start start 2023-10-06T05:25:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0ed99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e806dc00 0x7fc0e815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0ed99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 0x7fc0e80b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0ed99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0e81597f0 0x7fc0e80b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0ed99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e80b1550 con 0x7fc0e806dfd0 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0ed99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e80b1f20 con 0x7fc0e81597f0 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0ed99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e8157a80 con 0x7fc0e806dc00 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 0x7fc0e80b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 0x7fc0e80b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42312/0 (socket says 172.21.15.33:42312) 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.925+0000 7fc0e7fff700 1 -- 172.21.15.33:0/3333174912 learned_addr learned my addr 172.21.15.33:0/3333174912 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0ec99b700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e806dc00 0x7fc0e815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0ed19c700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0e81597f0 0x7fc0e80b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e7fff700 1 -- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e806dc00 msgr2=0x7fc0e815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e7fff700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e806dc00 0x7fc0e815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e7fff700 1 -- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0e81597f0 msgr2=0x7fc0e80b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e7fff700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0e81597f0 0x7fc0e80b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e7fff700 1 -- 172.21.15.33:0/3333174912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d4009420 con 0x7fc0e806dfd0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e7fff700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 0x7fc0e80b2250 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80894b0 tx=0x7fc0d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010c20 con 0x7fc0e806dfd0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4010d80 con 0x7fc0e806dfd0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e8157d60 con 0x7fc0e806dfd0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.926+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d401d660 con 0x7fc0e806dfd0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.927+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e814f300 con 0x7fc0e806dfd0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.927+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e8063ff0 con 0x7fc0e806dfd0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.928+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0d401da50 con 0x7fc0e806dfd0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.929+0000 7fc0e4ff9700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0cc066f30 0x7fc0cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.929+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc0d4085f50 con 0x7fc0e806dfd0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.930+0000 7fc0ec99b700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0cc066f30 0x7fc0cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.930+0000 7fc0ec99b700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0cc066f30 0x7fc0cc0693e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc001200 tx=0x7fc0dc000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:58.934+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0d4052f40 con 0x7fc0e806dfd0 2023-10-06T05:25:59.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.136+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0e80b1690 con 0x7fc0e806dfd0 2023-10-06T05:25:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.878+0000 7fc0e4ff9700 1 -- 172.21.15.33:0/3333174912 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v159) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc0d4056560 con 0x7fc0e806dfd0 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0cc066f30 msgr2=0x7fc0cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0cc066f30 0x7fc0cc0693e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc001200 tx=0x7fc0dc000bc0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 msgr2=0x7fc0e80b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 0x7fc0e80b2250 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80894b0 tx=0x7fc0d4017000 comp rx=0 tx=0).stop 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 shutdown_connections 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0cc066f30 0x7fc0cc0693e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0e81597f0 0x7fc0e80b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0e806dc00 0x7fc0e815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 --2- 172.21.15.33:0/3333174912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0e806dfd0 0x7fc0e80b2250 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 >> 172.21.15.33:0/3333174912 conn(0x7fc0e80d0550 msgr2=0x7fc0e80d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:59.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.885+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 shutdown_connections 2023-10-06T05:25:59.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:25:59.886+0000 7fc0ed99d700 1 -- 172.21.15.33:0/3333174912 wait complete. 2023-10-06T05:25:59.888 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:25:59.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:26:00.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.104+0000 7fb22999d700 1 Processor -- start 2023-10-06T05:26:00.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.104+0000 7fb22999d700 1 -- start start 2023-10-06T05:26:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.105+0000 7fb22999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22406dc00 0x7fb2241590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.105+0000 7fb22999d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb22406dfd0 0x7fb224159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.105+0000 7fb22999d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22415d280 con 0x7fb22415a030 2023-10-06T05:26:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.105+0000 7fb22999d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22415e290 con 0x7fb22406dfd0 2023-10-06T05:26:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.105+0000 7fb22999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22415ab90 con 0x7fb22406dc00 2023-10-06T05:26:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb22406dfd0 0x7fb224159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb22899b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22406dc00 0x7fb2241590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb22406dfd0 0x7fb224159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43058/0 (socket says 172.21.15.33:43058) 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 -- 172.21.15.33:0/3519819026 learned_addr learned my addr 172.21.15.33:0/3519819026 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb22919c700 1 --1- 172.21.15.33:0/3519819026 >> v1:172.21.15.167:6789/0 conn(0x7fb22415a030 0x7fb22415a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47742/0 (socket says 172.21.15.33:47742) 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 -- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22406dc00 msgr2=0x7fb2241590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 --2- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22406dc00 0x7fb2241590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 -- 172.21.15.33:0/3519819026 >> v1:172.21.15.167:6789/0 conn(0x7fb22415a030 legacy=0x7fb22415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb223fff700 1 -- 172.21.15.33:0/3519819026 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22415de00 con 0x7fb22406dfd0 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.106+0000 7fb2237fe700 1 -- 172.21.15.33:0/3519819026 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565519529 0 0) 0x7fb22415d280 con 0x7fb22415a030 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.107+0000 7fb223fff700 1 --2- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dfd0 0x7fb224159620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb2100099e0 tx=0x7fb210009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7edef531c3ed40d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.107+0000 7fb2237fe700 1 -- 172.21.15.33:0/3519819026 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000b820 con 0x7fb22406dfd0 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.107+0000 7fb2237fe700 1 -- 172.21.15.33:0/3519819026 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21000b980 con 0x7fb22406dfd0 2023-10-06T05:26:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.107+0000 7fb2237fe700 1 -- 172.21.15.33:0/3519819026 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000bd20 con 0x7fb22406dfd0 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.107+0000 7fb22999d700 1 -- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dfd0 msgr2=0x7fb224159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.107+0000 7fb22999d700 1 --2- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dfd0 0x7fb224159620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb2100099e0 tx=0x7fb210009250 comp rx=0 tx=0).stop 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.108+0000 7fb22999d700 1 -- 172.21.15.33:0/3519819026 shutdown_connections 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.108+0000 7fb22999d700 1 --2- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22406dc00 0x7fb2241590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.108+0000 7fb22999d700 1 --2- 172.21.15.33:0/3519819026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dfd0 0x7fb224159620 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.108+0000 7fb22999d700 1 -- 172.21.15.33:0/3519819026 >> 172.21.15.33:0/3519819026 conn(0x7fb2240d0550 msgr2=0x7fb2240d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.108+0000 7fb22999d700 1 -- 172.21.15.33:0/3519819026 shutdown_connections 2023-10-06T05:26:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.108+0000 7fb22999d700 1 -- 172.21.15.33:0/3519819026 wait complete. 2023-10-06T05:26:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.109+0000 7fb22999d700 1 Processor -- start 2023-10-06T05:26:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.109+0000 7fb22999d700 1 -- start start 2023-10-06T05:26:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb22999d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dc00 0x7fb22414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb22999d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 0x7fb2240b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb22999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22415a030 0x7fb224155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb22999d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240c8430 con 0x7fb22406dfd0 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb22999d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240b0d10 con 0x7fb22406dc00 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb22999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240b0ff0 con 0x7fb22415a030 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb223fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 0x7fb2240b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb223fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 0x7fb2240b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58938/0 (socket says 172.21.15.33:58938) 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.110+0000 7fb223fff700 1 -- 172.21.15.33:0/1878743699 learned_addr learned my addr 172.21.15.33:0/1878743699 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb22899b700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dc00 0x7fb22414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb22919c700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22415a030 0x7fb224155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb223fff700 1 -- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22415a030 msgr2=0x7fb224155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb223fff700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22415a030 0x7fb224155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb223fff700 1 -- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dc00 msgr2=0x7fb22414cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb223fff700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dc00 0x7fb22414cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb223fff700 1 -- 172.21.15.33:0/1878743699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100094f0 con 0x7fb22406dfd0 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb223fff700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 0x7fb2240b0330 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb2240894b0 tx=0x7fb210017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210011d70 con 0x7fb22406dfd0 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210024440 con 0x7fb22406dfd0 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.111+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210024800 con 0x7fb22406dfd0 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.112+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2240b12d0 con 0x7fb22406dfd0 2023-10-06T05:26:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.112+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240c7fc0 con 0x7fb22406dfd0 2023-10-06T05:26:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.113+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2240c85a0 con 0x7fb22406dfd0 2023-10-06T05:26:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.113+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb21002d440 con 0x7fb22406dfd0 2023-10-06T05:26:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.114+0000 7fb220ff9700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb208066f30 0x7fb2080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.114+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fb210085ec0 con 0x7fb22406dfd0 2023-10-06T05:26:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.115+0000 7fb22899b700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb208066f30 0x7fb2080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.116+0000 7fb22899b700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb208066f30 0x7fb2080693e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb218009d10 tx=0x7fb2180091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.118+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb210053060 con 0x7fb22406dfd0 2023-10-06T05:26:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.320+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb224063ff0 con 0x7fb22406dfd0 2023-10-06T05:26:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.889+0000 7fb220ff9700 1 -- 172.21.15.33:0/1878743699 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v160) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb210056680 con 0x7fb22406dfd0 2023-10-06T05:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb208066f30 msgr2=0x7fb2080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb208066f30 0x7fb2080693e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb218009d10 tx=0x7fb2180091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 msgr2=0x7fb2240b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 0x7fb2240b0330 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb2240894b0 tx=0x7fb210017000 comp rx=0 tx=0).stop 2023-10-06T05:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 shutdown_connections 2023-10-06T05:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb208066f30 0x7fb2080693e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22415a030 0x7fb224155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb22406dc00 0x7fb22414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 --2- 172.21.15.33:0/1878743699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22406dfd0 0x7fb2240b0330 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.893+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 >> 172.21.15.33:0/1878743699 conn(0x7fb2240d0550 msgr2=0x7fb224156890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.895+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 shutdown_connections 2023-10-06T05:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:00.895+0000 7fb22999d700 1 -- 172.21.15.33:0/1878743699 wait complete. 2023-10-06T05:26:00.898 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:26:00.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:26:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.109+0000 7f780bfff700 1 Processor -- start 2023-10-06T05:26:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.109+0000 7f780bfff700 1 -- start start 2023-10-06T05:26:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f780c0af540 0x7f780c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c15d660 con 0x7f780c0cced0 2023-10-06T05:26:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0ce890 con 0x7f780c0af540 2023-10-06T05:26:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c15aac0 con 0x7f780c15a650 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f780c0cced0 0x7f780c06c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40812/0 (socket says 172.21.15.33:40812) 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780b7fe700 1 -- 172.21.15.33:0/543110640 learned_addr learned my addr 172.21.15.33:0/543110640 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.110+0000 7f780affd700 1 --2- 172.21.15.33:0/543110640 >> v2:172.21.15.33:3300/0 conn(0x7f780c0af540 0x7f780c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f780a7fc700 1 --2- 172.21.15.33:0/543110640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f7809ffb700 1 -- 172.21.15.33:0/543110640 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673810577 0 0) 0x7f780c15d660 con 0x7f780c0cced0 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f7809ffb700 1 -- 172.21.15.33:0/543110640 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f4003570 con 0x7f780c0cced0 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f780affd700 1 -- 172.21.15.33:0/543110640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 msgr2=0x7f780c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f780affd700 1 --2- 172.21.15.33:0/543110640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f780affd700 1 -- 172.21.15.33:0/543110640 >> v1:172.21.15.33:6789/0 conn(0x7f780c0cced0 legacy=0x7f780c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f780affd700 1 -- 172.21.15.33:0/543110640 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c15d890 con 0x7f780c0af540 2023-10-06T05:26:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.111+0000 7f780affd700 1 --2- 172.21.15.33:0/543110640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c0af910 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098e0 tx=0x7f77fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86eb0547447424 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f7809ffb700 1 -- 172.21.15.33:0/543110640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc011630 con 0x7f780c0af540 2023-10-06T05:26:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f7809ffb700 1 -- 172.21.15.33:0/543110640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc011790 con 0x7f780c0af540 2023-10-06T05:26:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f7809ffb700 1 -- 172.21.15.33:0/543110640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc011b30 con 0x7f780c0af540 2023-10-06T05:26:01.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 -- 172.21.15.33:0/543110640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 msgr2=0x7f780c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 --2- 172.21.15.33:0/543110640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c0af910 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098e0 tx=0x7f77fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 -- 172.21.15.33:0/543110640 shutdown_connections 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 --2- 172.21.15.33:0/543110640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c0af910 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 --2- 172.21.15.33:0/543110640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 -- 172.21.15.33:0/543110640 >> 172.21.15.33:0/543110640 conn(0x7f780c06d9b0 msgr2=0x7f780c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 -- 172.21.15.33:0/543110640 shutdown_connections 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.112+0000 7f780bfff700 1 -- 172.21.15.33:0/543110640 wait complete. 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.114+0000 7f780bfff700 1 Processor -- start 2023-10-06T05:26:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.114+0000 7f780bfff700 1 -- start start 2023-10-06T05:26:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c14c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c0cced0 0x7f780c157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c16a890 con 0x7f780c0af540 2023-10-06T05:26:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c16de30 con 0x7f780c0cced0 2023-10-06T05:26:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c16d9a0 con 0x7f780c15a650 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c14c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c14c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58964/0 (socket says 172.21.15.33:58964) 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 -- 172.21.15.33:0/1173119190 learned_addr learned my addr 172.21.15.33:0/1173119190 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 -- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 msgr2=0x7f780c157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 -- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c0cced0 msgr2=0x7f780c157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c0cced0 0x7f780c157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.116+0000 7f780affd700 1 -- 172.21.15.33:0/1173119190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc009420 con 0x7f780c0af540 2023-10-06T05:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.117+0000 7f780affd700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c14c910 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009ed0 tx=0x7f77fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.117+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc011b10 con 0x7f780c0af540 2023-10-06T05:26:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.117+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc0253f0 con 0x7f780c0af540 2023-10-06T05:26:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.117+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc025790 con 0x7f780c0af540 2023-10-06T05:26:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.117+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c16d070 con 0x7f780c0af540 2023-10-06T05:26:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.117+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c16d630 con 0x7f780c0af540 2023-10-06T05:26:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.119+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c04c380 con 0x7f780c0af540 2023-10-06T05:26:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.119+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77fc025b80 con 0x7f780c0af540 2023-10-06T05:26:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.120+0000 7f77f2ffd700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77f8066f30 0x7f77f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.120+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f77fc085c70 con 0x7f780c0af540 2023-10-06T05:26:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.121+0000 7f780a7fc700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77f8066f30 0x7f77f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.121+0000 7f780a7fc700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77f8066f30 0x7f77f80693e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cde10 tx=0x7f78000091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.124+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77fc052fb0 con 0x7f780c0af540 2023-10-06T05:26:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.328+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f780c15db70 con 0x7f780c0af540 2023-10-06T05:26:01.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.331+0000 7f77f2ffd700 1 -- 172.21.15.33:0/1173119190 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77fc0565d0 con 0x7f780c0af540 2023-10-06T05:26:01.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77f8066f30 msgr2=0x7f77f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77f8066f30 0x7f77f80693e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f780c0cde10 tx=0x7f78000091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 msgr2=0x7f780c14c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c14c910 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009ed0 tx=0x7f77fc017000 comp rx=0 tx=0).stop 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 shutdown_connections 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f77f8066f30 0x7f77f80693e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c15a650 0x7f780c157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0af540 0x7f780c14c910 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 --2- 172.21.15.33:0/1173119190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c0cced0 0x7f780c157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 >> 172.21.15.33:0/1173119190 conn(0x7f780c06d9b0 msgr2=0x7f780c14dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:01.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 shutdown_connections 2023-10-06T05:26:01.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.334+0000 7f780bfff700 1 -- 172.21.15.33:0/1173119190 wait complete. 2023-10-06T05:26:01.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_mount_after_evicted_client' 2023-10-06T05:26:01.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.549+0000 7fc12b19d700 1 Processor -- start 2023-10-06T05:26:01.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.549+0000 7fc12b19d700 1 -- start start 2023-10-06T05:26:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.550+0000 7fc12b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1240cf3b0 0x7fc1241594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.550+0000 7fc12b19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240cbd80 con 0x7fc1240af0a0 2023-10-06T05:26:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.550+0000 7fc12b19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240af5f0 con 0x7fc1240ccfc0 2023-10-06T05:26:01.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.550+0000 7fc12b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1240af8d0 con 0x7fc1240cf3b0 2023-10-06T05:26:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc12a99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc1240af0a0 0x7fc1240b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40822/0 (socket says 172.21.15.33:40822) 2023-10-06T05:26:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc12a99c700 1 -- 172.21.15.33:0/3584000036 learned_addr learned my addr 172.21.15.33:0/3584000036 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:01.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc12999a700 1 --2- 172.21.15.33:0/3584000036 >> v2:172.21.15.33:3300/0 conn(0x7fc1240cf3b0 0x7fc1241594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc129199700 1 -- 172.21.15.33:0/3584000036 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826021993 0 0) 0x7fc1240cbd80 con 0x7fc1240af0a0 2023-10-06T05:26:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc129199700 1 -- 172.21.15.33:0/3584000036 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc110003570 con 0x7fc1240af0a0 2023-10-06T05:26:01.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc12999a700 1 -- 172.21.15.33:0/3584000036 >> v1:172.21.15.167:6789/0 conn(0x7fc1240ccfc0 legacy=0x7fc1240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc12999a700 1 -- 172.21.15.33:0/3584000036 >> v1:172.21.15.33:6789/0 conn(0x7fc1240af0a0 legacy=0x7fc1240b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.551+0000 7fc12999a700 1 -- 172.21.15.33:0/3584000036 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240afbb0 con 0x7fc1240cf3b0 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.552+0000 7fc12999a700 1 --2- 172.21.15.33:0/3584000036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240cf3b0 0x7fc1241594a0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc114009910 tx=0x7fc1140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788d8c08c794d981 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.552+0000 7fc129199700 1 -- 172.21.15.33:0/3584000036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400b640 con 0x7fc1240cf3b0 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.552+0000 7fc129199700 1 -- 172.21.15.33:0/3584000036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400b7a0 con 0x7fc1240cf3b0 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.552+0000 7fc129199700 1 -- 172.21.15.33:0/3584000036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400bb20 con 0x7fc1240cf3b0 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.552+0000 7fc12b19d700 1 -- 172.21.15.33:0/3584000036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240cf3b0 msgr2=0x7fc1241594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.552+0000 7fc12b19d700 1 --2- 172.21.15.33:0/3584000036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240cf3b0 0x7fc1241594a0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc114009910 tx=0x7fc1140091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.553+0000 7fc12b19d700 1 -- 172.21.15.33:0/3584000036 shutdown_connections 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.553+0000 7fc12b19d700 1 --2- 172.21.15.33:0/3584000036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240cf3b0 0x7fc1241594a0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.553+0000 7fc12b19d700 1 -- 172.21.15.33:0/3584000036 >> 172.21.15.33:0/3584000036 conn(0x7fc12406cff0 msgr2=0x7fc12406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:01.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.553+0000 7fc12b19d700 1 -- 172.21.15.33:0/3584000036 shutdown_connections 2023-10-06T05:26:01.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.553+0000 7fc12b19d700 1 -- 172.21.15.33:0/3584000036 wait complete. 2023-10-06T05:26:01.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.554+0000 7fc12b19d700 1 Processor -- start 2023-10-06T05:26:01.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.554+0000 7fc12b19d700 1 -- start start 2023-10-06T05:26:01.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 0x7fc1240b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240ccfc0 0x7fc1240c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1240b32a0 0x7fc1240c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc124159530 con 0x7fc1240af0a0 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12415f4d0 con 0x7fc1240b32a0 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240b4480 con 0x7fc1240ccfc0 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 0x7fc1240b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 0x7fc1240b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58978/0 (socket says 172.21.15.33:58978) 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.555+0000 7fc12a19b700 1 -- 172.21.15.33:0/2942731582 learned_addr learned my addr 172.21.15.33:0/2942731582 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12999a700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240ccfc0 0x7fc1240c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a99c700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1240b32a0 0x7fc1240c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a19b700 1 -- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240ccfc0 msgr2=0x7fc1240c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a19b700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240ccfc0 0x7fc1240c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a19b700 1 -- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1240b32a0 msgr2=0x7fc1240c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a19b700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1240b32a0 0x7fc1240c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a19b700 1 -- 172.21.15.33:0/2942731582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009420 con 0x7fc1240af0a0 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12999a700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240ccfc0 0x7fc1240c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12a19b700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 0x7fc1240b4910 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c590 tx=0x7fc12000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120011590 con 0x7fc1240af0a0 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1200116f0 con 0x7fc1240af0a0 2023-10-06T05:26:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1240b6080 con 0x7fc1240af0a0 2023-10-06T05:26:01.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120011ab0 con 0x7fc1240af0a0 2023-10-06T05:26:01.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.556+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1240b5da0 con 0x7fc1240af0a0 2023-10-06T05:26:01.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.558+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1240b4760 con 0x7fc1240af0a0 2023-10-06T05:26:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.558+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc12001b3b0 con 0x7fc1240af0a0 2023-10-06T05:26:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.559+0000 7fc11a7fc700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc108066f30 0x7fc1080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.559+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc1200838d0 con 0x7fc1240af0a0 2023-10-06T05:26:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.560+0000 7fc12999a700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc108066f30 0x7fc1080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.561+0000 7fc12999a700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc108066f30 0x7fc1080693e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc1240b0250 tx=0x7fc114017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:01.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.563+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc120050cb0 con 0x7fc1240af0a0 2023-10-06T05:26:01.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.844+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_mount_after_evicted_client"]} v 0) v1 -- 0x7fc12404c380 con 0x7fc1240af0a0 2023-10-06T05:26:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.892+0000 7fc11a7fc700 1 -- 172.21.15.33:0/2942731582 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_mount_after_evicted_client"]}]=0 v588) v1 ==== 163+0+0 (secure 0 0 0) 0x7fc120017070 con 0x7fc1240af0a0 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc108066f30 msgr2=0x7fc1080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc108066f30 0x7fc1080693e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc1240b0250 tx=0x7fc114017000 comp rx=0 tx=0).stop 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 msgr2=0x7fc1240b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 0x7fc1240b4910 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c590 tx=0x7fc12000c950 comp rx=0 tx=0).stop 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 shutdown_connections 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc108066f30 0x7fc1080693e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1240b32a0 0x7fc1240c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1240af0a0 0x7fc1240b4910 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 --2- 172.21.15.33:0/2942731582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240ccfc0 0x7fc1240c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 >> 172.21.15.33:0/2942731582 conn(0x7fc12406cff0 msgr2=0x7fc124155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 shutdown_connections 2023-10-06T05:26:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:01.895+0000 7fc12b19d700 1 -- 172.21.15.33:0/2942731582 wait complete. 2023-10-06T05:26:01.914 INFO:tasks.cephfs_test_runner:Test if a new mount of same fs works after client eviction. ... ok 2023-10-06T05:26:01.914 INFO:tasks.cephfs_test_runner:Starting test: test_reconnect_after_blocklisted (tasks.cephfs.test_client_recovery.TestClientRecovery) Test reconnect after blocklisted. 2023-10-06T05:26:01.914 INFO:tasks.cephfs_test_runner:test_reconnect_after_blocklisted (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:26:01.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_after_blocklisted' 2023-10-06T05:26:02.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.100+0000 7f915cd9d700 1 Processor -- start 2023-10-06T05:26:02.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.100+0000 7f915cd9d700 1 -- start start 2023-10-06T05:26:02.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f915cd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9158155af0 0x7f915806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f915cd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91580c7920 0x7f91580c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f915cd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915806ab60 con 0x7f915806a6a0 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f915cd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915806b7e0 con 0x7f91580c7920 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f915cd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915806ae40 con 0x7f9158155af0 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9157fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91580c7920 0x7f91580c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9156ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f915806a6a0 0x7f91580c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40824/0 (socket says 172.21.15.33:40824) 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9156ffd700 1 -- 172.21.15.33:0/2746380929 learned_addr learned my addr 172.21.15.33:0/2746380929 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f91577fe700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9158155af0 0x7f915806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9157fff700 1 -- 172.21.15.33:0/2746380929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9158155af0 msgr2=0x7f915806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9157fff700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9158155af0 0x7f915806a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9157fff700 1 -- 172.21.15.33:0/2746380929 >> v1:172.21.15.33:6789/0 conn(0x7f915806a6a0 legacy=0x7f91580c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f91567fc700 1 -- 172.21.15.33:0/2746380929 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089594788 0 0) 0x7f915806ab60 con 0x7f915806a6a0 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.102+0000 7f9157fff700 1 -- 172.21.15.33:0/2746380929 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915806b120 con 0x7f91580c7920 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.103+0000 7f91577fe700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9158155af0 0x7f915806a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.103+0000 7f9157fff700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91580c7920 0x7f91580c9e20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f915000af70 tx=0x7f915000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f7fce28817af52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.103+0000 7f91567fc700 1 -- 172.21.15.33:0/2746380929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915000f070 con 0x7f91580c7920 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.103+0000 7f91567fc700 1 -- 172.21.15.33:0/2746380929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915000a2e0 con 0x7f91580c7920 2023-10-06T05:26:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.103+0000 7f91567fc700 1 -- 172.21.15.33:0/2746380929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915000cd20 con 0x7f91580c7920 2023-10-06T05:26:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 -- 172.21.15.33:0/2746380929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91580c7920 msgr2=0x7f91580c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91580c7920 0x7f91580c9e20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f915000af70 tx=0x7f915000a890 comp rx=0 tx=0).stop 2023-10-06T05:26:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 -- 172.21.15.33:0/2746380929 shutdown_connections 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91580c7920 0x7f91580c9e20 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2746380929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9158155af0 0x7f915806a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 -- 172.21.15.33:0/2746380929 >> 172.21.15.33:0/2746380929 conn(0x7f915806d030 msgr2=0x7f915806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 -- 172.21.15.33:0/2746380929 shutdown_connections 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.104+0000 7f915cd9d700 1 -- 172.21.15.33:0/2746380929 wait complete. 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.105+0000 7f915cd9d700 1 Processor -- start 2023-10-06T05:26:02.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.105+0000 7f915cd9d700 1 -- start start 2023-10-06T05:26:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.106+0000 7f915cd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 0x7f915814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.106+0000 7f915cd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91580c7920 0x7f91580ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.106+0000 7f915cd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9158155af0 0x7f91580acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.106+0000 7f915cd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580abd20 con 0x7f915806a6a0 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.106+0000 7f915cd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580c50c0 con 0x7f9158155af0 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.106+0000 7f915cd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91580c53a0 con 0x7f91580c7920 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 0x7f915814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 0x7f915814f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58992/0 (socket says 172.21.15.33:58992) 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 -- 172.21.15.33:0/2467169152 learned_addr learned my addr 172.21.15.33:0/2467169152 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f9156ffd700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91580c7920 0x7f91580ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f9157fff700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9158155af0 0x7f91580acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 -- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91580c7920 msgr2=0x7f91580ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91580c7920 0x7f91580ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 -- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9158155af0 msgr2=0x7f91580acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9158155af0 0x7f91580acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 -- 172.21.15.33:0/2467169152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915000ab30 con 0x7f915806a6a0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f91577fe700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 0x7f915814f1d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f914800e8d0 tx=0x7f9148008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.107+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800ee70 con 0x7f915806a6a0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.108+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800c5e0 con 0x7f915806a6a0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.108+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800c980 con 0x7f915806a6a0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.108+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91580c5680 con 0x7f915806a6a0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.108+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580c5c40 con 0x7f915806a6a0 2023-10-06T05:26:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.109+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915804c380 con 0x7f915806a6a0 2023-10-06T05:26:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.109+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9148013410 con 0x7f915806a6a0 2023-10-06T05:26:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.111+0000 7f913f7fe700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9144066f30 0x7f91440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.111+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9148084250 con 0x7f915806a6a0 2023-10-06T05:26:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.111+0000 7f9156ffd700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9144066f30 0x7f91440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.112+0000 7f9156ffd700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9144066f30 0x7f91440693e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f91580abea0 tx=0x7f914c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:02.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.115+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9148051590 con 0x7f915806a6a0 2023-10-06T05:26:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.403+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_after_blocklisted"]} v 0) v1 -- 0x7f915806b400 con 0x7f915806a6a0 2023-10-06T05:26:02.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.894+0000 7f913f7fe700 1 -- 172.21.15.33:0/2467169152 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_after_blocklisted"]}]=0 v589) v1 ==== 167+0+0 (secure 0 0 0) 0x7f9148054bb0 con 0x7f915806a6a0 2023-10-06T05:26:02.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.897+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9144066f30 msgr2=0x7f91440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.897+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9144066f30 0x7f91440693e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f91580abea0 tx=0x7f914c009040 comp rx=0 tx=0).stop 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 msgr2=0x7f915814f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 0x7f915814f1d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f914800e8d0 tx=0x7f9148008040 comp rx=0 tx=0).stop 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 shutdown_connections 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9144066f30 0x7f91440693e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9158155af0 0x7f91580acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915806a6a0 0x7f915814f1d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 --2- 172.21.15.33:0/2467169152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91580c7920 0x7f91580ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 >> 172.21.15.33:0/2467169152 conn(0x7f915806d030 msgr2=0x7f915806e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 shutdown_connections 2023-10-06T05:26:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:02.898+0000 7f915cd9d700 1 -- 172.21.15.33:0/2467169152 wait complete. 2023-10-06T05:26:02.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:26:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.112+0000 7f0c8dd9d700 1 Processor -- start 2023-10-06T05:26:03.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.112+0000 7f0c8dd9d700 1 -- start start 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c8dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c88156370 0x7f0c880af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c8dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c880b01f0 0x7f0c880b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c8dd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880ad430 con 0x7f0c880cc7e0 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c8dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880afe20 con 0x7f0c880b01f0 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c8dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880b0b70 con 0x7f0c88156370 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c8d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c880b01f0 0x7f0c880b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.113+0000 7f0c87fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c880cc7e0 0x7f0c880c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40830/0 (socket says 172.21.15.33:40830) 2023-10-06T05:26:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c87fff700 1 -- 172.21.15.33:0/417487854 learned_addr learned my addr 172.21.15.33:0/417487854 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c8cd9b700 1 --2- 172.21.15.33:0/417487854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c88156370 0x7f0c880af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c8d59c700 1 -- 172.21.15.33:0/417487854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c88156370 msgr2=0x7f0c880af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c8d59c700 1 --2- 172.21.15.33:0/417487854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c88156370 0x7f0c880af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c8d59c700 1 -- 172.21.15.33:0/417487854 >> v1:172.21.15.33:6789/0 conn(0x7f0c880cc7e0 legacy=0x7f0c880c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c8d59c700 1 -- 172.21.15.33:0/417487854 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8806bc70 con 0x7f0c880b01f0 2023-10-06T05:26:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c877fe700 1 -- 172.21.15.33:0/417487854 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128030820 0 0) 0x7f0c880ad430 con 0x7f0c880cc7e0 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.114+0000 7f0c8d59c700 1 --2- 172.21.15.33:0/417487854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880b01f0 0x7f0c880b0630 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009e60 tx=0x7f0c80009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafc48a6261e0737 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c877fe700 1 -- 172.21.15.33:0/417487854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000e070 con 0x7f0c880b01f0 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c877fe700 1 -- 172.21.15.33:0/417487854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8000bb60 con 0x7f0c880b01f0 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c877fe700 1 -- 172.21.15.33:0/417487854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80012430 con 0x7f0c880b01f0 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/417487854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880b01f0 msgr2=0x7f0c880b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/417487854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880b01f0 0x7f0c880b0630 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009e60 tx=0x7f0c80009780 comp rx=0 tx=0).stop 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/417487854 shutdown_connections 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/417487854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880b01f0 0x7f0c880b0630 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/417487854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c88156370 0x7f0c880af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/417487854 >> 172.21.15.33:0/417487854 conn(0x7f0c8806d030 msgr2=0x7f0c880cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/417487854 shutdown_connections 2023-10-06T05:26:03.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.115+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/417487854 wait complete. 2023-10-06T05:26:03.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.117+0000 7f0c8dd9d700 1 Processor -- start 2023-10-06T05:26:03.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.117+0000 7f0c8dd9d700 1 -- start start 2023-10-06T05:26:03.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c880b01f0 0x7f0c881408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 0x7f0c88158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c88156370 0x7f0c88069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88158e90 con 0x7f0c880cc7e0 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88159170 con 0x7f0c88156370 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88159450 con 0x7f0c880b01f0 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 0x7f0c88158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 0x7f0c88158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59018/0 (socket says 172.21.15.33:59018) 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c87fff700 1 -- 172.21.15.33:0/4033252979 learned_addr learned my addr 172.21.15.33:0/4033252979 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8d59c700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c88156370 0x7f0c88069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.118+0000 7f0c8cd9b700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c880b01f0 0x7f0c881408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c87fff700 1 -- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c880b01f0 msgr2=0x7f0c881408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c87fff700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c880b01f0 0x7f0c881408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c87fff700 1 -- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c88156370 msgr2=0x7f0c88069a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c87fff700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c88156370 0x7f0c88069a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c87fff700 1 -- 172.21.15.33:0/4033252979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80009a20 con 0x7f0c880cc7e0 2023-10-06T05:26:03.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c87fff700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 0x7f0c88158440 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c810 tx=0x7f0c74007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74005c00 con 0x7f0c880cc7e0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74005d60 con 0x7f0c880cc7e0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c88159730 con 0x7f0c880cc7e0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74007320 con 0x7f0c880cc7e0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.119+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c88158980 con 0x7f0c880cc7e0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.120+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c88159c50 con 0x7f0c880cc7e0 2023-10-06T05:26:03.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.121+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c7401b400 con 0x7f0c880cc7e0 2023-10-06T05:26:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.122+0000 7f0c84ff9700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c6c066f30 0x7f0c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.122+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0c74083f40 con 0x7f0c880cc7e0 2023-10-06T05:26:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.123+0000 7f0c8cd9b700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c6c066f30 0x7f0c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.123+0000 7f0c8cd9b700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c6c066f30 0x7f0c6c0693e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009ca0 tx=0x7f0c7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.126+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c88159c50 con 0x7f0c880cc7e0 2023-10-06T05:26:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.330+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0c88159c50 con 0x7f0c880cc7e0 2023-10-06T05:26:03.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.336+0000 7f0c84ff9700 1 -- 172.21.15.33:0/4033252979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0c88159c50 con 0x7f0c880cc7e0 2023-10-06T05:26:03.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c6c066f30 msgr2=0x7f0c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c6c066f30 0x7f0c6c0693e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009ca0 tx=0x7f0c7c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:03.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 msgr2=0x7f0c88158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 0x7f0c88158440 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c810 tx=0x7f0c74007db0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 shutdown_connections 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c6c066f30 0x7f0c6c0693e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c88156370 0x7f0c88069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c880b01f0 0x7f0c881408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 --2- 172.21.15.33:0/4033252979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c880cc7e0 0x7f0c88158440 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.338+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 >> 172.21.15.33:0/4033252979 conn(0x7f0c8806d030 msgr2=0x7f0c8815b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.339+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 shutdown_connections 2023-10-06T05:26:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.339+0000 7f0c8dd9d700 1 -- 172.21.15.33:0/4033252979 wait complete. 2023-10-06T05:26:03.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:26:03.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:26:03.438 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:26:03.530 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:26:03.531 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:26:03.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:03.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.717+0000 7f5b9739d700 1 Processor -- start 2023-10-06T05:26:03.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.717+0000 7f5b9739d700 1 -- start start 2023-10-06T05:26:03.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b9739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9013faa0 0x7f5b9013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b9739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b900c7940 0x7f5b900c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b9739d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b900c9cb0 con 0x7f5b9006aad0 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b9739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9006b660 con 0x7f5b900c7940 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b9739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b900ca270 con 0x7f5b9013faa0 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b96b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b9006aad0 0x7f5b9006af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40846/0 (socket says 172.21.15.33:40846) 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b95b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b900c7940 0x7f5b900c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b96b9c700 1 -- 172.21.15.33:0/2794114017 learned_addr learned my addr 172.21.15.33:0/2794114017 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.719+0000 7f5b9639b700 1 --2- 172.21.15.33:0/2794114017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9013faa0 0x7f5b9013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95399700 1 -- 172.21.15.33:0/2794114017 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768623143 0 0) 0x7f5b900c9cb0 con 0x7f5b9006aad0 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95b9a700 1 -- 172.21.15.33:0/2794114017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9013faa0 msgr2=0x7f5b9013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95b9a700 1 --2- 172.21.15.33:0/2794114017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9013faa0 0x7f5b9013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95b9a700 1 -- 172.21.15.33:0/2794114017 >> v1:172.21.15.33:6789/0 conn(0x7f5b9006aad0 legacy=0x7f5b9006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95b9a700 1 -- 172.21.15.33:0/2794114017 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9006b840 con 0x7f5b900c7940 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95b9a700 1 --2- 172.21.15.33:0/2794114017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c7db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5b800098e0 tx=0x7f5b800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc693b20cfbcdd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95399700 1 -- 172.21.15.33:0/2794114017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80009e80 con 0x7f5b900c7940 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95399700 1 -- 172.21.15.33:0/2794114017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8000b860 con 0x7f5b900c7940 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.720+0000 7f5b95399700 1 -- 172.21.15.33:0/2794114017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000bbe0 con 0x7f5b900c7940 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 -- 172.21.15.33:0/2794114017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 msgr2=0x7f5b900c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 --2- 172.21.15.33:0/2794114017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c7db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5b800098e0 tx=0x7f5b800091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 -- 172.21.15.33:0/2794114017 shutdown_connections 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 --2- 172.21.15.33:0/2794114017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9013faa0 0x7f5b9013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 --2- 172.21.15.33:0/2794114017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c7db0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 -- 172.21.15.33:0/2794114017 >> 172.21.15.33:0/2794114017 conn(0x7f5b9006d030 msgr2=0x7f5b9006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 -- 172.21.15.33:0/2794114017 shutdown_connections 2023-10-06T05:26:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.721+0000 7f5b9739d700 1 -- 172.21.15.33:0/2794114017 wait complete. 2023-10-06T05:26:03.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.722+0000 7f5b9739d700 1 Processor -- start 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.723+0000 7f5b9739d700 1 -- start start 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.723+0000 7f5b9739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9006aad0 0x7f5b900c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.723+0000 7f5b9739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b9739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9013faa0 0x7f5b901656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b9739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90159710 con 0x7f5b900c7940 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b9739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b900c43b0 con 0x7f5b9013faa0 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b9739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9015a6c0 con 0x7f5b9006aad0 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59026/0 (socket says 172.21.15.33:59026) 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 -- 172.21.15.33:0/3192167956 learned_addr learned my addr 172.21.15.33:0/3192167956 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b9639b700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9006aad0 0x7f5b900c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b96b9c700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9013faa0 0x7f5b901656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 -- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9006aad0 msgr2=0x7f5b900c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9006aad0 0x7f5b900c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 -- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9013faa0 msgr2=0x7f5b901656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9013faa0 0x7f5b901656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 -- 172.21.15.33:0/3192167956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80009420 con 0x7f5b900c7940 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b9639b700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9006aad0 0x7f5b900c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b96b9c700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9013faa0 0x7f5b901656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.724+0000 7f5b95b9a700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c6e10 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5b900719c0 tx=0x7f5b80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.725+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8001f070 con 0x7f5b900c7940 2023-10-06T05:26:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.725+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80010470 con 0x7f5b900c7940 2023-10-06T05:26:03.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.725+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000bbe0 con 0x7f5b900c7940 2023-10-06T05:26:03.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.725+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b90159900 con 0x7f5b900c7940 2023-10-06T05:26:03.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.725+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b90159ec0 con 0x7f5b900c7940 2023-10-06T05:26:03.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.726+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9004c380 con 0x7f5b900c7940 2023-10-06T05:26:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.727+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b80023410 con 0x7f5b900c7940 2023-10-06T05:26:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.728+0000 7f5b867fc700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.728+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5b80014070 con 0x7f5b900c7940 2023-10-06T05:26:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.728+0000 7f5b9639b700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.729+0000 7f5b9639b700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5b900b1d90 tx=0x7f5b8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.732+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b80054c10 con 0x7f5b900c7940 2023-10-06T05:26:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.983+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b9006bb20 con 0x7f5b900c7940 2023-10-06T05:26:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.983+0000 7f5b867fc700 1 -- 172.21.15.33:0/3192167956 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+3540 (secure 0 0 0) 0x7f5b80058230 con 0x7f5b900c7940 2023-10-06T05:26:03.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:03.985 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3569228460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3569228460},{"type":"v1","addr":"172.21.15.33:6834","nonce":3569228460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":6977,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3195544212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3195544212},{"type":"v1","addr":"172.21.15.167:6836","nonce":3195544212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3480104623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3480104623},{"type":"v1","addr":"172.21.15.167:6837","nonce":3480104623}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":8053,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2454267259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2454267259},{"type":"v1","addr":"172.21.15.33:6835","nonce":2454267259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[]} 2023-10-06T05:26:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 msgr2=0x7f5b740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5b900b1d90 tx=0x7f5b8c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 msgr2=0x7f5b900c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c6e10 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5b900719c0 tx=0x7f5b80019000 comp rx=0 tx=0).stop 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 shutdown_connections 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9013faa0 0x7f5b901656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b9006aad0 0x7f5b900c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 --2- 172.21.15.33:0/3192167956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c7940 0x7f5b900c6e10 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 >> 172.21.15.33:0/3192167956 conn(0x7f5b9006d030 msgr2=0x7f5b900c27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 shutdown_connections 2023-10-06T05:26:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:03.986+0000 7f5b9739d700 1 -- 172.21.15.33:0/3192167956 wait complete. 2023-10-06T05:26:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 116 2023-10-06T05:26:04.005 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:26:04.005 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:26:04.006 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:26:04.006 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:04.006 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:26:04.006 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:26:04.007 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:26:04.007 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:04.007 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:26:04.007 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:26:04.007 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:26:04.008 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:04.008 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:26:04.008 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:26:04.008 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:26:04.009 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:04.019 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:26:04.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:26:04.023 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:26:04.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:26:04.024 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:26:04.024 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:26:04.064 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:26:04.064 DEBUG:teuthology.parallel:result is None 2023-10-06T05:26:04.068 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:26:04.069 DEBUG:teuthology.parallel:result is None 2023-10-06T05:26:04.072 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:26:04.073 DEBUG:teuthology.parallel:result is None 2023-10-06T05:26:04.194 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:26:04.208 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:26:04.208 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:26:04.209 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:26:04.208+0000 7f83e45fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T05:26:05.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T05:26:10.021 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:26:10.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:26:10.028 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:26:10.028 DEBUG:teuthology.parallel:result is None 2023-10-06T05:26:10.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:26:10.171 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:26:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.226+0000 7f6c8259d700 1 Processor -- start 2023-10-06T05:26:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.226+0000 7f6c8259d700 1 -- start start 2023-10-06T05:26:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.227+0000 7f6c8259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.227+0000 7f6c8259d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c06bd10 con 0x7f6c7c156880 2023-10-06T05:26:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.227+0000 7f6c8259d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c06a820 con 0x7f6c7c159e10 2023-10-06T05:26:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.227+0000 7f6c8259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0cc290 con 0x7f6c7c0c8090 2023-10-06T05:26:10.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c8159b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c7c156880 0x7f6c7c0c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50832/0 (socket says 172.21.15.33:50832) 2023-10-06T05:26:10.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c8159b700 1 -- 172.21.15.33:0/2684770518 learned_addr learned my addr 172.21.15.33:0/2684770518 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:10.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c80d9a700 1 --2- 172.21.15.33:0/2684770518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944134850 0 0) 0x7f6c7c06bd10 con 0x7f6c7c156880 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c68003570 con 0x7f6c7c156880 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329863738 0 0) 0x7f6c7c06a820 con 0x7f6c7c159e10 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c7c06bd10 con 0x7f6c7c159e10 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c80d9a700 1 -- 172.21.15.33:0/2684770518 >> v1:172.21.15.167:6789/0 conn(0x7f6c7c159e10 legacy=0x7f6c7c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c80d9a700 1 -- 172.21.15.33:0/2684770518 >> v1:172.21.15.33:6789/0 conn(0x7f6c7c156880 legacy=0x7f6c7c0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330643405 0 0) 0x7f6c68003570 con 0x7f6c7c156880 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.228+0000 7f6c80d9a700 1 -- 172.21.15.33:0/2684770518 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c15a2f0 con 0x7f6c7c0c8090 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.229+0000 7f6c80d9a700 1 --2- 172.21.15.33:0/2684770518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c156c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009910 tx=0x7f6c6c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a83949f1f180876 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.229+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78001470 con 0x7f6c7c0c8090 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.229+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c011420 con 0x7f6c7c0c8090 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.229+0000 7f6c73fff700 1 -- 172.21.15.33:0/2684770518 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0117c0 con 0x7f6c7c0c8090 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 -- 172.21.15.33:0/2684770518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 msgr2=0x7f6c7c156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 --2- 172.21.15.33:0/2684770518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c156c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009910 tx=0x7f6c6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 -- 172.21.15.33:0/2684770518 shutdown_connections 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 --2- 172.21.15.33:0/2684770518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c156c50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 -- 172.21.15.33:0/2684770518 >> 172.21.15.33:0/2684770518 conn(0x7f6c7c06c560 msgr2=0x7f6c7c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 -- 172.21.15.33:0/2684770518 shutdown_connections 2023-10-06T05:26:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.230+0000 7f6c8259d700 1 -- 172.21.15.33:0/2684770518 wait complete. 2023-10-06T05:26:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.231+0000 7f6c8259d700 1 Processor -- start 2023-10-06T05:26:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.231+0000 7f6c8259d700 1 -- start start 2023-10-06T05:26:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.232+0000 7f6c8259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c168db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.232+0000 7f6c8259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7c156880 0x7f6c7c0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.232+0000 7f6c8259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 0x7f6c7c13ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.232+0000 7f6c8259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c169c30 con 0x7f6c7c159e10 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.232+0000 7f6c8259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c169390 con 0x7f6c7c156880 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.232+0000 7f6c8259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c169670 con 0x7f6c7c0c8090 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 0x7f6c7c13ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 0x7f6c7c13ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50804/0 (socket says 172.21.15.33:50804) 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 -- 172.21.15.33:0/1539881151 learned_addr learned my addr 172.21.15.33:0/1539881151 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c8159b700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c80d9a700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7c156880 0x7f6c7c0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 -- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 msgr2=0x7f6c7c168db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c168db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 -- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7c156880 msgr2=0x7f6c7c0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7c156880 0x7f6c7c0b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 -- 172.21.15.33:0/1539881151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009420 con 0x7f6c7c159e10 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.233+0000 7f6c81d9c700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 0x7f6c7c13ba00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c910 tx=0x7f6c740027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.234+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400a7e0 con 0x7f6c7c159e10 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.234+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7400a960 con 0x7f6c7c159e10 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.234+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400ad20 con 0x7f6c7c159e10 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.234+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c169950 con 0x7f6c7c159e10 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.234+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c14b3f0 con 0x7f6c7c159e10 2023-10-06T05:26:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.235+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c0b02d0 con 0x7f6c7c159e10 2023-10-06T05:26:10.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.236+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c74012630 con 0x7f6c7c159e10 2023-10-06T05:26:10.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.236+0000 7f6c717fa700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c60066f30 0x7f6c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:10.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.237+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c74084920 con 0x7f6c7c159e10 2023-10-06T05:26:10.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.237+0000 7f6c8159b700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c60066f30 0x7f6c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:10.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.238+0000 7f6c8159b700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c60066f30 0x7f6c600693e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6c780015a0 tx=0x7f6c78000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:10.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.241+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c7c0b02d0 con 0x7f6c7c159e10 2023-10-06T05:26:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:10.459+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6c7c0b02d0 con 0x7f6c7c159e10 2023-10-06T05:26:11.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.937+0000 7f6c717fa700 1 -- 172.21.15.33:0/1539881151 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v165) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c7c0b02d0 con 0x7f6c7c159e10 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c60066f30 msgr2=0x7f6c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c60066f30 0x7f6c600693e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6c780015a0 tx=0x7f6c78000bc0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 msgr2=0x7f6c7c13ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 0x7f6c7c13ba00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c910 tx=0x7f6c740027a0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 shutdown_connections 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c60066f30 0x7f6c600693e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c159e10 0x7f6c7c13ba00 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7c0c8090 0x7f6c7c168db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 --2- 172.21.15.33:0/1539881151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7c156880 0x7f6c7c0b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 >> 172.21.15.33:0/1539881151 conn(0x7f6c7c06c560 msgr2=0x7f6c7c14ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:11.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 shutdown_connections 2023-10-06T05:26:11.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:11.941+0000 7f6c8259d700 1 -- 172.21.15.33:0/1539881151 wait complete. 2023-10-06T05:26:11.945 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:26:11.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:26:12.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T05:26:12.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.155+0000 7fd32679d700 1 Processor -- start 2023-10-06T05:26:12.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.156+0000 7fd32679d700 1 -- start start 2023-10-06T05:26:12.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.157+0000 7fd32679d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd3200c7920 0x7fd320160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.157+0000 7fd32679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32006ab60 con 0x7fd320155af0 2023-10-06T05:26:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.157+0000 7fd32679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32006ae40 con 0x7fd3200c8b50 2023-10-06T05:26:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.157+0000 7fd32679d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32006b7e0 con 0x7fd3200c7920 2023-10-06T05:26:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.157+0000 7fd32579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd320155af0 0x7fd32006a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50840/0 (socket says 172.21.15.33:50840) 2023-10-06T05:26:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.157+0000 7fd32579b700 1 -- 172.21.15.33:0/2176560486 learned_addr learned my addr 172.21.15.33:0/2176560486 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2176560486 >> v2:172.21.15.167:3300/0 conn(0x7fd3200c7920 0x7fd320160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256048919 0 0) 0x7fd32006ab60 con 0x7fd320155af0 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30c005780 con 0x7fd320155af0 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892638543 0 0) 0x7fd30c005780 con 0x7fd320155af0 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 >> v2:172.21.15.167:3300/0 conn(0x7fd3200c7920 msgr2=0x7fd320160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 --2- 172.21.15.33:0/2176560486 >> v2:172.21.15.167:3300/0 conn(0x7fd3200c7920 0x7fd320160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 >> v1:172.21.15.167:6789/0 conn(0x7fd3200c8b50 legacy=0x7fd3200c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32006b120 con 0x7fd320155af0 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fd31c001470 con 0x7fd320155af0 2023-10-06T05:26:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 >> v1:172.21.15.33:6789/0 conn(0x7fd320155af0 legacy=0x7fd32006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd30c005cc0 0x7fd30c0060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c006a20 0x7fd30c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd30c007420 0x7fd30c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32006b7e0 con 0x7fd30c007420 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c00e3c0 con 0x7fd30c005cc0 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c00e580 con 0x7fd30c006a20 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd30c007420 0x7fd30c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.158+0000 7fd32579b700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd30c005cc0 0x7fd30c0060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd324f9a700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c006a20 0x7fd30c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 -- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c006a20 msgr2=0x7fd30c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c006a20 0x7fd30c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 -- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd30c005cc0 msgr2=0x7fd30c0060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd30c005cc0 0x7fd30c0060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 -- 172.21.15.33:0/2176560486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c005780 con 0x7fd30c007420 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 -- 172.21.15.33:0/2176560486 reap_dead start 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd30c007420 0x7fd30c00dc70 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd3200c7d20 tx=0x7fd31800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61dab4cb33f67ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800f040 con 0x7fd30c007420 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318008b20 con 0x7fd30c007420 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.159+0000 7fd317fff700 1 -- 172.21.15.33:0/2176560486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318015400 con 0x7fd30c007420 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 -- 172.21.15.33:0/2176560486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd30c007420 msgr2=0x7fd30c00dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd30c007420 0x7fd30c00dc70 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd3200c7d20 tx=0x7fd31800c040 comp rx=0 tx=0).stop 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 -- 172.21.15.33:0/2176560486 shutdown_connections 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 --2- 172.21.15.33:0/2176560486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd30c007420 0x7fd30c00dc70 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 -- 172.21.15.33:0/2176560486 >> 172.21.15.33:0/2176560486 conn(0x7fd32006d030 msgr2=0x7fd32006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:12.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 -- 172.21.15.33:0/2176560486 shutdown_connections 2023-10-06T05:26:12.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.160+0000 7fd32679d700 1 -- 172.21.15.33:0/2176560486 wait complete. 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.161+0000 7fd32679d700 1 Processor -- start 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 -- start start 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c007420 0x7fd320166810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320168c40 0x7fd3201690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 0x7fd320157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3201695f0 con 0x7fd320157b00 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200af8e0 con 0x7fd320168c40 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.162+0000 7fd32679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd320166d50 con 0x7fd30c007420 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 0x7fd320157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 0x7fd320157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50818/0 (socket says 172.21.15.33:50818) 2023-10-06T05:26:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 -- 172.21.15.33:0/2124612019 learned_addr learned my addr 172.21.15.33:0/2124612019 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd324f9a700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320168c40 0x7fd3201690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd32579b700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c007420 0x7fd320166810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 -- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c007420 msgr2=0x7fd320166810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c007420 0x7fd320166810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 -- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320168c40 msgr2=0x7fd3201690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320168c40 0x7fd3201690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 -- 172.21.15.33:0/2124612019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320158860 con 0x7fd320157b00 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd324f9a700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320168c40 0x7fd3201690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd32579b700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c007420 0x7fd320166810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd325f9c700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 0x7fd320157fa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd31800bfd0 tx=0x7fd318018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.163+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318013790 con 0x7fd320157b00 2023-10-06T05:26:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.164+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318013910 con 0x7fd320157b00 2023-10-06T05:26:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.164+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318013cd0 con 0x7fd320157b00 2023-10-06T05:26:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.164+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3200aefb0 con 0x7fd320157b00 2023-10-06T05:26:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.164+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3200af570 con 0x7fd320157b00 2023-10-06T05:26:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.165+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3200afa90 con 0x7fd320157b00 2023-10-06T05:26:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.166+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd318003020 con 0x7fd320157b00 2023-10-06T05:26:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.166+0000 7fd3157fa700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd304066f30 0x7fd3040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.167+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd3180857e0 con 0x7fd320157b00 2023-10-06T05:26:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.167+0000 7fd32579b700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd304066f30 0x7fd3040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.167+0000 7fd32579b700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd304066f30 0x7fd3040693e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0015a0 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.171+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3200afa90 con 0x7fd320157b00 2023-10-06T05:26:12.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.450+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd3200afa50 con 0x7fd320157b00 2023-10-06T05:26:12.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.450+0000 7fd3157fa700 1 -- 172.21.15.33:0/2124612019 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd318023090 con 0x7fd320157b00 2023-10-06T05:26:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:12.453 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:26:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:26:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:26:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:26:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:26:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:26:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:26:12.469 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:26:12.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd304066f30 msgr2=0x7fd3040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd304066f30 0x7fd3040693e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0015a0 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 msgr2=0x7fd320157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 0x7fd320157fa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd31800bfd0 tx=0x7fd318018000 comp rx=0 tx=0).stop 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 shutdown_connections 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd304066f30 0x7fd3040693e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd320157b00 0x7fd320157fa0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd30c007420 0x7fd320166810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 --2- 172.21.15.33:0/2124612019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320168c40 0x7fd3201690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 >> 172.21.15.33:0/2124612019 conn(0x7fd32006d030 msgr2=0x7fd320159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:12.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 shutdown_connections 2023-10-06T05:26:12.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.453+0000 7fd32679d700 1 -- 172.21.15.33:0/2124612019 wait complete. 2023-10-06T05:26:12.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.658+0000 7f352bfff700 1 Processor -- start 2023-10-06T05:26:12.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.658+0000 7f352bfff700 1 -- start start 2023-10-06T05:26:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.659+0000 7f352bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f352c0b0a40 0x7f352c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.659+0000 7f352bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c15e2d0 con 0x7f352c0b0e10 2023-10-06T05:26:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.659+0000 7f352bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c15d6c0 con 0x7f352c15a0a0 2023-10-06T05:26:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.659+0000 7f352bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c15d9a0 con 0x7f352c0b0a40 2023-10-06T05:26:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f352a7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f352c0b0e10 0x7f352c0b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50852/0 (socket says 172.21.15.33:50852) 2023-10-06T05:26:12.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f352a7fc700 1 -- 172.21.15.33:0/3458964832 learned_addr learned my addr 172.21.15.33:0/3458964832 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f352affd700 1 --2- 172.21.15.33:0/3458964832 >> v2:172.21.15.167:3300/0 conn(0x7f352c0b0a40 0x7f352c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293285363 0 0) 0x7f352c15e2d0 con 0x7f352c0b0e10 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3514003570 con 0x7f352c0b0e10 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4096211149 0 0) 0x7f3514003570 con 0x7f352c0b0e10 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 >> v2:172.21.15.167:3300/0 conn(0x7f352c0b0a40 msgr2=0x7f352c06ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 --2- 172.21.15.33:0/3458964832 >> v2:172.21.15.167:3300/0 conn(0x7f352c0b0a40 0x7f352c06ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 >> v1:172.21.15.167:6789/0 conn(0x7f352c15a0a0 legacy=0x7f352c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.660+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c15dc80 con 0x7f352c0b0e10 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3520001470 con 0x7f352c0b0e10 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 >> v1:172.21.15.33:6789/0 conn(0x7f352c0b0e10 legacy=0x7f352c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3514003ab0 0x7f3514003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3514004810 0x7f3514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514005210 0x7f351400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c15d9a0 con 0x7f3514003ab0 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351400c1b0 con 0x7f3514004810 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351400c370 con 0x7f3514005210 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352affd700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3514003ab0 0x7f3514003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352a7fc700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3514004810 0x7f3514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352b7fe700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514005210 0x7f351400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352affd700 1 -- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514005210 msgr2=0x7f351400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352affd700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514005210 0x7f351400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352affd700 1 -- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3514004810 msgr2=0x7f3514004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352affd700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3514004810 0x7f3514004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352affd700 1 -- 172.21.15.33:0/3458964832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514003570 con 0x7f3514003ab0 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352b7fe700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514005210 0x7f351400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352a7fc700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3514004810 0x7f3514004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.661+0000 7f352b7fe700 1 -- 172.21.15.33:0/3458964832 reap_dead start 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f352affd700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3514003ab0 0x7f3514003ec0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f351c009c00 tx=0x7f351c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb330085365cb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c00e710 con 0x7f3514003ab0 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c00e870 con 0x7f3514003ab0 2023-10-06T05:26:12.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f3529ffb700 1 -- 172.21.15.33:0/3458964832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c00ec30 con 0x7f3514003ab0 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f352bfff700 1 -- 172.21.15.33:0/3458964832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3514003ab0 msgr2=0x7f3514003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f352bfff700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3514003ab0 0x7f3514003ec0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f351c009c00 tx=0x7f351c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f352bfff700 1 -- 172.21.15.33:0/3458964832 shutdown_connections 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f352bfff700 1 --2- 172.21.15.33:0/3458964832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3514003ab0 0x7f3514003ec0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.662+0000 7f352bfff700 1 -- 172.21.15.33:0/3458964832 >> 172.21.15.33:0/3458964832 conn(0x7f352c06e0f0 msgr2=0x7f352c0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.663+0000 7f352bfff700 1 -- 172.21.15.33:0/3458964832 shutdown_connections 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.663+0000 7f352bfff700 1 -- 172.21.15.33:0/3458964832 wait complete. 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.664+0000 7f352bfff700 1 Processor -- start 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.664+0000 7f352bfff700 1 -- start start 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514003ab0 0x7f352c14dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 0x7f352c0ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c151040 0x7f352c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0ab6e0 con 0x7f352c0aac20 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c16a220 con 0x7f352c151040 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c16a500 con 0x7f3514003ab0 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 0x7f352c0ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 0x7f352c0ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50840/0 (socket says 172.21.15.33:50840) 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.665+0000 7f352a7fc700 1 -- 172.21.15.33:0/547530641 learned_addr learned my addr 172.21.15.33:0/547530641 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352affd700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514003ab0 0x7f352c14dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352b7fe700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c151040 0x7f352c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352a7fc700 1 -- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514003ab0 msgr2=0x7f352c14dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352a7fc700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514003ab0 0x7f352c14dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352a7fc700 1 -- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c151040 msgr2=0x7f352c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352a7fc700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c151040 0x7f352c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352a7fc700 1 -- 172.21.15.33:0/547530641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c16a7e0 con 0x7f352c0aac20 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352a7fc700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 0x7f352c0ab090 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f3520000fd0 tx=0x7f3520001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520005e30 con 0x7f352c0aac20 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352000c7b0 con 0x7f352c0aac20 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352000cb70 con 0x7f352c0aac20 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c16aac0 con 0x7f352c0aac20 2023-10-06T05:26:12.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.666+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c151a00 con 0x7f352c0aac20 2023-10-06T05:26:12.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.667+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c06fcc0 con 0x7f352c0aac20 2023-10-06T05:26:12.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.668+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f352001d460 con 0x7f352c0aac20 2023-10-06T05:26:12.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.670+0000 7f3512ffd700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3518066f30 0x7f35180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.670+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3520084310 con 0x7f352c0aac20 2023-10-06T05:26:12.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.671+0000 7f352affd700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3518066f30 0x7f35180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.671+0000 7f352affd700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3518066f30 0x7f35180693e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f351c0091e0 tx=0x7f351c01a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.674+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35200516d0 con 0x7f352c0aac20 2023-10-06T05:26:12.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.877+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f352c16acb0 con 0x7f352c0aac20 2023-10-06T05:26:12.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.878+0000 7f3512ffd700 1 -- 172.21.15.33:0/547530641 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f3520054cf0 con 0x7f352c0aac20 2023-10-06T05:26:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:12.880 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":165,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:11.932808+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:12.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3518066f30 msgr2=0x7f35180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3518066f30 0x7f35180693e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f351c0091e0 tx=0x7f351c01a000 comp rx=0 tx=0).stop 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 msgr2=0x7f352c0ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 0x7f352c0ab090 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f3520000fd0 tx=0x7f3520001d50 comp rx=0 tx=0).stop 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 shutdown_connections 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3518066f30 0x7f35180693e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f352c151040 0x7f352c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514003ab0 0x7f352c14dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 --2- 172.21.15.33:0/547530641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f352c0aac20 0x7f352c0ab090 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 >> 172.21.15.33:0/547530641 conn(0x7f352c06e0f0 msgr2=0x7f352c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 shutdown_connections 2023-10-06T05:26:12.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:12.880+0000 7f352bfff700 1 -- 172.21.15.33:0/547530641 wait complete. 2023-10-06T05:26:12.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:26:12.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:26:13.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.091+0000 7f6342d9d700 1 Processor -- start 2023-10-06T05:26:13.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.091+0000 7f6342d9d700 1 -- start start 2023-10-06T05:26:13.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.092+0000 7f6342d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f633c0c9490 0x7f633c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.092+0000 7f6342d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0afb20 0x7f633c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.092+0000 7f6342d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0b06b0 con 0x7f633c155a50 2023-10-06T05:26:13.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.092+0000 7f6342d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c0c99e0 con 0x7f633c0c9490 2023-10-06T05:26:13.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.092+0000 7f6342d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0c9cc0 con 0x7f633c0afb20 2023-10-06T05:26:13.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.092+0000 7f634159a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f633c0c9490 0x7f633c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f633c0c9490 0x7f633c155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50850/0 (socket says 172.21.15.33:50850) 2023-10-06T05:26:13.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 -- 172.21.15.33:0/1196074894 learned_addr learned my addr 172.21.15.33:0/1196074894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:13.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634259c700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0afb20 0x7f633c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 -- 172.21.15.33:0/1196074894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0afb20 msgr2=0x7f633c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:13.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0afb20 0x7f633c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 -- 172.21.15.33:0/1196074894 >> v1:172.21.15.167:6789/0 conn(0x7f633c155a50 legacy=0x7f633c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 -- 172.21.15.33:0/1196074894 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0c9fa0 con 0x7f633c0c9490 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634259c700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0afb20 0x7f633c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f634159a700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0c9490 0x7f633c155e20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f632c0099b0 tx=0x7f632c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4448a61269fc83d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.093+0000 7f6340d99700 1 -- 172.21.15.33:0/1196074894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b820 con 0x7f633c0c9490 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6340d99700 1 -- 172.21.15.33:0/1196074894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00b980 con 0x7f633c0c9490 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6340d99700 1 -- 172.21.15.33:0/1196074894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00bd20 con 0x7f633c0c9490 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 -- 172.21.15.33:0/1196074894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0c9490 msgr2=0x7f633c155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0c9490 0x7f633c155e20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f632c0099b0 tx=0x7f632c009250 comp rx=0 tx=0).stop 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 -- 172.21.15.33:0/1196074894 shutdown_connections 2023-10-06T05:26:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0afb20 0x7f633c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 --2- 172.21.15.33:0/1196074894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0c9490 0x7f633c155e20 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 -- 172.21.15.33:0/1196074894 >> 172.21.15.33:0/1196074894 conn(0x7f633c06d030 msgr2=0x7f633c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 -- 172.21.15.33:0/1196074894 shutdown_connections 2023-10-06T05:26:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.094+0000 7f6342d9d700 1 -- 172.21.15.33:0/1196074894 wait complete. 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.096+0000 7f6342d9d700 1 Processor -- start 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.096+0000 7f6342d9d700 1 -- start start 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6342d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 0x7f633c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6342d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0c9490 0x7f633c1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6342d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633c155a50 0x7f633c14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6342d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c152910 con 0x7f633c0afb20 2023-10-06T05:26:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6342d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c151a30 con 0x7f633c155a50 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6342d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c151d00 con 0x7f633c0c9490 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 0x7f633c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 0x7f633c1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50856/0 (socket says 172.21.15.33:50856) 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 -- 172.21.15.33:0/2446090331 learned_addr learned my addr 172.21.15.33:0/2446090331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f634159a700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0c9490 0x7f633c1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f634259c700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633c155a50 0x7f633c14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 -- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0c9490 msgr2=0x7f633c1514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0c9490 0x7f633c1514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 -- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633c155a50 msgr2=0x7f633c14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633c155a50 0x7f633c14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.097+0000 7f6341d9b700 1 -- 172.21.15.33:0/2446090331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0094f0 con 0x7f633c0afb20 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f634159a700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0c9490 0x7f633c1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f634259c700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633c155a50 0x7f633c14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f6341d9b700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 0x7f633c1575a0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f633800c5d0 tx=0x7f6338007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011590 con 0x7f633c0afb20 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6338011710 con 0x7f633c0afb20 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c151fe0 con 0x7f633c0afb20 2023-10-06T05:26:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011ad0 con 0x7f633c0afb20 2023-10-06T05:26:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.098+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1525a0 con 0x7f633c0afb20 2023-10-06T05:26:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.099+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c158e30 con 0x7f633c0afb20 2023-10-06T05:26:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.100+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f633801c460 con 0x7f633c0afb20 2023-10-06T05:26:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.101+0000 7f6331ffb700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6320066f30 0x7f63200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.101+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f63380841e0 con 0x7f633c0afb20 2023-10-06T05:26:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.101+0000 7f634159a700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6320066f30 0x7f63200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.102+0000 7f634159a700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6320066f30 0x7f63200693e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f632c0099b0 tx=0x7f632c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.105+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f633c158e30 con 0x7f633c0afb20 2023-10-06T05:26:13.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.309+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f633c158e30 con 0x7f633c0afb20 2023-10-06T05:26:13.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.310+0000 7f6331ffb700 1 -- 172.21.15.33:0/2446090331 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7f633c158e30 con 0x7f633c0afb20 2023-10-06T05:26:13.312 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6320066f30 msgr2=0x7f63200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6320066f30 0x7f63200693e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f632c0099b0 tx=0x7f632c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 msgr2=0x7f633c1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 0x7f633c1575a0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f633800c5d0 tx=0x7f6338007f60 comp rx=0 tx=0).stop 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 shutdown_connections 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6320066f30 0x7f63200693e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633c155a50 0x7f633c14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c0afb20 0x7f633c1575a0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 --2- 172.21.15.33:0/2446090331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633c0c9490 0x7f633c1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 >> 172.21.15.33:0/2446090331 conn(0x7f633c06d030 msgr2=0x7f633c14d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 shutdown_connections 2023-10-06T05:26:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.312+0000 7f6342d9d700 1 -- 172.21.15.33:0/2446090331 wait complete. 2023-10-06T05:26:13.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:26:13.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.521+0000 7f4c73fff700 1 Processor -- start 2023-10-06T05:26:13.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.521+0000 7f4c73fff700 1 -- start start 2023-10-06T05:26:13.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.522+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c740af540 0x7f4c740af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.522+0000 7f4c73fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4c7415a650 0x7f4c7415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.522+0000 7f4c73fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7415d660 con 0x7f4c740cced0 2023-10-06T05:26:13.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.522+0000 7f4c73fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740ce890 con 0x7f4c7415a650 2023-10-06T05:26:13.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.522+0000 7f4c73fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415aac0 con 0x7f4c740af540 2023-10-06T05:26:13.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c737fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4c740cced0 0x7f4c7406c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50866/0 (socket says 172.21.15.33:50866) 2023-10-06T05:26:13.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c737fe700 1 -- 172.21.15.33:0/4213284867 learned_addr learned my addr 172.21.15.33:0/4213284867 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:13.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c72ffd700 1 --2- 172.21.15.33:0/4213284867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c740af540 0x7f4c740af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c727fc700 1 --2- 172.21.15.33:0/4213284867 >> v2:172.21.15.167:3300/0 conn(0x7f4c7415a650 0x7f4c7415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c71ffb700 1 -- 172.21.15.33:0/4213284867 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570666108 0 0) 0x7f4c7415d660 con 0x7f4c740cced0 2023-10-06T05:26:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c71ffb700 1 -- 172.21.15.33:0/4213284867 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c5c003570 con 0x7f4c740cced0 2023-10-06T05:26:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c72ffd700 1 -- 172.21.15.33:0/4213284867 >> v2:172.21.15.167:3300/0 conn(0x7f4c7415a650 msgr2=0x7f4c7415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c72ffd700 1 --2- 172.21.15.33:0/4213284867 >> v2:172.21.15.167:3300/0 conn(0x7f4c7415a650 0x7f4c7415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c72ffd700 1 -- 172.21.15.33:0/4213284867 >> v1:172.21.15.33:6789/0 conn(0x7f4c740cced0 legacy=0x7f4c7406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c72ffd700 1 -- 172.21.15.33:0/4213284867 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7415d890 con 0x7f4c740af540 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.523+0000 7f4c727fc700 1 --2- 172.21.15.33:0/4213284867 >> v2:172.21.15.167:3300/0 conn(0x7f4c7415a650 0x7f4c7415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.524+0000 7f4c72ffd700 1 --2- 172.21.15.33:0/4213284867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c740af540 0x7f4c740af910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c640098e0 tx=0x7f4c640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362d326c49ded3bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.524+0000 7f4c71ffb700 1 -- 172.21.15.33:0/4213284867 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011630 con 0x7f4c740af540 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.524+0000 7f4c71ffb700 1 -- 172.21.15.33:0/4213284867 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64011790 con 0x7f4c740af540 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.524+0000 7f4c71ffb700 1 -- 172.21.15.33:0/4213284867 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011b10 con 0x7f4c740af540 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.524+0000 7f4c73fff700 1 -- 172.21.15.33:0/4213284867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c740af540 msgr2=0x7f4c740af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.524+0000 7f4c73fff700 1 --2- 172.21.15.33:0/4213284867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c740af540 0x7f4c740af910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c640098e0 tx=0x7f4c640091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.525+0000 7f4c73fff700 1 -- 172.21.15.33:0/4213284867 shutdown_connections 2023-10-06T05:26:13.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.525+0000 7f4c73fff700 1 --2- 172.21.15.33:0/4213284867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c740af540 0x7f4c740af910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.525+0000 7f4c73fff700 1 --2- 172.21.15.33:0/4213284867 >> v2:172.21.15.167:3300/0 conn(0x7f4c7415a650 0x7f4c7415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.525+0000 7f4c73fff700 1 -- 172.21.15.33:0/4213284867 >> 172.21.15.33:0/4213284867 conn(0x7f4c7406d9b0 msgr2=0x7f4c7406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.525+0000 7f4c73fff700 1 -- 172.21.15.33:0/4213284867 shutdown_connections 2023-10-06T05:26:13.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.525+0000 7f4c73fff700 1 -- 172.21.15.33:0/4213284867 wait complete. 2023-10-06T05:26:13.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.526+0000 7f4c73fff700 1 Processor -- start 2023-10-06T05:26:13.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.526+0000 7f4c73fff700 1 -- start start 2023-10-06T05:26:13.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 0x7f4c740b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740cced0 0x7f4c7416de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c7415a650 0x7f4c740b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c73fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7416b110 con 0x7f4c740af540 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c73fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7416a870 con 0x7f4c740cced0 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c73fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7416ab50 con 0x7f4c7415a650 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 0x7f4c740b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 0x7f4c740b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50870/0 (socket says 172.21.15.33:50870) 2023-10-06T05:26:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.527+0000 7f4c72ffd700 1 -- 172.21.15.33:0/2447606229 learned_addr learned my addr 172.21.15.33:0/2447606229 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c727fc700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740cced0 0x7f4c7416de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c737fe700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c7415a650 0x7f4c740b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c72ffd700 1 -- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c7415a650 msgr2=0x7f4c740b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c72ffd700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c7415a650 0x7f4c740b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c72ffd700 1 -- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740cced0 msgr2=0x7f4c7416de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c72ffd700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740cced0 0x7f4c7416de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c72ffd700 1 -- 172.21.15.33:0/2447606229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c68009420 con 0x7f4c740af540 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c727fc700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740cced0 0x7f4c7416de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c72ffd700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 0x7f4c740b1c60 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f4c7415ac00 tx=0x7f4c64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.528+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011b10 con 0x7f4c740af540 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.529+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6401a430 con 0x7f4c740af540 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.529+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6401a7f0 con 0x7f4c740af540 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.529+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c64009420 con 0x7f4c740af540 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.529+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7416d910 con 0x7f4c740af540 2023-10-06T05:26:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.530+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7416d770 con 0x7f4c740af540 2023-10-06T05:26:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.531+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c6401aba0 con 0x7f4c740af540 2023-10-06T05:26:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.532+0000 7f4c5affd700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.532+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4c640860c0 con 0x7f4c740af540 2023-10-06T05:26:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.532+0000 7f4c727fc700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.533+0000 7f4c727fc700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c60066f30 0x7f4c600693e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4c680093f0 tx=0x7f4c68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.536+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c7416d770 con 0x7f4c740af540 2023-10-06T05:26:13.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.741+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c7416d770 con 0x7f4c740af540 2023-10-06T05:26:13.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.742+0000 7f4c5affd700 1 -- 172.21.15.33:0/2447606229 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v165) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c7416d770 con 0x7f4c740af540 2023-10-06T05:26:13.745 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c60066f30 msgr2=0x7f4c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c60066f30 0x7f4c600693e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4c680093f0 tx=0x7f4c68008040 comp rx=0 tx=0).stop 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 msgr2=0x7f4c740b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 0x7f4c740b1c60 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f4c7415ac00 tx=0x7f4c64017000 comp rx=0 tx=0).stop 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 shutdown_connections 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c7415a650 0x7f4c740b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c740af540 0x7f4c740b1c60 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 --2- 172.21.15.33:0/2447606229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c740cced0 0x7f4c7416de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 >> 172.21.15.33:0/2447606229 conn(0x7f4c7406d9b0 msgr2=0x7f4c74159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.744+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 shutdown_connections 2023-10-06T05:26:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.745+0000 7f4c73fff700 1 -- 172.21.15.33:0/2447606229 wait complete. 2023-10-06T05:26:13.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:26:13.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.949+0000 7fec68d9d700 1 Processor -- start 2023-10-06T05:26:13.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.949+0000 7fec68d9d700 1 -- start start 2023-10-06T05:26:13.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.950+0000 7fec68d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec64159530 0x7fec6406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.950+0000 7fec68d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fec640b04e0 0x7fec6415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.950+0000 7fec68d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6406e3e0 con 0x7fec6406df70 2023-10-06T05:26:13.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.950+0000 7fec68d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640b1e00 con 0x7fec64159530 2023-10-06T05:26:13.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.950+0000 7fec68d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6406f6b0 con 0x7fec640b04e0 2023-10-06T05:26:13.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec637fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec64159530 0x7fec6406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec62ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fec6406df70 0x7fec640afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50878/0 (socket says 172.21.15.33:50878) 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec62ffd700 1 -- 172.21.15.33:0/529658278 learned_addr learned my addr 172.21.15.33:0/529658278 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec63fff700 1 --2- 172.21.15.33:0/529658278 >> v2:172.21.15.167:3300/0 conn(0x7fec640b04e0 0x7fec6415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec627fc700 1 -- 172.21.15.33:0/529658278 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189245763 0 0) 0x7fec6406e3e0 con 0x7fec6406df70 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec627fc700 1 -- 172.21.15.33:0/529658278 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec4c003570 con 0x7fec6406df70 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec637fe700 1 -- 172.21.15.33:0/529658278 >> v2:172.21.15.167:3300/0 conn(0x7fec640b04e0 msgr2=0x7fec6415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec637fe700 1 --2- 172.21.15.33:0/529658278 >> v2:172.21.15.167:3300/0 conn(0x7fec640b04e0 0x7fec6415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec637fe700 1 -- 172.21.15.33:0/529658278 >> v1:172.21.15.33:6789/0 conn(0x7fec6406df70 legacy=0x7fec640afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec637fe700 1 -- 172.21.15.33:0/529658278 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6406f220 con 0x7fec64159530 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec63fff700 1 --2- 172.21.15.33:0/529658278 >> v2:172.21.15.167:3300/0 conn(0x7fec640b04e0 0x7fec6415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:13.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.951+0000 7fec637fe700 1 --2- 172.21.15.33:0/529658278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec6406da30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fec640cf3c0 tx=0x7fec540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19d450f31cf564d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.955+0000 7fec627fc700 1 -- 172.21.15.33:0/529658278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54011630 con 0x7fec64159530 2023-10-06T05:26:13.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec627fc700 1 -- 172.21.15.33:0/529658278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54011790 con 0x7fec64159530 2023-10-06T05:26:13.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec627fc700 1 -- 172.21.15.33:0/529658278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54011b30 con 0x7fec64159530 2023-10-06T05:26:13.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 -- 172.21.15.33:0/529658278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 msgr2=0x7fec6406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 --2- 172.21.15.33:0/529658278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec6406da30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fec640cf3c0 tx=0x7fec540091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 -- 172.21.15.33:0/529658278 shutdown_connections 2023-10-06T05:26:13.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 --2- 172.21.15.33:0/529658278 >> v2:172.21.15.167:3300/0 conn(0x7fec640b04e0 0x7fec6415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 --2- 172.21.15.33:0/529658278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec6406da30 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 -- 172.21.15.33:0/529658278 >> 172.21.15.33:0/529658278 conn(0x7fec6406c560 msgr2=0x7fec6406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 -- 172.21.15.33:0/529658278 shutdown_connections 2023-10-06T05:26:13.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.956+0000 7fec68d9d700 1 -- 172.21.15.33:0/529658278 wait complete. 2023-10-06T05:26:13.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.958+0000 7fec68d9d700 1 Processor -- start 2023-10-06T05:26:13.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.958+0000 7fec68d9d700 1 -- start start 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec68d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406df70 0x7fec6415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec68d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b04e0 0x7fec6416a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec68d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec64152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec68d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6414eab0 con 0x7fec64159530 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec68d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6415bbe0 con 0x7fec640b04e0 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec68d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6415aff0 con 0x7fec6406df70 2023-10-06T05:26:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec63fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec64152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec63fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec64152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50886/0 (socket says 172.21.15.33:50886) 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.959+0000 7fec63fff700 1 -- 172.21.15.33:0/3295615415 learned_addr learned my addr 172.21.15.33:0/3295615415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec62ffd700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b04e0 0x7fec6416a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec637fe700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406df70 0x7fec6415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec63fff700 1 -- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406df70 msgr2=0x7fec6415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec63fff700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406df70 0x7fec6415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec63fff700 1 -- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b04e0 msgr2=0x7fec6416a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec63fff700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b04e0 0x7fec6416a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec63fff700 1 -- 172.21.15.33:0/3295615415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54009420 con 0x7fec64159530 2023-10-06T05:26:13.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec63fff700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec64152210 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d180 tx=0x7fec5c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00de20 con 0x7fec64159530 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00f040 con 0x7fec64159530 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6415b2b0 con 0x7fec64159530 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.960+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c015690 con 0x7fec64159530 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.961+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6415b870 con 0x7fec64159530 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.962+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6404c380 con 0x7fec64159530 2023-10-06T05:26:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.963+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec5c01e3b0 con 0x7fec64159530 2023-10-06T05:26:13.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.963+0000 7fec4b7fe700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec50066f30 0x7fec500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.964+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fec5c0849f0 con 0x7fec64159530 2023-10-06T05:26:13.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.964+0000 7fec637fe700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec50066f30 0x7fec500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.965+0000 7fec637fe700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec50066f30 0x7fec500693e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fec540091e0 tx=0x7fec54017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:13.968+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec5c051db0 con 0x7fec64159530 2023-10-06T05:26:14.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.173+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fec6415bd20 con 0x7fec64159530 2023-10-06T05:26:14.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.173+0000 7fec4b7fe700 1 -- 172.21.15.33:0/3295615415 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v165) v1 ==== 84+0+126 (secure 0 0 0) 0x7fec5c01a090 con 0x7fec64159530 2023-10-06T05:26:14.176 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:14.176 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:26:14.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:14.176 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:26:14.177 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:26:14.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.175+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec50066f30 msgr2=0x7fec500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.175+0000 7fec68d9d700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec50066f30 0x7fec500693e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fec540091e0 tx=0x7fec54017000 comp rx=0 tx=0).stop 2023-10-06T05:26:14.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.175+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 msgr2=0x7fec64152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.175+0000 7fec68d9d700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec64152210 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d180 tx=0x7fec5c009a50 comp rx=0 tx=0).stop 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 shutdown_connections 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec50066f30 0x7fec500693e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64159530 0x7fec64152210 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec6406df70 0x7fec6415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 --2- 172.21.15.33:0/3295615415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b04e0 0x7fec6416a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 >> 172.21.15.33:0/3295615415 conn(0x7fec6406c560 msgr2=0x7fec641502c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 shutdown_connections 2023-10-06T05:26:14.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.176+0000 7fec68d9d700 1 -- 172.21.15.33:0/3295615415 wait complete. 2023-10-06T05:26:14.195 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:26:14.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:26:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.385+0000 7fa50db9d700 1 Processor -- start 2023-10-06T05:26:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.385+0000 7fa50db9d700 1 -- start start 2023-10-06T05:26:14.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5080ad3e0 0x7fa508156b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50db9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5081570a0 0x7fa508157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50db9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50806b1f0 con 0x7fa5080ad010 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50db9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50806a3f0 con 0x7fa5081570a0 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50806f110 con 0x7fa5080ad3e0 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50d39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5081570a0 0x7fa508157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50cb9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa5080ad010 0x7fa50806ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50888/0 (socket says 172.21.15.33:50888) 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50cb9b700 1 -- 172.21.15.33:0/2618925669 learned_addr learned my addr 172.21.15.33:0/2618925669 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa507fff700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5080ad3e0 0x7fa508156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50d39c700 1 -- 172.21.15.33:0/2618925669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5080ad3e0 msgr2=0x7fa508156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50d39c700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5080ad3e0 0x7fa508156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50d39c700 1 -- 172.21.15.33:0/2618925669 >> v1:172.21.15.33:6789/0 conn(0x7fa5080ad010 legacy=0x7fa50806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa50d39c700 1 -- 172.21.15.33:0/2618925669 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa508157a40 con 0x7fa5081570a0 2023-10-06T05:26:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa5077fe700 1 -- 172.21.15.33:0/2618925669 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123511580 0 0) 0x7fa50806b1f0 con 0x7fa5080ad010 2023-10-06T05:26:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.387+0000 7fa507fff700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5080ad3e0 0x7fa508156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa50d39c700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5081570a0 0x7fa508157500 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fa500009e60 tx=0x7fa500009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ff4e5900e58f89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa5077fe700 1 -- 172.21.15.33:0/2618925669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000e070 con 0x7fa5081570a0 2023-10-06T05:26:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa5077fe700 1 -- 172.21.15.33:0/2618925669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50000bcc0 con 0x7fa5081570a0 2023-10-06T05:26:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa5077fe700 1 -- 172.21.15.33:0/2618925669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500012680 con 0x7fa5081570a0 2023-10-06T05:26:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa50db9d700 1 -- 172.21.15.33:0/2618925669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5081570a0 msgr2=0x7fa508157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5081570a0 0x7fa508157500 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fa500009e60 tx=0x7fa500009780 comp rx=0 tx=0).stop 2023-10-06T05:26:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa50db9d700 1 -- 172.21.15.33:0/2618925669 shutdown_connections 2023-10-06T05:26:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5081570a0 0x7fa508157500 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.388+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2618925669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5080ad3e0 0x7fa508156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.389+0000 7fa50db9d700 1 -- 172.21.15.33:0/2618925669 >> 172.21.15.33:0/2618925669 conn(0x7fa508068090 msgr2=0x7fa50806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.389+0000 7fa50db9d700 1 -- 172.21.15.33:0/2618925669 shutdown_connections 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.389+0000 7fa50db9d700 1 -- 172.21.15.33:0/2618925669 wait complete. 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.390+0000 7fa50db9d700 1 Processor -- start 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.390+0000 7fa50db9d700 1 -- start start 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5080ad010 0x7fa5080a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 0x7fa508143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5081570a0 0x7fa5080a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508144490 con 0x7fa5080ad3e0 2023-10-06T05:26:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080a67f0 con 0x7fa5080ad010 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa508146a60 con 0x7fa5081570a0 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 0x7fa508143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 0x7fa508143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50904/0 (socket says 172.21.15.33:50904) 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa507fff700 1 -- 172.21.15.33:0/2949643355 learned_addr learned my addr 172.21.15.33:0/2949643355 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50cb9b700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5080ad010 0x7fa5080a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.391+0000 7fa50d39c700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5081570a0 0x7fa5080a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa507fff700 1 -- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5081570a0 msgr2=0x7fa5080a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa507fff700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5081570a0 0x7fa5080a6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa507fff700 1 -- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5080ad010 msgr2=0x7fa5080a7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa507fff700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5080ad010 0x7fa5080a7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa507fff700 1 -- 172.21.15.33:0/2949643355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500009a20 con 0x7fa5080ad3e0 2023-10-06T05:26:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa507fff700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 0x7fa508143f50 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e8d0 tx=0x7fa4f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4013410 con 0x7fa5080ad3e0 2023-10-06T05:26:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f4013570 con 0x7fa5080ad3e0 2023-10-06T05:26:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4013910 con 0x7fa5080ad3e0 2023-10-06T05:26:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.392+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa508146130 con 0x7fa5080ad3e0 2023-10-06T05:26:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.393+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5081466f0 con 0x7fa5080ad3e0 2023-10-06T05:26:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.393+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa508146bd0 con 0x7fa5080ad3e0 2023-10-06T05:26:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.394+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4f4013d40 con 0x7fa5080ad3e0 2023-10-06T05:26:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.395+0000 7fa504ff9700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa4ec066f30 0x7fa4ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.395+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa4f40840e0 con 0x7fa5080ad3e0 2023-10-06T05:26:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.396+0000 7fa50cb9b700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa4ec066f30 0x7fa4ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.396+0000 7fa50cb9b700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa4ec066f30 0x7fa4ec0693e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc007fd0 tx=0x7fa4fc007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.399+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4f40514a0 con 0x7fa5080ad3e0 2023-10-06T05:26:14.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.602+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa508063ff0 con 0x7fa5080ad3e0 2023-10-06T05:26:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.961+0000 7fa504ff9700 1 -- 172.21.15.33:0/2949643355 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v166) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa4f4054ac0 con 0x7fa5080ad3e0 2023-10-06T05:26:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.966+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa4ec066f30 msgr2=0x7fa4ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.966+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa4ec066f30 0x7fa4ec0693e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc007fd0 tx=0x7fa4fc007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.966+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 msgr2=0x7fa508143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.966+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 0x7fa508143f50 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e8d0 tx=0x7fa4f4008040 comp rx=0 tx=0).stop 2023-10-06T05:26:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 shutdown_connections 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa4ec066f30 0x7fa4ec0693e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5081570a0 0x7fa5080a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5080ad010 0x7fa5080a7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 --2- 172.21.15.33:0/2949643355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080ad3e0 0x7fa508143f50 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 >> 172.21.15.33:0/2949643355 conn(0x7fa508068090 msgr2=0x7fa5080af320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 shutdown_connections 2023-10-06T05:26:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:14.968+0000 7fa50db9d700 1 -- 172.21.15.33:0/2949643355 wait complete. 2023-10-06T05:26:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:26:14.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:26:15.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.182+0000 7fe7c199d700 1 Processor -- start 2023-10-06T05:26:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.182+0000 7fe7c199d700 1 -- start start 2023-10-06T05:26:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.183+0000 7fe7c199d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe7bc0af540 0x7fe7bc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.183+0000 7fe7c199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 0x7fe7bc0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.183+0000 7fe7c199d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc15ddb0 con 0x7fe7bc0cced0 2023-10-06T05:26:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.183+0000 7fe7c199d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cba60 con 0x7fe7bc0af540 2023-10-06T05:26:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.183+0000 7fe7c199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc15d1a0 con 0x7fe7bc15a7d0 2023-10-06T05:26:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7c119c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7bc0cced0 0x7fe7bc06c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50904/0 (socket says 172.21.15.33:50904) 2023-10-06T05:26:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7c119c700 1 -- 172.21.15.33:0/1853343777 learned_addr learned my addr 172.21.15.33:0/1853343777 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bbfff700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 0x7fe7bc0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7c099b700 1 --2- 172.21.15.33:0/1853343777 >> v2:172.21.15.167:3300/0 conn(0x7fe7bc0af540 0x7fe7bc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234387939 0 0) 0x7fe7bc15ddb0 con 0x7fe7bc0cced0 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ac003570 con 0x7fe7bc0cced0 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123549973 0 0) 0x7fe7ac003570 con 0x7fe7bc0cced0 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 msgr2=0x7fe7bc0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 0x7fe7bc0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 >> v2:172.21.15.167:3300/0 conn(0x7fe7bc0af540 msgr2=0x7fe7bc0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 --2- 172.21.15.33:0/1853343777 >> v2:172.21.15.167:3300/0 conn(0x7fe7bc0af540 0x7fe7bc0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc15d480 con 0x7fe7bc0cced0 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.184+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe7b40017f0 con 0x7fe7bc0cced0 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 >> v1:172.21.15.33:6789/0 conn(0x7fe7bc0cced0 legacy=0x7fe7bc06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 0x7fe7ac003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7ac003fe0 0x7fe7ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7ac004d50 0x7fe7ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cba60 con 0x7fe7ac003fe0 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc15d1a0 con 0x7fe7ac004d50 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac00ba90 con 0x7fe7bc15a7d0 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bbfff700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7ac003fe0 0x7fe7ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7c099b700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 0x7fe7ac003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7c119c700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7ac004d50 0x7fe7ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bbfff700 1 -- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 msgr2=0x7fe7ac003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bbfff700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7bc15a7d0 0x7fe7ac003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bbfff700 1 -- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7ac004d50 msgr2=0x7fe7ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bbfff700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7ac004d50 0x7fe7ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.185+0000 7fe7bbfff700 1 -- 172.21.15.33:0/1853343777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac003570 con 0x7fe7ac003fe0 2023-10-06T05:26:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.186+0000 7fe7bbfff700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7ac003fe0 0x7fe7ac004810 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009c00 tx=0x7fe7a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd21913159042298 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.186+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800e6b0 con 0x7fe7ac003fe0 2023-10-06T05:26:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.186+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800e810 con 0x7fe7ac003fe0 2023-10-06T05:26:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.186+0000 7fe7bb7fe700 1 -- 172.21.15.33:0/1853343777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800ebd0 con 0x7fe7ac003fe0 2023-10-06T05:26:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.186+0000 7fe7c199d700 1 -- 172.21.15.33:0/1853343777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7ac003fe0 msgr2=0x7fe7ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.186+0000 7fe7c199d700 1 --2- 172.21.15.33:0/1853343777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7ac003fe0 0x7fe7ac004810 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009c00 tx=0x7fe7a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.187+0000 7fe7c119c700 1 -- 172.21.15.33:0/1853343777 reap_dead start 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.187+0000 7fe7c199d700 1 -- 172.21.15.33:0/1853343777 shutdown_connections 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.187+0000 7fe7c199d700 1 -- 172.21.15.33:0/1853343777 >> 172.21.15.33:0/1853343777 conn(0x7fe7bc06d9b0 msgr2=0x7fe7bc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.187+0000 7fe7c199d700 1 -- 172.21.15.33:0/1853343777 shutdown_connections 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.187+0000 7fe7c199d700 1 -- 172.21.15.33:0/1853343777 wait complete. 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.188+0000 7fe7c199d700 1 Processor -- start 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.188+0000 7fe7c199d700 1 -- start start 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7ac003fe0 0x7fe7bc169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc0c7580 0x7fe7bc0c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 0x7fe7bc0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0c7f30 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0caf90 con 0x7fe7bc0c7580 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0ca380 con 0x7fe7ac003fe0 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.189+0000 7fe7c119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 0x7fe7bc0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 0x7fe7bc0c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50924/0 (socket says 172.21.15.33:50924) 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 -- 172.21.15.33:0/4156187518 learned_addr learned my addr 172.21.15.33:0/4156187518 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7bbfff700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc0c7580 0x7fe7bc0c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c099b700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7ac003fe0 0x7fe7bc169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 -- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7ac003fe0 msgr2=0x7fe7bc169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7ac003fe0 0x7fe7bc169bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 -- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc0c7580 msgr2=0x7fe7bc0c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc0c7580 0x7fe7bc0c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 -- 172.21.15.33:0/4156187518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0ca660 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7c119c700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 0x7fe7bc0c9a50 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f80 tx=0x7fe7b4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.190+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400ad80 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.191+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b40133f0 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.191+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40137b0 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.191+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0ca940 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.191+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc16a0f0 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.192+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc04c380 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.194+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7b4013b60 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.194+0000 7fe7b8ff9700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7a0066f30 0x7fe7a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.195+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fe7b4085840 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.195+0000 7fe7c099b700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7a0066f30 0x7fe7a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.196+0000 7fe7c099b700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7a0066f30 0x7fe7a00693e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc158d00 tx=0x7fe7b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.198+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7b4052c60 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.400+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe7bc15d760 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.966+0000 7fe7b8ff9700 1 -- 172.21.15.33:0/4156187518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v167) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe7b4056280 con 0x7fe7bc0c95b0 2023-10-06T05:26:15.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.976+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7a0066f30 msgr2=0x7fe7a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.976+0000 7fe7c199d700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7a0066f30 0x7fe7a00693e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc158d00 tx=0x7fe7b0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.976+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 msgr2=0x7fe7bc0c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.976+0000 7fe7c199d700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 0x7fe7bc0c9a50 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f80 tx=0x7fe7b4002210 comp rx=0 tx=0).stop 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.977+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 shutdown_connections 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.977+0000 7fe7c199d700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe7a0066f30 0x7fe7a00693e0 secure :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc158d00 tx=0x7fe7b0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.977+0000 7fe7c199d700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7bc0c95b0 0x7fe7bc0c9a50 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.977+0000 7fe7c199d700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe7ac003fe0 0x7fe7bc169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.977+0000 7fe7c199d700 1 --2- 172.21.15.33:0/4156187518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7bc0c7580 0x7fe7bc0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.977+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 >> 172.21.15.33:0/4156187518 conn(0x7fe7bc06d9b0 msgr2=0x7fe7bc0c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.980+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 shutdown_connections 2023-10-06T05:26:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:15.981+0000 7fe7c199d700 1 -- 172.21.15.33:0/4156187518 wait complete. 2023-10-06T05:26:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:26:16.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:26:16.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.200+0000 7fec6a39d700 1 Processor -- start 2023-10-06T05:26:16.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.200+0000 7fec6a39d700 1 -- start start 2023-10-06T05:26:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6a39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec64154f20 0x7fec640ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec640b18f0 0x7fec640cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6a39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec641586a0 con 0x7fec641552f0 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6a39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6406bdf0 con 0x7fec64154f20 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec64157ab0 con 0x7fec640b18f0 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec64154f20 0x7fec640ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fec64154f20 0x7fec640ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50940/0 (socket says 172.21.15.33:50940) 2023-10-06T05:26:16.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec6939b700 1 -- 172.21.15.33:0/3709955605 learned_addr learned my addr 172.21.15.33:0/3709955605 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.202+0000 7fec69b9c700 1 --2- 172.21.15.33:0/3709955605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec640b18f0 0x7fec640cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec6939b700 1 -- 172.21.15.33:0/3709955605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec640b18f0 msgr2=0x7fec640cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec6939b700 1 --2- 172.21.15.33:0/3709955605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec640b18f0 0x7fec640cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec6939b700 1 -- 172.21.15.33:0/3709955605 >> v1:172.21.15.167:6789/0 conn(0x7fec641552f0 legacy=0x7fec640b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec6939b700 1 -- 172.21.15.33:0/3709955605 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64157d70 con 0x7fec64154f20 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec6939b700 1 --2- 172.21.15.33:0/3709955605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec640ae720 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fec600099b0 tx=0x7fec60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cd86d84f6a2b7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec5bfff700 1 -- 172.21.15.33:0/3709955605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60009ea0 con 0x7fec64154f20 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec5bfff700 1 -- 172.21.15.33:0/3709955605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6000b860 con 0x7fec64154f20 2023-10-06T05:26:16.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.203+0000 7fec5bfff700 1 -- 172.21.15.33:0/3709955605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000bc00 con 0x7fec64154f20 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 -- 172.21.15.33:0/3709955605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 msgr2=0x7fec640ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 --2- 172.21.15.33:0/3709955605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec640ae720 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fec600099b0 tx=0x7fec60009250 comp rx=0 tx=0).stop 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 -- 172.21.15.33:0/3709955605 shutdown_connections 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 --2- 172.21.15.33:0/3709955605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec640ae720 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 --2- 172.21.15.33:0/3709955605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec640b18f0 0x7fec640cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 -- 172.21.15.33:0/3709955605 >> 172.21.15.33:0/3709955605 conn(0x7fec6406d030 msgr2=0x7fec640cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 -- 172.21.15.33:0/3709955605 shutdown_connections 2023-10-06T05:26:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.204+0000 7fec6a39d700 1 -- 172.21.15.33:0/3709955605 wait complete. 2023-10-06T05:26:16.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.206+0000 7fec6a39d700 1 Processor -- start 2023-10-06T05:26:16.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.206+0000 7fec6a39d700 1 -- start start 2023-10-06T05:26:16.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b18f0 0x7fec64148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec6415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec64154170 0x7fec6415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec641492f0 con 0x7fec64154f20 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6406ac30 con 0x7fec640b18f0 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6406af10 con 0x7fec64154170 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec6415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec6415b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50946/0 (socket says 172.21.15.33:50946) 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 -- 172.21.15.33:0/382317718 learned_addr learned my addr 172.21.15.33:0/382317718 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6939b700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b18f0 0x7fec64148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec69b9c700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec64154170 0x7fec6415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 -- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec64154170 msgr2=0x7fec6415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec64154170 0x7fec6415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 -- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b18f0 msgr2=0x7fec64148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b18f0 0x7fec64148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec68b9a700 1 -- 172.21.15.33:0/382317718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600094f0 con 0x7fec64154f20 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.207+0000 7fec6939b700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b18f0 0x7fec64148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.208+0000 7fec68b9a700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec6415b040 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c590 tx=0x7fec5400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.208+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54011590 con 0x7fec64154f20 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.208+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec540116f0 con 0x7fec64154f20 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.208+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54011ab0 con 0x7fec64154f20 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.208+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec640af170 con 0x7fec64154f20 2023-10-06T05:26:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.208+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6406a900 con 0x7fec64154f20 2023-10-06T05:26:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.209+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec640af610 con 0x7fec64154f20 2023-10-06T05:26:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.210+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec5401b3b0 con 0x7fec64154f20 2023-10-06T05:26:16.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.211+0000 7fec597fa700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:16.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.212+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fec54083c80 con 0x7fec64154f20 2023-10-06T05:26:16.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.212+0000 7fec6939b700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:16.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.213+0000 7fec6939b700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fec600099b0 tx=0x7fec60017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:16.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.216+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec54050d50 con 0x7fec64154f20 2023-10-06T05:26:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.464+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fec640aefd0 con 0x7fec64154f20 2023-10-06T05:26:16.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.982+0000 7fec597fa700 1 -- 172.21.15.33:0/382317718 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v120) v1 ==== 174+0+0 (secure 0 0 0) 0x7fec640aefd0 con 0x7fec64154f20 2023-10-06T05:26:16.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.992+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 msgr2=0x7fec480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:16.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.992+0000 7fec6a39d700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fec600099b0 tx=0x7fec60017000 comp rx=0 tx=0).stop 2023-10-06T05:26:16.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.992+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 msgr2=0x7fec6415b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:16.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.992+0000 7fec6a39d700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec6415b040 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c590 tx=0x7fec5400c950 comp rx=0 tx=0).stop 2023-10-06T05:26:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.994+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 shutdown_connections 2023-10-06T05:26:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.994+0000 7fec6a39d700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.994+0000 7fec6a39d700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec64154170 0x7fec6415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.994+0000 7fec6a39d700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec640b18f0 0x7fec64148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.994+0000 7fec6a39d700 1 --2- 172.21.15.33:0/382317718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec64154f20 0x7fec6415b040 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:16.994+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 >> 172.21.15.33:0/382317718 conn(0x7fec6406d030 msgr2=0x7fec640ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.006+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 shutdown_connections 2023-10-06T05:26:17.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.010+0000 7fec6a39d700 1 -- 172.21.15.33:0/382317718 wait complete. 2023-10-06T05:26:17.018 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 25 and data pool 26 2023-10-06T05:26:17.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:17.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.223+0000 7f410099d700 1 Processor -- start 2023-10-06T05:26:17.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.223+0000 7f410099d700 1 -- start start 2023-10-06T05:26:17.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f410099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40fc158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f410099d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f40fc0699c0 0x7f40fc069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f410099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc0c8e40 con 0x7f40fc0adf50 2023-10-06T05:26:17.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f410099d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc06a380 con 0x7f40fc0699c0 2023-10-06T05:26:17.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f410099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0c99e0 con 0x7f40fc155140 2023-10-06T05:26:17.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f40faffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f40fc0adf50 0x7f40fc0c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50906/0 (socket says 172.21.15.33:50906) 2023-10-06T05:26:17.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f40faffd700 1 -- 172.21.15.33:0/3638028946 learned_addr learned my addr 172.21.15.33:0/3638028946 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:17.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f40fbfff700 1 --2- 172.21.15.33:0/3638028946 >> v2:172.21.15.167:3300/0 conn(0x7f40fc0699c0 0x7f40fc069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40fc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385827657 0 0) 0x7f40fc0c8e40 con 0x7f40fc0adf50 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.225+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e4003570 con 0x7f40fc0adf50 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583077966 0 0) 0x7f40e4003570 con 0x7f40fc0adf50 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 msgr2=0x7f40fc158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40fc158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 >> v2:172.21.15.167:3300/0 conn(0x7f40fc0699c0 msgr2=0x7f40fc069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 --2- 172.21.15.33:0/3638028946 >> v2:172.21.15.167:3300/0 conn(0x7f40fc0699c0 0x7f40fc069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc0ae6c0 con 0x7f40fc0adf50 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f40f0001470 con 0x7f40fc0adf50 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 >> v1:172.21.15.33:6789/0 conn(0x7f40fc0adf50 legacy=0x7f40fc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40e4003ae0 0x7f40e4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4004d50 0x7f40e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06a380 con 0x7f40e4003ae0 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0c99e0 con 0x7f40e4004d50 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e400bb10 con 0x7f40fc155140 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40e4003ae0 0x7f40e4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40faffd700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.226+0000 7f40fbfff700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4004d50 0x7f40e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fb7fe700 1 -- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 msgr2=0x7f40e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fb7fe700 1 -- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4004d50 msgr2=0x7f40e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4004d50 0x7f40e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fb7fe700 1 -- 172.21.15.33:0/3638028946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e4003570 con 0x7f40e4003ae0 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fbfff700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4004d50 0x7f40e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40faffd700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc155140 0x7f40e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40e4003ae0 0x7f40e4003ef0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009c30 tx=0x7f40ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6348154c994a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00e710 con 0x7f40e4003ae0 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec00e870 con 0x7f40e4003ae0 2023-10-06T05:26:17.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.227+0000 7f40fa7fc700 1 -- 172.21.15.33:0/3638028946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00ec30 con 0x7f40e4003ae0 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f410099d700 1 -- 172.21.15.33:0/3638028946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40e4003ae0 msgr2=0x7f40e4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f410099d700 1 --2- 172.21.15.33:0/3638028946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40e4003ae0 0x7f40e4003ef0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009c30 tx=0x7f40ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f40fbfff700 1 -- 172.21.15.33:0/3638028946 reap_dead start 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f410099d700 1 -- 172.21.15.33:0/3638028946 shutdown_connections 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f410099d700 1 -- 172.21.15.33:0/3638028946 >> 172.21.15.33:0/3638028946 conn(0x7f40fc06d030 msgr2=0x7f40fc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f410099d700 1 -- 172.21.15.33:0/3638028946 shutdown_connections 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.228+0000 7f410099d700 1 -- 172.21.15.33:0/3638028946 wait complete. 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.229+0000 7f410099d700 1 Processor -- start 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.229+0000 7f410099d700 1 -- start start 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f410099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4003ae0 0x7f40fc0aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f410099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 0x7f40fc169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f410099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc169c40 0x7f40fc16a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f410099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc16a5f0 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f410099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc154400 con 0x7f40e4003ae0 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f410099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0ab230 con 0x7f40fc169c40 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.230+0000 7f40faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 0x7f40fc169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 0x7f40fc169700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50958/0 (socket says 172.21.15.33:50958) 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 -- 172.21.15.33:0/3404896742 learned_addr learned my addr 172.21.15.33:0/3404896742 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4003ae0 0x7f40fc0aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40fbfff700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc169c40 0x7f40fc16a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 -- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc169c40 msgr2=0x7f40fc16a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc169c40 0x7f40fc16a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 -- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4003ae0 msgr2=0x7f40fc0aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4003ae0 0x7f40fc0aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 -- 172.21.15.33:0/3404896742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc0ab510 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40faffd700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 0x7f40fc169700 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f40f00015e0 tx=0x7f40f0001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.231+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000ee70 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.232+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f000c760 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.232+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc0ab7f0 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.232+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f000cae0 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.232+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc0aae40 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.233+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc0abc90 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.234+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40f001d460 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.234+0000 7f40e37fe700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f40e8066f30 0x7f40e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.234+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f40f00845d0 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.235+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f40e8066f30 0x7f40e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.236+0000 7f40fb7fe700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f40e8066f30 0x7f40e80693e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0091e0 tx=0x7f40ec01a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.238+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40f0051640 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.440+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40fc0c8b20 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.441+0000 7f40e37fe700 1 -- 172.21.15.33:0/3404896742 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f40f0054c60 con 0x7f40fc0aa9d0 2023-10-06T05:26:17.444 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:17.444 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":168,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:16.466974+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:14.643519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:15.441428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.443+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f40e8066f30 msgr2=0x7f40e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f40e8066f30 0x7f40e80693e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0091e0 tx=0x7f40ec01a000 comp rx=0 tx=0).stop 2023-10-06T05:26:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 msgr2=0x7f40fc169700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 0x7f40fc169700 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f40f00015e0 tx=0x7f40f0001d90 comp rx=0 tx=0).stop 2023-10-06T05:26:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 shutdown_connections 2023-10-06T05:26:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f40e8066f30 0x7f40e80693e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40fc169c40 0x7f40fc16a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40e4003ae0 0x7f40fc0aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 --2- 172.21.15.33:0/3404896742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40fc0aa9d0 0x7f40fc169700 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 >> 172.21.15.33:0/3404896742 conn(0x7f40fc06d030 msgr2=0x7f40fc0c1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 shutdown_connections 2023-10-06T05:26:17.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.444+0000 7f410099d700 1 -- 172.21.15.33:0/3404896742 wait complete. 2023-10-06T05:26:17.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.648+0000 7f3375d9d700 1 Processor -- start 2023-10-06T05:26:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.648+0000 7f3375d9d700 1 -- start start 2023-10-06T05:26:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3375d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3370155af0 0x7f337006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3375d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f337006a6a0 0x7f33700c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3375d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337006ab60 con 0x7f33700c8b50 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3375d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337006b7e0 con 0x7f3370155af0 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3375d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337006ae40 con 0x7f337006a6a0 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3374d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3370155af0 0x7f337006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f337559c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f33700c8b50 0x7f33701598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50922/0 (socket says 172.21.15.33:50922) 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f337559c700 1 -- 172.21.15.33:0/2753482215 learned_addr learned my addr 172.21.15.33:0/2753482215 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f336ffff700 1 --2- 172.21.15.33:0/2753482215 >> v2:172.21.15.167:3300/0 conn(0x7f337006a6a0 0x7f33700c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3374d9b700 1 -- 172.21.15.33:0/2753482215 >> v2:172.21.15.167:3300/0 conn(0x7f337006a6a0 msgr2=0x7f33700c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3374d9b700 1 --2- 172.21.15.33:0/2753482215 >> v2:172.21.15.167:3300/0 conn(0x7f337006a6a0 0x7f33700c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f3374d9b700 1 -- 172.21.15.33:0/2753482215 >> v1:172.21.15.33:6789/0 conn(0x7f33700c8b50 legacy=0x7f33701598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.650+0000 7f336f7fe700 1 -- 172.21.15.33:0/2753482215 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951311304 0 0) 0x7f337006ab60 con 0x7f33700c8b50 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.651+0000 7f3374d9b700 1 -- 172.21.15.33:0/2753482215 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337006b120 con 0x7f3370155af0 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.651+0000 7f3374d9b700 1 --2- 172.21.15.33:0/2753482215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3370155af0 0x7f337006a160 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f337006f250 tx=0x7f33640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e8809299038ec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.651+0000 7f336f7fe700 1 -- 172.21.15.33:0/2753482215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364011630 con 0x7f3370155af0 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.651+0000 7f336f7fe700 1 -- 172.21.15.33:0/2753482215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3364011790 con 0x7f3370155af0 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.651+0000 7f336f7fe700 1 -- 172.21.15.33:0/2753482215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364011b30 con 0x7f3370155af0 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 -- 172.21.15.33:0/2753482215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3370155af0 msgr2=0x7f337006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 --2- 172.21.15.33:0/2753482215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3370155af0 0x7f337006a160 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f337006f250 tx=0x7f33640091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 -- 172.21.15.33:0/2753482215 shutdown_connections 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 --2- 172.21.15.33:0/2753482215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3370155af0 0x7f337006a160 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 --2- 172.21.15.33:0/2753482215 >> v2:172.21.15.167:3300/0 conn(0x7f337006a6a0 0x7f33700c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 -- 172.21.15.33:0/2753482215 >> 172.21.15.33:0/2753482215 conn(0x7f337006d030 msgr2=0x7f337006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 -- 172.21.15.33:0/2753482215 shutdown_connections 2023-10-06T05:26:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.652+0000 7f3375d9d700 1 -- 172.21.15.33:0/2753482215 wait complete. 2023-10-06T05:26:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.653+0000 7f3375d9d700 1 Processor -- start 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.653+0000 7f3375d9d700 1 -- start start 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.654+0000 7f3375d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f337006a6a0 0x7f33700c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.654+0000 7f3375d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 0x7f33700c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.654+0000 7f3375d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3370155af0 0x7f33700adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.654+0000 7f3375d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33700c5bd0 con 0x7f33700c8b50 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.654+0000 7f3375d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33700c3b70 con 0x7f337006a6a0 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.654+0000 7f3375d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33700c5580 con 0x7f3370155af0 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 0x7f33700c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 0x7f33700c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50984/0 (socket says 172.21.15.33:50984) 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 -- 172.21.15.33:0/1957882774 learned_addr learned my addr 172.21.15.33:0/1957882774 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f3374d9b700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f337006a6a0 0x7f33700c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f337559c700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3370155af0 0x7f33700adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 -- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3370155af0 msgr2=0x7f33700adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3370155af0 0x7f33700adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 -- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f337006a6a0 msgr2=0x7f33700c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f337006a6a0 0x7f33700c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 -- 172.21.15.33:0/1957882774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364009420 con 0x7f33700c8b50 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.655+0000 7f336ffff700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 0x7f33700c3190 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f335c00ec50 tx=0x7f335c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.656+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00c850 con 0x7f33700c8b50 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.656+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c00c9b0 con 0x7f33700c8b50 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.656+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00cd70 con 0x7f33700c8b50 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.656+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33700c5860 con 0x7f33700c8b50 2023-10-06T05:26:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.656+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33700c36d0 con 0x7f33700c8b50 2023-10-06T05:26:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.657+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337004c380 con 0x7f33700c8b50 2023-10-06T05:26:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.658+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f335c013680 con 0x7f33700c8b50 2023-10-06T05:26:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.659+0000 7f336cff9700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3354066f30 0x7f33540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.659+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f335c0849b0 con 0x7f33700c8b50 2023-10-06T05:26:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.659+0000 7f3374d9b700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3354066f30 0x7f33540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.660+0000 7f3374d9b700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3354066f30 0x7f33540693e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f33640091e0 tx=0x7f3364017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.663+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f335c051a20 con 0x7f33700c8b50 2023-10-06T05:26:17.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.866+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33700c5d10 con 0x7f33700c8b50 2023-10-06T05:26:17.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.867+0000 7f336cff9700 1 -- 172.21.15.33:0/1957882774 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f335c055040 con 0x7f33700c8b50 2023-10-06T05:26:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:17.870 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":168,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:16.466974+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:14.643519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:15.441428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3354066f30 msgr2=0x7f33540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3354066f30 0x7f33540693e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f33640091e0 tx=0x7f3364017000 comp rx=0 tx=0).stop 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 msgr2=0x7f33700c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 0x7f33700c3190 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f335c00ec50 tx=0x7f335c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 shutdown_connections 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3354066f30 0x7f33540693e0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3370155af0 0x7f33700adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f337006a6a0 0x7f33700c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 --2- 172.21.15.33:0/1957882774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33700c8b50 0x7f33700c3190 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 >> 172.21.15.33:0/1957882774 conn(0x7f337006d030 msgr2=0x7f337006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 shutdown_connections 2023-10-06T05:26:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:17.870+0000 7f3375d9d700 1 -- 172.21.15.33:0/1957882774 wait complete. 2023-10-06T05:26:17.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:26:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.080+0000 7f8a9a19d700 1 Processor -- start 2023-10-06T05:26:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.080+0000 7f8a9a19d700 1 -- start start 2023-10-06T05:26:18.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a941595d0 0x7f8a940aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ad460 0x7f8a940ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9a19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a940aa7d0 con 0x7f8a940ac180 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a940a9bc0 con 0x7f8a941595d0 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940a9ea0 con 0x7f8a940ad460 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a941595d0 0x7f8a940aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a941595d0 0x7f8a940aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50990/0 (socket says 172.21.15.33:50990) 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9919b700 1 -- 172.21.15.33:0/2487672494 learned_addr learned my addr 172.21.15.33:0/2487672494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.082+0000 7f8a9999c700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ad460 0x7f8a940ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a9919b700 1 -- 172.21.15.33:0/2487672494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ad460 msgr2=0x7f8a940ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a9919b700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ad460 0x7f8a940ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a9919b700 1 -- 172.21.15.33:0/2487672494 >> v1:172.21.15.167:6789/0 conn(0x7f8a940ac180 legacy=0x7f8a940acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a9919b700 1 -- 172.21.15.33:0/2487672494 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a940aa180 con 0x7f8a941595d0 2023-10-06T05:26:18.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a9999c700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ad460 0x7f8a940ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:18.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a9919b700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a941595d0 0x7f8a940aac60 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8a900099b0 tx=0x7f8a90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76c3dc3e8e831b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a8bfff700 1 -- 172.21.15.33:0/2487672494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000b860 con 0x7f8a941595d0 2023-10-06T05:26:18.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a8bfff700 1 -- 172.21.15.33:0/2487672494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9000b9c0 con 0x7f8a941595d0 2023-10-06T05:26:18.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.083+0000 7f8a8bfff700 1 -- 172.21.15.33:0/2487672494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000bd60 con 0x7f8a941595d0 2023-10-06T05:26:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 -- 172.21.15.33:0/2487672494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a941595d0 msgr2=0x7f8a940aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a941595d0 0x7f8a940aac60 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8a900099b0 tx=0x7f8a90009250 comp rx=0 tx=0).stop 2023-10-06T05:26:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 -- 172.21.15.33:0/2487672494 shutdown_connections 2023-10-06T05:26:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a941595d0 0x7f8a940aac60 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/2487672494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ad460 0x7f8a940ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 -- 172.21.15.33:0/2487672494 >> 172.21.15.33:0/2487672494 conn(0x7f8a9406d030 msgr2=0x7f8a940cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 -- 172.21.15.33:0/2487672494 shutdown_connections 2023-10-06T05:26:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.084+0000 7f8a9a19d700 1 -- 172.21.15.33:0/2487672494 wait complete. 2023-10-06T05:26:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.085+0000 7f8a9a19d700 1 Processor -- start 2023-10-06T05:26:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.085+0000 7f8a9a19d700 1 -- start start 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ac180 0x7f8a94143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 0x7f8a9415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a941595d0 0x7f8a940c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a941599a0 con 0x7f8a940ad460 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94159c80 con 0x7f8a941595d0 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415d180 con 0x7f8a940ac180 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.086+0000 7f8a9899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 0x7f8a9415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 0x7f8a9415bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50994/0 (socket says 172.21.15.33:50994) 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 -- 172.21.15.33:0/1693373838 learned_addr learned my addr 172.21.15.33:0/1693373838 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 -- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ac180 msgr2=0x7f8a94143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ac180 0x7f8a94143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9999c700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a941595d0 0x7f8a940c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 -- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a941595d0 msgr2=0x7f8a940c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a941595d0 0x7f8a940c17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 -- 172.21.15.33:0/1693373838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900094f0 con 0x7f8a940ad460 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9999c700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a941595d0 0x7f8a940c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a9899a700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 0x7f8a9415bac0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400c5c0 tx=0x7f8a8400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84011590 con 0x7f8a940ad460 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a84011710 con 0x7f8a940ad460 2023-10-06T05:26:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.087+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84011ad0 con 0x7f8a940ad460 2023-10-06T05:26:18.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.088+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9415d460 con 0x7f8a940ad460 2023-10-06T05:26:18.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.088+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9415ce50 con 0x7f8a940ad460 2023-10-06T05:26:18.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.089+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9404c380 con 0x7f8a940ad460 2023-10-06T05:26:18.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.090+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a8401b3b0 con 0x7f8a940ad460 2023-10-06T05:26:18.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.090+0000 7f8a897fa700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a78066f30 0x7f8a780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.091+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a84083cf0 con 0x7f8a940ad460 2023-10-06T05:26:18.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.091+0000 7f8a9919b700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a78066f30 0x7f8a780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.092+0000 7f8a9919b700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a78066f30 0x7f8a780693e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8a900099b0 tx=0x7f8a90017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.095+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a84050d60 con 0x7f8a940ad460 2023-10-06T05:26:18.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.342+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8a94159dc0 con 0x7f8a940ad460 2023-10-06T05:26:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:18.999+0000 7f8a897fa700 1 -- 172.21.15.33:0/1693373838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v123) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8a84017020 con 0x7f8a940ad460 2023-10-06T05:26:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a78066f30 msgr2=0x7f8a780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a78066f30 0x7f8a780693e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8a900099b0 tx=0x7f8a90017000 comp rx=0 tx=0).stop 2023-10-06T05:26:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 msgr2=0x7f8a9415bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 0x7f8a9415bac0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400c5c0 tx=0x7f8a8400c980 comp rx=0 tx=0).stop 2023-10-06T05:26:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 shutdown_connections 2023-10-06T05:26:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a78066f30 0x7f8a780693e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a941595d0 0x7f8a940c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a940ac180 0x7f8a94143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 --2- 172.21.15.33:0/1693373838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a940ad460 0x7f8a9415bac0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 >> 172.21.15.33:0/1693373838 conn(0x7f8a9406d030 msgr2=0x7f8a94153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 shutdown_connections 2023-10-06T05:26:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.002+0000 7f8a9a19d700 1 -- 172.21.15.33:0/1693373838 wait complete. 2023-10-06T05:26:19.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:26:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.218+0000 7fa135f9d700 1 Processor -- start 2023-10-06T05:26:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.218+0000 7fa135f9d700 1 -- start start 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa135f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa1300aff90 0x7fa1300a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa135f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13015d3a0 con 0x7fa13006c7b0 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa135f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1300b0470 con 0x7fa13006cb80 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa135f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1300686c0 con 0x7fa1300aff90 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa134f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa13006c7b0 0x7fa130067fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50930/0 (socket says 172.21.15.33:50930) 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa134f9b700 1 -- 172.21.15.33:0/870098754 learned_addr learned my addr 172.21.15.33:0/870098754 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.219+0000 7fa12ffff700 1 --2- 172.21.15.33:0/870098754 >> v2:172.21.15.167:3300/0 conn(0x7fa1300aff90 0x7fa1300a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.220+0000 7fa12f7fe700 1 -- 172.21.15.33:0/870098754 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368301323 0 0) 0x7fa13015d3a0 con 0x7fa13006c7b0 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.220+0000 7fa12f7fe700 1 -- 172.21.15.33:0/870098754 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa120003570 con 0x7fa13006c7b0 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.220+0000 7fa12ffff700 1 -- 172.21.15.33:0/870098754 >> v1:172.21.15.167:6789/0 conn(0x7fa13006cb80 legacy=0x7fa13015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.220+0000 7fa12ffff700 1 -- 172.21.15.33:0/870098754 >> v1:172.21.15.33:6789/0 conn(0x7fa13006c7b0 legacy=0x7fa130067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.220+0000 7fa12ffff700 1 -- 172.21.15.33:0/870098754 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300b0750 con 0x7fa1300aff90 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.220+0000 7fa12ffff700 1 --2- 172.21.15.33:0/870098754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1300aff90 0x7fa1300a9890 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098e0 tx=0x7fa11c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ead13e6cf47f1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa12f7fe700 1 -- 172.21.15.33:0/870098754 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00b640 con 0x7fa1300aff90 2023-10-06T05:26:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa12f7fe700 1 -- 172.21.15.33:0/870098754 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00b7a0 con 0x7fa1300aff90 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa12f7fe700 1 -- 172.21.15.33:0/870098754 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00bb40 con 0x7fa1300aff90 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa135f9d700 1 -- 172.21.15.33:0/870098754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1300aff90 msgr2=0x7fa1300a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa135f9d700 1 --2- 172.21.15.33:0/870098754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1300aff90 0x7fa1300a9890 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098e0 tx=0x7fa11c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa135f9d700 1 -- 172.21.15.33:0/870098754 shutdown_connections 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa135f9d700 1 --2- 172.21.15.33:0/870098754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1300aff90 0x7fa1300a9890 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa135f9d700 1 -- 172.21.15.33:0/870098754 >> 172.21.15.33:0/870098754 conn(0x7fa13006d670 msgr2=0x7fa13006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.221+0000 7fa135f9d700 1 -- 172.21.15.33:0/870098754 shutdown_connections 2023-10-06T05:26:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.222+0000 7fa135f9d700 1 -- 172.21.15.33:0/870098754 wait complete. 2023-10-06T05:26:19.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.223+0000 7fa135f9d700 1 Processor -- start 2023-10-06T05:26:19.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.223+0000 7fa135f9d700 1 -- start start 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa135f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13006c7b0 0x7fa13014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa135f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13006cb80 0x7fa130139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa135f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 0x7fa13016d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa135f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13014b9e0 con 0x7fa1300aff90 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa135f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13016e410 con 0x7fa13006c7b0 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa135f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13014a420 con 0x7fa13006cb80 2023-10-06T05:26:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa13579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 0x7fa13016d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa13579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 0x7fa13016d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51000/0 (socket says 172.21.15.33:51000) 2023-10-06T05:26:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.224+0000 7fa13579c700 1 -- 172.21.15.33:0/2198136619 learned_addr learned my addr 172.21.15.33:0/2198136619 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa12ffff700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13006cb80 0x7fa130139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa134f9b700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13006c7b0 0x7fa13014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa13579c700 1 -- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13006cb80 msgr2=0x7fa130139720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa13579c700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13006cb80 0x7fa130139720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa13579c700 1 -- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13006c7b0 msgr2=0x7fa13014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa13579c700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13006c7b0 0x7fa13014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa13579c700 1 -- 172.21.15.33:0/2198136619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c009420 con 0x7fa1300aff90 2023-10-06T05:26:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa13579c700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 0x7fa13016d5c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c8e0 tx=0x7fa128001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.225+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128012630 con 0x7fa1300aff90 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.226+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa128012790 con 0x7fa1300aff90 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.226+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128012b50 con 0x7fa1300aff90 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.226+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13016e130 con 0x7fa1300aff90 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.226+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13014b670 con 0x7fa1300aff90 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.227+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13014bb50 con 0x7fa1300aff90 2023-10-06T05:26:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.228+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa12801a3a0 con 0x7fa1300aff90 2023-10-06T05:26:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.228+0000 7fa12cff9700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa114066f60 0x7fa114069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.229+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1280848a0 con 0x7fa1300aff90 2023-10-06T05:26:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.229+0000 7fa134f9b700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa114066f60 0x7fa114069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.230+0000 7fa134f9b700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa114066f60 0x7fa114069410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa124001360 tx=0x7fa124009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.233+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa128051910 con 0x7fa1300aff90 2023-10-06T05:26:19.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:19.484+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa13004c380 con 0x7fa1300aff90 2023-10-06T05:26:19.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:19.782 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.012+0000 7fa12cff9700 1 -- 172.21.15.33:0/2198136619 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v124) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa128054f30 con 0x7fa1300aff90 2023-10-06T05:26:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa114066f60 msgr2=0x7fa114069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa114066f60 0x7fa114069410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa124001360 tx=0x7fa124009040 comp rx=0 tx=0).stop 2023-10-06T05:26:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 msgr2=0x7fa13016d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 0x7fa13016d5c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c8e0 tx=0x7fa128001710 comp rx=0 tx=0).stop 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 shutdown_connections 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa114066f60 0x7fa114069410 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1300aff90 0x7fa13016d5c0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13006c7b0 0x7fa13014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 --2- 172.21.15.33:0/2198136619 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13006cb80 0x7fa130139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 >> 172.21.15.33:0/2198136619 conn(0x7fa13006d670 msgr2=0x7fa1300ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 shutdown_connections 2023-10-06T05:26:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.015+0000 7fa135f9d700 1 -- 172.21.15.33:0/2198136619 wait complete. 2023-10-06T05:26:20.038 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:20.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.223+0000 7fd59359e700 1 Processor -- start 2023-10-06T05:26:20.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.223+0000 7fd59359e700 1 -- start start 2023-10-06T05:26:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.224+0000 7fd59359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd59413faa0 0x7fd59413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.224+0000 7fd59359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5940c7940 0x7fd5940c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.224+0000 7fd59359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5940c9cb0 con 0x7fd59406aad0 2023-10-06T05:26:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.224+0000 7fd59359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59406b660 con 0x7fd59413faa0 2023-10-06T05:26:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.224+0000 7fd59359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940ca270 con 0x7fd5940c7940 2023-10-06T05:26:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd592d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd59406aad0 0x7fd59406af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47986/0 (socket says 172.21.15.33:47986) 2023-10-06T05:26:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd592d9d700 1 -- 172.21.15.33:0/3127623932 learned_addr learned my addr 172.21.15.33:0/3127623932 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59259c700 1 --2- 172.21.15.33:0/3127623932 >> v2:172.21.15.33:3300/0 conn(0x7fd59413faa0 0x7fd59413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd591d9b700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5940c7940 0x7fd5940c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59159a700 1 -- 172.21.15.33:0/3127623932 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138939244 0 0) 0x7fd5940c9cb0 con 0x7fd59406aad0 2023-10-06T05:26:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59259c700 1 -- 172.21.15.33:0/3127623932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5940c7940 msgr2=0x7fd5940c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59259c700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5940c7940 0x7fd5940c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59259c700 1 -- 172.21.15.33:0/3127623932 >> v1:172.21.15.33:6789/0 conn(0x7fd59406aad0 legacy=0x7fd59406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59259c700 1 -- 172.21.15.33:0/3127623932 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59406b840 con 0x7fd59413faa0 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd591d9b700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5940c7940 0x7fd5940c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.225+0000 7fd59259c700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd59413fe70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd5840098e0 tx=0x7fd5840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0629a40f173a280 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59159a700 1 -- 172.21.15.33:0/3127623932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584009e80 con 0x7fd59413faa0 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59159a700 1 -- 172.21.15.33:0/3127623932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58400b860 con 0x7fd59413faa0 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59159a700 1 -- 172.21.15.33:0/3127623932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58400bbe0 con 0x7fd59413faa0 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 -- 172.21.15.33:0/3127623932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 msgr2=0x7fd59413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd59413fe70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd5840098e0 tx=0x7fd5840091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 -- 172.21.15.33:0/3127623932 shutdown_connections 2023-10-06T05:26:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd59413fe70 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 --2- 172.21.15.33:0/3127623932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5940c7940 0x7fd5940c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 -- 172.21.15.33:0/3127623932 >> 172.21.15.33:0/3127623932 conn(0x7fd59406d030 msgr2=0x7fd59406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 -- 172.21.15.33:0/3127623932 shutdown_connections 2023-10-06T05:26:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.226+0000 7fd59359e700 1 -- 172.21.15.33:0/3127623932 wait complete. 2023-10-06T05:26:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.228+0000 7fd59359e700 1 Processor -- start 2023-10-06T05:26:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.228+0000 7fd59359e700 1 -- start start 2023-10-06T05:26:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd59406aad0 0x7fd59413eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5940c7940 0x7fd5940c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd5941675d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941685e0 con 0x7fd59413faa0 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940c5830 con 0x7fd5940c7940 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd594168020 con 0x7fd59406aad0 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd592d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd5941675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd592d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd5941675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48320/0 (socket says 172.21.15.33:48320) 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd592d9d700 1 -- 172.21.15.33:0/617838500 learned_addr learned my addr 172.21.15.33:0/617838500 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd59259c700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd59406aad0 0x7fd59413eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.229+0000 7fd591d9b700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5940c7940 0x7fd5940c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd592d9d700 1 -- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd59406aad0 msgr2=0x7fd59413eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd592d9d700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd59406aad0 0x7fd59413eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd592d9d700 1 -- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5940c7940 msgr2=0x7fd5940c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd592d9d700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5940c7940 0x7fd5940c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd592d9d700 1 -- 172.21.15.33:0/617838500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584009420 con 0x7fd59413faa0 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd592d9d700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd5941675d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00cc90 tx=0x7fd58c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00a850 con 0x7fd59413faa0 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c00a9b0 con 0x7fd59413faa0 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00ad70 con 0x7fd59413faa0 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594168300 con 0x7fd59413faa0 2023-10-06T05:26:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.230+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5940c3350 con 0x7fd59413faa0 2023-10-06T05:26:20.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.231+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5940c5320 con 0x7fd59413faa0 2023-10-06T05:26:20.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.232+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd58c007300 con 0x7fd59413faa0 2023-10-06T05:26:20.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.233+0000 7fd5827fc700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd574066f30 0x7fd5740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.233+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd58c084a50 con 0x7fd59413faa0 2023-10-06T05:26:20.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.234+0000 7fd59259c700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd574066f30 0x7fd5740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.235+0000 7fd59259c700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd574066f30 0x7fd5740693e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd5840091e0 tx=0x7fd584017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.237+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5940c5320 con 0x7fd59413faa0 2023-10-06T05:26:20.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.487+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5940c5320 con 0x7fd59413faa0 2023-10-06T05:26:20.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.488+0000 7fd5827fc700 1 -- 172.21.15.33:0/617838500 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fd5940c5320 con 0x7fd59413faa0 2023-10-06T05:26:20.491 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:20.491 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:20.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd574066f30 msgr2=0x7fd5740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd574066f30 0x7fd5740693e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd5840091e0 tx=0x7fd584017000 comp rx=0 tx=0).stop 2023-10-06T05:26:20.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 msgr2=0x7fd5941675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd5941675d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00cc90 tx=0x7fd58c001a80 comp rx=0 tx=0).stop 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 shutdown_connections 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd574066f30 0x7fd5740693e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd59413faa0 0x7fd5941675d0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd59406aad0 0x7fd59413eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 --2- 172.21.15.33:0/617838500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5940c7940 0x7fd5940c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.490+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 >> 172.21.15.33:0/617838500 conn(0x7fd59406d030 msgr2=0x7fd5940c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.491+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 shutdown_connections 2023-10-06T05:26:20.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.491+0000 7fd59359e700 1 -- 172.21.15.33:0/617838500 wait complete. 2023-10-06T05:26:20.495 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:20.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:20.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.695+0000 7f0db2f9d700 1 Processor -- start 2023-10-06T05:26:20.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.695+0000 7f0db2f9d700 1 -- start start 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.696+0000 7f0db2f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dac0cb710 0x7f0dac06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.696+0000 7f0db2f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0c9cb0 con 0x7f0dac0c8050 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.696+0000 7f0db2f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0ca270 con 0x7f0dac13faa0 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.696+0000 7f0db2f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac06b840 con 0x7f0dac0cb710 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db179a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0dac0c8050 0x7f0dac06aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48000/0 (socket says 172.21.15.33:48000) 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db179a700 1 -- 172.21.15.33:0/1907765514 learned_addr learned my addr 172.21.15.33:0/1907765514 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db279c700 1 --2- 172.21.15.33:0/1907765514 >> v2:172.21.15.167:3300/0 conn(0x7f0dac0cb710 0x7f0dac06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db0f99700 1 -- 172.21.15.33:0/1907765514 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733166265 0 0) 0x7f0dac0c9cb0 con 0x7f0dac0c8050 2023-10-06T05:26:20.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db0f99700 1 -- 172.21.15.33:0/1907765514 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d98003570 con 0x7f0dac0c8050 2023-10-06T05:26:20.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db279c700 1 -- 172.21.15.33:0/1907765514 >> v1:172.21.15.167:6789/0 conn(0x7f0dac13faa0 legacy=0x7f0dac0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db279c700 1 -- 172.21.15.33:0/1907765514 >> v1:172.21.15.33:6789/0 conn(0x7f0dac0c8050 legacy=0x7f0dac06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.697+0000 7f0db279c700 1 -- 172.21.15.33:0/1907765514 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac06bb20 con 0x7f0dac0cb710 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.698+0000 7f0db279c700 1 --2- 172.21.15.33:0/1907765514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac06b1e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009e60 tx=0x7f0da4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5a696c62186f54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.698+0000 7f0db0f99700 1 -- 172.21.15.33:0/1907765514 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da400e070 con 0x7f0dac0cb710 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.698+0000 7f0db0f99700 1 -- 172.21.15.33:0/1907765514 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da400bb60 con 0x7f0dac0cb710 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.698+0000 7f0db0f99700 1 -- 172.21.15.33:0/1907765514 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4012430 con 0x7f0dac0cb710 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.698+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1907765514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 msgr2=0x7f0dac06b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.698+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1907765514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac06b1e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009e60 tx=0x7f0da4009780 comp rx=0 tx=0).stop 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.699+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1907765514 shutdown_connections 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.699+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1907765514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac06b1e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.699+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1907765514 >> 172.21.15.33:0/1907765514 conn(0x7f0dac06d030 msgr2=0x7f0dac06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.699+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1907765514 shutdown_connections 2023-10-06T05:26:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.699+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1907765514 wait complete. 2023-10-06T05:26:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.700+0000 7f0db2f9d700 1 Processor -- start 2023-10-06T05:26:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.700+0000 7f0db2f9d700 1 -- start start 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dac0c8050 0x7f0dac1678f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db2f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 0x7f0dac13be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db2f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac167e30 con 0x7f0dac13faa0 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac15d150 con 0x7f0dac0cb710 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac156190 con 0x7f0dac0c8050 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 0x7f0dac13be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 0x7f0dac13be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48336/0 (socket says 172.21.15.33:48336) 2023-10-06T05:26:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.701+0000 7f0db279c700 1 -- 172.21.15.33:0/1313674174 learned_addr learned my addr 172.21.15.33:0/1313674174 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db179a700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db1f9b700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dac0c8050 0x7f0dac1678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db279c700 1 -- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dac0c8050 msgr2=0x7f0dac1678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db279c700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dac0c8050 0x7f0dac1678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db279c700 1 -- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 msgr2=0x7f0dac155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db279c700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db279c700 1 -- 172.21.15.33:0/1313674174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4009a20 con 0x7f0dac13faa0 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db1f9b700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dac0c8050 0x7f0dac1678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db279c700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 0x7f0dac13be10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0719c0 tx=0x7f0da4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4012e90 con 0x7f0dac13faa0 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da401d630 con 0x7f0dac13faa0 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.702+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac13cb60 con 0x7f0dac13faa0 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.703+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da401d9f0 con 0x7f0dac13faa0 2023-10-06T05:26:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.703+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac15cb00 con 0x7f0dac13faa0 2023-10-06T05:26:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.703+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac15d2e0 con 0x7f0dac13faa0 2023-10-06T05:26:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.704+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0da4024070 con 0x7f0dac13faa0 2023-10-06T05:26:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.705+0000 7f0da1ffb700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d90066f30 0x7f0d900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.705+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0da4086fb0 con 0x7f0dac13faa0 2023-10-06T05:26:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.706+0000 7f0db1f9b700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d90066f30 0x7f0d900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.706+0000 7f0db1f9b700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d90066f30 0x7f0d900693e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0da80017a0 tx=0x7f0da8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.709+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0da4054020 con 0x7f0dac13faa0 2023-10-06T05:26:20.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.915+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dac04c3b0 con 0x7f0dac13faa0 2023-10-06T05:26:20.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.916+0000 7f0da1ffb700 1 -- 172.21.15.33:0/1313674174 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f0da4057640 con 0x7f0dac13faa0 2023-10-06T05:26:20.919 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:17.990991+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:14.643519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:15.441428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d90066f30 msgr2=0x7f0d900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d90066f30 0x7f0d900693e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0da80017a0 tx=0x7f0da8009040 comp rx=0 tx=0).stop 2023-10-06T05:26:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 msgr2=0x7f0dac13be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 0x7f0dac13be10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0719c0 tx=0x7f0da4019000 comp rx=0 tx=0).stop 2023-10-06T05:26:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 shutdown_connections 2023-10-06T05:26:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d90066f30 0x7f0d900693e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dac13faa0 0x7f0dac13be10 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dac0c8050 0x7f0dac1678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 --2- 172.21.15.33:0/1313674174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dac0cb710 0x7f0dac155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 >> 172.21.15.33:0/1313674174 conn(0x7f0dac06d030 msgr2=0x7f0dac0c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 shutdown_connections 2023-10-06T05:26:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:20.919+0000 7f0db2f9d700 1 -- 172.21.15.33:0/1313674174 wait complete. 2023-10-06T05:26:20.939 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:26:20.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:20.948 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:26:20.945+0000 7fcb3f1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:21.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.124+0000 7fc02499d700 1 Processor -- start 2023-10-06T05:26:21.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.124+0000 7fc02499d700 1 -- start start 2023-10-06T05:26:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc02499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0200c8b50 0x7fc0200c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc02499d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc020155ec0 0x7fc0200c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc02499d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02006ab60 con 0x7fc020155af0 2023-10-06T05:26:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc02499d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02006b7e0 con 0x7fc020155ec0 2023-10-06T05:26:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc02499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02006ae40 con 0x7fc0200c8b50 2023-10-06T05:26:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc01f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc020155af0 0x7fc02006a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48010/0 (socket says 172.21.15.33:48010) 2023-10-06T05:26:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc01f7fe700 1 -- 172.21.15.33:0/2351680865 learned_addr learned my addr 172.21.15.33:0/2351680865 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc01ffff700 1 --2- 172.21.15.33:0/2351680865 >> v2:172.21.15.33:3300/0 conn(0x7fc020155ec0 0x7fc0200c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc01e7fc700 1 -- 172.21.15.33:0/2351680865 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911088891 0 0) 0x7fc02006ab60 con 0x7fc020155af0 2023-10-06T05:26:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc01effd700 1 --2- 172.21.15.33:0/2351680865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0200c8b50 0x7fc0200c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.126+0000 7fc01e7fc700 1 -- 172.21.15.33:0/2351680865 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc008003570 con 0x7fc020155af0 2023-10-06T05:26:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01ffff700 1 -- 172.21.15.33:0/2351680865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0200c8b50 msgr2=0x7fc0200c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01ffff700 1 --2- 172.21.15.33:0/2351680865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0200c8b50 0x7fc0200c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01ffff700 1 -- 172.21.15.33:0/2351680865 >> v1:172.21.15.33:6789/0 conn(0x7fc020155af0 legacy=0x7fc02006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01ffff700 1 -- 172.21.15.33:0/2351680865 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02006b120 con 0x7fc020155ec0 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01ffff700 1 --2- 172.21.15.33:0/2351680865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc020155ec0 0x7fc0200c7750 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc01800af70 tx=0x7fc01800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e789158816f81a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01e7fc700 1 -- 172.21.15.33:0/2351680865 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01800f070 con 0x7fc020155ec0 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01e7fc700 1 -- 172.21.15.33:0/2351680865 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01800a2e0 con 0x7fc020155ec0 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.127+0000 7fc01e7fc700 1 -- 172.21.15.33:0/2351680865 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01800cc70 con 0x7fc020155ec0 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 -- 172.21.15.33:0/2351680865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc020155ec0 msgr2=0x7fc0200c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 --2- 172.21.15.33:0/2351680865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc020155ec0 0x7fc0200c7750 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc01800af70 tx=0x7fc01800a890 comp rx=0 tx=0).stop 2023-10-06T05:26:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 -- 172.21.15.33:0/2351680865 shutdown_connections 2023-10-06T05:26:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 --2- 172.21.15.33:0/2351680865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc020155ec0 0x7fc0200c7750 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 --2- 172.21.15.33:0/2351680865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0200c8b50 0x7fc0200c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 -- 172.21.15.33:0/2351680865 >> 172.21.15.33:0/2351680865 conn(0x7fc02006d030 msgr2=0x7fc02006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 -- 172.21.15.33:0/2351680865 shutdown_connections 2023-10-06T05:26:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.128+0000 7fc02499d700 1 -- 172.21.15.33:0/2351680865 wait complete. 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.129+0000 7fc02499d700 1 Processor -- start 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.129+0000 7fc02499d700 1 -- start start 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc02499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 0x7fc020151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc02499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020155af0 0x7fc0200ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc02499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc020155ec0 0x7fc020159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc02499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c3190 con 0x7fc0200c8b50 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc02499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c3750 con 0x7fc020155af0 2023-10-06T05:26:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc02499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201517d0 con 0x7fc020155ec0 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.130+0000 7fc01f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 0x7fc020151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 0x7fc020151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48368/0 (socket says 172.21.15.33:48368) 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 -- 172.21.15.33:0/3454301077 learned_addr learned my addr 172.21.15.33:0/3454301077 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 -- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc020155ec0 msgr2=0x7fc020159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01ffff700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc020155ec0 0x7fc020159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01effd700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020155af0 0x7fc0200ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc020155ec0 0x7fc020159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 -- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020155af0 msgr2=0x7fc0200ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020155af0 0x7fc0200ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 -- 172.21.15.33:0/3454301077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01800ab30 con 0x7fc0200c8b50 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01effd700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020155af0 0x7fc0200ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01ffff700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc020155ec0 0x7fc020159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc01f7fe700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 0x7fc020151290 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc01000c7c0 tx=0x7fc010007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010005b30 con 0x7fc0200c8b50 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010005c90 con 0x7fc0200c8b50 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.131+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010007340 con 0x7fc0200c8b50 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.132+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200ac810 con 0x7fc0200c8b50 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.132+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0200c33e0 con 0x7fc0200c8b50 2023-10-06T05:26:21.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.133+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02004c380 con 0x7fc0200c8b50 2023-10-06T05:26:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.134+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc01001b400 con 0x7fc0200c8b50 2023-10-06T05:26:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.135+0000 7fc0077fe700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc00c066f30 0x7fc00c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.135+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc010084240 con 0x7fc0200c8b50 2023-10-06T05:26:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.136+0000 7fc01effd700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc00c066f30 0x7fc00c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.136+0000 7fc01effd700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc00c066f30 0x7fc00c0693e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc02016ab70 tx=0x7fc014008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.139+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc010051330 con 0x7fc0200c8b50 2023-10-06T05:26:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.338+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc02006b400 con 0x7fc00c066f30 2023-10-06T05:26:21.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.349+0000 7fc0077fe700 1 -- 172.21.15.33:0/3454301077 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308942 (secure 0 0 0) 0x7fc02006b400 con 0x7fc00c066f30 2023-10-06T05:26:21.638 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:21.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.352+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc00c066f30 msgr2=0x7fc00c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.352+0000 7fc02499d700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc00c066f30 0x7fc00c0693e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc02016ab70 tx=0x7fc014008040 comp rx=0 tx=0).stop 2023-10-06T05:26:21.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.352+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 msgr2=0x7fc020151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.352+0000 7fc02499d700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 0x7fc020151290 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc01000c7c0 tx=0x7fc010007e00 comp rx=0 tx=0).stop 2023-10-06T05:26:21.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 shutdown_connections 2023-10-06T05:26:21.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc00c066f30 0x7fc00c0693e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc020155ec0 0x7fc020159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0200c8b50 0x7fc020151290 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 --2- 172.21.15.33:0/3454301077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc020155af0 0x7fc0200ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 >> 172.21.15.33:0/3454301077 conn(0x7fc02006d030 msgr2=0x7fc0200cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 shutdown_connections 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.353+0000 7fc02499d700 1 -- 172.21.15.33:0/3454301077 wait complete. 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:26:21.642 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":526,"stamp":"2023-10-06T05:26:19.995892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428648,"kb_used_data":17576,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546072,"statfs":{"total":773094113280,"available":770607177728,"internally_reserved":0,"allocated":17997824,"data_stored":8955280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.034529"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.979779+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:04.234830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976331+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:20.532361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971249+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:23.955976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971890+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:03.256316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.978679+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:30.880364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976831+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:56.462173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.975073+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:49.334859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.972770+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:38.213234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.978543+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:58.412839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.975886+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:09.004752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.977023+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:26.304792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971328+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.796608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.985776+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:03.189897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.973239+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:33.433911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976265+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:52.860509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.981067+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.796608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.981827+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:08.247894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.986053+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:26.116974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.983078+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:15.580935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.981429+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:03.256316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976215+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:03.829797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.975518+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:32.931988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971870+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:12.238141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.974903+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:26.116974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.973181+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:20.424498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.977345+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:31.121954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.973224+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:43.567138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.977408+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:42.450003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.975631+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:55.287981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.975694+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:19.986057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971685+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:40.244614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976767+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:36.010312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971690+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.887934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.988983+0000","last_change":"2023-10-06T05:26:15.988983+0000","last_active":"2023-10-06T05:26:15.988983+0000","last_peered":"2023-10-06T05:26:15.988983+0000","last_clean":"2023-10-06T05:26:15.988983+0000","last_became_active":"2023-10-06T05:26:15.988733+0000","last_became_peered":"2023-10-06T05:26:15.988733+0000","last_unstale":"2023-10-06T05:26:15.988983+0000","last_undegraded":"2023-10-06T05:26:15.988983+0000","last_fullsized":"2023-10-06T05:26:15.988983+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:08.247585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.978479+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:54.709241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.986339+0000","last_change":"2023-10-06T05:26:15.986339+0000","last_active":"2023-10-06T05:26:15.986339+0000","last_peered":"2023-10-06T05:26:15.986339+0000","last_clean":"2023-10-06T05:26:15.986339+0000","last_became_active":"2023-10-06T05:26:15.986049+0000","last_became_peered":"2023-10-06T05:26:15.986049+0000","last_unstale":"2023-10-06T05:26:15.986339+0000","last_undegraded":"2023-10-06T05:26:15.986339+0000","last_fullsized":"2023-10-06T05:26:15.986339+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:21.716854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.985502+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:53.327551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.990391+0000","last_change":"2023-10-06T05:26:15.990391+0000","last_active":"2023-10-06T05:26:15.990391+0000","last_peered":"2023-10-06T05:26:15.990391+0000","last_clean":"2023-10-06T05:26:15.990391+0000","last_became_active":"2023-10-06T05:26:15.989737+0000","last_became_peered":"2023-10-06T05:26:15.989737+0000","last_unstale":"2023-10-06T05:26:15.990391+0000","last_undegraded":"2023-10-06T05:26:15.990391+0000","last_fullsized":"2023-10-06T05:26:15.990391+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:46.587385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.974261+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:53.327551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.989306+0000","last_change":"2023-10-06T05:26:15.989306+0000","last_active":"2023-10-06T05:26:15.989306+0000","last_peered":"2023-10-06T05:26:15.989306+0000","last_clean":"2023-10-06T05:26:15.989306+0000","last_became_active":"2023-10-06T05:26:15.988911+0000","last_became_peered":"2023-10-06T05:26:15.988911+0000","last_unstale":"2023-10-06T05:26:15.989306+0000","last_undegraded":"2023-10-06T05:26:15.989306+0000","last_fullsized":"2023-10-06T05:26:15.989306+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:25.299022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.970480+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:32.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.992259+0000","last_change":"2023-10-06T05:26:15.992259+0000","last_active":"2023-10-06T05:26:15.992259+0000","last_peered":"2023-10-06T05:26:15.992259+0000","last_clean":"2023-10-06T05:26:15.992259+0000","last_became_active":"2023-10-06T05:26:15.991988+0000","last_became_peered":"2023-10-06T05:26:15.991988+0000","last_unstale":"2023-10-06T05:26:15.992259+0000","last_undegraded":"2023-10-06T05:26:15.992259+0000","last_fullsized":"2023-10-06T05:26:15.992259+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:30.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.977357+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:03.224756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.978359+0000","last_change":"2023-10-06T05:26:15.978359+0000","last_active":"2023-10-06T05:26:15.978359+0000","last_peered":"2023-10-06T05:26:15.978359+0000","last_clean":"2023-10-06T05:26:15.978359+0000","last_became_active":"2023-10-06T05:26:15.978104+0000","last_became_peered":"2023-10-06T05:26:15.978104+0000","last_unstale":"2023-10-06T05:26:15.978359+0000","last_undegraded":"2023-10-06T05:26:15.978359+0000","last_fullsized":"2023-10-06T05:26:15.978359+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:30.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.981786+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:28.219238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.989840+0000","last_change":"2023-10-06T05:26:15.989840+0000","last_active":"2023-10-06T05:26:15.989840+0000","last_peered":"2023-10-06T05:26:15.989840+0000","last_clean":"2023-10-06T05:26:15.989840+0000","last_became_active":"2023-10-06T05:26:15.989513+0000","last_became_peered":"2023-10-06T05:26:15.989513+0000","last_unstale":"2023-10-06T05:26:15.989840+0000","last_undegraded":"2023-10-06T05:26:15.989840+0000","last_fullsized":"2023-10-06T05:26:15.989840+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:22.175645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.985254+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:53.575234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.983222+0000","last_change":"2023-10-06T05:26:15.983222+0000","last_active":"2023-10-06T05:26:15.983222+0000","last_peered":"2023-10-06T05:26:15.983222+0000","last_clean":"2023-10-06T05:26:15.983222+0000","last_became_active":"2023-10-06T05:26:15.982969+0000","last_became_peered":"2023-10-06T05:26:15.982969+0000","last_unstale":"2023-10-06T05:26:15.983222+0000","last_undegraded":"2023-10-06T05:26:15.983222+0000","last_fullsized":"2023-10-06T05:26:15.983222+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:58.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.970361+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:52.480336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.983830+0000","last_change":"2023-10-06T05:26:15.983830+0000","last_active":"2023-10-06T05:26:15.983830+0000","last_peered":"2023-10-06T05:26:15.983830+0000","last_clean":"2023-10-06T05:26:15.983830+0000","last_became_active":"2023-10-06T05:26:15.983574+0000","last_became_peered":"2023-10-06T05:26:15.983574+0000","last_unstale":"2023-10-06T05:26:15.983830+0000","last_undegraded":"2023-10-06T05:26:15.983830+0000","last_fullsized":"2023-10-06T05:26:15.983830+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:17.508902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.977998+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:23.181415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.989957+0000","last_change":"2023-10-06T05:26:15.989957+0000","last_active":"2023-10-06T05:26:15.989957+0000","last_peered":"2023-10-06T05:26:15.989957+0000","last_clean":"2023-10-06T05:26:15.989957+0000","last_became_active":"2023-10-06T05:26:15.989449+0000","last_became_peered":"2023-10-06T05:26:15.989449+0000","last_unstale":"2023-10-06T05:26:15.989957+0000","last_undegraded":"2023-10-06T05:26:15.989957+0000","last_fullsized":"2023-10-06T05:26:15.989957+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:30.253659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976202+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:27.555065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.995078+0000","last_change":"2023-10-06T05:26:15.995078+0000","last_active":"2023-10-06T05:26:15.995078+0000","last_peered":"2023-10-06T05:26:15.995078+0000","last_clean":"2023-10-06T05:26:15.995078+0000","last_became_active":"2023-10-06T05:26:15.994684+0000","last_became_peered":"2023-10-06T05:26:15.994684+0000","last_unstale":"2023-10-06T05:26:15.995078+0000","last_undegraded":"2023-10-06T05:26:15.995078+0000","last_fullsized":"2023-10-06T05:26:15.995078+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:19.418728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971904+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:53.575234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.981157+0000","last_change":"2023-10-06T05:26:15.981157+0000","last_active":"2023-10-06T05:26:15.981157+0000","last_peered":"2023-10-06T05:26:15.981157+0000","last_clean":"2023-10-06T05:26:15.981157+0000","last_became_active":"2023-10-06T05:26:15.980886+0000","last_became_peered":"2023-10-06T05:26:15.980886+0000","last_unstale":"2023-10-06T05:26:15.981157+0000","last_undegraded":"2023-10-06T05:26:15.981157+0000","last_fullsized":"2023-10-06T05:26:15.981157+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:55.779205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971199+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:15.580935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.996997+0000","last_change":"2023-10-06T05:26:15.996997+0000","last_active":"2023-10-06T05:26:15.996997+0000","last_peered":"2023-10-06T05:26:15.996997+0000","last_clean":"2023-10-06T05:26:15.996997+0000","last_became_active":"2023-10-06T05:26:15.996768+0000","last_became_peered":"2023-10-06T05:26:15.996768+0000","last_unstale":"2023-10-06T05:26:15.996997+0000","last_undegraded":"2023-10-06T05:26:15.996997+0000","last_fullsized":"2023-10-06T05:26:15.996997+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:36.365849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.973166+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:08.247894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.979984+0000","last_change":"2023-10-06T05:26:15.979984+0000","last_active":"2023-10-06T05:26:15.979984+0000","last_peered":"2023-10-06T05:26:15.979984+0000","last_clean":"2023-10-06T05:26:15.979984+0000","last_became_active":"2023-10-06T05:26:15.979679+0000","last_became_peered":"2023-10-06T05:26:15.979679+0000","last_unstale":"2023-10-06T05:26:15.979984+0000","last_undegraded":"2023-10-06T05:26:15.979984+0000","last_fullsized":"2023-10-06T05:26:15.979984+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:02.218986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.973502+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:48.026485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.987088+0000","last_change":"2023-10-06T05:26:15.987088+0000","last_active":"2023-10-06T05:26:15.987088+0000","last_peered":"2023-10-06T05:26:15.987088+0000","last_clean":"2023-10-06T05:26:15.987088+0000","last_became_active":"2023-10-06T05:26:15.986754+0000","last_became_peered":"2023-10-06T05:26:15.986754+0000","last_unstale":"2023-10-06T05:26:15.987088+0000","last_undegraded":"2023-10-06T05:26:15.987088+0000","last_fullsized":"2023-10-06T05:26:15.987088+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:09.197792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.977596+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:31.259429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.979209+0000","last_change":"2023-10-06T05:26:15.979209+0000","last_active":"2023-10-06T05:26:15.979209+0000","last_peered":"2023-10-06T05:26:15.979209+0000","last_clean":"2023-10-06T05:26:15.979209+0000","last_became_active":"2023-10-06T05:26:15.978945+0000","last_became_peered":"2023-10-06T05:26:15.978945+0000","last_unstale":"2023-10-06T05:26:15.979209+0000","last_undegraded":"2023-10-06T05:26:15.979209+0000","last_fullsized":"2023-10-06T05:26:15.979209+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:48.329089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.969715+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:41.361021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.981648+0000","last_change":"2023-10-06T05:26:15.981648+0000","last_active":"2023-10-06T05:26:15.981648+0000","last_peered":"2023-10-06T05:26:15.981648+0000","last_clean":"2023-10-06T05:26:15.981648+0000","last_became_active":"2023-10-06T05:26:15.980699+0000","last_became_peered":"2023-10-06T05:26:15.980699+0000","last_unstale":"2023-10-06T05:26:15.981648+0000","last_undegraded":"2023-10-06T05:26:15.981648+0000","last_fullsized":"2023-10-06T05:26:15.981648+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:07.998982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.970656+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:28.219238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.986229+0000","last_change":"2023-10-06T05:26:15.986229+0000","last_active":"2023-10-06T05:26:15.986229+0000","last_peered":"2023-10-06T05:26:15.986229+0000","last_clean":"2023-10-06T05:26:15.986229+0000","last_became_active":"2023-10-06T05:26:15.985959+0000","last_became_peered":"2023-10-06T05:26:15.985959+0000","last_unstale":"2023-10-06T05:26:15.986229+0000","last_undegraded":"2023-10-06T05:26:15.986229+0000","last_fullsized":"2023-10-06T05:26:15.986229+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:34.700237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.972839+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:06.789461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.984823+0000","last_change":"2023-10-06T05:26:15.984823+0000","last_active":"2023-10-06T05:26:15.984823+0000","last_peered":"2023-10-06T05:26:15.984823+0000","last_clean":"2023-10-06T05:26:15.984823+0000","last_became_active":"2023-10-06T05:26:15.984551+0000","last_became_peered":"2023-10-06T05:26:15.984551+0000","last_unstale":"2023-10-06T05:26:15.984823+0000","last_undegraded":"2023-10-06T05:26:15.984823+0000","last_fullsized":"2023-10-06T05:26:15.984823+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:42.561369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.969995+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:10.608112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.993515+0000","last_change":"2023-10-06T05:26:15.993515+0000","last_active":"2023-10-06T05:26:15.993515+0000","last_peered":"2023-10-06T05:26:15.993515+0000","last_clean":"2023-10-06T05:26:15.993515+0000","last_became_active":"2023-10-06T05:26:15.993249+0000","last_became_peered":"2023-10-06T05:26:15.993249+0000","last_unstale":"2023-10-06T05:26:15.993515+0000","last_undegraded":"2023-10-06T05:26:15.993515+0000","last_fullsized":"2023-10-06T05:26:15.993515+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:45.111652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.970867+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:41.093677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.990744+0000","last_change":"2023-10-06T05:26:15.990744+0000","last_active":"2023-10-06T05:26:15.990744+0000","last_peered":"2023-10-06T05:26:15.990744+0000","last_clean":"2023-10-06T05:26:15.990744+0000","last_became_active":"2023-10-06T05:26:15.990214+0000","last_became_peered":"2023-10-06T05:26:15.990214+0000","last_unstale":"2023-10-06T05:26:15.990744+0000","last_undegraded":"2023-10-06T05:26:15.990744+0000","last_fullsized":"2023-10-06T05:26:15.990744+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:57.407069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.984907+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:24.507116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.989218+0000","last_change":"2023-10-06T05:26:15.989218+0000","last_active":"2023-10-06T05:26:15.989218+0000","last_peered":"2023-10-06T05:26:15.989218+0000","last_clean":"2023-10-06T05:26:15.989218+0000","last_became_active":"2023-10-06T05:26:15.988892+0000","last_became_peered":"2023-10-06T05:26:15.988892+0000","last_unstale":"2023-10-06T05:26:15.989218+0000","last_undegraded":"2023-10-06T05:26:15.989218+0000","last_fullsized":"2023-10-06T05:26:15.989218+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:19.526591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.969947+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:35.916680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.981438+0000","last_change":"2023-10-06T05:26:15.981438+0000","last_active":"2023-10-06T05:26:15.981438+0000","last_peered":"2023-10-06T05:26:15.981438+0000","last_clean":"2023-10-06T05:26:15.981438+0000","last_became_active":"2023-10-06T05:26:15.979239+0000","last_became_peered":"2023-10-06T05:26:15.979239+0000","last_unstale":"2023-10-06T05:26:15.981438+0000","last_undegraded":"2023-10-06T05:26:15.981438+0000","last_fullsized":"2023-10-06T05:26:15.981438+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:50.444147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.972489+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:37.180678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.981307+0000","last_change":"2023-10-06T05:26:15.981307+0000","last_active":"2023-10-06T05:26:15.981307+0000","last_peered":"2023-10-06T05:26:15.981307+0000","last_clean":"2023-10-06T05:26:15.981307+0000","last_became_active":"2023-10-06T05:26:15.981055+0000","last_became_peered":"2023-10-06T05:26:15.981055+0000","last_unstale":"2023-10-06T05:26:15.981307+0000","last_undegraded":"2023-10-06T05:26:15.981307+0000","last_fullsized":"2023-10-06T05:26:15.981307+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:32.428141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.972324+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:04.234830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.981474+0000","last_change":"2023-10-06T05:26:15.981474+0000","last_active":"2023-10-06T05:26:15.981474+0000","last_peered":"2023-10-06T05:26:15.981474+0000","last_clean":"2023-10-06T05:26:15.981474+0000","last_became_active":"2023-10-06T05:26:15.980370+0000","last_became_peered":"2023-10-06T05:26:15.980370+0000","last_unstale":"2023-10-06T05:26:15.981474+0000","last_undegraded":"2023-10-06T05:26:15.981474+0000","last_fullsized":"2023-10-06T05:26:15.981474+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:35.004542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.967701+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:26:14.967701+0000","last_peered":"2023-10-06T05:26:14.967701+0000","last_clean":"2023-10-06T05:26:14.967701+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:26:14.967701+0000","last_undegraded":"2023-10-06T05:26:14.967701+0000","last_fullsized":"2023-10-06T05:26:14.967701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.974521+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:03.189897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.989314+0000","last_change":"2023-10-06T05:26:15.989314+0000","last_active":"2023-10-06T05:26:15.989314+0000","last_peered":"2023-10-06T05:26:15.989314+0000","last_clean":"2023-10-06T05:26:15.989314+0000","last_became_active":"2023-10-06T05:26:15.988969+0000","last_became_peered":"2023-10-06T05:26:15.988969+0000","last_unstale":"2023-10-06T05:26:15.989314+0000","last_undegraded":"2023-10-06T05:26:15.989314+0000","last_fullsized":"2023-10-06T05:26:15.989314+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:26.549296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":132,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.966845+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:26:14.966845+0000","last_peered":"2023-10-06T05:26:14.966845+0000","last_clean":"2023-10-06T05:26:14.966845+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:26:14.966845+0000","last_undegraded":"2023-10-06T05:26:14.966845+0000","last_fullsized":"2023-10-06T05:26:14.966845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.973921+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:24.507116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.990882+0000","last_change":"2023-10-06T05:26:15.990882+0000","last_active":"2023-10-06T05:26:15.990882+0000","last_peered":"2023-10-06T05:26:15.990882+0000","last_clean":"2023-10-06T05:26:15.990882+0000","last_became_active":"2023-10-06T05:26:15.989632+0000","last_became_peered":"2023-10-06T05:26:15.989632+0000","last_unstale":"2023-10-06T05:26:15.990882+0000","last_undegraded":"2023-10-06T05:26:15.990882+0000","last_fullsized":"2023-10-06T05:26:15.990882+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:41.444233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.962105+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:26:14.962105+0000","last_peered":"2023-10-06T05:26:14.962105+0000","last_clean":"2023-10-06T05:26:14.962105+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:26:14.962105+0000","last_undegraded":"2023-10-06T05:26:14.962105+0000","last_fullsized":"2023-10-06T05:26:14.962105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.970503+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:17.817823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.988033+0000","last_change":"2023-10-06T05:26:15.988033+0000","last_active":"2023-10-06T05:26:15.988033+0000","last_peered":"2023-10-06T05:26:15.988033+0000","last_clean":"2023-10-06T05:26:15.988033+0000","last_became_active":"2023-10-06T05:26:15.987719+0000","last_became_peered":"2023-10-06T05:26:15.987719+0000","last_unstale":"2023-10-06T05:26:15.988033+0000","last_undegraded":"2023-10-06T05:26:15.988033+0000","last_fullsized":"2023-10-06T05:26:15.988033+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:19.842609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.966998+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:26:14.966998+0000","last_peered":"2023-10-06T05:26:14.966998+0000","last_clean":"2023-10-06T05:26:14.966998+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:26:14.966998+0000","last_undegraded":"2023-10-06T05:26:14.966998+0000","last_fullsized":"2023-10-06T05:26:14.966998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.968034+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:26:14.968034+0000","last_peered":"2023-10-06T05:26:14.968034+0000","last_clean":"2023-10-06T05:26:14.968034+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:26:14.968034+0000","last_undegraded":"2023-10-06T05:26:14.968034+0000","last_fullsized":"2023-10-06T05:26:14.968034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.981084+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:35.916680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.984905+0000","last_change":"2023-10-06T05:26:15.984905+0000","last_active":"2023-10-06T05:26:15.984905+0000","last_peered":"2023-10-06T05:26:15.984905+0000","last_clean":"2023-10-06T05:26:15.984905+0000","last_became_active":"2023-10-06T05:26:15.984636+0000","last_became_peered":"2023-10-06T05:26:15.984636+0000","last_unstale":"2023-10-06T05:26:15.984905+0000","last_undegraded":"2023-10-06T05:26:15.984905+0000","last_fullsized":"2023-10-06T05:26:15.984905+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:37.207464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.967979+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:26:14.967979+0000","last_peered":"2023-10-06T05:26:14.967979+0000","last_clean":"2023-10-06T05:26:14.967979+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:26:14.967979+0000","last_undegraded":"2023-10-06T05:26:14.967979+0000","last_fullsized":"2023-10-06T05:26:14.967979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.971141+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:42.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.987554+0000","last_change":"2023-10-06T05:26:15.987554+0000","last_active":"2023-10-06T05:26:15.987554+0000","last_peered":"2023-10-06T05:26:15.987554+0000","last_clean":"2023-10-06T05:26:15.987554+0000","last_became_active":"2023-10-06T05:26:15.987264+0000","last_became_peered":"2023-10-06T05:26:15.987264+0000","last_unstale":"2023-10-06T05:26:15.987554+0000","last_undegraded":"2023-10-06T05:26:15.987554+0000","last_fullsized":"2023-10-06T05:26:15.987554+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:05.783691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.968118+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:26:14.968118+0000","last_peered":"2023-10-06T05:26:14.968118+0000","last_clean":"2023-10-06T05:26:14.968118+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:26:14.968118+0000","last_undegraded":"2023-10-06T05:26:14.968118+0000","last_fullsized":"2023-10-06T05:26:14.968118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.976765+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:37.371618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.989016+0000","last_change":"2023-10-06T05:26:15.989016+0000","last_active":"2023-10-06T05:26:15.989016+0000","last_peered":"2023-10-06T05:26:15.989016+0000","last_clean":"2023-10-06T05:26:15.989016+0000","last_became_active":"2023-10-06T05:26:15.988637+0000","last_became_peered":"2023-10-06T05:26:15.988637+0000","last_unstale":"2023-10-06T05:26:15.989016+0000","last_undegraded":"2023-10-06T05:26:15.989016+0000","last_fullsized":"2023-10-06T05:26:15.989016+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.340712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.967862+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:26:14.967862+0000","last_peered":"2023-10-06T05:26:14.967862+0000","last_clean":"2023-10-06T05:26:14.967862+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:26:14.967862+0000","last_undegraded":"2023-10-06T05:26:14.967862+0000","last_fullsized":"2023-10-06T05:26:14.967862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:15.960021+0000","last_change":"2023-10-06T05:26:15.970580+0000","last_active":"2023-10-06T05:26:15.960021+0000","last_peered":"2023-10-06T05:26:15.960021+0000","last_clean":"2023-10-06T05:26:15.960021+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:15.960021+0000","last_undegraded":"2023-10-06T05:26:15.960021+0000","last_fullsized":"2023-10-06T05:26:15.960021+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:06.985236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T05:26:15.990119+0000","last_change":"2023-10-06T05:26:15.990119+0000","last_active":"2023-10-06T05:26:15.990119+0000","last_peered":"2023-10-06T05:26:15.990119+0000","last_clean":"2023-10-06T05:26:15.990119+0000","last_became_active":"2023-10-06T05:26:15.988477+0000","last_became_peered":"2023-10-06T05:26:15.988477+0000","last_unstale":"2023-10-06T05:26:15.990119+0000","last_undegraded":"2023-10-06T05:26:15.990119+0000","last_fullsized":"2023-10-06T05:26:15.990119+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:21.716854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:14.965051+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:26:14.965051+0000","last_peered":"2023-10-06T05:26:14.965051+0000","last_clean":"2023-10-06T05:26:14.965051+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:26:14.965051+0000","last_undegraded":"2023-10-06T05:26:14.965051+0000","last_fullsized":"2023-10-06T05:26:14.965051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304016,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067824,"statfs":{"total":96636764160,"available":96325451776,"internally_reserved":0,"allocated":2695168,"data_stored":1562186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.87,"5min":0.95999999999999996,"15min":1.1850000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":1.1339999999999999,"15min":1.169},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.268},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.96099999999999997,"15min":1.075},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.47899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97599999999999998,"5min":1.0780000000000001,"15min":1.093},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.22},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93200000000000005,"5min":1.0249999999999999,"15min":1.079},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.0960000000000001,"15min":1.202},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.625},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.0209999999999999,"15min":1.258},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.42199999999999999,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":1.109,"15min":1.109},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.746},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":1.0720000000000001,"15min":1.407},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.54800000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80000000000000004,"5min":1.1220000000000001,"15min":1.139},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":0.97799999999999998,"5min":1.135,"15min":1.3300000000000001},"last":0.89600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":971818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.51600000000000001,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.92800000000000005,"5min":0.98099999999999998,"15min":1.034},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.96499999999999997,"5min":0.995,"15min":1.2869999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.70599999999999996},"min":{"1min":0.56299999999999994,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.88700000000000001,"5min":0.93400000000000005,"15min":1},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.71699999999999997},"min":{"1min":0.58499999999999996,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.98899999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85699999999999998,"5min":0.95099999999999996,"15min":1.1599999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.73299999999999998},"min":{"1min":0.54700000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.89800000000000002,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.59199999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72899999999999998,"15min":0.77100000000000002},"min":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.5},"max":{"1min":0.88800000000000001,"5min":0.93200000000000005,"15min":1.1020000000000001},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.626,"15min":0.65100000000000002},"min":{"1min":0.52200000000000002,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95699999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64400000000000002,"15min":0.67500000000000004},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.42099999999999999},"max":{"1min":0.88200000000000001,"5min":0.89100000000000001,"15min":0.998},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67600000000000005,"15min":0.70699999999999996},"min":{"1min":0.52200000000000002,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0760000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71099999999999997,"15min":0.74099999999999999},"min":{"1min":0.52300000000000002,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":0.96499999999999997,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":0.94099999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.745,"15min":0.79200000000000004},"min":{"1min":0.62,"5min":0.496,"15min":0.436},"max":{"1min":1.006,"5min":1.006,"15min":1.1359999999999999},"last":0.76400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":971818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46999999999999997,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.69999999999999996,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.27800000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.376}]},{"osd":2,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.88200000000000001,"15min":2.7010000000000001},"last":0.40600000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.61799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.86299999999999999,"5min":0.88,"15min":2.7639999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.90000000000000002,"15min":2.8370000000000002},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50700000000000001},"max":{"1min":1.0249999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.79100000000000004},"min":{"1min":0.60599999999999998,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.0489999999999999,"5min":1.25,"15min":1.349},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.55600000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.073,"5min":1.2569999999999999,"15min":1.669},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.83899999999999997},"min":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0840000000000001,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.57099999999999995,"15min":0.503},"max":{"1min":1.0620000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1100000000000001,"5min":1.1870000000000001,"15min":1.528},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":1.0089999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304016,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067824,"statfs":{"total":96636764160,"available":96325451776,"internally_reserved":0,"allocated":2695168,"data_stored":1562186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30499999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.497},"min":{"1min":0.313,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70799999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.38400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.67200000000000004,"5min":0.70999999999999996,"15min":0.84999999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77000000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.442},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81100000000000005,"5min":0.84499999999999997,"15min":0.879},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78300000000000003,"15min":0.77000000000000002},"min":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.621},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.6629999999999998,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.77700000000000002},"min":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.159,"15min":1.218},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95499999999999996,"5min":1.089,"15min":1.1970000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.153,"5min":1.153,"15min":1.2629999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60299999999999998,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.044,"5min":1.0780000000000001,"15min":1.25},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59899999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.087,"5min":1.256,"15min":1.2789999999999999},"last":0.60699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":971818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.056,"15min":1.056},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72799999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.626,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.44900000000000001,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.79800000000000004,"5min":1.073,"15min":1.073},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46000000000000002,"15min":0.47099999999999997},"min":{"1min":0.35899999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.60899999999999999,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0940000000000001,"5min":1.109,"15min":1.1379999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.54500000000000004,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.071,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73999999999999999,"15min":0.76800000000000002},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.749},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":1.0640000000000001,"5min":1.1859999999999999,"15min":1.319},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.80800000000000005},"min":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0629999999999999,"5min":1.137,"15min":1.141},"last":0.77000000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.80400000000000005},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.98999999999999999,"5min":1.284,"15min":1.284},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78600000000000003,"15min":0.81899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0980000000000001,"5min":1.2,"15min":1.3520000000000001},"last":0.86199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":971818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45500000000000002,"15min":0.48699999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46600000000000003,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.54700000000000004,"5min":0.77100000000000002,"15min":1.772},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":1.913},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.68100000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.63500000000000001,"5min":0.79700000000000004,"15min":1.954},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82599999999999996,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.51400000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.042,"5min":1.101,"15min":1.282},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75700000000000001,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":1.117,"5min":1.1839999999999999,"15min":1.2609999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.79200000000000004},"min":{"1min":0.63800000000000001,"5min":0.54300000000000004,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.145,"15min":1.1930000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0940000000000001,"5min":1.21,"15min":1.3009999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78700000000000003,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.53900000000000003},"max":{"1min":1.042,"5min":1.1419999999999999,"15min":1.343},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.60199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":1.0649999999999999,"15min":1.3380000000000001},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79600000000000004,"15min":0.83299999999999996},"min":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.0800000000000001,"5min":1.194,"15min":1.3640000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.63300000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.0780000000000001,"5min":1.1579999999999999,"15min":1.214},"last":0.79600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":971818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.073},"last":0.628},{"interface":"front","average":{"1min":0.75,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.46100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0089999999999999,"15min":1.169},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44900000000000001,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69499999999999995,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.82299999999999995,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.69599999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.117},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.97099999999999997,"15min":1.002},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72099999999999997,"15min":0.73799999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91000000000000003,"5min":0.97599999999999998,"15min":1.0780000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.018,"15min":1.196},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.95299999999999996,"5min":0.999,"15min":1.375},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.97399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73999999999999999,"15min":0.77100000000000002},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0209999999999999,"15min":1.9390000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.48199999999999998},"max":{"1min":0.95899999999999996,"5min":1.05,"15min":1.466},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.218},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":971818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.54500000000000004,"5min":0.374,"15min":0.374},"max":{"1min":0.88200000000000001,"5min":0.93899999999999995,"15min":1.165},"last":2.4609999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.1890000000000001},"last":1.3460000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.98299999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.49199999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.995,"5min":1.048,"15min":1.048},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.55400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.204},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92400000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94599999999999995,"5min":0.95199999999999996,"15min":1.087},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.51900000000000002,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":0.93000000000000005,"5min":0.99299999999999999,"15min":1.278},"last":1.0700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.92800000000000005,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.44},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.004,"5min":1.1539999999999999,"15min":1.294},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":0.96899999999999997,"15min":1.141},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":0.95099999999999996,"5min":1.0429999999999999,"15min":1.254},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.57399999999999995,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":1.0189999999999999,"15min":1.337},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.248},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:21.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.881+0000 7f5d7c99d700 1 Processor -- start 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.882+0000 7f5d7c99d700 1 -- start start 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.883+0000 7f5d7c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 0x7f5d78160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.883+0000 7f5d7c99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7806ab60 con 0x7f5d78155af0 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.883+0000 7f5d7c99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7806ae40 con 0x7f5d780c8b50 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.883+0000 7f5d7c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7806b7e0 con 0x7f5d780c7920 2023-10-06T05:26:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.883+0000 7f5d777fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5d78155af0 0x7f5d7806a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48022/0 (socket says 172.21.15.33:48022) 2023-10-06T05:26:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.883+0000 7f5d777fe700 1 -- 172.21.15.33:0/4044261740 learned_addr learned my addr 172.21.15.33:0/4044261740 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d77fff700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 0x7f5d78160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863481037 0 0) 0x7f5d7806ab60 con 0x7f5d78155af0 2023-10-06T05:26:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d60004680 con 0x7f5d78155af0 2023-10-06T05:26:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665707918 0 0) 0x7f5d7806ae40 con 0x7f5d780c8b50 2023-10-06T05:26:21.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7806ab60 con 0x7f5d780c8b50 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858526263 0 0) 0x7f5d60004680 con 0x7f5d78155af0 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 msgr2=0x7f5d78160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 0x7f5d78160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 >> v1:172.21.15.167:6789/0 conn(0x7f5d780c8b50 legacy=0x7f5d780c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7806b120 con 0x7f5d78155af0 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f5d68001470 con 0x7f5d78155af0 2023-10-06T05:26:21.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 >> v1:172.21.15.33:6789/0 conn(0x7f5d78155af0 legacy=0x7f5d7806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60004d30 0x7f5d60005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d60005920 0x7f5d60005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 0x7f5d600062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7806b7e0 con 0x7f5d60004d30 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6000cc20 con 0x7f5d60005920 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.884+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6000cde0 con 0x7f5d780c7920 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60004d30 0x7f5d60005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d77fff700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 0x7f5d600062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d76ffd700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d60005920 0x7f5d60005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 -- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 msgr2=0x7f5d600062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780c7920 0x7f5d600062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 -- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d60005920 msgr2=0x7f5d60005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d60005920 0x7f5d60005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 -- 172.21.15.33:0/4044261740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7806b120 con 0x7f5d60004d30 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d777fe700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60004d30 0x7f5d60005120 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5d68007fd0 tx=0x7f5d680079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68812ffe511466c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800cce0 con 0x7f5d60004d30 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6800ce40 con 0x7f5d60004d30 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.885+0000 7f5d767fc700 1 -- 172.21.15.33:0/4044261740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68005b70 con 0x7f5d60004d30 2023-10-06T05:26:21.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d7c99d700 1 -- 172.21.15.33:0/4044261740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60004d30 msgr2=0x7f5d60005120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/4044261740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60004d30 0x7f5d60005120 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5d68007fd0 tx=0x7f5d680079d0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d77fff700 1 -- 172.21.15.33:0/4044261740 reap_dead start 2023-10-06T05:26:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d7c99d700 1 -- 172.21.15.33:0/4044261740 shutdown_connections 2023-10-06T05:26:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d7c99d700 1 -- 172.21.15.33:0/4044261740 >> 172.21.15.33:0/4044261740 conn(0x7f5d7806d030 msgr2=0x7f5d7806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d7c99d700 1 -- 172.21.15.33:0/4044261740 shutdown_connections 2023-10-06T05:26:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.886+0000 7f5d7c99d700 1 -- 172.21.15.33:0/4044261740 wait complete. 2023-10-06T05:26:21.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.888+0000 7f5d7c99d700 1 Processor -- start 2023-10-06T05:26:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.888+0000 7f5d7c99d700 1 -- start start 2023-10-06T05:26:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d7c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d60004d30 0x7f5d780b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d7c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 0x7f5d780aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d7c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780c4360 0x7f5d780c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d7c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780aff30 con 0x7f5d780ac600 2023-10-06T05:26:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d7c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780af320 con 0x7f5d780c4360 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d7c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780af600 con 0x7f5d60004d30 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 0x7f5d780aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 0x7f5d780aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48390/0 (socket says 172.21.15.33:48390) 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 -- 172.21.15.33:0/1631370038 learned_addr learned my addr 172.21.15.33:0/1631370038 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d777fe700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d60004d30 0x7f5d780b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d77fff700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780c4360 0x7f5d780c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 -- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d60004d30 msgr2=0x7f5d780b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d60004d30 0x7f5d780b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 -- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780c4360 msgr2=0x7f5d780c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780c4360 0x7f5d780c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.889+0000 7f5d76ffd700 1 -- 172.21.15.33:0/1631370038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780af8e0 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.890+0000 7f5d76ffd700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 0x7f5d780aca70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00f880 tx=0x7f5d6c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.890+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c013600 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.890+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c013760 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.890+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c013b00 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.890+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780afbc0 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.890+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d78150880 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.891+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d780b00a0 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.892+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d6c01c400 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.893+0000 7f5d5f7fe700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d64066f30 0x7f5d640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.893+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d6c0848c0 con 0x7f5d780ac600 2023-10-06T05:26:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.894+0000 7f5d777fe700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d64066f30 0x7f5d640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.894+0000 7f5d777fe700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d64066f30 0x7f5d640693e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d010 tx=0x7f5d68017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:21.897+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d6c051930 con 0x7f5d780ac600 2023-10-06T05:26:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.095+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d7804c380 con 0x7f5d64066f30 2023-10-06T05:26:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.105+0000 7f5d5f7fe700 1 -- 172.21.15.33:0/1631370038 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311843 (secure 0 0 0) 0x7f5d7804c380 con 0x7f5d64066f30 2023-10-06T05:26:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:22.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d64066f30 msgr2=0x7f5d640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d64066f30 0x7f5d640693e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d010 tx=0x7f5d68017000 comp rx=0 tx=0).stop 2023-10-06T05:26:22.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 msgr2=0x7f5d780aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 0x7f5d780aca70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00f880 tx=0x7f5d6c007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:22.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 shutdown_connections 2023-10-06T05:26:22.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d64066f30 0x7f5d640693e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780c4360 0x7f5d780c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d60004d30 0x7f5d780b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 --2- 172.21.15.33:0/1631370038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780ac600 0x7f5d780aca70 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 >> 172.21.15.33:0/1631370038 conn(0x7f5d7806d030 msgr2=0x7f5d780c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.109+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 shutdown_connections 2023-10-06T05:26:22.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.110+0000 7f5d7c99d700 1 -- 172.21.15.33:0/1631370038 wait complete. 2023-10-06T05:26:22.120 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:26:22.130 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2023-10-06T05:26:21.996328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":202,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428904,"kb_used_data":17832,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752545816,"statfs":{"total":773094113280,"available":770606915584,"internally_reserved":0,"allocated":18259968,"data_stored":9069948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.027013"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010389+0000","last_change":"2023-10-06T05:26:17.006285+0000","last_active":"2023-10-06T05:26:17.010389+0000","last_peered":"2023-10-06T05:26:17.010389+0000","last_clean":"2023-10-06T05:26:17.010389+0000","last_became_active":"2023-10-06T05:26:17.006033+0000","last_became_peered":"2023-10-06T05:26:17.006033+0000","last_unstale":"2023-10-06T05:26:17.010389+0000","last_undegraded":"2023-10-06T05:26:17.010389+0000","last_fullsized":"2023-10-06T05:26:17.010389+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:46.117421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.017483+0000","last_change":"2023-10-06T05:26:17.013352+0000","last_active":"2023-10-06T05:26:17.017483+0000","last_peered":"2023-10-06T05:26:17.017483+0000","last_clean":"2023-10-06T05:26:17.017483+0000","last_became_active":"2023-10-06T05:26:17.013028+0000","last_became_peered":"2023-10-06T05:26:17.013028+0000","last_unstale":"2023-10-06T05:26:17.017483+0000","last_undegraded":"2023-10-06T05:26:17.017483+0000","last_fullsized":"2023-10-06T05:26:17.017483+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:38.213234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.999097+0000","last_change":"2023-10-06T05:26:16.994652+0000","last_active":"2023-10-06T05:26:16.999097+0000","last_peered":"2023-10-06T05:26:16.999097+0000","last_clean":"2023-10-06T05:26:16.999097+0000","last_became_active":"2023-10-06T05:26:16.993873+0000","last_became_peered":"2023-10-06T05:26:16.993873+0000","last_unstale":"2023-10-06T05:26:16.999097+0000","last_undegraded":"2023-10-06T05:26:16.999097+0000","last_fullsized":"2023-10-06T05:26:16.999097+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:01.715201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010816+0000","last_change":"2023-10-06T05:26:17.009035+0000","last_active":"2023-10-06T05:26:17.010816+0000","last_peered":"2023-10-06T05:26:17.010816+0000","last_clean":"2023-10-06T05:26:17.010816+0000","last_became_active":"2023-10-06T05:26:17.008684+0000","last_became_peered":"2023-10-06T05:26:17.008684+0000","last_unstale":"2023-10-06T05:26:17.010816+0000","last_undegraded":"2023-10-06T05:26:17.010816+0000","last_fullsized":"2023-10-06T05:26:17.010816+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:35.896261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010898+0000","last_change":"2023-10-06T05:26:17.009230+0000","last_active":"2023-10-06T05:26:17.010898+0000","last_peered":"2023-10-06T05:26:17.010898+0000","last_clean":"2023-10-06T05:26:17.010898+0000","last_became_active":"2023-10-06T05:26:17.008904+0000","last_became_peered":"2023-10-06T05:26:17.008904+0000","last_unstale":"2023-10-06T05:26:17.010898+0000","last_undegraded":"2023-10-06T05:26:17.010898+0000","last_fullsized":"2023-10-06T05:26:17.010898+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:50.313717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.005595+0000","last_change":"2023-10-06T05:26:16.999938+0000","last_active":"2023-10-06T05:26:17.005595+0000","last_peered":"2023-10-06T05:26:17.005595+0000","last_clean":"2023-10-06T05:26:17.005595+0000","last_became_active":"2023-10-06T05:26:16.999659+0000","last_became_peered":"2023-10-06T05:26:16.999659+0000","last_unstale":"2023-10-06T05:26:17.005595+0000","last_undegraded":"2023-10-06T05:26:17.005595+0000","last_fullsized":"2023-10-06T05:26:17.005595+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:07.720075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006110+0000","last_change":"2023-10-06T05:26:17.002905+0000","last_active":"2023-10-06T05:26:17.006110+0000","last_peered":"2023-10-06T05:26:17.006110+0000","last_clean":"2023-10-06T05:26:17.006110+0000","last_became_active":"2023-10-06T05:26:17.002561+0000","last_became_peered":"2023-10-06T05:26:17.002561+0000","last_unstale":"2023-10-06T05:26:17.006110+0000","last_undegraded":"2023-10-06T05:26:17.006110+0000","last_fullsized":"2023-10-06T05:26:17.006110+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:41.093677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.015499+0000","last_change":"2023-10-06T05:26:17.014344+0000","last_active":"2023-10-06T05:26:17.015499+0000","last_peered":"2023-10-06T05:26:17.015499+0000","last_clean":"2023-10-06T05:26:17.015499+0000","last_became_active":"2023-10-06T05:26:17.014047+0000","last_became_peered":"2023-10-06T05:26:17.014047+0000","last_unstale":"2023-10-06T05:26:17.015499+0000","last_undegraded":"2023-10-06T05:26:17.015499+0000","last_fullsized":"2023-10-06T05:26:17.015499+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:47.593154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006377+0000","last_change":"2023-10-06T05:26:17.003859+0000","last_active":"2023-10-06T05:26:17.006377+0000","last_peered":"2023-10-06T05:26:17.006377+0000","last_clean":"2023-10-06T05:26:17.006377+0000","last_became_active":"2023-10-06T05:26:17.003539+0000","last_became_peered":"2023-10-06T05:26:17.003539+0000","last_unstale":"2023-10-06T05:26:17.006377+0000","last_undegraded":"2023-10-06T05:26:17.006377+0000","last_fullsized":"2023-10-06T05:26:17.006377+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:37.180678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.014385+0000","last_change":"2023-10-06T05:26:17.008248+0000","last_active":"2023-10-06T05:26:17.014385+0000","last_peered":"2023-10-06T05:26:17.014385+0000","last_clean":"2023-10-06T05:26:17.014385+0000","last_became_active":"2023-10-06T05:26:17.007940+0000","last_became_peered":"2023-10-06T05:26:17.007940+0000","last_unstale":"2023-10-06T05:26:17.014385+0000","last_undegraded":"2023-10-06T05:26:17.014385+0000","last_fullsized":"2023-10-06T05:26:17.014385+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:28.940732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016244+0000","last_change":"2023-10-06T05:26:17.009888+0000","last_active":"2023-10-06T05:26:17.016244+0000","last_peered":"2023-10-06T05:26:17.016244+0000","last_clean":"2023-10-06T05:26:17.016244+0000","last_became_active":"2023-10-06T05:26:17.009574+0000","last_became_peered":"2023-10-06T05:26:17.009574+0000","last_unstale":"2023-10-06T05:26:17.016244+0000","last_undegraded":"2023-10-06T05:26:17.016244+0000","last_fullsized":"2023-10-06T05:26:17.016244+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:11.209158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010834+0000","last_change":"2023-10-06T05:26:17.009107+0000","last_active":"2023-10-06T05:26:17.010834+0000","last_peered":"2023-10-06T05:26:17.010834+0000","last_clean":"2023-10-06T05:26:17.010834+0000","last_became_active":"2023-10-06T05:26:17.008834+0000","last_became_peered":"2023-10-06T05:26:17.008834+0000","last_unstale":"2023-10-06T05:26:17.010834+0000","last_undegraded":"2023-10-06T05:26:17.010834+0000","last_fullsized":"2023-10-06T05:26:17.010834+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:13.413249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.012607+0000","last_change":"2023-10-06T05:26:17.007641+0000","last_active":"2023-10-06T05:26:17.012607+0000","last_peered":"2023-10-06T05:26:17.012607+0000","last_clean":"2023-10-06T05:26:17.012607+0000","last_became_active":"2023-10-06T05:26:17.001790+0000","last_became_peered":"2023-10-06T05:26:17.001790+0000","last_unstale":"2023-10-06T05:26:17.012607+0000","last_undegraded":"2023-10-06T05:26:17.012607+0000","last_fullsized":"2023-10-06T05:26:17.012607+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:23.409856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.004845+0000","last_change":"2023-10-06T05:26:16.999913+0000","last_active":"2023-10-06T05:26:17.004845+0000","last_peered":"2023-10-06T05:26:17.004845+0000","last_clean":"2023-10-06T05:26:17.004845+0000","last_became_active":"2023-10-06T05:26:16.999641+0000","last_became_peered":"2023-10-06T05:26:16.999641+0000","last_unstale":"2023-10-06T05:26:17.004845+0000","last_undegraded":"2023-10-06T05:26:17.004845+0000","last_fullsized":"2023-10-06T05:26:17.004845+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.765873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016213+0000","last_change":"2023-10-06T05:26:17.012085+0000","last_active":"2023-10-06T05:26:17.016213+0000","last_peered":"2023-10-06T05:26:17.016213+0000","last_clean":"2023-10-06T05:26:17.016213+0000","last_became_active":"2023-10-06T05:26:17.011826+0000","last_became_peered":"2023-10-06T05:26:17.011826+0000","last_unstale":"2023-10-06T05:26:17.016213+0000","last_undegraded":"2023-10-06T05:26:17.016213+0000","last_fullsized":"2023-10-06T05:26:17.016213+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:36.010312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.018558+0000","last_change":"2023-10-06T05:26:17.015556+0000","last_active":"2023-10-06T05:26:17.018558+0000","last_peered":"2023-10-06T05:26:17.018558+0000","last_clean":"2023-10-06T05:26:17.018558+0000","last_became_active":"2023-10-06T05:26:17.014567+0000","last_became_peered":"2023-10-06T05:26:17.014567+0000","last_unstale":"2023-10-06T05:26:17.018558+0000","last_undegraded":"2023-10-06T05:26:17.018558+0000","last_fullsized":"2023-10-06T05:26:17.018558+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:29.792182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010033+0000","last_change":"2023-10-06T05:26:17.005640+0000","last_active":"2023-10-06T05:26:17.010033+0000","last_peered":"2023-10-06T05:26:17.010033+0000","last_clean":"2023-10-06T05:26:17.010033+0000","last_became_active":"2023-10-06T05:26:17.005361+0000","last_became_peered":"2023-10-06T05:26:17.005361+0000","last_unstale":"2023-10-06T05:26:17.010033+0000","last_undegraded":"2023-10-06T05:26:17.010033+0000","last_fullsized":"2023-10-06T05:26:17.010033+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:35.896261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.012642+0000","last_change":"2023-10-06T05:26:17.007381+0000","last_active":"2023-10-06T05:26:17.012642+0000","last_peered":"2023-10-06T05:26:17.012642+0000","last_clean":"2023-10-06T05:26:17.012642+0000","last_became_active":"2023-10-06T05:26:17.007114+0000","last_became_peered":"2023-10-06T05:26:17.007114+0000","last_unstale":"2023-10-06T05:26:17.012642+0000","last_undegraded":"2023-10-06T05:26:17.012642+0000","last_fullsized":"2023-10-06T05:26:17.012642+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:58.412839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.013446+0000","last_change":"2023-10-06T05:26:17.007768+0000","last_active":"2023-10-06T05:26:17.013446+0000","last_peered":"2023-10-06T05:26:17.013446+0000","last_clean":"2023-10-06T05:26:17.013446+0000","last_became_active":"2023-10-06T05:26:17.007413+0000","last_became_peered":"2023-10-06T05:26:17.007413+0000","last_unstale":"2023-10-06T05:26:17.013446+0000","last_undegraded":"2023-10-06T05:26:17.013446+0000","last_fullsized":"2023-10-06T05:26:17.013446+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:32.931988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010000+0000","last_change":"2023-10-06T05:26:17.004949+0000","last_active":"2023-10-06T05:26:17.010000+0000","last_peered":"2023-10-06T05:26:17.010000+0000","last_clean":"2023-10-06T05:26:17.010000+0000","last_became_active":"2023-10-06T05:26:17.004663+0000","last_became_peered":"2023-10-06T05:26:17.004663+0000","last_unstale":"2023-10-06T05:26:17.010000+0000","last_undegraded":"2023-10-06T05:26:17.010000+0000","last_fullsized":"2023-10-06T05:26:17.010000+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:25.219842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.002223+0000","last_change":"2023-10-06T05:26:17.001118+0000","last_active":"2023-10-06T05:26:17.002223+0000","last_peered":"2023-10-06T05:26:17.002223+0000","last_clean":"2023-10-06T05:26:17.002223+0000","last_became_active":"2023-10-06T05:26:17.000690+0000","last_became_peered":"2023-10-06T05:26:17.000690+0000","last_unstale":"2023-10-06T05:26:17.002223+0000","last_undegraded":"2023-10-06T05:26:17.002223+0000","last_fullsized":"2023-10-06T05:26:17.002223+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.887934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.009062+0000","last_change":"2023-10-06T05:26:17.004068+0000","last_active":"2023-10-06T05:26:17.009062+0000","last_peered":"2023-10-06T05:26:17.009062+0000","last_clean":"2023-10-06T05:26:17.009062+0000","last_became_active":"2023-10-06T05:26:17.003812+0000","last_became_peered":"2023-10-06T05:26:17.003812+0000","last_unstale":"2023-10-06T05:26:17.009062+0000","last_undegraded":"2023-10-06T05:26:17.009062+0000","last_fullsized":"2023-10-06T05:26:17.009062+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:23.409856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006634+0000","last_change":"2023-10-06T05:26:17.003673+0000","last_active":"2023-10-06T05:26:17.006634+0000","last_peered":"2023-10-06T05:26:17.006634+0000","last_clean":"2023-10-06T05:26:17.006634+0000","last_became_active":"2023-10-06T05:26:17.003310+0000","last_became_peered":"2023-10-06T05:26:17.003310+0000","last_unstale":"2023-10-06T05:26:17.006634+0000","last_undegraded":"2023-10-06T05:26:17.006634+0000","last_fullsized":"2023-10-06T05:26:17.006634+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:09.004752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.011399+0000","last_change":"2023-10-06T05:26:17.009902+0000","last_active":"2023-10-06T05:26:17.011399+0000","last_peered":"2023-10-06T05:26:17.011399+0000","last_clean":"2023-10-06T05:26:17.011399+0000","last_became_active":"2023-10-06T05:26:17.009641+0000","last_became_peered":"2023-10-06T05:26:17.009641+0000","last_unstale":"2023-10-06T05:26:17.011399+0000","last_undegraded":"2023-10-06T05:26:17.011399+0000","last_fullsized":"2023-10-06T05:26:17.011399+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:09.253354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.004613+0000","last_change":"2023-10-06T05:26:16.998641+0000","last_active":"2023-10-06T05:26:17.004613+0000","last_peered":"2023-10-06T05:26:17.004613+0000","last_clean":"2023-10-06T05:26:17.004613+0000","last_became_active":"2023-10-06T05:26:16.998163+0000","last_became_peered":"2023-10-06T05:26:16.998163+0000","last_unstale":"2023-10-06T05:26:17.004613+0000","last_undegraded":"2023-10-06T05:26:17.004613+0000","last_fullsized":"2023-10-06T05:26:17.004613+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:01.715201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.019396+0000","last_change":"2023-10-06T05:26:17.016990+0000","last_active":"2023-10-06T05:26:17.019396+0000","last_peered":"2023-10-06T05:26:17.019396+0000","last_clean":"2023-10-06T05:26:17.019396+0000","last_became_active":"2023-10-06T05:26:17.016621+0000","last_became_peered":"2023-10-06T05:26:17.016621+0000","last_unstale":"2023-10-06T05:26:17.019396+0000","last_undegraded":"2023-10-06T05:26:17.019396+0000","last_fullsized":"2023-10-06T05:26:17.019396+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:15.580935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.007568+0000","last_change":"2023-10-06T05:26:17.005932+0000","last_active":"2023-10-06T05:26:17.007568+0000","last_peered":"2023-10-06T05:26:17.007568+0000","last_clean":"2023-10-06T05:26:17.007568+0000","last_became_active":"2023-10-06T05:26:17.005666+0000","last_became_peered":"2023-10-06T05:26:17.005666+0000","last_unstale":"2023-10-06T05:26:17.007568+0000","last_undegraded":"2023-10-06T05:26:17.007568+0000","last_fullsized":"2023-10-06T05:26:17.007568+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:20.848379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.014886+0000","last_change":"2023-10-06T05:26:17.013105+0000","last_active":"2023-10-06T05:26:17.014886+0000","last_peered":"2023-10-06T05:26:17.014886+0000","last_clean":"2023-10-06T05:26:17.014886+0000","last_became_active":"2023-10-06T05:26:17.012673+0000","last_became_peered":"2023-10-06T05:26:17.012673+0000","last_unstale":"2023-10-06T05:26:17.014886+0000","last_undegraded":"2023-10-06T05:26:17.014886+0000","last_fullsized":"2023-10-06T05:26:17.014886+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:13.413249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.015840+0000","last_change":"2023-10-06T05:26:17.008926+0000","last_active":"2023-10-06T05:26:17.015840+0000","last_peered":"2023-10-06T05:26:17.015840+0000","last_clean":"2023-10-06T05:26:17.015840+0000","last_became_active":"2023-10-06T05:26:17.008368+0000","last_became_peered":"2023-10-06T05:26:17.008368+0000","last_unstale":"2023-10-06T05:26:17.015840+0000","last_undegraded":"2023-10-06T05:26:17.015840+0000","last_fullsized":"2023-10-06T05:26:17.015840+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:49.405271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016314+0000","last_change":"2023-10-06T05:26:17.009749+0000","last_active":"2023-10-06T05:26:17.016314+0000","last_peered":"2023-10-06T05:26:17.016314+0000","last_clean":"2023-10-06T05:26:17.016314+0000","last_became_active":"2023-10-06T05:26:17.009430+0000","last_became_peered":"2023-10-06T05:26:17.009430+0000","last_unstale":"2023-10-06T05:26:17.016314+0000","last_undegraded":"2023-10-06T05:26:17.016314+0000","last_fullsized":"2023-10-06T05:26:17.016314+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:22.722624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.004891+0000","last_change":"2023-10-06T05:26:16.999262+0000","last_active":"2023-10-06T05:26:17.004891+0000","last_peered":"2023-10-06T05:26:17.004891+0000","last_clean":"2023-10-06T05:26:17.004891+0000","last_became_active":"2023-10-06T05:26:16.998974+0000","last_became_peered":"2023-10-06T05:26:16.998974+0000","last_unstale":"2023-10-06T05:26:17.004891+0000","last_undegraded":"2023-10-06T05:26:17.004891+0000","last_fullsized":"2023-10-06T05:26:17.004891+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:09.903204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010864+0000","last_change":"2023-10-06T05:26:17.006736+0000","last_active":"2023-10-06T05:26:17.010864+0000","last_peered":"2023-10-06T05:26:17.010864+0000","last_clean":"2023-10-06T05:26:17.010864+0000","last_became_active":"2023-10-06T05:26:17.003134+0000","last_became_peered":"2023-10-06T05:26:17.003134+0000","last_unstale":"2023-10-06T05:26:17.010864+0000","last_undegraded":"2023-10-06T05:26:17.010864+0000","last_fullsized":"2023-10-06T05:26:17.010864+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:14.627179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.004689+0000","last_change":"2023-10-06T05:26:17.000823+0000","last_active":"2023-10-06T05:26:17.004689+0000","last_peered":"2023-10-06T05:26:17.004689+0000","last_clean":"2023-10-06T05:26:17.004689+0000","last_became_active":"2023-10-06T05:26:17.000540+0000","last_became_peered":"2023-10-06T05:26:17.000540+0000","last_unstale":"2023-10-06T05:26:17.004689+0000","last_undegraded":"2023-10-06T05:26:17.004689+0000","last_fullsized":"2023-10-06T05:26:17.004689+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:56.462173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.988335+0000","last_change":"2023-10-06T05:26:15.988983+0000","last_active":"2023-10-06T05:26:16.988335+0000","last_peered":"2023-10-06T05:26:16.988335+0000","last_clean":"2023-10-06T05:26:16.988335+0000","last_became_active":"2023-10-06T05:26:15.988733+0000","last_became_peered":"2023-10-06T05:26:15.988733+0000","last_unstale":"2023-10-06T05:26:16.988335+0000","last_undegraded":"2023-10-06T05:26:16.988335+0000","last_fullsized":"2023-10-06T05:26:16.988335+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:08.247585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.018546+0000","last_change":"2023-10-06T05:26:17.015618+0000","last_active":"2023-10-06T05:26:17.018546+0000","last_peered":"2023-10-06T05:26:17.018546+0000","last_clean":"2023-10-06T05:26:17.018546+0000","last_became_active":"2023-10-06T05:26:17.015007+0000","last_became_peered":"2023-10-06T05:26:17.015007+0000","last_unstale":"2023-10-06T05:26:17.018546+0000","last_undegraded":"2023-10-06T05:26:17.018546+0000","last_fullsized":"2023-10-06T05:26:17.018546+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:57.763573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.994382+0000","last_change":"2023-10-06T05:26:15.986339+0000","last_active":"2023-10-06T05:26:16.994382+0000","last_peered":"2023-10-06T05:26:16.994382+0000","last_clean":"2023-10-06T05:26:16.994382+0000","last_became_active":"2023-10-06T05:26:15.986049+0000","last_became_peered":"2023-10-06T05:26:15.986049+0000","last_unstale":"2023-10-06T05:26:16.994382+0000","last_undegraded":"2023-10-06T05:26:16.994382+0000","last_fullsized":"2023-10-06T05:26:16.994382+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:21.716854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.017302+0000","last_change":"2023-10-06T05:26:17.015344+0000","last_active":"2023-10-06T05:26:17.017302+0000","last_peered":"2023-10-06T05:26:17.017302+0000","last_clean":"2023-10-06T05:26:17.017302+0000","last_became_active":"2023-10-06T05:26:17.015023+0000","last_became_peered":"2023-10-06T05:26:17.015023+0000","last_unstale":"2023-10-06T05:26:17.017302+0000","last_undegraded":"2023-10-06T05:26:17.017302+0000","last_fullsized":"2023-10-06T05:26:17.017302+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:52.860509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"168'3","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.019910+0000","last_change":"2023-10-06T05:26:15.990391+0000","last_active":"2023-10-06T05:26:17.019910+0000","last_peered":"2023-10-06T05:26:17.019910+0000","last_clean":"2023-10-06T05:26:17.019910+0000","last_became_active":"2023-10-06T05:26:15.989737+0000","last_became_peered":"2023-10-06T05:26:15.989737+0000","last_unstale":"2023-10-06T05:26:17.019910+0000","last_undegraded":"2023-10-06T05:26:17.019910+0000","last_fullsized":"2023-10-06T05:26:17.019910+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:46.587385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016637+0000","last_change":"2023-10-06T05:26:17.012816+0000","last_active":"2023-10-06T05:26:17.016637+0000","last_peered":"2023-10-06T05:26:17.016637+0000","last_clean":"2023-10-06T05:26:17.016637+0000","last_became_active":"2023-10-06T05:26:17.012548+0000","last_became_peered":"2023-10-06T05:26:17.012548+0000","last_unstale":"2023-10-06T05:26:17.016637+0000","last_undegraded":"2023-10-06T05:26:17.016637+0000","last_fullsized":"2023-10-06T05:26:17.016637+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:12.238141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"168'6","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.022517+0000","last_change":"2023-10-06T05:26:15.989306+0000","last_active":"2023-10-06T05:26:17.022517+0000","last_peered":"2023-10-06T05:26:17.022517+0000","last_clean":"2023-10-06T05:26:17.022517+0000","last_became_active":"2023-10-06T05:26:15.988911+0000","last_became_peered":"2023-10-06T05:26:15.988911+0000","last_unstale":"2023-10-06T05:26:17.022517+0000","last_undegraded":"2023-10-06T05:26:17.022517+0000","last_fullsized":"2023-10-06T05:26:17.022517+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:25.299022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006191+0000","last_change":"2023-10-06T05:26:17.003307+0000","last_active":"2023-10-06T05:26:17.006191+0000","last_peered":"2023-10-06T05:26:17.006191+0000","last_clean":"2023-10-06T05:26:17.006191+0000","last_became_active":"2023-10-06T05:26:17.002814+0000","last_became_peered":"2023-10-06T05:26:17.002814+0000","last_unstale":"2023-10-06T05:26:17.006191+0000","last_undegraded":"2023-10-06T05:26:17.006191+0000","last_fullsized":"2023-10-06T05:26:17.006191+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:21.282021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.020578+0000","last_change":"2023-10-06T05:26:15.992259+0000","last_active":"2023-10-06T05:26:17.020578+0000","last_peered":"2023-10-06T05:26:17.020578+0000","last_clean":"2023-10-06T05:26:17.020578+0000","last_became_active":"2023-10-06T05:26:15.991988+0000","last_became_peered":"2023-10-06T05:26:15.991988+0000","last_unstale":"2023-10-06T05:26:17.020578+0000","last_undegraded":"2023-10-06T05:26:17.020578+0000","last_fullsized":"2023-10-06T05:26:17.020578+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:30.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.001372+0000","last_change":"2023-10-06T05:26:16.994015+0000","last_active":"2023-10-06T05:26:17.001372+0000","last_peered":"2023-10-06T05:26:17.001372+0000","last_clean":"2023-10-06T05:26:17.001372+0000","last_became_active":"2023-10-06T05:26:16.993743+0000","last_became_peered":"2023-10-06T05:26:16.993743+0000","last_unstale":"2023-10-06T05:26:17.001372+0000","last_undegraded":"2023-10-06T05:26:17.001372+0000","last_fullsized":"2023-10-06T05:26:17.001372+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:18.349059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.995057+0000","last_change":"2023-10-06T05:26:15.978359+0000","last_active":"2023-10-06T05:26:16.995057+0000","last_peered":"2023-10-06T05:26:16.995057+0000","last_clean":"2023-10-06T05:26:16.995057+0000","last_became_active":"2023-10-06T05:26:15.978104+0000","last_became_peered":"2023-10-06T05:26:15.978104+0000","last_unstale":"2023-10-06T05:26:16.995057+0000","last_undegraded":"2023-10-06T05:26:16.995057+0000","last_fullsized":"2023-10-06T05:26:16.995057+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:30.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.012565+0000","last_change":"2023-10-06T05:26:17.006788+0000","last_active":"2023-10-06T05:26:17.012565+0000","last_peered":"2023-10-06T05:26:17.012565+0000","last_clean":"2023-10-06T05:26:17.012565+0000","last_became_active":"2023-10-06T05:26:17.005693+0000","last_became_peered":"2023-10-06T05:26:17.005693+0000","last_unstale":"2023-10-06T05:26:17.012565+0000","last_undegraded":"2023-10-06T05:26:17.012565+0000","last_fullsized":"2023-10-06T05:26:17.012565+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:03.829797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.993738+0000","last_change":"2023-10-06T05:26:15.989840+0000","last_active":"2023-10-06T05:26:16.993738+0000","last_peered":"2023-10-06T05:26:16.993738+0000","last_clean":"2023-10-06T05:26:16.993738+0000","last_became_active":"2023-10-06T05:26:15.989513+0000","last_became_peered":"2023-10-06T05:26:15.989513+0000","last_unstale":"2023-10-06T05:26:16.993738+0000","last_undegraded":"2023-10-06T05:26:16.993738+0000","last_fullsized":"2023-10-06T05:26:16.993738+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:22.175645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.015854+0000","last_change":"2023-10-06T05:26:17.010487+0000","last_active":"2023-10-06T05:26:17.015854+0000","last_peered":"2023-10-06T05:26:17.015854+0000","last_clean":"2023-10-06T05:26:17.015854+0000","last_became_active":"2023-10-06T05:26:17.009801+0000","last_became_peered":"2023-10-06T05:26:17.009801+0000","last_unstale":"2023-10-06T05:26:17.015854+0000","last_undegraded":"2023-10-06T05:26:17.015854+0000","last_fullsized":"2023-10-06T05:26:17.015854+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.887934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"168'8","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.020701+0000","last_change":"2023-10-06T05:26:15.983222+0000","last_active":"2023-10-06T05:26:17.020701+0000","last_peered":"2023-10-06T05:26:17.020701+0000","last_clean":"2023-10-06T05:26:17.020701+0000","last_became_active":"2023-10-06T05:26:15.982969+0000","last_became_peered":"2023-10-06T05:26:15.982969+0000","last_unstale":"2023-10-06T05:26:17.020701+0000","last_undegraded":"2023-10-06T05:26:17.020701+0000","last_fullsized":"2023-10-06T05:26:17.020701+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:58.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.004972+0000","last_change":"2023-10-06T05:26:17.002525+0000","last_active":"2023-10-06T05:26:17.004972+0000","last_peered":"2023-10-06T05:26:17.004972+0000","last_clean":"2023-10-06T05:26:17.004972+0000","last_became_active":"2023-10-06T05:26:17.002288+0000","last_became_peered":"2023-10-06T05:26:17.002288+0000","last_unstale":"2023-10-06T05:26:17.004972+0000","last_undegraded":"2023-10-06T05:26:17.004972+0000","last_fullsized":"2023-10-06T05:26:17.004972+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:23.955976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.020288+0000","last_change":"2023-10-06T05:26:15.983830+0000","last_active":"2023-10-06T05:26:17.020288+0000","last_peered":"2023-10-06T05:26:17.020288+0000","last_clean":"2023-10-06T05:26:17.020288+0000","last_became_active":"2023-10-06T05:26:15.983574+0000","last_became_peered":"2023-10-06T05:26:15.983574+0000","last_unstale":"2023-10-06T05:26:17.020288+0000","last_undegraded":"2023-10-06T05:26:17.020288+0000","last_fullsized":"2023-10-06T05:26:17.020288+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:17.508902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.014459+0000","last_change":"2023-10-06T05:26:17.009077+0000","last_active":"2023-10-06T05:26:17.014459+0000","last_peered":"2023-10-06T05:26:17.014459+0000","last_clean":"2023-10-06T05:26:17.014459+0000","last_became_active":"2023-10-06T05:26:17.008715+0000","last_became_peered":"2023-10-06T05:26:17.008715+0000","last_unstale":"2023-10-06T05:26:17.014459+0000","last_undegraded":"2023-10-06T05:26:17.014459+0000","last_fullsized":"2023-10-06T05:26:17.014459+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:46.038362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.017158+0000","last_change":"2023-10-06T05:26:15.989957+0000","last_active":"2023-10-06T05:26:17.017158+0000","last_peered":"2023-10-06T05:26:17.017158+0000","last_clean":"2023-10-06T05:26:17.017158+0000","last_became_active":"2023-10-06T05:26:15.989449+0000","last_became_peered":"2023-10-06T05:26:15.989449+0000","last_unstale":"2023-10-06T05:26:17.017158+0000","last_undegraded":"2023-10-06T05:26:17.017158+0000","last_fullsized":"2023-10-06T05:26:17.017158+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:30.253659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016800+0000","last_change":"2023-10-06T05:26:17.010825+0000","last_active":"2023-10-06T05:26:17.016800+0000","last_peered":"2023-10-06T05:26:17.016800+0000","last_clean":"2023-10-06T05:26:17.016800+0000","last_became_active":"2023-10-06T05:26:17.010155+0000","last_became_peered":"2023-10-06T05:26:17.010155+0000","last_unstale":"2023-10-06T05:26:17.016800+0000","last_undegraded":"2023-10-06T05:26:17.016800+0000","last_fullsized":"2023-10-06T05:26:17.016800+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:28.219238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.020043+0000","last_change":"2023-10-06T05:26:15.995078+0000","last_active":"2023-10-06T05:26:17.020043+0000","last_peered":"2023-10-06T05:26:17.020043+0000","last_clean":"2023-10-06T05:26:17.020043+0000","last_became_active":"2023-10-06T05:26:15.994684+0000","last_became_peered":"2023-10-06T05:26:15.994684+0000","last_unstale":"2023-10-06T05:26:17.020043+0000","last_undegraded":"2023-10-06T05:26:17.020043+0000","last_fullsized":"2023-10-06T05:26:17.020043+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:19.418728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.008757+0000","last_change":"2023-10-06T05:26:17.005179+0000","last_active":"2023-10-06T05:26:17.008757+0000","last_peered":"2023-10-06T05:26:17.008757+0000","last_clean":"2023-10-06T05:26:17.008757+0000","last_became_active":"2023-10-06T05:26:17.004939+0000","last_became_peered":"2023-10-06T05:26:17.004939+0000","last_unstale":"2023-10-06T05:26:17.008757+0000","last_undegraded":"2023-10-06T05:26:17.008757+0000","last_fullsized":"2023-10-06T05:26:17.008757+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:56.462173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.999534+0000","last_change":"2023-10-06T05:26:15.981157+0000","last_active":"2023-10-06T05:26:16.999534+0000","last_peered":"2023-10-06T05:26:16.999534+0000","last_clean":"2023-10-06T05:26:16.999534+0000","last_became_active":"2023-10-06T05:26:15.980886+0000","last_became_peered":"2023-10-06T05:26:15.980886+0000","last_unstale":"2023-10-06T05:26:16.999534+0000","last_undegraded":"2023-10-06T05:26:16.999534+0000","last_fullsized":"2023-10-06T05:26:16.999534+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:55.779205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016179+0000","last_change":"2023-10-06T05:26:17.011289+0000","last_active":"2023-10-06T05:26:17.016179+0000","last_peered":"2023-10-06T05:26:17.016179+0000","last_clean":"2023-10-06T05:26:17.016179+0000","last_became_active":"2023-10-06T05:26:17.011003+0000","last_became_peered":"2023-10-06T05:26:17.011003+0000","last_unstale":"2023-10-06T05:26:17.016179+0000","last_undegraded":"2023-10-06T05:26:17.016179+0000","last_fullsized":"2023-10-06T05:26:17.016179+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:20.848379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.019088+0000","last_change":"2023-10-06T05:26:15.996997+0000","last_active":"2023-10-06T05:26:17.019088+0000","last_peered":"2023-10-06T05:26:17.019088+0000","last_clean":"2023-10-06T05:26:17.019088+0000","last_became_active":"2023-10-06T05:26:15.996768+0000","last_became_peered":"2023-10-06T05:26:15.996768+0000","last_unstale":"2023-10-06T05:26:17.019088+0000","last_undegraded":"2023-10-06T05:26:17.019088+0000","last_fullsized":"2023-10-06T05:26:17.019088+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:36.365849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.011875+0000","last_change":"2023-10-06T05:26:17.010366+0000","last_active":"2023-10-06T05:26:17.011875+0000","last_peered":"2023-10-06T05:26:17.011875+0000","last_clean":"2023-10-06T05:26:17.011875+0000","last_became_active":"2023-10-06T05:26:17.010104+0000","last_became_peered":"2023-10-06T05:26:17.010104+0000","last_unstale":"2023-10-06T05:26:17.011875+0000","last_undegraded":"2023-10-06T05:26:17.011875+0000","last_fullsized":"2023-10-06T05:26:17.011875+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:50.313717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.993704+0000","last_change":"2023-10-06T05:26:15.979984+0000","last_active":"2023-10-06T05:26:16.993704+0000","last_peered":"2023-10-06T05:26:16.993704+0000","last_clean":"2023-10-06T05:26:16.993704+0000","last_became_active":"2023-10-06T05:26:15.979679+0000","last_became_peered":"2023-10-06T05:26:15.979679+0000","last_unstale":"2023-10-06T05:26:16.993704+0000","last_undegraded":"2023-10-06T05:26:16.993704+0000","last_fullsized":"2023-10-06T05:26:16.993704+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:02.218986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.014314+0000","last_change":"2023-10-06T05:26:17.012464+0000","last_active":"2023-10-06T05:26:17.014314+0000","last_peered":"2023-10-06T05:26:17.014314+0000","last_clean":"2023-10-06T05:26:17.014314+0000","last_became_active":"2023-10-06T05:26:17.012186+0000","last_became_peered":"2023-10-06T05:26:17.012186+0000","last_unstale":"2023-10-06T05:26:17.014314+0000","last_undegraded":"2023-10-06T05:26:17.014314+0000","last_fullsized":"2023-10-06T05:26:17.014314+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:25.219842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.982493+0000","last_change":"2023-10-06T05:26:15.987088+0000","last_active":"2023-10-06T05:26:16.982493+0000","last_peered":"2023-10-06T05:26:16.982493+0000","last_clean":"2023-10-06T05:26:16.982493+0000","last_became_active":"2023-10-06T05:26:15.986754+0000","last_became_peered":"2023-10-06T05:26:15.986754+0000","last_unstale":"2023-10-06T05:26:16.982493+0000","last_undegraded":"2023-10-06T05:26:16.982493+0000","last_fullsized":"2023-10-06T05:26:16.982493+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:09.197792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.005738+0000","last_change":"2023-10-06T05:26:16.997319+0000","last_active":"2023-10-06T05:26:17.005738+0000","last_peered":"2023-10-06T05:26:17.005738+0000","last_clean":"2023-10-06T05:26:17.005738+0000","last_became_active":"2023-10-06T05:26:16.996600+0000","last_became_peered":"2023-10-06T05:26:16.996600+0000","last_unstale":"2023-10-06T05:26:17.005738+0000","last_undegraded":"2023-10-06T05:26:17.005738+0000","last_fullsized":"2023-10-06T05:26:17.005738+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:48.026485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.986242+0000","last_change":"2023-10-06T05:26:15.979209+0000","last_active":"2023-10-06T05:26:16.986242+0000","last_peered":"2023-10-06T05:26:16.986242+0000","last_clean":"2023-10-06T05:26:16.986242+0000","last_became_active":"2023-10-06T05:26:15.978945+0000","last_became_peered":"2023-10-06T05:26:15.978945+0000","last_unstale":"2023-10-06T05:26:16.986242+0000","last_undegraded":"2023-10-06T05:26:16.986242+0000","last_fullsized":"2023-10-06T05:26:16.986242+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:48.329089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016449+0000","last_change":"2023-10-06T05:26:17.011840+0000","last_active":"2023-10-06T05:26:17.016449+0000","last_peered":"2023-10-06T05:26:17.016449+0000","last_clean":"2023-10-06T05:26:17.016449+0000","last_became_active":"2023-10-06T05:26:17.011542+0000","last_became_peered":"2023-10-06T05:26:17.011542+0000","last_unstale":"2023-10-06T05:26:17.016449+0000","last_undegraded":"2023-10-06T05:26:17.016449+0000","last_fullsized":"2023-10-06T05:26:17.016449+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:07.916170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010703+0000","last_change":"2023-10-06T05:26:15.981648+0000","last_active":"2023-10-06T05:26:17.010703+0000","last_peered":"2023-10-06T05:26:17.010703+0000","last_clean":"2023-10-06T05:26:17.010703+0000","last_became_active":"2023-10-06T05:26:15.980699+0000","last_became_peered":"2023-10-06T05:26:15.980699+0000","last_unstale":"2023-10-06T05:26:17.010703+0000","last_undegraded":"2023-10-06T05:26:17.010703+0000","last_fullsized":"2023-10-06T05:26:17.010703+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:07.998982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.010454+0000","last_change":"2023-10-06T05:26:17.006992+0000","last_active":"2023-10-06T05:26:17.010454+0000","last_peered":"2023-10-06T05:26:17.010454+0000","last_clean":"2023-10-06T05:26:17.010454+0000","last_became_active":"2023-10-06T05:26:17.006724+0000","last_became_peered":"2023-10-06T05:26:17.006724+0000","last_unstale":"2023-10-06T05:26:17.010454+0000","last_undegraded":"2023-10-06T05:26:17.010454+0000","last_fullsized":"2023-10-06T05:26:17.010454+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:06.789461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-06T05:26:18.286314+0000","last_change":"2023-10-06T05:26:15.986229+0000","last_active":"2023-10-06T05:26:18.286314+0000","last_peered":"2023-10-06T05:26:18.286314+0000","last_clean":"2023-10-06T05:26:18.286314+0000","last_became_active":"2023-10-06T05:26:15.985959+0000","last_became_peered":"2023-10-06T05:26:15.985959+0000","last_unstale":"2023-10-06T05:26:18.286314+0000","last_undegraded":"2023-10-06T05:26:18.286314+0000","last_fullsized":"2023-10-06T05:26:18.286314+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:34.700237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.001812+0000","last_change":"2023-10-06T05:26:17.000304+0000","last_active":"2023-10-06T05:26:17.001812+0000","last_peered":"2023-10-06T05:26:17.001812+0000","last_clean":"2023-10-06T05:26:17.001812+0000","last_became_active":"2023-10-06T05:26:17.000004+0000","last_became_peered":"2023-10-06T05:26:17.000004+0000","last_unstale":"2023-10-06T05:26:17.001812+0000","last_undegraded":"2023-10-06T05:26:17.001812+0000","last_fullsized":"2023-10-06T05:26:17.001812+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:46.117421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.987714+0000","last_change":"2023-10-06T05:26:15.984823+0000","last_active":"2023-10-06T05:26:16.987714+0000","last_peered":"2023-10-06T05:26:16.987714+0000","last_clean":"2023-10-06T05:26:16.987714+0000","last_became_active":"2023-10-06T05:26:15.984551+0000","last_became_peered":"2023-10-06T05:26:15.984551+0000","last_unstale":"2023-10-06T05:26:16.987714+0000","last_undegraded":"2023-10-06T05:26:16.987714+0000","last_fullsized":"2023-10-06T05:26:16.987714+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:42.561369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.017666+0000","last_change":"2023-10-06T05:26:17.013096+0000","last_active":"2023-10-06T05:26:17.017666+0000","last_peered":"2023-10-06T05:26:17.017666+0000","last_clean":"2023-10-06T05:26:17.017666+0000","last_became_active":"2023-10-06T05:26:17.012835+0000","last_became_peered":"2023-10-06T05:26:17.012835+0000","last_unstale":"2023-10-06T05:26:17.017666+0000","last_undegraded":"2023-10-06T05:26:17.017666+0000","last_fullsized":"2023-10-06T05:26:17.017666+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:06.985236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.987061+0000","last_change":"2023-10-06T05:26:15.993515+0000","last_active":"2023-10-06T05:26:16.987061+0000","last_peered":"2023-10-06T05:26:16.987061+0000","last_clean":"2023-10-06T05:26:16.987061+0000","last_became_active":"2023-10-06T05:26:15.993249+0000","last_became_peered":"2023-10-06T05:26:15.993249+0000","last_unstale":"2023-10-06T05:26:16.987061+0000","last_undegraded":"2023-10-06T05:26:16.987061+0000","last_fullsized":"2023-10-06T05:26:16.987061+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:45.111652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006530+0000","last_change":"2023-10-06T05:26:17.003094+0000","last_active":"2023-10-06T05:26:17.006530+0000","last_peered":"2023-10-06T05:26:17.006530+0000","last_clean":"2023-10-06T05:26:17.006530+0000","last_became_active":"2023-10-06T05:26:17.002700+0000","last_became_peered":"2023-10-06T05:26:17.002700+0000","last_unstale":"2023-10-06T05:26:17.006530+0000","last_undegraded":"2023-10-06T05:26:17.006530+0000","last_fullsized":"2023-10-06T05:26:17.006530+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:09.903204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.991207+0000","last_change":"2023-10-06T05:26:15.990744+0000","last_active":"2023-10-06T05:26:16.991207+0000","last_peered":"2023-10-06T05:26:16.991207+0000","last_clean":"2023-10-06T05:26:16.991207+0000","last_became_active":"2023-10-06T05:26:15.990214+0000","last_became_peered":"2023-10-06T05:26:15.990214+0000","last_unstale":"2023-10-06T05:26:16.991207+0000","last_undegraded":"2023-10-06T05:26:16.991207+0000","last_fullsized":"2023-10-06T05:26:16.991207+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:57.407069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006174+0000","last_change":"2023-10-06T05:26:17.001276+0000","last_active":"2023-10-06T05:26:17.006174+0000","last_peered":"2023-10-06T05:26:17.006174+0000","last_clean":"2023-10-06T05:26:17.006174+0000","last_became_active":"2023-10-06T05:26:17.000985+0000","last_became_peered":"2023-10-06T05:26:17.000985+0000","last_unstale":"2023-10-06T05:26:17.006174+0000","last_undegraded":"2023-10-06T05:26:17.006174+0000","last_fullsized":"2023-10-06T05:26:17.006174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:20.532361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.991396+0000","last_change":"2023-10-06T05:26:15.989218+0000","last_active":"2023-10-06T05:26:16.991396+0000","last_peered":"2023-10-06T05:26:16.991396+0000","last_clean":"2023-10-06T05:26:16.991396+0000","last_became_active":"2023-10-06T05:26:15.988892+0000","last_became_peered":"2023-10-06T05:26:15.988892+0000","last_unstale":"2023-10-06T05:26:16.991396+0000","last_undegraded":"2023-10-06T05:26:16.991396+0000","last_fullsized":"2023-10-06T05:26:16.991396+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:19.526591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.014046+0000","last_change":"2023-10-06T05:26:17.010616+0000","last_active":"2023-10-06T05:26:17.014046+0000","last_peered":"2023-10-06T05:26:17.014046+0000","last_clean":"2023-10-06T05:26:17.014046+0000","last_became_active":"2023-10-06T05:26:17.010325+0000","last_became_peered":"2023-10-06T05:26:17.010325+0000","last_unstale":"2023-10-06T05:26:17.014046+0000","last_undegraded":"2023-10-06T05:26:17.014046+0000","last_fullsized":"2023-10-06T05:26:17.014046+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:47.593154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.982866+0000","last_change":"2023-10-06T05:26:15.981438+0000","last_active":"2023-10-06T05:26:16.982866+0000","last_peered":"2023-10-06T05:26:16.982866+0000","last_clean":"2023-10-06T05:26:16.982866+0000","last_became_active":"2023-10-06T05:26:15.979239+0000","last_became_peered":"2023-10-06T05:26:15.979239+0000","last_unstale":"2023-10-06T05:26:16.982866+0000","last_undegraded":"2023-10-06T05:26:16.982866+0000","last_fullsized":"2023-10-06T05:26:16.982866+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:50.444147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.006680+0000","last_change":"2023-10-06T05:26:17.004415+0000","last_active":"2023-10-06T05:26:17.006680+0000","last_peered":"2023-10-06T05:26:17.006680+0000","last_clean":"2023-10-06T05:26:17.006680+0000","last_became_active":"2023-10-06T05:26:17.004164+0000","last_became_peered":"2023-10-06T05:26:17.004164+0000","last_unstale":"2023-10-06T05:26:17.006680+0000","last_undegraded":"2023-10-06T05:26:17.006680+0000","last_fullsized":"2023-10-06T05:26:17.006680+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:36.010312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.986359+0000","last_change":"2023-10-06T05:26:15.981307+0000","last_active":"2023-10-06T05:26:16.986359+0000","last_peered":"2023-10-06T05:26:16.986359+0000","last_clean":"2023-10-06T05:26:16.986359+0000","last_became_active":"2023-10-06T05:26:15.981055+0000","last_became_peered":"2023-10-06T05:26:15.981055+0000","last_unstale":"2023-10-06T05:26:16.986359+0000","last_undegraded":"2023-10-06T05:26:16.986359+0000","last_fullsized":"2023-10-06T05:26:16.986359+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:32.428141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.014007+0000","last_change":"2023-10-06T05:26:17.011846+0000","last_active":"2023-10-06T05:26:17.014007+0000","last_peered":"2023-10-06T05:26:17.014007+0000","last_clean":"2023-10-06T05:26:17.014007+0000","last_became_active":"2023-10-06T05:26:17.011499+0000","last_became_peered":"2023-10-06T05:26:17.011499+0000","last_unstale":"2023-10-06T05:26:17.014007+0000","last_undegraded":"2023-10-06T05:26:17.014007+0000","last_fullsized":"2023-10-06T05:26:17.014007+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:09.253354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"168'4","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.015038+0000","last_change":"2023-10-06T05:26:15.981474+0000","last_active":"2023-10-06T05:26:17.015038+0000","last_peered":"2023-10-06T05:26:17.015038+0000","last_clean":"2023-10-06T05:26:17.015038+0000","last_became_active":"2023-10-06T05:26:15.980370+0000","last_became_peered":"2023-10-06T05:26:15.980370+0000","last_unstale":"2023-10-06T05:26:17.015038+0000","last_undegraded":"2023-10-06T05:26:17.015038+0000","last_fullsized":"2023-10-06T05:26:17.015038+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:35.004542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.997133+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:26:16.997133+0000","last_peered":"2023-10-06T05:26:16.997133+0000","last_clean":"2023-10-06T05:26:16.997133+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:26:16.997133+0000","last_undegraded":"2023-10-06T05:26:16.997133+0000","last_fullsized":"2023-10-06T05:26:16.997133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.002280+0000","last_change":"2023-10-06T05:26:17.000930+0000","last_active":"2023-10-06T05:26:17.002280+0000","last_peered":"2023-10-06T05:26:17.002280+0000","last_clean":"2023-10-06T05:26:17.002280+0000","last_became_active":"2023-10-06T05:26:17.000583+0000","last_became_peered":"2023-10-06T05:26:17.000583+0000","last_unstale":"2023-10-06T05:26:17.002280+0000","last_undegraded":"2023-10-06T05:26:17.002280+0000","last_fullsized":"2023-10-06T05:26:17.002280+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:09.004752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.990811+0000","last_change":"2023-10-06T05:26:15.989314+0000","last_active":"2023-10-06T05:26:16.990811+0000","last_peered":"2023-10-06T05:26:16.990811+0000","last_clean":"2023-10-06T05:26:16.990811+0000","last_became_active":"2023-10-06T05:26:15.988969+0000","last_became_peered":"2023-10-06T05:26:15.988969+0000","last_unstale":"2023-10-06T05:26:16.990811+0000","last_undegraded":"2023-10-06T05:26:16.990811+0000","last_fullsized":"2023-10-06T05:26:16.990811+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:26.549296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.000760+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:26:17.000760+0000","last_peered":"2023-10-06T05:26:17.000760+0000","last_clean":"2023-10-06T05:26:17.000760+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:26:17.000760+0000","last_undegraded":"2023-10-06T05:26:17.000760+0000","last_fullsized":"2023-10-06T05:26:17.000760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.011250+0000","last_change":"2023-10-06T05:26:17.008956+0000","last_active":"2023-10-06T05:26:17.011250+0000","last_peered":"2023-10-06T05:26:17.011250+0000","last_clean":"2023-10-06T05:26:17.011250+0000","last_became_active":"2023-10-06T05:26:17.008491+0000","last_became_peered":"2023-10-06T05:26:17.008491+0000","last_unstale":"2023-10-06T05:26:17.011250+0000","last_undegraded":"2023-10-06T05:26:17.011250+0000","last_fullsized":"2023-10-06T05:26:17.011250+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:43.567138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.018481+0000","last_change":"2023-10-06T05:26:15.990882+0000","last_active":"2023-10-06T05:26:17.018481+0000","last_peered":"2023-10-06T05:26:17.018481+0000","last_clean":"2023-10-06T05:26:17.018481+0000","last_became_active":"2023-10-06T05:26:15.989632+0000","last_became_peered":"2023-10-06T05:26:15.989632+0000","last_unstale":"2023-10-06T05:26:17.018481+0000","last_undegraded":"2023-10-06T05:26:17.018481+0000","last_fullsized":"2023-10-06T05:26:17.018481+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:41.444233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.982956+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:26:16.982956+0000","last_peered":"2023-10-06T05:26:16.982956+0000","last_clean":"2023-10-06T05:26:16.982956+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:26:16.982956+0000","last_undegraded":"2023-10-06T05:26:16.982956+0000","last_fullsized":"2023-10-06T05:26:16.982956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.999258+0000","last_change":"2023-10-06T05:26:16.996144+0000","last_active":"2023-10-06T05:26:16.999258+0000","last_peered":"2023-10-06T05:26:16.999258+0000","last_clean":"2023-10-06T05:26:16.999258+0000","last_became_active":"2023-10-06T05:26:16.995869+0000","last_became_peered":"2023-10-06T05:26:16.995869+0000","last_unstale":"2023-10-06T05:26:16.999258+0000","last_undegraded":"2023-10-06T05:26:16.999258+0000","last_fullsized":"2023-10-06T05:26:16.999258+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:35.202260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.994377+0000","last_change":"2023-10-06T05:26:15.988033+0000","last_active":"2023-10-06T05:26:16.994377+0000","last_peered":"2023-10-06T05:26:16.994377+0000","last_clean":"2023-10-06T05:26:16.994377+0000","last_became_active":"2023-10-06T05:26:15.987719+0000","last_became_peered":"2023-10-06T05:26:15.987719+0000","last_unstale":"2023-10-06T05:26:16.994377+0000","last_undegraded":"2023-10-06T05:26:16.994377+0000","last_fullsized":"2023-10-06T05:26:16.994377+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:19.842609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.994963+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:26:16.994963+0000","last_peered":"2023-10-06T05:26:16.994963+0000","last_clean":"2023-10-06T05:26:16.994963+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:26:16.994963+0000","last_undegraded":"2023-10-06T05:26:16.994963+0000","last_fullsized":"2023-10-06T05:26:16.994963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.993114+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:26:16.993114+0000","last_peered":"2023-10-06T05:26:16.993114+0000","last_clean":"2023-10-06T05:26:16.993114+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:26:16.993114+0000","last_undegraded":"2023-10-06T05:26:16.993114+0000","last_fullsized":"2023-10-06T05:26:16.993114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.998899+0000","last_change":"2023-10-06T05:26:16.997711+0000","last_active":"2023-10-06T05:26:16.998899+0000","last_peered":"2023-10-06T05:26:16.998899+0000","last_clean":"2023-10-06T05:26:16.998899+0000","last_became_active":"2023-10-06T05:26:16.997179+0000","last_became_peered":"2023-10-06T05:26:16.997179+0000","last_unstale":"2023-10-06T05:26:16.998899+0000","last_undegraded":"2023-10-06T05:26:16.998899+0000","last_fullsized":"2023-10-06T05:26:16.998899+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:12.238141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.019265+0000","last_change":"2023-10-06T05:26:15.984905+0000","last_active":"2023-10-06T05:26:17.019265+0000","last_peered":"2023-10-06T05:26:17.019265+0000","last_clean":"2023-10-06T05:26:17.019265+0000","last_became_active":"2023-10-06T05:26:15.984636+0000","last_became_peered":"2023-10-06T05:26:15.984636+0000","last_unstale":"2023-10-06T05:26:17.019265+0000","last_undegraded":"2023-10-06T05:26:17.019265+0000","last_fullsized":"2023-10-06T05:26:17.019265+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:37.207464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":132,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.993023+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:26:16.993023+0000","last_peered":"2023-10-06T05:26:16.993023+0000","last_clean":"2023-10-06T05:26:16.993023+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:26:16.993023+0000","last_undegraded":"2023-10-06T05:26:16.993023+0000","last_fullsized":"2023-10-06T05:26:16.993023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.004890+0000","last_change":"2023-10-06T05:26:17.002845+0000","last_active":"2023-10-06T05:26:17.004890+0000","last_peered":"2023-10-06T05:26:17.004890+0000","last_clean":"2023-10-06T05:26:17.004890+0000","last_became_active":"2023-10-06T05:26:17.002502+0000","last_became_peered":"2023-10-06T05:26:17.002502+0000","last_unstale":"2023-10-06T05:26:17.004890+0000","last_undegraded":"2023-10-06T05:26:17.004890+0000","last_fullsized":"2023-10-06T05:26:17.004890+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:17.817823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"168'1","reported_seq":11,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.019516+0000","last_change":"2023-10-06T05:26:15.987554+0000","last_active":"2023-10-06T05:26:17.019516+0000","last_peered":"2023-10-06T05:26:17.019516+0000","last_clean":"2023-10-06T05:26:17.019516+0000","last_became_active":"2023-10-06T05:26:15.987264+0000","last_became_peered":"2023-10-06T05:26:15.987264+0000","last_unstale":"2023-10-06T05:26:17.019516+0000","last_undegraded":"2023-10-06T05:26:17.019516+0000","last_fullsized":"2023-10-06T05:26:17.019516+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:05.783691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":132,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.991551+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:26:16.991551+0000","last_peered":"2023-10-06T05:26:16.991551+0000","last_clean":"2023-10-06T05:26:16.991551+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:26:16.991551+0000","last_undegraded":"2023-10-06T05:26:16.991551+0000","last_fullsized":"2023-10-06T05:26:16.991551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.016821+0000","last_change":"2023-10-06T05:26:17.011002+0000","last_active":"2023-10-06T05:26:17.016821+0000","last_peered":"2023-10-06T05:26:17.016821+0000","last_clean":"2023-10-06T05:26:17.016821+0000","last_became_active":"2023-10-06T05:26:17.010441+0000","last_became_peered":"2023-10-06T05:26:17.010441+0000","last_unstale":"2023-10-06T05:26:17.016821+0000","last_undegraded":"2023-10-06T05:26:17.016821+0000","last_fullsized":"2023-10-06T05:26:17.016821+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:35.916680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.018601+0000","last_change":"2023-10-06T05:26:15.989016+0000","last_active":"2023-10-06T05:26:17.018601+0000","last_peered":"2023-10-06T05:26:17.018601+0000","last_clean":"2023-10-06T05:26:17.018601+0000","last_became_active":"2023-10-06T05:26:15.988637+0000","last_became_peered":"2023-10-06T05:26:15.988637+0000","last_unstale":"2023-10-06T05:26:17.018601+0000","last_undegraded":"2023-10-06T05:26:17.018601+0000","last_fullsized":"2023-10-06T05:26:17.018601+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.340712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.994477+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:26:16.994477+0000","last_peered":"2023-10-06T05:26:16.994477+0000","last_clean":"2023-10-06T05:26:16.994477+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:26:16.994477+0000","last_undegraded":"2023-10-06T05:26:16.994477+0000","last_fullsized":"2023-10-06T05:26:16.994477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.015944+0000","last_change":"2023-10-06T05:26:17.013375+0000","last_active":"2023-10-06T05:26:17.015944+0000","last_peered":"2023-10-06T05:26:17.015944+0000","last_clean":"2023-10-06T05:26:17.015944+0000","last_became_active":"2023-10-06T05:26:17.013088+0000","last_became_peered":"2023-10-06T05:26:17.013088+0000","last_unstale":"2023-10-06T05:26:17.015944+0000","last_undegraded":"2023-10-06T05:26:17.015944+0000","last_fullsized":"2023-10-06T05:26:17.015944+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:15.960021+0000","last_clean_scrub_stamp":"2023-10-06T05:26:15.960021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.765873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"168'2","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:17.017817+0000","last_change":"2023-10-06T05:26:15.990119+0000","last_active":"2023-10-06T05:26:17.017817+0000","last_peered":"2023-10-06T05:26:17.017817+0000","last_clean":"2023-10-06T05:26:17.017817+0000","last_became_active":"2023-10-06T05:26:15.988477+0000","last_became_peered":"2023-10-06T05:26:15.988477+0000","last_unstale":"2023-10-06T05:26:17.017817+0000","last_undegraded":"2023-10-06T05:26:17.017817+0000","last_fullsized":"2023-10-06T05:26:17.017817+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:14.954251+0000","last_clean_scrub_stamp":"2023-10-06T05:26:14.954251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:21.716854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-10-06T05:26:16.985831+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:26:16.985831+0000","last_peered":"2023-10-06T05:26:16.985831+0000","last_clean":"2023-10-06T05:26:16.985831+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:26:16.985831+0000","last_undegraded":"2023-10-06T05:26:16.985831+0000","last_fullsized":"2023-10-06T05:26:16.985831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304040,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067800,"statfs":{"total":96636764160,"available":96325427200,"internally_reserved":0,"allocated":2719744,"data_stored":1575943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.87,"5min":0.95999999999999996,"15min":1.1850000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":1.1339999999999999,"15min":1.169},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.47199999999999998,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.268},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.96099999999999997,"15min":1.075},"last":0.627}]},{"osd":2,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.47899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97599999999999998,"5min":1.0780000000000001,"15min":1.093},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.22},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93200000000000005,"5min":1.0249999999999999,"15min":1.079},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.0960000000000001,"15min":1.202},"last":0.749}]},{"osd":4,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.625},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.0209999999999999,"15min":1.258},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.42199999999999999,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":1.109,"15min":1.109},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.746},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":1.0720000000000001,"15min":1.407},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.54800000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80000000000000004,"5min":1.1220000000000001,"15min":1.139},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":0.97799999999999998,"5min":1.135,"15min":1.3300000000000001},"last":1.0489999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303464,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068376,"statfs":{"total":96636764160,"available":96326017024,"internally_reserved":0,"allocated":2129920,"data_stored":985645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.51600000000000001,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.92800000000000005,"5min":0.98099999999999998,"15min":1.034},"last":0.752},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":0.96499999999999997,"5min":0.995,"15min":1.2869999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.70599999999999996},"min":{"1min":0.56299999999999994,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":0.88700000000000001,"5min":0.93400000000000005,"15min":1},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.71699999999999997},"min":{"1min":0.58499999999999996,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.98899999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.72899999999999998},"min":{"1min":0.54300000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85699999999999998,"5min":0.95099999999999996,"15min":1.1599999999999999},"last":0.505},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.73299999999999998},"min":{"1min":0.54700000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.89800000000000002,"5min":0.93799999999999994,"15min":0.98599999999999999},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.59199999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72899999999999998,"15min":0.77100000000000002},"min":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.5},"max":{"1min":0.88800000000000001,"5min":0.93200000000000005,"15min":1.1020000000000001},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.626,"15min":0.65100000000000002},"min":{"1min":0.52200000000000002,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95699999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64400000000000002,"15min":0.67500000000000004},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.42099999999999999},"max":{"1min":0.88200000000000001,"5min":0.89100000000000001,"15min":0.998},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67600000000000005,"15min":0.70699999999999996},"min":{"1min":0.52200000000000002,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0760000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71099999999999997,"15min":0.74099999999999999},"min":{"1min":0.52300000000000002,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":0.96499999999999997,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70899999999999996,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":0.94099999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.745,"15min":0.79200000000000004},"min":{"1min":0.62,"5min":0.496,"15min":0.436},"max":{"1min":1.006,"5min":1.006,"15min":1.1359999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":985575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46999999999999997,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.69999999999999996,"5min":0.81799999999999995,"15min":2.7349999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.27800000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":2.6520000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.88200000000000001,"15min":2.7010000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.61799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.86299999999999999,"5min":0.88,"15min":2.7639999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52500000000000002,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.90000000000000002,"15min":2.8370000000000002},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50700000000000001},"max":{"1min":1.0249999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.79100000000000004},"min":{"1min":0.60599999999999998,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.876}]},{"osd":5,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.0489999999999999,"5min":1.25,"15min":1.349},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.55600000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.073,"5min":1.2569999999999999,"15min":1.669},"last":0.755}]},{"osd":6,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.83899999999999997},"min":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0840000000000001,"5min":1.1910000000000001,"15min":1.6040000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.57099999999999995,"15min":0.503},"max":{"1min":1.0620000000000001,"5min":1.3280000000000001,"15min":1.573},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1100000000000001,"5min":1.1870000000000001,"15min":1.528},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":1.0089999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304056,"kb_used_data":2672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067784,"statfs":{"total":96636764160,"available":96325410816,"internally_reserved":0,"allocated":2736128,"data_stored":1576643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30499999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.497},"min":{"1min":0.313,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70799999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.36699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.67200000000000004,"5min":0.70999999999999996,"15min":0.84999999999999998},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77000000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81100000000000005,"5min":0.84499999999999997,"15min":0.879},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78300000000000003,"15min":0.77000000000000002},"min":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":3.6629999999999998,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.77700000000000002},"min":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.159,"15min":1.218},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95499999999999996,"5min":1.089,"15min":1.1970000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.1779999999999999,"15min":1.1839999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.153,"5min":1.153,"15min":1.2629999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60299999999999998,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.044,"5min":1.0780000000000001,"15min":1.25},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.59899999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.087,"5min":1.256,"15min":1.2789999999999999},"last":0.72199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303464,"kb_used_data":2080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068376,"statfs":{"total":96636764160,"available":96326017024,"internally_reserved":0,"allocated":2129920,"data_stored":986129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":1.056,"15min":1.056},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72799999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.443}]},{"osd":1,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.626,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.495},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.44900000000000001,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.79800000000000004,"5min":1.073,"15min":1.073},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46000000000000002,"15min":0.47099999999999997},"min":{"1min":0.35899999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.60899999999999999,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.432},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.81200000000000006,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0940000000000001,"5min":1.109,"15min":1.1379999999999999},"last":0.623},{"interface":"front","average":{"1min":0.748,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.54500000000000004,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":1.071,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73999999999999999,"15min":0.76800000000000002},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":1.0640000000000001,"5min":1.1859999999999999,"15min":1.319},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.80800000000000005},"min":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0309999999999999,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0629999999999999,"5min":1.137,"15min":1.141},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.80400000000000005},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.52100000000000002},"max":{"1min":0.98999999999999999,"5min":1.284,"15min":1.284},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78600000000000003,"15min":0.81899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0980000000000001,"5min":1.2,"15min":1.3520000000000001},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303468,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068372,"statfs":{"total":96636764160,"available":96326012928,"internally_reserved":0,"allocated":2134016,"data_stored":985735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45500000000000002,"15min":0.48699999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":2.0950000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46600000000000003,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.54700000000000004,"5min":0.77100000000000002,"15min":1.772},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":1.913},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.55400000000000005},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.68100000000000005,"5min":0.95899999999999996,"15min":2.0089999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.63500000000000001,"5min":0.79700000000000004,"15min":1.954},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82599999999999996,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.51400000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.042,"5min":1.101,"15min":1.282},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75700000000000001,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":1.117,"5min":1.1839999999999999,"15min":1.2609999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.79200000000000004},"min":{"1min":0.63800000000000001,"5min":0.54300000000000004,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.145,"15min":1.1930000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0940000000000001,"5min":1.21,"15min":1.3009999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78700000000000003,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.53900000000000003},"max":{"1min":1.042,"5min":1.1419999999999999,"15min":1.343},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.60199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":1.0649999999999999,"15min":1.3380000000000001},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79600000000000004,"15min":0.83299999999999996},"min":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.0800000000000001,"5min":1.194,"15min":1.3640000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.63300000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.0780000000000001,"5min":1.1579999999999999,"15min":1.214},"last":0.79600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303476,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068364,"statfs":{"total":96636764160,"available":96326004736,"internally_reserved":0,"allocated":2142208,"data_stored":987145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.073},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.46100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0089999999999999,"15min":1.169},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44900000000000001,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":1.0569999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69499999999999995,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.82299999999999995,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.117},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.97099999999999997,"15min":1.002},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72099999999999997,"15min":0.73799999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91000000000000003,"5min":0.97599999999999998,"15min":1.0780000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.44600000000000001},"max":{"1min":0.90700000000000003,"5min":1.018,"15min":1.196},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.92400000000000004,"15min":1.8740000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.95299999999999996,"5min":0.999,"15min":1.375},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.97399999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73999999999999999,"15min":0.77100000000000002},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0209999999999999,"15min":1.9390000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.78400000000000003},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.48199999999999998},"max":{"1min":0.95899999999999996,"5min":1.05,"15min":1.466},"last":0.874},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.218},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303476,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068364,"statfs":{"total":96636764160,"available":96326004736,"internally_reserved":0,"allocated":2142208,"data_stored":987133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.54500000000000004,"5min":0.374,"15min":0.374},"max":{"1min":0.88200000000000001,"5min":0.93899999999999995,"15min":1.165},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.1890000000000001},"last":1.0349999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.98299999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.49199999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.995,"5min":1.048,"15min":1.048},"last":0.93600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.55400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.204},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92400000000000004,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.876}]},{"osd":3,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94599999999999995,"5min":0.95199999999999996,"15min":1.087},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.51900000000000002,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":0.93000000000000005,"5min":0.99299999999999999,"15min":1.278},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.92800000000000005,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.496},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.004,"5min":1.1539999999999999,"15min":1.294},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":0.96899999999999997,"15min":1.141},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":0.95099999999999996,"5min":1.0429999999999999,"15min":1.254},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.57399999999999995,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.90200000000000002,"5min":1.0189999999999999,"15min":1.337},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.248},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:22.139 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:26:22.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:26:22.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.362+0000 7fb6bb59d700 1 Processor -- start 2023-10-06T05:26:22.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.363+0000 7fb6bb59d700 1 -- start start 2023-10-06T05:26:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6bb59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6b41594c0 0x7fb6b406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6bb59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb6b4159890 0x7fb6b40cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6bb59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b415dbe0 con 0x7fb6b40c95c0 2023-10-06T05:26:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6bb59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b40c9e30 con 0x7fb6b41594c0 2023-10-06T05:26:22.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6bb59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b40ca110 con 0x7fb6b4159890 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6ba59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6b41594c0 0x7fb6b406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6ba59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6b41594c0 0x7fb6b406a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48392/0 (socket says 172.21.15.33:48392) 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3780296023 learned_addr learned my addr 172.21.15.33:0/3780296023 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3780296023 >> v2:172.21.15.167:3300/0 conn(0x7fb6b4159890 msgr2=0x7fb6b40cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.364+0000 7fb6bad9c700 1 --2- 172.21.15.33:0/3780296023 >> v2:172.21.15.167:3300/0 conn(0x7fb6b4159890 0x7fb6b40cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6ba59b700 1 --2- 172.21.15.33:0/3780296023 >> v2:172.21.15.167:3300/0 conn(0x7fb6b4159890 0x7fb6b40cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3780296023 >> v1:172.21.15.167:6789/0 conn(0x7fb6b40c95c0 legacy=0x7fb6b406a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3780296023 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b415de30 con 0x7fb6b41594c0 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6bad9c700 1 --2- 172.21.15.33:0/3780296023 >> v2:172.21.15.167:3300/0 conn(0x7fb6b4159890 0x7fb6b40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:22.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6ba59b700 1 --2- 172.21.15.33:0/3780296023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b41594c0 0x7fb6b406a310 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4069dc0 tx=0x7fb6b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8292794cb2d52f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6b9599700 1 -- 172.21.15.33:0/3780296023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000b820 con 0x7fb6b41594c0 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6b9599700 1 -- 172.21.15.33:0/3780296023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000b980 con 0x7fb6b41594c0 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.365+0000 7fb6b9599700 1 -- 172.21.15.33:0/3780296023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000bd20 con 0x7fb6b41594c0 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3780296023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b41594c0 msgr2=0x7fb6b406a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3780296023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b41594c0 0x7fb6b406a310 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4069dc0 tx=0x7fb6b0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3780296023 shutdown_connections 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3780296023 >> v2:172.21.15.167:3300/0 conn(0x7fb6b4159890 0x7fb6b40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3780296023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b41594c0 0x7fb6b406a310 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3780296023 >> 172.21.15.33:0/3780296023 conn(0x7fb6b40cd3b0 msgr2=0x7fb6b40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3780296023 shutdown_connections 2023-10-06T05:26:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.366+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3780296023 wait complete. 2023-10-06T05:26:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.367+0000 7fb6bb59d700 1 Processor -- start 2023-10-06T05:26:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.367+0000 7fb6bb59d700 1 -- start start 2023-10-06T05:26:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.368+0000 7fb6bb59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 0x7fb6b40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.368+0000 7fb6bb59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6b41594c0 0x7fb6b416d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.368+0000 7fb6bb59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6b4159890 0x7fb6b416df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.368+0000 7fb6bb59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40c75a0 con 0x7fb6b40c95c0 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.368+0000 7fb6bb59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40c7880 con 0x7fb6b4159890 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.368+0000 7fb6bb59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40c7b60 con 0x7fb6b41594c0 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 0x7fb6b40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 0x7fb6b40c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48402/0 (socket says 172.21.15.33:48402) 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3827699674 learned_addr learned my addr 172.21.15.33:0/3827699674 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6bad9c700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6b4159890 0x7fb6b416df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6b9d9a700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6b41594c0 0x7fb6b416d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6b41594c0 msgr2=0x7fb6b416d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6b41594c0 0x7fb6b416d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6b4159890 msgr2=0x7fb6b416df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6b4159890 0x7fb6b416df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 -- 172.21.15.33:0/3827699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b00094f0 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6bad9c700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6b4159890 0x7fb6b416df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.369+0000 7fb6ba59b700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 0x7fb6b40c6b00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009290 tx=0x7fb6b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.370+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0011e20 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.370+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b001b680 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.370+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b001ba40 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.370+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b40c7e40 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.370+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b415a490 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.371+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b404c380 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.372+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6b001c450 con 0x7fb6b40c95c0 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.372+0000 7fb6aa7fc700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb698066f30 0x7fb6980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.373+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb6b0086490 con 0x7fb6b40c95c0 2023-10-06T05:26:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.373+0000 7fb6b9d9a700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb698066f30 0x7fb6980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.374+0000 7fb6b9d9a700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb698066f30 0x7fb6980693e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40a6060 tx=0x7fb6a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.377+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6b0053500 con 0x7fb6b40c95c0 2023-10-06T05:26:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.658+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb6b40c8e20 con 0x7fb6b40c95c0 2023-10-06T05:26:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.663+0000 7fb6aa7fc700 1 -- 172.21.15.33:0/3827699674 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb6b0056b20 con 0x7fb6b40c95c0 2023-10-06T05:26:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb698066f30 msgr2=0x7fb6980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb698066f30 0x7fb6980693e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40a6060 tx=0x7fb6a4009040 comp rx=0 tx=0).stop 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 msgr2=0x7fb6b40c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 0x7fb6b40c6b00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009290 tx=0x7fb6b0016000 comp rx=0 tx=0).stop 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 shutdown_connections 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb698066f30 0x7fb6980693e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6b4159890 0x7fb6b416df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b40c95c0 0x7fb6b40c6b00 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 --2- 172.21.15.33:0/3827699674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6b41594c0 0x7fb6b416d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.665+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 >> 172.21.15.33:0/3827699674 conn(0x7fb6b40cd3b0 msgr2=0x7fb6b4157f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.666+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 shutdown_connections 2023-10-06T05:26:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.666+0000 7fb6bb59d700 1 -- 172.21.15.33:0/3827699674 wait complete. 2023-10-06T05:26:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:26:22.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:22.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.870+0000 7fbdc89fe700 1 Processor -- start 2023-10-06T05:26:22.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.870+0000 7fbdc89fe700 1 -- start start 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc89fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdc40cad00 0x7fbdc40c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc89fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc406d670 con 0x7fbdc4159440 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc89fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc406ca60 con 0x7fbdc406db00 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc89fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc406cd40 con 0x7fbdc40cad00 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc2d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbdc4159440 0x7fbdc40ca5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48032/0 (socket says 172.21.15.33:48032) 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc2d9d700 1 -- 172.21.15.33:0/4141959941 learned_addr learned my addr 172.21.15.33:0/4141959941 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:22.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.871+0000 7fbdc359e700 1 --2- 172.21.15.33:0/4141959941 >> v2:172.21.15.33:3300/0 conn(0x7fbdc40cad00 0x7fbdc40c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594696540 0 0) 0x7fbdc406d670 con 0x7fbdc4159440 2023-10-06T05:26:22.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdac003570 con 0x7fbdc4159440 2023-10-06T05:26:22.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464958616 0 0) 0x7fbdc406ca60 con 0x7fbdc406db00 2023-10-06T05:26:22.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc406d670 con 0x7fbdc406db00 2023-10-06T05:26:22.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288950275 0 0) 0x7fbdac003570 con 0x7fbdc4159440 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc359e700 1 -- 172.21.15.33:0/4141959941 >> v1:172.21.15.167:6789/0 conn(0x7fbdc406db00 legacy=0x7fbdc40c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc359e700 1 -- 172.21.15.33:0/4141959941 >> v1:172.21.15.33:6789/0 conn(0x7fbdc4159440 legacy=0x7fbdc40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc359e700 1 -- 172.21.15.33:0/4141959941 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc406d020 con 0x7fbdc40cad00 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.872+0000 7fbdc359e700 1 --2- 172.21.15.33:0/4141959941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc40cad00 0x7fbdc40c7560 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009e60 tx=0x7fbdbc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddff7e13e96221fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc00e070 con 0x7fbdc40cad00 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc00b7a0 con 0x7fbdc40cad00 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc1d9b700 1 -- 172.21.15.33:0/4141959941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4001470 con 0x7fbdc40cad00 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 -- 172.21.15.33:0/4141959941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc40cad00 msgr2=0x7fbdc40c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/4141959941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc40cad00 0x7fbdc40c7560 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009e60 tx=0x7fbdbc009780 comp rx=0 tx=0).stop 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 -- 172.21.15.33:0/4141959941 shutdown_connections 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/4141959941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc40cad00 0x7fbdc40c7560 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 -- 172.21.15.33:0/4141959941 >> 172.21.15.33:0/4141959941 conn(0x7fbdc40cc8b0 msgr2=0x7fbdc40cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 -- 172.21.15.33:0/4141959941 shutdown_connections 2023-10-06T05:26:22.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.873+0000 7fbdc89fe700 1 -- 172.21.15.33:0/4141959941 wait complete. 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.875+0000 7fbdc89fe700 1 Processor -- start 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.875+0000 7fbdc89fe700 1 -- start start 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc89fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 0x7fbdc4154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc89fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc40cad00 0x7fbdc413c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc4159440 0x7fbdc4155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc89fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41542d0 con 0x7fbdc406db00 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc89fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41536c0 con 0x7fbdc40cad00 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41539a0 con 0x7fbdc4159440 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 0x7fbdc4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 0x7fbdc4154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48424/0 (socket says 172.21.15.33:48424) 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 -- 172.21.15.33:0/3636512816 learned_addr learned my addr 172.21.15.33:0/3636512816 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc259c700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc40cad00 0x7fbdc413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc359e700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc4159440 0x7fbdc4155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 -- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc4159440 msgr2=0x7fbdc4155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc4159440 0x7fbdc4155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 -- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc40cad00 msgr2=0x7fbdc413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc40cad00 0x7fbdc413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc2d9d700 1 -- 172.21.15.33:0/3636512816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc009a20 con 0x7fbdc406db00 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.876+0000 7fbdc359e700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc4159440 0x7fbdc4155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.877+0000 7fbdc2d9d700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 0x7fbdc4154760 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c7f0 tx=0x7fbdb4001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.877+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4007360 con 0x7fbdc406db00 2023-10-06T05:26:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.877+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb40074e0 con 0x7fbdc406db00 2023-10-06T05:26:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.877+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc4153c80 con 0x7fbdc406db00 2023-10-06T05:26:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.877+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4004830 con 0x7fbdc406db00 2023-10-06T05:26:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.877+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc4154ca0 con 0x7fbdc406db00 2023-10-06T05:26:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.878+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc4154120 con 0x7fbdc406db00 2023-10-06T05:26:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.879+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdb4004c00 con 0x7fbdc406db00 2023-10-06T05:26:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.880+0000 7fbdaaffd700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdb0066f30 0x7fbdb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.880+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbdb4083dd0 con 0x7fbdc406db00 2023-10-06T05:26:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.880+0000 7fbdc259c700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdb0066f30 0x7fbdb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.881+0000 7fbdc259c700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdb0066f30 0x7fbdb00693e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4154f90 tx=0x7fbdb8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:22.884+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdb4050f60 con 0x7fbdc406db00 2023-10-06T05:26:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.131+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdc41545b0 con 0x7fbdc406db00 2023-10-06T05:26:23.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.131+0000 7fbdaaffd700 1 -- 172.21.15.33:0/3636512816 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fbdb4018020 con 0x7fbdc406db00 2023-10-06T05:26:23.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:23.134 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdb0066f30 msgr2=0x7fbdb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdb0066f30 0x7fbdb00693e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4154f90 tx=0x7fbdb8007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 msgr2=0x7fbdc4154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 0x7fbdc4154760 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c7f0 tx=0x7fbdb4001a10 comp rx=0 tx=0).stop 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 shutdown_connections 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbdb0066f30 0x7fbdb00693e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdc4159440 0x7fbdc4155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc406db00 0x7fbdc4154760 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 --2- 172.21.15.33:0/3636512816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdc40cad00 0x7fbdc413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 >> 172.21.15.33:0/3636512816 conn(0x7fbdc40cc8b0 msgr2=0x7fbdc41521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 shutdown_connections 2023-10-06T05:26:23.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.134+0000 7fbdc89fe700 1 -- 172.21.15.33:0/3636512816 wait complete. 2023-10-06T05:26:23.138 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:23.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:26:16.467314+0000', 'modified': '2023-10-06T05:26:20.007449+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9066}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9066': {'gid': 9066, 'name': 'a', 'rank': 0, 'incarnation': 120, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2543247533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 2543247533}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2543247533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:26:23.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:26:23.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:26:23.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.744+0000 7ffb8159d700 1 Processor -- start 2023-10-06T05:26:23.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.744+0000 7ffb8159d700 1 -- start start 2023-10-06T05:26:23.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb8159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156880 0x7ffb7c0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb8159d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb7c159e10 0x7ffb7c0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb8159d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffb7c156c50 0x7ffb7c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb8159d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cc290 con 0x7ffb7c159e10 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb8159d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c06a820 con 0x7ffb7c156c50 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb8159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c15a2f0 con 0x7ffb7c156880 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb7c159e10 0x7ffb7c0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb7c159e10 0x7ffb7c0c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48432/0 (socket says 172.21.15.33:48432) 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3450656819 learned_addr learned my addr 172.21.15.33:0/3450656819 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3450656819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156880 0x7ffb7c0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/3450656819 >> v2:172.21.15.167:3300/0 conn(0x7ffb7c156c50 0x7ffb7c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3450656819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156880 msgr2=0x7ffb7c0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3450656819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156880 0x7ffb7c0c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3450656819 >> v2:172.21.15.167:3300/0 conn(0x7ffb7c156c50 msgr2=0x7ffb7c1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3450656819 >> v2:172.21.15.167:3300/0 conn(0x7ffb7c156c50 0x7ffb7c1570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.746+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3450656819 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c15a5d0 con 0x7ffb7c159e10 2023-10-06T05:26:23.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.747+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3450656819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c0c7ec0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0c8650 tx=0x7ffb70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44af78b5a57f1204 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.747+0000 7ffb7affd700 1 -- 172.21.15.33:0/3450656819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000b860 con 0x7ffb7c159e10 2023-10-06T05:26:23.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.747+0000 7ffb7affd700 1 -- 172.21.15.33:0/3450656819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7000b9c0 con 0x7ffb7c159e10 2023-10-06T05:26:23.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.747+0000 7ffb7affd700 1 -- 172.21.15.33:0/3450656819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000bd60 con 0x7ffb7c159e10 2023-10-06T05:26:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.747+0000 7ffb8159d700 1 -- 172.21.15.33:0/3450656819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 msgr2=0x7ffb7c0c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.747+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3450656819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c0c7ec0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0c8650 tx=0x7ffb70009250 comp rx=0 tx=0).stop 2023-10-06T05:26:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 -- 172.21.15.33:0/3450656819 shutdown_connections 2023-10-06T05:26:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3450656819 >> v2:172.21.15.167:3300/0 conn(0x7ffb7c156c50 0x7ffb7c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3450656819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156880 0x7ffb7c0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3450656819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c0c7ec0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 -- 172.21.15.33:0/3450656819 >> 172.21.15.33:0/3450656819 conn(0x7ffb7c06c560 msgr2=0x7ffb7c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 -- 172.21.15.33:0/3450656819 shutdown_connections 2023-10-06T05:26:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.748+0000 7ffb8159d700 1 -- 172.21.15.33:0/3450656819 wait complete. 2023-10-06T05:26:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.749+0000 7ffb8159d700 1 Processor -- start 2023-10-06T05:26:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.749+0000 7ffb8159d700 1 -- start start 2023-10-06T05:26:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb8159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb7c156880 0x7ffb7c0c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb8159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156c50 0x7ffb7c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb8159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb8159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c16ae80 con 0x7ffb7c159e10 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb8159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0c4a80 con 0x7ffb7c156880 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb8159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0c4d60 con 0x7ffb7c156c50 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb80d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb80d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c1525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48434/0 (socket says 172.21.15.33:48434) 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.750+0000 7ffb80d9c700 1 -- 172.21.15.33:0/3087479309 learned_addr learned my addr 172.21.15.33:0/3087479309 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb80d9c700 1 -- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156c50 msgr2=0x7ffb7c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156c50 0x7ffb7c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb80d9c700 1 -- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb7c156880 msgr2=0x7ffb7c0c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb7c156880 0x7ffb7c0c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb80d9c700 1 -- 172.21.15.33:0/3087479309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700094f0 con 0x7ffb7c159e10 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c1525f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009fa0 tx=0x7ffb74009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb63fff700 1 -- 172.21.15.33:0/3087479309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400ed20 con 0x7ffb7c159e10 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb63fff700 1 -- 172.21.15.33:0/3087479309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7400ee80 con 0x7ffb7c159e10 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c0c5040 con 0x7ffb7c159e10 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb63fff700 1 -- 172.21.15.33:0/3087479309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74014670 con 0x7ffb7c159e10 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.751+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c0b1a10 con 0x7ffb7c159e10 2023-10-06T05:26:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.754+0000 7ffb63fff700 1 -- 172.21.15.33:0/3087479309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb740149e0 con 0x7ffb7c159e10 2023-10-06T05:26:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.755+0000 7ffb63fff700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb68066f30 0x7ffb680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.755+0000 7ffb63fff700 1 -- 172.21.15.33:0/3087479309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb74084f30 con 0x7ffb7c159e10 2023-10-06T05:26:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.756+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb68066f30 0x7ffb680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.756+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb68066f30 0x7ffb680693e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009a60 tx=0x7ffb6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.757+0000 7ffb90f6cb80 1 Processor -- start 2023-10-06T05:26:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.757+0000 7ffb90f6cb80 1 -- start start 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deadecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deafc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601dea35c80 con 0x5601deab6dd0 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601dea0ca70 con 0x5601deadf210 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601dea0a270 con 0x5601deab5950 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deadecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deadecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48436/0 (socket says 172.21.15.33:48436) 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3943016793 learned_addr learned my addr 172.21.15.33:0/3943016793 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deafc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 msgr2=0x5601deab5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.758+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 msgr2=0x5601deafc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deafc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3943016793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601de8a0270 con 0x5601deab6dd0 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deadecd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b3c0 tx=0x7ffb70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b39a261bcc22871 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb617fa700 1 -- 172.21.15.33:0/3943016793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70000c90 con 0x5601deab6dd0 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb617fa700 1 -- 172.21.15.33:0/3943016793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7000bb30 con 0x5601deab6dd0 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.759+0000 7ffb617fa700 1 -- 172.21.15.33:0/3943016793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70011630 con 0x5601deab6dd0 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3943016793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 msgr2=0x5601deadecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deadecd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b3c0 tx=0x7ffb70017000 comp rx=0 tx=0).stop 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3943016793 shutdown_connections 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deafc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3943016793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deadecd0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3943016793 >> 172.21.15.33:0/3943016793 conn(0x5601deab7270 msgr2=0x5601deab7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3943016793 shutdown_connections 2023-10-06T05:26:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.760+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3943016793 wait complete. 2023-10-06T05:26:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.761+0000 7ffb90f6cb80 1 -- start start 2023-10-06T05:26:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.761+0000 7ffb90f6cb80 1 Processor -- start 2023-10-06T05:26:23.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deadfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb09fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deadf210 0x5601deab2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601dea06060 con 0x5601deab6dd0 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de9b1890 con 0x5601deab5950 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601de9ccea0 con 0x5601deadf210 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb90f6cb80 10 client.0 _ll_register_callbacks cb 0x5601de8a2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb09fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb09fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48448/0 (socket says 172.21.15.33:48448) 2023-10-06T05:26:23.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/2636954815 learned_addr learned my addr 172.21.15.33:0/2636954815 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deadfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deadf210 0x5601deab2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.762+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deadf210 msgr2=0x5601deab2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deadf210 0x5601deab2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 msgr2=0x5601deadfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deadfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/2636954815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c15a5d0 con 0x5601deab6dd0 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deadf210 0x5601deab2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb09fa0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009290 tx=0x7ffb7000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1d3b8681ea6b69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb5a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:26:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/2636954815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000bca0 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/2636954815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb7000be00 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601deaaa690 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/2636954815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb70011c30 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.763+0000 7ffb90f6cb80 10 client.9210 fetch_fsmap learned FSMap version 124 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x5601de842fe0 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/2636954815 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601deaaa690 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/2636954815 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 124) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ffb7002c070 con 0x5601deab6dd0 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 10 client.9210 fetch_fsmap finished waiting for FSMap version 124 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 10 client.9210 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 20 client.9210 populate_metadata read hostname 'smithi033' 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601dead9a30 0x5601dead9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 4 client.9210 mds_command: new command op to 9066 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 --> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601dea875c0 con 0x5601dead9a30 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.764+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601dead9a30 0x5601dead9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.765+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601dead9a30 0x5601dead9e00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9540340aff3310d1 server_cookie=8a524318141f065d in_seq=0 out_seq=0 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.765+0000 7ffb5a7fc700 10 client.9210 ms_handle_connect on v2:172.21.15.33:6834/2543247533 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.765+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/2636954815 <== mds.0 v2:172.21.15.33:6834/2543247533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5601dea875c0 con 0x5601dead9a30 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.765+0000 7ffb5a7fc700 10 client.9210 handle_command_reply: tid=0 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 client.9210 shutdown 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 msgr2=0x5601deb09fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb09fa0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009290 tx=0x7ffb7000b050 comp rx=0 tx=0).stop 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 shutdown_connections 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deadf210 0x5601deab2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deadfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601dead9a30 0x5601dead9e00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/2636954815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb09fa0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 >> 172.21.15.33:0/2636954815 conn(0x5601deab7270 msgr2=0x5601deaac790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb5a7fc700 0 client.9210 ms_handle_reset on v2:172.21.15.33:6834/2543247533 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 shutdown_connections 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/2636954815 wait complete. 2023-10-06T05:26:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.766+0000 7ffb90f6cb80 20 client.9210 trim_cache size 0 max 16384 2023-10-06T05:26:23.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.784+0000 7ffb90f6cb80 1 Processor -- start 2023-10-06T05:26:23.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.784+0000 7ffb90f6cb80 1 -- start start 2023-10-06T05:26:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab6dd0 0x5601deab6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 0x5601deadce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601deb09970 con 0x5601deadf210 2023-10-06T05:26:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de888e30 con 0x5601deab5950 2023-10-06T05:26:23.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601deaf6ba0 con 0x5601deab6dd0 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 0x5601deadce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 0x5601deadce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48454/0 (socket says 172.21.15.33:48454) 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 -- 172.21.15.33:0/98281784 learned_addr learned my addr 172.21.15.33:0/98281784 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab6dd0 0x5601deab6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 -- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab6dd0 msgr2=0x5601deab6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab6dd0 0x5601deab6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 -- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 msgr2=0x5601deab5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.785+0000 7ffb80d9c700 1 -- 172.21.15.33:0/98281784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601de9d4780 con 0x5601deadf210 2023-10-06T05:26:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 0x5601deadce20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ffb74007fb0 tx=0x7ffb74016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02757f50d648db3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb61ffb700 1 -- 172.21.15.33:0/98281784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74055c60 con 0x5601deadf210 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb61ffb700 1 -- 172.21.15.33:0/98281784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb74058eb0 con 0x5601deadf210 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb61ffb700 1 -- 172.21.15.33:0/98281784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7405cc00 con 0x5601deadf210 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/98281784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 msgr2=0x5601deadce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 0x5601deadce20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ffb74007fb0 tx=0x7ffb74016040 comp rx=0 tx=0).stop 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/98281784 shutdown_connections 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deadf210 0x5601deadce20 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deab5950 0x5601deab5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/98281784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab6dd0 0x5601deab6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.786+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/98281784 >> 172.21.15.33:0/98281784 conn(0x5601deab7270 msgr2=0x5601deab8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.787+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/98281784 shutdown_connections 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.787+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/98281784 wait complete. 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.787+0000 7ffb90f6cb80 1 -- start start 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.787+0000 7ffb90f6cb80 1 Processor -- start 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deb60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deae4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601deb13ba0 con 0x5601deab6dd0 2023-10-06T05:26:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601deae5910 con 0x5601deadf210 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601deb1b880 con 0x5601deab5950 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.788+0000 7ffb90f6cb80 10 client.0 _ll_register_callbacks cb 0x5601de8a2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb66fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48462/0 (socket says 172.21.15.33:48462) 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3192767345 learned_addr learned my addr 172.21.15.33:0/3192767345 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deb60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb80d9c700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deae4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 msgr2=0x5601deb60750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deb60750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 msgr2=0x5601deae4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deae4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 -- 172.21.15.33:0/3192767345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c0b1a10 con 0x5601deab6dd0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deb60750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb7b7fe700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb66fe0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ffb7002e9f0 tx=0x7ffb70028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f327c10fa164a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb5a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/3192767345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000bca0 con 0x5601deab6dd0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/3192767345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb7000be00 con 0x5601deab6dd0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.789+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb7000bb30 con 0x5601deab6dd0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/3192767345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb7002e660 con 0x5601deab6dd0 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 10 client.9216 fetch_fsmap learned FSMap version 124 2023-10-06T05:26:23.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x5601de9bb470 con 0x5601deab6dd0 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/3192767345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb70043020 con 0x5601deab6dd0 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/3192767345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 124) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ffb70009a60 con 0x5601deab6dd0 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 10 client.9216 fetch_fsmap finished waiting for FSMap version 124 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 10 client.9216 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 20 client.9216 populate_metadata read hostname 'smithi033' 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601deab5d20 0x5601deab60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 4 client.9216 mds_command: new command op to 9066 tid=0[{"prefix": "status"}] 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 --> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601de9a8e00 con 0x5601deab5d20 2023-10-06T05:26:23.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.790+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601deab5d20 0x5601deab60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb7bfff700 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601deab5d20 0x5601deab60f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75820dd104abb231 server_cookie=51ad3ce9c240b405 in_seq=0 out_seq=0 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb5a7fc700 10 client.9216 ms_handle_connect on v2:172.21.15.33:6834/2543247533 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb5a7fc700 1 -- 172.21.15.33:0/3192767345 <== mds.0 v2:172.21.15.33:6834/2543247533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5601de9a8e00 con 0x5601deab5d20 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb5a7fc700 10 client.9216 handle_command_reply: tid=0 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 client.9216 shutdown 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 msgr2=0x5601deb66fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb66fe0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ffb7002e9f0 tx=0x7ffb70028000 comp rx=0 tx=0).stop 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 shutdown_connections 2023-10-06T05:26:23.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601deadf210 0x5601deae4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601deab5950 0x5601deb60750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:6834/2543247533,v1:172.21.15.33:6835/2543247533] conn(0x5601deab5d20 0x5601deab60f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 --2- 172.21.15.33:0/3192767345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601deab6dd0 0x5601deb66fe0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 >> 172.21.15.33:0/3192767345 conn(0x5601deab7270 msgr2=0x5601deaf50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb5a7fc700 0 client.9216 ms_handle_reset on v2:172.21.15.33:6834/2543247533 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 shutdown_connections 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.791+0000 7ffb90f6cb80 1 -- 172.21.15.33:0/3192767345 wait complete. 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb90f6cb80 20 client.9216 trim_cache size 0 max 16384 2023-10-06T05:26:23.800 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 9066, 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.8130254380000004, 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 124, 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 169, 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 169, 2023-10-06T05:26:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 13.6190651 2023-10-06T05:26:23.802 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:26:23.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb68066f30 msgr2=0x7ffb680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb68066f30 0x7ffb680693e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009a60 tx=0x7ffb6c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:23.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 msgr2=0x7ffb7c1525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c1525f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009fa0 tx=0x7ffb74009900 comp rx=0 tx=0).stop 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 shutdown_connections 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb68066f30 0x7ffb680693e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb7c159e10 0x7ffb7c1525f0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb7c156880 0x7ffb7c0c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 --2- 172.21.15.33:0/3087479309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb7c156c50 0x7ffb7c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 >> 172.21.15.33:0/3087479309 conn(0x7ffb7c06c560 msgr2=0x7ffb7c1592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 shutdown_connections 2023-10-06T05:26:23.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.792+0000 7ffb8159d700 1 -- 172.21.15.33:0/3087479309 wait complete. 2023-10-06T05:26:23.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.998+0000 7f41ad19d700 1 Processor -- start 2023-10-06T05:26:24.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.998+0000 7f41ad19d700 1 -- start start 2023-10-06T05:26:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.999+0000 7f41ad19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155f60 0x7f41a80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.999+0000 7f41ad19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a806ab60 con 0x7f41a8155af0 2023-10-06T05:26:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.999+0000 7f41ad19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a806ae40 con 0x7f41a80c8b50 2023-10-06T05:26:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:23.999+0000 7f41ad19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a806b7e0 con 0x7f41a8155f60 2023-10-06T05:26:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41a77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f41a8155af0 0x7f41a806a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48036/0 (socket says 172.21.15.33:48036) 2023-10-06T05:26:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41a77fe700 1 -- 172.21.15.33:0/1418719297 learned_addr learned my addr 172.21.15.33:0/1418719297 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41ac99c700 1 --2- 172.21.15.33:0/1418719297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155f60 0x7f41a80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41a6ffd700 1 -- 172.21.15.33:0/1418719297 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232478713 0 0) 0x7f41a806ab60 con 0x7f41a8155af0 2023-10-06T05:26:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41a6ffd700 1 -- 172.21.15.33:0/1418719297 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4190003570 con 0x7f41a8155af0 2023-10-06T05:26:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41ac99c700 1 -- 172.21.15.33:0/1418719297 >> v1:172.21.15.167:6789/0 conn(0x7f41a80c8b50 legacy=0x7f41a806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41a6ffd700 1 -- 172.21.15.33:0/1418719297 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907507169 0 0) 0x7f41a806ae40 con 0x7f41a80c8b50 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41ac99c700 1 -- 172.21.15.33:0/1418719297 >> v1:172.21.15.33:6789/0 conn(0x7f41a8155af0 legacy=0x7f41a806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.000+0000 7f41ac99c700 1 -- 172.21.15.33:0/1418719297 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a806b120 con 0x7f41a8155f60 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.001+0000 7f41ac99c700 1 --2- 172.21.15.33:0/1418719297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155f60 0x7f41a80c7210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009e90 tx=0x7f41a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930e47d650e5e9e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.001+0000 7f41a6ffd700 1 -- 172.21.15.33:0/1418719297 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000e070 con 0x7f41a8155f60 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.001+0000 7f41a6ffd700 1 -- 172.21.15.33:0/1418719297 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a000b920 con 0x7f41a8155f60 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.001+0000 7f41a6ffd700 1 -- 172.21.15.33:0/1418719297 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c001470 con 0x7f41a8155f60 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.001+0000 7f41ad19d700 1 -- 172.21.15.33:0/1418719297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155f60 msgr2=0x7f41a80c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.001+0000 7f41ad19d700 1 --2- 172.21.15.33:0/1418719297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155f60 0x7f41a80c7210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009e90 tx=0x7f41a0009780 comp rx=0 tx=0).stop 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.002+0000 7f41ad19d700 1 -- 172.21.15.33:0/1418719297 shutdown_connections 2023-10-06T05:26:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.002+0000 7f41ad19d700 1 --2- 172.21.15.33:0/1418719297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155f60 0x7f41a80c7210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.002+0000 7f41ad19d700 1 -- 172.21.15.33:0/1418719297 >> 172.21.15.33:0/1418719297 conn(0x7f41a806d030 msgr2=0x7f41a806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:24.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.002+0000 7f41ad19d700 1 -- 172.21.15.33:0/1418719297 shutdown_connections 2023-10-06T05:26:24.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.002+0000 7f41ad19d700 1 -- 172.21.15.33:0/1418719297 wait complete. 2023-10-06T05:26:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.003+0000 7f41ad19d700 1 Processor -- start 2023-10-06T05:26:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.003+0000 7f41ad19d700 1 -- start start 2023-10-06T05:26:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41ad19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 0x7f41a816dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41ad19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155af0 0x7f41a8169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41ad19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a8155f60 0x7f41a80c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41ad19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a816ac20 con 0x7f41a80c8b50 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41ad19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a816e1a0 con 0x7f41a8155f60 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41ad19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a816a120 con 0x7f41a8155af0 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 0x7f41a816dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 0x7f41a816dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48468/0 (socket says 172.21.15.33:48468) 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.004+0000 7f41a7fff700 1 -- 172.21.15.33:0/4252799539 learned_addr learned my addr 172.21.15.33:0/4252799539 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a7fff700 1 -- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155af0 msgr2=0x7f41a8169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a77fe700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155af0 0x7f41a8169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41ac99c700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a8155f60 0x7f41a80c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a7fff700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155af0 0x7f41a8169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a7fff700 1 -- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a8155f60 msgr2=0x7f41a80c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a7fff700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a8155f60 0x7f41a80c2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a7fff700 1 -- 172.21.15.33:0/4252799539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a0009a20 con 0x7f41a80c8b50 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a77fe700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155af0 0x7f41a8169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41a7fff700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 0x7f41a816dc60 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f419800c4e0 tx=0x7f419800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198011590 con 0x7f41a80c8b50 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4198011710 con 0x7f41a80c8b50 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a816a380 con 0x7f41a80c8b50 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.005+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198011ad0 con 0x7f41a80c8b50 2023-10-06T05:26:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.006+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a816a9c0 con 0x7f41a80c8b50 2023-10-06T05:26:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.007+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a816ade0 con 0x7f41a80c8b50 2023-10-06T05:26:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.007+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f419801b3b0 con 0x7f41a80c8b50 2023-10-06T05:26:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.008+0000 7f418ffff700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4194066f30 0x7f41940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.008+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4198083bc0 con 0x7f41a80c8b50 2023-10-06T05:26:24.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.009+0000 7f41a77fe700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4194066f30 0x7f41940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.009+0000 7f41a77fe700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4194066f30 0x7f41940693e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f419c0011e0 tx=0x7f419c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:24.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.013+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41a816ade0 con 0x7f41a80c8b50 2023-10-06T05:26:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.261+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f41a816ade0 con 0x7f41a80c8b50 2023-10-06T05:26:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.261+0000 7f418ffff700 1 -- 172.21.15.33:0/4252799539 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v124) v1 ==== 53+0+73 (secure 0 0 0) 0x7f41a816ade0 con 0x7f41a80c8b50 2023-10-06T05:26:24.264 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:24.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4194066f30 msgr2=0x7f41940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4194066f30 0x7f41940693e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f419c0011e0 tx=0x7f419c009040 comp rx=0 tx=0).stop 2023-10-06T05:26:24.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 msgr2=0x7f41a816dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 0x7f41a816dc60 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f419800c4e0 tx=0x7f419800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 shutdown_connections 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4194066f30 0x7f41940693e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a8155f60 0x7f41a80c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a80c8b50 0x7f41a816dc60 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 --2- 172.21.15.33:0/4252799539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a8155af0 0x7f41a8169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.263+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 >> 172.21.15.33:0/4252799539 conn(0x7f41a806d030 msgr2=0x7f41a80c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.264+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 shutdown_connections 2023-10-06T05:26:24.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:24.264+0000 7f41ad19d700 1 -- 172.21.15.33:0/4252799539 wait complete. 2023-10-06T05:26:24.283 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:24.284 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:24.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:24.285 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:26:24.321 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41573sec preferred_lft 41573sec 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:26:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:26:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:24.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:24.327 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:24.422 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:26:24.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:24.426 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:26:24.465 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:26:24.465 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:24.465 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:26:24.465 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:26:24.465 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:24.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:24.466 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:26:24.466 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:26:24.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:26:24.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:26:24.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:26:24.467 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:26:24.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:24.666 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:26:24.702 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:26:24.703 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:26:24.703 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:26:24.703 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:24.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:24.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:26:24.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:24.805 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:24.842 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:24.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:24.842 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:24.918 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:26:24.919 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:24.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:26:25.055 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:26:25.055 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:26:25.056 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:26:25.072 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:26:25.072 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:26:25.072 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:26:25.072 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:26:25.072 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:26:25.095 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:26:25.095 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:26:25.095 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:26:25.096 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:26:25.096 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:26:25.096 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:26:25.096 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:26:25.096 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:26:25.100 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:26:25.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:25.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:25.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:25.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.346+0000 7f49c699d700 1 Processor -- start 2023-10-06T05:26:25.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.346+0000 7f49c699d700 1 -- start start 2023-10-06T05:26:25.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.347+0000 7f49c699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00ccfc0 0x7f49c00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.347+0000 7f49c699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49c00cf3b0 0x7f49c01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.347+0000 7f49c699d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49c00cc750 0x7f49c01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.347+0000 7f49c699d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00cbd80 con 0x7f49c00cc750 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.347+0000 7f49c699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00af5f0 con 0x7f49c00cf3b0 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.347+0000 7f49c699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00af8d0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49c00cc750 0x7f49c01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49c00cc750 0x7f49c01599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48476/0 (socket says 172.21.15.33:48476) 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 -- 172.21.15.33:0/2341822129 learned_addr learned my addr 172.21.15.33:0/2341822129 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 -- 172.21.15.33:0/2341822129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00ccfc0 msgr2=0x7f49c00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 --2- 172.21.15.33:0/2341822129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00ccfc0 0x7f49c00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 -- 172.21.15.33:0/2341822129 >> v2:172.21.15.167:3300/0 conn(0x7f49c00cf3b0 msgr2=0x7f49c01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c519a700 1 --2- 172.21.15.33:0/2341822129 >> v2:172.21.15.167:3300/0 conn(0x7f49c00cf3b0 0x7f49c01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 --2- 172.21.15.33:0/2341822129 >> v2:172.21.15.167:3300/0 conn(0x7f49c00cf3b0 0x7f49c01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 -- 172.21.15.33:0/2341822129 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00afbb0 con 0x7f49c00cc750 2023-10-06T05:26:25.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c519a700 1 --2- 172.21.15.33:0/2341822129 >> v2:172.21.15.167:3300/0 conn(0x7f49c00cf3b0 0x7f49c01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:25.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.348+0000 7f49c619c700 1 --2- 172.21.15.33:0/2341822129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00cc750 0x7f49c01599e0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f49c0071260 tx=0x7f49b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eca26b63840b09e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c4999700 1 -- 172.21.15.33:0/2341822129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800e070 con 0x7f49c00cc750 2023-10-06T05:26:25.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c4999700 1 -- 172.21.15.33:0/2341822129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b800ba50 con 0x7f49c00cc750 2023-10-06T05:26:25.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c4999700 1 -- 172.21.15.33:0/2341822129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80092e0 con 0x7f49c00cc750 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 -- 172.21.15.33:0/2341822129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00cc750 msgr2=0x7f49c01599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 --2- 172.21.15.33:0/2341822129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00cc750 0x7f49c01599e0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f49c0071260 tx=0x7f49b8009840 comp rx=0 tx=0).stop 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 -- 172.21.15.33:0/2341822129 shutdown_connections 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 --2- 172.21.15.33:0/2341822129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00cc750 0x7f49c01599e0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 --2- 172.21.15.33:0/2341822129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00ccfc0 0x7f49c00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 --2- 172.21.15.33:0/2341822129 >> v2:172.21.15.167:3300/0 conn(0x7f49c00cf3b0 0x7f49c01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 -- 172.21.15.33:0/2341822129 >> 172.21.15.33:0/2341822129 conn(0x7f49c006cff0 msgr2=0x7f49c006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 -- 172.21.15.33:0/2341822129 shutdown_connections 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.349+0000 7f49c699d700 1 -- 172.21.15.33:0/2341822129 wait complete. 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.351+0000 7f49c699d700 1 Processor -- start 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.351+0000 7f49c699d700 1 -- start start 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00cc750 0x7f49c016cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 0x7f49c0169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c00b1760 0x7f49c00b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c9600 con 0x7f49c00ccfc0 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b2110 con 0x7f49c00b1760 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b37b0 con 0x7f49c00cc750 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 0x7f49c0169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c599b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00cc750 0x7f49c016cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 0x7f49c0169bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48490/0 (socket says 172.21.15.33:48490) 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c519a700 1 -- 172.21.15.33:0/2136413639 learned_addr learned my addr 172.21.15.33:0/2136413639 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.352+0000 7f49c619c700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c00b1760 0x7f49c00b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c519a700 1 -- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00cc750 msgr2=0x7f49c016cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c519a700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00cc750 0x7f49c016cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c519a700 1 -- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c00b1760 msgr2=0x7f49c00b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c519a700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c00b1760 0x7f49c00b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c519a700 1 -- 172.21.15.33:0/2136413639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009ae0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c519a700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 0x7f49c0169bd0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f49b000c8b0 tx=0x7f49b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0011600 con 0x7f49c00ccfc0 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0011760 con 0x7f49c00ccfc0 2023-10-06T05:26:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0011b20 con 0x7f49c00ccfc0 2023-10-06T05:26:25.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00c81c0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.353+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c00b3440 con 0x7f49c00ccfc0 2023-10-06T05:26:25.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.354+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00c86e0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.355+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49b0022350 con 0x7f49c00ccfc0 2023-10-06T05:26:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.356+0000 7f49b5ffb700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49a4066f30 0x7f49a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.357+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f49b0084570 con 0x7f49c00ccfc0 2023-10-06T05:26:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.357+0000 7f49c599b700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49a4066f30 0x7f49a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.358+0000 7f49c599b700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49a4066f30 0x7f49a40693e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f49c00cf050 tx=0x7f49bc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.361+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49c00c86e0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.608+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f49c00c86e0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.609+0000 7f49b5ffb700 1 -- 172.21.15.33:0/2136413639 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v124) v1 ==== 53+0+73 (secure 0 0 0) 0x7f49c00c86e0 con 0x7f49c00ccfc0 2023-10-06T05:26:25.612 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49a4066f30 msgr2=0x7f49a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49a4066f30 0x7f49a40693e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f49c00cf050 tx=0x7f49bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 msgr2=0x7f49c0169bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 0x7f49c0169bd0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f49b000c8b0 tx=0x7f49b0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 shutdown_connections 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49a4066f30 0x7f49a40693e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c00b1760 0x7f49c00b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c00cc750 0x7f49c016cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 --2- 172.21.15.33:0/2136413639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c00ccfc0 0x7f49c0169bd0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.611+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 >> 172.21.15.33:0/2136413639 conn(0x7f49c006cff0 msgr2=0x7f49c00c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.612+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 shutdown_connections 2023-10-06T05:26:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:25.612+0000 7f49c699d700 1 -- 172.21.15.33:0/2136413639 wait complete. 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:26:25.632 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:26:25.633 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:25.633 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:26:25.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:25.633 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:26:25.674 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:25.675 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41575sec preferred_lft 41575sec 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:26:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:25.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:25.678 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:25.777 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:26:25.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:25.780 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:26:25.819 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:26:25.819 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:25.820 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:26:25.820 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:26:25.820 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:25.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:25.821 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:26:25.821 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:26:25.821 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:26:25.821 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:26:25.821 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:26:25.821 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:26:26.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:26.018 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:26:26.055 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:26:26.055 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:26:26.055 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:26:26.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:26.055 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:26:26.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:26.157 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:26.192 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:26:26.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:26.193 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:26.267 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:26:26.268 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:26.268 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:26:26.397 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:26:26.397 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:26:26.397 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:26:26.413 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:26:26.413 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:26:26.414 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:26:26.414 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:26:26.414 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:26:26.435 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:26:26.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:26:26.435 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:26:26.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:26:26.435 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:26:26.436 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:26:26.436 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:26:26.436 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:26:26.439 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:26:26.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:26.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:26.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:26:26.718 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:26:26.719 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:26:26.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:26:26.926 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:26:26.926 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:26:26.926 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:26:26.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:26.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:27.013 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:26:27.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.013 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:27.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:27.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.278+0000 7fe84699d700 1 Processor -- start 2023-10-06T05:26:27.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.278+0000 7fe84699d700 1 -- start start 2023-10-06T05:26:27.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe8400ccfc0 0x7fe8400cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84699d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8400cc750 0x7fe8400b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84699d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400cbd80 con 0x7fe8400cf3b0 2023-10-06T05:26:27.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84699d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400af5f0 con 0x7fe8400cc750 2023-10-06T05:26:27.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400af8d0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8400cc750 0x7fe8400b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84619c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8400cc750 0x7fe8400b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48504/0 (socket says 172.21.15.33:48504) 2023-10-06T05:26:27.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.279+0000 7fe84619c700 1 -- 172.21.15.33:0/1719489036 learned_addr learned my addr 172.21.15.33:0/1719489036 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:27.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe84599b700 1 --2- 172.21.15.33:0/1719489036 >> v2:172.21.15.167:3300/0 conn(0x7fe8400ccfc0 0x7fe8400cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe84619c700 1 -- 172.21.15.33:0/1719489036 >> v2:172.21.15.167:3300/0 conn(0x7fe8400ccfc0 msgr2=0x7fe8400cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe84619c700 1 --2- 172.21.15.33:0/1719489036 >> v2:172.21.15.167:3300/0 conn(0x7fe8400ccfc0 0x7fe8400cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe84619c700 1 -- 172.21.15.33:0/1719489036 >> v1:172.21.15.33:6789/0 conn(0x7fe8400cf3b0 legacy=0x7fe8401594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe84619c700 1 -- 172.21.15.33:0/1719489036 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400afbb0 con 0x7fe8400cc750 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe84619c700 1 --2- 172.21.15.33:0/1719489036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400cc750 0x7fe8400b0460 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe838009f20 tx=0x7fe838009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f4335fb8e47724 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe844999700 1 -- 172.21.15.33:0/1719489036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800e070 con 0x7fe8400cc750 2023-10-06T05:26:27.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe844999700 1 -- 172.21.15.33:0/1719489036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83800b990 con 0x7fe8400cc750 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.280+0000 7fe844999700 1 -- 172.21.15.33:0/1719489036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8380092e0 con 0x7fe8400cc750 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 -- 172.21.15.33:0/1719489036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400cc750 msgr2=0x7fe8400b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 --2- 172.21.15.33:0/1719489036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400cc750 0x7fe8400b0460 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe838009f20 tx=0x7fe838009840 comp rx=0 tx=0).stop 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 -- 172.21.15.33:0/1719489036 shutdown_connections 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 --2- 172.21.15.33:0/1719489036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400cc750 0x7fe8400b0460 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 --2- 172.21.15.33:0/1719489036 >> v2:172.21.15.167:3300/0 conn(0x7fe8400ccfc0 0x7fe8400cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 -- 172.21.15.33:0/1719489036 >> 172.21.15.33:0/1719489036 conn(0x7fe84006cff0 msgr2=0x7fe84006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 -- 172.21.15.33:0/1719489036 shutdown_connections 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.281+0000 7fe84699d700 1 -- 172.21.15.33:0/1719489036 wait complete. 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.282+0000 7fe84699d700 1 Processor -- start 2023-10-06T05:26:27.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.283+0000 7fe84699d700 1 -- start start 2023-10-06T05:26:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.283+0000 7fe84699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8400cc750 0x7fe84014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 0x7fe8400b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe84016aa90 0x7fe8400b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b3340 con 0x7fe8400ccfc0 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84014d2e0 con 0x7fe8400cc750 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b35e0 con 0x7fe84016aa90 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 0x7fe8400b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 0x7fe8400b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48506/0 (socket says 172.21.15.33:48506) 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 -- 172.21.15.33:0/1125228089 learned_addr learned my addr 172.21.15.33:0/1125228089 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84599b700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8400cc750 0x7fe84014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84619c700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe84016aa90 0x7fe8400b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 -- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe84016aa90 msgr2=0x7fe8400b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe84016aa90 0x7fe8400b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 -- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8400cc750 msgr2=0x7fe84014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8400cc750 0x7fe84014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.284+0000 7fe84519a700 1 -- 172.21.15.33:0/1125228089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838009ae0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.285+0000 7fe84519a700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 0x7fe8400b15a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c560 tx=0x7fe830007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.285+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830011590 con 0x7fe8400ccfc0 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.285+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830011710 con 0x7fe8400ccfc0 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.285+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830011ad0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.285+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8400b38c0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.285+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84016a7f0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.286+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8400b3de0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.287+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe83001a360 con 0x7fe8400ccfc0 2023-10-06T05:26:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.288+0000 7fe835ffb700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe824066f30 0x7fe8240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.288+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8300835b0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.288+0000 7fe84599b700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe824066f30 0x7fe8240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.289+0000 7fe84599b700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe824066f30 0x7fe8240693e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe8400cac70 tx=0x7fe83c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.292+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8400b3de0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.538+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8400b3de0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.539+0000 7fe835ffb700 1 -- 172.21.15.33:0/1125228089 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v124) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8400b3de0 con 0x7fe8400ccfc0 2023-10-06T05:26:27.541 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe824066f30 msgr2=0x7fe8240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe824066f30 0x7fe8240693e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe8400cac70 tx=0x7fe83c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 msgr2=0x7fe8400b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 0x7fe8400b15a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c560 tx=0x7fe830007f20 comp rx=0 tx=0).stop 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 shutdown_connections 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe824066f30 0x7fe8240693e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe84016aa90 0x7fe8400b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8400cc750 0x7fe84014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 --2- 172.21.15.33:0/1125228089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8400ccfc0 0x7fe8400b15a0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 >> 172.21.15.33:0/1125228089 conn(0x7fe84006cff0 msgr2=0x7fe8400cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 shutdown_connections 2023-10-06T05:26:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:27.541+0000 7fe84699d700 1 -- 172.21.15.33:0/1125228089 wait complete. 2023-10-06T05:26:27.560 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:26:27.560 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:27.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:26:27.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:26:27.561 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:27.561 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:26:27.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:26:27.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:27.561 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:27.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.561 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:26:27.598 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41570sec preferred_lft 41570sec 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:26:27.601 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:26:27.602 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.602 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:27.602 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:26:27.602 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:26:27.602 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:27.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.602 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:27.698 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:26:27.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.701 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:26:27.740 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:26:27.740 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:27.740 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:26:27.741 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:26:27.741 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:27.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.742 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:26:27.742 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:26:27.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:26:27.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:26:27.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:26:27.742 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:26:27.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.941 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:26:27.977 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:26:27.977 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:26:27.978 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:26:27.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:27.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:27.978 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:26:28.080 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:28.081 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:28.118 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:28.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:28.118 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:28.194 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:26:28.194 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:28.194 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,recover_session=clean 2023-10-06T05:26:28.327 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,recover_session=clean 2023-10-06T05:26:28.328 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,recover_session=clean". 2023-10-06T05:26:28.328 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:26:28.343 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,recover_session=clean 2023-10-06T05:26:28.343 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,recover_session=clean". 2023-10-06T05:26:28.343 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:26:28.344 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:26:28.344 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,recover_session=clean 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,recover_session=clean". 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:26:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,recover_session=clean,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:26:28.369 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:26:28.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> python3 -c ' 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> import sys 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> import fcntl 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> import errno 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.432 DEBUG:teuthology.orchestra.run.smithi033:> fd1 = os.open("/home/ubuntu/cephtest/mnt.0/testfile_reconnect_after_blocklisted.1", os.O_RDWR | os.O_CREAT, 0O666) 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> fd2 = os.open("/home/ubuntu/cephtest/mnt.0/testfile_reconnect_after_blocklisted.1", os.O_RDONLY) 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> fd3 = os.open("/home/ubuntu/cephtest/mnt.0/testfile_reconnect_after_blocklisted.2", os.O_RDWR | os.O_CREAT, 0O666) 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> fd4 = os.open("/home/ubuntu/cephtest/mnt.0/testfile_reconnect_after_blocklisted.2", os.O_RDONLY) 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> os.write(fd1, b'"'"'content'"'"') 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> os.read(fd2, 1); 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> os.write(fd3, b'"'"'content'"'"') 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> os.read(fd4, 1); 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> fcntl.flock(fd4, fcntl.LOCK_SH | fcntl.LOCK_NB) 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> print("blocklist") 2023-10-06T05:26:28.433 DEBUG:teuthology.orchestra.run.smithi033:> sys.stdout.flush() 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> sys.stdin.readline() 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> # wait for mds to close session 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(10); 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> # trigger '"'"'open session'"'"' message. kclient relies on '"'"'session reject'"'"' message 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> # to detect if itself is blocklisted 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> try: 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> os.stat("/home/ubuntu/cephtest/mnt.0/testfile_reconnect_after_blocklisted.1") 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> except: 2023-10-06T05:26:28.434 DEBUG:teuthology.orchestra.run.smithi033:> pass 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> # wait for auto reconnect 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(10); 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> try: 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> os.write(fd1, b'"'"'content'"'"') 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> except OSError as e: 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> if e.errno != errno.EBADF: 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> raise 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> else: 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("write() failed to raise error") 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> os.read(fd2, 1); 2023-10-06T05:26:28.435 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> try: 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> os.read(fd4, 1) 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> except OSError as e: 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> if e.errno != errno.EIO: 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> raise 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> else: 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("read() failed to raise error") 2023-10-06T05:26:28.436 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:26:28.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:28.504 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi033.stdout:client7307 2023-10-06T05:26:28.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:28.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.765+0000 7fcd4719d700 1 Processor -- start 2023-10-06T05:26:28.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.765+0000 7fcd4719d700 1 -- start start 2023-10-06T05:26:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd40159520 0x7fcd400c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcd400c6170 0x7fcd400c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4006c820 con 0x7fcd400c80d0 2023-10-06T05:26:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400c6610 con 0x7fcd40159520 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4006b670 con 0x7fcd400c6170 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd40159520 0x7fcd400c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd40159520 0x7fcd400c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48510/0 (socket says 172.21.15.33:48510) 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4699c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcd400c80d0 0x7fcd40067ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48052/0 (socket says 172.21.15.33:48052) 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.767+0000 7fcd4619b700 1 -- 172.21.15.33:0/4229144595 learned_addr learned my addr 172.21.15.33:0/4229144595 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4599a700 1 --2- 172.21.15.33:0/4229144595 >> v2:172.21.15.167:3300/0 conn(0x7fcd400c6170 0x7fcd400c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4619b700 1 -- 172.21.15.33:0/4229144595 >> v2:172.21.15.167:3300/0 conn(0x7fcd400c6170 msgr2=0x7fcd400c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4619b700 1 --2- 172.21.15.33:0/4229144595 >> v2:172.21.15.167:3300/0 conn(0x7fcd400c6170 0x7fcd400c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4619b700 1 -- 172.21.15.33:0/4229144595 >> v1:172.21.15.33:6789/0 conn(0x7fcd400c80d0 legacy=0x7fcd40067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4619b700 1 -- 172.21.15.33:0/4229144595 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4006a160 con 0x7fcd40159520 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4599a700 1 --2- 172.21.15.33:0/4229144595 >> v2:172.21.15.167:3300/0 conn(0x7fcd400c6170 0x7fcd400c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd4619b700 1 --2- 172.21.15.33:0/4229144595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40159520 0x7fcd400c5c30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fcd400c6e50 tx=0x7fcd3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d111de791405a0eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd45199700 1 -- 172.21.15.33:0/4229144595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00b860 con 0x7fcd40159520 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd45199700 1 -- 172.21.15.33:0/4229144595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c00b9c0 con 0x7fcd40159520 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.768+0000 7fcd45199700 1 -- 172.21.15.33:0/4229144595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00bd60 con 0x7fcd40159520 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 -- 172.21.15.33:0/4229144595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40159520 msgr2=0x7fcd400c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 --2- 172.21.15.33:0/4229144595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40159520 0x7fcd400c5c30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fcd400c6e50 tx=0x7fcd3c009250 comp rx=0 tx=0).stop 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 -- 172.21.15.33:0/4229144595 shutdown_connections 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 --2- 172.21.15.33:0/4229144595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40159520 0x7fcd400c5c30 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 --2- 172.21.15.33:0/4229144595 >> v2:172.21.15.167:3300/0 conn(0x7fcd400c6170 0x7fcd400c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 -- 172.21.15.33:0/4229144595 >> 172.21.15.33:0/4229144595 conn(0x7fcd400cc930 msgr2=0x7fcd400ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 -- 172.21.15.33:0/4229144595 shutdown_connections 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.769+0000 7fcd4719d700 1 -- 172.21.15.33:0/4229144595 wait complete. 2023-10-06T05:26:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.770+0000 7fcd4719d700 1 Processor -- start 2023-10-06T05:26:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.771+0000 7fcd4719d700 1 -- start start 2023-10-06T05:26:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.771+0000 7fcd4719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd400c6170 0x7fcd400b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.771+0000 7fcd4719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 0x7fcd4015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd40159520 0x7fcd4015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4015b010 con 0x7fcd400c80d0 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40153e60 con 0x7fcd40159520 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40153500 con 0x7fcd400c6170 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 0x7fcd4015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 0x7fcd4015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48514/0 (socket says 172.21.15.33:48514) 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 -- 172.21.15.33:0/482481925 learned_addr learned my addr 172.21.15.33:0/482481925 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4619b700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd400c6170 0x7fcd400b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4699c700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd40159520 0x7fcd4015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 -- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd400c6170 msgr2=0x7fcd400b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd400c6170 0x7fcd400b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 -- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd40159520 msgr2=0x7fcd4015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd40159520 0x7fcd4015a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 -- 172.21.15.33:0/482481925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c0094f0 con 0x7fcd400c80d0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.772+0000 7fcd4599a700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 0x7fcd4015c250 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c830 tx=0x7fcd30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.773+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30011600 con 0x7fcd400c80d0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.773+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30011760 con 0x7fcd400c80d0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.773+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30011b20 con 0x7fcd400c80d0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.773+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd401537a0 con 0x7fcd400c80d0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.773+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4015b5c0 con 0x7fcd400c80d0 2023-10-06T05:26:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.774+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4004c380 con 0x7fcd400c80d0 2023-10-06T05:26:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.775+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd3000a3e0 con 0x7fcd400c80d0 2023-10-06T05:26:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.776+0000 7fcd367fc700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd24066f30 0x7fcd240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.776+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcd30084b20 con 0x7fcd400c80d0 2023-10-06T05:26:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.776+0000 7fcd4619b700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd24066f30 0x7fcd240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.777+0000 7fcd4619b700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd24066f30 0x7fcd240693e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcd400c6e50 tx=0x7fcd3c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:28.780+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd30051c30 con 0x7fcd400c80d0 2023-10-06T05:26:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.029+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd4015ad30 con 0x7fcd400c80d0 2023-10-06T05:26:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.030+0000 7fcd367fc700 1 -- 172.21.15.33:0/482481925 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fcd30015090 con 0x7fcd400c80d0 2023-10-06T05:26:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:29.032 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd24066f30 msgr2=0x7fcd240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd24066f30 0x7fcd240693e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcd400c6e50 tx=0x7fcd3c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 msgr2=0x7fcd4015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 0x7fcd4015c250 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c830 tx=0x7fcd30008040 comp rx=0 tx=0).stop 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 shutdown_connections 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcd24066f30 0x7fcd240693e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd40159520 0x7fcd4015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd400c6170 0x7fcd400b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 --2- 172.21.15.33:0/482481925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400c80d0 0x7fcd4015c250 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 >> 172.21.15.33:0/482481925 conn(0x7fcd400cc930 msgr2=0x7fcd4006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 shutdown_connections 2023-10-06T05:26:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:29.032+0000 7fcd4719d700 1 -- 172.21.15.33:0/482481925 wait complete. 2023-10-06T05:26:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:29.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session evict 7307 2023-10-06T05:26:29.884 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:26:49.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:49.885 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:49.913 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:26:49.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:26:50.038 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:26:50.116 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:26:50.116 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:26:50.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:50.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:50.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:26:50.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:50.198 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:50.235 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:26:50.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:50.235 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:50.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:26:50.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:50.317 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:26:50.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.540+0000 7f301759e700 1 Processor -- start 2023-10-06T05:26:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.540+0000 7f301759e700 1 -- start start 2023-10-06T05:26:50.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f301759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f301813faa0 0x7f301813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f301759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30180c9cb0 con 0x7f301806aad0 2023-10-06T05:26:50.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f301759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30180ca270 con 0x7f30180c7940 2023-10-06T05:26:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f301759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301806b640 con 0x7f301813faa0 2023-10-06T05:26:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f3016d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f301806aad0 0x7f301806af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46254/0 (socket says 172.21.15.33:46254) 2023-10-06T05:26:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f3016d9d700 1 -- 172.21.15.33:0/3599210747 learned_addr learned my addr 172.21.15.33:0/3599210747 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f301659c700 1 --2- 172.21.15.33:0/3599210747 >> v2:172.21.15.167:3300/0 conn(0x7f301813faa0 0x7f301813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.542+0000 7f301559a700 1 -- 172.21.15.33:0/3599210747 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201053224 0 0) 0x7f30180c9cb0 con 0x7f301806aad0 2023-10-06T05:26:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301559a700 1 -- 172.21.15.33:0/3599210747 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3000003570 con 0x7f301806aad0 2023-10-06T05:26:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301659c700 1 -- 172.21.15.33:0/3599210747 >> v1:172.21.15.167:6789/0 conn(0x7f30180c7940 legacy=0x7f30180c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301659c700 1 -- 172.21.15.33:0/3599210747 >> v1:172.21.15.33:6789/0 conn(0x7f301806aad0 legacy=0x7f301806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301659c700 1 -- 172.21.15.33:0/3599210747 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301806b840 con 0x7f301813faa0 2023-10-06T05:26:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301659c700 1 --2- 172.21.15.33:0/3599210747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301813faa0 0x7f301813fe70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f30080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa1efd2b08bcd28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301559a700 1 -- 172.21.15.33:0/3599210747 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300800f6b0 con 0x7f301813faa0 2023-10-06T05:26:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301559a700 1 -- 172.21.15.33:0/3599210747 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300800f810 con 0x7f301813faa0 2023-10-06T05:26:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.543+0000 7f301559a700 1 -- 172.21.15.33:0/3599210747 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300800fbb0 con 0x7f301813faa0 2023-10-06T05:26:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 -- 172.21.15.33:0/3599210747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301813faa0 msgr2=0x7f301813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 --2- 172.21.15.33:0/3599210747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301813faa0 0x7f301813fe70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f30080091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 -- 172.21.15.33:0/3599210747 shutdown_connections 2023-10-06T05:26:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 --2- 172.21.15.33:0/3599210747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301813faa0 0x7f301813fe70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 -- 172.21.15.33:0/3599210747 >> 172.21.15.33:0/3599210747 conn(0x7f301806d030 msgr2=0x7f301806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 -- 172.21.15.33:0/3599210747 shutdown_connections 2023-10-06T05:26:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.544+0000 7f301759e700 1 -- 172.21.15.33:0/3599210747 wait complete. 2023-10-06T05:26:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.545+0000 7f301759e700 1 Processor -- start 2023-10-06T05:26:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.546+0000 7f301759e700 1 -- start start 2023-10-06T05:26:50.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.546+0000 7f301759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806aad0 0x7f3018168b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f301759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30180c7940 0x7f3018165b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f301759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 0x7f3018159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f301759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301813e830 con 0x7f301813faa0 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f301759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180afde0 con 0x7f301806aad0 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f301759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30180af4f0 con 0x7f30180c7940 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 0x7f3018159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f301659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806aad0 0x7f3018168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 0x7f3018159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47916/0 (socket says 172.21.15.33:47916) 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 -- 172.21.15.33:0/885431084 learned_addr learned my addr 172.21.15.33:0/885431084 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3015d9b700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30180c7940 0x7f3018165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 -- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30180c7940 msgr2=0x7f3018165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30180c7940 0x7f3018165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 -- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806aad0 msgr2=0x7f3018168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806aad0 0x7f3018168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.547+0000 7f3016d9d700 1 -- 172.21.15.33:0/885431084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3008009420 con 0x7f301813faa0 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.548+0000 7f3016d9d700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 0x7f3018159260 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f3010001460 tx=0x7f3010001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.548+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000a890 con 0x7f301813faa0 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.548+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301000a9f0 con 0x7f301813faa0 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.548+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000ad70 con 0x7f301813faa0 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.548+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30180af790 con 0x7f301813faa0 2023-10-06T05:26:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.548+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3018158670 con 0x7f301813faa0 2023-10-06T05:26:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.549+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30180aff70 con 0x7f301813faa0 2023-10-06T05:26:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.550+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3010007300 con 0x7f301813faa0 2023-10-06T05:26:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.551+0000 7f30067fc700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ff8066f30 0x7f2ff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.551+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3010084ae0 con 0x7f301813faa0 2023-10-06T05:26:50.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.551+0000 7f301659c700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ff8066f30 0x7f2ff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.552+0000 7f301659c700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ff8066f30 0x7f2ff80693e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f30080093f0 tx=0x7f3008019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.555+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3010051ba0 con 0x7f301813faa0 2023-10-06T05:26:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.802+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f301804c380 con 0x7f301813faa0 2023-10-06T05:26:50.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.803+0000 7f30067fc700 1 -- 172.21.15.33:0/885431084 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f30100551c0 con 0x7f301813faa0 2023-10-06T05:26:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:50.805 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ff8066f30 msgr2=0x7f2ff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ff8066f30 0x7f2ff80693e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f30080093f0 tx=0x7f3008019000 comp rx=0 tx=0).stop 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 msgr2=0x7f3018159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 0x7f3018159260 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f3010001460 tx=0x7f3010001ae0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 shutdown_connections 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2ff8066f30 0x7f2ff80693e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301813faa0 0x7f3018159260 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806aad0 0x7f3018168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 --2- 172.21.15.33:0/885431084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30180c7940 0x7f3018165b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 >> 172.21.15.33:0/885431084 conn(0x7f301806d030 msgr2=0x7f301813ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 shutdown_connections 2023-10-06T05:26:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:50.805+0000 7f301759e700 1 -- 172.21.15.33:0/885431084 wait complete. 2023-10-06T05:26:50.808 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:50.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:51.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.011+0000 7fed04b9d700 1 Processor -- start 2023-10-06T05:26:51.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.011+0000 7fed04b9d700 1 -- start start 2023-10-06T05:26:51.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fed04b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed000cf3b0 0x7fed001594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fed04b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fed000af0a0 0x7fed001599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fed04b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed000cbd80 con 0x7fed000ccfc0 2023-10-06T05:26:51.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fed04b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed000af5f0 con 0x7fed000af0a0 2023-10-06T05:26:51.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fed04b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed000af8d0 con 0x7fed000cf3b0 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fed000af0a0 0x7fed001599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fed000af0a0 0x7fed001599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47924/0 (socket says 172.21.15.33:47924) 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 -- 172.21.15.33:0/2181756524 learned_addr learned my addr 172.21.15.33:0/2181756524 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfeffd700 1 --2- 172.21.15.33:0/2181756524 >> v2:172.21.15.167:3300/0 conn(0x7fed000cf3b0 0x7fed001594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 -- 172.21.15.33:0/2181756524 >> v2:172.21.15.167:3300/0 conn(0x7fed000cf3b0 msgr2=0x7fed001594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 --2- 172.21.15.33:0/2181756524 >> v2:172.21.15.167:3300/0 conn(0x7fed000cf3b0 0x7fed001594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 -- 172.21.15.33:0/2181756524 >> v1:172.21.15.167:6789/0 conn(0x7fed000ccfc0 legacy=0x7fed000cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfffff700 1 -- 172.21.15.33:0/2181756524 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000afbb0 con 0x7fed000af0a0 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.013+0000 7fecfe7fc700 1 -- 172.21.15.33:0/2181756524 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386119728 0 0) 0x7fed000cbd80 con 0x7fed000ccfc0 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fecfffff700 1 --2- 172.21.15.33:0/2181756524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed000af0a0 0x7fed001599e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009f20 tx=0x7fecf8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b624ddbd842c565 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fecfe7fc700 1 -- 172.21.15.33:0/2181756524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf800e070 con 0x7fed000af0a0 2023-10-06T05:26:51.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fecfe7fc700 1 -- 172.21.15.33:0/2181756524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf800b990 con 0x7fed000af0a0 2023-10-06T05:26:51.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fecfe7fc700 1 -- 172.21.15.33:0/2181756524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80092e0 con 0x7fed000af0a0 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fed04b9d700 1 -- 172.21.15.33:0/2181756524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed000af0a0 msgr2=0x7fed001599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fed04b9d700 1 --2- 172.21.15.33:0/2181756524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed000af0a0 0x7fed001599e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009f20 tx=0x7fecf8009840 comp rx=0 tx=0).stop 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fed04b9d700 1 -- 172.21.15.33:0/2181756524 shutdown_connections 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fed04b9d700 1 --2- 172.21.15.33:0/2181756524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed000af0a0 0x7fed001599e0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fed04b9d700 1 --2- 172.21.15.33:0/2181756524 >> v2:172.21.15.167:3300/0 conn(0x7fed000cf3b0 0x7fed001594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.014+0000 7fed04b9d700 1 -- 172.21.15.33:0/2181756524 >> 172.21.15.33:0/2181756524 conn(0x7fed0006cff0 msgr2=0x7fed0006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.015+0000 7fed04b9d700 1 -- 172.21.15.33:0/2181756524 shutdown_connections 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.015+0000 7fed04b9d700 1 -- 172.21.15.33:0/2181756524 wait complete. 2023-10-06T05:26:51.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.016+0000 7fed04b9d700 1 Processor -- start 2023-10-06T05:26:51.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.016+0000 7fed04b9d700 1 -- start start 2023-10-06T05:26:51.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fed04b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed000af0a0 0x7fed000b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fed04b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed000ccfc0 0x7fed00147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fed04b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 0x7fed000b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fed04b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed000b44a0 con 0x7fed00154540 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fed04b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed000b4780 con 0x7fed000ccfc0 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fed04b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed000b4a60 con 0x7fed000af0a0 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fecfffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 0x7fed000b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fecfffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 0x7fed000b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47930/0 (socket says 172.21.15.33:47930) 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fecfffff700 1 -- 172.21.15.33:0/4153353445 learned_addr learned my addr 172.21.15.33:0/4153353445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fecfeffd700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed000ccfc0 0x7fed00147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.017+0000 7fecff7fe700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed000af0a0 0x7fed000b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fecfffff700 1 -- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed000af0a0 msgr2=0x7fed000b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fecfffff700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed000af0a0 0x7fed000b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fecfffff700 1 -- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed000ccfc0 msgr2=0x7fed00147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fecfffff700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed000ccfc0 0x7fed00147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fecfffff700 1 -- 172.21.15.33:0/4153353445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8009ae0 con 0x7fed00154540 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fecfffff700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 0x7fed000b2430 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fecf8012000 tx=0x7fecf8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8008be0 con 0x7fed00154540 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf8008d40 con 0x7fed00154540 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed000b4d40 con 0x7fed00154540 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80256b0 con 0x7fed00154540 2023-10-06T05:26:51.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.018+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed000b41a0 con 0x7fed00154540 2023-10-06T05:26:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.020+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed000b5240 con 0x7fed00154540 2023-10-06T05:26:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.020+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecf8003040 con 0x7fed00154540 2023-10-06T05:26:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.021+0000 7fece77fe700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecec066f60 0x7fecec069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.021+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fecf8086730 con 0x7fed00154540 2023-10-06T05:26:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.022+0000 7fecff7fe700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecec066f60 0x7fecec069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.022+0000 7fecff7fe700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecec066f60 0x7fecec069410 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fecf0003820 tx=0x7fecf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.026+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecf8053770 con 0x7fed00154540 2023-10-06T05:26:51.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.227+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed0004c380 con 0x7fed00154540 2023-10-06T05:26:51.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.228+0000 7fece77fe700 1 -- 172.21.15.33:0/4153353445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16941 (secure 0 0 0) 0x7fecf8056d90 con 0x7fed00154540 2023-10-06T05:26:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:51.230 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":170,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:29.807750+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:14.643519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:15.441428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2408692474":"2023-10-06T06:26:29.245947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:51.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecec066f60 msgr2=0x7fecec069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecec066f60 0x7fecec069410 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fecf0003820 tx=0x7fecf0009040 comp rx=0 tx=0).stop 2023-10-06T05:26:51.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 msgr2=0x7fed000b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 0x7fed000b2430 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fecf8012000 tx=0x7fecf8018000 comp rx=0 tx=0).stop 2023-10-06T05:26:51.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 shutdown_connections 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fecec066f60 0x7fecec069410 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed00154540 0x7fed000b2430 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed000af0a0 0x7fed000b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 --2- 172.21.15.33:0/4153353445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed000ccfc0 0x7fed00147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 >> 172.21.15.33:0/4153353445 conn(0x7fed0006cff0 msgr2=0x7fed001561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 shutdown_connections 2023-10-06T05:26:51.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.230+0000 7fed04b9d700 1 -- 172.21.15.33:0/4153353445 wait complete. 2023-10-06T05:26:51.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:26:14.643519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:26:15.441428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:26:51.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:26:51.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.439+0000 7fb88bfff700 1 Processor -- start 2023-10-06T05:26:51.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.439+0000 7fb88bfff700 1 -- start start 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c159850 0x7fb88c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb88c15acd0 0x7fb88c06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb88c06d140 0x7fb88c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0b0cd0 con 0x7fb88c15acd0 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c15a840 con 0x7fb88c06d140 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c159f10 con 0x7fb88c159850 2023-10-06T05:26:51.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.440+0000 7fb88a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb88c15acd0 0x7fb88c06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb88c15acd0 0x7fb88c06edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47938/0 (socket says 172.21.15.33:47938) 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2770496921 learned_addr learned my addr 172.21.15.33:0/2770496921 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88b7fe700 1 --2- 172.21.15.33:0/2770496921 >> v2:172.21.15.167:3300/0 conn(0x7fb88c06d140 0x7fb88c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88affd700 1 --2- 172.21.15.33:0/2770496921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c159850 0x7fb88c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2770496921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c159850 msgr2=0x7fb88c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 --2- 172.21.15.33:0/2770496921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c159850 0x7fb88c06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2770496921 >> v2:172.21.15.167:3300/0 conn(0x7fb88c06d140 msgr2=0x7fb88c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 --2- 172.21.15.33:0/2770496921 >> v2:172.21.15.167:3300/0 conn(0x7fb88c06d140 0x7fb88c06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2770496921 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c15a1f0 con 0x7fb88c15acd0 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.441+0000 7fb88a7fc700 1 --2- 172.21.15.33:0/2770496921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c15acd0 0x7fb88c06edd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0cdf90 tx=0x7fb880009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b8e1b884f50ae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb889ffb700 1 -- 172.21.15.33:0/2770496921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000b640 con 0x7fb88c15acd0 2023-10-06T05:26:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb889ffb700 1 -- 172.21.15.33:0/2770496921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000b7a0 con 0x7fb88c15acd0 2023-10-06T05:26:51.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb889ffb700 1 -- 172.21.15.33:0/2770496921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000bb60 con 0x7fb88c15acd0 2023-10-06T05:26:51.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 -- 172.21.15.33:0/2770496921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c15acd0 msgr2=0x7fb88c06edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2770496921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c15acd0 0x7fb88c06edd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0cdf90 tx=0x7fb880009250 comp rx=0 tx=0).stop 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 -- 172.21.15.33:0/2770496921 shutdown_connections 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2770496921 >> v2:172.21.15.167:3300/0 conn(0x7fb88c06d140 0x7fb88c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2770496921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c159850 0x7fb88c06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2770496921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c15acd0 0x7fb88c06edd0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 -- 172.21.15.33:0/2770496921 >> 172.21.15.33:0/2770496921 conn(0x7fb88c06cd40 msgr2=0x7fb88c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 -- 172.21.15.33:0/2770496921 shutdown_connections 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.442+0000 7fb88bfff700 1 -- 172.21.15.33:0/2770496921 wait complete. 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.444+0000 7fb88bfff700 1 Processor -- start 2023-10-06T05:26:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.444+0000 7fb88bfff700 1 -- start start 2023-10-06T05:26:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c06d140 0x7fb88c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 0x7fb88c14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb88c15acd0 0x7fb88c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0cba10 con 0x7fb88c159850 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c14e790 con 0x7fb88c15acd0 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0ad090 con 0x7fb88c06d140 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 0x7fb88c14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 0x7fb88c14e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47946/0 (socket says 172.21.15.33:47946) 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2932896923 learned_addr learned my addr 172.21.15.33:0/2932896923 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88affd700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c06d140 0x7fb88c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.445+0000 7fb88b7fe700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb88c15acd0 0x7fb88c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c06d140 msgr2=0x7fb88c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88a7fc700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c06d140 0x7fb88c0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb88c15acd0 msgr2=0x7fb88c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88a7fc700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb88c15acd0 0x7fb88c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88a7fc700 1 -- 172.21.15.33:0/2932896923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800094f0 con 0x7fb88c159850 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88a7fc700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 0x7fb88c14e250 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb88c065080 tx=0x7fb880017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000b910 con 0x7fb88c159850 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000ba70 con 0x7fb88c159850 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880011630 con 0x7fb88c159850 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c0ad370 con 0x7fb88c159850 2023-10-06T05:26:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.446+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c0cbc60 con 0x7fb88c159850 2023-10-06T05:26:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.447+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c04c380 con 0x7fb88c159850 2023-10-06T05:26:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.448+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8800119a0 con 0x7fb88c159850 2023-10-06T05:26:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.450+0000 7fb872ffd700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb878066f30 0x7fb8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.450+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb880085d80 con 0x7fb88c159850 2023-10-06T05:26:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.450+0000 7fb88affd700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb878066f30 0x7fb8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.451+0000 7fb88affd700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb878066f30 0x7fb8780693e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009db0 tx=0x7fb87c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.454+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb880052dc0 con 0x7fb88c159850 2023-10-06T05:26:51.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.658+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb88c0caaf0 con 0x7fb88c159850 2023-10-06T05:26:51.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.658+0000 7fb872ffd700 1 -- 172.21.15.33:0/2932896923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8800563e0 con 0x7fb88c159850 2023-10-06T05:26:51.660 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.660+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb878066f30 msgr2=0x7fb8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.660+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb878066f30 0x7fb8780693e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009db0 tx=0x7fb87c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.660+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 msgr2=0x7fb88c14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.660+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 0x7fb88c14e250 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb88c065080 tx=0x7fb880017000 comp rx=0 tx=0).stop 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 shutdown_connections 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb878066f30 0x7fb8780693e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb88c15acd0 0x7fb88c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb88c06d140 0x7fb88c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 --2- 172.21.15.33:0/2932896923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb88c159850 0x7fb88c14e250 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 >> 172.21.15.33:0/2932896923 conn(0x7fb88c06cd40 msgr2=0x7fb88c15c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 shutdown_connections 2023-10-06T05:26:51.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.661+0000 7fb88bfff700 1 -- 172.21.15.33:0/2932896923 wait complete. 2023-10-06T05:26:51.679 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:26:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.869+0000 7ff4effff700 1 Processor -- start 2023-10-06T05:26:51.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.869+0000 7ff4effff700 1 -- start start 2023-10-06T05:26:51.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4effff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4f006f330 0x7ff4f006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4effff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f006fcf0 con 0x7ff4f0139140 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4effff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f00c7580 con 0x7ff4f006f330 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f006a7b0 con 0x7ff4f006ac70 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4ee7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4f006f330 0x7ff4f006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4ee7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4f006f330 0x7ff4f006f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47950/0 (socket says 172.21.15.33:47950) 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/677957657 learned_addr learned my addr 172.21.15.33:0/677957657 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.870+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/677957657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/677957657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 msgr2=0x7ff4f006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4ee7fc700 1 --2- 172.21.15.33:0/677957657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/677957657 >> v1:172.21.15.167:6789/0 conn(0x7ff4f0139140 legacy=0x7ff4f00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/677957657 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00c6a00 con 0x7ff4f006f330 2023-10-06T05:26:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4ee7fc700 1 --2- 172.21.15.33:0/677957657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f006f330 0x7ff4f006f7b0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40099b0 tx=0x7ff4e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c814e6e308344457 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4edffb700 1 -- 172.21.15.33:0/677957657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4009ea0 con 0x7ff4f006f330 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4edffb700 1 -- 172.21.15.33:0/677957657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e400b860 con 0x7ff4f006f330 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.871+0000 7ff4edffb700 1 -- 172.21.15.33:0/677957657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400bc00 con 0x7ff4f006f330 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 -- 172.21.15.33:0/677957657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f006f330 msgr2=0x7ff4f006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 --2- 172.21.15.33:0/677957657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f006f330 0x7ff4f006f7b0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40099b0 tx=0x7ff4e4009250 comp rx=0 tx=0).stop 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 -- 172.21.15.33:0/677957657 shutdown_connections 2023-10-06T05:26:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 --2- 172.21.15.33:0/677957657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 --2- 172.21.15.33:0/677957657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f006f330 0x7ff4f006f7b0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 -- 172.21.15.33:0/677957657 >> 172.21.15.33:0/677957657 conn(0x7ff4f006cfe0 msgr2=0x7ff4f00cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 -- 172.21.15.33:0/677957657 shutdown_connections 2023-10-06T05:26:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.872+0000 7ff4effff700 1 -- 172.21.15.33:0/677957657 wait complete. 2023-10-06T05:26:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.873+0000 7ff4effff700 1 Processor -- start 2023-10-06T05:26:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 -- start start 2023-10-06T05:26:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f0155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f006f330 0x7ff4f00ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 0x7ff4f00ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c0ff0 con 0x7ff4f0139140 2023-10-06T05:26:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f0151530 con 0x7ff4f006f330 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.874+0000 7ff4effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f01560d0 con 0x7ff4f006ac70 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 0x7ff4f00ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 0x7ff4f00ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47966/0 (socket says 172.21.15.33:47966) 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 -- 172.21.15.33:0/587552204 learned_addr learned my addr 172.21.15.33:0/587552204 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ee7fc700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f006f330 0x7ff4f00ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f0155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 -- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 msgr2=0x7ff4f0155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f0155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 -- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f006f330 msgr2=0x7ff4f00ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f006f330 0x7ff4f00ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 -- 172.21.15.33:0/587552204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40094f0 con 0x7ff4f0139140 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ee7fc700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f006f330 0x7ff4f00ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.875+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 0x7ff4f00ab940 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009e90 tx=0x7ff4e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.876+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800cc20 con 0x7ff4f0139140 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.876+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e800f040 con 0x7ff4f0139140 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.876+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8014480 con 0x7ff4f0139140 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.876+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f01563b0 con 0x7ff4f0139140 2023-10-06T05:26:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.876+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f00ae910 con 0x7ff4f0139140 2023-10-06T05:26:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.877+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f01568d0 con 0x7ff4f0139140 2023-10-06T05:26:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.878+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4e8014830 con 0x7ff4f0139140 2023-10-06T05:26:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.879+0000 7ff4d6ffd700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.879+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4e8085180 con 0x7ff4f0139140 2023-10-06T05:26:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.879+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.880+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0003720 tx=0x7ff4e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:51.883+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4f01568d0 con 0x7ff4f0139140 2023-10-06T05:26:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.086+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4f00c0dc0 con 0x7ff4f0139140 2023-10-06T05:26:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.087+0000 7ff4d6ffd700 1 -- 172.21.15.33:0/587552204 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v170) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4f00c0dc0 con 0x7ff4f0139140 2023-10-06T05:26:52.088 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 msgr2=0x7ff4dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0003720 tx=0x7ff4e0009040 comp rx=0 tx=0).stop 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 msgr2=0x7ff4f00ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 0x7ff4f00ab940 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009e90 tx=0x7ff4e800b040 comp rx=0 tx=0).stop 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 shutdown_connections 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f0139140 0x7ff4f00ab940 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f006ac70 0x7ff4f0155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 --2- 172.21.15.33:0/587552204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f006f330 0x7ff4f00ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 >> 172.21.15.33:0/587552204 conn(0x7ff4f006cfe0 msgr2=0x7ff4f0152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 shutdown_connections 2023-10-06T05:26:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.089+0000 7ff4effff700 1 -- 172.21.15.33:0/587552204 wait complete. 2023-10-06T05:26:52.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:26:52.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.296+0000 7f9785b9d700 1 Processor -- start 2023-10-06T05:26:52.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.297+0000 7f9785b9d700 1 -- start start 2023-10-06T05:26:52.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f9785b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97800cc7e0 0x7f97800af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f9785b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97800ad430 con 0x7f97800b01f0 2023-10-06T05:26:52.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f9785b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97800afe20 con 0x7f9780156370 2023-10-06T05:26:52.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f9785b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978006bc70 con 0x7f97800cc7e0 2023-10-06T05:26:52.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f978539c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97800b01f0 0x7f97800ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46262/0 (socket says 172.21.15.33:46262) 2023-10-06T05:26:52.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f978539c700 1 -- 172.21.15.33:0/308477408 learned_addr learned my addr 172.21.15.33:0/308477408 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.298+0000 7f977ffff700 1 --2- 172.21.15.33:0/308477408 >> v2:172.21.15.167:3300/0 conn(0x7f97800cc7e0 0x7f97800af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361061932 0 0) 0x7f97800ad430 con 0x7f97800b01f0 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9770003570 con 0x7f97800b01f0 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851931552 0 0) 0x7f97800afe20 con 0x7f9780156370 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97800ad430 con 0x7f9780156370 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304378611 0 0) 0x7f9770003570 con 0x7f97800b01f0 2023-10-06T05:26:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 >> v2:172.21.15.167:3300/0 conn(0x7f97800cc7e0 msgr2=0x7f97800af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 --2- 172.21.15.33:0/308477408 >> v2:172.21.15.167:3300/0 conn(0x7f97800cc7e0 0x7f97800af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 >> v1:172.21.15.167:6789/0 conn(0x7f9780156370 legacy=0x7f97800c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800cccc0 con 0x7f97800b01f0 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f97780017f0 con 0x7f97800b01f0 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 >> v1:172.21.15.33:6789/0 conn(0x7f97800b01f0 legacy=0x7f97800ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9770003c40 0x7f9770004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9770004810 0x7f9770004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f977000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978006bc70 con 0x7f9770005210 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977000c1b0 con 0x7f9770003c40 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977000e580 con 0x7f9770004810 2023-10-06T05:26:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f978539c700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f977000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f977ffff700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9770004810 0x7f9770004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.299+0000 7f9784b9b700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9770003c40 0x7f9770004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 -- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9770004810 msgr2=0x7f9770004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9770004810 0x7f9770004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 -- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9770003c40 msgr2=0x7f9770004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9770003c40 0x7f9770004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 -- 172.21.15.33:0/308477408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770003570 con 0x7f9770005210 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 -- 172.21.15.33:0/308477408 reap_dead start 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f978539c700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f977000ba60 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f977800a7d0 tx=0x7f97780020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5129751dad17aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780070d0 con 0x7f9770005210 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9778007230 con 0x7f9770005210 2023-10-06T05:26:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.300+0000 7f977f7fe700 1 -- 172.21.15.33:0/308477408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780075f0 con 0x7f9770005210 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 -- 172.21.15.33:0/308477408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 msgr2=0x7f977000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f977000ba60 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f977800a7d0 tx=0x7f97780020c0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 -- 172.21.15.33:0/308477408 shutdown_connections 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 --2- 172.21.15.33:0/308477408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f977000ba60 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 -- 172.21.15.33:0/308477408 >> 172.21.15.33:0/308477408 conn(0x7f978006d030 msgr2=0x7f97800cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 -- 172.21.15.33:0/308477408 shutdown_connections 2023-10-06T05:26:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.301+0000 7f9785b9d700 1 -- 172.21.15.33:0/308477408 wait complete. 2023-10-06T05:26:52.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.302+0000 7f9785b9d700 1 Processor -- start 2023-10-06T05:26:52.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.303+0000 7f9785b9d700 1 -- start start 2023-10-06T05:26:52.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.303+0000 7f9785b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f9780148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.303+0000 7f9785b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9780154ad0 0x7f97800a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.303+0000 7f9785b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9780154f40 0x7f97800a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9785b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780155ac0 con 0x7f9770005210 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9785b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97800a67a0 con 0x7f9780154ad0 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9785b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97801493a0 con 0x7f9780154f40 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f9780148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f9780148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47984/0 (socket says 172.21.15.33:47984) 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 -- 172.21.15.33:0/990593032 learned_addr learned my addr 172.21.15.33:0/990593032 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f977ffff700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9780154ad0 0x7f97800a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f978539c700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9780154f40 0x7f97800a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 -- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9780154f40 msgr2=0x7f97800a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9780154f40 0x7f97800a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 -- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9780154ad0 msgr2=0x7f97800a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9780154ad0 0x7f97800a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 -- 172.21.15.33:0/990593032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780155500 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.304+0000 7f9784b9b700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f9780148e60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9774002b80 tx=0x7f9774007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.305+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977400fa00 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.305+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977400fb80 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.305+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774006820 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.305+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97801557e0 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.305+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97800ab110 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.306+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97800a6270 con 0x7f9770005210 2023-10-06T05:26:52.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.307+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9774006bd0 con 0x7f9770005210 2023-10-06T05:26:52.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.307+0000 7f977cff9700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9764066f30 0x7f97640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.308+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9774084c30 con 0x7f9770005210 2023-10-06T05:26:52.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.308+0000 7f977ffff700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9764066f30 0x7f97640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.309+0000 7f977ffff700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9764066f30 0x7f97640693e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f976c006ed0 tx=0x7f976c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.312+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9774051c70 con 0x7f9770005210 2023-10-06T05:26:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.511+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f978004c380 con 0x7f9770005210 2023-10-06T05:26:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.512+0000 7f977cff9700 1 -- 172.21.15.33:0/990593032 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9774055290 con 0x7f9770005210 2023-10-06T05:26:52.514 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:26:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.514+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9764066f30 msgr2=0x7f97640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.514+0000 7f9785b9d700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9764066f30 0x7f97640693e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f976c006ed0 tx=0x7f976c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.514+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 msgr2=0x7f9780148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.514+0000 7f9785b9d700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f9780148e60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9774002b80 tx=0x7f9774007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 shutdown_connections 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9764066f30 0x7f97640693e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9780154f40 0x7f97800a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9770005210 0x7f9780148e60 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 --2- 172.21.15.33:0/990593032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9780154ad0 0x7f97800a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 >> 172.21.15.33:0/990593032 conn(0x7f978006d030 msgr2=0x7f978006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 shutdown_connections 2023-10-06T05:26:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.515+0000 7f9785b9d700 1 -- 172.21.15.33:0/990593032 wait complete. 2023-10-06T05:26:52.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:26:52.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.721+0000 7f2356d9d700 1 Processor -- start 2023-10-06T05:26:52.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.721+0000 7f2356d9d700 1 -- start start 2023-10-06T05:26:52.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2356d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f235006c7b0 0x7f2350067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2356d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23500aff90 0x7f23500a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2356d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235015d3a0 con 0x7f235006cb80 2023-10-06T05:26:52.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2356d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23500b0470 con 0x7f23500aff90 2023-10-06T05:26:52.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2356d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23500b0750 con 0x7f235006c7b0 2023-10-06T05:26:52.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235559a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23500aff90 0x7f23500a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f235006cb80 0x7f235015d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46272/0 (socket says 172.21.15.33:46272) 2023-10-06T05:26:52.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235659c700 1 -- 172.21.15.33:0/2513695155 learned_addr learned my addr 172.21.15.33:0/2513695155 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:52.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2355d9b700 1 --2- 172.21.15.33:0/2513695155 >> v2:172.21.15.167:3300/0 conn(0x7f235006c7b0 0x7f2350067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235559a700 1 -- 172.21.15.33:0/2513695155 >> v2:172.21.15.167:3300/0 conn(0x7f235006c7b0 msgr2=0x7f2350067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235559a700 1 --2- 172.21.15.33:0/2513695155 >> v2:172.21.15.167:3300/0 conn(0x7f235006c7b0 0x7f2350067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235559a700 1 -- 172.21.15.33:0/2513695155 >> v1:172.21.15.33:6789/0 conn(0x7f235006cb80 legacy=0x7f235015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f2354d99700 1 -- 172.21.15.33:0/2513695155 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212812629 0 0) 0x7f235015d3a0 con 0x7f235006cb80 2023-10-06T05:26:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.723+0000 7f235559a700 1 -- 172.21.15.33:0/2513695155 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500b0a30 con 0x7f23500aff90 2023-10-06T05:26:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.724+0000 7f235559a700 1 --2- 172.21.15.33:0/2513695155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23500aff90 0x7f23500a9890 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f23500afc10 tx=0x7f23400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df721ffbfd5ea888 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.724+0000 7f2354d99700 1 -- 172.21.15.33:0/2513695155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000b6b0 con 0x7f23500aff90 2023-10-06T05:26:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.724+0000 7f2354d99700 1 -- 172.21.15.33:0/2513695155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000b810 con 0x7f23500aff90 2023-10-06T05:26:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.724+0000 7f2354d99700 1 -- 172.21.15.33:0/2513695155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000bbb0 con 0x7f23500aff90 2023-10-06T05:26:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.724+0000 7f2356d9d700 1 -- 172.21.15.33:0/2513695155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23500aff90 msgr2=0x7f23500a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.724+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2513695155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23500aff90 0x7f23500a9890 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f23500afc10 tx=0x7f23400091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.725+0000 7f2356d9d700 1 -- 172.21.15.33:0/2513695155 shutdown_connections 2023-10-06T05:26:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.725+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2513695155 >> v2:172.21.15.167:3300/0 conn(0x7f235006c7b0 0x7f2350067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.725+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2513695155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23500aff90 0x7f23500a9890 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.725+0000 7f2356d9d700 1 -- 172.21.15.33:0/2513695155 >> 172.21.15.33:0/2513695155 conn(0x7f235006d670 msgr2=0x7f235006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.725+0000 7f2356d9d700 1 -- 172.21.15.33:0/2513695155 shutdown_connections 2023-10-06T05:26:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.725+0000 7f2356d9d700 1 -- 172.21.15.33:0/2513695155 wait complete. 2023-10-06T05:26:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.726+0000 7f2356d9d700 1 Processor -- start 2023-10-06T05:26:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.726+0000 7f2356d9d700 1 -- start start 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2356d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 0x7f235015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2356d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235006cb80 0x7f23501429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2356d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500aff90 0x7f2350142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2356d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23501595b0 con 0x7f235006c7b0 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2356d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2350159890 con 0x7f23500aff90 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2356d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2350159b70 con 0x7f235006cb80 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2355d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 0x7f235015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2355d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 0x7f235015bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48006/0 (socket says 172.21.15.33:48006) 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.727+0000 7f2355d9b700 1 -- 172.21.15.33:0/2538777606 learned_addr learned my addr 172.21.15.33:0/2538777606 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f235559a700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235006cb80 0x7f23501429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f235659c700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500aff90 0x7f2350142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2355d9b700 1 -- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235006cb80 msgr2=0x7f23501429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2355d9b700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235006cb80 0x7f23501429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2355d9b700 1 -- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500aff90 msgr2=0x7f2350142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2355d9b700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500aff90 0x7f2350142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2355d9b700 1 -- 172.21.15.33:0/2538777606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009420 con 0x7f235006c7b0 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f235559a700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235006cb80 0x7f23501429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f235659c700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500aff90 0x7f2350142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2355d9b700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 0x7f235015bfd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f234c00d810 tx=0x7f234c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c0125f0 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c012750 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c012b10 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2350159e50 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.728+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235015ed50 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.729+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235015a1c0 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.730+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f234c01e460 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.731+0000 7f2345ffb700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2334066f30 0x7f23340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.731+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f234c084df0 con 0x7f235006c7b0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.732+0000 7f235559a700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2334066f30 0x7f23340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.732+0000 7f235559a700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2334066f30 0x7f23340693e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f23500b03a0 tx=0x7f2340017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.735+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f234c051e30 con 0x7f235006c7b0 2023-10-06T05:26:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.938+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23501596f0 con 0x7f235006c7b0 2023-10-06T05:26:52.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.939+0000 7f2345ffb700 1 -- 172.21.15.33:0/2538777606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23501596f0 con 0x7f235006c7b0 2023-10-06T05:26:52.941 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:26:52.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2334066f30 msgr2=0x7f23340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2334066f30 0x7f23340693e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f23500b03a0 tx=0x7f2340017000 comp rx=0 tx=0).stop 2023-10-06T05:26:52.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 msgr2=0x7f235015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 0x7f235015bfd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f234c00d810 tx=0x7f234c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:52.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 shutdown_connections 2023-10-06T05:26:52.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2334066f30 0x7f23340693e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500aff90 0x7f2350142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f235006c7b0 0x7f235015bfd0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 --2- 172.21.15.33:0/2538777606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f235006cb80 0x7f23501429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 >> 172.21.15.33:0/2538777606 conn(0x7f235006d670 msgr2=0x7f2350154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 shutdown_connections 2023-10-06T05:26:52.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:52.941+0000 7f2356d9d700 1 -- 172.21.15.33:0/2538777606 wait complete. 2023-10-06T05:26:52.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:53.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.146+0000 7faffcf9d700 1 Processor -- start 2023-10-06T05:26:53.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.146+0000 7faffcf9d700 1 -- start start 2023-10-06T05:26:53.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.147+0000 7faffcf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faff813faa0 0x7faff813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.147+0000 7faffcf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff80c7940 0x7faff80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.147+0000 7faffcf9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff80c9cb0 con 0x7faff806aad0 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.147+0000 7faffcf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff806b660 con 0x7faff813faa0 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.147+0000 7faffcf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80ca270 con 0x7faff80c7940 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faff813faa0 0x7faff813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff6ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff80c7940 0x7faff80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faff813faa0 0x7faff813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:54024/0 (socket says 172.21.15.33:54024) 2023-10-06T05:26:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 -- 172.21.15.33:0/3938728300 learned_addr learned my addr 172.21.15.33:0/3938728300 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 -- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff80c7940 msgr2=0x7faff80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 --2- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff80c7940 0x7faff80c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 -- 172.21.15.33:0/3938728300 >> v1:172.21.15.167:6789/0 conn(0x7faff806aad0 legacy=0x7faff806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff77fe700 1 -- 172.21.15.33:0/3938728300 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff806b840 con 0x7faff813faa0 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff6ffd700 1 --2- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff80c7940 0x7faff80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.148+0000 7faff67fc700 1 -- 172.21.15.33:0/3938728300 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632007656 0 0) 0x7faff80c9cb0 con 0x7faff806aad0 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faff77fe700 1 --2- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff813faa0 0x7faff813fe70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fafe80099b0 tx=0x7fafe8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83e9bfb8bee3780 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faff67fc700 1 -- 172.21.15.33:0/3938728300 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8009ea0 con 0x7faff813faa0 2023-10-06T05:26:53.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faff67fc700 1 -- 172.21.15.33:0/3938728300 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe800b860 con 0x7faff813faa0 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faff67fc700 1 -- 172.21.15.33:0/3938728300 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe800bc00 con 0x7faff813faa0 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faffcf9d700 1 -- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff813faa0 msgr2=0x7faff813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faffcf9d700 1 --2- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff813faa0 0x7faff813fe70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fafe80099b0 tx=0x7fafe8009250 comp rx=0 tx=0).stop 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faffcf9d700 1 -- 172.21.15.33:0/3938728300 shutdown_connections 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faffcf9d700 1 --2- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff813faa0 0x7faff813fe70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faffcf9d700 1 --2- 172.21.15.33:0/3938728300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff80c7940 0x7faff80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.149+0000 7faffcf9d700 1 -- 172.21.15.33:0/3938728300 >> 172.21.15.33:0/3938728300 conn(0x7faff806d030 msgr2=0x7faff806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.150+0000 7faffcf9d700 1 -- 172.21.15.33:0/3938728300 shutdown_connections 2023-10-06T05:26:53.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.150+0000 7faffcf9d700 1 -- 172.21.15.33:0/3938728300 wait complete. 2023-10-06T05:26:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.151+0000 7faffcf9d700 1 Processor -- start 2023-10-06T05:26:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.151+0000 7faffcf9d700 1 -- start start 2023-10-06T05:26:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faffcf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff806aad0 0x7faff80a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faffcf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 0x7faff80c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faffcf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff813faa0 0x7faff813edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faffcf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80ab310 con 0x7faff80c7940 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faffcf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80c4790 con 0x7faff806aad0 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faffcf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff816e2e0 con 0x7faff813faa0 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faff6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 0x7faff80c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faff6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 0x7faff80c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48018/0 (socket says 172.21.15.33:48018) 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faff6ffd700 1 -- 172.21.15.33:0/2347052799 learned_addr learned my addr 172.21.15.33:0/2347052799 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faff77fe700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff806aad0 0x7faff80a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.152+0000 7faff7fff700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff813faa0 0x7faff813edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff6ffd700 1 -- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff813faa0 msgr2=0x7faff813edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff6ffd700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff813faa0 0x7faff813edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff6ffd700 1 -- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff806aad0 msgr2=0x7faff80a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff6ffd700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff806aad0 0x7faff80a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff6ffd700 1 -- 172.21.15.33:0/2347052799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec00b6d0 con 0x7faff80c7940 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff7fff700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff813faa0 0x7faff813edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faff6ffd700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 0x7faff80c4250 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fafec00edc0 tx=0x7fafec00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00dde0 con 0x7faff80c7940 2023-10-06T05:26:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec010440 con 0x7faff80c7940 2023-10-06T05:26:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe80094f0 con 0x7faff80c7940 2023-10-06T05:26:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0107c0 con 0x7faff80c7940 2023-10-06T05:26:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.153+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff80a9e70 con 0x7faff80c7940 2023-10-06T05:26:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.154+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff804c380 con 0x7faff80c7940 2023-10-06T05:26:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.155+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafec010b70 con 0x7faff80c7940 2023-10-06T05:26:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.156+0000 7fafdf7fe700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafe4066f30 0x7fafe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.156+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafec084be0 con 0x7faff80c7940 2023-10-06T05:26:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.157+0000 7faff77fe700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafe4066f30 0x7fafe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.157+0000 7faff77fe700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafe4066f30 0x7fafe40693e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fafe80099b0 tx=0x7fafe8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.160+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafec051c20 con 0x7faff80c7940 2023-10-06T05:26:53.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.403+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faff806bb50 con 0x7faff80c7940 2023-10-06T05:26:53.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.404+0000 7fafdf7fe700 1 -- 172.21.15.33:0/2347052799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fafec01d090 con 0x7faff80c7940 2023-10-06T05:26:53.405 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:53.406 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafe4066f30 msgr2=0x7fafe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafe4066f30 0x7fafe40693e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fafe80099b0 tx=0x7fafe8017000 comp rx=0 tx=0).stop 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 msgr2=0x7faff80c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 0x7faff80c4250 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fafec00edc0 tx=0x7fafec00b430 comp rx=0 tx=0).stop 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 shutdown_connections 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafe4066f30 0x7fafe40693e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faff813faa0 0x7faff813edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faff806aad0 0x7faff80a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 --2- 172.21.15.33:0/2347052799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faff80c7940 0x7faff80c4250 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 >> 172.21.15.33:0/2347052799 conn(0x7faff806d030 msgr2=0x7faff806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 shutdown_connections 2023-10-06T05:26:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.406+0000 7faffcf9d700 1 -- 172.21.15.33:0/2347052799 wait complete. 2023-10-06T05:26:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:53.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:53.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.612+0000 7f050df9d700 1 Processor -- start 2023-10-06T05:26:53.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.612+0000 7f050df9d700 1 -- start start 2023-10-06T05:26:53.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f050df9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0508158700 0x7f05080adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f050df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f0508069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f050df9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0508158ba0 con 0x7f0508155140 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f050df9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050806a380 con 0x7f0508158700 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f050df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05080c8e40 con 0x7f05080699c0 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f0507fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0508158700 0x7f05080adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f0507fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0508158700 0x7f05080adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48034/0 (socket says 172.21.15.33:48034) 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f0507fff700 1 -- 172.21.15.33:0/632682451 learned_addr learned my addr 172.21.15.33:0/632682451 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f050d79c700 1 --2- 172.21.15.33:0/632682451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f0508069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f0507fff700 1 -- 172.21.15.33:0/632682451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 msgr2=0x7f0508069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f0507fff700 1 --2- 172.21.15.33:0/632682451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f0508069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f0507fff700 1 -- 172.21.15.33:0/632682451 >> v1:172.21.15.167:6789/0 conn(0x7f0508155140 legacy=0x7f05080c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.614+0000 7f05077fe700 1 -- 172.21.15.33:0/632682451 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418160430 0 0) 0x7f0508158ba0 con 0x7f0508155140 2023-10-06T05:26:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.615+0000 7f0507fff700 1 -- 172.21.15.33:0/632682451 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05080ae490 con 0x7f0508158700 2023-10-06T05:26:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.615+0000 7f0507fff700 1 --2- 172.21.15.33:0/632682451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508158700 0x7f05080adf50 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f04f40099e0 tx=0x7f04f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1f5612f41e0dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.615+0000 7f05077fe700 1 -- 172.21.15.33:0/632682451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400b820 con 0x7f0508158700 2023-10-06T05:26:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.615+0000 7f05077fe700 1 -- 172.21.15.33:0/632682451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f400b980 con 0x7f0508158700 2023-10-06T05:26:53.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.615+0000 7f05077fe700 1 -- 172.21.15.33:0/632682451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400bd20 con 0x7f0508158700 2023-10-06T05:26:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 -- 172.21.15.33:0/632682451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508158700 msgr2=0x7f05080adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 --2- 172.21.15.33:0/632682451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508158700 0x7f05080adf50 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f04f40099e0 tx=0x7f04f4009250 comp rx=0 tx=0).stop 2023-10-06T05:26:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 -- 172.21.15.33:0/632682451 shutdown_connections 2023-10-06T05:26:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 --2- 172.21.15.33:0/632682451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f0508069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 --2- 172.21.15.33:0/632682451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508158700 0x7f05080adf50 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 -- 172.21.15.33:0/632682451 >> 172.21.15.33:0/632682451 conn(0x7f050806d030 msgr2=0x7f050806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 -- 172.21.15.33:0/632682451 shutdown_connections 2023-10-06T05:26:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.616+0000 7f050df9d700 1 -- 172.21.15.33:0/632682451 wait complete. 2023-10-06T05:26:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.617+0000 7f050df9d700 1 Processor -- start 2023-10-06T05:26:53.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.617+0000 7f050df9d700 1 -- start start 2023-10-06T05:26:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.618+0000 7f050df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f050816d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.618+0000 7f050df9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 0x7f050814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.618+0000 7f050df9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508158700 0x7f05080c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.618+0000 7f050df9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05080c34a0 con 0x7f0508155140 2023-10-06T05:26:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.618+0000 7f050df9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05080b12f0 con 0x7f0508158700 2023-10-06T05:26:53.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.618+0000 7f050df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05080b15d0 con 0x7f05080699c0 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 0x7f050814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 0x7f050814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48038/0 (socket says 172.21.15.33:48038) 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 -- 172.21.15.33:0/3176972290 learned_addr learned my addr 172.21.15.33:0/3176972290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f050cf9b700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f050816d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f050d79c700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508158700 0x7f05080c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 -- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 msgr2=0x7f050816d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f050816d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 -- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508158700 msgr2=0x7f05080c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508158700 0x7f05080c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 -- 172.21.15.33:0/3176972290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40094f0 con 0x7f0508155140 2023-10-06T05:26:53.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f050cf9b700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f050816d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f050d79c700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508158700 0x7f05080c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0507fff700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 0x7f050814ff50 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0508063610 tx=0x7f04f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.619+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f401a450 con 0x7f0508155140 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.620+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f401a5f0 con 0x7f0508155140 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.620+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05080b18b0 con 0x7f0508155140 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.620+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f401a9b0 con 0x7f0508155140 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.620+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05080b1e70 con 0x7f0508155140 2023-10-06T05:26:53.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.621+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05080af750 con 0x7f0508155140 2023-10-06T05:26:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.622+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04f401b430 con 0x7f0508155140 2023-10-06T05:26:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.622+0000 7f0504ff9700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04ec066f30 0x7f04ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.622+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04f40862c0 con 0x7f0508155140 2023-10-06T05:26:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.623+0000 7f050cf9b700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04ec066f30 0x7f04ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.623+0000 7f050cf9b700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04ec066f30 0x7f04ec0693e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0011e0 tx=0x7f04fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.627+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05080af750 con 0x7f0508155140 2023-10-06T05:26:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.832+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05080af750 con 0x7f0508155140 2023-10-06T05:26:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.833+0000 7f0504ff9700 1 -- 172.21.15.33:0/3176972290 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f05080af750 con 0x7f0508155140 2023-10-06T05:26:53.835 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:53.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":170,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:29.807750+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:14.643519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:15.441428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2408692474":"2023-10-06T06:26:29.245947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:53.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.835+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04ec066f30 msgr2=0x7f04ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.835+0000 7f050df9d700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04ec066f30 0x7f04ec0693e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0011e0 tx=0x7f04fc009040 comp rx=0 tx=0).stop 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.835+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 msgr2=0x7f050814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.835+0000 7f050df9d700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 0x7f050814ff50 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0508063610 tx=0x7f04f4017000 comp rx=0 tx=0).stop 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 shutdown_connections 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f04ec066f30 0x7f04ec0693e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508158700 0x7f05080c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05080699c0 0x7f050816d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 --2- 172.21.15.33:0/3176972290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0508155140 0x7f050814ff50 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 >> 172.21.15.33:0/3176972290 conn(0x7f050806d030 msgr2=0x7f05080c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:53.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 shutdown_connections 2023-10-06T05:26:53.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:53.836+0000 7f050df9d700 1 -- 172.21.15.33:0/3176972290 wait complete. 2023-10-06T05:26:53.855 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:26:53.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:54.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.041+0000 7f61db59e700 1 Processor -- start 2023-10-06T05:26:54.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.041+0000 7f61db59e700 1 -- start start 2023-10-06T05:26:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61db59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61dc0ac770 0x7f61dc0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61db59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0c8960 con 0x7f61dc0b1840 2023-10-06T05:26:54.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61db59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc0c8680 con 0x7f61dc0ac770 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc1560d0 con 0x7f61dc0acb40 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61da59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61dc0ac770 0x7f61dc0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61da59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61dc0ac770 0x7f61dc0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48042/0 (socket says 172.21.15.33:48042) 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.042+0000 7f61da59c700 1 -- 172.21.15.33:0/2339854173 learned_addr learned my addr 172.21.15.33:0/2339854173 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61da59c700 1 -- 172.21.15.33:0/2339854173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 msgr2=0x7f61dc0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61da59c700 1 --2- 172.21.15.33:0/2339854173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61da59c700 1 -- 172.21.15.33:0/2339854173 >> v1:172.21.15.33:6789/0 conn(0x7f61dc0b1840 legacy=0x7f61dc159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61da59c700 1 -- 172.21.15.33:0/2339854173 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc1563b0 con 0x7f61dc0ac770 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61da59c700 1 --2- 172.21.15.33:0/2339854173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0ac770 0x7f61dc0c93c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00bc40 tx=0x7f61cc00b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61266daabf6e652 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61d959a700 1 -- 172.21.15.33:0/2339854173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00db40 con 0x7f61dc0ac770 2023-10-06T05:26:54.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61d959a700 1 -- 172.21.15.33:0/2339854173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc00dca0 con 0x7f61dc0ac770 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.043+0000 7f61d959a700 1 -- 172.21.15.33:0/2339854173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc017640 con 0x7f61dc0ac770 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 -- 172.21.15.33:0/2339854173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0ac770 msgr2=0x7f61dc0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 --2- 172.21.15.33:0/2339854173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0ac770 0x7f61dc0c93c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00bc40 tx=0x7f61cc00b4e0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 -- 172.21.15.33:0/2339854173 shutdown_connections 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 --2- 172.21.15.33:0/2339854173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0ac770 0x7f61dc0c93c0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 --2- 172.21.15.33:0/2339854173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc0b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 -- 172.21.15.33:0/2339854173 >> 172.21.15.33:0/2339854173 conn(0x7f61dc06c560 msgr2=0x7f61dc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 -- 172.21.15.33:0/2339854173 shutdown_connections 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.044+0000 7f61db59e700 1 -- 172.21.15.33:0/2339854173 wait complete. 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.046+0000 7f61db59e700 1 Processor -- start 2023-10-06T05:26:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.046+0000 7f61db59e700 1 -- start start 2023-10-06T05:26:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61db59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc0ac770 0x7f61dc159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc14e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61db59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 0x7f61dc14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61db59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc155210 con 0x7f61dc0b1840 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61db59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc153ac0 con 0x7f61dc0ac770 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc150a10 con 0x7f61dc0acb40 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 0x7f61dc14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 0x7f61dc14edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48054/0 (socket says 172.21.15.33:48054) 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61dad9d700 1 -- 172.21.15.33:0/3748273978 learned_addr learned my addr 172.21.15.33:0/3748273978 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61d9d9b700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc14e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61da59c700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc0ac770 0x7f61dc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61dad9d700 1 -- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 msgr2=0x7f61dc14e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.047+0000 7f61dad9d700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc14e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61dad9d700 1 -- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc0ac770 msgr2=0x7f61dc159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61dad9d700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc0ac770 0x7f61dc159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61dad9d700 1 -- 172.21.15.33:0/3748273978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc00b780 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61da59c700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc0ac770 0x7f61dc159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61dad9d700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 0x7f61dc14edb0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c8e0 tx=0x7f61d4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d400ce80 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d400a7e0 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc150cf0 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d400ab80 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.048+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc159cb0 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.049+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc1511f0 con 0x7f61dc0b1840 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.050+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61d4012420 con 0x7f61dc0b1840 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.051+0000 7f61ca7fc700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61bc066f30 0x7f61bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.051+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61d4084a50 con 0x7f61dc0b1840 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.052+0000 7f61da59c700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61bc066f30 0x7f61bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.052+0000 7f61da59c700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61bc066f30 0x7f61bc0693e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b520 tx=0x7f61cc019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.055+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61d4051a90 con 0x7f61dc0b1840 2023-10-06T05:26:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.303+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61dc04c380 con 0x7f61dc0b1840 2023-10-06T05:26:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.303+0000 7f61ca7fc700 1 -- 172.21.15.33:0/3748273978 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f61d40550b0 con 0x7f61dc0b1840 2023-10-06T05:26:54.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.305+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61bc066f30 msgr2=0x7f61bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.305+0000 7f61db59e700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61bc066f30 0x7f61bc0693e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b520 tx=0x7f61cc019000 comp rx=0 tx=0).stop 2023-10-06T05:26:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.305+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 msgr2=0x7f61dc14edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.305+0000 7f61db59e700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 0x7f61dc14edb0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c8e0 tx=0x7f61d4001710 comp rx=0 tx=0).stop 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 shutdown_connections 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61bc066f30 0x7f61bc0693e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61dc0b1840 0x7f61dc14edb0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61dc0ac770 0x7f61dc159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 --2- 172.21.15.33:0/3748273978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61dc0acb40 0x7f61dc14e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 >> 172.21.15.33:0/3748273978 conn(0x7f61dc06c560 msgr2=0x7f61dc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 shutdown_connections 2023-10-06T05:26:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.306+0000 7f61db59e700 1 -- 172.21.15.33:0/3748273978 wait complete. 2023-10-06T05:26:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:54.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.510+0000 7f509239d700 1 Processor -- start 2023-10-06T05:26:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.510+0000 7f509239d700 1 -- start start 2023-10-06T05:26:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509239d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f508c155af0 0x7f508c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509239d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f508c06a6a0 0x7f508c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c0c8b50 0x7f508c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509239d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c06b7e0 con 0x7f508c06a6a0 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509239d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c06ab60 con 0x7f508c155af0 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c06ae40 con 0x7f508c0c8b50 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f508c06a6a0 0x7f508c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f508c06a6a0 0x7f508c0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48064/0 (socket says 172.21.15.33:48064) 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 -- 172.21.15.33:0/488170129 learned_addr learned my addr 172.21.15.33:0/488170129 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f509139b700 1 --2- 172.21.15.33:0/488170129 >> v2:172.21.15.167:3300/0 conn(0x7f508c155af0 0x7f508c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 -- 172.21.15.33:0/488170129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c0c8b50 msgr2=0x7f508c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5091b9c700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c0c8b50 0x7f508c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c0c8b50 0x7f508c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 -- 172.21.15.33:0/488170129 >> v2:172.21.15.167:3300/0 conn(0x7f508c155af0 msgr2=0x7f508c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 --2- 172.21.15.33:0/488170129 >> v2:172.21.15.167:3300/0 conn(0x7f508c155af0 0x7f508c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.512+0000 7f5090b9a700 1 -- 172.21.15.33:0/488170129 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c06b120 con 0x7f508c06a6a0 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.513+0000 7f509139b700 1 --2- 172.21.15.33:0/488170129 >> v2:172.21.15.167:3300/0 conn(0x7f508c155af0 0x7f508c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.513+0000 7f5091b9c700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c0c8b50 0x7f508c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.513+0000 7f5090b9a700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c06a6a0 0x7f508c0c7210 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f508c0697c0 tx=0x7f507c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac143f909a640ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.513+0000 7f5083fff700 1 -- 172.21.15.33:0/488170129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00dc50 con 0x7f508c06a6a0 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.513+0000 7f5083fff700 1 -- 172.21.15.33:0/488170129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c00ddb0 con 0x7f508c06a6a0 2023-10-06T05:26:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.513+0000 7f5083fff700 1 -- 172.21.15.33:0/488170129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0196d0 con 0x7f508c06a6a0 2023-10-06T05:26:54.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 -- 172.21.15.33:0/488170129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c06a6a0 msgr2=0x7f508c0c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c06a6a0 0x7f508c0c7210 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f508c0697c0 tx=0x7f507c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 -- 172.21.15.33:0/488170129 shutdown_connections 2023-10-06T05:26:54.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c0c8b50 0x7f508c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 --2- 172.21.15.33:0/488170129 >> v2:172.21.15.167:3300/0 conn(0x7f508c155af0 0x7f508c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 --2- 172.21.15.33:0/488170129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c06a6a0 0x7f508c0c7210 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 -- 172.21.15.33:0/488170129 >> 172.21.15.33:0/488170129 conn(0x7f508c06d030 msgr2=0x7f508c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 -- 172.21.15.33:0/488170129 shutdown_connections 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.514+0000 7f509239d700 1 -- 172.21.15.33:0/488170129 wait complete. 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.515+0000 7f509239d700 1 Processor -- start 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.515+0000 7f509239d700 1 -- start start 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f509239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c06a6a0 0x7f508c16d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f509239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 0x7f508c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f509239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508c155af0 0x7f508c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f509239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0ab190 con 0x7f508c0c8b50 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f509239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0aa580 con 0x7f508c155af0 2023-10-06T05:26:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f509239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0aa860 con 0x7f508c06a6a0 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.516+0000 7f5090b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 0x7f508c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 0x7f508c14db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48074/0 (socket says 172.21.15.33:48074) 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 -- 172.21.15.33:0/3041203135 learned_addr learned my addr 172.21.15.33:0/3041203135 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f509139b700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c06a6a0 0x7f508c16d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5091b9c700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508c155af0 0x7f508c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 -- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c06a6a0 msgr2=0x7f508c16d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c06a6a0 0x7f508c16d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 -- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508c155af0 msgr2=0x7f508c14e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508c155af0 0x7f508c14e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 -- 172.21.15.33:0/3041203135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c00b790 con 0x7f508c0c8b50 2023-10-06T05:26:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f5090b9a700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 0x7f508c14db50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f507c006fd0 tx=0x7f507c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0107d0 con 0x7f508c0c8b50 2023-10-06T05:26:54.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c010950 con 0x7f508c0c8b50 2023-10-06T05:26:54.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c0aab40 con 0x7f508c0c8b50 2023-10-06T05:26:54.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.517+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c010d10 con 0x7f508c0c8b50 2023-10-06T05:26:54.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.518+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0aa240 con 0x7f508c0c8b50 2023-10-06T05:26:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.519+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c04c380 con 0x7f508c0c8b50 2023-10-06T05:26:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.519+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f507c02c430 con 0x7f508c0c8b50 2023-10-06T05:26:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.520+0000 7f50817fa700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5070066f30 0x7f50700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.520+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f507c014070 con 0x7f508c0c8b50 2023-10-06T05:26:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.521+0000 7f509139b700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5070066f30 0x7f50700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.521+0000 7f509139b700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5070066f30 0x7f50700693e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f508800ac00 tx=0x7f5088006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.524+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f507c053960 con 0x7f508c0c8b50 2023-10-06T05:26:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.773+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f508c06b400 con 0x7f508c0c8b50 2023-10-06T05:26:54.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.774+0000 7f50817fa700 1 -- 172.21.15.33:0/3041203135 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f507c056f80 con 0x7f508c0c8b50 2023-10-06T05:26:54.776 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:54.776 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":124,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:16.467314+0000","modified":"2023-10-06T05:26:20.007449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":120,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2543247533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2543247533},{"type":"v1","addr":"172.21.15.33:6835","nonce":2543247533}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:26:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5070066f30 msgr2=0x7f50700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5070066f30 0x7f50700693e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f508800ac00 tx=0x7f5088006f90 comp rx=0 tx=0).stop 2023-10-06T05:26:54.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 msgr2=0x7f508c14db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 0x7f508c14db50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f507c006fd0 tx=0x7f507c01b000 comp rx=0 tx=0).stop 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 shutdown_connections 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5070066f30 0x7f50700693e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f508c155af0 0x7f508c14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f508c06a6a0 0x7f508c16d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.776+0000 7f509239d700 1 --2- 172.21.15.33:0/3041203135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f508c0c8b50 0x7f508c14db50 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.777+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 >> 172.21.15.33:0/3041203135 conn(0x7f508c06d030 msgr2=0x7f508c0cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.777+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 shutdown_connections 2023-10-06T05:26:54.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.777+0000 7f509239d700 1 -- 172.21.15.33:0/3041203135 wait complete. 2023-10-06T05:26:54.780 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 124 2023-10-06T05:26:54.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.982+0000 7f494e39d700 1 Processor -- start 2023-10-06T05:26:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.982+0000 7f494e39d700 1 -- start start 2023-10-06T05:26:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.983+0000 7f494e39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49480ccfc0 0x7f49480cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.983+0000 7f494e39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cc750 0x7f49480b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.983+0000 7f494e39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480cbd80 con 0x7f49480cf3b0 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.983+0000 7f494e39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480af5f0 con 0x7f49480ccfc0 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.983+0000 7f494e39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480af8d0 con 0x7f49480cc750 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.983+0000 7f494cb9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f49480cf3b0 0x7f49481594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46288/0 (socket says 172.21.15.33:46288) 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494cb9a700 1 -- 172.21.15.33:0/996653050 learned_addr learned my addr 172.21.15.33:0/996653050 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494d39b700 1 --2- 172.21.15.33:0/996653050 >> v2:172.21.15.167:3300/0 conn(0x7f49480ccfc0 0x7f49480cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494db9c700 1 --2- 172.21.15.33:0/996653050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cc750 0x7f49480b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f493ffff700 1 -- 172.21.15.33:0/996653050 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247225979 0 0) 0x7f49480cbd80 con 0x7f49480cf3b0 2023-10-06T05:26:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f493ffff700 1 -- 172.21.15.33:0/996653050 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4934003570 con 0x7f49480cf3b0 2023-10-06T05:26:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494db9c700 1 -- 172.21.15.33:0/996653050 >> v2:172.21.15.167:3300/0 conn(0x7f49480ccfc0 msgr2=0x7f49480cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494db9c700 1 --2- 172.21.15.33:0/996653050 >> v2:172.21.15.167:3300/0 conn(0x7f49480ccfc0 0x7f49480cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494db9c700 1 -- 172.21.15.33:0/996653050 >> v1:172.21.15.33:6789/0 conn(0x7f49480cf3b0 legacy=0x7f49481594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494db9c700 1 -- 172.21.15.33:0/996653050 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480afbb0 con 0x7f49480cc750 2023-10-06T05:26:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.984+0000 7f494d39b700 1 --2- 172.21.15.33:0/996653050 >> v2:172.21.15.167:3300/0 conn(0x7f49480ccfc0 0x7f49480cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:54.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494db9c700 1 --2- 172.21.15.33:0/996653050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cc750 0x7f49480b0460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4940009e90 tx=0x7f4940009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3497b270d68c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f493ffff700 1 -- 172.21.15.33:0/996653050 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000e070 con 0x7f49480cc750 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f493ffff700 1 -- 172.21.15.33:0/996653050 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494000bb40 con 0x7f49480cc750 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f493ffff700 1 -- 172.21.15.33:0/996653050 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940012430 con 0x7f49480cc750 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494e39d700 1 -- 172.21.15.33:0/996653050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cc750 msgr2=0x7f49480b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494e39d700 1 --2- 172.21.15.33:0/996653050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cc750 0x7f49480b0460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4940009e90 tx=0x7f4940009780 comp rx=0 tx=0).stop 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494e39d700 1 -- 172.21.15.33:0/996653050 shutdown_connections 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494e39d700 1 --2- 172.21.15.33:0/996653050 >> v2:172.21.15.167:3300/0 conn(0x7f49480ccfc0 0x7f49480cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494e39d700 1 --2- 172.21.15.33:0/996653050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480cc750 0x7f49480b0460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.985+0000 7f494e39d700 1 -- 172.21.15.33:0/996653050 >> 172.21.15.33:0/996653050 conn(0x7f494806cff0 msgr2=0x7f494806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.986+0000 7f494e39d700 1 -- 172.21.15.33:0/996653050 shutdown_connections 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.986+0000 7f494e39d700 1 -- 172.21.15.33:0/996653050 wait complete. 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.987+0000 7f494e39d700 1 Processor -- start 2023-10-06T05:26:54.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.987+0000 7f494e39d700 1 -- start start 2023-10-06T05:26:54.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494e39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49480cc750 0x7f49480c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494e39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480ccfc0 0x7f49480b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494e39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 0x7f49480b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494e39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480b4a20 con 0x7f49480c9df0 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494e39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480b4cc0 con 0x7f49480cc750 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494e39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480b4fa0 con 0x7f49480ccfc0 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494db9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 0x7f49480b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494db9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 0x7f49480b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48086/0 (socket says 172.21.15.33:48086) 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494db9c700 1 -- 172.21.15.33:0/247828215 learned_addr learned my addr 172.21.15.33:0/247828215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494cb9a700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480ccfc0 0x7f49480b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.988+0000 7f494d39b700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49480cc750 0x7f49480c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494db9c700 1 -- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480ccfc0 msgr2=0x7f49480b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494db9c700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480ccfc0 0x7f49480b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494db9c700 1 -- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49480cc750 msgr2=0x7f49480c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494db9c700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49480cc750 0x7f49480c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494db9c700 1 -- 172.21.15.33:0/247828215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009420 con 0x7f49480c9df0 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494db9c700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 0x7f49480b0420 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f49480afa10 tx=0x7f4940019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940012e60 con 0x7f49480c9df0 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940027470 con 0x7f49480c9df0 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940027850 con 0x7f49480c9df0 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.989+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4940009a20 con 0x7f49480c9df0 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.990+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49480ca300 con 0x7f49480c9df0 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.990+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494804c380 con 0x7f49480c9df0 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.991+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4940024070 con 0x7f49480c9df0 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.992+0000 7f493d7fa700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f492c066f30 0x7f492c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.992+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4940086f00 con 0x7f49480c9df0 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.993+0000 7f494d39b700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f492c066f30 0x7f492c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.993+0000 7f494d39b700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f492c066f30 0x7f492c0693e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4944009f40 tx=0x7f49440091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:54.996+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4940053f40 con 0x7f49480c9df0 2023-10-06T05:26:55.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.199+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4948159bb0 con 0x7f49480c9df0 2023-10-06T05:26:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.200+0000 7f493d7fa700 1 -- 172.21.15.33:0/247828215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f494002c090 con 0x7f49480c9df0 2023-10-06T05:26:55.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:26:55.202 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":170,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:26:29.807750+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:14.643519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:15.441428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2408692474":"2023-10-06T06:26:29.245947+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:55.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f492c066f30 msgr2=0x7f492c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f492c066f30 0x7f492c0693e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4944009f40 tx=0x7f49440091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 msgr2=0x7f49480b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 0x7f49480b0420 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f49480afa10 tx=0x7f4940019000 comp rx=0 tx=0).stop 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 shutdown_connections 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f492c066f30 0x7f492c0693e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49480c9df0 0x7f49480b0420 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49480cc750 0x7f49480c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 --2- 172.21.15.33:0/247828215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49480ccfc0 0x7f49480b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 >> 172.21.15.33:0/247828215 conn(0x7f494806cff0 msgr2=0x7f4948156d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 shutdown_connections 2023-10-06T05:26:55.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.203+0000 7f494e39d700 1 -- 172.21.15.33:0/247828215 wait complete. 2023-10-06T05:26:55.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.409+0000 7fa980d9d700 1 Processor -- start 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.409+0000 7fa980d9d700 1 -- start start 2023-10-06T05:26:55.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.410+0000 7fa980d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa97c1557f0 0x7fa97c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.410+0000 7fa980d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa97c0c9220 0x7fa97c0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.410+0000 7fa980d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97c06a410 con 0x7fa97c155bc0 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.410+0000 7fa980d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c0cccd0 con 0x7fa97c1557f0 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.410+0000 7fa980d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c0c7500 con 0x7fa97c0c9220 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa97c1557f0 0x7fa97c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa97c1557f0 0x7fa97c0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48090/0 (socket says 172.21.15.33:48090) 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 -- 172.21.15.33:0/2456149939 learned_addr learned my addr 172.21.15.33:0/2456149939 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:55.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97bfff700 1 --2- 172.21.15.33:0/2456149939 >> v2:172.21.15.167:3300/0 conn(0x7fa97c0c9220 0x7fa97c0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 -- 172.21.15.33:0/2456149939 >> v2:172.21.15.167:3300/0 conn(0x7fa97c0c9220 msgr2=0x7fa97c0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 --2- 172.21.15.33:0/2456149939 >> v2:172.21.15.167:3300/0 conn(0x7fa97c0c9220 0x7fa97c0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 -- 172.21.15.33:0/2456149939 >> v1:172.21.15.167:6789/0 conn(0x7fa97c155bc0 legacy=0x7fa97c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 -- 172.21.15.33:0/2456149939 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0c77e0 con 0x7fa97c1557f0 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.411+0000 7fa97b7fe700 1 --2- 172.21.15.33:0/2456149939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c1557f0 0x7fa97c0c6960 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa97c06d280 tx=0x7fa96c00b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1812a1868167ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa97a7fc700 1 -- 172.21.15.33:0/2456149939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00da60 con 0x7fa97c1557f0 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa97a7fc700 1 -- 172.21.15.33:0/2456149939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c00dbc0 con 0x7fa97c1557f0 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa97a7fc700 1 -- 172.21.15.33:0/2456149939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c017410 con 0x7fa97c1557f0 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 -- 172.21.15.33:0/2456149939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c1557f0 msgr2=0x7fa97c0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 --2- 172.21.15.33:0/2456149939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c1557f0 0x7fa97c0c6960 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa97c06d280 tx=0x7fa96c00b4e0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 -- 172.21.15.33:0/2456149939 shutdown_connections 2023-10-06T05:26:55.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 --2- 172.21.15.33:0/2456149939 >> v2:172.21.15.167:3300/0 conn(0x7fa97c0c9220 0x7fa97c0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 --2- 172.21.15.33:0/2456149939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c1557f0 0x7fa97c0c6960 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 -- 172.21.15.33:0/2456149939 >> 172.21.15.33:0/2456149939 conn(0x7fa97c06c560 msgr2=0x7fa97c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 -- 172.21.15.33:0/2456149939 shutdown_connections 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.412+0000 7fa980d9d700 1 -- 172.21.15.33:0/2456149939 wait complete. 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.414+0000 7fa980d9d700 1 Processor -- start 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.414+0000 7fa980d9d700 1 -- start start 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa980d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 0x7fa97c14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa980d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa97c1557f0 0x7fa97c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa980d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa97c155bc0 0x7fa97c14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa980d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c152e80 con 0x7fa97c0c9220 2023-10-06T05:26:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa980d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c152270 con 0x7fa97c155bc0 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa980d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c152550 con 0x7fa97c1557f0 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa97b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 0x7fa97c14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa97b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 0x7fa97c14be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48098/0 (socket says 172.21.15.33:48098) 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.415+0000 7fa97b7fe700 1 -- 172.21.15.33:0/3186380996 learned_addr learned my addr 172.21.15.33:0/3186380996 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97affd700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa97c1557f0 0x7fa97c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97bfff700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa97c155bc0 0x7fa97c14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97b7fe700 1 -- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa97c1557f0 msgr2=0x7fa97c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97b7fe700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa97c1557f0 0x7fa97c14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97b7fe700 1 -- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa97c155bc0 msgr2=0x7fa97c14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97b7fe700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa97c155bc0 0x7fa97c14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97b7fe700 1 -- 172.21.15.33:0/3186380996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c00b780 con 0x7fa97c0c9220 2023-10-06T05:26:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa97b7fe700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 0x7fa97c14be50 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0636f0 tx=0x7fa96c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c022610 con 0x7fa97c0c9220 2023-10-06T05:26:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c022770 con 0x7fa97c0c9220 2023-10-06T05:26:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.416+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c022b10 con 0x7fa97c0c9220 2023-10-06T05:26:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.417+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c152830 con 0x7fa97c0c9220 2023-10-06T05:26:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.417+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c14c390 con 0x7fa97c0c9220 2023-10-06T05:26:55.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.418+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97c153160 con 0x7fa97c0c9220 2023-10-06T05:26:55.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.419+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa96c015070 con 0x7fa97c0c9220 2023-10-06T05:26:55.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.419+0000 7fa9637fe700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa968066f30 0x7fa9680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.419+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa96c088670 con 0x7fa97c0c9220 2023-10-06T05:26:55.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.420+0000 7fa97affd700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa968066f30 0x7fa9680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.421+0000 7fa97affd700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa968066f30 0x7fa9680693e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa97c154630 tx=0x7fa970009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.423+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa96c0556b0 con 0x7fa97c0c9220 2023-10-06T05:26:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.672+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa97c04c380 con 0x7fa97c0c9220 2023-10-06T05:26:55.677 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:26:55.673+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.678+0000 7fa9637fe700 1 -- 172.21.15.33:0/3186380996 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v125) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa96c006d50 con 0x7fa97c0c9220 2023-10-06T05:26:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.680+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa968066f30 msgr2=0x7fa9680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.680+0000 7fa980d9d700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa968066f30 0x7fa9680693e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa97c154630 tx=0x7fa970009040 comp rx=0 tx=0).stop 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.680+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 msgr2=0x7fa97c14be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.680+0000 7fa980d9d700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 0x7fa97c14be50 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0636f0 tx=0x7fa96c019000 comp rx=0 tx=0).stop 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 shutdown_connections 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa968066f30 0x7fa9680693e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa97c155bc0 0x7fa97c14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa97c0c9220 0x7fa97c14be50 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 --2- 172.21.15.33:0/3186380996 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa97c1557f0 0x7fa97c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 >> 172.21.15.33:0/3186380996 conn(0x7fa97c06c560 msgr2=0x7fa97c0cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 shutdown_connections 2023-10-06T05:26:55.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.681+0000 7fa980d9d700 1 -- 172.21.15.33:0/3186380996 wait complete. 2023-10-06T05:26:55.684 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:55.687 INFO:tasks.ceph.mds.a.smithi033.stderr: -4099> 2023-10-06T05:26:20.945+0000 7fcb3f1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:55.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:26:55.732 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:26:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.891+0000 7f898b59d700 1 Processor -- start 2023-10-06T05:26:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.891+0000 7f898b59d700 1 -- start start 2023-10-06T05:26:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898b59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8984159850 0x7f898406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898b59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f898415acd0 0x7f898406edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898b59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89840b0cd0 con 0x7f898406d140 2023-10-06T05:26:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898b59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898415a840 con 0x7f8984159850 2023-10-06T05:26:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898b59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8984159f10 con 0x7f898415acd0 2023-10-06T05:26:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898a59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8984159850 0x7f898406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898ad9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f898406d140 0x7f898406d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46294/0 (socket says 172.21.15.33:46294) 2023-10-06T05:26:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f898ad9c700 1 -- 172.21.15.33:0/2306773174 learned_addr learned my addr 172.21.15.33:0/2306773174 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.892+0000 7f8989d9a700 1 --2- 172.21.15.33:0/2306773174 >> v2:172.21.15.167:3300/0 conn(0x7f898415acd0 0x7f898406edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f898a59b700 1 -- 172.21.15.33:0/2306773174 >> v2:172.21.15.167:3300/0 conn(0x7f898415acd0 msgr2=0x7f898406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f898a59b700 1 --2- 172.21.15.33:0/2306773174 >> v2:172.21.15.167:3300/0 conn(0x7f898415acd0 0x7f898406edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f898a59b700 1 -- 172.21.15.33:0/2306773174 >> v1:172.21.15.33:6789/0 conn(0x7f898406d140 legacy=0x7f898406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f898a59b700 1 -- 172.21.15.33:0/2306773174 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898415a1f0 con 0x7f8984159850 2023-10-06T05:26:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f898a59b700 1 --2- 172.21.15.33:0/2306773174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898406e890 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f898406a730 tx=0x7f89800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f53384c63f859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f8989599700 1 -- 172.21.15.33:0/2306773174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980011630 con 0x7f8984159850 2023-10-06T05:26:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f8989599700 1 -- 172.21.15.33:0/2306773174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8980011790 con 0x7f8984159850 2023-10-06T05:26:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.893+0000 7f8989599700 1 -- 172.21.15.33:0/2306773174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980011b30 con 0x7f8984159850 2023-10-06T05:26:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 -- 172.21.15.33:0/2306773174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 msgr2=0x7f898406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 --2- 172.21.15.33:0/2306773174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898406e890 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f898406a730 tx=0x7f89800091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 -- 172.21.15.33:0/2306773174 shutdown_connections 2023-10-06T05:26:55.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 --2- 172.21.15.33:0/2306773174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898406e890 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 --2- 172.21.15.33:0/2306773174 >> v2:172.21.15.167:3300/0 conn(0x7f898415acd0 0x7f898406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 -- 172.21.15.33:0/2306773174 >> 172.21.15.33:0/2306773174 conn(0x7f898406cd40 msgr2=0x7f89840ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:55.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 -- 172.21.15.33:0/2306773174 shutdown_connections 2023-10-06T05:26:55.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.894+0000 7f898b59d700 1 -- 172.21.15.33:0/2306773174 wait complete. 2023-10-06T05:26:55.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.896+0000 7f898b59d700 1 Processor -- start 2023-10-06T05:26:55.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.896+0000 7f898b59d700 1 -- start start 2023-10-06T05:26:55.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.896+0000 7f898b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f898406d140 0x7f898414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898416da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898415acd0 0x7f89840c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89840cbc90 con 0x7f8984159850 2023-10-06T05:26:55.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898414e4a0 con 0x7f898406d140 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898416df70 con 0x7f898415acd0 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898416da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898416da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48122/0 (socket says 172.21.15.33:48122) 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 -- 172.21.15.33:0/3897846518 learned_addr learned my addr 172.21.15.33:0/3897846518 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898ad9c700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898415acd0 0x7f89840c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898a59b700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f898406d140 0x7f898414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 -- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898415acd0 msgr2=0x7f89840c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898415acd0 0x7f89840c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 -- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f898406d140 msgr2=0x7f898414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f898406d140 0x7f898414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f8989d9a700 1 -- 172.21.15.33:0/3897846518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980009420 con 0x7f8984159850 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898ad9c700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898415acd0 0x7f89840c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.897+0000 7f898a59b700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f898406d140 0x7f898414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:55.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.898+0000 7f8989d9a700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898416da30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f897400ecb0 tx=0x7f89740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.898+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400c9d0 con 0x7f8984159850 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.898+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897400cb30 con 0x7f8984159850 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.898+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400ced0 con 0x7f8984159850 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.898+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89840cb080 con 0x7f8984159850 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.898+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89840cb6c0 con 0x7f8984159850 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.899+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89840cbe40 con 0x7f8984159850 2023-10-06T05:26:55.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.900+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8974013680 con 0x7f8984159850 2023-10-06T05:26:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.901+0000 7f897a7fc700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8968066f30 0x7f89680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.901+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8974084b70 con 0x7f8984159850 2023-10-06T05:26:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.901+0000 7f898a59b700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8968066f30 0x7f89680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.902+0000 7f898a59b700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8968066f30 0x7f89680693e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8980003080 tx=0x7f8980017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:55.905+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89840cbe40 con 0x7f8984159850 2023-10-06T05:26:56.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.155+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f89840cbe40 con 0x7f8984159850 2023-10-06T05:26:56.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.679+0000 7f897a7fc700 1 -- 172.21.15.33:0/3897846518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v126) v1 ==== 104+0+0 (secure 0 0 0) 0x7f89840cbe40 con 0x7f8984159850 2023-10-06T05:26:56.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8968066f30 msgr2=0x7f89680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:56.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8968066f30 0x7f89680693e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8980003080 tx=0x7f8980017000 comp rx=0 tx=0).stop 2023-10-06T05:26:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 msgr2=0x7f898416da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898416da30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f897400ecb0 tx=0x7f89740091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 shutdown_connections 2023-10-06T05:26:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8968066f30 0x7f89680693e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898415acd0 0x7f89840c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f898406d140 0x7f898414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 --2- 172.21.15.33:0/3897846518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8984159850 0x7f898416da30 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 >> 172.21.15.33:0/3897846518 conn(0x7f898406cd40 msgr2=0x7f89841582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:56.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 shutdown_connections 2023-10-06T05:26:56.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.682+0000 7f898b59d700 1 -- 172.21.15.33:0/3897846518 wait complete. 2023-10-06T05:26:56.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:26:56.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.936+0000 7f1cb0f9d700 1 Processor -- start 2023-10-06T05:26:56.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.936+0000 7f1cb0f9d700 1 -- start start 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1cb0f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1cac1597f0 0x7f1cac15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1cb0f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dfd0 0x7f1cac15a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1cb0f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac15d280 con 0x7f1cac06dc00 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1cb0f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac15e290 con 0x7f1cac1597f0 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1cb0f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15de00 con 0x7f1cac06dfd0 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1caaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1cac1597f0 0x7f1cac15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1caaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1cac1597f0 0x7f1cac15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48126/0 (socket says 172.21.15.33:48126) 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1caaffd700 1 -- 172.21.15.33:0/125074069 learned_addr learned my addr 172.21.15.33:0/125074069 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.938+0000 7f1cabfff700 1 --2- 172.21.15.33:0/125074069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dfd0 0x7f1cac15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caaffd700 1 -- 172.21.15.33:0/125074069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dfd0 msgr2=0x7f1cac15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caaffd700 1 --2- 172.21.15.33:0/125074069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dfd0 0x7f1cac15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caa7fc700 1 -- 172.21.15.33:0/125074069 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303334229 0 0) 0x7f1cac15d280 con 0x7f1cac06dc00 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caaffd700 1 -- 172.21.15.33:0/125074069 >> v1:172.21.15.167:6789/0 conn(0x7f1cac06dc00 legacy=0x7f1cac1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caaffd700 1 -- 172.21.15.33:0/125074069 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac15d4d0 con 0x7f1cac1597f0 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caaffd700 1 --2- 172.21.15.33:0/125074069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac1597f0 0x7f1cac15a030 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00099b0 tx=0x7f1ca0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b1afade64484e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caa7fc700 1 -- 172.21.15.33:0/125074069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0009ea0 con 0x7f1cac1597f0 2023-10-06T05:26:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caa7fc700 1 -- 172.21.15.33:0/125074069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca000b860 con 0x7f1cac1597f0 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.939+0000 7f1caa7fc700 1 -- 172.21.15.33:0/125074069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000bc00 con 0x7f1cac1597f0 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/125074069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac1597f0 msgr2=0x7f1cac15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/125074069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac1597f0 0x7f1cac15a030 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00099b0 tx=0x7f1ca0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/125074069 shutdown_connections 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/125074069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dfd0 0x7f1cac15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/125074069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac1597f0 0x7f1cac15a030 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/125074069 >> 172.21.15.33:0/125074069 conn(0x7f1cac0d0550 msgr2=0x7f1cac0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/125074069 shutdown_connections 2023-10-06T05:26:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.940+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/125074069 wait complete. 2023-10-06T05:26:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.941+0000 7f1cb0f9d700 1 Processor -- start 2023-10-06T05:26:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.941+0000 7f1cb0f9d700 1 -- start start 2023-10-06T05:26:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.942+0000 7f1cb0f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dc00 0x7f1cac0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.942+0000 7f1cb0f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 0x7f1cac16d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.942+0000 7f1cb0f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cac1597f0 0x7f1cac16d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.942+0000 7f1cb0f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15c350 con 0x7f1cac06dfd0 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.942+0000 7f1cb0f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15b780 con 0x7f1cac1597f0 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.942+0000 7f1cb0f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15ba20 con 0x7f1cac06dc00 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 0x7f1cac16d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 0x7f1cac16d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48128/0 (socket says 172.21.15.33:48128) 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 -- 172.21.15.33:0/3112527362 learned_addr learned my addr 172.21.15.33:0/3112527362 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1cab7fe700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dc00 0x7f1cac0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1cabfff700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cac1597f0 0x7f1cac16d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 -- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dc00 msgr2=0x7f1cac0c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dc00 0x7f1cac0c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 -- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cac1597f0 msgr2=0x7f1cac16d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cac1597f0 0x7f1cac16d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 -- 172.21.15.33:0/3112527362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00094f0 con 0x7f1cac06dfd0 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.943+0000 7f1caaffd700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 0x7f1cac16d1a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0894b0 tx=0x7f1ca0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.944+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0000d10 con 0x7f1cac06dfd0 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.944+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca000bbe0 con 0x7f1cac06dfd0 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.944+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca001a6c0 con 0x7f1cac06dfd0 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.944+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac15bd00 con 0x7f1cac06dfd0 2023-10-06T05:26:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.944+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac0c9360 con 0x7f1cac06dfd0 2023-10-06T05:26:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.945+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac15c4c0 con 0x7f1cac06dfd0 2023-10-06T05:26:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.946+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ca001aab0 con 0x7f1cac06dfd0 2023-10-06T05:26:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.946+0000 7f1c937fe700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.947+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1ca0086090 con 0x7f1cac06dfd0 2023-10-06T05:26:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.947+0000 7f1cab7fe700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.948+0000 7f1cab7fe700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c98066f30 0x7f1c980693e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c007fd0 tx=0x7f1c9c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:56.951+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ca0053080 con 0x7f1cac06dfd0 2023-10-06T05:26:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.157+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1cac063ff0 con 0x7f1cac06dfd0 2023-10-06T05:26:57.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.899+0000 7f1c937fe700 1 -- 172.21.15.33:0/3112527362 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1ca00566a0 con 0x7f1cac06dfd0 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c98066f30 msgr2=0x7f1c980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c98066f30 0x7f1c980693e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c007fd0 tx=0x7f1c9c007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 msgr2=0x7f1cac16d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 0x7f1cac16d1a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0894b0 tx=0x7f1ca0017000 comp rx=0 tx=0).stop 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 shutdown_connections 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cac1597f0 0x7f1cac16d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cac06dc00 0x7f1cac0c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 --2- 172.21.15.33:0/3112527362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cac06dfd0 0x7f1cac16d1a0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 >> 172.21.15.33:0/3112527362 conn(0x7f1cac0d0550 msgr2=0x7f1cac0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 shutdown_connections 2023-10-06T05:26:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:57.903+0000 7f1cb0f9d700 1 -- 172.21.15.33:0/3112527362 wait complete. 2023-10-06T05:26:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:26:57.927 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:26:58.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.122+0000 7f4a87fff700 1 Processor -- start 2023-10-06T05:26:58.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.122+0000 7f4a87fff700 1 -- start start 2023-10-06T05:26:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.123+0000 7f4a87fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a88159400 0x7f4a880a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.123+0000 7f4a87fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4a8806dbf0 0x7f4a88067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.123+0000 7f4a87fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a880a9dd0 con 0x7f4a881597d0 2023-10-06T05:26:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.123+0000 7f4a87fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8806d760 con 0x7f4a88159400 2023-10-06T05:26:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.123+0000 7f4a87fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a880aa8c0 con 0x7f4a8806dbf0 2023-10-06T05:26:58.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a877fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4a881597d0 0x7f4a8806a260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46296/0 (socket says 172.21.15.33:46296) 2023-10-06T05:26:58.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a877fe700 1 -- 172.21.15.33:0/3746796625 learned_addr learned my addr 172.21.15.33:0/3746796625 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:58.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a86ffd700 1 --2- 172.21.15.33:0/3746796625 >> v2:172.21.15.33:3300/0 conn(0x7f4a88159400 0x7f4a880a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a867fc700 1 --2- 172.21.15.33:0/3746796625 >> v2:172.21.15.167:3300/0 conn(0x7f4a8806dbf0 0x7f4a88067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a85ffb700 1 -- 172.21.15.33:0/3746796625 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048778561 0 0) 0x7f4a80001490 con 0x7f4a881597d0 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a86ffd700 1 -- 172.21.15.33:0/3746796625 >> v2:172.21.15.167:3300/0 conn(0x7f4a8806dbf0 msgr2=0x7f4a88067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a86ffd700 1 --2- 172.21.15.33:0/3746796625 >> v2:172.21.15.167:3300/0 conn(0x7f4a8806dbf0 0x7f4a88067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a86ffd700 1 -- 172.21.15.33:0/3746796625 >> v1:172.21.15.33:6789/0 conn(0x7f4a881597d0 legacy=0x7f4a8806a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a86ffd700 1 -- 172.21.15.33:0/3746796625 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880aa5e0 con 0x7f4a88159400 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.124+0000 7f4a86ffd700 1 --2- 172.21.15.33:0/3746796625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a880a9890 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4a780098e0 tx=0x7f4a780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258f7b154775fce5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a85ffb700 1 -- 172.21.15.33:0/3746796625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78009e80 con 0x7f4a88159400 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a85ffb700 1 -- 172.21.15.33:0/3746796625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7800b820 con 0x7f4a88159400 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a85ffb700 1 -- 172.21.15.33:0/3746796625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800bba0 con 0x7f4a88159400 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 -- 172.21.15.33:0/3746796625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 msgr2=0x7f4a880a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 --2- 172.21.15.33:0/3746796625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a880a9890 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4a780098e0 tx=0x7f4a780091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 -- 172.21.15.33:0/3746796625 shutdown_connections 2023-10-06T05:26:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 --2- 172.21.15.33:0/3746796625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a880a9890 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 --2- 172.21.15.33:0/3746796625 >> v2:172.21.15.167:3300/0 conn(0x7f4a8806dbf0 0x7f4a88067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 -- 172.21.15.33:0/3746796625 >> 172.21.15.33:0/3746796625 conn(0x7f4a8806ebf0 msgr2=0x7f4a8806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 -- 172.21.15.33:0/3746796625 shutdown_connections 2023-10-06T05:26:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.125+0000 7f4a87fff700 1 -- 172.21.15.33:0/3746796625 wait complete. 2023-10-06T05:26:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.127+0000 7f4a87fff700 1 Processor -- start 2023-10-06T05:26:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.127+0000 7f4a87fff700 1 -- start start 2023-10-06T05:26:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a87fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8806dbf0 0x7f4a88142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a88169670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a87fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a881597d0 0x7f4a8815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a87fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8815e350 con 0x7f4a88159400 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a87fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88152be0 con 0x7f4a8806dbf0 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a87fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88169bb0 con 0x7f4a881597d0 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a88169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a88169670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48146/0 (socket says 172.21.15.33:48146) 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a867fc700 1 -- 172.21.15.33:0/1547754918 learned_addr learned my addr 172.21.15.33:0/1547754918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a86ffd700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8806dbf0 0x7f4a88142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.128+0000 7f4a877fe700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a881597d0 0x7f4a8815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a867fc700 1 -- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a881597d0 msgr2=0x7f4a8815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a867fc700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a881597d0 0x7f4a8815d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a867fc700 1 -- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8806dbf0 msgr2=0x7f4a88142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a867fc700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8806dbf0 0x7f4a88142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a867fc700 1 -- 172.21.15.33:0/1547754918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78009420 con 0x7f4a88159400 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a867fc700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a88169670 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00d860 tx=0x7f4a7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0125f0 con 0x7f4a88159400 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c012770 con 0x7f4a88159400 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c012b30 con 0x7f4a88159400 2023-10-06T05:26:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8815e070 con 0x7f4a88159400 2023-10-06T05:26:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.129+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a88152870 con 0x7f4a88159400 2023-10-06T05:26:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.130+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a881523f0 con 0x7f4a88159400 2023-10-06T05:26:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.131+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a7c013480 con 0x7f4a88159400 2023-10-06T05:26:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.132+0000 7f4a6effd700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a74066f30 0x7f4a740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.132+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f4a7c084810 con 0x7f4a88159400 2023-10-06T05:26:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.133+0000 7f4a86ffd700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a74066f30 0x7f4a740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.133+0000 7f4a86ffd700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a74066f30 0x7f4a740693e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4a780093f0 tx=0x7f4a78017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.136+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a881523f0 con 0x7f4a88159400 2023-10-06T05:26:58.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.338+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a88152d90 con 0x7f4a88159400 2023-10-06T05:26:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.913+0000 7f4a6effd700 1 -- 172.21.15.33:0/1547754918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4a88152d90 con 0x7f4a88159400 2023-10-06T05:26:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a74066f30 msgr2=0x7f4a740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a74066f30 0x7f4a740693e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4a780093f0 tx=0x7f4a78017000 comp rx=0 tx=0).stop 2023-10-06T05:26:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 msgr2=0x7f4a88169670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a88169670 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00d860 tx=0x7f4a7c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 shutdown_connections 2023-10-06T05:26:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a74066f30 0x7f4a740693e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a881597d0 0x7f4a8815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a8806dbf0 0x7f4a88142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 --2- 172.21.15.33:0/1547754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a88159400 0x7f4a88169670 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 >> 172.21.15.33:0/1547754918 conn(0x7f4a8806ebf0 msgr2=0x7f4a8806efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 shutdown_connections 2023-10-06T05:26:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:58.917+0000 7f4a87fff700 1 -- 172.21.15.33:0/1547754918 wait complete. 2023-10-06T05:26:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:26:58.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:26:59.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.130+0000 7f0c08f9d700 1 Processor -- start 2023-10-06T05:26:59.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.131+0000 7f0c08f9d700 1 -- start start 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c08f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c040cc7e0 0x7f0c040af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c08f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040b01f0 0x7f0c040b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c08f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c040ad430 con 0x7f0c04156370 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c08f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040afe20 con 0x7f0c040cc7e0 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c08f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040b0b70 con 0x7f0c040b01f0 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c02ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c040cc7e0 0x7f0c040af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c037fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c04156370 0x7f0c040c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46312/0 (socket says 172.21.15.33:46312) 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c037fe700 1 -- 172.21.15.33:0/2167937907 learned_addr learned my addr 172.21.15.33:0/2167937907 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.132+0000 7f0c03fff700 1 --2- 172.21.15.33:0/2167937907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040b01f0 0x7f0c040b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c02ffd700 1 -- 172.21.15.33:0/2167937907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040b01f0 msgr2=0x7f0c040b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c02ffd700 1 --2- 172.21.15.33:0/2167937907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040b01f0 0x7f0c040b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c02ffd700 1 -- 172.21.15.33:0/2167937907 >> v1:172.21.15.33:6789/0 conn(0x7f0c04156370 legacy=0x7f0c040c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c02ffd700 1 -- 172.21.15.33:0/2167937907 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0406bc70 con 0x7f0c040cc7e0 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c02ffd700 1 --2- 172.21.15.33:0/2167937907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c040cc7e0 0x7f0c040af8e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80098e0 tx=0x7f0bf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e53fc4bde638b62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c027fc700 1 -- 172.21.15.33:0/2167937907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8009e80 con 0x7f0c040cc7e0 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c027fc700 1 -- 172.21.15.33:0/2167937907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf800b860 con 0x7f0c040cc7e0 2023-10-06T05:26:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.133+0000 7f0c027fc700 1 -- 172.21.15.33:0/2167937907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800bbe0 con 0x7f0c040cc7e0 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 -- 172.21.15.33:0/2167937907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c040cc7e0 msgr2=0x7f0c040af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/2167937907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c040cc7e0 0x7f0c040af8e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80098e0 tx=0x7f0bf80091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 -- 172.21.15.33:0/2167937907 shutdown_connections 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/2167937907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040b01f0 0x7f0c040b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/2167937907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c040cc7e0 0x7f0c040af8e0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 -- 172.21.15.33:0/2167937907 >> 172.21.15.33:0/2167937907 conn(0x7f0c0406d030 msgr2=0x7f0c040cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 -- 172.21.15.33:0/2167937907 shutdown_connections 2023-10-06T05:26:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.134+0000 7f0c08f9d700 1 -- 172.21.15.33:0/2167937907 wait complete. 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.135+0000 7f0c08f9d700 1 Processor -- start 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.135+0000 7f0c08f9d700 1 -- start start 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.136+0000 7f0c08f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c040b01f0 0x7f0c040caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.136+0000 7f0c08f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040cc7e0 0x7f0c04157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.136+0000 7f0c08f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 0x7f0c04156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.136+0000 7f0c08f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c041584a0 con 0x7f0c04156370 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.136+0000 7f0c08f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04158740 con 0x7f0c040b01f0 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.136+0000 7f0c08f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04158a20 con 0x7f0c040cc7e0 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 0x7f0c04156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 0x7f0c04156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48152/0 (socket says 172.21.15.33:48152) 2023-10-06T05:26:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 -- 172.21.15.33:0/3115451200 learned_addr learned my addr 172.21.15.33:0/3115451200 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c02ffd700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040cc7e0 0x7f0c04157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c037fe700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c040b01f0 0x7f0c040caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 -- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040cc7e0 msgr2=0x7f0c04157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040cc7e0 0x7f0c04157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 -- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c040b01f0 msgr2=0x7f0c040caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c040b01f0 0x7f0c040caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 -- 172.21.15.33:0/3115451200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf8009420 con 0x7f0c04156370 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c037fe700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c040b01f0 0x7f0c040caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c03fff700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 0x7f0c04156740 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc007f20 tx=0x7f0bfc009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00eca0 con 0x7f0c04156370 2023-10-06T05:26:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc00ee20 con 0x7f0c04156370 2023-10-06T05:26:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc014670 con 0x7f0c04156370 2023-10-06T05:26:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.137+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c04158d00 con 0x7f0c04156370 2023-10-06T05:26:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.138+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c040a6140 con 0x7f0c04156370 2023-10-06T05:26:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.139+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0404c380 con 0x7f0c04156370 2023-10-06T05:26:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.140+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0bfc0149e0 con 0x7f0c04156370 2023-10-06T05:26:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.140+0000 7f0beb7fe700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0bf0066f30 0x7f0bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.140+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0bfc084b90 con 0x7f0c04156370 2023-10-06T05:26:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.141+0000 7f0c037fe700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0bf0066f30 0x7f0bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.142+0000 7f0c037fe700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0bf0066f30 0x7f0bf00693e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4001820 tx=0x7f0bf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.145+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bfc051ed0 con 0x7f0c04156370 2023-10-06T05:26:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.346+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0c0406f310 con 0x7f0c04156370 2023-10-06T05:26:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.350+0000 7f0beb7fe700 1 -- 172.21.15.33:0/3115451200 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0bfc0554f0 con 0x7f0c04156370 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0bf0066f30 msgr2=0x7f0bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0bf0066f30 0x7f0bf00693e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4001820 tx=0x7f0bf4009040 comp rx=0 tx=0).stop 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 msgr2=0x7f0c04156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 0x7f0c04156740 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc007f20 tx=0x7f0bfc009860 comp rx=0 tx=0).stop 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 shutdown_connections 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0bf0066f30 0x7f0bf00693e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c04156370 0x7f0c04156740 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c040b01f0 0x7f0c040caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 --2- 172.21.15.33:0/3115451200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c040cc7e0 0x7f0c04157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 >> 172.21.15.33:0/3115451200 conn(0x7f0c0406d030 msgr2=0x7f0c040ac540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 shutdown_connections 2023-10-06T05:26:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.353+0000 7f0c08f9d700 1 -- 172.21.15.33:0/3115451200 wait complete. 2023-10-06T05:26:59.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_after_blocklisted' 2023-10-06T05:26:59.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.558+0000 7f25af59e700 1 Processor -- start 2023-10-06T05:26:59.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.558+0000 7f25af59e700 1 -- start start 2023-10-06T05:26:59.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.559+0000 7f25af59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25b00c8050 0x7f25b013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.559+0000 7f25af59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 0x7f25b006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.559+0000 7f25af59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00c9cb0 con 0x7f25b013faa0 2023-10-06T05:26:59.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.559+0000 7f25af59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00ca270 con 0x7f25b00c8050 2023-10-06T05:26:59.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.559+0000 7f25af59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b006b840 con 0x7f25b006aad0 2023-10-06T05:26:59.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ae59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f25b013faa0 0x7f25b00c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46320/0 (socket says 172.21.15.33:46320) 2023-10-06T05:26:59.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ae59c700 1 -- 172.21.15.33:0/4281253528 learned_addr learned my addr 172.21.15.33:0/4281253528 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25aed9d700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 0x7f25b006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25add9b700 1 --2- 172.21.15.33:0/4281253528 >> v2:172.21.15.167:3300/0 conn(0x7f25b00c8050 0x7f25b013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941361320 0 0) 0x7f25b00c9cb0 con 0x7f25b013faa0 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2598003570 con 0x7f25b013faa0 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786905691 0 0) 0x7f2598003570 con 0x7f25b013faa0 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 msgr2=0x7f25b006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 0x7f25b006af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 >> v2:172.21.15.167:3300/0 conn(0x7f25b00c8050 msgr2=0x7f25b013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 --2- 172.21.15.33:0/4281253528 >> v2:172.21.15.167:3300/0 conn(0x7f25b00c8050 0x7f25b013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b006bb20 con 0x7f25b013faa0 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.560+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f25a0001470 con 0x7f25b013faa0 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 >> v1:172.21.15.33:6789/0 conn(0x7f25b013faa0 legacy=0x7f25b00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 0x7f2598003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598003fe0 0x7f2598004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ca270 con 0x7f2598004d50 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b006b840 con 0x7f2598003fe0 2023-10-06T05:26:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259800dca0 con 0x7f25b006aad0 2023-10-06T05:26:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25ae59c700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 0x7f2598003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25aed9d700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25add9b700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598003fe0 0x7f2598004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25aed9d700 1 -- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 msgr2=0x7f2598003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25aed9d700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b006aad0 0x7f2598003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25aed9d700 1 -- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598003fe0 msgr2=0x7f2598004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25aed9d700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598003fe0 0x7f2598004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.561+0000 7f25aed9d700 1 -- 172.21.15.33:0/4281253528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598003570 con 0x7f2598004d50 2023-10-06T05:26:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25aed9d700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25980051c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f25a800e210 tx=0x7f25a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c4df6ea140e4fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800ea60 con 0x7f2598004d50 2023-10-06T05:26:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a800f040 con 0x7f2598004d50 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25ad59a700 1 -- 172.21.15.33:0/4281253528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800ee10 con 0x7f2598004d50 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25af59e700 1 -- 172.21.15.33:0/4281253528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 msgr2=0x7f25980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25af59e700 1 --2- 172.21.15.33:0/4281253528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25980051c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f25a800e210 tx=0x7f25a800b040 comp rx=0 tx=0).stop 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25aed9d700 1 -- 172.21.15.33:0/4281253528 reap_dead start 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25af59e700 1 -- 172.21.15.33:0/4281253528 shutdown_connections 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.562+0000 7f25af59e700 1 -- 172.21.15.33:0/4281253528 >> 172.21.15.33:0/4281253528 conn(0x7f25b006d030 msgr2=0x7f25b006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.563+0000 7f25af59e700 1 -- 172.21.15.33:0/4281253528 shutdown_connections 2023-10-06T05:26:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.563+0000 7f25af59e700 1 -- 172.21.15.33:0/4281253528 wait complete. 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.564+0000 7f25af59e700 1 Processor -- start 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.564+0000 7f25af59e700 1 -- start start 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25af59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25b00c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25af59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25b013e780 0x7f25b0165480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25af59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b013ebf0 0x7f25b00c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25af59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b0158980 con 0x7f2598004d50 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25af59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b0157d70 con 0x7f25b013e780 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25af59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b0158050 con 0x7f25b013ebf0 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25b00c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25b00c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48168/0 (socket says 172.21.15.33:48168) 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25ae59c700 1 -- 172.21.15.33:0/1744535014 learned_addr learned my addr 172.21.15.33:0/1744535014 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25add9b700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25b013e780 0x7f25b0165480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.565+0000 7f25aed9d700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b013ebf0 0x7f25b00c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25ae59c700 1 -- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b013ebf0 msgr2=0x7f25b00c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25ae59c700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b013ebf0 0x7f25b00c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25ae59c700 1 -- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25b013e780 msgr2=0x7f25b0165480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25ae59c700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25b013e780 0x7f25b0165480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25ae59c700 1 -- 172.21.15.33:0/1744535014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0158330 con 0x7f2598004d50 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25ae59c700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25b00c3630 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f25a0007370 tx=0x7f25b006be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0018030 con 0x7f2598004d50 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000f9a0 con 0x7f2598004d50 2023-10-06T05:26:59.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000fd40 con 0x7f2598004d50 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b0158610 con 0x7f2598004d50 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.566+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b00a55f0 con 0x7f2598004d50 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.567+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b004c380 con 0x7f2598004d50 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.568+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25a001b070 con 0x7f2598004d50 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.569+0000 7f259e7fc700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2590066f30 0x7f25900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.569+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f25a0019070 con 0x7f2598004d50 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.570+0000 7f25add9b700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2590066f30 0x7f25900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.570+0000 7f25add9b700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2590066f30 0x7f25900693e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f25b01689c0 tx=0x7f25a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.573+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25a0051840 con 0x7f2598004d50 2023-10-06T05:26:59.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.859+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_after_blocklisted"]} v 0) v1 -- 0x7f25b00c9f00 con 0x7f2598004d50 2023-10-06T05:26:59.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.913+0000 7f259e7fc700 1 -- 172.21.15.33:0/1744535014 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_after_blocklisted"]}]=0 v635) v1 ==== 164+0+0 (secure 0 0 0) 0x7f25a000fea0 con 0x7f2598004d50 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2590066f30 msgr2=0x7f25900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2590066f30 0x7f25900693e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f25b01689c0 tx=0x7f25a4008040 comp rx=0 tx=0).stop 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 msgr2=0x7f25b00c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25b00c3630 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f25a0007370 tx=0x7f25b006be00 comp rx=0 tx=0).stop 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 shutdown_connections 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2590066f30 0x7f25900693e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25b013ebf0 0x7f25b00c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2598004d50 0x7f25b00c3630 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 --2- 172.21.15.33:0/1744535014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25b013e780 0x7f25b0165480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 >> 172.21.15.33:0/1744535014 conn(0x7f25b006d030 msgr2=0x7f25b006e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 shutdown_connections 2023-10-06T05:26:59.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:26:59.915+0000 7f25af59e700 1 -- 172.21.15.33:0/1744535014 wait complete. 2023-10-06T05:26:59.934 INFO:tasks.cephfs_test_runner:Test reconnect after blocklisted. ... ok 2023-10-06T05:26:59.934 INFO:tasks.cephfs_test_runner:Starting test: test_reconnect_eviction (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:26:59.934 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_eviction' 2023-10-06T05:27:00.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.121+0000 7ff92939d700 1 Processor -- start 2023-10-06T05:27:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.121+0000 7ff92939d700 1 -- start start 2023-10-06T05:27:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff92939d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff9240ccfc0 0x7ff9240cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff92939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9240cf3b0 0x7ff9241594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff92939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9240cc750 0x7ff9241599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff92939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240cbd80 con 0x7ff9240cf3b0 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff92939d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240af5f0 con 0x7ff9240ccfc0 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff92939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240af8d0 con 0x7ff9240cc750 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.122+0000 7ff9237fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9240cf3b0 0x7ff9241594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff9240cf3b0 0x7ff9241594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39882/0 (socket says 172.21.15.33:39882) 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 -- 172.21.15.33:0/949726652 learned_addr learned my addr 172.21.15.33:0/949726652 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff923fff700 1 --2- 172.21.15.33:0/949726652 >> v2:172.21.15.167:3300/0 conn(0x7ff9240ccfc0 0x7ff9240cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff928b9c700 1 --2- 172.21.15.33:0/949726652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9240cc750 0x7ff9241599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 -- 172.21.15.33:0/949726652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9240cc750 msgr2=0x7ff9241599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 --2- 172.21.15.33:0/949726652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9240cc750 0x7ff9241599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 -- 172.21.15.33:0/949726652 >> v2:172.21.15.167:3300/0 conn(0x7ff9240ccfc0 msgr2=0x7ff9240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 --2- 172.21.15.33:0/949726652 >> v2:172.21.15.167:3300/0 conn(0x7ff9240ccfc0 0x7ff9240cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 -- 172.21.15.33:0/949726652 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240afbb0 con 0x7ff9240cf3b0 2023-10-06T05:27:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff9237fe700 1 --2- 172.21.15.33:0/949726652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240cf3b0 0x7ff9241594a0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff9180099b0 tx=0x7ff918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa25101125f213a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.123+0000 7ff922ffd700 1 -- 172.21.15.33:0/949726652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800b820 con 0x7ff9240cf3b0 2023-10-06T05:27:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff922ffd700 1 -- 172.21.15.33:0/949726652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91800b980 con 0x7ff9240cf3b0 2023-10-06T05:27:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff922ffd700 1 -- 172.21.15.33:0/949726652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800bd20 con 0x7ff9240cf3b0 2023-10-06T05:27:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 -- 172.21.15.33:0/949726652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240cf3b0 msgr2=0x7ff9241594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 --2- 172.21.15.33:0/949726652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240cf3b0 0x7ff9241594a0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff9180099b0 tx=0x7ff918009250 comp rx=0 tx=0).stop 2023-10-06T05:27:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 -- 172.21.15.33:0/949726652 shutdown_connections 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 --2- 172.21.15.33:0/949726652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9240cc750 0x7ff9241599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 --2- 172.21.15.33:0/949726652 >> v2:172.21.15.167:3300/0 conn(0x7ff9240ccfc0 0x7ff9240cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 --2- 172.21.15.33:0/949726652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240cf3b0 0x7ff9241594a0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 -- 172.21.15.33:0/949726652 >> 172.21.15.33:0/949726652 conn(0x7ff92406cff0 msgr2=0x7ff92406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 -- 172.21.15.33:0/949726652 shutdown_connections 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.124+0000 7ff92939d700 1 -- 172.21.15.33:0/949726652 wait complete. 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.126+0000 7ff92939d700 1 Processor -- start 2023-10-06T05:27:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.126+0000 7ff92939d700 1 -- start start 2023-10-06T05:27:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff92939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9240cc750 0x7ff92416acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff92939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 0x7ff9240c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff92939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff924154700 0x7ff924154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff92939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240b32a0 con 0x7ff9240ccfc0 2023-10-06T05:27:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff92939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240b34b0 con 0x7ff9240cc750 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff92939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240b3790 con 0x7ff924154700 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 0x7ff9240c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 0x7ff9240c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39894/0 (socket says 172.21.15.33:39894) 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 -- 172.21.15.33:0/651107787 learned_addr learned my addr 172.21.15.33:0/651107787 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff923fff700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9240cc750 0x7ff92416acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff928b9c700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff924154700 0x7ff924154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 -- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff924154700 msgr2=0x7ff924154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff924154700 0x7ff924154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 -- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9240cc750 msgr2=0x7ff92416acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9240cc750 0x7ff92416acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff9237fe700 1 -- 172.21.15.33:0/651107787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180094f0 con 0x7ff9240ccfc0 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.127+0000 7ff923fff700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9240cc750 0x7ff92416acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.128+0000 7ff9237fe700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 0x7ff9240c90e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff9180094c0 tx=0x7ff918017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.128+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91801a450 con 0x7ff9240ccfc0 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.128+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91801a5b0 con 0x7ff9240ccfc0 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.128+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91801a970 con 0x7ff9240ccfc0 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.128+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9240b3a70 con 0x7ff9240ccfc0 2023-10-06T05:27:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.128+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9241550e0 con 0x7ff9240ccfc0 2023-10-06T05:27:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.129+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9240b3f90 con 0x7ff9240ccfc0 2023-10-06T05:27:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.130+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff918025440 con 0x7ff9240ccfc0 2023-10-06T05:27:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.131+0000 7ff90bfff700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff910066f30 0x7ff9100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.131+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff918085fd0 con 0x7ff9240ccfc0 2023-10-06T05:27:00.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.131+0000 7ff923fff700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff910066f30 0x7ff9100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.132+0000 7ff923fff700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff910066f30 0x7ff9100693e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff914006fd0 tx=0x7ff914008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.135+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9240b3f90 con 0x7ff9240ccfc0 2023-10-06T05:27:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.424+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_eviction"]} v 0) v1 -- 0x7ff9240b3f90 con 0x7ff9240ccfc0 2023-10-06T05:27:00.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.919+0000 7ff90bfff700 1 -- 172.21.15.33:0/651107787 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_eviction"]}]=0 v636) v1 ==== 158+0+0 (secure 0 0 0) 0x7ff9240b3f90 con 0x7ff9240ccfc0 2023-10-06T05:27:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff910066f30 msgr2=0x7ff9100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff910066f30 0x7ff9100693e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff914006fd0 tx=0x7ff914008040 comp rx=0 tx=0).stop 2023-10-06T05:27:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 msgr2=0x7ff9240c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 0x7ff9240c90e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff9180094c0 tx=0x7ff918017000 comp rx=0 tx=0).stop 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 shutdown_connections 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff910066f30 0x7ff9100693e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff924154700 0x7ff924154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9240cc750 0x7ff92416acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 --2- 172.21.15.33:0/651107787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9240ccfc0 0x7ff9240c90e0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 >> 172.21.15.33:0/651107787 conn(0x7ff92406cff0 msgr2=0x7ff9240c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 shutdown_connections 2023-10-06T05:27:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:00.922+0000 7ff92939d700 1 -- 172.21.15.33:0/651107787 wait complete. 2023-10-06T05:27:00.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:27:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.133+0000 7f6f6bfff700 1 Processor -- start 2023-10-06T05:27:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.133+0000 7f6f6bfff700 1 -- start start 2023-10-06T05:27:01.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.134+0000 7f6f6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f6c0af540 0x7f6f6c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.134+0000 7f6f6bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f6c0cd5e0 0x7f6f6c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.134+0000 7f6f6bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c15d660 con 0x7f6f6c15a650 2023-10-06T05:27:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.134+0000 7f6f6bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c0ce890 con 0x7f6f6c0af540 2023-10-06T05:27:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.134+0000 7f6f6bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c15aac0 con 0x7f6f6c0cd5e0 2023-10-06T05:27:01.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6a7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6f6c15a650 0x7f6f6c0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34054/0 (socket says 172.21.15.33:34054) 2023-10-06T05:27:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6a7fc700 1 -- 172.21.15.33:0/975446302 learned_addr learned my addr 172.21.15.33:0/975446302 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6affd700 1 --2- 172.21.15.33:0/975446302 >> v2:172.21.15.33:3300/0 conn(0x7f6f6c0af540 0x7f6f6c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6b7fe700 1 --2- 172.21.15.33:0/975446302 >> v2:172.21.15.167:3300/0 conn(0x7f6f6c0cd5e0 0x7f6f6c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f69ffb700 1 -- 172.21.15.33:0/975446302 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414111258 0 0) 0x7f6f6c15d660 con 0x7f6f6c15a650 2023-10-06T05:27:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f69ffb700 1 -- 172.21.15.33:0/975446302 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f54003570 con 0x7f6f6c15a650 2023-10-06T05:27:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6affd700 1 -- 172.21.15.33:0/975446302 >> v2:172.21.15.167:3300/0 conn(0x7f6f6c0cd5e0 msgr2=0x7f6f6c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6affd700 1 --2- 172.21.15.33:0/975446302 >> v2:172.21.15.167:3300/0 conn(0x7f6f6c0cd5e0 0x7f6f6c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6affd700 1 -- 172.21.15.33:0/975446302 >> v1:172.21.15.33:6789/0 conn(0x7f6f6c15a650 legacy=0x7f6f6c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6affd700 1 -- 172.21.15.33:0/975446302 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c15d890 con 0x7f6f6c0af540 2023-10-06T05:27:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.135+0000 7f6f6b7fe700 1 --2- 172.21.15.33:0/975446302 >> v2:172.21.15.167:3300/0 conn(0x7f6f6c0cd5e0 0x7f6f6c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6affd700 1 --2- 172.21.15.33:0/975446302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c0af910 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0ce1c0 tx=0x7f6f5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660ea2c1fd860302 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f69ffb700 1 -- 172.21.15.33:0/975446302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c011630 con 0x7f6f6c0af540 2023-10-06T05:27:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f69ffb700 1 -- 172.21.15.33:0/975446302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c011790 con 0x7f6f6c0af540 2023-10-06T05:27:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f69ffb700 1 -- 172.21.15.33:0/975446302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c011b10 con 0x7f6f6c0af540 2023-10-06T05:27:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 -- 172.21.15.33:0/975446302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 msgr2=0x7f6f6c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/975446302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c0af910 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0ce1c0 tx=0x7f6f5c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 -- 172.21.15.33:0/975446302 shutdown_connections 2023-10-06T05:27:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/975446302 >> v2:172.21.15.167:3300/0 conn(0x7f6f6c0cd5e0 0x7f6f6c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/975446302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c0af910 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 -- 172.21.15.33:0/975446302 >> 172.21.15.33:0/975446302 conn(0x7f6f6c06d9b0 msgr2=0x7f6f6c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 -- 172.21.15.33:0/975446302 shutdown_connections 2023-10-06T05:27:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.136+0000 7f6f6bfff700 1 -- 172.21.15.33:0/975446302 wait complete. 2023-10-06T05:27:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.138+0000 7f6f6bfff700 1 Processor -- start 2023-10-06T05:27:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.138+0000 7f6f6bfff700 1 -- start start 2023-10-06T05:27:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c14a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f6c0cd5e0 0x7f6f6c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6c15a650 0x7f6f6c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c16a4f0 con 0x7f6f6c0af540 2023-10-06T05:27:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c169bc0 con 0x7f6f6c0cd5e0 2023-10-06T05:27:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c169ea0 con 0x7f6f6c15a650 2023-10-06T05:27:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c14a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c14a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39904/0 (socket says 172.21.15.33:39904) 2023-10-06T05:27:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6affd700 1 -- 172.21.15.33:0/2088607922 learned_addr learned my addr 172.21.15.33:0/2088607922 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.139+0000 7f6f6a7fc700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f6c0cd5e0 0x7f6f6c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6b7fe700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6c15a650 0x7f6f6c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6affd700 1 -- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6c15a650 msgr2=0x7f6f6c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6affd700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6c15a650 0x7f6f6c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6affd700 1 -- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f6c0cd5e0 msgr2=0x7f6f6c15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6affd700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f6c0cd5e0 0x7f6f6c15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6affd700 1 -- 172.21.15.33:0/2088607922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c009420 con 0x7f6f6c0af540 2023-10-06T05:27:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6b7fe700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6c15a650 0x7f6f6c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6affd700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c14a0b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0c7320 tx=0x7f6f5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c011b10 con 0x7f6f6c0af540 2023-10-06T05:27:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c000cc0 con 0x7f6f6c0af540 2023-10-06T05:27:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c16a180 con 0x7f6f6c0af540 2023-10-06T05:27:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c025620 con 0x7f6f6c0af540 2023-10-06T05:27:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.140+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c0b3270 con 0x7f6f6c0af540 2023-10-06T05:27:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.141+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6c04c380 con 0x7f6f6c0af540 2023-10-06T05:27:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.142+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f5c025a10 con 0x7f6f6c0af540 2023-10-06T05:27:01.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.143+0000 7f6f52ffd700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f58066f30 0x7f6f580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.143+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6f5c085c40 con 0x7f6f6c0af540 2023-10-06T05:27:01.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.144+0000 7f6f6a7fc700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f58066f30 0x7f6f580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.144+0000 7f6f6a7fc700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f58066f30 0x7f6f580693e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c155850 tx=0x7f6f60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.147+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f5c053000 con 0x7f6f6c0af540 2023-10-06T05:27:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.352+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6f6c15db70 con 0x7f6f6c0af540 2023-10-06T05:27:01.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.358+0000 7f6f52ffd700 1 -- 172.21.15.33:0/2088607922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f5c056620 con 0x7f6f6c0af540 2023-10-06T05:27:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f58066f30 msgr2=0x7f6f580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f58066f30 0x7f6f580693e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c155850 tx=0x7f6f60009040 comp rx=0 tx=0).stop 2023-10-06T05:27:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 msgr2=0x7f6f6c14a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c14a0b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0c7320 tx=0x7f6f5c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 shutdown_connections 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f58066f30 0x7f6f580693e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6c15a650 0x7f6f6c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6c0af540 0x7f6f6c14a0b0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 --2- 172.21.15.33:0/2088607922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f6c0cd5e0 0x7f6f6c15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 >> 172.21.15.33:0/2088607922 conn(0x7f6f6c06d9b0 msgr2=0x7f6f6c158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 shutdown_connections 2023-10-06T05:27:01.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.360+0000 7f6f6bfff700 1 -- 172.21.15.33:0/2088607922 wait complete. 2023-10-06T05:27:01.379 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:27:01.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:01.470 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:01.555 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:27:01.556 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:27:01.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:01.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.741+0000 7f8183fff700 1 Processor -- start 2023-10-06T05:27:01.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.742+0000 7f8183fff700 1 -- start start 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f8183fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8184155140 0x7f8184158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f8183fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81840c8e40 con 0x7f81840699c0 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f8183fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81840c99e0 con 0x7f81840adf50 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f8183fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840ae3f0 con 0x7f8184155140 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f8182ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8184155140 0x7f8184158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f81837fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81840699c0 0x7f81840c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34070/0 (socket says 172.21.15.33:34070) 2023-10-06T05:27:01.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.743+0000 7f81837fe700 1 -- 172.21.15.33:0/1733077581 learned_addr learned my addr 172.21.15.33:0/1733077581 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:01.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8182ffd700 1 -- 172.21.15.33:0/1733077581 >> v1:172.21.15.167:6789/0 conn(0x7f81840adf50 legacy=0x7f81840c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:01.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8182ffd700 1 -- 172.21.15.33:0/1733077581 >> v1:172.21.15.33:6789/0 conn(0x7f81840699c0 legacy=0x7f81840c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8182ffd700 1 -- 172.21.15.33:0/1733077581 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840ae6c0 con 0x7f8184155140 2023-10-06T05:27:01.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8182ffd700 1 --2- 172.21.15.33:0/1733077581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8184155140 0x7f8184158700 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8174009910 tx=0x7f81740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2860194dd8baa59d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8181ffb700 1 -- 172.21.15.33:0/1733077581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400b820 con 0x7f8184155140 2023-10-06T05:27:01.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8181ffb700 1 -- 172.21.15.33:0/1733077581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817400b980 con 0x7f8184155140 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.744+0000 7f8181ffb700 1 -- 172.21.15.33:0/1733077581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400bd20 con 0x7f8184155140 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 -- 172.21.15.33:0/1733077581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8184155140 msgr2=0x7f8184158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 --2- 172.21.15.33:0/1733077581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8184155140 0x7f8184158700 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8174009910 tx=0x7f81740091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 -- 172.21.15.33:0/1733077581 shutdown_connections 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 --2- 172.21.15.33:0/1733077581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8184155140 0x7f8184158700 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 -- 172.21.15.33:0/1733077581 >> 172.21.15.33:0/1733077581 conn(0x7f818406d030 msgr2=0x7f818406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 -- 172.21.15.33:0/1733077581 shutdown_connections 2023-10-06T05:27:01.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.745+0000 7f8183fff700 1 -- 172.21.15.33:0/1733077581 wait complete. 2023-10-06T05:27:01.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.746+0000 7f8183fff700 1 Processor -- start 2023-10-06T05:27:01.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.746+0000 7f8183fff700 1 -- start start 2023-10-06T05:27:01.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.747+0000 7f8183fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81840699c0 0x7f818414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.747+0000 7f8183fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 0x7f81840c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.747+0000 7f8183fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8184155140 0x7f81840c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.747+0000 7f8183fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c3190 con 0x7f81840adf50 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.747+0000 7f8183fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818414f8c0 con 0x7f81840699c0 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.747+0000 7f8183fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c33e0 con 0x7f8184155140 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 0x7f81840c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 0x7f81840c1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39934/0 (socket says 172.21.15.33:39934) 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 -- 172.21.15.33:0/3010813494 learned_addr learned my addr 172.21.15.33:0/3010813494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81837fe700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8184155140 0x7f81840c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f8182ffd700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81840699c0 0x7f818414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 -- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8184155140 msgr2=0x7f81840c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8184155140 0x7f81840c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 -- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81840699c0 msgr2=0x7f818414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81840699c0 0x7f818414f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 -- 172.21.15.33:0/3010813494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009420 con 0x7f81840adf50 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f8182ffd700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81840699c0 0x7f818414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.748+0000 7f81827fc700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 0x7f81840c1f00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f817800c590 tx=0x7f817800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.749+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178011590 con 0x7f81840adf50 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.749+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81780116f0 con 0x7f81840adf50 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.749+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178011ab0 con 0x7f81840adf50 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.749+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81840c36c0 con 0x7f81840adf50 2023-10-06T05:27:01.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.749+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81840c3c80 con 0x7f81840adf50 2023-10-06T05:27:01.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.750+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81840c3e70 con 0x7f81840adf50 2023-10-06T05:27:01.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.751+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f817801b3b0 con 0x7f81840adf50 2023-10-06T05:27:01.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.752+0000 7f816affd700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8170066f30 0x7f81700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.752+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8178083a10 con 0x7f81840adf50 2023-10-06T05:27:01.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.752+0000 7f8182ffd700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8170066f30 0x7f81700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.753+0000 7f8182ffd700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8170066f30 0x7f81700693e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f81740093f0 tx=0x7f8174017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:01.756+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8178050d50 con 0x7f81840adf50 2023-10-06T05:27:02.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.002+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f818404c380 con 0x7f81840adf50 2023-10-06T05:27:02.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.003+0000 7f816affd700 1 -- 172.21.15.33:0/3010813494 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+3537 (secure 0 0 0) 0x7f8178017020 con 0x7f81840adf50 2023-10-06T05:27:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:02.005 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":126,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8335,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3411130742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3411130742},{"type":"v1","addr":"172.21.15.167:6836","nonce":3411130742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4032354134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4032354134},{"type":"v1","addr":"172.21.15.33:6833","nonce":4032354134}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":9057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2758204064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2758204064},{"type":"v1","addr":"172.21.15.167:6837","nonce":2758204064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":9330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/796187001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":796187001},{"type":"v1","addr":"172.21.15.33:6835","nonce":796187001}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[]} 2023-10-06T05:27:02.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8170066f30 msgr2=0x7f81700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8170066f30 0x7f81700693e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f81740093f0 tx=0x7f8174017000 comp rx=0 tx=0).stop 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 msgr2=0x7f81840c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 0x7f81840c1f00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f817800c590 tx=0x7f817800c950 comp rx=0 tx=0).stop 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 shutdown_connections 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8170066f30 0x7f81700693e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8184155140 0x7f81840c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81840699c0 0x7f818414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 --2- 172.21.15.33:0/3010813494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81840adf50 0x7f81840c1f00 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.005+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 >> 172.21.15.33:0/3010813494 conn(0x7f818406d030 msgr2=0x7f818406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.006+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 shutdown_connections 2023-10-06T05:27:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.006+0000 7f8183fff700 1 -- 172.21.15.33:0/3010813494 wait complete. 2023-10-06T05:27:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 126 2023-10-06T05:27:02.025 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:27:02.025 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:27:02.025 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:27:02.025 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:02.025 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:27:02.026 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:27:02.026 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:27:02.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:02.026 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:27:02.026 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:27:02.026 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:27:02.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:02.027 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:27:02.027 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:27:02.027 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:27:02.027 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:02.041 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:27:02.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:27:02.043 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:27:02.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:27:02.045 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:27:02.045 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:27:02.046 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:27:02.046 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:27:02.084 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:27:02.084 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:02.087 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:27:02.087 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:02.091 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:27:02.092 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:02.093 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:27:02.093 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:02.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:27:02.223 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:27:02.223 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:27:02.229 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:27:02.229 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:27:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.290+0000 7f289759e700 1 Processor -- start 2023-10-06T05:27:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.290+0000 7f289759e700 1 -- start start 2023-10-06T05:27:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.291+0000 7f289759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2898156880 0x7f28980c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.291+0000 7f289759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28980c8320 con 0x7f2898159e10 2023-10-06T05:27:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.291+0000 7f289759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289806a820 con 0x7f28980c7ec0 2023-10-06T05:27:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.291+0000 7f289759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980cc290 con 0x7f2898156880 2023-10-06T05:27:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f2895d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2898159e10 0x7f2898156c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34084/0 (socket says 172.21.15.33:34084) 2023-10-06T05:27:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f2895d9b700 1 -- 172.21.15.33:0/3146270412 learned_addr learned my addr 172.21.15.33:0/3146270412 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289659c700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2898156880 0x7f28980c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626247181 0 0) 0x7f288c001110 con 0x7f2898159e10 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2880003570 con 0x7f2898159e10 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772939028 0 0) 0x7f289806a820 con 0x7f28980c7ec0 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f288c001110 con 0x7f28980c7ec0 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680604943 0 0) 0x7f2880003570 con 0x7f2898159e10 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2898156880 msgr2=0x7f28980c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2898156880 0x7f28980c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 >> v1:172.21.15.167:6789/0 conn(0x7f28980c7ec0 legacy=0x7f289806ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.292+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289815a2f0 con 0x7f2898159e10 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f288c001600 con 0x7f2898159e10 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289659c700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2898156880 0x7f28980c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 >> v1:172.21.15.33:6789/0 conn(0x7f2898159e10 legacy=0x7f2898156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2880003c20 0x7f2880004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2880004810 0x7f2880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f288000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980cc290 con 0x7f28800051c0 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288000c020 con 0x7f2880003c20 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288000c1e0 con 0x7f2880004810 2023-10-06T05:27:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2896d9d700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f288000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f289659c700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2880003c20 0x7f2880004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2895d9b700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2880004810 0x7f2880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2896d9d700 1 -- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2880004810 msgr2=0x7f2880004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2896d9d700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2880004810 0x7f2880004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2896d9d700 1 -- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2880003c20 msgr2=0x7f2880004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2896d9d700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2880003c20 0x7f2880004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.293+0000 7f2896d9d700 1 -- 172.21.15.33:0/3146270412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880003570 con 0x7f28800051c0 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f2896d9d700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f288000ba60 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f2890002120 tx=0x7f28900017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c616c0a4e72b38ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890001ab0 con 0x7f28800051c0 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289000c980 con 0x7f28800051c0 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289559a700 1 -- 172.21.15.33:0/3146270412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000cd20 con 0x7f28800051c0 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289759e700 1 -- 172.21.15.33:0/3146270412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 msgr2=0x7f288000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289759e700 1 --2- 172.21.15.33:0/3146270412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f288000ba60 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f2890002120 tx=0x7f28900017a0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f2896d9d700 1 -- 172.21.15.33:0/3146270412 reap_dead start 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289759e700 1 -- 172.21.15.33:0/3146270412 shutdown_connections 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.294+0000 7f289759e700 1 -- 172.21.15.33:0/3146270412 >> 172.21.15.33:0/3146270412 conn(0x7f289806c560 msgr2=0x7f289806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.295+0000 7f289759e700 1 -- 172.21.15.33:0/3146270412 shutdown_connections 2023-10-06T05:27:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.295+0000 7f289759e700 1 -- 172.21.15.33:0/3146270412 wait complete. 2023-10-06T05:27:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.296+0000 7f289759e700 1 Processor -- start 2023-10-06T05:27:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.296+0000 7f289759e700 1 -- start start 2023-10-06T05:27:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f28980ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f289813d740 0x7f289813dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f289815be80 0x7f289815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289813e0f0 con 0x7f28800051c0 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289815ec70 con 0x7f289815be80 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289815e060 con 0x7f289813d740 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f28980ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f28980ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39976/0 (socket says 172.21.15.33:39976) 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.297+0000 7f289659c700 1 -- 172.21.15.33:0/2801813484 learned_addr learned my addr 172.21.15.33:0/2801813484 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f2895d9b700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f289813d740 0x7f289813dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f2896d9d700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f289815be80 0x7f289815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289659c700 1 -- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f289813d740 msgr2=0x7f289813dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289659c700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f289813d740 0x7f289813dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289659c700 1 -- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f289815be80 msgr2=0x7f289815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289659c700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f289815be80 0x7f289815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289659c700 1 -- 172.21.15.33:0/2801813484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289815e340 con 0x7f28800051c0 2023-10-06T05:27:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f2896d9d700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f289815be80 0x7f289815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289659c700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f28980ca000 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f2888011d20 tx=0x7f28880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800fb50 con 0x7f28800051c0 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288800fcb0 con 0x7f28800051c0 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288801d690 con 0x7f28800051c0 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.298+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289815e620 con 0x7f28800051c0 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.299+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289815c7f0 con 0x7f28800051c0 2023-10-06T05:27:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.299+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289804c380 con 0x7f28800051c0 2023-10-06T05:27:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.300+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f288801dac0 con 0x7f28800051c0 2023-10-06T05:27:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.302+0000 7f28867fc700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2878066f30 0x7f28780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.302+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2888018070 con 0x7f28800051c0 2023-10-06T05:27:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.302+0000 7f2895d9b700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2878066f30 0x7f28780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.303+0000 7f2895d9b700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2878066f30 0x7f28780693e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f288c001e10 tx=0x7f288c0079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.306+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2888052650 con 0x7f28800051c0 2023-10-06T05:27:02.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.523+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f289815a5d0 con 0x7f28800051c0 2023-10-06T05:27:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.938+0000 7f28867fc700 1 -- 172.21.15.33:0/2801813484 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v175) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2888055c70 con 0x7f28800051c0 2023-10-06T05:27:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2878066f30 msgr2=0x7f28780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2878066f30 0x7f28780693e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f288c001e10 tx=0x7f288c0079d0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 msgr2=0x7f28980ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f28980ca000 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f2888011d20 tx=0x7f28880091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 shutdown_connections 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2878066f30 0x7f28780693e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f289815be80 0x7f289815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28800051c0 0x7f28980ca000 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 --2- 172.21.15.33:0/2801813484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f289813d740 0x7f289813dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 >> 172.21.15.33:0/2801813484 conn(0x7f289806c560 msgr2=0x7f289814f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 shutdown_connections 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:02.941+0000 7f289759e700 1 -- 172.21.15.33:0/2801813484 wait complete. 2023-10-06T05:27:02.944 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:27:02.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:27:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.151+0000 7f554d5ff700 1 Processor -- start 2023-10-06T05:27:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.151+0000 7f554d5ff700 1 -- start start 2023-10-06T05:27:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.152+0000 7f554d5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55480c9e10 0x7f55480ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.152+0000 7f554d5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55480ca8d0 con 0x7f554813fb70 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.152+0000 7f554d5ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55480cabb0 con 0x7f55480af000 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.152+0000 7f554d5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554806ad00 con 0x7f55480c9e10 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55480c9e10 0x7f55480ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55480c9e10 0x7f55480ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39984/0 (socket says 172.21.15.33:39984) 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 -- 172.21.15.33:0/3089738410 learned_addr learned my addr 172.21.15.33:0/3089738410 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f5546ffd700 1 --1- 172.21.15.33:0/3089738410 >> v1:172.21.15.33:6789/0 conn(0x7f554813fb70 0x7f554806b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34098/0 (socket says 172.21.15.33:34098) 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 -- 172.21.15.33:0/3089738410 >> v1:172.21.15.167:6789/0 conn(0x7f55480af000 legacy=0x7f55480c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 -- 172.21.15.33:0/3089738410 >> v1:172.21.15.33:6789/0 conn(0x7f554813fb70 legacy=0x7f554806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 -- 172.21.15.33:0/3089738410 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554806afe0 con 0x7f55480c9e10 2023-10-06T05:27:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.153+0000 7f55467fc700 1 --2- 172.21.15.33:0/3089738410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f55480ca290 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f553c0099b0 tx=0x7f553c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686f7f23be4b9e21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f5545ffb700 1 -- 172.21.15.33:0/3089738410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00b820 con 0x7f55480c9e10 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f5545ffb700 1 -- 172.21.15.33:0/3089738410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c00b980 con 0x7f55480c9e10 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f5545ffb700 1 -- 172.21.15.33:0/3089738410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00bd20 con 0x7f55480c9e10 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 -- 172.21.15.33:0/3089738410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 msgr2=0x7f55480ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 --2- 172.21.15.33:0/3089738410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f55480ca290 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f553c0099b0 tx=0x7f553c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 -- 172.21.15.33:0/3089738410 shutdown_connections 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 --2- 172.21.15.33:0/3089738410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f55480ca290 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 -- 172.21.15.33:0/3089738410 >> 172.21.15.33:0/3089738410 conn(0x7f55480675f0 msgr2=0x7f55480679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 -- 172.21.15.33:0/3089738410 shutdown_connections 2023-10-06T05:27:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.154+0000 7f554d5ff700 1 -- 172.21.15.33:0/3089738410 wait complete. 2023-10-06T05:27:03.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.156+0000 7f554d5ff700 1 Processor -- start 2023-10-06T05:27:03.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.156+0000 7f554d5ff700 1 -- start start 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f554d5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55480af000 0x7f5548132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f554d5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f5548156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f554d5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f554813fb70 0x7f55480c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f554d5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481581e0 con 0x7f55480c9e10 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f554d5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481575d0 con 0x7f55480af000 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f554d5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55481578b0 con 0x7f554813fb70 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f55467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f5548156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f55467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f5548156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39998/0 (socket says 172.21.15.33:39998) 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f55467fc700 1 -- 172.21.15.33:0/971392424 learned_addr learned my addr 172.21.15.33:0/971392424 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f5546ffd700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55480af000 0x7f5548132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.157+0000 7f55477fe700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f554813fb70 0x7f55480c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f55467fc700 1 -- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f554813fb70 msgr2=0x7f55480c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f55467fc700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f554813fb70 0x7f55480c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f55467fc700 1 -- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55480af000 msgr2=0x7f5548132b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f55467fc700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55480af000 0x7f5548132b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f55467fc700 1 -- 172.21.15.33:0/971392424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c0094f0 con 0x7f55480c9e10 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f55467fc700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f5548156b30 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f553c00c010 tx=0x7f553c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c011e10 con 0x7f55480c9e10 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c020070 con 0x7f55480c9e10 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5548157b90 con 0x7f55480c9e10 2023-10-06T05:27:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c01b6a0 con 0x7f55480c9e10 2023-10-06T05:27:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.158+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5548133080 con 0x7f55480c9e10 2023-10-06T05:27:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.159+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5548063ff0 con 0x7f55480c9e10 2023-10-06T05:27:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.160+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f553c022030 con 0x7f55480c9e10 2023-10-06T05:27:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.161+0000 7f552effd700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5534066f30 0x7f55340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.161+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f553c0863e0 con 0x7f55480c9e10 2023-10-06T05:27:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.162+0000 7f5546ffd700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5534066f30 0x7f55340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.162+0000 7f5546ffd700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5534066f30 0x7f55340693e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5538007fd0 tx=0x7f5538007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.165+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f553c0537a0 con 0x7f55480c9e10 2023-10-06T05:27:03.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.447+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5548063ff0 con 0x7f55480c9e10 2023-10-06T05:27:03.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.448+0000 7f552effd700 1 -- 172.21.15.33:0/971392424 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f553c056dc0 con 0x7f55480c9e10 2023-10-06T05:27:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:03.449 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:27:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:27:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:27:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:27:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:27:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:27:03.465 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:27:03.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5534066f30 msgr2=0x7f55340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5534066f30 0x7f55340693e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5538007fd0 tx=0x7f5538007d70 comp rx=0 tx=0).stop 2023-10-06T05:27:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 msgr2=0x7f5548156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f5548156b30 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f553c00c010 tx=0x7f553c016000 comp rx=0 tx=0).stop 2023-10-06T05:27:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 shutdown_connections 2023-10-06T05:27:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5534066f30 0x7f55340693e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f554813fb70 0x7f55480c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55480af000 0x7f5548132b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 --2- 172.21.15.33:0/971392424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55480c9e10 0x7f5548156b30 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 >> 172.21.15.33:0/971392424 conn(0x7f55480675f0 msgr2=0x7f554813ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 shutdown_connections 2023-10-06T05:27:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.450+0000 7f554d5ff700 1 -- 172.21.15.33:0/971392424 wait complete. 2023-10-06T05:27:03.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:03.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.655+0000 7f1f9b59e700 1 Processor -- start 2023-10-06T05:27:03.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.655+0000 7f1f9b59e700 1 -- start start 2023-10-06T05:27:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f9c0c8050 0x7f1f9c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0c9cb0 con 0x7f1f9c13faa0 2023-10-06T05:27:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0ca270 con 0x7f1f9c06aad0 2023-10-06T05:27:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c06b640 con 0x7f1f9c0c8050 2023-10-06T05:27:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1f9c13faa0 0x7f1f9c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34114/0 (socket says 172.21.15.33:34114) 2023-10-06T05:27:03.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9a59c700 1 -- 172.21.15.33:0/1054366942 learned_addr learned my addr 172.21.15.33:0/1054366942 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f99d9b700 1 --2- 172.21.15.33:0/1054366942 >> v2:172.21.15.167:3300/0 conn(0x7f1f9c0c8050 0x7f1f9c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591893585 0 0) 0x7f1f9c0c9cb0 con 0x7f1f9c13faa0 2023-10-06T05:27:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.657+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f84003570 con 0x7f1f9c13faa0 2023-10-06T05:27:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1682408077 0 0) 0x7f1f84003570 con 0x7f1f9c13faa0 2023-10-06T05:27:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 >> v2:172.21.15.167:3300/0 conn(0x7f1f9c0c8050 msgr2=0x7f1f9c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 --2- 172.21.15.33:0/1054366942 >> v2:172.21.15.167:3300/0 conn(0x7f1f9c0c8050 0x7f1f9c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 >> v1:172.21.15.167:6789/0 conn(0x7f1f9c06aad0 legacy=0x7f1f9c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c06b840 con 0x7f1f9c13faa0 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1f8c001470 con 0x7f1f9c13faa0 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 >> v1:172.21.15.33:6789/0 conn(0x7f1f9c13faa0 legacy=0x7f1f9c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f84003a60 0x7f1f84003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f84004810 0x7f1f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f840051c0 0x7f1f8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c06b640 con 0x7f1f840051c0 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8400c050 con 0x7f1f84004810 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8400c210 con 0x7f1f84003a60 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f840051c0 0x7f1f8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f99d9b700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f84004810 0x7f1f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.658+0000 7f1f9a59c700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f84003a60 0x7f1f84003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f84003a60 msgr2=0x7f1f84003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f84003a60 0x7f1f84003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f84004810 msgr2=0x7f1f84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f84004810 0x7f1f84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/1054366942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84003570 con 0x7f1f840051c0 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/1054366942 reap_dead start 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f840051c0 0x7f1f8400ba60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1f940020a0 tx=0x7f1f940016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d32a4c95b0a9bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940019e0 con 0x7f1f840051c0 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9400c8b0 con 0x7f1f840051c0 2023-10-06T05:27:03.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.659+0000 7f1f9959a700 1 -- 172.21.15.33:0/1054366942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400cc50 con 0x7f1f840051c0 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 -- 172.21.15.33:0/1054366942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f840051c0 msgr2=0x7f1f8400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f840051c0 0x7f1f8400ba60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1f940020a0 tx=0x7f1f940016d0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 -- 172.21.15.33:0/1054366942 shutdown_connections 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/1054366942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f840051c0 0x7f1f8400ba60 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 -- 172.21.15.33:0/1054366942 >> 172.21.15.33:0/1054366942 conn(0x7f1f9c06d030 msgr2=0x7f1f9c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 -- 172.21.15.33:0/1054366942 shutdown_connections 2023-10-06T05:27:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.660+0000 7f1f9b59e700 1 -- 172.21.15.33:0/1054366942 wait complete. 2023-10-06T05:27:03.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.661+0000 7f1f9b59e700 1 Processor -- start 2023-10-06T05:27:03.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 -- start start 2023-10-06T05:27:03.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f840051c0 0x7f1f9c0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c0b1140 0x7f1f9c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 0x7f1f9c0cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0cb780 con 0x7f1f9c0cada0 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0ae380 con 0x7f1f9c0b1140 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.662+0000 7f1f9b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0ad770 con 0x7f1f840051c0 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 0x7f1f9c0cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 0x7f1f9c0cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40002/0 (socket says 172.21.15.33:40002) 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/3457541805 learned_addr learned my addr 172.21.15.33:0/3457541805 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9a59c700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f840051c0 0x7f1f9c0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f99d9b700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c0b1140 0x7f1f9c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f840051c0 msgr2=0x7f1f9c0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f840051c0 0x7f1f9c0ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c0b1140 msgr2=0x7f1f9c158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c0b1140 0x7f1f9c158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 -- 172.21.15.33:0/3457541805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0ada50 con 0x7f1f9c0cada0 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f99d9b700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c0b1140 0x7f1f9c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.663+0000 7f1f9ad9d700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 0x7f1f9c0cb240 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0ca1e0 tx=0x7f1f94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.664+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940019e0 con 0x7f1f9c0cada0 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.664+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f940012d0 con 0x7f1f9c0cada0 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.664+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94007a20 con 0x7f1f9c0cada0 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.664+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c0add30 con 0x7f1f9c0cada0 2023-10-06T05:27:03.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.664+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c159370 con 0x7f1f9c0cada0 2023-10-06T05:27:03.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.665+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c0ae510 con 0x7f1f9c0cada0 2023-10-06T05:27:03.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.666+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f9401d630 con 0x7f1f9c0cada0 2023-10-06T05:27:03.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.667+0000 7f1f8a7fc700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f7c066f30 0x7f1f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.667+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1f94085c70 con 0x7f1f9c0cada0 2023-10-06T05:27:03.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.667+0000 7f1f9a59c700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f7c066f30 0x7f1f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.668+0000 7f1f9a59c700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f7c066f30 0x7f1f7c0693e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004dc0 tx=0x7f1f8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.671+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f94053030 con 0x7f1f9c0cada0 2023-10-06T05:27:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.875+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f9c04c380 con 0x7f1f9c0cada0 2023-10-06T05:27:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.875+0000 7f1f8a7fc700 1 -- 172.21.15.33:0/3457541805 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f1f94022090 con 0x7f1f9c0cada0 2023-10-06T05:27:03.877 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:03.877 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":177,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:02.946677+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:03.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f7c066f30 msgr2=0x7f1f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f7c066f30 0x7f1f7c0693e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004dc0 tx=0x7f1f8c009040 comp rx=0 tx=0).stop 2023-10-06T05:27:03.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 msgr2=0x7f1f9c0cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 0x7f1f9c0cb240 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0ca1e0 tx=0x7f1f94016000 comp rx=0 tx=0).stop 2023-10-06T05:27:03.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 shutdown_connections 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f7c066f30 0x7f1f7c0693e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c0cada0 0x7f1f9c0cb240 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f840051c0 0x7f1f9c0ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 --2- 172.21.15.33:0/3457541805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c0b1140 0x7f1f9c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 >> 172.21.15.33:0/3457541805 conn(0x7f1f9c06d030 msgr2=0x7f1f9c0c13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 shutdown_connections 2023-10-06T05:27:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:03.878+0000 7f1f9b59e700 1 -- 172.21.15.33:0/3457541805 wait complete. 2023-10-06T05:27:03.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:27:03.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:27:04.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.089+0000 7f87b679d700 1 Processor -- start 2023-10-06T05:27:04.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.090+0000 7f87b679d700 1 -- start start 2023-10-06T05:27:04.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b679d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87b015acd0 0x7f87b00abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 0x7f87b00ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00b0cd0 con 0x7f87b0159850 2023-10-06T05:27:04.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b679d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b015a840 con 0x7f87b015acd0 2023-10-06T05:27:04.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0159f10 con 0x7f87b006efa0 2023-10-06T05:27:04.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87b0159850 0x7f87b006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34116/0 (socket says 172.21.15.33:34116) 2023-10-06T05:27:04.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b579b700 1 -- 172.21.15.33:0/3806619267 learned_addr learned my addr 172.21.15.33:0/3806619267 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:04.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/3806619267 >> v2:172.21.15.167:3300/0 conn(0x7f87b015acd0 0x7f87b00abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.091+0000 7f87b5f9c700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 0x7f87b00ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383267685 0 0) 0x7f87b00b0cd0 con 0x7f87b0159850 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f879c003570 con 0x7f87b0159850 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76475001 0 0) 0x7f879c003570 con 0x7f87b0159850 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 msgr2=0x7f87b00ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 0x7f87b00ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 >> v2:172.21.15.167:3300/0 conn(0x7f87b015acd0 msgr2=0x7f87b00abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 --2- 172.21.15.33:0/3806619267 >> v2:172.21.15.167:3300/0 conn(0x7f87b015acd0 0x7f87b00abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b015a1f0 con 0x7f87b0159850 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f87ac001470 con 0x7f87b0159850 2023-10-06T05:27:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 >> v1:172.21.15.33:6789/0 conn(0x7f87b0159850 legacy=0x7f87b006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 0x7f879c003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f879c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f879c004d50 0x7f879c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b015a840 con 0x7f879c0040b0 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0159f10 con 0x7f879c004d50 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c00bb60 con 0x7f87b006efa0 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f879c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.092+0000 7f87b579b700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 0x7f879c003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b5f9c700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f879c004d50 0x7f879c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b4f9a700 1 -- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 msgr2=0x7f879c003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b006efa0 0x7f879c003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b4f9a700 1 -- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f879c004d50 msgr2=0x7f879c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f879c004d50 0x7f879c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b4f9a700 1 -- 172.21.15.33:0/3806619267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c003570 con 0x7f879c0040b0 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b5f9c700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f879c004d50 0x7f879c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f879c004810 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f87a0006f00 tx=0x7f87a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32e0d9fbc4d921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000e710 con 0x7f879c0040b0 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000e870 con 0x7f879c0040b0 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.093+0000 7f87a7fff700 1 -- 172.21.15.33:0/3806619267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000ec30 con 0x7f879c0040b0 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b679d700 1 -- 172.21.15.33:0/3806619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 msgr2=0x7f879c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b679d700 1 --2- 172.21.15.33:0/3806619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f879c004810 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f87a0006f00 tx=0x7f87a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b5f9c700 1 -- 172.21.15.33:0/3806619267 reap_dead start 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b679d700 1 -- 172.21.15.33:0/3806619267 shutdown_connections 2023-10-06T05:27:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b679d700 1 -- 172.21.15.33:0/3806619267 >> 172.21.15.33:0/3806619267 conn(0x7f87b006cd40 msgr2=0x7f87b00ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b679d700 1 -- 172.21.15.33:0/3806619267 shutdown_connections 2023-10-06T05:27:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.094+0000 7f87b679d700 1 -- 172.21.15.33:0/3806619267 wait complete. 2023-10-06T05:27:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.095+0000 7f87b679d700 1 Processor -- start 2023-10-06T05:27:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.096+0000 7f87b679d700 1 -- start start 2023-10-06T05:27:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.096+0000 7f87b679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f87b016d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.096+0000 7f87b679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b00c7ed0 0x7f87b00c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b00cbf00 0x7f87b00cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b10b0 con 0x7f879c0040b0 2023-10-06T05:27:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b1c30 con 0x7f87b00c7ed0 2023-10-06T05:27:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b1300 con 0x7f87b00cbf00 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f87b016d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f87b016d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40024/0 (socket says 172.21.15.33:40024) 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 -- 172.21.15.33:0/2966268455 learned_addr learned my addr 172.21.15.33:0/2966268455 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b5f9c700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b00cbf00 0x7f87b00cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b00c7ed0 0x7f87b00c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 -- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b00cbf00 msgr2=0x7f87b00cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b00cbf00 0x7f87b00cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 -- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b00c7ed0 msgr2=0x7f87b00c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b00c7ed0 0x7f87b00c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 -- 172.21.15.33:0/2966268455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00b15e0 con 0x7f879c0040b0 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.097+0000 7f87b579b700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f87b016d080 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00eb90 tx=0x7f87ac000d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.098+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00c4f0 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.098+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac00c670 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.098+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00ca30 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.098+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b00b18c0 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.098+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b016d5c0 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.099+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b004c380 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.100+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87ac01d460 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.101+0000 7f87a57fa700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8794066f30 0x7f87940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.101+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f87ac0842c0 con 0x7f879c0040b0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.101+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8794066f30 0x7f87940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.102+0000 7f87b4f9a700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8794066f30 0x7f87940693e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f87a00091e0 tx=0x7f87a001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.105+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87ac0551c0 con 0x7f879c0040b0 2023-10-06T05:27:04.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.307+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87b00b0c80 con 0x7f879c0040b0 2023-10-06T05:27:04.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.308+0000 7f87a57fa700 1 -- 172.21.15.33:0/2966268455 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87ac054d50 con 0x7f879c0040b0 2023-10-06T05:27:04.310 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:27:04.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8794066f30 msgr2=0x7f87940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8794066f30 0x7f87940693e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f87a00091e0 tx=0x7f87a001a000 comp rx=0 tx=0).stop 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 msgr2=0x7f87b016d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f87b016d080 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00eb90 tx=0x7f87ac000d00 comp rx=0 tx=0).stop 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 shutdown_connections 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8794066f30 0x7f87940693e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b00cbf00 0x7f87b00cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879c0040b0 0x7f87b016d080 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 --2- 172.21.15.33:0/2966268455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b00c7ed0 0x7f87b00c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 >> 172.21.15.33:0/2966268455 conn(0x7f87b006cd40 msgr2=0x7f87b00afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 shutdown_connections 2023-10-06T05:27:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.310+0000 7f87b679d700 1 -- 172.21.15.33:0/2966268455 wait complete. 2023-10-06T05:27:04.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:27:04.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.518+0000 7f23d719d700 1 Processor -- start 2023-10-06T05:27:04.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.518+0000 7f23d719d700 1 -- start start 2023-10-06T05:27:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.519+0000 7f23d719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f23d015acd0 0x7f23d00abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.519+0000 7f23d719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00b0cd0 con 0x7f23d0159850 2023-10-06T05:27:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.519+0000 7f23d719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d015a840 con 0x7f23d006efa0 2023-10-06T05:27:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.519+0000 7f23d719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d0159f10 con 0x7f23d015acd0 2023-10-06T05:27:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.519+0000 7f23d619b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f23d0159850 0x7f23d006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34124/0 (socket says 172.21.15.33:34124) 2023-10-06T05:27:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.519+0000 7f23d619b700 1 -- 172.21.15.33:0/140216424 learned_addr learned my addr 172.21.15.33:0/140216424 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d599a700 1 --2- 172.21.15.33:0/140216424 >> v2:172.21.15.167:3300/0 conn(0x7f23d015acd0 0x7f23d00abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277735493 0 0) 0x7f23d00b0cd0 con 0x7f23d0159850 2023-10-06T05:27:04.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23bc003570 con 0x7f23d0159850 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188827373 0 0) 0x7f23d015a840 con 0x7f23d006efa0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d00b0cd0 con 0x7f23d006efa0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d599a700 1 -- 172.21.15.33:0/140216424 >> v1:172.21.15.167:6789/0 conn(0x7f23d006efa0 legacy=0x7f23d00ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1787103096 0 0) 0x7f23bc003570 con 0x7f23d0159850 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d599a700 1 -- 172.21.15.33:0/140216424 >> v1:172.21.15.33:6789/0 conn(0x7f23d0159850 legacy=0x7f23d006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.520+0000 7f23d599a700 1 -- 172.21.15.33:0/140216424 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d015a1f0 con 0x7f23d015acd0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d599a700 1 --2- 172.21.15.33:0/140216424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d015acd0 0x7f23d00abe90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098e0 tx=0x7f23c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d0b68039bd190 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc001470 con 0x7f23d015acd0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0011420 con 0x7f23d015acd0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d5199700 1 -- 172.21.15.33:0/140216424 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c00117c0 con 0x7f23d015acd0 2023-10-06T05:27:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d719d700 1 -- 172.21.15.33:0/140216424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d015acd0 msgr2=0x7f23d00abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d719d700 1 --2- 172.21.15.33:0/140216424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d015acd0 0x7f23d00abe90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098e0 tx=0x7f23c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d719d700 1 -- 172.21.15.33:0/140216424 shutdown_connections 2023-10-06T05:27:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d719d700 1 --2- 172.21.15.33:0/140216424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d015acd0 0x7f23d00abe90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.521+0000 7f23d719d700 1 -- 172.21.15.33:0/140216424 >> 172.21.15.33:0/140216424 conn(0x7f23d006cd40 msgr2=0x7f23d00ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.522+0000 7f23d719d700 1 -- 172.21.15.33:0/140216424 shutdown_connections 2023-10-06T05:27:04.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.522+0000 7f23d719d700 1 -- 172.21.15.33:0/140216424 wait complete. 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.523+0000 7f23d719d700 1 Processor -- start 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.523+0000 7f23d719d700 1 -- start start 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 0x7f23d00ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d0159850 0x7f23d0168f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d015acd0 0x7f23d00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d01694a0 con 0x7f23d006efa0 2023-10-06T05:27:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00ac350 con 0x7f23d0159850 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00ac630 con 0x7f23d015acd0 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 0x7f23d00ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d619b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 0x7f23d00ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40028/0 (socket says 172.21.15.33:40028) 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d619b700 1 -- 172.21.15.33:0/3650557840 learned_addr learned my addr 172.21.15.33:0/3650557840 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d599a700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d0159850 0x7f23d0168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.524+0000 7f23d699c700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d015acd0 0x7f23d00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d619b700 1 -- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d015acd0 msgr2=0x7f23d00c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d619b700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d015acd0 0x7f23d00c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d619b700 1 -- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d0159850 msgr2=0x7f23d0168f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d619b700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d0159850 0x7f23d0168f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d619b700 1 -- 172.21.15.33:0/3650557840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0009420 con 0x7f23d006efa0 2023-10-06T05:27:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d619b700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 0x7f23d00ad3f0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00cb70 tx=0x7f23cc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc007360 con 0x7f23d006efa0 2023-10-06T05:27:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc007500 con 0x7f23d006efa0 2023-10-06T05:27:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d00ac910 con 0x7f23d006efa0 2023-10-06T05:27:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc004a80 con 0x7f23d006efa0 2023-10-06T05:27:04.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.525+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d00aced0 con 0x7f23d006efa0 2023-10-06T05:27:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.526+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d004c380 con 0x7f23d006efa0 2023-10-06T05:27:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.527+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23cc00a590 con 0x7f23d006efa0 2023-10-06T05:27:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.528+0000 7f23c67fc700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23b4066f30 0x7f23b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.529+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f23cc083920 con 0x7f23d006efa0 2023-10-06T05:27:04.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.529+0000 7f23d599a700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23b4066f30 0x7f23b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.529+0000 7f23d599a700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23b4066f30 0x7f23b40693e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f23c00093f0 tx=0x7f23c0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.533+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23cc050d60 con 0x7f23d006efa0 2023-10-06T05:27:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.733+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f23d00ae660 con 0x7f23d006efa0 2023-10-06T05:27:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.734+0000 7f23c67fc700 1 -- 172.21.15.33:0/3650557840 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v177) v1 ==== 92+0+10 (secure 0 0 0) 0x7f23cc018020 con 0x7f23d006efa0 2023-10-06T05:27:04.736 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:27:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23b4066f30 msgr2=0x7f23b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23b4066f30 0x7f23b40693e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f23c00093f0 tx=0x7f23c0016000 comp rx=0 tx=0).stop 2023-10-06T05:27:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 msgr2=0x7f23d00ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 0x7f23d00ad3f0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00cb70 tx=0x7f23cc001b50 comp rx=0 tx=0).stop 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 shutdown_connections 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f23b4066f30 0x7f23b40693e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23d015acd0 0x7f23d00c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23d006efa0 0x7f23d00ad3f0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 --2- 172.21.15.33:0/3650557840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23d0159850 0x7f23d0168f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.736+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 >> 172.21.15.33:0/3650557840 conn(0x7f23d006cd40 msgr2=0x7f23d015b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.737+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 shutdown_connections 2023-10-06T05:27:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.737+0000 7f23d719d700 1 -- 172.21.15.33:0/3650557840 wait complete. 2023-10-06T05:27:04.755 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:27:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.940+0000 7f7a7f19d700 1 Processor -- start 2023-10-06T05:27:04.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.940+0000 7f7a7f19d700 1 -- start start 2023-10-06T05:27:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.941+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a78155af0 0x7f7a7806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.941+0000 7f7a7f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a780c7920 0x7f7a780c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.941+0000 7f7a7f19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7806ab60 con 0x7f7a7806a6a0 2023-10-06T05:27:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.941+0000 7f7a7f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7806b7e0 con 0x7f7a780c7920 2023-10-06T05:27:04.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.941+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7806ae40 con 0x7f7a78155af0 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a780c7920 0x7f7a780c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a780c7920 0x7f7a780c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40030/0 (socket says 172.21.15.33:40030) 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 -- 172.21.15.33:0/4082541825 learned_addr learned my addr 172.21.15.33:0/4082541825 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e19b700 1 --2- 172.21.15.33:0/4082541825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a78155af0 0x7f7a7806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 -- 172.21.15.33:0/4082541825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a78155af0 msgr2=0x7f7a7806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 --2- 172.21.15.33:0/4082541825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a78155af0 0x7f7a7806a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 -- 172.21.15.33:0/4082541825 >> v1:172.21.15.167:6789/0 conn(0x7f7a7806a6a0 legacy=0x7f7a780c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 -- 172.21.15.33:0/4082541825 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7806b120 con 0x7f7a780c7920 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.942+0000 7f7a7e99c700 1 --2- 172.21.15.33:0/4082541825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780c9e20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009f20 tx=0x7f7a70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f660442ffabc14bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7d199700 1 -- 172.21.15.33:0/4082541825 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a700092e0 con 0x7f7a780c7920 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7d199700 1 -- 172.21.15.33:0/4082541825 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7000b9d0 con 0x7f7a780c7920 2023-10-06T05:27:04.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7d199700 1 -- 172.21.15.33:0/4082541825 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000bd90 con 0x7f7a780c7920 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 -- 172.21.15.33:0/4082541825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 msgr2=0x7f7a780c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/4082541825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780c9e20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009f20 tx=0x7f7a70009840 comp rx=0 tx=0).stop 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 -- 172.21.15.33:0/4082541825 shutdown_connections 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/4082541825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780c9e20 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/4082541825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a78155af0 0x7f7a7806a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 -- 172.21.15.33:0/4082541825 >> 172.21.15.33:0/4082541825 conn(0x7f7a7806d030 msgr2=0x7f7a7806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 -- 172.21.15.33:0/4082541825 shutdown_connections 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.943+0000 7f7a7f19d700 1 -- 172.21.15.33:0/4082541825 wait complete. 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.945+0000 7f7a7f19d700 1 Processor -- start 2023-10-06T05:27:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.945+0000 7f7a7f19d700 1 -- start start 2023-10-06T05:27:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a7806a6a0 0x7f7a780c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a78155af0 0x7f7a7816cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780b0a60 con 0x7f7a780c7920 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780b0150 con 0x7f7a78155af0 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780ab740 con 0x7f7a7806a6a0 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7e99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a78155af0 0x7f7a7816cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7e19b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a7806a6a0 0x7f7a780c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40040/0 (socket says 172.21.15.33:40040) 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.946+0000 7f7a7d99a700 1 -- 172.21.15.33:0/3789421097 learned_addr learned my addr 172.21.15.33:0/3789421097 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7d99a700 1 -- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a7806a6a0 msgr2=0x7f7a780c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7d99a700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a7806a6a0 0x7f7a780c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7d99a700 1 -- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a78155af0 msgr2=0x7f7a7816cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7d99a700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a78155af0 0x7f7a7816cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7d99a700 1 -- 172.21.15.33:0/3789421097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70009ae0 con 0x7f7a780c7920 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7d99a700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780aac80 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c7c0 tx=0x7f7a68007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68011590 con 0x7f7a780c7920 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a680116f0 con 0x7f7a780c7920 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68011ab0 con 0x7f7a780c7920 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.947+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a780b03a0 con 0x7f7a780c7920 2023-10-06T05:27:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.948+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78159210 con 0x7f7a780c7920 2023-10-06T05:27:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.949+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a780b0ba0 con 0x7f7a780c7920 2023-10-06T05:27:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.949+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a6801b3b0 con 0x7f7a780c7920 2023-10-06T05:27:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.950+0000 7f7a6e7fc700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.950+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7a68083790 con 0x7f7a780c7920 2023-10-06T05:27:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.951+0000 7f7a7e19b700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.952+0000 7f7a7e19b700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a5c066f30 0x7f7a5c0693e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009d10 tx=0x7f7a740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:04.956+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a780b0ba0 con 0x7f7a780c7920 2023-10-06T05:27:05.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.162+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7a780b0ba0 con 0x7f7a780c7920 2023-10-06T05:27:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.162+0000 7f7a6e7fc700 1 -- 172.21.15.33:0/3789421097 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v177) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7a780b0ba0 con 0x7f7a780c7920 2023-10-06T05:27:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:05.164 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:27:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:27:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:27:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:27:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:27:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:05.165 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:27:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a5c066f30 msgr2=0x7f7a5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a5c066f30 0x7f7a5c0693e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009d10 tx=0x7f7a740091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 msgr2=0x7f7a780aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780aac80 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c7c0 tx=0x7f7a68007f50 comp rx=0 tx=0).stop 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 shutdown_connections 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a78155af0 0x7f7a7816cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a7806a6a0 0x7f7a780c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 --2- 172.21.15.33:0/3789421097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a780c7920 0x7f7a780aac80 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 >> 172.21.15.33:0/3789421097 conn(0x7f7a7806d030 msgr2=0x7f7a7806e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.164+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 shutdown_connections 2023-10-06T05:27:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.165+0000 7f7a7f19d700 1 -- 172.21.15.33:0/3789421097 wait complete. 2023-10-06T05:27:05.184 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:27:05.184 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:27:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.373+0000 7f429c99d700 1 Processor -- start 2023-10-06T05:27:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.373+0000 7f429c99d700 1 -- start start 2023-10-06T05:27:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.374+0000 7f429c99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42980ad520 0x7f42980aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.374+0000 7f429c99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980afc40 con 0x7f429806c560 2023-10-06T05:27:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.374+0000 7f429c99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429815d7c0 con 0x7f42980c9720 2023-10-06T05:27:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.374+0000 7f429c99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42980ad9c0 con 0x7f42980ad520 2023-10-06T05:27:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4296ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42980ad520 0x7f42980aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4297fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f429806c560 0x7f429806c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34130/0 (socket says 172.21.15.33:34130) 2023-10-06T05:27:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4297fff700 1 -- 172.21.15.33:0/2538885609 learned_addr learned my addr 172.21.15.33:0/2538885609 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4296ffd700 1 -- 172.21.15.33:0/2538885609 >> v1:172.21.15.167:6789/0 conn(0x7f42980c9720 legacy=0x7f429815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4296ffd700 1 -- 172.21.15.33:0/2538885609 >> v1:172.21.15.33:6789/0 conn(0x7f429806c560 legacy=0x7f429806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4296ffd700 1 -- 172.21.15.33:0/2538885609 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298159bc0 con 0x7f42980ad520 2023-10-06T05:27:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f42967fc700 1 -- 172.21.15.33:0/2538885609 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128491956 0 0) 0x7f42980afc40 con 0x7f429806c560 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.375+0000 7f4296ffd700 1 --2- 172.21.15.33:0/2538885609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42980ad520 0x7f42980aeda0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f428c0098e0 tx=0x7f428c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877a6928cd3475b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f42967fc700 1 -- 172.21.15.33:0/2538885609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c009e80 con 0x7f42980ad520 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f42967fc700 1 -- 172.21.15.33:0/2538885609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c00b820 con 0x7f42980ad520 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f42967fc700 1 -- 172.21.15.33:0/2538885609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00bba0 con 0x7f42980ad520 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 -- 172.21.15.33:0/2538885609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42980ad520 msgr2=0x7f42980aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 --2- 172.21.15.33:0/2538885609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42980ad520 0x7f42980aeda0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f428c0098e0 tx=0x7f428c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 -- 172.21.15.33:0/2538885609 shutdown_connections 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 --2- 172.21.15.33:0/2538885609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42980ad520 0x7f42980aeda0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 -- 172.21.15.33:0/2538885609 >> 172.21.15.33:0/2538885609 conn(0x7f42980675f0 msgr2=0x7f42980679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 -- 172.21.15.33:0/2538885609 shutdown_connections 2023-10-06T05:27:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.376+0000 7f429c99d700 1 -- 172.21.15.33:0/2538885609 wait complete. 2023-10-06T05:27:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.378+0000 7f429c99d700 1 Processor -- start 2023-10-06T05:27:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.378+0000 7f429c99d700 1 -- start start 2023-10-06T05:27:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f429c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 0x7f42980cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f429c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42980ad520 0x7f42981582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f429c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42980c9720 0x7f429814c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f429c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981587e0 con 0x7f429806c560 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f429c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429814c940 con 0x7f42980ad520 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f429c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4298157e10 con 0x7f42980c9720 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f42977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 0x7f42980cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f42977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 0x7f42980cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40068/0 (socket says 172.21.15.33:40068) 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f42977fe700 1 -- 172.21.15.33:0/2300268991 learned_addr learned my addr 172.21.15.33:0/2300268991 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f4296ffd700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42980ad520 0x7f42981582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.379+0000 7f4297fff700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42980c9720 0x7f429814c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f42977fe700 1 -- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42980c9720 msgr2=0x7f429814c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f42977fe700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42980c9720 0x7f429814c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f42977fe700 1 -- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42980ad520 msgr2=0x7f42981582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f42977fe700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42980ad520 0x7f42981582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f42977fe700 1 -- 172.21.15.33:0/2300268991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c009420 con 0x7f429806c560 2023-10-06T05:27:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f42977fe700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 0x7f42980cd5d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f428800b5e0 tx=0x7f4288009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800f510 con 0x7f429806c560 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428800f670 con 0x7f429806c560 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4298157200 con 0x7f429806c560 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800fa30 con 0x7f429806c560 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.380+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42981577c0 con 0x7f429806c560 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.381+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4298063ff0 con 0x7f429806c560 2023-10-06T05:27:05.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.382+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f428801a400 con 0x7f429806c560 2023-10-06T05:27:05.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.383+0000 7f427f7fe700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.383+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4288083f00 con 0x7f429806c560 2023-10-06T05:27:05.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.384+0000 7f4296ffd700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.384+0000 7f4296ffd700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4284066f30 0x7f42840693e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f428c0030c0 tx=0x7f428c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.387+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4288051340 con 0x7f429806c560 2023-10-06T05:27:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.588+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42980c9eb0 con 0x7f429806c560 2023-10-06T05:27:05.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.957+0000 7f427f7fe700 1 -- 172.21.15.33:0/2300268991 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v178) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4288054960 con 0x7f429806c560 2023-10-06T05:27:05.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4284066f30 msgr2=0x7f42840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4284066f30 0x7f42840693e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f428c0030c0 tx=0x7f428c016000 comp rx=0 tx=0).stop 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 msgr2=0x7f42980cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 0x7f42980cd5d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f428800b5e0 tx=0x7f4288009120 comp rx=0 tx=0).stop 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 shutdown_connections 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42980c9720 0x7f429814c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f429806c560 0x7f42980cd5d0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 --2- 172.21.15.33:0/2300268991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42980ad520 0x7f42981582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 >> 172.21.15.33:0/2300268991 conn(0x7f42980675f0 msgr2=0x7f42980b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 shutdown_connections 2023-10-06T05:27:05.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:05.961+0000 7f429c99d700 1 -- 172.21.15.33:0/2300268991 wait complete. 2023-10-06T05:27:05.967 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:27:05.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:27:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.176+0000 7f140ffff700 1 Processor -- start 2023-10-06T05:27:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.176+0000 7f140ffff700 1 -- start start 2023-10-06T05:27:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.177+0000 7f140ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f14100b0740 0x7f141015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141015a5f0 con 0x7f14100cea20 2023-10-06T05:27:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14100cf170 con 0x7f141006efa0 2023-10-06T05:27:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1410159fa0 con 0x7f14100b0740 2023-10-06T05:27:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140e7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f14100cea20 0x7f141006e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34136/0 (socket says 172.21.15.33:34136) 2023-10-06T05:27:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140e7fc700 1 -- 172.21.15.33:0/1366537673 learned_addr learned my addr 172.21.15.33:0/1366537673 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140effd700 1 --2- 172.21.15.33:0/1366537673 >> v2:172.21.15.167:3300/0 conn(0x7f14100b0740 0x7f141015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353172210 0 0) 0x7f141015a5f0 con 0x7f14100cea20 2023-10-06T05:27:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.178+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f8003570 con 0x7f14100cea20 2023-10-06T05:27:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354747744 0 0) 0x7f14100cf170 con 0x7f141006efa0 2023-10-06T05:27:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141015a5f0 con 0x7f141006efa0 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248052207 0 0) 0x7f13f8003570 con 0x7f14100cea20 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140effd700 1 -- 172.21.15.33:0/1366537673 >> v1:172.21.15.167:6789/0 conn(0x7f141006efa0 legacy=0x7f14100abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140effd700 1 -- 172.21.15.33:0/1366537673 >> v1:172.21.15.33:6789/0 conn(0x7f14100cea20 legacy=0x7f141006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140effd700 1 -- 172.21.15.33:0/1366537673 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141015a280 con 0x7f14100b0740 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140effd700 1 --2- 172.21.15.33:0/1366537673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100b0740 0x7f141015dd10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f14000098e0 tx=0x7f14000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89edf394a26fbf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404001470 con 0x7f14100b0740 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1400011420 con 0x7f14100b0740 2023-10-06T05:27:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.179+0000 7f140dffb700 1 -- 172.21.15.33:0/1366537673 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14000117c0 con 0x7f14100b0740 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 -- 172.21.15.33:0/1366537673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100b0740 msgr2=0x7f141015dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 --2- 172.21.15.33:0/1366537673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100b0740 0x7f141015dd10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f14000098e0 tx=0x7f14000091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 -- 172.21.15.33:0/1366537673 shutdown_connections 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 --2- 172.21.15.33:0/1366537673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100b0740 0x7f141015dd10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 -- 172.21.15.33:0/1366537673 >> 172.21.15.33:0/1366537673 conn(0x7f141006cd40 msgr2=0x7f14100ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 -- 172.21.15.33:0/1366537673 shutdown_connections 2023-10-06T05:27:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.180+0000 7f140ffff700 1 -- 172.21.15.33:0/1366537673 wait complete. 2023-10-06T05:27:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.181+0000 7f140ffff700 1 Processor -- start 2023-10-06T05:27:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 -- start start 2023-10-06T05:27:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 0x7f14100b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14100b0740 0x7f14100c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100cea20 0x7f1410148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14100b1e90 con 0x7f141006efa0 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141015bb80 con 0x7f14100cea20 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.182+0000 7f140ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141015be60 con 0x7f14100b0740 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 0x7f14100b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 0x7f14100b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40080/0 (socket says 172.21.15.33:40080) 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 -- 172.21.15.33:0/2550362994 learned_addr learned my addr 172.21.15.33:0/2550362994 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140f7fe700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100cea20 0x7f1410148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140e7fc700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14100b0740 0x7f14100c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 -- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14100b0740 msgr2=0x7f14100c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14100b0740 0x7f14100c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 -- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100cea20 msgr2=0x7f1410148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100cea20 0x7f1410148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 -- 172.21.15.33:0/2550362994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400009420 con 0x7f141006efa0 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140f7fe700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100cea20 0x7f1410148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.183+0000 7f140effd700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 0x7f14100b1950 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f14100ade40 tx=0x7f1400016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.184+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140001cc90 con 0x7f141006efa0 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.184+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140001cdf0 con 0x7f141006efa0 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.184+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400025680 con 0x7f141006efa0 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.184+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141015c140 con 0x7f141006efa0 2023-10-06T05:27:06.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.184+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141015c700 con 0x7f141006efa0 2023-10-06T05:27:06.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.185+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141004c380 con 0x7f141006efa0 2023-10-06T05:27:06.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.186+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14000193f0 con 0x7f141006efa0 2023-10-06T05:27:06.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.187+0000 7f13f6ffd700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13fc066f30 0x7f13fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.187+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f1400085880 con 0x7f141006efa0 2023-10-06T05:27:06.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.188+0000 7f140e7fc700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13fc066f30 0x7f13fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.188+0000 7f140e7fc700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13fc066f30 0x7f13fc0693e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1404001ed0 tx=0x7f1404000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:06.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.192+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1400052b50 con 0x7f141006efa0 2023-10-06T05:27:06.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.393+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f141006d350 con 0x7f141006efa0 2023-10-06T05:27:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.966+0000 7f13f6ffd700 1 -- 172.21.15.33:0/2550362994 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v179) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1400056170 con 0x7f141006efa0 2023-10-06T05:27:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.975+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13fc066f30 msgr2=0x7f13fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.975+0000 7f140ffff700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13fc066f30 0x7f13fc0693e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1404001ed0 tx=0x7f1404000bc0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.975+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 msgr2=0x7f14100b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.975+0000 7f140ffff700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 0x7f14100b1950 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f14100ade40 tx=0x7f1400016000 comp rx=0 tx=0).stop 2023-10-06T05:27:06.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.978+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 shutdown_connections 2023-10-06T05:27:06.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.978+0000 7f140ffff700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13fc066f30 0x7f13fc0693e0 secure :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1404001ed0 tx=0x7f1404000bc0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.978+0000 7f140ffff700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14100cea20 0x7f1410148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.978+0000 7f140ffff700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f141006efa0 0x7f14100b1950 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.978+0000 7f140ffff700 1 --2- 172.21.15.33:0/2550362994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14100b0740 0x7f14100c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.978+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 >> 172.21.15.33:0/2550362994 conn(0x7f141006cd40 msgr2=0x7f14101548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:06.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.979+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 shutdown_connections 2023-10-06T05:27:06.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:06.980+0000 7f140ffff700 1 -- 172.21.15.33:0/2550362994 wait complete. 2023-10-06T05:27:06.988 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:27:07.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.198+0000 7f514f59d700 1 Processor -- start 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.199+0000 7f514f59d700 1 -- start start 2023-10-06T05:27:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5148156370 0x7f51480af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514f59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51480cc7e0 0x7f51480c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51480b01f0 0x7f51480b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514f59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480ad430 con 0x7f51480cc7e0 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480afe20 con 0x7f51480b01f0 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480b0b70 con 0x7f5148156370 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51480cc7e0 0x7f51480c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514dd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f51480cc7e0 0x7f51480c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40094/0 (socket says 172.21.15.33:40094) 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.200+0000 7f514dd9a700 1 -- 172.21.15.33:0/2460025596 learned_addr learned my addr 172.21.15.33:0/2460025596 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514e59b700 1 --2- 172.21.15.33:0/2460025596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5148156370 0x7f51480af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514ed9c700 1 --2- 172.21.15.33:0/2460025596 >> v2:172.21.15.167:3300/0 conn(0x7f51480b01f0 0x7f51480b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514dd9a700 1 -- 172.21.15.33:0/2460025596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5148156370 msgr2=0x7f51480af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514dd9a700 1 --2- 172.21.15.33:0/2460025596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5148156370 0x7f51480af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514dd9a700 1 -- 172.21.15.33:0/2460025596 >> v2:172.21.15.167:3300/0 conn(0x7f51480b01f0 msgr2=0x7f51480b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514dd9a700 1 --2- 172.21.15.33:0/2460025596 >> v2:172.21.15.167:3300/0 conn(0x7f51480b01f0 0x7f51480b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514dd9a700 1 -- 172.21.15.33:0/2460025596 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514806bc70 con 0x7f51480cc7e0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514dd9a700 1 --2- 172.21.15.33:0/2460025596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480cc7e0 0x7f51480c8db0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f5148069810 tx=0x7f5138009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157cb51086b41ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514d599700 1 -- 172.21.15.33:0/2460025596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800b860 con 0x7f51480cc7e0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514d599700 1 -- 172.21.15.33:0/2460025596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513800b9c0 con 0x7f51480cc7e0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.201+0000 7f514d599700 1 -- 172.21.15.33:0/2460025596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800bd60 con 0x7f51480cc7e0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 -- 172.21.15.33:0/2460025596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480cc7e0 msgr2=0x7f51480c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 --2- 172.21.15.33:0/2460025596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480cc7e0 0x7f51480c8db0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f5148069810 tx=0x7f5138009250 comp rx=0 tx=0).stop 2023-10-06T05:27:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 -- 172.21.15.33:0/2460025596 shutdown_connections 2023-10-06T05:27:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 --2- 172.21.15.33:0/2460025596 >> v2:172.21.15.167:3300/0 conn(0x7f51480b01f0 0x7f51480b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 --2- 172.21.15.33:0/2460025596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5148156370 0x7f51480af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 --2- 172.21.15.33:0/2460025596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480cc7e0 0x7f51480c8db0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 -- 172.21.15.33:0/2460025596 >> 172.21.15.33:0/2460025596 conn(0x7f514806d030 msgr2=0x7f51480cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 -- 172.21.15.33:0/2460025596 shutdown_connections 2023-10-06T05:27:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.202+0000 7f514f59d700 1 -- 172.21.15.33:0/2460025596 wait complete. 2023-10-06T05:27:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.204+0000 7f514f59d700 1 Processor -- start 2023-10-06T05:27:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.204+0000 7f514f59d700 1 -- start start 2023-10-06T05:27:07.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 0x7f5148145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51480cc7e0 0x7f51480aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5148156370 0x7f51480abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480a9b30 con 0x7f51480b01f0 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480aa990 con 0x7f5148156370 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480a9d80 con 0x7f51480cc7e0 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 0x7f5148145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 0x7f5148145ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40100/0 (socket says 172.21.15.33:40100) 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 -- 172.21.15.33:0/3880057741 learned_addr learned my addr 172.21.15.33:0/3880057741 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514ed9c700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5148156370 0x7f51480abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514dd9a700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51480cc7e0 0x7f51480aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 -- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51480cc7e0 msgr2=0x7f51480aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51480cc7e0 0x7f51480aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 -- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5148156370 msgr2=0x7f51480abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5148156370 0x7f51480abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.205+0000 7f514e59b700 1 -- 172.21.15.33:0/3880057741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380094f0 con 0x7f51480b01f0 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.206+0000 7f514e59b700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 0x7f5148145ea0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f514400d890 tx=0x7f5144008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.206+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144009610 con 0x7f51480b01f0 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.206+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144009770 con 0x7f51480b01f0 2023-10-06T05:27:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.206+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144009b30 con 0x7f51480b01f0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.206+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51480aa060 con 0x7f51480b01f0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.206+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51480aa620 con 0x7f51480b01f0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.207+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514804c380 con 0x7f51480b01f0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.208+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f514401e460 con 0x7f51480b01f0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.209+0000 7f513e7fc700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f512c066f30 0x7f512c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.209+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f5144084d40 con 0x7f51480b01f0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.209+0000 7f514dd9a700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f512c066f30 0x7f512c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.210+0000 7f514dd9a700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f512c066f30 0x7f512c0693e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5148069810 tx=0x7f5138017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.213+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5144051e10 con 0x7f51480b01f0 2023-10-06T05:27:07.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.462+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f514806f310 con 0x7f51480b01f0 2023-10-06T05:27:07.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.973+0000 7f513e7fc700 1 -- 172.21.15.33:0/3880057741 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v129) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5144055430 con 0x7f51480b01f0 2023-10-06T05:27:07.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.983+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f512c066f30 msgr2=0x7f512c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.983+0000 7f514f59d700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f512c066f30 0x7f512c0693e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5148069810 tx=0x7f5138017000 comp rx=0 tx=0).stop 2023-10-06T05:27:07.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.983+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 msgr2=0x7f5148145ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.983+0000 7f514f59d700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 0x7f5148145ea0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f514400d890 tx=0x7f5144008040 comp rx=0 tx=0).stop 2023-10-06T05:27:07.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.984+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 shutdown_connections 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.984+0000 7f514f59d700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f512c066f30 0x7f512c0693e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.984+0000 7f514f59d700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5148156370 0x7f51480abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.984+0000 7f514f59d700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51480b01f0 0x7f5148145ea0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.984+0000 7f514f59d700 1 --2- 172.21.15.33:0/3880057741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51480cc7e0 0x7f51480aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.984+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 >> 172.21.15.33:0/3880057741 conn(0x7f514806d030 msgr2=0x7f514806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.985+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 shutdown_connections 2023-10-06T05:27:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:07.986+0000 7f514f59d700 1 -- 172.21.15.33:0/3880057741 wait complete. 2023-10-06T05:27:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 27 and data pool 28 2023-10-06T05:27:08.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.206+0000 7f6d9119d700 1 Processor -- start 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.207+0000 7f6d9119d700 1 -- start start 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d9119d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d8c155140 0x7f6d8c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d9119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d8c0adf50 0x7f6d8c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d9119d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0c8e40 con 0x7f6d8c0c9810 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d9119d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0ae3f0 con 0x7f6d8c155140 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d9119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0ae6c0 con 0x7f6d8c0adf50 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d8bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d8c155140 0x7f6d8c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d8bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d8c155140 0x7f6d8c158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40116/0 (socket says 172.21.15.33:40116) 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d8bfff700 1 -- 172.21.15.33:0/3196118847 learned_addr learned my addr 172.21.15.33:0/3196118847 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d9099c700 1 --1- 172.21.15.33:0/3196118847 >> v1:172.21.15.33:6789/0 conn(0x7f6d8c0c9810 0x7f6d8c0699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34140/0 (socket says 172.21.15.33:34140) 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.208+0000 7f6d8b7fe700 1 --2- 172.21.15.33:0/3196118847 >> v2:172.21.15.167:3300/0 conn(0x7f6d8c0adf50 0x7f6d8c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8bfff700 1 -- 172.21.15.33:0/3196118847 >> v2:172.21.15.167:3300/0 conn(0x7f6d8c0adf50 msgr2=0x7f6d8c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8bfff700 1 --2- 172.21.15.33:0/3196118847 >> v2:172.21.15.167:3300/0 conn(0x7f6d8c0adf50 0x7f6d8c0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8bfff700 1 -- 172.21.15.33:0/3196118847 >> v1:172.21.15.33:6789/0 conn(0x7f6d8c0c9810 legacy=0x7f6d8c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8bfff700 1 -- 172.21.15.33:0/3196118847 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0ae9a0 con 0x7f6d8c155140 2023-10-06T05:27:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8bfff700 1 --2- 172.21.15.33:0/3196118847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c158700 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099b0 tx=0x7f6d7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf4ecb769eeb1df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8affd700 1 -- 172.21.15.33:0/3196118847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00b820 con 0x7f6d8c155140 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8affd700 1 -- 172.21.15.33:0/3196118847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c00b980 con 0x7f6d8c155140 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.209+0000 7f6d8affd700 1 -- 172.21.15.33:0/3196118847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00bd20 con 0x7f6d8c155140 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 -- 172.21.15.33:0/3196118847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 msgr2=0x7f6d8c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 --2- 172.21.15.33:0/3196118847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c158700 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099b0 tx=0x7f6d7c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 -- 172.21.15.33:0/3196118847 shutdown_connections 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 --2- 172.21.15.33:0/3196118847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c158700 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 --2- 172.21.15.33:0/3196118847 >> v2:172.21.15.167:3300/0 conn(0x7f6d8c0adf50 0x7f6d8c0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 -- 172.21.15.33:0/3196118847 >> 172.21.15.33:0/3196118847 conn(0x7f6d8c06d030 msgr2=0x7f6d8c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 -- 172.21.15.33:0/3196118847 shutdown_connections 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.210+0000 7f6d9119d700 1 -- 172.21.15.33:0/3196118847 wait complete. 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.211+0000 7f6d9119d700 1 Processor -- start 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 -- start start 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c0adf50 0x7f6d8c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0c9810 0x7f6d8c152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0c4b30 con 0x7f6d8c155140 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0c4200 con 0x7f6d8c0adf50 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.212+0000 7f6d9119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0c44e0 con 0x7f6d8c0c9810 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c147f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40132/0 (socket says 172.21.15.33:40132) 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 -- 172.21.15.33:0/4067843295 learned_addr learned my addr 172.21.15.33:0/4067843295 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d8bfff700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c0adf50 0x7f6d8c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d8b7fe700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0c9810 0x7f6d8c152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 -- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0c9810 msgr2=0x7f6d8c152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0c9810 0x7f6d8c152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 -- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c0adf50 msgr2=0x7f6d8c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c0adf50 0x7f6d8c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 -- 172.21.15.33:0/4067843295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0094f0 con 0x7f6d8c155140 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d8bfff700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c0adf50 0x7f6d8c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.213+0000 7f6d9099c700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c147f70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400cd00 tx=0x7f6d84001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.214+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84005df0 con 0x7f6d8c155140 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.214+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8400a960 con 0x7f6d8c155140 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.214+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400ad00 con 0x7f6d8c155140 2023-10-06T05:27:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.214+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c0c47c0 con 0x7f6d8c155140 2023-10-06T05:27:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.214+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c153690 con 0x7f6d8c155140 2023-10-06T05:27:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.215+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c0c4ce0 con 0x7f6d8c155140 2023-10-06T05:27:08.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.216+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d84007300 con 0x7f6d8c155140 2023-10-06T05:27:08.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.216+0000 7f6d73fff700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.217+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d84084850 con 0x7f6d8c155140 2023-10-06T05:27:08.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.217+0000 7f6d8bfff700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.218+0000 7f6d8bfff700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099b0 tx=0x7f6d7c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.221+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d8c0c4ce0 con 0x7f6d8c155140 2023-10-06T05:27:08.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.422+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d8c04c380 con 0x7f6d8c155140 2023-10-06T05:27:08.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.423+0000 7f6d73fff700 1 -- 172.21.15.33:0/4067843295 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f6d84051940 con 0x7f6d8c155140 2023-10-06T05:27:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:08.425 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":180,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:07.463456+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:05.629452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:06.434007+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:08.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 msgr2=0x7f6d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0099b0 tx=0x7f6d7c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:08.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 msgr2=0x7f6d8c147f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c147f70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400cd00 tx=0x7f6d84001b10 comp rx=0 tx=0).stop 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 shutdown_connections 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d78066f30 0x7f6d780693e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d8c155140 0x7f6d8c147f70 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d8c0adf50 0x7f6d8c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 --2- 172.21.15.33:0/4067843295 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d8c0c9810 0x7f6d8c152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 >> 172.21.15.33:0/4067843295 conn(0x7f6d8c06d030 msgr2=0x7f6d8c0c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 shutdown_connections 2023-10-06T05:27:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.425+0000 7f6d9119d700 1 -- 172.21.15.33:0/4067843295 wait complete. 2023-10-06T05:27:08.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.631+0000 7f08f0b9d700 1 Processor -- start 2023-10-06T05:27:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.631+0000 7f08f0b9d700 1 -- start start 2023-10-06T05:27:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.632+0000 7f08f0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 0x7f08ec0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.632+0000 7f08f0b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f08ec0cc750 0x7f08ec0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.632+0000 7f08f0b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0cbd80 con 0x7f08ec0cf3b0 2023-10-06T05:27:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08f0b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec0af5f0 con 0x7f08ec0cc750 2023-10-06T05:27:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08f0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0af8d0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08eb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 0x7f08ec0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08eb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 0x7f08ec0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35490/0 (socket says 172.21.15.33:35490) 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08eb7fe700 1 -- 172.21.15.33:0/3707890823 learned_addr learned my addr 172.21.15.33:0/3707890823 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08eaffd700 1 --1- 172.21.15.33:0/3707890823 >> v1:172.21.15.167:6789/0 conn(0x7f08ec0cf3b0 0x7f08ec1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:35684/0 (socket says 172.21.15.33:35684) 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.633+0000 7f08ebfff700 1 --2- 172.21.15.33:0/3707890823 >> v2:172.21.15.167:3300/0 conn(0x7f08ec0cc750 0x7f08ec0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08eb7fe700 1 -- 172.21.15.33:0/3707890823 >> v2:172.21.15.167:3300/0 conn(0x7f08ec0cc750 msgr2=0x7f08ec0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08eb7fe700 1 --2- 172.21.15.33:0/3707890823 >> v2:172.21.15.167:3300/0 conn(0x7f08ec0cc750 0x7f08ec0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08eb7fe700 1 -- 172.21.15.33:0/3707890823 >> v1:172.21.15.167:6789/0 conn(0x7f08ec0cf3b0 legacy=0x7f08ec1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08eb7fe700 1 -- 172.21.15.33:0/3707890823 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0afbb0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08eb7fe700 1 --2- 172.21.15.33:0/3707890823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 0x7f08ec0cc210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0099b0 tx=0x7f08dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9b9c2c83ea0456 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08ea7fc700 1 -- 172.21.15.33:0/3707890823 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc009ea0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08ea7fc700 1 -- 172.21.15.33:0/3707890823 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00b860 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.634+0000 7f08ea7fc700 1 -- 172.21.15.33:0/3707890823 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00bc00 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 -- 172.21.15.33:0/3707890823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 msgr2=0x7f08ec0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/3707890823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 0x7f08ec0cc210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0099b0 tx=0x7f08dc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 -- 172.21.15.33:0/3707890823 shutdown_connections 2023-10-06T05:27:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/3707890823 >> v2:172.21.15.167:3300/0 conn(0x7f08ec0cc750 0x7f08ec0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/3707890823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec0ccfc0 0x7f08ec0cc210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 -- 172.21.15.33:0/3707890823 >> 172.21.15.33:0/3707890823 conn(0x7f08ec06cff0 msgr2=0x7f08ec06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 -- 172.21.15.33:0/3707890823 shutdown_connections 2023-10-06T05:27:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.635+0000 7f08f0b9d700 1 -- 172.21.15.33:0/3707890823 wait complete. 2023-10-06T05:27:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.636+0000 7f08f0b9d700 1 Processor -- start 2023-10-06T05:27:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.637+0000 7f08f0b9d700 1 -- start start 2023-10-06T05:27:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08f0b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec0cc750 0x7f08ec0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08f0b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 0x7f08ec0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08f0b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec15a230 0x7f08ec15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08f0b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15cf10 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08f0b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15ac10 con 0x7f08ec0cc750 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08f0b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15c0f0 con 0x7f08ec15a230 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 0x7f08ec0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 0x7f08ec0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40144/0 (socket says 172.21.15.33:40144) 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 -- 172.21.15.33:0/4197258157 learned_addr learned my addr 172.21.15.33:0/4197258157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eb7fe700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec0cc750 0x7f08ec0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08ebfff700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec15a230 0x7f08ec15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 -- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec15a230 msgr2=0x7f08ec15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec15a230 0x7f08ec15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 -- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec0cc750 msgr2=0x7f08ec0b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec0cc750 0x7f08ec0b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.638+0000 7f08eaffd700 1 -- 172.21.15.33:0/4197258157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0094f0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.639+0000 7f08eaffd700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 0x7f08ec0cace0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c560 tx=0x7f08e0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.639+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0005a70 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.639+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0005bd0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.639+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec15c390 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.639+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00070d0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.639+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec15c950 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.640+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec0ce310 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.641+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08e0007480 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.642+0000 7f08d37fe700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.642+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08e0083a80 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.642+0000 7f08eb7fe700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.643+0000 7f08eb7fe700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0099b0 tx=0x7f08dc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.646+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08e0050af0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.850+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08ec159bb0 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.851+0000 7f08d37fe700 1 -- 172.21.15.33:0/4197258157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f08e0017020 con 0x7f08ec0ccfc0 2023-10-06T05:27:08.853 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:08.853 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":180,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:07.463456+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:05.629452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:06.434007+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.853+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d8066f30 msgr2=0x7f08d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.853+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0099b0 tx=0x7f08dc017000 comp rx=0 tx=0).stop 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.853+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 msgr2=0x7f08ec0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.853+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 0x7f08ec0cace0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c560 tx=0x7f08e0007f40 comp rx=0 tx=0).stop 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 shutdown_connections 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08ec15a230 0x7f08ec15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08ec0cc750 0x7f08ec0b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 --2- 172.21.15.33:0/4197258157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08ec0ccfc0 0x7f08ec0cace0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 >> 172.21.15.33:0/4197258157 conn(0x7f08ec06cff0 msgr2=0x7f08ec14f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 shutdown_connections 2023-10-06T05:27:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:08.854+0000 7f08f0b9d700 1 -- 172.21.15.33:0/4197258157 wait complete. 2023-10-06T05:27:08.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:27:08.950 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:27:08.949+0000 7f2ae0df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.064+0000 7ff2c3fff700 1 Processor -- start 2023-10-06T05:27:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.064+0000 7ff2c3fff700 1 -- start start 2023-10-06T05:27:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2c40cc7e0 0x7ff2c40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40b01f0 0x7ff2c40b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c40ad430 con 0x7ff2c4156370 2023-10-06T05:27:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c40afe20 con 0x7ff2c40cc7e0 2023-10-06T05:27:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c40b0b70 con 0x7ff2c40b01f0 2023-10-06T05:27:09.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c27fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2c40cc7e0 0x7ff2c40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff2c4156370 0x7ff2c40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34142/0 (socket says 172.21.15.33:34142) 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.065+0000 7ff2c2ffd700 1 -- 172.21.15.33:0/1590791566 learned_addr learned my addr 172.21.15.33:0/1590791566 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c37fe700 1 --2- 172.21.15.33:0/1590791566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40b01f0 0x7ff2c40b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c27fc700 1 -- 172.21.15.33:0/1590791566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40b01f0 msgr2=0x7ff2c40b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c27fc700 1 --2- 172.21.15.33:0/1590791566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40b01f0 0x7ff2c40b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c27fc700 1 -- 172.21.15.33:0/1590791566 >> v1:172.21.15.33:6789/0 conn(0x7ff2c4156370 legacy=0x7ff2c40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c27fc700 1 -- 172.21.15.33:0/1590791566 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c406bc70 con 0x7ff2c40cc7e0 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c1ffb700 1 -- 172.21.15.33:0/1590791566 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867884907 0 0) 0x7ff2c40ad430 con 0x7ff2c4156370 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c27fc700 1 --2- 172.21.15.33:0/1590791566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c40cc7e0 0x7ff2c40af8e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80098e0 tx=0x7ff2b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edafbf77a3cdd58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c1ffb700 1 -- 172.21.15.33:0/1590791566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800b640 con 0x7ff2c40cc7e0 2023-10-06T05:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c1ffb700 1 -- 172.21.15.33:0/1590791566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b800b7a0 con 0x7ff2c40cc7e0 2023-10-06T05:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.066+0000 7ff2c1ffb700 1 -- 172.21.15.33:0/1590791566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800bb40 con 0x7ff2c40cc7e0 2023-10-06T05:27:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 -- 172.21.15.33:0/1590791566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c40cc7e0 msgr2=0x7ff2c40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/1590791566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c40cc7e0 0x7ff2c40af8e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80098e0 tx=0x7ff2b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 -- 172.21.15.33:0/1590791566 shutdown_connections 2023-10-06T05:27:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/1590791566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40b01f0 0x7ff2c40b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/1590791566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c40cc7e0 0x7ff2c40af8e0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 -- 172.21.15.33:0/1590791566 >> 172.21.15.33:0/1590791566 conn(0x7ff2c406d030 msgr2=0x7ff2c40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 -- 172.21.15.33:0/1590791566 shutdown_connections 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.067+0000 7ff2c3fff700 1 -- 172.21.15.33:0/1590791566 wait complete. 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.068+0000 7ff2c3fff700 1 Processor -- start 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.069+0000 7ff2c3fff700 1 -- start start 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.069+0000 7ff2c3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2c40b01f0 0x7ff2c414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40cc7e0 0x7ff2c4159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 0x7ff2c415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4157110 con 0x7ff2c4156370 2023-10-06T05:27:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c40ab3a0 con 0x7ff2c40b01f0 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41567e0 con 0x7ff2c40cc7e0 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 0x7ff2c415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 0x7ff2c415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40150/0 (socket says 172.21.15.33:40150) 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 -- 172.21.15.33:0/2564303697 learned_addr learned my addr 172.21.15.33:0/2564303697 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c27fc700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40cc7e0 0x7ff2c4159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c2ffd700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2c40b01f0 0x7ff2c414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 -- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40cc7e0 msgr2=0x7ff2c4159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40cc7e0 0x7ff2c4159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 -- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2c40b01f0 msgr2=0x7ff2c414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2c40b01f0 0x7ff2c414df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c37fe700 1 -- 172.21.15.33:0/2564303697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b8009420 con 0x7ff2c4156370 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.070+0000 7ff2c27fc700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40cc7e0 0x7ff2c4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.071+0000 7ff2c37fe700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 0x7ff2c415a510 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009df0 tx=0x7ff2bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.071+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00cb50 con 0x7ff2c4156370 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.071+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc00f040 con 0x7ff2c4156370 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.071+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc014480 con 0x7ff2c4156370 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.071+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c4156ac0 con 0x7ff2c4156370 2023-10-06T05:27:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.071+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c414e460 con 0x7ff2c4156370 2023-10-06T05:27:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.072+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c406a350 con 0x7ff2c4156370 2023-10-06T05:27:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.073+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2bc014830 con 0x7ff2c4156370 2023-10-06T05:27:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.074+0000 7ff2aaffd700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2b0066f30 0x7ff2b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.074+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff2bc0853e0 con 0x7ff2c4156370 2023-10-06T05:27:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.075+0000 7ff2c2ffd700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2b0066f30 0x7ff2b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.075+0000 7ff2c2ffd700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2b0066f30 0x7ff2b00693e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40011e0 tx=0x7ff2b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:09.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.078+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2bc052450 con 0x7ff2c4156370 2023-10-06T05:27:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.326+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff2c404c380 con 0x7ff2c4156370 2023-10-06T05:27:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.995+0000 7ff2aaffd700 1 -- 172.21.15.33:0/2564303697 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v131) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff2bc055a70 con 0x7ff2c4156370 2023-10-06T05:27:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2b0066f30 msgr2=0x7ff2b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2b0066f30 0x7ff2b00693e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40011e0 tx=0x7ff2b4009040 comp rx=0 tx=0).stop 2023-10-06T05:27:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 msgr2=0x7ff2c415a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 0x7ff2c415a510 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009df0 tx=0x7ff2bc00b040 comp rx=0 tx=0).stop 2023-10-06T05:27:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 shutdown_connections 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff2b0066f30 0x7ff2b00693e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c4156370 0x7ff2c415a510 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2c40b01f0 0x7ff2c414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 --2- 172.21.15.33:0/2564303697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2c40cc7e0 0x7ff2c4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 >> 172.21.15.33:0/2564303697 conn(0x7ff2c406d030 msgr2=0x7ff2c4069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 shutdown_connections 2023-10-06T05:27:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:09.998+0000 7ff2c3fff700 1 -- 172.21.15.33:0/2564303697 wait complete. 2023-10-06T05:27:10.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:27:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.215+0000 7f3ba499d700 1 Processor -- start 2023-10-06T05:27:10.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.215+0000 7f3ba499d700 1 -- start start 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3ba499d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3ba00abbd0 0x7f3ba0067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3ba499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3ba0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3ba499d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba006ae10 con 0x7f3ba00b1f90 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3ba499d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba00ac050 con 0x7f3ba00abbd0 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3ba499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba006a2d0 con 0x7f3ba0068140 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3b9f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3ba00b1f90 0x7f3ba0070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37420/0 (socket says 172.21.15.33:37420) 2023-10-06T05:27:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.216+0000 7f3b9f7fe700 1 -- 172.21.15.33:0/950561125 learned_addr learned my addr 172.21.15.33:0/950561125 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9effd700 1 --2- 172.21.15.33:0/950561125 >> v2:172.21.15.167:3300/0 conn(0x7f3ba00abbd0 0x7f3ba0067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3ba0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400270149 0 0) 0x7f3ba006ae10 con 0x7f3ba00b1f90 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b88003570 con 0x7f3ba00b1f90 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042949581 0 0) 0x7f3b88003570 con 0x7f3ba00b1f90 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 msgr2=0x7f3ba0159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3ba0159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 >> v2:172.21.15.167:3300/0 conn(0x7f3ba00abbd0 msgr2=0x7f3ba0067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 --2- 172.21.15.33:0/950561125 >> v2:172.21.15.167:3300/0 conn(0x7f3ba00abbd0 0x7f3ba0067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba0068580 con 0x7f3ba00b1f90 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3b90001470 con 0x7f3ba00b1f90 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 >> v1:172.21.15.33:6789/0 conn(0x7f3ba00b1f90 legacy=0x7f3ba0070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b88003ae0 0x7f3b88003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3b88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b88004d50 0x7f3b880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00ac050 con 0x7f3b88004d50 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.217+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba006a2d0 con 0x7f3b88003ae0 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8800bb10 con 0x7f3ba0068140 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b88004d50 0x7f3b880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9f7fe700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b88003ae0 0x7f3b88003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9effd700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3b88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 -- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 msgr2=0x7f3b88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3b88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 -- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b88003ae0 msgr2=0x7f3b88003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b88003ae0 0x7f3b88003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 -- 172.21.15.33:0/950561125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88003570 con 0x7f3b88004d50 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9effd700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ba0068140 0x7f3b88004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.218+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b88004d50 0x7f3b880051c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e240 tx=0x7f3b9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98717c1a238a41c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800eaf0 con 0x7f3b88004d50 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9800f040 con 0x7f3b88004d50 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3b9e7fc700 1 -- 172.21.15.33:0/950561125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800eea0 con 0x7f3b88004d50 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3ba499d700 1 -- 172.21.15.33:0/950561125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b88004d50 msgr2=0x7f3b880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3ba499d700 1 --2- 172.21.15.33:0/950561125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b88004d50 0x7f3b880051c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e240 tx=0x7f3b9800b040 comp rx=0 tx=0).stop 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3b9ffff700 1 -- 172.21.15.33:0/950561125 reap_dead start 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3ba499d700 1 -- 172.21.15.33:0/950561125 shutdown_connections 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3ba499d700 1 -- 172.21.15.33:0/950561125 >> 172.21.15.33:0/950561125 conn(0x7f3ba006ea70 msgr2=0x7f3ba006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3ba499d700 1 -- 172.21.15.33:0/950561125 shutdown_connections 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.219+0000 7f3ba499d700 1 -- 172.21.15.33:0/950561125 wait complete. 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.221+0000 7f3ba499d700 1 Processor -- start 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.221+0000 7f3ba499d700 1 -- start start 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3ba499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b88004d50 0x7f3ba0150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3ba499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba0155670 0x7f3ba0156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3ba499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 0x7f3ba0156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3ba499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba01551e0 con 0x7f3ba0155ae0 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3ba499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba01545d0 con 0x7f3ba0155670 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3ba499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba01548b0 con 0x7f3b88004d50 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 0x7f3ba0156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 0x7f3ba0156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58106/0 (socket says 172.21.15.33:58106) 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 -- 172.21.15.33:0/1787696395 learned_addr learned my addr 172.21.15.33:0/1787696395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9f7fe700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b88004d50 0x7f3ba0150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9effd700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba0155670 0x7f3ba0156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 -- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b88004d50 msgr2=0x7f3ba0150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b88004d50 0x7f3ba0150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 -- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba0155670 msgr2=0x7f3ba0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba0155670 0x7f3ba0156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9ffff700 1 -- 172.21.15.33:0/1787696395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba0154b90 con 0x7f3ba0155ae0 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.222+0000 7f3b9f7fe700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b88004d50 0x7f3ba0150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.223+0000 7f3b9ffff700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 0x7f3ba0156fa0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e210 tx=0x7f3b9801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.223+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801d630 con 0x7f3ba0155ae0 2023-10-06T05:27:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.223+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9801d7b0 con 0x7f3ba0155ae0 2023-10-06T05:27:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.223+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba0154e70 con 0x7f3ba0155ae0 2023-10-06T05:27:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.223+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801db70 con 0x7f3ba0155ae0 2023-10-06T05:27:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.223+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba015daa0 con 0x7f3ba0155ae0 2023-10-06T05:27:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.224+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba004c380 con 0x7f3ba0155ae0 2023-10-06T05:27:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.225+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b98003040 con 0x7f3ba0155ae0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.226+0000 7f3b877fe700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b8c066f30 0x7f3b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.226+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b98085a90 con 0x7f3ba0155ae0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.226+0000 7f3b9f7fe700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b8c066f30 0x7f3b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.227+0000 7f3b9f7fe700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b8c066f30 0x7f3b8c0693e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3b90000e10 tx=0x7f3b90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.230+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b98052b00 con 0x7f3ba0155ae0 2023-10-06T05:27:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:10.477+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3ba0155320 con 0x7f3ba0155ae0 2023-10-06T05:27:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.003+0000 7f3b877fe700 1 -- 172.21.15.33:0/1787696395 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v132) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b98018020 con 0x7f3ba0155ae0 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b8c066f30 msgr2=0x7f3b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b8c066f30 0x7f3b8c0693e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3b90000e10 tx=0x7f3b90009040 comp rx=0 tx=0).stop 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 msgr2=0x7f3ba0156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 0x7f3ba0156fa0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e210 tx=0x7f3b9801a000 comp rx=0 tx=0).stop 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 shutdown_connections 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b8c066f30 0x7f3b8c0693e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ba0155ae0 0x7f3ba0156fa0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b88004d50 0x7f3ba0150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 --2- 172.21.15.33:0/1787696395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba0155670 0x7f3ba0156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 >> 172.21.15.33:0/1787696395 conn(0x7f3ba006ea70 msgr2=0x7f3ba00ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 shutdown_connections 2023-10-06T05:27:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.006+0000 7f3ba499d700 1 -- 172.21.15.33:0/1787696395 wait complete. 2023-10-06T05:27:11.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.214+0000 7fe6fdb9d700 1 Processor -- start 2023-10-06T05:27:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.214+0000 7fe6fdb9d700 1 -- start start 2023-10-06T05:27:11.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fdb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6f80c8b50 0x7fe6f806e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fdb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6f8155f60 0x7fe6f80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fdb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f806ab60 con 0x7fe6f8155af0 2023-10-06T05:27:11.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fdb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f806b7e0 con 0x7fe6f8155f60 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fdb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f806ae40 con 0x7fe6f80c8b50 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fd39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6f8155f60 0x7fe6f80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fd39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6f8155f60 0x7fe6f80c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58112/0 (socket says 172.21.15.33:58112) 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.215+0000 7fe6fd39c700 1 -- 172.21.15.33:0/604980064 learned_addr learned my addr 172.21.15.33:0/604980064 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fcb9b700 1 --2- 172.21.15.33:0/604980064 >> v2:172.21.15.167:3300/0 conn(0x7fe6f80c8b50 0x7fe6f806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fd39c700 1 -- 172.21.15.33:0/604980064 >> v2:172.21.15.167:3300/0 conn(0x7fe6f80c8b50 msgr2=0x7fe6f806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fd39c700 1 --2- 172.21.15.33:0/604980064 >> v2:172.21.15.167:3300/0 conn(0x7fe6f80c8b50 0x7fe6f806e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fd39c700 1 -- 172.21.15.33:0/604980064 >> v1:172.21.15.167:6789/0 conn(0x7fe6f8155af0 legacy=0x7fe6f806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fd39c700 1 -- 172.21.15.33:0/604980064 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f806b120 con 0x7fe6f8155f60 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fcb9b700 1 --2- 172.21.15.33:0/604980064 >> v2:172.21.15.167:3300/0 conn(0x7fe6f80c8b50 0x7fe6f806e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6fd39c700 1 --2- 172.21.15.33:0/604980064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f80c7210 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80697c0 tx=0x7fe6f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e163817d14d1bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6f77fe700 1 -- 172.21.15.33:0/604980064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000e070 con 0x7fe6f8155f60 2023-10-06T05:27:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6f77fe700 1 -- 172.21.15.33:0/604980064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f000ba50 con 0x7fe6f8155f60 2023-10-06T05:27:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.216+0000 7fe6f77fe700 1 -- 172.21.15.33:0/604980064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00092e0 con 0x7fe6f8155f60 2023-10-06T05:27:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/604980064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 msgr2=0x7fe6f80c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/604980064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f80c7210 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80697c0 tx=0x7fe6f0009840 comp rx=0 tx=0).stop 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/604980064 shutdown_connections 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/604980064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f80c7210 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/604980064 >> v2:172.21.15.167:3300/0 conn(0x7fe6f80c8b50 0x7fe6f806e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/604980064 >> 172.21.15.33:0/604980064 conn(0x7fe6f806d030 msgr2=0x7fe6f806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/604980064 shutdown_connections 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.217+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/604980064 wait complete. 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.219+0000 7fe6fdb9d700 1 Processor -- start 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.219+0000 7fe6fdb9d700 1 -- start start 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fdb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6f80c8b50 0x7fe6f816abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fdb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8155af0 0x7fe6f81567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fdb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f8158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fdb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c4160 con 0x7fe6f8155f60 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fdb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c49c0 con 0x7fe6f8155af0 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fdb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f8159470 con 0x7fe6f80c8b50 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f8158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fd39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f8158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58128/0 (socket says 172.21.15.33:58128) 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fd39c700 1 -- 172.21.15.33:0/1436840898 learned_addr learned my addr 172.21.15.33:0/1436840898 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6fcb9b700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6f80c8b50 0x7fe6f816abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.220+0000 7fe6f7fff700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8155af0 0x7fe6f81567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fd39c700 1 -- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6f80c8b50 msgr2=0x7fe6f816abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fd39c700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6f80c8b50 0x7fe6f816abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fd39c700 1 -- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8155af0 msgr2=0x7fe6f81567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fd39c700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8155af0 0x7fe6f81567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fd39c700 1 -- 172.21.15.33:0/1436840898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009ae0 con 0x7fe6f8155f60 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fd39c700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f8158740 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80635d0 tx=0x7fe6f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000be60 con 0x7fe6f8155f60 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0008ce0 con 0x7fe6f8155f60 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00256b0 con 0x7fe6f8155f60 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f80c4370 con 0x7fe6f8155f60 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.221+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f8156d30 con 0x7fe6f8155f60 2023-10-06T05:27:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.222+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f80c8fc0 con 0x7fe6f8155f60 2023-10-06T05:27:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.223+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6f0003040 con 0x7fe6f8155f60 2023-10-06T05:27:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.224+0000 7fe6f4ff9700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6dc066f30 0x7fe6dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.224+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe6f0085c40 con 0x7fe6f8155f60 2023-10-06T05:27:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.224+0000 7fe6fcb9b700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6dc066f30 0x7fe6dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.225+0000 7fe6fcb9b700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6dc066f30 0x7fe6dc0693e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80cc4a0 tx=0x7fe6ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.228+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6f0052d60 con 0x7fe6f8155f60 2023-10-06T05:27:11.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.478+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6f80c4b30 con 0x7fe6f8155f60 2023-10-06T05:27:11.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.479+0000 7fe6f4ff9700 1 -- 172.21.15.33:0/1436840898 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fe6f0056380 con 0x7fe6f8155f60 2023-10-06T05:27:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:11.481 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":132,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:10.994747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9405":{"gid":9405,"name":"c","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/950449971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":950449971},{"type":"v1","addr":"172.21.15.33:6835","nonce":950449971}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.481+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6dc066f30 msgr2=0x7fe6dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.481+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6dc066f30 0x7fe6dc0693e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80cc4a0 tx=0x7fe6ec008040 comp rx=0 tx=0).stop 2023-10-06T05:27:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.481+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 msgr2=0x7fe6f8158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.481+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f8158740 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80635d0 tx=0x7fe6f0018000 comp rx=0 tx=0).stop 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.481+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 shutdown_connections 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.481+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe6dc066f30 0x7fe6dc0693e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.482+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8155f60 0x7fe6f8158740 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.482+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6f80c8b50 0x7fe6f816abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.482+0000 7fe6fdb9d700 1 --2- 172.21.15.33:0/1436840898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8155af0 0x7fe6f81567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.482+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 >> 172.21.15.33:0/1436840898 conn(0x7fe6f806d030 msgr2=0x7fe6f80c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.482+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 shutdown_connections 2023-10-06T05:27:11.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.482+0000 7fe6fdb9d700 1 -- 172.21.15.33:0/1436840898 wait complete. 2023-10-06T05:27:11.485 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 132 2023-10-06T05:27:11.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:11.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.687+0000 7f9bf919d700 1 Processor -- start 2023-10-06T05:27:11.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.687+0000 7f9bf919d700 1 -- start start 2023-10-06T05:27:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.688+0000 7f9bf919d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9bf4158700 0x7f9bf40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf919d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf4158ba0 con 0x7f9bf40699c0 2023-10-06T05:27:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf919d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf40ae490 con 0x7f9bf4155140 2023-10-06T05:27:11.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf919d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40c8e40 con 0x7f9bf4158700 2023-10-06T05:27:11.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf899c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9bf40699c0 0x7f9bf40c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37424/0 (socket says 172.21.15.33:37424) 2023-10-06T05:27:11.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf899c700 1 -- 172.21.15.33:0/360162889 learned_addr learned my addr 172.21.15.33:0/360162889 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:11.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf37fe700 1 --2- 172.21.15.33:0/360162889 >> v2:172.21.15.167:3300/0 conn(0x7f9bf4158700 0x7f9bf40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655673241 0 0) 0x7f9bf4158ba0 con 0x7f9bf40699c0 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.689+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bdc003570 con 0x7f9bf40699c0 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685295692 0 0) 0x7f9bf40ae490 con 0x7f9bf4155140 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf4158ba0 con 0x7f9bf4155140 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf37fe700 1 -- 172.21.15.33:0/360162889 >> v1:172.21.15.167:6789/0 conn(0x7f9bf4155140 legacy=0x7f9bf40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1524130893 0 0) 0x7f9bdc003570 con 0x7f9bf40699c0 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf37fe700 1 -- 172.21.15.33:0/360162889 >> v1:172.21.15.33:6789/0 conn(0x7f9bf40699c0 legacy=0x7f9bf40c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf37fe700 1 -- 172.21.15.33:0/360162889 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40c99e0 con 0x7f9bf4158700 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf37fe700 1 --2- 172.21.15.33:0/360162889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4158700 0x7f9bf40adf50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009910 tx=0x7f9be80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ffff604dae260fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0017f0 con 0x7f9bf4158700 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8011420 con 0x7f9bf4158700 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.690+0000 7f9bf2ffd700 1 -- 172.21.15.33:0/360162889 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80117c0 con 0x7f9bf4158700 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 -- 172.21.15.33:0/360162889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4158700 msgr2=0x7f9bf40adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 --2- 172.21.15.33:0/360162889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4158700 0x7f9bf40adf50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009910 tx=0x7f9be80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 -- 172.21.15.33:0/360162889 shutdown_connections 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 --2- 172.21.15.33:0/360162889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4158700 0x7f9bf40adf50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 -- 172.21.15.33:0/360162889 >> 172.21.15.33:0/360162889 conn(0x7f9bf406d030 msgr2=0x7f9bf406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 -- 172.21.15.33:0/360162889 shutdown_connections 2023-10-06T05:27:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.691+0000 7f9bf919d700 1 -- 172.21.15.33:0/360162889 wait complete. 2023-10-06T05:27:11.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.692+0000 7f9bf919d700 1 Processor -- start 2023-10-06T05:27:11.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.693+0000 7f9bf919d700 1 -- start start 2023-10-06T05:27:11.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.693+0000 7f9bf919d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 0x7f9bf40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.693+0000 7f9bf919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4155140 0x7f9bf40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf4158700 0x7f9bf4144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf919d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4155920 con 0x7f9bf40699c0 2023-10-06T05:27:11.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40b1800 con 0x7f9bf4155140 2023-10-06T05:27:11.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4168c30 con 0x7f9bf4158700 2023-10-06T05:27:11.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 0x7f9bf40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 0x7f9bf40af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58144/0 (socket says 172.21.15.33:58144) 2023-10-06T05:27:11.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 -- 172.21.15.33:0/2115467019 learned_addr learned my addr 172.21.15.33:0/2115467019 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf37fe700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4155140 0x7f9bf40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf899c700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf4158700 0x7f9bf4144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 -- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf4158700 msgr2=0x7f9bf4144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf4158700 0x7f9bf4144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 -- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4155140 msgr2=0x7f9bf40b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4155140 0x7f9bf40b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf3fff700 1 -- 172.21.15.33:0/2115467019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8009420 con 0x7f9bf40699c0 2023-10-06T05:27:11.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.694+0000 7f9bf899c700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf4158700 0x7f9bf4144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.695+0000 7f9bf3fff700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 0x7f9bf40af9c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f9be400c790 tx=0x7f9be4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.695+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4010510 con 0x7f9bf40699c0 2023-10-06T05:27:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.695+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be4010670 con 0x7f9bf40699c0 2023-10-06T05:27:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.695+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf40b0660 con 0x7f9bf40699c0 2023-10-06T05:27:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.695+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4010a30 con 0x7f9bf40699c0 2023-10-06T05:27:11.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.695+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf40b0380 con 0x7f9bf40699c0 2023-10-06T05:27:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.696+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf404c380 con 0x7f9bf40699c0 2023-10-06T05:27:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.697+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9be401b3b0 con 0x7f9bf40699c0 2023-10-06T05:27:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.698+0000 7f9bdbfff700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9be0066f30 0x7f9be00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.698+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9be4084050 con 0x7f9bf40699c0 2023-10-06T05:27:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.698+0000 7f9bf37fe700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9be0066f30 0x7f9be00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.699+0000 7f9bf37fe700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9be0066f30 0x7f9be00693e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9be80093f0 tx=0x7f9be8016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.702+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9be40b55a0 con 0x7f9bf40699c0 2023-10-06T05:27:11.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.903+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf406ec70 con 0x7f9bf40699c0 2023-10-06T05:27:11.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.904+0000 7f9bdbfff700 1 -- 172.21.15.33:0/2115467019 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f9be4017020 con 0x7f9bf40699c0 2023-10-06T05:27:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:11.906 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":181,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:08.974784+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:05.629452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:06.434007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:11.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9be0066f30 msgr2=0x7f9be00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9be0066f30 0x7f9be00693e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9be80093f0 tx=0x7f9be8016000 comp rx=0 tx=0).stop 2023-10-06T05:27:11.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 msgr2=0x7f9bf40af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 0x7f9bf40af9c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f9be400c790 tx=0x7f9be4007e10 comp rx=0 tx=0).stop 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 shutdown_connections 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9be0066f30 0x7f9be00693e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bf4158700 0x7f9bf4144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bf40699c0 0x7f9bf40af9c0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 --2- 172.21.15.33:0/2115467019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4155140 0x7f9bf40b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 >> 172.21.15.33:0/2115467019 conn(0x7f9bf406d030 msgr2=0x7f9bf4158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 shutdown_connections 2023-10-06T05:27:11.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:11.906+0000 7f9bf919d700 1 -- 172.21.15.33:0/2115467019 wait complete. 2023-10-06T05:27:11.926 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:27:11.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:27:12.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.112+0000 7fa8a719d700 1 Processor -- start 2023-10-06T05:27:12.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.113+0000 7fa8a719d700 1 -- start start 2023-10-06T05:27:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a013faa0 0x7fa8a013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00c9cb0 con 0x7fa8a013faa0 2023-10-06T05:27:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00ca270 con 0x7fa8a006aad0 2023-10-06T05:27:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a006b840 con 0x7fa8a00c7940 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a013faa0 0x7fa8a013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a013faa0 0x7fa8a013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58152/0 (socket says 172.21.15.33:58152) 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a619b700 1 -- 172.21.15.33:0/3643615391 learned_addr learned my addr 172.21.15.33:0/3643615391 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a599a700 1 --2- 172.21.15.33:0/3643615391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.114+0000 7fa8a699c700 1 --2- 172.21.15.33:0/3643615391 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a619b700 1 -- 172.21.15.33:0/3643615391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 msgr2=0x7fa8a00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a619b700 1 --2- 172.21.15.33:0/3643615391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a619b700 1 -- 172.21.15.33:0/3643615391 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 msgr2=0x7fa8a006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a619b700 1 --2- 172.21.15.33:0/3643615391 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a619b700 1 -- 172.21.15.33:0/3643615391 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a006bb20 con 0x7fa8a013faa0 2023-10-06T05:27:12.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a619b700 1 --2- 172.21.15.33:0/3643615391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a013fe70 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00cc0b0 tx=0x7fa89c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294db5a9b46ced29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a5199700 1 -- 172.21.15.33:0/3643615391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c00b860 con 0x7fa8a013faa0 2023-10-06T05:27:12.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a5199700 1 -- 172.21.15.33:0/3643615391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c00b9c0 con 0x7fa8a013faa0 2023-10-06T05:27:12.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.115+0000 7fa8a5199700 1 -- 172.21.15.33:0/3643615391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c00bd60 con 0x7fa8a013faa0 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 -- 172.21.15.33:0/3643615391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 msgr2=0x7fa8a013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 --2- 172.21.15.33:0/3643615391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a013fe70 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00cc0b0 tx=0x7fa89c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 -- 172.21.15.33:0/3643615391 shutdown_connections 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 --2- 172.21.15.33:0/3643615391 >> v2:172.21.15.167:3300/0 conn(0x7fa8a006aad0 0x7fa8a006af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 --2- 172.21.15.33:0/3643615391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a013fe70 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 --2- 172.21.15.33:0/3643615391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 -- 172.21.15.33:0/3643615391 >> 172.21.15.33:0/3643615391 conn(0x7fa8a006d030 msgr2=0x7fa8a006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 -- 172.21.15.33:0/3643615391 shutdown_connections 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.116+0000 7fa8a719d700 1 -- 172.21.15.33:0/3643615391 wait complete. 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.117+0000 7fa8a719d700 1 Processor -- start 2023-10-06T05:27:12.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.118+0000 7fa8a719d700 1 -- start start 2023-10-06T05:27:12.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.118+0000 7fa8a719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a013dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.118+0000 7fa8a719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0167860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c6210 con 0x7fa8a013faa0 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c5600 con 0x7fa8a006aad0 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c58e0 con 0x7fa8a00c7940 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0167860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58156/0 (socket says 172.21.15.33:58156) 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 -- 172.21.15.33:0/819591038 learned_addr learned my addr 172.21.15.33:0/819591038 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a619b700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a013dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a599a700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 -- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 msgr2=0x7fa8a00c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 -- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 msgr2=0x7fa8a013dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a013dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.119+0000 7fa8a699c700 1 -- 172.21.15.33:0/819591038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0094f0 con 0x7fa8a013faa0 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.120+0000 7fa8a699c700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0167860 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa898009fa0 tx=0x7fa898009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.120+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89800ebc0 con 0x7fa8a013faa0 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.120+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89800ed20 con 0x7fa8a013faa0 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.120+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a00c5bc0 con 0x7fa8a013faa0 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.120+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898014670 con 0x7fa8a013faa0 2023-10-06T05:27:12.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.120+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a00a68b0 con 0x7fa8a013faa0 2023-10-06T05:27:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.121+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00c63c0 con 0x7fa8a013faa0 2023-10-06T05:27:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.122+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8980149e0 con 0x7fa8a013faa0 2023-10-06T05:27:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.123+0000 7fa8967fc700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa884066f30 0x7fa8840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.123+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa898084d90 con 0x7fa8a013faa0 2023-10-06T05:27:12.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.124+0000 7fa8a619b700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa884066f30 0x7fa8840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.124+0000 7fa8a619b700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa884066f30 0x7fa8840693e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00cc0b0 tx=0x7fa89c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.127+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8a00c63c0 con 0x7fa8a013faa0 2023-10-06T05:27:12.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.325+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8a004c380 con 0x7fa884066f30 2023-10-06T05:27:12.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.335+0000 7fa8967fc700 1 -- 172.21.15.33:0/819591038 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311677 (secure 0 0 0) 0x7fa8a004c380 con 0x7fa884066f30 2023-10-06T05:27:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:12.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa884066f30 msgr2=0x7fa8840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa884066f30 0x7fa8840693e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00cc0b0 tx=0x7fa89c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 msgr2=0x7fa8a0167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0167860 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa898009fa0 tx=0x7fa898009900 comp rx=0 tx=0).stop 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 shutdown_connections 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa884066f30 0x7fa8840693e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a013faa0 0x7fa8a0167860 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a006aad0 0x7fa8a013dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 --2- 172.21.15.33:0/819591038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00c7940 0x7fa8a00c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 >> 172.21.15.33:0/819591038 conn(0x7fa8a006d030 msgr2=0x7fa8a015a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 shutdown_connections 2023-10-06T05:27:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.339+0000 7fa8a719d700 1 -- 172.21.15.33:0/819591038 wait complete. 2023-10-06T05:27:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:27:12.359 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":564,"stamp":"2023-10-06T05:27:12.009081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430108,"kb_used_data":19036,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544612,"statfs":{"total":773094113280,"available":770605682688,"internally_reserved":0,"allocated":19492864,"data_stored":9651259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001272"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996247+0000","last_change":"2023-10-06T05:27:07.991642+0000","last_active":"2023-10-06T05:27:07.996247+0000","last_peered":"2023-10-06T05:27:07.996247+0000","last_clean":"2023-10-06T05:27:07.996247+0000","last_became_active":"2023-10-06T05:27:07.991389+0000","last_became_peered":"2023-10-06T05:27:07.991389+0000","last_unstale":"2023-10-06T05:27:07.996247+0000","last_undegraded":"2023-10-06T05:27:07.996247+0000","last_fullsized":"2023-10-06T05:27:07.996247+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:16.784766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988617+0000","last_change":"2023-10-06T05:27:07.985031+0000","last_active":"2023-10-06T05:27:07.988617+0000","last_peered":"2023-10-06T05:27:07.988617+0000","last_clean":"2023-10-06T05:27:07.988617+0000","last_became_active":"2023-10-06T05:27:07.984712+0000","last_became_peered":"2023-10-06T05:27:07.984712+0000","last_unstale":"2023-10-06T05:27:07.988617+0000","last_undegraded":"2023-10-06T05:27:07.988617+0000","last_fullsized":"2023-10-06T05:27:07.988617+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:18.241590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997445+0000","last_change":"2023-10-06T05:27:07.995267+0000","last_active":"2023-10-06T05:27:07.997445+0000","last_peered":"2023-10-06T05:27:07.997445+0000","last_clean":"2023-10-06T05:27:07.997445+0000","last_became_active":"2023-10-06T05:27:07.994971+0000","last_became_peered":"2023-10-06T05:27:07.994971+0000","last_unstale":"2023-10-06T05:27:07.997445+0000","last_undegraded":"2023-10-06T05:27:07.997445+0000","last_fullsized":"2023-10-06T05:27:07.997445+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:50.860043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990403+0000","last_change":"2023-10-06T05:27:07.984037+0000","last_active":"2023-10-06T05:27:07.990403+0000","last_peered":"2023-10-06T05:27:07.990403+0000","last_clean":"2023-10-06T05:27:07.990403+0000","last_became_active":"2023-10-06T05:27:07.983746+0000","last_became_peered":"2023-10-06T05:27:07.983746+0000","last_unstale":"2023-10-06T05:27:07.990403+0000","last_undegraded":"2023-10-06T05:27:07.990403+0000","last_fullsized":"2023-10-06T05:27:07.990403+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:11.454394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994078+0000","last_change":"2023-10-06T05:27:07.989964+0000","last_active":"2023-10-06T05:27:07.994078+0000","last_peered":"2023-10-06T05:27:07.994078+0000","last_clean":"2023-10-06T05:27:07.994078+0000","last_became_active":"2023-10-06T05:27:07.989268+0000","last_became_peered":"2023-10-06T05:27:07.989268+0000","last_unstale":"2023-10-06T05:27:07.994078+0000","last_undegraded":"2023-10-06T05:27:07.994078+0000","last_fullsized":"2023-10-06T05:27:07.994078+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:16.216941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993621+0000","last_change":"2023-10-06T05:27:07.988801+0000","last_active":"2023-10-06T05:27:07.993621+0000","last_peered":"2023-10-06T05:27:07.993621+0000","last_clean":"2023-10-06T05:27:07.993621+0000","last_became_active":"2023-10-06T05:27:07.988533+0000","last_became_peered":"2023-10-06T05:27:07.988533+0000","last_unstale":"2023-10-06T05:27:07.993621+0000","last_undegraded":"2023-10-06T05:27:07.993621+0000","last_fullsized":"2023-10-06T05:27:07.993621+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:47.711821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.995588+0000","last_change":"2023-10-06T05:27:07.993741+0000","last_active":"2023-10-06T05:27:07.995588+0000","last_peered":"2023-10-06T05:27:07.995588+0000","last_clean":"2023-10-06T05:27:07.995588+0000","last_became_active":"2023-10-06T05:27:07.993467+0000","last_became_peered":"2023-10-06T05:27:07.993467+0000","last_unstale":"2023-10-06T05:27:07.995588+0000","last_undegraded":"2023-10-06T05:27:07.995588+0000","last_fullsized":"2023-10-06T05:27:07.995588+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:14.178515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994533+0000","last_change":"2023-10-06T05:27:07.986376+0000","last_active":"2023-10-06T05:27:07.994533+0000","last_peered":"2023-10-06T05:27:07.994533+0000","last_clean":"2023-10-06T05:27:07.994533+0000","last_became_active":"2023-10-06T05:27:07.986100+0000","last_became_peered":"2023-10-06T05:27:07.986100+0000","last_unstale":"2023-10-06T05:27:07.994533+0000","last_undegraded":"2023-10-06T05:27:07.994533+0000","last_fullsized":"2023-10-06T05:27:07.994533+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:20.789282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999797+0000","last_change":"2023-10-06T05:27:07.992250+0000","last_active":"2023-10-06T05:27:07.999797+0000","last_peered":"2023-10-06T05:27:07.999797+0000","last_clean":"2023-10-06T05:27:07.999797+0000","last_became_active":"2023-10-06T05:27:07.991022+0000","last_became_peered":"2023-10-06T05:27:07.991022+0000","last_unstale":"2023-10-06T05:27:07.999797+0000","last_undegraded":"2023-10-06T05:27:07.999797+0000","last_fullsized":"2023-10-06T05:27:07.999797+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:45.326029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996191+0000","last_change":"2023-10-06T05:27:07.990152+0000","last_active":"2023-10-06T05:27:07.996191+0000","last_peered":"2023-10-06T05:27:07.996191+0000","last_clean":"2023-10-06T05:27:07.996191+0000","last_became_active":"2023-10-06T05:27:07.989388+0000","last_became_peered":"2023-10-06T05:27:07.989388+0000","last_unstale":"2023-10-06T05:27:07.996191+0000","last_undegraded":"2023-10-06T05:27:07.996191+0000","last_fullsized":"2023-10-06T05:27:07.996191+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:57:27.877492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994464+0000","last_change":"2023-10-06T05:27:07.993293+0000","last_active":"2023-10-06T05:27:07.994464+0000","last_peered":"2023-10-06T05:27:07.994464+0000","last_clean":"2023-10-06T05:27:07.994464+0000","last_became_active":"2023-10-06T05:27:07.992651+0000","last_became_peered":"2023-10-06T05:27:07.992651+0000","last_unstale":"2023-10-06T05:27:07.994464+0000","last_undegraded":"2023-10-06T05:27:07.994464+0000","last_fullsized":"2023-10-06T05:27:07.994464+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:32.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989353+0000","last_change":"2023-10-06T05:27:07.981638+0000","last_active":"2023-10-06T05:27:07.989353+0000","last_peered":"2023-10-06T05:27:07.989353+0000","last_clean":"2023-10-06T05:27:07.989353+0000","last_became_active":"2023-10-06T05:27:07.981285+0000","last_became_peered":"2023-10-06T05:27:07.981285+0000","last_unstale":"2023-10-06T05:27:07.989353+0000","last_undegraded":"2023-10-06T05:27:07.989353+0000","last_fullsized":"2023-10-06T05:27:07.989353+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:26.893361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.005853+0000","last_change":"2023-10-06T05:27:07.999647+0000","last_active":"2023-10-06T05:27:08.005853+0000","last_peered":"2023-10-06T05:27:08.005853+0000","last_clean":"2023-10-06T05:27:08.005853+0000","last_became_active":"2023-10-06T05:27:07.999109+0000","last_became_peered":"2023-10-06T05:27:07.999109+0000","last_unstale":"2023-10-06T05:27:08.005853+0000","last_undegraded":"2023-10-06T05:27:08.005853+0000","last_fullsized":"2023-10-06T05:27:08.005853+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:28.210701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998278+0000","last_change":"2023-10-06T05:27:07.995619+0000","last_active":"2023-10-06T05:27:07.998278+0000","last_peered":"2023-10-06T05:27:07.998278+0000","last_clean":"2023-10-06T05:27:07.998278+0000","last_became_active":"2023-10-06T05:27:07.995318+0000","last_became_peered":"2023-10-06T05:27:07.995318+0000","last_unstale":"2023-10-06T05:27:07.998278+0000","last_undegraded":"2023-10-06T05:27:07.998278+0000","last_fullsized":"2023-10-06T05:27:07.998278+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:58.717175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.984529+0000","last_change":"2023-10-06T05:27:07.982150+0000","last_active":"2023-10-06T05:27:07.984529+0000","last_peered":"2023-10-06T05:27:07.984529+0000","last_clean":"2023-10-06T05:27:07.984529+0000","last_became_active":"2023-10-06T05:27:07.981841+0000","last_became_peered":"2023-10-06T05:27:07.981841+0000","last_unstale":"2023-10-06T05:27:07.984529+0000","last_undegraded":"2023-10-06T05:27:07.984529+0000","last_fullsized":"2023-10-06T05:27:07.984529+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:48.118214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989981+0000","last_change":"2023-10-06T05:27:07.988162+0000","last_active":"2023-10-06T05:27:07.989981+0000","last_peered":"2023-10-06T05:27:07.989981+0000","last_clean":"2023-10-06T05:27:07.989981+0000","last_became_active":"2023-10-06T05:27:07.987912+0000","last_became_peered":"2023-10-06T05:27:07.987912+0000","last_unstale":"2023-10-06T05:27:07.989981+0000","last_undegraded":"2023-10-06T05:27:07.989981+0000","last_fullsized":"2023-10-06T05:27:07.989981+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:11.454394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988174+0000","last_change":"2023-10-06T05:27:07.986863+0000","last_active":"2023-10-06T05:27:07.988174+0000","last_peered":"2023-10-06T05:27:07.988174+0000","last_clean":"2023-10-06T05:27:07.988174+0000","last_became_active":"2023-10-06T05:27:07.986554+0000","last_became_peered":"2023-10-06T05:27:07.986554+0000","last_unstale":"2023-10-06T05:27:07.988174+0000","last_undegraded":"2023-10-06T05:27:07.988174+0000","last_fullsized":"2023-10-06T05:27:07.988174+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:03.557080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.992771+0000","last_change":"2023-10-06T05:27:07.990735+0000","last_active":"2023-10-06T05:27:07.992771+0000","last_peered":"2023-10-06T05:27:07.992771+0000","last_clean":"2023-10-06T05:27:07.992771+0000","last_became_active":"2023-10-06T05:27:07.988575+0000","last_became_peered":"2023-10-06T05:27:07.988575+0000","last_unstale":"2023-10-06T05:27:07.992771+0000","last_undegraded":"2023-10-06T05:27:07.992771+0000","last_fullsized":"2023-10-06T05:27:07.992771+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:07.824570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988176+0000","last_change":"2023-10-06T05:27:07.982588+0000","last_active":"2023-10-06T05:27:07.988176+0000","last_peered":"2023-10-06T05:27:07.988176+0000","last_clean":"2023-10-06T05:27:07.988176+0000","last_became_active":"2023-10-06T05:27:07.981946+0000","last_became_peered":"2023-10-06T05:27:07.981946+0000","last_unstale":"2023-10-06T05:27:07.988176+0000","last_undegraded":"2023-10-06T05:27:07.988176+0000","last_fullsized":"2023-10-06T05:27:07.988176+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:04.410349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988114+0000","last_change":"2023-10-06T05:27:07.983911+0000","last_active":"2023-10-06T05:27:07.988114+0000","last_peered":"2023-10-06T05:27:07.988114+0000","last_clean":"2023-10-06T05:27:07.988114+0000","last_became_active":"2023-10-06T05:27:07.983655+0000","last_became_peered":"2023-10-06T05:27:07.983655+0000","last_unstale":"2023-10-06T05:27:07.988114+0000","last_undegraded":"2023-10-06T05:27:07.988114+0000","last_fullsized":"2023-10-06T05:27:07.988114+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:16.216941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996489+0000","last_change":"2023-10-06T05:27:07.990126+0000","last_active":"2023-10-06T05:27:07.996489+0000","last_peered":"2023-10-06T05:27:07.996489+0000","last_clean":"2023-10-06T05:27:07.996489+0000","last_became_active":"2023-10-06T05:27:07.989603+0000","last_became_peered":"2023-10-06T05:27:07.989603+0000","last_unstale":"2023-10-06T05:27:07.996489+0000","last_undegraded":"2023-10-06T05:27:07.996489+0000","last_fullsized":"2023-10-06T05:27:07.996489+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:27.864903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998259+0000","last_change":"2023-10-06T05:27:07.995017+0000","last_active":"2023-10-06T05:27:07.998259+0000","last_peered":"2023-10-06T05:27:07.998259+0000","last_clean":"2023-10-06T05:27:07.998259+0000","last_became_active":"2023-10-06T05:27:07.994725+0000","last_became_peered":"2023-10-06T05:27:07.994725+0000","last_unstale":"2023-10-06T05:27:07.998259+0000","last_undegraded":"2023-10-06T05:27:07.998259+0000","last_fullsized":"2023-10-06T05:27:07.998259+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:48.760673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997206+0000","last_change":"2023-10-06T05:27:07.995673+0000","last_active":"2023-10-06T05:27:07.997206+0000","last_peered":"2023-10-06T05:27:07.997206+0000","last_clean":"2023-10-06T05:27:07.997206+0000","last_became_active":"2023-10-06T05:27:07.995374+0000","last_became_peered":"2023-10-06T05:27:07.995374+0000","last_unstale":"2023-10-06T05:27:07.997206+0000","last_undegraded":"2023-10-06T05:27:07.997206+0000","last_fullsized":"2023-10-06T05:27:07.997206+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:51.828771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999368+0000","last_change":"2023-10-06T05:27:07.997721+0000","last_active":"2023-10-06T05:27:07.999368+0000","last_peered":"2023-10-06T05:27:07.999368+0000","last_clean":"2023-10-06T05:27:07.999368+0000","last_became_active":"2023-10-06T05:27:07.997338+0000","last_became_peered":"2023-10-06T05:27:07.997338+0000","last_unstale":"2023-10-06T05:27:07.999368+0000","last_undegraded":"2023-10-06T05:27:07.999368+0000","last_fullsized":"2023-10-06T05:27:07.999368+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:39.023585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.992348+0000","last_change":"2023-10-06T05:27:07.990039+0000","last_active":"2023-10-06T05:27:07.992348+0000","last_peered":"2023-10-06T05:27:07.992348+0000","last_clean":"2023-10-06T05:27:07.992348+0000","last_became_active":"2023-10-06T05:27:07.989750+0000","last_became_peered":"2023-10-06T05:27:07.989750+0000","last_unstale":"2023-10-06T05:27:07.992348+0000","last_undegraded":"2023-10-06T05:27:07.992348+0000","last_fullsized":"2023-10-06T05:27:07.992348+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:10.321422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996497+0000","last_change":"2023-10-06T05:27:07.992261+0000","last_active":"2023-10-06T05:27:07.996497+0000","last_peered":"2023-10-06T05:27:07.996497+0000","last_clean":"2023-10-06T05:27:07.996497+0000","last_became_active":"2023-10-06T05:27:07.992014+0000","last_became_peered":"2023-10-06T05:27:07.992014+0000","last_unstale":"2023-10-06T05:27:07.996497+0000","last_undegraded":"2023-10-06T05:27:07.996497+0000","last_fullsized":"2023-10-06T05:27:07.996497+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:16.872804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994446+0000","last_change":"2023-10-06T05:27:07.993013+0000","last_active":"2023-10-06T05:27:07.994446+0000","last_peered":"2023-10-06T05:27:07.994446+0000","last_clean":"2023-10-06T05:27:07.994446+0000","last_became_active":"2023-10-06T05:27:07.992195+0000","last_became_peered":"2023-10-06T05:27:07.992195+0000","last_unstale":"2023-10-06T05:27:07.994446+0000","last_undegraded":"2023-10-06T05:27:07.994446+0000","last_fullsized":"2023-10-06T05:27:07.994446+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:18.893704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989376+0000","last_change":"2023-10-06T05:27:07.984408+0000","last_active":"2023-10-06T05:27:07.989376+0000","last_peered":"2023-10-06T05:27:07.989376+0000","last_clean":"2023-10-06T05:27:07.989376+0000","last_became_active":"2023-10-06T05:27:07.984137+0000","last_became_peered":"2023-10-06T05:27:07.984137+0000","last_unstale":"2023-10-06T05:27:07.989376+0000","last_undegraded":"2023-10-06T05:27:07.989376+0000","last_fullsized":"2023-10-06T05:27:07.989376+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:41.310817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.000430+0000","last_change":"2023-10-06T05:27:07.997629+0000","last_active":"2023-10-06T05:27:08.000430+0000","last_peered":"2023-10-06T05:27:08.000430+0000","last_clean":"2023-10-06T05:27:08.000430+0000","last_became_active":"2023-10-06T05:27:07.997387+0000","last_became_peered":"2023-10-06T05:27:07.997387+0000","last_unstale":"2023-10-06T05:27:08.000430+0000","last_undegraded":"2023-10-06T05:27:08.000430+0000","last_fullsized":"2023-10-06T05:27:08.000430+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:15.267008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996217+0000","last_change":"2023-10-06T05:27:07.990036+0000","last_active":"2023-10-06T05:27:07.996217+0000","last_peered":"2023-10-06T05:27:07.996217+0000","last_clean":"2023-10-06T05:27:07.996217+0000","last_became_active":"2023-10-06T05:27:07.989767+0000","last_became_peered":"2023-10-06T05:27:07.989767+0000","last_unstale":"2023-10-06T05:27:07.996217+0000","last_undegraded":"2023-10-06T05:27:07.996217+0000","last_fullsized":"2023-10-06T05:27:07.996217+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:27.762816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994543+0000","last_change":"2023-10-06T05:27:07.987112+0000","last_active":"2023-10-06T05:27:07.994543+0000","last_peered":"2023-10-06T05:27:07.994543+0000","last_clean":"2023-10-06T05:27:07.994543+0000","last_became_active":"2023-10-06T05:27:07.986786+0000","last_became_peered":"2023-10-06T05:27:07.986786+0000","last_unstale":"2023-10-06T05:27:07.994543+0000","last_undegraded":"2023-10-06T05:27:07.994543+0000","last_fullsized":"2023-10-06T05:27:07.994543+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:34.747786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997763+0000","last_change":"2023-10-06T05:27:07.992743+0000","last_active":"2023-10-06T05:27:07.997763+0000","last_peered":"2023-10-06T05:27:07.997763+0000","last_clean":"2023-10-06T05:27:07.997763+0000","last_became_active":"2023-10-06T05:27:07.992480+0000","last_became_peered":"2023-10-06T05:27:07.992480+0000","last_unstale":"2023-10-06T05:27:07.997763+0000","last_undegraded":"2023-10-06T05:27:07.997763+0000","last_fullsized":"2023-10-06T05:27:07.997763+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:21.534869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.984603+0000","last_change":"2023-10-06T05:27:07.981995+0000","last_active":"2023-10-06T05:27:07.984603+0000","last_peered":"2023-10-06T05:27:07.984603+0000","last_clean":"2023-10-06T05:27:07.984603+0000","last_became_active":"2023-10-06T05:27:07.981679+0000","last_became_peered":"2023-10-06T05:27:07.981679+0000","last_unstale":"2023-10-06T05:27:07.984603+0000","last_undegraded":"2023-10-06T05:27:07.984603+0000","last_fullsized":"2023-10-06T05:27:07.984603+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:16.784766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.968702+0000","last_change":"2023-10-06T05:27:06.988233+0000","last_active":"2023-10-06T05:27:07.968702+0000","last_peered":"2023-10-06T05:27:07.968702+0000","last_clean":"2023-10-06T05:27:07.968702+0000","last_became_active":"2023-10-06T05:27:06.987901+0000","last_became_peered":"2023-10-06T05:27:06.987901+0000","last_unstale":"2023-10-06T05:27:07.968702+0000","last_undegraded":"2023-10-06T05:27:07.968702+0000","last_fullsized":"2023-10-06T05:27:07.968702+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:10.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993450+0000","last_change":"2023-10-06T05:27:07.991400+0000","last_active":"2023-10-06T05:27:07.993450+0000","last_peered":"2023-10-06T05:27:07.993450+0000","last_clean":"2023-10-06T05:27:07.993450+0000","last_became_active":"2023-10-06T05:27:07.991101+0000","last_became_peered":"2023-10-06T05:27:07.991101+0000","last_unstale":"2023-10-06T05:27:07.993450+0000","last_undegraded":"2023-10-06T05:27:07.993450+0000","last_fullsized":"2023-10-06T05:27:07.993450+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979957+0000","last_change":"2023-10-06T05:27:06.990409+0000","last_active":"2023-10-06T05:27:07.979957+0000","last_peered":"2023-10-06T05:27:07.979957+0000","last_clean":"2023-10-06T05:27:07.979957+0000","last_became_active":"2023-10-06T05:27:06.990038+0000","last_became_peered":"2023-10-06T05:27:06.990038+0000","last_unstale":"2023-10-06T05:27:07.979957+0000","last_undegraded":"2023-10-06T05:27:07.979957+0000","last_fullsized":"2023-10-06T05:27:07.979957+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:26.755989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999719+0000","last_change":"2023-10-06T05:27:07.992949+0000","last_active":"2023-10-06T05:27:07.999719+0000","last_peered":"2023-10-06T05:27:07.999719+0000","last_clean":"2023-10-06T05:27:07.999719+0000","last_became_active":"2023-10-06T05:27:07.991076+0000","last_became_peered":"2023-10-06T05:27:07.991076+0000","last_unstale":"2023-10-06T05:27:07.999719+0000","last_undegraded":"2023-10-06T05:27:07.999719+0000","last_fullsized":"2023-10-06T05:27:07.999719+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:30.178173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"180'3","reported_seq":13,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998253+0000","last_change":"2023-10-06T05:27:06.981081+0000","last_active":"2023-10-06T05:27:07.998253+0000","last_peered":"2023-10-06T05:27:07.998253+0000","last_clean":"2023-10-06T05:27:07.998253+0000","last_became_active":"2023-10-06T05:27:06.980366+0000","last_became_peered":"2023-10-06T05:27:06.980366+0000","last_unstale":"2023-10-06T05:27:07.998253+0000","last_undegraded":"2023-10-06T05:27:07.998253+0000","last_fullsized":"2023-10-06T05:27:07.998253+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:33.740958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990047+0000","last_change":"2023-10-06T05:27:07.982074+0000","last_active":"2023-10-06T05:27:07.990047+0000","last_peered":"2023-10-06T05:27:07.990047+0000","last_clean":"2023-10-06T05:27:07.990047+0000","last_became_active":"2023-10-06T05:27:07.981810+0000","last_became_peered":"2023-10-06T05:27:07.981810+0000","last_unstale":"2023-10-06T05:27:07.990047+0000","last_undegraded":"2023-10-06T05:27:07.990047+0000","last_fullsized":"2023-10-06T05:27:07.990047+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:20.789282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"180'6","reported_seq":16,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.006107+0000","last_change":"2023-10-06T05:27:06.983941+0000","last_active":"2023-10-06T05:27:08.006107+0000","last_peered":"2023-10-06T05:27:08.006107+0000","last_clean":"2023-10-06T05:27:08.006107+0000","last_became_active":"2023-10-06T05:27:06.983665+0000","last_became_peered":"2023-10-06T05:27:06.983665+0000","last_unstale":"2023-10-06T05:27:08.006107+0000","last_undegraded":"2023-10-06T05:27:08.006107+0000","last_fullsized":"2023-10-06T05:27:08.006107+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:13.579090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989039+0000","last_change":"2023-10-06T05:27:07.985258+0000","last_active":"2023-10-06T05:27:07.989039+0000","last_peered":"2023-10-06T05:27:07.989039+0000","last_clean":"2023-10-06T05:27:07.989039+0000","last_became_active":"2023-10-06T05:27:07.984565+0000","last_became_peered":"2023-10-06T05:27:07.984565+0000","last_unstale":"2023-10-06T05:27:07.989039+0000","last_undegraded":"2023-10-06T05:27:07.989039+0000","last_fullsized":"2023-10-06T05:27:07.989039+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:58.087999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.000142+0000","last_change":"2023-10-06T05:27:06.989280+0000","last_active":"2023-10-06T05:27:08.000142+0000","last_peered":"2023-10-06T05:27:08.000142+0000","last_clean":"2023-10-06T05:27:08.000142+0000","last_became_active":"2023-10-06T05:27:06.988952+0000","last_became_peered":"2023-10-06T05:27:06.988952+0000","last_unstale":"2023-10-06T05:27:08.000142+0000","last_undegraded":"2023-10-06T05:27:08.000142+0000","last_fullsized":"2023-10-06T05:27:08.000142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:01.310099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990012+0000","last_change":"2023-10-06T05:27:07.988980+0000","last_active":"2023-10-06T05:27:07.990012+0000","last_peered":"2023-10-06T05:27:07.990012+0000","last_clean":"2023-10-06T05:27:07.990012+0000","last_became_active":"2023-10-06T05:27:07.988707+0000","last_became_peered":"2023-10-06T05:27:07.988707+0000","last_unstale":"2023-10-06T05:27:07.990012+0000","last_undegraded":"2023-10-06T05:27:07.990012+0000","last_fullsized":"2023-10-06T05:27:07.990012+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:22.842577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976420+0000","last_change":"2023-10-06T05:27:06.981974+0000","last_active":"2023-10-06T05:27:07.976420+0000","last_peered":"2023-10-06T05:27:07.976420+0000","last_clean":"2023-10-06T05:27:07.976420+0000","last_became_active":"2023-10-06T05:27:06.981590+0000","last_became_peered":"2023-10-06T05:27:06.981590+0000","last_unstale":"2023-10-06T05:27:07.976420+0000","last_undegraded":"2023-10-06T05:27:07.976420+0000","last_fullsized":"2023-10-06T05:27:07.976420+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:37.739301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999690+0000","last_change":"2023-10-06T05:27:07.992673+0000","last_active":"2023-10-06T05:27:07.999690+0000","last_peered":"2023-10-06T05:27:07.999690+0000","last_clean":"2023-10-06T05:27:07.999690+0000","last_became_active":"2023-10-06T05:27:07.991372+0000","last_became_peered":"2023-10-06T05:27:07.991372+0000","last_unstale":"2023-10-06T05:27:07.999690+0000","last_undegraded":"2023-10-06T05:27:07.999690+0000","last_fullsized":"2023-10-06T05:27:07.999690+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:47.460002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976276+0000","last_change":"2023-10-06T05:27:06.980027+0000","last_active":"2023-10-06T05:27:07.976276+0000","last_peered":"2023-10-06T05:27:07.976276+0000","last_clean":"2023-10-06T05:27:07.976276+0000","last_became_active":"2023-10-06T05:27:06.979718+0000","last_became_peered":"2023-10-06T05:27:06.979718+0000","last_unstale":"2023-10-06T05:27:07.976276+0000","last_undegraded":"2023-10-06T05:27:07.976276+0000","last_fullsized":"2023-10-06T05:27:07.976276+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:23.972834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996724+0000","last_change":"2023-10-06T05:27:07.992111+0000","last_active":"2023-10-06T05:27:07.996724+0000","last_peered":"2023-10-06T05:27:07.996724+0000","last_clean":"2023-10-06T05:27:07.996724+0000","last_became_active":"2023-10-06T05:27:07.991829+0000","last_became_peered":"2023-10-06T05:27:07.991829+0000","last_unstale":"2023-10-06T05:27:07.996724+0000","last_undegraded":"2023-10-06T05:27:07.996724+0000","last_fullsized":"2023-10-06T05:27:07.996724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:24.979661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"180'8","reported_seq":18,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.007669+0000","last_change":"2023-10-06T05:27:06.986822+0000","last_active":"2023-10-06T05:27:08.007669+0000","last_peered":"2023-10-06T05:27:08.007669+0000","last_clean":"2023-10-06T05:27:08.007669+0000","last_became_active":"2023-10-06T05:27:06.986496+0000","last_became_peered":"2023-10-06T05:27:06.986496+0000","last_unstale":"2023-10-06T05:27:08.007669+0000","last_undegraded":"2023-10-06T05:27:08.007669+0000","last_fullsized":"2023-10-06T05:27:08.007669+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:58.995025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998183+0000","last_change":"2023-10-06T05:27:07.995086+0000","last_active":"2023-10-06T05:27:07.998183+0000","last_peered":"2023-10-06T05:27:07.998183+0000","last_clean":"2023-10-06T05:27:07.998183+0000","last_became_active":"2023-10-06T05:27:07.993671+0000","last_became_peered":"2023-10-06T05:27:07.993671+0000","last_unstale":"2023-10-06T05:27:07.998183+0000","last_undegraded":"2023-10-06T05:27:07.998183+0000","last_fullsized":"2023-10-06T05:27:07.998183+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:19.937832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.001180+0000","last_change":"2023-10-06T05:27:06.987881+0000","last_active":"2023-10-06T05:27:08.001180+0000","last_peered":"2023-10-06T05:27:08.001180+0000","last_clean":"2023-10-06T05:27:08.001180+0000","last_became_active":"2023-10-06T05:27:06.987586+0000","last_became_peered":"2023-10-06T05:27:06.987586+0000","last_unstale":"2023-10-06T05:27:08.001180+0000","last_undegraded":"2023-10-06T05:27:08.001180+0000","last_fullsized":"2023-10-06T05:27:08.001180+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:17.361942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996776+0000","last_change":"2023-10-06T05:27:07.991139+0000","last_active":"2023-10-06T05:27:07.996776+0000","last_peered":"2023-10-06T05:27:07.996776+0000","last_clean":"2023-10-06T05:27:07.996776+0000","last_became_active":"2023-10-06T05:27:07.990861+0000","last_became_peered":"2023-10-06T05:27:07.990861+0000","last_unstale":"2023-10-06T05:27:07.996776+0000","last_undegraded":"2023-10-06T05:27:07.996776+0000","last_fullsized":"2023-10-06T05:27:07.996776+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:06.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996439+0000","last_change":"2023-10-06T05:27:06.985851+0000","last_active":"2023-10-06T05:27:07.996439+0000","last_peered":"2023-10-06T05:27:07.996439+0000","last_clean":"2023-10-06T05:27:07.996439+0000","last_became_active":"2023-10-06T05:27:06.985549+0000","last_became_peered":"2023-10-06T05:27:06.985549+0000","last_unstale":"2023-10-06T05:27:07.996439+0000","last_undegraded":"2023-10-06T05:27:07.996439+0000","last_fullsized":"2023-10-06T05:27:07.996439+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:35.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996828+0000","last_change":"2023-10-06T05:27:07.992748+0000","last_active":"2023-10-06T05:27:07.996828+0000","last_peered":"2023-10-06T05:27:07.996828+0000","last_clean":"2023-10-06T05:27:07.996828+0000","last_became_active":"2023-10-06T05:27:07.992475+0000","last_became_peered":"2023-10-06T05:27:07.992475+0000","last_unstale":"2023-10-06T05:27:07.996828+0000","last_undegraded":"2023-10-06T05:27:07.996828+0000","last_fullsized":"2023-10-06T05:27:07.996828+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:44.294386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.001794+0000","last_change":"2023-10-06T05:27:06.985453+0000","last_active":"2023-10-06T05:27:08.001794+0000","last_peered":"2023-10-06T05:27:08.001794+0000","last_clean":"2023-10-06T05:27:08.001794+0000","last_became_active":"2023-10-06T05:27:06.985167+0000","last_became_peered":"2023-10-06T05:27:06.985167+0000","last_unstale":"2023-10-06T05:27:08.001794+0000","last_undegraded":"2023-10-06T05:27:08.001794+0000","last_fullsized":"2023-10-06T05:27:08.001794+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:36.096363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.991904+0000","last_change":"2023-10-06T05:27:07.986527+0000","last_active":"2023-10-06T05:27:07.991904+0000","last_peered":"2023-10-06T05:27:07.991904+0000","last_clean":"2023-10-06T05:27:07.991904+0000","last_became_active":"2023-10-06T05:27:07.986197+0000","last_became_peered":"2023-10-06T05:27:07.986197+0000","last_unstale":"2023-10-06T05:27:07.991904+0000","last_undegraded":"2023-10-06T05:27:07.991904+0000","last_fullsized":"2023-10-06T05:27:07.991904+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:26.893361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.969415+0000","last_change":"2023-10-06T05:27:06.987452+0000","last_active":"2023-10-06T05:27:07.969415+0000","last_peered":"2023-10-06T05:27:07.969415+0000","last_clean":"2023-10-06T05:27:07.969415+0000","last_became_active":"2023-10-06T05:27:06.986312+0000","last_became_peered":"2023-10-06T05:27:06.986312+0000","last_unstale":"2023-10-06T05:27:07.969415+0000","last_undegraded":"2023-10-06T05:27:07.969415+0000","last_fullsized":"2023-10-06T05:27:07.969415+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:26.000585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989065+0000","last_change":"2023-10-06T05:27:07.985441+0000","last_active":"2023-10-06T05:27:07.989065+0000","last_peered":"2023-10-06T05:27:07.989065+0000","last_clean":"2023-10-06T05:27:07.989065+0000","last_became_active":"2023-10-06T05:27:07.984889+0000","last_became_peered":"2023-10-06T05:27:07.984889+0000","last_unstale":"2023-10-06T05:27:07.989065+0000","last_undegraded":"2023-10-06T05:27:07.989065+0000","last_fullsized":"2023-10-06T05:27:07.989065+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:31.065787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.004006+0000","last_change":"2023-10-06T05:27:06.988284+0000","last_active":"2023-10-06T05:27:08.004006+0000","last_peered":"2023-10-06T05:27:08.004006+0000","last_clean":"2023-10-06T05:27:08.004006+0000","last_became_active":"2023-10-06T05:27:06.988024+0000","last_became_peered":"2023-10-06T05:27:06.988024+0000","last_unstale":"2023-10-06T05:27:08.004006+0000","last_undegraded":"2023-10-06T05:27:08.004006+0000","last_fullsized":"2023-10-06T05:27:08.004006+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:37.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997798+0000","last_change":"2023-10-06T05:27:07.992349+0000","last_active":"2023-10-06T05:27:07.997798+0000","last_peered":"2023-10-06T05:27:07.997798+0000","last_clean":"2023-10-06T05:27:07.997798+0000","last_became_active":"2023-10-06T05:27:07.992065+0000","last_became_peered":"2023-10-06T05:27:07.992065+0000","last_unstale":"2023-10-06T05:27:07.997798+0000","last_undegraded":"2023-10-06T05:27:07.997798+0000","last_fullsized":"2023-10-06T05:27:07.997798+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:26.794709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.969901+0000","last_change":"2023-10-06T05:27:06.985860+0000","last_active":"2023-10-06T05:27:07.969901+0000","last_peered":"2023-10-06T05:27:07.969901+0000","last_clean":"2023-10-06T05:27:07.969901+0000","last_became_active":"2023-10-06T05:27:06.985492+0000","last_became_peered":"2023-10-06T05:27:06.985492+0000","last_unstale":"2023-10-06T05:27:07.969901+0000","last_undegraded":"2023-10-06T05:27:07.969901+0000","last_fullsized":"2023-10-06T05:27:07.969901+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:10.396573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999821+0000","last_change":"2023-10-06T05:27:07.995392+0000","last_active":"2023-10-06T05:27:07.999821+0000","last_peered":"2023-10-06T05:27:07.999821+0000","last_clean":"2023-10-06T05:27:07.999821+0000","last_became_active":"2023-10-06T05:27:07.995121+0000","last_became_peered":"2023-10-06T05:27:07.995121+0000","last_unstale":"2023-10-06T05:27:07.999821+0000","last_undegraded":"2023-10-06T05:27:07.999821+0000","last_fullsized":"2023-10-06T05:27:07.999821+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:18.241590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.974486+0000","last_change":"2023-10-06T05:27:06.978724+0000","last_active":"2023-10-06T05:27:07.974486+0000","last_peered":"2023-10-06T05:27:07.974486+0000","last_clean":"2023-10-06T05:27:07.974486+0000","last_became_active":"2023-10-06T05:27:06.978461+0000","last_became_peered":"2023-10-06T05:27:06.978461+0000","last_unstale":"2023-10-06T05:27:07.974486+0000","last_undegraded":"2023-10-06T05:27:07.974486+0000","last_fullsized":"2023-10-06T05:27:07.974486+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:28.251732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990392+0000","last_change":"2023-10-06T05:27:07.985197+0000","last_active":"2023-10-06T05:27:07.990392+0000","last_peered":"2023-10-06T05:27:07.990392+0000","last_clean":"2023-10-06T05:27:07.990392+0000","last_became_active":"2023-10-06T05:27:07.984889+0000","last_became_peered":"2023-10-06T05:27:07.984889+0000","last_unstale":"2023-10-06T05:27:07.990392+0000","last_undegraded":"2023-10-06T05:27:07.990392+0000","last_fullsized":"2023-10-06T05:27:07.990392+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:05.624279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979159+0000","last_change":"2023-10-06T05:27:06.984946+0000","last_active":"2023-10-06T05:27:07.979159+0000","last_peered":"2023-10-06T05:27:07.979159+0000","last_clean":"2023-10-06T05:27:07.979159+0000","last_became_active":"2023-10-06T05:27:06.984133+0000","last_became_peered":"2023-10-06T05:27:06.984133+0000","last_unstale":"2023-10-06T05:27:07.979159+0000","last_undegraded":"2023-10-06T05:27:07.979159+0000","last_fullsized":"2023-10-06T05:27:07.979159+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:09.899166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998163+0000","last_change":"2023-10-06T05:27:07.994897+0000","last_active":"2023-10-06T05:27:07.998163+0000","last_peered":"2023-10-06T05:27:07.998163+0000","last_clean":"2023-10-06T05:27:07.998163+0000","last_became_active":"2023-10-06T05:27:07.994359+0000","last_became_peered":"2023-10-06T05:27:07.994359+0000","last_unstale":"2023-10-06T05:27:07.998163+0000","last_undegraded":"2023-10-06T05:27:07.998163+0000","last_fullsized":"2023-10-06T05:27:07.998163+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:37.035462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999194+0000","last_change":"2023-10-06T05:27:06.979383+0000","last_active":"2023-10-06T05:27:07.999194+0000","last_peered":"2023-10-06T05:27:07.999194+0000","last_clean":"2023-10-06T05:27:07.999194+0000","last_became_active":"2023-10-06T05:27:06.979127+0000","last_became_peered":"2023-10-06T05:27:06.979127+0000","last_unstale":"2023-10-06T05:27:07.999194+0000","last_undegraded":"2023-10-06T05:27:07.999194+0000","last_fullsized":"2023-10-06T05:27:07.999194+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:42.829460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994413+0000","last_change":"2023-10-06T05:27:07.992413+0000","last_active":"2023-10-06T05:27:07.994413+0000","last_peered":"2023-10-06T05:27:07.994413+0000","last_clean":"2023-10-06T05:27:07.994413+0000","last_became_active":"2023-10-06T05:27:07.992094+0000","last_became_peered":"2023-10-06T05:27:07.992094+0000","last_unstale":"2023-10-06T05:27:07.994413+0000","last_undegraded":"2023-10-06T05:27:07.994413+0000","last_fullsized":"2023-10-06T05:27:07.994413+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:39.448394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.971266+0000","last_change":"2023-10-06T05:27:06.976303+0000","last_active":"2023-10-06T05:27:07.971266+0000","last_peered":"2023-10-06T05:27:07.971266+0000","last_clean":"2023-10-06T05:27:07.971266+0000","last_became_active":"2023-10-06T05:27:06.976014+0000","last_became_peered":"2023-10-06T05:27:06.976014+0000","last_unstale":"2023-10-06T05:27:07.971266+0000","last_undegraded":"2023-10-06T05:27:07.971266+0000","last_fullsized":"2023-10-06T05:27:07.971266+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:37.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994531+0000","last_change":"2023-10-06T05:27:07.990802+0000","last_active":"2023-10-06T05:27:07.994531+0000","last_peered":"2023-10-06T05:27:07.994531+0000","last_clean":"2023-10-06T05:27:07.994531+0000","last_became_active":"2023-10-06T05:27:07.990508+0000","last_became_peered":"2023-10-06T05:27:07.990508+0000","last_unstale":"2023-10-06T05:27:07.994531+0000","last_undegraded":"2023-10-06T05:27:07.994531+0000","last_fullsized":"2023-10-06T05:27:07.994531+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:51.828771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.974422+0000","last_change":"2023-10-06T05:27:06.981482+0000","last_active":"2023-10-06T05:27:07.974422+0000","last_peered":"2023-10-06T05:27:07.974422+0000","last_clean":"2023-10-06T05:27:07.974422+0000","last_became_active":"2023-10-06T05:27:06.981215+0000","last_became_peered":"2023-10-06T05:27:06.981215+0000","last_unstale":"2023-10-06T05:27:07.974422+0000","last_undegraded":"2023-10-06T05:27:07.974422+0000","last_fullsized":"2023-10-06T05:27:07.974422+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:59.413229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996643+0000","last_change":"2023-10-06T05:27:07.989278+0000","last_active":"2023-10-06T05:27:07.996643+0000","last_peered":"2023-10-06T05:27:07.996643+0000","last_clean":"2023-10-06T05:27:07.996643+0000","last_became_active":"2023-10-06T05:27:07.988846+0000","last_became_peered":"2023-10-06T05:27:07.988846+0000","last_unstale":"2023-10-06T05:27:07.996643+0000","last_undegraded":"2023-10-06T05:27:07.996643+0000","last_fullsized":"2023-10-06T05:27:07.996643+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:50.819400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979153+0000","last_change":"2023-10-06T05:27:06.984272+0000","last_active":"2023-10-06T05:27:07.979153+0000","last_peered":"2023-10-06T05:27:07.979153+0000","last_clean":"2023-10-06T05:27:07.979153+0000","last_became_active":"2023-10-06T05:27:06.984006+0000","last_became_peered":"2023-10-06T05:27:06.984006+0000","last_unstale":"2023-10-06T05:27:07.979153+0000","last_undegraded":"2023-10-06T05:27:07.979153+0000","last_fullsized":"2023-10-06T05:27:07.979153+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:47.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990000+0000","last_change":"2023-10-06T05:27:07.985717+0000","last_active":"2023-10-06T05:27:07.990000+0000","last_peered":"2023-10-06T05:27:07.990000+0000","last_clean":"2023-10-06T05:27:07.990000+0000","last_became_active":"2023-10-06T05:27:07.985464+0000","last_became_peered":"2023-10-06T05:27:07.985464+0000","last_unstale":"2023-10-06T05:27:07.990000+0000","last_undegraded":"2023-10-06T05:27:07.990000+0000","last_fullsized":"2023-10-06T05:27:07.990000+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:41.310817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.970800+0000","last_change":"2023-10-06T05:27:06.984401+0000","last_active":"2023-10-06T05:27:07.970800+0000","last_peered":"2023-10-06T05:27:07.970800+0000","last_clean":"2023-10-06T05:27:07.970800+0000","last_became_active":"2023-10-06T05:27:06.984052+0000","last_became_peered":"2023-10-06T05:27:06.984052+0000","last_unstale":"2023-10-06T05:27:07.970800+0000","last_undegraded":"2023-10-06T05:27:07.970800+0000","last_fullsized":"2023-10-06T05:27:07.970800+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:36.107694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.005812+0000","last_change":"2023-10-06T05:27:07.999284+0000","last_active":"2023-10-06T05:27:08.005812+0000","last_peered":"2023-10-06T05:27:08.005812+0000","last_clean":"2023-10-06T05:27:08.005812+0000","last_became_active":"2023-10-06T05:27:07.999031+0000","last_became_peered":"2023-10-06T05:27:07.999031+0000","last_unstale":"2023-10-06T05:27:08.005812+0000","last_undegraded":"2023-10-06T05:27:08.005812+0000","last_fullsized":"2023-10-06T05:27:08.005812+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:32.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.977393+0000","last_change":"2023-10-06T05:27:06.977765+0000","last_active":"2023-10-06T05:27:07.977393+0000","last_peered":"2023-10-06T05:27:07.977393+0000","last_clean":"2023-10-06T05:27:07.977393+0000","last_became_active":"2023-10-06T05:27:06.975275+0000","last_became_peered":"2023-10-06T05:27:06.975275+0000","last_unstale":"2023-10-06T05:27:07.977393+0000","last_undegraded":"2023-10-06T05:27:07.977393+0000","last_fullsized":"2023-10-06T05:27:07.977393+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:25.787882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.003205+0000","last_change":"2023-10-06T05:27:07.998587+0000","last_active":"2023-10-06T05:27:08.003205+0000","last_peered":"2023-10-06T05:27:08.003205+0000","last_clean":"2023-10-06T05:27:08.003205+0000","last_became_active":"2023-10-06T05:27:07.998132+0000","last_became_peered":"2023-10-06T05:27:07.998132+0000","last_unstale":"2023-10-06T05:27:08.003205+0000","last_undegraded":"2023-10-06T05:27:08.003205+0000","last_fullsized":"2023-10-06T05:27:08.003205+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:43.836288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979023+0000","last_change":"2023-10-06T05:27:06.986788+0000","last_active":"2023-10-06T05:27:07.979023+0000","last_peered":"2023-10-06T05:27:07.979023+0000","last_clean":"2023-10-06T05:27:07.979023+0000","last_became_active":"2023-10-06T05:27:06.986517+0000","last_became_peered":"2023-10-06T05:27:06.986517+0000","last_unstale":"2023-10-06T05:27:07.979023+0000","last_undegraded":"2023-10-06T05:27:07.979023+0000","last_fullsized":"2023-10-06T05:27:07.979023+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:22.462248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994991+0000","last_change":"2023-10-06T05:27:07.993576+0000","last_active":"2023-10-06T05:27:07.994991+0000","last_peered":"2023-10-06T05:27:07.994991+0000","last_clean":"2023-10-06T05:27:07.994991+0000","last_became_active":"2023-10-06T05:27:07.993303+0000","last_became_peered":"2023-10-06T05:27:07.993303+0000","last_unstale":"2023-10-06T05:27:07.994991+0000","last_undegraded":"2023-10-06T05:27:07.994991+0000","last_fullsized":"2023-10-06T05:27:07.994991+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:25.449792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.968602+0000","last_change":"2023-10-06T05:27:06.985051+0000","last_active":"2023-10-06T05:27:07.968602+0000","last_peered":"2023-10-06T05:27:07.968602+0000","last_clean":"2023-10-06T05:27:07.968602+0000","last_became_active":"2023-10-06T05:27:06.984516+0000","last_became_peered":"2023-10-06T05:27:06.984516+0000","last_unstale":"2023-10-06T05:27:07.968602+0000","last_undegraded":"2023-10-06T05:27:07.968602+0000","last_fullsized":"2023-10-06T05:27:07.968602+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:46.452445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996283+0000","last_change":"2023-10-06T05:27:07.985106+0000","last_active":"2023-10-06T05:27:07.996283+0000","last_peered":"2023-10-06T05:27:07.996283+0000","last_clean":"2023-10-06T05:27:07.996283+0000","last_became_active":"2023-10-06T05:27:07.984798+0000","last_became_peered":"2023-10-06T05:27:07.984798+0000","last_unstale":"2023-10-06T05:27:07.996283+0000","last_undegraded":"2023-10-06T05:27:07.996283+0000","last_fullsized":"2023-10-06T05:27:07.996283+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:09.346159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.001242+0000","last_change":"2023-10-06T05:27:06.988368+0000","last_active":"2023-10-06T05:27:08.001242+0000","last_peered":"2023-10-06T05:27:08.001242+0000","last_clean":"2023-10-06T05:27:08.001242+0000","last_became_active":"2023-10-06T05:27:06.988094+0000","last_became_peered":"2023-10-06T05:27:06.988094+0000","last_unstale":"2023-10-06T05:27:08.001242+0000","last_undegraded":"2023-10-06T05:27:08.001242+0000","last_fullsized":"2023-10-06T05:27:08.001242+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:09.719362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":145,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976839+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:27:07.976839+0000","last_peered":"2023-10-06T05:27:07.976839+0000","last_clean":"2023-10-06T05:27:07.976839+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:27:07.976839+0000","last_undegraded":"2023-10-06T05:27:07.976839+0000","last_fullsized":"2023-10-06T05:27:07.976839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993977+0000","last_change":"2023-10-06T05:27:07.989387+0000","last_active":"2023-10-06T05:27:07.993977+0000","last_peered":"2023-10-06T05:27:07.993977+0000","last_clean":"2023-10-06T05:27:07.993977+0000","last_became_active":"2023-10-06T05:27:07.989130+0000","last_became_peered":"2023-10-06T05:27:07.989130+0000","last_unstale":"2023-10-06T05:27:07.993977+0000","last_undegraded":"2023-10-06T05:27:07.993977+0000","last_fullsized":"2023-10-06T05:27:07.993977+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:14.585917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998051+0000","last_change":"2023-10-06T05:27:06.987525+0000","last_active":"2023-10-06T05:27:07.998051+0000","last_peered":"2023-10-06T05:27:07.998051+0000","last_clean":"2023-10-06T05:27:07.998051+0000","last_became_active":"2023-10-06T05:27:06.987238+0000","last_became_peered":"2023-10-06T05:27:06.987238+0000","last_unstale":"2023-10-06T05:27:07.998051+0000","last_undegraded":"2023-10-06T05:27:07.998051+0000","last_fullsized":"2023-10-06T05:27:07.998051+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:48.771305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.978774+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:27:07.978774+0000","last_peered":"2023-10-06T05:27:07.978774+0000","last_clean":"2023-10-06T05:27:07.978774+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:27:07.978774+0000","last_undegraded":"2023-10-06T05:27:07.978774+0000","last_fullsized":"2023-10-06T05:27:07.978774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999822+0000","last_change":"2023-10-06T05:27:07.996214+0000","last_active":"2023-10-06T05:27:07.999822+0000","last_peered":"2023-10-06T05:27:07.999822+0000","last_clean":"2023-10-06T05:27:07.999822+0000","last_became_active":"2023-10-06T05:27:07.995945+0000","last_became_peered":"2023-10-06T05:27:07.995945+0000","last_unstale":"2023-10-06T05:27:07.999822+0000","last_undegraded":"2023-10-06T05:27:07.999822+0000","last_fullsized":"2023-10-06T05:27:07.999822+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:31.065787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.977844+0000","last_change":"2023-10-06T05:27:06.979223+0000","last_active":"2023-10-06T05:27:07.977844+0000","last_peered":"2023-10-06T05:27:07.977844+0000","last_clean":"2023-10-06T05:27:07.977844+0000","last_became_active":"2023-10-06T05:27:06.978972+0000","last_became_peered":"2023-10-06T05:27:06.978972+0000","last_unstale":"2023-10-06T05:27:07.977844+0000","last_undegraded":"2023-10-06T05:27:07.977844+0000","last_fullsized":"2023-10-06T05:27:07.977844+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:15.865977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.969240+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:27:07.969240+0000","last_peered":"2023-10-06T05:27:07.969240+0000","last_clean":"2023-10-06T05:27:07.969240+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:27:07.969240+0000","last_undegraded":"2023-10-06T05:27:07.969240+0000","last_fullsized":"2023-10-06T05:27:07.969240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.995604+0000","last_change":"2023-10-06T05:27:07.988312+0000","last_active":"2023-10-06T05:27:07.995604+0000","last_peered":"2023-10-06T05:27:07.995604+0000","last_clean":"2023-10-06T05:27:07.995604+0000","last_became_active":"2023-10-06T05:27:07.988019+0000","last_became_peered":"2023-10-06T05:27:07.988019+0000","last_unstale":"2023-10-06T05:27:07.995604+0000","last_undegraded":"2023-10-06T05:27:07.995604+0000","last_fullsized":"2023-10-06T05:27:07.995604+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:38.746128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997773+0000","last_change":"2023-10-06T05:27:06.981344+0000","last_active":"2023-10-06T05:27:07.997773+0000","last_peered":"2023-10-06T05:27:07.997773+0000","last_clean":"2023-10-06T05:27:07.997773+0000","last_became_active":"2023-10-06T05:27:06.980236+0000","last_became_peered":"2023-10-06T05:27:06.980236+0000","last_unstale":"2023-10-06T05:27:07.997773+0000","last_undegraded":"2023-10-06T05:27:07.997773+0000","last_fullsized":"2023-10-06T05:27:07.997773+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:20.528042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979362+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:27:07.979362+0000","last_peered":"2023-10-06T05:27:07.979362+0000","last_clean":"2023-10-06T05:27:07.979362+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:27:07.979362+0000","last_undegraded":"2023-10-06T05:27:07.979362+0000","last_fullsized":"2023-10-06T05:27:07.979362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":295,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976528+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:27:07.976528+0000","last_peered":"2023-10-06T05:27:07.976528+0000","last_clean":"2023-10-06T05:27:07.976528+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:27:07.976528+0000","last_undegraded":"2023-10-06T05:27:07.976528+0000","last_fullsized":"2023-10-06T05:27:07.976528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993896+0000","last_change":"2023-10-06T05:27:07.988732+0000","last_active":"2023-10-06T05:27:07.993896+0000","last_peered":"2023-10-06T05:27:07.993896+0000","last_clean":"2023-10-06T05:27:07.993896+0000","last_became_active":"2023-10-06T05:27:07.988458+0000","last_became_peered":"2023-10-06T05:27:07.988458+0000","last_unstale":"2023-10-06T05:27:07.993896+0000","last_undegraded":"2023-10-06T05:27:07.993896+0000","last_fullsized":"2023-10-06T05:27:07.993896+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:49.778132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976652+0000","last_change":"2023-10-06T05:27:06.988126+0000","last_active":"2023-10-06T05:27:07.976652+0000","last_peered":"2023-10-06T05:27:07.976652+0000","last_clean":"2023-10-06T05:27:07.976652+0000","last_became_active":"2023-10-06T05:27:06.987867+0000","last_became_peered":"2023-10-06T05:27:06.987867+0000","last_unstale":"2023-10-06T05:27:07.976652+0000","last_undegraded":"2023-10-06T05:27:07.976652+0000","last_fullsized":"2023-10-06T05:27:07.976652+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:47.111387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976433+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:27:07.976433+0000","last_peered":"2023-10-06T05:27:07.976433+0000","last_clean":"2023-10-06T05:27:07.976433+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:27:07.976433+0000","last_undegraded":"2023-10-06T05:27:07.976433+0000","last_fullsized":"2023-10-06T05:27:07.976433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994383+0000","last_change":"2023-10-06T05:27:07.991295+0000","last_active":"2023-10-06T05:27:07.994383+0000","last_peered":"2023-10-06T05:27:07.994383+0000","last_clean":"2023-10-06T05:27:07.994383+0000","last_became_active":"2023-10-06T05:27:07.991046+0000","last_became_peered":"2023-10-06T05:27:07.991046+0000","last_unstale":"2023-10-06T05:27:07.994383+0000","last_undegraded":"2023-10-06T05:27:07.994383+0000","last_fullsized":"2023-10-06T05:27:07.994383+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:04.410349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998337+0000","last_change":"2023-10-06T05:27:06.979266+0000","last_active":"2023-10-06T05:27:07.998337+0000","last_peered":"2023-10-06T05:27:07.998337+0000","last_clean":"2023-10-06T05:27:07.998337+0000","last_became_active":"2023-10-06T05:27:06.978989+0000","last_became_peered":"2023-10-06T05:27:06.978989+0000","last_unstale":"2023-10-06T05:27:07.998337+0000","last_undegraded":"2023-10-06T05:27:07.998337+0000","last_fullsized":"2023-10-06T05:27:07.998337+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:27.203874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.977838+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:27:07.977838+0000","last_peered":"2023-10-06T05:27:07.977838+0000","last_clean":"2023-10-06T05:27:07.977838+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:27:07.977838+0000","last_undegraded":"2023-10-06T05:27:07.977838+0000","last_fullsized":"2023-10-06T05:27:07.977838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996968+0000","last_change":"2023-10-06T05:27:07.994984+0000","last_active":"2023-10-06T05:27:07.996968+0000","last_peered":"2023-10-06T05:27:07.996968+0000","last_clean":"2023-10-06T05:27:07.996968+0000","last_became_active":"2023-10-06T05:27:07.994713+0000","last_became_peered":"2023-10-06T05:27:07.994713+0000","last_unstale":"2023-10-06T05:27:07.996968+0000","last_undegraded":"2023-10-06T05:27:07.996968+0000","last_fullsized":"2023-10-06T05:27:07.996968+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:11.586742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"180'4","reported_seq":14,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.000890+0000","last_change":"2023-10-06T05:27:06.979448+0000","last_active":"2023-10-06T05:27:08.000890+0000","last_peered":"2023-10-06T05:27:08.000890+0000","last_clean":"2023-10-06T05:27:08.000890+0000","last_became_active":"2023-10-06T05:27:06.979111+0000","last_became_peered":"2023-10-06T05:27:06.979111+0000","last_unstale":"2023-10-06T05:27:08.000890+0000","last_undegraded":"2023-10-06T05:27:08.000890+0000","last_fullsized":"2023-10-06T05:27:08.000890+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.135245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976162+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:27:07.976162+0000","last_peered":"2023-10-06T05:27:07.976162+0000","last_clean":"2023-10-06T05:27:07.976162+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:27:07.976162+0000","last_undegraded":"2023-10-06T05:27:07.976162+0000","last_fullsized":"2023-10-06T05:27:07.976162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996852+0000","last_change":"2023-10-06T05:27:07.995504+0000","last_active":"2023-10-06T05:27:07.996852+0000","last_peered":"2023-10-06T05:27:07.996852+0000","last_clean":"2023-10-06T05:27:07.996852+0000","last_became_active":"2023-10-06T05:27:07.995145+0000","last_became_peered":"2023-10-06T05:27:07.995145+0000","last_unstale":"2023-10-06T05:27:07.996852+0000","last_undegraded":"2023-10-06T05:27:07.996852+0000","last_fullsized":"2023-10-06T05:27:07.996852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:46.040029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.002107+0000","last_change":"2023-10-06T05:27:06.988055+0000","last_active":"2023-10-06T05:27:08.002107+0000","last_peered":"2023-10-06T05:27:08.002107+0000","last_clean":"2023-10-06T05:27:08.002107+0000","last_became_active":"2023-10-06T05:27:06.987653+0000","last_became_peered":"2023-10-06T05:27:06.987653+0000","last_unstale":"2023-10-06T05:27:08.002107+0000","last_undegraded":"2023-10-06T05:27:08.002107+0000","last_fullsized":"2023-10-06T05:27:08.002107+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:11.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":168,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.970444+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:27:07.970444+0000","last_peered":"2023-10-06T05:27:07.970444+0000","last_clean":"2023-10-06T05:27:07.970444+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:27:07.970444+0000","last_undegraded":"2023-10-06T05:27:07.970444+0000","last_fullsized":"2023-10-06T05:27:07.970444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304196,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067644,"statfs":{"total":96636764160,"available":96325267456,"internally_reserved":0,"allocated":2879488,"data_stored":1644402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.47099999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.03,"5min":1.1339999999999999,"15min":1.169},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0249999999999999,"15min":1.268},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.075},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.58099999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.876,"5min":1.0009999999999999,"15min":1.093},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":1.22},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.57099999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.079},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.75800000000000001},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":0.98899999999999999,"5min":1.0960000000000001,"15min":1.202},"last":0.89600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.258},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.42199999999999999,"15min":0.39400000000000002},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":1.032}]},{"osd":5,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.109,"15min":1.109},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.497,"5min":0.44,"15min":0.44},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.407},"last":1.0800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.117},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.0489999999999999,"5min":1.135,"15min":1.3300000000000001},"last":1.097}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303628,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068212,"statfs":{"total":96636764160,"available":96325849088,"internally_reserved":0,"allocated":2297856,"data_stored":1061159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.46899999999999997,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69299999999999995,"15min":0.72599999999999998},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.44900000000000001,"15min":0.442},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.57299999999999995,"5min":0.45300000000000001,"15min":0.40899999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.73099999999999998,"15min":0.76600000000000001},"min":{"1min":0.56599999999999995,"5min":0.56499999999999995,"15min":0.5},"max":{"1min":0.92500000000000004,"5min":0.93200000000000005,"15min":1.1020000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.42099999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.998},"last":0.504}]},{"osd":5,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0760000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":0.94899999999999995,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":0.94199999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.78000000000000003},"min":{"1min":0.57199999999999995,"5min":0.496,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.006,"15min":1.1359999999999999},"last":0.55900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303612,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068228,"statfs":{"total":96636764160,"available":96325865472,"internally_reserved":0,"allocated":2281472,"data_stored":1059069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.28999999999999998},"max":{"1min":1.165,"5min":1.165,"15min":2.6520000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":1.077,"5min":1.077,"15min":2.7010000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.629},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.40000000000000002,"15min":0.35199999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.76900000000000002},"min":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.50700000000000001},"max":{"1min":1.111,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.60199999999999998,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.087,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79600000000000004},"min":{"1min":0.64300000000000002,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.1379999999999999,"5min":1.25,"15min":1.349},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.68200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.159,"5min":1.2569999999999999,"15min":1.669},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.83199999999999996,"15min":0.83699999999999997},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.503},"max":{"1min":1.1299999999999999,"5min":1.3280000000000001,"15min":1.573},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.196,"5min":1.196,"15min":1.528},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82699999999999996,"15min":0.83399999999999996},"min":{"1min":0.65600000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":1.1879999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304196,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067644,"statfs":{"total":96636764160,"available":96325267456,"internally_reserved":0,"allocated":2879488,"data_stored":1649527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.32200000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.41699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79900000000000004,"15min":0.77500000000000002},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":1.258,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79100000000000004,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0609999999999999,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.218},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.748},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.121,"5min":1.153,"15min":1.2629999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.65300000000000002,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.25},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.60699999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.2789999999999999},"last":0.69999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303616,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068224,"statfs":{"total":96636764160,"available":96325861376,"internally_reserved":0,"allocated":2285568,"data_stored":1059159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":0.86899999999999999,"5min":1.056,"15min":1.056},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.67400000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.80600000000000005,"5min":1.073,"15min":1.073},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93700000000000006,"5min":1.109,"15min":1.1379999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.67500000000000004,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.99399999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.746}]},{"osd":5,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74399999999999999,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93899999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77900000000000003,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":0.97799999999999998,"5min":1.1859999999999999,"15min":1.319},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77700000000000002,"15min":0.78800000000000003},"min":{"1min":0.63100000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0209999999999999,"5min":1.137,"15min":1.141},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.52100000000000002},"max":{"1min":0.95699999999999996,"5min":1.284,"15min":1.284},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79800000000000004,"15min":0.81899999999999995},"min":{"1min":0.68899999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97599999999999998,"5min":1.2,"15min":1.3520000000000001},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303624,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068216,"statfs":{"total":96636764160,"available":96325853184,"internally_reserved":0,"allocated":2293760,"data_stored":1059195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":2.0950000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46800000000000003,"15min":0.48899999999999999},"min":{"1min":0.33000000000000002,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.772},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.369,"5min":1.369,"15min":1.913},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.55200000000000005},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.307},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.954},"last":0.505},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.81599999999999995,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.54800000000000004,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.282},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76000000000000001,"15min":0.78500000000000003},"min":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.51100000000000001},"max":{"1min":1.0509999999999999,"5min":1.0840000000000001,"15min":1.1930000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.65200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.79900000000000004,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.53900000000000003},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.343},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.83099999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.1419999999999999,"5min":1.194,"15min":1.3640000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.66700000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.036,"5min":1.1579999999999999,"15min":1.214},"last":0.748}]}]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303616,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068224,"statfs":{"total":96636764160,"available":96325861376,"internally_reserved":0,"allocated":2285568,"data_stored":1059623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89300000000000002,"5min":0.98699999999999999,"15min":1.073},"last":0.621},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.46100000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.169},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.41199999999999998},"max":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":1.0569999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68400000000000005,"15min":0.71399999999999997},"min":{"1min":0.52600000000000002,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.117},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.60899999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.002},"last":0.877}]},{"osd":3,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.875,"5min":0.94999999999999996,"15min":1.0780000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.753},"min":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":1.018,"15min":1.196},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.8740000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.375},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.72099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.081,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.98599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.466},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.218},"last":0.97099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303620,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068220,"statfs":{"total":96636764160,"available":96325857280,"internally_reserved":0,"allocated":2289664,"data_stored":1059125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.71199999999999997,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.374},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.746}]},{"osd":1,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.54500000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":0.98199999999999998,"15min":1.204},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.89900000000000002,"5min":0.95199999999999996,"15min":1.087},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.278},"last":0.72099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":1.004,"15min":1.294},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.89900000000000002,"15min":1.141},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":0.93100000000000005,"5min":0.995,"15min":1.254},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.337},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.75700000000000001},"min":{"1min":0.52000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94499999999999995,"5min":1.0600000000000001,"15min":1.248},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:27:12.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:27:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.585+0000 7f39de59d700 1 Processor -- start 2023-10-06T05:27:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.585+0000 7f39de59d700 1 -- start start 2023-10-06T05:27:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.586+0000 7f39de59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39d80afdd0 0x7f39d80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.586+0000 7f39de59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d806f220 con 0x7f39d806e140 2023-10-06T05:27:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.586+0000 7f39de59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d806e610 con 0x7f39d8159530 2023-10-06T05:27:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.586+0000 7f39de59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80b1e00 con 0x7f39d80afdd0 2023-10-06T05:27:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39dcd9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f39d806e140 0x7f39d815a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37434/0 (socket says 172.21.15.33:37434) 2023-10-06T05:27:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39dcd9a700 1 -- 172.21.15.33:0/3718141467 learned_addr learned my addr 172.21.15.33:0/3718141467 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39ddd9c700 1 --2- 172.21.15.33:0/3718141467 >> v2:172.21.15.167:3300/0 conn(0x7f39d80afdd0 0x7f39d80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32538779 0 0) 0x7f39d806f220 con 0x7f39d806e140 2023-10-06T05:27:12.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c4003570 con 0x7f39d806e140 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267937479 0 0) 0x7f39d806e610 con 0x7f39d8159530 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d806f220 con 0x7f39d8159530 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39ddd9c700 1 -- 172.21.15.33:0/3718141467 >> v1:172.21.15.167:6789/0 conn(0x7f39d8159530 legacy=0x7f39d806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39ddd9c700 1 -- 172.21.15.33:0/3718141467 >> v1:172.21.15.33:6789/0 conn(0x7f39d806e140 legacy=0x7f39d815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088393574 0 0) 0x7f39c4003570 con 0x7f39d806e140 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.587+0000 7f39ddd9c700 1 -- 172.21.15.33:0/3718141467 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d806e8f0 con 0x7f39d80afdd0 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.588+0000 7f39ddd9c700 1 --2- 172.21.15.33:0/3718141467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d80afdd0 0x7f39d80b01a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009e60 tx=0x7f39d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2a9562ecc3987c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.588+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000e070 con 0x7f39d80afdd0 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.588+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d000b7a0 con 0x7f39d80afdd0 2023-10-06T05:27:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.588+0000 7f39cffff700 1 -- 172.21.15.33:0/3718141467 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8001470 con 0x7f39d80afdd0 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 -- 172.21.15.33:0/3718141467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d80afdd0 msgr2=0x7f39d80b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 --2- 172.21.15.33:0/3718141467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d80afdd0 0x7f39d80b01a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009e60 tx=0x7f39d0009780 comp rx=0 tx=0).stop 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 -- 172.21.15.33:0/3718141467 shutdown_connections 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 --2- 172.21.15.33:0/3718141467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d80afdd0 0x7f39d80b01a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 -- 172.21.15.33:0/3718141467 >> 172.21.15.33:0/3718141467 conn(0x7f39d806c560 msgr2=0x7f39d806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 -- 172.21.15.33:0/3718141467 shutdown_connections 2023-10-06T05:27:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.589+0000 7f39de59d700 1 -- 172.21.15.33:0/3718141467 wait complete. 2023-10-06T05:27:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.590+0000 7f39de59d700 1 Processor -- start 2023-10-06T05:27:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.590+0000 7f39de59d700 1 -- start start 2023-10-06T05:27:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39de59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d806e140 0x7f39d8141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39de59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 0x7f39d80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39de59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39d8159530 0x7f39d80cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39de59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80c9420 con 0x7f39d80afdd0 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39de59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80c8810 con 0x7f39d806e140 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39de59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80c8af0 con 0x7f39d8159530 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.591+0000 7f39dcd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 0x7f39d80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 0x7f39d80c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58164/0 (socket says 172.21.15.33:58164) 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 -- 172.21.15.33:0/3972868904 learned_addr learned my addr 172.21.15.33:0/3972868904 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dd59b700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d806e140 0x7f39d8141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 -- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39d8159530 msgr2=0x7f39d80cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39d8159530 0x7f39d80cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 -- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d806e140 msgr2=0x7f39d8141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d806e140 0x7f39d8141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 -- 172.21.15.33:0/3972868904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d0009a20 con 0x7f39d80afdd0 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dd59b700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d806e140 0x7f39d8141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39dcd9a700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 0x7f39d80c7b30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c860 tx=0x7f39c8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.592+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8007360 con 0x7f39d80afdd0 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.593+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c80074c0 con 0x7f39d80afdd0 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.593+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d80c8dd0 con 0x7f39d80afdd0 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.593+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8004830 con 0x7f39d80afdd0 2023-10-06T05:27:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.593+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d80c84a0 con 0x7f39d80afdd0 2023-10-06T05:27:12.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.594+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d80c95d0 con 0x7f39d80afdd0 2023-10-06T05:27:12.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.595+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f39c8004bc0 con 0x7f39d80afdd0 2023-10-06T05:27:12.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.595+0000 7f39cd7fa700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f39bc066f60 0x7f39bc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.596+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f39c8083df0 con 0x7f39d80afdd0 2023-10-06T05:27:12.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.596+0000 7f39dd59b700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f39bc066f60 0x7f39bc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.597+0000 7f39dd59b700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f39bc066f60 0x7f39bc069410 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f39d4007fd0 tx=0x7f39d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.600+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39c8050ee0 con 0x7f39d80afdd0 2023-10-06T05:27:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.794+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39d80b36f0 con 0x7f39bc066f60 2023-10-06T05:27:12.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.804+0000 7f39cd7fa700 1 -- 172.21.15.33:0/3972868904 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311677 (secure 0 0 0) 0x7f39d80b36f0 con 0x7f39bc066f60 2023-10-06T05:27:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f39bc066f60 msgr2=0x7f39bc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f39bc066f60 0x7f39bc069410 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f39d4007fd0 tx=0x7f39d4009040 comp rx=0 tx=0).stop 2023-10-06T05:27:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 msgr2=0x7f39d80c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 0x7f39d80c7b30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c860 tx=0x7f39c8001a30 comp rx=0 tx=0).stop 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 shutdown_connections 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f39bc066f60 0x7f39bc069410 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39d8159530 0x7f39d80cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39d806e140 0x7f39d8141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 --2- 172.21.15.33:0/3972868904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d80afdd0 0x7f39d80c7b30 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.808+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 >> 172.21.15.33:0/3972868904 conn(0x7f39d806c560 msgr2=0x7f39d80af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.809+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 shutdown_connections 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:12.809+0000 7f39de59d700 1 -- 172.21.15.33:0/3972868904 wait complete. 2023-10-06T05:27:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:27:12.828 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":564,"stamp":"2023-10-06T05:27:12.009081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430108,"kb_used_data":19036,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544612,"statfs":{"total":773094113280,"available":770605682688,"internally_reserved":0,"allocated":19492864,"data_stored":9651259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001272"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996247+0000","last_change":"2023-10-06T05:27:07.991642+0000","last_active":"2023-10-06T05:27:07.996247+0000","last_peered":"2023-10-06T05:27:07.996247+0000","last_clean":"2023-10-06T05:27:07.996247+0000","last_became_active":"2023-10-06T05:27:07.991389+0000","last_became_peered":"2023-10-06T05:27:07.991389+0000","last_unstale":"2023-10-06T05:27:07.996247+0000","last_undegraded":"2023-10-06T05:27:07.996247+0000","last_fullsized":"2023-10-06T05:27:07.996247+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:16.784766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988617+0000","last_change":"2023-10-06T05:27:07.985031+0000","last_active":"2023-10-06T05:27:07.988617+0000","last_peered":"2023-10-06T05:27:07.988617+0000","last_clean":"2023-10-06T05:27:07.988617+0000","last_became_active":"2023-10-06T05:27:07.984712+0000","last_became_peered":"2023-10-06T05:27:07.984712+0000","last_unstale":"2023-10-06T05:27:07.988617+0000","last_undegraded":"2023-10-06T05:27:07.988617+0000","last_fullsized":"2023-10-06T05:27:07.988617+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:18.241590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997445+0000","last_change":"2023-10-06T05:27:07.995267+0000","last_active":"2023-10-06T05:27:07.997445+0000","last_peered":"2023-10-06T05:27:07.997445+0000","last_clean":"2023-10-06T05:27:07.997445+0000","last_became_active":"2023-10-06T05:27:07.994971+0000","last_became_peered":"2023-10-06T05:27:07.994971+0000","last_unstale":"2023-10-06T05:27:07.997445+0000","last_undegraded":"2023-10-06T05:27:07.997445+0000","last_fullsized":"2023-10-06T05:27:07.997445+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:50.860043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990403+0000","last_change":"2023-10-06T05:27:07.984037+0000","last_active":"2023-10-06T05:27:07.990403+0000","last_peered":"2023-10-06T05:27:07.990403+0000","last_clean":"2023-10-06T05:27:07.990403+0000","last_became_active":"2023-10-06T05:27:07.983746+0000","last_became_peered":"2023-10-06T05:27:07.983746+0000","last_unstale":"2023-10-06T05:27:07.990403+0000","last_undegraded":"2023-10-06T05:27:07.990403+0000","last_fullsized":"2023-10-06T05:27:07.990403+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:11.454394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994078+0000","last_change":"2023-10-06T05:27:07.989964+0000","last_active":"2023-10-06T05:27:07.994078+0000","last_peered":"2023-10-06T05:27:07.994078+0000","last_clean":"2023-10-06T05:27:07.994078+0000","last_became_active":"2023-10-06T05:27:07.989268+0000","last_became_peered":"2023-10-06T05:27:07.989268+0000","last_unstale":"2023-10-06T05:27:07.994078+0000","last_undegraded":"2023-10-06T05:27:07.994078+0000","last_fullsized":"2023-10-06T05:27:07.994078+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:16.216941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993621+0000","last_change":"2023-10-06T05:27:07.988801+0000","last_active":"2023-10-06T05:27:07.993621+0000","last_peered":"2023-10-06T05:27:07.993621+0000","last_clean":"2023-10-06T05:27:07.993621+0000","last_became_active":"2023-10-06T05:27:07.988533+0000","last_became_peered":"2023-10-06T05:27:07.988533+0000","last_unstale":"2023-10-06T05:27:07.993621+0000","last_undegraded":"2023-10-06T05:27:07.993621+0000","last_fullsized":"2023-10-06T05:27:07.993621+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:47.711821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.995588+0000","last_change":"2023-10-06T05:27:07.993741+0000","last_active":"2023-10-06T05:27:07.995588+0000","last_peered":"2023-10-06T05:27:07.995588+0000","last_clean":"2023-10-06T05:27:07.995588+0000","last_became_active":"2023-10-06T05:27:07.993467+0000","last_became_peered":"2023-10-06T05:27:07.993467+0000","last_unstale":"2023-10-06T05:27:07.995588+0000","last_undegraded":"2023-10-06T05:27:07.995588+0000","last_fullsized":"2023-10-06T05:27:07.995588+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:14.178515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994533+0000","last_change":"2023-10-06T05:27:07.986376+0000","last_active":"2023-10-06T05:27:07.994533+0000","last_peered":"2023-10-06T05:27:07.994533+0000","last_clean":"2023-10-06T05:27:07.994533+0000","last_became_active":"2023-10-06T05:27:07.986100+0000","last_became_peered":"2023-10-06T05:27:07.986100+0000","last_unstale":"2023-10-06T05:27:07.994533+0000","last_undegraded":"2023-10-06T05:27:07.994533+0000","last_fullsized":"2023-10-06T05:27:07.994533+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:20.789282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999797+0000","last_change":"2023-10-06T05:27:07.992250+0000","last_active":"2023-10-06T05:27:07.999797+0000","last_peered":"2023-10-06T05:27:07.999797+0000","last_clean":"2023-10-06T05:27:07.999797+0000","last_became_active":"2023-10-06T05:27:07.991022+0000","last_became_peered":"2023-10-06T05:27:07.991022+0000","last_unstale":"2023-10-06T05:27:07.999797+0000","last_undegraded":"2023-10-06T05:27:07.999797+0000","last_fullsized":"2023-10-06T05:27:07.999797+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:45.326029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996191+0000","last_change":"2023-10-06T05:27:07.990152+0000","last_active":"2023-10-06T05:27:07.996191+0000","last_peered":"2023-10-06T05:27:07.996191+0000","last_clean":"2023-10-06T05:27:07.996191+0000","last_became_active":"2023-10-06T05:27:07.989388+0000","last_became_peered":"2023-10-06T05:27:07.989388+0000","last_unstale":"2023-10-06T05:27:07.996191+0000","last_undegraded":"2023-10-06T05:27:07.996191+0000","last_fullsized":"2023-10-06T05:27:07.996191+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:57:27.877492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994464+0000","last_change":"2023-10-06T05:27:07.993293+0000","last_active":"2023-10-06T05:27:07.994464+0000","last_peered":"2023-10-06T05:27:07.994464+0000","last_clean":"2023-10-06T05:27:07.994464+0000","last_became_active":"2023-10-06T05:27:07.992651+0000","last_became_peered":"2023-10-06T05:27:07.992651+0000","last_unstale":"2023-10-06T05:27:07.994464+0000","last_undegraded":"2023-10-06T05:27:07.994464+0000","last_fullsized":"2023-10-06T05:27:07.994464+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:32.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989353+0000","last_change":"2023-10-06T05:27:07.981638+0000","last_active":"2023-10-06T05:27:07.989353+0000","last_peered":"2023-10-06T05:27:07.989353+0000","last_clean":"2023-10-06T05:27:07.989353+0000","last_became_active":"2023-10-06T05:27:07.981285+0000","last_became_peered":"2023-10-06T05:27:07.981285+0000","last_unstale":"2023-10-06T05:27:07.989353+0000","last_undegraded":"2023-10-06T05:27:07.989353+0000","last_fullsized":"2023-10-06T05:27:07.989353+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:26.893361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.005853+0000","last_change":"2023-10-06T05:27:07.999647+0000","last_active":"2023-10-06T05:27:08.005853+0000","last_peered":"2023-10-06T05:27:08.005853+0000","last_clean":"2023-10-06T05:27:08.005853+0000","last_became_active":"2023-10-06T05:27:07.999109+0000","last_became_peered":"2023-10-06T05:27:07.999109+0000","last_unstale":"2023-10-06T05:27:08.005853+0000","last_undegraded":"2023-10-06T05:27:08.005853+0000","last_fullsized":"2023-10-06T05:27:08.005853+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:28.210701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998278+0000","last_change":"2023-10-06T05:27:07.995619+0000","last_active":"2023-10-06T05:27:07.998278+0000","last_peered":"2023-10-06T05:27:07.998278+0000","last_clean":"2023-10-06T05:27:07.998278+0000","last_became_active":"2023-10-06T05:27:07.995318+0000","last_became_peered":"2023-10-06T05:27:07.995318+0000","last_unstale":"2023-10-06T05:27:07.998278+0000","last_undegraded":"2023-10-06T05:27:07.998278+0000","last_fullsized":"2023-10-06T05:27:07.998278+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:58.717175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.984529+0000","last_change":"2023-10-06T05:27:07.982150+0000","last_active":"2023-10-06T05:27:07.984529+0000","last_peered":"2023-10-06T05:27:07.984529+0000","last_clean":"2023-10-06T05:27:07.984529+0000","last_became_active":"2023-10-06T05:27:07.981841+0000","last_became_peered":"2023-10-06T05:27:07.981841+0000","last_unstale":"2023-10-06T05:27:07.984529+0000","last_undegraded":"2023-10-06T05:27:07.984529+0000","last_fullsized":"2023-10-06T05:27:07.984529+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:48.118214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989981+0000","last_change":"2023-10-06T05:27:07.988162+0000","last_active":"2023-10-06T05:27:07.989981+0000","last_peered":"2023-10-06T05:27:07.989981+0000","last_clean":"2023-10-06T05:27:07.989981+0000","last_became_active":"2023-10-06T05:27:07.987912+0000","last_became_peered":"2023-10-06T05:27:07.987912+0000","last_unstale":"2023-10-06T05:27:07.989981+0000","last_undegraded":"2023-10-06T05:27:07.989981+0000","last_fullsized":"2023-10-06T05:27:07.989981+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:11.454394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988174+0000","last_change":"2023-10-06T05:27:07.986863+0000","last_active":"2023-10-06T05:27:07.988174+0000","last_peered":"2023-10-06T05:27:07.988174+0000","last_clean":"2023-10-06T05:27:07.988174+0000","last_became_active":"2023-10-06T05:27:07.986554+0000","last_became_peered":"2023-10-06T05:27:07.986554+0000","last_unstale":"2023-10-06T05:27:07.988174+0000","last_undegraded":"2023-10-06T05:27:07.988174+0000","last_fullsized":"2023-10-06T05:27:07.988174+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:03.557080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.992771+0000","last_change":"2023-10-06T05:27:07.990735+0000","last_active":"2023-10-06T05:27:07.992771+0000","last_peered":"2023-10-06T05:27:07.992771+0000","last_clean":"2023-10-06T05:27:07.992771+0000","last_became_active":"2023-10-06T05:27:07.988575+0000","last_became_peered":"2023-10-06T05:27:07.988575+0000","last_unstale":"2023-10-06T05:27:07.992771+0000","last_undegraded":"2023-10-06T05:27:07.992771+0000","last_fullsized":"2023-10-06T05:27:07.992771+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:07.824570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988176+0000","last_change":"2023-10-06T05:27:07.982588+0000","last_active":"2023-10-06T05:27:07.988176+0000","last_peered":"2023-10-06T05:27:07.988176+0000","last_clean":"2023-10-06T05:27:07.988176+0000","last_became_active":"2023-10-06T05:27:07.981946+0000","last_became_peered":"2023-10-06T05:27:07.981946+0000","last_unstale":"2023-10-06T05:27:07.988176+0000","last_undegraded":"2023-10-06T05:27:07.988176+0000","last_fullsized":"2023-10-06T05:27:07.988176+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:04.410349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.988114+0000","last_change":"2023-10-06T05:27:07.983911+0000","last_active":"2023-10-06T05:27:07.988114+0000","last_peered":"2023-10-06T05:27:07.988114+0000","last_clean":"2023-10-06T05:27:07.988114+0000","last_became_active":"2023-10-06T05:27:07.983655+0000","last_became_peered":"2023-10-06T05:27:07.983655+0000","last_unstale":"2023-10-06T05:27:07.988114+0000","last_undegraded":"2023-10-06T05:27:07.988114+0000","last_fullsized":"2023-10-06T05:27:07.988114+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:16.216941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996489+0000","last_change":"2023-10-06T05:27:07.990126+0000","last_active":"2023-10-06T05:27:07.996489+0000","last_peered":"2023-10-06T05:27:07.996489+0000","last_clean":"2023-10-06T05:27:07.996489+0000","last_became_active":"2023-10-06T05:27:07.989603+0000","last_became_peered":"2023-10-06T05:27:07.989603+0000","last_unstale":"2023-10-06T05:27:07.996489+0000","last_undegraded":"2023-10-06T05:27:07.996489+0000","last_fullsized":"2023-10-06T05:27:07.996489+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:27.864903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998259+0000","last_change":"2023-10-06T05:27:07.995017+0000","last_active":"2023-10-06T05:27:07.998259+0000","last_peered":"2023-10-06T05:27:07.998259+0000","last_clean":"2023-10-06T05:27:07.998259+0000","last_became_active":"2023-10-06T05:27:07.994725+0000","last_became_peered":"2023-10-06T05:27:07.994725+0000","last_unstale":"2023-10-06T05:27:07.998259+0000","last_undegraded":"2023-10-06T05:27:07.998259+0000","last_fullsized":"2023-10-06T05:27:07.998259+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:48.760673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997206+0000","last_change":"2023-10-06T05:27:07.995673+0000","last_active":"2023-10-06T05:27:07.997206+0000","last_peered":"2023-10-06T05:27:07.997206+0000","last_clean":"2023-10-06T05:27:07.997206+0000","last_became_active":"2023-10-06T05:27:07.995374+0000","last_became_peered":"2023-10-06T05:27:07.995374+0000","last_unstale":"2023-10-06T05:27:07.997206+0000","last_undegraded":"2023-10-06T05:27:07.997206+0000","last_fullsized":"2023-10-06T05:27:07.997206+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:51.828771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999368+0000","last_change":"2023-10-06T05:27:07.997721+0000","last_active":"2023-10-06T05:27:07.999368+0000","last_peered":"2023-10-06T05:27:07.999368+0000","last_clean":"2023-10-06T05:27:07.999368+0000","last_became_active":"2023-10-06T05:27:07.997338+0000","last_became_peered":"2023-10-06T05:27:07.997338+0000","last_unstale":"2023-10-06T05:27:07.999368+0000","last_undegraded":"2023-10-06T05:27:07.999368+0000","last_fullsized":"2023-10-06T05:27:07.999368+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:39.023585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.992348+0000","last_change":"2023-10-06T05:27:07.990039+0000","last_active":"2023-10-06T05:27:07.992348+0000","last_peered":"2023-10-06T05:27:07.992348+0000","last_clean":"2023-10-06T05:27:07.992348+0000","last_became_active":"2023-10-06T05:27:07.989750+0000","last_became_peered":"2023-10-06T05:27:07.989750+0000","last_unstale":"2023-10-06T05:27:07.992348+0000","last_undegraded":"2023-10-06T05:27:07.992348+0000","last_fullsized":"2023-10-06T05:27:07.992348+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:10.321422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996497+0000","last_change":"2023-10-06T05:27:07.992261+0000","last_active":"2023-10-06T05:27:07.996497+0000","last_peered":"2023-10-06T05:27:07.996497+0000","last_clean":"2023-10-06T05:27:07.996497+0000","last_became_active":"2023-10-06T05:27:07.992014+0000","last_became_peered":"2023-10-06T05:27:07.992014+0000","last_unstale":"2023-10-06T05:27:07.996497+0000","last_undegraded":"2023-10-06T05:27:07.996497+0000","last_fullsized":"2023-10-06T05:27:07.996497+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:16.872804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994446+0000","last_change":"2023-10-06T05:27:07.993013+0000","last_active":"2023-10-06T05:27:07.994446+0000","last_peered":"2023-10-06T05:27:07.994446+0000","last_clean":"2023-10-06T05:27:07.994446+0000","last_became_active":"2023-10-06T05:27:07.992195+0000","last_became_peered":"2023-10-06T05:27:07.992195+0000","last_unstale":"2023-10-06T05:27:07.994446+0000","last_undegraded":"2023-10-06T05:27:07.994446+0000","last_fullsized":"2023-10-06T05:27:07.994446+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:18.893704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989376+0000","last_change":"2023-10-06T05:27:07.984408+0000","last_active":"2023-10-06T05:27:07.989376+0000","last_peered":"2023-10-06T05:27:07.989376+0000","last_clean":"2023-10-06T05:27:07.989376+0000","last_became_active":"2023-10-06T05:27:07.984137+0000","last_became_peered":"2023-10-06T05:27:07.984137+0000","last_unstale":"2023-10-06T05:27:07.989376+0000","last_undegraded":"2023-10-06T05:27:07.989376+0000","last_fullsized":"2023-10-06T05:27:07.989376+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:41.310817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.000430+0000","last_change":"2023-10-06T05:27:07.997629+0000","last_active":"2023-10-06T05:27:08.000430+0000","last_peered":"2023-10-06T05:27:08.000430+0000","last_clean":"2023-10-06T05:27:08.000430+0000","last_became_active":"2023-10-06T05:27:07.997387+0000","last_became_peered":"2023-10-06T05:27:07.997387+0000","last_unstale":"2023-10-06T05:27:08.000430+0000","last_undegraded":"2023-10-06T05:27:08.000430+0000","last_fullsized":"2023-10-06T05:27:08.000430+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:15.267008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996217+0000","last_change":"2023-10-06T05:27:07.990036+0000","last_active":"2023-10-06T05:27:07.996217+0000","last_peered":"2023-10-06T05:27:07.996217+0000","last_clean":"2023-10-06T05:27:07.996217+0000","last_became_active":"2023-10-06T05:27:07.989767+0000","last_became_peered":"2023-10-06T05:27:07.989767+0000","last_unstale":"2023-10-06T05:27:07.996217+0000","last_undegraded":"2023-10-06T05:27:07.996217+0000","last_fullsized":"2023-10-06T05:27:07.996217+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:27.762816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994543+0000","last_change":"2023-10-06T05:27:07.987112+0000","last_active":"2023-10-06T05:27:07.994543+0000","last_peered":"2023-10-06T05:27:07.994543+0000","last_clean":"2023-10-06T05:27:07.994543+0000","last_became_active":"2023-10-06T05:27:07.986786+0000","last_became_peered":"2023-10-06T05:27:07.986786+0000","last_unstale":"2023-10-06T05:27:07.994543+0000","last_undegraded":"2023-10-06T05:27:07.994543+0000","last_fullsized":"2023-10-06T05:27:07.994543+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:34.747786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997763+0000","last_change":"2023-10-06T05:27:07.992743+0000","last_active":"2023-10-06T05:27:07.997763+0000","last_peered":"2023-10-06T05:27:07.997763+0000","last_clean":"2023-10-06T05:27:07.997763+0000","last_became_active":"2023-10-06T05:27:07.992480+0000","last_became_peered":"2023-10-06T05:27:07.992480+0000","last_unstale":"2023-10-06T05:27:07.997763+0000","last_undegraded":"2023-10-06T05:27:07.997763+0000","last_fullsized":"2023-10-06T05:27:07.997763+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:21.534869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.984603+0000","last_change":"2023-10-06T05:27:07.981995+0000","last_active":"2023-10-06T05:27:07.984603+0000","last_peered":"2023-10-06T05:27:07.984603+0000","last_clean":"2023-10-06T05:27:07.984603+0000","last_became_active":"2023-10-06T05:27:07.981679+0000","last_became_peered":"2023-10-06T05:27:07.981679+0000","last_unstale":"2023-10-06T05:27:07.984603+0000","last_undegraded":"2023-10-06T05:27:07.984603+0000","last_fullsized":"2023-10-06T05:27:07.984603+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:16.784766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.968702+0000","last_change":"2023-10-06T05:27:06.988233+0000","last_active":"2023-10-06T05:27:07.968702+0000","last_peered":"2023-10-06T05:27:07.968702+0000","last_clean":"2023-10-06T05:27:07.968702+0000","last_became_active":"2023-10-06T05:27:06.987901+0000","last_became_peered":"2023-10-06T05:27:06.987901+0000","last_unstale":"2023-10-06T05:27:07.968702+0000","last_undegraded":"2023-10-06T05:27:07.968702+0000","last_fullsized":"2023-10-06T05:27:07.968702+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:10.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993450+0000","last_change":"2023-10-06T05:27:07.991400+0000","last_active":"2023-10-06T05:27:07.993450+0000","last_peered":"2023-10-06T05:27:07.993450+0000","last_clean":"2023-10-06T05:27:07.993450+0000","last_became_active":"2023-10-06T05:27:07.991101+0000","last_became_peered":"2023-10-06T05:27:07.991101+0000","last_unstale":"2023-10-06T05:27:07.993450+0000","last_undegraded":"2023-10-06T05:27:07.993450+0000","last_fullsized":"2023-10-06T05:27:07.993450+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979957+0000","last_change":"2023-10-06T05:27:06.990409+0000","last_active":"2023-10-06T05:27:07.979957+0000","last_peered":"2023-10-06T05:27:07.979957+0000","last_clean":"2023-10-06T05:27:07.979957+0000","last_became_active":"2023-10-06T05:27:06.990038+0000","last_became_peered":"2023-10-06T05:27:06.990038+0000","last_unstale":"2023-10-06T05:27:07.979957+0000","last_undegraded":"2023-10-06T05:27:07.979957+0000","last_fullsized":"2023-10-06T05:27:07.979957+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:26.755989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999719+0000","last_change":"2023-10-06T05:27:07.992949+0000","last_active":"2023-10-06T05:27:07.999719+0000","last_peered":"2023-10-06T05:27:07.999719+0000","last_clean":"2023-10-06T05:27:07.999719+0000","last_became_active":"2023-10-06T05:27:07.991076+0000","last_became_peered":"2023-10-06T05:27:07.991076+0000","last_unstale":"2023-10-06T05:27:07.999719+0000","last_undegraded":"2023-10-06T05:27:07.999719+0000","last_fullsized":"2023-10-06T05:27:07.999719+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:30.178173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"180'3","reported_seq":13,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998253+0000","last_change":"2023-10-06T05:27:06.981081+0000","last_active":"2023-10-06T05:27:07.998253+0000","last_peered":"2023-10-06T05:27:07.998253+0000","last_clean":"2023-10-06T05:27:07.998253+0000","last_became_active":"2023-10-06T05:27:06.980366+0000","last_became_peered":"2023-10-06T05:27:06.980366+0000","last_unstale":"2023-10-06T05:27:07.998253+0000","last_undegraded":"2023-10-06T05:27:07.998253+0000","last_fullsized":"2023-10-06T05:27:07.998253+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:33.740958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990047+0000","last_change":"2023-10-06T05:27:07.982074+0000","last_active":"2023-10-06T05:27:07.990047+0000","last_peered":"2023-10-06T05:27:07.990047+0000","last_clean":"2023-10-06T05:27:07.990047+0000","last_became_active":"2023-10-06T05:27:07.981810+0000","last_became_peered":"2023-10-06T05:27:07.981810+0000","last_unstale":"2023-10-06T05:27:07.990047+0000","last_undegraded":"2023-10-06T05:27:07.990047+0000","last_fullsized":"2023-10-06T05:27:07.990047+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:20.789282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"180'6","reported_seq":16,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.006107+0000","last_change":"2023-10-06T05:27:06.983941+0000","last_active":"2023-10-06T05:27:08.006107+0000","last_peered":"2023-10-06T05:27:08.006107+0000","last_clean":"2023-10-06T05:27:08.006107+0000","last_became_active":"2023-10-06T05:27:06.983665+0000","last_became_peered":"2023-10-06T05:27:06.983665+0000","last_unstale":"2023-10-06T05:27:08.006107+0000","last_undegraded":"2023-10-06T05:27:08.006107+0000","last_fullsized":"2023-10-06T05:27:08.006107+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:13.579090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989039+0000","last_change":"2023-10-06T05:27:07.985258+0000","last_active":"2023-10-06T05:27:07.989039+0000","last_peered":"2023-10-06T05:27:07.989039+0000","last_clean":"2023-10-06T05:27:07.989039+0000","last_became_active":"2023-10-06T05:27:07.984565+0000","last_became_peered":"2023-10-06T05:27:07.984565+0000","last_unstale":"2023-10-06T05:27:07.989039+0000","last_undegraded":"2023-10-06T05:27:07.989039+0000","last_fullsized":"2023-10-06T05:27:07.989039+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:58.087999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.000142+0000","last_change":"2023-10-06T05:27:06.989280+0000","last_active":"2023-10-06T05:27:08.000142+0000","last_peered":"2023-10-06T05:27:08.000142+0000","last_clean":"2023-10-06T05:27:08.000142+0000","last_became_active":"2023-10-06T05:27:06.988952+0000","last_became_peered":"2023-10-06T05:27:06.988952+0000","last_unstale":"2023-10-06T05:27:08.000142+0000","last_undegraded":"2023-10-06T05:27:08.000142+0000","last_fullsized":"2023-10-06T05:27:08.000142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:01.310099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990012+0000","last_change":"2023-10-06T05:27:07.988980+0000","last_active":"2023-10-06T05:27:07.990012+0000","last_peered":"2023-10-06T05:27:07.990012+0000","last_clean":"2023-10-06T05:27:07.990012+0000","last_became_active":"2023-10-06T05:27:07.988707+0000","last_became_peered":"2023-10-06T05:27:07.988707+0000","last_unstale":"2023-10-06T05:27:07.990012+0000","last_undegraded":"2023-10-06T05:27:07.990012+0000","last_fullsized":"2023-10-06T05:27:07.990012+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:22.842577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976420+0000","last_change":"2023-10-06T05:27:06.981974+0000","last_active":"2023-10-06T05:27:07.976420+0000","last_peered":"2023-10-06T05:27:07.976420+0000","last_clean":"2023-10-06T05:27:07.976420+0000","last_became_active":"2023-10-06T05:27:06.981590+0000","last_became_peered":"2023-10-06T05:27:06.981590+0000","last_unstale":"2023-10-06T05:27:07.976420+0000","last_undegraded":"2023-10-06T05:27:07.976420+0000","last_fullsized":"2023-10-06T05:27:07.976420+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:37.739301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999690+0000","last_change":"2023-10-06T05:27:07.992673+0000","last_active":"2023-10-06T05:27:07.999690+0000","last_peered":"2023-10-06T05:27:07.999690+0000","last_clean":"2023-10-06T05:27:07.999690+0000","last_became_active":"2023-10-06T05:27:07.991372+0000","last_became_peered":"2023-10-06T05:27:07.991372+0000","last_unstale":"2023-10-06T05:27:07.999690+0000","last_undegraded":"2023-10-06T05:27:07.999690+0000","last_fullsized":"2023-10-06T05:27:07.999690+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:47.460002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976276+0000","last_change":"2023-10-06T05:27:06.980027+0000","last_active":"2023-10-06T05:27:07.976276+0000","last_peered":"2023-10-06T05:27:07.976276+0000","last_clean":"2023-10-06T05:27:07.976276+0000","last_became_active":"2023-10-06T05:27:06.979718+0000","last_became_peered":"2023-10-06T05:27:06.979718+0000","last_unstale":"2023-10-06T05:27:07.976276+0000","last_undegraded":"2023-10-06T05:27:07.976276+0000","last_fullsized":"2023-10-06T05:27:07.976276+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:23.972834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996724+0000","last_change":"2023-10-06T05:27:07.992111+0000","last_active":"2023-10-06T05:27:07.996724+0000","last_peered":"2023-10-06T05:27:07.996724+0000","last_clean":"2023-10-06T05:27:07.996724+0000","last_became_active":"2023-10-06T05:27:07.991829+0000","last_became_peered":"2023-10-06T05:27:07.991829+0000","last_unstale":"2023-10-06T05:27:07.996724+0000","last_undegraded":"2023-10-06T05:27:07.996724+0000","last_fullsized":"2023-10-06T05:27:07.996724+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:24.979661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"180'8","reported_seq":18,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.007669+0000","last_change":"2023-10-06T05:27:06.986822+0000","last_active":"2023-10-06T05:27:08.007669+0000","last_peered":"2023-10-06T05:27:08.007669+0000","last_clean":"2023-10-06T05:27:08.007669+0000","last_became_active":"2023-10-06T05:27:06.986496+0000","last_became_peered":"2023-10-06T05:27:06.986496+0000","last_unstale":"2023-10-06T05:27:08.007669+0000","last_undegraded":"2023-10-06T05:27:08.007669+0000","last_fullsized":"2023-10-06T05:27:08.007669+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:58.995025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998183+0000","last_change":"2023-10-06T05:27:07.995086+0000","last_active":"2023-10-06T05:27:07.998183+0000","last_peered":"2023-10-06T05:27:07.998183+0000","last_clean":"2023-10-06T05:27:07.998183+0000","last_became_active":"2023-10-06T05:27:07.993671+0000","last_became_peered":"2023-10-06T05:27:07.993671+0000","last_unstale":"2023-10-06T05:27:07.998183+0000","last_undegraded":"2023-10-06T05:27:07.998183+0000","last_fullsized":"2023-10-06T05:27:07.998183+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:19.937832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.001180+0000","last_change":"2023-10-06T05:27:06.987881+0000","last_active":"2023-10-06T05:27:08.001180+0000","last_peered":"2023-10-06T05:27:08.001180+0000","last_clean":"2023-10-06T05:27:08.001180+0000","last_became_active":"2023-10-06T05:27:06.987586+0000","last_became_peered":"2023-10-06T05:27:06.987586+0000","last_unstale":"2023-10-06T05:27:08.001180+0000","last_undegraded":"2023-10-06T05:27:08.001180+0000","last_fullsized":"2023-10-06T05:27:08.001180+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:17.361942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996776+0000","last_change":"2023-10-06T05:27:07.991139+0000","last_active":"2023-10-06T05:27:07.996776+0000","last_peered":"2023-10-06T05:27:07.996776+0000","last_clean":"2023-10-06T05:27:07.996776+0000","last_became_active":"2023-10-06T05:27:07.990861+0000","last_became_peered":"2023-10-06T05:27:07.990861+0000","last_unstale":"2023-10-06T05:27:07.996776+0000","last_undegraded":"2023-10-06T05:27:07.996776+0000","last_fullsized":"2023-10-06T05:27:07.996776+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:06.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996439+0000","last_change":"2023-10-06T05:27:06.985851+0000","last_active":"2023-10-06T05:27:07.996439+0000","last_peered":"2023-10-06T05:27:07.996439+0000","last_clean":"2023-10-06T05:27:07.996439+0000","last_became_active":"2023-10-06T05:27:06.985549+0000","last_became_peered":"2023-10-06T05:27:06.985549+0000","last_unstale":"2023-10-06T05:27:07.996439+0000","last_undegraded":"2023-10-06T05:27:07.996439+0000","last_fullsized":"2023-10-06T05:27:07.996439+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:35.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996828+0000","last_change":"2023-10-06T05:27:07.992748+0000","last_active":"2023-10-06T05:27:07.996828+0000","last_peered":"2023-10-06T05:27:07.996828+0000","last_clean":"2023-10-06T05:27:07.996828+0000","last_became_active":"2023-10-06T05:27:07.992475+0000","last_became_peered":"2023-10-06T05:27:07.992475+0000","last_unstale":"2023-10-06T05:27:07.996828+0000","last_undegraded":"2023-10-06T05:27:07.996828+0000","last_fullsized":"2023-10-06T05:27:07.996828+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:44.294386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.001794+0000","last_change":"2023-10-06T05:27:06.985453+0000","last_active":"2023-10-06T05:27:08.001794+0000","last_peered":"2023-10-06T05:27:08.001794+0000","last_clean":"2023-10-06T05:27:08.001794+0000","last_became_active":"2023-10-06T05:27:06.985167+0000","last_became_peered":"2023-10-06T05:27:06.985167+0000","last_unstale":"2023-10-06T05:27:08.001794+0000","last_undegraded":"2023-10-06T05:27:08.001794+0000","last_fullsized":"2023-10-06T05:27:08.001794+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:36.096363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.991904+0000","last_change":"2023-10-06T05:27:07.986527+0000","last_active":"2023-10-06T05:27:07.991904+0000","last_peered":"2023-10-06T05:27:07.991904+0000","last_clean":"2023-10-06T05:27:07.991904+0000","last_became_active":"2023-10-06T05:27:07.986197+0000","last_became_peered":"2023-10-06T05:27:07.986197+0000","last_unstale":"2023-10-06T05:27:07.991904+0000","last_undegraded":"2023-10-06T05:27:07.991904+0000","last_fullsized":"2023-10-06T05:27:07.991904+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:26.893361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.969415+0000","last_change":"2023-10-06T05:27:06.987452+0000","last_active":"2023-10-06T05:27:07.969415+0000","last_peered":"2023-10-06T05:27:07.969415+0000","last_clean":"2023-10-06T05:27:07.969415+0000","last_became_active":"2023-10-06T05:27:06.986312+0000","last_became_peered":"2023-10-06T05:27:06.986312+0000","last_unstale":"2023-10-06T05:27:07.969415+0000","last_undegraded":"2023-10-06T05:27:07.969415+0000","last_fullsized":"2023-10-06T05:27:07.969415+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:26.000585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.989065+0000","last_change":"2023-10-06T05:27:07.985441+0000","last_active":"2023-10-06T05:27:07.989065+0000","last_peered":"2023-10-06T05:27:07.989065+0000","last_clean":"2023-10-06T05:27:07.989065+0000","last_became_active":"2023-10-06T05:27:07.984889+0000","last_became_peered":"2023-10-06T05:27:07.984889+0000","last_unstale":"2023-10-06T05:27:07.989065+0000","last_undegraded":"2023-10-06T05:27:07.989065+0000","last_fullsized":"2023-10-06T05:27:07.989065+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:31.065787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.004006+0000","last_change":"2023-10-06T05:27:06.988284+0000","last_active":"2023-10-06T05:27:08.004006+0000","last_peered":"2023-10-06T05:27:08.004006+0000","last_clean":"2023-10-06T05:27:08.004006+0000","last_became_active":"2023-10-06T05:27:06.988024+0000","last_became_peered":"2023-10-06T05:27:06.988024+0000","last_unstale":"2023-10-06T05:27:08.004006+0000","last_undegraded":"2023-10-06T05:27:08.004006+0000","last_fullsized":"2023-10-06T05:27:08.004006+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:37.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997798+0000","last_change":"2023-10-06T05:27:07.992349+0000","last_active":"2023-10-06T05:27:07.997798+0000","last_peered":"2023-10-06T05:27:07.997798+0000","last_clean":"2023-10-06T05:27:07.997798+0000","last_became_active":"2023-10-06T05:27:07.992065+0000","last_became_peered":"2023-10-06T05:27:07.992065+0000","last_unstale":"2023-10-06T05:27:07.997798+0000","last_undegraded":"2023-10-06T05:27:07.997798+0000","last_fullsized":"2023-10-06T05:27:07.997798+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:26.794709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.969901+0000","last_change":"2023-10-06T05:27:06.985860+0000","last_active":"2023-10-06T05:27:07.969901+0000","last_peered":"2023-10-06T05:27:07.969901+0000","last_clean":"2023-10-06T05:27:07.969901+0000","last_became_active":"2023-10-06T05:27:06.985492+0000","last_became_peered":"2023-10-06T05:27:06.985492+0000","last_unstale":"2023-10-06T05:27:07.969901+0000","last_undegraded":"2023-10-06T05:27:07.969901+0000","last_fullsized":"2023-10-06T05:27:07.969901+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:10.396573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999821+0000","last_change":"2023-10-06T05:27:07.995392+0000","last_active":"2023-10-06T05:27:07.999821+0000","last_peered":"2023-10-06T05:27:07.999821+0000","last_clean":"2023-10-06T05:27:07.999821+0000","last_became_active":"2023-10-06T05:27:07.995121+0000","last_became_peered":"2023-10-06T05:27:07.995121+0000","last_unstale":"2023-10-06T05:27:07.999821+0000","last_undegraded":"2023-10-06T05:27:07.999821+0000","last_fullsized":"2023-10-06T05:27:07.999821+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:18.241590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.974486+0000","last_change":"2023-10-06T05:27:06.978724+0000","last_active":"2023-10-06T05:27:07.974486+0000","last_peered":"2023-10-06T05:27:07.974486+0000","last_clean":"2023-10-06T05:27:07.974486+0000","last_became_active":"2023-10-06T05:27:06.978461+0000","last_became_peered":"2023-10-06T05:27:06.978461+0000","last_unstale":"2023-10-06T05:27:07.974486+0000","last_undegraded":"2023-10-06T05:27:07.974486+0000","last_fullsized":"2023-10-06T05:27:07.974486+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:28.251732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990392+0000","last_change":"2023-10-06T05:27:07.985197+0000","last_active":"2023-10-06T05:27:07.990392+0000","last_peered":"2023-10-06T05:27:07.990392+0000","last_clean":"2023-10-06T05:27:07.990392+0000","last_became_active":"2023-10-06T05:27:07.984889+0000","last_became_peered":"2023-10-06T05:27:07.984889+0000","last_unstale":"2023-10-06T05:27:07.990392+0000","last_undegraded":"2023-10-06T05:27:07.990392+0000","last_fullsized":"2023-10-06T05:27:07.990392+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:05.624279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979159+0000","last_change":"2023-10-06T05:27:06.984946+0000","last_active":"2023-10-06T05:27:07.979159+0000","last_peered":"2023-10-06T05:27:07.979159+0000","last_clean":"2023-10-06T05:27:07.979159+0000","last_became_active":"2023-10-06T05:27:06.984133+0000","last_became_peered":"2023-10-06T05:27:06.984133+0000","last_unstale":"2023-10-06T05:27:07.979159+0000","last_undegraded":"2023-10-06T05:27:07.979159+0000","last_fullsized":"2023-10-06T05:27:07.979159+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:09.899166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998163+0000","last_change":"2023-10-06T05:27:07.994897+0000","last_active":"2023-10-06T05:27:07.998163+0000","last_peered":"2023-10-06T05:27:07.998163+0000","last_clean":"2023-10-06T05:27:07.998163+0000","last_became_active":"2023-10-06T05:27:07.994359+0000","last_became_peered":"2023-10-06T05:27:07.994359+0000","last_unstale":"2023-10-06T05:27:07.998163+0000","last_undegraded":"2023-10-06T05:27:07.998163+0000","last_fullsized":"2023-10-06T05:27:07.998163+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:37.035462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999194+0000","last_change":"2023-10-06T05:27:06.979383+0000","last_active":"2023-10-06T05:27:07.999194+0000","last_peered":"2023-10-06T05:27:07.999194+0000","last_clean":"2023-10-06T05:27:07.999194+0000","last_became_active":"2023-10-06T05:27:06.979127+0000","last_became_peered":"2023-10-06T05:27:06.979127+0000","last_unstale":"2023-10-06T05:27:07.999194+0000","last_undegraded":"2023-10-06T05:27:07.999194+0000","last_fullsized":"2023-10-06T05:27:07.999194+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:42.829460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994413+0000","last_change":"2023-10-06T05:27:07.992413+0000","last_active":"2023-10-06T05:27:07.994413+0000","last_peered":"2023-10-06T05:27:07.994413+0000","last_clean":"2023-10-06T05:27:07.994413+0000","last_became_active":"2023-10-06T05:27:07.992094+0000","last_became_peered":"2023-10-06T05:27:07.992094+0000","last_unstale":"2023-10-06T05:27:07.994413+0000","last_undegraded":"2023-10-06T05:27:07.994413+0000","last_fullsized":"2023-10-06T05:27:07.994413+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:39.448394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.971266+0000","last_change":"2023-10-06T05:27:06.976303+0000","last_active":"2023-10-06T05:27:07.971266+0000","last_peered":"2023-10-06T05:27:07.971266+0000","last_clean":"2023-10-06T05:27:07.971266+0000","last_became_active":"2023-10-06T05:27:06.976014+0000","last_became_peered":"2023-10-06T05:27:06.976014+0000","last_unstale":"2023-10-06T05:27:07.971266+0000","last_undegraded":"2023-10-06T05:27:07.971266+0000","last_fullsized":"2023-10-06T05:27:07.971266+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:37.583427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994531+0000","last_change":"2023-10-06T05:27:07.990802+0000","last_active":"2023-10-06T05:27:07.994531+0000","last_peered":"2023-10-06T05:27:07.994531+0000","last_clean":"2023-10-06T05:27:07.994531+0000","last_became_active":"2023-10-06T05:27:07.990508+0000","last_became_peered":"2023-10-06T05:27:07.990508+0000","last_unstale":"2023-10-06T05:27:07.994531+0000","last_undegraded":"2023-10-06T05:27:07.994531+0000","last_fullsized":"2023-10-06T05:27:07.994531+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:51.828771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.974422+0000","last_change":"2023-10-06T05:27:06.981482+0000","last_active":"2023-10-06T05:27:07.974422+0000","last_peered":"2023-10-06T05:27:07.974422+0000","last_clean":"2023-10-06T05:27:07.974422+0000","last_became_active":"2023-10-06T05:27:06.981215+0000","last_became_peered":"2023-10-06T05:27:06.981215+0000","last_unstale":"2023-10-06T05:27:07.974422+0000","last_undegraded":"2023-10-06T05:27:07.974422+0000","last_fullsized":"2023-10-06T05:27:07.974422+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:59.413229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996643+0000","last_change":"2023-10-06T05:27:07.989278+0000","last_active":"2023-10-06T05:27:07.996643+0000","last_peered":"2023-10-06T05:27:07.996643+0000","last_clean":"2023-10-06T05:27:07.996643+0000","last_became_active":"2023-10-06T05:27:07.988846+0000","last_became_peered":"2023-10-06T05:27:07.988846+0000","last_unstale":"2023-10-06T05:27:07.996643+0000","last_undegraded":"2023-10-06T05:27:07.996643+0000","last_fullsized":"2023-10-06T05:27:07.996643+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:50.819400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979153+0000","last_change":"2023-10-06T05:27:06.984272+0000","last_active":"2023-10-06T05:27:07.979153+0000","last_peered":"2023-10-06T05:27:07.979153+0000","last_clean":"2023-10-06T05:27:07.979153+0000","last_became_active":"2023-10-06T05:27:06.984006+0000","last_became_peered":"2023-10-06T05:27:06.984006+0000","last_unstale":"2023-10-06T05:27:07.979153+0000","last_undegraded":"2023-10-06T05:27:07.979153+0000","last_fullsized":"2023-10-06T05:27:07.979153+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:47.440284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.990000+0000","last_change":"2023-10-06T05:27:07.985717+0000","last_active":"2023-10-06T05:27:07.990000+0000","last_peered":"2023-10-06T05:27:07.990000+0000","last_clean":"2023-10-06T05:27:07.990000+0000","last_became_active":"2023-10-06T05:27:07.985464+0000","last_became_peered":"2023-10-06T05:27:07.985464+0000","last_unstale":"2023-10-06T05:27:07.990000+0000","last_undegraded":"2023-10-06T05:27:07.990000+0000","last_fullsized":"2023-10-06T05:27:07.990000+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:41.310817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.970800+0000","last_change":"2023-10-06T05:27:06.984401+0000","last_active":"2023-10-06T05:27:07.970800+0000","last_peered":"2023-10-06T05:27:07.970800+0000","last_clean":"2023-10-06T05:27:07.970800+0000","last_became_active":"2023-10-06T05:27:06.984052+0000","last_became_peered":"2023-10-06T05:27:06.984052+0000","last_unstale":"2023-10-06T05:27:07.970800+0000","last_undegraded":"2023-10-06T05:27:07.970800+0000","last_fullsized":"2023-10-06T05:27:07.970800+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:36.107694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.005812+0000","last_change":"2023-10-06T05:27:07.999284+0000","last_active":"2023-10-06T05:27:08.005812+0000","last_peered":"2023-10-06T05:27:08.005812+0000","last_clean":"2023-10-06T05:27:08.005812+0000","last_became_active":"2023-10-06T05:27:07.999031+0000","last_became_peered":"2023-10-06T05:27:07.999031+0000","last_unstale":"2023-10-06T05:27:08.005812+0000","last_undegraded":"2023-10-06T05:27:08.005812+0000","last_fullsized":"2023-10-06T05:27:08.005812+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:32.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.977393+0000","last_change":"2023-10-06T05:27:06.977765+0000","last_active":"2023-10-06T05:27:07.977393+0000","last_peered":"2023-10-06T05:27:07.977393+0000","last_clean":"2023-10-06T05:27:07.977393+0000","last_became_active":"2023-10-06T05:27:06.975275+0000","last_became_peered":"2023-10-06T05:27:06.975275+0000","last_unstale":"2023-10-06T05:27:07.977393+0000","last_undegraded":"2023-10-06T05:27:07.977393+0000","last_fullsized":"2023-10-06T05:27:07.977393+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:25.787882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.003205+0000","last_change":"2023-10-06T05:27:07.998587+0000","last_active":"2023-10-06T05:27:08.003205+0000","last_peered":"2023-10-06T05:27:08.003205+0000","last_clean":"2023-10-06T05:27:08.003205+0000","last_became_active":"2023-10-06T05:27:07.998132+0000","last_became_peered":"2023-10-06T05:27:07.998132+0000","last_unstale":"2023-10-06T05:27:08.003205+0000","last_undegraded":"2023-10-06T05:27:08.003205+0000","last_fullsized":"2023-10-06T05:27:08.003205+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:43.836288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979023+0000","last_change":"2023-10-06T05:27:06.986788+0000","last_active":"2023-10-06T05:27:07.979023+0000","last_peered":"2023-10-06T05:27:07.979023+0000","last_clean":"2023-10-06T05:27:07.979023+0000","last_became_active":"2023-10-06T05:27:06.986517+0000","last_became_peered":"2023-10-06T05:27:06.986517+0000","last_unstale":"2023-10-06T05:27:07.979023+0000","last_undegraded":"2023-10-06T05:27:07.979023+0000","last_fullsized":"2023-10-06T05:27:07.979023+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:22.462248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994991+0000","last_change":"2023-10-06T05:27:07.993576+0000","last_active":"2023-10-06T05:27:07.994991+0000","last_peered":"2023-10-06T05:27:07.994991+0000","last_clean":"2023-10-06T05:27:07.994991+0000","last_became_active":"2023-10-06T05:27:07.993303+0000","last_became_peered":"2023-10-06T05:27:07.993303+0000","last_unstale":"2023-10-06T05:27:07.994991+0000","last_undegraded":"2023-10-06T05:27:07.994991+0000","last_fullsized":"2023-10-06T05:27:07.994991+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:25.449792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.968602+0000","last_change":"2023-10-06T05:27:06.985051+0000","last_active":"2023-10-06T05:27:07.968602+0000","last_peered":"2023-10-06T05:27:07.968602+0000","last_clean":"2023-10-06T05:27:07.968602+0000","last_became_active":"2023-10-06T05:27:06.984516+0000","last_became_peered":"2023-10-06T05:27:06.984516+0000","last_unstale":"2023-10-06T05:27:07.968602+0000","last_undegraded":"2023-10-06T05:27:07.968602+0000","last_fullsized":"2023-10-06T05:27:07.968602+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:46.452445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996283+0000","last_change":"2023-10-06T05:27:07.985106+0000","last_active":"2023-10-06T05:27:07.996283+0000","last_peered":"2023-10-06T05:27:07.996283+0000","last_clean":"2023-10-06T05:27:07.996283+0000","last_became_active":"2023-10-06T05:27:07.984798+0000","last_became_peered":"2023-10-06T05:27:07.984798+0000","last_unstale":"2023-10-06T05:27:07.996283+0000","last_undegraded":"2023-10-06T05:27:07.996283+0000","last_fullsized":"2023-10-06T05:27:07.996283+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:09.346159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.001242+0000","last_change":"2023-10-06T05:27:06.988368+0000","last_active":"2023-10-06T05:27:08.001242+0000","last_peered":"2023-10-06T05:27:08.001242+0000","last_clean":"2023-10-06T05:27:08.001242+0000","last_became_active":"2023-10-06T05:27:06.988094+0000","last_became_peered":"2023-10-06T05:27:06.988094+0000","last_unstale":"2023-10-06T05:27:08.001242+0000","last_undegraded":"2023-10-06T05:27:08.001242+0000","last_fullsized":"2023-10-06T05:27:08.001242+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:09.719362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":145,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976839+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:27:07.976839+0000","last_peered":"2023-10-06T05:27:07.976839+0000","last_clean":"2023-10-06T05:27:07.976839+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:27:07.976839+0000","last_undegraded":"2023-10-06T05:27:07.976839+0000","last_fullsized":"2023-10-06T05:27:07.976839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993977+0000","last_change":"2023-10-06T05:27:07.989387+0000","last_active":"2023-10-06T05:27:07.993977+0000","last_peered":"2023-10-06T05:27:07.993977+0000","last_clean":"2023-10-06T05:27:07.993977+0000","last_became_active":"2023-10-06T05:27:07.989130+0000","last_became_peered":"2023-10-06T05:27:07.989130+0000","last_unstale":"2023-10-06T05:27:07.993977+0000","last_undegraded":"2023-10-06T05:27:07.993977+0000","last_fullsized":"2023-10-06T05:27:07.993977+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:14.585917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998051+0000","last_change":"2023-10-06T05:27:06.987525+0000","last_active":"2023-10-06T05:27:07.998051+0000","last_peered":"2023-10-06T05:27:07.998051+0000","last_clean":"2023-10-06T05:27:07.998051+0000","last_became_active":"2023-10-06T05:27:06.987238+0000","last_became_peered":"2023-10-06T05:27:06.987238+0000","last_unstale":"2023-10-06T05:27:07.998051+0000","last_undegraded":"2023-10-06T05:27:07.998051+0000","last_fullsized":"2023-10-06T05:27:07.998051+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:48.771305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.978774+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:27:07.978774+0000","last_peered":"2023-10-06T05:27:07.978774+0000","last_clean":"2023-10-06T05:27:07.978774+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:27:07.978774+0000","last_undegraded":"2023-10-06T05:27:07.978774+0000","last_fullsized":"2023-10-06T05:27:07.978774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.999822+0000","last_change":"2023-10-06T05:27:07.996214+0000","last_active":"2023-10-06T05:27:07.999822+0000","last_peered":"2023-10-06T05:27:07.999822+0000","last_clean":"2023-10-06T05:27:07.999822+0000","last_became_active":"2023-10-06T05:27:07.995945+0000","last_became_peered":"2023-10-06T05:27:07.995945+0000","last_unstale":"2023-10-06T05:27:07.999822+0000","last_undegraded":"2023-10-06T05:27:07.999822+0000","last_fullsized":"2023-10-06T05:27:07.999822+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:31.065787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.977844+0000","last_change":"2023-10-06T05:27:06.979223+0000","last_active":"2023-10-06T05:27:07.977844+0000","last_peered":"2023-10-06T05:27:07.977844+0000","last_clean":"2023-10-06T05:27:07.977844+0000","last_became_active":"2023-10-06T05:27:06.978972+0000","last_became_peered":"2023-10-06T05:27:06.978972+0000","last_unstale":"2023-10-06T05:27:07.977844+0000","last_undegraded":"2023-10-06T05:27:07.977844+0000","last_fullsized":"2023-10-06T05:27:07.977844+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:15.865977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.969240+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:27:07.969240+0000","last_peered":"2023-10-06T05:27:07.969240+0000","last_clean":"2023-10-06T05:27:07.969240+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:27:07.969240+0000","last_undegraded":"2023-10-06T05:27:07.969240+0000","last_fullsized":"2023-10-06T05:27:07.969240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.995604+0000","last_change":"2023-10-06T05:27:07.988312+0000","last_active":"2023-10-06T05:27:07.995604+0000","last_peered":"2023-10-06T05:27:07.995604+0000","last_clean":"2023-10-06T05:27:07.995604+0000","last_became_active":"2023-10-06T05:27:07.988019+0000","last_became_peered":"2023-10-06T05:27:07.988019+0000","last_unstale":"2023-10-06T05:27:07.995604+0000","last_undegraded":"2023-10-06T05:27:07.995604+0000","last_fullsized":"2023-10-06T05:27:07.995604+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:38.746128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.997773+0000","last_change":"2023-10-06T05:27:06.981344+0000","last_active":"2023-10-06T05:27:07.997773+0000","last_peered":"2023-10-06T05:27:07.997773+0000","last_clean":"2023-10-06T05:27:07.997773+0000","last_became_active":"2023-10-06T05:27:06.980236+0000","last_became_peered":"2023-10-06T05:27:06.980236+0000","last_unstale":"2023-10-06T05:27:07.997773+0000","last_undegraded":"2023-10-06T05:27:07.997773+0000","last_fullsized":"2023-10-06T05:27:07.997773+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:20.528042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.979362+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:27:07.979362+0000","last_peered":"2023-10-06T05:27:07.979362+0000","last_clean":"2023-10-06T05:27:07.979362+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:27:07.979362+0000","last_undegraded":"2023-10-06T05:27:07.979362+0000","last_fullsized":"2023-10-06T05:27:07.979362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":295,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976528+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:27:07.976528+0000","last_peered":"2023-10-06T05:27:07.976528+0000","last_clean":"2023-10-06T05:27:07.976528+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:27:07.976528+0000","last_undegraded":"2023-10-06T05:27:07.976528+0000","last_fullsized":"2023-10-06T05:27:07.976528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.993896+0000","last_change":"2023-10-06T05:27:07.988732+0000","last_active":"2023-10-06T05:27:07.993896+0000","last_peered":"2023-10-06T05:27:07.993896+0000","last_clean":"2023-10-06T05:27:07.993896+0000","last_became_active":"2023-10-06T05:27:07.988458+0000","last_became_peered":"2023-10-06T05:27:07.988458+0000","last_unstale":"2023-10-06T05:27:07.993896+0000","last_undegraded":"2023-10-06T05:27:07.993896+0000","last_fullsized":"2023-10-06T05:27:07.993896+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:49.778132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976652+0000","last_change":"2023-10-06T05:27:06.988126+0000","last_active":"2023-10-06T05:27:07.976652+0000","last_peered":"2023-10-06T05:27:07.976652+0000","last_clean":"2023-10-06T05:27:07.976652+0000","last_became_active":"2023-10-06T05:27:06.987867+0000","last_became_peered":"2023-10-06T05:27:06.987867+0000","last_unstale":"2023-10-06T05:27:07.976652+0000","last_undegraded":"2023-10-06T05:27:07.976652+0000","last_fullsized":"2023-10-06T05:27:07.976652+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:47.111387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976433+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:27:07.976433+0000","last_peered":"2023-10-06T05:27:07.976433+0000","last_clean":"2023-10-06T05:27:07.976433+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:27:07.976433+0000","last_undegraded":"2023-10-06T05:27:07.976433+0000","last_fullsized":"2023-10-06T05:27:07.976433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.994383+0000","last_change":"2023-10-06T05:27:07.991295+0000","last_active":"2023-10-06T05:27:07.994383+0000","last_peered":"2023-10-06T05:27:07.994383+0000","last_clean":"2023-10-06T05:27:07.994383+0000","last_became_active":"2023-10-06T05:27:07.991046+0000","last_became_peered":"2023-10-06T05:27:07.991046+0000","last_unstale":"2023-10-06T05:27:07.994383+0000","last_undegraded":"2023-10-06T05:27:07.994383+0000","last_fullsized":"2023-10-06T05:27:07.994383+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:04.410349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"180'2","reported_seq":12,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.998337+0000","last_change":"2023-10-06T05:27:06.979266+0000","last_active":"2023-10-06T05:27:07.998337+0000","last_peered":"2023-10-06T05:27:07.998337+0000","last_clean":"2023-10-06T05:27:07.998337+0000","last_became_active":"2023-10-06T05:27:06.978989+0000","last_became_peered":"2023-10-06T05:27:06.978989+0000","last_unstale":"2023-10-06T05:27:07.998337+0000","last_undegraded":"2023-10-06T05:27:07.998337+0000","last_fullsized":"2023-10-06T05:27:07.998337+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:27.203874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.977838+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:27:07.977838+0000","last_peered":"2023-10-06T05:27:07.977838+0000","last_clean":"2023-10-06T05:27:07.977838+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:27:07.977838+0000","last_undegraded":"2023-10-06T05:27:07.977838+0000","last_fullsized":"2023-10-06T05:27:07.977838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996968+0000","last_change":"2023-10-06T05:27:07.994984+0000","last_active":"2023-10-06T05:27:07.996968+0000","last_peered":"2023-10-06T05:27:07.996968+0000","last_clean":"2023-10-06T05:27:07.996968+0000","last_became_active":"2023-10-06T05:27:07.994713+0000","last_became_peered":"2023-10-06T05:27:07.994713+0000","last_unstale":"2023-10-06T05:27:07.996968+0000","last_undegraded":"2023-10-06T05:27:07.996968+0000","last_fullsized":"2023-10-06T05:27:07.996968+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:11.586742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"180'4","reported_seq":14,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.000890+0000","last_change":"2023-10-06T05:27:06.979448+0000","last_active":"2023-10-06T05:27:08.000890+0000","last_peered":"2023-10-06T05:27:08.000890+0000","last_clean":"2023-10-06T05:27:08.000890+0000","last_became_active":"2023-10-06T05:27:06.979111+0000","last_became_peered":"2023-10-06T05:27:06.979111+0000","last_unstale":"2023-10-06T05:27:08.000890+0000","last_undegraded":"2023-10-06T05:27:08.000890+0000","last_fullsized":"2023-10-06T05:27:08.000890+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:04.135245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.976162+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:27:07.976162+0000","last_peered":"2023-10-06T05:27:07.976162+0000","last_clean":"2023-10-06T05:27:07.976162+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:27:07.976162+0000","last_undegraded":"2023-10-06T05:27:07.976162+0000","last_fullsized":"2023-10-06T05:27:07.976162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.996852+0000","last_change":"2023-10-06T05:27:07.995504+0000","last_active":"2023-10-06T05:27:07.996852+0000","last_peered":"2023-10-06T05:27:07.996852+0000","last_clean":"2023-10-06T05:27:07.996852+0000","last_became_active":"2023-10-06T05:27:07.995145+0000","last_became_peered":"2023-10-06T05:27:07.995145+0000","last_unstale":"2023-10-06T05:27:07.996852+0000","last_undegraded":"2023-10-06T05:27:07.996852+0000","last_fullsized":"2023-10-06T05:27:07.996852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:06.957121+0000","last_clean_scrub_stamp":"2023-10-06T05:27:06.957121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:46.040029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"180'1","reported_seq":11,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:08.002107+0000","last_change":"2023-10-06T05:27:06.988055+0000","last_active":"2023-10-06T05:27:08.002107+0000","last_peered":"2023-10-06T05:27:08.002107+0000","last_clean":"2023-10-06T05:27:08.002107+0000","last_became_active":"2023-10-06T05:27:06.987653+0000","last_became_peered":"2023-10-06T05:27:06.987653+0000","last_unstale":"2023-10-06T05:27:08.002107+0000","last_undegraded":"2023-10-06T05:27:08.002107+0000","last_fullsized":"2023-10-06T05:27:08.002107+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:05.950293+0000","last_clean_scrub_stamp":"2023-10-06T05:27:05.950293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:11.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":168,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-06T05:27:07.970444+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:27:07.970444+0000","last_peered":"2023-10-06T05:27:07.970444+0000","last_clean":"2023-10-06T05:27:07.970444+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:27:07.970444+0000","last_undegraded":"2023-10-06T05:27:07.970444+0000","last_fullsized":"2023-10-06T05:27:07.970444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304196,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067644,"statfs":{"total":96636764160,"available":96325267456,"internally_reserved":0,"allocated":2879488,"data_stored":1644402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.47099999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.03,"5min":1.1339999999999999,"15min":1.169},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0249999999999999,"15min":1.268},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.075},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.58099999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.876,"5min":1.0009999999999999,"15min":1.093},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":1.22},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.57099999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.079},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.75800000000000001},"min":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.46600000000000003},"max":{"1min":0.98899999999999999,"5min":1.0960000000000001,"15min":1.202},"last":0.89600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.32200000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.258},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.42199999999999999,"15min":0.39400000000000002},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":1.032}]},{"osd":5,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.109,"15min":1.109},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.497,"5min":0.44,"15min":0.44},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.407},"last":1.0800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.117},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.0489999999999999,"5min":1.135,"15min":1.3300000000000001},"last":1.097}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303628,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068212,"statfs":{"total":96636764160,"available":96325849088,"internally_reserved":0,"allocated":2297856,"data_stored":1061159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.67200000000000004,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.373},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.46899999999999997,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69299999999999995,"15min":0.72599999999999998},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.44900000000000001,"15min":0.442},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.57299999999999995,"5min":0.45300000000000001,"15min":0.40899999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.73099999999999998,"15min":0.76600000000000001},"min":{"1min":0.56599999999999995,"5min":0.56499999999999995,"15min":0.5},"max":{"1min":0.92500000000000004,"5min":0.93200000000000005,"15min":1.1020000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.42099999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.998},"last":0.504}]},{"osd":5,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0760000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":0.94899999999999995,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":0.94199999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.78000000000000003},"min":{"1min":0.57199999999999995,"5min":0.496,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.006,"15min":1.1359999999999999},"last":0.55900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303612,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068228,"statfs":{"total":96636764160,"available":96325865472,"internally_reserved":0,"allocated":2281472,"data_stored":1059069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.28999999999999998},"max":{"1min":1.165,"5min":1.165,"15min":2.6520000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":1.077,"5min":1.077,"15min":2.7010000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.629},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.40000000000000002,"15min":0.35199999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75,"15min":0.76900000000000002},"min":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.50700000000000001},"max":{"1min":1.111,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.60199999999999998,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.087,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.79200000000000004,"15min":0.79600000000000004},"min":{"1min":0.64300000000000002,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.1379999999999999,"5min":1.25,"15min":1.349},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.68200000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.159,"5min":1.2569999999999999,"15min":1.669},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.83199999999999996,"15min":0.83699999999999997},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.503},"max":{"1min":1.1299999999999999,"5min":1.3280000000000001,"15min":1.573},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.196,"5min":1.196,"15min":1.528},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82699999999999996,"15min":0.83399999999999996},"min":{"1min":0.65600000000000003,"5min":0.53400000000000003,"15min":0.48699999999999999},"max":{"1min":1.1879999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304196,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067644,"statfs":{"total":96636764160,"available":96325267456,"internally_reserved":0,"allocated":2879488,"data_stored":1649527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.32200000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.41699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.33500000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79900000000000004,"15min":0.77500000000000002},"min":{"1min":0.621,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":1.258,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79100000000000004,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0609999999999999,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.218},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.748},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.53400000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.121,"5min":1.153,"15min":1.2629999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.65300000000000002,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.25},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.60699999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.2789999999999999},"last":0.69999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303616,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068224,"statfs":{"total":96636764160,"available":96325861376,"internally_reserved":0,"allocated":2285568,"data_stored":1059159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":0.86899999999999999,"5min":1.056,"15min":1.056},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.67400000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.80600000000000005,"5min":1.073,"15min":1.073},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79100000000000004},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93700000000000006,"5min":1.109,"15min":1.1379999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.67500000000000004,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.99399999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.746}]},{"osd":5,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74399999999999999,"15min":0.76700000000000002},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93899999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77900000000000003,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":0.97799999999999998,"5min":1.1859999999999999,"15min":1.319},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.80600000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77700000000000002,"15min":0.78800000000000003},"min":{"1min":0.63100000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0209999999999999,"5min":1.137,"15min":1.141},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78800000000000003,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.52100000000000002},"max":{"1min":0.95699999999999996,"5min":1.284,"15min":1.284},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79800000000000004,"15min":0.81899999999999995},"min":{"1min":0.68899999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97599999999999998,"5min":1.2,"15min":1.3520000000000001},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303624,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068216,"statfs":{"total":96636764160,"available":96325853184,"internally_reserved":0,"allocated":2293760,"data_stored":1059195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45700000000000002,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":2.0950000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46800000000000003,"15min":0.48899999999999999},"min":{"1min":0.33000000000000002,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.772},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.369,"5min":1.369,"15min":1.913},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.55200000000000005},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.307},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.32900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.954},"last":0.505},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.81599999999999995,"5min":0.86599999999999999,"15min":2.0539999999999998},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.748,"15min":0.76400000000000001},"min":{"1min":0.54800000000000004,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.282},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76000000000000001,"15min":0.78500000000000003},"min":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.48899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.51100000000000001},"max":{"1min":1.0509999999999999,"5min":1.0840000000000001,"15min":1.1930000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.65200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.79900000000000004,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.53900000000000003},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.343},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:26:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.80700000000000005,"15min":0.83099999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.56799999999999995},"max":{"1min":1.1419999999999999,"5min":1.194,"15min":1.3640000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82099999999999995,"15min":0.83399999999999996},"min":{"1min":0.66700000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.036,"5min":1.1579999999999999,"15min":1.214},"last":0.748}]}]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303616,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068224,"statfs":{"total":96636764160,"available":96325861376,"internally_reserved":0,"allocated":2285568,"data_stored":1059623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89300000000000002,"5min":0.98699999999999999,"15min":1.073},"last":0.621},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.46100000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.169},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.41199999999999998},"max":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":1.0569999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.57699999999999996,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":1.0309999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68400000000000005,"15min":0.71399999999999997},"min":{"1min":0.52600000000000002,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.117},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.60899999999999999,"5min":0.42999999999999999,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97099999999999997,"15min":1.002},"last":0.877}]},{"osd":3,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.875,"5min":0.94999999999999996,"15min":1.0780000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.747,"15min":0.753},"min":{"1min":0.58299999999999996,"5min":0.54400000000000004,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":1.018,"15min":1.196},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.8740000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.375},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.72099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.081,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.98599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.466},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.218},"last":0.97099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303620,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068220,"statfs":{"total":96636764160,"available":96325857280,"internally_reserved":0,"allocated":2289664,"data_stored":1059125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.71199999999999997,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.374},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.746}]},{"osd":1,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72199999999999998,"15min":0.70899999999999996},"min":{"1min":0.54500000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95099999999999996,"5min":0.98199999999999998,"15min":1.204},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.89900000000000002,"5min":0.95199999999999996,"15min":1.087},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.57299999999999995,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.278},"last":0.72099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":1.004,"15min":1.294},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.89900000000000002,"15min":1.141},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":0.93100000000000005,"5min":0.995,"15min":1.254},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.337},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.75700000000000001},"min":{"1min":0.52000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94499999999999995,"5min":1.0600000000000001,"15min":1.248},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:27:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:27:12.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:27:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.062+0000 7faedb59d700 1 Processor -- start 2023-10-06T05:27:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.062+0000 7faedb59d700 1 -- start start 2023-10-06T05:27:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.063+0000 7faedb59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faed415a7d0 0x7faed40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.063+0000 7faedb59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed415ddb0 con 0x7faed40cced0 2023-10-06T05:27:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.063+0000 7faedb59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed415d1a0 con 0x7faed40af540 2023-10-06T05:27:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.063+0000 7faedb59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed415d480 con 0x7faed415a7d0 2023-10-06T05:27:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faed415a7d0 0x7faed40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faed415a7d0 0x7faed40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58174/0 (socket says 172.21.15.33:58174) 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 -- 172.21.15.33:0/1113482386 learned_addr learned my addr 172.21.15.33:0/1113482386 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faedad9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faed40cced0 0x7faed40ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37436/0 (socket says 172.21.15.33:37436) 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 -- 172.21.15.33:0/1113482386 >> v1:172.21.15.167:6789/0 conn(0x7faed40af540 legacy=0x7faed40cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 -- 172.21.15.33:0/1113482386 >> v1:172.21.15.33:6789/0 conn(0x7faed40cced0 legacy=0x7faed40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 -- 172.21.15.33:0/1113482386 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed415d760 con 0x7faed415a7d0 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9599700 1 -- 172.21.15.33:0/1113482386 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217051896 0 0) 0x7faed415ddb0 con 0x7faed40cced0 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.064+0000 7faed9d9a700 1 --2- 172.21.15.33:0/1113482386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed40af910 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7faec40099b0 tx=0x7faec4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb5d0a9d65ff03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faed9599700 1 -- 172.21.15.33:0/1113482386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400b860 con 0x7faed415a7d0 2023-10-06T05:27:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faed9599700 1 -- 172.21.15.33:0/1113482386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec400b9c0 con 0x7faed415a7d0 2023-10-06T05:27:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faed9599700 1 -- 172.21.15.33:0/1113482386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400bd60 con 0x7faed415a7d0 2023-10-06T05:27:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 -- 172.21.15.33:0/1113482386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 msgr2=0x7faed40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 --2- 172.21.15.33:0/1113482386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed40af910 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7faec40099b0 tx=0x7faec4009250 comp rx=0 tx=0).stop 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 -- 172.21.15.33:0/1113482386 shutdown_connections 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 --2- 172.21.15.33:0/1113482386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed40af910 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 -- 172.21.15.33:0/1113482386 >> 172.21.15.33:0/1113482386 conn(0x7faed406d9b0 msgr2=0x7faed406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 -- 172.21.15.33:0/1113482386 shutdown_connections 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.065+0000 7faedb59d700 1 -- 172.21.15.33:0/1113482386 wait complete. 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.067+0000 7faedb59d700 1 Processor -- start 2023-10-06T05:27:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.067+0000 7faedb59d700 1 -- start start 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedb59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed40af540 0x7faed40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedb59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed40cced0 0x7faed4159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedb59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedb59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed416d060 con 0x7faed415a7d0 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedb59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed416d8c0 con 0x7faed40cced0 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedb59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed416a7c0 con 0x7faed40af540 2023-10-06T05:27:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58190/0 (socket says 172.21.15.33:58190) 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faedad9c700 1 -- 172.21.15.33:0/3742606805 learned_addr learned my addr 172.21.15.33:0/3742606805 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faed9d9a700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed40cced0 0x7faed4159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.068+0000 7faeda59b700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed40af540 0x7faed40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedad9c700 1 -- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed40af540 msgr2=0x7faed40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedad9c700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed40af540 0x7faed40c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedad9c700 1 -- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed40cced0 msgr2=0x7faed4159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedad9c700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed40cced0 0x7faed4159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedad9c700 1 -- 172.21.15.33:0/3742606805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40094f0 con 0x7faed415a7d0 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedad9c700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed415a060 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7faecc00cd30 tx=0x7faecc001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ce90 con 0x7faed415a7d0 2023-10-06T05:27:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc005df0 con 0x7faed415a7d0 2023-10-06T05:27:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00aaa0 con 0x7faed415a7d0 2023-10-06T05:27:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed416d270 con 0x7faed415a7d0 2023-10-06T05:27:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.069+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed40c9d10 con 0x7faed415a7d0 2023-10-06T05:27:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.070+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed416da50 con 0x7faed415a7d0 2023-10-06T05:27:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.071+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faecc0070f0 con 0x7faed415a7d0 2023-10-06T05:27:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.072+0000 7faeca7fc700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faeb8066f30 0x7faeb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.072+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faecc0846f0 con 0x7faed415a7d0 2023-10-06T05:27:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.073+0000 7faeda59b700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faeb8066f30 0x7faeb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.073+0000 7faeda59b700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faeb8066f30 0x7faeb80693e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faed406e720 tx=0x7faed0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.076+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faecc051880 con 0x7faed415a7d0 2023-10-06T05:27:13.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.360+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faed404c380 con 0x7faed415a7d0 2023-10-06T05:27:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.364+0000 7faeca7fc700 1 -- 172.21.15.33:0/3742606805 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7faecc054ea0 con 0x7faed415a7d0 2023-10-06T05:27:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faeb8066f30 msgr2=0x7faeb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faeb8066f30 0x7faeb80693e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faed406e720 tx=0x7faed0009040 comp rx=0 tx=0).stop 2023-10-06T05:27:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 msgr2=0x7faed415a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed415a060 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7faecc00cd30 tx=0x7faecc001ac0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 shutdown_connections 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faeb8066f30 0x7faeb80693e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faed415a7d0 0x7faed415a060 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed40af540 0x7faed40c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 --2- 172.21.15.33:0/3742606805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed40cced0 0x7faed4159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 >> 172.21.15.33:0/3742606805 conn(0x7faed406d9b0 msgr2=0x7faed41569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 shutdown_connections 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.367+0000 7faedb59d700 1 -- 172.21.15.33:0/3742606805 wait complete. 2023-10-06T05:27:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:27:13.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.572+0000 7f0363fff700 1 Processor -- start 2023-10-06T05:27:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.573+0000 7f0363fff700 1 -- start start 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f0363fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f036413faa0 0x7f036413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f0363fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f036406aad0 0x7f036406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f0363fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03640c9cb0 con 0x7f03640c7940 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f0363fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640ca270 con 0x7f036406aad0 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f0363fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036406b840 con 0x7f036413faa0 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f03637fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f036406aad0 0x7f036406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f03637fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f036406aad0 0x7f036406af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58192/0 (socket says 172.21.15.33:58192) 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.574+0000 7f03637fe700 1 -- 172.21.15.33:0/4238206071 learned_addr learned my addr 172.21.15.33:0/4238206071 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f03637fe700 1 -- 172.21.15.33:0/4238206071 >> v2:172.21.15.167:3300/0 conn(0x7f036413faa0 msgr2=0x7f036413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f0362ffd700 1 --2- 172.21.15.33:0/4238206071 >> v2:172.21.15.167:3300/0 conn(0x7f036413faa0 0x7f036413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f03637fe700 1 --2- 172.21.15.33:0/4238206071 >> v2:172.21.15.167:3300/0 conn(0x7f036413faa0 0x7f036413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f03637fe700 1 -- 172.21.15.33:0/4238206071 >> v1:172.21.15.167:6789/0 conn(0x7f03640c7940 legacy=0x7f03640c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f03637fe700 1 -- 172.21.15.33:0/4238206071 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036406bb20 con 0x7f036406aad0 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f0362ffd700 1 --2- 172.21.15.33:0/4238206071 >> v2:172.21.15.167:3300/0 conn(0x7f036413faa0 0x7f036413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f03637fe700 1 --2- 172.21.15.33:0/4238206071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f036406aad0 0x7f036406af30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f036406fe50 tx=0x7f035c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed44819aa45aafc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f0361ffb700 1 -- 172.21.15.33:0/4238206071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00e070 con 0x7f036406aad0 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f0361ffb700 1 -- 172.21.15.33:0/4238206071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c00ba10 con 0x7f036406aad0 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.575+0000 7f0361ffb700 1 -- 172.21.15.33:0/4238206071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0092e0 con 0x7f036406aad0 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 -- 172.21.15.33:0/4238206071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f036406aad0 msgr2=0x7f036406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 --2- 172.21.15.33:0/4238206071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f036406aad0 0x7f036406af30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f036406fe50 tx=0x7f035c009840 comp rx=0 tx=0).stop 2023-10-06T05:27:13.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 -- 172.21.15.33:0/4238206071 shutdown_connections 2023-10-06T05:27:13.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 --2- 172.21.15.33:0/4238206071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f036406aad0 0x7f036406af30 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 --2- 172.21.15.33:0/4238206071 >> v2:172.21.15.167:3300/0 conn(0x7f036413faa0 0x7f036413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 -- 172.21.15.33:0/4238206071 >> 172.21.15.33:0/4238206071 conn(0x7f036406d030 msgr2=0x7f036406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:13.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 -- 172.21.15.33:0/4238206071 shutdown_connections 2023-10-06T05:27:13.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.576+0000 7f0363fff700 1 -- 172.21.15.33:0/4238206071 wait complete. 2023-10-06T05:27:13.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.577+0000 7f0363fff700 1 Processor -- start 2023-10-06T05:27:13.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.578+0000 7f0363fff700 1 -- start start 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.578+0000 7f0363fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036406aad0 0x7f03640c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0363fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 0x7f0364156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0363fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f036413faa0 0x7f0364156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0363fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036413e810 con 0x7f03640c7940 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0363fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364156da0 con 0x7f036406aad0 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0363fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036413df70 con 0x7f036413faa0 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 0x7f0364156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 0x7f0364156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58200/0 (socket says 172.21.15.33:58200) 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 -- 172.21.15.33:0/3783991375 learned_addr learned my addr 172.21.15.33:0/3783991375 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0362ffd700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036406aad0 0x7f03640c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03637fe700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f036413faa0 0x7f0364156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 -- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f036413faa0 msgr2=0x7f0364156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f036413faa0 0x7f0364156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 -- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036406aad0 msgr2=0x7f03640c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036406aad0 0x7f03640c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 -- 172.21.15.33:0/3783991375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c009ae0 con 0x7f03640c7940 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f0362ffd700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036406aad0 0x7f03640c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03637fe700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f036413faa0 0x7f0364156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:13.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.579+0000 7f03627fc700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 0x7f0364156050 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f035800c600 tx=0x7f0358007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.580+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358011590 con 0x7f03640c7940 2023-10-06T05:27:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.580+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0358011710 con 0x7f03640c7940 2023-10-06T05:27:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.580+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358011ad0 con 0x7f03640c7940 2023-10-06T05:27:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.580+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036413e250 con 0x7f03640c7940 2023-10-06T05:27:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.580+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03640c2610 con 0x7f03640c7940 2023-10-06T05:27:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.581+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03640c2830 con 0x7f03640c7940 2023-10-06T05:27:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.582+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f035801c460 con 0x7f03640c7940 2023-10-06T05:27:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.583+0000 7f034affd700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0350066f30 0x7f03500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.584+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0358084510 con 0x7f03640c7940 2023-10-06T05:27:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.584+0000 7f0362ffd700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0350066f30 0x7f03500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.585+0000 7f0362ffd700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0350066f30 0x7f03500693e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.587+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0358051580 con 0x7f03640c7940 2023-10-06T05:27:13.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.834+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f036404c380 con 0x7f03640c7940 2023-10-06T05:27:13.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.834+0000 7f034affd700 1 -- 172.21.15.33:0/3783991375 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f0358054ba0 con 0x7f03640c7940 2023-10-06T05:27:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:13.837 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":132,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:10.994747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9405":{"gid":9405,"name":"c","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/950449971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":950449971},{"type":"v1","addr":"172.21.15.33:6835","nonce":950449971}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:13.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0350066f30 msgr2=0x7f03500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0350066f30 0x7f03500693e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354008040 comp rx=0 tx=0).stop 2023-10-06T05:27:13.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 msgr2=0x7f0364156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 0x7f0364156050 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f035800c600 tx=0x7f0358007f60 comp rx=0 tx=0).stop 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 shutdown_connections 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0350066f30 0x7f03500693e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f036413faa0 0x7f0364156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036406aad0 0x7f03640c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 --2- 172.21.15.33:0/3783991375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03640c7940 0x7f0364156050 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.836+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 >> 172.21.15.33:0/3783991375 conn(0x7f036406d030 msgr2=0x7f03640b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.837+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 shutdown_connections 2023-10-06T05:27:13.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:13.837+0000 7f0363fff700 1 -- 172.21.15.33:0/3783991375 wait complete. 2023-10-06T05:27:13.841 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 132 2023-10-06T05:27:14.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:27:07.463944+0000', 'modified': '2023-10-06T05:27:10.994747+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9405}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9405': {'gid': 9405, 'name': 'c', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/950449971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 950449971}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 950449971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:27:15.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:27:15.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:27:15.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.502+0000 7fdeb09fe700 1 Processor -- start 2023-10-06T05:27:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.502+0000 7fdeb09fe700 1 -- start start 2023-10-06T05:27:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.503+0000 7fdeb09fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdeac0c7070 0x7fdeac0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.503+0000 7fdeb09fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdeac155bc0 0x7fdeac0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.503+0000 7fdeb09fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac06a410 con 0x7fdeac1557f0 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.503+0000 7fdeb09fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0cccd0 con 0x7fdeac155bc0 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.503+0000 7fdeb09fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0c7500 con 0x7fdeac0c7070 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdeac155bc0 0x7fdeac0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdeac155bc0 0x7fdeac0c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58210/0 (socket says 172.21.15.33:58210) 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 -- 172.21.15.33:0/1355376184 learned_addr learned my addr 172.21.15.33:0/1355376184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/1355376184 >> v2:172.21.15.167:3300/0 conn(0x7fdeac0c7070 0x7fdeac0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:15.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 -- 172.21.15.33:0/1355376184 >> v2:172.21.15.167:3300/0 conn(0x7fdeac0c7070 msgr2=0x7fdeac0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 --2- 172.21.15.33:0/1355376184 >> v2:172.21.15.167:3300/0 conn(0x7fdeac0c7070 0x7fdeac0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 -- 172.21.15.33:0/1355376184 >> v1:172.21.15.167:6789/0 conn(0x7fdeac1557f0 legacy=0x7fdeac0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 -- 172.21.15.33:0/1355376184 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0c77e0 con 0x7fdeac155bc0 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.504+0000 7fdeab59e700 1 --2- 172.21.15.33:0/1355376184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac155bc0 0x7fdeac0c9050 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0b22f0 tx=0x7fdea4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fc44e2462d944b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdea9d9b700 1 -- 172.21.15.33:0/1355376184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400e070 con 0x7fdeac155bc0 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdea9d9b700 1 -- 172.21.15.33:0/1355376184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400ba00 con 0x7fdeac155bc0 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdea9d9b700 1 -- 172.21.15.33:0/1355376184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40092e0 con 0x7fdeac155bc0 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1355376184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac155bc0 msgr2=0x7fdeac0c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1355376184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac155bc0 0x7fdeac0c9050 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0b22f0 tx=0x7fdea4009840 comp rx=0 tx=0).stop 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1355376184 shutdown_connections 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1355376184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac155bc0 0x7fdeac0c9050 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1355376184 >> v2:172.21.15.167:3300/0 conn(0x7fdeac0c7070 0x7fdeac0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1355376184 >> 172.21.15.33:0/1355376184 conn(0x7fdeac06c560 msgr2=0x7fdeac06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1355376184 shutdown_connections 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.505+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1355376184 wait complete. 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.507+0000 7fdeb09fe700 1 Processor -- start 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.507+0000 7fdeb09fe700 1 -- start start 2023-10-06T05:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeb09fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdeac0c7070 0x7fdeac0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeb09fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 0x7fdeac164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeb09fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeac155bc0 0x7fdeac157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeb09fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b14e0 con 0x7fdeac1557f0 2023-10-06T05:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeb09fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac154480 con 0x7fdeac155bc0 2023-10-06T05:27:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeb09fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac165e50 con 0x7fdeac0c7070 2023-10-06T05:27:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeaa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 0x7fdeac164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeaa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 0x7fdeac164ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58216/0 (socket says 172.21.15.33:58216) 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeaa59c700 1 -- 172.21.15.33:0/1922290137 learned_addr learned my addr 172.21.15.33:0/1922290137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdeac0c7070 0x7fdeac0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.508+0000 7fdeab59e700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeac155bc0 0x7fdeac157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaa59c700 1 -- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdeac0c7070 msgr2=0x7fdeac0b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdeac0c7070 0x7fdeac0b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaa59c700 1 -- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeac155bc0 msgr2=0x7fdeac157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeac155bc0 0x7fdeac157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaa59c700 1 -- 172.21.15.33:0/1922290137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4009ae0 con 0x7fdeac1557f0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeab59e700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeac155bc0 0x7fdeac157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdeac0c7070 0x7fdeac0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 0x7fdeac164ee0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fdea000d860 tx=0x7fdea0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fde92ffd700 1 -- 172.21.15.33:0/1922290137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00125f0 con 0x7fdeac1557f0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fde92ffd700 1 -- 172.21.15.33:0/1922290137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea0012750 con 0x7fdeac1557f0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac165890 con 0x7fdeac1557f0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fde92ffd700 1 -- 172.21.15.33:0/1922290137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0012b10 con 0x7fdeac1557f0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.509+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0b1100 con 0x7fdeac1557f0 2023-10-06T05:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.511+0000 7fde92ffd700 1 -- 172.21.15.33:0/1922290137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdea001e460 con 0x7fdeac1557f0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.512+0000 7fde92ffd700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde98066f30 0x7fde980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.512+0000 7fde92ffd700 1 -- 172.21.15.33:0/1922290137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdea0084d50 con 0x7fdeac1557f0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.513+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde98066f30 0x7fde980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.514+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde98066f30 0x7fde980693e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fde9c007fd0 tx=0x7fde9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.514+0000 7fdebf994b80 1 Processor -- start 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.514+0000 7fdebf994b80 1 -- start start 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b420750d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420779cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b42077a210 0x55b420797290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdebf994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4206d0c80 con 0x55b420750950 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4206a7a70 con 0x55b420751dd0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4206a5270 con 0x55b42077a210 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdeaad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b420750d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdeaad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b420750d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58226/0 (socket says 172.21.15.33:58226) 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.515+0000 7fdeaad9d700 1 -- 172.21.15.33:0/350094906 learned_addr learned my addr 172.21.15.33:0/350094906 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420779cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeab59e700 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b42077a210 0x55b420797290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaad9d700 1 -- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b42077a210 msgr2=0x55b420797290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b42077a210 0x55b420797290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaad9d700 1 -- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 msgr2=0x55b420779cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420779cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaad9d700 1 -- 172.21.15.33:0/350094906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b42053b270 con 0x55b420750950 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeab59e700 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b42077a210 0x55b420797290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b420750d20 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00e9a0 tx=0x7fde9c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edfa4da3e028c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fde8bfff700 1 -- 172.21.15.33:0/350094906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0157f0 con 0x55b420750950 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fde8bfff700 1 -- 172.21.15.33:0/350094906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c015950 con 0x55b420750950 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.516+0000 7fde8bfff700 1 -- 172.21.15.33:0/350094906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c015cd0 con 0x55b420750950 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 -- 172.21.15.33:0/350094906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 msgr2=0x55b420750d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b420750d20 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00e9a0 tx=0x7fde9c015000 comp rx=0 tx=0).stop 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 -- 172.21.15.33:0/350094906 shutdown_connections 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b42077a210 0x55b420797290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b420750d20 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 --2- 172.21.15.33:0/350094906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420779cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 -- 172.21.15.33:0/350094906 >> 172.21.15.33:0/350094906 conn(0x55b420752270 msgr2=0x55b420752640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 -- 172.21.15.33:0/350094906 shutdown_connections 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.517+0000 7fdebf994b80 1 -- 172.21.15.33:0/350094906 wait complete. 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.518+0000 7fdebf994b80 1 -- start start 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.518+0000 7fdebf994b80 1 Processor -- start 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b42077d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420776710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4206a1060 con 0x55b42077a210 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42064c890 con 0x55b420751dd0 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b420667ea0 con 0x55b420750950 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdebf994b80 10 client.0 _ll_register_callbacks cb 0x55b42053df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdeab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdeab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58234/0 (socket says 172.21.15.33:58234) 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.519+0000 7fdeab59e700 1 -- 172.21.15.33:0/3631215864 learned_addr learned my addr 172.21.15.33:0/3631215864 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b42077d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420776710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeab59e700 1 -- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 msgr2=0x55b42077d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b42077d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeab59e700 1 -- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 msgr2=0x55b420776710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420776710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeab59e700 1 -- 172.21.15.33:0/3631215864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c00f040 con 0x55b42077a210 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078f3b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fdea4003000 tx=0x7fdea4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136470e200cc9cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fde897fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:27:16.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.520+0000 7fde897fa700 1 -- 172.21.15.33:0/3631215864 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4009960 con 0x55b42077a210 2023-10-06T05:27:16.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b420745690 con 0x55b42077a210 2023-10-06T05:27:16.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fde897fa700 1 -- 172.21.15.33:0/3631215864 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdea4012d00 con 0x55b42077a210 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fde897fa700 1 -- 172.21.15.33:0/3631215864 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea4028680 con 0x55b42077a210 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 10 client.9549 fetch_fsmap learned FSMap version 132 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) v3 -- 0x55b4204ddfe0 con 0x55b42077a210 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fde897fa700 1 -- 172.21.15.33:0/3631215864 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b420745690 con 0x55b42077a210 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fde897fa700 1 -- 172.21.15.33:0/3631215864 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 132) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdea40287e0 con 0x55b42077a210 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 10 client.9549 fetch_fsmap finished waiting for FSMap version 132 2023-10-06T05:27:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 10 client.9549 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 20 client.9549 populate_metadata read hostname 'smithi033' 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207785b0 0x55b42077fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 4 client.9549 mds_command: new command op to 9405 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 --> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4207225c0 con 0x55b4207785b0 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.521+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207785b0 0x55b42077fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.522+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207785b0 0x55b42077fe50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f894939be28ef4d server_cookie=a2afe6a109dfc19b in_seq=0 out_seq=0 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.522+0000 7fde897fa700 10 client.9549 ms_handle_connect on v2:172.21.15.33:6833/950449971 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fde897fa700 1 -- 172.21.15.33:0/3631215864 <== mds.0 v2:172.21.15.33:6833/950449971 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b4207225c0 con 0x55b4207785b0 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fde897fa700 10 client.9549 handle_command_reply: tid=0 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 client.9549 shutdown 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 msgr2=0x55b42078f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078f3b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fdea4003000 tx=0x7fdea4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 shutdown_connections 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207785b0 0x55b42077fe50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078f3b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b42077d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fde897fa700 0 client.9549 ms_handle_reset on v2:172.21.15.33:6833/950449971 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3631215864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420776710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 >> 172.21.15.33:0/3631215864 conn(0x55b420752270 msgr2=0x55b420750d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 shutdown_connections 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 1 -- 172.21.15.33:0/3631215864 wait complete. 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.523+0000 7fdebf994b80 20 client.9549 trim_cache size 0 max 16384 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.541+0000 7fdebf994b80 1 Processor -- start 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.541+0000 7fdebf994b80 1 -- start start 2023-10-06T05:27:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b4207475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420747ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdebf994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b420523e30 con 0x55b42077a210 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4207766f0 con 0x55b420751dd0 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b42077b420 con 0x55b420750950 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58236/0 (socket says 172.21.15.33:58236) 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 -- 172.21.15.33:0/3362481395 learned_addr learned my addr 172.21.15.33:0/3362481395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 -- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 msgr2=0x55b4207475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b4207475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 -- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 msgr2=0x55b420747ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420747ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420747ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 -- 172.21.15.33:0/3362481395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b42066f780 con 0x55b42077a210 2023-10-06T05:27:16.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420747ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.542+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078ed60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009ea0 tx=0x7fdea4021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6466a6458cc34fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fde90ff9700 1 -- 172.21.15.33:0/3362481395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4012970 con 0x55b42077a210 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fde90ff9700 1 -- 172.21.15.33:0/3362481395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea40092e0 con 0x55b42077a210 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fde90ff9700 1 -- 172.21.15.33:0/3362481395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400be80 con 0x55b42077a210 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 -- 172.21.15.33:0/3362481395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 msgr2=0x55b42078ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078ed60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009ea0 tx=0x7fdea4021070 comp rx=0 tx=0).stop 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 -- 172.21.15.33:0/3362481395 shutdown_connections 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b42077a210 0x55b42078ed60 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420750950 0x55b4207475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3362481395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b420751dd0 0x55b420747ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 -- 172.21.15.33:0/3362481395 >> 172.21.15.33:0/3362481395 conn(0x55b420752270 msgr2=0x55b42077bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 -- 172.21.15.33:0/3362481395 shutdown_connections 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.543+0000 7fdebf994b80 1 -- 172.21.15.33:0/3362481395 wait complete. 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.544+0000 7fdebf994b80 1 -- start start 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.544+0000 7fdebf994b80 1 Processor -- start 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b4207911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420751dd0 0x55b42078ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b42077a210 0x55b42078f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b420801190 con 0x55b420750950 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b420779a00 con 0x55b42077a210 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b420779f30 con 0x55b420751dd0 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdebf994b80 10 client.0 _ll_register_callbacks cb 0x55b42053df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdeaad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b4207911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdeaad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b4207911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58246/0 (socket says 172.21.15.33:58246) 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.545+0000 7fdeaad9d700 1 -- 172.21.15.33:0/3338354678 learned_addr learned my addr 172.21.15.33:0/3338354678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeab59e700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b42077a210 0x55b42078f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420751dd0 0x55b42078ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaad9d700 1 -- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420751dd0 msgr2=0x55b42078ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420751dd0 0x55b42078ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaad9d700 1 -- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b42077a210 msgr2=0x55b42078f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b42077a210 0x55b42078f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaad9d700 1 -- 172.21.15.33:0/3338354678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4204ddfe0 con 0x55b420750950 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420751dd0 0x55b42078ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdeaad9d700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b4207911a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fde9c020000 tx=0x7fde9c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7301cf8cabbca253 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fde897fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fde897fa700 1 -- 172.21.15.33:0/3338354678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c000cb0 con 0x55b420750950 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fde897fa700 1 -- 172.21.15.33:0/3338354678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde9c015cd0 con 0x55b420750950 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde9c015950 con 0x55b420750950 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fde897fa700 1 -- 172.21.15.33:0/3338354678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde9c015e90 con 0x55b420750950 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.546+0000 7fdebf994b80 10 client.9555 fetch_fsmap learned FSMap version 132 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) v3 -- 0x55b420656470 con 0x55b420750950 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fde897fa700 1 -- 172.21.15.33:0/3338354678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c03a070 con 0x55b420750950 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fde897fa700 1 -- 172.21.15.33:0/3338354678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 132) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fde9c006a30 con 0x55b420750950 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 10 client.9555 fetch_fsmap finished waiting for FSMap version 132 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 10 client.9555 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 20 client.9555 populate_metadata read hostname 'smithi033' 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207940d0 0x55b4207944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 4 client.9555 mds_command: new command op to 9405 tid=0[{"prefix": "status"}] 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 --> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b420643e00 con 0x55b4207940d0 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.547+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207940d0 0x55b4207944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdeaa59c700 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207940d0 0x55b4207944a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d5d17b68012388 server_cookie=eb74b5de48d0efd3 in_seq=0 out_seq=0 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fde897fa700 10 client.9555 ms_handle_connect on v2:172.21.15.33:6833/950449971 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fde897fa700 1 -- 172.21.15.33:0/3338354678 <== mds.0 v2:172.21.15.33:6833/950449971 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55b420643e00 con 0x55b4207940d0 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fde897fa700 10 client.9555 handle_command_reply: tid=0 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 client.9555 shutdown 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 msgr2=0x55b4207911a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b4207911a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fde9c020000 tx=0x7fde9c01e000 comp rx=0 tx=0).stop 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 shutdown_connections 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b42077a210 0x55b42078f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:6833/950449971,v1:172.21.15.33:6835/950449971] conn(0x55b4207940d0 0x55b4207944a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b420750950 0x55b4207911a0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 --2- 172.21.15.33:0/3338354678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55b420751dd0 0x55b42078ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 >> 172.21.15.33:0/3338354678 conn(0x55b420752270 msgr2=0x55b420796600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fde897fa700 0 client.9555 ms_handle_reset on v2:172.21.15.33:6833/950449971 2023-10-06T05:27:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 shutdown_connections 2023-10-06T05:27:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.548+0000 7fdebf994b80 1 -- 172.21.15.33:0/3338354678 wait complete. 2023-10-06T05:27:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdebf994b80 20 client.9555 trim_cache size 0 max 16384 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 9405, 2023-10-06T05:27:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 7.5836026590000003, 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 132, 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 181, 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 181, 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 13.322142962999999 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde98066f30 msgr2=0x7fde980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde98066f30 0x7fde980693e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fde9c007fd0 tx=0x7fde9c009040 comp rx=0 tx=0).stop 2023-10-06T05:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 msgr2=0x7fdeac164ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 0x7fdeac164ee0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fdea000d860 tx=0x7fdea0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 shutdown_connections 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fde98066f30 0x7fde980693e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeac155bc0 0x7fdeac157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdeac0c7070 0x7fdeac0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 --2- 172.21.15.33:0/1922290137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeac1557f0 0x7fdeac164ee0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 >> 172.21.15.33:0/1922290137 conn(0x7fdeac06c560 msgr2=0x7fdeac158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 shutdown_connections 2023-10-06T05:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:15.549+0000 7fdeb09fe700 1 -- 172.21.15.33:0/1922290137 wait complete. 2023-10-06T05:27:16.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:27:16.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.586+0000 7f481d79d700 1 Processor -- start 2023-10-06T05:27:16.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.586+0000 7f481d79d700 1 -- start start 2023-10-06T05:27:16.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f481d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48180c8050 0x7f481813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f481d79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180c9cb0 con 0x7f481806aad0 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f481d79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180ca270 con 0x7f481813faa0 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f481d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481806b640 con 0x7f48180c8050 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f48177fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48180c8050 0x7f481813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f48177fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48180c8050 0x7f481813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58260/0 (socket says 172.21.15.33:58260) 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f48177fe700 1 -- 172.21.15.33:0/1739060802 learned_addr learned my addr 172.21.15.33:0/1739060802 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f481cf9c700 1 --1- 172.21.15.33:0/1739060802 >> v1:172.21.15.33:6789/0 conn(0x7f481806aad0 0x7f481806af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37440/0 (socket says 172.21.15.33:37440) 2023-10-06T05:27:16.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f48177fe700 1 -- 172.21.15.33:0/1739060802 >> v1:172.21.15.167:6789/0 conn(0x7f481813faa0 legacy=0x7f48180c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f48177fe700 1 -- 172.21.15.33:0/1739060802 >> v1:172.21.15.33:6789/0 conn(0x7f481806aad0 legacy=0x7f481806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.588+0000 7f48177fe700 1 -- 172.21.15.33:0/1739060802 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481806b840 con 0x7f48180c8050 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.589+0000 7f48177fe700 1 --2- 172.21.15.33:0/1739060802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f481813fe70 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f480c0099b0 tx=0x7f480c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9fd4b9f01000f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.589+0000 7f4816ffd700 1 -- 172.21.15.33:0/1739060802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00b820 con 0x7f48180c8050 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.589+0000 7f4816ffd700 1 -- 172.21.15.33:0/1739060802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c00b980 con 0x7f48180c8050 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.589+0000 7f4816ffd700 1 -- 172.21.15.33:0/1739060802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00bd20 con 0x7f48180c8050 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.589+0000 7f481d79d700 1 -- 172.21.15.33:0/1739060802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 msgr2=0x7f481813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.589+0000 7f481d79d700 1 --2- 172.21.15.33:0/1739060802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f481813fe70 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f480c0099b0 tx=0x7f480c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.590+0000 7f481d79d700 1 -- 172.21.15.33:0/1739060802 shutdown_connections 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.590+0000 7f481d79d700 1 --2- 172.21.15.33:0/1739060802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f481813fe70 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.590+0000 7f481d79d700 1 -- 172.21.15.33:0/1739060802 >> 172.21.15.33:0/1739060802 conn(0x7f481806d030 msgr2=0x7f481806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.590+0000 7f481d79d700 1 -- 172.21.15.33:0/1739060802 shutdown_connections 2023-10-06T05:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.590+0000 7f481d79d700 1 -- 172.21.15.33:0/1739060802 wait complete. 2023-10-06T05:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.591+0000 7f481d79d700 1 Processor -- start 2023-10-06T05:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.591+0000 7f481d79d700 1 -- start start 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f481d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481806aad0 0x7f481816b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f481d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f48181686b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f481d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f481813faa0 0x7f48181679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f481d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180a7db0 con 0x7f48180c8050 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f481d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180a5480 con 0x7f481806aad0 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f481d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481816beb0 con 0x7f481813faa0 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f48177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f48181686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f48177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f48181686b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58272/0 (socket says 172.21.15.33:58272) 2023-10-06T05:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.592+0000 7f48177fe700 1 -- 172.21.15.33:0/2840531661 learned_addr learned my addr 172.21.15.33:0/2840531661 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f481cf9c700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f481813faa0 0x7f48181679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f48177fe700 1 -- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f481813faa0 msgr2=0x7f48181679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f4817fff700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481806aad0 0x7f481816b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f48177fe700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f481813faa0 0x7f48181679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f48177fe700 1 -- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481806aad0 msgr2=0x7f481816b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f48177fe700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481806aad0 0x7f481816b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f48177fe700 1 -- 172.21.15.33:0/2840531661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0094f0 con 0x7f48180c8050 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f481cf9c700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f481813faa0 0x7f48181679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f4817fff700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481806aad0 0x7f481816b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f48177fe700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f48181686b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c010 tx=0x7f480c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011e10 con 0x7f48180c8050 2023-10-06T05:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c020070 con 0x7f48180c8050 2023-10-06T05:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818168bf0 con 0x7f48180c8050 2023-10-06T05:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.593+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c01b6a0 con 0x7f48180c8050 2023-10-06T05:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.594+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48180a77f0 con 0x7f48180c8050 2023-10-06T05:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.595+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48180a55f0 con 0x7f48180c8050 2023-10-06T05:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.595+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f480c022030 con 0x7f48180c8050 2023-10-06T05:27:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.596+0000 7f47fffff700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.596+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f480c086690 con 0x7f48180c8050 2023-10-06T05:27:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.597+0000 7f4817fff700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.598+0000 7f4817fff700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4804066f30 0x7f48040693e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4808006fd0 tx=0x7f4808008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.601+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48180a55f0 con 0x7f48180c8050 2023-10-06T05:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.849+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f48180a55f0 con 0x7f48180c8050 2023-10-06T05:27:16.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.850+0000 7f47fffff700 1 -- 172.21.15.33:0/2840531661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) v1 ==== 53+0+73 (secure 0 0 0) 0x7f48180a55f0 con 0x7f48180c8050 2023-10-06T05:27:16.876 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4804066f30 msgr2=0x7f48040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4804066f30 0x7f48040693e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4808006fd0 tx=0x7f4808008040 comp rx=0 tx=0).stop 2023-10-06T05:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 msgr2=0x7f48181686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f48181686b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c010 tx=0x7f480c016000 comp rx=0 tx=0).stop 2023-10-06T05:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 shutdown_connections 2023-10-06T05:27:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f481813faa0 0x7f48181679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f481806aad0 0x7f481816b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 --2- 172.21.15.33:0/2840531661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48180c8050 0x7f48181686b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 >> 172.21.15.33:0/2840531661 conn(0x7f481806d030 msgr2=0x7f48180c1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 shutdown_connections 2023-10-06T05:27:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:16.852+0000 7f481d79d700 1 -- 172.21.15.33:0/2840531661 wait complete. 2023-10-06T05:27:16.878 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:27:16.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:27:16.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:27:16.879 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:27:16.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:16.879 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:27:16.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:27:16.879 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:27:16.879 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:16.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:16.879 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:27:16.917 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:27:16.917 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:27:16.917 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:27:16.917 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.917 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:27:16.917 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:27:16.918 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41520sec preferred_lft 41520sec 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:27:16.919 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:27:16.920 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:27:16.921 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:16.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:16.921 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:27:17.017 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:27:17.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:17.020 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:27:17.061 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:27:17.062 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:27:17.062 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:27:17.062 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:27:17.062 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:27:17.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:17.062 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:27:17.062 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:27:17.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:27:17.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:27:17.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:27:17.063 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:27:17.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:17.300 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:27:17.337 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:27:17.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:27:17.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:27:17.609 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:17.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:17.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:27:17.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:17.757 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:17.794 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:17.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:17.795 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:17.872 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:27:17.872 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:27:17.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:18.005 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:18.005 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:27:18.005 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:27:18.022 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:18.022 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:27:18.022 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:27:18.022 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:27:18.022 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:27:18.044 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:27:18.045 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:27:18.051 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:27:18.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:18.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:18.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:27:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.300+0000 7f8b9cb9d700 1 Processor -- start 2023-10-06T05:27:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.300+0000 7f8b9cb9d700 1 -- start start 2023-10-06T05:27:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.301+0000 7f8b9cb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8b980ca590 0x7f8b980c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.301+0000 7f8b9cb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b980c92d0 con 0x7f8b98159a20 2023-10-06T05:27:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.301+0000 7f8b9cb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b980c86c0 con 0x7f8b98159650 2023-10-06T05:27:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.301+0000 7f8b9cb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b980c89a0 con 0x7f8b980ca590 2023-10-06T05:27:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b97fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b98159a20 0x7f8b980a9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37452/0 (socket says 172.21.15.33:37452) 2023-10-06T05:27:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b97fff700 1 -- 172.21.15.33:0/3027347457 learned_addr learned my addr 172.21.15.33:0/3027347457 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b96ffd700 1 --2- 172.21.15.33:0/3027347457 >> v2:172.21.15.167:3300/0 conn(0x7f8b980ca590 0x7f8b980c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113438338 0 0) 0x7f8b980c92d0 con 0x7f8b98159a20 2023-10-06T05:27:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b80003570 con 0x7f8b98159a20 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031844847 0 0) 0x7f8b980c86c0 con 0x7f8b98159650 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b980c92d0 con 0x7f8b98159650 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b96ffd700 1 -- 172.21.15.33:0/3027347457 >> v1:172.21.15.167:6789/0 conn(0x7f8b98159650 legacy=0x7f8b980c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b96ffd700 1 -- 172.21.15.33:0/3027347457 >> v1:172.21.15.33:6789/0 conn(0x7f8b98159a20 legacy=0x7f8b980a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.302+0000 7f8b96ffd700 1 -- 172.21.15.33:0/3027347457 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b980c8c80 con 0x7f8b980ca590 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.303+0000 7f8b96ffd700 1 --2- 172.21.15.33:0/3027347457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980ca590 0x7f8b980c8030 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098e0 tx=0x7f8b8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=528635790829bb39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.303+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c011630 con 0x7f8b980ca590 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.303+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c011790 con 0x7f8b980ca590 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.303+0000 7f8b967fc700 1 -- 172.21.15.33:0/3027347457 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c011b30 con 0x7f8b980ca590 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.303+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/3027347457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980ca590 msgr2=0x7f8b980c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.303+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/3027347457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980ca590 0x7f8b980c8030 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098e0 tx=0x7f8b8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.304+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/3027347457 shutdown_connections 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.304+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/3027347457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980ca590 0x7f8b980c8030 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.304+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/3027347457 >> 172.21.15.33:0/3027347457 conn(0x7f8b9806d030 msgr2=0x7f8b9806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.304+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/3027347457 shutdown_connections 2023-10-06T05:27:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.304+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/3027347457 wait complete. 2023-10-06T05:27:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.305+0000 7f8b9cb9d700 1 Processor -- start 2023-10-06T05:27:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.305+0000 7f8b9cb9d700 1 -- start start 2023-10-06T05:27:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b9cb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 0x7f8b98151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b9cb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b98159650 0x7f8b980ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b9cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b98159a20 0x7f8b980ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b9cb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980c3e80 con 0x7f8b980ca590 2023-10-06T05:27:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b9cb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980c4160 con 0x7f8b98159650 2023-10-06T05:27:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b9cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980c4440 con 0x7f8b98159a20 2023-10-06T05:27:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.306+0000 7f8b977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 0x7f8b98151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 0x7f8b98151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58282/0 (socket says 172.21.15.33:58282) 2023-10-06T05:27:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 -- 172.21.15.33:0/2906752788 learned_addr learned my addr 172.21.15.33:0/2906752788 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b96ffd700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b98159650 0x7f8b980ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b97fff700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b98159a20 0x7f8b980ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 -- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b98159a20 msgr2=0x7f8b980ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b98159a20 0x7f8b980ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 -- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b98159650 msgr2=0x7f8b980ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b98159650 0x7f8b980ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 -- 172.21.15.33:0/2906752788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009420 con 0x7f8b980ca590 2023-10-06T05:27:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b96ffd700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b98159650 0x7f8b980ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:18.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b977fe700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 0x7f8b98151ee0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800c790 tx=0x7f8b88007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.307+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88010510 con 0x7f8b980ca590 2023-10-06T05:27:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.308+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b88010670 con 0x7f8b980ca590 2023-10-06T05:27:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.308+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88010a10 con 0x7f8b980ca590 2023-10-06T05:27:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.308+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b980c4720 con 0x7f8b980ca590 2023-10-06T05:27:18.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.308+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b980c3b70 con 0x7f8b980ca590 2023-10-06T05:27:18.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.309+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9804c380 con 0x7f8b980ca590 2023-10-06T05:27:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.310+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b8801b400 con 0x7f8b980ca590 2023-10-06T05:27:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.311+0000 7f8b7f7fe700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b84066f30 0x7f8b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.311+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b88084500 con 0x7f8b980ca590 2023-10-06T05:27:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.311+0000 7f8b96ffd700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b84066f30 0x7f8b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.312+0000 7f8b96ffd700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b84066f30 0x7f8b840693e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8b980ca990 tx=0x7f8b8c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.315+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b88051570 con 0x7f8b980ca590 2023-10-06T05:27:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.563+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b980c8f60 con 0x7f8b980ca590 2023-10-06T05:27:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.563+0000 7f8b7f7fe700 1 -- 172.21.15.33:0/2906752788 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b88054b90 con 0x7f8b980ca590 2023-10-06T05:27:18.566 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:27:18.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b84066f30 msgr2=0x7f8b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b84066f30 0x7f8b840693e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8b980ca990 tx=0x7f8b8c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 msgr2=0x7f8b98151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 0x7f8b98151ee0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800c790 tx=0x7f8b88007e10 comp rx=0 tx=0).stop 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 shutdown_connections 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8b84066f30 0x7f8b840693e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b98159a20 0x7f8b980ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b980ca590 0x7f8b98151ee0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 --2- 172.21.15.33:0/2906752788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b98159650 0x7f8b980ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 >> 172.21.15.33:0/2906752788 conn(0x7f8b9806d030 msgr2=0x7f8b98153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 shutdown_connections 2023-10-06T05:27:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:18.566+0000 7f8b9cb9d700 1 -- 172.21.15.33:0/2906752788 wait complete. 2023-10-06T05:27:18.586 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:27:18.586 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:27:18.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:27:18.587 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:27:18.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:27:18.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:27:18.588 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:27:18.588 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:27:18.588 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:27:18.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:18.588 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:27:18.629 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:18.630 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41522sec preferred_lft 41522sec 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:18.631 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:27:18.632 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:18.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:18.633 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:27:18.731 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:27:18.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:18.734 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:27:18.772 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:27:18.772 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:27:18.772 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:27:18.772 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:27:18.773 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:27:18.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:18.774 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:27:18.774 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:27:18.774 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:27:18.774 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:27:18.774 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:27:18.774 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:27:18.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:18.972 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:27:19.009 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:27:19.009 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:27:19.009 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:27:19.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:19.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:27:19.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:19.111 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:27:19.147 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:27:19.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:19.147 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:27:19.223 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:27:19.223 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:27:19.223 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:27:19.353 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:27:19.353 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:27:19.353 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:27:19.369 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:27:19.369 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:27:19.370 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:27:19.370 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:27:19.370 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:27:19.390 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:27:19.390 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:27:19.390 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:27:19.390 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:27:19.391 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:27:19.391 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:27:19.391 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:27:19.391 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:27:19.394 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:27:19.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:19.395 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:27:19.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:27:19.672 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:27:19.672 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:27:19.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:27:19.879 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:27:19.879 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:27:19.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:19.880 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:27:19.952 INFO:teuthology.orchestra.run.smithi033.stdout:client8761 2023-10-06T05:27:19.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:27:19.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:20.003 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.142+0000 7fdb5b59e700 1 Processor -- start 2023-10-06T05:27:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.142+0000 7fdb5b59e700 1 -- start start 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb5c1557f0 0x7fdb5c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c0c9220 0x7fdb5c0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c06a410 con 0x7fdb5c155bc0 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c0cccd0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0c7500 con 0x7fdb5c0c9220 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb59d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb5c155bc0 0x7fdb5c0c8b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55176/0 (socket says 172.21.15.33:55176) 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb59d9b700 1 -- 172.21.15.33:0/780685546 learned_addr learned my addr 172.21.15.33:0/780685546 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5ad9d700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c0c9220 0x7fdb5c0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5a59c700 1 --2- 172.21.15.33:0/780685546 >> v2:172.21.15.167:3300/0 conn(0x7fdb5c1557f0 0x7fdb5c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5959a700 1 -- 172.21.15.33:0/780685546 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893018667 0 0) 0x7fdb5c06a410 con 0x7fdb5c155bc0 2023-10-06T05:27:20.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.144+0000 7fdb5959a700 1 -- 172.21.15.33:0/780685546 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb44003570 con 0x7fdb5c155bc0 2023-10-06T05:27:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5959a700 1 -- 172.21.15.33:0/780685546 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407061245 0 0) 0x7fdb44003570 con 0x7fdb5c155bc0 2023-10-06T05:27:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5a59c700 1 -- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c0c9220 msgr2=0x7fdb5c0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5a59c700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c0c9220 0x7fdb5c0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5a59c700 1 -- 172.21.15.33:0/780685546 >> v1:172.21.15.33:6789/0 conn(0x7fdb5c155bc0 legacy=0x7fdb5c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5a59c700 1 -- 172.21.15.33:0/780685546 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c0c77e0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5ad9d700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c0c9220 0x7fdb5c0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5a59c700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c0c6960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098e0 tx=0x7fdb4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f3cac2c53d2b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5959a700 1 -- 172.21.15.33:0/780685546 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50001470 con 0x7fdb5c1557f0 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5959a700 1 -- 172.21.15.33:0/780685546 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c011420 con 0x7fdb5c1557f0 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.145+0000 7fdb5959a700 1 -- 172.21.15.33:0/780685546 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0117c0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 -- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c1557f0 msgr2=0x7fdb5c0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c0c6960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098e0 tx=0x7fdb4c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 -- 172.21.15.33:0/780685546 shutdown_connections 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c0c6960 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/780685546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c0c9220 0x7fdb5c0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 -- 172.21.15.33:0/780685546 >> 172.21.15.33:0/780685546 conn(0x7fdb5c06c560 msgr2=0x7fdb5c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 -- 172.21.15.33:0/780685546 shutdown_connections 2023-10-06T05:27:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.146+0000 7fdb5b59e700 1 -- 172.21.15.33:0/780685546 wait complete. 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.147+0000 7fdb5b59e700 1 Processor -- start 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 -- start start 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c0c9220 0x7fdb5c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c155bc0 0x7fdb5c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0b1df0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c152390 con 0x7fdb5c0c9220 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.148+0000 7fdb5b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c152670 con 0x7fdb5c155bc0 2023-10-06T05:27:20.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44996/0 (socket says 172.21.15.33:44996) 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 -- 172.21.15.33:0/930778540 learned_addr learned my addr 172.21.15.33:0/930778540 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb5ad9d700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c155bc0 0x7fdb5c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb5a59c700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c0c9220 0x7fdb5c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 -- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c155bc0 msgr2=0x7fdb5c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c155bc0 0x7fdb5c151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 -- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c0c9220 msgr2=0x7fdb5c14df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c0c9220 0x7fdb5c14df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 -- 172.21.15.33:0/930778540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5400ab30 con 0x7fdb5c1557f0 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.149+0000 7fdb59d9b700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c151820 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000cb90 tx=0x7fdb50001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.150+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50007150 con 0x7fdb5c1557f0 2023-10-06T05:27:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.150+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb500072b0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.150+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c009420 con 0x7fdb5c1557f0 2023-10-06T05:27:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.150+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50007650 con 0x7fdb5c1557f0 2023-10-06T05:27:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.150+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c152c30 con 0x7fdb5c1557f0 2023-10-06T05:27:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.151+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c04c380 con 0x7fdb5c1557f0 2023-10-06T05:27:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.152+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb50004a40 con 0x7fdb5c1557f0 2023-10-06T05:27:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.153+0000 7fdb4a7fc700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb3c066f30 0x7fdb3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.153+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb50083ec0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.153+0000 7fdb5a59c700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb3c066f30 0x7fdb3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.154+0000 7fdb5a59c700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb3c066f30 0x7fdb3c0693e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0030c0 tx=0x7fdb4c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:20.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.157+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb50050fb0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.405+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb5c0c1ee0 con 0x7fdb5c1557f0 2023-10-06T05:27:20.408 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:27:20.406+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:27:20.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.413+0000 7fdb4a7fc700 1 -- 172.21.15.33:0/930778540 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v133) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdb50018020 con 0x7fdb5c1557f0 2023-10-06T05:27:20.417 INFO:tasks.ceph.mds.c.smithi033.stderr: -728> 2023-10-06T05:27:08.949+0000 7f2ae0df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb3c066f30 msgr2=0x7fdb3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb3c066f30 0x7fdb3c0693e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0030c0 tx=0x7fdb4c016000 comp rx=0 tx=0).stop 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 msgr2=0x7fdb5c151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c151820 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000cb90 tx=0x7fdb50001a80 comp rx=0 tx=0).stop 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 shutdown_connections 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdb3c066f30 0x7fdb3c0693e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb5c155bc0 0x7fdb5c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb5c0c9220 0x7fdb5c14df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 --2- 172.21.15.33:0/930778540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb5c1557f0 0x7fdb5c151820 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 >> 172.21.15.33:0/930778540 conn(0x7fdb5c06c560 msgr2=0x7fdb5c0c2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 shutdown_connections 2023-10-06T05:27:20.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:20.416+0000 7fdb5b59e700 1 -- 172.21.15.33:0/930778540 wait complete. 2023-10-06T05:27:20.420 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:27:20.438 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T05:27:20.438 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:20.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:20.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:27:20.455 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:27:25.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:27:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.291+0000 7f0736f9d700 1 Processor -- start 2023-10-06T05:27:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.291+0000 7f0736f9d700 1 -- start start 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.292+0000 7f0736f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f07300cc7e0 0x7f07300af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.292+0000 7f0736f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07300b01f0 0x7f07300b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.292+0000 7f0736f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07300ad430 con 0x7f0730156370 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.292+0000 7f0736f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07300afe20 con 0x7f07300b01f0 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.292+0000 7f0736f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07300b0b70 con 0x7f07300cc7e0 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07300b01f0 0x7f07300b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07300b01f0 0x7f07300b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45016/0 (socket says 172.21.15.33:45016) 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 -- 172.21.15.33:0/1641979610 learned_addr learned my addr 172.21.15.33:0/1641979610 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073579a700 1 --2- 172.21.15.33:0/1641979610 >> v2:172.21.15.167:3300/0 conn(0x7f07300cc7e0 0x7f07300af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 -- 172.21.15.33:0/1641979610 >> v2:172.21.15.167:3300/0 conn(0x7f07300cc7e0 msgr2=0x7f07300af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 --2- 172.21.15.33:0/1641979610 >> v2:172.21.15.167:3300/0 conn(0x7f07300cc7e0 0x7f07300af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 -- 172.21.15.33:0/1641979610 >> v1:172.21.15.167:6789/0 conn(0x7f0730156370 legacy=0x7f07300c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.293+0000 7f073679c700 1 -- 172.21.15.33:0/1641979610 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073006bc70 con 0x7f07300b01f0 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f073679c700 1 --2- 172.21.15.33:0/1641979610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07300b01f0 0x7f07300b0630 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f07300c94c0 tx=0x7f0728009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3174d56194f748b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0734f99700 1 -- 172.21.15.33:0/1641979610 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800e070 con 0x7f07300b01f0 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0734f99700 1 -- 172.21.15.33:0/1641979610 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072800ba20 con 0x7f07300b01f0 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0734f99700 1 -- 172.21.15.33:0/1641979610 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280092e0 con 0x7f07300b01f0 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0736f9d700 1 -- 172.21.15.33:0/1641979610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07300b01f0 msgr2=0x7f07300b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0736f9d700 1 --2- 172.21.15.33:0/1641979610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07300b01f0 0x7f07300b0630 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f07300c94c0 tx=0x7f0728009840 comp rx=0 tx=0).stop 2023-10-06T05:27:26.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0736f9d700 1 -- 172.21.15.33:0/1641979610 shutdown_connections 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.294+0000 7f0736f9d700 1 --2- 172.21.15.33:0/1641979610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07300b01f0 0x7f07300b0630 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.295+0000 7f0736f9d700 1 --2- 172.21.15.33:0/1641979610 >> v2:172.21.15.167:3300/0 conn(0x7f07300cc7e0 0x7f07300af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.295+0000 7f0736f9d700 1 -- 172.21.15.33:0/1641979610 >> 172.21.15.33:0/1641979610 conn(0x7f073006d030 msgr2=0x7f07300cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.295+0000 7f0736f9d700 1 -- 172.21.15.33:0/1641979610 shutdown_connections 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.295+0000 7f0736f9d700 1 -- 172.21.15.33:0/1641979610 wait complete. 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.296+0000 7f0736f9d700 1 Processor -- start 2023-10-06T05:27:26.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.296+0000 7f0736f9d700 1 -- start start 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f0736f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07300b01f0 0x7f073006aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f0736f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300cc7e0 0x7f07300a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f0736f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 0x7f073016d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f0736f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073016cfb0 con 0x7f0730156370 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f0736f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073016cc60 con 0x7f07300cc7e0 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f0736f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0730158090 con 0x7f07300b01f0 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f073679c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 0x7f073016d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f073679c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 0x7f073016d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45022/0 (socket says 172.21.15.33:45022) 2023-10-06T05:27:26.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.297+0000 7f073679c700 1 -- 172.21.15.33:0/532725634 learned_addr learned my addr 172.21.15.33:0/532725634 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:26.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073679c700 1 -- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07300b01f0 msgr2=0x7f073006aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073579a700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300cc7e0 0x7f07300a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f0735f9b700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07300b01f0 0x7f073006aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073679c700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07300b01f0 0x7f073006aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073679c700 1 -- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300cc7e0 msgr2=0x7f07300a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073679c700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300cc7e0 0x7f07300a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073679c700 1 -- 172.21.15.33:0/532725634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728009ae0 con 0x7f0730156370 2023-10-06T05:27:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073579a700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300cc7e0 0x7f07300a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f0735f9b700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07300b01f0 0x7f073006aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:26.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f073679c700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 0x7f073016d440 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f0728009880 tx=0x7f0728018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728008be0 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728008d40 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0730158370 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.298+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280256b0 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.299+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073006b030 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.299+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073016d160 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.300+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0728003040 con 0x7f0730156370 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.301+0000 7f0725ffb700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0714066f30 0x7f07140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.301+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f0728085d70 con 0x7f0730156370 2023-10-06T05:27:26.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.302+0000 7f0735f9b700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0714066f30 0x7f07140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.302+0000 7f0735f9b700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0714066f30 0x7f07140693e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f072c001840 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.306+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f073016d160 con 0x7f0730156370 2023-10-06T05:27:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.553+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f073004c380 con 0x7f0730156370 2023-10-06T05:27:26.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.855+0000 7f0725ffb700 1 -- 172.21.15.33:0/532725634 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v135) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0728052e60 con 0x7f0730156370 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0714066f30 msgr2=0x7f07140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0714066f30 0x7f07140693e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f072c001840 tx=0x7f072c009040 comp rx=0 tx=0).stop 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 msgr2=0x7f073016d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 0x7f073016d440 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f0728009880 tx=0x7f0728018000 comp rx=0 tx=0).stop 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 shutdown_connections 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0714066f30 0x7f07140693e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0730156370 0x7f073016d440 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07300b01f0 0x7f073006aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 --2- 172.21.15.33:0/532725634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300cc7e0 0x7f07300a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 >> 172.21.15.33:0/532725634 conn(0x7f073006d030 msgr2=0x7f07300abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 shutdown_connections 2023-10-06T05:27:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:26.858+0000 7f0736f9d700 1 -- 172.21.15.33:0/532725634 wait complete. 2023-10-06T05:27:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:27:26.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.072+0000 7f323e19d700 1 Processor -- start 2023-10-06T05:27:27.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.072+0000 7f323e19d700 1 -- start start 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323e19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3238155140 0x7f3238158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380699c0 0x7f3238069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323e19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380c8e40 con 0x7f32380adf50 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323e19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323806a380 con 0x7f3238155140 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380c99e0 con 0x7f32380699c0 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323d19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3238155140 0x7f3238158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323d19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3238155140 0x7f3238158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45030/0 (socket says 172.21.15.33:45030) 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323d19b700 1 -- 172.21.15.33:0/1636817932 learned_addr learned my addr 172.21.15.33:0/1636817932 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:27.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323d99c700 1 --2- 172.21.15.33:0/1636817932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380699c0 0x7f3238069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.074+0000 7f323d19b700 1 -- 172.21.15.33:0/1636817932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380699c0 msgr2=0x7f3238069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f323d19b700 1 --2- 172.21.15.33:0/1636817932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380699c0 0x7f3238069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f323d19b700 1 -- 172.21.15.33:0/1636817932 >> v1:172.21.15.167:6789/0 conn(0x7f32380adf50 legacy=0x7f32380c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f323d19b700 1 -- 172.21.15.33:0/1636817932 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380ae3f0 con 0x7f3238155140 2023-10-06T05:27:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f322ffff700 1 -- 172.21.15.33:0/1636817932 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219718885 0 0) 0x7f32380c8e40 con 0x7f32380adf50 2023-10-06T05:27:27.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f323d19b700 1 --2- 172.21.15.33:0/1636817932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3238155140 0x7f3238158700 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f32340099e0 tx=0x7f3234009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10a5287d53aeaa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f322ffff700 1 -- 172.21.15.33:0/1636817932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400b820 con 0x7f3238155140 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f322ffff700 1 -- 172.21.15.33:0/1636817932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323400b980 con 0x7f3238155140 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.075+0000 7f322ffff700 1 -- 172.21.15.33:0/1636817932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400bd20 con 0x7f3238155140 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 -- 172.21.15.33:0/1636817932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3238155140 msgr2=0x7f3238158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 --2- 172.21.15.33:0/1636817932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3238155140 0x7f3238158700 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f32340099e0 tx=0x7f3234009250 comp rx=0 tx=0).stop 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 -- 172.21.15.33:0/1636817932 shutdown_connections 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 --2- 172.21.15.33:0/1636817932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3238155140 0x7f3238158700 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 --2- 172.21.15.33:0/1636817932 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380699c0 0x7f3238069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 -- 172.21.15.33:0/1636817932 >> 172.21.15.33:0/1636817932 conn(0x7f323806d030 msgr2=0x7f323806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 -- 172.21.15.33:0/1636817932 shutdown_connections 2023-10-06T05:27:27.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.076+0000 7f323e19d700 1 -- 172.21.15.33:0/1636817932 wait complete. 2023-10-06T05:27:27.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.077+0000 7f323e19d700 1 Processor -- start 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.077+0000 7f323e19d700 1 -- start start 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 0x7f323814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380adf50 0x7f32380c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3238155140 0x7f32380b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380cb890 con 0x7f32380699c0 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380c4f20 con 0x7f3238155140 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323816dbb0 con 0x7f32380adf50 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.078+0000 7f323d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 0x7f323814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 0x7f323814fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45040/0 (socket says 172.21.15.33:45040) 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 -- 172.21.15.33:0/654968509 learned_addr learned my addr 172.21.15.33:0/654968509 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d99c700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3238155140 0x7f32380b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323c99a700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380adf50 0x7f32380c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 -- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380adf50 msgr2=0x7f32380c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380adf50 0x7f32380c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 -- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3238155140 msgr2=0x7f32380b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3238155140 0x7f32380b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 -- 172.21.15.33:0/654968509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32340094f0 con 0x7f32380699c0 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d99c700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3238155140 0x7f32380b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323c99a700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380adf50 0x7f32380c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323d19b700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 0x7f323814fd30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3234006fd0 tx=0x7f3234017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234011e20 con 0x7f32380699c0 2023-10-06T05:27:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234024440 con 0x7f32380699c0 2023-10-06T05:27:27.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.079+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32380c5130 con 0x7f32380699c0 2023-10-06T05:27:27.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.080+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340247e0 con 0x7f32380699c0 2023-10-06T05:27:27.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.080+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32380c56f0 con 0x7f32380699c0 2023-10-06T05:27:27.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.081+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32380c58e0 con 0x7f32380699c0 2023-10-06T05:27:27.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.081+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f323401a420 con 0x7f32380699c0 2023-10-06T05:27:27.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.083+0000 7f322d7fa700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f321c066f30 0x7f321c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.083+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3234086670 con 0x7f32380699c0 2023-10-06T05:27:27.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.083+0000 7f323c99a700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f321c066f30 0x7f321c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.084+0000 7f323c99a700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f321c066f30 0x7f321c0693e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f32280011e0 tx=0x7f3228009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.087+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32340536b0 con 0x7f32380699c0 2023-10-06T05:27:27.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.335+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f323804c380 con 0x7f32380699c0 2023-10-06T05:27:27.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.336+0000 7f322d7fa700 1 -- 172.21.15.33:0/654968509 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f3234056cd0 con 0x7f32380699c0 2023-10-06T05:27:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:27.339 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:26.857666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f321c066f30 msgr2=0x7f321c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f321c066f30 0x7f321c0693e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f32280011e0 tx=0x7f3228009040 comp rx=0 tx=0).stop 2023-10-06T05:27:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 msgr2=0x7f323814fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 0x7f323814fd30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3234006fd0 tx=0x7f3234017000 comp rx=0 tx=0).stop 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 shutdown_connections 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f321c066f30 0x7f321c0693e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3238155140 0x7f32380b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32380699c0 0x7f323814fd30 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 --2- 172.21.15.33:0/654968509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32380adf50 0x7f32380c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.338+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 >> 172.21.15.33:0/654968509 conn(0x7f323806d030 msgr2=0x7f32380c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.339+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 shutdown_connections 2023-10-06T05:27:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:27.339+0000 7f323e19d700 1 -- 172.21.15.33:0/654968509 wait complete. 2023-10-06T05:27:27.343 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 136 2023-10-06T05:27:27.359 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2023-10-06T05:27:28.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.557+0000 7fab2b59e700 1 Processor -- start 2023-10-06T05:27:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.557+0000 7fab2b59e700 1 -- start start 2023-10-06T05:27:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab2c06e140 0x7fab2c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c06f220 con 0x7fab2c159530 2023-10-06T05:27:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0b1e00 con 0x7fab2c06e140 2023-10-06T05:27:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c06f6b0 con 0x7fab2c0afdd0 2023-10-06T05:27:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab29d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab2c06e140 0x7fab2c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab2c159530 0x7fab2c06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55192/0 (socket says 172.21.15.33:55192) 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.558+0000 7fab2a59c700 1 -- 172.21.15.33:0/4051827124 learned_addr learned my addr 172.21.15.33:0/4051827124 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab2ad9d700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab2959a700 1 -- 172.21.15.33:0/4051827124 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854970635 0 0) 0x7fab2c06f220 con 0x7fab2c159530 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab2959a700 1 -- 172.21.15.33:0/4051827124 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab14003570 con 0x7fab2c159530 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab29d9b700 1 -- 172.21.15.33:0/4051827124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 msgr2=0x7fab2c0b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab29d9b700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab29d9b700 1 -- 172.21.15.33:0/4051827124 >> v1:172.21.15.33:6789/0 conn(0x7fab2c159530 legacy=0x7fab2c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab29d9b700 1 -- 172.21.15.33:0/4051827124 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c15ae50 con 0x7fab2c06e140 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab2ad9d700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab29d9b700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c15a910 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fab200098e0 tx=0x7fab200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56da77eae6b8111 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.559+0000 7fab2959a700 1 -- 172.21.15.33:0/4051827124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000b640 con 0x7fab2c06e140 2023-10-06T05:27:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2959a700 1 -- 172.21.15.33:0/4051827124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2000b7a0 con 0x7fab2c06e140 2023-10-06T05:27:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2959a700 1 -- 172.21.15.33:0/4051827124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000bb40 con 0x7fab2c06e140 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 -- 172.21.15.33:0/4051827124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 msgr2=0x7fab2c15a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c15a910 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fab200098e0 tx=0x7fab200091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 -- 172.21.15.33:0/4051827124 shutdown_connections 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 --2- 172.21.15.33:0/4051827124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c15a910 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 -- 172.21.15.33:0/4051827124 >> 172.21.15.33:0/4051827124 conn(0x7fab2c06c560 msgr2=0x7fab2c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 -- 172.21.15.33:0/4051827124 shutdown_connections 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.560+0000 7fab2b59e700 1 -- 172.21.15.33:0/4051827124 wait complete. 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.562+0000 7fab2b59e700 1 Processor -- start 2023-10-06T05:27:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.562+0000 7fab2b59e700 1 -- start start 2023-10-06T05:27:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2c159530 0x7fab2c155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0c85b0 con 0x7fab2c06e140 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0ce420 con 0x7fab2c159530 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0c8800 con 0x7fab2c0afdd0 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45058/0 (socket says 172.21.15.33:45058) 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2a59c700 1 -- 172.21.15.33:0/1719917545 learned_addr learned my addr 172.21.15.33:0/1719917545 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab29d9b700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.563+0000 7fab2ad9d700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2c159530 0x7fab2c155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2a59c700 1 -- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 msgr2=0x7fab2c0cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2a59c700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2a59c700 1 -- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2c159530 msgr2=0x7fab2c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2a59c700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2c159530 0x7fab2c155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2a59c700 1 -- 172.21.15.33:0/1719917545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab20009420 con 0x7fab2c06e140 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2ad9d700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2c159530 0x7fab2c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2a59c700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c143df0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c7c0 tx=0x7fab1c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c005b30 con 0x7fab2c06e140 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c005c90 con 0x7fab2c06e140 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c007340 con 0x7fab2c06e140 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c0c8ae0 con 0x7fab2c06e140 2023-10-06T05:27:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.564+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c144330 con 0x7fab2c06e140 2023-10-06T05:27:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.565+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c069a10 con 0x7fab2c06e140 2023-10-06T05:27:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.566+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab1c01b400 con 0x7fab2c06e140 2023-10-06T05:27:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.567+0000 7fab1a7fc700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab0c066f30 0x7fab0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.567+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fab1c084270 con 0x7fab2c06e140 2023-10-06T05:27:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.568+0000 7fab29d9b700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab0c066f30 0x7fab0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.568+0000 7fab29d9b700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab0c066f30 0x7fab0c0693e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fab200093f0 tx=0x7fab20017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.571+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab1c051330 con 0x7fab2c06e140 2023-10-06T05:27:28.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.816+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab2c04c380 con 0x7fab2c06e140 2023-10-06T05:27:28.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.817+0000 7fab1a7fc700 1 -- 172.21.15.33:0/1719917545 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fab1c054950 con 0x7fab2c06e140 2023-10-06T05:27:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:28.819 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":137,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:27.860889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab0c066f30 msgr2=0x7fab0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab0c066f30 0x7fab0c0693e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fab200093f0 tx=0x7fab20017000 comp rx=0 tx=0).stop 2023-10-06T05:27:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 msgr2=0x7fab2c143df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c143df0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c7c0 tx=0x7fab1c007e00 comp rx=0 tx=0).stop 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 shutdown_connections 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fab0c066f30 0x7fab0c0693e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab2c159530 0x7fab2c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c06e140 0x7fab2c143df0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.819+0000 7fab2b59e700 1 --2- 172.21.15.33:0/1719917545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab2c0afdd0 0x7fab2c0cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.820+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 >> 172.21.15.33:0/1719917545 conn(0x7fab2c06c560 msgr2=0x7fab2c1587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.820+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 shutdown_connections 2023-10-06T05:27:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:28.820+0000 7fab2b59e700 1 -- 172.21.15.33:0/1719917545 wait complete. 2023-10-06T05:27:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 137 2023-10-06T05:27:28.842 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to up:reconnect 2023-10-06T05:27:28.842 DEBUG:tasks.cephfs.filesystem:reached state 'up:reconnect' in 1.960040807723999s 2023-10-06T05:27:28.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:29.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.027+0000 7fb9aa39d700 1 Processor -- start 2023-10-06T05:27:29.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.027+0000 7fb9aa39d700 1 -- start start 2023-10-06T05:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9aa39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 0x7fb9a4158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9aa39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb9a40adf50 0x7fb9a40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9aa39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a40c8e40 con 0x7fb9a40c9810 2023-10-06T05:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9aa39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a40ae6c0 con 0x7fb9a40adf50 2023-10-06T05:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9aa39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a40ae9a0 con 0x7fb9a4155140 2023-10-06T05:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9a9b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb9a40c9810 0x7fb9a40699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55202/0 (socket says 172.21.15.33:55202) 2023-10-06T05:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9a9b9c700 1 -- 172.21.15.33:0/1062652446 learned_addr learned my addr 172.21.15.33:0/1062652446 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9a8b9a700 1 --2- 172.21.15.33:0/1062652446 >> v2:172.21.15.167:3300/0 conn(0x7fb9a40adf50 0x7fb9a40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb9a939b700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 0x7fb9a4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856517341 0 0) 0x7fb9a40c8e40 con 0x7fb9a40c9810 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb990003570 con 0x7fb9a40c9810 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371387031 0 0) 0x7fb990003570 con 0x7fb9a40c9810 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 msgr2=0x7fb9a4158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 0x7fb9a4158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 >> v2:172.21.15.167:3300/0 conn(0x7fb9a40adf50 msgr2=0x7fb9a40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 --2- 172.21.15.33:0/1062652446 >> v2:172.21.15.167:3300/0 conn(0x7fb9a40adf50 0x7fb9a40c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a40c8ad0 con 0x7fb9a40c9810 2023-10-06T05:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.029+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb99c0017f0 con 0x7fb9a40c9810 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 >> v1:172.21.15.33:6789/0 conn(0x7fb9a40c9810 legacy=0x7fb9a40699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990003a60 0x7fb990003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 0x7fb990004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb990004d50 0x7fb9900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a40ae6c0 con 0x7fb990004d50 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a40ae9a0 con 0x7fb990003a60 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99000fe70 con 0x7fb9a4155140 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a9b9c700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb990004d50 0x7fb9900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a8b9a700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 0x7fb990004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a939b700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990003a60 0x7fb990003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a9b9c700 1 -- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 msgr2=0x7fb990004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a9b9c700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a4155140 0x7fb990004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a9b9c700 1 -- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990003a60 msgr2=0x7fb990003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a9b9c700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990003a60 0x7fb990003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.030+0000 7fb9a9b9c700 1 -- 172.21.15.33:0/1062652446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb990003570 con 0x7fb990004d50 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb9a9b9c700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb990004d50 0x7fb9900051c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fb9a406da70 tx=0x7fb99c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e2761bf7ea32f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0070d0 con 0x7fb990004d50 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c007230 con 0x7fb990004d50 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb99bfff700 1 -- 172.21.15.33:0/1062652446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0075f0 con 0x7fb990004d50 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb9aa39d700 1 -- 172.21.15.33:0/1062652446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb990004d50 msgr2=0x7fb9900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/1062652446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb990004d50 0x7fb9900051c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fb9a406da70 tx=0x7fb99c002210 comp rx=0 tx=0).stop 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb9a9b9c700 1 -- 172.21.15.33:0/1062652446 reap_dead start 2023-10-06T05:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb9aa39d700 1 -- 172.21.15.33:0/1062652446 shutdown_connections 2023-10-06T05:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.031+0000 7fb9aa39d700 1 -- 172.21.15.33:0/1062652446 >> 172.21.15.33:0/1062652446 conn(0x7fb9a406d030 msgr2=0x7fb9a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.032+0000 7fb9aa39d700 1 -- 172.21.15.33:0/1062652446 shutdown_connections 2023-10-06T05:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.032+0000 7fb9aa39d700 1 -- 172.21.15.33:0/1062652446 wait complete. 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.033+0000 7fb9aa39d700 1 Processor -- start 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.033+0000 7fb9aa39d700 1 -- start start 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9aa39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990004d50 0x7fb9a4153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9aa39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 0x7fb9a414c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9aa39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a414ef20 0x7fb9a414f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9aa39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a414f900 con 0x7fb9a40c3b30 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9aa39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a40c5b00 con 0x7fb990004d50 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9aa39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a41540e0 con 0x7fb9a414ef20 2023-10-06T05:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9a8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 0x7fb9a414c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9a8b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 0x7fb9a414c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45080/0 (socket says 172.21.15.33:45080) 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.034+0000 7fb9a8b9a700 1 -- 172.21.15.33:0/2856066287 learned_addr learned my addr 172.21.15.33:0/2856066287 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a939b700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990004d50 0x7fb9a4153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a9b9c700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a414ef20 0x7fb9a414f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a8b9a700 1 -- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a414ef20 msgr2=0x7fb9a414f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a8b9a700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a414ef20 0x7fb9a414f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a8b9a700 1 -- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990004d50 msgr2=0x7fb9a4153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a8b9a700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990004d50 0x7fb9a4153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a8b9a700 1 -- 172.21.15.33:0/2856066287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a41543c0 con 0x7fb9a40c3b30 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9a8b9a700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 0x7fb9a414c540 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fb994006ac0 tx=0x7fb9940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994011ec0 con 0x7fb9a40c3b30 2023-10-06T05:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99400f940 con 0x7fb9a40c3b30 2023-10-06T05:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.035+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a41546a0 con 0x7fb9a40c3b30 2023-10-06T05:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.036+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99400fcc0 con 0x7fb9a40c3b30 2023-10-06T05:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.036+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a414ca80 con 0x7fb9a40c3b30 2023-10-06T05:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.037+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a40c5cb0 con 0x7fb9a40c3b30 2023-10-06T05:27:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.038+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb994014650 con 0x7fb9a40c3b30 2023-10-06T05:27:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.040+0000 7fb9997fa700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb988066f30 0x7fb9880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.040+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb994018030 con 0x7fb9a40c3b30 2023-10-06T05:27:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.040+0000 7fb9a939b700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb988066f30 0x7fb9880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.040+0000 7fb9a939b700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb988066f30 0x7fb9880693e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0006f00 tx=0x7fb9a00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:29.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.044+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9a40c5cb0 con 0x7fb9a40c3b30 2023-10-06T05:27:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.286+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9a40c5c70 con 0x7fb9a40c3b30 2023-10-06T05:27:29.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.287+0000 7fb9997fa700 1 -- 172.21.15.33:0/2856066287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fb994022090 con 0x7fb9a40c3b30 2023-10-06T05:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":137,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:27.860889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb988066f30 msgr2=0x7fb9880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb988066f30 0x7fb9880693e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0006f00 tx=0x7fb9a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 msgr2=0x7fb9a414c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 0x7fb9a414c540 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fb994006ac0 tx=0x7fb9940091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 shutdown_connections 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb988066f30 0x7fb9880693e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9a414ef20 0x7fb9a414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb990004d50 0x7fb9a4153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 --2- 172.21.15.33:0/2856066287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a40c3b30 0x7fb9a414c540 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 >> 172.21.15.33:0/2856066287 conn(0x7fb9a406d030 msgr2=0x7fb9a40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 shutdown_connections 2023-10-06T05:27:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.289+0000 7fb9aa39d700 1 -- 172.21.15.33:0/2856066287 wait complete. 2023-10-06T05:27:29.294 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 137 2023-10-06T05:27:29.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:27:29.520 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":7502,"entity":{"name":{"type":"client","num":7502},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1381827131}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":2.6349584300000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99624334414031135,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7502 v1:192.168.0.1:0/1381827131","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":8761,"entity":{"name":{"type":"client","num":8761},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":1699314688}},"state":"open","num_leases":0,"num_caps":0,"request_load_avg":0,"uptime":2.6349584300000002,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":true,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8761 v1:172.21.15.33:0/1699314688","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:27:29.520 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1381827131, "type": "v1" }, "name": { "num": 7502, "type": "client" } }, "id": 7502, "inst": "client.7502 v1:192.168.0.1:0/1381827131", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9962433441403113 }, "state": "open", "uptime": 2.63495843 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 1699314688, "type": "v1" }, "name": { "num": 8761, "type": "client" } }, "id": 8761, "inst": "client.8761 v1:172.21.15.33:0/1699314688", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 0, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": true, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0 }, "state": "open", "uptime": 2.63495843 } ] 2023-10-06T05:27:29.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.704+0000 7ff1f5d9d700 1 Processor -- start 2023-10-06T05:27:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1f5d9d700 1 -- start start 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1f5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 0x7ff1f0160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1f5d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006ab60 con 0x7ff1f00c8b50 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1f5d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006ae40 con 0x7ff1f0155af0 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1f5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f006b7e0 con 0x7ff1f00c7920 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1effff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1f00c8b50 0x7ff1f00c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55204/0 (socket says 172.21.15.33:55204) 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.706+0000 7ff1effff700 1 -- 172.21.15.33:0/695632891 learned_addr learned my addr 172.21.15.33:0/695632891 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1f559c700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 0x7ff1f0160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215773978 0 0) 0x7ff1f006ab60 con 0x7ff1f00c8b50 2023-10-06T05:27:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e0003570 con 0x7ff1f00c8b50 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450786778 0 0) 0x7ff1f006ae40 con 0x7ff1f0155af0 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1f006ab60 con 0x7ff1f0155af0 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3763290109 0 0) 0x7ff1e0003570 con 0x7ff1f00c8b50 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 msgr2=0x7ff1f0160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 0x7ff1f0160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 >> v1:172.21.15.167:6789/0 conn(0x7ff1f0155af0 legacy=0x7ff1f006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f006b120 con 0x7ff1f00c8b50 2023-10-06T05:27:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7ff1dc001470 con 0x7ff1f00c8b50 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 >> v1:172.21.15.33:6789/0 conn(0x7ff1f00c8b50 legacy=0x7ff1f00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 0x7ff1e0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e0004d50 0x7ff1e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f006b7e0 con 0x7ff1e0004d50 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e000de00 con 0x7ff1e00041d0 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.707+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e000dfc0 con 0x7ff1f00c7920 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1f559c700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e0004d50 0x7ff1e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1f4d9b700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 0x7ff1e0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 -- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 msgr2=0x7ff1e0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00c7920 0x7ff1e0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 -- 172.21.15.33:0/695632891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e0004d50 msgr2=0x7ff1e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e0004d50 0x7ff1e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 -- 172.21.15.33:0/695632891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0003570 con 0x7ff1e00041d0 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1f559c700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e0004d50 0x7ff1e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.708+0000 7ff1effff700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1e0004810 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c210 tx=0x7ff1dc001d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1701ff732c4c4b24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00cc10 con 0x7ff1e00041d0 2023-10-06T05:27:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc00cd70 con 0x7ff1e00041d0 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1ef7fe700 1 -- 172.21.15.33:0/695632891 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc005b70 con 0x7ff1e00041d0 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 msgr2=0x7ff1e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/695632891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1e0004810 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c210 tx=0x7ff1dc001d40 comp rx=0 tx=0).stop 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f559c700 1 -- 172.21.15.33:0/695632891 reap_dead start 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/695632891 shutdown_connections 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/695632891 >> 172.21.15.33:0/695632891 conn(0x7ff1f006d030 msgr2=0x7ff1f006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/695632891 shutdown_connections 2023-10-06T05:27:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.709+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/695632891 wait complete. 2023-10-06T05:27:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.711+0000 7ff1f5d9d700 1 Processor -- start 2023-10-06T05:27:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.711+0000 7ff1f5d9d700 1 -- start start 2023-10-06T05:27:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1f00b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 0x7ff1f0154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00ac890 0x7ff1f0154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0155ec0 con 0x7ff1f00ac420 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c4180 con 0x7ff1e00041d0 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c3570 con 0x7ff1f00ac890 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 0x7ff1f0154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 0x7ff1f0154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45102/0 (socket says 172.21.15.33:45102) 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 -- 172.21.15.33:0/332952144 learned_addr learned my addr 172.21.15.33:0/332952144 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f559c700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00ac890 0x7ff1f0154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f4d9b700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1f00b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 -- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00ac890 msgr2=0x7ff1f0154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00ac890 0x7ff1f0154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 -- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 msgr2=0x7ff1f00b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1f00b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 -- 172.21.15.33:0/332952144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00c3850 con 0x7ff1f00ac420 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f559c700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00ac890 0x7ff1f0154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1effff700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 0x7ff1f0154110 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c930 tx=0x7ff1dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc001650 con 0x7ff1f00ac420 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0070d0 con 0x7ff1f00ac420 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0074d0 con 0x7ff1f00ac420 2023-10-06T05:27:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f00c3b30 con 0x7ff1f00ac420 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.713+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f00c3190 con 0x7ff1f00ac420 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.714+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f00c4310 con 0x7ff1f00ac420 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.715+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1dc019030 con 0x7ff1f00ac420 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.717+0000 7ff1ecff9700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1d4066f30 0x7ff1d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.717+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff1dc085c60 con 0x7ff1f00ac420 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.717+0000 7ff1f4d9b700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1d4066f30 0x7ff1d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.717+0000 7ff1f4d9b700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1d4066f30 0x7ff1d40693e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4002c40 tx=0x7ff1e4007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.721+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1dc052ca0 con 0x7ff1f00ac420 2023-10-06T05:27:29.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.971+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1f004c380 con 0x7ff1f00ac420 2023-10-06T05:27:29.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.973+0000 7ff1ecff9700 1 -- 172.21.15.33:0/332952144 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4618 (secure 0 0 0) 0x7ff1dc0562c0 con 0x7ff1f00ac420 2023-10-06T05:27:29.975 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:29.975 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":137,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:27.860889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:29.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1d4066f30 msgr2=0x7ff1d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1d4066f30 0x7ff1d40693e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4002c40 tx=0x7ff1e4007d30 comp rx=0 tx=0).stop 2023-10-06T05:27:29.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 msgr2=0x7ff1f0154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:29.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 0x7ff1f0154110 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c930 tx=0x7ff1dc015000 comp rx=0 tx=0).stop 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 shutdown_connections 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1d4066f30 0x7ff1d40693e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1f00ac890 0x7ff1f0154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e00041d0 0x7ff1f00b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 --2- 172.21.15.33:0/332952144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f00ac420 0x7ff1f0154110 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 >> 172.21.15.33:0/332952144 conn(0x7ff1f006d030 msgr2=0x7ff1f0159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 shutdown_connections 2023-10-06T05:27:29.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:29.974+0000 7ff1f5d9d700 1 -- 172.21.15.33:0/332952144 wait complete. 2023-10-06T05:27:29.979 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 137 2023-10-06T05:27:29.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session evict 8761 2023-10-06T05:27:30.905 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:27:30.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:31.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.101+0000 7fc6889fe700 1 Processor -- start 2023-10-06T05:27:31.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.101+0000 7fc6889fe700 1 -- start start 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.102+0000 7fc6889fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc684159530 0x7fc68406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.102+0000 7fc6889fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.102+0000 7fc6889fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68406e3e0 con 0x7fc6840b0310 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.102+0000 7fc6889fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840b1e00 con 0x7fc684159530 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.102+0000 7fc6889fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406f6b0 con 0x7fc68406df70 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68359e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc6840b0310 0x7fc684159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50264/0 (socket says 172.21.15.33:50264) 2023-10-06T05:27:31.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68359e700 1 -- 172.21.15.33:0/1262608302 learned_addr learned my addr 172.21.15.33:0/1262608302 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68259c700 1 --2- 172.21.15.33:0/1262608302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc682d9d700 1 --2- 172.21.15.33:0/1262608302 >> v2:172.21.15.167:3300/0 conn(0x7fc684159530 0x7fc68406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc681d9b700 1 -- 172.21.15.33:0/1262608302 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836407739 0 0) 0x7fc68406e3e0 con 0x7fc6840b0310 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc681d9b700 1 -- 172.21.15.33:0/1262608302 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc66c003570 con 0x7fc6840b0310 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68259c700 1 -- 172.21.15.33:0/1262608302 >> v2:172.21.15.167:3300/0 conn(0x7fc684159530 msgr2=0x7fc68406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68259c700 1 --2- 172.21.15.33:0/1262608302 >> v2:172.21.15.167:3300/0 conn(0x7fc684159530 0x7fc68406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68259c700 1 -- 172.21.15.33:0/1262608302 >> v1:172.21.15.33:6789/0 conn(0x7fc6840b0310 legacy=0x7fc684159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc68259c700 1 -- 172.21.15.33:0/1262608302 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68406f220 con 0x7fc68406df70 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.103+0000 7fc682d9d700 1 --2- 172.21.15.33:0/1262608302 >> v2:172.21.15.167:3300/0 conn(0x7fc684159530 0x7fc68406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.104+0000 7fc68259c700 1 --2- 172.21.15.33:0/1262608302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840afdd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6780098e0 tx=0x7fc6780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1417ce62fd7ea7e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.104+0000 7fc681d9b700 1 -- 172.21.15.33:0/1262608302 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011630 con 0x7fc68406df70 2023-10-06T05:27:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.104+0000 7fc681d9b700 1 -- 172.21.15.33:0/1262608302 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678011790 con 0x7fc68406df70 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.104+0000 7fc681d9b700 1 -- 172.21.15.33:0/1262608302 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011b30 con 0x7fc68406df70 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.104+0000 7fc6889fe700 1 -- 172.21.15.33:0/1262608302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 msgr2=0x7fc6840afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.104+0000 7fc6889fe700 1 --2- 172.21.15.33:0/1262608302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840afdd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6780098e0 tx=0x7fc6780091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.105+0000 7fc6889fe700 1 -- 172.21.15.33:0/1262608302 shutdown_connections 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.105+0000 7fc6889fe700 1 --2- 172.21.15.33:0/1262608302 >> v2:172.21.15.167:3300/0 conn(0x7fc684159530 0x7fc68406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.105+0000 7fc6889fe700 1 --2- 172.21.15.33:0/1262608302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840afdd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.105+0000 7fc6889fe700 1 -- 172.21.15.33:0/1262608302 >> 172.21.15.33:0/1262608302 conn(0x7fc68406c560 msgr2=0x7fc68406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.105+0000 7fc6889fe700 1 -- 172.21.15.33:0/1262608302 shutdown_connections 2023-10-06T05:27:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.105+0000 7fc6889fe700 1 -- 172.21.15.33:0/1262608302 wait complete. 2023-10-06T05:27:31.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.106+0000 7fc6889fe700 1 Processor -- start 2023-10-06T05:27:31.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.106+0000 7fc6889fe700 1 -- start start 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.107+0000 7fc6889fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.107+0000 7fc6889fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 0x7fc6840c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.107+0000 7fc6889fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc684159530 0x7fc6840cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.107+0000 7fc6889fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840c9f00 con 0x7fc6840b0310 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.107+0000 7fc6889fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840ccb10 con 0x7fc684159530 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.107+0000 7fc6889fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cbf00 con 0x7fc68406df70 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 0x7fc6840c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 0x7fc6840c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41588/0 (socket says 172.21.15.33:41588) 2023-10-06T05:27:31.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 -- 172.21.15.33:0/3279583929 learned_addr learned my addr 172.21.15.33:0/3279583929 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc682d9d700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68359e700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc684159530 0x7fc6840cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 -- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 msgr2=0x7fc6840cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 -- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc684159530 msgr2=0x7fc6840cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc684159530 0x7fc6840cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 -- 172.21.15.33:0/3279583929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc674009420 con 0x7fc6840b0310 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.108+0000 7fc68259c700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 0x7fc6840c99c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc6840719a0 tx=0x7fc678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.109+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678000d10 con 0x7fc6840b0310 2023-10-06T05:27:31.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.109+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678011b10 con 0x7fc6840b0310 2023-10-06T05:27:31.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.109+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc678009420 con 0x7fc6840b0310 2023-10-06T05:27:31.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.109+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678025660 con 0x7fc6840b0310 2023-10-06T05:27:31.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.109+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6840cc4c0 con 0x7fc6840b0310 2023-10-06T05:27:31.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.110+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68404c380 con 0x7fc6840b0310 2023-10-06T05:27:31.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.111+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc678025a50 con 0x7fc6840b0310 2023-10-06T05:27:31.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.112+0000 7fc66affd700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc670066f30 0x7fc6700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.112+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc678085f10 con 0x7fc6840b0310 2023-10-06T05:27:31.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.112+0000 7fc682d9d700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc670066f30 0x7fc6700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.113+0000 7fc682d9d700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc670066f30 0x7fc6700693e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc674009eb0 tx=0x7fc6740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:31.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.116+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc678052f00 con 0x7fc6840b0310 2023-10-06T05:27:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.366+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc684070f80 con 0x7fc6840b0310 2023-10-06T05:27:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.366+0000 7fc66affd700 1 -- 172.21.15.33:0/3279583929 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fc678056520 con 0x7fc6840b0310 2023-10-06T05:27:31.368 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:31.368 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":137,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:27.860889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc670066f30 msgr2=0x7fc6700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc670066f30 0x7fc6700693e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc674009eb0 tx=0x7fc6740091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 msgr2=0x7fc6840c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 0x7fc6840c99c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc6840719a0 tx=0x7fc678017000 comp rx=0 tx=0).stop 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 shutdown_connections 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc670066f30 0x7fc6700693e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc684159530 0x7fc6840cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc68406df70 0x7fc6840cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 --2- 172.21.15.33:0/3279583929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6840b0310 0x7fc6840c99c0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 >> 172.21.15.33:0/3279583929 conn(0x7fc68406c560 msgr2=0x7fc6840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:31.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 shutdown_connections 2023-10-06T05:27:31.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.369+0000 7fc6889fe700 1 -- 172.21.15.33:0/3279583929 wait complete. 2023-10-06T05:27:31.372 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 137 2023-10-06T05:27:31.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:27:31.599 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":7502,"entity":{"name":{"type":"client","num":7502},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1381827131}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.714925612,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99146713884655568,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7502 v1:192.168.0.1:0/1381827131","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:27:31.599 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1381827131, "type": "v1" }, "name": { "num": 7502, "type": "client" } }, "id": 7502, "inst": "client.7502 v1:192.168.0.1:0/1381827131", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9914671388465557 }, "state": "open", "uptime": 4.714925612 } ] 2023-10-06T05:27:31.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.785+0000 7f80d939d700 1 Processor -- start 2023-10-06T05:27:31.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.785+0000 7f80d939d700 1 -- start start 2023-10-06T05:27:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.786+0000 7f80d939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80d40cb710 0x7f80d406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.786+0000 7f80d939d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40c9cb0 con 0x7f80d40c8050 2023-10-06T05:27:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.786+0000 7f80d939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40ca270 con 0x7f80d413faa0 2023-10-06T05:27:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.786+0000 7f80d939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d406b840 con 0x7f80d40cb710 2023-10-06T05:27:31.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f80d40c8050 0x7f80d406aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50270/0 (socket says 172.21.15.33:50270) 2023-10-06T05:27:31.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d37fe700 1 -- 172.21.15.33:0/2466327384 learned_addr learned my addr 172.21.15.33:0/2466327384 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:31.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d8b9c700 1 --2- 172.21.15.33:0/2466327384 >> v2:172.21.15.33:3300/0 conn(0x7f80d40cb710 0x7f80d406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d2ffd700 1 -- 172.21.15.33:0/2466327384 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215163613 0 0) 0x7f80d40c9cb0 con 0x7f80d40c8050 2023-10-06T05:27:31.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d2ffd700 1 -- 172.21.15.33:0/2466327384 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80bc003570 con 0x7f80d40c8050 2023-10-06T05:27:31.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d8b9c700 1 -- 172.21.15.33:0/2466327384 >> v1:172.21.15.167:6789/0 conn(0x7f80d413faa0 legacy=0x7f80d40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d2ffd700 1 -- 172.21.15.33:0/2466327384 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197635407 0 0) 0x7f80d40ca270 con 0x7f80d413faa0 2023-10-06T05:27:31.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d8b9c700 1 -- 172.21.15.33:0/2466327384 >> v1:172.21.15.33:6789/0 conn(0x7f80d40c8050 legacy=0x7f80d406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d8b9c700 1 -- 172.21.15.33:0/2466327384 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d406bb20 con 0x7f80d40cb710 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.787+0000 7f80d8b9c700 1 --2- 172.21.15.33:0/2466327384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40cb710 0x7f80d406b1e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009e60 tx=0x7f80cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea767aa9e70835c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d2ffd700 1 -- 172.21.15.33:0/2466327384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00e070 con 0x7f80d40cb710 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d2ffd700 1 -- 172.21.15.33:0/2466327384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc00bb60 con 0x7f80d40cb710 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d2ffd700 1 -- 172.21.15.33:0/2466327384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc012430 con 0x7f80d40cb710 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 -- 172.21.15.33:0/2466327384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40cb710 msgr2=0x7f80d406b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 --2- 172.21.15.33:0/2466327384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40cb710 0x7f80d406b1e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009e60 tx=0x7f80cc009780 comp rx=0 tx=0).stop 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 -- 172.21.15.33:0/2466327384 shutdown_connections 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 --2- 172.21.15.33:0/2466327384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40cb710 0x7f80d406b1e0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 -- 172.21.15.33:0/2466327384 >> 172.21.15.33:0/2466327384 conn(0x7f80d406d030 msgr2=0x7f80d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 -- 172.21.15.33:0/2466327384 shutdown_connections 2023-10-06T05:27:31.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.788+0000 7f80d939d700 1 -- 172.21.15.33:0/2466327384 wait complete. 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.790+0000 7f80d939d700 1 Processor -- start 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.790+0000 7f80d939d700 1 -- start start 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 0x7f80d4139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80d40cb710 0x7f80d4168790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80d413faa0 0x7f80d41675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d413bf00 con 0x7f80d40c8050 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d413b8f0 con 0x7f80d413faa0 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d41680a0 con 0x7f80d40cb710 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 0x7f80d4139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 0x7f80d4139dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41594/0 (socket says 172.21.15.33:41594) 2023-10-06T05:27:31.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d3fff700 1 -- 172.21.15.33:0/1199347230 learned_addr learned my addr 172.21.15.33:0/1199347230 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:31.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d37fe700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80d40cb710 0x7f80d4168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.791+0000 7f80d8b9c700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80d413faa0 0x7f80d41675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d3fff700 1 -- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80d40cb710 msgr2=0x7f80d4168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d3fff700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80d40cb710 0x7f80d4168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d3fff700 1 -- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80d413faa0 msgr2=0x7f80d41675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d3fff700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80d413faa0 0x7f80d41675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d3fff700 1 -- 172.21.15.33:0/1199347230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc009a20 con 0x7f80d40c8050 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d3fff700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 0x7f80d4139dd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c530 tx=0x7f80c400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4011590 con 0x7f80d40c8050 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c4011710 con 0x7f80d40c8050 2023-10-06T05:27:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4011ad0 con 0x7f80d40c8050 2023-10-06T05:27:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d4168380 con 0x7f80d40c8050 2023-10-06T05:27:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.792+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d413a310 con 0x7f80d40c8050 2023-10-06T05:27:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.793+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d406f900 con 0x7f80d40c8050 2023-10-06T05:27:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.794+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80c4001010 con 0x7f80d40c8050 2023-10-06T05:27:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.796+0000 7f80bbfff700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80c0066f30 0x7f80c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:31.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.796+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f80c4083b20 con 0x7f80d40c8050 2023-10-06T05:27:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.796+0000 7f80d37fe700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80c0066f30 0x7f80c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.797+0000 7f80d37fe700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80c0066f30 0x7f80c00693e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f80d415a040 tx=0x7f80c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:31.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:31.800+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80c4050b10 con 0x7f80d40c8050 2023-10-06T05:27:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.053+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80d404c380 con 0x7f80d40c8050 2023-10-06T05:27:32.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.053+0000 7f80bbfff700 1 -- 172.21.15.33:0/1199347230 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f80c4017070 con 0x7f80d40c8050 2023-10-06T05:27:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":138,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:31.881161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:rejoin","state_seq":5,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80c0066f30 msgr2=0x7f80c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:32.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80c0066f30 0x7f80c00693e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f80d415a040 tx=0x7f80c8009040 comp rx=0 tx=0).stop 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 msgr2=0x7f80d4139dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 0x7f80d4139dd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c530 tx=0x7f80c400c840 comp rx=0 tx=0).stop 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 shutdown_connections 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80c0066f30 0x7f80c00693e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80d413faa0 0x7f80d41675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d40c8050 0x7f80d4139dd0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 --2- 172.21.15.33:0/1199347230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80d40cb710 0x7f80d4168790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.055+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 >> 172.21.15.33:0/1199347230 conn(0x7f80d406d030 msgr2=0x7f80d40c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.056+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 shutdown_connections 2023-10-06T05:27:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:32.056+0000 7f80d939d700 1 -- 172.21.15.33:0/1199347230 wait complete. 2023-10-06T05:27:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 138 2023-10-06T05:27:32.074 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2023-10-06T05:27:33.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.272+0000 7f089159d700 1 Processor -- start 2023-10-06T05:27:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.272+0000 7f089159d700 1 -- start start 2023-10-06T05:27:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.273+0000 7f089159d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f088c0ca590 0x7f088c0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.273+0000 7f089159d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f088c159a20 0x7f088c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.273+0000 7f089159d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0c92d0 con 0x7f088c159650 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.273+0000 7f089159d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0c7bc0 con 0x7f088c159a20 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.273+0000 7f089159d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0c86c0 con 0x7f088c0ca590 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.273+0000 7f0890d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f088c159a20 0x7f088c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f088c159a20 0x7f088c0c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41604/0 (socket says 172.21.15.33:41604) 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 -- 172.21.15.33:0/3987040338 learned_addr learned my addr 172.21.15.33:0/3987040338 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f088b7fe700 1 --2- 172.21.15.33:0/3987040338 >> v2:172.21.15.167:3300/0 conn(0x7f088c0ca590 0x7f088c0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 -- 172.21.15.33:0/3987040338 >> v2:172.21.15.167:3300/0 conn(0x7f088c0ca590 msgr2=0x7f088c0c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 --2- 172.21.15.33:0/3987040338 >> v2:172.21.15.167:3300/0 conn(0x7f088c0ca590 0x7f088c0c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 -- 172.21.15.33:0/3987040338 >> v1:172.21.15.33:6789/0 conn(0x7f088c159650 legacy=0x7f088c0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 -- 172.21.15.33:0/3987040338 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0c89a0 con 0x7f088c159a20 2023-10-06T05:27:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.274+0000 7f0890d9c700 1 --2- 172.21.15.33:0/3987040338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159a20 0x7f088c0c7680 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f088c068540 tx=0x7f0884009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8fa4c22ed68b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f088affd700 1 -- 172.21.15.33:0/3987040338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400e070 con 0x7f088c159a20 2023-10-06T05:27:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f088affd700 1 -- 172.21.15.33:0/3987040338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088400ba50 con 0x7f088c159a20 2023-10-06T05:27:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f088affd700 1 -- 172.21.15.33:0/3987040338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840092e0 con 0x7f088c159a20 2023-10-06T05:27:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 -- 172.21.15.33:0/3987040338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159a20 msgr2=0x7f088c0c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 --2- 172.21.15.33:0/3987040338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159a20 0x7f088c0c7680 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f088c068540 tx=0x7f0884009840 comp rx=0 tx=0).stop 2023-10-06T05:27:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 -- 172.21.15.33:0/3987040338 shutdown_connections 2023-10-06T05:27:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 --2- 172.21.15.33:0/3987040338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159a20 0x7f088c0c7680 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 --2- 172.21.15.33:0/3987040338 >> v2:172.21.15.167:3300/0 conn(0x7f088c0ca590 0x7f088c0c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 -- 172.21.15.33:0/3987040338 >> 172.21.15.33:0/3987040338 conn(0x7f088c06d030 msgr2=0x7f088c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 -- 172.21.15.33:0/3987040338 shutdown_connections 2023-10-06T05:27:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.275+0000 7f089159d700 1 -- 172.21.15.33:0/3987040338 wait complete. 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.277+0000 7f089159d700 1 Processor -- start 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.277+0000 7f089159d700 1 -- start start 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f089159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c0ca590 0x7f088c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f089159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 0x7f088c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f089159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f088c159a20 0x7f088c15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f089159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0afc60 con 0x7f088c159650 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f089159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0af050 con 0x7f088c0ca590 2023-10-06T05:27:33.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f089159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0af330 con 0x7f088c159a20 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 0x7f088c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 0x7f088c159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41610/0 (socket says 172.21.15.33:41610) 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 -- 172.21.15.33:0/4101365055 learned_addr learned my addr 172.21.15.33:0/4101365055 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088bfff700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c0ca590 0x7f088c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f0890d9c700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f088c159a20 0x7f088c15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 -- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f088c159a20 msgr2=0x7f088c15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f088c159a20 0x7f088c15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 -- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c0ca590 msgr2=0x7f088c153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c0ca590 0x7f088c153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.278+0000 7f088b7fe700 1 -- 172.21.15.33:0/4101365055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884009ae0 con 0x7f088c159650 2023-10-06T05:27:33.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.279+0000 7f088b7fe700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 0x7f088c159eb0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f088000d890 tx=0x7f0880008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.279+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800125f0 con 0x7f088c159650 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.279+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0880012770 con 0x7f088c159650 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.279+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880012b30 con 0x7f088c159650 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.279+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c0af610 con 0x7f088c159650 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.279+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c154350 con 0x7f088c159650 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.280+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c0afdd0 con 0x7f088c159650 2023-10-06T05:27:33.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.281+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f088001e460 con 0x7f088c159650 2023-10-06T05:27:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.283+0000 7f0873fff700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0878066f30 0x7f08780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.283+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0880084d80 con 0x7f088c159650 2023-10-06T05:27:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.283+0000 7f088bfff700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0878066f30 0x7f08780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.284+0000 7f088bfff700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0878066f30 0x7f08780693e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f087c003a80 tx=0x7f087c007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.287+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0880051d70 con 0x7f088c159650 2023-10-06T05:27:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.532+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f088c04c380 con 0x7f088c159650 2023-10-06T05:27:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.533+0000 7f0873fff700 1 -- 172.21.15.33:0/4101365055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f0880055390 con 0x7f088c159650 2023-10-06T05:27:33.535 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:33.535 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":139,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:32.888818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":6,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0878066f30 msgr2=0x7f08780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0878066f30 0x7f08780693e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f087c003a80 tx=0x7f087c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 msgr2=0x7f088c159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 0x7f088c159eb0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f088000d890 tx=0x7f0880008040 comp rx=0 tx=0).stop 2023-10-06T05:27:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 shutdown_connections 2023-10-06T05:27:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0878066f30 0x7f08780693e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f088c159a20 0x7f088c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f088c0ca590 0x7f088c153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 --2- 172.21.15.33:0/4101365055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088c159650 0x7f088c159eb0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 >> 172.21.15.33:0/4101365055 conn(0x7f088c06d030 msgr2=0x7f088c15ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:33.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 shutdown_connections 2023-10-06T05:27:33.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.535+0000 7f089159d700 1 -- 172.21.15.33:0/4101365055 wait complete. 2023-10-06T05:27:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 139 2023-10-06T05:27:33.556 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:27:33.556 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 1.9562995433807373s 2023-10-06T05:27:33.556 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:27:33.556 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:27:33.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:33.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:27:33.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:27:33.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:33.633 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:27:33.679 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:27:33.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.907+0000 7f64d359e700 1 Processor -- start 2023-10-06T05:27:33.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.907+0000 7f64d359e700 1 -- start start 2023-10-06T05:27:33.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.908+0000 7f64d359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f64d4155af0 0x7f64d406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.908+0000 7f64d359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d406a6a0 0x7f64d40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.908+0000 7f64d359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64d40c8b50 0x7f64d40c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.908+0000 7f64d359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d406b7e0 con 0x7f64d40c8b50 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.908+0000 7f64d359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d406ab60 con 0x7f64d4155af0 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.908+0000 7f64d359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d406ae40 con 0x7f64d406a6a0 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64d40c8b50 0x7f64d40c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64d40c8b50 0x7f64d40c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41614/0 (socket says 172.21.15.33:41614) 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 -- 172.21.15.33:0/4222661840 learned_addr learned my addr 172.21.15.33:0/4222661840 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d259c700 1 --2- 172.21.15.33:0/4222661840 >> v2:172.21.15.167:3300/0 conn(0x7f64d4155af0 0x7f64d406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d1d9b700 1 --2- 172.21.15.33:0/4222661840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d406a6a0 0x7f64d40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 -- 172.21.15.33:0/4222661840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d406a6a0 msgr2=0x7f64d40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:33.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 --2- 172.21.15.33:0/4222661840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d406a6a0 0x7f64d40c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 -- 172.21.15.33:0/4222661840 >> v2:172.21.15.167:3300/0 conn(0x7f64d4155af0 msgr2=0x7f64d406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 --2- 172.21.15.33:0/4222661840 >> v2:172.21.15.167:3300/0 conn(0x7f64d4155af0 0x7f64d406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 -- 172.21.15.33:0/4222661840 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d406b120 con 0x7f64d40c8b50 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d259c700 1 --2- 172.21.15.33:0/4222661840 >> v2:172.21.15.167:3300/0 conn(0x7f64d4155af0 0x7f64d406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.909+0000 7f64d2d9d700 1 --2- 172.21.15.33:0/4222661840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40c7750 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f64d40697c0 tx=0x7f64cc00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1283f5ef60fefa60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d159a700 1 -- 172.21.15.33:0/4222661840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc010070 con 0x7f64d40c8b50 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d159a700 1 -- 172.21.15.33:0/4222661840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00dc80 con 0x7f64d40c8b50 2023-10-06T05:27:33.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d159a700 1 -- 172.21.15.33:0/4222661840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc016400 con 0x7f64d40c8b50 2023-10-06T05:27:33.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 -- 172.21.15.33:0/4222661840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 msgr2=0x7f64d40c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 --2- 172.21.15.33:0/4222661840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40c7750 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f64d40697c0 tx=0x7f64cc00ba50 comp rx=0 tx=0).stop 2023-10-06T05:27:33.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 -- 172.21.15.33:0/4222661840 shutdown_connections 2023-10-06T05:27:33.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 --2- 172.21.15.33:0/4222661840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40c7750 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 --2- 172.21.15.33:0/4222661840 >> v2:172.21.15.167:3300/0 conn(0x7f64d4155af0 0x7f64d406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 --2- 172.21.15.33:0/4222661840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d406a6a0 0x7f64d40c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.910+0000 7f64d359e700 1 -- 172.21.15.33:0/4222661840 >> 172.21.15.33:0/4222661840 conn(0x7f64d406d030 msgr2=0x7f64d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.911+0000 7f64d359e700 1 -- 172.21.15.33:0/4222661840 shutdown_connections 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.911+0000 7f64d359e700 1 -- 172.21.15.33:0/4222661840 wait complete. 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.912+0000 7f64d359e700 1 Processor -- start 2023-10-06T05:27:33.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.912+0000 7f64d359e700 1 -- start start 2023-10-06T05:27:33.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d406a6a0 0x7f64d40b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d4155af0 0x7f64d41454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d41694b0 con 0x7f64d40c8b50 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c3a60 con 0x7f64d406a6a0 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b0410 con 0x7f64d4155af0 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41620/0 (socket says 172.21.15.33:41620) 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.913+0000 7f64d1d9b700 1 -- 172.21.15.33:0/629677988 learned_addr learned my addr 172.21.15.33:0/629677988 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d2d9d700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d4155af0 0x7f64d41454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d259c700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d406a6a0 0x7f64d40b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d1d9b700 1 -- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d4155af0 msgr2=0x7f64d41454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d1d9b700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d4155af0 0x7f64d41454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d1d9b700 1 -- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d406a6a0 msgr2=0x7f64d40b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d1d9b700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d406a6a0 0x7f64d40b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d1d9b700 1 -- 172.21.15.33:0/629677988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc00bcf0 con 0x7f64d40c8b50 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d259c700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d406a6a0 0x7f64d40b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d1d9b700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40ae4c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f64c800d7e0 tx=0x7f64c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80125f0 con 0x7f64d40c8b50 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8012750 con 0x7f64d40c8b50 2023-10-06T05:27:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8012b10 con 0x7f64d40c8b50 2023-10-06T05:27:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d40b06f0 con 0x7f64d40c8b50 2023-10-06T05:27:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.914+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d40b0cb0 con 0x7f64d40c8b50 2023-10-06T05:27:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.916+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d404c380 con 0x7f64d40c8b50 2023-10-06T05:27:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.916+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64c801e460 con 0x7f64d40c8b50 2023-10-06T05:27:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.917+0000 7f64c27fc700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b4066f30 0x7f64b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.917+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f64c8084e20 con 0x7f64d40c8b50 2023-10-06T05:27:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.918+0000 7f64d259c700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b4066f30 0x7f64b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.918+0000 7f64d259c700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b4066f30 0x7f64b40693e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006f00 tx=0x7f64c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:33.922+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64c8051e10 con 0x7f64d40c8b50 2023-10-06T05:27:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.168+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64d406b400 con 0x7f64d40c8b50 2023-10-06T05:27:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.168+0000 7f64c27fc700 1 -- 172.21.15.33:0/629677988 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v139) v1 ==== 53+0+73 (secure 0 0 0) 0x7f64c8055430 con 0x7f64d40c8b50 2023-10-06T05:27:34.170 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b4066f30 msgr2=0x7f64b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b4066f30 0x7f64b40693e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006f00 tx=0x7f64c4008040 comp rx=0 tx=0).stop 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 msgr2=0x7f64d40ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40ae4c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f64c800d7e0 tx=0x7f64c8008040 comp rx=0 tx=0).stop 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 shutdown_connections 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b4066f30 0x7f64b40693e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64d4155af0 0x7f64d41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d406a6a0 0x7f64d40b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:34.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 --2- 172.21.15.33:0/629677988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64d40c8b50 0x7f64d40ae4c0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:34.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 >> 172.21.15.33:0/629677988 conn(0x7f64d406d030 msgr2=0x7f64d406e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:34.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 shutdown_connections 2023-10-06T05:27:34.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:34.171+0000 7f64d359e700 1 -- 172.21.15.33:0/629677988 wait complete. 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:27:34.190 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:34.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:34.191 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:27:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:27:34.229 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:27:34.229 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:27:34.230 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41503sec preferred_lft 41503sec 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:27:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:27:34.233 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:34.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:34.233 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:27:34.330 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:27:34.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:34.333 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:27:34.371 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:27:34.372 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:27:34.372 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:27:34.372 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:27:34.372 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:27:34.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:34.373 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:27:34.373 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:27:34.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:27:34.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:27:34.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:27:34.373 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:27:34.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:34.572 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:27:34.608 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:27:34.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:27:34.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:27:34.610 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:34.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:34.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:27:34.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:34.720 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:34.758 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:34.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:34.758 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:34.834 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:27:34.834 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:27:34.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:34.967 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:34.967 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:27:34.968 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:27:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:27:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:27:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:27:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:27:35.005 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:27:35.005 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:27:35.005 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:27:35.005 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:27:35.005 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:27:35.006 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:27:35.006 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:27:35.006 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:27:35.010 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:27:35.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:35.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:35.073 DEBUG:tasks.cephfs.mount:Creating test file 2023-10-06 05:27:35.073074 0 2023-10-06T05:27:35.073 DEBUG:teuthology.orchestra.run.smithi033:> touch '/home/ubuntu/cephtest/mnt.0/2023-10-06 05:27:35.073074 0' 2023-10-06T05:27:35.113 DEBUG:tasks.cephfs.mount:Deleting test file 2023-10-06 05:27:35.073074 0 2023-10-06T05:27:35.113 DEBUG:teuthology.orchestra.run.smithi033:> rm -f '/home/ubuntu/cephtest/mnt.0/2023-10-06 05:27:35.073074 0' 2023-10-06T05:27:35.191 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:27:35.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:35.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:35.398 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:27:35.399 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:27:35.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:35.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:35.486 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:27:35.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:35.486 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:27:35.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:27:35.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:35.523 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:27:35.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:27:35.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:35.612 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:27:35.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.834+0000 7f2b6c99d700 1 Processor -- start 2023-10-06T05:27:35.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.834+0000 7f2b6c99d700 1 -- start start 2023-10-06T05:27:35.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.835+0000 7f2b6c99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2b6806ab00 0x7f2b680b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:35.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.835+0000 7f2b6c99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6815e2d0 con 0x7f2b680b2e30 2023-10-06T05:27:35.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.835+0000 7f2b6c99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6806afa0 con 0x7f2b680b0a40 2023-10-06T05:27:35.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.835+0000 7f2b6c99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6815d6c0 con 0x7f2b6806ab00 2023-10-06T05:27:35.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b66ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2b6806ab00 0x7f2b680b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:35.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b67fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2b680b2e30 0x7f2b6815a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50284/0 (socket says 172.21.15.33:50284) 2023-10-06T05:27:35.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b67fff700 1 -- 172.21.15.33:0/2241385416 learned_addr learned my addr 172.21.15.33:0/2241385416 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b66ffd700 1 -- 172.21.15.33:0/2241385416 >> v1:172.21.15.167:6789/0 conn(0x7f2b680b0a40 legacy=0x7f2b680b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b66ffd700 1 -- 172.21.15.33:0/2241385416 >> v1:172.21.15.33:6789/0 conn(0x7f2b680b2e30 legacy=0x7f2b6815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b66ffd700 1 -- 172.21.15.33:0/2241385416 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6815d9a0 con 0x7f2b6806ab00 2023-10-06T05:27:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b667fc700 1 -- 172.21.15.33:0/2241385416 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106154409 0 0) 0x7f2b6815e2d0 con 0x7f2b680b2e30 2023-10-06T05:27:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.836+0000 7f2b66ffd700 1 --2- 172.21.15.33:0/2241385416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b6806ab00 0x7f2b680b28f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098e0 tx=0x7f2b5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2566ecf07b35d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b667fc700 1 -- 172.21.15.33:0/2241385416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c009e80 con 0x7f2b6806ab00 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b667fc700 1 -- 172.21.15.33:0/2241385416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c00b860 con 0x7f2b6806ab00 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b667fc700 1 -- 172.21.15.33:0/2241385416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00bbe0 con 0x7f2b6806ab00 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 -- 172.21.15.33:0/2241385416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b6806ab00 msgr2=0x7f2b680b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/2241385416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b6806ab00 0x7f2b680b28f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098e0 tx=0x7f2b5c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 -- 172.21.15.33:0/2241385416 shutdown_connections 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/2241385416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b6806ab00 0x7f2b680b28f0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 -- 172.21.15.33:0/2241385416 >> 172.21.15.33:0/2241385416 conn(0x7f2b6806e0f0 msgr2=0x7f2b680675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 -- 172.21.15.33:0/2241385416 shutdown_connections 2023-10-06T05:27:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.837+0000 7f2b6c99d700 1 -- 172.21.15.33:0/2241385416 wait complete. 2023-10-06T05:27:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.839+0000 7f2b6c99d700 1 Processor -- start 2023-10-06T05:27:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.839+0000 7f2b6c99d700 1 -- start start 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b6c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b6806ab00 0x7f2b6815c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b6c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 0x7f2b681488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b6c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b680b2e30 0x7f2b68148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b6c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6816ae00 con 0x7f2b680b0a40 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b6c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6816ba90 con 0x7f2b680b2e30 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b6c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6816b160 con 0x7f2b6806ab00 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 0x7f2b681488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:35.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 0x7f2b681488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41638/0 (socket says 172.21.15.33:41638) 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b66ffd700 1 -- 172.21.15.33:0/1568894225 learned_addr learned my addr 172.21.15.33:0/1568894225 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b677fe700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b6806ab00 0x7f2b6815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.840+0000 7f2b67fff700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b680b2e30 0x7f2b68148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b66ffd700 1 -- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b6806ab00 msgr2=0x7f2b6815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b66ffd700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b6806ab00 0x7f2b6815c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b66ffd700 1 -- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b680b2e30 msgr2=0x7f2b68148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b66ffd700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b680b2e30 0x7f2b68148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b66ffd700 1 -- 172.21.15.33:0/1568894225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c009420 con 0x7f2b680b0a40 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b66ffd700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 0x7f2b681488e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806b130 tx=0x7f2b5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01f070 con 0x7f2b680b0a40 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c000c90 con 0x7f2b680b0a40 2023-10-06T05:27:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c025420 con 0x7f2b680b0a40 2023-10-06T05:27:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6816b440 con 0x7f2b680b0a40 2023-10-06T05:27:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.841+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6815c880 con 0x7f2b680b0a40 2023-10-06T05:27:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.843+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6815caf0 con 0x7f2b680b0a40 2023-10-06T05:27:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.843+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b5c023070 con 0x7f2b680b0a40 2023-10-06T05:27:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.844+0000 7f2b4f7fe700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2b54066f30 0x7f2b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.844+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2b5c085c60 con 0x7f2b680b0a40 2023-10-06T05:27:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.845+0000 7f2b677fe700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2b54066f30 0x7f2b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.845+0000 7f2b677fe700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2b54066f30 0x7f2b540693e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2b6815ce30 tx=0x7f2b58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:35.849+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b6815caf0 con 0x7f2b680b0a40 2023-10-06T05:27:36.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.101+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b6815caf0 con 0x7f2b680b0a40 2023-10-06T05:27:36.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.102+0000 7f2b4f7fe700 1 -- 172.21.15.33:0/1568894225 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f2b6815caf0 con 0x7f2b680b0a40 2023-10-06T05:27:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:36.104 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":139,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:32.888818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":6,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:36.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2b54066f30 msgr2=0x7f2b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2b54066f30 0x7f2b540693e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2b6815ce30 tx=0x7f2b58009040 comp rx=0 tx=0).stop 2023-10-06T05:27:36.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 msgr2=0x7f2b681488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 0x7f2b681488e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806b130 tx=0x7f2b5c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 shutdown_connections 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2b54066f30 0x7f2b540693e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b680b2e30 0x7f2b68148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2b6806ab00 0x7f2b6815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 --2- 172.21.15.33:0/1568894225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b680b0a40 0x7f2b681488e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 >> 172.21.15.33:0/1568894225 conn(0x7f2b6806e0f0 msgr2=0x7f2b68157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 shutdown_connections 2023-10-06T05:27:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.104+0000 7f2b6c99d700 1 -- 172.21.15.33:0/1568894225 wait complete. 2023-10-06T05:27:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 139 2023-10-06T05:27:36.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.309+0000 7ff76ef9d700 1 Processor -- start 2023-10-06T05:27:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.310+0000 7ff76ef9d700 1 -- start start 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff76813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76ef9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680c9cb0 con 0x7ff76813faa0 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76ef9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680ca270 con 0x7ff76806aad0 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76806b660 con 0x7ff7680c8050 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76df9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff76813faa0 0x7ff7680c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50292/0 (socket says 172.21.15.33:50292) 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76df9b700 1 -- 172.21.15.33:0/3232690151 learned_addr learned my addr 172.21.15.33:0/3232690151 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.311+0000 7ff76d79a700 1 --2- 172.21.15.33:0/3232690151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff76813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340019725 0 0) 0x7ff7680c9cb0 con 0x7ff76813faa0 2023-10-06T05:27:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff754003570 con 0x7ff76813faa0 2023-10-06T05:27:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179636564 0 0) 0x7ff7680ca270 con 0x7ff76806aad0 2023-10-06T05:27:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7680c9cb0 con 0x7ff76806aad0 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76d79a700 1 -- 172.21.15.33:0/3232690151 >> v1:172.21.15.167:6789/0 conn(0x7ff76806aad0 legacy=0x7ff76806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76d79a700 1 -- 172.21.15.33:0/3232690151 >> v1:172.21.15.33:6789/0 conn(0x7ff76813faa0 legacy=0x7ff7680c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76d79a700 1 -- 172.21.15.33:0/3232690151 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76806b840 con 0x7ff7680c8050 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76d79a700 1 --2- 172.21.15.33:0/3232690151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff76813fe70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff758009910 tx=0x7ff7580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e50283427d143c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.312+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011630 con 0x7ff7680c8050 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758011790 con 0x7ff7680c8050 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76cf99700 1 -- 172.21.15.33:0/3232690151 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011b30 con 0x7ff7680c8050 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3232690151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 msgr2=0x7ff76813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3232690151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff76813fe70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff758009910 tx=0x7ff7580091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3232690151 shutdown_connections 2023-10-06T05:27:36.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3232690151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff76813fe70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3232690151 >> 172.21.15.33:0/3232690151 conn(0x7ff76806d030 msgr2=0x7ff76806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:36.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3232690151 shutdown_connections 2023-10-06T05:27:36.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.313+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3232690151 wait complete. 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.315+0000 7ff76ef9d700 1 Processor -- start 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.315+0000 7ff76ef9d700 1 -- start start 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76ef9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 0x7ff76813b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff768160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76ef9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76813faa0 0x7ff7680b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76ef9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76813c060 con 0x7ff76806aad0 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76ef9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76813c340 con 0x7ff76813faa0 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76813c620 con 0x7ff7680c8050 2023-10-06T05:27:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76df9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 0x7ff76813b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76df9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 0x7ff76813b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41648/0 (socket says 172.21.15.33:41648) 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76df9b700 1 -- 172.21.15.33:0/3798868053 learned_addr learned my addr 172.21.15.33:0/3798868053 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76d79a700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff768160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.316+0000 7ff76e79c700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76813faa0 0x7ff7680b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76df9b700 1 -- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 msgr2=0x7ff768160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76df9b700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff768160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76df9b700 1 -- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76813faa0 msgr2=0x7ff7680b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76df9b700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76813faa0 0x7ff7680b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76df9b700 1 -- 172.21.15.33:0/3798868053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758009420 con 0x7ff76806aad0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76e79c700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76813faa0 0x7ff7680b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76df9b700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 0x7ff76813b9a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff76400c790 tx=0x7ff764007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764005b30 con 0x7ff76806aad0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff764005c90 con 0x7ff76806aad0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640072e0 con 0x7ff76806aad0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76813c900 con 0x7ff76806aad0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.317+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7680b10c0 con 0x7ff76806aad0 2023-10-06T05:27:36.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.318+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76804c380 con 0x7ff76806aad0 2023-10-06T05:27:36.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.319+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff764007690 con 0x7ff76806aad0 2023-10-06T05:27:36.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.320+0000 7ff75dffb700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff74c066f30 0x7ff74c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.320+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff764083c10 con 0x7ff76806aad0 2023-10-06T05:27:36.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.320+0000 7ff76d79a700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff74c066f30 0x7ff74c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.321+0000 7ff76d79a700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff74c066f30 0x7ff74c0693e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff758009860 tx=0x7ff758017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:36.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.324+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff76404b5a0 con 0x7ff76806aad0 2023-10-06T05:27:36.420 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:27:36.419+0000 7f260a1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.529+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff76806bb20 con 0x7ff76806aad0 2023-10-06T05:27:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.530+0000 7ff75dffb700 1 -- 172.21.15.33:0/3798868053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17069 (secure 0 0 0) 0x7ff764017020 con 0x7ff76806aad0 2023-10-06T05:27:36.532 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":183,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:30.873128+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:05.629452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:06.434007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1699314688":"2023-10-06T06:27:30.190772+0000","172.21.15.33:6835/950449971":"2023-10-07T05:27:20.407691+0000","172.21.15.33:6833/950449971":"2023-10-07T05:27:20.407691+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.532+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff74c066f30 msgr2=0x7ff74c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.532+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff74c066f30 0x7ff74c0693e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff758009860 tx=0x7ff758017000 comp rx=0 tx=0).stop 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.532+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 msgr2=0x7ff76813b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.532+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 0x7ff76813b9a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff76400c790 tx=0x7ff764007ea0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 shutdown_connections 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff74c066f30 0x7ff74c0693e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff76813faa0 0x7ff7680b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff76806aad0 0x7ff76813b9a0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 --2- 172.21.15.33:0/3798868053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7680c8050 0x7ff768160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 >> 172.21.15.33:0/3798868053 conn(0x7ff76806d030 msgr2=0x7ff7680c1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 shutdown_connections 2023-10-06T05:27:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.533+0000 7ff76ef9d700 1 -- 172.21.15.33:0/3798868053 wait complete. 2023-10-06T05:27:36.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:27:05.629452+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:27:06.434007+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:27:36.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:27:36.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.741+0000 7f659a59d700 1 Processor -- start 2023-10-06T05:27:36.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.741+0000 7f659a59d700 1 -- start start 2023-10-06T05:27:36.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.742+0000 7f659a59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6594156370 0x7f65940af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.742+0000 7f659a59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940ad430 con 0x7f65940cc7e0 2023-10-06T05:27:36.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.742+0000 7f659a59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940afe20 con 0x7f65940b01f0 2023-10-06T05:27:36.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.742+0000 7f659a59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659406bc70 con 0x7f6594156370 2023-10-06T05:27:36.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f659959b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6594156370 0x7f65940af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f6598d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f65940cc7e0 0x7f65940c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50294/0 (socket says 172.21.15.33:50294) 2023-10-06T05:27:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f6598d9a700 1 -- 172.21.15.33:0/4260481633 learned_addr learned my addr 172.21.15.33:0/4260481633 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f659959b700 1 -- 172.21.15.33:0/4260481633 >> v1:172.21.15.167:6789/0 conn(0x7f65940b01f0 legacy=0x7f65940ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f659959b700 1 -- 172.21.15.33:0/4260481633 >> v1:172.21.15.33:6789/0 conn(0x7f65940cc7e0 legacy=0x7f65940c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f659959b700 1 -- 172.21.15.33:0/4260481633 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940cccc0 con 0x7f6594156370 2023-10-06T05:27:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f658bfff700 1 -- 172.21.15.33:0/4260481633 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638970401 0 0) 0x7f65940ad430 con 0x7f65940cc7e0 2023-10-06T05:27:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.743+0000 7f659959b700 1 --2- 172.21.15.33:0/4260481633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6594156370 0x7f65940af8e0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f65900098e0 tx=0x7f65900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f337ebf76beda7a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f658bfff700 1 -- 172.21.15.33:0/4260481633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590009e80 con 0x7f6594156370 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f658bfff700 1 -- 172.21.15.33:0/4260481633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659000b820 con 0x7f6594156370 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f658bfff700 1 -- 172.21.15.33:0/4260481633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000bba0 con 0x7f6594156370 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 -- 172.21.15.33:0/4260481633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6594156370 msgr2=0x7f65940af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 --2- 172.21.15.33:0/4260481633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6594156370 0x7f65940af8e0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f65900098e0 tx=0x7f65900091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 -- 172.21.15.33:0/4260481633 shutdown_connections 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 --2- 172.21.15.33:0/4260481633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6594156370 0x7f65940af8e0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 -- 172.21.15.33:0/4260481633 >> 172.21.15.33:0/4260481633 conn(0x7f659406d030 msgr2=0x7f65940cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 -- 172.21.15.33:0/4260481633 shutdown_connections 2023-10-06T05:27:36.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.744+0000 7f659a59d700 1 -- 172.21.15.33:0/4260481633 wait complete. 2023-10-06T05:27:36.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.746+0000 7f659a59d700 1 Processor -- start 2023-10-06T05:27:36.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.746+0000 7f659a59d700 1 -- start start 2023-10-06T05:27:36.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940b01f0 0x7f6594151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659a59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 0x7f65940ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659a59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6594156370 0x7f65940ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659a59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594154350 con 0x7f65940cc7e0 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659a59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594153740 con 0x7f6594156370 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6594153a20 con 0x7f65940b01f0 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f6598d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 0x7f65940ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f6598d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 0x7f65940ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41664/0 (socket says 172.21.15.33:41664) 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f6598d9a700 1 -- 172.21.15.33:0/288668233 learned_addr learned my addr 172.21.15.33:0/288668233 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f659959b700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940b01f0 0x7f6594151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.747+0000 7f6599d9c700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6594156370 0x7f65940ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f6598d9a700 1 -- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940b01f0 msgr2=0x7f6594151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f6598d9a700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940b01f0 0x7f6594151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f6598d9a700 1 -- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6594156370 msgr2=0x7f65940ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f6598d9a700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6594156370 0x7f65940ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f6598d9a700 1 -- 172.21.15.33:0/288668233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009420 con 0x7f65940cc7e0 2023-10-06T05:27:36.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f6598d9a700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 0x7f65940ad8c0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f658400c7e0 tx=0x7f6584007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584005b30 con 0x7f65940cc7e0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6584005c90 con 0x7f65940cc7e0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584007300 con 0x7f65940cc7e0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6594153d00 con 0x7f65940cc7e0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.748+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6594152440 con 0x7f65940cc7e0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.749+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594154500 con 0x7f65940cc7e0 2023-10-06T05:27:36.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.750+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f658401b400 con 0x7f65940cc7e0 2023-10-06T05:27:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.751+0000 7f65897fa700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.751+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f65840842f0 con 0x7f65940cc7e0 2023-10-06T05:27:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.751+0000 7f659959b700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:36.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.752+0000 7f659959b700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f65900030c0 tx=0x7f6590016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:36.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.755+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6594154500 con 0x7f65940cc7e0 2023-10-06T05:27:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.959+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f659404c380 con 0x7f65940cc7e0 2023-10-06T05:27:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.959+0000 7f65897fa700 1 -- 172.21.15.33:0/288668233 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6584054df0 con 0x7f65940cc7e0 2023-10-06T05:27:36.961 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:27:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.961+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 msgr2=0x7f65780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.961+0000 7f659a59d700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f65900030c0 tx=0x7f6590016000 comp rx=0 tx=0).stop 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.961+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 msgr2=0x7f65940ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.961+0000 7f659a59d700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 0x7f65940ad8c0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f658400c7e0 tx=0x7f6584007df0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 shutdown_connections 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6578066f30 0x7f65780693e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6594156370 0x7f65940ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65940b01f0 0x7f6594151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 --2- 172.21.15.33:0/288668233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65940cc7e0 0x7f65940ad8c0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 >> 172.21.15.33:0/288668233 conn(0x7f659406d030 msgr2=0x7f659406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 shutdown_connections 2023-10-06T05:27:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:36.962+0000 7f659a59d700 1 -- 172.21.15.33:0/288668233 wait complete. 2023-10-06T05:27:36.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:27:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.169+0000 7f9ddbfff700 1 Processor -- start 2023-10-06T05:27:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.170+0000 7f9ddbfff700 1 -- start start 2023-10-06T05:27:37.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.171+0000 7f9ddbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 0x7f9ddc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.171+0000 7f9ddbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0ad430 con 0x7f9ddc0b01f0 2023-10-06T05:27:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.171+0000 7f9ddbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0afe20 con 0x7f9ddc0cc7e0 2023-10-06T05:27:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.171+0000 7f9ddbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc06bc70 con 0x7f9ddc156370 2023-10-06T05:27:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.171+0000 7f9ddb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ddc0b01f0 0x7f9ddc0ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50304/0 (socket says 172.21.15.33:50304) 2023-10-06T05:27:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.171+0000 7f9ddb7fe700 1 -- 172.21.15.33:0/1447613479 learned_addr learned my addr 172.21.15.33:0/1447613479 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378196409 0 0) 0x7f9ddc0ad430 con 0x7f9ddc0b01f0 2023-10-06T05:27:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc4003570 con 0x7f9ddc0b01f0 2023-10-06T05:27:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 0x7f9ddc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3790543130 0 0) 0x7f9dc4003570 con 0x7f9ddc0b01f0 2023-10-06T05:27:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 msgr2=0x7f9ddc0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 0x7f9ddc0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 >> v1:172.21.15.167:6789/0 conn(0x7f9ddc0cc7e0 legacy=0x7f9ddc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0cccc0 con 0x7f9ddc0b01f0 2023-10-06T05:27:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9dd40017f0 con 0x7f9ddc0b01f0 2023-10-06T05:27:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 >> v1:172.21.15.33:6789/0 conn(0x7f9ddc0b01f0 legacy=0x7f9ddc0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4003a90 0x7f9dc4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 0x7f9dc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004e70 0x7f9dc40052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc06bc70 con 0x7f9dc4003a90 2023-10-06T05:27:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc400bc70 con 0x7f9dc4004e70 2023-10-06T05:27:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc400be30 con 0x7f9ddc156370 2023-10-06T05:27:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4003a90 0x7f9dc4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.172+0000 7f9ddb7fe700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004e70 0x7f9dc40052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9dda7fc700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 0x7f9dc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9ddaffd700 1 -- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 msgr2=0x7f9dc4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ddc156370 0x7f9dc4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9ddaffd700 1 -- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004e70 msgr2=0x7f9dc40052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc4004e70 0x7f9dc40052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9ddaffd700 1 -- 172.21.15.33:0/1447613479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4003570 con 0x7f9dc4003a90 2023-10-06T05:27:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4003a90 0x7f9dc4003ea0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00bb60 tx=0x7f9dcc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9e849ac3afbff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00d5d0 con 0x7f9dc4003a90 2023-10-06T05:27:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc00d730 con 0x7f9dc4003a90 2023-10-06T05:27:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.173+0000 7f9dd9ffb700 1 -- 172.21.15.33:0/1447613479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00daf0 con 0x7f9dc4003a90 2023-10-06T05:27:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddbfff700 1 -- 172.21.15.33:0/1447613479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4003a90 msgr2=0x7f9dc4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/1447613479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4003a90 0x7f9dc4003ea0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00bb60 tx=0x7f9dcc008040 comp rx=0 tx=0).stop 2023-10-06T05:27:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddb7fe700 1 -- 172.21.15.33:0/1447613479 reap_dead start 2023-10-06T05:27:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddbfff700 1 -- 172.21.15.33:0/1447613479 shutdown_connections 2023-10-06T05:27:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddbfff700 1 -- 172.21.15.33:0/1447613479 >> 172.21.15.33:0/1447613479 conn(0x7f9ddc06d030 msgr2=0x7f9ddc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddbfff700 1 -- 172.21.15.33:0/1447613479 shutdown_connections 2023-10-06T05:27:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.174+0000 7f9ddbfff700 1 -- 172.21.15.33:0/1447613479 wait complete. 2023-10-06T05:27:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.175+0000 7f9ddbfff700 1 Processor -- start 2023-10-06T05:27:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.175+0000 7f9ddbfff700 1 -- start start 2023-10-06T05:27:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.176+0000 7f9ddbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003a90 0x7f9ddc14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.176+0000 7f9ddbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ddc0aaa10 0x7f9ddc0aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.176+0000 7f9ddbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 0x7f9ddc152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.176+0000 7f9ddbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc151f40 con 0x7f9ddc152b50 2023-10-06T05:27:37.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.176+0000 7f9ddbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc154b90 con 0x7f9ddc0aaa10 2023-10-06T05:27:37.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.176+0000 7f9ddbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0ab3c0 con 0x7f9dc4003a90 2023-10-06T05:27:37.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 0x7f9ddc152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 0x7f9ddc152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41674/0 (socket says 172.21.15.33:41674) 2023-10-06T05:27:37.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 -- 172.21.15.33:0/683068563 learned_addr learned my addr 172.21.15.33:0/683068563 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:37.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003a90 0x7f9ddc14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9dda7fc700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ddc0aaa10 0x7f9ddc0aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 -- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003a90 msgr2=0x7f9ddc14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003a90 0x7f9ddc14f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 -- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ddc0aaa10 msgr2=0x7f9ddc0aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ddc0aaa10 0x7f9ddc0aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 -- 172.21.15.33:0/683068563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc151ab0 con 0x7f9ddc152b50 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.177+0000 7f9ddb7fe700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 0x7f9ddc152fd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400bfc0 tx=0x7f9dd4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.178+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400e8a0 con 0x7f9ddc152b50 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.178+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd400ea40 con 0x7f9ddc152b50 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.178+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc150ea0 con 0x7f9ddc152b50 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.178+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400ee00 con 0x7f9ddc152b50 2023-10-06T05:27:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.178+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc151460 con 0x7f9ddc152b50 2023-10-06T05:27:37.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.179+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc151c40 con 0x7f9ddc152b50 2023-10-06T05:27:37.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.180+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9dd4013630 con 0x7f9ddc152b50 2023-10-06T05:27:37.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.180+0000 7f9dc2ffd700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9dc8066f30 0x7f9dc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.180+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9dd4084a50 con 0x7f9ddc152b50 2023-10-06T05:27:37.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.181+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9dc8066f30 0x7f9dc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.182+0000 7f9ddaffd700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9dc8066f30 0x7f9dc80693e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00bb60 tx=0x7f9dcc01b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:37.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.184+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dd4051a40 con 0x7f9ddc152b50 2023-10-06T05:27:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.388+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ddc04c380 con 0x7f9ddc152b50 2023-10-06T05:27:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.389+0000 7f9dc2ffd700 1 -- 172.21.15.33:0/683068563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v183) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9dd4055060 con 0x7f9ddc152b50 2023-10-06T05:27:37.391 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:27:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9dc8066f30 msgr2=0x7f9dc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9dc8066f30 0x7f9dc80693e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00bb60 tx=0x7f9dcc01b000 comp rx=0 tx=0).stop 2023-10-06T05:27:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 msgr2=0x7f9ddc152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 0x7f9ddc152fd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400bfc0 tx=0x7f9dd4000f40 comp rx=0 tx=0).stop 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 shutdown_connections 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9dc8066f30 0x7f9dc80693e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ddc152b50 0x7f9ddc152fd0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc4003a90 0x7f9ddc14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.391+0000 7f9ddbfff700 1 --2- 172.21.15.33:0/683068563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ddc0aaa10 0x7f9ddc0aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.392+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 >> 172.21.15.33:0/683068563 conn(0x7f9ddc06d030 msgr2=0x7f9ddc06a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.392+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 shutdown_connections 2023-10-06T05:27:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.392+0000 7f9ddbfff700 1 -- 172.21.15.33:0/683068563 wait complete. 2023-10-06T05:27:37.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:27:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.598+0000 7f1f11d9d700 1 Processor -- start 2023-10-06T05:27:37.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.599+0000 7f1f11d9d700 1 -- start start 2023-10-06T05:27:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f11d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f11d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f0c0cc7e0 0x7f1f0c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f11d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1f0c0b01f0 0x7f1f0c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f11d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0ad430 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f11d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0afe20 con 0x7f1f0c0b01f0 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f11d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0b0b70 con 0x7f1f0c156370 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f0bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f0c0cc7e0 0x7f1f0c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f0bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f0c0cc7e0 0x7f1f0c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41678/0 (socket says 172.21.15.33:41678) 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f0bfff700 1 -- 172.21.15.33:0/3746113675 learned_addr learned my addr 172.21.15.33:0/3746113675 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.600+0000 7f1f10d9b700 1 --2- 172.21.15.33:0/3746113675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f1159c700 1 --2- 172.21.15.33:0/3746113675 >> v2:172.21.15.167:3300/0 conn(0x7f1f0c0b01f0 0x7f1f0c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0bfff700 1 -- 172.21.15.33:0/3746113675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 msgr2=0x7f1f0c0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0bfff700 1 --2- 172.21.15.33:0/3746113675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0bfff700 1 -- 172.21.15.33:0/3746113675 >> v2:172.21.15.167:3300/0 conn(0x7f1f0c0b01f0 msgr2=0x7f1f0c0b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0bfff700 1 --2- 172.21.15.33:0/3746113675 >> v2:172.21.15.167:3300/0 conn(0x7f1f0c0b01f0 0x7f1f0c0b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0bfff700 1 -- 172.21.15.33:0/3746113675 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c06bc70 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0bfff700 1 --2- 172.21.15.33:0/3746113675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0c8db0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c069810 tx=0x7f1ef8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a9da01abab89b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0b7fe700 1 -- 172.21.15.33:0/3746113675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8009e70 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0b7fe700 1 -- 172.21.15.33:0/3746113675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800b860 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.601+0000 7f1f0b7fe700 1 -- 172.21.15.33:0/3746113675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800bc00 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 -- 172.21.15.33:0/3746113675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 msgr2=0x7f1f0c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/3746113675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0c8db0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c069810 tx=0x7f1ef8009250 comp rx=0 tx=0).stop 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 -- 172.21.15.33:0/3746113675 shutdown_connections 2023-10-06T05:27:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/3746113675 >> v2:172.21.15.167:3300/0 conn(0x7f1f0c0b01f0 0x7f1f0c0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/3746113675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/3746113675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0c8db0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 -- 172.21.15.33:0/3746113675 >> 172.21.15.33:0/3746113675 conn(0x7f1f0c06d030 msgr2=0x7f1f0c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 -- 172.21.15.33:0/3746113675 shutdown_connections 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.602+0000 7f1f11d9d700 1 -- 172.21.15.33:0/3746113675 wait complete. 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.603+0000 7f1f11d9d700 1 Processor -- start 2023-10-06T05:27:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 -- start start 2023-10-06T05:27:37.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f0c0b01f0 0x7f1f0c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0a68e0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1507c0 con 0x7f1f0c0b01f0 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.604+0000 7f1f11d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0a5cd0 con 0x7f1f0c156370 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0a8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41692/0 (socket says 172.21.15.33:41692) 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 -- 172.21.15.33:0/1280252988 learned_addr learned my addr 172.21.15.33:0/1280252988 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f1159c700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f10d9b700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f0c0b01f0 0x7f1f0c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 -- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 msgr2=0x7f1f0c0a6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0a6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 -- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f0c0b01f0 msgr2=0x7f1f0c0a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f0c0b01f0 0x7f1f0c0a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 -- 172.21.15.33:0/1280252988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80094f0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.605+0000 7f1f0bfff700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0a8c20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c06d4d0 tx=0x7f1ef8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.606+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8000d10 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.606+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800bbe0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.606+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011670 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.606+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c0a5fb0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.606+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c0a6570 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.607+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c04c380 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.608+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ef8011aa0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.608+0000 7f1f08ff9700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ef0066f30 0x7f1ef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.609+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1ef8085d00 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.609+0000 7f1f10d9b700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ef0066f30 0x7f1ef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:37.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.610+0000 7f1f10d9b700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ef0066f30 0x7f1ef00693e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ca0 tx=0x7f1f00008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.613+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ef8052cf0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.813+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f0c0cc5c0 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.814+0000 7f1f08ff9700 1 -- 172.21.15.33:0/1280252988 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ef8056310 con 0x7f1f0c0cc7e0 2023-10-06T05:27:37.818 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ef0066f30 msgr2=0x7f1ef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ef0066f30 0x7f1ef00693e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ca0 tx=0x7f1f00008040 comp rx=0 tx=0).stop 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 msgr2=0x7f1f0c0a8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0a8c20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c06d4d0 tx=0x7f1ef8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 shutdown_connections 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ef0066f30 0x7f1ef00693e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1f0c156370 0x7f1f0c0a6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f0c0b01f0 0x7f1f0c0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 --2- 172.21.15.33:0/1280252988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f0c0cc7e0 0x7f1f0c0a8c20 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 >> 172.21.15.33:0/1280252988 conn(0x7f1f0c06d030 msgr2=0x7f1f0c150d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 shutdown_connections 2023-10-06T05:27:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:37.816+0000 7f1f11d9d700 1 -- 172.21.15.33:0/1280252988 wait complete. 2023-10-06T05:27:37.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:27:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.025+0000 7fce26d9d700 1 Processor -- start 2023-10-06T05:27:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.025+0000 7fce26d9d700 1 -- start start 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.026+0000 7fce26d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 0x7fce200af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.026+0000 7fce26d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce200ce890 con 0x7fce200af540 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.026+0000 7fce26d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2015aad0 con 0x7fce2015a650 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.026+0000 7fce26d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2015d890 con 0x7fce200cd5e0 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce25d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fce200af540 0x7fce200cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:50308/0 (socket says 172.21.15.33:50308) 2023-10-06T05:27:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce25d9b700 1 -- 172.21.15.33:0/2860578649 learned_addr learned my addr 172.21.15.33:0/2860578649 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49498645 0 0) 0x7fce200ce890 con 0x7fce200af540 2023-10-06T05:27:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce2559a700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 0x7fce200af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce0c003570 con 0x7fce200af540 2023-10-06T05:27:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289570437 0 0) 0x7fce0c003570 con 0x7fce200af540 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 msgr2=0x7fce200af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 0x7fce200af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 >> v1:172.21.15.167:6789/0 conn(0x7fce2015a650 legacy=0x7fce200ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2015db70 con 0x7fce200af540 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fce1c001470 con 0x7fce200af540 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.027+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 >> v1:172.21.15.33:6789/0 conn(0x7fce200af540 legacy=0x7fce200cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce24d99700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c003a90 0x7fce0c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce24d99700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce0c004810 0x7fce0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce24d99700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 0x7fce0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2015d890 con 0x7fce0c003a90 2023-10-06T05:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c00bb40 con 0x7fce0c004810 2023-10-06T05:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c00bd00 con 0x7fce200cd5e0 2023-10-06T05:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce25d9b700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c003a90 0x7fce0c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce2559a700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce0c004810 0x7fce0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce2659c700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 0x7fce0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce25d9b700 1 -- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 msgr2=0x7fce0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce25d9b700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce200cd5e0 0x7fce0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce25d9b700 1 -- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce0c004810 msgr2=0x7fce0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce25d9b700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce0c004810 0x7fce0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.028+0000 7fce25d9b700 1 -- 172.21.15.33:0/2860578649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c003570 con 0x7fce0c003a90 2023-10-06T05:27:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce25d9b700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c003a90 0x7fce0c003e60 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00c2a0 tx=0x7fce200ce800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d22da1b01ee72b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00cb70 con 0x7fce0c003a90 2023-10-06T05:27:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00ccd0 con 0x7fce0c003a90 2023-10-06T05:27:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce24d99700 1 -- 172.21.15.33:0/2860578649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c005b70 con 0x7fce0c003a90 2023-10-06T05:27:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce26d9d700 1 -- 172.21.15.33:0/2860578649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c003a90 msgr2=0x7fce0c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce26d9d700 1 --2- 172.21.15.33:0/2860578649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c003a90 0x7fce0c003e60 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00c2a0 tx=0x7fce200ce800 comp rx=0 tx=0).stop 2023-10-06T05:27:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce2659c700 1 -- 172.21.15.33:0/2860578649 reap_dead start 2023-10-06T05:27:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.029+0000 7fce26d9d700 1 -- 172.21.15.33:0/2860578649 shutdown_connections 2023-10-06T05:27:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.030+0000 7fce26d9d700 1 -- 172.21.15.33:0/2860578649 >> 172.21.15.33:0/2860578649 conn(0x7fce2006d9b0 msgr2=0x7fce2006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.030+0000 7fce26d9d700 1 -- 172.21.15.33:0/2860578649 shutdown_connections 2023-10-06T05:27:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.030+0000 7fce26d9d700 1 -- 172.21.15.33:0/2860578649 wait complete. 2023-10-06T05:27:38.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.031+0000 7fce26d9d700 1 Processor -- start 2023-10-06T05:27:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.031+0000 7fce26d9d700 1 -- start start 2023-10-06T05:27:38.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce26d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce0c003a90 0x7fce201572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce26d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce201448c0 0x7fce200c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce26d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 0x7fce200caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce26d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce200cb470 con 0x7fce200caac0 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce26d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce201584e0 con 0x7fce201448c0 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce26d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce201578d0 con 0x7fce0c003a90 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 0x7fce200caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 0x7fce200caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41712/0 (socket says 172.21.15.33:41712) 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce2659c700 1 -- 172.21.15.33:0/399055388 learned_addr learned my addr 172.21.15.33:0/399055388 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce25d9b700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce0c003a90 0x7fce201572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.032+0000 7fce2559a700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce201448c0 0x7fce200c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2659c700 1 -- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce0c003a90 msgr2=0x7fce201572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2659c700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce0c003a90 0x7fce201572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2659c700 1 -- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce201448c0 msgr2=0x7fce200c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2659c700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce201448c0 0x7fce200c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2659c700 1 -- 172.21.15.33:0/399055388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20157bb0 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2559a700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce201448c0 0x7fce200c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce2659c700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 0x7fce200caf30 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fce200caa90 tx=0x7fce180016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1800c7b0 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1800c930 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce20157e90 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1800ccf0 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.033+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce20144d30 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.035+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce20158650 con 0x7fce200caac0 2023-10-06T05:27:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.035+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce18015640 con 0x7fce200caac0 2023-10-06T05:27:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.036+0000 7fce15ffb700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fce04066f30 0x7fce040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.036+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce18084cd0 con 0x7fce200caac0 2023-10-06T05:27:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.037+0000 7fce25d9b700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fce04066f30 0x7fce040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.038+0000 7fce25d9b700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fce04066f30 0x7fce040693e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fce200b64a0 tx=0x7fce1c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.040+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce18051d60 con 0x7fce200caac0 2023-10-06T05:27:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.243+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce2004c380 con 0x7fce200caac0 2023-10-06T05:27:38.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.244+0000 7fce15ffb700 1 -- 172.21.15.33:0/399055388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce18004120 con 0x7fce200caac0 2023-10-06T05:27:38.246 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:27:38.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fce04066f30 msgr2=0x7fce040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fce04066f30 0x7fce040693e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fce200b64a0 tx=0x7fce1c015000 comp rx=0 tx=0).stop 2023-10-06T05:27:38.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 msgr2=0x7fce200caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 0x7fce200caf30 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fce200caa90 tx=0x7fce180016d0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 shutdown_connections 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fce04066f30 0x7fce040693e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce200caac0 0x7fce200caf30 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce0c003a90 0x7fce201572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 --2- 172.21.15.33:0/399055388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce201448c0 0x7fce200c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 >> 172.21.15.33:0/399055388 conn(0x7fce2006d9b0 msgr2=0x7fce200b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 shutdown_connections 2023-10-06T05:27:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.246+0000 7fce26d9d700 1 -- 172.21.15.33:0/399055388 wait complete. 2023-10-06T05:27:38.265 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:38.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.451+0000 7fe4c359e700 1 Processor -- start 2023-10-06T05:27:38.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.452+0000 7fe4c359e700 1 -- start start 2023-10-06T05:27:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4c413faa0 0x7fe4c413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c40c9cb0 con 0x7fe4c406aad0 2023-10-06T05:27:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c406b660 con 0x7fe4c413faa0 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40ca270 con 0x7fe4c40c7940 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4c413faa0 0x7fe4c413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe4c413faa0 0x7fe4c413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41728/0 (socket says 172.21.15.33:41728) 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c259c700 1 -- 172.21.15.33:0/3249889105 learned_addr learned my addr 172.21.15.33:0/3249889105 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.453+0000 7fe4c1d9b700 1 --2- 172.21.15.33:0/3249889105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c259c700 1 -- 172.21.15.33:0/3249889105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 msgr2=0x7fe4c40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c259c700 1 --2- 172.21.15.33:0/3249889105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c40c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c259c700 1 -- 172.21.15.33:0/3249889105 >> v1:172.21.15.167:6789/0 conn(0x7fe4c406aad0 legacy=0x7fe4c406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c259c700 1 -- 172.21.15.33:0/3249889105 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c406b840 con 0x7fe4c413faa0 2023-10-06T05:27:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c259c700 1 --2- 172.21.15.33:0/3249889105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c413faa0 0x7fe4c413fe70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40099b0 tx=0x7fe4b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6d80db369958cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c159a700 1 -- 172.21.15.33:0/3249889105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4009ea0 con 0x7fe4c413faa0 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c159a700 1 -- 172.21.15.33:0/3249889105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b400b860 con 0x7fe4c413faa0 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.454+0000 7fe4c159a700 1 -- 172.21.15.33:0/3249889105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400bc00 con 0x7fe4c413faa0 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 -- 172.21.15.33:0/3249889105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c413faa0 msgr2=0x7fe4c413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3249889105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c413faa0 0x7fe4c413fe70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40099b0 tx=0x7fe4b4009250 comp rx=0 tx=0).stop 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 -- 172.21.15.33:0/3249889105 shutdown_connections 2023-10-06T05:27:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3249889105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c413faa0 0x7fe4c413fe70 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3249889105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 -- 172.21.15.33:0/3249889105 >> 172.21.15.33:0/3249889105 conn(0x7fe4c406d030 msgr2=0x7fe4c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 -- 172.21.15.33:0/3249889105 shutdown_connections 2023-10-06T05:27:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.455+0000 7fe4c359e700 1 -- 172.21.15.33:0/3249889105 wait complete. 2023-10-06T05:27:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.456+0000 7fe4c359e700 1 Processor -- start 2023-10-06T05:27:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.456+0000 7fe4c359e700 1 -- start start 2023-10-06T05:27:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 0x7fe4c4157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c413faa0 0x7fe4c4158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4158060 con 0x7fe4c406aad0 2023-10-06T05:27:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40ae510 con 0x7fe4c413faa0 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c415dcd0 con 0x7fe4c40c7940 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.457+0000 7fe4c259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 0x7fe4c4157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 0x7fe4c4157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41730/0 (socket says 172.21.15.33:41730) 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 -- 172.21.15.33:0/3657548331 learned_addr learned my addr 172.21.15.33:0/3657548331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c1d9b700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c413ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c2d9d700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c413faa0 0x7fe4c4158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 -- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 msgr2=0x7fe4c413ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c413ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 -- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c413faa0 msgr2=0x7fe4c4158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c413faa0 0x7fe4c4158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 -- 172.21.15.33:0/3657548331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b40094f0 con 0x7fe4c406aad0 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c2d9d700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c413faa0 0x7fe4c4158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c259c700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 0x7fe4c4157b20 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fd0 tx=0x7fe4b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400bbe0 con 0x7fe4c406aad0 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4011420 con 0x7fe4c406aad0 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40117c0 con 0x7fe4c406aad0 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.458+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c4158270 con 0x7fe4c406aad0 2023-10-06T05:27:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.459+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c415d680 con 0x7fe4c406aad0 2023-10-06T05:27:38.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.460+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c404c380 con 0x7fe4c406aad0 2023-10-06T05:27:38.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.460+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4b4011bf0 con 0x7fe4c406aad0 2023-10-06T05:27:38.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.461+0000 7fe4b27fc700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe4a4066f30 0x7fe4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.461+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe4b4085fd0 con 0x7fe4c406aad0 2023-10-06T05:27:38.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.462+0000 7fe4c1d9b700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe4a4066f30 0x7fe4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.462+0000 7fe4c1d9b700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe4a4066f30 0x7fe4a40693e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40ca900 tx=0x7fe4b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.465+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4b4052fc0 con 0x7fe4c406aad0 2023-10-06T05:27:38.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.715+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4c406bb20 con 0x7fe4c406aad0 2023-10-06T05:27:38.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.716+0000 7fe4b27fc700 1 -- 172.21.15.33:0/3657548331 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fe4b40565e0 con 0x7fe4c406aad0 2023-10-06T05:27:38.718 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:38.718 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":139,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:32.888818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":6,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.718+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe4a4066f30 msgr2=0x7fe4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.718+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe4a4066f30 0x7fe4a40693e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40ca900 tx=0x7fe4b8008040 comp rx=0 tx=0).stop 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 msgr2=0x7fe4c4157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 0x7fe4c4157b20 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fd0 tx=0x7fe4b4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 shutdown_connections 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe4a4066f30 0x7fe4a40693e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4c413faa0 0x7fe4c4158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406aad0 0x7fe4c4157b20 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 --2- 172.21.15.33:0/3657548331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4c40c7940 0x7fe4c413ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 >> 172.21.15.33:0/3657548331 conn(0x7fe4c406d030 msgr2=0x7fe4c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 shutdown_connections 2023-10-06T05:27:38.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.719+0000 7fe4c359e700 1 -- 172.21.15.33:0/3657548331 wait complete. 2023-10-06T05:27:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 139 2023-10-06T05:27:38.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.924+0000 7fa12db9d700 1 Processor -- start 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.924+0000 7fa12db9d700 1 -- start start 2023-10-06T05:27:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.925+0000 7fa12db9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa128153b10 0x7fa1280c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.925+0000 7fa12db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280c6b30 0x7fa1280c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.925+0000 7fa12db9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa1280675f0 0x7fa128067a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.925+0000 7fa12db9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280c74f0 con 0x7fa128153b10 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.925+0000 7fa12db9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12815abd0 con 0x7fa1280675f0 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.925+0000 7fa12db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c9b00 con 0x7fa1280c6b30 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa128153b10 0x7fa1280c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa128153b10 0x7fa1280c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41746/0 (socket says 172.21.15.33:41746) 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 -- 172.21.15.33:0/846291592 learned_addr learned my addr 172.21.15.33:0/846291592 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12d39c700 1 --2- 172.21.15.33:0/846291592 >> v2:172.21.15.167:3300/0 conn(0x7fa1280675f0 0x7fa128067a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa127fff700 1 --2- 172.21.15.33:0/846291592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280c6b30 0x7fa1280c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 -- 172.21.15.33:0/846291592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280c6b30 msgr2=0x7fa1280c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 --2- 172.21.15.33:0/846291592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280c6b30 0x7fa1280c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 -- 172.21.15.33:0/846291592 >> v2:172.21.15.167:3300/0 conn(0x7fa1280675f0 msgr2=0x7fa128067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 --2- 172.21.15.33:0/846291592 >> v2:172.21.15.167:3300/0 conn(0x7fa1280675f0 0x7fa128067a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 -- 172.21.15.33:0/846291592 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12815a050 con 0x7fa128153b10 2023-10-06T05:27:38.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.926+0000 7fa12cb9b700 1 --2- 172.21.15.33:0/846291592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c95c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa128065a70 tx=0x7fa11c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6614fb7e3efaffd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa1277fe700 1 -- 172.21.15.33:0/846291592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00b860 con 0x7fa128153b10 2023-10-06T05:27:38.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa1277fe700 1 -- 172.21.15.33:0/846291592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00b9c0 con 0x7fa128153b10 2023-10-06T05:27:38.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa1277fe700 1 -- 172.21.15.33:0/846291592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00bd60 con 0x7fa128153b10 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 -- 172.21.15.33:0/846291592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 msgr2=0x7fa1280c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 --2- 172.21.15.33:0/846291592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c95c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa128065a70 tx=0x7fa11c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 -- 172.21.15.33:0/846291592 shutdown_connections 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 --2- 172.21.15.33:0/846291592 >> v2:172.21.15.167:3300/0 conn(0x7fa1280675f0 0x7fa128067a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 --2- 172.21.15.33:0/846291592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c95c0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 --2- 172.21.15.33:0/846291592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280c6b30 0x7fa1280c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 -- 172.21.15.33:0/846291592 >> 172.21.15.33:0/846291592 conn(0x7fa12806c560 msgr2=0x7fa12806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 -- 172.21.15.33:0/846291592 shutdown_connections 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.927+0000 7fa12db9d700 1 -- 172.21.15.33:0/846291592 wait complete. 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.929+0000 7fa12db9d700 1 Processor -- start 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.929+0000 7fa12db9d700 1 -- start start 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280675f0 0x7fa1280afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1280c6b30 0x7fa128152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280b0400 con 0x7fa128153b10 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280af8c0 con 0x7fa1280c6b30 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280af290 con 0x7fa1280675f0 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12d39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41748/0 (socket says 172.21.15.33:41748) 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12d39c700 1 -- 172.21.15.33:0/1968565927 learned_addr learned my addr 172.21.15.33:0/1968565927 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa12cb9b700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280675f0 0x7fa1280afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.930+0000 7fa127fff700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1280c6b30 0x7fa128152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12d39c700 1 -- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280675f0 msgr2=0x7fa1280afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12d39c700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280675f0 0x7fa1280afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12d39c700 1 -- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1280c6b30 msgr2=0x7fa128152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12d39c700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1280c6b30 0x7fa128152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12d39c700 1 -- 172.21.15.33:0/1968565927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0094f0 con 0x7fa128153b10 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12d39c700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c5ec0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fa120007f50 tx=0x7fa120009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000eb70 con 0x7fa128153b10 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000ecf0 con 0x7fa128153b10 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120014670 con 0x7fa128153b10 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280b0650 con 0x7fa128153b10 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.931+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1280af550 con 0x7fa128153b10 2023-10-06T05:27:38.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.932+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280afa50 con 0x7fa128153b10 2023-10-06T05:27:38.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.933+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1200149e0 con 0x7fa128153b10 2023-10-06T05:27:38.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.934+0000 7fa124ff9700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa10c066f60 0x7fa10c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.934+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa120084c90 con 0x7fa128153b10 2023-10-06T05:27:38.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.935+0000 7fa12cb9b700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa10c066f60 0x7fa10c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.935+0000 7fa12cb9b700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa10c066f60 0x7fa10c069410 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa128065a70 tx=0x7fa11c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:38.938+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa120051d20 con 0x7fa128153b10 2023-10-06T05:27:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.144+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1280aeaf0 con 0x7fa128153b10 2023-10-06T05:27:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.145+0000 7fa124ff9700 1 -- 172.21.15.33:0/1968565927 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17069 (secure 0 0 0) 0x7fa120055340 con 0x7fa128153b10 2023-10-06T05:27:39.147 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:39.147 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":183,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:30.873128+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:05.629452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:06.434007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1699314688":"2023-10-06T06:27:30.190772+0000","172.21.15.33:6835/950449971":"2023-10-07T05:27:20.407691+0000","172.21.15.33:6833/950449971":"2023-10-07T05:27:20.407691+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:39.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.147+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa10c066f60 msgr2=0x7fa10c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.147+0000 7fa12db9d700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa10c066f60 0x7fa10c069410 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa128065a70 tx=0x7fa11c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.147+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 msgr2=0x7fa1280c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.147+0000 7fa12db9d700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c5ec0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fa120007f50 tx=0x7fa120009860 comp rx=0 tx=0).stop 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 shutdown_connections 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa10c066f60 0x7fa10c069410 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa128153b10 0x7fa1280c5ec0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1280675f0 0x7fa1280afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 --2- 172.21.15.33:0/1968565927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1280c6b30 0x7fa128152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 >> 172.21.15.33:0/1968565927 conn(0x7fa12806c560 msgr2=0x7fa128153290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 shutdown_connections 2023-10-06T05:27:39.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.148+0000 7fa12db9d700 1 -- 172.21.15.33:0/1968565927 wait complete. 2023-10-06T05:27:39.167 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:27:39.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:39.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.353+0000 7ff05a19d700 1 Processor -- start 2023-10-06T05:27:39.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.353+0000 7ff05a19d700 1 -- start start 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff05406ab00 0x7ff0540b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b2e30 0x7ff05415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05a19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05415e2d0 con 0x7ff0540b0a40 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05406afa0 con 0x7ff05406ab00 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05415d6c0 con 0x7ff0540b2e30 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff05406ab00 0x7ff0540b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05899a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff05406ab00 0x7ff0540b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41764/0 (socket says 172.21.15.33:41764) 2023-10-06T05:27:39.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.354+0000 7ff05899a700 1 -- 172.21.15.33:0/3263576998 learned_addr learned my addr 172.21.15.33:0/3263576998 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:39.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05999c700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b2e30 0x7ff05415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05899a700 1 -- 172.21.15.33:0/3263576998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b2e30 msgr2=0x7ff05415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:39.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05899a700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b2e30 0x7ff05415a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05899a700 1 -- 172.21.15.33:0/3263576998 >> v1:172.21.15.167:6789/0 conn(0x7ff0540b0a40 legacy=0x7ff0540b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05899a700 1 -- 172.21.15.33:0/3263576998 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05415d9a0 con 0x7ff05406ab00 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05999c700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b2e30 0x7ff05415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff05899a700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff0540b28f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff0440099b0 tx=0x7ff044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4e1a5ec414074a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.355+0000 7ff04bfff700 1 -- 172.21.15.33:0/3263576998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400b860 con 0x7ff05406ab00 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff04bfff700 1 -- 172.21.15.33:0/3263576998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400b9c0 con 0x7ff05406ab00 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff04bfff700 1 -- 172.21.15.33:0/3263576998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400bd60 con 0x7ff05406ab00 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 -- 172.21.15.33:0/3263576998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 msgr2=0x7ff0540b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff0540b28f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff0440099b0 tx=0x7ff044009250 comp rx=0 tx=0).stop 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 -- 172.21.15.33:0/3263576998 shutdown_connections 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b2e30 0x7ff05415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 --2- 172.21.15.33:0/3263576998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff0540b28f0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 -- 172.21.15.33:0/3263576998 >> 172.21.15.33:0/3263576998 conn(0x7ff05406e0f0 msgr2=0x7ff0540675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:39.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 -- 172.21.15.33:0/3263576998 shutdown_connections 2023-10-06T05:27:39.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.356+0000 7ff05a19d700 1 -- 172.21.15.33:0/3263576998 wait complete. 2023-10-06T05:27:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.358+0000 7ff05a19d700 1 Processor -- start 2023-10-06T05:27:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.358+0000 7ff05a19d700 1 -- start start 2023-10-06T05:27:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff05415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b0a40 0x7ff0540c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0540b2e30 0x7ff054169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c4490 con 0x7ff05406ab00 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c35e0 con 0x7ff0540b2e30 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c3880 con 0x7ff0540b0a40 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff05415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff05415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41770/0 (socket says 172.21.15.33:41770) 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 -- 172.21.15.33:0/1702636824 learned_addr learned my addr 172.21.15.33:0/1702636824 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05899a700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b0a40 0x7ff0540c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 -- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b0a40 msgr2=0x7ff0540c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b0a40 0x7ff0540c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05999c700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0540b2e30 0x7ff054169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 -- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0540b2e30 msgr2=0x7ff054169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.359+0000 7ff05919b700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0540b2e30 0x7ff054169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff05919b700 1 -- 172.21.15.33:0/1702636824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440094f0 con 0x7ff05406ab00 2023-10-06T05:27:39.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff05999c700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0540b2e30 0x7ff054169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:39.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff05919b700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff05415c6a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c4e0 tx=0x7ff05000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050011590 con 0x7ff05406ab00 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff050011710 con 0x7ff05406ab00 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0540c3b60 con 0x7ff05406ab00 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050011ad0 con 0x7ff05406ab00 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.360+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0540c4120 con 0x7ff05406ab00 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.361+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05406fcc0 con 0x7ff05406ab00 2023-10-06T05:27:39.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.362+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff05001b3b0 con 0x7ff05406ab00 2023-10-06T05:27:39.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.363+0000 7ff0497fa700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff038066f30 0x7ff0380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.363+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff050083cf0 con 0x7ff05406ab00 2023-10-06T05:27:39.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.363+0000 7ff05899a700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff038066f30 0x7ff0380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.364+0000 7ff05899a700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff038066f30 0x7ff0380693e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff0440099b0 tx=0x7ff044017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.367+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff050050ce0 con 0x7ff05406ab00 2023-10-06T05:27:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.610+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff05404c380 con 0x7ff05406ab00 2023-10-06T05:27:39.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.611+0000 7ff0497fa700 1 -- 172.21.15.33:0/1702636824 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4615 (secure 0 0 0) 0x7ff050017070 con 0x7ff05406ab00 2023-10-06T05:27:39.615 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:39.615 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":139,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:32.888818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":6,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff038066f30 msgr2=0x7ff0380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff038066f30 0x7ff0380693e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff0440099b0 tx=0x7ff044017000 comp rx=0 tx=0).stop 2023-10-06T05:27:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 msgr2=0x7ff05415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff05415c6a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c4e0 tx=0x7ff05000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 shutdown_connections 2023-10-06T05:27:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff038066f30 0x7ff0380693e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0540b2e30 0x7ff054169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff05406ab00 0x7ff05415c6a0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 --2- 172.21.15.33:0/1702636824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0540b0a40 0x7ff0540c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 >> 172.21.15.33:0/1702636824 conn(0x7ff05406e0f0 msgr2=0x7ff0540abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 shutdown_connections 2023-10-06T05:27:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.613+0000 7ff05a19d700 1 -- 172.21.15.33:0/1702636824 wait complete. 2023-10-06T05:27:39.619 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 139 2023-10-06T05:27:39.632 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:39.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.818+0000 7f5123fff700 1 Processor -- start 2023-10-06T05:27:39.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.818+0000 7f5123fff700 1 -- start start 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.819+0000 7f5123fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f512413faa0 0x7f512413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.819+0000 7f5123fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51240c7940 0x7f51240c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.819+0000 7f5123fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51240c9cb0 con 0x7f512406aad0 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.819+0000 7f5123fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512406b660 con 0x7f512413faa0 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.819+0000 7f5123fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240ca270 con 0x7f51240c7940 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f512413faa0 0x7f512413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f512413faa0 0x7f512413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41784/0 (socket says 172.21.15.33:41784) 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 -- 172.21.15.33:0/98641197 learned_addr learned my addr 172.21.15.33:0/98641197 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f51227fc700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51240c7940 0x7f51240c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 -- 172.21.15.33:0/98641197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51240c7940 msgr2=0x7f51240c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51240c7940 0x7f51240c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 -- 172.21.15.33:0/98641197 >> v1:172.21.15.167:6789/0 conn(0x7f512406aad0 legacy=0x7f512406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f5122ffd700 1 -- 172.21.15.33:0/98641197 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512406b840 con 0x7f512413faa0 2023-10-06T05:27:39.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.820+0000 7f51227fc700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51240c7940 0x7f51240c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5122ffd700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512413faa0 0x7f512413fe70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f51140099b0 tx=0x7f5114009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c0c0d8f6699260 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5121ffb700 1 -- 172.21.15.33:0/98641197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400b860 con 0x7f512413faa0 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5121ffb700 1 -- 172.21.15.33:0/98641197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511400b9c0 con 0x7f512413faa0 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5121ffb700 1 -- 172.21.15.33:0/98641197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400bd60 con 0x7f512413faa0 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5123fff700 1 -- 172.21.15.33:0/98641197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512413faa0 msgr2=0x7f512413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5123fff700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512413faa0 0x7f512413fe70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f51140099b0 tx=0x7f5114009250 comp rx=0 tx=0).stop 2023-10-06T05:27:39.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5123fff700 1 -- 172.21.15.33:0/98641197 shutdown_connections 2023-10-06T05:27:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5123fff700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512413faa0 0x7f512413fe70 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5123fff700 1 --2- 172.21.15.33:0/98641197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51240c7940 0x7f51240c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.821+0000 7f5123fff700 1 -- 172.21.15.33:0/98641197 >> 172.21.15.33:0/98641197 conn(0x7f512406d030 msgr2=0x7f512406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.822+0000 7f5123fff700 1 -- 172.21.15.33:0/98641197 shutdown_connections 2023-10-06T05:27:39.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.822+0000 7f5123fff700 1 -- 172.21.15.33:0/98641197 wait complete. 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.823+0000 7f5123fff700 1 Processor -- start 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.823+0000 7f5123fff700 1 -- start start 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5123fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 0x7f51240c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5123fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51240c7940 0x7f51240c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5123fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512413faa0 0x7f51240c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5123fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240c6460 con 0x7f512406aad0 2023-10-06T05:27:39.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5123fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240c55b0 con 0x7f51240c7940 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5123fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240c5850 con 0x7f512413faa0 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5122ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 0x7f51240c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5122ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 0x7f51240c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41794/0 (socket says 172.21.15.33:41794) 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.824+0000 7f5122ffd700 1 -- 172.21.15.33:0/3917259575 learned_addr learned my addr 172.21.15.33:0/3917259575 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5122ffd700 1 -- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512413faa0 msgr2=0x7f51240c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5122ffd700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512413faa0 0x7f51240c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f51227fc700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51240c7940 0x7f51240c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5122ffd700 1 -- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51240c7940 msgr2=0x7f51240c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5122ffd700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51240c7940 0x7f51240c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5122ffd700 1 -- 172.21.15.33:0/3917259575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51140094f0 con 0x7f512406aad0 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f51227fc700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51240c7940 0x7f51240c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5122ffd700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 0x7f51240c68f0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f5114009290 tx=0x7f5114017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114026480 con 0x7f512406aad0 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511401f070 con 0x7f512406aad0 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114026830 con 0x7f512406aad0 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51240c5b30 con 0x7f512406aad0 2023-10-06T05:27:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.825+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51240c60f0 con 0x7f512406aad0 2023-10-06T05:27:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.826+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512404c380 con 0x7f512406aad0 2023-10-06T05:27:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.827+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5114026ba0 con 0x7f512406aad0 2023-10-06T05:27:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.828+0000 7f510affd700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5110066f30 0x7f51100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.828+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5114021070 con 0x7f512406aad0 2023-10-06T05:27:39.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.829+0000 7f51227fc700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5110066f30 0x7f51100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.830+0000 7f51227fc700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5110066f30 0x7f51100693e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5118006f00 tx=0x7f5118008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:39.832+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5114053590 con 0x7f512406aad0 2023-10-06T05:27:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.085+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f512406bb20 con 0x7f512406aad0 2023-10-06T05:27:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.086+0000 7f510affd700 1 -- 172.21.15.33:0/3917259575 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f5114056bb0 con 0x7f512406aad0 2023-10-06T05:27:40.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:40.088 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":139,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:07.463944+0000","modified":"2023-10-06T05:27:32.888818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":182,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9582},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9582":{"gid":9582,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":6,"addr":"172.21.15.33:6835/1947720392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1947720392},{"type":"v1","addr":"172.21.15.33:6835","nonce":1947720392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:27:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5110066f30 msgr2=0x7f51100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5110066f30 0x7f51100693e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5118006f00 tx=0x7f5118008040 comp rx=0 tx=0).stop 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 msgr2=0x7f51240c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 0x7f51240c68f0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f5114009290 tx=0x7f5114017000 comp rx=0 tx=0).stop 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 shutdown_connections 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5110066f30 0x7f51100693e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512413faa0 0x7f51240c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f512406aad0 0x7f51240c68f0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 --2- 172.21.15.33:0/3917259575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51240c7940 0x7f51240c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.088+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 >> 172.21.15.33:0/3917259575 conn(0x7f512406d030 msgr2=0x7f512406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.089+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 shutdown_connections 2023-10-06T05:27:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.089+0000 7f5123fff700 1 -- 172.21.15.33:0/3917259575 wait complete. 2023-10-06T05:27:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 139 2023-10-06T05:27:40.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:40.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.294+0000 7f0f8359e700 1 Processor -- start 2023-10-06T05:27:40.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.294+0000 7f0f8359e700 1 -- start start 2023-10-06T05:27:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f8359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f84159520 0x7f0f840c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f8359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0f840c6170 0x7f0f840c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f8359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f840c80d0 0x7f0f84067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f8359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f840c65e0 con 0x7f0f840c80d0 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f8359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8406c820 con 0x7f0f840c6170 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f8359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8406a160 con 0x7f0f84159520 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.295+0000 7f0f82d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f840c80d0 0x7f0f84067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f840c80d0 0x7f0f84067ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50958/0 (socket says 172.21.15.33:50958) 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 -- 172.21.15.33:0/3958421139 learned_addr learned my addr 172.21.15.33:0/3958421139 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f81d9b700 1 --2- 172.21.15.33:0/3958421139 >> v2:172.21.15.167:3300/0 conn(0x7f0f840c6170 0x7f0f840c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f8259c700 1 --2- 172.21.15.33:0/3958421139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f84159520 0x7f0f840c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 -- 172.21.15.33:0/3958421139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f84159520 msgr2=0x7f0f840c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 --2- 172.21.15.33:0/3958421139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f84159520 0x7f0f840c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 -- 172.21.15.33:0/3958421139 >> v2:172.21.15.167:3300/0 conn(0x7f0f840c6170 msgr2=0x7f0f840c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 --2- 172.21.15.33:0/3958421139 >> v2:172.21.15.167:3300/0 conn(0x7f0f840c6170 0x7f0f840c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 -- 172.21.15.33:0/3958421139 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f840c8800 con 0x7f0f840c80d0 2023-10-06T05:27:40.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f82d9d700 1 --2- 172.21.15.33:0/3958421139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f84067ba0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0f840636c0 tx=0x7f0f7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e83ddafcc908c6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.296+0000 7f0f8159a700 1 -- 172.21.15.33:0/3958421139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00e070 con 0x7f0f840c80d0 2023-10-06T05:27:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8159a700 1 -- 172.21.15.33:0/3958421139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c00b970 con 0x7f0f840c80d0 2023-10-06T05:27:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8159a700 1 -- 172.21.15.33:0/3958421139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0092e0 con 0x7f0f840c80d0 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 -- 172.21.15.33:0/3958421139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 msgr2=0x7f0f84067ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 --2- 172.21.15.33:0/3958421139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f84067ba0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0f840636c0 tx=0x7f0f7c009840 comp rx=0 tx=0).stop 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 -- 172.21.15.33:0/3958421139 shutdown_connections 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 --2- 172.21.15.33:0/3958421139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f84067ba0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 --2- 172.21.15.33:0/3958421139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f84159520 0x7f0f840c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 --2- 172.21.15.33:0/3958421139 >> v2:172.21.15.167:3300/0 conn(0x7f0f840c6170 0x7f0f840c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 -- 172.21.15.33:0/3958421139 >> 172.21.15.33:0/3958421139 conn(0x7f0f840cc930 msgr2=0x7f0f840ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 -- 172.21.15.33:0/3958421139 shutdown_connections 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.297+0000 7f0f8359e700 1 -- 172.21.15.33:0/3958421139 wait complete. 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.299+0000 7f0f8359e700 1 Processor -- start 2023-10-06T05:27:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.299+0000 7f0f8359e700 1 -- start start 2023-10-06T05:27:40.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f840c6170 0x7f0f8415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f840b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f84159520 0x7f0f840b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8415a3c0 con 0x7f0f840c80d0 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f84152be0 con 0x7f0f84159520 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f84159a90 con 0x7f0f840c6170 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f81d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f840b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f81d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f840b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50960/0 (socket says 172.21.15.33:50960) 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f81d9b700 1 -- 172.21.15.33:0/2303998339 learned_addr learned my addr 172.21.15.33:0/2303998339 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f8259c700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f840c6170 0x7f0f8415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f82d9d700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f84159520 0x7f0f840b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f81d9b700 1 -- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f840c6170 msgr2=0x7f0f8415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.300+0000 7f0f81d9b700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f840c6170 0x7f0f8415c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f81d9b700 1 -- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f84159520 msgr2=0x7f0f840b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f81d9b700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f84159520 0x7f0f840b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f81d9b700 1 -- 172.21.15.33:0/2303998339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009ae0 con 0x7f0f840c80d0 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f82d9d700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f84159520 0x7f0f840b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f81d9b700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f840b0d00 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800ec80 tx=0x7f0f780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800c850 con 0x7f0f840c80d0 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7800c9b0 con 0x7f0f840c80d0 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f84159d70 con 0x7f0f840c80d0 2023-10-06T05:27:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800cd50 con 0x7f0f840c80d0 2023-10-06T05:27:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.301+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8415c650 con 0x7f0f840c80d0 2023-10-06T05:27:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.302+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8404c380 con 0x7f0f840c80d0 2023-10-06T05:27:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.303+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f78013680 con 0x7f0f840c80d0 2023-10-06T05:27:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.304+0000 7f0f727fc700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f64066f30 0x7f0f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.304+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0f78084dd0 con 0x7f0f840c80d0 2023-10-06T05:27:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.304+0000 7f0f8259c700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f64066f30 0x7f0f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.305+0000 7f0f8259c700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f64066f30 0x7f0f640693e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009d10 tx=0x7f0f740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.308+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f78051dc0 con 0x7f0f840c80d0 2023-10-06T05:27:40.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.511+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f840c8ae0 con 0x7f0f840c80d0 2023-10-06T05:27:40.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.512+0000 7f0f727fc700 1 -- 172.21.15.33:0/2303998339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17069 (secure 0 0 0) 0x7f0f78017090 con 0x7f0f840c80d0 2023-10-06T05:27:40.514 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:40.514 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":183,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:30.873128+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:05.629452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:06.434007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1699314688":"2023-10-06T06:27:30.190772+0000","172.21.15.33:6835/950449971":"2023-10-07T05:27:20.407691+0000","172.21.15.33:6833/950449971":"2023-10-07T05:27:20.407691+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f64066f30 msgr2=0x7f0f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f64066f30 0x7f0f640693e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009d10 tx=0x7f0f740091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 msgr2=0x7f0f840b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f840b0d00 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800ec80 tx=0x7f0f780091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 shutdown_connections 2023-10-06T05:27:40.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0f64066f30 0x7f0f640693e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f84159520 0x7f0f840b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f840c6170 0x7f0f8415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 --2- 172.21.15.33:0/2303998339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f840c80d0 0x7f0f840b0d00 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 >> 172.21.15.33:0/2303998339 conn(0x7f0f840cc930 msgr2=0x7f0f8406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 shutdown_connections 2023-10-06T05:27:40.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.514+0000 7f0f8359e700 1 -- 172.21.15.33:0/2303998339 wait complete. 2023-10-06T05:27:40.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:27:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.720+0000 7fa8bbfff700 1 Processor -- start 2023-10-06T05:27:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.720+0000 7fa8bbfff700 1 -- start start 2023-10-06T05:27:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8bc1596c0 0x7fa8bc159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc0af600 0x7fa8bc0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8bc06ec40 con 0x7fa8bc06f0d0 2023-10-06T05:27:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bbfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc0aff90 con 0x7fa8bc1596c0 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc06dd80 con 0x7fa8bc0af600 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8bc1596c0 0x7fa8bc159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8bc1596c0 0x7fa8bc159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50962/0 (socket says 172.21.15.33:50962) 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 -- 172.21.15.33:0/2594321907 learned_addr learned my addr 172.21.15.33:0/2594321907 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bb7fe700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc0af600 0x7fa8bc0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 -- 172.21.15.33:0/2594321907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc0af600 msgr2=0x7fa8bc0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc0af600 0x7fa8bc0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 -- 172.21.15.33:0/2594321907 >> v1:172.21.15.167:6789/0 conn(0x7fa8bc06f0d0 legacy=0x7fa8bc06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8baffd700 1 -- 172.21.15.33:0/2594321907 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc06e030 con 0x7fa8bc1596c0 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.722+0000 7fa8bb7fe700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc0af600 0x7fa8bc0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.723+0000 7fa8baffd700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc159a90 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0099b0 tx=0x7fa8ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d11a2556d3d002 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.723+0000 7fa8b9ffb700 1 -- 172.21.15.33:0/2594321907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac00b860 con 0x7fa8bc1596c0 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.723+0000 7fa8b9ffb700 1 -- 172.21.15.33:0/2594321907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac00b9c0 con 0x7fa8bc1596c0 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.723+0000 7fa8b9ffb700 1 -- 172.21.15.33:0/2594321907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac00bd60 con 0x7fa8bc1596c0 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.723+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2594321907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 msgr2=0x7fa8bc159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.723+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc159a90 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0099b0 tx=0x7fa8ac009250 comp rx=0 tx=0).stop 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.724+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2594321907 shutdown_connections 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.724+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc159a90 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.724+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2594321907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc0af600 0x7fa8bc0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.724+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2594321907 >> 172.21.15.33:0/2594321907 conn(0x7fa8bc087fb0 msgr2=0x7fa8bc0883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.724+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2594321907 shutdown_connections 2023-10-06T05:27:40.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.724+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2594321907 wait complete. 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.725+0000 7fa8bbfff700 1 Processor -- start 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.725+0000 7fa8bbfff700 1 -- start start 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc06f0d0 0x7fa8bc0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8bc0af600 0x7fa8bc1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0ca7a0 con 0x7fa8bc1596c0 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0caa80 con 0x7fa8bc0af600 2023-10-06T05:27:40.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0cad60 con 0x7fa8bc06f0d0 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc0c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50966/0 (socket says 172.21.15.33:50966) 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.726+0000 7fa8bb7fe700 1 -- 172.21.15.33:0/2468644264 learned_addr learned my addr 172.21.15.33:0/2468644264 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8baffd700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc06f0d0 0x7fa8bc0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8ba7fc700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8bc0af600 0x7fa8bc1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bb7fe700 1 -- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc06f0d0 msgr2=0x7fa8bc0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bb7fe700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc06f0d0 0x7fa8bc0c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bb7fe700 1 -- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8bc0af600 msgr2=0x7fa8bc1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bb7fe700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8bc0af600 0x7fa8bc1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bb7fe700 1 -- 172.21.15.33:0/2468644264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0094f0 con 0x7fa8bc1596c0 2023-10-06T05:27:40.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bb7fe700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc0c9f70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400a380 tx=0x7fa8b4007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4015440 con 0x7fa8bc1596c0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc0c4b80 con 0x7fa8bc1596c0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40155a0 con 0x7fa8bc1596c0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.727+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4015960 con 0x7fa8bc1596c0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.728+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc0ca4b0 con 0x7fa8bc1596c0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.728+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc0c50a0 con 0x7fa8bc1596c0 2023-10-06T05:27:40.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.729+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8b400d440 con 0x7fa8bc1596c0 2023-10-06T05:27:40.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.730+0000 7fa8a2ffd700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa8a8066f30 0x7fa8a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.730+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa8b4085340 con 0x7fa8bc1596c0 2023-10-06T05:27:40.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.731+0000 7fa8baffd700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa8a8066f30 0x7fa8a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.732+0000 7fa8baffd700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa8a8066f30 0x7fa8a80693e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0099b0 tx=0x7fa8ac017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.735+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8b4052330 con 0x7fa8bc1596c0 2023-10-06T05:27:40.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.983+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8bc04c380 con 0x7fa8bc1596c0 2023-10-06T05:27:40.985 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:27:40.984+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:27:40.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.989+0000 7fa8a2ffd700 1 -- 172.21.15.33:0/2468644264 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v140) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8b4055950 con 0x7fa8bc1596c0 2023-10-06T05:27:40.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa8a8066f30 msgr2=0x7fa8a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa8a8066f30 0x7fa8a80693e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0099b0 tx=0x7fa8ac017000 comp rx=0 tx=0).stop 2023-10-06T05:27:40.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 msgr2=0x7fa8bc0c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc0c9f70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400a380 tx=0x7fa8b4007900 comp rx=0 tx=0).stop 2023-10-06T05:27:40.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 shutdown_connections 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa8a8066f30 0x7fa8a80693e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8bc1596c0 0x7fa8bc0c9f70 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8bc06f0d0 0x7fa8bc0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 --2- 172.21.15.33:0/2468644264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8bc0af600 0x7fa8bc1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 >> 172.21.15.33:0/2468644264 conn(0x7fa8bc087fb0 msgr2=0x7fa8bc068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.991+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 shutdown_connections 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:40.992+0000 7fa8bbfff700 1 -- 172.21.15.33:0/2468644264 wait complete. 2023-10-06T05:27:40.994 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:27:40.999 INFO:tasks.ceph.mds.c.smithi033.stderr: -227> 2023-10-06T05:27:36.419+0000 7f260a1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:41.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:27:41.034 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:27:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.200+0000 7f486dd9d700 1 Processor -- start 2023-10-06T05:27:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.200+0000 7f486dd9d700 1 -- start start 2023-10-06T05:27:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.201+0000 7f486dd9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4868159850 0x7f486806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.201+0000 7f486dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f486806d140 0x7f486806edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.201+0000 7f486dd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680b0cd0 con 0x7f486815acd0 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.201+0000 7f486dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486815a840 con 0x7f486806d140 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.201+0000 7f486dd9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4868159f10 con 0x7f4868159850 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.201+0000 7f486d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f486806d140 0x7f486806edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f486806d140 0x7f486806edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50992/0 (socket says 172.21.15.33:50992) 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 -- 172.21.15.33:0/1044289815 learned_addr learned my addr 172.21.15.33:0/1044289815 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486cd9b700 1 --2- 172.21.15.33:0/1044289815 >> v2:172.21.15.167:3300/0 conn(0x7f4868159850 0x7f486806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 -- 172.21.15.33:0/1044289815 >> v2:172.21.15.167:3300/0 conn(0x7f4868159850 msgr2=0x7f486806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 --2- 172.21.15.33:0/1044289815 >> v2:172.21.15.167:3300/0 conn(0x7f4868159850 0x7f486806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 -- 172.21.15.33:0/1044289815 >> v1:172.21.15.167:6789/0 conn(0x7f486815acd0 legacy=0x7f486806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 -- 172.21.15.33:0/1044289815 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486815a1f0 con 0x7f486806d140 2023-10-06T05:27:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f486d59c700 1 --2- 172.21.15.33:0/1044289815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486806d140 0x7f486806edd0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f4868063620 tx=0x7f4860009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28503296b148eed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.202+0000 7f48677fe700 1 -- 172.21.15.33:0/1044289815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486000e070 con 0x7f486806d140 2023-10-06T05:27:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f48677fe700 1 -- 172.21.15.33:0/1044289815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486000ba20 con 0x7f486806d140 2023-10-06T05:27:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f48677fe700 1 -- 172.21.15.33:0/1044289815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48600092e0 con 0x7f486806d140 2023-10-06T05:27:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 -- 172.21.15.33:0/1044289815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486806d140 msgr2=0x7f486806edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 --2- 172.21.15.33:0/1044289815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486806d140 0x7f486806edd0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f4868063620 tx=0x7f4860009840 comp rx=0 tx=0).stop 2023-10-06T05:27:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 -- 172.21.15.33:0/1044289815 shutdown_connections 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 --2- 172.21.15.33:0/1044289815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486806d140 0x7f486806edd0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 --2- 172.21.15.33:0/1044289815 >> v2:172.21.15.167:3300/0 conn(0x7f4868159850 0x7f486806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 -- 172.21.15.33:0/1044289815 >> 172.21.15.33:0/1044289815 conn(0x7f486806cd40 msgr2=0x7f48680ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 -- 172.21.15.33:0/1044289815 shutdown_connections 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.203+0000 7f486dd9d700 1 -- 172.21.15.33:0/1044289815 wait complete. 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.205+0000 7f486dd9d700 1 Processor -- start 2023-10-06T05:27:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.205+0000 7f486dd9d700 1 -- start start 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486806d140 0x7f48680b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4868159850 0x7f48680b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 0x7f48680ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48680ad490 con 0x7f486815acd0 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486816a220 con 0x7f486806d140 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680b15f0 con 0x7f4868159850 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 0x7f48680ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 0x7f48680ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51006/0 (socket says 172.21.15.33:51006) 2023-10-06T05:27:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 -- 172.21.15.33:0/4014164895 learned_addr learned my addr 172.21.15.33:0/4014164895 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486cd9b700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486806d140 0x7f48680b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f4867fff700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4868159850 0x7f48680b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 -- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4868159850 msgr2=0x7f48680b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4868159850 0x7f48680b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 -- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486806d140 msgr2=0x7f48680b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486806d140 0x7f48680b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.206+0000 7f486d59c700 1 -- 172.21.15.33:0/4014164895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860009ae0 con 0x7f486815acd0 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.207+0000 7f486d59c700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 0x7f48680ac1c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f4860012000 tx=0x7f4860018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.207+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48600089b0 con 0x7f486815acd0 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.207+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4860008b10 con 0x7f486815acd0 2023-10-06T05:27:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.207+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860025460 con 0x7f486815acd0 2023-10-06T05:27:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.207+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48681696a0 con 0x7f486815acd0 2023-10-06T05:27:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.207+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4868169c60 con 0x7f486815acd0 2023-10-06T05:27:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.208+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48680b25c0 con 0x7f486815acd0 2023-10-06T05:27:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.209+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4860003040 con 0x7f486815acd0 2023-10-06T05:27:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.210+0000 7f4864ff9700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f484c066f30 0x7f484c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:41.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.210+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f48600862c0 con 0x7f486815acd0 2023-10-06T05:27:41.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.210+0000 7f486cd9b700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f484c066f30 0x7f484c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:41.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.211+0000 7f486cd9b700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f484c066f30 0x7f484c0693e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f485c009db0 tx=0x7f485c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:41.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.214+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48680b25c0 con 0x7f486815acd0 2023-10-06T05:27:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.463+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f48680b25c0 con 0x7f486815acd0 2023-10-06T05:27:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.991+0000 7f4864ff9700 1 -- 172.21.15.33:0/4014164895 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v141) v1 ==== 104+0+0 (secure 0 0 0) 0x7f48680b25c0 con 0x7f486815acd0 2023-10-06T05:27:42.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.993+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f484c066f30 msgr2=0x7f484c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.993+0000 7f486dd9d700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f484c066f30 0x7f484c0693e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f485c009db0 tx=0x7f485c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 msgr2=0x7f48680ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 0x7f48680ac1c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f4860012000 tx=0x7f4860018000 comp rx=0 tx=0).stop 2023-10-06T05:27:42.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 shutdown_connections 2023-10-06T05:27:42.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f484c066f30 0x7f484c0693e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486815acd0 0x7f48680ac1c0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486806d140 0x7f48680b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 --2- 172.21.15.33:0/4014164895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4868159850 0x7f48680b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 >> 172.21.15.33:0/4014164895 conn(0x7f486806cd40 msgr2=0x7f486815b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 shutdown_connections 2023-10-06T05:27:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:41.994+0000 7f486dd9d700 1 -- 172.21.15.33:0/4014164895 wait complete. 2023-10-06T05:27:42.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:27:42.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.248+0000 7f9e43fff700 1 Processor -- start 2023-10-06T05:27:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.248+0000 7f9e43fff700 1 -- start start 2023-10-06T05:27:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.249+0000 7f9e43fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9e4406dfd0 0x7f9e4415a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.249+0000 7f9e43fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4415d280 con 0x7f9e441597f0 2023-10-06T05:27:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.249+0000 7f9e43fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4415de00 con 0x7f9e4406dc00 2023-10-06T05:27:43.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.249+0000 7f9e43fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4415e290 con 0x7f9e4406dfd0 2023-10-06T05:27:43.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.249+0000 7f9e427fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e441597f0 0x7f9e4415a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55436/0 (socket says 172.21.15.33:55436) 2023-10-06T05:27:43.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.249+0000 7f9e427fc700 1 -- 172.21.15.33:0/2395706164 learned_addr learned my addr 172.21.15.33:0/2395706164 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:43.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e437fe700 1 --2- 172.21.15.33:0/2395706164 >> v2:172.21.15.167:3300/0 conn(0x7f9e4406dfd0 0x7f9e4415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587122284 0 0) 0x7f9e4415d280 con 0x7f9e441597f0 2023-10-06T05:27:43.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e2c003570 con 0x7f9e441597f0 2023-10-06T05:27:43.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742930761 0 0) 0x7f9e4415de00 con 0x7f9e4406dc00 2023-10-06T05:27:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e4415d280 con 0x7f9e4406dc00 2023-10-06T05:27:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 353885778 0 0) 0x7f9e2c003570 con 0x7f9e441597f0 2023-10-06T05:27:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 >> v2:172.21.15.167:3300/0 conn(0x7f9e4406dfd0 msgr2=0x7f9e4415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 --2- 172.21.15.33:0/2395706164 >> v2:172.21.15.167:3300/0 conn(0x7f9e4406dfd0 0x7f9e4415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 >> v1:172.21.15.167:6789/0 conn(0x7f9e4406dc00 legacy=0x7f9e441590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4415d4d0 con 0x7f9e441597f0 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9e38001470 con 0x7f9e441597f0 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 >> v1:172.21.15.33:6789/0 conn(0x7f9e441597f0 legacy=0x7f9e4415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c003c40 0x7f9e2c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e2c004810 0x7f9e2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e2c005210 0x7f9e2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c00c130 con 0x7f9e2c005210 2023-10-06T05:27:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c00c270 con 0x7f9e2c003c40 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.250+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c00c400 con 0x7f9e2c004810 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e2c005210 0x7f9e2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e427fc700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e2c004810 0x7f9e2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e42ffd700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c003c40 0x7f9e2c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 -- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e2c004810 msgr2=0x7f9e2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e2c004810 0x7f9e2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 -- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c003c40 msgr2=0x7f9e2c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c003c40 0x7f9e2c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 -- 172.21.15.33:0/2395706164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c003570 con 0x7f9e2c005210 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 -- 172.21.15.33:0/2395706164 reap_dead start 2023-10-06T05:27:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e437fe700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e2c005210 0x7f9e2c00ba60 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00e240 tx=0x7f9e3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2679f0df85ad7d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c009ac0 con 0x7f9e2c005210 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.251+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c00f040 con 0x7f9e2c005210 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e41ffb700 1 -- 172.21.15.33:0/2395706164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00ed40 con 0x7f9e2c005210 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 -- 172.21.15.33:0/2395706164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e2c005210 msgr2=0x7f9e2c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e2c005210 0x7f9e2c00ba60 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00e240 tx=0x7f9e3c00b040 comp rx=0 tx=0).stop 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 -- 172.21.15.33:0/2395706164 shutdown_connections 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 --2- 172.21.15.33:0/2395706164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e2c005210 0x7f9e2c00ba60 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 -- 172.21.15.33:0/2395706164 >> 172.21.15.33:0/2395706164 conn(0x7f9e440d0550 msgr2=0x7f9e440d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 -- 172.21.15.33:0/2395706164 shutdown_connections 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.252+0000 7f9e43fff700 1 -- 172.21.15.33:0/2395706164 wait complete. 2023-10-06T05:27:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.254+0000 7f9e43fff700 1 Processor -- start 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.254+0000 7f9e43fff700 1 -- start start 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c005210 0x7f9e4416d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 0x7f9e440c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e441533d0 0x7f9e440c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e43fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440c84c0 con 0x7f9e44153d10 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e43fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4416de50 con 0x7f9e2c005210 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e43fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440c7f00 con 0x7f9e441533d0 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 0x7f9e440c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 0x7f9e440c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51022/0 (socket says 172.21.15.33:51022) 2023-10-06T05:27:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 -- 172.21.15.33:0/1336139779 learned_addr learned my addr 172.21.15.33:0/1336139779 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e437fe700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e441533d0 0x7f9e440c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e42ffd700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c005210 0x7f9e4416d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 -- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e441533d0 msgr2=0x7f9e440c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e441533d0 0x7f9e440c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 -- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c005210 msgr2=0x7f9e4416d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c005210 0x7f9e4416d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 -- 172.21.15.33:0/1336139779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440c81e0 con 0x7f9e44153d10 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.255+0000 7f9e427fc700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 0x7f9e440c7600 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9e38001e30 tx=0x7f9e38001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.256+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380077d0 con 0x7f9e44153d10 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.256+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38000cb0 con 0x7f9e44153d10 2023-10-06T05:27:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.256+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800fb00 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.256+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4416d110 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.256+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e440c7b40 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.257+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4416cf70 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.258+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e3801d460 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.259+0000 7f9e2affd700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e30066f60 0x7f9e30069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.259+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f9e380848d0 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.259+0000 7f9e42ffd700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e30066f60 0x7f9e30069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.260+0000 7f9e42ffd700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e30066f60 0x7f9e30069410 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9e34007e40 tx=0x7f9e34009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.262+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e4416cf70 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.467+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9e4416d630 con 0x7f9e44153d10 2023-10-06T05:27:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:42.998+0000 7f9e2affd700 1 -- 172.21.15.33:0/1336139779 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v185) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9e4416d630 con 0x7f9e44153d10 2023-10-06T05:27:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e30066f60 msgr2=0x7f9e30069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e30066f60 0x7f9e30069410 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9e34007e40 tx=0x7f9e34009040 comp rx=0 tx=0).stop 2023-10-06T05:27:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 msgr2=0x7f9e440c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 0x7f9e440c7600 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9e38001e30 tx=0x7f9e38001d20 comp rx=0 tx=0).stop 2023-10-06T05:27:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 shutdown_connections 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e30066f60 0x7f9e30069410 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e441533d0 0x7f9e440c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e2c005210 0x7f9e4416d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 --2- 172.21.15.33:0/1336139779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e44153d10 0x7f9e440c7600 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.003+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 >> 172.21.15.33:0/1336139779 conn(0x7f9e440d0550 msgr2=0x7f9e440c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.004+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 shutdown_connections 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.004+0000 7f9e43fff700 1 -- 172.21.15.33:0/1336139779 wait complete. 2023-10-06T05:27:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:27:43.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:27:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.748+0000 7fca3359e700 1 Processor -- start 2023-10-06T05:27:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.748+0000 7fca3359e700 1 -- start start 2023-10-06T05:27:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.749+0000 7fca3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca34156b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.749+0000 7fca3359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca341570a0 0x7fca34157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.749+0000 7fca3359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3406b1f0 con 0x7fca340ad010 2023-10-06T05:27:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.749+0000 7fca3359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3406a3f0 con 0x7fca341570a0 2023-10-06T05:27:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.749+0000 7fca3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3406f110 con 0x7fca340ad3e0 2023-10-06T05:27:43.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca3259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca340ad010 0x7fca3406ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55450/0 (socket says 172.21.15.33:55450) 2023-10-06T05:27:43.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca3259c700 1 -- 172.21.15.33:0/2027925639 learned_addr learned my addr 172.21.15.33:0/2027925639 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca32d9d700 1 --2- 172.21.15.33:0/2027925639 >> v2:172.21.15.167:3300/0 conn(0x7fca341570a0 0x7fca34157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca31d9b700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca34156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433023720 0 0) 0x7fca3406b1f0 con 0x7fca340ad010 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca1c003570 con 0x7fca340ad010 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.750+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433254146 0 0) 0x7fca1c003570 con 0x7fca340ad010 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 msgr2=0x7fca34156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca34156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 >> v2:172.21.15.167:3300/0 conn(0x7fca341570a0 msgr2=0x7fca34157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 --2- 172.21.15.33:0/2027925639 >> v2:172.21.15.167:3300/0 conn(0x7fca341570a0 0x7fca34157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34157a40 con 0x7fca340ad010 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fca24001470 con 0x7fca340ad010 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 >> v1:172.21.15.33:6789/0 conn(0x7fca340ad010 legacy=0x7fca3406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca1c004810 0x7fca1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3406f110 con 0x7fca1c003a60 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00ba60 con 0x7fca1c004810 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c00bba0 con 0x7fca340ad3e0 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3259c700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca31d9b700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca1c004810 0x7fca1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca32d9d700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3259c700 1 -- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 msgr2=0x7fca1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3259c700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3259c700 1 -- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca1c004810 msgr2=0x7fca1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3259c700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca1c004810 0x7fca1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca3259c700 1 -- 172.21.15.33:0/2027925639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c003570 con 0x7fca1c003a60 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.751+0000 7fca32d9d700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340ad3e0 0x7fca1c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca3259c700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca1c003e70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fca2400c3f0 tx=0x7fca3406b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ea42e2bff54e07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400cca0 con 0x7fca1c003a60 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2400ce00 con 0x7fca1c003a60 2023-10-06T05:27:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca3159a700 1 -- 172.21.15.33:0/2027925639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24005b70 con 0x7fca1c003a60 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca3359e700 1 -- 172.21.15.33:0/2027925639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 msgr2=0x7fca1c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca3359e700 1 --2- 172.21.15.33:0/2027925639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca1c003e70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fca2400c3f0 tx=0x7fca3406b160 comp rx=0 tx=0).stop 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.752+0000 7fca32d9d700 1 -- 172.21.15.33:0/2027925639 reap_dead start 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.753+0000 7fca3359e700 1 -- 172.21.15.33:0/2027925639 shutdown_connections 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.753+0000 7fca3359e700 1 -- 172.21.15.33:0/2027925639 >> 172.21.15.33:0/2027925639 conn(0x7fca34068090 msgr2=0x7fca3406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.753+0000 7fca3359e700 1 -- 172.21.15.33:0/2027925639 shutdown_connections 2023-10-06T05:27:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.753+0000 7fca3359e700 1 -- 172.21.15.33:0/2027925639 wait complete. 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.754+0000 7fca3359e700 1 Processor -- start 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.754+0000 7fca3359e700 1 -- start start 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca341441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340b15e0 0x7fca340b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34146660 0x7fca34146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340b1f90 con 0x7fca1c003a60 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340a9aa0 con 0x7fca34146660 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca341473c0 con 0x7fca340b15e0 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca341441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca341441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51052/0 (socket says 172.21.15.33:51052) 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.755+0000 7fca3259c700 1 -- 172.21.15.33:0/1143545995 learned_addr learned my addr 172.21.15.33:0/1143545995 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca32d9d700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34146660 0x7fca34146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca31d9b700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340b15e0 0x7fca340b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3259c700 1 -- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340b15e0 msgr2=0x7fca340b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3259c700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340b15e0 0x7fca340b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3259c700 1 -- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34146660 msgr2=0x7fca34146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3259c700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34146660 0x7fca34146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3259c700 1 -- 172.21.15.33:0/1143545995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca341476a0 con 0x7fca1c003a60 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3259c700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca341441f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fca1c004700 tx=0x7fca24012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400cca0 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca240070d0 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240074b0 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca34147980 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.756+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca340cb050 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.757+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34063ff0 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.758+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca24028460 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.759+0000 7fca227fc700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca14066f30 0x7fca140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.759+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fca24085970 con 0x7fca1c003a60 2023-10-06T05:27:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.760+0000 7fca31d9b700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca14066f30 0x7fca140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.761+0000 7fca31d9b700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca14066f30 0x7fca140693e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fca3416ca60 tx=0x7fca28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:43.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.763+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca24052b60 con 0x7fca1c003a60 2023-10-06T05:27:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:43.969+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fca340a9d80 con 0x7fca1c003a60 2023-10-06T05:27:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.829+0000 7fca227fc700 1 -- 172.21.15.33:0/1143545995 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v186) v1 ==== 169+0+0 (secure 0 0 0) 0x7fca24017020 con 0x7fca1c003a60 2023-10-06T05:27:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.836+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca14066f30 msgr2=0x7fca140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.836+0000 7fca3359e700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca14066f30 0x7fca140693e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fca3416ca60 tx=0x7fca28008040 comp rx=0 tx=0).stop 2023-10-06T05:27:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.836+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 msgr2=0x7fca341441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.836+0000 7fca3359e700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca341441f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fca1c004700 tx=0x7fca24012000 comp rx=0 tx=0).stop 2023-10-06T05:27:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 shutdown_connections 2023-10-06T05:27:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fca14066f30 0x7fca140693e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34146660 0x7fca34146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca1c003a60 0x7fca341441f0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 --2- 172.21.15.33:0/1143545995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca340b15e0 0x7fca340b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 >> 172.21.15.33:0/1143545995 conn(0x7fca34068090 msgr2=0x7fca3414d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.837+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 shutdown_connections 2023-10-06T05:27:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:44.838+0000 7fca3359e700 1 -- 172.21.15.33:0/1143545995 wait complete. 2023-10-06T05:27:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:27:44.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:27:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.059+0000 7fb3c9b9d700 1 Processor -- start 2023-10-06T05:27:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.060+0000 7fb3c9b9d700 1 -- start start 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c9b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c4155a50 0x7fb3c4155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c9b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3c40afb20 0x7fb3c40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c9b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c40b06b0 con 0x7fb3c40c9490 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c9b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c40c99e0 con 0x7fb3c40afb20 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c9b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40c9cc0 con 0x7fb3c4155a50 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3c40afb20 0x7fb3c40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c939c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3c40afb20 0x7fb3c40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51060/0 (socket says 172.21.15.33:51060) 2023-10-06T05:27:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c939c700 1 -- 172.21.15.33:0/2705398723 learned_addr learned my addr 172.21.15.33:0/2705398723 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.061+0000 7fb3c8b9b700 1 --2- 172.21.15.33:0/2705398723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c4155a50 0x7fb3c4155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c939c700 1 -- 172.21.15.33:0/2705398723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c4155a50 msgr2=0x7fb3c4155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c939c700 1 --2- 172.21.15.33:0/2705398723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c4155a50 0x7fb3c4155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c939c700 1 -- 172.21.15.33:0/2705398723 >> v1:172.21.15.167:6789/0 conn(0x7fb3c40c9490 legacy=0x7fb3c40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c939c700 1 -- 172.21.15.33:0/2705398723 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c40c9fa0 con 0x7fb3c40afb20 2023-10-06T05:27:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c939c700 1 --2- 172.21.15.33:0/2705398723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c40afb20 0x7fb3c40caa80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009f50 tx=0x7fb3bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adce318ec0f8fe29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c37fe700 1 -- 172.21.15.33:0/2705398723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00e070 con 0x7fb3c40afb20 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c37fe700 1 -- 172.21.15.33:0/2705398723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc00ba70 con 0x7fb3c40afb20 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.062+0000 7fb3c37fe700 1 -- 172.21.15.33:0/2705398723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0092e0 con 0x7fb3c40afb20 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/2705398723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c40afb20 msgr2=0x7fb3c40caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/2705398723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c40afb20 0x7fb3c40caa80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009f50 tx=0x7fb3bc009840 comp rx=0 tx=0).stop 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/2705398723 shutdown_connections 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/2705398723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c40afb20 0x7fb3c40caa80 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/2705398723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c4155a50 0x7fb3c4155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/2705398723 >> 172.21.15.33:0/2705398723 conn(0x7fb3c406d030 msgr2=0x7fb3c40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/2705398723 shutdown_connections 2023-10-06T05:27:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.063+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/2705398723 wait complete. 2023-10-06T05:27:45.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.064+0000 7fb3c9b9d700 1 Processor -- start 2023-10-06T05:27:45.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.065+0000 7fb3c9b9d700 1 -- start start 2023-10-06T05:27:45.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.065+0000 7fb3c9b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c40afb20 0x7fb3c416cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.065+0000 7fb3c9b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c40c9490 0x7fb3c4140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c9b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 0x7fb3c4169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c9b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4141320 con 0x7fb3c4155a50 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c9b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40a5620 con 0x7fb3c40c9490 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c9b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c416d5a0 con 0x7fb3c40afb20 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 0x7fb3c4169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 0x7fb3c4169960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51066/0 (socket says 172.21.15.33:51066) 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 -- 172.21.15.33:0/386486810 learned_addr learned my addr 172.21.15.33:0/386486810 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c8b9b700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c40afb20 0x7fb3c416cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c3fff700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c40c9490 0x7fb3c4140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 -- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c40afb20 msgr2=0x7fb3c416cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c40afb20 0x7fb3c416cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 -- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c40c9490 msgr2=0x7fb3c4140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c40c9490 0x7fb3c4140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 -- 172.21.15.33:0/386486810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc009ae0 con 0x7fb3c4155a50 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c8b9b700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c40afb20 0x7fb3c416cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.066+0000 7fb3c939c700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 0x7fb3c4169960 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb3c406d4d0 tx=0x7fb3bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.067+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0032f0 con 0x7fb3c4155a50 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.067+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc008e40 con 0x7fb3c4155a50 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.067+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc026690 con 0x7fb3c4155a50 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.067+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c416d880 con 0x7fb3c4155a50 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.067+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c40a73e0 con 0x7fb3c4155a50 2023-10-06T05:27:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.068+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c40a57d0 con 0x7fb3c4155a50 2023-10-06T05:27:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.069+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3bc003040 con 0x7fb3c4155a50 2023-10-06T05:27:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.070+0000 7fb3c0ff9700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3a8066f30 0x7fb3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.070+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb3bc086400 con 0x7fb3c4155a50 2023-10-06T05:27:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.070+0000 7fb3c8b9b700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3a8066f30 0x7fb3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.071+0000 7fb3c8b9b700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3a8066f30 0x7fb3a80693e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800bbb0 tx=0x7fb3b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.074+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3c40a57d0 con 0x7fb3c4155a50 2023-10-06T05:27:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.276+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3c40a5790 con 0x7fb3c4155a50 2023-10-06T05:27:45.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.280+0000 7fb3c0ff9700 1 -- 172.21.15.33:0/386486810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3bc0536f0 con 0x7fb3c4155a50 2023-10-06T05:27:45.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3a8066f30 msgr2=0x7fb3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3a8066f30 0x7fb3a80693e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800bbb0 tx=0x7fb3b8008040 comp rx=0 tx=0).stop 2023-10-06T05:27:45.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 msgr2=0x7fb3c4169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 0x7fb3c4169960 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb3c406d4d0 tx=0x7fb3bc018000 comp rx=0 tx=0).stop 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 shutdown_connections 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3a8066f30 0x7fb3a80693e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3c4155a50 0x7fb3c4169960 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3c40afb20 0x7fb3c416cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 --2- 172.21.15.33:0/386486810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c40c9490 0x7fb3c4140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.282+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 >> 172.21.15.33:0/386486810 conn(0x7fb3c406d030 msgr2=0x7fb3c40cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.283+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 shutdown_connections 2023-10-06T05:27:45.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.283+0000 7fb3c9b9d700 1 -- 172.21.15.33:0/386486810 wait complete. 2023-10-06T05:27:45.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_eviction' 2023-10-06T05:27:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.487+0000 7efee219d700 1 Processor -- start 2023-10-06T05:27:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.487+0000 7efee219d700 1 -- start start 2023-10-06T05:27:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.488+0000 7efee219d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efedc13faa0 0x7efedc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee219d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efedc0c7940 0x7efedc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee219d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0c9cb0 con 0x7efedc0c7940 2023-10-06T05:27:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee219d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0ca270 con 0x7efedc13faa0 2023-10-06T05:27:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc06b840 con 0x7efedc06aad0 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efedc0c7940 0x7efedc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efedc0c7940 0x7efedc0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51078/0 (socket says 172.21.15.33:51078) 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 -- 172.21.15.33:0/3513865995 learned_addr learned my addr 172.21.15.33:0/3513865995 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee119b700 1 --2- 172.21.15.33:0/3513865995 >> v2:172.21.15.167:3300/0 conn(0x7efedc13faa0 0x7efedc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee199c700 1 --2- 172.21.15.33:0/3513865995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 -- 172.21.15.33:0/3513865995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 msgr2=0x7efedc06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 --2- 172.21.15.33:0/3513865995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 -- 172.21.15.33:0/3513865995 >> v2:172.21.15.167:3300/0 conn(0x7efedc13faa0 msgr2=0x7efedc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 --2- 172.21.15.33:0/3513865995 >> v2:172.21.15.167:3300/0 conn(0x7efedc13faa0 0x7efedc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.489+0000 7efee099a700 1 -- 172.21.15.33:0/3513865995 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc06bb20 con 0x7efedc0c7940 2023-10-06T05:27:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.490+0000 7efee099a700 1 --2- 172.21.15.33:0/3513865995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc0c7940 0x7efedc0c7db0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7efedc06fe50 tx=0x7efecc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8b12f7eaebd666 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.490+0000 7efed3fff700 1 -- 172.21.15.33:0/3513865995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00b860 con 0x7efedc0c7940 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.490+0000 7efed3fff700 1 -- 172.21.15.33:0/3513865995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc00b9c0 con 0x7efedc0c7940 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.490+0000 7efed3fff700 1 -- 172.21.15.33:0/3513865995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00bd60 con 0x7efedc0c7940 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.490+0000 7efee219d700 1 -- 172.21.15.33:0/3513865995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc0c7940 msgr2=0x7efedc0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.490+0000 7efee219d700 1 --2- 172.21.15.33:0/3513865995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc0c7940 0x7efedc0c7db0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7efedc06fe50 tx=0x7efecc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 -- 172.21.15.33:0/3513865995 shutdown_connections 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 --2- 172.21.15.33:0/3513865995 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 --2- 172.21.15.33:0/3513865995 >> v2:172.21.15.167:3300/0 conn(0x7efedc13faa0 0x7efedc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 --2- 172.21.15.33:0/3513865995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc0c7940 0x7efedc0c7db0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 -- 172.21.15.33:0/3513865995 >> 172.21.15.33:0/3513865995 conn(0x7efedc06d030 msgr2=0x7efedc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 -- 172.21.15.33:0/3513865995 shutdown_connections 2023-10-06T05:27:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.491+0000 7efee219d700 1 -- 172.21.15.33:0/3513865995 wait complete. 2023-10-06T05:27:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.492+0000 7efee219d700 1 Processor -- start 2023-10-06T05:27:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.492+0000 7efee219d700 1 -- start start 2023-10-06T05:27:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efedc0c7940 0x7efedc1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 0x7efedc15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15ece0 con 0x7efedc13faa0 2023-10-06T05:27:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc1699d0 con 0x7efedc0c7940 2023-10-06T05:27:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc169c70 con 0x7efedc06aad0 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 0x7efedc15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee199c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 0x7efedc15dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51092/0 (socket says 172.21.15.33:51092) 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.493+0000 7efee199c700 1 -- 172.21.15.33:0/4274539309 learned_addr learned my addr 172.21.15.33:0/4274539309 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee099a700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efedc0c7940 0x7efedc1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee119b700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee199c700 1 -- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 msgr2=0x7efedc15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee199c700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc15f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee199c700 1 -- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efedc0c7940 msgr2=0x7efedc1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee199c700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efedc0c7940 0x7efedc1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee199c700 1 -- 172.21.15.33:0/4274539309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0094f0 con 0x7efedc13faa0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee099a700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efedc0c7940 0x7efedc1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee119b700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee199c700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 0x7efedc15dc80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fa0 tx=0x7efed4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400ed20 con 0x7efedc13faa0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400ee80 con 0x7efedc13faa0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc15e1c0 con 0x7efedc13faa0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.494+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4014670 con 0x7efedc13faa0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.495+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc15e720 con 0x7efedc13faa0 2023-10-06T05:27:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.495+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc169de0 con 0x7efedc13faa0 2023-10-06T05:27:45.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.496+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efed4014a20 con 0x7efedc13faa0 2023-10-06T05:27:45.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.497+0000 7efed17fa700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efec0066f30 0x7efec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.497+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efed4084c10 con 0x7efedc13faa0 2023-10-06T05:27:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.498+0000 7efee119b700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efec0066f30 0x7efec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.498+0000 7efee119b700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efec0066f30 0x7efec00693e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efed8006fd0 tx=0x7efed8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.501+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efed4051f00 con 0x7efedc13faa0 2023-10-06T05:27:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.782+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_eviction"]} v 0) v1 -- 0x7efedc04c380 con 0x7efedc13faa0 2023-10-06T05:27:45.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.830+0000 7efed17fa700 1 -- 172.21.15.33:0/4274539309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_eviction"]}]=0 v676) v1 ==== 155+0+0 (secure 0 0 0) 0x7efed4055520 con 0x7efedc13faa0 2023-10-06T05:27:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efec0066f30 msgr2=0x7efec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efec0066f30 0x7efec00693e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efed8006fd0 tx=0x7efed8008040 comp rx=0 tx=0).stop 2023-10-06T05:27:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 msgr2=0x7efedc15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 0x7efedc15dc80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fa0 tx=0x7efed4009900 comp rx=0 tx=0).stop 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 shutdown_connections 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efec0066f30 0x7efec00693e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efedc13faa0 0x7efedc15dc80 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efedc06aad0 0x7efedc15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 --2- 172.21.15.33:0/4274539309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efedc0c7940 0x7efedc1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 >> 172.21.15.33:0/4274539309 conn(0x7efedc06d030 msgr2=0x7efedc0cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 shutdown_connections 2023-10-06T05:27:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:45.832+0000 7efee219d700 1 -- 172.21.15.33:0/4274539309 wait complete. 2023-10-06T05:27:45.851 INFO:tasks.cephfs_test_runner:test_reconnect_eviction (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:27:45.852 INFO:tasks.cephfs_test_runner:Starting test: test_reconnect_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:27:45.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_timeout' 2023-10-06T05:27:46.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.038+0000 7f6f6759e700 1 Processor -- start 2023-10-06T05:27:46.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.039+0000 7f6f6759e700 1 -- start start 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f6813faa0 0x7f6f6813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f6806aad0 0x7f6f6806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680c9cb0 con 0x7f6f680c7940 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680ca270 con 0x7f6f6813faa0 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6806b840 con 0x7f6f6806aad0 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f6813faa0 0x7f6f6813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f6813faa0 0x7f6f6813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51102/0 (socket says 172.21.15.33:51102) 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f6659c700 1 -- 172.21.15.33:0/4251439915 learned_addr learned my addr 172.21.15.33:0/4251439915 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:46.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.040+0000 7f6f66d9d700 1 --2- 172.21.15.33:0/4251439915 >> v2:172.21.15.167:3300/0 conn(0x7f6f6806aad0 0x7f6f6806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6659c700 1 -- 172.21.15.33:0/4251439915 >> v2:172.21.15.167:3300/0 conn(0x7f6f6806aad0 msgr2=0x7f6f6806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6659c700 1 --2- 172.21.15.33:0/4251439915 >> v2:172.21.15.167:3300/0 conn(0x7f6f6806aad0 0x7f6f6806af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6659c700 1 -- 172.21.15.33:0/4251439915 >> v1:172.21.15.167:6789/0 conn(0x7f6f680c7940 legacy=0x7f6f680c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6659c700 1 -- 172.21.15.33:0/4251439915 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6806bb20 con 0x7f6f6813faa0 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f66d9d700 1 --2- 172.21.15.33:0/4251439915 >> v2:172.21.15.167:3300/0 conn(0x7f6f6806aad0 0x7f6f6806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6659c700 1 --2- 172.21.15.33:0/4251439915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6813fe70 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6f68156b90 tx=0x7f6f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1bfc6db24402c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6559a700 1 -- 172.21.15.33:0/4251439915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800b860 con 0x7f6f6813faa0 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6559a700 1 -- 172.21.15.33:0/4251439915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5800b9c0 con 0x7f6f6813faa0 2023-10-06T05:27:46.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.041+0000 7f6f6559a700 1 -- 172.21.15.33:0/4251439915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800bd60 con 0x7f6f6813faa0 2023-10-06T05:27:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 -- 172.21.15.33:0/4251439915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 msgr2=0x7f6f6813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 --2- 172.21.15.33:0/4251439915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6813fe70 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6f68156b90 tx=0x7f6f58009250 comp rx=0 tx=0).stop 2023-10-06T05:27:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 -- 172.21.15.33:0/4251439915 shutdown_connections 2023-10-06T05:27:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 --2- 172.21.15.33:0/4251439915 >> v2:172.21.15.167:3300/0 conn(0x7f6f6806aad0 0x7f6f6806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 --2- 172.21.15.33:0/4251439915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6813fe70 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 -- 172.21.15.33:0/4251439915 >> 172.21.15.33:0/4251439915 conn(0x7f6f6806d030 msgr2=0x7f6f6806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 -- 172.21.15.33:0/4251439915 shutdown_connections 2023-10-06T05:27:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.042+0000 7f6f6759e700 1 -- 172.21.15.33:0/4251439915 wait complete. 2023-10-06T05:27:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.043+0000 7f6f6759e700 1 Processor -- start 2023-10-06T05:27:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.043+0000 7f6f6759e700 1 -- start start 2023-10-06T05:27:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.044+0000 7f6f6759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6806aad0 0x7f6f6813df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.044+0000 7f6f6759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f680c7940 0x7f6f6816a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.044+0000 7f6f6759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6816a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.044+0000 7f6f6759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6813e8d0 con 0x7f6f6813faa0 2023-10-06T05:27:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.044+0000 7f6f6759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6813ebb0 con 0x7f6f680c7940 2023-10-06T05:27:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.044+0000 7f6f6759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6813ee90 con 0x7f6f6806aad0 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6816a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6816a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51106/0 (socket says 172.21.15.33:51106) 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 -- 172.21.15.33:0/1024698503 learned_addr learned my addr 172.21.15.33:0/1024698503 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f6659c700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6806aad0 0x7f6f6813df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f65d9b700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f680c7940 0x7f6f6816a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 -- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6806aad0 msgr2=0x7f6f6813df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6806aad0 0x7f6f6813df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 -- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f680c7940 msgr2=0x7f6f6816a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f680c7940 0x7f6f6816a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 -- 172.21.15.33:0/1024698503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580094f0 con 0x7f6f6813faa0 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f6659c700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6806aad0 0x7f6f6813df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f65d9b700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f680c7940 0x7f6f6816a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f66d9d700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6816a8a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000d180 tx=0x7f6f60009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.045+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000de20 con 0x7f6f6813faa0 2023-10-06T05:27:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.046+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6000f040 con 0x7f6f6813faa0 2023-10-06T05:27:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.046+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6813f170 con 0x7f6f6813faa0 2023-10-06T05:27:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.046+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60015690 con 0x7f6f6813faa0 2023-10-06T05:27:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.046+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6813e660 con 0x7f6f6813faa0 2023-10-06T05:27:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.047+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6804c380 con 0x7f6f6813faa0 2023-10-06T05:27:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.047+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f6001e3b0 con 0x7f6f6813faa0 2023-10-06T05:27:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.049+0000 7f6f567fc700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f48066f30 0x7f6f480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.049+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6f600849f0 con 0x7f6f6813faa0 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.049+0000 7f6f6659c700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f48066f30 0x7f6f480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.050+0000 7f6f6659c700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f48066f30 0x7f6f480693e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6f68156b90 tx=0x7f6f58017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.053+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f60051d80 con 0x7f6f6813faa0 2023-10-06T05:27:46.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.336+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_timeout"]} v 0) v1 -- 0x7f6f680c9f00 con 0x7f6f6813faa0 2023-10-06T05:27:46.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.832+0000 7f6f567fc700 1 -- 172.21.15.33:0/1024698503 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_timeout"]}]=0 v677) v1 ==== 157+0+0 (secure 0 0 0) 0x7f6f6001a0e0 con 0x7f6f6813faa0 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f48066f30 msgr2=0x7f6f480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f48066f30 0x7f6f480693e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6f68156b90 tx=0x7f6f58017000 comp rx=0 tx=0).stop 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 msgr2=0x7f6f6816a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6816a8a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000d180 tx=0x7f6f60009a50 comp rx=0 tx=0).stop 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 shutdown_connections 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f48066f30 0x7f6f480693e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f6813faa0 0x7f6f6816a8a0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f6806aad0 0x7f6f6813df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 --2- 172.21.15.33:0/1024698503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f680c7940 0x7f6f6816a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.834+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 >> 172.21.15.33:0/1024698503 conn(0x7f6f6806d030 msgr2=0x7f6f680b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:46.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.835+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 shutdown_connections 2023-10-06T05:27:46.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:46.835+0000 7f6f6759e700 1 -- 172.21.15.33:0/1024698503 wait complete. 2023-10-06T05:27:46.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:27:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.048+0000 7f9feb59d700 1 Processor -- start 2023-10-06T05:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.049+0000 7f9feb59d700 1 -- start start 2023-10-06T05:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.050+0000 7f9feb59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9fe4156370 0x7f9fe40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.050+0000 7f9feb59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe40cc7e0 0x7f9fe40c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.050+0000 7f9feb59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe40ad430 con 0x7f9fe40b01f0 2023-10-06T05:27:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.050+0000 7f9feb59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe40afe20 con 0x7f9fe4156370 2023-10-06T05:27:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.050+0000 7f9feb59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe406bc70 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fead9c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9fe40b01f0 0x7f9fe40ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:51962/0 (socket says 172.21.15.33:51962) 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fead9c700 1 -- 172.21.15.33:0/2101865906 learned_addr learned my addr 172.21.15.33:0/2101865906 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9d9a700 1 --2- 172.21.15.33:0/2101865906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe40cc7e0 0x7f9fe40c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fea59b700 1 --2- 172.21.15.33:0/2101865906 >> v2:172.21.15.167:3300/0 conn(0x7f9fe4156370 0x7f9fe40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9599700 1 -- 172.21.15.33:0/2101865906 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538094264 0 0) 0x7f9fe40ad430 con 0x7f9fe40b01f0 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9599700 1 -- 172.21.15.33:0/2101865906 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd0003570 con 0x7f9fe40b01f0 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/2101865906 >> v2:172.21.15.167:3300/0 conn(0x7f9fe4156370 msgr2=0x7f9fe40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9d9a700 1 --2- 172.21.15.33:0/2101865906 >> v2:172.21.15.167:3300/0 conn(0x7f9fe4156370 0x7f9fe40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/2101865906 >> v1:172.21.15.167:6789/0 conn(0x7f9fe40b01f0 legacy=0x7f9fe40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/2101865906 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40c92f0 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.051+0000 7f9fea59b700 1 --2- 172.21.15.33:0/2101865906 >> v2:172.21.15.167:3300/0 conn(0x7f9fe4156370 0x7f9fe40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:47.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9fe9d9a700 1 --2- 172.21.15.33:0/2101865906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe40cc7e0 0x7f9fe40c8db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40098e0 tx=0x7f9fd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5beb94e01f83858 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9fe9599700 1 -- 172.21.15.33:0/2101865906 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4011630 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9fe9599700 1 -- 172.21.15.33:0/2101865906 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd4011790 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9fe9599700 1 -- 172.21.15.33:0/2101865906 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4011b30 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9feb59d700 1 -- 172.21.15.33:0/2101865906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe40cc7e0 msgr2=0x7f9fe40c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9feb59d700 1 --2- 172.21.15.33:0/2101865906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe40cc7e0 0x7f9fe40c8db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40098e0 tx=0x7f9fd40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9feb59d700 1 -- 172.21.15.33:0/2101865906 shutdown_connections 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9feb59d700 1 --2- 172.21.15.33:0/2101865906 >> v2:172.21.15.167:3300/0 conn(0x7f9fe4156370 0x7f9fe40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9feb59d700 1 --2- 172.21.15.33:0/2101865906 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe40cc7e0 0x7f9fe40c8db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.052+0000 7f9feb59d700 1 -- 172.21.15.33:0/2101865906 >> 172.21.15.33:0/2101865906 conn(0x7f9fe406d030 msgr2=0x7f9fe40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.053+0000 7f9feb59d700 1 -- 172.21.15.33:0/2101865906 shutdown_connections 2023-10-06T05:27:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.053+0000 7f9feb59d700 1 -- 172.21.15.33:0/2101865906 wait complete. 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.054+0000 7f9feb59d700 1 Processor -- start 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.054+0000 7f9feb59d700 1 -- start start 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9feb59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fe40b01f0 0x7f9fe4150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9feb59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 0x7f9fe4154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9feb59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe4156370 0x7f9fe4169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9feb59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4155140 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9feb59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4151f20 con 0x7f9fe40b01f0 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9feb59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4151bf0 con 0x7f9fe4156370 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9fe9d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 0x7f9fe4154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9fe9d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 0x7f9fe4154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51114/0 (socket says 172.21.15.33:51114) 2023-10-06T05:27:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.055+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/1692204113 learned_addr learned my addr 172.21.15.33:0/1692204113 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fead9c700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe4156370 0x7f9fe4169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fea59b700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fe40b01f0 0x7f9fe4150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe4156370 msgr2=0x7f9fe4169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fe9d9a700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe4156370 0x7f9fe4169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fe40b01f0 msgr2=0x7f9fe4150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fe9d9a700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fe40b01f0 0x7f9fe4150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fe9d9a700 1 -- 172.21.15.33:0/1692204113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0009420 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fe9d9a700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 0x7f9fe4154280 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4013000 tx=0x7f9fd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4011b10 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd40253f0 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd4009420 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.056+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4025790 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.057+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe4154e60 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.057+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4151a50 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.058+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fd4025b80 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.059+0000 7f9fda7fc700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fc8066f30 0x7f9fc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.059+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f9fd4085cf0 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.060+0000 7f9fea59b700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fc8066f30 0x7f9fc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.060+0000 7f9fea59b700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fc8066f30 0x7f9fc80693e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00093f0 tx=0x7f9fe0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.063+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fe4151a50 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.265+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9fe4151a50 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.269+0000 7f9fda7fc700 1 -- 172.21.15.33:0/1692204113 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9fe4151a50 con 0x7f9fe40cc7e0 2023-10-06T05:27:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fc8066f30 msgr2=0x7f9fc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fc8066f30 0x7f9fc80693e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00093f0 tx=0x7f9fe0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 msgr2=0x7f9fe4154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 0x7f9fe4154280 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4013000 tx=0x7f9fd4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 shutdown_connections 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fc8066f30 0x7f9fc80693e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fe4156370 0x7f9fe4169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fe40b01f0 0x7f9fe4150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 --2- 172.21.15.33:0/1692204113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fe40cc7e0 0x7f9fe4154280 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 >> 172.21.15.33:0/1692204113 conn(0x7f9fe406d030 msgr2=0x7f9fe406a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 shutdown_connections 2023-10-06T05:27:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.272+0000 7f9feb59d700 1 -- 172.21.15.33:0/1692204113 wait complete. 2023-10-06T05:27:47.291 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:27:47.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:47.381 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:47.467 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:27:47.467 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:27:47.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:47.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.653+0000 7f5c109fe700 1 Processor -- start 2023-10-06T05:27:47.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.653+0000 7f5c109fe700 1 -- start start 2023-10-06T05:27:47.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c109fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c159530 0x7f5c0c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c109fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5c0c06df70 0x7f5c0c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c109fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c06e3e0 con 0x7f5c0c0b0310 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c109fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b1e00 con 0x7f5c0c06df70 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c109fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c06f6b0 con 0x7f5c0c159530 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c0a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5c0c06df70 0x7f5c0c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.654+0000 7f5c0b59e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5c0c0b0310 0x7f5c0c159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55454/0 (socket says 172.21.15.33:55454) 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0b59e700 1 -- 172.21.15.33:0/2820242117 learned_addr learned my addr 172.21.15.33:0/2820242117 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0ad9d700 1 --2- 172.21.15.33:0/2820242117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c159530 0x7f5c0c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0a59c700 1 -- 172.21.15.33:0/2820242117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c159530 msgr2=0x7f5c0c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c09d9b700 1 -- 172.21.15.33:0/2820242117 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207723089 0 0) 0x7f5c0c06e3e0 con 0x7f5c0c0b0310 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0a59c700 1 --2- 172.21.15.33:0/2820242117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c159530 0x7f5c0c06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0a59c700 1 -- 172.21.15.33:0/2820242117 >> v1:172.21.15.33:6789/0 conn(0x7f5c0c0b0310 legacy=0x7f5c0c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0a59c700 1 -- 172.21.15.33:0/2820242117 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c06f220 con 0x7f5c0c06df70 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c0a59c700 1 --2- 172.21.15.33:0/2820242117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c06df70 0x7f5c0c0afdd0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f5c000098e0 tx=0x7f5c000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce99eb43fbe53211 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c09d9b700 1 -- 172.21.15.33:0/2820242117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00009e80 con 0x7f5c0c06df70 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.655+0000 7f5c09d9b700 1 -- 172.21.15.33:0/2820242117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000b860 con 0x7f5c0c06df70 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c09d9b700 1 -- 172.21.15.33:0/2820242117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000bbe0 con 0x7f5c0c06df70 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 -- 172.21.15.33:0/2820242117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c06df70 msgr2=0x7f5c0c0afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2820242117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c06df70 0x7f5c0c0afdd0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f5c000098e0 tx=0x7f5c000091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 -- 172.21.15.33:0/2820242117 shutdown_connections 2023-10-06T05:27:47.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2820242117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c159530 0x7f5c0c06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2820242117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c06df70 0x7f5c0c0afdd0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 -- 172.21.15.33:0/2820242117 >> 172.21.15.33:0/2820242117 conn(0x7f5c0c06c560 msgr2=0x7f5c0c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 -- 172.21.15.33:0/2820242117 shutdown_connections 2023-10-06T05:27:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.656+0000 7f5c109fe700 1 -- 172.21.15.33:0/2820242117 wait complete. 2023-10-06T05:27:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.658+0000 7f5c109fe700 1 Processor -- start 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.658+0000 7f5c109fe700 1 -- start start 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c109fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c06df70 0x7f5c0c14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c109fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0b0310 0x7f5c0c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c109fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 0x7f5c0c1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c109fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c169e00 con 0x7f5c0c159530 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c109fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c16a0d0 con 0x7f5c0c0b0310 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c109fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c16a3b0 con 0x7f5c0c06df70 2023-10-06T05:27:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 0x7f5c0c1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0ad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c06df70 0x7f5c0c14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 0x7f5c0c1569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51134/0 (socket says 172.21.15.33:51134) 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 -- 172.21.15.33:0/2517127981 learned_addr learned my addr 172.21.15.33:0/2517127981 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0a59c700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0b0310 0x7f5c0c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 -- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c06df70 msgr2=0x7f5c0c14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c06df70 0x7f5c0c14aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 -- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0b0310 msgr2=0x7f5c0c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0b0310 0x7f5c0c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.659+0000 7f5c0b59e700 1 -- 172.21.15.33:0/2517127981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00009420 con 0x7f5c0c159530 2023-10-06T05:27:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.660+0000 7f5c0b59e700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 0x7f5c0c1569a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400cc90 tx=0x7f5c04001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.660+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400a8b0 con 0x7f5c0c159530 2023-10-06T05:27:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.660+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400aa10 con 0x7f5c0c159530 2023-10-06T05:27:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.660+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c16a690 con 0x7f5c0c159530 2023-10-06T05:27:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.660+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400add0 con 0x7f5c0c159530 2023-10-06T05:27:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.660+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c14b530 con 0x7f5c0c159530 2023-10-06T05:27:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.661+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c04c380 con 0x7f5c0c159530 2023-10-06T05:27:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.662+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c04007300 con 0x7f5c0c159530 2023-10-06T05:27:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.663+0000 7f5bf2ffd700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bf8066f30 0x7f5bf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.663+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5c040847d0 con 0x7f5c0c159530 2023-10-06T05:27:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.663+0000 7f5c0ad9d700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bf8066f30 0x7f5bf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.664+0000 7f5c0ad9d700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bf8066f30 0x7f5bf80693e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc006fd0 tx=0x7f5bfc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.667+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c04051b40 con 0x7f5c0c159530 2023-10-06T05:27:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.913+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c0c0b42a0 con 0x7f5c0c159530 2023-10-06T05:27:47.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.914+0000 7f5bf2ffd700 1 -- 172.21.15.33:0/2517127981 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+3540 (secure 0 0 0) 0x7f5c04055160 con 0x7f5c0c159530 2023-10-06T05:27:47.915 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:47.916 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":141,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1469695805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1469695805},{"type":"v1","addr":"172.21.15.167:6836","nonce":1469695805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2207313329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2207313329},{"type":"v1","addr":"172.21.15.167:6837","nonce":2207313329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2295761811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2295761811},{"type":"v1","addr":"172.21.15.33:6834","nonce":2295761811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":9729,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2216821936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2216821936},{"type":"v1","addr":"172.21.15.33:6835","nonce":2216821936}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[]} 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bf8066f30 msgr2=0x7f5bf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bf8066f30 0x7f5bf80693e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc006fd0 tx=0x7f5bfc008040 comp rx=0 tx=0).stop 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 msgr2=0x7f5c0c1569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 0x7f5c0c1569a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400cc90 tx=0x7f5c04001a80 comp rx=0 tx=0).stop 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 shutdown_connections 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5bf8066f30 0x7f5bf80693e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c159530 0x7f5c0c1569a0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c06df70 0x7f5c0c14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 --2- 172.21.15.33:0/2517127981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0b0310 0x7f5c0c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 >> 172.21.15.33:0/2517127981 conn(0x7f5c0c06c560 msgr2=0x7f5c0c159990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 shutdown_connections 2023-10-06T05:27:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:47.916+0000 7f5c109fe700 1 -- 172.21.15.33:0/2517127981 wait complete. 2023-10-06T05:27:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 141 2023-10-06T05:27:47.936 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:27:47.936 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:27:47.936 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:27:47.936 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:47.937 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:27:47.937 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:27:47.937 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:27:47.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:47.937 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:27:47.937 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:27:47.937 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:27:47.938 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:47.938 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:27:47.938 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:27:47.938 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:27:47.938 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:47.949 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:27:47.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:27:47.951 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:27:47.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:27:47.953 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:27:47.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:27:47.954 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:27:47.954 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:27:47.994 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:27:47.995 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:47.998 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:27:47.998 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:47.999 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:27:47.999 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:48.002 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:27:48.002 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:48.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:27:48.132 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:27:48.133 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:27:48.136 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:27:48.137 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:27:48.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.197+0000 7ff3b759e700 1 Processor -- start 2023-10-06T05:27:48.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.197+0000 7ff3b759e700 1 -- start start 2023-10-06T05:27:48.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3b806cc70 0x7ff3b81644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80b1160 con 0x7ff3b8159400 2023-10-06T05:27:48.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80d0ed0 con 0x7ff3b81597d0 2023-10-06T05:27:48.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b80cb6d0 con 0x7ff3b806cc70 2023-10-06T05:27:48.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff3b8159400 0x7ff3b80cab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55474/0 (socket says 172.21.15.33:55474) 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b659c700 1 -- 172.21.15.33:0/755694402 learned_addr learned my addr 172.21.15.33:0/755694402 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b6d9d700 1 --2- 172.21.15.33:0/755694402 >> v2:172.21.15.167:3300/0 conn(0x7ff3b806cc70 0x7ff3b81644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b559a700 1 -- 172.21.15.33:0/755694402 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507307902 0 0) 0x7ff3b80d0ed0 con 0x7ff3b81597d0 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.199+0000 7ff3b559a700 1 -- 172.21.15.33:0/755694402 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a0003570 con 0x7ff3b81597d0 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b559a700 1 -- 172.21.15.33:0/755694402 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060227295 0 0) 0x7ff3a8001110 con 0x7ff3b8159400 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b6d9d700 1 -- 172.21.15.33:0/755694402 >> v1:172.21.15.167:6789/0 conn(0x7ff3b81597d0 legacy=0x7ff3b806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b6d9d700 1 -- 172.21.15.33:0/755694402 >> v1:172.21.15.33:6789/0 conn(0x7ff3b8159400 legacy=0x7ff3b80cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b6d9d700 1 -- 172.21.15.33:0/755694402 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80cb9b0 con 0x7ff3b806cc70 2023-10-06T05:27:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b6d9d700 1 --2- 172.21.15.33:0/755694402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b806cc70 0x7ff3b81644d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009e60 tx=0x7ff3b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7d1ddc5675ebf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b559a700 1 -- 172.21.15.33:0/755694402 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000e070 con 0x7ff3b806cc70 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b559a700 1 -- 172.21.15.33:0/755694402 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b000bb60 con 0x7ff3b806cc70 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.200+0000 7ff3b559a700 1 -- 172.21.15.33:0/755694402 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0012430 con 0x7ff3b806cc70 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 -- 172.21.15.33:0/755694402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b806cc70 msgr2=0x7ff3b81644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 --2- 172.21.15.33:0/755694402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b806cc70 0x7ff3b81644d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009e60 tx=0x7ff3b0009780 comp rx=0 tx=0).stop 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 -- 172.21.15.33:0/755694402 shutdown_connections 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 --2- 172.21.15.33:0/755694402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b806cc70 0x7ff3b81644d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 -- 172.21.15.33:0/755694402 >> 172.21.15.33:0/755694402 conn(0x7ff3b806e4b0 msgr2=0x7ff3b80d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 -- 172.21.15.33:0/755694402 shutdown_connections 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.201+0000 7ff3b759e700 1 -- 172.21.15.33:0/755694402 wait complete. 2023-10-06T05:27:48.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.202+0000 7ff3b759e700 1 Processor -- start 2023-10-06T05:27:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.202+0000 7ff3b759e700 1 -- start start 2023-10-06T05:27:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.203+0000 7ff3b759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3b806cc70 0x7ff3b80b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.203+0000 7ff3b759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 0x7ff3b814b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.203+0000 7ff3b759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b81597d0 0x7ff3b815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.203+0000 7ff3b759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b815de40 con 0x7ff3b8159400 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.203+0000 7ff3b759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b814b5a0 con 0x7ff3b81597d0 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.203+0000 7ff3b759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80b47e0 con 0x7ff3b806cc70 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 0x7ff3b814b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 0x7ff3b814b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51176/0 (socket says 172.21.15.33:51176) 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 -- 172.21.15.33:0/3420903479 learned_addr learned my addr 172.21.15.33:0/3420903479 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b659c700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3b806cc70 0x7ff3b80b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b6d9d700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b81597d0 0x7ff3b815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 -- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3b806cc70 msgr2=0x7ff3b80b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3b806cc70 0x7ff3b80b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 -- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b81597d0 msgr2=0x7ff3b815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b81597d0 0x7ff3b815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 -- 172.21.15.33:0/3420903479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009a20 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.204+0000 7ff3b5d9b700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 0x7ff3b814b060 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c790 tx=0x7ff3ac007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.205+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac010510 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.205+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac010670 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.205+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac010a10 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.205+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b815d7f0 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.205+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b815d4a0 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.206+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b815dfb0 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.207+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3ac01b3b0 con 0x7ff3b8159400 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.209+0000 7ff3a67fc700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff398066f30 0x7ff3980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.209+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff3ac083d50 con 0x7ff3b8159400 2023-10-06T05:27:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.209+0000 7ff3b659c700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff398066f30 0x7ff3980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.210+0000 7ff3b659c700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff398066f30 0x7ff3980693e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8006fc0 tx=0x7ff3a8007b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.213+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3ac051040 con 0x7ff3b8159400 2023-10-06T05:27:48.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.430+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3b804c380 con 0x7ff3b8159400 2023-10-06T05:27:48.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.849+0000 7ff3a67fc700 1 -- 172.21.15.33:0/3420903479 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v189) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3ac017020 con 0x7ff3b8159400 2023-10-06T05:27:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff398066f30 msgr2=0x7ff3980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff398066f30 0x7ff3980693e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8006fc0 tx=0x7ff3a8007b90 comp rx=0 tx=0).stop 2023-10-06T05:27:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 msgr2=0x7ff3b814b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 0x7ff3b814b060 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00c790 tx=0x7ff3ac007e10 comp rx=0 tx=0).stop 2023-10-06T05:27:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 shutdown_connections 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff398066f30 0x7ff3980693e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3b81597d0 0x7ff3b815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3b806cc70 0x7ff3b80b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 --2- 172.21.15.33:0/3420903479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3b8159400 0x7ff3b814b060 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 >> 172.21.15.33:0/3420903479 conn(0x7ff3b806e4b0 msgr2=0x7ff3b80c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 shutdown_connections 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:48.852+0000 7ff3b759e700 1 -- 172.21.15.33:0/3420903479 wait complete. 2023-10-06T05:27:48.855 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:27:48.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:27:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.060+0000 7f551359d700 1 Processor -- start 2023-10-06T05:27:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.060+0000 7f551359d700 1 -- start start 2023-10-06T05:27:49.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551359d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f550c0ccfc0 0x7f550c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551359d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f550c0cc750 0x7f550c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551359d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c0cbd80 con 0x7f550c0cf3b0 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551359d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c0af5f0 con 0x7f550c0ccfc0 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551359d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c0af8d0 con 0x7f550c0cc750 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f550c0ccfc0 0x7f550c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f550c0ccfc0 0x7f550c0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51180/0 (socket says 172.21.15.33:51180) 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f551259b700 1 -- 172.21.15.33:0/153881490 learned_addr learned my addr 172.21.15.33:0/153881490 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f5511d9a700 1 --1- 172.21.15.33:0/153881490 >> v1:172.21.15.33:6789/0 conn(0x7f550c0cf3b0 0x7f550c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55484/0 (socket says 172.21.15.33:55484) 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.062+0000 7f5512d9c700 1 --2- 172.21.15.33:0/153881490 >> v2:172.21.15.167:3300/0 conn(0x7f550c0cc750 0x7f550c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f551259b700 1 -- 172.21.15.33:0/153881490 >> v2:172.21.15.167:3300/0 conn(0x7f550c0cc750 msgr2=0x7f550c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f551259b700 1 --2- 172.21.15.33:0/153881490 >> v2:172.21.15.167:3300/0 conn(0x7f550c0cc750 0x7f550c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f551259b700 1 -- 172.21.15.33:0/153881490 >> v1:172.21.15.33:6789/0 conn(0x7f550c0cf3b0 legacy=0x7f550c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f551259b700 1 -- 172.21.15.33:0/153881490 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0afbb0 con 0x7f550c0ccfc0 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f5511599700 1 -- 172.21.15.33:0/153881490 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041177160 0 0) 0x7f550c0cbd80 con 0x7f550c0cf3b0 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f551259b700 1 --2- 172.21.15.33:0/153881490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0ccfc0 0x7f550c0cc210 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f55080099b0 tx=0x7f5508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175e48eb0091ae8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f5511599700 1 -- 172.21.15.33:0/153881490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508009ea0 con 0x7f550c0ccfc0 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f5511599700 1 -- 172.21.15.33:0/153881490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550800b820 con 0x7f550c0ccfc0 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.063+0000 7f5511599700 1 -- 172.21.15.33:0/153881490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550800bbc0 con 0x7f550c0ccfc0 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 -- 172.21.15.33:0/153881490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0ccfc0 msgr2=0x7f550c0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 --2- 172.21.15.33:0/153881490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0ccfc0 0x7f550c0cc210 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f55080099b0 tx=0x7f5508009250 comp rx=0 tx=0).stop 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 -- 172.21.15.33:0/153881490 shutdown_connections 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 --2- 172.21.15.33:0/153881490 >> v2:172.21.15.167:3300/0 conn(0x7f550c0cc750 0x7f550c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 --2- 172.21.15.33:0/153881490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0ccfc0 0x7f550c0cc210 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 -- 172.21.15.33:0/153881490 >> 172.21.15.33:0/153881490 conn(0x7f550c06cff0 msgr2=0x7f550c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 -- 172.21.15.33:0/153881490 shutdown_connections 2023-10-06T05:27:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.064+0000 7f551359d700 1 -- 172.21.15.33:0/153881490 wait complete. 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.065+0000 7f551359d700 1 Processor -- start 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.065+0000 7f551359d700 1 -- start start 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.066+0000 7f551359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f550c0cc750 0x7f550c14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.066+0000 7f551359d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f550c0ccfc0 0x7f550c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.066+0000 7f551359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 0x7f550c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.066+0000 7f551359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0c77e0 con 0x7f550c0b37e0 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.066+0000 7f551359d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b3d60 con 0x7f550c0ccfc0 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.066+0000 7f551359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b4040 con 0x7f550c0cc750 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 0x7f550c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 0x7f550c0b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51182/0 (socket says 172.21.15.33:51182) 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 -- 172.21.15.33:0/2404247360 learned_addr learned my addr 172.21.15.33:0/2404247360 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5511d9a700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f550c0ccfc0 0x7f550c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f551259b700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f550c0cc750 0x7f550c14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 -- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f550c0cc750 msgr2=0x7f550c14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f550c0cc750 0x7f550c14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 -- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f550c0ccfc0 msgr2=0x7f550c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f550c0ccfc0 0x7f550c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 -- 172.21.15.33:0/2404247360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55080094f0 con 0x7f550c0b37e0 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f5512d9c700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 0x7f550c0b49d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5504009e90 tx=0x7f550400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400cc20 con 0x7f550c0b37e0 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400f040 con 0x7f550c0b37e0 2023-10-06T05:27:49.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504014480 con 0x7f550c0b37e0 2023-10-06T05:27:49.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.067+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c0b4320 con 0x7f550c0b37e0 2023-10-06T05:27:49.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.068+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c14d6d0 con 0x7f550c0b37e0 2023-10-06T05:27:49.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.069+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550c0c75b0 con 0x7f550c0b37e0 2023-10-06T05:27:49.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.069+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5504014830 con 0x7f550c0b37e0 2023-10-06T05:27:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.070+0000 7f55027fc700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54f0066f30 0x7f54f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.070+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5504084e00 con 0x7f550c0b37e0 2023-10-06T05:27:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.071+0000 7f551259b700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54f0066f30 0x7f54f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.071+0000 7f551259b700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54f0066f30 0x7f54f00693e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f55080099b0 tx=0x7f5508017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.074+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f550c0c75b0 con 0x7f550c0b37e0 2023-10-06T05:27:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.350+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f550c0c75b0 con 0x7f550c0b37e0 2023-10-06T05:27:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.351+0000 7f55027fc700 1 -- 172.21.15.33:0/2404247360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f550c0c75b0 con 0x7f550c0b37e0 2023-10-06T05:27:49.354 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:49.355 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:27:49.356 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:27:49.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.356 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:27:49.357 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:27:49.357 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.358 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.358 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:49.358 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:49.359 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.359 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.359 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:27:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:27:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.361 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:27:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:27:49.363 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.363 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.363 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:49.363 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:49.363 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:27:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:27:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.366 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:27:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:27:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.368 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:27:49.368 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:27:49.368 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:27:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:27:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.371 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.371 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.372 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:27:49.372 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:27:49.373 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.373 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.374 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.374 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.374 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.378 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:27:49.380 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:27:49.381 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:27:49.382 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:27:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:27:49.384 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.384 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:49.384 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:27:49.384 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:27:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54f0066f30 msgr2=0x7f54f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54f0066f30 0x7f54f00693e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f55080099b0 tx=0x7f5508017000 comp rx=0 tx=0).stop 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 msgr2=0x7f550c0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 0x7f550c0b49d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5504009e90 tx=0x7f550400b040 comp rx=0 tx=0).stop 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 shutdown_connections 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54f0066f30 0x7f54f00693e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f550c0b37e0 0x7f550c0b49d0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f550c0cc750 0x7f550c14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 --2- 172.21.15.33:0/2404247360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f550c0ccfc0 0x7f550c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 >> 172.21.15.33:0/2404247360 conn(0x7f550c06cff0 msgr2=0x7f550c158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.353+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 shutdown_connections 2023-10-06T05:27:49.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.354+0000 7f551359d700 1 -- 172.21.15.33:0/2404247360 wait complete. 2023-10-06T05:27:49.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:49.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.577+0000 7fc48ffff700 1 Processor -- start 2023-10-06T05:27:49.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.577+0000 7fc48ffff700 1 -- start start 2023-10-06T05:27:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.578+0000 7fc48ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc49013faa0 0x7fc49013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4900c9cb0 con 0x7fc4900c7940 2023-10-06T05:27:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4900ca270 con 0x7fc49006aad0 2023-10-06T05:27:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49006b660 con 0x7fc49013faa0 2023-10-06T05:27:49.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48e7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc4900c7940 0x7fc4900c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55488/0 (socket says 172.21.15.33:55488) 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48e7fc700 1 -- 172.21.15.33:0/61415374 learned_addr learned my addr 172.21.15.33:0/61415374 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48effd700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc49013faa0 0x7fc49013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224647207 0 0) 0x7fc4900c9cb0 con 0x7fc4900c7940 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc478003570 con 0x7fc4900c7940 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638744428 0 0) 0x7fc4900ca270 con 0x7fc49006aad0 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.579+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4900c9cb0 con 0x7fc49006aad0 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533442503 0 0) 0x7fc478003570 con 0x7fc4900c7940 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc49013faa0 msgr2=0x7fc49013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc49013faa0 0x7fc49013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 >> v1:172.21.15.167:6789/0 conn(0x7fc49006aad0 legacy=0x7fc49006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49006b840 con 0x7fc4900c7940 2023-10-06T05:27:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc484001470 con 0x7fc4900c7940 2023-10-06T05:27:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 >> v1:172.21.15.33:6789/0 conn(0x7fc4900c7940 legacy=0x7fc4900c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478003c20 0x7fc478004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478004810 0x7fc478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc47800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49006b660 con 0x7fc4780051c0 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800e230 con 0x7fc478004810 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800e3f0 con 0x7fc478003c20 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48effd700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc49013faa0 0x7fc49013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48f7fe700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc47800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48e7fc700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478004810 0x7fc478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.580+0000 7fc48effd700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478003c20 0x7fc478004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48f7fe700 1 -- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478003c20 msgr2=0x7fc478004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48f7fe700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478003c20 0x7fc478004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48f7fe700 1 -- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478004810 msgr2=0x7fc478004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48f7fe700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc478004810 0x7fc478004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48f7fe700 1 -- 172.21.15.33:0/61415374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc478003570 con 0x7fc4780051c0 2023-10-06T05:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48f7fe700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc47800ba60 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fc488002120 tx=0x7fc4880017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ae723c26e2b86e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488001ab0 con 0x7fc4780051c0 2023-10-06T05:27:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48800c980 con 0x7fc4780051c0 2023-10-06T05:27:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.581+0000 7fc48dffb700 1 -- 172.21.15.33:0/61415374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800cd20 con 0x7fc4780051c0 2023-10-06T05:27:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48ffff700 1 -- 172.21.15.33:0/61415374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 msgr2=0x7fc47800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48ffff700 1 --2- 172.21.15.33:0/61415374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc47800ba60 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fc488002120 tx=0x7fc4880017a0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48f7fe700 1 -- 172.21.15.33:0/61415374 reap_dead start 2023-10-06T05:27:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48ffff700 1 -- 172.21.15.33:0/61415374 shutdown_connections 2023-10-06T05:27:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48ffff700 1 -- 172.21.15.33:0/61415374 >> 172.21.15.33:0/61415374 conn(0x7fc49006d030 msgr2=0x7fc49006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48ffff700 1 -- 172.21.15.33:0/61415374 shutdown_connections 2023-10-06T05:27:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.582+0000 7fc48ffff700 1 -- 172.21.15.33:0/61415374 wait complete. 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.583+0000 7fc48ffff700 1 Processor -- start 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.583+0000 7fc48ffff700 1 -- start start 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc4900c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4901561a0 0x7fc4900c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc490156610 0x7fc490165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900c4ff0 con 0x7fc4780051c0 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900c52d0 con 0x7fc4901561a0 2023-10-06T05:27:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900c55b0 con 0x7fc490156610 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc4900c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc4900c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51198/0 (socket says 172.21.15.33:51198) 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.584+0000 7fc48effd700 1 -- 172.21.15.33:0/1510525289 learned_addr learned my addr 172.21.15.33:0/1510525289 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48e7fc700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4901561a0 0x7fc4900c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48f7fe700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc490156610 0x7fc490165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48effd700 1 -- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc490156610 msgr2=0x7fc490165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48effd700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc490156610 0x7fc490165440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48effd700 1 -- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4901561a0 msgr2=0x7fc4900c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48effd700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4901561a0 0x7fc4900c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48effd700 1 -- 172.21.15.33:0/1510525289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900c5890 con 0x7fc4780051c0 2023-10-06T05:27:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48f7fe700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc490156610 0x7fc490165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48effd700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc4900c6090 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fc480011d20 tx=0x7fc4800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48000fb50 con 0x7fc4780051c0 2023-10-06T05:27:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48000fcb0 con 0x7fc4780051c0 2023-10-06T05:27:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4900c5b70 con 0x7fc4780051c0 2023-10-06T05:27:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.585+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48001d690 con 0x7fc4780051c0 2023-10-06T05:27:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.586+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4900c4ae0 con 0x7fc4780051c0 2023-10-06T05:27:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.586+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4900c4d60 con 0x7fc4780051c0 2023-10-06T05:27:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.587+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc48001dac0 con 0x7fc4780051c0 2023-10-06T05:27:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.589+0000 7fc476ffd700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc47c066f30 0x7fc47c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.589+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc480018070 con 0x7fc4780051c0 2023-10-06T05:27:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.589+0000 7fc48e7fc700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc47c066f30 0x7fc47c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.590+0000 7fc48e7fc700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc47c066f30 0x7fc47c0693e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc484001200 tx=0x7fc484009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.593+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc480052700 con 0x7fc4780051c0 2023-10-06T05:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.795+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc49004c380 con 0x7fc4780051c0 2023-10-06T05:27:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.796+0000 7fc476ffd700 1 -- 172.21.15.33:0/1510525289 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fc480055d20 con 0x7fc4780051c0 2023-10-06T05:27:49.798 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":190,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:48.849966+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.798+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc47c066f30 msgr2=0x7fc47c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.798+0000 7fc48ffff700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc47c066f30 0x7fc47c0693e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc484001200 tx=0x7fc484009040 comp rx=0 tx=0).stop 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.798+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 msgr2=0x7fc4900c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.798+0000 7fc48ffff700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc4900c6090 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fc480011d20 tx=0x7fc4800091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 shutdown_connections 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc47c066f30 0x7fc47c0693e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc490156610 0x7fc490165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4780051c0 0x7fc4900c6090 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 --2- 172.21.15.33:0/1510525289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4901561a0 0x7fc4900c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 >> 172.21.15.33:0/1510525289 conn(0x7fc49006d030 msgr2=0x7fc49013ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 shutdown_connections 2023-10-06T05:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:49.799+0000 7fc48ffff700 1 -- 172.21.15.33:0/1510525289 wait complete. 2023-10-06T05:27:49.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:27:49.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:27:50.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.011+0000 7f5af179d700 1 Processor -- start 2023-10-06T05:27:50.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.011+0000 7f5af179d700 1 -- start start 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5af179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5aec154f20 0x7f5aec0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5af179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5aec1552f0 0x7f5aec0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5af179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec1586a0 con 0x7f5aec0b11e0 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5af179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec06bdf0 con 0x7f5aec154f20 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5af179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec157ab0 con 0x7f5aec1552f0 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5aebfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5aec154f20 0x7f5aec0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5aebfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5aec154f20 0x7f5aec0ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34448/0 (socket says 172.21.15.33:34448) 2023-10-06T05:27:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.012+0000 7f5aebfff700 1 -- 172.21.15.33:0/212891659 learned_addr learned my addr 172.21.15.33:0/212891659 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aeb7fe700 1 --2- 172.21.15.33:0/212891659 >> v2:172.21.15.167:3300/0 conn(0x7f5aec1552f0 0x7f5aec0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aebfff700 1 -- 172.21.15.33:0/212891659 >> v2:172.21.15.167:3300/0 conn(0x7f5aec1552f0 msgr2=0x7f5aec0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aebfff700 1 --2- 172.21.15.33:0/212891659 >> v2:172.21.15.167:3300/0 conn(0x7f5aec1552f0 0x7f5aec0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aebfff700 1 -- 172.21.15.33:0/212891659 >> v1:172.21.15.167:6789/0 conn(0x7f5aec0b11e0 legacy=0x7f5aec0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aebfff700 1 -- 172.21.15.33:0/212891659 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec157d70 con 0x7f5aec154f20 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aeb7fe700 1 --2- 172.21.15.33:0/212891659 >> v2:172.21.15.167:3300/0 conn(0x7f5aec1552f0 0x7f5aec0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aebfff700 1 --2- 172.21.15.33:0/212891659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec0ae720 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0b21a0 tx=0x7f5adc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5721542d3df3c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aeaffd700 1 -- 172.21.15.33:0/212891659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00b820 con 0x7f5aec154f20 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aeaffd700 1 -- 172.21.15.33:0/212891659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc00b980 con 0x7f5aec154f20 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.013+0000 7f5aeaffd700 1 -- 172.21.15.33:0/212891659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00bd20 con 0x7f5aec154f20 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 -- 172.21.15.33:0/212891659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 msgr2=0x7f5aec0ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 --2- 172.21.15.33:0/212891659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec0ae720 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0b21a0 tx=0x7f5adc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 -- 172.21.15.33:0/212891659 shutdown_connections 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 --2- 172.21.15.33:0/212891659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec0ae720 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 --2- 172.21.15.33:0/212891659 >> v2:172.21.15.167:3300/0 conn(0x7f5aec1552f0 0x7f5aec0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 -- 172.21.15.33:0/212891659 >> 172.21.15.33:0/212891659 conn(0x7f5aec06d030 msgr2=0x7f5aec0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 -- 172.21.15.33:0/212891659 shutdown_connections 2023-10-06T05:27:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.014+0000 7f5af179d700 1 -- 172.21.15.33:0/212891659 wait complete. 2023-10-06T05:27:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.015+0000 7f5af179d700 1 Processor -- start 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 -- start start 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0b11e0 0x7f5aec16a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec069a30 0x7f5aec06af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec16ac00 con 0x7f5aec154f20 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06aad0 con 0x7f5aec069a30 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.016+0000 7f5af179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec153a10 con 0x7f5aec0b11e0 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec1534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34454/0 (socket says 172.21.15.33:34454) 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 -- 172.21.15.33:0/2223240269 learned_addr learned my addr 172.21.15.33:0/2223240269 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aebfff700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0b11e0 0x7f5aec16a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 -- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0b11e0 msgr2=0x7f5aec16a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0b11e0 0x7f5aec16a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 -- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec069a30 msgr2=0x7f5aec06af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5af0f9c700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec069a30 0x7f5aec06af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec069a30 0x7f5aec06af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 -- 172.21.15.33:0/2223240269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0094f0 con 0x7f5aec154f20 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aebfff700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0b11e0 0x7f5aec16a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5af0f9c700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec069a30 0x7f5aec06af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.017+0000 7f5aeb7fe700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec1534d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c830 tx=0x7f5ae0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.018+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0011600 con 0x7f5aec154f20 2023-10-06T05:27:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.018+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0011760 con 0x7f5aec154f20 2023-10-06T05:27:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.018+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec069ec0 con 0x7f5aec154f20 2023-10-06T05:27:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.018+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0011b20 con 0x7f5aec154f20 2023-10-06T05:27:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.018+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec06a480 con 0x7f5aec154f20 2023-10-06T05:27:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.019+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec04c380 con 0x7f5aec154f20 2023-10-06T05:27:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.020+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ae000a3e0 con 0x7f5aec154f20 2023-10-06T05:27:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.021+0000 7f5ad3fff700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ad8066f30 0x7f5ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.021+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5ae00848a0 con 0x7f5aec154f20 2023-10-06T05:27:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.021+0000 7f5aebfff700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ad8066f30 0x7f5ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.022+0000 7f5aebfff700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ad8066f30 0x7f5ad80693e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0b21a0 tx=0x7f5adc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.025+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ae0051c60 con 0x7f5aec154f20 2023-10-06T05:27:50.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:50.111 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.231+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5aec06ac10 con 0x7f5aec154f20 2023-10-06T05:27:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.231+0000 7f5ad3fff700 1 -- 172.21.15.33:0/2223240269 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ae00150a0 con 0x7f5aec154f20 2023-10-06T05:27:50.233 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:27:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ad8066f30 msgr2=0x7f5ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ad8066f30 0x7f5ad80693e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0b21a0 tx=0x7f5adc016000 comp rx=0 tx=0).stop 2023-10-06T05:27:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 msgr2=0x7f5aec1534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec1534d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c830 tx=0x7f5ae0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:50.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 shutdown_connections 2023-10-06T05:27:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5ad8066f30 0x7f5ad80693e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aec069a30 0x7f5aec06af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5aec0b11e0 0x7f5aec16a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 --2- 172.21.15.33:0/2223240269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aec154f20 0x7f5aec1534d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 >> 172.21.15.33:0/2223240269 conn(0x7f5aec06d030 msgr2=0x7f5aec0cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 shutdown_connections 2023-10-06T05:27:50.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.233+0000 7f5af179d700 1 -- 172.21.15.33:0/2223240269 wait complete. 2023-10-06T05:27:50.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:27:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.482+0000 7fa164d9d700 1 Processor -- start 2023-10-06T05:27:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.483+0000 7fa164d9d700 1 -- start start 2023-10-06T05:27:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa164d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa160159850 0x7fa16006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa164d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa16015acd0 0x7fa16006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa164d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600b0cd0 con 0x7fa16006d140 2023-10-06T05:27:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa164d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16015a840 con 0x7fa160159850 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa164d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa160159f10 con 0x7fa16015acd0 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa15f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa160159850 0x7fa16006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa15f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa160159850 0x7fa16006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34460/0 (socket says 172.21.15.33:34460) 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.484+0000 7fa15f7fe700 1 -- 172.21.15.33:0/4210210781 learned_addr learned my addr 172.21.15.33:0/4210210781 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15effd700 1 --2- 172.21.15.33:0/4210210781 >> v2:172.21.15.167:3300/0 conn(0x7fa16015acd0 0x7fa16006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15f7fe700 1 -- 172.21.15.33:0/4210210781 >> v2:172.21.15.167:3300/0 conn(0x7fa16015acd0 msgr2=0x7fa16006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15f7fe700 1 --2- 172.21.15.33:0/4210210781 >> v2:172.21.15.167:3300/0 conn(0x7fa16015acd0 0x7fa16006edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15f7fe700 1 -- 172.21.15.33:0/4210210781 >> v1:172.21.15.167:6789/0 conn(0x7fa16006d140 legacy=0x7fa16006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15f7fe700 1 -- 172.21.15.33:0/4210210781 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16015a1f0 con 0x7fa160159850 2023-10-06T05:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15f7fe700 1 --2- 172.21.15.33:0/4210210781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa160159850 0x7fa16006e890 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fa16006a730 tx=0x7fa150009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b02105d1d5564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15e7fc700 1 -- 172.21.15.33:0/4210210781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150009e70 con 0x7fa160159850 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15e7fc700 1 -- 172.21.15.33:0/4210210781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15000b860 con 0x7fa160159850 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.485+0000 7fa15e7fc700 1 -- 172.21.15.33:0/4210210781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15000bc00 con 0x7fa160159850 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 -- 172.21.15.33:0/4210210781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa160159850 msgr2=0x7fa16006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 --2- 172.21.15.33:0/4210210781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa160159850 0x7fa16006e890 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fa16006a730 tx=0x7fa150009250 comp rx=0 tx=0).stop 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 -- 172.21.15.33:0/4210210781 shutdown_connections 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 --2- 172.21.15.33:0/4210210781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa160159850 0x7fa16006e890 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 --2- 172.21.15.33:0/4210210781 >> v2:172.21.15.167:3300/0 conn(0x7fa16015acd0 0x7fa16006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 -- 172.21.15.33:0/4210210781 >> 172.21.15.33:0/4210210781 conn(0x7fa16006cd40 msgr2=0x7fa1600ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 -- 172.21.15.33:0/4210210781 shutdown_connections 2023-10-06T05:27:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.486+0000 7fa164d9d700 1 -- 172.21.15.33:0/4210210781 wait complete. 2023-10-06T05:27:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.487+0000 7fa164d9d700 1 Processor -- start 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 -- start start 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa16006d140 0x7fa1600cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa160159850 0x7fa1600abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 0x7fa1600ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160169b90 con 0x7fa16015acd0 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160169e70 con 0x7fa160159850 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.488+0000 7fa164d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16016a150 con 0x7fa16006d140 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 0x7fa1600ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 0x7fa1600ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34468/0 (socket says 172.21.15.33:34468) 2023-10-06T05:27:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 -- 172.21.15.33:0/693840255 learned_addr learned my addr 172.21.15.33:0/693840255 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15effd700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa160159850 0x7fa1600abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15f7fe700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa16006d140 0x7fa1600cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 -- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa16006d140 msgr2=0x7fa1600cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa16006d140 0x7fa1600cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 -- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa160159850 msgr2=0x7fa1600abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa160159850 0x7fa1600abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 -- 172.21.15.33:0/693840255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500094f0 con 0x7fa16015acd0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.489+0000 7fa15ffff700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 0x7fa1600ac390 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c810 tx=0x7fa158001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.490+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158012630 con 0x7fa16015acd0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.490+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158012790 con 0x7fa16015acd0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.490+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16016a430 con 0x7fa16015acd0 2023-10-06T05:27:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.490+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158012b50 con 0x7fa16015acd0 2023-10-06T05:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.490+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa160169900 con 0x7fa16015acd0 2023-10-06T05:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.491+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16004c380 con 0x7fa16015acd0 2023-10-06T05:27:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.492+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa158003030 con 0x7fa16015acd0 2023-10-06T05:27:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.493+0000 7fa1477fe700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa14c066f30 0x7fa14c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.493+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa1580842f0 con 0x7fa16015acd0 2023-10-06T05:27:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.493+0000 7fa15f7fe700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa14c066f30 0x7fa14c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.494+0000 7fa15f7fe700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa14c066f30 0x7fa14c0693e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa16006a730 tx=0x7fa150017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.497+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1580516b0 con 0x7fa16015acd0 2023-10-06T05:27:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.698+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa16014e5f0 con 0x7fa16015acd0 2023-10-06T05:27:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.699+0000 7fa1477fe700 1 -- 172.21.15.33:0/693840255 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v190) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa158054cd0 con 0x7fa16015acd0 2023-10-06T05:27:50.701 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:27:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa14c066f30 msgr2=0x7fa14c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa14c066f30 0x7fa14c0693e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa16006a730 tx=0x7fa150017000 comp rx=0 tx=0).stop 2023-10-06T05:27:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 msgr2=0x7fa1600ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 0x7fa1600ac390 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c810 tx=0x7fa158001650 comp rx=0 tx=0).stop 2023-10-06T05:27:50.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 shutdown_connections 2023-10-06T05:27:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa14c066f30 0x7fa14c0693e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa16015acd0 0x7fa1600ac390 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa16006d140 0x7fa1600cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 --2- 172.21.15.33:0/693840255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa160159850 0x7fa1600abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 >> 172.21.15.33:0/693840255 conn(0x7fa16006cd40 msgr2=0x7fa16015ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 shutdown_connections 2023-10-06T05:27:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.701+0000 7fa164d9d700 1 -- 172.21.15.33:0/693840255 wait complete. 2023-10-06T05:27:50.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:27:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.909+0000 7f8ab339d700 1 Processor -- start 2023-10-06T05:27:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.909+0000 7f8ab339d700 1 -- start start 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab339d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8aac0ccfc0 0x7f8aac0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0cf3b0 0x7f8aac1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab339d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8aac0cc750 0x7f8aac1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab339d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0cbd80 con 0x7f8aac0ccfc0 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab339d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0af5f0 con 0x7f8aac0cc750 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0af8d0 con 0x7f8aac0cf3b0 2023-10-06T05:27:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.910+0000 7f8ab239b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8aac0ccfc0 0x7f8aac0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8aac0ccfc0 0x7f8aac0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34482/0 (socket says 172.21.15.33:34482) 2023-10-06T05:27:50.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 -- 172.21.15.33:0/2209216445 learned_addr learned my addr 172.21.15.33:0/2209216445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab2b9c700 1 --2- 172.21.15.33:0/2209216445 >> v2:172.21.15.167:3300/0 conn(0x7f8aac0cc750 0x7f8aac1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab1b9a700 1 --2- 172.21.15.33:0/2209216445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0cf3b0 0x7f8aac1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 -- 172.21.15.33:0/2209216445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0cf3b0 msgr2=0x7f8aac1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 --2- 172.21.15.33:0/2209216445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0cf3b0 0x7f8aac1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 -- 172.21.15.33:0/2209216445 >> v2:172.21.15.167:3300/0 conn(0x7f8aac0cc750 msgr2=0x7f8aac1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 --2- 172.21.15.33:0/2209216445 >> v2:172.21.15.167:3300/0 conn(0x7f8aac0cc750 0x7f8aac1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 -- 172.21.15.33:0/2209216445 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0afbb0 con 0x7f8aac0ccfc0 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab2b9c700 1 --2- 172.21.15.33:0/2209216445 >> v2:172.21.15.167:3300/0 conn(0x7f8aac0cc750 0x7f8aac1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.911+0000 7f8ab239b700 1 --2- 172.21.15.33:0/2209216445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac0ccfc0 0x7f8aac0cc210 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0cdb60 tx=0x7f8aa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e248c69ea7fb92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab1399700 1 -- 172.21.15.33:0/2209216445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800b860 con 0x7f8aac0ccfc0 2023-10-06T05:27:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab1399700 1 -- 172.21.15.33:0/2209216445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa800b9c0 con 0x7f8aac0ccfc0 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab1399700 1 -- 172.21.15.33:0/2209216445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800bd60 con 0x7f8aac0ccfc0 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 -- 172.21.15.33:0/2209216445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac0ccfc0 msgr2=0x7f8aac0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 --2- 172.21.15.33:0/2209216445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac0ccfc0 0x7f8aac0cc210 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0cdb60 tx=0x7f8aa8009250 comp rx=0 tx=0).stop 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 -- 172.21.15.33:0/2209216445 shutdown_connections 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 --2- 172.21.15.33:0/2209216445 >> v2:172.21.15.167:3300/0 conn(0x7f8aac0cc750 0x7f8aac1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 --2- 172.21.15.33:0/2209216445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac0ccfc0 0x7f8aac0cc210 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 --2- 172.21.15.33:0/2209216445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0cf3b0 0x7f8aac1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 -- 172.21.15.33:0/2209216445 >> 172.21.15.33:0/2209216445 conn(0x7f8aac06cff0 msgr2=0x7f8aac06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 -- 172.21.15.33:0/2209216445 shutdown_connections 2023-10-06T05:27:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.912+0000 7f8ab339d700 1 -- 172.21.15.33:0/2209216445 wait complete. 2023-10-06T05:27:50.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.914+0000 7f8ab339d700 1 Processor -- start 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.914+0000 7f8ab339d700 1 -- start start 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8aac0cc750 0x7f8aac0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0ccfc0 0x7f8aac144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 0x7f8aac0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1456f0 con 0x7f8aac1452b0 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b4160 con 0x7f8aac0cc750 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b4440 con 0x7f8aac0ccfc0 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 0x7f8aac0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab2b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 0x7f8aac0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34494/0 (socket says 172.21.15.33:34494) 2023-10-06T05:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab2b9c700 1 -- 172.21.15.33:0/454365159 learned_addr learned my addr 172.21.15.33:0/454365159 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab239b700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8aac0cc750 0x7f8aac0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.915+0000 7f8ab1b9a700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0ccfc0 0x7f8aac144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab2b9c700 1 -- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0ccfc0 msgr2=0x7f8aac144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab2b9c700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0ccfc0 0x7f8aac144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab2b9c700 1 -- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8aac0cc750 msgr2=0x7f8aac0b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab2b9c700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8aac0cc750 0x7f8aac0b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab2b9c700 1 -- 172.21.15.33:0/454365159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80094f0 con 0x7f8aac1452b0 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab1b9a700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0ccfc0 0x7f8aac144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab2b9c700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 0x7f8aac0b33c0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d180 tx=0x7f8aa4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400dd10 con 0x7f8aac1452b0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400f040 con 0x7f8aac1452b0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4015650 con 0x7f8aac1452b0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac0b4720 con 0x7f8aac1452b0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.916+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac0b5960 con 0x7f8aac1452b0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.917+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac04c380 con 0x7f8aac1452b0 2023-10-06T05:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.918+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8aa4015a40 con 0x7f8aac1452b0 2023-10-06T05:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.919+0000 7f8aa27fc700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a90066f30 0x7f8a900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.919+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8aa4084c20 con 0x7f8aac1452b0 2023-10-06T05:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.920+0000 7f8ab239b700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a90066f30 0x7f8a900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.920+0000 7f8ab239b700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a90066f30 0x7f8a900693e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0cdb60 tx=0x7f8aa8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:50.923+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8aa4051fe0 con 0x7f8aac1452b0 2023-10-06T05:27:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.128+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8aac0b2e70 con 0x7f8aac1452b0 2023-10-06T05:27:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.129+0000 7f8aa27fc700 1 -- 172.21.15.33:0/454365159 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v190) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8aa4055600 con 0x7f8aac1452b0 2023-10-06T05:27:51.130 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:27:51.131 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:27:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a90066f30 msgr2=0x7f8a900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a90066f30 0x7f8a900693e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0cdb60 tx=0x7f8aa8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 msgr2=0x7f8aac0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 0x7f8aac0b33c0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d180 tx=0x7f8aa4009a50 comp rx=0 tx=0).stop 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 shutdown_connections 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a90066f30 0x7f8a900693e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aac1452b0 0x7f8aac0b33c0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8aac0cc750 0x7f8aac0b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 --2- 172.21.15.33:0/454365159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8aac0ccfc0 0x7f8aac144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 >> 172.21.15.33:0/454365159 conn(0x7f8aac06cff0 msgr2=0x7f8aac0ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 shutdown_connections 2023-10-06T05:27:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.131+0000 7f8ab339d700 1 -- 172.21.15.33:0/454365159 wait complete. 2023-10-06T05:27:51.150 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:27:51.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:27:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.340+0000 7fe31e19d700 1 Processor -- start 2023-10-06T05:27:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.340+0000 7fe31e19d700 1 -- start start 2023-10-06T05:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3180acd00 0x7fe3180aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe318155e70 0x7fe3180c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31e19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe318069ad0 con 0x7fe318155aa0 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe318069db0 con 0x7fe318155e70 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31806a090 con 0x7fe3180acd00 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31d19b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe318155aa0 0x7fe3180ac5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52158/0 (socket says 172.21.15.33:52158) 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31d19b700 1 -- 172.21.15.33:0/1034607798 learned_addr learned my addr 172.21.15.33:0/1034607798 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31c99a700 1 --2- 172.21.15.33:0/1034607798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3180acd00 0x7fe3180aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe31d99c700 1 --2- 172.21.15.33:0/1034607798 >> v2:172.21.15.167:3300/0 conn(0x7fe318155e70 0x7fe3180c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe30ffff700 1 -- 172.21.15.33:0/1034607798 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525924131 0 0) 0x7fe318069ad0 con 0x7fe318155aa0 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.342+0000 7fe30ffff700 1 -- 172.21.15.33:0/1034607798 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe304003570 con 0x7fe318155aa0 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe31c99a700 1 -- 172.21.15.33:0/1034607798 >> v2:172.21.15.167:3300/0 conn(0x7fe318155e70 msgr2=0x7fe3180c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe31c99a700 1 --2- 172.21.15.33:0/1034607798 >> v2:172.21.15.167:3300/0 conn(0x7fe318155e70 0x7fe3180c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe30ffff700 1 -- 172.21.15.33:0/1034607798 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280522634 0 0) 0x7fe304003570 con 0x7fe318155aa0 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe31c99a700 1 -- 172.21.15.33:0/1034607798 >> v1:172.21.15.33:6789/0 conn(0x7fe318155aa0 legacy=0x7fe3180ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe31c99a700 1 -- 172.21.15.33:0/1034607798 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31806a370 con 0x7fe3180acd00 2023-10-06T05:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe31c99a700 1 --2- 172.21.15.33:0/1034607798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3180acd00 0x7fe3180aef40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe3080098e0 tx=0x7fe3080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c59028dee09906f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe30ffff700 1 -- 172.21.15.33:0/1034607798 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314001470 con 0x7fe3180acd00 2023-10-06T05:27:51.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe30ffff700 1 -- 172.21.15.33:0/1034607798 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe308011420 con 0x7fe3180acd00 2023-10-06T05:27:51.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.343+0000 7fe30ffff700 1 -- 172.21.15.33:0/1034607798 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3080117c0 con 0x7fe3180acd00 2023-10-06T05:27:51.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 -- 172.21.15.33:0/1034607798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3180acd00 msgr2=0x7fe3180aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 --2- 172.21.15.33:0/1034607798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3180acd00 0x7fe3180aef40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe3080098e0 tx=0x7fe3080091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 -- 172.21.15.33:0/1034607798 shutdown_connections 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 --2- 172.21.15.33:0/1034607798 >> v2:172.21.15.167:3300/0 conn(0x7fe318155e70 0x7fe3180c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 --2- 172.21.15.33:0/1034607798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3180acd00 0x7fe3180aef40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 -- 172.21.15.33:0/1034607798 >> 172.21.15.33:0/1034607798 conn(0x7fe318068090 msgr2=0x7fe31806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 -- 172.21.15.33:0/1034607798 shutdown_connections 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.344+0000 7fe31e19d700 1 -- 172.21.15.33:0/1034607798 wait complete. 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.345+0000 7fe31e19d700 1 Processor -- start 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 -- start start 2023-10-06T05:27:51.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3180acd00 0x7fe3181543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe318155aa0 0x7fe3181589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 0x7fe3180c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180c1fe0 con 0x7fe318155e70 2023-10-06T05:27:51.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181548e0 con 0x7fe3180acd00 2023-10-06T05:27:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.346+0000 7fe31e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181576c0 con 0x7fe318155aa0 2023-10-06T05:27:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 0x7fe3180c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 0x7fe3180c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34498/0 (socket says 172.21.15.33:34498) 2023-10-06T05:27:51.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 -- 172.21.15.33:0/2748807290 learned_addr learned my addr 172.21.15.33:0/2748807290 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d19b700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3180acd00 0x7fe3181543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31c99a700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe318155aa0 0x7fe3181589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 -- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe318155aa0 msgr2=0x7fe3181589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe318155aa0 0x7fe3181589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 -- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3180acd00 msgr2=0x7fe3181543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3180acd00 0x7fe3181543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 -- 172.21.15.33:0/2748807290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009420 con 0x7fe318155e70 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.347+0000 7fe31d99c700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 0x7fe3180c3000 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fe310009ef0 tx=0x7fe31000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.348+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000cc50 con 0x7fe318155e70 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.348+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31000f040 con 0x7fe318155e70 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.348+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310014480 con 0x7fe318155e70 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.348+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe318158520 con 0x7fe318155e70 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.348+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe318157ed0 con 0x7fe318155e70 2023-10-06T05:27:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.349+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3181586b0 con 0x7fe318155e70 2023-10-06T05:27:51.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.350+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe310014830 con 0x7fe318155e70 2023-10-06T05:27:51.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.351+0000 7fe30d7fa700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe2fc066f30 0x7fe2fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.351+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe310085210 con 0x7fe318155e70 2023-10-06T05:27:51.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.351+0000 7fe31d19b700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe2fc066f30 0x7fe2fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.352+0000 7fe31d19b700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe2fc066f30 0x7fe2fc0693e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe314005c20 tx=0x7fe314001120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.355+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3100525d0 con 0x7fe318155e70 2023-10-06T05:27:51.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.558+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe318063ff0 con 0x7fe318155e70 2023-10-06T05:27:51.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.869+0000 7fe30d7fa700 1 -- 172.21.15.33:0/2748807290 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v191) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe310055bf0 con 0x7fe318155e70 2023-10-06T05:27:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe2fc066f30 msgr2=0x7fe2fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe2fc066f30 0x7fe2fc0693e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe314005c20 tx=0x7fe314001120 comp rx=0 tx=0).stop 2023-10-06T05:27:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 msgr2=0x7fe3180c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 0x7fe3180c3000 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fe310009ef0 tx=0x7fe31000b040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 shutdown_connections 2023-10-06T05:27:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe2fc066f30 0x7fe2fc0693e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe318155e70 0x7fe3180c3000 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3180acd00 0x7fe3181543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 --2- 172.21.15.33:0/2748807290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe318155aa0 0x7fe3181589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 >> 172.21.15.33:0/2748807290 conn(0x7fe318068090 msgr2=0x7fe3180a9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.872+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 shutdown_connections 2023-10-06T05:27:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:51.873+0000 7fe31e19d700 1 -- 172.21.15.33:0/2748807290 wait complete. 2023-10-06T05:27:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:27:51.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:27:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.087+0000 7f9ae8f9d700 1 Processor -- start 2023-10-06T05:27:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.087+0000 7f9ae8f9d700 1 -- start start 2023-10-06T05:27:52.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae4155bd0 0x7f9ae40c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae8f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ae40ac460 0x7f9ae40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae8f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae406aab0 con 0x7f9ae406af40 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae8f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae40acdf0 con 0x7f9ae40ac460 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae4069ea0 con 0x7f9ae4155bd0 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ae406af40 0x7f9ae406b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52160/0 (socket says 172.21.15.33:52160) 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae3fff700 1 -- 172.21.15.33:0/3716485981 learned_addr learned my addr 172.21.15.33:0/3716485981 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/3716485981 >> v2:172.21.15.167:3300/0 conn(0x7f9ae40ac460 0x7f9ae40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae4155bd0 0x7f9ae40c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.089+0000 7f9ae27fc700 1 -- 172.21.15.33:0/3716485981 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017380280 0 0) 0x7f9ae406aab0 con 0x7f9ae406af40 2023-10-06T05:27:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae27fc700 1 -- 172.21.15.33:0/3716485981 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc003570 con 0x7f9ae406af40 2023-10-06T05:27:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae2ffd700 1 -- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae4155bd0 msgr2=0x7f9ae40c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae4155bd0 0x7f9ae40c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae2ffd700 1 -- 172.21.15.33:0/3716485981 >> v1:172.21.15.33:6789/0 conn(0x7f9ae406af40 legacy=0x7f9ae406b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae2ffd700 1 -- 172.21.15.33:0/3716485981 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae406a180 con 0x7f9ae40ac460 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae4155bd0 0x7f9ae40c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.090+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40ac460 0x7f9ae40ac8b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098e0 tx=0x7f9ad80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9227336a4f116c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae27fc700 1 -- 172.21.15.33:0/3716485981 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8011630 con 0x7f9ae40ac460 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae27fc700 1 -- 172.21.15.33:0/3716485981 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8011790 con 0x7f9ae40ac460 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae27fc700 1 -- 172.21.15.33:0/3716485981 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8011b30 con 0x7f9ae40ac460 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40ac460 msgr2=0x7f9ae40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40ac460 0x7f9ae40ac8b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098e0 tx=0x7f9ad80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/3716485981 shutdown_connections 2023-10-06T05:27:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae4155bd0 0x7f9ae40c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/3716485981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40ac460 0x7f9ae40ac8b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/3716485981 >> 172.21.15.33:0/3716485981 conn(0x7f9ae406c560 msgr2=0x7f9ae406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/3716485981 shutdown_connections 2023-10-06T05:27:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.091+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/3716485981 wait complete. 2023-10-06T05:27:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.093+0000 7f9ae8f9d700 1 Processor -- start 2023-10-06T05:27:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.093+0000 7f9ae8f9d700 1 -- start start 2023-10-06T05:27:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae8f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 0x7f9ae40af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ac460 0x7f9ae40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae8f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae4155bd0 0x7f9ae4169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae8f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b0460 con 0x7f9ae406af40 2023-10-06T05:27:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae8f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b0710 con 0x7f9ae4155bd0 2023-10-06T05:27:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae416ab60 con 0x7f9ae40ac460 2023-10-06T05:27:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 0x7f9ae40af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 0x7f9ae40af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34502/0 (socket says 172.21.15.33:34502) 2023-10-06T05:27:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.094+0000 7f9ae37fe700 1 -- 172.21.15.33:0/1707615846 learned_addr learned my addr 172.21.15.33:0/1707615846 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae3fff700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae4155bd0 0x7f9ae4169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ac460 0x7f9ae40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae37fe700 1 -- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ac460 msgr2=0x7f9ae40aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ac460 0x7f9ae40aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae37fe700 1 -- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae4155bd0 msgr2=0x7f9ae4169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae4155bd0 0x7f9ae4169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae37fe700 1 -- 172.21.15.33:0/1707615846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009420 con 0x7f9ae406af40 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ac460 0x7f9ae40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 0x7f9ae40af9e0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400ec80 tx=0x7f9ad40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400c9d0 con 0x7f9ae406af40 2023-10-06T05:27:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400cb30 con 0x7f9ae406af40 2023-10-06T05:27:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8009420 con 0x7f9ae406af40 2023-10-06T05:27:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400ceb0 con 0x7f9ae406af40 2023-10-06T05:27:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.095+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae416b120 con 0x7f9ae406af40 2023-10-06T05:27:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.097+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae404c380 con 0x7f9ae406af40 2023-10-06T05:27:52.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.098+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ad4013680 con 0x7f9ae406af40 2023-10-06T05:27:52.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.099+0000 7f9acb7fe700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.099+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f9ad40848f0 con 0x7f9ae406af40 2023-10-06T05:27:52.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.099+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.100+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40cc980 tx=0x7f9ad8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.103+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad4051b40 con 0x7f9ae406af40 2023-10-06T05:27:52.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.305+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9ae406dec0 con 0x7f9ae406af40 2023-10-06T05:27:52.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.877+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1707615846 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v192) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9ad4055160 con 0x7f9ae406af40 2023-10-06T05:27:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.885+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 msgr2=0x7f9ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.885+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40cc980 tx=0x7f9ad8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.885+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 msgr2=0x7f9ae40af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.885+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 0x7f9ae40af9e0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400ec80 tx=0x7f9ad40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.886+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 shutdown_connections 2023-10-06T05:27:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.887+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40cc980 tx=0x7f9ad8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.887+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae4155bd0 0x7f9ae4169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.887+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae406af40 0x7f9ae40af9e0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.887+0000 7f9ae8f9d700 1 --2- 172.21.15.33:0/1707615846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ac460 0x7f9ae40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.887+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 >> 172.21.15.33:0/1707615846 conn(0x7f9ae406c560 msgr2=0x7f9ae40c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.888+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 shutdown_connections 2023-10-06T05:27:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:52.889+0000 7f9ae8f9d700 1 -- 172.21.15.33:0/1707615846 wait complete. 2023-10-06T05:27:52.895 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:27:52.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:27:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.105+0000 7fd21499d700 1 Processor -- start 2023-10-06T05:27:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.106+0000 7fd21499d700 1 -- start start 2023-10-06T05:27:53.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd21499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2101552f0 0x7fd2100ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd21499d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2100b18f0 0x7fd2100cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd21499d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2101586a0 con 0x7fd210154f20 2023-10-06T05:27:53.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd21499d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21006bdf0 con 0x7fd2100b18f0 2023-10-06T05:27:53.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd21499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd210157ab0 con 0x7fd2101552f0 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd20ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2100b18f0 0x7fd2100cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd20effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2101552f0 0x7fd2100ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.107+0000 7fd20f7fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd210154f20 0x7fd2100b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:39094/0 (socket says 172.21.15.33:39094) 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20f7fe700 1 -- 172.21.15.33:0/1546685216 learned_addr learned my addr 172.21.15.33:0/1546685216 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20e7fc700 1 -- 172.21.15.33:0/1546685216 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483119482 0 0) 0x7fd2101586a0 con 0x7fd210154f20 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20e7fc700 1 -- 172.21.15.33:0/1546685216 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f8003570 con 0x7fd210154f20 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20effd700 1 -- 172.21.15.33:0/1546685216 >> v2:172.21.15.167:3300/0 conn(0x7fd2100b18f0 msgr2=0x7fd2100cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20effd700 1 --2- 172.21.15.33:0/1546685216 >> v2:172.21.15.167:3300/0 conn(0x7fd2100b18f0 0x7fd2100cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20effd700 1 -- 172.21.15.33:0/1546685216 >> v1:172.21.15.167:6789/0 conn(0x7fd210154f20 legacy=0x7fd2100b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20effd700 1 -- 172.21.15.33:0/1546685216 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210157d70 con 0x7fd2101552f0 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20ffff700 1 --2- 172.21.15.33:0/1546685216 >> v2:172.21.15.167:3300/0 conn(0x7fd2100b18f0 0x7fd2100cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.108+0000 7fd20effd700 1 --2- 172.21.15.33:0/1546685216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2101552f0 0x7fd2100ae720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098e0 tx=0x7fd2040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af345e72f17ae726 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd20e7fc700 1 -- 172.21.15.33:0/1546685216 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011630 con 0x7fd2101552f0 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd20e7fc700 1 -- 172.21.15.33:0/1546685216 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204011790 con 0x7fd2101552f0 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd20e7fc700 1 -- 172.21.15.33:0/1546685216 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011b30 con 0x7fd2101552f0 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 -- 172.21.15.33:0/1546685216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2101552f0 msgr2=0x7fd2100ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 --2- 172.21.15.33:0/1546685216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2101552f0 0x7fd2100ae720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098e0 tx=0x7fd2040091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 -- 172.21.15.33:0/1546685216 shutdown_connections 2023-10-06T05:27:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 --2- 172.21.15.33:0/1546685216 >> v2:172.21.15.167:3300/0 conn(0x7fd2100b18f0 0x7fd2100cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 --2- 172.21.15.33:0/1546685216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2101552f0 0x7fd2100ae720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 -- 172.21.15.33:0/1546685216 >> 172.21.15.33:0/1546685216 conn(0x7fd21006d030 msgr2=0x7fd2100cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 -- 172.21.15.33:0/1546685216 shutdown_connections 2023-10-06T05:27:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.109+0000 7fd21499d700 1 -- 172.21.15.33:0/1546685216 wait complete. 2023-10-06T05:27:53.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.111+0000 7fd21499d700 1 Processor -- start 2023-10-06T05:27:53.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.111+0000 7fd21499d700 1 -- start start 2023-10-06T05:27:53.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd21499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2100b18f0 0x7fd210147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd21499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 0x7fd21015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd21499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2100af4e0 0x7fd21015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd21499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015d190 con 0x7fd210154f20 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd21499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21006abc0 con 0x7fd2100af4e0 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd21499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015f0f0 con 0x7fd2100b18f0 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd20effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 0x7fd21015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd20effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 0x7fd21015e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34516/0 (socket says 172.21.15.33:34516) 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd20effd700 1 -- 172.21.15.33:0/460354449 learned_addr learned my addr 172.21.15.33:0/460354449 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd20f7fe700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2100b18f0 0x7fd210147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.112+0000 7fd20ffff700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2100af4e0 0x7fd21015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd20effd700 1 -- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2100b18f0 msgr2=0x7fd210147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd20effd700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2100b18f0 0x7fd210147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd20effd700 1 -- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2100af4e0 msgr2=0x7fd21015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd20effd700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2100af4e0 0x7fd21015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd20effd700 1 -- 172.21.15.33:0/460354449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009a20 con 0x7fd210154f20 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd20effd700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 0x7fd21015e670 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd210157bf0 tx=0x7fd204017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011b10 con 0x7fd210154f20 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204000cc0 con 0x7fd210154f20 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd204009420 con 0x7fd210154f20 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.113+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204023600 con 0x7fd210154f20 2023-10-06T05:27:53.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.114+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21006a570 con 0x7fd210154f20 2023-10-06T05:27:53.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.115+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21004c380 con 0x7fd210154f20 2023-10-06T05:27:53.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.116+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd20402c3b0 con 0x7fd210154f20 2023-10-06T05:27:53.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.117+0000 7fd1f77fe700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1fc066f60 0x7fd1fc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.117+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fd204085e50 con 0x7fd210154f20 2023-10-06T05:27:53.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.117+0000 7fd20f7fe700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1fc066f60 0x7fd1fc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.118+0000 7fd20f7fe700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1fc066f60 0x7fd1fc069410 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd200007fd0 tx=0x7fd200009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.121+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2040530e0 con 0x7fd210154f20 2023-10-06T05:27:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.372+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd21006f370 con 0x7fd210154f20 2023-10-06T05:27:53.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.875+0000 7fd1f77fe700 1 -- 172.21.15.33:0/460354449 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v144) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd204056700 con 0x7fd210154f20 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.878+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1fc066f60 msgr2=0x7fd1fc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.878+0000 7fd21499d700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1fc066f60 0x7fd1fc069410 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd200007fd0 tx=0x7fd200009040 comp rx=0 tx=0).stop 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.878+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 msgr2=0x7fd21015e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.878+0000 7fd21499d700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 0x7fd21015e670 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd210157bf0 tx=0x7fd204017000 comp rx=0 tx=0).stop 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 shutdown_connections 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1fc066f60 0x7fd1fc069410 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2100af4e0 0x7fd21015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2100b18f0 0x7fd210147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 --2- 172.21.15.33:0/460354449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd210154f20 0x7fd21015e670 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 >> 172.21.15.33:0/460354449 conn(0x7fd21006d030 msgr2=0x7fd2100caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 shutdown_connections 2023-10-06T05:27:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:53.879+0000 7fd21499d700 1 -- 172.21.15.33:0/460354449 wait complete. 2023-10-06T05:27:53.883 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 29 and data pool 30 2023-10-06T05:27:53.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:54.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.088+0000 7f039139d700 1 Processor -- start 2023-10-06T05:27:54.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.088+0000 7f039139d700 1 -- start start 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f039139d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f038c155140 0x7f038c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f039139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f039139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c0c8e40 con 0x7f038c0adf50 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f039139d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c06a380 con 0x7f038c155140 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f039139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0c99e0 con 0x7f038c0699c0 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f038bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f038c155140 0x7f038c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f038bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f038c155140 0x7f038c158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34526/0 (socket says 172.21.15.33:34526) 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f038bfff700 1 -- 172.21.15.33:0/911250015 learned_addr learned my addr 172.21.15.33:0/911250015 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.090+0000 7f0390b9c700 1 --2- 172.21.15.33:0/911250015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038bfff700 1 -- 172.21.15.33:0/911250015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 msgr2=0x7f038c069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038bfff700 1 --2- 172.21.15.33:0/911250015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038bfff700 1 -- 172.21.15.33:0/911250015 >> v1:172.21.15.167:6789/0 conn(0x7f038c0adf50 legacy=0x7f038c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038bfff700 1 -- 172.21.15.33:0/911250015 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0ae3f0 con 0x7f038c155140 2023-10-06T05:27:54.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038bfff700 1 --2- 172.21.15.33:0/911250015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c158700 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f037c0099e0 tx=0x7f037c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04c80169bac52a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038affd700 1 -- 172.21.15.33:0/911250015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00b820 con 0x7f038c155140 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038affd700 1 -- 172.21.15.33:0/911250015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00b980 con 0x7f038c155140 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.091+0000 7f038affd700 1 -- 172.21.15.33:0/911250015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00bd20 con 0x7f038c155140 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 -- 172.21.15.33:0/911250015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 msgr2=0x7f038c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 --2- 172.21.15.33:0/911250015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c158700 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f037c0099e0 tx=0x7f037c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 -- 172.21.15.33:0/911250015 shutdown_connections 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 --2- 172.21.15.33:0/911250015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c158700 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 --2- 172.21.15.33:0/911250015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 -- 172.21.15.33:0/911250015 >> 172.21.15.33:0/911250015 conn(0x7f038c06d030 msgr2=0x7f038c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 -- 172.21.15.33:0/911250015 shutdown_connections 2023-10-06T05:27:54.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.092+0000 7f039139d700 1 -- 172.21.15.33:0/911250015 wait complete. 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.093+0000 7f039139d700 1 Processor -- start 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.094+0000 7f039139d700 1 -- start start 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.094+0000 7f039139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.094+0000 7f039139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038c0adf50 0x7f038c0c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.094+0000 7f039139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c14fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.094+0000 7f039139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0c38d0 con 0x7f038c155140 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f039139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0c3310 con 0x7f038c0adf50 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f039139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0c35f0 con 0x7f038c0699c0 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c14fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c14fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34528/0 (socket says 172.21.15.33:34528) 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 -- 172.21.15.33:0/3496641631 learned_addr learned my addr 172.21.15.33:0/3496641631 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f038bfff700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f038b7fe700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038c0adf50 0x7f038c0c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 -- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 msgr2=0x7f038c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 -- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038c0adf50 msgr2=0x7f038c0c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038c0adf50 0x7f038c0c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.095+0000 7f0390b9c700 1 -- 172.21.15.33:0/3496641631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c0094f0 con 0x7f038c155140 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.096+0000 7f0390b9c700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c14fb70 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f0384009df0 tx=0x7f038400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.096+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038400caa0 con 0x7f038c155140 2023-10-06T05:27:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.096+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038400f040 con 0x7f038c155140 2023-10-06T05:27:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.096+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c0b0830 con 0x7f038c155140 2023-10-06T05:27:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.096+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038400ce50 con 0x7f038c155140 2023-10-06T05:27:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.096+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c169f70 con 0x7f038c155140 2023-10-06T05:27:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.097+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c06edb0 con 0x7f038c155140 2023-10-06T05:27:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.098+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0384014690 con 0x7f038c155140 2023-10-06T05:27:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.099+0000 7f0373fff700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0378066f60 0x7f0378069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.099+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0384085160 con 0x7f038c155140 2023-10-06T05:27:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.099+0000 7f038bfff700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0378066f60 0x7f0378069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.100+0000 7f038bfff700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0378066f60 0x7f0378069410 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f037c0099e0 tx=0x7f037c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.103+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03840521d0 con 0x7f038c155140 2023-10-06T05:27:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.306+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f038c04c380 con 0x7f038c155140 2023-10-06T05:27:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.307+0000 7f0373fff700 1 -- 172.21.15.33:0/3496641631 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f03840557f0 con 0x7f038c155140 2023-10-06T05:27:54.309 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:54.310 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":193,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:53.374019+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:51.598576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:52.345977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:54.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0378066f60 msgr2=0x7f0378069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0378066f60 0x7f0378069410 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f037c0099e0 tx=0x7f037c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 msgr2=0x7f038c14fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c14fb70 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f0384009df0 tx=0x7f038400b040 comp rx=0 tx=0).stop 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 shutdown_connections 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0378066f60 0x7f0378069410 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f038c155140 0x7f038c14fb70 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f038c0699c0 0x7f038c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 --2- 172.21.15.33:0/3496641631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038c0adf50 0x7f038c0c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 >> 172.21.15.33:0/3496641631 conn(0x7f038c06d030 msgr2=0x7f038c0c7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 shutdown_connections 2023-10-06T05:27:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.310+0000 7f039139d700 1 -- 172.21.15.33:0/3496641631 wait complete. 2023-10-06T05:27:54.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:54.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.515+0000 7f42e159d700 1 Processor -- start 2023-10-06T05:27:54.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.515+0000 7f42e159d700 1 -- start start 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42e159d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42dc0b0a40 0x7f42dc06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42e159d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42dc0b0e10 0x7f42dc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42e159d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc15e2d0 con 0x7f42dc0b2e30 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42e159d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc15d6c0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42e159d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc15d9a0 con 0x7f42dc0b0e10 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42dbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42dc0b0a40 0x7f42dc06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42dbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42dc0b0a40 0x7f42dc06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34542/0 (socket says 172.21.15.33:34542) 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42dbfff700 1 -- 172.21.15.33:0/1254245348 learned_addr learned my addr 172.21.15.33:0/1254245348 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:54.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42db7fe700 1 --2- 172.21.15.33:0/1254245348 >> v2:172.21.15.167:3300/0 conn(0x7f42dc0b0e10 0x7f42dc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42dbfff700 1 -- 172.21.15.33:0/1254245348 >> v2:172.21.15.167:3300/0 conn(0x7f42dc0b0e10 msgr2=0x7f42dc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42dbfff700 1 --2- 172.21.15.33:0/1254245348 >> v2:172.21.15.167:3300/0 conn(0x7f42dc0b0e10 0x7f42dc0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.517+0000 7f42dbfff700 1 -- 172.21.15.33:0/1254245348 >> v1:172.21.15.167:6789/0 conn(0x7f42dc0b2e30 legacy=0x7f42dc15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.518+0000 7f42dbfff700 1 -- 172.21.15.33:0/1254245348 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc15dc80 con 0x7f42dc0b0a40 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.518+0000 7f42db7fe700 1 --2- 172.21.15.33:0/1254245348 >> v2:172.21.15.167:3300/0 conn(0x7f42dc0b0e10 0x7f42dc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.518+0000 7f42dbfff700 1 --2- 172.21.15.33:0/1254245348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc06ab00 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f42dc065020 tx=0x7f42cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11aa6f3b52695ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.518+0000 7f42daffd700 1 -- 172.21.15.33:0/1254245348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00b820 con 0x7f42dc0b0a40 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.518+0000 7f42daffd700 1 -- 172.21.15.33:0/1254245348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc00b980 con 0x7f42dc0b0a40 2023-10-06T05:27:54.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.518+0000 7f42daffd700 1 -- 172.21.15.33:0/1254245348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00bd40 con 0x7f42dc0b0a40 2023-10-06T05:27:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 -- 172.21.15.33:0/1254245348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 msgr2=0x7f42dc06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 --2- 172.21.15.33:0/1254245348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc06ab00 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f42dc065020 tx=0x7f42cc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 -- 172.21.15.33:0/1254245348 shutdown_connections 2023-10-06T05:27:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 --2- 172.21.15.33:0/1254245348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc06ab00 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 --2- 172.21.15.33:0/1254245348 >> v2:172.21.15.167:3300/0 conn(0x7f42dc0b0e10 0x7f42dc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 -- 172.21.15.33:0/1254245348 >> 172.21.15.33:0/1254245348 conn(0x7f42dc06e0f0 msgr2=0x7f42dc0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 -- 172.21.15.33:0/1254245348 shutdown_connections 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.519+0000 7f42e159d700 1 -- 172.21.15.33:0/1254245348 wait complete. 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.520+0000 7f42e159d700 1 Processor -- start 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.520+0000 7f42e159d700 1 -- start start 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.521+0000 7f42e159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc0c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.521+0000 7f42e159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42dc0b0e10 0x7f42dc15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.521+0000 7f42e159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42dc0b2e30 0x7f42dc158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.521+0000 7f42e159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc16b0b0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.521+0000 7f42e159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c3040 con 0x7f42dc0b0e10 2023-10-06T05:27:54.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.521+0000 7f42e159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c2510 con 0x7f42dc0b2e30 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc0c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc0c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34556/0 (socket says 172.21.15.33:34556) 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 -- 172.21.15.33:0/4290509809 learned_addr learned my addr 172.21.15.33:0/4290509809 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42e0d9c700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42dc0b2e30 0x7f42dc158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42db7fe700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42dc0b0e10 0x7f42dc15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 -- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42dc0b2e30 msgr2=0x7f42dc158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42dc0b2e30 0x7f42dc158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 -- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42dc0b0e10 msgr2=0x7f42dc15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42dc0b0e10 0x7f42dc15c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 -- 172.21.15.33:0/4290509809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0094f0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42e0d9c700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42dc0b2e30 0x7f42dc158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42dbfff700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc0c1fd0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0ce290 tx=0x7f42cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.522+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc011d50 con 0x7f42dc0b0a40 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.523+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc020070 con 0x7f42dc0b0a40 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.523+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc01b6a0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.523+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc15ca80 con 0x7f42dc0b0a40 2023-10-06T05:27:54.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.523+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc16add0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.524+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc06fcc0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.524+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42cc022030 con 0x7f42dc0b0a40 2023-10-06T05:27:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.525+0000 7f42c3fff700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42c8066f30 0x7f42c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.525+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f42cc086570 con 0x7f42dc0b0a40 2023-10-06T05:27:54.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.526+0000 7f42db7fe700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42c8066f30 0x7f42c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.526+0000 7f42db7fe700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42c8066f30 0x7f42c80693e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f42d000ac00 tx=0x7f42d0006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.529+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42cc0535e0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.734+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42dc06fcc0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.735+0000 7f42c3fff700 1 -- 172.21.15.33:0/4290509809 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f42cc0092d0 con 0x7f42dc0b0a40 2023-10-06T05:27:54.737 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":193,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:53.374019+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:51.598576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:52.345977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42c8066f30 msgr2=0x7f42c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42c8066f30 0x7f42c80693e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f42d000ac00 tx=0x7f42d0006f90 comp rx=0 tx=0).stop 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 msgr2=0x7f42dc0c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc0c1fd0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0ce290 tx=0x7f42cc016000 comp rx=0 tx=0).stop 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 shutdown_connections 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42c8066f30 0x7f42c80693e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42dc0b2e30 0x7f42dc158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42dc0b0a40 0x7f42dc0c1fd0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 --2- 172.21.15.33:0/4290509809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42dc0b0e10 0x7f42dc15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 >> 172.21.15.33:0/4290509809 conn(0x7f42dc06e0f0 msgr2=0x7f42dc157900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 shutdown_connections 2023-10-06T05:27:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.738+0000 7f42e159d700 1 -- 172.21.15.33:0/4290509809 wait complete. 2023-10-06T05:27:54.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:27:54.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.949+0000 7f22db59e700 1 Processor -- start 2023-10-06T05:27:54.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.949+0000 7f22db59e700 1 -- start start 2023-10-06T05:27:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.950+0000 7f22db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0af540 0x7f22dc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.950+0000 7f22db59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f22dc0cd5e0 0x7f22dc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.950+0000 7f22db59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc15d660 con 0x7f22dc15a650 2023-10-06T05:27:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.950+0000 7f22db59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc0ce890 con 0x7f22dc0cd5e0 2023-10-06T05:27:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.950+0000 7f22db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc15aac0 con 0x7f22dc0af540 2023-10-06T05:27:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.950+0000 7f22dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f22dc0cd5e0 0x7f22dc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f22dc0cd5e0 0x7f22dc15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34572/0 (socket says 172.21.15.33:34572) 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 -- 172.21.15.33:0/3106329068 learned_addr learned my addr 172.21.15.33:0/3106329068 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 -- 172.21.15.33:0/3106329068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0af540 msgr2=0x7f22dc0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22da59c700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0af540 0x7f22dc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0af540 0x7f22dc0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 -- 172.21.15.33:0/3106329068 >> v1:172.21.15.167:6789/0 conn(0x7f22dc15a650 legacy=0x7f22dc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 -- 172.21.15.33:0/3106329068 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc15d890 con 0x7f22dc0cd5e0 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22da59c700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0af540 0x7f22dc0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22dad9d700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc0cd5e0 0x7f22dc15d120 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009f20 tx=0x7f22d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1257853f167c15ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.951+0000 7f22d959a700 1 -- 172.21.15.33:0/3106329068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400e070 con 0x7f22dc0cd5e0 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22d959a700 1 -- 172.21.15.33:0/3106329068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d400ba80 con 0x7f22dc0cd5e0 2023-10-06T05:27:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22d959a700 1 -- 172.21.15.33:0/3106329068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40092e0 con 0x7f22dc0cd5e0 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 -- 172.21.15.33:0/3106329068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc0cd5e0 msgr2=0x7f22dc15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc0cd5e0 0x7f22dc15d120 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009f20 tx=0x7f22d4009840 comp rx=0 tx=0).stop 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 -- 172.21.15.33:0/3106329068 shutdown_connections 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc0cd5e0 0x7f22dc15d120 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 --2- 172.21.15.33:0/3106329068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0af540 0x7f22dc0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 -- 172.21.15.33:0/3106329068 >> 172.21.15.33:0/3106329068 conn(0x7f22dc06d9b0 msgr2=0x7f22dc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 -- 172.21.15.33:0/3106329068 shutdown_connections 2023-10-06T05:27:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.952+0000 7f22db59e700 1 -- 172.21.15.33:0/3106329068 wait complete. 2023-10-06T05:27:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.954+0000 7f22db59e700 1 Processor -- start 2023-10-06T05:27:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.954+0000 7f22db59e700 1 -- start start 2023-10-06T05:27:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22db59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22dc0af540 0x7f22dc148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0cd5e0 0x7f22dc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22db59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc15a650 0x7f22dc0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22db59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0b4990 con 0x7f22dc15a650 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22db59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1494c0 con 0x7f22dc0af540 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc16a040 con 0x7f22dc0cd5e0 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22dad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc15a650 0x7f22dc0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22da59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22dc0af540 0x7f22dc148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22da59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22dc0af540 0x7f22dc148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:46102/0 (socket says 172.21.15.33:46102) 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22da59c700 1 -- 172.21.15.33:0/435237853 learned_addr learned my addr 172.21.15.33:0/435237853 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22d9d9b700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0cd5e0 0x7f22dc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22dad9d700 1 -- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0cd5e0 msgr2=0x7f22dc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22dad9d700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0cd5e0 0x7f22dc0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22dad9d700 1 -- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22dc0af540 msgr2=0x7f22dc148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22dad9d700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22dc0af540 0x7f22dc148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.955+0000 7f22dad9d700 1 -- 172.21.15.33:0/435237853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009ae0 con 0x7f22dc15a650 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.956+0000 7f22dad9d700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc15a650 0x7f22dc0c7580 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f22d4015000 tx=0x7f22d4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.956+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40094e0 con 0x7f22dc15a650 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.956+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4008d20 con 0x7f22dc15a650 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.956+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40266b0 con 0x7f22dc15a650 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.956+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc16a320 con 0x7f22dc15a650 2023-10-06T05:27:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.956+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc16a8e0 con 0x7f22dc15a650 2023-10-06T05:27:54.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.957+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc04c380 con 0x7f22dc15a650 2023-10-06T05:27:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.958+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22d4003040 con 0x7f22dc15a650 2023-10-06T05:27:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.959+0000 7f22ca7fc700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f22bc066f30 0x7f22bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.959+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f22d4086700 con 0x7f22dc15a650 2023-10-06T05:27:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.959+0000 7f22da59c700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f22bc066f30 0x7f22bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.960+0000 7f22da59c700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f22bc066f30 0x7f22bc0693e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:54.963+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22d4053820 con 0x7f22dc15a650 2023-10-06T05:27:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.211+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f22dc15db70 con 0x7f22dc15a650 2023-10-06T05:27:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.893+0000 7f22ca7fc700 1 -- 172.21.15.33:0/435237853 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v147) v1 ==== 118+0+0 (secure 0 0 0) 0x7f22d4056e40 con 0x7f22dc15a650 2023-10-06T05:27:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f22bc066f30 msgr2=0x7f22bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f22bc066f30 0x7f22bc0693e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2023-10-06T05:27:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc15a650 msgr2=0x7f22dc0c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc15a650 0x7f22dc0c7580 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f22d4015000 tx=0x7f22d4018000 comp rx=0 tx=0).stop 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 shutdown_connections 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f22bc066f30 0x7f22bc0693e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22dc15a650 0x7f22dc0c7580 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22dc0af540 0x7f22dc148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 --2- 172.21.15.33:0/435237853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22dc0cd5e0 0x7f22dc0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 >> 172.21.15.33:0/435237853 conn(0x7f22dc06d9b0 msgr2=0x7f22dc158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.896+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 shutdown_connections 2023-10-06T05:27:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:55.897+0000 7f22db59e700 1 -- 172.21.15.33:0/435237853 wait complete. 2023-10-06T05:27:55.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:27:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.112+0000 7f4c2bfff700 1 Processor -- start 2023-10-06T05:27:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.113+0000 7f4c2bfff700 1 -- start start 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c155a50 0x7f4c2c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c2c0c9490 0x7f4c2c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4c2c0af950 0x7f4c2c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c0b06b0 con 0x7f4c2c0c9490 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c0c99e0 con 0x7f4c2c0af950 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0c9cc0 con 0x7f4c2c155a50 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c2c0c9490 0x7f4c2c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2a7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c2c0c9490 0x7f4c2c0af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34578/0 (socket says 172.21.15.33:34578) 2023-10-06T05:27:56.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2a7fc700 1 -- 172.21.15.33:0/3360258853 learned_addr learned my addr 172.21.15.33:0/3360258853 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:56.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2affd700 1 --2- 172.21.15.33:0/3360258853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c155a50 0x7f4c2c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.114+0000 7f4c2b7fe700 1 --2- 172.21.15.33:0/3360258853 >> v2:172.21.15.167:3300/0 conn(0x7f4c2c0af950 0x7f4c2c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c2a7fc700 1 -- 172.21.15.33:0/3360258853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c155a50 msgr2=0x7f4c2c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:56.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c2a7fc700 1 --2- 172.21.15.33:0/3360258853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c155a50 0x7f4c2c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c2a7fc700 1 -- 172.21.15.33:0/3360258853 >> v2:172.21.15.167:3300/0 conn(0x7f4c2c0af950 msgr2=0x7f4c2c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c2a7fc700 1 --2- 172.21.15.33:0/3360258853 >> v2:172.21.15.167:3300/0 conn(0x7f4c2c0af950 0x7f4c2c0afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c2a7fc700 1 -- 172.21.15.33:0/3360258853 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0c9fa0 con 0x7f4c2c0c9490 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c2a7fc700 1 --2- 172.21.15.33:0/3360258853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0c9490 0x7f4c2c0af410 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f4c200099b0 tx=0x7f4c20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e21b67b7621595a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c29ffb700 1 -- 172.21.15.33:0/3360258853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20009ea0 con 0x7f4c2c0c9490 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c29ffb700 1 -- 172.21.15.33:0/3360258853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2000b820 con 0x7f4c2c0c9490 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.115+0000 7f4c29ffb700 1 -- 172.21.15.33:0/3360258853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000bbc0 con 0x7f4c2c0c9490 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 -- 172.21.15.33:0/3360258853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0c9490 msgr2=0x7f4c2c0af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/3360258853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0c9490 0x7f4c2c0af410 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f4c200099b0 tx=0x7f4c20009250 comp rx=0 tx=0).stop 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 -- 172.21.15.33:0/3360258853 shutdown_connections 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/3360258853 >> v2:172.21.15.167:3300/0 conn(0x7f4c2c0af950 0x7f4c2c0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/3360258853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c155a50 0x7f4c2c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/3360258853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0c9490 0x7f4c2c0af410 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 -- 172.21.15.33:0/3360258853 >> 172.21.15.33:0/3360258853 conn(0x7f4c2c06d030 msgr2=0x7f4c2c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 -- 172.21.15.33:0/3360258853 shutdown_connections 2023-10-06T05:27:56.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.116+0000 7f4c2bfff700 1 -- 172.21.15.33:0/3360258853 wait complete. 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.117+0000 7f4c2bfff700 1 Processor -- start 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.118+0000 7f4c2bfff700 1 -- start start 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.118+0000 7f4c2bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 0x7f4c2c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.118+0000 7f4c2bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c0c9490 0x7f4c2c154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c2c155a50 0x7f4c2c14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0a5cf0 con 0x7f4c2c0af950 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0a5fd0 con 0x7f4c2c155a50 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0a62b0 con 0x7f4c2c0c9490 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 0x7f4c2c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 0x7f4c2c0a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34580/0 (socket says 172.21.15.33:34580) 2023-10-06T05:27:56.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 -- 172.21.15.33:0/801491947 learned_addr learned my addr 172.21.15.33:0/801491947 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2a7fc700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c0c9490 0x7f4c2c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2b7fe700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c2c155a50 0x7f4c2c14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 -- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c0c9490 msgr2=0x7f4c2c154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c0c9490 0x7f4c2c154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 -- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c2c155a50 msgr2=0x7f4c2c14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c2c155a50 0x7f4c2c14eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 -- 172.21.15.33:0/801491947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200094f0 con 0x7f4c2c0af950 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.119+0000 7f4c2affd700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 0x7f4c2c0a5480 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00de70 tx=0x7f4c1c00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.120+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c009b00 con 0x7f4c2c0af950 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.120+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c009c80 con 0x7f4c2c0af950 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.120+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c0a6590 con 0x7f4c2c0af950 2023-10-06T05:27:56.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.120+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0173f0 con 0x7f4c2c0af950 2023-10-06T05:27:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.120+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c0a59c0 con 0x7f4c2c0af950 2023-10-06T05:27:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.121+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c04c380 con 0x7f4c2c0af950 2023-10-06T05:27:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.122+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c1c017760 con 0x7f4c2c0af950 2023-10-06T05:27:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.122+0000 7f4c12ffd700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c18066f30 0x7f4c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.123+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c1c084b40 con 0x7f4c2c0af950 2023-10-06T05:27:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.123+0000 7f4c2a7fc700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c18066f30 0x7f4c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.124+0000 7f4c2a7fc700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c18066f30 0x7f4c180693e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4c200099b0 tx=0x7f4c20017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.127+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c1c051bb0 con 0x7f4c2c0af950 2023-10-06T05:27:56.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.379+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4c2c06bb10 con 0x7f4c2c0af950 2023-10-06T05:27:56.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.905+0000 7f4c12ffd700 1 -- 172.21.15.33:0/801491947 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v148) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c1c0551d0 con 0x7f4c2c0af950 2023-10-06T05:27:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c18066f30 msgr2=0x7f4c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c18066f30 0x7f4c180693e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4c200099b0 tx=0x7f4c20017000 comp rx=0 tx=0).stop 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 msgr2=0x7f4c2c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 0x7f4c2c0a5480 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00de70 tx=0x7f4c1c00f040 comp rx=0 tx=0).stop 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 shutdown_connections 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4c18066f30 0x7f4c180693e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c2c155a50 0x7f4c2c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c0af950 0x7f4c2c0a5480 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 --2- 172.21.15.33:0/801491947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2c0c9490 0x7f4c2c154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 >> 172.21.15.33:0/801491947 conn(0x7f4c2c06d030 msgr2=0x7f4c2c0c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 shutdown_connections 2023-10-06T05:27:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:56.908+0000 7f4c2bfff700 1 -- 172.21.15.33:0/801491947 wait complete. 2023-10-06T05:27:56.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:57.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.119+0000 7f9e703ff700 1 Processor -- start 2023-10-06T05:27:57.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.120+0000 7f9e703ff700 1 -- start start 2023-10-06T05:27:57.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e703ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9e6813fb70 0x7f9e6806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e703ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680ca8d0 con 0x7f9e6806ba40 2023-10-06T05:27:57.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e703ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680cabb0 con 0x7f9e680af000 2023-10-06T05:27:57.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e703ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6806ad00 con 0x7f9e6813fb70 2023-10-06T05:27:57.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e6d99a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e6806ba40 0x7f9e680c9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52168/0 (socket says 172.21.15.33:52168) 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e6d99a700 1 -- 172.21.15.33:0/1224304040 learned_addr learned my addr 172.21.15.33:0/1224304040 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.121+0000 7f9e6e19b700 1 --2- 172.21.15.33:0/1224304040 >> v2:172.21.15.167:3300/0 conn(0x7f9e6813fb70 0x7f9e6806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423531953 0 0) 0x7f9e680ca8d0 con 0x7f9e6806ba40 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e54003570 con 0x7f9e6806ba40 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044746315 0 0) 0x7f9e680cabb0 con 0x7f9e680af000 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e680ca8d0 con 0x7f9e680af000 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203621667 0 0) 0x7f9e54003570 con 0x7f9e6806ba40 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 >> v2:172.21.15.167:3300/0 conn(0x7f9e6813fb70 msgr2=0x7f9e6806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 --2- 172.21.15.33:0/1224304040 >> v2:172.21.15.167:3300/0 conn(0x7f9e6813fb70 0x7f9e6806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 >> v1:172.21.15.167:6789/0 conn(0x7f9e680af000 legacy=0x7f9e680c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6806afe0 con 0x7f9e6806ba40 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9e58001470 con 0x7f9e6806ba40 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 >> v1:172.21.15.33:6789/0 conn(0x7f9e6806ba40 legacy=0x7f9e680c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54003c40 0x7f9e54004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e54004810 0x7f9e54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e54005210 0x7f9e5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6806ad00 con 0x7f9e54005210 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5400c130 con 0x7f9e54004810 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5400c2c0 con 0x7f9e54003c40 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.122+0000 7f9e6e19b700 1 --2- 172.21.15.33:0/1224304040 >> v2:172.21.15.167:3300/0 conn(0x7f9e6813fb70 0x7f9e6806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e54005210 0x7f9e5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6d99a700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e54004810 0x7f9e54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e19b700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54003c40 0x7f9e54004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 -- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54003c40 msgr2=0x7f9e54004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54003c40 0x7f9e54004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 -- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e54004810 msgr2=0x7f9e54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e54004810 0x7f9e54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 -- 172.21.15.33:0/1224304040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54003570 con 0x7f9e54005210 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 -- 172.21.15.33:0/1224304040 reap_dead start 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.123+0000 7f9e6e99c700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e54005210 0x7f9e5400ba60 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9e60002150 tx=0x7f9e600017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d612e9bd8f16f55a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000c940 con 0x7f9e54005210 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6000caa0 con 0x7f9e54005210 2023-10-06T05:27:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e6d199700 1 -- 172.21.15.33:0/1224304040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000cc10 con 0x7f9e54005210 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 -- 172.21.15.33:0/1224304040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e54005210 msgr2=0x7f9e5400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e54005210 0x7f9e5400ba60 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9e60002150 tx=0x7f9e600017a0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 -- 172.21.15.33:0/1224304040 shutdown_connections 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 --2- 172.21.15.33:0/1224304040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e54005210 0x7f9e5400ba60 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 -- 172.21.15.33:0/1224304040 >> 172.21.15.33:0/1224304040 conn(0x7f9e680675f0 msgr2=0x7f9e680679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 -- 172.21.15.33:0/1224304040 shutdown_connections 2023-10-06T05:27:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.124+0000 7f9e703ff700 1 -- 172.21.15.33:0/1224304040 wait complete. 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.126+0000 7f9e703ff700 1 Processor -- start 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.126+0000 7f9e703ff700 1 -- start start 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e703ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54005210 0x7f9e68155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e703ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 0x7f9e680c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e703ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e68169990 0x7f9e68169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e703ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68167230 con 0x7f9e680c7700 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e703ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6816b3d0 con 0x7f9e68169990 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e703ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680c80b0 con 0x7f9e54005210 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e6d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 0x7f9e680c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e6d99a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 0x7f9e680c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34610/0 (socket says 172.21.15.33:34610) 2023-10-06T05:27:57.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e6d99a700 1 -- 172.21.15.33:0/638280048 learned_addr learned my addr 172.21.15.33:0/638280048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e6e19b700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54005210 0x7f9e68155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e6e99c700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e68169990 0x7f9e68169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.127+0000 7f9e6d99a700 1 -- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54005210 msgr2=0x7f9e68155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e6d99a700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54005210 0x7f9e68155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e6d99a700 1 -- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e68169990 msgr2=0x7f9e68169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e6d99a700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e68169990 0x7f9e68169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e6d99a700 1 -- 172.21.15.33:0/638280048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68155e50 con 0x7f9e680c7700 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e6e99c700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e68169990 0x7f9e68169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e6d99a700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 0x7f9e680c7b70 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f9e580073c0 tx=0x7f9e58001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58007780 con 0x7f9e680c7700 2023-10-06T05:27:57.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58000f50 con 0x7f9e680c7700 2023-10-06T05:27:57.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800fac0 con 0x7f9e680c7700 2023-10-06T05:27:57.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6816a7c0 con 0x7f9e680c7700 2023-10-06T05:27:57.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.128+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6816ad80 con 0x7f9e680c7700 2023-10-06T05:27:57.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.129+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6816b560 con 0x7f9e680c7700 2023-10-06T05:27:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.130+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e5801d460 con 0x7f9e680c7700 2023-10-06T05:27:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.131+0000 7f9e5e7fc700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e4c066f30 0x7f9e4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.131+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9e58084750 con 0x7f9e680c7700 2023-10-06T05:27:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.132+0000 7f9e6e19b700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e4c066f30 0x7f9e4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.132+0000 7f9e6e19b700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e4c066f30 0x7f9e4c0693e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9e6813def0 tx=0x7f9e640091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.135+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e58055250 con 0x7f9e680c7700 2023-10-06T05:27:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.383+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e68063ff0 con 0x7f9e680c7700 2023-10-06T05:27:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.384+0000 7f9e5e7fc700 1 -- 172.21.15.33:0/638280048 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f9e58054de0 con 0x7f9e680c7700 2023-10-06T05:27:57.385 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:57.386 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":148,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142},{"gid":9801,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:27:56.896156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9810},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9810":{"gid":9810,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3335696661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3335696661},{"type":"v1","addr":"172.21.15.167:6837","nonce":3335696661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:27:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e4c066f30 msgr2=0x7f9e4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e4c066f30 0x7f9e4c0693e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9e6813def0 tx=0x7f9e640091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 msgr2=0x7f9e680c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 0x7f9e680c7b70 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f9e580073c0 tx=0x7f9e58001d20 comp rx=0 tx=0).stop 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 shutdown_connections 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e4c066f30 0x7f9e4c0693e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e68169990 0x7f9e68169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e54005210 0x7f9e68155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 --2- 172.21.15.33:0/638280048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680c7700 0x7f9e680c7b70 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.386+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 >> 172.21.15.33:0/638280048 conn(0x7f9e680675f0 msgr2=0x7f9e680cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.387+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 shutdown_connections 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.387+0000 7f9e703ff700 1 -- 172.21.15.33:0/638280048 wait complete. 2023-10-06T05:27:57.389 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 148 2023-10-06T05:27:57.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:57.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.592+0000 7f59e2fff700 1 Processor -- start 2023-10-06T05:27:57.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.592+0000 7f59e2fff700 1 -- start start 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e2fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0c9e10 0x7f59dc0ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e2fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59dc0af000 0x7f59dc0af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e2fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc0ca8d0 con 0x7f59dc13fb70 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e2fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc0af9a0 con 0x7f59dc0af000 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e2fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0cabb0 con 0x7f59dc0c9e10 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59dc0af000 0x7f59dc0af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e159c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59dc0af000 0x7f59dc0af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34620/0 (socket says 172.21.15.33:34620) 2023-10-06T05:27:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e159c700 1 -- 172.21.15.33:0/1065044533 learned_addr learned my addr 172.21.15.33:0/1065044533 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59dbfff700 1 --2- 172.21.15.33:0/1065044533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0c9e10 0x7f59dc0ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e159c700 1 -- 172.21.15.33:0/1065044533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0c9e10 msgr2=0x7f59dc0ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e159c700 1 --2- 172.21.15.33:0/1065044533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0c9e10 0x7f59dc0ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.594+0000 7f59e159c700 1 -- 172.21.15.33:0/1065044533 >> v1:172.21.15.33:6789/0 conn(0x7f59dc13fb70 legacy=0x7f59dc06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.595+0000 7f59e159c700 1 -- 172.21.15.33:0/1065044533 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc06ad00 con 0x7f59dc0af000 2023-10-06T05:27:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.595+0000 7f59e159c700 1 --2- 172.21.15.33:0/1065044533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc0af000 0x7f59dc0af460 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009f20 tx=0x7f59d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b2187fb204edef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.595+0000 7f59db7fe700 1 -- 172.21.15.33:0/1065044533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d400e070 con 0x7f59dc0af000 2023-10-06T05:27:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.595+0000 7f59db7fe700 1 -- 172.21.15.33:0/1065044533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d400ba80 con 0x7f59dc0af000 2023-10-06T05:27:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.595+0000 7f59db7fe700 1 -- 172.21.15.33:0/1065044533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40092e0 con 0x7f59dc0af000 2023-10-06T05:27:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 -- 172.21.15.33:0/1065044533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc0af000 msgr2=0x7f59dc0af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 --2- 172.21.15.33:0/1065044533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc0af000 0x7f59dc0af460 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009f20 tx=0x7f59d4009840 comp rx=0 tx=0).stop 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 -- 172.21.15.33:0/1065044533 shutdown_connections 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 --2- 172.21.15.33:0/1065044533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc0af000 0x7f59dc0af460 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 --2- 172.21.15.33:0/1065044533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0c9e10 0x7f59dc0ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 -- 172.21.15.33:0/1065044533 >> 172.21.15.33:0/1065044533 conn(0x7f59dc0675f0 msgr2=0x7f59dc0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 -- 172.21.15.33:0/1065044533 shutdown_connections 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.596+0000 7f59e2fff700 1 -- 172.21.15.33:0/1065044533 wait complete. 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.597+0000 7f59e2fff700 1 Processor -- start 2023-10-06T05:27:57.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.597+0000 7f59e2fff700 1 -- start start 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.598+0000 7f59e2fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0af000 0x7f59dc0a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.598+0000 7f59e2fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0c9e10 0x7f59dc13e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.598+0000 7f59e2fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 0x7f59dc1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.598+0000 7f59e2fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0c3390 con 0x7f59dc13fb70 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.598+0000 7f59e2fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0c3670 con 0x7f59dc0c9e10 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.598+0000 7f59e2fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0c3950 con 0x7f59dc0af000 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 0x7f59dc1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 0x7f59dc1593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34622/0 (socket says 172.21.15.33:34622) 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 -- 172.21.15.33:0/424912188 learned_addr learned my addr 172.21.15.33:0/424912188 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e0d9b700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0af000 0x7f59dc0a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59dbfff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0c9e10 0x7f59dc13e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 -- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0af000 msgr2=0x7f59dc0a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0af000 0x7f59dc0a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 -- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0c9e10 msgr2=0x7f59dc13e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0c9e10 0x7f59dc13e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 -- 172.21.15.33:0/424912188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d4009ae0 con 0x7f59dc13fb70 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.599+0000 7f59e159c700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 0x7f59dc1593b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06f510 tx=0x7f59d4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.600+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40032f0 con 0x7f59dc13fb70 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.600+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4008e80 con 0x7f59dc13fb70 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.600+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d401c640 con 0x7f59dc13fb70 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.600+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc0c3c30 con 0x7f59dc13fb70 2023-10-06T05:27:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.600+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc13eeb0 con 0x7f59dc13fb70 2023-10-06T05:27:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.601+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc0c4150 con 0x7f59dc13fb70 2023-10-06T05:27:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.602+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59d4003040 con 0x7f59dc13fb70 2023-10-06T05:27:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.603+0000 7f59d8ff9700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.603+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f59d40860a0 con 0x7f59dc13fb70 2023-10-06T05:27:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.603+0000 7f59e0d9b700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.604+0000 7f59e0d9b700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c0066f30 0x7f59c00693e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f59d0003860 tx=0x7f59d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.607+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59d40531b0 con 0x7f59dc13fb70 2023-10-06T05:27:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.813+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59dc063ff0 con 0x7f59dc13fb70 2023-10-06T05:27:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.814+0000 7f59d8ff9700 1 -- 172.21.15.33:0/424912188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f59d40567d0 con 0x7f59dc13fb70 2023-10-06T05:27:57.816 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:57.817 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":194,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:27:54.827626+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:51.598576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:52.345977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c0066f30 msgr2=0x7f59c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c0066f30 0x7f59c00693e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f59d0003860 tx=0x7f59d0009040 comp rx=0 tx=0).stop 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 msgr2=0x7f59dc1593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 0x7f59dc1593b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06f510 tx=0x7f59d4018000 comp rx=0 tx=0).stop 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 shutdown_connections 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59dc13fb70 0x7f59dc1593b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc0af000 0x7f59dc0a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 --2- 172.21.15.33:0/424912188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0c9e10 0x7f59dc13e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 >> 172.21.15.33:0/424912188 conn(0x7f59dc0675f0 msgr2=0x7f59dc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 shutdown_connections 2023-10-06T05:27:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:57.817+0000 7f59e2fff700 1 -- 172.21.15.33:0/424912188 wait complete. 2023-10-06T05:27:57.836 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:27:57.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:27:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.022+0000 7f98c89ff700 1 Processor -- start 2023-10-06T05:27:58.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.022+0000 7f98c89ff700 1 -- start start 2023-10-06T05:27:58.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.023+0000 7f98c89ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98c4157210 0x7f98c4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.023+0000 7f98c89ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c415a150 0x7f98c40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.023+0000 7f98c89ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c406dfb0 con 0x7f98c40b0340 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.023+0000 7f98c89ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c40b0780 con 0x7f98c4157210 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.023+0000 7f98c89ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c415a730 con 0x7f98c415a150 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98c4157210 0x7f98c4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98c4157210 0x7f98c4159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34626/0 (socket says 172.21.15.33:34626) 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 -- 172.21.15.33:0/1960634893 learned_addr learned my addr 172.21.15.33:0/1960634893 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c1d9b700 1 --2- 172.21.15.33:0/1960634893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c415a150 0x7f98c40afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 -- 172.21.15.33:0/1960634893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c415a150 msgr2=0x7f98c40afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:58.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 --2- 172.21.15.33:0/1960634893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c415a150 0x7f98c40afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 -- 172.21.15.33:0/1960634893 >> v1:172.21.15.33:6789/0 conn(0x7f98c40b0340 legacy=0x7f98c406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 -- 172.21.15.33:0/1960634893 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c415aa10 con 0x7f98c4157210 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.024+0000 7f98c259c700 1 --2- 172.21.15.33:0/1960634893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c4157210 0x7f98c4159c10 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f98b40099b0 tx=0x7f98b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b854816286bbe7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c159a700 1 -- 172.21.15.33:0/1960634893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4009ea0 con 0x7f98c4157210 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c159a700 1 -- 172.21.15.33:0/1960634893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b400b860 con 0x7f98c4157210 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c159a700 1 -- 172.21.15.33:0/1960634893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b400bc00 con 0x7f98c4157210 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 -- 172.21.15.33:0/1960634893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c4157210 msgr2=0x7f98c4159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 --2- 172.21.15.33:0/1960634893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c4157210 0x7f98c4159c10 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f98b40099b0 tx=0x7f98b4009250 comp rx=0 tx=0).stop 2023-10-06T05:27:58.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 -- 172.21.15.33:0/1960634893 shutdown_connections 2023-10-06T05:27:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 --2- 172.21.15.33:0/1960634893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c4157210 0x7f98c4159c10 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 --2- 172.21.15.33:0/1960634893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c415a150 0x7f98c40afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 -- 172.21.15.33:0/1960634893 >> 172.21.15.33:0/1960634893 conn(0x7f98c406c560 msgr2=0x7f98c406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 -- 172.21.15.33:0/1960634893 shutdown_connections 2023-10-06T05:27:58.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.025+0000 7f98c89ff700 1 -- 172.21.15.33:0/1960634893 wait complete. 2023-10-06T05:27:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.027+0000 7f98c89ff700 1 Processor -- start 2023-10-06T05:27:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.027+0000 7f98c89ff700 1 -- start start 2023-10-06T05:27:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c89ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c40b0340 0x7f98c40cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c89ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c4157210 0x7f98c4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c89ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 0x7f98c4153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c89ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40cb130 con 0x7f98c415a150 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c89ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40afab0 con 0x7f98c4157210 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c89ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c416d110 con 0x7f98c40b0340 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 0x7f98c4153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 0x7f98c4153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34638/0 (socket says 172.21.15.33:34638) 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.028+0000 7f98c2d9d700 1 -- 172.21.15.33:0/3217913116 learned_addr learned my addr 172.21.15.33:0/3217913116 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c259c700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c40b0340 0x7f98c40cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c1d9b700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c4157210 0x7f98c4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c2d9d700 1 -- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c40b0340 msgr2=0x7f98c40cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c2d9d700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c40b0340 0x7f98c40cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c2d9d700 1 -- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c4157210 msgr2=0x7f98c4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c2d9d700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c4157210 0x7f98c4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c2d9d700 1 -- 172.21.15.33:0/3217913116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40094f0 con 0x7f98c415a150 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c259c700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c40b0340 0x7f98c40cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c1d9b700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c4157210 0x7f98c4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c2d9d700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 0x7f98c4153b40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c8e0 tx=0x7f98bc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00ce80 con 0x7f98c415a150 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc00a7e0 con 0x7f98c415a150 2023-10-06T05:27:58.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c416d3f0 con 0x7f98c415a150 2023-10-06T05:27:58.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.029+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00ab80 con 0x7f98c415a150 2023-10-06T05:27:58.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.030+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c416d9b0 con 0x7f98c415a150 2023-10-06T05:27:58.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.031+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c404c380 con 0x7f98c415a150 2023-10-06T05:27:58.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.031+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98bc012420 con 0x7f98c415a150 2023-10-06T05:27:58.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.032+0000 7f98b27fc700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98a4066f30 0x7f98a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.032+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98bc084a20 con 0x7f98c415a150 2023-10-06T05:27:58.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.033+0000 7f98c259c700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98a4066f30 0x7f98a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.034+0000 7f98c259c700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98a4066f30 0x7f98a40693e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f98b40099b0 tx=0x7f98b4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.036+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98bc051a90 con 0x7f98c415a150 2023-10-06T05:27:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.234+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98c40b0120 con 0x7f98a4066f30 2023-10-06T05:27:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.245+0000 7f98b27fc700 1 -- 172.21.15.33:0/3217913116 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311443 (secure 0 0 0) 0x7f98c40b0120 con 0x7f98a4066f30 2023-10-06T05:27:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98a4066f30 msgr2=0x7f98a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98a4066f30 0x7f98a40693e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f98b40099b0 tx=0x7f98b4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 msgr2=0x7f98c4153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 0x7f98c4153b40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c8e0 tx=0x7f98bc001710 comp rx=0 tx=0).stop 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 shutdown_connections 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98a4066f30 0x7f98a40693e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98c415a150 0x7f98c4153b40 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c40b0340 0x7f98c40cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 --2- 172.21.15.33:0/3217913116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c4157210 0x7f98c4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 >> 172.21.15.33:0/3217913116 conn(0x7f98c406c560 msgr2=0x7f98c415bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 shutdown_connections 2023-10-06T05:27:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.249+0000 7f98c89ff700 1 -- 172.21.15.33:0/3217913116 wait complete. 2023-10-06T05:27:58.255 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:27:58.269 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":600,"stamp":"2023-10-06T05:27:58.021030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431432,"kb_used_data":20360,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752543288,"statfs":{"total":773094113280,"available":770604326912,"internally_reserved":0,"allocated":20848640,"data_stored":10302948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001391"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846718+0000","last_change":"2023-10-06T05:27:53.843979+0000","last_active":"2023-10-06T05:27:53.846718+0000","last_peered":"2023-10-06T05:27:53.846718+0000","last_clean":"2023-10-06T05:27:53.846718+0000","last_became_active":"2023-10-06T05:27:53.843710+0000","last_became_peered":"2023-10-06T05:27:53.843710+0000","last_unstale":"2023-10-06T05:27:53.846718+0000","last_undegraded":"2023-10-06T05:27:53.846718+0000","last_fullsized":"2023-10-06T05:27:53.846718+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:58.861720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.849829+0000","last_change":"2023-10-06T05:27:53.846800+0000","last_active":"2023-10-06T05:27:53.849829+0000","last_peered":"2023-10-06T05:27:53.849829+0000","last_clean":"2023-10-06T05:27:53.849829+0000","last_became_active":"2023-10-06T05:27:53.846537+0000","last_became_peered":"2023-10-06T05:27:53.846537+0000","last_unstale":"2023-10-06T05:27:53.849829+0000","last_undegraded":"2023-10-06T05:27:53.849829+0000","last_fullsized":"2023-10-06T05:27:53.849829+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:37.249054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850749+0000","last_change":"2023-10-06T05:27:53.847718+0000","last_active":"2023-10-06T05:27:53.850749+0000","last_peered":"2023-10-06T05:27:53.850749+0000","last_clean":"2023-10-06T05:27:53.850749+0000","last_became_active":"2023-10-06T05:27:53.847421+0000","last_became_peered":"2023-10-06T05:27:53.847421+0000","last_unstale":"2023-10-06T05:27:53.850749+0000","last_undegraded":"2023-10-06T05:27:53.850749+0000","last_fullsized":"2023-10-06T05:27:53.850749+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:15.298112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855550+0000","last_change":"2023-10-06T05:27:53.853667+0000","last_active":"2023-10-06T05:27:53.855550+0000","last_peered":"2023-10-06T05:27:53.855550+0000","last_clean":"2023-10-06T05:27:53.855550+0000","last_became_active":"2023-10-06T05:27:53.853326+0000","last_became_peered":"2023-10-06T05:27:53.853326+0000","last_unstale":"2023-10-06T05:27:53.855550+0000","last_undegraded":"2023-10-06T05:27:53.855550+0000","last_fullsized":"2023-10-06T05:27:53.855550+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:17.334537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850707+0000","last_change":"2023-10-06T05:27:53.846468+0000","last_active":"2023-10-06T05:27:53.850707+0000","last_peered":"2023-10-06T05:27:53.850707+0000","last_clean":"2023-10-06T05:27:53.850707+0000","last_became_active":"2023-10-06T05:27:53.845896+0000","last_became_peered":"2023-10-06T05:27:53.845896+0000","last_unstale":"2023-10-06T05:27:53.850707+0000","last_undegraded":"2023-10-06T05:27:53.850707+0000","last_fullsized":"2023-10-06T05:27:53.850707+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:06.689371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.860662+0000","last_change":"2023-10-06T05:27:53.856756+0000","last_active":"2023-10-06T05:27:53.860662+0000","last_peered":"2023-10-06T05:27:53.860662+0000","last_clean":"2023-10-06T05:27:53.860662+0000","last_became_active":"2023-10-06T05:27:53.856489+0000","last_became_peered":"2023-10-06T05:27:53.856489+0000","last_unstale":"2023-10-06T05:27:53.860662+0000","last_undegraded":"2023-10-06T05:27:53.860662+0000","last_fullsized":"2023-10-06T05:27:53.860662+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:39.142251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850786+0000","last_change":"2023-10-06T05:27:53.842169+0000","last_active":"2023-10-06T05:27:53.850786+0000","last_peered":"2023-10-06T05:27:53.850786+0000","last_clean":"2023-10-06T05:27:53.850786+0000","last_became_active":"2023-10-06T05:27:53.841915+0000","last_became_peered":"2023-10-06T05:27:53.841915+0000","last_unstale":"2023-10-06T05:27:53.850786+0000","last_undegraded":"2023-10-06T05:27:53.850786+0000","last_fullsized":"2023-10-06T05:27:53.850786+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:59.465527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.860692+0000","last_change":"2023-10-06T05:27:53.856128+0000","last_active":"2023-10-06T05:27:53.860692+0000","last_peered":"2023-10-06T05:27:53.860692+0000","last_clean":"2023-10-06T05:27:53.860692+0000","last_became_active":"2023-10-06T05:27:53.854732+0000","last_became_peered":"2023-10-06T05:27:53.854732+0000","last_unstale":"2023-10-06T05:27:53.860692+0000","last_undegraded":"2023-10-06T05:27:53.860692+0000","last_fullsized":"2023-10-06T05:27:53.860692+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:17.334537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848791+0000","last_change":"2023-10-06T05:27:53.846658+0000","last_active":"2023-10-06T05:27:53.848791+0000","last_peered":"2023-10-06T05:27:53.848791+0000","last_clean":"2023-10-06T05:27:53.848791+0000","last_became_active":"2023-10-06T05:27:53.846398+0000","last_became_peered":"2023-10-06T05:27:53.846398+0000","last_unstale":"2023-10-06T05:27:53.848791+0000","last_undegraded":"2023-10-06T05:27:53.848791+0000","last_fullsized":"2023-10-06T05:27:53.848791+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:14.120267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.853639+0000","last_change":"2023-10-06T05:27:53.848757+0000","last_active":"2023-10-06T05:27:53.853639+0000","last_peered":"2023-10-06T05:27:53.853639+0000","last_clean":"2023-10-06T05:27:53.853639+0000","last_became_active":"2023-10-06T05:27:53.848480+0000","last_became_peered":"2023-10-06T05:27:53.848480+0000","last_unstale":"2023-10-06T05:27:53.853639+0000","last_undegraded":"2023-10-06T05:27:53.853639+0000","last_fullsized":"2023-10-06T05:27:53.853639+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:17.817077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855604+0000","last_change":"2023-10-06T05:27:53.854408+0000","last_active":"2023-10-06T05:27:53.855604+0000","last_peered":"2023-10-06T05:27:53.855604+0000","last_clean":"2023-10-06T05:27:53.855604+0000","last_became_active":"2023-10-06T05:27:53.854138+0000","last_became_peered":"2023-10-06T05:27:53.854138+0000","last_unstale":"2023-10-06T05:27:53.855604+0000","last_undegraded":"2023-10-06T05:27:53.855604+0000","last_fullsized":"2023-10-06T05:27:53.855604+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:03.407461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852179+0000","last_change":"2023-10-06T05:27:53.849980+0000","last_active":"2023-10-06T05:27:53.852179+0000","last_peered":"2023-10-06T05:27:53.852179+0000","last_clean":"2023-10-06T05:27:53.852179+0000","last_became_active":"2023-10-06T05:27:53.849713+0000","last_became_peered":"2023-10-06T05:27:53.849713+0000","last_unstale":"2023-10-06T05:27:53.852179+0000","last_undegraded":"2023-10-06T05:27:53.852179+0000","last_fullsized":"2023-10-06T05:27:53.852179+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:31.636043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.858957+0000","last_change":"2023-10-06T05:27:53.855056+0000","last_active":"2023-10-06T05:27:53.858957+0000","last_peered":"2023-10-06T05:27:53.858957+0000","last_clean":"2023-10-06T05:27:53.858957+0000","last_became_active":"2023-10-06T05:27:53.854777+0000","last_became_peered":"2023-10-06T05:27:53.854777+0000","last_unstale":"2023-10-06T05:27:53.858957+0000","last_undegraded":"2023-10-06T05:27:53.858957+0000","last_fullsized":"2023-10-06T05:27:53.858957+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:51.020996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.851635+0000","last_change":"2023-10-06T05:27:53.849507+0000","last_active":"2023-10-06T05:27:53.851635+0000","last_peered":"2023-10-06T05:27:53.851635+0000","last_clean":"2023-10-06T05:27:53.851635+0000","last_became_active":"2023-10-06T05:27:53.848662+0000","last_became_peered":"2023-10-06T05:27:53.848662+0000","last_unstale":"2023-10-06T05:27:53.851635+0000","last_undegraded":"2023-10-06T05:27:53.851635+0000","last_fullsized":"2023-10-06T05:27:53.851635+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:47.806726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.844358+0000","last_change":"2023-10-06T05:27:53.843438+0000","last_active":"2023-10-06T05:27:53.844358+0000","last_peered":"2023-10-06T05:27:53.844358+0000","last_clean":"2023-10-06T05:27:53.844358+0000","last_became_active":"2023-10-06T05:27:53.843180+0000","last_became_peered":"2023-10-06T05:27:53.843180+0000","last_unstale":"2023-10-06T05:27:53.844358+0000","last_undegraded":"2023-10-06T05:27:53.844358+0000","last_fullsized":"2023-10-06T05:27:53.844358+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:10.048081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.845471+0000","last_change":"2023-10-06T05:27:53.840835+0000","last_active":"2023-10-06T05:27:53.845471+0000","last_peered":"2023-10-06T05:27:53.845471+0000","last_clean":"2023-10-06T05:27:53.845471+0000","last_became_active":"2023-10-06T05:27:53.840532+0000","last_became_peered":"2023-10-06T05:27:53.840532+0000","last_unstale":"2023-10-06T05:27:53.845471+0000","last_undegraded":"2023-10-06T05:27:53.845471+0000","last_fullsized":"2023-10-06T05:27:53.845471+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:22.008412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.851563+0000","last_change":"2023-10-06T05:27:53.849408+0000","last_active":"2023-10-06T05:27:53.851563+0000","last_peered":"2023-10-06T05:27:53.851563+0000","last_clean":"2023-10-06T05:27:53.851563+0000","last_became_active":"2023-10-06T05:27:53.849054+0000","last_became_peered":"2023-10-06T05:27:53.849054+0000","last_unstale":"2023-10-06T05:27:53.851563+0000","last_undegraded":"2023-10-06T05:27:53.851563+0000","last_fullsized":"2023-10-06T05:27:53.851563+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.565485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856463+0000","last_change":"2023-10-06T05:27:53.854553+0000","last_active":"2023-10-06T05:27:53.856463+0000","last_peered":"2023-10-06T05:27:53.856463+0000","last_clean":"2023-10-06T05:27:53.856463+0000","last_became_active":"2023-10-06T05:27:53.854304+0000","last_became_peered":"2023-10-06T05:27:53.854304+0000","last_unstale":"2023-10-06T05:27:53.856463+0000","last_undegraded":"2023-10-06T05:27:53.856463+0000","last_fullsized":"2023-10-06T05:27:53.856463+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:48.950557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852368+0000","last_change":"2023-10-06T05:27:53.846850+0000","last_active":"2023-10-06T05:27:53.852368+0000","last_peered":"2023-10-06T05:27:53.852368+0000","last_clean":"2023-10-06T05:27:53.852368+0000","last_became_active":"2023-10-06T05:27:53.846310+0000","last_became_peered":"2023-10-06T05:27:53.846310+0000","last_unstale":"2023-10-06T05:27:53.852368+0000","last_undegraded":"2023-10-06T05:27:53.852368+0000","last_fullsized":"2023-10-06T05:27:53.852368+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:31.157221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.847222+0000","last_change":"2023-10-06T05:27:53.841749+0000","last_active":"2023-10-06T05:27:53.847222+0000","last_peered":"2023-10-06T05:27:53.847222+0000","last_clean":"2023-10-06T05:27:53.847222+0000","last_became_active":"2023-10-06T05:27:53.841437+0000","last_became_peered":"2023-10-06T05:27:53.841437+0000","last_unstale":"2023-10-06T05:27:53.847222+0000","last_undegraded":"2023-10-06T05:27:53.847222+0000","last_fullsized":"2023-10-06T05:27:53.847222+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:23.268651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850799+0000","last_change":"2023-10-06T05:27:53.847211+0000","last_active":"2023-10-06T05:27:53.850799+0000","last_peered":"2023-10-06T05:27:53.850799+0000","last_clean":"2023-10-06T05:27:53.850799+0000","last_became_active":"2023-10-06T05:27:53.846958+0000","last_became_peered":"2023-10-06T05:27:53.846958+0000","last_unstale":"2023-10-06T05:27:53.850799+0000","last_undegraded":"2023-10-06T05:27:53.850799+0000","last_fullsized":"2023-10-06T05:27:53.850799+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:18.780395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.844121+0000","last_change":"2023-10-06T05:27:53.841978+0000","last_active":"2023-10-06T05:27:53.844121+0000","last_peered":"2023-10-06T05:27:53.844121+0000","last_clean":"2023-10-06T05:27:53.844121+0000","last_became_active":"2023-10-06T05:27:53.841700+0000","last_became_peered":"2023-10-06T05:27:53.841700+0000","last_unstale":"2023-10-06T05:27:53.844121+0000","last_undegraded":"2023-10-06T05:27:53.844121+0000","last_fullsized":"2023-10-06T05:27:53.844121+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:27.427455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.842664+0000","last_change":"2023-10-06T05:27:53.839540+0000","last_active":"2023-10-06T05:27:53.842664+0000","last_peered":"2023-10-06T05:27:53.842664+0000","last_clean":"2023-10-06T05:27:53.842664+0000","last_became_active":"2023-10-06T05:27:53.839263+0000","last_became_peered":"2023-10-06T05:27:53.839263+0000","last_unstale":"2023-10-06T05:27:53.842664+0000","last_undegraded":"2023-10-06T05:27:53.842664+0000","last_fullsized":"2023-10-06T05:27:53.842664+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:20.657352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846827+0000","last_change":"2023-10-06T05:27:53.844929+0000","last_active":"2023-10-06T05:27:53.846827+0000","last_peered":"2023-10-06T05:27:53.846827+0000","last_clean":"2023-10-06T05:27:53.846827+0000","last_became_active":"2023-10-06T05:27:53.844625+0000","last_became_peered":"2023-10-06T05:27:53.844625+0000","last_unstale":"2023-10-06T05:27:53.846827+0000","last_undegraded":"2023-10-06T05:27:53.846827+0000","last_fullsized":"2023-10-06T05:27:53.846827+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:29.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846796+0000","last_change":"2023-10-06T05:27:53.844892+0000","last_active":"2023-10-06T05:27:53.846796+0000","last_peered":"2023-10-06T05:27:53.846796+0000","last_clean":"2023-10-06T05:27:53.846796+0000","last_became_active":"2023-10-06T05:27:53.844578+0000","last_became_peered":"2023-10-06T05:27:53.844578+0000","last_unstale":"2023-10-06T05:27:53.846796+0000","last_undegraded":"2023-10-06T05:27:53.846796+0000","last_fullsized":"2023-10-06T05:27:53.846796+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:39.804459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.854488+0000","last_change":"2023-10-06T05:27:53.850975+0000","last_active":"2023-10-06T05:27:53.854488+0000","last_peered":"2023-10-06T05:27:53.854488+0000","last_clean":"2023-10-06T05:27:53.854488+0000","last_became_active":"2023-10-06T05:27:53.849056+0000","last_became_peered":"2023-10-06T05:27:53.849056+0000","last_unstale":"2023-10-06T05:27:53.854488+0000","last_undegraded":"2023-10-06T05:27:53.854488+0000","last_fullsized":"2023-10-06T05:27:53.854488+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:05.961239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855581+0000","last_change":"2023-10-06T05:27:53.852957+0000","last_active":"2023-10-06T05:27:53.855581+0000","last_peered":"2023-10-06T05:27:53.855581+0000","last_clean":"2023-10-06T05:27:53.855581+0000","last_became_active":"2023-10-06T05:27:53.852636+0000","last_became_peered":"2023-10-06T05:27:53.852636+0000","last_unstale":"2023-10-06T05:27:53.855581+0000","last_undegraded":"2023-10-06T05:27:53.855581+0000","last_fullsized":"2023-10-06T05:27:53.855581+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:39.142251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850553+0000","last_change":"2023-10-06T05:27:53.846883+0000","last_active":"2023-10-06T05:27:53.850553+0000","last_peered":"2023-10-06T05:27:53.850553+0000","last_clean":"2023-10-06T05:27:53.850553+0000","last_became_active":"2023-10-06T05:27:53.846628+0000","last_became_peered":"2023-10-06T05:27:53.846628+0000","last_unstale":"2023-10-06T05:27:53.850553+0000","last_undegraded":"2023-10-06T05:27:53.850553+0000","last_fullsized":"2023-10-06T05:27:53.850553+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:13:50.588026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856780+0000","last_change":"2023-10-06T05:27:53.855371+0000","last_active":"2023-10-06T05:27:53.856780+0000","last_peered":"2023-10-06T05:27:53.856780+0000","last_clean":"2023-10-06T05:27:53.856780+0000","last_became_active":"2023-10-06T05:27:53.855069+0000","last_became_peered":"2023-10-06T05:27:53.855069+0000","last_unstale":"2023-10-06T05:27:53.856780+0000","last_undegraded":"2023-10-06T05:27:53.856780+0000","last_fullsized":"2023-10-06T05:27:53.856780+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:42.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856675+0000","last_change":"2023-10-06T05:27:53.849850+0000","last_active":"2023-10-06T05:27:53.856675+0000","last_peered":"2023-10-06T05:27:53.856675+0000","last_clean":"2023-10-06T05:27:53.856675+0000","last_became_active":"2023-10-06T05:27:53.849556+0000","last_became_peered":"2023-10-06T05:27:53.849556+0000","last_unstale":"2023-10-06T05:27:53.856675+0000","last_undegraded":"2023-10-06T05:27:53.856675+0000","last_fullsized":"2023-10-06T05:27:53.856675+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:02.967469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839708+0000","last_change":"2023-10-06T05:27:53.836945+0000","last_active":"2023-10-06T05:27:53.839708+0000","last_peered":"2023-10-06T05:27:53.839708+0000","last_clean":"2023-10-06T05:27:53.839708+0000","last_became_active":"2023-10-06T05:27:53.836621+0000","last_became_peered":"2023-10-06T05:27:53.836621+0000","last_unstale":"2023-10-06T05:27:53.839708+0000","last_undegraded":"2023-10-06T05:27:53.839708+0000","last_fullsized":"2023-10-06T05:27:53.839708+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:10.889227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856238+0000","last_change":"2023-10-06T05:27:53.851187+0000","last_active":"2023-10-06T05:27:53.856238+0000","last_peered":"2023-10-06T05:27:53.856238+0000","last_clean":"2023-10-06T05:27:53.856238+0000","last_became_active":"2023-10-06T05:27:53.850936+0000","last_became_peered":"2023-10-06T05:27:53.850936+0000","last_unstale":"2023-10-06T05:27:53.856238+0000","last_undegraded":"2023-10-06T05:27:53.856238+0000","last_fullsized":"2023-10-06T05:27:53.856238+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:45.961016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855953+0000","last_change":"2023-10-06T05:27:53.854784+0000","last_active":"2023-10-06T05:27:53.855953+0000","last_peered":"2023-10-06T05:27:53.855953+0000","last_clean":"2023-10-06T05:27:53.855953+0000","last_became_active":"2023-10-06T05:27:53.854519+0000","last_became_peered":"2023-10-06T05:27:53.854519+0000","last_unstale":"2023-10-06T05:27:53.855953+0000","last_undegraded":"2023-10-06T05:27:53.855953+0000","last_fullsized":"2023-10-06T05:27:53.855953+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:11.218258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.835885+0000","last_change":"2023-10-06T05:27:52.892875+0000","last_active":"2023-10-06T05:27:53.835885+0000","last_peered":"2023-10-06T05:27:53.835885+0000","last_clean":"2023-10-06T05:27:53.835885+0000","last_became_active":"2023-10-06T05:27:52.892618+0000","last_became_peered":"2023-10-06T05:27:52.892618+0000","last_unstale":"2023-10-06T05:27:53.835885+0000","last_undegraded":"2023-10-06T05:27:53.835885+0000","last_fullsized":"2023-10-06T05:27:53.835885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:37.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850405+0000","last_change":"2023-10-06T05:27:53.845797+0000","last_active":"2023-10-06T05:27:53.850405+0000","last_peered":"2023-10-06T05:27:53.850405+0000","last_clean":"2023-10-06T05:27:53.850405+0000","last_became_active":"2023-10-06T05:27:53.845460+0000","last_became_peered":"2023-10-06T05:27:53.845460+0000","last_unstale":"2023-10-06T05:27:53.850405+0000","last_undegraded":"2023-10-06T05:27:53.850405+0000","last_fullsized":"2023-10-06T05:27:53.850405+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.214242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848982+0000","last_change":"2023-10-06T05:27:52.900113+0000","last_active":"2023-10-06T05:27:53.848982+0000","last_peered":"2023-10-06T05:27:53.848982+0000","last_clean":"2023-10-06T05:27:53.848982+0000","last_became_active":"2023-10-06T05:27:52.899814+0000","last_became_peered":"2023-10-06T05:27:52.899814+0000","last_unstale":"2023-10-06T05:27:53.848982+0000","last_undegraded":"2023-10-06T05:27:53.848982+0000","last_fullsized":"2023-10-06T05:27:53.848982+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:11.527906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848849+0000","last_change":"2023-10-06T05:27:53.842031+0000","last_active":"2023-10-06T05:27:53.848849+0000","last_peered":"2023-10-06T05:27:53.848849+0000","last_clean":"2023-10-06T05:27:53.848849+0000","last_became_active":"2023-10-06T05:27:53.841712+0000","last_became_peered":"2023-10-06T05:27:53.841712+0000","last_unstale":"2023-10-06T05:27:53.848849+0000","last_undegraded":"2023-10-06T05:27:53.848849+0000","last_fullsized":"2023-10-06T05:27:53.848849+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:53.529237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"193'3","reported_seq":13,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.885816+0000","last_change":"2023-10-06T05:27:52.894676+0000","last_active":"2023-10-06T05:27:53.885816+0000","last_peered":"2023-10-06T05:27:53.885816+0000","last_clean":"2023-10-06T05:27:53.885816+0000","last_became_active":"2023-10-06T05:27:52.894386+0000","last_became_peered":"2023-10-06T05:27:52.894386+0000","last_unstale":"2023-10-06T05:27:53.885816+0000","last_undegraded":"2023-10-06T05:27:53.885816+0000","last_fullsized":"2023-10-06T05:27:53.885816+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:14.073674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855191+0000","last_change":"2023-10-06T05:27:53.852617+0000","last_active":"2023-10-06T05:27:53.855191+0000","last_peered":"2023-10-06T05:27:53.855191+0000","last_clean":"2023-10-06T05:27:53.855191+0000","last_became_active":"2023-10-06T05:27:53.852336+0000","last_became_peered":"2023-10-06T05:27:53.852336+0000","last_unstale":"2023-10-06T05:27:53.855191+0000","last_undegraded":"2023-10-06T05:27:53.855191+0000","last_fullsized":"2023-10-06T05:27:53.855191+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:22.008412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"193'6","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.891839+0000","last_change":"2023-10-06T05:27:52.901867+0000","last_active":"2023-10-06T05:27:53.891839+0000","last_peered":"2023-10-06T05:27:53.891839+0000","last_clean":"2023-10-06T05:27:53.891839+0000","last_became_active":"2023-10-06T05:27:52.901578+0000","last_became_peered":"2023-10-06T05:27:52.901578+0000","last_unstale":"2023-10-06T05:27:53.891839+0000","last_undegraded":"2023-10-06T05:27:53.891839+0000","last_fullsized":"2023-10-06T05:27:53.891839+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:09.217790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.854694+0000","last_change":"2023-10-06T05:27:53.853295+0000","last_active":"2023-10-06T05:27:53.854694+0000","last_peered":"2023-10-06T05:27:53.854694+0000","last_clean":"2023-10-06T05:27:53.854694+0000","last_became_active":"2023-10-06T05:27:53.852837+0000","last_became_peered":"2023-10-06T05:27:53.852837+0000","last_unstale":"2023-10-06T05:27:53.854694+0000","last_undegraded":"2023-10-06T05:27:53.854694+0000","last_fullsized":"2023-10-06T05:27:53.854694+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:58.630568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.889972+0000","last_change":"2023-10-06T05:27:52.903325+0000","last_active":"2023-10-06T05:27:53.889972+0000","last_peered":"2023-10-06T05:27:53.889972+0000","last_clean":"2023-10-06T05:27:53.889972+0000","last_became_active":"2023-10-06T05:27:52.902997+0000","last_became_peered":"2023-10-06T05:27:52.902997+0000","last_unstale":"2023-10-06T05:27:53.889972+0000","last_undegraded":"2023-10-06T05:27:53.889972+0000","last_fullsized":"2023-10-06T05:27:53.889972+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:00.167089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852150+0000","last_change":"2023-10-06T05:27:53.849606+0000","last_active":"2023-10-06T05:27:53.852150+0000","last_peered":"2023-10-06T05:27:53.852150+0000","last_clean":"2023-10-06T05:27:53.852150+0000","last_became_active":"2023-10-06T05:27:53.849343+0000","last_became_peered":"2023-10-06T05:27:53.849343+0000","last_unstale":"2023-10-06T05:27:53.852150+0000","last_undegraded":"2023-10-06T05:27:53.852150+0000","last_fullsized":"2023-10-06T05:27:53.852150+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:34.090516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.835063+0000","last_change":"2023-10-06T05:27:52.892190+0000","last_active":"2023-10-06T05:27:53.835063+0000","last_peered":"2023-10-06T05:27:53.835063+0000","last_clean":"2023-10-06T05:27:53.835063+0000","last_became_active":"2023-10-06T05:27:52.891936+0000","last_became_peered":"2023-10-06T05:27:52.891936+0000","last_unstale":"2023-10-06T05:27:53.835063+0000","last_undegraded":"2023-10-06T05:27:53.835063+0000","last_fullsized":"2023-10-06T05:27:53.835063+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:30.147736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848573+0000","last_change":"2023-10-06T05:27:53.842157+0000","last_active":"2023-10-06T05:27:53.848573+0000","last_peered":"2023-10-06T05:27:53.848573+0000","last_clean":"2023-10-06T05:27:53.848573+0000","last_became_active":"2023-10-06T05:27:53.841891+0000","last_became_peered":"2023-10-06T05:27:53.841891+0000","last_unstale":"2023-10-06T05:27:53.848573+0000","last_undegraded":"2023-10-06T05:27:53.848573+0000","last_fullsized":"2023-10-06T05:27:53.848573+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:42.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.832503+0000","last_change":"2023-10-06T05:27:52.896976+0000","last_active":"2023-10-06T05:27:53.832503+0000","last_peered":"2023-10-06T05:27:53.832503+0000","last_clean":"2023-10-06T05:27:53.832503+0000","last_became_active":"2023-10-06T05:27:52.896648+0000","last_became_peered":"2023-10-06T05:27:52.896648+0000","last_unstale":"2023-10-06T05:27:53.832503+0000","last_undegraded":"2023-10-06T05:27:53.832503+0000","last_fullsized":"2023-10-06T05:27:53.832503+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:11.527906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.843862+0000","last_change":"2023-10-06T05:27:53.841086+0000","last_active":"2023-10-06T05:27:53.843862+0000","last_peered":"2023-10-06T05:27:53.843862+0000","last_clean":"2023-10-06T05:27:53.843862+0000","last_became_active":"2023-10-06T05:27:53.840711+0000","last_became_peered":"2023-10-06T05:27:53.840711+0000","last_unstale":"2023-10-06T05:27:53.843862+0000","last_undegraded":"2023-10-06T05:27:53.843862+0000","last_fullsized":"2023-10-06T05:27:53.843862+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:29.986397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"193'8","reported_seq":18,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.891708+0000","last_change":"2023-10-06T05:27:52.893625+0000","last_active":"2023-10-06T05:27:53.891708+0000","last_peered":"2023-10-06T05:27:53.891708+0000","last_clean":"2023-10-06T05:27:53.891708+0000","last_became_active":"2023-10-06T05:27:52.893337+0000","last_became_peered":"2023-10-06T05:27:52.893337+0000","last_unstale":"2023-10-06T05:27:53.891708+0000","last_undegraded":"2023-10-06T05:27:53.891708+0000","last_fullsized":"2023-10-06T05:27:53.891708+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:19.006016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.847003+0000","last_change":"2023-10-06T05:27:53.844309+0000","last_active":"2023-10-06T05:27:53.847003+0000","last_peered":"2023-10-06T05:27:53.847003+0000","last_clean":"2023-10-06T05:27:53.847003+0000","last_became_active":"2023-10-06T05:27:53.844059+0000","last_became_peered":"2023-10-06T05:27:53.844059+0000","last_unstale":"2023-10-06T05:27:53.847003+0000","last_undegraded":"2023-10-06T05:27:53.847003+0000","last_fullsized":"2023-10-06T05:27:53.847003+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:19.763732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.884378+0000","last_change":"2023-10-06T05:27:52.904384+0000","last_active":"2023-10-06T05:27:53.884378+0000","last_peered":"2023-10-06T05:27:53.884378+0000","last_clean":"2023-10-06T05:27:53.884378+0000","last_became_active":"2023-10-06T05:27:52.904102+0000","last_became_peered":"2023-10-06T05:27:52.904102+0000","last_unstale":"2023-10-06T05:27:53.884378+0000","last_undegraded":"2023-10-06T05:27:53.884378+0000","last_fullsized":"2023-10-06T05:27:53.884378+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.743229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.858857+0000","last_change":"2023-10-06T05:27:53.857906+0000","last_active":"2023-10-06T05:27:53.858857+0000","last_peered":"2023-10-06T05:27:53.858857+0000","last_clean":"2023-10-06T05:27:53.858857+0000","last_became_active":"2023-10-06T05:27:53.857592+0000","last_became_peered":"2023-10-06T05:27:53.857592+0000","last_unstale":"2023-10-06T05:27:53.858857+0000","last_undegraded":"2023-10-06T05:27:53.858857+0000","last_fullsized":"2023-10-06T05:27:53.858857+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:30.076619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.883439+0000","last_change":"2023-10-06T05:27:52.895270+0000","last_active":"2023-10-06T05:27:53.883439+0000","last_peered":"2023-10-06T05:27:53.883439+0000","last_clean":"2023-10-06T05:27:53.883439+0000","last_became_active":"2023-10-06T05:27:52.894694+0000","last_became_peered":"2023-10-06T05:27:52.894694+0000","last_unstale":"2023-10-06T05:27:53.883439+0000","last_undegraded":"2023-10-06T05:27:53.883439+0000","last_fullsized":"2023-10-06T05:27:53.883439+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:58.512050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855865+0000","last_change":"2023-10-06T05:27:53.853506+0000","last_active":"2023-10-06T05:27:53.855865+0000","last_peered":"2023-10-06T05:27:53.855865+0000","last_clean":"2023-10-06T05:27:53.855865+0000","last_became_active":"2023-10-06T05:27:53.851377+0000","last_became_peered":"2023-10-06T05:27:53.851377+0000","last_unstale":"2023-10-06T05:27:53.855865+0000","last_undegraded":"2023-10-06T05:27:53.855865+0000","last_fullsized":"2023-10-06T05:27:53.855865+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:58.630568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890099+0000","last_change":"2023-10-06T05:27:52.895108+0000","last_active":"2023-10-06T05:27:53.890099+0000","last_peered":"2023-10-06T05:27:53.890099+0000","last_clean":"2023-10-06T05:27:53.890099+0000","last_became_active":"2023-10-06T05:27:52.893584+0000","last_became_peered":"2023-10-06T05:27:52.893584+0000","last_unstale":"2023-10-06T05:27:53.890099+0000","last_undegraded":"2023-10-06T05:27:53.890099+0000","last_fullsized":"2023-10-06T05:27:53.890099+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:42.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.859192+0000","last_change":"2023-10-06T05:27:53.855552+0000","last_active":"2023-10-06T05:27:53.859192+0000","last_peered":"2023-10-06T05:27:53.859192+0000","last_clean":"2023-10-06T05:27:53.859192+0000","last_became_active":"2023-10-06T05:27:53.855253+0000","last_became_peered":"2023-10-06T05:27:53.855253+0000","last_unstale":"2023-10-06T05:27:53.859192+0000","last_undegraded":"2023-10-06T05:27:53.859192+0000","last_fullsized":"2023-10-06T05:27:53.859192+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:27.189784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.838898+0000","last_change":"2023-10-06T05:27:52.892912+0000","last_active":"2023-10-06T05:27:53.838898+0000","last_peered":"2023-10-06T05:27:53.838898+0000","last_clean":"2023-10-06T05:27:53.838898+0000","last_became_active":"2023-10-06T05:27:52.892586+0000","last_became_peered":"2023-10-06T05:27:52.892586+0000","last_unstale":"2023-10-06T05:27:53.838898+0000","last_undegraded":"2023-10-06T05:27:53.838898+0000","last_fullsized":"2023-10-06T05:27:53.838898+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:56.486823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850673+0000","last_change":"2023-10-06T05:27:53.847768+0000","last_active":"2023-10-06T05:27:53.850673+0000","last_peered":"2023-10-06T05:27:53.850673+0000","last_clean":"2023-10-06T05:27:53.850673+0000","last_became_active":"2023-10-06T05:27:53.847459+0000","last_became_peered":"2023-10-06T05:27:53.847459+0000","last_unstale":"2023-10-06T05:27:53.850673+0000","last_undegraded":"2023-10-06T05:27:53.850673+0000","last_fullsized":"2023-10-06T05:27:53.850673+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:35.732868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890789+0000","last_change":"2023-10-06T05:27:52.895569+0000","last_active":"2023-10-06T05:27:53.890789+0000","last_peered":"2023-10-06T05:27:53.890789+0000","last_clean":"2023-10-06T05:27:53.890789+0000","last_became_active":"2023-10-06T05:27:52.895260+0000","last_became_peered":"2023-10-06T05:27:52.895260+0000","last_unstale":"2023-10-06T05:27:53.890789+0000","last_undegraded":"2023-10-06T05:27:53.890789+0000","last_fullsized":"2023-10-06T05:27:53.890789+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:05.453895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846756+0000","last_change":"2023-10-06T05:27:53.844391+0000","last_active":"2023-10-06T05:27:53.846756+0000","last_peered":"2023-10-06T05:27:53.846756+0000","last_clean":"2023-10-06T05:27:53.846756+0000","last_became_active":"2023-10-06T05:27:53.844088+0000","last_became_peered":"2023-10-06T05:27:53.844088+0000","last_unstale":"2023-10-06T05:27:53.846756+0000","last_undegraded":"2023-10-06T05:27:53.846756+0000","last_fullsized":"2023-10-06T05:27:53.846756+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:00.934914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.837938+0000","last_change":"2023-10-06T05:27:52.892977+0000","last_active":"2023-10-06T05:27:53.837938+0000","last_peered":"2023-10-06T05:27:53.837938+0000","last_clean":"2023-10-06T05:27:53.837938+0000","last_became_active":"2023-10-06T05:27:52.892711+0000","last_became_peered":"2023-10-06T05:27:52.892711+0000","last_unstale":"2023-10-06T05:27:53.837938+0000","last_undegraded":"2023-10-06T05:27:53.837938+0000","last_fullsized":"2023-10-06T05:27:53.837938+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:52.760427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.849926+0000","last_change":"2023-10-06T05:27:53.847722+0000","last_active":"2023-10-06T05:27:53.849926+0000","last_peered":"2023-10-06T05:27:53.849926+0000","last_clean":"2023-10-06T05:27:53.849926+0000","last_became_active":"2023-10-06T05:27:53.847452+0000","last_became_peered":"2023-10-06T05:27:53.847452+0000","last_unstale":"2023-10-06T05:27:53.849926+0000","last_undegraded":"2023-10-06T05:27:53.849926+0000","last_fullsized":"2023-10-06T05:27:53.849926+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:59.465527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.837921+0000","last_change":"2023-10-06T05:27:52.886594+0000","last_active":"2023-10-06T05:27:53.837921+0000","last_peered":"2023-10-06T05:27:53.837921+0000","last_clean":"2023-10-06T05:27:53.837921+0000","last_became_active":"2023-10-06T05:27:52.886337+0000","last_became_peered":"2023-10-06T05:27:52.886337+0000","last_unstale":"2023-10-06T05:27:53.837921+0000","last_undegraded":"2023-10-06T05:27:53.837921+0000","last_fullsized":"2023-10-06T05:27:53.837921+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:59.679323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.854367+0000","last_change":"2023-10-06T05:27:53.850701+0000","last_active":"2023-10-06T05:27:53.854367+0000","last_peered":"2023-10-06T05:27:53.854367+0000","last_clean":"2023-10-06T05:27:53.854367+0000","last_became_active":"2023-10-06T05:27:53.848886+0000","last_became_peered":"2023-10-06T05:27:53.848886+0000","last_unstale":"2023-10-06T05:27:53.854367+0000","last_undegraded":"2023-10-06T05:27:53.854367+0000","last_fullsized":"2023-10-06T05:27:53.854367+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:10.710631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.830001+0000","last_change":"2023-10-06T05:27:52.894905+0000","last_active":"2023-10-06T05:27:53.830001+0000","last_peered":"2023-10-06T05:27:53.830001+0000","last_clean":"2023-10-06T05:27:53.830001+0000","last_became_active":"2023-10-06T05:27:52.894632+0000","last_became_peered":"2023-10-06T05:27:52.894632+0000","last_unstale":"2023-10-06T05:27:53.830001+0000","last_undegraded":"2023-10-06T05:27:53.830001+0000","last_fullsized":"2023-10-06T05:27:53.830001+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:24.348475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.844522+0000","last_change":"2023-10-06T05:27:53.841916+0000","last_active":"2023-10-06T05:27:53.844522+0000","last_peered":"2023-10-06T05:27:53.844522+0000","last_clean":"2023-10-06T05:27:53.844522+0000","last_became_active":"2023-10-06T05:27:53.841634+0000","last_became_peered":"2023-10-06T05:27:53.841634+0000","last_unstale":"2023-10-06T05:27:53.844522+0000","last_undegraded":"2023-10-06T05:27:53.844522+0000","last_fullsized":"2023-10-06T05:27:53.844522+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:56.723977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890970+0000","last_change":"2023-10-06T05:27:52.900858+0000","last_active":"2023-10-06T05:27:53.890970+0000","last_peered":"2023-10-06T05:27:53.890970+0000","last_clean":"2023-10-06T05:27:53.890970+0000","last_became_active":"2023-10-06T05:27:52.898620+0000","last_became_peered":"2023-10-06T05:27:52.898620+0000","last_unstale":"2023-10-06T05:27:53.890970+0000","last_undegraded":"2023-10-06T05:27:53.890970+0000","last_fullsized":"2023-10-06T05:27:53.890970+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:22.003271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852388+0000","last_change":"2023-10-06T05:27:53.847373+0000","last_active":"2023-10-06T05:27:53.852388+0000","last_peered":"2023-10-06T05:27:53.852388+0000","last_clean":"2023-10-06T05:27:53.852388+0000","last_became_active":"2023-10-06T05:27:53.847043+0000","last_became_peered":"2023-10-06T05:27:53.847043+0000","last_unstale":"2023-10-06T05:27:53.852388+0000","last_undegraded":"2023-10-06T05:27:53.852388+0000","last_fullsized":"2023-10-06T05:27:53.852388+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:09.402475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840512+0000","last_change":"2023-10-06T05:27:52.887276+0000","last_active":"2023-10-06T05:27:53.840512+0000","last_peered":"2023-10-06T05:27:53.840512+0000","last_clean":"2023-10-06T05:27:53.840512+0000","last_became_active":"2023-10-06T05:27:52.887007+0000","last_became_peered":"2023-10-06T05:27:52.887007+0000","last_unstale":"2023-10-06T05:27:53.840512+0000","last_undegraded":"2023-10-06T05:27:53.840512+0000","last_fullsized":"2023-10-06T05:27:53.840512+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:52.765913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856231+0000","last_change":"2023-10-06T05:27:53.850573+0000","last_active":"2023-10-06T05:27:53.856231+0000","last_peered":"2023-10-06T05:27:53.856231+0000","last_clean":"2023-10-06T05:27:53.856231+0000","last_became_active":"2023-10-06T05:27:53.850324+0000","last_became_peered":"2023-10-06T05:27:53.850324+0000","last_unstale":"2023-10-06T05:27:53.856231+0000","last_undegraded":"2023-10-06T05:27:53.856231+0000","last_fullsized":"2023-10-06T05:27:53.856231+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:19.452322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840197+0000","last_change":"2023-10-06T05:27:52.884987+0000","last_active":"2023-10-06T05:27:53.840197+0000","last_peered":"2023-10-06T05:27:53.840197+0000","last_clean":"2023-10-06T05:27:53.840197+0000","last_became_active":"2023-10-06T05:27:52.884556+0000","last_became_peered":"2023-10-06T05:27:52.884556+0000","last_unstale":"2023-10-06T05:27:53.840197+0000","last_undegraded":"2023-10-06T05:27:53.840197+0000","last_fullsized":"2023-10-06T05:27:53.840197+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:31.535290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848225+0000","last_change":"2023-10-06T05:27:53.843350+0000","last_active":"2023-10-06T05:27:53.848225+0000","last_peered":"2023-10-06T05:27:53.848225+0000","last_clean":"2023-10-06T05:27:53.848225+0000","last_became_active":"2023-10-06T05:27:53.843060+0000","last_became_peered":"2023-10-06T05:27:53.843060+0000","last_unstale":"2023-10-06T05:27:53.848225+0000","last_undegraded":"2023-10-06T05:27:53.848225+0000","last_fullsized":"2023-10-06T05:27:53.848225+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:07.444435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.838887+0000","last_change":"2023-10-06T05:27:52.895618+0000","last_active":"2023-10-06T05:27:53.838887+0000","last_peered":"2023-10-06T05:27:53.838887+0000","last_clean":"2023-10-06T05:27:53.838887+0000","last_became_active":"2023-10-06T05:27:52.895339+0000","last_became_peered":"2023-10-06T05:27:52.895339+0000","last_unstale":"2023-10-06T05:27:53.838887+0000","last_undegraded":"2023-10-06T05:27:53.838887+0000","last_fullsized":"2023-10-06T05:27:53.838887+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:13.082692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855845+0000","last_change":"2023-10-06T05:27:53.851921+0000","last_active":"2023-10-06T05:27:53.855845+0000","last_peered":"2023-10-06T05:27:53.855845+0000","last_clean":"2023-10-06T05:27:53.855845+0000","last_became_active":"2023-10-06T05:27:53.851626+0000","last_became_peered":"2023-10-06T05:27:53.851626+0000","last_unstale":"2023-10-06T05:27:53.855845+0000","last_undegraded":"2023-10-06T05:27:53.855845+0000","last_fullsized":"2023-10-06T05:27:53.855845+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:49.858880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839383+0000","last_change":"2023-10-06T05:27:52.904755+0000","last_active":"2023-10-06T05:27:53.839383+0000","last_peered":"2023-10-06T05:27:53.839383+0000","last_clean":"2023-10-06T05:27:53.839383+0000","last_became_active":"2023-10-06T05:27:52.903048+0000","last_became_peered":"2023-10-06T05:27:52.903048+0000","last_unstale":"2023-10-06T05:27:53.839383+0000","last_undegraded":"2023-10-06T05:27:53.839383+0000","last_fullsized":"2023-10-06T05:27:53.839383+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:31.918975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856514+0000","last_change":"2023-10-06T05:27:53.855315+0000","last_active":"2023-10-06T05:27:53.856514+0000","last_peered":"2023-10-06T05:27:53.856514+0000","last_clean":"2023-10-06T05:27:53.856514+0000","last_became_active":"2023-10-06T05:27:53.855042+0000","last_became_peered":"2023-10-06T05:27:53.855042+0000","last_unstale":"2023-10-06T05:27:53.856514+0000","last_undegraded":"2023-10-06T05:27:53.856514+0000","last_fullsized":"2023-10-06T05:27:53.856514+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:19.763732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848835+0000","last_change":"2023-10-06T05:27:52.888091+0000","last_active":"2023-10-06T05:27:53.848835+0000","last_peered":"2023-10-06T05:27:53.848835+0000","last_clean":"2023-10-06T05:27:53.848835+0000","last_became_active":"2023-10-06T05:27:52.887799+0000","last_became_peered":"2023-10-06T05:27:52.887799+0000","last_unstale":"2023-10-06T05:27:53.848835+0000","last_undegraded":"2023-10-06T05:27:53.848835+0000","last_fullsized":"2023-10-06T05:27:53.848835+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:33.326457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.857052+0000","last_change":"2023-10-06T05:27:53.856261+0000","last_active":"2023-10-06T05:27:53.857052+0000","last_peered":"2023-10-06T05:27:53.857052+0000","last_clean":"2023-10-06T05:27:53.857052+0000","last_became_active":"2023-10-06T05:27:53.855912+0000","last_became_peered":"2023-10-06T05:27:53.855912+0000","last_unstale":"2023-10-06T05:27:53.857052+0000","last_undegraded":"2023-10-06T05:27:53.857052+0000","last_fullsized":"2023-10-06T05:27:53.857052+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:29.745854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836227+0000","last_change":"2023-10-06T05:27:52.893719+0000","last_active":"2023-10-06T05:27:53.836227+0000","last_peered":"2023-10-06T05:27:53.836227+0000","last_clean":"2023-10-06T05:27:53.836227+0000","last_became_active":"2023-10-06T05:27:52.893443+0000","last_became_peered":"2023-10-06T05:27:52.893443+0000","last_unstale":"2023-10-06T05:27:53.836227+0000","last_undegraded":"2023-10-06T05:27:53.836227+0000","last_fullsized":"2023-10-06T05:27:53.836227+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:12.990386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.860681+0000","last_change":"2023-10-06T05:27:53.858012+0000","last_active":"2023-10-06T05:27:53.860681+0000","last_peered":"2023-10-06T05:27:53.860681+0000","last_clean":"2023-10-06T05:27:53.860681+0000","last_became_active":"2023-10-06T05:27:53.857747+0000","last_became_peered":"2023-10-06T05:27:53.857747+0000","last_unstale":"2023-10-06T05:27:53.860681+0000","last_undegraded":"2023-10-06T05:27:53.860681+0000","last_fullsized":"2023-10-06T05:27:53.860681+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:50.310597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839165+0000","last_change":"2023-10-06T05:27:52.893297+0000","last_active":"2023-10-06T05:27:53.839165+0000","last_peered":"2023-10-06T05:27:53.839165+0000","last_clean":"2023-10-06T05:27:53.839165+0000","last_became_active":"2023-10-06T05:27:52.893030+0000","last_became_peered":"2023-10-06T05:27:52.893030+0000","last_unstale":"2023-10-06T05:27:53.839165+0000","last_undegraded":"2023-10-06T05:27:53.839165+0000","last_fullsized":"2023-10-06T05:27:53.839165+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:55.626270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855826+0000","last_change":"2023-10-06T05:27:53.851659+0000","last_active":"2023-10-06T05:27:53.855826+0000","last_peered":"2023-10-06T05:27:53.855826+0000","last_clean":"2023-10-06T05:27:53.855826+0000","last_became_active":"2023-10-06T05:27:53.851355+0000","last_became_peered":"2023-10-06T05:27:53.851355+0000","last_unstale":"2023-10-06T05:27:53.855826+0000","last_undegraded":"2023-10-06T05:27:53.855826+0000","last_fullsized":"2023-10-06T05:27:53.855826+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:09.436722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.889379+0000","last_change":"2023-10-06T05:27:52.898958+0000","last_active":"2023-10-06T05:27:53.889379+0000","last_peered":"2023-10-06T05:27:53.889379+0000","last_clean":"2023-10-06T05:27:53.889379+0000","last_became_active":"2023-10-06T05:27:52.898625+0000","last_became_peered":"2023-10-06T05:27:52.898625+0000","last_unstale":"2023-10-06T05:27:53.889379+0000","last_undegraded":"2023-10-06T05:27:53.889379+0000","last_fullsized":"2023-10-06T05:27:53.889379+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:26.175340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840200+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:27:53.840200+0000","last_peered":"2023-10-06T05:27:53.840200+0000","last_clean":"2023-10-06T05:27:53.840200+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:27:53.840200+0000","last_undegraded":"2023-10-06T05:27:53.840200+0000","last_fullsized":"2023-10-06T05:27:53.840200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850702+0000","last_change":"2023-10-06T05:27:53.849046+0000","last_active":"2023-10-06T05:27:53.850702+0000","last_peered":"2023-10-06T05:27:53.850702+0000","last_clean":"2023-10-06T05:27:53.850702+0000","last_became_active":"2023-10-06T05:27:53.848447+0000","last_became_peered":"2023-10-06T05:27:53.848447+0000","last_unstale":"2023-10-06T05:27:53.850702+0000","last_undegraded":"2023-10-06T05:27:53.850702+0000","last_fullsized":"2023-10-06T05:27:53.850702+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:26.553126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.887832+0000","last_change":"2023-10-06T05:27:52.901867+0000","last_active":"2023-10-06T05:27:53.887832+0000","last_peered":"2023-10-06T05:27:53.887832+0000","last_clean":"2023-10-06T05:27:53.887832+0000","last_became_active":"2023-10-06T05:27:52.901595+0000","last_became_peered":"2023-10-06T05:27:52.901595+0000","last_unstale":"2023-10-06T05:27:53.887832+0000","last_undegraded":"2023-10-06T05:27:53.887832+0000","last_fullsized":"2023-10-06T05:27:53.887832+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:44.156324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.832036+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:27:53.832036+0000","last_peered":"2023-10-06T05:27:53.832036+0000","last_clean":"2023-10-06T05:27:53.832036+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:27:53.832036+0000","last_undegraded":"2023-10-06T05:27:53.832036+0000","last_fullsized":"2023-10-06T05:27:53.832036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850722+0000","last_change":"2023-10-06T05:27:53.846847+0000","last_active":"2023-10-06T05:27:53.850722+0000","last_peered":"2023-10-06T05:27:53.850722+0000","last_clean":"2023-10-06T05:27:53.850722+0000","last_became_active":"2023-10-06T05:27:53.844035+0000","last_became_peered":"2023-10-06T05:27:53.844035+0000","last_unstale":"2023-10-06T05:27:53.850722+0000","last_undegraded":"2023-10-06T05:27:53.850722+0000","last_fullsized":"2023-10-06T05:27:53.850722+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:37.249054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890485+0000","last_change":"2023-10-06T05:27:52.893056+0000","last_active":"2023-10-06T05:27:53.890485+0000","last_peered":"2023-10-06T05:27:53.890485+0000","last_clean":"2023-10-06T05:27:53.890485+0000","last_became_active":"2023-10-06T05:27:52.892724+0000","last_became_peered":"2023-10-06T05:27:52.892724+0000","last_unstale":"2023-10-06T05:27:53.890485+0000","last_undegraded":"2023-10-06T05:27:53.890485+0000","last_fullsized":"2023-10-06T05:27:53.890485+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.724651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.830555+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:27:53.830555+0000","last_peered":"2023-10-06T05:27:53.830555+0000","last_clean":"2023-10-06T05:27:53.830555+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:27:53.830555+0000","last_undegraded":"2023-10-06T05:27:53.830555+0000","last_fullsized":"2023-10-06T05:27:53.830555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856114+0000","last_change":"2023-10-06T05:27:53.851751+0000","last_active":"2023-10-06T05:27:53.856114+0000","last_peered":"2023-10-06T05:27:53.856114+0000","last_clean":"2023-10-06T05:27:53.856114+0000","last_became_active":"2023-10-06T05:27:53.851405+0000","last_became_peered":"2023-10-06T05:27:53.851405+0000","last_unstale":"2023-10-06T05:27:53.856114+0000","last_undegraded":"2023-10-06T05:27:53.856114+0000","last_fullsized":"2023-10-06T05:27:53.856114+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.666423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.884570+0000","last_change":"2023-10-06T05:27:52.898272+0000","last_active":"2023-10-06T05:27:53.884570+0000","last_peered":"2023-10-06T05:27:53.884570+0000","last_clean":"2023-10-06T05:27:53.884570+0000","last_became_active":"2023-10-06T05:27:52.897987+0000","last_became_peered":"2023-10-06T05:27:52.897987+0000","last_unstale":"2023-10-06T05:27:53.884570+0000","last_undegraded":"2023-10-06T05:27:53.884570+0000","last_fullsized":"2023-10-06T05:27:53.884570+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:28.976912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.833197+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:27:53.833197+0000","last_peered":"2023-10-06T05:27:53.833197+0000","last_clean":"2023-10-06T05:27:53.833197+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:27:53.833197+0000","last_undegraded":"2023-10-06T05:27:53.833197+0000","last_fullsized":"2023-10-06T05:27:53.833197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.835662+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:27:53.835662+0000","last_peered":"2023-10-06T05:27:53.835662+0000","last_clean":"2023-10-06T05:27:53.835662+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:27:53.835662+0000","last_undegraded":"2023-10-06T05:27:53.835662+0000","last_fullsized":"2023-10-06T05:27:53.835662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850620+0000","last_change":"2023-10-06T05:27:53.846645+0000","last_active":"2023-10-06T05:27:53.850620+0000","last_peered":"2023-10-06T05:27:53.850620+0000","last_clean":"2023-10-06T05:27:53.850620+0000","last_became_active":"2023-10-06T05:27:53.843484+0000","last_became_peered":"2023-10-06T05:27:53.843484+0000","last_unstale":"2023-10-06T05:27:53.850620+0000","last_undegraded":"2023-10-06T05:27:53.850620+0000","last_fullsized":"2023-10-06T05:27:53.850620+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:19.531745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"193'4","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.889482+0000","last_change":"2023-10-06T05:27:52.895926+0000","last_active":"2023-10-06T05:27:53.889482+0000","last_peered":"2023-10-06T05:27:53.889482+0000","last_clean":"2023-10-06T05:27:53.889482+0000","last_became_active":"2023-10-06T05:27:52.895594+0000","last_became_peered":"2023-10-06T05:27:52.895594+0000","last_unstale":"2023-10-06T05:27:53.889482+0000","last_undegraded":"2023-10-06T05:27:53.889482+0000","last_fullsized":"2023-10-06T05:27:53.889482+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:45.597973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836261+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:27:53.836261+0000","last_peered":"2023-10-06T05:27:53.836261+0000","last_clean":"2023-10-06T05:27:53.836261+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:27:53.836261+0000","last_undegraded":"2023-10-06T05:27:53.836261+0000","last_fullsized":"2023-10-06T05:27:53.836261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850806+0000","last_change":"2023-10-06T05:27:53.847587+0000","last_active":"2023-10-06T05:27:53.850806+0000","last_peered":"2023-10-06T05:27:53.850806+0000","last_clean":"2023-10-06T05:27:53.850806+0000","last_became_active":"2023-10-06T05:27:53.846707+0000","last_became_peered":"2023-10-06T05:27:53.846707+0000","last_unstale":"2023-10-06T05:27:53.850806+0000","last_undegraded":"2023-10-06T05:27:53.850806+0000","last_fullsized":"2023-10-06T05:27:53.850806+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:45.947811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.885130+0000","last_change":"2023-10-06T05:27:52.897389+0000","last_active":"2023-10-06T05:27:53.885130+0000","last_peered":"2023-10-06T05:27:53.885130+0000","last_clean":"2023-10-06T05:27:53.885130+0000","last_became_active":"2023-10-06T05:27:52.897055+0000","last_became_peered":"2023-10-06T05:27:52.897055+0000","last_unstale":"2023-10-06T05:27:53.885130+0000","last_undegraded":"2023-10-06T05:27:53.885130+0000","last_fullsized":"2023-10-06T05:27:53.885130+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:52.760427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836280+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:27:53.836280+0000","last_peered":"2023-10-06T05:27:53.836280+0000","last_clean":"2023-10-06T05:27:53.836280+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:27:53.836280+0000","last_undegraded":"2023-10-06T05:27:53.836280+0000","last_fullsized":"2023-10-06T05:27:53.836280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850430+0000","last_change":"2023-10-06T05:27:53.846020+0000","last_active":"2023-10-06T05:27:53.850430+0000","last_peered":"2023-10-06T05:27:53.850430+0000","last_clean":"2023-10-06T05:27:53.850430+0000","last_became_active":"2023-10-06T05:27:53.845705+0000","last_became_peered":"2023-10-06T05:27:53.845705+0000","last_unstale":"2023-10-06T05:27:53.850430+0000","last_undegraded":"2023-10-06T05:27:53.850430+0000","last_fullsized":"2023-10-06T05:27:53.850430+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.752715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848724+0000","last_change":"2023-10-06T05:27:52.899129+0000","last_active":"2023-10-06T05:27:53.848724+0000","last_peered":"2023-10-06T05:27:53.848724+0000","last_clean":"2023-10-06T05:27:53.848724+0000","last_became_active":"2023-10-06T05:27:52.898785+0000","last_became_peered":"2023-10-06T05:27:52.898785+0000","last_unstale":"2023-10-06T05:27:53.848724+0000","last_undegraded":"2023-10-06T05:27:53.848724+0000","last_fullsized":"2023-10-06T05:27:53.848724+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:48.785945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840599+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:27:53.840599+0000","last_peered":"2023-10-06T05:27:53.840599+0000","last_clean":"2023-10-06T05:27:53.840599+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:27:53.840599+0000","last_undegraded":"2023-10-06T05:27:53.840599+0000","last_fullsized":"2023-10-06T05:27:53.840599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848843+0000","last_change":"2023-10-06T05:27:53.844213+0000","last_active":"2023-10-06T05:27:53.848843+0000","last_peered":"2023-10-06T05:27:53.848843+0000","last_clean":"2023-10-06T05:27:53.848843+0000","last_became_active":"2023-10-06T05:27:53.842863+0000","last_became_peered":"2023-10-06T05:27:53.842863+0000","last_unstale":"2023-10-06T05:27:53.848843+0000","last_undegraded":"2023-10-06T05:27:53.848843+0000","last_fullsized":"2023-10-06T05:27:53.848843+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:12.704276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836707+0000","last_change":"2023-10-06T05:27:52.889906+0000","last_active":"2023-10-06T05:27:53.836707+0000","last_peered":"2023-10-06T05:27:53.836707+0000","last_clean":"2023-10-06T05:27:53.836707+0000","last_became_active":"2023-10-06T05:27:52.889643+0000","last_became_peered":"2023-10-06T05:27:52.889643+0000","last_unstale":"2023-10-06T05:27:53.836707+0000","last_undegraded":"2023-10-06T05:27:53.836707+0000","last_fullsized":"2023-10-06T05:27:53.836707+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:09.038596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839470+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:27:53.839470+0000","last_peered":"2023-10-06T05:27:53.839470+0000","last_clean":"2023-10-06T05:27:53.839470+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:27:53.839470+0000","last_undegraded":"2023-10-06T05:27:53.839470+0000","last_fullsized":"2023-10-06T05:27:53.839470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304364,"kb_used_data":2980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067476,"statfs":{"total":96636764160,"available":96325095424,"internally_reserved":0,"allocated":3051520,"data_stored":1730712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.876},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1339999999999999,"15min":1.169},"last":0.62}]},{"osd":1,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.504,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":1.012,"5min":1.0249999999999999,"15min":1.268},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.48099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":1.0609999999999999,"15min":1.075},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.99099999999999999,"15min":1.22},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.5,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0960000000000001,"15min":1.202},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.32200000000000001},"max":{"1min":1.0209999999999999,"5min":1.109,"15min":1.258},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.47299999999999998,"15min":0.39400000000000002},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.94699999999999995,"5min":1.109,"15min":1.109},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.0800000000000001,"5min":1.2290000000000001,"15min":1.407},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.55700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.117,"5min":1.169,"15min":1.169},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.097,"5min":1.135,"15min":1.3300000000000001},"last":0.68799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303800,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068040,"statfs":{"total":96636764160,"available":96325672960,"internally_reserved":0,"allocated":2473984,"data_stored":1141362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69199999999999995},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.96499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.373},"max":{"1min":0.94099999999999995,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":1.018,"5min":1.135,"15min":1.135},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.63300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.054,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.496,"5min":0.44900000000000001,"15min":0.442},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.53600000000000003,"5min":0.45300000000000001,"15min":0.40899999999999997},"max":{"1min":0.999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73299999999999998,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.1020000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45700000000000002,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.91000000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.441,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.998},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.0760000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":1.071,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70299999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":1.1000000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.745,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1359999999999999},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303776,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068064,"statfs":{"total":96636764160,"available":96325697536,"internally_reserved":0,"allocated":2449408,"data_stored":1139700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.27800000000000002},"max":{"1min":0.70299999999999996,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.28999999999999998},"max":{"1min":0.88800000000000001,"5min":1.165,"15min":2.6520000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.434,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.70999999999999996,"5min":1.077,"15min":2.7010000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.96099999999999997,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.90600000000000003,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.5389999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":0.96399999999999997,"5min":1.087,"15min":1.4690000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":0.99399999999999999,"5min":1.145,"15min":1.349},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.64600000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.034,"5min":1.2569999999999999,"15min":1.669},"last":0.87}]},{"osd":6,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.65800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.75},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.012,"5min":1.204,"15min":1.573},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82099999999999995,"15min":0.82199999999999995},"min":{"1min":0.60799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.196,"15min":1.528},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.48699999999999999},"max":{"1min":1.024,"5min":1.3049999999999999,"15min":2.6240000000000001},"last":0.68899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304356,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067484,"statfs":{"total":96636764160,"available":96325103616,"internally_reserved":0,"allocated":3043328,"data_stored":1730068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.245},"max":{"1min":0.68700000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.33200000000000002,"15min":0.307},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.622},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.59499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.503,"15min":0.496},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.56299999999999994,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":1.05,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.76700000000000002},"min":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.47799999999999998},"max":{"1min":1.1140000000000001,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.218},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.49199999999999999},"max":{"1min":1.097,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.875}]},{"osd":6,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":1.0660000000000001,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81899999999999995,"15min":0.80800000000000005},"min":{"1min":0.59199999999999997,"5min":0.53400000000000003,"15min":0.45500000000000002},"max":{"1min":1.141,"5min":1.153,"15min":1.2629999999999999},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82499999999999996,"15min":0.81299999999999994},"min":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.48599999999999999},"max":{"1min":1.1399999999999999,"5min":1.2010000000000001,"15min":1.25},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":1.046,"5min":1.274,"15min":1.2789999999999999},"last":0.88900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2457600,"data_stored":1140288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":0.58599999999999997,"5min":1.056,"15min":1.056},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.29699999999999999},"max":{"1min":0.61099999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.39800000000000002,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.66900000000000004,"5min":1.073,"15min":1.073},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.495,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.498},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.53700000000000003},"max":{"1min":0.95199999999999996,"5min":1.109,"15min":1.1379999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.47499999999999998},"max":{"1min":0.96299999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.50700000000000001,"15min":0.47299999999999998},"max":{"1min":0.97099999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.66900000000000004,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":0.96399999999999997,"5min":1.1859999999999999,"15min":1.319},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.63300000000000001,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.99399999999999999,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81000000000000005,"15min":0.79300000000000004},"min":{"1min":0.68000000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.032,"5min":1.137,"15min":1.141},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":1.284,"15min":1.284},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.82399999999999995,"15min":0.82099999999999995},"min":{"1min":0.63700000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.014,"5min":1.2,"15min":1.3520000000000001},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2453504,"data_stored":1139722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.47599999999999998},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.76000000000000001,"15min":2.0950000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.61399999999999999,"5min":0.77200000000000002,"15min":1.772},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.016,"5min":1.369,"15min":1.913},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54500000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.32900000000000001},"max":{"1min":0.71799999999999997,"5min":0.90200000000000002,"15min":1.954},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":2.0539999999999998},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73799999999999999,"15min":0.75900000000000001},"min":{"1min":0.54700000000000004,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":0.95299999999999996,"5min":1.101,"15min":1.282},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.78200000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":1.1080000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79100000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51100000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.1930000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.79900000000000004,"15min":0.81499999999999995},"min":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.53900000000000003},"max":{"1min":1.026,"5min":1.3360000000000001,"15min":1.343},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.80900000000000005},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0780000000000001,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.194,"15min":1.3640000000000001},"last":0.873},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80900000000000005,"15min":0.82799999999999996},"min":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.0980000000000001,"5min":1.1579999999999999,"15min":1.214},"last":0.89100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575016,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2453504,"data_stored":1139790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.40699999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.073},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93100000000000005,"5min":1.05,"15min":1.169},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.41199999999999998},"max":{"1min":0.85499999999999998,"5min":0.91700000000000004,"15min":1.0569999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.91100000000000003,"5min":0.96799999999999997,"15min":1.0309999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":1.117},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41499999999999998},"max":{"1min":0.877,"5min":0.97099999999999997,"15min":1.002},"last":0.78700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91400000000000003,"5min":0.94999999999999996,"15min":1.0780000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.752},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":0.91100000000000003,"5min":1.018,"15min":1.196},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":1.044,"15min":1.8740000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.97199999999999998,"5min":1.014,"15min":1.375},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54800000000000004,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.073,"15min":1.466},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.56599999999999995,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.97099999999999997,"5min":1.0569999999999999,"15min":1.218},"last":0.73999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303792,"kb_used_data":2408,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068048,"statfs":{"total":96636764160,"available":96325681152,"internally_reserved":0,"allocated":2465792,"data_stored":1141306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.374},"max":{"1min":0.82699999999999996,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.47899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.093,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.204},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71599999999999997,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0089999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.49099999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":0.94599999999999995,"15min":1.087},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.72899999999999998},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":1.145,"5min":1.145,"15min":1.278},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.294},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.81599999999999995,"5min":0.89900000000000002,"15min":1.141},"last":0.754},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.49399999999999999,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.98599999999999999,"15min":1.254},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.748},"min":{"1min":0.57199999999999995,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":1.0189999999999999,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.54700000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86599999999999999,"5min":1.0600000000000001,"15min":1.248},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:27:58.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:27:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.494+0000 7f3bd5f9d700 1 Processor -- start 2023-10-06T05:27:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.495+0000 7f3bd5f9d700 1 -- start start 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bd5f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3bd006a0d0 0x7f3bd00adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bd5f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd0158ba0 con 0x7f3bd0158700 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bd5f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd00ae490 con 0x7f3bd0155140 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bd5f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00c8e40 con 0x7f3bd006a0d0 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bcffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3bd0158700 0x7f3bd00699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52202/0 (socket says 172.21.15.33:52202) 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bcffff700 1 -- 172.21.15.33:0/3103819777 learned_addr learned my addr 172.21.15.33:0/3103819777 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:27:58.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.496+0000 7f3bd579c700 1 --2- 172.21.15.33:0/3103819777 >> v2:172.21.15.33:3300/0 conn(0x7f3bd006a0d0 0x7f3bd00adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bd579c700 1 -- 172.21.15.33:0/3103819777 >> v1:172.21.15.167:6789/0 conn(0x7f3bd0155140 legacy=0x7f3bd00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bd579c700 1 -- 172.21.15.33:0/3103819777 >> v1:172.21.15.33:6789/0 conn(0x7f3bd0158700 legacy=0x7f3bd00699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bd579c700 1 -- 172.21.15.33:0/3103819777 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00c99e0 con 0x7f3bd006a0d0 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3103819777 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099317164 0 0) 0x7f3bd0158ba0 con 0x7f3bd0158700 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bd579c700 1 --2- 172.21.15.33:0/3103819777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd006a0d0 0x7f3bd00adf50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009e90 tx=0x7f3bc8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42776709503eae60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3103819777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800e070 con 0x7f3bd006a0d0 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3103819777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800bbf0 con 0x7f3bd006a0d0 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.497+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3103819777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8012430 con 0x7f3bd006a0d0 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/3103819777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd006a0d0 msgr2=0x7f3bd00adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/3103819777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd006a0d0 0x7f3bd00adf50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009e90 tx=0x7f3bc8009780 comp rx=0 tx=0).stop 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/3103819777 shutdown_connections 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/3103819777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd006a0d0 0x7f3bd00adf50 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/3103819777 >> 172.21.15.33:0/3103819777 conn(0x7f3bd006d030 msgr2=0x7f3bd006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/3103819777 shutdown_connections 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.498+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/3103819777 wait complete. 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.499+0000 7f3bd5f9d700 1 Processor -- start 2023-10-06T05:27:58.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 -- start start 2023-10-06T05:27:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd006a0d0 0x7f3bd00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 0x7f3bd0156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd0158700 0x7f3bd00c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00b11c0 con 0x7f3bd0155140 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0157380 con 0x7f3bd0158700 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.500+0000 7f3bd5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0157660 con 0x7f3bd006a0d0 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 0x7f3bd0156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 0x7f3bd0156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34666/0 (socket says 172.21.15.33:34666) 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 -- 172.21.15.33:0/2842715681 learned_addr learned my addr 172.21.15.33:0/2842715681 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bd579c700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd0158700 0x7f3bd00c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bd4f9b700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd006a0d0 0x7f3bd00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 -- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd006a0d0 msgr2=0x7f3bd00b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd006a0d0 0x7f3bd00b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 -- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd0158700 msgr2=0x7f3bd00c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd0158700 0x7f3bd00c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 -- 172.21.15.33:0/2842715681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009a20 con 0x7f3bd0155140 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bd579c700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd0158700 0x7f3bd00c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.501+0000 7f3bcffff700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 0x7f3bd0156a50 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c810 tx=0x7f3bbc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.502+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc005c00 con 0x7f3bd0155140 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.502+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc005d60 con 0x7f3bd0155140 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.502+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc007340 con 0x7f3bd0155140 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.502+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd0157940 con 0x7f3bd0155140 2023-10-06T05:27:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.502+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd0157f00 con 0x7f3bd0155140 2023-10-06T05:27:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.503+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0158120 con 0x7f3bd0155140 2023-10-06T05:27:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.504+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bbc01b400 con 0x7f3bd0155140 2023-10-06T05:27:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.504+0000 7f3bccff9700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bb4066f30 0x7f3bb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.505+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3bbc0841e0 con 0x7f3bd0155140 2023-10-06T05:27:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.505+0000 7f3bd4f9b700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bb4066f30 0x7f3bb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.506+0000 7f3bd4f9b700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bb4066f30 0x7f3bb40693e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4003860 tx=0x7f3bc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.508+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bbc0512d0 con 0x7f3bd0155140 2023-10-06T05:27:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.705+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bd004c380 con 0x7f3bb4066f30 2023-10-06T05:27:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.715+0000 7f3bccff9700 1 -- 172.21.15.33:0/2842715681 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311443 (secure 0 0 0) 0x7f3bd004c380 con 0x7f3bb4066f30 2023-10-06T05:27:58.721 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bb4066f30 msgr2=0x7f3bb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bb4066f30 0x7f3bb40693e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4003860 tx=0x7f3bc4009040 comp rx=0 tx=0).stop 2023-10-06T05:27:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 msgr2=0x7f3bd0156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 0x7f3bd0156a50 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c810 tx=0x7f3bbc007d90 comp rx=0 tx=0).stop 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 shutdown_connections 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bb4066f30 0x7f3bb40693e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd0158700 0x7f3bd00c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd006a0d0 0x7f3bd00b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 --2- 172.21.15.33:0/2842715681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd0155140 0x7f3bd0156a50 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 >> 172.21.15.33:0/2842715681 conn(0x7f3bd006d030 msgr2=0x7f3bd00cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 shutdown_connections 2023-10-06T05:27:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.719+0000 7f3bd5f9d700 1 -- 172.21.15.33:0/2842715681 wait complete. 2023-10-06T05:27:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:27:58.739 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":600,"stamp":"2023-10-06T05:27:58.021030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431432,"kb_used_data":20360,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752543288,"statfs":{"total":773094113280,"available":770604326912,"internally_reserved":0,"allocated":20848640,"data_stored":10302948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001391"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846718+0000","last_change":"2023-10-06T05:27:53.843979+0000","last_active":"2023-10-06T05:27:53.846718+0000","last_peered":"2023-10-06T05:27:53.846718+0000","last_clean":"2023-10-06T05:27:53.846718+0000","last_became_active":"2023-10-06T05:27:53.843710+0000","last_became_peered":"2023-10-06T05:27:53.843710+0000","last_unstale":"2023-10-06T05:27:53.846718+0000","last_undegraded":"2023-10-06T05:27:53.846718+0000","last_fullsized":"2023-10-06T05:27:53.846718+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:58.861720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.849829+0000","last_change":"2023-10-06T05:27:53.846800+0000","last_active":"2023-10-06T05:27:53.849829+0000","last_peered":"2023-10-06T05:27:53.849829+0000","last_clean":"2023-10-06T05:27:53.849829+0000","last_became_active":"2023-10-06T05:27:53.846537+0000","last_became_peered":"2023-10-06T05:27:53.846537+0000","last_unstale":"2023-10-06T05:27:53.849829+0000","last_undegraded":"2023-10-06T05:27:53.849829+0000","last_fullsized":"2023-10-06T05:27:53.849829+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:37.249054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850749+0000","last_change":"2023-10-06T05:27:53.847718+0000","last_active":"2023-10-06T05:27:53.850749+0000","last_peered":"2023-10-06T05:27:53.850749+0000","last_clean":"2023-10-06T05:27:53.850749+0000","last_became_active":"2023-10-06T05:27:53.847421+0000","last_became_peered":"2023-10-06T05:27:53.847421+0000","last_unstale":"2023-10-06T05:27:53.850749+0000","last_undegraded":"2023-10-06T05:27:53.850749+0000","last_fullsized":"2023-10-06T05:27:53.850749+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:15.298112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855550+0000","last_change":"2023-10-06T05:27:53.853667+0000","last_active":"2023-10-06T05:27:53.855550+0000","last_peered":"2023-10-06T05:27:53.855550+0000","last_clean":"2023-10-06T05:27:53.855550+0000","last_became_active":"2023-10-06T05:27:53.853326+0000","last_became_peered":"2023-10-06T05:27:53.853326+0000","last_unstale":"2023-10-06T05:27:53.855550+0000","last_undegraded":"2023-10-06T05:27:53.855550+0000","last_fullsized":"2023-10-06T05:27:53.855550+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:17.334537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850707+0000","last_change":"2023-10-06T05:27:53.846468+0000","last_active":"2023-10-06T05:27:53.850707+0000","last_peered":"2023-10-06T05:27:53.850707+0000","last_clean":"2023-10-06T05:27:53.850707+0000","last_became_active":"2023-10-06T05:27:53.845896+0000","last_became_peered":"2023-10-06T05:27:53.845896+0000","last_unstale":"2023-10-06T05:27:53.850707+0000","last_undegraded":"2023-10-06T05:27:53.850707+0000","last_fullsized":"2023-10-06T05:27:53.850707+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:06.689371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.860662+0000","last_change":"2023-10-06T05:27:53.856756+0000","last_active":"2023-10-06T05:27:53.860662+0000","last_peered":"2023-10-06T05:27:53.860662+0000","last_clean":"2023-10-06T05:27:53.860662+0000","last_became_active":"2023-10-06T05:27:53.856489+0000","last_became_peered":"2023-10-06T05:27:53.856489+0000","last_unstale":"2023-10-06T05:27:53.860662+0000","last_undegraded":"2023-10-06T05:27:53.860662+0000","last_fullsized":"2023-10-06T05:27:53.860662+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:39.142251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850786+0000","last_change":"2023-10-06T05:27:53.842169+0000","last_active":"2023-10-06T05:27:53.850786+0000","last_peered":"2023-10-06T05:27:53.850786+0000","last_clean":"2023-10-06T05:27:53.850786+0000","last_became_active":"2023-10-06T05:27:53.841915+0000","last_became_peered":"2023-10-06T05:27:53.841915+0000","last_unstale":"2023-10-06T05:27:53.850786+0000","last_undegraded":"2023-10-06T05:27:53.850786+0000","last_fullsized":"2023-10-06T05:27:53.850786+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:59.465527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.860692+0000","last_change":"2023-10-06T05:27:53.856128+0000","last_active":"2023-10-06T05:27:53.860692+0000","last_peered":"2023-10-06T05:27:53.860692+0000","last_clean":"2023-10-06T05:27:53.860692+0000","last_became_active":"2023-10-06T05:27:53.854732+0000","last_became_peered":"2023-10-06T05:27:53.854732+0000","last_unstale":"2023-10-06T05:27:53.860692+0000","last_undegraded":"2023-10-06T05:27:53.860692+0000","last_fullsized":"2023-10-06T05:27:53.860692+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:17.334537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848791+0000","last_change":"2023-10-06T05:27:53.846658+0000","last_active":"2023-10-06T05:27:53.848791+0000","last_peered":"2023-10-06T05:27:53.848791+0000","last_clean":"2023-10-06T05:27:53.848791+0000","last_became_active":"2023-10-06T05:27:53.846398+0000","last_became_peered":"2023-10-06T05:27:53.846398+0000","last_unstale":"2023-10-06T05:27:53.848791+0000","last_undegraded":"2023-10-06T05:27:53.848791+0000","last_fullsized":"2023-10-06T05:27:53.848791+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:14.120267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.853639+0000","last_change":"2023-10-06T05:27:53.848757+0000","last_active":"2023-10-06T05:27:53.853639+0000","last_peered":"2023-10-06T05:27:53.853639+0000","last_clean":"2023-10-06T05:27:53.853639+0000","last_became_active":"2023-10-06T05:27:53.848480+0000","last_became_peered":"2023-10-06T05:27:53.848480+0000","last_unstale":"2023-10-06T05:27:53.853639+0000","last_undegraded":"2023-10-06T05:27:53.853639+0000","last_fullsized":"2023-10-06T05:27:53.853639+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:17.817077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855604+0000","last_change":"2023-10-06T05:27:53.854408+0000","last_active":"2023-10-06T05:27:53.855604+0000","last_peered":"2023-10-06T05:27:53.855604+0000","last_clean":"2023-10-06T05:27:53.855604+0000","last_became_active":"2023-10-06T05:27:53.854138+0000","last_became_peered":"2023-10-06T05:27:53.854138+0000","last_unstale":"2023-10-06T05:27:53.855604+0000","last_undegraded":"2023-10-06T05:27:53.855604+0000","last_fullsized":"2023-10-06T05:27:53.855604+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:03.407461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852179+0000","last_change":"2023-10-06T05:27:53.849980+0000","last_active":"2023-10-06T05:27:53.852179+0000","last_peered":"2023-10-06T05:27:53.852179+0000","last_clean":"2023-10-06T05:27:53.852179+0000","last_became_active":"2023-10-06T05:27:53.849713+0000","last_became_peered":"2023-10-06T05:27:53.849713+0000","last_unstale":"2023-10-06T05:27:53.852179+0000","last_undegraded":"2023-10-06T05:27:53.852179+0000","last_fullsized":"2023-10-06T05:27:53.852179+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:31.636043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.858957+0000","last_change":"2023-10-06T05:27:53.855056+0000","last_active":"2023-10-06T05:27:53.858957+0000","last_peered":"2023-10-06T05:27:53.858957+0000","last_clean":"2023-10-06T05:27:53.858957+0000","last_became_active":"2023-10-06T05:27:53.854777+0000","last_became_peered":"2023-10-06T05:27:53.854777+0000","last_unstale":"2023-10-06T05:27:53.858957+0000","last_undegraded":"2023-10-06T05:27:53.858957+0000","last_fullsized":"2023-10-06T05:27:53.858957+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:51.020996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.851635+0000","last_change":"2023-10-06T05:27:53.849507+0000","last_active":"2023-10-06T05:27:53.851635+0000","last_peered":"2023-10-06T05:27:53.851635+0000","last_clean":"2023-10-06T05:27:53.851635+0000","last_became_active":"2023-10-06T05:27:53.848662+0000","last_became_peered":"2023-10-06T05:27:53.848662+0000","last_unstale":"2023-10-06T05:27:53.851635+0000","last_undegraded":"2023-10-06T05:27:53.851635+0000","last_fullsized":"2023-10-06T05:27:53.851635+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:47.806726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.844358+0000","last_change":"2023-10-06T05:27:53.843438+0000","last_active":"2023-10-06T05:27:53.844358+0000","last_peered":"2023-10-06T05:27:53.844358+0000","last_clean":"2023-10-06T05:27:53.844358+0000","last_became_active":"2023-10-06T05:27:53.843180+0000","last_became_peered":"2023-10-06T05:27:53.843180+0000","last_unstale":"2023-10-06T05:27:53.844358+0000","last_undegraded":"2023-10-06T05:27:53.844358+0000","last_fullsized":"2023-10-06T05:27:53.844358+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:10.048081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.845471+0000","last_change":"2023-10-06T05:27:53.840835+0000","last_active":"2023-10-06T05:27:53.845471+0000","last_peered":"2023-10-06T05:27:53.845471+0000","last_clean":"2023-10-06T05:27:53.845471+0000","last_became_active":"2023-10-06T05:27:53.840532+0000","last_became_peered":"2023-10-06T05:27:53.840532+0000","last_unstale":"2023-10-06T05:27:53.845471+0000","last_undegraded":"2023-10-06T05:27:53.845471+0000","last_fullsized":"2023-10-06T05:27:53.845471+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:22.008412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.851563+0000","last_change":"2023-10-06T05:27:53.849408+0000","last_active":"2023-10-06T05:27:53.851563+0000","last_peered":"2023-10-06T05:27:53.851563+0000","last_clean":"2023-10-06T05:27:53.851563+0000","last_became_active":"2023-10-06T05:27:53.849054+0000","last_became_peered":"2023-10-06T05:27:53.849054+0000","last_unstale":"2023-10-06T05:27:53.851563+0000","last_undegraded":"2023-10-06T05:27:53.851563+0000","last_fullsized":"2023-10-06T05:27:53.851563+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.565485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856463+0000","last_change":"2023-10-06T05:27:53.854553+0000","last_active":"2023-10-06T05:27:53.856463+0000","last_peered":"2023-10-06T05:27:53.856463+0000","last_clean":"2023-10-06T05:27:53.856463+0000","last_became_active":"2023-10-06T05:27:53.854304+0000","last_became_peered":"2023-10-06T05:27:53.854304+0000","last_unstale":"2023-10-06T05:27:53.856463+0000","last_undegraded":"2023-10-06T05:27:53.856463+0000","last_fullsized":"2023-10-06T05:27:53.856463+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:48.950557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852368+0000","last_change":"2023-10-06T05:27:53.846850+0000","last_active":"2023-10-06T05:27:53.852368+0000","last_peered":"2023-10-06T05:27:53.852368+0000","last_clean":"2023-10-06T05:27:53.852368+0000","last_became_active":"2023-10-06T05:27:53.846310+0000","last_became_peered":"2023-10-06T05:27:53.846310+0000","last_unstale":"2023-10-06T05:27:53.852368+0000","last_undegraded":"2023-10-06T05:27:53.852368+0000","last_fullsized":"2023-10-06T05:27:53.852368+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:31.157221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.847222+0000","last_change":"2023-10-06T05:27:53.841749+0000","last_active":"2023-10-06T05:27:53.847222+0000","last_peered":"2023-10-06T05:27:53.847222+0000","last_clean":"2023-10-06T05:27:53.847222+0000","last_became_active":"2023-10-06T05:27:53.841437+0000","last_became_peered":"2023-10-06T05:27:53.841437+0000","last_unstale":"2023-10-06T05:27:53.847222+0000","last_undegraded":"2023-10-06T05:27:53.847222+0000","last_fullsized":"2023-10-06T05:27:53.847222+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:23.268651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850799+0000","last_change":"2023-10-06T05:27:53.847211+0000","last_active":"2023-10-06T05:27:53.850799+0000","last_peered":"2023-10-06T05:27:53.850799+0000","last_clean":"2023-10-06T05:27:53.850799+0000","last_became_active":"2023-10-06T05:27:53.846958+0000","last_became_peered":"2023-10-06T05:27:53.846958+0000","last_unstale":"2023-10-06T05:27:53.850799+0000","last_undegraded":"2023-10-06T05:27:53.850799+0000","last_fullsized":"2023-10-06T05:27:53.850799+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:18.780395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.844121+0000","last_change":"2023-10-06T05:27:53.841978+0000","last_active":"2023-10-06T05:27:53.844121+0000","last_peered":"2023-10-06T05:27:53.844121+0000","last_clean":"2023-10-06T05:27:53.844121+0000","last_became_active":"2023-10-06T05:27:53.841700+0000","last_became_peered":"2023-10-06T05:27:53.841700+0000","last_unstale":"2023-10-06T05:27:53.844121+0000","last_undegraded":"2023-10-06T05:27:53.844121+0000","last_fullsized":"2023-10-06T05:27:53.844121+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:27.427455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.842664+0000","last_change":"2023-10-06T05:27:53.839540+0000","last_active":"2023-10-06T05:27:53.842664+0000","last_peered":"2023-10-06T05:27:53.842664+0000","last_clean":"2023-10-06T05:27:53.842664+0000","last_became_active":"2023-10-06T05:27:53.839263+0000","last_became_peered":"2023-10-06T05:27:53.839263+0000","last_unstale":"2023-10-06T05:27:53.842664+0000","last_undegraded":"2023-10-06T05:27:53.842664+0000","last_fullsized":"2023-10-06T05:27:53.842664+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:20.657352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846827+0000","last_change":"2023-10-06T05:27:53.844929+0000","last_active":"2023-10-06T05:27:53.846827+0000","last_peered":"2023-10-06T05:27:53.846827+0000","last_clean":"2023-10-06T05:27:53.846827+0000","last_became_active":"2023-10-06T05:27:53.844625+0000","last_became_peered":"2023-10-06T05:27:53.844625+0000","last_unstale":"2023-10-06T05:27:53.846827+0000","last_undegraded":"2023-10-06T05:27:53.846827+0000","last_fullsized":"2023-10-06T05:27:53.846827+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:29.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846796+0000","last_change":"2023-10-06T05:27:53.844892+0000","last_active":"2023-10-06T05:27:53.846796+0000","last_peered":"2023-10-06T05:27:53.846796+0000","last_clean":"2023-10-06T05:27:53.846796+0000","last_became_active":"2023-10-06T05:27:53.844578+0000","last_became_peered":"2023-10-06T05:27:53.844578+0000","last_unstale":"2023-10-06T05:27:53.846796+0000","last_undegraded":"2023-10-06T05:27:53.846796+0000","last_fullsized":"2023-10-06T05:27:53.846796+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:39.804459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.854488+0000","last_change":"2023-10-06T05:27:53.850975+0000","last_active":"2023-10-06T05:27:53.854488+0000","last_peered":"2023-10-06T05:27:53.854488+0000","last_clean":"2023-10-06T05:27:53.854488+0000","last_became_active":"2023-10-06T05:27:53.849056+0000","last_became_peered":"2023-10-06T05:27:53.849056+0000","last_unstale":"2023-10-06T05:27:53.854488+0000","last_undegraded":"2023-10-06T05:27:53.854488+0000","last_fullsized":"2023-10-06T05:27:53.854488+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:05.961239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855581+0000","last_change":"2023-10-06T05:27:53.852957+0000","last_active":"2023-10-06T05:27:53.855581+0000","last_peered":"2023-10-06T05:27:53.855581+0000","last_clean":"2023-10-06T05:27:53.855581+0000","last_became_active":"2023-10-06T05:27:53.852636+0000","last_became_peered":"2023-10-06T05:27:53.852636+0000","last_unstale":"2023-10-06T05:27:53.855581+0000","last_undegraded":"2023-10-06T05:27:53.855581+0000","last_fullsized":"2023-10-06T05:27:53.855581+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:39.142251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850553+0000","last_change":"2023-10-06T05:27:53.846883+0000","last_active":"2023-10-06T05:27:53.850553+0000","last_peered":"2023-10-06T05:27:53.850553+0000","last_clean":"2023-10-06T05:27:53.850553+0000","last_became_active":"2023-10-06T05:27:53.846628+0000","last_became_peered":"2023-10-06T05:27:53.846628+0000","last_unstale":"2023-10-06T05:27:53.850553+0000","last_undegraded":"2023-10-06T05:27:53.850553+0000","last_fullsized":"2023-10-06T05:27:53.850553+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:13:50.588026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856780+0000","last_change":"2023-10-06T05:27:53.855371+0000","last_active":"2023-10-06T05:27:53.856780+0000","last_peered":"2023-10-06T05:27:53.856780+0000","last_clean":"2023-10-06T05:27:53.856780+0000","last_became_active":"2023-10-06T05:27:53.855069+0000","last_became_peered":"2023-10-06T05:27:53.855069+0000","last_unstale":"2023-10-06T05:27:53.856780+0000","last_undegraded":"2023-10-06T05:27:53.856780+0000","last_fullsized":"2023-10-06T05:27:53.856780+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:42.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856675+0000","last_change":"2023-10-06T05:27:53.849850+0000","last_active":"2023-10-06T05:27:53.856675+0000","last_peered":"2023-10-06T05:27:53.856675+0000","last_clean":"2023-10-06T05:27:53.856675+0000","last_became_active":"2023-10-06T05:27:53.849556+0000","last_became_peered":"2023-10-06T05:27:53.849556+0000","last_unstale":"2023-10-06T05:27:53.856675+0000","last_undegraded":"2023-10-06T05:27:53.856675+0000","last_fullsized":"2023-10-06T05:27:53.856675+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:02.967469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839708+0000","last_change":"2023-10-06T05:27:53.836945+0000","last_active":"2023-10-06T05:27:53.839708+0000","last_peered":"2023-10-06T05:27:53.839708+0000","last_clean":"2023-10-06T05:27:53.839708+0000","last_became_active":"2023-10-06T05:27:53.836621+0000","last_became_peered":"2023-10-06T05:27:53.836621+0000","last_unstale":"2023-10-06T05:27:53.839708+0000","last_undegraded":"2023-10-06T05:27:53.839708+0000","last_fullsized":"2023-10-06T05:27:53.839708+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:10.889227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856238+0000","last_change":"2023-10-06T05:27:53.851187+0000","last_active":"2023-10-06T05:27:53.856238+0000","last_peered":"2023-10-06T05:27:53.856238+0000","last_clean":"2023-10-06T05:27:53.856238+0000","last_became_active":"2023-10-06T05:27:53.850936+0000","last_became_peered":"2023-10-06T05:27:53.850936+0000","last_unstale":"2023-10-06T05:27:53.856238+0000","last_undegraded":"2023-10-06T05:27:53.856238+0000","last_fullsized":"2023-10-06T05:27:53.856238+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:45.961016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855953+0000","last_change":"2023-10-06T05:27:53.854784+0000","last_active":"2023-10-06T05:27:53.855953+0000","last_peered":"2023-10-06T05:27:53.855953+0000","last_clean":"2023-10-06T05:27:53.855953+0000","last_became_active":"2023-10-06T05:27:53.854519+0000","last_became_peered":"2023-10-06T05:27:53.854519+0000","last_unstale":"2023-10-06T05:27:53.855953+0000","last_undegraded":"2023-10-06T05:27:53.855953+0000","last_fullsized":"2023-10-06T05:27:53.855953+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:11.218258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.835885+0000","last_change":"2023-10-06T05:27:52.892875+0000","last_active":"2023-10-06T05:27:53.835885+0000","last_peered":"2023-10-06T05:27:53.835885+0000","last_clean":"2023-10-06T05:27:53.835885+0000","last_became_active":"2023-10-06T05:27:52.892618+0000","last_became_peered":"2023-10-06T05:27:52.892618+0000","last_unstale":"2023-10-06T05:27:53.835885+0000","last_undegraded":"2023-10-06T05:27:53.835885+0000","last_fullsized":"2023-10-06T05:27:53.835885+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:37.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850405+0000","last_change":"2023-10-06T05:27:53.845797+0000","last_active":"2023-10-06T05:27:53.850405+0000","last_peered":"2023-10-06T05:27:53.850405+0000","last_clean":"2023-10-06T05:27:53.850405+0000","last_became_active":"2023-10-06T05:27:53.845460+0000","last_became_peered":"2023-10-06T05:27:53.845460+0000","last_unstale":"2023-10-06T05:27:53.850405+0000","last_undegraded":"2023-10-06T05:27:53.850405+0000","last_fullsized":"2023-10-06T05:27:53.850405+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:19.214242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848982+0000","last_change":"2023-10-06T05:27:52.900113+0000","last_active":"2023-10-06T05:27:53.848982+0000","last_peered":"2023-10-06T05:27:53.848982+0000","last_clean":"2023-10-06T05:27:53.848982+0000","last_became_active":"2023-10-06T05:27:52.899814+0000","last_became_peered":"2023-10-06T05:27:52.899814+0000","last_unstale":"2023-10-06T05:27:53.848982+0000","last_undegraded":"2023-10-06T05:27:53.848982+0000","last_fullsized":"2023-10-06T05:27:53.848982+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:11.527906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848849+0000","last_change":"2023-10-06T05:27:53.842031+0000","last_active":"2023-10-06T05:27:53.848849+0000","last_peered":"2023-10-06T05:27:53.848849+0000","last_clean":"2023-10-06T05:27:53.848849+0000","last_became_active":"2023-10-06T05:27:53.841712+0000","last_became_peered":"2023-10-06T05:27:53.841712+0000","last_unstale":"2023-10-06T05:27:53.848849+0000","last_undegraded":"2023-10-06T05:27:53.848849+0000","last_fullsized":"2023-10-06T05:27:53.848849+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:53.529237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"193'3","reported_seq":13,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.885816+0000","last_change":"2023-10-06T05:27:52.894676+0000","last_active":"2023-10-06T05:27:53.885816+0000","last_peered":"2023-10-06T05:27:53.885816+0000","last_clean":"2023-10-06T05:27:53.885816+0000","last_became_active":"2023-10-06T05:27:52.894386+0000","last_became_peered":"2023-10-06T05:27:52.894386+0000","last_unstale":"2023-10-06T05:27:53.885816+0000","last_undegraded":"2023-10-06T05:27:53.885816+0000","last_fullsized":"2023-10-06T05:27:53.885816+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:14.073674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855191+0000","last_change":"2023-10-06T05:27:53.852617+0000","last_active":"2023-10-06T05:27:53.855191+0000","last_peered":"2023-10-06T05:27:53.855191+0000","last_clean":"2023-10-06T05:27:53.855191+0000","last_became_active":"2023-10-06T05:27:53.852336+0000","last_became_peered":"2023-10-06T05:27:53.852336+0000","last_unstale":"2023-10-06T05:27:53.855191+0000","last_undegraded":"2023-10-06T05:27:53.855191+0000","last_fullsized":"2023-10-06T05:27:53.855191+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:22.008412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"193'6","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.891839+0000","last_change":"2023-10-06T05:27:52.901867+0000","last_active":"2023-10-06T05:27:53.891839+0000","last_peered":"2023-10-06T05:27:53.891839+0000","last_clean":"2023-10-06T05:27:53.891839+0000","last_became_active":"2023-10-06T05:27:52.901578+0000","last_became_peered":"2023-10-06T05:27:52.901578+0000","last_unstale":"2023-10-06T05:27:53.891839+0000","last_undegraded":"2023-10-06T05:27:53.891839+0000","last_fullsized":"2023-10-06T05:27:53.891839+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:09.217790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.854694+0000","last_change":"2023-10-06T05:27:53.853295+0000","last_active":"2023-10-06T05:27:53.854694+0000","last_peered":"2023-10-06T05:27:53.854694+0000","last_clean":"2023-10-06T05:27:53.854694+0000","last_became_active":"2023-10-06T05:27:53.852837+0000","last_became_peered":"2023-10-06T05:27:53.852837+0000","last_unstale":"2023-10-06T05:27:53.854694+0000","last_undegraded":"2023-10-06T05:27:53.854694+0000","last_fullsized":"2023-10-06T05:27:53.854694+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:58.630568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.889972+0000","last_change":"2023-10-06T05:27:52.903325+0000","last_active":"2023-10-06T05:27:53.889972+0000","last_peered":"2023-10-06T05:27:53.889972+0000","last_clean":"2023-10-06T05:27:53.889972+0000","last_became_active":"2023-10-06T05:27:52.902997+0000","last_became_peered":"2023-10-06T05:27:52.902997+0000","last_unstale":"2023-10-06T05:27:53.889972+0000","last_undegraded":"2023-10-06T05:27:53.889972+0000","last_fullsized":"2023-10-06T05:27:53.889972+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:00.167089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852150+0000","last_change":"2023-10-06T05:27:53.849606+0000","last_active":"2023-10-06T05:27:53.852150+0000","last_peered":"2023-10-06T05:27:53.852150+0000","last_clean":"2023-10-06T05:27:53.852150+0000","last_became_active":"2023-10-06T05:27:53.849343+0000","last_became_peered":"2023-10-06T05:27:53.849343+0000","last_unstale":"2023-10-06T05:27:53.852150+0000","last_undegraded":"2023-10-06T05:27:53.852150+0000","last_fullsized":"2023-10-06T05:27:53.852150+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:34.090516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.835063+0000","last_change":"2023-10-06T05:27:52.892190+0000","last_active":"2023-10-06T05:27:53.835063+0000","last_peered":"2023-10-06T05:27:53.835063+0000","last_clean":"2023-10-06T05:27:53.835063+0000","last_became_active":"2023-10-06T05:27:52.891936+0000","last_became_peered":"2023-10-06T05:27:52.891936+0000","last_unstale":"2023-10-06T05:27:53.835063+0000","last_undegraded":"2023-10-06T05:27:53.835063+0000","last_fullsized":"2023-10-06T05:27:53.835063+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:30.147736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848573+0000","last_change":"2023-10-06T05:27:53.842157+0000","last_active":"2023-10-06T05:27:53.848573+0000","last_peered":"2023-10-06T05:27:53.848573+0000","last_clean":"2023-10-06T05:27:53.848573+0000","last_became_active":"2023-10-06T05:27:53.841891+0000","last_became_peered":"2023-10-06T05:27:53.841891+0000","last_unstale":"2023-10-06T05:27:53.848573+0000","last_undegraded":"2023-10-06T05:27:53.848573+0000","last_fullsized":"2023-10-06T05:27:53.848573+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:42.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.832503+0000","last_change":"2023-10-06T05:27:52.896976+0000","last_active":"2023-10-06T05:27:53.832503+0000","last_peered":"2023-10-06T05:27:53.832503+0000","last_clean":"2023-10-06T05:27:53.832503+0000","last_became_active":"2023-10-06T05:27:52.896648+0000","last_became_peered":"2023-10-06T05:27:52.896648+0000","last_unstale":"2023-10-06T05:27:53.832503+0000","last_undegraded":"2023-10-06T05:27:53.832503+0000","last_fullsized":"2023-10-06T05:27:53.832503+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:11.527906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.843862+0000","last_change":"2023-10-06T05:27:53.841086+0000","last_active":"2023-10-06T05:27:53.843862+0000","last_peered":"2023-10-06T05:27:53.843862+0000","last_clean":"2023-10-06T05:27:53.843862+0000","last_became_active":"2023-10-06T05:27:53.840711+0000","last_became_peered":"2023-10-06T05:27:53.840711+0000","last_unstale":"2023-10-06T05:27:53.843862+0000","last_undegraded":"2023-10-06T05:27:53.843862+0000","last_fullsized":"2023-10-06T05:27:53.843862+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:29.986397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"193'8","reported_seq":18,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.891708+0000","last_change":"2023-10-06T05:27:52.893625+0000","last_active":"2023-10-06T05:27:53.891708+0000","last_peered":"2023-10-06T05:27:53.891708+0000","last_clean":"2023-10-06T05:27:53.891708+0000","last_became_active":"2023-10-06T05:27:52.893337+0000","last_became_peered":"2023-10-06T05:27:52.893337+0000","last_unstale":"2023-10-06T05:27:53.891708+0000","last_undegraded":"2023-10-06T05:27:53.891708+0000","last_fullsized":"2023-10-06T05:27:53.891708+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:19.006016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.847003+0000","last_change":"2023-10-06T05:27:53.844309+0000","last_active":"2023-10-06T05:27:53.847003+0000","last_peered":"2023-10-06T05:27:53.847003+0000","last_clean":"2023-10-06T05:27:53.847003+0000","last_became_active":"2023-10-06T05:27:53.844059+0000","last_became_peered":"2023-10-06T05:27:53.844059+0000","last_unstale":"2023-10-06T05:27:53.847003+0000","last_undegraded":"2023-10-06T05:27:53.847003+0000","last_fullsized":"2023-10-06T05:27:53.847003+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:19.763732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.884378+0000","last_change":"2023-10-06T05:27:52.904384+0000","last_active":"2023-10-06T05:27:53.884378+0000","last_peered":"2023-10-06T05:27:53.884378+0000","last_clean":"2023-10-06T05:27:53.884378+0000","last_became_active":"2023-10-06T05:27:52.904102+0000","last_became_peered":"2023-10-06T05:27:52.904102+0000","last_unstale":"2023-10-06T05:27:53.884378+0000","last_undegraded":"2023-10-06T05:27:53.884378+0000","last_fullsized":"2023-10-06T05:27:53.884378+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:05.743229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.858857+0000","last_change":"2023-10-06T05:27:53.857906+0000","last_active":"2023-10-06T05:27:53.858857+0000","last_peered":"2023-10-06T05:27:53.858857+0000","last_clean":"2023-10-06T05:27:53.858857+0000","last_became_active":"2023-10-06T05:27:53.857592+0000","last_became_peered":"2023-10-06T05:27:53.857592+0000","last_unstale":"2023-10-06T05:27:53.858857+0000","last_undegraded":"2023-10-06T05:27:53.858857+0000","last_fullsized":"2023-10-06T05:27:53.858857+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:30.076619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.883439+0000","last_change":"2023-10-06T05:27:52.895270+0000","last_active":"2023-10-06T05:27:53.883439+0000","last_peered":"2023-10-06T05:27:53.883439+0000","last_clean":"2023-10-06T05:27:53.883439+0000","last_became_active":"2023-10-06T05:27:52.894694+0000","last_became_peered":"2023-10-06T05:27:52.894694+0000","last_unstale":"2023-10-06T05:27:53.883439+0000","last_undegraded":"2023-10-06T05:27:53.883439+0000","last_fullsized":"2023-10-06T05:27:53.883439+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:58.512050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855865+0000","last_change":"2023-10-06T05:27:53.853506+0000","last_active":"2023-10-06T05:27:53.855865+0000","last_peered":"2023-10-06T05:27:53.855865+0000","last_clean":"2023-10-06T05:27:53.855865+0000","last_became_active":"2023-10-06T05:27:53.851377+0000","last_became_peered":"2023-10-06T05:27:53.851377+0000","last_unstale":"2023-10-06T05:27:53.855865+0000","last_undegraded":"2023-10-06T05:27:53.855865+0000","last_fullsized":"2023-10-06T05:27:53.855865+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:58.630568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890099+0000","last_change":"2023-10-06T05:27:52.895108+0000","last_active":"2023-10-06T05:27:53.890099+0000","last_peered":"2023-10-06T05:27:53.890099+0000","last_clean":"2023-10-06T05:27:53.890099+0000","last_became_active":"2023-10-06T05:27:52.893584+0000","last_became_peered":"2023-10-06T05:27:52.893584+0000","last_unstale":"2023-10-06T05:27:53.890099+0000","last_undegraded":"2023-10-06T05:27:53.890099+0000","last_fullsized":"2023-10-06T05:27:53.890099+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:42.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.859192+0000","last_change":"2023-10-06T05:27:53.855552+0000","last_active":"2023-10-06T05:27:53.859192+0000","last_peered":"2023-10-06T05:27:53.859192+0000","last_clean":"2023-10-06T05:27:53.859192+0000","last_became_active":"2023-10-06T05:27:53.855253+0000","last_became_peered":"2023-10-06T05:27:53.855253+0000","last_unstale":"2023-10-06T05:27:53.859192+0000","last_undegraded":"2023-10-06T05:27:53.859192+0000","last_fullsized":"2023-10-06T05:27:53.859192+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:27.189784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.838898+0000","last_change":"2023-10-06T05:27:52.892912+0000","last_active":"2023-10-06T05:27:53.838898+0000","last_peered":"2023-10-06T05:27:53.838898+0000","last_clean":"2023-10-06T05:27:53.838898+0000","last_became_active":"2023-10-06T05:27:52.892586+0000","last_became_peered":"2023-10-06T05:27:52.892586+0000","last_unstale":"2023-10-06T05:27:53.838898+0000","last_undegraded":"2023-10-06T05:27:53.838898+0000","last_fullsized":"2023-10-06T05:27:53.838898+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:56.486823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850673+0000","last_change":"2023-10-06T05:27:53.847768+0000","last_active":"2023-10-06T05:27:53.850673+0000","last_peered":"2023-10-06T05:27:53.850673+0000","last_clean":"2023-10-06T05:27:53.850673+0000","last_became_active":"2023-10-06T05:27:53.847459+0000","last_became_peered":"2023-10-06T05:27:53.847459+0000","last_unstale":"2023-10-06T05:27:53.850673+0000","last_undegraded":"2023-10-06T05:27:53.850673+0000","last_fullsized":"2023-10-06T05:27:53.850673+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:35.732868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890789+0000","last_change":"2023-10-06T05:27:52.895569+0000","last_active":"2023-10-06T05:27:53.890789+0000","last_peered":"2023-10-06T05:27:53.890789+0000","last_clean":"2023-10-06T05:27:53.890789+0000","last_became_active":"2023-10-06T05:27:52.895260+0000","last_became_peered":"2023-10-06T05:27:52.895260+0000","last_unstale":"2023-10-06T05:27:53.890789+0000","last_undegraded":"2023-10-06T05:27:53.890789+0000","last_fullsized":"2023-10-06T05:27:53.890789+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:05.453895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.846756+0000","last_change":"2023-10-06T05:27:53.844391+0000","last_active":"2023-10-06T05:27:53.846756+0000","last_peered":"2023-10-06T05:27:53.846756+0000","last_clean":"2023-10-06T05:27:53.846756+0000","last_became_active":"2023-10-06T05:27:53.844088+0000","last_became_peered":"2023-10-06T05:27:53.844088+0000","last_unstale":"2023-10-06T05:27:53.846756+0000","last_undegraded":"2023-10-06T05:27:53.846756+0000","last_fullsized":"2023-10-06T05:27:53.846756+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:00.934914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.837938+0000","last_change":"2023-10-06T05:27:52.892977+0000","last_active":"2023-10-06T05:27:53.837938+0000","last_peered":"2023-10-06T05:27:53.837938+0000","last_clean":"2023-10-06T05:27:53.837938+0000","last_became_active":"2023-10-06T05:27:52.892711+0000","last_became_peered":"2023-10-06T05:27:52.892711+0000","last_unstale":"2023-10-06T05:27:53.837938+0000","last_undegraded":"2023-10-06T05:27:53.837938+0000","last_fullsized":"2023-10-06T05:27:53.837938+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:52.760427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.849926+0000","last_change":"2023-10-06T05:27:53.847722+0000","last_active":"2023-10-06T05:27:53.849926+0000","last_peered":"2023-10-06T05:27:53.849926+0000","last_clean":"2023-10-06T05:27:53.849926+0000","last_became_active":"2023-10-06T05:27:53.847452+0000","last_became_peered":"2023-10-06T05:27:53.847452+0000","last_unstale":"2023-10-06T05:27:53.849926+0000","last_undegraded":"2023-10-06T05:27:53.849926+0000","last_fullsized":"2023-10-06T05:27:53.849926+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:59.465527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.837921+0000","last_change":"2023-10-06T05:27:52.886594+0000","last_active":"2023-10-06T05:27:53.837921+0000","last_peered":"2023-10-06T05:27:53.837921+0000","last_clean":"2023-10-06T05:27:53.837921+0000","last_became_active":"2023-10-06T05:27:52.886337+0000","last_became_peered":"2023-10-06T05:27:52.886337+0000","last_unstale":"2023-10-06T05:27:53.837921+0000","last_undegraded":"2023-10-06T05:27:53.837921+0000","last_fullsized":"2023-10-06T05:27:53.837921+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:59.679323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.854367+0000","last_change":"2023-10-06T05:27:53.850701+0000","last_active":"2023-10-06T05:27:53.854367+0000","last_peered":"2023-10-06T05:27:53.854367+0000","last_clean":"2023-10-06T05:27:53.854367+0000","last_became_active":"2023-10-06T05:27:53.848886+0000","last_became_peered":"2023-10-06T05:27:53.848886+0000","last_unstale":"2023-10-06T05:27:53.854367+0000","last_undegraded":"2023-10-06T05:27:53.854367+0000","last_fullsized":"2023-10-06T05:27:53.854367+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:10.710631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.830001+0000","last_change":"2023-10-06T05:27:52.894905+0000","last_active":"2023-10-06T05:27:53.830001+0000","last_peered":"2023-10-06T05:27:53.830001+0000","last_clean":"2023-10-06T05:27:53.830001+0000","last_became_active":"2023-10-06T05:27:52.894632+0000","last_became_peered":"2023-10-06T05:27:52.894632+0000","last_unstale":"2023-10-06T05:27:53.830001+0000","last_undegraded":"2023-10-06T05:27:53.830001+0000","last_fullsized":"2023-10-06T05:27:53.830001+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:24.348475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.844522+0000","last_change":"2023-10-06T05:27:53.841916+0000","last_active":"2023-10-06T05:27:53.844522+0000","last_peered":"2023-10-06T05:27:53.844522+0000","last_clean":"2023-10-06T05:27:53.844522+0000","last_became_active":"2023-10-06T05:27:53.841634+0000","last_became_peered":"2023-10-06T05:27:53.841634+0000","last_unstale":"2023-10-06T05:27:53.844522+0000","last_undegraded":"2023-10-06T05:27:53.844522+0000","last_fullsized":"2023-10-06T05:27:53.844522+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:56.723977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890970+0000","last_change":"2023-10-06T05:27:52.900858+0000","last_active":"2023-10-06T05:27:53.890970+0000","last_peered":"2023-10-06T05:27:53.890970+0000","last_clean":"2023-10-06T05:27:53.890970+0000","last_became_active":"2023-10-06T05:27:52.898620+0000","last_became_peered":"2023-10-06T05:27:52.898620+0000","last_unstale":"2023-10-06T05:27:53.890970+0000","last_undegraded":"2023-10-06T05:27:53.890970+0000","last_fullsized":"2023-10-06T05:27:53.890970+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:22.003271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.852388+0000","last_change":"2023-10-06T05:27:53.847373+0000","last_active":"2023-10-06T05:27:53.852388+0000","last_peered":"2023-10-06T05:27:53.852388+0000","last_clean":"2023-10-06T05:27:53.852388+0000","last_became_active":"2023-10-06T05:27:53.847043+0000","last_became_peered":"2023-10-06T05:27:53.847043+0000","last_unstale":"2023-10-06T05:27:53.852388+0000","last_undegraded":"2023-10-06T05:27:53.852388+0000","last_fullsized":"2023-10-06T05:27:53.852388+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:09.402475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840512+0000","last_change":"2023-10-06T05:27:52.887276+0000","last_active":"2023-10-06T05:27:53.840512+0000","last_peered":"2023-10-06T05:27:53.840512+0000","last_clean":"2023-10-06T05:27:53.840512+0000","last_became_active":"2023-10-06T05:27:52.887007+0000","last_became_peered":"2023-10-06T05:27:52.887007+0000","last_unstale":"2023-10-06T05:27:53.840512+0000","last_undegraded":"2023-10-06T05:27:53.840512+0000","last_fullsized":"2023-10-06T05:27:53.840512+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:52.765913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856231+0000","last_change":"2023-10-06T05:27:53.850573+0000","last_active":"2023-10-06T05:27:53.856231+0000","last_peered":"2023-10-06T05:27:53.856231+0000","last_clean":"2023-10-06T05:27:53.856231+0000","last_became_active":"2023-10-06T05:27:53.850324+0000","last_became_peered":"2023-10-06T05:27:53.850324+0000","last_unstale":"2023-10-06T05:27:53.856231+0000","last_undegraded":"2023-10-06T05:27:53.856231+0000","last_fullsized":"2023-10-06T05:27:53.856231+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:19.452322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840197+0000","last_change":"2023-10-06T05:27:52.884987+0000","last_active":"2023-10-06T05:27:53.840197+0000","last_peered":"2023-10-06T05:27:53.840197+0000","last_clean":"2023-10-06T05:27:53.840197+0000","last_became_active":"2023-10-06T05:27:52.884556+0000","last_became_peered":"2023-10-06T05:27:52.884556+0000","last_unstale":"2023-10-06T05:27:53.840197+0000","last_undegraded":"2023-10-06T05:27:53.840197+0000","last_fullsized":"2023-10-06T05:27:53.840197+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:31.535290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848225+0000","last_change":"2023-10-06T05:27:53.843350+0000","last_active":"2023-10-06T05:27:53.848225+0000","last_peered":"2023-10-06T05:27:53.848225+0000","last_clean":"2023-10-06T05:27:53.848225+0000","last_became_active":"2023-10-06T05:27:53.843060+0000","last_became_peered":"2023-10-06T05:27:53.843060+0000","last_unstale":"2023-10-06T05:27:53.848225+0000","last_undegraded":"2023-10-06T05:27:53.848225+0000","last_fullsized":"2023-10-06T05:27:53.848225+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:07.444435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.838887+0000","last_change":"2023-10-06T05:27:52.895618+0000","last_active":"2023-10-06T05:27:53.838887+0000","last_peered":"2023-10-06T05:27:53.838887+0000","last_clean":"2023-10-06T05:27:53.838887+0000","last_became_active":"2023-10-06T05:27:52.895339+0000","last_became_peered":"2023-10-06T05:27:52.895339+0000","last_unstale":"2023-10-06T05:27:53.838887+0000","last_undegraded":"2023-10-06T05:27:53.838887+0000","last_fullsized":"2023-10-06T05:27:53.838887+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:13.082692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855845+0000","last_change":"2023-10-06T05:27:53.851921+0000","last_active":"2023-10-06T05:27:53.855845+0000","last_peered":"2023-10-06T05:27:53.855845+0000","last_clean":"2023-10-06T05:27:53.855845+0000","last_became_active":"2023-10-06T05:27:53.851626+0000","last_became_peered":"2023-10-06T05:27:53.851626+0000","last_unstale":"2023-10-06T05:27:53.855845+0000","last_undegraded":"2023-10-06T05:27:53.855845+0000","last_fullsized":"2023-10-06T05:27:53.855845+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:49.858880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839383+0000","last_change":"2023-10-06T05:27:52.904755+0000","last_active":"2023-10-06T05:27:53.839383+0000","last_peered":"2023-10-06T05:27:53.839383+0000","last_clean":"2023-10-06T05:27:53.839383+0000","last_became_active":"2023-10-06T05:27:52.903048+0000","last_became_peered":"2023-10-06T05:27:52.903048+0000","last_unstale":"2023-10-06T05:27:53.839383+0000","last_undegraded":"2023-10-06T05:27:53.839383+0000","last_fullsized":"2023-10-06T05:27:53.839383+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:31.918975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856514+0000","last_change":"2023-10-06T05:27:53.855315+0000","last_active":"2023-10-06T05:27:53.856514+0000","last_peered":"2023-10-06T05:27:53.856514+0000","last_clean":"2023-10-06T05:27:53.856514+0000","last_became_active":"2023-10-06T05:27:53.855042+0000","last_became_peered":"2023-10-06T05:27:53.855042+0000","last_unstale":"2023-10-06T05:27:53.856514+0000","last_undegraded":"2023-10-06T05:27:53.856514+0000","last_fullsized":"2023-10-06T05:27:53.856514+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:19.763732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848835+0000","last_change":"2023-10-06T05:27:52.888091+0000","last_active":"2023-10-06T05:27:53.848835+0000","last_peered":"2023-10-06T05:27:53.848835+0000","last_clean":"2023-10-06T05:27:53.848835+0000","last_became_active":"2023-10-06T05:27:52.887799+0000","last_became_peered":"2023-10-06T05:27:52.887799+0000","last_unstale":"2023-10-06T05:27:53.848835+0000","last_undegraded":"2023-10-06T05:27:53.848835+0000","last_fullsized":"2023-10-06T05:27:53.848835+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:33.326457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.857052+0000","last_change":"2023-10-06T05:27:53.856261+0000","last_active":"2023-10-06T05:27:53.857052+0000","last_peered":"2023-10-06T05:27:53.857052+0000","last_clean":"2023-10-06T05:27:53.857052+0000","last_became_active":"2023-10-06T05:27:53.855912+0000","last_became_peered":"2023-10-06T05:27:53.855912+0000","last_unstale":"2023-10-06T05:27:53.857052+0000","last_undegraded":"2023-10-06T05:27:53.857052+0000","last_fullsized":"2023-10-06T05:27:53.857052+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:29.745854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836227+0000","last_change":"2023-10-06T05:27:52.893719+0000","last_active":"2023-10-06T05:27:53.836227+0000","last_peered":"2023-10-06T05:27:53.836227+0000","last_clean":"2023-10-06T05:27:53.836227+0000","last_became_active":"2023-10-06T05:27:52.893443+0000","last_became_peered":"2023-10-06T05:27:52.893443+0000","last_unstale":"2023-10-06T05:27:53.836227+0000","last_undegraded":"2023-10-06T05:27:53.836227+0000","last_fullsized":"2023-10-06T05:27:53.836227+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:12.990386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.860681+0000","last_change":"2023-10-06T05:27:53.858012+0000","last_active":"2023-10-06T05:27:53.860681+0000","last_peered":"2023-10-06T05:27:53.860681+0000","last_clean":"2023-10-06T05:27:53.860681+0000","last_became_active":"2023-10-06T05:27:53.857747+0000","last_became_peered":"2023-10-06T05:27:53.857747+0000","last_unstale":"2023-10-06T05:27:53.860681+0000","last_undegraded":"2023-10-06T05:27:53.860681+0000","last_fullsized":"2023-10-06T05:27:53.860681+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:50.310597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839165+0000","last_change":"2023-10-06T05:27:52.893297+0000","last_active":"2023-10-06T05:27:53.839165+0000","last_peered":"2023-10-06T05:27:53.839165+0000","last_clean":"2023-10-06T05:27:53.839165+0000","last_became_active":"2023-10-06T05:27:52.893030+0000","last_became_peered":"2023-10-06T05:27:52.893030+0000","last_unstale":"2023-10-06T05:27:53.839165+0000","last_undegraded":"2023-10-06T05:27:53.839165+0000","last_fullsized":"2023-10-06T05:27:53.839165+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:19:55.626270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.855826+0000","last_change":"2023-10-06T05:27:53.851659+0000","last_active":"2023-10-06T05:27:53.855826+0000","last_peered":"2023-10-06T05:27:53.855826+0000","last_clean":"2023-10-06T05:27:53.855826+0000","last_became_active":"2023-10-06T05:27:53.851355+0000","last_became_peered":"2023-10-06T05:27:53.851355+0000","last_unstale":"2023-10-06T05:27:53.855826+0000","last_undegraded":"2023-10-06T05:27:53.855826+0000","last_fullsized":"2023-10-06T05:27:53.855826+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:09.436722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.889379+0000","last_change":"2023-10-06T05:27:52.898958+0000","last_active":"2023-10-06T05:27:53.889379+0000","last_peered":"2023-10-06T05:27:53.889379+0000","last_clean":"2023-10-06T05:27:53.889379+0000","last_became_active":"2023-10-06T05:27:52.898625+0000","last_became_peered":"2023-10-06T05:27:52.898625+0000","last_unstale":"2023-10-06T05:27:53.889379+0000","last_undegraded":"2023-10-06T05:27:53.889379+0000","last_fullsized":"2023-10-06T05:27:53.889379+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:26.175340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840200+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:27:53.840200+0000","last_peered":"2023-10-06T05:27:53.840200+0000","last_clean":"2023-10-06T05:27:53.840200+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:27:53.840200+0000","last_undegraded":"2023-10-06T05:27:53.840200+0000","last_fullsized":"2023-10-06T05:27:53.840200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850702+0000","last_change":"2023-10-06T05:27:53.849046+0000","last_active":"2023-10-06T05:27:53.850702+0000","last_peered":"2023-10-06T05:27:53.850702+0000","last_clean":"2023-10-06T05:27:53.850702+0000","last_became_active":"2023-10-06T05:27:53.848447+0000","last_became_peered":"2023-10-06T05:27:53.848447+0000","last_unstale":"2023-10-06T05:27:53.850702+0000","last_undegraded":"2023-10-06T05:27:53.850702+0000","last_fullsized":"2023-10-06T05:27:53.850702+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:26.553126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.887832+0000","last_change":"2023-10-06T05:27:52.901867+0000","last_active":"2023-10-06T05:27:53.887832+0000","last_peered":"2023-10-06T05:27:53.887832+0000","last_clean":"2023-10-06T05:27:53.887832+0000","last_became_active":"2023-10-06T05:27:52.901595+0000","last_became_peered":"2023-10-06T05:27:52.901595+0000","last_unstale":"2023-10-06T05:27:53.887832+0000","last_undegraded":"2023-10-06T05:27:53.887832+0000","last_fullsized":"2023-10-06T05:27:53.887832+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:44.156324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.832036+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:27:53.832036+0000","last_peered":"2023-10-06T05:27:53.832036+0000","last_clean":"2023-10-06T05:27:53.832036+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:27:53.832036+0000","last_undegraded":"2023-10-06T05:27:53.832036+0000","last_fullsized":"2023-10-06T05:27:53.832036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850722+0000","last_change":"2023-10-06T05:27:53.846847+0000","last_active":"2023-10-06T05:27:53.850722+0000","last_peered":"2023-10-06T05:27:53.850722+0000","last_clean":"2023-10-06T05:27:53.850722+0000","last_became_active":"2023-10-06T05:27:53.844035+0000","last_became_peered":"2023-10-06T05:27:53.844035+0000","last_unstale":"2023-10-06T05:27:53.850722+0000","last_undegraded":"2023-10-06T05:27:53.850722+0000","last_fullsized":"2023-10-06T05:27:53.850722+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:37.249054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.890485+0000","last_change":"2023-10-06T05:27:52.893056+0000","last_active":"2023-10-06T05:27:53.890485+0000","last_peered":"2023-10-06T05:27:53.890485+0000","last_clean":"2023-10-06T05:27:53.890485+0000","last_became_active":"2023-10-06T05:27:52.892724+0000","last_became_peered":"2023-10-06T05:27:52.892724+0000","last_unstale":"2023-10-06T05:27:53.890485+0000","last_undegraded":"2023-10-06T05:27:53.890485+0000","last_fullsized":"2023-10-06T05:27:53.890485+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:52.724651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.830555+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:27:53.830555+0000","last_peered":"2023-10-06T05:27:53.830555+0000","last_clean":"2023-10-06T05:27:53.830555+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:27:53.830555+0000","last_undegraded":"2023-10-06T05:27:53.830555+0000","last_fullsized":"2023-10-06T05:27:53.830555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.856114+0000","last_change":"2023-10-06T05:27:53.851751+0000","last_active":"2023-10-06T05:27:53.856114+0000","last_peered":"2023-10-06T05:27:53.856114+0000","last_clean":"2023-10-06T05:27:53.856114+0000","last_became_active":"2023-10-06T05:27:53.851405+0000","last_became_peered":"2023-10-06T05:27:53.851405+0000","last_unstale":"2023-10-06T05:27:53.856114+0000","last_undegraded":"2023-10-06T05:27:53.856114+0000","last_fullsized":"2023-10-06T05:27:53.856114+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.666423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"193'2","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.884570+0000","last_change":"2023-10-06T05:27:52.898272+0000","last_active":"2023-10-06T05:27:53.884570+0000","last_peered":"2023-10-06T05:27:53.884570+0000","last_clean":"2023-10-06T05:27:53.884570+0000","last_became_active":"2023-10-06T05:27:52.897987+0000","last_became_peered":"2023-10-06T05:27:52.897987+0000","last_unstale":"2023-10-06T05:27:53.884570+0000","last_undegraded":"2023-10-06T05:27:53.884570+0000","last_fullsized":"2023-10-06T05:27:53.884570+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:28.976912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.833197+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:27:53.833197+0000","last_peered":"2023-10-06T05:27:53.833197+0000","last_clean":"2023-10-06T05:27:53.833197+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:27:53.833197+0000","last_undegraded":"2023-10-06T05:27:53.833197+0000","last_fullsized":"2023-10-06T05:27:53.833197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.835662+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:27:53.835662+0000","last_peered":"2023-10-06T05:27:53.835662+0000","last_clean":"2023-10-06T05:27:53.835662+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:27:53.835662+0000","last_undegraded":"2023-10-06T05:27:53.835662+0000","last_fullsized":"2023-10-06T05:27:53.835662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850620+0000","last_change":"2023-10-06T05:27:53.846645+0000","last_active":"2023-10-06T05:27:53.850620+0000","last_peered":"2023-10-06T05:27:53.850620+0000","last_clean":"2023-10-06T05:27:53.850620+0000","last_became_active":"2023-10-06T05:27:53.843484+0000","last_became_peered":"2023-10-06T05:27:53.843484+0000","last_unstale":"2023-10-06T05:27:53.850620+0000","last_undegraded":"2023-10-06T05:27:53.850620+0000","last_fullsized":"2023-10-06T05:27:53.850620+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:19.531745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"193'4","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.889482+0000","last_change":"2023-10-06T05:27:52.895926+0000","last_active":"2023-10-06T05:27:53.889482+0000","last_peered":"2023-10-06T05:27:53.889482+0000","last_clean":"2023-10-06T05:27:53.889482+0000","last_became_active":"2023-10-06T05:27:52.895594+0000","last_became_peered":"2023-10-06T05:27:52.895594+0000","last_unstale":"2023-10-06T05:27:53.889482+0000","last_undegraded":"2023-10-06T05:27:53.889482+0000","last_fullsized":"2023-10-06T05:27:53.889482+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:45.597973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836261+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:27:53.836261+0000","last_peered":"2023-10-06T05:27:53.836261+0000","last_clean":"2023-10-06T05:27:53.836261+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:27:53.836261+0000","last_undegraded":"2023-10-06T05:27:53.836261+0000","last_fullsized":"2023-10-06T05:27:53.836261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850806+0000","last_change":"2023-10-06T05:27:53.847587+0000","last_active":"2023-10-06T05:27:53.850806+0000","last_peered":"2023-10-06T05:27:53.850806+0000","last_clean":"2023-10-06T05:27:53.850806+0000","last_became_active":"2023-10-06T05:27:53.846707+0000","last_became_peered":"2023-10-06T05:27:53.846707+0000","last_unstale":"2023-10-06T05:27:53.850806+0000","last_undegraded":"2023-10-06T05:27:53.850806+0000","last_fullsized":"2023-10-06T05:27:53.850806+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:45.947811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"193'1","reported_seq":11,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.885130+0000","last_change":"2023-10-06T05:27:52.897389+0000","last_active":"2023-10-06T05:27:53.885130+0000","last_peered":"2023-10-06T05:27:53.885130+0000","last_clean":"2023-10-06T05:27:53.885130+0000","last_became_active":"2023-10-06T05:27:52.897055+0000","last_became_peered":"2023-10-06T05:27:52.897055+0000","last_unstale":"2023-10-06T05:27:53.885130+0000","last_undegraded":"2023-10-06T05:27:53.885130+0000","last_fullsized":"2023-10-06T05:27:53.885130+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:52.760427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836280+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:27:53.836280+0000","last_peered":"2023-10-06T05:27:53.836280+0000","last_clean":"2023-10-06T05:27:53.836280+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:27:53.836280+0000","last_undegraded":"2023-10-06T05:27:53.836280+0000","last_fullsized":"2023-10-06T05:27:53.836280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.850430+0000","last_change":"2023-10-06T05:27:53.846020+0000","last_active":"2023-10-06T05:27:53.850430+0000","last_peered":"2023-10-06T05:27:53.850430+0000","last_clean":"2023-10-06T05:27:53.850430+0000","last_became_active":"2023-10-06T05:27:53.845705+0000","last_became_peered":"2023-10-06T05:27:53.845705+0000","last_unstale":"2023-10-06T05:27:53.850430+0000","last_undegraded":"2023-10-06T05:27:53.850430+0000","last_fullsized":"2023-10-06T05:27:53.850430+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:06.752715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848724+0000","last_change":"2023-10-06T05:27:52.899129+0000","last_active":"2023-10-06T05:27:53.848724+0000","last_peered":"2023-10-06T05:27:53.848724+0000","last_clean":"2023-10-06T05:27:53.848724+0000","last_became_active":"2023-10-06T05:27:52.898785+0000","last_became_peered":"2023-10-06T05:27:52.898785+0000","last_unstale":"2023-10-06T05:27:53.848724+0000","last_undegraded":"2023-10-06T05:27:53.848724+0000","last_fullsized":"2023-10-06T05:27:53.848724+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:48.785945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.840599+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:27:53.840599+0000","last_peered":"2023-10-06T05:27:53.840599+0000","last_clean":"2023-10-06T05:27:53.840599+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:27:53.840599+0000","last_undegraded":"2023-10-06T05:27:53.840599+0000","last_fullsized":"2023-10-06T05:27:53.840599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.848843+0000","last_change":"2023-10-06T05:27:53.844213+0000","last_active":"2023-10-06T05:27:53.848843+0000","last_peered":"2023-10-06T05:27:53.848843+0000","last_clean":"2023-10-06T05:27:53.848843+0000","last_became_active":"2023-10-06T05:27:53.842863+0000","last_became_peered":"2023-10-06T05:27:53.842863+0000","last_unstale":"2023-10-06T05:27:53.848843+0000","last_undegraded":"2023-10-06T05:27:53.848843+0000","last_fullsized":"2023-10-06T05:27:53.848843+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:52.866687+0000","last_clean_scrub_stamp":"2023-10-06T05:27:52.866687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:12.704276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.836707+0000","last_change":"2023-10-06T05:27:52.889906+0000","last_active":"2023-10-06T05:27:53.836707+0000","last_peered":"2023-10-06T05:27:53.836707+0000","last_clean":"2023-10-06T05:27:53.836707+0000","last_became_active":"2023-10-06T05:27:52.889643+0000","last_became_peered":"2023-10-06T05:27:52.889643+0000","last_unstale":"2023-10-06T05:27:53.836707+0000","last_undegraded":"2023-10-06T05:27:53.836707+0000","last_fullsized":"2023-10-06T05:27:53.836707+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:51.857202+0000","last_clean_scrub_stamp":"2023-10-06T05:27:51.857202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:09.038596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":193,"state":"active+clean","last_fresh":"2023-10-06T05:27:53.839470+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:27:53.839470+0000","last_peered":"2023-10-06T05:27:53.839470+0000","last_clean":"2023-10-06T05:27:53.839470+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:27:53.839470+0000","last_undegraded":"2023-10-06T05:27:53.839470+0000","last_fullsized":"2023-10-06T05:27:53.839470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304364,"kb_used_data":2980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067476,"statfs":{"total":96636764160,"available":96325095424,"internally_reserved":0,"allocated":3051520,"data_stored":1730712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.876},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1339999999999999,"15min":1.169},"last":0.62}]},{"osd":1,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.504,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":1.012,"5min":1.0249999999999999,"15min":1.268},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.48099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":1.0609999999999999,"15min":1.075},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.92700000000000005,"5min":0.99099999999999999,"15min":1.22},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.5,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0960000000000001,"15min":1.202},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.32200000000000001},"max":{"1min":1.0209999999999999,"5min":1.109,"15min":1.258},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.47299999999999998,"15min":0.39400000000000002},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.2849999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.94699999999999995,"5min":1.109,"15min":1.109},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.745,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.0800000000000001,"5min":1.2290000000000001,"15min":1.407},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.55700000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.117,"5min":1.169,"15min":1.169},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.097,"5min":1.135,"15min":1.3300000000000001},"last":0.68799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303800,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068040,"statfs":{"total":96636764160,"available":96325672960,"internally_reserved":0,"allocated":2473984,"data_stored":1141362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69199999999999995},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":0.96499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.373},"max":{"1min":0.94099999999999995,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":1.018,"5min":1.135,"15min":1.135},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.63300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.054,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.496,"5min":0.44900000000000001,"15min":0.442},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.53600000000000003,"5min":0.45300000000000001,"15min":0.40899999999999997},"max":{"1min":0.999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73299999999999998,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.1020000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45700000000000002,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.91000000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.441,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.998},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.0760000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":1.071,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70299999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":1.1000000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.745,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.1359999999999999},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303776,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068064,"statfs":{"total":96636764160,"available":96325697536,"internally_reserved":0,"allocated":2449408,"data_stored":1139700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.27800000000000002},"max":{"1min":0.70299999999999996,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.28999999999999998},"max":{"1min":0.88800000000000001,"5min":1.165,"15min":2.6520000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.434,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.70999999999999996,"5min":1.077,"15min":2.7010000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.96099999999999997,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.90600000000000003,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.5389999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":0.96399999999999997,"5min":1.087,"15min":1.4690000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":0.99399999999999999,"5min":1.145,"15min":1.349},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.64600000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.034,"5min":1.2569999999999999,"15min":1.669},"last":0.87}]},{"osd":6,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.65800000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.75},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83199999999999996,"15min":0.83899999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.012,"5min":1.204,"15min":1.573},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82099999999999995,"15min":0.82199999999999995},"min":{"1min":0.60799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.196,"15min":1.528},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.48699999999999999},"max":{"1min":1.024,"5min":1.3049999999999999,"15min":2.6240000000000001},"last":0.68899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304356,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067484,"statfs":{"total":96636764160,"available":96325103616,"internally_reserved":0,"allocated":3043328,"data_stored":1730068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.245},"max":{"1min":0.68700000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.36499999999999999,"5min":0.33200000000000002,"15min":0.307},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.622},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.59499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.503,"15min":0.496},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45500000000000002,"5min":0.41299999999999998,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.56299999999999994,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":1.05,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.76700000000000002},"min":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.47799999999999998},"max":{"1min":1.1140000000000001,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.218},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.49199999999999999},"max":{"1min":1.097,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.875}]},{"osd":6,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":1.0660000000000001,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81899999999999995,"15min":0.80800000000000005},"min":{"1min":0.59199999999999997,"5min":0.53400000000000003,"15min":0.45500000000000002},"max":{"1min":1.141,"5min":1.153,"15min":1.2629999999999999},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82499999999999996,"15min":0.81299999999999994},"min":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.48599999999999999},"max":{"1min":1.1399999999999999,"5min":1.2010000000000001,"15min":1.25},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":1.046,"5min":1.274,"15min":1.2789999999999999},"last":0.88900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2457600,"data_stored":1140288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.27200000000000002},"max":{"1min":0.58599999999999997,"5min":1.056,"15min":1.056},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.29699999999999999},"max":{"1min":0.61099999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.39800000000000002,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.63700000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.41199999999999998},"max":{"1min":0.66900000000000004,"5min":1.073,"15min":1.073},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.495,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.498},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.53700000000000003},"max":{"1min":0.95199999999999996,"5min":1.109,"15min":1.1379999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.47499999999999998},"max":{"1min":0.96299999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.50700000000000001,"15min":0.47299999999999998},"max":{"1min":0.97099999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.66900000000000004,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":0.96399999999999997,"5min":1.1859999999999999,"15min":1.319},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.63300000000000001,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.99399999999999999,"5min":1.1830000000000001,"15min":1.5289999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81000000000000005,"15min":0.79300000000000004},"min":{"1min":0.68000000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.032,"5min":1.137,"15min":1.141},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 05:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.64900000000000002,"5min":0.59699999999999998,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":1.284,"15min":1.284},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.82399999999999995,"15min":0.82099999999999995},"min":{"1min":0.63700000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.014,"5min":1.2,"15min":1.3520000000000001},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2453504,"data_stored":1139722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45400000000000001,"15min":0.47599999999999998},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.76000000000000001,"15min":2.0950000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.61399999999999999,"5min":0.77200000000000002,"15min":1.772},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.016,"5min":1.369,"15min":1.913},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54500000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.32900000000000001},"max":{"1min":0.71799999999999997,"5min":0.90200000000000002,"15min":1.954},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":2.0539999999999998},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73799999999999999,"15min":0.75900000000000001},"min":{"1min":0.54700000000000004,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":0.95299999999999996,"5min":1.101,"15min":1.282},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.78200000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":1.1080000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.79100000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51100000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.1930000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.79900000000000004,"15min":0.81499999999999995},"min":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.53900000000000003},"max":{"1min":1.026,"5min":1.3360000000000001,"15min":1.343},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.80900000000000005},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0780000000000001,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"min":{"1min":0.628,"5min":0.59999999999999998,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.194,"15min":1.3640000000000001},"last":0.873},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80900000000000005,"15min":0.82799999999999996},"min":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.0980000000000001,"5min":1.1579999999999999,"15min":1.214},"last":0.89100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575016,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2453504,"data_stored":1139790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.40699999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.073},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93100000000000005,"5min":1.05,"15min":1.169},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.41199999999999998},"max":{"1min":0.85499999999999998,"5min":0.91700000000000004,"15min":1.0569999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.91100000000000003,"5min":0.96799999999999997,"15min":1.0309999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":1.117},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41499999999999998},"max":{"1min":0.877,"5min":0.97099999999999997,"15min":1.002},"last":0.78700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91400000000000003,"5min":0.94999999999999996,"15min":1.0780000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.752},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":0.91100000000000003,"5min":1.018,"15min":1.196},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":1.044,"15min":1.8740000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.97199999999999998,"5min":1.014,"15min":1.375},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54800000000000004,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.073,"15min":1.466},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.56599999999999995,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.97099999999999997,"5min":1.0569999999999999,"15min":1.218},"last":0.73999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303792,"kb_used_data":2408,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068048,"statfs":{"total":96636764160,"available":96325681152,"internally_reserved":0,"allocated":2465792,"data_stored":1141306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.374},"max":{"1min":0.82699999999999996,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.47899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.093,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.204},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71599999999999997,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0089999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.49099999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":0.94599999999999995,"15min":1.087},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72199999999999998,"15min":0.72899999999999998},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":1.145,"5min":1.145,"15min":1.278},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.0089999999999999,"15min":1.1379999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.294},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.81599999999999995,"5min":0.89900000000000002,"15min":1.141},"last":0.754},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.49399999999999999,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.98599999999999999,"15min":1.254},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.748},"min":{"1min":0.57199999999999995,"5min":0.44800000000000001,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":1.0189999999999999,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.54700000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86599999999999999,"5min":1.0600000000000001,"15min":1.248},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:27:58.745 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:27:58.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:27:58.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.976+0000 7f5e4dd9d700 1 Processor -- start 2023-10-06T05:27:58.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.976+0000 7f5e4dd9d700 1 -- start start 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.977+0000 7f5e4dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e48155bd0 0x7f5e480c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.977+0000 7f5e4dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5e480ac8b0 0x7f5e4806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.977+0000 7f5e4dd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4806aab0 con 0x7f5e480ac460 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.977+0000 7f5e4dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e48069ea0 con 0x7f5e480ac8b0 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.977+0000 7f5e4dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4806a180 con 0x7f5e48155bd0 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5e480ac8b0 0x7f5e4806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5e480ac8b0 0x7f5e4806b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34674/0 (socket says 172.21.15.33:34674) 2023-10-06T05:27:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 -- 172.21.15.33:0/1085191441 learned_addr learned my addr 172.21.15.33:0/1085191441 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4cd9b700 1 --2- 172.21.15.33:0/1085191441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e48155bd0 0x7f5e480c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 -- 172.21.15.33:0/1085191441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e48155bd0 msgr2=0x7f5e480c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 --2- 172.21.15.33:0/1085191441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e48155bd0 0x7f5e480c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 -- 172.21.15.33:0/1085191441 >> v1:172.21.15.167:6789/0 conn(0x7f5e480ac460 legacy=0x7f5e4806af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 -- 172.21.15.33:0/1085191441 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4806a460 con 0x7f5e480ac8b0 2023-10-06T05:27:58.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.978+0000 7f5e4d59c700 1 --2- 172.21.15.33:0/1085191441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac8b0 0x7f5e4806b650 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009f20 tx=0x7f5e40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c290f564eb7cb49d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e477fe700 1 -- 172.21.15.33:0/1085191441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000e070 con 0x7f5e480ac8b0 2023-10-06T05:27:58.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e477fe700 1 -- 172.21.15.33:0/1085191441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4000ba80 con 0x7f5e480ac8b0 2023-10-06T05:27:58.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e477fe700 1 -- 172.21.15.33:0/1085191441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e400092e0 con 0x7f5e480ac8b0 2023-10-06T05:27:58.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1085191441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac8b0 msgr2=0x7f5e4806b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1085191441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac8b0 0x7f5e4806b650 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009f20 tx=0x7f5e40009840 comp rx=0 tx=0).stop 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1085191441 shutdown_connections 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1085191441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac8b0 0x7f5e4806b650 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1085191441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e48155bd0 0x7f5e480c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1085191441 >> 172.21.15.33:0/1085191441 conn(0x7f5e4806c560 msgr2=0x7f5e4806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1085191441 shutdown_connections 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.979+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1085191441 wait complete. 2023-10-06T05:27:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.981+0000 7f5e4dd9d700 1 Processor -- start 2023-10-06T05:27:58.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.981+0000 7f5e4dd9d700 1 -- start start 2023-10-06T05:27:58.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 0x7f5e480ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e480ac8b0 0x7f5e4815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e48155bd0 0x7f5e48152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480aaa70 con 0x7f5e480ac460 2023-10-06T05:27:58.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480cad10 con 0x7f5e48155bd0 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480caff0 con 0x7f5e480ac8b0 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 0x7f5e480ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 0x7f5e480ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34678/0 (socket says 172.21.15.33:34678) 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 -- 172.21.15.33:0/1330687163 learned_addr learned my addr 172.21.15.33:0/1330687163 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e47fff700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e480ac8b0 0x7f5e4815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4d59c700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e48155bd0 0x7f5e48152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 -- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e480ac8b0 msgr2=0x7f5e4815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e480ac8b0 0x7f5e4815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 -- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e48155bd0 msgr2=0x7f5e48152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.982+0000 7f5e4cd9b700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e48155bd0 0x7f5e48152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e4cd9b700 1 -- 172.21.15.33:0/1330687163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40009ae0 con 0x7f5e480ac460 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e4cd9b700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 0x7f5e480ca410 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00d860 tx=0x7f5e3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0125f0 con 0x7f5e480ac460 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c012750 con 0x7f5e480ac460 2023-10-06T05:27:58.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480cb2d0 con 0x7f5e480ac460 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c012b10 con 0x7f5e480ac460 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.983+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480ca950 con 0x7f5e480ac460 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.984+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e480aa8d0 con 0x7f5e480ac460 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.985+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e3c01e460 con 0x7f5e480ac460 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.986+0000 7f5e44ff9700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e2c066f30 0x7f5e2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.986+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e3c084e70 con 0x7f5e480ac460 2023-10-06T05:27:58.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.986+0000 7f5e47fff700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e2c066f30 0x7f5e2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.987+0000 7f5e47fff700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e2c066f30 0x7f5e2c0693e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5e480b0010 tx=0x7f5e34009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:58.991+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e480aa8d0 con 0x7f5e480ac460 2023-10-06T05:27:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.277+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5e480aa8d0 con 0x7f5e480ac460 2023-10-06T05:27:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.281+0000 7f5e44ff9700 1 -- 172.21.15.33:0/1330687163 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5e480aa8d0 con 0x7f5e480ac460 2023-10-06T05:27:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.283+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e2c066f30 msgr2=0x7f5e2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.283+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e2c066f30 0x7f5e2c0693e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5e480b0010 tx=0x7f5e34009040 comp rx=0 tx=0).stop 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 msgr2=0x7f5e480ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 0x7f5e480ca410 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00d860 tx=0x7f5e3c008040 comp rx=0 tx=0).stop 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 shutdown_connections 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5e2c066f30 0x7f5e2c0693e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e48155bd0 0x7f5e48152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e480ac460 0x7f5e480ca410 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 --2- 172.21.15.33:0/1330687163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5e480ac8b0 0x7f5e4815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 >> 172.21.15.33:0/1330687163 conn(0x7f5e4806c560 msgr2=0x7f5e48154ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 shutdown_connections 2023-10-06T05:27:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.284+0000 7f5e4dd9d700 1 -- 172.21.15.33:0/1330687163 wait complete. 2023-10-06T05:27:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:27:59.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.488+0000 7f0fe6b9d700 1 Processor -- start 2023-10-06T05:27:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.488+0000 7f0fe6b9d700 1 -- start start 2023-10-06T05:27:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.489+0000 7f0fe6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe0159530 0x7f0fe006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.489+0000 7f0fe6b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe006df70 0x7f0fe00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.489+0000 7f0fe6b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe006e3e0 con 0x7f0fe00b0310 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.489+0000 7f0fe6b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00b1e00 con 0x7f0fe006df70 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.489+0000 7f0fe6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe006f6b0 con 0x7f0fe0159530 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe006df70 0x7f0fe00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe006df70 0x7f0fe00afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34694/0 (socket says 172.21.15.33:34694) 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 -- 172.21.15.33:0/2775200770 learned_addr learned my addr 172.21.15.33:0/2775200770 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe5b9b700 1 --2- 172.21.15.33:0/2775200770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe0159530 0x7f0fe006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 -- 172.21.15.33:0/2775200770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe0159530 msgr2=0x7f0fe006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 --2- 172.21.15.33:0/2775200770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe0159530 0x7f0fe006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 -- 172.21.15.33:0/2775200770 >> v1:172.21.15.33:6789/0 conn(0x7f0fe00b0310 legacy=0x7f0fe0159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 -- 172.21.15.33:0/2775200770 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe006f220 con 0x7f0fe006df70 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.490+0000 7f0fe539a700 1 --2- 172.21.15.33:0/2775200770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe00afdd0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00099b0 tx=0x7f0fd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf99a1dd0044505 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe4b99700 1 -- 172.21.15.33:0/2775200770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0009ea0 con 0x7f0fe006df70 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe4b99700 1 -- 172.21.15.33:0/2775200770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd000b860 con 0x7f0fe006df70 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe4b99700 1 -- 172.21.15.33:0/2775200770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000bc00 con 0x7f0fe006df70 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/2775200770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 msgr2=0x7f0fe00afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/2775200770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe00afdd0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00099b0 tx=0x7f0fd0009250 comp rx=0 tx=0).stop 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/2775200770 shutdown_connections 2023-10-06T05:27:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/2775200770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe0159530 0x7f0fe006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/2775200770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe00afdd0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/2775200770 >> 172.21.15.33:0/2775200770 conn(0x7f0fe006c560 msgr2=0x7f0fe006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/2775200770 shutdown_connections 2023-10-06T05:27:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.491+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/2775200770 wait complete. 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.493+0000 7f0fe6b9d700 1 Processor -- start 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.493+0000 7f0fe6b9d700 1 -- start start 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe6b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe0144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe00b0310 0x7f0fe00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe6b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe0159530 0x7f0fe015e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe6b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ca200 con 0x7f0fe006df70 2023-10-06T05:27:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe6b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ccc60 con 0x7f0fe0159530 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00c98d0 con 0x7f0fe00b0310 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe0144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe0144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34706/0 (socket says 172.21.15.33:34706) 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe5b9b700 1 -- 172.21.15.33:0/1990527773 learned_addr learned my addr 172.21.15.33:0/1990527773 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.494+0000 7f0fe539a700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe00b0310 0x7f0fe00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe639c700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe0159530 0x7f0fe015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe5b9b700 1 -- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe00b0310 msgr2=0x7f0fe00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe5b9b700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe00b0310 0x7f0fe00cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe5b9b700 1 -- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe0159530 msgr2=0x7f0fe015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe5b9b700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe0159530 0x7f0fe015e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe5b9b700 1 -- 172.21.15.33:0/1990527773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00094f0 con 0x7f0fe006df70 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe639c700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe0159530 0x7f0fe015e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe5b9b700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe0144cf0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00ec80 tx=0x7f0fdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00c850 con 0x7f0fe006df70 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00c9b0 con 0x7f0fe006df70 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00cd50 con 0x7f0fe006df70 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe00c9bb0 con 0x7f0fe006df70 2023-10-06T05:27:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.495+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe00c94b0 con 0x7f0fe006df70 2023-10-06T05:27:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.496+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe004c380 con 0x7f0fe006df70 2023-10-06T05:27:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.497+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fdc013680 con 0x7f0fe006df70 2023-10-06T05:27:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.498+0000 7f0fd5ffb700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc4066f30 0x7f0fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.498+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0fdc0849f0 con 0x7f0fe006df70 2023-10-06T05:27:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.499+0000 7f0fe539a700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc4066f30 0x7f0fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.499+0000 7f0fe539a700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc4066f30 0x7f0fc40693e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00099b0 tx=0x7f0fd0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.502+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fdc051a60 con 0x7f0fe006df70 2023-10-06T05:27:59.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.746+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe0155760 con 0x7f0fe006df70 2023-10-06T05:27:59.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.746+0000 7f0fd5ffb700 1 -- 172.21.15.33:0/1990527773 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f0fdc055080 con 0x7f0fe006df70 2023-10-06T05:27:59.748 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:27:59.748 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":148,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142},{"gid":9801,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:27:56.896156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9810},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9810":{"gid":9810,"name":"b","rank":0,"incarnation":144,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3335696661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3335696661},{"type":"v1","addr":"172.21.15.167:6837","nonce":3335696661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:27:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc4066f30 msgr2=0x7f0fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc4066f30 0x7f0fc40693e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00099b0 tx=0x7f0fd0017000 comp rx=0 tx=0).stop 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 msgr2=0x7f0fe0144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe0144cf0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00ec80 tx=0x7f0fdc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 shutdown_connections 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc4066f30 0x7f0fc40693e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe0159530 0x7f0fe015e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe006df70 0x7f0fe0144cf0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 --2- 172.21.15.33:0/1990527773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe00b0310 0x7f0fe00cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.748+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 >> 172.21.15.33:0/1990527773 conn(0x7f0fe006c560 msgr2=0x7f0fe0154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.749+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 shutdown_connections 2023-10-06T05:27:59.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:27:59.749+0000 7f0fe6b9d700 1 -- 172.21.15.33:0/1990527773 wait complete. 2023-10-06T05:27:59.752 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 148 2023-10-06T05:27:59.857 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:27:59.856+0000 7ff24a9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:28:00.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:27:53.374342+0000', 'modified': '2023-10-06T05:27:56.896156+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9810}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9810': {'gid': 9810, 'name': 'b', 'rank': 0, 'incarnation': 144, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/3335696661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 3335696661}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3335696661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:28:00.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:28:00.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:28:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.359+0000 7f13d6b9d700 1 Processor -- start 2023-10-06T05:28:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.360+0000 7f13d6b9d700 1 -- start start 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 0x7f13d00cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d6b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f13d006c560 0x7f13d00aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d6b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d00b03a0 con 0x7f13d01592e0 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d6b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d00b0f40 con 0x7f13d006c560 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d006ccd0 con 0x7f13d01596b0 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d5b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f13d01592e0 0x7f13d00b0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60100/0 (socket says 172.21.15.33:60100) 2023-10-06T05:28:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.361+0000 7f13d5b9b700 1 -- 172.21.15.33:0/894578370 learned_addr learned my addr 172.21.15.33:0/894578370 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d639c700 1 --2- 172.21.15.33:0/894578370 >> v2:172.21.15.167:3300/0 conn(0x7f13d006c560 0x7f13d00aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d539a700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 0x7f13d00cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028659688 0 0) 0x7f13d00b03a0 con 0x7f13d01592e0 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc003570 con 0x7f13d01592e0 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3868237739 0 0) 0x7f13bc003570 con 0x7f13d01592e0 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 msgr2=0x7f13d00cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 0x7f13d00cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 >> v2:172.21.15.167:3300/0 conn(0x7f13d006c560 msgr2=0x7f13d00aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 --2- 172.21.15.33:0/894578370 >> v2:172.21.15.167:3300/0 conn(0x7f13d006c560 0x7f13d00aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d006cfb0 con 0x7f13d01592e0 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f13cc001470 con 0x7f13d01592e0 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 >> v1:172.21.15.33:6789/0 conn(0x7f13d01592e0 legacy=0x7f13d00b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc003a60 0x7f13bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 0x7f13bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004d50 0x7f13bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d00b0f40 con 0x7f13bc004d50 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d006ccd0 con 0x7f13bc003a60 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc00ba60 con 0x7f13d01596b0 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d639c700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004d50 0x7f13bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.362+0000 7f13d539a700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 0x7f13bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc003a60 0x7f13bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d639c700 1 -- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 msgr2=0x7f13bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d639c700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13d01596b0 0x7f13bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d639c700 1 -- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc003a60 msgr2=0x7f13bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d639c700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13bc003a60 0x7f13bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d639c700 1 -- 172.21.15.33:0/894578370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc003570 con 0x7f13bc004d50 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d639c700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004d50 0x7f13bc0051c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e210 tx=0x7f13c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52dafa8c01b08e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c800ea60 con 0x7f13bc004d50 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c800f040 con 0x7f13bc004d50 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.363+0000 7f13d4b99700 1 -- 172.21.15.33:0/894578370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c800ee10 con 0x7f13bc004d50 2023-10-06T05:28:00.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d6b9d700 1 -- 172.21.15.33:0/894578370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004d50 msgr2=0x7f13bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/894578370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc004d50 0x7f13bc0051c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e210 tx=0x7f13c800b040 comp rx=0 tx=0).stop 2023-10-06T05:28:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d639c700 1 -- 172.21.15.33:0/894578370 reap_dead start 2023-10-06T05:28:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d6b9d700 1 -- 172.21.15.33:0/894578370 shutdown_connections 2023-10-06T05:28:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d6b9d700 1 -- 172.21.15.33:0/894578370 >> 172.21.15.33:0/894578370 conn(0x7f13d00675f0 msgr2=0x7f13d00679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d6b9d700 1 -- 172.21.15.33:0/894578370 shutdown_connections 2023-10-06T05:28:00.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.364+0000 7f13d6b9d700 1 -- 172.21.15.33:0/894578370 wait complete. 2023-10-06T05:28:00.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.366+0000 7f13d6b9d700 1 Processor -- start 2023-10-06T05:28:00.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.366+0000 7f13d6b9d700 1 -- start start 2023-10-06T05:28:00.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc004d50 0x7f13d014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d6b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 0x7f13d0154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d6b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13d015e330 0x7f13d0151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d6b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d015dea0 con 0x7f13d015cd10 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d6b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d015d290 con 0x7f13d015e330 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d015d570 con 0x7f13bc004d50 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 0x7f13d0154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 0x7f13d0154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50074/0 (socket says 172.21.15.33:50074) 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 -- 172.21.15.33:0/4254910294 learned_addr learned my addr 172.21.15.33:0/4254910294 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc004d50 0x7f13d014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d639c700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13d015e330 0x7f13d0151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 -- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc004d50 msgr2=0x7f13d014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc004d50 0x7f13d014db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 -- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13d015e330 msgr2=0x7f13d0151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13d015e330 0x7f13d0151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.367+0000 7f13d539a700 1 -- 172.21.15.33:0/4254910294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d015d850 con 0x7f13d015cd10 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.368+0000 7f13d539a700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 0x7f13d0154bf0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006ac0 tx=0x7f13c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.368+0000 7f13c5ffb700 1 -- 172.21.15.33:0/4254910294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c000f940 con 0x7f13d015cd10 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.368+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d015db30 con 0x7f13d015cd10 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.368+0000 7f13c5ffb700 1 -- 172.21.15.33:0/4254910294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c000faa0 con 0x7f13d015cd10 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.368+0000 7f13c5ffb700 1 -- 172.21.15.33:0/4254910294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c000fe20 con 0x7f13d015cd10 2023-10-06T05:28:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.368+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d0155130 con 0x7f13d015cd10 2023-10-06T05:28:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.371+0000 7f13c5ffb700 1 -- 172.21.15.33:0/4254910294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13c0014650 con 0x7f13d015cd10 2023-10-06T05:28:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.372+0000 7f13c5ffb700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13b4066f30 0x7f13b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.372+0000 7f13c5ffb700 1 -- 172.21.15.33:0/4254910294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f13c0018030 con 0x7f13d015cd10 2023-10-06T05:28:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.372+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13b4066f30 0x7f13b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.373+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13b4066f30 0x7f13b40693e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f13d00cdf70 tx=0x7f13d00cda90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.373+0000 7f13e65c1b80 1 Processor -- start 2023-10-06T05:28:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 -- start start 2023-10-06T05:28:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc21bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc244920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc217990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc19bc80 con 0x5586bc265bc0 2023-10-06T05:28:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc172a70 con 0x5586bc244e60 2023-10-06T05:28:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.374+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586bc170270 con 0x5586bc21b950 2023-10-06T05:28:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc244920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc244920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50088/0 (socket says 172.21.15.33:50088) 2023-10-06T05:28:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 -- 172.21.15.33:0/3259211117 learned_addr learned my addr 172.21.15.33:0/3259211117 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d639c700 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc217990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc21bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 -- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 msgr2=0x5586bc21bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc21bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 -- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 msgr2=0x5586bc217990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc217990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 -- 172.21.15.33:0/3259211117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586bc006270 con 0x5586bc265bc0 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d639c700 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc217990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13d539a700 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc244920 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009860 tx=0x7f13c0083450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda7cceb30608e4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.375+0000 7f13baffd700 1 -- 172.21.15.33:0/3259211117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00591b0 con 0x5586bc265bc0 2023-10-06T05:28:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13baffd700 1 -- 172.21.15.33:0/3259211117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c0059e90 con 0x5586bc265bc0 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13baffd700 1 -- 172.21.15.33:0/3259211117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c005cea0 con 0x5586bc265bc0 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3259211117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 msgr2=0x5586bc244920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc244920 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009860 tx=0x7f13c0083450 comp rx=0 tx=0).stop 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3259211117 shutdown_connections 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc217990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc21bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3259211117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc244920 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3259211117 >> 172.21.15.33:0/3259211117 conn(0x5586bc21d270 msgr2=0x5586bc21d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3259211117 shutdown_connections 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.376+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3259211117 wait complete. 2023-10-06T05:28:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.377+0000 7f13e65c1b80 1 -- start start 2023-10-06T05:28:00.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.377+0000 7f13e65c1b80 1 Processor -- start 2023-10-06T05:28:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc275e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc2126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc16c060 con 0x5586bc21b950 2023-10-06T05:28:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc117890 con 0x5586bc244e60 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586bc132ea0 con 0x5586bc265bc0 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13e65c1b80 10 client.0 _ll_register_callbacks cb 0x5586bc008f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.378+0000 7f13d5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc275e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc275e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50096/0 (socket says 172.21.15.33:50096) 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 -- 172.21.15.33:0/1923293175 learned_addr learned my addr 172.21.15.33:0/1923293175 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d639c700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc2126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d539a700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 -- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 msgr2=0x5586bc2126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc2126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 -- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 msgr2=0x5586bc2121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 -- 172.21.15.33:0/1923293175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d0155130 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc275e70 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f13cc015aa0 tx=0x7f13cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b49d43c6af9ba0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13affff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13affff700 1 -- 172.21.15.33:0/1923293175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01a040 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13affff700 1 -- 172.21.15.33:0/1923293175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f13cc024c60 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.379+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586bc210690 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13affff700 1 -- 172.21.15.33:0/1923293175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13cc014c50 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 10 client.9948 fetch_fsmap learned FSMap version 148 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=2+}) v3 -- 0x5586bbfa8fe0 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13affff700 1 -- 172.21.15.33:0/1923293175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586bc210690 con 0x5586bc21b950 2023-10-06T05:28:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13affff700 1 -- 172.21.15.33:0/1923293175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 148) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f13cc020050 con 0x5586bc21b950 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 10 client.9948 fetch_fsmap finished waiting for FSMap version 148 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 10 client.9948 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 20 client.9948 populate_metadata read hostname 'smithi033' 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2413b0 0x5586bc25d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 4 client.9948 mds_command: new command op to 9810 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.380+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 --> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586bc1ed5c0 con 0x5586bc2413b0 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.381+0000 7f13d539a700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2413b0 0x5586bc25d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.381+0000 7f13d539a700 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2413b0 0x5586bc25d890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7020b37728c411 server_cookie=56d666a55081bc03 in_seq=0 out_seq=0 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.381+0000 7f13affff700 10 client.9948 ms_handle_connect on v2:172.21.15.167:6835/3335696661 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13affff700 1 -- 172.21.15.33:0/1923293175 <== mds.0 v2:172.21.15.167:6835/3335696661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5586bc1ed5c0 con 0x5586bc2413b0 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13affff700 10 client.9948 handle_command_reply: tid=0 2023-10-06T05:28:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 client.9948 shutdown 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 msgr2=0x5586bc275e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc275e70 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f13cc015aa0 tx=0x7f13cc017000 comp rx=0 tx=0).stop 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 shutdown_connections 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc2126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc275e70 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2413b0 0x5586bc25d890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1923293175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13affff700 0 client.9948 ms_handle_reset on v2:172.21.15.167:6835/3335696661 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.382+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 >> 172.21.15.33:0/1923293175 conn(0x5586bc21d270 msgr2=0x5586bc2180a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.383+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 shutdown_connections 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.383+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1923293175 wait complete. 2023-10-06T05:28:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.383+0000 7f13e65c1b80 20 client.9948 trim_cache size 0 max 16384 2023-10-06T05:28:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 Processor -- start 2023-10-06T05:28:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 -- start start 2023-10-06T05:28:00.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc2445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc275d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc21bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bbfeee30 con 0x5586bc21b950 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc25a060 con 0x5586bc244e60 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.401+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586bc219b10 con 0x5586bc265bc0 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc2445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc2445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50110/0 (socket says 172.21.15.33:50110) 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 -- 172.21.15.33:0/3420745847 learned_addr learned my addr 172.21.15.33:0/3420745847 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d539a700 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc275d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d639c700 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc21bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 -- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 msgr2=0x5586bc21bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc21bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 -- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 msgr2=0x5586bc275d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc275d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 -- 172.21.15.33:0/3420745847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586bc13a780 con 0x5586bc21b950 2023-10-06T05:28:00.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.402+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc2445d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f13cc014680 tx=0x7f13cc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a019b7177022630f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13bb7fe700 1 -- 172.21.15.33:0/3420745847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01a040 con 0x5586bc21b950 2023-10-06T05:28:00.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13bb7fe700 1 -- 172.21.15.33:0/3420745847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc014c50 con 0x5586bc21b950 2023-10-06T05:28:00.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13bb7fe700 1 -- 172.21.15.33:0/3420745847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc02f430 con 0x5586bc21b950 2023-10-06T05:28:00.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3420745847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 msgr2=0x5586bc2445d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc2445d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f13cc014680 tx=0x7f13cc01b000 comp rx=0 tx=0).stop 2023-10-06T05:28:00.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3420745847 shutdown_connections 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc265bc0 0x5586bc21bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc21b950 0x5586bc2445d0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/3420745847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc275d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3420745847 >> 172.21.15.33:0/3420745847 conn(0x5586bc21d270 msgr2=0x5586bc21d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3420745847 shutdown_connections 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.403+0000 7f13e65c1b80 1 -- 172.21.15.33:0/3420745847 wait complete. 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.404+0000 7f13e65c1b80 1 -- start start 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.404+0000 7f13e65c1b80 1 Processor -- start 2023-10-06T05:28:00.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc23cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc21c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc2c8140 con 0x5586bc265bc0 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586bc2c8560 con 0x5586bc244e60 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586bc23d220 con 0x5586bc21b950 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.405+0000 7f13e65c1b80 10 client.0 _ll_register_callbacks cb 0x5586bc008f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc21c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc21c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50118/0 (socket says 172.21.15.33:50118) 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 -- 172.21.15.33:0/1294805666 learned_addr learned my addr 172.21.15.33:0/1294805666 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d539a700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc23cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 -- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 msgr2=0x5586bc23cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc23cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 -- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 msgr2=0x5586bc2c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 -- 172.21.15.33:0/1294805666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586bbfa8fe0 con 0x5586bc265bc0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d539a700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13d639c700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc21c350 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f13c8016000 tx=0x7f13c801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a504436261b956c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13affff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.406+0000 7f13affff700 1 -- 172.21.15.33:0/1294805666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8024070 con 0x5586bc265bc0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586bc219180 con 0x5586bc265bc0 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13affff700 1 -- 172.21.15.33:0/1294805666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f13c802bc30 con 0x5586bc265bc0 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13affff700 1 -- 172.21.15.33:0/1294805666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c80148e0 con 0x5586bc265bc0 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 10 client.9954 fetch_fsmap learned FSMap version 148 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=148,monmap=2+}) v3 -- 0x5586bc121470 con 0x5586bc265bc0 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13affff700 1 -- 172.21.15.33:0/1294805666 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586bc219180 con 0x5586bc265bc0 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13affff700 1 -- 172.21.15.33:0/1294805666 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 148) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f13c8031020 con 0x5586bc265bc0 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 10 client.9954 fetch_fsmap finished waiting for FSMap version 148 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 10 client.9954 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 20 client.9954 populate_metadata read hostname 'smithi033' 2023-10-06T05:28:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2ad3f0 0x5586bc2ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 4 client.9954 mds_command: new command op to 9810 tid=0[{"prefix": "status"}] 2023-10-06T05:28:00.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.407+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 --> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f13c0059e90 con 0x5586bc2ad3f0 2023-10-06T05:28:00.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.408+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2ad3f0 0x5586bc2ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.408+0000 7f13d5b9b700 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2ad3f0 0x5586bc2ad7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba18602844435149 server_cookie=dc0f76bd063e7bd9 in_seq=0 out_seq=0 2023-10-06T05:28:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.408+0000 7f13affff700 10 client.9954 ms_handle_connect on v2:172.21.15.167:6835/3335696661 2023-10-06T05:28:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.408+0000 7f13affff700 1 -- 172.21.15.33:0/1294805666 <== mds.0 v2:172.21.15.167:6835/3335696661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x7f13c0059e90 con 0x5586bc2ad3f0 2023-10-06T05:28:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.408+0000 7f13affff700 10 client.9954 handle_command_reply: tid=0 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 client.9954 shutdown 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 msgr2=0x5586bc21c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc21c350 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f13c8016000 tx=0x7f13c801a000 comp rx=0 tx=0).stop 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 shutdown_connections 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5586bc265bc0 0x5586bc21c350 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5586bc21b950 0x5586bc23cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:6835/3335696661,v1:172.21.15.167:6837/3335696661] conn(0x5586bc2ad3f0 0x5586bc2ad7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 --2- 172.21.15.33:0/1294805666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586bc244e60 0x5586bc2c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 >> 172.21.15.33:0/1294805666 conn(0x5586bc21d270 msgr2=0x5586bc25d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13affff700 0 client.9954 ms_handle_reset on v2:172.21.15.167:6835/3335696661 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 shutdown_connections 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 1 -- 172.21.15.33:0/1294805666 wait complete. 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.409+0000 7f13e65c1b80 20 client.9954 trim_cache size 0 max 16384 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 9810, 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:28:00.417 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5341908929999999, 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 148, 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 194, 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 194, 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.268814026999999 2023-10-06T05:28:00.418 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:28:00.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13b4066f30 msgr2=0x7f13b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13b4066f30 0x7f13b40693e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f13d00cdf70 tx=0x7f13d00cda90 comp rx=0 tx=0).stop 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 msgr2=0x7f13d0154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 0x7f13d0154bf0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006ac0 tx=0x7f13c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 shutdown_connections 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f13b4066f30 0x7f13b40693e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13d015e330 0x7f13d0151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13bc004d50 0x7f13d014db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 --2- 172.21.15.33:0/4254910294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13d015cd10 0x7f13d0154bf0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 >> 172.21.15.33:0/4254910294 conn(0x7f13d00675f0 msgr2=0x7f13d00ac140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 shutdown_connections 2023-10-06T05:28:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.410+0000 7f13d6b9d700 1 -- 172.21.15.33:0/4254910294 wait complete. 2023-10-06T05:28:00.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:28:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.614+0000 7efd7dd9d700 1 Processor -- start 2023-10-06T05:28:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.615+0000 7efd7dd9d700 1 -- start start 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd7813faa0 0x7efd7813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd780c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7dd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd780c9cb0 con 0x7efd7806aad0 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7806b660 con 0x7efd7813faa0 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd780ca270 con 0x7efd780c7940 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7cd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd7813faa0 0x7efd7813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7cd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd7813faa0 0x7efd7813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50130/0 (socket says 172.21.15.33:50130) 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.616+0000 7efd7cd9b700 1 -- 172.21.15.33:0/258734120 learned_addr learned my addr 172.21.15.33:0/258734120 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd77fff700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd780c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd7cd9b700 1 -- 172.21.15.33:0/258734120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 msgr2=0x7efd780c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd7cd9b700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd780c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd7cd9b700 1 -- 172.21.15.33:0/258734120 >> v1:172.21.15.33:6789/0 conn(0x7efd7806aad0 legacy=0x7efd7806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd7cd9b700 1 -- 172.21.15.33:0/258734120 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7806b840 con 0x7efd7813faa0 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd77fff700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd780c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd7cd9b700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7813faa0 0x7efd7813fe70 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00bc50 tx=0x7efd6c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f017122b358750 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd777fe700 1 -- 172.21.15.33:0/258734120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00be60 con 0x7efd7813faa0 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd777fe700 1 -- 172.21.15.33:0/258734120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c00de00 con 0x7efd7813faa0 2023-10-06T05:28:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.617+0000 7efd777fe700 1 -- 172.21.15.33:0/258734120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c0196b0 con 0x7efd7813faa0 2023-10-06T05:28:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 -- 172.21.15.33:0/258734120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7813faa0 msgr2=0x7efd7813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7813faa0 0x7efd7813fe70 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00bc50 tx=0x7efd6c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 -- 172.21.15.33:0/258734120 shutdown_connections 2023-10-06T05:28:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7813faa0 0x7efd7813fe70 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/258734120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd780c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 -- 172.21.15.33:0/258734120 >> 172.21.15.33:0/258734120 conn(0x7efd7806d030 msgr2=0x7efd7806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 -- 172.21.15.33:0/258734120 shutdown_connections 2023-10-06T05:28:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.618+0000 7efd7dd9d700 1 -- 172.21.15.33:0/258734120 wait complete. 2023-10-06T05:28:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.619+0000 7efd7dd9d700 1 Processor -- start 2023-10-06T05:28:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.620+0000 7efd7dd9d700 1 -- start start 2023-10-06T05:28:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.620+0000 7efd7dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 0x7efd7815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.620+0000 7efd7dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd7813ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.620+0000 7efd7dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd7813faa0 0x7efd78158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780b1a60 con 0x7efd7806aad0 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7815f480 con 0x7efd7813faa0 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7813b2b0 con 0x7efd780c7940 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 0x7efd7815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 0x7efd7815dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50138/0 (socket says 172.21.15.33:50138) 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 -- 172.21.15.33:0/1131667791 learned_addr learned my addr 172.21.15.33:0/1131667791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd77fff700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd7813ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7d59c700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd7813faa0 0x7efd78158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 -- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 msgr2=0x7efd7813ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd7813ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 -- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd7813faa0 msgr2=0x7efd78158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd7813faa0 0x7efd78158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 -- 172.21.15.33:0/1131667791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c00b790 con 0x7efd7806aad0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.621+0000 7efd7cd9b700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 0x7efd7815dcd0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7efd780719c0 tx=0x7efd6c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.622+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c010850 con 0x7efd7806aad0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.622+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c0109b0 con 0x7efd7806aad0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.622+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c010d70 con 0x7efd7806aad0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.622+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7815eb50 con 0x7efd7806aad0 2023-10-06T05:28:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.622+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7815f110 con 0x7efd7806aad0 2023-10-06T05:28:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.623+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7804c380 con 0x7efd7806aad0 2023-10-06T05:28:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.624+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd6c023600 con 0x7efd7806aad0 2023-10-06T05:28:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.625+0000 7efd74ff9700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd5c066f30 0x7efd5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.625+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efd6c014070 con 0x7efd7806aad0 2023-10-06T05:28:00.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.626+0000 7efd77fff700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd5c066f30 0x7efd5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.626+0000 7efd77fff700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd5c066f30 0x7efd5c0693e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efd7813f2a0 tx=0x7efd64006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.630+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd6c0539f0 con 0x7efd7806aad0 2023-10-06T05:28:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.878+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd7806bb20 con 0x7efd7806aad0 2023-10-06T05:28:00.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.879+0000 7efd74ff9700 1 -- 172.21.15.33:0/1131667791 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v148) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd6c057010 con 0x7efd7806aad0 2023-10-06T05:28:00.881 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd5c066f30 msgr2=0x7efd5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd5c066f30 0x7efd5c0693e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efd7813f2a0 tx=0x7efd64006f90 comp rx=0 tx=0).stop 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 msgr2=0x7efd7815dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 0x7efd7815dcd0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7efd780719c0 tx=0x7efd6c01b000 comp rx=0 tx=0).stop 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 shutdown_connections 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd5c066f30 0x7efd5c0693e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd7813faa0 0x7efd78158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7806aad0 0x7efd7815dcd0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 --2- 172.21.15.33:0/1131667791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd780c7940 0x7efd7813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 >> 172.21.15.33:0/1131667791 conn(0x7efd7806d030 msgr2=0x7efd780c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 shutdown_connections 2023-10-06T05:28:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:00.881+0000 7efd7dd9d700 1 -- 172.21.15.33:0/1131667791 wait complete. 2023-10-06T05:28:00.901 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:28:00.902 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:28:00.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:00.902 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.940 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:28:00.941 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41476sec preferred_lft 41476sec 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:28:00.942 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:28:00.943 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.944 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:00.944 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:28:00.944 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:28:00.944 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:00.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:00.944 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:28:01.040 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:28:01.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.044 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:28:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:28:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:28:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:28:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:28:01.083 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:28:01.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.084 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:28:01.084 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:28:01.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:28:01.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:28:01.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:28:01.084 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:28:01.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.282 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:28:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:28:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:28:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:28:01.320 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:28:01.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:28:01.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:28:01.423 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:01.460 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:28:01.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:28:01.460 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:01.536 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:28:01.536 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:28:01.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:01.668 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:01.669 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:28:01.669 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:28:01.685 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:01.685 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:28:01.685 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:28:01.685 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:28:01.685 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:28:01.708 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:28:01.712 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:28:01.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:01.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:28:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.966+0000 7fb18199d700 1 Processor -- start 2023-10-06T05:28:02.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.966+0000 7fb18199d700 1 -- start start 2023-10-06T05:28:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.967+0000 7fb18199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.967+0000 7fb18199d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb17c0c7920 0x7fb17c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.967+0000 7fb18199d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c06ab60 con 0x7fb17c06a6a0 2023-10-06T05:28:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.967+0000 7fb18199d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c06b7e0 con 0x7fb17c0c7920 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.967+0000 7fb18199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c06ae40 con 0x7fb17c155af0 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb17c0c7920 0x7fb17c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18099b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb17c0c7920 0x7fb17c0c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:50170/0 (socket says 172.21.15.33:50170) 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18099b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c06a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41350/0 (socket says 172.21.15.33:41350) 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 -- 172.21.15.33:0/1279633547 learned_addr learned my addr 172.21.15.33:0/1279633547 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb17bfff700 1 --1- 172.21.15.33:0/1279633547 >> v1:172.21.15.167:6789/0 conn(0x7fb17c06a6a0 0x7fb17c0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:59824/0 (socket says 172.21.15.33:59824) 2023-10-06T05:28:02.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb17b7fe700 1 -- 172.21.15.33:0/1279633547 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657632707 0 0) 0x7fb17c06ab60 con 0x7fb17c06a6a0 2023-10-06T05:28:02.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb17b7fe700 1 -- 172.21.15.33:0/1279633547 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb16c003570 con 0x7fb17c06a6a0 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 -- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 msgr2=0x7fb17c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 --2- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 -- 172.21.15.33:0/1279633547 >> v1:172.21.15.167:6789/0 conn(0x7fb17c06a6a0 legacy=0x7fb17c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18119c700 1 -- 172.21.15.33:0/1279633547 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c06b120 con 0x7fb17c0c7920 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.968+0000 7fb18099b700 1 --2- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.969+0000 7fb18119c700 1 --2- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0c7920 0x7fb17c0c9e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb174009f20 tx=0x7fb174009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59aca23a42ec36fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.969+0000 7fb17b7fe700 1 -- 172.21.15.33:0/1279633547 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400e070 con 0x7fb17c0c7920 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.969+0000 7fb17b7fe700 1 -- 172.21.15.33:0/1279633547 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17400b970 con 0x7fb17c0c7920 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.969+0000 7fb17b7fe700 1 -- 172.21.15.33:0/1279633547 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1740092e0 con 0x7fb17c0c7920 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.969+0000 7fb18199d700 1 -- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0c7920 msgr2=0x7fb17c0c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.969+0000 7fb18199d700 1 --2- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0c7920 0x7fb17c0c9e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb174009f20 tx=0x7fb174009840 comp rx=0 tx=0).stop 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.970+0000 7fb18199d700 1 -- 172.21.15.33:0/1279633547 shutdown_connections 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.970+0000 7fb18199d700 1 --2- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0c7920 0x7fb17c0c9e20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.970+0000 7fb18199d700 1 --2- 172.21.15.33:0/1279633547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.970+0000 7fb18199d700 1 -- 172.21.15.33:0/1279633547 >> 172.21.15.33:0/1279633547 conn(0x7fb17c06d030 msgr2=0x7fb17c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.970+0000 7fb18199d700 1 -- 172.21.15.33:0/1279633547 shutdown_connections 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.970+0000 7fb18199d700 1 -- 172.21.15.33:0/1279633547 wait complete. 2023-10-06T05:28:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.971+0000 7fb18199d700 1 Processor -- start 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.971+0000 7fb18199d700 1 -- start start 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb18199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c06a6a0 0x7fb17c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb18199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 0x7fb17c16c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb18199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c0adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb18199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c16d910 con 0x7fb17c0c7920 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb18199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0ac000 con 0x7fb17c06a6a0 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb18199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c16d350 con 0x7fb17c155af0 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb17bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 0x7fb17c16c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb17bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 0x7fb17c16c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50146/0 (socket says 172.21.15.33:50146) 2023-10-06T05:28:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.972+0000 7fb17bfff700 1 -- 172.21.15.33:0/2803952719 learned_addr learned my addr 172.21.15.33:0/2803952719 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb18099b700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c06a6a0 0x7fb17c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb18119c700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c0adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb17bfff700 1 -- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 msgr2=0x7fb17c0adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb17bfff700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c0adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb17bfff700 1 -- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c06a6a0 msgr2=0x7fb17c152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb17bfff700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c06a6a0 0x7fb17c152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb17bfff700 1 -- 172.21.15.33:0/2803952719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170009420 con 0x7fb17c0c7920 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb18099b700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c06a6a0 0x7fb17c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb18119c700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c0adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb17bfff700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 0x7fb17c16c970 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c560 tx=0x7fb168007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168005b40 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb168005cc0 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168007320 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.973+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb174009ae0 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.974+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c0abc90 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.975+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c04c380 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.975+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb16801b3b0 con 0x7fb17c0c7920 2023-10-06T05:28:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.976+0000 7fb178ff9700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb160066f30 0x7fb1600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.976+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb168083c80 con 0x7fb17c0c7920 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.977+0000 7fb18099b700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb160066f30 0x7fb1600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.978+0000 7fb18099b700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb160066f30 0x7fb1600693e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb1700093f0 tx=0x7fb170008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:01.980+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb168050cf0 con 0x7fb17c0c7920 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.228+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb17c06b400 con 0x7fb17c0c7920 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.228+0000 7fb178ff9700 1 -- 172.21.15.33:0/2803952719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v148) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb168017070 con 0x7fb17c0c7920 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb160066f30 msgr2=0x7fb1600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb160066f30 0x7fb1600693e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb1700093f0 tx=0x7fb170008040 comp rx=0 tx=0).stop 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 msgr2=0x7fb17c16c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 0x7fb17c16c970 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c560 tx=0x7fb168007dd0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 shutdown_connections 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb160066f30 0x7fb1600693e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c155af0 0x7fb17c0adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c06a6a0 0x7fb17c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 --2- 172.21.15.33:0/2803952719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c0c7920 0x7fb17c16c970 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 >> 172.21.15.33:0/2803952719 conn(0x7fb17c06d030 msgr2=0x7fb17c06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 shutdown_connections 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:02.231+0000 7fb18199d700 1 -- 172.21.15.33:0/2803952719 wait complete. 2023-10-06T05:28:02.267 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:28:02.267 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:28:02.267 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:28:02.267 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:28:02.267 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:28:02.267 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:28:02.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:28:02.268 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:28:02.268 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:02.268 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:02.309 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:28:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41479sec preferred_lft 41479sec 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.311 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:28:02.312 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:02.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:02.313 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:28:02.416 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:28:02.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:02.552 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:28:02.593 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:28:02.593 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:28:02.593 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:28:02.593 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:28:02.594 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:28:02.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:02.595 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:28:02.595 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:28:02.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:28:02.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:28:02.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:28:02.595 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:28:02.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:02.791 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:28:02.827 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:28:02.827 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:28:02.828 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:28:02.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:02.828 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:28:02.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:28:02.931 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:28:02.967 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:28:02.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:28:02.968 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:28:03.043 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:28:03.043 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:28:03.043 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:28:03.176 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:28:03.176 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:28:03.177 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:28:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:28:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:28:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:28:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:28:03.193 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:28:03.213 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:28:03.213 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:28:03.213 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:28:03.214 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:28:03.214 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:28:03.214 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:28:03.214 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:28:03.214 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:28:03.217 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:28:03.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:03.217 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:28:03.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:28:03.495 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:28:03.496 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:28:03.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:28:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:28:03.702 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:28:03.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:28:03.702 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:28:03.775 INFO:teuthology.orchestra.run.smithi033.stdout:client7772 2023-10-06T05:28:03.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:28:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.962+0000 7f995b7ff700 1 Processor -- start 2023-10-06T05:28:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.963+0000 7f995b7ff700 1 -- start start 2023-10-06T05:28:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.964+0000 7f995b7ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9954156390 0x7f99540ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.964+0000 7f995b7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156760 0x7f99540ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.964+0000 7f995b7ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540c7220 con 0x7f99540ca8c0 2023-10-06T05:28:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.964+0000 7f995b7ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99540cb5b0 con 0x7f9954156390 2023-10-06T05:28:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.964+0000 7f995b7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540cb890 con 0x7f9954156760 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f9958d9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156760 0x7f99540ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9954156390 0x7f99540ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9954156390 0x7f99540ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:50190/0 (socket says 172.21.15.33:50190) 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f9958d9a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156760 0x7f99540ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41366/0 (socket says 172.21.15.33:41366) 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 -- 172.21.15.33:0/3103868796 learned_addr learned my addr 172.21.15.33:0/3103868796 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 -- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156760 msgr2=0x7f99540ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 --2- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156760 0x7f99540ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 -- 172.21.15.33:0/3103868796 >> v1:172.21.15.167:6789/0 conn(0x7f99540ca8c0 legacy=0x7f99540cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 -- 172.21.15.33:0/3103868796 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540c6bd0 con 0x7f9954156390 2023-10-06T05:28:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.965+0000 7f995959b700 1 --2- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156390 0x7f99540ae140 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f99500099b0 tx=0x7f9950009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be18ecc934625eee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f994bfff700 1 -- 172.21.15.33:0/3103868796 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950009ea0 con 0x7f9954156390 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f994bfff700 1 -- 172.21.15.33:0/3103868796 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000b860 con 0x7f9954156390 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f994bfff700 1 -- 172.21.15.33:0/3103868796 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000bc00 con 0x7f9954156390 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f995b7ff700 1 -- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156390 msgr2=0x7f99540ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f995b7ff700 1 --2- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156390 0x7f99540ae140 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f99500099b0 tx=0x7f9950009250 comp rx=0 tx=0).stop 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f995b7ff700 1 -- 172.21.15.33:0/3103868796 shutdown_connections 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f995b7ff700 1 --2- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156390 0x7f99540ae140 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f995b7ff700 1 --2- 172.21.15.33:0/3103868796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156760 0x7f99540ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.966+0000 7f995b7ff700 1 -- 172.21.15.33:0/3103868796 >> 172.21.15.33:0/3103868796 conn(0x7f995406d030 msgr2=0x7f99540cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.967+0000 7f995b7ff700 1 -- 172.21.15.33:0/3103868796 shutdown_connections 2023-10-06T05:28:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.967+0000 7f995b7ff700 1 -- 172.21.15.33:0/3103868796 wait complete. 2023-10-06T05:28:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.968+0000 7f995b7ff700 1 Processor -- start 2023-10-06T05:28:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.968+0000 7f995b7ff700 1 -- start start 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995b7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 0x7f9954153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995b7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156390 0x7f9954142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995b7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156760 0x7f9954142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995b7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954144060 con 0x7f99540ca8c0 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995b7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540c4490 con 0x7f9954156760 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995b7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540c41b0 con 0x7f9954156390 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 0x7f9954153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 0x7f9954153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50154/0 (socket says 172.21.15.33:50154) 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f995959b700 1 -- 172.21.15.33:0/732359184 learned_addr learned my addr 172.21.15.33:0/732359184 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.969+0000 7f9959d9c700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156760 0x7f9954142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f9958d9a700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156390 0x7f9954142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995959b700 1 -- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156390 msgr2=0x7f9954142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995959b700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156390 0x7f9954142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995959b700 1 -- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156760 msgr2=0x7f9954142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995959b700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156760 0x7f9954142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995959b700 1 -- 172.21.15.33:0/732359184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500094f0 con 0x7f99540ca8c0 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995959b700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 0x7f9954153800 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f99540648b0 tx=0x7f9950017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000bbe0 con 0x7f99540ca8c0 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950011420 con 0x7f99540ca8c0 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99500117c0 con 0x7f99540ca8c0 2023-10-06T05:28:03.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.970+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9954143a10 con 0x7f99540ca8c0 2023-10-06T05:28:03.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.971+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99540c3550 con 0x7f99540ca8c0 2023-10-06T05:28:03.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.972+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9954143870 con 0x7f99540ca8c0 2023-10-06T05:28:03.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.972+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9950011bf0 con 0x7f99540ca8c0 2023-10-06T05:28:03.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.974+0000 7f99497fa700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9938066f30 0x7f99380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.974+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f99500860e0 con 0x7f99540ca8c0 2023-10-06T05:28:03.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.974+0000 7f9958d9a700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9938066f30 0x7f99380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.975+0000 7f9958d9a700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9938066f30 0x7f99380693e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f99540c3b10 tx=0x7f9944008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:03.979+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9954143870 con 0x7f99540ca8c0 2023-10-06T05:28:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.231+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9954143870 con 0x7f99540ca8c0 2023-10-06T05:28:04.233 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:28:04.232+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:28:04.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.236+0000 7f99497fa700 1 -- 172.21.15.33:0/732359184 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v149) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9954143870 con 0x7f99540ca8c0 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9938066f30 msgr2=0x7f99380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9938066f30 0x7f99380693e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f99540c3b10 tx=0x7f9944008040 comp rx=0 tx=0).stop 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 msgr2=0x7f9954153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 0x7f9954153800 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f99540648b0 tx=0x7f9950017000 comp rx=0 tx=0).stop 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 shutdown_connections 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9938066f30 0x7f99380693e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9954156760 0x7f9954142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99540ca8c0 0x7f9954153800 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 --2- 172.21.15.33:0/732359184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954156390 0x7f9954142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 >> 172.21.15.33:0/732359184 conn(0x7f995406d030 msgr2=0x7f9954152b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 shutdown_connections 2023-10-06T05:28:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.239+0000 7f995b7ff700 1 -- 172.21.15.33:0/732359184 wait complete. 2023-10-06T05:28:04.242 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:28:04.243 INFO:tasks.ceph.mds.b.smithi167.stderr: -538> 2023-10-06T05:27:59.856+0000 7ff24a9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:28:04.259 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:28:04.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:28:04.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:28:04.280 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:28:04.334 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:28:04.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:04.335 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:28:04.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:28:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.602+0000 7f3ca139d700 1 Processor -- start 2023-10-06T05:28:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.602+0000 7f3ca139d700 1 -- start start 2023-10-06T05:28:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.603+0000 7f3ca139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 0x7f3c9c0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.603+0000 7f3ca139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c1586a0 con 0x7f3c9c1552f0 2023-10-06T05:28:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.603+0000 7f3ca139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c157ab0 con 0x7f3c9c154f20 2023-10-06T05:28:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.603+0000 7f3ca139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c06bdf0 con 0x7f3c9c0b18f0 2023-10-06T05:28:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.603+0000 7f3c9b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c9c1552f0 0x7f3c9c0c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60122/0 (socket says 172.21.15.33:60122) 2023-10-06T05:28:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.603+0000 7f3c9b7fe700 1 -- 172.21.15.33:0/2538869890 learned_addr learned my addr 172.21.15.33:0/2538869890 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444677072 0 0) 0x7f3c9c1586a0 con 0x7f3c9c1552f0 2023-10-06T05:28:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c84003570 con 0x7f3c9c1552f0 2023-10-06T05:28:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4024154360 0 0) 0x7f3c84003570 con 0x7f3c9c1552f0 2023-10-06T05:28:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 msgr2=0x7f3c9c0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 0x7f3c9c0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 >> v1:172.21.15.167:6789/0 conn(0x7f3c9c154f20 legacy=0x7f3c9c0b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c157d70 con 0x7f3c9c1552f0 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3c90001470 con 0x7f3c9c1552f0 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 >> v1:172.21.15.33:6789/0 conn(0x7f3c9c1552f0 legacy=0x7f3c9c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c84003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 0x7f3c84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c84004e70 0x7f3c840052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c06bdf0 con 0x7f3c84003a90 2023-10-06T05:28:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8400bc90 con 0x7f3c84004e70 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.604+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8400be50 con 0x7f3c9c0b18f0 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c84003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9b7fe700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 0x7f3c84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3ca0b9c700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c84004e70 0x7f3c840052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 -- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 msgr2=0x7f3c84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0b18f0 0x7f3c84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 -- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c84004e70 msgr2=0x7f3c840052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c84004e70 0x7f3c840052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 -- 172.21.15.33:0/2538869890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84003570 con 0x7f3c84003a90 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.605+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c84003e80 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0027f0 tx=0x7f3c8c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1aa9ec5aac84b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c016070 con 0x7f3c84003a90 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c006820 con 0x7f3c84003a90 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3c9affd700 1 -- 172.21.15.33:0/2538869890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c006bc0 con 0x7f3c84003a90 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca139d700 1 -- 172.21.15.33:0/2538869890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 msgr2=0x7f3c84003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca139d700 1 --2- 172.21.15.33:0/2538869890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c84003e80 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0027f0 tx=0x7f3c8c007d30 comp rx=0 tx=0).stop 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca0b9c700 1 -- 172.21.15.33:0/2538869890 reap_dead start 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca139d700 1 -- 172.21.15.33:0/2538869890 shutdown_connections 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca139d700 1 -- 172.21.15.33:0/2538869890 >> 172.21.15.33:0/2538869890 conn(0x7f3c9c06d030 msgr2=0x7f3c9c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca139d700 1 -- 172.21.15.33:0/2538869890 shutdown_connections 2023-10-06T05:28:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.606+0000 7f3ca139d700 1 -- 172.21.15.33:0/2538869890 wait complete. 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.608+0000 7f3ca139d700 1 Processor -- start 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.608+0000 7f3ca139d700 1 -- start start 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3ca139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c9c0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3ca139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c9c169790 0x7f3c9c0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3ca139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0cc0b0 0x7f3c9c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3ca139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0cca50 con 0x7f3c84003a90 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3ca139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0b04e0 con 0x7f3c9c169790 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3ca139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0af8d0 con 0x7f3c9c0cc0b0 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c9c0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c9c0aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50168/0 (socket says 172.21.15.33:50168) 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 -- 172.21.15.33:0/4103065453 learned_addr learned my addr 172.21.15.33:0/4103065453 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 -- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0cc0b0 msgr2=0x7f3c9c0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0cc0b0 0x7f3c9c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 -- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c9c169790 msgr2=0x7f3c9c0c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c9c169790 0x7f3c9c0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.609+0000 7f3c9bfff700 1 -- 172.21.15.33:0/4103065453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0afbb0 con 0x7f3c84003a90 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.610+0000 7f3c9bfff700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c9c0aefd0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c002de0 tx=0x7f3c8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.610+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c007730 con 0x7f3c84003a90 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.610+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c01e650 con 0x7f3c84003a90 2023-10-06T05:28:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.610+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c0afe90 con 0x7f3c84003a90 2023-10-06T05:28:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.610+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c01ea50 con 0x7f3c84003a90 2023-10-06T05:28:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.610+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9c0af510 con 0x7f3c84003a90 2023-10-06T05:28:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.611+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9c0cca50 con 0x7f3c84003a90 2023-10-06T05:28:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.612+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c8c012420 con 0x7f3c84003a90 2023-10-06T05:28:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.613+0000 7f3c83fff700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c88066f30 0x7f3c880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.613+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3c8c0862a0 con 0x7f3c84003a90 2023-10-06T05:28:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.613+0000 7f3c9b7fe700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c88066f30 0x7f3c880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.614+0000 7f3c9b7fe700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c88066f30 0x7f3c880693e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c151760 tx=0x7f3c90007940 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.617+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c9c0cca50 con 0x7f3c84003a90 2023-10-06T05:28:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:04.867+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3c9c0cca50 con 0x7f3c84003a90 2023-10-06T05:28:05.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.237+0000 7f3c83fff700 1 -- 172.21.15.33:0/4103065453 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v150) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3c9c0cca50 con 0x7f3c84003a90 2023-10-06T05:28:05.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c88066f30 msgr2=0x7f3c880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c88066f30 0x7f3c880693e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c151760 tx=0x7f3c90007940 comp rx=0 tx=0).stop 2023-10-06T05:28:05.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 msgr2=0x7f3c9c0aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c9c0aefd0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c002de0 tx=0x7f3c8c016000 comp rx=0 tx=0).stop 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 shutdown_connections 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c88066f30 0x7f3c880693e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c9c0cc0b0 0x7f3c9c0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c84003a90 0x7f3c9c0aefd0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 --2- 172.21.15.33:0/4103065453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c9c169790 0x7f3c9c0c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 >> 172.21.15.33:0/4103065453 conn(0x7f3c9c06d030 msgr2=0x7f3c9c15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 shutdown_connections 2023-10-06T05:28:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.240+0000 7f3ca139d700 1 -- 172.21.15.33:0/4103065453 wait complete. 2023-10-06T05:28:05.244 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:28:05.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:05.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.450+0000 7f92a9f9d700 1 Processor -- start 2023-10-06T05:28:05.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.450+0000 7f92a9f9d700 1 -- start start 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a9f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92a413faa0 0x7f92a413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a9f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40c9cb0 con 0x7f92a406aad0 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a9f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40ca270 con 0x7f92a40c7940 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a9f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a406b640 con 0x7f92a413faa0 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a8f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92a413faa0 0x7f92a413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a979c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f92a406aad0 0x7f92a406af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60132/0 (socket says 172.21.15.33:60132) 2023-10-06T05:28:05.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.451+0000 7f92a979c700 1 -- 172.21.15.33:0/4194916695 learned_addr learned my addr 172.21.15.33:0/4194916695 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a8f9b700 1 -- 172.21.15.33:0/4194916695 >> v1:172.21.15.167:6789/0 conn(0x7f92a40c7940 legacy=0x7f92a40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a8f9b700 1 -- 172.21.15.33:0/4194916695 >> v1:172.21.15.33:6789/0 conn(0x7f92a406aad0 legacy=0x7f92a406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a8f9b700 1 -- 172.21.15.33:0/4194916695 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a406b840 con 0x7f92a413faa0 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a37fe700 1 -- 172.21.15.33:0/4194916695 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718611066 0 0) 0x7f92a40c9cb0 con 0x7f92a406aad0 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a8f9b700 1 --2- 172.21.15.33:0/4194916695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a413faa0 0x7f92a413fe70 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f92980098e0 tx=0x7f92980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec810e4be7a2cab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a37fe700 1 -- 172.21.15.33:0/4194916695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298009e80 con 0x7f92a413faa0 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a37fe700 1 -- 172.21.15.33:0/4194916695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929800b820 con 0x7f92a413faa0 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.452+0000 7f92a37fe700 1 -- 172.21.15.33:0/4194916695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800bba0 con 0x7f92a413faa0 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 -- 172.21.15.33:0/4194916695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a413faa0 msgr2=0x7f92a413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/4194916695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a413faa0 0x7f92a413fe70 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f92980098e0 tx=0x7f92980091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 -- 172.21.15.33:0/4194916695 shutdown_connections 2023-10-06T05:28:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/4194916695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a413faa0 0x7f92a413fe70 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 -- 172.21.15.33:0/4194916695 >> 172.21.15.33:0/4194916695 conn(0x7f92a406d030 msgr2=0x7f92a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 -- 172.21.15.33:0/4194916695 shutdown_connections 2023-10-06T05:28:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.453+0000 7f92a9f9d700 1 -- 172.21.15.33:0/4194916695 wait complete. 2023-10-06T05:28:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.455+0000 7f92a9f9d700 1 Processor -- start 2023-10-06T05:28:05.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.455+0000 7f92a9f9d700 1 -- start start 2023-10-06T05:28:05.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a9f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a406aad0 0x7f92a40c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a9f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 0x7f92a40c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a9f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92a413faa0 0x7f92a40c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a9f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c56c0 con 0x7f92a40c7940 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a9f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c4ab0 con 0x7f92a406aad0 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a9f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c4d90 con 0x7f92a413faa0 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 0x7f92a40c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 0x7f92a40c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50190/0 (socket says 172.21.15.33:50190) 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 -- 172.21.15.33:0/3902222129 learned_addr learned my addr 172.21.15.33:0/3902222129 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a8f9b700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a406aad0 0x7f92a40c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a979c700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92a413faa0 0x7f92a40c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 -- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92a413faa0 msgr2=0x7f92a40c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92a413faa0 0x7f92a40c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 -- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a406aad0 msgr2=0x7f92a40c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a406aad0 0x7f92a40c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.456+0000 7f92a3fff700 1 -- 172.21.15.33:0/3902222129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9298009420 con 0x7f92a40c7940 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.457+0000 7f92a3fff700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 0x7f92a40c1a40 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f929000b600 tx=0x7f92900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.457+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000f510 con 0x7f92a40c7940 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.457+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929000f670 con 0x7f92a40c7940 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.457+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40c5070 con 0x7f92a40c7940 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.457+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000fa30 con 0x7f92a40c7940 2023-10-06T05:28:05.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.457+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a40c46a0 con 0x7f92a40c7940 2023-10-06T05:28:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.458+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a40c5870 con 0x7f92a40c7940 2023-10-06T05:28:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.459+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f929001a400 con 0x7f92a40c7940 2023-10-06T05:28:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.460+0000 7f92a0ff9700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.460+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f92900842c0 con 0x7f92a40c7940 2023-10-06T05:28:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.460+0000 7f92a8f9b700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.461+0000 7f92a8f9b700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f92980030c0 tx=0x7f9298016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.464+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92a40c5870 con 0x7f92a40c7940 2023-10-06T05:28:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.712+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92a404c380 con 0x7f92a40c7940 2023-10-06T05:28:05.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.713+0000 7f92a0ff9700 1 -- 172.21.15.33:0/3902222129 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f9290051380 con 0x7f92a40c7940 2023-10-06T05:28:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:05.715 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":151,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:05.238399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:05.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.715+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9288066f30 msgr2=0x7f92880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.715+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9288066f30 0x7f92880693e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f92980030c0 tx=0x7f9298016000 comp rx=0 tx=0).stop 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.715+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 msgr2=0x7f92a40c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.715+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 0x7f92a40c1a40 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f929000b600 tx=0x7f92900091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 shutdown_connections 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9288066f30 0x7f92880693e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92a413faa0 0x7f92a40c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a406aad0 0x7f92a40c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 --2- 172.21.15.33:0/3902222129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a40c7940 0x7f92a40c1a40 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 >> 172.21.15.33:0/3902222129 conn(0x7f92a406d030 msgr2=0x7f92a413e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 shutdown_connections 2023-10-06T05:28:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:05.716+0000 7f92a9f9d700 1 -- 172.21.15.33:0/3902222129 wait complete. 2023-10-06T05:28:05.719 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 151 2023-10-06T05:28:05.735 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2023-10-06T05:28:06.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:06.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.932+0000 7f955319d700 1 Processor -- start 2023-10-06T05:28:06.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.933+0000 7f955319d700 1 -- start start 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955319d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f954c0c8050 0x7f954c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c06aad0 0x7f954c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955319d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954c0c9cb0 con 0x7f954c13faa0 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955319d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c0ca270 con 0x7f954c0c8050 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c06b840 con 0x7f954c06aad0 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955299c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c06aad0 0x7f954c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955199a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f954c0c8050 0x7f954c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955299c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c06aad0 0x7f954c06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41424/0 (socket says 172.21.15.33:41424) 2023-10-06T05:28:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955299c700 1 -- 172.21.15.33:0/640844762 learned_addr learned my addr 172.21.15.33:0/640844762 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.934+0000 7f955219b700 1 --1- 172.21.15.33:0/640844762 >> v1:172.21.15.167:6789/0 conn(0x7f954c13faa0 0x7f954c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:59862/0 (socket says 172.21.15.33:59862) 2023-10-06T05:28:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.935+0000 7f955199a700 1 -- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c06aad0 msgr2=0x7f954c06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.935+0000 7f955199a700 1 --2- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c06aad0 0x7f954c06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.935+0000 7f955199a700 1 -- 172.21.15.33:0/640844762 >> v1:172.21.15.167:6789/0 conn(0x7f954c13faa0 legacy=0x7f954c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.935+0000 7f955199a700 1 -- 172.21.15.33:0/640844762 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c06bb20 con 0x7f954c0c8050 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.935+0000 7f955199a700 1 --2- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c0c8050 0x7f954c13fe70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098e0 tx=0x7f953c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d098fecf96a13cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.935+0000 7f9551199700 1 -- 172.21.15.33:0/640844762 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00b640 con 0x7f954c0c8050 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f9551199700 1 -- 172.21.15.33:0/640844762 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c00b7a0 con 0x7f954c0c8050 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f9551199700 1 -- 172.21.15.33:0/640844762 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00bb40 con 0x7f954c0c8050 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 -- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c0c8050 msgr2=0x7f954c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 --2- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c0c8050 0x7f954c13fe70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098e0 tx=0x7f953c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 -- 172.21.15.33:0/640844762 shutdown_connections 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 --2- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c06aad0 0x7f954c06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 --2- 172.21.15.33:0/640844762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c0c8050 0x7f954c13fe70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 -- 172.21.15.33:0/640844762 >> 172.21.15.33:0/640844762 conn(0x7f954c06d030 msgr2=0x7f954c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 -- 172.21.15.33:0/640844762 shutdown_connections 2023-10-06T05:28:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.936+0000 7f955319d700 1 -- 172.21.15.33:0/640844762 wait complete. 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.938+0000 7f955319d700 1 Processor -- start 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.938+0000 7f955319d700 1 -- start start 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 0x7f954c13cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c0c8050 0x7f954c166360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c13faa0 0x7f954c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c0c4d80 con 0x7f954c06aad0 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c1668a0 con 0x7f954c13faa0 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c159c10 con 0x7f954c0c8050 2023-10-06T05:28:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 0x7f954c13cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 0x7f954c13cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50206/0 (socket says 172.21.15.33:50206) 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 -- 172.21.15.33:0/2244421485 learned_addr learned my addr 172.21.15.33:0/2244421485 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955299c700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c13faa0 0x7f954c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955199a700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c0c8050 0x7f954c166360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 -- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c0c8050 msgr2=0x7f954c166360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c0c8050 0x7f954c166360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 -- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c13faa0 msgr2=0x7f954c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c13faa0 0x7f954c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.939+0000 7f955219b700 1 -- 172.21.15.33:0/2244421485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c009420 con 0x7f954c06aad0 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.940+0000 7f955219b700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 0x7f954c13cd90 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f954800c560 tx=0x7f9548007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.940+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548005a70 con 0x7f954c06aad0 2023-10-06T05:28:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.940+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9548005bd0 con 0x7f954c06aad0 2023-10-06T05:28:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.940+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480070d0 con 0x7f954c06aad0 2023-10-06T05:28:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.940+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c0c4450 con 0x7f954c06aad0 2023-10-06T05:28:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.940+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c0c4a10 con 0x7f954c06aad0 2023-10-06T05:28:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.941+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c0c8590 con 0x7f954c06aad0 2023-10-06T05:28:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.942+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9548007480 con 0x7f954c06aad0 2023-10-06T05:28:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.943+0000 7f95427fc700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9530066f30 0x7f95300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.943+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9548083ad0 con 0x7f954c06aad0 2023-10-06T05:28:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.943+0000 7f955199a700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9530066f30 0x7f95300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.944+0000 7f955199a700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9530066f30 0x7f95300693e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f953c0093f0 tx=0x7f953c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:06.947+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9548050b10 con 0x7f954c06aad0 2023-10-06T05:28:07.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.194+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f954c0c9af0 con 0x7f954c06aad0 2023-10-06T05:28:07.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.195+0000 7f95427fc700 1 -- 172.21.15.33:0/2244421485 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f9548017020 con 0x7f954c06aad0 2023-10-06T05:28:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:07.197 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9530066f30 msgr2=0x7f95300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9530066f30 0x7f95300693e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f953c0093f0 tx=0x7f953c017000 comp rx=0 tx=0).stop 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 msgr2=0x7f954c13cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 0x7f954c13cd90 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f954800c560 tx=0x7f9548007f70 comp rx=0 tx=0).stop 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 shutdown_connections 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9530066f30 0x7f95300693e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f954c13faa0 0x7f954c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f954c06aad0 0x7f954c13cd90 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 --2- 172.21.15.33:0/2244421485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f954c0c8050 0x7f954c166360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 >> 172.21.15.33:0/2244421485 conn(0x7f954c06d030 msgr2=0x7f954c13ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 shutdown_connections 2023-10-06T05:28:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.197+0000 7f955319d700 1 -- 172.21.15.33:0/2244421485 wait complete. 2023-10-06T05:28:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:07.217 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to up:reconnect 2023-10-06T05:28:07.217 DEBUG:tasks.cephfs.filesystem:reached state 'up:reconnect' in 1.9544620513916016s 2023-10-06T05:28:07.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:07.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.403+0000 7f16c359e700 1 Processor -- start 2023-10-06T05:28:07.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.403+0000 7f16c359e700 1 -- start start 2023-10-06T05:28:07.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.404+0000 7f16c359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f16c413faa0 0x7f16c413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16c406aad0 0x7f16c406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40c9cb0 con 0x7f16c40c7940 2023-10-06T05:28:07.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c40ca270 con 0x7f16c406aad0 2023-10-06T05:28:07.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c406b840 con 0x7f16c413faa0 2023-10-06T05:28:07.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16c406aad0 0x7f16c406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c1d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f16c40c7940 0x7f16c40c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60134/0 (socket says 172.21.15.33:60134) 2023-10-06T05:28:07.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c1d9b700 1 -- 172.21.15.33:0/2925557107 learned_addr learned my addr 172.21.15.33:0/2925557107 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:07.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c259c700 1 --2- 172.21.15.33:0/2925557107 >> v2:172.21.15.167:3300/0 conn(0x7f16c413faa0 0x7f16c413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c2d9d700 1 -- 172.21.15.33:0/2925557107 >> v2:172.21.15.167:3300/0 conn(0x7f16c413faa0 msgr2=0x7f16c413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:07.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c2d9d700 1 --2- 172.21.15.33:0/2925557107 >> v2:172.21.15.167:3300/0 conn(0x7f16c413faa0 0x7f16c413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c2d9d700 1 -- 172.21.15.33:0/2925557107 >> v1:172.21.15.33:6789/0 conn(0x7f16c40c7940 legacy=0x7f16c40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.405+0000 7f16c2d9d700 1 -- 172.21.15.33:0/2925557107 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c406bb20 con 0x7f16c406aad0 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.406+0000 7f16c2d9d700 1 --2- 172.21.15.33:0/2925557107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c406aad0 0x7f16c406af30 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f16c406fe50 tx=0x7f16bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13ca071a1a327e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.406+0000 7f16c159a700 1 -- 172.21.15.33:0/2925557107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00e070 con 0x7f16c406aad0 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.406+0000 7f16c159a700 1 -- 172.21.15.33:0/2925557107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc00bc30 con 0x7f16c406aad0 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.406+0000 7f16c159a700 1 -- 172.21.15.33:0/2925557107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc012430 con 0x7f16c406aad0 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.406+0000 7f16c359e700 1 -- 172.21.15.33:0/2925557107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c406aad0 msgr2=0x7f16c406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.406+0000 7f16c359e700 1 --2- 172.21.15.33:0/2925557107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c406aad0 0x7f16c406af30 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f16c406fe50 tx=0x7f16bc009780 comp rx=0 tx=0).stop 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.407+0000 7f16c359e700 1 -- 172.21.15.33:0/2925557107 shutdown_connections 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.407+0000 7f16c359e700 1 --2- 172.21.15.33:0/2925557107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c406aad0 0x7f16c406af30 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.407+0000 7f16c359e700 1 --2- 172.21.15.33:0/2925557107 >> v2:172.21.15.167:3300/0 conn(0x7f16c413faa0 0x7f16c413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.407+0000 7f16c359e700 1 -- 172.21.15.33:0/2925557107 >> 172.21.15.33:0/2925557107 conn(0x7f16c406d030 msgr2=0x7f16c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.407+0000 7f16c359e700 1 -- 172.21.15.33:0/2925557107 shutdown_connections 2023-10-06T05:28:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.407+0000 7f16c359e700 1 -- 172.21.15.33:0/2925557107 wait complete. 2023-10-06T05:28:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.408+0000 7f16c359e700 1 Processor -- start 2023-10-06T05:28:07.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.408+0000 7f16c359e700 1 -- start start 2023-10-06T05:28:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.409+0000 7f16c359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c406aad0 0x7f16c413e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.409+0000 7f16c359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c40c7940 0x7f16c416d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.409+0000 7f16c359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 0x7f16c413d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.409+0000 7f16c359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4159aa0 con 0x7f16c413faa0 2023-10-06T05:28:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.409+0000 7f16c359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40b0230 con 0x7f16c40c7940 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.409+0000 7f16c359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4159260 con 0x7f16c406aad0 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 0x7f16c413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 0x7f16c413d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50224/0 (socket says 172.21.15.33:50224) 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 -- 172.21.15.33:0/4082521831 learned_addr learned my addr 172.21.15.33:0/4082521831 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c1d9b700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c40c7940 0x7f16c416d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c259c700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c406aad0 0x7f16c413e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 -- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c406aad0 msgr2=0x7f16c413e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c406aad0 0x7f16c413e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 -- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c40c7940 msgr2=0x7f16c416d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c40c7940 0x7f16c416d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 -- 172.21.15.33:0/4082521831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc009a20 con 0x7f16c413faa0 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c1d9b700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c40c7940 0x7f16c416d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c259c700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c406aad0 0x7f16c413e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16c2d9d700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 0x7f16c413d970 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f16bc013000 tx=0x7f16bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.410+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc01e630 con 0x7f16c413faa0 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.411+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc01e7d0 con 0x7f16c413faa0 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.411+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c41594e0 con 0x7f16c413faa0 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.411+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc01eb90 con 0x7f16c413faa0 2023-10-06T05:28:07.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.411+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c40afec0 con 0x7f16c413faa0 2023-10-06T05:28:07.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.412+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c404c380 con 0x7f16c413faa0 2023-10-06T05:28:07.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.412+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16bc024070 con 0x7f16c413faa0 2023-10-06T05:28:07.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.413+0000 7f16b27fc700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f60 0x7f16a4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.414+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f16bc087080 con 0x7f16c413faa0 2023-10-06T05:28:07.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.414+0000 7f16c259c700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f60 0x7f16a4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.415+0000 7f16c259c700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f60 0x7f16a4069410 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009990 tx=0x7f16b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.417+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16bc0b85a0 con 0x7f16c413faa0 2023-10-06T05:28:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.663+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16c40b0370 con 0x7f16c413faa0 2023-10-06T05:28:07.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.664+0000 7f16b27fc700 1 -- 172.21.15.33:0/4082521831 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f16bc0540c0 con 0x7f16c413faa0 2023-10-06T05:28:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:07.666 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f60 msgr2=0x7f16a4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f60 0x7f16a4069410 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009990 tx=0x7f16b4008040 comp rx=0 tx=0).stop 2023-10-06T05:28:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 msgr2=0x7f16c413d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 0x7f16c413d970 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f16bc013000 tx=0x7f16bc019000 comp rx=0 tx=0).stop 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 shutdown_connections 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16a4066f60 0x7f16a4069410 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c413faa0 0x7f16c413d970 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16c406aad0 0x7f16c413e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 --2- 172.21.15.33:0/4082521831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c40c7940 0x7f16c416d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.666+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 >> 172.21.15.33:0/4082521831 conn(0x7f16c406d030 msgr2=0x7f16c415aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.667+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 shutdown_connections 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:07.667+0000 7f16c359e700 1 -- 172.21.15.33:0/4082521831 wait complete. 2023-10-06T05:28:07.669 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:07.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-06T05:28:07.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"cluster_fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","whoami":0,"id":9801,"want_state":"up:reconnect","state":"up:reconnect","fs_name":"cephfs","reconnect_status":{"client_reconnect_gather":"7772"},"rank_uptime":2.6377250409999999,"mdsmap_epoch":152,"osdmap_epoch":195,"osdmap_epoch_barrier":0,"uptime":19.743790558000001} 2023-10-06T05:28:07.895 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", "fs_name": "cephfs", "id": 9801, "mdsmap_epoch": 152, "osdmap_epoch": 195, "osdmap_epoch_barrier": 0, "rank_uptime": 2.637725041, "reconnect_status": { "client_reconnect_gather": "7772" }, "state": "up:reconnect", "uptime": 19.743790558, "want_state": "up:reconnect", "whoami": 0 } 2023-10-06T05:28:07.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:08.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.084+0000 7f307ffff700 1 Processor -- start 2023-10-06T05:28:08.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.084+0000 7f307ffff700 1 -- start start 2023-10-06T05:28:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3080155af0 0x7f308006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308006a6a0 0x7f30800c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f30800c8b50 0x7f30800c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308006b7e0 con 0x7f3080155af0 2023-10-06T05:28:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308006ab60 con 0x7f30800c8b50 2023-10-06T05:28:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308006ae40 con 0x7f308006a6a0 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3080155af0 0x7f308006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3080155af0 0x7f308006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50238/0 (socket says 172.21.15.33:50238) 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307effd700 1 -- 172.21.15.33:0/258975843 learned_addr learned my addr 172.21.15.33:0/258975843 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307e7fc700 1 --2- 172.21.15.33:0/258975843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308006a6a0 0x7f30800c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307f7fe700 1 --2- 172.21.15.33:0/258975843 >> v2:172.21.15.167:3300/0 conn(0x7f30800c8b50 0x7f30800c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307effd700 1 -- 172.21.15.33:0/258975843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308006a6a0 msgr2=0x7f30800c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.086+0000 7f307effd700 1 --2- 172.21.15.33:0/258975843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308006a6a0 0x7f30800c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307effd700 1 -- 172.21.15.33:0/258975843 >> v2:172.21.15.167:3300/0 conn(0x7f30800c8b50 msgr2=0x7f30800c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307effd700 1 --2- 172.21.15.33:0/258975843 >> v2:172.21.15.167:3300/0 conn(0x7f30800c8b50 0x7f30800c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307effd700 1 -- 172.21.15.33:0/258975843 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308006b120 con 0x7f3080155af0 2023-10-06T05:28:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307effd700 1 --2- 172.21.15.33:0/258975843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080155af0 0x7f308006a160 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f30800c66b0 tx=0x7f3070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339e7ffa874ee628 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307dffb700 1 -- 172.21.15.33:0/258975843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000b860 con 0x7f3080155af0 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307dffb700 1 -- 172.21.15.33:0/258975843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307000b9c0 con 0x7f3080155af0 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.087+0000 7f307dffb700 1 -- 172.21.15.33:0/258975843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000bd60 con 0x7f3080155af0 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 -- 172.21.15.33:0/258975843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080155af0 msgr2=0x7f308006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 --2- 172.21.15.33:0/258975843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080155af0 0x7f308006a160 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f30800c66b0 tx=0x7f3070009250 comp rx=0 tx=0).stop 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 -- 172.21.15.33:0/258975843 shutdown_connections 2023-10-06T05:28:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 --2- 172.21.15.33:0/258975843 >> v2:172.21.15.167:3300/0 conn(0x7f30800c8b50 0x7f30800c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 --2- 172.21.15.33:0/258975843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080155af0 0x7f308006a160 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 --2- 172.21.15.33:0/258975843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f308006a6a0 0x7f30800c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 -- 172.21.15.33:0/258975843 >> 172.21.15.33:0/258975843 conn(0x7f308006d030 msgr2=0x7f308006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 -- 172.21.15.33:0/258975843 shutdown_connections 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.088+0000 7f307ffff700 1 -- 172.21.15.33:0/258975843 wait complete. 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.089+0000 7f307ffff700 1 Processor -- start 2023-10-06T05:28:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.089+0000 7f307ffff700 1 -- start start 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.090+0000 7f307ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 0x7f30800cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.090+0000 7f307ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30800c8b50 0x7f308014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.090+0000 7f307ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3080155af0 0x7f308014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.090+0000 7f307ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800ac8a0 con 0x7f308006a6a0 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.090+0000 7f307ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308014b730 con 0x7f30800c8b50 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.090+0000 7f307ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30800b0ce0 con 0x7f3080155af0 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 0x7f30800cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 0x7f30800cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50246/0 (socket says 172.21.15.33:50246) 2023-10-06T05:28:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 -- 172.21.15.33:0/3514770445 learned_addr learned my addr 172.21.15.33:0/3514770445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307e7fc700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30800c8b50 0x7f308014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307f7fe700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3080155af0 0x7f308014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 -- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3080155af0 msgr2=0x7f308014b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3080155af0 0x7f308014b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 -- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30800c8b50 msgr2=0x7f308014acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30800c8b50 0x7f308014acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 -- 172.21.15.33:0/3514770445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700094f0 con 0x7f308006a6a0 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307f7fe700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3080155af0 0x7f308014b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307e7fc700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30800c8b50 0x7f308014acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.091+0000 7f307effd700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 0x7f30800cc130 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f30800635d0 tx=0x7f3070017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.092+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070024440 con 0x7f308006a6a0 2023-10-06T05:28:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.092+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700245a0 con 0x7f308006a6a0 2023-10-06T05:28:08.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.092+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070024960 con 0x7f308006a6a0 2023-10-06T05:28:08.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.092+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30800abf70 con 0x7f308006a6a0 2023-10-06T05:28:08.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.092+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30800ac530 con 0x7f308006a6a0 2023-10-06T05:28:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.093+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308004c380 con 0x7f308006a6a0 2023-10-06T05:28:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.093+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f307001a420 con 0x7f308006a6a0 2023-10-06T05:28:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.095+0000 7f3066ffd700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f306c066f30 0x7f306c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.095+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f3070086670 con 0x7f308006a6a0 2023-10-06T05:28:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.095+0000 7f307e7fc700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f306c066f30 0x7f306c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.096+0000 7f307e7fc700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f306c066f30 0x7f306c0693e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3074006fd0 tx=0x7f3074008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.099+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3070053610 con 0x7f308006a6a0 2023-10-06T05:28:08.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.352+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f308006b400 con 0x7f308006a6a0 2023-10-06T05:28:08.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.352+0000 7f3066ffd700 1 -- 172.21.15.33:0/3514770445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f3070056c30 con 0x7f308006a6a0 2023-10-06T05:28:08.354 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:08.354 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f306c066f30 msgr2=0x7f306c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f306c066f30 0x7f306c0693e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3074006fd0 tx=0x7f3074008040 comp rx=0 tx=0).stop 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 msgr2=0x7f30800cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 0x7f30800cc130 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f30800635d0 tx=0x7f3070017000 comp rx=0 tx=0).stop 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 shutdown_connections 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f306c066f30 0x7f306c0693e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3080155af0 0x7f308014b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308006a6a0 0x7f30800cc130 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 --2- 172.21.15.33:0/3514770445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30800c8b50 0x7f308014acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 >> 172.21.15.33:0/3514770445 conn(0x7f308006d030 msgr2=0x7f3080154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 shutdown_connections 2023-10-06T05:28:08.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.355+0000 7f307ffff700 1 -- 172.21.15.33:0/3514770445 wait complete. 2023-10-06T05:28:08.358 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:08.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:28:08.587 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":7778,"entity":{"name":{"type":"client","num":7778},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1820721818}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.3199476200000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99466327935300414,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7778 v1:192.168.0.1:0/1820721818","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":7772,"entity":{"name":{"type":"client","num":7772},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":1091933932}},"state":"open","num_leases":0,"num_caps":0,"request_load_avg":0,"uptime":3.3199476200000002,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":true,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7772 v1:172.21.15.33:0/1091933932","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:28:08.588 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1820721818, "type": "v1" }, "name": { "num": 7778, "type": "client" } }, "id": 7778, "inst": "client.7778 v1:192.168.0.1:0/1820721818", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9946632793530041 }, "state": "open", "uptime": 3.31994762 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 1091933932, "type": "v1" }, "name": { "num": 7772, "type": "client" } }, "id": 7772, "inst": "client.7772 v1:172.21.15.33:0/1091933932", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 0, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": true, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0 }, "state": "open", "uptime": 3.31994762 } ] 2023-10-06T05:28:08.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:08.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.773+0000 7f94a919d700 1 Processor -- start 2023-10-06T05:28:08.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.773+0000 7f94a919d700 1 -- start start 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a919d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94a40cb710 0x7f94a406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a919d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a40c9cb0 con 0x7f94a40c8050 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a919d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a40ca270 con 0x7f94a413faa0 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a919d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a406b840 con 0x7f94a40cb710 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a899c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94a40cb710 0x7f94a406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f94a40c8050 0x7f94a406aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60150/0 (socket says 172.21.15.33:60150) 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.775+0000 7f94a37fe700 1 -- 172.21.15.33:0/3294797618 learned_addr learned my addr 172.21.15.33:0/3294797618 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a899c700 1 -- 172.21.15.33:0/3294797618 >> v1:172.21.15.167:6789/0 conn(0x7f94a413faa0 legacy=0x7f94a40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a899c700 1 -- 172.21.15.33:0/3294797618 >> v1:172.21.15.33:6789/0 conn(0x7f94a40c8050 legacy=0x7f94a406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a899c700 1 -- 172.21.15.33:0/3294797618 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a406bb20 con 0x7f94a40cb710 2023-10-06T05:28:08.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a2ffd700 1 -- 172.21.15.33:0/3294797618 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267723375 0 0) 0x7f94a40c9cb0 con 0x7f94a40c8050 2023-10-06T05:28:08.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a899c700 1 --2- 172.21.15.33:0/3294797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a40cb710 0x7f94a406b1e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f949c009e60 tx=0x7f949c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404325966fa5e7df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a2ffd700 1 -- 172.21.15.33:0/3294797618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00e070 con 0x7f94a40cb710 2023-10-06T05:28:08.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a2ffd700 1 -- 172.21.15.33:0/3294797618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c00bc30 con 0x7f94a40cb710 2023-10-06T05:28:08.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.776+0000 7f94a2ffd700 1 -- 172.21.15.33:0/3294797618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c012430 con 0x7f94a40cb710 2023-10-06T05:28:08.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 -- 172.21.15.33:0/3294797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a40cb710 msgr2=0x7f94a406b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 --2- 172.21.15.33:0/3294797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a40cb710 0x7f94a406b1e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f949c009e60 tx=0x7f949c009780 comp rx=0 tx=0).stop 2023-10-06T05:28:08.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 -- 172.21.15.33:0/3294797618 shutdown_connections 2023-10-06T05:28:08.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 --2- 172.21.15.33:0/3294797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a40cb710 0x7f94a406b1e0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 -- 172.21.15.33:0/3294797618 >> 172.21.15.33:0/3294797618 conn(0x7f94a406d030 msgr2=0x7f94a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 -- 172.21.15.33:0/3294797618 shutdown_connections 2023-10-06T05:28:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.777+0000 7f94a919d700 1 -- 172.21.15.33:0/3294797618 wait complete. 2023-10-06T05:28:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.778+0000 7f94a919d700 1 Processor -- start 2023-10-06T05:28:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.778+0000 7f94a919d700 1 -- start start 2023-10-06T05:28:08.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94a40c8050 0x7f94a416cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a40cb710 0x7f94a4139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a919d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 0x7f94a40c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a919d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a416de40 con 0x7f94a413faa0 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4169a00 con 0x7f94a40cb710 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a413c370 con 0x7f94a40c8050 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.779+0000 7f94a899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 0x7f94a40c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 0x7f94a40c0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50256/0 (socket says 172.21.15.33:50256) 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 -- 172.21.15.33:0/2524912323 learned_addr learned my addr 172.21.15.33:0/2524912323 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a37fe700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a40cb710 0x7f94a4139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a3fff700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94a40c8050 0x7f94a416cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 -- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94a40c8050 msgr2=0x7f94a416cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94a40c8050 0x7f94a416cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 -- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a40cb710 msgr2=0x7f94a4139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a40cb710 0x7f94a4139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 -- 172.21.15.33:0/2524912323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c009a20 con 0x7f94a413faa0 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a899c700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 0x7f94a40c0f80 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f949c013000 tx=0x7f949c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c01b630 con 0x7f94a413faa0 2023-10-06T05:28:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c01b7b0 con 0x7f94a413faa0 2023-10-06T05:28:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a416d880 con 0x7f94a413faa0 2023-10-06T05:28:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.780+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c01bb70 con 0x7f94a413faa0 2023-10-06T05:28:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.781+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a413c000 con 0x7f94a413faa0 2023-10-06T05:28:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.781+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a413c520 con 0x7f94a413faa0 2023-10-06T05:28:08.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.782+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f949c01c400 con 0x7f94a413faa0 2023-10-06T05:28:08.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.783+0000 7f948bfff700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9490066f30 0x7f94900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.783+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f949c0867b0 con 0x7f94a413faa0 2023-10-06T05:28:08.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.784+0000 7f94a3fff700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9490066f30 0x7f94900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.785+0000 7f94a3fff700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9490066f30 0x7f94900693e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f94940037c0 tx=0x7f9494009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:08.787+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94a413c520 con 0x7f94a413faa0 2023-10-06T05:28:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.039+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94a413c4e0 con 0x7f94a413faa0 2023-10-06T05:28:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.039+0000 7f948bfff700 1 -- 172.21.15.33:0/2524912323 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f949c053750 con 0x7f94a413faa0 2023-10-06T05:28:09.041 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:09.041 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9490066f30 msgr2=0x7f94900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9490066f30 0x7f94900693e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f94940037c0 tx=0x7f9494009040 comp rx=0 tx=0).stop 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 msgr2=0x7f94a40c0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 0x7f94a40c0f80 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f949c013000 tx=0x7f949c019000 comp rx=0 tx=0).stop 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 shutdown_connections 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9490066f30 0x7f94900693e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94a413faa0 0x7f94a40c0f80 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94a40c8050 0x7f94a416cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 --2- 172.21.15.33:0/2524912323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a40cb710 0x7f94a4139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 >> 172.21.15.33:0/2524912323 conn(0x7f94a406d030 msgr2=0x7f94a40c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 shutdown_connections 2023-10-06T05:28:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.042+0000 7f94a919d700 1 -- 172.21.15.33:0/2524912323 wait complete. 2023-10-06T05:28:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:09.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:28:09.272 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":7778,"entity":{"name":{"type":"client","num":7778},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1820721818}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.0049368120000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99309030955001198,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7778 v1:192.168.0.1:0/1820721818","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":7772,"entity":{"name":{"type":"client","num":7772},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":1091933932}},"state":"open","num_leases":0,"num_caps":0,"request_load_avg":0,"uptime":4.0049368120000004,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":true,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7772 v1:172.21.15.33:0/1091933932","completed_requests":[{"tid":2,"created_ino":"0x0"}],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:28:09.272 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1820721818, "type": "v1" }, "name": { "num": 7778, "type": "client" } }, "id": 7778, "inst": "client.7778 v1:192.168.0.1:0/1820721818", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.993090309550012 }, "state": "open", "uptime": 4.004936812 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 2 } ], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 1091933932, "type": "v1" }, "name": { "num": 7772, "type": "client" } }, "id": 7772, "inst": "client.7772 v1:172.21.15.33:0/1091933932", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 0, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": true, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0 }, "state": "open", "uptime": 4.004936812 } ] 2023-10-06T05:28:09.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:09.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.460+0000 7fdfa339d700 1 Processor -- start 2023-10-06T05:28:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.460+0000 7fdfa339d700 1 -- start start 2023-10-06T05:28:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.461+0000 7fdfa339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf9c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.461+0000 7fdfa339d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c06e3e0 con 0x7fdf9c06df70 2023-10-06T05:28:09.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.461+0000 7fdfa339d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c06f220 con 0x7fdf9c0b04e0 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.461+0000 7fdfa339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b1e00 con 0x7fdf9c159530 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdf9c06df70 0x7fdf9c0afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60158/0 (socket says 172.21.15.33:60158) 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1b9a700 1 -- 172.21.15.33:0/436736807 learned_addr learned my addr 172.21.15.33:0/436736807 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa239b700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf9c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442685913 0 0) 0x7fdf9c06e3e0 con 0x7fdf9c06df70 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf88003570 con 0x7fdf9c06df70 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155667625 0 0) 0x7fdf9c06f220 con 0x7fdf9c0b04e0 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf9c06e3e0 con 0x7fdf9c0b04e0 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397724054 0 0) 0x7fdf88003570 con 0x7fdf9c06df70 2023-10-06T05:28:09.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 msgr2=0x7fdf9c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf9c06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 >> v1:172.21.15.167:6789/0 conn(0x7fdf9c0b04e0 legacy=0x7fdf9c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c06e610 con 0x7fdf9c06df70 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fdf8c001470 con 0x7fdf9c06df70 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.462+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 >> v1:172.21.15.33:6789/0 conn(0x7fdf9c06df70 legacy=0x7fdf9c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1399700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf88003c20 0x7fdf88003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1399700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1399700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88004d50 0x7fdf880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b1e00 con 0x7fdf88003c20 2023-10-06T05:28:09.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8800bb10 con 0x7fdf88004d50 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8800bcd0 con 0x7fdf9c159530 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa239b700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf88003c20 0x7fdf88003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1b9a700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa2b9c700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88004d50 0x7fdf880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa239b700 1 -- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 msgr2=0x7fdf88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa239b700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa239b700 1 -- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88004d50 msgr2=0x7fdf880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa239b700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88004d50 0x7fdf880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa239b700 1 -- 172.21.15.33:0/436736807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88003570 con 0x7fdf88003c20 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.463+0000 7fdfa1b9a700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c159530 0x7fdf88004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa239b700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf88003c20 0x7fdf88003ff0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009c00 tx=0x7fdf980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16defc8e37b245ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800e710 con 0x7fdf88003c20 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800e870 con 0x7fdf88003c20 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa1399700 1 -- 172.21.15.33:0/436736807 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800ec30 con 0x7fdf88003c20 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa339d700 1 -- 172.21.15.33:0/436736807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf88003c20 msgr2=0x7fdf88003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa339d700 1 --2- 172.21.15.33:0/436736807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf88003c20 0x7fdf88003ff0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009c00 tx=0x7fdf980091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa2b9c700 1 -- 172.21.15.33:0/436736807 reap_dead start 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa339d700 1 -- 172.21.15.33:0/436736807 shutdown_connections 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa339d700 1 -- 172.21.15.33:0/436736807 >> 172.21.15.33:0/436736807 conn(0x7fdf9c06c560 msgr2=0x7fdf9c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:09.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa339d700 1 -- 172.21.15.33:0/436736807 shutdown_connections 2023-10-06T05:28:09.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.464+0000 7fdfa339d700 1 -- 172.21.15.33:0/436736807 wait complete. 2023-10-06T05:28:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.466+0000 7fdfa339d700 1 Processor -- start 2023-10-06T05:28:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.466+0000 7fdfa339d700 1 -- start start 2023-10-06T05:28:09.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88003c20 0x7fdf9c14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 0x7fdf9c150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c151490 0x7fdf9c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c150b30 con 0x7fdf9c1529e0 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c14feb0 con 0x7fdf88003c20 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c150190 con 0x7fdf9c151490 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa1b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 0x7fdf9c150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa1b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 0x7fdf9c150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50280/0 (socket says 172.21.15.33:50280) 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa1b9a700 1 -- 172.21.15.33:0/2698215289 learned_addr learned my addr 172.21.15.33:0/2698215289 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:09.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa2b9c700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c151490 0x7fdf9c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.467+0000 7fdfa239b700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88003c20 0x7fdf9c14de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa1b9a700 1 -- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c151490 msgr2=0x7fdf9c0c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa1b9a700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c151490 0x7fdf9c0c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa1b9a700 1 -- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88003c20 msgr2=0x7fdf9c14de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa1b9a700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88003c20 0x7fdf9c14de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa1b9a700 1 -- 172.21.15.33:0/2698215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c150470 con 0x7fdf9c1529e0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa1b9a700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 0x7fdf9c150f50 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00eb90 tx=0x7fdf9c06f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c005e90 con 0x7fdf9c1529e0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c00c7e0 con 0x7fdf9c1529e0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00cba0 con 0x7fdf9c1529e0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c150750 con 0x7fdf9c1529e0 2023-10-06T05:28:09.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.468+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c14e500 con 0x7fdf9c1529e0 2023-10-06T05:28:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.469+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c150c70 con 0x7fdf9c1529e0 2023-10-06T05:28:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.470+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf8c01d460 con 0x7fdf9c1529e0 2023-10-06T05:28:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.471+0000 7fdf927fc700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf80066f30 0x7fdf800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.471+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fdf8c0846e0 con 0x7fdf9c1529e0 2023-10-06T05:28:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.472+0000 7fdfa239b700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf80066f30 0x7fdf800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.472+0000 7fdfa239b700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf80066f30 0x7fdf800693e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdf980091e0 tx=0x7fdf9801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:09.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.475+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf9c150c70 con 0x7fdf9c1529e0 2023-10-06T05:28:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.722+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf9c150c70 con 0x7fdf9c1529e0 2023-10-06T05:28:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.723+0000 7fdf927fc700 1 -- 172.21.15.33:0/2698215289 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fdf9c150c70 con 0x7fdf9c1529e0 2023-10-06T05:28:09.724 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:09.724 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf80066f30 msgr2=0x7fdf800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf80066f30 0x7fdf800693e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdf980091e0 tx=0x7fdf9801a000 comp rx=0 tx=0).stop 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 msgr2=0x7fdf9c150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 0x7fdf9c150f50 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00eb90 tx=0x7fdf9c06f190 comp rx=0 tx=0).stop 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 shutdown_connections 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf80066f30 0x7fdf800693e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf9c151490 0x7fdf9c0c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf88003c20 0x7fdf9c14de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 --2- 172.21.15.33:0/2698215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c1529e0 0x7fdf9c150f50 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 >> 172.21.15.33:0/2698215289 conn(0x7fdf9c06c560 msgr2=0x7fdf9c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 shutdown_connections 2023-10-06T05:28:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:09.725+0000 7fdfa339d700 1 -- 172.21.15.33:0/2698215289 wait complete. 2023-10-06T05:28:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:09.744 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:10.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:10.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.941+0000 7fb53099d700 1 Processor -- start 2023-10-06T05:28:10.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.941+0000 7fb53099d700 1 -- start start 2023-10-06T05:28:10.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.942+0000 7fb53099d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb52c159530 0x7fb52c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:10.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.942+0000 7fb53099d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb52c06df70 0x7fb52c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:10.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.942+0000 7fb53099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c06e3e0 con 0x7fb52c0b0310 2023-10-06T05:28:10.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.942+0000 7fb53099d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c0b1e00 con 0x7fb52c06df70 2023-10-06T05:28:10.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.942+0000 7fb53099d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c06f6b0 con 0x7fb52c159530 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb52c0b0310 0x7fb52c159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54536/0 (socket says 172.21.15.33:54536) 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb52c06df70 0x7fb52c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52bfff700 1 -- 172.21.15.33:0/3319178082 learned_addr learned my addr 172.21.15.33:0/3319178082 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52b7fe700 1 --2- 172.21.15.33:0/3319178082 >> v2:172.21.15.167:3300/0 conn(0x7fb52c159530 0x7fb52c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52affd700 1 -- 172.21.15.33:0/3319178082 >> v2:172.21.15.167:3300/0 conn(0x7fb52c159530 msgr2=0x7fb52c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52affd700 1 --2- 172.21.15.33:0/3319178082 >> v2:172.21.15.167:3300/0 conn(0x7fb52c159530 0x7fb52c06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:10.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52affd700 1 -- 172.21.15.33:0/3319178082 >> v1:172.21.15.33:6789/0 conn(0x7fb52c0b0310 legacy=0x7fb52c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52affd700 1 -- 172.21.15.33:0/3319178082 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c06f220 con 0x7fb52c06df70 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52b7fe700 1 --2- 172.21.15.33:0/3319178082 >> v2:172.21.15.167:3300/0 conn(0x7fb52c159530 0x7fb52c06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52a7fc700 1 -- 172.21.15.33:0/3319178082 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33258594 0 0) 0x7fb52c06e3e0 con 0x7fb52c0b0310 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.943+0000 7fb52affd700 1 --2- 172.21.15.33:0/3319178082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afdd0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0cf4d0 tx=0x7fb5200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35edbf2c3fc66f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb52a7fc700 1 -- 172.21.15.33:0/3319178082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520009e80 con 0x7fb52c06df70 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb52a7fc700 1 -- 172.21.15.33:0/3319178082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000b820 con 0x7fb52c06df70 2023-10-06T05:28:10.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb52a7fc700 1 -- 172.21.15.33:0/3319178082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000bba0 con 0x7fb52c06df70 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 -- 172.21.15.33:0/3319178082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 msgr2=0x7fb52c0afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 --2- 172.21.15.33:0/3319178082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afdd0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0cf4d0 tx=0x7fb5200091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 -- 172.21.15.33:0/3319178082 shutdown_connections 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 --2- 172.21.15.33:0/3319178082 >> v2:172.21.15.167:3300/0 conn(0x7fb52c159530 0x7fb52c06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 --2- 172.21.15.33:0/3319178082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afdd0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 -- 172.21.15.33:0/3319178082 >> 172.21.15.33:0/3319178082 conn(0x7fb52c06c560 msgr2=0x7fb52c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:10.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 -- 172.21.15.33:0/3319178082 shutdown_connections 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.944+0000 7fb53099d700 1 -- 172.21.15.33:0/3319178082 wait complete. 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.946+0000 7fb53099d700 1 Processor -- start 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.946+0000 7fb53099d700 1 -- start start 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb53099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb53099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb52c0b0310 0x7fb52c16a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb53099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb52c159530 0x7fb52c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:10.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb53099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b01b0 con 0x7fb52c06df70 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb53099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c16a9b0 con 0x7fb52c159530 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb53099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0cb250 con 0x7fb52c0b0310 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51998/0 (socket says 172.21.15.33:51998) 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb52b7fe700 1 -- 172.21.15.33:0/3829103279 learned_addr learned my addr 172.21.15.33:0/3829103279 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.947+0000 7fb52bfff700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb52c159530 0x7fb52c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52affd700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb52c0b0310 0x7fb52c16a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:10.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52b7fe700 1 -- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb52c0b0310 msgr2=0x7fb52c16a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52b7fe700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb52c0b0310 0x7fb52c16a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52b7fe700 1 -- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb52c159530 msgr2=0x7fb52c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52b7fe700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb52c159530 0x7fb52c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52b7fe700 1 -- 172.21.15.33:0/3829103279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009420 con 0x7fb52c06df70 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb52b7fe700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afc70 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00d860 tx=0x7fb51c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0125f0 con 0x7fb52c06df70 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c012750 con 0x7fb52c06df70 2023-10-06T05:28:10.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c0cbc20 con 0x7fb52c06df70 2023-10-06T05:28:10.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c012ad0 con 0x7fb52c06df70 2023-10-06T05:28:10.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.948+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c0c8db0 con 0x7fb52c06df70 2023-10-06T05:28:10.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.949+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0cb530 con 0x7fb52c06df70 2023-10-06T05:28:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.950+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb51c014430 con 0x7fb52c06df70 2023-10-06T05:28:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.951+0000 7fb5137fe700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb518066f30 0x7fb5180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.951+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb51c084c80 con 0x7fb52c06df70 2023-10-06T05:28:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.951+0000 7fb52affd700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb518066f30 0x7fb5180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:10.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.952+0000 7fb52affd700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb518066f30 0x7fb5180693e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb5200093f0 tx=0x7fb520017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:10.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:10.955+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb51c051c20 con 0x7fb52c06df70 2023-10-06T05:28:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.202+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb52c04c380 con 0x7fb52c06df70 2023-10-06T05:28:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.202+0000 7fb5137fe700 1 -- 172.21.15.33:0/3829103279 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fb51c055240 con 0x7fb52c06df70 2023-10-06T05:28:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:11.204 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb518066f30 msgr2=0x7fb5180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb518066f30 0x7fb5180693e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb5200093f0 tx=0x7fb520017000 comp rx=0 tx=0).stop 2023-10-06T05:28:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 msgr2=0x7fb52c0afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afc70 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00d860 tx=0x7fb51c008040 comp rx=0 tx=0).stop 2023-10-06T05:28:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 shutdown_connections 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb518066f30 0x7fb5180693e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb52c159530 0x7fb52c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb52c06df70 0x7fb52c0afc70 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 --2- 172.21.15.33:0/3829103279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb52c0b0310 0x7fb52c16a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 >> 172.21.15.33:0/3829103279 conn(0x7fb52c06c560 msgr2=0x7fb52c159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 shutdown_connections 2023-10-06T05:28:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:11.205+0000 7fb53099d700 1 -- 172.21.15.33:0/3829103279 wait complete. 2023-10-06T05:28:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:11.224 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:12.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:12.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.422+0000 7f31d159d700 1 Processor -- start 2023-10-06T05:28:12.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.422+0000 7f31d159d700 1 -- start start 2023-10-06T05:28:12.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31d159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31cc0b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31d159d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc06f220 con 0x7f31cc159530 2023-10-06T05:28:12.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31d159d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc0b0730 con 0x7f31cc06e140 2023-10-06T05:28:12.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31d159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0b1e00 con 0x7f31cc0afdd0 2023-10-06T05:28:12.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31cbfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f31cc159530 0x7f31cc06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54540/0 (socket says 172.21.15.33:54540) 2023-10-06T05:28:12.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31cbfff700 1 -- 172.21.15.33:0/1376755124 learned_addr learned my addr 172.21.15.33:0/1376755124 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31d0d9c700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31cc0b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658704390 0 0) 0x7f31cc06f220 con 0x7f31cc159530 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.424+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b4003570 con 0x7f31cc159530 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978231146 0 0) 0x7f31cc0b0730 con 0x7f31cc06e140 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31cc06f220 con 0x7f31cc06e140 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304597469 0 0) 0x7f31b4003570 con 0x7f31cc159530 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 msgr2=0x7f31cc0b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31cc0b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 >> v1:172.21.15.167:6789/0 conn(0x7f31cc06e140 legacy=0x7f31cc15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc06e610 con 0x7f31cc159530 2023-10-06T05:28:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f31bc001470 con 0x7f31cc159530 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 >> v1:172.21.15.33:6789/0 conn(0x7f31cc159530 legacy=0x7f31cc06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b4003c20 0x7f31b4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31b4004d50 0x7f31b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0b1e00 con 0x7f31b4003c20 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b400bb40 con 0x7f31b4004d50 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b400bd00 con 0x7f31cc0afdd0 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31cbfff700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b4003c20 0x7f31b4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31cb7fe700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.425+0000 7f31d0d9c700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31b4004d50 0x7f31b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cbfff700 1 -- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 msgr2=0x7f31b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cbfff700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cbfff700 1 -- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31b4004d50 msgr2=0x7f31b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cbfff700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31b4004d50 0x7f31b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cbfff700 1 -- 172.21.15.33:0/1376755124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4003570 con 0x7f31b4003c20 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31d0d9c700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31b4004d50 0x7f31b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cb7fe700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31cc0afdd0 0x7f31b4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31cbfff700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b4003c20 0x7f31b4004010 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00c2d0 tx=0x7f31bc001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53f27c1559c9b9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00cba0 con 0x7f31b4003c20 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc00cd00 con 0x7f31b4003c20 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.426+0000 7f31caffd700 1 -- 172.21.15.33:0/1376755124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc005b70 con 0x7f31b4003c20 2023-10-06T05:28:12.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d159d700 1 -- 172.21.15.33:0/1376755124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b4003c20 msgr2=0x7f31b4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d159d700 1 --2- 172.21.15.33:0/1376755124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31b4003c20 0x7f31b4004010 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00c2d0 tx=0x7f31bc001e10 comp rx=0 tx=0).stop 2023-10-06T05:28:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d0d9c700 1 -- 172.21.15.33:0/1376755124 reap_dead start 2023-10-06T05:28:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d159d700 1 -- 172.21.15.33:0/1376755124 shutdown_connections 2023-10-06T05:28:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d159d700 1 -- 172.21.15.33:0/1376755124 >> 172.21.15.33:0/1376755124 conn(0x7f31cc06c560 msgr2=0x7f31cc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d159d700 1 -- 172.21.15.33:0/1376755124 shutdown_connections 2023-10-06T05:28:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.427+0000 7f31d159d700 1 -- 172.21.15.33:0/1376755124 wait complete. 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.428+0000 7f31d159d700 1 Processor -- start 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.428+0000 7f31d159d700 1 -- start start 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31b4003c20 0x7f31cc146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc157cc0 0x7f31cc158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 0x7f31cc0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc158670 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0cea20 con 0x7f31cc157cc0 2023-10-06T05:28:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0ced00 con 0x7f31b4003c20 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.429+0000 7f31d0d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 0x7f31cc0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 0x7f31cc0ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52016/0 (socket says 172.21.15.33:52016) 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 -- 172.21.15.33:0/3789118229 learned_addr learned my addr 172.21.15.33:0/3789118229 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31cbfff700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31b4003c20 0x7f31cc146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 -- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31b4003c20 msgr2=0x7f31cc146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31b4003c20 0x7f31cc146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 -- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc157cc0 msgr2=0x7f31cc158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc157cc0 0x7f31cc158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 -- 172.21.15.33:0/3789118229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc0cefe0 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d0d9c700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 0x7f31cc0ce130 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0b0a90 tx=0x7f31c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c401b070 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c400f040 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.430+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc0c92d0 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.431+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400e050 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.431+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1472c0 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.431+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc0c9130 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.432+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31c4003020 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.433+0000 7f31b3fff700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31b8066f30 0x7f31b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:12.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.433+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f31c4084d50 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.434+0000 7f31cbfff700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31b8066f30 0x7f31b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:12.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.435+0000 7f31cbfff700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31b8066f30 0x7f31b80693e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f31bc007c90 tx=0x7f31bc015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.438+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31cc0c9130 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.682+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31cc0c9130 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.683+0000 7f31b3fff700 1 -- 172.21.15.33:0/3789118229 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f31cc0c9130 con 0x7f31cc0cdcb0 2023-10-06T05:28:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:12.685 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31b8066f30 msgr2=0x7f31b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31b8066f30 0x7f31b80693e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f31bc007c90 tx=0x7f31bc015000 comp rx=0 tx=0).stop 2023-10-06T05:28:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 msgr2=0x7f31cc0ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 0x7f31cc0ce130 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0b0a90 tx=0x7f31c400b040 comp rx=0 tx=0).stop 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 shutdown_connections 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31b8066f30 0x7f31b80693e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31cc0cdcb0 0x7f31cc0ce130 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31b4003c20 0x7f31cc146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 --2- 172.21.15.33:0/3789118229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31cc157cc0 0x7f31cc158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.685+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 >> 172.21.15.33:0/3789118229 conn(0x7f31cc06c560 msgr2=0x7f31cc15b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.686+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 shutdown_connections 2023-10-06T05:28:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:12.686+0000 7f31d159d700 1 -- 172.21.15.33:0/3789118229 wait complete. 2023-10-06T05:28:12.689 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:12.704 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:13.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:13.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.905+0000 7f5475b9d700 1 Processor -- start 2023-10-06T05:28:13.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.905+0000 7f5475b9d700 1 -- start start 2023-10-06T05:28:13.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.906+0000 7f5475b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 0x7f547013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.906+0000 7f5475b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700c9cb0 con 0x7f547006aad0 2023-10-06T05:28:13.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.906+0000 7f5475b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700ca270 con 0x7f54700c7940 2023-10-06T05:28:13.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.906+0000 7f5475b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547006b660 con 0x7f547013faa0 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.906+0000 7f547539c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f547006aad0 0x7f547006af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54542/0 (socket says 172.21.15.33:54542) 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.906+0000 7f547539c700 1 -- 172.21.15.33:0/814682338 learned_addr learned my addr 172.21.15.33:0/814682338 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f5474b9b700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 0x7f547013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985053795 0 0) 0x7f54700c9cb0 con 0x7f547006aad0 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5460003570 con 0x7f547006aad0 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238526050 0 0) 0x7f54700ca270 con 0x7f54700c7940 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54700c9cb0 con 0x7f54700c7940 2023-10-06T05:28:13.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323051019 0 0) 0x7f5460003570 con 0x7f547006aad0 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 msgr2=0x7f547013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 0x7f547013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 >> v1:172.21.15.167:6789/0 conn(0x7f54700c7940 legacy=0x7f54700c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547006b840 con 0x7f547006aad0 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f54680017f0 con 0x7f547006aad0 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 >> v1:172.21.15.33:6789/0 conn(0x7f547006aad0 legacy=0x7f547006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f5460004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 0x7f5460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5460004d50 0x7f54600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547006b660 con 0x7f5460003c20 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546000bb40 con 0x7f5460004d50 2023-10-06T05:28:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.907+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546000bd00 con 0x7f547013faa0 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f5460004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f546ffff700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 0x7f5460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f547539c700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5460004d50 0x7f54600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 -- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 msgr2=0x7f5460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f547013faa0 0x7f5460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 -- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5460004d50 msgr2=0x7f54600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5460004d50 0x7f54600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 -- 172.21.15.33:0/814682338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460003570 con 0x7f5460003c20 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f5474b9b700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f5460004010 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f5464009c00 tx=0x7f54640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e27057b4f39197 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400e710 con 0x7f5460003c20 2023-10-06T05:28:13.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400e870 con 0x7f5460003c20 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.908+0000 7f546f7fe700 1 -- 172.21.15.33:0/814682338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400ec30 con 0x7f5460003c20 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f5475b9d700 1 -- 172.21.15.33:0/814682338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 msgr2=0x7f5460004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f5475b9d700 1 --2- 172.21.15.33:0/814682338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f5460004010 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f5464009c00 tx=0x7f54640091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f547539c700 1 -- 172.21.15.33:0/814682338 reap_dead start 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f5475b9d700 1 -- 172.21.15.33:0/814682338 shutdown_connections 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f5475b9d700 1 -- 172.21.15.33:0/814682338 >> 172.21.15.33:0/814682338 conn(0x7f547006d030 msgr2=0x7f547006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f5475b9d700 1 -- 172.21.15.33:0/814682338 shutdown_connections 2023-10-06T05:28:13.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.909+0000 7f5475b9d700 1 -- 172.21.15.33:0/814682338 wait complete. 2023-10-06T05:28:13.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.911+0000 7f5475b9d700 1 Processor -- start 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.911+0000 7f5475b9d700 1 -- start start 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5475b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f54700c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5475b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54700ae890 0x7f5470169620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5475b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470169b60 0x7f547016a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5475b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547016a540 con 0x7f5460003c20 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5475b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c49f0 con 0x7f5470169b60 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5475b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c40c0 con 0x7f54700ae890 2023-10-06T05:28:13.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f54700c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f54700c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52032/0 (socket says 172.21.15.33:52032) 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 -- 172.21.15.33:0/4203615563 learned_addr learned my addr 172.21.15.33:0/4203615563 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f546ffff700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54700ae890 0x7f5470169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f547539c700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470169b60 0x7f547016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 -- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54700ae890 msgr2=0x7f5470169620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54700ae890 0x7f5470169620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 -- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470169b60 msgr2=0x7f547016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:13.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470169b60 0x7f547016a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.912+0000 7f5474b9b700 1 -- 172.21.15.33:0/4203615563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700c43a0 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.913+0000 7f5474b9b700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f54700c5fd0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f5460004790 tx=0x7f546401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.913+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464022070 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.913+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400ebf0 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.913+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54640276c0 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.913+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54700c4680 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.913+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54700c3d90 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.914+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547004c380 con 0x7f5460003c20 2023-10-06T05:28:13.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.915+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f546401c450 con 0x7f5460003c20 2023-10-06T05:28:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.916+0000 7f546cff9700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5454066f30 0x7f54540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.916+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f5464086460 con 0x7f5460003c20 2023-10-06T05:28:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.916+0000 7f546ffff700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5454066f30 0x7f54540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:13.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.917+0000 7f546ffff700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5454066f30 0x7f54540693e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f54700c6eb0 tx=0x7f545c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:13.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:13.920+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5464053400 con 0x7f5460003c20 2023-10-06T05:28:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.168+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f547006bb20 con 0x7f5460003c20 2023-10-06T05:28:14.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.168+0000 7f546cff9700 1 -- 172.21.15.33:0/4203615563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f5464017090 con 0x7f5460003c20 2023-10-06T05:28:14.170 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:14.171 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:14.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5454066f30 msgr2=0x7f54540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5454066f30 0x7f54540693e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f54700c6eb0 tx=0x7f545c009040 comp rx=0 tx=0).stop 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 msgr2=0x7f54700c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f54700c5fd0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f5460004790 tx=0x7f546401a000 comp rx=0 tx=0).stop 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 shutdown_connections 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5454066f30 0x7f54540693e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5470169b60 0x7f547016a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5460003c20 0x7f54700c5fd0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:14.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 --2- 172.21.15.33:0/4203615563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54700ae890 0x7f5470169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 >> 172.21.15.33:0/4203615563 conn(0x7f547006d030 msgr2=0x7f54700ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 shutdown_connections 2023-10-06T05:28:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:14.171+0000 7f5475b9d700 1 -- 172.21.15.33:0/4203615563 wait complete. 2023-10-06T05:28:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:14.190 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:15.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.389+0000 7f26c5f9d700 1 Processor -- start 2023-10-06T05:28:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.389+0000 7f26c5f9d700 1 -- start start 2023-10-06T05:28:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.390+0000 7f26c5f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f26c0155af0 0x7f26c006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.390+0000 7f26c5f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c006ab60 con 0x7f26c00c7920 2023-10-06T05:28:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.390+0000 7f26c5f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c006ae40 con 0x7f26c006a6a0 2023-10-06T05:28:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.390+0000 7f26c5f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c006b7e0 con 0x7f26c0155af0 2023-10-06T05:28:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26c579c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f26c00c7920 0x7f26c00c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54556/0 (socket says 172.21.15.33:54556) 2023-10-06T05:28:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26c579c700 1 -- 172.21.15.33:0/3801818417 learned_addr learned my addr 172.21.15.33:0/3801818417 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:15.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26c4f9b700 1 --2- 172.21.15.33:0/3801818417 >> v2:172.21.15.167:3300/0 conn(0x7f26c0155af0 0x7f26c006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:15.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26bf7fe700 1 -- 172.21.15.33:0/3801818417 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274531955 0 0) 0x7f26c006ab60 con 0x7f26c00c7920 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26bf7fe700 1 -- 172.21.15.33:0/3801818417 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b0003570 con 0x7f26c00c7920 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26c4f9b700 1 -- 172.21.15.33:0/3801818417 >> v1:172.21.15.167:6789/0 conn(0x7f26c006a6a0 legacy=0x7f26c00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26bf7fe700 1 -- 172.21.15.33:0/3801818417 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661621590 0 0) 0x7f26c006ae40 con 0x7f26c006a6a0 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.391+0000 7f26c4f9b700 1 -- 172.21.15.33:0/3801818417 >> v1:172.21.15.33:6789/0 conn(0x7f26c00c7920 legacy=0x7f26c00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.392+0000 7f26c4f9b700 1 -- 172.21.15.33:0/3801818417 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c006b120 con 0x7f26c0155af0 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.392+0000 7f26c4f9b700 1 --2- 172.21.15.33:0/3801818417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c0155af0 0x7f26c006a160 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f26b40098e0 tx=0x7f26b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69c225b0eb93765 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.392+0000 7f26bf7fe700 1 -- 172.21.15.33:0/3801818417 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011630 con 0x7f26c0155af0 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.392+0000 7f26bf7fe700 1 -- 172.21.15.33:0/3801818417 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b4011790 con 0x7f26c0155af0 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.392+0000 7f26bf7fe700 1 -- 172.21.15.33:0/3801818417 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011b30 con 0x7f26c0155af0 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 -- 172.21.15.33:0/3801818417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c0155af0 msgr2=0x7f26c006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/3801818417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c0155af0 0x7f26c006a160 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f26b40098e0 tx=0x7f26b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 -- 172.21.15.33:0/3801818417 shutdown_connections 2023-10-06T05:28:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/3801818417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c0155af0 0x7f26c006a160 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 -- 172.21.15.33:0/3801818417 >> 172.21.15.33:0/3801818417 conn(0x7f26c006d030 msgr2=0x7f26c006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 -- 172.21.15.33:0/3801818417 shutdown_connections 2023-10-06T05:28:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.393+0000 7f26c5f9d700 1 -- 172.21.15.33:0/3801818417 wait complete. 2023-10-06T05:28:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.394+0000 7f26c5f9d700 1 Processor -- start 2023-10-06T05:28:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 -- start start 2023-10-06T05:28:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 0x7f26c014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c00c7920 0x7f26c016afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26c0155af0 0x7f26c016b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00af6f0 con 0x7f26c006a6a0 2023-10-06T05:28:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00b1d70 con 0x7f26c00c7920 2023-10-06T05:28:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.395+0000 7f26c5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00af0e0 con 0x7f26c0155af0 2023-10-06T05:28:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 0x7f26c014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 0x7f26c014a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52048/0 (socket says 172.21.15.33:52048) 2023-10-06T05:28:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 -- 172.21.15.33:0/2847180380 learned_addr learned my addr 172.21.15.33:0/2847180380 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c579c700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26c0155af0 0x7f26c016b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 -- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26c0155af0 msgr2=0x7f26c016b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26c0155af0 0x7f26c016b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26bffff700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c00c7920 0x7f26c016afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 -- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c00c7920 msgr2=0x7f26c016afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c00c7920 0x7f26c016afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 -- 172.21.15.33:0/2847180380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4009420 con 0x7f26c006a6a0 2023-10-06T05:28:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c579c700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26c0155af0 0x7f26c016b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26bffff700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c00c7920 0x7f26c016afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.396+0000 7f26c4f9b700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 0x7f26c014a6b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f26b40091e0 tx=0x7f26b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.397+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011b10 con 0x7f26c006a6a0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.397+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b401a450 con 0x7f26c006a6a0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.397+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b401a7f0 con 0x7f26c006a6a0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.397+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c00b1a90 con 0x7f26c006a6a0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.397+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c00b16b0 con 0x7f26c006a6a0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.398+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c004c380 con 0x7f26c006a6a0 2023-10-06T05:28:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.398+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26b401ab60 con 0x7f26c006a6a0 2023-10-06T05:28:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.399+0000 7f26bcff9700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26a4066f30 0x7f26a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.399+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f26b40866d0 con 0x7f26c006a6a0 2023-10-06T05:28:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.400+0000 7f26bffff700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26a4066f30 0x7f26a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.401+0000 7f26bffff700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26a4066f30 0x7f26a40693e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0011e0 tx=0x7f26ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.404+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26b4053670 con 0x7f26c006a6a0 2023-10-06T05:28:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.651+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26c004c380 con 0x7f26c006a6a0 2023-10-06T05:28:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.651+0000 7f26bcff9700 1 -- 172.21.15.33:0/2847180380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f26b4056c90 con 0x7f26c006a6a0 2023-10-06T05:28:15.653 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:15.653 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26a4066f30 msgr2=0x7f26a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26a4066f30 0x7f26a40693e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0011e0 tx=0x7f26ac009040 comp rx=0 tx=0).stop 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 msgr2=0x7f26c014a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 0x7f26c014a6b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f26b40091e0 tx=0x7f26b4016000 comp rx=0 tx=0).stop 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 shutdown_connections 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26a4066f30 0x7f26a40693e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26c0155af0 0x7f26c016b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26c006a6a0 0x7f26c014a6b0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 --2- 172.21.15.33:0/2847180380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26c00c7920 0x7f26c016afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 >> 172.21.15.33:0/2847180380 conn(0x7f26c006d030 msgr2=0x7f26c00cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 shutdown_connections 2023-10-06T05:28:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:15.654+0000 7f26c5f9d700 1 -- 172.21.15.33:0/2847180380 wait complete. 2023-10-06T05:28:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:15.673 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:16.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.871+0000 7f43effff700 1 Processor -- start 2023-10-06T05:28:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.872+0000 7f43effff700 1 -- start start 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43effff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43f01572d0 0x7f43f00abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00b0f10 0x7f43f00cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43effff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f43f00cc4e0 0x7f43f00ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43effff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f00ad020 con 0x7f43f01572d0 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43effff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f00cc920 con 0x7f43f00cc4e0 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00ac410 con 0x7f43f00b0f10 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43eeffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43f01572d0 0x7f43f00abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43eeffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43f01572d0 0x7f43f00abe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52056/0 (socket says 172.21.15.33:52056) 2023-10-06T05:28:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43eeffd700 1 -- 172.21.15.33:0/275736172 learned_addr learned my addr 172.21.15.33:0/275736172 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43ef7fe700 1 --2- 172.21.15.33:0/275736172 >> v2:172.21.15.167:3300/0 conn(0x7f43f00cc4e0 0x7f43f00ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.873+0000 7f43ee7fc700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00b0f10 0x7f43f00cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43eeffd700 1 -- 172.21.15.33:0/275736172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00b0f10 msgr2=0x7f43f00cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43eeffd700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00b0f10 0x7f43f00cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43eeffd700 1 -- 172.21.15.33:0/275736172 >> v2:172.21.15.167:3300/0 conn(0x7f43f00cc4e0 msgr2=0x7f43f00ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43eeffd700 1 --2- 172.21.15.33:0/275736172 >> v2:172.21.15.167:3300/0 conn(0x7f43f00cc4e0 0x7f43f00ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43eeffd700 1 -- 172.21.15.33:0/275736172 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00ac6f0 con 0x7f43f01572d0 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43ef7fe700 1 --2- 172.21.15.33:0/275736172 >> v2:172.21.15.167:3300/0 conn(0x7f43f00cc4e0 0x7f43f00ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43ee7fc700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00b0f10 0x7f43f00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:16.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43eeffd700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f01572d0 0x7f43f00abe30 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f43f0069950 tx=0x7f43e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8f48e207ba1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:16.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43edffb700 1 -- 172.21.15.33:0/275736172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000b860 con 0x7f43f01572d0 2023-10-06T05:28:16.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43edffb700 1 -- 172.21.15.33:0/275736172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000b9c0 con 0x7f43f01572d0 2023-10-06T05:28:16.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.874+0000 7f43edffb700 1 -- 172.21.15.33:0/275736172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000bd60 con 0x7f43f01572d0 2023-10-06T05:28:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 -- 172.21.15.33:0/275736172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f01572d0 msgr2=0x7f43f00abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f01572d0 0x7f43f00abe30 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f43f0069950 tx=0x7f43e0009250 comp rx=0 tx=0).stop 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 -- 172.21.15.33:0/275736172 shutdown_connections 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 --2- 172.21.15.33:0/275736172 >> v2:172.21.15.167:3300/0 conn(0x7f43f00cc4e0 0x7f43f00ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f01572d0 0x7f43f00abe30 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 --2- 172.21.15.33:0/275736172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00b0f10 0x7f43f00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 -- 172.21.15.33:0/275736172 >> 172.21.15.33:0/275736172 conn(0x7f43f006c560 msgr2=0x7f43f006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 -- 172.21.15.33:0/275736172 shutdown_connections 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.875+0000 7f43effff700 1 -- 172.21.15.33:0/275736172 wait complete. 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.876+0000 7f43effff700 1 Processor -- start 2023-10-06T05:28:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 -- start start 2023-10-06T05:28:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 0x7f43f0152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00cc4e0 0x7f43f00a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f01572d0 0x7f43f01548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0154420 con 0x7f43f00b0f10 2023-10-06T05:28:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0153810 con 0x7f43f01572d0 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.877+0000 7f43effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0153af0 con 0x7f43f00cc4e0 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 0x7f43f0152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 0x7f43f0152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52066/0 (socket says 172.21.15.33:52066) 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 -- 172.21.15.33:0/707122951 learned_addr learned my addr 172.21.15.33:0/707122951 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43ee7fc700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00cc4e0 0x7f43f00a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43ef7fe700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f01572d0 0x7f43f01548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 -- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00cc4e0 msgr2=0x7f43f00a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00cc4e0 0x7f43f00a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 -- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f01572d0 msgr2=0x7f43f01548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f01572d0 0x7f43f01548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 -- 172.21.15.33:0/707122951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00094f0 con 0x7f43f00b0f10 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43ef7fe700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f01572d0 0x7f43f01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.878+0000 7f43eeffd700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 0x7f43f0152de0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009290 tx=0x7f43e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.879+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0026480 con 0x7f43f00b0f10 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.879+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e001f070 con 0x7f43f00b0f10 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.879+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f0153dd0 con 0x7f43f00b0f10 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.879+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0026830 con 0x7f43f00b0f10 2023-10-06T05:28:16.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.879+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f0153320 con 0x7f43f00b0f10 2023-10-06T05:28:16.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.880+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f004c380 con 0x7f43f00b0f10 2023-10-06T05:28:16.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.881+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43e0026ba0 con 0x7f43f00b0f10 2023-10-06T05:28:16.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.881+0000 7f43d6ffd700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.882+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f43e0021070 con 0x7f43f00b0f10 2023-10-06T05:28:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.882+0000 7f43ee7fc700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.883+0000 7f43ee7fc700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006f00 tx=0x7f43e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:16.886+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e0053650 con 0x7f43f00b0f10 2023-10-06T05:28:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.134+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43f0154700 con 0x7f43f00b0f10 2023-10-06T05:28:17.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.134+0000 7f43d6ffd700 1 -- 172.21.15.33:0/707122951 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f43e0056c70 con 0x7f43f00b0f10 2023-10-06T05:28:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:17.137 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.136+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43dc066f30 msgr2=0x7f43dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.136+0000 7f43effff700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006f00 tx=0x7f43e4008040 comp rx=0 tx=0).stop 2023-10-06T05:28:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.136+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 msgr2=0x7f43f0152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.136+0000 7f43effff700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 0x7f43f0152de0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009290 tx=0x7f43e0017000 comp rx=0 tx=0).stop 2023-10-06T05:28:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 shutdown_connections 2023-10-06T05:28:17.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f01572d0 0x7f43f01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43f00b0f10 0x7f43f0152de0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 --2- 172.21.15.33:0/707122951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43f00cc4e0 0x7f43f00a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 >> 172.21.15.33:0/707122951 conn(0x7f43f006c560 msgr2=0x7f43f00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 shutdown_connections 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:17.137+0000 7f43effff700 1 -- 172.21.15.33:0/707122951 wait complete. 2023-10-06T05:28:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:17.157 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:18.158 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:18.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.356+0000 7feb08b9d700 1 Processor -- start 2023-10-06T05:28:18.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.356+0000 7feb08b9d700 1 -- start start 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.357+0000 7feb08b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04155140 0x7feb04158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.357+0000 7feb08b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb040699c0 0x7feb04069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb08b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040c8e40 con 0x7feb040adf50 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb08b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0406a380 con 0x7feb040699c0 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb08b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040c99e0 con 0x7feb04155140 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb040699c0 0x7feb04069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb040699c0 0x7feb04069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52076/0 (socket says 172.21.15.33:52076) 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 -- 172.21.15.33:0/4118632765 learned_addr learned my addr 172.21.15.33:0/4118632765 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb037fe700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04155140 0x7feb04158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 -- 172.21.15.33:0/4118632765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04155140 msgr2=0x7feb04158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04155140 0x7feb04158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 -- 172.21.15.33:0/4118632765 >> v1:172.21.15.167:6789/0 conn(0x7feb040adf50 legacy=0x7feb040c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb03fff700 1 -- 172.21.15.33:0/4118632765 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb040ae6c0 con 0x7feb040699c0 2023-10-06T05:28:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.358+0000 7feb037fe700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04155140 0x7feb04158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.359+0000 7feb03fff700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb040699c0 0x7feb04069e40 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7feafc009f20 tx=0x7feafc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81b1cd7579c0af4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.359+0000 7feb027fc700 1 -- 172.21.15.33:0/4118632765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00e070 con 0x7feb040699c0 2023-10-06T05:28:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.359+0000 7feb027fc700 1 -- 172.21.15.33:0/4118632765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00ba80 con 0x7feb040699c0 2023-10-06T05:28:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.359+0000 7feb027fc700 1 -- 172.21.15.33:0/4118632765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0092e0 con 0x7feb040699c0 2023-10-06T05:28:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.359+0000 7feb08b9d700 1 -- 172.21.15.33:0/4118632765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb040699c0 msgr2=0x7feb04069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.359+0000 7feb08b9d700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb040699c0 0x7feb04069e40 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7feafc009f20 tx=0x7feafc009840 comp rx=0 tx=0).stop 2023-10-06T05:28:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.360+0000 7feb08b9d700 1 -- 172.21.15.33:0/4118632765 shutdown_connections 2023-10-06T05:28:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.360+0000 7feb08b9d700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb040699c0 0x7feb04069e40 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.360+0000 7feb08b9d700 1 --2- 172.21.15.33:0/4118632765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb04155140 0x7feb04158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.360+0000 7feb08b9d700 1 -- 172.21.15.33:0/4118632765 >> 172.21.15.33:0/4118632765 conn(0x7feb0406d030 msgr2=0x7feb0406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.360+0000 7feb08b9d700 1 -- 172.21.15.33:0/4118632765 shutdown_connections 2023-10-06T05:28:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.360+0000 7feb08b9d700 1 -- 172.21.15.33:0/4118632765 wait complete. 2023-10-06T05:28:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.361+0000 7feb08b9d700 1 Processor -- start 2023-10-06T05:28:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.361+0000 7feb08b9d700 1 -- start start 2023-10-06T05:28:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb08b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb040699c0 0x7feb04152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb08b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb040adf50 0x7feb040c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb08b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 0x7feb040c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb08b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04153520 con 0x7feb04155140 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb08b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb040c66d0 con 0x7feb040699c0 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb08b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb04152fa0 con 0x7feb040adf50 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 0x7feb040c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 0x7feb040c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52088/0 (socket says 172.21.15.33:52088) 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb03fff700 1 -- 172.21.15.33:0/716361266 learned_addr learned my addr 172.21.15.33:0/716361266 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.362+0000 7feb02ffd700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb040adf50 0x7feb040c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb037fe700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb040699c0 0x7feb04152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb03fff700 1 -- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb040adf50 msgr2=0x7feb040c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb03fff700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb040adf50 0x7feb040c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb03fff700 1 -- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb040699c0 msgr2=0x7feb04152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb03fff700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb040699c0 0x7feb04152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb03fff700 1 -- 172.21.15.33:0/716361266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009ae0 con 0x7feb04155140 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb02ffd700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb040adf50 0x7feb040c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb037fe700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb040699c0 0x7feb04152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb03fff700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 0x7feb040c5420 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7feb04063610 tx=0x7feafc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc008e80 con 0x7feb04155140 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc01c410 con 0x7feb04155140 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb040c5ac0 con 0x7feb04155140 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc01c7b0 con 0x7feb04155140 2023-10-06T05:28:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.363+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb040c6080 con 0x7feb04155140 2023-10-06T05:28:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.364+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb040c6840 con 0x7feb04155140 2023-10-06T05:28:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.365+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feafc003040 con 0x7feb04155140 2023-10-06T05:28:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.366+0000 7feaeb7fe700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feaf0066f30 0x7feaf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.366+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7feafc0862b0 con 0x7feb04155140 2023-10-06T05:28:18.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.367+0000 7feb037fe700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feaf0066f30 0x7feaf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:18.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.367+0000 7feb037fe700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feaf0066f30 0x7feaf00693e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7feaf400ac10 tx=0x7feaf4006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.370+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feafc053250 con 0x7feb04155140 2023-10-06T05:28:18.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.619+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb0404c380 con 0x7feb04155140 2023-10-06T05:28:18.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.619+0000 7feaeb7fe700 1 -- 172.21.15.33:0/716361266 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7feafc056870 con 0x7feb04155140 2023-10-06T05:28:18.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:18.622 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feaf0066f30 msgr2=0x7feaf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feaf0066f30 0x7feaf00693e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7feaf400ac10 tx=0x7feaf4006f90 comp rx=0 tx=0).stop 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 msgr2=0x7feb040c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 0x7feb040c5420 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7feb04063610 tx=0x7feafc018000 comp rx=0 tx=0).stop 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 shutdown_connections 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feaf0066f30 0x7feaf00693e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb04155140 0x7feb040c5420 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb040699c0 0x7feb04152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 --2- 172.21.15.33:0/716361266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb040adf50 0x7feb040c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 >> 172.21.15.33:0/716361266 conn(0x7feb0406d030 msgr2=0x7feb040c1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 shutdown_connections 2023-10-06T05:28:18.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:18.622+0000 7feb08b9d700 1 -- 172.21.15.33:0/716361266 wait complete. 2023-10-06T05:28:18.626 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:18.642 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:19.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.840+0000 7faf51d9d700 1 Processor -- start 2023-10-06T05:28:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.840+0000 7faf51d9d700 1 -- start start 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf51d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf4c147390 0x7faf4c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf51d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c0c7040 0x7faf4c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf51d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c0cb5f0 con 0x7faf4c147760 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf51d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0ca9e0 con 0x7faf4c147390 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf51d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0cacc0 con 0x7faf4c0c7040 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf4c147390 0x7faf4c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf4c147390 0x7faf4c0cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52092/0 (socket says 172.21.15.33:52092) 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 -- 172.21.15.33:0/3972549644 learned_addr learned my addr 172.21.15.33:0/3972549644 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf5159c700 1 --2- 172.21.15.33:0/3972549644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c0c7040 0x7faf4c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 -- 172.21.15.33:0/3972549644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c0c7040 msgr2=0x7faf4c0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 --2- 172.21.15.33:0/3972549644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c0c7040 0x7faf4c0c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 -- 172.21.15.33:0/3972549644 >> v1:172.21.15.167:6789/0 conn(0x7faf4c147760 legacy=0x7faf4c0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.842+0000 7faf50d9b700 1 -- 172.21.15.33:0/3972549644 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c0cafa0 con 0x7faf4c147390 2023-10-06T05:28:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.843+0000 7faf50d9b700 1 --2- 172.21.15.33:0/3972549644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c147390 0x7faf4c0cba80 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7faf400099b0 tx=0x7faf40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e3a4e1192f877e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.843+0000 7faf4b7fe700 1 -- 172.21.15.33:0/3972549644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4000b860 con 0x7faf4c147390 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.843+0000 7faf4b7fe700 1 -- 172.21.15.33:0/3972549644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4000b9c0 con 0x7faf4c147390 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.843+0000 7faf4b7fe700 1 -- 172.21.15.33:0/3972549644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4000bd60 con 0x7faf4c147390 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.843+0000 7faf51d9d700 1 -- 172.21.15.33:0/3972549644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c147390 msgr2=0x7faf4c0cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.843+0000 7faf51d9d700 1 --2- 172.21.15.33:0/3972549644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c147390 0x7faf4c0cba80 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7faf400099b0 tx=0x7faf40009250 comp rx=0 tx=0).stop 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.844+0000 7faf51d9d700 1 -- 172.21.15.33:0/3972549644 shutdown_connections 2023-10-06T05:28:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.844+0000 7faf51d9d700 1 --2- 172.21.15.33:0/3972549644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c147390 0x7faf4c0cba80 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.844+0000 7faf51d9d700 1 --2- 172.21.15.33:0/3972549644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c0c7040 0x7faf4c0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.844+0000 7faf51d9d700 1 -- 172.21.15.33:0/3972549644 >> 172.21.15.33:0/3972549644 conn(0x7faf4c06d030 msgr2=0x7faf4c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.844+0000 7faf51d9d700 1 -- 172.21.15.33:0/3972549644 shutdown_connections 2023-10-06T05:28:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.844+0000 7faf51d9d700 1 -- 172.21.15.33:0/3972549644 wait complete. 2023-10-06T05:28:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.845+0000 7faf51d9d700 1 Processor -- start 2023-10-06T05:28:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.845+0000 7faf51d9d700 1 -- start start 2023-10-06T05:28:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf51d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 0x7faf4c161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf51d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf4c147390 0x7faf4c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf51d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c147760 0x7faf4c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:19.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf51d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c16ee20 con 0x7faf4c0c7040 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf51d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c16f100 con 0x7faf4c147390 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf51d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c16f410 con 0x7faf4c147760 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf50d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 0x7faf4c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf50d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 0x7faf4c161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52096/0 (socket says 172.21.15.33:52096) 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.846+0000 7faf50d9b700 1 -- 172.21.15.33:0/2912642204 learned_addr learned my addr 172.21.15.33:0/2912642204 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf5159c700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c147760 0x7faf4c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf4bfff700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf4c147390 0x7faf4c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf50d9b700 1 -- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c147760 msgr2=0x7faf4c0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf50d9b700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c147760 0x7faf4c0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf50d9b700 1 -- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf4c147390 msgr2=0x7faf4c0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf50d9b700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf4c147390 0x7faf4c0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf50d9b700 1 -- 172.21.15.33:0/2912642204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400094f0 con 0x7faf4c0c7040 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf50d9b700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 0x7faf4c161880 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7faf4c063610 tx=0x7faf40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40024440 con 0x7faf4c0c7040 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf400245a0 con 0x7faf4c0c7040 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40024940 con 0x7faf4c0c7040 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.847+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c16f6f0 con 0x7faf4c0c7040 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.848+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c0b07a0 con 0x7faf4c0c7040 2023-10-06T05:28:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.848+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c04c380 con 0x7faf4c0c7040 2023-10-06T05:28:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.850+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf4001a420 con 0x7faf4c0c7040 2023-10-06T05:28:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.850+0000 7faf48ff9700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf30066f30 0x7faf300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.851+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7faf400865f0 con 0x7faf4c0c7040 2023-10-06T05:28:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.851+0000 7faf4bfff700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf30066f30 0x7faf300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.851+0000 7faf4bfff700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf30066f30 0x7faf300693e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0a5530 tx=0x7faf38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:19.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:19.857+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf40053610 con 0x7faf4c0c7040 2023-10-06T05:28:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.108+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf4c16ef60 con 0x7faf4c0c7040 2023-10-06T05:28:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.108+0000 7faf48ff9700 1 -- 172.21.15.33:0/2912642204 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7faf4c16ef60 con 0x7faf4c0c7040 2023-10-06T05:28:20.111 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:20.112 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:20.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf30066f30 msgr2=0x7faf300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:20.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf30066f30 0x7faf300693e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0a5530 tx=0x7faf38009040 comp rx=0 tx=0).stop 2023-10-06T05:28:20.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 msgr2=0x7faf4c161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:20.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 0x7faf4c161880 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7faf4c063610 tx=0x7faf40017000 comp rx=0 tx=0).stop 2023-10-06T05:28:20.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 shutdown_connections 2023-10-06T05:28:20.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf30066f30 0x7faf300693e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:20.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf4c147760 0x7faf4c0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:20.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c0c7040 0x7faf4c161880 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:20.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 --2- 172.21.15.33:0/2912642204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf4c147390 0x7faf4c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:20.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 >> 172.21.15.33:0/2912642204 conn(0x7faf4c06d030 msgr2=0x7faf4c1464f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:20.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 shutdown_connections 2023-10-06T05:28:20.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:20.111+0000 7faf51d9d700 1 -- 172.21.15.33:0/2912642204 wait complete. 2023-10-06T05:28:20.117 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:20.131 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:20.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:20.221 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:21.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.329+0000 7f0fea59d700 1 Processor -- start 2023-10-06T05:28:21.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.329+0000 7f0fea59d700 1 -- start start 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fea59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fe413faa0 0x7f0fe413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fea59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe40c7940 0x7f0fe40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fea59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe40c9cb0 con 0x7f0fe406aad0 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fea59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe406b640 con 0x7f0fe40c7940 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fea59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40ca270 con 0x7f0fe413faa0 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fe8d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0fe40c7940 0x7f0fe40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:21.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fe9d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0fe406aad0 0x7f0fe406af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58266/0 (socket says 172.21.15.33:58266) 2023-10-06T05:28:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.330+0000 7f0fe9d9c700 1 -- 172.21.15.33:0/3525881204 learned_addr learned my addr 172.21.15.33:0/3525881204 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fe959b700 1 --2- 172.21.15.33:0/3525881204 >> v2:172.21.15.167:3300/0 conn(0x7f0fe413faa0 0x7f0fe413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/3525881204 >> v2:172.21.15.167:3300/0 conn(0x7f0fe413faa0 msgr2=0x7f0fe413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fe8d9a700 1 --2- 172.21.15.33:0/3525881204 >> v2:172.21.15.167:3300/0 conn(0x7f0fe413faa0 0x7f0fe413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/3525881204 >> v1:172.21.15.33:6789/0 conn(0x7f0fe406aad0 legacy=0x7f0fe406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/3525881204 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe406b840 con 0x7f0fe40c7940 2023-10-06T05:28:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fdbfff700 1 -- 172.21.15.33:0/3525881204 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132960752 0 0) 0x7f0fe40c9cb0 con 0x7f0fe406aad0 2023-10-06T05:28:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fe8d9a700 1 --2- 172.21.15.33:0/3525881204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe40c7db0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f0fe406fe50 tx=0x7f0fd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6caf1906846d471b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fdbfff700 1 -- 172.21.15.33:0/3525881204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4011630 con 0x7f0fe40c7940 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fdbfff700 1 -- 172.21.15.33:0/3525881204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd4011790 con 0x7f0fe40c7940 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.331+0000 7f0fdbfff700 1 -- 172.21.15.33:0/3525881204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4011b30 con 0x7f0fe40c7940 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 -- 172.21.15.33:0/3525881204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 msgr2=0x7f0fe40c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 --2- 172.21.15.33:0/3525881204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe40c7db0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f0fe406fe50 tx=0x7f0fd40091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 -- 172.21.15.33:0/3525881204 shutdown_connections 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 --2- 172.21.15.33:0/3525881204 >> v2:172.21.15.167:3300/0 conn(0x7f0fe413faa0 0x7f0fe413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 --2- 172.21.15.33:0/3525881204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe40c7db0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 -- 172.21.15.33:0/3525881204 >> 172.21.15.33:0/3525881204 conn(0x7f0fe406d030 msgr2=0x7f0fe406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 -- 172.21.15.33:0/3525881204 shutdown_connections 2023-10-06T05:28:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.332+0000 7f0fea59d700 1 -- 172.21.15.33:0/3525881204 wait complete. 2023-10-06T05:28:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.334+0000 7f0fea59d700 1 Processor -- start 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.334+0000 7f0fea59d700 1 -- start start 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fea59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe406aad0 0x7f0fe40c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fea59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe416af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fea59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe413faa0 0x7f0fe416b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fea59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415cf40 con 0x7f0fe40c7940 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fea59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415d220 con 0x7f0fe406aad0 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fea59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415d500 con 0x7f0fe413faa0 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe416af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe416af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56250/0 (socket says 172.21.15.33:56250) 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/2367746352 learned_addr learned my addr 172.21.15.33:0/2367746352 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe959b700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe406aad0 0x7f0fe40c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe9d9c700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe413faa0 0x7f0fe416b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe413faa0 msgr2=0x7f0fe416b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe413faa0 0x7f0fe416b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe406aad0 msgr2=0x7f0fe40c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe406aad0 0x7f0fe40c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.335+0000 7f0fe8d9a700 1 -- 172.21.15.33:0/2367746352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009420 con 0x7f0fe40c7940 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.336+0000 7f0fe8d9a700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe416af70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40719c0 tx=0x7f0fd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.336+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4011b10 con 0x7f0fe40c7940 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.336+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd40253f0 con 0x7f0fe40c7940 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.336+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4025790 con 0x7f0fe40c7940 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.336+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe415d7e0 con 0x7f0fe40c7940 2023-10-06T05:28:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.336+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe40aea00 con 0x7f0fe40c7940 2023-10-06T05:28:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.337+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe415dce0 con 0x7f0fe40c7940 2023-10-06T05:28:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.338+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fd4025b80 con 0x7f0fe40c7940 2023-10-06T05:28:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.339+0000 7f0fd97fa700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc8066f30 0x7f0fc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:21.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.339+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f0fd4085fe0 con 0x7f0fe40c7940 2023-10-06T05:28:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.339+0000 7f0fe959b700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc8066f30 0x7f0fc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:21.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.340+0000 7f0fe959b700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc8066f30 0x7f0fc80693e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009c80 tx=0x7f0fe00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.343+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fd4052fc0 con 0x7f0fe40c7940 2023-10-06T05:28:21.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.594+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe404c380 con 0x7f0fe40c7940 2023-10-06T05:28:21.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.594+0000 7f0fd97fa700 1 -- 172.21.15.33:0/2367746352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f0fd40565e0 con 0x7f0fe40c7940 2023-10-06T05:28:21.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:21.597 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc8066f30 msgr2=0x7f0fc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc8066f30 0x7f0fc80693e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009c80 tx=0x7f0fe00091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 msgr2=0x7f0fe416af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe416af70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40719c0 tx=0x7f0fd4017000 comp rx=0 tx=0).stop 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 shutdown_connections 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0fc8066f30 0x7f0fc80693e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe413faa0 0x7f0fe416b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe406aad0 0x7f0fe40c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 --2- 172.21.15.33:0/2367746352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c7940 0x7f0fe416af70 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 >> 172.21.15.33:0/2367746352 conn(0x7f0fe406d030 msgr2=0x7f0fe40c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 shutdown_connections 2023-10-06T05:28:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:21.597+0000 7f0fea59d700 1 -- 172.21.15.33:0/2367746352 wait complete. 2023-10-06T05:28:21.601 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:21.617 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:22.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.814+0000 7f4ee759d700 1 Processor -- start 2023-10-06T05:28:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.814+0000 7f4ee759d700 1 -- start start 2023-10-06T05:28:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee759d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ee00b0a40 0x7f4ee006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee759d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ee00b0e10 0x7f4ee00b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee759d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b2e30 0x7f4ee015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee759d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee015e2d0 con 0x7f4ee00b0e10 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee759d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee015d6c0 con 0x7f4ee00b0a40 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee759d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015d9a0 con 0x7f4ee00b2e30 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ee00b0e10 0x7f4ee00b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4ee00b0e10 0x7f4ee00b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56256/0 (socket says 172.21.15.33:56256) 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 -- 172.21.15.33:0/1268532016 learned_addr learned my addr 172.21.15.33:0/1268532016 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee6d9c700 1 --2- 172.21.15.33:0/1268532016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b2e30 0x7f4ee015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee659b700 1 --2- 172.21.15.33:0/1268532016 >> v2:172.21.15.167:3300/0 conn(0x7f4ee00b0a40 0x7f4ee006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 -- 172.21.15.33:0/1268532016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b2e30 msgr2=0x7f4ee015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 --2- 172.21.15.33:0/1268532016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b2e30 0x7f4ee015a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 -- 172.21.15.33:0/1268532016 >> v2:172.21.15.167:3300/0 conn(0x7f4ee00b0a40 msgr2=0x7f4ee006ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 --2- 172.21.15.33:0/1268532016 >> v2:172.21.15.167:3300/0 conn(0x7f4ee00b0a40 0x7f4ee006ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.816+0000 7f4ee5d9a700 1 -- 172.21.15.33:0/1268532016 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee015dc80 con 0x7f4ee00b0e10 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.817+0000 7f4ee5d9a700 1 --2- 172.21.15.33:0/1268532016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0e10 0x7f4ee00b28f0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006a230 tx=0x7f4ed0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9455099be481feca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.817+0000 7f4ee5599700 1 -- 172.21.15.33:0/1268532016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000b860 con 0x7f4ee00b0e10 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.817+0000 7f4ee5599700 1 -- 172.21.15.33:0/1268532016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed000b9c0 con 0x7f4ee00b0e10 2023-10-06T05:28:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.817+0000 7f4ee5599700 1 -- 172.21.15.33:0/1268532016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000bd60 con 0x7f4ee00b0e10 2023-10-06T05:28:22.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.817+0000 7f4ee759d700 1 -- 172.21.15.33:0/1268532016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0e10 msgr2=0x7f4ee00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:22.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.817+0000 7f4ee759d700 1 --2- 172.21.15.33:0/1268532016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0e10 0x7f4ee00b28f0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006a230 tx=0x7f4ed0009250 comp rx=0 tx=0).stop 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 -- 172.21.15.33:0/1268532016 shutdown_connections 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 --2- 172.21.15.33:0/1268532016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b2e30 0x7f4ee015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 --2- 172.21.15.33:0/1268532016 >> v2:172.21.15.167:3300/0 conn(0x7f4ee00b0a40 0x7f4ee006ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 --2- 172.21.15.33:0/1268532016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0e10 0x7f4ee00b28f0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 -- 172.21.15.33:0/1268532016 >> 172.21.15.33:0/1268532016 conn(0x7f4ee006e0f0 msgr2=0x7f4ee00675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 -- 172.21.15.33:0/1268532016 shutdown_connections 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.818+0000 7f4ee759d700 1 -- 172.21.15.33:0/1268532016 wait complete. 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.819+0000 7f4ee759d700 1 Processor -- start 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.819+0000 7f4ee759d700 1 -- start start 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.820+0000 7f4ee759d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 0x7f4ee015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.820+0000 7f4ee759d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b0e10 0x7f4ee00c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.820+0000 7f4ee759d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ee00b2e30 0x7f4ee016d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.820+0000 7f4ee759d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015b460 con 0x7f4ee00b0a40 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.820+0000 7f4ee759d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015b740 con 0x7f4ee00b2e30 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.820+0000 7f4ee759d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015ba20 con 0x7f4ee00b0e10 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 0x7f4ee015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 0x7f4ee015add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56260/0 (socket says 172.21.15.33:56260) 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 -- 172.21.15.33:0/2856830177 learned_addr learned my addr 172.21.15.33:0/2856830177 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 -- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b0e10 msgr2=0x7f4ee00c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b0e10 0x7f4ee00c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 -- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ee00b2e30 msgr2=0x7f4ee016d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ee00b2e30 0x7f4ee016d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 -- 172.21.15.33:0/2856830177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00094f0 con 0x7f4ee00b0a40 2023-10-06T05:28:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ee659b700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 0x7f4ee015add0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d890 tx=0x7f4edc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:22.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00b600 con 0x7f4ee00b0a40 2023-10-06T05:28:22.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc00b760 con 0x7f4ee00b0a40 2023-10-06T05:28:22.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.821+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00bae0 con 0x7f4ee00b0a40 2023-10-06T05:28:22.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.822+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee015bd00 con 0x7f4ee00b0a40 2023-10-06T05:28:22.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.822+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee016dc50 con 0x7f4ee00b0a40 2023-10-06T05:28:22.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.823+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee016de70 con 0x7f4ee00b0a40 2023-10-06T05:28:22.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.824+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4edc01e460 con 0x7f4ee00b0a40 2023-10-06T05:28:22.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.824+0000 7f4ed67fc700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ec4066f30 0x7f4ec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:22.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.824+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f4edc084eb0 con 0x7f4ee00b0a40 2023-10-06T05:28:22.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.825+0000 7f4ee5d9a700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ec4066f30 0x7f4ec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:22.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.825+0000 7f4ee5d9a700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ec4066f30 0x7f4ec40693e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006a230 tx=0x7f4ed0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:22.829+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4edc051e50 con 0x7f4ee00b0a40 2023-10-06T05:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.075+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ee006fcc0 con 0x7f4ee00b0a40 2023-10-06T05:28:23.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.076+0000 7f4ed67fc700 1 -- 172.21.15.33:0/2856830177 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f4edc055470 con 0x7f4ee00b0a40 2023-10-06T05:28:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:23.079 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ec4066f30 msgr2=0x7f4ec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ec4066f30 0x7f4ec40693e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006a230 tx=0x7f4ed0017000 comp rx=0 tx=0).stop 2023-10-06T05:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 msgr2=0x7f4ee015add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 0x7f4ee015add0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d890 tx=0x7f4edc008040 comp rx=0 tx=0).stop 2023-10-06T05:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 shutdown_connections 2023-10-06T05:28:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4ec4066f30 0x7f4ec40693e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ee00b2e30 0x7f4ee016d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee00b0a40 0x7f4ee015add0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 --2- 172.21.15.33:0/2856830177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ee00b0e10 0x7f4ee00c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 >> 172.21.15.33:0/2856830177 conn(0x7f4ee006e0f0 msgr2=0x7f4ee00abac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 shutdown_connections 2023-10-06T05:28:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:23.078+0000 7f4ee759d700 1 -- 172.21.15.33:0/2856830177 wait complete. 2023-10-06T05:28:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:23.098 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:24.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.295+0000 7f1da979d700 1 Processor -- start 2023-10-06T05:28:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.295+0000 7f1da979d700 1 -- start start 2023-10-06T05:28:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da979d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1da4155af0 0x7f1da406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da979d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da406ab60 con 0x7f1da40c7920 2023-10-06T05:28:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da406ae40 con 0x7f1da406a6a0 2023-10-06T05:28:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da979d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da406b7e0 con 0x7f1da4155af0 2023-10-06T05:28:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da8f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1da40c7920 0x7f1da40c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58274/0 (socket says 172.21.15.33:58274) 2023-10-06T05:28:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da8f9c700 1 -- 172.21.15.33:0/1569111476 learned_addr learned my addr 172.21.15.33:0/1569111476 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1569111476 >> v2:172.21.15.167:3300/0 conn(0x7f1da4155af0 0x7f1da406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631103069 0 0) 0x7f1da406ab60 con 0x7f1da40c7920 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.297+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d8c004680 con 0x7f1da40c7920 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624112226 0 0) 0x7f1da406ae40 con 0x7f1da406a6a0 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da406ab60 con 0x7f1da406a6a0 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1854994632 0 0) 0x7f1d8c004680 con 0x7f1da40c7920 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 >> v2:172.21.15.167:3300/0 conn(0x7f1da4155af0 msgr2=0x7f1da406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 --2- 172.21.15.33:0/1569111476 >> v2:172.21.15.167:3300/0 conn(0x7f1da4155af0 0x7f1da406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 >> v1:172.21.15.167:6789/0 conn(0x7f1da406a6a0 legacy=0x7f1da40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da406b120 con 0x7f1da40c7920 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1d9c0017f0 con 0x7f1da40c7920 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 >> v1:172.21.15.33:6789/0 conn(0x7f1da40c7920 legacy=0x7f1da40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1d8c005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d8c005920 0x7f1d8c005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d8c006320 0x7f1d8c00cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da406b7e0 con 0x7f1d8c004d50 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00d2c0 con 0x7f1d8c005920 2023-10-06T05:28:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c00d480 con 0x7f1d8c006320 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1d8c005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da37fe700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d8c005920 0x7f1d8c005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.298+0000 7f1da8f9c700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d8c006320 0x7f1d8c00cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da3fff700 1 -- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d8c006320 msgr2=0x7f1d8c00cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d8c006320 0x7f1d8c00cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da3fff700 1 -- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d8c005920 msgr2=0x7f1d8c005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d8c005920 0x7f1d8c005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da3fff700 1 -- 172.21.15.33:0/1569111476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c004680 con 0x7f1d8c004d50 2023-10-06T05:28:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da8f9c700 1 -- 172.21.15.33:0/1569111476 reap_dead start 2023-10-06T05:28:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1d8c005140 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006f00 tx=0x7f1d940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c139972c531a9b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400e710 con 0x7f1d8c004d50 2023-10-06T05:28:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9400e870 con 0x7f1d8c004d50 2023-10-06T05:28:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.299+0000 7f1da2ffd700 1 -- 172.21.15.33:0/1569111476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400ec30 con 0x7f1d8c004d50 2023-10-06T05:28:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 -- 172.21.15.33:0/1569111476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 msgr2=0x7f1d8c005140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1d8c005140 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006f00 tx=0x7f1d940091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 -- 172.21.15.33:0/1569111476 shutdown_connections 2023-10-06T05:28:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 --2- 172.21.15.33:0/1569111476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1d8c005140 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 -- 172.21.15.33:0/1569111476 >> 172.21.15.33:0/1569111476 conn(0x7f1da406d030 msgr2=0x7f1da406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 -- 172.21.15.33:0/1569111476 shutdown_connections 2023-10-06T05:28:24.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.300+0000 7f1da979d700 1 -- 172.21.15.33:0/1569111476 wait complete. 2023-10-06T05:28:24.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.301+0000 7f1da979d700 1 Processor -- start 2023-10-06T05:28:24.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.301+0000 7f1da979d700 1 -- start start 2023-10-06T05:28:24.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.302+0000 7f1da979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1da416a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.302+0000 7f1da979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da40abd40 0x7f1da416ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.302+0000 7f1da979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da40ac1b0 0x7f1da4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.302+0000 7f1da979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4156e90 con 0x7f1d8c004d50 2023-10-06T05:28:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.302+0000 7f1da979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4157170 con 0x7f1da40ac1b0 2023-10-06T05:28:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.302+0000 7f1da979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4157450 con 0x7f1da40abd40 2023-10-06T05:28:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1da416a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1da416a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56272/0 (socket says 172.21.15.33:56272) 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 -- 172.21.15.33:0/1775648843 learned_addr learned my addr 172.21.15.33:0/1775648843 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da8f9c700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da40ac1b0 0x7f1da4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da37fe700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da40abd40 0x7f1da416ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 -- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da40abd40 msgr2=0x7f1da416ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da40abd40 0x7f1da416ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 -- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da40ac1b0 msgr2=0x7f1da4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da40ac1b0 0x7f1da4157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 -- 172.21.15.33:0/1775648843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da4157730 con 0x7f1d8c004d50 2023-10-06T05:28:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.303+0000 7f1da3fff700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1da416a160 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400e4b0 tx=0x7f1d9401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.304+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94022070 con 0x7f1d8c004d50 2023-10-06T05:28:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.304+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94013af0 con 0x7f1d8c004d50 2023-10-06T05:28:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.304+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9401c450 con 0x7f1d8c004d50 2023-10-06T05:28:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.304+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da4157a10 con 0x7f1d8c004d50 2023-10-06T05:28:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.304+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da416a860 con 0x7f1d8c004d50 2023-10-06T05:28:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.305+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da404c380 con 0x7f1d8c004d50 2023-10-06T05:28:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.306+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d9401c880 con 0x7f1d8c004d50 2023-10-06T05:28:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.307+0000 7f1d8bfff700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d90066f30 0x7f1d900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.307+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f1d94085e40 con 0x7f1d8c004d50 2023-10-06T05:28:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.307+0000 7f1da37fe700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d90066f30 0x7f1d900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.308+0000 7f1da37fe700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d90066f30 0x7f1d900693e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007fd0 tx=0x7f1d98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.311+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d94052de0 con 0x7f1d8c004d50 2023-10-06T05:28:24.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.561+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1da406b400 con 0x7f1d8c004d50 2023-10-06T05:28:24.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.561+0000 7f1d8bfff700 1 -- 172.21.15.33:0/1775648843 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f1d94056400 con 0x7f1d8c004d50 2023-10-06T05:28:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:24.564 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:24.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d90066f30 msgr2=0x7f1d900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d90066f30 0x7f1d900693e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007fd0 tx=0x7f1d98009040 comp rx=0 tx=0).stop 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 msgr2=0x7f1da416a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1da416a160 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400e4b0 tx=0x7f1d9401a000 comp rx=0 tx=0).stop 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 shutdown_connections 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1d90066f30 0x7f1d900693e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da40ac1b0 0x7f1da4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d8c004d50 0x7f1da416a160 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 --2- 172.21.15.33:0/1775648843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da40abd40 0x7f1da416ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 >> 172.21.15.33:0/1775648843 conn(0x7f1da406d030 msgr2=0x7f1da40c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 shutdown_connections 2023-10-06T05:28:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:24.564+0000 7f1da979d700 1 -- 172.21.15.33:0/1775648843 wait complete. 2023-10-06T05:28:24.568 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:24.584 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:25.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:25.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.781+0000 7fb43319d700 1 Processor -- start 2023-10-06T05:28:25.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.781+0000 7fb43319d700 1 -- start start 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.782+0000 7fb43319d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb42c13faa0 0x7fb42c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43319d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb42c0c7940 0x7fb42c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42c0c9cb0 con 0x7fb42c06aad0 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43319d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42c06b640 con 0x7fb42c13faa0 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43319d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42c0ca270 con 0x7fb42c0c7940 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43219b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb42c13faa0 0x7fb42c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43299c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb42c06aad0 0x7fb42c06af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58276/0 (socket says 172.21.15.33:58276) 2023-10-06T05:28:25.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43299c700 1 -- 172.21.15.33:0/1858543517 learned_addr learned my addr 172.21.15.33:0/1858543517 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43199a700 1 --2- 172.21.15.33:0/1858543517 >> v2:172.21.15.167:3300/0 conn(0x7fb42c0c7940 0x7fb42c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb431199700 1 -- 172.21.15.33:0/1858543517 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932850061 0 0) 0x7fb42c0c9cb0 con 0x7fb42c06aad0 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43219b700 1 -- 172.21.15.33:0/1858543517 >> v2:172.21.15.167:3300/0 conn(0x7fb42c0c7940 msgr2=0x7fb42c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43219b700 1 --2- 172.21.15.33:0/1858543517 >> v2:172.21.15.167:3300/0 conn(0x7fb42c0c7940 0x7fb42c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43219b700 1 -- 172.21.15.33:0/1858543517 >> v1:172.21.15.33:6789/0 conn(0x7fb42c06aad0 legacy=0x7fb42c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.783+0000 7fb43219b700 1 -- 172.21.15.33:0/1858543517 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c06b840 con 0x7fb42c13faa0 2023-10-06T05:28:25.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43219b700 1 --2- 172.21.15.33:0/1858543517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c13faa0 0x7fb42c13fe70 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0cbef0 tx=0x7fb4280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123cae3a52b2a0c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:25.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb431199700 1 -- 172.21.15.33:0/1858543517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800f6b0 con 0x7fb42c13faa0 2023-10-06T05:28:25.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb431199700 1 -- 172.21.15.33:0/1858543517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42800f810 con 0x7fb42c13faa0 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb431199700 1 -- 172.21.15.33:0/1858543517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800fbb0 con 0x7fb42c13faa0 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 -- 172.21.15.33:0/1858543517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c13faa0 msgr2=0x7fb42c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 --2- 172.21.15.33:0/1858543517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c13faa0 0x7fb42c13fe70 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0cbef0 tx=0x7fb4280091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 -- 172.21.15.33:0/1858543517 shutdown_connections 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 --2- 172.21.15.33:0/1858543517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c13faa0 0x7fb42c13fe70 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 --2- 172.21.15.33:0/1858543517 >> v2:172.21.15.167:3300/0 conn(0x7fb42c0c7940 0x7fb42c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 -- 172.21.15.33:0/1858543517 >> 172.21.15.33:0/1858543517 conn(0x7fb42c06d030 msgr2=0x7fb42c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.784+0000 7fb43319d700 1 -- 172.21.15.33:0/1858543517 shutdown_connections 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.785+0000 7fb43319d700 1 -- 172.21.15.33:0/1858543517 wait complete. 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.786+0000 7fb43319d700 1 Processor -- start 2023-10-06T05:28:25.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.786+0000 7fb43319d700 1 -- start start 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06aad0 0x7fb42c13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 0x7fb42c169a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c13faa0 0x7fb42c0c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c159350 con 0x7fb42c0c7940 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c0a5890 con 0x7fb42c13faa0 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c158dc0 con 0x7fb42c06aad0 2023-10-06T05:28:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 0x7fb42c169a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 0x7fb42c169a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56286/0 (socket says 172.21.15.33:56286) 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.787+0000 7fb43199a700 1 -- 172.21.15.33:0/207989141 learned_addr learned my addr 172.21.15.33:0/207989141 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43219b700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06aad0 0x7fb42c13cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43299c700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c13faa0 0x7fb42c0c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43199a700 1 -- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06aad0 msgr2=0x7fb42c13cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43199a700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06aad0 0x7fb42c13cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43199a700 1 -- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c13faa0 msgr2=0x7fb42c0c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43199a700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c13faa0 0x7fb42c0c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43199a700 1 -- 172.21.15.33:0/207989141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb428009420 con 0x7fb42c0c7940 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43299c700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c13faa0 0x7fb42c0c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:25.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43199a700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 0x7fb42c169a00 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00d890 tx=0x7fb41c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00b600 con 0x7fb42c0c7940 2023-10-06T05:28:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c00b760 con 0x7fb42c0c7940 2023-10-06T05:28:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c158800 con 0x7fb42c0c7940 2023-10-06T05:28:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00bae0 con 0x7fb42c0c7940 2023-10-06T05:28:25.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.788+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c0a5520 con 0x7fb42c0c7940 2023-10-06T05:28:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.789+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c0a5a20 con 0x7fb42c0c7940 2023-10-06T05:28:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.790+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb41c01e460 con 0x7fb42c0c7940 2023-10-06T05:28:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.791+0000 7fb4227fc700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb410066f30 0x7fb4100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.791+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb41c084ec0 con 0x7fb42c0c7940 2023-10-06T05:28:25.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.792+0000 7fb43219b700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb410066f30 0x7fb4100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:25.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.792+0000 7fb43219b700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb410066f30 0x7fb4100693e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb4280093f0 tx=0x7fb428019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:25.795+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb41c051e60 con 0x7fb42c0c7940 2023-10-06T05:28:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.047+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb42c04c380 con 0x7fb42c0c7940 2023-10-06T05:28:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.048+0000 7fb4227fc700 1 -- 172.21.15.33:0/207989141 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fb41c055480 con 0x7fb42c0c7940 2023-10-06T05:28:26.051 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb410066f30 msgr2=0x7fb4100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb410066f30 0x7fb4100693e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb4280093f0 tx=0x7fb428019000 comp rx=0 tx=0).stop 2023-10-06T05:28:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 msgr2=0x7fb42c169a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 0x7fb42c169a00 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00d890 tx=0x7fb41c008040 comp rx=0 tx=0).stop 2023-10-06T05:28:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 shutdown_connections 2023-10-06T05:28:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb410066f30 0x7fb4100693e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c13faa0 0x7fb42c0c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb42c06aad0 0x7fb42c13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 --2- 172.21.15.33:0/207989141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb42c0c7940 0x7fb42c169a00 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 >> 172.21.15.33:0/207989141 conn(0x7fb42c06d030 msgr2=0x7fb42c0c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 shutdown_connections 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:26.050+0000 7fb43319d700 1 -- 172.21.15.33:0/207989141 wait complete. 2023-10-06T05:28:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:26.071 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:27.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:27.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.266+0000 7f64d179d700 1 Processor -- start 2023-10-06T05:28:27.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.266+0000 7f64d179d700 1 -- start start 2023-10-06T05:28:27.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64d179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64cc0b0a40 0x7f64cc06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64d179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f64cc15a0a0 0x7f64cc15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64d179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc15e2d0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64d179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc15d6c0 con 0x7f64cc0b0a40 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64d179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc15d9a0 con 0x7f64cc15a0a0 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64cbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64cc0b0a40 0x7f64cc06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64cbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f64cc0b0a40 0x7f64cc06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56290/0 (socket says 172.21.15.33:56290) 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64cbfff700 1 -- 172.21.15.33:0/1433778805 learned_addr learned my addr 172.21.15.33:0/1433778805 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64cbfff700 1 -- 172.21.15.33:0/1433778805 >> v2:172.21.15.167:3300/0 conn(0x7f64cc15a0a0 msgr2=0x7f64cc15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64d0f9c700 1 --2- 172.21.15.33:0/1433778805 >> v2:172.21.15.167:3300/0 conn(0x7f64cc15a0a0 0x7f64cc15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64cbfff700 1 --2- 172.21.15.33:0/1433778805 >> v2:172.21.15.167:3300/0 conn(0x7f64cc15a0a0 0x7f64cc15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.268+0000 7f64cbfff700 1 -- 172.21.15.33:0/1433778805 >> v1:172.21.15.167:6789/0 conn(0x7f64cc0b0e10 legacy=0x7f64cc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:27.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64cbfff700 1 -- 172.21.15.33:0/1433778805 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc15dc80 con 0x7f64cc0b0a40 2023-10-06T05:28:27.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64d0f9c700 1 --2- 172.21.15.33:0/1433778805 >> v2:172.21.15.167:3300/0 conn(0x7f64cc15a0a0 0x7f64cc15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:27.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64cbfff700 1 --2- 172.21.15.33:0/1433778805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0a40 0x7f64cc06ab00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0b0740 tx=0x7f64bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1a4f4a2030da33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:27.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64caffd700 1 -- 172.21.15.33:0/1433778805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00b820 con 0x7f64cc0b0a40 2023-10-06T05:28:27.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64caffd700 1 -- 172.21.15.33:0/1433778805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc00b980 con 0x7f64cc0b0a40 2023-10-06T05:28:27.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64caffd700 1 -- 172.21.15.33:0/1433778805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00bd20 con 0x7f64cc0b0a40 2023-10-06T05:28:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64d179d700 1 -- 172.21.15.33:0/1433778805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0a40 msgr2=0x7f64cc06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.269+0000 7f64d179d700 1 --2- 172.21.15.33:0/1433778805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0a40 0x7f64cc06ab00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0b0740 tx=0x7f64bc009250 comp rx=0 tx=0).stop 2023-10-06T05:28:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.270+0000 7f64d179d700 1 -- 172.21.15.33:0/1433778805 shutdown_connections 2023-10-06T05:28:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.270+0000 7f64d179d700 1 --2- 172.21.15.33:0/1433778805 >> v2:172.21.15.167:3300/0 conn(0x7f64cc15a0a0 0x7f64cc15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.270+0000 7f64d179d700 1 --2- 172.21.15.33:0/1433778805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0a40 0x7f64cc06ab00 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.270+0000 7f64d179d700 1 -- 172.21.15.33:0/1433778805 >> 172.21.15.33:0/1433778805 conn(0x7f64cc06e0f0 msgr2=0x7f64cc0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:27.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.270+0000 7f64d179d700 1 -- 172.21.15.33:0/1433778805 shutdown_connections 2023-10-06T05:28:27.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.270+0000 7f64d179d700 1 -- 172.21.15.33:0/1433778805 wait complete. 2023-10-06T05:28:27.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.271+0000 7f64d179d700 1 Processor -- start 2023-10-06T05:28:27.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.271+0000 7f64d179d700 1 -- start start 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64d179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64cc0b0a40 0x7f64cc16ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64d179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 0x7f64cc153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64d179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64cc15a0a0 0x7f64cc15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64d179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc16b380 con 0x7f64cc0b0e10 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64d179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc16b720 con 0x7f64cc15a0a0 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64d179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc16ba00 con 0x7f64cc0b0a40 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.272+0000 7f64cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 0x7f64cc153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 0x7f64cc153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56306/0 (socket says 172.21.15.33:56306) 2023-10-06T05:28:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 -- 172.21.15.33:0/2904217161 learned_addr learned my addr 172.21.15.33:0/2904217161 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64d0f9c700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64cc15a0a0 0x7f64cc15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cbfff700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64cc0b0a40 0x7f64cc16ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 -- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64cc0b0a40 msgr2=0x7f64cc16ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64cc0b0a40 0x7f64cc16ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 -- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64cc15a0a0 msgr2=0x7f64cc15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64cc15a0a0 0x7f64cc15b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 -- 172.21.15.33:0/2904217161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0094f0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64cb7fe700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 0x7f64cc153e00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c5d0 tx=0x7f64c0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0011590 con 0x7f64cc0b0e10 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c0011710 con 0x7f64cc0b0e10 2023-10-06T05:28:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0011ad0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.273+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc16bce0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.274+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc16c2a0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.275+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc06fcc0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.276+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64c001c460 con 0x7f64cc0b0e10 2023-10-06T05:28:27.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.277+0000 7f64b3fff700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b8066f30 0x7f64b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:27.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.277+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f64c0084600 con 0x7f64cc0b0e10 2023-10-06T05:28:27.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.277+0000 7f64cbfff700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b8066f30 0x7f64b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:27.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.278+0000 7f64cbfff700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b8066f30 0x7f64b80693e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0b0740 tx=0x7f64bc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:27.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.281+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64c00515a0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.524+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64cc16c4c0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.524+0000 7f64b3fff700 1 -- 172.21.15.33:0/2904217161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f64c0054bc0 con 0x7f64cc0b0e10 2023-10-06T05:28:27.527 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:27.527 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b8066f30 msgr2=0x7f64b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b8066f30 0x7f64b80693e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0b0740 tx=0x7f64bc016000 comp rx=0 tx=0).stop 2023-10-06T05:28:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 msgr2=0x7f64cc153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 0x7f64cc153e00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c5d0 tx=0x7f64c0007f60 comp rx=0 tx=0).stop 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 shutdown_connections 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f64b8066f30 0x7f64b80693e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64cc15a0a0 0x7f64cc15b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64cc0b0a40 0x7f64cc16ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 --2- 172.21.15.33:0/2904217161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc0b0e10 0x7f64cc153e00 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 >> 172.21.15.33:0/2904217161 conn(0x7f64cc06e0f0 msgr2=0x7f64cc06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 shutdown_connections 2023-10-06T05:28:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:27.527+0000 7f64d179d700 1 -- 172.21.15.33:0/2904217161 wait complete. 2023-10-06T05:28:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:27.547 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:28.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.743+0000 7fdff639d700 1 Processor -- start 2023-10-06T05:28:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.744+0000 7fdff639d700 1 -- start start 2023-10-06T05:28:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff639d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdff0155140 0x7fdff0158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff00adf50 0x7fdff00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff639d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 0x7fdff00699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff639d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00c8e40 con 0x7fdff00c9810 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff639d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00ae6c0 con 0x7fdff0155140 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00ae9a0 con 0x7fdff00adf50 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff5b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 0x7fdff00699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff5b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 0x7fdff00699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56308/0 (socket says 172.21.15.33:56308) 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff539b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdff0155140 0x7fdff0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff5b9c700 1 -- 172.21.15.33:0/2110029892 learned_addr learned my addr 172.21.15.33:0/2110029892 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.745+0000 7fdff4b9a700 1 --2- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff00adf50 0x7fdff00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff539b700 1 -- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff00adf50 msgr2=0x7fdff00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff539b700 1 --2- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff00adf50 0x7fdff00c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff539b700 1 -- 172.21.15.33:0/2110029892 >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 msgr2=0x7fdff00699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff539b700 1 --2- 172.21.15.33:0/2110029892 >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 0x7fdff00699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff539b700 1 -- 172.21.15.33:0/2110029892 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00c8ad0 con 0x7fdff0155140 2023-10-06T05:28:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff5b9c700 1 --2- 172.21.15.33:0/2110029892 >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 0x7fdff00699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdff539b700 1 --2- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff0155140 0x7fdff0158700 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdff0071970 tx=0x7fdfec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc697db671d588b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:28.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdfe7fff700 1 -- 172.21.15.33:0/2110029892 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec011630 con 0x7fdff0155140 2023-10-06T05:28:28.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdfe7fff700 1 -- 172.21.15.33:0/2110029892 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec0117b0 con 0x7fdff0155140 2023-10-06T05:28:28.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.746+0000 7fdfe7fff700 1 -- 172.21.15.33:0/2110029892 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec011b70 con 0x7fdff0155140 2023-10-06T05:28:28.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 -- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff0155140 msgr2=0x7fdff0158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 --2- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff0155140 0x7fdff0158700 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdff0071970 tx=0x7fdfec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 -- 172.21.15.33:0/2110029892 shutdown_connections 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 --2- 172.21.15.33:0/2110029892 >> v2:172.21.15.33:3300/0 conn(0x7fdff00c9810 0x7fdff00699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 --2- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff0155140 0x7fdff0158700 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 --2- 172.21.15.33:0/2110029892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff00adf50 0x7fdff00c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 -- 172.21.15.33:0/2110029892 >> 172.21.15.33:0/2110029892 conn(0x7fdff006d030 msgr2=0x7fdff006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 -- 172.21.15.33:0/2110029892 shutdown_connections 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.747+0000 7fdff639d700 1 -- 172.21.15.33:0/2110029892 wait complete. 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.748+0000 7fdff639d700 1 Processor -- start 2023-10-06T05:28:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.749+0000 7fdff639d700 1 -- start start 2023-10-06T05:28:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.749+0000 7fdff639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff00adf50 0x7fdff00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.749+0000 7fdff639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 0x7fdff0149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff0155140 0x7fdff014b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff0151cb0 con 0x7fdff00c9810 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b0ff0 con 0x7fdff00adf50 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff0151660 con 0x7fdff0155140 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 0x7fdff0149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 0x7fdff0149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56310/0 (socket says 172.21.15.33:56310) 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 -- 172.21.15.33:0/159241546 learned_addr learned my addr 172.21.15.33:0/159241546 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff539b700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff00adf50 0x7fdff00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff5b9c700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff0155140 0x7fdff014b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 -- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff0155140 msgr2=0x7fdff014b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff0155140 0x7fdff014b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 -- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff00adf50 msgr2=0x7fdff00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff00adf50 0x7fdff00b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.750+0000 7fdff4b9a700 1 -- 172.21.15.33:0/159241546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8009ae0 con 0x7fdff00c9810 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.751+0000 7fdff4b9a700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 0x7fdff0149210 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000de80 tx=0x7fdfe000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.751+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000d2e0 con 0x7fdff00c9810 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.751+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0013040 con 0x7fdff00c9810 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.751+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0010b00 con 0x7fdff00c9810 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.751+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec009420 con 0x7fdff00c9810 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.751+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff00b0ce0 con 0x7fdff00c9810 2023-10-06T05:28:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.752+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0151e20 con 0x7fdff00c9810 2023-10-06T05:28:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.753+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfe0018480 con 0x7fdff00c9810 2023-10-06T05:28:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.754+0000 7fdfe57fa700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfd4066f30 0x7fdfd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.754+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fdfe0088840 con 0x7fdff00c9810 2023-10-06T05:28:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.754+0000 7fdff539b700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfd4066f30 0x7fdfd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:28.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.755+0000 7fdff539b700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfd4066f30 0x7fdfd40693e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdff0071970 tx=0x7fdfec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:28.758+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfe0055890 con 0x7fdff00c9810 2023-10-06T05:28:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.005+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdff004c380 con 0x7fdff00c9810 2023-10-06T05:28:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.006+0000 7fdfe57fa700 1 -- 172.21.15.33:0/159241546 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fdfe0058eb0 con 0x7fdff00c9810 2023-10-06T05:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:29.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.008+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfd4066f30 msgr2=0x7fdfd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.008+0000 7fdff639d700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfd4066f30 0x7fdfd40693e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdff0071970 tx=0x7fdfec017000 comp rx=0 tx=0).stop 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.008+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 msgr2=0x7fdff0149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 0x7fdff0149210 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000de80 tx=0x7fdfe000f040 comp rx=0 tx=0).stop 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 shutdown_connections 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdfd4066f30 0x7fdfd40693e0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdff0155140 0x7fdff014b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff00adf50 0x7fdff00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 --2- 172.21.15.33:0/159241546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdff00c9810 0x7fdff0149210 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 >> 172.21.15.33:0/159241546 conn(0x7fdff006d030 msgr2=0x7fdff00c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 shutdown_connections 2023-10-06T05:28:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:29.009+0000 7fdff639d700 1 -- 172.21.15.33:0/159241546 wait complete. 2023-10-06T05:28:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:29.029 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:30.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.225+0000 7fa35cb9d700 1 Processor -- start 2023-10-06T05:28:30.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.225+0000 7fa35cb9d700 1 -- start start 2023-10-06T05:28:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa35cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa35cb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa358158ba0 con 0x7fa358155140 2023-10-06T05:28:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa35cb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580ae490 con 0x7fa358158700 2023-10-06T05:28:30.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa35cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580c8e40 con 0x7fa35806a0d0 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa3577fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa358155140 0x7fa3580c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51966/0 (socket says 172.21.15.33:51966) 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa3577fe700 1 -- 172.21.15.33:0/1275357302 learned_addr learned my addr 172.21.15.33:0/1275357302 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa357fff700 1 --2- 172.21.15.33:0/1275357302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852913144 0 0) 0x7fa358158ba0 con 0x7fa358155140 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.227+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa340003570 con 0x7fa358155140 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442133690 0 0) 0x7fa3580ae490 con 0x7fa358158700 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa358158ba0 con 0x7fa358158700 2023-10-06T05:28:30.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602505117 0 0) 0x7fa340003570 con 0x7fa358155140 2023-10-06T05:28:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa357fff700 1 -- 172.21.15.33:0/1275357302 >> v1:172.21.15.167:6789/0 conn(0x7fa358158700 legacy=0x7fa3580699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa357fff700 1 -- 172.21.15.33:0/1275357302 >> v1:172.21.15.33:6789/0 conn(0x7fa358155140 legacy=0x7fa3580c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa357fff700 1 -- 172.21.15.33:0/1275357302 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580c99e0 con 0x7fa35806a0d0 2023-10-06T05:28:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa357fff700 1 --2- 172.21.15.33:0/1275357302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580adf50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa350009ec0 tx=0x7fa350009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33ada6aaccb0b71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000e070 con 0x7fa35806a0d0 2023-10-06T05:28:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35000b870 con 0x7fa35806a0d0 2023-10-06T05:28:30.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.228+0000 7fa3567fc700 1 -- 172.21.15.33:0/1275357302 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348001470 con 0x7fa35806a0d0 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1275357302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 msgr2=0x7fa3580adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1275357302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580adf50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa350009ec0 tx=0x7fa350009780 comp rx=0 tx=0).stop 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1275357302 shutdown_connections 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1275357302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580adf50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1275357302 >> 172.21.15.33:0/1275357302 conn(0x7fa35806d030 msgr2=0x7fa35806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1275357302 shutdown_connections 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.229+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1275357302 wait complete. 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.231+0000 7fa35cb9d700 1 Processor -- start 2023-10-06T05:28:30.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.231+0000 7fa35cb9d700 1 -- start start 2023-10-06T05:28:30.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa35cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa35cb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa358155140 0x7fa358159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa35cb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 0x7fa35815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa35cb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580c60a0 con 0x7fa358158700 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa35cb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580afee0 con 0x7fa358155140 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa35cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580c5ae0 con 0x7fa35806a0d0 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 0x7fa35815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 0x7fa35815a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56330/0 (socket says 172.21.15.33:56330) 2023-10-06T05:28:30.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 -- 172.21.15.33:0/1198008894 learned_addr learned my addr 172.21.15.33:0/1198008894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa3577fe700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa356ffd700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa358155140 0x7fa358159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 -- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 msgr2=0x7fa3580af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 -- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa358155140 msgr2=0x7fa358159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa358155140 0x7fa358159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa357fff700 1 -- 172.21.15.33:0/1198008894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350009a20 con 0x7fa358158700 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa3577fe700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:30.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.232+0000 7fa356ffd700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa358155140 0x7fa358159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.233+0000 7fa357fff700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 0x7fa35815a3f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fa3500097c0 tx=0x7fa350017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.233+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350026400 con 0x7fa358158700 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.233+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa350026580 con 0x7fa358158700 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.233+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350026940 con 0x7fa358158700 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.233+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3580c5dc0 con 0x7fa358158700 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.233+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3580af680 con 0x7fa358158700 2023-10-06T05:28:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.234+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3580b0050 con 0x7fa358158700 2023-10-06T05:28:30.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.235+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa350019030 con 0x7fa358158700 2023-10-06T05:28:30.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.236+0000 7fa33f7fe700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa344066f30 0x7fa3440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.236+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fa350086a40 con 0x7fa358158700 2023-10-06T05:28:30.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.236+0000 7fa3577fe700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa344066f30 0x7fa3440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.237+0000 7fa3577fe700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa344066f30 0x7fa3440693e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa348000ff0 tx=0x7fa3480019d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:30.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.240+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3500539e0 con 0x7fa358158700 2023-10-06T05:28:30.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.486+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa35804c380 con 0x7fa358158700 2023-10-06T05:28:30.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.486+0000 7fa33f7fe700 1 -- 172.21.15.33:0/1198008894 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fa350057000 con 0x7fa358158700 2023-10-06T05:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:30.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa344066f30 msgr2=0x7fa3440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa344066f30 0x7fa3440693e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa348000ff0 tx=0x7fa3480019d0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 msgr2=0x7fa35815a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 0x7fa35815a3f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fa3500097c0 tx=0x7fa350017000 comp rx=0 tx=0).stop 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 shutdown_connections 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa344066f30 0x7fa3440693e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa358158700 0x7fa35815a3f0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa35806a0d0 0x7fa3580af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 --2- 172.21.15.33:0/1198008894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa358155140 0x7fa358159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.488+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 >> 172.21.15.33:0/1198008894 conn(0x7fa35806d030 msgr2=0x7fa3580c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.489+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 shutdown_connections 2023-10-06T05:28:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:30.489+0000 7fa35cb9d700 1 -- 172.21.15.33:0/1198008894 wait complete. 2023-10-06T05:28:30.493 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:30.508 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:31.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.706+0000 7fd9a2f9d700 1 Processor -- start 2023-10-06T05:28:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.706+0000 7fd9a2f9d700 1 -- start start 2023-10-06T05:28:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.707+0000 7fd9a2f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd99c155af0 0x7fd99c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.707+0000 7fd9a2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:31.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.707+0000 7fd9a2f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c06ab60 con 0x7fd99c0c8b50 2023-10-06T05:28:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.707+0000 7fd9a2f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99c06b7e0 con 0x7fd99c155af0 2023-10-06T05:28:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c06ae40 con 0x7fd99c06a6a0 2023-10-06T05:28:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a279c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd99c0c8b50 0x7fd99c1598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51982/0 (socket says 172.21.15.33:51982) 2023-10-06T05:28:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a1f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd99c155af0 0x7fd99c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a279c700 1 -- 172.21.15.33:0/2864847429 learned_addr learned my addr 172.21.15.33:0/2864847429 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a179a700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a0f99700 1 -- 172.21.15.33:0/2864847429 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776092506 0 0) 0x7fd99c06ab60 con 0x7fd99c0c8b50 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a0f99700 1 -- 172.21.15.33:0/2864847429 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd988003570 con 0x7fd99c0c8b50 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a1f9b700 1 -- 172.21.15.33:0/2864847429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 msgr2=0x7fd99c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a1f9b700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a1f9b700 1 -- 172.21.15.33:0/2864847429 >> v1:172.21.15.33:6789/0 conn(0x7fd99c0c8b50 legacy=0x7fd99c1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a1f9b700 1 -- 172.21.15.33:0/2864847429 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c06b120 con 0x7fd99c155af0 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.708+0000 7fd9a179a700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a1f9b700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c155af0 0x7fd99c06a160 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fd9980098e0 tx=0x7fd9980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e4cd60a73d10c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a0f99700 1 -- 172.21.15.33:0/2864847429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99800b640 con 0x7fd99c155af0 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a0f99700 1 -- 172.21.15.33:0/2864847429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99800b7a0 con 0x7fd99c155af0 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a0f99700 1 -- 172.21.15.33:0/2864847429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99800bb40 con 0x7fd99c155af0 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2864847429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c155af0 msgr2=0x7fd99c06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c155af0 0x7fd99c06a160 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fd9980098e0 tx=0x7fd9980091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2864847429 shutdown_connections 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.709+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c155af0 0x7fd99c06a160 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.710+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2864847429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.710+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2864847429 >> 172.21.15.33:0/2864847429 conn(0x7fd99c06d030 msgr2=0x7fd99c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.710+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2864847429 shutdown_connections 2023-10-06T05:28:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.710+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2864847429 wait complete. 2023-10-06T05:28:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.711+0000 7fd9a2f9d700 1 Processor -- start 2023-10-06T05:28:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.711+0000 7fd9a2f9d700 1 -- start start 2023-10-06T05:28:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a2f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 0x7fd99c167f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c155af0 0x7fd99c156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a2f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c168490 con 0x7fd99c0c8b50 2023-10-06T05:28:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c156040 con 0x7fd99c155af0 2023-10-06T05:28:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c156320 con 0x7fd99c06a6a0 2023-10-06T05:28:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 0x7fd99c167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a179a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 0x7fd99c167f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56338/0 (socket says 172.21.15.33:56338) 2023-10-06T05:28:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.712+0000 7fd9a179a700 1 -- 172.21.15.33:0/2829307636 learned_addr learned my addr 172.21.15.33:0/2829307636 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a179a700 1 -- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 msgr2=0x7fd99c149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a179a700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a179a700 1 -- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c155af0 msgr2=0x7fd99c156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a179a700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c155af0 0x7fd99c156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a179a700 1 -- 172.21.15.33:0/2829307636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998009420 con 0x7fd99c0c8b50 2023-10-06T05:28:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a179a700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 0x7fd99c167f50 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00e8a0 tx=0x7fd98c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00c430 con 0x7fd99c0c8b50 2023-10-06T05:28:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c00c590 con 0x7fd99c0c8b50 2023-10-06T05:28:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c1686f0 con 0x7fd99c0c8b50 2023-10-06T05:28:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00c950 con 0x7fd99c0c8b50 2023-10-06T05:28:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.713+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c168cb0 con 0x7fd99c0c8b50 2023-10-06T05:28:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.714+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c0cb8a0 con 0x7fd99c0c8b50 2023-10-06T05:28:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.715+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd98c00cd00 con 0x7fd99c0c8b50 2023-10-06T05:28:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.716+0000 7fd991ffb700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd980066f30 0x7fd9800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.716+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fd98c0845b0 con 0x7fd99c0c8b50 2023-10-06T05:28:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.716+0000 7fd9a1f9b700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd980066f30 0x7fd9800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.717+0000 7fd9a1f9b700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd980066f30 0x7fd9800693e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd99800a010 tx=0x7fd998017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.720+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd98c054fe0 con 0x7fd99c0c8b50 2023-10-06T05:28:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.966+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd99c04c380 con 0x7fd99c0c8b50 2023-10-06T05:28:31.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.966+0000 7fd991ffb700 1 -- 172.21.15.33:0/2829307636 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fd98c054b70 con 0x7fd99c0c8b50 2023-10-06T05:28:31.969 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:31.969 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd980066f30 msgr2=0x7fd9800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd980066f30 0x7fd9800693e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd99800a010 tx=0x7fd998017000 comp rx=0 tx=0).stop 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 msgr2=0x7fd99c167f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 0x7fd99c167f50 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00e8a0 tx=0x7fd98c008040 comp rx=0 tx=0).stop 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 shutdown_connections 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd980066f30 0x7fd9800693e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd99c155af0 0x7fd99c156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd99c06a6a0 0x7fd99c149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 --2- 172.21.15.33:0/2829307636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd99c0c8b50 0x7fd99c167f50 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 >> 172.21.15.33:0/2829307636 conn(0x7fd99c06d030 msgr2=0x7fd99c0c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 shutdown_connections 2023-10-06T05:28:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:31.969+0000 7fd9a2f9d700 1 -- 172.21.15.33:0/2829307636 wait complete. 2023-10-06T05:28:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:31.989 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:32.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.188+0000 7f3ac6b9d700 1 Processor -- start 2023-10-06T05:28:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.188+0000 7f3ac6b9d700 1 -- start start 2023-10-06T05:28:33.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac6b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3ac0159530 0x7f3ac006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:33.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 0x7f3ac015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:33.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac6b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac006e3e0 con 0x7f3ac006df70 2023-10-06T05:28:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac6b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00b1e00 con 0x7f3ac0159530 2023-10-06T05:28:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac006f6b0 con 0x7f3ac00b04e0 2023-10-06T05:28:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac5b9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3ac0159530 0x7f3ac006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac639c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 0x7f3ac015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac639c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 0x7f3ac015a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:57886/0 (socket says 172.21.15.33:57886) 2023-10-06T05:28:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.189+0000 7f3ac639c700 1 -- 172.21.15.33:0/4036307792 learned_addr learned my addr 172.21.15.33:0/4036307792 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac5b9b700 1 -- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 msgr2=0x7f3ac015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac5b9b700 1 --2- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 0x7f3ac015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac5b9b700 1 -- 172.21.15.33:0/4036307792 >> v1:172.21.15.167:6789/0 conn(0x7f3ac006df70 legacy=0x7f3ac00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac5b9b700 1 -- 172.21.15.33:0/4036307792 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac006f220 con 0x7f3ac0159530 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac639c700 1 --2- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 0x7f3ac015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac4b99700 1 -- 172.21.15.33:0/4036307792 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385324229 0 0) 0x7f3ac006e3e0 con 0x7f3ac006df70 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.190+0000 7f3ac5b9b700 1 --2- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac0159530 0x7f3ac006da30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009910 tx=0x7f3abc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c47f552df90a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac4b99700 1 -- 172.21.15.33:0/4036307792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00b640 con 0x7f3ac0159530 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac4b99700 1 -- 172.21.15.33:0/4036307792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc00b7a0 con 0x7f3ac0159530 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac4b99700 1 -- 172.21.15.33:0/4036307792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00bb60 con 0x7f3ac0159530 2023-10-06T05:28:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac0159530 msgr2=0x7f3ac006da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac0159530 0x7f3ac006da30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009910 tx=0x7f3abc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/4036307792 shutdown_connections 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac0159530 0x7f3ac006da30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/4036307792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac00b04e0 0x7f3ac015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/4036307792 >> 172.21.15.33:0/4036307792 conn(0x7f3ac006c560 msgr2=0x7f3ac006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/4036307792 shutdown_connections 2023-10-06T05:28:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.191+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/4036307792 wait complete. 2023-10-06T05:28:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.194+0000 7f3ac6b9d700 1 Processor -- start 2023-10-06T05:28:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.194+0000 7f3ac6b9d700 1 -- start start 2023-10-06T05:28:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac006df70 0x7f3ac015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 0x7f3ac0152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac0159530 0x7f3ac00af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00c8410 con 0x7f3ac00b04e0 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00c86f0 con 0x7f3ac006df70 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00c89d0 con 0x7f3ac0159530 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 0x7f3ac0152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac5b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac006df70 0x7f3ac015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 0x7f3ac0152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56340/0 (socket says 172.21.15.33:56340) 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 -- 172.21.15.33:0/3404232509 learned_addr learned my addr 172.21.15.33:0/3404232509 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac639c700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac0159530 0x7f3ac00af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 -- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac0159530 msgr2=0x7f3ac00af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac0159530 0x7f3ac00af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 -- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac006df70 msgr2=0x7f3ac015d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac006df70 0x7f3ac015d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 -- 172.21.15.33:0/3404232509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8009ae0 con 0x7f3ac00b04e0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac5b9b700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac006df70 0x7f3ac015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac539a700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 0x7f3ac0152690 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c530 tx=0x7f3ab000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0011590 con 0x7f3ac00b04e0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab0011710 con 0x7f3ac00b04e0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0011ad0 con 0x7f3ac00b04e0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc009420 con 0x7f3ac00b04e0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.195+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac00c9010 con 0x7f3ac00b04e0 2023-10-06T05:28:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.196+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac00c91a0 con 0x7f3ac00b04e0 2023-10-06T05:28:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.198+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ab001b3b0 con 0x7f3ac00b04e0 2023-10-06T05:28:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.199+0000 7f3ab5ffb700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3aa4066f30 0x7f3aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.199+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f3ab0083c40 con 0x7f3ac00b04e0 2023-10-06T05:28:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.200+0000 7f3ac5b9b700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3aa4066f30 0x7f3aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.200+0000 7f3ac5b9b700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3aa4066f30 0x7f3aa40693e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00b02d0 tx=0x7f3abc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:33.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.203+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ab0050be0 con 0x7f3ac00b04e0 2023-10-06T05:28:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.453+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ac004c380 con 0x7f3ac00b04e0 2023-10-06T05:28:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.454+0000 7f3ab5ffb700 1 -- 172.21.15.33:0/3404232509 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f3ab0017070 con 0x7f3ac00b04e0 2023-10-06T05:28:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:33.456 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3aa4066f30 msgr2=0x7f3aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3aa4066f30 0x7f3aa40693e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00b02d0 tx=0x7f3abc017000 comp rx=0 tx=0).stop 2023-10-06T05:28:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 msgr2=0x7f3ac0152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 0x7f3ac0152690 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c530 tx=0x7f3ab000c840 comp rx=0 tx=0).stop 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 shutdown_connections 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3aa4066f30 0x7f3aa40693e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ac0159530 0x7f3ac00af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac006df70 0x7f3ac015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 --2- 172.21.15.33:0/3404232509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac00b04e0 0x7f3ac0152690 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 >> 172.21.15.33:0/3404232509 conn(0x7f3ac006c560 msgr2=0x7f3ac0158640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.456+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 shutdown_connections 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:33.457+0000 7f3ac6b9d700 1 -- 172.21.15.33:0/3404232509 wait complete. 2023-10-06T05:28:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:33.475 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:34.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.724+0000 7fa94ef9d700 1 Processor -- start 2023-10-06T05:28:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.724+0000 7fa94ef9d700 1 -- start start 2023-10-06T05:28:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.725+0000 7fa94ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 0x7fa9480c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.725+0000 7fa94ef9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa948155ec0 0x7fa9480c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.725+0000 7fa94ef9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94806ab60 con 0x7fa948155af0 2023-10-06T05:28:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.725+0000 7fa94ef9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94806b7e0 con 0x7fa948155ec0 2023-10-06T05:28:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.725+0000 7fa94ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94806ae40 con 0x7fa9480c8b50 2023-10-06T05:28:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94df9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa948155af0 0x7fa94806a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51990/0 (socket says 172.21.15.33:51990) 2023-10-06T05:28:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94df9b700 1 -- 172.21.15.33:0/2372336104 learned_addr learned my addr 172.21.15.33:0/2372336104 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94e79c700 1 --2- 172.21.15.33:0/2372336104 >> v2:172.21.15.167:3300/0 conn(0x7fa948155ec0 0x7fa9480c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94d79a700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 0x7fa9480c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922105462 0 0) 0x7fa94806ab60 con 0x7fa948155af0 2023-10-06T05:28:34.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa934005780 con 0x7fa948155af0 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2103045064 0 0) 0x7fa934005780 con 0x7fa948155af0 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 msgr2=0x7fa9480c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 0x7fa9480c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 >> v2:172.21.15.167:3300/0 conn(0x7fa948155ec0 msgr2=0x7fa9480c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 --2- 172.21.15.33:0/2372336104 >> v2:172.21.15.167:3300/0 conn(0x7fa948155ec0 0x7fa9480c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94806b120 con 0x7fa948155af0 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.726+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa944001470 con 0x7fa948155af0 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 >> v1:172.21.15.33:6789/0 conn(0x7fa948155af0 legacy=0x7fa94806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa934006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa934006a20 0x7fa934006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 0x7fa9340073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94806b7e0 con 0x7fa934005c70 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94806ae40 con 0x7fa934006a20 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93400dc70 con 0x7fa9480c8b50 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94df9b700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa934006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94d79a700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa934006a20 0x7fa934006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94e79c700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 0x7fa9340073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94df9b700 1 -- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 msgr2=0x7fa9340073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94df9b700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c8b50 0x7fa9340073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94df9b700 1 -- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa934006a20 msgr2=0x7fa934006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94df9b700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa934006a20 0x7fa934006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.727+0000 7fa94df9b700 1 -- 172.21.15.33:0/2372336104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934005780 con 0x7fa934005c70 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94df9b700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa934006080 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c3f0 tx=0x7fa944001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d897eadc591b630 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400cc80 con 0x7fa934005c70 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400ce20 con 0x7fa934005c70 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94cf99700 1 -- 172.21.15.33:0/2372336104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944005b70 con 0x7fa934005c70 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94ef9d700 1 -- 172.21.15.33:0/2372336104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 msgr2=0x7fa934006080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/2372336104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa934006080 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c3f0 tx=0x7fa944001e90 comp rx=0 tx=0).stop 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94e79c700 1 -- 172.21.15.33:0/2372336104 reap_dead start 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94ef9d700 1 -- 172.21.15.33:0/2372336104 shutdown_connections 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.728+0000 7fa94ef9d700 1 -- 172.21.15.33:0/2372336104 >> 172.21.15.33:0/2372336104 conn(0x7fa94806d030 msgr2=0x7fa94806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.729+0000 7fa94ef9d700 1 -- 172.21.15.33:0/2372336104 shutdown_connections 2023-10-06T05:28:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.729+0000 7fa94ef9d700 1 -- 172.21.15.33:0/2372336104 wait complete. 2023-10-06T05:28:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.730+0000 7fa94ef9d700 1 Processor -- start 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.730+0000 7fa94ef9d700 1 -- start start 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94ef9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa9481671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c5a30 0x7fa94816a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94ef9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9481583f0 0x7fa948158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94ef9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948157a60 con 0x7fa934005c70 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94ef9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948158db0 con 0x7fa9481583f0 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9481575d0 con 0x7fa9480c5a30 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94df9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa9481671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94df9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa9481671f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56358/0 (socket says 172.21.15.33:56358) 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94df9b700 1 -- 172.21.15.33:0/3616787501 learned_addr learned my addr 172.21.15.33:0/3616787501 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.731+0000 7fa94d79a700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c5a30 0x7fa94816a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94e79c700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9481583f0 0x7fa948158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94df9b700 1 -- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c5a30 msgr2=0x7fa94816a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94df9b700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c5a30 0x7fa94816a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94df9b700 1 -- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9481583f0 msgr2=0x7fa948158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94df9b700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9481583f0 0x7fa948158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94df9b700 1 -- 172.21.15.33:0/3616787501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94816ab70 con 0x7fa934005c70 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94df9b700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa9481671f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fa9340068e0 tx=0x7fa944015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944000f50 con 0x7fa934005c70 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9440072e0 con 0x7fa934005c70 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9481569e0 con 0x7fa934005c70 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440076a0 con 0x7fa934005c70 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.732+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa948156f80 con 0x7fa934005c70 2023-10-06T05:28:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.733+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94804c380 con 0x7fa934005c70 2023-10-06T05:28:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.734+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa944019030 con 0x7fa934005c70 2023-10-06T05:28:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.735+0000 7fa93dffb700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa92c066f30 0x7fa92c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.735+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fa944085a80 con 0x7fa934005c70 2023-10-06T05:28:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.736+0000 7fa94d79a700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa92c066f30 0x7fa92c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:34.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.736+0000 7fa94d79a700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa92c066f30 0x7fa92c0693e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa938006f00 tx=0x7fa938008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.740+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa944052a20 con 0x7fa934005c70 2023-10-06T05:28:34.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.988+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa94806b400 con 0x7fa934005c70 2023-10-06T05:28:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.989+0000 7fa93dffb700 1 -- 172.21.15.33:0/3616787501 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fa944056040 con 0x7fa934005c70 2023-10-06T05:28:34.991 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:34.991 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:34.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa92c066f30 msgr2=0x7fa92c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:34.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa92c066f30 0x7fa92c0693e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa938006f00 tx=0x7fa938008040 comp rx=0 tx=0).stop 2023-10-06T05:28:34.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 msgr2=0x7fa9481671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:34.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa9481671f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fa9340068e0 tx=0x7fa944015000 comp rx=0 tx=0).stop 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 shutdown_connections 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa92c066f30 0x7fa92c0693e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.991+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9481583f0 0x7fa948158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.992+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa934005c70 0x7fa9481671f0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.992+0000 7fa94ef9d700 1 --2- 172.21.15.33:0/3616787501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9480c5a30 0x7fa94816a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.992+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 >> 172.21.15.33:0/3616787501 conn(0x7fa94806d030 msgr2=0x7fa9480caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.992+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 shutdown_connections 2023-10-06T05:28:34.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:34.992+0000 7fa94ef9d700 1 -- 172.21.15.33:0/3616787501 wait complete. 2023-10-06T05:28:34.995 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:35.011 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:36.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.209+0000 7efd4b19d700 1 Processor -- start 2023-10-06T05:28:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.210+0000 7efd4b19d700 1 -- start start 2023-10-06T05:28:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4b19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efd4413faa0 0x7efd4413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd440c7940 0x7efd440c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4b19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440c9cb0 con 0x7efd4406aad0 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4406b640 con 0x7efd440c7940 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4b19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd440ca270 con 0x7efd4413faa0 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd440c7940 0x7efd440c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd440c7940 0x7efd440c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56364/0 (socket says 172.21.15.33:56364) 2023-10-06T05:28:36.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4999a700 1 -- 172.21.15.33:0/2565555253 learned_addr learned my addr 172.21.15.33:0/2565555253 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.211+0000 7efd4a19b700 1 --2- 172.21.15.33:0/2565555253 >> v2:172.21.15.167:3300/0 conn(0x7efd4413faa0 0x7efd4413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd4999a700 1 -- 172.21.15.33:0/2565555253 >> v2:172.21.15.167:3300/0 conn(0x7efd4413faa0 msgr2=0x7efd4413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd4999a700 1 --2- 172.21.15.33:0/2565555253 >> v2:172.21.15.167:3300/0 conn(0x7efd4413faa0 0x7efd4413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd4999a700 1 -- 172.21.15.33:0/2565555253 >> v1:172.21.15.167:6789/0 conn(0x7efd4406aad0 legacy=0x7efd4406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd4999a700 1 -- 172.21.15.33:0/2565555253 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4406b840 con 0x7efd440c7940 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd4999a700 1 --2- 172.21.15.33:0/2565555253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd440c7940 0x7efd440c7db0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7efd4406fe50 tx=0x7efd34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0476da581c52af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd49199700 1 -- 172.21.15.33:0/2565555253 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400b860 con 0x7efd440c7940 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd49199700 1 -- 172.21.15.33:0/2565555253 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400b9c0 con 0x7efd440c7940 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.212+0000 7efd49199700 1 -- 172.21.15.33:0/2565555253 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400bd60 con 0x7efd440c7940 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 -- 172.21.15.33:0/2565555253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd440c7940 msgr2=0x7efd440c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:36.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 --2- 172.21.15.33:0/2565555253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd440c7940 0x7efd440c7db0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7efd4406fe50 tx=0x7efd34009250 comp rx=0 tx=0).stop 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 -- 172.21.15.33:0/2565555253 shutdown_connections 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 --2- 172.21.15.33:0/2565555253 >> v2:172.21.15.167:3300/0 conn(0x7efd4413faa0 0x7efd4413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 --2- 172.21.15.33:0/2565555253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd440c7940 0x7efd440c7db0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 -- 172.21.15.33:0/2565555253 >> 172.21.15.33:0/2565555253 conn(0x7efd4406d030 msgr2=0x7efd4406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 -- 172.21.15.33:0/2565555253 shutdown_connections 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.213+0000 7efd4b19d700 1 -- 172.21.15.33:0/2565555253 wait complete. 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.214+0000 7efd4b19d700 1 Processor -- start 2023-10-06T05:28:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.214+0000 7efd4b19d700 1 -- start start 2023-10-06T05:28:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.215+0000 7efd4b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 0x7efd4413ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.215+0000 7efd4b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd440c7940 0x7efd440b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.215+0000 7efd4b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4413faa0 0x7efd44131be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.215+0000 7efd4b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd440b1d00 con 0x7efd4406aad0 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.215+0000 7efd4b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4413f280 con 0x7efd4413faa0 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.215+0000 7efd4b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4416d480 con 0x7efd440c7940 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 0x7efd4413ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 0x7efd4413ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56378/0 (socket says 172.21.15.33:56378) 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 -- 172.21.15.33:0/468463157 learned_addr learned my addr 172.21.15.33:0/468463157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4999a700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd440c7940 0x7efd440b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a99c700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4413faa0 0x7efd44131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 -- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd440c7940 msgr2=0x7efd440b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd440c7940 0x7efd440b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 -- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4413faa0 msgr2=0x7efd44131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4413faa0 0x7efd44131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 -- 172.21.15.33:0/468463157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340094f0 con 0x7efd4406aad0 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a99c700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4413faa0 0x7efd44131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.216+0000 7efd4a19b700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 0x7efd4413ea60 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7efd4000d890 tx=0x7efd40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.217+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400125f0 con 0x7efd4406aad0 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.217+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40012770 con 0x7efd4406aad0 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.217+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40012b30 con 0x7efd4406aad0 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.217+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd440c26c0 con 0x7efd4406aad0 2023-10-06T05:28:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.217+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd44132120 con 0x7efd4406aad0 2023-10-06T05:28:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.218+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4416d640 con 0x7efd4406aad0 2023-10-06T05:28:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.219+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd4001e460 con 0x7efd4406aad0 2023-10-06T05:28:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.219+0000 7efd3a7fc700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd28066f30 0x7efd280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.219+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7efd40084e80 con 0x7efd4406aad0 2023-10-06T05:28:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.220+0000 7efd4999a700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd28066f30 0x7efd280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.221+0000 7efd4999a700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd28066f30 0x7efd280693e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7efd4406fe50 tx=0x7efd34017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.223+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd40051e20 con 0x7efd4406aad0 2023-10-06T05:28:36.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.470+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd4404c380 con 0x7efd4406aad0 2023-10-06T05:28:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.470+0000 7efd3a7fc700 1 -- 172.21.15.33:0/468463157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7efd40055440 con 0x7efd4406aad0 2023-10-06T05:28:36.472 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:36.472 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.472+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd28066f30 msgr2=0x7efd280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.472+0000 7efd4b19d700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd28066f30 0x7efd280693e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7efd4406fe50 tx=0x7efd34017000 comp rx=0 tx=0).stop 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.472+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 msgr2=0x7efd4413ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.472+0000 7efd4b19d700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 0x7efd4413ea60 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7efd4000d890 tx=0x7efd40008040 comp rx=0 tx=0).stop 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 shutdown_connections 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd28066f30 0x7efd280693e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4413faa0 0x7efd44131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd4406aad0 0x7efd4413ea60 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 --2- 172.21.15.33:0/468463157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd440c7940 0x7efd440b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 >> 172.21.15.33:0/468463157 conn(0x7efd4406d030 msgr2=0x7efd440c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 shutdown_connections 2023-10-06T05:28:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:36.473+0000 7efd4b19d700 1 -- 172.21.15.33:0/468463157 wait complete. 2023-10-06T05:28:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:36.492 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:37.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.690+0000 7f6dff59e700 1 Processor -- start 2023-10-06T05:28:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.691+0000 7f6dff59e700 1 -- start start 2023-10-06T05:28:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dff59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e000ccfc0 0x7f6e000cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dff59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6e000cc750 0x7f6e000b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dff59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e000cbd80 con 0x7f6e000cf3b0 2023-10-06T05:28:37.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dff59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e000af5f0 con 0x7f6e000ccfc0 2023-10-06T05:28:37.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dff59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e000af8d0 con 0x7f6e000cc750 2023-10-06T05:28:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dfe59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e000ccfc0 0x7f6e000cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dfdd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6e000cf3b0 0x7f6e001594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52000/0 (socket says 172.21.15.33:52000) 2023-10-06T05:28:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dfdd9b700 1 -- 172.21.15.33:0/1314897400 learned_addr learned my addr 172.21.15.33:0/1314897400 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.692+0000 7f6dfed9d700 1 --2- 172.21.15.33:0/1314897400 >> v2:172.21.15.167:3300/0 conn(0x7f6e000cc750 0x7f6e000b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfe59c700 1 -- 172.21.15.33:0/1314897400 >> v2:172.21.15.167:3300/0 conn(0x7f6e000cc750 msgr2=0x7f6e000b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfe59c700 1 --2- 172.21.15.33:0/1314897400 >> v2:172.21.15.167:3300/0 conn(0x7f6e000cc750 0x7f6e000b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfe59c700 1 -- 172.21.15.33:0/1314897400 >> v1:172.21.15.33:6789/0 conn(0x7f6e000cf3b0 legacy=0x7f6e001594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfe59c700 1 -- 172.21.15.33:0/1314897400 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e000afbb0 con 0x7f6e000ccfc0 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfe59c700 1 --2- 172.21.15.33:0/1314897400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000ccfc0 0x7f6e000cc210 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f6df00098e0 tx=0x7f6df00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b23c6a666ff5f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfd59a700 1 -- 172.21.15.33:0/1314897400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0011630 con 0x7f6e000ccfc0 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfd59a700 1 -- 172.21.15.33:0/1314897400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df0011790 con 0x7f6e000ccfc0 2023-10-06T05:28:37.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.693+0000 7f6dfd59a700 1 -- 172.21.15.33:0/1314897400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0011b30 con 0x7f6e000ccfc0 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 -- 172.21.15.33:0/1314897400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000ccfc0 msgr2=0x7f6e000cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 --2- 172.21.15.33:0/1314897400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000ccfc0 0x7f6e000cc210 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f6df00098e0 tx=0x7f6df00091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 -- 172.21.15.33:0/1314897400 shutdown_connections 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 --2- 172.21.15.33:0/1314897400 >> v2:172.21.15.167:3300/0 conn(0x7f6e000cc750 0x7f6e000b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 --2- 172.21.15.33:0/1314897400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000ccfc0 0x7f6e000cc210 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 -- 172.21.15.33:0/1314897400 >> 172.21.15.33:0/1314897400 conn(0x7f6e0006cff0 msgr2=0x7f6e0006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 -- 172.21.15.33:0/1314897400 shutdown_connections 2023-10-06T05:28:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.694+0000 7f6dff59e700 1 -- 172.21.15.33:0/1314897400 wait complete. 2023-10-06T05:28:37.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.695+0000 7f6dff59e700 1 Processor -- start 2023-10-06T05:28:37.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.695+0000 7f6dff59e700 1 -- start start 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.696+0000 7f6dff59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e000cc750 0x7f6e0014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.696+0000 7f6dff59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e000ccfc0 0x7f6e000c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.696+0000 7f6dff59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 0x7f6e000d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.696+0000 7f6dff59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000c7d80 con 0x7f6e000c8210 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.696+0000 7f6dff59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0014d2e0 con 0x7f6e000ccfc0 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.696+0000 7f6dff59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000c9e30 con 0x7f6e000cc750 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 0x7f6e000d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 0x7f6e000d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56388/0 (socket says 172.21.15.33:56388) 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 -- 172.21.15.33:0/148607684 learned_addr learned my addr 172.21.15.33:0/148607684 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:37.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfe59c700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e000cc750 0x7f6e0014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfdd9b700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e000ccfc0 0x7f6e000c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 -- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e000cc750 msgr2=0x7f6e0014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e000cc750 0x7f6e0014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 -- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e000ccfc0 msgr2=0x7f6e000c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e000ccfc0 0x7f6e000c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 -- 172.21.15.33:0/148607684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0009420 con 0x7f6e000c8210 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfdd9b700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e000ccfc0 0x7f6e000c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.697+0000 7f6dfed9d700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 0x7f6e000d02d0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009e90 tx=0x7f6df800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.698+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800cc20 con 0x7f6e000c8210 2023-10-06T05:28:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.698+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800f040 con 0x7f6e000c8210 2023-10-06T05:28:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.698+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8014480 con 0x7f6e000c8210 2023-10-06T05:28:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.698+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e000c7730 con 0x7f6e000c8210 2023-10-06T05:28:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.698+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0014d5f0 con 0x7f6e000c8210 2023-10-06T05:28:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.699+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0004c380 con 0x7f6e000c8210 2023-10-06T05:28:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.700+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6df8014830 con 0x7f6e000c8210 2023-10-06T05:28:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.701+0000 7f6dee7fc700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.701+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f6df8085490 con 0x7f6e000c8210 2023-10-06T05:28:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.701+0000 7f6dfe59c700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.702+0000 7f6dfe59c700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6df00091e0 tx=0x7f6df0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.705+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6df8055f60 con 0x7f6e000c8210 2023-10-06T05:28:37.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.954+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e000c6590 con 0x7f6e000c8210 2023-10-06T05:28:37.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.954+0000 7f6dee7fc700 1 -- 172.21.15.33:0/148607684 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f6df8055af0 con 0x7f6e000c8210 2023-10-06T05:28:37.956 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:37.956 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:37.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 msgr2=0x7f6de00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6df00091e0 tx=0x7f6df0017000 comp rx=0 tx=0).stop 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 msgr2=0x7f6e000d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 0x7f6e000d02d0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009e90 tx=0x7f6df800b040 comp rx=0 tx=0).stop 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 shutdown_connections 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e000c8210 0x7f6e000d02d0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e000cc750 0x7f6e0014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 --2- 172.21.15.33:0/148607684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e000ccfc0 0x7f6e000c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 >> 172.21.15.33:0/148607684 conn(0x7f6e0006cff0 msgr2=0x7f6e0014f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 shutdown_connections 2023-10-06T05:28:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:37.957+0000 7f6dff59e700 1 -- 172.21.15.33:0/148607684 wait complete. 2023-10-06T05:28:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:37.976 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:38.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.174+0000 7fb21ffff700 1 Processor -- start 2023-10-06T05:28:39.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.174+0000 7fb21ffff700 1 -- start start 2023-10-06T05:28:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.175+0000 7fb21ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb22013faa0 0x7fb22013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.175+0000 7fb21ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2200c7940 0x7fb2200c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.175+0000 7fb21ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2200c9cb0 con 0x7fb22006aad0 2023-10-06T05:28:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.175+0000 7fb21ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22006b640 con 0x7fb2200c7940 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.175+0000 7fb21ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200ca270 con 0x7fb22013faa0 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2200c7940 0x7fb2200c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb22006aad0 0x7fb22006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52006/0 (socket says 172.21.15.33:52006) 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21f7fe700 1 -- 172.21.15.33:0/2368636145 learned_addr learned my addr 172.21.15.33:0/2368636145 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21effd700 1 --2- 172.21.15.33:0/2368636145 >> v2:172.21.15.167:3300/0 conn(0x7fb22013faa0 0x7fb22013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21e7fc700 1 -- 172.21.15.33:0/2368636145 >> v2:172.21.15.167:3300/0 conn(0x7fb22013faa0 msgr2=0x7fb22013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21e7fc700 1 --2- 172.21.15.33:0/2368636145 >> v2:172.21.15.167:3300/0 conn(0x7fb22013faa0 0x7fb22013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21e7fc700 1 -- 172.21.15.33:0/2368636145 >> v1:172.21.15.33:6789/0 conn(0x7fb22006aad0 legacy=0x7fb22006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21e7fc700 1 -- 172.21.15.33:0/2368636145 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22006b840 con 0x7fb2200c7940 2023-10-06T05:28:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21dffb700 1 -- 172.21.15.33:0/2368636145 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495991272 0 0) 0x7fb2200c9cb0 con 0x7fb22006aad0 2023-10-06T05:28:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.176+0000 7fb21e7fc700 1 --2- 172.21.15.33:0/2368636145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2200c7940 0x7fb2200c7db0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fb22006fe50 tx=0x7fb2140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91a15d47ab96bb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21dffb700 1 -- 172.21.15.33:0/2368636145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400b6b0 con 0x7fb2200c7940 2023-10-06T05:28:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21dffb700 1 -- 172.21.15.33:0/2368636145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21400b810 con 0x7fb2200c7940 2023-10-06T05:28:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21dffb700 1 -- 172.21.15.33:0/2368636145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400bbb0 con 0x7fb2200c7940 2023-10-06T05:28:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 -- 172.21.15.33:0/2368636145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2200c7940 msgr2=0x7fb2200c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 --2- 172.21.15.33:0/2368636145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2200c7940 0x7fb2200c7db0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fb22006fe50 tx=0x7fb2140091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 -- 172.21.15.33:0/2368636145 shutdown_connections 2023-10-06T05:28:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 --2- 172.21.15.33:0/2368636145 >> v2:172.21.15.167:3300/0 conn(0x7fb22013faa0 0x7fb22013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 --2- 172.21.15.33:0/2368636145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2200c7940 0x7fb2200c7db0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 -- 172.21.15.33:0/2368636145 >> 172.21.15.33:0/2368636145 conn(0x7fb22006d030 msgr2=0x7fb22006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 -- 172.21.15.33:0/2368636145 shutdown_connections 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.177+0000 7fb21ffff700 1 -- 172.21.15.33:0/2368636145 wait complete. 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.179+0000 7fb21ffff700 1 Processor -- start 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.179+0000 7fb21ffff700 1 -- start start 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22006aad0 0x7fb220156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2200c7940 0x7fb2201676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 0x7fb220167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb220168fb0 con 0x7fb22013faa0 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200b0230 con 0x7fb2200c7940 2023-10-06T05:28:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2201682a0 con 0x7fb22006aad0 2023-10-06T05:28:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 0x7fb220167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 0x7fb220167c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56412/0 (socket says 172.21.15.33:56412) 2023-10-06T05:28:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22006aad0 0x7fb220156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21f7fe700 1 -- 172.21.15.33:0/1172140967 learned_addr learned my addr 172.21.15.33:0/1172140967 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.180+0000 7fb21e7fc700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2200c7940 0x7fb2201676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21f7fe700 1 -- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22006aad0 msgr2=0x7fb220156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21f7fe700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22006aad0 0x7fb220156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21f7fe700 1 -- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2200c7940 msgr2=0x7fb2201676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21f7fe700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2200c7940 0x7fb2201676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21f7fe700 1 -- 172.21.15.33:0/1172140967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb214009420 con 0x7fb22013faa0 2023-10-06T05:28:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21f7fe700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 0x7fb220167c00 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fb21800ccc0 tx=0x7fb218001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800a8a0 con 0x7fb22013faa0 2023-10-06T05:28:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21800aa00 con 0x7fb22013faa0 2023-10-06T05:28:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb220168580 con 0x7fb22013faa0 2023-10-06T05:28:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800adc0 con 0x7fb22013faa0 2023-10-06T05:28:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.181+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb220168c40 con 0x7fb22013faa0 2023-10-06T05:28:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.182+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22004c380 con 0x7fb22013faa0 2023-10-06T05:28:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.183+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb218007300 con 0x7fb22013faa0 2023-10-06T05:28:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.184+0000 7fb206ffd700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb20c066f30 0x7fb20c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.184+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb218084b20 con 0x7fb22013faa0 2023-10-06T05:28:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.185+0000 7fb21effd700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb20c066f30 0x7fb20c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.185+0000 7fb21effd700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb20c066f30 0x7fb20c0693e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.188+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb218051b40 con 0x7fb22013faa0 2023-10-06T05:28:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.434+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb22006bb20 con 0x7fb22013faa0 2023-10-06T05:28:39.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.434+0000 7fb206ffd700 1 -- 172.21.15.33:0/1172140967 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fb218055160 con 0x7fb22013faa0 2023-10-06T05:28:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:39.436 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:39.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb20c066f30 msgr2=0x7fb20c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:39.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb20c066f30 0x7fb20c0693e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb210006fd0 tx=0x7fb210008040 comp rx=0 tx=0).stop 2023-10-06T05:28:39.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 msgr2=0x7fb220167c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:39.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 0x7fb220167c00 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fb21800ccc0 tx=0x7fb218001a80 comp rx=0 tx=0).stop 2023-10-06T05:28:39.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 shutdown_connections 2023-10-06T05:28:39.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb20c066f30 0x7fb20c0693e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb22013faa0 0x7fb220167c00 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb22006aad0 0x7fb220156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 --2- 172.21.15.33:0/1172140967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2200c7940 0x7fb2201676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 >> 172.21.15.33:0/1172140967 conn(0x7fb22006d030 msgr2=0x7fb22006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 shutdown_connections 2023-10-06T05:28:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:39.437+0000 7fb21ffff700 1 -- 172.21.15.33:0/1172140967 wait complete. 2023-10-06T05:28:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:39.456 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:40.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:40.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.655+0000 7f6eaed9d700 1 Processor -- start 2023-10-06T05:28:40.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.655+0000 7f6eaed9d700 1 -- start start 2023-10-06T05:28:40.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6eaed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ea80c8050 0x7f6ea813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6eaed9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ea806aad0 0x7f6ea806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6eaed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea80c9cb0 con 0x7f6ea813faa0 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6eaed9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea80ca270 con 0x7f6ea80c8050 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6eaed9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea806b840 con 0x7f6ea806aad0 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6ead59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ea80c8050 0x7f6ea813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6ead59a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ea80c8050 0x7f6ea813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37256/0 (socket says 172.21.15.33:37256) 2023-10-06T05:28:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6ead59a700 1 -- 172.21.15.33:0/1605130086 learned_addr learned my addr 172.21.15.33:0/1605130086 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6eae59c700 1 --2- 172.21.15.33:0/1605130086 >> v2:172.21.15.167:3300/0 conn(0x7f6ea806aad0 0x7f6ea806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6ead59a700 1 -- 172.21.15.33:0/1605130086 >> v2:172.21.15.167:3300/0 conn(0x7f6ea806aad0 msgr2=0x7f6ea806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6ead59a700 1 --2- 172.21.15.33:0/1605130086 >> v2:172.21.15.167:3300/0 conn(0x7f6ea806aad0 0x7f6ea806af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.657+0000 7f6ead59a700 1 -- 172.21.15.33:0/1605130086 >> v1:172.21.15.167:6789/0 conn(0x7f6ea813faa0 legacy=0x7f6ea80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.658+0000 7f6ead59a700 1 -- 172.21.15.33:0/1605130086 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea806bb20 con 0x7f6ea80c8050 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.658+0000 7f6ead59a700 1 --2- 172.21.15.33:0/1605130086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea80c8050 0x7f6ea813fe70 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8156b90 tx=0x7f6e98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71240f1428255414 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.658+0000 7f6eacd99700 1 -- 172.21.15.33:0/1605130086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800b860 con 0x7f6ea80c8050 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.658+0000 7f6eacd99700 1 -- 172.21.15.33:0/1605130086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9800b9c0 con 0x7f6ea80c8050 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.658+0000 7f6eacd99700 1 -- 172.21.15.33:0/1605130086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800bd60 con 0x7f6ea80c8050 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 -- 172.21.15.33:0/1605130086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea80c8050 msgr2=0x7f6ea813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/1605130086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea80c8050 0x7f6ea813fe70 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8156b90 tx=0x7f6e98009250 comp rx=0 tx=0).stop 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 -- 172.21.15.33:0/1605130086 shutdown_connections 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/1605130086 >> v2:172.21.15.167:3300/0 conn(0x7f6ea806aad0 0x7f6ea806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/1605130086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea80c8050 0x7f6ea813fe70 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 -- 172.21.15.33:0/1605130086 >> 172.21.15.33:0/1605130086 conn(0x7f6ea806d030 msgr2=0x7f6ea806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 -- 172.21.15.33:0/1605130086 shutdown_connections 2023-10-06T05:28:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.659+0000 7f6eaed9d700 1 -- 172.21.15.33:0/1605130086 wait complete. 2023-10-06T05:28:40.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.660+0000 7f6eaed9d700 1 Processor -- start 2023-10-06T05:28:40.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.660+0000 7f6eaed9d700 1 -- start start 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.661+0000 7f6eaed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 0x7f6ea813de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.661+0000 7f6eaed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea80c8050 0x7f6ea80c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.661+0000 7f6eaed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea813faa0 0x7f6ea8158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.661+0000 7f6eaed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8166780 con 0x7f6ea806aad0 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.661+0000 7f6eaed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80c58f0 con 0x7f6ea80c8050 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.661+0000 7f6eaed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80c5bd0 con 0x7f6ea813faa0 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 0x7f6ea813de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 0x7f6ea813de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37258/0 (socket says 172.21.15.33:37258) 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 -- 172.21.15.33:0/3025319505 learned_addr learned my addr 172.21.15.33:0/3025319505 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6ead59a700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea80c8050 0x7f6ea80c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:40.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eae59c700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea813faa0 0x7f6ea8158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 -- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea813faa0 msgr2=0x7f6ea8158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea813faa0 0x7f6ea8158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 -- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea80c8050 msgr2=0x7f6ea80c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea80c8050 0x7f6ea80c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 -- 172.21.15.33:0/3025319505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980094f0 con 0x7f6ea806aad0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.662+0000 7f6eadd9b700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 0x7f6ea813de10 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c4e0 tx=0x7f6ea400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.663+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011590 con 0x7f6ea806aad0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.663+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011710 con 0x7f6ea806aad0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.663+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011ad0 con 0x7f6ea806aad0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.663+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea80c5eb0 con 0x7f6ea806aad0 2023-10-06T05:28:40.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.663+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea813e350 con 0x7f6ea806aad0 2023-10-06T05:28:40.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.664+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea804c380 con 0x7f6ea806aad0 2023-10-06T05:28:40.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.665+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ea401b3b0 con 0x7f6ea806aad0 2023-10-06T05:28:40.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.665+0000 7f6e9dffb700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e8c066f30 0x7f6e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:40.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.665+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f6ea4083d30 con 0x7f6ea806aad0 2023-10-06T05:28:40.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.666+0000 7f6ead59a700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e8c066f30 0x7f6e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:40.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.667+0000 7f6ead59a700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e8c066f30 0x7f6e8c0693e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8156b90 tx=0x7f6e98017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:40.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.670+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ea4050cd0 con 0x7f6ea806aad0 2023-10-06T05:28:40.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.918+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ea80c9f00 con 0x7f6ea806aad0 2023-10-06T05:28:40.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.919+0000 7f6e9dffb700 1 -- 172.21.15.33:0/3025319505 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f6ea4017070 con 0x7f6ea806aad0 2023-10-06T05:28:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:40.921 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e8c066f30 msgr2=0x7f6e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e8c066f30 0x7f6e8c0693e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8156b90 tx=0x7f6e98017000 comp rx=0 tx=0).stop 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 msgr2=0x7f6ea813de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 0x7f6ea813de10 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c4e0 tx=0x7f6ea400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 shutdown_connections 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e8c066f30 0x7f6e8c0693e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea813faa0 0x7f6ea8158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea806aad0 0x7f6ea813de10 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 --2- 172.21.15.33:0/3025319505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea80c8050 0x7f6ea80c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 >> 172.21.15.33:0/3025319505 conn(0x7f6ea806d030 msgr2=0x7f6ea80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 shutdown_connections 2023-10-06T05:28:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:40.921+0000 7f6eaed9d700 1 -- 172.21.15.33:0/3025319505 wait complete. 2023-10-06T05:28:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:40.943 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:41.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:42.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.144+0000 7f741099d700 1 Processor -- start 2023-10-06T05:28:42.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.144+0000 7f741099d700 1 -- start start 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f741099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147390 0x7f740c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f741099d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f740c147760 0x7f740c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f741099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0c72d0 con 0x7f740c0c6e70 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f741099d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0cb5f0 con 0x7f740c147760 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f741099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0ca9e0 con 0x7f740c147390 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f740bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f740c0c6e70 0x7f740c0c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47038/0 (socket says 172.21.15.33:47038) 2023-10-06T05:28:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f740bfff700 1 -- 172.21.15.33:0/2697621278 learned_addr learned my addr 172.21.15.33:0/2697621278 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f740b7fe700 1 --2- 172.21.15.33:0/2697621278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147390 0x7f740c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.146+0000 7f740affd700 1 --2- 172.21.15.33:0/2697621278 >> v2:172.21.15.167:3300/0 conn(0x7f740c147760 0x7f740c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740a7fc700 1 -- 172.21.15.33:0/2697621278 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608824566 0 0) 0x7f7404001490 con 0x7f740c0c6e70 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740a7fc700 1 -- 172.21.15.33:0/2697621278 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f4003570 con 0x7f740c0c6e70 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740b7fe700 1 -- 172.21.15.33:0/2697621278 >> v2:172.21.15.167:3300/0 conn(0x7f740c147760 msgr2=0x7f740c0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740b7fe700 1 --2- 172.21.15.33:0/2697621278 >> v2:172.21.15.167:3300/0 conn(0x7f740c147760 0x7f740c0c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740b7fe700 1 -- 172.21.15.33:0/2697621278 >> v1:172.21.15.33:6789/0 conn(0x7f740c0c6e70 legacy=0x7f740c0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740b7fe700 1 -- 172.21.15.33:0/2697621278 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0cacc0 con 0x7f740c147390 2023-10-06T05:28:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740b7fe700 1 --2- 172.21.15.33:0/2697621278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147390 0x7f740c0cba80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0098e0 tx=0x7f73fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16ea3c5a7cccee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740a7fc700 1 -- 172.21.15.33:0/2697621278 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc011630 con 0x7f740c147390 2023-10-06T05:28:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740a7fc700 1 -- 172.21.15.33:0/2697621278 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc011790 con 0x7f740c147390 2023-10-06T05:28:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.147+0000 7f740a7fc700 1 -- 172.21.15.33:0/2697621278 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc011b10 con 0x7f740c147390 2023-10-06T05:28:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 -- 172.21.15.33:0/2697621278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147390 msgr2=0x7f740c0cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 --2- 172.21.15.33:0/2697621278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147390 0x7f740c0cba80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0098e0 tx=0x7f73fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 -- 172.21.15.33:0/2697621278 shutdown_connections 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 --2- 172.21.15.33:0/2697621278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147390 0x7f740c0cba80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 --2- 172.21.15.33:0/2697621278 >> v2:172.21.15.167:3300/0 conn(0x7f740c147760 0x7f740c0c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 -- 172.21.15.33:0/2697621278 >> 172.21.15.33:0/2697621278 conn(0x7f740c06d030 msgr2=0x7f740c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 -- 172.21.15.33:0/2697621278 shutdown_connections 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.148+0000 7f741099d700 1 -- 172.21.15.33:0/2697621278 wait complete. 2023-10-06T05:28:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.150+0000 7f741099d700 1 Processor -- start 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.150+0000 7f741099d700 1 -- start start 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f741099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 0x7f740c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f741099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f740c147390 0x7f740c145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f741099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147760 0x7f740c0c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f741099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c16de10 con 0x7f740c0c6e70 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f741099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c16e0f0 con 0x7f740c147390 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f741099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c16e3d0 con 0x7f740c147760 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 0x7f740c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 0x7f740c15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37268/0 (socket says 172.21.15.33:37268) 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 -- 172.21.15.33:0/1886617488 learned_addr learned my addr 172.21.15.33:0/1886617488 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 -- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147760 msgr2=0x7f740c0c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147760 0x7f740c0c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 -- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f740c147390 msgr2=0x7f740c145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740affd700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f740c147390 0x7f740c145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f740c147390 0x7f740c145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740b7fe700 1 -- 172.21.15.33:0/1886617488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc009420 con 0x7f740c0c6e70 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.151+0000 7f740affd700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f740c147390 0x7f740c145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.152+0000 7f740b7fe700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 0x7f740c15e200 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f740c068570 tx=0x7f73fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.152+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc011b10 con 0x7f740c0c6e70 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.152+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc000cc0 con 0x7f740c0c6e70 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.152+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01a680 con 0x7f740c0c6e70 2023-10-06T05:28:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.152+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c16e6b0 con 0x7f740c0c6e70 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.152+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c145940 con 0x7f740c0c6e70 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.153+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c16e840 con 0x7f740c0c6e70 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.154+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73fc01aa70 con 0x7f740c0c6e70 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.155+0000 7f73f37fe700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73f8066f30 0x7f73f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.155+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f73fc086430 con 0x7f740c0c6e70 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.155+0000 7f740affd700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73f8066f30 0x7f73f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.156+0000 7f740affd700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73f8066f30 0x7f73f80693e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7400006f00 tx=0x7f7400008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.159+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73fc0533d0 con 0x7f740c0c6e70 2023-10-06T05:28:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.408+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f740c04c380 con 0x7f740c0c6e70 2023-10-06T05:28:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.409+0000 7f73f37fe700 1 -- 172.21.15.33:0/1886617488 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f73fc0569f0 con 0x7f740c0c6e70 2023-10-06T05:28:42.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:42.411 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73f8066f30 msgr2=0x7f73f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73f8066f30 0x7f73f80693e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7400006f00 tx=0x7f7400008040 comp rx=0 tx=0).stop 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 msgr2=0x7f740c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 0x7f740c15e200 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f740c068570 tx=0x7f73fc017000 comp rx=0 tx=0).stop 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 shutdown_connections 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73f8066f30 0x7f73f80693e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f740c147760 0x7f740c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f740c0c6e70 0x7f740c15e200 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 --2- 172.21.15.33:0/1886617488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f740c147390 0x7f740c145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 >> 172.21.15.33:0/1886617488 conn(0x7f740c06d030 msgr2=0x7f740c146390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 shutdown_connections 2023-10-06T05:28:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:42.411+0000 7f741099d700 1 -- 172.21.15.33:0/1886617488 wait complete. 2023-10-06T05:28:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:42.430 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:43.431 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:43.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.628+0000 7fba69b9d700 1 Processor -- start 2023-10-06T05:28:43.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.628+0000 7fba69b9d700 1 -- start start 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba69b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba6413faa0 0x7fba6413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba69b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640c7940 0x7fba640c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba69b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba640c9cb0 con 0x7fba6406aad0 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba69b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6406b660 con 0x7fba6413faa0 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba69b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640ca270 con 0x7fba640c7940 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba68b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba6413faa0 0x7fba6413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:43.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba68b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba6413faa0 0x7fba6413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37284/0 (socket says 172.21.15.33:37284) 2023-10-06T05:28:43.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.629+0000 7fba68b9b700 1 -- 172.21.15.33:0/2583809437 learned_addr learned my addr 172.21.15.33:0/2583809437 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:43.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba63fff700 1 --2- 172.21.15.33:0/2583809437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640c7940 0x7fba640c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:43.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba68b9b700 1 -- 172.21.15.33:0/2583809437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640c7940 msgr2=0x7fba640c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:43.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba68b9b700 1 --2- 172.21.15.33:0/2583809437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640c7940 0x7fba640c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba68b9b700 1 -- 172.21.15.33:0/2583809437 >> v1:172.21.15.167:6789/0 conn(0x7fba6406aad0 legacy=0x7fba6406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba68b9b700 1 -- 172.21.15.33:0/2583809437 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6406b840 con 0x7fba6413faa0 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba68b9b700 1 --2- 172.21.15.33:0/2583809437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba6413faa0 0x7fba6413fe70 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fba5800bc50 tx=0x7fba5800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea61eed3dda955f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.630+0000 7fba637fe700 1 -- 172.21.15.33:0/2583809437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5800dbf0 con 0x7fba6413faa0 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba637fe700 1 -- 172.21.15.33:0/2583809437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5800dd50 con 0x7fba6413faa0 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba637fe700 1 -- 172.21.15.33:0/2583809437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba580196b0 con 0x7fba6413faa0 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 -- 172.21.15.33:0/2583809437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba6413faa0 msgr2=0x7fba6413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 --2- 172.21.15.33:0/2583809437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba6413faa0 0x7fba6413fe70 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fba5800bc50 tx=0x7fba5800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 -- 172.21.15.33:0/2583809437 shutdown_connections 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 --2- 172.21.15.33:0/2583809437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba6413faa0 0x7fba6413fe70 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 --2- 172.21.15.33:0/2583809437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640c7940 0x7fba640c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 -- 172.21.15.33:0/2583809437 >> 172.21.15.33:0/2583809437 conn(0x7fba6406d030 msgr2=0x7fba6406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 -- 172.21.15.33:0/2583809437 shutdown_connections 2023-10-06T05:28:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.631+0000 7fba69b9d700 1 -- 172.21.15.33:0/2583809437 wait complete. 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.633+0000 7fba69b9d700 1 Processor -- start 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.633+0000 7fba69b9d700 1 -- start start 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba69b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba6406aad0 0x7fba640c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba69b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 0x7fba640c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba69b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba6413faa0 0x7fba6413c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba69b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c4e60 con 0x7fba640c7940 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba69b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c3630 con 0x7fba6406aad0 2023-10-06T05:28:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba69b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c48a0 con 0x7fba6413faa0 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 0x7fba640c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 0x7fba640c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37294/0 (socket says 172.21.15.33:37294) 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 -- 172.21.15.33:0/1719860761 learned_addr learned my addr 172.21.15.33:0/1719860761 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba68b9b700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba6406aad0 0x7fba640c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba6939c700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba6413faa0 0x7fba6413c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 -- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba6413faa0 msgr2=0x7fba6413c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba6413faa0 0x7fba6413c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 -- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba6406aad0 msgr2=0x7fba640c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba6406aad0 0x7fba640c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba63fff700 1 -- 172.21.15.33:0/1719860761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5800b790 con 0x7fba640c7940 2023-10-06T05:28:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.634+0000 7fba68b9b700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba6406aad0 0x7fba640c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.635+0000 7fba63fff700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 0x7fba640c3ac0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ec80 tx=0x7fba500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.635+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000c9d0 con 0x7fba640c7940 2023-10-06T05:28:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.635+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5000cb30 con 0x7fba640c7940 2023-10-06T05:28:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.635+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640c4b80 con 0x7fba640c7940 2023-10-06T05:28:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.635+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50013470 con 0x7fba640c7940 2023-10-06T05:28:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.635+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba64159b00 con 0x7fba640c7940 2023-10-06T05:28:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.636+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba640c37a0 con 0x7fba640c7940 2023-10-06T05:28:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.637+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba50013820 con 0x7fba640c7940 2023-10-06T05:28:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.638+0000 7fba60ff9700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.638+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fba50084b70 con 0x7fba640c7940 2023-10-06T05:28:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.639+0000 7fba68b9b700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.640+0000 7fba68b9b700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba48066f30 0x7fba480693e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fba5800bc50 tx=0x7fba5801b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.642+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba50051b10 con 0x7fba640c7940 2023-10-06T05:28:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.893+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba6404c380 con 0x7fba640c7940 2023-10-06T05:28:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.893+0000 7fba60ff9700 1 -- 172.21.15.33:0/1719860761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fba50055130 con 0x7fba640c7940 2023-10-06T05:28:43.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:43.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba48066f30 msgr2=0x7fba480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba48066f30 0x7fba480693e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fba5800bc50 tx=0x7fba5801b000 comp rx=0 tx=0).stop 2023-10-06T05:28:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 msgr2=0x7fba640c3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 0x7fba640c3ac0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ec80 tx=0x7fba500091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 shutdown_connections 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba6413faa0 0x7fba6413c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba6406aad0 0x7fba640c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 --2- 172.21.15.33:0/1719860761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba640c7940 0x7fba640c3ac0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.895+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 >> 172.21.15.33:0/1719860761 conn(0x7fba6406d030 msgr2=0x7fba640cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.896+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 shutdown_connections 2023-10-06T05:28:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:43.896+0000 7fba69b9d700 1 -- 172.21.15.33:0/1719860761 wait complete. 2023-10-06T05:28:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:43.915 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:44.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:45.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.113+0000 7fb728f9d700 1 Processor -- start 2023-10-06T05:28:45.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.113+0000 7fb728f9d700 1 -- start start 2023-10-06T05:28:45.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.114+0000 7fb728f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb72406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:45.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.114+0000 7fb728f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72406ab60 con 0x7fb7240c7920 2023-10-06T05:28:45.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb728f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72406ae40 con 0x7fb72406a6a0 2023-10-06T05:28:45.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb728f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72406b7e0 con 0x7fb724155af0 2023-10-06T05:28:45.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb723fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb7240c7920 0x7fb7240c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47040/0 (socket says 172.21.15.33:47040) 2023-10-06T05:28:45.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb723fff700 1 -- 172.21.15.33:0/777302563 learned_addr learned my addr 172.21.15.33:0/777302563 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:45.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7237fe700 1 --2- 172.21.15.33:0/777302563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb72406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:45.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7227fc700 1 -- 172.21.15.33:0/777302563 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313760631 0 0) 0x7fb72406ab60 con 0x7fb7240c7920 2023-10-06T05:28:45.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7227fc700 1 -- 172.21.15.33:0/777302563 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb70c004680 con 0x7fb7240c7920 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7237fe700 1 -- 172.21.15.33:0/777302563 >> v1:172.21.15.167:6789/0 conn(0x7fb72406a6a0 legacy=0x7fb7240c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7227fc700 1 -- 172.21.15.33:0/777302563 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517072002 0 0) 0x7fb72406ae40 con 0x7fb72406a6a0 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7237fe700 1 -- 172.21.15.33:0/777302563 >> v1:172.21.15.33:6789/0 conn(0x7fb7240c7920 legacy=0x7fb7240c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.115+0000 7fb7237fe700 1 -- 172.21.15.33:0/777302563 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72406b120 con 0x7fb724155af0 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.116+0000 7fb7237fe700 1 --2- 172.21.15.33:0/777302563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb72406a160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb714009910 tx=0x7fb7140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a3f19f5e0b8c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.116+0000 7fb7227fc700 1 -- 172.21.15.33:0/777302563 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400b680 con 0x7fb724155af0 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.116+0000 7fb7227fc700 1 -- 172.21.15.33:0/777302563 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71400b7e0 con 0x7fb724155af0 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.116+0000 7fb7227fc700 1 -- 172.21.15.33:0/777302563 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400bba0 con 0x7fb724155af0 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 -- 172.21.15.33:0/777302563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 msgr2=0x7fb72406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 --2- 172.21.15.33:0/777302563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb72406a160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb714009910 tx=0x7fb7140091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 -- 172.21.15.33:0/777302563 shutdown_connections 2023-10-06T05:28:45.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 --2- 172.21.15.33:0/777302563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb72406a160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 -- 172.21.15.33:0/777302563 >> 172.21.15.33:0/777302563 conn(0x7fb72406d030 msgr2=0x7fb72406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:45.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 -- 172.21.15.33:0/777302563 shutdown_connections 2023-10-06T05:28:45.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.117+0000 7fb728f9d700 1 -- 172.21.15.33:0/777302563 wait complete. 2023-10-06T05:28:45.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.118+0000 7fb728f9d700 1 Processor -- start 2023-10-06T05:28:45.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.118+0000 7fb728f9d700 1 -- start start 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.119+0000 7fb728f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb72406a6a0 0x7fb7241691f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.119+0000 7fb728f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 0x7fb7240c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.119+0000 7fb728f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb7240adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.119+0000 7fb728f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240b1bc0 con 0x7fb7240c7920 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.119+0000 7fb728f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240b0fb0 con 0x7fb72406a6a0 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.119+0000 7fb728f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240b1290 con 0x7fb724155af0 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 0x7fb7240c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 0x7fb7240c3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37310/0 (socket says 172.21.15.33:37310) 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 -- 172.21.15.33:0/4136708923 learned_addr learned my addr 172.21.15.33:0/4136708923 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb7237fe700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb72406a6a0 0x7fb7241691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:45.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb723fff700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb7240adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 -- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 msgr2=0x7fb7240adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb7240adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 -- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb72406a6a0 msgr2=0x7fb7241691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb72406a6a0 0x7fb7241691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 -- 172.21.15.33:0/4136708923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714009420 con 0x7fb7240c7920 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb723fff700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb7240adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.120+0000 7fb722ffd700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 0x7fb7240c3c40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fb71800c7e0 tx=0x7fb718007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.121+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718011590 con 0x7fb7240c7920 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.121+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7180116f0 con 0x7fb7240c7920 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.121+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718011ab0 con 0x7fb7240c7920 2023-10-06T05:28:45.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.121+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7240b1570 con 0x7fb7240c7920 2023-10-06T05:28:45.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.121+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb724169730 con 0x7fb7240c7920 2023-10-06T05:28:45.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.122+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72404bb40 con 0x7fb7240c7920 2023-10-06T05:28:45.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.123+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb71801b3b0 con 0x7fb7240c7920 2023-10-06T05:28:45.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.124+0000 7fb70b7fe700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb710066f30 0x7fb7100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:45.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.124+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fb718083cc0 con 0x7fb7240c7920 2023-10-06T05:28:45.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.125+0000 7fb7237fe700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb710066f30 0x7fb7100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.125+0000 7fb7237fe700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb710066f30 0x7fb7100693e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb7140093f0 tx=0x7fb714017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:45.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.129+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb71808a080 con 0x7fb7240c7920 2023-10-06T05:28:45.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.373+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb72404c380 con 0x7fb7240c7920 2023-10-06T05:28:45.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.374+0000 7fb70b7fe700 1 -- 172.21.15.33:0/4136708923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fb718017070 con 0x7fb7240c7920 2023-10-06T05:28:45.376 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:45.376 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:45.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb710066f30 msgr2=0x7fb7100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb710066f30 0x7fb7100693e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb7140093f0 tx=0x7fb714017000 comp rx=0 tx=0).stop 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 msgr2=0x7fb7240c3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 0x7fb7240c3c40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fb71800c7e0 tx=0x7fb718007dc0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 shutdown_connections 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb710066f30 0x7fb7100693e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb724155af0 0x7fb7240adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb72406a6a0 0x7fb7241691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 --2- 172.21.15.33:0/4136708923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7240c7920 0x7fb7240c3c40 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 >> 172.21.15.33:0/4136708923 conn(0x7fb72406d030 msgr2=0x7fb7240cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 shutdown_connections 2023-10-06T05:28:45.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:45.376+0000 7fb728f9d700 1 -- 172.21.15.33:0/4136708923 wait complete. 2023-10-06T05:28:45.381 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:45.395 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:46.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.593+0000 7fe8b3fff700 1 Processor -- start 2023-10-06T05:28:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.593+0000 7fe8b3fff700 1 -- start start 2023-10-06T05:28:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.594+0000 7fe8b3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8b4153b10 0x7fe8b40c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.594+0000 7fe8b3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b415abd0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b40c9b00 con 0x7fe8b40c2610 2023-10-06T05:28:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b4068410 con 0x7fe8b4153b10 2023-10-06T05:28:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8b4153b10 0x7fe8b40c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b27fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe8b40c6b30 0x7fe8b40675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47054/0 (socket says 172.21.15.33:47054) 2023-10-06T05:28:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b27fc700 1 -- 172.21.15.33:0/3705983877 learned_addr learned my addr 172.21.15.33:0/3705983877 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b2ffd700 1 -- 172.21.15.33:0/3705983877 >> v1:172.21.15.167:6789/0 conn(0x7fe8b40c2610 legacy=0x7fe8b4067d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b2ffd700 1 -- 172.21.15.33:0/3705983877 >> v1:172.21.15.33:6789/0 conn(0x7fe8b40c6b30 legacy=0x7fe8b40675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b2ffd700 1 -- 172.21.15.33:0/3705983877 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b415a050 con 0x7fe8b4153b10 2023-10-06T05:28:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.595+0000 7fe8b1ffb700 1 -- 172.21.15.33:0/3705983877 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934427884 0 0) 0x7fe8b415abd0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b2ffd700 1 --2- 172.21.15.33:0/3705983877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c95c0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098e0 tx=0x7fe8a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d46122df1bc1be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b1ffb700 1 -- 172.21.15.33:0/3705983877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4009e80 con 0x7fe8b4153b10 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b1ffb700 1 -- 172.21.15.33:0/3705983877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a400b820 con 0x7fe8b4153b10 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b1ffb700 1 -- 172.21.15.33:0/3705983877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400bba0 con 0x7fe8b4153b10 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 -- 172.21.15.33:0/3705983877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b4153b10 msgr2=0x7fe8b40c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/3705983877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c95c0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098e0 tx=0x7fe8a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 -- 172.21.15.33:0/3705983877 shutdown_connections 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/3705983877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c95c0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 -- 172.21.15.33:0/3705983877 >> 172.21.15.33:0/3705983877 conn(0x7fe8b406c560 msgr2=0x7fe8b406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 -- 172.21.15.33:0/3705983877 shutdown_connections 2023-10-06T05:28:46.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.596+0000 7fe8b3fff700 1 -- 172.21.15.33:0/3705983877 wait complete. 2023-10-06T05:28:46.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.598+0000 7fe8b3fff700 1 Processor -- start 2023-10-06T05:28:46.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.598+0000 7fe8b3fff700 1 -- start start 2023-10-06T05:28:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b40c2610 0x7fe8b413f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 0x7fe8b4169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b41531f0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b413cc60 con 0x7fe8b4153b10 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b413cf40 con 0x7fe8b40c2610 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 0x7fe8b4169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 0x7fe8b4169960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37322/0 (socket says 172.21.15.33:37322) 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b27fc700 1 -- 172.21.15.33:0/2821706925 learned_addr learned my addr 172.21.15.33:0/2821706925 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b2ffd700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b40c2610 0x7fe8b413f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.599+0000 7fe8b37fe700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b27fc700 1 -- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b40c2610 msgr2=0x7fe8b413f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b27fc700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b40c2610 0x7fe8b413f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b27fc700 1 -- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b4153b10 msgr2=0x7fe8b40c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b27fc700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b27fc700 1 -- 172.21.15.33:0/2821706925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009420 con 0x7fe8b40c6b30 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b37fe700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b27fc700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 0x7fe8b4169960 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c810 tx=0x7fe8a8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005b30 con 0x7fe8b40c6b30 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8005c90 con 0x7fe8b40c6b30 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80072e0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b40c5d20 con 0x7fe8b40c6b30 2023-10-06T05:28:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.600+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b40c62e0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.601+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b413d0d0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.602+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8a8007690 con 0x7fe8b40c6b30 2023-10-06T05:28:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.604+0000 7fe89affd700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8a0066f30 0x7fe8a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.604+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fe8a8083b60 con 0x7fe8b40c6b30 2023-10-06T05:28:46.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.605+0000 7fe8b2ffd700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8a0066f30 0x7fe8a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:46.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.605+0000 7fe8b2ffd700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8a0066f30 0x7fe8a00693e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40030c0 tx=0x7fe8a4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:46.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.608+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8a804b5a0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.854+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8b404c380 con 0x7fe8b40c6b30 2023-10-06T05:28:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.855+0000 7fe89affd700 1 -- 172.21.15.33:0/2821706925 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fe8a80546e0 con 0x7fe8b40c6b30 2023-10-06T05:28:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8a0066f30 msgr2=0x7fe8a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8a0066f30 0x7fe8a00693e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40030c0 tx=0x7fe8a4016000 comp rx=0 tx=0).stop 2023-10-06T05:28:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 msgr2=0x7fe8b4169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 0x7fe8b4169960 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c810 tx=0x7fe8a8007df0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 shutdown_connections 2023-10-06T05:28:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe8a0066f30 0x7fe8a00693e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b4153b10 0x7fe8b40c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8b40c2610 0x7fe8b413f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 --2- 172.21.15.33:0/2821706925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8b40c6b30 0x7fe8b4169960 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 >> 172.21.15.33:0/2821706925 conn(0x7fe8b406c560 msgr2=0x7fe8b413bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 shutdown_connections 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:46.857+0000 7fe8b3fff700 1 -- 172.21.15.33:0/2821706925 wait complete. 2023-10-06T05:28:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:46.877 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:47.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:48.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.076+0000 7f1afab9d700 1 Processor -- start 2023-10-06T05:28:48.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.076+0000 7f1afab9d700 1 -- start start 2023-10-06T05:28:48.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.077+0000 7f1afab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1af413faa0 0x7f1af413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:48.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.077+0000 7f1afab9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1af406aad0 0x7f1af406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:48.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1afab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af40c9cb0 con 0x7f1af40c7940 2023-10-06T05:28:48.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1afab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af40ca270 con 0x7f1af413faa0 2023-10-06T05:28:48.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1afab9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af406b840 con 0x7f1af406aad0 2023-10-06T05:28:48.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af9b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1af413faa0 0x7f1af413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af939a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1af40c7940 0x7f1af40c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47068/0 (socket says 172.21.15.33:47068) 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af939a700 1 -- 172.21.15.33:0/523461064 learned_addr learned my addr 172.21.15.33:0/523461064 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af9b9b700 1 -- 172.21.15.33:0/523461064 >> v2:172.21.15.167:3300/0 conn(0x7f1af406aad0 msgr2=0x7f1af406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af9b9b700 1 --2- 172.21.15.33:0/523461064 >> v2:172.21.15.167:3300/0 conn(0x7f1af406aad0 0x7f1af406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af9b9b700 1 -- 172.21.15.33:0/523461064 >> v1:172.21.15.33:6789/0 conn(0x7f1af40c7940 legacy=0x7f1af40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.078+0000 7f1af9b9b700 1 -- 172.21.15.33:0/523461064 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af406bb20 con 0x7f1af413faa0 2023-10-06T05:28:48.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1af9b9b700 1 --2- 172.21.15.33:0/523461064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af413faa0 0x7f1af413fe70 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f1af4156b90 tx=0x7f1af00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03b0007221458fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:48.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1af8b99700 1 -- 172.21.15.33:0/523461064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000f6b0 con 0x7f1af413faa0 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1af8b99700 1 -- 172.21.15.33:0/523461064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af000f810 con 0x7f1af413faa0 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1af8b99700 1 -- 172.21.15.33:0/523461064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000fbb0 con 0x7f1af413faa0 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1afab9d700 1 -- 172.21.15.33:0/523461064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af413faa0 msgr2=0x7f1af413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1afab9d700 1 --2- 172.21.15.33:0/523461064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af413faa0 0x7f1af413fe70 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f1af4156b90 tx=0x7f1af00091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1afab9d700 1 -- 172.21.15.33:0/523461064 shutdown_connections 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1afab9d700 1 --2- 172.21.15.33:0/523461064 >> v2:172.21.15.167:3300/0 conn(0x7f1af406aad0 0x7f1af406af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1afab9d700 1 --2- 172.21.15.33:0/523461064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af413faa0 0x7f1af413fe70 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.079+0000 7f1afab9d700 1 -- 172.21.15.33:0/523461064 >> 172.21.15.33:0/523461064 conn(0x7f1af406d030 msgr2=0x7f1af406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.080+0000 7f1afab9d700 1 -- 172.21.15.33:0/523461064 shutdown_connections 2023-10-06T05:28:48.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.080+0000 7f1afab9d700 1 -- 172.21.15.33:0/523461064 wait complete. 2023-10-06T05:28:48.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.081+0000 7f1afab9d700 1 Processor -- start 2023-10-06T05:28:48.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.081+0000 7f1afab9d700 1 -- start start 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1afab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 0x7f1af41699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1afab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1af40c7940 0x7f1af40c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1afab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1af413faa0 0x7f1af40c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1afab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40c53d0 con 0x7f1af406aad0 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1afab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40c5d10 con 0x7f1af413faa0 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1afab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40c4110 con 0x7f1af40c7940 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1af9b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 0x7f1af41699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1af9b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 0x7f1af41699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37346/0 (socket says 172.21.15.33:37346) 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.082+0000 7f1af9b9b700 1 -- 172.21.15.33:0/1403292399 learned_addr learned my addr 172.21.15.33:0/1403292399 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1afa39c700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1af413faa0 0x7f1af40c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af9b9b700 1 -- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1af40c7940 msgr2=0x7f1af40c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af939a700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1af40c7940 0x7f1af40c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af9b9b700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1af40c7940 0x7f1af40c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af9b9b700 1 -- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1af413faa0 msgr2=0x7f1af40c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af9b9b700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1af413faa0 0x7f1af40c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af9b9b700 1 -- 172.21.15.33:0/1403292399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af0009420 con 0x7f1af406aad0 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1afa39c700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1af413faa0 0x7f1af40c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af939a700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1af40c7940 0x7f1af40c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1af9b9b700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 0x7f1af41699c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f1af40719c0 tx=0x7f1af0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af001e070 con 0x7f1af406aad0 2023-10-06T05:28:48.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af000f980 con 0x7f1af406aad0 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0010470 con 0x7f1af406aad0 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af40c43b0 con 0x7f1af406aad0 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.083+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af40c4970 con 0x7f1af406aad0 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.084+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af40c4b90 con 0x7f1af406aad0 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.085+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1af0022070 con 0x7f1af406aad0 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.086+0000 7f1ae9ffb700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ad8066f30 0x7f1ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:48.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.086+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f1af0014070 con 0x7f1af406aad0 2023-10-06T05:28:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.087+0000 7f1af939a700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ad8066f30 0x7f1ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.087+0000 7f1af939a700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ad8066f30 0x7f1ad80693e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1af40cb950 tx=0x7f1ae4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.090+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1af0054950 con 0x7f1af406aad0 2023-10-06T05:28:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.337+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1af404c380 con 0x7f1af406aad0 2023-10-06T05:28:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.337+0000 7f1ae9ffb700 1 -- 172.21.15.33:0/1403292399 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f1af0057f70 con 0x7f1af406aad0 2023-10-06T05:28:48.340 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:48.340 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ad8066f30 msgr2=0x7f1ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ad8066f30 0x7f1ad80693e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1af40cb950 tx=0x7f1ae4009040 comp rx=0 tx=0).stop 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 msgr2=0x7f1af41699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 0x7f1af41699c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f1af40719c0 tx=0x7f1af0019000 comp rx=0 tx=0).stop 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 shutdown_connections 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ad8066f30 0x7f1ad80693e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1af413faa0 0x7f1af40c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af406aad0 0x7f1af41699c0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 --2- 172.21.15.33:0/1403292399 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1af40c7940 0x7f1af40c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 >> 172.21.15.33:0/1403292399 conn(0x7f1af406d030 msgr2=0x7f1af413e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 shutdown_connections 2023-10-06T05:28:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:48.340+0000 7f1afab9d700 1 -- 172.21.15.33:0/1403292399 wait complete. 2023-10-06T05:28:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:48.359 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:49.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:49.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.554+0000 7f335759e700 1 Processor -- start 2023-10-06T05:28:49.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.554+0000 7f335759e700 1 -- start start 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3358155af0 0x7f335806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f33580c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335806ab60 con 0x7f335806a6a0 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335806b7e0 con 0x7f3358155af0 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335806ae40 con 0x7f33580c7920 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f3355d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f335806a6a0 0x7f33580c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47082/0 (socket says 172.21.15.33:47082) 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f3355d9b700 1 -- 172.21.15.33:0/4203202282 learned_addr learned my addr 172.21.15.33:0/4203202282 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f3356d9d700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f33580c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335659c700 1 --2- 172.21.15.33:0/4203202282 >> v2:172.21.15.167:3300/0 conn(0x7f3358155af0 0x7f335806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570054747 0 0) 0x7f335806ab60 con 0x7f335806a6a0 2023-10-06T05:28:49.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.556+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3340004680 con 0x7f335806a6a0 2023-10-06T05:28:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704007343 0 0) 0x7f3340004680 con 0x7f335806a6a0 2023-10-06T05:28:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 msgr2=0x7f33580c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f33580c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 >> v2:172.21.15.167:3300/0 conn(0x7f3358155af0 msgr2=0x7f335806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 --2- 172.21.15.33:0/4203202282 >> v2:172.21.15.167:3300/0 conn(0x7f3358155af0 0x7f335806a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335806b120 con 0x7f335806a6a0 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f334c001470 con 0x7f335806a6a0 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 >> v1:172.21.15.33:6789/0 conn(0x7f335806a6a0 legacy=0x7f33580c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f3340004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33400051c0 0x7f3340005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3340005e60 0x7f33400062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335806b7e0 con 0x7f33400051c0 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335806ae40 con 0x7f3340005e60 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334000cc70 con 0x7f33580c7920 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f3355d9b700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33400051c0 0x7f3340005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f3356d9d700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3340005e60 0x7f33400062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.557+0000 7f335659c700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f3340004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f3355d9b700 1 -- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 msgr2=0x7f3340004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f3355d9b700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f3340004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f3355d9b700 1 -- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3340005e60 msgr2=0x7f33400062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f3355d9b700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3340005e60 0x7f33400062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f3355d9b700 1 -- 172.21.15.33:0/4203202282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340004680 con 0x7f33400051c0 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f335659c700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33580c7920 0x7f3340004bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f3355d9b700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33400051c0 0x7f3340005920 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f335806aa90 tx=0x7f334c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5e249a75f50541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00cb40 con 0x7f33400051c0 2023-10-06T05:28:49.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c00cca0 con 0x7f33400051c0 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f335559a700 1 -- 172.21.15.33:0/4203202282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c005b70 con 0x7f33400051c0 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f335759e700 1 -- 172.21.15.33:0/4203202282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33400051c0 msgr2=0x7f3340005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.558+0000 7f335759e700 1 --2- 172.21.15.33:0/4203202282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33400051c0 0x7f3340005920 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f335806aa90 tx=0x7f334c001dc0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.559+0000 7f3356d9d700 1 -- 172.21.15.33:0/4203202282 reap_dead start 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.559+0000 7f335759e700 1 -- 172.21.15.33:0/4203202282 shutdown_connections 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.559+0000 7f335759e700 1 -- 172.21.15.33:0/4203202282 >> 172.21.15.33:0/4203202282 conn(0x7f335806d030 msgr2=0x7f335806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.559+0000 7f335759e700 1 -- 172.21.15.33:0/4203202282 shutdown_connections 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.559+0000 7f335759e700 1 -- 172.21.15.33:0/4203202282 wait complete. 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.560+0000 7f335759e700 1 Processor -- start 2023-10-06T05:28:49.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.560+0000 7f335759e700 1 -- start start 2023-10-06T05:28:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f335759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33400051c0 0x7f33580ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f335759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335816a930 0x7f335814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f335759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 0x7f33580c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f335759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580c7580 con 0x7f33580c6bd0 2023-10-06T05:28:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f335759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580ad220 con 0x7f335816a930 2023-10-06T05:28:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f335759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33580afb00 con 0x7f33400051c0 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f3356d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 0x7f33580c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f3356d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 0x7f33580c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37354/0 (socket says 172.21.15.33:37354) 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.561+0000 7f3356d9d700 1 -- 172.21.15.33:0/3117767781 learned_addr learned my addr 172.21.15.33:0/3117767781 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f335659c700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33400051c0 0x7f33580ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3356d9d700 1 -- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33400051c0 msgr2=0x7f33580ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3356d9d700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33400051c0 0x7f33580ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3355d9b700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335816a930 0x7f335814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3356d9d700 1 -- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335816a930 msgr2=0x7f335814ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3356d9d700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335816a930 0x7f335814ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3356d9d700 1 -- 172.21.15.33:0/3117767781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33580acf40 con 0x7f33580c6bd0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3355d9b700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335816a930 0x7f335814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f3356d9d700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 0x7f33580c7040 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f33580c7d20 tx=0x7f335000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335000e040 con 0x7f33580c6bd0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335000f040 con 0x7f33580c6bd0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335000fe50 con 0x7f33580c6bd0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33580af4b0 con 0x7f33580c6bd0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.562+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33580acc50 con 0x7f33580c6bd0 2023-10-06T05:28:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.563+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335804c380 con 0x7f33580c6bd0 2023-10-06T05:28:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.564+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3350015640 con 0x7f33580c6bd0 2023-10-06T05:28:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.565+0000 7f33467fc700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3338066f30 0x7f33380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.565+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f3350085140 con 0x7f33580c6bd0 2023-10-06T05:28:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.566+0000 7f335659c700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3338066f30 0x7f33380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.566+0000 7f335659c700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3338066f30 0x7f33380693e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3348009d60 tx=0x7f3348008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.570+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33500520e0 con 0x7f33580c6bd0 2023-10-06T05:28:49.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.818+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33580afc40 con 0x7f33580c6bd0 2023-10-06T05:28:49.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.819+0000 7f33467fc700 1 -- 172.21.15.33:0/3117767781 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f3350055700 con 0x7f33580c6bd0 2023-10-06T05:28:49.821 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:49.821 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:49.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3338066f30 msgr2=0x7f33380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3338066f30 0x7f33380693e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3348009d60 tx=0x7f3348008040 comp rx=0 tx=0).stop 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 msgr2=0x7f33580c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 0x7f33580c7040 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f33580c7d20 tx=0x7f335000b040 comp rx=0 tx=0).stop 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 shutdown_connections 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3338066f30 0x7f33380693e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33580c6bd0 0x7f33580c7040 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33400051c0 0x7f33580ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 --2- 172.21.15.33:0/3117767781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f335816a930 0x7f335814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 >> 172.21.15.33:0/3117767781 conn(0x7f335806d030 msgr2=0x7f3358159100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 shutdown_connections 2023-10-06T05:28:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:49.821+0000 7f335759e700 1 -- 172.21.15.33:0/3117767781 wait complete. 2023-10-06T05:28:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:49.840 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:50.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:50.348 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:50.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:51.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.030+0000 7f11f7fff700 1 Processor -- start 2023-10-06T05:28:51.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.030+0000 7f11f7fff700 1 -- start start 2023-10-06T05:28:51.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.031+0000 7f11f7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f11f80ccfc0 0x7f11f80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.031+0000 7f11f7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11f80cf3b0 0x7f11f81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80cc750 0x7f11f81599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80cbd80 con 0x7f11f80cf3b0 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80af5f0 con 0x7f11f80ccfc0 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80af8d0 con 0x7f11f80cc750 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11f80cf3b0 0x7f11f81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11f80cf3b0 0x7f11f81594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38140/0 (socket says 172.21.15.33:38140) 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 -- 172.21.15.33:0/1424083460 learned_addr learned my addr 172.21.15.33:0/1424083460 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/1424083460 >> v2:172.21.15.167:3300/0 conn(0x7f11f80ccfc0 0x7f11f80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f77fe700 1 --2- 172.21.15.33:0/1424083460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80cc750 0x7f11f81599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 -- 172.21.15.33:0/1424083460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80cc750 msgr2=0x7f11f81599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 --2- 172.21.15.33:0/1424083460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80cc750 0x7f11f81599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 -- 172.21.15.33:0/1424083460 >> v2:172.21.15.167:3300/0 conn(0x7f11f80ccfc0 msgr2=0x7f11f80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 --2- 172.21.15.33:0/1424083460 >> v2:172.21.15.167:3300/0 conn(0x7f11f80ccfc0 0x7f11f80cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f67fc700 1 -- 172.21.15.33:0/1424083460 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80afbb0 con 0x7f11f80cf3b0 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.032+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/1424083460 >> v2:172.21.15.167:3300/0 conn(0x7f11f80ccfc0 0x7f11f80cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f67fc700 1 --2- 172.21.15.33:0/1424083460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f80cf3b0 0x7f11f81594a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0099b0 tx=0x7f11ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3784d6eb86615ed2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f5ffb700 1 -- 172.21.15.33:0/1424083460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00b820 con 0x7f11f80cf3b0 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f5ffb700 1 -- 172.21.15.33:0/1424083460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec00b980 con 0x7f11f80cf3b0 2023-10-06T05:28:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f5ffb700 1 -- 172.21.15.33:0/1424083460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00bd20 con 0x7f11f80cf3b0 2023-10-06T05:28:51.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f7fff700 1 -- 172.21.15.33:0/1424083460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f80cf3b0 msgr2=0x7f11f81594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:51.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f7fff700 1 --2- 172.21.15.33:0/1424083460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f80cf3b0 0x7f11f81594a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0099b0 tx=0x7f11ec009250 comp rx=0 tx=0).stop 2023-10-06T05:28:51.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f7fff700 1 -- 172.21.15.33:0/1424083460 shutdown_connections 2023-10-06T05:28:51.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f7fff700 1 --2- 172.21.15.33:0/1424083460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80cc750 0x7f11f81599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f7fff700 1 --2- 172.21.15.33:0/1424083460 >> v2:172.21.15.167:3300/0 conn(0x7f11f80ccfc0 0x7f11f80cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.033+0000 7f11f7fff700 1 --2- 172.21.15.33:0/1424083460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f80cf3b0 0x7f11f81594a0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.034+0000 7f11f7fff700 1 -- 172.21.15.33:0/1424083460 >> 172.21.15.33:0/1424083460 conn(0x7f11f806cff0 msgr2=0x7f11f806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:51.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.034+0000 7f11f7fff700 1 -- 172.21.15.33:0/1424083460 shutdown_connections 2023-10-06T05:28:51.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.034+0000 7f11f7fff700 1 -- 172.21.15.33:0/1424083460 wait complete. 2023-10-06T05:28:51.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.035+0000 7f11f7fff700 1 Processor -- start 2023-10-06T05:28:51.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.035+0000 7f11f7fff700 1 -- start start 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80cc750 0x7f11f80b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80ccfc0 0x7f11f80b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 0x7f11f8154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b2a50 con 0x7f11f8154b70 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b1370 con 0x7f11f80cc750 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8155510 con 0x7f11f80ccfc0 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.036+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 0x7f11f8154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 0x7f11f8154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38152/0 (socket says 172.21.15.33:38152) 2023-10-06T05:28:51.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 -- 172.21.15.33:0/2249367162 learned_addr learned my addr 172.21.15.33:0/2249367162 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80cc750 0x7f11f80b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f67fc700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80ccfc0 0x7f11f80b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 -- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80ccfc0 msgr2=0x7f11f80b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80ccfc0 0x7f11f80b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 -- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80cc750 msgr2=0x7f11f80b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80cc750 0x7f11f80b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 -- 172.21.15.33:0/2249367162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0094f0 con 0x7f11f8154b70 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f67fc700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80ccfc0 0x7f11f80b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11f77fe700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 0x7f11f8154fd0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fa0 tx=0x7f11f0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.037+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000ec10 con 0x7f11f8154b70 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.038+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000ed70 con 0x7f11f8154b70 2023-10-06T05:28:51.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.038+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0014670 con 0x7f11f8154b70 2023-10-06T05:28:51.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.038+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f80b1dd0 con 0x7f11f8154b70 2023-10-06T05:28:51.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.038+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80b2390 con 0x7f11f8154b70 2023-10-06T05:28:51.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.039+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f804c380 con 0x7f11f8154b70 2023-10-06T05:28:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.039+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11f00149e0 con 0x7f11f8154b70 2023-10-06T05:28:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.041+0000 7f11deffd700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f30 0x7f11e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.041+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f11f0084e90 con 0x7f11f8154b70 2023-10-06T05:28:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.041+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f30 0x7f11e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.042+0000 7f11f6ffd700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f30 0x7f11e40693e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006f00 tx=0x7f11e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:51.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.045+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f0051e30 con 0x7f11f8154b70 2023-10-06T05:28:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.292+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11f8158eb0 con 0x7f11f8154b70 2023-10-06T05:28:51.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.292+0000 7f11deffd700 1 -- 172.21.15.33:0/2249367162 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f11f0086090 con 0x7f11f8154b70 2023-10-06T05:28:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:51.294 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f30 msgr2=0x7f11e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f30 0x7f11e40693e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006f00 tx=0x7f11e8008040 comp rx=0 tx=0).stop 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 msgr2=0x7f11f8154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 0x7f11f8154fd0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fa0 tx=0x7f11f0009900 comp rx=0 tx=0).stop 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 shutdown_connections 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11e4066f30 0x7f11e40693e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f8154b70 0x7f11f8154fd0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f80cc750 0x7f11f80b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 --2- 172.21.15.33:0/2249367162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f80ccfc0 0x7f11f80b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 >> 172.21.15.33:0/2249367162 conn(0x7f11f806cff0 msgr2=0x7f11f81560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 shutdown_connections 2023-10-06T05:28:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:51.295+0000 7f11f7fff700 1 -- 172.21.15.33:0/2249367162 wait complete. 2023-10-06T05:28:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:51.314 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:52.315 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.512+0000 7f2f9ed9d700 1 Processor -- start 2023-10-06T05:28:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.512+0000 7f2f9ed9d700 1 -- start start 2023-10-06T05:28:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.513+0000 7f2f9ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f980ccfc0 0x7f2f980cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.513+0000 7f2f9ed9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2f980cf3b0 0x7f2f981594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.513+0000 7f2f9ed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980cbd80 con 0x7f2f980cc750 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.513+0000 7f2f9ed9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980af5f0 con 0x7f2f980ccfc0 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.513+0000 7f2f9ed9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980af8d0 con 0x7f2f980cf3b0 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.513+0000 7f2f9dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f980ccfc0 0x7f2f980cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2f980ccfc0 0x7f2f980cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38156/0 (socket says 172.21.15.33:38156) 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 -- 172.21.15.33:0/2449162936 learned_addr learned my addr 172.21.15.33:0/2449162936 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9d59a700 1 --2- 172.21.15.33:0/2449162936 >> v2:172.21.15.167:3300/0 conn(0x7f2f980cf3b0 0x7f2f981594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 -- 172.21.15.33:0/2449162936 >> v2:172.21.15.167:3300/0 conn(0x7f2f980cf3b0 msgr2=0x7f2f981594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 --2- 172.21.15.33:0/2449162936 >> v2:172.21.15.167:3300/0 conn(0x7f2f980cf3b0 0x7f2f981594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 -- 172.21.15.33:0/2449162936 >> v1:172.21.15.167:6789/0 conn(0x7f2f980cc750 legacy=0x7f2f980b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 -- 172.21.15.33:0/2449162936 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980afbb0 con 0x7f2f980ccfc0 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9dd9b700 1 --2- 172.21.15.33:0/2449162936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f980ccfc0 0x7f2f980cc210 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f2f940099b0 tx=0x7f2f94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6a8e9fd3cdc3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.514+0000 7f2f9cd99700 1 -- 172.21.15.33:0/2449162936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400b7e0 con 0x7f2f980ccfc0 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9cd99700 1 -- 172.21.15.33:0/2449162936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9400b940 con 0x7f2f980ccfc0 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9cd99700 1 -- 172.21.15.33:0/2449162936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400bce0 con 0x7f2f980ccfc0 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2449162936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f980ccfc0 msgr2=0x7f2f980cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2449162936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f980ccfc0 0x7f2f980cc210 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f2f940099b0 tx=0x7f2f94009250 comp rx=0 tx=0).stop 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2449162936 shutdown_connections 2023-10-06T05:28:52.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2449162936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f980ccfc0 0x7f2f980cc210 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2449162936 >> v2:172.21.15.167:3300/0 conn(0x7f2f980cf3b0 0x7f2f981594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2449162936 >> 172.21.15.33:0/2449162936 conn(0x7f2f9806cff0 msgr2=0x7f2f9806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:52.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2449162936 shutdown_connections 2023-10-06T05:28:52.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.515+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2449162936 wait complete. 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.517+0000 7f2f9ed9d700 1 Processor -- start 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.517+0000 7f2f9ed9d700 1 -- start start 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f980cc750 0x7f2f980c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f980ccfc0 0x7f2f9814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 0x7f2f980ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9ed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980ca430 con 0x7f2f9816dab0 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9ed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9814fa20 con 0x7f2f980ccfc0 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980b3660 con 0x7f2f980cc750 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 0x7f2f980ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:52.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 0x7f2f980ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38170/0 (socket says 172.21.15.33:38170) 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 -- 172.21.15.33:0/2992494530 learned_addr learned my addr 172.21.15.33:0/2992494530 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9dd9b700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f980cc750 0x7f2f980c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9d59a700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f980ccfc0 0x7f2f9814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 -- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f980cc750 msgr2=0x7f2f980c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f980cc750 0x7f2f980c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 -- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f980ccfc0 msgr2=0x7f2f9814f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f980ccfc0 0x7f2f9814f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9e59c700 1 -- 172.21.15.33:0/2992494530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940094f0 con 0x7f2f9816dab0 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.518+0000 7f2f9d59a700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f980ccfc0 0x7f2f9814f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:52.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.519+0000 7f2f9e59c700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 0x7f2f980ce520 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c930 tx=0x7f2f900026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:52.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.519+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000ce70 con 0x7f2f9816dab0 2023-10-06T05:28:52.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.519+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9000a7e0 con 0x7f2f9816dab0 2023-10-06T05:28:52.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.519+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000aba0 con 0x7f2f9816dab0 2023-10-06T05:28:52.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.519+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980b3940 con 0x7f2f9816dab0 2023-10-06T05:28:52.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.519+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980b3f00 con 0x7f2f9816dab0 2023-10-06T05:28:52.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.520+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f98157820 con 0x7f2f9816dab0 2023-10-06T05:28:52.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.521+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f90012420 con 0x7f2f9816dab0 2023-10-06T05:28:52.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.522+0000 7f2f8dffb700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f7c066f30 0x7f2f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:52.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.522+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f2f90084c90 con 0x7f2f9816dab0 2023-10-06T05:28:52.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.522+0000 7f2f9dd9b700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f7c066f30 0x7f2f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:52.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.523+0000 7f2f9dd9b700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f7c066f30 0x7f2f7c0693e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2f940099b0 tx=0x7f2f94016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:52.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.526+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f90051ce0 con 0x7f2f9816dab0 2023-10-06T05:28:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.775+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f9804c380 con 0x7f2f9816dab0 2023-10-06T05:28:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.775+0000 7f2f8dffb700 1 -- 172.21.15.33:0/2992494530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f2f90016090 con 0x7f2f9816dab0 2023-10-06T05:28:52.777 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:52.777 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f7c066f30 msgr2=0x7f2f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f7c066f30 0x7f2f7c0693e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2f940099b0 tx=0x7f2f94016000 comp rx=0 tx=0).stop 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 msgr2=0x7f2f980ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 0x7f2f980ce520 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c930 tx=0x7f2f900026e0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 shutdown_connections 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2f7c066f30 0x7f2f7c0693e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f9816dab0 0x7f2f980ce520 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f980cc750 0x7f2f980c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 --2- 172.21.15.33:0/2992494530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f980ccfc0 0x7f2f9814f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 >> 172.21.15.33:0/2992494530 conn(0x7f2f9806cff0 msgr2=0x7f2f98157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 shutdown_connections 2023-10-06T05:28:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:52.778+0000 7f2f9ed9d700 1 -- 172.21.15.33:0/2992494530 wait complete. 2023-10-06T05:28:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:52.797 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:53.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:53.999+0000 7f9fa1f9d700 1 Processor -- start 2023-10-06T05:28:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:53.999+0000 7f9fa1f9d700 1 -- start start 2023-10-06T05:28:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.000+0000 7f9fa1f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9f9c159bb0 0x7f9f9c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.000+0000 7f9fa1f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c0cbd80 con 0x7f9f9c0cf3b0 2023-10-06T05:28:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.000+0000 7f9fa1f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c0af5f0 con 0x7f9f9c0ccfc0 2023-10-06T05:28:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.000+0000 7f9fa1f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c0af8d0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9f9bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9f9c0cf3b0 0x7f9f9c1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48512/0 (socket says 172.21.15.33:48512) 2023-10-06T05:28:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9f9bfff700 1 -- 172.21.15.33:0/24188979 learned_addr learned my addr 172.21.15.33:0/24188979 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9fa179c700 1 --2- 172.21.15.33:0/24188979 >> v2:172.21.15.33:3300/0 conn(0x7f9f9c159bb0 0x7f9f9c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9f9b7fe700 1 -- 172.21.15.33:0/24188979 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127717454 0 0) 0x7f9f9c0cbd80 con 0x7f9f9c0cf3b0 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9f9b7fe700 1 -- 172.21.15.33:0/24188979 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c003570 con 0x7f9f9c0cf3b0 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9fa179c700 1 -- 172.21.15.33:0/24188979 >> v1:172.21.15.167:6789/0 conn(0x7f9f9c0ccfc0 legacy=0x7f9f9c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9fa179c700 1 -- 172.21.15.33:0/24188979 >> v1:172.21.15.33:6789/0 conn(0x7f9f9c0cf3b0 legacy=0x7f9f9c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9fa179c700 1 -- 172.21.15.33:0/24188979 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c0afbb0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.001+0000 7f9fa179c700 1 --2- 172.21.15.33:0/24188979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b0460 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009e90 tx=0x7f9f94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6adee092f31435 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9f9b7fe700 1 -- 172.21.15.33:0/24188979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400e070 con 0x7f9f9c159bb0 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9f9b7fe700 1 -- 172.21.15.33:0/24188979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9400bc40 con 0x7f9f9c159bb0 2023-10-06T05:28:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9f9b7fe700 1 -- 172.21.15.33:0/24188979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94012430 con 0x7f9f9c159bb0 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/24188979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 msgr2=0x7f9f9c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/24188979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b0460 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009e90 tx=0x7f9f94009780 comp rx=0 tx=0).stop 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/24188979 shutdown_connections 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/24188979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b0460 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/24188979 >> 172.21.15.33:0/24188979 conn(0x7f9f9c06cff0 msgr2=0x7f9f9c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/24188979 shutdown_connections 2023-10-06T05:28:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.002+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/24188979 wait complete. 2023-10-06T05:28:54.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.004+0000 7f9fa1f9d700 1 Processor -- start 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.004+0000 7f9fa1f9d700 1 -- start start 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f9c0ccfc0 0x7f9f9c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa1f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa1f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f9c16e760 0x7f9f9c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa1f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c16e2d0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa1f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c16dcc0 con 0x7f9f9c16e760 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0b5f10 con 0x7f9f9c0ccfc0 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9f9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9f9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38198/0 (socket says 172.21.15.33:38198) 2023-10-06T05:28:54.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9f9bfff700 1 -- 172.21.15.33:0/962061315 learned_addr learned my addr 172.21.15.33:0/962061315 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa179c700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f9c16e760 0x7f9f9c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.005+0000 7f9fa0f9b700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f9c0ccfc0 0x7f9f9c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f9bfff700 1 -- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f9c0ccfc0 msgr2=0x7f9f9c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f9bfff700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f9c0ccfc0 0x7f9f9c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f9bfff700 1 -- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f9c16e760 msgr2=0x7f9f9c151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f9bfff700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f9c16e760 0x7f9f9c151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f9bfff700 1 -- 172.21.15.33:0/962061315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94009a20 con 0x7f9f9c159bb0 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f9bfff700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b5420 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800c790 tx=0x7f9f88007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88005c00 con 0x7f9f9c159bb0 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f88005d60 con 0x7f9f9c159bb0 2023-10-06T05:28:54.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880072e0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c16df60 con 0x7f9f9c159bb0 2023-10-06T05:28:54.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.006+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c0b3e90 con 0x7f9f9c159bb0 2023-10-06T05:28:54.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.007+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c04c380 con 0x7f9f9c159bb0 2023-10-06T05:28:54.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.008+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f8801b3b0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.009+0000 7f9f98ff9700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:54.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.010+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9f88083c20 con 0x7f9f9c159bb0 2023-10-06T05:28:54.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.010+0000 7f9fa0f9b700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:54.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.011+0000 7f9fa0f9b700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9f900037c0 tx=0x7f9f90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:54.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.014+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f8804b5a0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.264+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f9c0cc9b0 con 0x7f9f9c159bb0 2023-10-06T05:28:54.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.265+0000 7f9f98ff9700 1 -- 172.21.15.33:0/962061315 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f9f88017020 con 0x7f9f9c159bb0 2023-10-06T05:28:54.266 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:54.267 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":152,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:06.241759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f80066f30 msgr2=0x7f9f800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f80066f30 0x7f9f800693e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9f900037c0 tx=0x7f9f90009040 comp rx=0 tx=0).stop 2023-10-06T05:28:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 msgr2=0x7f9f9c0b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b5420 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800c790 tx=0x7f9f88007db0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 shutdown_connections 2023-10-06T05:28:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f80066f30 0x7f9f800693e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f9c16e760 0x7f9f9c151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f9c0ccfc0 0x7f9f9c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 --2- 172.21.15.33:0/962061315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f9c159bb0 0x7f9f9c0b5420 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 >> 172.21.15.33:0/962061315 conn(0x7f9f9c06cff0 msgr2=0x7f9f9c152550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 shutdown_connections 2023-10-06T05:28:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:54.267+0000 7f9fa1f9d700 1 -- 172.21.15.33:0/962061315 wait complete. 2023-10-06T05:28:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 152 2023-10-06T05:28:54.286 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:28:55.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.484+0000 7f253b39d700 1 Processor -- start 2023-10-06T05:28:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.485+0000 7f253b39d700 1 -- start start 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f253b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f253413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f253b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340c9cb0 con 0x7f253413faa0 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f253b39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340ca270 con 0x7f253406aad0 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f253b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253406b660 con 0x7f25340c8050 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f253a39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f253413faa0 0x7f25340c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48528/0 (socket says 172.21.15.33:48528) 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f253a39b700 1 -- 172.21.15.33:0/1610357730 learned_addr learned my addr 172.21.15.33:0/1610357730 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.486+0000 7f2539b9a700 1 --2- 172.21.15.33:0/1610357730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f253413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760061214 0 0) 0x7f25340c9cb0 con 0x7f253413faa0 2023-10-06T05:28:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2520003570 con 0x7f253413faa0 2023-10-06T05:28:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351490593 0 0) 0x7f25340ca270 con 0x7f253406aad0 2023-10-06T05:28:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25340c9cb0 con 0x7f253406aad0 2023-10-06T05:28:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539b9a700 1 -- 172.21.15.33:0/1610357730 >> v1:172.21.15.167:6789/0 conn(0x7f253406aad0 legacy=0x7f253406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726014208 0 0) 0x7f2520003570 con 0x7f253413faa0 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539b9a700 1 -- 172.21.15.33:0/1610357730 >> v1:172.21.15.33:6789/0 conn(0x7f253413faa0 legacy=0x7f25340c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539b9a700 1 -- 172.21.15.33:0/1610357730 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253406b840 con 0x7f25340c8050 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539b9a700 1 --2- 172.21.15.33:0/1610357730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f253413fe70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2524009910 tx=0x7f25240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe334dfe3bc19df3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.487+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530001470 con 0x7f25340c8050 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2524011420 con 0x7f25340c8050 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f2539399700 1 -- 172.21.15.33:0/1610357730 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240117c0 con 0x7f25340c8050 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 -- 172.21.15.33:0/1610357730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 msgr2=0x7f253413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 --2- 172.21.15.33:0/1610357730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f253413fe70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2524009910 tx=0x7f25240091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 -- 172.21.15.33:0/1610357730 shutdown_connections 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 --2- 172.21.15.33:0/1610357730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f253413fe70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 -- 172.21.15.33:0/1610357730 >> 172.21.15.33:0/1610357730 conn(0x7f253406d030 msgr2=0x7f253406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:55.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 -- 172.21.15.33:0/1610357730 shutdown_connections 2023-10-06T05:28:55.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.488+0000 7f253b39d700 1 -- 172.21.15.33:0/1610357730 wait complete. 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.490+0000 7f253b39d700 1 Processor -- start 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.490+0000 7f253b39d700 1 -- start start 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253406aad0 0x7f253413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f25340c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 0x7f25340c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340c4d20 con 0x7f253413faa0 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340c5000 con 0x7f253406aad0 2023-10-06T05:28:55.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340c52e0 con 0x7f25340c8050 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 0x7f25340c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 0x7f25340c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38208/0 (socket says 172.21.15.33:38208) 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253ab9c700 1 -- 172.21.15.33:0/1916157557 learned_addr learned my addr 172.21.15.33:0/1916157557 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f253a39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253406aad0 0x7f253413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.491+0000 7f2539b9a700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f25340c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253ab9c700 1 -- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 msgr2=0x7f25340c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253ab9c700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f25340c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253ab9c700 1 -- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253406aad0 msgr2=0x7f253413ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253ab9c700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253406aad0 0x7f253413ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253ab9c700 1 -- 172.21.15.33:0/1916157557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009420 con 0x7f253413faa0 2023-10-06T05:28:55.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253ab9c700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 0x7f25340c8b30 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f252c00c910 tx=0x7f252c0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:55.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c012630 con 0x7f253413faa0 2023-10-06T05:28:55.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c0127b0 con 0x7f253413faa0 2023-10-06T05:28:55.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c012b70 con 0x7f253413faa0 2023-10-06T05:28:55.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25340c55c0 con 0x7f253413faa0 2023-10-06T05:28:55.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.492+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25340c5dc0 con 0x7f253413faa0 2023-10-06T05:28:55.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.493+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25340c5ae0 con 0x7f253413faa0 2023-10-06T05:28:55.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.494+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f252c01b400 con 0x7f253413faa0 2023-10-06T05:28:55.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.495+0000 7f252a7fc700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2518066f30 0x7f25180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:55.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.495+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f252c084bc0 con 0x7f253413faa0 2023-10-06T05:28:55.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.496+0000 7f253a39b700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2518066f30 0x7f25180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:55.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.496+0000 7f253a39b700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2518066f30 0x7f25180693e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f25300011e0 tx=0x7f2530000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:55.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.500+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25340c5ae0 con 0x7f253413faa0 2023-10-06T05:28:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.746+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25340c5ae0 con 0x7f253413faa0 2023-10-06T05:28:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.747+0000 7f252a7fc700 1 -- 172.21.15.33:0/1916157557 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f25340c5ae0 con 0x7f253413faa0 2023-10-06T05:28:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:55.749 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":153,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:54.978514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.749+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2518066f30 msgr2=0x7f25180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.749+0000 7f253b39d700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2518066f30 0x7f25180693e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f25300011e0 tx=0x7f2530000bc0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.749+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 msgr2=0x7f25340c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.749+0000 7f253b39d700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 0x7f25340c8b30 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f252c00c910 tx=0x7f252c0027a0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 shutdown_connections 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2518066f30 0x7f25180693e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f253413faa0 0x7f25340c8b30 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253406aad0 0x7f253413ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 --2- 172.21.15.33:0/1916157557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25340c8050 0x7f25340c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 >> 172.21.15.33:0/1916157557 conn(0x7f253406d030 msgr2=0x7f25340c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 shutdown_connections 2023-10-06T05:28:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:55.750+0000 7f253b39d700 1 -- 172.21.15.33:0/1916157557 wait complete. 2023-10-06T05:28:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 153 2023-10-06T05:28:55.769 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2023-10-06T05:28:56.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.968+0000 7fbbd299d700 1 Processor -- start 2023-10-06T05:28:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.968+0000 7fbbd299d700 1 -- start start 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd299d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbbcc155af0 0x7fbbcc06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd299d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbbcc0c7920 0x7fbbcc0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd299d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc06ab60 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd299d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc06b7e0 con 0x7fbbcc0c7920 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd299d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc06ae40 con 0x7fbbcc155af0 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbbcc0c7920 0x7fbbcc0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd119a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbbcc06a6a0 0x7fbbcc0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48532/0 (socket says 172.21.15.33:48532) 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd119a700 1 -- 172.21.15.33:0/3782370480 learned_addr learned my addr 172.21.15.33:0/3782370480 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.970+0000 7fbbd199b700 1 --2- 172.21.15.33:0/3782370480 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc155af0 0x7fbbcc06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd219c700 1 -- 172.21.15.33:0/3782370480 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc155af0 msgr2=0x7fbbcc06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd219c700 1 --2- 172.21.15.33:0/3782370480 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc155af0 0x7fbbcc06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd219c700 1 -- 172.21.15.33:0/3782370480 >> v1:172.21.15.33:6789/0 conn(0x7fbbcc06a6a0 legacy=0x7fbbcc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd219c700 1 -- 172.21.15.33:0/3782370480 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc06b120 con 0x7fbbcc0c7920 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd219c700 1 --2- 172.21.15.33:0/3782370480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc0c7920 0x7fbbcc0c9e20 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0697c0 tx=0x7fbbc4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f386395414265a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd0999700 1 -- 172.21.15.33:0/3782370480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400bb60 con 0x7fbbcc0c7920 2023-10-06T05:28:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd0999700 1 -- 172.21.15.33:0/3782370480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc400bcc0 con 0x7fbbcc0c7920 2023-10-06T05:28:56.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.971+0000 7fbbd0999700 1 -- 172.21.15.33:0/3782370480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40116b0 con 0x7fbbcc0c7920 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 -- 172.21.15.33:0/3782370480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc0c7920 msgr2=0x7fbbcc0c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 --2- 172.21.15.33:0/3782370480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc0c7920 0x7fbbcc0c9e20 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0697c0 tx=0x7fbbc4009780 comp rx=0 tx=0).stop 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 -- 172.21.15.33:0/3782370480 shutdown_connections 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 --2- 172.21.15.33:0/3782370480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc0c7920 0x7fbbcc0c9e20 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 --2- 172.21.15.33:0/3782370480 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc155af0 0x7fbbcc06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 -- 172.21.15.33:0/3782370480 >> 172.21.15.33:0/3782370480 conn(0x7fbbcc06d030 msgr2=0x7fbbcc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 -- 172.21.15.33:0/3782370480 shutdown_connections 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.972+0000 7fbbd299d700 1 -- 172.21.15.33:0/3782370480 wait complete. 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.973+0000 7fbbd299d700 1 Processor -- start 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.973+0000 7fbbd299d700 1 -- start start 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.974+0000 7fbbd299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 0x7fbbcc0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.974+0000 7fbbd299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc0c7920 0x7fbbcc0ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.974+0000 7fbbd299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc155af0 0x7fbbcc168f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.974+0000 7fbbd299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc0c3c00 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.974+0000 7fbbd299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc0ac900 con 0x7fbbcc155af0 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.974+0000 7fbbd299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc169450 con 0x7fbbcc0c7920 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 0x7fbbcc0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 0x7fbbcc0b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38220/0 (socket says 172.21.15.33:38220) 2023-10-06T05:28:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 -- 172.21.15.33:0/2483650542 learned_addr learned my addr 172.21.15.33:0/2483650542 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd119a700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc0c7920 0x7fbbcc0ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd219c700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc155af0 0x7fbbcc168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 -- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc0c7920 msgr2=0x7fbbcc0ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc0c7920 0x7fbbcc0ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 -- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc155af0 msgr2=0x7fbbcc168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc155af0 0x7fbbcc168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 -- 172.21.15.33:0/2483650542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4009a20 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd119a700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc0c7920 0x7fbbcc0ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd219c700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc155af0 0x7fbbcc168f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbd199b700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 0x7fbbcc0b0de0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800ea20 tx=0x7fbbc800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.975+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800a7a0 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.976+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc800c8f0 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.976+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc0b1480 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.976+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800cc90 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.976+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc0b1a40 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.977+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc04c380 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.978+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbbc80113f0 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.978+0000 7fbbc1ffb700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbb0066f30 0x7fbbb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.978+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fbbc8015030 con 0x7fbbcc06a6a0 2023-10-06T05:28:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.979+0000 7fbbd119a700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbb0066f30 0x7fbbb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.980+0000 7fbbd119a700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbb0066f30 0x7fbbb00693e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0cb010 tx=0x7fbbbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:56.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:56.983+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbc8055080 con 0x7fbbcc06a6a0 2023-10-06T05:28:57.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.230+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbcc06b400 con 0x7fbbcc06a6a0 2023-10-06T05:28:57.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.230+0000 7fbbc1ffb700 1 -- 172.21.15.33:0/2483650542 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fbbc8054c10 con 0x7fbbcc06a6a0 2023-10-06T05:28:57.232 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:57.232 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:57.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.232+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbb0066f30 msgr2=0x7fbbb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.232+0000 7fbbd299d700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbb0066f30 0x7fbbb00693e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0cb010 tx=0x7fbbbc009040 comp rx=0 tx=0).stop 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.232+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 msgr2=0x7fbbcc0b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.232+0000 7fbbd299d700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 0x7fbbcc0b0de0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800ea20 tx=0x7fbbc800a330 comp rx=0 tx=0).stop 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 shutdown_connections 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbbb0066f30 0x7fbbb00693e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc155af0 0x7fbbcc168f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a6a0 0x7fbbcc0b0de0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 --2- 172.21.15.33:0/2483650542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc0c7920 0x7fbbcc0ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 >> 172.21.15.33:0/2483650542 conn(0x7fbbcc06d030 msgr2=0x7fbbcc0ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 shutdown_connections 2023-10-06T05:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.233+0000 7fbbd299d700 1 -- 172.21.15.33:0/2483650542 wait complete. 2023-10-06T05:28:57.236 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:28:57.252 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:28:57.252 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 47.979270219802856s 2023-10-06T05:28:57.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.438+0000 7fee97fff700 1 Processor -- start 2023-10-06T05:28:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.438+0000 7fee97fff700 1 -- start start 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.439+0000 7fee97fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee980c8050 0x7fee9813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.439+0000 7fee97fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee980c9cb0 con 0x7fee9806aad0 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.439+0000 7fee97fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee980ca270 con 0x7fee9813faa0 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.439+0000 7fee97fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9806b640 con 0x7fee980c8050 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.439+0000 7fee977fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fee9806aad0 0x7fee9806af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48536/0 (socket says 172.21.15.33:48536) 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.439+0000 7fee977fe700 1 -- 172.21.15.33:0/502337795 learned_addr learned my addr 172.21.15.33:0/502337795 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:28:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee967fc700 1 --2- 172.21.15.33:0/502337795 >> v2:172.21.15.167:3300/0 conn(0x7fee980c8050 0x7fee9813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756114859 0 0) 0x7fee980c9cb0 con 0x7fee9806aad0 2023-10-06T05:28:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee80003570 con 0x7fee9806aad0 2023-10-06T05:28:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 521277194 0 0) 0x7fee80003570 con 0x7fee9806aad0 2023-10-06T05:28:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 >> v2:172.21.15.167:3300/0 conn(0x7fee980c8050 msgr2=0x7fee9813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 --2- 172.21.15.33:0/502337795 >> v2:172.21.15.167:3300/0 conn(0x7fee980c8050 0x7fee9813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 >> v1:172.21.15.167:6789/0 conn(0x7fee9813faa0 legacy=0x7fee980c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9806b840 con 0x7fee9806aad0 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fee900017f0 con 0x7fee9806aad0 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 >> v1:172.21.15.33:6789/0 conn(0x7fee9806aad0 legacy=0x7fee9806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee80003a60 0x7fee80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80004810 0x7fee80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee800051c0 0x7fee8000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9806b640 con 0x7fee800051c0 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8000c0d0 con 0x7fee80003a60 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.440+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8000c290 con 0x7fee80004810 2023-10-06T05:28:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee977fe700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee800051c0 0x7fee8000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee80003a60 0x7fee80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee967fc700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80004810 0x7fee80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 -- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80004810 msgr2=0x7fee80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80004810 0x7fee80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 -- 172.21.15.33:0/502337795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee800051c0 msgr2=0x7fee8000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee800051c0 0x7fee8000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 -- 172.21.15.33:0/502337795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80003570 con 0x7fee80003a60 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee977fe700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee800051c0 0x7fee8000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee977fe700 1 -- 172.21.15.33:0/502337795 reap_dead start 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.441+0000 7fee96ffd700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee80003a60 0x7fee80003e70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fee880018b0 tx=0x7fee88007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=726957cd7a634781 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88012420 con 0x7fee80003a60 2023-10-06T05:28:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee880125a0 con 0x7fee80003a60 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee95ffb700 1 -- 172.21.15.33:0/502337795 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88012960 con 0x7fee80003a60 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 -- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee80003a60 msgr2=0x7fee80003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee80003a60 0x7fee80003e70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fee880018b0 tx=0x7fee88007d30 comp rx=0 tx=0).stop 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 -- 172.21.15.33:0/502337795 shutdown_connections 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 --2- 172.21.15.33:0/502337795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee80003a60 0x7fee80003e70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 -- 172.21.15.33:0/502337795 >> 172.21.15.33:0/502337795 conn(0x7fee9806d030 msgr2=0x7fee9806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 -- 172.21.15.33:0/502337795 shutdown_connections 2023-10-06T05:28:57.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.442+0000 7fee97fff700 1 -- 172.21.15.33:0/502337795 wait complete. 2023-10-06T05:28:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.444+0000 7fee97fff700 1 Processor -- start 2023-10-06T05:28:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.444+0000 7fee97fff700 1 -- start start 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee97fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80003a60 0x7fee98138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 0x7fee981523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee97fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee981394a0 0x7fee98139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee97fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98152fe0 con 0x7fee980b0dd0 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee97fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98139e50 con 0x7fee981394a0 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee97fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980b0940 con 0x7fee80003a60 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 0x7fee981523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 0x7fee981523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38248/0 (socket says 172.21.15.33:38248) 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee967fc700 1 -- 172.21.15.33:0/4001283972 learned_addr learned my addr 172.21.15.33:0/4001283972 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee96ffd700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80003a60 0x7fee98138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.445+0000 7fee977fe700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee981394a0 0x7fee98139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee967fc700 1 -- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80003a60 msgr2=0x7fee98138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee967fc700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80003a60 0x7fee98138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee967fc700 1 -- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee981394a0 msgr2=0x7fee98139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee967fc700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee981394a0 0x7fee98139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee967fc700 1 -- 172.21.15.33:0/4001283972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98139190 con 0x7fee980b0dd0 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee96ffd700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80003a60 0x7fee98138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee977fe700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee981394a0 0x7fee98139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee967fc700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 0x7fee981523f0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fee8c011a40 tx=0x7fee8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00f940 con 0x7fee980b0dd0 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c00faa0 con 0x7fee980b0dd0 2023-10-06T05:28:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00fe20 con 0x7fee980b0dd0 2023-10-06T05:28:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980afd30 con 0x7fee980b0dd0 2023-10-06T05:28:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.446+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980b02f0 con 0x7fee980b0dd0 2023-10-06T05:28:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.447+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee980b0af0 con 0x7fee980b0dd0 2023-10-06T05:28:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.448+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee8c014650 con 0x7fee980b0dd0 2023-10-06T05:28:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.450+0000 7fee7effd700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee84066f30 0x7fee840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.450+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fee8c018030 con 0x7fee980b0dd0 2023-10-06T05:28:57.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.451+0000 7fee96ffd700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee84066f30 0x7fee840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:57.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.451+0000 7fee96ffd700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee84066f30 0x7fee840693e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fee88001690 tx=0x7fee88015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:57.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.454+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee980b0af0 con 0x7fee980b0dd0 2023-10-06T05:28:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.702+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee980b0ab0 con 0x7fee980b0dd0 2023-10-06T05:28:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.703+0000 7fee7effd700 1 -- 172.21.15.33:0/4001283972 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fee8c022090 con 0x7fee980b0dd0 2023-10-06T05:28:57.705 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:57.705 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:57.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee84066f30 msgr2=0x7fee840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee84066f30 0x7fee840693e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fee88001690 tx=0x7fee88015000 comp rx=0 tx=0).stop 2023-10-06T05:28:57.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 msgr2=0x7fee981523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:57.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 0x7fee981523f0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fee8c011a40 tx=0x7fee8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 shutdown_connections 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee84066f30 0x7fee840693e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee981394a0 0x7fee98139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee80003a60 0x7fee98138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 --2- 172.21.15.33:0/4001283972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee980b0dd0 0x7fee981523f0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 >> 172.21.15.33:0/4001283972 conn(0x7fee9806d030 msgr2=0x7fee9813abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 shutdown_connections 2023-10-06T05:28:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:57.705+0000 7fee97fff700 1 -- 172.21.15.33:0/4001283972 wait complete. 2023-10-06T05:28:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:28:57.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:28:57.938 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":7778,"entity":{"name":{"type":"client","num":7778},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1820721818}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":52.671170150000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.88747356174908276,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7778 v1:192.168.0.1:0/1820721818","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:28:57.938 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1820721818, "type": "v1" }, "name": { "num": 7778, "type": "client" } }, "id": 7778, "inst": "client.7778 v1:192.168.0.1:0/1820721818", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.8874735617490828 }, "state": "open", "uptime": 52.67117015 } ] 2023-10-06T05:28:57.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:28:58.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.128+0000 7f4f4b59e700 1 Processor -- start 2023-10-06T05:28:58.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.128+0000 7f4f4b59e700 1 -- start start 2023-10-06T05:28:58.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c155140 0x7f4f4c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f4c0699c0 0x7f4f4c069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0c8e40 con 0x7f4f4c0adf50 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c06a380 con 0x7f4f4c0699c0 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c99e0 con 0x7f4f4c155140 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f4c0699c0 0x7f4f4c069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.129+0000 7f4f4ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4f4c0699c0 0x7f4f4c069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38258/0 (socket says 172.21.15.33:38258) 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4ad9d700 1 -- 172.21.15.33:0/3095351572 learned_addr learned my addr 172.21.15.33:0/3095351572 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4a59c700 1 --2- 172.21.15.33:0/3095351572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c155140 0x7f4f4c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4ad9d700 1 -- 172.21.15.33:0/3095351572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c155140 msgr2=0x7f4f4c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:58.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4ad9d700 1 --2- 172.21.15.33:0/3095351572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c155140 0x7f4f4c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4ad9d700 1 -- 172.21.15.33:0/3095351572 >> v1:172.21.15.167:6789/0 conn(0x7f4f4c0adf50 legacy=0x7f4f4c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4ad9d700 1 -- 172.21.15.33:0/3095351572 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0ae6c0 con 0x7f4f4c0699c0 2023-10-06T05:28:58.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4ad9d700 1 --2- 172.21.15.33:0/3095351572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0699c0 0x7f4f4c069e40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009f20 tx=0x7f4f44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691fb2579b6e07de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4959a700 1 -- 172.21.15.33:0/3095351572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400e070 con 0x7f4f4c0699c0 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.130+0000 7f4f4959a700 1 -- 172.21.15.33:0/3095351572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4400ba80 con 0x7f4f4c0699c0 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4959a700 1 -- 172.21.15.33:0/3095351572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440092e0 con 0x7f4f4c0699c0 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 -- 172.21.15.33:0/3095351572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0699c0 msgr2=0x7f4f4c069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/3095351572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0699c0 0x7f4f4c069e40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009f20 tx=0x7f4f44009840 comp rx=0 tx=0).stop 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 -- 172.21.15.33:0/3095351572 shutdown_connections 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/3095351572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0699c0 0x7f4f4c069e40 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/3095351572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c155140 0x7f4f4c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 -- 172.21.15.33:0/3095351572 >> 172.21.15.33:0/3095351572 conn(0x7f4f4c06d030 msgr2=0x7f4f4c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 -- 172.21.15.33:0/3095351572 shutdown_connections 2023-10-06T05:28:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.131+0000 7f4f4b59e700 1 -- 172.21.15.33:0/3095351572 wait complete. 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.133+0000 7f4f4b59e700 1 Processor -- start 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.133+0000 7f4f4b59e700 1 -- start start 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c0699c0 0x7f4f4c0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 0x7f4f4c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f4c155140 0x7f4f4c0a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c5950 con 0x7f4f4c0adf50 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c4aa0 con 0x7f4f4c155140 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c4d40 con 0x7f4f4c0699c0 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f49d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 0x7f4f4c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:58.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f49d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 0x7f4f4c0c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38264/0 (socket says 172.21.15.33:38264) 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f49d9b700 1 -- 172.21.15.33:0/1194653856 learned_addr learned my addr 172.21.15.33:0/1194653856 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4a59c700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c0699c0 0x7f4f4c0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.134+0000 7f4f4ad9d700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f4c155140 0x7f4f4c0a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f49d9b700 1 -- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c0699c0 msgr2=0x7f4f4c0a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f49d9b700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c0699c0 0x7f4f4c0a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f49d9b700 1 -- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f4c155140 msgr2=0x7f4f4c0a6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f49d9b700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f4c155140 0x7f4f4c0a6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f49d9b700 1 -- 172.21.15.33:0/1194653856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009ae0 con 0x7f4f4c0adf50 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f49d9b700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 0x7f4f4c0c4560 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c590 tx=0x7f4f4000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40011590 con 0x7f4f4c0adf50 2023-10-06T05:28:58.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f40011710 con 0x7f4f4c0adf50 2023-10-06T05:28:58.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c0c5020 con 0x7f4f4c0adf50 2023-10-06T05:28:58.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40011ad0 con 0x7f4f4c0adf50 2023-10-06T05:28:58.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.135+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c0c55e0 con 0x7f4f4c0adf50 2023-10-06T05:28:58.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.136+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c04c380 con 0x7f4f4c0adf50 2023-10-06T05:28:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.137+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f4001b3b0 con 0x7f4f4c0adf50 2023-10-06T05:28:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.138+0000 7f4f3a7fc700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.139+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4f40083c50 con 0x7f4f4c0adf50 2023-10-06T05:28:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.139+0000 7f4f4a59c700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:58.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.140+0000 7f4f4a59c700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c006fd0 tx=0x7f4f3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:58.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.143+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f40050bd0 con 0x7f4f4c0adf50 2023-10-06T05:28:58.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.391+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f4c0c5a90 con 0x7f4f4c0adf50 2023-10-06T05:28:58.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.391+0000 7f4f3a7fc700 1 -- 172.21.15.33:0/1194653856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v154) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f40017070 con 0x7f4f4c0adf50 2023-10-06T05:28:58.393 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:28:58.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f2c066f30 msgr2=0x7f4f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c006fd0 tx=0x7f4f3c008040 comp rx=0 tx=0).stop 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 msgr2=0x7f4f4c0c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 0x7f4f4c0c4560 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c590 tx=0x7f4f4000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 shutdown_connections 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f4c155140 0x7f4f4c0a6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f4c0699c0 0x7f4f4c0a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 --2- 172.21.15.33:0/1194653856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f4c0adf50 0x7f4f4c0c4560 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 >> 172.21.15.33:0/1194653856 conn(0x7f4f4c06d030 msgr2=0x7f4f4c0aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 shutdown_connections 2023-10-06T05:28:58.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:58.394+0000 7f4f4b59e700 1 -- 172.21.15.33:0/1194653856 wait complete. 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:28:58.413 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:28:58.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:58.414 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.451 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41419sec preferred_lft 41419sec 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:28:58.452 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:28:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:28:58.454 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:28:58.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:58.455 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:28:58.553 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:28:58.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:58.555 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:28:58.594 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:28:58.594 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:28:58.594 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:28:58.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:28:58.594 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:28:58.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:58.595 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:28:58.595 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:28:58.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:28:58.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:28:58.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:28:58.596 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:28:58.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:58.794 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:28:58.831 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:28:58.831 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:28:58.831 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:28:58.831 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:28:58.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:58.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:28:58.852 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T05:28:58.850+0000 7ff1529f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:28:58.915 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:28:58.915 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:58.952 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:28:58.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:28:58.953 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:59.030 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:28:59.030 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:28:59.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:59.164 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:59.164 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:28:59.164 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:28:59.180 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:59.180 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:28:59.180 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:28:59.181 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:28:59.181 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:28:59.202 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:28:59.202 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:28:59.202 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:28:59.202 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:28:59.203 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:28:59.203 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:28:59.203 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:28:59.203 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:28:59.207 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:28:59.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:59.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:28:59.270 DEBUG:tasks.cephfs.mount:Creating test file 2023-10-06 05:28:59.270455 0 2023-10-06T05:28:59.270 DEBUG:teuthology.orchestra.run.smithi033:> touch '/home/ubuntu/cephtest/mnt.0/2023-10-06 05:28:59.270455 0' 2023-10-06T05:28:59.310 DEBUG:tasks.cephfs.mount:Deleting test file 2023-10-06 05:28:59.270455 0 2023-10-06T05:28:59.311 DEBUG:teuthology.orchestra.run.smithi033:> rm -f '/home/ubuntu/cephtest/mnt.0/2023-10-06 05:28:59.270455 0' 2023-10-06T05:28:59.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:59.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.612+0000 7fb69179d700 1 Processor -- start 2023-10-06T05:28:59.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.612+0000 7fb69179d700 1 -- start start 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.613+0000 7fb69179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb68c155af0 0x7fb68c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.613+0000 7fb69179d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb68c0c7920 0x7fb68c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb69179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c06ab60 con 0x7fb68c06a6a0 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb69179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c06b7e0 con 0x7fb68c155af0 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb69179d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c06ae40 con 0x7fb68c0c7920 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb68c155af0 0x7fb68c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb68c155af0 0x7fb68c06a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38272/0 (socket says 172.21.15.33:38272) 2023-10-06T05:28:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 -- 172.21.15.33:0/1094289732 learned_addr learned my addr 172.21.15.33:0/1094289732 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb690f9c700 1 --2- 172.21.15.33:0/1094289732 >> v2:172.21.15.167:3300/0 conn(0x7fb68c0c7920 0x7fb68c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 -- 172.21.15.33:0/1094289732 >> v2:172.21.15.167:3300/0 conn(0x7fb68c0c7920 msgr2=0x7fb68c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 --2- 172.21.15.33:0/1094289732 >> v2:172.21.15.167:3300/0 conn(0x7fb68c0c7920 0x7fb68c0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 -- 172.21.15.33:0/1094289732 >> v1:172.21.15.167:6789/0 conn(0x7fb68c06a6a0 legacy=0x7fb68c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.614+0000 7fb68bfff700 1 -- 172.21.15.33:0/1094289732 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c06b120 con 0x7fb68c155af0 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.615+0000 7fb68bfff700 1 --2- 172.21.15.33:0/1094289732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c155af0 0x7fb68c06a160 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fb68c070fb0 tx=0x7fb67c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43772dc77d1d5e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.615+0000 7fb68affd700 1 -- 172.21.15.33:0/1094289732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00be60 con 0x7fb68c155af0 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.615+0000 7fb68affd700 1 -- 172.21.15.33:0/1094289732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67c00ddc0 con 0x7fb68c155af0 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.615+0000 7fb68affd700 1 -- 172.21.15.33:0/1094289732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0196b0 con 0x7fb68c155af0 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.615+0000 7fb69179d700 1 -- 172.21.15.33:0/1094289732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c155af0 msgr2=0x7fb68c06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 --2- 172.21.15.33:0/1094289732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c155af0 0x7fb68c06a160 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fb68c070fb0 tx=0x7fb67c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 -- 172.21.15.33:0/1094289732 shutdown_connections 2023-10-06T05:28:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 --2- 172.21.15.33:0/1094289732 >> v2:172.21.15.167:3300/0 conn(0x7fb68c0c7920 0x7fb68c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 --2- 172.21.15.33:0/1094289732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c155af0 0x7fb68c06a160 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 -- 172.21.15.33:0/1094289732 >> 172.21.15.33:0/1094289732 conn(0x7fb68c06d030 msgr2=0x7fb68c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 -- 172.21.15.33:0/1094289732 shutdown_connections 2023-10-06T05:28:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.616+0000 7fb69179d700 1 -- 172.21.15.33:0/1094289732 wait complete. 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.617+0000 7fb69179d700 1 Processor -- start 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.617+0000 7fb69179d700 1 -- start start 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb69179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb68c06a6a0 0x7fb68c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb69179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 0x7fb68c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb69179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb68c155af0 0x7fb68c0c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb69179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1531a0 con 0x7fb68c0c7920 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb69179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c153420 con 0x7fb68c155af0 2023-10-06T05:28:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb69179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c153700 con 0x7fb68c06a6a0 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb68b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 0x7fb68c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb68b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 0x7fb68c1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38278/0 (socket says 172.21.15.33:38278) 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.618+0000 7fb68b7fe700 1 -- 172.21.15.33:0/4271226076 learned_addr learned my addr 172.21.15.33:0/4271226076 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68bfff700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb68c06a6a0 0x7fb68c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb690f9c700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb68c155af0 0x7fb68c0c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68b7fe700 1 -- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb68c06a6a0 msgr2=0x7fb68c1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68b7fe700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb68c06a6a0 0x7fb68c1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68b7fe700 1 -- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb68c155af0 msgr2=0x7fb68c0c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:59.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68b7fe700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb68c155af0 0x7fb68c0c1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68b7fe700 1 -- 172.21.15.33:0/4271226076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c00b790 con 0x7fb68c0c7920 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb68b7fe700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 0x7fb68c1515d0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fb68000c590 tx=0x7fb680007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680011590 con 0x7fb68c0c7920 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680011710 con 0x7fb68c0c7920 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680011ad0 con 0x7fb68c0c7920 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.619+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c1539e0 con 0x7fb68c0c7920 2023-10-06T05:28:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.620+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c154720 con 0x7fb68c0c7920 2023-10-06T05:28:59.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.620+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c153ee0 con 0x7fb68c0c7920 2023-10-06T05:28:59.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.622+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb68001a360 con 0x7fb68c0c7920 2023-10-06T05:28:59.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.622+0000 7fb673fff700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb678066f30 0x7fb6780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:59.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.623+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb680083670 con 0x7fb68c0c7920 2023-10-06T05:28:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.623+0000 7fb68bfff700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb678066f30 0x7fb6780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.624+0000 7fb68bfff700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb678066f30 0x7fb6780693e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb68c070fb0 tx=0x7fb67c01b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.627+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6800505f0 con 0x7fb68c0c7920 2023-10-06T05:28:59.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.873+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb68c04c380 con 0x7fb68c0c7920 2023-10-06T05:28:59.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.874+0000 7fb673fff700 1 -- 172.21.15.33:0/4271226076 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fb680053c10 con 0x7fb68c0c7920 2023-10-06T05:28:59.876 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:28:59.876 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:28:59.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb678066f30 msgr2=0x7fb6780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:59.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb678066f30 0x7fb6780693e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb68c070fb0 tx=0x7fb67c01b000 comp rx=0 tx=0).stop 2023-10-06T05:28:59.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 msgr2=0x7fb68c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:59.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 0x7fb68c1515d0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fb68000c590 tx=0x7fb680007f50 comp rx=0 tx=0).stop 2023-10-06T05:28:59.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 shutdown_connections 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb678066f30 0x7fb6780693e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb68c155af0 0x7fb68c0c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb68c06a6a0 0x7fb68c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 --2- 172.21.15.33:0/4271226076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb68c0c7920 0x7fb68c1515d0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 >> 172.21.15.33:0/4271226076 conn(0x7fb68c06d030 msgr2=0x7fb68c0c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 shutdown_connections 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:28:59.876+0000 7fb69179d700 1 -- 172.21.15.33:0/4271226076 wait complete. 2023-10-06T05:28:59.879 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:28:59.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:29:00.110 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":7946,"entity":{"name":{"type":"client","num":7946},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":2373724761}},"state":"open","num_leases":1,"num_caps":2,"request_load_avg":2,"uptime":0.89798585799999997,"requests_in_flight":1,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":5.9890169702192964,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7946 v1:172.21.15.33:0/2373724761","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":7778,"entity":{"name":{"type":"client","num":7778},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1820721818}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":54.844135927000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.8793864876637625,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7778 v1:192.168.0.1:0/1820721818","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:29:00.111 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 2373724761, "type": "v1" }, "name": { "num": 7946, "type": "client" } }, "id": 7946, "inst": "client.7946 v1:172.21.15.33:0/2373724761", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 1, "session_cache_liveness": { "halflife": 300, "value": 5.989016970219296 }, "state": "open", "uptime": 0.897985858 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1820721818, "type": "v1" }, "name": { "num": 7778, "type": "client" } }, "id": 7778, "inst": "client.7778 v1:192.168.0.1:0/1820721818", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.8793864876637625 }, "state": "open", "uptime": 54.844135927 } ] 2023-10-06T05:29:00.112 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:29:00.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:00.192 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:00.276 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:29:00.277 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:29:00.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:29:00.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:29:00.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:29:00.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:00.362 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:29:00.399 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:29:00.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:29:00.399 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:29:00.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:29:00.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:00.494 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:29:00.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:00.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.717+0000 7f3bf119d700 1 Processor -- start 2023-10-06T05:29:00.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.717+0000 7f3bf119d700 1 -- start start 2023-10-06T05:29:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.718+0000 7f3bf119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3bec155af0 0x7f3bec06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.718+0000 7f3bf119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec06a6a0 0x7f3bec0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.718+0000 7f3bf119d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3bec0c8b50 0x7f3bec0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.718+0000 7f3bf119d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec06b7e0 con 0x7f3bec0c8b50 2023-10-06T05:29:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.718+0000 7f3bf119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec06ab60 con 0x7f3bec155af0 2023-10-06T05:29:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.718+0000 7f3bf119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec06ae40 con 0x7f3bec06a6a0 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3bec0c8b50 0x7f3bec0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3bec0c8b50 0x7f3bec0c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45726/0 (socket says 172.21.15.33:45726) 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 -- 172.21.15.33:0/853119860 learned_addr learned my addr 172.21.15.33:0/853119860 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3beb7fe700 1 --2- 172.21.15.33:0/853119860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec06a6a0 0x7f3bec0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bebfff700 1 --2- 172.21.15.33:0/853119860 >> v2:172.21.15.167:3300/0 conn(0x7f3bec155af0 0x7f3bec06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 -- 172.21.15.33:0/853119860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec06a6a0 msgr2=0x7f3bec0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 --2- 172.21.15.33:0/853119860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec06a6a0 0x7f3bec0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 -- 172.21.15.33:0/853119860 >> v2:172.21.15.167:3300/0 conn(0x7f3bec155af0 msgr2=0x7f3bec06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 --2- 172.21.15.33:0/853119860 >> v2:172.21.15.167:3300/0 conn(0x7f3bec155af0 0x7f3bec06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.719+0000 7f3bf099c700 1 -- 172.21.15.33:0/853119860 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec06b120 con 0x7f3bec0c8b50 2023-10-06T05:29:00.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.720+0000 7f3bf099c700 1 --2- 172.21.15.33:0/853119860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0c7750 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0697c0 tx=0x7f3be4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52aa8903a969c0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.720+0000 7f3beaffd700 1 -- 172.21.15.33:0/853119860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400e070 con 0x7f3bec0c8b50 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.720+0000 7f3beaffd700 1 -- 172.21.15.33:0/853119860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400b970 con 0x7f3bec0c8b50 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.720+0000 7f3beaffd700 1 -- 172.21.15.33:0/853119860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be40092e0 con 0x7f3bec0c8b50 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.720+0000 7f3bf119d700 1 -- 172.21.15.33:0/853119860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec0c8b50 msgr2=0x7f3bec0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.720+0000 7f3bf119d700 1 --2- 172.21.15.33:0/853119860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0c7750 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0697c0 tx=0x7f3be4009840 comp rx=0 tx=0).stop 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 -- 172.21.15.33:0/853119860 shutdown_connections 2023-10-06T05:29:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 --2- 172.21.15.33:0/853119860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0c7750 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 --2- 172.21.15.33:0/853119860 >> v2:172.21.15.167:3300/0 conn(0x7f3bec155af0 0x7f3bec06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 --2- 172.21.15.33:0/853119860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec06a6a0 0x7f3bec0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 -- 172.21.15.33:0/853119860 >> 172.21.15.33:0/853119860 conn(0x7f3bec06d030 msgr2=0x7f3bec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 -- 172.21.15.33:0/853119860 shutdown_connections 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.721+0000 7f3bf119d700 1 -- 172.21.15.33:0/853119860 wait complete. 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.722+0000 7f3bf119d700 1 Processor -- start 2023-10-06T05:29:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.722+0000 7f3bf119d700 1 -- start start 2023-10-06T05:29:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.723+0000 7f3bf119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 0x7f3bec1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.723+0000 7f3bf119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.723+0000 7f3bf119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec155af0 0x7f3bec0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.723+0000 7f3bf119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec153a20 con 0x7f3bec06a6a0 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.723+0000 7f3bf119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0ac320 con 0x7f3bec0c8b50 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.723+0000 7f3bf119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0ac600 con 0x7f3bec155af0 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 0x7f3bec1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 0x7f3bec1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45742/0 (socket says 172.21.15.33:45742) 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 -- 172.21.15.33:0/426815450 learned_addr learned my addr 172.21.15.33:0/426815450 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bf099c700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec155af0 0x7f3bec0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3beb7fe700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 -- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec155af0 msgr2=0x7f3bec0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec155af0 0x7f3bec0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 -- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bec0c8b50 msgr2=0x7f3bec0abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 -- 172.21.15.33:0/426815450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4009ae0 con 0x7f3bec06a6a0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bebfff700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 0x7f3bec1534e0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00e9a0 tx=0x7f3bdc00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00a620 con 0x7f3bec06a6a0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc010630 con 0x7f3bec06a6a0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.724+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0ac8e0 con 0x7f3bec06a6a0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.725+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0109d0 con 0x7f3bec06a6a0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.725+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0acf20 con 0x7f3bec06a6a0 2023-10-06T05:29:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.726+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec04c380 con 0x7f3bec06a6a0 2023-10-06T05:29:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.727+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bdc0113f0 con 0x7f3bec06a6a0 2023-10-06T05:29:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.727+0000 7f3bd3fff700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bd8066f30 0x7f3bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.728+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3bdc015030 con 0x7f3bec06a6a0 2023-10-06T05:29:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.728+0000 7f3beb7fe700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bd8066f30 0x7f3bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.728+0000 7f3beb7fe700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bd8066f30 0x7f3bd80693e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0aa980 tx=0x7f3be00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.732+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bdc054dd0 con 0x7f3bec06a6a0 2023-10-06T05:29:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.979+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bec06b400 con 0x7f3bec06a6a0 2023-10-06T05:29:00.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.979+0000 7f3bd3fff700 1 -- 172.21.15.33:0/426815450 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f3bdc054960 con 0x7f3bec06a6a0 2023-10-06T05:29:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:00.981 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:29:00.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bd8066f30 msgr2=0x7f3bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bd8066f30 0x7f3bd80693e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0aa980 tx=0x7f3be00091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 msgr2=0x7f3bec1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 0x7f3bec1534e0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00e9a0 tx=0x7f3bdc00a330 comp rx=0 tx=0).stop 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 shutdown_connections 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3bd8066f30 0x7f3bd80693e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bec155af0 0x7f3bec0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bec06a6a0 0x7f3bec1534e0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 --2- 172.21.15.33:0/426815450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bec0c8b50 0x7f3bec0abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 >> 172.21.15.33:0/426815450 conn(0x7f3bec06d030 msgr2=0x7f3bec0cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:00.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 shutdown_connections 2023-10-06T05:29:00.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:00.982+0000 7f3bf119d700 1 -- 172.21.15.33:0/426815450 wait complete. 2023-10-06T05:29:00.985 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:29:01.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:01.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.186+0000 7fc4889fe700 1 Processor -- start 2023-10-06T05:29:01.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.186+0000 7fc4889fe700 1 -- start start 2023-10-06T05:29:01.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.187+0000 7fc4889fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc48413faa0 0x7fc48413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.187+0000 7fc4889fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc4840c7940 0x7fc4840c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.187+0000 7fc4889fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4840c9cb0 con 0x7fc48406aad0 2023-10-06T05:29:01.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.187+0000 7fc4889fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48406b640 con 0x7fc48413faa0 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.187+0000 7fc4889fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4840ca270 con 0x7fc4840c7940 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc48413faa0 0x7fc48413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc48413faa0 0x7fc48413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45754/0 (socket says 172.21.15.33:45754) 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 -- 172.21.15.33:0/893879232 learned_addr learned my addr 172.21.15.33:0/893879232 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc48259c700 1 --2- 172.21.15.33:0/893879232 >> v2:172.21.15.167:3300/0 conn(0x7fc4840c7940 0x7fc4840c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 -- 172.21.15.33:0/893879232 >> v2:172.21.15.167:3300/0 conn(0x7fc4840c7940 msgr2=0x7fc4840c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 --2- 172.21.15.33:0/893879232 >> v2:172.21.15.167:3300/0 conn(0x7fc4840c7940 0x7fc4840c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 -- 172.21.15.33:0/893879232 >> v1:172.21.15.167:6789/0 conn(0x7fc48406aad0 legacy=0x7fc48406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc482d9d700 1 -- 172.21.15.33:0/893879232 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48406b840 con 0x7fc48413faa0 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.188+0000 7fc48259c700 1 --2- 172.21.15.33:0/893879232 >> v2:172.21.15.167:3300/0 conn(0x7fc4840c7940 0x7fc4840c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.189+0000 7fc482d9d700 1 --2- 172.21.15.33:0/893879232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48413faa0 0x7fc48413fe70 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc4840cbef0 tx=0x7fc474009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c265134dee0163 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.189+0000 7fc481d9b700 1 -- 172.21.15.33:0/893879232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400b820 con 0x7fc48413faa0 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.189+0000 7fc481d9b700 1 -- 172.21.15.33:0/893879232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47400b980 con 0x7fc48413faa0 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.189+0000 7fc481d9b700 1 -- 172.21.15.33:0/893879232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400bd20 con 0x7fc48413faa0 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 -- 172.21.15.33:0/893879232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48413faa0 msgr2=0x7fc48413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 --2- 172.21.15.33:0/893879232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48413faa0 0x7fc48413fe70 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc4840cbef0 tx=0x7fc474009250 comp rx=0 tx=0).stop 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 -- 172.21.15.33:0/893879232 shutdown_connections 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 --2- 172.21.15.33:0/893879232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48413faa0 0x7fc48413fe70 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 --2- 172.21.15.33:0/893879232 >> v2:172.21.15.167:3300/0 conn(0x7fc4840c7940 0x7fc4840c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 -- 172.21.15.33:0/893879232 >> 172.21.15.33:0/893879232 conn(0x7fc48406d030 msgr2=0x7fc48406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 -- 172.21.15.33:0/893879232 shutdown_connections 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.190+0000 7fc4889fe700 1 -- 172.21.15.33:0/893879232 wait complete. 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.191+0000 7fc4889fe700 1 Processor -- start 2023-10-06T05:29:01.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.191+0000 7fc4889fe700 1 -- start start 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc4889fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 0x7fc484155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc4889fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4840c7940 0x7fc4840a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc4889fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc48413faa0 0x7fc484158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc4889fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840a6d10 con 0x7fc48406aad0 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc4889fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840a6ff0 con 0x7fc48413faa0 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc4889fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840c8cf0 con 0x7fc4840c7940 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.192+0000 7fc482d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 0x7fc484155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 0x7fc484155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45770/0 (socket says 172.21.15.33:45770) 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 -- 172.21.15.33:0/3764356266 learned_addr learned my addr 172.21.15.33:0/3764356266 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc48359e700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc48413faa0 0x7fc484158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc48259c700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4840c7940 0x7fc4840a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 -- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4840c7940 msgr2=0x7fc4840a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4840c7940 0x7fc4840a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 -- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc48413faa0 msgr2=0x7fc484158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc48413faa0 0x7fc484158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 -- 172.21.15.33:0/3764356266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4740094f0 con 0x7fc48406aad0 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.193+0000 7fc482d9d700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 0x7fc484155e00 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fc4740030c0 tx=0x7fc474016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.194+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474011d00 con 0x7fc48406aad0 2023-10-06T05:29:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.194+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc474020070 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.194+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47401b6a0 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.194+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4840c8fd0 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.194+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc484156340 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.195+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4840c8b50 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.196+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc474022030 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.197+0000 7fc46affd700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.197+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc4740865a0 con 0x7fc48406aad0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.197+0000 7fc48259c700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.198+0000 7fc48259c700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc470066f30 0x7fc4700693e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc47800adc0 tx=0x7fc478006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.201+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4840c8b50 con 0x7fc48406aad0 2023-10-06T05:29:01.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.406+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4840c8b50 con 0x7fc48406aad0 2023-10-06T05:29:01.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.407+0000 7fc46affd700 1 -- 172.21.15.33:0/3764356266 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17265 (secure 0 0 0) 0x7fc4840c8b50 con 0x7fc48406aad0 2023-10-06T05:29:01.409 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:01.409 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":198,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:28:53.971341+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:51.598576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:52.345977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1091933932":"2023-10-06T06:28:53.141475+0000","172.21.15.167:6837/3335696661":"2023-10-07T05:28:04.232599+0000","172.21.15.167:6835/3335696661":"2023-10-07T05:28:04.232599+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc470066f30 msgr2=0x7fc4700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc470066f30 0x7fc4700693e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc47800adc0 tx=0x7fc478006f90 comp rx=0 tx=0).stop 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 msgr2=0x7fc484155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 0x7fc484155e00 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fc4740030c0 tx=0x7fc474016000 comp rx=0 tx=0).stop 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 shutdown_connections 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc48413faa0 0x7fc484158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48406aad0 0x7fc484155e00 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 --2- 172.21.15.33:0/3764356266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc4840c7940 0x7fc4840a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.409+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 >> 172.21.15.33:0/3764356266 conn(0x7fc48406d030 msgr2=0x7fc48413ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.410+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 shutdown_connections 2023-10-06T05:29:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.410+0000 7fc4889fe700 1 -- 172.21.15.33:0/3764356266 wait complete. 2023-10-06T05:29:01.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:27:51.598576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:27:52.345977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:29:01.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:29:01.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.617+0000 7fac9559d700 1 Processor -- start 2023-10-06T05:29:01.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.617+0000 7fac9559d700 1 -- start start 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac9559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac900af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac9559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac900b01f0 0x7fac900b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac9559d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900ad430 con 0x7fac900cc7e0 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac9559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900afe20 con 0x7fac900b01f0 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac9559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac900b0b70 con 0x7fac90156370 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac94d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac900b01f0 0x7fac900b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac8f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fac900cc7e0 0x7fac900c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36870/0 (socket says 172.21.15.33:36870) 2023-10-06T05:29:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.618+0000 7fac8f7fe700 1 -- 172.21.15.33:0/2236908084 learned_addr learned my addr 172.21.15.33:0/2236908084 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac8ffff700 1 --2- 172.21.15.33:0/2236908084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac900af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac8effd700 1 -- 172.21.15.33:0/2236908084 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213673836 0 0) 0x7fac900ad430 con 0x7fac900cc7e0 2023-10-06T05:29:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac94d9c700 1 -- 172.21.15.33:0/2236908084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 msgr2=0x7fac900af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac94d9c700 1 --2- 172.21.15.33:0/2236908084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac900af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac94d9c700 1 -- 172.21.15.33:0/2236908084 >> v1:172.21.15.33:6789/0 conn(0x7fac900cc7e0 legacy=0x7fac900c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac94d9c700 1 -- 172.21.15.33:0/2236908084 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9006bc70 con 0x7fac900b01f0 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac94d9c700 1 --2- 172.21.15.33:0/2236908084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900b01f0 0x7fac900b0630 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e60 tx=0x7fac88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be8a0ea9baa1d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac8effd700 1 -- 172.21.15.33:0/2236908084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800e070 con 0x7fac900b01f0 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac8effd700 1 -- 172.21.15.33:0/2236908084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8800bc30 con 0x7fac900b01f0 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.619+0000 7fac8effd700 1 -- 172.21.15.33:0/2236908084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88012430 con 0x7fac900b01f0 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 -- 172.21.15.33:0/2236908084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900b01f0 msgr2=0x7fac900b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 --2- 172.21.15.33:0/2236908084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900b01f0 0x7fac900b0630 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e60 tx=0x7fac88009780 comp rx=0 tx=0).stop 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 -- 172.21.15.33:0/2236908084 shutdown_connections 2023-10-06T05:29:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 --2- 172.21.15.33:0/2236908084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900b01f0 0x7fac900b0630 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 --2- 172.21.15.33:0/2236908084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac900af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 -- 172.21.15.33:0/2236908084 >> 172.21.15.33:0/2236908084 conn(0x7fac9006d030 msgr2=0x7fac900cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 -- 172.21.15.33:0/2236908084 shutdown_connections 2023-10-06T05:29:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.620+0000 7fac9559d700 1 -- 172.21.15.33:0/2236908084 wait complete. 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.622+0000 7fac9559d700 1 Processor -- start 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.622+0000 7fac9559d700 1 -- start start 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac9559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac900b01f0 0x7fac901656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac9559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 0x7fac900aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac9559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac90069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac9559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90165bf0 con 0x7fac900cc7e0 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac9559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90158450 con 0x7fac900b01f0 2023-10-06T05:29:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac9559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90152670 con 0x7fac90156370 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 0x7fac900aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 0x7fac900aad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45794/0 (socket says 172.21.15.33:45794) 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 -- 172.21.15.33:0/472094021 learned_addr learned my addr 172.21.15.33:0/472094021 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8ffff700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac900b01f0 0x7fac901656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac94d9c700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac90069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 -- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 msgr2=0x7fac90069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac90069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 -- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac900b01f0 msgr2=0x7fac901656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac900b01f0 0x7fac901656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.623+0000 7fac8f7fe700 1 -- 172.21.15.33:0/472094021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88009a20 con 0x7fac900cc7e0 2023-10-06T05:29:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.624+0000 7fac8f7fe700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 0x7fac900aad20 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c840 tx=0x7fac84007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.624+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84005c00 con 0x7fac900cc7e0 2023-10-06T05:29:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.624+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac84005d60 con 0x7fac900cc7e0 2023-10-06T05:29:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.624+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac840072e0 con 0x7fac900cc7e0 2023-10-06T05:29:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.624+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac90151ac0 con 0x7fac900cc7e0 2023-10-06T05:29:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.624+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac90152300 con 0x7fac900cc7e0 2023-10-06T05:29:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.625+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9004c380 con 0x7fac900cc7e0 2023-10-06T05:29:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.626+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac8401b3b0 con 0x7fac900cc7e0 2023-10-06T05:29:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.627+0000 7fac77fff700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac7c066f30 0x7fac7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.627+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fac84083ca0 con 0x7fac900cc7e0 2023-10-06T05:29:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.627+0000 7fac8ffff700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac7c066f30 0x7fac7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:01.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.628+0000 7fac8ffff700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac7c066f30 0x7fac7c0693e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fac80006fd0 tx=0x7fac80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.631+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac8404b5a0 con 0x7fac900cc7e0 2023-10-06T05:29:01.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.830+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac901527b0 con 0x7fac900cc7e0 2023-10-06T05:29:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.831+0000 7fac77fff700 1 -- 172.21.15.33:0/472094021 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac840170a0 con 0x7fac900cc7e0 2023-10-06T05:29:01.833 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:29:01.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac7c066f30 msgr2=0x7fac7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac7c066f30 0x7fac7c0693e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fac80006fd0 tx=0x7fac80008040 comp rx=0 tx=0).stop 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 msgr2=0x7fac900aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 0x7fac900aad20 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c840 tx=0x7fac84007db0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 shutdown_connections 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac7c066f30 0x7fac7c0693e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac90156370 0x7fac90069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac900b01f0 0x7fac901656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 --2- 172.21.15.33:0/472094021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900cc7e0 0x7fac900aad20 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 >> 172.21.15.33:0/472094021 conn(0x7fac9006d030 msgr2=0x7fac900ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 shutdown_connections 2023-10-06T05:29:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:01.833+0000 7fac9559d700 1 -- 172.21.15.33:0/472094021 wait complete. 2023-10-06T05:29:01.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:29:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.041+0000 7fa07699d700 1 Processor -- start 2023-10-06T05:29:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.041+0000 7fa07699d700 1 -- start start 2023-10-06T05:29:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0700c9490 0x7fa070155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa0700caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07699d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700b06b0 con 0x7fa070155a50 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0700c99e0 con 0x7fa0700c9490 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c9cc0 con 0x7fa0700afb20 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07619c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa0700caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07519a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0700c9490 0x7fa070155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07619c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa0700caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:34854/0 (socket says 172.21.15.33:34854) 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.043+0000 7fa07619c700 1 -- 172.21.15.33:0/3194328327 learned_addr learned my addr 172.21.15.33:0/3194328327 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa074999700 1 -- 172.21.15.33:0/3194328327 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929683583 0 0) 0x7fa0700b06b0 con 0x7fa070155a50 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa074999700 1 -- 172.21.15.33:0/3194328327 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003570 con 0x7fa070155a50 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa07519a700 1 -- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 msgr2=0x7fa0700caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:02.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa07519a700 1 --2- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa0700caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa07519a700 1 -- 172.21.15.33:0/3194328327 >> v1:172.21.15.167:6789/0 conn(0x7fa070155a50 legacy=0x7fa0700af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa07519a700 1 -- 172.21.15.33:0/3194328327 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700c9fa0 con 0x7fa0700c9490 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa07619c700 1 --2- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa0700caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.044+0000 7fa07519a700 1 --2- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0700c9490 0x7fa070155e20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa06000aa90 tx=0x7fa06000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8748687f28a4c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa074999700 1 -- 172.21.15.33:0/3194328327 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000c970 con 0x7fa0700c9490 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa074999700 1 -- 172.21.15.33:0/3194328327 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06000cad0 con 0x7fa0700c9490 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa074999700 1 -- 172.21.15.33:0/3194328327 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000ce70 con 0x7fa0700c9490 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 -- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0700c9490 msgr2=0x7fa070155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 --2- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0700c9490 0x7fa070155e20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa06000aa90 tx=0x7fa06000a330 comp rx=0 tx=0).stop 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 -- 172.21.15.33:0/3194328327 shutdown_connections 2023-10-06T05:29:02.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 --2- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa0700caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 --2- 172.21.15.33:0/3194328327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0700c9490 0x7fa070155e20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 -- 172.21.15.33:0/3194328327 >> 172.21.15.33:0/3194328327 conn(0x7fa07006d030 msgr2=0x7fa0700cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:02.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 -- 172.21.15.33:0/3194328327 shutdown_connections 2023-10-06T05:29:02.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.045+0000 7fa07699d700 1 -- 172.21.15.33:0/3194328327 wait complete. 2023-10-06T05:29:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.047+0000 7fa07699d700 1 Processor -- start 2023-10-06T05:29:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.047+0000 7fa07699d700 1 -- start start 2023-10-06T05:29:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa070159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 0x7fa070151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa070155a50 0x7fa0701523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701698d0 con 0x7fa0700c9490 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa070152a70 con 0x7fa070155a50 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa070152d50 con 0x7fa0700afb20 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 0x7fa070151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 0x7fa070151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45802/0 (socket says 172.21.15.33:45802) 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07519a700 1 -- 172.21.15.33:0/827804760 learned_addr learned my addr 172.21.15.33:0/827804760 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07599b700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa070159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.048+0000 7fa07619c700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa070155a50 0x7fa0701523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07519a700 1 -- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 msgr2=0x7fa070159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07519a700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa070159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07519a700 1 -- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa070155a50 msgr2=0x7fa0701523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07519a700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa070155a50 0x7fa0701523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07519a700 1 -- 172.21.15.33:0/827804760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009ae0 con 0x7fa0700c9490 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07519a700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 0x7fa070151e90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fa07006d4d0 tx=0x7fa06001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060019e70 con 0x7fa0700c9490 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060021430 con 0x7fa0700c9490 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600217d0 con 0x7fa0700c9490 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06000a5d0 con 0x7fa0700c9490 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.049+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa070153310 con 0x7fa0700c9490 2023-10-06T05:29:02.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.050+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07004c380 con 0x7fa0700c9490 2023-10-06T05:29:02.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.051+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0600103f0 con 0x7fa0700c9490 2023-10-06T05:29:02.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.052+0000 7fa065ffb700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa054066f30 0x7fa0540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.052+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa060085cf0 con 0x7fa0700c9490 2023-10-06T05:29:02.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.053+0000 7fa07599b700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa054066f30 0x7fa0540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.053+0000 7fa07599b700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa054066f30 0x7fa0540693e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007fd0 tx=0x7fa06c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.057+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa060052c70 con 0x7fa0700c9490 2023-10-06T05:29:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.258+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa07006bb10 con 0x7fa0700c9490 2023-10-06T05:29:02.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.258+0000 7fa065ffb700 1 -- 172.21.15.33:0/827804760 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v198) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa060056290 con 0x7fa0700c9490 2023-10-06T05:29:02.260 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa054066f30 msgr2=0x7fa0540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa054066f30 0x7fa0540693e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007fd0 tx=0x7fa06c007d50 comp rx=0 tx=0).stop 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 msgr2=0x7fa070151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 0x7fa070151e90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fa07006d4d0 tx=0x7fa06001a000 comp rx=0 tx=0).stop 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 shutdown_connections 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa054066f30 0x7fa0540693e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa070155a50 0x7fa0701523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0700afb20 0x7fa070159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 --2- 172.21.15.33:0/827804760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0700c9490 0x7fa070151e90 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 >> 172.21.15.33:0/827804760 conn(0x7fa07006d030 msgr2=0x7fa07006ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 shutdown_connections 2023-10-06T05:29:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.261+0000 7fa07699d700 1 -- 172.21.15.33:0/827804760 wait complete. 2023-10-06T05:29:02.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:29:02.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.468+0000 7faca5d9d700 1 Processor -- start 2023-10-06T05:29:02.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.468+0000 7faca5d9d700 1 -- start start 2023-10-06T05:29:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.469+0000 7faca5d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faca00af540 0x7faca00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.469+0000 7faca5d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faca00cd5e0 0x7faca015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.469+0000 7faca5d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca015d660 con 0x7faca015a650 2023-10-06T05:29:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.469+0000 7faca5d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca00ce890 con 0x7faca00cd5e0 2023-10-06T05:29:02.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.469+0000 7faca5d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca015aac0 con 0x7faca00af540 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7fac9ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faca015a650 0x7faca00cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36876/0 (socket says 172.21.15.33:36876) 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7fac9ffff700 1 -- 172.21.15.33:0/3564304358 learned_addr learned my addr 172.21.15.33:0/3564304358 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca559c700 1 --2- 172.21.15.33:0/3564304358 >> v2:172.21.15.33:3300/0 conn(0x7faca00cd5e0 0x7faca015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca4d9b700 1 --2- 172.21.15.33:0/3564304358 >> v2:172.21.15.167:3300/0 conn(0x7faca00af540 0x7faca00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7fac9f7fe700 1 -- 172.21.15.33:0/3564304358 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842936479 0 0) 0x7faca015d660 con 0x7faca015a650 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7fac9f7fe700 1 -- 172.21.15.33:0/3564304358 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac90003570 con 0x7faca015a650 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca559c700 1 -- 172.21.15.33:0/3564304358 >> v2:172.21.15.167:3300/0 conn(0x7faca00af540 msgr2=0x7faca00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca559c700 1 --2- 172.21.15.33:0/3564304358 >> v2:172.21.15.167:3300/0 conn(0x7faca00af540 0x7faca00af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca559c700 1 -- 172.21.15.33:0/3564304358 >> v1:172.21.15.33:6789/0 conn(0x7faca015a650 legacy=0x7faca00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca559c700 1 -- 172.21.15.33:0/3564304358 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca015d890 con 0x7faca00cd5e0 2023-10-06T05:29:02.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.470+0000 7faca559c700 1 --2- 172.21.15.33:0/3564304358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca00cd5e0 0x7faca015d120 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7faca00c7530 tx=0x7fac98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8899a864210993 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7fac9f7fe700 1 -- 172.21.15.33:0/3564304358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800e070 con 0x7faca00cd5e0 2023-10-06T05:29:02.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7fac9f7fe700 1 -- 172.21.15.33:0/3564304358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9800bab0 con 0x7faca00cd5e0 2023-10-06T05:29:02.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7fac9f7fe700 1 -- 172.21.15.33:0/3564304358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800be70 con 0x7faca00cd5e0 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 -- 172.21.15.33:0/3564304358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca00cd5e0 msgr2=0x7faca015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3564304358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca00cd5e0 0x7faca015d120 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7faca00c7530 tx=0x7fac98009780 comp rx=0 tx=0).stop 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 -- 172.21.15.33:0/3564304358 shutdown_connections 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3564304358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca00cd5e0 0x7faca015d120 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3564304358 >> v2:172.21.15.167:3300/0 conn(0x7faca00af540 0x7faca00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 -- 172.21.15.33:0/3564304358 >> 172.21.15.33:0/3564304358 conn(0x7faca006d9b0 msgr2=0x7faca006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.471+0000 7faca5d9d700 1 -- 172.21.15.33:0/3564304358 shutdown_connections 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.472+0000 7faca5d9d700 1 -- 172.21.15.33:0/3564304358 wait complete. 2023-10-06T05:29:02.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.473+0000 7faca5d9d700 1 Processor -- start 2023-10-06T05:29:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.473+0000 7faca5d9d700 1 -- start start 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca5d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faca00af540 0x7faca01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faca00cd5e0 0x7faca00b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca5d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 0x7faca00b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca5d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca016a610 con 0x7faca015a650 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca5d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0169a00 con 0x7faca00af540 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca0169ce0 con 0x7faca00cd5e0 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 0x7faca00b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 0x7faca00b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45812/0 (socket says 172.21.15.33:45812) 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.474+0000 7faca559c700 1 -- 172.21.15.33:0/3207634543 learned_addr learned my addr 172.21.15.33:0/3207634543 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:02.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca4d9b700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faca00af540 0x7faca01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7fac9ffff700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faca00cd5e0 0x7faca00b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca559c700 1 -- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faca00cd5e0 msgr2=0x7faca00b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca559c700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faca00cd5e0 0x7faca00b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca559c700 1 -- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faca00af540 msgr2=0x7faca01589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca559c700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faca00af540 0x7faca01589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca559c700 1 -- 172.21.15.33:0/3207634543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009a20 con 0x7faca015a650 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca559c700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 0x7faca00b15e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fac980099f0 tx=0x7fac98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98027470 con 0x7faca015a650 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac980275f0 con 0x7faca015a650 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980279b0 con 0x7faca015a650 2023-10-06T05:29:02.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca0169fc0 con 0x7faca015a650 2023-10-06T05:29:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.475+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca0158f30 con 0x7faca015a650 2023-10-06T05:29:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.476+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca004c380 con 0x7faca015a650 2023-10-06T05:29:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.477+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac98024070 con 0x7faca015a650 2023-10-06T05:29:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.478+0000 7fac9cff9700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac84066f30 0x7fac840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.478+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fac98087240 con 0x7faca015a650 2023-10-06T05:29:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.479+0000 7faca4d9b700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac84066f30 0x7fac840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.479+0000 7faca4d9b700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac84066f30 0x7fac840693e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7faca00c9bf0 tx=0x7fac94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.482+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac980541c0 con 0x7faca015a650 2023-10-06T05:29:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.684+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faca015db70 con 0x7faca015a650 2023-10-06T05:29:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.685+0000 7fac9cff9700 1 -- 172.21.15.33:0/3207634543 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7fac980577e0 con 0x7faca015a650 2023-10-06T05:29:02.687 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:29:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.687+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac84066f30 msgr2=0x7fac840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.687+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac84066f30 0x7fac840693e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7faca00c9bf0 tx=0x7fac94008040 comp rx=0 tx=0).stop 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.687+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 msgr2=0x7faca00b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.687+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 0x7faca00b15e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fac980099f0 tx=0x7fac98019000 comp rx=0 tx=0).stop 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.687+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 shutdown_connections 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac84066f30 0x7fac840693e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca015a650 0x7faca00b15e0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faca00af540 0x7faca01589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 --2- 172.21.15.33:0/3207634543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faca00cd5e0 0x7faca00b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 >> 172.21.15.33:0/3207634543 conn(0x7faca006d9b0 msgr2=0x7faca0159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:02.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 shutdown_connections 2023-10-06T05:29:02.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.688+0000 7faca5d9d700 1 -- 172.21.15.33:0/3207634543 wait complete. 2023-10-06T05:29:02.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:29:02.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.894+0000 7f737139d700 1 Processor -- start 2023-10-06T05:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.895+0000 7f737139d700 1 -- start start 2023-10-06T05:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f737139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c15acd0 0x7f736c0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f737139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736c0b0cd0 con 0x7f736c06efa0 2023-10-06T05:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f737139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736c15a840 con 0x7f736c159850 2023-10-06T05:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f737139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c159f10 con 0x7f736c15acd0 2023-10-06T05:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f7370b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f736c06efa0 0x7f736c0ac3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36888/0 (socket says 172.21.15.33:36888) 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f7370b9c700 1 -- 172.21.15.33:0/300058945 learned_addr learned my addr 172.21.15.33:0/300058945 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.896+0000 7f736b7fe700 1 --2- 172.21.15.33:0/300058945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c15acd0 0x7f736c0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574252889 0 0) 0x7f736c0b0cd0 con 0x7f736c06efa0 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7354003570 con 0x7f736c06efa0 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125586032 0 0) 0x7f736c15a840 con 0x7f736c159850 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f736c0b0cd0 con 0x7f736c159850 2023-10-06T05:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736b7fe700 1 -- 172.21.15.33:0/300058945 >> v1:172.21.15.167:6789/0 conn(0x7f736c159850 legacy=0x7f736c06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736b7fe700 1 -- 172.21.15.33:0/300058945 >> v1:172.21.15.33:6789/0 conn(0x7f736c06efa0 legacy=0x7f736c0ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736b7fe700 1 -- 172.21.15.33:0/300058945 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c15a1f0 con 0x7f736c15acd0 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.897+0000 7f736b7fe700 1 --2- 172.21.15.33:0/300058945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c15acd0 0x7f736c0abe90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7360009910 tx=0x7f73600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97530eefbea98824 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360011630 con 0x7f736c15acd0 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360011790 con 0x7f736c15acd0 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f736affd700 1 -- 172.21.15.33:0/300058945 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360011b30 con 0x7f736c15acd0 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 -- 172.21.15.33:0/300058945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c15acd0 msgr2=0x7f736c0abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 --2- 172.21.15.33:0/300058945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c15acd0 0x7f736c0abe90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7360009910 tx=0x7f73600091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 -- 172.21.15.33:0/300058945 shutdown_connections 2023-10-06T05:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 --2- 172.21.15.33:0/300058945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c15acd0 0x7f736c0abe90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 -- 172.21.15.33:0/300058945 >> 172.21.15.33:0/300058945 conn(0x7f736c06cd40 msgr2=0x7f736c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 -- 172.21.15.33:0/300058945 shutdown_connections 2023-10-06T05:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.898+0000 7f737139d700 1 -- 172.21.15.33:0/300058945 wait complete. 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.900+0000 7f737139d700 1 Processor -- start 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.900+0000 7f737139d700 1 -- start start 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f737139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736c06efa0 0x7f736c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f737139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c159850 0x7f736c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f737139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 0x7f736c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f737139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b2960 con 0x7f736c15acd0 2023-10-06T05:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f737139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b2c40 con 0x7f736c06efa0 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f737139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b2f20 con 0x7f736c159850 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f7370b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 0x7f736c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f7370b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 0x7f736c0c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45828/0 (socket says 172.21.15.33:45828) 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f7370b9c700 1 -- 172.21.15.33:0/3898025071 learned_addr learned my addr 172.21.15.33:0/3898025071 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f736bfff700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736c06efa0 0x7f736c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.901+0000 7f736b7fe700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c159850 0x7f736c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7370b9c700 1 -- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c159850 msgr2=0x7f736c0b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7370b9c700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c159850 0x7f736c0b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7370b9c700 1 -- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736c06efa0 msgr2=0x7f736c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7370b9c700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736c06efa0 0x7f736c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7370b9c700 1 -- 172.21.15.33:0/3898025071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009420 con 0x7f736c15acd0 2023-10-06T05:29:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7370b9c700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 0x7f736c0c8390 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f736400cc00 tx=0x7f7364001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736400a8b0 con 0x7f736c15acd0 2023-10-06T05:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736400aa30 con 0x7f736c15acd0 2023-10-06T05:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736400adf0 con 0x7f736c15acd0 2023-10-06T05:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c0cc5b0 con 0x7f736c15acd0 2023-10-06T05:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.902+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c158370 con 0x7f736c15acd0 2023-10-06T05:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.904+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c0ccad0 con 0x7f736c15acd0 2023-10-06T05:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.904+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7364007300 con 0x7f736c15acd0 2023-10-06T05:29:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.905+0000 7f7353fff700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7358066f30 0x7f73580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.905+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7364084ad0 con 0x7f736c15acd0 2023-10-06T05:29:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.906+0000 7f736bfff700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7358066f30 0x7f73580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.907+0000 7f736bfff700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7358066f30 0x7f73580693e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f736c0b4120 tx=0x7f735c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:02.909+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f736c0ccad0 con 0x7f736c15acd0 2023-10-06T05:29:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.111+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f736c0ccad0 con 0x7f736c15acd0 2023-10-06T05:29:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.112+0000 7f7353fff700 1 -- 172.21.15.33:0/3898025071 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7f736c0ccad0 con 0x7f736c15acd0 2023-10-06T05:29:03.114 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:29:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7358066f30 msgr2=0x7f73580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7358066f30 0x7f73580693e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f736c0b4120 tx=0x7f735c007d50 comp rx=0 tx=0).stop 2023-10-06T05:29:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 msgr2=0x7f736c0c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 0x7f736c0c8390 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f736400cc00 tx=0x7f7364001a80 comp rx=0 tx=0).stop 2023-10-06T05:29:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 shutdown_connections 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7358066f30 0x7f73580693e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f736c15acd0 0x7f736c0c8390 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736c06efa0 0x7f736c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 --2- 172.21.15.33:0/3898025071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f736c159850 0x7f736c0b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 >> 172.21.15.33:0/3898025071 conn(0x7f736c06cd40 msgr2=0x7f736c069340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 shutdown_connections 2023-10-06T05:29:03.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.114+0000 7f737139d700 1 -- 172.21.15.33:0/3898025071 wait complete. 2023-10-06T05:29:03.133 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.319+0000 7fb38759e700 1 Processor -- start 2023-10-06T05:29:03.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.319+0000 7fb38759e700 1 -- start start 2023-10-06T05:29:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.320+0000 7fb38759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb388155af0 0x7fb38806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.320+0000 7fb38759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb3880c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.320+0000 7fb38759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38806ab60 con 0x7fb38806a6a0 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.320+0000 7fb38759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38806b7e0 con 0x7fb388155af0 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.320+0000 7fb38759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38806ae40 con 0x7fb3880c7920 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb385d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb38806a6a0 0x7fb3880c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36894/0 (socket says 172.21.15.33:36894) 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb385d9b700 1 -- 172.21.15.33:0/3952784896 learned_addr learned my addr 172.21.15.33:0/3952784896 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb386d9d700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb3880c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38659c700 1 --2- 172.21.15.33:0/3952784896 >> v2:172.21.15.167:3300/0 conn(0x7fb388155af0 0x7fb38806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256074398 0 0) 0x7fb38806ab60 con 0x7fb38806a6a0 2023-10-06T05:29:03.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb370003570 con 0x7fb38806a6a0 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164215644 0 0) 0x7fb370003570 con 0x7fb38806a6a0 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 msgr2=0x7fb3880c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb3880c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 >> v2:172.21.15.167:3300/0 conn(0x7fb388155af0 msgr2=0x7fb38806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 --2- 172.21.15.33:0/3952784896 >> v2:172.21.15.167:3300/0 conn(0x7fb388155af0 0x7fb38806a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38806b120 con 0x7fb38806a6a0 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.321+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb37c001470 con 0x7fb38806a6a0 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 >> v1:172.21.15.33:6789/0 conn(0x7fb38806a6a0 legacy=0x7fb3880c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb370003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb370004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb370004d50 0x7fb3700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38806b7e0 con 0x7fb370003fe0 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38806ae40 con 0x7fb370004d50 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37000ba90 con 0x7fb3880c7920 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb370004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38659c700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb370003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb386d9d700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb370004d50 0x7fb3700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb385d9b700 1 -- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 msgr2=0x7fb370003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb370003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb385d9b700 1 -- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb370004d50 msgr2=0x7fb3700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb370004d50 0x7fb3700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb385d9b700 1 -- 172.21.15.33:0/3952784896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb370003570 con 0x7fb370003fe0 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.322+0000 7fb38659c700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880c7920 0x7fb370003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb370004810 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c390 tx=0x7fb37c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdd3cd34c152828 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00cc00 con 0x7fb370003fe0 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00cda0 con 0x7fb370003fe0 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb38559a700 1 -- 172.21.15.33:0/3952784896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c005b70 con 0x7fb370003fe0 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb38759e700 1 -- 172.21.15.33:0/3952784896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 msgr2=0x7fb370004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb38759e700 1 --2- 172.21.15.33:0/3952784896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb370004810 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c390 tx=0x7fb37c001e90 comp rx=0 tx=0).stop 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.323+0000 7fb386d9d700 1 -- 172.21.15.33:0/3952784896 reap_dead start 2023-10-06T05:29:03.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.324+0000 7fb38759e700 1 -- 172.21.15.33:0/3952784896 shutdown_connections 2023-10-06T05:29:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.324+0000 7fb38759e700 1 -- 172.21.15.33:0/3952784896 >> 172.21.15.33:0/3952784896 conn(0x7fb38806d030 msgr2=0x7fb38806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.324+0000 7fb38759e700 1 -- 172.21.15.33:0/3952784896 shutdown_connections 2023-10-06T05:29:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.324+0000 7fb38759e700 1 -- 172.21.15.33:0/3952784896 wait complete. 2023-10-06T05:29:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.325+0000 7fb38759e700 1 Processor -- start 2023-10-06T05:29:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.325+0000 7fb38759e700 1 -- start start 2023-10-06T05:29:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb388146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880abaf0 0x7fb3880abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3880b0400 0x7fb3880b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880c3190 con 0x7fb370003fe0 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880b0db0 con 0x7fb3880b0400 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880ad390 con 0x7fb3880abaf0 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb388146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb388146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45846/0 (socket says 172.21.15.33:45846) 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.326+0000 7fb38659c700 1 -- 172.21.15.33:0/3908539974 learned_addr learned my addr 172.21.15.33:0/3908539974 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880abaf0 0x7fb3880abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb386d9d700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3880b0400 0x7fb3880b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38659c700 1 -- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880abaf0 msgr2=0x7fb3880abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38659c700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880abaf0 0x7fb3880abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38659c700 1 -- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3880b0400 msgr2=0x7fb3880b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38659c700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3880b0400 0x7fb3880b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38659c700 1 -- 172.21.15.33:0/3908539974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880ac5b0 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880abaf0 0x7fb3880abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38659c700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb388146c10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb378011da0 tx=0x7fb3780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800fbf0 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800fd50 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37801d6b0 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.327+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3880ac890 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.328+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388158bb0 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.329+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb388068480 con 0x7fb370003fe0 2023-10-06T05:29:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.329+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb37801dae0 con 0x7fb370003fe0 2023-10-06T05:29:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.330+0000 7fb3767fc700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb368066f30 0x7fb3680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.330+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb378018070 con 0x7fb370003fe0 2023-10-06T05:29:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.331+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb368066f30 0x7fb3680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.332+0000 7fb385d9b700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb368066f30 0x7fb3680693e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c390 tx=0x7fb37c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:03.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.334+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb378052750 con 0x7fb370003fe0 2023-10-06T05:29:03.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.581+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb38804c380 con 0x7fb370003fe0 2023-10-06T05:29:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.582+0000 7fb3767fc700 1 -- 172.21.15.33:0/3908539974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fb378055d70 con 0x7fb370003fe0 2023-10-06T05:29:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:03.583 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:29:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb368066f30 msgr2=0x7fb3680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb368066f30 0x7fb3680693e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c390 tx=0x7fb37c015000 comp rx=0 tx=0).stop 2023-10-06T05:29:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 msgr2=0x7fb388146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb388146c10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb378011da0 tx=0x7fb3780091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 shutdown_connections 2023-10-06T05:29:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb368066f30 0x7fb3680693e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3880b0400 0x7fb3880b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb370003fe0 0x7fb388146c10 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 --2- 172.21.15.33:0/3908539974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3880abaf0 0x7fb3880abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 >> 172.21.15.33:0/3908539974 conn(0x7fb38806d030 msgr2=0x7fb3880cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 shutdown_connections 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.584+0000 7fb38759e700 1 -- 172.21.15.33:0/3908539974 wait complete. 2023-10-06T05:29:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:29:03.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.788+0000 7f9ae8d9d700 1 Processor -- start 2023-10-06T05:29:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.788+0000 7f9ae8d9d700 1 -- start start 2023-10-06T05:29:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.789+0000 7f9ae8d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9ae40cf3b0 0x7f9ae41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.789+0000 7f9ae8d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40af0a0 0x7f9ae41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.789+0000 7f9ae8d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae40cbd80 con 0x7f9ae40ccfc0 2023-10-06T05:29:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.789+0000 7f9ae8d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae40af5f0 con 0x7f9ae40cf3b0 2023-10-06T05:29:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.789+0000 7f9ae8d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40af8d0 con 0x7f9ae40af0a0 2023-10-06T05:29:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9ae40ccfc0 0x7f9ae40cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36900/0 (socket says 172.21.15.33:36900) 2023-10-06T05:29:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae37fe700 1 -- 172.21.15.33:0/1599613347 learned_addr learned my addr 172.21.15.33:0/1599613347 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1599613347 >> v2:172.21.15.167:3300/0 conn(0x7f9ae40cf3b0 0x7f9ae41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae3fff700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40af0a0 0x7f9ae41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae27fc700 1 -- 172.21.15.33:0/1599613347 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047975720 0 0) 0x7f9ae40cbd80 con 0x7f9ae40ccfc0 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae27fc700 1 -- 172.21.15.33:0/1599613347 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc003570 con 0x7f9ae40ccfc0 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae2ffd700 1 -- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40af0a0 msgr2=0x7f9ae41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40af0a0 0x7f9ae41599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae2ffd700 1 -- 172.21.15.33:0/1599613347 >> v1:172.21.15.33:6789/0 conn(0x7f9ae40ccfc0 legacy=0x7f9ae40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae2ffd700 1 -- 172.21.15.33:0/1599613347 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40afbb0 con 0x7f9ae40cf3b0 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.790+0000 7f9ae3fff700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40af0a0 0x7f9ae41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.791+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40cf3b0 0x7f9ae41594a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009910 tx=0x7f9ad80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579a9121eaf3ba49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.791+0000 7f9ae27fc700 1 -- 172.21.15.33:0/1599613347 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8011630 con 0x7f9ae40cf3b0 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.791+0000 7f9ae27fc700 1 -- 172.21.15.33:0/1599613347 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8011790 con 0x7f9ae40cf3b0 2023-10-06T05:29:03.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.791+0000 7f9ae27fc700 1 -- 172.21.15.33:0/1599613347 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8011b30 con 0x7f9ae40cf3b0 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40cf3b0 msgr2=0x7f9ae41594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40cf3b0 0x7f9ae41594a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009910 tx=0x7f9ad80091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1599613347 shutdown_connections 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40af0a0 0x7f9ae41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1599613347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40cf3b0 0x7f9ae41594a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1599613347 >> 172.21.15.33:0/1599613347 conn(0x7f9ae406cff0 msgr2=0x7f9ae406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1599613347 shutdown_connections 2023-10-06T05:29:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.792+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1599613347 wait complete. 2023-10-06T05:29:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.793+0000 7f9ae8d9d700 1 Processor -- start 2023-10-06T05:29:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.793+0000 7f9ae8d9d700 1 -- start start 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae8d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40af0a0 0x7f9ae40c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae8d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ccfc0 0x7f9ae414e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae8d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 0x7f9ae40c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae8d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c92c0 con 0x7f9ae40c8200 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae8d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c95a0 con 0x7f9ae40af0a0 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae8d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c9880 con 0x7f9ae40ccfc0 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 0x7f9ae40c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.794+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 0x7f9ae40c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45862/0 (socket says 172.21.15.33:45862) 2023-10-06T05:29:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 -- 172.21.15.33:0/1596201299 learned_addr learned my addr 172.21.15.33:0/1596201299 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ccfc0 0x7f9ae414e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40af0a0 0x7f9ae40c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 -- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ccfc0 msgr2=0x7f9ae414e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ccfc0 0x7f9ae414e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 -- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40af0a0 msgr2=0x7f9ae40c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40af0a0 0x7f9ae40c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 -- 172.21.15.33:0/1596201299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc009a20 con 0x7f9ae40c8200 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae2ffd700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ccfc0 0x7f9ae414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae3fff700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 0x7f9ae40c65f0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009ef0 tx=0x7f9adc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00cc50 con 0x7f9ae40c8200 2023-10-06T05:29:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc00f040 con 0x7f9ae40c8200 2023-10-06T05:29:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8009420 con 0x7f9ae40c8200 2023-10-06T05:29:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.795+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc014480 con 0x7f9ae40c8200 2023-10-06T05:29:03.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.796+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae40c9e40 con 0x7f9ae40c8200 2023-10-06T05:29:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.796+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae40ca030 con 0x7f9ae40c8200 2023-10-06T05:29:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.797+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9adc0147f0 con 0x7f9ae40c8200 2023-10-06T05:29:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.798+0000 7f9acb7fe700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.798+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9adc085630 con 0x7f9ae40c8200 2023-10-06T05:29:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.799+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:03.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.800+0000 7f9ae37fe700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40058d0 tx=0x7f9ad4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:03.803+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9adc0525b0 con 0x7f9ae40c8200 2023-10-06T05:29:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.005+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ae404c380 con 0x7f9ae40c8200 2023-10-06T05:29:04.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.006+0000 7f9acb7fe700 1 -- 172.21.15.33:0/1596201299 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17265 (secure 0 0 0) 0x7f9adc055bd0 con 0x7f9ae40c8200 2023-10-06T05:29:04.008 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:04.009 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":198,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:28:53.971341+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:51.598576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:52.345977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1091933932":"2023-10-06T06:28:53.141475+0000","172.21.15.167:6837/3335696661":"2023-10-07T05:28:04.232599+0000","172.21.15.167:6835/3335696661":"2023-10-07T05:28:04.232599+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:04.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 msgr2=0x7f9ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40058d0 tx=0x7f9ad4009040 comp rx=0 tx=0).stop 2023-10-06T05:29:04.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 msgr2=0x7f9ae40c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 0x7f9ae40c65f0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009ef0 tx=0x7f9adc00b040 comp rx=0 tx=0).stop 2023-10-06T05:29:04.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 shutdown_connections 2023-10-06T05:29:04.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ad0066f30 0x7f9ad00693e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ae40c8200 0x7f9ae40c65f0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ae40af0a0 0x7f9ae40c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 --2- 172.21.15.33:0/1596201299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ae40ccfc0 0x7f9ae414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 >> 172.21.15.33:0/1596201299 conn(0x7f9ae406cff0 msgr2=0x7f9ae4157180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:04.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 shutdown_connections 2023-10-06T05:29:04.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.009+0000 7f9ae8d9d700 1 -- 172.21.15.33:0/1596201299 wait complete. 2023-10-06T05:29:04.029 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:29:04.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.215+0000 7ffaf2f9d700 1 Processor -- start 2023-10-06T05:29:04.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.215+0000 7ffaf2f9d700 1 -- start start 2023-10-06T05:29:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.216+0000 7ffaf2f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffaec0cf3b0 0x7ffaec1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.216+0000 7ffaf2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0af0a0 0x7ffaec1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.216+0000 7ffaf2f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec0cbd80 con 0x7ffaec0ccfc0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.216+0000 7ffaf2f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec0af5f0 con 0x7ffaec0cf3b0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.216+0000 7ffaf2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0af8d0 con 0x7ffaec0af0a0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf1f9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffaec0ccfc0 0x7ffaec0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36904/0 (socket says 172.21.15.33:36904) 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf179a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffaec0cf3b0 0x7ffaec1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf1f9b700 1 -- 172.21.15.33:0/2463118151 learned_addr learned my addr 172.21.15.33:0/2463118151 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf0f99700 1 -- 172.21.15.33:0/2463118151 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023893573 0 0) 0x7ffaec0cbd80 con 0x7ffaec0ccfc0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf0f99700 1 -- 172.21.15.33:0/2463118151 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad8003570 con 0x7ffaec0ccfc0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf279c700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0af0a0 0x7ffaec1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf179a700 1 -- 172.21.15.33:0/2463118151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0af0a0 msgr2=0x7ffaec1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:04.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf179a700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0af0a0 0x7ffaec1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf179a700 1 -- 172.21.15.33:0/2463118151 >> v1:172.21.15.33:6789/0 conn(0x7ffaec0ccfc0 legacy=0x7ffaec0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf179a700 1 -- 172.21.15.33:0/2463118151 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0afbb0 con 0x7ffaec0cf3b0 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.217+0000 7ffaf279c700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0af0a0 0x7ffaec1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf179a700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0cf3b0 0x7ffaec1594a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009910 tx=0x7ffadc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f7874457f3897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf0f99700 1 -- 172.21.15.33:0/2463118151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00b640 con 0x7ffaec0cf3b0 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf0f99700 1 -- 172.21.15.33:0/2463118151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc00b7a0 con 0x7ffaec0cf3b0 2023-10-06T05:29:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf0f99700 1 -- 172.21.15.33:0/2463118151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00bb60 con 0x7ffaec0cf3b0 2023-10-06T05:29:04.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/2463118151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0cf3b0 msgr2=0x7ffaec1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0cf3b0 0x7ffaec1594a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009910 tx=0x7ffadc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/2463118151 shutdown_connections 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0af0a0 0x7ffaec1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/2463118151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0cf3b0 0x7ffaec1594a0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.218+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/2463118151 >> 172.21.15.33:0/2463118151 conn(0x7ffaec06cff0 msgr2=0x7ffaec06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.219+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/2463118151 shutdown_connections 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.219+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/2463118151 wait complete. 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.220+0000 7ffaf2f9d700 1 Processor -- start 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.220+0000 7ffaf2f9d700 1 -- start start 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf2f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 0x7ffaec14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaec0ccfc0 0x7ffaec0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0b2880 0x7ffaec0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf2f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec14ea80 con 0x7ffaec0af0a0 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0b2d00 con 0x7ffaec0ccfc0 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0c82e0 con 0x7ffaec0b2880 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf1f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 0x7ffaec14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf1f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 0x7ffaec14e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45882/0 (socket says 172.21.15.33:45882) 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf1f9b700 1 -- 172.21.15.33:0/1244636125 learned_addr learned my addr 172.21.15.33:0/1244636125 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.221+0000 7ffaf279c700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0b2880 0x7ffaec0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf179a700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaec0ccfc0 0x7ffaec0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf1f9b700 1 -- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0b2880 msgr2=0x7ffaec0c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf1f9b700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0b2880 0x7ffaec0c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf1f9b700 1 -- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaec0ccfc0 msgr2=0x7ffaec0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf1f9b700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaec0ccfc0 0x7ffaec0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf1f9b700 1 -- 172.21.15.33:0/1244636125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc009420 con 0x7ffaec0af0a0 2023-10-06T05:29:04.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf1f9b700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 0x7ffaec14e540 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c7c0 tx=0x7ffae8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:04.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8005b30 con 0x7ffaec0af0a0 2023-10-06T05:29:04.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8005c90 con 0x7ffaec0af0a0 2023-10-06T05:29:04.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8007340 con 0x7ffaec0af0a0 2023-10-06T05:29:04.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec0c85c0 con 0x7ffaec0af0a0 2023-10-06T05:29:04.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.222+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec0c8b80 con 0x7ffaec0af0a0 2023-10-06T05:29:04.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.223+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec0b32a0 con 0x7ffaec0af0a0 2023-10-06T05:29:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.224+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffae801b400 con 0x7ffaec0af0a0 2023-10-06T05:29:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.226+0000 7ffae1ffb700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffad0067000 0x7ffad00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.226+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffae8084260 con 0x7ffaec0af0a0 2023-10-06T05:29:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.226+0000 7ffaf179a700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffad0067000 0x7ffad00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.227+0000 7ffaf179a700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffad0067000 0x7ffad00694b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0b0250 tx=0x7ffadc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.230+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffae80512e0 con 0x7ffaec0af0a0 2023-10-06T05:29:04.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.477+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaec04c380 con 0x7ffaec0af0a0 2023-10-06T05:29:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.477+0000 7ffae1ffb700 1 -- 172.21.15.33:0/1244636125 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7ffae8054900 con 0x7ffaec0af0a0 2023-10-06T05:29:04.479 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:04.479 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffad0067000 msgr2=0x7ffad00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffad0067000 0x7ffad00694b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0b0250 tx=0x7ffadc017000 comp rx=0 tx=0).stop 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 msgr2=0x7ffaec14e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 0x7ffaec14e540 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c7c0 tx=0x7ffae8007e00 comp rx=0 tx=0).stop 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 shutdown_connections 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffad0067000 0x7ffad00694b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffaec0b2880 0x7ffaec0c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaec0af0a0 0x7ffaec14e540 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 --2- 172.21.15.33:0/1244636125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffaec0ccfc0 0x7ffaec0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 >> 172.21.15.33:0/1244636125 conn(0x7ffaec06cff0 msgr2=0x7ffaec0ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 shutdown_connections 2023-10-06T05:29:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.480+0000 7ffaf2f9d700 1 -- 172.21.15.33:0/1244636125 wait complete. 2023-10-06T05:29:04.484 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:29:04.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.684+0000 7f6ea899d700 1 Processor -- start 2023-10-06T05:29:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.684+0000 7f6ea899d700 1 -- start start 2023-10-06T05:29:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.685+0000 7f6ea899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6ea406e140 0x7f6ea415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.685+0000 7f6ea899d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea406f220 con 0x7f6ea40afdd0 2023-10-06T05:29:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.685+0000 7f6ea899d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea415ae50 con 0x7f6ea4159530 2023-10-06T05:29:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.685+0000 7f6ea899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40b1e00 con 0x7f6ea406e140 2023-10-06T05:29:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6ea40afdd0 0x7f6ea40b01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36916/0 (socket says 172.21.15.33:36916) 2023-10-06T05:29:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea3fff700 1 -- 172.21.15.33:0/64005059 learned_addr learned my addr 172.21.15.33:0/64005059 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea2ffd700 1 --2- 172.21.15.33:0/64005059 >> v2:172.21.15.167:3300/0 conn(0x7f6ea406e140 0x7f6ea415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483104822 0 0) 0x7f6ea406f220 con 0x7f6ea40afdd0 2023-10-06T05:29:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e8c003570 con 0x7f6ea40afdd0 2023-10-06T05:29:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695092135 0 0) 0x7f6ea415ae50 con 0x7f6ea4159530 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea406f220 con 0x7f6ea4159530 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea2ffd700 1 -- 172.21.15.33:0/64005059 >> v1:172.21.15.167:6789/0 conn(0x7f6ea4159530 legacy=0x7f6ea406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea2ffd700 1 -- 172.21.15.33:0/64005059 >> v1:172.21.15.33:6789/0 conn(0x7f6ea40afdd0 legacy=0x7f6ea40b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.686+0000 7f6ea2ffd700 1 -- 172.21.15.33:0/64005059 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea406e610 con 0x7f6ea406e140 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.687+0000 7f6ea2ffd700 1 --2- 172.21.15.33:0/64005059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea406e140 0x7f6ea415a910 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6e980098e0 tx=0x7f6e980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d997cf6a9508e3f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.687+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98011630 con 0x7f6ea406e140 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.687+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98011790 con 0x7f6ea406e140 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.687+0000 7f6ea27fc700 1 -- 172.21.15.33:0/64005059 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98011b30 con 0x7f6ea406e140 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 -- 172.21.15.33:0/64005059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea406e140 msgr2=0x7f6ea415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 --2- 172.21.15.33:0/64005059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea406e140 0x7f6ea415a910 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6e980098e0 tx=0x7f6e980091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 -- 172.21.15.33:0/64005059 shutdown_connections 2023-10-06T05:29:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 --2- 172.21.15.33:0/64005059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea406e140 0x7f6ea415a910 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 -- 172.21.15.33:0/64005059 >> 172.21.15.33:0/64005059 conn(0x7f6ea406c560 msgr2=0x7f6ea406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 -- 172.21.15.33:0/64005059 shutdown_connections 2023-10-06T05:29:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.688+0000 7f6ea899d700 1 -- 172.21.15.33:0/64005059 wait complete. 2023-10-06T05:29:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.689+0000 7f6ea899d700 1 Processor -- start 2023-10-06T05:29:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.689+0000 7f6ea899d700 1 -- start start 2023-10-06T05:29:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea406e140 0x7f6ea4140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea40afdd0 0x7f6ea40c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 0x7f6ea40c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40c8cf0 con 0x7f6ea4159530 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40c80e0 con 0x7f6ea40afdd0 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40c83c0 con 0x7f6ea406e140 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.690+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 0x7f6ea40c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 0x7f6ea40c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45884/0 (socket says 172.21.15.33:45884) 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 -- 172.21.15.33:0/855339361 learned_addr learned my addr 172.21.15.33:0/855339361 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea2ffd700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea40afdd0 0x7f6ea40c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea37fe700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea406e140 0x7f6ea4140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 -- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea406e140 msgr2=0x7f6ea4140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea406e140 0x7f6ea4140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 -- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea40afdd0 msgr2=0x7f6ea40c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea40afdd0 0x7f6ea40c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 -- 172.21.15.33:0/855339361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98009420 con 0x7f6ea4159530 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea37fe700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea406e140 0x7f6ea4140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea2ffd700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea40afdd0 0x7f6ea40c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.691+0000 7f6ea3fff700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 0x7f6ea40c7af0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00cc60 tx=0x7f6e9c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.692+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00ce70 con 0x7f6ea4159530 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.692+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c00ab70 con 0x7f6ea4159530 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.692+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0070f0 con 0x7f6ea4159530 2023-10-06T05:29:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.692+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea40c86a0 con 0x7f6ea4159530 2023-10-06T05:29:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.692+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea41552b0 con 0x7f6ea4159530 2023-10-06T05:29:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.693+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea404c380 con 0x7f6ea4159530 2023-10-06T05:29:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.694+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e9c0074a0 con 0x7f6ea4159530 2023-10-06T05:29:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.694+0000 7f6e8b7fe700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.694+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6e9c084cf0 con 0x7f6ea4159530 2023-10-06T05:29:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.695+0000 7f6ea37fe700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.696+0000 7f6ea37fe700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e90066f30 0x7f6e900693e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6e94003990 tx=0x7f6e94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.699+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e9c055780 con 0x7f6ea4159530 2023-10-06T05:29:04.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.946+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ea40c8e30 con 0x7f6ea4159530 2023-10-06T05:29:04.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.946+0000 7f6e8b7fe700 1 -- 172.21.15.33:0/855339361 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f6e9c055310 con 0x7f6ea4159530 2023-10-06T05:29:04.948 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:04.948 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":154,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:53.374342+0000","modified":"2023-10-06T05:28:55.980424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":195,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9801":{"gid":9801,"name":"c","rank":0,"incarnation":151,"state":"up:active","state_seq":20,"addr":"172.21.15.33:6835/4280620355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":4280620355},{"type":"v1","addr":"172.21.15.33:6835","nonce":4280620355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T05:29:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e90066f30 msgr2=0x7f6e900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e90066f30 0x7f6e900693e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6e94003990 tx=0x7f6e94009040 comp rx=0 tx=0).stop 2023-10-06T05:29:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 msgr2=0x7f6ea40c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 0x7f6ea40c7af0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00cc60 tx=0x7f6e9c001ae0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 shutdown_connections 2023-10-06T05:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ea4159530 0x7f6ea40c7af0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ea406e140 0x7f6ea4140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 --2- 172.21.15.33:0/855339361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ea40afdd0 0x7f6ea40c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 >> 172.21.15.33:0/855339361 conn(0x7f6ea406c560 msgr2=0x7f6ea406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 shutdown_connections 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:04.949+0000 7f6ea899d700 1 -- 172.21.15.33:0/855339361 wait complete. 2023-10-06T05:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 154 2023-10-06T05:29:04.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.154+0000 7f2de7fff700 1 Processor -- start 2023-10-06T05:29:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.154+0000 7f2de7fff700 1 -- start start 2023-10-06T05:29:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.155+0000 7f2de7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2de8159bb0 0x7f2de80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.155+0000 7f2de7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80cbd80 con 0x7f2de80cf3b0 2023-10-06T05:29:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.155+0000 7f2de7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80af5f0 con 0x7f2de80ccfc0 2023-10-06T05:29:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.155+0000 7f2de7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de80af8d0 con 0x7f2de8159bb0 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.155+0000 7f2de67fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2de80cf3b0 0x7f2de81594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36928/0 (socket says 172.21.15.33:36928) 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de67fc700 1 -- 172.21.15.33:0/1412311710 learned_addr learned my addr 172.21.15.33:0/1412311710 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de77fe700 1 --2- 172.21.15.33:0/1412311710 >> v2:172.21.15.33:3300/0 conn(0x7f2de8159bb0 0x7f2de80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de5ffb700 1 -- 172.21.15.33:0/1412311710 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316404553 0 0) 0x7f2de80cbd80 con 0x7f2de80cf3b0 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de5ffb700 1 -- 172.21.15.33:0/1412311710 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd0003570 con 0x7f2de80cf3b0 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de77fe700 1 -- 172.21.15.33:0/1412311710 >> v1:172.21.15.167:6789/0 conn(0x7f2de80ccfc0 legacy=0x7f2de80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de77fe700 1 -- 172.21.15.33:0/1412311710 >> v1:172.21.15.33:6789/0 conn(0x7f2de80cf3b0 legacy=0x7f2de81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de77fe700 1 -- 172.21.15.33:0/1412311710 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de80afbb0 con 0x7f2de8159bb0 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.156+0000 7f2de77fe700 1 --2- 172.21.15.33:0/1412311710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8159bb0 0x7f2de80b0460 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009e90 tx=0x7f2de0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4845b4a89337c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:05.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de5ffb700 1 -- 172.21.15.33:0/1412311710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de000e070 con 0x7f2de8159bb0 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de5ffb700 1 -- 172.21.15.33:0/1412311710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de000bc40 con 0x7f2de8159bb0 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de5ffb700 1 -- 172.21.15.33:0/1412311710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0012430 con 0x7f2de8159bb0 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 -- 172.21.15.33:0/1412311710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8159bb0 msgr2=0x7f2de80b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 --2- 172.21.15.33:0/1412311710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8159bb0 0x7f2de80b0460 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009e90 tx=0x7f2de0009780 comp rx=0 tx=0).stop 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 -- 172.21.15.33:0/1412311710 shutdown_connections 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 --2- 172.21.15.33:0/1412311710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8159bb0 0x7f2de80b0460 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 -- 172.21.15.33:0/1412311710 >> 172.21.15.33:0/1412311710 conn(0x7f2de806cff0 msgr2=0x7f2de806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 -- 172.21.15.33:0/1412311710 shutdown_connections 2023-10-06T05:29:05.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.157+0000 7f2de7fff700 1 -- 172.21.15.33:0/1412311710 wait complete. 2023-10-06T05:29:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.159+0000 7f2de7fff700 1 Processor -- start 2023-10-06T05:29:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.159+0000 7f2de7fff700 1 -- start start 2023-10-06T05:29:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de80ccfc0 0x7f2de813ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8159bb0 0x7f2de81511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 0x7f2de8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8158700 con 0x7f2de8151710 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8151b70 con 0x7f2de8159bb0 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80c8810 con 0x7f2de80ccfc0 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 0x7f2de8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 0x7f2de8157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45904/0 (socket says 172.21.15.33:45904) 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de77fe700 1 -- 172.21.15.33:0/3527919143 learned_addr learned my addr 172.21.15.33:0/3527919143 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de67fc700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8159bb0 0x7f2de81511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de6ffd700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de80ccfc0 0x7f2de813ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.160+0000 7f2de77fe700 1 -- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de80ccfc0 msgr2=0x7f2de813ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de77fe700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de80ccfc0 0x7f2de813ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de77fe700 1 -- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8159bb0 msgr2=0x7f2de81511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de77fe700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8159bb0 0x7f2de81511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de77fe700 1 -- 172.21.15.33:0/3527919143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de0009a20 con 0x7f2de8151710 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de67fc700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8159bb0 0x7f2de81511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de6ffd700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de80ccfc0 0x7f2de813ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de77fe700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 0x7f2de8157990 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f2de806be20 tx=0x7f2de0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de001b630 con 0x7f2de8151710 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de001b7d0 con 0x7f2de8151710 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de80c7ee0 con 0x7f2de8151710 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de001bb90 con 0x7f2de8151710 2023-10-06T05:29:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.161+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de80c84a0 con 0x7f2de8151710 2023-10-06T05:29:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.162+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de804c380 con 0x7f2de8151710 2023-10-06T05:29:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.163+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2de001c400 con 0x7f2de8151710 2023-10-06T05:29:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.164+0000 7f2dceffd700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.164+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f2de0086610 con 0x7f2de8151710 2023-10-06T05:29:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.165+0000 7f2de6ffd700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.165+0000 7f2de6ffd700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2dd4066f30 0x7f2dd40693e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2de8152380 tx=0x7f2dd8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.168+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2de0053640 con 0x7f2de8151710 2023-10-06T05:29:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.372+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2de80cc920 con 0x7f2de8151710 2023-10-06T05:29:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.373+0000 7f2dceffd700 1 -- 172.21.15.33:0/3527919143 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17265 (secure 0 0 0) 0x7f2de0056c60 con 0x7f2de8151710 2023-10-06T05:29:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:05.375 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":198,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:28:53.971341+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:51.598576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:52.345977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1091933932":"2023-10-06T06:28:53.141475+0000","172.21.15.167:6837/3335696661":"2023-10-07T05:28:04.232599+0000","172.21.15.167:6835/3335696661":"2023-10-07T05:28:04.232599+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.375+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2dd4066f30 msgr2=0x7f2dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.375+0000 7f2de7fff700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2dd4066f30 0x7f2dd40693e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2de8152380 tx=0x7f2dd8009040 comp rx=0 tx=0).stop 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 msgr2=0x7f2de8157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 0x7f2de8157990 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f2de806be20 tx=0x7f2de0019000 comp rx=0 tx=0).stop 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 shutdown_connections 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2de8151710 0x7f2de8157990 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2de80ccfc0 0x7f2de813ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 --2- 172.21.15.33:0/3527919143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2de8159bb0 0x7f2de81511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 >> 172.21.15.33:0/3527919143 conn(0x7f2de806cff0 msgr2=0x7f2de8152550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 shutdown_connections 2023-10-06T05:29:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.376+0000 7f2de7fff700 1 -- 172.21.15.33:0/3527919143 wait complete. 2023-10-06T05:29:05.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:29:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.581+0000 7f743d79d700 1 Processor -- start 2023-10-06T05:29:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.581+0000 7f743d79d700 1 -- start start 2023-10-06T05:29:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.582+0000 7f743d79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f74380c8090 0x7f7438156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.582+0000 7f743d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7438159e10 0x7f7438157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.582+0000 7f743d79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743806a820 con 0x7f7438156880 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.582+0000 7f743d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74380cc290 con 0x7f7438159e10 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.582+0000 7f743d79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743806bd10 con 0x7f74380c8090 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f7437fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7438156880 0x7f74380c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36940/0 (socket says 172.21.15.33:36940) 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f743cf9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7438159e10 0x7f7438157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f7437fff700 1 -- 172.21.15.33:0/2949363102 learned_addr learned my addr 172.21.15.33:0/2949363102 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f74377fe700 1 --2- 172.21.15.33:0/2949363102 >> v2:172.21.15.167:3300/0 conn(0x7f74380c8090 0x7f7438156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f743cf9c700 1 -- 172.21.15.33:0/2949363102 >> v2:172.21.15.167:3300/0 conn(0x7f74380c8090 msgr2=0x7f7438156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f743cf9c700 1 --2- 172.21.15.33:0/2949363102 >> v2:172.21.15.167:3300/0 conn(0x7f74380c8090 0x7f7438156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f743cf9c700 1 -- 172.21.15.33:0/2949363102 >> v1:172.21.15.33:6789/0 conn(0x7f7438156880 legacy=0x7f74380c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f743cf9c700 1 -- 172.21.15.33:0/2949363102 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743815a2f0 con 0x7f7438159e10 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.583+0000 7f7436ffd700 1 -- 172.21.15.33:0/2949363102 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302094128 0 0) 0x7f743806a820 con 0x7f7438156880 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743cf9c700 1 --2- 172.21.15.33:0/2949363102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7438159e10 0x7f7438157190 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f7430009e60 tx=0x7f7430009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf705751d3429cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f7436ffd700 1 -- 172.21.15.33:0/2949363102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743000e070 con 0x7f7438159e10 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f7436ffd700 1 -- 172.21.15.33:0/2949363102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743000bb20 con 0x7f7438159e10 2023-10-06T05:29:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f7436ffd700 1 -- 172.21.15.33:0/2949363102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430012430 con 0x7f7438159e10 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743d79d700 1 -- 172.21.15.33:0/2949363102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7438159e10 msgr2=0x7f7438157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743d79d700 1 --2- 172.21.15.33:0/2949363102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7438159e10 0x7f7438157190 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f7430009e60 tx=0x7f7430009780 comp rx=0 tx=0).stop 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743d79d700 1 -- 172.21.15.33:0/2949363102 shutdown_connections 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743d79d700 1 --2- 172.21.15.33:0/2949363102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7438159e10 0x7f7438157190 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743d79d700 1 --2- 172.21.15.33:0/2949363102 >> v2:172.21.15.167:3300/0 conn(0x7f74380c8090 0x7f7438156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.584+0000 7f743d79d700 1 -- 172.21.15.33:0/2949363102 >> 172.21.15.33:0/2949363102 conn(0x7f743806c560 msgr2=0x7f743806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.585+0000 7f743d79d700 1 -- 172.21.15.33:0/2949363102 shutdown_connections 2023-10-06T05:29:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.585+0000 7f743d79d700 1 -- 172.21.15.33:0/2949363102 wait complete. 2023-10-06T05:29:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.586+0000 7f743d79d700 1 Processor -- start 2023-10-06T05:29:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.586+0000 7f743d79d700 1 -- start start 2023-10-06T05:29:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f743d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 0x7f743815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f743d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7438156880 0x7f7438155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f743d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7438159e10 0x7f743815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f743d79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381505d0 con 0x7f74380c8090 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f743d79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743815e7a0 con 0x7f7438159e10 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f743d79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743815ea80 con 0x7f7438156880 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f7437fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 0x7f743815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f7437fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 0x7f743815e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45922/0 (socket says 172.21.15.33:45922) 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.587+0000 7f7437fff700 1 -- 172.21.15.33:0/3056343226 learned_addr learned my addr 172.21.15.33:0/3056343226 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f7437fff700 1 -- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7438156880 msgr2=0x7f7438155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f7437fff700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7438156880 0x7f7438155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f7437fff700 1 -- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7438159e10 msgr2=0x7f743815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f7437fff700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7438159e10 0x7f743815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f7437fff700 1 -- 172.21.15.33:0/3056343226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430009a20 con 0x7f74380c8090 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f7437fff700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 0x7f743815e1c0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f742800c7e0 tx=0x7f7428007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428005b30 con 0x7f74380c8090 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428005c90 con 0x7f74380c8090 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428007300 con 0x7f74380c8090 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743815ed60 con 0x7f74380c8090 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.588+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743815df00 con 0x7f74380c8090 2023-10-06T05:29:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.589+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743804c380 con 0x7f74380c8090 2023-10-06T05:29:05.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.590+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f742801b400 con 0x7f74380c8090 2023-10-06T05:29:05.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.592+0000 7f741ffff700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:05.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.592+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f74280842d0 con 0x7f74380c8090 2023-10-06T05:29:05.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.592+0000 7f74377fe700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:05.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.593+0000 7f74377fe700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f74380cd220 tx=0x7f742c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.596+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74280512d0 con 0x7f74380c8090 2023-10-06T05:29:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.844+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7438169050 con 0x7f74380c8090 2023-10-06T05:29:05.846 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:29:05.845+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:29:05.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.850+0000 7f741ffff700 1 -- 172.21.15.33:0/3056343226 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v155) v1 ==== 161+0+0 (secure 0 0 0) 0x7f74280548f0 con 0x7f74380c8090 2023-10-06T05:29:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 msgr2=0x7f74240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f74380cd220 tx=0x7f742c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 msgr2=0x7f743815e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 0x7f743815e1c0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f742800c7e0 tx=0x7f7428007df0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 shutdown_connections 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7424066f30 0x7f74240693e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7438159e10 0x7f743815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74380c8090 0x7f743815e1c0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 --2- 172.21.15.33:0/3056343226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7438156880 0x7f7438155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 >> 172.21.15.33:0/3056343226 conn(0x7f743806c560 msgr2=0x7f74380c26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 shutdown_connections 2023-10-06T05:29:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:05.854+0000 7f743d79d700 1 -- 172.21.15.33:0/3056343226 wait complete. 2023-10-06T05:29:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:29:05.860 INFO:tasks.ceph.mds.c.smithi033.stderr: -1301> 2023-10-06T05:28:58.850+0000 7ff1529f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:29:05.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:29:05.902 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:29:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.069+0000 7f3d5f59e700 1 Processor -- start 2023-10-06T05:29:06.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.070+0000 7f3d5f59e700 1 -- start start 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d60156370 0x7f3d600af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d600ad430 con 0x7f3d600b01f0 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d600afe20 con 0x7f3d600cc7e0 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6006bc70 con 0x7f3d60156370 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5ed9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3d600b01f0 0x7f3d600ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36958/0 (socket says 172.21.15.33:36958) 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/1651239741 learned_addr learned my addr 172.21.15.33:0/1651239741 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:06.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.071+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/1651239741 >> v2:172.21.15.167:3300/0 conn(0x7f3d60156370 0x7f3d600af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244326834 0 0) 0x7f3d600ad430 con 0x7f3d600b01f0 2023-10-06T05:29:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d48003570 con 0x7f3d600b01f0 2023-10-06T05:29:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571038342 0 0) 0x7f3d600afe20 con 0x7f3d600cc7e0 2023-10-06T05:29:06.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d600ad430 con 0x7f3d600cc7e0 2023-10-06T05:29:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995442088 0 0) 0x7f3d48003570 con 0x7f3d600b01f0 2023-10-06T05:29:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 >> v2:172.21.15.167:3300/0 conn(0x7f3d60156370 msgr2=0x7f3d600af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 --2- 172.21.15.33:0/1651239741 >> v2:172.21.15.167:3300/0 conn(0x7f3d60156370 0x7f3d600af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 >> v1:172.21.15.167:6789/0 conn(0x7f3d600cc7e0 legacy=0x7f3d600c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600cccc0 con 0x7f3d600b01f0 2023-10-06T05:29:06.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3d580017f0 con 0x7f3d600b01f0 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 >> v1:172.21.15.33:6789/0 conn(0x7f3d600b01f0 legacy=0x7f3d600ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d48003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d48004810 0x7f3d48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d480051c0 0x7f3d4800dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6006bc70 con 0x7f3d48003bf0 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4800e370 con 0x7f3d48004810 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4800e530 con 0x7f3d480051c0 2023-10-06T05:29:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.072+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d48003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d48004810 0x7f3d48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d480051c0 0x7f3d4800dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5e59c700 1 -- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d480051c0 msgr2=0x7f3d4800dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d480051c0 0x7f3d4800dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5e59c700 1 -- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d48004810 msgr2=0x7f3d48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d48004810 0x7f3d48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5e59c700 1 -- 172.21.15.33:0/1651239741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48003570 con 0x7f3d48003bf0 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/1651239741 reap_dead start 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d48003fe0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006f00 tx=0x7f3d500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c986ef114d5a2d1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:06.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000e710 con 0x7f3d48003bf0 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5000e870 con 0x7f3d48003bf0 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.073+0000 7f3d5d59a700 1 -- 172.21.15.33:0/1651239741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000ec30 con 0x7f3d48003bf0 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 -- 172.21.15.33:0/1651239741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 msgr2=0x7f3d48003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d48003fe0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006f00 tx=0x7f3d500091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 -- 172.21.15.33:0/1651239741 shutdown_connections 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/1651239741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d48003fe0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 -- 172.21.15.33:0/1651239741 >> 172.21.15.33:0/1651239741 conn(0x7f3d6006d030 msgr2=0x7f3d600cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 -- 172.21.15.33:0/1651239741 shutdown_connections 2023-10-06T05:29:06.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.074+0000 7f3d5f59e700 1 -- 172.21.15.33:0/1651239741 wait complete. 2023-10-06T05:29:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.075+0000 7f3d5f59e700 1 Processor -- start 2023-10-06T05:29:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.075+0000 7f3d5f59e700 1 -- start start 2023-10-06T05:29:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.076+0000 7f3d5f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d60147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.076+0000 7f3d5f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d60157810 0x7f3d6015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.076+0000 7f3d5f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015d5d0 0x7f3d6015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.076+0000 7f3d5f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60169e10 con 0x7f3d48003bf0 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.076+0000 7f3d5f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6015df40 con 0x7f3d6015d5d0 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.076+0000 7f3d5f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60157c80 con 0x7f3d60157810 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d60147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d60147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45942/0 (socket says 172.21.15.33:45942) 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 -- 172.21.15.33:0/2069089761 learned_addr learned my addr 172.21.15.33:0/2069089761 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015d5d0 0x7f3d6015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d60157810 0x7f3d6015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 -- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d60157810 msgr2=0x7f3d6015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d60157810 0x7f3d6015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 -- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015d5d0 msgr2=0x7f3d6015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015d5d0 0x7f3d6015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 -- 172.21.15.33:0/2069089761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60157f10 con 0x7f3d48003bf0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d60147ee0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006ed0 tx=0x7f3d5001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50022070 con 0x7f3d48003bf0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d50013c40 con 0x7f3d48003bf0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.077+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5001c450 con 0x7f3d48003bf0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.078+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d601581f0 con 0x7f3d48003bf0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.078+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d601587b0 con 0x7f3d48003bf0 2023-10-06T05:29:06.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.079+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6016a0f0 con 0x7f3d48003bf0 2023-10-06T05:29:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.079+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d5001c880 con 0x7f3d48003bf0 2023-10-06T05:29:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.081+0000 7f3d4e7fc700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d40066f30 0x7f3d400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.081+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3d50085ea0 con 0x7f3d48003bf0 2023-10-06T05:29:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.081+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d40066f30 0x7f3d400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.082+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d40066f30 0x7f3d400693e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3d6015cfc0 tx=0x7f3d54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:06.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.085+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d50052dc0 con 0x7f3d48003bf0 2023-10-06T05:29:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.333+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3d6004c380 con 0x7f3d48003bf0 2023-10-06T05:29:06.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.851+0000 7f3d4e7fc700 1 -- 172.21.15.33:0/2069089761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v156) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d50017020 con 0x7f3d48003bf0 2023-10-06T05:29:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.854+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d40066f30 msgr2=0x7f3d400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.854+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d40066f30 0x7f3d400693e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3d6015cfc0 tx=0x7f3d54009040 comp rx=0 tx=0).stop 2023-10-06T05:29:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 msgr2=0x7f3d60147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d60147ee0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006ed0 tx=0x7f3d5001a000 comp rx=0 tx=0).stop 2023-10-06T05:29:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 shutdown_connections 2023-10-06T05:29:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d40066f30 0x7f3d400693e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d6015d5d0 0x7f3d6015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d48003bf0 0x7f3d60147ee0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 --2- 172.21.15.33:0/2069089761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d60157810 0x7f3d6015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 >> 172.21.15.33:0/2069089761 conn(0x7f3d6006d030 msgr2=0x7f3d60069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 shutdown_connections 2023-10-06T05:29:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:06.855+0000 7f3d5f59e700 1 -- 172.21.15.33:0/2069089761 wait complete. 2023-10-06T05:29:06.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:29:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.114+0000 7fc0b6f9d700 1 Processor -- start 2023-10-06T05:29:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.115+0000 7fc0b6f9d700 1 -- start start 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b6f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc0b00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b6f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b0069ad0 con 0x7fc0b00acd00 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b6f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b0069db0 con 0x7fc0b0155aa0 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b6f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b006a090 con 0x7fc0b0155e70 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b579a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc0b00acd00 0x7fc0b00c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36968/0 (socket says 172.21.15.33:36968) 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b579a700 1 -- 172.21.15.33:0/4138571692 learned_addr learned my addr 172.21.15.33:0/4138571692 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b679c700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc0b00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.116+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626038447 0 0) 0x7fc0b0069ad0 con 0x7fc0b00acd00 2023-10-06T05:29:07.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09c003570 con 0x7fc0b00acd00 2023-10-06T05:29:07.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520336698 0 0) 0x7fc0b0069db0 con 0x7fc0b0155aa0 2023-10-06T05:29:07.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b0069ad0 con 0x7fc0b0155aa0 2023-10-06T05:29:07.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820655127 0 0) 0x7fc09c003570 con 0x7fc0b00acd00 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 msgr2=0x7fc0b00aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc0b00aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 >> v1:172.21.15.167:6789/0 conn(0x7fc0b0155aa0 legacy=0x7fc0b00ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b006a370 con 0x7fc0b00acd00 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc0a0001470 con 0x7fc0b00acd00 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b679c700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc0b00aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 >> v1:172.21.15.33:6789/0 conn(0x7fc0b00acd00 legacy=0x7fc0b00c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c003c20 0x7fc09c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc09c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc09c004d50 0x7fc09c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b006a090 con 0x7fc09c004d50 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c00bb70 con 0x7fc09c003c20 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c00df40 con 0x7fc0b0155e70 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b679c700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc09c004d50 0x7fc09c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b579a700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc09c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.117+0000 7fc0b5f9b700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c003c20 0x7fc09c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b679c700 1 -- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 msgr2=0x7fc09c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b679c700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0155e70 0x7fc09c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b679c700 1 -- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c003c20 msgr2=0x7fc09c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b679c700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c003c20 0x7fc09c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:07.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b679c700 1 -- 172.21.15.33:0/4138571692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a8009a20 con 0x7fc09c004d50 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b5f9b700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c003c20 0x7fc09c004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b679c700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc09c004d50 0x7fc09c0051c0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fc0b01562d0 tx=0x7fc0a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4ff234620e46fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8009c50 con 0x7fc09c004d50 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a800f040 con 0x7fc09c004d50 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.118+0000 7fc0b4f99700 1 -- 172.21.15.33:0/4138571692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a800ee70 con 0x7fc09c004d50 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/4138571692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc09c004d50 msgr2=0x7fc09c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/4138571692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc09c004d50 0x7fc09c0051c0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fc0b01562d0 tx=0x7fc0a800b040 comp rx=0 tx=0).stop 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b679c700 1 -- 172.21.15.33:0/4138571692 reap_dead start 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/4138571692 shutdown_connections 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/4138571692 >> 172.21.15.33:0/4138571692 conn(0x7fc0b0068090 msgr2=0x7fc0b006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:07.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/4138571692 shutdown_connections 2023-10-06T05:29:07.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.119+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/4138571692 wait complete. 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.120+0000 7fc0b6f9d700 1 Processor -- start 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.121+0000 7fc0b6f9d700 1 -- start start 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.121+0000 7fc0b6f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c004d50 0x7fc0b0166440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.121+0000 7fc0b6f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 0x7fc0b014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.121+0000 7fc0b6f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0149d60 0x7fc0b00cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b6f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b014b520 con 0x7fc0b014a480 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b6f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00ac080 con 0x7fc09c004d50 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b6f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b014a910 con 0x7fc0b0149d60 2023-10-06T05:29:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 0x7fc0b014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 0x7fc0b014b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45960/0 (socket says 172.21.15.33:45960) 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 -- 172.21.15.33:0/1441431115 learned_addr learned my addr 172.21.15.33:0/1441431115 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b5f9b700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c004d50 0x7fc0b0166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b679c700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0149d60 0x7fc0b00cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 -- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0149d60 msgr2=0x7fc0b00cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0149d60 0x7fc0b00cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 -- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c004d50 msgr2=0x7fc0b0166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c004d50 0x7fc0b0166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 -- 172.21.15.33:0/1441431115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b014abf0 con 0x7fc0b014a480 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b679c700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0149d60 0x7fc0b00cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.122+0000 7fc0b579a700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 0x7fc0b014b9b0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00073c0 tx=0x7fc0a0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.123+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0000cc0 con 0x7fc0b014a480 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.123+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a0007780 con 0x7fc0b014a480 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.123+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000fb00 con 0x7fc0b014a480 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.123+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b014aed0 con 0x7fc0b014a480 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.123+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b014a1e0 con 0x7fc0b014a480 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.124+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b014b6d0 con 0x7fc0b014a480 2023-10-06T05:29:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.125+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0a001d460 con 0x7fc0b014a480 2023-10-06T05:29:07.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.126+0000 7fc0a5ffb700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:07.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.126+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fc0a0084980 con 0x7fc0b014a480 2023-10-06T05:29:07.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.126+0000 7fc0b5f9b700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:07.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.127+0000 7fc0b5f9b700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc094066f30 0x7fc0940693e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000c00 tx=0x7fc0ac00c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:07.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.130+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0b014b6d0 con 0x7fc0b014a480 2023-10-06T05:29:07.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:07.337+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0b0063ff0 con 0x7fc0b014a480 2023-10-06T05:29:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.049+0000 7fc0a5ffb700 1 -- 172.21.15.33:0/1441431115 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v200) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc0a00518a0 con 0x7fc0b014a480 2023-10-06T05:29:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.053+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc094066f30 msgr2=0x7fc0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.053+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc094066f30 0x7fc0940693e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000c00 tx=0x7fc0ac00c040 comp rx=0 tx=0).stop 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.053+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 msgr2=0x7fc0b014b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.053+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 0x7fc0b014b9b0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00073c0 tx=0x7fc0a0001d40 comp rx=0 tx=0).stop 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 shutdown_connections 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0b0149d60 0x7fc0b00cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc09c004d50 0x7fc0b0166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 --2- 172.21.15.33:0/1441431115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0b014a480 0x7fc0b014b9b0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 >> 172.21.15.33:0/1441431115 conn(0x7fc0b0068090 msgr2=0x7fc0b0153470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 shutdown_connections 2023-10-06T05:29:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.054+0000 7fc0b6f9d700 1 -- 172.21.15.33:0/1441431115 wait complete. 2023-10-06T05:29:08.331 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:29:08.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:29:08.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.530+0000 7fb86d99d700 1 Processor -- start 2023-10-06T05:29:09.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.530+0000 7fb86d99d700 1 -- start start 2023-10-06T05:29:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb868157320 0x7fb868159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 0x7fb8680ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680ad3a0 con 0x7fb86806e710 2023-10-06T05:29:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8680ac580 con 0x7fb868157320 2023-10-06T05:29:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680afcc0 con 0x7fb8680ad830 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb86806e710 0x7fb86806d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36978/0 (socket says 172.21.15.33:36978) 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86d19c700 1 -- 172.21.15.33:0/1650177105 learned_addr learned my addr 172.21.15.33:0/1650177105 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb86c99b700 1 --2- 172.21.15.33:0/1650177105 >> v2:172.21.15.167:3300/0 conn(0x7fb868157320 0x7fb868159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.532+0000 7fb867fff700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 0x7fb8680ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406789474 0 0) 0x7fb8680ad3a0 con 0x7fb86806e710 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb858003570 con 0x7fb86806e710 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2681783835 0 0) 0x7fb858003570 con 0x7fb86806e710 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 msgr2=0x7fb8680ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 0x7fb8680ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 >> v2:172.21.15.167:3300/0 conn(0x7fb868157320 msgr2=0x7fb868159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 --2- 172.21.15.33:0/1650177105 >> v2:172.21.15.167:3300/0 conn(0x7fb868157320 0x7fb868159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8681576f0 con 0x7fb86806e710 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb8600017f0 con 0x7fb86806e710 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 >> v1:172.21.15.33:6789/0 conn(0x7fb86806e710 legacy=0x7fb86806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb858003a60 0x7fb858003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb858004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 0x7fb8580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680ac580 con 0x7fb858004810 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680afcc0 con 0x7fb858003a60 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85800ba60 con 0x7fb8680ad830 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb867fff700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb858004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.533+0000 7fb86c99b700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb858003a60 0x7fb858003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb86d19c700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 0x7fb8580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb867fff700 1 -- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 msgr2=0x7fb8580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb867fff700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680ad830 0x7fb8580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb867fff700 1 -- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb858003a60 msgr2=0x7fb858003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb867fff700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb858003a60 0x7fb858003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb867fff700 1 -- 172.21.15.33:0/1650177105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858003570 con 0x7fb858004810 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb867fff700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb858004c80 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fb854009c00 tx=0x7fb8540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bbaafa88c0f31f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400e710 con 0x7fb858004810 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85400e870 con 0x7fb858004810 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.534+0000 7fb8677fe700 1 -- 172.21.15.33:0/1650177105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400ec30 con 0x7fb858004810 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d99d700 1 -- 172.21.15.33:0/1650177105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 msgr2=0x7fb858004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:09.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d99d700 1 --2- 172.21.15.33:0/1650177105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb858004c80 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fb854009c00 tx=0x7fb8540091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d19c700 1 -- 172.21.15.33:0/1650177105 reap_dead start 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d99d700 1 -- 172.21.15.33:0/1650177105 shutdown_connections 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d99d700 1 -- 172.21.15.33:0/1650177105 >> 172.21.15.33:0/1650177105 conn(0x7fb86806cd40 msgr2=0x7fb8680ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d99d700 1 -- 172.21.15.33:0/1650177105 shutdown_connections 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.535+0000 7fb86d99d700 1 -- 172.21.15.33:0/1650177105 wait complete. 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.536+0000 7fb86d99d700 1 Processor -- start 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 -- start start 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb8681535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8681692b0 0x7fb8680b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680b3810 0x7fb86815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680b3cb0 con 0x7fb858004810 2023-10-06T05:29:09.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8681578b0 con 0x7fb8681692b0 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.537+0000 7fb86d99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb868158150 con 0x7fb8680b3810 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb8681535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb8681535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45970/0 (socket says 172.21.15.33:45970) 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 -- 172.21.15.33:0/3417036326 learned_addr learned my addr 172.21.15.33:0/3417036326 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb867fff700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8681692b0 0x7fb8680b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 -- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680b3810 msgr2=0x7fb86815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680b3810 0x7fb86815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 -- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8681692b0 msgr2=0x7fb8680b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8681692b0 0x7fb8680b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 -- 172.21.15.33:0/3417036326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86815cb40 con 0x7fb858004810 2023-10-06T05:29:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb867fff700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8681692b0 0x7fb8680b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.538+0000 7fb86c99b700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb8681535b0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb85c011d70 tx=0x7fb85c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.539+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c009e20 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.539+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c00fc70 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.539+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c014440 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.539+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb868157b00 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.539+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86815c320 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.540+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb868063ff0 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.541+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb85c014870 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.541+0000 7fb864ff9700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb84c066f30 0x7fb84c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.542+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fb85c018030 con 0x7fb858004810 2023-10-06T05:29:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.542+0000 7fb867fff700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb84c066f30 0x7fb84c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.543+0000 7fb867fff700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb84c066f30 0x7fb84c0693e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb86815a6a0 tx=0x7fb85401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.545+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb85c052300 con 0x7fb858004810 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:08.818+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8680ac790 con 0x7fb858004810 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.058+0000 7fb864ff9700 1 -- 172.21.15.33:0/3417036326 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v201) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb85c055920 con 0x7fb858004810 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.063+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb84c066f30 msgr2=0x7fb84c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.063+0000 7fb86d99d700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb84c066f30 0x7fb84c0693e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb86815a6a0 tx=0x7fb85401a000 comp rx=0 tx=0).stop 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.063+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 msgr2=0x7fb8681535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.063+0000 7fb86d99d700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb8681535b0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb85c011d70 tx=0x7fb85c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 shutdown_connections 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb84c066f30 0x7fb84c0693e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8680b3810 0x7fb86815bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb858004810 0x7fb8681535b0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 --2- 172.21.15.33:0/3417036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8681692b0 0x7fb8680b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 >> 172.21.15.33:0/3417036326 conn(0x7fb86806cd40 msgr2=0x7fb868068720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 shutdown_connections 2023-10-06T05:29:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:09.064+0000 7fb86d99d700 1 -- 172.21.15.33:0/3417036326 wait complete. 2023-10-06T05:29:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:29:09.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:29:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.089+0000 7f8eb2d9d700 1 Processor -- start 2023-10-06T05:29:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.089+0000 7f8eb2d9d700 1 -- start start 2023-10-06T05:29:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.090+0000 7f8eb2d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8eac0b18f0 0x7f8eac0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.090+0000 7f8eb2d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac1586a0 con 0x7f8eac1552f0 2023-10-06T05:29:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.090+0000 7f8eb2d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac157ab0 con 0x7f8eac154f20 2023-10-06T05:29:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.090+0000 7f8eb2d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac06bdf0 con 0x7f8eac0b18f0 2023-10-06T05:29:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb159a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8eac1552f0 0x7f8eac0c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55454/0 (socket says 172.21.15.33:55454) 2023-10-06T05:29:10.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb159a700 1 -- 172.21.15.33:0/2972187366 learned_addr learned my addr 172.21.15.33:0/2972187366 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:10.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb259c700 1 --2- 172.21.15.33:0/2972187366 >> v2:172.21.15.167:3300/0 conn(0x7f8eac0b18f0 0x7f8eac0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050717758 0 0) 0x7f8eac1586a0 con 0x7f8eac1552f0 2023-10-06T05:29:10.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e98003570 con 0x7f8eac1552f0 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16557460 0 0) 0x7f8e98003570 con 0x7f8eac1552f0 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 >> v2:172.21.15.167:3300/0 conn(0x7f8eac0b18f0 msgr2=0x7f8eac0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 --2- 172.21.15.33:0/2972187366 >> v2:172.21.15.167:3300/0 conn(0x7f8eac0b18f0 0x7f8eac0ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 >> v1:172.21.15.167:6789/0 conn(0x7f8eac154f20 legacy=0x7f8eac0b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac157d70 con 0x7f8eac1552f0 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.091+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f8e9c001470 con 0x7f8eac1552f0 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 >> v1:172.21.15.33:6789/0 conn(0x7f8eac1552f0 legacy=0x7f8eac0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e98003a60 0x7f8e98003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e98004810 0x7f8e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e980051c0 0x7f8e9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac06bdf0 con 0x7f8e98003a60 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9800c0e0 con 0x7f8e98004810 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9800c2a0 con 0x7f8e980051c0 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb1d9b700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e98003a60 0x7f8e98003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb159a700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e98004810 0x7f8e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb259c700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e980051c0 0x7f8e9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb1d9b700 1 -- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e980051c0 msgr2=0x7f8e9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb1d9b700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e980051c0 0x7f8e9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb1d9b700 1 -- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e98004810 msgr2=0x7f8e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:10.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb1d9b700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e98004810 0x7f8e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb1d9b700 1 -- 172.21.15.33:0/2972187366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98003570 con 0x7f8e98003a60 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.092+0000 7f8eb259c700 1 -- 172.21.15.33:0/2972187366 reap_dead start 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.093+0000 7f8eb1d9b700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e98003a60 0x7f8e98003e30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80018b0 tx=0x7f8ea8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d344aa1738a361 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.093+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8012420 con 0x7f8e98003a60 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.093+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80125a0 con 0x7f8e98003a60 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.093+0000 7f8eb0d99700 1 -- 172.21.15.33:0/2972187366 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8012960 con 0x7f8e98003a60 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.093+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/2972187366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e98003a60 msgr2=0x7f8e98003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.093+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e98003a60 0x7f8e98003e30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80018b0 tx=0x7f8ea8007d30 comp rx=0 tx=0).stop 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.094+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/2972187366 shutdown_connections 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.094+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/2972187366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e98003a60 0x7f8e98003e30 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.094+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/2972187366 >> 172.21.15.33:0/2972187366 conn(0x7f8eac06d030 msgr2=0x7f8eac0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:10.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.094+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/2972187366 shutdown_connections 2023-10-06T05:29:10.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.094+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/2972187366 wait complete. 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.095+0000 7f8eb2d9d700 1 Processor -- start 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.095+0000 7f8eb2d9d700 1 -- start start 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.096+0000 7f8eb2d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e98003a60 0x7f8eac15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.096+0000 7f8eb2d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eac153040 0x7f8eac0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.096+0000 7f8eb2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 0x7f8eac0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.096+0000 7f8eb2d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac15b0c0 con 0x7f8eac152330 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.096+0000 7f8eb2d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac15b360 con 0x7f8eac153040 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.096+0000 7f8eb2d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac15b640 con 0x7f8e98003a60 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 0x7f8eac0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb1d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e98003a60 0x7f8eac15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 0x7f8eac0aa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36330/0 (socket says 172.21.15.33:36330) 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 -- 172.21.15.33:0/4244195026 learned_addr learned my addr 172.21.15.33:0/4244195026 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb159a700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eac153040 0x7f8eac0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 -- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e98003a60 msgr2=0x7f8eac15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e98003a60 0x7f8eac15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 -- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eac153040 msgr2=0x7f8eac0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eac153040 0x7f8eac0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 -- 172.21.15.33:0/4244195026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac15b920 con 0x7f8eac152330 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.097+0000 7f8eb259c700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 0x7f8eac0aa950 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40094d0 tx=0x7f8ea400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.098+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400e040 con 0x7f8eac152330 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.098+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea400f040 con 0x7f8eac152330 2023-10-06T05:29:10.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.098+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac0cb3d0 con 0x7f8eac152330 2023-10-06T05:29:10.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.098+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400fc30 con 0x7f8eac152330 2023-10-06T05:29:10.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.098+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac0aae90 con 0x7f8eac152330 2023-10-06T05:29:10.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.099+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac15bb10 con 0x7f8eac152330 2023-10-06T05:29:10.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.100+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ea400fd90 con 0x7f8eac152330 2023-10-06T05:29:10.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.100+0000 7f8ea1ffb700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e90066f30 0x7f8e900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.101+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8ea4084d10 con 0x7f8eac152330 2023-10-06T05:29:10.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.101+0000 7f8eb1d9b700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e90066f30 0x7f8e900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.102+0000 7f8eb1d9b700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e90066f30 0x7f8e900693e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001010 tx=0x7f8ea8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:10.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.105+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ea4052000 con 0x7f8eac152330 2023-10-06T05:29:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.310+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8eac04c380 con 0x7f8eac152330 2023-10-06T05:29:10.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.314+0000 7f8ea1ffb700 1 -- 172.21.15.33:0/4244195026 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ea4055620 con 0x7f8eac152330 2023-10-06T05:29:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e90066f30 msgr2=0x7f8e900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e90066f30 0x7f8e900693e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001010 tx=0x7f8ea8015000 comp rx=0 tx=0).stop 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 msgr2=0x7f8eac0aa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 0x7f8eac0aa950 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40094d0 tx=0x7f8ea400b040 comp rx=0 tx=0).stop 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 shutdown_connections 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e90066f30 0x7f8e900693e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac152330 0x7f8eac0aa950 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e98003a60 0x7f8eac15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 --2- 172.21.15.33:0/4244195026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eac153040 0x7f8eac0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 >> 172.21.15.33:0/4244195026 conn(0x7f8eac06d030 msgr2=0x7f8eac0c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 shutdown_connections 2023-10-06T05:29:10.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.317+0000 7f8eb2d9d700 1 -- 172.21.15.33:0/4244195026 wait complete. 2023-10-06T05:29:10.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_timeout' 2023-10-06T05:29:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.523+0000 7f5a8579d700 1 Processor -- start 2023-10-06T05:29:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.523+0000 7f5a8579d700 1 -- start start 2023-10-06T05:29:10.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a8579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80155140 0x7f5a80158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a8579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a800c8e40 con 0x7f5a800699c0 2023-10-06T05:29:10.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a8579d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a800c99e0 con 0x7f5a800adf50 2023-10-06T05:29:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a8579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800ae6c0 con 0x7f5a80155140 2023-10-06T05:29:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a84f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5a800699c0 0x7f5a800c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55466/0 (socket says 172.21.15.33:55466) 2023-10-06T05:29:10.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a84f9c700 1 -- 172.21.15.33:0/1079373193 learned_addr learned my addr 172.21.15.33:0/1079373193 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a7ffff700 1 --2- 172.21.15.33:0/1079373193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80155140 0x7f5a80158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a7effd700 1 -- 172.21.15.33:0/1079373193 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506229033 0 0) 0x7f5a800c8e40 con 0x7f5a800699c0 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.525+0000 7f5a7effd700 1 -- 172.21.15.33:0/1079373193 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a68003570 con 0x7f5a800699c0 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7ffff700 1 -- 172.21.15.33:0/1079373193 >> v1:172.21.15.167:6789/0 conn(0x7f5a800adf50 legacy=0x7f5a800c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7effd700 1 -- 172.21.15.33:0/1079373193 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368912712 0 0) 0x7f5a800c99e0 con 0x7f5a800adf50 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7ffff700 1 -- 172.21.15.33:0/1079373193 >> v1:172.21.15.33:6789/0 conn(0x7f5a800699c0 legacy=0x7f5a800c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7ffff700 1 -- 172.21.15.33:0/1079373193 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a800ae9a0 con 0x7f5a80155140 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7ffff700 1 --2- 172.21.15.33:0/1079373193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80155140 0x7f5a80158700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009910 tx=0x7f5a700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbb2cd92c0f5601 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7effd700 1 -- 172.21.15.33:0/1079373193 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000b680 con 0x7f5a80155140 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7effd700 1 -- 172.21.15.33:0/1079373193 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7000b7e0 con 0x7f5a80155140 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.526+0000 7f5a7effd700 1 -- 172.21.15.33:0/1079373193 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000bba0 con 0x7f5a80155140 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 -- 172.21.15.33:0/1079373193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80155140 msgr2=0x7f5a80158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1079373193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80155140 0x7f5a80158700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009910 tx=0x7f5a700091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 -- 172.21.15.33:0/1079373193 shutdown_connections 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1079373193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a80155140 0x7f5a80158700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 -- 172.21.15.33:0/1079373193 >> 172.21.15.33:0/1079373193 conn(0x7f5a8006d030 msgr2=0x7f5a8006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 -- 172.21.15.33:0/1079373193 shutdown_connections 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.527+0000 7f5a8579d700 1 -- 172.21.15.33:0/1079373193 wait complete. 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.528+0000 7f5a8579d700 1 Processor -- start 2023-10-06T05:29:10.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.529+0000 7f5a8579d700 1 -- start start 2023-10-06T05:29:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.529+0000 7f5a8579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800699c0 0x7f5a8013fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.529+0000 7f5a8579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800adf50 0x7f5a800b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a8579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 0x7f5a80151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a8579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800b17e0 con 0x7f5a80155140 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a8579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800b0bd0 con 0x7f5a800adf50 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a8579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800b0eb0 con 0x7f5a800699c0 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 0x7f5a80151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 0x7f5a80151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36340/0 (socket says 172.21.15.33:36340) 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 -- 172.21.15.33:0/1475261727 learned_addr learned my addr 172.21.15.33:0/1475261727 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a7ffff700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800699c0 0x7f5a8013fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a7f7fe700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800adf50 0x7f5a800b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 -- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800699c0 msgr2=0x7f5a8013fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800699c0 0x7f5a8013fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 -- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800adf50 msgr2=0x7f5a800b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800adf50 0x7f5a800b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a84f9c700 1 -- 172.21.15.33:0/1475261727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a70009420 con 0x7f5a80155140 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.530+0000 7f5a7f7fe700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800adf50 0x7f5a800b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.531+0000 7f5a84f9c700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 0x7f5a80151600 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f5a78001460 tx=0x7f5a78001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.531+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800a940 con 0x7f5a80155140 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.531+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7800aaa0 con 0x7f5a80155140 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.531+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800ae20 con 0x7f5a80155140 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.531+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a800b1190 con 0x7f5a80155140 2023-10-06T05:29:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.531+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a80140200 con 0x7f5a80155140 2023-10-06T05:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.532+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a800b1990 con 0x7f5a80155140 2023-10-06T05:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.533+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a78007300 con 0x7f5a80155140 2023-10-06T05:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.534+0000 7f5a67fff700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a6c066f30 0x7f5a6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.534+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5a780848d0 con 0x7f5a80155140 2023-10-06T05:29:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.534+0000 7f5a7ffff700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a6c066f30 0x7f5a6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.535+0000 7f5a7ffff700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a6c066f30 0x7f5a6c0693e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5a700093f0 tx=0x7f5a70017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:10.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.538+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a78051c40 con 0x7f5a80155140 2023-10-06T05:29:10.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:10.820+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_timeout"]} v 0) v1 -- 0x7f5a8004c380 con 0x7f5a80155140 2023-10-06T05:29:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.066+0000 7f5a67fff700 1 -- 172.21.15.33:0/1475261727 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_reconnect_timeout"]}]=0 v751) v1 ==== 154+0+0 (secure 0 0 0) 0x7f5a78055260 con 0x7f5a80155140 2023-10-06T05:29:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a6c066f30 msgr2=0x7f5a6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a6c066f30 0x7f5a6c0693e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5a700093f0 tx=0x7f5a70017000 comp rx=0 tx=0).stop 2023-10-06T05:29:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 msgr2=0x7f5a80151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 0x7f5a80151600 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f5a78001460 tx=0x7f5a78001ae0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 shutdown_connections 2023-10-06T05:29:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5a6c066f30 0x7f5a6c0693e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a80155140 0x7f5a80151600 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800699c0 0x7f5a8013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 --2- 172.21.15.33:0/1475261727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800adf50 0x7f5a800b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 >> 172.21.15.33:0/1475261727 conn(0x7f5a8006d030 msgr2=0x7f5a800c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 shutdown_connections 2023-10-06T05:29:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.069+0000 7f5a8579d700 1 -- 172.21.15.33:0/1475261727 wait complete. 2023-10-06T05:29:11.089 INFO:tasks.cephfs_test_runner:test_reconnect_timeout (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:29:11.089 INFO:tasks.cephfs_test_runner:Starting test: test_restart (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:29:11.089 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_restart' 2023-10-06T05:29:11.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.278+0000 7f377df9d700 1 Processor -- start 2023-10-06T05:29:11.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.278+0000 7f377df9d700 1 -- start start 2023-10-06T05:29:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377df9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f377813faa0 0x7f377813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:11.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377df9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f377806aad0 0x7f377806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377df9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37780c9cb0 con 0x7f37780c7940 2023-10-06T05:29:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377df9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780ca270 con 0x7f377806aad0 2023-10-06T05:29:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377df9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377806b840 con 0x7f377813faa0 2023-10-06T05:29:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377d79c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f377806aad0 0x7f377806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f3777fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f37780c7940 0x7f37780c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55470/0 (socket says 172.21.15.33:55470) 2023-10-06T05:29:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f3777fff700 1 -- 172.21.15.33:0/3628210491 learned_addr learned my addr 172.21.15.33:0/3628210491 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377cf9b700 1 --2- 172.21.15.33:0/3628210491 >> v2:172.21.15.167:3300/0 conn(0x7f377813faa0 0x7f377813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377d79c700 1 -- 172.21.15.33:0/3628210491 >> v2:172.21.15.167:3300/0 conn(0x7f377813faa0 msgr2=0x7f377813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377d79c700 1 --2- 172.21.15.33:0/3628210491 >> v2:172.21.15.167:3300/0 conn(0x7f377813faa0 0x7f377813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f377d79c700 1 -- 172.21.15.33:0/3628210491 >> v1:172.21.15.33:6789/0 conn(0x7f37780c7940 legacy=0x7f37780c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.280+0000 7f37777fe700 1 -- 172.21.15.33:0/3628210491 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874249886 0 0) 0x7f37780c9cb0 con 0x7f37780c7940 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.281+0000 7f377d79c700 1 -- 172.21.15.33:0/3628210491 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377806bb20 con 0x7f377806aad0 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.281+0000 7f377d79c700 1 --2- 172.21.15.33:0/3628210491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f377806af30 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f377806fe50 tx=0x7f377000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c4c0972398d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.281+0000 7f37777fe700 1 -- 172.21.15.33:0/3628210491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770010070 con 0x7f377806aad0 2023-10-06T05:29:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.281+0000 7f37777fe700 1 -- 172.21.15.33:0/3628210491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377000ddc0 con 0x7f377806aad0 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.281+0000 7f37777fe700 1 -- 172.21.15.33:0/3628210491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000b320 con 0x7f377806aad0 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.281+0000 7f377df9d700 1 -- 172.21.15.33:0/3628210491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 msgr2=0x7f377806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 --2- 172.21.15.33:0/3628210491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f377806af30 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f377806fe50 tx=0x7f377000b990 comp rx=0 tx=0).stop 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 -- 172.21.15.33:0/3628210491 shutdown_connections 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 --2- 172.21.15.33:0/3628210491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f377806af30 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 --2- 172.21.15.33:0/3628210491 >> v2:172.21.15.167:3300/0 conn(0x7f377813faa0 0x7f377813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 -- 172.21.15.33:0/3628210491 >> 172.21.15.33:0/3628210491 conn(0x7f377806d030 msgr2=0x7f377806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 -- 172.21.15.33:0/3628210491 shutdown_connections 2023-10-06T05:29:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.282+0000 7f377df9d700 1 -- 172.21.15.33:0/3628210491 wait complete. 2023-10-06T05:29:11.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.283+0000 7f377df9d700 1 Processor -- start 2023-10-06T05:29:11.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.283+0000 7f377df9d700 1 -- start start 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377df9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f3778155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377df9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37780c7940 0x7f37780c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377df9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f377813faa0 0x7f37780c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377df9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377813cd60 con 0x7f377806aad0 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377df9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780c3de0 con 0x7f37780c7940 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377df9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780c40c0 con 0x7f377813faa0 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377cf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f3778155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377cf9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f3778155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36358/0 (socket says 172.21.15.33:36358) 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.284+0000 7f377cf9b700 1 -- 172.21.15.33:0/3719003947 learned_addr learned my addr 172.21.15.33:0/3719003947 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377cf9b700 1 -- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f377813faa0 msgr2=0x7f37780c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377cf9b700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f377813faa0 0x7f37780c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f3777fff700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37780c7940 0x7f37780c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377cf9b700 1 -- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37780c7940 msgr2=0x7f37780c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377cf9b700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37780c7940 0x7f37780c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377cf9b700 1 -- 172.21.15.33:0/3719003947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377000bc30 con 0x7f377806aad0 2023-10-06T05:29:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f3777fff700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37780c7940 0x7f37780c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377cf9b700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f3778155f00 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f376c00d810 tx=0x7f376c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c0125f0 con 0x7f377806aad0 2023-10-06T05:29:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c012750 con 0x7f377806aad0 2023-10-06T05:29:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37780c43a0 con 0x7f377806aad0 2023-10-06T05:29:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.285+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c012b10 con 0x7f377806aad0 2023-10-06T05:29:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.286+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3778156440 con 0x7f377806aad0 2023-10-06T05:29:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.286+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377813d040 con 0x7f377806aad0 2023-10-06T05:29:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.287+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f376c01e460 con 0x7f377806aad0 2023-10-06T05:29:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.288+0000 7f3774ff9700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f375c066f30 0x7f375c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.288+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f376c084ac0 con 0x7f377806aad0 2023-10-06T05:29:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.289+0000 7f3777fff700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f375c066f30 0x7f375c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.289+0000 7f3777fff700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f375c066f30 0x7f375c0693e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f37780cb1a0 tx=0x7f3764009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:11.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.292+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f376c051db0 con 0x7f377806aad0 2023-10-06T05:29:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:11.579+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_restart"]} v 0) v1 -- 0x7f377813cb30 con 0x7f377806aad0 2023-10-06T05:29:12.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.074+0000 7f3774ff9700 1 -- 172.21.15.33:0/3719003947 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_restart"]}]=0 v752) v1 ==== 147+0+0 (secure 0 0 0) 0x7f376c0553d0 con 0x7f377806aad0 2023-10-06T05:29:12.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f375c066f30 msgr2=0x7f375c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f375c066f30 0x7f375c0693e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f37780cb1a0 tx=0x7f3764009040 comp rx=0 tx=0).stop 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 msgr2=0x7f3778155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f3778155f00 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f376c00d810 tx=0x7f376c008040 comp rx=0 tx=0).stop 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 shutdown_connections 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f375c066f30 0x7f375c0693e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f377813faa0 0x7f37780c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f377806aad0 0x7f3778155f00 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 --2- 172.21.15.33:0/3719003947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37780c7940 0x7f37780c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 >> 172.21.15.33:0/3719003947 conn(0x7f377806d030 msgr2=0x7f377806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:12.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 shutdown_connections 2023-10-06T05:29:12.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.077+0000 7f377df9d700 1 -- 172.21.15.33:0/3719003947 wait complete. 2023-10-06T05:29:12.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:29:12.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.291+0000 7f71fab9d700 1 Processor -- start 2023-10-06T05:29:12.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.291+0000 7f71fab9d700 1 -- start start 2023-10-06T05:29:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71fab9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f71f4156370 0x7f71f40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71fab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f40ad430 con 0x7f71f40cc7e0 2023-10-06T05:29:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71fab9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f40afe20 con 0x7f71f40b01f0 2023-10-06T05:29:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71fab9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f406bc70 con 0x7f71f4156370 2023-10-06T05:29:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f939a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f71f40cc7e0 0x7f71f40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55482/0 (socket says 172.21.15.33:55482) 2023-10-06T05:29:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f939a700 1 -- 172.21.15.33:0/4177659376 learned_addr learned my addr 172.21.15.33:0/4177659376 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f9b9b700 1 --2- 172.21.15.33:0/4177659376 >> v2:172.21.15.167:3300/0 conn(0x7f71f4156370 0x7f71f40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375332057 0 0) 0x7f71f40ad430 con 0x7f71f40cc7e0 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e0003570 con 0x7f71f40cc7e0 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500142721 0 0) 0x7f71f40afe20 con 0x7f71f40b01f0 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.293+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f40ad430 con 0x7f71f40b01f0 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f9b9b700 1 -- 172.21.15.33:0/4177659376 >> v1:172.21.15.167:6789/0 conn(0x7f71f40b01f0 legacy=0x7f71f40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f9b9b700 1 -- 172.21.15.33:0/4177659376 >> v1:172.21.15.33:6789/0 conn(0x7f71f40cc7e0 legacy=0x7f71f40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f9b9b700 1 -- 172.21.15.33:0/4177659376 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40cccc0 con 0x7f71f4156370 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f9b9b700 1 --2- 172.21.15.33:0/4177659376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f4156370 0x7f71f40af8e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f71f00098e0 tx=0x7f71f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c86e83147d1d851 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011630 con 0x7f71f4156370 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0011790 con 0x7f71f4156370 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.294+0000 7f71f8b99700 1 -- 172.21.15.33:0/4177659376 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011b30 con 0x7f71f4156370 2023-10-06T05:29:12.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 -- 172.21.15.33:0/4177659376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f4156370 msgr2=0x7f71f40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 --2- 172.21.15.33:0/4177659376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f4156370 0x7f71f40af8e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f71f00098e0 tx=0x7f71f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 -- 172.21.15.33:0/4177659376 shutdown_connections 2023-10-06T05:29:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 --2- 172.21.15.33:0/4177659376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f4156370 0x7f71f40af8e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 -- 172.21.15.33:0/4177659376 >> 172.21.15.33:0/4177659376 conn(0x7f71f406d030 msgr2=0x7f71f40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 -- 172.21.15.33:0/4177659376 shutdown_connections 2023-10-06T05:29:12.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.295+0000 7f71fab9d700 1 -- 172.21.15.33:0/4177659376 wait complete. 2023-10-06T05:29:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.296+0000 7f71fab9d700 1 Processor -- start 2023-10-06T05:29:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 -- start start 2023-10-06T05:29:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b01f0 0x7f71f4140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40cc7e0 0x7f71f4148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 0x7f71f4148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4158860 con 0x7f71f4156370 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4148d80 con 0x7f71f40cc7e0 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.297+0000 7f71fab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4147260 con 0x7f71f40b01f0 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 0x7f71f4148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 0x7f71f4148840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36374/0 (socket says 172.21.15.33:36374) 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 -- 172.21.15.33:0/1672329034 learned_addr learned my addr 172.21.15.33:0/1672329034 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71f939a700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40cc7e0 0x7f71f4148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71f9b9b700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b01f0 0x7f71f4140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 -- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b01f0 msgr2=0x7f71f4140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b01f0 0x7f71f4140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 -- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40cc7e0 msgr2=0x7f71f4148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40cc7e0 0x7f71f4148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 -- 172.21.15.33:0/1672329034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0009420 con 0x7f71f4156370 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.298+0000 7f71fa39c700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 0x7f71f4148840 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00c860 tx=0x7f71ec002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.299+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec012630 con 0x7f71f4156370 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.299+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec0127b0 con 0x7f71f4156370 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.299+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec012b70 con 0x7f71f4156370 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.299+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f4147540 con 0x7f71f4156370 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.299+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f4147b00 con 0x7f71f4156370 2023-10-06T05:29:12.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.300+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f404c380 con 0x7f71f4156370 2023-10-06T05:29:12.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.301+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71ec01b400 con 0x7f71f4156370 2023-10-06T05:29:12.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.301+0000 7f71e9ffb700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f71d8066f60 0x7f71d8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.302+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f71ec084390 con 0x7f71f4156370 2023-10-06T05:29:12.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.302+0000 7f71f9b9b700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f71d8066f60 0x7f71d8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.303+0000 7f71f9b9b700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f71d8066f60 0x7f71d8069410 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f71f00093f0 tx=0x7f71f0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:12.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.306+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71ec0517d0 con 0x7f71f4156370 2023-10-06T05:29:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.509+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f71f406f310 con 0x7f71f4156370 2023-10-06T05:29:12.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.514+0000 7f71e9ffb700 1 -- 172.21.15.33:0/1672329034 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f71ec054df0 con 0x7f71f4156370 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f71d8066f60 msgr2=0x7f71d8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f71d8066f60 0x7f71d8069410 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f71f00093f0 tx=0x7f71f0017000 comp rx=0 tx=0).stop 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 msgr2=0x7f71f4148840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 0x7f71f4148840 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00c860 tx=0x7f71ec002700 comp rx=0 tx=0).stop 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 shutdown_connections 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f71d8066f60 0x7f71d8069410 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f4156370 0x7f71f4148840 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f40b01f0 0x7f71f4140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 --2- 172.21.15.33:0/1672329034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f40cc7e0 0x7f71f4148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 >> 172.21.15.33:0/1672329034 conn(0x7f71f406d030 msgr2=0x7f71f40ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:12.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 shutdown_connections 2023-10-06T05:29:12.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.516+0000 7f71fab9d700 1 -- 172.21.15.33:0/1672329034 wait complete. 2023-10-06T05:29:12.535 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:29:12.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:12.615 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:12.704 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:29:12.705 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:29:12.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.890+0000 7fe669b9d700 1 Processor -- start 2023-10-06T05:29:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.890+0000 7fe669b9d700 1 -- start start 2023-10-06T05:29:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe669b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe664159530 0x7fe66406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe669b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe66406df70 0x7fe6640afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe669b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66406e3e0 con 0x7fe6640b0310 2023-10-06T05:29:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe669b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6640b1e00 con 0x7fe664159530 2023-10-06T05:29:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe669b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66406f6b0 con 0x7fe66406df70 2023-10-06T05:29:12.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe668b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe664159530 0x7fe66406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe668b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe664159530 0x7fe66406da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36380/0 (socket says 172.21.15.33:36380) 2023-10-06T05:29:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.891+0000 7fe668b9b700 1 -- 172.21.15.33:0/3293369160 learned_addr learned my addr 172.21.15.33:0/3293369160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:12.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe663fff700 1 --2- 172.21.15.33:0/3293369160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe66406df70 0x7fe6640afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe668b9b700 1 -- 172.21.15.33:0/3293369160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe66406df70 msgr2=0x7fe6640afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe668b9b700 1 --2- 172.21.15.33:0/3293369160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe66406df70 0x7fe6640afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe668b9b700 1 -- 172.21.15.33:0/3293369160 >> v1:172.21.15.167:6789/0 conn(0x7fe6640b0310 legacy=0x7fe664159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe668b9b700 1 -- 172.21.15.33:0/3293369160 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66406f220 con 0x7fe664159530 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe6637fe700 1 -- 172.21.15.33:0/3293369160 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667111009 0 0) 0x7fe66406e3e0 con 0x7fe6640b0310 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe668b9b700 1 --2- 172.21.15.33:0/3293369160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe66406da30 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fe6580099b0 tx=0x7fe658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ad24aa55d350c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe6637fe700 1 -- 172.21.15.33:0/3293369160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658009ea0 con 0x7fe664159530 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe6637fe700 1 -- 172.21.15.33:0/3293369160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65800b860 con 0x7fe664159530 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.892+0000 7fe6637fe700 1 -- 172.21.15.33:0/3293369160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65800bc00 con 0x7fe664159530 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 -- 172.21.15.33:0/3293369160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 msgr2=0x7fe66406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 --2- 172.21.15.33:0/3293369160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe66406da30 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fe6580099b0 tx=0x7fe658009250 comp rx=0 tx=0).stop 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 -- 172.21.15.33:0/3293369160 shutdown_connections 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 --2- 172.21.15.33:0/3293369160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe66406da30 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 --2- 172.21.15.33:0/3293369160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe66406df70 0x7fe6640afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 -- 172.21.15.33:0/3293369160 >> 172.21.15.33:0/3293369160 conn(0x7fe66406c560 msgr2=0x7fe66406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 -- 172.21.15.33:0/3293369160 shutdown_connections 2023-10-06T05:29:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.893+0000 7fe669b9d700 1 -- 172.21.15.33:0/3293369160 wait complete. 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.895+0000 7fe669b9d700 1 Processor -- start 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.895+0000 7fe669b9d700 1 -- start start 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe669b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 0x7fe6641409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe669b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640b0310 0x7fe664141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe669b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe6640ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe669b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640cafb0 con 0x7fe664159530 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe669b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640ca3a0 con 0x7fe66406df70 2023-10-06T05:29:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe669b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640ca680 con 0x7fe6640b0310 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 0x7fe6641409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe66939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe6640ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 0x7fe6641409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:35534/0 (socket says 172.21.15.33:35534) 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 -- 172.21.15.33:0/2537728829 learned_addr learned my addr 172.21.15.33:0/2537728829 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe663fff700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640b0310 0x7fe664141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 -- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640b0310 msgr2=0x7fe664141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640b0310 0x7fe664141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 -- 172.21.15.33:0/2537728829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 msgr2=0x7fe6640ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe6640ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe668b9b700 1 -- 172.21.15.33:0/2537728829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580094f0 con 0x7fe66406df70 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.896+0000 7fe66939c700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe6640ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.897+0000 7fe668b9b700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 0x7fe6641409f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe66406b870 tx=0x7fe658017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.897+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658000d10 con 0x7fe66406df70 2023-10-06T05:29:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.897+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65800bbe0 con 0x7fe66406df70 2023-10-06T05:29:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.897+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658011690 con 0x7fe66406df70 2023-10-06T05:29:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.897+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6640ca960 con 0x7fe66406df70 2023-10-06T05:29:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.897+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe664140f30 con 0x7fe66406df70 2023-10-06T05:29:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.898+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66404c380 con 0x7fe66406df70 2023-10-06T05:29:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.899+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe658011ac0 con 0x7fe66406df70 2023-10-06T05:29:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.900+0000 7fe660ff9700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe648066f30 0x7fe6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.900+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe658085fb0 con 0x7fe66406df70 2023-10-06T05:29:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.900+0000 7fe663fff700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe648066f30 0x7fe6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.901+0000 7fe663fff700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe648066f30 0x7fe6480693e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe66404bc50 tx=0x7fe6500091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:12.905+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe658053320 con 0x7fe66406df70 2023-10-06T05:29:13.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.153+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe66406dd50 con 0x7fe66406df70 2023-10-06T05:29:13.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.154+0000 7fe660ff9700 1 -- 172.21.15.33:0/2537728829 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+3541 (secure 0 0 0) 0x7fe658015df0 con 0x7fe66406df70 2023-10-06T05:29:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:13.158 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":156,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3729098419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3729098419},{"type":"v1","addr":"172.21.15.167:6836","nonce":3729098419}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":143},{"gid":7790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1349195474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1349195474},{"type":"v1","addr":"172.21.15.167:6837","nonce":1349195474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":150},{"gid":8953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1394539499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1394539499},{"type":"v1","addr":"172.21.15.33:6834","nonce":1394539499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142},{"gid":10353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2336217320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2336217320},{"type":"v1","addr":"172.21.15.33:6835","nonce":2336217320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[]} 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe648066f30 msgr2=0x7fe6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe648066f30 0x7fe6480693e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe66404bc50 tx=0x7fe6500091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 msgr2=0x7fe6641409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 0x7fe6641409f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe66406b870 tx=0x7fe658017000 comp rx=0 tx=0).stop 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 shutdown_connections 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe648066f30 0x7fe6480693e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe664159530 0x7fe6640ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe66406df70 0x7fe6641409f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 --2- 172.21.15.33:0/2537728829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6640b0310 0x7fe664141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 >> 172.21.15.33:0/2537728829 conn(0x7fe66406c560 msgr2=0x7fe66415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:13.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 shutdown_connections 2023-10-06T05:29:13.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.157+0000 7fe669b9d700 1 -- 172.21.15.33:0/2537728829 wait complete. 2023-10-06T05:29:13.160 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 156 2023-10-06T05:29:13.177 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:29:13.177 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:29:13.177 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:29:13.178 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:13.178 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:29:13.178 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:29:13.178 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:29:13.178 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:13.179 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:29:13.179 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:29:13.179 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:29:13.179 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:13.179 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:29:13.179 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:29:13.180 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:29:13.180 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:13.190 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:29:13.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:29:13.192 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:29:13.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:29:13.195 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:29:13.196 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:29:13.196 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:29:13.197 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:29:13.235 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:29:13.235 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:13.237 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:29:13.238 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:13.240 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:29:13.240 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:13.243 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:29:13.243 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:13.243 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:29:13.374 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:29:13.375 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:29:13.378 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:29:13.378 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:29:13.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.436+0000 7f32a139d700 1 Processor -- start 2023-10-06T05:29:13.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.436+0000 7f32a139d700 1 -- start start 2023-10-06T05:29:13.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c0c7070 0x7f329c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:13.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a139d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f329c155bc0 0x7f329c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:13.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329c06a410 con 0x7f329c1557f0 2023-10-06T05:29:13.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a139d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c0cccd0 con 0x7f329c155bc0 2023-10-06T05:29:13.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0c7500 con 0x7f329c0c7070 2023-10-06T05:29:13.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f329bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f329c1557f0 0x7f329c0c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55510/0 (socket says 172.21.15.33:55510) 2023-10-06T05:29:13.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f329bfff700 1 -- 172.21.15.33:0/2815510412 learned_addr learned my addr 172.21.15.33:0/2815510412 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:13.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a0b9c700 1 --2- 172.21.15.33:0/2815510412 >> v2:172.21.15.167:3300/0 conn(0x7f329c155bc0 0x7f329c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:13.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f329b7fe700 1 --2- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c0c7070 0x7f329c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f329affd700 1 -- 172.21.15.33:0/2815510412 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668294612 0 0) 0x7f329c06a410 con 0x7f329c1557f0 2023-10-06T05:29:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f329affd700 1 -- 172.21.15.33:0/2815510412 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3284003570 con 0x7f329c1557f0 2023-10-06T05:29:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a0b9c700 1 -- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c0c7070 msgr2=0x7f329c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a0b9c700 1 --2- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c0c7070 0x7f329c0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a0b9c700 1 -- 172.21.15.33:0/2815510412 >> v1:172.21.15.33:6789/0 conn(0x7f329c1557f0 legacy=0x7f329c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.438+0000 7f32a0b9c700 1 -- 172.21.15.33:0/2815510412 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c0c77e0 con 0x7f329c155bc0 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.439+0000 7f32a0b9c700 1 --2- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c155bc0 0x7f329c0c9050 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3294009e60 tx=0x7f3294009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cc0e1fa03a479e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.439+0000 7f329affd700 1 -- 172.21.15.33:0/2815510412 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400bab0 con 0x7f329c155bc0 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.439+0000 7f329affd700 1 -- 172.21.15.33:0/2815510412 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400bc10 con 0x7f329c155bc0 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.439+0000 7f329affd700 1 -- 172.21.15.33:0/2815510412 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011460 con 0x7f329c155bc0 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.439+0000 7f32a139d700 1 -- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c155bc0 msgr2=0x7f329c0c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.439+0000 7f32a139d700 1 --2- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c155bc0 0x7f329c0c9050 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3294009e60 tx=0x7f3294009780 comp rx=0 tx=0).stop 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.440+0000 7f32a139d700 1 -- 172.21.15.33:0/2815510412 shutdown_connections 2023-10-06T05:29:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.440+0000 7f32a139d700 1 --2- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c155bc0 0x7f329c0c9050 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.440+0000 7f32a139d700 1 --2- 172.21.15.33:0/2815510412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c0c7070 0x7f329c0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.440+0000 7f32a139d700 1 -- 172.21.15.33:0/2815510412 >> 172.21.15.33:0/2815510412 conn(0x7f329c06c560 msgr2=0x7f329c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.440+0000 7f32a139d700 1 -- 172.21.15.33:0/2815510412 shutdown_connections 2023-10-06T05:29:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.440+0000 7f32a139d700 1 -- 172.21.15.33:0/2815510412 wait complete. 2023-10-06T05:29:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.441+0000 7f32a139d700 1 Processor -- start 2023-10-06T05:29:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.441+0000 7f32a139d700 1 -- start start 2023-10-06T05:29:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.442+0000 7f32a139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c0c7070 0x7f329c165bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.442+0000 7f32a139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 0x7f329c0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.442+0000 7f32a139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c155bc0 0x7f329c0c1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.442+0000 7f32a139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0b1310 con 0x7f329c1557f0 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.442+0000 7f32a139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0af880 con 0x7f329c0c7070 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.442+0000 7f32a139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0b1fc0 con 0x7f329c155bc0 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 0x7f329c0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 0x7f329c0c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36408/0 (socket says 172.21.15.33:36408) 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 -- 172.21.15.33:0/1580364968 learned_addr learned my addr 172.21.15.33:0/1580364968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f32a0b9c700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c155bc0 0x7f329c0c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329bfff700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c0c7070 0x7f329c165bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 -- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c155bc0 msgr2=0x7f329c0c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c155bc0 0x7f329c0c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 -- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c0c7070 msgr2=0x7f329c165bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c0c7070 0x7f329c165bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 -- 172.21.15.33:0/1580364968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294009a20 con 0x7f329c1557f0 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329bfff700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c0c7070 0x7f329c165bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.443+0000 7f329b7fe700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 0x7f329c0c4ac0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f329000ecb0 tx=0x7f32900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.444+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329000c920 con 0x7f329c1557f0 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.444+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329000ca80 con 0x7f329c1557f0 2023-10-06T05:29:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.444+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c166100 con 0x7f329c1557f0 2023-10-06T05:29:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.444+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329000ce20 con 0x7f329c1557f0 2023-10-06T05:29:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.444+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c0af1c0 con 0x7f329c1557f0 2023-10-06T05:29:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.445+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c0af9c0 con 0x7f329c1557f0 2023-10-06T05:29:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.446+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3290013680 con 0x7f329c1557f0 2023-10-06T05:29:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.446+0000 7f3283fff700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3288066fb0 0x7f3288069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.446+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3290084790 con 0x7f329c1557f0 2023-10-06T05:29:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.447+0000 7f329bfff700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3288066fb0 0x7f3288069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.447+0000 7f329bfff700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3288066fb0 0x7f3288069460 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f328c0058d0 tx=0x7f328c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:13.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.450+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f329c0af9c0 con 0x7f329c1557f0 2023-10-06T05:29:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:13.666+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f329c0af9c0 con 0x7f329c1557f0 2023-10-06T05:29:14.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.874+0000 7f3283fff700 1 -- 172.21.15.33:0/1580364968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v206) v1 ==== 97+0+0 (secure 0 0 0) 0x7f329c0af9c0 con 0x7f329c1557f0 2023-10-06T05:29:14.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3288066fb0 msgr2=0x7f3288069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:14.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3288066fb0 0x7f3288069460 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f328c0058d0 tx=0x7f328c009040 comp rx=0 tx=0).stop 2023-10-06T05:29:14.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 msgr2=0x7f329c0c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:14.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 0x7f329c0c4ac0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f329000ecb0 tx=0x7f32900091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:14.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 shutdown_connections 2023-10-06T05:29:14.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3288066fb0 0x7f3288069460 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f329c155bc0 0x7f329c0c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f329c0c7070 0x7f329c165bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 --2- 172.21.15.33:0/1580364968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c1557f0 0x7f329c0c4ac0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.877+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 >> 172.21.15.33:0/1580364968 conn(0x7f329c06c560 msgr2=0x7f329c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.878+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 shutdown_connections 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:14.878+0000 7f32a139d700 1 -- 172.21.15.33:0/1580364968 wait complete. 2023-10-06T05:29:14.881 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:29:14.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:29:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.090+0000 7f9ebc99d700 1 Processor -- start 2023-10-06T05:29:15.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.090+0000 7f9ebc99d700 1 -- start start 2023-10-06T05:29:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9ebc99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9eb8155af0 0x7f9eb806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9ebc99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9eb806a6a0 0x7f9eb80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9ebc99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb806ab60 con 0x7f9eb80c8b50 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9ebc99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb806b7e0 con 0x7f9eb8155af0 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9ebc99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb806ae40 con 0x7f9eb806a6a0 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9eb77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9eb8155af0 0x7f9eb806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9eb77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9eb8155af0 0x7f9eb806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36410/0 (socket says 172.21.15.33:36410) 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.091+0000 7f9eb77fe700 1 -- 172.21.15.33:0/1963917299 learned_addr learned my addr 172.21.15.33:0/1963917299 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb6ffd700 1 --2- 172.21.15.33:0/1963917299 >> v2:172.21.15.167:3300/0 conn(0x7f9eb806a6a0 0x7f9eb80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb77fe700 1 -- 172.21.15.33:0/1963917299 >> v2:172.21.15.167:3300/0 conn(0x7f9eb806a6a0 msgr2=0x7f9eb80c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb77fe700 1 --2- 172.21.15.33:0/1963917299 >> v2:172.21.15.167:3300/0 conn(0x7f9eb806a6a0 0x7f9eb80c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb77fe700 1 -- 172.21.15.33:0/1963917299 >> v1:172.21.15.33:6789/0 conn(0x7f9eb80c8b50 legacy=0x7f9eb81598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb77fe700 1 -- 172.21.15.33:0/1963917299 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb806b120 con 0x7f9eb8155af0 2023-10-06T05:29:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb67fc700 1 -- 172.21.15.33:0/1963917299 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965624620 0 0) 0x7f9eb806ab60 con 0x7f9eb80c8b50 2023-10-06T05:29:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb77fe700 1 --2- 172.21.15.33:0/1963917299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb8155af0 0x7f9eb806a160 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f9eb806f250 tx=0x7f9ea800a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c783dad5a51bdf5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb67fc700 1 -- 172.21.15.33:0/1963917299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800ca40 con 0x7f9eb8155af0 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb67fc700 1 -- 172.21.15.33:0/1963917299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea800cba0 con 0x7f9eb8155af0 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.092+0000 7f9eb67fc700 1 -- 172.21.15.33:0/1963917299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8018400 con 0x7f9eb8155af0 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 -- 172.21.15.33:0/1963917299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb8155af0 msgr2=0x7f9eb806a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/1963917299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb8155af0 0x7f9eb806a160 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f9eb806f250 tx=0x7f9ea800a3f0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 -- 172.21.15.33:0/1963917299 shutdown_connections 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/1963917299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb8155af0 0x7f9eb806a160 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/1963917299 >> v2:172.21.15.167:3300/0 conn(0x7f9eb806a6a0 0x7f9eb80c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 -- 172.21.15.33:0/1963917299 >> 172.21.15.33:0/1963917299 conn(0x7f9eb806d030 msgr2=0x7f9eb806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:15.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 -- 172.21.15.33:0/1963917299 shutdown_connections 2023-10-06T05:29:15.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.093+0000 7f9ebc99d700 1 -- 172.21.15.33:0/1963917299 wait complete. 2023-10-06T05:29:15.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.095+0000 7f9ebc99d700 1 Processor -- start 2023-10-06T05:29:15.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.095+0000 7f9ebc99d700 1 -- start start 2023-10-06T05:29:15.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9ebc99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb806a6a0 0x7f9eb80aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9ebc99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 0x7f9eb8146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9ebc99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eb8155af0 0x7f9eb80aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9ebc99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80a9d70 con 0x7f9eb80c8b50 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9ebc99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80aa050 con 0x7f9eb806a6a0 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9ebc99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80aa330 con 0x7f9eb8155af0 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 0x7f9eb8146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 0x7f9eb8146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36414/0 (socket says 172.21.15.33:36414) 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 -- 172.21.15.33:0/2782918514 learned_addr learned my addr 172.21.15.33:0/2782918514 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb77fe700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb806a6a0 0x7f9eb80aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 -- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eb8155af0 msgr2=0x7f9eb80aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb7fff700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eb8155af0 0x7f9eb80aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eb8155af0 0x7f9eb80aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 -- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb806a6a0 msgr2=0x7f9eb80aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb806a6a0 0x7f9eb80aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb6ffd700 1 -- 172.21.15.33:0/2782918514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea800a690 con 0x7f9eb80c8b50 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb7fff700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eb8155af0 0x7f9eb80aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.096+0000 7f9eb77fe700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb806a6a0 0x7f9eb80aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.097+0000 7f9eb6ffd700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 0x7f9eb8146e30 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00d860 tx=0x7f9eac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.097+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0125f0 con 0x7f9eb80c8b50 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.097+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac012750 con 0x7f9eb80c8b50 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.097+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac012ad0 con 0x7f9eb80c8b50 2023-10-06T05:29:15.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.097+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb80aa610 con 0x7f9eb80c8b50 2023-10-06T05:29:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.097+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb8147370 con 0x7f9eb80c8b50 2023-10-06T05:29:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.098+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb80aab30 con 0x7f9eb80c8b50 2023-10-06T05:29:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.099+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9eac01e460 con 0x7f9eb80c8b50 2023-10-06T05:29:15.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.100+0000 7f9e9f7fe700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ea4066f30 0x7f9ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.100+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9eac084a60 con 0x7f9eb80c8b50 2023-10-06T05:29:15.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.100+0000 7f9eb77fe700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ea4066f30 0x7f9ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.101+0000 7f9eb77fe700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ea4066f30 0x7f9ea40693e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9eb806f250 tx=0x7f9ea8018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:15.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.104+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9eb80aab30 con 0x7f9eb80c8b50 2023-10-06T05:29:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.384+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9eb80aab30 con 0x7f9eb80c8b50 2023-10-06T05:29:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.384+0000 7f9e9f7fe700 1 -- 172.21.15.33:0/2782918514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9eb80aab30 con 0x7f9eb80c8b50 2023-10-06T05:29:15.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:15.386 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:29:15.386 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:29:15.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:29:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:29:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:29:15.390 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.391 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:29:15.394 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.398 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:29:15.400 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:29:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:29:15.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:15.403 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.403 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:15.403 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:29:15.403 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:29:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ea4066f30 msgr2=0x7f9ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ea4066f30 0x7f9ea40693e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9eb806f250 tx=0x7f9ea8018000 comp rx=0 tx=0).stop 2023-10-06T05:29:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 msgr2=0x7f9eb8146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 0x7f9eb8146e30 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00d860 tx=0x7f9eac008040 comp rx=0 tx=0).stop 2023-10-06T05:29:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 shutdown_connections 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9ea4066f30 0x7f9ea40693e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eb8155af0 0x7f9eb80aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb806a6a0 0x7f9eb80aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 --2- 172.21.15.33:0/2782918514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9eb80c8b50 0x7f9eb8146e30 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 >> 172.21.15.33:0/2782918514 conn(0x7f9eb806d030 msgr2=0x7f9eb8154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 shutdown_connections 2023-10-06T05:29:15.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.387+0000 7f9ebc99d700 1 -- 172.21.15.33:0/2782918514 wait complete. 2023-10-06T05:29:15.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:15.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.591+0000 7f5950fff700 1 Processor -- start 2023-10-06T05:29:15.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.592+0000 7f5950fff700 1 -- start start 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f5950fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f594c156150 0x7f594c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f5950fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c06e850 0x7f594c0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f5950fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594c150670 con 0x7f594c0cb760 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f5950fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594c0c9c30 con 0x7f594c156150 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f5950fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c06b5e0 con 0x7f594c06e850 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f594a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f594c0cb760 0x7f594c06a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55530/0 (socket says 172.21.15.33:55530) 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f594a59c700 1 -- 172.21.15.33:0/4014616383 learned_addr learned my addr 172.21.15.33:0/4014616383 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f5949d9b700 1 --2- 172.21.15.33:0/4014616383 >> v2:172.21.15.33:3300/0 conn(0x7f594c156150 0x7f594c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.593+0000 7f594ad9d700 1 --2- 172.21.15.33:0/4014616383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c06e850 0x7f594c0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f594959a700 1 -- 172.21.15.33:0/4014616383 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983191128 0 0) 0x7f594c150670 con 0x7f594c0cb760 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f594959a700 1 -- 172.21.15.33:0/4014616383 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5934003570 con 0x7f594c0cb760 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f5949d9b700 1 -- 172.21.15.33:0/4014616383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c06e850 msgr2=0x7f594c0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f5949d9b700 1 --2- 172.21.15.33:0/4014616383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c06e850 0x7f594c0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f5949d9b700 1 -- 172.21.15.33:0/4014616383 >> v1:172.21.15.33:6789/0 conn(0x7f594c0cb760 legacy=0x7f594c06a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f5949d9b700 1 -- 172.21.15.33:0/4014616383 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c06ec60 con 0x7f594c156150 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f5949d9b700 1 --2- 172.21.15.33:0/4014616383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c06e310 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f594000aa90 tx=0x7f594000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbd80bf91aade41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f594959a700 1 -- 172.21.15.33:0/4014616383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000c970 con 0x7f594c156150 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f594959a700 1 -- 172.21.15.33:0/4014616383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594000cad0 con 0x7f594c156150 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.594+0000 7f594959a700 1 -- 172.21.15.33:0/4014616383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000ce50 con 0x7f594c156150 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 -- 172.21.15.33:0/4014616383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 msgr2=0x7f594c06e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 --2- 172.21.15.33:0/4014616383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c06e310 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f594000aa90 tx=0x7f594000a330 comp rx=0 tx=0).stop 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 -- 172.21.15.33:0/4014616383 shutdown_connections 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 --2- 172.21.15.33:0/4014616383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c06e850 0x7f594c0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 --2- 172.21.15.33:0/4014616383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c06e310 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 -- 172.21.15.33:0/4014616383 >> 172.21.15.33:0/4014616383 conn(0x7f594c0675f0 msgr2=0x7f594c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 -- 172.21.15.33:0/4014616383 shutdown_connections 2023-10-06T05:29:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.595+0000 7f5950fff700 1 -- 172.21.15.33:0/4014616383 wait complete. 2023-10-06T05:29:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.596+0000 7f5950fff700 1 Processor -- start 2023-10-06T05:29:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.597+0000 7f5950fff700 1 -- start start 2023-10-06T05:29:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.597+0000 7f5950fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f594c06e850 0x7f594c156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.597+0000 7f5950fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c0cb760 0x7f594c146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.597+0000 7f5950fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f5950fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c158260 con 0x7f594c156150 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f5950fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c157650 con 0x7f594c06e850 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f5950fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c157930 con 0x7f594c0cb760 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36438/0 (socket says 172.21.15.33:36438) 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 -- 172.21.15.33:0/2121068334 learned_addr learned my addr 172.21.15.33:0/2121068334 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f5949d9b700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c0cb760 0x7f594c146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594a59c700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f594c06e850 0x7f594c156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 -- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c0cb760 msgr2=0x7f594c146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c0cb760 0x7f594c146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 -- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f594c06e850 msgr2=0x7f594c156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f594c06e850 0x7f594c156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 -- 172.21.15.33:0/2121068334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594000a5d0 con 0x7f594c156150 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.598+0000 7f594ad9d700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c0afa80 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f5944009e90 tx=0x7f594400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.599+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594400cc20 con 0x7f594c156150 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.599+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594400f040 con 0x7f594c156150 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.599+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944014480 con 0x7f594c156150 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.599+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c157c10 con 0x7f594c156150 2023-10-06T05:29:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.599+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c1471d0 con 0x7f594c156150 2023-10-06T05:29:15.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.600+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c158410 con 0x7f594c156150 2023-10-06T05:29:15.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.601+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5944014830 con 0x7f594c156150 2023-10-06T05:29:15.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.602+0000 7f593a7fc700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f592c066f30 0x7f592c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:15.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.602+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f59440850e0 con 0x7f594c156150 2023-10-06T05:29:15.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.602+0000 7f594a59c700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f592c066f30 0x7f592c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:15.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.603+0000 7f594a59c700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f592c066f30 0x7f592c0693e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f594c156d30 tx=0x7f593c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:15.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.606+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5944055f30 con 0x7f594c156150 2023-10-06T05:29:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.810+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f594c063ff0 con 0x7f594c156150 2023-10-06T05:29:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.811+0000 7f593a7fc700 1 -- 172.21.15.33:0/2121068334 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f5944055ac0 con 0x7f594c156150 2023-10-06T05:29:15.813 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:15.813 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":206,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:14.868230+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.813+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f592c066f30 msgr2=0x7f592c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.813+0000 7f5950fff700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f592c066f30 0x7f592c0693e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f594c156d30 tx=0x7f593c009040 comp rx=0 tx=0).stop 2023-10-06T05:29:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.813+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 msgr2=0x7f594c0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.813+0000 7f5950fff700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c0afa80 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f5944009e90 tx=0x7f594400b040 comp rx=0 tx=0).stop 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 shutdown_connections 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f592c066f30 0x7f592c0693e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f594c156150 0x7f594c0afa80 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f594c06e850 0x7f594c156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 --2- 172.21.15.33:0/2121068334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f594c0cb760 0x7f594c146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 >> 172.21.15.33:0/2121068334 conn(0x7f594c0675f0 msgr2=0x7f594c0c7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 shutdown_connections 2023-10-06T05:29:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:15.814+0000 7f5950fff700 1 -- 172.21.15.33:0/2121068334 wait complete. 2023-10-06T05:29:15.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:29:15.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:29:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.024+0000 7f564dd9d700 1 Processor -- start 2023-10-06T05:29:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.024+0000 7f564dd9d700 1 -- start start 2023-10-06T05:29:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564dd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56480c9490 0x7f5648155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564dd9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f56480afb20 0x7f56480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564dd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56480b06b0 con 0x7f5648155a50 2023-10-06T05:29:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564dd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56480c99e0 con 0x7f56480c9490 2023-10-06T05:29:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564dd9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56480c9cc0 con 0x7f56480afb20 2023-10-06T05:29:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f5647fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56480c9490 0x7f5648155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564cd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5648155a50 0x7f56480af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55538/0 (socket says 172.21.15.33:55538) 2023-10-06T05:29:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564cd9b700 1 -- 172.21.15.33:0/1205811074 learned_addr learned my addr 172.21.15.33:0/1205811074 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564d59c700 1 --2- 172.21.15.33:0/1205811074 >> v2:172.21.15.167:3300/0 conn(0x7f56480afb20 0x7f56480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f56477fe700 1 -- 172.21.15.33:0/1205811074 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609563113 0 0) 0x7f56480b06b0 con 0x7f5648155a50 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f56477fe700 1 -- 172.21.15.33:0/1205811074 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5638003570 con 0x7f5648155a50 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f5647fff700 1 -- 172.21.15.33:0/1205811074 >> v2:172.21.15.167:3300/0 conn(0x7f56480afb20 msgr2=0x7f56480caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f5647fff700 1 --2- 172.21.15.33:0/1205811074 >> v2:172.21.15.167:3300/0 conn(0x7f56480afb20 0x7f56480caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f5647fff700 1 -- 172.21.15.33:0/1205811074 >> v1:172.21.15.33:6789/0 conn(0x7f5648155a50 legacy=0x7f56480af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f5647fff700 1 -- 172.21.15.33:0/1205811074 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480c9fa0 con 0x7f56480c9490 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.026+0000 7f564d59c700 1 --2- 172.21.15.33:0/1205811074 >> v2:172.21.15.167:3300/0 conn(0x7f56480afb20 0x7f56480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.027+0000 7f5647fff700 1 --2- 172.21.15.33:0/1205811074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648155e20 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f564806f1f0 tx=0x7f563400a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5bbcbf5a1dff5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.027+0000 7f56477fe700 1 -- 172.21.15.33:0/1205811074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400c970 con 0x7f56480c9490 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.027+0000 7f56477fe700 1 -- 172.21.15.33:0/1205811074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563400cad0 con 0x7f56480c9490 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.027+0000 7f56477fe700 1 -- 172.21.15.33:0/1205811074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400ce70 con 0x7f56480c9490 2023-10-06T05:29:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.027+0000 7f564dd9d700 1 -- 172.21.15.33:0/1205811074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 msgr2=0x7f5648155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.027+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1205811074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648155e20 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f564806f1f0 tx=0x7f563400a330 comp rx=0 tx=0).stop 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.028+0000 7f564dd9d700 1 -- 172.21.15.33:0/1205811074 shutdown_connections 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.028+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1205811074 >> v2:172.21.15.167:3300/0 conn(0x7f56480afb20 0x7f56480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.028+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1205811074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648155e20 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.028+0000 7f564dd9d700 1 -- 172.21.15.33:0/1205811074 >> 172.21.15.33:0/1205811074 conn(0x7f564806d030 msgr2=0x7f56480cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.028+0000 7f564dd9d700 1 -- 172.21.15.33:0/1205811074 shutdown_connections 2023-10-06T05:29:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.028+0000 7f564dd9d700 1 -- 172.21.15.33:0/1205811074 wait complete. 2023-10-06T05:29:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.029+0000 7f564dd9d700 1 Processor -- start 2023-10-06T05:29:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.029+0000 7f564dd9d700 1 -- start start 2023-10-06T05:29:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f564dd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56480afb20 0x7f564814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f564dd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f564dd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5648155a50 0x7f5648157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f564dd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564816b5e0 con 0x7f56480c9490 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f564dd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564814e480 con 0x7f5648155a50 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f564dd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564816a9d0 con 0x7f56480afb20 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f5647fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f5647fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36468/0 (socket says 172.21.15.33:36468) 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.030+0000 7f5647fff700 1 -- 172.21.15.33:0/1689794321 learned_addr learned my addr 172.21.15.33:0/1689794321 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f564cd9b700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56480afb20 0x7f564814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f564d59c700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5648155a50 0x7f5648157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5647fff700 1 -- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56480afb20 msgr2=0x7f564814df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5647fff700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56480afb20 0x7f564814df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5647fff700 1 -- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5648155a50 msgr2=0x7f5648157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5647fff700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5648155a50 0x7f5648157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5647fff700 1 -- 172.21.15.33:0/1689794321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563400a5d0 con 0x7f56480c9490 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5647fff700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648150cf0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f563400a370 tx=0x7f563401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634010600 con 0x7f56480c9490 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634010780 con 0x7f56480c9490 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634010b40 con 0x7f56480c9490 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.031+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564816acb0 con 0x7f56480c9490 2023-10-06T05:29:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.032+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564816b270 con 0x7f56480c9490 2023-10-06T05:29:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.033+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564804c380 con 0x7f56480c9490 2023-10-06T05:29:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.034+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f563402a3b0 con 0x7f56480c9490 2023-10-06T05:29:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.034+0000 7f5644ff9700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f562c066f30 0x7f562c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.034+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5634085a40 con 0x7f56480c9490 2023-10-06T05:29:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.035+0000 7f564cd9b700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f562c066f30 0x7f562c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.036+0000 7f564cd9b700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f562c066f30 0x7f562c0693e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f563c0058d0 tx=0x7f563c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.039+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5634052e00 con 0x7f56480c9490 2023-10-06T05:29:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.242+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f564816b720 con 0x7f56480c9490 2023-10-06T05:29:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.243+0000 7f5644ff9700 1 -- 172.21.15.33:0/1689794321 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v206) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5634056420 con 0x7f56480c9490 2023-10-06T05:29:16.244 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f562c066f30 msgr2=0x7f562c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f562c066f30 0x7f562c0693e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f563c0058d0 tx=0x7f563c009040 comp rx=0 tx=0).stop 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 msgr2=0x7f5648150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648150cf0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f563400a370 tx=0x7f563401a000 comp rx=0 tx=0).stop 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 shutdown_connections 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f562c066f30 0x7f562c0693e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5648155a50 0x7f5648157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56480afb20 0x7f564814df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 --2- 172.21.15.33:0/1689794321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56480c9490 0x7f5648150cf0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 >> 172.21.15.33:0/1689794321 conn(0x7f564806d030 msgr2=0x7f564806b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 shutdown_connections 2023-10-06T05:29:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.245+0000 7f564dd9d700 1 -- 172.21.15.33:0/1689794321 wait complete. 2023-10-06T05:29:16.265 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:29:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.454+0000 7fafce59d700 1 Processor -- start 2023-10-06T05:29:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.454+0000 7fafce59d700 1 -- start start 2023-10-06T05:29:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafce59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafc80cc7e0 0x7fafc80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafce59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80ad430 con 0x7fafc80b01f0 2023-10-06T05:29:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafce59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80afe20 con 0x7fafc8156370 2023-10-06T05:29:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafce59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc806bc70 con 0x7fafc80cc7e0 2023-10-06T05:29:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafccd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafc80cc7e0 0x7fafc80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafccd9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fafc80cc7e0 0x7fafc80af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36472/0 (socket says 172.21.15.33:36472) 2023-10-06T05:29:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.455+0000 7fafccd9a700 1 -- 172.21.15.33:0/232413097 learned_addr learned my addr 172.21.15.33:0/232413097 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafccd9a700 1 -- 172.21.15.33:0/232413097 >> v1:172.21.15.167:6789/0 conn(0x7fafc8156370 legacy=0x7fafc80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafccd9a700 1 -- 172.21.15.33:0/232413097 >> v1:172.21.15.33:6789/0 conn(0x7fafc80b01f0 legacy=0x7fafc80ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafccd9a700 1 -- 172.21.15.33:0/232413097 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80cccc0 con 0x7fafc80cc7e0 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafbffff700 1 -- 172.21.15.33:0/232413097 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608774638 0 0) 0x7fafc80ad430 con 0x7fafc80b01f0 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafccd9a700 1 --2- 172.21.15.33:0/232413097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc80af8e0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fafb800bc50 tx=0x7fafb800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497afd3bad23f7ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafbffff700 1 -- 172.21.15.33:0/232413097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800be60 con 0x7fafc80cc7e0 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafbffff700 1 -- 172.21.15.33:0/232413097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800de00 con 0x7fafc80cc7e0 2023-10-06T05:29:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.456+0000 7fafbffff700 1 -- 172.21.15.33:0/232413097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb80196b0 con 0x7fafc80cc7e0 2023-10-06T05:29:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 -- 172.21.15.33:0/232413097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 msgr2=0x7fafc80af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 --2- 172.21.15.33:0/232413097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc80af8e0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fafb800bc50 tx=0x7fafb800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 -- 172.21.15.33:0/232413097 shutdown_connections 2023-10-06T05:29:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 --2- 172.21.15.33:0/232413097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc80af8e0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 -- 172.21.15.33:0/232413097 >> 172.21.15.33:0/232413097 conn(0x7fafc806d030 msgr2=0x7fafc80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 -- 172.21.15.33:0/232413097 shutdown_connections 2023-10-06T05:29:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.457+0000 7fafce59d700 1 -- 172.21.15.33:0/232413097 wait complete. 2023-10-06T05:29:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.458+0000 7fafce59d700 1 Processor -- start 2023-10-06T05:29:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.459+0000 7fafce59d700 1 -- start start 2023-10-06T05:29:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.459+0000 7fafce59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafc80b01f0 0x7fafc81529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.459+0000 7fafce59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc8150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafce59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc8156370 0x7fafc80caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafce59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8152520 con 0x7fafc80cc7e0 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafce59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8151bf0 con 0x7fafc8156370 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafce59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8151ed0 con 0x7fafc80b01f0 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc8150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc8150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36484/0 (socket says 172.21.15.33:36484) 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 -- 172.21.15.33:0/2790492101 learned_addr learned my addr 172.21.15.33:0/2790492101 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafcdd9c700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc8156370 0x7fafc80caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafcd59b700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafc80b01f0 0x7fafc81529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 -- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafc80b01f0 msgr2=0x7fafc81529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafc80b01f0 0x7fafc81529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 -- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc8156370 msgr2=0x7fafc80caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc8156370 0x7fafc80caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 -- 172.21.15.33:0/2790492101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb800b790 con 0x7fafc80cc7e0 2023-10-06T05:29:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafcdd9c700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc8156370 0x7fafc80caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafcd59b700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafc80b01f0 0x7fafc81529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.460+0000 7fafccd9a700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc8150ff0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fafc806d4d0 tx=0x7fafb801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.461+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8010750 con 0x7fafc80cc7e0 2023-10-06T05:29:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.461+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb80108d0 con 0x7fafc80cc7e0 2023-10-06T05:29:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.461+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc81521b0 con 0x7fafc80cc7e0 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.461+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8010c90 con 0x7fafc80cc7e0 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.461+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc8151530 con 0x7fafc80cc7e0 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.462+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc8151720 con 0x7fafc80cc7e0 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.463+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafb802c440 con 0x7fafc80cc7e0 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.463+0000 7fafbd7fa700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafac066f30 0x7fafac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.463+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fafb8014070 con 0x7fafc80cc7e0 2023-10-06T05:29:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.464+0000 7fafcd59b700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafac066f30 0x7fafac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.465+0000 7fafcd59b700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafac066f30 0x7fafac0693e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fafc4001840 tx=0x7fafc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.467+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafb80534f0 con 0x7fafc80cc7e0 2023-10-06T05:29:16.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.671+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fafc804c380 con 0x7fafc80cc7e0 2023-10-06T05:29:16.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.671+0000 7fafbd7fa700 1 -- 172.21.15.33:0/2790492101 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v206) v1 ==== 92+0+10 (secure 0 0 0) 0x7fafb8056b10 con 0x7fafc80cc7e0 2023-10-06T05:29:16.673 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafac066f30 msgr2=0x7fafac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafac066f30 0x7fafac0693e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fafc4001840 tx=0x7fafc4009040 comp rx=0 tx=0).stop 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 msgr2=0x7fafc8150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc8150ff0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fafc806d4d0 tx=0x7fafb801b000 comp rx=0 tx=0).stop 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 shutdown_connections 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fafac066f30 0x7fafac0693e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc8156370 0x7fafc80caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafc80b01f0 0x7fafc81529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 --2- 172.21.15.33:0/2790492101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafc80cc7e0 0x7fafc8150ff0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 >> 172.21.15.33:0/2790492101 conn(0x7fafc806d030 msgr2=0x7fafc806e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:16.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 shutdown_connections 2023-10-06T05:29:16.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.674+0000 7fafce59d700 1 -- 172.21.15.33:0/2790492101 wait complete. 2023-10-06T05:29:16.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:29:16.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.878+0000 7f4aa759e700 1 Processor -- start 2023-10-06T05:29:16.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.878+0000 7f4aa759e700 1 -- start start 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4aa8159530 0x7f4aa806da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa806e3e0 con 0x7f4aa806df70 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa806f220 con 0x7f4aa80b04e0 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa80b1e00 con 0x7f4aa8159530 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4aa8159530 0x7f4aa806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4aa8159530 0x7f4aa806da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36498/0 (socket says 172.21.15.33:36498) 2023-10-06T05:29:16.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa5d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4aa806df70 0x7f4aa80afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55548/0 (socket says 172.21.15.33:55548) 2023-10-06T05:29:16.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.879+0000 7f4aa659c700 1 -- 172.21.15.33:0/2346189301 learned_addr learned my addr 172.21.15.33:0/2346189301 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.880+0000 7f4aa659c700 1 -- 172.21.15.33:0/2346189301 >> v1:172.21.15.167:6789/0 conn(0x7f4aa80b04e0 legacy=0x7f4aa815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.880+0000 7f4aa659c700 1 -- 172.21.15.33:0/2346189301 >> v1:172.21.15.33:6789/0 conn(0x7f4aa806df70 legacy=0x7f4aa80afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.880+0000 7f4aa659c700 1 -- 172.21.15.33:0/2346189301 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa806e610 con 0x7f4aa8159530 2023-10-06T05:29:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.880+0000 7f4aa559a700 1 -- 172.21.15.33:0/2346189301 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882008610 0 0) 0x7f4aa806e3e0 con 0x7f4aa806df70 2023-10-06T05:29:16.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.880+0000 7f4aa659c700 1 --2- 172.21.15.33:0/2346189301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa8159530 0x7f4aa806da30 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f4a980099b0 tx=0x7f4a98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6c8e39c42c3d5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.880+0000 7f4aa559a700 1 -- 172.21.15.33:0/2346189301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800b860 con 0x7f4aa8159530 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa559a700 1 -- 172.21.15.33:0/2346189301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9800b9c0 con 0x7f4aa8159530 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa559a700 1 -- 172.21.15.33:0/2346189301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800bd60 con 0x7f4aa8159530 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 -- 172.21.15.33:0/2346189301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa8159530 msgr2=0x7f4aa806da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 --2- 172.21.15.33:0/2346189301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa8159530 0x7f4aa806da30 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f4a980099b0 tx=0x7f4a98009250 comp rx=0 tx=0).stop 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 -- 172.21.15.33:0/2346189301 shutdown_connections 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 --2- 172.21.15.33:0/2346189301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa8159530 0x7f4aa806da30 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 -- 172.21.15.33:0/2346189301 >> 172.21.15.33:0/2346189301 conn(0x7f4aa806c560 msgr2=0x7f4aa806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 -- 172.21.15.33:0/2346189301 shutdown_connections 2023-10-06T05:29:16.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.881+0000 7f4aa759e700 1 -- 172.21.15.33:0/2346189301 wait complete. 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.883+0000 7f4aa759e700 1 Processor -- start 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.883+0000 7f4aa759e700 1 -- start start 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aa806df70 0x7f4aa81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 0x7f4aa816a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aa8159530 0x7f4aa80c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa815cac0 con 0x7f4aa80b04e0 2023-10-06T05:29:16.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa815beb0 con 0x7f4aa806df70 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa815c190 con 0x7f4aa8159530 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 0x7f4aa816a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 0x7f4aa816a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36500/0 (socket says 172.21.15.33:36500) 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 -- 172.21.15.33:0/4039453324 learned_addr learned my addr 172.21.15.33:0/4039453324 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa659c700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aa806df70 0x7f4aa81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa6d9d700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aa8159530 0x7f4aa80c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 -- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aa8159530 msgr2=0x7f4aa80c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aa8159530 0x7f4aa80c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 -- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aa806df70 msgr2=0x7f4aa81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aa806df70 0x7f4aa81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.884+0000 7f4aa5d9b700 1 -- 172.21.15.33:0/4039453324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980094f0 con 0x7f4aa80b04e0 2023-10-06T05:29:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.885+0000 7f4aa5d9b700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 0x7f4aa816a1d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c560 tx=0x7f4a9c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.885+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c005a70 con 0x7f4aa80b04e0 2023-10-06T05:29:16.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.885+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c005bd0 con 0x7f4aa80b04e0 2023-10-06T05:29:16.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.885+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0070d0 con 0x7f4aa80b04e0 2023-10-06T05:29:16.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.885+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa815c470 con 0x7f4aa80b04e0 2023-10-06T05:29:16.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.885+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa81536e0 con 0x7f4aa80b04e0 2023-10-06T05:29:16.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.886+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa806da30 con 0x7f4aa80b04e0 2023-10-06T05:29:16.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.887+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a9c0074c0 con 0x7f4aa80b04e0 2023-10-06T05:29:16.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.888+0000 7f4a967fc700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a88066f30 0x7f4a880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:16.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.888+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4a9c083950 con 0x7f4aa80b04e0 2023-10-06T05:29:16.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.888+0000 7f4aa659c700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a88066f30 0x7f4a880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:16.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.889+0000 7f4aa659c700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a88066f30 0x7f4a880693e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4a980099b0 tx=0x7f4a98017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:16.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:16.892+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a9c050d10 con 0x7f4aa80b04e0 2023-10-06T05:29:17.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.097+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4aa815cc00 con 0x7f4aa80b04e0 2023-10-06T05:29:17.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.098+0000 7f4a967fc700 1 -- 172.21.15.33:0/4039453324 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v206) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4a9c017070 con 0x7f4aa80b04e0 2023-10-06T05:29:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:17.100 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:29:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:29:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:29:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:29:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:29:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:17.102 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:29:17.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a88066f30 msgr2=0x7f4a880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:17.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a88066f30 0x7f4a880693e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4a980099b0 tx=0x7f4a98017000 comp rx=0 tx=0).stop 2023-10-06T05:29:17.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 msgr2=0x7f4aa816a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:17.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 0x7f4aa816a1d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c560 tx=0x7f4a9c007f60 comp rx=0 tx=0).stop 2023-10-06T05:29:17.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 shutdown_connections 2023-10-06T05:29:17.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4a88066f30 0x7f4a880693e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aa8159530 0x7f4aa80c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aa806df70 0x7f4aa81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 --2- 172.21.15.33:0/4039453324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa80b04e0 0x7f4aa816a1d0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 >> 172.21.15.33:0/4039453324 conn(0x7f4aa806c560 msgr2=0x7f4aa815a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:17.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 shutdown_connections 2023-10-06T05:29:17.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.100+0000 7f4aa759e700 1 -- 172.21.15.33:0/4039453324 wait complete. 2023-10-06T05:29:17.119 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:29:17.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:29:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.310+0000 7fe609f9d700 1 Processor -- start 2023-10-06T05:29:17.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.310+0000 7fe609f9d700 1 -- start start 2023-10-06T05:29:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.311+0000 7fe609f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6040ad010 0x7fe60406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.311+0000 7fe609f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6040ad3e0 0x7fe60415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe609f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60406b1f0 con 0x7fe60415ab00 2023-10-06T05:29:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe609f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406a3f0 con 0x7fe6040ad010 2023-10-06T05:29:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe609f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406a5e0 con 0x7fe6040ad3e0 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6040ad010 0x7fe60406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6040ad010 0x7fe60406ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36506/0 (socket says 172.21.15.33:36506) 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 -- 172.21.15.33:0/2217137814 learned_addr learned my addr 172.21.15.33:0/2217137814 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe603fff700 1 --2- 172.21.15.33:0/2217137814 >> v2:172.21.15.167:3300/0 conn(0x7fe6040ad3e0 0x7fe60415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 -- 172.21.15.33:0/2217137814 >> v2:172.21.15.167:3300/0 conn(0x7fe6040ad3e0 msgr2=0x7fe60415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 --2- 172.21.15.33:0/2217137814 >> v2:172.21.15.167:3300/0 conn(0x7fe6040ad3e0 0x7fe60415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 -- 172.21.15.33:0/2217137814 >> v1:172.21.15.167:6789/0 conn(0x7fe60415ab00 legacy=0x7fe604156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.312+0000 7fe608f9b700 1 -- 172.21.15.33:0/2217137814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60406a8c0 con 0x7fe6040ad010 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.313+0000 7fe608f9b700 1 --2- 172.21.15.33:0/2217137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6040ad010 0x7fe60406ea00 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fe6040b2230 tx=0x7fe5f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cb72b7ca840897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.313+0000 7fe6037fe700 1 -- 172.21.15.33:0/2217137814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8009e70 con 0x7fe6040ad010 2023-10-06T05:29:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.313+0000 7fe6037fe700 1 -- 172.21.15.33:0/2217137814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f800b8f0 con 0x7fe6040ad010 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.313+0000 7fe6037fe700 1 -- 172.21.15.33:0/2217137814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800bc90 con 0x7fe6040ad010 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.313+0000 7fe609f9d700 1 -- 172.21.15.33:0/2217137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6040ad010 msgr2=0x7fe60406ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.313+0000 7fe609f9d700 1 --2- 172.21.15.33:0/2217137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6040ad010 0x7fe60406ea00 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fe6040b2230 tx=0x7fe5f8009250 comp rx=0 tx=0).stop 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.314+0000 7fe609f9d700 1 -- 172.21.15.33:0/2217137814 shutdown_connections 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.314+0000 7fe609f9d700 1 --2- 172.21.15.33:0/2217137814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6040ad010 0x7fe60406ea00 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.314+0000 7fe609f9d700 1 --2- 172.21.15.33:0/2217137814 >> v2:172.21.15.167:3300/0 conn(0x7fe6040ad3e0 0x7fe60415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.314+0000 7fe609f9d700 1 -- 172.21.15.33:0/2217137814 >> 172.21.15.33:0/2217137814 conn(0x7fe604068090 msgr2=0x7fe60406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.314+0000 7fe609f9d700 1 -- 172.21.15.33:0/2217137814 shutdown_connections 2023-10-06T05:29:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.314+0000 7fe609f9d700 1 -- 172.21.15.33:0/2217137814 wait complete. 2023-10-06T05:29:17.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.315+0000 7fe609f9d700 1 Processor -- start 2023-10-06T05:29:17.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.315+0000 7fe609f9d700 1 -- start start 2023-10-06T05:29:17.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe609f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6040ad010 0x7fe6040b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:17.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe609f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6040ad3e0 0x7fe60413a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:17.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe609f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 0x7fe6040b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:17.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe609f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60416d460 con 0x7fe60415ab00 2023-10-06T05:29:17.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe609f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60416d740 con 0x7fe6040ad3e0 2023-10-06T05:29:17.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe609f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60416da20 con 0x7fe6040ad010 2023-10-06T05:29:17.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.316+0000 7fe60979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 0x7fe6040b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:17.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 0x7fe6040b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36522/0 (socket says 172.21.15.33:36522) 2023-10-06T05:29:17.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 -- 172.21.15.33:0/4266197700 learned_addr learned my addr 172.21.15.33:0/4266197700 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:17.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe603fff700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6040ad3e0 0x7fe60413a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:17.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe608f9b700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6040ad010 0x7fe6040b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:17.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 -- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6040ad010 msgr2=0x7fe6040b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6040ad010 0x7fe6040b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 -- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6040ad3e0 msgr2=0x7fe60413a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6040ad3e0 0x7fe60413a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 -- 172.21.15.33:0/4266197700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80094f0 con 0x7fe60415ab00 2023-10-06T05:29:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe608f9b700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6040ad010 0x7fe6040b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:17.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe60979c700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 0x7fe6040b0140 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00c840 tx=0x7fe5fc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:17.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.317+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc012630 con 0x7fe60415ab00 2023-10-06T05:29:17.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.318+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc0127b0 con 0x7fe60415ab00 2023-10-06T05:29:17.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.318+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc012b70 con 0x7fe60415ab00 2023-10-06T05:29:17.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.318+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60416dd00 con 0x7fe60415ab00 2023-10-06T05:29:17.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.318+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60414d210 con 0x7fe60415ab00 2023-10-06T05:29:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.319+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe604169bc0 con 0x7fe60415ab00 2023-10-06T05:29:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.320+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5fc003030 con 0x7fe60415ab00 2023-10-06T05:29:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.320+0000 7fe600ff9700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5e8066f30 0x7fe5e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.320+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe5fc0842e0 con 0x7fe60415ab00 2023-10-06T05:29:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.321+0000 7fe608f9b700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5e8066f30 0x7fe5e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.322+0000 7fe608f9b700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5e8066f30 0x7fe5e80693e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe6040b2230 tx=0x7fe5f8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.325+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe604169bc0 con 0x7fe60415ab00 2023-10-06T05:29:17.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:17.530+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe604169bc0 con 0x7fe60415ab00 2023-10-06T05:29:18.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.099+0000 7fe600ff9700 1 -- 172.21.15.33:0/4266197700 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v207) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe604169bc0 con 0x7fe60415ab00 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.103+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5e8066f30 msgr2=0x7fe5e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.103+0000 7fe609f9d700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5e8066f30 0x7fe5e80693e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe6040b2230 tx=0x7fe5f8017000 comp rx=0 tx=0).stop 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 msgr2=0x7fe6040b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 0x7fe6040b0140 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00c840 tx=0x7fe5fc001670 comp rx=0 tx=0).stop 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 shutdown_connections 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5e8066f30 0x7fe5e80693e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe60415ab00 0x7fe6040b0140 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6040ad010 0x7fe6040b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 --2- 172.21.15.33:0/4266197700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6040ad3e0 0x7fe60413a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.104+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 >> 172.21.15.33:0/4266197700 conn(0x7fe604068090 msgr2=0x7fe6040c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.105+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 shutdown_connections 2023-10-06T05:29:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.105+0000 7fe609f9d700 1 -- 172.21.15.33:0/4266197700 wait complete. 2023-10-06T05:29:18.111 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:29:18.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:29:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.325+0000 7ff6b319d700 1 Processor -- start 2023-10-06T05:29:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.325+0000 7ff6b319d700 1 -- start start 2023-10-06T05:29:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.326+0000 7ff6b319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac14e580 0x7ff6ac0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.326+0000 7ff6b319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0cca00 con 0x7ff6ac0c8ff0 2023-10-06T05:29:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.326+0000 7ff6b319d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0b0870 con 0x7ff6ac0cc590 2023-10-06T05:29:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.326+0000 7ff6b319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0c9470 con 0x7ff6ac14e580 2023-10-06T05:29:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b299c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff6ac0c8ff0 0x7ff6ac0b0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:55554/0 (socket says 172.21.15.33:55554) 2023-10-06T05:29:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b299c700 1 -- 172.21.15.33:0/2906323642 learned_addr learned my addr 172.21.15.33:0/2906323642 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b219b700 1 --2- 172.21.15.33:0/2906323642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac14e580 0x7ff6ac0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142937321 0 0) 0x7ff6ac0cca00 con 0x7ff6ac0c8ff0 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff698003570 con 0x7ff6ac0c8ff0 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067980966 0 0) 0x7ff6ac0b0870 con 0x7ff6ac0cc590 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ac0cca00 con 0x7ff6ac0cc590 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b219b700 1 -- 172.21.15.33:0/2906323642 >> v1:172.21.15.167:6789/0 conn(0x7ff6ac0cc590 legacy=0x7ff6ac0c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b219b700 1 -- 172.21.15.33:0/2906323642 >> v1:172.21.15.33:6789/0 conn(0x7ff6ac0c8ff0 legacy=0x7ff6ac0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.327+0000 7ff6b219b700 1 -- 172.21.15.33:0/2906323642 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0c9930 con 0x7ff6ac14e580 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b219b700 1 --2- 172.21.15.33:0/2906323642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac14e580 0x7ff6ac0cc050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009910 tx=0x7ff6a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7911c4ea96e8afb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8011630 con 0x7ff6ac14e580 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a8011790 con 0x7ff6ac14e580 2023-10-06T05:29:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b1199700 1 -- 172.21.15.33:0/2906323642 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8011b30 con 0x7ff6ac14e580 2023-10-06T05:29:18.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b319d700 1 -- 172.21.15.33:0/2906323642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac14e580 msgr2=0x7ff6ac0cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b319d700 1 --2- 172.21.15.33:0/2906323642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac14e580 0x7ff6ac0cc050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009910 tx=0x7ff6a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.328+0000 7ff6b319d700 1 -- 172.21.15.33:0/2906323642 shutdown_connections 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.329+0000 7ff6b319d700 1 --2- 172.21.15.33:0/2906323642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac14e580 0x7ff6ac0cc050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.329+0000 7ff6b319d700 1 -- 172.21.15.33:0/2906323642 >> 172.21.15.33:0/2906323642 conn(0x7ff6ac06c560 msgr2=0x7ff6ac06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.329+0000 7ff6b319d700 1 -- 172.21.15.33:0/2906323642 shutdown_connections 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.329+0000 7ff6b319d700 1 -- 172.21.15.33:0/2906323642 wait complete. 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.330+0000 7ff6b319d700 1 Processor -- start 2023-10-06T05:29:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.330+0000 7ff6b319d700 1 -- start start 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0c8ff0 0x7ff6ac1490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 0x7ff6ac16a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac14e580 0x7ff6ac16af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac14d600 con 0x7ff6ac0cc590 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac14f630 con 0x7ff6ac14e580 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac14d320 con 0x7ff6ac0c8ff0 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.331+0000 7ff6b199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 0x7ff6ac16a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 0x7ff6ac16a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36524/0 (socket says 172.21.15.33:36524) 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 -- 172.21.15.33:0/3398119226 learned_addr learned my addr 172.21.15.33:0/3398119226 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:18.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b299c700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac14e580 0x7ff6ac16af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b219b700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0c8ff0 0x7ff6ac1490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 -- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0c8ff0 msgr2=0x7ff6ac1490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0c8ff0 0x7ff6ac1490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 -- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac14e580 msgr2=0x7ff6ac16af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac14e580 0x7ff6ac16af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 -- 172.21.15.33:0/3398119226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a8009420 con 0x7ff6ac0cc590 2023-10-06T05:29:18.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.332+0000 7ff6b199a700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 0x7ff6ac16a9c0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c790 tx=0x7ff69c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:18.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.333+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010510 con 0x7ff6ac0cc590 2023-10-06T05:29:18.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.333+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c010690 con 0x7ff6ac0cc590 2023-10-06T05:29:18.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.333+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010a50 con 0x7ff6ac0cc590 2023-10-06T05:29:18.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.333+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac14efe0 con 0x7ff6ac0cc590 2023-10-06T05:29:18.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.333+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1673e0 con 0x7ff6ac0cc590 2023-10-06T05:29:18.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.334+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac14f7a0 con 0x7ff6ac0cc590 2023-10-06T05:29:18.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.335+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff69c01b3b0 con 0x7ff6ac0cc590 2023-10-06T05:29:18.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.336+0000 7ff6a27fc700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff690066f30 0x7ff6900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:18.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.336+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5576+0+0 (secure 0 0 0) 0x7ff69c083d20 con 0x7ff6ac0cc590 2023-10-06T05:29:18.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.336+0000 7ff6b219b700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff690066f30 0x7ff6900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:18.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.337+0000 7ff6b219b700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff690066f30 0x7ff6900693e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac06cfe0 tx=0x7ff6a8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:18.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.340+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff69c050f70 con 0x7ff6ac0cc590 2023-10-06T05:29:18.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:18.542+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff6ac04c380 con 0x7ff6ac0cc590 2023-10-06T05:29:19.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.105+0000 7ff6a27fc700 1 -- 172.21.15.33:0/3398119226 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v208) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff69c017020 con 0x7ff6ac0cc590 2023-10-06T05:29:19.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.115+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff690066f30 msgr2=0x7ff6900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:19.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.115+0000 7ff6b319d700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff690066f30 0x7ff6900693e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac06cfe0 tx=0x7ff6a8017000 comp rx=0 tx=0).stop 2023-10-06T05:29:19.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.115+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 msgr2=0x7ff6ac16a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:19.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.115+0000 7ff6b319d700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 0x7ff6ac16a9c0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c790 tx=0x7ff69c007e10 comp rx=0 tx=0).stop 2023-10-06T05:29:19.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.116+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 shutdown_connections 2023-10-06T05:29:19.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.116+0000 7ff6b319d700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff690066f30 0x7ff6900693e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.116+0000 7ff6b319d700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac14e580 0x7ff6ac16af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.116+0000 7ff6b319d700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0c8ff0 0x7ff6ac1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.116+0000 7ff6b319d700 1 --2- 172.21.15.33:0/3398119226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0cc590 0x7ff6ac16a9c0 secure :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c790 tx=0x7ff69c007e10 comp rx=0 tx=0).stop 2023-10-06T05:29:19.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.116+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 >> 172.21.15.33:0/3398119226 conn(0x7ff6ac06c560 msgr2=0x7ff6ac149d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:19.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.118+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 shutdown_connections 2023-10-06T05:29:19.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.118+0000 7ff6b319d700 1 -- 172.21.15.33:0/3398119226 wait complete. 2023-10-06T05:29:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:29:19.146 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:29:19.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.337+0000 7f177959d700 1 Processor -- start 2023-10-06T05:29:19.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.337+0000 7f177959d700 1 -- start start 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f177959d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f177406c7b0 0x7f1774067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f177959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17740aff90 0x7f17740a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f177959d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177415d3a0 con 0x7f177406cb80 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f177959d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17740b0470 con 0x7f177406c7b0 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f177959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17740b0750 con 0x7f17740aff90 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f1773fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f177406c7b0 0x7f1774067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f1773fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f177406c7b0 0x7f1774067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36528/0 (socket says 172.21.15.33:36528) 2023-10-06T05:29:19.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.338+0000 7f1773fff700 1 -- 172.21.15.33:0/3998054474 learned_addr learned my addr 172.21.15.33:0/3998054474 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1773fff700 1 -- 172.21.15.33:0/3998054474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17740aff90 msgr2=0x7f17740a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1773fff700 1 --2- 172.21.15.33:0/3998054474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17740aff90 0x7f17740a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1773fff700 1 -- 172.21.15.33:0/3998054474 >> v1:172.21.15.167:6789/0 conn(0x7f177406cb80 legacy=0x7f177415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1773fff700 1 -- 172.21.15.33:0/3998054474 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740b0a30 con 0x7f177406c7b0 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1773fff700 1 --2- 172.21.15.33:0/3998054474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f177406c7b0 0x7f1774067fb0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f1764009990 tx=0x7f1764009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e0bb09a6aceb82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1772ffd700 1 -- 172.21.15.33:0/3998054474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764005c10 con 0x7f177406c7b0 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1772ffd700 1 -- 172.21.15.33:0/3998054474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764005d70 con 0x7f177406c7b0 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.339+0000 7f1772ffd700 1 -- 172.21.15.33:0/3998054474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400b840 con 0x7f177406c7b0 2023-10-06T05:29:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 -- 172.21.15.33:0/3998054474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f177406c7b0 msgr2=0x7f1774067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 --2- 172.21.15.33:0/3998054474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f177406c7b0 0x7f1774067fb0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f1764009990 tx=0x7f1764009220 comp rx=0 tx=0).stop 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 -- 172.21.15.33:0/3998054474 shutdown_connections 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 --2- 172.21.15.33:0/3998054474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f177406c7b0 0x7f1774067fb0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 --2- 172.21.15.33:0/3998054474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17740aff90 0x7f17740a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 -- 172.21.15.33:0/3998054474 >> 172.21.15.33:0/3998054474 conn(0x7f177406d670 msgr2=0x7f177406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 -- 172.21.15.33:0/3998054474 shutdown_connections 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.340+0000 7f177959d700 1 -- 172.21.15.33:0/3998054474 wait complete. 2023-10-06T05:29:19.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.341+0000 7f177959d700 1 Processor -- start 2023-10-06T05:29:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.342+0000 7f177959d700 1 -- start start 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.342+0000 7f177959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f177406c7b0 0x7f1774160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.342+0000 7f177959d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177406cb80 0x7f177414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f177959d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 0x7f1774151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f177959d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17741517d0 con 0x7f17740aff90 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f177959d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17740adc40 con 0x7f177406cb80 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f177959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17740adee0 con 0x7f177406c7b0 2023-10-06T05:29:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 0x7f1774151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 0x7f1774151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36544/0 (socket says 172.21.15.33:36544) 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 -- 172.21.15.33:0/1531688692 learned_addr learned my addr 172.21.15.33:0/1531688692 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1773fff700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f177406c7b0 0x7f1774160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 -- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f177406c7b0 msgr2=0x7f1774160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f177406c7b0 0x7f1774160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f17737fe700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177406cb80 0x7f177414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 -- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177406cb80 msgr2=0x7f177414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177406cb80 0x7f177414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f1778d9c700 1 -- 172.21.15.33:0/1531688692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764009480 con 0x7f17740aff90 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.343+0000 7f17737fe700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177406cb80 0x7f177414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.344+0000 7f1778d9c700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 0x7f1774151290 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f176c00bbb0 tx=0x7f176c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:19.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.344+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c0092e0 con 0x7f17740aff90 2023-10-06T05:29:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.344+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c00c7a0 con 0x7f17740aff90 2023-10-06T05:29:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.344+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17740ae1c0 con 0x7f17740aff90 2023-10-06T05:29:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.344+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c00cb40 con 0x7f17740aff90 2023-10-06T05:29:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.344+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1774151a00 con 0x7f17740aff90 2023-10-06T05:29:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.345+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177404c380 con 0x7f17740aff90 2023-10-06T05:29:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.346+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f176c012070 con 0x7f17740aff90 2023-10-06T05:29:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.347+0000 7f175bfff700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1760066f30 0x7f17600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.347+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f176c084410 con 0x7f17740aff90 2023-10-06T05:29:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.347+0000 7f1773fff700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1760066f30 0x7f17600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.348+0000 7f1773fff700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1760066f30 0x7f17600693e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1764003080 tx=0x7f1764015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.352+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f176c0514e0 con 0x7f17740aff90 2023-10-06T05:29:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:19.599+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f17740b0d10 con 0x7f17740aff90 2023-10-06T05:29:20.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.122+0000 7f175bfff700 1 -- 172.21.15.33:0/1531688692 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v158) v1 ==== 174+0+0 (secure 0 0 0) 0x7f176c054b00 con 0x7f17740aff90 2023-10-06T05:29:20.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1760066f30 msgr2=0x7f17600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1760066f30 0x7f17600693e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1764003080 tx=0x7f1764015000 comp rx=0 tx=0).stop 2023-10-06T05:29:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 msgr2=0x7f1774151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 0x7f1774151290 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f176c00bbb0 tx=0x7f176c009840 comp rx=0 tx=0).stop 2023-10-06T05:29:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 shutdown_connections 2023-10-06T05:29:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1760066f30 0x7f17600693e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17740aff90 0x7f1774151290 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f177406c7b0 0x7f1774160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 --2- 172.21.15.33:0/1531688692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177406cb80 0x7f177414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.138+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 >> 172.21.15.33:0/1531688692 conn(0x7f177406d670 msgr2=0x7f1774155dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.139+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 shutdown_connections 2023-10-06T05:29:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.139+0000 7f177959d700 1 -- 172.21.15.33:0/1531688692 wait complete. 2023-10-06T05:29:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 31 and data pool 32 2023-10-06T05:29:20.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:20.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.356+0000 7f6acb59e700 1 Processor -- start 2023-10-06T05:29:20.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.356+0000 7f6acb59e700 1 -- start start 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.357+0000 7f6acb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6acc13faa0 0x7f6acc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.357+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc0c7940 0x7f6acc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.357+0000 7f6acb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6acc06aad0 0x7f6acc06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.357+0000 7f6acb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6acc0c9cb0 con 0x7f6acc13faa0 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.357+0000 7f6acb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6acc0ca270 con 0x7f6acc06aad0 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.357+0000 7f6acb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc06b840 con 0x7f6acc0c7940 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6acc13faa0 0x7f6acc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6acc13faa0 0x7f6acc13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48802/0 (socket says 172.21.15.33:48802) 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 -- 172.21.15.33:0/3742351305 learned_addr learned my addr 172.21.15.33:0/3742351305 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/3742351305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc0c7940 0x7f6acc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6acad9d700 1 --2- 172.21.15.33:0/3742351305 >> v2:172.21.15.167:3300/0 conn(0x7f6acc06aad0 0x7f6acc06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 -- 172.21.15.33:0/3742351305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc0c7940 msgr2=0x7f6acc0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 --2- 172.21.15.33:0/3742351305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc0c7940 0x7f6acc0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 -- 172.21.15.33:0/3742351305 >> v2:172.21.15.167:3300/0 conn(0x7f6acc06aad0 msgr2=0x7f6acc06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 --2- 172.21.15.33:0/3742351305 >> v2:172.21.15.167:3300/0 conn(0x7f6acc06aad0 0x7f6acc06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.358+0000 7f6aca59c700 1 -- 172.21.15.33:0/3742351305 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc06bb20 con 0x7f6acc13faa0 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.359+0000 7f6aca59c700 1 --2- 172.21.15.33:0/3742351305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc13faa0 0x7f6acc13fe70 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0cc0b0 tx=0x7f6abc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2771577566851439 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.359+0000 7f6ac959a700 1 -- 172.21.15.33:0/3742351305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00b860 con 0x7f6acc13faa0 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.359+0000 7f6ac959a700 1 -- 172.21.15.33:0/3742351305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc00b9c0 con 0x7f6acc13faa0 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.359+0000 7f6ac959a700 1 -- 172.21.15.33:0/3742351305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00bd60 con 0x7f6acc13faa0 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.359+0000 7f6acb59e700 1 -- 172.21.15.33:0/3742351305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc13faa0 msgr2=0x7f6acc13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.359+0000 7f6acb59e700 1 --2- 172.21.15.33:0/3742351305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc13faa0 0x7f6acc13fe70 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0cc0b0 tx=0x7f6abc009250 comp rx=0 tx=0).stop 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 -- 172.21.15.33:0/3742351305 shutdown_connections 2023-10-06T05:29:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 --2- 172.21.15.33:0/3742351305 >> v2:172.21.15.167:3300/0 conn(0x7f6acc06aad0 0x7f6acc06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 --2- 172.21.15.33:0/3742351305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc13faa0 0x7f6acc13fe70 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 --2- 172.21.15.33:0/3742351305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc0c7940 0x7f6acc0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 -- 172.21.15.33:0/3742351305 >> 172.21.15.33:0/3742351305 conn(0x7f6acc06d030 msgr2=0x7f6acc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 -- 172.21.15.33:0/3742351305 shutdown_connections 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.360+0000 7f6acb59e700 1 -- 172.21.15.33:0/3742351305 wait complete. 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.361+0000 7f6acb59e700 1 Processor -- start 2023-10-06T05:29:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.361+0000 7f6acb59e700 1 -- start start 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6acc06aad0 0x7f6acc12a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 0x7f6acc15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6acb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc13faa0 0x7f6acc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6acb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc156fc0 con 0x7f6acc0c7940 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6acb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1572a0 con 0x7f6acc06aad0 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6acb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc157580 con 0x7f6acc13faa0 2023-10-06T05:29:20.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.362+0000 7f6ac9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 0x7f6acc15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 0x7f6acc15c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48808/0 (socket says 172.21.15.33:48808) 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2823322740 learned_addr learned my addr 172.21.15.33:0/2823322740 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6acad9d700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc13faa0 0x7f6acc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6aca59c700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6acc06aad0 0x7f6acc12a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc13faa0 msgr2=0x7f6acc15c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc13faa0 0x7f6acc15c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6acc06aad0 msgr2=0x7f6acc12a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6acc06aad0 0x7f6acc12a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 -- 172.21.15.33:0/2823322740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0094f0 con 0x7f6acc0c7940 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6ac9d9b700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 0x7f6acc15c180 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000d890 tx=0x7f6ac0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.363+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000b600 con 0x7f6acc0c7940 2023-10-06T05:29:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.364+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000b760 con 0x7f6acc0c7940 2023-10-06T05:29:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.364+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000bae0 con 0x7f6acc0c7940 2023-10-06T05:29:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.364+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc157860 con 0x7f6acc0c7940 2023-10-06T05:29:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.364+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc156c00 con 0x7f6acc0c7940 2023-10-06T05:29:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.365+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc157d00 con 0x7f6acc0c7940 2023-10-06T05:29:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.366+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ac001e460 con 0x7f6acc0c7940 2023-10-06T05:29:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.366+0000 7f6aba7fc700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6aac066f30 0x7f6aac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.367+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ac0084de0 con 0x7f6acc0c7940 2023-10-06T05:29:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.367+0000 7f6aca59c700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6aac066f30 0x7f6aac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.368+0000 7f6aca59c700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6aac066f30 0x7f6aac0693e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0cc0b0 tx=0x7f6abc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:20.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.371+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ac0051e50 con 0x7f6acc0c7940 2023-10-06T05:29:20.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:29:20.456 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:29:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.574+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6acc04c380 con 0x7f6acc0c7940 2023-10-06T05:29:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.575+0000 7f6aba7fc700 1 -- 172.21.15.33:0/2823322740 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f6ac0055470 con 0x7f6acc0c7940 2023-10-06T05:29:20.578 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:20.578 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":209,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:19.601466+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:17.571482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:18.582693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.577+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6aac066f30 msgr2=0x7f6aac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.577+0000 7f6acb59e700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6aac066f30 0x7f6aac0693e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0cc0b0 tx=0x7f6abc017000 comp rx=0 tx=0).stop 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.577+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 msgr2=0x7f6acc15c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.577+0000 7f6acb59e700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 0x7f6acc15c180 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000d890 tx=0x7f6ac0008040 comp rx=0 tx=0).stop 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 shutdown_connections 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6aac066f30 0x7f6aac0693e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6acc13faa0 0x7f6acc15c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6acc06aad0 0x7f6acc12a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 --2- 172.21.15.33:0/2823322740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0c7940 0x7f6acc15c180 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 >> 172.21.15.33:0/2823322740 conn(0x7f6acc06d030 msgr2=0x7f6acc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 shutdown_connections 2023-10-06T05:29:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.578+0000 7f6acb59e700 1 -- 172.21.15.33:0/2823322740 wait complete. 2023-10-06T05:29:20.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.823+0000 7fb35199d700 1 Processor -- start 2023-10-06T05:29:20.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.823+0000 7fb35199d700 1 -- start start 2023-10-06T05:29:20.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.824+0000 7fb35199d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb34c0c6340 0x7fb34c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.824+0000 7fb35199d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c06c820 con 0x7fb34c159520 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.824+0000 7fb35199d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c06a160 con 0x7fb34c0c80d0 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.824+0000 7fb35199d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0c8800 con 0x7fb34c0c6340 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.824+0000 7fb34bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb34c0c6340 0x7fb34c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb34c0c6340 0x7fb34c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48816/0 (socket says 172.21.15.33:48816) 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb35099b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb34c159520 0x7fb34c0c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58312/0 (socket says 172.21.15.33:58312) 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34bfff700 1 -- 172.21.15.33:0/874045705 learned_addr learned my addr 172.21.15.33:0/874045705 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34bfff700 1 -- 172.21.15.33:0/874045705 >> v1:172.21.15.167:6789/0 conn(0x7fb34c0c80d0 legacy=0x7fb34c06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34bfff700 1 -- 172.21.15.33:0/874045705 >> v1:172.21.15.33:6789/0 conn(0x7fb34c159520 legacy=0x7fb34c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34bfff700 1 -- 172.21.15.33:0/874045705 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0c8ae0 con 0x7fb34c0c6340 2023-10-06T05:29:20.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34b7fe700 1 -- 172.21.15.33:0/874045705 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497856066 0 0) 0x7fb34c06c820 con 0x7fb34c159520 2023-10-06T05:29:20.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34bfff700 1 --2- 172.21.15.33:0/874045705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c6340 0x7fb34c0c7b90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fb3380099b0 tx=0x7fb338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7512526bb77ad78d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.825+0000 7fb34b7fe700 1 -- 172.21.15.33:0/874045705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800b860 con 0x7fb34c0c6340 2023-10-06T05:29:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb34b7fe700 1 -- 172.21.15.33:0/874045705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800b9c0 con 0x7fb34c0c6340 2023-10-06T05:29:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb34b7fe700 1 -- 172.21.15.33:0/874045705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800bd60 con 0x7fb34c0c6340 2023-10-06T05:29:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 -- 172.21.15.33:0/874045705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c6340 msgr2=0x7fb34c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 --2- 172.21.15.33:0/874045705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c6340 0x7fb34c0c7b90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fb3380099b0 tx=0x7fb338009250 comp rx=0 tx=0).stop 2023-10-06T05:29:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 -- 172.21.15.33:0/874045705 shutdown_connections 2023-10-06T05:29:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 --2- 172.21.15.33:0/874045705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c6340 0x7fb34c0c7b90 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 -- 172.21.15.33:0/874045705 >> 172.21.15.33:0/874045705 conn(0x7fb34c0cc930 msgr2=0x7fb34c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 -- 172.21.15.33:0/874045705 shutdown_connections 2023-10-06T05:29:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.826+0000 7fb35199d700 1 -- 172.21.15.33:0/874045705 wait complete. 2023-10-06T05:29:20.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.828+0000 7fb35199d700 1 Processor -- start 2023-10-06T05:29:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.828+0000 7fb35199d700 1 -- start start 2023-10-06T05:29:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb34c0c6340 0x7fb34c14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 0x7fb34c1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34c159520 0x7fb34c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c14aa00 con 0x7fb34c0c80d0 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0abf80 con 0x7fb34c0c6340 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c15a260 con 0x7fb34c159520 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 0x7fb34c1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 0x7fb34c1598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48824/0 (socket says 172.21.15.33:48824) 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 -- 172.21.15.33:0/404484921 learned_addr learned my addr 172.21.15.33:0/404484921 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35099b700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb34c0c6340 0x7fb34c14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35119c700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34c159520 0x7fb34c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 -- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34c159520 msgr2=0x7fb34c15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34c159520 0x7fb34c15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 -- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb34c0c6340 msgr2=0x7fb34c14a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb34c0c6340 0x7fb34c14a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb34bfff700 1 -- 172.21.15.33:0/404484921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380094f0 con 0x7fb34c0c80d0 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.829+0000 7fb35099b700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb34c0c6340 0x7fb34c14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.830+0000 7fb34bfff700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 0x7fb34c1598f0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fb3380094c0 tx=0x7fb338017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.830+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33801f070 con 0x7fb34c0c80d0 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.830+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338011c40 con 0x7fb34c0c80d0 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.830+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338025420 con 0x7fb34c0c80d0 2023-10-06T05:29:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.830+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c15a540 con 0x7fb34c0c80d0 2023-10-06T05:29:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.830+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c0abc10 con 0x7fb34c0c80d0 2023-10-06T05:29:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.831+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c0ac0f0 con 0x7fb34c0c80d0 2023-10-06T05:29:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.832+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb338023070 con 0x7fb34c0c80d0 2023-10-06T05:29:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.833+0000 7fb348ff9700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.833+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb3380859e0 con 0x7fb34c0c80d0 2023-10-06T05:29:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.834+0000 7fb35099b700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.835+0000 7fb35099b700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb340007fd0 tx=0x7fb340007e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:20.838+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb338052a90 con 0x7fb34c0c80d0 2023-10-06T05:29:21.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.041+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb34c04c380 con 0x7fb34c0c80d0 2023-10-06T05:29:21.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.042+0000 7fb348ff9700 1 -- 172.21.15.33:0/404484921 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fb33802a090 con 0x7fb34c0c80d0 2023-10-06T05:29:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:21.045 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":209,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:19.601466+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:17.571482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:18.582693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:21.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 msgr2=0x7fb3300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:21.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb340007fd0 tx=0x7fb340007e70 comp rx=0 tx=0).stop 2023-10-06T05:29:21.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 msgr2=0x7fb34c1598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:21.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 0x7fb34c1598f0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fb3380094c0 tx=0x7fb338017000 comp rx=0 tx=0).stop 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 shutdown_connections 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34c159520 0x7fb34c15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb34c0c6340 0x7fb34c14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 --2- 172.21.15.33:0/404484921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb34c0c80d0 0x7fb34c1598f0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 >> 172.21.15.33:0/404484921 conn(0x7fb34c0cc930 msgr2=0x7fb34c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 shutdown_connections 2023-10-06T05:29:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.044+0000 7fb35199d700 1 -- 172.21.15.33:0/404484921 wait complete. 2023-10-06T05:29:21.065 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:29:21.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.258+0000 7f1135d9d700 1 Processor -- start 2023-10-06T05:29:21.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.258+0000 7f1135d9d700 1 -- start start 2023-10-06T05:29:21.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.259+0000 7f1135d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11300abbd0 0x7f1130067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:21.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f1135d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1130068140 0x7f1130159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:21.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f1135d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11300ac020 con 0x7f11300b1f90 2023-10-06T05:29:21.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f1135d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113006a2d0 con 0x7f1130068140 2023-10-06T05:29:21.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f1135d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1130154290 con 0x7f11300abbd0 2023-10-06T05:29:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f113559c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1130068140 0x7f1130159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f112ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11300abbd0 0x7f1130067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f1134d9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f11300b1f90 0x7f1130070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55120/0 (socket says 172.21.15.33:55120) 2023-10-06T05:29:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.260+0000 7f1134d9b700 1 -- 172.21.15.33:0/144861497 learned_addr learned my addr 172.21.15.33:0/144861497 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112f7fe700 1 -- 172.21.15.33:0/144861497 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778253227 0 0) 0x7f11300ac020 con 0x7f11300b1f90 2023-10-06T05:29:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112f7fe700 1 -- 172.21.15.33:0/144861497 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1120003570 con 0x7f11300b1f90 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112ffff700 1 -- 172.21.15.33:0/144861497 >> v2:172.21.15.167:3300/0 conn(0x7f1130068140 msgr2=0x7f1130159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112ffff700 1 --2- 172.21.15.33:0/144861497 >> v2:172.21.15.167:3300/0 conn(0x7f1130068140 0x7f1130159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112ffff700 1 -- 172.21.15.33:0/144861497 >> v1:172.21.15.167:6789/0 conn(0x7f11300b1f90 legacy=0x7f1130070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112ffff700 1 -- 172.21.15.33:0/144861497 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113006ae10 con 0x7f11300abbd0 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f113559c700 1 --2- 172.21.15.33:0/144861497 >> v2:172.21.15.167:3300/0 conn(0x7f1130068140 0x7f1130159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.261+0000 7f112ffff700 1 --2- 172.21.15.33:0/144861497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11300abbd0 0x7f1130067c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f111c0098e0 tx=0x7f111c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90983123ad4d3c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f112f7fe700 1 -- 172.21.15.33:0/144861497 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011630 con 0x7f11300abbd0 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f112f7fe700 1 -- 172.21.15.33:0/144861497 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c011790 con 0x7f11300abbd0 2023-10-06T05:29:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f112f7fe700 1 -- 172.21.15.33:0/144861497 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011b30 con 0x7f11300abbd0 2023-10-06T05:29:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 -- 172.21.15.33:0/144861497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11300abbd0 msgr2=0x7f1130067c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 --2- 172.21.15.33:0/144861497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11300abbd0 0x7f1130067c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f111c0098e0 tx=0x7f111c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 -- 172.21.15.33:0/144861497 shutdown_connections 2023-10-06T05:29:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 --2- 172.21.15.33:0/144861497 >> v2:172.21.15.167:3300/0 conn(0x7f1130068140 0x7f1130159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 --2- 172.21.15.33:0/144861497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11300abbd0 0x7f1130067c00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 -- 172.21.15.33:0/144861497 >> 172.21.15.33:0/144861497 conn(0x7f113006ea70 msgr2=0x7f113006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:21.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 -- 172.21.15.33:0/144861497 shutdown_connections 2023-10-06T05:29:21.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.262+0000 7f1135d9d700 1 -- 172.21.15.33:0/144861497 wait complete. 2023-10-06T05:29:21.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.264+0000 7f1135d9d700 1 Processor -- start 2023-10-06T05:29:21.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.264+0000 7f1135d9d700 1 -- start start 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1135d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1130068140 0x7f11300cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1135d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11300abbd0 0x7f113014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1135d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 0x7f113014b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1135d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130171110 con 0x7f11300b1f90 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1135d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130171f70 con 0x7f11300abbd0 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1135d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1130171360 con 0x7f1130068140 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f113559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 0x7f113014b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f113559c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 0x7f113014b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48836/0 (socket says 172.21.15.33:48836) 2023-10-06T05:29:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f1134d9b700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1130068140 0x7f11300cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f113559c700 1 -- 172.21.15.33:0/2773412110 learned_addr learned my addr 172.21.15.33:0/2773412110 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.265+0000 7f112ffff700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11300abbd0 0x7f113014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f113559c700 1 -- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1130068140 msgr2=0x7f11300cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f113559c700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1130068140 0x7f11300cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f113559c700 1 -- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11300abbd0 msgr2=0x7f113014d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f113559c700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11300abbd0 0x7f113014d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f113559c700 1 -- 172.21.15.33:0/2773412110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128009a20 con 0x7f11300b1f90 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f113559c700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 0x7f113014b5d0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f1128009f50 tx=0x7f112800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800cb50 con 0x7f11300b1f90 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800f040 con 0x7f11300b1f90 2023-10-06T05:29:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128014480 con 0x7f11300b1f90 2023-10-06T05:29:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c009420 con 0x7f11300b1f90 2023-10-06T05:29:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.266+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1130171920 con 0x7f11300b1f90 2023-10-06T05:29:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.267+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113004c380 con 0x7f11300b1f90 2023-10-06T05:29:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.268+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1128003020 con 0x7f11300b1f90 2023-10-06T05:29:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.270+0000 7f112cff9700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1114066f60 0x7f1114069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.270+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1128084f40 con 0x7f11300b1f90 2023-10-06T05:29:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.270+0000 7f1134d9b700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1114066f60 0x7f1114069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.271+0000 7f1134d9b700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1114066f60 0x7f1114069410 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1124007fd0 tx=0x7f1124009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.274+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1128051fb0 con 0x7f11300b1f90 2023-10-06T05:29:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:21.521+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f11301720b0 con 0x7f11300b1f90 2023-10-06T05:29:22.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.149+0000 7f112cff9700 1 -- 172.21.15.33:0/2773412110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v161) v1 ==== 118+0+0 (secure 0 0 0) 0x7f11301720b0 con 0x7f11300b1f90 2023-10-06T05:29:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.152+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1114066f60 msgr2=0x7f1114069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:22.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.152+0000 7f1135d9d700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1114066f60 0x7f1114069410 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1124007fd0 tx=0x7f1124009040 comp rx=0 tx=0).stop 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.152+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 msgr2=0x7f113014b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.152+0000 7f1135d9d700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 0x7f113014b5d0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f1128009f50 tx=0x7f112800b040 comp rx=0 tx=0).stop 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 shutdown_connections 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1114066f60 0x7f1114069410 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11300b1f90 0x7f113014b5d0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1130068140 0x7f11300cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 --2- 172.21.15.33:0/2773412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11300abbd0 0x7f113014d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 >> 172.21.15.33:0/2773412110 conn(0x7f113006ea70 msgr2=0x7f113006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:22.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 shutdown_connections 2023-10-06T05:29:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.153+0000 7f1135d9d700 1 -- 172.21.15.33:0/2773412110 wait complete. 2023-10-06T05:29:22.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:29:22.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.367+0000 7f6eccb9d700 1 Processor -- start 2023-10-06T05:29:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.367+0000 7f6eccb9d700 1 -- start start 2023-10-06T05:29:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.368+0000 7f6eccb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ec8156370 0x7f6ec80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.368+0000 7f6eccb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80b01f0 0x7f6ec80b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.368+0000 7f6eccb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80ad430 con 0x7f6ec80cc7e0 2023-10-06T05:29:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.368+0000 7f6eccb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec80afe20 con 0x7f6ec8156370 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.368+0000 7f6eccb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80b0b70 con 0x7f6ec80b01f0 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.368+0000 7f6ec77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ec8156370 0x7f6ec80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ec8156370 0x7f6ec80af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48846/0 (socket says 172.21.15.33:48846) 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 -- 172.21.15.33:0/1941039517 learned_addr learned my addr 172.21.15.33:0/1941039517 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec7fff700 1 --2- 172.21.15.33:0/1941039517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80b01f0 0x7f6ec80b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 -- 172.21.15.33:0/1941039517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80b01f0 msgr2=0x7f6ec80b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 --2- 172.21.15.33:0/1941039517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80b01f0 0x7f6ec80b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 -- 172.21.15.33:0/1941039517 >> v1:172.21.15.167:6789/0 conn(0x7f6ec80cc7e0 legacy=0x7f6ec80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 -- 172.21.15.33:0/1941039517 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec806bc70 con 0x7f6ec8156370 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec77fe700 1 --2- 172.21.15.33:0/1941039517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec80af8e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80099b0 tx=0x7f6eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55af11525a7997d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:22.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.369+0000 7f6ec67fc700 1 -- 172.21.15.33:0/1941039517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800b860 con 0x7f6ec8156370 2023-10-06T05:29:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6ec67fc700 1 -- 172.21.15.33:0/1941039517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb800b9c0 con 0x7f6ec8156370 2023-10-06T05:29:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6ec67fc700 1 -- 172.21.15.33:0/1941039517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800bd60 con 0x7f6ec8156370 2023-10-06T05:29:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 -- 172.21.15.33:0/1941039517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 msgr2=0x7f6ec80af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/1941039517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec80af8e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80099b0 tx=0x7f6eb8009250 comp rx=0 tx=0).stop 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 -- 172.21.15.33:0/1941039517 shutdown_connections 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/1941039517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec80af8e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/1941039517 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80b01f0 0x7f6ec80b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 -- 172.21.15.33:0/1941039517 >> 172.21.15.33:0/1941039517 conn(0x7f6ec806d030 msgr2=0x7f6ec80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 -- 172.21.15.33:0/1941039517 shutdown_connections 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.370+0000 7f6eccb9d700 1 -- 172.21.15.33:0/1941039517 wait complete. 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.372+0000 7f6eccb9d700 1 Processor -- start 2023-10-06T05:29:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.372+0000 7f6eccb9d700 1 -- start start 2023-10-06T05:29:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6eccb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ec80b01f0 0x7f6ec814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6eccb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80cc7e0 0x7f6ec814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6eccb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6eccb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80a5590 con 0x7f6ec8156370 2023-10-06T05:29:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6eccb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec81548f0 con 0x7f6ec80b01f0 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6eccb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8169760 con 0x7f6ec80cc7e0 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6ec7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6ec7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec814fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48848/0 (socket says 172.21.15.33:48848) 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6ec7fff700 1 -- 172.21.15.33:0/2940464991 learned_addr learned my addr 172.21.15.33:0/2940464991 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6ec77fe700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ec80b01f0 0x7f6ec814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.373+0000 7f6ec6ffd700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80cc7e0 0x7f6ec814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6ec7fff700 1 -- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80cc7e0 msgr2=0x7f6ec814f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6ec7fff700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80cc7e0 0x7f6ec814f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6ec7fff700 1 -- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ec80b01f0 msgr2=0x7f6ec814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6ec7fff700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ec80b01f0 0x7f6ec814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6ec7fff700 1 -- 172.21.15.33:0/2940464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80094f0 con 0x7f6ec8156370 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6ec7fff700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec814fc10 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0007f50 tx=0x7f6ec0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000eba0 con 0x7f6ec8156370 2023-10-06T05:29:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec000ed20 con 0x7f6ec8156370 2023-10-06T05:29:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0014670 con 0x7f6ec8156370 2023-10-06T05:29:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8169150 con 0x7f6ec8156370 2023-10-06T05:29:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.374+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec8154610 con 0x7f6ec8156370 2023-10-06T05:29:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.376+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec8168fb0 con 0x7f6ec8156370 2023-10-06T05:29:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.376+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ec00149e0 con 0x7f6ec8156370 2023-10-06T05:29:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.377+0000 7f6eaf7fe700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6eb4066f30 0x7f6eb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.377+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ec0084d70 con 0x7f6ec8156370 2023-10-06T05:29:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.378+0000 7f6ec77fe700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6eb4066f30 0x7f6eb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.378+0000 7f6ec77fe700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6eb4066f30 0x7f6eb40693e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80099b0 tx=0x7f6eb8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.381+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ec8168fb0 con 0x7f6ec8156370 2023-10-06T05:29:22.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:22.628+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6ec8168fb0 con 0x7f6ec8156370 2023-10-06T05:29:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.151+0000 7f6eaf7fe700 1 -- 172.21.15.33:0/2940464991 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v162) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ec8168fb0 con 0x7f6ec8156370 2023-10-06T05:29:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6eb4066f30 msgr2=0x7f6eb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6eb4066f30 0x7f6eb40693e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80099b0 tx=0x7f6eb8017000 comp rx=0 tx=0).stop 2023-10-06T05:29:23.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 msgr2=0x7f6ec814fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec814fc10 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0007f50 tx=0x7f6ec0009860 comp rx=0 tx=0).stop 2023-10-06T05:29:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 shutdown_connections 2023-10-06T05:29:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6eb4066f30 0x7f6eb40693e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ec8156370 0x7f6ec814fc10 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ec80b01f0 0x7f6ec814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 --2- 172.21.15.33:0/2940464991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ec80cc7e0 0x7f6ec814f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 >> 172.21.15.33:0/2940464991 conn(0x7f6ec806d030 msgr2=0x7f6ec806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.154+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 shutdown_connections 2023-10-06T05:29:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.155+0000 7f6eccb9d700 1 -- 172.21.15.33:0/2940464991 wait complete. 2023-10-06T05:29:23.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:23.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.366+0000 7f1ed93ff700 1 Processor -- start 2023-10-06T05:29:23.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.367+0000 7f1ed93ff700 1 -- start start 2023-10-06T05:29:23.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed93ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 0x7f1ed40c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed93ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1ed406a160 0x7f1ed40c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed93ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed415b0c0 con 0x7f1ed4157370 2023-10-06T05:29:23.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed93ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed415c0d0 con 0x7f1ed406a160 2023-10-06T05:29:23.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed93ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed415bc40 con 0x7f1ed4157740 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed27fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 0x7f1ed40c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed27fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 0x7f1ed40c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:58698/0 (socket says 172.21.15.33:58698) 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed27fc700 1 -- 172.21.15.33:0/434254555 learned_addr learned my addr 172.21.15.33:0/434254555 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed2ffd700 1 --1- 172.21.15.33:0/434254555 >> v1:172.21.15.167:6789/0 conn(0x7f1ed4157370 0x7f1ed4069a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:55142/0 (socket says 172.21.15.33:55142) 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.368+0000 7f1ed37fe700 1 --2- 172.21.15.33:0/434254555 >> v2:172.21.15.167:3300/0 conn(0x7f1ed406a160 0x7f1ed40c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.369+0000 7f1ed27fc700 1 -- 172.21.15.33:0/434254555 >> v2:172.21.15.167:3300/0 conn(0x7f1ed406a160 msgr2=0x7f1ed40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.369+0000 7f1ed27fc700 1 --2- 172.21.15.33:0/434254555 >> v2:172.21.15.167:3300/0 conn(0x7f1ed406a160 0x7f1ed40c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.369+0000 7f1ed27fc700 1 -- 172.21.15.33:0/434254555 >> v1:172.21.15.167:6789/0 conn(0x7f1ed4157370 legacy=0x7f1ed4069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.369+0000 7f1ed27fc700 1 -- 172.21.15.33:0/434254555 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40c95a0 con 0x7f1ed4157740 2023-10-06T05:29:23.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.369+0000 7f1ed1ffb700 1 -- 172.21.15.33:0/434254555 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388027373 0 0) 0x7f1ed415b0c0 con 0x7f1ed4157370 2023-10-06T05:29:23.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.369+0000 7f1ed27fc700 1 --2- 172.21.15.33:0/434254555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 0x7f1ed40c8b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80099b0 tx=0x7f1ec8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85e7f5a067f9e81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed1ffb700 1 -- 172.21.15.33:0/434254555 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8009ea0 con 0x7f1ed4157740 2023-10-06T05:29:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed1ffb700 1 -- 172.21.15.33:0/434254555 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800b860 con 0x7f1ed4157740 2023-10-06T05:29:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed1ffb700 1 -- 172.21.15.33:0/434254555 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800bc00 con 0x7f1ed4157740 2023-10-06T05:29:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed93ff700 1 -- 172.21.15.33:0/434254555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 msgr2=0x7f1ed40c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/434254555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 0x7f1ed40c8b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80099b0 tx=0x7f1ec8009250 comp rx=0 tx=0).stop 2023-10-06T05:29:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed93ff700 1 -- 172.21.15.33:0/434254555 shutdown_connections 2023-10-06T05:29:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/434254555 >> v2:172.21.15.167:3300/0 conn(0x7f1ed406a160 0x7f1ed40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/434254555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed4157740 0x7f1ed40c8b20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.370+0000 7f1ed93ff700 1 -- 172.21.15.33:0/434254555 >> 172.21.15.33:0/434254555 conn(0x7f1ed40675f0 msgr2=0x7f1ed40679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.371+0000 7f1ed93ff700 1 -- 172.21.15.33:0/434254555 shutdown_connections 2023-10-06T05:29:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.371+0000 7f1ed93ff700 1 -- 172.21.15.33:0/434254555 wait complete. 2023-10-06T05:29:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.372+0000 7f1ed93ff700 1 Processor -- start 2023-10-06T05:29:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.372+0000 7f1ed93ff700 1 -- start start 2023-10-06T05:29:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed93ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed406a160 0x7f1ed4149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed93ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ed4157370 0x7f1ed416a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed93ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 0x7f1ed4155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed93ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40c6060 con 0x7f1ed4157740 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed93ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed416a9e0 con 0x7f1ed4157370 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed93ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed414a4a0 con 0x7f1ed406a160 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 0x7f1ed4155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 0x7f1ed4155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48850/0 (socket says 172.21.15.33:48850) 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.373+0000 7f1ed37fe700 1 -- 172.21.15.33:0/3604808596 learned_addr learned my addr 172.21.15.33:0/3604808596 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed2ffd700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed406a160 0x7f1ed4149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed27fc700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ed4157370 0x7f1ed416a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed37fe700 1 -- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed406a160 msgr2=0x7f1ed4149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed37fe700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed406a160 0x7f1ed4149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed37fe700 1 -- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ed4157370 msgr2=0x7f1ed416a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed37fe700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ed4157370 0x7f1ed416a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed37fe700 1 -- 172.21.15.33:0/3604808596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80094f0 con 0x7f1ed4157740 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed27fc700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ed4157370 0x7f1ed416a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed2ffd700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed406a160 0x7f1ed4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed37fe700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 0x7f1ed4155e70 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009ec0 tx=0x7f1ecc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00cc20 con 0x7f1ed4157740 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc00f040 con 0x7f1ed4157740 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc014480 con 0x7f1ed4157740 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.374+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed40c5450 con 0x7f1ed4157740 2023-10-06T05:29:23.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.375+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed40c5a10 con 0x7f1ed4157740 2023-10-06T05:29:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.376+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed4063ff0 con 0x7f1ed4157740 2023-10-06T05:29:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.377+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ecc014810 con 0x7f1ed4157740 2023-10-06T05:29:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.378+0000 7f1ebaffd700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ec0066f30 0x7f1ec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.378+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1ecc0851e0 con 0x7f1ed4157740 2023-10-06T05:29:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.378+0000 7f1ed2ffd700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ec0066f30 0x7f1ec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.379+0000 7f1ed2ffd700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ec0066f30 0x7f1ec00693e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4153310 tx=0x7f1ec4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:23.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.381+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ecc052250 con 0x7f1ed4157740 2023-10-06T05:29:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.626+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed40c61a0 con 0x7f1ed4157740 2023-10-06T05:29:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.627+0000 7f1ebaffd700 1 -- 172.21.15.33:0/3604808596 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f1ed40c61a0 con 0x7f1ed4157740 2023-10-06T05:29:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:23.630 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":162,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1335406981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1335406981},{"type":"v1","addr":"172.21.15.167:6836","nonce":1335406981}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157},{"gid":9463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1379577645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1379577645},{"type":"v1","addr":"172.21.15.33:6835","nonce":1379577645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157},{"gid":9469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1485763855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1485763855},{"type":"v1","addr":"172.21.15.167:6837","nonce":1485763855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:23.149786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10419":{"gid":10419,"name":"a","rank":0,"incarnation":158,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/963232630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":963232630},{"type":"v1","addr":"172.21.15.33:6834","nonce":963232630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ec0066f30 msgr2=0x7f1ec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ec0066f30 0x7f1ec00693e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4153310 tx=0x7f1ec4009040 comp rx=0 tx=0).stop 2023-10-06T05:29:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 msgr2=0x7f1ed4155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 0x7f1ed4155e70 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009ec0 tx=0x7f1ecc00b040 comp rx=0 tx=0).stop 2023-10-06T05:29:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 shutdown_connections 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1ec0066f30 0x7f1ec00693e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ed4157740 0x7f1ed4155e70 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ed406a160 0x7f1ed4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 --2- 172.21.15.33:0/3604808596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ed4157370 0x7f1ed416a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.629+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 >> 172.21.15.33:0/3604808596 conn(0x7f1ed40675f0 msgr2=0x7f1ed40c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.630+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 shutdown_connections 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.630+0000 7f1ed93ff700 1 -- 172.21.15.33:0/3604808596 wait complete. 2023-10-06T05:29:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 162 2023-10-06T05:29:23.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.834+0000 7ff28579d700 1 Processor -- start 2023-10-06T05:29:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.834+0000 7ff28579d700 1 -- start start 2023-10-06T05:29:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff28579d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff2800c8050 0x7ff28013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff28579d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff28006aad0 0x7ff28006af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff28579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800c9cb0 con 0x7ff28013faa0 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff28579d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2800ca270 con 0x7ff28006aad0 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff28579d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28006b840 con 0x7ff2800c8050 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff27ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff28013faa0 0x7ff2800c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58322/0 (socket says 172.21.15.33:58322) 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff27ffff700 1 -- 172.21.15.33:0/1594349469 learned_addr learned my addr 172.21.15.33:0/1594349469 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff284f9c700 1 --2- 172.21.15.33:0/1594349469 >> v2:172.21.15.33:3300/0 conn(0x7ff28006aad0 0x7ff28006af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff27f7fe700 1 --2- 172.21.15.33:0/1594349469 >> v2:172.21.15.167:3300/0 conn(0x7ff2800c8050 0x7ff28013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff27effd700 1 -- 172.21.15.33:0/1594349469 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274639380 0 0) 0x7ff2800c9cb0 con 0x7ff28013faa0 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.836+0000 7ff27effd700 1 -- 172.21.15.33:0/1594349469 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff268003570 con 0x7ff28013faa0 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff284f9c700 1 -- 172.21.15.33:0/1594349469 >> v2:172.21.15.167:3300/0 conn(0x7ff2800c8050 msgr2=0x7ff28013fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff284f9c700 1 --2- 172.21.15.33:0/1594349469 >> v2:172.21.15.167:3300/0 conn(0x7ff2800c8050 0x7ff28013fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff284f9c700 1 -- 172.21.15.33:0/1594349469 >> v1:172.21.15.33:6789/0 conn(0x7ff28013faa0 legacy=0x7ff2800c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff284f9c700 1 -- 172.21.15.33:0/1594349469 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28006bb20 con 0x7ff28006aad0 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff284f9c700 1 --2- 172.21.15.33:0/1594349469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff28006af30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7ff2800cbef0 tx=0x7ff278009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4528178d845e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff27effd700 1 -- 172.21.15.33:0/1594349469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800e070 con 0x7ff28006aad0 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff27effd700 1 -- 172.21.15.33:0/1594349469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800bab0 con 0x7ff28006aad0 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.837+0000 7ff27effd700 1 -- 172.21.15.33:0/1594349469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800be70 con 0x7ff28006aad0 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 -- 172.21.15.33:0/1594349469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 msgr2=0x7ff28006af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 --2- 172.21.15.33:0/1594349469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff28006af30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7ff2800cbef0 tx=0x7ff278009780 comp rx=0 tx=0).stop 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 -- 172.21.15.33:0/1594349469 shutdown_connections 2023-10-06T05:29:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 --2- 172.21.15.33:0/1594349469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff28006af30 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 --2- 172.21.15.33:0/1594349469 >> v2:172.21.15.167:3300/0 conn(0x7ff2800c8050 0x7ff28013fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 -- 172.21.15.33:0/1594349469 >> 172.21.15.33:0/1594349469 conn(0x7ff28006d030 msgr2=0x7ff28006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 -- 172.21.15.33:0/1594349469 shutdown_connections 2023-10-06T05:29:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.838+0000 7ff28579d700 1 -- 172.21.15.33:0/1594349469 wait complete. 2023-10-06T05:29:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.839+0000 7ff28579d700 1 Processor -- start 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.839+0000 7ff28579d700 1 -- start start 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff28579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff2800c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff28579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2800c8050 0x7ff28013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff28579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff28013faa0 0x7ff2800c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff28579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c5b30 con 0x7ff28006aad0 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff28579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c6500 con 0x7ff28013faa0 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff28579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c5260 con 0x7ff2800c8050 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.840+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff2800c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff2800c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48862/0 (socket says 172.21.15.33:48862) 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 -- 172.21.15.33:0/1988543846 learned_addr learned my addr 172.21.15.33:0/1988543846 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 -- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2800c8050 msgr2=0x7ff28013ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2800c8050 0x7ff28013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 -- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff28013faa0 msgr2=0x7ff2800c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff28013faa0 0x7ff2800c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 -- 172.21.15.33:0/1988543846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009a20 con 0x7ff28006aad0 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff27ffff700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff2800c5fc0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c5c0 tx=0x7ff270007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270005b30 con 0x7ff28006aad0 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270005cb0 con 0x7ff28006aad0 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270007320 con 0x7ff28006aad0 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.841+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2800afee0 con 0x7ff28006aad0 2023-10-06T05:29:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.842+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2800c57c0 con 0x7ff28006aad0 2023-10-06T05:29:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.842+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28004c380 con 0x7ff28006aad0 2023-10-06T05:29:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.844+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff27001b3b0 con 0x7ff28006aad0 2023-10-06T05:29:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.844+0000 7ff267fff700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff26c066f30 0x7ff26c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.845+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff270083cd0 con 0x7ff28006aad0 2023-10-06T05:29:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.845+0000 7ff27f7fe700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff26c066f30 0x7ff26c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.845+0000 7ff27f7fe700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff26c066f30 0x7ff26c0693e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff2800b1810 tx=0x7ff2740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:23.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:23.849+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff270050d40 con 0x7ff28006aad0 2023-10-06T05:29:24.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.055+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2800c9f00 con 0x7ff28006aad0 2023-10-06T05:29:24.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.056+0000 7ff267fff700 1 -- 172.21.15.33:0/1988543846 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+16880 (secure 0 0 0) 0x7ff270017020 con 0x7ff28006aad0 2023-10-06T05:29:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:24.059 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:21.116935+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:17.571482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:18.582693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:24.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff26c066f30 msgr2=0x7ff26c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff26c066f30 0x7ff26c0693e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff2800b1810 tx=0x7ff2740091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 msgr2=0x7ff2800c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff2800c5fc0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c5c0 tx=0x7ff270007e00 comp rx=0 tx=0).stop 2023-10-06T05:29:24.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 shutdown_connections 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff26c066f30 0x7ff26c0693e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff28013faa0 0x7ff2800c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff28006aad0 0x7ff2800c5fc0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 --2- 172.21.15.33:0/1988543846 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2800c8050 0x7ff28013ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 >> 172.21.15.33:0/1988543846 conn(0x7ff28006d030 msgr2=0x7ff2800c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 shutdown_connections 2023-10-06T05:29:24.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.058+0000 7ff28579d700 1 -- 172.21.15.33:0/1988543846 wait complete. 2023-10-06T05:29:24.079 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:29:24.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:29:24.085 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:29:24.082+0000 7f36673f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:29:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.265+0000 7fe143fff700 1 Processor -- start 2023-10-06T05:29:24.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.265+0000 7fe143fff700 1 -- start start 2023-10-06T05:29:24.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.266+0000 7fe143fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.266+0000 7fe143fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1440cc750 0x7fe1440b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.266+0000 7fe143fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440cbd80 con 0x7fe1440cf3b0 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.266+0000 7fe143fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440af5f0 con 0x7fe1440cc750 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.266+0000 7fe143fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440af8d0 con 0x7fe1440ccfc0 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1440cc750 0x7fe1440b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1440cc750 0x7fe1440b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48878/0 (socket says 172.21.15.33:48878) 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 -- 172.21.15.33:0/1242659645 learned_addr learned my addr 172.21.15.33:0/1242659645 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe142ffd700 1 --2- 172.21.15.33:0/1242659645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 -- 172.21.15.33:0/1242659645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 msgr2=0x7fe1440cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:24.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 --2- 172.21.15.33:0/1242659645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 -- 172.21.15.33:0/1242659645 >> v1:172.21.15.167:6789/0 conn(0x7fe1440cf3b0 legacy=0x7fe1441594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 -- 172.21.15.33:0/1242659645 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440afbb0 con 0x7fe1440cc750 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.267+0000 7fe1437fe700 1 --2- 172.21.15.33:0/1242659645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440cc750 0x7fe1440b0460 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009f20 tx=0x7fe13c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d59bf99c917666 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe141ffb700 1 -- 172.21.15.33:0/1242659645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00e070 con 0x7fe1440cc750 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe141ffb700 1 -- 172.21.15.33:0/1242659645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c00ba80 con 0x7fe1440cc750 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe141ffb700 1 -- 172.21.15.33:0/1242659645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0092e0 con 0x7fe1440cc750 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 -- 172.21.15.33:0/1242659645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440cc750 msgr2=0x7fe1440b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 --2- 172.21.15.33:0/1242659645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440cc750 0x7fe1440b0460 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009f20 tx=0x7fe13c009840 comp rx=0 tx=0).stop 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 -- 172.21.15.33:0/1242659645 shutdown_connections 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 --2- 172.21.15.33:0/1242659645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440cc750 0x7fe1440b0460 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 --2- 172.21.15.33:0/1242659645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 -- 172.21.15.33:0/1242659645 >> 172.21.15.33:0/1242659645 conn(0x7fe14406cff0 msgr2=0x7fe14406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:24.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.268+0000 7fe143fff700 1 -- 172.21.15.33:0/1242659645 shutdown_connections 2023-10-06T05:29:24.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.269+0000 7fe143fff700 1 -- 172.21.15.33:0/1242659645 wait complete. 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.270+0000 7fe143fff700 1 Processor -- start 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.270+0000 7fe143fff700 1 -- start start 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe143fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1440cc750 0x7fe1441537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe143fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe143fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 0x7fe1440b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe143fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1441699e0 con 0x7fe1440c6a50 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe143fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c6ed0 con 0x7fe1440cc750 2023-10-06T05:29:24.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe143fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c75b0 con 0x7fe1440ccfc0 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe1437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 0x7fe1440b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe1437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 0x7fe1440b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48886/0 (socket says 172.21.15.33:48886) 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe1437fe700 1 -- 172.21.15.33:0/4091726618 learned_addr learned my addr 172.21.15.33:0/4091726618 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe142ffd700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1440cc750 0x7fe1441537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.271+0000 7fe1427fc700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe1437fe700 1 -- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 msgr2=0x7fe1440c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe1437fe700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe1437fe700 1 -- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1440cc750 msgr2=0x7fe1441537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe1437fe700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1440cc750 0x7fe1441537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe1437fe700 1 -- 172.21.15.33:0/4091726618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c009ae0 con 0x7fe1440c6a50 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe1437fe700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 0x7fe1440b4680 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b6c0 tx=0x7fe13c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:24.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0032f0 con 0x7fe1440c6a50 2023-10-06T05:29:24.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c008ce0 con 0x7fe1440c6a50 2023-10-06T05:29:24.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440c7890 con 0x7fe1440c6a50 2023-10-06T05:29:24.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c026670 con 0x7fe1440c6a50 2023-10-06T05:29:24.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.272+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1440c7e50 con 0x7fe1440c6a50 2023-10-06T05:29:24.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.273+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe144156b20 con 0x7fe1440c6a50 2023-10-06T05:29:24.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.274+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe13c003040 con 0x7fe1440c6a50 2023-10-06T05:29:24.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.275+0000 7fe12affd700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe130066f30 0x7fe1300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.275+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe13c0866c0 con 0x7fe1440c6a50 2023-10-06T05:29:24.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.276+0000 7fe142ffd700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe130066f30 0x7fe1300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.276+0000 7fe142ffd700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe130066f30 0x7fe1300693e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe134006fd0 tx=0x7fe134008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:24.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.280+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe13c053730 con 0x7fe1440c6a50 2023-10-06T05:29:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.478+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe144159bb0 con 0x7fe130066f30 2023-10-06T05:29:24.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.488+0000 7fe12affd700 1 -- 172.21.15.33:0/4091726618 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311324 (secure 0 0 0) 0x7fe144159bb0 con 0x7fe130066f30 2023-10-06T05:29:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe130066f30 msgr2=0x7fe1300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe130066f30 0x7fe1300693e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe134006fd0 tx=0x7fe134008040 comp rx=0 tx=0).stop 2023-10-06T05:29:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 msgr2=0x7fe1440b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 0x7fe1440b4680 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b6c0 tx=0x7fe13c018000 comp rx=0 tx=0).stop 2023-10-06T05:29:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 shutdown_connections 2023-10-06T05:29:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe130066f30 0x7fe1300693e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1440c6a50 0x7fe1440b4680 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1440cc750 0x7fe1441537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 --2- 172.21.15.33:0/4091726618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1440ccfc0 0x7fe1440c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.492+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 >> 172.21.15.33:0/4091726618 conn(0x7fe14406cff0 msgr2=0x7fe144156050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.493+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 shutdown_connections 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.493+0000 7fe143fff700 1 -- 172.21.15.33:0/4091726618 wait complete. 2023-10-06T05:29:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:29:24.513 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":659,"stamp":"2023-10-06T05:29:24.044411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432996,"kb_used_data":21924,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752541724,"statfs":{"total":773094113280,"available":770602725376,"internally_reserved":0,"allocated":22450176,"data_stored":11081385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.941597"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.151677+0000","last_change":"2023-10-06T05:29:20.148259+0000","last_active":"2023-10-06T05:29:20.151677+0000","last_peered":"2023-10-06T05:29:20.151677+0000","last_clean":"2023-10-06T05:29:20.151677+0000","last_became_active":"2023-10-06T05:29:20.147981+0000","last_became_peered":"2023-10-06T05:29:20.147981+0000","last_unstale":"2023-10-06T05:29:20.151677+0000","last_undegraded":"2023-10-06T05:29:20.151677+0000","last_fullsized":"2023-10-06T05:29:20.151677+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:39.748897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157534+0000","last_change":"2023-10-06T05:29:20.149065+0000","last_active":"2023-10-06T05:29:20.157534+0000","last_peered":"2023-10-06T05:29:20.157534+0000","last_clean":"2023-10-06T05:29:20.157534+0000","last_became_active":"2023-10-06T05:29:20.148813+0000","last_became_peered":"2023-10-06T05:29:20.148813+0000","last_unstale":"2023-10-06T05:29:20.157534+0000","last_undegraded":"2023-10-06T05:29:20.157534+0000","last_fullsized":"2023-10-06T05:29:20.157534+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:33.221702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152168+0000","last_change":"2023-10-06T05:29:20.144003+0000","last_active":"2023-10-06T05:29:20.152168+0000","last_peered":"2023-10-06T05:29:20.152168+0000","last_clean":"2023-10-06T05:29:20.152168+0000","last_became_active":"2023-10-06T05:29:20.143694+0000","last_became_peered":"2023-10-06T05:29:20.143694+0000","last_unstale":"2023-10-06T05:29:20.152168+0000","last_undegraded":"2023-10-06T05:29:20.152168+0000","last_fullsized":"2023-10-06T05:29:20.152168+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:13.493549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152153+0000","last_change":"2023-10-06T05:29:20.147626+0000","last_active":"2023-10-06T05:29:20.152153+0000","last_peered":"2023-10-06T05:29:20.152153+0000","last_clean":"2023-10-06T05:29:20.152153+0000","last_became_active":"2023-10-06T05:29:20.147362+0000","last_became_peered":"2023-10-06T05:29:20.147362+0000","last_unstale":"2023-10-06T05:29:20.152153+0000","last_undegraded":"2023-10-06T05:29:20.152153+0000","last_fullsized":"2023-10-06T05:29:20.152153+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:28.292786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148670+0000","last_change":"2023-10-06T05:29:20.142749+0000","last_active":"2023-10-06T05:29:20.148670+0000","last_peered":"2023-10-06T05:29:20.148670+0000","last_clean":"2023-10-06T05:29:20.148670+0000","last_became_active":"2023-10-06T05:29:20.141280+0000","last_became_peered":"2023-10-06T05:29:20.141280+0000","last_unstale":"2023-10-06T05:29:20.148670+0000","last_undegraded":"2023-10-06T05:29:20.148670+0000","last_fullsized":"2023-10-06T05:29:20.148670+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:43.968575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142259+0000","last_change":"2023-10-06T05:29:20.129470+0000","last_active":"2023-10-06T05:29:20.142259+0000","last_peered":"2023-10-06T05:29:20.142259+0000","last_clean":"2023-10-06T05:29:20.142259+0000","last_became_active":"2023-10-06T05:29:20.129135+0000","last_became_peered":"2023-10-06T05:29:20.129135+0000","last_unstale":"2023-10-06T05:29:20.142259+0000","last_undegraded":"2023-10-06T05:29:20.142259+0000","last_fullsized":"2023-10-06T05:29:20.142259+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:29.764934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142843+0000","last_change":"2023-10-06T05:29:20.140046+0000","last_active":"2023-10-06T05:29:20.142843+0000","last_peered":"2023-10-06T05:29:20.142843+0000","last_clean":"2023-10-06T05:29:20.142843+0000","last_became_active":"2023-10-06T05:29:20.139386+0000","last_became_peered":"2023-10-06T05:29:20.139386+0000","last_unstale":"2023-10-06T05:29:20.142843+0000","last_undegraded":"2023-10-06T05:29:20.142843+0000","last_fullsized":"2023-10-06T05:29:20.142843+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:23.361080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134527+0000","last_change":"2023-10-06T05:29:20.133263+0000","last_active":"2023-10-06T05:29:20.134527+0000","last_peered":"2023-10-06T05:29:20.134527+0000","last_clean":"2023-10-06T05:29:20.134527+0000","last_became_active":"2023-10-06T05:29:20.132987+0000","last_became_peered":"2023-10-06T05:29:20.132987+0000","last_unstale":"2023-10-06T05:29:20.134527+0000","last_undegraded":"2023-10-06T05:29:20.134527+0000","last_fullsized":"2023-10-06T05:29:20.134527+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:25.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142417+0000","last_change":"2023-10-06T05:29:20.136532+0000","last_active":"2023-10-06T05:29:20.142417+0000","last_peered":"2023-10-06T05:29:20.142417+0000","last_clean":"2023-10-06T05:29:20.142417+0000","last_became_active":"2023-10-06T05:29:20.130776+0000","last_became_peered":"2023-10-06T05:29:20.130776+0000","last_unstale":"2023-10-06T05:29:20.142417+0000","last_undegraded":"2023-10-06T05:29:20.142417+0000","last_fullsized":"2023-10-06T05:29:20.142417+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:48.943820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155721+0000","last_change":"2023-10-06T05:29:20.150764+0000","last_active":"2023-10-06T05:29:20.155721+0000","last_peered":"2023-10-06T05:29:20.155721+0000","last_clean":"2023-10-06T05:29:20.155721+0000","last_became_active":"2023-10-06T05:29:20.150512+0000","last_became_peered":"2023-10-06T05:29:20.150512+0000","last_unstale":"2023-10-06T05:29:20.155721+0000","last_undegraded":"2023-10-06T05:29:20.155721+0000","last_fullsized":"2023-10-06T05:29:20.155721+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:11.016883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155696+0000","last_change":"2023-10-06T05:29:20.149546+0000","last_active":"2023-10-06T05:29:20.155696+0000","last_peered":"2023-10-06T05:29:20.155696+0000","last_clean":"2023-10-06T05:29:20.155696+0000","last_became_active":"2023-10-06T05:29:20.149295+0000","last_became_peered":"2023-10-06T05:29:20.149295+0000","last_unstale":"2023-10-06T05:29:20.155696+0000","last_undegraded":"2023-10-06T05:29:20.155696+0000","last_fullsized":"2023-10-06T05:29:20.155696+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.899449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148119+0000","last_change":"2023-10-06T05:29:20.142968+0000","last_active":"2023-10-06T05:29:20.148119+0000","last_peered":"2023-10-06T05:29:20.148119+0000","last_clean":"2023-10-06T05:29:20.148119+0000","last_became_active":"2023-10-06T05:29:20.142709+0000","last_became_peered":"2023-10-06T05:29:20.142709+0000","last_unstale":"2023-10-06T05:29:20.148119+0000","last_undegraded":"2023-10-06T05:29:20.148119+0000","last_fullsized":"2023-10-06T05:29:20.148119+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:48.713611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.153048+0000","last_change":"2023-10-06T05:29:20.148877+0000","last_active":"2023-10-06T05:29:20.153048+0000","last_peered":"2023-10-06T05:29:20.153048+0000","last_clean":"2023-10-06T05:29:20.153048+0000","last_became_active":"2023-10-06T05:29:20.148597+0000","last_became_peered":"2023-10-06T05:29:20.148597+0000","last_unstale":"2023-10-06T05:29:20.153048+0000","last_undegraded":"2023-10-06T05:29:20.153048+0000","last_fullsized":"2023-10-06T05:29:20.153048+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:48.156668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144406+0000","last_change":"2023-10-06T05:29:20.139828+0000","last_active":"2023-10-06T05:29:20.144406+0000","last_peered":"2023-10-06T05:29:20.144406+0000","last_clean":"2023-10-06T05:29:20.144406+0000","last_became_active":"2023-10-06T05:29:20.139490+0000","last_became_peered":"2023-10-06T05:29:20.139490+0000","last_unstale":"2023-10-06T05:29:20.144406+0000","last_undegraded":"2023-10-06T05:29:20.144406+0000","last_fullsized":"2023-10-06T05:29:20.144406+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:48.713611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.145709+0000","last_change":"2023-10-06T05:29:20.143500+0000","last_active":"2023-10-06T05:29:20.145709+0000","last_peered":"2023-10-06T05:29:20.145709+0000","last_clean":"2023-10-06T05:29:20.145709+0000","last_became_active":"2023-10-06T05:29:20.143118+0000","last_became_peered":"2023-10-06T05:29:20.143118+0000","last_unstale":"2023-10-06T05:29:20.145709+0000","last_undegraded":"2023-10-06T05:29:20.145709+0000","last_fullsized":"2023-10-06T05:29:20.145709+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:50.376311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.158303+0000","last_change":"2023-10-06T05:29:20.153625+0000","last_active":"2023-10-06T05:29:20.158303+0000","last_peered":"2023-10-06T05:29:20.158303+0000","last_clean":"2023-10-06T05:29:20.158303+0000","last_became_active":"2023-10-06T05:29:20.153362+0000","last_became_peered":"2023-10-06T05:29:20.153362+0000","last_unstale":"2023-10-06T05:29:20.158303+0000","last_undegraded":"2023-10-06T05:29:20.158303+0000","last_fullsized":"2023-10-06T05:29:20.158303+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:28.561696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152615+0000","last_change":"2023-10-06T05:29:20.151079+0000","last_active":"2023-10-06T05:29:20.152615+0000","last_peered":"2023-10-06T05:29:20.152615+0000","last_clean":"2023-10-06T05:29:20.152615+0000","last_became_active":"2023-10-06T05:29:20.150492+0000","last_became_peered":"2023-10-06T05:29:20.150492+0000","last_unstale":"2023-10-06T05:29:20.152615+0000","last_undegraded":"2023-10-06T05:29:20.152615+0000","last_fullsized":"2023-10-06T05:29:20.152615+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:28.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152217+0000","last_change":"2023-10-06T05:29:20.147400+0000","last_active":"2023-10-06T05:29:20.152217+0000","last_peered":"2023-10-06T05:29:20.152217+0000","last_clean":"2023-10-06T05:29:20.152217+0000","last_became_active":"2023-10-06T05:29:20.147081+0000","last_became_peered":"2023-10-06T05:29:20.147081+0000","last_unstale":"2023-10-06T05:29:20.152217+0000","last_undegraded":"2023-10-06T05:29:20.152217+0000","last_fullsized":"2023-10-06T05:29:20.152217+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:29.147674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.159493+0000","last_change":"2023-10-06T05:29:20.155549+0000","last_active":"2023-10-06T05:29:20.159493+0000","last_peered":"2023-10-06T05:29:20.159493+0000","last_clean":"2023-10-06T05:29:20.159493+0000","last_became_active":"2023-10-06T05:29:20.155236+0000","last_became_peered":"2023-10-06T05:29:20.155236+0000","last_unstale":"2023-10-06T05:29:20.159493+0000","last_undegraded":"2023-10-06T05:29:20.159493+0000","last_fullsized":"2023-10-06T05:29:20.159493+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:56.660751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.140336+0000","last_change":"2023-10-06T05:29:20.136861+0000","last_active":"2023-10-06T05:29:20.140336+0000","last_peered":"2023-10-06T05:29:20.140336+0000","last_clean":"2023-10-06T05:29:20.140336+0000","last_became_active":"2023-10-06T05:29:20.136444+0000","last_became_peered":"2023-10-06T05:29:20.136444+0000","last_unstale":"2023-10-06T05:29:20.140336+0000","last_undegraded":"2023-10-06T05:29:20.140336+0000","last_fullsized":"2023-10-06T05:29:20.140336+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:30.399866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.138371+0000","last_change":"2023-10-06T05:29:20.133119+0000","last_active":"2023-10-06T05:29:20.138371+0000","last_peered":"2023-10-06T05:29:20.138371+0000","last_clean":"2023-10-06T05:29:20.138371+0000","last_became_active":"2023-10-06T05:29:20.132847+0000","last_became_peered":"2023-10-06T05:29:20.132847+0000","last_unstale":"2023-10-06T05:29:20.138371+0000","last_undegraded":"2023-10-06T05:29:20.138371+0000","last_fullsized":"2023-10-06T05:29:20.138371+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:27.823924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152664+0000","last_change":"2023-10-06T05:29:20.148875+0000","last_active":"2023-10-06T05:29:20.152664+0000","last_peered":"2023-10-06T05:29:20.152664+0000","last_clean":"2023-10-06T05:29:20.152664+0000","last_became_active":"2023-10-06T05:29:20.147477+0000","last_became_peered":"2023-10-06T05:29:20.147477+0000","last_unstale":"2023-10-06T05:29:20.152664+0000","last_undegraded":"2023-10-06T05:29:20.152664+0000","last_fullsized":"2023-10-06T05:29:20.152664+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:27.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152253+0000","last_change":"2023-10-06T05:29:20.150540+0000","last_active":"2023-10-06T05:29:20.152253+0000","last_peered":"2023-10-06T05:29:20.152253+0000","last_clean":"2023-10-06T05:29:20.152253+0000","last_became_active":"2023-10-06T05:29:20.150264+0000","last_became_peered":"2023-10-06T05:29:20.150264+0000","last_unstale":"2023-10-06T05:29:20.152253+0000","last_undegraded":"2023-10-06T05:29:20.152253+0000","last_fullsized":"2023-10-06T05:29:20.152253+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:20.401755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148647+0000","last_change":"2023-10-06T05:29:20.141677+0000","last_active":"2023-10-06T05:29:20.148647+0000","last_peered":"2023-10-06T05:29:20.148647+0000","last_clean":"2023-10-06T05:29:20.148647+0000","last_became_active":"2023-10-06T05:29:20.141395+0000","last_became_peered":"2023-10-06T05:29:20.141395+0000","last_unstale":"2023-10-06T05:29:20.148647+0000","last_undegraded":"2023-10-06T05:29:20.148647+0000","last_fullsized":"2023-10-06T05:29:20.148647+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:33.897100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.147505+0000","last_change":"2023-10-06T05:29:20.146356+0000","last_active":"2023-10-06T05:29:20.147505+0000","last_peered":"2023-10-06T05:29:20.147505+0000","last_clean":"2023-10-06T05:29:20.147505+0000","last_became_active":"2023-10-06T05:29:20.145994+0000","last_became_peered":"2023-10-06T05:29:20.145994+0000","last_unstale":"2023-10-06T05:29:20.147505+0000","last_undegraded":"2023-10-06T05:29:20.147505+0000","last_fullsized":"2023-10-06T05:29:20.147505+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:34.146881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142468+0000","last_change":"2023-10-06T05:29:20.140556+0000","last_active":"2023-10-06T05:29:20.142468+0000","last_peered":"2023-10-06T05:29:20.142468+0000","last_clean":"2023-10-06T05:29:20.142468+0000","last_became_active":"2023-10-06T05:29:20.140231+0000","last_became_peered":"2023-10-06T05:29:20.140231+0000","last_unstale":"2023-10-06T05:29:20.142468+0000","last_undegraded":"2023-10-06T05:29:20.142468+0000","last_fullsized":"2023-10-06T05:29:20.142468+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:33.897100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.149064+0000","last_change":"2023-10-06T05:29:20.145949+0000","last_active":"2023-10-06T05:29:20.149064+0000","last_peered":"2023-10-06T05:29:20.149064+0000","last_clean":"2023-10-06T05:29:20.149064+0000","last_became_active":"2023-10-06T05:29:20.145492+0000","last_became_peered":"2023-10-06T05:29:20.145492+0000","last_unstale":"2023-10-06T05:29:20.149064+0000","last_undegraded":"2023-10-06T05:29:20.149064+0000","last_fullsized":"2023-10-06T05:29:20.149064+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:45.216186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142338+0000","last_change":"2023-10-06T05:29:20.130304+0000","last_active":"2023-10-06T05:29:20.142338+0000","last_peered":"2023-10-06T05:29:20.142338+0000","last_clean":"2023-10-06T05:29:20.142338+0000","last_became_active":"2023-10-06T05:29:20.130043+0000","last_became_peered":"2023-10-06T05:29:20.130043+0000","last_unstale":"2023-10-06T05:29:20.142338+0000","last_undegraded":"2023-10-06T05:29:20.142338+0000","last_fullsized":"2023-10-06T05:29:20.142338+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:21.683933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.145919+0000","last_change":"2023-10-06T05:29:20.141091+0000","last_active":"2023-10-06T05:29:20.145919+0000","last_peered":"2023-10-06T05:29:20.145919+0000","last_clean":"2023-10-06T05:29:20.145919+0000","last_became_active":"2023-10-06T05:29:20.140811+0000","last_became_peered":"2023-10-06T05:29:20.140811+0000","last_unstale":"2023-10-06T05:29:20.145919+0000","last_undegraded":"2023-10-06T05:29:20.145919+0000","last_fullsized":"2023-10-06T05:29:20.145919+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:39.748897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.141044+0000","last_change":"2023-10-06T05:29:20.126709+0000","last_active":"2023-10-06T05:29:20.141044+0000","last_peered":"2023-10-06T05:29:20.141044+0000","last_clean":"2023-10-06T05:29:20.141044+0000","last_became_active":"2023-10-06T05:29:20.126411+0000","last_became_peered":"2023-10-06T05:29:20.126411+0000","last_unstale":"2023-10-06T05:29:20.141044+0000","last_undegraded":"2023-10-06T05:29:20.141044+0000","last_fullsized":"2023-10-06T05:29:20.141044+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:29.764934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.140513+0000","last_change":"2023-10-06T05:29:20.139090+0000","last_active":"2023-10-06T05:29:20.140513+0000","last_peered":"2023-10-06T05:29:20.140513+0000","last_clean":"2023-10-06T05:29:20.140513+0000","last_became_active":"2023-10-06T05:29:20.138426+0000","last_became_peered":"2023-10-06T05:29:20.138426+0000","last_unstale":"2023-10-06T05:29:20.140513+0000","last_undegraded":"2023-10-06T05:29:20.140513+0000","last_fullsized":"2023-10-06T05:29:20.140513+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:58.850616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142309+0000","last_change":"2023-10-06T05:29:20.131355+0000","last_active":"2023-10-06T05:29:20.142309+0000","last_peered":"2023-10-06T05:29:20.142309+0000","last_clean":"2023-10-06T05:29:20.142309+0000","last_became_active":"2023-10-06T05:29:20.131018+0000","last_became_peered":"2023-10-06T05:29:20.131018+0000","last_unstale":"2023-10-06T05:29:20.142309+0000","last_undegraded":"2023-10-06T05:29:20.142309+0000","last_fullsized":"2023-10-06T05:29:20.142309+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:19.992039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144320+0000","last_change":"2023-10-06T05:29:20.142170+0000","last_active":"2023-10-06T05:29:20.144320+0000","last_peered":"2023-10-06T05:29:20.144320+0000","last_clean":"2023-10-06T05:29:20.144320+0000","last_became_active":"2023-10-06T05:29:20.141675+0000","last_became_peered":"2023-10-06T05:29:20.141675+0000","last_unstale":"2023-10-06T05:29:20.144320+0000","last_undegraded":"2023-10-06T05:29:20.144320+0000","last_fullsized":"2023-10-06T05:29:20.144320+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:28.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122758+0000","last_change":"2023-10-06T05:29:19.124506+0000","last_active":"2023-10-06T05:29:20.122758+0000","last_peered":"2023-10-06T05:29:20.122758+0000","last_clean":"2023-10-06T05:29:20.122758+0000","last_became_active":"2023-10-06T05:29:19.124229+0000","last_became_peered":"2023-10-06T05:29:19.124229+0000","last_unstale":"2023-10-06T05:29:20.122758+0000","last_undegraded":"2023-10-06T05:29:20.122758+0000","last_fullsized":"2023-10-06T05:29:20.122758+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:32.164964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.149966+0000","last_change":"2023-10-06T05:29:20.144895+0000","last_active":"2023-10-06T05:29:20.149966+0000","last_peered":"2023-10-06T05:29:20.149966+0000","last_clean":"2023-10-06T05:29:20.149966+0000","last_became_active":"2023-10-06T05:29:20.144598+0000","last_became_peered":"2023-10-06T05:29:20.144598+0000","last_unstale":"2023-10-06T05:29:20.149966+0000","last_undegraded":"2023-10-06T05:29:20.149966+0000","last_fullsized":"2023-10-06T05:29:20.149966+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:27.823924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.139078+0000","last_change":"2023-10-06T05:29:19.120534+0000","last_active":"2023-10-06T05:29:20.139078+0000","last_peered":"2023-10-06T05:29:20.139078+0000","last_clean":"2023-10-06T05:29:20.139078+0000","last_became_active":"2023-10-06T05:29:19.120256+0000","last_became_peered":"2023-10-06T05:29:19.120256+0000","last_unstale":"2023-10-06T05:29:20.139078+0000","last_undegraded":"2023-10-06T05:29:20.139078+0000","last_fullsized":"2023-10-06T05:29:20.139078+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.808874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152291+0000","last_change":"2023-10-06T05:29:20.147952+0000","last_active":"2023-10-06T05:29:20.152291+0000","last_peered":"2023-10-06T05:29:20.152291+0000","last_clean":"2023-10-06T05:29:20.152291+0000","last_became_active":"2023-10-06T05:29:20.146719+0000","last_became_peered":"2023-10-06T05:29:20.146719+0000","last_unstale":"2023-10-06T05:29:20.152291+0000","last_undegraded":"2023-10-06T05:29:20.152291+0000","last_fullsized":"2023-10-06T05:29:20.152291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:56.660751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"209'3","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.153287+0000","last_change":"2023-10-06T05:29:19.124209+0000","last_active":"2023-10-06T05:29:20.153287+0000","last_peered":"2023-10-06T05:29:20.153287+0000","last_clean":"2023-10-06T05:29:20.153287+0000","last_became_active":"2023-10-06T05:29:19.123770+0000","last_became_peered":"2023-10-06T05:29:19.123770+0000","last_unstale":"2023-10-06T05:29:20.153287+0000","last_undegraded":"2023-10-06T05:29:20.153287+0000","last_fullsized":"2023-10-06T05:29:20.153287+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:10.185226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.147116+0000","last_change":"2023-10-06T05:29:20.144871+0000","last_active":"2023-10-06T05:29:20.147116+0000","last_peered":"2023-10-06T05:29:20.147116+0000","last_clean":"2023-10-06T05:29:20.147116+0000","last_became_active":"2023-10-06T05:29:20.144550+0000","last_became_peered":"2023-10-06T05:29:20.144550+0000","last_unstale":"2023-10-06T05:29:20.147116+0000","last_undegraded":"2023-10-06T05:29:20.147116+0000","last_fullsized":"2023-10-06T05:29:20.147116+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:00.967071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"209'6","reported_seq":16,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.166525+0000","last_change":"2023-10-06T05:29:19.115617+0000","last_active":"2023-10-06T05:29:20.166525+0000","last_peered":"2023-10-06T05:29:20.166525+0000","last_clean":"2023-10-06T05:29:20.166525+0000","last_became_active":"2023-10-06T05:29:19.115323+0000","last_became_peered":"2023-10-06T05:29:19.115323+0000","last_unstale":"2023-10-06T05:29:20.166525+0000","last_undegraded":"2023-10-06T05:29:20.166525+0000","last_fullsized":"2023-10-06T05:29:20.166525+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:52.004531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157582+0000","last_change":"2023-10-06T05:29:20.148463+0000","last_active":"2023-10-06T05:29:20.157582+0000","last_peered":"2023-10-06T05:29:20.157582+0000","last_clean":"2023-10-06T05:29:20.157582+0000","last_became_active":"2023-10-06T05:29:20.148148+0000","last_became_peered":"2023-10-06T05:29:20.148148+0000","last_unstale":"2023-10-06T05:29:20.157582+0000","last_undegraded":"2023-10-06T05:29:20.157582+0000","last_fullsized":"2023-10-06T05:29:20.157582+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:50.357267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155676+0000","last_change":"2023-10-06T05:29:19.116548+0000","last_active":"2023-10-06T05:29:20.155676+0000","last_peered":"2023-10-06T05:29:20.155676+0000","last_clean":"2023-10-06T05:29:20.155676+0000","last_became_active":"2023-10-06T05:29:19.116271+0000","last_became_peered":"2023-10-06T05:29:19.116271+0000","last_unstale":"2023-10-06T05:29:20.155676+0000","last_undegraded":"2023-10-06T05:29:20.155676+0000","last_fullsized":"2023-10-06T05:29:20.155676+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:22.456167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.145348+0000","last_change":"2023-10-06T05:29:20.142745+0000","last_active":"2023-10-06T05:29:20.145348+0000","last_peered":"2023-10-06T05:29:20.145348+0000","last_clean":"2023-10-06T05:29:20.145348+0000","last_became_active":"2023-10-06T05:29:20.142481+0000","last_became_peered":"2023-10-06T05:29:20.142481+0000","last_unstale":"2023-10-06T05:29:20.145348+0000","last_undegraded":"2023-10-06T05:29:20.145348+0000","last_fullsized":"2023-10-06T05:29:20.145348+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:05.082060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122335+0000","last_change":"2023-10-06T05:29:19.128908+0000","last_active":"2023-10-06T05:29:20.122335+0000","last_peered":"2023-10-06T05:29:20.122335+0000","last_clean":"2023-10-06T05:29:20.122335+0000","last_became_active":"2023-10-06T05:29:19.128627+0000","last_became_peered":"2023-10-06T05:29:19.128627+0000","last_unstale":"2023-10-06T05:29:20.122335+0000","last_undegraded":"2023-10-06T05:29:20.122335+0000","last_fullsized":"2023-10-06T05:29:20.122335+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:33.829071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.149351+0000","last_change":"2023-10-06T05:29:20.146303+0000","last_active":"2023-10-06T05:29:20.149351+0000","last_peered":"2023-10-06T05:29:20.149351+0000","last_clean":"2023-10-06T05:29:20.149351+0000","last_became_active":"2023-10-06T05:29:20.145976+0000","last_became_peered":"2023-10-06T05:29:20.145976+0000","last_unstale":"2023-10-06T05:29:20.149351+0000","last_undegraded":"2023-10-06T05:29:20.149351+0000","last_fullsized":"2023-10-06T05:29:20.149351+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.899449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122729+0000","last_change":"2023-10-06T05:29:19.120038+0000","last_active":"2023-10-06T05:29:20.122729+0000","last_peered":"2023-10-06T05:29:20.122729+0000","last_clean":"2023-10-06T05:29:20.122729+0000","last_became_active":"2023-10-06T05:29:19.119774+0000","last_became_peered":"2023-10-06T05:29:19.119774+0000","last_unstale":"2023-10-06T05:29:20.122729+0000","last_undegraded":"2023-10-06T05:29:20.122729+0000","last_fullsized":"2023-10-06T05:29:20.122729+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:14.771802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.143563+0000","last_change":"2023-10-06T05:29:20.137235+0000","last_active":"2023-10-06T05:29:20.143563+0000","last_peered":"2023-10-06T05:29:20.143563+0000","last_clean":"2023-10-06T05:29:20.143563+0000","last_became_active":"2023-10-06T05:29:20.136977+0000","last_became_peered":"2023-10-06T05:29:20.136977+0000","last_unstale":"2023-10-06T05:29:20.143563+0000","last_undegraded":"2023-10-06T05:29:20.143563+0000","last_fullsized":"2023-10-06T05:29:20.143563+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:34.990845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"209'8","reported_seq":18,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157582+0000","last_change":"2023-10-06T05:29:19.118674+0000","last_active":"2023-10-06T05:29:20.157582+0000","last_peered":"2023-10-06T05:29:20.157582+0000","last_clean":"2023-10-06T05:29:20.157582+0000","last_became_active":"2023-10-06T05:29:19.118418+0000","last_became_peered":"2023-10-06T05:29:19.118418+0000","last_unstale":"2023-10-06T05:29:20.157582+0000","last_undegraded":"2023-10-06T05:29:20.157582+0000","last_fullsized":"2023-10-06T05:29:20.157582+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:21.078436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148168+0000","last_change":"2023-10-06T05:29:20.141146+0000","last_active":"2023-10-06T05:29:20.148168+0000","last_peered":"2023-10-06T05:29:20.148168+0000","last_clean":"2023-10-06T05:29:20.148168+0000","last_became_active":"2023-10-06T05:29:20.140836+0000","last_became_peered":"2023-10-06T05:29:20.140836+0000","last_unstale":"2023-10-06T05:29:20.148168+0000","last_undegraded":"2023-10-06T05:29:20.148168+0000","last_fullsized":"2023-10-06T05:29:20.148168+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:55.933108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.150552+0000","last_change":"2023-10-06T05:29:19.124764+0000","last_active":"2023-10-06T05:29:20.150552+0000","last_peered":"2023-10-06T05:29:20.150552+0000","last_clean":"2023-10-06T05:29:20.150552+0000","last_became_active":"2023-10-06T05:29:19.124439+0000","last_became_peered":"2023-10-06T05:29:19.124439+0000","last_unstale":"2023-10-06T05:29:20.150552+0000","last_undegraded":"2023-10-06T05:29:20.150552+0000","last_fullsized":"2023-10-06T05:29:20.150552+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:55.684382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148413+0000","last_change":"2023-10-06T05:29:20.142379+0000","last_active":"2023-10-06T05:29:20.148413+0000","last_peered":"2023-10-06T05:29:20.148413+0000","last_clean":"2023-10-06T05:29:20.148413+0000","last_became_active":"2023-10-06T05:29:20.141214+0000","last_became_peered":"2023-10-06T05:29:20.141214+0000","last_unstale":"2023-10-06T05:29:20.148413+0000","last_undegraded":"2023-10-06T05:29:20.148413+0000","last_fullsized":"2023-10-06T05:29:20.148413+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:17.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148089+0000","last_change":"2023-10-06T05:29:19.121504+0000","last_active":"2023-10-06T05:29:20.148089+0000","last_peered":"2023-10-06T05:29:20.148089+0000","last_clean":"2023-10-06T05:29:20.148089+0000","last_became_active":"2023-10-06T05:29:19.121244+0000","last_became_peered":"2023-10-06T05:29:19.121244+0000","last_unstale":"2023-10-06T05:29:20.148089+0000","last_undegraded":"2023-10-06T05:29:20.148089+0000","last_fullsized":"2023-10-06T05:29:20.148089+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:45.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.140548+0000","last_change":"2023-10-06T05:29:20.136254+0000","last_active":"2023-10-06T05:29:20.140548+0000","last_peered":"2023-10-06T05:29:20.140548+0000","last_clean":"2023-10-06T05:29:20.140548+0000","last_became_active":"2023-10-06T05:29:20.135984+0000","last_became_peered":"2023-10-06T05:29:20.135984+0000","last_unstale":"2023-10-06T05:29:20.140548+0000","last_undegraded":"2023-10-06T05:29:20.140548+0000","last_fullsized":"2023-10-06T05:29:20.140548+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:16.688762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.159719+0000","last_change":"2023-10-06T05:29:19.128302+0000","last_active":"2023-10-06T05:29:20.159719+0000","last_peered":"2023-10-06T05:29:20.159719+0000","last_clean":"2023-10-06T05:29:20.159719+0000","last_became_active":"2023-10-06T05:29:19.128011+0000","last_became_peered":"2023-10-06T05:29:19.128011+0000","last_unstale":"2023-10-06T05:29:20.159719+0000","last_undegraded":"2023-10-06T05:29:20.159719+0000","last_fullsized":"2023-10-06T05:29:20.159719+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:24.687020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155722+0000","last_change":"2023-10-06T05:29:20.150181+0000","last_active":"2023-10-06T05:29:20.155722+0000","last_peered":"2023-10-06T05:29:20.155722+0000","last_clean":"2023-10-06T05:29:20.155722+0000","last_became_active":"2023-10-06T05:29:20.149936+0000","last_became_peered":"2023-10-06T05:29:20.149936+0000","last_unstale":"2023-10-06T05:29:20.155722+0000","last_undegraded":"2023-10-06T05:29:20.155722+0000","last_fullsized":"2023-10-06T05:29:20.155722+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:59.102372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.129659+0000","last_change":"2023-10-06T05:29:19.122512+0000","last_active":"2023-10-06T05:29:20.129659+0000","last_peered":"2023-10-06T05:29:20.129659+0000","last_clean":"2023-10-06T05:29:20.129659+0000","last_became_active":"2023-10-06T05:29:19.122215+0000","last_became_peered":"2023-10-06T05:29:19.122215+0000","last_unstale":"2023-10-06T05:29:20.129659+0000","last_undegraded":"2023-10-06T05:29:20.129659+0000","last_fullsized":"2023-10-06T05:29:20.129659+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:09.193150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157645+0000","last_change":"2023-10-06T05:29:20.154260+0000","last_active":"2023-10-06T05:29:20.157645+0000","last_peered":"2023-10-06T05:29:20.157645+0000","last_clean":"2023-10-06T05:29:20.157645+0000","last_became_active":"2023-10-06T05:29:20.153993+0000","last_became_peered":"2023-10-06T05:29:20.153993+0000","last_unstale":"2023-10-06T05:29:20.157645+0000","last_undegraded":"2023-10-06T05:29:20.157645+0000","last_fullsized":"2023-10-06T05:29:20.157645+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:04.392114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.154514+0000","last_change":"2023-10-06T05:29:19.123872+0000","last_active":"2023-10-06T05:29:20.154514+0000","last_peered":"2023-10-06T05:29:20.154514+0000","last_clean":"2023-10-06T05:29:20.154514+0000","last_became_active":"2023-10-06T05:29:19.123598+0000","last_became_peered":"2023-10-06T05:29:19.123598+0000","last_unstale":"2023-10-06T05:29:20.154514+0000","last_undegraded":"2023-10-06T05:29:20.154514+0000","last_fullsized":"2023-10-06T05:29:20.154514+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:48.165151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157772+0000","last_change":"2023-10-06T05:29:20.152714+0000","last_active":"2023-10-06T05:29:20.157772+0000","last_peered":"2023-10-06T05:29:20.157772+0000","last_clean":"2023-10-06T05:29:20.157772+0000","last_became_active":"2023-10-06T05:29:20.152473+0000","last_became_peered":"2023-10-06T05:29:20.152473+0000","last_unstale":"2023-10-06T05:29:20.157772+0000","last_undegraded":"2023-10-06T05:29:20.157772+0000","last_fullsized":"2023-10-06T05:29:20.157772+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:00.118747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134024+0000","last_change":"2023-10-06T05:29:19.128363+0000","last_active":"2023-10-06T05:29:20.134024+0000","last_peered":"2023-10-06T05:29:20.134024+0000","last_clean":"2023-10-06T05:29:20.134024+0000","last_became_active":"2023-10-06T05:29:19.128072+0000","last_became_peered":"2023-10-06T05:29:19.128072+0000","last_unstale":"2023-10-06T05:29:20.134024+0000","last_undegraded":"2023-10-06T05:29:20.134024+0000","last_fullsized":"2023-10-06T05:29:20.134024+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:17.458208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152636+0000","last_change":"2023-10-06T05:29:20.150928+0000","last_active":"2023-10-06T05:29:20.152636+0000","last_peered":"2023-10-06T05:29:20.152636+0000","last_clean":"2023-10-06T05:29:20.152636+0000","last_became_active":"2023-10-06T05:29:20.150274+0000","last_became_peered":"2023-10-06T05:29:20.150274+0000","last_unstale":"2023-10-06T05:29:20.152636+0000","last_undegraded":"2023-10-06T05:29:20.152636+0000","last_fullsized":"2023-10-06T05:29:20.152636+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:47.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.128911+0000","last_change":"2023-10-06T05:29:19.127593+0000","last_active":"2023-10-06T05:29:20.128911+0000","last_peered":"2023-10-06T05:29:20.128911+0000","last_clean":"2023-10-06T05:29:20.128911+0000","last_became_active":"2023-10-06T05:29:19.126087+0000","last_became_peered":"2023-10-06T05:29:19.126087+0000","last_unstale":"2023-10-06T05:29:20.128911+0000","last_undegraded":"2023-10-06T05:29:20.128911+0000","last_fullsized":"2023-10-06T05:29:20.128911+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:10.505144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152589+0000","last_change":"2023-10-06T05:29:20.150750+0000","last_active":"2023-10-06T05:29:20.152589+0000","last_peered":"2023-10-06T05:29:20.152589+0000","last_clean":"2023-10-06T05:29:20.152589+0000","last_became_active":"2023-10-06T05:29:20.150121+0000","last_became_peered":"2023-10-06T05:29:20.150121+0000","last_unstale":"2023-10-06T05:29:20.152589+0000","last_undegraded":"2023-10-06T05:29:20.152589+0000","last_fullsized":"2023-10-06T05:29:20.152589+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:05.082060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.121568+0000","last_change":"2023-10-06T05:29:19.123013+0000","last_active":"2023-10-06T05:29:20.121568+0000","last_peered":"2023-10-06T05:29:20.121568+0000","last_clean":"2023-10-06T05:29:20.121568+0000","last_became_active":"2023-10-06T05:29:19.122688+0000","last_became_peered":"2023-10-06T05:29:19.122688+0000","last_unstale":"2023-10-06T05:29:20.121568+0000","last_undegraded":"2023-10-06T05:29:20.121568+0000","last_fullsized":"2023-10-06T05:29:20.121568+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:40.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142262+0000","last_change":"2023-10-06T05:29:20.139762+0000","last_active":"2023-10-06T05:29:20.142262+0000","last_peered":"2023-10-06T05:29:20.142262+0000","last_clean":"2023-10-06T05:29:20.142262+0000","last_became_active":"2023-10-06T05:29:20.139445+0000","last_became_peered":"2023-10-06T05:29:20.139445+0000","last_unstale":"2023-10-06T05:29:20.142262+0000","last_undegraded":"2023-10-06T05:29:20.142262+0000","last_fullsized":"2023-10-06T05:29:20.142262+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:27.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.153660+0000","last_change":"2023-10-06T05:29:19.116860+0000","last_active":"2023-10-06T05:29:20.153660+0000","last_peered":"2023-10-06T05:29:20.153660+0000","last_clean":"2023-10-06T05:29:20.153660+0000","last_became_active":"2023-10-06T05:29:19.116528+0000","last_became_peered":"2023-10-06T05:29:19.116528+0000","last_unstale":"2023-10-06T05:29:20.153660+0000","last_undegraded":"2023-10-06T05:29:20.153660+0000","last_fullsized":"2023-10-06T05:29:20.153660+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:30.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142749+0000","last_change":"2023-10-06T05:29:20.138146+0000","last_active":"2023-10-06T05:29:20.142749+0000","last_peered":"2023-10-06T05:29:20.142749+0000","last_clean":"2023-10-06T05:29:20.142749+0000","last_became_active":"2023-10-06T05:29:20.137859+0000","last_became_peered":"2023-10-06T05:29:20.137859+0000","last_unstale":"2023-10-06T05:29:20.142749+0000","last_undegraded":"2023-10-06T05:29:20.142749+0000","last_fullsized":"2023-10-06T05:29:20.142749+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:28.561696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122407+0000","last_change":"2023-10-06T05:29:19.126125+0000","last_active":"2023-10-06T05:29:20.122407+0000","last_peered":"2023-10-06T05:29:20.122407+0000","last_clean":"2023-10-06T05:29:20.122407+0000","last_became_active":"2023-10-06T05:29:19.125186+0000","last_became_peered":"2023-10-06T05:29:19.125186+0000","last_unstale":"2023-10-06T05:29:20.122407+0000","last_undegraded":"2023-10-06T05:29:20.122407+0000","last_fullsized":"2023-10-06T05:29:20.122407+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:33.119207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155686+0000","last_change":"2023-10-06T05:29:20.151574+0000","last_active":"2023-10-06T05:29:20.155686+0000","last_peered":"2023-10-06T05:29:20.155686+0000","last_clean":"2023-10-06T05:29:20.155686+0000","last_became_active":"2023-10-06T05:29:20.151134+0000","last_became_peered":"2023-10-06T05:29:20.151134+0000","last_unstale":"2023-10-06T05:29:20.155686+0000","last_undegraded":"2023-10-06T05:29:20.155686+0000","last_fullsized":"2023-10-06T05:29:20.155686+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:20.401755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122374+0000","last_change":"2023-10-06T05:29:19.125212+0000","last_active":"2023-10-06T05:29:20.122374+0000","last_peered":"2023-10-06T05:29:20.122374+0000","last_clean":"2023-10-06T05:29:20.122374+0000","last_became_active":"2023-10-06T05:29:19.124953+0000","last_became_peered":"2023-10-06T05:29:19.124953+0000","last_unstale":"2023-10-06T05:29:20.122374+0000","last_undegraded":"2023-10-06T05:29:20.122374+0000","last_fullsized":"2023-10-06T05:29:20.122374+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:47.580175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148652+0000","last_change":"2023-10-06T05:29:20.137889+0000","last_active":"2023-10-06T05:29:20.148652+0000","last_peered":"2023-10-06T05:29:20.148652+0000","last_clean":"2023-10-06T05:29:20.148652+0000","last_became_active":"2023-10-06T05:29:20.137617+0000","last_became_peered":"2023-10-06T05:29:20.137617+0000","last_unstale":"2023-10-06T05:29:20.148652+0000","last_undegraded":"2023-10-06T05:29:20.148652+0000","last_fullsized":"2023-10-06T05:29:20.148652+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:02.935235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122498+0000","last_change":"2023-10-06T05:29:19.123577+0000","last_active":"2023-10-06T05:29:20.122498+0000","last_peered":"2023-10-06T05:29:20.122498+0000","last_clean":"2023-10-06T05:29:20.122498+0000","last_became_active":"2023-10-06T05:29:19.123338+0000","last_became_peered":"2023-10-06T05:29:19.123338+0000","last_unstale":"2023-10-06T05:29:20.122498+0000","last_undegraded":"2023-10-06T05:29:20.122498+0000","last_fullsized":"2023-10-06T05:29:20.122498+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:44.835632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.141480+0000","last_change":"2023-10-06T05:29:20.132262+0000","last_active":"2023-10-06T05:29:20.141480+0000","last_peered":"2023-10-06T05:29:20.141480+0000","last_clean":"2023-10-06T05:29:20.141480+0000","last_became_active":"2023-10-06T05:29:20.132023+0000","last_became_peered":"2023-10-06T05:29:20.132023+0000","last_unstale":"2023-10-06T05:29:20.141480+0000","last_undegraded":"2023-10-06T05:29:20.141480+0000","last_fullsized":"2023-10-06T05:29:20.141480+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:03.419340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.131597+0000","last_change":"2023-10-06T05:29:19.127795+0000","last_active":"2023-10-06T05:29:20.131597+0000","last_peered":"2023-10-06T05:29:20.131597+0000","last_clean":"2023-10-06T05:29:20.131597+0000","last_became_active":"2023-10-06T05:29:19.127216+0000","last_became_peered":"2023-10-06T05:29:19.127216+0000","last_unstale":"2023-10-06T05:29:20.131597+0000","last_undegraded":"2023-10-06T05:29:20.131597+0000","last_fullsized":"2023-10-06T05:29:20.131597+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:32.486205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142443+0000","last_change":"2023-10-06T05:29:20.129613+0000","last_active":"2023-10-06T05:29:20.142443+0000","last_peered":"2023-10-06T05:29:20.142443+0000","last_clean":"2023-10-06T05:29:20.142443+0000","last_became_active":"2023-10-06T05:29:20.129339+0000","last_became_peered":"2023-10-06T05:29:20.129339+0000","last_unstale":"2023-10-06T05:29:20.142443+0000","last_undegraded":"2023-10-06T05:29:20.142443+0000","last_fullsized":"2023-10-06T05:29:20.142443+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:03.419340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122479+0000","last_change":"2023-10-06T05:29:19.122769+0000","last_active":"2023-10-06T05:29:20.122479+0000","last_peered":"2023-10-06T05:29:20.122479+0000","last_clean":"2023-10-06T05:29:20.122479+0000","last_became_active":"2023-10-06T05:29:19.122452+0000","last_became_peered":"2023-10-06T05:29:19.122452+0000","last_unstale":"2023-10-06T05:29:20.122479+0000","last_undegraded":"2023-10-06T05:29:20.122479+0000","last_fullsized":"2023-10-06T05:29:20.122479+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:19.519119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157598+0000","last_change":"2023-10-06T05:29:20.151460+0000","last_active":"2023-10-06T05:29:20.157598+0000","last_peered":"2023-10-06T05:29:20.157598+0000","last_clean":"2023-10-06T05:29:20.157598+0000","last_became_active":"2023-10-06T05:29:20.151216+0000","last_became_peered":"2023-10-06T05:29:20.151216+0000","last_unstale":"2023-10-06T05:29:20.157598+0000","last_undegraded":"2023-10-06T05:29:20.157598+0000","last_fullsized":"2023-10-06T05:29:20.157598+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:05.449456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.135363+0000","last_change":"2023-10-06T05:29:19.121182+0000","last_active":"2023-10-06T05:29:20.135363+0000","last_peered":"2023-10-06T05:29:20.135363+0000","last_clean":"2023-10-06T05:29:20.135363+0000","last_became_active":"2023-10-06T05:29:19.120908+0000","last_became_peered":"2023-10-06T05:29:19.120908+0000","last_unstale":"2023-10-06T05:29:20.135363+0000","last_undegraded":"2023-10-06T05:29:20.135363+0000","last_fullsized":"2023-10-06T05:29:20.135363+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:08.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.147739+0000","last_change":"2023-10-06T05:29:20.145143+0000","last_active":"2023-10-06T05:29:20.147739+0000","last_peered":"2023-10-06T05:29:20.147739+0000","last_clean":"2023-10-06T05:29:20.147739+0000","last_became_active":"2023-10-06T05:29:20.144731+0000","last_became_peered":"2023-10-06T05:29:20.144731+0000","last_unstale":"2023-10-06T05:29:20.147739+0000","last_undegraded":"2023-10-06T05:29:20.147739+0000","last_fullsized":"2023-10-06T05:29:20.147739+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:12.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.133977+0000","last_change":"2023-10-06T05:29:19.121069+0000","last_active":"2023-10-06T05:29:20.133977+0000","last_peered":"2023-10-06T05:29:20.133977+0000","last_clean":"2023-10-06T05:29:20.133977+0000","last_became_active":"2023-10-06T05:29:19.120748+0000","last_became_peered":"2023-10-06T05:29:19.120748+0000","last_unstale":"2023-10-06T05:29:20.133977+0000","last_undegraded":"2023-10-06T05:29:20.133977+0000","last_fullsized":"2023-10-06T05:29:20.133977+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:10.129669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.158450+0000","last_change":"2023-10-06T05:29:20.153804+0000","last_active":"2023-10-06T05:29:20.158450+0000","last_peered":"2023-10-06T05:29:20.158450+0000","last_clean":"2023-10-06T05:29:20.158450+0000","last_became_active":"2023-10-06T05:29:20.153560+0000","last_became_peered":"2023-10-06T05:29:20.153560+0000","last_unstale":"2023-10-06T05:29:20.158450+0000","last_undegraded":"2023-10-06T05:29:20.158450+0000","last_fullsized":"2023-10-06T05:29:20.158450+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:50.376311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.135358+0000","last_change":"2023-10-06T05:29:19.118394+0000","last_active":"2023-10-06T05:29:20.135358+0000","last_peered":"2023-10-06T05:29:20.135358+0000","last_clean":"2023-10-06T05:29:20.135358+0000","last_became_active":"2023-10-06T05:29:19.118139+0000","last_became_peered":"2023-10-06T05:29:19.118139+0000","last_unstale":"2023-10-06T05:29:20.135358+0000","last_undegraded":"2023-10-06T05:29:20.135358+0000","last_fullsized":"2023-10-06T05:29:20.135358+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:18.129121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134158+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:29:20.134158+0000","last_peered":"2023-10-06T05:29:20.134158+0000","last_clean":"2023-10-06T05:29:20.134158+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:29:20.134158+0000","last_undegraded":"2023-10-06T05:29:20.134158+0000","last_fullsized":"2023-10-06T05:29:20.134158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157468+0000","last_change":"2023-10-06T05:29:20.152136+0000","last_active":"2023-10-06T05:29:20.157468+0000","last_peered":"2023-10-06T05:29:20.157468+0000","last_clean":"2023-10-06T05:29:20.157468+0000","last_became_active":"2023-10-06T05:29:20.151887+0000","last_became_peered":"2023-10-06T05:29:20.151887+0000","last_unstale":"2023-10-06T05:29:20.157468+0000","last_undegraded":"2023-10-06T05:29:20.157468+0000","last_fullsized":"2023-10-06T05:29:20.157468+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:23.692229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"209'4","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.159885+0000","last_change":"2023-10-06T05:29:19.119785+0000","last_active":"2023-10-06T05:29:20.159885+0000","last_peered":"2023-10-06T05:29:20.159885+0000","last_clean":"2023-10-06T05:29:20.159885+0000","last_became_active":"2023-10-06T05:29:19.119467+0000","last_became_peered":"2023-10-06T05:29:19.119467+0000","last_unstale":"2023-10-06T05:29:20.159885+0000","last_undegraded":"2023-10-06T05:29:20.159885+0000","last_fullsized":"2023-10-06T05:29:20.159885+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:40.050699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.128818+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:29:20.128818+0000","last_peered":"2023-10-06T05:29:20.128818+0000","last_clean":"2023-10-06T05:29:20.128818+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:29:20.128818+0000","last_undegraded":"2023-10-06T05:29:20.128818+0000","last_fullsized":"2023-10-06T05:29:20.128818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142405+0000","last_change":"2023-10-06T05:29:20.138747+0000","last_active":"2023-10-06T05:29:20.142405+0000","last_peered":"2023-10-06T05:29:20.142405+0000","last_clean":"2023-10-06T05:29:20.142405+0000","last_became_active":"2023-10-06T05:29:20.138426+0000","last_became_peered":"2023-10-06T05:29:20.138426+0000","last_unstale":"2023-10-06T05:29:20.142405+0000","last_undegraded":"2023-10-06T05:29:20.142405+0000","last_fullsized":"2023-10-06T05:29:20.142405+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:45.353232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.143833+0000","last_change":"2023-10-06T05:29:19.117956+0000","last_active":"2023-10-06T05:29:20.143833+0000","last_peered":"2023-10-06T05:29:20.143833+0000","last_clean":"2023-10-06T05:29:20.143833+0000","last_became_active":"2023-10-06T05:29:19.117592+0000","last_became_peered":"2023-10-06T05:29:19.117592+0000","last_unstale":"2023-10-06T05:29:20.143833+0000","last_undegraded":"2023-10-06T05:29:20.143833+0000","last_fullsized":"2023-10-06T05:29:20.143833+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:21.948779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":171,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.132868+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:29:20.132868+0000","last_peered":"2023-10-06T05:29:20.132868+0000","last_clean":"2023-10-06T05:29:20.132868+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:29:20.132868+0000","last_undegraded":"2023-10-06T05:29:20.132868+0000","last_fullsized":"2023-10-06T05:29:20.132868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148192+0000","last_change":"2023-10-06T05:29:20.142655+0000","last_active":"2023-10-06T05:29:20.148192+0000","last_peered":"2023-10-06T05:29:20.148192+0000","last_clean":"2023-10-06T05:29:20.148192+0000","last_became_active":"2023-10-06T05:29:20.141953+0000","last_became_peered":"2023-10-06T05:29:20.141953+0000","last_unstale":"2023-10-06T05:29:20.148192+0000","last_undegraded":"2023-10-06T05:29:20.148192+0000","last_fullsized":"2023-10-06T05:29:20.148192+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:19.715122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155539+0000","last_change":"2023-10-06T05:29:19.119799+0000","last_active":"2023-10-06T05:29:20.155539+0000","last_peered":"2023-10-06T05:29:20.155539+0000","last_clean":"2023-10-06T05:29:20.155539+0000","last_became_active":"2023-10-06T05:29:19.118352+0000","last_became_peered":"2023-10-06T05:29:19.118352+0000","last_unstale":"2023-10-06T05:29:20.155539+0000","last_undegraded":"2023-10-06T05:29:20.155539+0000","last_fullsized":"2023-10-06T05:29:20.155539+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:24.687020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.128600+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:29:20.128600+0000","last_peered":"2023-10-06T05:29:20.128600+0000","last_clean":"2023-10-06T05:29:20.128600+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:29:20.128600+0000","last_undegraded":"2023-10-06T05:29:20.128600+0000","last_fullsized":"2023-10-06T05:29:20.128600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.129186+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:29:20.129186+0000","last_peered":"2023-10-06T05:29:20.129186+0000","last_clean":"2023-10-06T05:29:20.129186+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:29:20.129186+0000","last_undegraded":"2023-10-06T05:29:20.129186+0000","last_fullsized":"2023-10-06T05:29:20.129186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152952+0000","last_change":"2023-10-06T05:29:20.149603+0000","last_active":"2023-10-06T05:29:20.152952+0000","last_peered":"2023-10-06T05:29:20.152952+0000","last_clean":"2023-10-06T05:29:20.152952+0000","last_became_active":"2023-10-06T05:29:20.149320+0000","last_became_peered":"2023-10-06T05:29:20.149320+0000","last_unstale":"2023-10-06T05:29:20.152952+0000","last_undegraded":"2023-10-06T05:29:20.152952+0000","last_fullsized":"2023-10-06T05:29:20.152952+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:59.102372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.158785+0000","last_change":"2023-10-06T05:29:19.128409+0000","last_active":"2023-10-06T05:29:20.158785+0000","last_peered":"2023-10-06T05:29:20.158785+0000","last_clean":"2023-10-06T05:29:20.158785+0000","last_became_active":"2023-10-06T05:29:19.127679+0000","last_became_peered":"2023-10-06T05:29:19.127679+0000","last_unstale":"2023-10-06T05:29:20.158785+0000","last_undegraded":"2023-10-06T05:29:20.158785+0000","last_fullsized":"2023-10-06T05:29:20.158785+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:19.782239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.129090+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:29:20.129090+0000","last_peered":"2023-10-06T05:29:20.129090+0000","last_clean":"2023-10-06T05:29:20.129090+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:29:20.129090+0000","last_undegraded":"2023-10-06T05:29:20.129090+0000","last_fullsized":"2023-10-06T05:29:20.129090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134490+0000","last_change":"2023-10-06T05:29:20.132682+0000","last_active":"2023-10-06T05:29:20.134490+0000","last_peered":"2023-10-06T05:29:20.134490+0000","last_clean":"2023-10-06T05:29:20.134490+0000","last_became_active":"2023-10-06T05:29:20.131574+0000","last_became_peered":"2023-10-06T05:29:20.131574+0000","last_unstale":"2023-10-06T05:29:20.134490+0000","last_undegraded":"2023-10-06T05:29:20.134490+0000","last_fullsized":"2023-10-06T05:29:20.134490+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:39.430627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.121895+0000","last_change":"2023-10-06T05:29:19.116628+0000","last_active":"2023-10-06T05:29:20.121895+0000","last_peered":"2023-10-06T05:29:20.121895+0000","last_clean":"2023-10-06T05:29:20.121895+0000","last_became_active":"2023-10-06T05:29:19.112389+0000","last_became_peered":"2023-10-06T05:29:19.112389+0000","last_unstale":"2023-10-06T05:29:20.121895+0000","last_undegraded":"2023-10-06T05:29:20.121895+0000","last_fullsized":"2023-10-06T05:29:20.121895+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:40.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.133123+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:29:20.133123+0000","last_peered":"2023-10-06T05:29:20.133123+0000","last_clean":"2023-10-06T05:29:20.133123+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:29:20.133123+0000","last_undegraded":"2023-10-06T05:29:20.133123+0000","last_fullsized":"2023-10-06T05:29:20.133123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144450+0000","last_change":"2023-10-06T05:29:20.137960+0000","last_active":"2023-10-06T05:29:20.144450+0000","last_peered":"2023-10-06T05:29:20.144450+0000","last_clean":"2023-10-06T05:29:20.144450+0000","last_became_active":"2023-10-06T05:29:20.137675+0000","last_became_peered":"2023-10-06T05:29:20.137675+0000","last_unstale":"2023-10-06T05:29:20.144450+0000","last_undegraded":"2023-10-06T05:29:20.144450+0000","last_fullsized":"2023-10-06T05:29:20.144450+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:45.353232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155668+0000","last_change":"2023-10-06T05:29:19.118622+0000","last_active":"2023-10-06T05:29:20.155668+0000","last_peered":"2023-10-06T05:29:20.155668+0000","last_clean":"2023-10-06T05:29:20.155668+0000","last_became_active":"2023-10-06T05:29:19.118351+0000","last_became_peered":"2023-10-06T05:29:19.118351+0000","last_unstale":"2023-10-06T05:29:20.155668+0000","last_undegraded":"2023-10-06T05:29:20.155668+0000","last_fullsized":"2023-10-06T05:29:20.155668+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:24.904793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.139155+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:29:20.139155+0000","last_peered":"2023-10-06T05:29:20.139155+0000","last_clean":"2023-10-06T05:29:20.139155+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:29:20.139155+0000","last_undegraded":"2023-10-06T05:29:20.139155+0000","last_fullsized":"2023-10-06T05:29:20.139155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142871+0000","last_change":"2023-10-06T05:29:20.140679+0000","last_active":"2023-10-06T05:29:20.142871+0000","last_peered":"2023-10-06T05:29:20.142871+0000","last_clean":"2023-10-06T05:29:20.142871+0000","last_became_active":"2023-10-06T05:29:20.140365+0000","last_became_peered":"2023-10-06T05:29:20.140365+0000","last_unstale":"2023-10-06T05:29:20.142871+0000","last_undegraded":"2023-10-06T05:29:20.142871+0000","last_fullsized":"2023-10-06T05:29:20.142871+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:47.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144325+0000","last_change":"2023-10-06T05:29:19.118550+0000","last_active":"2023-10-06T05:29:20.144325+0000","last_peered":"2023-10-06T05:29:20.144325+0000","last_clean":"2023-10-06T05:29:20.144325+0000","last_became_active":"2023-10-06T05:29:19.118121+0000","last_became_peered":"2023-10-06T05:29:19.118121+0000","last_unstale":"2023-10-06T05:29:20.144325+0000","last_undegraded":"2023-10-06T05:29:20.144325+0000","last_fullsized":"2023-10-06T05:29:20.144325+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:30.080647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.123421+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:29:20.123421+0000","last_peered":"2023-10-06T05:29:20.123421+0000","last_clean":"2023-10-06T05:29:20.123421+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:29:20.123421+0000","last_undegraded":"2023-10-06T05:29:20.123421+0000","last_fullsized":"2023-10-06T05:29:20.123421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304552,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067288,"statfs":{"total":96636764160,"available":96324902912,"internally_reserved":0,"allocated":3244032,"data_stored":1823346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.88100000000000001,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88200000000000001,"5min":1.1339999999999999,"15min":1.169},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68600000000000005,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":1.0249999999999999,"15min":1.268},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96399999999999997,"5min":1.163,"15min":1.163},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":1.22},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.109,"15min":1.109},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.753},"min":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":1.0960000000000001,"15min":1.202},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.627},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":1.109,"15min":1.258},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.39400000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.2849999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.98099999999999998,"5min":1.109,"15min":1.109},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.752,"15min":0.749},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.407},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.999,"5min":1.169,"15min":1.169},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.52700000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.0349999999999999,"5min":1.135,"15min":1.3300000000000001},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1239607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":1.044,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.373},"max":{"1min":0.98099999999999998,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69299999999999995,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.44600000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.53400000000000003,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.91000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.48099999999999998,"15min":0.47999999999999998},"max":{"1min":0.90800000000000003,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.44900000000000001,"15min":0.442},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.871}]},{"osd":3,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.57499999999999996,"5min":0.45300000000000001,"15min":0.40899999999999997},"max":{"1min":0.93400000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.68300000000000005,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.1020000000000001},"last":0.96399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.95299999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93000000000000005,"5min":0.97799999999999998,"15min":0.998},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52500000000000002,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.0760000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":1.002,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.64400000000000002,"5min":0.497,"15min":0.436},"max":{"1min":1.0620000000000001,"5min":1.0860000000000001,"15min":1.1359999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303964,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067876,"statfs":{"total":96636764160,"available":96325505024,"internally_reserved":0,"allocated":2641920,"data_stored":1232854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.621,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27800000000000002},"max":{"1min":0.56200000000000006,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":1.165,"15min":2.6520000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.76600000000000001,"5min":1.077,"15min":2.7010000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.72399999999999998,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35199999999999998},"max":{"1min":0.69299999999999995,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.026,"5min":1.111,"15min":1.5389999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.4690000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.349},"last":0.752},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.50600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.669},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.48199999999999998},"max":{"1min":1.2290000000000001,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.573},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.528},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.48699999999999999},"max":{"1min":1.073,"5min":1.3049999999999999,"15min":2.6240000000000001},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304568,"kb_used_data":3184,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067272,"statfs":{"total":96636764160,"available":96324886528,"internally_reserved":0,"allocated":3260416,"data_stored":1831399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46200000000000002,"15min":0.45400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.245},"max":{"1min":0.59199999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.498},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.26900000000000002},"max":{"1min":0.70699999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.307},"max":{"1min":0.624,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69299999999999995,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79500000000000004,"15min":0.78000000000000003},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":1,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.244,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":0.95299999999999996,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.56100000000000005,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.45500000000000002},"max":{"1min":1.099,"5min":1.153,"15min":1.2629999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.0980000000000001,"5min":1.2010000000000001,"15min":1.25},"last":1.018},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.50600000000000001},"max":{"1min":1.167,"5min":1.274,"15min":1.2789999999999999},"last":0.871}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1240119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":1.03,"15min":1.056},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33700000000000002,"5min":0.309,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.38200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.42999999999999999,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.622,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.63,"5min":0.94399999999999995,"15min":1.073},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48999999999999999,"15min":0.47699999999999998},"min":{"1min":0.36099999999999999,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.31},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64500000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.53700000000000003},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.77100000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.47499999999999998},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.754,"15min":0.76500000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.47299999999999998},"max":{"1min":0.91700000000000004,"5min":1.153,"15min":1.1639999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":0.94699999999999995,"5min":1.1859999999999999,"15min":1.319},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.59599999999999997,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.175,"15min":1.5289999999999999},"last":0.746},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80200000000000005,"15min":0.79200000000000004},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96399999999999997,"5min":1.137,"15min":1.141},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.52100000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80400000000000005,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88400000000000001,"5min":1.2,"15min":1.3520000000000001},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304000,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067840,"statfs":{"total":96636764160,"available":96325468160,"internally_reserved":0,"allocated":2678784,"data_stored":1241641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45700000000000002,"15min":0.46800000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":2.0950000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":1.772},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60799999999999998,"5min":1.369,"15min":1.913},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.307},"max":{"1min":0.71099999999999997,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.90200000000000002,"15min":1.954},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38300000000000001},"max":{"1min":0.70099999999999996,"5min":0.82599999999999996,"15min":2.0539999999999998},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.282},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76200000000000001,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":0.92700000000000005,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.111,"15min":1.1930000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97299999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":1.0089999999999999,"5min":1.3360000000000001,"15min":1.343},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.64100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.68700000000000006,"5min":0.59999999999999998,"15min":0.56799999999999995},"max":{"1min":1.0649999999999999,"5min":1.194,"15min":1.3640000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.61299999999999999,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.1579999999999999,"15min":1.214},"last":0.88}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1239565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":1.073},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72899999999999998,"15min":0.75},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90800000000000003,"5min":1.05,"15min":1.169},"last":0.86199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":1.0569999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.88800000000000001,"5min":0.96799999999999997,"15min":1.0309999999999999},"last":0.91600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.82899999999999996,"5min":0.96199999999999997,"15min":1.117},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71599999999999997,"15min":0.71899999999999997},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41499999999999998},"max":{"1min":0.871,"5min":0.97099999999999997,"15min":1.002},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72899999999999998},"min":{"1min":0.55400000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":0.94999999999999996,"15min":1.0780000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.196},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":1.044,"15min":1.8740000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.70299999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":1.014,"15min":1.375},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.70599999999999996},"min":{"1min":0.51800000000000002,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.90400000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.505,"5min":0.505,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48199999999999998},"max":{"1min":0.93700000000000006,"5min":1.073,"15min":1.466},"last":1.081},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97099999999999997,"5min":1.0569999999999999,"15min":1.218},"last":1.0660000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575035,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303964,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067876,"statfs":{"total":96636764160,"available":96325505024,"internally_reserved":0,"allocated":2641920,"data_stored":1232854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.441,"5min":0.435,"15min":0.374},"max":{"1min":1.127,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93899999999999995,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.0529999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":0.999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0289999999999999,"15min":1.204},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.091,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.46899999999999997},"max":{"1min":0.88700000000000001,"5min":0.94599999999999995,"15min":1.087},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":0.90000000000000002,"5min":1.145,"15min":1.278},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.49399999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.1379999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.876,"5min":1.004,"15min":1.294},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.442,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.80200000000000005,"5min":0.89900000000000002,"15min":1.141},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71899999999999997,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.254},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0189999999999999,"15min":1.337},"last":0.752},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98699999999999999,"5min":1.0600000000000001,"15min":1.248},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:29:24.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:29:24.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.738+0000 7fc10b59d700 1 Processor -- start 2023-10-06T05:29:24.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.738+0000 7fc10b59d700 1 -- start start 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.739+0000 7fc10b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc104155af0 0x7fc10406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.739+0000 7fc10b59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1040c7920 0x7fc1040c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.739+0000 7fc10b59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10406ab60 con 0x7fc10406a6a0 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.739+0000 7fc10b59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10406b7e0 con 0x7fc1040c7920 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.739+0000 7fc10b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10406ae40 con 0x7fc104155af0 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1040c7920 0x7fc1040c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1040c7920 0x7fc1040c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48900/0 (socket says 172.21.15.33:48900) 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 -- 172.21.15.33:0/2009270421 learned_addr learned my addr 172.21.15.33:0/2009270421 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:24.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10a59b700 1 --2- 172.21.15.33:0/2009270421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc104155af0 0x7fc10406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 -- 172.21.15.33:0/2009270421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc104155af0 msgr2=0x7fc10406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 --2- 172.21.15.33:0/2009270421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc104155af0 0x7fc10406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 -- 172.21.15.33:0/2009270421 >> v1:172.21.15.167:6789/0 conn(0x7fc10406a6a0 legacy=0x7fc1040c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 -- 172.21.15.33:0/2009270421 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10406b120 con 0x7fc1040c7920 2023-10-06T05:29:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.740+0000 7fc10ad9c700 1 --2- 172.21.15.33:0/2009270421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c9e20 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c0d0 tx=0x7fc0fc00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2882897c8254eda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc109599700 1 -- 172.21.15.33:0/2009270421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00f070 con 0x7fc1040c7920 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc109599700 1 -- 172.21.15.33:0/2009270421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00cdf0 con 0x7fc1040c7920 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc109599700 1 -- 172.21.15.33:0/2009270421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00a320 con 0x7fc1040c7920 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 -- 172.21.15.33:0/2009270421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 msgr2=0x7fc1040c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2009270421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c9e20 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c0d0 tx=0x7fc0fc00a950 comp rx=0 tx=0).stop 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 -- 172.21.15.33:0/2009270421 shutdown_connections 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2009270421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c9e20 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2009270421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc104155af0 0x7fc10406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 -- 172.21.15.33:0/2009270421 >> 172.21.15.33:0/2009270421 conn(0x7fc10406d030 msgr2=0x7fc10406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:24.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 -- 172.21.15.33:0/2009270421 shutdown_connections 2023-10-06T05:29:24.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.741+0000 7fc10b59d700 1 -- 172.21.15.33:0/2009270421 wait complete. 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.743+0000 7fc10b59d700 1 Processor -- start 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.743+0000 7fc10b59d700 1 -- start start 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10b59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc10406a6a0 0x7fc104146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10b59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10b59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc104155af0 0x7fc1040c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10b59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040aca70 con 0x7fc1040c7920 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10b59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ae030 con 0x7fc104155af0 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10b59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ad700 con 0x7fc10406a6a0 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc109d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc109d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48910/0 (socket says 172.21.15.33:48910) 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc109d9a700 1 -- 172.21.15.33:0/2225237926 learned_addr learned my addr 172.21.15.33:0/2225237926 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10a59b700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc10406a6a0 0x7fc104146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.744+0000 7fc10ad9c700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc104155af0 0x7fc1040c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc109d9a700 1 -- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc10406a6a0 msgr2=0x7fc104146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc109d9a700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc10406a6a0 0x7fc104146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc109d9a700 1 -- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc104155af0 msgr2=0x7fc1040c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc109d9a700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc104155af0 0x7fc1040c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc109d9a700 1 -- 172.21.15.33:0/2225237926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc00abf0 con 0x7fc1040c7920 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc10a59b700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc10406a6a0 0x7fc104146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc10ad9c700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc104155af0 0x7fc1040c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc109d9a700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c52e0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c7c0 tx=0x7fc0f4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4011590 con 0x7fc1040c7920 2023-10-06T05:29:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f40116f0 con 0x7fc1040c7920 2023-10-06T05:29:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4011ab0 con 0x7fc1040c7920 2023-10-06T05:29:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1040ad9e0 con 0x7fc1040c7920 2023-10-06T05:29:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.745+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1040ae4c0 con 0x7fc1040c7920 2023-10-06T05:29:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.746+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1040ae1a0 con 0x7fc1040c7920 2023-10-06T05:29:24.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.747+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0f401b3b0 con 0x7fc1040c7920 2023-10-06T05:29:24.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.748+0000 7fc0fa7fc700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0e8066f30 0x7fc0e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:24.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.748+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0f4083c90 con 0x7fc1040c7920 2023-10-06T05:29:24.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.748+0000 7fc10a59b700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0e8066f30 0x7fc0e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:24.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.749+0000 7fc10a59b700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0e8066f30 0x7fc0e80693e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc100006fd0 tx=0x7fc100008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:24.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.752+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0f4050d00 con 0x7fc1040c7920 2023-10-06T05:29:24.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.951+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc10404c380 con 0x7fc0e8066f30 2023-10-06T05:29:24.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.960+0000 7fc0fa7fc700 1 -- 172.21.15.33:0/2225237926 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311324 (secure 0 0 0) 0x7fc10404c380 con 0x7fc0e8066f30 2023-10-06T05:29:24.967 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0e8066f30 msgr2=0x7fc0e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0e8066f30 0x7fc0e80693e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc100006fd0 tx=0x7fc100008040 comp rx=0 tx=0).stop 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 msgr2=0x7fc1040c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c52e0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c7c0 tx=0x7fc0f4007f50 comp rx=0 tx=0).stop 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 shutdown_connections 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc0e8066f30 0x7fc0e80693e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc104155af0 0x7fc1040c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc10406a6a0 0x7fc104146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 --2- 172.21.15.33:0/2225237926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1040c7920 0x7fc1040c52e0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:24.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 >> 172.21.15.33:0/2225237926 conn(0x7fc10406d030 msgr2=0x7fc104154f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:24.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 shutdown_connections 2023-10-06T05:29:24.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:24.965+0000 7fc10b59d700 1 -- 172.21.15.33:0/2225237926 wait complete. 2023-10-06T05:29:24.971 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:29:24.985 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":659,"stamp":"2023-10-06T05:29:24.044411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432996,"kb_used_data":21924,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752541724,"statfs":{"total":773094113280,"available":770602725376,"internally_reserved":0,"allocated":22450176,"data_stored":11081385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.941597"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.151677+0000","last_change":"2023-10-06T05:29:20.148259+0000","last_active":"2023-10-06T05:29:20.151677+0000","last_peered":"2023-10-06T05:29:20.151677+0000","last_clean":"2023-10-06T05:29:20.151677+0000","last_became_active":"2023-10-06T05:29:20.147981+0000","last_became_peered":"2023-10-06T05:29:20.147981+0000","last_unstale":"2023-10-06T05:29:20.151677+0000","last_undegraded":"2023-10-06T05:29:20.151677+0000","last_fullsized":"2023-10-06T05:29:20.151677+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:39.748897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157534+0000","last_change":"2023-10-06T05:29:20.149065+0000","last_active":"2023-10-06T05:29:20.157534+0000","last_peered":"2023-10-06T05:29:20.157534+0000","last_clean":"2023-10-06T05:29:20.157534+0000","last_became_active":"2023-10-06T05:29:20.148813+0000","last_became_peered":"2023-10-06T05:29:20.148813+0000","last_unstale":"2023-10-06T05:29:20.157534+0000","last_undegraded":"2023-10-06T05:29:20.157534+0000","last_fullsized":"2023-10-06T05:29:20.157534+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:33.221702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152168+0000","last_change":"2023-10-06T05:29:20.144003+0000","last_active":"2023-10-06T05:29:20.152168+0000","last_peered":"2023-10-06T05:29:20.152168+0000","last_clean":"2023-10-06T05:29:20.152168+0000","last_became_active":"2023-10-06T05:29:20.143694+0000","last_became_peered":"2023-10-06T05:29:20.143694+0000","last_unstale":"2023-10-06T05:29:20.152168+0000","last_undegraded":"2023-10-06T05:29:20.152168+0000","last_fullsized":"2023-10-06T05:29:20.152168+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:13.493549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152153+0000","last_change":"2023-10-06T05:29:20.147626+0000","last_active":"2023-10-06T05:29:20.152153+0000","last_peered":"2023-10-06T05:29:20.152153+0000","last_clean":"2023-10-06T05:29:20.152153+0000","last_became_active":"2023-10-06T05:29:20.147362+0000","last_became_peered":"2023-10-06T05:29:20.147362+0000","last_unstale":"2023-10-06T05:29:20.152153+0000","last_undegraded":"2023-10-06T05:29:20.152153+0000","last_fullsized":"2023-10-06T05:29:20.152153+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:28.292786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148670+0000","last_change":"2023-10-06T05:29:20.142749+0000","last_active":"2023-10-06T05:29:20.148670+0000","last_peered":"2023-10-06T05:29:20.148670+0000","last_clean":"2023-10-06T05:29:20.148670+0000","last_became_active":"2023-10-06T05:29:20.141280+0000","last_became_peered":"2023-10-06T05:29:20.141280+0000","last_unstale":"2023-10-06T05:29:20.148670+0000","last_undegraded":"2023-10-06T05:29:20.148670+0000","last_fullsized":"2023-10-06T05:29:20.148670+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:43.968575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142259+0000","last_change":"2023-10-06T05:29:20.129470+0000","last_active":"2023-10-06T05:29:20.142259+0000","last_peered":"2023-10-06T05:29:20.142259+0000","last_clean":"2023-10-06T05:29:20.142259+0000","last_became_active":"2023-10-06T05:29:20.129135+0000","last_became_peered":"2023-10-06T05:29:20.129135+0000","last_unstale":"2023-10-06T05:29:20.142259+0000","last_undegraded":"2023-10-06T05:29:20.142259+0000","last_fullsized":"2023-10-06T05:29:20.142259+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:29.764934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142843+0000","last_change":"2023-10-06T05:29:20.140046+0000","last_active":"2023-10-06T05:29:20.142843+0000","last_peered":"2023-10-06T05:29:20.142843+0000","last_clean":"2023-10-06T05:29:20.142843+0000","last_became_active":"2023-10-06T05:29:20.139386+0000","last_became_peered":"2023-10-06T05:29:20.139386+0000","last_unstale":"2023-10-06T05:29:20.142843+0000","last_undegraded":"2023-10-06T05:29:20.142843+0000","last_fullsized":"2023-10-06T05:29:20.142843+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:23.361080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134527+0000","last_change":"2023-10-06T05:29:20.133263+0000","last_active":"2023-10-06T05:29:20.134527+0000","last_peered":"2023-10-06T05:29:20.134527+0000","last_clean":"2023-10-06T05:29:20.134527+0000","last_became_active":"2023-10-06T05:29:20.132987+0000","last_became_peered":"2023-10-06T05:29:20.132987+0000","last_unstale":"2023-10-06T05:29:20.134527+0000","last_undegraded":"2023-10-06T05:29:20.134527+0000","last_fullsized":"2023-10-06T05:29:20.134527+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:25.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142417+0000","last_change":"2023-10-06T05:29:20.136532+0000","last_active":"2023-10-06T05:29:20.142417+0000","last_peered":"2023-10-06T05:29:20.142417+0000","last_clean":"2023-10-06T05:29:20.142417+0000","last_became_active":"2023-10-06T05:29:20.130776+0000","last_became_peered":"2023-10-06T05:29:20.130776+0000","last_unstale":"2023-10-06T05:29:20.142417+0000","last_undegraded":"2023-10-06T05:29:20.142417+0000","last_fullsized":"2023-10-06T05:29:20.142417+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:48.943820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155721+0000","last_change":"2023-10-06T05:29:20.150764+0000","last_active":"2023-10-06T05:29:20.155721+0000","last_peered":"2023-10-06T05:29:20.155721+0000","last_clean":"2023-10-06T05:29:20.155721+0000","last_became_active":"2023-10-06T05:29:20.150512+0000","last_became_peered":"2023-10-06T05:29:20.150512+0000","last_unstale":"2023-10-06T05:29:20.155721+0000","last_undegraded":"2023-10-06T05:29:20.155721+0000","last_fullsized":"2023-10-06T05:29:20.155721+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:11.016883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155696+0000","last_change":"2023-10-06T05:29:20.149546+0000","last_active":"2023-10-06T05:29:20.155696+0000","last_peered":"2023-10-06T05:29:20.155696+0000","last_clean":"2023-10-06T05:29:20.155696+0000","last_became_active":"2023-10-06T05:29:20.149295+0000","last_became_peered":"2023-10-06T05:29:20.149295+0000","last_unstale":"2023-10-06T05:29:20.155696+0000","last_undegraded":"2023-10-06T05:29:20.155696+0000","last_fullsized":"2023-10-06T05:29:20.155696+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.899449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148119+0000","last_change":"2023-10-06T05:29:20.142968+0000","last_active":"2023-10-06T05:29:20.148119+0000","last_peered":"2023-10-06T05:29:20.148119+0000","last_clean":"2023-10-06T05:29:20.148119+0000","last_became_active":"2023-10-06T05:29:20.142709+0000","last_became_peered":"2023-10-06T05:29:20.142709+0000","last_unstale":"2023-10-06T05:29:20.148119+0000","last_undegraded":"2023-10-06T05:29:20.148119+0000","last_fullsized":"2023-10-06T05:29:20.148119+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:48.713611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.153048+0000","last_change":"2023-10-06T05:29:20.148877+0000","last_active":"2023-10-06T05:29:20.153048+0000","last_peered":"2023-10-06T05:29:20.153048+0000","last_clean":"2023-10-06T05:29:20.153048+0000","last_became_active":"2023-10-06T05:29:20.148597+0000","last_became_peered":"2023-10-06T05:29:20.148597+0000","last_unstale":"2023-10-06T05:29:20.153048+0000","last_undegraded":"2023-10-06T05:29:20.153048+0000","last_fullsized":"2023-10-06T05:29:20.153048+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:48.156668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144406+0000","last_change":"2023-10-06T05:29:20.139828+0000","last_active":"2023-10-06T05:29:20.144406+0000","last_peered":"2023-10-06T05:29:20.144406+0000","last_clean":"2023-10-06T05:29:20.144406+0000","last_became_active":"2023-10-06T05:29:20.139490+0000","last_became_peered":"2023-10-06T05:29:20.139490+0000","last_unstale":"2023-10-06T05:29:20.144406+0000","last_undegraded":"2023-10-06T05:29:20.144406+0000","last_fullsized":"2023-10-06T05:29:20.144406+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:48.713611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.145709+0000","last_change":"2023-10-06T05:29:20.143500+0000","last_active":"2023-10-06T05:29:20.145709+0000","last_peered":"2023-10-06T05:29:20.145709+0000","last_clean":"2023-10-06T05:29:20.145709+0000","last_became_active":"2023-10-06T05:29:20.143118+0000","last_became_peered":"2023-10-06T05:29:20.143118+0000","last_unstale":"2023-10-06T05:29:20.145709+0000","last_undegraded":"2023-10-06T05:29:20.145709+0000","last_fullsized":"2023-10-06T05:29:20.145709+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:50.376311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.158303+0000","last_change":"2023-10-06T05:29:20.153625+0000","last_active":"2023-10-06T05:29:20.158303+0000","last_peered":"2023-10-06T05:29:20.158303+0000","last_clean":"2023-10-06T05:29:20.158303+0000","last_became_active":"2023-10-06T05:29:20.153362+0000","last_became_peered":"2023-10-06T05:29:20.153362+0000","last_unstale":"2023-10-06T05:29:20.158303+0000","last_undegraded":"2023-10-06T05:29:20.158303+0000","last_fullsized":"2023-10-06T05:29:20.158303+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:28.561696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152615+0000","last_change":"2023-10-06T05:29:20.151079+0000","last_active":"2023-10-06T05:29:20.152615+0000","last_peered":"2023-10-06T05:29:20.152615+0000","last_clean":"2023-10-06T05:29:20.152615+0000","last_became_active":"2023-10-06T05:29:20.150492+0000","last_became_peered":"2023-10-06T05:29:20.150492+0000","last_unstale":"2023-10-06T05:29:20.152615+0000","last_undegraded":"2023-10-06T05:29:20.152615+0000","last_fullsized":"2023-10-06T05:29:20.152615+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:28.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152217+0000","last_change":"2023-10-06T05:29:20.147400+0000","last_active":"2023-10-06T05:29:20.152217+0000","last_peered":"2023-10-06T05:29:20.152217+0000","last_clean":"2023-10-06T05:29:20.152217+0000","last_became_active":"2023-10-06T05:29:20.147081+0000","last_became_peered":"2023-10-06T05:29:20.147081+0000","last_unstale":"2023-10-06T05:29:20.152217+0000","last_undegraded":"2023-10-06T05:29:20.152217+0000","last_fullsized":"2023-10-06T05:29:20.152217+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:29.147674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.159493+0000","last_change":"2023-10-06T05:29:20.155549+0000","last_active":"2023-10-06T05:29:20.159493+0000","last_peered":"2023-10-06T05:29:20.159493+0000","last_clean":"2023-10-06T05:29:20.159493+0000","last_became_active":"2023-10-06T05:29:20.155236+0000","last_became_peered":"2023-10-06T05:29:20.155236+0000","last_unstale":"2023-10-06T05:29:20.159493+0000","last_undegraded":"2023-10-06T05:29:20.159493+0000","last_fullsized":"2023-10-06T05:29:20.159493+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:56.660751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.140336+0000","last_change":"2023-10-06T05:29:20.136861+0000","last_active":"2023-10-06T05:29:20.140336+0000","last_peered":"2023-10-06T05:29:20.140336+0000","last_clean":"2023-10-06T05:29:20.140336+0000","last_became_active":"2023-10-06T05:29:20.136444+0000","last_became_peered":"2023-10-06T05:29:20.136444+0000","last_unstale":"2023-10-06T05:29:20.140336+0000","last_undegraded":"2023-10-06T05:29:20.140336+0000","last_fullsized":"2023-10-06T05:29:20.140336+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:30.399866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.138371+0000","last_change":"2023-10-06T05:29:20.133119+0000","last_active":"2023-10-06T05:29:20.138371+0000","last_peered":"2023-10-06T05:29:20.138371+0000","last_clean":"2023-10-06T05:29:20.138371+0000","last_became_active":"2023-10-06T05:29:20.132847+0000","last_became_peered":"2023-10-06T05:29:20.132847+0000","last_unstale":"2023-10-06T05:29:20.138371+0000","last_undegraded":"2023-10-06T05:29:20.138371+0000","last_fullsized":"2023-10-06T05:29:20.138371+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:27.823924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152664+0000","last_change":"2023-10-06T05:29:20.148875+0000","last_active":"2023-10-06T05:29:20.152664+0000","last_peered":"2023-10-06T05:29:20.152664+0000","last_clean":"2023-10-06T05:29:20.152664+0000","last_became_active":"2023-10-06T05:29:20.147477+0000","last_became_peered":"2023-10-06T05:29:20.147477+0000","last_unstale":"2023-10-06T05:29:20.152664+0000","last_undegraded":"2023-10-06T05:29:20.152664+0000","last_fullsized":"2023-10-06T05:29:20.152664+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:27.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152253+0000","last_change":"2023-10-06T05:29:20.150540+0000","last_active":"2023-10-06T05:29:20.152253+0000","last_peered":"2023-10-06T05:29:20.152253+0000","last_clean":"2023-10-06T05:29:20.152253+0000","last_became_active":"2023-10-06T05:29:20.150264+0000","last_became_peered":"2023-10-06T05:29:20.150264+0000","last_unstale":"2023-10-06T05:29:20.152253+0000","last_undegraded":"2023-10-06T05:29:20.152253+0000","last_fullsized":"2023-10-06T05:29:20.152253+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:20.401755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148647+0000","last_change":"2023-10-06T05:29:20.141677+0000","last_active":"2023-10-06T05:29:20.148647+0000","last_peered":"2023-10-06T05:29:20.148647+0000","last_clean":"2023-10-06T05:29:20.148647+0000","last_became_active":"2023-10-06T05:29:20.141395+0000","last_became_peered":"2023-10-06T05:29:20.141395+0000","last_unstale":"2023-10-06T05:29:20.148647+0000","last_undegraded":"2023-10-06T05:29:20.148647+0000","last_fullsized":"2023-10-06T05:29:20.148647+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:33.897100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.147505+0000","last_change":"2023-10-06T05:29:20.146356+0000","last_active":"2023-10-06T05:29:20.147505+0000","last_peered":"2023-10-06T05:29:20.147505+0000","last_clean":"2023-10-06T05:29:20.147505+0000","last_became_active":"2023-10-06T05:29:20.145994+0000","last_became_peered":"2023-10-06T05:29:20.145994+0000","last_unstale":"2023-10-06T05:29:20.147505+0000","last_undegraded":"2023-10-06T05:29:20.147505+0000","last_fullsized":"2023-10-06T05:29:20.147505+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:34.146881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142468+0000","last_change":"2023-10-06T05:29:20.140556+0000","last_active":"2023-10-06T05:29:20.142468+0000","last_peered":"2023-10-06T05:29:20.142468+0000","last_clean":"2023-10-06T05:29:20.142468+0000","last_became_active":"2023-10-06T05:29:20.140231+0000","last_became_peered":"2023-10-06T05:29:20.140231+0000","last_unstale":"2023-10-06T05:29:20.142468+0000","last_undegraded":"2023-10-06T05:29:20.142468+0000","last_fullsized":"2023-10-06T05:29:20.142468+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:33.897100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.149064+0000","last_change":"2023-10-06T05:29:20.145949+0000","last_active":"2023-10-06T05:29:20.149064+0000","last_peered":"2023-10-06T05:29:20.149064+0000","last_clean":"2023-10-06T05:29:20.149064+0000","last_became_active":"2023-10-06T05:29:20.145492+0000","last_became_peered":"2023-10-06T05:29:20.145492+0000","last_unstale":"2023-10-06T05:29:20.149064+0000","last_undegraded":"2023-10-06T05:29:20.149064+0000","last_fullsized":"2023-10-06T05:29:20.149064+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:45.216186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142338+0000","last_change":"2023-10-06T05:29:20.130304+0000","last_active":"2023-10-06T05:29:20.142338+0000","last_peered":"2023-10-06T05:29:20.142338+0000","last_clean":"2023-10-06T05:29:20.142338+0000","last_became_active":"2023-10-06T05:29:20.130043+0000","last_became_peered":"2023-10-06T05:29:20.130043+0000","last_unstale":"2023-10-06T05:29:20.142338+0000","last_undegraded":"2023-10-06T05:29:20.142338+0000","last_fullsized":"2023-10-06T05:29:20.142338+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:21.683933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.145919+0000","last_change":"2023-10-06T05:29:20.141091+0000","last_active":"2023-10-06T05:29:20.145919+0000","last_peered":"2023-10-06T05:29:20.145919+0000","last_clean":"2023-10-06T05:29:20.145919+0000","last_became_active":"2023-10-06T05:29:20.140811+0000","last_became_peered":"2023-10-06T05:29:20.140811+0000","last_unstale":"2023-10-06T05:29:20.145919+0000","last_undegraded":"2023-10-06T05:29:20.145919+0000","last_fullsized":"2023-10-06T05:29:20.145919+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:39.748897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.141044+0000","last_change":"2023-10-06T05:29:20.126709+0000","last_active":"2023-10-06T05:29:20.141044+0000","last_peered":"2023-10-06T05:29:20.141044+0000","last_clean":"2023-10-06T05:29:20.141044+0000","last_became_active":"2023-10-06T05:29:20.126411+0000","last_became_peered":"2023-10-06T05:29:20.126411+0000","last_unstale":"2023-10-06T05:29:20.141044+0000","last_undegraded":"2023-10-06T05:29:20.141044+0000","last_fullsized":"2023-10-06T05:29:20.141044+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:29.764934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.140513+0000","last_change":"2023-10-06T05:29:20.139090+0000","last_active":"2023-10-06T05:29:20.140513+0000","last_peered":"2023-10-06T05:29:20.140513+0000","last_clean":"2023-10-06T05:29:20.140513+0000","last_became_active":"2023-10-06T05:29:20.138426+0000","last_became_peered":"2023-10-06T05:29:20.138426+0000","last_unstale":"2023-10-06T05:29:20.140513+0000","last_undegraded":"2023-10-06T05:29:20.140513+0000","last_fullsized":"2023-10-06T05:29:20.140513+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:58.850616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142309+0000","last_change":"2023-10-06T05:29:20.131355+0000","last_active":"2023-10-06T05:29:20.142309+0000","last_peered":"2023-10-06T05:29:20.142309+0000","last_clean":"2023-10-06T05:29:20.142309+0000","last_became_active":"2023-10-06T05:29:20.131018+0000","last_became_peered":"2023-10-06T05:29:20.131018+0000","last_unstale":"2023-10-06T05:29:20.142309+0000","last_undegraded":"2023-10-06T05:29:20.142309+0000","last_fullsized":"2023-10-06T05:29:20.142309+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:19.992039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144320+0000","last_change":"2023-10-06T05:29:20.142170+0000","last_active":"2023-10-06T05:29:20.144320+0000","last_peered":"2023-10-06T05:29:20.144320+0000","last_clean":"2023-10-06T05:29:20.144320+0000","last_became_active":"2023-10-06T05:29:20.141675+0000","last_became_peered":"2023-10-06T05:29:20.141675+0000","last_unstale":"2023-10-06T05:29:20.144320+0000","last_undegraded":"2023-10-06T05:29:20.144320+0000","last_fullsized":"2023-10-06T05:29:20.144320+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:28.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122758+0000","last_change":"2023-10-06T05:29:19.124506+0000","last_active":"2023-10-06T05:29:20.122758+0000","last_peered":"2023-10-06T05:29:20.122758+0000","last_clean":"2023-10-06T05:29:20.122758+0000","last_became_active":"2023-10-06T05:29:19.124229+0000","last_became_peered":"2023-10-06T05:29:19.124229+0000","last_unstale":"2023-10-06T05:29:20.122758+0000","last_undegraded":"2023-10-06T05:29:20.122758+0000","last_fullsized":"2023-10-06T05:29:20.122758+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:32.164964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.149966+0000","last_change":"2023-10-06T05:29:20.144895+0000","last_active":"2023-10-06T05:29:20.149966+0000","last_peered":"2023-10-06T05:29:20.149966+0000","last_clean":"2023-10-06T05:29:20.149966+0000","last_became_active":"2023-10-06T05:29:20.144598+0000","last_became_peered":"2023-10-06T05:29:20.144598+0000","last_unstale":"2023-10-06T05:29:20.149966+0000","last_undegraded":"2023-10-06T05:29:20.149966+0000","last_fullsized":"2023-10-06T05:29:20.149966+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:27.823924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.139078+0000","last_change":"2023-10-06T05:29:19.120534+0000","last_active":"2023-10-06T05:29:20.139078+0000","last_peered":"2023-10-06T05:29:20.139078+0000","last_clean":"2023-10-06T05:29:20.139078+0000","last_became_active":"2023-10-06T05:29:19.120256+0000","last_became_peered":"2023-10-06T05:29:19.120256+0000","last_unstale":"2023-10-06T05:29:20.139078+0000","last_undegraded":"2023-10-06T05:29:20.139078+0000","last_fullsized":"2023-10-06T05:29:20.139078+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:54.808874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152291+0000","last_change":"2023-10-06T05:29:20.147952+0000","last_active":"2023-10-06T05:29:20.152291+0000","last_peered":"2023-10-06T05:29:20.152291+0000","last_clean":"2023-10-06T05:29:20.152291+0000","last_became_active":"2023-10-06T05:29:20.146719+0000","last_became_peered":"2023-10-06T05:29:20.146719+0000","last_unstale":"2023-10-06T05:29:20.152291+0000","last_undegraded":"2023-10-06T05:29:20.152291+0000","last_fullsized":"2023-10-06T05:29:20.152291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:56.660751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"209'3","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.153287+0000","last_change":"2023-10-06T05:29:19.124209+0000","last_active":"2023-10-06T05:29:20.153287+0000","last_peered":"2023-10-06T05:29:20.153287+0000","last_clean":"2023-10-06T05:29:20.153287+0000","last_became_active":"2023-10-06T05:29:19.123770+0000","last_became_peered":"2023-10-06T05:29:19.123770+0000","last_unstale":"2023-10-06T05:29:20.153287+0000","last_undegraded":"2023-10-06T05:29:20.153287+0000","last_fullsized":"2023-10-06T05:29:20.153287+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:10.185226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.147116+0000","last_change":"2023-10-06T05:29:20.144871+0000","last_active":"2023-10-06T05:29:20.147116+0000","last_peered":"2023-10-06T05:29:20.147116+0000","last_clean":"2023-10-06T05:29:20.147116+0000","last_became_active":"2023-10-06T05:29:20.144550+0000","last_became_peered":"2023-10-06T05:29:20.144550+0000","last_unstale":"2023-10-06T05:29:20.147116+0000","last_undegraded":"2023-10-06T05:29:20.147116+0000","last_fullsized":"2023-10-06T05:29:20.147116+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:00.967071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"209'6","reported_seq":16,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.166525+0000","last_change":"2023-10-06T05:29:19.115617+0000","last_active":"2023-10-06T05:29:20.166525+0000","last_peered":"2023-10-06T05:29:20.166525+0000","last_clean":"2023-10-06T05:29:20.166525+0000","last_became_active":"2023-10-06T05:29:19.115323+0000","last_became_peered":"2023-10-06T05:29:19.115323+0000","last_unstale":"2023-10-06T05:29:20.166525+0000","last_undegraded":"2023-10-06T05:29:20.166525+0000","last_fullsized":"2023-10-06T05:29:20.166525+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:52.004531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157582+0000","last_change":"2023-10-06T05:29:20.148463+0000","last_active":"2023-10-06T05:29:20.157582+0000","last_peered":"2023-10-06T05:29:20.157582+0000","last_clean":"2023-10-06T05:29:20.157582+0000","last_became_active":"2023-10-06T05:29:20.148148+0000","last_became_peered":"2023-10-06T05:29:20.148148+0000","last_unstale":"2023-10-06T05:29:20.157582+0000","last_undegraded":"2023-10-06T05:29:20.157582+0000","last_fullsized":"2023-10-06T05:29:20.157582+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:50.357267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155676+0000","last_change":"2023-10-06T05:29:19.116548+0000","last_active":"2023-10-06T05:29:20.155676+0000","last_peered":"2023-10-06T05:29:20.155676+0000","last_clean":"2023-10-06T05:29:20.155676+0000","last_became_active":"2023-10-06T05:29:19.116271+0000","last_became_peered":"2023-10-06T05:29:19.116271+0000","last_unstale":"2023-10-06T05:29:20.155676+0000","last_undegraded":"2023-10-06T05:29:20.155676+0000","last_fullsized":"2023-10-06T05:29:20.155676+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:22.456167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.145348+0000","last_change":"2023-10-06T05:29:20.142745+0000","last_active":"2023-10-06T05:29:20.145348+0000","last_peered":"2023-10-06T05:29:20.145348+0000","last_clean":"2023-10-06T05:29:20.145348+0000","last_became_active":"2023-10-06T05:29:20.142481+0000","last_became_peered":"2023-10-06T05:29:20.142481+0000","last_unstale":"2023-10-06T05:29:20.145348+0000","last_undegraded":"2023-10-06T05:29:20.145348+0000","last_fullsized":"2023-10-06T05:29:20.145348+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:05.082060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122335+0000","last_change":"2023-10-06T05:29:19.128908+0000","last_active":"2023-10-06T05:29:20.122335+0000","last_peered":"2023-10-06T05:29:20.122335+0000","last_clean":"2023-10-06T05:29:20.122335+0000","last_became_active":"2023-10-06T05:29:19.128627+0000","last_became_peered":"2023-10-06T05:29:19.128627+0000","last_unstale":"2023-10-06T05:29:20.122335+0000","last_undegraded":"2023-10-06T05:29:20.122335+0000","last_fullsized":"2023-10-06T05:29:20.122335+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:33.829071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.149351+0000","last_change":"2023-10-06T05:29:20.146303+0000","last_active":"2023-10-06T05:29:20.149351+0000","last_peered":"2023-10-06T05:29:20.149351+0000","last_clean":"2023-10-06T05:29:20.149351+0000","last_became_active":"2023-10-06T05:29:20.145976+0000","last_became_peered":"2023-10-06T05:29:20.145976+0000","last_unstale":"2023-10-06T05:29:20.149351+0000","last_undegraded":"2023-10-06T05:29:20.149351+0000","last_fullsized":"2023-10-06T05:29:20.149351+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:49.899449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122729+0000","last_change":"2023-10-06T05:29:19.120038+0000","last_active":"2023-10-06T05:29:20.122729+0000","last_peered":"2023-10-06T05:29:20.122729+0000","last_clean":"2023-10-06T05:29:20.122729+0000","last_became_active":"2023-10-06T05:29:19.119774+0000","last_became_peered":"2023-10-06T05:29:19.119774+0000","last_unstale":"2023-10-06T05:29:20.122729+0000","last_undegraded":"2023-10-06T05:29:20.122729+0000","last_fullsized":"2023-10-06T05:29:20.122729+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:14.771802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.143563+0000","last_change":"2023-10-06T05:29:20.137235+0000","last_active":"2023-10-06T05:29:20.143563+0000","last_peered":"2023-10-06T05:29:20.143563+0000","last_clean":"2023-10-06T05:29:20.143563+0000","last_became_active":"2023-10-06T05:29:20.136977+0000","last_became_peered":"2023-10-06T05:29:20.136977+0000","last_unstale":"2023-10-06T05:29:20.143563+0000","last_undegraded":"2023-10-06T05:29:20.143563+0000","last_fullsized":"2023-10-06T05:29:20.143563+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:34.990845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"209'8","reported_seq":18,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157582+0000","last_change":"2023-10-06T05:29:19.118674+0000","last_active":"2023-10-06T05:29:20.157582+0000","last_peered":"2023-10-06T05:29:20.157582+0000","last_clean":"2023-10-06T05:29:20.157582+0000","last_became_active":"2023-10-06T05:29:19.118418+0000","last_became_peered":"2023-10-06T05:29:19.118418+0000","last_unstale":"2023-10-06T05:29:20.157582+0000","last_undegraded":"2023-10-06T05:29:20.157582+0000","last_fullsized":"2023-10-06T05:29:20.157582+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:21.078436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148168+0000","last_change":"2023-10-06T05:29:20.141146+0000","last_active":"2023-10-06T05:29:20.148168+0000","last_peered":"2023-10-06T05:29:20.148168+0000","last_clean":"2023-10-06T05:29:20.148168+0000","last_became_active":"2023-10-06T05:29:20.140836+0000","last_became_peered":"2023-10-06T05:29:20.140836+0000","last_unstale":"2023-10-06T05:29:20.148168+0000","last_undegraded":"2023-10-06T05:29:20.148168+0000","last_fullsized":"2023-10-06T05:29:20.148168+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:55.933108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.150552+0000","last_change":"2023-10-06T05:29:19.124764+0000","last_active":"2023-10-06T05:29:20.150552+0000","last_peered":"2023-10-06T05:29:20.150552+0000","last_clean":"2023-10-06T05:29:20.150552+0000","last_became_active":"2023-10-06T05:29:19.124439+0000","last_became_peered":"2023-10-06T05:29:19.124439+0000","last_unstale":"2023-10-06T05:29:20.150552+0000","last_undegraded":"2023-10-06T05:29:20.150552+0000","last_fullsized":"2023-10-06T05:29:20.150552+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:55.684382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148413+0000","last_change":"2023-10-06T05:29:20.142379+0000","last_active":"2023-10-06T05:29:20.148413+0000","last_peered":"2023-10-06T05:29:20.148413+0000","last_clean":"2023-10-06T05:29:20.148413+0000","last_became_active":"2023-10-06T05:29:20.141214+0000","last_became_peered":"2023-10-06T05:29:20.141214+0000","last_unstale":"2023-10-06T05:29:20.148413+0000","last_undegraded":"2023-10-06T05:29:20.148413+0000","last_fullsized":"2023-10-06T05:29:20.148413+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:17.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148089+0000","last_change":"2023-10-06T05:29:19.121504+0000","last_active":"2023-10-06T05:29:20.148089+0000","last_peered":"2023-10-06T05:29:20.148089+0000","last_clean":"2023-10-06T05:29:20.148089+0000","last_became_active":"2023-10-06T05:29:19.121244+0000","last_became_peered":"2023-10-06T05:29:19.121244+0000","last_unstale":"2023-10-06T05:29:20.148089+0000","last_undegraded":"2023-10-06T05:29:20.148089+0000","last_fullsized":"2023-10-06T05:29:20.148089+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:45.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.140548+0000","last_change":"2023-10-06T05:29:20.136254+0000","last_active":"2023-10-06T05:29:20.140548+0000","last_peered":"2023-10-06T05:29:20.140548+0000","last_clean":"2023-10-06T05:29:20.140548+0000","last_became_active":"2023-10-06T05:29:20.135984+0000","last_became_peered":"2023-10-06T05:29:20.135984+0000","last_unstale":"2023-10-06T05:29:20.140548+0000","last_undegraded":"2023-10-06T05:29:20.140548+0000","last_fullsized":"2023-10-06T05:29:20.140548+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:16.688762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.159719+0000","last_change":"2023-10-06T05:29:19.128302+0000","last_active":"2023-10-06T05:29:20.159719+0000","last_peered":"2023-10-06T05:29:20.159719+0000","last_clean":"2023-10-06T05:29:20.159719+0000","last_became_active":"2023-10-06T05:29:19.128011+0000","last_became_peered":"2023-10-06T05:29:19.128011+0000","last_unstale":"2023-10-06T05:29:20.159719+0000","last_undegraded":"2023-10-06T05:29:20.159719+0000","last_fullsized":"2023-10-06T05:29:20.159719+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:24.687020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155722+0000","last_change":"2023-10-06T05:29:20.150181+0000","last_active":"2023-10-06T05:29:20.155722+0000","last_peered":"2023-10-06T05:29:20.155722+0000","last_clean":"2023-10-06T05:29:20.155722+0000","last_became_active":"2023-10-06T05:29:20.149936+0000","last_became_peered":"2023-10-06T05:29:20.149936+0000","last_unstale":"2023-10-06T05:29:20.155722+0000","last_undegraded":"2023-10-06T05:29:20.155722+0000","last_fullsized":"2023-10-06T05:29:20.155722+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:59.102372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.129659+0000","last_change":"2023-10-06T05:29:19.122512+0000","last_active":"2023-10-06T05:29:20.129659+0000","last_peered":"2023-10-06T05:29:20.129659+0000","last_clean":"2023-10-06T05:29:20.129659+0000","last_became_active":"2023-10-06T05:29:19.122215+0000","last_became_peered":"2023-10-06T05:29:19.122215+0000","last_unstale":"2023-10-06T05:29:20.129659+0000","last_undegraded":"2023-10-06T05:29:20.129659+0000","last_fullsized":"2023-10-06T05:29:20.129659+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:17:09.193150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157645+0000","last_change":"2023-10-06T05:29:20.154260+0000","last_active":"2023-10-06T05:29:20.157645+0000","last_peered":"2023-10-06T05:29:20.157645+0000","last_clean":"2023-10-06T05:29:20.157645+0000","last_became_active":"2023-10-06T05:29:20.153993+0000","last_became_peered":"2023-10-06T05:29:20.153993+0000","last_unstale":"2023-10-06T05:29:20.157645+0000","last_undegraded":"2023-10-06T05:29:20.157645+0000","last_fullsized":"2023-10-06T05:29:20.157645+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:04.392114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.154514+0000","last_change":"2023-10-06T05:29:19.123872+0000","last_active":"2023-10-06T05:29:20.154514+0000","last_peered":"2023-10-06T05:29:20.154514+0000","last_clean":"2023-10-06T05:29:20.154514+0000","last_became_active":"2023-10-06T05:29:19.123598+0000","last_became_peered":"2023-10-06T05:29:19.123598+0000","last_unstale":"2023-10-06T05:29:20.154514+0000","last_undegraded":"2023-10-06T05:29:20.154514+0000","last_fullsized":"2023-10-06T05:29:20.154514+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:48.165151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157772+0000","last_change":"2023-10-06T05:29:20.152714+0000","last_active":"2023-10-06T05:29:20.157772+0000","last_peered":"2023-10-06T05:29:20.157772+0000","last_clean":"2023-10-06T05:29:20.157772+0000","last_became_active":"2023-10-06T05:29:20.152473+0000","last_became_peered":"2023-10-06T05:29:20.152473+0000","last_unstale":"2023-10-06T05:29:20.157772+0000","last_undegraded":"2023-10-06T05:29:20.157772+0000","last_fullsized":"2023-10-06T05:29:20.157772+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:00.118747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134024+0000","last_change":"2023-10-06T05:29:19.128363+0000","last_active":"2023-10-06T05:29:20.134024+0000","last_peered":"2023-10-06T05:29:20.134024+0000","last_clean":"2023-10-06T05:29:20.134024+0000","last_became_active":"2023-10-06T05:29:19.128072+0000","last_became_peered":"2023-10-06T05:29:19.128072+0000","last_unstale":"2023-10-06T05:29:20.134024+0000","last_undegraded":"2023-10-06T05:29:20.134024+0000","last_fullsized":"2023-10-06T05:29:20.134024+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:17.458208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152636+0000","last_change":"2023-10-06T05:29:20.150928+0000","last_active":"2023-10-06T05:29:20.152636+0000","last_peered":"2023-10-06T05:29:20.152636+0000","last_clean":"2023-10-06T05:29:20.152636+0000","last_became_active":"2023-10-06T05:29:20.150274+0000","last_became_peered":"2023-10-06T05:29:20.150274+0000","last_unstale":"2023-10-06T05:29:20.152636+0000","last_undegraded":"2023-10-06T05:29:20.152636+0000","last_fullsized":"2023-10-06T05:29:20.152636+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:47.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.128911+0000","last_change":"2023-10-06T05:29:19.127593+0000","last_active":"2023-10-06T05:29:20.128911+0000","last_peered":"2023-10-06T05:29:20.128911+0000","last_clean":"2023-10-06T05:29:20.128911+0000","last_became_active":"2023-10-06T05:29:19.126087+0000","last_became_peered":"2023-10-06T05:29:19.126087+0000","last_unstale":"2023-10-06T05:29:20.128911+0000","last_undegraded":"2023-10-06T05:29:20.128911+0000","last_fullsized":"2023-10-06T05:29:20.128911+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:10.505144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152589+0000","last_change":"2023-10-06T05:29:20.150750+0000","last_active":"2023-10-06T05:29:20.152589+0000","last_peered":"2023-10-06T05:29:20.152589+0000","last_clean":"2023-10-06T05:29:20.152589+0000","last_became_active":"2023-10-06T05:29:20.150121+0000","last_became_peered":"2023-10-06T05:29:20.150121+0000","last_unstale":"2023-10-06T05:29:20.152589+0000","last_undegraded":"2023-10-06T05:29:20.152589+0000","last_fullsized":"2023-10-06T05:29:20.152589+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:05.082060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.121568+0000","last_change":"2023-10-06T05:29:19.123013+0000","last_active":"2023-10-06T05:29:20.121568+0000","last_peered":"2023-10-06T05:29:20.121568+0000","last_clean":"2023-10-06T05:29:20.121568+0000","last_became_active":"2023-10-06T05:29:19.122688+0000","last_became_peered":"2023-10-06T05:29:19.122688+0000","last_unstale":"2023-10-06T05:29:20.121568+0000","last_undegraded":"2023-10-06T05:29:20.121568+0000","last_fullsized":"2023-10-06T05:29:20.121568+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:40.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142262+0000","last_change":"2023-10-06T05:29:20.139762+0000","last_active":"2023-10-06T05:29:20.142262+0000","last_peered":"2023-10-06T05:29:20.142262+0000","last_clean":"2023-10-06T05:29:20.142262+0000","last_became_active":"2023-10-06T05:29:20.139445+0000","last_became_peered":"2023-10-06T05:29:20.139445+0000","last_unstale":"2023-10-06T05:29:20.142262+0000","last_undegraded":"2023-10-06T05:29:20.142262+0000","last_fullsized":"2023-10-06T05:29:20.142262+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:27.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.153660+0000","last_change":"2023-10-06T05:29:19.116860+0000","last_active":"2023-10-06T05:29:20.153660+0000","last_peered":"2023-10-06T05:29:20.153660+0000","last_clean":"2023-10-06T05:29:20.153660+0000","last_became_active":"2023-10-06T05:29:19.116528+0000","last_became_peered":"2023-10-06T05:29:19.116528+0000","last_unstale":"2023-10-06T05:29:20.153660+0000","last_undegraded":"2023-10-06T05:29:20.153660+0000","last_fullsized":"2023-10-06T05:29:20.153660+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:30.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142749+0000","last_change":"2023-10-06T05:29:20.138146+0000","last_active":"2023-10-06T05:29:20.142749+0000","last_peered":"2023-10-06T05:29:20.142749+0000","last_clean":"2023-10-06T05:29:20.142749+0000","last_became_active":"2023-10-06T05:29:20.137859+0000","last_became_peered":"2023-10-06T05:29:20.137859+0000","last_unstale":"2023-10-06T05:29:20.142749+0000","last_undegraded":"2023-10-06T05:29:20.142749+0000","last_fullsized":"2023-10-06T05:29:20.142749+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:28.561696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122407+0000","last_change":"2023-10-06T05:29:19.126125+0000","last_active":"2023-10-06T05:29:20.122407+0000","last_peered":"2023-10-06T05:29:20.122407+0000","last_clean":"2023-10-06T05:29:20.122407+0000","last_became_active":"2023-10-06T05:29:19.125186+0000","last_became_peered":"2023-10-06T05:29:19.125186+0000","last_unstale":"2023-10-06T05:29:20.122407+0000","last_undegraded":"2023-10-06T05:29:20.122407+0000","last_fullsized":"2023-10-06T05:29:20.122407+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:33.119207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155686+0000","last_change":"2023-10-06T05:29:20.151574+0000","last_active":"2023-10-06T05:29:20.155686+0000","last_peered":"2023-10-06T05:29:20.155686+0000","last_clean":"2023-10-06T05:29:20.155686+0000","last_became_active":"2023-10-06T05:29:20.151134+0000","last_became_peered":"2023-10-06T05:29:20.151134+0000","last_unstale":"2023-10-06T05:29:20.155686+0000","last_undegraded":"2023-10-06T05:29:20.155686+0000","last_fullsized":"2023-10-06T05:29:20.155686+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:20.401755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122374+0000","last_change":"2023-10-06T05:29:19.125212+0000","last_active":"2023-10-06T05:29:20.122374+0000","last_peered":"2023-10-06T05:29:20.122374+0000","last_clean":"2023-10-06T05:29:20.122374+0000","last_became_active":"2023-10-06T05:29:19.124953+0000","last_became_peered":"2023-10-06T05:29:19.124953+0000","last_unstale":"2023-10-06T05:29:20.122374+0000","last_undegraded":"2023-10-06T05:29:20.122374+0000","last_fullsized":"2023-10-06T05:29:20.122374+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:47.580175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148652+0000","last_change":"2023-10-06T05:29:20.137889+0000","last_active":"2023-10-06T05:29:20.148652+0000","last_peered":"2023-10-06T05:29:20.148652+0000","last_clean":"2023-10-06T05:29:20.148652+0000","last_became_active":"2023-10-06T05:29:20.137617+0000","last_became_peered":"2023-10-06T05:29:20.137617+0000","last_unstale":"2023-10-06T05:29:20.148652+0000","last_undegraded":"2023-10-06T05:29:20.148652+0000","last_fullsized":"2023-10-06T05:29:20.148652+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:02.935235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122498+0000","last_change":"2023-10-06T05:29:19.123577+0000","last_active":"2023-10-06T05:29:20.122498+0000","last_peered":"2023-10-06T05:29:20.122498+0000","last_clean":"2023-10-06T05:29:20.122498+0000","last_became_active":"2023-10-06T05:29:19.123338+0000","last_became_peered":"2023-10-06T05:29:19.123338+0000","last_unstale":"2023-10-06T05:29:20.122498+0000","last_undegraded":"2023-10-06T05:29:20.122498+0000","last_fullsized":"2023-10-06T05:29:20.122498+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:44.835632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.141480+0000","last_change":"2023-10-06T05:29:20.132262+0000","last_active":"2023-10-06T05:29:20.141480+0000","last_peered":"2023-10-06T05:29:20.141480+0000","last_clean":"2023-10-06T05:29:20.141480+0000","last_became_active":"2023-10-06T05:29:20.132023+0000","last_became_peered":"2023-10-06T05:29:20.132023+0000","last_unstale":"2023-10-06T05:29:20.141480+0000","last_undegraded":"2023-10-06T05:29:20.141480+0000","last_fullsized":"2023-10-06T05:29:20.141480+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:03.419340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.131597+0000","last_change":"2023-10-06T05:29:19.127795+0000","last_active":"2023-10-06T05:29:20.131597+0000","last_peered":"2023-10-06T05:29:20.131597+0000","last_clean":"2023-10-06T05:29:20.131597+0000","last_became_active":"2023-10-06T05:29:19.127216+0000","last_became_peered":"2023-10-06T05:29:19.127216+0000","last_unstale":"2023-10-06T05:29:20.131597+0000","last_undegraded":"2023-10-06T05:29:20.131597+0000","last_fullsized":"2023-10-06T05:29:20.131597+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:32.486205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142443+0000","last_change":"2023-10-06T05:29:20.129613+0000","last_active":"2023-10-06T05:29:20.142443+0000","last_peered":"2023-10-06T05:29:20.142443+0000","last_clean":"2023-10-06T05:29:20.142443+0000","last_became_active":"2023-10-06T05:29:20.129339+0000","last_became_peered":"2023-10-06T05:29:20.129339+0000","last_unstale":"2023-10-06T05:29:20.142443+0000","last_undegraded":"2023-10-06T05:29:20.142443+0000","last_fullsized":"2023-10-06T05:29:20.142443+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:03.419340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.122479+0000","last_change":"2023-10-06T05:29:19.122769+0000","last_active":"2023-10-06T05:29:20.122479+0000","last_peered":"2023-10-06T05:29:20.122479+0000","last_clean":"2023-10-06T05:29:20.122479+0000","last_became_active":"2023-10-06T05:29:19.122452+0000","last_became_peered":"2023-10-06T05:29:19.122452+0000","last_unstale":"2023-10-06T05:29:20.122479+0000","last_undegraded":"2023-10-06T05:29:20.122479+0000","last_fullsized":"2023-10-06T05:29:20.122479+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:19.519119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157598+0000","last_change":"2023-10-06T05:29:20.151460+0000","last_active":"2023-10-06T05:29:20.157598+0000","last_peered":"2023-10-06T05:29:20.157598+0000","last_clean":"2023-10-06T05:29:20.157598+0000","last_became_active":"2023-10-06T05:29:20.151216+0000","last_became_peered":"2023-10-06T05:29:20.151216+0000","last_unstale":"2023-10-06T05:29:20.157598+0000","last_undegraded":"2023-10-06T05:29:20.157598+0000","last_fullsized":"2023-10-06T05:29:20.157598+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:05.449456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.135363+0000","last_change":"2023-10-06T05:29:19.121182+0000","last_active":"2023-10-06T05:29:20.135363+0000","last_peered":"2023-10-06T05:29:20.135363+0000","last_clean":"2023-10-06T05:29:20.135363+0000","last_became_active":"2023-10-06T05:29:19.120908+0000","last_became_peered":"2023-10-06T05:29:19.120908+0000","last_unstale":"2023-10-06T05:29:20.135363+0000","last_undegraded":"2023-10-06T05:29:20.135363+0000","last_fullsized":"2023-10-06T05:29:20.135363+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:08.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.147739+0000","last_change":"2023-10-06T05:29:20.145143+0000","last_active":"2023-10-06T05:29:20.147739+0000","last_peered":"2023-10-06T05:29:20.147739+0000","last_clean":"2023-10-06T05:29:20.147739+0000","last_became_active":"2023-10-06T05:29:20.144731+0000","last_became_peered":"2023-10-06T05:29:20.144731+0000","last_unstale":"2023-10-06T05:29:20.147739+0000","last_undegraded":"2023-10-06T05:29:20.147739+0000","last_fullsized":"2023-10-06T05:29:20.147739+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:12.336540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.133977+0000","last_change":"2023-10-06T05:29:19.121069+0000","last_active":"2023-10-06T05:29:20.133977+0000","last_peered":"2023-10-06T05:29:20.133977+0000","last_clean":"2023-10-06T05:29:20.133977+0000","last_became_active":"2023-10-06T05:29:19.120748+0000","last_became_peered":"2023-10-06T05:29:19.120748+0000","last_unstale":"2023-10-06T05:29:20.133977+0000","last_undegraded":"2023-10-06T05:29:20.133977+0000","last_fullsized":"2023-10-06T05:29:20.133977+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:10.129669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.158450+0000","last_change":"2023-10-06T05:29:20.153804+0000","last_active":"2023-10-06T05:29:20.158450+0000","last_peered":"2023-10-06T05:29:20.158450+0000","last_clean":"2023-10-06T05:29:20.158450+0000","last_became_active":"2023-10-06T05:29:20.153560+0000","last_became_peered":"2023-10-06T05:29:20.153560+0000","last_unstale":"2023-10-06T05:29:20.158450+0000","last_undegraded":"2023-10-06T05:29:20.158450+0000","last_fullsized":"2023-10-06T05:29:20.158450+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:50.376311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.135358+0000","last_change":"2023-10-06T05:29:19.118394+0000","last_active":"2023-10-06T05:29:20.135358+0000","last_peered":"2023-10-06T05:29:20.135358+0000","last_clean":"2023-10-06T05:29:20.135358+0000","last_became_active":"2023-10-06T05:29:19.118139+0000","last_became_peered":"2023-10-06T05:29:19.118139+0000","last_unstale":"2023-10-06T05:29:20.135358+0000","last_undegraded":"2023-10-06T05:29:20.135358+0000","last_fullsized":"2023-10-06T05:29:20.135358+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:18.129121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134158+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:29:20.134158+0000","last_peered":"2023-10-06T05:29:20.134158+0000","last_clean":"2023-10-06T05:29:20.134158+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:29:20.134158+0000","last_undegraded":"2023-10-06T05:29:20.134158+0000","last_fullsized":"2023-10-06T05:29:20.134158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.157468+0000","last_change":"2023-10-06T05:29:20.152136+0000","last_active":"2023-10-06T05:29:20.157468+0000","last_peered":"2023-10-06T05:29:20.157468+0000","last_clean":"2023-10-06T05:29:20.157468+0000","last_became_active":"2023-10-06T05:29:20.151887+0000","last_became_peered":"2023-10-06T05:29:20.151887+0000","last_unstale":"2023-10-06T05:29:20.157468+0000","last_undegraded":"2023-10-06T05:29:20.157468+0000","last_fullsized":"2023-10-06T05:29:20.157468+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:23.692229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"209'4","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.159885+0000","last_change":"2023-10-06T05:29:19.119785+0000","last_active":"2023-10-06T05:29:20.159885+0000","last_peered":"2023-10-06T05:29:20.159885+0000","last_clean":"2023-10-06T05:29:20.159885+0000","last_became_active":"2023-10-06T05:29:19.119467+0000","last_became_peered":"2023-10-06T05:29:19.119467+0000","last_unstale":"2023-10-06T05:29:20.159885+0000","last_undegraded":"2023-10-06T05:29:20.159885+0000","last_fullsized":"2023-10-06T05:29:20.159885+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:40.050699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.128818+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:29:20.128818+0000","last_peered":"2023-10-06T05:29:20.128818+0000","last_clean":"2023-10-06T05:29:20.128818+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:29:20.128818+0000","last_undegraded":"2023-10-06T05:29:20.128818+0000","last_fullsized":"2023-10-06T05:29:20.128818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142405+0000","last_change":"2023-10-06T05:29:20.138747+0000","last_active":"2023-10-06T05:29:20.142405+0000","last_peered":"2023-10-06T05:29:20.142405+0000","last_clean":"2023-10-06T05:29:20.142405+0000","last_became_active":"2023-10-06T05:29:20.138426+0000","last_became_peered":"2023-10-06T05:29:20.138426+0000","last_unstale":"2023-10-06T05:29:20.142405+0000","last_undegraded":"2023-10-06T05:29:20.142405+0000","last_fullsized":"2023-10-06T05:29:20.142405+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:45.353232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.143833+0000","last_change":"2023-10-06T05:29:19.117956+0000","last_active":"2023-10-06T05:29:20.143833+0000","last_peered":"2023-10-06T05:29:20.143833+0000","last_clean":"2023-10-06T05:29:20.143833+0000","last_became_active":"2023-10-06T05:29:19.117592+0000","last_became_peered":"2023-10-06T05:29:19.117592+0000","last_unstale":"2023-10-06T05:29:20.143833+0000","last_undegraded":"2023-10-06T05:29:20.143833+0000","last_fullsized":"2023-10-06T05:29:20.143833+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:21.948779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":171,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.132868+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:29:20.132868+0000","last_peered":"2023-10-06T05:29:20.132868+0000","last_clean":"2023-10-06T05:29:20.132868+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:29:20.132868+0000","last_undegraded":"2023-10-06T05:29:20.132868+0000","last_fullsized":"2023-10-06T05:29:20.132868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.148192+0000","last_change":"2023-10-06T05:29:20.142655+0000","last_active":"2023-10-06T05:29:20.148192+0000","last_peered":"2023-10-06T05:29:20.148192+0000","last_clean":"2023-10-06T05:29:20.148192+0000","last_became_active":"2023-10-06T05:29:20.141953+0000","last_became_peered":"2023-10-06T05:29:20.141953+0000","last_unstale":"2023-10-06T05:29:20.148192+0000","last_undegraded":"2023-10-06T05:29:20.148192+0000","last_fullsized":"2023-10-06T05:29:20.148192+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:19.715122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155539+0000","last_change":"2023-10-06T05:29:19.119799+0000","last_active":"2023-10-06T05:29:20.155539+0000","last_peered":"2023-10-06T05:29:20.155539+0000","last_clean":"2023-10-06T05:29:20.155539+0000","last_became_active":"2023-10-06T05:29:19.118352+0000","last_became_peered":"2023-10-06T05:29:19.118352+0000","last_unstale":"2023-10-06T05:29:20.155539+0000","last_undegraded":"2023-10-06T05:29:20.155539+0000","last_fullsized":"2023-10-06T05:29:20.155539+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:24.687020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.128600+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:29:20.128600+0000","last_peered":"2023-10-06T05:29:20.128600+0000","last_clean":"2023-10-06T05:29:20.128600+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:29:20.128600+0000","last_undegraded":"2023-10-06T05:29:20.128600+0000","last_fullsized":"2023-10-06T05:29:20.128600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.129186+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:29:20.129186+0000","last_peered":"2023-10-06T05:29:20.129186+0000","last_clean":"2023-10-06T05:29:20.129186+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:29:20.129186+0000","last_undegraded":"2023-10-06T05:29:20.129186+0000","last_fullsized":"2023-10-06T05:29:20.129186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.152952+0000","last_change":"2023-10-06T05:29:20.149603+0000","last_active":"2023-10-06T05:29:20.152952+0000","last_peered":"2023-10-06T05:29:20.152952+0000","last_clean":"2023-10-06T05:29:20.152952+0000","last_became_active":"2023-10-06T05:29:20.149320+0000","last_became_peered":"2023-10-06T05:29:20.149320+0000","last_unstale":"2023-10-06T05:29:20.152952+0000","last_undegraded":"2023-10-06T05:29:20.152952+0000","last_fullsized":"2023-10-06T05:29:20.152952+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:59.102372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.158785+0000","last_change":"2023-10-06T05:29:19.128409+0000","last_active":"2023-10-06T05:29:20.158785+0000","last_peered":"2023-10-06T05:29:20.158785+0000","last_clean":"2023-10-06T05:29:20.158785+0000","last_became_active":"2023-10-06T05:29:19.127679+0000","last_became_peered":"2023-10-06T05:29:19.127679+0000","last_unstale":"2023-10-06T05:29:20.158785+0000","last_undegraded":"2023-10-06T05:29:20.158785+0000","last_fullsized":"2023-10-06T05:29:20.158785+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:19.782239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.129090+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:29:20.129090+0000","last_peered":"2023-10-06T05:29:20.129090+0000","last_clean":"2023-10-06T05:29:20.129090+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:29:20.129090+0000","last_undegraded":"2023-10-06T05:29:20.129090+0000","last_fullsized":"2023-10-06T05:29:20.129090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.134490+0000","last_change":"2023-10-06T05:29:20.132682+0000","last_active":"2023-10-06T05:29:20.134490+0000","last_peered":"2023-10-06T05:29:20.134490+0000","last_clean":"2023-10-06T05:29:20.134490+0000","last_became_active":"2023-10-06T05:29:20.131574+0000","last_became_peered":"2023-10-06T05:29:20.131574+0000","last_unstale":"2023-10-06T05:29:20.134490+0000","last_undegraded":"2023-10-06T05:29:20.134490+0000","last_fullsized":"2023-10-06T05:29:20.134490+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:39.430627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.121895+0000","last_change":"2023-10-06T05:29:19.116628+0000","last_active":"2023-10-06T05:29:20.121895+0000","last_peered":"2023-10-06T05:29:20.121895+0000","last_clean":"2023-10-06T05:29:20.121895+0000","last_became_active":"2023-10-06T05:29:19.112389+0000","last_became_peered":"2023-10-06T05:29:19.112389+0000","last_unstale":"2023-10-06T05:29:20.121895+0000","last_undegraded":"2023-10-06T05:29:20.121895+0000","last_fullsized":"2023-10-06T05:29:20.121895+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:40.088591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.133123+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:29:20.133123+0000","last_peered":"2023-10-06T05:29:20.133123+0000","last_clean":"2023-10-06T05:29:20.133123+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:29:20.133123+0000","last_undegraded":"2023-10-06T05:29:20.133123+0000","last_fullsized":"2023-10-06T05:29:20.133123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144450+0000","last_change":"2023-10-06T05:29:20.137960+0000","last_active":"2023-10-06T05:29:20.144450+0000","last_peered":"2023-10-06T05:29:20.144450+0000","last_clean":"2023-10-06T05:29:20.144450+0000","last_became_active":"2023-10-06T05:29:20.137675+0000","last_became_peered":"2023-10-06T05:29:20.137675+0000","last_unstale":"2023-10-06T05:29:20.144450+0000","last_undegraded":"2023-10-06T05:29:20.144450+0000","last_fullsized":"2023-10-06T05:29:20.144450+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:45.353232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.155668+0000","last_change":"2023-10-06T05:29:19.118622+0000","last_active":"2023-10-06T05:29:20.155668+0000","last_peered":"2023-10-06T05:29:20.155668+0000","last_clean":"2023-10-06T05:29:20.155668+0000","last_became_active":"2023-10-06T05:29:19.118351+0000","last_became_peered":"2023-10-06T05:29:19.118351+0000","last_unstale":"2023-10-06T05:29:20.155668+0000","last_undegraded":"2023-10-06T05:29:20.155668+0000","last_fullsized":"2023-10-06T05:29:20.155668+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:24.904793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.139155+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:29:20.139155+0000","last_peered":"2023-10-06T05:29:20.139155+0000","last_clean":"2023-10-06T05:29:20.139155+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:29:20.139155+0000","last_undegraded":"2023-10-06T05:29:20.139155+0000","last_fullsized":"2023-10-06T05:29:20.139155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.142871+0000","last_change":"2023-10-06T05:29:20.140679+0000","last_active":"2023-10-06T05:29:20.142871+0000","last_peered":"2023-10-06T05:29:20.142871+0000","last_clean":"2023-10-06T05:29:20.142871+0000","last_became_active":"2023-10-06T05:29:20.140365+0000","last_became_peered":"2023-10-06T05:29:20.140365+0000","last_unstale":"2023-10-06T05:29:20.142871+0000","last_undegraded":"2023-10-06T05:29:20.142871+0000","last_fullsized":"2023-10-06T05:29:20.142871+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:19.097863+0000","last_clean_scrub_stamp":"2023-10-06T05:29:19.097863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:47.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.144325+0000","last_change":"2023-10-06T05:29:19.118550+0000","last_active":"2023-10-06T05:29:20.144325+0000","last_peered":"2023-10-06T05:29:20.144325+0000","last_clean":"2023-10-06T05:29:20.144325+0000","last_became_active":"2023-10-06T05:29:19.118121+0000","last_became_peered":"2023-10-06T05:29:19.118121+0000","last_unstale":"2023-10-06T05:29:20.144325+0000","last_undegraded":"2023-10-06T05:29:20.144325+0000","last_fullsized":"2023-10-06T05:29:20.144325+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:18.091209+0000","last_clean_scrub_stamp":"2023-10-06T05:29:18.091209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:30.080647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:29:20.123421+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:29:20.123421+0000","last_peered":"2023-10-06T05:29:20.123421+0000","last_clean":"2023-10-06T05:29:20.123421+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:29:20.123421+0000","last_undegraded":"2023-10-06T05:29:20.123421+0000","last_fullsized":"2023-10-06T05:29:20.123421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304552,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067288,"statfs":{"total":96636764160,"available":96324902912,"internally_reserved":0,"allocated":3244032,"data_stored":1823346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.88100000000000001,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88200000000000001,"5min":1.1339999999999999,"15min":1.169},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68600000000000005,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":1.0249999999999999,"15min":1.268},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96399999999999997,"5min":1.163,"15min":1.163},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":1.22},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.109,"15min":1.109},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.753},"min":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":1.0960000000000001,"15min":1.202},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.627},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":1.109,"15min":1.258},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.39400000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.2849999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.98099999999999998,"5min":1.109,"15min":1.109},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.752,"15min":0.749},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.407},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.999,"5min":1.169,"15min":1.169},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.52700000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.0349999999999999,"5min":1.135,"15min":1.3300000000000001},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1239607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.39800000000000002,"15min":0.34200000000000003},"max":{"1min":1.044,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.434,"5min":0.42699999999999999,"15min":0.373},"max":{"1min":0.98099999999999998,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69299999999999995,"15min":0.69999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.44600000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.53400000000000003,"5min":0.44600000000000001,"15min":0.40799999999999997},"max":{"1min":1.097,"5min":1.135,"15min":1.135},"last":0.91000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.48099999999999998,"15min":0.47999999999999998},"max":{"1min":0.90800000000000003,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.44900000000000001,"15min":0.442},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.871}]},{"osd":3,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.57499999999999996,"5min":0.45300000000000001,"15min":0.40899999999999997},"max":{"1min":0.93400000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.68300000000000005,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.1020000000000001},"last":0.96399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.95299999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93000000000000005,"5min":0.97799999999999998,"15min":0.998},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52500000000000002,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.0760000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.499,"5min":0.42699999999999999,"15min":0.40000000000000002},"max":{"1min":1.002,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.432},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.64400000000000002,"5min":0.497,"15min":0.436},"max":{"1min":1.0620000000000001,"5min":1.0860000000000001,"15min":1.1359999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303964,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067876,"statfs":{"total":96636764160,"available":96325505024,"internally_reserved":0,"allocated":2641920,"data_stored":1232854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.621,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27800000000000002},"max":{"1min":0.56200000000000006,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":1.165,"15min":2.6520000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.76600000000000001,"5min":1.077,"15min":2.7010000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.72399999999999998,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35199999999999998},"max":{"1min":0.69299999999999995,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.026,"5min":1.111,"15min":1.5389999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.4690000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.349},"last":0.752},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.50600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.669},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.48199999999999998},"max":{"1min":1.2290000000000001,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.573},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.528},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.48699999999999999},"max":{"1min":1.073,"5min":1.3049999999999999,"15min":2.6240000000000001},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304568,"kb_used_data":3184,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067272,"statfs":{"total":96636764160,"available":96324886528,"internally_reserved":0,"allocated":3260416,"data_stored":1831399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46200000000000002,"15min":0.45400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.245},"max":{"1min":0.59199999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.498},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.26900000000000002},"max":{"1min":0.70699999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.307},"max":{"1min":0.624,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69299999999999995,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79500000000000004,"15min":0.78000000000000003},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":1,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.77300000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.244,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.54500000000000004},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":0.95299999999999996,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.56100000000000005,"5min":0.52800000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.81000000000000005},"min":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.45500000000000002},"max":{"1min":1.099,"5min":1.153,"15min":1.2629999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.0980000000000001,"5min":1.2010000000000001,"15min":1.25},"last":1.018},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.50600000000000001},"max":{"1min":1.167,"5min":1.274,"15min":1.2789999999999999},"last":0.871}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1240119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":1.03,"15min":1.056},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.33700000000000002,"5min":0.309,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.38200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.42999999999999999,"5min":0.34200000000000003,"15min":0.30599999999999999},"max":{"1min":0.622,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.63,"5min":0.94399999999999995,"15min":1.073},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48999999999999999,"15min":0.47699999999999998},"min":{"1min":0.36099999999999999,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.61599999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.31},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64500000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.53700000000000003},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.77100000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.47499999999999998},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.754,"15min":0.76500000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.47299999999999998},"max":{"1min":0.91700000000000004,"5min":1.153,"15min":1.1639999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.54800000000000004,"15min":0.53000000000000003},"max":{"1min":0.94699999999999995,"5min":1.1859999999999999,"15min":1.319},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.59599999999999997,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.94599999999999995,"5min":1.175,"15min":1.5289999999999999},"last":0.746},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80200000000000005,"15min":0.79200000000000004},"min":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96399999999999997,"5min":1.137,"15min":1.141},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.52100000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80400000000000005,"15min":0.81499999999999995},"min":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88400000000000001,"5min":1.2,"15min":1.3520000000000001},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304000,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067840,"statfs":{"total":96636764160,"available":96325468160,"internally_reserved":0,"allocated":2678784,"data_stored":1241641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45700000000000002,"15min":0.46800000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":2.0950000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.34300000000000003,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":1.772},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60799999999999998,"5min":1.369,"15min":1.913},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.307},"max":{"1min":0.71099999999999997,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.90200000000000002,"15min":1.954},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38300000000000001},"max":{"1min":0.70099999999999996,"5min":0.82599999999999996,"15min":2.0539999999999998},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.59999999999999998,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.282},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76200000000000001,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":0.92700000000000005,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.111,"15min":1.1930000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.51800000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97299999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":1.0089999999999999,"5min":1.3360000000000001,"15min":1.343},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.64100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.68700000000000006,"5min":0.59999999999999998,"15min":0.56799999999999995},"max":{"1min":1.0649999999999999,"5min":1.194,"15min":1.3640000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.61299999999999999,"5min":0.54400000000000004,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.1579999999999999,"15min":1.214},"last":0.88}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1239565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":1.073},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72899999999999998,"15min":0.75},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90800000000000003,"5min":1.05,"15min":1.169},"last":0.86199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":1.0569999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.46200000000000002,"15min":0.432},"max":{"1min":0.88800000000000001,"5min":0.96799999999999997,"15min":1.0309999999999999},"last":0.91600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.70399999999999996},"min":{"1min":0.46999999999999997,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.82899999999999996,"5min":0.96199999999999997,"15min":1.117},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71599999999999997,"15min":0.71899999999999997},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41499999999999998},"max":{"1min":0.871,"5min":0.97099999999999997,"15min":1.002},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.72899999999999998},"min":{"1min":0.55400000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":0.94999999999999996,"15min":1.0780000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.196},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":1.044,"15min":1.8740000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.70299999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":1.014,"15min":1.375},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.70599999999999996},"min":{"1min":0.51800000000000002,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.90400000000000003,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.505,"5min":0.505,"15min":0.45300000000000001},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48199999999999998},"max":{"1min":0.93700000000000006,"5min":1.073,"15min":1.466},"last":1.081},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97099999999999997,"5min":1.0569999999999999,"15min":1.218},"last":1.0660000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575035,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303964,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067876,"statfs":{"total":96636764160,"available":96325505024,"internally_reserved":0,"allocated":2641920,"data_stored":1232854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.441,"5min":0.435,"15min":0.374},"max":{"1min":1.127,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93899999999999995,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.0529999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":0.999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0289999999999999,"15min":1.204},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.091,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.46899999999999997},"max":{"1min":0.88700000000000001,"5min":0.94599999999999995,"15min":1.087},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.433},"max":{"1min":0.90000000000000002,"5min":1.145,"15min":1.278},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.49399999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.1379999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.876,"5min":1.004,"15min":1.294},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.442,"5min":0.41899999999999998,"15min":0.38900000000000001},"max":{"1min":0.80200000000000005,"5min":0.89900000000000002,"15min":1.141},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71899999999999997,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.441,"15min":0.40500000000000003},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.254},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0189999999999999,"15min":1.337},"last":0.752},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98699999999999999,"5min":1.0600000000000001,"15min":1.248},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:29:24.992 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:29:24.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:29:25.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.217+0000 7f747319d700 1 Processor -- start 2023-10-06T05:29:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.217+0000 7f747319d700 1 -- start start 2023-10-06T05:29:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c15a7d0 0x7f746c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747319d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f746c0cced0 0x7f746c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c15ddb0 con 0x7f746c0af540 2023-10-06T05:29:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747319d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c0cd890 con 0x7f746c0cced0 2023-10-06T05:29:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15d1a0 con 0x7f746c15a7d0 2023-10-06T05:29:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747219b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f746c0af540 0x7f746c0cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58328/0 (socket says 172.21.15.33:58328) 2023-10-06T05:29:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747219b700 1 -- 172.21.15.33:0/3690030048 learned_addr learned my addr 172.21.15.33:0/3690030048 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.218+0000 7f747299c700 1 --2- 172.21.15.33:0/3690030048 >> v2:172.21.15.33:3300/0 conn(0x7f746c0cced0 0x7f746c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747199a700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c15a7d0 0x7f746c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f7471199700 1 -- 172.21.15.33:0/3690030048 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149285684 0 0) 0x7f746c15ddb0 con 0x7f746c0af540 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f7471199700 1 -- 172.21.15.33:0/3690030048 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7458003570 con 0x7f746c0af540 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747299c700 1 -- 172.21.15.33:0/3690030048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c15a7d0 msgr2=0x7f746c0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747299c700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c15a7d0 0x7f746c0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747299c700 1 -- 172.21.15.33:0/3690030048 >> v1:172.21.15.33:6789/0 conn(0x7f746c0af540 legacy=0x7f746c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747299c700 1 -- 172.21.15.33:0/3690030048 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c15d480 con 0x7f746c0cced0 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747199a700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c15a7d0 0x7f746c0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f747299c700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c0cd350 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f7464009e60 tx=0x7f7464009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f259ff9c1be9476 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.219+0000 7f7471199700 1 -- 172.21.15.33:0/3690030048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400e070 con 0x7f746c0cced0 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f7471199700 1 -- 172.21.15.33:0/3690030048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746400bb60 con 0x7f746c0cced0 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f7471199700 1 -- 172.21.15.33:0/3690030048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464012430 con 0x7f746c0cced0 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 -- 172.21.15.33:0/3690030048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 msgr2=0x7f746c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c0cd350 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f7464009e60 tx=0x7f7464009780 comp rx=0 tx=0).stop 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 -- 172.21.15.33:0/3690030048 shutdown_connections 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c0cd350 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 --2- 172.21.15.33:0/3690030048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c15a7d0 0x7f746c0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 -- 172.21.15.33:0/3690030048 >> 172.21.15.33:0/3690030048 conn(0x7f746c06d9b0 msgr2=0x7f746c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 -- 172.21.15.33:0/3690030048 shutdown_connections 2023-10-06T05:29:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.220+0000 7f747319d700 1 -- 172.21.15.33:0/3690030048 wait complete. 2023-10-06T05:29:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.222+0000 7f747319d700 1 Processor -- start 2023-10-06T05:29:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.222+0000 7f747319d700 1 -- start start 2023-10-06T05:29:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c0af540 0x7f746c15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c15a7d0 0x7f746c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c160270 con 0x7f746c0cced0 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15f8a0 con 0x7f746c15a7d0 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c15ec90 con 0x7f746c0af540 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48932/0 (socket says 172.21.15.33:48932) 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 -- 172.21.15.33:0/2183422605 learned_addr learned my addr 172.21.15.33:0/2183422605 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747219b700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c0af540 0x7f746c15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747299c700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c15a7d0 0x7f746c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 -- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c0af540 msgr2=0x7f746c15fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c0af540 0x7f746c15fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 -- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c15a7d0 msgr2=0x7f746c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c15a7d0 0x7f746c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747199a700 1 -- 172.21.15.33:0/2183422605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009a20 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.223+0000 7f747299c700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c15a7d0 0x7f746c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.224+0000 7f747199a700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c142030 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f745c00e8d0 tx=0x7f745c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.224+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c013410 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.224+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c013570 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.224+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c013910 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.224+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c15ef70 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.224+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c15f530 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.225+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c04c380 con 0x7f746c0cced0 2023-10-06T05:29:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.226+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f745c013d00 con 0x7f746c0cced0 2023-10-06T05:29:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.227+0000 7f74627fc700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7450066f30 0x7f74500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.227+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f745c084490 con 0x7f746c0cced0 2023-10-06T05:29:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.227+0000 7f747219b700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7450066f30 0x7f74500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.228+0000 7f747219b700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7450066f30 0x7f74500693e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7468007fd0 tx=0x7f7468009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:25.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.231+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f745c0515a0 con 0x7f746c0cced0 2023-10-06T05:29:25.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.517+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f746c15d760 con 0x7f746c0cced0 2023-10-06T05:29:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.521+0000 7f74627fc700 1 -- 172.21.15.33:0/2183422605 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7f745c054bc0 con 0x7f746c0cced0 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7450066f30 msgr2=0x7f74500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7450066f30 0x7f74500693e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7468007fd0 tx=0x7f7468009040 comp rx=0 tx=0).stop 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 msgr2=0x7f746c142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c142030 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f745c00e8d0 tx=0x7f745c008040 comp rx=0 tx=0).stop 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 shutdown_connections 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7450066f30 0x7f74500693e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c15a7d0 0x7f746c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f746c0af540 0x7f746c15fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 --2- 172.21.15.33:0/2183422605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746c0cced0 0x7f746c142030 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 >> 172.21.15.33:0/2183422605 conn(0x7f746c06d9b0 msgr2=0x7f746c0c6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 shutdown_connections 2023-10-06T05:29:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.524+0000 7f747319d700 1 -- 172.21.15.33:0/2183422605 wait complete. 2023-10-06T05:29:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:29:25.544 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:25.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.728+0000 7f36fd79d700 1 Processor -- start 2023-10-06T05:29:25.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.729+0000 7f36fd79d700 1 -- start start 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36fd79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c6340 0x7f36f80c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36fd79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36f80c80d0 0x7f36f806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36fd79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f806c820 con 0x7f36f8159520 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36fd79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f806a160 con 0x7f36f80c80d0 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36fd79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80c8800 con 0x7f36f80c6340 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36f7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f36f8159520 0x7f36f80c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58338/0 (socket says 172.21.15.33:58338) 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36f7fff700 1 -- 172.21.15.33:0/3478779825 learned_addr learned my addr 172.21.15.33:0/3478779825 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:25.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.730+0000 7f36fcf9c700 1 --2- 172.21.15.33:0/3478779825 >> v2:172.21.15.33:3300/0 conn(0x7f36f80c80d0 0x7f36f806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3478779825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c6340 0x7f36f80c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3478779825 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765049619 0 0) 0x7f36f806c820 con 0x7f36f8159520 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3478779825 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e0003570 con 0x7f36f8159520 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36fcf9c700 1 -- 172.21.15.33:0/3478779825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c6340 msgr2=0x7f36f80c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36fcf9c700 1 --2- 172.21.15.33:0/3478779825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c6340 0x7f36f80c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36fcf9c700 1 -- 172.21.15.33:0/3478779825 >> v1:172.21.15.33:6789/0 conn(0x7f36f8159520 legacy=0x7f36f80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36fcf9c700 1 -- 172.21.15.33:0/3478779825 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f80c8ae0 con 0x7f36f80c80d0 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36fcf9c700 1 --2- 172.21.15.33:0/3478779825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c80d0 0x7f36f806a660 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e60 tx=0x7f36f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9198279b72426656 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3478779825 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000e070 con 0x7f36f80c80d0 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3478779825 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f000bab0 con 0x7f36f80c80d0 2023-10-06T05:29:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.731+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3478779825 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000be70 con 0x7f36f80c80d0 2023-10-06T05:29:25.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 -- 172.21.15.33:0/3478779825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c80d0 msgr2=0x7f36f806a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 --2- 172.21.15.33:0/3478779825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c80d0 0x7f36f806a660 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e60 tx=0x7f36f0009780 comp rx=0 tx=0).stop 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 -- 172.21.15.33:0/3478779825 shutdown_connections 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 --2- 172.21.15.33:0/3478779825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c80d0 0x7f36f806a660 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 --2- 172.21.15.33:0/3478779825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c6340 0x7f36f80c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 -- 172.21.15.33:0/3478779825 >> 172.21.15.33:0/3478779825 conn(0x7f36f80cc930 msgr2=0x7f36f80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 -- 172.21.15.33:0/3478779825 shutdown_connections 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.732+0000 7f36fd79d700 1 -- 172.21.15.33:0/3478779825 wait complete. 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.734+0000 7f36fd79d700 1 Processor -- start 2023-10-06T05:29:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.734+0000 7f36fd79d700 1 -- start start 2023-10-06T05:29:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fd79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 0x7f36f8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fd79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c80d0 0x7f36f813b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fd79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f8159520 0x7f36f80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fd79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f813aec0 con 0x7f36f80c6340 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fd79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f813c7d0 con 0x7f36f8159520 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fd79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f813bbc0 con 0x7f36f80c80d0 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 0x7f36f8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 0x7f36f8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48948/0 (socket says 172.21.15.33:48948) 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 -- 172.21.15.33:0/2535376050 learned_addr learned my addr 172.21.15.33:0/2535376050 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f77fe700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c80d0 0x7f36f813b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36fcf9c700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f8159520 0x7f36f80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 -- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c80d0 msgr2=0x7f36f813b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c80d0 0x7f36f813b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 -- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f8159520 msgr2=0x7f36f80b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f8159520 0x7f36f80b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.735+0000 7f36f7fff700 1 -- 172.21.15.33:0/2535376050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0009a20 con 0x7f36f80c6340 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.736+0000 7f36f7fff700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 0x7f36f8155d90 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c590 tx=0x7f36e8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.736+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8005b30 con 0x7f36f80c6340 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.736+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8005cb0 con 0x7f36f80c6340 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.736+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8007320 con 0x7f36f80c6340 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.736+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f813bea0 con 0x7f36f80c6340 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.736+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f813c460 con 0x7f36f80c6340 2023-10-06T05:29:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.737+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f804c380 con 0x7f36f80c6340 2023-10-06T05:29:25.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.738+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36e801b3b0 con 0x7f36f80c6340 2023-10-06T05:29:25.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.739+0000 7f36dffff700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36e4066f30 0x7f36e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:25.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.739+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f36e8083bb0 con 0x7f36f80c6340 2023-10-06T05:29:25.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.739+0000 7f36f77fe700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36e4066f30 0x7f36e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:25.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.740+0000 7f36f77fe700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36e4066f30 0x7f36e40693e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f36f8158d20 tx=0x7f36ec0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:25.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.743+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36e8050c20 con 0x7f36f80c6340 2023-10-06T05:29:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.992+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36f80c8dc0 con 0x7f36f80c6340 2023-10-06T05:29:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.992+0000 7f36dffff700 1 -- 172.21.15.33:0/2535376050 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f36e8017020 con 0x7f36f80c6340 2023-10-06T05:29:25.995 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:25.995 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":162,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1335406981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1335406981},{"type":"v1","addr":"172.21.15.167:6836","nonce":1335406981}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157},{"gid":9463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1379577645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1379577645},{"type":"v1","addr":"172.21.15.33:6835","nonce":1379577645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157},{"gid":9469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1485763855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1485763855},{"type":"v1","addr":"172.21.15.167:6837","nonce":1485763855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:23.149786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10419":{"gid":10419,"name":"a","rank":0,"incarnation":158,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/963232630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":963232630},{"type":"v1","addr":"172.21.15.33:6834","nonce":963232630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36e4066f30 msgr2=0x7f36e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36e4066f30 0x7f36e40693e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f36f8158d20 tx=0x7f36ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 msgr2=0x7f36f8155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 0x7f36f8155d90 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c590 tx=0x7f36e8007e00 comp rx=0 tx=0).stop 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 shutdown_connections 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f36e4066f30 0x7f36e40693e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f8159520 0x7f36f80b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80c6340 0x7f36f8155d90 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 --2- 172.21.15.33:0/2535376050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f80c80d0 0x7f36f813b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 >> 172.21.15.33:0/2535376050 conn(0x7f36f80cc930 msgr2=0x7f36f8151510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 shutdown_connections 2023-10-06T05:29:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:25.995+0000 7f36fd79d700 1 -- 172.21.15.33:0/2535376050 wait complete. 2023-10-06T05:29:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 162 2023-10-06T05:29:26.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:29:19.601836+0000', 'modified': '2023-10-06T05:29:23.149786+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10419': {'gid': 10419, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6834/963232630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 963232630}, {'type': 'v1', 'addr': '172.21.15.33:6834', 'nonce': 963232630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:29:26.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:29:26.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:29:26.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.606+0000 7fe2063ff700 1 Processor -- start 2023-10-06T05:29:26.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.606+0000 7fe2063ff700 1 -- start start 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe2063ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe2001597c0 0x7fe2000a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe2063ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2000c9d20 0x7fe200159b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe2063ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe2000c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe2063ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20006cf30 con 0x7fe2000c9d20 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe2063ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20006b630 con 0x7fe2001597c0 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe2063ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20006c600 con 0x7fe20006acd0 2023-10-06T05:29:26.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.607+0000 7fe1ff7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2000c9d20 0x7fe200159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2000c9d20 0x7fe200159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48962/0 (socket says 172.21.15.33:48962) 2023-10-06T05:29:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/36751720 learned_addr learned my addr 172.21.15.33:0/36751720 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/36751720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 msgr2=0x7fe2000c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe20499c700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe2000c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1fffff700 1 --2- 172.21.15.33:0/36751720 >> v2:172.21.15.167:3300/0 conn(0x7fe2001597c0 0x7fe2000a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe2000c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/36751720 >> v2:172.21.15.167:3300/0 conn(0x7fe2001597c0 msgr2=0x7fe2000a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/36751720 >> v2:172.21.15.167:3300/0 conn(0x7fe2001597c0 0x7fe2000a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/36751720 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20006c8e0 con 0x7fe2000c9d20 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe20499c700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe2000c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1fffff700 1 --2- 172.21.15.33:0/36751720 >> v2:172.21.15.167:3300/0 conn(0x7fe2001597c0 0x7fe2000a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.608+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe200159b90 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fe20006e570 tx=0x7fe1f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f678de2118b536f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe1feffd700 1 -- 172.21.15.33:0/36751720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400b860 con 0x7fe2000c9d20 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe1feffd700 1 -- 172.21.15.33:0/36751720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f400b9c0 con 0x7fe2000c9d20 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe1feffd700 1 -- 172.21.15.33:0/36751720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400bd60 con 0x7fe2000c9d20 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 -- 172.21.15.33:0/36751720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 msgr2=0x7fe200159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe200159b90 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fe20006e570 tx=0x7fe1f4009250 comp rx=0 tx=0).stop 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 -- 172.21.15.33:0/36751720 shutdown_connections 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe2000c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 --2- 172.21.15.33:0/36751720 >> v2:172.21.15.167:3300/0 conn(0x7fe2001597c0 0x7fe2000a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 --2- 172.21.15.33:0/36751720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe200159b90 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 -- 172.21.15.33:0/36751720 >> 172.21.15.33:0/36751720 conn(0x7fe2000cd600 msgr2=0x7fe2000c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 -- 172.21.15.33:0/36751720 shutdown_connections 2023-10-06T05:29:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.609+0000 7fe2063ff700 1 -- 172.21.15.33:0/36751720 wait complete. 2023-10-06T05:29:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.611+0000 7fe2063ff700 1 Processor -- start 2023-10-06T05:29:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.611+0000 7fe2063ff700 1 -- start start 2023-10-06T05:29:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe2063ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe200159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe2063ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe20015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe2063ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2001597c0 0x7fe2000c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe2063ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20015b230 con 0x7fe2000c9d20 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe2063ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c4710 con 0x7fe2001597c0 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe2063ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c43c0 con 0x7fe20006acd0 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe1ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe20015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe1ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe20015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48964/0 (socket says 172.21.15.33:48964) 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/3122001080 learned_addr learned my addr 172.21.15.33:0/3122001080 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe1fffff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe200159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.612+0000 7fe20499c700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2001597c0 0x7fe2000c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 msgr2=0x7fe200159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe200159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2001597c0 msgr2=0x7fe2000c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2001597c0 0x7fe2000c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/3122001080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40094f0 con 0x7fe2000c9d20 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe20015a1f0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fe20006e5d0 tx=0x7fe1f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1e7fff700 1 -- 172.21.15.33:0/3122001080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4011dc0 con 0x7fe2000c9d20 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1e7fff700 1 -- 172.21.15.33:0/3122001080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4024440 con 0x7fe2000c9d20 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe1e7fff700 1 -- 172.21.15.33:0/3122001080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4024800 con 0x7fe2000c9d20 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20015ac70 con 0x7fe2000c9d20 2023-10-06T05:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.613+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe200159550 con 0x7fe2000c9d20 2023-10-06T05:29:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.615+0000 7fe1e7fff700 1 -- 172.21.15.33:0/3122001080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1f401a420 con 0x7fe2000c9d20 2023-10-06T05:29:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.616+0000 7fe1e7fff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ec066f30 0x7fe1ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.616+0000 7fe1e7fff700 1 -- 172.21.15.33:0/3122001080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1f4086550 con 0x7fe2000c9d20 2023-10-06T05:29:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.617+0000 7fe1fffff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ec066f30 0x7fe1ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.617+0000 7fe1fffff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ec066f30 0x7fe1ec0693e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0006fd0 tx=0x7fe1f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.618+0000 7fe214c1eb80 1 Processor -- start 2023-10-06T05:29:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.618+0000 7fe214c1eb80 1 -- start start 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e2fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099de8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099d68c80 con 0x557099deb9a0 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099d3fa70 con 0x557099de9bd0 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557099d3d270 con 0x557099e2fa90 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e2fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e2fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:58832/0 (socket says 172.21.15.33:58832) 2023-10-06T05:29:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1fffff700 1 -- 172.21.15.33:0/1752528294 learned_addr learned my addr 172.21.15.33:0/1752528294 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe20499c700 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099de8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 msgr2=0x557099e2fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e2fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 msgr2=0x557099de8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.619+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099de8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.620+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/1752528294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557099bd3270 con 0x557099deb9a0 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.620+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de8580 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40848d0 tx=0x7fe1f4003040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34756784bde9b48e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.620+0000 7fe1e57fa700 1 -- 172.21.15.33:0/1752528294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4057980 con 0x557099deb9a0 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.620+0000 7fe1e57fa700 1 -- 172.21.15.33:0/1752528294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4057280 con 0x557099deb9a0 2023-10-06T05:29:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.620+0000 7fe1e57fa700 1 -- 172.21.15.33:0/1752528294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f405a4d0 con 0x557099deb9a0 2023-10-06T05:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1752528294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 msgr2=0x557099de8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de8580 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40848d0 tx=0x7fe1f4003040 comp rx=0 tx=0).stop 2023-10-06T05:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1752528294 shutdown_connections 2023-10-06T05:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099de8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e2fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1752528294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de8580 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1752528294 >> 172.21.15.33:0/1752528294 conn(0x557099dea070 msgr2=0x557099dea440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1752528294 shutdown_connections 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1752528294 wait complete. 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.621+0000 7fe214c1eb80 1 -- start start 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.622+0000 7fe214c1eb80 1 Processor -- start 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099e2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099deb9a0 0x557099e269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099e2fa90 0x557099e26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099d39060 con 0x557099de9bd0 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099ce4890 con 0x557099e2fa90 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557099cffea0 con 0x557099deb9a0 2023-10-06T05:29:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe214c1eb80 10 client.0 _ll_register_callbacks cb 0x557099bd5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099e2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099e2c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48982/0 (socket says 172.21.15.33:48982) 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 -- 172.21.15.33:0/1476886135 learned_addr learned my addr 172.21.15.33:0/1476886135 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099deb9a0 0x557099e269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe20499c700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099e2fa90 0x557099e26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 -- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099deb9a0 msgr2=0x557099e269d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099deb9a0 0x557099e269d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 -- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099e2fa90 msgr2=0x557099e26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099e2fa90 0x557099e26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.623+0000 7fe1fffff700 1 -- 172.21.15.33:0/1476886135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe200159550 con 0x557099de9bd0 2023-10-06T05:29:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe1fffff700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099e2c030 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0015b80 tx=0x7fe1f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ed7d1345aaf4a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe1de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe1de7fc700 1 -- 172.21.15.33:0/1476886135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0021690 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe1de7fc700 1 -- 172.21.15.33:0/1476886135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1f001a040 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557099ddd690 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe1de7fc700 1 -- 172.21.15.33:0/1476886135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=162) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f0021a00 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe214c1eb80 10 client.10575 fetch_fsmap learned FSMap version 162 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=162,monmap=2+}) v3 -- 0x557099b75fe0 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.624+0000 7fe1de7fc700 1 -- 172.21.15.33:0/1476886135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557099ddd690 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe1de7fc700 1 -- 172.21.15.33:0/1476886135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 162) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe1f0021b60 con 0x557099de9bd0 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe214c1eb80 10 client.10575 fetch_fsmap finished waiting for FSMap version 162 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe214c1eb80 10 client.10575 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:29:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe214c1eb80 20 client.10575 populate_metadata read hostname 'smithi033' 2023-10-06T05:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e29090 0x557099e29460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe214c1eb80 4 client.10575 mds_command: new command op to 10419 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 --> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557099dba5c0 con 0x557099e29090 2023-10-06T05:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e29090 0x557099e29460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe1de7fc700 10 client.10575 ms_handle_connect on v2:172.21.15.33:6832/963232630 2023-10-06T05:29:26.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.625+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e29090 0x557099e29460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56817a7d431d001c server_cookie=6b13fd2c2e5668cc in_seq=0 out_seq=0 2023-10-06T05:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe1de7fc700 1 -- 172.21.15.33:0/1476886135 <== mds.0 v2:172.21.15.33:6832/963232630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557099dba5c0 con 0x557099e29090 2023-10-06T05:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe1de7fc700 10 client.10575 handle_command_reply: tid=0 2023-10-06T05:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 client.10575 shutdown 2023-10-06T05:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 msgr2=0x557099e2c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099e2c030 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0015b80 tx=0x7fe1f0017000 comp rx=0 tx=0).stop 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 shutdown_connections 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e29090 0x557099e29460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099e2fa90 0x557099e26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.627+0000 7fe1de7fc700 0 client.10575 ms_handle_reset on v2:172.21.15.33:6832/963232630 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.626+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099e2c030 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.627+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/1476886135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099deb9a0 0x557099e269d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.627+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 >> 172.21.15.33:0/1476886135 conn(0x557099dea070 msgr2=0x557099e2d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.627+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 shutdown_connections 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.627+0000 7fe214c1eb80 1 -- 172.21.15.33:0/1476886135 wait complete. 2023-10-06T05:29:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.627+0000 7fe214c1eb80 20 client.10575 trim_cache size 0 max 16384 2023-10-06T05:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.644+0000 7fe214c1eb80 1 Processor -- start 2023-10-06T05:29:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.644+0000 7fe214c1eb80 1 -- start start 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099dea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099deb9a0 0x557099de4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e26ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099bbbe30 con 0x557099de9bd0 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099e303e0 con 0x557099deb9a0 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557099de30c0 con 0x557099e2fa90 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099dea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.645+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099dea440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48988/0 (socket says 172.21.15.33:48988) 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 -- 172.21.15.33:0/175244201 learned_addr learned my addr 172.21.15.33:0/175244201 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe20499c700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e26ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099deb9a0 0x557099de4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 -- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 msgr2=0x557099e26ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e26ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 -- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099deb9a0 msgr2=0x557099de4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099deb9a0 0x557099de4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 -- 172.21.15.33:0/175244201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557099d07780 con 0x557099de9bd0 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe20499c700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e26ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099deb9a0 0x557099de4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:26.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1fffff700 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099dea440 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000a170 tx=0x7fe1f0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f605b35e0876172 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1e5ffb700 1 -- 172.21.15.33:0/175244201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0021690 con 0x557099de9bd0 2023-10-06T05:29:26.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1e5ffb700 1 -- 172.21.15.33:0/175244201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0039070 con 0x557099de9bd0 2023-10-06T05:29:26.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.646+0000 7fe1e5ffb700 1 -- 172.21.15.33:0/175244201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0021c10 con 0x557099de9bd0 2023-10-06T05:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 -- 172.21.15.33:0/175244201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 msgr2=0x557099dea440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099dea440 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000a170 tx=0x7fe1f0024000 comp rx=0 tx=0).stop 2023-10-06T05:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 -- 172.21.15.33:0/175244201 shutdown_connections 2023-10-06T05:29:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e26ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099de9bd0 0x557099dea440 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/175244201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099deb9a0 0x557099de4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 -- 172.21.15.33:0/175244201 >> 172.21.15.33:0/175244201 conn(0x557099dea070 msgr2=0x557099e29460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 -- 172.21.15.33:0/175244201 shutdown_connections 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.647+0000 7fe214c1eb80 1 -- 172.21.15.33:0/175244201 wait complete. 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.648+0000 7fe214c1eb80 1 -- start start 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.648+0000 7fe214c1eb80 1 Processor -- start 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099e41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e94140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099e94e40 con 0x557099deb9a0 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557099e2a1c0 con 0x557099de9bd0 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557099e41370 con 0x557099e2fa90 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe214c1eb80 10 client.0 _ll_register_callbacks cb 0x557099bd5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49000/0 (socket says 172.21.15.33:49000) 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/962514031 learned_addr learned my addr 172.21.15.33:0/962514031 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1fffff700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099e41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe20499c700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e94140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 msgr2=0x557099e94140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e94140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 msgr2=0x557099e41800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099e41800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.649+0000 7fe1ff7fe700 1 -- 172.21.15.33:0/962514031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557099b75fe0 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1ff7fe700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de4810 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40848a0 tx=0x7fe1f4080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e50f08b2e84fd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1de7fc700 1 -- 172.21.15.33:0/962514031 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4057980 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1de7fc700 1 -- 172.21.15.33:0/962514031 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1f405e220 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1f4057280 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1de7fc700 1 -- 172.21.15.33:0/962514031 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=162) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f405a4d0 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 10 client.10581 fetch_fsmap learned FSMap version 162 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=162,monmap=2+}) v3 -- 0x557099cee470 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1de7fc700 1 -- 172.21.15.33:0/962514031 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4057280 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe1de7fc700 1 -- 172.21.15.33:0/962514031 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 162) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe1f402b340 con 0x557099deb9a0 2023-10-06T05:29:26.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 10 client.10581 fetch_fsmap finished waiting for FSMap version 162 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 10 client.10581 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 20 client.10581 populate_metadata read hostname 'smithi033' 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e105d0 0x557099e94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 4 client.10581 mds_command: new command op to 10419 tid=0[{"prefix": "status"}] 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.650+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 --> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557099cdbe00 con 0x557099e105d0 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.651+0000 7fe1fffff700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e105d0 0x557099e94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.651+0000 7fe1fffff700 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e105d0 0x557099e94680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71196daa34796d65 server_cookie=ef32ff733d7977bc in_seq=0 out_seq=0 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.651+0000 7fe1de7fc700 10 client.10581 ms_handle_connect on v2:172.21.15.33:6832/963232630 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.651+0000 7fe1de7fc700 1 -- 172.21.15.33:0/962514031 <== mds.0 v2:172.21.15.33:6832/963232630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x557099cdbe00 con 0x557099e105d0 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.651+0000 7fe1de7fc700 10 client.10581 handle_command_reply: tid=0 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.651+0000 7fe214c1eb80 1 client.10581 shutdown 2023-10-06T05:29:26.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 msgr2=0x557099de4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de4810 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40848a0 tx=0x7fe1f4080040 comp rx=0 tx=0).stop 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 shutdown_connections 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:6832/963232630,v1:172.21.15.33:6834/963232630] conn(0x557099e105d0 0x557099e94680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557099e2fa90 0x557099e94140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557099de9bd0 0x557099e41800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 --2- 172.21.15.33:0/962514031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557099deb9a0 0x557099de4810 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe1de7fc700 0 client.10581 ms_handle_reset on v2:172.21.15.33:6832/963232630 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 >> 172.21.15.33:0/962514031 conn(0x557099dea070 msgr2=0x557099e276b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 shutdown_connections 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 1 -- 172.21.15.33:0/962514031 wait complete. 2023-10-06T05:29:26.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.652+0000 7fe214c1eb80 20 client.10581 trim_cache size 0 max 16384 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 10419, 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5360011010000001, 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 162, 2023-10-06T05:29:26.661 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 210, 2023-10-06T05:29:26.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 210, 2023-10-06T05:29:26.662 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 13.276656123 2023-10-06T05:29:26.662 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ec066f30 msgr2=0x7fe1ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ec066f30 0x7fe1ec0693e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0006fd0 tx=0x7fe1f0008040 comp rx=0 tx=0).stop 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 msgr2=0x7fe20015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe20015a1f0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fe20006e5d0 tx=0x7fe1f4017000 comp rx=0 tx=0).stop 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 shutdown_connections 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe1ec066f30 0x7fe1ec0693e0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2001597c0 0x7fe2000c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20006acd0 0x7fe200159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 --2- 172.21.15.33:0/3122001080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2000c9d20 0x7fe20015a1f0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 >> 172.21.15.33:0/3122001080 conn(0x7fe2000cd600 msgr2=0x7fe2000c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 shutdown_connections 2023-10-06T05:29:26.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.653+0000 7fe2063ff700 1 -- 172.21.15.33:0/3122001080 wait complete. 2023-10-06T05:29:26.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:29:26.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.856+0000 7fc66759e700 1 Processor -- start 2023-10-06T05:29:26.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.856+0000 7fc66759e700 1 -- start start 2023-10-06T05:29:26.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc6680c6340 0x7fc6680c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680c80d0 0x7fc66806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66806c820 con 0x7fc668159520 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66806a160 con 0x7fc6680c6340 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680c8800 con 0x7fc6680c80d0 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc665d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc6680c6340 0x7fc6680c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc668159520 0x7fc6680c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58342/0 (socket says 172.21.15.33:58342) 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66659c700 1 -- 172.21.15.33:0/362640260 learned_addr learned my addr 172.21.15.33:0/362640260 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:26.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc666d9d700 1 --2- 172.21.15.33:0/362640260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680c80d0 0x7fc66806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66559a700 1 -- 172.21.15.33:0/362640260 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029818737 0 0) 0x7fc66806c820 con 0x7fc668159520 2023-10-06T05:29:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc66559a700 1 -- 172.21.15.33:0/362640260 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc650003570 con 0x7fc668159520 2023-10-06T05:29:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc665d9b700 1 -- 172.21.15.33:0/362640260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680c80d0 msgr2=0x7fc66806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:26.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc665d9b700 1 --2- 172.21.15.33:0/362640260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680c80d0 0x7fc66806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc665d9b700 1 -- 172.21.15.33:0/362640260 >> v1:172.21.15.33:6789/0 conn(0x7fc668159520 legacy=0x7fc6680c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.858+0000 7fc665d9b700 1 -- 172.21.15.33:0/362640260 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680c8ae0 con 0x7fc6680c6340 2023-10-06T05:29:26.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.859+0000 7fc665d9b700 1 --2- 172.21.15.33:0/362640260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc6680c7b90 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0098e0 tx=0x7fc65c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df7d69d3c7764e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66559a700 1 -- 172.21.15.33:0/362640260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011630 con 0x7fc6680c6340 2023-10-06T05:29:26.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66559a700 1 -- 172.21.15.33:0/362640260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c011790 con 0x7fc6680c6340 2023-10-06T05:29:26.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66559a700 1 -- 172.21.15.33:0/362640260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011b30 con 0x7fc6680c6340 2023-10-06T05:29:26.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 -- 172.21.15.33:0/362640260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 msgr2=0x7fc6680c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 --2- 172.21.15.33:0/362640260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc6680c7b90 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0098e0 tx=0x7fc65c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 -- 172.21.15.33:0/362640260 shutdown_connections 2023-10-06T05:29:26.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 --2- 172.21.15.33:0/362640260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680c80d0 0x7fc66806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 --2- 172.21.15.33:0/362640260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc6680c7b90 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 -- 172.21.15.33:0/362640260 >> 172.21.15.33:0/362640260 conn(0x7fc6680cc930 msgr2=0x7fc6680ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:26.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 -- 172.21.15.33:0/362640260 shutdown_connections 2023-10-06T05:29:26.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.861+0000 7fc66759e700 1 -- 172.21.15.33:0/362640260 wait complete. 2023-10-06T05:29:26.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.863+0000 7fc66759e700 1 Processor -- start 2023-10-06T05:29:26.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.863+0000 7fc66759e700 1 -- start start 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc66814e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680c80d0 0x7fc66814f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc668159520 0x7fc66815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815dd70 con 0x7fc6680c6340 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66814eb50 con 0x7fc6680c80d0 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815d160 con 0x7fc668159520 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc66814e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc66814e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49032/0 (socket says 172.21.15.33:49032) 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.864+0000 7fc66659c700 1 -- 172.21.15.33:0/2499908886 learned_addr learned my addr 172.21.15.33:0/2499908886 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66659c700 1 -- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc668159520 msgr2=0x7fc66815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66659c700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc668159520 0x7fc66815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66659c700 1 -- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680c80d0 msgr2=0x7fc66814f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc665d9b700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680c80d0 0x7fc66814f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66659c700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680c80d0 0x7fc66814f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66659c700 1 -- 172.21.15.33:0/2499908886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009420 con 0x7fc6680c6340 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66659c700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc66814e610 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c590 tx=0x7fc658007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658005b30 con 0x7fc6680c6340 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658005cb0 con 0x7fc6680c6340 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66815d440 con 0x7fc6680c6340 2023-10-06T05:29:26.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658007320 con 0x7fc6680c6340 2023-10-06T05:29:26.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.865+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66815da00 con 0x7fc6680c6340 2023-10-06T05:29:26.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.866+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66804c380 con 0x7fc6680c6340 2023-10-06T05:29:26.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.867+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc65801b3b0 con 0x7fc6680c6340 2023-10-06T05:29:26.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.868+0000 7fc6567fc700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc648066f30 0x7fc6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:26.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.868+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc658083cf0 con 0x7fc6680c6340 2023-10-06T05:29:26.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.868+0000 7fc665d9b700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc648066f30 0x7fc6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:26.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.869+0000 7fc665d9b700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc648066f30 0x7fc6480693e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc6680c6740 tx=0x7fc65c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:26.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:26.872+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc658050d60 con 0x7fc6680c6340 2023-10-06T05:29:27.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.123+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc6680c8dc0 con 0x7fc6680c6340 2023-10-06T05:29:27.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.124+0000 7fc6567fc700 1 -- 172.21.15.33:0/2499908886 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v162) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc658017020 con 0x7fc6680c6340 2023-10-06T05:29:27.126 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:29:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc648066f30 msgr2=0x7fc6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc648066f30 0x7fc6480693e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc6680c6740 tx=0x7fc65c017000 comp rx=0 tx=0).stop 2023-10-06T05:29:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 msgr2=0x7fc66814e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:27.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc66814e610 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c590 tx=0x7fc658007e00 comp rx=0 tx=0).stop 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 shutdown_connections 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc648066f30 0x7fc6480693e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc668159520 0x7fc66815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680c6340 0x7fc66814e610 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 --2- 172.21.15.33:0/2499908886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6680c80d0 0x7fc66814f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 >> 172.21.15.33:0/2499908886 conn(0x7fc6680cc930 msgr2=0x7fc668151450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 shutdown_connections 2023-10-06T05:29:27.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:27.126+0000 7fc66759e700 1 -- 172.21.15.33:0/2499908886 wait complete. 2023-10-06T05:29:27.146 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:29:27.146 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:29:27.147 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:29:27.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.147 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:29:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:27.186 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41390sec preferred_lft 41390sec 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:29:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:29:27.189 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:29:27.189 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:27.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.189 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:29:27.285 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:29:27.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.289 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:29:27.327 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:29:27.328 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:29:27.328 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:29:27.328 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:29:27.328 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:29:27.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.329 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:29:27.329 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:29:27.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:29:27.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:29:27.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:29:27.330 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:29:27.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.528 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:29:27.564 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:29:27.564 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:29:27.565 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:29:27.565 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:29:27.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:29:27.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:29:27.667 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:29:27.704 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:29:27.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:29:27.704 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:29:27.780 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:29:27.780 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:29:27.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:29:27.914 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:29:27.914 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:29:27.914 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:29:27.930 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:29:27.930 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:29:27.931 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:29:27.931 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:29:27.931 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:29:27.950 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:29:27.951 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:29:27.956 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:29:27.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:27.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:29:28.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:29:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.207+0000 7faa26b9d700 1 Processor -- start 2023-10-06T05:29:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.207+0000 7faa26b9d700 1 -- start start 2023-10-06T05:29:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa26b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faa20155af0 0x7faa2006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa26b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2006ab60 con 0x7faa2006a6a0 2023-10-06T05:29:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa26b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2006ae40 con 0x7faa200c7920 2023-10-06T05:29:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa26b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2006b7e0 con 0x7faa20155af0 2023-10-06T05:29:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa2539a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faa2006a6a0 0x7faa200c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58344/0 (socket says 172.21.15.33:58344) 2023-10-06T05:29:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa2539a700 1 -- 172.21.15.33:0/597390842 learned_addr learned my addr 172.21.15.33:0/597390842 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa25b9b700 1 --2- 172.21.15.33:0/597390842 >> v2:172.21.15.167:3300/0 conn(0x7faa20155af0 0x7faa2006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524222374 0 0) 0x7faa2006ab60 con 0x7faa2006a6a0 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.209+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0c003570 con 0x7faa2006a6a0 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 504545354 0 0) 0x7faa0c003570 con 0x7faa2006a6a0 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 >> v2:172.21.15.167:3300/0 conn(0x7faa20155af0 msgr2=0x7faa2006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 --2- 172.21.15.33:0/597390842 >> v2:172.21.15.167:3300/0 conn(0x7faa20155af0 0x7faa2006a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 >> v1:172.21.15.167:6789/0 conn(0x7faa200c7920 legacy=0x7faa200c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2006b120 con 0x7faa2006a6a0 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7faa10001470 con 0x7faa2006a6a0 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 >> v1:172.21.15.33:6789/0 conn(0x7faa2006a6a0 legacy=0x7faa200c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa0c003a60 0x7faa0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2006b7e0 con 0x7faa0c0051c0 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c00e370 con 0x7faa0c004810 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c00e530 con 0x7faa0c003a60 2023-10-06T05:29:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa2639c700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa25b9b700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa0c003a60 0x7faa0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.210+0000 7faa2539a700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 -- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa0c003a60 msgr2=0x7faa0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa0c003a60 0x7faa0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 -- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c004810 msgr2=0x7faa0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c004810 0x7faa0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 -- 172.21.15.33:0/597390842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c003570 con 0x7faa0c0051c0 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 -- 172.21.15.33:0/597390842 reap_dead start 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa2639c700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7faa200c7d20 tx=0x7faa18001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc54b2b268010c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18001950 con 0x7faa0c0051c0 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18004030 con 0x7faa0c0051c0 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.211+0000 7faa24b99700 1 -- 172.21.15.33:0/597390842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800cae0 con 0x7faa0c0051c0 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 -- 172.21.15.33:0/597390842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0c0051c0 msgr2=0x7faa0c00dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7faa200c7d20 tx=0x7faa18001640 comp rx=0 tx=0).stop 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 -- 172.21.15.33:0/597390842 shutdown_connections 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 --2- 172.21.15.33:0/597390842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0c0051c0 0x7faa0c00dc70 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 -- 172.21.15.33:0/597390842 >> 172.21.15.33:0/597390842 conn(0x7faa2006d030 msgr2=0x7faa2006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 -- 172.21.15.33:0/597390842 shutdown_connections 2023-10-06T05:29:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.212+0000 7faa26b9d700 1 -- 172.21.15.33:0/597390842 wait complete. 2023-10-06T05:29:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.213+0000 7faa26b9d700 1 Processor -- start 2023-10-06T05:29:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.213+0000 7faa26b9d700 1 -- start start 2023-10-06T05:29:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.214+0000 7faa26b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0051c0 0x7faa20166ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.214+0000 7faa26b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 0x7faa200ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.214+0000 7faa26b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa200ad950 0x7faa20168ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.214+0000 7faa26b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200adf00 con 0x7faa200c5940 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.214+0000 7faa26b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200af050 con 0x7faa0c0051c0 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.214+0000 7faa26b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200af330 con 0x7faa200ad950 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 0x7faa200ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 0x7faa200ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49052/0 (socket says 172.21.15.33:49052) 2023-10-06T05:29:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 -- 172.21.15.33:0/288968727 learned_addr learned my addr 172.21.15.33:0/288968727 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa25b9b700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0051c0 0x7faa20166ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2639c700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa200ad950 0x7faa20168ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 -- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa200ad950 msgr2=0x7faa20168ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa200ad950 0x7faa20168ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 -- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0051c0 msgr2=0x7faa20166ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0051c0 0x7faa20166ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 -- 172.21.15.33:0/288968727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200af610 con 0x7faa200c5940 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2639c700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa200ad950 0x7faa20168ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa2539a700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 0x7faa200ad410 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7faa100073c0 tx=0x7faa10001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.215+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10018030 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.216+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa10001040 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.216+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f930 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.216+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa200af8f0 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.216+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200b1720 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.217+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa200afae0 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.218+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa1001b070 con 0x7faa200c5940 2023-10-06T05:29:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.218+0000 7faa15ffb700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa04066f30 0x7faa040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.219+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa10019070 con 0x7faa200c5940 2023-10-06T05:29:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.219+0000 7faa25b9b700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa04066f30 0x7faa040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.220+0000 7faa25b9b700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa04066f30 0x7faa040693e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7faa20166a70 tx=0x7faa1c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.223+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa100514b0 con 0x7faa200c5940 2023-10-06T05:29:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.473+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa2004c380 con 0x7faa200c5940 2023-10-06T05:29:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.473+0000 7faa15ffb700 1 -- 172.21.15.33:0/288968727 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v162) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa10054ad0 con 0x7faa200c5940 2023-10-06T05:29:28.476 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:29:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa04066f30 msgr2=0x7faa040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa04066f30 0x7faa040693e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7faa20166a70 tx=0x7faa1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 msgr2=0x7faa200ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 0x7faa200ad410 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7faa100073c0 tx=0x7faa10001e50 comp rx=0 tx=0).stop 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 shutdown_connections 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa04066f30 0x7faa040693e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa200ad950 0x7faa20168ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0c0051c0 0x7faa20166ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 --2- 172.21.15.33:0/288968727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa200c5940 0x7faa200ad410 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 >> 172.21.15.33:0/288968727 conn(0x7faa2006d030 msgr2=0x7faa200c6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.475+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 shutdown_connections 2023-10-06T05:29:28.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:28.476+0000 7faa26b9d700 1 -- 172.21.15.33:0/288968727 wait complete. 2023-10-06T05:29:28.495 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:29:28.495 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:29:28.495 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:29:28.496 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:29:28.496 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:29:28.496 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:29:28.496 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:29:28.496 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:29:28.496 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:29:28.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:28.496 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:29:28.538 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41393sec preferred_lft 41393sec 2023-10-06T05:29:28.539 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.540 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:29:28.541 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:29:28.541 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:29:28.541 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:29:28.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:28.541 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:29:28.641 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:29:28.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:28.644 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:29:28.683 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:29:28.683 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:29:28.683 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:29:28.683 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:29:28.683 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:29:28.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:28.685 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:29:28.685 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:29:28.685 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:29:28.685 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:29:28.685 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:29:28.685 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:29:28.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:28.880 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:29:28.916 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:29:28.917 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:29:28.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:29:28.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:28.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:29:29.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:29:29.018 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:29:29.054 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:29:29.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:29:29.055 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:29:29.130 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:29:29.130 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:29:29.130 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:29:29.260 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:29:29.260 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:29:29.260 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:29:29.276 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:29:29.276 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:29:29.276 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:29:29.277 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:29:29.277 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:29:29.297 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:29:29.297 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:29:29.298 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:29:29.298 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:29:29.298 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:29:29.298 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:29:29.298 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:29:29.298 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:29:29.301 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:29:29.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:29.302 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:29:29.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:29:29.580 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:29:29.581 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:29:29.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:29:29.787 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:29:29.787 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:29:29.787 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:29:29.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:29.788 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:29:29.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:29.788 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:29:29.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:29.789 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:29:29.789 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:29.801 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:29:29.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2023-10-06T05:29:29.802 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:29:29.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2023-10-06T05:29:29.803 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:29:29.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2023-10-06T05:29:29.806 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:29:29.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2023-10-06T05:29:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.042+0000 7f532d79d700 1 Processor -- start 2023-10-06T05:29:30.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.042+0000 7f81db59e700 1 Processor -- start 2023-10-06T05:29:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.042+0000 7f29e0f9d700 1 Processor -- start 2023-10-06T05:29:30.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.043+0000 7f81db59e700 1 -- start start 2023-10-06T05:29:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.043+0000 7f532d79d700 1 -- start start 2023-10-06T05:29:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.043+0000 7f29e0f9d700 1 -- start start 2023-10-06T05:29:30.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f81db59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f81dc140d20 0x7f81dc0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f29e0f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 0x7f29dc1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f81db59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f81dc0c9210 0x7f81dc0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f81db59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc15bcb0 con 0x7f81dc069f00 2023-10-06T05:29:30.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f29e0f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc15a380 con 0x7f29dc0c8090 2023-10-06T05:29:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f81db59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc0c9690 con 0x7f81dc140d20 2023-10-06T05:29:30.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f29e0f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc15a660 con 0x7f29dc0cc9b0 2023-10-06T05:29:30.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f81db59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc1410f0 con 0x7f81dc0c9210 2023-10-06T05:29:30.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f81d9d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f81dc0c9210 0x7f81dc0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f81d9d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f81dc0c9210 0x7f81dc0699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:39930/0 (socket says 172.21.15.33:39930) 2023-10-06T05:29:30.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f81d9d9b700 1 -- 172.21.15.33:0/1512504646 learned_addr learned my addr 172.21.15.33:0/1512504646 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f81da59c700 1 --2- 172.21.15.33:0/1512504646 >> v2:172.21.15.33:3300/0 conn(0x7f81dc140d20 0x7f81dc0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.044+0000 7f29db7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 0x7f29dc1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f29db7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 0x7f29dc1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55002/0 (socket says 172.21.15.33:55002) 2023-10-06T05:29:30.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f29db7fe700 1 -- 172.21.15.33:0/1158083609 learned_addr learned my addr 172.21.15.33:0/1158083609 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f29e0f9d700 1 -- 172.21.15.33:0/1158083609 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0cc210 con 0x7f29dc0cc5e0 2023-10-06T05:29:30.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f81d959a700 1 -- 172.21.15.33:0/1512504646 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773560175 0 0) 0x7f81dc15bcb0 con 0x7f81dc069f00 2023-10-06T05:29:30.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f81d959a700 1 -- 172.21.15.33:0/1512504646 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c4003570 con 0x7f81dc069f00 2023-10-06T05:29:30.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f29db7fe700 1 -- 172.21.15.33:0/1158083609 >> v1:172.21.15.167:6789/0 conn(0x7f29dc0cc9b0 legacy=0x7f29dc0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f532d79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5328159400 0x7f53280cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f532d79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f53281597d0 0x7f53280cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f532d79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53280d0ed0 con 0x7f53280cc770 2023-10-06T05:29:30.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f532d79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280b1160 con 0x7f53281597d0 2023-10-06T05:29:30.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f29db7fe700 1 -- 172.21.15.33:0/1158083609 >> v1:172.21.15.33:6789/0 conn(0x7f29dc0c8090 legacy=0x7f29dc06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.045+0000 7f532d79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280cb6d0 con 0x7f5328159400 2023-10-06T05:29:30.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5328159400 0x7f53280cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5328159400 0x7f53280cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:39932/0 (socket says 172.21.15.33:39932) 2023-10-06T05:29:30.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 -- 172.21.15.33:0/3315388102 learned_addr learned my addr 172.21.15.33:0/3315388102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f81d9d9b700 1 -- 172.21.15.33:0/1512504646 >> v2:172.21.15.33:3300/0 conn(0x7f81dc140d20 msgr2=0x7f81dc0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:30.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f81d9d9b700 1 --2- 172.21.15.33:0/1512504646 >> v2:172.21.15.33:3300/0 conn(0x7f81dc140d20 0x7f81dc0c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f81d9d9b700 1 -- 172.21.15.33:0/1512504646 >> v1:172.21.15.167:6789/0 conn(0x7f81dc069f00 legacy=0x7f81dc0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f81d9d9b700 1 -- 172.21.15.33:0/1512504646 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc06a2f0 con 0x7f81dc0c9210 2023-10-06T05:29:30.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f29db7fe700 1 -- 172.21.15.33:0/1158083609 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc15a940 con 0x7f29dc0cc5e0 2023-10-06T05:29:30.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f81d9d9b700 1 --2- 172.21.15.33:0/1512504646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0699c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0c66e0 tx=0x7f81d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30c2392f7ba4cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f53277fe700 1 --2- 172.21.15.33:0/3315388102 >> v2:172.21.15.33:3300/0 conn(0x7f53281597d0 0x7f53280cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f29db7fe700 1 --2- 172.21.15.33:0/1158083609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 0x7f29dc1571d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0099e0 tx=0x7f29cc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8798258d8d349a08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 -- 172.21.15.33:0/3315388102 >> v2:172.21.15.33:3300/0 conn(0x7f53281597d0 msgr2=0x7f53280cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:30.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 --2- 172.21.15.33:0/3315388102 >> v2:172.21.15.33:3300/0 conn(0x7f53281597d0 0x7f53280cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f81d959a700 1 -- 172.21.15.33:0/1512504646 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000b640 con 0x7f81dc0c9210 2023-10-06T05:29:30.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 -- 172.21.15.33:0/3315388102 >> v1:172.21.15.33:6789/0 conn(0x7f53280cc770 legacy=0x7f532806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81d959a700 1 -- 172.21.15.33:0/1512504646 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d000b7a0 con 0x7f81dc0c9210 2023-10-06T05:29:30.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81d959a700 1 -- 172.21.15.33:0/1512504646 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000bb60 con 0x7f81dc0c9210 2023-10-06T05:29:30.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f29da7fc700 1 -- 172.21.15.33:0/1158083609 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00b860 con 0x7f29dc0cc5e0 2023-10-06T05:29:30.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f29da7fc700 1 -- 172.21.15.33:0/1158083609 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc00b9c0 con 0x7f29dc0cc5e0 2023-10-06T05:29:30.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.046+0000 7f5327fff700 1 -- 172.21.15.33:0/3315388102 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280cb9b0 con 0x7f5328159400 2023-10-06T05:29:30.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f29da7fc700 1 -- 172.21.15.33:0/1158083609 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00bd60 con 0x7f29dc0cc5e0 2023-10-06T05:29:30.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f5327fff700 1 --2- 172.21.15.33:0/3315388102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f53280cab60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f532806c030 tx=0x7f5318009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b46d0fe3f91cc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f5326ffd700 1 -- 172.21.15.33:0/3315388102 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531800b860 con 0x7f5328159400 2023-10-06T05:29:30.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81db59e700 1 -- 172.21.15.33:0/1512504646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 msgr2=0x7f81dc0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f5326ffd700 1 -- 172.21.15.33:0/3315388102 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531800b9c0 con 0x7f5328159400 2023-10-06T05:29:30.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f5326ffd700 1 -- 172.21.15.33:0/3315388102 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531800bd60 con 0x7f5328159400 2023-10-06T05:29:30.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81db59e700 1 --2- 172.21.15.33:0/1512504646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0699c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0c66e0 tx=0x7f81d0009250 comp rx=0 tx=0).stop 2023-10-06T05:29:30.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f29e0f9d700 1 -- 172.21.15.33:0/1158083609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 msgr2=0x7f29dc1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/1158083609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 0x7f29dc1571d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0099e0 tx=0x7f29cc009250 comp rx=0 tx=0).stop 2023-10-06T05:29:30.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81db59e700 1 -- 172.21.15.33:0/1512504646 shutdown_connections 2023-10-06T05:29:30.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81db59e700 1 --2- 172.21.15.33:0/1512504646 >> v2:172.21.15.33:3300/0 conn(0x7f81dc140d20 0x7f81dc0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.047+0000 7f81db59e700 1 --2- 172.21.15.33:0/1512504646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0699c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f81db59e700 1 -- 172.21.15.33:0/1512504646 >> 172.21.15.33:0/1512504646 conn(0x7f81dc06c560 msgr2=0x7f81dc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f29e0f9d700 1 -- 172.21.15.33:0/1158083609 shutdown_connections 2023-10-06T05:29:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/1158083609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc5e0 0x7f29dc1571d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f29e0f9d700 1 -- 172.21.15.33:0/1158083609 >> 172.21.15.33:0/1158083609 conn(0x7f29dc06c560 msgr2=0x7f29dc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f81db59e700 1 -- 172.21.15.33:0/1512504646 shutdown_connections 2023-10-06T05:29:30.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f81db59e700 1 -- 172.21.15.33:0/1512504646 wait complete. 2023-10-06T05:29:30.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f29e0f9d700 1 -- 172.21.15.33:0/1158083609 shutdown_connections 2023-10-06T05:29:30.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 -- 172.21.15.33:0/3315388102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 msgr2=0x7f53280cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 --2- 172.21.15.33:0/3315388102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f53280cab60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f532806c030 tx=0x7f5318009250 comp rx=0 tx=0).stop 2023-10-06T05:29:30.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 -- 172.21.15.33:0/3315388102 shutdown_connections 2023-10-06T05:29:30.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 --2- 172.21.15.33:0/3315388102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f53280cab60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 --2- 172.21.15.33:0/3315388102 >> v2:172.21.15.33:3300/0 conn(0x7f53281597d0 0x7f53280cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 -- 172.21.15.33:0/3315388102 >> 172.21.15.33:0/3315388102 conn(0x7f532806e4b0 msgr2=0x7f53280d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f29e0f9d700 1 -- 172.21.15.33:0/1158083609 wait complete. 2023-10-06T05:29:30.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.048+0000 7f532d79d700 1 -- 172.21.15.33:0/3315388102 shutdown_connections 2023-10-06T05:29:30.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.049+0000 7f532d79d700 1 -- 172.21.15.33:0/3315388102 wait complete. 2023-10-06T05:29:30.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.049+0000 7f81db59e700 1 Processor -- start 2023-10-06T05:29:30.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.050+0000 7f81db59e700 1 -- start start 2023-10-06T05:29:30.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.050+0000 7f29e0f9d700 1 Processor -- start 2023-10-06T05:29:30.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.050+0000 7f29e0f9d700 1 -- start start 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.050+0000 7f81db59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 0x7f81dc1695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81db59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc140d20 0x7f81dc13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81db59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc169ae0 con 0x7f81dc069f00 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81db59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0c40b0 con 0x7f81dc0c9210 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0c31c0 con 0x7f81dc140d20 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 0x7f81dc1695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 0x7f81dc1695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59568/0 (socket says 172.21.15.33:59568) 2023-10-06T05:29:30.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81da59c700 1 -- 172.21.15.33:0/1158450370 learned_addr learned my addr 172.21.15.33:0/1158450370 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f532d79d700 1 Processor -- start 2023-10-06T05:29:30.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81d9d9b700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f532d79d700 1 -- start start 2023-10-06T05:29:30.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.051+0000 7f81dad9d700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc140d20 0x7f81dc13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81da59c700 1 -- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc140d20 msgr2=0x7f81dc13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81da59c700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc140d20 0x7f81dc13eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81da59c700 1 -- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 msgr2=0x7f81dc0cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81da59c700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81da59c700 1 -- 172.21.15.33:0/1158450370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00094f0 con 0x7f81dc069f00 2023-10-06T05:29:30.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81d9d9b700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:30.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29e0f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 0x7f29dc153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f81da59c700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 0x7f81dc1695a0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00e930 tx=0x7f81cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f532d79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 0x7f532816d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29e0f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29dc0cc5e0 0x7f29dc0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29e0f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc9b0 0x7f29dc0c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29e0f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0b1a60 con 0x7f29dc0cc5e0 2023-10-06T05:29:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29e0f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0b1d40 con 0x7f29dc0c8090 2023-10-06T05:29:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29e0f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc152880 con 0x7f29dc0cc9b0 2023-10-06T05:29:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f532d79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f532814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f532d79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53281597d0 0x7f532806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.052+0000 7f29db7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 0x7f29dc153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 0x7f29dc153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:39950/0 (socket says 172.21.15.33:39950) 2023-10-06T05:29:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 -- 172.21.15.33:0/371841195 learned_addr learned my addr 172.21.15.33:0/371841195 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f5327fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 0x7f532816d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29dbfff700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc9b0 0x7f29dc0c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f5327fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 0x7f532816d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55020/0 (socket says 172.21.15.33:55020) 2023-10-06T05:29:30.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00eed0 con 0x7f81dc069f00 2023-10-06T05:29:30.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc00c850 con 0x7f81dc069f00 2023-10-06T05:29:30.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00cbf0 con 0x7f81dc069f00 2023-10-06T05:29:30.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f53277fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f532814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f53277fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f532814c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:39966/0 (socket says 172.21.15.33:39966) 2023-10-06T05:29:30.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29daffd700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29dc0cc5e0 0x7f29dc0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532816de80 con 0x7f53281597d0 2023-10-06T05:29:30.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328169d20 con 0x7f5328159400 2023-10-06T05:29:30.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc0c34a0 con 0x7f81dc069f00 2023-10-06T05:29:30.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532816a000 con 0x7f53280cc770 2023-10-06T05:29:30.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f5327fff700 1 -- 172.21.15.33:0/1695604150 learned_addr learned my addr 172.21.15.33:0/1695604150 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 -- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc9b0 msgr2=0x7f29dc0c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc9b0 0x7f29dc0c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 -- 172.21.15.33:0/371841195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29dc0cc5e0 msgr2=0x7f29dc0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc0c3a60 con 0x7f81dc069f00 2023-10-06T05:29:30.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29dc0cc5e0 0x7f29dc0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29db7fe700 1 -- 172.21.15.33:0/371841195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0094f0 con 0x7f29dc0c8090 2023-10-06T05:29:30.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f532cf9c700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53281597d0 0x7f532806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.053+0000 7f29daffd700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29dc0cc5e0 0x7f29dc0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:30.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f5327fff700 1 -- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 msgr2=0x7f532814c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f5327fff700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f532814c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f5327fff700 1 -- 172.21.15.33:0/1695604150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53281597d0 msgr2=0x7f532806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f5327fff700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53281597d0 0x7f532806c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f5327fff700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180094f0 con 0x7f53280cc770 2023-10-06T05:29:30.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f532cf9c700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53281597d0 0x7f532806c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:30.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f29db7fe700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 0x7f29dc153360 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009290 tx=0x7f29cc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f5327fff700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 0x7f532816d940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5318009290 tx=0x7f5318017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01a630 con 0x7f29dc0c8090 2023-10-06T05:29:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc01a7b0 con 0x7f29dc0c8090 2023-10-06T05:29:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc152b60 con 0x7f29dc0c8090 2023-10-06T05:29:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01ab70 con 0x7f29dc0c8090 2023-10-06T05:29:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531801a630 con 0x7f53280cc770 2023-10-06T05:29:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc0af1e0 con 0x7f29dc0c8090 2023-10-06T05:29:30.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531801a7b0 con 0x7f53280cc770 2023-10-06T05:29:30.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531801ab70 con 0x7f53280cc770 2023-10-06T05:29:30.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.054+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532816a2e0 con 0x7f53280cc770 2023-10-06T05:29:30.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.055+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532816a8a0 con 0x7f53280cc770 2023-10-06T05:29:30.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.055+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc0c4390 con 0x7f81dc069f00 2023-10-06T05:29:30.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.056+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc0af370 con 0x7f29dc0c8090 2023-10-06T05:29:30.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.056+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53280b32c0 con 0x7f53280cc770 2023-10-06T05:29:30.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.057+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f531802d440 con 0x7f53280cc770 2023-10-06T05:29:30.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81cc013470 con 0x7f81dc069f00 2023-10-06T05:29:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29cc021400 con 0x7f29dc0c8090 2023-10-06T05:29:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7f530ffff700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5314067080 0x7f5314069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f53180862f0 con 0x7f53280cc770 2023-10-06T05:29:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7fb3549fe700 1 Processor -- start 2023-10-06T05:29:30.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7f53277fe700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5314067080 0x7f5314069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.058+0000 7fb3549fe700 1 -- start start 2023-10-06T05:29:30.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.059+0000 7f29c37fe700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f29c8066f30 0x7f29c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.059+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f29cc088260 con 0x7f29dc0c8090 2023-10-06T05:29:30.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.059+0000 7f53277fe700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5314067080 0x7f5314069530 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5328157a80 tx=0x7f531c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.059+0000 7f29daffd700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f29c8066f30 0x7f29c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.059+0000 7f81ca7fc700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f81bc067000 0x7f81bc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.059+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f81cc084840 con 0x7f81dc069f00 2023-10-06T05:29:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7f81d9d9b700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f81bc067000 0x7f81bc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7f29daffd700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f29c8066f30 0x7f29c80693e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f29d0007fd0 tx=0x7f29d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7fb3549fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb350159400 0x7fb3500cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7fb3549fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7fb3549fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3500d0ed0 con 0x7fb3500cc770 2023-10-06T05:29:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7fb3549fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3500b1160 con 0x7fb350159400 2023-10-06T05:29:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7fb3549fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500cb6d0 con 0x7fb3501597d0 2023-10-06T05:29:30.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.060+0000 7f81d9d9b700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f81bc067000 0x7f81bc0694b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0c66e0 tx=0x7f81d0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb350159400 0x7fb3500cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb350159400 0x7fb3500cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59590/0 (socket says 172.21.15.33:59590) 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 -- 172.21.15.33:0/690334569 learned_addr learned my addr 172.21.15.33:0/690334569 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34659c700 1 --2- 172.21.15.33:0/690334569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 -- 172.21.15.33:0/690334569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 msgr2=0x7fb3500cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 --2- 172.21.15.33:0/690334569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 -- 172.21.15.33:0/690334569 >> v1:172.21.15.33:6789/0 conn(0x7fb3500cc770 legacy=0x7fb35006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 -- 172.21.15.33:0/690334569 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500cb9b0 con 0x7fb350159400 2023-10-06T05:29:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.061+0000 7fb34ed9d700 1 --2- 172.21.15.33:0/690334569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb3500cab60 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fb3380099b0 tx=0x7fb338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba97318ace113fc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.063+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53280b32c0 con 0x7f53280cc770 2023-10-06T05:29:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.064+0000 7fb34e59c700 1 -- 172.21.15.33:0/690334569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338009ea0 con 0x7fb350159400 2023-10-06T05:29:30.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.064+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29cc055350 con 0x7f29dc0c8090 2023-10-06T05:29:30.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.064+0000 7fb34e59c700 1 -- 172.21.15.33:0/690334569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800b860 con 0x7fb350159400 2023-10-06T05:29:30.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.064+0000 7fb34e59c700 1 -- 172.21.15.33:0/690334569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800bc00 con 0x7fb350159400 2023-10-06T05:29:30.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.064+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81cc0519d0 con 0x7f81dc069f00 2023-10-06T05:29:30.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.065+0000 7fb3549fe700 1 -- 172.21.15.33:0/690334569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 msgr2=0x7fb3500cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.065+0000 7fb3549fe700 1 --2- 172.21.15.33:0/690334569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb3500cab60 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fb3380099b0 tx=0x7fb338009250 comp rx=0 tx=0).stop 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.067+0000 7fb3549fe700 1 -- 172.21.15.33:0/690334569 shutdown_connections 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.067+0000 7fb3549fe700 1 --2- 172.21.15.33:0/690334569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb3500cab60 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.067+0000 7fb3549fe700 1 --2- 172.21.15.33:0/690334569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.067+0000 7fb3549fe700 1 -- 172.21.15.33:0/690334569 >> 172.21.15.33:0/690334569 conn(0x7fb35006e4b0 msgr2=0x7fb3500d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.067+0000 7fb3549fe700 1 -- 172.21.15.33:0/690334569 shutdown_connections 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.067+0000 7fb3549fe700 1 -- 172.21.15.33:0/690334569 wait complete. 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.069+0000 7fb3549fe700 1 Processor -- start 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.069+0000 7fb3549fe700 1 -- start start 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb3549fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3500cc770 0x7fb350148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb3549fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb35015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb3549fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb3549fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb350157010 con 0x7fb350159400 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb3549fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35015e850 con 0x7fb3500cc770 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb3549fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35015e570 con 0x7fb3501597d0 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.071+0000 7fb34659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb35015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb35015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59596/0 (socket says 172.21.15.33:59596) 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3500cc770 0x7fb350148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34659c700 1 -- 172.21.15.33:0/209453883 learned_addr learned my addr 172.21.15.33:0/209453883 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34f59e700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 -- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 msgr2=0x7fb3500b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 -- 172.21.15.33:0/209453883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 msgr2=0x7fb35015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb35015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 -- 172.21.15.33:0/209453883 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380094f0 con 0x7fb3500cc770 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34f59e700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34659c700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb35015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.072+0000 7fb34ed9d700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3500cc770 0x7fb350148f40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb3500cb5e0 tx=0x7fb338017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.073+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338000d10 con 0x7fb3500cc770 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.073+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800b9d0 con 0x7fb3500cc770 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.073+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338026640 con 0x7fb3500cc770 2023-10-06T05:29:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.073+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3501569c0 con 0x7fb3500cc770 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.073+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb350149480 con 0x7fb3500cc770 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.075+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb350156820 con 0x7fb3500cc770 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.075+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb338023030 con 0x7fb3500cc770 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.076+0000 7fb3477fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.076+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb338085e60 con 0x7fb3500cc770 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.077+0000 7fb34659c700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.077+0000 7fb34659c700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb3500b5f70 tx=0x7fb3400091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.081+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb350156820 con 0x7fb3500cc770 2023-10-06T05:29:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.374+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f29dc04c380 con 0x7f29dc0c8090 2023-10-06T05:29:30.378 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:29:30.375+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:29:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.377+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f532804c380 con 0x7f53280cc770 2023-10-06T05:29:30.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.381+0000 7f29c37fe700 1 -- 172.21.15.33:0/371841195 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 10419 v163) v1 ==== 96+0+0 (secure 0 0 0) 0x7f29cc058970 con 0x7f29dc0c8090 2023-10-06T05:29:30.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f29c8066f30 msgr2=0x7f29c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f29c8066f30 0x7f29c80693e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f29d0007fd0 tx=0x7f29d0009040 comp rx=0 tx=0).stop 2023-10-06T05:29:30.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 msgr2=0x7f29dc153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 0x7f29dc153360 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009290 tx=0x7f29cc017000 comp rx=0 tx=0).stop 2023-10-06T05:29:30.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 shutdown_connections 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f29c8066f30 0x7f29c80693e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29dc0cc9b0 0x7f29dc0c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29dc0c8090 0x7f29dc153360 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 --2- 172.21.15.33:0/371841195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29dc0cc5e0 0x7f29dc0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 >> 172.21.15.33:0/371841195 conn(0x7f29dc06c560 msgr2=0x7f29dc1505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 shutdown_connections 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.384+0000 7f29e0f9d700 1 -- 172.21.15.33:0/371841195 wait complete. 2023-10-06T05:29:30.388 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 10419 2023-10-06T05:29:30.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.400+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f81dc04c380 con 0x7f81dc069f00 2023-10-06T05:29:30.414 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:29:30.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:29:30.419 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:29:30.419 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:30.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.420+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fb350156820 con 0x7fb3500cc770 2023-10-06T05:29:30.551 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:29:30.874 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:29:30.871+0000 7f50273ec700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:29:30.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.879+0000 7f81ca7fc700 1 -- 172.21.15.33:0/1158450370 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 8045 v165) v1 ==== 95+0+0 (secure 0 0 0) 0x7f81cc054ff0 con 0x7f81dc069f00 2023-10-06T05:29:30.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.880+0000 7f530ffff700 1 -- 172.21.15.33:0/1695604150 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 9463 v165) v1 ==== 95+0+0 (secure 0 0 0) 0x7f5318053460 con 0x7f53280cc770 2023-10-06T05:29:30.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.880+0000 7fb3477fe700 1 -- 172.21.15.33:0/209453883 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 9469 v165) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb350156820 con 0x7fb3500cc770 2023-10-06T05:29:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f81bc067000 msgr2=0x7f81bc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f81bc067000 0x7f81bc0694b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0c66e0 tx=0x7f81d0017000 comp rx=0 tx=0).stop 2023-10-06T05:29:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 msgr2=0x7f81dc1695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 0x7f81dc1695a0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00e930 tx=0x7f81cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 shutdown_connections 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f81bc067000 0x7f81bc0694b0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc140d20 0x7f81dc13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81dc069f00 0x7f81dc1695a0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 --2- 172.21.15.33:0/1158450370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0c9210 0x7f81dc0cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 >> 172.21.15.33:0/1158450370 conn(0x7f81dc06c560 msgr2=0x7f81dc0c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.883+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 shutdown_connections 2023-10-06T05:29:30.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f81db59e700 1 -- 172.21.15.33:0/1158450370 wait complete. 2023-10-06T05:29:30.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5314067080 msgr2=0x7f5314069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5314067080 0x7f5314069530 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5328157a80 tx=0x7f531c008040 comp rx=0 tx=0).stop 2023-10-06T05:29:30.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 msgr2=0x7f532816d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 0x7f532816d940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5318009290 tx=0x7f5318017000 comp rx=0 tx=0).stop 2023-10-06T05:29:30.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 msgr2=0x7fb3300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb3500b5f70 tx=0x7fb3400091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3500cc770 msgr2=0x7fb350148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3500cc770 0x7fb350148f40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb3500cb5e0 tx=0x7fb338017000 comp rx=0 tx=0).stop 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 shutdown_connections 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3501597d0 0x7fb3500b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3500cc770 0x7fb350148f40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 --2- 172.21.15.33:0/209453883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb350159400 0x7fb35015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 >> 172.21.15.33:0/209453883 conn(0x7fb35006e4b0 msgr2=0x7fb35015b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 shutdown_connections 2023-10-06T05:29:30.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7fb3549fe700 1 -- 172.21.15.33:0/209453883 wait complete. 2023-10-06T05:29:30.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 shutdown_connections 2023-10-06T05:29:30.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5314067080 0x7f5314069530 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53281597d0 0x7f532806c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53280cc770 0x7f532816d940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 --2- 172.21.15.33:0/1695604150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5328159400 0x7f532814c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:30.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.884+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 >> 172.21.15.33:0/1695604150 conn(0x7f532806e4b0 msgr2=0x7f53281531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:30.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.885+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 shutdown_connections 2023-10-06T05:29:30.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:30.885+0000 7f532d79d700 1 -- 172.21.15.33:0/1695604150 wait complete. 2023-10-06T05:29:30.892 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 8045 2023-10-06T05:29:30.892 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 9469 2023-10-06T05:29:30.893 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 9463 2023-10-06T05:29:30.915 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:29:30.916 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:29:30.917 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:29:30.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:29:30.918 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:29:30.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:29:30.922 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:29:30.923 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:30.925 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:29:30.925 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:30.960 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:29:30.960 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:30.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:31.057 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:29:31.058 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:29:31.096 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:29:31.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.150+0000 7f0debfff700 1 Processor -- start 2023-10-06T05:29:31.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.150+0000 7f0debfff700 1 -- start start 2023-10-06T05:29:31.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0debfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dec0c6340 0x7f0dec0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:31.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0debfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0dec0c80d0 0x7f0dec06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0debfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec06c820 con 0x7f0dec159520 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0debfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec06a160 con 0x7f0dec0c80d0 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0debfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec0c8800 con 0x7f0dec0c6340 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0deaffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0dec159520 0x7f0dec0c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46222/0 (socket says 172.21.15.33:46222) 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0deaffd700 1 -- 172.21.15.33:0/2609291757 learned_addr learned my addr 172.21.15.33:0/2609291757 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0dea7fc700 1 --2- 172.21.15.33:0/2609291757 >> v2:172.21.15.167:3300/0 conn(0x7f0dec0c6340 0x7f0dec0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0deb7fe700 1 --2- 172.21.15.33:0/2609291757 >> v2:172.21.15.33:3300/0 conn(0x7f0dec0c80d0 0x7f0dec06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0de9ffb700 1 -- 172.21.15.33:0/2609291757 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965826231 0 0) 0x7f0dec06c820 con 0x7f0dec159520 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0de9ffb700 1 -- 172.21.15.33:0/2609291757 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd4003570 con 0x7f0dec159520 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0dea7fc700 1 -- 172.21.15.33:0/2609291757 >> v2:172.21.15.33:3300/0 conn(0x7f0dec0c80d0 msgr2=0x7f0dec06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0dea7fc700 1 --2- 172.21.15.33:0/2609291757 >> v2:172.21.15.33:3300/0 conn(0x7f0dec0c80d0 0x7f0dec06a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0dea7fc700 1 -- 172.21.15.33:0/2609291757 >> v1:172.21.15.33:6789/0 conn(0x7f0dec159520 legacy=0x7f0dec0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0dea7fc700 1 -- 172.21.15.33:0/2609291757 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0c8ae0 con 0x7f0dec0c6340 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.152+0000 7f0deb7fe700 1 --2- 172.21.15.33:0/2609291757 >> v2:172.21.15.33:3300/0 conn(0x7f0dec0c80d0 0x7f0dec06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.153+0000 7f0dea7fc700 1 --2- 172.21.15.33:0/2609291757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0c6340 0x7f0dec0c7b90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0c6e50 tx=0x7f0de00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291ef300394beb7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.153+0000 7f0de9ffb700 1 -- 172.21.15.33:0/2609291757 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011630 con 0x7f0dec0c6340 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.153+0000 7f0de9ffb700 1 -- 172.21.15.33:0/2609291757 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de00117b0 con 0x7f0dec0c6340 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.153+0000 7f0de9ffb700 1 -- 172.21.15.33:0/2609291757 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011b30 con 0x7f0dec0c6340 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 -- 172.21.15.33:0/2609291757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0c6340 msgr2=0x7f0dec0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 --2- 172.21.15.33:0/2609291757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0c6340 0x7f0dec0c7b90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0c6e50 tx=0x7f0de00091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 -- 172.21.15.33:0/2609291757 shutdown_connections 2023-10-06T05:29:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 --2- 172.21.15.33:0/2609291757 >> v2:172.21.15.33:3300/0 conn(0x7f0dec0c80d0 0x7f0dec06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 --2- 172.21.15.33:0/2609291757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0c6340 0x7f0dec0c7b90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 -- 172.21.15.33:0/2609291757 >> 172.21.15.33:0/2609291757 conn(0x7f0dec0cc930 msgr2=0x7f0dec0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 -- 172.21.15.33:0/2609291757 shutdown_connections 2023-10-06T05:29:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.154+0000 7f0debfff700 1 -- 172.21.15.33:0/2609291757 wait complete. 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.155+0000 7f0debfff700 1 Processor -- start 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 -- start start 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec0c6340 0x7f0dec0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 0x7f0dec14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec159520 0x7f0dec0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec0af030 con 0x7f0dec0c80d0 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec15a680 con 0x7f0dec159520 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.156+0000 7f0debfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec14a970 con 0x7f0dec0c6340 2023-10-06T05:29:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 0x7f0dec14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 0x7f0dec14a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59658/0 (socket says 172.21.15.33:59658) 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 -- 172.21.15.33:0/365834233 learned_addr learned my addr 172.21.15.33:0/365834233 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0deaffd700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec0c6340 0x7f0dec0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0deb7fe700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec159520 0x7f0dec0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 -- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec0c6340 msgr2=0x7f0dec0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec0c6340 0x7f0dec0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 -- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec159520 msgr2=0x7f0dec0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec159520 0x7f0dec0b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 -- 172.21.15.33:0/365834233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de4009a20 con 0x7f0dec0c80d0 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0deb7fe700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec159520 0x7f0dec0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0deaffd700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec0c6340 0x7f0dec0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.157+0000 7f0dea7fc700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 0x7f0dec14a430 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f0de00093f0 tx=0x7f0de0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.158+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011920 con 0x7f0dec0c80d0 2023-10-06T05:29:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.158+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0011a80 con 0x7f0dec0c80d0 2023-10-06T05:29:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.158+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de0009420 con 0x7f0dec0c80d0 2023-10-06T05:29:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.158+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de001a680 con 0x7f0dec0c80d0 2023-10-06T05:29:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.158+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec159d50 con 0x7f0dec0c80d0 2023-10-06T05:29:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.159+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec15a830 con 0x7f0dec0c80d0 2023-10-06T05:29:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.160+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0de001aa30 con 0x7f0dec0c80d0 2023-10-06T05:29:31.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.161+0000 7f0dd2ffd700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0dd8066f30 0x7f0dd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.161+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f0de0086400 con 0x7f0dec0c80d0 2023-10-06T05:29:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.161+0000 7f0deaffd700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0dd8066f30 0x7f0dd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.162+0000 7f0deaffd700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0dd8066f30 0x7f0dd80693e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0058d0 tx=0x7f0ddc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:31.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.165+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dec15a830 con 0x7f0dec0c80d0 2023-10-06T05:29:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.411+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dec15a7f0 con 0x7f0dec0c80d0 2023-10-06T05:29:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.412+0000 7f0dd2ffd700 1 -- 172.21.15.33:0/365834233 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+2339 (secure 0 0 0) 0x7f0de00533c0 con 0x7f0dec0c80d0 2023-10-06T05:29:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:31.415 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":166,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:30.881587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0dd8066f30 msgr2=0x7f0dd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0dd8066f30 0x7f0dd80693e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0058d0 tx=0x7f0ddc009040 comp rx=0 tx=0).stop 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 msgr2=0x7f0dec14a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 0x7f0dec14a430 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f0de00093f0 tx=0x7f0de0017000 comp rx=0 tx=0).stop 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 shutdown_connections 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0dd8066f30 0x7f0dd80693e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec159520 0x7f0dec0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec0c6340 0x7f0dec0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 --2- 172.21.15.33:0/365834233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0dec0c80d0 0x7f0dec14a430 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 >> 172.21.15.33:0/365834233 conn(0x7f0dec0cc930 msgr2=0x7f0dec0cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 shutdown_connections 2023-10-06T05:29:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:31.414+0000 7f0debfff700 1 -- 172.21.15.33:0/365834233 wait complete. 2023-10-06T05:29:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 166 2023-10-06T05:29:31.434 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2023-10-06T05:29:32.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.631+0000 7f307559d700 1 Processor -- start 2023-10-06T05:29:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.631+0000 7f307559d700 1 -- start start 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f307559d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f30701557a0 0x7f307006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f307559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30700682b0 0x7f3070155b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f307559d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307006c770 con 0x7f307015a090 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f307559d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307015a530 con 0x7f30701557a0 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f307559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307006a820 con 0x7f30700682b0 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f306ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f30701557a0 0x7f307006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.632+0000 7f306ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f30701557a0 0x7f307006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59668/0 (socket says 172.21.15.33:59668) 2023-10-06T05:29:32.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f306ffff700 1 -- 172.21.15.33:0/3538935499 learned_addr learned my addr 172.21.15.33:0/3538935499 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:32.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f3074d9c700 1 --2- 172.21.15.33:0/3538935499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30700682b0 0x7f3070155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:32.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f306ffff700 1 -- 172.21.15.33:0/3538935499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30700682b0 msgr2=0x7f3070155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:32.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f306ffff700 1 --2- 172.21.15.33:0/3538935499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30700682b0 0x7f3070155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f306ffff700 1 -- 172.21.15.33:0/3538935499 >> v1:172.21.15.167:6789/0 conn(0x7f307015a090 legacy=0x7f3070067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:32.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f306ffff700 1 -- 172.21.15.33:0/3538935499 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307006ab00 con 0x7f30701557a0 2023-10-06T05:29:32.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.633+0000 7f306ffff700 1 --2- 172.21.15.33:0/3538935499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30701557a0 0x7f307006a160 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f306000ab50 tx=0x7f306000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b1db4530a0be42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:32.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f306effd700 1 -- 172.21.15.33:0/3538935499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000ca60 con 0x7f30701557a0 2023-10-06T05:29:32.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f306effd700 1 -- 172.21.15.33:0/3538935499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306000cbc0 con 0x7f30701557a0 2023-10-06T05:29:32.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f306effd700 1 -- 172.21.15.33:0/3538935499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060018400 con 0x7f30701557a0 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 -- 172.21.15.33:0/3538935499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30701557a0 msgr2=0x7f307006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 --2- 172.21.15.33:0/3538935499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30701557a0 0x7f307006a160 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f306000ab50 tx=0x7f306000a3f0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 -- 172.21.15.33:0/3538935499 shutdown_connections 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 --2- 172.21.15.33:0/3538935499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30701557a0 0x7f307006a160 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 --2- 172.21.15.33:0/3538935499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30700682b0 0x7f3070155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 -- 172.21.15.33:0/3538935499 >> 172.21.15.33:0/3538935499 conn(0x7f30700cc930 msgr2=0x7f30700ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 -- 172.21.15.33:0/3538935499 shutdown_connections 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.634+0000 7f307559d700 1 -- 172.21.15.33:0/3538935499 wait complete. 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.636+0000 7f307559d700 1 Processor -- start 2023-10-06T05:29:32.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.636+0000 7f307559d700 1 -- start start 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f307559d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30700682b0 0x7f30700b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f307559d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30701557a0 0x7f307014e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f307559d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 0x7f3070151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f307559d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700b2860 con 0x7f307015a090 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f307559d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307014ea90 con 0x7f30700682b0 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f307559d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30701678a0 con 0x7f30701557a0 2023-10-06T05:29:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f3074d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 0x7f3070151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f306ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30700682b0 0x7f30700b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f3074d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 0x7f3070151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59676/0 (socket says 172.21.15.33:59676) 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f3074d9c700 1 -- 172.21.15.33:0/1465699956 learned_addr learned my addr 172.21.15.33:0/1465699956 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.637+0000 7f306f7fe700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30701557a0 0x7f307014e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3074d9c700 1 -- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30701557a0 msgr2=0x7f307014e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3074d9c700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30701557a0 0x7f307014e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3074d9c700 1 -- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30700682b0 msgr2=0x7f30700b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3074d9c700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30700682b0 0x7f30700b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3074d9c700 1 -- 172.21.15.33:0/1465699956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306000a690 con 0x7f307015a090 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3074d9c700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 0x7f3070151720 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f3068007f60 tx=0x7f30680098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:32.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800eb90 con 0x7f307015a090 2023-10-06T05:29:32.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306800ecf0 con 0x7f307015a090 2023-10-06T05:29:32.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3070167b80 con 0x7f307015a090 2023-10-06T05:29:32.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068014670 con 0x7f307015a090 2023-10-06T05:29:32.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.638+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3070168140 con 0x7f307015a090 2023-10-06T05:29:32.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.639+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307004c380 con 0x7f307015a090 2023-10-06T05:29:32.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.640+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3068014a60 con 0x7f307015a090 2023-10-06T05:29:32.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.641+0000 7f3057fff700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f305c066f30 0x7f305c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:32.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.641+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f3068084dc0 con 0x7f307015a090 2023-10-06T05:29:32.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.641+0000 7f306ffff700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f305c066f30 0x7f305c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:32.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.642+0000 7f306ffff700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f305c066f30 0x7f305c0693e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f306000ab50 tx=0x7f306001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:32.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.645+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3068055810 con 0x7f307015a090 2023-10-06T05:29:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.891+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f307006ade0 con 0x7f307015a090 2023-10-06T05:29:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.892+0000 7f3057fff700 1 -- 172.21.15.33:0/1465699956 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f3068009a20 con 0x7f307015a090 2023-10-06T05:29:32.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:32.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":168,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:32.885465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:32.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.894+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f305c066f30 msgr2=0x7f305c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.894+0000 7f307559d700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f305c066f30 0x7f305c0693e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f306000ab50 tx=0x7f306001a000 comp rx=0 tx=0).stop 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.894+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 msgr2=0x7f3070151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.894+0000 7f307559d700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 0x7f3070151720 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f3068007f60 tx=0x7f30680098e0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 shutdown_connections 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f305c066f30 0x7f305c0693e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f307015a090 0x7f3070151720 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30700682b0 0x7f30700b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 --2- 172.21.15.33:0/1465699956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30701557a0 0x7f307014e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 >> 172.21.15.33:0/1465699956 conn(0x7f30700cc930 msgr2=0x7f3070155d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:32.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 shutdown_connections 2023-10-06T05:29:32.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:32.895+0000 7f307559d700 1 -- 172.21.15.33:0/1465699956 wait complete. 2023-10-06T05:29:32.901 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 168 2023-10-06T05:29:32.915 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2023-10-06T05:29:33.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:34.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.111+0000 7ff4f499d700 1 Processor -- start 2023-10-06T05:29:34.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.111+0000 7ff4f499d700 1 -- start start 2023-10-06T05:29:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4f499d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4f00c6340 0x7ff4f00c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4f499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f00c80d0 0x7ff4f006a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4f499d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f006c820 con 0x7ff4f0159520 2023-10-06T05:29:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4f499d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f006a160 con 0x7ff4f00c6340 2023-10-06T05:29:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4f499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c8800 con 0x7ff4f00c80d0 2023-10-06T05:29:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4eeffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4f00c6340 0x7ff4f00c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f00c80d0 0x7ff4f006a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.113+0000 7ff4ef7fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff4f0159520 0x7ff4f00c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:53016/0 (socket says 172.21.15.33:53016) 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4ef7fe700 1 -- 172.21.15.33:0/2429788701 learned_addr learned my addr 172.21.15.33:0/2429788701 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4eeffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4f00c6340 0x7ff4f00c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:40040/0 (socket says 172.21.15.33:40040) 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/2429788701 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832809711 0 0) 0x7ff4f006c820 con 0x7ff4f0159520 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/2429788701 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d8003570 con 0x7ff4f0159520 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4eeffd700 1 -- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f00c80d0 msgr2=0x7ff4f006a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f00c80d0 0x7ff4f006a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4eeffd700 1 -- 172.21.15.33:0/2429788701 >> v1:172.21.15.167:6789/0 conn(0x7ff4f0159520 legacy=0x7ff4f00c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4eeffd700 1 -- 172.21.15.33:0/2429788701 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00c8ae0 con 0x7ff4f00c6340 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4effff700 1 --2- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f00c80d0 0x7ff4f006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.114+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f00c7b90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40098e0 tx=0x7ff4e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33bc6b385f8a323 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/2429788701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4011630 con 0x7ff4f00c6340 2023-10-06T05:29:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/2429788701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e4011790 con 0x7ff4f00c6340 2023-10-06T05:29:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4ee7fc700 1 -- 172.21.15.33:0/2429788701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4011b30 con 0x7ff4f00c6340 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 -- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 msgr2=0x7ff4f00c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 --2- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f00c7b90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40098e0 tx=0x7ff4e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 -- 172.21.15.33:0/2429788701 shutdown_connections 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 --2- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f00c80d0 0x7ff4f006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 --2- 172.21.15.33:0/2429788701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f00c7b90 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 -- 172.21.15.33:0/2429788701 >> 172.21.15.33:0/2429788701 conn(0x7ff4f00cc930 msgr2=0x7ff4f00ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 -- 172.21.15.33:0/2429788701 shutdown_connections 2023-10-06T05:29:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.115+0000 7ff4f499d700 1 -- 172.21.15.33:0/2429788701 wait complete. 2023-10-06T05:29:34.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.117+0000 7ff4f499d700 1 Processor -- start 2023-10-06T05:29:34.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.117+0000 7ff4f499d700 1 -- start start 2023-10-06T05:29:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4f499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f016a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4f499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 0x7ff4f0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4f499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f0159520 0x7ff4f0153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4f499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f016a650 con 0x7ff4f00c80d0 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4f499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f016ca80 con 0x7ff4f00c6340 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4f499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f0151a30 con 0x7ff4f0159520 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 0x7ff4f0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 0x7ff4f0153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59678/0 (socket says 172.21.15.33:59678) 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4eeffd700 1 -- 172.21.15.33:0/1861081581 learned_addr learned my addr 172.21.15.33:0/1861081581 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f016a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.118+0000 7ff4effff700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f0159520 0x7ff4f0153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4eeffd700 1 -- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f0159520 msgr2=0x7ff4f0153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f0159520 0x7ff4f0153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4eeffd700 1 -- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 msgr2=0x7ff4f016a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f016a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4eeffd700 1 -- 172.21.15.33:0/1861081581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e8009a20 con 0x7ff4f00c80d0 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4eeffd700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 0x7ff4f0153140 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4013000 tx=0x7ff4e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4011b10 con 0x7ff4f00c80d0 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e40253f0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e4009420 con 0x7ff4f00c80d0 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.119+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40257b0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.120+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f01525e0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.120+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f004c380 con 0x7ff4f00c80d0 2023-10-06T05:29:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.121+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4e4025ba0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.123+0000 7ff4d77fe700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.123+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7ff4e4085f40 con 0x7ff4f00c80d0 2023-10-06T05:29:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.123+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.124+0000 7ff4ef7fe700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0007fd0 tx=0x7ff4e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:34.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.127+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4e4052fc0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.377+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4f00c8dc0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.378+0000 7ff4d77fe700 1 -- 172.21.15.33:0/1861081581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+4619 (secure 0 0 0) 0x7ff4e40565e0 con 0x7ff4f00c80d0 2023-10-06T05:29:34.381 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:34.381 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:33.890660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 msgr2=0x7ff4dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0007fd0 tx=0x7ff4e0009040 comp rx=0 tx=0).stop 2023-10-06T05:29:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 msgr2=0x7ff4f0153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:34.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 0x7ff4f0153140 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4013000 tx=0x7ff4e4017000 comp rx=0 tx=0).stop 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 shutdown_connections 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff4dc066f30 0x7ff4dc0693e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4f0159520 0x7ff4f0153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4f00c6340 0x7ff4f016a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.380+0000 7ff4f499d700 1 --2- 172.21.15.33:0/1861081581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00c80d0 0x7ff4f0153140 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.381+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 >> 172.21.15.33:0/1861081581 conn(0x7ff4f00cc930 msgr2=0x7ff4f0157870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.381+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 shutdown_connections 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:34.381+0000 7ff4f499d700 1 -- 172.21.15.33:0/1861081581 wait complete. 2023-10-06T05:29:34.384 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 169 2023-10-06T05:29:34.401 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:29:34.401 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 3.4404714107513428s 2023-10-06T05:29:34.401 DEBUG:tasks.cephfs.mount:Creating test file 2023-10-06 05:29:34.401446 0 2023-10-06T05:29:34.401 DEBUG:teuthology.orchestra.run.smithi033:> touch '/home/ubuntu/cephtest/mnt.0/2023-10-06 05:29:34.401446 0' 2023-10-06T05:29:34.441 DEBUG:tasks.cephfs.mount:Deleting test file 2023-10-06 05:29:34.401446 0 2023-10-06T05:29:34.442 DEBUG:teuthology.orchestra.run.smithi033:> rm -f '/home/ubuntu/cephtest/mnt.0/2023-10-06 05:29:34.401446 0' 2023-10-06T05:29:34.519 DEBUG:tasks.cephfs.mount:Creating test file 2023-10-06 05:29:34.519205 1 2023-10-06T05:29:34.519 DEBUG:teuthology.orchestra.run.smithi167:> touch '/home/ubuntu/cephtest/mnt.1/2023-10-06 05:29:34.519205 1' 2023-10-06T05:29:34.564 DEBUG:tasks.cephfs.mount:Deleting test file 2023-10-06 05:29:34.519205 1 2023-10-06T05:29:34.564 DEBUG:teuthology.orchestra.run.smithi167:> rm -f '/home/ubuntu/cephtest/mnt.1/2023-10-06 05:29:34.519205 1' 2023-10-06T05:29:34.646 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:29:34.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:34.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:34.807 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:29:34.808 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:29:34.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:29:34.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:29:34.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:29:34.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:34.891 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:29:34.930 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:29:34.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:29:34.930 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:29:35.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:29:35.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:29:35.018 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:29:35.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.242+0000 7f310739d700 1 Processor -- start 2023-10-06T05:29:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.242+0000 7f310739d700 1 -- start start 2023-10-06T05:29:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f310739d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31000c7920 0x7f3100160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f310739d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310006ab60 con 0x7f31000c8b50 2023-10-06T05:29:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f310739d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310006ae40 con 0x7f3100155af0 2023-10-06T05:29:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f310739d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310006b7e0 con 0x7f31000c7920 2023-10-06T05:29:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3105b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f31000c8b50 0x7f31000c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46232/0 (socket says 172.21.15.33:46232) 2023-10-06T05:29:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3105b9a700 1 -- 172.21.15.33:0/3917095364 learned_addr learned my addr 172.21.15.33:0/3917095364 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3106b9c700 1 --2- 172.21.15.33:0/3917095364 >> v2:172.21.15.167:3300/0 conn(0x7f31000c7920 0x7f3100160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389008655 0 0) 0x7f310006ab60 con 0x7f31000c8b50 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30ec004680 con 0x7f31000c8b50 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199241489 0 0) 0x7f310006ae40 con 0x7f3100155af0 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.244+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f310006ab60 con 0x7f3100155af0 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3106b9c700 1 -- 172.21.15.33:0/3917095364 >> v1:172.21.15.167:6789/0 conn(0x7f3100155af0 legacy=0x7f310006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3106b9c700 1 -- 172.21.15.33:0/3917095364 >> v1:172.21.15.33:6789/0 conn(0x7f31000c8b50 legacy=0x7f31000c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3106b9c700 1 -- 172.21.15.33:0/3917095364 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310006b120 con 0x7f31000c7920 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3106b9c700 1 --2- 172.21.15.33:0/3917095364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c7920 0x7f3100160c30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009e60 tx=0x7f30f8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a42300635b5f470 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800e070 con 0x7f31000c7920 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f800bab0 con 0x7f31000c7920 2023-10-06T05:29:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.245+0000 7f3105399700 1 -- 172.21.15.33:0/3917095364 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800be70 con 0x7f31000c7920 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 -- 172.21.15.33:0/3917095364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c7920 msgr2=0x7f3100160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 --2- 172.21.15.33:0/3917095364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c7920 0x7f3100160c30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009e60 tx=0x7f30f8009780 comp rx=0 tx=0).stop 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 -- 172.21.15.33:0/3917095364 shutdown_connections 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 --2- 172.21.15.33:0/3917095364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31000c7920 0x7f3100160c30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 -- 172.21.15.33:0/3917095364 >> 172.21.15.33:0/3917095364 conn(0x7f310006d030 msgr2=0x7f310006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 -- 172.21.15.33:0/3917095364 shutdown_connections 2023-10-06T05:29:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.246+0000 7f310739d700 1 -- 172.21.15.33:0/3917095364 wait complete. 2023-10-06T05:29:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.247+0000 7f310739d700 1 Processor -- start 2023-10-06T05:29:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.247+0000 7f310739d700 1 -- start start 2023-10-06T05:29:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f310739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31000c7920 0x7f31000ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f310739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 0x7f31000ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f310739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3100155af0 0x7f31000c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f310739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31001694c0 con 0x7f31000c8b50 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f310739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310016d7f0 con 0x7f3100155af0 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f310739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310016cbe0 con 0x7f31000c7920 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.248+0000 7f3105b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 0x7f31000ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 0x7f31000ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59682/0 (socket says 172.21.15.33:59682) 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 -- 172.21.15.33:0/1157597596 learned_addr learned my addr 172.21.15.33:0/1157597596 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3106b9c700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3100155af0 0x7f31000c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f310639b700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31000c7920 0x7f31000ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 -- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31000c7920 msgr2=0x7f31000ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31000c7920 0x7f31000ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 -- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3100155af0 msgr2=0x7f31000c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3100155af0 0x7f31000c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 -- 172.21.15.33:0/1157597596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8009a20 con 0x7f31000c8b50 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3106b9c700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3100155af0 0x7f31000c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f3105b9a700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 0x7f31000ae4e0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c7c0 tx=0x7f30f0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0005b30 con 0x7f31000c8b50 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0005c90 con 0x7f31000c8b50 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310016cec0 con 0x7f31000c8b50 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.249+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0007300 con 0x7f31000c8b50 2023-10-06T05:29:35.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.250+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310016d480 con 0x7f31000c8b50 2023-10-06T05:29:35.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.250+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31000ab8a0 con 0x7f31000c8b50 2023-10-06T05:29:35.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.251+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30f0007670 con 0x7f31000c8b50 2023-10-06T05:29:35.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.252+0000 7f30f67fc700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e4066f30 0x7f30e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.252+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f30f00842a0 con 0x7f31000c8b50 2023-10-06T05:29:35.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.253+0000 7f310639b700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e4066f30 0x7f30e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.253+0000 7f310639b700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e4066f30 0x7f30e40693e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f30fc003870 tx=0x7f30fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:35.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.256+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30f00512e0 con 0x7f31000c8b50 2023-10-06T05:29:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.504+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f310004c380 con 0x7f31000c8b50 2023-10-06T05:29:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.504+0000 7f30f67fc700 1 -- 172.21.15.33:0/1157597596 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f30f0054900 con 0x7f31000c8b50 2023-10-06T05:29:35.508 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:35.508 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:33.890660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e4066f30 msgr2=0x7f30e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e4066f30 0x7f30e40693e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f30fc003870 tx=0x7f30fc009040 comp rx=0 tx=0).stop 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 msgr2=0x7f31000ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 0x7f31000ae4e0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c7c0 tx=0x7f30f0007ea0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 shutdown_connections 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f30e4066f30 0x7f30e40693e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3100155af0 0x7f31000c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31000c7920 0x7f31000ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 --2- 172.21.15.33:0/1157597596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31000c8b50 0x7f31000ae4e0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.506+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 >> 172.21.15.33:0/1157597596 conn(0x7f310006d030 msgr2=0x7f31000cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.507+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 shutdown_connections 2023-10-06T05:29:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.507+0000 7f310739d700 1 -- 172.21.15.33:0/1157597596 wait complete. 2023-10-06T05:29:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 169 2023-10-06T05:29:35.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.712+0000 7f74c3fff700 1 Processor -- start 2023-10-06T05:29:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.712+0000 7f74c3fff700 1 -- start start 2023-10-06T05:29:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74c40ccfc0 0x7f74c40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f74c40cf3b0 0x7f74c41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40cc750 0x7f74c41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c3fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c40cbd80 con 0x7f74c40ccfc0 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c40af5f0 con 0x7f74c40cf3b0 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40af8d0 con 0x7f74c40cc750 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74c40ccfc0 0x7f74c40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74c40ccfc0 0x7f74c40cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59698/0 (socket says 172.21.15.33:59698) 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.713+0000 7f74c2ffd700 1 -- 172.21.15.33:0/2473206257 learned_addr learned my addr 172.21.15.33:0/2473206257 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c37fe700 1 --2- 172.21.15.33:0/2473206257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40cc750 0x7f74c41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c27fc700 1 --2- 172.21.15.33:0/2473206257 >> v2:172.21.15.167:3300/0 conn(0x7f74c40cf3b0 0x7f74c41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c2ffd700 1 -- 172.21.15.33:0/2473206257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40cc750 msgr2=0x7f74c41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c2ffd700 1 --2- 172.21.15.33:0/2473206257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40cc750 0x7f74c41599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c2ffd700 1 -- 172.21.15.33:0/2473206257 >> v2:172.21.15.167:3300/0 conn(0x7f74c40cf3b0 msgr2=0x7f74c41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:35.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c2ffd700 1 --2- 172.21.15.33:0/2473206257 >> v2:172.21.15.167:3300/0 conn(0x7f74c40cf3b0 0x7f74c41594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c2ffd700 1 -- 172.21.15.33:0/2473206257 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c40afbb0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c2ffd700 1 --2- 172.21.15.33:0/2473206257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c40cc210 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bce48d918f2369 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:35.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.714+0000 7f74c1ffb700 1 -- 172.21.15.33:0/2473206257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400b640 con 0x7f74c40ccfc0 2023-10-06T05:29:35.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c1ffb700 1 -- 172.21.15.33:0/2473206257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400b7a0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c1ffb700 1 -- 172.21.15.33:0/2473206257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400bb40 con 0x7f74c40ccfc0 2023-10-06T05:29:35.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 -- 172.21.15.33:0/2473206257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 msgr2=0x7f74c40cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2473206257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c40cc210 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4009250 comp rx=0 tx=0).stop 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 -- 172.21.15.33:0/2473206257 shutdown_connections 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2473206257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40cc750 0x7f74c41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2473206257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c40cc210 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2473206257 >> v2:172.21.15.167:3300/0 conn(0x7f74c40cf3b0 0x7f74c41594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 -- 172.21.15.33:0/2473206257 >> 172.21.15.33:0/2473206257 conn(0x7f74c406cff0 msgr2=0x7f74c406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 -- 172.21.15.33:0/2473206257 shutdown_connections 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.715+0000 7f74c3fff700 1 -- 172.21.15.33:0/2473206257 wait complete. 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.717+0000 7f74c3fff700 1 Processor -- start 2023-10-06T05:29:35.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.717+0000 7f74c3fff700 1 -- start start 2023-10-06T05:29:35.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74c40cc750 0x7f74c40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c4169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40c85e0 0x7f74c416a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40ca400 con 0x7f74c40ccfc0 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b1f30 con 0x7f74c40cc750 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40c9360 con 0x7f74c40c85e0 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c4169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c4169c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59714/0 (socket says 172.21.15.33:59714) 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 -- 172.21.15.33:0/2403472145 learned_addr learned my addr 172.21.15.33:0/2403472145 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c2ffd700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74c40cc750 0x7f74c40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c37fe700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40c85e0 0x7f74c416a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 -- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40c85e0 msgr2=0x7f74c416a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40c85e0 0x7f74c416a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 -- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74c40cc750 msgr2=0x7f74c40c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74c40cc750 0x7f74c40c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.718+0000 7f74c27fc700 1 -- 172.21.15.33:0/2403472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094f0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.719+0000 7f74c27fc700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c4169c90 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f74b800e8d0 tx=0x7f74b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.719+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8013680 con 0x7f74c40ccfc0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.719+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b80137e0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.719+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8013ba0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.719+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c40c9640 con 0x7f74c40ccfc0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.719+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c40c9c00 con 0x7f74c40ccfc0 2023-10-06T05:29:35.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.720+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c404c380 con 0x7f74c40ccfc0 2023-10-06T05:29:35.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.721+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74b80123d0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.722+0000 7f74aaffd700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b0066f30 0x7f74b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:35.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.722+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f74b80849a0 con 0x7f74c40ccfc0 2023-10-06T05:29:35.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.722+0000 7f74c2ffd700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b0066f30 0x7f74b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:35.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.723+0000 7f74c2ffd700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b0066f30 0x7f74b00693e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:35.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.726+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74b8051960 con 0x7f74c40ccfc0 2023-10-06T05:29:35.890 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:29:35.888+0000 7f50633f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:29:35.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.931+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c4157e80 con 0x7f74c40ccfc0 2023-10-06T05:29:35.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.932+0000 7f74aaffd700 1 -- 172.21.15.33:0/2403472145 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+17139 (secure 0 0 0) 0x7f74b8017090 con 0x7f74c40ccfc0 2023-10-06T05:29:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:35.935 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":214,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:30.873187+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:17.571482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:18.582693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/1485763855":"2023-10-07T05:29:30.873167+0000","172.21.15.167:6835/1485763855":"2023-10-07T05:29:30.873167+0000","172.21.15.33:6834/963232630":"2023-10-07T05:29:30.377199+0000","172.21.15.33:6832/963232630":"2023-10-07T05:29:30.377199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b0066f30 msgr2=0x7f74b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b0066f30 0x7f74b00693e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4017000 comp rx=0 tx=0).stop 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 msgr2=0x7f74c4169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c4169c90 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f74b800e8d0 tx=0x7f74b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 shutdown_connections 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b0066f30 0x7f74b00693e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74c40c85e0 0x7f74c416a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74c40cc750 0x7f74c40c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 --2- 172.21.15.33:0/2403472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c40ccfc0 0x7f74c4169c90 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:35.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 >> 172.21.15.33:0/2403472145 conn(0x7f74c406cff0 msgr2=0x7f74c40c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:35.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 shutdown_connections 2023-10-06T05:29:35.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:35.935+0000 7f74c3fff700 1 -- 172.21.15.33:0/2403472145 wait complete. 2023-10-06T05:29:35.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:29:17.571482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:29:18.582693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:29:35.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:29:36.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.147+0000 7f246ef9d700 1 Processor -- start 2023-10-06T05:29:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.147+0000 7f246ef9d700 1 -- start start 2023-10-06T05:29:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.148+0000 7f246ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680cd5e0 0x7f24680af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.148+0000 7f246ef9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24680ce890 con 0x7f246815a650 2023-10-06T05:29:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.148+0000 7f246ef9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246815aad0 con 0x7f24680af540 2023-10-06T05:29:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.148+0000 7f246ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246815d890 con 0x7f24680cd5e0 2023-10-06T05:29:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246e79c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f246815a650 0x7f24680ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46246/0 (socket says 172.21.15.33:46246) 2023-10-06T05:29:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246e79c700 1 -- 172.21.15.33:0/268373169 learned_addr learned my addr 172.21.15.33:0/268373169 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246d79a700 1 --2- 172.21.15.33:0/268373169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680cd5e0 0x7f24680af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569468567 0 0) 0x7f24680ce890 con 0x7f246815a650 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2454003570 con 0x7f246815a650 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339716695 0 0) 0x7f246815aad0 con 0x7f24680af540 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24680ce890 con 0x7f24680af540 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246d79a700 1 -- 172.21.15.33:0/268373169 >> v1:172.21.15.167:6789/0 conn(0x7f24680af540 legacy=0x7f24680cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.149+0000 7f246d79a700 1 -- 172.21.15.33:0/268373169 >> v1:172.21.15.33:6789/0 conn(0x7f246815a650 legacy=0x7f24680ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.150+0000 7f246d79a700 1 -- 172.21.15.33:0/268373169 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246815db70 con 0x7f24680cd5e0 2023-10-06T05:29:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.150+0000 7f246d79a700 1 --2- 172.21.15.33:0/268373169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680cd5e0 0x7f24680af910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2458009910 tx=0x7f24580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746a6dbb9cec6a5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.150+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458011630 con 0x7f24680cd5e0 2023-10-06T05:29:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.150+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2458011790 con 0x7f24680cd5e0 2023-10-06T05:29:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.150+0000 7f246cf99700 1 -- 172.21.15.33:0/268373169 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458011b30 con 0x7f24680cd5e0 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 -- 172.21.15.33:0/268373169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680cd5e0 msgr2=0x7f24680af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 --2- 172.21.15.33:0/268373169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680cd5e0 0x7f24680af910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2458009910 tx=0x7f24580091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 -- 172.21.15.33:0/268373169 shutdown_connections 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 --2- 172.21.15.33:0/268373169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680cd5e0 0x7f24680af910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 -- 172.21.15.33:0/268373169 >> 172.21.15.33:0/268373169 conn(0x7f246806d9b0 msgr2=0x7f246806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 -- 172.21.15.33:0/268373169 shutdown_connections 2023-10-06T05:29:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.151+0000 7f246ef9d700 1 -- 172.21.15.33:0/268373169 wait complete. 2023-10-06T05:29:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.153+0000 7f246ef9d700 1 Processor -- start 2023-10-06T05:29:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.153+0000 7f246ef9d700 1 -- start start 2023-10-06T05:29:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246ef9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680af540 0x7f246816a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246ef9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 0x7f24680b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246ef9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246815a650 0x7f24681575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246ef9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680b5100 con 0x7f24680cd5e0 2023-10-06T05:29:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246ef9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680b5380 con 0x7f246815a650 2023-10-06T05:29:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246ef9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680b5660 con 0x7f24680af540 2023-10-06T05:29:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246d79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 0x7f24680b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246df9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680af540 0x7f246816a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246e79c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246815a650 0x7f24681575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246d79a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 0x7f24680b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59722/0 (socket says 172.21.15.33:59722) 2023-10-06T05:29:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.154+0000 7f246d79a700 1 -- 172.21.15.33:0/2737384276 learned_addr learned my addr 172.21.15.33:0/2737384276 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246df9b700 1 -- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246815a650 msgr2=0x7f24681575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246df9b700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246815a650 0x7f24681575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246df9b700 1 -- 172.21.15.33:0/2737384276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 msgr2=0x7f24680b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246df9b700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 0x7f24680b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246df9b700 1 -- 172.21.15.33:0/2737384276 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2458009420 con 0x7f24680af540 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246e79c700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246815a650 0x7f24681575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246d79a700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 0x7f24680b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246df9b700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680af540 0x7f246816a620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f246400c4b0 tx=0x7f2464007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400ce60 con 0x7f24680af540 2023-10-06T05:29:36.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2464011590 con 0x7f24680af540 2023-10-06T05:29:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24680b5940 con 0x7f24680af540 2023-10-06T05:29:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464011950 con 0x7f24680af540 2023-10-06T05:29:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.155+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2468157b00 con 0x7f24680af540 2023-10-06T05:29:36.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.156+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246804c380 con 0x7f24680af540 2023-10-06T05:29:36.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.158+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2464011d00 con 0x7f24680af540 2023-10-06T05:29:36.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.158+0000 7f245dffb700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f244c066f30 0x7f244c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.158+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f2464084810 con 0x7f24680af540 2023-10-06T05:29:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.159+0000 7f246d79a700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f244c066f30 0x7f244c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.159+0000 7f246d79a700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f244c066f30 0x7f244c0693e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f24680c55e0 tx=0x7f2458017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.163+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24640517d0 con 0x7f24680af540 2023-10-06T05:29:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.363+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24680b57a0 con 0x7f24680af540 2023-10-06T05:29:36.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.364+0000 7f245dffb700 1 -- 172.21.15.33:0/2737384276 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v214) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24680b57a0 con 0x7f24680af540 2023-10-06T05:29:36.366 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:29:36.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f244c066f30 msgr2=0x7f244c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f244c066f30 0x7f244c0693e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f24680c55e0 tx=0x7f2458017000 comp rx=0 tx=0).stop 2023-10-06T05:29:36.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680af540 msgr2=0x7f246816a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680af540 0x7f246816a620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f246400c4b0 tx=0x7f2464007d50 comp rx=0 tx=0).stop 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 shutdown_connections 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f244c066f30 0x7f244c0693e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f246815a650 0x7f24681575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24680af540 0x7f246816a620 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 --2- 172.21.15.33:0/2737384276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24680cd5e0 0x7f24680b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 >> 172.21.15.33:0/2737384276 conn(0x7f246806d9b0 msgr2=0x7f24681590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 shutdown_connections 2023-10-06T05:29:36.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.366+0000 7f246ef9d700 1 -- 172.21.15.33:0/2737384276 wait complete. 2023-10-06T05:29:36.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:29:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.574+0000 7ff78f59e700 1 Processor -- start 2023-10-06T05:29:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.574+0000 7ff78f59e700 1 -- start start 2023-10-06T05:29:36.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7900b0900 0x7ff7900af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7900ad430 con 0x7ff790156370 2023-10-06T05:29:36.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7900afe20 con 0x7ff7900cc7e0 2023-10-06T05:29:36.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79006bc70 con 0x7ff7900b0900 2023-10-06T05:29:36.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff790156370 0x7ff7900c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46254/0 (socket says 172.21.15.33:46254) 2023-10-06T05:29:36.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78e59c700 1 -- 172.21.15.33:0/1551455094 learned_addr learned my addr 172.21.15.33:0/1551455094 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:36.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78ed9d700 1 --2- 172.21.15.33:0/1551455094 >> v2:172.21.15.167:3300/0 conn(0x7ff7900b0900 0x7ff7900af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878724316 0 0) 0x7ff7900ad430 con 0x7ff790156370 2023-10-06T05:29:36.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff778003570 con 0x7ff790156370 2023-10-06T05:29:36.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427728057 0 0) 0x7ff7900afe20 con 0x7ff7900cc7e0 2023-10-06T05:29:36.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7900ad430 con 0x7ff7900cc7e0 2023-10-06T05:29:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1551455094 >> v1:172.21.15.167:6789/0 conn(0x7ff7900cc7e0 legacy=0x7ff7900b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1551455094 >> v1:172.21.15.33:6789/0 conn(0x7ff790156370 legacy=0x7ff7900c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.575+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1551455094 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900cccc0 con 0x7ff7900b0900 2023-10-06T05:29:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.576+0000 7ff78ed9d700 1 --2- 172.21.15.33:0/1551455094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900af8e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff788009e60 tx=0x7ff788009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c47f61edfbf272d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.576+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800e070 con 0x7ff7900b0900 2023-10-06T05:29:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.576+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800bab0 con 0x7ff7900b0900 2023-10-06T05:29:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.576+0000 7ff78d59a700 1 -- 172.21.15.33:0/1551455094 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800be70 con 0x7ff7900b0900 2023-10-06T05:29:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 -- 172.21.15.33:0/1551455094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 msgr2=0x7ff7900af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1551455094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900af8e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff788009e60 tx=0x7ff788009780 comp rx=0 tx=0).stop 2023-10-06T05:29:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 -- 172.21.15.33:0/1551455094 shutdown_connections 2023-10-06T05:29:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1551455094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900af8e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 -- 172.21.15.33:0/1551455094 >> 172.21.15.33:0/1551455094 conn(0x7ff79006d030 msgr2=0x7ff7900cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 -- 172.21.15.33:0/1551455094 shutdown_connections 2023-10-06T05:29:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.577+0000 7ff78f59e700 1 -- 172.21.15.33:0/1551455094 wait complete. 2023-10-06T05:29:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.578+0000 7ff78f59e700 1 Processor -- start 2023-10-06T05:29:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.578+0000 7ff78f59e700 1 -- start start 2023-10-06T05:29:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.579+0000 7ff78f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.579+0000 7ff78f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7900cc7e0 0x7ff790157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.579+0000 7ff78f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 0x7ff7900c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.579+0000 7ff78f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790156b80 con 0x7ff790156370 2023-10-06T05:29:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.579+0000 7ff78f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901579a0 con 0x7ff7900b0900 2023-10-06T05:29:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.579+0000 7ff78f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff790149750 con 0x7ff7900cc7e0 2023-10-06T05:29:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 0x7ff7900c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 0x7ff7900c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59738/0 (socket says 172.21.15.33:59738) 2023-10-06T05:29:36.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1720382658 learned_addr learned my addr 172.21.15.33:0/1720382658 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78e59c700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78dd9b700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7900cc7e0 0x7ff790157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7900cc7e0 msgr2=0x7ff790157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7900cc7e0 0x7ff790157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 msgr2=0x7ff7900c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 -- 172.21.15.33:0/1720382658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788009a20 con 0x7ff790156370 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78dd9b700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7900cc7e0 0x7ff790157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.580+0000 7ff78ed9d700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 0x7ff7900c98d0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7ff788013000 tx=0x7ff788019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.581+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788012e60 con 0x7ff790156370 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.581+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788026480 con 0x7ff790156370 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.581+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff790156d90 con 0x7ff790156370 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.581+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788026840 con 0x7ff790156370 2023-10-06T05:29:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.581+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff790157350 con 0x7ff790156370 2023-10-06T05:29:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.582+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790157b10 con 0x7ff790156370 2023-10-06T05:29:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.582+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff78801c400 con 0x7ff790156370 2023-10-06T05:29:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.584+0000 7ff77e7fc700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff770066f30 0x7ff7700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.584+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7ff788086620 con 0x7ff790156370 2023-10-06T05:29:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.584+0000 7ff78e59c700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff770066f30 0x7ff7700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.585+0000 7ff78e59c700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff770066f30 0x7ff7700693e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff780007fd0 tx=0x7ff780009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.589+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7880535e0 con 0x7ff790156370 2023-10-06T05:29:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.792+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff79004c380 con 0x7ff790156370 2023-10-06T05:29:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.793+0000 7ff77e7fc700 1 -- 172.21.15.33:0/1720382658 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v214) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff788056c00 con 0x7ff790156370 2023-10-06T05:29:36.796 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:29:36.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff770066f30 msgr2=0x7ff7700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff770066f30 0x7ff7700693e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff780007fd0 tx=0x7ff780009040 comp rx=0 tx=0).stop 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 msgr2=0x7ff7900c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 0x7ff7900c98d0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7ff788013000 tx=0x7ff788019000 comp rx=0 tx=0).stop 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 shutdown_connections 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff770066f30 0x7ff7700693e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff790156370 0x7ff7900c98d0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7900b0900 0x7ff7900c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 --2- 172.21.15.33:0/1720382658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7900cc7e0 0x7ff790157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 >> 172.21.15.33:0/1720382658 conn(0x7ff79006d030 msgr2=0x7ff7900abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 shutdown_connections 2023-10-06T05:29:36.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:36.795+0000 7ff78f59e700 1 -- 172.21.15.33:0/1720382658 wait complete. 2023-10-06T05:29:36.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:29:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.006+0000 7f3c5dfff700 1 Processor -- start 2023-10-06T05:29:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.006+0000 7f3c5dfff700 1 -- start start 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c5dfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3c5806e8f0 0x7f3c58156140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c5dfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5806abb0 con 0x7f3c58155d70 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c5dfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5806a5a0 con 0x7f3c580c1000 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c5dfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c58156d90 con 0x7f3c5806e8f0 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c577fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c58155d70 0x7f3c5806e1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46264/0 (socket says 172.21.15.33:46264) 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c577fe700 1 -- 172.21.15.33:0/4175179702 learned_addr learned my addr 172.21.15.33:0/4175179702 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c56ffd700 1 --2- 172.21.15.33:0/4175179702 >> v2:172.21.15.167:3300/0 conn(0x7f3c5806e8f0 0x7f3c58156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783566627 0 0) 0x7f3c5806abb0 con 0x7f3c58155d70 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.008+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c40003570 con 0x7f3c58155d70 2023-10-06T05:29:37.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711824048 0 0) 0x7f3c40003570 con 0x7f3c58155d70 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 >> v2:172.21.15.167:3300/0 conn(0x7f3c5806e8f0 msgr2=0x7f3c58156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 --2- 172.21.15.33:0/4175179702 >> v2:172.21.15.167:3300/0 conn(0x7f3c5806e8f0 0x7f3c58156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 >> v1:172.21.15.167:6789/0 conn(0x7f3c580c1000 legacy=0x7f3c58156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58156fb0 con 0x7f3c58155d70 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3c48001470 con 0x7f3c58155d70 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 >> v1:172.21.15.33:6789/0 conn(0x7f3c58155d70 legacy=0x7f3c5806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c40003a60 0x7f3c40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40004810 0x7f3c40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c400051c0 0x7f3c40005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58156d90 con 0x7f3c40003a60 2023-10-06T05:29:37.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400060a0 con 0x7f3c40004810 2023-10-06T05:29:37.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40006260 con 0x7f3c400051c0 2023-10-06T05:29:37.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c577fe700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c40003a60 0x7f3c40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c56ffd700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40004810 0x7f3c40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.009+0000 7f3c57fff700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c400051c0 0x7f3c40005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c577fe700 1 -- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c400051c0 msgr2=0x7f3c40005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c577fe700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c400051c0 0x7f3c40005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c577fe700 1 -- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40004810 msgr2=0x7f3c40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c577fe700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40004810 0x7f3c40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c577fe700 1 -- 172.21.15.33:0/4175179702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c40003570 con 0x7f3c40003a60 2023-10-06T05:29:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c57fff700 1 -- 172.21.15.33:0/4175179702 reap_dead start 2023-10-06T05:29:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c577fe700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c40003a60 0x7f3c40003e70 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800c3c0 tx=0x7f3c48001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824239f08acd222d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800cc90 con 0x7f3c40003a60 2023-10-06T05:29:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4800cdf0 con 0x7f3c40003a60 2023-10-06T05:29:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.010+0000 7f3c567fc700 1 -- 172.21.15.33:0/4175179702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48005b50 con 0x7f3c40003a60 2023-10-06T05:29:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 -- 172.21.15.33:0/4175179702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c40003a60 msgr2=0x7f3c40003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c40003a60 0x7f3c40003e70 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800c3c0 tx=0x7f3c48001e50 comp rx=0 tx=0).stop 2023-10-06T05:29:37.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 -- 172.21.15.33:0/4175179702 shutdown_connections 2023-10-06T05:29:37.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/4175179702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c40003a60 0x7f3c40003e70 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 -- 172.21.15.33:0/4175179702 >> 172.21.15.33:0/4175179702 conn(0x7f3c5806d030 msgr2=0x7f3c580cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:37.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 -- 172.21.15.33:0/4175179702 shutdown_connections 2023-10-06T05:29:37.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.011+0000 7f3c5dfff700 1 -- 172.21.15.33:0/4175179702 wait complete. 2023-10-06T05:29:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.012+0000 7f3c5dfff700 1 Processor -- start 2023-10-06T05:29:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 -- start start 2023-10-06T05:29:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c40003a60 0x7f3c58166300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58152e40 0x7f3c58168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 0x7f3c580adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58166840 con 0x7f3c58169380 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58165e70 con 0x7f3c58152e40 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.013+0000 7f3c5dfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58169900 con 0x7f3c40003a60 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 0x7f3c580adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 0x7f3c580adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59756/0 (socket says 172.21.15.33:59756) 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 -- 172.21.15.33:0/3570482683 learned_addr learned my addr 172.21.15.33:0/3570482683 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c577fe700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c40003a60 0x7f3c58166300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c56ffd700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58152e40 0x7f3c58168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 -- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c40003a60 msgr2=0x7f3c58166300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c40003a60 0x7f3c58166300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 -- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58152e40 msgr2=0x7f3c58168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58152e40 0x7f3c58168e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 -- 172.21.15.33:0/3570482683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58169be0 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c56ffd700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58152e40 0x7f3c58168e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c577fe700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c40003a60 0x7f3c58166300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.014+0000 7f3c57fff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 0x7f3c580adda0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f3c580c1420 tx=0x7f3c50001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.015+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50006db0 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.015+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5000c5d0 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.015+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5000c990 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.015+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c58165b00 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.015+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580b06f0 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.016+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c58166000 con 0x7f3c58169380 2023-10-06T05:29:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.017+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c5000cd00 con 0x7f3c58169380 2023-10-06T05:29:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.017+0000 7f3c3f7fe700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.018+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f3c50084130 con 0x7f3c58169380 2023-10-06T05:29:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.018+0000 7f3c577fe700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.019+0000 7f3c577fe700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c44066f30 0x7f3c440693e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3c48007cd0 tx=0x7f3c48015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.022+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c50051170 con 0x7f3c58169380 2023-10-06T05:29:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.221+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c5804c380 con 0x7f3c58169380 2023-10-06T05:29:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.222+0000 7f3c3f7fe700 1 -- 172.21.15.33:0/3570482683 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v214) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c50054790 con 0x7f3c58169380 2023-10-06T05:29:37.223 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c44066f30 msgr2=0x7f3c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c44066f30 0x7f3c440693e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3c48007cd0 tx=0x7f3c48015000 comp rx=0 tx=0).stop 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 msgr2=0x7f3c580adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 0x7f3c580adda0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f3c580c1420 tx=0x7f3c50001640 comp rx=0 tx=0).stop 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 shutdown_connections 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58169380 0x7f3c580adda0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c40003a60 0x7f3c58166300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 --2- 172.21.15.33:0/3570482683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c58152e40 0x7f3c58168e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 >> 172.21.15.33:0/3570482683 conn(0x7f3c5806d030 msgr2=0x7f3c580c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 shutdown_connections 2023-10-06T05:29:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.224+0000 7f3c5dfff700 1 -- 172.21.15.33:0/3570482683 wait complete. 2023-10-06T05:29:37.243 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:29:37.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.432+0000 7fb91ffff700 1 Processor -- start 2023-10-06T05:29:37.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.432+0000 7fb91ffff700 1 -- start start 2023-10-06T05:29:37.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006c7b0 0x7fb920067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9200aff90 0x7fb9200a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92015d3a0 con 0x7fb92006cb80 2023-10-06T05:29:37.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9200b0470 con 0x7fb9200aff90 2023-10-06T05:29:37.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200b0750 con 0x7fb92006c7b0 2023-10-06T05:29:37.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9200aff90 0x7fb9200a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.433+0000 7fb91f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb92006cb80 0x7fb92015d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46266/0 (socket says 172.21.15.33:46266) 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91f7fe700 1 -- 172.21.15.33:0/2151116246 learned_addr learned my addr 172.21.15.33:0/2151116246 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91effd700 1 --2- 172.21.15.33:0/2151116246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006c7b0 0x7fb920067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91dffb700 1 -- 172.21.15.33:0/2151116246 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313348696 0 0) 0x7fb92015d3a0 con 0x7fb92006cb80 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91dffb700 1 -- 172.21.15.33:0/2151116246 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb908003570 con 0x7fb92006cb80 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91e7fc700 1 -- 172.21.15.33:0/2151116246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006c7b0 msgr2=0x7fb920067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91e7fc700 1 --2- 172.21.15.33:0/2151116246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006c7b0 0x7fb920067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91e7fc700 1 -- 172.21.15.33:0/2151116246 >> v1:172.21.15.33:6789/0 conn(0x7fb92006cb80 legacy=0x7fb92015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91e7fc700 1 -- 172.21.15.33:0/2151116246 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200b0a30 con 0x7fb9200aff90 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91e7fc700 1 --2- 172.21.15.33:0/2151116246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200a9890 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb9140098e0 tx=0x7fb9140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb71fec684570cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.434+0000 7fb91dffb700 1 -- 172.21.15.33:0/2151116246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914011630 con 0x7fb9200aff90 2023-10-06T05:29:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91dffb700 1 -- 172.21.15.33:0/2151116246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb914011790 con 0x7fb9200aff90 2023-10-06T05:29:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91dffb700 1 -- 172.21.15.33:0/2151116246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914011b30 con 0x7fb9200aff90 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 -- 172.21.15.33:0/2151116246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 msgr2=0x7fb9200a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 --2- 172.21.15.33:0/2151116246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200a9890 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb9140098e0 tx=0x7fb9140091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 -- 172.21.15.33:0/2151116246 shutdown_connections 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 --2- 172.21.15.33:0/2151116246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006c7b0 0x7fb920067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 --2- 172.21.15.33:0/2151116246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200a9890 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 -- 172.21.15.33:0/2151116246 >> 172.21.15.33:0/2151116246 conn(0x7fb92006d670 msgr2=0x7fb92006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 -- 172.21.15.33:0/2151116246 shutdown_connections 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.435+0000 7fb91ffff700 1 -- 172.21.15.33:0/2151116246 wait complete. 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.437+0000 7fb91ffff700 1 Processor -- start 2023-10-06T05:29:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.437+0000 7fb91ffff700 1 -- start start 2023-10-06T05:29:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92006c7b0 0x7fb92013f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006cb80 0x7fb9200acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920142f00 con 0x7fb9200aff90 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9201422f0 con 0x7fb92006c7b0 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9201425d0 con 0x7fb92006cb80 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59774/0 (socket says 172.21.15.33:59774) 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 -- 172.21.15.33:0/3582693644 learned_addr learned my addr 172.21.15.33:0/3582693644 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91effd700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92006c7b0 0x7fb92013f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91e7fc700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006cb80 0x7fb9200acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 -- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006cb80 msgr2=0x7fb9200acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006cb80 0x7fb9200acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 -- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92006c7b0 msgr2=0x7fb92013f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92006c7b0 0x7fb92013f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.438+0000 7fb91f7fe700 1 -- 172.21.15.33:0/3582693644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914009420 con 0x7fb9200aff90 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb91e7fc700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006cb80 0x7fb9200acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb91effd700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92006c7b0 0x7fb92013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb91f7fe700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200ad080 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fb918001460 tx=0x7fb918001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800ce80 con 0x7fb9200aff90 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91800ab70 con 0x7fb9200aff90 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180070f0 con 0x7fb9200aff90 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9201428b0 con 0x7fb9200aff90 2023-10-06T05:29:37.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.439+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92013fcc0 con 0x7fb9200aff90 2023-10-06T05:29:37.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.440+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92004c380 con 0x7fb9200aff90 2023-10-06T05:29:37.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.441+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb918007520 con 0x7fb9200aff90 2023-10-06T05:29:37.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.442+0000 7fb906ffd700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb90c066f30 0x7fb90c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.442+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fb918084cd0 con 0x7fb9200aff90 2023-10-06T05:29:37.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.443+0000 7fb91effd700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb90c066f30 0x7fb90c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.444+0000 7fb91effd700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb90c066f30 0x7fb90c0693e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb920154ef0 tx=0x7fb910008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.446+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb918051d10 con 0x7fb9200aff90 2023-10-06T05:29:37.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.649+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb920143040 con 0x7fb9200aff90 2023-10-06T05:29:37.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.649+0000 7fb906ffd700 1 -- 172.21.15.33:0/3582693644 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v214) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb918055330 con 0x7fb9200aff90 2023-10-06T05:29:37.651 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:29:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.651+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb90c066f30 msgr2=0x7fb90c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.651+0000 7fb91ffff700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb90c066f30 0x7fb90c0693e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb920154ef0 tx=0x7fb910008040 comp rx=0 tx=0).stop 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.651+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 msgr2=0x7fb9200ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.651+0000 7fb91ffff700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200ad080 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fb918001460 tx=0x7fb918001ae0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 shutdown_connections 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb90c066f30 0x7fb90c0693e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200aff90 0x7fb9200ad080 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb92006c7b0 0x7fb92013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 --2- 172.21.15.33:0/3582693644 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb92006cb80 0x7fb9200acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 >> 172.21.15.33:0/3582693644 conn(0x7fb92006d670 msgr2=0x7fb9200ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 shutdown_connections 2023-10-06T05:29:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.652+0000 7fb91ffff700 1 -- 172.21.15.33:0/3582693644 wait complete. 2023-10-06T05:29:37.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.857+0000 7f417279d700 1 Processor -- start 2023-10-06T05:29:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.857+0000 7f417279d700 1 -- start start 2023-10-06T05:29:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f416c0b0e10 0x7f416c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f416c0b2e30 0x7f416c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c15e2d0 con 0x7f416c0b0e10 2023-10-06T05:29:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c15d6c0 con 0x7f416c0b2e30 2023-10-06T05:29:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c15d9a0 con 0x7f416c0b0a40 2023-10-06T05:29:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f416c0b0e10 0x7f416c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f416c0b0e10 0x7f416c0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59776/0 (socket says 172.21.15.33:59776) 2023-10-06T05:29:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 -- 172.21.15.33:0/4238183496 learned_addr learned my addr 172.21.15.33:0/4238183496 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4171f9c700 1 --2- 172.21.15.33:0/4238183496 >> v2:172.21.15.167:3300/0 conn(0x7f416c0b2e30 0x7f416c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f417179b700 1 --2- 172.21.15.33:0/4238183496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 -- 172.21.15.33:0/4238183496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 msgr2=0x7f416c06ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 --2- 172.21.15.33:0/4238183496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c06ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 -- 172.21.15.33:0/4238183496 >> v2:172.21.15.167:3300/0 conn(0x7f416c0b2e30 msgr2=0x7f416c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 --2- 172.21.15.33:0/4238183496 >> v2:172.21.15.167:3300/0 conn(0x7f416c0b2e30 0x7f416c15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4170f9a700 1 -- 172.21.15.33:0/4238183496 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c15dc80 con 0x7f416c0b0e10 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.859+0000 7f4171f9c700 1 --2- 172.21.15.33:0/4238183496 >> v2:172.21.15.167:3300/0 conn(0x7f416c0b2e30 0x7f416c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f4170f9a700 1 --2- 172.21.15.33:0/4238183496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b0e10 0x7f416c0b28f0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f416c0b32b0 tx=0x7f415c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fdda5e8c20daa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f4163fff700 1 -- 172.21.15.33:0/4238183496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00b860 con 0x7f416c0b0e10 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f4163fff700 1 -- 172.21.15.33:0/4238183496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c00b9c0 con 0x7f416c0b0e10 2023-10-06T05:29:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f4163fff700 1 -- 172.21.15.33:0/4238183496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00bd60 con 0x7f416c0b0e10 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 -- 172.21.15.33:0/4238183496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b0e10 msgr2=0x7f416c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 --2- 172.21.15.33:0/4238183496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b0e10 0x7f416c0b28f0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f416c0b32b0 tx=0x7f415c009250 comp rx=0 tx=0).stop 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 -- 172.21.15.33:0/4238183496 shutdown_connections 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 --2- 172.21.15.33:0/4238183496 >> v2:172.21.15.167:3300/0 conn(0x7f416c0b2e30 0x7f416c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 --2- 172.21.15.33:0/4238183496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c06ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 --2- 172.21.15.33:0/4238183496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b0e10 0x7f416c0b28f0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.860+0000 7f417279d700 1 -- 172.21.15.33:0/4238183496 >> 172.21.15.33:0/4238183496 conn(0x7f416c06e0f0 msgr2=0x7f416c0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.861+0000 7f417279d700 1 -- 172.21.15.33:0/4238183496 shutdown_connections 2023-10-06T05:29:37.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.861+0000 7f417279d700 1 -- 172.21.15.33:0/4238183496 wait complete. 2023-10-06T05:29:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.862+0000 7f417279d700 1 Processor -- start 2023-10-06T05:29:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.862+0000 7f417279d700 1 -- start start 2023-10-06T05:29:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f417279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c0aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f417279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0b0e10 0x7f416c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f417279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 0x7f416c0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f417279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c15ca70 con 0x7f416c0b2e30 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f417279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c156b60 con 0x7f416c0b0e10 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f417279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c169c80 con 0x7f416c0b0a40 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f4171f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 0x7f416c0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f4171f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 0x7f416c0c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59790/0 (socket says 172.21.15.33:59790) 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.863+0000 7f4171f9c700 1 -- 172.21.15.33:0/4138345220 learned_addr learned my addr 172.21.15.33:0/4138345220 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f417179b700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c0aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4170f9a700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0b0e10 0x7f416c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4171f9c700 1 -- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 msgr2=0x7f416c0aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4171f9c700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c0aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4171f9c700 1 -- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0b0e10 msgr2=0x7f416c156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:37.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4171f9c700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0b0e10 0x7f416c156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4171f9c700 1 -- 172.21.15.33:0/4138345220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0094f0 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4170f9a700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0b0e10 0x7f416c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f4171f9c700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 0x7f416c0c4310 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f416400d100 tx=0x7f41640099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400dda0 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400f040 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164015690 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c15bef0 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.864+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c15c4b0 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.865+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c06fcc0 con 0x7f416c0b2e30 2023-10-06T05:29:37.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.866+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f416401e3b0 con 0x7f416c0b2e30 2023-10-06T05:29:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.867+0000 7f41617fa700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4150066f30 0x7f41500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.867+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f4164084d60 con 0x7f416c0b2e30 2023-10-06T05:29:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.868+0000 7f417179b700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4150066f30 0x7f41500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:37.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.868+0000 7f417179b700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4150066f30 0x7f41500693e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4168006fd0 tx=0x7f4168008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:37.871+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4164051d20 con 0x7f416c0b2e30 2023-10-06T05:29:38.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.121+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f416c169e50 con 0x7f416c0b2e30 2023-10-06T05:29:38.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.122+0000 7f41617fa700 1 -- 172.21.15.33:0/4138345220 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f416401a090 con 0x7f416c0b2e30 2023-10-06T05:29:38.124 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:38.124 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:33.890660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4150066f30 msgr2=0x7f41500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4150066f30 0x7f41500693e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4168006fd0 tx=0x7f4168008040 comp rx=0 tx=0).stop 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 msgr2=0x7f416c0c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 0x7f416c0c4310 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f416400d100 tx=0x7f41640099b0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 shutdown_connections 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4150066f30 0x7f41500693e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f416c0b2e30 0x7f416c0c4310 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c0b0a40 0x7f416c0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 --2- 172.21.15.33:0/4138345220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0b0e10 0x7f416c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 >> 172.21.15.33:0/4138345220 conn(0x7f416c06e0f0 msgr2=0x7f416c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 shutdown_connections 2023-10-06T05:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.124+0000 7f417279d700 1 -- 172.21.15.33:0/4138345220 wait complete. 2023-10-06T05:29:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 169 2023-10-06T05:29:38.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:38.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.329+0000 7f54bd59d700 1 Processor -- start 2023-10-06T05:29:38.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.329+0000 7f54bd59d700 1 -- start start 2023-10-06T05:29:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bd59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b8155140 0x7f54b8158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bd59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f54b80adf50 0x7f54b80c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bd59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54b80c9810 0x7f54b80699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bd59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b80c8e40 con 0x7f54b80c9810 2023-10-06T05:29:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bd59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b80ae6c0 con 0x7f54b80adf50 2023-10-06T05:29:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bd59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80ae9a0 con 0x7f54b8155140 2023-10-06T05:29:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.330+0000 7f54bcd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54b80c9810 0x7f54b80699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54b80c9810 0x7f54b80699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59800/0 (socket says 172.21.15.33:59800) 2023-10-06T05:29:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 -- 172.21.15.33:0/887680330 learned_addr learned my addr 172.21.15.33:0/887680330 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54b7fff700 1 --2- 172.21.15.33:0/887680330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b8155140 0x7f54b8158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54b77fe700 1 --2- 172.21.15.33:0/887680330 >> v2:172.21.15.167:3300/0 conn(0x7f54b80adf50 0x7f54b80c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 -- 172.21.15.33:0/887680330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b8155140 msgr2=0x7f54b8158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 --2- 172.21.15.33:0/887680330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b8155140 0x7f54b8158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 -- 172.21.15.33:0/887680330 >> v2:172.21.15.167:3300/0 conn(0x7f54b80adf50 msgr2=0x7f54b80c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 --2- 172.21.15.33:0/887680330 >> v2:172.21.15.167:3300/0 conn(0x7f54b80adf50 0x7f54b80c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 -- 172.21.15.33:0/887680330 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b80c8ad0 con 0x7f54b80c9810 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54bcd9c700 1 --2- 172.21.15.33:0/887680330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80c9810 0x7f54b80699c0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f54b80719a0 tx=0x7f54b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f38c27531ebcf26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.331+0000 7f54b6ffd700 1 -- 172.21.15.33:0/887680330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000e070 con 0x7f54b80c9810 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54b6ffd700 1 -- 172.21.15.33:0/887680330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b000ba50 con 0x7f54b80c9810 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54b6ffd700 1 -- 172.21.15.33:0/887680330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b00092e0 con 0x7f54b80c9810 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 -- 172.21.15.33:0/887680330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80c9810 msgr2=0x7f54b80699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 --2- 172.21.15.33:0/887680330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80c9810 0x7f54b80699c0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f54b80719a0 tx=0x7f54b0009840 comp rx=0 tx=0).stop 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 -- 172.21.15.33:0/887680330 shutdown_connections 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 --2- 172.21.15.33:0/887680330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80c9810 0x7f54b80699c0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 --2- 172.21.15.33:0/887680330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b8155140 0x7f54b8158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 --2- 172.21.15.33:0/887680330 >> v2:172.21.15.167:3300/0 conn(0x7f54b80adf50 0x7f54b80c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 -- 172.21.15.33:0/887680330 >> 172.21.15.33:0/887680330 conn(0x7f54b806d030 msgr2=0x7f54b806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 -- 172.21.15.33:0/887680330 shutdown_connections 2023-10-06T05:29:38.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.332+0000 7f54bd59d700 1 -- 172.21.15.33:0/887680330 wait complete. 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.334+0000 7f54bd59d700 1 Processor -- start 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.334+0000 7f54bd59d700 1 -- start start 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bd59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 0x7f54b814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bd59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b80c9810 0x7f54b814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bd59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b8155140 0x7f54b80aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bd59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b814bda0 con 0x7f54b80adf50 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bd59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b814ac00 con 0x7f54b8155140 2023-10-06T05:29:38.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bd59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b814bfb0 con 0x7f54b80c9810 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 0x7f54b814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 0x7f54b814b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59816/0 (socket says 172.21.15.33:59816) 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 -- 172.21.15.33:0/2262778278 learned_addr learned my addr 172.21.15.33:0/2262778278 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54bcd9c700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b8155140 0x7f54b80aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b77fe700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b80c9810 0x7f54b814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 -- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b80c9810 msgr2=0x7f54b814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b80c9810 0x7f54b814ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 -- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b8155140 msgr2=0x7f54b80aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b8155140 0x7f54b80aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b7fff700 1 -- 172.21.15.33:0/2262778278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0009ae0 con 0x7f54b80adf50 2023-10-06T05:29:38.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.335+0000 7f54b77fe700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b80c9810 0x7f54b814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.336+0000 7f54b7fff700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 0x7f54b814b860 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f54a800d860 tx=0x7f54a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.336+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80125f0 con 0x7f54b80adf50 2023-10-06T05:29:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.336+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8012750 con 0x7f54b80adf50 2023-10-06T05:29:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.336+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b814c290 con 0x7f54b80adf50 2023-10-06T05:29:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.336+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8012b10 con 0x7f54b80adf50 2023-10-06T05:29:38.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.336+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b814c850 con 0x7f54b80adf50 2023-10-06T05:29:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.337+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b804c380 con 0x7f54b80adf50 2023-10-06T05:29:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.338+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54a801e460 con 0x7f54b80adf50 2023-10-06T05:29:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.339+0000 7f549ffff700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54a4066f30 0x7f54a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.339+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f54a8084e20 con 0x7f54b80adf50 2023-10-06T05:29:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.339+0000 7f54b77fe700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54a4066f30 0x7f54a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.340+0000 7f54b77fe700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54a4066f30 0x7f54a40693e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f54b80b1350 tx=0x7f54ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:38.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.343+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54a8051de0 con 0x7f54b80adf50 2023-10-06T05:29:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.545+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b806fa40 con 0x7f54b80adf50 2023-10-06T05:29:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.546+0000 7f549ffff700 1 -- 172.21.15.33:0/2262778278 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+17139 (secure 0 0 0) 0x7f54a8055400 con 0x7f54b80adf50 2023-10-06T05:29:38.548 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:38.548 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":214,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:30.873187+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:17.571482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:18.582693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/1485763855":"2023-10-07T05:29:30.873167+0000","172.21.15.167:6835/1485763855":"2023-10-07T05:29:30.873167+0000","172.21.15.33:6834/963232630":"2023-10-07T05:29:30.377199+0000","172.21.15.33:6832/963232630":"2023-10-07T05:29:30.377199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:38.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.548+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54a4066f30 msgr2=0x7f54a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.548+0000 7f54bd59d700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54a4066f30 0x7f54a40693e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f54b80b1350 tx=0x7f54ac008040 comp rx=0 tx=0).stop 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.548+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 msgr2=0x7f54b814b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.548+0000 7f54bd59d700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 0x7f54b814b860 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f54a800d860 tx=0x7f54a8008040 comp rx=0 tx=0).stop 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 shutdown_connections 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f54a4066f30 0x7f54a40693e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54b8155140 0x7f54b80aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54b80adf50 0x7f54b814b860 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 --2- 172.21.15.33:0/2262778278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54b80c9810 0x7f54b814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 >> 172.21.15.33:0/2262778278 conn(0x7f54b806d030 msgr2=0x7f54b80c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 shutdown_connections 2023-10-06T05:29:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.549+0000 7f54bd59d700 1 -- 172.21.15.33:0/2262778278 wait complete. 2023-10-06T05:29:38.569 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:29:38.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:38.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.755+0000 7f0103fff700 1 Processor -- start 2023-10-06T05:29:38.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.755+0000 7f0103fff700 1 -- start start 2023-10-06T05:29:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f0103fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01040c6340 0x7f01040c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f0103fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010406c820 con 0x7f01040c80d0 2023-10-06T05:29:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f0103fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010406a160 con 0x7f0104159520 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f0103fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01040c8800 con 0x7f01040c6340 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f01027fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01040c6340 0x7f01040c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f01027fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01040c6340 0x7f01040c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59832/0 (socket says 172.21.15.33:59832) 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f01037fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01040c80d0 0x7f010406a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46276/0 (socket says 172.21.15.33:46276) 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f01027fc700 1 -- 172.21.15.33:0/1722698052 learned_addr learned my addr 172.21.15.33:0/1722698052 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.757+0000 7f01027fc700 1 -- 172.21.15.33:0/1722698052 >> v1:172.21.15.167:6789/0 conn(0x7f0104159520 legacy=0x7f01040c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f01027fc700 1 -- 172.21.15.33:0/1722698052 >> v1:172.21.15.33:6789/0 conn(0x7f01040c80d0 legacy=0x7f010406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f01027fc700 1 -- 172.21.15.33:0/1722698052 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01040c8ae0 con 0x7f01040c6340 2023-10-06T05:29:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f0101ffb700 1 -- 172.21.15.33:0/1722698052 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884551342 0 0) 0x7f010406c820 con 0x7f01040c80d0 2023-10-06T05:29:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f01027fc700 1 --2- 172.21.15.33:0/1722698052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01040c6340 0x7f01040c7b90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f00f80099b0 tx=0x7f00f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb5f509e90793f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f0101ffb700 1 -- 172.21.15.33:0/1722698052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800b860 con 0x7f01040c6340 2023-10-06T05:29:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f0101ffb700 1 -- 172.21.15.33:0/1722698052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f800b9c0 con 0x7f01040c6340 2023-10-06T05:29:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.758+0000 7f0101ffb700 1 -- 172.21.15.33:0/1722698052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800bd60 con 0x7f01040c6340 2023-10-06T05:29:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 -- 172.21.15.33:0/1722698052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01040c6340 msgr2=0x7f01040c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 --2- 172.21.15.33:0/1722698052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01040c6340 0x7f01040c7b90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f00f80099b0 tx=0x7f00f8009250 comp rx=0 tx=0).stop 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 -- 172.21.15.33:0/1722698052 shutdown_connections 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 --2- 172.21.15.33:0/1722698052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01040c6340 0x7f01040c7b90 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 -- 172.21.15.33:0/1722698052 >> 172.21.15.33:0/1722698052 conn(0x7f01040cc930 msgr2=0x7f01040ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 -- 172.21.15.33:0/1722698052 shutdown_connections 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.759+0000 7f0103fff700 1 -- 172.21.15.33:0/1722698052 wait complete. 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.760+0000 7f0103fff700 1 Processor -- start 2023-10-06T05:29:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.760+0000 7f0103fff700 1 -- start start 2023-10-06T05:29:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.761+0000 7f0103fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01040c6340 0x7f01040abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.761+0000 7f0103fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01040c80d0 0x7f01040ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.761+0000 7f0103fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 0x7f01040cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.761+0000 7f0103fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010416a200 con 0x7f0104159520 2023-10-06T05:29:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.761+0000 7f0103fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01040aef10 con 0x7f01040c80d0 2023-10-06T05:29:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.761+0000 7f0103fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010416a690 con 0x7f01040c6340 2023-10-06T05:29:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 0x7f01040cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 0x7f01040cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59844/0 (socket says 172.21.15.33:59844) 2023-10-06T05:29:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 -- 172.21.15.33:0/102524306 learned_addr learned my addr 172.21.15.33:0/102524306 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01027fc700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01040c80d0 0x7f01040ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f0102ffd700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01040c6340 0x7f01040abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 -- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01040c6340 msgr2=0x7f01040abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01040c6340 0x7f01040abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 -- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01040c80d0 msgr2=0x7f01040ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01040c80d0 0x7f01040ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 -- 172.21.15.33:0/102524306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80094f0 con 0x7f0104159520 2023-10-06T05:29:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01027fc700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01040c80d0 0x7f01040ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.762+0000 7f01037fe700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 0x7f01040cb2c0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00cd30 tx=0x7f00fc001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.763+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc005df0 con 0x7f0104159520 2023-10-06T05:29:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.763+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc00a920 con 0x7f0104159520 2023-10-06T05:29:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.763+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01041698d0 con 0x7f0104159520 2023-10-06T05:29:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.763+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00ace0 con 0x7f0104159520 2023-10-06T05:29:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.763+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0104169e90 con 0x7f0104159520 2023-10-06T05:29:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.764+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010404c380 con 0x7f0104159520 2023-10-06T05:29:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.765+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00fc007300 con 0x7f0104159520 2023-10-06T05:29:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.765+0000 7f00eaffd700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f0066f30 0x7f00f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.766+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f00fc084990 con 0x7f0104159520 2023-10-06T05:29:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.766+0000 7f0102ffd700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f0066f30 0x7f00f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.767+0000 7f0102ffd700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f0066f30 0x7f00f00693e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f01040cda60 tx=0x7f00f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:38.770+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00fc0519d0 con 0x7f0104159520 2023-10-06T05:29:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.019+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01040c8dc0 con 0x7f0104159520 2023-10-06T05:29:39.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.020+0000 7f00eaffd700 1 -- 172.21.15.33:0/102524306 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f00fc054ff0 con 0x7f0104159520 2023-10-06T05:29:39.021 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:33.890660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f0066f30 msgr2=0x7f00f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f0066f30 0x7f00f00693e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f01040cda60 tx=0x7f00f4009040 comp rx=0 tx=0).stop 2023-10-06T05:29:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 msgr2=0x7f01040cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 0x7f01040cb2c0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00cd30 tx=0x7f00fc001ac0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 shutdown_connections 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f00f0066f30 0x7f00f00693e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0104159520 0x7f01040cb2c0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01040c6340 0x7f01040abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 --2- 172.21.15.33:0/102524306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01040c80d0 0x7f01040ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 >> 172.21.15.33:0/102524306 conn(0x7f01040cc930 msgr2=0x7f0104157080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 shutdown_connections 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.022+0000 7f0103fff700 1 -- 172.21.15.33:0/102524306 wait complete. 2023-10-06T05:29:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 169 2023-10-06T05:29:39.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:39.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.228+0000 7fda6359e700 1 Processor -- start 2023-10-06T05:29:39.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.228+0000 7fda6359e700 1 -- start start 2023-10-06T05:29:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.229+0000 7fda6359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fda64155140 0x7fda64158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.229+0000 7fda6359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda640c8e40 con 0x7fda640699c0 2023-10-06T05:29:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.229+0000 7fda6359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda640c99e0 con 0x7fda640adf50 2023-10-06T05:29:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.229+0000 7fda6359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640ae3f0 con 0x7fda64155140 2023-10-06T05:29:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda62d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fda640699c0 0x7fda640c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:46278/0 (socket says 172.21.15.33:46278) 2023-10-06T05:29:39.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda62d9d700 1 -- 172.21.15.33:0/1343654152 learned_addr learned my addr 172.21.15.33:0/1343654152 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6259c700 1 --2- 172.21.15.33:0/1343654152 >> v2:172.21.15.167:3300/0 conn(0x7fda64155140 0x7fda64158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912618523 0 0) 0x7fda640c8e40 con 0x7fda640699c0 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda4c003570 con 0x7fda640699c0 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798213607 0 0) 0x7fda4c003570 con 0x7fda640699c0 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 >> v2:172.21.15.167:3300/0 conn(0x7fda64155140 msgr2=0x7fda64158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 --2- 172.21.15.33:0/1343654152 >> v2:172.21.15.167:3300/0 conn(0x7fda64155140 0x7fda64158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 >> v1:172.21.15.167:6789/0 conn(0x7fda640adf50 legacy=0x7fda640c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.230+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640ae6c0 con 0x7fda640699c0 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fda5c0017f0 con 0x7fda640699c0 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 >> v1:172.21.15.33:6789/0 conn(0x7fda640699c0 legacy=0x7fda640c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4c003a60 0x7fda4c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda4c004810 0x7fda4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c0051c0 0x7fda4c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640ae3f0 con 0x7fda4c003a60 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c00c0b0 con 0x7fda4c004810 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c00c270 con 0x7fda4c0051c0 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6259c700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4c003a60 0x7fda4c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda61d9b700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda4c004810 0x7fda4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda62d9d700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c0051c0 0x7fda4c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6259c700 1 -- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c0051c0 msgr2=0x7fda4c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6259c700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c0051c0 0x7fda4c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6259c700 1 -- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda4c004810 msgr2=0x7fda4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6259c700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda4c004810 0x7fda4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda6259c700 1 -- 172.21.15.33:0/1343654152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c003570 con 0x7fda4c003a60 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.231+0000 7fda62d9d700 1 -- 172.21.15.33:0/1343654152 reap_dead start 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6259c700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4c003a60 0x7fda4c003e30 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fda54009c00 tx=0x7fda540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187a307d9a239e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400e710 con 0x7fda4c003a60 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5400e870 con 0x7fda4c003a60 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6159a700 1 -- 172.21.15.33:0/1343654152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400ec30 con 0x7fda4c003a60 2023-10-06T05:29:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 -- 172.21.15.33:0/1343654152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4c003a60 msgr2=0x7fda4c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4c003a60 0x7fda4c003e30 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fda54009c00 tx=0x7fda540091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 -- 172.21.15.33:0/1343654152 shutdown_connections 2023-10-06T05:29:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 --2- 172.21.15.33:0/1343654152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda4c003a60 0x7fda4c003e30 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 -- 172.21.15.33:0/1343654152 >> 172.21.15.33:0/1343654152 conn(0x7fda6406d030 msgr2=0x7fda6406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 -- 172.21.15.33:0/1343654152 shutdown_connections 2023-10-06T05:29:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.232+0000 7fda6359e700 1 -- 172.21.15.33:0/1343654152 wait complete. 2023-10-06T05:29:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.234+0000 7fda6359e700 1 Processor -- start 2023-10-06T05:29:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.234+0000 7fda6359e700 1 -- start start 2023-10-06T05:29:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c003a60 0x7fda640b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda6414efb0 0x7fda64141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 0x7fda64153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6414f840 con 0x7fda64152bd0 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641545d0 con 0x7fda6414efb0 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda640b1b50 con 0x7fda4c003a60 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 0x7fda64153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 0x7fda64153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59854/0 (socket says 172.21.15.33:59854) 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda62d9d700 1 -- 172.21.15.33:0/1374832463 learned_addr learned my addr 172.21.15.33:0/1374832463 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda6259c700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c003a60 0x7fda640b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.235+0000 7fda61d9b700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda6414efb0 0x7fda64141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda62d9d700 1 -- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c003a60 msgr2=0x7fda640b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda62d9d700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c003a60 0x7fda640b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda62d9d700 1 -- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda6414efb0 msgr2=0x7fda64141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda62d9d700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda6414efb0 0x7fda64141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda62d9d700 1 -- 172.21.15.33:0/1374832463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64154140 con 0x7fda64152bd0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda61d9b700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda6414efb0 0x7fda64141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda6259c700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c003a60 0x7fda640b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda62d9d700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 0x7fda64153020 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fda64068540 tx=0x7fda5c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c017070 con 0x7fda64152bd0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c00ec60 con 0x7fda64152bd0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c013420 con 0x7fda64152bd0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda64142340 con 0x7fda64152bd0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.236+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda64153af0 con 0x7fda64152bd0 2023-10-06T05:29:39.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.237+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda641542d0 con 0x7fda64152bd0 2023-10-06T05:29:39.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.238+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda5c013850 con 0x7fda64152bd0 2023-10-06T05:29:39.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.239+0000 7fda527fc700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda44066f30 0x7fda440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.239+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fda5c084710 con 0x7fda64152bd0 2023-10-06T05:29:39.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.240+0000 7fda6259c700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda44066f30 0x7fda440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.241+0000 7fda6259c700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda44066f30 0x7fda440693e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fda540091e0 tx=0x7fda5401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:39.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.244+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda5c0516d0 con 0x7fda64152bd0 2023-10-06T05:29:39.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.489+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda6404c380 con 0x7fda64152bd0 2023-10-06T05:29:39.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.489+0000 7fda527fc700 1 -- 172.21.15.33:0/1374832463 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fda5c054cf0 con 0x7fda64152bd0 2023-10-06T05:29:39.491 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:39.491 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:19.601836+0000","modified":"2023-10-06T05:29:33.890660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":214,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6833/2044336703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2044336703},{"type":"v1","addr":"172.21.15.33:6833","nonce":2044336703}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.491+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda44066f30 msgr2=0x7fda440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.491+0000 7fda6359e700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda44066f30 0x7fda440693e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fda540091e0 tx=0x7fda5401a000 comp rx=0 tx=0).stop 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.491+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 msgr2=0x7fda64153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.491+0000 7fda6359e700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 0x7fda64153020 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fda64068540 tx=0x7fda5c00e440 comp rx=0 tx=0).stop 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 shutdown_connections 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fda44066f30 0x7fda440693e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda64152bd0 0x7fda64153020 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda4c003a60 0x7fda640b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 --2- 172.21.15.33:0/1374832463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda6414efb0 0x7fda64141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 >> 172.21.15.33:0/1374832463 conn(0x7fda6406d030 msgr2=0x7fda640cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 shutdown_connections 2023-10-06T05:29:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.492+0000 7fda6359e700 1 -- 172.21.15.33:0/1374832463 wait complete. 2023-10-06T05:29:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 169 2023-10-06T05:29:39.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.697+0000 7f5acd99d700 1 Processor -- start 2023-10-06T05:29:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.697+0000 7f5acd99d700 1 -- start start 2023-10-06T05:29:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.698+0000 7f5acd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac8159520 0x7f5ac80c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.698+0000 7f5acd99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ac80c6170 0x7f5ac80c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.698+0000 7f5acd99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac80c65e0 con 0x7f5ac80c80d0 2023-10-06T05:29:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.698+0000 7f5acd99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac806c820 con 0x7f5ac80c6170 2023-10-06T05:29:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.698+0000 7f5acd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac806b670 con 0x7f5ac8159520 2023-10-06T05:29:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.698+0000 7f5ac7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ac80c6170 0x7f5ac80c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5ac80c6170 0x7f5ac80c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59870/0 (socket says 172.21.15.33:59870) 2023-10-06T05:29:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 -- 172.21.15.33:0/894898162 learned_addr learned my addr 172.21.15.33:0/894898162 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:39.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5acc99b700 1 --2- 172.21.15.33:0/894898162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac8159520 0x7f5ac80c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 -- 172.21.15.33:0/894898162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac8159520 msgr2=0x7f5ac80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 --2- 172.21.15.33:0/894898162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac8159520 0x7f5ac80c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 -- 172.21.15.33:0/894898162 >> v1:172.21.15.167:6789/0 conn(0x7f5ac80c80d0 legacy=0x7f5ac8067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 -- 172.21.15.33:0/894898162 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac806a160 con 0x7f5ac80c6170 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.699+0000 7f5ac7fff700 1 --2- 172.21.15.33:0/894898162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c6170 0x7f5ac80c7b90 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40099b0 tx=0x7f5ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef9aab836882442 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5ac77fe700 1 -- 172.21.15.33:0/894898162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400b860 con 0x7f5ac80c6170 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5ac77fe700 1 -- 172.21.15.33:0/894898162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab400b9c0 con 0x7f5ac80c6170 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5ac77fe700 1 -- 172.21.15.33:0/894898162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400bd60 con 0x7f5ac80c6170 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 -- 172.21.15.33:0/894898162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c6170 msgr2=0x7f5ac80c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 --2- 172.21.15.33:0/894898162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c6170 0x7f5ac80c7b90 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40099b0 tx=0x7f5ab4009250 comp rx=0 tx=0).stop 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 -- 172.21.15.33:0/894898162 shutdown_connections 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 --2- 172.21.15.33:0/894898162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac8159520 0x7f5ac80c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 --2- 172.21.15.33:0/894898162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c6170 0x7f5ac80c7b90 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 -- 172.21.15.33:0/894898162 >> 172.21.15.33:0/894898162 conn(0x7f5ac80cc930 msgr2=0x7f5ac80ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 -- 172.21.15.33:0/894898162 shutdown_connections 2023-10-06T05:29:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.700+0000 7f5acd99d700 1 -- 172.21.15.33:0/894898162 wait complete. 2023-10-06T05:29:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.702+0000 7f5acd99d700 1 Processor -- start 2023-10-06T05:29:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.702+0000 7f5acd99d700 1 -- start start 2023-10-06T05:29:39.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac80c6170 0x7f5ac816a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 0x7f5ac814a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac8159520 0x7f5ac815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8156f90 con 0x7f5ac80c80d0 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac816a820 con 0x7f5ac8159520 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8157250 con 0x7f5ac80c6170 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 0x7f5ac814a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 0x7f5ac814a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59878/0 (socket says 172.21.15.33:59878) 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 -- 172.21.15.33:0/3261044967 learned_addr learned my addr 172.21.15.33:0/3261044967 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acc99b700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac80c6170 0x7f5ac816a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5acd19c700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac8159520 0x7f5ac815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 -- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac80c6170 msgr2=0x7f5ac816a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac80c6170 0x7f5ac816a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 -- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac8159520 msgr2=0x7f5ac815c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac8159520 0x7f5ac815c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.703+0000 7f5ac7fff700 1 -- 172.21.15.33:0/3261044967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40094f0 con 0x7f5ac80c80d0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.704+0000 7f5ac7fff700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 0x7f5ac814a3c0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80636f0 tx=0x7f5ab4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.704+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4011e40 con 0x7f5ac80c80d0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.704+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4024440 con 0x7f5ac80c80d0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.704+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4024800 con 0x7f5ac80c80d0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.704+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac8157530 con 0x7f5ac80c80d0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.704+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac815bfe0 con 0x7f5ac80c80d0 2023-10-06T05:29:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.705+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac815c200 con 0x7f5ac80c80d0 2023-10-06T05:29:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.706+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ab401a420 con 0x7f5ac80c80d0 2023-10-06T05:29:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.707+0000 7f5ac4ff9700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5aac066f30 0x7f5aac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.707+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f5ab4086560 con 0x7f5ac80c80d0 2023-10-06T05:29:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.707+0000 7f5acc99b700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5aac066f30 0x7f5aac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:39.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.708+0000 7f5acc99b700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5aac066f30 0x7f5aac0693e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80ac7d0 tx=0x7f5abc0097b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.711+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ab40535c0 con 0x7f5ac80c80d0 2023-10-06T05:29:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.917+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ac804c380 con 0x7f5ac80c80d0 2023-10-06T05:29:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.918+0000 7f5ac4ff9700 1 -- 172.21.15.33:0/3261044967 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+17139 (secure 0 0 0) 0x7f5ab4056be0 con 0x7f5ac80c80d0 2023-10-06T05:29:39.920 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:39.920 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":214,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:30.873187+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:17.571482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:18.582693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/1485763855":"2023-10-07T05:29:30.873167+0000","172.21.15.167:6835/1485763855":"2023-10-07T05:29:30.873167+0000","172.21.15.33:6834/963232630":"2023-10-07T05:29:30.377199+0000","172.21.15.33:6832/963232630":"2023-10-07T05:29:30.377199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5aac066f30 msgr2=0x7f5aac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5aac066f30 0x7f5aac0693e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80ac7d0 tx=0x7f5abc0097b0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 msgr2=0x7f5ac814a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 0x7f5ac814a3c0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80636f0 tx=0x7f5ab4017000 comp rx=0 tx=0).stop 2023-10-06T05:29:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 shutdown_connections 2023-10-06T05:29:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5aac066f30 0x7f5aac0693e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac8159520 0x7f5ac815c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ac80c6170 0x7f5ac816a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 --2- 172.21.15.33:0/3261044967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac80c80d0 0x7f5ac814a3c0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 >> 172.21.15.33:0/3261044967 conn(0x7f5ac80cc930 msgr2=0x7f5ac80b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 shutdown_connections 2023-10-06T05:29:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:39.920+0000 7f5acd99d700 1 -- 172.21.15.33:0/3261044967 wait complete. 2023-10-06T05:29:39.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:29:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.127+0000 7f3fc2f9d700 1 Processor -- start 2023-10-06T05:29:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.127+0000 7f3fc2f9d700 1 -- start start 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc2f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fbc0c7070 0x7f3fbc0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc2f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc06a410 con 0x7f3fbc155bc0 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc2f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc0cccd0 con 0x7f3fbc1557f0 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc2f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc0c7500 con 0x7f3fbc0c7070 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc279c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3fbc155bc0 0x7f3fbc06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32830/0 (socket says 172.21.15.33:32830) 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc279c700 1 -- 172.21.15.33:0/2857250143 learned_addr learned my addr 172.21.15.33:0/2857250143 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc179a700 1 --2- 172.21.15.33:0/2857250143 >> v2:172.21.15.167:3300/0 conn(0x7f3fbc0c7070 0x7f3fbc0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409397502 0 0) 0x7f3fbc06a410 con 0x7f3fbc155bc0 2023-10-06T05:29:40.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.129+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa8004680 con 0x7f3fbc155bc0 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907410487 0 0) 0x7f3fbc0cccd0 con 0x7f3fbc1557f0 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fbc06a410 con 0x7f3fbc1557f0 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36248640 0 0) 0x7f3fa8004680 con 0x7f3fbc155bc0 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 >> v2:172.21.15.167:3300/0 conn(0x7f3fbc0c7070 msgr2=0x7f3fbc0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 --2- 172.21.15.33:0/2857250143 >> v2:172.21.15.167:3300/0 conn(0x7f3fbc0c7070 0x7f3fbc0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 >> v1:172.21.15.167:6789/0 conn(0x7f3fbc1557f0 legacy=0x7f3fbc0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0c77e0 con 0x7f3fbc155bc0 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f3fb40017f0 con 0x7f3fbc155bc0 2023-10-06T05:29:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 >> v1:172.21.15.33:6789/0 conn(0x7f3fbc155bc0 legacy=0x7f3fbc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa8004d50 0x7f3fa8005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fa8005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8006320 0x7f3fa8003490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0c7500 con 0x7f3fa8005920 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8003bc0 con 0x7f3fa8004d50 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8003d80 con 0x7f3fa8006320 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc179a700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fa8005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc1f9b700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa8004d50 0x7f3fa8005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.130+0000 7f3fc279c700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8006320 0x7f3fa8003490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc179a700 1 -- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8006320 msgr2=0x7f3fa8003490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc179a700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8006320 0x7f3fa8003490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc179a700 1 -- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa8004d50 msgr2=0x7f3fa8005140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc179a700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa8004d50 0x7f3fa8005140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc179a700 1 -- 172.21.15.33:0/2857250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8004680 con 0x7f3fa8005920 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc1f9b700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa8004d50 0x7f3fa8005140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc279c700 1 -- 172.21.15.33:0/2857250143 reap_dead start 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc179a700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fa8005d90 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009c00 tx=0x7f3fac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d874304c097d33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac00e710 con 0x7f3fa8005920 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac00e870 con 0x7f3fa8005920 2023-10-06T05:29:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.131+0000 7f3fc0f99700 1 -- 172.21.15.33:0/2857250143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac00ec30 con 0x7f3fa8005920 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/2857250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 msgr2=0x7f3fa8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fa8005d90 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009c00 tx=0x7f3fac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/2857250143 shutdown_connections 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/2857250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fa8005d90 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/2857250143 >> 172.21.15.33:0/2857250143 conn(0x7f3fbc06c560 msgr2=0x7f3fbc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/2857250143 shutdown_connections 2023-10-06T05:29:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.132+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/2857250143 wait complete. 2023-10-06T05:29:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.133+0000 7f3fc2f9d700 1 Processor -- start 2023-10-06T05:29:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.133+0000 7f3fc2f9d700 1 -- start start 2023-10-06T05:29:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc2f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fbc167ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc2f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fbc0c63e0 0x7f3fbc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fbc168530 0x7f3fbc1689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc2f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc168ef0 con 0x7f3fa8005920 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b0aa0 con 0x7f3fbc168530 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc2f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b0d80 con 0x7f3fbc0c63e0 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc1f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fbc167ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.134+0000 7f3fc1f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fbc167ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45708/0 (socket says 172.21.15.33:45708) 2023-10-06T05:29:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 -- 172.21.15.33:0/1200958605 learned_addr learned my addr 172.21.15.33:0/1200958605 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc179a700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fbc0c63e0 0x7f3fbc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc279c700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fbc168530 0x7f3fbc1689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 -- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fbc0c63e0 msgr2=0x7f3fbc0a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fbc0c63e0 0x7f3fbc0a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 -- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fbc168530 msgr2=0x7f3fbc1689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fbc168530 0x7f3fbc1689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 -- 172.21.15.33:0/1200958605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0b1060 con 0x7f3fa8005920 2023-10-06T05:29:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc279c700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fbc168530 0x7f3fbc1689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc1f9b700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fbc167ce0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e810 tx=0x7f3fb8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8006820 con 0x7f3fa8005920 2023-10-06T05:29:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb8006980 con 0x7f3fa8005920 2023-10-06T05:29:40.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8006d00 con 0x7f3fa8005920 2023-10-06T05:29:40.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.135+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc0b1340 con 0x7f3fa8005920 2023-10-06T05:29:40.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.136+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc168220 con 0x7f3fa8005920 2023-10-06T05:29:40.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.137+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc0b1840 con 0x7f3fa8005920 2023-10-06T05:29:40.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.137+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3fb8007240 con 0x7f3fa8005920 2023-10-06T05:29:40.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.138+0000 7f3fb1ffb700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3fa0066f30 0x7f3fa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.138+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f3fb8084ae0 con 0x7f3fa8005920 2023-10-06T05:29:40.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.139+0000 7f3fc179a700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3fa0066f30 0x7f3fa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.139+0000 7f3fc179a700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3fa0066f30 0x7f3fa00693e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0091e0 tx=0x7f3fac01a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.142+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fb8051aa0 con 0x7f3fa8005920 2023-10-06T05:29:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.392+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3fbc04c380 con 0x7f3fa8005920 2023-10-06T05:29:40.395 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:29:40.393+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:29:40.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.398+0000 7f3fb1ffb700 1 -- 172.21.15.33:0/1200958605 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v170) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3fb8084030 con 0x7f3fa8005920 2023-10-06T05:29:40.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.400+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3fa0066f30 msgr2=0x7f3fa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.400+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3fa0066f30 0x7f3fa00693e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0091e0 tx=0x7f3fac01a000 comp rx=0 tx=0).stop 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.400+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 msgr2=0x7f3fbc167ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.400+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fbc167ce0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e810 tx=0x7f3fb8009040 comp rx=0 tx=0).stop 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 shutdown_connections 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3fa0066f30 0x7f3fa00693e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fbc168530 0x7f3fbc1689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa8005920 0x7f3fbc167ce0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 --2- 172.21.15.33:0/1200958605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fbc0c63e0 0x7f3fbc0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 >> 172.21.15.33:0/1200958605 conn(0x7f3fbc06c560 msgr2=0x7f3fbc0c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:40.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 shutdown_connections 2023-10-06T05:29:40.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.401+0000 7f3fc2f9d700 1 -- 172.21.15.33:0/1200958605 wait complete. 2023-10-06T05:29:40.404 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:29:40.411 INFO:tasks.ceph.mds.a.smithi033.stderr: -150> 2023-10-06T05:29:35.888+0000 7f50633f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:29:40.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:29:40.447 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:29:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.611+0000 7ff18259d700 1 Processor -- start 2023-10-06T05:29:40.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.612+0000 7ff18259d700 1 -- start start 2023-10-06T05:29:40.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.613+0000 7ff18259d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff17c155a50 0x7ff17c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.613+0000 7ff18259d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0b06b0 con 0x7ff17c0afb20 2023-10-06T05:29:40.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.613+0000 7ff18259d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0c99e0 con 0x7ff17c0c9490 2023-10-06T05:29:40.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.613+0000 7ff18259d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0c9cc0 con 0x7ff17c155a50 2023-10-06T05:29:40.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.613+0000 7ff181d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff17c0afb20 0x7ff17c0caa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32846/0 (socket says 172.21.15.33:32846) 2023-10-06T05:29:40.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.613+0000 7ff181d9c700 1 -- 172.21.15.33:0/1659180321 learned_addr learned my addr 172.21.15.33:0/1659180321 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:40.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff18159b700 1 --2- 172.21.15.33:0/1659180321 >> v2:172.21.15.167:3300/0 conn(0x7ff17c155a50 0x7ff17c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff173fff700 1 -- 172.21.15.33:0/1659180321 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735230734 0 0) 0x7ff17c0b06b0 con 0x7ff17c0afb20 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff173fff700 1 -- 172.21.15.33:0/1659180321 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff168003570 con 0x7ff17c0afb20 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff18159b700 1 -- 172.21.15.33:0/1659180321 >> v1:172.21.15.167:6789/0 conn(0x7ff17c0c9490 legacy=0x7ff17c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff18159b700 1 -- 172.21.15.33:0/1659180321 >> v1:172.21.15.33:6789/0 conn(0x7ff17c0afb20 legacy=0x7ff17c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff173fff700 1 -- 172.21.15.33:0/1659180321 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126714512 0 0) 0x7ff168003570 con 0x7ff17c0afb20 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.614+0000 7ff18159b700 1 -- 172.21.15.33:0/1659180321 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0c9fa0 con 0x7ff17c155a50 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff18159b700 1 --2- 172.21.15.33:0/1659180321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c155e20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff17800aa90 tx=0x7ff17800a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4f5bea8afa5272 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff173fff700 1 -- 172.21.15.33:0/1659180321 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1740017f0 con 0x7ff17c155a50 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff173fff700 1 -- 172.21.15.33:0/1659180321 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17800c660 con 0x7ff17c155a50 2023-10-06T05:29:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff173fff700 1 -- 172.21.15.33:0/1659180321 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800ca00 con 0x7ff17c155a50 2023-10-06T05:29:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff18259d700 1 -- 172.21.15.33:0/1659180321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 msgr2=0x7ff17c155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff18259d700 1 --2- 172.21.15.33:0/1659180321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c155e20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff17800aa90 tx=0x7ff17800a330 comp rx=0 tx=0).stop 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff18259d700 1 -- 172.21.15.33:0/1659180321 shutdown_connections 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff18259d700 1 --2- 172.21.15.33:0/1659180321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c155e20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.615+0000 7ff18259d700 1 -- 172.21.15.33:0/1659180321 >> 172.21.15.33:0/1659180321 conn(0x7ff17c06d030 msgr2=0x7ff17c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.616+0000 7ff18259d700 1 -- 172.21.15.33:0/1659180321 shutdown_connections 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.616+0000 7ff18259d700 1 -- 172.21.15.33:0/1659180321 wait complete. 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.617+0000 7ff18259d700 1 Processor -- start 2023-10-06T05:29:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.617+0000 7ff18259d700 1 -- start start 2023-10-06T05:29:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff18259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17c0afb20 0x7ff17c154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff18259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 0x7ff17c149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff18259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff18259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c169790 con 0x7ff17c0c9490 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff18259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c154b70 con 0x7ff17c155a50 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff18259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c154e50 con 0x7ff17c0afb20 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff180d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 0x7ff17c149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff180d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 0x7ff17c149d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45728/0 (socket says 172.21.15.33:45728) 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.618+0000 7ff180d9a700 1 -- 172.21.15.33:0/2734301450 learned_addr learned my addr 172.21.15.33:0/2734301450 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff181d9c700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff18159b700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17c0afb20 0x7ff17c154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff180d9a700 1 -- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17c0afb20 msgr2=0x7ff17c154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff180d9a700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17c0afb20 0x7ff17c154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff180d9a700 1 -- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 msgr2=0x7ff17c14c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff180d9a700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c14c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff180d9a700 1 -- 172.21.15.33:0/2734301450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17800a5d0 con 0x7ff17c0c9490 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff181d9c700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff18159b700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17c0afb20 0x7ff17c154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff180d9a700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 0x7ff17c149d70 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c790 tx=0x7ff16c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012630 con 0x7ff17c0c9490 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c0127b0 con 0x7ff17c0c9490 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012b70 con 0x7ff17c0c9490 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c14f5f0 con 0x7ff17c0c9490 2023-10-06T05:29:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.619+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c151ce0 con 0x7ff17c0c9490 2023-10-06T05:29:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.620+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c1562a0 con 0x7ff17c0c9490 2023-10-06T05:29:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.621+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff16c01b400 con 0x7ff17c0c9490 2023-10-06T05:29:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.622+0000 7ff1717fa700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff160066f60 0x7ff160069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.622+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6316+0+0 (secure 0 0 0) 0x7ff16c084340 con 0x7ff17c0c9490 2023-10-06T05:29:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.622+0000 7ff18159b700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff160066f60 0x7ff160069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.623+0000 7ff18159b700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff160066f60 0x7ff160069410 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff17800d010 tx=0x7ff178019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.626+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff16c0512b0 con 0x7ff17c0c9490 2023-10-06T05:29:40.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:40.870+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff17c04c380 con 0x7ff17c0c9490 2023-10-06T05:29:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.398+0000 7ff1717fa700 1 -- 172.21.15.33:0/2734301450 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v171) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff16c0548d0 con 0x7ff17c0c9490 2023-10-06T05:29:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff160066f60 msgr2=0x7ff160069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff160066f60 0x7ff160069410 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff17800d010 tx=0x7ff178019000 comp rx=0 tx=0).stop 2023-10-06T05:29:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 msgr2=0x7ff17c149d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 0x7ff17c149d70 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c790 tx=0x7ff16c007f60 comp rx=0 tx=0).stop 2023-10-06T05:29:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 shutdown_connections 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff160066f60 0x7ff160069410 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c155a50 0x7ff17c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff17c0afb20 0x7ff17c154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 --2- 172.21.15.33:0/2734301450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c0c9490 0x7ff17c149d70 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.401+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 >> 172.21.15.33:0/2734301450 conn(0x7ff17c06d030 msgr2=0x7ff17c06aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.402+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 shutdown_connections 2023-10-06T05:29:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.402+0000 7ff18259d700 1 -- 172.21.15.33:0/2734301450 wait complete. 2023-10-06T05:29:41.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:29:41.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.656+0000 7f5b1c99d700 1 Processor -- start 2023-10-06T05:29:41.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.656+0000 7f5b1c99d700 1 -- start start 2023-10-06T05:29:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.657+0000 7f5b1c99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b18068260 0x7f5b180a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.657+0000 7f5b1c99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b1806dbf0 0x7f5b180af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.657+0000 7f5b1c99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b180a9dd0 con 0x7f5b18159400 2023-10-06T05:29:41.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.657+0000 7f5b1c99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1806d760 con 0x7f5b1806dbf0 2023-10-06T05:29:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.657+0000 7f5b1c99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b180aa8c0 con 0x7f5b18068260 2023-10-06T05:29:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b17fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b1806dbf0 0x7f5b180af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b177fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b18159400 0x7f5b18067b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32860/0 (socket says 172.21.15.33:32860) 2023-10-06T05:29:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b177fe700 1 -- 172.21.15.33:0/3377106661 learned_addr learned my addr 172.21.15.33:0/3377106661 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:41.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b16ffd700 1 --2- 172.21.15.33:0/3377106661 >> v2:172.21.15.167:3300/0 conn(0x7f5b18068260 0x7f5b180a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:41.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b17fff700 1 -- 172.21.15.33:0/3377106661 >> v2:172.21.15.167:3300/0 conn(0x7f5b18068260 msgr2=0x7f5b180a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b17fff700 1 --2- 172.21.15.33:0/3377106661 >> v2:172.21.15.167:3300/0 conn(0x7f5b18068260 0x7f5b180a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b17fff700 1 -- 172.21.15.33:0/3377106661 >> v1:172.21.15.33:6789/0 conn(0x7f5b18159400 legacy=0x7f5b18067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b17fff700 1 -- 172.21.15.33:0/3377106661 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b180aa5e0 con 0x7f5b1806dbf0 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b16ffd700 1 --2- 172.21.15.33:0/3377106661 >> v2:172.21.15.167:3300/0 conn(0x7f5b18068260 0x7f5b180a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.658+0000 7f5b17fff700 1 --2- 172.21.15.33:0/3377106661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b1806dbf0 0x7f5b180af830 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009e60 tx=0x7f5b10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2803cf13843f699 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b167fc700 1 -- 172.21.15.33:0/3377106661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000e070 con 0x7f5b1806dbf0 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b167fc700 1 -- 172.21.15.33:0/3377106661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000bbf0 con 0x7f5b1806dbf0 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b167fc700 1 -- 172.21.15.33:0/3377106661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10012430 con 0x7f5b1806dbf0 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3377106661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b1806dbf0 msgr2=0x7f5b180af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:41.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3377106661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b1806dbf0 0x7f5b180af830 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009e60 tx=0x7f5b10009780 comp rx=0 tx=0).stop 2023-10-06T05:29:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3377106661 shutdown_connections 2023-10-06T05:29:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3377106661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b1806dbf0 0x7f5b180af830 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3377106661 >> v2:172.21.15.167:3300/0 conn(0x7f5b18068260 0x7f5b180a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3377106661 >> 172.21.15.33:0/3377106661 conn(0x7f5b1806ebf0 msgr2=0x7f5b1806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3377106661 shutdown_connections 2023-10-06T05:29:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.659+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3377106661 wait complete. 2023-10-06T05:29:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.661+0000 7f5b1c99d700 1 Processor -- start 2023-10-06T05:29:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.661+0000 7f5b1c99d700 1 -- start start 2023-10-06T05:29:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b1c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 0x7f5b18148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b1c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1806dbf0 0x7f5b1806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b1c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b18159400 0x7f5b1806f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b1c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181695c0 con 0x7f5b18068260 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b1c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1815cf50 con 0x7f5b18159400 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b1c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1815d230 con 0x7f5b1806dbf0 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 0x7f5b18148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 0x7f5b18148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45752/0 (socket says 172.21.15.33:45752) 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 -- 172.21.15.33:0/3017390388 learned_addr learned my addr 172.21.15.33:0/3017390388 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b16ffd700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1806dbf0 0x7f5b1806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b17fff700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b18159400 0x7f5b1806f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 -- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1806dbf0 msgr2=0x7f5b1806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1806dbf0 0x7f5b1806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 -- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b18159400 msgr2=0x7f5b1806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b18159400 0x7f5b1806f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b177fe700 1 -- 172.21.15.33:0/3017390388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009a20 con 0x7f5b18068260 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b16ffd700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1806dbf0 0x7f5b1806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.662+0000 7f5b17fff700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b18159400 0x7f5b1806f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.663+0000 7f5b177fe700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 0x7f5b18148380 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c560 tx=0x7f5b08001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.663+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08005b30 con 0x7f5b18068260 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.663+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08005cb0 con 0x7f5b18068260 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.663+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1815d510 con 0x7f5b18068260 2023-10-06T05:29:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.663+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08007320 con 0x7f5b18068260 2023-10-06T05:29:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.663+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b181488c0 con 0x7f5b18068260 2023-10-06T05:29:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.664+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18063ff0 con 0x7f5b18068260 2023-10-06T05:29:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.665+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b0801b3b0 con 0x7f5b18068260 2023-10-06T05:29:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.666+0000 7f5aff7fe700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.666+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5b08083b20 con 0x7f5b18068260 2023-10-06T05:29:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.666+0000 7f5b16ffd700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.667+0000 7f5b16ffd700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b04066f30 0x7f5b040693e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c006f00 tx=0x7f5b0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:41.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.670+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b08050a90 con 0x7f5b18068260 2023-10-06T05:29:41.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:41.874+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5b180a9dd0 con 0x7f5b18068260 2023-10-06T05:29:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.402+0000 7f5aff7fe700 1 -- 172.21.15.33:0/3017390388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v216) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5b180a9dd0 con 0x7f5b18068260 2023-10-06T05:29:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b04066f30 msgr2=0x7f5b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b04066f30 0x7f5b040693e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c006f00 tx=0x7f5b0c008040 comp rx=0 tx=0).stop 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 msgr2=0x7f5b18148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 0x7f5b18148380 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c560 tx=0x7f5b08001020 comp rx=0 tx=0).stop 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 shutdown_connections 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b18159400 0x7f5b1806f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b18068260 0x7f5b18148380 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 --2- 172.21.15.33:0/3017390388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b1806dbf0 0x7f5b1806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.406+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 >> 172.21.15.33:0/3017390388 conn(0x7f5b1806ebf0 msgr2=0x7f5b180afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.407+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 shutdown_connections 2023-10-06T05:29:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.407+0000 7f5b1c99d700 1 -- 172.21.15.33:0/3017390388 wait complete. 2023-10-06T05:29:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:29:42.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:29:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.621+0000 7f6b6b59e700 1 Processor -- start 2023-10-06T05:29:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.621+0000 7f6b6b59e700 1 -- start start 2023-10-06T05:29:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.622+0000 7f6b6b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b6c157320 0x7f6b6c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.622+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.622+0000 7f6b6b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0ad3a0 con 0x7f6b6c0ad830 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.622+0000 7f6b6b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ac580 con 0x7f6b6c157320 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.622+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0afcc0 con 0x7f6b6c06ee20 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b6c157320 0x7f6b6c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b6c157320 0x7f6b6c159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45760/0 (socket says 172.21.15.33:45760) 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 -- 172.21.15.33:0/3908347096 learned_addr learned my addr 172.21.15.33:0/3908347096 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6ad9d700 1 --2- 172.21.15.33:0/3908347096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 -- 172.21.15.33:0/3908347096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 msgr2=0x7f6b6c0ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 --2- 172.21.15.33:0/3908347096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 -- 172.21.15.33:0/3908347096 >> v1:172.21.15.167:6789/0 conn(0x7f6b6c0ad830 legacy=0x7f6b6c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 -- 172.21.15.33:0/3908347096 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c1576f0 con 0x7f6b6c157320 2023-10-06T05:29:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.623+0000 7f6b6a59c700 1 --2- 172.21.15.33:0/3908347096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c159cd0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0099b0 tx=0x7f6b5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263c723c7beea582 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6959a700 1 -- 172.21.15.33:0/3908347096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00b860 con 0x7f6b6c157320 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6959a700 1 -- 172.21.15.33:0/3908347096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c00b9c0 con 0x7f6b6c157320 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6959a700 1 -- 172.21.15.33:0/3908347096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00bd60 con 0x7f6b6c157320 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 -- 172.21.15.33:0/3908347096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 msgr2=0x7f6b6c159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/3908347096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c159cd0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0099b0 tx=0x7f6b5c009250 comp rx=0 tx=0).stop 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 -- 172.21.15.33:0/3908347096 shutdown_connections 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/3908347096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c159cd0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/3908347096 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 -- 172.21.15.33:0/3908347096 >> 172.21.15.33:0/3908347096 conn(0x7f6b6c06cd40 msgr2=0x7f6b6c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 -- 172.21.15.33:0/3908347096 shutdown_connections 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.624+0000 7f6b6b59e700 1 -- 172.21.15.33:0/3908347096 wait complete. 2023-10-06T05:29:42.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.626+0000 7f6b6b59e700 1 Processor -- start 2023-10-06T05:29:42.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.626+0000 7f6b6b59e700 1 -- start start 2023-10-06T05:29:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:42.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b6c0ad830 0x7f6b6c0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ca440 con 0x7f6b6c157320 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c15a020 con 0x7f6b6c0ad830 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0c79a0 con 0x7f6b6c06ee20 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c0c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45768/0 (socket says 172.21.15.33:45768) 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6ad9d700 1 -- 172.21.15.33:0/1360253317 learned_addr learned my addr 172.21.15.33:0/1360253317 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b69d9b700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b6c0ad830 0x7f6b6c0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.627+0000 7f6b6a59c700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:42.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6ad9d700 1 -- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 msgr2=0x7f6b6c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6ad9d700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6ad9d700 1 -- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b6c0ad830 msgr2=0x7f6b6c0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6ad9d700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b6c0ad830 0x7f6b6c0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6ad9d700 1 -- 172.21.15.33:0/1360253317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0094f0 con 0x7f6b6c157320 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6ad9d700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c0c9f00 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007f30 tx=0x7f6b640098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400eb90 con 0x7f6b6c157320 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6400ecf0 con 0x7f6b6c157320 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c0b1d10 con 0x7f6b6c157320 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64014670 con 0x7f6b6c157320 2023-10-06T05:29:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.628+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0c7630 con 0x7f6b6c157320 2023-10-06T05:29:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.629+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c0c7b50 con 0x7f6b6c157320 2023-10-06T05:29:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.630+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b640149e0 con 0x7f6b6c157320 2023-10-06T05:29:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.632+0000 7f6b5a7fc700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.632+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 5878+0+0 (secure 0 0 0) 0x7f6b64084c40 con 0x7f6b6c157320 2023-10-06T05:29:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.633+0000 7f6b6a59c700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:42.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.633+0000 7f6b6a59c700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0099b0 tx=0x7f6b5c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.636+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b6c0c7b50 con 0x7f6b6c157320 2023-10-06T05:29:42.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:42.843+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b6c0c7b10 con 0x7f6b6c157320 2023-10-06T05:29:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.409+0000 7f6b5a7fc700 1 -- 172.21.15.33:0/1360253317 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v217) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6b64055560 con 0x7f6b6c157320 2023-10-06T05:29:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.417+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 msgr2=0x7f6b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.417+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0099b0 tx=0x7f6b5c017000 comp rx=0 tx=0).stop 2023-10-06T05:29:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.417+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 msgr2=0x7f6b6c0c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.417+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c0c9f00 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007f30 tx=0x7f6b640098e0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 shutdown_connections 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b4c066f30 0x7f6b4c0693e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b6c157320 0x7f6b6c0c9f00 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b6c06ee20 0x7f6b6c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 --2- 172.21.15.33:0/1360253317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b6c0ad830 0x7f6b6c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 >> 172.21.15.33:0/1360253317 conn(0x7f6b6c06cd40 msgr2=0x7f6b6c068cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.418+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 shutdown_connections 2023-10-06T05:29:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.419+0000 7f6b6b59e700 1 -- 172.21.15.33:0/1360253317 wait complete. 2023-10-06T05:29:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:29:43.442 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:29:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.632+0000 7f74d739d700 1 Processor -- start 2023-10-06T05:29:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.633+0000 7f74d739d700 1 -- start start 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74d0154f20 0x7f74d00ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d739d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f74d01552f0 0x7f74d00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d739d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d01586a0 con 0x7f74d00b11e0 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d006bdf0 con 0x7f74d0154f20 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d739d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d0157ab0 con 0x7f74d01552f0 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d639b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74d0154f20 0x7f74d00ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d639b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74d0154f20 0x7f74d00ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45772/0 (socket says 172.21.15.33:45772) 2023-10-06T05:29:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.634+0000 7f74d639b700 1 -- 172.21.15.33:0/3065003503 learned_addr learned my addr 172.21.15.33:0/3065003503 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d639b700 1 -- 172.21.15.33:0/3065003503 >> v2:172.21.15.167:3300/0 conn(0x7f74d01552f0 msgr2=0x7f74d00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d5b9a700 1 --2- 172.21.15.33:0/3065003503 >> v2:172.21.15.167:3300/0 conn(0x7f74d01552f0 0x7f74d00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d639b700 1 --2- 172.21.15.33:0/3065003503 >> v2:172.21.15.167:3300/0 conn(0x7f74d01552f0 0x7f74d00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d639b700 1 -- 172.21.15.33:0/3065003503 >> v1:172.21.15.167:6789/0 conn(0x7f74d00b11e0 legacy=0x7f74d00b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d639b700 1 -- 172.21.15.33:0/3065003503 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0157d70 con 0x7f74d0154f20 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d5b9a700 1 --2- 172.21.15.33:0/3065003503 >> v2:172.21.15.167:3300/0 conn(0x7f74d01552f0 0x7f74d00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d639b700 1 --2- 172.21.15.33:0/3065003503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0154f20 0x7f74d00ae720 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f74d00b21a0 tx=0x7f74cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8281ce1227a50b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d5399700 1 -- 172.21.15.33:0/3065003503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00b820 con 0x7f74d0154f20 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d5399700 1 -- 172.21.15.33:0/3065003503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc00b980 con 0x7f74d0154f20 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.635+0000 7f74d5399700 1 -- 172.21.15.33:0/3065003503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00bd20 con 0x7f74d0154f20 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 -- 172.21.15.33:0/3065003503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0154f20 msgr2=0x7f74d00ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 --2- 172.21.15.33:0/3065003503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0154f20 0x7f74d00ae720 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f74d00b21a0 tx=0x7f74cc009250 comp rx=0 tx=0).stop 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 -- 172.21.15.33:0/3065003503 shutdown_connections 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 --2- 172.21.15.33:0/3065003503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0154f20 0x7f74d00ae720 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 --2- 172.21.15.33:0/3065003503 >> v2:172.21.15.167:3300/0 conn(0x7f74d01552f0 0x7f74d00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 -- 172.21.15.33:0/3065003503 >> 172.21.15.33:0/3065003503 conn(0x7f74d006d030 msgr2=0x7f74d00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 -- 172.21.15.33:0/3065003503 shutdown_connections 2023-10-06T05:29:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.636+0000 7f74d739d700 1 -- 172.21.15.33:0/3065003503 wait complete. 2023-10-06T05:29:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.637+0000 7f74d739d700 1 Processor -- start 2023-10-06T05:29:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.637+0000 7f74d739d700 1 -- start start 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.638+0000 7f74d739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74d00b11e0 0x7f74d016a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.638+0000 7f74d739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0154f20 0x7f74d01534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.638+0000 7f74d739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 0x7f74d006af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.638+0000 7f74d739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d016ac00 con 0x7f74d0069a30 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.638+0000 7f74d739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d006aad0 con 0x7f74d0154f20 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.638+0000 7f74d739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0153a10 con 0x7f74d00b11e0 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 0x7f74d006af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 0x7f74d006af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45786/0 (socket says 172.21.15.33:45786) 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 -- 172.21.15.33:0/2938569197 learned_addr learned my addr 172.21.15.33:0/2938569197 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d639b700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74d00b11e0 0x7f74d016a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d5b9a700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0154f20 0x7f74d01534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 -- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74d00b11e0 msgr2=0x7f74d016a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74d00b11e0 0x7f74d016a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 -- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0154f20 msgr2=0x7f74d01534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0154f20 0x7f74d01534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 -- 172.21.15.33:0/2938569197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0094f0 con 0x7f74d0069a30 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.639+0000 7f74d6b9c700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 0x7f74d006af60 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f74c800bc30 tx=0x7f74c8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.640+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800c9b0 con 0x7f74d0069a30 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.640+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c800cb50 con 0x7f74d0069a30 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.640+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8014410 con 0x7f74d0069a30 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.640+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d0069ec0 con 0x7f74d0069a30 2023-10-06T05:29:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.640+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d006a480 con 0x7f74d0069a30 2023-10-06T05:29:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.641+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d006ac60 con 0x7f74d0069a30 2023-10-06T05:29:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.642+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74c8014800 con 0x7f74d0069a30 2023-10-06T05:29:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.643+0000 7f74c67fc700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b4066f30 0x7f74b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:43.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.643+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f74c80845d0 con 0x7f74d0069a30 2023-10-06T05:29:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.643+0000 7f74d639b700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b4066f30 0x7f74b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.644+0000 7f74d639b700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b4066f30 0x7f74b40693e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f74d00b21a0 tx=0x7f74cc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:43.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.647+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74c8051890 con 0x7f74d0069a30 2023-10-06T05:29:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.848+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f74d004c380 con 0x7f74d0069a30 2023-10-06T05:29:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.852+0000 7f74c67fc700 1 -- 172.21.15.33:0/2938569197 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f74c8054eb0 con 0x7f74d0069a30 2023-10-06T05:29:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.854+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b4066f30 msgr2=0x7f74b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.854+0000 7f74d739d700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b4066f30 0x7f74b40693e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f74d00b21a0 tx=0x7f74cc016000 comp rx=0 tx=0).stop 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.854+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 msgr2=0x7f74d006af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.854+0000 7f74d739d700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 0x7f74d006af60 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f74c800bc30 tx=0x7f74c8009820 comp rx=0 tx=0).stop 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.854+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 shutdown_connections 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74b4066f30 0x7f74b40693e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74d0069a30 0x7f74d006af60 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74d00b11e0 0x7f74d016a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 --2- 172.21.15.33:0/2938569197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0154f20 0x7f74d01534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 >> 172.21.15.33:0/2938569197 conn(0x7f74d006d030 msgr2=0x7f74d00cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 shutdown_connections 2023-10-06T05:29:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:43.855+0000 7f74d739d700 1 -- 172.21.15.33:0/2938569197 wait complete. 2023-10-06T05:29:43.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_restart' 2023-10-06T05:29:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.067+0000 7fb0f09fe700 1 Processor -- start 2023-10-06T05:29:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.068+0000 7fb0f09fe700 1 -- start start 2023-10-06T05:29:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0f09fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb0ec13faa0 0x7fb0ec13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0f09fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec0c9cb0 con 0x7fb0ec0c7940 2023-10-06T05:29:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0f09fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec0ca270 con 0x7fb0ec06aad0 2023-10-06T05:29:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0f09fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec06b640 con 0x7fb0ec13faa0 2023-10-06T05:29:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0ea59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb0ec0c7940 0x7fb0ec0c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32864/0 (socket says 172.21.15.33:32864) 2023-10-06T05:29:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0ea59c700 1 -- 172.21.15.33:0/320111413 learned_addr learned my addr 172.21.15.33:0/320111413 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.069+0000 7fb0ead9d700 1 --2- 172.21.15.33:0/320111413 >> v2:172.21.15.167:3300/0 conn(0x7fb0ec13faa0 0x7fb0ec13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0e9d9b700 1 -- 172.21.15.33:0/320111413 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974655434 0 0) 0x7fb0ec0c9cb0 con 0x7fb0ec0c7940 2023-10-06T05:29:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0e9d9b700 1 -- 172.21.15.33:0/320111413 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d4003570 con 0x7fb0ec0c7940 2023-10-06T05:29:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0e9d9b700 1 -- 172.21.15.33:0/320111413 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665358525 0 0) 0x7fb0d4003570 con 0x7fb0ec0c7940 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0ead9d700 1 -- 172.21.15.33:0/320111413 >> v1:172.21.15.167:6789/0 conn(0x7fb0ec06aad0 legacy=0x7fb0ec06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0ead9d700 1 -- 172.21.15.33:0/320111413 >> v1:172.21.15.33:6789/0 conn(0x7fb0ec0c7940 legacy=0x7fb0ec0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0ead9d700 1 -- 172.21.15.33:0/320111413 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec06b840 con 0x7fb0ec13faa0 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.070+0000 7fb0ead9d700 1 --2- 172.21.15.33:0/320111413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13fe70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0098e0 tx=0x7fb0dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86bda2a07d0b79a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0e9d9b700 1 -- 172.21.15.33:0/320111413 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0001470 con 0x7fb0ec13faa0 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0e9d9b700 1 -- 172.21.15.33:0/320111413 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc011420 con 0x7fb0ec13faa0 2023-10-06T05:29:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0e9d9b700 1 -- 172.21.15.33:0/320111413 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0117c0 con 0x7fb0ec13faa0 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 -- 172.21.15.33:0/320111413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec13faa0 msgr2=0x7fb0ec13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/320111413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13fe70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0098e0 tx=0x7fb0dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 -- 172.21.15.33:0/320111413 shutdown_connections 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/320111413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13fe70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 -- 172.21.15.33:0/320111413 >> 172.21.15.33:0/320111413 conn(0x7fb0ec06d030 msgr2=0x7fb0ec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 -- 172.21.15.33:0/320111413 shutdown_connections 2023-10-06T05:29:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.071+0000 7fb0f09fe700 1 -- 172.21.15.33:0/320111413 wait complete. 2023-10-06T05:29:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.073+0000 7fb0f09fe700 1 Processor -- start 2023-10-06T05:29:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.073+0000 7fb0f09fe700 1 -- start start 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0f09fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0ec06aad0 0x7fb0ec0b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0f09fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec0c7940 0x7fb0ec0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0f09fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0f09fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec156e30 con 0x7fb0ec13faa0 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0f09fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0c64f0 con 0x7fb0ec0c7940 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0f09fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0b1000 con 0x7fb0ec06aad0 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0eb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0eb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45796/0 (socket says 172.21.15.33:45796) 2023-10-06T05:29:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.074+0000 7fb0eb59e700 1 -- 172.21.15.33:0/3345187794 learned_addr learned my addr 172.21.15.33:0/3345187794 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0ead9d700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0ec06aad0 0x7fb0ec0b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0ea59c700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec0c7940 0x7fb0ec0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0eb59e700 1 -- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0ec06aad0 msgr2=0x7fb0ec0b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0eb59e700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0ec06aad0 0x7fb0ec0b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0eb59e700 1 -- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec0c7940 msgr2=0x7fb0ec0c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0eb59e700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec0c7940 0x7fb0ec0c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0eb59e700 1 -- 172.21.15.33:0/3345187794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc009420 con 0x7fb0ec13faa0 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0eb59e700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13dfd0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c980 tx=0x7fb0e4001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400a7e0 con 0x7fb0ec13faa0 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400a960 con 0x7fb0ec13faa0 2023-10-06T05:29:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400ad20 con 0x7fb0ec13faa0 2023-10-06T05:29:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec0b12e0 con 0x7fb0ec13faa0 2023-10-06T05:29:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.075+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec13e510 con 0x7fb0ec13faa0 2023-10-06T05:29:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.076+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec0b1800 con 0x7fb0ec13faa0 2023-10-06T05:29:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.077+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0e4003030 con 0x7fb0ec13faa0 2023-10-06T05:29:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.079+0000 7fb0d2ffd700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0d8066f30 0x7fb0d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.079+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb0e4084600 con 0x7fb0ec13faa0 2023-10-06T05:29:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.079+0000 7fb0ead9d700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0d8066f30 0x7fb0d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.080+0000 7fb0ead9d700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0d8066f30 0x7fb0d80693e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0091e0 tx=0x7fb0dc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.083+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0ec0b1800 con 0x7fb0ec13faa0 2023-10-06T05:29:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.366+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_restart"]} v 0) v1 -- 0x7fb0ec0b1800 con 0x7fb0ec13faa0 2023-10-06T05:29:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.411+0000 7fb0d2ffd700 1 -- 172.21.15.33:0/3345187794 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_restart"]}]=0 v783) v1 ==== 144+0+0 (secure 0 0 0) 0x7fb0ec0b1800 con 0x7fb0ec13faa0 2023-10-06T05:29:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0d8066f30 msgr2=0x7fb0d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0d8066f30 0x7fb0d80693e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0091e0 tx=0x7fb0dc017000 comp rx=0 tx=0).stop 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 msgr2=0x7fb0ec13dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13dfd0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c980 tx=0x7fb0e4001770 comp rx=0 tx=0).stop 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 shutdown_connections 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0d8066f30 0x7fb0d80693e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0ec13faa0 0x7fb0ec13dfd0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0ec06aad0 0x7fb0ec0b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 --2- 172.21.15.33:0/3345187794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0ec0c7940 0x7fb0ec0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.413+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 >> 172.21.15.33:0/3345187794 conn(0x7fb0ec06d030 msgr2=0x7fb0ec0c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.414+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 shutdown_connections 2023-10-06T05:29:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.414+0000 7fb0f09fe700 1 -- 172.21.15.33:0/3345187794 wait complete. 2023-10-06T05:29:44.433 INFO:tasks.cephfs_test_runner:test_restart (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:29:44.433 INFO:tasks.cephfs_test_runner:Starting test: test_stale_read_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:29:44.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_read_caps' 2023-10-06T05:29:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.619+0000 7fa99119d700 1 Processor -- start 2023-10-06T05:29:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.619+0000 7fa99119d700 1 -- start start 2023-10-06T05:29:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa99119d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa98c158700 0x7fa98c0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa99119d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c158ba0 con 0x7fa98c155140 2023-10-06T05:29:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa99119d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c0ae490 con 0x7fa98c0699c0 2023-10-06T05:29:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa99119d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0c8e40 con 0x7fa98c158700 2023-10-06T05:29:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa98bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa98c155140 0x7fa98c0c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32866/0 (socket says 172.21.15.33:32866) 2023-10-06T05:29:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa98bfff700 1 -- 172.21.15.33:0/3599591402 learned_addr learned my addr 172.21.15.33:0/3599591402 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa98b7fe700 1 --2- 172.21.15.33:0/3599591402 >> v2:172.21.15.33:3300/0 conn(0x7fa98c158700 0x7fa98c0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.621+0000 7fa98affd700 1 -- 172.21.15.33:0/3599591402 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960626417 0 0) 0x7fa98c158ba0 con 0x7fa98c155140 2023-10-06T05:29:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98affd700 1 -- 172.21.15.33:0/3599591402 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa974003570 con 0x7fa98c155140 2023-10-06T05:29:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98b7fe700 1 -- 172.21.15.33:0/3599591402 >> v1:172.21.15.167:6789/0 conn(0x7fa98c0699c0 legacy=0x7fa98c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98affd700 1 -- 172.21.15.33:0/3599591402 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950007754 0 0) 0x7fa98c0ae490 con 0x7fa98c0699c0 2023-10-06T05:29:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98b7fe700 1 -- 172.21.15.33:0/3599591402 >> v1:172.21.15.33:6789/0 conn(0x7fa98c155140 legacy=0x7fa98c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98b7fe700 1 -- 172.21.15.33:0/3599591402 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0c99e0 con 0x7fa98c158700 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98b7fe700 1 --2- 172.21.15.33:0/3599591402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c0adf50 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fa980009910 tx=0x7fa9800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1570f2b8fedb2c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98affd700 1 -- 172.21.15.33:0/3599591402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000b640 con 0x7fa98c158700 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98affd700 1 -- 172.21.15.33:0/3599591402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000b7a0 con 0x7fa98c158700 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.622+0000 7fa98affd700 1 -- 172.21.15.33:0/3599591402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000bb60 con 0x7fa98c158700 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 -- 172.21.15.33:0/3599591402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 msgr2=0x7fa98c0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 --2- 172.21.15.33:0/3599591402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c0adf50 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fa980009910 tx=0x7fa9800091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 -- 172.21.15.33:0/3599591402 shutdown_connections 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 --2- 172.21.15.33:0/3599591402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c0adf50 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 -- 172.21.15.33:0/3599591402 >> 172.21.15.33:0/3599591402 conn(0x7fa98c06d030 msgr2=0x7fa98c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 -- 172.21.15.33:0/3599591402 shutdown_connections 2023-10-06T05:29:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.623+0000 7fa99119d700 1 -- 172.21.15.33:0/3599591402 wait complete. 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.624+0000 7fa99119d700 1 Processor -- start 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 -- start start 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa98c0699c0 0x7fa98c159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa98c155140 0x7fa98c0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c14b760 con 0x7fa98c158700 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c159e30 con 0x7fa98c0699c0 2023-10-06T05:29:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.625+0000 7fa99119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c15a110 con 0x7fa98c155140 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c14da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45806/0 (socket says 172.21.15.33:45806) 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 -- 172.21.15.33:0/1986568635 learned_addr learned my addr 172.21.15.33:0/1986568635 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa98bfff700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa98c0699c0 0x7fa98c159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa98b7fe700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa98c155140 0x7fa98c0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 -- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa98c155140 msgr2=0x7fa98c0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa98c155140 0x7fa98c0addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 -- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa98c0699c0 msgr2=0x7fa98c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa98c0699c0 0x7fa98c159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa99099c700 1 -- 172.21.15.33:0/1986568635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa980009420 con 0x7fa98c158700 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.626+0000 7fa98b7fe700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa98c155140 0x7fa98c0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.627+0000 7fa99099c700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c14da70 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fa984001560 tx=0x7fa984001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.627+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984003030 con 0x7fa98c158700 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.627+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98400a7e0 con 0x7fa98c158700 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.627+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c15a3f0 con 0x7fa98c158700 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.627+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98400aba0 con 0x7fa98c158700 2023-10-06T05:29:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.627+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c15a780 con 0x7fa98c158700 2023-10-06T05:29:44.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.628+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c0ae3c0 con 0x7fa98c158700 2023-10-06T05:29:44.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.629+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa984012420 con 0x7fa98c158700 2023-10-06T05:29:44.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.630+0000 7fa973fff700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa978066f30 0x7fa9780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:44.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.630+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa984004070 con 0x7fa98c158700 2023-10-06T05:29:44.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.630+0000 7fa98bfff700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa978066f30 0x7fa9780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.631+0000 7fa98bfff700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa978066f30 0x7fa9780693e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa97c007fd0 tx=0x7fa97c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.634+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa984051680 con 0x7fa98c158700 2023-10-06T05:29:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:44.917+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_read_caps"]} v 0) v1 -- 0x7fa98c04c380 con 0x7fa98c158700 2023-10-06T05:29:45.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.413+0000 7fa973fff700 1 -- 172.21.15.33:0/1986568635 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_read_caps"]}]=0 v784) v1 ==== 155+0+0 (secure 0 0 0) 0x7fa984054ca0 con 0x7fa98c158700 2023-10-06T05:29:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.415+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa978066f30 msgr2=0x7fa9780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.415+0000 7fa99119d700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa978066f30 0x7fa9780693e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa97c007fd0 tx=0x7fa97c009040 comp rx=0 tx=0).stop 2023-10-06T05:29:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.415+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 msgr2=0x7fa98c14da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.415+0000 7fa99119d700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c14da70 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fa984001560 tx=0x7fa984001770 comp rx=0 tx=0).stop 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 shutdown_connections 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa978066f30 0x7fa9780693e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa98c158700 0x7fa98c14da70 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa98c0699c0 0x7fa98c159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 --2- 172.21.15.33:0/1986568635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa98c155140 0x7fa98c0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 >> 172.21.15.33:0/1986568635 conn(0x7fa98c06d030 msgr2=0x7fa98c0c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 shutdown_connections 2023-10-06T05:29:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.416+0000 7fa99119d700 1 -- 172.21.15.33:0/1986568635 wait complete. 2023-10-06T05:29:45.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:29:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.630+0000 7fe536f9d700 1 Processor -- start 2023-10-06T05:29:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.630+0000 7fe536f9d700 1 -- start start 2023-10-06T05:29:45.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.631+0000 7fe536f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe53006c7b0 0x7fe530067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:45.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.631+0000 7fe536f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe5300a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:45.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.631+0000 7fe536f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53015d3a0 con 0x7fe53006cb80 2023-10-06T05:29:45.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.631+0000 7fe536f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5300b0470 con 0x7fe53006c7b0 2023-10-06T05:29:45.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.631+0000 7fe536f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300b0750 con 0x7fe5300aff90 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe53006c7b0 0x7fe530067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe53006c7b0 0x7fe530067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45820/0 (socket says 172.21.15.33:45820) 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 -- 172.21.15.33:0/3779290638 learned_addr learned my addr 172.21.15.33:0/3779290638 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe53579a700 1 --2- 172.21.15.33:0/3779290638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe5300a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 -- 172.21.15.33:0/3779290638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 msgr2=0x7fe5300a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 --2- 172.21.15.33:0/3779290638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe5300a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 -- 172.21.15.33:0/3779290638 >> v1:172.21.15.167:6789/0 conn(0x7fe53006cb80 legacy=0x7fe53015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 -- 172.21.15.33:0/3779290638 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300b0a30 con 0x7fe53006c7b0 2023-10-06T05:29:45.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.632+0000 7fe535f9b700 1 --2- 172.21.15.33:0/3779290638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006c7b0 0x7fe530067fb0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009990 tx=0x7fe52c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7e12051daa680c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe534f99700 1 -- 172.21.15.33:0/3779290638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c005c10 con 0x7fe53006c7b0 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe534f99700 1 -- 172.21.15.33:0/3779290638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c005d70 con 0x7fe53006c7b0 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe534f99700 1 -- 172.21.15.33:0/3779290638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00b880 con 0x7fe53006c7b0 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 -- 172.21.15.33:0/3779290638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006c7b0 msgr2=0x7fe530067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 --2- 172.21.15.33:0/3779290638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006c7b0 0x7fe530067fb0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009990 tx=0x7fe52c009220 comp rx=0 tx=0).stop 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 -- 172.21.15.33:0/3779290638 shutdown_connections 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 --2- 172.21.15.33:0/3779290638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006c7b0 0x7fe530067fb0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 --2- 172.21.15.33:0/3779290638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe5300a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 -- 172.21.15.33:0/3779290638 >> 172.21.15.33:0/3779290638 conn(0x7fe53006d670 msgr2=0x7fe53006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 -- 172.21.15.33:0/3779290638 shutdown_connections 2023-10-06T05:29:45.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.633+0000 7fe536f9d700 1 -- 172.21.15.33:0/3779290638 wait complete. 2023-10-06T05:29:45.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.635+0000 7fe536f9d700 1 Processor -- start 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.635+0000 7fe536f9d700 1 -- start start 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe536f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53006c7b0 0x7fe530142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe536f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 0x7fe53015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe536f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe53015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe536f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5301599a0 con 0x7fe53006cb80 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe536f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530159090 con 0x7fe53006c7b0 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe536f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300ae430 con 0x7fe5300aff90 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe53579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 0x7fe53015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe53579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 0x7fe53015ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45830/0 (socket says 172.21.15.33:45830) 2023-10-06T05:29:45.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe53579a700 1 -- 172.21.15.33:0/2517369241 learned_addr learned my addr 172.21.15.33:0/2517369241 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe535f9b700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53006c7b0 0x7fe530142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe53679c700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe53015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe53579a700 1 -- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 msgr2=0x7fe53015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.636+0000 7fe53579a700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe53015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe53579a700 1 -- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53006c7b0 msgr2=0x7fe530142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe53579a700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53006c7b0 0x7fe530142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe53579a700 1 -- 172.21.15.33:0/2517369241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009480 con 0x7fe53006cb80 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe53679c700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe53015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe53579a700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 0x7fe53015ba40 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fe520006b00 tx=0x7fe520008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5200125f0 con 0x7fe53006cb80 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe520012770 con 0x7fe53006cb80 2023-10-06T05:29:45.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520012b30 con 0x7fe53006cb80 2023-10-06T05:29:45.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe530159350 con 0x7fe53006cb80 2023-10-06T05:29:45.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.637+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe530143270 con 0x7fe53006cb80 2023-10-06T05:29:45.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.638+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53004c380 con 0x7fe53006cb80 2023-10-06T05:29:45.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.639+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe52001e460 con 0x7fe53006cb80 2023-10-06T05:29:45.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.640+0000 7fe525ffb700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe514066f30 0x7fe5140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:45.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.640+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe520084ae0 con 0x7fe53006cb80 2023-10-06T05:29:45.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.641+0000 7fe535f9b700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe514066f30 0x7fe5140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:45.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.641+0000 7fe535f9b700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe514066f30 0x7fe5140693e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009260 tx=0x7fe52c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:45.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.645+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe520051da0 con 0x7fe53006cb80 2023-10-06T05:29:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.848+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe5300b0d10 con 0x7fe53006cb80 2023-10-06T05:29:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.852+0000 7fe525ffb700 1 -- 172.21.15.33:0/2517369241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5200553c0 con 0x7fe53006cb80 2023-10-06T05:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe514066f30 msgr2=0x7fe5140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe514066f30 0x7fe5140693e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009260 tx=0x7fe52c016000 comp rx=0 tx=0).stop 2023-10-06T05:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 msgr2=0x7fe53015ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 0x7fe53015ba40 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fe520006b00 tx=0x7fe520008040 comp rx=0 tx=0).stop 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 shutdown_connections 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe514066f30 0x7fe5140693e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5300aff90 0x7fe53015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53006c7b0 0x7fe530142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 --2- 172.21.15.33:0/2517369241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53006cb80 0x7fe53015ba40 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.854+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 >> 172.21.15.33:0/2517369241 conn(0x7fe53006d670 msgr2=0x7fe530155c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.855+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 shutdown_connections 2023-10-06T05:29:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:45.855+0000 7fe536f9d700 1 -- 172.21.15.33:0/2517369241 wait complete. 2023-10-06T05:29:45.874 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:29:45.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:45.958 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:29:46.047 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:29:46.047 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:29:46.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.233+0000 7f5b95b9d700 1 Processor -- start 2023-10-06T05:29:46.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.233+0000 7f5b95b9d700 1 -- start start 2023-10-06T05:29:46.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b95b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b900c8050 0x7f5b9013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b95b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b9006aad0 0x7f5b9006af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b95b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b900c9cb0 con 0x7f5b9013faa0 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b95b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b900ca270 con 0x7f5b900c8050 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b95b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9006b840 con 0x7f5b9006aad0 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b900c8050 0x7f5b9013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b900c8050 0x7f5b9013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45848/0 (socket says 172.21.15.33:45848) 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 -- 172.21.15.33:0/1994009427 learned_addr learned my addr 172.21.15.33:0/1994009427 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:46.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b9539c700 1 --2- 172.21.15.33:0/1994009427 >> v2:172.21.15.167:3300/0 conn(0x7f5b9006aad0 0x7f5b9006af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 -- 172.21.15.33:0/1994009427 >> v2:172.21.15.167:3300/0 conn(0x7f5b9006aad0 msgr2=0x7f5b9006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 --2- 172.21.15.33:0/1994009427 >> v2:172.21.15.167:3300/0 conn(0x7f5b9006aad0 0x7f5b9006af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 -- 172.21.15.33:0/1994009427 >> v1:172.21.15.167:6789/0 conn(0x7f5b9013faa0 legacy=0x7f5b900c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.235+0000 7f5b8ffff700 1 -- 172.21.15.33:0/1994009427 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9006bb20 con 0x7f5b900c8050 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.236+0000 7f5b8ffff700 1 --2- 172.21.15.33:0/1994009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c8050 0x7f5b9013fe70 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f5b90156b90 tx=0x7f5b7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbca0a43ebc1436 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.236+0000 7f5b8f7fe700 1 -- 172.21.15.33:0/1994009427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c009e70 con 0x7f5b900c8050 2023-10-06T05:29:46.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.236+0000 7f5b8f7fe700 1 -- 172.21.15.33:0/1994009427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c00b860 con 0x7f5b900c8050 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.236+0000 7f5b8f7fe700 1 -- 172.21.15.33:0/1994009427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00bc00 con 0x7f5b900c8050 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.236+0000 7f5b95b9d700 1 -- 172.21.15.33:0/1994009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c8050 msgr2=0x7f5b9013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.236+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/1994009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c8050 0x7f5b9013fe70 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f5b90156b90 tx=0x7f5b7c009250 comp rx=0 tx=0).stop 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.237+0000 7f5b95b9d700 1 -- 172.21.15.33:0/1994009427 shutdown_connections 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.237+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/1994009427 >> v2:172.21.15.167:3300/0 conn(0x7f5b9006aad0 0x7f5b9006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.237+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/1994009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b900c8050 0x7f5b9013fe70 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.237+0000 7f5b95b9d700 1 -- 172.21.15.33:0/1994009427 >> 172.21.15.33:0/1994009427 conn(0x7f5b9006d030 msgr2=0x7f5b9006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.237+0000 7f5b95b9d700 1 -- 172.21.15.33:0/1994009427 shutdown_connections 2023-10-06T05:29:46.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.237+0000 7f5b95b9d700 1 -- 172.21.15.33:0/1994009427 wait complete. 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.238+0000 7f5b95b9d700 1 Processor -- start 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.238+0000 7f5b95b9d700 1 -- start start 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b95b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9006aad0 0x7f5b9016de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b95b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b900c8050 0x7f5b900c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b95b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 0x7f5b90169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b95b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b900c33a0 con 0x7f5b9013faa0 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b95b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9016a370 con 0x7f5b9006aad0 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b95b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9016a5a0 con 0x7f5b900c8050 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.239+0000 7f5b9539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 0x7f5b90169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 0x7f5b90169e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45864/0 (socket says 172.21.15.33:45864) 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 -- 172.21.15.33:0/508116205 learned_addr learned my addr 172.21.15.33:0/508116205 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:46.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b94b9b700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9006aad0 0x7f5b9016de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b8ffff700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b900c8050 0x7f5b900c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 -- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b900c8050 msgr2=0x7f5b900c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b900c8050 0x7f5b900c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 -- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9006aad0 msgr2=0x7f5b9016de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9006aad0 0x7f5b9016de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 -- 172.21.15.33:0/508116205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0094f0 con 0x7f5b9013faa0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b9539c700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 0x7f5b90169e30 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009d50 tx=0x7f5b88009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800f040 con 0x7f5b9013faa0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8800eb10 con 0x7f5b9013faa0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.240+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88014460 con 0x7f5b9013faa0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.241+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9016a880 con 0x7f5b9013faa0 2023-10-06T05:29:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.241+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9013c8c0 con 0x7f5b9013faa0 2023-10-06T05:29:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.242+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9016ada0 con 0x7f5b9013faa0 2023-10-06T05:29:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.242+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b88012070 con 0x7f5b9013faa0 2023-10-06T05:29:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.243+0000 7f5b8cff9700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.243+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5b880848a0 con 0x7f5b9013faa0 2023-10-06T05:29:46.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.244+0000 7f5b94b9b700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.244+0000 7f5b94b9b700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5b840039e0 tx=0x7f5b84007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.247+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b88051b60 con 0x7f5b9013faa0 2023-10-06T05:29:46.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.496+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b9004c380 con 0x7f5b9013faa0 2023-10-06T05:29:46.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.497+0000 7f5b8cff9700 1 -- 172.21.15.33:0/508116205 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+3542 (secure 0 0 0) 0x7f5b88055180 con 0x7f5b9013faa0 2023-10-06T05:29:46.499 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:46.499 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":171,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8177,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3835394974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3835394974},{"type":"v1","addr":"172.21.15.167:6837","nonce":3835394974}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":9628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1414007180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1414007180},{"type":"v1","addr":"172.21.15.167:6836","nonce":1414007180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10647,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1282162299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1282162299},{"type":"v1","addr":"172.21.15.33:6835","nonce":1282162299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167},{"gid":10749,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/1320628382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1320628382},{"type":"v1","addr":"172.21.15.33:6833","nonce":1320628382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":171}],"filesystems":[]} 2023-10-06T05:29:46.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 msgr2=0x7f5b740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5b840039e0 tx=0x7f5b84007ec0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 msgr2=0x7f5b90169e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 0x7f5b90169e30 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009d50 tx=0x7f5b88009740 comp rx=0 tx=0).stop 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 shutdown_connections 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5b74066f30 0x7f5b740693e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b9013faa0 0x7f5b90169e30 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b9006aad0 0x7f5b9016de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 --2- 172.21.15.33:0/508116205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b900c8050 0x7f5b900c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 >> 172.21.15.33:0/508116205 conn(0x7f5b9006d030 msgr2=0x7f5b9006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 shutdown_connections 2023-10-06T05:29:46.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.499+0000 7f5b95b9d700 1 -- 172.21.15.33:0/508116205 wait complete. 2023-10-06T05:29:46.503 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 171 2023-10-06T05:29:46.519 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:29:46.519 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:29:46.519 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:29:46.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:46.520 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:29:46.520 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:29:46.520 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:29:46.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:46.520 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:29:46.520 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:29:46.521 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:29:46.521 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:46.521 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:29:46.521 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:29:46.521 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:29:46.521 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:29:46.533 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:29:46.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:29:46.534 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:29:46.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:29:46.537 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:29:46.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:29:46.538 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:29:46.538 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:29:46.578 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:29:46.579 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:46.581 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:29:46.581 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:46.583 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:29:46.583 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:46.586 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:29:46.586 DEBUG:teuthology.parallel:result is None 2023-10-06T05:29:46.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:29:46.716 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:29:46.716 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:29:46.723 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:29:46.723 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:29:46.724 INFO:tasks.ceph.mds.d.smithi167.stderr:2023-10-06T05:29:46.722+0000 7f0b7e1fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T05:29:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.782+0000 7f42cffff700 1 Processor -- start 2023-10-06T05:29:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.782+0000 7f42cffff700 1 -- start start 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42cffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42d00af600 0x7f42d00afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42cffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d006ec40 con 0x7f42d0159a90 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42cffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d006dd80 con 0x7f42d01596c0 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42cffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00affc0 con 0x7f42d00af600 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42ce7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f42d0159a90 0x7f42d006d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32898/0 (socket says 172.21.15.33:32898) 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42ce7fc700 1 -- 172.21.15.33:0/2750015942 learned_addr learned my addr 172.21.15.33:0/2750015942 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.783+0000 7f42cf7fe700 1 --2- 172.21.15.33:0/2750015942 >> v2:172.21.15.167:3300/0 conn(0x7f42d00af600 0x7f42d00afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723334234 0 0) 0x7f42d006ec40 con 0x7f42d0159a90 2023-10-06T05:29:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b8003570 con 0x7f42d0159a90 2023-10-06T05:29:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883912698 0 0) 0x7f42d006dd80 con 0x7f42d01596c0 2023-10-06T05:29:46.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d006ec40 con 0x7f42d01596c0 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cf7fe700 1 -- 172.21.15.33:0/2750015942 >> v1:172.21.15.167:6789/0 conn(0x7f42d01596c0 legacy=0x7f42d006f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cf7fe700 1 -- 172.21.15.33:0/2750015942 >> v1:172.21.15.33:6789/0 conn(0x7f42d0159a90 legacy=0x7f42d006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cf7fe700 1 -- 172.21.15.33:0/2750015942 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d006e030 con 0x7f42d00af600 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cf7fe700 1 --2- 172.21.15.33:0/2750015942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d00af600 0x7f42d00afa80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009e60 tx=0x7f42c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6188bf732e42d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c800e070 con 0x7f42d00af600 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.784+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c800bab0 con 0x7f42d00af600 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cdffb700 1 -- 172.21.15.33:0/2750015942 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c800be70 con 0x7f42d00af600 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 -- 172.21.15.33:0/2750015942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d00af600 msgr2=0x7f42d00afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 --2- 172.21.15.33:0/2750015942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d00af600 0x7f42d00afa80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009e60 tx=0x7f42c8009780 comp rx=0 tx=0).stop 2023-10-06T05:29:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 -- 172.21.15.33:0/2750015942 shutdown_connections 2023-10-06T05:29:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 --2- 172.21.15.33:0/2750015942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d00af600 0x7f42d00afa80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 -- 172.21.15.33:0/2750015942 >> 172.21.15.33:0/2750015942 conn(0x7f42d0087fb0 msgr2=0x7f42d00883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 -- 172.21.15.33:0/2750015942 shutdown_connections 2023-10-06T05:29:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.785+0000 7f42cffff700 1 -- 172.21.15.33:0/2750015942 wait complete. 2023-10-06T05:29:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.787+0000 7f42cffff700 1 Processor -- start 2023-10-06T05:29:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.787+0000 7f42cffff700 1 -- start start 2023-10-06T05:29:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.787+0000 7f42cffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d00af600 0x7f42d0157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d01596c0 0x7f42d00cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 0x7f42d0157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d016a6b0 con 0x7f42d0159a90 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00cbc40 con 0x7f42d01596c0 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00ce5c0 con 0x7f42d00af600 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 0x7f42d0157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 0x7f42d0157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45892/0 (socket says 172.21.15.33:45892) 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 -- 172.21.15.33:0/3284347579 learned_addr learned my addr 172.21.15.33:0/3284347579 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42ceffd700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d00af600 0x7f42d0157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42ce7fc700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d01596c0 0x7f42d00cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 -- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d00af600 msgr2=0x7f42d0157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d00af600 0x7f42d0157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 -- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d01596c0 msgr2=0x7f42d00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d01596c0 0x7f42d00cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.788+0000 7f42cf7fe700 1 -- 172.21.15.33:0/3284347579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009a20 con 0x7f42d0159a90 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.789+0000 7f42cf7fe700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 0x7f42d0157740 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f42d006a970 tx=0x7f42c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.789+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8012db0 con 0x7f42d0159a90 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.789+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c8026480 con 0x7f42d0159a90 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.789+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d00cb350 con 0x7f42d0159a90 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.789+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8026840 con 0x7f42d0159a90 2023-10-06T05:29:46.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.789+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d00cb8d0 con 0x7f42d0159a90 2023-10-06T05:29:46.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.790+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d00cbdd0 con 0x7f42d0159a90 2023-10-06T05:29:46.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.791+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42c801c400 con 0x7f42d0159a90 2023-10-06T05:29:46.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.792+0000 7f42b6ffd700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42bc066f30 0x7f42bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:46.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.792+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f42c8086340 con 0x7f42d0159a90 2023-10-06T05:29:46.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.792+0000 7f42ceffd700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42bc066f30 0x7f42bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:46.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.793+0000 7f42ceffd700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42bc066f30 0x7f42bc0693e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f42c0007fd0 tx=0x7f42c0007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:46.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:46.796+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42c8053600 con 0x7f42d0159a90 2023-10-06T05:29:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.017+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f42d004c380 con 0x7f42d0159a90 2023-10-06T05:29:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.877+0000 7f42b6ffd700 1 -- 172.21.15.33:0/3284347579 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v222) v1 ==== 97+0+0 (secure 0 0 0) 0x7f42c8056c20 con 0x7f42d0159a90 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42bc066f30 msgr2=0x7f42bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42bc066f30 0x7f42bc0693e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f42c0007fd0 tx=0x7f42c0007d50 comp rx=0 tx=0).stop 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 msgr2=0x7f42d0157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 0x7f42d0157740 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f42d006a970 tx=0x7f42c8019000 comp rx=0 tx=0).stop 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 shutdown_connections 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42bc066f30 0x7f42bc0693e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0159a90 0x7f42d0157740 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d00af600 0x7f42d0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 --2- 172.21.15.33:0/3284347579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d01596c0 0x7f42d00cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 >> 172.21.15.33:0/3284347579 conn(0x7f42d0087fb0 msgr2=0x7f42d00b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 shutdown_connections 2023-10-06T05:29:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:47.881+0000 7f42cffff700 1 -- 172.21.15.33:0/3284347579 wait complete. 2023-10-06T05:29:47.884 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:29:47.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:29:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.094+0000 7fadb119d700 1 Processor -- start 2023-10-06T05:29:48.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.094+0000 7fadb119d700 1 -- start start 2023-10-06T05:29:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.095+0000 7fadb119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fadac159530 0x7fadac06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.095+0000 7fadb119d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fadac06df70 0x7fadac0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.095+0000 7fadb119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac0b0310 0x7fadac159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.095+0000 7fadb119d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac0b1e00 con 0x7fadac06df70 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.095+0000 7fadb119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac06f6b0 con 0x7fadac159530 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.095+0000 7fadb119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac06e3e0 con 0x7fadac0b0310 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fadac06df70 0x7fadac0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fadac06df70 0x7fadac0afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45902/0 (socket says 172.21.15.33:45902) 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 -- 172.21.15.33:0/4077753482 learned_addr learned my addr 172.21.15.33:0/4077753482 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadb099c700 1 --2- 172.21.15.33:0/4077753482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac0b0310 0x7fadac159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadabfff700 1 --2- 172.21.15.33:0/4077753482 >> v2:172.21.15.167:3300/0 conn(0x7fadac159530 0x7fadac06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 -- 172.21.15.33:0/4077753482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac0b0310 msgr2=0x7fadac159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 --2- 172.21.15.33:0/4077753482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac0b0310 0x7fadac159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 -- 172.21.15.33:0/4077753482 >> v2:172.21.15.167:3300/0 conn(0x7fadac159530 msgr2=0x7fadac06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 --2- 172.21.15.33:0/4077753482 >> v2:172.21.15.167:3300/0 conn(0x7fadac159530 0x7fadac06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 -- 172.21.15.33:0/4077753482 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac06f220 con 0x7fadac06df70 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.096+0000 7fadab7fe700 1 --2- 172.21.15.33:0/4077753482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac06df70 0x7fadac0afdd0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fadac0cf4d0 tx=0x7fada0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3510b06a611a3804 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadaaffd700 1 -- 172.21.15.33:0/4077753482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000b640 con 0x7fadac06df70 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadaaffd700 1 -- 172.21.15.33:0/4077753482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada000b7a0 con 0x7fadac06df70 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadaaffd700 1 -- 172.21.15.33:0/4077753482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000bb40 con 0x7fadac06df70 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 -- 172.21.15.33:0/4077753482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac06df70 msgr2=0x7fadac0afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 --2- 172.21.15.33:0/4077753482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac06df70 0x7fadac0afdd0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fadac0cf4d0 tx=0x7fada0009250 comp rx=0 tx=0).stop 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 -- 172.21.15.33:0/4077753482 shutdown_connections 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 --2- 172.21.15.33:0/4077753482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac0b0310 0x7fadac159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 --2- 172.21.15.33:0/4077753482 >> v2:172.21.15.167:3300/0 conn(0x7fadac159530 0x7fadac06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 --2- 172.21.15.33:0/4077753482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac06df70 0x7fadac0afdd0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 -- 172.21.15.33:0/4077753482 >> 172.21.15.33:0/4077753482 conn(0x7fadac06c560 msgr2=0x7fadac06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 -- 172.21.15.33:0/4077753482 shutdown_connections 2023-10-06T05:29:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.097+0000 7fadb119d700 1 -- 172.21.15.33:0/4077753482 wait complete. 2023-10-06T05:29:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.099+0000 7fadb119d700 1 Processor -- start 2023-10-06T05:29:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.099+0000 7fadb119d700 1 -- start start 2023-10-06T05:29:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac06df70 0x7fadac15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 0x7fadac1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac159530 0x7fadac0c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac15b050 con 0x7fadac0b0310 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0c9a90 con 0x7fadac159530 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac15aa00 con 0x7fadac06df70 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 0x7fadac1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 0x7fadac1533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45912/0 (socket says 172.21.15.33:45912) 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 -- 172.21.15.33:0/1584993164 learned_addr learned my addr 172.21.15.33:0/1584993164 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 -- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac06df70 msgr2=0x7fadac15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac06df70 0x7fadac15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 -- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac159530 msgr2=0x7fadac0c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb099c700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac159530 0x7fadac0c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac159530 0x7fadac0c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadab7fe700 1 -- 172.21.15.33:0/1584993164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00094f0 con 0x7fadac0b0310 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.100+0000 7fadb099c700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac159530 0x7fadac0c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.101+0000 7fadab7fe700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 0x7fadac1533a0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fada00099b0 tx=0x7fada0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.101+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000bb20 con 0x7fadac0b0310 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.101+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada0011420 con 0x7fadac0b0310 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.101+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac15ace0 con 0x7fadac0b0310 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.101+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00117c0 con 0x7fadac0b0310 2023-10-06T05:29:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.101+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac1538e0 con 0x7fadac0b0310 2023-10-06T05:29:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.102+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac04c380 con 0x7fadac0b0310 2023-10-06T05:29:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.103+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fada0011bb0 con 0x7fadac0b0310 2023-10-06T05:29:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.104+0000 7fad93fff700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.104+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fada0085e20 con 0x7fadac0b0310 2023-10-06T05:29:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.104+0000 7fadabfff700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.105+0000 7fadabfff700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009bb0 tx=0x7fad9c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.108+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada00531e0 con 0x7fadac0b0310 2023-10-06T05:29:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.390+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fadac15b190 con 0x7fadac0b0310 2023-10-06T05:29:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.391+0000 7fad93fff700 1 -- 172.21.15.33:0/1584993164 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fada0011e60 con 0x7fadac0b0310 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:48.393 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:48.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.397 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:29:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.399 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:29:48.401 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:29:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:29:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:29:48.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:29:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:29:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad98066f30 msgr2=0x7fad980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009bb0 tx=0x7fad9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 msgr2=0x7fadac1533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 0x7fadac1533a0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fada00099b0 tx=0x7fada0017000 comp rx=0 tx=0).stop 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 shutdown_connections 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadac159530 0x7fadac0c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac06df70 0x7fadac15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 --2- 172.21.15.33:0/1584993164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadac0b0310 0x7fadac1533a0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 >> 172.21.15.33:0/1584993164 conn(0x7fadac06c560 msgr2=0x7fadac156f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 shutdown_connections 2023-10-06T05:29:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.393+0000 7fadb119d700 1 -- 172.21.15.33:0/1584993164 wait complete. 2023-10-06T05:29:48.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.598+0000 7f5219b9d700 1 Processor -- start 2023-10-06T05:29:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.598+0000 7f5219b9d700 1 -- start start 2023-10-06T05:29:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.599+0000 7f5219b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f521406a0d0 0x7f52140adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.599+0000 7f5219b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5214158ba0 con 0x7f5214155140 2023-10-06T05:29:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.599+0000 7f5219b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52140ae490 con 0x7f5214158700 2023-10-06T05:29:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.599+0000 7f5219b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c8e40 con 0x7f521406a0d0 2023-10-06T05:29:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f5218b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5214155140 0x7f52140c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32914/0 (socket says 172.21.15.33:32914) 2023-10-06T05:29:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f5218b9b700 1 -- 172.21.15.33:0/458616860 learned_addr learned my addr 172.21.15.33:0/458616860 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f521939c700 1 --2- 172.21.15.33:0/458616860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f521406a0d0 0x7f52140adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097640317 0 0) 0x7f5214158ba0 con 0x7f5214155140 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5204003570 con 0x7f5214155140 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541287775 0 0) 0x7f52140ae490 con 0x7f5214158700 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5214158ba0 con 0x7f5214158700 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f521939c700 1 -- 172.21.15.33:0/458616860 >> v1:172.21.15.167:6789/0 conn(0x7f5214158700 legacy=0x7f52140699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270422289 0 0) 0x7f5204003570 con 0x7f5214155140 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.600+0000 7f521939c700 1 -- 172.21.15.33:0/458616860 >> v1:172.21.15.33:6789/0 conn(0x7f5214155140 legacy=0x7f52140c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.601+0000 7f521939c700 1 -- 172.21.15.33:0/458616860 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140c99e0 con 0x7f521406a0d0 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.601+0000 7f521939c700 1 --2- 172.21.15.33:0/458616860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f521406a0d0 0x7f52140adf50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f520c009ec0 tx=0x7f520c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978ce3a0e739622 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.601+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c00e070 con 0x7f521406a0d0 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.601+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c00b870 con 0x7f521406a0d0 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.601+0000 7f52137fe700 1 -- 172.21.15.33:0/458616860 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208001470 con 0x7f521406a0d0 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 -- 172.21.15.33:0/458616860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f521406a0d0 msgr2=0x7f52140adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 --2- 172.21.15.33:0/458616860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f521406a0d0 0x7f52140adf50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f520c009ec0 tx=0x7f520c009780 comp rx=0 tx=0).stop 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 -- 172.21.15.33:0/458616860 shutdown_connections 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 --2- 172.21.15.33:0/458616860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f521406a0d0 0x7f52140adf50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 -- 172.21.15.33:0/458616860 >> 172.21.15.33:0/458616860 conn(0x7f521406d030 msgr2=0x7f521406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 -- 172.21.15.33:0/458616860 shutdown_connections 2023-10-06T05:29:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.602+0000 7f5219b9d700 1 -- 172.21.15.33:0/458616860 wait complete. 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.603+0000 7f5219b9d700 1 Processor -- start 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 -- start start 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f521406a0d0 0x7f52141535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5214155140 0x7f52140c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 0x7f52140c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5214153b20 con 0x7f5214158700 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c4160 con 0x7f521406a0d0 2023-10-06T05:29:48.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.604+0000 7f5219b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140ca600 con 0x7f5214155140 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 0x7f52140c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 0x7f52140c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45928/0 (socket says 172.21.15.33:45928) 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 -- 172.21.15.33:0/3764890598 learned_addr learned my addr 172.21.15.33:0/3764890598 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f5213fff700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5214155140 0x7f52140c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f5218b9b700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f521406a0d0 0x7f52141535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 -- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5214155140 msgr2=0x7f52140c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5214155140 0x7f52140c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 -- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f521406a0d0 msgr2=0x7f52141535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f521406a0d0 0x7f52141535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 -- 172.21.15.33:0/3764890598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c009a20 con 0x7f5214158700 2023-10-06T05:29:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.605+0000 7f521939c700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 0x7f52140c7ab0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f520c009e90 tx=0x7f520c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.606+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c026400 con 0x7f5214158700 2023-10-06T05:29:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.606+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c0265a0 con 0x7f5214158700 2023-10-06T05:29:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.606+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52140ca8e0 con 0x7f5214158700 2023-10-06T05:29:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.606+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c026960 con 0x7f5214158700 2023-10-06T05:29:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.606+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52140caea0 con 0x7f5214158700 2023-10-06T05:29:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.607+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521404c380 con 0x7f5214158700 2023-10-06T05:29:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.608+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f520c019030 con 0x7f5214158700 2023-10-06T05:29:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.608+0000 7f5210ff9700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51f8066f30 0x7f51f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.608+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f520c0865f0 con 0x7f5214158700 2023-10-06T05:29:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.609+0000 7f5218b9b700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51f8066f30 0x7f51f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:48.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.610+0000 7f5218b9b700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51f8066f30 0x7f51f80693e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5208001200 tx=0x7f5208001190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.613+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f520c0539b0 con 0x7f5214158700 2023-10-06T05:29:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.815+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52140b2090 con 0x7f5214158700 2023-10-06T05:29:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.816+0000 7f5210ff9700 1 -- 172.21.15.33:0/3764890598 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f520c056fd0 con 0x7f5214158700 2023-10-06T05:29:48.818 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:48.818 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":222,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:47.871611+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51f8066f30 msgr2=0x7f51f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51f8066f30 0x7f51f80693e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5208001200 tx=0x7f5208001190 comp rx=0 tx=0).stop 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 msgr2=0x7f52140c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 0x7f52140c7ab0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f520c009e90 tx=0x7f520c017000 comp rx=0 tx=0).stop 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 shutdown_connections 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51f8066f30 0x7f51f80693e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5214158700 0x7f52140c7ab0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f521406a0d0 0x7f52141535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 --2- 172.21.15.33:0/3764890598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5214155140 0x7f52140c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 >> 172.21.15.33:0/3764890598 conn(0x7f521406d030 msgr2=0x7f5214154220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 shutdown_connections 2023-10-06T05:29:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:48.819+0000 7f5219b9d700 1 -- 172.21.15.33:0/3764890598 wait complete. 2023-10-06T05:29:48.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:29:48.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:29:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.028+0000 7faf98f9d700 1 Processor -- start 2023-10-06T05:29:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.028+0000 7faf98f9d700 1 -- start start 2023-10-06T05:29:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.029+0000 7faf98f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf940af910 0x7faf9415a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.029+0000 7faf98f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf940ce890 con 0x7faf940af540 2023-10-06T05:29:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.029+0000 7faf98f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9415d890 con 0x7faf940cd5e0 2023-10-06T05:29:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.029+0000 7faf98f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf940afd50 con 0x7faf940af910 2023-10-06T05:29:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf937fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf940af540 0x7faf940cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32918/0 (socket says 172.21.15.33:32918) 2023-10-06T05:29:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf937fe700 1 -- 172.21.15.33:0/4125866720 learned_addr learned my addr 172.21.15.33:0/4125866720 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf93fff700 1 --2- 172.21.15.33:0/4125866720 >> v2:172.21.15.167:3300/0 conn(0x7faf940af910 0x7faf9415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf927fc700 1 -- 172.21.15.33:0/4125866720 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685888648 0 0) 0x7faf940ce890 con 0x7faf940af540 2023-10-06T05:29:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf927fc700 1 -- 172.21.15.33:0/4125866720 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf7c003570 con 0x7faf940af540 2023-10-06T05:29:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf93fff700 1 -- 172.21.15.33:0/4125866720 >> v1:172.21.15.167:6789/0 conn(0x7faf940cd5e0 legacy=0x7faf940ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf93fff700 1 -- 172.21.15.33:0/4125866720 >> v1:172.21.15.33:6789/0 conn(0x7faf940af540 legacy=0x7faf940cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf927fc700 1 -- 172.21.15.33:0/4125866720 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017677070 0 0) 0x7faf7c003570 con 0x7faf940af540 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.030+0000 7faf93fff700 1 -- 172.21.15.33:0/4125866720 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9415db70 con 0x7faf940af910 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.031+0000 7faf93fff700 1 --2- 172.21.15.33:0/4125866720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9415a650 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009e60 tx=0x7faf8c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5413826949610bd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.031+0000 7faf927fc700 1 -- 172.21.15.33:0/4125866720 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00e070 con 0x7faf940af910 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.031+0000 7faf927fc700 1 -- 172.21.15.33:0/4125866720 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c00b850 con 0x7faf940af910 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.031+0000 7faf927fc700 1 -- 172.21.15.33:0/4125866720 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84001470 con 0x7faf940af910 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 -- 172.21.15.33:0/4125866720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 msgr2=0x7faf9415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 --2- 172.21.15.33:0/4125866720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9415a650 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009e60 tx=0x7faf8c009780 comp rx=0 tx=0).stop 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 -- 172.21.15.33:0/4125866720 shutdown_connections 2023-10-06T05:29:49.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 --2- 172.21.15.33:0/4125866720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9415a650 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 -- 172.21.15.33:0/4125866720 >> 172.21.15.33:0/4125866720 conn(0x7faf9406d9b0 msgr2=0x7faf9406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:49.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 -- 172.21.15.33:0/4125866720 shutdown_connections 2023-10-06T05:29:49.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.032+0000 7faf98f9d700 1 -- 172.21.15.33:0/4125866720 wait complete. 2023-10-06T05:29:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.033+0000 7faf98f9d700 1 Processor -- start 2023-10-06T05:29:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.033+0000 7faf98f9d700 1 -- start start 2023-10-06T05:29:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.034+0000 7faf98f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 0x7faf940c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.034+0000 7faf98f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.034+0000 7faf98f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf940cd5e0 0x7faf9416d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.034+0000 7faf98f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf940ca6e0 con 0x7faf940af540 2023-10-06T05:29:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.034+0000 7faf98f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf940c7ac0 con 0x7faf940af910 2023-10-06T05:29:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.034+0000 7faf98f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9414b550 con 0x7faf940cd5e0 2023-10-06T05:29:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 0x7faf940c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 0x7faf940c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45930/0 (socket says 172.21.15.33:45930) 2023-10-06T05:29:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 -- 172.21.15.33:0/2597767574 learned_addr learned my addr 172.21.15.33:0/2597767574 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 -- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf940cd5e0 msgr2=0x7faf9416d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf93fff700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf940cd5e0 0x7faf9416d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf940cd5e0 0x7faf9416d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 -- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 msgr2=0x7faf9414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 -- 172.21.15.33:0/2597767574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009a20 con 0x7faf940af540 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf93fff700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf940cd5e0 0x7faf9416d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.035+0000 7faf937fe700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 0x7faf940c7580 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7faf8400c820 tx=0x7faf840019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.036+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84007360 con 0x7faf940af540 2023-10-06T05:29:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.036+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf840074c0 con 0x7faf940af540 2023-10-06T05:29:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.036+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84004830 con 0x7faf940af540 2023-10-06T05:29:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.036+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf940b0260 con 0x7faf940af540 2023-10-06T05:29:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.036+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf940ca120 con 0x7faf940af540 2023-10-06T05:29:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.037+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf940cd360 con 0x7faf940af540 2023-10-06T05:29:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.038+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf84004c20 con 0x7faf940af540 2023-10-06T05:29:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.039+0000 7faf7b7fe700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf80066f60 0x7faf80069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.039+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 5206+0+0 (secure 0 0 0) 0x7faf84083a40 con 0x7faf940af540 2023-10-06T05:29:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.039+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf80066f60 0x7faf80069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.040+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf80066f60 0x7faf80069410 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7faf940c8d60 tx=0x7faf88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.043+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf8404c5a0 con 0x7faf940af540 2023-10-06T05:29:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.244+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf9404c380 con 0x7faf940af540 2023-10-06T05:29:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.245+0000 7faf7b7fe700 1 -- 172.21.15.33:0/2597767574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v222) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf84018070 con 0x7faf940af540 2023-10-06T05:29:49.247 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:29:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf80066f60 msgr2=0x7faf80069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf80066f60 0x7faf80069410 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7faf940c8d60 tx=0x7faf88009040 comp rx=0 tx=0).stop 2023-10-06T05:29:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 msgr2=0x7faf940c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 0x7faf940c7580 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7faf8400c820 tx=0x7faf840019e0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 shutdown_connections 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faf80066f60 0x7faf80069410 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf940cd5e0 0x7faf9416d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf940af540 0x7faf940c7580 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 --2- 172.21.15.33:0/2597767574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf940af910 0x7faf9414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 >> 172.21.15.33:0/2597767574 conn(0x7faf9406d9b0 msgr2=0x7faf9406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 shutdown_connections 2023-10-06T05:29:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.247+0000 7faf98f9d700 1 -- 172.21.15.33:0/2597767574 wait complete. 2023-10-06T05:29:49.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:29:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.455+0000 7facee79d700 1 Processor -- start 2023-10-06T05:29:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.455+0000 7facee79d700 1 -- start start 2023-10-06T05:29:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7facee79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7face80af540 0x7face80af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7facee79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80ce890 con 0x7face815a650 2023-10-06T05:29:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7facee79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face815aac0 con 0x7face80cd5e0 2023-10-06T05:29:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7facee79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face815d890 con 0x7face80af540 2023-10-06T05:29:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7faced79b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7face80af540 0x7face80af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7facecf9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7face815a650 0x7face80cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:32926/0 (socket says 172.21.15.33:32926) 2023-10-06T05:29:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.456+0000 7facecf9a700 1 -- 172.21.15.33:0/1062592599 learned_addr learned my addr 172.21.15.33:0/1062592599 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7facdffff700 1 -- 172.21.15.33:0/1062592599 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460863857 0 0) 0x7face80ce890 con 0x7face815a650 2023-10-06T05:29:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7facdffff700 1 -- 172.21.15.33:0/1062592599 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd4003570 con 0x7face815a650 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7faced79b700 1 -- 172.21.15.33:0/1062592599 >> v1:172.21.15.167:6789/0 conn(0x7face80cd5e0 legacy=0x7face80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7faced79b700 1 -- 172.21.15.33:0/1062592599 >> v1:172.21.15.33:6789/0 conn(0x7face815a650 legacy=0x7face80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7faced79b700 1 -- 172.21.15.33:0/1062592599 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face815db70 con 0x7face80af540 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7faced79b700 1 --2- 172.21.15.33:0/1062592599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face80af540 0x7face80af910 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7face40098e0 tx=0x7face40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7783b33770c8f761 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.457+0000 7facdffff700 1 -- 172.21.15.33:0/1062592599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4011630 con 0x7face80af540 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facdffff700 1 -- 172.21.15.33:0/1062592599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4011790 con 0x7face80af540 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facdffff700 1 -- 172.21.15.33:0/1062592599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4011b30 con 0x7face80af540 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 -- 172.21.15.33:0/1062592599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face80af540 msgr2=0x7face80af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 --2- 172.21.15.33:0/1062592599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face80af540 0x7face80af910 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7face40098e0 tx=0x7face40091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 -- 172.21.15.33:0/1062592599 shutdown_connections 2023-10-06T05:29:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 --2- 172.21.15.33:0/1062592599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face80af540 0x7face80af910 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 -- 172.21.15.33:0/1062592599 >> 172.21.15.33:0/1062592599 conn(0x7face806d9b0 msgr2=0x7face806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 -- 172.21.15.33:0/1062592599 shutdown_connections 2023-10-06T05:29:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.458+0000 7facee79d700 1 -- 172.21.15.33:0/1062592599 wait complete. 2023-10-06T05:29:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.460+0000 7facee79d700 1 Processor -- start 2023-10-06T05:29:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.460+0000 7facee79d700 1 -- start start 2023-10-06T05:29:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facee79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7face80af540 0x7face80c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facee79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7face80cd5e0 0x7face80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facee79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 0x7face80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facee79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c87f0 con 0x7face815a650 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facee79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c7580 con 0x7face80cd5e0 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facee79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80c92f0 con 0x7face80af540 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 0x7face80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 0x7face80ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45944/0 (socket says 172.21.15.33:45944) 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 -- 172.21.15.33:0/1852498374 learned_addr learned my addr 172.21.15.33:0/1852498374 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7faced79b700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7face80af540 0x7face80c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facecf9a700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7face80cd5e0 0x7face80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 -- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7face80af540 msgr2=0x7face80c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7face80af540 0x7face80c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 -- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7face80cd5e0 msgr2=0x7face80c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7face80cd5e0 0x7face80c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7facedf9c700 1 -- 172.21.15.33:0/1852498374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009420 con 0x7face815a650 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.461+0000 7faced79b700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7face80af540 0x7face80c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.462+0000 7facedf9c700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 0x7face80ae410 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7face000c890 tx=0x7face00026a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.462+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0012630 con 0x7face815a650 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.462+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0012790 con 0x7face815a650 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.462+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0012b50 con 0x7face815a650 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.462+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face80c77d0 con 0x7face815a650 2023-10-06T05:29:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.462+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face80c7d90 con 0x7face815a650 2023-10-06T05:29:49.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.463+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face80c7fb0 con 0x7face815a650 2023-10-06T05:29:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.464+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7face000a420 con 0x7face815a650 2023-10-06T05:29:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.465+0000 7facdd7fa700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faccc066f30 0x7faccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.465+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 5206+0+0 (secure 0 0 0) 0x7face0084710 con 0x7face815a650 2023-10-06T05:29:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.465+0000 7faced79b700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faccc066f30 0x7faccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.466+0000 7faced79b700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faccc066f30 0x7faccc0693e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7face40091e0 tx=0x7face4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.470+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7face0051ad0 con 0x7face815a650 2023-10-06T05:29:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.669+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7face804c380 con 0x7face815a650 2023-10-06T05:29:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.669+0000 7facdd7fa700 1 -- 172.21.15.33:0/1852498374 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v222) v1 ==== 92+0+10 (secure 0 0 0) 0x7face00550f0 con 0x7face815a650 2023-10-06T05:29:49.671 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.671+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faccc066f30 msgr2=0x7faccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.671+0000 7facee79d700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faccc066f30 0x7faccc0693e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7face40091e0 tx=0x7face4016000 comp rx=0 tx=0).stop 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 msgr2=0x7face80ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 0x7face80ae410 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7face000c890 tx=0x7face00026a0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 shutdown_connections 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faccc066f30 0x7faccc0693e0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7face815a650 0x7face80ae410 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7face80af540 0x7face80c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 --2- 172.21.15.33:0/1852498374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7face80cd5e0 0x7face80c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 >> 172.21.15.33:0/1852498374 conn(0x7face806d9b0 msgr2=0x7face80cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:49.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 shutdown_connections 2023-10-06T05:29:49.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.672+0000 7facee79d700 1 -- 172.21.15.33:0/1852498374 wait complete. 2023-10-06T05:29:49.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:29:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.876+0000 7f242759e700 1 Processor -- start 2023-10-06T05:29:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.877+0000 7f242759e700 1 -- start start 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f242759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2428155140 0x7f2428158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f242759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280699c0 0x7f2428069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f242759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280c8e40 con 0x7f24280adf50 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f242759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242806a380 con 0x7f2428155140 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f242759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c99e0 con 0x7f24280699c0 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f2425d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f24280adf50 0x7f24280c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56112/0 (socket says 172.21.15.33:56112) 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f2425d9b700 1 -- 172.21.15.33:0/2565311983 learned_addr learned my addr 172.21.15.33:0/2565311983 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f242659c700 1 --2- 172.21.15.33:0/2565311983 >> v2:172.21.15.167:3300/0 conn(0x7f2428155140 0x7f2428158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.878+0000 7f2426d9d700 1 --2- 172.21.15.33:0/2565311983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280699c0 0x7f2428069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f242559a700 1 -- 172.21.15.33:0/2565311983 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909881457 0 0) 0x7f24280c8e40 con 0x7f24280adf50 2023-10-06T05:29:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f242559a700 1 -- 172.21.15.33:0/2565311983 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2410003570 con 0x7f24280adf50 2023-10-06T05:29:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f242559a700 1 -- 172.21.15.33:0/2565311983 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 764804446 0 0) 0x7f2410003570 con 0x7f24280adf50 2023-10-06T05:29:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f2426d9d700 1 -- 172.21.15.33:0/2565311983 >> v2:172.21.15.167:3300/0 conn(0x7f2428155140 msgr2=0x7f2428158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f2426d9d700 1 --2- 172.21.15.33:0/2565311983 >> v2:172.21.15.167:3300/0 conn(0x7f2428155140 0x7f2428158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f2426d9d700 1 -- 172.21.15.33:0/2565311983 >> v1:172.21.15.33:6789/0 conn(0x7f24280adf50 legacy=0x7f24280c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f2426d9d700 1 -- 172.21.15.33:0/2565311983 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280ae3f0 con 0x7f24280699c0 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.879+0000 7f2426d9d700 1 --2- 172.21.15.33:0/2565311983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280699c0 0x7f2428069e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2420009e90 tx=0x7f2420009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed484a7995057926 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242559a700 1 -- 172.21.15.33:0/2565311983 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000e070 con 0x7f24280699c0 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242559a700 1 -- 172.21.15.33:0/2565311983 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242000b870 con 0x7f24280699c0 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242559a700 1 -- 172.21.15.33:0/2565311983 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c001470 con 0x7f24280699c0 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 -- 172.21.15.33:0/2565311983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280699c0 msgr2=0x7f2428069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 --2- 172.21.15.33:0/2565311983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280699c0 0x7f2428069e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2420009e90 tx=0x7f2420009780 comp rx=0 tx=0).stop 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 -- 172.21.15.33:0/2565311983 shutdown_connections 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 --2- 172.21.15.33:0/2565311983 >> v2:172.21.15.167:3300/0 conn(0x7f2428155140 0x7f2428158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 --2- 172.21.15.33:0/2565311983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280699c0 0x7f2428069e40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 -- 172.21.15.33:0/2565311983 >> 172.21.15.33:0/2565311983 conn(0x7f242806d030 msgr2=0x7f242806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 -- 172.21.15.33:0/2565311983 shutdown_connections 2023-10-06T05:29:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.880+0000 7f242759e700 1 -- 172.21.15.33:0/2565311983 wait complete. 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.882+0000 7f242759e700 1 Processor -- start 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.882+0000 7f242759e700 1 -- start start 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24280699c0 0x7f24280c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280adf50 0x7f24280ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 0x7f2428147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242816dfc0 con 0x7f2428155140 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242816d440 con 0x7f24280699c0 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242816d720 con 0x7f24280adf50 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 0x7f2428147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 0x7f2428147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49906/0 (socket says 172.21.15.33:49906) 2023-10-06T05:29:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 -- 172.21.15.33:0/4009395128 learned_addr learned my addr 172.21.15.33:0/4009395128 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2425d9b700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280adf50 0x7f24280ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f242659c700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24280699c0 0x7f24280c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 -- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280adf50 msgr2=0x7f24280ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280adf50 0x7f24280ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 -- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24280699c0 msgr2=0x7f24280c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24280699c0 0x7f24280c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.883+0000 7f2426d9d700 1 -- 172.21.15.33:0/4009395128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420009a20 con 0x7f2428155140 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.884+0000 7f2426d9d700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 0x7f2428147ca0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f24200097c0 tx=0x7f2420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.884+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420012c20 con 0x7f2428155140 2023-10-06T05:29:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.884+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2420012d80 con 0x7f2428155140 2023-10-06T05:29:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.884+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242001e6a0 con 0x7f2428155140 2023-10-06T05:29:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.884+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242816da00 con 0x7f2428155140 2023-10-06T05:29:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.884+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280ca920 con 0x7f2428155140 2023-10-06T05:29:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.885+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242804c380 con 0x7f2428155140 2023-10-06T05:29:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.886+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f242001ea50 con 0x7f2428155140 2023-10-06T05:29:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.887+0000 7f24167fc700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2408066f30 0x7f24080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.887+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2420086100 con 0x7f2428155140 2023-10-06T05:29:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.887+0000 7f242659c700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2408066f30 0x7f24080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.888+0000 7f242659c700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2408066f30 0x7f24080693e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2418009db0 tx=0x7f24180091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:49.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:49.892+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2420053580 con 0x7f2428155140 2023-10-06T05:29:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.099+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f242804bb40 con 0x7f2428155140 2023-10-06T05:29:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.100+0000 7f24167fc700 1 -- 172.21.15.33:0/4009395128 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v222) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2420056ba0 con 0x7f2428155140 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:29:50.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:29:50.103 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:29:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2408066f30 msgr2=0x7f24080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2408066f30 0x7f24080693e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2418009db0 tx=0x7f24180091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 msgr2=0x7f2428147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 0x7f2428147ca0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f24200097c0 tx=0x7f2420017000 comp rx=0 tx=0).stop 2023-10-06T05:29:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 shutdown_connections 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2408066f30 0x7f24080693e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2428155140 0x7f2428147ca0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24280699c0 0x7f24280c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 --2- 172.21.15.33:0/4009395128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280adf50 0x7f24280ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 >> 172.21.15.33:0/4009395128 conn(0x7f242806d030 msgr2=0x7f24280c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 shutdown_connections 2023-10-06T05:29:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.102+0000 7f242759e700 1 -- 172.21.15.33:0/4009395128 wait complete. 2023-10-06T05:29:50.121 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:29:50.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:29:50.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.311+0000 7f8abb59e700 1 Processor -- start 2023-10-06T05:29:50.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.311+0000 7f8abb59e700 1 -- start start 2023-10-06T05:29:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.312+0000 7f8abb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8abc06dfd0 0x7f8abc15a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.312+0000 7f8abb59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc15d280 con 0x7f8abc06dc00 2023-10-06T05:29:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.312+0000 7f8abb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc15de00 con 0x7f8abc1597f0 2023-10-06T05:29:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.312+0000 7f8abb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc15e290 con 0x7f8abc06dfd0 2023-10-06T05:29:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.312+0000 7f8aba59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8abc06dc00 0x7f8abc1590e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56120/0 (socket says 172.21.15.33:56120) 2023-10-06T05:29:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.312+0000 7f8aba59c700 1 -- 172.21.15.33:0/61836775 learned_addr learned my addr 172.21.15.33:0/61836775 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8abad9d700 1 --2- 172.21.15.33:0/61836775 >> v2:172.21.15.167:3300/0 conn(0x7f8abc06dfd0 0x7f8abc15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8ab959a700 1 -- 172.21.15.33:0/61836775 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892873820 0 0) 0x7f8abc15d280 con 0x7f8abc06dc00 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8ab959a700 1 -- 172.21.15.33:0/61836775 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa4003570 con 0x7f8abc06dc00 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8abad9d700 1 -- 172.21.15.33:0/61836775 >> v1:172.21.15.167:6789/0 conn(0x7f8abc1597f0 legacy=0x7f8abc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8abad9d700 1 -- 172.21.15.33:0/61836775 >> v1:172.21.15.33:6789/0 conn(0x7f8abc06dc00 legacy=0x7f8abc1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8abad9d700 1 -- 172.21.15.33:0/61836775 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc15d4d0 con 0x7f8abc06dfd0 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.313+0000 7f8abad9d700 1 --2- 172.21.15.33:0/61836775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc06dfd0 0x7f8abc15a740 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009e60 tx=0x7f8ab4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a21ce869a74bde79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8ab959a700 1 -- 172.21.15.33:0/61836775 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400e070 con 0x7f8abc06dfd0 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8ab959a700 1 -- 172.21.15.33:0/61836775 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400bb60 con 0x7f8abc06dfd0 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8ab959a700 1 -- 172.21.15.33:0/61836775 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4012430 con 0x7f8abc06dfd0 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 -- 172.21.15.33:0/61836775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc06dfd0 msgr2=0x7f8abc15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 --2- 172.21.15.33:0/61836775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc06dfd0 0x7f8abc15a740 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009e60 tx=0x7f8ab4009780 comp rx=0 tx=0).stop 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 -- 172.21.15.33:0/61836775 shutdown_connections 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 --2- 172.21.15.33:0/61836775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc06dfd0 0x7f8abc15a740 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 -- 172.21.15.33:0/61836775 >> 172.21.15.33:0/61836775 conn(0x7f8abc0d0550 msgr2=0x7f8abc0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 -- 172.21.15.33:0/61836775 shutdown_connections 2023-10-06T05:29:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.314+0000 7f8abb59e700 1 -- 172.21.15.33:0/61836775 wait complete. 2023-10-06T05:29:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.316+0000 7f8abb59e700 1 Processor -- start 2023-10-06T05:29:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.316+0000 7f8abb59e700 1 -- start start 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 0x7f8abc140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8abc06dfd0 0x7f8abc15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc1597f0 0x7f8abc0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0b0bb0 con 0x7f8abc06dc00 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15b8c0 con 0x7f8abc1597f0 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15bba0 con 0x7f8abc06dfd0 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 0x7f8abc140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 0x7f8abc140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49912/0 (socket says 172.21.15.33:49912) 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 -- 172.21.15.33:0/457549784 learned_addr learned my addr 172.21.15.33:0/457549784 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:50.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abad9d700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc1597f0 0x7f8abc0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8ab9d9b700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8abc06dfd0 0x7f8abc15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 -- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8abc06dfd0 msgr2=0x7f8abc15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8abc06dfd0 0x7f8abc15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 -- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc1597f0 msgr2=0x7f8abc0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc1597f0 0x7f8abc0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8aba59c700 1 -- 172.21.15.33:0/457549784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009a20 con 0x7f8abc06dc00 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8abad9d700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc1597f0 0x7f8abc0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.317+0000 7f8ab9d9b700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8abc06dfd0 0x7f8abc15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.318+0000 7f8aba59c700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 0x7f8abc140dc0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c7c0 tx=0x7f8aac007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.318+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac005b30 con 0x7f8abc06dc00 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.318+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005c90 con 0x7f8abc06dc00 2023-10-06T05:29:50.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.318+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc15be80 con 0x7f8abc06dc00 2023-10-06T05:29:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.318+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007340 con 0x7f8abc06dc00 2023-10-06T05:29:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.318+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc15cd10 con 0x7f8abc06dc00 2023-10-06T05:29:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.319+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc063ff0 con 0x7f8abc06dc00 2023-10-06T05:29:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.320+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8aac01b400 con 0x7f8abc06dc00 2023-10-06T05:29:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.321+0000 7f8aaa7fc700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a9c066f30 0x7f8a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.321+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8aac083f50 con 0x7f8abc06dc00 2023-10-06T05:29:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.321+0000 7f8ab9d9b700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a9c066f30 0x7f8a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.322+0000 7f8ab9d9b700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a9c066f30 0x7f8a9c0693e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007fd0 tx=0x7f8ab0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.325+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8aac051310 con 0x7f8abc06dc00 2023-10-06T05:29:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:50.528+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8abc15d7b0 con 0x7f8abc06dc00 2023-10-06T05:29:50.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:29:50.562 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:29:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.443+0000 7f8aaa7fc700 1 -- 172.21.15.33:0/457549784 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v223) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8aac054930 con 0x7f8abc06dc00 2023-10-06T05:29:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.449+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a9c066f30 msgr2=0x7f8a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.449+0000 7f8abb59e700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a9c066f30 0x7f8a9c0693e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007fd0 tx=0x7f8ab0009040 comp rx=0 tx=0).stop 2023-10-06T05:29:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.449+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 msgr2=0x7f8abc140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.449+0000 7f8abb59e700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 0x7f8abc140dc0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c7c0 tx=0x7f8aac007e00 comp rx=0 tx=0).stop 2023-10-06T05:29:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.450+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 shutdown_connections 2023-10-06T05:29:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.450+0000 7f8abb59e700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8a9c066f30 0x7f8a9c0693e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.450+0000 7f8abb59e700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8abc1597f0 0x7f8abc0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.450+0000 7f8abb59e700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8abc06dc00 0x7f8abc140dc0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.450+0000 7f8abb59e700 1 --2- 172.21.15.33:0/457549784 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8abc06dfd0 0x7f8abc15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.450+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 >> 172.21.15.33:0/457549784 conn(0x7f8abc0d0550 msgr2=0x7f8abc0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.451+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 shutdown_connections 2023-10-06T05:29:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.452+0000 7f8abb59e700 1 -- 172.21.15.33:0/457549784 wait complete. 2023-10-06T05:29:51.456 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:29:51.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:29:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.673+0000 7f92ba59d700 1 Processor -- start 2023-10-06T05:29:51.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.673+0000 7f92ba59d700 1 -- start start 2023-10-06T05:29:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.674+0000 7f92ba59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f92b40c95c0 0x7f92b406aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.674+0000 7f92ba59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92b4159890 0x7f92b40cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.674+0000 7f92ba59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b415dbe0 con 0x7f92b41594c0 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.674+0000 7f92ba59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40c9e30 con 0x7f92b4159890 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.674+0000 7f92ba59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40ca110 con 0x7f92b40c95c0 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92b4159890 0x7f92b40cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92b4159890 0x7f92b40cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49928/0 (socket says 172.21.15.33:49928) 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 -- 172.21.15.33:0/3196381474 learned_addr learned my addr 172.21.15.33:0/3196381474 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b959b700 1 --1- 172.21.15.33:0/3196381474 >> v1:172.21.15.33:6789/0 conn(0x7f92b41594c0 0x7f92b406a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56128/0 (socket says 172.21.15.33:56128) 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b8d9a700 1 --2- 172.21.15.33:0/3196381474 >> v2:172.21.15.167:3300/0 conn(0x7f92b40c95c0 0x7f92b406aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 -- 172.21.15.33:0/3196381474 >> v2:172.21.15.167:3300/0 conn(0x7f92b40c95c0 msgr2=0x7f92b406aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 --2- 172.21.15.33:0/3196381474 >> v2:172.21.15.167:3300/0 conn(0x7f92b40c95c0 0x7f92b406aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 -- 172.21.15.33:0/3196381474 >> v1:172.21.15.33:6789/0 conn(0x7f92b41594c0 legacy=0x7f92b406a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:51.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 -- 172.21.15.33:0/3196381474 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b415de30 con 0x7f92b4159890 2023-10-06T05:29:51.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.675+0000 7f92b9d9c700 1 --2- 172.21.15.33:0/3196381474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4159890 0x7f92b40cb780 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f92b40cd9c0 tx=0x7f92ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7f102dee02f699 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:51.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92abfff700 1 -- 172.21.15.33:0/3196381474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00e070 con 0x7f92b4159890 2023-10-06T05:29:51.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92abfff700 1 -- 172.21.15.33:0/3196381474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac00ba20 con 0x7f92b4159890 2023-10-06T05:29:51.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92abfff700 1 -- 172.21.15.33:0/3196381474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0092e0 con 0x7f92b4159890 2023-10-06T05:29:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 -- 172.21.15.33:0/3196381474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4159890 msgr2=0x7f92b40cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 --2- 172.21.15.33:0/3196381474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4159890 0x7f92b40cb780 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f92b40cd9c0 tx=0x7f92ac009840 comp rx=0 tx=0).stop 2023-10-06T05:29:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 -- 172.21.15.33:0/3196381474 shutdown_connections 2023-10-06T05:29:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 --2- 172.21.15.33:0/3196381474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b4159890 0x7f92b40cb780 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 --2- 172.21.15.33:0/3196381474 >> v2:172.21.15.167:3300/0 conn(0x7f92b40c95c0 0x7f92b406aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 -- 172.21.15.33:0/3196381474 >> 172.21.15.33:0/3196381474 conn(0x7f92b40cd3b0 msgr2=0x7f92b40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 -- 172.21.15.33:0/3196381474 shutdown_connections 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.676+0000 7f92ba59d700 1 -- 172.21.15.33:0/3196381474 wait complete. 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.678+0000 7f92ba59d700 1 Processor -- start 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.678+0000 7f92ba59d700 1 -- start start 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92ba59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 0x7f92b414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92ba59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92b41594c0 0x7f92b40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92ba59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b4159890 0x7f92b40cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92ba59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4150fa0 con 0x7f92b40c95c0 2023-10-06T05:29:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92ba59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4150b10 con 0x7f92b4159890 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92ba59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41501e0 con 0x7f92b41594c0 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 0x7f92b414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b959b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 0x7f92b414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49938/0 (socket says 172.21.15.33:49938) 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b959b700 1 -- 172.21.15.33:0/1909213332 learned_addr learned my addr 172.21.15.33:0/1909213332 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b959b700 1 -- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92b41594c0 msgr2=0x7f92b40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b8d9a700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92b41594c0 0x7f92b40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b9d9c700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b4159890 0x7f92b40cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.679+0000 7f92b959b700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92b41594c0 0x7f92b40cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92b959b700 1 -- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b4159890 msgr2=0x7f92b40cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92b959b700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b4159890 0x7f92b40cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92b959b700 1 -- 172.21.15.33:0/1909213332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac009ae0 con 0x7f92b40c95c0 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92b8d9a700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92b41594c0 0x7f92b40cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92b9d9c700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b4159890 0x7f92b40cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92b959b700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 0x7f92b414d1b0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c7c0 tx=0x7f92b0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0005a70 con 0x7f92b40c95c0 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0005bf0 con 0x7f92b40c95c0 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b00070f0 con 0x7f92b40c95c0 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b41504c0 con 0x7f92b40c95c0 2023-10-06T05:29:51.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.680+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b414fe90 con 0x7f92b40c95c0 2023-10-06T05:29:51.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.681+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b404c380 con 0x7f92b40c95c0 2023-10-06T05:29:51.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.682+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92b0007460 con 0x7f92b40c95c0 2023-10-06T05:29:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.683+0000 7f92a97fa700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9298066f30 0x7f92980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.683+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f92b0083fd0 con 0x7f92b40c95c0 2023-10-06T05:29:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.684+0000 7f92b8d9a700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9298066f30 0x7f92980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.684+0000 7f92b8d9a700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9298066f30 0x7f92980693e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006f00 tx=0x7f92a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.688+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92b00512a0 con 0x7f92b40c95c0 2023-10-06T05:29:51.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:51.892+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f92b4150960 con 0x7f92b40c95c0 2023-10-06T05:29:52.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.456+0000 7f92a97fa700 1 -- 172.21.15.33:0/1909213332 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v224) v1 ==== 170+0+0 (secure 0 0 0) 0x7f92b00548c0 con 0x7f92b40c95c0 2023-10-06T05:29:52.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.463+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9298066f30 msgr2=0x7f92980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:52.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.463+0000 7f92ba59d700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9298066f30 0x7f92980693e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006f00 tx=0x7f92a4008040 comp rx=0 tx=0).stop 2023-10-06T05:29:52.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.463+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 msgr2=0x7f92b414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:52.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.463+0000 7f92ba59d700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 0x7f92b414d1b0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c7c0 tx=0x7f92b0007f70 comp rx=0 tx=0).stop 2023-10-06T05:29:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.464+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 shutdown_connections 2023-10-06T05:29:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.464+0000 7f92ba59d700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9298066f30 0x7f92980693e0 secure :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f92a4006f00 tx=0x7f92a4008040 comp rx=0 tx=0).stop 2023-10-06T05:29:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.464+0000 7f92ba59d700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b4159890 0x7f92b40cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.464+0000 7f92ba59d700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92b40c95c0 0x7f92b414d1b0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.464+0000 7f92ba59d700 1 --2- 172.21.15.33:0/1909213332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f92b41594c0 0x7f92b40cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.465+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 >> 172.21.15.33:0/1909213332 conn(0x7f92b40cd3b0 msgr2=0x7f92b40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:52.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.465+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 shutdown_connections 2023-10-06T05:29:52.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.467+0000 7f92ba59d700 1 -- 172.21.15.33:0/1909213332 wait complete. 2023-10-06T05:29:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:29:52.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:29:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.688+0000 7f626f59d700 1 Processor -- start 2023-10-06T05:29:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.689+0000 7f626f59d700 1 -- start start 2023-10-06T05:29:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6268156370 0x7f62680af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62680cc7e0 0x7f62680c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62680ad430 con 0x7f62680b01f0 2023-10-06T05:29:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62680afe20 con 0x7f62680cc7e0 2023-10-06T05:29:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626806bc70 con 0x7f6268156370 2023-10-06T05:29:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626ed9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f62680b01f0 0x7f62680ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56134/0 (socket says 172.21.15.33:56134) 2023-10-06T05:29:52.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626ed9c700 1 -- 172.21.15.33:0/2583713727 learned_addr learned my addr 172.21.15.33:0/2583713727 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:52.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626dd9a700 1 --2- 172.21.15.33:0/2583713727 >> v2:172.21.15.167:3300/0 conn(0x7f62680cc7e0 0x7f62680c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:52.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.690+0000 7f626e59b700 1 --2- 172.21.15.33:0/2583713727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6268156370 0x7f62680af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:52.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626d599700 1 -- 172.21.15.33:0/2583713727 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648904147 0 0) 0x7f62680ad430 con 0x7f62680b01f0 2023-10-06T05:29:52.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626d599700 1 -- 172.21.15.33:0/2583713727 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6254003570 con 0x7f62680b01f0 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626e59b700 1 -- 172.21.15.33:0/2583713727 >> v2:172.21.15.167:3300/0 conn(0x7f62680cc7e0 msgr2=0x7f62680c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626e59b700 1 --2- 172.21.15.33:0/2583713727 >> v2:172.21.15.167:3300/0 conn(0x7f62680cc7e0 0x7f62680c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626e59b700 1 -- 172.21.15.33:0/2583713727 >> v1:172.21.15.33:6789/0 conn(0x7f62680b01f0 legacy=0x7f62680ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626e59b700 1 -- 172.21.15.33:0/2583713727 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680c92f0 con 0x7f6268156370 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626dd9a700 1 --2- 172.21.15.33:0/2583713727 >> v2:172.21.15.167:3300/0 conn(0x7f62680cc7e0 0x7f62680c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.691+0000 7f626e59b700 1 --2- 172.21.15.33:0/2583713727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6268156370 0x7f62680af8e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62640098e0 tx=0x7f62640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8610f884f0163a82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626d599700 1 -- 172.21.15.33:0/2583713727 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264011630 con 0x7f6268156370 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626d599700 1 -- 172.21.15.33:0/2583713727 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6264011790 con 0x7f6268156370 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626d599700 1 -- 172.21.15.33:0/2583713727 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264011b30 con 0x7f6268156370 2023-10-06T05:29:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 -- 172.21.15.33:0/2583713727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6268156370 msgr2=0x7f62680af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 --2- 172.21.15.33:0/2583713727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6268156370 0x7f62680af8e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62640098e0 tx=0x7f62640091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 -- 172.21.15.33:0/2583713727 shutdown_connections 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 --2- 172.21.15.33:0/2583713727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6268156370 0x7f62680af8e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 --2- 172.21.15.33:0/2583713727 >> v2:172.21.15.167:3300/0 conn(0x7f62680cc7e0 0x7f62680c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 -- 172.21.15.33:0/2583713727 >> 172.21.15.33:0/2583713727 conn(0x7f626806d030 msgr2=0x7f62680cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 -- 172.21.15.33:0/2583713727 shutdown_connections 2023-10-06T05:29:52.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.692+0000 7f626f59d700 1 -- 172.21.15.33:0/2583713727 wait complete. 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.694+0000 7f626f59d700 1 Processor -- start 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.694+0000 7f626f59d700 1 -- start start 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62680b01f0 0x7f62680cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62680cc7e0 0x7f6268158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 0x7f62680ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62681668f0 con 0x7f6268156370 2023-10-06T05:29:52.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62680a5520 con 0x7f62680cc7e0 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62680a5800 con 0x7f62680b01f0 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 0x7f62680ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 0x7f62680ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49954/0 (socket says 172.21.15.33:49954) 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626ed9c700 1 -- 172.21.15.33:0/3373891404 learned_addr learned my addr 172.21.15.33:0/3373891404 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626dd9a700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62680cc7e0 0x7f6268158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.695+0000 7f626e59b700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62680b01f0 0x7f62680cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626ed9c700 1 -- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62680b01f0 msgr2=0x7f62680cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626ed9c700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62680b01f0 0x7f62680cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626ed9c700 1 -- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62680cc7e0 msgr2=0x7f6268158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626ed9c700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62680cc7e0 0x7f6268158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:52.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626ed9c700 1 -- 172.21.15.33:0/3373891404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625800b6d0 con 0x7f6268156370 2023-10-06T05:29:52.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626ed9c700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 0x7f62680ca690 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f6260001460 tx=0x7f6260001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:52.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626000a890 con 0x7f6268156370 2023-10-06T05:29:52.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626000a9f0 con 0x7f6268156370 2023-10-06T05:29:52.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626000ad70 con 0x7f6268156370 2023-10-06T05:29:52.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6264009420 con 0x7f6268156370 2023-10-06T05:29:52.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.696+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62680a5ae0 con 0x7f6268156370 2023-10-06T05:29:52.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.697+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626804c380 con 0x7f6268156370 2023-10-06T05:29:52.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.698+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6260007300 con 0x7f6268156370 2023-10-06T05:29:52.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.699+0000 7f625e7fc700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f624c066f30 0x7f624c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:52.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.699+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f6260084b10 con 0x7f6268156370 2023-10-06T05:29:52.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.700+0000 7f626e59b700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f624c066f30 0x7f624c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:52.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.700+0000 7f626e59b700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f624c066f30 0x7f624c0693e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f62640091e0 tx=0x7f6264017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:52.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.704+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6260051b50 con 0x7f6268156370 2023-10-06T05:29:52.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:52.951+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6268151470 con 0x7f6268156370 2023-10-06T05:29:53.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.461+0000 7f625e7fc700 1 -- 172.21.15.33:0/3373891404 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 33 and data pool 34 v173) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6260055170 con 0x7f6268156370 2023-10-06T05:29:53.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.470+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f624c066f30 msgr2=0x7f624c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.470+0000 7f626f59d700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f624c066f30 0x7f624c0693e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f62640091e0 tx=0x7f6264017000 comp rx=0 tx=0).stop 2023-10-06T05:29:53.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.470+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 msgr2=0x7f62680ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.470+0000 7f626f59d700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 0x7f62680ca690 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f6260001460 tx=0x7f6260001ae0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.471+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 shutdown_connections 2023-10-06T05:29:53.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.471+0000 7f626f59d700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f624c066f30 0x7f624c0693e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.471+0000 7f626f59d700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6268156370 0x7f62680ca690 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.471+0000 7f626f59d700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62680b01f0 0x7f62680cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.471+0000 7f626f59d700 1 --2- 172.21.15.33:0/3373891404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62680cc7e0 0x7f6268158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.471+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 >> 172.21.15.33:0/3373891404 conn(0x7f626806d030 msgr2=0x7f626806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:53.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.473+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 shutdown_connections 2023-10-06T05:29:53.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.474+0000 7f626f59d700 1 -- 172.21.15.33:0/3373891404 wait complete. 2023-10-06T05:29:53.479 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 33 and data pool 34 2023-10-06T05:29:53.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:53.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.697+0000 7f51ebfff700 1 Processor -- start 2023-10-06T05:29:53.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.697+0000 7f51ebfff700 1 -- start start 2023-10-06T05:29:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.698+0000 7f51ebfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51ec0ccfc0 0x7f51ec0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.698+0000 7f51ebfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0cbd80 con 0x7f51ec0cf3b0 2023-10-06T05:29:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.698+0000 7f51ebfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0af5f0 con 0x7f51ec0cc750 2023-10-06T05:29:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.698+0000 7f51ebfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec0af8d0 con 0x7f51ec0ccfc0 2023-10-06T05:29:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51ea7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f51ec0cf3b0 0x7f51ec1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56146/0 (socket says 172.21.15.33:56146) 2023-10-06T05:29:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51ea7fc700 1 -- 172.21.15.33:0/1585765221 learned_addr learned my addr 172.21.15.33:0/1585765221 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51eaffd700 1 --2- 172.21.15.33:0/1585765221 >> v2:172.21.15.167:3300/0 conn(0x7f51ec0ccfc0 0x7f51ec0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51e9ffb700 1 -- 172.21.15.33:0/1585765221 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478931038 0 0) 0x7f51ec0cbd80 con 0x7f51ec0cf3b0 2023-10-06T05:29:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51e9ffb700 1 -- 172.21.15.33:0/1585765221 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d4003570 con 0x7f51ec0cf3b0 2023-10-06T05:29:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51eaffd700 1 -- 172.21.15.33:0/1585765221 >> v1:172.21.15.167:6789/0 conn(0x7f51ec0cc750 legacy=0x7f51ec0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51eaffd700 1 -- 172.21.15.33:0/1585765221 >> v1:172.21.15.33:6789/0 conn(0x7f51ec0cf3b0 legacy=0x7f51ec1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.699+0000 7f51eaffd700 1 -- 172.21.15.33:0/1585765221 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0afbb0 con 0x7f51ec0ccfc0 2023-10-06T05:29:53.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51eaffd700 1 --2- 172.21.15.33:0/1585765221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0ccfc0 0x7f51ec0cc210 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009910 tx=0x7f51dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd4e21f9b4487f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51e9ffb700 1 -- 172.21.15.33:0/1585765221 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00b640 con 0x7f51ec0ccfc0 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51e9ffb700 1 -- 172.21.15.33:0/1585765221 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00b7a0 con 0x7f51ec0ccfc0 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51e9ffb700 1 -- 172.21.15.33:0/1585765221 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00bb60 con 0x7f51ec0ccfc0 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51ebfff700 1 -- 172.21.15.33:0/1585765221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0ccfc0 msgr2=0x7f51ec0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51ebfff700 1 --2- 172.21.15.33:0/1585765221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0ccfc0 0x7f51ec0cc210 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009910 tx=0x7f51dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51ebfff700 1 -- 172.21.15.33:0/1585765221 shutdown_connections 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51ebfff700 1 --2- 172.21.15.33:0/1585765221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0ccfc0 0x7f51ec0cc210 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.700+0000 7f51ebfff700 1 -- 172.21.15.33:0/1585765221 >> 172.21.15.33:0/1585765221 conn(0x7f51ec06cff0 msgr2=0x7f51ec06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.701+0000 7f51ebfff700 1 -- 172.21.15.33:0/1585765221 shutdown_connections 2023-10-06T05:29:53.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.701+0000 7f51ebfff700 1 -- 172.21.15.33:0/1585765221 wait complete. 2023-10-06T05:29:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.702+0000 7f51ebfff700 1 Processor -- start 2023-10-06T05:29:53.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.702+0000 7f51ebfff700 1 -- start start 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 0x7f51ec16dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51ec0ccfc0 0x7f51ec0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0c9be0 0x7f51ec0c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ebfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0c7ee0 con 0x7f51ec0cc750 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ebfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec16d4b0 con 0x7f51ec0c9be0 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ebfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ca0d0 con 0x7f51ec0ccfc0 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 0x7f51ec16dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 0x7f51ec16dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49970/0 (socket says 172.21.15.33:49970) 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51eaffd700 1 -- 172.21.15.33:0/288053890 learned_addr learned my addr 172.21.15.33:0/288053890 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51ea7fc700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51ec0ccfc0 0x7f51ec0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.703+0000 7f51eb7fe700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0c9be0 0x7f51ec0c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51eaffd700 1 -- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51ec0ccfc0 msgr2=0x7f51ec0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51eaffd700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51ec0ccfc0 0x7f51ec0c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51eaffd700 1 -- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0c9be0 msgr2=0x7f51ec0c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51eaffd700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0c9be0 0x7f51ec0c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51eaffd700 1 -- 172.21.15.33:0/288053890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc009420 con 0x7f51ec0cc750 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51eaffd700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 0x7f51ec16dfb0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f51ec06be20 tx=0x7f51dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc01f070 con 0x7f51ec0cc750 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00bb20 con 0x7f51ec0cc750 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc011670 con 0x7f51ec0cc750 2023-10-06T05:29:53.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0c75b0 con 0x7f51ec0cc750 2023-10-06T05:29:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.704+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec0c7b70 con 0x7f51ec0cc750 2023-10-06T05:29:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.705+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec0c81c0 con 0x7f51ec0cc750 2023-10-06T05:29:53.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.706+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51dc0119e0 con 0x7f51ec0cc750 2023-10-06T05:29:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.707+0000 7f51d2ffd700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51d8066f60 0x7f51d8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.707+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51dc0859c0 con 0x7f51ec0cc750 2023-10-06T05:29:53.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.708+0000 7f51ea7fc700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51d8066f60 0x7f51d8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.709+0000 7f51ea7fc700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51d8066f60 0x7f51d8069410 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f51ec156640 tx=0x7f51e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:53.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.711+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51dc052ae0 con 0x7f51ec0cc750 2023-10-06T05:29:53.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.917+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51ec04c380 con 0x7f51ec0cc750 2023-10-06T05:29:53.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.918+0000 7f51d2ffd700 1 -- 172.21.15.33:0/288053890 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f51dc056100 con 0x7f51ec0cc750 2023-10-06T05:29:53.920 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:53.920 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":225,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:52.952467+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:50.569789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:51.932764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:53.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.920+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51d8066f60 msgr2=0x7f51d8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.920+0000 7f51ebfff700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51d8066f60 0x7f51d8069410 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f51ec156640 tx=0x7f51e0009040 comp rx=0 tx=0).stop 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.920+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 msgr2=0x7f51ec16dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.920+0000 7f51ebfff700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 0x7f51ec16dfb0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f51ec06be20 tx=0x7f51dc017000 comp rx=0 tx=0).stop 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.920+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 shutdown_connections 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51d8066f60 0x7f51d8069410 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51ec0c9be0 0x7f51ec0c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51ec0cc750 0x7f51ec16dfb0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 --2- 172.21.15.33:0/288053890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51ec0ccfc0 0x7f51ec0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 >> 172.21.15.33:0/288053890 conn(0x7f51ec06cff0 msgr2=0x7f51ec150640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 shutdown_connections 2023-10-06T05:29:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:53.921+0000 7f51ebfff700 1 -- 172.21.15.33:0/288053890 wait complete. 2023-10-06T05:29:53.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:54.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.127+0000 7f73d319d700 1 Processor -- start 2023-10-06T05:29:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.127+0000 7f73d319d700 1 -- start start 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d319d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73cc159bb0 0x7f73cc0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d319d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73cc0cbd80 con 0x7f73cc0cf3b0 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d319d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73cc0af5f0 con 0x7f73cc0ccfc0 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d319d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73cc0af8d0 con 0x7f73cc159bb0 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d299c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73cc159bb0 0x7f73cc0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d199a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73cc0cf3b0 0x7f73cc1594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56148/0 (socket says 172.21.15.33:56148) 2023-10-06T05:29:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d199a700 1 -- 172.21.15.33:0/311343306 learned_addr learned my addr 172.21.15.33:0/311343306 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.128+0000 7f73d299c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73cc159bb0 0x7f73cc0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49980/0 (socket says 172.21.15.33:49980) 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d299c700 1 -- 172.21.15.33:0/311343306 >> v1:172.21.15.167:6789/0 conn(0x7f73cc0ccfc0 legacy=0x7f73cc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d299c700 1 -- 172.21.15.33:0/311343306 >> v1:172.21.15.33:6789/0 conn(0x7f73cc0cf3b0 legacy=0x7f73cc1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d299c700 1 -- 172.21.15.33:0/311343306 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0afbb0 con 0x7f73cc159bb0 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d1199700 1 -- 172.21.15.33:0/311343306 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45778636 0 0) 0x7f73cc0cbd80 con 0x7f73cc0cf3b0 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d299c700 1 --2- 172.21.15.33:0/311343306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0b0460 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009e90 tx=0x7f73c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de33d850c2c9068 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d1199700 1 -- 172.21.15.33:0/311343306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400e070 con 0x7f73cc159bb0 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d1199700 1 -- 172.21.15.33:0/311343306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c400bbf0 con 0x7f73cc159bb0 2023-10-06T05:29:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.129+0000 7f73d1199700 1 -- 172.21.15.33:0/311343306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4012430 con 0x7f73cc159bb0 2023-10-06T05:29:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 -- 172.21.15.33:0/311343306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 msgr2=0x7f73cc0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 --2- 172.21.15.33:0/311343306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0b0460 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009e90 tx=0x7f73c4009780 comp rx=0 tx=0).stop 2023-10-06T05:29:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 -- 172.21.15.33:0/311343306 shutdown_connections 2023-10-06T05:29:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 --2- 172.21.15.33:0/311343306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0b0460 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 -- 172.21.15.33:0/311343306 >> 172.21.15.33:0/311343306 conn(0x7f73cc06cff0 msgr2=0x7f73cc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 -- 172.21.15.33:0/311343306 shutdown_connections 2023-10-06T05:29:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.130+0000 7f73d319d700 1 -- 172.21.15.33:0/311343306 wait complete. 2023-10-06T05:29:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.131+0000 7f73d319d700 1 Processor -- start 2023-10-06T05:29:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.132+0000 7f73d319d700 1 -- start start 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.132+0000 7f73d319d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73cc0ccfc0 0x7f73cc1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.132+0000 7f73d319d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.132+0000 7f73d319d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73cc0c8850 0x7f73cc0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d319d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc0b4650 con 0x7f73cc159bb0 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d319d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc0b47e0 con 0x7f73cc0c8850 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d319d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc0c9720 con 0x7f73cc0ccfc0 2023-10-06T05:29:54.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49992/0 (socket says 172.21.15.33:49992) 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 -- 172.21.15.33:0/3290344766 learned_addr learned my addr 172.21.15.33:0/3290344766 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d299c700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73cc0c8850 0x7f73cc0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d219b700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73cc0ccfc0 0x7f73cc1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 -- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73cc0ccfc0 msgr2=0x7f73cc1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73cc0ccfc0 0x7f73cc1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 -- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73cc0c8850 msgr2=0x7f73cc0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73cc0c8850 0x7f73cc0c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.133+0000 7f73d199a700 1 -- 172.21.15.33:0/3290344766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4009a20 con 0x7f73cc159bb0 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.134+0000 7f73d199a700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0c8310 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c5b0 tx=0x7f73bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:54.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.134+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc005b30 con 0x7f73cc159bb0 2023-10-06T05:29:54.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.134+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc005c90 con 0x7f73cc159bb0 2023-10-06T05:29:54.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.134+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc007320 con 0x7f73cc159bb0 2023-10-06T05:29:54.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.134+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc0c9a00 con 0x7f73cc159bb0 2023-10-06T05:29:54.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.134+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc0c9fc0 con 0x7f73cc159bb0 2023-10-06T05:29:54.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.135+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc0b4970 con 0x7f73cc159bb0 2023-10-06T05:29:54.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.136+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73bc01b3b0 con 0x7f73cc159bb0 2023-10-06T05:29:54.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.138+0000 7f73c27fc700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73b0066f30 0x7f73b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.138+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f73bc083ba0 con 0x7f73cc159bb0 2023-10-06T05:29:54.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.138+0000 7f73d219b700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73b0066f30 0x7f73b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.139+0000 7f73d219b700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73b0066f30 0x7f73b00693e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f73c80039e0 tx=0x7f73c8007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.142+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73bc050c10 con 0x7f73cc159bb0 2023-10-06T05:29:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.347+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73cc04c380 con 0x7f73cc159bb0 2023-10-06T05:29:54.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.348+0000 7f73c27fc700 1 -- 172.21.15.33:0/3290344766 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f73bc017090 con 0x7f73cc159bb0 2023-10-06T05:29:54.350 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:54.350 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":225,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:52.952467+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:50.569789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:51.932764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.350+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73b0066f30 msgr2=0x7f73b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.350+0000 7f73d319d700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73b0066f30 0x7f73b00693e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f73c80039e0 tx=0x7f73c8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.350+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 msgr2=0x7f73cc0c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.350+0000 7f73d319d700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0c8310 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c5b0 tx=0x7f73bc007d30 comp rx=0 tx=0).stop 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 shutdown_connections 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f73b0066f30 0x7f73b00693e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73cc0c8850 0x7f73cc0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73cc0ccfc0 0x7f73cc1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 --2- 172.21.15.33:0/3290344766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73cc159bb0 0x7f73cc0c8310 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 >> 172.21.15.33:0/3290344766 conn(0x7f73cc06cff0 msgr2=0x7f73cc155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 shutdown_connections 2023-10-06T05:29:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.351+0000 7f73d319d700 1 -- 172.21.15.33:0/3290344766 wait complete. 2023-10-06T05:29:54.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:29:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.562+0000 7fae3e79d700 1 Processor -- start 2023-10-06T05:29:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.562+0000 7fae3e79d700 1 -- start start 2023-10-06T05:29:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.563+0000 7fae3e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.563+0000 7fae3e79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 0x7fae380caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.563+0000 7fae3e79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae380b06b0 con 0x7fae380c9490 2023-10-06T05:29:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.563+0000 7fae3e79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae380c99e0 con 0x7fae380afb20 2023-10-06T05:29:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.563+0000 7fae3e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae380c9cc0 con 0x7fae38155a50 2023-10-06T05:29:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3d79b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3df9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 0x7fae380caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3df9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 0x7fae380caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44686/0 (socket says 172.21.15.33:44686) 2023-10-06T05:29:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3df9c700 1 -- 172.21.15.33:0/1150325753 learned_addr learned my addr 172.21.15.33:0/1150325753 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3d79b700 1 --2- 172.21.15.33:0/1150325753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:45406/0 (socket says 172.21.15.33:45406) 2023-10-06T05:29:54.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3d79b700 1 -- 172.21.15.33:0/1150325753 >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 msgr2=0x7fae380caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3d79b700 1 --2- 172.21.15.33:0/1150325753 >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 0x7fae380caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3d79b700 1 -- 172.21.15.33:0/1150325753 >> v1:172.21.15.167:6789/0 conn(0x7fae380c9490 legacy=0x7fae380af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3d79b700 1 -- 172.21.15.33:0/1150325753 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae380c9fa0 con 0x7fae38155a50 2023-10-06T05:29:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.564+0000 7fae3df9c700 1 --2- 172.21.15.33:0/1150325753 >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 0x7fae380caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.565+0000 7fae3d79b700 1 --2- 172.21.15.33:0/1150325753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38155e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae34009910 tx=0x7fae340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955574cbe57a8bb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.565+0000 7fae2ffff700 1 -- 172.21.15.33:0/1150325753 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400f6b0 con 0x7fae38155a50 2023-10-06T05:29:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.565+0000 7fae2ffff700 1 -- 172.21.15.33:0/1150325753 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3400f810 con 0x7fae38155a50 2023-10-06T05:29:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.565+0000 7fae2ffff700 1 -- 172.21.15.33:0/1150325753 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400fbd0 con 0x7fae38155a50 2023-10-06T05:29:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 -- 172.21.15.33:0/1150325753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 msgr2=0x7fae38155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 --2- 172.21.15.33:0/1150325753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38155e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae34009910 tx=0x7fae340091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 -- 172.21.15.33:0/1150325753 shutdown_connections 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 --2- 172.21.15.33:0/1150325753 >> v2:172.21.15.167:3300/0 conn(0x7fae380afb20 0x7fae380caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 --2- 172.21.15.33:0/1150325753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38155e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 -- 172.21.15.33:0/1150325753 >> 172.21.15.33:0/1150325753 conn(0x7fae3806d030 msgr2=0x7fae380cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 -- 172.21.15.33:0/1150325753 shutdown_connections 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.566+0000 7fae3e79d700 1 -- 172.21.15.33:0/1150325753 wait complete. 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.567+0000 7fae3e79d700 1 Processor -- start 2023-10-06T05:29:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 -- start start 2023-10-06T05:29:54.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 0x7fae38157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae380c9490 0x7fae380c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38165170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3815a2d0 con 0x7fae380afb20 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38158580 con 0x7fae380c9490 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.568+0000 7fae3e79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae38158860 con 0x7fae38155a50 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 0x7fae38157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 0x7fae38157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50002/0 (socket says 172.21.15.33:50002) 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 -- 172.21.15.33:0/4107921771 learned_addr learned my addr 172.21.15.33:0/4107921771 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:54.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3cf9a700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae380c9490 0x7fae380c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3df9c700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38165170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 -- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 msgr2=0x7fae38165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38165170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 -- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae380c9490 msgr2=0x7fae380c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae380c9490 0x7fae380c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3d79b700 1 -- 172.21.15.33:0/4107921771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009ae0 con 0x7fae380afb20 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3cf9a700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae380c9490 0x7fae380c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.569+0000 7fae3df9c700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38165170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.570+0000 7fae3d79b700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 0x7fae38157d30 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fae380c9e00 tx=0x7fae34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.570+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3401f070 con 0x7fae380afb20 2023-10-06T05:29:54.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.570+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34000d10 con 0x7fae380afb20 2023-10-06T05:29:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.570+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340106a0 con 0x7fae380afb20 2023-10-06T05:29:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.570+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae34009420 con 0x7fae380afb20 2023-10-06T05:29:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.570+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae38158e20 con 0x7fae380afb20 2023-10-06T05:29:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.571+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3815a4d0 con 0x7fae380afb20 2023-10-06T05:29:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.572+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae34023410 con 0x7fae380afb20 2023-10-06T05:29:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.573+0000 7fae2d7fa700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.573+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fae34014070 con 0x7fae380afb20 2023-10-06T05:29:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.573+0000 7fae3cf9a700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:54.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.574+0000 7fae3cf9a700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fae1c066f30 0x7fae1c0693e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fae380c1350 tx=0x7fae28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:54.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.577+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae3815a4d0 con 0x7fae380afb20 2023-10-06T05:29:54.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:54.825+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae3815a4d0 con 0x7fae380afb20 2023-10-06T05:29:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.479+0000 7fae2d7fa700 1 -- 172.21.15.33:0/4107921771 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v175) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae3815a4d0 con 0x7fae380afb20 2023-10-06T05:29:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fae1c066f30 msgr2=0x7fae1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fae1c066f30 0x7fae1c0693e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fae380c1350 tx=0x7fae28009040 comp rx=0 tx=0).stop 2023-10-06T05:29:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 msgr2=0x7fae38157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 0x7fae38157d30 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fae380c9e00 tx=0x7fae34019000 comp rx=0 tx=0).stop 2023-10-06T05:29:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 shutdown_connections 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae38155a50 0x7fae38165170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae380afb20 0x7fae38157d30 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 --2- 172.21.15.33:0/4107921771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae380c9490 0x7fae380c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 >> 172.21.15.33:0/4107921771 conn(0x7fae3806d030 msgr2=0x7fae3806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 shutdown_connections 2023-10-06T05:29:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.482+0000 7fae3e79d700 1 -- 172.21.15.33:0/4107921771 wait complete. 2023-10-06T05:29:55.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:29:55.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.699+0000 7f575ed9d700 1 Processor -- start 2023-10-06T05:29:55.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.699+0000 7f575ed9d700 1 -- start start 2023-10-06T05:29:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.700+0000 7f575ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57580c9490 0x7f57580cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.700+0000 7f575ed9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57580b0230 con 0x7f5758155a50 2023-10-06T05:29:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.700+0000 7f575ed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57580b06b0 con 0x7f5758155ec0 2023-10-06T05:29:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.700+0000 7f575ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5758156320 con 0x7f57580c9490 2023-10-06T05:29:55.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575d59a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5758155a50 0x7f57580af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56164/0 (socket says 172.21.15.33:56164) 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575d59a700 1 -- 172.21.15.33:0/2393914925 learned_addr learned my addr 172.21.15.33:0/2393914925 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575dd9b700 1 --2- 172.21.15.33:0/2393914925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57580c9490 0x7f57580cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290270742 0 0) 0x7f57580b0230 con 0x7f5758155a50 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5744003570 con 0x7f5758155a50 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889324532 0 0) 0x7f57580b06b0 con 0x7f5758155ec0 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57580b0230 con 0x7f5758155ec0 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575dd9b700 1 -- 172.21.15.33:0/2393914925 >> v1:172.21.15.167:6789/0 conn(0x7f5758155ec0 legacy=0x7f57580afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.701+0000 7f575dd9b700 1 -- 172.21.15.33:0/2393914925 >> v1:172.21.15.33:6789/0 conn(0x7f5758155a50 legacy=0x7f57580af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.702+0000 7f575dd9b700 1 -- 172.21.15.33:0/2393914925 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580c99e0 con 0x7f57580c9490 2023-10-06T05:29:55.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.702+0000 7f575dd9b700 1 --2- 172.21.15.33:0/2393914925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57580c9490 0x7f57580cc7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5754009910 tx=0x7f57540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c291f058ecac7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:55.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.702+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754011630 con 0x7f57580c9490 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.702+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5754011790 con 0x7f57580c9490 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.702+0000 7f575cd99700 1 -- 172.21.15.33:0/2393914925 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754011b30 con 0x7f57580c9490 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 -- 172.21.15.33:0/2393914925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57580c9490 msgr2=0x7f57580cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 --2- 172.21.15.33:0/2393914925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57580c9490 0x7f57580cc7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5754009910 tx=0x7f57540091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 -- 172.21.15.33:0/2393914925 shutdown_connections 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 --2- 172.21.15.33:0/2393914925 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57580c9490 0x7f57580cc7e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 -- 172.21.15.33:0/2393914925 >> 172.21.15.33:0/2393914925 conn(0x7f575806d030 msgr2=0x7f57580cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 -- 172.21.15.33:0/2393914925 shutdown_connections 2023-10-06T05:29:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.703+0000 7f575ed9d700 1 -- 172.21.15.33:0/2393914925 wait complete. 2023-10-06T05:29:55.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.704+0000 7f575ed9d700 1 Processor -- start 2023-10-06T05:29:55.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.705+0000 7f575ed9d700 1 -- start start 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.705+0000 7f575ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57580c9490 0x7f575816a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5758155a50 0x7f5758145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 0x7f57581517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575ed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5758154a80 con 0x7f5758155ec0 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575ed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5758153e70 con 0x7f57580c9490 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5758154150 con 0x7f5758155a50 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 0x7f57581517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 0x7f57581517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50012/0 (socket says 172.21.15.33:50012) 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 -- 172.21.15.33:0/4177516783 learned_addr learned my addr 172.21.15.33:0/4177516783 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575d59a700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5758155a50 0x7f5758145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575dd9b700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57580c9490 0x7f575816a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 -- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5758155a50 msgr2=0x7f5758145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5758155a50 0x7f5758145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 -- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57580c9490 msgr2=0x7f575816a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57580c9490 0x7f575816a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.706+0000 7f575e59c700 1 -- 172.21.15.33:0/4177516783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5754009420 con 0x7f5758155ec0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.707+0000 7f575e59c700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 0x7f57581517b0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f575000c860 tx=0x7f5750002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.707+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750002030 con 0x7f5758155ec0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.707+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57500021b0 con 0x7f5758155ec0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.707+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750012630 con 0x7f5758155ec0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.707+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5758154430 con 0x7f5758155ec0 2023-10-06T05:29:55.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.707+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5758146350 con 0x7f5758155ec0 2023-10-06T05:29:55.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.708+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5758154c30 con 0x7f5758155ec0 2023-10-06T05:29:55.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.709+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5750012a20 con 0x7f5758155ec0 2023-10-06T05:29:55.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.710+0000 7f574dffb700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f573c066f30 0x7f573c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:55.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.710+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5750084620 con 0x7f5758155ec0 2023-10-06T05:29:55.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.710+0000 7f575dd9b700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f573c066f30 0x7f573c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:55.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.711+0000 7f575dd9b700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f573c066f30 0x7f573c0693e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5754009860 tx=0x7f5754017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:55.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.714+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5758154c30 con 0x7f5758155ec0 2023-10-06T05:29:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:55.962+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f575804c380 con 0x7f5758155ec0 2023-10-06T05:29:56.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.488+0000 7f574dffb700 1 -- 172.21.15.33:0/4177516783 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v176) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5750051690 con 0x7f5758155ec0 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f573c066f30 msgr2=0x7f573c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f573c066f30 0x7f573c0693e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5754009860 tx=0x7f5754017000 comp rx=0 tx=0).stop 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 msgr2=0x7f57581517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 0x7f57581517b0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f575000c860 tx=0x7f5750002700 comp rx=0 tx=0).stop 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 shutdown_connections 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f573c066f30 0x7f573c0693e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5758155ec0 0x7f57581517b0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57580c9490 0x7f575816a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 --2- 172.21.15.33:0/4177516783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5758155a50 0x7f5758145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 >> 172.21.15.33:0/4177516783 conn(0x7f575806d030 msgr2=0x7f5758069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:56.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 shutdown_connections 2023-10-06T05:29:56.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.491+0000 7f575ed9d700 1 -- 172.21.15.33:0/4177516783 wait complete. 2023-10-06T05:29:56.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:56.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.697+0000 7fa61219d700 1 Processor -- start 2023-10-06T05:29:56.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.697+0000 7fa61219d700 1 -- start start 2023-10-06T05:29:56.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.698+0000 7fa61219d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa60c0c6340 0x7fa60c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:56.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.698+0000 7fa61219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c0c80d0 0x7fa60c06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:56.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.698+0000 7fa61219d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c06c820 con 0x7fa60c159520 2023-10-06T05:29:56.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.698+0000 7fa61219d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c06a160 con 0x7fa60c0c6340 2023-10-06T05:29:56.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.698+0000 7fa61219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0c8800 con 0x7fa60c0c80d0 2023-10-06T05:29:56.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa60c0c6340 0x7fa60c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:56.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa60c0c6340 0x7fa60c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50014/0 (socket says 172.21.15.33:50014) 2023-10-06T05:29:56.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 -- 172.21.15.33:0/827628890 learned_addr learned my addr 172.21.15.33:0/827628890 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61199c700 1 --2- 172.21.15.33:0/827628890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c0c80d0 0x7fa60c06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 -- 172.21.15.33:0/827628890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c0c80d0 msgr2=0x7fa60c06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 --2- 172.21.15.33:0/827628890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c0c80d0 0x7fa60c06a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 -- 172.21.15.33:0/827628890 >> v1:172.21.15.167:6789/0 conn(0x7fa60c159520 legacy=0x7fa60c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.699+0000 7fa61099a700 1 -- 172.21.15.33:0/827628890 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c0c8ae0 con 0x7fa60c0c6340 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61099a700 1 --2- 172.21.15.33:0/827628890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c6340 0x7fa60c0c7b90 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0099b0 tx=0x7fa5fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0b8ccfce6790c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa603fff700 1 -- 172.21.15.33:0/827628890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc009ea0 con 0x7fa60c0c6340 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa603fff700 1 -- 172.21.15.33:0/827628890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00b860 con 0x7fa60c0c6340 2023-10-06T05:29:56.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa603fff700 1 -- 172.21.15.33:0/827628890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00bc00 con 0x7fa60c0c6340 2023-10-06T05:29:56.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61219d700 1 -- 172.21.15.33:0/827628890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c6340 msgr2=0x7fa60c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61219d700 1 --2- 172.21.15.33:0/827628890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c6340 0x7fa60c0c7b90 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0099b0 tx=0x7fa5fc009250 comp rx=0 tx=0).stop 2023-10-06T05:29:56.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61219d700 1 -- 172.21.15.33:0/827628890 shutdown_connections 2023-10-06T05:29:56.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61219d700 1 --2- 172.21.15.33:0/827628890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c0c80d0 0x7fa60c06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61219d700 1 --2- 172.21.15.33:0/827628890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c6340 0x7fa60c0c7b90 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.700+0000 7fa61219d700 1 -- 172.21.15.33:0/827628890 >> 172.21.15.33:0/827628890 conn(0x7fa60c0cc930 msgr2=0x7fa60c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:56.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.701+0000 7fa61219d700 1 -- 172.21.15.33:0/827628890 shutdown_connections 2023-10-06T05:29:56.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.701+0000 7fa61219d700 1 -- 172.21.15.33:0/827628890 wait complete. 2023-10-06T05:29:56.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.702+0000 7fa61219d700 1 Processor -- start 2023-10-06T05:29:56.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.702+0000 7fa61219d700 1 -- start start 2023-10-06T05:29:56.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61219d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60c0c6340 0x7fa60c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:56.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61219d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 0x7fa60c06ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:56.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61219d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c159520 0x7fa60c06b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61219d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c1553d0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61219d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c15c710 con 0x7fa60c0c6340 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61219d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c1550f0 con 0x7fa60c159520 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 0x7fa60c06ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61099a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 0x7fa60c06ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50018/0 (socket says 172.21.15.33:50018) 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.703+0000 7fa61099a700 1 -- 172.21.15.33:0/2857790484 learned_addr learned my addr 172.21.15.33:0/2857790484 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61199c700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c159520 0x7fa60c06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61119b700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60c0c6340 0x7fa60c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61099a700 1 -- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c159520 msgr2=0x7fa60c06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61099a700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c159520 0x7fa60c06b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61099a700 1 -- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60c0c6340 msgr2=0x7fa60c151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61099a700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60c0c6340 0x7fa60c151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61099a700 1 -- 172.21.15.33:0/2857790484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc0094f0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61099a700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 0x7fa60c06ac00 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0636f0 tx=0x7fa5fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00bbe0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc011420 con 0x7fa60c0c80d0 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0117c0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.704+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60c169f20 con 0x7fa60c0c80d0 2023-10-06T05:29:56.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.705+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c16a4e0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.705+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60c169d80 con 0x7fa60c0c80d0 2023-10-06T05:29:56.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.706+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5fc011bf0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.707+0000 7fa6017fa700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5f0066f30 0x7fa5f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:56.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.707+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa5fc0860e0 con 0x7fa60c0c80d0 2023-10-06T05:29:56.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.708+0000 7fa61119b700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5f0066f30 0x7fa5f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:56.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.708+0000 7fa61119b700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5f0066f30 0x7fa5f00693e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa60c15e670 tx=0x7fa608009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:56.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.711+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa60c169d80 con 0x7fa60c0c80d0 2023-10-06T05:29:56.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.959+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa60c169d80 con 0x7fa60c0c80d0 2023-10-06T05:29:56.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.959+0000 7fa6017fa700 1 -- 172.21.15.33:0/2857790484 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7fa60c169d80 con 0x7fa60c0c80d0 2023-10-06T05:29:56.962 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:56.962 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:29:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5f0066f30 msgr2=0x7fa5f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5f0066f30 0x7fa5f00693e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa60c15e670 tx=0x7fa608009040 comp rx=0 tx=0).stop 2023-10-06T05:29:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 msgr2=0x7fa60c06ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 0x7fa60c06ac00 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0636f0 tx=0x7fa5fc017000 comp rx=0 tx=0).stop 2023-10-06T05:29:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 shutdown_connections 2023-10-06T05:29:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5f0066f30 0x7fa5f00693e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60c159520 0x7fa60c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa60c0c6340 0x7fa60c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 --2- 172.21.15.33:0/2857790484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c0c80d0 0x7fa60c06ac00 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.961+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 >> 172.21.15.33:0/2857790484 conn(0x7fa60c0cc930 msgr2=0x7fa60c0c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.962+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 shutdown_connections 2023-10-06T05:29:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:56.962+0000 7fa61219d700 1 -- 172.21.15.33:0/2857790484 wait complete. 2023-10-06T05:29:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:29:56.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:29:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.167+0000 7f26d9f9d700 1 Processor -- start 2023-10-06T05:29:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.167+0000 7f26d9f9d700 1 -- start start 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.168+0000 7f26d9f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f26d413faa0 0x7f26d413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d9f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26d406aad0 0x7f26d406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d9f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d40c9cb0 con 0x7f26d40c7940 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d9f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d40ca270 con 0x7f26d406aad0 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d9f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d406b840 con 0x7f26d413faa0 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26d406aad0 0x7f26d406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26d406aad0 0x7f26d406af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50022/0 (socket says 172.21.15.33:50022) 2023-10-06T05:29:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 -- 172.21.15.33:0/232312755 learned_addr learned my addr 172.21.15.33:0/232312755 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d8f9b700 1 --2- 172.21.15.33:0/232312755 >> v2:172.21.15.167:3300/0 conn(0x7f26d413faa0 0x7f26d413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 -- 172.21.15.33:0/232312755 >> v2:172.21.15.167:3300/0 conn(0x7f26d413faa0 msgr2=0x7f26d413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 --2- 172.21.15.33:0/232312755 >> v2:172.21.15.167:3300/0 conn(0x7f26d413faa0 0x7f26d413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 -- 172.21.15.33:0/232312755 >> v1:172.21.15.167:6789/0 conn(0x7f26d40c7940 legacy=0x7f26d40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.169+0000 7f26d979c700 1 -- 172.21.15.33:0/232312755 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d406bb20 con 0x7f26d406aad0 2023-10-06T05:29:57.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.170+0000 7f26d979c700 1 --2- 172.21.15.33:0/232312755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d406af30 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f26d406fe50 tx=0x7f26cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709182317fb0b285 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:57.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.170+0000 7f26d37fe700 1 -- 172.21.15.33:0/232312755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00e070 con 0x7f26d406aad0 2023-10-06T05:29:57.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.170+0000 7f26d37fe700 1 -- 172.21.15.33:0/232312755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc00ba50 con 0x7f26d406aad0 2023-10-06T05:29:57.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.170+0000 7f26d37fe700 1 -- 172.21.15.33:0/232312755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0092e0 con 0x7f26d406aad0 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.170+0000 7f26d9f9d700 1 -- 172.21.15.33:0/232312755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 msgr2=0x7f26d406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.170+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/232312755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d406af30 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f26d406fe50 tx=0x7f26cc009840 comp rx=0 tx=0).stop 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.171+0000 7f26d9f9d700 1 -- 172.21.15.33:0/232312755 shutdown_connections 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.171+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/232312755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d406af30 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.171+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/232312755 >> v2:172.21.15.167:3300/0 conn(0x7f26d413faa0 0x7f26d413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.171+0000 7f26d9f9d700 1 -- 172.21.15.33:0/232312755 >> 172.21.15.33:0/232312755 conn(0x7f26d406d030 msgr2=0x7f26d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.171+0000 7f26d9f9d700 1 -- 172.21.15.33:0/232312755 shutdown_connections 2023-10-06T05:29:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.171+0000 7f26d9f9d700 1 -- 172.21.15.33:0/232312755 wait complete. 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.172+0000 7f26d9f9d700 1 Processor -- start 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.172+0000 7f26d9f9d700 1 -- start start 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.173+0000 7f26d9f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d4159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.173+0000 7f26d9f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26d40c7940 0x7f26d413d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.173+0000 7f26d9f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26d413faa0 0x7f26d413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.173+0000 7f26d9f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40c3220 con 0x7f26d406aad0 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.173+0000 7f26d9f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d413dbf0 con 0x7f26d413faa0 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.173+0000 7f26d9f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d413e110 con 0x7f26d40c7940 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d4159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d4159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50026/0 (socket says 172.21.15.33:50026) 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 -- 172.21.15.33:0/3061716883 learned_addr learned my addr 172.21.15.33:0/3061716883 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d979c700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26d413faa0 0x7f26d413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d3fff700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26d40c7940 0x7f26d413d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 -- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26d40c7940 msgr2=0x7f26d413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26d40c7940 0x7f26d413d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 -- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26d413faa0 msgr2=0x7f26d413e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26d413faa0 0x7f26d413e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 -- 172.21.15.33:0/3061716883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc009ae0 con 0x7f26d406aad0 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d3fff700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26d40c7940 0x7f26d413d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d8f9b700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d4159960 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f26c800d860 tx=0x7f26c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.174+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80125f0 con 0x7f26d406aad0 2023-10-06T05:29:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.175+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c8012750 con 0x7f26d406aad0 2023-10-06T05:29:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.175+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d416aaf0 con 0x7f26d406aad0 2023-10-06T05:29:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.175+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8012b10 con 0x7f26d406aad0 2023-10-06T05:29:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.175+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d41585e0 con 0x7f26d406aad0 2023-10-06T05:29:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.175+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d4159ea0 con 0x7f26d406aad0 2023-10-06T05:29:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.176+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26c801e460 con 0x7f26d406aad0 2023-10-06T05:29:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.177+0000 7f26d0ff9700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26b8066f30 0x7f26b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.177+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26c8084df0 con 0x7f26d406aad0 2023-10-06T05:29:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.178+0000 7f26d3fff700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26b8066f30 0x7f26b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.179+0000 7f26d3fff700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26b8066f30 0x7f26b80693e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f26c0001840 tx=0x7f26c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.181+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26c80558f0 con 0x7f26d406aad0 2023-10-06T05:29:57.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.390+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26d404c380 con 0x7f26d406aad0 2023-10-06T05:29:57.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.391+0000 7f26d0ff9700 1 -- 172.21.15.33:0/3061716883 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f26c8055480 con 0x7f26d406aad0 2023-10-06T05:29:57.393 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:57.393 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":226,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:29:54.468345+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:50.569789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:51.932764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:29:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26b8066f30 msgr2=0x7f26b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26b8066f30 0x7f26b80693e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f26c0001840 tx=0x7f26c0009040 comp rx=0 tx=0).stop 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 msgr2=0x7f26d4159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d4159960 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f26c800d860 tx=0x7f26c8008040 comp rx=0 tx=0).stop 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 shutdown_connections 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f26b8066f30 0x7f26b80693e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26d413faa0 0x7f26d413e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d406aad0 0x7f26d4159960 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 --2- 172.21.15.33:0/3061716883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26d40c7940 0x7f26d413d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 >> 172.21.15.33:0/3061716883 conn(0x7f26d406d030 msgr2=0x7f26d40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.393+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 shutdown_connections 2023-10-06T05:29:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.394+0000 7f26d9f9d700 1 -- 172.21.15.33:0/3061716883 wait complete. 2023-10-06T05:29:57.413 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:29:57.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:29:57.423 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:29:57.422+0000 7fefc5ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:29:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.600+0000 7fb17f59d700 1 Processor -- start 2023-10-06T05:29:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.600+0000 7fb17f59d700 1 -- start start 2023-10-06T05:29:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1780b04e0 0x7fb1780ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1781575c0 0x7fb1780cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1780b1ae0 con 0x7fb1781571f0 2023-10-06T05:29:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780ad8c0 con 0x7fb1781575c0 2023-10-06T05:29:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b17b0 con 0x7fb1780b04e0 2023-10-06T05:29:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb1781571f0 0x7fb1780afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56178/0 (socket says 172.21.15.33:56178) 2023-10-06T05:29:57.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17e59b700 1 -- 172.21.15.33:0/3232293037 learned_addr learned my addr 172.21.15.33:0/3232293037 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17ed9c700 1 --2- 172.21.15.33:0/3232293037 >> v2:172.21.15.167:3300/0 conn(0x7fb1781575c0 0x7fb1780cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.602+0000 7fb17dd9a700 1 --2- 172.21.15.33:0/3232293037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1780b04e0 0x7fb1780ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17d599700 1 -- 172.21.15.33:0/3232293037 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111802938 0 0) 0x7fb1780b1ae0 con 0x7fb1781571f0 2023-10-06T05:29:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17d599700 1 -- 172.21.15.33:0/3232293037 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb164003570 con 0x7fb1781571f0 2023-10-06T05:29:57.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17dd9a700 1 -- 172.21.15.33:0/3232293037 >> v2:172.21.15.167:3300/0 conn(0x7fb1781575c0 msgr2=0x7fb1780cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17dd9a700 1 --2- 172.21.15.33:0/3232293037 >> v2:172.21.15.167:3300/0 conn(0x7fb1781575c0 0x7fb1780cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17dd9a700 1 -- 172.21.15.33:0/3232293037 >> v1:172.21.15.33:6789/0 conn(0x7fb1781571f0 legacy=0x7fb1780afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17dd9a700 1 -- 172.21.15.33:0/3232293037 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780cd9f0 con 0x7fb1780b04e0 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17ed9c700 1 --2- 172.21.15.33:0/3232293037 >> v2:172.21.15.167:3300/0 conn(0x7fb1781575c0 0x7fb1780cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17dd9a700 1 --2- 172.21.15.33:0/3232293037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1780b04e0 0x7fb1780ccda0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1680098e0 tx=0x7fb1680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287d02d2ac95352d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.603+0000 7fb17d599700 1 -- 172.21.15.33:0/3232293037 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168011630 con 0x7fb1780b04e0 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17d599700 1 -- 172.21.15.33:0/3232293037 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb168011790 con 0x7fb1780b04e0 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17d599700 1 -- 172.21.15.33:0/3232293037 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168011b30 con 0x7fb1780b04e0 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 -- 172.21.15.33:0/3232293037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1780b04e0 msgr2=0x7fb1780ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 --2- 172.21.15.33:0/3232293037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1780b04e0 0x7fb1780ccda0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1680098e0 tx=0x7fb1680091a0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 -- 172.21.15.33:0/3232293037 shutdown_connections 2023-10-06T05:29:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 --2- 172.21.15.33:0/3232293037 >> v2:172.21.15.167:3300/0 conn(0x7fb1781575c0 0x7fb1780cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 --2- 172.21.15.33:0/3232293037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1780b04e0 0x7fb1780ccda0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 -- 172.21.15.33:0/3232293037 >> 172.21.15.33:0/3232293037 conn(0x7fb17806c560 msgr2=0x7fb17806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 -- 172.21.15.33:0/3232293037 shutdown_connections 2023-10-06T05:29:57.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.604+0000 7fb17f59d700 1 -- 172.21.15.33:0/3232293037 wait complete. 2023-10-06T05:29:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.606+0000 7fb17f59d700 1 Processor -- start 2023-10-06T05:29:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.606+0000 7fb17f59d700 1 -- start start 2023-10-06T05:29:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1780b04e0 0x7fb178145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 0x7fb1780c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1781575c0 0x7fb17815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b2f10 con 0x7fb1781571f0 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b2060 con 0x7fb1780b04e0 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b2300 con 0x7fb1781575c0 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 0x7fb1780c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 0x7fb1780c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50030/0 (socket says 172.21.15.33:50030) 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17dd9a700 1 -- 172.21.15.33:0/1302029827 learned_addr learned my addr 172.21.15.33:0/1302029827 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17ed9c700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1781575c0 0x7fb17815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.607+0000 7fb17e59b700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1780b04e0 0x7fb178145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17dd9a700 1 -- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1781575c0 msgr2=0x7fb17815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17dd9a700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1781575c0 0x7fb17815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17dd9a700 1 -- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1780b04e0 msgr2=0x7fb178145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17dd9a700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1780b04e0 0x7fb178145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17dd9a700 1 -- 172.21.15.33:0/1302029827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170009a20 con 0x7fb1781571f0 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17dd9a700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 0x7fb1780c3ad0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b18f0 tx=0x7fb168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168011b10 con 0x7fb1781571f0 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1680253f0 con 0x7fb1781571f0 2023-10-06T05:29:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb168009420 con 0x7fb1781571f0 2023-10-06T05:29:57.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680257b0 con 0x7fb1781571f0 2023-10-06T05:29:57.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.608+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1780b28c0 con 0x7fb1781571f0 2023-10-06T05:29:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.609+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17804c380 con 0x7fb1781571f0 2023-10-06T05:29:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.610+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb168025ba0 con 0x7fb1781571f0 2023-10-06T05:29:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.611+0000 7fb16e7fc700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb15c066f60 0x7fb15c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.611+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb168085eb0 con 0x7fb1781571f0 2023-10-06T05:29:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.612+0000 7fb17e59b700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb15c066f60 0x7fb15c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.612+0000 7fb17e59b700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb15c066f60 0x7fb15c069410 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb174007fd0 tx=0x7fb174009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.615+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb168052fe0 con 0x7fb1781571f0 2023-10-06T05:29:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.814+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1780ce960 con 0x7fb15c066f60 2023-10-06T05:29:57.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.823+0000 7fb16e7fc700 1 -- 172.21.15.33:0/1302029827 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300381 (secure 0 0 0) 0x7fb1780ce960 con 0x7fb15c066f60 2023-10-06T05:29:57.834 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb15c066f60 msgr2=0x7fb15c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb15c066f60 0x7fb15c069410 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb174007fd0 tx=0x7fb174009040 comp rx=0 tx=0).stop 2023-10-06T05:29:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 msgr2=0x7fb1780c3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 0x7fb1780c3ad0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b18f0 tx=0x7fb168017000 comp rx=0 tx=0).stop 2023-10-06T05:29:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 shutdown_connections 2023-10-06T05:29:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb15c066f60 0x7fb15c069410 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1781575c0 0x7fb17815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1780b04e0 0x7fb178145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 --2- 172.21.15.33:0/1302029827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1781571f0 0x7fb1780c3ad0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 >> 172.21.15.33:0/1302029827 conn(0x7fb17806c560 msgr2=0x7fb178153950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 shutdown_connections 2023-10-06T05:29:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:57.828+0000 7fb17f59d700 1 -- 172.21.15.33:0/1302029827 wait complete. 2023-10-06T05:29:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:29:57.848 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":691,"stamp":"2023-10-06T05:29:56.052662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2434220,"kb_used_data":23148,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752540500,"statfs":{"total":773094113280,"available":770601472000,"internally_reserved":0,"allocated":23703552,"data_stored":11699346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.607650"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.449026+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:10.379257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447894+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:45.827072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.452448+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:50.356871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447392+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:01.481198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.453307+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:03.390349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.444859+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:01.003882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447314+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:32.462441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.451602+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:41.457035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.448857+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:22.915360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447302+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:58.244971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.450376+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:23.653866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.449811+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:33.300595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.452802+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:50.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.449262+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:57.688224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447334+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:47.945210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447348+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:34.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.450219+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:52.169893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.448515+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:51.549085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.451300+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:04.177286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.449232+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:39.013488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.452122+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:00.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.451717+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:52.169893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.444923+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:50.191614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.449750+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:20.310675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.448285+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:30.285047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447941+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:02.506956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.450619+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:20.920352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.451529+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:50.356871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447846+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:08.860299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.449397+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:55.694604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.450548+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:03.420825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:29:51.431387+0000","last_change":"2023-10-06T05:29:51.447762+0000","last_active":"2023-10-06T05:29:51.431387+0000","last_peered":"2023-10-06T05:29:51.431387+0000","last_clean":"2023-10-06T05:29:51.431387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:51.431387+0000","last_undegraded":"2023-10-06T05:29:51.431387+0000","last_fullsized":"2023-10-06T05:29:51.431387+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:59.455088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":179,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:47.899176+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:29:47.899176+0000","last_peered":"2023-10-06T05:29:47.899176+0000","last_clean":"2023-10-06T05:29:47.899176+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:29:47.899176+0000","last_undegraded":"2023-10-06T05:29:47.899176+0000","last_fullsized":"2023-10-06T05:29:47.899176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":177,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:47.907734+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:29:47.907734+0000","last_peered":"2023-10-06T05:29:47.907734+0000","last_clean":"2023-10-06T05:29:47.907734+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:29:47.907734+0000","last_undegraded":"2023-10-06T05:29:47.907734+0000","last_fullsized":"2023-10-06T05:29:47.907734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":180,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:49.008800+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:29:49.008800+0000","last_peered":"2023-10-06T05:29:49.008800+0000","last_clean":"2023-10-06T05:29:49.008800+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:29:49.008800+0000","last_undegraded":"2023-10-06T05:29:49.008800+0000","last_fullsized":"2023-10-06T05:29:49.008800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":177,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:47.907735+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:29:47.907735+0000","last_peered":"2023-10-06T05:29:47.907735+0000","last_clean":"2023-10-06T05:29:47.907735+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:29:47.907735+0000","last_undegraded":"2023-10-06T05:29:47.907735+0000","last_fullsized":"2023-10-06T05:29:47.907735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":327,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:49.009111+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:29:49.009111+0000","last_peered":"2023-10-06T05:29:49.009111+0000","last_clean":"2023-10-06T05:29:49.009111+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:29:49.009111+0000","last_undegraded":"2023-10-06T05:29:49.009111+0000","last_fullsized":"2023-10-06T05:29:49.009111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":174,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:49.009111+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:29:49.009111+0000","last_peered":"2023-10-06T05:29:49.009111+0000","last_clean":"2023-10-06T05:29:49.009111+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:29:49.009111+0000","last_undegraded":"2023-10-06T05:29:49.009111+0000","last_fullsized":"2023-10-06T05:29:49.009111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":174,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:49.009114+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:29:49.009114+0000","last_peered":"2023-10-06T05:29:49.009114+0000","last_clean":"2023-10-06T05:29:49.009114+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:29:49.009114+0000","last_undegraded":"2023-10-06T05:29:49.009114+0000","last_fullsized":"2023-10-06T05:29:49.009114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":179,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:47.899172+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:29:47.899172+0000","last_peered":"2023-10-06T05:29:47.899172+0000","last_clean":"2023-10-06T05:29:47.899172+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:29:47.899172+0000","last_undegraded":"2023-10-06T05:29:47.899172+0000","last_fullsized":"2023-10-06T05:29:47.899172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:29:52.442232+0000","last_change":"2023-10-06T05:29:52.442232+0000","last_active":"2023-10-06T05:29:52.442232+0000","last_peered":"2023-10-06T05:29:52.442232+0000","last_clean":"2023-10-06T05:29:52.442232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:29:52.442232+0000","last_undegraded":"2023-10-06T05:29:52.442232+0000","last_fullsized":"2023-10-06T05:29:52.442232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":210,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-10-06T05:29:47.879747+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:29:47.879747+0000","last_peered":"2023-10-06T05:29:47.879747+0000","last_clean":"2023-10-06T05:29:47.879747+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:29:47.879747+0000","last_undegraded":"2023-10-06T05:29:47.879747+0000","last_fullsized":"2023-10-06T05:29:47.879747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575041,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304708,"kb_used_data":3324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067132,"statfs":{"total":96636764160,"available":96324743168,"internally_reserved":0,"allocated":3403776,"data_stored":1902885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.32300000000000001},"max":{"1min":0.88100000000000001,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88200000000000001,"5min":1.1339999999999999,"15min":1.169},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68600000000000005,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":1.0249999999999999,"15min":1.268},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46700000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96399999999999997,"5min":1.163,"15min":1.163},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":1.22},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.109,"15min":1.109},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.753},"min":{"1min":0.47299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":1.0960000000000001,"15min":1.202},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.627},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.84499999999999997,"5min":1.109,"15min":1.258},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.55200000000000005,"5min":0.47299999999999998,"15min":0.39400000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.2849999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"max":{"1min":0.98099999999999998,"5min":1.109,"15min":1.109},"last":0.754},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.752,"15min":0.749},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.407},"last":0.38500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.999,"5min":1.169,"15min":1.169},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.52700000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.0349999999999999,"5min":1.135,"15min":1.3300000000000001},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304128,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067712,"statfs":{"total":96636764160,"available":96325337088,"internally_reserved":0,"allocated":2809856,"data_stored":1312517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34200000000000003},"max":{"1min":0.93799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.47399999999999998,"15min":0.44600000000000001},"max":{"1min":0.999,"5min":1.137,"15min":1.137},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72899999999999998,"15min":0.71099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":1.135,"15min":1.135},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53000000000000003,"5min":0.496,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.115,"15min":1.115},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72399999999999998,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.748,"15min":0.749},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.1020000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.503},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.97799999999999998,"15min":0.998},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0760000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.73299999999999998,"15min":0.71599999999999997},"min":{"1min":0.56399999999999995,"5min":0.47999999999999998,"15min":0.40000000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.4610000000000001},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.432},"max":{"1min":0.96399999999999997,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.436},"max":{"1min":1.032,"5min":1.0860000000000001,"15min":1.1359999999999999},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834575040,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304140,"kb_used_data":2756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067700,"statfs":{"total":96636764160,"available":96325324800,"internally_reserved":0,"allocated":2822144,"data_stored":1318675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.621,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27800000000000002},"max":{"1min":0.56200000000000006,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.86099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":1.165,"15min":2.6520000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.503,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.76600000000000001,"5min":1.077,"15min":2.7010000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.72399999999999998,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35199999999999998},"max":{"1min":0.69299999999999995,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.026,"5min":1.111,"15min":1.5389999999999999},"last":1.012},{"interface":"front","average":{"1min":0.745,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.45200000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.4690000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.79800000000000004,"15min":0.78800000000000003},"min":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.349},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.50600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.669},"last":0.97199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.48199999999999998},"max":{"1min":1.2290000000000001,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.573},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:28:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.46999999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.528},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.48699999999999999},"max":{"1min":1.073,"5min":1.3049999999999999,"15min":2.6240000000000001},"last":0.76000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575040,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304708,"kb_used_data":3324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067132,"statfs":{"total":96636764160,"available":96324743168,"internally_reserved":0,"allocated":3403776,"data_stored":1902885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45700000000000002,"15min":0.45400000000000001},"min":{"1min":0.29999999999999999,"5min":0.28799999999999998,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.26900000000000002},"max":{"1min":0.72799999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.66600000000000004,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.66700000000000004,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.39100000000000001},"max":{"1min":0.73999999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.47299999999999998},"max":{"1min":0.90200000000000002,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":1.008},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.0600000000000001,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":1.032}]},{"osd":5,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.54500000000000004},"max":{"1min":0.996,"5min":1.27,"15min":1.27},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":0.90900000000000003,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97599999999999998,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80700000000000005,"15min":0.80600000000000005},"min":{"1min":0.57599999999999996,"5min":0.53400000000000003,"15min":0.45500000000000002},"max":{"1min":0.93500000000000005,"5min":1.153,"15min":1.2629999999999999},"last":0.877}]},{"osd":7,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.81000000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.48599999999999999},"max":{"1min":1.018,"5min":1.2010000000000001,"15min":1.25},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.50600000000000001},"max":{"1min":0.95899999999999996,"5min":1.274,"15min":1.2789999999999999},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575040,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304128,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067712,"statfs":{"total":96636764160,"available":96325337088,"internally_reserved":0,"allocated":2809856,"data_stored":1312517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.435,"15min":0.44800000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27200000000000002},"max":{"1min":0.48099999999999998,"5min":0.86899999999999999,"15min":1.056},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45600000000000002,"15min":0.47699999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":1.0649999999999999},"last":0.439},{"interface":"front","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68500000000000005,"5min":0.80600000000000005,"15min":1.073},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.56999999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.53000000000000003,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.96499999999999997},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77200000000000002,"15min":0.77800000000000002},"min":{"1min":0.56299999999999994,"5min":0.54100000000000004,"15min":0.53700000000000003},"max":{"1min":0.89800000000000002,"5min":1.486,"15min":1.486},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89000000000000001,"5min":1.5,"15min":1.5},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0069999999999999,"15min":1.1639999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.53000000000000003},"max":{"1min":0.86899999999999999,"5min":1.0640000000000001,"15min":1.319},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.77000000000000002,"15min":0.78900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86599999999999999,"5min":1.0309999999999999,"15min":1.5289999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78500000000000003,"15min":0.78800000000000003},"min":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.141},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.52100000000000002},"max":{"1min":0.87,"5min":1.538,"15min":1.538},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78200000000000003,"15min":0.80600000000000005},"min":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97699999999999998,"5min":1.0980000000000001,"15min":1.3520000000000001},"last":0.61599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575041,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304140,"kb_used_data":2756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067700,"statfs":{"total":96636764160,"available":96325324800,"internally_reserved":0,"allocated":2822144,"data_stored":1318675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45200000000000001,"15min":0.45900000000000002},"min":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.57999999999999996,"5min":0.76700000000000002,"15min":2.0950000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":0.77200000000000002,"15min":1.772},"last":0.379}]},{"osd":1,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.369,"15min":1.913},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.97899999999999998,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.72999999999999998,"5min":0.90200000000000002,"15min":1.954},"last":0.501},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56799999999999995,"15min":0.57399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.754,"5min":0.82599999999999996,"15min":2.0539999999999998},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.57199999999999995,"5min":0.51400000000000001,"15min":0.46000000000000002},"max":{"1min":0.86899999999999999,"5min":1.1100000000000001,"15min":1.282},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75700000000000001,"15min":0.77200000000000002},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":0.93799999999999994,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.998}]},{"osd":5,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.51100000000000001},"max":{"1min":0.97799999999999998,"5min":1.111,"15min":1.1930000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77400000000000002,"15min":0.79500000000000004},"min":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.97399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":0.92300000000000004,"5min":1.3360000000000001,"15min":1.343},"last":0.873},{"interface":"front","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":0.93700000000000006,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.98499999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56799999999999995},"max":{"1min":0.91700000000000004,"5min":1.1419999999999999,"15min":1.3640000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53000000000000003},"max":{"1min":0.90900000000000003,"5min":1.0980000000000001,"15min":1.214},"last":0.92600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304140,"kb_used_data":2756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067700,"statfs":{"total":96636764160,"available":96325324800,"internally_reserved":0,"allocated":2822144,"data_stored":1318675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.46500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.92900000000000005,"15min":1.073},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.74399999999999999},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86199999999999999,"5min":1.05,"15min":1.169},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.0569999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.432},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.79200000000000004,"5min":0.96199999999999997,"15min":1.117},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73799999999999999,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":1.196},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.8740000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.375},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.0189999999999999,"5min":1.081,"15min":1.0920000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.45300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.282,"5min":1.282,"15min":1.466},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78800000000000003},"min":{"1min":0.53700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575041,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304128,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067712,"statfs":{"total":96636764160,"available":96325337088,"internally_reserved":0,"allocated":2809856,"data_stored":1312517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.03,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68799999999999994,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":1.0289999999999999,"15min":1.204},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72199999999999998,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":1.091,"15min":1.1000000000000001},"last":0.754}]},{"osd":3,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.49099999999999999,"15min":0.46899999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.087},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.53600000000000003,"5min":0.498,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.145,"15min":1.278},"last":0.623}]},{"osd":4,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":1.1379999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.46300000000000002,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":1.004,"15min":1.294},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.79800000000000004,"5min":0.85899999999999999,"15min":1.141},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.254},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.42399999999999999},"max":{"1min":0.86699999999999999,"5min":0.93899999999999995,"15min":1.337},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72799999999999998,"15min":0.73899999999999999},"min":{"1min":0.51200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.872,"5min":1.0600000000000001,"15min":1.248},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:29:57.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:29:58.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.076+0000 7fee9f59d700 1 Processor -- start 2023-10-06T05:29:58.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.076+0000 7fee9f59d700 1 -- start start 2023-10-06T05:29:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.077+0000 7fee9f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee98155f60 0x7fee980c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.077+0000 7fee9f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9806ab60 con 0x7fee980c8b50 2023-10-06T05:29:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.077+0000 7fee9f59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9806ae40 con 0x7fee98155af0 2023-10-06T05:29:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.077+0000 7fee9f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9806b7e0 con 0x7fee98155f60 2023-10-06T05:29:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.077+0000 7fee9e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fee980c8b50 0x7fee9806e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56194/0 (socket says 172.21.15.33:56194) 2023-10-06T05:29:58.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.077+0000 7fee9e59b700 1 -- 172.21.15.33:0/778519975 learned_addr learned my addr 172.21.15.33:0/778519975 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:29:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/778519975 >> v2:172.21.15.167:3300/0 conn(0x7fee98155f60 0x7fee980c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291400259 0 0) 0x7fee9806ab60 con 0x7fee980c8b50 2023-10-06T05:29:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee84003570 con 0x7fee980c8b50 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467356003 0 0) 0x7fee9806ae40 con 0x7fee98155af0 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9806ab60 con 0x7fee98155af0 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1706589269 0 0) 0x7fee84003570 con 0x7fee980c8b50 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 >> v2:172.21.15.167:3300/0 conn(0x7fee98155f60 msgr2=0x7fee980c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 --2- 172.21.15.33:0/778519975 >> v2:172.21.15.167:3300/0 conn(0x7fee98155f60 0x7fee980c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 >> v1:172.21.15.167:6789/0 conn(0x7fee98155af0 legacy=0x7fee9806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9806b120 con 0x7fee980c8b50 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fee94001470 con 0x7fee980c8b50 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 >> v1:172.21.15.33:6789/0 conn(0x7fee980c8b50 legacy=0x7fee9806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee84003bf0 0x7fee84003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9806b7e0 con 0x7fee840051c0 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400c160 con 0x7fee84004810 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.078+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400c320 con 0x7fee84003bf0 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/778519975 >> v2:172.21.15.167:3300/0 conn(0x7fee98155f60 0x7fee980c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9e59b700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee84003bf0 0x7fee84003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9dd9a700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 -- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee84003bf0 msgr2=0x7fee84003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee84003bf0 0x7fee84003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 -- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84004810 msgr2=0x7fee84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:58.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 -- 172.21.15.33:0/778519975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84003570 con 0x7fee840051c0 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9dd9a700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 -- 172.21.15.33:0/778519975 reap_dead start 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.079+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fee900094d0 tx=0x7fee9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d121c5e6bf8319 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90009cb0 con 0x7fee840051c0 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9000f040 con 0x7fee840051c0 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9d599700 1 -- 172.21.15.33:0/778519975 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9000ee90 con 0x7fee840051c0 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 -- 172.21.15.33:0/778519975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee840051c0 msgr2=0x7fee8400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fee900094d0 tx=0x7fee9000b040 comp rx=0 tx=0).stop 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 -- 172.21.15.33:0/778519975 shutdown_connections 2023-10-06T05:29:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 --2- 172.21.15.33:0/778519975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 -- 172.21.15.33:0/778519975 >> 172.21.15.33:0/778519975 conn(0x7fee9806d030 msgr2=0x7fee9806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:58.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 -- 172.21.15.33:0/778519975 shutdown_connections 2023-10-06T05:29:58.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.080+0000 7fee9f59d700 1 -- 172.21.15.33:0/778519975 wait complete. 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.082+0000 7fee9f59d700 1 Processor -- start 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.082+0000 7fee9f59d700 1 -- start start 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee840051c0 0x7fee980ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee98156dc0 0x7fee980ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 0x7fee980c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980adfd0 con 0x7fee98147c10 2023-10-06T05:29:58.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980c3960 con 0x7fee840051c0 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee98157230 con 0x7fee98156dc0 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 0x7fee980c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 0x7fee980c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50050/0 (socket says 172.21.15.33:50050) 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9ed9c700 1 -- 172.21.15.33:0/2075907393 learned_addr learned my addr 172.21.15.33:0/2075907393 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9e59b700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee840051c0 0x7fee980ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.083+0000 7fee9dd9a700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee98156dc0 0x7fee980ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9ed9c700 1 -- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee98156dc0 msgr2=0x7fee980ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee98156dc0 0x7fee980ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9ed9c700 1 -- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee840051c0 msgr2=0x7fee980ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee840051c0 0x7fee980ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9ed9c700 1 -- 172.21.15.33:0/2075907393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980adcf0 con 0x7fee98147c10 2023-10-06T05:29:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9ed9c700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 0x7fee980c5480 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fee9000afd0 tx=0x7fee90019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9001e650 con 0x7fee98147c10 2023-10-06T05:29:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9001e7f0 con 0x7fee98147c10 2023-10-06T05:29:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9001ebb0 con 0x7fee98147c10 2023-10-06T05:29:58.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980c3310 con 0x7fee98147c10 2023-10-06T05:29:58.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.084+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980acc10 con 0x7fee98147c10 2023-10-06T05:29:58.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.085+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9804c380 con 0x7fee98147c10 2023-10-06T05:29:58.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.086+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee9001d440 con 0x7fee98147c10 2023-10-06T05:29:58.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.087+0000 7fee8e7fc700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee7c066f30 0x7fee7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.087+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fee900868c0 con 0x7fee98147c10 2023-10-06T05:29:58.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.087+0000 7fee9e59b700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee7c066f30 0x7fee7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.088+0000 7fee9e59b700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee7c066f30 0x7fee7c0693e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee94001ed0 tx=0x7fee94007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:58.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.091+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee900539f0 con 0x7fee98147c10 2023-10-06T05:29:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.288+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee980adfd0 con 0x7fee7c066f30 2023-10-06T05:29:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.297+0000 7fee8e7fc700 1 -- 172.21.15.33:0/2075907393 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311478 (secure 0 0 0) 0x7fee980adfd0 con 0x7fee7c066f30 2023-10-06T05:29:58.321 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee7c066f30 msgr2=0x7fee7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee7c066f30 0x7fee7c0693e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee94001ed0 tx=0x7fee94007040 comp rx=0 tx=0).stop 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 msgr2=0x7fee980c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 0x7fee980c5480 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fee9000afd0 tx=0x7fee90019000 comp rx=0 tx=0).stop 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 shutdown_connections 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee7c066f30 0x7fee7c0693e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee98147c10 0x7fee980c5480 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee840051c0 0x7fee980ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 --2- 172.21.15.33:0/2075907393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee98156dc0 0x7fee980ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 >> 172.21.15.33:0/2075907393 conn(0x7fee9806d030 msgr2=0x7fee980cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:58.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 shutdown_connections 2023-10-06T05:29:58.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.302+0000 7fee9f59d700 1 -- 172.21.15.33:0/2075907393 wait complete. 2023-10-06T05:29:58.333 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:29:58.333 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":692,"stamp":"2023-10-06T05:29:58.053146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2434632,"kb_used_data":23560,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752540088,"statfs":{"total":773094113280,"available":770601050112,"internally_reserved":0,"allocated":24125440,"data_stored":11897732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001532"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.493883+0000","last_change":"2023-10-06T05:29:53.492926+0000","last_active":"2023-10-06T05:29:53.493883+0000","last_peered":"2023-10-06T05:29:53.493883+0000","last_clean":"2023-10-06T05:29:53.493883+0000","last_became_active":"2023-10-06T05:29:53.492588+0000","last_became_peered":"2023-10-06T05:29:53.492588+0000","last_unstale":"2023-10-06T05:29:53.493883+0000","last_undegraded":"2023-10-06T05:29:53.493883+0000","last_fullsized":"2023-10-06T05:29:53.493883+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:50.195848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.484418+0000","last_change":"2023-10-06T05:29:53.476803+0000","last_active":"2023-10-06T05:29:53.484418+0000","last_peered":"2023-10-06T05:29:53.484418+0000","last_clean":"2023-10-06T05:29:53.484418+0000","last_became_active":"2023-10-06T05:29:53.476497+0000","last_became_peered":"2023-10-06T05:29:53.476497+0000","last_unstale":"2023-10-06T05:29:53.484418+0000","last_undegraded":"2023-10-06T05:29:53.484418+0000","last_fullsized":"2023-10-06T05:29:53.484418+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:42.148068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.483053+0000","last_change":"2023-10-06T05:29:53.472161+0000","last_active":"2023-10-06T05:29:53.483053+0000","last_peered":"2023-10-06T05:29:53.483053+0000","last_clean":"2023-10-06T05:29:53.483053+0000","last_became_active":"2023-10-06T05:29:53.471870+0000","last_became_peered":"2023-10-06T05:29:53.471870+0000","last_unstale":"2023-10-06T05:29:53.483053+0000","last_undegraded":"2023-10-06T05:29:53.483053+0000","last_fullsized":"2023-10-06T05:29:53.483053+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:05.796047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486325+0000","last_change":"2023-10-06T05:29:53.479768+0000","last_active":"2023-10-06T05:29:53.486325+0000","last_peered":"2023-10-06T05:29:53.486325+0000","last_clean":"2023-10-06T05:29:53.486325+0000","last_became_active":"2023-10-06T05:29:53.479465+0000","last_became_peered":"2023-10-06T05:29:53.479465+0000","last_unstale":"2023-10-06T05:29:53.486325+0000","last_undegraded":"2023-10-06T05:29:53.486325+0000","last_fullsized":"2023-10-06T05:29:53.486325+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:23.206027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.491854+0000","last_change":"2023-10-06T05:29:53.487233+0000","last_active":"2023-10-06T05:29:53.491854+0000","last_peered":"2023-10-06T05:29:53.491854+0000","last_clean":"2023-10-06T05:29:53.491854+0000","last_became_active":"2023-10-06T05:29:53.486939+0000","last_became_peered":"2023-10-06T05:29:53.486939+0000","last_unstale":"2023-10-06T05:29:53.491854+0000","last_undegraded":"2023-10-06T05:29:53.491854+0000","last_fullsized":"2023-10-06T05:29:53.491854+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:50.093258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482054+0000","last_change":"2023-10-06T05:29:53.477072+0000","last_active":"2023-10-06T05:29:53.482054+0000","last_peered":"2023-10-06T05:29:53.482054+0000","last_clean":"2023-10-06T05:29:53.482054+0000","last_became_active":"2023-10-06T05:29:53.476769+0000","last_became_peered":"2023-10-06T05:29:53.476769+0000","last_unstale":"2023-10-06T05:29:53.482054+0000","last_undegraded":"2023-10-06T05:29:53.482054+0000","last_fullsized":"2023-10-06T05:29:53.482054+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:14.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.480544+0000","last_change":"2023-10-06T05:29:53.469678+0000","last_active":"2023-10-06T05:29:53.480544+0000","last_peered":"2023-10-06T05:29:53.480544+0000","last_clean":"2023-10-06T05:29:53.480544+0000","last_became_active":"2023-10-06T05:29:53.469347+0000","last_became_peered":"2023-10-06T05:29:53.469347+0000","last_unstale":"2023-10-06T05:29:53.480544+0000","last_undegraded":"2023-10-06T05:29:53.480544+0000","last_fullsized":"2023-10-06T05:29:53.480544+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:59.851097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.479998+0000","last_change":"2023-10-06T05:29:53.478349+0000","last_active":"2023-10-06T05:29:53.479998+0000","last_peered":"2023-10-06T05:29:53.479998+0000","last_clean":"2023-10-06T05:29:53.479998+0000","last_became_active":"2023-10-06T05:29:53.478059+0000","last_became_peered":"2023-10-06T05:29:53.478059+0000","last_unstale":"2023-10-06T05:29:53.479998+0000","last_undegraded":"2023-10-06T05:29:53.479998+0000","last_fullsized":"2023-10-06T05:29:53.479998+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:06.697393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482009+0000","last_change":"2023-10-06T05:29:53.477217+0000","last_active":"2023-10-06T05:29:53.482009+0000","last_peered":"2023-10-06T05:29:53.482009+0000","last_clean":"2023-10-06T05:29:53.482009+0000","last_became_active":"2023-10-06T05:29:53.476349+0000","last_became_peered":"2023-10-06T05:29:53.476349+0000","last_unstale":"2023-10-06T05:29:53.482009+0000","last_undegraded":"2023-10-06T05:29:53.482009+0000","last_fullsized":"2023-10-06T05:29:53.482009+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:50.093258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486008+0000","last_change":"2023-10-06T05:29:53.482083+0000","last_active":"2023-10-06T05:29:53.486008+0000","last_peered":"2023-10-06T05:29:53.486008+0000","last_clean":"2023-10-06T05:29:53.486008+0000","last_became_active":"2023-10-06T05:29:53.481686+0000","last_became_peered":"2023-10-06T05:29:53.481686+0000","last_unstale":"2023-10-06T05:29:53.486008+0000","last_undegraded":"2023-10-06T05:29:53.486008+0000","last_fullsized":"2023-10-06T05:29:53.486008+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:19.186654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.484583+0000","last_change":"2023-10-06T05:29:53.477762+0000","last_active":"2023-10-06T05:29:53.484583+0000","last_peered":"2023-10-06T05:29:53.484583+0000","last_clean":"2023-10-06T05:29:53.484583+0000","last_became_active":"2023-10-06T05:29:53.477456+0000","last_became_peered":"2023-10-06T05:29:53.477456+0000","last_unstale":"2023-10-06T05:29:53.484583+0000","last_undegraded":"2023-10-06T05:29:53.484583+0000","last_fullsized":"2023-10-06T05:29:53.484583+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:33.340916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482337+0000","last_change":"2023-10-06T05:29:53.479340+0000","last_active":"2023-10-06T05:29:53.482337+0000","last_peered":"2023-10-06T05:29:53.482337+0000","last_clean":"2023-10-06T05:29:53.482337+0000","last_became_active":"2023-10-06T05:29:53.479039+0000","last_became_peered":"2023-10-06T05:29:53.479039+0000","last_unstale":"2023-10-06T05:29:53.482337+0000","last_undegraded":"2023-10-06T05:29:53.482337+0000","last_fullsized":"2023-10-06T05:29:53.482337+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:19.232574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.477705+0000","last_change":"2023-10-06T05:29:53.474851+0000","last_active":"2023-10-06T05:29:53.477705+0000","last_peered":"2023-10-06T05:29:53.477705+0000","last_clean":"2023-10-06T05:29:53.477705+0000","last_became_active":"2023-10-06T05:29:53.474579+0000","last_became_peered":"2023-10-06T05:29:53.474579+0000","last_unstale":"2023-10-06T05:29:53.477705+0000","last_undegraded":"2023-10-06T05:29:53.477705+0000","last_fullsized":"2023-10-06T05:29:53.477705+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:42.148068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.476104+0000","last_change":"2023-10-06T05:29:53.468186+0000","last_active":"2023-10-06T05:29:53.476104+0000","last_peered":"2023-10-06T05:29:53.476104+0000","last_clean":"2023-10-06T05:29:53.476104+0000","last_became_active":"2023-10-06T05:29:53.467923+0000","last_became_peered":"2023-10-06T05:29:53.467923+0000","last_unstale":"2023-10-06T05:29:53.476104+0000","last_undegraded":"2023-10-06T05:29:53.476104+0000","last_fullsized":"2023-10-06T05:29:53.476104+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:13.633512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490552+0000","last_change":"2023-10-06T05:29:53.479515+0000","last_active":"2023-10-06T05:29:53.490552+0000","last_peered":"2023-10-06T05:29:53.490552+0000","last_clean":"2023-10-06T05:29:53.490552+0000","last_became_active":"2023-10-06T05:29:53.478146+0000","last_became_peered":"2023-10-06T05:29:53.478146+0000","last_unstale":"2023-10-06T05:29:53.490552+0000","last_undegraded":"2023-10-06T05:29:53.490552+0000","last_fullsized":"2023-10-06T05:29:53.490552+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:23.831130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.471515+0000","last_change":"2023-10-06T05:29:53.466665+0000","last_active":"2023-10-06T05:29:53.471515+0000","last_peered":"2023-10-06T05:29:53.471515+0000","last_clean":"2023-10-06T05:29:53.471515+0000","last_became_active":"2023-10-06T05:29:53.466383+0000","last_became_peered":"2023-10-06T05:29:53.466383+0000","last_unstale":"2023-10-06T05:29:53.471515+0000","last_undegraded":"2023-10-06T05:29:53.471515+0000","last_fullsized":"2023-10-06T05:29:53.471515+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:14.401722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.479660+0000","last_change":"2023-10-06T05:29:53.475237+0000","last_active":"2023-10-06T05:29:53.479660+0000","last_peered":"2023-10-06T05:29:53.479660+0000","last_clean":"2023-10-06T05:29:53.479660+0000","last_became_active":"2023-10-06T05:29:53.474980+0000","last_became_peered":"2023-10-06T05:29:53.474980+0000","last_unstale":"2023-10-06T05:29:53.479660+0000","last_undegraded":"2023-10-06T05:29:53.479660+0000","last_fullsized":"2023-10-06T05:29:53.479660+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:30.035405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.483380+0000","last_change":"2023-10-06T05:29:53.481852+0000","last_active":"2023-10-06T05:29:53.483380+0000","last_peered":"2023-10-06T05:29:53.483380+0000","last_clean":"2023-10-06T05:29:53.483380+0000","last_became_active":"2023-10-06T05:29:53.481245+0000","last_became_peered":"2023-10-06T05:29:53.481245+0000","last_unstale":"2023-10-06T05:29:53.483380+0000","last_undegraded":"2023-10-06T05:29:53.483380+0000","last_fullsized":"2023-10-06T05:29:53.483380+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:18.706629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.476279+0000","last_change":"2023-10-06T05:29:53.473994+0000","last_active":"2023-10-06T05:29:53.476279+0000","last_peered":"2023-10-06T05:29:53.476279+0000","last_clean":"2023-10-06T05:29:53.476279+0000","last_became_active":"2023-10-06T05:29:53.473667+0000","last_became_peered":"2023-10-06T05:29:53.473667+0000","last_unstale":"2023-10-06T05:29:53.476279+0000","last_undegraded":"2023-10-06T05:29:53.476279+0000","last_fullsized":"2023-10-06T05:29:53.476279+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:56.299802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486299+0000","last_change":"2023-10-06T05:29:53.483261+0000","last_active":"2023-10-06T05:29:53.486299+0000","last_peered":"2023-10-06T05:29:53.486299+0000","last_clean":"2023-10-06T05:29:53.486299+0000","last_became_active":"2023-10-06T05:29:53.482845+0000","last_became_peered":"2023-10-06T05:29:53.482845+0000","last_unstale":"2023-10-06T05:29:53.486299+0000","last_undegraded":"2023-10-06T05:29:53.486299+0000","last_fullsized":"2023-10-06T05:29:53.486299+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:00.131259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.489626+0000","last_change":"2023-10-06T05:29:53.486476+0000","last_active":"2023-10-06T05:29:53.489626+0000","last_peered":"2023-10-06T05:29:53.489626+0000","last_clean":"2023-10-06T05:29:53.489626+0000","last_became_active":"2023-10-06T05:29:53.486059+0000","last_became_peered":"2023-10-06T05:29:53.486059+0000","last_unstale":"2023-10-06T05:29:53.489626+0000","last_undegraded":"2023-10-06T05:29:53.489626+0000","last_fullsized":"2023-10-06T05:29:53.489626+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:14.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.488182+0000","last_change":"2023-10-06T05:29:53.485163+0000","last_active":"2023-10-06T05:29:53.488182+0000","last_peered":"2023-10-06T05:29:53.488182+0000","last_clean":"2023-10-06T05:29:53.488182+0000","last_became_active":"2023-10-06T05:29:53.484859+0000","last_became_peered":"2023-10-06T05:29:53.484859+0000","last_unstale":"2023-10-06T05:29:53.488182+0000","last_undegraded":"2023-10-06T05:29:53.488182+0000","last_fullsized":"2023-10-06T05:29:53.488182+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:04.363721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.491553+0000","last_change":"2023-10-06T05:29:53.489351+0000","last_active":"2023-10-06T05:29:53.491553+0000","last_peered":"2023-10-06T05:29:53.491553+0000","last_clean":"2023-10-06T05:29:53.491553+0000","last_became_active":"2023-10-06T05:29:53.483229+0000","last_became_peered":"2023-10-06T05:29:53.483229+0000","last_unstale":"2023-10-06T05:29:53.491553+0000","last_undegraded":"2023-10-06T05:29:53.491553+0000","last_fullsized":"2023-10-06T05:29:53.491553+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:22.193285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486312+0000","last_change":"2023-10-06T05:29:53.484163+0000","last_active":"2023-10-06T05:29:53.486312+0000","last_peered":"2023-10-06T05:29:53.486312+0000","last_clean":"2023-10-06T05:29:53.486312+0000","last_became_active":"2023-10-06T05:29:53.483013+0000","last_became_peered":"2023-10-06T05:29:53.483013+0000","last_unstale":"2023-10-06T05:29:53.486312+0000","last_undegraded":"2023-10-06T05:29:53.486312+0000","last_fullsized":"2023-10-06T05:29:53.486312+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:42.404230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.483730+0000","last_change":"2023-10-06T05:29:53.479303+0000","last_active":"2023-10-06T05:29:53.483730+0000","last_peered":"2023-10-06T05:29:53.483730+0000","last_clean":"2023-10-06T05:29:53.483730+0000","last_became_active":"2023-10-06T05:29:53.479017+0000","last_became_peered":"2023-10-06T05:29:53.479017+0000","last_unstale":"2023-10-06T05:29:53.483730+0000","last_undegraded":"2023-10-06T05:29:53.483730+0000","last_fullsized":"2023-10-06T05:29:53.483730+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:14.439614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"225'4","reported_seq":14,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.494294+0000","last_change":"2023-10-06T05:29:52.477185+0000","last_active":"2023-10-06T05:29:53.494294+0000","last_peered":"2023-10-06T05:29:53.494294+0000","last_clean":"2023-10-06T05:29:53.494294+0000","last_became_active":"2023-10-06T05:29:52.476905+0000","last_became_peered":"2023-10-06T05:29:52.476905+0000","last_unstale":"2023-10-06T05:29:53.494294+0000","last_undegraded":"2023-10-06T05:29:53.494294+0000","last_fullsized":"2023-10-06T05:29:53.494294+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:00.157448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.476840+0000","last_change":"2023-10-06T05:29:53.474231+0000","last_active":"2023-10-06T05:29:53.476840+0000","last_peered":"2023-10-06T05:29:53.476840+0000","last_clean":"2023-10-06T05:29:53.476840+0000","last_became_active":"2023-10-06T05:29:53.473933+0000","last_became_peered":"2023-10-06T05:29:53.473933+0000","last_unstale":"2023-10-06T05:29:53.476840+0000","last_undegraded":"2023-10-06T05:29:53.476840+0000","last_fullsized":"2023-10-06T05:29:53.476840+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:52.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"225'1","reported_seq":11,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.488869+0000","last_change":"2023-10-06T05:29:52.467812+0000","last_active":"2023-10-06T05:29:53.488869+0000","last_peered":"2023-10-06T05:29:53.488869+0000","last_clean":"2023-10-06T05:29:53.488869+0000","last_became_active":"2023-10-06T05:29:52.467501+0000","last_became_peered":"2023-10-06T05:29:52.467501+0000","last_unstale":"2023-10-06T05:29:53.488869+0000","last_undegraded":"2023-10-06T05:29:53.488869+0000","last_fullsized":"2023-10-06T05:29:53.488869+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:36.725638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.476877+0000","last_change":"2023-10-06T05:29:53.473252+0000","last_active":"2023-10-06T05:29:53.476877+0000","last_peered":"2023-10-06T05:29:53.476877+0000","last_clean":"2023-10-06T05:29:53.476877+0000","last_became_active":"2023-10-06T05:29:53.472903+0000","last_became_peered":"2023-10-06T05:29:53.472903+0000","last_unstale":"2023-10-06T05:29:53.476877+0000","last_undegraded":"2023-10-06T05:29:53.476877+0000","last_fullsized":"2023-10-06T05:29:53.476877+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:45.273332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"225'2","reported_seq":12,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.495295+0000","last_change":"2023-10-06T05:29:52.479331+0000","last_active":"2023-10-06T05:29:53.495295+0000","last_peered":"2023-10-06T05:29:53.495295+0000","last_clean":"2023-10-06T05:29:53.495295+0000","last_became_active":"2023-10-06T05:29:52.479120+0000","last_became_peered":"2023-10-06T05:29:52.479120+0000","last_unstale":"2023-10-06T05:29:53.495295+0000","last_undegraded":"2023-10-06T05:29:53.495295+0000","last_fullsized":"2023-10-06T05:29:53.495295+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:00.946705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.481884+0000","last_change":"2023-10-06T05:29:53.473123+0000","last_active":"2023-10-06T05:29:53.481884+0000","last_peered":"2023-10-06T05:29:53.481884+0000","last_clean":"2023-10-06T05:29:53.481884+0000","last_became_active":"2023-10-06T05:29:53.472878+0000","last_became_peered":"2023-10-06T05:29:53.472878+0000","last_unstale":"2023-10-06T05:29:53.481884+0000","last_undegraded":"2023-10-06T05:29:53.481884+0000","last_fullsized":"2023-10-06T05:29:53.481884+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:02.558739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"225'2","reported_seq":12,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.495079+0000","last_change":"2023-10-06T05:29:52.468893+0000","last_active":"2023-10-06T05:29:53.495079+0000","last_peered":"2023-10-06T05:29:53.495079+0000","last_clean":"2023-10-06T05:29:53.495079+0000","last_became_active":"2023-10-06T05:29:52.468636+0000","last_became_peered":"2023-10-06T05:29:52.468636+0000","last_unstale":"2023-10-06T05:29:53.495079+0000","last_undegraded":"2023-10-06T05:29:53.495079+0000","last_fullsized":"2023-10-06T05:29:53.495079+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:43.350406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490487+0000","last_change":"2023-10-06T05:29:53.482597+0000","last_active":"2023-10-06T05:29:53.490487+0000","last_peered":"2023-10-06T05:29:53.490487+0000","last_clean":"2023-10-06T05:29:53.490487+0000","last_became_active":"2023-10-06T05:29:53.482307+0000","last_became_peered":"2023-10-06T05:29:53.482307+0000","last_unstale":"2023-10-06T05:29:53.490487+0000","last_undegraded":"2023-10-06T05:29:53.490487+0000","last_fullsized":"2023-10-06T05:29:53.490487+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:00.964897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"225'1","reported_seq":11,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.498305+0000","last_change":"2023-10-06T05:29:52.472867+0000","last_active":"2023-10-06T05:29:53.498305+0000","last_peered":"2023-10-06T05:29:53.498305+0000","last_clean":"2023-10-06T05:29:53.498305+0000","last_became_active":"2023-10-06T05:29:52.472577+0000","last_became_peered":"2023-10-06T05:29:52.472577+0000","last_unstale":"2023-10-06T05:29:53.498305+0000","last_undegraded":"2023-10-06T05:29:53.498305+0000","last_fullsized":"2023-10-06T05:29:53.498305+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:02.256682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490569+0000","last_change":"2023-10-06T05:29:53.483523+0000","last_active":"2023-10-06T05:29:53.490569+0000","last_peered":"2023-10-06T05:29:53.490569+0000","last_clean":"2023-10-06T05:29:53.490569+0000","last_became_active":"2023-10-06T05:29:53.479218+0000","last_became_peered":"2023-10-06T05:29:53.479218+0000","last_unstale":"2023-10-06T05:29:53.490569+0000","last_undegraded":"2023-10-06T05:29:53.490569+0000","last_fullsized":"2023-10-06T05:29:53.490569+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:23.206027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"225'1","reported_seq":11,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490619+0000","last_change":"2023-10-06T05:29:52.467609+0000","last_active":"2023-10-06T05:29:53.490619+0000","last_peered":"2023-10-06T05:29:53.490619+0000","last_clean":"2023-10-06T05:29:53.490619+0000","last_became_active":"2023-10-06T05:29:52.467331+0000","last_became_peered":"2023-10-06T05:29:52.467331+0000","last_unstale":"2023-10-06T05:29:53.490619+0000","last_undegraded":"2023-10-06T05:29:53.490619+0000","last_fullsized":"2023-10-06T05:29:53.490619+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:09.748345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.481673+0000","last_change":"2023-10-06T05:29:53.473710+0000","last_active":"2023-10-06T05:29:53.481673+0000","last_peered":"2023-10-06T05:29:53.481673+0000","last_clean":"2023-10-06T05:29:53.481673+0000","last_became_active":"2023-10-06T05:29:53.473352+0000","last_became_peered":"2023-10-06T05:29:53.473352+0000","last_unstale":"2023-10-06T05:29:53.481673+0000","last_undegraded":"2023-10-06T05:29:53.481673+0000","last_fullsized":"2023-10-06T05:29:53.481673+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:33.340916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.458470+0000","last_change":"2023-10-06T05:29:52.470333+0000","last_active":"2023-10-06T05:29:53.458470+0000","last_peered":"2023-10-06T05:29:53.458470+0000","last_clean":"2023-10-06T05:29:53.458470+0000","last_became_active":"2023-10-06T05:29:52.469904+0000","last_became_peered":"2023-10-06T05:29:52.469904+0000","last_unstale":"2023-10-06T05:29:53.458470+0000","last_undegraded":"2023-10-06T05:29:53.458470+0000","last_fullsized":"2023-10-06T05:29:53.458470+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:22.709835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.477684+0000","last_change":"2023-10-06T05:29:53.472357+0000","last_active":"2023-10-06T05:29:53.477684+0000","last_peered":"2023-10-06T05:29:53.477684+0000","last_clean":"2023-10-06T05:29:53.477684+0000","last_became_active":"2023-10-06T05:29:53.472021+0000","last_became_peered":"2023-10-06T05:29:53.472021+0000","last_unstale":"2023-10-06T05:29:53.477684+0000","last_undegraded":"2023-10-06T05:29:53.477684+0000","last_fullsized":"2023-10-06T05:29:53.477684+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:33.213882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.470447+0000","last_change":"2023-10-06T05:29:52.474698+0000","last_active":"2023-10-06T05:29:53.470447+0000","last_peered":"2023-10-06T05:29:53.470447+0000","last_clean":"2023-10-06T05:29:53.470447+0000","last_became_active":"2023-10-06T05:29:52.474438+0000","last_became_peered":"2023-10-06T05:29:52.474438+0000","last_unstale":"2023-10-06T05:29:53.470447+0000","last_undegraded":"2023-10-06T05:29:53.470447+0000","last_fullsized":"2023-10-06T05:29:53.470447+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:11.611677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.478060+0000","last_change":"2023-10-06T05:29:53.473688+0000","last_active":"2023-10-06T05:29:53.478060+0000","last_peered":"2023-10-06T05:29:53.478060+0000","last_clean":"2023-10-06T05:29:53.478060+0000","last_became_active":"2023-10-06T05:29:53.473420+0000","last_became_peered":"2023-10-06T05:29:53.473420+0000","last_unstale":"2023-10-06T05:29:53.478060+0000","last_undegraded":"2023-10-06T05:29:53.478060+0000","last_fullsized":"2023-10-06T05:29:53.478060+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:00.964897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.459048+0000","last_change":"2023-10-06T05:29:52.471078+0000","last_active":"2023-10-06T05:29:53.459048+0000","last_peered":"2023-10-06T05:29:53.459048+0000","last_clean":"2023-10-06T05:29:53.459048+0000","last_became_active":"2023-10-06T05:29:52.470807+0000","last_became_peered":"2023-10-06T05:29:52.470807+0000","last_unstale":"2023-10-06T05:29:53.459048+0000","last_undegraded":"2023-10-06T05:29:53.459048+0000","last_fullsized":"2023-10-06T05:29:53.459048+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:18.315706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.476822+0000","last_change":"2023-10-06T05:29:53.473888+0000","last_active":"2023-10-06T05:29:53.476822+0000","last_peered":"2023-10-06T05:29:53.476822+0000","last_clean":"2023-10-06T05:29:53.476822+0000","last_became_active":"2023-10-06T05:29:53.473570+0000","last_became_peered":"2023-10-06T05:29:53.473570+0000","last_unstale":"2023-10-06T05:29:53.476822+0000","last_undegraded":"2023-10-06T05:29:53.476822+0000","last_fullsized":"2023-10-06T05:29:53.476822+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:42.613477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.459134+0000","last_change":"2023-10-06T05:29:52.464924+0000","last_active":"2023-10-06T05:29:53.459134+0000","last_peered":"2023-10-06T05:29:53.459134+0000","last_clean":"2023-10-06T05:29:53.459134+0000","last_became_active":"2023-10-06T05:29:52.464564+0000","last_became_peered":"2023-10-06T05:29:52.464564+0000","last_unstale":"2023-10-06T05:29:53.459134+0000","last_undegraded":"2023-10-06T05:29:53.459134+0000","last_fullsized":"2023-10-06T05:29:53.459134+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:00.626310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482261+0000","last_change":"2023-10-06T05:29:53.477525+0000","last_active":"2023-10-06T05:29:53.482261+0000","last_peered":"2023-10-06T05:29:53.482261+0000","last_clean":"2023-10-06T05:29:53.482261+0000","last_became_active":"2023-10-06T05:29:53.477272+0000","last_became_peered":"2023-10-06T05:29:53.477272+0000","last_unstale":"2023-10-06T05:29:53.482261+0000","last_undegraded":"2023-10-06T05:29:53.482261+0000","last_fullsized":"2023-10-06T05:29:53.482261+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:20.193266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"225'3","reported_seq":13,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.500673+0000","last_change":"2023-10-06T05:29:52.469383+0000","last_active":"2023-10-06T05:29:53.500673+0000","last_peered":"2023-10-06T05:29:53.500673+0000","last_clean":"2023-10-06T05:29:53.500673+0000","last_became_active":"2023-10-06T05:29:52.469113+0000","last_became_peered":"2023-10-06T05:29:52.469113+0000","last_unstale":"2023-10-06T05:29:53.500673+0000","last_undegraded":"2023-10-06T05:29:53.500673+0000","last_fullsized":"2023-10-06T05:29:53.500673+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:20.009885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486095+0000","last_change":"2023-10-06T05:29:53.478136+0000","last_active":"2023-10-06T05:29:53.486095+0000","last_peered":"2023-10-06T05:29:53.486095+0000","last_clean":"2023-10-06T05:29:53.486095+0000","last_became_active":"2023-10-06T05:29:53.477779+0000","last_became_peered":"2023-10-06T05:29:53.477779+0000","last_unstale":"2023-10-06T05:29:53.486095+0000","last_undegraded":"2023-10-06T05:29:53.486095+0000","last_fullsized":"2023-10-06T05:29:53.486095+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:12.978914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"225'6","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.501101+0000","last_change":"2023-10-06T05:29:52.473744+0000","last_active":"2023-10-06T05:29:53.501101+0000","last_peered":"2023-10-06T05:29:53.501101+0000","last_clean":"2023-10-06T05:29:53.501101+0000","last_became_active":"2023-10-06T05:29:52.473354+0000","last_became_peered":"2023-10-06T05:29:52.473354+0000","last_unstale":"2023-10-06T05:29:53.501101+0000","last_undegraded":"2023-10-06T05:29:53.501101+0000","last_fullsized":"2023-10-06T05:29:53.501101+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:44.670064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.484521+0000","last_change":"2023-10-06T05:29:53.475945+0000","last_active":"2023-10-06T05:29:53.484521+0000","last_peered":"2023-10-06T05:29:53.484521+0000","last_clean":"2023-10-06T05:29:53.484521+0000","last_became_active":"2023-10-06T05:29:53.475576+0000","last_became_peered":"2023-10-06T05:29:53.475576+0000","last_unstale":"2023-10-06T05:29:53.484521+0000","last_undegraded":"2023-10-06T05:29:53.484521+0000","last_fullsized":"2023-10-06T05:29:53.484521+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:09.743580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"225'2","reported_seq":12,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.498412+0000","last_change":"2023-10-06T05:29:52.475485+0000","last_active":"2023-10-06T05:29:53.498412+0000","last_peered":"2023-10-06T05:29:53.498412+0000","last_clean":"2023-10-06T05:29:53.498412+0000","last_became_active":"2023-10-06T05:29:52.475154+0000","last_became_peered":"2023-10-06T05:29:52.475154+0000","last_unstale":"2023-10-06T05:29:53.498412+0000","last_undegraded":"2023-10-06T05:29:53.498412+0000","last_fullsized":"2023-10-06T05:29:53.498412+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:47.711826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.493894+0000","last_change":"2023-10-06T05:29:53.492990+0000","last_active":"2023-10-06T05:29:53.493894+0000","last_peered":"2023-10-06T05:29:53.493894+0000","last_clean":"2023-10-06T05:29:53.493894+0000","last_became_active":"2023-10-06T05:29:53.492542+0000","last_became_peered":"2023-10-06T05:29:53.492542+0000","last_unstale":"2023-10-06T05:29:53.493894+0000","last_undegraded":"2023-10-06T05:29:53.493894+0000","last_fullsized":"2023-10-06T05:29:53.493894+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:31.528191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.460332+0000","last_change":"2023-10-06T05:29:52.471480+0000","last_active":"2023-10-06T05:29:53.460332+0000","last_peered":"2023-10-06T05:29:53.460332+0000","last_clean":"2023-10-06T05:29:53.460332+0000","last_became_active":"2023-10-06T05:29:52.470072+0000","last_became_peered":"2023-10-06T05:29:52.470072+0000","last_unstale":"2023-10-06T05:29:53.460332+0000","last_undegraded":"2023-10-06T05:29:53.460332+0000","last_fullsized":"2023-10-06T05:29:53.460332+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:19.851660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.485309+0000","last_change":"2023-10-06T05:29:53.483927+0000","last_active":"2023-10-06T05:29:53.485309+0000","last_peered":"2023-10-06T05:29:53.485309+0000","last_clean":"2023-10-06T05:29:53.485309+0000","last_became_active":"2023-10-06T05:29:53.483645+0000","last_became_peered":"2023-10-06T05:29:53.483645+0000","last_unstale":"2023-10-06T05:29:53.485309+0000","last_undegraded":"2023-10-06T05:29:53.485309+0000","last_fullsized":"2023-10-06T05:29:53.485309+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:23.097375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.458800+0000","last_change":"2023-10-06T05:29:52.465259+0000","last_active":"2023-10-06T05:29:53.458800+0000","last_peered":"2023-10-06T05:29:53.458800+0000","last_clean":"2023-10-06T05:29:53.458800+0000","last_became_active":"2023-10-06T05:29:52.464759+0000","last_became_peered":"2023-10-06T05:29:52.464759+0000","last_unstale":"2023-10-06T05:29:53.458800+0000","last_undegraded":"2023-10-06T05:29:53.458800+0000","last_fullsized":"2023-10-06T05:29:53.458800+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:43.834156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.489605+0000","last_change":"2023-10-06T05:29:53.486307+0000","last_active":"2023-10-06T05:29:53.489605+0000","last_peered":"2023-10-06T05:29:53.489605+0000","last_clean":"2023-10-06T05:29:53.489605+0000","last_became_active":"2023-10-06T05:29:53.485702+0000","last_became_peered":"2023-10-06T05:29:53.485702+0000","last_unstale":"2023-10-06T05:29:53.489605+0000","last_undegraded":"2023-10-06T05:29:53.489605+0000","last_fullsized":"2023-10-06T05:29:53.489605+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:00.131259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"225'8","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.501749+0000","last_change":"2023-10-06T05:29:52.461350+0000","last_active":"2023-10-06T05:29:53.501749+0000","last_peered":"2023-10-06T05:29:53.501749+0000","last_clean":"2023-10-06T05:29:53.501749+0000","last_became_active":"2023-10-06T05:29:52.460985+0000","last_became_peered":"2023-10-06T05:29:52.460985+0000","last_unstale":"2023-10-06T05:29:53.501749+0000","last_undegraded":"2023-10-06T05:29:53.501749+0000","last_fullsized":"2023-10-06T05:29:53.501749+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:06.356744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486196+0000","last_change":"2023-10-06T05:29:53.481472+0000","last_active":"2023-10-06T05:29:53.486196+0000","last_peered":"2023-10-06T05:29:53.486196+0000","last_clean":"2023-10-06T05:29:53.486196+0000","last_became_active":"2023-10-06T05:29:53.481131+0000","last_became_peered":"2023-10-06T05:29:53.481131+0000","last_unstale":"2023-10-06T05:29:53.486196+0000","last_undegraded":"2023-10-06T05:29:53.486196+0000","last_fullsized":"2023-10-06T05:29:53.486196+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:30.652663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"225'1","reported_seq":11,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.496737+0000","last_change":"2023-10-06T05:29:52.480362+0000","last_active":"2023-10-06T05:29:53.496737+0000","last_peered":"2023-10-06T05:29:53.496737+0000","last_clean":"2023-10-06T05:29:53.496737+0000","last_became_active":"2023-10-06T05:29:52.480023+0000","last_became_peered":"2023-10-06T05:29:52.480023+0000","last_unstale":"2023-10-06T05:29:53.496737+0000","last_undegraded":"2023-10-06T05:29:53.496737+0000","last_fullsized":"2023-10-06T05:29:53.496737+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:00.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.487987+0000","last_change":"2023-10-06T05:29:53.484705+0000","last_active":"2023-10-06T05:29:53.487987+0000","last_peered":"2023-10-06T05:29:53.487987+0000","last_clean":"2023-10-06T05:29:53.487987+0000","last_became_active":"2023-10-06T05:29:53.484343+0000","last_became_peered":"2023-10-06T05:29:53.484343+0000","last_unstale":"2023-10-06T05:29:53.487987+0000","last_undegraded":"2023-10-06T05:29:53.487987+0000","last_fullsized":"2023-10-06T05:29:53.487987+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:05.796047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"225'2","reported_seq":12,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.492059+0000","last_change":"2023-10-06T05:29:52.475125+0000","last_active":"2023-10-06T05:29:53.492059+0000","last_peered":"2023-10-06T05:29:53.492059+0000","last_clean":"2023-10-06T05:29:53.492059+0000","last_became_active":"2023-10-06T05:29:52.474550+0000","last_became_peered":"2023-10-06T05:29:52.474550+0000","last_unstale":"2023-10-06T05:29:53.492059+0000","last_undegraded":"2023-10-06T05:29:53.492059+0000","last_fullsized":"2023-10-06T05:29:53.492059+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:00.816259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.487771+0000","last_change":"2023-10-06T05:29:53.480212+0000","last_active":"2023-10-06T05:29:53.487771+0000","last_peered":"2023-10-06T05:29:53.487771+0000","last_clean":"2023-10-06T05:29:53.487771+0000","last_became_active":"2023-10-06T05:29:53.479925+0000","last_became_peered":"2023-10-06T05:29:53.479925+0000","last_unstale":"2023-10-06T05:29:53.487771+0000","last_undegraded":"2023-10-06T05:29:53.487771+0000","last_fullsized":"2023-10-06T05:29:53.487771+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:56.238907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"225'1","reported_seq":11,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.493020+0000","last_change":"2023-10-06T05:29:52.481456+0000","last_active":"2023-10-06T05:29:53.493020+0000","last_peered":"2023-10-06T05:29:53.493020+0000","last_clean":"2023-10-06T05:29:53.493020+0000","last_became_active":"2023-10-06T05:29:52.481190+0000","last_became_peered":"2023-10-06T05:29:52.481190+0000","last_unstale":"2023-10-06T05:29:53.493020+0000","last_undegraded":"2023-10-06T05:29:53.493020+0000","last_fullsized":"2023-10-06T05:29:53.493020+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:54.721830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.488135+0000","last_change":"2023-10-06T05:29:53.485837+0000","last_active":"2023-10-06T05:29:53.488135+0000","last_peered":"2023-10-06T05:29:53.488135+0000","last_clean":"2023-10-06T05:29:53.488135+0000","last_became_active":"2023-10-06T05:29:53.485111+0000","last_became_peered":"2023-10-06T05:29:53.485111+0000","last_unstale":"2023-10-06T05:29:53.488135+0000","last_undegraded":"2023-10-06T05:29:53.488135+0000","last_fullsized":"2023-10-06T05:29:53.488135+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:26.361103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.465252+0000","last_change":"2023-10-06T05:29:52.477797+0000","last_active":"2023-10-06T05:29:53.465252+0000","last_peered":"2023-10-06T05:29:53.465252+0000","last_clean":"2023-10-06T05:29:53.465252+0000","last_became_active":"2023-10-06T05:29:52.477497+0000","last_became_peered":"2023-10-06T05:29:52.477497+0000","last_unstale":"2023-10-06T05:29:53.465252+0000","last_undegraded":"2023-10-06T05:29:53.465252+0000","last_fullsized":"2023-10-06T05:29:53.465252+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:59.591328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482364+0000","last_change":"2023-10-06T05:29:53.471006+0000","last_active":"2023-10-06T05:29:53.482364+0000","last_peered":"2023-10-06T05:29:53.482364+0000","last_clean":"2023-10-06T05:29:53.482364+0000","last_became_active":"2023-10-06T05:29:53.470702+0000","last_became_peered":"2023-10-06T05:29:53.470702+0000","last_unstale":"2023-10-06T05:29:53.482364+0000","last_undegraded":"2023-10-06T05:29:53.482364+0000","last_fullsized":"2023-10-06T05:29:53.482364+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:33.213882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"225'2","reported_seq":12,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.500855+0000","last_change":"2023-10-06T05:29:52.478219+0000","last_active":"2023-10-06T05:29:53.500855+0000","last_peered":"2023-10-06T05:29:53.500855+0000","last_clean":"2023-10-06T05:29:53.500855+0000","last_became_active":"2023-10-06T05:29:52.477918+0000","last_became_peered":"2023-10-06T05:29:52.477918+0000","last_unstale":"2023-10-06T05:29:53.500855+0000","last_undegraded":"2023-10-06T05:29:53.500855+0000","last_fullsized":"2023-10-06T05:29:53.500855+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:37.867026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486141+0000","last_change":"2023-10-06T05:29:53.479136+0000","last_active":"2023-10-06T05:29:53.486141+0000","last_peered":"2023-10-06T05:29:53.486141+0000","last_clean":"2023-10-06T05:29:53.486141+0000","last_became_active":"2023-10-06T05:29:53.478854+0000","last_became_peered":"2023-10-06T05:29:53.478854+0000","last_unstale":"2023-10-06T05:29:53.486141+0000","last_undegraded":"2023-10-06T05:29:53.486141+0000","last_fullsized":"2023-10-06T05:29:53.486141+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:49.538123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.468597+0000","last_change":"2023-10-06T05:29:52.481544+0000","last_active":"2023-10-06T05:29:53.468597+0000","last_peered":"2023-10-06T05:29:53.468597+0000","last_clean":"2023-10-06T05:29:53.468597+0000","last_became_active":"2023-10-06T05:29:52.481058+0000","last_became_peered":"2023-10-06T05:29:52.481058+0000","last_unstale":"2023-10-06T05:29:53.468597+0000","last_undegraded":"2023-10-06T05:29:53.468597+0000","last_fullsized":"2023-10-06T05:29:53.468597+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:51.589632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.477497+0000","last_change":"2023-10-06T05:29:53.473052+0000","last_active":"2023-10-06T05:29:53.477497+0000","last_peered":"2023-10-06T05:29:53.477497+0000","last_clean":"2023-10-06T05:29:53.477497+0000","last_became_active":"2023-10-06T05:29:53.472781+0000","last_became_peered":"2023-10-06T05:29:53.472781+0000","last_unstale":"2023-10-06T05:29:53.477497+0000","last_undegraded":"2023-10-06T05:29:53.477497+0000","last_fullsized":"2023-10-06T05:29:53.477497+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:13.633512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.459418+0000","last_change":"2023-10-06T05:29:52.462192+0000","last_active":"2023-10-06T05:29:53.459418+0000","last_peered":"2023-10-06T05:29:53.459418+0000","last_clean":"2023-10-06T05:29:53.459418+0000","last_became_active":"2023-10-06T05:29:52.461855+0000","last_became_peered":"2023-10-06T05:29:52.461855+0000","last_unstale":"2023-10-06T05:29:53.459418+0000","last_undegraded":"2023-10-06T05:29:53.459418+0000","last_fullsized":"2023-10-06T05:29:53.459418+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:34.922753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490432+0000","last_change":"2023-10-06T05:29:53.482720+0000","last_active":"2023-10-06T05:29:53.490432+0000","last_peered":"2023-10-06T05:29:53.490432+0000","last_clean":"2023-10-06T05:29:53.490432+0000","last_became_active":"2023-10-06T05:29:53.481868+0000","last_became_peered":"2023-10-06T05:29:53.481868+0000","last_unstale":"2023-10-06T05:29:53.490432+0000","last_undegraded":"2023-10-06T05:29:53.490432+0000","last_fullsized":"2023-10-06T05:29:53.490432+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:08.293479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.460155+0000","last_change":"2023-10-06T05:29:52.474334+0000","last_active":"2023-10-06T05:29:53.460155+0000","last_peered":"2023-10-06T05:29:53.460155+0000","last_clean":"2023-10-06T05:29:53.460155+0000","last_became_active":"2023-10-06T05:29:52.472214+0000","last_became_peered":"2023-10-06T05:29:52.472214+0000","last_unstale":"2023-10-06T05:29:53.460155+0000","last_undegraded":"2023-10-06T05:29:53.460155+0000","last_fullsized":"2023-10-06T05:29:53.460155+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:14.596125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.484229+0000","last_change":"2023-10-06T05:29:53.482096+0000","last_active":"2023-10-06T05:29:53.484229+0000","last_peered":"2023-10-06T05:29:53.484229+0000","last_clean":"2023-10-06T05:29:53.484229+0000","last_became_active":"2023-10-06T05:29:53.481756+0000","last_became_peered":"2023-10-06T05:29:53.481756+0000","last_unstale":"2023-10-06T05:29:53.484229+0000","last_undegraded":"2023-10-06T05:29:53.484229+0000","last_fullsized":"2023-10-06T05:29:53.484229+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:36.783707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"225'2","reported_seq":12,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.500585+0000","last_change":"2023-10-06T05:29:52.472551+0000","last_active":"2023-10-06T05:29:53.500585+0000","last_peered":"2023-10-06T05:29:53.500585+0000","last_clean":"2023-10-06T05:29:53.500585+0000","last_became_active":"2023-10-06T05:29:52.472203+0000","last_became_peered":"2023-10-06T05:29:52.472203+0000","last_unstale":"2023-10-06T05:29:53.500585+0000","last_undegraded":"2023-10-06T05:29:53.500585+0000","last_fullsized":"2023-10-06T05:29:53.500585+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:30.269795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482320+0000","last_change":"2023-10-06T05:29:53.478221+0000","last_active":"2023-10-06T05:29:53.482320+0000","last_peered":"2023-10-06T05:29:53.482320+0000","last_clean":"2023-10-06T05:29:53.482320+0000","last_became_active":"2023-10-06T05:29:53.477972+0000","last_became_peered":"2023-10-06T05:29:53.477972+0000","last_unstale":"2023-10-06T05:29:53.482320+0000","last_undegraded":"2023-10-06T05:29:53.482320+0000","last_fullsized":"2023-10-06T05:29:53.482320+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:09.743580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"226'1","reported_seq":11,"reported_epoch":226,"state":"active+clean","last_fresh":"2023-10-06T05:29:54.478988+0000","last_change":"2023-10-06T05:29:52.478450+0000","last_active":"2023-10-06T05:29:54.478988+0000","last_peered":"2023-10-06T05:29:54.478988+0000","last_clean":"2023-10-06T05:29:54.478988+0000","last_became_active":"2023-10-06T05:29:52.478147+0000","last_became_peered":"2023-10-06T05:29:52.478147+0000","last_unstale":"2023-10-06T05:29:54.478988+0000","last_undegraded":"2023-10-06T05:29:54.478988+0000","last_fullsized":"2023-10-06T05:29:54.478988+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.415584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.485363+0000","last_change":"2023-10-06T05:29:53.482304+0000","last_active":"2023-10-06T05:29:53.485363+0000","last_peered":"2023-10-06T05:29:53.485363+0000","last_clean":"2023-10-06T05:29:53.485363+0000","last_became_active":"2023-10-06T05:29:53.482022+0000","last_became_peered":"2023-10-06T05:29:53.482022+0000","last_unstale":"2023-10-06T05:29:53.485363+0000","last_undegraded":"2023-10-06T05:29:53.485363+0000","last_fullsized":"2023-10-06T05:29:53.485363+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:53.336407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.464908+0000","last_change":"2023-10-06T05:29:52.469620+0000","last_active":"2023-10-06T05:29:53.464908+0000","last_peered":"2023-10-06T05:29:53.464908+0000","last_clean":"2023-10-06T05:29:53.464908+0000","last_became_active":"2023-10-06T05:29:52.469053+0000","last_became_peered":"2023-10-06T05:29:52.469053+0000","last_unstale":"2023-10-06T05:29:53.464908+0000","last_undegraded":"2023-10-06T05:29:53.464908+0000","last_fullsized":"2023-10-06T05:29:53.464908+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:00.946705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.482179+0000","last_change":"2023-10-06T05:29:53.477753+0000","last_active":"2023-10-06T05:29:53.482179+0000","last_peered":"2023-10-06T05:29:53.482179+0000","last_clean":"2023-10-06T05:29:53.482179+0000","last_became_active":"2023-10-06T05:29:53.477507+0000","last_became_peered":"2023-10-06T05:29:53.477507+0000","last_unstale":"2023-10-06T05:29:53.482179+0000","last_undegraded":"2023-10-06T05:29:53.482179+0000","last_fullsized":"2023-10-06T05:29:53.482179+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:23.097375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.464565+0000","last_change":"2023-10-06T05:29:52.481196+0000","last_active":"2023-10-06T05:29:53.464565+0000","last_peered":"2023-10-06T05:29:53.464565+0000","last_clean":"2023-10-06T05:29:53.464565+0000","last_became_active":"2023-10-06T05:29:52.480957+0000","last_became_peered":"2023-10-06T05:29:52.480957+0000","last_unstale":"2023-10-06T05:29:53.464565+0000","last_undegraded":"2023-10-06T05:29:53.464565+0000","last_fullsized":"2023-10-06T05:29:53.464565+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:52.048646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.487424+0000","last_change":"2023-10-06T05:29:53.479610+0000","last_active":"2023-10-06T05:29:53.487424+0000","last_peered":"2023-10-06T05:29:53.487424+0000","last_clean":"2023-10-06T05:29:53.487424+0000","last_became_active":"2023-10-06T05:29:53.479341+0000","last_became_peered":"2023-10-06T05:29:53.479341+0000","last_unstale":"2023-10-06T05:29:53.487424+0000","last_undegraded":"2023-10-06T05:29:53.487424+0000","last_fullsized":"2023-10-06T05:29:53.487424+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:23.831130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.467133+0000","last_change":"2023-10-06T05:29:52.478638+0000","last_active":"2023-10-06T05:29:53.467133+0000","last_peered":"2023-10-06T05:29:53.467133+0000","last_clean":"2023-10-06T05:29:53.467133+0000","last_became_active":"2023-10-06T05:29:52.475554+0000","last_became_peered":"2023-10-06T05:29:52.475554+0000","last_unstale":"2023-10-06T05:29:53.467133+0000","last_undegraded":"2023-10-06T05:29:53.467133+0000","last_fullsized":"2023-10-06T05:29:53.467133+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:06.230624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490606+0000","last_change":"2023-10-06T05:29:53.488722+0000","last_active":"2023-10-06T05:29:53.490606+0000","last_peered":"2023-10-06T05:29:53.490606+0000","last_clean":"2023-10-06T05:29:53.490606+0000","last_became_active":"2023-10-06T05:29:53.479234+0000","last_became_peered":"2023-10-06T05:29:53.479234+0000","last_unstale":"2023-10-06T05:29:53.490606+0000","last_undegraded":"2023-10-06T05:29:53.490606+0000","last_fullsized":"2023-10-06T05:29:53.490606+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:41.503136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.466631+0000","last_change":"2023-10-06T05:29:52.468614+0000","last_active":"2023-10-06T05:29:53.466631+0000","last_peered":"2023-10-06T05:29:53.466631+0000","last_clean":"2023-10-06T05:29:53.466631+0000","last_became_active":"2023-10-06T05:29:52.468163+0000","last_became_peered":"2023-10-06T05:29:52.468163+0000","last_unstale":"2023-10-06T05:29:53.466631+0000","last_undegraded":"2023-10-06T05:29:53.466631+0000","last_fullsized":"2023-10-06T05:29:53.466631+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:19.799372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.476129+0000","last_change":"2023-10-06T05:29:53.470534+0000","last_active":"2023-10-06T05:29:53.476129+0000","last_peered":"2023-10-06T05:29:53.476129+0000","last_clean":"2023-10-06T05:29:53.476129+0000","last_became_active":"2023-10-06T05:29:53.470235+0000","last_became_peered":"2023-10-06T05:29:53.470235+0000","last_unstale":"2023-10-06T05:29:53.476129+0000","last_undegraded":"2023-10-06T05:29:53.476129+0000","last_fullsized":"2023-10-06T05:29:53.476129+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:15.299903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.464574+0000","last_change":"2023-10-06T05:29:52.464820+0000","last_active":"2023-10-06T05:29:53.464574+0000","last_peered":"2023-10-06T05:29:53.464574+0000","last_clean":"2023-10-06T05:29:53.464574+0000","last_became_active":"2023-10-06T05:29:52.464555+0000","last_became_peered":"2023-10-06T05:29:52.464555+0000","last_unstale":"2023-10-06T05:29:53.464574+0000","last_undegraded":"2023-10-06T05:29:53.464574+0000","last_fullsized":"2023-10-06T05:29:53.464574+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:10.873171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.483693+0000","last_change":"2023-10-06T05:29:53.476780+0000","last_active":"2023-10-06T05:29:53.483693+0000","last_peered":"2023-10-06T05:29:53.483693+0000","last_clean":"2023-10-06T05:29:53.483693+0000","last_became_active":"2023-10-06T05:29:53.475925+0000","last_became_peered":"2023-10-06T05:29:53.475925+0000","last_unstale":"2023-10-06T05:29:53.483693+0000","last_undegraded":"2023-10-06T05:29:53.483693+0000","last_fullsized":"2023-10-06T05:29:53.483693+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:26.355553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.466299+0000","last_change":"2023-10-06T05:29:52.468132+0000","last_active":"2023-10-06T05:29:53.466299+0000","last_peered":"2023-10-06T05:29:53.466299+0000","last_clean":"2023-10-06T05:29:53.466299+0000","last_became_active":"2023-10-06T05:29:52.467815+0000","last_became_peered":"2023-10-06T05:29:52.467815+0000","last_unstale":"2023-10-06T05:29:53.466299+0000","last_undegraded":"2023-10-06T05:29:53.466299+0000","last_fullsized":"2023-10-06T05:29:53.466299+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:51.431387+0000","last_clean_scrub_stamp":"2023-10-06T05:29:51.431387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:28.097483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.493924+0000","last_change":"2023-10-06T05:29:53.493019+0000","last_active":"2023-10-06T05:29:53.493924+0000","last_peered":"2023-10-06T05:29:53.493924+0000","last_clean":"2023-10-06T05:29:53.493924+0000","last_became_active":"2023-10-06T05:29:53.492663+0000","last_became_peered":"2023-10-06T05:29:53.492663+0000","last_unstale":"2023-10-06T05:29:53.493924+0000","last_undegraded":"2023-10-06T05:29:53.493924+0000","last_fullsized":"2023-10-06T05:29:53.493924+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:23.451445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.468644+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:29:53.468644+0000","last_peered":"2023-10-06T05:29:53.468644+0000","last_clean":"2023-10-06T05:29:53.468644+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:29:53.468644+0000","last_undegraded":"2023-10-06T05:29:53.468644+0000","last_fullsized":"2023-10-06T05:29:53.468644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.479119+0000","last_change":"2023-10-06T05:29:53.476434+0000","last_active":"2023-10-06T05:29:53.479119+0000","last_peered":"2023-10-06T05:29:53.479119+0000","last_clean":"2023-10-06T05:29:53.479119+0000","last_became_active":"2023-10-06T05:29:53.476160+0000","last_became_peered":"2023-10-06T05:29:53.476160+0000","last_unstale":"2023-10-06T05:29:53.479119+0000","last_undegraded":"2023-10-06T05:29:53.479119+0000","last_fullsized":"2023-10-06T05:29:53.479119+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:54.080077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":180,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.464792+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:29:53.464792+0000","last_peered":"2023-10-06T05:29:53.464792+0000","last_clean":"2023-10-06T05:29:53.464792+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:29:53.464792+0000","last_undegraded":"2023-10-06T05:29:53.464792+0000","last_fullsized":"2023-10-06T05:29:53.464792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490587+0000","last_change":"2023-10-06T05:29:53.483710+0000","last_active":"2023-10-06T05:29:53.490587+0000","last_peered":"2023-10-06T05:29:53.490587+0000","last_clean":"2023-10-06T05:29:53.490587+0000","last_became_active":"2023-10-06T05:29:53.479606+0000","last_became_peered":"2023-10-06T05:29:53.479606+0000","last_unstale":"2023-10-06T05:29:53.490587+0000","last_undegraded":"2023-10-06T05:29:53.490587+0000","last_fullsized":"2023-10-06T05:29:53.490587+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:15.982506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":183,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.465016+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:29:53.465016+0000","last_peered":"2023-10-06T05:29:53.465016+0000","last_clean":"2023-10-06T05:29:53.465016+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:29:53.465016+0000","last_undegraded":"2023-10-06T05:29:53.465016+0000","last_fullsized":"2023-10-06T05:29:53.465016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.477721+0000","last_change":"2023-10-06T05:29:53.475601+0000","last_active":"2023-10-06T05:29:53.477721+0000","last_peered":"2023-10-06T05:29:53.477721+0000","last_clean":"2023-10-06T05:29:53.477721+0000","last_became_active":"2023-10-06T05:29:53.475293+0000","last_became_peered":"2023-10-06T05:29:53.475293+0000","last_unstale":"2023-10-06T05:29:53.477721+0000","last_undegraded":"2023-10-06T05:29:53.477721+0000","last_fullsized":"2023-10-06T05:29:53.477721+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:59.725104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":180,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.464653+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:29:53.464653+0000","last_peered":"2023-10-06T05:29:53.464653+0000","last_clean":"2023-10-06T05:29:53.464653+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:29:53.464653+0000","last_undegraded":"2023-10-06T05:29:53.464653+0000","last_fullsized":"2023-10-06T05:29:53.464653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":330,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.465850+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:29:53.465850+0000","last_peered":"2023-10-06T05:29:53.465850+0000","last_clean":"2023-10-06T05:29:53.465850+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:29:53.465850+0000","last_undegraded":"2023-10-06T05:29:53.465850+0000","last_fullsized":"2023-10-06T05:29:53.465850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.490467+0000","last_change":"2023-10-06T05:29:53.482485+0000","last_active":"2023-10-06T05:29:53.490467+0000","last_peered":"2023-10-06T05:29:53.490467+0000","last_clean":"2023-10-06T05:29:53.490467+0000","last_became_active":"2023-10-06T05:29:53.482176+0000","last_became_peered":"2023-10-06T05:29:53.482176+0000","last_unstale":"2023-10-06T05:29:53.490467+0000","last_undegraded":"2023-10-06T05:29:53.490467+0000","last_fullsized":"2023-10-06T05:29:53.490467+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:59.641398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":177,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.465752+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:29:53.465752+0000","last_peered":"2023-10-06T05:29:53.465752+0000","last_clean":"2023-10-06T05:29:53.465752+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:29:53.465752+0000","last_undegraded":"2023-10-06T05:29:53.465752+0000","last_fullsized":"2023-10-06T05:29:53.465752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.486561+0000","last_change":"2023-10-06T05:29:53.478379+0000","last_active":"2023-10-06T05:29:53.486561+0000","last_peered":"2023-10-06T05:29:53.486561+0000","last_clean":"2023-10-06T05:29:53.486561+0000","last_became_active":"2023-10-06T05:29:53.478089+0000","last_became_peered":"2023-10-06T05:29:53.478089+0000","last_unstale":"2023-10-06T05:29:53.486561+0000","last_undegraded":"2023-10-06T05:29:53.486561+0000","last_fullsized":"2023-10-06T05:29:53.486561+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.121364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.463954+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:29:53.463954+0000","last_peered":"2023-10-06T05:29:53.463954+0000","last_clean":"2023-10-06T05:29:53.463954+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:29:53.463954+0000","last_undegraded":"2023-10-06T05:29:53.463954+0000","last_fullsized":"2023-10-06T05:29:53.463954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.484439+0000","last_change":"2023-10-06T05:29:53.477546+0000","last_active":"2023-10-06T05:29:53.484439+0000","last_peered":"2023-10-06T05:29:53.484439+0000","last_clean":"2023-10-06T05:29:53.484439+0000","last_became_active":"2023-10-06T05:29:53.477255+0000","last_became_peered":"2023-10-06T05:29:53.477255+0000","last_unstale":"2023-10-06T05:29:53.484439+0000","last_undegraded":"2023-10-06T05:29:53.484439+0000","last_fullsized":"2023-10-06T05:29:53.484439+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:59.725104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":182,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.470009+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:29:53.470009+0000","last_peered":"2023-10-06T05:29:53.470009+0000","last_clean":"2023-10-06T05:29:53.470009+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:29:53.470009+0000","last_undegraded":"2023-10-06T05:29:53.470009+0000","last_fullsized":"2023-10-06T05:29:53.470009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.481739+0000","last_change":"2023-10-06T05:29:53.477593+0000","last_active":"2023-10-06T05:29:53.481739+0000","last_peered":"2023-10-06T05:29:53.481739+0000","last_clean":"2023-10-06T05:29:53.481739+0000","last_became_active":"2023-10-06T05:29:53.477310+0000","last_became_peered":"2023-10-06T05:29:53.477310+0000","last_unstale":"2023-10-06T05:29:53.481739+0000","last_undegraded":"2023-10-06T05:29:53.481739+0000","last_fullsized":"2023-10-06T05:29:53.481739+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:29:52.442232+0000","last_clean_scrub_stamp":"2023-10-06T05:29:52.442232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:56.332194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":225,"state":"active+clean","last_fresh":"2023-10-06T05:29:53.459261+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:29:53.459261+0000","last_peered":"2023-10-06T05:29:53.459261+0000","last_clean":"2023-10-06T05:29:53.459261+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:29:53.459261+0000","last_undegraded":"2023-10-06T05:29:53.459261+0000","last_fullsized":"2023-10-06T05:29:53.459261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304764,"kb_used_data":3380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067076,"statfs":{"total":96636764160,"available":96324685824,"internally_reserved":0,"allocated":3461120,"data_stored":1929992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.97399999999999998,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96399999999999997,"5min":1.03,"15min":1.169},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.38900000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.268},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.499,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.41499999999999998},"max":{"1min":1.0509999999999999,"5min":1.163,"15min":1.163},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.45600000000000002,"15min":0.45300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.22},"last":0.502}]},{"osd":3,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.006,"5min":1.0529999999999999,"15min":1.202},"last":0.622}]},{"osd":4,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.91500000000000004,"5min":1.109,"15min":1.258},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.39400000000000002},"max":{"1min":1.0940000000000001,"5min":1.234,"15min":1.2849999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.70699999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90200000000000002,"5min":0.98099999999999998,"15min":1.109},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75,"15min":0.747},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":1.2290000000000001,"15min":1.407},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.96299999999999997,"5min":1.169,"15min":1.169},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.3300000000000001},"last":0.82299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575040,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304180,"kb_used_data":2796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067660,"statfs":{"total":96636764160,"available":96325283840,"internally_reserved":0,"allocated":2863104,"data_stored":1339082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70499999999999996,"15min":0.68799999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34200000000000003},"max":{"1min":0.93799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":1.0369999999999999,"15min":1.2869999999999999},"last":0.752}]},{"osd":1,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.47399999999999998,"15min":0.44600000000000001},"max":{"1min":0.999,"5min":1.137,"15min":1.137},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72899999999999998,"15min":0.71099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91000000000000003,"5min":1.135,"15min":1.135},"last":0.88500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91700000000000004,"5min":1.0980000000000001,"15min":1.1599999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53000000000000003,"5min":0.496,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.115,"15min":1.115},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72399999999999998,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.748,"15min":0.749},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.1020000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.97799999999999998,"15min":0.998},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0760000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.73299999999999998,"15min":0.71599999999999997},"min":{"1min":0.56399999999999995,"5min":0.47999999999999998,"15min":0.40000000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.4610000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.432},"max":{"1min":0.96399999999999997,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.436},"max":{"1min":1.032,"5min":1.0860000000000001,"15min":1.1359999999999999},"last":0.67900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304184,"kb_used_data":2800,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067656,"statfs":{"total":96636764160,"available":96325279744,"internally_reserved":0,"allocated":2867200,"data_stored":1339172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":2.7349999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.27800000000000002},"max":{"1min":0.86099999999999999,"5min":0.88100000000000001,"15min":2.8050000000000002},"last":0.47499999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47699999999999998,"15min":0.47399999999999998},"min":{"1min":0.32700000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":1.165,"15min":2.6520000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.68999999999999995,"5min":1.077,"15min":2.7010000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.42699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":1.2230000000000001,"15min":2.7639999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35199999999999998},"max":{"1min":0.75800000000000001,"5min":1.2430000000000001,"15min":2.8370000000000002},"last":0.435}]},{"osd":4,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.50700000000000001},"max":{"1min":1.012,"5min":1.111,"15min":1.5389999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.45200000000000001},"max":{"1min":0.91700000000000004,"5min":1.119,"15min":1.4690000000000001},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.47899999999999998},"max":{"1min":0.98399999999999999,"5min":1.248,"15min":1.349},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.79100000000000004,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.50600000000000001},"max":{"1min":0.97199999999999998,"5min":1.26,"15min":1.669},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.68400000000000005,"5min":0.50700000000000001,"15min":0.48199999999999998},"max":{"1min":0.93300000000000005,"5min":1.2909999999999999,"15min":1.6040000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83699999999999997,"15min":0.82599999999999996},"min":{"1min":0.63300000000000001,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.028,"5min":1.2849999999999999,"15min":1.573},"last":0.94799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.80200000000000005,"15min":0.80700000000000005},"min":{"1min":0.61699999999999999,"5min":0.52200000000000002,"15min":0.46999999999999997},"max":{"1min":1.0329999999999999,"5min":1.1990000000000001,"15min":1.528},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80300000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":0.97699999999999998,"5min":1.1879999999999999,"15min":2.6240000000000001},"last":0.76500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304760,"kb_used_data":3376,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067080,"statfs":{"total":96636764160,"available":96324689920,"internally_reserved":0,"allocated":3457024,"data_stored":1929486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45700000000000002,"15min":0.45400000000000001},"min":{"1min":0.29999999999999999,"5min":0.28799999999999998,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.26900000000000002},"max":{"1min":0.72799999999999998,"5min":0.73699999999999999,"15min":0.82099999999999995},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.66600000000000004,"5min":0.83199999999999996,"15min":0.84999999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.66700000000000004,"5min":0.77000000000000002,"15min":0.93600000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.39100000000000001},"max":{"1min":0.73999999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.627}]},{"osd":4,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.47299999999999998},"max":{"1min":0.90200000000000002,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.0600000000000001,"5min":3.6629999999999998,"15min":3.6629999999999998},"last":0.748}]},{"osd":5,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.54500000000000004},"max":{"1min":0.996,"5min":1.27,"15min":1.27},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":0.90900000000000003,"5min":1.1339999999999999,"15min":1.1970000000000001},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97599999999999998,"5min":1.1830000000000001,"15min":1.1839999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80700000000000005,"15min":0.80600000000000005},"min":{"1min":0.57599999999999996,"5min":0.53400000000000003,"15min":0.45500000000000002},"max":{"1min":0.93500000000000005,"5min":1.153,"15min":1.2629999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:29:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.81000000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.48599999999999999},"max":{"1min":1.018,"5min":1.2010000000000001,"15min":1.25},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.50600000000000001},"max":{"1min":0.95899999999999996,"5min":1.274,"15min":1.2789999999999999},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575041,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304196,"kb_used_data":2812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067644,"statfs":{"total":96636764160,"available":96325267456,"internally_reserved":0,"allocated":2879488,"data_stored":1341160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.435,"15min":0.44800000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27200000000000002},"max":{"1min":0.48099999999999998,"5min":0.86899999999999999,"15min":1.056},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45600000000000002,"15min":0.47699999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51100000000000001,"15min":0.53100000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":1.0649999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68500000000000005,"5min":0.80600000000000005,"15min":1.073},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.56999999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.436},{"interface":"front","average":{"1min":0.505,"5min":0.53000000000000003,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.96499999999999997},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77200000000000002,"15min":0.77800000000000002},"min":{"1min":0.56299999999999994,"5min":0.54100000000000004,"15min":0.53700000000000003},"max":{"1min":0.89800000000000002,"5min":1.486,"15min":1.486},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89000000000000001,"5min":1.5,"15min":1.5},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0069999999999999,"15min":1.1639999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.53000000000000003},"max":{"1min":0.86899999999999999,"5min":1.0640000000000001,"15min":1.319},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.77000000000000002,"15min":0.78900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86599999999999999,"5min":1.0309999999999999,"15min":1.5289999999999999},"last":0.87},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.78500000000000003,"15min":0.78800000000000003},"min":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":0.89900000000000002,"5min":1.0629999999999999,"15min":1.141},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.52100000000000002},"max":{"1min":0.87,"5min":1.538,"15min":1.538},"last":0.755},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78200000000000003,"15min":0.80600000000000005},"min":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97699999999999998,"5min":1.0980000000000001,"15min":1.3520000000000001},"last":0.84299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304180,"kb_used_data":2796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067660,"statfs":{"total":96636764160,"available":96325283840,"internally_reserved":0,"allocated":2863104,"data_stored":1339138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45200000000000001,"15min":0.45900000000000002},"min":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.254},"max":{"1min":0.57999999999999996,"5min":0.76700000000000002,"15min":2.0950000000000002},"last":0.316},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":0.77200000000000002,"15min":1.772},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":1.369,"15min":1.913},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.97899999999999998,"5min":1.5369999999999999,"15min":2.0089999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53100000000000003,"15min":0.54400000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.72999999999999998,"5min":0.90200000000000002,"15min":1.954},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56799999999999995,"15min":0.57399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.754,"5min":0.82599999999999996,"15min":2.0539999999999998},"last":0.55700000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.57199999999999995,"5min":0.51400000000000001,"15min":0.46000000000000002},"max":{"1min":0.86899999999999999,"5min":1.1100000000000001,"15min":1.282},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75700000000000001,"15min":0.77200000000000002},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":0.93799999999999994,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.51100000000000001},"max":{"1min":0.97799999999999998,"5min":1.111,"15min":1.1930000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77400000000000002,"15min":0.79500000000000004},"min":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":0.92000000000000004,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":0.92300000000000004,"5min":1.3360000000000001,"15min":1.343},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":0.93700000000000006,"5min":1.1339999999999999,"15min":1.3380000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56799999999999995},"max":{"1min":0.91700000000000004,"5min":1.1419999999999999,"15min":1.3640000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53000000000000003},"max":{"1min":0.90900000000000003,"5min":1.0980000000000001,"15min":1.214},"last":0.754}]}]},{"osd":4,"up_from":13,"seq":55834575040,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304188,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067652,"statfs":{"total":96636764160,"available":96325275648,"internally_reserved":0,"allocated":2871296,"data_stored":1340584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.46500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.92900000000000005,"15min":1.073},"last":0.88},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.74399999999999999},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86199999999999999,"5min":1.05,"15min":1.169},"last":0.93700000000000006}]},{"osd":1,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.0569999999999999},"last":0.753},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.432},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.42699999999999999},"max":{"1min":0.79200000000000004,"5min":0.96199999999999997,"15min":1.117},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73799999999999999,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":1.196},"last":0.89000000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.8740000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.375},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":1.0189999999999999,"5min":1.081,"15min":1.0920000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.45300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0609999999999999,"15min":1.9390000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.282,"5min":1.282,"15min":1.466},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.78800000000000003},"min":{"1min":0.53700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304180,"kb_used_data":2796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067660,"statfs":{"total":96636764160,"available":96325283840,"internally_reserved":0,"allocated":2863104,"data_stored":1339118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67900000000000005,"15min":0.69299999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.03,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68799999999999994,"15min":0.69099999999999995},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":1.0289999999999999,"15min":1.204},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72199999999999998,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":1.091,"15min":1.1000000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.59199999999999997,"5min":0.49099999999999999,"15min":0.46899999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.087},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.53600000000000003,"5min":0.498,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.145,"15min":1.278},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.95899999999999996,"15min":1.1379999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.46300000000000002,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":1.004,"15min":1.294},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":0.79800000000000004,"5min":0.85899999999999999,"15min":1.141},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0109999999999999,"15min":1.254},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.42399999999999999},"max":{"1min":0.86699999999999999,"5min":0.93899999999999995,"15min":1.337},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72799999999999998,"15min":0.73899999999999999},"min":{"1min":0.51200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.872,"5min":1.0600000000000001,"15min":1.248},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:29:58.521 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:29:58.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:29:58.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.794+0000 7fd7009fe700 1 Processor -- start 2023-10-06T05:29:58.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.794+0000 7fd7009fe700 1 -- start start 2023-10-06T05:29:58.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.795+0000 7fd7009fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6fc156370 0x7fd6fc0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.795+0000 7fd7009fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.795+0000 7fd7009fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc0c8900 con 0x7fd6fc156740 2023-10-06T05:29:58.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.795+0000 7fd7009fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc06beb0 con 0x7fd6fc156370 2023-10-06T05:29:58.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.795+0000 7fd7009fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc06b330 con 0x7fd6fc0c7f60 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.795+0000 7fd6fad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6fc156370 0x7fd6fc0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6fc156370 0x7fd6fc0aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50062/0 (socket says 172.21.15.33:50062) 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 -- 172.21.15.33:0/23921041 learned_addr learned my addr 172.21.15.33:0/23921041 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fa59c700 1 --1- 172.21.15.33:0/23921041 >> v1:172.21.15.33:6789/0 conn(0x7fd6fc156740 0x7fd6fc0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56208/0 (socket says 172.21.15.33:56208) 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fb59e700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 -- 172.21.15.33:0/23921041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 msgr2=0x7fd6fc0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 -- 172.21.15.33:0/23921041 >> v1:172.21.15.33:6789/0 conn(0x7fd6fc156740 legacy=0x7fd6fc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 -- 172.21.15.33:0/23921041 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc06b610 con 0x7fd6fc156370 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fb59e700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6f9d9b700 1 -- 172.21.15.33:0/23921041 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451080399 0 0) 0x7fd6f0001110 con 0x7fd6fc156740 2023-10-06T05:29:58.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.796+0000 7fd6fad9d700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc0aef80 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099b0 tx=0x7fd6ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fc97891c8f40ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:58.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd6f9d9b700 1 -- 172.21.15.33:0/23921041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00b860 con 0x7fd6fc156370 2023-10-06T05:29:58.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd6f9d9b700 1 -- 172.21.15.33:0/23921041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec00b9c0 con 0x7fd6fc156370 2023-10-06T05:29:58.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd6f9d9b700 1 -- 172.21.15.33:0/23921041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00bd60 con 0x7fd6fc156370 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 -- 172.21.15.33:0/23921041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 msgr2=0x7fd6fc0aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc0aef80 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099b0 tx=0x7fd6ec009250 comp rx=0 tx=0).stop 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 -- 172.21.15.33:0/23921041 shutdown_connections 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc0aef80 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 --2- 172.21.15.33:0/23921041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 -- 172.21.15.33:0/23921041 >> 172.21.15.33:0/23921041 conn(0x7fd6fc06c560 msgr2=0x7fd6fc06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 -- 172.21.15.33:0/23921041 shutdown_connections 2023-10-06T05:29:58.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.797+0000 7fd7009fe700 1 -- 172.21.15.33:0/23921041 wait complete. 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.799+0000 7fd7009fe700 1 Processor -- start 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.799+0000 7fd7009fe700 1 -- start start 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd7009fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc16dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd7009fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd7009fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc156740 0x7fd6fc152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd7009fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc15ab10 con 0x7fd6fc156370 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd7009fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0ad020 con 0x7fd6fc156740 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd7009fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0acd40 con 0x7fd6fc0c7f60 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50072/0 (socket says 172.21.15.33:50072) 2023-10-06T05:29:58.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 -- 172.21.15.33:0/2820872435 learned_addr learned my addr 172.21.15.33:0/2820872435 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:58.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 -- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 msgr2=0x7fd6fc16dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc16dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 -- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc156740 msgr2=0x7fd6fc152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:58.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc156740 0x7fd6fc152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:58.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.800+0000 7fd6fa59c700 1 -- 172.21.15.33:0/2820872435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0094f0 con 0x7fd6fc156370 2023-10-06T05:29:58.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.801+0000 7fd6fa59c700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc147c80 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c810 tx=0x7fd6f0007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:58.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.801+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0006300 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.801+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f0006460 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.801+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0004810 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.801+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc149200 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.801+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc1497c0 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.802+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc1499e0 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.803+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6f0004be0 con 0x7fd6fc156370 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.804+0000 7fd6e2ffd700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6e8066f30 0x7fd6e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:58.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.804+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd6f0084070 con 0x7fd6fc156370 2023-10-06T05:29:58.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.805+0000 7fd6fad9d700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6e8066f30 0x7fd6e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:58.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.805+0000 7fd6fad9d700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6e8066f30 0x7fd6e80693e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099b0 tx=0x7fd6ec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:58.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:58.808+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6f0051160 con 0x7fd6fc156370 2023-10-06T05:29:59.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.092+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6fc0c8900 con 0x7fd6fc156370 2023-10-06T05:29:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.097+0000 7fd6e2ffd700 1 -- 172.21.15.33:0/2820872435 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd6fc0c8900 con 0x7fd6fc156370 2023-10-06T05:29:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.099+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6e8066f30 msgr2=0x7fd6e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.099+0000 7fd7009fe700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6e8066f30 0x7fd6e80693e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099b0 tx=0x7fd6ec017000 comp rx=0 tx=0).stop 2023-10-06T05:29:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.099+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 msgr2=0x7fd6fc147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.099+0000 7fd7009fe700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc147c80 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c810 tx=0x7fd6f0007c30 comp rx=0 tx=0).stop 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 shutdown_connections 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd6e8066f30 0x7fd6e80693e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc156740 0x7fd6fc152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6fc0c7f60 0x7fd6fc16dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 --2- 172.21.15.33:0/2820872435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6fc156370 0x7fd6fc147c80 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 >> 172.21.15.33:0/2820872435 conn(0x7fd6fc06c560 msgr2=0x7fd6fc0cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 shutdown_connections 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.100+0000 7fd7009fe700 1 -- 172.21.15.33:0/2820872435 wait complete. 2023-10-06T05:29:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:29:59.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:29:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.305+0000 7f682eb9d700 1 Processor -- start 2023-10-06T05:29:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.305+0000 7f682eb9d700 1 -- start start 2023-10-06T05:29:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.306+0000 7f682eb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68280cf3b0 0x7f68281594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.306+0000 7f682eb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68280af0a0 0x7f68281599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.306+0000 7f682eb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68280cbd80 con 0x7f68280ccfc0 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.306+0000 7f682eb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68280af5f0 con 0x7f68280cf3b0 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.306+0000 7f682eb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68280af8d0 con 0x7f68280af0a0 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68280cf3b0 0x7f68281594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f68280cf3b0 0x7f68281594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50084/0 (socket says 172.21.15.33:50084) 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 -- 172.21.15.33:0/2130897388 learned_addr learned my addr 172.21.15.33:0/2130897388 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682e39c700 1 --2- 172.21.15.33:0/2130897388 >> v2:172.21.15.167:3300/0 conn(0x7f68280af0a0 0x7f68281599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 -- 172.21.15.33:0/2130897388 >> v2:172.21.15.167:3300/0 conn(0x7f68280af0a0 msgr2=0x7f68281599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 --2- 172.21.15.33:0/2130897388 >> v2:172.21.15.167:3300/0 conn(0x7f68280af0a0 0x7f68281599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 -- 172.21.15.33:0/2130897388 >> v1:172.21.15.167:6789/0 conn(0x7f68280ccfc0 legacy=0x7f68280cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 -- 172.21.15.33:0/2130897388 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280afbb0 con 0x7f68280cf3b0 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.307+0000 7f682d39a700 1 --2- 172.21.15.33:0/2130897388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68280cf3b0 0x7f68281594a0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f68180099b0 tx=0x7f6818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c5b84606767cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682cb99700 1 -- 172.21.15.33:0/2130897388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800b820 con 0x7f68280cf3b0 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682cb99700 1 -- 172.21.15.33:0/2130897388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681800b980 con 0x7f68280cf3b0 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682cb99700 1 -- 172.21.15.33:0/2130897388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800bd20 con 0x7f68280cf3b0 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 -- 172.21.15.33:0/2130897388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68280cf3b0 msgr2=0x7f68281594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 --2- 172.21.15.33:0/2130897388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68280cf3b0 0x7f68281594a0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f68180099b0 tx=0x7f6818009250 comp rx=0 tx=0).stop 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 -- 172.21.15.33:0/2130897388 shutdown_connections 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 --2- 172.21.15.33:0/2130897388 >> v2:172.21.15.167:3300/0 conn(0x7f68280af0a0 0x7f68281599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 --2- 172.21.15.33:0/2130897388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68280cf3b0 0x7f68281594a0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 -- 172.21.15.33:0/2130897388 >> 172.21.15.33:0/2130897388 conn(0x7f682806cff0 msgr2=0x7f682806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 -- 172.21.15.33:0/2130897388 shutdown_connections 2023-10-06T05:29:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.308+0000 7f682eb9d700 1 -- 172.21.15.33:0/2130897388 wait complete. 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.310+0000 7f682eb9d700 1 Processor -- start 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.310+0000 7f682eb9d700 1 -- start start 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68280af0a0 0x7f68280c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68280ccfc0 0x7f6828154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 0x7f68280c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68280c8540 con 0x7f682815ee40 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682815a1d0 con 0x7f68280ccfc0 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68280c9090 con 0x7f68280af0a0 2023-10-06T05:29:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682e39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 0x7f68280c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682e39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 0x7f68280c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50092/0 (socket says 172.21.15.33:50092) 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682e39c700 1 -- 172.21.15.33:0/1912683794 learned_addr learned my addr 172.21.15.33:0/1912683794 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682d39a700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68280ccfc0 0x7f6828154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.311+0000 7f682db9b700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68280af0a0 0x7f68280c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682e39c700 1 -- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68280af0a0 msgr2=0x7f68280c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682e39c700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68280af0a0 0x7f68280c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682e39c700 1 -- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68280ccfc0 msgr2=0x7f6828154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682e39c700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68280ccfc0 0x7f6828154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682e39c700 1 -- 172.21.15.33:0/1912683794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180094f0 con 0x7f682815ee40 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682db9b700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68280af0a0 0x7f68280c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682e39c700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 0x7f68280c6480 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f6820007f20 tx=0x7f6820009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682000ec20 con 0x7f682815ee40 2023-10-06T05:29:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682000ed80 con 0x7f682815ee40 2023-10-06T05:29:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820014670 con 0x7f682815ee40 2023-10-06T05:29:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68280c75b0 con 0x7f682815ee40 2023-10-06T05:29:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.312+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68280c7b70 con 0x7f682815ee40 2023-10-06T05:29:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.313+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68280c7d00 con 0x7f682815ee40 2023-10-06T05:29:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.314+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6820014a20 con 0x7f682815ee40 2023-10-06T05:29:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.315+0000 7f681dffb700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f680c066f30 0x7f680c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:29:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.315+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6820084e20 con 0x7f682815ee40 2023-10-06T05:29:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.315+0000 7f682db9b700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f680c066f30 0x7f680c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:29:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.316+0000 7f682db9b700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f680c066f30 0x7f680c0693e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6824007fd0 tx=0x7f6824009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:29:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.319+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6820051e90 con 0x7f682815ee40 2023-10-06T05:29:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.563+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f682804c380 con 0x7f682815ee40 2023-10-06T05:29:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.564+0000 7f681dffb700 1 -- 172.21.15.33:0/1912683794 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f6820009980 con 0x7f682815ee40 2023-10-06T05:29:59.835 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:29:59.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f680c066f30 msgr2=0x7f680c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f680c066f30 0x7f680c0693e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6824007fd0 tx=0x7f6824009040 comp rx=0 tx=0).stop 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 msgr2=0x7f68280c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 0x7f68280c6480 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f6820007f20 tx=0x7f6820009840 comp rx=0 tx=0).stop 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 shutdown_connections 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f680c066f30 0x7f680c0693e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682815ee40 0x7f68280c6480 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68280af0a0 0x7f68280c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 --2- 172.21.15.33:0/1912683794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68280ccfc0 0x7f6828154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:29:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 >> 172.21.15.33:0/1912683794 conn(0x7f682806cff0 msgr2=0x7f6828155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:29:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 shutdown_connections 2023-10-06T05:29:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:29:59.566+0000 7f682eb9d700 1 -- 172.21.15.33:0/1912683794 wait complete. 2023-10-06T05:29:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:30:00.241 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 176, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:29:52.952828+0000', 'modified': '2023-10-06T05:29:56.481932+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10824}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10824': {'gid': 10824, 'name': 'a', 'rank': 0, 'incarnation': 173, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/3371957868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 3371957868}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 3371957868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:30:00.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:30:00.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:30:00.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.518+0000 7f629f59d700 1 Processor -- start 2023-10-06T05:30:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.519+0000 7f629f59d700 1 -- start start 2023-10-06T05:30:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629f59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6298156880 0x7f62980c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62980c7ec0 0x7f6298157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629806a820 con 0x7f6298159e10 2023-10-06T05:30:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629f59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62980cc290 con 0x7f6298156880 2023-10-06T05:30:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c8360 con 0x7f62980c7ec0 2023-10-06T05:30:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629e59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6298156880 0x7f62980c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629dd9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6298159e10 0x7f6298156c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52300/0 (socket says 172.21.15.33:52300) 2023-10-06T05:30:00.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.520+0000 7f629dd9a700 1 -- 172.21.15.33:0/2042362127 learned_addr learned my addr 172.21.15.33:0/2042362127 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:30:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629ed9c700 1 --2- 172.21.15.33:0/2042362127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62980c7ec0 0x7f6298157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629e59b700 1 -- 172.21.15.33:0/2042362127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62980c7ec0 msgr2=0x7f6298157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629e59b700 1 --2- 172.21.15.33:0/2042362127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62980c7ec0 0x7f6298157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629e59b700 1 -- 172.21.15.33:0/2042362127 >> v1:172.21.15.33:6789/0 conn(0x7f6298159e10 legacy=0x7f6298156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629e59b700 1 -- 172.21.15.33:0/2042362127 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629806bd10 con 0x7f6298156880 2023-10-06T05:30:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629d599700 1 -- 172.21.15.33:0/2042362127 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744773591 0 0) 0x7f629806a820 con 0x7f6298159e10 2023-10-06T05:30:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629e59b700 1 --2- 172.21.15.33:0/2042362127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6298156880 0x7f62980c7980 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f6294009910 tx=0x7f62940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480e49bc23dedb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629d599700 1 -- 172.21.15.33:0/2042362127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400b820 con 0x7f6298156880 2023-10-06T05:30:00.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629d599700 1 -- 172.21.15.33:0/2042362127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629400b980 con 0x7f6298156880 2023-10-06T05:30:00.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.521+0000 7f629d599700 1 -- 172.21.15.33:0/2042362127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400bd20 con 0x7f6298156880 2023-10-06T05:30:00.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 -- 172.21.15.33:0/2042362127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6298156880 msgr2=0x7f62980c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 --2- 172.21.15.33:0/2042362127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6298156880 0x7f62980c7980 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f6294009910 tx=0x7f62940091a0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 -- 172.21.15.33:0/2042362127 shutdown_connections 2023-10-06T05:30:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 --2- 172.21.15.33:0/2042362127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6298156880 0x7f62980c7980 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 --2- 172.21.15.33:0/2042362127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62980c7ec0 0x7f6298157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 -- 172.21.15.33:0/2042362127 >> 172.21.15.33:0/2042362127 conn(0x7f629806c560 msgr2=0x7f629806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 -- 172.21.15.33:0/2042362127 shutdown_connections 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.522+0000 7f629f59d700 1 -- 172.21.15.33:0/2042362127 wait complete. 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.523+0000 7f629f59d700 1 Processor -- start 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.523+0000 7f629f59d700 1 -- start start 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.524+0000 7f629f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 0x7f629814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.524+0000 7f629f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6298156880 0x7f62980c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.524+0000 7f629f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6298159e10 0x7f62980c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.524+0000 7f629f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298169890 con 0x7f62980c7ec0 2023-10-06T05:30:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.524+0000 7f629f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629814fdf0 con 0x7f6298156880 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.524+0000 7f629f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c5fb0 con 0x7f6298159e10 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 0x7f629814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 0x7f629814f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55260/0 (socket says 172.21.15.33:55260) 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 -- 172.21.15.33:0/2148403817 learned_addr learned my addr 172.21.15.33:0/2148403817 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629ed9c700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6298159e10 0x7f62980c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629dd9a700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6298156880 0x7f62980c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 -- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6298159e10 msgr2=0x7f62980c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6298159e10 0x7f62980c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 -- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6298156880 msgr2=0x7f62980c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6298156880 0x7f62980c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 -- 172.21.15.33:0/2148403817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6294009420 con 0x7f62980c7ec0 2023-10-06T05:30:00.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f629e59b700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 0x7f629814f8b0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f629400b3c0 tx=0x7f6294017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.525+0000 7f628e7fc700 1 -- 172.21.15.33:0/2148403817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294024440 con 0x7f62980c7ec0 2023-10-06T05:30:00.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.526+0000 7f628e7fc700 1 -- 172.21.15.33:0/2148403817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62940245e0 con 0x7f62980c7ec0 2023-10-06T05:30:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.526+0000 7f628e7fc700 1 -- 172.21.15.33:0/2148403817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940249a0 con 0x7f62980c7ec0 2023-10-06T05:30:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.526+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62980afdd0 con 0x7f62980c7ec0 2023-10-06T05:30:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.526+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62980c57c0 con 0x7f62980c7ec0 2023-10-06T05:30:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.528+0000 7f628e7fc700 1 -- 172.21.15.33:0/2148403817 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f629401a420 con 0x7f62980c7ec0 2023-10-06T05:30:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.529+0000 7f628e7fc700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f627c066f30 0x7f627c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.529+0000 7f628e7fc700 1 -- 172.21.15.33:0/2148403817 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f62940865b0 con 0x7f62980c7ec0 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.530+0000 7f629dd9a700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f627c066f30 0x7f627c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.531+0000 7f629dd9a700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f627c066f30 0x7f627c0693e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f629806f040 tx=0x7f6288009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.531+0000 7f62aeea0b80 1 Processor -- start 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.531+0000 7f62aeea0b80 1 -- start start 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87832d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8787cbc0 0x558a8785b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a8782e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a877b2c80 con 0x558a8785be60 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a87789a70 con 0x558a8787cbc0 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a87787270 con 0x558a87832950 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f629ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a8782e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f629ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a8782e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55272/0 (socket says 172.21.15.33:55272) 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f629ed9c700 1 -- 172.21.15.33:0/1097111175 learned_addr learned my addr 172.21.15.33:0/1097111175 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f629dd9a700 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8787cbc0 0x558a8785b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.532+0000 7f629e59b700 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87832d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f629ed9c700 1 -- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 msgr2=0x558a87832d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f629ed9c700 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87832d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f629ed9c700 1 -- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8787cbc0 msgr2=0x558a8785b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f629ed9c700 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8787cbc0 0x558a8785b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f629ed9c700 1 -- 172.21.15.33:0/1097111175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a8761d270 con 0x558a8785be60 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f629ed9c700 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a8782e990 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f6290007370 tx=0x7f629000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2228aa6a2ef626fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f627b7fe700 1 -- 172.21.15.33:0/1097111175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290015470 con 0x558a8785be60 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f627b7fe700 1 -- 172.21.15.33:0/1097111175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000f050 con 0x558a8785be60 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.533+0000 7f627b7fe700 1 -- 172.21.15.33:0/1097111175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62900157e0 con 0x558a8785be60 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 -- 172.21.15.33:0/1097111175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 msgr2=0x558a8782e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a8782e990 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f6290007370 tx=0x7f629000b040 comp rx=0 tx=0).stop 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 -- 172.21.15.33:0/1097111175 shutdown_connections 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a8782e990 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87832d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/1097111175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8787cbc0 0x558a8785b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 -- 172.21.15.33:0/1097111175 >> 172.21.15.33:0/1097111175 conn(0x558a87834270 msgr2=0x558a87834640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 -- 172.21.15.33:0/1097111175 shutdown_connections 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.534+0000 7f62aeea0b80 1 -- 172.21.15.33:0/1097111175 wait complete. 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.535+0000 7f62aeea0b80 1 -- start start 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.535+0000 7f62aeea0b80 1 Processor -- start 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87874a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8785be60 0x558a878737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a87857ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a87783060 con 0x558a8787cbc0 2023-10-06T05:30:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8772e890 con 0x558a8785be60 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a87749ea0 con 0x558a87832950 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f62aeea0b80 10 client.0 _ll_register_callbacks cb 0x558a8761ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a87857ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a87857ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55286/0 (socket says 172.21.15.33:55286) 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 -- 172.21.15.33:0/196752192 learned_addr learned my addr 172.21.15.33:0/196752192 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 -- 172.21.15.33:0/196752192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 msgr2=0x558a87874a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87874a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 -- 172.21.15.33:0/196752192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8785be60 msgr2=0x558a878737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8785be60 0x558a878737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.536+0000 7f629ed9c700 1 -- 172.21.15.33:0/196752192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6290009cc0 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f629ed9c700 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a87857ae0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f6290007eb0 tx=0x7f629001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0fb29c58f234e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f6278ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f6278ff9700 1 -- 172.21.15.33:0/196752192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290015780 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f6278ff9700 1 -- 172.21.15.33:0/196752192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f629001f400 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a87827690 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f6278ff9700 1 -- 172.21.15.33:0/196752192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629001f7d0 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f62aeea0b80 10 client.10968 fetch_fsmap learned FSMap version 176 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=176,monmap=2+}) v3 -- 0x558a875bffe0 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.537+0000 7f6278ff9700 1 -- 172.21.15.33:0/196752192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a87827690 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f6278ff9700 1 -- 172.21.15.33:0/196752192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 176) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f629001f930 con 0x558a8787cbc0 2023-10-06T05:30:00.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f62aeea0b80 10 client.10968 fetch_fsmap finished waiting for FSMap version 176 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f62aeea0b80 10 client.10968 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f62aeea0b80 20 client.10968 populate_metadata read hostname 'smithi033' 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87870ae0 0x558a87870ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f62aeea0b80 4 client.10968 mds_command: new command op to 10824 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 --> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a878045c0 con 0x558a87870ae0 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f629e59b700 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87870ae0 0x558a87870ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f629e59b700 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87870ae0 0x558a87870ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d7a0c2299b166d server_cookie=6913ef6c13492d3f in_seq=0 out_seq=0 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.538+0000 7f6278ff9700 10 client.10968 ms_handle_connect on v2:172.21.15.33:6833/3371957868 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f6278ff9700 1 -- 172.21.15.33:0/196752192 <== mds.0 v2:172.21.15.33:6833/3371957868 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558a878045c0 con 0x558a87870ae0 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f6278ff9700 10 client.10968 handle_command_reply: tid=0 2023-10-06T05:30:00.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f62aeea0b80 1 client.10968 shutdown 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 msgr2=0x558a87857ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a87857ae0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f6290007eb0 tx=0x7f629001c000 comp rx=0 tx=0).stop 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 shutdown_connections 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87870ae0 0x558a87870ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.539+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a87857ae0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a87832950 0x558a87874a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/196752192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a8785be60 0x558a878737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f6278ff9700 0 client.10968 ms_handle_reset on v2:172.21.15.33:6833/3371957868 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 >> 172.21.15.33:0/196752192 conn(0x558a87834270 msgr2=0x558a878291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 shutdown_connections 2023-10-06T05:30:00.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f62aeea0b80 1 -- 172.21.15.33:0/196752192 wait complete. 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.540+0000 7f62aeea0b80 20 client.10968 trim_cache size 0 max 16384 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.557+0000 7f62aeea0b80 1 Processor -- start 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.557+0000 7f62aeea0b80 1 -- start start 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a8788ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8785be60 0x558a878332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a878291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a87605e30 con 0x558a8787cbc0 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8785ac50 con 0x558a87832950 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a87856640 con 0x558a8785be60 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f629ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a878291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f629ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a878291a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55298/0 (socket says 172.21.15.33:55298) 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.558+0000 7f629ed9c700 1 -- 172.21.15.33:0/3854991154 learned_addr learned my addr 172.21.15.33:0/3854991154 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f629ed9c700 1 -- 172.21.15.33:0/3854991154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8785be60 msgr2=0x558a878332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f629ed9c700 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8785be60 0x558a878332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f629ed9c700 1 -- 172.21.15.33:0/3854991154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 msgr2=0x558a8788ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f629ed9c700 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a8788ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f629ed9c700 1 -- 172.21.15.33:0/3854991154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a87751780 con 0x558a8787cbc0 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f629ed9c700 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a878291a0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f6290002230 tx=0x7f629000b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9303c8e06ee554bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f627bfff700 1 -- 172.21.15.33:0/3854991154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290009750 con 0x558a8787cbc0 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f627bfff700 1 -- 172.21.15.33:0/3854991154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629001f400 con 0x558a8787cbc0 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.559+0000 7f627bfff700 1 -- 172.21.15.33:0/3854991154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290015780 con 0x558a8787cbc0 2023-10-06T05:30:00.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 -- 172.21.15.33:0/3854991154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 msgr2=0x558a878291a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a878291a0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f6290002230 tx=0x7f629000b070 comp rx=0 tx=0).stop 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 -- 172.21.15.33:0/3854991154 shutdown_connections 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8787cbc0 0x558a878291a0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a8788ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/3854991154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8785be60 0x558a878332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 -- 172.21.15.33:0/3854991154 >> 172.21.15.33:0/3854991154 conn(0x558a87834270 msgr2=0x558a87872070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 -- 172.21.15.33:0/3854991154 shutdown_connections 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 -- 172.21.15.33:0/3854991154 wait complete. 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.560+0000 7f62aeea0b80 1 -- start start 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.561+0000 7f62aeea0b80 1 Processor -- start 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a878deee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a878729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8787cbc0 0x558a8788c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a878df420 con 0x558a8785be60 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a87873670 con 0x558a87832950 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a8782edb0 con 0x558a8787cbc0 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f62aeea0b80 10 client.0 _ll_register_callbacks cb 0x558a8761ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a878729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a878729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55304/0 (socket says 172.21.15.33:55304) 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 -- 172.21.15.33:0/2002164250 learned_addr learned my addr 172.21.15.33:0/2002164250 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629e59b700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a878deee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629ed9c700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8787cbc0 0x558a8788c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 -- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8787cbc0 msgr2=0x558a8788c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8787cbc0 0x558a8788c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 -- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 msgr2=0x558a878deee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a878deee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 -- 172.21.15.33:0/2002164250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a875bffe0 con 0x558a8785be60 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629e59b700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a878deee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f629dd9a700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a878729f0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f62880011e0 tx=0x7f6288013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e2e50518c911c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.562+0000 7f6278ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f6278ff9700 1 -- 172.21.15.33:0/2002164250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288007b20 con 0x558a8785be60 2023-10-06T05:30:00.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f6278ff9700 1 -- 172.21.15.33:0/2002164250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f628800e9c0 con 0x558a8785be60 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f629000f050 con 0x558a8785be60 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f6278ff9700 1 -- 172.21.15.33:0/2002164250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=176) v2 ==== 24+0+0 (secure 0 0 0) 0x7f628800ed90 con 0x558a8785be60 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 10 client.10974 fetch_fsmap learned FSMap version 176 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=176,monmap=2+}) v3 -- 0x558a87738470 con 0x558a8785be60 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f6278ff9700 1 -- 172.21.15.33:0/2002164250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000f050 con 0x558a8785be60 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f6278ff9700 1 -- 172.21.15.33:0/2002164250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 176) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6288020070 con 0x558a8785be60 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 10 client.10974 fetch_fsmap finished waiting for FSMap version 176 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 10 client.10974 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 20 client.10974 populate_metadata read hostname 'smithi033' 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87862260 0x558a8788c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 4 client.10974 mds_command: new command op to 10824 tid=0[{"prefix": "status"}] 2023-10-06T05:30:00.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.563+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 --> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a87725e00 con 0x558a87862260 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f629e59b700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87862260 0x558a8788c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f629e59b700 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87862260 0x558a8788c6b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5321531818d234 server_cookie=1e276ea26503830e in_seq=0 out_seq=0 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f6278ff9700 10 client.10974 ms_handle_connect on v2:172.21.15.33:6833/3371957868 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f6278ff9700 1 -- 172.21.15.33:0/2002164250 <== mds.0 v2:172.21.15.33:6833/3371957868 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x558a87725e00 con 0x558a87862260 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f6278ff9700 10 client.10974 handle_command_reply: tid=0 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 client.10974 shutdown 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 msgr2=0x558a878729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a878729f0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f62880011e0 tx=0x7f6288013000 comp rx=0 tx=0).stop 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 shutdown_connections 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:6833/3371957868,v1:172.21.15.33:6835/3371957868] conn(0x558a87862260 0x558a8788c6b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x558a8787cbc0 0x558a8788c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a87832950 0x558a878deee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f6278ff9700 0 client.10974 ms_handle_reset on v2:172.21.15.33:6833/3371957868 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 --2- 172.21.15.33:0/2002164250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a8785be60 0x558a878729f0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.564+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 >> 172.21.15.33:0/2002164250 conn(0x558a87834270 msgr2=0x558a87877a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 shutdown_connections 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f62aeea0b80 1 -- 172.21.15.33:0/2002164250 wait complete. 2023-10-06T05:30:00.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f62aeea0b80 20 client.10974 trim_cache size 0 max 16384 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 10824, 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:30:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 7.1105056529999997, 2023-10-06T05:30:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 176, 2023-10-06T05:30:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 226, 2023-10-06T05:30:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 226, 2023-10-06T05:30:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 13.845682909000001 2023-10-06T05:30:00.889 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:30:00.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f627c066f30 msgr2=0x7f627c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f627c066f30 0x7f627c0693e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f629806f040 tx=0x7f6288009040 comp rx=0 tx=0).stop 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 msgr2=0x7f629814f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 0x7f629814f8b0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f629400b3c0 tx=0x7f6294017000 comp rx=0 tx=0).stop 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 shutdown_connections 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f627c066f30 0x7f627c0693e0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6298159e10 0x7f62980c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62980c7ec0 0x7f629814f8b0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 --2- 172.21.15.33:0/2148403817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6298156880 0x7f62980c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.565+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 >> 172.21.15.33:0/2148403817 conn(0x7f629806c560 msgr2=0x7f62981505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:00.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.566+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 shutdown_connections 2023-10-06T05:30:00.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:00.566+0000 7f629f59d700 1 -- 172.21.15.33:0/2148403817 wait complete. 2023-10-06T05:30:00.892 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:30:01.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.080+0000 7f49e579d700 1 Processor -- start 2023-10-06T05:30:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.080+0000 7f49e579d700 1 -- start start 2023-10-06T05:30:01.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.081+0000 7f49e579d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49e00c8050 0x7f49e013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.081+0000 7f49e579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00c9cb0 con 0x7f49e006aad0 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.081+0000 7f49e579d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00ca270 con 0x7f49e013faa0 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.082+0000 7f49e579d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e006b640 con 0x7f49e00c8050 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.082+0000 7f49e4f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f49e006aad0 0x7f49e006af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52310/0 (socket says 172.21.15.33:52310) 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.082+0000 7f49e4f9c700 1 -- 172.21.15.33:0/1743968168 learned_addr learned my addr 172.21.15.33:0/1743968168 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.082+0000 7f49df7fe700 1 --2- 172.21.15.33:0/1743968168 >> v2:172.21.15.167:3300/0 conn(0x7f49e00c8050 0x7f49e013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.082+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605764121 0 0) 0x7f49e00c9cb0 con 0x7f49e006aad0 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.082+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c8003570 con 0x7f49e006aad0 2023-10-06T05:30:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097785564 0 0) 0x7f49e00ca270 con 0x7f49e013faa0 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e00c9cb0 con 0x7f49e013faa0 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461285463 0 0) 0x7f49c8003570 con 0x7f49e006aad0 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49df7fe700 1 -- 172.21.15.33:0/1743968168 >> v1:172.21.15.167:6789/0 conn(0x7f49e013faa0 legacy=0x7f49e00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49df7fe700 1 -- 172.21.15.33:0/1743968168 >> v1:172.21.15.33:6789/0 conn(0x7f49e006aad0 legacy=0x7f49e006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49df7fe700 1 -- 172.21.15.33:0/1743968168 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e006b840 con 0x7f49e00c8050 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49df7fe700 1 --2- 172.21.15.33:0/1743968168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e00c8050 0x7f49e013fe70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098e0 tx=0x7f49d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a759aae7427b763 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8001770 con 0x7f49e00c8050 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4011420 con 0x7f49e00c8050 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.083+0000 7f49deffd700 1 -- 172.21.15.33:0/1743968168 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40117c0 con 0x7f49e00c8050 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 -- 172.21.15.33:0/1743968168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e00c8050 msgr2=0x7f49e013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 --2- 172.21.15.33:0/1743968168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e00c8050 0x7f49e013fe70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098e0 tx=0x7f49d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 -- 172.21.15.33:0/1743968168 shutdown_connections 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 --2- 172.21.15.33:0/1743968168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e00c8050 0x7f49e013fe70 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 -- 172.21.15.33:0/1743968168 >> 172.21.15.33:0/1743968168 conn(0x7f49e006d030 msgr2=0x7f49e006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 -- 172.21.15.33:0/1743968168 shutdown_connections 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.084+0000 7f49e579d700 1 -- 172.21.15.33:0/1743968168 wait complete. 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.085+0000 7f49e579d700 1 Processor -- start 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.085+0000 7f49e579d700 1 -- start start 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.086+0000 7f49e579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e006aad0 0x7f49e0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.086+0000 7f49e579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 0x7f49e016a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.086+0000 7f49e579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49e013faa0 0x7f49e016a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.086+0000 7f49e579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01586c0 con 0x7f49e00c8050 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.086+0000 7f49e579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0157b80 con 0x7f49e006aad0 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.086+0000 7f49e579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0157e20 con 0x7f49e013faa0 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 0x7f49e016a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 0x7f49e016a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55316/0 (socket says 172.21.15.33:55316) 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 -- 172.21.15.33:0/2079432456 learned_addr learned my addr 172.21.15.33:0/2079432456 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49e4f9c700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49e013faa0 0x7f49e016a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49dffff700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e006aad0 0x7f49e0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 -- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49e013faa0 msgr2=0x7f49e016a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49e013faa0 0x7f49e016a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 -- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e006aad0 msgr2=0x7f49e0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e006aad0 0x7f49e0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 -- 172.21.15.33:0/2079432456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009420 con 0x7f49e00c8050 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49dffff700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e006aad0 0x7f49e0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.087+0000 7f49df7fe700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 0x7f49e016a010 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f49e00719c0 tx=0x7f49d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.088+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d401cc90 con 0x7f49e00c8050 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.088+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d401cdf0 con 0x7f49e00c8050 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.088+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d401a6a0 con 0x7f49e00c8050 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.088+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e0158100 con 0x7f49e00c8050 2023-10-06T05:30:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.088+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e00c32c0 con 0x7f49e00c8050 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.089+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0159eb0 con 0x7f49e00c8050 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.090+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49d401aa90 con 0x7f49e00c8050 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.091+0000 7f49c7fff700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49cc066f30 0x7f49cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.091+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f49d4085e30 con 0x7f49e00c8050 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.091+0000 7f49dffff700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49cc066f30 0x7f49cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.092+0000 7f49dffff700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49cc066f30 0x7f49cc0693e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f49e00c6310 tx=0x7f49d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.095+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49d4052ea0 con 0x7f49e00c8050 2023-10-06T05:30:01.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.342+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f49e01569d0 con 0x7f49e00c8050 2023-10-06T05:30:01.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.342+0000 7f49c7fff700 1 -- 172.21.15.33:0/2079432456 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v176) v1 ==== 53+0+73 (secure 0 0 0) 0x7f49d4029060 con 0x7f49e00c8050 2023-10-06T05:30:01.344 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:30:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49cc066f30 msgr2=0x7f49cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49cc066f30 0x7f49cc0693e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f49e00c6310 tx=0x7f49d0009040 comp rx=0 tx=0).stop 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 msgr2=0x7f49e016a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 0x7f49e016a010 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f49e00719c0 tx=0x7f49d4016000 comp rx=0 tx=0).stop 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 shutdown_connections 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49cc066f30 0x7f49cc0693e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49e013faa0 0x7f49e016a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49e006aad0 0x7f49e0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 --2- 172.21.15.33:0/2079432456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e00c8050 0x7f49e016a010 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.344+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 >> 172.21.15.33:0/2079432456 conn(0x7f49e006d030 msgr2=0x7f49e00c1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.345+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 shutdown_connections 2023-10-06T05:30:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:01.345+0000 7f49e579d700 1 -- 172.21.15.33:0/2079432456 wait complete. 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:30:01.364 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:01.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:01.365 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41356sec preferred_lft 41356sec 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.404 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:30:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:30:01.406 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:01.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:01.407 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:30:01.505 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:30:01.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:01.515 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:30:01.554 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:30:01.554 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:30:01.554 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:30:01.554 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:30:01.554 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:30:01.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:01.555 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:30:01.556 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:30:01.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:30:01.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:30:01.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:30:01.556 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:30:01.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:01.755 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:30:01.791 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:30:02.007 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:30:02.008 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:30:02.008 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:02.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:02.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:30:02.072 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:30:02.097 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:02.134 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:02.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:30:02.135 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:02.210 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:30:02.210 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:30:02.210 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:02.343 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:02.343 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:30:02.344 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:30:02.360 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:02.360 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:30:02.360 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:30:02.360 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:30:02.361 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:30:02.383 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:30:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:30:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:30:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:30:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:30:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:30:02.385 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:30:02.389 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:30:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:02.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:02.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:30:02.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.637+0000 7f37ab59e700 1 Processor -- start 2023-10-06T05:30:02.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.637+0000 7f37ab59e700 1 -- start start 2023-10-06T05:30:02.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37ab59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 0x7f37ac06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:02.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37ab59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37ac0c7920 0x7f37ac0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:02.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37ab59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06ab60 con 0x7f37ac06a6a0 2023-10-06T05:30:02.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37ab59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac06b7e0 con 0x7f37ac0c7920 2023-10-06T05:30:02.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37ab59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac06ae40 con 0x7f37ac155af0 2023-10-06T05:30:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37aa59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 0x7f37ac06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37aad9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37ac0c7920 0x7f37ac0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37aa59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 0x7f37ac06a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:42412/0 (socket says 172.21.15.33:42412) 2023-10-06T05:30:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37aa59c700 1 -- 172.21.15.33:0/452609840 learned_addr learned my addr 172.21.15.33:0/452609840 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.639+0000 7f37a9d9b700 1 --1- 172.21.15.33:0/452609840 >> v1:172.21.15.167:6789/0 conn(0x7f37ac06a6a0 0x7f37ac0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:37498/0 (socket says 172.21.15.33:37498) 2023-10-06T05:30:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37aa59c700 1 -- 172.21.15.33:0/452609840 >> v2:172.21.15.167:3300/0 conn(0x7f37ac0c7920 msgr2=0x7f37ac0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37a959a700 1 -- 172.21.15.33:0/452609840 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246222026 0 0) 0x7f37ac06ab60 con 0x7f37ac06a6a0 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37aa59c700 1 --2- 172.21.15.33:0/452609840 >> v2:172.21.15.167:3300/0 conn(0x7f37ac0c7920 0x7f37ac0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37aa59c700 1 -- 172.21.15.33:0/452609840 >> v1:172.21.15.167:6789/0 conn(0x7f37ac06a6a0 legacy=0x7f37ac0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37aa59c700 1 -- 172.21.15.33:0/452609840 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac06b120 con 0x7f37ac155af0 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37aad9d700 1 --2- 172.21.15.33:0/452609840 >> v2:172.21.15.167:3300/0 conn(0x7f37ac0c7920 0x7f37ac0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.640+0000 7f37aa59c700 1 --2- 172.21.15.33:0/452609840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 0x7f37ac06a160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f379c0099b0 tx=0x7f379c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfabdda47a1ca48f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37a959a700 1 -- 172.21.15.33:0/452609840 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c009ea0 con 0x7f37ac155af0 2023-10-06T05:30:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37a959a700 1 -- 172.21.15.33:0/452609840 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00b860 con 0x7f37ac155af0 2023-10-06T05:30:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37a959a700 1 -- 172.21.15.33:0/452609840 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00bc00 con 0x7f37ac155af0 2023-10-06T05:30:02.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 -- 172.21.15.33:0/452609840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 msgr2=0x7f37ac06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:02.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 --2- 172.21.15.33:0/452609840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 0x7f37ac06a160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f379c0099b0 tx=0x7f379c009250 comp rx=0 tx=0).stop 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 -- 172.21.15.33:0/452609840 shutdown_connections 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 --2- 172.21.15.33:0/452609840 >> v2:172.21.15.167:3300/0 conn(0x7f37ac0c7920 0x7f37ac0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 --2- 172.21.15.33:0/452609840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac155af0 0x7f37ac06a160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 -- 172.21.15.33:0/452609840 >> 172.21.15.33:0/452609840 conn(0x7f37ac06d030 msgr2=0x7f37ac06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 -- 172.21.15.33:0/452609840 shutdown_connections 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.641+0000 7f37ab59e700 1 -- 172.21.15.33:0/452609840 wait complete. 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.643+0000 7f37ab59e700 1 Processor -- start 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.643+0000 7f37ab59e700 1 -- start start 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37ab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 0x7f37ac0ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37ab59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac0c7920 0x7f37ac16a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:02.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37ab59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37ac155af0 0x7f37ac16c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37ab59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0adb90 con 0x7f37ac06a6a0 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37ab59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0ae9d0 con 0x7f37ac155af0 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37ab59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac16aba0 con 0x7f37ac0c7920 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 0x7f37ac0ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 0x7f37ac0ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55332/0 (socket says 172.21.15.33:55332) 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37aa59c700 1 -- 172.21.15.33:0/485840515 learned_addr learned my addr 172.21.15.33:0/485840515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37a9d9b700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac0c7920 0x7f37ac16a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.644+0000 7f37aad9d700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37ac155af0 0x7f37ac16c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37aa59c700 1 -- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac0c7920 msgr2=0x7f37ac16a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37aa59c700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac0c7920 0x7f37ac16a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37aa59c700 1 -- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37ac155af0 msgr2=0x7f37ac16c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37aa59c700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37ac155af0 0x7f37ac16c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37aa59c700 1 -- 172.21.15.33:0/485840515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009a20 con 0x7f37ac06a6a0 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37aa59c700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 0x7f37ac0ad650 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b3c0 tx=0x7f379c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c000d10 con 0x7f37ac06a6a0 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00b9d0 con 0x7f37ac06a6a0 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c025420 con 0x7f37ac06a6a0 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c0094f0 con 0x7f37ac06a6a0 2023-10-06T05:30:02.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.645+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac0ae0a0 con 0x7f37ac06a6a0 2023-10-06T05:30:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.646+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac0c8540 con 0x7f37ac06a6a0 2023-10-06T05:30:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.647+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f379c023070 con 0x7f37ac06a6a0 2023-10-06T05:30:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.649+0000 7f379a7fc700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f378c066f30 0x7f378c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:02.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.649+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f379c085ad0 con 0x7f37ac06a6a0 2023-10-06T05:30:02.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.650+0000 7f37a9d9b700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f378c066f30 0x7f378c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:02.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.650+0000 7f37a9d9b700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f378c066f30 0x7f378c0693e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f37ac14ccf0 tx=0x7f37a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:02.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.654+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f379c052c00 con 0x7f37ac06a6a0 2023-10-06T05:30:02.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.903+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37ac06b400 con 0x7f37ac06a6a0 2023-10-06T05:30:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.903+0000 7f379a7fc700 1 -- 172.21.15.33:0/485840515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v176) v1 ==== 53+0+73 (secure 0 0 0) 0x7f379c015070 con 0x7f37ac06a6a0 2023-10-06T05:30:02.905 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:30:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f378c066f30 msgr2=0x7f378c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f378c066f30 0x7f378c0693e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f37ac14ccf0 tx=0x7f37a0009040 comp rx=0 tx=0).stop 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 msgr2=0x7f37ac0ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 0x7f37ac0ad650 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b3c0 tx=0x7f379c017000 comp rx=0 tx=0).stop 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 shutdown_connections 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f378c066f30 0x7f378c0693e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37ac155af0 0x7f37ac16c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37ac06a6a0 0x7f37ac0ad650 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 --2- 172.21.15.33:0/485840515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37ac0c7920 0x7f37ac16a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:02.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 >> 172.21.15.33:0/485840515 conn(0x7f37ac06d030 msgr2=0x7f37ac1530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 shutdown_connections 2023-10-06T05:30:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:02.906+0000 7f37ab59e700 1 -- 172.21.15.33:0/485840515 wait complete. 2023-10-06T05:30:02.925 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:30:02.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:30:02.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:30:02.926 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:30:02.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:30:02.926 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:30:02.926 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:30:02.926 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:30:02.926 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:30:02.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:02.927 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:30:02.970 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:30:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:30:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:30:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:02.971 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41358sec preferred_lft 41358sec 2023-10-06T05:30:02.972 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.973 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:30:02.974 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:02.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:02.975 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:30:03.080 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:30:03.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:03.084 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:30:03.122 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:30:03.123 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:30:03.123 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:30:03.123 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:30:03.123 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:30:03.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:03.124 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:30:03.124 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:30:03.124 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:30:03.124 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:30:03.124 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:30:03.125 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:30:03.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:03.320 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:30:03.356 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:30:03.356 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:30:03.357 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:30:03.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:03.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:30:03.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:30:03.458 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:30:03.494 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:30:03.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:30:03.495 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:30:03.570 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:30:03.570 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:30:03.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:30:03.701 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:30:03.701 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:30:03.701 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:30:03.718 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:30:03.718 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:30:03.718 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:30:03.718 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:30:03.718 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:30:03.738 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:30:03.738 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:30:03.739 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:30:03.739 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:30:03.739 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:30:03.739 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:30:03.739 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:30:03.739 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:30:03.744 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:30:03.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:03.745 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:30:03.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:30:04.024 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:30:04.025 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:30:04.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:30:04.230 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:30:04.231 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:30:04.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:30:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.418+0000 7f6679f9d700 1 Processor -- start 2023-10-06T05:30:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.419+0000 7f6679f9d700 1 -- start start 2023-10-06T05:30:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.420+0000 7f6679f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f667406e140 0x7f667415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.420+0000 7f6679f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f66740b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.420+0000 7f6679f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667406f220 con 0x7f6674159530 2023-10-06T05:30:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.420+0000 7f6679f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66740b1e00 con 0x7f667406e140 2023-10-06T05:30:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.420+0000 7f6679f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667406f6b0 con 0x7f66740afdd0 2023-10-06T05:30:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f6673fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f667406e140 0x7f667415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f6678f9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6674159530 0x7f667406da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:37514/0 (socket says 172.21.15.33:37514) 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f6678f9b700 1 -- 172.21.15.33:0/3097963304 learned_addr learned my addr 172.21.15.33:0/3097963304 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f667979c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f66740b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f66737fe700 1 -- 172.21.15.33:0/3097963304 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363130916 0 0) 0x7f667406f220 con 0x7f6674159530 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f66737fe700 1 -- 172.21.15.33:0/3097963304 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6664003570 con 0x7f6674159530 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f667979c700 1 -- 172.21.15.33:0/3097963304 >> v2:172.21.15.167:3300/0 conn(0x7f667406e140 msgr2=0x7f667415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f667979c700 1 --2- 172.21.15.33:0/3097963304 >> v2:172.21.15.167:3300/0 conn(0x7f667406e140 0x7f667415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f667979c700 1 -- 172.21.15.33:0/3097963304 >> v1:172.21.15.167:6789/0 conn(0x7f6674159530 legacy=0x7f667406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f667979c700 1 -- 172.21.15.33:0/3097963304 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667415ae50 con 0x7f66740afdd0 2023-10-06T05:30:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f6673fff700 1 --2- 172.21.15.33:0/3097963304 >> v2:172.21.15.167:3300/0 conn(0x7f667406e140 0x7f667415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:30:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.421+0000 7f667979c700 1 --2- 172.21.15.33:0/3097963304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f66740b01f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f666c009e60 tx=0x7f666c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f518396e3e7027c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f66737fe700 1 -- 172.21.15.33:0/3097963304 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c00e070 con 0x7f66740afdd0 2023-10-06T05:30:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f66737fe700 1 -- 172.21.15.33:0/3097963304 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c00bab0 con 0x7f66740afdd0 2023-10-06T05:30:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f66737fe700 1 -- 172.21.15.33:0/3097963304 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c00be70 con 0x7f66740afdd0 2023-10-06T05:30:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f6679f9d700 1 -- 172.21.15.33:0/3097963304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 msgr2=0x7f66740b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f6679f9d700 1 --2- 172.21.15.33:0/3097963304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f66740b01f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f666c009e60 tx=0x7f666c009780 comp rx=0 tx=0).stop 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f6679f9d700 1 -- 172.21.15.33:0/3097963304 shutdown_connections 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f6679f9d700 1 --2- 172.21.15.33:0/3097963304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f66740b01f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f6679f9d700 1 --2- 172.21.15.33:0/3097963304 >> v2:172.21.15.167:3300/0 conn(0x7f667406e140 0x7f667415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.422+0000 7f6679f9d700 1 -- 172.21.15.33:0/3097963304 >> 172.21.15.33:0/3097963304 conn(0x7f667406c560 msgr2=0x7f667406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.423+0000 7f6679f9d700 1 -- 172.21.15.33:0/3097963304 shutdown_connections 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.423+0000 7f6679f9d700 1 -- 172.21.15.33:0/3097963304 wait complete. 2023-10-06T05:30:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.424+0000 7f6679f9d700 1 Processor -- start 2023-10-06T05:30:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.424+0000 7f6679f9d700 1 -- start start 2023-10-06T05:30:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f6679f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667406e140 0x7f6674140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f6679f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f6674157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f6679f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 0x7f6674158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f6679f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740ced40 con 0x7f6674159530 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f6679f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667416de20 con 0x7f667406e140 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f6679f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667416d990 con 0x7f66740afdd0 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f667979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 0x7f6674158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f667979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 0x7f6674158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55348/0 (socket says 172.21.15.33:55348) 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.425+0000 7f667979c700 1 -- 172.21.15.33:0/955390530 learned_addr learned my addr 172.21.15.33:0/955390530 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6673fff700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f6674157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6678f9b700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667406e140 0x7f6674140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f667979c700 1 -- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 msgr2=0x7f6674157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f667979c700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f6674157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f667979c700 1 -- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667406e140 msgr2=0x7f6674140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f667979c700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667406e140 0x7f6674140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f667979c700 1 -- 172.21.15.33:0/955390530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660009420 con 0x7f6674159530 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6678f9b700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667406e140 0x7f6674140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f667979c700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 0x7f6674158080 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f666c00b610 tx=0x7f666c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c027470 con 0x7f6674159530 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c0275f0 con 0x7f6674159530 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c0279b0 con 0x7f6674159530 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.426+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c009a20 con 0x7f6674159530 2023-10-06T05:30:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.427+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667416d340 con 0x7f6674159530 2023-10-06T05:30:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.428+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667404c380 con 0x7f6674159530 2023-10-06T05:30:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.429+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f666c024070 con 0x7f6674159530 2023-10-06T05:30:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.429+0000 7f6670ff9700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6658066f30 0x7f66580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.430+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f666c087040 con 0x7f6674159530 2023-10-06T05:30:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.430+0000 7f6678f9b700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6658066f30 0x7f66580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.430+0000 7f6678f9b700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6658066f30 0x7f66580693e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f66680058d0 tx=0x7f6668009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:04.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.434+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f666c054150 con 0x7f6674159530 2023-10-06T05:30:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.684+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66740b0730 con 0x7f6674159530 2023-10-06T05:30:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.685+0000 7f6670ff9700 1 -- 172.21.15.33:0/955390530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f666c057770 con 0x7f6674159530 2023-10-06T05:30:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:30:04.686 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6658066f30 msgr2=0x7f66580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6658066f30 0x7f66580693e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f66680058d0 tx=0x7f6668009040 comp rx=0 tx=0).stop 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 msgr2=0x7f6674158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 0x7f6674158080 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f666c00b610 tx=0x7f666c019000 comp rx=0 tx=0).stop 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 shutdown_connections 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6658066f30 0x7f66580693e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6674159530 0x7f6674158080 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667406e140 0x7f6674140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 --2- 172.21.15.33:0/955390530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66740afdd0 0x7f6674157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 >> 172.21.15.33:0/955390530 conn(0x7f667406c560 msgr2=0x7f667415b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 shutdown_connections 2023-10-06T05:30:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:04.687+0000 7f6679f9d700 1 -- 172.21.15.33:0/955390530 wait complete. 2023-10-06T05:30:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:30:04.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:30:04.707 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch background_file) 2023-10-06T05:30:04.748 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:30:04.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:30:04.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:04.879 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:30:04.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:04.880 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:30:04.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:30:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.144+0000 7f338359e700 1 Processor -- start 2023-10-06T05:30:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.144+0000 7f338359e700 1 -- start start 2023-10-06T05:30:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.145+0000 7f338359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33840c8b50 0x7f338406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.145+0000 7f338359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3384155f60 0x7f33840c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.145+0000 7f338359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338406ab60 con 0x7f3384155af0 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.145+0000 7f338359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338406b7e0 con 0x7f33840c8b50 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.145+0000 7f338359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338406ae40 con 0x7f3384155f60 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.145+0000 7f338259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33840c8b50 0x7f338406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f33840c8b50 0x7f338406e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55358/0 (socket says 172.21.15.33:55358) 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 -- 172.21.15.33:0/2784389371 learned_addr learned my addr 172.21.15.33:0/2784389371 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f3381d9b700 1 --1- 172.21.15.33:0/2784389371 >> v1:172.21.15.33:6789/0 conn(0x7f3384155af0 0x7f338406a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52316/0 (socket says 172.21.15.33:52316) 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f3382d9d700 1 --2- 172.21.15.33:0/2784389371 >> v2:172.21.15.167:3300/0 conn(0x7f3384155f60 0x7f33840c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 -- 172.21.15.33:0/2784389371 >> v2:172.21.15.167:3300/0 conn(0x7f3384155f60 msgr2=0x7f33840c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:30:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 --2- 172.21.15.33:0/2784389371 >> v2:172.21.15.167:3300/0 conn(0x7f3384155f60 0x7f33840c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 -- 172.21.15.33:0/2784389371 >> v1:172.21.15.33:6789/0 conn(0x7f3384155af0 legacy=0x7f338406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:05.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 -- 172.21.15.33:0/2784389371 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338406b120 con 0x7f33840c8b50 2023-10-06T05:30:05.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338159a700 1 -- 172.21.15.33:0/2784389371 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795161188 0 0) 0x7f338406ab60 con 0x7f3384155af0 2023-10-06T05:30:05.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338259c700 1 --2- 172.21.15.33:0/2784389371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33840c8b50 0x7f338406e960 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f338406fdf0 tx=0x7f3374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc67f79449d1953 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:05.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.146+0000 7f338159a700 1 -- 172.21.15.33:0/2784389371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337400b860 con 0x7f33840c8b50 2023-10-06T05:30:05.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338159a700 1 -- 172.21.15.33:0/2784389371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337400b9c0 con 0x7f33840c8b50 2023-10-06T05:30:05.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338159a700 1 -- 172.21.15.33:0/2784389371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337400bd60 con 0x7f33840c8b50 2023-10-06T05:30:05.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 -- 172.21.15.33:0/2784389371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33840c8b50 msgr2=0x7f338406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:05.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 --2- 172.21.15.33:0/2784389371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33840c8b50 0x7f338406e960 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f338406fdf0 tx=0x7f3374009250 comp rx=0 tx=0).stop 2023-10-06T05:30:05.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 -- 172.21.15.33:0/2784389371 shutdown_connections 2023-10-06T05:30:05.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 --2- 172.21.15.33:0/2784389371 >> v2:172.21.15.167:3300/0 conn(0x7f3384155f60 0x7f33840c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 --2- 172.21.15.33:0/2784389371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33840c8b50 0x7f338406e960 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 -- 172.21.15.33:0/2784389371 >> 172.21.15.33:0/2784389371 conn(0x7f338406d030 msgr2=0x7f338406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:05.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 -- 172.21.15.33:0/2784389371 shutdown_connections 2023-10-06T05:30:05.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.147+0000 7f338359e700 1 -- 172.21.15.33:0/2784389371 wait complete. 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.149+0000 7f338359e700 1 Processor -- start 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.149+0000 7f338359e700 1 -- start start 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33840c8b50 0x7f33841533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 0x7f33840af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384155f60 0x7f33840c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33840afbf0 con 0x7f3384155af0 2023-10-06T05:30:05.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33840c5ae0 con 0x7f3384155f60 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33841542f0 con 0x7f33840c8b50 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f3381d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 0x7f33840af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f3381d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 0x7f33840af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55368/0 (socket says 172.21.15.33:55368) 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f3381d9b700 1 -- 172.21.15.33:0/2651772264 learned_addr learned my addr 172.21.15.33:0/2651772264 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f338259c700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33840c8b50 0x7f33841533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.150+0000 7f3382d9d700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384155f60 0x7f33840c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f3381d9b700 1 -- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33840c8b50 msgr2=0x7f33841533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f3381d9b700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33840c8b50 0x7f33841533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f3381d9b700 1 -- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384155f60 msgr2=0x7f33840c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f3381d9b700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384155f60 0x7f33840c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f3381d9b700 1 -- 172.21.15.33:0/2651772264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740094f0 con 0x7f3384155af0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f3381d9b700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 0x7f33840af6b0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f337800c5b0 tx=0x7f3378001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378005a70 con 0x7f3384155af0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3378005bd0 con 0x7f3384155af0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780070d0 con 0x7f3384155af0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3384153a50 con 0x7f3384155af0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.151+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3384154010 con 0x7f3384155af0 2023-10-06T05:30:05.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.153+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338404c380 con 0x7f3384155af0 2023-10-06T05:30:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.153+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3378007480 con 0x7f3384155af0 2023-10-06T05:30:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.154+0000 7f33727fc700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3364066f30 0x7f33640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.155+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3378083c30 con 0x7f3384155af0 2023-10-06T05:30:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.155+0000 7f338259c700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3364066f30 0x7f33640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:05.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.156+0000 7f338259c700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3364066f30 0x7f33640693e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f338406fdf0 tx=0x7f3374017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:05.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.158+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3378050ca0 con 0x7f3384155af0 2023-10-06T05:30:05.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.406+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f338406b400 con 0x7f3384155af0 2023-10-06T05:30:05.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.407+0000 7f33727fc700 1 -- 172.21.15.33:0/2651772264 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v176) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3378017020 con 0x7f3384155af0 2023-10-06T05:30:05.411 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:30:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3364066f30 msgr2=0x7f33640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3364066f30 0x7f33640693e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f338406fdf0 tx=0x7f3374017000 comp rx=0 tx=0).stop 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 msgr2=0x7f33840af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 0x7f33840af6b0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f337800c5b0 tx=0x7f3378001040 comp rx=0 tx=0).stop 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 shutdown_connections 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3364066f30 0x7f33640693e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384155f60 0x7f33840c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33840c8b50 0x7f33841533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 --2- 172.21.15.33:0/2651772264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384155af0 0x7f33840af6b0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 >> 172.21.15.33:0/2651772264 conn(0x7f338406d030 msgr2=0x7f33840cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 shutdown_connections 2023-10-06T05:30:05.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:05.409+0000 7f338359e700 1 -- 172.21.15.33:0/2651772264 wait complete. 2023-10-06T05:30:05.428 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:30:05.428 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:30:05.428 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:30:05.429 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:30:05.429 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:05.429 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:30:05.429 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:30:05.429 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:30:05.429 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:05.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:05.429 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:30:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:05.467 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41352sec preferred_lft 41352sec 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:30:05.468 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:30:05.469 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.470 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:30:05.470 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:30:05.470 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:30:05.470 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:30:05.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:05.470 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:30:05.566 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:30:05.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:05.569 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:30:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:30:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:30:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:30:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:30:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:30:05.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:05.609 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:30:05.609 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:30:05.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:30:05.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:30:05.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:30:05.610 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:30:05.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:05.808 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:30:05.845 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:30:05.845 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:30:05.845 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:30:05.845 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:05.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:05.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:30:05.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:30:05.950 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:05.986 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:05.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:30:05.987 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:06.062 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:30:06.062 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:30:06.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:06.195 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:06.195 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:30:06.195 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:30:06.211 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:06.211 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:30:06.211 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:30:06.211 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:30:06.211 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:30:06.233 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:30:06.237 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:30:06.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:06.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:30:06.299 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:30:06.299 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:30:06.299 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:30:06.299 DEBUG:teuthology.orchestra.run.smithi033:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'r'"'"') as f: 2023-10-06T05:30:06.299 DEBUG:teuthology.orchestra.run.smithi033:> while True: 2023-10-06T05:30:06.299 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(1) 2023-10-06T05:30:06.300 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:30:06.304 DEBUG:teuthology.orchestra.run.smithi033:> stat /home/ubuntu/cephtest/mnt.0/background_file 2023-10-06T05:30:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0/background_file 2023-10-06T05:30:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 4194304 regular empty file 2023-10-06T05:30:06.343 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 2bh/43d Inode: 1099511627776 Links: 1 2023-10-06T05:30:06.343 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0664/-rw-rw-r--) Uid: ( 1000/ ubuntu) Gid: ( 1275/ ubuntu) 2023-10-06T05:30:06.343 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-06T05:30:06.343 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 05:30:04.746289433 +0000 2023-10-06T05:30:06.344 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 05:30:04.746289433 +0000 2023-10-06T05:30:06.344 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 05:30:04.746289433 +0000 2023-10-06T05:30:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T05:30:06.344 INFO:tasks.cephfs.mount:File background_file became visible from 0 after 0s 2023-10-06T05:30:06.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:30:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.573+0000 7f0e5b19d700 1 Processor -- start 2023-10-06T05:30:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.573+0000 7f0e5b19d700 1 -- start start 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.574+0000 7f0e5b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e54158700 0x7f0e540adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.574+0000 7f0e5b19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e54158ba0 con 0x7f0e54155140 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.574+0000 7f0e5b19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e540ae490 con 0x7f0e540699c0 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.574+0000 7f0e5b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e540c8e40 con 0x7f0e54158700 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e54158700 0x7f0e540adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5999a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e54158700 0x7f0e540adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55378/0 (socket says 172.21.15.33:55378) 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5999a700 1 -- 172.21.15.33:0/2639332012 learned_addr learned my addr 172.21.15.33:0/2639332012 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5a19b700 1 --1- 172.21.15.33:0/2639332012 >> v1:172.21.15.33:6789/0 conn(0x7f0e54155140 0x7f0e540c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52328/0 (socket says 172.21.15.33:52328) 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5999a700 1 -- 172.21.15.33:0/2639332012 >> v1:172.21.15.167:6789/0 conn(0x7f0e540699c0 legacy=0x7f0e540c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5999a700 1 -- 172.21.15.33:0/2639332012 >> v1:172.21.15.33:6789/0 conn(0x7f0e54155140 legacy=0x7f0e540c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.575+0000 7f0e5999a700 1 -- 172.21.15.33:0/2639332012 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540c99e0 con 0x7f0e54158700 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5999a700 1 --2- 172.21.15.33:0/2639332012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e540adf50 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f0e440099e0 tx=0x7f0e44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0121f000b1dadfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e59199700 1 -- 172.21.15.33:0/2639332012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400b760 con 0x7f0e54158700 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e59199700 1 -- 172.21.15.33:0/2639332012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4400b8c0 con 0x7f0e54158700 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e59199700 1 -- 172.21.15.33:0/2639332012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400bc60 con 0x7f0e54158700 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5b19d700 1 -- 172.21.15.33:0/2639332012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 msgr2=0x7f0e540adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/2639332012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e540adf50 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f0e440099e0 tx=0x7f0e44009250 comp rx=0 tx=0).stop 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5b19d700 1 -- 172.21.15.33:0/2639332012 shutdown_connections 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/2639332012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e540adf50 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5b19d700 1 -- 172.21.15.33:0/2639332012 >> 172.21.15.33:0/2639332012 conn(0x7f0e5406d030 msgr2=0x7f0e5406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.576+0000 7f0e5b19d700 1 -- 172.21.15.33:0/2639332012 shutdown_connections 2023-10-06T05:30:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.577+0000 7f0e5b19d700 1 -- 172.21.15.33:0/2639332012 wait complete. 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.578+0000 7f0e5b19d700 1 Processor -- start 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.578+0000 7f0e5b19d700 1 -- start start 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e540699c0 0x7f0e540b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e54155140 0x7f0e541531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e54157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e540ae260 con 0x7f0e54158700 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541578d0 con 0x7f0e540699c0 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541557d0 con 0x7f0e54155140 2023-10-06T05:30:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5a99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e54157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5a99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e54157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55394/0 (socket says 172.21.15.33:55394) 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5a99c700 1 -- 172.21.15.33:0/37984809 learned_addr learned my addr 172.21.15.33:0/37984809 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5a19b700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e540699c0 0x7f0e540b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.579+0000 7f0e5999a700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e54155140 0x7f0e541531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a99c700 1 -- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e54155140 msgr2=0x7f0e541531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a99c700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e54155140 0x7f0e541531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a99c700 1 -- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e540699c0 msgr2=0x7f0e540b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a99c700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e540699c0 0x7f0e540b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a99c700 1 -- 172.21.15.33:0/37984809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440094f0 con 0x7f0e54158700 2023-10-06T05:30:06.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a19b700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e540699c0 0x7f0e540b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:30:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5a99c700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e54157d60 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00c910 tx=0x7f0e4c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c012630 con 0x7f0e54158700 2023-10-06T05:30:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c012790 con 0x7f0e54158700 2023-10-06T05:30:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e54156fa0 con 0x7f0e54158700 2023-10-06T05:30:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c012b30 con 0x7f0e54158700 2023-10-06T05:30:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.580+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e54157560 con 0x7f0e54158700 2023-10-06T05:30:06.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.581+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e54157a60 con 0x7f0e54158700 2023-10-06T05:30:06.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.582+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e4c00a3b0 con 0x7f0e54158700 2023-10-06T05:30:06.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.584+0000 7f0e4a7fc700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e38066f30 0x7f0e380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:30:06.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.584+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e4c084980 con 0x7f0e54158700 2023-10-06T05:30:06.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.584+0000 7f0e5a19b700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e38066f30 0x7f0e380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:30:06.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.585+0000 7f0e5a19b700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e38066f30 0x7f0e380693e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0e50007fd0 tx=0x7f0e50007db0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:30:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.588+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e4c0519f0 con 0x7f0e54158700 2023-10-06T05:30:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.835+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e5404c380 con 0x7f0e54158700 2023-10-06T05:30:06.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.836+0000 7f0e4a7fc700 1 -- 172.21.15.33:0/37984809 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f0e4c055010 con 0x7f0e54158700 2023-10-06T05:30:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:30:06.837 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e38066f30 msgr2=0x7f0e380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e38066f30 0x7f0e380693e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0e50007fd0 tx=0x7f0e50007db0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 msgr2=0x7f0e54157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e54157d60 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00c910 tx=0x7f0e4c001710 comp rx=0 tx=0).stop 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 shutdown_connections 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0e38066f30 0x7f0e380693e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e54158700 0x7f0e54157d60 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e540699c0 0x7f0e540b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 --2- 172.21.15.33:0/37984809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e54155140 0x7f0e541531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:30:06.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 >> 172.21.15.33:0/37984809 conn(0x7f0e5406d030 msgr2=0x7f0e540cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:30:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 shutdown_connections 2023-10-06T05:30:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:30:06.838+0000 7f0e5b19d700 1 -- 172.21.15.33:0/37984809 wait complete. 2023-10-06T05:30:06.841 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:30:06.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T05:30:07.069 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":8390,"entity":{"name":{"type":"client","num":8390},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":1211457731}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":1,"uptime":0.82498703900000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9964589177685892,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8390 v1:172.21.15.33:0/1211457731","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":9967,"entity":{"name":{"type":"client","num":9967},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1654832570}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.3189478590000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9817242415801712,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.9967 v1:192.168.0.1:0/1654832570","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:30:07.070 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 1211457731, "type": "v1" }, "name": { "num": 8390, "type": "client" } }, "id": 8390, "inst": "client.8390 v1:172.21.15.33:0/1211457731", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9964589177685892 }, "state": "open", "uptime": 0.824987039 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1654832570, "type": "v1" }, "name": { "num": 9967, "type": "client" } }, "id": 9967, "inst": "client.9967 v1:192.168.0.1:0/1654832570", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9817242415801712 }, "state": "open", "uptime": 3.318947859 } ] 2023-10-06T05:30:07.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:30:07.071 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:30:07.142 INFO:teuthology.orchestra.run.smithi033.stdout:client8390 2023-10-06T05:30:07.142 DEBUG:teuthology.orchestra.run.smithi167:> stat --printf=%s /home/ubuntu/cephtest/mnt.1/background_file 2023-10-06T05:30:07.184 INFO:teuthology.orchestra.run.smithi167.stdout:0 2023-10-06T05:30:07.184 INFO:tasks.cephfs.mount:File background_file became visible with size 0 from 1 after 0s 2023-10-06T05:30:07.184 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:30:07.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:30:07.184 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:30:12.256 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:30:12.256 DEBUG:teuthology.orchestra.run.smithi167:> import os 2023-10-06T05:30:12.256 DEBUG:teuthology.orchestra.run.smithi167:> import time 2023-10-06T05:30:12.256 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> while True: 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> os.write(fd, b'"'"'content'"'"') 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> time.sleep(1) 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> if not False: 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> break 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> os.close(fd) 2023-10-06T05:30:12.257 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:30:20.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:30:20.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:30:50.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:30:50.805 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:31:12.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.049+0000 7feebd99d700 1 Processor -- start 2023-10-06T05:31:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.049+0000 7feebd99d700 1 -- start start 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.050+0000 7feebd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.050+0000 7feebd99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feeb80c7040 0x7feeb80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.050+0000 7feebd99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb80cb5f0 con 0x7feeb8147760 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feebd99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80ca9e0 con 0x7feeb80c7040 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feebd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb80cacc0 con 0x7feeb8147390 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feeb7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7feeb8147760 0x7feeb80c6930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39866/0 (socket says 172.21.15.33:39866) 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feeb7fff700 1 -- 172.21.15.33:0/1056874390 learned_addr learned my addr 172.21.15.33:0/1056874390 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feebc99b700 1 --2- 172.21.15.33:0/1056874390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feebd19c700 1 --2- 172.21.15.33:0/1056874390 >> v2:172.21.15.167:3300/0 conn(0x7feeb80c7040 0x7feeb80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feeb77fe700 1 -- 172.21.15.33:0/1056874390 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326914429 0 0) 0x7feeb80cb5f0 con 0x7feeb8147760 2023-10-06T05:31:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.051+0000 7feeb77fe700 1 -- 172.21.15.33:0/1056874390 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea8003570 con 0x7feeb8147760 2023-10-06T05:31:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feebc99b700 1 -- 172.21.15.33:0/1056874390 >> v2:172.21.15.167:3300/0 conn(0x7feeb80c7040 msgr2=0x7feeb80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feebc99b700 1 --2- 172.21.15.33:0/1056874390 >> v2:172.21.15.167:3300/0 conn(0x7feeb80c7040 0x7feeb80c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feebc99b700 1 -- 172.21.15.33:0/1056874390 >> v1:172.21.15.33:6789/0 conn(0x7feeb8147760 legacy=0x7feeb80c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feebc99b700 1 -- 172.21.15.33:0/1056874390 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80cafa0 con 0x7feeb8147390 2023-10-06T05:31:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feebd19c700 1 --2- 172.21.15.33:0/1056874390 >> v2:172.21.15.167:3300/0 conn(0x7feeb80c7040 0x7feeb80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:13.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feebc99b700 1 --2- 172.21.15.33:0/1056874390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80cba80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feeac0098e0 tx=0x7feeac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cfc8775fa52e645 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feeb77fe700 1 -- 172.21.15.33:0/1056874390 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac011630 con 0x7feeb8147390 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feeb77fe700 1 -- 172.21.15.33:0/1056874390 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac011790 con 0x7feeb8147390 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.052+0000 7feeb77fe700 1 -- 172.21.15.33:0/1056874390 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac011b30 con 0x7feeb8147390 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 -- 172.21.15.33:0/1056874390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 msgr2=0x7feeb80cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 --2- 172.21.15.33:0/1056874390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80cba80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feeac0098e0 tx=0x7feeac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 -- 172.21.15.33:0/1056874390 shutdown_connections 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 --2- 172.21.15.33:0/1056874390 >> v2:172.21.15.167:3300/0 conn(0x7feeb80c7040 0x7feeb80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 --2- 172.21.15.33:0/1056874390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80cba80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 -- 172.21.15.33:0/1056874390 >> 172.21.15.33:0/1056874390 conn(0x7feeb806d030 msgr2=0x7feeb806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 -- 172.21.15.33:0/1056874390 shutdown_connections 2023-10-06T05:31:13.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.053+0000 7feebd99d700 1 -- 172.21.15.33:0/1056874390 wait complete. 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.054+0000 7feebd99d700 1 Processor -- start 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.055+0000 7feebd99d700 1 -- start start 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 0x7feeb8145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb8147760 0x7feeb815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb80c5c40 con 0x7feeb80c7040 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb80c5ec0 con 0x7feeb8147760 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb80c61a0 con 0x7feeb8147390 2023-10-06T05:31:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 0x7feeb8145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 0x7feeb8145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54176/0 (socket says 172.21.15.33:54176) 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 -- 172.21.15.33:0/424348345 learned_addr learned my addr 172.21.15.33:0/424348345 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feeb7fff700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebd19c700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb8147760 0x7feeb815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 -- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 msgr2=0x7feeb80c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 -- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb8147760 msgr2=0x7feeb815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb8147760 0x7feeb815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.056+0000 7feebc99b700 1 -- 172.21.15.33:0/424348345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0009a20 con 0x7feeb80c7040 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.057+0000 7feebc99b700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 0x7feeb8145e30 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7feeb80cae00 tx=0x7feeac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.057+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac011b10 con 0x7feeb80c7040 2023-10-06T05:31:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.057+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac026450 con 0x7feeb80c7040 2023-10-06T05:31:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.057+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac0267d0 con 0x7feeb80c7040 2023-10-06T05:31:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.057+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac009420 con 0x7feeb80c7040 2023-10-06T05:31:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.057+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb80c6760 con 0x7feeb80c7040 2023-10-06T05:31:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.058+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb804c380 con 0x7feeb80c7040 2023-10-06T05:31:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.059+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeac026bc0 con 0x7feeb80c7040 2023-10-06T05:31:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.060+0000 7feeb4ff9700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.060+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feeac086180 con 0x7feeb80c7040 2023-10-06T05:31:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.060+0000 7feeb7fff700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.061+0000 7feeb7fff700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7feeb80c9b20 tx=0x7feea4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.064+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeac053190 con 0x7feeb80c7040 2023-10-06T05:31:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.316+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feeb806fbf0 con 0x7feeb80c7040 2023-10-06T05:31:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.317+0000 7feeb4ff9700 1 -- 172.21.15.33:0/424348345 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7feeac0567b0 con 0x7feeb80c7040 2023-10-06T05:31:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:13.319 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:31:13.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee9c066f30 msgr2=0x7fee9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:13.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7feeb80c9b20 tx=0x7feea4009040 comp rx=0 tx=0).stop 2023-10-06T05:31:13.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 msgr2=0x7feeb8145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 0x7feeb8145e30 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7feeb80cae00 tx=0x7feeac017000 comp rx=0 tx=0).stop 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 shutdown_connections 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb8147760 0x7feeb815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb80c7040 0x7feeb8145e30 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 --2- 172.21.15.33:0/424348345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb8147390 0x7feeb80c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 >> 172.21.15.33:0/424348345 conn(0x7feeb806d030 msgr2=0x7feeb80b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 shutdown_connections 2023-10-06T05:31:13.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:13.319+0000 7feebd99d700 1 -- 172.21.15.33:0/424348345 wait complete. 2023-10-06T05:31:13.323 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:31:13.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T05:31:13.553 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":8390,"entity":{"name":{"type":"client","num":8390},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":1211457731}},"state":"stale","num_leases":0,"num_caps":2,"request_load_avg":0,"uptime":67.309945709000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":5.5678351252008955,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8390 v1:172.21.15.33:0/1211457731","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":9967,"entity":{"name":{"type":"client","num":9967},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1654832570}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":0,"uptime":69.803906529000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":4.2832044333659178,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.9967 v1:192.168.0.1:0/1654832570","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:31:13.554 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 1211457731, "type": "v1" }, "name": { "num": 8390, "type": "client" } }, "id": 8390, "inst": "client.8390 v1:172.21.15.33:0/1211457731", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 5.5678351252008955 }, "state": "stale", "uptime": 67.309945709 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1654832570, "type": "v1" }, "name": { "num": 9967, "type": "client" } }, "id": 9967, "inst": "client.9967 v1:192.168.0.1:0/1654832570", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4.283204433365918 }, "state": "open", "uptime": 69.803906529 } ] 2023-10-06T05:31:13.554 INFO:tasks.cephfs.test_client_recovery:cap_waiter waited 60.5815863609314s 2023-10-06T05:31:13.554 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:31:13.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:13.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:31:13.632 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:31:13.632 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:31:13.764 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:31:13.851 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:31:13.851 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:31:13.852 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:31:13.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:31:13.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:31:20.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:31:20.916 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:31:30.984 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:31:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:30.984 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:31:31.025 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:31:31.025 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:31:31.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:31:31.025 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:31:31.113 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:31:31.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:31.113 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:31:31.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.340+0000 7fa5fb59d700 1 Processor -- start 2023-10-06T05:31:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.340+0000 7fa5fb59d700 1 -- start start 2023-10-06T05:31:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.341+0000 7fa5fb59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5f4155140 0x7fa5f4158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.341+0000 7fa5fb59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40adf50 0x7fa5f40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.341+0000 7fa5fb59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f40c8e40 con 0x7fa5f40c9810 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fb59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40ae6c0 con 0x7fa5f4155140 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fb59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f40ae9a0 con 0x7fa5f40adf50 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fa59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5f4155140 0x7fa5f4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fad9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa5f40c9810 0x7fa5f40699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34068/0 (socket says 172.21.15.33:34068) 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fad9c700 1 -- 172.21.15.33:0/2228407300 learned_addr learned my addr 172.21.15.33:0/2228407300 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5f9d9a700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40adf50 0x7fa5f40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fa59b700 1 -- 172.21.15.33:0/2228407300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40adf50 msgr2=0x7fa5f40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fa59b700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40adf50 0x7fa5f40c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fa59b700 1 -- 172.21.15.33:0/2228407300 >> v1:172.21.15.33:6789/0 conn(0x7fa5f40c9810 legacy=0x7fa5f40699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5fa59b700 1 -- 172.21.15.33:0/2228407300 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40c8ad0 con 0x7fa5f4155140 2023-10-06T05:31:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5f9d9a700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40adf50 0x7fa5f40c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.342+0000 7fa5f9599700 1 -- 172.21.15.33:0/2228407300 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324574897 0 0) 0x7fa5f40c8e40 con 0x7fa5f40c9810 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5fa59b700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f4158700 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00098e0 tx=0x7fa5f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d40129001b74e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5f9599700 1 -- 172.21.15.33:0/2228407300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0009e80 con 0x7fa5f4155140 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5f9599700 1 -- 172.21.15.33:0/2228407300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000b860 con 0x7fa5f4155140 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5f9599700 1 -- 172.21.15.33:0/2228407300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000bbe0 con 0x7fa5f4155140 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5fb59d700 1 -- 172.21.15.33:0/2228407300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 msgr2=0x7fa5f4158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f4158700 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00098e0 tx=0x7fa5f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5fb59d700 1 -- 172.21.15.33:0/2228407300 shutdown_connections 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f4158700 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.343+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/2228407300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40adf50 0x7fa5f40c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.344+0000 7fa5fb59d700 1 -- 172.21.15.33:0/2228407300 >> 172.21.15.33:0/2228407300 conn(0x7fa5f406d030 msgr2=0x7fa5f406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.344+0000 7fa5fb59d700 1 -- 172.21.15.33:0/2228407300 shutdown_connections 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.344+0000 7fa5fb59d700 1 -- 172.21.15.33:0/2228407300 wait complete. 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.345+0000 7fa5fb59d700 1 Processor -- start 2023-10-06T05:31:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.345+0000 7fa5fb59d700 1 -- start start 2023-10-06T05:31:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fb59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5f40adf50 0x7fa5f40a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fb59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40c9810 0x7fa5f40ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fb59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fb59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f4153580 con 0x7fa5f4155140 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fb59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f40caad0 con 0x7fa5f40adf50 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fb59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f4159d10 con 0x7fa5f40c9810 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.346+0000 7fa5fad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f40caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58788/0 (socket says 172.21.15.33:58788) 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 -- 172.21.15.33:0/1259725782 learned_addr learned my addr 172.21.15.33:0/1259725782 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:31.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5f9d9a700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40c9810 0x7fa5f40ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fa59b700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5f40adf50 0x7fa5f40a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 -- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40c9810 msgr2=0x7fa5f40ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40c9810 0x7fa5f40ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 -- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5f40adf50 msgr2=0x7fa5f40a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5f40adf50 0x7fa5f40a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 -- 172.21.15.33:0/1259725782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0009420 con 0x7fa5f4155140 2023-10-06T05:31:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fa59b700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5f40adf50 0x7fa5f40a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5fad9c700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f40caf60 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00cc90 tx=0x7fa5ec001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.347+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00cea0 con 0x7fa5f4155140 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.348+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec00ab10 con 0x7fa5f4155140 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.348+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f4159ff0 con 0x7fa5f4155140 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.348+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0070f0 con 0x7fa5f4155140 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.348+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f40adc40 con 0x7fa5f4155140 2023-10-06T05:31:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.349+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f406bad0 con 0x7fa5f4155140 2023-10-06T05:31:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.350+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5ec0074e0 con 0x7fa5f4155140 2023-10-06T05:31:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.350+0000 7fa5ea7fc700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5d8066f30 0x7fa5d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.351+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa5ec084c10 con 0x7fa5f4155140 2023-10-06T05:31:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.351+0000 7fa5fa59b700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5d8066f30 0x7fa5d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.351+0000 7fa5fa59b700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5d8066f30 0x7fa5d80693e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40cbdd0 tx=0x7fa5f0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.355+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5ec051ca0 con 0x7fa5f4155140 2023-10-06T05:31:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.601+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5f404c380 con 0x7fa5f4155140 2023-10-06T05:31:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.602+0000 7fa5ea7fc700 1 -- 172.21.15.33:0/1259725782 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7fa5ec0552c0 con 0x7fa5f4155140 2023-10-06T05:31:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:31.605 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:31:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5d8066f30 msgr2=0x7fa5d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5d8066f30 0x7fa5d80693e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40cbdd0 tx=0x7fa5f0017000 comp rx=0 tx=0).stop 2023-10-06T05:31:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 msgr2=0x7fa5f40caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f40caf60 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00cc90 tx=0x7fa5ec001a00 comp rx=0 tx=0).stop 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 shutdown_connections 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5d8066f30 0x7fa5d80693e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5f4155140 0x7fa5f40caf60 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5f40adf50 0x7fa5f40a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 --2- 172.21.15.33:0/1259725782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5f40c9810 0x7fa5f40ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 >> 172.21.15.33:0/1259725782 conn(0x7fa5f406d030 msgr2=0x7fa5f4153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 shutdown_connections 2023-10-06T05:31:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.604+0000 7fa5fb59d700 1 -- 172.21.15.33:0/1259725782 wait complete. 2023-10-06T05:31:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:31:31.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.810+0000 7fa488b9d700 1 Processor -- start 2023-10-06T05:31:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.810+0000 7fa488b9d700 1 -- start start 2023-10-06T05:31:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.811+0000 7fa488b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa484155af0 0x7fa48406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.811+0000 7fa488b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.811+0000 7fa488b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48406ab60 con 0x7fa48406a6a0 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.811+0000 7fa488b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48406b7e0 con 0x7fa484155af0 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.811+0000 7fa488b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48406ae40 con 0x7fa4840c7920 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa484155af0 0x7fa48406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa483fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa484155af0 0x7fa48406a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:60460/0 (socket says 172.21.15.33:60460) 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 -- 172.21.15.33:0/1584134871 learned_addr learned my addr 172.21.15.33:0/1584134871 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:31.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa482ffd700 1 --1- 172.21.15.33:0/1584134871 >> v1:172.21.15.167:6789/0 conn(0x7fa48406a6a0 0x7fa4840c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:42420/0 (socket says 172.21.15.33:42420) 2023-10-06T05:31:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4827fc700 1 -- 172.21.15.33:0/1584134871 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692350916 0 0) 0x7fa48406ab60 con 0x7fa48406a6a0 2023-10-06T05:31:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4827fc700 1 -- 172.21.15.33:0/1584134871 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa46c003570 con 0x7fa48406a6a0 2023-10-06T05:31:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 -- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 msgr2=0x7fa4840c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 --2- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 -- 172.21.15.33:0/1584134871 >> v1:172.21.15.167:6789/0 conn(0x7fa48406a6a0 legacy=0x7fa4840c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa4837fe700 1 -- 172.21.15.33:0/1584134871 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48406b120 con 0x7fa484155af0 2023-10-06T05:31:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.812+0000 7fa483fff700 1 --2- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa4837fe700 1 --2- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa484155af0 0x7fa48406a160 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa47400bc50 tx=0x7fa47400b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458610d3b35e658c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa4827fc700 1 -- 172.21.15.33:0/1584134871 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47400db40 con 0x7fa484155af0 2023-10-06T05:31:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa4827fc700 1 -- 172.21.15.33:0/1584134871 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47400dca0 con 0x7fa484155af0 2023-10-06T05:31:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa4827fc700 1 -- 172.21.15.33:0/1584134871 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740196b0 con 0x7fa484155af0 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa488b9d700 1 -- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa484155af0 msgr2=0x7fa48406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa488b9d700 1 --2- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa484155af0 0x7fa48406a160 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa47400bc50 tx=0x7fa47400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa488b9d700 1 -- 172.21.15.33:0/1584134871 shutdown_connections 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa488b9d700 1 --2- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa484155af0 0x7fa48406a160 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa488b9d700 1 --2- 172.21.15.33:0/1584134871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.813+0000 7fa488b9d700 1 -- 172.21.15.33:0/1584134871 >> 172.21.15.33:0/1584134871 conn(0x7fa48406d030 msgr2=0x7fa48406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.814+0000 7fa488b9d700 1 -- 172.21.15.33:0/1584134871 shutdown_connections 2023-10-06T05:31:31.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.814+0000 7fa488b9d700 1 -- 172.21.15.33:0/1584134871 wait complete. 2023-10-06T05:31:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.815+0000 7fa488b9d700 1 Processor -- start 2023-10-06T05:31:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.815+0000 7fa488b9d700 1 -- start start 2023-10-06T05:31:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.816+0000 7fa488b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48406a6a0 0x7fa484150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.816+0000 7fa488b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.816+0000 7fa488b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 0x7fa4840c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.816+0000 7fa488b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48416aac0 con 0x7fa484155af0 2023-10-06T05:31:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.816+0000 7fa488b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840a9a10 con 0x7fa48406a6a0 2023-10-06T05:31:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.816+0000 7fa488b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840a9cf0 con 0x7fa4840c7920 2023-10-06T05:31:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 0x7fa4840c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 0x7fa4840c33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58804/0 (socket says 172.21.15.33:58804) 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa4837fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48406a6a0 0x7fa484150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 -- 172.21.15.33:0/624238829 learned_addr learned my addr 172.21.15.33:0/624238829 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa482ffd700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 -- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 msgr2=0x7fa4840afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 -- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48406a6a0 msgr2=0x7fa484150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48406a6a0 0x7fa484150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 -- 172.21.15.33:0/624238829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c009a20 con 0x7fa484155af0 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.817+0000 7fa483fff700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 0x7fa4840c33a0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009f90 tx=0x7fa47c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.818+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00cb70 con 0x7fa484155af0 2023-10-06T05:31:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.818+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c00f040 con 0x7fa484155af0 2023-10-06T05:31:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.818+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c014480 con 0x7fa484155af0 2023-10-06T05:31:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.818+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47400b790 con 0x7fa484155af0 2023-10-06T05:31:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.818+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4840c38e0 con 0x7fa484155af0 2023-10-06T05:31:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.819+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48404c380 con 0x7fa484155af0 2023-10-06T05:31:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.820+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa47c0147f0 con 0x7fa484155af0 2023-10-06T05:31:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.820+0000 7fa46b7fe700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa470066f30 0x7fa4700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.821+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa47c0853f0 con 0x7fa484155af0 2023-10-06T05:31:31.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.821+0000 7fa4837fe700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa470066f30 0x7fa4700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:31.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.822+0000 7fa4837fe700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa470066f30 0x7fa4700693e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa47400bc50 tx=0x7fa47401b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:31.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:31.824+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa47c052400 con 0x7fa484155af0 2023-10-06T05:31:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.030+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa48406b400 con 0x7fa484155af0 2023-10-06T05:31:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.031+0000 7fa46b7fe700 1 -- 172.21.15.33:0/624238829 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v228) v1 ==== 74+0+17025 (secure 0 0 0) 0x7fa47c055a20 con 0x7fa484155af0 2023-10-06T05:31:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:32.034 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":228,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:30:09.896755+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:50.569789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:51.932764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.2","mappings":[{"from":0,"to":2}]},{"pgid":"34.20","mappings":[{"from":0,"to":1}]},{"pgid":"34.29","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa470066f30 msgr2=0x7fa4700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa470066f30 0x7fa4700693e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa47400bc50 tx=0x7fa47401b000 comp rx=0 tx=0).stop 2023-10-06T05:31:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 msgr2=0x7fa4840c33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 0x7fa4840c33a0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009f90 tx=0x7fa47c00b040 comp rx=0 tx=0).stop 2023-10-06T05:31:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 shutdown_connections 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa470066f30 0x7fa4700693e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa484155af0 0x7fa4840c33a0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48406a6a0 0x7fa484150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 --2- 172.21.15.33:0/624238829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4840c7920 0x7fa4840afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 >> 172.21.15.33:0/624238829 conn(0x7fa48406d030 msgr2=0x7fa4840cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 shutdown_connections 2023-10-06T05:31:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.033+0000 7fa488b9d700 1 -- 172.21.15.33:0/624238829 wait complete. 2023-10-06T05:31:32.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 33, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:29:50.569789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '225', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 34, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:29:51.932764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '226', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:31:32.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:31:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.242+0000 7fd00259d700 1 Processor -- start 2023-10-06T05:31:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.242+0000 7fd00259d700 1 -- start start 2023-10-06T05:31:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd00259d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcffc0c9490 0x7fcffc155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd00259d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0b06b0 con 0x7fcffc155a50 2023-10-06T05:31:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd00259d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0c99e0 con 0x7fcffc0afb20 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd00259d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc0c9cc0 con 0x7fcffc0c9490 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd000d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcffc0c9490 0x7fcffc155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd000d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcffc0c9490 0x7fcffc155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58818/0 (socket says 172.21.15.33:58818) 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd00159b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcffc155a50 0x7fcffc0af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34080/0 (socket says 172.21.15.33:34080) 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd000d9a700 1 -- 172.21.15.33:0/337891452 learned_addr learned my addr 172.21.15.33:0/337891452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd000d9a700 1 -- 172.21.15.33:0/337891452 >> v1:172.21.15.167:6789/0 conn(0x7fcffc0afb20 legacy=0x7fcffc0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd000d9a700 1 -- 172.21.15.33:0/337891452 >> v1:172.21.15.33:6789/0 conn(0x7fcffc155a50 legacy=0x7fcffc0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fd000d9a700 1 -- 172.21.15.33:0/337891452 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0c9fa0 con 0x7fcffc0c9490 2023-10-06T05:31:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.244+0000 7fcff3fff700 1 -- 172.21.15.33:0/337891452 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593888148 0 0) 0x7fcffc0b06b0 con 0x7fcffc155a50 2023-10-06T05:31:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd000d9a700 1 --2- 172.21.15.33:0/337891452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc155e20 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0099b0 tx=0x7fcfec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337a160e5a94c58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fcff3fff700 1 -- 172.21.15.33:0/337891452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00b860 con 0x7fcffc0c9490 2023-10-06T05:31:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fcff3fff700 1 -- 172.21.15.33:0/337891452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec00b9c0 con 0x7fcffc0c9490 2023-10-06T05:31:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fcff3fff700 1 -- 172.21.15.33:0/337891452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00bd60 con 0x7fcffc0c9490 2023-10-06T05:31:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd00259d700 1 -- 172.21.15.33:0/337891452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 msgr2=0x7fcffc155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd00259d700 1 --2- 172.21.15.33:0/337891452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc155e20 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0099b0 tx=0x7fcfec009250 comp rx=0 tx=0).stop 2023-10-06T05:31:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd00259d700 1 -- 172.21.15.33:0/337891452 shutdown_connections 2023-10-06T05:31:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd00259d700 1 --2- 172.21.15.33:0/337891452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc155e20 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd00259d700 1 -- 172.21.15.33:0/337891452 >> 172.21.15.33:0/337891452 conn(0x7fcffc06d030 msgr2=0x7fcffc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.245+0000 7fd00259d700 1 -- 172.21.15.33:0/337891452 shutdown_connections 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.246+0000 7fd00259d700 1 -- 172.21.15.33:0/337891452 wait complete. 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.247+0000 7fd00259d700 1 Processor -- start 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.247+0000 7fd00259d700 1 -- start start 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcffc0afb20 0x7fcffc141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcffc155a50 0x7fcffc15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15d4b0 con 0x7fcffc0c9490 2023-10-06T05:31:32.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15d790 con 0x7fcffc155a50 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15da70 con 0x7fcffc0afb20 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd000d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd000d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58834/0 (socket says 172.21.15.33:58834) 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd000d9a700 1 -- 172.21.15.33:0/3505383614 learned_addr learned my addr 172.21.15.33:0/3505383614 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd00159b700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcffc0afb20 0x7fcffc141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.248+0000 7fd001d9c700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcffc155a50 0x7fcffc15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd000d9a700 1 -- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcffc0afb20 msgr2=0x7fcffc141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd000d9a700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcffc0afb20 0x7fcffc141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd000d9a700 1 -- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcffc155a50 msgr2=0x7fcffc15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd000d9a700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcffc155a50 0x7fcffc15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd000d9a700 1 -- 172.21.15.33:0/3505383614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0094f0 con 0x7fcffc0c9490 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd001d9c700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcffc155a50 0x7fcffc15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd00159b700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcffc0afb20 0x7fcffc141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd000d9a700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc15ecb0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009290 tx=0x7fcfec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:32.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec011e10 con 0x7fcffc0c9490 2023-10-06T05:31:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec01b680 con 0x7fcffc0c9490 2023-10-06T05:31:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec01ba40 con 0x7fcffc0c9490 2023-10-06T05:31:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc15dd50 con 0x7fcffc0c9490 2023-10-06T05:31:32.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.249+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc1420f0 con 0x7fcffc0c9490 2023-10-06T05:31:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.250+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc15d5f0 con 0x7fcffc0c9490 2023-10-06T05:31:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.251+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfec01c450 con 0x7fcffc0c9490 2023-10-06T05:31:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.252+0000 7fcff17fa700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfe0066f30 0x7fcfe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.252+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fcfec086530 con 0x7fcffc0c9490 2023-10-06T05:31:32.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.253+0000 7fd00159b700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfe0066f30 0x7fcfe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.253+0000 7fd00159b700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfe0066f30 0x7fcfe00693e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fcff80011e0 tx=0x7fcff8007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.256+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcffc15d5f0 con 0x7fcffc0c9490 2023-10-06T05:31:32.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.458+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcffc15d5f0 con 0x7fcffc0c9490 2023-10-06T05:31:32.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.459+0000 7fcff17fa700 1 -- 172.21.15.33:0/3505383614 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v228) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcffc15d5f0 con 0x7fcffc0c9490 2023-10-06T05:31:32.461 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfe0066f30 msgr2=0x7fcfe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfe0066f30 0x7fcfe00693e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fcff80011e0 tx=0x7fcff8007c90 comp rx=0 tx=0).stop 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 msgr2=0x7fcffc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc15ecb0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009290 tx=0x7fcfec016000 comp rx=0 tx=0).stop 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 shutdown_connections 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcfe0066f30 0x7fcfe00693e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcffc155a50 0x7fcffc15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcffc0afb20 0x7fcffc141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 --2- 172.21.15.33:0/3505383614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcffc0c9490 0x7fcffc15ecb0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 >> 172.21.15.33:0/3505383614 conn(0x7fcffc06d030 msgr2=0x7fcffc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:32.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 shutdown_connections 2023-10-06T05:31:32.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.461+0000 7fd00259d700 1 -- 172.21.15.33:0/3505383614 wait complete. 2023-10-06T05:31:32.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:31:32.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.667+0000 7f8e4a59d700 1 Processor -- start 2023-10-06T05:31:32.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.667+0000 7f8e4a59d700 1 -- start start 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e4a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e44159760 0x7f8e440aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e4a59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8e44159b30 0x7f8e440aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e4a59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440aacf0 con 0x7f8e440af6a0 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e4a59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440aaf20 con 0x7f8e44159b30 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e4a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440ab200 con 0x7f8e44159760 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e49d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8e440af6a0 0x7f8e440afb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34096/0 (socket says 172.21.15.33:34096) 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e49d9c700 1 -- 172.21.15.33:0/1335315695 learned_addr learned my addr 172.21.15.33:0/1335315695 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e4959b700 1 --2- 172.21.15.33:0/1335315695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e44159760 0x7f8e440aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.669+0000 7f8e48d9a700 1 --2- 172.21.15.33:0/1335315695 >> v2:172.21.15.167:3300/0 conn(0x7f8e44159b30 0x7f8e440aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e3bfff700 1 -- 172.21.15.33:0/1335315695 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032680728 0 0) 0x7f8e440aacf0 con 0x7f8e440af6a0 2023-10-06T05:31:32.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e3bfff700 1 -- 172.21.15.33:0/1335315695 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e30003570 con 0x7f8e440af6a0 2023-10-06T05:31:32.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e4959b700 1 -- 172.21.15.33:0/1335315695 >> v2:172.21.15.167:3300/0 conn(0x7f8e44159b30 msgr2=0x7f8e440aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:32.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e4959b700 1 --2- 172.21.15.33:0/1335315695 >> v2:172.21.15.167:3300/0 conn(0x7f8e44159b30 0x7f8e440aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e4959b700 1 -- 172.21.15.33:0/1335315695 >> v1:172.21.15.33:6789/0 conn(0x7f8e440af6a0 legacy=0x7f8e440afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e4959b700 1 -- 172.21.15.33:0/1335315695 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440ab4e0 con 0x7f8e44159760 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e48d9a700 1 --2- 172.21.15.33:0/1335315695 >> v2:172.21.15.167:3300/0 conn(0x7f8e44159b30 0x7f8e440aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e4959b700 1 --2- 172.21.15.33:0/1335315695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e44159760 0x7f8e440aa270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e400098e0 tx=0x7f8e400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf98deaf3c5a9ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e3bfff700 1 -- 172.21.15.33:0/1335315695 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011630 con 0x7f8e44159760 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e3bfff700 1 -- 172.21.15.33:0/1335315695 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40011790 con 0x7f8e44159760 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.670+0000 7f8e3bfff700 1 -- 172.21.15.33:0/1335315695 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011b10 con 0x7f8e44159760 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 -- 172.21.15.33:0/1335315695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e44159760 msgr2=0x7f8e440aa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/1335315695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e44159760 0x7f8e440aa270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e400098e0 tx=0x7f8e400091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 -- 172.21.15.33:0/1335315695 shutdown_connections 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/1335315695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e44159760 0x7f8e440aa270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/1335315695 >> v2:172.21.15.167:3300/0 conn(0x7f8e44159b30 0x7f8e440aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 -- 172.21.15.33:0/1335315695 >> 172.21.15.33:0/1335315695 conn(0x7f8e440cd000 msgr2=0x7f8e4406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 -- 172.21.15.33:0/1335315695 shutdown_connections 2023-10-06T05:31:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.671+0000 7f8e4a59d700 1 -- 172.21.15.33:0/1335315695 wait complete. 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.672+0000 7f8e4a59d700 1 Processor -- start 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.673+0000 7f8e4a59d700 1 -- start start 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.673+0000 7f8e4a59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e440af6a0 0x7f8e441518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e4a59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e44159760 0x7f8e4414af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e4a59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 0x7f8e44156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e4a59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4414caf0 con 0x7f8e44159b30 2023-10-06T05:31:32.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e4a59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e44151e10 con 0x7f8e44159760 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e4a59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4414cf60 con 0x7f8e440af6a0 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 0x7f8e44156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 0x7f8e44156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58838/0 (socket says 172.21.15.33:58838) 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 -- 172.21.15.33:0/3047124224 learned_addr learned my addr 172.21.15.33:0/3047124224 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e4959b700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e440af6a0 0x7f8e441518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e48d9a700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e44159760 0x7f8e4414af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 -- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e440af6a0 msgr2=0x7f8e441518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e440af6a0 0x7f8e441518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 -- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e44159760 msgr2=0x7f8e4414af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e44159760 0x7f8e4414af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.674+0000 7f8e49d9c700 1 -- 172.21.15.33:0/3047124224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34009420 con 0x7f8e44159b30 2023-10-06T05:31:32.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.675+0000 7f8e49d9c700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 0x7f8e44156d20 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c001460 tx=0x7f8e3c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:32.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.675+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00a870 con 0x7f8e44159b30 2023-10-06T05:31:32.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.675+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00a9d0 con 0x7f8e44159b30 2023-10-06T05:31:32.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.675+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40009420 con 0x7f8e44159b30 2023-10-06T05:31:32.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.675+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00ad50 con 0x7f8e44159b30 2023-10-06T05:31:32.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.675+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4414c810 con 0x7f8e44159b30 2023-10-06T05:31:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.676+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440ae7d0 con 0x7f8e44159b30 2023-10-06T05:31:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.677+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e3c007300 con 0x7f8e44159b30 2023-10-06T05:31:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.678+0000 7f8e397fa700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e28066f30 0x7f8e280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.678+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8e3c084ad0 con 0x7f8e44159b30 2023-10-06T05:31:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.678+0000 7f8e4959b700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e28066f30 0x7f8e280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:32.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.679+0000 7f8e4959b700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e28066f30 0x7f8e280693e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8e400091e0 tx=0x7f8e40017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:32.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.682+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e3c051c10 con 0x7f8e44159b30 2023-10-06T05:31:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.886+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e4404c380 con 0x7f8e44159b30 2023-10-06T05:31:32.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.886+0000 7f8e397fa700 1 -- 172.21.15.33:0/3047124224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v228) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e3c055230 con 0x7f8e44159b30 2023-10-06T05:31:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:31:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e28066f30 msgr2=0x7f8e280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e28066f30 0x7f8e280693e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8e400091e0 tx=0x7f8e40017000 comp rx=0 tx=0).stop 2023-10-06T05:31:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 msgr2=0x7f8e44156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 0x7f8e44156d20 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c001460 tx=0x7f8e3c001a40 comp rx=0 tx=0).stop 2023-10-06T05:31:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 shutdown_connections 2023-10-06T05:31:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8e28066f30 0x7f8e280693e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e44159b30 0x7f8e44156d20 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e440af6a0 0x7f8e441518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 --2- 172.21.15.33:0/3047124224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e44159760 0x7f8e4414af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 >> 172.21.15.33:0/3047124224 conn(0x7f8e440cd000 msgr2=0x7f8e440b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 shutdown_connections 2023-10-06T05:31:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:32.889+0000 7f8e4a59d700 1 -- 172.21.15.33:0/3047124224 wait complete. 2023-10-06T05:31:32.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:31:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.100+0000 7f811759e700 1 Processor -- start 2023-10-06T05:31:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.100+0000 7f811759e700 1 -- start start 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.101+0000 7f811759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f81180cd5e0 0x7f81180af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.101+0000 7f811759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f811815a650 0x7f811815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.101+0000 7f811759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811815d660 con 0x7f81180af540 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.101+0000 7f811759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81180ce890 con 0x7f811815a650 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.101+0000 7f811759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811815aa60 con 0x7f81180cd5e0 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.101+0000 7f8116d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f811815a650 0x7f811815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f811815a650 0x7f811815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58854/0 (socket says 172.21.15.33:58854) 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f811659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81180af540 0x7f81180cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34110/0 (socket says 172.21.15.33:34110) 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 -- 172.21.15.33:0/183579378 learned_addr learned my addr 172.21.15.33:0/183579378 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8115d9b700 1 --2- 172.21.15.33:0/183579378 >> v2:172.21.15.167:3300/0 conn(0x7f81180cd5e0 0x7f81180af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 -- 172.21.15.33:0/183579378 >> v2:172.21.15.167:3300/0 conn(0x7f81180cd5e0 msgr2=0x7f81180af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 --2- 172.21.15.33:0/183579378 >> v2:172.21.15.167:3300/0 conn(0x7f81180cd5e0 0x7f81180af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 -- 172.21.15.33:0/183579378 >> v1:172.21.15.33:6789/0 conn(0x7f81180af540 legacy=0x7f81180cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 -- 172.21.15.33:0/183579378 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811815d890 con 0x7f811815a650 2023-10-06T05:31:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.102+0000 7f8116d9d700 1 --2- 172.21.15.33:0/183579378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f811815d120 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f81180893f0 tx=0x7f8110009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5be8a0d89fbb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811559a700 1 -- 172.21.15.33:0/183579378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811000e070 con 0x7f811815a650 2023-10-06T05:31:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811559a700 1 -- 172.21.15.33:0/183579378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811000ba50 con 0x7f811815a650 2023-10-06T05:31:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811559a700 1 -- 172.21.15.33:0/183579378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100092e0 con 0x7f811815a650 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 -- 172.21.15.33:0/183579378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 msgr2=0x7f811815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 --2- 172.21.15.33:0/183579378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f811815d120 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f81180893f0 tx=0x7f8110009840 comp rx=0 tx=0).stop 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 -- 172.21.15.33:0/183579378 shutdown_connections 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 --2- 172.21.15.33:0/183579378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f811815d120 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 --2- 172.21.15.33:0/183579378 >> v2:172.21.15.167:3300/0 conn(0x7f81180cd5e0 0x7f81180af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 -- 172.21.15.33:0/183579378 >> 172.21.15.33:0/183579378 conn(0x7f811806d9b0 msgr2=0x7f811806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 -- 172.21.15.33:0/183579378 shutdown_connections 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.103+0000 7f811759e700 1 -- 172.21.15.33:0/183579378 wait complete. 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.105+0000 7f811759e700 1 Processor -- start 2023-10-06T05:31:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.105+0000 7f811759e700 1 -- start start 2023-10-06T05:31:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81180af540 0x7f81180b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81180cd5e0 0x7f811814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f81180ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811816a790 con 0x7f811815a650 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811814c580 con 0x7f81180af540 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81180ca6c0 con 0x7f81180cd5e0 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f81180ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f81180ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58864/0 (socket says 172.21.15.33:58864) 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 -- 172.21.15.33:0/3282260960 learned_addr learned my addr 172.21.15.33:0/3282260960 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f811659c700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81180af540 0x7f81180b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8115d9b700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81180cd5e0 0x7f811814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 -- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81180cd5e0 msgr2=0x7f811814c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81180cd5e0 0x7f811814c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 -- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81180af540 msgr2=0x7f81180b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81180af540 0x7f81180b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.106+0000 7f8116d9d700 1 -- 172.21.15.33:0/3282260960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009ae0 con 0x7f811815a650 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.107+0000 7f8116d9d700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f81180ca180 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f81180c7320 tx=0x7f8110018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.107+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110008ce0 con 0x7f811815a650 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.107+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8110008e60 con 0x7f811815a650 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.107+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100266b0 con 0x7f811815a650 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.107+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8118169b80 con 0x7f811815a650 2023-10-06T05:31:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.107+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811816a140 con 0x7f811815a650 2023-10-06T05:31:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.108+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811804c380 con 0x7f811815a650 2023-10-06T05:31:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.109+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8110003040 con 0x7f811815a650 2023-10-06T05:31:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.110+0000 7f81067fc700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.110+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f81100866b0 con 0x7f811815a650 2023-10-06T05:31:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.111+0000 7f811659c700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.111+0000 7f811659c700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8108004fd0 tx=0x7f8108007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.114+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81100536c0 con 0x7f811815a650 2023-10-06T05:31:33.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.314+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f811815db70 con 0x7f811815a650 2023-10-06T05:31:33.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.315+0000 7f81067fc700 1 -- 172.21.15.33:0/3282260960 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v228) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8110056ce0 con 0x7f811815a650 2023-10-06T05:31:33.318 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 msgr2=0x7f80f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8108004fd0 tx=0x7f8108007eb0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 msgr2=0x7f81180ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f81180ca180 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f81180c7320 tx=0x7f8110018000 comp rx=0 tx=0).stop 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 shutdown_connections 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f80f8066f30 0x7f80f80693e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f811815a650 0x7f81180ca180 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81180af540 0x7f81180b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 --2- 172.21.15.33:0/3282260960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81180cd5e0 0x7f811814c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 >> 172.21.15.33:0/3282260960 conn(0x7f811806d9b0 msgr2=0x7f81180c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:33.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 shutdown_connections 2023-10-06T05:31:33.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.317+0000 7f811759e700 1 -- 172.21.15.33:0/3282260960 wait complete. 2023-10-06T05:31:33.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:31:33.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.525+0000 7f846b59e700 1 Processor -- start 2023-10-06T05:31:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.525+0000 7f846b59e700 1 -- start start 2023-10-06T05:31:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.526+0000 7f846b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 0x7f846c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.526+0000 7f846b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c0ce890 con 0x7f846c0af540 2023-10-06T05:31:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.526+0000 7f846b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c15d890 con 0x7f846c0cd5e0 2023-10-06T05:31:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.526+0000 7f846b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0afd50 con 0x7f846c0af910 2023-10-06T05:31:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.526+0000 7f846a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f846c0af540 0x7f846c0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34112/0 (socket says 172.21.15.33:34112) 2023-10-06T05:31:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.526+0000 7f846a59c700 1 -- 172.21.15.33:0/2671071222 learned_addr learned my addr 172.21.15.33:0/2671071222 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846ad9d700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 0x7f846c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330592301 0 0) 0x7f846c0ce890 con 0x7f846c0af540 2023-10-06T05:31:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454003570 con 0x7f846c0af540 2023-10-06T05:31:33.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470449986 0 0) 0x7f8454003570 con 0x7f846c0af540 2023-10-06T05:31:33.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 msgr2=0x7f846c15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 0x7f846c15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 >> v1:172.21.15.167:6789/0 conn(0x7f846c0cd5e0 legacy=0x7f846c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c15db70 con 0x7f846c0af540 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f845c001470 con 0x7f846c0af540 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 >> v1:172.21.15.33:6789/0 conn(0x7f846c0af540 legacy=0x7f846c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 0x7f8454003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8454003fd0 0x7f8454004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f84540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0afd50 con 0x7f8454004d50 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845400bb40 con 0x7f8454003fd0 2023-10-06T05:31:33.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.527+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845400bd00 con 0x7f846c0af910 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f84540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846a59c700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 0x7f8454003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f8469d9b700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8454003fd0 0x7f8454004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 -- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 msgr2=0x7f8454003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0af910 0x7f8454003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 -- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8454003fd0 msgr2=0x7f8454004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8454003fd0 0x7f8454004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 -- 172.21.15.33:0/2671071222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454003570 con 0x7f8454004d50 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.528+0000 7f846ad9d700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f84540051c0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f846400e210 tx=0x7f846400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd2969755f066c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400eac0 con 0x7f8454004d50 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846400f040 con 0x7f8454004d50 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846959a700 1 -- 172.21.15.33:0/2671071222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400ee70 con 0x7f8454004d50 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846b59e700 1 -- 172.21.15.33:0/2671071222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 msgr2=0x7f84540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846b59e700 1 --2- 172.21.15.33:0/2671071222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f84540051c0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f846400e210 tx=0x7f846400b040 comp rx=0 tx=0).stop 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846ad9d700 1 -- 172.21.15.33:0/2671071222 reap_dead start 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846b59e700 1 -- 172.21.15.33:0/2671071222 shutdown_connections 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846b59e700 1 -- 172.21.15.33:0/2671071222 >> 172.21.15.33:0/2671071222 conn(0x7f846c06d9b0 msgr2=0x7f846c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846b59e700 1 -- 172.21.15.33:0/2671071222 shutdown_connections 2023-10-06T05:31:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.529+0000 7f846b59e700 1 -- 172.21.15.33:0/2671071222 wait complete. 2023-10-06T05:31:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.531+0000 7f846b59e700 1 Processor -- start 2023-10-06T05:31:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.531+0000 7f846b59e700 1 -- start start 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f846c0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c14dac0 0x7f846c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c15b090 0x7f846c15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c16cf80 con 0x7f8454004d50 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0b47b0 con 0x7f846c14dac0 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0b4a90 con 0x7f846c15b090 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f846c0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f846c0b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58890/0 (socket says 172.21.15.33:58890) 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 -- 172.21.15.33:0/2133594984 learned_addr learned my addr 172.21.15.33:0/2133594984 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f8469d9b700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c14dac0 0x7f846c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846ad9d700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c15b090 0x7f846c15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 -- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c15b090 msgr2=0x7f846c15b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c15b090 0x7f846c15b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 -- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c14dac0 msgr2=0x7f846c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c14dac0 0x7f846c0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.532+0000 7f846a59c700 1 -- 172.21.15.33:0/2133594984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c16d1d0 con 0x7f8454004d50 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.533+0000 7f846a59c700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f846c0b1380 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f845c0073c0 tx=0x7f845c001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.533+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c018030 con 0x7f8454004d50 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.533+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c007830 con 0x7f8454004d50 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.533+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c16d4b0 con 0x7f8454004d50 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.533+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00fb20 con 0x7f8454004d50 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.533+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c15bc30 con 0x7f8454004d50 2023-10-06T05:31:33.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.534+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c16d6c0 con 0x7f8454004d50 2023-10-06T05:31:33.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.535+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f845c01b070 con 0x7f8454004d50 2023-10-06T05:31:33.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.536+0000 7f845a7fc700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f844c066fb0 0x7f844c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.536+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f845c019070 con 0x7f8454004d50 2023-10-06T05:31:33.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.537+0000 7f8469d9b700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f844c066fb0 0x7f844c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.537+0000 7f8469d9b700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f844c066fb0 0x7f844c069460 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f846c159170 tx=0x7f8460009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.540+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f845c0516f0 con 0x7f8454004d50 2023-10-06T05:31:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.743+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f846c04c380 con 0x7f8454004d50 2023-10-06T05:31:33.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.743+0000 7f845a7fc700 1 -- 172.21.15.33:0/2133594984 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v228) v1 ==== 100+0+11 (secure 0 0 0) 0x7f845c054d10 con 0x7f8454004d50 2023-10-06T05:31:33.746 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f844c066fb0 msgr2=0x7f844c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f844c066fb0 0x7f844c069460 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f846c159170 tx=0x7f8460009040 comp rx=0 tx=0).stop 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 msgr2=0x7f846c0b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f846c0b1380 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f845c0073c0 tx=0x7f845c001df0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 shutdown_connections 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f844c066fb0 0x7f844c069460 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c15b090 0x7f846c15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454004d50 0x7f846c0b1380 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 --2- 172.21.15.33:0/2133594984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c14dac0 0x7f846c0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 >> 172.21.15.33:0/2133594984 conn(0x7f846c06d9b0 msgr2=0x7f846c157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 shutdown_connections 2023-10-06T05:31:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.746+0000 7f846b59e700 1 -- 172.21.15.33:0/2133594984 wait complete. 2023-10-06T05:31:33.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:33.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.952+0000 7fc81a39d700 1 Processor -- start 2023-10-06T05:31:33.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.952+0000 7fc81a39d700 1 -- start start 2023-10-06T05:31:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.953+0000 7fc81a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.953+0000 7fc81a39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc81406aad0 0x7fc81406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.953+0000 7fc81a39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8140c9cb0 con 0x7fc8140c7940 2023-10-06T05:31:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.953+0000 7fc81a39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8140ca270 con 0x7fc81406aad0 2023-10-06T05:31:33.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.953+0000 7fc81a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81406b840 con 0x7fc81413faa0 2023-10-06T05:31:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc818b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc8140c7940 0x7fc8140c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34128/0 (socket says 172.21.15.33:34128) 2023-10-06T05:31:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc818b9a700 1 -- 172.21.15.33:0/128852033 learned_addr learned my addr 172.21.15.33:0/128852033 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc81939b700 1 --2- 172.21.15.33:0/128852033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc819b9c700 1 --2- 172.21.15.33:0/128852033 >> v2:172.21.15.167:3300/0 conn(0x7fc81406aad0 0x7fc81406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc80bfff700 1 -- 172.21.15.33:0/128852033 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875796511 0 0) 0x7fc8140c9cb0 con 0x7fc8140c7940 2023-10-06T05:31:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc80bfff700 1 -- 172.21.15.33:0/128852033 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc800003570 con 0x7fc8140c7940 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc81939b700 1 -- 172.21.15.33:0/128852033 >> v2:172.21.15.167:3300/0 conn(0x7fc81406aad0 msgr2=0x7fc81406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc81939b700 1 --2- 172.21.15.33:0/128852033 >> v2:172.21.15.167:3300/0 conn(0x7fc81406aad0 0x7fc81406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc81939b700 1 -- 172.21.15.33:0/128852033 >> v1:172.21.15.33:6789/0 conn(0x7fc8140c7940 legacy=0x7fc8140c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.954+0000 7fc81939b700 1 -- 172.21.15.33:0/128852033 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81406bb20 con 0x7fc81413faa0 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.955+0000 7fc81939b700 1 --2- 172.21.15.33:0/128852033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81413fe70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8100098e0 tx=0x7fc8100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30dc2832c8366d29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.955+0000 7fc80bfff700 1 -- 172.21.15.33:0/128852033 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810011630 con 0x7fc81413faa0 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.955+0000 7fc80bfff700 1 -- 172.21.15.33:0/128852033 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc810011790 con 0x7fc81413faa0 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.955+0000 7fc80bfff700 1 -- 172.21.15.33:0/128852033 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810011b30 con 0x7fc81413faa0 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.955+0000 7fc81a39d700 1 -- 172.21.15.33:0/128852033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 msgr2=0x7fc81413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.955+0000 7fc81a39d700 1 --2- 172.21.15.33:0/128852033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81413fe70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8100098e0 tx=0x7fc8100091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.956+0000 7fc81a39d700 1 -- 172.21.15.33:0/128852033 shutdown_connections 2023-10-06T05:31:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.956+0000 7fc81a39d700 1 --2- 172.21.15.33:0/128852033 >> v2:172.21.15.167:3300/0 conn(0x7fc81406aad0 0x7fc81406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.956+0000 7fc81a39d700 1 --2- 172.21.15.33:0/128852033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81413fe70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.956+0000 7fc81a39d700 1 -- 172.21.15.33:0/128852033 >> 172.21.15.33:0/128852033 conn(0x7fc81406d030 msgr2=0x7fc81406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.956+0000 7fc81a39d700 1 -- 172.21.15.33:0/128852033 shutdown_connections 2023-10-06T05:31:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.956+0000 7fc81a39d700 1 -- 172.21.15.33:0/128852033 wait complete. 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.957+0000 7fc81a39d700 1 Processor -- start 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.957+0000 7fc81a39d700 1 -- start start 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 0x7fc8141577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8140c7940 0x7fc814155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814158300 con 0x7fc81406aad0 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140c3d70 con 0x7fc8140c7940 2023-10-06T05:31:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc814158020 con 0x7fc81413faa0 2023-10-06T05:31:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 0x7fc8141577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 0x7fc8141577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58894/0 (socket says 172.21.15.33:58894) 2023-10-06T05:31:33.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.958+0000 7fc81939b700 1 -- 172.21.15.33:0/744859159 learned_addr learned my addr 172.21.15.33:0/744859159 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc818b9a700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8140c7940 0x7fc814155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81939b700 1 -- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 msgr2=0x7fc81415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc819b9c700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81939b700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81939b700 1 -- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8140c7940 msgr2=0x7fc814155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81939b700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8140c7940 0x7fc814155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81939b700 1 -- 172.21.15.33:0/744859159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810009420 con 0x7fc81406aad0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc818b9a700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8140c7940 0x7fc814155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc819b9c700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81939b700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 0x7fc8141577f0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fc81406b980 tx=0x7fc810017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810011900 con 0x7fc81406aad0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc810011aa0 con 0x7fc81406aad0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810025670 con 0x7fc81406aad0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.959+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8140c3720 con 0x7fc81406aad0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.960+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc814157d30 con 0x7fc81406aad0 2023-10-06T05:31:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.961+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81404c380 con 0x7fc81406aad0 2023-10-06T05:31:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.961+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc810023070 con 0x7fc81406aad0 2023-10-06T05:31:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.963+0000 7fc8097fa700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7f8066f30 0x7fc7f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.963+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc810085b30 con 0x7fc81406aad0 2023-10-06T05:31:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.963+0000 7fc818b9a700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7f8066f30 0x7fc7f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.964+0000 7fc818b9a700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7f8066f30 0x7fc7f80693e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc804007fd0 tx=0x7fc804009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:33.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:33.968+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc810052bf0 con 0x7fc81406aad0 2023-10-06T05:31:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.212+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8140c9f00 con 0x7fc81406aad0 2023-10-06T05:31:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.213+0000 7fc8097fa700 1 -- 172.21.15.33:0/744859159 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7fc81002a020 con 0x7fc81406aad0 2023-10-06T05:31:34.216 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:34.216 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7f8066f30 msgr2=0x7fc7f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7f8066f30 0x7fc7f80693e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc804007fd0 tx=0x7fc804009040 comp rx=0 tx=0).stop 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 msgr2=0x7fc8141577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 0x7fc8141577f0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fc81406b980 tx=0x7fc810017000 comp rx=0 tx=0).stop 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 shutdown_connections 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7f8066f30 0x7fc7f80693e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc81413faa0 0x7fc81415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc81406aad0 0x7fc8141577f0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 --2- 172.21.15.33:0/744859159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8140c7940 0x7fc814155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 >> 172.21.15.33:0/744859159 conn(0x7fc81406d030 msgr2=0x7fc8140ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 shutdown_connections 2023-10-06T05:31:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.215+0000 7fc81a39d700 1 -- 172.21.15.33:0/744859159 wait complete. 2023-10-06T05:31:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:31:34.236 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:34.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.420+0000 7fc67a79d700 1 Processor -- start 2023-10-06T05:31:34.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.420+0000 7fc67a79d700 1 -- start start 2023-10-06T05:31:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc67a79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67413faa0 0x7fc67413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc67a79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 0x7fc67406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc67a79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6740c9cb0 con 0x7fc6740c7940 2023-10-06T05:31:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc67a79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6740ca270 con 0x7fc67406aad0 2023-10-06T05:31:34.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc67a79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67406b840 con 0x7fc67413faa0 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc679f9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 0x7fc67406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc67979b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67413faa0 0x7fc67413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc679f9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 0x7fc67406af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:60564/0 (socket says 172.21.15.33:60564) 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.422+0000 7fc679f9c700 1 -- 172.21.15.33:0/2768164072 learned_addr learned my addr 172.21.15.33:0/2768164072 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc66bfff700 1 -- 172.21.15.33:0/2768164072 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295852989 0 0) 0x7fc6740c9cb0 con 0x7fc6740c7940 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc67979b700 1 -- 172.21.15.33:0/2768164072 >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 msgr2=0x7fc67406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc67979b700 1 --2- 172.21.15.33:0/2768164072 >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 0x7fc67406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc67979b700 1 -- 172.21.15.33:0/2768164072 >> v1:172.21.15.167:6789/0 conn(0x7fc6740c7940 legacy=0x7fc6740c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc67979b700 1 -- 172.21.15.33:0/2768164072 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67406bb20 con 0x7fc67413faa0 2023-10-06T05:31:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc679f9c700 1 --2- 172.21.15.33:0/2768164072 >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 0x7fc67406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.423+0000 7fc67979b700 1 --2- 172.21.15.33:0/2768164072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67413faa0 0x7fc67413fe70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc67000bb90 tx=0x7fc67000b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d28df5a1824533 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc66bfff700 1 -- 172.21.15.33:0/2768164072 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000db40 con 0x7fc67413faa0 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc66bfff700 1 -- 172.21.15.33:0/2768164072 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67000dca0 con 0x7fc67413faa0 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc66bfff700 1 -- 172.21.15.33:0/2768164072 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6700196b0 con 0x7fc67413faa0 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 -- 172.21.15.33:0/2768164072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67413faa0 msgr2=0x7fc67413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 --2- 172.21.15.33:0/2768164072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67413faa0 0x7fc67413fe70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc67000bb90 tx=0x7fc67000b430 comp rx=0 tx=0).stop 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 -- 172.21.15.33:0/2768164072 shutdown_connections 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 --2- 172.21.15.33:0/2768164072 >> v2:172.21.15.167:3300/0 conn(0x7fc67406aad0 0x7fc67406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 --2- 172.21.15.33:0/2768164072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67413faa0 0x7fc67413fe70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 -- 172.21.15.33:0/2768164072 >> 172.21.15.33:0/2768164072 conn(0x7fc67406d030 msgr2=0x7fc67406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 -- 172.21.15.33:0/2768164072 shutdown_connections 2023-10-06T05:31:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.424+0000 7fc67a79d700 1 -- 172.21.15.33:0/2768164072 wait complete. 2023-10-06T05:31:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.426+0000 7fc67a79d700 1 Processor -- start 2023-10-06T05:31:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.426+0000 7fc67a79d700 1 -- start start 2023-10-06T05:31:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67a79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67406aad0 0x7fc674158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67a79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 0x7fc6740c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67a79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc67413faa0 0x7fc6740ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67a79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c65c0 con 0x7fc6740c7940 2023-10-06T05:31:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67a79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c67b0 con 0x7fc67413faa0 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67a79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c4300 con 0x7fc67406aad0 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc678f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 0x7fc6740c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc678f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 0x7fc6740c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58908/0 (socket says 172.21.15.33:58908) 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc678f9a700 1 -- 172.21.15.33:0/3199436821 learned_addr learned my addr 172.21.15.33:0/3199436821 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc67979b700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67406aad0 0x7fc674158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.427+0000 7fc679f9c700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc67413faa0 0x7fc6740ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc678f9a700 1 -- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67406aad0 msgr2=0x7fc674158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc678f9a700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67406aad0 0x7fc674158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc678f9a700 1 -- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc67413faa0 msgr2=0x7fc6740ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc678f9a700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc67413faa0 0x7fc6740ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc678f9a700 1 -- 172.21.15.33:0/3199436821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c009ae0 con 0x7fc6740c7940 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc678f9a700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 0x7fc6740c6080 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fc66400c810 tx=0x7fc664007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664011590 con 0x7fc6740c7940 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc664011710 con 0x7fc6740c7940 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664011ad0 con 0x7fc6740c7940 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67000b6d0 con 0x7fc6740c7940 2023-10-06T05:31:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.428+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6740c4880 con 0x7fc6740c7940 2023-10-06T05:31:34.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.429+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67404c380 con 0x7fc6740c7940 2023-10-06T05:31:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.430+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc66401c460 con 0x7fc6740c7940 2023-10-06T05:31:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.431+0000 7fc6697fa700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc658066f30 0x7fc6580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.431+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc6640844c0 con 0x7fc6740c7940 2023-10-06T05:31:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.432+0000 7fc67979b700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc658066f30 0x7fc6580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.432+0000 7fc67979b700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc658066f30 0x7fc6580693e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b6a0 tx=0x7fc67001b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:34.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.436+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6640514d0 con 0x7fc6740c7940 2023-10-06T05:31:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.642+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc67406fc10 con 0x7fc6740c7940 2023-10-06T05:31:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.643+0000 7fc6697fa700 1 -- 172.21.15.33:0/3199436821 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v228) v1 ==== 74+0+17025 (secure 0 0 0) 0x7fc664054af0 con 0x7fc6740c7940 2023-10-06T05:31:34.646 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:34.646 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":228,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:30:09.896755+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:50.569789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:51.932764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.2","mappings":[{"from":0,"to":2}]},{"pgid":"34.20","mappings":[{"from":0,"to":1}]},{"pgid":"34.29","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc658066f30 msgr2=0x7fc6580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc658066f30 0x7fc6580693e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b6a0 tx=0x7fc67001b000 comp rx=0 tx=0).stop 2023-10-06T05:31:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 msgr2=0x7fc6740c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 0x7fc6740c6080 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fc66400c810 tx=0x7fc664007df0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 shutdown_connections 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc658066f30 0x7fc6580693e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc67413faa0 0x7fc6740ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc67406aad0 0x7fc674158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 --2- 172.21.15.33:0/3199436821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6740c7940 0x7fc6740c6080 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 >> 172.21.15.33:0/3199436821 conn(0x7fc67406d030 msgr2=0x7fc6740b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 shutdown_connections 2023-10-06T05:31:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.645+0000 7fc67a79d700 1 -- 172.21.15.33:0/3199436821 wait complete. 2023-10-06T05:31:34.666 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:31:34.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:34.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.849+0000 7f55a939d700 1 Processor -- start 2023-10-06T05:31:34.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.849+0000 7f55a939d700 1 -- start start 2023-10-06T05:31:34.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.850+0000 7f55a939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a4158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.850+0000 7f55a939d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55a40adf50 0x7f55a40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.850+0000 7f55a939d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a40c8e40 con 0x7f55a40c9810 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.850+0000 7f55a939d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a40ae6c0 con 0x7f55a40adf50 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.850+0000 7f55a939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40ae9a0 con 0x7f55a4155140 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55a40adf50 0x7f55a40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55a40adf50 0x7f55a40c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58918/0 (socket says 172.21.15.33:58918) 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 -- 172.21.15.33:0/2734848306 learned_addr learned my addr 172.21.15.33:0/2734848306 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a3fff700 1 --2- 172.21.15.33:0/2734848306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 -- 172.21.15.33:0/2734848306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 msgr2=0x7f55a4158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:34.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 --2- 172.21.15.33:0/2734848306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a4158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 -- 172.21.15.33:0/2734848306 >> v1:172.21.15.167:6789/0 conn(0x7f55a40c9810 legacy=0x7f55a40699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 -- 172.21.15.33:0/2734848306 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a40c8ad0 con 0x7f55a40adf50 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.851+0000 7f55a37fe700 1 --2- 172.21.15.33:0/2734848306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40adf50 0x7f55a40c92d0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f55980099b0 tx=0x7f5598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e685d5611ef53f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a2ffd700 1 -- 172.21.15.33:0/2734848306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800b860 con 0x7f55a40adf50 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a2ffd700 1 -- 172.21.15.33:0/2734848306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800b9c0 con 0x7f55a40adf50 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a2ffd700 1 -- 172.21.15.33:0/2734848306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800bd60 con 0x7f55a40adf50 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 -- 172.21.15.33:0/2734848306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40adf50 msgr2=0x7f55a40c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 --2- 172.21.15.33:0/2734848306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40adf50 0x7f55a40c92d0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f55980099b0 tx=0x7f5598009250 comp rx=0 tx=0).stop 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 -- 172.21.15.33:0/2734848306 shutdown_connections 2023-10-06T05:31:34.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 --2- 172.21.15.33:0/2734848306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a4158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 --2- 172.21.15.33:0/2734848306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40adf50 0x7f55a40c92d0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 -- 172.21.15.33:0/2734848306 >> 172.21.15.33:0/2734848306 conn(0x7f55a406d030 msgr2=0x7f55a406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 -- 172.21.15.33:0/2734848306 shutdown_connections 2023-10-06T05:31:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.852+0000 7f55a939d700 1 -- 172.21.15.33:0/2734848306 wait complete. 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.854+0000 7f55a939d700 1 Processor -- start 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.854+0000 7f55a939d700 1 -- start start 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a939d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a40adf50 0x7f55a414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a939d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 0x7f55a40c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a939d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a40ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a939d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4152a10 con 0x7f55a40c9810 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a939d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41532b0 con 0x7f55a40adf50 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a939d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40c5f90 con 0x7f55a4155140 2023-10-06T05:31:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 0x7f55a40c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 0x7f55a40c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58928/0 (socket says 172.21.15.33:58928) 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a37fe700 1 -- 172.21.15.33:0/3980613498 learned_addr learned my addr 172.21.15.33:0/3980613498 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.855+0000 7f55a3fff700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a40adf50 0x7f55a414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a8b9c700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a40ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a37fe700 1 -- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 msgr2=0x7f55a40ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a37fe700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a40ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a37fe700 1 -- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a40adf50 msgr2=0x7f55a414d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a37fe700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a40adf50 0x7f55a414d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a37fe700 1 -- 172.21.15.33:0/3980613498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55980094f0 con 0x7f55a40c9810 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a3fff700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a40adf50 0x7f55a414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a8b9c700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a40ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a37fe700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 0x7f55a40c5620 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598024440 con 0x7f55a40c9810 2023-10-06T05:31:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55980245c0 con 0x7f55a40c9810 2023-10-06T05:31:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598024980 con 0x7f55a40c9810 2023-10-06T05:31:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a4152c60 con 0x7f55a40c9810 2023-10-06T05:31:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.856+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a414d9e0 con 0x7f55a40c9810 2023-10-06T05:31:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.857+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a404c380 con 0x7f55a40c9810 2023-10-06T05:31:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.858+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f559801a420 con 0x7f55a40c9810 2023-10-06T05:31:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.859+0000 7f558bfff700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5590066f60 0x7f5590069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.859+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5598086520 con 0x7f55a40c9810 2023-10-06T05:31:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.859+0000 7f55a3fff700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5590066f60 0x7f5590069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.860+0000 7f55a3fff700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5590066f60 0x7f5590069410 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5594006fd0 tx=0x7f5594008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:34.863+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5598053530 con 0x7f55a40c9810 2023-10-06T05:31:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.112+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55a41533f0 con 0x7f55a40c9810 2023-10-06T05:31:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.113+0000 7f558bfff700 1 -- 172.21.15.33:0/3980613498 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f5598056b50 con 0x7f55a40c9810 2023-10-06T05:31:35.116 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:35.116 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:31:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5590066f60 msgr2=0x7f5590069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5590066f60 0x7f5590069410 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5594006fd0 tx=0x7f5594008040 comp rx=0 tx=0).stop 2023-10-06T05:31:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 msgr2=0x7f55a40c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 0x7f55a40c5620 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598017000 comp rx=0 tx=0).stop 2023-10-06T05:31:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 shutdown_connections 2023-10-06T05:31:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5590066f60 0x7f5590069410 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55a4155140 0x7f55a40ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55a40adf50 0x7f55a414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 --2- 172.21.15.33:0/3980613498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55a40c9810 0x7f55a40c5620 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 >> 172.21.15.33:0/3980613498 conn(0x7f55a406d030 msgr2=0x7f55a40ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 shutdown_connections 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.115+0000 7f55a939d700 1 -- 172.21.15.33:0/3980613498 wait complete. 2023-10-06T05:31:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:31:35.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:35.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.321+0000 7f942d5ff700 1 Processor -- start 2023-10-06T05:31:35.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.321+0000 7f942d5ff700 1 -- start start 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f942d5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f942813fb70 0x7f942806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f942d5ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f942806ba40 0x7f94280c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f942d5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94280ca350 0x7f94280af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f942d5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94280ca8d0 con 0x7f942813fb70 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f942d5ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94280cabb0 con 0x7f942806ba40 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f942d5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942806ad00 con 0x7f94280ca350 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f9426ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f942813fb70 0x7f942806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f9426ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f942813fb70 0x7f942806b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58944/0 (socket says 172.21.15.33:58944) 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f9426ffd700 1 -- 172.21.15.33:0/184290191 learned_addr learned my addr 172.21.15.33:0/184290191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f94267fc700 1 --2- 172.21.15.33:0/184290191 >> v2:172.21.15.167:3300/0 conn(0x7f942806ba40 0x7f94280c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.323+0000 7f94277fe700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94280ca350 0x7f94280af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9426ffd700 1 -- 172.21.15.33:0/184290191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94280ca350 msgr2=0x7f94280af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9426ffd700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94280ca350 0x7f94280af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9426ffd700 1 -- 172.21.15.33:0/184290191 >> v2:172.21.15.167:3300/0 conn(0x7f942806ba40 msgr2=0x7f94280c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9426ffd700 1 --2- 172.21.15.33:0/184290191 >> v2:172.21.15.167:3300/0 conn(0x7f942806ba40 0x7f94280c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9426ffd700 1 -- 172.21.15.33:0/184290191 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942806afe0 con 0x7f942813fb70 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f94277fe700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94280ca350 0x7f94280af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9426ffd700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f942806b500 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f94280c6810 tx=0x7f9418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cdec32d76673ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9425ffb700 1 -- 172.21.15.33:0/184290191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418009e70 con 0x7f942813fb70 2023-10-06T05:31:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9425ffb700 1 -- 172.21.15.33:0/184290191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941800b860 con 0x7f942813fb70 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.324+0000 7f9425ffb700 1 -- 172.21.15.33:0/184290191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941800bc00 con 0x7f942813fb70 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 -- 172.21.15.33:0/184290191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 msgr2=0x7f942806b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f942806b500 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f94280c6810 tx=0x7f9418009250 comp rx=0 tx=0).stop 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 -- 172.21.15.33:0/184290191 shutdown_connections 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94280ca350 0x7f94280af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 --2- 172.21.15.33:0/184290191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f942806b500 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 --2- 172.21.15.33:0/184290191 >> v2:172.21.15.167:3300/0 conn(0x7f942806ba40 0x7f94280c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 -- 172.21.15.33:0/184290191 >> 172.21.15.33:0/184290191 conn(0x7f94280675f0 msgr2=0x7f94280679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 -- 172.21.15.33:0/184290191 shutdown_connections 2023-10-06T05:31:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.325+0000 7f942d5ff700 1 -- 172.21.15.33:0/184290191 wait complete. 2023-10-06T05:31:35.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.326+0000 7f942d5ff700 1 Processor -- start 2023-10-06T05:31:35.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.327+0000 7f942d5ff700 1 -- start start 2023-10-06T05:31:35.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f942d5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942806ba40 0x7f942813df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f942d5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94280ca350 0x7f94280aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f942d5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f94280b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f942d5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94280c4080 con 0x7f942813fb70 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f942d5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942813e460 con 0x7f94280ca350 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f942d5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94280c3470 con 0x7f942806ba40 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f94280b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f94280b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58956/0 (socket says 172.21.15.33:58956) 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 -- 172.21.15.33:0/3370399785 learned_addr learned my addr 172.21.15.33:0/3370399785 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94267fc700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94280ca350 0x7f94280aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f9426ffd700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942806ba40 0x7f942813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 -- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942806ba40 msgr2=0x7f942813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942806ba40 0x7f942813df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 -- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94280ca350 msgr2=0x7f94280aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94280ca350 0x7f94280aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.328+0000 7f94277fe700 1 -- 172.21.15.33:0/3370399785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180094f0 con 0x7f942813fb70 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.329+0000 7f94277fe700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f94280b1130 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f942000d1b0 tx=0x7f9420009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.329+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000de50 con 0x7f942813fb70 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.329+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942000f040 con 0x7f942813fb70 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.329+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94280c3750 con 0x7f942813fb70 2023-10-06T05:31:35.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.329+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420015690 con 0x7f942813fb70 2023-10-06T05:31:35.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.329+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94280c3d10 con 0x7f942813fb70 2023-10-06T05:31:35.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.330+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94280c4230 con 0x7f942813fb70 2023-10-06T05:31:35.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.331+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f942001e3b0 con 0x7f942813fb70 2023-10-06T05:31:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.332+0000 7f940effd700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9414066f30 0x7f94140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.332+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f9420084da0 con 0x7f942813fb70 2023-10-06T05:31:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.332+0000 7f9426ffd700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9414066f30 0x7f94140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.333+0000 7f9426ffd700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9414066f30 0x7f94140693e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f94280c6810 tx=0x7f9418017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:35.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.336+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94280c4230 con 0x7f942813fb70 2023-10-06T05:31:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.587+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94280c4230 con 0x7f942813fb70 2023-10-06T05:31:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.588+0000 7f940effd700 1 -- 172.21.15.33:0/3370399785 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f94280c4230 con 0x7f942813fb70 2023-10-06T05:31:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:35.591 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":176,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:29:52.952828+0000","modified":"2023-10-06T05:29:56.481932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10824":{"gid":10824,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3371957868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3371957868},{"type":"v1","addr":"172.21.15.33:6835","nonce":3371957868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T05:31:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9414066f30 msgr2=0x7f94140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9414066f30 0x7f94140693e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f94280c6810 tx=0x7f9418017000 comp rx=0 tx=0).stop 2023-10-06T05:31:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 msgr2=0x7f94280b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f94280b1130 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f942000d1b0 tx=0x7f9420009a50 comp rx=0 tx=0).stop 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 shutdown_connections 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9414066f30 0x7f94140693e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942813fb70 0x7f94280b1130 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942806ba40 0x7f942813df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 --2- 172.21.15.33:0/3370399785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94280ca350 0x7f94280aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 >> 172.21.15.33:0/3370399785 conn(0x7f94280675f0 msgr2=0x7f94280c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.590+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 shutdown_connections 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.591+0000 7f942d5ff700 1 -- 172.21.15.33:0/3370399785 wait complete. 2023-10-06T05:31:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 176 2023-10-06T05:31:35.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.796+0000 7fd85b39d700 1 Processor -- start 2023-10-06T05:31:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.796+0000 7fd85b39d700 1 -- start start 2023-10-06T05:31:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.797+0000 7fd85b39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd85406e140 0x7fd85415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.797+0000 7fd85b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85406f220 con 0x7fd854159530 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.797+0000 7fd85b39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85415ae50 con 0x7fd8540afdd0 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.797+0000 7fd85b39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8540b1e00 con 0x7fd85406e140 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.797+0000 7fd859b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd85406e140 0x7fd85415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd85a39b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd854159530 0x7fd85406da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34136/0 (socket says 172.21.15.33:34136) 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd859b9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd85406e140 0x7fd85415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58966/0 (socket says 172.21.15.33:58966) 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd85a39b700 1 -- 172.21.15.33:0/1179484764 learned_addr learned my addr 172.21.15.33:0/1179484764 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd859b9a700 1 -- 172.21.15.33:0/1179484764 >> v1:172.21.15.167:6789/0 conn(0x7fd8540afdd0 legacy=0x7fd8540b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd859b9a700 1 -- 172.21.15.33:0/1179484764 >> v1:172.21.15.33:6789/0 conn(0x7fd854159530 legacy=0x7fd85406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd859b9a700 1 -- 172.21.15.33:0/1179484764 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85406e610 con 0x7fd85406e140 2023-10-06T05:31:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd859b9a700 1 --2- 172.21.15.33:0/1179484764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd85406e140 0x7fd85415a910 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd8440098e0 tx=0x7fd8440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975457ebe761d6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:35.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.798+0000 7fd859399700 1 -- 172.21.15.33:0/1179484764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844009e80 con 0x7fd85406e140 2023-10-06T05:31:35.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd859399700 1 -- 172.21.15.33:0/1179484764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84400b760 con 0x7fd85406e140 2023-10-06T05:31:35.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd859399700 1 -- 172.21.15.33:0/1179484764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400bb00 con 0x7fd85406e140 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 -- 172.21.15.33:0/1179484764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd85406e140 msgr2=0x7fd85415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 --2- 172.21.15.33:0/1179484764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd85406e140 0x7fd85415a910 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd8440098e0 tx=0x7fd8440091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 -- 172.21.15.33:0/1179484764 shutdown_connections 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 --2- 172.21.15.33:0/1179484764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd85406e140 0x7fd85415a910 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 -- 172.21.15.33:0/1179484764 >> 172.21.15.33:0/1179484764 conn(0x7fd85406c560 msgr2=0x7fd85406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 -- 172.21.15.33:0/1179484764 shutdown_connections 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.799+0000 7fd85b39d700 1 -- 172.21.15.33:0/1179484764 wait complete. 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.801+0000 7fd85b39d700 1 Processor -- start 2023-10-06T05:31:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.801+0000 7fd85b39d700 1 -- start start 2023-10-06T05:31:35.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd85406e140 0x7fd85416da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8540afdd0 0x7fd8540cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 0x7fd854152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541523f0 con 0x7fd854159530 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85416df80 con 0x7fd8540afdd0 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8541517e0 con 0x7fd85406e140 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 0x7fd854152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 0x7fd854152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58976/0 (socket says 172.21.15.33:58976) 2023-10-06T05:31:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 -- 172.21.15.33:0/3945175445 learned_addr learned my addr 172.21.15.33:0/3945175445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd859b9a700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8540afdd0 0x7fd8540cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85a39b700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd85406e140 0x7fd85416da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 -- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd85406e140 msgr2=0x7fd85416da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd85406e140 0x7fd85416da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 -- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8540afdd0 msgr2=0x7fd8540cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8540afdd0 0x7fd8540cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85ab9c700 1 -- 172.21.15.33:0/3945175445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd844009420 con 0x7fd854159530 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.802+0000 7fd85a39b700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd85406e140 0x7fd85416da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.803+0000 7fd85ab9c700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 0x7fd854152880 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c930 tx=0x7fd84c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.803+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c012630 con 0x7fd854159530 2023-10-06T05:31:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.803+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c0127d0 con 0x7fd854159530 2023-10-06T05:31:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.803+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd854151ac0 con 0x7fd854159530 2023-10-06T05:31:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.803+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c012b90 con 0x7fd854159530 2023-10-06T05:31:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.803+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd854152080 con 0x7fd854159530 2023-10-06T05:31:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.804+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85404c380 con 0x7fd854159530 2023-10-06T05:31:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.805+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd84c00a3b0 con 0x7fd854159530 2023-10-06T05:31:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.806+0000 7fd84a7fc700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd838066f30 0x7fd8380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.806+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd84c0844a0 con 0x7fd854159530 2023-10-06T05:31:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.807+0000 7fd85a39b700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd838066f30 0x7fd8380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.807+0000 7fd85a39b700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd838066f30 0x7fd8380693e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd850007fd0 tx=0x7fd850009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:35.810+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd84c0514b0 con 0x7fd854159530 2023-10-06T05:31:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.012+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85415b420 con 0x7fd854159530 2023-10-06T05:31:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.013+0000 7fd84a7fc700 1 -- 172.21.15.33:0/3945175445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v228) v1 ==== 74+0+17025 (secure 0 0 0) 0x7fd84c054ad0 con 0x7fd854159530 2023-10-06T05:31:36.016 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:36.016 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":228,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:30:09.896755+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:29:50.569789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-06T05:29:51.932764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.2","mappings":[{"from":0,"to":2}]},{"pgid":"34.20","mappings":[{"from":0,"to":1}]},{"pgid":"34.29","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd838066f30 msgr2=0x7fd8380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd838066f30 0x7fd8380693e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd850007fd0 tx=0x7fd850009040 comp rx=0 tx=0).stop 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 msgr2=0x7fd854152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 0x7fd854152880 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c930 tx=0x7fd84c0026e0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 shutdown_connections 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd838066f30 0x7fd8380693e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd854159530 0x7fd854152880 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd85406e140 0x7fd85416da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 --2- 172.21.15.33:0/3945175445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8540afdd0 0x7fd8540cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.015+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 >> 172.21.15.33:0/3945175445 conn(0x7fd85406c560 msgr2=0x7fd854153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.016+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 shutdown_connections 2023-10-06T05:31:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.016+0000 7fd85b39d700 1 -- 172.21.15.33:0/3945175445 wait complete. 2023-10-06T05:31:36.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:31:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.223+0000 7f495a99d700 1 Processor -- start 2023-10-06T05:31:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.223+0000 7f495a99d700 1 -- start start 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495a99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4954159400 0x7f49540cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495a99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49541597d0 0x7f49540cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495a99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49540d0ed0 con 0x7f49540cc770 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495a99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49540b1160 con 0x7f49541597d0 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495a99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49540cb6d0 con 0x7f4954159400 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49541597d0 0x7f49540cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495919a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49541597d0 0x7f49540cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58992/0 (socket says 172.21.15.33:58992) 2023-10-06T05:31:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.224+0000 7f495919a700 1 -- 172.21.15.33:0/1870701129 learned_addr learned my addr 172.21.15.33:0/1870701129 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f495999b700 1 --2- 172.21.15.33:0/1870701129 >> v2:172.21.15.167:3300/0 conn(0x7f4954159400 0x7f49540cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f495919a700 1 -- 172.21.15.33:0/1870701129 >> v2:172.21.15.167:3300/0 conn(0x7f4954159400 msgr2=0x7f49540cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f495919a700 1 --2- 172.21.15.33:0/1870701129 >> v2:172.21.15.167:3300/0 conn(0x7f4954159400 0x7f49540cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f495919a700 1 -- 172.21.15.33:0/1870701129 >> v1:172.21.15.33:6789/0 conn(0x7f49540cc770 legacy=0x7f495406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f495919a700 1 -- 172.21.15.33:0/1870701129 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540cb9b0 con 0x7f49541597d0 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f4958999700 1 -- 172.21.15.33:0/1870701129 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662867670 0 0) 0x7f49540d0ed0 con 0x7f49540cc770 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f495919a700 1 --2- 172.21.15.33:0/1870701129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49541597d0 0x7f49540cb0a0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f495406c030 tx=0x7f4944009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac1a5b493c6033e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f4958999700 1 -- 172.21.15.33:0/1870701129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944009e70 con 0x7f49541597d0 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f4958999700 1 -- 172.21.15.33:0/1870701129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494400b860 con 0x7f49541597d0 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.225+0000 7f4958999700 1 -- 172.21.15.33:0/1870701129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494400bc00 con 0x7f49541597d0 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 -- 172.21.15.33:0/1870701129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49541597d0 msgr2=0x7f49540cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 --2- 172.21.15.33:0/1870701129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49541597d0 0x7f49540cb0a0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f495406c030 tx=0x7f4944009250 comp rx=0 tx=0).stop 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 -- 172.21.15.33:0/1870701129 shutdown_connections 2023-10-06T05:31:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 --2- 172.21.15.33:0/1870701129 >> v2:172.21.15.167:3300/0 conn(0x7f4954159400 0x7f49540cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 --2- 172.21.15.33:0/1870701129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49541597d0 0x7f49540cb0a0 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 -- 172.21.15.33:0/1870701129 >> 172.21.15.33:0/1870701129 conn(0x7f495406e4b0 msgr2=0x7f49540d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 -- 172.21.15.33:0/1870701129 shutdown_connections 2023-10-06T05:31:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.226+0000 7f495a99d700 1 -- 172.21.15.33:0/1870701129 wait complete. 2023-10-06T05:31:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.228+0000 7f495a99d700 1 Processor -- start 2023-10-06T05:31:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.228+0000 7f495a99d700 1 -- start start 2023-10-06T05:31:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 0x7f4954152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4954159400 0x7f4954169570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49541597d0 0x7f4954155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495416a5d0 con 0x7f49540cc770 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540c7ae0 con 0x7f49541597d0 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4954153180 con 0x7f4954159400 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 0x7f4954152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 0x7f4954152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59008/0 (socket says 172.21.15.33:59008) 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 -- 172.21.15.33:0/1664354565 learned_addr learned my addr 172.21.15.33:0/1664354565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495919a700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4954159400 0x7f4954169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495a19c700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49541597d0 0x7f4954155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 -- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4954159400 msgr2=0x7f4954169570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4954159400 0x7f4954169570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 -- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49541597d0 msgr2=0x7f4954155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49541597d0 0x7f4954155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.229+0000 7f495999b700 1 -- 172.21.15.33:0/1664354565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440094f0 con 0x7f49540cc770 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.230+0000 7f495999b700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 0x7f4954152c40 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f495000ec80 tx=0x7f49500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.230+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495000c9d0 con 0x7f49540cc770 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.230+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495000cb30 con 0x7f49540cc770 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.230+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495416a010 con 0x7f49540cc770 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.230+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950013470 con 0x7f49540cc770 2023-10-06T05:31:36.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.230+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49540c7770 con 0x7f49540cc770 2023-10-06T05:31:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.231+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49540c7dc0 con 0x7f49540cc770 2023-10-06T05:31:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.232+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49500137e0 con 0x7f49540cc770 2023-10-06T05:31:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.233+0000 7f4949ffb700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4938066f30 0x7f49380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.233+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4950084b60 con 0x7f49540cc770 2023-10-06T05:31:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.233+0000 7f495919a700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4938066f30 0x7f49380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.234+0000 7f495919a700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4938066f30 0x7f49380693e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f495406c030 tx=0x7f4944017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.237+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4950051b70 con 0x7f49540cc770 2023-10-06T05:31:36.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.487+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f495404c380 con 0x7f49540cc770 2023-10-06T05:31:36.490 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:31:36.487+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:31:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.492+0000 7f4949ffb700 1 -- 172.21.15.33:0/1664354565 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v177) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4950055190 con 0x7f49540cc770 2023-10-06T05:31:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4938066f30 msgr2=0x7f49380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4938066f30 0x7f49380693e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f495406c030 tx=0x7f4944017000 comp rx=0 tx=0).stop 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 msgr2=0x7f4954152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 0x7f4954152c40 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f495000ec80 tx=0x7f49500091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 shutdown_connections 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4938066f30 0x7f49380693e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49541597d0 0x7f4954155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49540cc770 0x7f4954152c40 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 --2- 172.21.15.33:0/1664354565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4954159400 0x7f4954169570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.495+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 >> 172.21.15.33:0/1664354565 conn(0x7f495406e4b0 msgr2=0x7f49540b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.496+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 shutdown_connections 2023-10-06T05:31:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.496+0000 7f495a99d700 1 -- 172.21.15.33:0/1664354565 wait complete. 2023-10-06T05:31:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:31:36.505 INFO:tasks.ceph.mds.a.smithi033.stderr: -6386> 2023-10-06T05:29:57.422+0000 7fefc5ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:31:36.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:31:36.552 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:31:36.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.704+0000 7fea81d9d700 1 Processor -- start 2023-10-06T05:31:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.704+0000 7fea81d9d700 1 -- start start 2023-10-06T05:31:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.705+0000 7fea81d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fea7c156370 0x7fea7c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.705+0000 7fea81d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea7c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.705+0000 7fea81d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0ad430 con 0x7fea7c0b01f0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.705+0000 7fea81d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7c0afe20 con 0x7fea7c156370 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.705+0000 7fea81d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c06bc70 con 0x7fea7c0cc7e0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea8159c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fea7c0b01f0 0x7fea7c0ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34156/0 (socket says 172.21.15.33:34156) 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea8159c700 1 -- 172.21.15.33:0/1778773087 learned_addr learned my addr 172.21.15.33:0/1778773087 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea80d9b700 1 --2- 172.21.15.33:0/1778773087 >> v2:172.21.15.167:3300/0 conn(0x7fea7c156370 0x7fea7c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7bfff700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea7c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729330428 0 0) 0x7fea7c0ad430 con 0x7fea7c0b01f0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea6c003570 con 0x7fea7c0b01f0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075607335 0 0) 0x7fea6c003570 con 0x7fea7c0b01f0 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 msgr2=0x7fea7c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:36.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea7c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 >> v2:172.21.15.167:3300/0 conn(0x7fea7c156370 msgr2=0x7fea7c0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 --2- 172.21.15.33:0/1778773087 >> v2:172.21.15.167:3300/0 conn(0x7fea7c156370 0x7fea7c0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c0c92f0 con 0x7fea7c0b01f0 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fea740017f0 con 0x7fea7c0b01f0 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea80d9b700 1 --2- 172.21.15.33:0/1778773087 >> v2:172.21.15.167:3300/0 conn(0x7fea7c156370 0x7fea7c0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.706+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 >> v1:172.21.15.33:6789/0 conn(0x7fea7c0b01f0 legacy=0x7fea7c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7b7fe700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c003ae0 0x7fea6c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7b7fe700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7b7fe700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0afe20 con 0x7fea6c004d50 2023-10-06T05:31:36.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c06bc70 con 0x7fea6c003ae0 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c00dd50 con 0x7fea7c0cc7e0 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea8159c700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7bfff700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea80d9b700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c003ae0 0x7fea6c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea8159c700 1 -- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 msgr2=0x7fea6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea8159c700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea8159c700 1 -- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c003ae0 msgr2=0x7fea6c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea8159c700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea6c003ae0 0x7fea6c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea8159c700 1 -- 172.21.15.33:0/1778773087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c003570 con 0x7fea6c004d50 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.707+0000 7fea7bfff700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c0cc7e0 0x7fea6c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea8159c700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea6c0051c0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fea7400a8a0 tx=0x7fea74002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a382d88d3b532d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740070f0 con 0x7fea6c004d50 2023-10-06T05:31:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74007250 con 0x7fea6c004d50 2023-10-06T05:31:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea7b7fe700 1 -- 172.21.15.33:0/1778773087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74007610 con 0x7fea6c004d50 2023-10-06T05:31:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea81d9d700 1 -- 172.21.15.33:0/1778773087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 msgr2=0x7fea6c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea81d9d700 1 --2- 172.21.15.33:0/1778773087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea6c0051c0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fea7400a8a0 tx=0x7fea74002140 comp rx=0 tx=0).stop 2023-10-06T05:31:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea8159c700 1 -- 172.21.15.33:0/1778773087 reap_dead start 2023-10-06T05:31:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea81d9d700 1 -- 172.21.15.33:0/1778773087 shutdown_connections 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea81d9d700 1 -- 172.21.15.33:0/1778773087 >> 172.21.15.33:0/1778773087 conn(0x7fea7c06d030 msgr2=0x7fea7c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea81d9d700 1 -- 172.21.15.33:0/1778773087 shutdown_connections 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.708+0000 7fea81d9d700 1 -- 172.21.15.33:0/1778773087 wait complete. 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.710+0000 7fea81d9d700 1 Processor -- start 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.710+0000 7fea81d9d700 1 -- start start 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea81d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea7c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea81d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c14fed0 0x7fea7c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea81d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c150340 0x7fea7c150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea81d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c155760 con 0x7fea6c004d50 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea81d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c156c50 con 0x7fea7c150340 2023-10-06T05:31:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea81d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1551a0 con 0x7fea7c14fed0 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea80d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea7c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea80d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea7c0c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59026/0 (socket says 172.21.15.33:59026) 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea80d9b700 1 -- 172.21.15.33:0/746256396 learned_addr learned my addr 172.21.15.33:0/746256396 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea8159c700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c150340 0x7fea7c150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.711+0000 7fea7bfff700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c14fed0 0x7fea7c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea80d9b700 1 -- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c14fed0 msgr2=0x7fea7c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea80d9b700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c14fed0 0x7fea7c1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea80d9b700 1 -- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c150340 msgr2=0x7fea7c150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea80d9b700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c150340 0x7fea7c150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea80d9b700 1 -- 172.21.15.33:0/746256396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c155480 con 0x7fea6c004d50 2023-10-06T05:31:36.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea80d9b700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea7c0c7860 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fea70011da0 tx=0x7fea700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7000fcd0 con 0x7fea6c004d50 2023-10-06T05:31:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7000fe30 con 0x7fea6c004d50 2023-10-06T05:31:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7001d6d0 con 0x7fea6c004d50 2023-10-06T05:31:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c1568e0 con 0x7fea6c004d50 2023-10-06T05:31:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.712+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c1586b0 con 0x7fea6c004d50 2023-10-06T05:31:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.713+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c156740 con 0x7fea6c004d50 2023-10-06T05:31:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.714+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea7001da80 con 0x7fea6c004d50 2023-10-06T05:31:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.715+0000 7fea78ff9700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea60066f30 0x7fea600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.716+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fea70018070 con 0x7fea6c004d50 2023-10-06T05:31:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.716+0000 7fea7bfff700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea60066f30 0x7fea600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.717+0000 7fea7bfff700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea60066f30 0x7fea600693e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fea68006eb0 tx=0x7fea68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.720+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea7c156740 con 0x7fea6c004d50 2023-10-06T05:31:36.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:36.973+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fea7c156740 con 0x7fea6c004d50 2023-10-06T05:31:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.493+0000 7fea78ff9700 1 -- 172.21.15.33:0/746256396 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v178) v1 ==== 104+0+0 (secure 0 0 0) 0x7fea7c156740 con 0x7fea6c004d50 2023-10-06T05:31:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea60066f30 msgr2=0x7fea600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea60066f30 0x7fea600693e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fea68006eb0 tx=0x7fea68008040 comp rx=0 tx=0).stop 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 msgr2=0x7fea7c0c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea7c0c7860 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fea70011da0 tx=0x7fea700091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 shutdown_connections 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea60066f30 0x7fea600693e0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c150340 0x7fea7c150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea6c004d50 0x7fea7c0c7860 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.496+0000 7fea81d9d700 1 --2- 172.21.15.33:0/746256396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea7c14fed0 0x7fea7c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.497+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 >> 172.21.15.33:0/746256396 conn(0x7fea7c06d030 msgr2=0x7fea7c0cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.497+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 shutdown_connections 2023-10-06T05:31:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.497+0000 7fea81d9d700 1 -- 172.21.15.33:0/746256396 wait complete. 2023-10-06T05:31:37.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:31:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.751+0000 7f4912b9d700 1 Processor -- start 2023-10-06T05:31:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.751+0000 7f4912b9d700 1 -- start start 2023-10-06T05:31:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4912b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f490c0ad3e0 0x7f490c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4912b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06b1f0 con 0x7f490c0ad010 2023-10-06T05:31:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4912b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06a5e0 con 0x7f490c1570a0 2023-10-06T05:31:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4912b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c06a3f0 con 0x7f490c0ad3e0 2023-10-06T05:31:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4911b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f490c0ad010 0x7f490c06ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34162/0 (socket says 172.21.15.33:34162) 2023-10-06T05:31:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4911b9b700 1 -- 172.21.15.33:0/1688258830 learned_addr learned my addr 172.21.15.33:0/1688258830 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:37.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f491139a700 1 --2- 172.21.15.33:0/1688258830 >> v2:172.21.15.167:3300/0 conn(0x7f490c0ad3e0 0x7f490c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823005098 0 0) 0x7f490c06b1f0 con 0x7f490c0ad010 2023-10-06T05:31:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f8003570 con 0x7f490c0ad010 2023-10-06T05:31:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.753+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532521303 0 0) 0x7f490c06a5e0 con 0x7f490c1570a0 2023-10-06T05:31:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490c06b1f0 con 0x7f490c1570a0 2023-10-06T05:31:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f491139a700 1 -- 172.21.15.33:0/1688258830 >> v1:172.21.15.167:6789/0 conn(0x7f490c1570a0 legacy=0x7f490c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f491139a700 1 -- 172.21.15.33:0/1688258830 >> v1:172.21.15.33:6789/0 conn(0x7f490c0ad010 legacy=0x7f490c06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f491139a700 1 -- 172.21.15.33:0/1688258830 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c06a8c0 con 0x7f490c0ad3e0 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f491139a700 1 --2- 172.21.15.33:0/1688258830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c0ad3e0 0x7f490c156b60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0098e0 tx=0x7f48fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1e39b34e45f30b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011630 con 0x7f490c0ad3e0 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc011790 con 0x7f490c0ad3e0 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.754+0000 7f4910b99700 1 -- 172.21.15.33:0/1688258830 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011b30 con 0x7f490c0ad3e0 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 -- 172.21.15.33:0/1688258830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c0ad3e0 msgr2=0x7f490c156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1688258830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c0ad3e0 0x7f490c156b60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0098e0 tx=0x7f48fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 -- 172.21.15.33:0/1688258830 shutdown_connections 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1688258830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c0ad3e0 0x7f490c156b60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 -- 172.21.15.33:0/1688258830 >> 172.21.15.33:0/1688258830 conn(0x7f490c068090 msgr2=0x7f490c06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 -- 172.21.15.33:0/1688258830 shutdown_connections 2023-10-06T05:31:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.755+0000 7f4912b9d700 1 -- 172.21.15.33:0/1688258830 wait complete. 2023-10-06T05:31:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.756+0000 7f4912b9d700 1 Processor -- start 2023-10-06T05:31:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 -- start start 2023-10-06T05:31:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 0x7f490c13f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f490c0ad3e0 0x7f490c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c1570a0 0x7f490c0a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c150930 con 0x7f490c0ad010 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c154b20 con 0x7f490c1570a0 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.757+0000 7f4912b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c154e00 con 0x7f490c0ad3e0 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 0x7f490c13f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 0x7f490c13f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59034/0 (socket says 172.21.15.33:59034) 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 -- 172.21.15.33:0/1314872711 learned_addr learned my addr 172.21.15.33:0/1314872711 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 -- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f490c0ad3e0 msgr2=0x7f490c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f491139a700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f490c0ad3e0 0x7f490c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f491239c700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c1570a0 0x7f490c0a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f490c0ad3e0 0x7f490c1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 -- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c1570a0 msgr2=0x7f490c0a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c1570a0 0x7f490c0a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 -- 172.21.15.33:0/1314872711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc009420 con 0x7f490c0ad010 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f491139a700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f490c0ad3e0 0x7f490c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f491239c700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c1570a0 0x7f490c0a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.758+0000 7f4911b9b700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 0x7f490c13f000 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f490800c790 tx=0x7f4908007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.759+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908005b30 con 0x7f490c0ad010 2023-10-06T05:31:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.759+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908005c90 con 0x7f490c0ad010 2023-10-06T05:31:37.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.759+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c150b10 con 0x7f490c0ad010 2023-10-06T05:31:37.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.759+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908007300 con 0x7f490c0ad010 2023-10-06T05:31:37.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.759+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c13f540 con 0x7f490c0ad010 2023-10-06T05:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.760+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c063ff0 con 0x7f490c0ad010 2023-10-06T05:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.761+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f490801b400 con 0x7f490c0ad010 2023-10-06T05:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.761+0000 7f4901ffb700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48f0066f30 0x7f48f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.762+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f49080842a0 con 0x7f490c0ad010 2023-10-06T05:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.762+0000 7f491139a700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48f0066f30 0x7f48f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.762+0000 7f491139a700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48f0066f30 0x7f48f00693e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f490c06bfc0 tx=0x7f48fc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:37.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.766+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49080512e0 con 0x7f490c0ad010 2023-10-06T05:31:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:37.970+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f490c161670 con 0x7f490c0ad010 2023-10-06T05:31:38.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.496+0000 7f4901ffb700 1 -- 172.21.15.33:0/1314872711 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v230) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4908054900 con 0x7f490c0ad010 2023-10-06T05:31:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48f0066f30 msgr2=0x7f48f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48f0066f30 0x7f48f00693e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f490c06bfc0 tx=0x7f48fc017000 comp rx=0 tx=0).stop 2023-10-06T05:31:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 msgr2=0x7f490c13f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 0x7f490c13f000 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f490800c790 tx=0x7f4908007ea0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 shutdown_connections 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f48f0066f30 0x7f48f00693e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f490c1570a0 0x7f490c0a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f490c0ad010 0x7f490c13f000 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 --2- 172.21.15.33:0/1314872711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f490c0ad3e0 0x7f490c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 >> 172.21.15.33:0/1314872711 conn(0x7f490c068090 msgr2=0x7f490c0afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.500+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 shutdown_connections 2023-10-06T05:31:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.501+0000 7f4912b9d700 1 -- 172.21.15.33:0/1314872711 wait complete. 2023-10-06T05:31:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:31:38.526 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:31:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.719+0000 7fe61eb9d700 1 Processor -- start 2023-10-06T05:31:38.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.719+0000 7fe61eb9d700 1 -- start start 2023-10-06T05:31:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.720+0000 7fe61eb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6180c9720 0x7fe6180ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.720+0000 7fe61eb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe61806c560 0x7fe61806c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:38.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.720+0000 7fe61eb9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180afc40 con 0x7fe61815dc50 2023-10-06T05:31:38.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.720+0000 7fe61eb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61815d7c0 con 0x7fe6180c9720 2023-10-06T05:31:38.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.720+0000 7fe61eb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe618159bc0 con 0x7fe61806c560 2023-10-06T05:31:38.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61db9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6180c9720 0x7fe6180ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:38.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61d39a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe61815dc50 0x7fe6180aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34170/0 (socket says 172.21.15.33:34170) 2023-10-06T05:31:38.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61d39a700 1 -- 172.21.15.33:0/910853408 learned_addr learned my addr 172.21.15.33:0/910853408 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:38.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61db9b700 1 -- 172.21.15.33:0/910853408 >> v2:172.21.15.167:3300/0 conn(0x7fe61806c560 msgr2=0x7fe61806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61e39c700 1 --2- 172.21.15.33:0/910853408 >> v2:172.21.15.167:3300/0 conn(0x7fe61806c560 0x7fe61806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61db9b700 1 --2- 172.21.15.33:0/910853408 >> v2:172.21.15.167:3300/0 conn(0x7fe61806c560 0x7fe61806c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61db9b700 1 -- 172.21.15.33:0/910853408 >> v1:172.21.15.33:6789/0 conn(0x7fe61815dc50 legacy=0x7fe6180aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61db9b700 1 -- 172.21.15.33:0/910853408 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61806ced0 con 0x7fe6180c9720 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61cb99700 1 -- 172.21.15.33:0/910853408 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845770873 0 0) 0x7fe6180afc40 con 0x7fe61815dc50 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.721+0000 7fe61e39c700 1 --2- 172.21.15.33:0/910853408 >> v2:172.21.15.167:3300/0 conn(0x7fe61806c560 0x7fe61806c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61db9b700 1 --2- 172.21.15.33:0/910853408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ad520 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fe6180696b0 tx=0x7fe6140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33ce103f6c0dc33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61cb99700 1 -- 172.21.15.33:0/910853408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400b6b0 con 0x7fe6180c9720 2023-10-06T05:31:38.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61cb99700 1 -- 172.21.15.33:0/910853408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61400b810 con 0x7fe6180c9720 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61cb99700 1 -- 172.21.15.33:0/910853408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400bbb0 con 0x7fe6180c9720 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61eb9d700 1 -- 172.21.15.33:0/910853408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 msgr2=0x7fe6180ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/910853408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ad520 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fe6180696b0 tx=0x7fe6140091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61eb9d700 1 -- 172.21.15.33:0/910853408 shutdown_connections 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/910853408 >> v2:172.21.15.167:3300/0 conn(0x7fe61806c560 0x7fe61806c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/910853408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ad520 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.722+0000 7fe61eb9d700 1 -- 172.21.15.33:0/910853408 >> 172.21.15.33:0/910853408 conn(0x7fe6180675f0 msgr2=0x7fe6180679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.723+0000 7fe61eb9d700 1 -- 172.21.15.33:0/910853408 shutdown_connections 2023-10-06T05:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.723+0000 7fe61eb9d700 1 -- 172.21.15.33:0/910853408 wait complete. 2023-10-06T05:31:38.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.724+0000 7fe61eb9d700 1 Processor -- start 2023-10-06T05:31:38.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.724+0000 7fe61eb9d700 1 -- start start 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61806c560 0x7fe6180b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe61815dc50 0x7fe6180cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618150770 con 0x7fe6180c9720 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180ac140 con 0x7fe61806c560 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180cc3d0 con 0x7fe61815dc50 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61d39a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59062/0 (socket says 172.21.15.33:59062) 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.725+0000 7fe61d39a700 1 -- 172.21.15.33:0/1100854905 learned_addr learned my addr 172.21.15.33:0/1100854905 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61db9b700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61806c560 0x7fe6180b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61e39c700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe61815dc50 0x7fe6180cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61d39a700 1 -- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe61815dc50 msgr2=0x7fe6180cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61d39a700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe61815dc50 0x7fe6180cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61d39a700 1 -- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61806c560 msgr2=0x7fe6180b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61d39a700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61806c560 0x7fe6180b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61d39a700 1 -- 172.21.15.33:0/1100854905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614009420 con 0x7fe6180c9720 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe61d39a700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ab680 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fe60800c5b0 tx=0x7fe608001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.726+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608005b30 con 0x7fe6180c9720 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.727+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608005c90 con 0x7fe6180c9720 2023-10-06T05:31:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.727+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608007300 con 0x7fe6180c9720 2023-10-06T05:31:38.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.727+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6180cc6b0 con 0x7fe6180c9720 2023-10-06T05:31:38.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.727+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6180b0590 con 0x7fe6180c9720 2023-10-06T05:31:38.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.728+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6180ccbd0 con 0x7fe6180c9720 2023-10-06T05:31:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.729+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe60801b3b0 con 0x7fe6180c9720 2023-10-06T05:31:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.729+0000 7fe60dffb700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5fc066f30 0x7fe5fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.730+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe608083a30 con 0x7fe6180c9720 2023-10-06T05:31:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.730+0000 7fe61db9b700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5fc066f30 0x7fe5fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.731+0000 7fe61db9b700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5fc066f30 0x7fe5fc0693e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe6140093f0 tx=0x7fe614017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:38.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.733+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6180ccbd0 con 0x7fe6180c9720 2023-10-06T05:31:38.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:38.936+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6180ccbd0 con 0x7fe6180c9720 2023-10-06T05:31:39.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.889+0000 7fe60dffb700 1 -- 172.21.15.33:0/1100854905 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v231) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe6180ccbd0 con 0x7fe6180c9720 2023-10-06T05:31:39.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5fc066f30 msgr2=0x7fe5fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5fc066f30 0x7fe5fc0693e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe6140093f0 tx=0x7fe614017000 comp rx=0 tx=0).stop 2023-10-06T05:31:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 msgr2=0x7fe6180ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ab680 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fe60800c5b0 tx=0x7fe608001070 comp rx=0 tx=0).stop 2023-10-06T05:31:39.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 shutdown_connections 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe5fc066f30 0x7fe5fc0693e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe61815dc50 0x7fe6180cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61806c560 0x7fe6180b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 --2- 172.21.15.33:0/1100854905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6180c9720 0x7fe6180ab680 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.893+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 >> 172.21.15.33:0/1100854905 conn(0x7fe6180675f0 msgr2=0x7fe618150fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.894+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 shutdown_connections 2023-10-06T05:31:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:39.894+0000 7fe61eb9d700 1 -- 172.21.15.33:0/1100854905 wait complete. 2023-10-06T05:31:39.898 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:31:39.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:31:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.111+0000 7f476719d700 1 Processor -- start 2023-10-06T05:31:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.112+0000 7f476719d700 1 -- start start 2023-10-06T05:31:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4760156370 0x7f47600af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600b01f0 0x7f47600b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47600ad430 con 0x7f47600cc7e0 2023-10-06T05:31:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47600afe20 con 0x7f4760156370 2023-10-06T05:31:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600b0b70 con 0x7f47600b01f0 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4760156370 0x7f47600af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476619b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4760156370 0x7f47600af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33126/0 (socket says 172.21.15.33:33126) 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f47600cc7e0 0x7f47600c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53444/0 (socket says 172.21.15.33:53444) 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.113+0000 7f476619b700 1 -- 172.21.15.33:0/3767238636 learned_addr learned my addr 172.21.15.33:0/3767238636 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f476699c700 1 --2- 172.21.15.33:0/3767238636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600b01f0 0x7f47600b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f476619b700 1 -- 172.21.15.33:0/3767238636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600b01f0 msgr2=0x7f47600b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f476619b700 1 --2- 172.21.15.33:0/3767238636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600b01f0 0x7f47600b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f476619b700 1 -- 172.21.15.33:0/3767238636 >> v1:172.21.15.33:6789/0 conn(0x7f47600cc7e0 legacy=0x7f47600c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f476619b700 1 -- 172.21.15.33:0/3767238636 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476006bc70 con 0x7f4760156370 2023-10-06T05:31:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f476619b700 1 --2- 172.21.15.33:0/3767238636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47600af8e0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f475c0099b0 tx=0x7f475c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9a8869027c8cbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f4765199700 1 -- 172.21.15.33:0/3767238636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c00b860 con 0x7f4760156370 2023-10-06T05:31:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f4765199700 1 -- 172.21.15.33:0/3767238636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c00b9c0 con 0x7f4760156370 2023-10-06T05:31:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.114+0000 7f4765199700 1 -- 172.21.15.33:0/3767238636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c00bd60 con 0x7f4760156370 2023-10-06T05:31:40.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 -- 172.21.15.33:0/3767238636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 msgr2=0x7f47600af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 --2- 172.21.15.33:0/3767238636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47600af8e0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f475c0099b0 tx=0x7f475c009250 comp rx=0 tx=0).stop 2023-10-06T05:31:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 -- 172.21.15.33:0/3767238636 shutdown_connections 2023-10-06T05:31:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 --2- 172.21.15.33:0/3767238636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47600af8e0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 --2- 172.21.15.33:0/3767238636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600b01f0 0x7f47600b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 -- 172.21.15.33:0/3767238636 >> 172.21.15.33:0/3767238636 conn(0x7f476006d030 msgr2=0x7f47600cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:40.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 -- 172.21.15.33:0/3767238636 shutdown_connections 2023-10-06T05:31:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.115+0000 7f476719d700 1 -- 172.21.15.33:0/3767238636 wait complete. 2023-10-06T05:31:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.116+0000 7f476719d700 1 Processor -- start 2023-10-06T05:31:40.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.117+0000 7f476719d700 1 -- start start 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.117+0000 7f476719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47600b01f0 0x7f4760146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600cc7e0 0x7f4760158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47601510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4760147380 con 0x7f4760156370 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4760153470 con 0x7f47600b01f0 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4760153750 con 0x7f47600cc7e0 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47601510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47601510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33136/0 (socket says 172.21.15.33:33136) 2023-10-06T05:31:40.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 -- 172.21.15.33:0/2209114197 learned_addr learned my addr 172.21.15.33:0/2209114197 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476619b700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47600b01f0 0x7f4760146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 -- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600cc7e0 msgr2=0x7f4760158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600cc7e0 0x7f4760158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 -- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47600b01f0 msgr2=0x7f4760146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47600b01f0 0x7f4760146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476699c700 1 -- 172.21.15.33:0/2209114197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0094f0 con 0x7f4760156370 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.118+0000 7f476619b700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47600b01f0 0x7f4760146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.119+0000 7f476699c700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47601510a0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f4758007f20 tx=0x7f4758009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.119+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475800eca0 con 0x7f4760156370 2023-10-06T05:31:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.119+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475800ee20 con 0x7f4760156370 2023-10-06T05:31:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.119+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758014670 con 0x7f4760156370 2023-10-06T05:31:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.119+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4760153a30 con 0x7f4760156370 2023-10-06T05:31:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.119+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4760153ff0 con 0x7f4760156370 2023-10-06T05:31:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.120+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4760154510 con 0x7f4760156370 2023-10-06T05:31:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.121+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47580149e0 con 0x7f4760156370 2023-10-06T05:31:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.122+0000 7f47567fc700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4744066f30 0x7f47440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.122+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4758084b60 con 0x7f4760156370 2023-10-06T05:31:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.122+0000 7f476619b700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4744066f30 0x7f47440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.123+0000 7f476619b700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4744066f30 0x7f47440693e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f475c0099b0 tx=0x7f475c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:40.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.126+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4760154510 con 0x7f4760156370 2023-10-06T05:31:40.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.330+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4760154510 con 0x7f4760156370 2023-10-06T05:31:40.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.334+0000 7f47567fc700 1 -- 172.21.15.33:0/2209114197 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4760154510 con 0x7f4760156370 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.336+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4744066f30 msgr2=0x7f47440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.336+0000 7f476719d700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4744066f30 0x7f47440693e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f475c0099b0 tx=0x7f475c017000 comp rx=0 tx=0).stop 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.336+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 msgr2=0x7f47601510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.336+0000 7f476719d700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47601510a0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f4758007f20 tx=0x7f4758009860 comp rx=0 tx=0).stop 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 shutdown_connections 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4744066f30 0x7f47440693e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4760156370 0x7f47601510a0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47600b01f0 0x7f4760146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 --2- 172.21.15.33:0/2209114197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47600cc7e0 0x7f4760158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 >> 172.21.15.33:0/2209114197 conn(0x7f476006d030 msgr2=0x7f47600abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 shutdown_connections 2023-10-06T05:31:40.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.337+0000 7f476719d700 1 -- 172.21.15.33:0/2209114197 wait complete. 2023-10-06T05:31:40.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_read_caps' 2023-10-06T05:31:40.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.541+0000 7f56989fe700 1 Processor -- start 2023-10-06T05:31:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.542+0000 7f56989fe700 1 -- start start 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f56989fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f569406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f56989fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f569406a6a0 0x7f56940c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f56989fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569406ab60 con 0x7f56940c8b50 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f56989fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569406b7e0 con 0x7f569406a6a0 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f56989fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569406ae40 con 0x7f5694155af0 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f569259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f569406a6a0 0x7f56940c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f569259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f569406a6a0 0x7f56940c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33142/0 (socket says 172.21.15.33:33142) 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.543+0000 7f569259c700 1 -- 172.21.15.33:0/1556500416 learned_addr learned my addr 172.21.15.33:0/1556500416 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f569259c700 1 -- 172.21.15.33:0/1556500416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 msgr2=0x7f569406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f5692d9d700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f569406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f569259c700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f569406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f569259c700 1 -- 172.21.15.33:0/1556500416 >> v1:172.21.15.33:6789/0 conn(0x7f56940c8b50 legacy=0x7f56941598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f569259c700 1 -- 172.21.15.33:0/1556500416 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569406b120 con 0x7f569406a6a0 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f5691d9b700 1 -- 172.21.15.33:0/1556500416 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001119964 0 0) 0x7f569406ab60 con 0x7f56940c8b50 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f5692d9d700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f569406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f569259c700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940c7210 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f56880099b0 tx=0x7f5688009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b64b27cf27ebf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f5691d9b700 1 -- 172.21.15.33:0/1556500416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568800b860 con 0x7f569406a6a0 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f5691d9b700 1 -- 172.21.15.33:0/1556500416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568800b9c0 con 0x7f569406a6a0 2023-10-06T05:31:40.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.544+0000 7f5691d9b700 1 -- 172.21.15.33:0/1556500416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568800bd60 con 0x7f569406a6a0 2023-10-06T05:31:40.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 -- 172.21.15.33:0/1556500416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 msgr2=0x7f56940c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940c7210 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f56880099b0 tx=0x7f5688009250 comp rx=0 tx=0).stop 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 -- 172.21.15.33:0/1556500416 shutdown_connections 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f569406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 --2- 172.21.15.33:0/1556500416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940c7210 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 -- 172.21.15.33:0/1556500416 >> 172.21.15.33:0/1556500416 conn(0x7f569406d030 msgr2=0x7f569406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 -- 172.21.15.33:0/1556500416 shutdown_connections 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.545+0000 7f56989fe700 1 -- 172.21.15.33:0/1556500416 wait complete. 2023-10-06T05:31:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.546+0000 7f56989fe700 1 Processor -- start 2023-10-06T05:31:40.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.547+0000 7f56989fe700 1 -- start start 2023-10-06T05:31:40.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f56989fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f56989fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56940c8b50 0x7f56940ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f56989fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f56941537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f56989fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56940b0490 con 0x7f569406a6a0 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f56989fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56940ac4a0 con 0x7f56940c8b50 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f56989fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56940ac780 con 0x7f5694155af0 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33154/0 (socket says 172.21.15.33:33154) 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 -- 172.21.15.33:0/2279698395 learned_addr learned my addr 172.21.15.33:0/2279698395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 -- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 msgr2=0x7f56941537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f569359e700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f56941537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f569259c700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56940c8b50 0x7f56940ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f56941537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 -- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56940c8b50 msgr2=0x7f56940ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56940c8b50 0x7f56940ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f5692d9d700 1 -- 172.21.15.33:0/2279698395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880094f0 con 0x7f569406a6a0 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f569259c700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56940c8b50 0x7f56940ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.548+0000 7f569359e700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f56941537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.549+0000 7f5692d9d700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940abaf0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f568400c830 tx=0x7f5684008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.549+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684011600 con 0x7f569406a6a0 2023-10-06T05:31:40.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.549+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684011760 con 0x7f569406a6a0 2023-10-06T05:31:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.549+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56940aca60 con 0x7f569406a6a0 2023-10-06T05:31:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.549+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684011b20 con 0x7f569406a6a0 2023-10-06T05:31:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.549+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56940ac030 con 0x7f569406a6a0 2023-10-06T05:31:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.550+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569404c380 con 0x7f569406a6a0 2023-10-06T05:31:40.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.551+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f568400a3e0 con 0x7f569406a6a0 2023-10-06T05:31:40.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.552+0000 7f567affd700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:40.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.552+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f56840848f0 con 0x7f569406a6a0 2023-10-06T05:31:40.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.552+0000 7f569259c700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:40.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.553+0000 7f569259c700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5680066f30 0x7f56800693e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f56880099b0 tx=0x7f5688018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:40.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.556+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5684051c60 con 0x7f569406a6a0 2023-10-06T05:31:40.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.839+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_read_caps"]} v 0) v1 -- 0x7f569406b400 con 0x7f569406a6a0 2023-10-06T05:31:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.892+0000 7f567affd700 1 -- 172.21.15.33:0/2279698395 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_read_caps"]}]=0 v857) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5684055280 con 0x7f569406a6a0 2023-10-06T05:31:40.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5680066f30 msgr2=0x7f56800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5680066f30 0x7f56800693e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f56880099b0 tx=0x7f5688018000 comp rx=0 tx=0).stop 2023-10-06T05:31:40.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 msgr2=0x7f56940abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:40.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940abaf0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f568400c830 tx=0x7f5684008040 comp rx=0 tx=0).stop 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 shutdown_connections 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5694155af0 0x7f56941537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f569406a6a0 0x7f56940abaf0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 --2- 172.21.15.33:0/2279698395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56940c8b50 0x7f56940ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 >> 172.21.15.33:0/2279698395 conn(0x7f569406d030 msgr2=0x7f569406e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 shutdown_connections 2023-10-06T05:31:40.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:40.894+0000 7f56989fe700 1 -- 172.21.15.33:0/2279698395 wait complete. 2023-10-06T05:31:40.915 INFO:tasks.cephfs_test_runner:test_stale_read_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:31:40.915 INFO:tasks.cephfs_test_runner:Starting test: test_stale_renew (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:31:40.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_renew' 2023-10-06T05:31:41.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.103+0000 7fac3359e700 1 Processor -- start 2023-10-06T05:31:41.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.103+0000 7fac3359e700 1 -- start start 2023-10-06T05:31:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.104+0000 7fac3359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fac34155af0 0x7fac3406a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.104+0000 7fac3359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac3406a6a0 0x7fac340c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.104+0000 7fac3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac340c8b50 0x7fac340c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.104+0000 7fac3359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3406b7e0 con 0x7fac3406a6a0 2023-10-06T05:31:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.104+0000 7fac3359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3406ab60 con 0x7fac34155af0 2023-10-06T05:31:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.104+0000 7fac3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3406ae40 con 0x7fac340c8b50 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac3406a6a0 0x7fac340c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac3406a6a0 0x7fac340c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33158/0 (socket says 172.21.15.33:33158) 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 -- 172.21.15.33:0/1840812391 learned_addr learned my addr 172.21.15.33:0/1840812391 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac32d9d700 1 --2- 172.21.15.33:0/1840812391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac340c8b50 0x7fac340c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac3259c700 1 --2- 172.21.15.33:0/1840812391 >> v2:172.21.15.167:3300/0 conn(0x7fac34155af0 0x7fac3406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 -- 172.21.15.33:0/1840812391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac340c8b50 msgr2=0x7fac340c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 --2- 172.21.15.33:0/1840812391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac340c8b50 0x7fac340c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 -- 172.21.15.33:0/1840812391 >> v2:172.21.15.167:3300/0 conn(0x7fac34155af0 msgr2=0x7fac3406a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 --2- 172.21.15.33:0/1840812391 >> v2:172.21.15.167:3300/0 conn(0x7fac34155af0 0x7fac3406a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 -- 172.21.15.33:0/1840812391 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3406b120 con 0x7fac3406a6a0 2023-10-06T05:31:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.105+0000 7fac31d9b700 1 --2- 172.21.15.33:0/1840812391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac340c7210 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fac340697c0 tx=0x7fac28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabfbe0359ac2862 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3159a700 1 -- 172.21.15.33:0/1840812391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800b860 con 0x7fac3406a6a0 2023-10-06T05:31:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3159a700 1 -- 172.21.15.33:0/1840812391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2800b9c0 con 0x7fac3406a6a0 2023-10-06T05:31:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3159a700 1 -- 172.21.15.33:0/1840812391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800bd60 con 0x7fac3406a6a0 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 -- 172.21.15.33:0/1840812391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 msgr2=0x7fac340c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 --2- 172.21.15.33:0/1840812391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac340c7210 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fac340697c0 tx=0x7fac28009250 comp rx=0 tx=0).stop 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 -- 172.21.15.33:0/1840812391 shutdown_connections 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 --2- 172.21.15.33:0/1840812391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac340c8b50 0x7fac340c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 --2- 172.21.15.33:0/1840812391 >> v2:172.21.15.167:3300/0 conn(0x7fac34155af0 0x7fac3406a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 --2- 172.21.15.33:0/1840812391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac340c7210 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 -- 172.21.15.33:0/1840812391 >> 172.21.15.33:0/1840812391 conn(0x7fac3406d030 msgr2=0x7fac3406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 -- 172.21.15.33:0/1840812391 shutdown_connections 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.106+0000 7fac3359e700 1 -- 172.21.15.33:0/1840812391 wait complete. 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.108+0000 7fac3359e700 1 Processor -- start 2023-10-06T05:31:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.108+0000 7fac3359e700 1 -- start start 2023-10-06T05:31:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac34152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac340c8b50 0x7fac340c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac34155af0 0x7fac340c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340c62a0 con 0x7fac3406a6a0 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340c4950 con 0x7fac340c8b50 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac340c40b0 con 0x7fac34155af0 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac34152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac34152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33172/0 (socket says 172.21.15.33:33172) 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 -- 172.21.15.33:0/1053353494 learned_addr learned my addr 172.21.15.33:0/1053353494 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac32d9d700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac34155af0 0x7fac340c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac31d9b700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac340c8b50 0x7fac340c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 -- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac34155af0 msgr2=0x7fac340c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac34155af0 0x7fac340c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 -- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac340c8b50 msgr2=0x7fac340c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac340c8b50 0x7fac340c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.109+0000 7fac3259c700 1 -- 172.21.15.33:0/1053353494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280094f0 con 0x7fac3406a6a0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.110+0000 7fac3259c700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac34152000 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fac2400d890 tx=0x7fac24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.110+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400b600 con 0x7fac3406a6a0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.110+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2400b760 con 0x7fac3406a6a0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.110+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac340c4390 con 0x7fac3406a6a0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.110+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400bae0 con 0x7fac3406a6a0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.110+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac340ad070 con 0x7fac3406a6a0 2023-10-06T05:31:41.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.111+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3404c380 con 0x7fac3406a6a0 2023-10-06T05:31:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.112+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac2401e460 con 0x7fac3406a6a0 2023-10-06T05:31:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.113+0000 7fac227fc700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac14066f30 0x7fac140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.113+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fac24084ae0 con 0x7fac3406a6a0 2023-10-06T05:31:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.113+0000 7fac31d9b700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac14066f30 0x7fac140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.114+0000 7fac31d9b700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac14066f30 0x7fac140693e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fac340697c0 tx=0x7fac28017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.117+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac24051e50 con 0x7fac3406a6a0 2023-10-06T05:31:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.398+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_renew"]} v 0) v1 -- 0x7fac3406b400 con 0x7fac3406a6a0 2023-10-06T05:31:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.894+0000 7fac227fc700 1 -- 172.21.15.33:0/1053353494 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_renew"]}]=0 v858) v1 ==== 151+0+0 (secure 0 0 0) 0x7fac24055470 con 0x7fac3406a6a0 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac14066f30 msgr2=0x7fac140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac14066f30 0x7fac140693e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fac340697c0 tx=0x7fac28017000 comp rx=0 tx=0).stop 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 msgr2=0x7fac34152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac34152000 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fac2400d890 tx=0x7fac24008040 comp rx=0 tx=0).stop 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 shutdown_connections 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac14066f30 0x7fac140693e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac34155af0 0x7fac340c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3406a6a0 0x7fac34152000 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 --2- 172.21.15.33:0/1053353494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac340c8b50 0x7fac340c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 >> 172.21.15.33:0/1053353494 conn(0x7fac3406d030 msgr2=0x7fac340ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 shutdown_connections 2023-10-06T05:31:41.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:41.897+0000 7fac3359e700 1 -- 172.21.15.33:0/1053353494 wait complete. 2023-10-06T05:31:41.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:31:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.109+0000 7f0d3519d700 1 Processor -- start 2023-10-06T05:31:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.110+0000 7f0d3519d700 1 -- start start 2023-10-06T05:31:42.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d3519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d30156370 0x7f0d300af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d3519d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300ad430 con 0x7f0d300b01f0 2023-10-06T05:31:42.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d3519d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300afe20 con 0x7f0d300cc7e0 2023-10-06T05:31:42.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d3519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3006bc70 con 0x7f0d30156370 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d3499c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0d300b01f0 0x7f0d300ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53460/0 (socket says 172.21.15.33:53460) 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d3499c700 1 -- 172.21.15.33:0/2204465631 learned_addr learned my addr 172.21.15.33:0/2204465631 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.111+0000 7f0d2ffff700 1 --2- 172.21.15.33:0/2204465631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d30156370 0x7f0d300af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087334499 0 0) 0x7f0d300ad430 con 0x7f0d300b01f0 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d18003570 con 0x7f0d300b01f0 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424835331 0 0) 0x7f0d300afe20 con 0x7f0d300cc7e0 2023-10-06T05:31:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d300ad430 con 0x7f0d300cc7e0 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2ffff700 1 -- 172.21.15.33:0/2204465631 >> v1:172.21.15.167:6789/0 conn(0x7f0d300cc7e0 legacy=0x7f0d300c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2ffff700 1 -- 172.21.15.33:0/2204465631 >> v1:172.21.15.33:6789/0 conn(0x7f0d300b01f0 legacy=0x7f0d300ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2ffff700 1 -- 172.21.15.33:0/2204465631 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300cccc0 con 0x7f0d30156370 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.112+0000 7f0d2ffff700 1 --2- 172.21.15.33:0/2204465631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d30156370 0x7f0d300af8e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009910 tx=0x7f0d200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9146c6bb3660a61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011630 con 0x7f0d30156370 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d20011790 con 0x7f0d30156370 2023-10-06T05:31:42.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d2effd700 1 -- 172.21.15.33:0/2204465631 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011b30 con 0x7f0d30156370 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 -- 172.21.15.33:0/2204465631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d30156370 msgr2=0x7f0d300af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2204465631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d30156370 0x7f0d300af8e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009910 tx=0x7f0d200091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 -- 172.21.15.33:0/2204465631 shutdown_connections 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2204465631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d30156370 0x7f0d300af8e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 -- 172.21.15.33:0/2204465631 >> 172.21.15.33:0/2204465631 conn(0x7f0d3006d030 msgr2=0x7f0d300cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 -- 172.21.15.33:0/2204465631 shutdown_connections 2023-10-06T05:31:42.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.113+0000 7f0d3519d700 1 -- 172.21.15.33:0/2204465631 wait complete. 2023-10-06T05:31:42.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.115+0000 7f0d3519d700 1 Processor -- start 2023-10-06T05:31:42.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.115+0000 7f0d3519d700 1 -- start start 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d3519d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d300b01f0 0x7f0d3016a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d3519d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 0x7f0d30157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d3519d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d30156370 0x7f0d300ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d3519d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30157070 con 0x7f0d300cc7e0 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d3519d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3016a890 con 0x7f0d30156370 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d3519d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30157a40 con 0x7f0d300b01f0 2023-10-06T05:31:42.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d2f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 0x7f0d30157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d2f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 0x7f0d30157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33174/0 (socket says 172.21.15.33:33174) 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.116+0000 7f0d2f7fe700 1 -- 172.21.15.33:0/2389754738 learned_addr learned my addr 172.21.15.33:0/2389754738 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2ffff700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d300b01f0 0x7f0d3016a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d3499c700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d30156370 0x7f0d300ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2f7fe700 1 -- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d300b01f0 msgr2=0x7f0d3016a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2f7fe700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d300b01f0 0x7f0d3016a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2f7fe700 1 -- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d30156370 msgr2=0x7f0d300ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2f7fe700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d30156370 0x7f0d300ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2f7fe700 1 -- 172.21.15.33:0/2389754738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20009420 con 0x7f0d300cc7e0 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d2f7fe700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 0x7f0d30157500 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c7c0 tx=0x7f0d24007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:42.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24010510 con 0x7f0d300cc7e0 2023-10-06T05:31:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24010690 con 0x7f0d300cc7e0 2023-10-06T05:31:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d300cbc30 con 0x7f0d300cc7e0 2023-10-06T05:31:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24010a50 con 0x7f0d300cc7e0 2023-10-06T05:31:42.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.117+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d30156a20 con 0x7f0d300cc7e0 2023-10-06T05:31:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.118+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d301571e0 con 0x7f0d300cc7e0 2023-10-06T05:31:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.119+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d2401b400 con 0x7f0d300cc7e0 2023-10-06T05:31:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.120+0000 7f0d17fff700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d1c066f30 0x7f0d1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.120+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0d240841e0 con 0x7f0d300cc7e0 2023-10-06T05:31:42.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.121+0000 7f0d2ffff700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d1c066f30 0x7f0d1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.121+0000 7f0d2ffff700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d1c066f30 0x7f0d1c0693e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0d200091e0 tx=0x7f0d20017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.124+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d24051550 con 0x7f0d300cc7e0 2023-10-06T05:31:42.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.327+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0d3004c380 con 0x7f0d300cc7e0 2023-10-06T05:31:42.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.331+0000 7f0d17fff700 1 -- 172.21.15.33:0/2389754738 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d24054b70 con 0x7f0d300cc7e0 2023-10-06T05:31:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d1c066f30 msgr2=0x7f0d1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d1c066f30 0x7f0d1c0693e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0d200091e0 tx=0x7f0d20017000 comp rx=0 tx=0).stop 2023-10-06T05:31:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 msgr2=0x7f0d30157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 0x7f0d30157500 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c7c0 tx=0x7f0d24007de0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 shutdown_connections 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d1c066f30 0x7f0d1c0693e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d30156370 0x7f0d300ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d300b01f0 0x7f0d3016a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 --2- 172.21.15.33:0/2389754738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d300cc7e0 0x7f0d30157500 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 >> 172.21.15.33:0/2389754738 conn(0x7f0d3006d030 msgr2=0x7f0d300add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 shutdown_connections 2023-10-06T05:31:42.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.334+0000 7f0d3519d700 1 -- 172.21.15.33:0/2389754738 wait complete. 2023-10-06T05:31:42.354 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:31:42.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:31:42.433 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:31:42.518 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:31:42.518 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:31:42.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:42.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.703+0000 7f3d5ed9d700 1 Processor -- start 2023-10-06T05:31:42.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.703+0000 7f3d5ed9d700 1 -- start start 2023-10-06T05:31:42.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.704+0000 7f3d5ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.704+0000 7f3d5ed9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d580cc750 0x7f3d580b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.704+0000 7f3d5ed9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580cbd80 con 0x7f3d580cf3b0 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.704+0000 7f3d5ed9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d580af5f0 con 0x7f3d580cc750 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.704+0000 7f3d5ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580af8d0 con 0x7f3d580ccfc0 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d580cc750 0x7f3d580b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5dd9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3d580cc750 0x7f3d580b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:50112/0 (socket says 172.21.15.33:50112) 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 -- 172.21.15.33:0/3986278094 learned_addr learned my addr 172.21.15.33:0/3986278094 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:42.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5dd9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40998/0 (socket says 172.21.15.33:40998) 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 -- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 msgr2=0x7f3d580cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 -- 172.21.15.33:0/3986278094 >> v1:172.21.15.167:6789/0 conn(0x7f3d580cf3b0 legacy=0x7f3d581594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.705+0000 7f3d5e59c700 1 -- 172.21.15.33:0/3986278094 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580afbb0 con 0x7f3d580cc750 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.706+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d580b0460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009f20 tx=0x7f3d50009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679791ac815d13de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.706+0000 7f3d5cd99700 1 -- 172.21.15.33:0/3986278094 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000e070 con 0x7f3d580cc750 2023-10-06T05:31:42.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.706+0000 7f3d5cd99700 1 -- 172.21.15.33:0/3986278094 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5000b9d0 con 0x7f3d580cc750 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.706+0000 7f3d5cd99700 1 -- 172.21.15.33:0/3986278094 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500092e0 con 0x7f3d580cc750 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.706+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 msgr2=0x7f3d580b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.706+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d580b0460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009f20 tx=0x7f3d50009840 comp rx=0 tx=0).stop 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.707+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/3986278094 shutdown_connections 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.707+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d580b0460 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.707+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/3986278094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.707+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/3986278094 >> 172.21.15.33:0/3986278094 conn(0x7f3d5806cff0 msgr2=0x7f3d5806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.707+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/3986278094 shutdown_connections 2023-10-06T05:31:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.707+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/3986278094 wait complete. 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.708+0000 7f3d5ed9d700 1 Processor -- start 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.708+0000 7f3d5ed9d700 1 -- start start 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d58153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 0x7f3d580b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5ed9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580b4320 con 0x7f3d580b37e0 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5ed9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580c7a50 con 0x7f3d580cc750 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5ed9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580b3d60 con 0x7f3d580ccfc0 2023-10-06T05:31:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 0x7f3d580b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 0x7f3d580b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33178/0 (socket says 172.21.15.33:33178) 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.709+0000 7f3d5e59c700 1 -- 172.21.15.33:0/519246765 learned_addr learned my addr 172.21.15.33:0/519246765 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d58153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5d59a700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5e59c700 1 -- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 msgr2=0x7f3d580b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5e59c700 1 -- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 msgr2=0x7f3d58153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d58153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5e59c700 1 -- 172.21.15.33:0/519246765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d50009ae0 con 0x7f3d580b37e0 2023-10-06T05:31:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5e59c700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 0x7f3d580b4e80 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f3d5806be20 tx=0x7f3d50018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50008ad0 con 0x7f3d580b37e0 2023-10-06T05:31:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d50008c30 con 0x7f3d580b37e0 2023-10-06T05:31:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.710+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d580b4040 con 0x7f3d580b37e0 2023-10-06T05:31:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.711+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50026460 con 0x7f3d580b37e0 2023-10-06T05:31:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.711+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d581699b0 con 0x7f3d580b37e0 2023-10-06T05:31:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.712+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d580717a0 con 0x7f3d580b37e0 2023-10-06T05:31:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.712+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d50003040 con 0x7f3d580b37e0 2023-10-06T05:31:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.713+0000 7f3d4dffb700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d3c066f30 0x7f3d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.713+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3d50086280 con 0x7f3d580b37e0 2023-10-06T05:31:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.714+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d3c066f30 0x7f3d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.714+0000 7f3d5dd9b700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d3c066f30 0x7f3d3c0693e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009db0 tx=0x7f3d540091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.717+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d500535f0 con 0x7f3d580b37e0 2023-10-06T05:31:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.966+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d5804c380 con 0x7f3d580b37e0 2023-10-06T05:31:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.966+0000 7f3d4dffb700 1 -- 172.21.15.33:0/519246765 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+3542 (secure 0 0 0) 0x7f3d50021df0 con 0x7f3d580b37e0 2023-10-06T05:31:42.970 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:42.970 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":178,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2591296902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2591296902},{"type":"v1","addr":"172.21.15.167:6837","nonce":2591296902}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":9814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1301533698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1301533698},{"type":"v1","addr":"172.21.15.167:6836","nonce":1301533698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":10818,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2427801599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2427801599},{"type":"v1","addr":"172.21.15.33:6834","nonce":2427801599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172},{"gid":11094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1807920462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1807920462},{"type":"v1","addr":"172.21.15.33:6835","nonce":1807920462}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":178}],"filesystems":[]} 2023-10-06T05:31:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d3c066f30 msgr2=0x7f3d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d3c066f30 0x7f3d3c0693e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009db0 tx=0x7f3d540091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 msgr2=0x7f3d580b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 0x7f3d580b4e80 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f3d5806be20 tx=0x7f3d50018000 comp rx=0 tx=0).stop 2023-10-06T05:31:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 shutdown_connections 2023-10-06T05:31:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3d3c066f30 0x7f3d3c0693e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d580b37e0 0x7f3d580b4e80 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d580cc750 0x7f3d58153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 --2- 172.21.15.33:0/519246765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d580ccfc0 0x7f3d580b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 >> 172.21.15.33:0/519246765 conn(0x7f3d5806cff0 msgr2=0x7f3d58155f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 shutdown_connections 2023-10-06T05:31:42.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:42.969+0000 7f3d5ed9d700 1 -- 172.21.15.33:0/519246765 wait complete. 2023-10-06T05:31:42.977 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 178 2023-10-06T05:31:42.990 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:31:42.990 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:31:42.990 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:31:42.990 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:31:42.990 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:31:42.990 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:31:42.991 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:31:42.991 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:31:42.991 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:31:42.991 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:31:42.991 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:31:42.991 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:31:42.992 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:31:42.992 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:31:42.992 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:31:42.992 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:31:43.007 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:31:43.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:31:43.008 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:31:43.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:31:43.009 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:31:43.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:31:43.010 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:31:43.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:31:43.053 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:31:43.053 DEBUG:teuthology.parallel:result is None 2023-10-06T05:31:43.054 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:31:43.054 DEBUG:teuthology.parallel:result is None 2023-10-06T05:31:43.055 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:31:43.055 DEBUG:teuthology.parallel:result is None 2023-10-06T05:31:43.057 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:31:43.057 DEBUG:teuthology.parallel:result is None 2023-10-06T05:31:43.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:31:43.193 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:31:43.193 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:31:43.195 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:31:43.195 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:31:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.256+0000 7f4fbf59e700 1 Processor -- start 2023-10-06T05:31:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.257+0000 7f4fbf59e700 1 -- start start 2023-10-06T05:31:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.258+0000 7f4fbf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc00c8090 0x7f4fc0156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.258+0000 7f4fbf59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc006bd10 con 0x7f4fc0156880 2023-10-06T05:31:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.258+0000 7f4fbf59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc006a820 con 0x7f4fc0159e10 2023-10-06T05:31:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.258+0000 7f4fbf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00cc290 con 0x7f4fc00c8090 2023-10-06T05:31:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.258+0000 7f4fbe59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4fc0156880 0x7f4fc00c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53480/0 (socket says 172.21.15.33:53480) 2023-10-06T05:31:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.258+0000 7f4fbe59c700 1 -- 172.21.15.33:0/2260270889 learned_addr learned my addr 172.21.15.33:0/2260270889 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbdd9b700 1 --2- 172.21.15.33:0/2260270889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc00c8090 0x7f4fc0156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540865527 0 0) 0x7f4fc006bd10 con 0x7f4fc0156880 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa8003570 con 0x7f4fc0156880 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126760198 0 0) 0x7f4fc006a820 con 0x7f4fc0159e10 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc006bd10 con 0x7f4fc0159e10 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbdd9b700 1 -- 172.21.15.33:0/2260270889 >> v1:172.21.15.167:6789/0 conn(0x7f4fc0159e10 legacy=0x7f4fc0157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbdd9b700 1 -- 172.21.15.33:0/2260270889 >> v1:172.21.15.33:6789/0 conn(0x7f4fc0156880 legacy=0x7f4fc00c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247482270 0 0) 0x7f4fa8003570 con 0x7f4fc0156880 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbdd9b700 1 -- 172.21.15.33:0/2260270889 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc015a2f0 con 0x7f4fc00c8090 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.259+0000 7f4fbdd9b700 1 --2- 172.21.15.33:0/2260270889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc00c8090 0x7f4fc0156c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009910 tx=0x7f4fb40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3609521463b2f227 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0001470 con 0x7f4fc00c8090 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4011420 con 0x7f4fc00c8090 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbd59a700 1 -- 172.21.15.33:0/2260270889 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40117c0 con 0x7f4fc00c8090 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2260270889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc00c8090 msgr2=0x7f4fc0156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2260270889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc00c8090 0x7f4fc0156c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009910 tx=0x7f4fb40091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2260270889 shutdown_connections 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2260270889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc00c8090 0x7f4fc0156c50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2260270889 >> 172.21.15.33:0/2260270889 conn(0x7f4fc006c560 msgr2=0x7f4fc006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:43.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2260270889 shutdown_connections 2023-10-06T05:31:43.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.260+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2260270889 wait complete. 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.262+0000 7f4fbf59e700 1 Processor -- start 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.262+0000 7f4fbf59e700 1 -- start start 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 0x7f4fc00b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc0156880 0x7f4fc0155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbf59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc0159e10 0x7f4fc013a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbf59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0147070 con 0x7f4fc00c8090 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbf59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0146460 con 0x7f4fc0159e10 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0146740 con 0x7f4fc0156880 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 0x7f4fc00b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:43.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 0x7f4fc00b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33218/0 (socket says 172.21.15.33:33218) 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbe59c700 1 -- 172.21.15.33:0/2140653262 learned_addr learned my addr 172.21.15.33:0/2140653262 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbdd9b700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc0156880 0x7f4fc0155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.263+0000 7f4fbed9d700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc0159e10 0x7f4fc013a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbe59c700 1 -- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc0156880 msgr2=0x7f4fc0155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbe59c700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc0156880 0x7f4fc0155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbe59c700 1 -- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc0159e10 msgr2=0x7f4fc013a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbe59c700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc0159e10 0x7f4fc013a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbe59c700 1 -- 172.21.15.33:0/2140653262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4009420 con 0x7f4fc00c8090 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbe59c700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 0x7f4fc00b0a50 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000cb90 tx=0x7f4fb0001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0007150 con 0x7f4fc00c8090 2023-10-06T05:31:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb00072f0 con 0x7f4fc00c8090 2023-10-06T05:31:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0146a20 con 0x7f4fc00c8090 2023-10-06T05:31:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00076b0 con 0x7f4fc00c8090 2023-10-06T05:31:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.264+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc00b0f90 con 0x7f4fc00c8090 2023-10-06T05:31:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.265+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc004c380 con 0x7f4fc00c8090 2023-10-06T05:31:43.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.266+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fb0004a40 con 0x7f4fc00c8090 2023-10-06T05:31:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.267+0000 7f4fae7fc700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4fa0066f30 0x7f4fa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.267+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4fb0083a20 con 0x7f4fc00c8090 2023-10-06T05:31:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.268+0000 7f4fbdd9b700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4fa0066f30 0x7f4fa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:43.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.269+0000 7f4fbdd9b700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4fa0066f30 0x7f4fa00693e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40030c0 tx=0x7f4fb4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:43.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.271+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fb004c5a0 con 0x7f4fc00c8090 2023-10-06T05:31:43.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.490+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4fc00cc500 con 0x7f4fc00c8090 2023-10-06T05:31:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.911+0000 7f4fae7fc700 1 -- 172.21.15.33:0/2140653262 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v235) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4fb0018020 con 0x7f4fc00c8090 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4fa0066f30 msgr2=0x7f4fa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4fa0066f30 0x7f4fa00693e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40030c0 tx=0x7f4fb4016000 comp rx=0 tx=0).stop 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 msgr2=0x7f4fc00b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 0x7f4fc00b0a50 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000cb90 tx=0x7f4fb0001ab0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 shutdown_connections 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4fa0066f30 0x7f4fa00693e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc0159e10 0x7f4fc013a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fc00c8090 0x7f4fc00b0a50 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 --2- 172.21.15.33:0/2140653262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fc0156880 0x7f4fc0155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 >> 172.21.15.33:0/2140653262 conn(0x7f4fc006c560 msgr2=0x7f4fc006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 shutdown_connections 2023-10-06T05:31:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:43.914+0000 7f4fbf59e700 1 -- 172.21.15.33:0/2140653262 wait complete. 2023-10-06T05:31:43.918 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:31:43.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:31:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.120+0000 7feb5cb9d700 1 Processor -- start 2023-10-06T05:31:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.120+0000 7feb5cb9d700 1 -- start start 2023-10-06T05:31:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb58155510 0x7feb581558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c8f40 0x7feb580c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5806ac00 con 0x7feb580c70f0 2023-10-06T05:31:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5806b090 con 0x7feb58155510 2023-10-06T05:31:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb58155e20 con 0x7feb580c8f40 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb58155510 0x7feb581558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb58155510 0x7feb581558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33230/0 (socket says 172.21.15.33:33230) 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 -- 172.21.15.33:0/1507515451 learned_addr learned my addr 172.21.15.33:0/1507515451 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 -- 172.21.15.33:0/1507515451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c8f40 msgr2=0x7feb580c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 --2- 172.21.15.33:0/1507515451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c8f40 0x7feb580c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 -- 172.21.15.33:0/1507515451 >> v1:172.21.15.167:6789/0 conn(0x7feb580c70f0 legacy=0x7feb580c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 -- 172.21.15.33:0/1507515451 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5806ec70 con 0x7feb58155510 2023-10-06T05:31:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb577fe700 1 --2- 172.21.15.33:0/1507515451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb58155510 0x7feb581558e0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7feb480099b0 tx=0x7feb48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c05ea74e0b6cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb567fc700 1 -- 172.21.15.33:0/1507515451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800b820 con 0x7feb58155510 2023-10-06T05:31:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb567fc700 1 -- 172.21.15.33:0/1507515451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4800b980 con 0x7feb58155510 2023-10-06T05:31:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb567fc700 1 -- 172.21.15.33:0/1507515451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800bd20 con 0x7feb58155510 2023-10-06T05:31:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1507515451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb58155510 msgr2=0x7feb581558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.122+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1507515451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb58155510 0x7feb581558e0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7feb480099b0 tx=0x7feb48009250 comp rx=0 tx=0).stop 2023-10-06T05:31:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.123+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1507515451 shutdown_connections 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.123+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1507515451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb58155510 0x7feb581558e0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.123+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1507515451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c8f40 0x7feb580c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.123+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1507515451 >> 172.21.15.33:0/1507515451 conn(0x7feb5806d030 msgr2=0x7feb5806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.123+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1507515451 shutdown_connections 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.123+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1507515451 wait complete. 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.124+0000 7feb5cb9d700 1 Processor -- start 2023-10-06T05:31:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.124+0000 7feb5cb9d700 1 -- start start 2023-10-06T05:31:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb5cb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c70f0 0x7feb5816d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb5cb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 0x7feb58153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb5cb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58155510 0x7feb581431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb5cb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb58151980 con 0x7feb580c8f40 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb5cb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb58153540 con 0x7feb58155510 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb5cb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5816dba0 con 0x7feb580c70f0 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 0x7feb58153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 0x7feb58153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33238/0 (socket says 172.21.15.33:33238) 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.125+0000 7feb56ffd700 1 -- 172.21.15.33:0/1175608299 learned_addr learned my addr 172.21.15.33:0/1175608299 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb57fff700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58155510 0x7feb581431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb577fe700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c70f0 0x7feb5816d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb56ffd700 1 -- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c70f0 msgr2=0x7feb5816d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb56ffd700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c70f0 0x7feb5816d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb56ffd700 1 -- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58155510 msgr2=0x7feb581431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb56ffd700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58155510 0x7feb581431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb56ffd700 1 -- 172.21.15.33:0/1175608299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480094f0 con 0x7feb580c8f40 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb57fff700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58155510 0x7feb581431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb577fe700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c70f0 0x7feb5816d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb56ffd700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 0x7feb58153000 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00b5e0 tx=0x7feb4c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00f510 con 0x7feb580c8f40 2023-10-06T05:31:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c00f670 con 0x7feb580c8f40 2023-10-06T05:31:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00fa30 con 0x7feb580c8f40 2023-10-06T05:31:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.126+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5816de80 con 0x7feb580c8f40 2023-10-06T05:31:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.127+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5816e440 con 0x7feb580c8f40 2023-10-06T05:31:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.127+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb58151ac0 con 0x7feb580c8f40 2023-10-06T05:31:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.128+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb4c01a400 con 0x7feb580c8f40 2023-10-06T05:31:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.129+0000 7feb3f7fe700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb44066f30 0x7feb440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.129+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5206+0+0 (secure 0 0 0) 0x7feb4c083ed0 con 0x7feb580c8f40 2023-10-06T05:31:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.130+0000 7feb577fe700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb44066f30 0x7feb440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.132+0000 7feb577fe700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb44066f30 0x7feb440693e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7feb480099b0 tx=0x7feb48016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.134+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb58151ac0 con 0x7feb580c8f40 2023-10-06T05:31:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.413+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7feb58151ac0 con 0x7feb580c8f40 2023-10-06T05:31:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.413+0000 7feb3f7fe700 1 -- 172.21.15.33:0/1175608299 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) v1 ==== 80+0+5304 (secure 0 0 0) 0x7feb58151ac0 con 0x7feb580c8f40 2023-10-06T05:31:44.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:44.416 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.419 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:31:44.420 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:31:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:31:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:31:44.426 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.427 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:31:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:31:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:31:44.430 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:31:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:44.434 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:31:44.434 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb44066f30 msgr2=0x7feb440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb44066f30 0x7feb440693e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7feb480099b0 tx=0x7feb48016000 comp rx=0 tx=0).stop 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 msgr2=0x7feb58153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 0x7feb58153000 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00b5e0 tx=0x7feb4c009120 comp rx=0 tx=0).stop 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 shutdown_connections 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feb44066f30 0x7feb440693e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb58155510 0x7feb581431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb580c70f0 0x7feb5816d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 --2- 172.21.15.33:0/1175608299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb580c8f40 0x7feb58153000 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 >> 172.21.15.33:0/1175608299 conn(0x7feb5806d030 msgr2=0x7feb580af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:44.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 shutdown_connections 2023-10-06T05:31:44.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.416+0000 7feb5cb9d700 1 -- 172.21.15.33:0/1175608299 wait complete. 2023-10-06T05:31:44.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.623+0000 7f9c9159d700 1 Processor -- start 2023-10-06T05:31:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.623+0000 7f9c9159d700 1 -- start start 2023-10-06T05:31:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c9159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c9159d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c8c0c7920 0x7f9c8c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c9159d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8c06ab60 con 0x7f9c8c06a6a0 2023-10-06T05:31:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c9159d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c06b7e0 con 0x7f9c8c0c7920 2023-10-06T05:31:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c9159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c06ae40 con 0x7f9c8c155af0 2023-10-06T05:31:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c8b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9c8c06a6a0 0x7f9c8c0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53490/0 (socket says 172.21.15.33:53490) 2023-10-06T05:31:44.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c8b7fe700 1 -- 172.21.15.33:0/3936368073 learned_addr learned my addr 172.21.15.33:0/3936368073 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c8bfff700 1 --2- 172.21.15.33:0/3936368073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c90d9c700 1 --2- 172.21.15.33:0/3936368073 >> v2:172.21.15.167:3300/0 conn(0x7f9c8c0c7920 0x7f9c8c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.625+0000 7f9c8affd700 1 -- 172.21.15.33:0/3936368073 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328183346 0 0) 0x7f9c8c06ab60 con 0x7f9c8c06a6a0 2023-10-06T05:31:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8affd700 1 -- 172.21.15.33:0/3936368073 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c74003570 con 0x7f9c8c06a6a0 2023-10-06T05:31:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8affd700 1 -- 172.21.15.33:0/3936368073 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1581173738 0 0) 0x7f9c74003570 con 0x7f9c8c06a6a0 2023-10-06T05:31:44.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8bfff700 1 -- 172.21.15.33:0/3936368073 >> v2:172.21.15.167:3300/0 conn(0x7f9c8c0c7920 msgr2=0x7f9c8c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8bfff700 1 --2- 172.21.15.33:0/3936368073 >> v2:172.21.15.167:3300/0 conn(0x7f9c8c0c7920 0x7f9c8c0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8bfff700 1 -- 172.21.15.33:0/3936368073 >> v1:172.21.15.33:6789/0 conn(0x7f9c8c06a6a0 legacy=0x7f9c8c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8bfff700 1 -- 172.21.15.33:0/3936368073 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c06b120 con 0x7f9c8c155af0 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8bfff700 1 --2- 172.21.15.33:0/3936368073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c06a160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0098e0 tx=0x7f9c7c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0f5f85e6538b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.626+0000 7f9c8affd700 1 -- 172.21.15.33:0/3936368073 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80001470 con 0x7f9c8c155af0 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c8affd700 1 -- 172.21.15.33:0/3936368073 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c011420 con 0x7f9c8c155af0 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c8affd700 1 -- 172.21.15.33:0/3936368073 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0117c0 con 0x7f9c8c155af0 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 -- 172.21.15.33:0/3936368073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 msgr2=0x7f9c8c06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 --2- 172.21.15.33:0/3936368073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c06a160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0098e0 tx=0x7f9c7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 -- 172.21.15.33:0/3936368073 shutdown_connections 2023-10-06T05:31:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 --2- 172.21.15.33:0/3936368073 >> v2:172.21.15.167:3300/0 conn(0x7f9c8c0c7920 0x7f9c8c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 --2- 172.21.15.33:0/3936368073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c06a160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 -- 172.21.15.33:0/3936368073 >> 172.21.15.33:0/3936368073 conn(0x7f9c8c06d030 msgr2=0x7f9c8c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 -- 172.21.15.33:0/3936368073 shutdown_connections 2023-10-06T05:31:44.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.627+0000 7f9c9159d700 1 -- 172.21.15.33:0/3936368073 wait complete. 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.629+0000 7f9c9159d700 1 Processor -- start 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.629+0000 7f9c9159d700 1 -- start start 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c9159d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8c06a6a0 0x7f9c8c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c9159d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 0x7f9c8c14c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c9159d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c9159d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b1a70 con 0x7f9c8c0c7920 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c9159d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c16d3d0 con 0x7f9c8c06a6a0 2023-10-06T05:31:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c9159d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0ccdb0 con 0x7f9c8c155af0 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 0x7f9c8c14c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 0x7f9c8c14c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33240/0 (socket says 172.21.15.33:33240) 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c8b7fe700 1 -- 172.21.15.33:0/1692581809 learned_addr learned my addr 172.21.15.33:0/1692581809 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c90d9c700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c8bfff700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8c06a6a0 0x7f9c8c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c90d9c700 1 -- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8c06a6a0 msgr2=0x7f9c8c0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c90d9c700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8c06a6a0 0x7f9c8c0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c90d9c700 1 -- 172.21.15.33:0/1692581809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 msgr2=0x7f9c8c14c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c90d9c700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 0x7f9c8c14c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c90d9c700 1 -- 172.21.15.33:0/1692581809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c009420 con 0x7f9c8c155af0 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.630+0000 7f9c8b7fe700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 0x7f9c8c14c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.631+0000 7f9c90d9c700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c0acb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0c7d20 tx=0x7f9c8400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.631+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400caa0 con 0x7f9c8c155af0 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.631+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8400f040 con 0x7f9c8c155af0 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.631+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400ce50 con 0x7f9c8c155af0 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.631+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c0b11d0 con 0x7f9c8c155af0 2023-10-06T05:31:44.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.631+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c0b1790 con 0x7f9c8c155af0 2023-10-06T05:31:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.632+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c16d540 con 0x7f9c8c155af0 2023-10-06T05:31:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.633+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c84014690 con 0x7f9c8c155af0 2023-10-06T05:31:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.634+0000 7f9c73fff700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9c78066f30 0x7f9c780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.634+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9c84085300 con 0x7f9c8c155af0 2023-10-06T05:31:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.635+0000 7f9c8bfff700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9c78066f30 0x7f9c780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:44.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.635+0000 7f9c8bfff700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9c78066f30 0x7f9c780693e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0030c0 tx=0x7f9c7c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.638+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c840561d0 con 0x7f9c8c155af0 2023-10-06T05:31:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.839+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c8c04c380 con 0x7f9c8c155af0 2023-10-06T05:31:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.841+0000 7f9c73fff700 1 -- 172.21.15.33:0/1692581809 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f9c84055d60 con 0x7f9c8c155af0 2023-10-06T05:31:44.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:44.843 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":235,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:43.909102+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":227,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.843+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9c78066f30 msgr2=0x7f9c780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.843+0000 7f9c9159d700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9c78066f30 0x7f9c780693e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0030c0 tx=0x7f9c7c016000 comp rx=0 tx=0).stop 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.843+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 msgr2=0x7f9c8c0acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.843+0000 7f9c9159d700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c0acb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0c7d20 tx=0x7f9c8400b040 comp rx=0 tx=0).stop 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 shutdown_connections 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9c78066f30 0x7f9c780693e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c8c155af0 0x7f9c8c0acb30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c8c06a6a0 0x7f9c8c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 --2- 172.21.15.33:0/1692581809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c8c0c7920 0x7f9c8c14c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 >> 172.21.15.33:0/1692581809 conn(0x7f9c8c06d030 msgr2=0x7f9c8c0c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 shutdown_connections 2023-10-06T05:31:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:44.844+0000 7f9c9159d700 1 -- 172.21.15.33:0/1692581809 wait complete. 2023-10-06T05:31:44.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:31:44.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:31:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.055+0000 7f50b49fe700 1 Processor -- start 2023-10-06T05:31:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.055+0000 7f50b49fe700 1 -- start start 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50b49fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50b00cd5e0 0x7f50b00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50b49fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b00ce890 con 0x7f50b00af540 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50b49fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b015aa60 con 0x7f50b015a650 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50b49fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b015d890 con 0x7f50b00cd5e0 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50ae59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50b00cd5e0 0x7f50b00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50aed9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50b00af540 0x7f50b00cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53492/0 (socket says 172.21.15.33:53492) 2023-10-06T05:31:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.056+0000 7f50aed9d700 1 -- 172.21.15.33:0/3906572041 learned_addr learned my addr 172.21.15.33:0/3906572041 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50ae59c700 1 -- 172.21.15.33:0/3906572041 >> v1:172.21.15.167:6789/0 conn(0x7f50b015a650 legacy=0x7f50b00ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50ae59c700 1 -- 172.21.15.33:0/3906572041 >> v1:172.21.15.33:6789/0 conn(0x7f50b00af540 legacy=0x7f50b00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50ae59c700 1 -- 172.21.15.33:0/3906572041 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b015db70 con 0x7f50b00cd5e0 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50ae59c700 1 --2- 172.21.15.33:0/3906572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b00af910 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f50a40098e0 tx=0x7f50a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8234dd1eb6c49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50add9b700 1 -- 172.21.15.33:0/3906572041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4009e80 con 0x7f50b00cd5e0 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50add9b700 1 -- 172.21.15.33:0/3906572041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a400b820 con 0x7f50b00cd5e0 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.057+0000 7f50add9b700 1 -- 172.21.15.33:0/3906572041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400bba0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 -- 172.21.15.33:0/3906572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 msgr2=0x7f50b00af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 --2- 172.21.15.33:0/3906572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b00af910 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f50a40098e0 tx=0x7f50a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 -- 172.21.15.33:0/3906572041 shutdown_connections 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 --2- 172.21.15.33:0/3906572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b00af910 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 -- 172.21.15.33:0/3906572041 >> 172.21.15.33:0/3906572041 conn(0x7f50b006d9b0 msgr2=0x7f50b006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 -- 172.21.15.33:0/3906572041 shutdown_connections 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.058+0000 7f50b49fe700 1 -- 172.21.15.33:0/3906572041 wait complete. 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.059+0000 7f50b49fe700 1 Processor -- start 2023-10-06T05:31:45.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.060+0000 7f50b49fe700 1 -- start start 2023-10-06T05:31:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50b49fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b00af540 0x7f50b0146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50b49fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b0169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50b49fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b015a650 0x7f50b016a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50b49fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00ca1a0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50b49fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0147040 con 0x7f50b00af540 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50b49fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b016a850 con 0x7f50b015a650 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b0169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b0169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33252/0 (socket says 172.21.15.33:33252) 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 -- 172.21.15.33:0/1349110216 learned_addr learned my addr 172.21.15.33:0/1349110216 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50aed9d700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b00af540 0x7f50b0146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50af59e700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b015a650 0x7f50b016a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 -- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b015a650 msgr2=0x7f50b016a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b015a650 0x7f50b016a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 -- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b00af540 msgr2=0x7f50b0146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b00af540 0x7f50b0146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.061+0000 7f50ae59c700 1 -- 172.21.15.33:0/1349110216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a4009420 con 0x7f50b00cd5e0 2023-10-06T05:31:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.062+0000 7f50ae59c700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b0169d30 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f50a40030c0 tx=0x7f50a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.062+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400b990 con 0x7f50b00cd5e0 2023-10-06T05:31:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.062+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a4020070 con 0x7f50b00cd5e0 2023-10-06T05:31:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.062+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b016ab30 con 0x7f50b00cd5e0 2023-10-06T05:31:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.062+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a401b450 con 0x7f50b00cd5e0 2023-10-06T05:31:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.062+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b00c79e0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.063+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b00c7bd0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.064+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50a4022030 con 0x7f50b00cd5e0 2023-10-06T05:31:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.065+0000 7f5096ffd700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f509c066f30 0x7f509c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.065+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f50a4086200 con 0x7f50b00cd5e0 2023-10-06T05:31:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.065+0000 7f50aed9d700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f509c066f30 0x7f509c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.066+0000 7f50aed9d700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f509c066f30 0x7f509c0693e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f50a0001950 tx=0x7f50a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.069+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50a40535c0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.271+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50b00c7bd0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.272+0000 7f5096ffd700 1 -- 172.21.15.33:0/1349110216 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v235) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50a4056be0 con 0x7f50b00cd5e0 2023-10-06T05:31:45.273 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:31:45.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f509c066f30 msgr2=0x7f509c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f509c066f30 0x7f509c0693e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f50a0001950 tx=0x7f50a0009040 comp rx=0 tx=0).stop 2023-10-06T05:31:45.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 msgr2=0x7f50b0169d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b0169d30 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f50a40030c0 tx=0x7f50a4016000 comp rx=0 tx=0).stop 2023-10-06T05:31:45.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 shutdown_connections 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f509c066f30 0x7f509c0693e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b015a650 0x7f50b016a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b00af540 0x7f50b0146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 --2- 172.21.15.33:0/1349110216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50b00cd5e0 0x7f50b0169d30 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 >> 172.21.15.33:0/1349110216 conn(0x7f50b006d9b0 msgr2=0x7f50b015c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 shutdown_connections 2023-10-06T05:31:45.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.274+0000 7f50b49fe700 1 -- 172.21.15.33:0/1349110216 wait complete. 2023-10-06T05:31:45.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:31:45.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.483+0000 7f0cc1f9d700 1 Processor -- start 2023-10-06T05:31:45.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.483+0000 7f0cc1f9d700 1 -- start start 2023-10-06T05:31:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.484+0000 7f0cc1f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0cbc0cd5e0 0x7f0cbc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.484+0000 7f0cc1f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0cbc15a650 0x7f0cbc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.484+0000 7f0cc1f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cbc15d660 con 0x7f0cbc0af540 2023-10-06T05:31:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.484+0000 7f0cc1f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc0ce890 con 0x7f0cbc15a650 2023-10-06T05:31:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.484+0000 7f0cc1f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc15aa60 con 0x7f0cbc0cd5e0 2023-10-06T05:31:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0cbc15a650 0x7f0cbc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0cbc15a650 0x7f0cbc15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33264/0 (socket says 172.21.15.33:33264) 2023-10-06T05:31:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 -- 172.21.15.33:0/476404137 learned_addr learned my addr 172.21.15.33:0/476404137 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:45.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cbbfff700 1 --2- 172.21.15.33:0/476404137 >> v2:172.21.15.167:3300/0 conn(0x7f0cbc0cd5e0 0x7f0cbc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 -- 172.21.15.33:0/476404137 >> v2:172.21.15.167:3300/0 conn(0x7f0cbc0cd5e0 msgr2=0x7f0cbc0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 --2- 172.21.15.33:0/476404137 >> v2:172.21.15.167:3300/0 conn(0x7f0cbc0cd5e0 0x7f0cbc0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 -- 172.21.15.33:0/476404137 >> v1:172.21.15.167:6789/0 conn(0x7f0cbc0af540 legacy=0x7f0cbc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 -- 172.21.15.33:0/476404137 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc15d890 con 0x7f0cbc15a650 2023-10-06T05:31:45.489 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.485+0000 7f0cc179c700 1 --2- 172.21.15.33:0/476404137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc15a650 0x7f0cbc15d120 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0893f0 tx=0x7f0cb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b49f60a3c48dc2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cbb7fe700 1 -- 172.21.15.33:0/476404137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400e070 con 0x7f0cbc15a650 2023-10-06T05:31:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cbb7fe700 1 -- 172.21.15.33:0/476404137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb400ba20 con 0x7f0cbc15a650 2023-10-06T05:31:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cbb7fe700 1 -- 172.21.15.33:0/476404137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40092e0 con 0x7f0cbc15a650 2023-10-06T05:31:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/476404137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc15a650 msgr2=0x7f0cbc15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/476404137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc15a650 0x7f0cbc15d120 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0893f0 tx=0x7f0cb4009840 comp rx=0 tx=0).stop 2023-10-06T05:31:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/476404137 shutdown_connections 2023-10-06T05:31:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/476404137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc15a650 0x7f0cbc15d120 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/476404137 >> v2:172.21.15.167:3300/0 conn(0x7f0cbc0cd5e0 0x7f0cbc0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/476404137 >> 172.21.15.33:0/476404137 conn(0x7f0cbc06d9b0 msgr2=0x7f0cbc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/476404137 shutdown_connections 2023-10-06T05:31:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.486+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/476404137 wait complete. 2023-10-06T05:31:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.488+0000 7f0cc1f9d700 1 Processor -- start 2023-10-06T05:31:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.488+0000 7f0cc1f9d700 1 -- start start 2023-10-06T05:31:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc1f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 0x7f0cbc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc1f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cbc0cd5e0 0x7f0cbc14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc1f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cbc15a650 0x7f0cbc0ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc1f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc16a790 con 0x7f0cbc0af540 2023-10-06T05:31:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc1f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc14c580 con 0x7f0cbc0cd5e0 2023-10-06T05:31:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc1f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0ca6c0 con 0x7f0cbc15a650 2023-10-06T05:31:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc0f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 0x7f0cbc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc0f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 0x7f0cbc0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33280/0 (socket says 172.21.15.33:33280) 2023-10-06T05:31:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc0f9b700 1 -- 172.21.15.33:0/1454547443 learned_addr learned my addr 172.21.15.33:0/1454547443 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cbbfff700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cbc0cd5e0 0x7f0cbc14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.489+0000 7f0cc179c700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cbc15a650 0x7f0cbc0ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc0f9b700 1 -- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cbc15a650 msgr2=0x7f0cbc0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc0f9b700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cbc15a650 0x7f0cbc0ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc0f9b700 1 -- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cbc0cd5e0 msgr2=0x7f0cbc14c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc0f9b700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cbc0cd5e0 0x7f0cbc14c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc0f9b700 1 -- 172.21.15.33:0/1454547443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4009ae0 con 0x7f0cbc0af540 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc0f9b700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 0x7f0cbc0b4650 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000c4e0 tx=0x7f0cb000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0011590 con 0x7f0cbc0af540 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0011710 con 0x7f0cbc0af540 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc169b80 con 0x7f0cbc0af540 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0011ad0 con 0x7f0cbc0af540 2023-10-06T05:31:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.490+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc16a140 con 0x7f0cbc0af540 2023-10-06T05:31:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.491+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc04c380 con 0x7f0cbc0af540 2023-10-06T05:31:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.492+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cb001b3b0 con 0x7f0cbc0af540 2023-10-06T05:31:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.493+0000 7f0cb8ff9700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca0066f30 0x7f0ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.493+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0cb0083910 con 0x7f0cbc0af540 2023-10-06T05:31:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.494+0000 7f0cbbfff700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca0066f30 0x7f0ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.494+0000 7f0cbbfff700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca0066f30 0x7f0ca00693e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc159610 tx=0x7f0ca8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.497+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cb0050cd0 con 0x7f0cbc0af540 2023-10-06T05:31:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.704+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0cbc15db70 con 0x7f0cbc0af540 2023-10-06T05:31:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.704+0000 7f0cb8ff9700 1 -- 172.21.15.33:0/1454547443 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v235) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0cb0017070 con 0x7f0cbc0af540 2023-10-06T05:31:45.706 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca0066f30 msgr2=0x7f0ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca0066f30 0x7f0ca00693e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc159610 tx=0x7f0ca8008040 comp rx=0 tx=0).stop 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 msgr2=0x7f0cbc0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 0x7f0cbc0b4650 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000c4e0 tx=0x7f0cb000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 shutdown_connections 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca0066f30 0x7f0ca00693e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cbc15a650 0x7f0cbc0ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cbc0af540 0x7f0cbc0b4650 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 --2- 172.21.15.33:0/1454547443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cbc0cd5e0 0x7f0cbc14c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 >> 172.21.15.33:0/1454547443 conn(0x7f0cbc06d9b0 msgr2=0x7f0cbc0c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 shutdown_connections 2023-10-06T05:31:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.707+0000 7f0cc1f9d700 1 -- 172.21.15.33:0/1454547443 wait complete. 2023-10-06T05:31:45.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:31:45.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.912+0000 7fc7da19d700 1 Processor -- start 2023-10-06T05:31:45.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.912+0000 7fc7da19d700 1 -- start start 2023-10-06T05:31:45.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.913+0000 7fc7da19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7d40ccfc0 0x7fc7d40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.913+0000 7fc7da19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40cf3b0 0x7fc7d41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.913+0000 7fc7da19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d40cbd80 con 0x7fc7d40cc750 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.913+0000 7fc7da19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d40af5f0 con 0x7fc7d40ccfc0 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.913+0000 7fc7da19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40af8d0 con 0x7fc7d40cf3b0 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7d40ccfc0 0x7fc7d40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d999c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc7d40cc750 0x7fc7d40b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53494/0 (socket says 172.21.15.33:53494) 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d999c700 1 -- 172.21.15.33:0/2731191357 learned_addr learned my addr 172.21.15.33:0/2731191357 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7d40ccfc0 0x7fc7d40cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33296/0 (socket says 172.21.15.33:33296) 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 -- 172.21.15.33:0/2731191357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40cf3b0 msgr2=0x7fc7d41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d899a700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40cf3b0 0x7fc7d41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40cf3b0 0x7fc7d41594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 -- 172.21.15.33:0/2731191357 >> v1:172.21.15.33:6789/0 conn(0x7fc7d40cc750 legacy=0x7fc7d40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 -- 172.21.15.33:0/2731191357 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40afbb0 con 0x7fc7d40ccfc0 2023-10-06T05:31:45.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d899a700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40cf3b0 0x7fc7d41594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:45.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.914+0000 7fc7d919b700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40ccfc0 0x7fc7d40cc210 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00098e0 tx=0x7fc7d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4423609dd05185 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.915+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2731191357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0009e80 con 0x7fc7d40ccfc0 2023-10-06T05:31:45.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.915+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2731191357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d000b860 con 0x7fc7d40ccfc0 2023-10-06T05:31:45.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.915+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2731191357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000bbe0 con 0x7fc7d40ccfc0 2023-10-06T05:31:45.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.915+0000 7fc7da19d700 1 -- 172.21.15.33:0/2731191357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40ccfc0 msgr2=0x7fc7d40cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.915+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40ccfc0 0x7fc7d40cc210 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00098e0 tx=0x7fc7d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.916+0000 7fc7da19d700 1 -- 172.21.15.33:0/2731191357 shutdown_connections 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.916+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40ccfc0 0x7fc7d40cc210 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.916+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2731191357 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40cf3b0 0x7fc7d41594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.916+0000 7fc7da19d700 1 -- 172.21.15.33:0/2731191357 >> 172.21.15.33:0/2731191357 conn(0x7fc7d406cff0 msgr2=0x7fc7d406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.916+0000 7fc7da19d700 1 -- 172.21.15.33:0/2731191357 shutdown_connections 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.916+0000 7fc7da19d700 1 -- 172.21.15.33:0/2731191357 wait complete. 2023-10-06T05:31:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.917+0000 7fc7da19d700 1 Processor -- start 2023-10-06T05:31:45.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.917+0000 7fc7da19d700 1 -- start start 2023-10-06T05:31:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.918+0000 7fc7da19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d40cc750 0x7fc7d414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.918+0000 7fc7da19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40ccfc0 0x7fc7d40c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.918+0000 7fc7da19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 0x7fc7d40ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.918+0000 7fc7da19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c9620 con 0x7fc7d40c8500 2023-10-06T05:31:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.918+0000 7fc7da19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c8a10 con 0x7fc7d40cc750 2023-10-06T05:31:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.918+0000 7fc7da19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c8cf0 con 0x7fc7d40ccfc0 2023-10-06T05:31:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 0x7fc7d40ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d899a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40ccfc0 0x7fc7d40c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 0x7fc7d40ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33298/0 (socket says 172.21.15.33:33298) 2023-10-06T05:31:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 -- 172.21.15.33:0/2249165169 learned_addr learned my addr 172.21.15.33:0/2249165169 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d919b700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d40cc750 0x7fc7d414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 -- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40ccfc0 msgr2=0x7fc7d40c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40ccfc0 0x7fc7d40c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 -- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d40cc750 msgr2=0x7fc7d414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d40cc750 0x7fc7d414cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 -- 172.21.15.33:0/2249165169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d0009420 con 0x7fc7d40c8500 2023-10-06T05:31:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7d999c700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 0x7fc7d40ca8f0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00cbc0 tx=0x7fc7cc001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.919+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00a7a0 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.920+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc00a900 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.920+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00ac80 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.920+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d40c8fd0 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.920+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d414d4b0 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.921+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d40b3860 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.922+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7cc007300 con 0x7fc7d40c8500 2023-10-06T05:31:45.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.922+0000 7fc7c97fa700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7b8066f30 0x7fc7b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.923+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc7cc084640 con 0x7fc7d40c8500 2023-10-06T05:31:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.923+0000 7fc7d919b700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7b8066f30 0x7fc7b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.924+0000 7fc7d919b700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7b8066f30 0x7fc7b80693e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00091e0 tx=0x7fc7d0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:45.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:45.927+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7cc0519f0 con 0x7fc7d40c8500 2023-10-06T05:31:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.131+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc7d404c380 con 0x7fc7d40c8500 2023-10-06T05:31:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.132+0000 7fc7c97fa700 1 -- 172.21.15.33:0/2249165169 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v235) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc7cc055010 con 0x7fc7d40c8500 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:31:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:31:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:31:46.135 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:31:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7b8066f30 msgr2=0x7fc7b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7b8066f30 0x7fc7b80693e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00091e0 tx=0x7fc7d0017000 comp rx=0 tx=0).stop 2023-10-06T05:31:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 msgr2=0x7fc7d40ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 0x7fc7d40ca8f0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00cbc0 tx=0x7fc7cc001990 comp rx=0 tx=0).stop 2023-10-06T05:31:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 shutdown_connections 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7b8066f30 0x7fc7b80693e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d40c8500 0x7fc7d40ca8f0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d40cc750 0x7fc7d414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 --2- 172.21.15.33:0/2249165169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d40ccfc0 0x7fc7d40c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 >> 172.21.15.33:0/2249165169 conn(0x7fc7d406cff0 msgr2=0x7fc7d414fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 shutdown_connections 2023-10-06T05:31:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.134+0000 7fc7da19d700 1 -- 172.21.15.33:0/2249165169 wait complete. 2023-10-06T05:31:46.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:31:46.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:31:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.343+0000 7fe9c6d9d700 1 Processor -- start 2023-10-06T05:31:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.344+0000 7fe9c6d9d700 1 -- start start 2023-10-06T05:31:46.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c6d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c00ae620 0x7fe9c00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:46.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c6d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe9c01576f0 0x7fe9c00b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:46.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c6d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00b0840 con 0x7fe9c0157320 2023-10-06T05:31:46.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c6d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c01585c0 con 0x7fe9c01576f0 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c6d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00b1ff0 con 0x7fe9c00ae620 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c5d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe9c0157320 0x7fe9c00adf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53500/0 (socket says 172.21.15.33:53500) 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c5d9b700 1 -- 172.21.15.33:0/2238666914 learned_addr learned my addr 172.21.15.33:0/2238666914 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c659c700 1 --2- 172.21.15.33:0/2238666914 >> v2:172.21.15.33:3300/0 conn(0x7fe9c01576f0 0x7fe9c00b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.345+0000 7fe9c559a700 1 --2- 172.21.15.33:0/2238666914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c00ae620 0x7fe9c00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c659c700 1 -- 172.21.15.33:0/2238666914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c00ae620 msgr2=0x7fe9c00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c659c700 1 --2- 172.21.15.33:0/2238666914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c00ae620 0x7fe9c00b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c659c700 1 -- 172.21.15.33:0/2238666914 >> v1:172.21.15.33:6789/0 conn(0x7fe9c0157320 legacy=0x7fe9c00adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c659c700 1 -- 172.21.15.33:0/2238666914 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0157c90 con 0x7fe9c01576f0 2023-10-06T05:31:46.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c659c700 1 --2- 172.21.15.33:0/2238666914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c01576f0 0x7fe9c00b1910 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009e60 tx=0x7fe9b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d481ce26da06f6be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:46.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c4d99700 1 -- 172.21.15.33:0/2238666914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800e070 con 0x7fe9c01576f0 2023-10-06T05:31:46.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c4d99700 1 -- 172.21.15.33:0/2238666914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b800bcc0 con 0x7fe9c01576f0 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.346+0000 7fe9c4d99700 1 -- 172.21.15.33:0/2238666914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8012680 con 0x7fe9c01576f0 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2238666914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c01576f0 msgr2=0x7fe9c00b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2238666914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c01576f0 0x7fe9c00b1910 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009e60 tx=0x7fe9b8009780 comp rx=0 tx=0).stop 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2238666914 shutdown_connections 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2238666914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c01576f0 0x7fe9c00b1910 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2238666914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c00ae620 0x7fe9c00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2238666914 >> 172.21.15.33:0/2238666914 conn(0x7fe9c0068090 msgr2=0x7fe9c006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2238666914 shutdown_connections 2023-10-06T05:31:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.347+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2238666914 wait complete. 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.348+0000 7fe9c6d9d700 1 Processor -- start 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 -- start start 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 0x7fe9c014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c0157320 0x7fe9c006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c01576f0 0x7fe9c014a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c014dcd0 con 0x7fe9c00ae620 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0156990 con 0x7fe9c0157320 2023-10-06T05:31:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.349+0000 7fe9c6d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c014da00 con 0x7fe9c01576f0 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 0x7fe9c014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 0x7fe9c014d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33328/0 (socket says 172.21.15.33:33328) 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 -- 172.21.15.33:0/2757964984 learned_addr learned my addr 172.21.15.33:0/2757964984 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c659c700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c01576f0 0x7fe9c014a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c559a700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c0157320 0x7fe9c006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 -- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c01576f0 msgr2=0x7fe9c014a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c01576f0 0x7fe9c014a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 -- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c0157320 msgr2=0x7fe9c006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c0157320 0x7fe9c006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 -- 172.21.15.33:0/2757964984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8009a20 con 0x7fe9c00ae620 2023-10-06T05:31:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.350+0000 7fe9c5d9b700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 0x7fe9c014d4c0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c7c0 tx=0x7fe9bc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.351+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc005b30 con 0x7fe9c00ae620 2023-10-06T05:31:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.351+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc005c90 con 0x7fe9c00ae620 2023-10-06T05:31:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.351+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c0153810 con 0x7fe9c00ae620 2023-10-06T05:31:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.351+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc004a80 con 0x7fe9c00ae620 2023-10-06T05:31:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.351+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c0154050 con 0x7fe9c00ae620 2023-10-06T05:31:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.352+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0154210 con 0x7fe9c00ae620 2023-10-06T05:31:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.353+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9bc0070d0 con 0x7fe9c00ae620 2023-10-06T05:31:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.354+0000 7fe9b5ffb700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe9a4066f30 0x7fe9a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.354+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe9bc0836d0 con 0x7fe9c00ae620 2023-10-06T05:31:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.355+0000 7fe9c559a700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe9a4066f30 0x7fe9a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:46.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.355+0000 7fe9c559a700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe9a4066f30 0x7fe9a40693e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe9c006ee00 tx=0x7fe9b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:46.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.358+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9bc04b5a0 con 0x7fe9c00ae620 2023-10-06T05:31:46.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.561+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe9c0063ff0 con 0x7fe9c00ae620 2023-10-06T05:31:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.925+0000 7fe9b5ffb700 1 -- 172.21.15.33:0/2757964984 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v236) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe9bc050a90 con 0x7fe9c00ae620 2023-10-06T05:31:46.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.928+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe9a4066f30 msgr2=0x7fe9a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.928+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe9a4066f30 0x7fe9a40693e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe9c006ee00 tx=0x7fe9b0008040 comp rx=0 tx=0).stop 2023-10-06T05:31:46.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.928+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 msgr2=0x7fe9c014d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:46.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.928+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 0x7fe9c014d4c0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c7c0 tx=0x7fe9bc007f70 comp rx=0 tx=0).stop 2023-10-06T05:31:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 shutdown_connections 2023-10-06T05:31:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe9a4066f30 0x7fe9a40693e0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9c01576f0 0x7fe9c014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c00ae620 0x7fe9c014d4c0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 --2- 172.21.15.33:0/2757964984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c0157320 0x7fe9c006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 >> 172.21.15.33:0/2757964984 conn(0x7fe9c0068090 msgr2=0x7fe9c00ca3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 shutdown_connections 2023-10-06T05:31:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:46.929+0000 7fe9c6d9d700 1 -- 172.21.15.33:0/2757964984 wait complete. 2023-10-06T05:31:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:31:46.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:31:47.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.144+0000 7f2d93fff700 1 Processor -- start 2023-10-06T05:31:47.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.144+0000 7f2d93fff700 1 -- start start 2023-10-06T05:31:47.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.145+0000 7f2d93fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2d941594c0 0x7f2d9406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:47.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.145+0000 7f2d93fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d9406a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:47.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.145+0000 7f2d93fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9415dbe0 con 0x7f2d94159890 2023-10-06T05:31:47.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.145+0000 7f2d93fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d940c9e30 con 0x7f2d941594c0 2023-10-06T05:31:47.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.145+0000 7f2d93fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940ca110 con 0x7f2d940c95c0 2023-10-06T05:31:47.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.146+0000 7f2d937fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2d94159890 0x7f2d9406c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:53506/0 (socket says 172.21.15.33:53506) 2023-10-06T05:31:47.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.146+0000 7f2d937fe700 1 -- 172.21.15.33:0/1637435850 learned_addr learned my addr 172.21.15.33:0/1637435850 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:47.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.146+0000 7f2d927fc700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d9406a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.146+0000 7f2d92ffd700 1 --2- 172.21.15.33:0/1637435850 >> v2:172.21.15.167:3300/0 conn(0x7f2d941594c0 0x7f2d9406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.146+0000 7f2d91ffb700 1 -- 172.21.15.33:0/1637435850 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64392457 0 0) 0x7f2d9415dbe0 con 0x7f2d94159890 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.146+0000 7f2d91ffb700 1 -- 172.21.15.33:0/1637435850 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d7c003570 con 0x7f2d94159890 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d92ffd700 1 -- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 msgr2=0x7f2d9406a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d92ffd700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d9406a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d92ffd700 1 -- 172.21.15.33:0/1637435850 >> v1:172.21.15.167:6789/0 conn(0x7f2d94159890 legacy=0x7f2d9406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d92ffd700 1 -- 172.21.15.33:0/1637435850 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9415de30 con 0x7f2d941594c0 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d927fc700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d9406a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d92ffd700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d941594c0 0x7f2d9406a310 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2d840098e0 tx=0x7f2d840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fb8d920828ec0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:47.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d91ffb700 1 -- 172.21.15.33:0/1637435850 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011630 con 0x7f2d941594c0 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d91ffb700 1 -- 172.21.15.33:0/1637435850 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84011790 con 0x7f2d941594c0 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.147+0000 7f2d91ffb700 1 -- 172.21.15.33:0/1637435850 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011b30 con 0x7f2d941594c0 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 -- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d941594c0 msgr2=0x7f2d9406a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d941594c0 0x7f2d9406a310 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2d840098e0 tx=0x7f2d840091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 -- 172.21.15.33:0/1637435850 shutdown_connections 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d941594c0 0x7f2d9406a310 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 --2- 172.21.15.33:0/1637435850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d9406a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 -- 172.21.15.33:0/1637435850 >> 172.21.15.33:0/1637435850 conn(0x7f2d940cd3b0 msgr2=0x7f2d940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 -- 172.21.15.33:0/1637435850 shutdown_connections 2023-10-06T05:31:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.148+0000 7f2d93fff700 1 -- 172.21.15.33:0/1637435850 wait complete. 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.149+0000 7f2d93fff700 1 Processor -- start 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.149+0000 7f2d93fff700 1 -- start start 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.150+0000 7f2d93fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d94148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.150+0000 7f2d93fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 0x7f2d9414afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.150+0000 7f2d93fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d94159890 0x7f2d9414ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.150+0000 7f2d93fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c6de0 con 0x7f2d941594c0 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.150+0000 7f2d93fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9414a980 con 0x7f2d94159890 2023-10-06T05:31:47.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.150+0000 7f2d93fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9414b520 con 0x7f2d940c95c0 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 0x7f2d9414afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 0x7f2d9414afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33332/0 (socket says 172.21.15.33:33332) 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 -- 172.21.15.33:0/3128657926 learned_addr learned my addr 172.21.15.33:0/3128657926 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d937fe700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d94159890 0x7f2d9414ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d92ffd700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d94148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 -- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 msgr2=0x7f2d94148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d94148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 -- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d94159890 msgr2=0x7f2d9414ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d94159890 0x7f2d9414ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 -- 172.21.15.33:0/3128657926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d88009420 con 0x7f2d941594c0 2023-10-06T05:31:47.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.151+0000 7f2d927fc700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 0x7f2d9414afe0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800ecb0 tx=0x7f2d880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:47.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.152+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800c920 con 0x7f2d941594c0 2023-10-06T05:31:47.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.152+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8800ca80 con 0x7f2d941594c0 2023-10-06T05:31:47.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.152+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800ce00 con 0x7f2d941594c0 2023-10-06T05:31:47.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.152+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d84009420 con 0x7f2d941594c0 2023-10-06T05:31:47.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.152+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d94150880 con 0x7f2d941594c0 2023-10-06T05:31:47.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.153+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d940b1c00 con 0x7f2d941594c0 2023-10-06T05:31:47.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.154+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d88013680 con 0x7f2d941594c0 2023-10-06T05:31:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.155+0000 7f2d7affd700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d80066f30 0x7f2d800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.155+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f2d88084fb0 con 0x7f2d941594c0 2023-10-06T05:31:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.156+0000 7f2d92ffd700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d80066f30 0x7f2d800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:47.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.157+0000 7f2d92ffd700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d80066f30 0x7f2d800693e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2d840091e0 tx=0x7f2d84017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.159+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d88052280 con 0x7f2d941594c0 2023-10-06T05:31:47.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.362+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2d9404c380 con 0x7f2d941594c0 2023-10-06T05:31:47.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.934+0000 7f2d7affd700 1 -- 172.21.15.33:0/3128657926 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v237) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2d880558a0 con 0x7f2d941594c0 2023-10-06T05:31:47.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.945+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d80066f30 msgr2=0x7f2d800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:47.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.945+0000 7f2d93fff700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d80066f30 0x7f2d800693e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2d840091e0 tx=0x7f2d84017000 comp rx=0 tx=0).stop 2023-10-06T05:31:47.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.945+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 msgr2=0x7f2d9414afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.945+0000 7f2d93fff700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 0x7f2d9414afe0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800ecb0 tx=0x7f2d880091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.946+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 shutdown_connections 2023-10-06T05:31:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.947+0000 7f2d93fff700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2d80066f30 0x7f2d800693e0 secure :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2d840091e0 tx=0x7f2d84017000 comp rx=0 tx=0).stop 2023-10-06T05:31:47.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.947+0000 7f2d93fff700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d94159890 0x7f2d9414ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.947+0000 7f2d93fff700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d940c95c0 0x7f2d94148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.947+0000 7f2d93fff700 1 --2- 172.21.15.33:0/3128657926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d941594c0 0x7f2d9414afe0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:47.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.947+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 >> 172.21.15.33:0/3128657926 conn(0x7f2d940cd3b0 msgr2=0x7f2d9406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:47.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.950+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 shutdown_connections 2023-10-06T05:31:47.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:47.950+0000 7f2d93fff700 1 -- 172.21.15.33:0/3128657926 wait complete. 2023-10-06T05:31:47.954 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:31:47.974 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:31:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.165+0000 7f84a679d700 1 Processor -- start 2023-10-06T05:31:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.165+0000 7f84a679d700 1 -- start start 2023-10-06T05:31:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.166+0000 7f84a679d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f84a00b1f90 0x7f84a00abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.166+0000 7f84a679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 0x7f84a0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.166+0000 7f84a679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a006ae10 con 0x7f84a0070440 2023-10-06T05:31:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.166+0000 7f84a679d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a006a2d0 con 0x7f84a00b1f90 2023-10-06T05:31:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.166+0000 7f84a679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a0154290 con 0x7f84a0068310 2023-10-06T05:31:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 0x7f84a0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 0x7f84a0159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41178/0 (socket says 172.21.15.33:41178) 2023-10-06T05:31:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a4f9a700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f84a0070440 0x7f84a0067c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:53514/0 (socket says 172.21.15.33:53514) 2023-10-06T05:31:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 -- 172.21.15.33:0/965181565 learned_addr learned my addr 172.21.15.33:0/965181565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a579b700 1 --2- 172.21.15.33:0/965181565 >> v2:172.21.15.167:3300/0 conn(0x7f84a00b1f90 0x7f84a00abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 -- 172.21.15.33:0/965181565 >> v2:172.21.15.167:3300/0 conn(0x7f84a00b1f90 msgr2=0x7f84a00abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 --2- 172.21.15.33:0/965181565 >> v2:172.21.15.167:3300/0 conn(0x7f84a00b1f90 0x7f84a00abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 -- 172.21.15.33:0/965181565 >> v1:172.21.15.167:6789/0 conn(0x7f84a0070440 legacy=0x7f84a0067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f84a5f9c700 1 -- 172.21.15.33:0/965181565 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a006a4e0 con 0x7f84a0068310 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.167+0000 7f8497fff700 1 -- 172.21.15.33:0/965181565 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795812186 0 0) 0x7f84a006ae10 con 0x7f84a0070440 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f84a5f9c700 1 --2- 172.21.15.33:0/965181565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 0x7f84a0159dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8498009f20 tx=0x7f8498009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af765969c7f9d7d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f8497fff700 1 -- 172.21.15.33:0/965181565 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800e070 con 0x7f84a0068310 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f8497fff700 1 -- 172.21.15.33:0/965181565 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849800b9d0 con 0x7f84a0068310 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f8497fff700 1 -- 172.21.15.33:0/965181565 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980092e0 con 0x7f84a0068310 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f84a679d700 1 -- 172.21.15.33:0/965181565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 msgr2=0x7f84a0159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f84a679d700 1 --2- 172.21.15.33:0/965181565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 0x7f84a0159dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8498009f20 tx=0x7f8498009840 comp rx=0 tx=0).stop 2023-10-06T05:31:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f84a679d700 1 -- 172.21.15.33:0/965181565 shutdown_connections 2023-10-06T05:31:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f84a679d700 1 --2- 172.21.15.33:0/965181565 >> v2:172.21.15.167:3300/0 conn(0x7f84a00b1f90 0x7f84a00abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.168+0000 7f84a679d700 1 --2- 172.21.15.33:0/965181565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0068310 0x7f84a0159dd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.169+0000 7f84a679d700 1 -- 172.21.15.33:0/965181565 >> 172.21.15.33:0/965181565 conn(0x7f84a006ea70 msgr2=0x7f84a006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.169+0000 7f84a679d700 1 -- 172.21.15.33:0/965181565 shutdown_connections 2023-10-06T05:31:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.169+0000 7f84a679d700 1 -- 172.21.15.33:0/965181565 wait complete. 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.170+0000 7f84a679d700 1 Processor -- start 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.170+0000 7f84a679d700 1 -- start start 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0068310 0x7f84a013d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0070440 0x7f84a015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 0x7f84a015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a013d9d0 con 0x7f84a00b1f90 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a013df90 con 0x7f84a0068310 2023-10-06T05:31:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a015b370 con 0x7f84a0070440 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a5f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 0x7f84a015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a5f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 0x7f84a015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33340/0 (socket says 172.21.15.33:33340) 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.171+0000 7f84a5f9c700 1 -- 172.21.15.33:0/1761058781 learned_addr learned my addr 172.21.15.33:0/1761058781 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a4f9a700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0070440 0x7f84a015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a579b700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0068310 0x7f84a013d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a5f9c700 1 -- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0070440 msgr2=0x7f84a015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a5f9c700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0070440 0x7f84a015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a5f9c700 1 -- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0068310 msgr2=0x7f84a013d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a5f9c700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0068310 0x7f84a013d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a5f9c700 1 -- 172.21.15.33:0/1761058781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498009ae0 con 0x7f84a00b1f90 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a579b700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0068310 0x7f84a013d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:48.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84a5f9c700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 0x7f84a015ab40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f84a006a410 tx=0x7f8498018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498008c30 con 0x7f84a00b1f90 2023-10-06T05:31:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.172+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8498008d90 con 0x7f84a00b1f90 2023-10-06T05:31:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.173+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a015b650 con 0x7f84a00b1f90 2023-10-06T05:31:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.173+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980266b0 con 0x7f84a00b1f90 2023-10-06T05:31:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.173+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a015b080 con 0x7f84a00b1f90 2023-10-06T05:31:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.174+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a013e120 con 0x7f84a00b1f90 2023-10-06T05:31:48.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.175+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8498003040 con 0x7f84a00b1f90 2023-10-06T05:31:48.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.176+0000 7f84957fa700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8484066f30 0x7f84840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:48.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.176+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f8498086610 con 0x7f84a00b1f90 2023-10-06T05:31:48.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.176+0000 7f84a579b700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8484066f30 0x7f84840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:48.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.177+0000 7f84a579b700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8484066f30 0x7f84840693e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f849c006f00 tx=0x7f849c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:48.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.181+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8498053790 con 0x7f84a00b1f90 2023-10-06T05:31:48.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.428+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f84a004c380 con 0x7f84a00b1f90 2023-10-06T05:31:48.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.946+0000 7f84957fa700 1 -- 172.21.15.33:0/1761058781 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 35 and data pool 36 v181) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8498056db0 con 0x7f84a00b1f90 2023-10-06T05:31:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.950+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8484066f30 msgr2=0x7f84840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.950+0000 7f84a679d700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8484066f30 0x7f84840693e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f849c006f00 tx=0x7f849c008040 comp rx=0 tx=0).stop 2023-10-06T05:31:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.950+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 msgr2=0x7f84a015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.950+0000 7f84a679d700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 0x7f84a015ab40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f84a006a410 tx=0x7f8498018000 comp rx=0 tx=0).stop 2023-10-06T05:31:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.950+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 shutdown_connections 2023-10-06T05:31:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.951+0000 7f84a679d700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8484066f30 0x7f84840693e0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.951+0000 7f84a679d700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a00b1f90 0x7f84a015ab40 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.951+0000 7f84a679d700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a0068310 0x7f84a013d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.951+0000 7f84a679d700 1 --2- 172.21.15.33:0/1761058781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f84a0070440 0x7f84a015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.951+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 >> 172.21.15.33:0/1761058781 conn(0x7f84a006ea70 msgr2=0x7f84a00ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.952+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 shutdown_connections 2023-10-06T05:31:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:48.952+0000 7f84a679d700 1 -- 172.21.15.33:0/1761058781 wait complete. 2023-10-06T05:31:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 35 and data pool 36 2023-10-06T05:31:48.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:49.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.164+0000 7faa0b39d700 1 Processor -- start 2023-10-06T05:31:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.164+0000 7faa0b39d700 1 -- start start 2023-10-06T05:31:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.165+0000 7faa0b39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faa04159530 0x7faa0406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.165+0000 7faa0b39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0406e3e0 con 0x7faa0406df70 2023-10-06T05:31:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.165+0000 7faa0b39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0406f220 con 0x7faa040b04e0 2023-10-06T05:31:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.165+0000 7faa0b39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040b1e00 con 0x7faa04159530 2023-10-06T05:31:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.165+0000 7faa09b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faa0406df70 0x7faa040afdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53506/0 (socket says 172.21.15.33:53506) 2023-10-06T05:31:49.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.165+0000 7faa09b9a700 1 -- 172.21.15.33:0/4011330313 learned_addr learned my addr 172.21.15.33:0/4011330313 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:49.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa0a39b700 1 --2- 172.21.15.33:0/4011330313 >> v2:172.21.15.167:3300/0 conn(0x7faa04159530 0x7faa0406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216687788 0 0) 0x7faa0406e3e0 con 0x7faa0406df70 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f0003570 con 0x7faa0406df70 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720102011 0 0) 0x7faa0406f220 con 0x7faa040b04e0 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0406e3e0 con 0x7faa040b04e0 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165870670 0 0) 0x7fa9f0003570 con 0x7faa0406df70 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 >> v2:172.21.15.167:3300/0 conn(0x7faa04159530 msgr2=0x7faa0406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 --2- 172.21.15.33:0/4011330313 >> v2:172.21.15.167:3300/0 conn(0x7faa04159530 0x7faa0406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 >> v1:172.21.15.167:6789/0 conn(0x7faa040b04e0 legacy=0x7faa0415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0406e610 con 0x7faa0406df70 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa9f4001470 con 0x7faa0406df70 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 >> v1:172.21.15.33:6789/0 conn(0x7faa0406df70 legacy=0x7faa040afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0003c40 0x7fa9f0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.166+0000 7faa09399700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0005210 0x7fa9f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa040b1e00 con 0x7fa9f0004810 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f000c130 con 0x7fa9f0005210 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f000c2c0 con 0x7fa9f0003c40 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa0a39b700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0003c40 0x7fa9f0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa0ab9c700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0005210 0x7fa9f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 -- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0003c40 msgr2=0x7fa9f0004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0003c40 0x7fa9f0004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 -- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0005210 msgr2=0x7fa9f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0005210 0x7fa9f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 -- 172.21.15.33:0/4011330313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0003570 con 0x7fa9f0004810 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa0ab9c700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9f0005210 0x7fa9f000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa0ab9c700 1 -- 172.21.15.33:0/4011330313 reap_dead start 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa0a39b700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0003c40 0x7fa9f0004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.167+0000 7faa09b9a700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c80 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c210 tx=0x7fa9f4001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b21316ad41c9da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400cb00 con 0x7fa9f0004810 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f400cc80 con 0x7fa9f0004810 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa09399700 1 -- 172.21.15.33:0/4011330313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005920 con 0x7fa9f0004810 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 -- 172.21.15.33:0/4011330313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f0004810 msgr2=0x7fa9f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c80 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c210 tx=0x7fa9f4001d40 comp rx=0 tx=0).stop 2023-10-06T05:31:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 -- 172.21.15.33:0/4011330313 shutdown_connections 2023-10-06T05:31:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 --2- 172.21.15.33:0/4011330313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f0004810 0x7fa9f0004c80 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 -- 172.21.15.33:0/4011330313 >> 172.21.15.33:0/4011330313 conn(0x7faa0406c560 msgr2=0x7faa0406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 -- 172.21.15.33:0/4011330313 shutdown_connections 2023-10-06T05:31:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.168+0000 7faa0b39d700 1 -- 172.21.15.33:0/4011330313 wait complete. 2023-10-06T05:31:49.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.170+0000 7faa0b39d700 1 Processor -- start 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.170+0000 7faa0b39d700 1 -- start start 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0b39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0004810 0x7faa04150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0b39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0414dcf0 0x7faa0414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 0x7faa041568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04154e20 con 0x7faa0414e160 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0b39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0414e8d0 con 0x7faa0414dcf0 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0b39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa04152410 con 0x7fa9f0004810 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 0x7faa041568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 0x7faa041568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33360/0 (socket says 172.21.15.33:33360) 2023-10-06T05:31:49.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 -- 172.21.15.33:0/3449718513 learned_addr learned my addr 172.21.15.33:0/3449718513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0a39b700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0004810 0x7faa04150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa09b9a700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0414dcf0 0x7faa0414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 -- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0004810 msgr2=0x7faa04150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0004810 0x7faa04150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 -- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0414dcf0 msgr2=0x7faa0414f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0414dcf0 0x7faa0414f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0ab9c700 1 -- 172.21.15.33:0/3449718513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa041550a0 con 0x7faa0414e160 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.171+0000 7faa0a39b700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0004810 0x7faa04150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.172+0000 7faa0ab9c700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 0x7faa041568d0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7faa040b0a90 tx=0x7fa9fc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.172+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc017070 con 0x7faa0414e160 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.172+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc006ba0 con 0x7faa0414e160 2023-10-06T05:31:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.172+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0032e0 con 0x7faa0414e160 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.172+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa04155380 con 0x7faa0414e160 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.172+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa04155660 con 0x7faa0414e160 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.173+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0404c380 con 0x7faa0414e160 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.174+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9fc0016c0 con 0x7faa0414e160 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.175+0000 7fa9fa7fc700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9e8066f30 0x7fa9e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.175+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa9fc084360 con 0x7faa0414e160 2023-10-06T05:31:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.175+0000 7faa0a39b700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9e8066f30 0x7fa9e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.176+0000 7faa0a39b700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9e8066f30 0x7fa9e80693e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7faa0415bb20 tx=0x7faa00008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:49.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.179+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9fc0513d0 con 0x7faa0414e160 2023-10-06T05:31:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.382+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa04159ff0 con 0x7faa0414e160 2023-10-06T05:31:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.383+0000 7fa9fa7fc700 1 -- 172.21.15.33:0/3449718513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fa9fc0549f0 con 0x7faa0414e160 2023-10-06T05:31:49.385 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:49.385 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":238,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:48.429445+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:31:46.602215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-06T05:31:47.403110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9e8066f30 msgr2=0x7fa9e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9e8066f30 0x7fa9e80693e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7faa0415bb20 tx=0x7faa00008040 comp rx=0 tx=0).stop 2023-10-06T05:31:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 msgr2=0x7faa041568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 0x7faa041568d0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7faa040b0a90 tx=0x7fa9fc009040 comp rx=0 tx=0).stop 2023-10-06T05:31:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 shutdown_connections 2023-10-06T05:31:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa9e8066f30 0x7fa9e80693e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa0414e160 0x7faa041568d0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9f0004810 0x7faa04150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 --2- 172.21.15.33:0/3449718513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa0414dcf0 0x7faa0414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 >> 172.21.15.33:0/3449718513 conn(0x7faa0406c560 msgr2=0x7faa041586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 shutdown_connections 2023-10-06T05:31:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.386+0000 7faa0b39d700 1 -- 172.21.15.33:0/3449718513 wait complete. 2023-10-06T05:31:49.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.591+0000 7f906ffff700 1 Processor -- start 2023-10-06T05:31:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.591+0000 7f906ffff700 1 -- start start 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.592+0000 7f906ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f907013c5c0 0x7f90700c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f90700c7020 0x7f90700c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700c7490 con 0x7f90700c93e0 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907013c990 con 0x7f90700c7020 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9070159bd0 con 0x7f907013c5c0 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f90700c93e0 0x7f90700675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53510/0 (socket says 172.21.15.33:53510) 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906f7fe700 1 -- 172.21.15.33:0/3452833918 learned_addr learned my addr 172.21.15.33:0/3452833918 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906e7fc700 1 --2- 172.21.15.33:0/3452833918 >> v2:172.21.15.167:3300/0 conn(0x7f90700c7020 0x7f90700c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906effd700 1 --2- 172.21.15.33:0/3452833918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f907013c5c0 0x7f90700c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906dffb700 1 -- 172.21.15.33:0/3452833918 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905412067 0 0) 0x7f90700c7490 con 0x7f90700c93e0 2023-10-06T05:31:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.593+0000 7f906dffb700 1 -- 172.21.15.33:0/3452833918 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9058003570 con 0x7f90700c93e0 2023-10-06T05:31:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906effd700 1 -- 172.21.15.33:0/3452833918 >> v2:172.21.15.167:3300/0 conn(0x7f90700c7020 msgr2=0x7f90700c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906effd700 1 --2- 172.21.15.33:0/3452833918 >> v2:172.21.15.167:3300/0 conn(0x7f90700c7020 0x7f90700c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906dffb700 1 -- 172.21.15.33:0/3452833918 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 476997990 0 0) 0x7f9058003570 con 0x7f90700c93e0 2023-10-06T05:31:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906effd700 1 -- 172.21.15.33:0/3452833918 >> v1:172.21.15.33:6789/0 conn(0x7f90700c93e0 legacy=0x7f90700675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906effd700 1 -- 172.21.15.33:0/3452833918 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907015a710 con 0x7f907013c5c0 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906effd700 1 --2- 172.21.15.33:0/3452833918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f907013c5c0 0x7f90700c6ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90600098e0 tx=0x7f90600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8770c60ddc4a26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906dffb700 1 -- 172.21.15.33:0/3452833918 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680017f0 con 0x7f907013c5c0 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906dffb700 1 -- 172.21.15.33:0/3452833918 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9060011420 con 0x7f907013c5c0 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.594+0000 7f906dffb700 1 -- 172.21.15.33:0/3452833918 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600117c0 con 0x7f907013c5c0 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 -- 172.21.15.33:0/3452833918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f907013c5c0 msgr2=0x7f90700c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 --2- 172.21.15.33:0/3452833918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f907013c5c0 0x7f90700c6ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90600098e0 tx=0x7f90600091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 -- 172.21.15.33:0/3452833918 shutdown_connections 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 --2- 172.21.15.33:0/3452833918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f907013c5c0 0x7f90700c6ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 --2- 172.21.15.33:0/3452833918 >> v2:172.21.15.167:3300/0 conn(0x7f90700c7020 0x7f90700c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 -- 172.21.15.33:0/3452833918 >> 172.21.15.33:0/3452833918 conn(0x7f907006c560 msgr2=0x7f907006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 -- 172.21.15.33:0/3452833918 shutdown_connections 2023-10-06T05:31:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.595+0000 7f906ffff700 1 -- 172.21.15.33:0/3452833918 wait complete. 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.596+0000 7f906ffff700 1 Processor -- start 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 -- start start 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90700c7020 0x7f907015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90700c93e0 0x7f90700c2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 0x7f90700c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907015b5f0 con 0x7f907013c5c0 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907015b8d0 con 0x7f90700c7020 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.597+0000 7f906ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907015bbb0 con 0x7f90700c93e0 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 0x7f90700c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 0x7f90700c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33376/0 (socket says 172.21.15.33:33376) 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 -- 172.21.15.33:0/3259567572 learned_addr learned my addr 172.21.15.33:0/3259567572 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906effd700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90700c7020 0x7f907015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906e7fc700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90700c93e0 0x7f90700c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 -- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90700c93e0 msgr2=0x7f90700c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90700c93e0 0x7f90700c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 -- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90700c7020 msgr2=0x7f907015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90700c7020 0x7f907015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 -- 172.21.15.33:0/3259567572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060009420 con 0x7f907013c5c0 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.598+0000 7f906f7fe700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 0x7f90700c2750 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f906800cb90 tx=0x7f9068007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.599+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068004dc0 con 0x7f907013c5c0 2023-10-06T05:31:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.599+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800a660 con 0x7f907013c5c0 2023-10-06T05:31:49.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.599+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907015be90 con 0x7f907013c5c0 2023-10-06T05:31:49.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.599+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800aa20 con 0x7f907013c5c0 2023-10-06T05:31:49.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.599+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907015b2e0 con 0x7f907013c5c0 2023-10-06T05:31:49.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.600+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907015c390 con 0x7f907013c5c0 2023-10-06T05:31:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.601+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9068007150 con 0x7f907013c5c0 2023-10-06T05:31:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.602+0000 7f9056ffd700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f905c066f60 0x7f905c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.602+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9068084200 con 0x7f907013c5c0 2023-10-06T05:31:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.602+0000 7f906effd700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f905c066f60 0x7f905c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:49.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.603+0000 7f906effd700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f905c066f60 0x7f905c069410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f90600030c0 tx=0x7f9060016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.606+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90680512f0 con 0x7f907013c5c0 2023-10-06T05:31:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.809+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f907004c380 con 0x7f907013c5c0 2023-10-06T05:31:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.810+0000 7f9056ffd700 1 -- 172.21.15.33:0/3259567572 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f9068054910 con 0x7f907013c5c0 2023-10-06T05:31:49.812 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:49.812 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":238,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:48.429445+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:31:46.602215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-06T05:31:47.403110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f905c066f60 msgr2=0x7f905c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f905c066f60 0x7f905c069410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f90600030c0 tx=0x7f9060016000 comp rx=0 tx=0).stop 2023-10-06T05:31:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 msgr2=0x7f90700c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 0x7f90700c2750 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f906800cb90 tx=0x7f9068007ca0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 shutdown_connections 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f905c066f60 0x7f905c069410 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907013c5c0 0x7f90700c2750 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90700c7020 0x7f907015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 --2- 172.21.15.33:0/3259567572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90700c93e0 0x7f90700c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 >> 172.21.15.33:0/3259567572 conn(0x7f907006c560 msgr2=0x7f90700c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 shutdown_connections 2023-10-06T05:31:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:49.812+0000 7f906ffff700 1 -- 172.21.15.33:0/3259567572 wait complete. 2023-10-06T05:31:49.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:31:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.023+0000 7fb5ce59d700 1 Processor -- start 2023-10-06T05:31:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.023+0000 7fb5ce59d700 1 -- start start 2023-10-06T05:31:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5ce59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5ce59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5c80b01f0 0x7fb5c80b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5ce59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80ad430 con 0x7fb5c8156370 2023-10-06T05:31:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5ce59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80afe20 con 0x7fb5c80b01f0 2023-10-06T05:31:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5ce59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80b0b70 con 0x7fb5c80cc7e0 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cdd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5c80b01f0 0x7fb5c80b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cdd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5c80b01f0 0x7fb5c80b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60766/0 (socket says 172.21.15.33:60766) 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/339839459 learned_addr learned my addr 172.21.15.33:0/339839459 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cd59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb5c8156370 0x7fb5c80c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35028/0 (socket says 172.21.15.33:35028) 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5ccd9a700 1 --2- 172.21.15.33:0/339839459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/339839459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 msgr2=0x7fb5c80af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cdd9c700 1 --2- 172.21.15.33:0/339839459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c80af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.025+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/339839459 >> v1:172.21.15.33:6789/0 conn(0x7fb5c8156370 legacy=0x7fb5c80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.026+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/339839459 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c806bc70 con 0x7fb5c80b01f0 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.026+0000 7fb5bffff700 1 -- 172.21.15.33:0/339839459 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902853624 0 0) 0x7fb5c80ad430 con 0x7fb5c8156370 2023-10-06T05:31:50.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.026+0000 7fb5cdd9c700 1 --2- 172.21.15.33:0/339839459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80b01f0 0x7fb5c80b0630 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009f20 tx=0x7fb5c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3a691062a67fe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.026+0000 7fb5bffff700 1 -- 172.21.15.33:0/339839459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000e070 con 0x7fb5c80b01f0 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.026+0000 7fb5bffff700 1 -- 172.21.15.33:0/339839459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000ba80 con 0x7fb5c80b01f0 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.026+0000 7fb5bffff700 1 -- 172.21.15.33:0/339839459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00092e0 con 0x7fb5c80b01f0 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 -- 172.21.15.33:0/339839459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80b01f0 msgr2=0x7fb5c80b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/339839459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80b01f0 0x7fb5c80b0630 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009f20 tx=0x7fb5c0009840 comp rx=0 tx=0).stop 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 -- 172.21.15.33:0/339839459 shutdown_connections 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/339839459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c80b01f0 0x7fb5c80b0630 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/339839459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c80af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 -- 172.21.15.33:0/339839459 >> 172.21.15.33:0/339839459 conn(0x7fb5c806d030 msgr2=0x7fb5c80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 -- 172.21.15.33:0/339839459 shutdown_connections 2023-10-06T05:31:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.027+0000 7fb5ce59d700 1 -- 172.21.15.33:0/339839459 wait complete. 2023-10-06T05:31:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.028+0000 7fb5ce59d700 1 Processor -- start 2023-10-06T05:31:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.028+0000 7fb5ce59d700 1 -- start start 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5ce59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c80b01f0 0x7fb5c8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5ce59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c8146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5ce59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 0x7fb5c80adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5ce59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c814a900 con 0x7fb5c8156370 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5ce59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c81594b0 con 0x7fb5c80b01f0 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5ce59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c81596e0 con 0x7fb5c80cc7e0 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.029+0000 7fb5cdd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 0x7fb5c80adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 0x7fb5c80adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60774/0 (socket says 172.21.15.33:60774) 2023-10-06T05:31:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/3008775217 learned_addr learned my addr 172.21.15.33:0/3008775217 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5ccd9a700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c8146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cd59b700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c80b01f0 0x7fb5c8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 msgr2=0x7fb5c8146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c8146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c80b01f0 msgr2=0x7fb5c8158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c80b01f0 0x7fb5c8158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 -- 172.21.15.33:0/3008775217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009ae0 con 0x7fb5c8156370 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5cdd9c700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 0x7fb5c80adba0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000b6c0 tx=0x7fb5c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.030+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00032f0 con 0x7fb5c8156370 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.031+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0008ce0 con 0x7fb5c8156370 2023-10-06T05:31:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.031+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c81689d0 con 0x7fb5c8156370 2023-10-06T05:31:50.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.031+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c001c620 con 0x7fb5c8156370 2023-10-06T05:31:50.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.031+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c8167ef0 con 0x7fb5c8156370 2023-10-06T05:31:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.032+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c804c380 con 0x7fb5c8156370 2023-10-06T05:31:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.033+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5c0003040 con 0x7fb5c8156370 2023-10-06T05:31:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.033+0000 7fb5bd7fa700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5ac066f30 0x7fb5ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.034+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5c0085fa0 con 0x7fb5c8156370 2023-10-06T05:31:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.034+0000 7fb5cd59b700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5ac066f30 0x7fb5ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.035+0000 7fb5cd59b700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5ac066f30 0x7fb5ac0693e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4007fd0 tx=0x7fb5c4007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:50.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.038+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5c0053010 con 0x7fb5c8156370 2023-10-06T05:31:50.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.285+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5c806f310 con 0x7fb5c8156370 2023-10-06T05:31:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.966+0000 7fb5bd7fa700 1 -- 172.21.15.33:0/3008775217 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v184) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb5c0056630 con 0x7fb5c8156370 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5ac066f30 msgr2=0x7fb5ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5ac066f30 0x7fb5ac0693e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4007fd0 tx=0x7fb5c4007d50 comp rx=0 tx=0).stop 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 msgr2=0x7fb5c80adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 0x7fb5c80adba0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000b6c0 tx=0x7fb5c0018000 comp rx=0 tx=0).stop 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 shutdown_connections 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb5ac066f30 0x7fb5ac0693e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c8156370 0x7fb5c80adba0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c80b01f0 0x7fb5c8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 --2- 172.21.15.33:0/3008775217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5c80cc7e0 0x7fb5c8146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 >> 172.21.15.33:0/3008775217 conn(0x7fb5c806d030 msgr2=0x7fb5c80ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 shutdown_connections 2023-10-06T05:31:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:50.969+0000 7fb5ce59d700 1 -- 172.21.15.33:0/3008775217 wait complete. 2023-10-06T05:31:50.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:31:51.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:31:51.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:31:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.185+0000 7f41d279d700 1 Processor -- start 2023-10-06T05:31:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.185+0000 7f41d279d700 1 -- start start 2023-10-06T05:31:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41cc0cc7e0 0x7f41cc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d279d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc0ad430 con 0x7f41cc0b01f0 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc0afe20 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc06bc70 con 0x7f41cc156370 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d0f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41cc0cc7e0 0x7f41cc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d0f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41cc0cc7e0 0x7f41cc0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60786/0 (socket says 172.21.15.33:60786) 2023-10-06T05:31:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d0f9a700 1 -- 172.21.15.33:0/2815043863 learned_addr learned my addr 172.21.15.33:0/2815043863 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d179b700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d0f9a700 1 -- 172.21.15.33:0/2815043863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 msgr2=0x7f41cc0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d0f9a700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.187+0000 7f41d0f9a700 1 -- 172.21.15.33:0/2815043863 >> v1:172.21.15.167:6789/0 conn(0x7f41cc0b01f0 legacy=0x7f41cc0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41d0f9a700 1 -- 172.21.15.33:0/2815043863 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc0c92f0 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41d179b700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41d0f9a700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0c8db0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0099b0 tx=0x7f41bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb1da0a6bfd2b2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41c3fff700 1 -- 172.21.15.33:0/2815043863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00b820 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41c3fff700 1 -- 172.21.15.33:0/2815043863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc00b980 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41c3fff700 1 -- 172.21.15.33:0/2815043863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00bd20 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41d279d700 1 -- 172.21.15.33:0/2815043863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 msgr2=0x7f41cc0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.188+0000 7f41d279d700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0c8db0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0099b0 tx=0x7f41bc009250 comp rx=0 tx=0).stop 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.189+0000 7f41d279d700 1 -- 172.21.15.33:0/2815043863 shutdown_connections 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.189+0000 7f41d279d700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.189+0000 7f41d279d700 1 --2- 172.21.15.33:0/2815043863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0c8db0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.189+0000 7f41d279d700 1 -- 172.21.15.33:0/2815043863 >> 172.21.15.33:0/2815043863 conn(0x7f41cc06d030 msgr2=0x7f41cc0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.189+0000 7f41d279d700 1 -- 172.21.15.33:0/2815043863 shutdown_connections 2023-10-06T05:31:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.189+0000 7f41d279d700 1 -- 172.21.15.33:0/2815043863 wait complete. 2023-10-06T05:31:51.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.190+0000 7f41d279d700 1 Processor -- start 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.190+0000 7f41d279d700 1 -- start start 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc0b01f0 0x7f41cc14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc1594b0 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc16d6d0 con 0x7f41cc0b01f0 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc16d9b0 con 0x7f41cc156370 2023-10-06T05:31:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.191+0000 7f41d0f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60794/0 (socket says 172.21.15.33:60794) 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 -- 172.21.15.33:0/3782118446 learned_addr learned my addr 172.21.15.33:0/3782118446 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d179b700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc0b01f0 0x7f41cc14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d1f9c700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 -- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 msgr2=0x7f41cc0aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 -- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc0b01f0 msgr2=0x7f41cc14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc0b01f0 0x7f41cc14eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 -- 172.21.15.33:0/3782118446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0094f0 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d179b700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc0b01f0 0x7f41cc14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:51.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d1f9c700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:31:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d0f9a700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0aaa10 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009290 tx=0x7f41bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc011d70 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc020070 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.192+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc16dc90 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.193+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc01b6a0 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.193+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc16e490 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.194+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc04c380 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.194+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41bc022030 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.196+0000 7f41c17fa700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41b0066f30 0x7f41b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:51.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.196+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41bc086720 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.196+0000 7f41d179b700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41b0066f30 0x7f41b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:51.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.197+0000 7f41d179b700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41b0066f30 0x7f41b00693e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f41c8006fd0 tx=0x7f41c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:51.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.200+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41bc053790 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.444+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f41cc04c380 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.972+0000 7f41c17fa700 1 -- 172.21.15.33:0/3782118446 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v185) v1 ==== 122+0+0 (secure 0 0 0) 0x7f41bc084e60 con 0x7f41cc0cc7e0 2023-10-06T05:31:51.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41b0066f30 msgr2=0x7f41b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:51.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41b0066f30 0x7f41b00693e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f41c8006fd0 tx=0x7f41c8008040 comp rx=0 tx=0).stop 2023-10-06T05:31:51.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 msgr2=0x7f41cc0aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:51.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0aaa10 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009290 tx=0x7f41bc016000 comp rx=0 tx=0).stop 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 shutdown_connections 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f41b0066f30 0x7f41b00693e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41cc156370 0x7f41cc0aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41cc0b01f0 0x7f41cc14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 --2- 172.21.15.33:0/3782118446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41cc0cc7e0 0x7f41cc0aaa10 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 >> 172.21.15.33:0/3782118446 conn(0x7f41cc06d030 msgr2=0x7f41cc159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 shutdown_connections 2023-10-06T05:31:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:51.975+0000 7f41d279d700 1 -- 172.21.15.33:0/3782118446 wait complete. 2023-10-06T05:31:51.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.183+0000 7fb6dab9d700 1 Processor -- start 2023-10-06T05:31:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.183+0000 7fb6dab9d700 1 -- start start 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6dab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6d40cda10 0x7fb6d40b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6dab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d40b14b0 con 0x7fb6d4067d00 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6dab9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d40b08a0 con 0x7fb6d40cd640 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6dab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d40b0b80 con 0x7fb6d40cda10 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6d939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6d40cda10 0x7fb6d40b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6d939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6d40cda10 0x7fb6d40b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60798/0 (socket says 172.21.15.33:60798) 2023-10-06T05:31:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6da39c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb6d4067d00 0x7fb6d40ca5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35034/0 (socket says 172.21.15.33:35034) 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.185+0000 7fb6d939a700 1 -- 172.21.15.33:0/3532407919 learned_addr learned my addr 172.21.15.33:0/3532407919 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d939a700 1 -- 172.21.15.33:0/3532407919 >> v1:172.21.15.167:6789/0 conn(0x7fb6d40cd640 legacy=0x7fb6d40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d939a700 1 -- 172.21.15.33:0/3532407919 >> v1:172.21.15.33:6789/0 conn(0x7fb6d4067d00 legacy=0x7fb6d40ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d939a700 1 -- 172.21.15.33:0/3532407919 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40b0e60 con 0x7fb6d40cda10 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d8b99700 1 -- 172.21.15.33:0/3532407919 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538538639 0 0) 0x7fb6d40b14b0 con 0x7fb6d4067d00 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d939a700 1 --2- 172.21.15.33:0/3532407919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40b1740 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40099b0 tx=0x7fb6c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b17b0351404bdd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d8b99700 1 -- 172.21.15.33:0/3532407919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400b860 con 0x7fb6d40cda10 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d8b99700 1 -- 172.21.15.33:0/3532407919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c400b9c0 con 0x7fb6d40cda10 2023-10-06T05:31:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.186+0000 7fb6d8b99700 1 -- 172.21.15.33:0/3532407919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400bd60 con 0x7fb6d40cda10 2023-10-06T05:31:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 -- 172.21.15.33:0/3532407919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 msgr2=0x7fb6d40b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/3532407919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40b1740 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40099b0 tx=0x7fb6c4009250 comp rx=0 tx=0).stop 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 -- 172.21.15.33:0/3532407919 shutdown_connections 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/3532407919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40b1740 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 -- 172.21.15.33:0/3532407919 >> 172.21.15.33:0/3532407919 conn(0x7fb6d406e0f0 msgr2=0x7fb6d40675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 -- 172.21.15.33:0/3532407919 shutdown_connections 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.187+0000 7fb6dab9d700 1 -- 172.21.15.33:0/3532407919 wait complete. 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.188+0000 7fb6dab9d700 1 Processor -- start 2023-10-06T05:31:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.188+0000 7fb6dab9d700 1 -- start start 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.189+0000 7fb6dab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d4067d00 0x7fb6d40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.189+0000 7fb6dab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6d40cd640 0x7fb6d4153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.189+0000 7fb6dab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.189+0000 7fb6dab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40a5de0 con 0x7fb6d40cda10 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.189+0000 7fb6dab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40a5520 con 0x7fb6d4067d00 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.189+0000 7fb6dab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40a5790 con 0x7fb6d40cd640 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60804/0 (socket says 172.21.15.33:60804) 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 -- 172.21.15.33:0/1920331048 learned_addr learned my addr 172.21.15.33:0/1920331048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6d9b9b700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d4067d00 0x7fb6d40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6d939a700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6d40cd640 0x7fb6d4153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 -- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6d40cd640 msgr2=0x7fb6d4153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6d40cd640 0x7fb6d4153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 -- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d4067d00 msgr2=0x7fb6d40a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d4067d00 0x7fb6d40a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 -- 172.21.15.33:0/1920331048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c40094f0 con 0x7fb6d40cda10 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6da39c700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40af120 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00ccf0 tx=0x7fb6cc001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00ce30 con 0x7fb6d40cda10 2023-10-06T05:31:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc005df0 con 0x7fb6d40cda10 2023-10-06T05:31:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.190+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d40a5a70 con 0x7fb6d40cda10 2023-10-06T05:31:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.191+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00ab70 con 0x7fb6d40cda10 2023-10-06T05:31:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.191+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d4159930 con 0x7fb6d40cda10 2023-10-06T05:31:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.191+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d406fcc0 con 0x7fb6d40cda10 2023-10-06T05:31:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.192+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6cc0070f0 con 0x7fb6d40cda10 2023-10-06T05:31:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.193+0000 7fb6c9ffb700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.193+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb6cc084760 con 0x7fb6d40cda10 2023-10-06T05:31:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.194+0000 7fb6d9b9b700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.194+0000 7fb6d9b9b700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6b8066f30 0x7fb6b80693e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0003860 tx=0x7fb6d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:52.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.198+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6cc0518f0 con 0x7fb6d40cda10 2023-10-06T05:31:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.445+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d404c380 con 0x7fb6d40cda10 2023-10-06T05:31:52.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.445+0000 7fb6c9ffb700 1 -- 172.21.15.33:0/1920331048 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fb6cc054f10 con 0x7fb6d40cda10 2023-10-06T05:31:52.447 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:52.447 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:31:48.429792+0000","modified":"2023-10-06T05:31:51.962941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11169},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11169":{"gid":11169,"name":"a","rank":0,"incarnation":181,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1245472058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1245472058},{"type":"v1","addr":"172.21.15.33:6835","nonce":1245472058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2023-10-06T05:31:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6b8066f30 msgr2=0x7fb6b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6b8066f30 0x7fb6b80693e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0003860 tx=0x7fb6d0009040 comp rx=0 tx=0).stop 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 msgr2=0x7fb6d40af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40af120 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00ccf0 tx=0x7fb6cc001a70 comp rx=0 tx=0).stop 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 shutdown_connections 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d40cda10 0x7fb6d40af120 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d4067d00 0x7fb6d40a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 --2- 172.21.15.33:0/1920331048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6d40cd640 0x7fb6d4153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 >> 172.21.15.33:0/1920331048 conn(0x7fb6d406e0f0 msgr2=0x7fb6d40accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 shutdown_connections 2023-10-06T05:31:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.448+0000 7fb6dab9d700 1 -- 172.21.15.33:0/1920331048 wait complete. 2023-10-06T05:31:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 185 2023-10-06T05:31:52.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:52.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.652+0000 7f602739d700 1 Processor -- start 2023-10-06T05:31:52.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.652+0000 7f602739d700 1 -- start start 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.653+0000 7f602739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6020155af0 0x7f602006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602739d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602006ab60 con 0x7f60200c7920 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602739d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602006ae40 con 0x7f602006a6a0 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602006b7e0 con 0x7f6020155af0 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602639b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6020155af0 0x7f602006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602639b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6020155af0 0x7f602006a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60812/0 (socket says 172.21.15.33:60812) 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602639b700 1 -- 172.21.15.33:0/3582505507 learned_addr learned my addr 172.21.15.33:0/3582505507 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:52.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f6026b9c700 1 --1- 172.21.15.33:0/3582505507 >> v1:172.21.15.33:6789/0 conn(0x7f60200c7920 0x7f60200c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35048/0 (socket says 172.21.15.33:35048) 2023-10-06T05:31:52.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602639b700 1 -- 172.21.15.33:0/3582505507 >> v1:172.21.15.167:6789/0 conn(0x7f602006a6a0 legacy=0x7f60200c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602639b700 1 -- 172.21.15.33:0/3582505507 >> v1:172.21.15.33:6789/0 conn(0x7f60200c7920 legacy=0x7f60200c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f602639b700 1 -- 172.21.15.33:0/3582505507 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602006b120 con 0x7f6020155af0 2023-10-06T05:31:52.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.654+0000 7f6025399700 1 -- 172.21.15.33:0/3582505507 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487723575 0 0) 0x7f602006ab60 con 0x7f60200c7920 2023-10-06T05:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.655+0000 7f602639b700 1 --2- 172.21.15.33:0/3582505507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020155af0 0x7f602006a160 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099b0 tx=0x7f601c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7149207aaac3cf5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.655+0000 7f6025399700 1 -- 172.21.15.33:0/3582505507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00b860 con 0x7f6020155af0 2023-10-06T05:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.655+0000 7f6025399700 1 -- 172.21.15.33:0/3582505507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00b9c0 con 0x7f6020155af0 2023-10-06T05:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.655+0000 7f6025399700 1 -- 172.21.15.33:0/3582505507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00bd60 con 0x7f6020155af0 2023-10-06T05:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.655+0000 7f602739d700 1 -- 172.21.15.33:0/3582505507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020155af0 msgr2=0x7f602006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.655+0000 7f602739d700 1 --2- 172.21.15.33:0/3582505507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020155af0 0x7f602006a160 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099b0 tx=0x7f601c009250 comp rx=0 tx=0).stop 2023-10-06T05:31:52.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.656+0000 7f602739d700 1 -- 172.21.15.33:0/3582505507 shutdown_connections 2023-10-06T05:31:52.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.656+0000 7f602739d700 1 --2- 172.21.15.33:0/3582505507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6020155af0 0x7f602006a160 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.656+0000 7f602739d700 1 -- 172.21.15.33:0/3582505507 >> 172.21.15.33:0/3582505507 conn(0x7f602006d030 msgr2=0x7f602006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:52.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.656+0000 7f602739d700 1 -- 172.21.15.33:0/3582505507 shutdown_connections 2023-10-06T05:31:52.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.656+0000 7f602739d700 1 -- 172.21.15.33:0/3582505507 wait complete. 2023-10-06T05:31:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.657+0000 7f602739d700 1 Processor -- start 2023-10-06T05:31:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.657+0000 7f602739d700 1 -- start start 2023-10-06T05:31:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f602739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f602006a6a0 0x7f6020151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f602739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 0x7f6020157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f602739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6020155af0 0x7f60201582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f602739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6020151cc0 con 0x7f60200c7920 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f602739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6020168e60 con 0x7f602006a6a0 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f602739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6020168530 con 0x7f6020155af0 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f6025b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 0x7f6020157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f6025b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 0x7f6020157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60814/0 (socket says 172.21.15.33:60814) 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f6025b9a700 1 -- 172.21.15.33:0/3993878445 learned_addr learned my addr 172.21.15.33:0/3993878445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.658+0000 7f6026b9c700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6020155af0 0x7f60201582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f602639b700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f602006a6a0 0x7f6020151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f6025b9a700 1 -- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6020155af0 msgr2=0x7f60201582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f6025b9a700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6020155af0 0x7f60201582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f6025b9a700 1 -- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f602006a6a0 msgr2=0x7f6020151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f6025b9a700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f602006a6a0 0x7f6020151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f6025b9a700 1 -- 172.21.15.33:0/3993878445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0094f0 con 0x7f60200c7920 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f6025b9a700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 0x7f6020157d70 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f601000c7c0 tx=0x7f6010007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010011590 con 0x7f60200c7920 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60100116f0 con 0x7f60200c7920 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010011ab0 con 0x7f60200c7920 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6020168810 con 0x7f60200c7920 2023-10-06T05:31:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.659+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6020158950 con 0x7f60200c7920 2023-10-06T05:31:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.661+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602004c380 con 0x7f60200c7920 2023-10-06T05:31:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.661+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f601001b3b0 con 0x7f60200c7920 2023-10-06T05:31:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.662+0000 7f60167fc700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6004066f30 0x7f60040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.662+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6010083c90 con 0x7f60200c7920 2023-10-06T05:31:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.663+0000 7f602639b700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6004066f30 0x7f60040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.664+0000 7f602639b700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6004066f30 0x7f60040693e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099b0 tx=0x7f601c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:52.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.666+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6010050d00 con 0x7f60200c7920 2023-10-06T05:31:52.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.871+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6020168fa0 con 0x7f60200c7920 2023-10-06T05:31:52.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.872+0000 7f60167fc700 1 -- 172.21.15.33:0/3993878445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f6010017020 con 0x7f60200c7920 2023-10-06T05:31:52.873 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":239,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:49.887625+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:31:46.602215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-06T05:31:47.403110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:31:52.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6004066f30 msgr2=0x7f60040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6004066f30 0x7f60040693e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099b0 tx=0x7f601c017000 comp rx=0 tx=0).stop 2023-10-06T05:31:52.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 msgr2=0x7f6020157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:52.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 0x7f6020157d70 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f601000c7c0 tx=0x7f6010007f50 comp rx=0 tx=0).stop 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 shutdown_connections 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6004066f30 0x7f60040693e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6020155af0 0x7f60201582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f602006a6a0 0x7f6020151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 --2- 172.21.15.33:0/3993878445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60200c7920 0x7f6020157d70 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 >> 172.21.15.33:0/3993878445 conn(0x7f602006d030 msgr2=0x7f60200ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 shutdown_connections 2023-10-06T05:31:52.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:52.874+0000 7f602739d700 1 -- 172.21.15.33:0/3993878445 wait complete. 2023-10-06T05:31:52.894 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:31:52.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:31:53.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.175+0000 7f02b7fff700 1 Processor -- start 2023-10-06T05:31:53.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.175+0000 7f02b7fff700 1 -- start start 2023-10-06T05:31:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.176+0000 7f02b7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02b80ccfc0 0x7f02b80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.176+0000 7f02b7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b80cbd80 con 0x7f02b80cf3b0 2023-10-06T05:31:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.176+0000 7f02b7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b80af5f0 con 0x7f02b80cc750 2023-10-06T05:31:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.176+0000 7f02b7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b80af8d0 con 0x7f02b80ccfc0 2023-10-06T05:31:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.176+0000 7f02b6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02b80ccfc0 0x7f02b80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b67fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f02b80cf3b0 0x7f02b81594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35050/0 (socket says 172.21.15.33:35050) 2023-10-06T05:31:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b67fc700 1 -- 172.21.15.33:0/3225455036 learned_addr learned my addr 172.21.15.33:0/3225455036 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b6ffd700 1 -- 172.21.15.33:0/3225455036 >> v1:172.21.15.167:6789/0 conn(0x7f02b80cc750 legacy=0x7f02b80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b6ffd700 1 -- 172.21.15.33:0/3225455036 >> v1:172.21.15.33:6789/0 conn(0x7f02b80cf3b0 legacy=0x7f02b81594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b6ffd700 1 -- 172.21.15.33:0/3225455036 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80afbb0 con 0x7f02b80ccfc0 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b6ffd700 1 --2- 172.21.15.33:0/3225455036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80ccfc0 0x7f02b80cc210 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009910 tx=0x7f02a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56f2e90b6fa1e27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.177+0000 7f02b5ffb700 1 -- 172.21.15.33:0/3225455036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800b820 con 0x7f02b80ccfc0 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b5ffb700 1 -- 172.21.15.33:0/3225455036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a800b980 con 0x7f02b80ccfc0 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b5ffb700 1 -- 172.21.15.33:0/3225455036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800bd20 con 0x7f02b80ccfc0 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 -- 172.21.15.33:0/3225455036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80ccfc0 msgr2=0x7f02b80cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3225455036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80ccfc0 0x7f02b80cc210 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009910 tx=0x7f02a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 -- 172.21.15.33:0/3225455036 shutdown_connections 2023-10-06T05:31:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3225455036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b80ccfc0 0x7f02b80cc210 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 -- 172.21.15.33:0/3225455036 >> 172.21.15.33:0/3225455036 conn(0x7f02b806cff0 msgr2=0x7f02b806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 -- 172.21.15.33:0/3225455036 shutdown_connections 2023-10-06T05:31:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.178+0000 7f02b7fff700 1 -- 172.21.15.33:0/3225455036 wait complete. 2023-10-06T05:31:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.180+0000 7f02b7fff700 1 Processor -- start 2023-10-06T05:31:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.180+0000 7f02b7fff700 1 -- start start 2023-10-06T05:31:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80cc750 0x7f02b815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80ccfc0 0x7f02b814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 0x7f02b814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80b1e00 con 0x7f02b815ec80 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80b11f0 con 0x7f02b80ccfc0 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80b14d0 con 0x7f02b80cc750 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 0x7f02b814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 0x7f02b814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60828/0 (socket says 172.21.15.33:60828) 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 -- 172.21.15.33:0/3082834188 learned_addr learned my addr 172.21.15.33:0/3082834188 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b6ffd700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80cc750 0x7f02b815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b67fc700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80ccfc0 0x7f02b814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 -- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80cc750 msgr2=0x7f02b815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80cc750 0x7f02b815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 -- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80ccfc0 msgr2=0x7f02b814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80ccfc0 0x7f02b814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b77fe700 1 -- 172.21.15.33:0/3082834188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8009420 con 0x7f02b815ec80 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.181+0000 7f02b67fc700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80ccfc0 0x7f02b814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.182+0000 7f02b77fe700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 0x7f02b814f3e0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c8e0 tx=0x7f02b0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.182+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0012630 con 0x7f02b815ec80 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.182+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0012790 con 0x7f02b815ec80 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.182+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b80b17b0 con 0x7f02b815ec80 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.182+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0012b50 con 0x7f02b815ec80 2023-10-06T05:31:53.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.182+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b80b0e60 con 0x7f02b815ec80 2023-10-06T05:31:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.183+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b80d1390 con 0x7f02b815ec80 2023-10-06T05:31:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.184+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02b000a3b0 con 0x7f02b815ec80 2023-10-06T05:31:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.185+0000 7f029effd700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f02a4066f30 0x7f02a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.185+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f02b00849c0 con 0x7f02b815ec80 2023-10-06T05:31:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.185+0000 7f02b6ffd700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f02a4066f30 0x7f02a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.186+0000 7f02b6ffd700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f02a4066f30 0x7f02a40693e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f02a80093f0 tx=0x7f02a8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:53.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.189+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02b0051a30 con 0x7f02b815ec80 2023-10-06T05:31:53.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.384+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02b804c380 con 0x7f02a4066f30 2023-10-06T05:31:53.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.395+0000 7f029effd700 1 -- 172.21.15.33:0/3082834188 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311730 (secure 0 0 0) 0x7f02b804c380 con 0x7f02a4066f30 2023-10-06T05:31:53.403 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:53.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f02a4066f30 msgr2=0x7f02a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f02a4066f30 0x7f02a40693e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f02a80093f0 tx=0x7f02a8017000 comp rx=0 tx=0).stop 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 msgr2=0x7f02b814f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 0x7f02b814f3e0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c8e0 tx=0x7f02b0001710 comp rx=0 tx=0).stop 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 shutdown_connections 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f02a4066f30 0x7f02a40693e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b815ec80 0x7f02b814f3e0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80cc750 0x7f02b815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 --2- 172.21.15.33:0/3082834188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80ccfc0 0x7f02b814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 >> 172.21.15.33:0/3082834188 conn(0x7f02b806cff0 msgr2=0x7f02b80c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 shutdown_connections 2023-10-06T05:31:53.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.399+0000 7f02b7fff700 1 -- 172.21.15.33:0/3082834188 wait complete. 2023-10-06T05:31:53.407 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:31:53.418 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":762,"stamp":"2023-10-06T05:31:52.086007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440040,"kb_used_data":24872,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752534680,"statfs":{"total":773094113280,"available":770595512320,"internally_reserved":0,"allocated":25468928,"data_stored":12538732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.159143"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917637+0000","last_change":"2023-10-06T05:31:48.911533+0000","last_active":"2023-10-06T05:31:48.917637+0000","last_peered":"2023-10-06T05:31:48.917637+0000","last_clean":"2023-10-06T05:31:48.917637+0000","last_became_active":"2023-10-06T05:31:48.909401+0000","last_became_peered":"2023-10-06T05:31:48.909401+0000","last_unstale":"2023-10-06T05:31:48.917637+0000","last_undegraded":"2023-10-06T05:31:48.917637+0000","last_fullsized":"2023-10-06T05:31:48.917637+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:30.379004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909120+0000","last_change":"2023-10-06T05:31:48.903476+0000","last_active":"2023-10-06T05:31:48.909120+0000","last_peered":"2023-10-06T05:31:48.909120+0000","last_clean":"2023-10-06T05:31:48.909120+0000","last_became_active":"2023-10-06T05:31:48.903185+0000","last_became_peered":"2023-10-06T05:31:48.903185+0000","last_unstale":"2023-10-06T05:31:48.909120+0000","last_undegraded":"2023-10-06T05:31:48.909120+0000","last_fullsized":"2023-10-06T05:31:48.909120+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:28.209326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915923+0000","last_change":"2023-10-06T05:31:48.906545+0000","last_active":"2023-10-06T05:31:48.915923+0000","last_peered":"2023-10-06T05:31:48.915923+0000","last_clean":"2023-10-06T05:31:48.915923+0000","last_became_active":"2023-10-06T05:31:48.906238+0000","last_became_peered":"2023-10-06T05:31:48.906238+0000","last_unstale":"2023-10-06T05:31:48.915923+0000","last_undegraded":"2023-10-06T05:31:48.915923+0000","last_fullsized":"2023-10-06T05:31:48.915923+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:13.121708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915907+0000","last_change":"2023-10-06T05:31:48.905955+0000","last_active":"2023-10-06T05:31:48.915907+0000","last_peered":"2023-10-06T05:31:48.915907+0000","last_clean":"2023-10-06T05:31:48.915907+0000","last_became_active":"2023-10-06T05:31:48.905661+0000","last_became_peered":"2023-10-06T05:31:48.905661+0000","last_unstale":"2023-10-06T05:31:48.915907+0000","last_undegraded":"2023-10-06T05:31:48.915907+0000","last_fullsized":"2023-10-06T05:31:48.915907+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:19.730450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909219+0000","last_change":"2023-10-06T05:31:48.907884+0000","last_active":"2023-10-06T05:31:48.909219+0000","last_peered":"2023-10-06T05:31:48.909219+0000","last_clean":"2023-10-06T05:31:48.909219+0000","last_became_active":"2023-10-06T05:31:48.907610+0000","last_became_peered":"2023-10-06T05:31:48.907610+0000","last_unstale":"2023-10-06T05:31:48.909219+0000","last_undegraded":"2023-10-06T05:31:48.909219+0000","last_fullsized":"2023-10-06T05:31:48.909219+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:42.859104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908003+0000","last_change":"2023-10-06T05:31:48.903453+0000","last_active":"2023-10-06T05:31:48.908003+0000","last_peered":"2023-10-06T05:31:48.908003+0000","last_clean":"2023-10-06T05:31:48.908003+0000","last_became_active":"2023-10-06T05:31:48.903140+0000","last_became_peered":"2023-10-06T05:31:48.903140+0000","last_unstale":"2023-10-06T05:31:48.908003+0000","last_undegraded":"2023-10-06T05:31:48.908003+0000","last_fullsized":"2023-10-06T05:31:48.908003+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:14.525280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.910728+0000","last_change":"2023-10-06T05:31:48.904072+0000","last_active":"2023-10-06T05:31:48.910728+0000","last_peered":"2023-10-06T05:31:48.910728+0000","last_clean":"2023-10-06T05:31:48.910728+0000","last_became_active":"2023-10-06T05:31:48.903747+0000","last_became_peered":"2023-10-06T05:31:48.903747+0000","last_unstale":"2023-10-06T05:31:48.910728+0000","last_undegraded":"2023-10-06T05:31:48.910728+0000","last_fullsized":"2023-10-06T05:31:48.910728+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:21.613566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914526+0000","last_change":"2023-10-06T05:31:48.904932+0000","last_active":"2023-10-06T05:31:48.914526+0000","last_peered":"2023-10-06T05:31:48.914526+0000","last_clean":"2023-10-06T05:31:48.914526+0000","last_became_active":"2023-10-06T05:31:48.904645+0000","last_became_peered":"2023-10-06T05:31:48.904645+0000","last_unstale":"2023-10-06T05:31:48.914526+0000","last_undegraded":"2023-10-06T05:31:48.914526+0000","last_fullsized":"2023-10-06T05:31:48.914526+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:24.120897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914501+0000","last_change":"2023-10-06T05:31:48.911546+0000","last_active":"2023-10-06T05:31:48.914501+0000","last_peered":"2023-10-06T05:31:48.914501+0000","last_clean":"2023-10-06T05:31:48.914501+0000","last_became_active":"2023-10-06T05:31:48.910461+0000","last_became_peered":"2023-10-06T05:31:48.910461+0000","last_unstale":"2023-10-06T05:31:48.914501+0000","last_undegraded":"2023-10-06T05:31:48.914501+0000","last_fullsized":"2023-10-06T05:31:48.914501+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.296854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.919266+0000","last_change":"2023-10-06T05:31:48.918357+0000","last_active":"2023-10-06T05:31:48.919266+0000","last_peered":"2023-10-06T05:31:48.919266+0000","last_clean":"2023-10-06T05:31:48.919266+0000","last_became_active":"2023-10-06T05:31:48.918043+0000","last_became_peered":"2023-10-06T05:31:48.918043+0000","last_unstale":"2023-10-06T05:31:48.919266+0000","last_undegraded":"2023-10-06T05:31:48.919266+0000","last_fullsized":"2023-10-06T05:31:48.919266+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:06.772610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.918648+0000","last_change":"2023-10-06T05:31:48.918190+0000","last_active":"2023-10-06T05:31:48.918648+0000","last_peered":"2023-10-06T05:31:48.918648+0000","last_clean":"2023-10-06T05:31:48.918648+0000","last_became_active":"2023-10-06T05:31:48.917933+0000","last_became_peered":"2023-10-06T05:31:48.917933+0000","last_unstale":"2023-10-06T05:31:48.918648+0000","last_undegraded":"2023-10-06T05:31:48.918648+0000","last_fullsized":"2023-10-06T05:31:48.918648+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:42.546145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917401+0000","last_change":"2023-10-06T05:31:48.908226+0000","last_active":"2023-10-06T05:31:48.917401+0000","last_peered":"2023-10-06T05:31:48.917401+0000","last_clean":"2023-10-06T05:31:48.917401+0000","last_became_active":"2023-10-06T05:31:48.907843+0000","last_became_peered":"2023-10-06T05:31:48.907843+0000","last_unstale":"2023-10-06T05:31:48.917401+0000","last_undegraded":"2023-10-06T05:31:48.917401+0000","last_fullsized":"2023-10-06T05:31:48.917401+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:31.888428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.905955+0000","last_change":"2023-10-06T05:31:48.899190+0000","last_active":"2023-10-06T05:31:48.905955+0000","last_peered":"2023-10-06T05:31:48.905955+0000","last_clean":"2023-10-06T05:31:48.905955+0000","last_became_active":"2023-10-06T05:31:48.898903+0000","last_became_peered":"2023-10-06T05:31:48.898903+0000","last_unstale":"2023-10-06T05:31:48.905955+0000","last_undegraded":"2023-10-06T05:31:48.905955+0000","last_fullsized":"2023-10-06T05:31:48.905955+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:09.835703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914843+0000","last_change":"2023-10-06T05:31:48.912805+0000","last_active":"2023-10-06T05:31:48.914843+0000","last_peered":"2023-10-06T05:31:48.914843+0000","last_clean":"2023-10-06T05:31:48.914843+0000","last_became_active":"2023-10-06T05:31:48.912534+0000","last_became_peered":"2023-10-06T05:31:48.912534+0000","last_unstale":"2023-10-06T05:31:48.914843+0000","last_undegraded":"2023-10-06T05:31:48.914843+0000","last_fullsized":"2023-10-06T05:31:48.914843+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:47.386588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916253+0000","last_change":"2023-10-06T05:31:48.912841+0000","last_active":"2023-10-06T05:31:48.916253+0000","last_peered":"2023-10-06T05:31:48.916253+0000","last_clean":"2023-10-06T05:31:48.916253+0000","last_became_active":"2023-10-06T05:31:48.912545+0000","last_became_peered":"2023-10-06T05:31:48.912545+0000","last_unstale":"2023-10-06T05:31:48.916253+0000","last_undegraded":"2023-10-06T05:31:48.916253+0000","last_fullsized":"2023-10-06T05:31:48.916253+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:13.302591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907129+0000","last_change":"2023-10-06T05:31:48.903262+0000","last_active":"2023-10-06T05:31:48.907129+0000","last_peered":"2023-10-06T05:31:48.907129+0000","last_clean":"2023-10-06T05:31:48.907129+0000","last_became_active":"2023-10-06T05:31:48.902983+0000","last_became_peered":"2023-10-06T05:31:48.902983+0000","last_unstale":"2023-10-06T05:31:48.907129+0000","last_undegraded":"2023-10-06T05:31:48.907129+0000","last_fullsized":"2023-10-06T05:31:48.907129+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:45.153641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917614+0000","last_change":"2023-10-06T05:31:48.912688+0000","last_active":"2023-10-06T05:31:48.917614+0000","last_peered":"2023-10-06T05:31:48.917614+0000","last_clean":"2023-10-06T05:31:48.917614+0000","last_became_active":"2023-10-06T05:31:48.912420+0000","last_became_peered":"2023-10-06T05:31:48.912420+0000","last_unstale":"2023-10-06T05:31:48.917614+0000","last_undegraded":"2023-10-06T05:31:48.917614+0000","last_fullsized":"2023-10-06T05:31:48.917614+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:51.624561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915899+0000","last_change":"2023-10-06T05:31:48.906293+0000","last_active":"2023-10-06T05:31:48.915899+0000","last_peered":"2023-10-06T05:31:48.915899+0000","last_clean":"2023-10-06T05:31:48.915899+0000","last_became_active":"2023-10-06T05:31:48.906015+0000","last_became_peered":"2023-10-06T05:31:48.906015+0000","last_unstale":"2023-10-06T05:31:48.915899+0000","last_undegraded":"2023-10-06T05:31:48.915899+0000","last_fullsized":"2023-10-06T05:31:48.915899+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:01.008339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911100+0000","last_change":"2023-10-06T05:31:48.906609+0000","last_active":"2023-10-06T05:31:48.911100+0000","last_peered":"2023-10-06T05:31:48.911100+0000","last_clean":"2023-10-06T05:31:48.911100+0000","last_became_active":"2023-10-06T05:31:48.906323+0000","last_became_peered":"2023-10-06T05:31:48.906323+0000","last_unstale":"2023-10-06T05:31:48.911100+0000","last_undegraded":"2023-10-06T05:31:48.911100+0000","last_fullsized":"2023-10-06T05:31:48.911100+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:29.118160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908107+0000","last_change":"2023-10-06T05:31:48.903980+0000","last_active":"2023-10-06T05:31:48.908107+0000","last_peered":"2023-10-06T05:31:48.908107+0000","last_clean":"2023-10-06T05:31:48.908107+0000","last_became_active":"2023-10-06T05:31:48.903624+0000","last_became_peered":"2023-10-06T05:31:48.903624+0000","last_unstale":"2023-10-06T05:31:48.908107+0000","last_undegraded":"2023-10-06T05:31:48.908107+0000","last_fullsized":"2023-10-06T05:31:48.908107+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:25.209531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917471+0000","last_change":"2023-10-06T05:31:48.915364+0000","last_active":"2023-10-06T05:31:48.917471+0000","last_peered":"2023-10-06T05:31:48.917471+0000","last_clean":"2023-10-06T05:31:48.917471+0000","last_became_active":"2023-10-06T05:31:48.915077+0000","last_became_peered":"2023-10-06T05:31:48.915077+0000","last_unstale":"2023-10-06T05:31:48.917471+0000","last_undegraded":"2023-10-06T05:31:48.917471+0000","last_fullsized":"2023-10-06T05:31:48.917471+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:21.147549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909170+0000","last_change":"2023-10-06T05:31:48.907670+0000","last_active":"2023-10-06T05:31:48.909170+0000","last_peered":"2023-10-06T05:31:48.909170+0000","last_clean":"2023-10-06T05:31:48.909170+0000","last_became_active":"2023-10-06T05:31:48.907295+0000","last_became_peered":"2023-10-06T05:31:48.907295+0000","last_unstale":"2023-10-06T05:31:48.909170+0000","last_undegraded":"2023-10-06T05:31:48.909170+0000","last_fullsized":"2023-10-06T05:31:48.909170+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:20.520473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912904+0000","last_change":"2023-10-06T05:31:48.910558+0000","last_active":"2023-10-06T05:31:48.912904+0000","last_peered":"2023-10-06T05:31:48.912904+0000","last_clean":"2023-10-06T05:31:48.912904+0000","last_became_active":"2023-10-06T05:31:48.910184+0000","last_became_peered":"2023-10-06T05:31:48.910184+0000","last_unstale":"2023-10-06T05:31:48.912904+0000","last_undegraded":"2023-10-06T05:31:48.912904+0000","last_fullsized":"2023-10-06T05:31:48.912904+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:16.078845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915757+0000","last_change":"2023-10-06T05:31:48.911578+0000","last_active":"2023-10-06T05:31:48.915757+0000","last_peered":"2023-10-06T05:31:48.915757+0000","last_clean":"2023-10-06T05:31:48.915757+0000","last_became_active":"2023-10-06T05:31:48.911185+0000","last_became_peered":"2023-10-06T05:31:48.911185+0000","last_unstale":"2023-10-06T05:31:48.915757+0000","last_undegraded":"2023-10-06T05:31:48.915757+0000","last_fullsized":"2023-10-06T05:31:48.915757+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:05.302502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914042+0000","last_change":"2023-10-06T05:31:48.910903+0000","last_active":"2023-10-06T05:31:48.914042+0000","last_peered":"2023-10-06T05:31:48.914042+0000","last_clean":"2023-10-06T05:31:48.914042+0000","last_became_active":"2023-10-06T05:31:48.910634+0000","last_became_peered":"2023-10-06T05:31:48.910634+0000","last_unstale":"2023-10-06T05:31:48.914042+0000","last_undegraded":"2023-10-06T05:31:48.914042+0000","last_fullsized":"2023-10-06T05:31:48.914042+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:08.340788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"238'4","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.960353+0000","last_change":"2023-10-06T05:31:47.952906+0000","last_active":"2023-10-06T05:31:48.960353+0000","last_peered":"2023-10-06T05:31:48.960353+0000","last_clean":"2023-10-06T05:31:48.960353+0000","last_became_active":"2023-10-06T05:31:47.952564+0000","last_became_peered":"2023-10-06T05:31:47.952564+0000","last_unstale":"2023-10-06T05:31:48.960353+0000","last_undegraded":"2023-10-06T05:31:48.960353+0000","last_fullsized":"2023-10-06T05:31:48.960353+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:05.046935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912886+0000","last_change":"2023-10-06T05:31:48.910160+0000","last_active":"2023-10-06T05:31:48.912886+0000","last_peered":"2023-10-06T05:31:48.912886+0000","last_clean":"2023-10-06T05:31:48.912886+0000","last_became_active":"2023-10-06T05:31:48.909807+0000","last_became_peered":"2023-10-06T05:31:48.909807+0000","last_unstale":"2023-10-06T05:31:48.912886+0000","last_undegraded":"2023-10-06T05:31:48.912886+0000","last_fullsized":"2023-10-06T05:31:48.912886+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:59.724009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959887+0000","last_change":"2023-10-06T05:31:47.945386+0000","last_active":"2023-10-06T05:31:48.959887+0000","last_peered":"2023-10-06T05:31:48.959887+0000","last_clean":"2023-10-06T05:31:48.959887+0000","last_became_active":"2023-10-06T05:31:47.945103+0000","last_became_peered":"2023-10-06T05:31:47.945103+0000","last_unstale":"2023-10-06T05:31:48.959887+0000","last_undegraded":"2023-10-06T05:31:48.959887+0000","last_fullsized":"2023-10-06T05:31:48.959887+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:31.300931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914337+0000","last_change":"2023-10-06T05:31:48.904809+0000","last_active":"2023-10-06T05:31:48.914337+0000","last_peered":"2023-10-06T05:31:48.914337+0000","last_clean":"2023-10-06T05:31:48.914337+0000","last_became_active":"2023-10-06T05:31:48.904516+0000","last_became_peered":"2023-10-06T05:31:48.904516+0000","last_unstale":"2023-10-06T05:31:48.914337+0000","last_undegraded":"2023-10-06T05:31:48.914337+0000","last_fullsized":"2023-10-06T05:31:48.914337+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:41.441891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959427+0000","last_change":"2023-10-06T05:31:47.944906+0000","last_active":"2023-10-06T05:31:48.959427+0000","last_peered":"2023-10-06T05:31:48.959427+0000","last_clean":"2023-10-06T05:31:48.959427+0000","last_became_active":"2023-10-06T05:31:47.944638+0000","last_became_peered":"2023-10-06T05:31:47.944638+0000","last_unstale":"2023-10-06T05:31:48.959427+0000","last_undegraded":"2023-10-06T05:31:48.959427+0000","last_fullsized":"2023-10-06T05:31:48.959427+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:33.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912467+0000","last_change":"2023-10-06T05:31:48.910974+0000","last_active":"2023-10-06T05:31:48.912467+0000","last_peered":"2023-10-06T05:31:48.912467+0000","last_clean":"2023-10-06T05:31:48.912467+0000","last_became_active":"2023-10-06T05:31:48.910675+0000","last_became_peered":"2023-10-06T05:31:48.910675+0000","last_unstale":"2023-10-06T05:31:48.912467+0000","last_undegraded":"2023-10-06T05:31:48.912467+0000","last_fullsized":"2023-10-06T05:31:48.912467+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:14.525280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959739+0000","last_change":"2023-10-06T05:31:47.944302+0000","last_active":"2023-10-06T05:31:48.959739+0000","last_peered":"2023-10-06T05:31:48.959739+0000","last_clean":"2023-10-06T05:31:48.959739+0000","last_became_active":"2023-10-06T05:31:47.944022+0000","last_became_peered":"2023-10-06T05:31:47.944022+0000","last_unstale":"2023-10-06T05:31:48.959739+0000","last_undegraded":"2023-10-06T05:31:48.959739+0000","last_fullsized":"2023-10-06T05:31:48.959739+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:56.906395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915929+0000","last_change":"2023-10-06T05:31:48.906579+0000","last_active":"2023-10-06T05:31:48.915929+0000","last_peered":"2023-10-06T05:31:48.915929+0000","last_clean":"2023-10-06T05:31:48.915929+0000","last_became_active":"2023-10-06T05:31:48.906306+0000","last_became_peered":"2023-10-06T05:31:48.906306+0000","last_unstale":"2023-10-06T05:31:48.915929+0000","last_undegraded":"2023-10-06T05:31:48.915929+0000","last_fullsized":"2023-10-06T05:31:48.915929+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:21.060836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959027+0000","last_change":"2023-10-06T05:31:47.954390+0000","last_active":"2023-10-06T05:31:48.959027+0000","last_peered":"2023-10-06T05:31:48.959027+0000","last_clean":"2023-10-06T05:31:48.959027+0000","last_became_active":"2023-10-06T05:31:47.953253+0000","last_became_peered":"2023-10-06T05:31:47.953253+0000","last_unstale":"2023-10-06T05:31:48.959027+0000","last_undegraded":"2023-10-06T05:31:48.959027+0000","last_fullsized":"2023-10-06T05:31:48.959027+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:42.083066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.913003+0000","last_change":"2023-10-06T05:31:48.908592+0000","last_active":"2023-10-06T05:31:48.913003+0000","last_peered":"2023-10-06T05:31:48.913003+0000","last_clean":"2023-10-06T05:31:48.913003+0000","last_became_active":"2023-10-06T05:31:48.908333+0000","last_became_peered":"2023-10-06T05:31:48.908333+0000","last_unstale":"2023-10-06T05:31:48.913003+0000","last_undegraded":"2023-10-06T05:31:48.913003+0000","last_fullsized":"2023-10-06T05:31:48.913003+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:08.610133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.958935+0000","last_change":"2023-10-06T05:31:47.957431+0000","last_active":"2023-10-06T05:31:48.958935+0000","last_peered":"2023-10-06T05:31:48.958935+0000","last_clean":"2023-10-06T05:31:48.958935+0000","last_became_active":"2023-10-06T05:31:47.957167+0000","last_became_peered":"2023-10-06T05:31:47.957167+0000","last_unstale":"2023-10-06T05:31:48.958935+0000","last_undegraded":"2023-10-06T05:31:48.958935+0000","last_fullsized":"2023-10-06T05:31:48.958935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:26.909628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916977+0000","last_change":"2023-10-06T05:31:48.915563+0000","last_active":"2023-10-06T05:31:48.916977+0000","last_peered":"2023-10-06T05:31:48.916977+0000","last_clean":"2023-10-06T05:31:48.916977+0000","last_became_active":"2023-10-06T05:31:48.915310+0000","last_became_peered":"2023-10-06T05:31:48.915310+0000","last_unstale":"2023-10-06T05:31:48.916977+0000","last_undegraded":"2023-10-06T05:31:48.916977+0000","last_fullsized":"2023-10-06T05:31:48.916977+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:38.983840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.891661+0000","last_change":"2023-10-06T05:31:47.944456+0000","last_active":"2023-10-06T05:31:48.891661+0000","last_peered":"2023-10-06T05:31:48.891661+0000","last_clean":"2023-10-06T05:31:48.891661+0000","last_became_active":"2023-10-06T05:31:47.944186+0000","last_became_peered":"2023-10-06T05:31:47.944186+0000","last_unstale":"2023-10-06T05:31:48.891661+0000","last_undegraded":"2023-10-06T05:31:48.891661+0000","last_fullsized":"2023-10-06T05:31:48.891661+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:52.951970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908134+0000","last_change":"2023-10-06T05:31:48.905363+0000","last_active":"2023-10-06T05:31:48.908134+0000","last_peered":"2023-10-06T05:31:48.908134+0000","last_clean":"2023-10-06T05:31:48.908134+0000","last_became_active":"2023-10-06T05:31:48.905055+0000","last_became_peered":"2023-10-06T05:31:48.905055+0000","last_unstale":"2023-10-06T05:31:48.908134+0000","last_undegraded":"2023-10-06T05:31:48.908134+0000","last_fullsized":"2023-10-06T05:31:48.908134+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:09.169488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.891264+0000","last_change":"2023-10-06T05:31:47.954190+0000","last_active":"2023-10-06T05:31:48.891264+0000","last_peered":"2023-10-06T05:31:48.891264+0000","last_clean":"2023-10-06T05:31:48.891264+0000","last_became_active":"2023-10-06T05:31:47.953860+0000","last_became_peered":"2023-10-06T05:31:47.953860+0000","last_unstale":"2023-10-06T05:31:48.891264+0000","last_undegraded":"2023-10-06T05:31:48.891264+0000","last_fullsized":"2023-10-06T05:31:48.891264+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:50.959815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.905989+0000","last_change":"2023-10-06T05:31:48.899990+0000","last_active":"2023-10-06T05:31:48.905989+0000","last_peered":"2023-10-06T05:31:48.905989+0000","last_clean":"2023-10-06T05:31:48.905989+0000","last_became_active":"2023-10-06T05:31:48.899738+0000","last_became_peered":"2023-10-06T05:31:48.899738+0000","last_unstale":"2023-10-06T05:31:48.905989+0000","last_undegraded":"2023-10-06T05:31:48.905989+0000","last_fullsized":"2023-10-06T05:31:48.905989+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:30.977276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897705+0000","last_change":"2023-10-06T05:31:47.945250+0000","last_active":"2023-10-06T05:31:48.897705+0000","last_peered":"2023-10-06T05:31:48.897705+0000","last_clean":"2023-10-06T05:31:48.897705+0000","last_became_active":"2023-10-06T05:31:47.944998+0000","last_became_peered":"2023-10-06T05:31:47.944998+0000","last_unstale":"2023-10-06T05:31:48.897705+0000","last_undegraded":"2023-10-06T05:31:48.897705+0000","last_fullsized":"2023-10-06T05:31:48.897705+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:01.493691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917660+0000","last_change":"2023-10-06T05:31:48.911796+0000","last_active":"2023-10-06T05:31:48.917660+0000","last_peered":"2023-10-06T05:31:48.917660+0000","last_clean":"2023-10-06T05:31:48.917660+0000","last_became_active":"2023-10-06T05:31:48.909460+0000","last_became_peered":"2023-10-06T05:31:48.909460+0000","last_unstale":"2023-10-06T05:31:48.917660+0000","last_undegraded":"2023-10-06T05:31:48.917660+0000","last_fullsized":"2023-10-06T05:31:48.917660+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:09.029152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892617+0000","last_change":"2023-10-06T05:31:47.945842+0000","last_active":"2023-10-06T05:31:48.892617+0000","last_peered":"2023-10-06T05:31:48.892617+0000","last_clean":"2023-10-06T05:31:48.892617+0000","last_became_active":"2023-10-06T05:31:47.945515+0000","last_became_peered":"2023-10-06T05:31:47.945515+0000","last_unstale":"2023-10-06T05:31:48.892617+0000","last_undegraded":"2023-10-06T05:31:48.892617+0000","last_fullsized":"2023-10-06T05:31:48.892617+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.266037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916282+0000","last_change":"2023-10-06T05:31:48.911006+0000","last_active":"2023-10-06T05:31:48.916282+0000","last_peered":"2023-10-06T05:31:48.916282+0000","last_clean":"2023-10-06T05:31:48.916282+0000","last_became_active":"2023-10-06T05:31:48.910045+0000","last_became_peered":"2023-10-06T05:31:48.910045+0000","last_unstale":"2023-10-06T05:31:48.916282+0000","last_undegraded":"2023-10-06T05:31:48.916282+0000","last_fullsized":"2023-10-06T05:31:48.916282+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:01.628857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"238'3","reported_seq":13,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.958498+0000","last_change":"2023-10-06T05:31:47.954873+0000","last_active":"2023-10-06T05:31:48.958498+0000","last_peered":"2023-10-06T05:31:48.958498+0000","last_clean":"2023-10-06T05:31:48.958498+0000","last_became_active":"2023-10-06T05:31:47.954543+0000","last_became_peered":"2023-10-06T05:31:47.954543+0000","last_unstale":"2023-10-06T05:31:48.958498+0000","last_undegraded":"2023-10-06T05:31:48.958498+0000","last_fullsized":"2023-10-06T05:31:48.958498+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.266037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.918070+0000","last_change":"2023-10-06T05:31:48.917169+0000","last_active":"2023-10-06T05:31:48.918070+0000","last_peered":"2023-10-06T05:31:48.918070+0000","last_clean":"2023-10-06T05:31:48.918070+0000","last_became_active":"2023-10-06T05:31:48.916852+0000","last_became_peered":"2023-10-06T05:31:48.916852+0000","last_unstale":"2023-10-06T05:31:48.918070+0000","last_undegraded":"2023-10-06T05:31:48.918070+0000","last_fullsized":"2023-10-06T05:31:48.918070+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:21.613566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"238'6","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.963637+0000","last_change":"2023-10-06T05:31:47.955074+0000","last_active":"2023-10-06T05:31:48.963637+0000","last_peered":"2023-10-06T05:31:48.963637+0000","last_clean":"2023-10-06T05:31:48.963637+0000","last_became_active":"2023-10-06T05:31:47.954190+0000","last_became_peered":"2023-10-06T05:31:47.954190+0000","last_unstale":"2023-10-06T05:31:48.963637+0000","last_undegraded":"2023-10-06T05:31:48.963637+0000","last_fullsized":"2023-10-06T05:31:48.963637+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:39.937926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909059+0000","last_change":"2023-10-06T05:31:48.901775+0000","last_active":"2023-10-06T05:31:48.909059+0000","last_peered":"2023-10-06T05:31:48.909059+0000","last_clean":"2023-10-06T05:31:48.909059+0000","last_became_active":"2023-10-06T05:31:48.901402+0000","last_became_peered":"2023-10-06T05:31:48.901402+0000","last_unstale":"2023-10-06T05:31:48.909059+0000","last_undegraded":"2023-10-06T05:31:48.909059+0000","last_fullsized":"2023-10-06T05:31:48.909059+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:35.249864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.961746+0000","last_change":"2023-10-06T05:31:47.955833+0000","last_active":"2023-10-06T05:31:48.961746+0000","last_peered":"2023-10-06T05:31:48.961746+0000","last_clean":"2023-10-06T05:31:48.961746+0000","last_became_active":"2023-10-06T05:31:47.955574+0000","last_became_peered":"2023-10-06T05:31:47.955574+0000","last_unstale":"2023-10-06T05:31:48.961746+0000","last_undegraded":"2023-10-06T05:31:48.961746+0000","last_fullsized":"2023-10-06T05:31:48.961746+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:27.270314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.903820+0000","last_change":"2023-10-06T05:31:48.897129+0000","last_active":"2023-10-06T05:31:48.903820+0000","last_peered":"2023-10-06T05:31:48.903820+0000","last_clean":"2023-10-06T05:31:48.903820+0000","last_became_active":"2023-10-06T05:31:48.896845+0000","last_became_peered":"2023-10-06T05:31:48.896845+0000","last_unstale":"2023-10-06T05:31:48.903820+0000","last_undegraded":"2023-10-06T05:31:48.903820+0000","last_fullsized":"2023-10-06T05:31:48.903820+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:56.470940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.895220+0000","last_change":"2023-10-06T05:31:47.939790+0000","last_active":"2023-10-06T05:31:48.895220+0000","last_peered":"2023-10-06T05:31:48.895220+0000","last_clean":"2023-10-06T05:31:48.895220+0000","last_became_active":"2023-10-06T05:31:47.939462+0000","last_became_peered":"2023-10-06T05:31:47.939462+0000","last_unstale":"2023-10-06T05:31:48.895220+0000","last_undegraded":"2023-10-06T05:31:48.895220+0000","last_fullsized":"2023-10-06T05:31:48.895220+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:12.115237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911037+0000","last_change":"2023-10-06T05:31:48.905732+0000","last_active":"2023-10-06T05:31:48.911037+0000","last_peered":"2023-10-06T05:31:48.911037+0000","last_clean":"2023-10-06T05:31:48.911037+0000","last_became_active":"2023-10-06T05:31:48.905259+0000","last_became_peered":"2023-10-06T05:31:48.905259+0000","last_unstale":"2023-10-06T05:31:48.911037+0000","last_undegraded":"2023-10-06T05:31:48.911037+0000","last_fullsized":"2023-10-06T05:31:48.911037+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:20.785238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894673+0000","last_change":"2023-10-06T05:31:47.947135+0000","last_active":"2023-10-06T05:31:48.894673+0000","last_peered":"2023-10-06T05:31:48.894673+0000","last_clean":"2023-10-06T05:31:48.894673+0000","last_became_active":"2023-10-06T05:31:47.946865+0000","last_became_peered":"2023-10-06T05:31:47.946865+0000","last_unstale":"2023-10-06T05:31:48.894673+0000","last_undegraded":"2023-10-06T05:31:48.894673+0000","last_fullsized":"2023-10-06T05:31:48.894673+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:29.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908114+0000","last_change":"2023-10-06T05:31:48.903789+0000","last_active":"2023-10-06T05:31:48.908114+0000","last_peered":"2023-10-06T05:31:48.908114+0000","last_clean":"2023-10-06T05:31:48.908114+0000","last_became_active":"2023-10-06T05:31:48.903500+0000","last_became_peered":"2023-10-06T05:31:48.903500+0000","last_unstale":"2023-10-06T05:31:48.908114+0000","last_undegraded":"2023-10-06T05:31:48.908114+0000","last_fullsized":"2023-10-06T05:31:48.908114+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:12.535465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"238'8","reported_seq":18,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.962892+0000","last_change":"2023-10-06T05:31:47.953208+0000","last_active":"2023-10-06T05:31:48.962892+0000","last_peered":"2023-10-06T05:31:48.962892+0000","last_clean":"2023-10-06T05:31:48.962892+0000","last_became_active":"2023-10-06T05:31:47.952770+0000","last_became_peered":"2023-10-06T05:31:47.952770+0000","last_unstale":"2023-10-06T05:31:48.962892+0000","last_undegraded":"2023-10-06T05:31:48.962892+0000","last_fullsized":"2023-10-06T05:31:48.962892+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:13.530112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907394+0000","last_change":"2023-10-06T05:31:48.902861+0000","last_active":"2023-10-06T05:31:48.907394+0000","last_peered":"2023-10-06T05:31:48.907394+0000","last_clean":"2023-10-06T05:31:48.907394+0000","last_became_active":"2023-10-06T05:31:48.902544+0000","last_became_peered":"2023-10-06T05:31:48.902544+0000","last_unstale":"2023-10-06T05:31:48.907394+0000","last_undegraded":"2023-10-06T05:31:48.907394+0000","last_fullsized":"2023-10-06T05:31:48.907394+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:26.419754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.960077+0000","last_change":"2023-10-06T05:31:47.945586+0000","last_active":"2023-10-06T05:31:48.960077+0000","last_peered":"2023-10-06T05:31:48.960077+0000","last_clean":"2023-10-06T05:31:48.960077+0000","last_became_active":"2023-10-06T05:31:47.945326+0000","last_became_peered":"2023-10-06T05:31:47.945326+0000","last_unstale":"2023-10-06T05:31:48.960077+0000","last_undegraded":"2023-10-06T05:31:48.960077+0000","last_fullsized":"2023-10-06T05:31:48.960077+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:00.801937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909646+0000","last_change":"2023-10-06T05:31:48.907128+0000","last_active":"2023-10-06T05:31:48.909646+0000","last_peered":"2023-10-06T05:31:48.909646+0000","last_clean":"2023-10-06T05:31:48.909646+0000","last_became_active":"2023-10-06T05:31:48.903132+0000","last_became_peered":"2023-10-06T05:31:48.903132+0000","last_unstale":"2023-10-06T05:31:48.909646+0000","last_undegraded":"2023-10-06T05:31:48.909646+0000","last_fullsized":"2023-10-06T05:31:48.909646+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:41.109366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.957184+0000","last_change":"2023-10-06T05:31:47.955383+0000","last_active":"2023-10-06T05:31:48.957184+0000","last_peered":"2023-10-06T05:31:48.957184+0000","last_clean":"2023-10-06T05:31:48.957184+0000","last_became_active":"2023-10-06T05:31:47.955043+0000","last_became_peered":"2023-10-06T05:31:47.955043+0000","last_unstale":"2023-10-06T05:31:48.957184+0000","last_undegraded":"2023-10-06T05:31:48.957184+0000","last_fullsized":"2023-10-06T05:31:48.957184+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:11.081466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908711+0000","last_change":"2023-10-06T05:31:48.906847+0000","last_active":"2023-10-06T05:31:48.908711+0000","last_peered":"2023-10-06T05:31:48.908711+0000","last_clean":"2023-10-06T05:31:48.908711+0000","last_became_active":"2023-10-06T05:31:48.906579+0000","last_became_peered":"2023-10-06T05:31:48.906579+0000","last_unstale":"2023-10-06T05:31:48.908711+0000","last_undegraded":"2023-10-06T05:31:48.908711+0000","last_fullsized":"2023-10-06T05:31:48.908711+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:14.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.958832+0000","last_change":"2023-10-06T05:31:47.953015+0000","last_active":"2023-10-06T05:31:48.958832+0000","last_peered":"2023-10-06T05:31:48.958832+0000","last_clean":"2023-10-06T05:31:48.958832+0000","last_became_active":"2023-10-06T05:31:47.952651+0000","last_became_peered":"2023-10-06T05:31:47.952651+0000","last_unstale":"2023-10-06T05:31:48.958832+0000","last_undegraded":"2023-10-06T05:31:48.958832+0000","last_fullsized":"2023-10-06T05:31:48.958832+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:03.260271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.903787+0000","last_change":"2023-10-06T05:31:48.898071+0000","last_active":"2023-10-06T05:31:48.903787+0000","last_peered":"2023-10-06T05:31:48.903787+0000","last_clean":"2023-10-06T05:31:48.903787+0000","last_became_active":"2023-10-06T05:31:48.897334+0000","last_became_peered":"2023-10-06T05:31:48.897334+0000","last_unstale":"2023-10-06T05:31:48.903787+0000","last_undegraded":"2023-10-06T05:31:48.903787+0000","last_fullsized":"2023-10-06T05:31:48.903787+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.482035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892325+0000","last_change":"2023-10-06T05:31:47.943786+0000","last_active":"2023-10-06T05:31:48.892325+0000","last_peered":"2023-10-06T05:31:48.892325+0000","last_clean":"2023-10-06T05:31:48.892325+0000","last_became_active":"2023-10-06T05:31:47.943458+0000","last_became_peered":"2023-10-06T05:31:47.943458+0000","last_unstale":"2023-10-06T05:31:48.892325+0000","last_undegraded":"2023-10-06T05:31:48.892325+0000","last_fullsized":"2023-10-06T05:31:48.892325+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:34.635886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915645+0000","last_change":"2023-10-06T05:31:48.911092+0000","last_active":"2023-10-06T05:31:48.915645+0000","last_peered":"2023-10-06T05:31:48.915645+0000","last_clean":"2023-10-06T05:31:48.915645+0000","last_became_active":"2023-10-06T05:31:48.910797+0000","last_became_peered":"2023-10-06T05:31:48.910797+0000","last_unstale":"2023-10-06T05:31:48.915645+0000","last_undegraded":"2023-10-06T05:31:48.915645+0000","last_fullsized":"2023-10-06T05:31:48.915645+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:13.405614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.961651+0000","last_change":"2023-10-06T05:31:47.945188+0000","last_active":"2023-10-06T05:31:48.961651+0000","last_peered":"2023-10-06T05:31:48.961651+0000","last_clean":"2023-10-06T05:31:48.961651+0000","last_became_active":"2023-10-06T05:31:47.944879+0000","last_became_peered":"2023-10-06T05:31:47.944879+0000","last_unstale":"2023-10-06T05:31:48.961651+0000","last_undegraded":"2023-10-06T05:31:48.961651+0000","last_fullsized":"2023-10-06T05:31:48.961651+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:02.072557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.910295+0000","last_change":"2023-10-06T05:31:48.907998+0000","last_active":"2023-10-06T05:31:48.910295+0000","last_peered":"2023-10-06T05:31:48.910295+0000","last_clean":"2023-10-06T05:31:48.910295+0000","last_became_active":"2023-10-06T05:31:48.907678+0000","last_became_peered":"2023-10-06T05:31:48.907678+0000","last_unstale":"2023-10-06T05:31:48.910295+0000","last_undegraded":"2023-10-06T05:31:48.910295+0000","last_fullsized":"2023-10-06T05:31:48.910295+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:21.147549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.889718+0000","last_change":"2023-10-06T05:31:47.956209+0000","last_active":"2023-10-06T05:31:48.889718+0000","last_peered":"2023-10-06T05:31:48.889718+0000","last_clean":"2023-10-06T05:31:48.889718+0000","last_became_active":"2023-10-06T05:31:47.955904+0000","last_became_peered":"2023-10-06T05:31:47.955904+0000","last_unstale":"2023-10-06T05:31:48.889718+0000","last_undegraded":"2023-10-06T05:31:48.889718+0000","last_fullsized":"2023-10-06T05:31:48.889718+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:29.242754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911014+0000","last_change":"2023-10-06T05:31:48.905545+0000","last_active":"2023-10-06T05:31:48.911014+0000","last_peered":"2023-10-06T05:31:48.911014+0000","last_clean":"2023-10-06T05:31:48.911014+0000","last_became_active":"2023-10-06T05:31:48.904927+0000","last_became_peered":"2023-10-06T05:31:48.904927+0000","last_unstale":"2023-10-06T05:31:48.911014+0000","last_undegraded":"2023-10-06T05:31:48.911014+0000","last_fullsized":"2023-10-06T05:31:48.911014+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:19.141959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894696+0000","last_change":"2023-10-06T05:31:47.941963+0000","last_active":"2023-10-06T05:31:48.894696+0000","last_peered":"2023-10-06T05:31:48.894696+0000","last_clean":"2023-10-06T05:31:48.894696+0000","last_became_active":"2023-10-06T05:31:47.941641+0000","last_became_peered":"2023-10-06T05:31:47.941641+0000","last_unstale":"2023-10-06T05:31:48.894696+0000","last_undegraded":"2023-10-06T05:31:48.894696+0000","last_fullsized":"2023-10-06T05:31:48.894696+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:51.696951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912193+0000","last_change":"2023-10-06T05:31:48.908603+0000","last_active":"2023-10-06T05:31:48.912193+0000","last_peered":"2023-10-06T05:31:48.912193+0000","last_clean":"2023-10-06T05:31:48.912193+0000","last_became_active":"2023-10-06T05:31:48.908349+0000","last_became_peered":"2023-10-06T05:31:48.908349+0000","last_unstale":"2023-10-06T05:31:48.912193+0000","last_undegraded":"2023-10-06T05:31:48.912193+0000","last_fullsized":"2023-10-06T05:31:48.912193+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:14.031440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.891847+0000","last_change":"2023-10-06T05:31:47.948871+0000","last_active":"2023-10-06T05:31:48.891847+0000","last_peered":"2023-10-06T05:31:48.891847+0000","last_clean":"2023-10-06T05:31:48.891847+0000","last_became_active":"2023-10-06T05:31:47.948041+0000","last_became_peered":"2023-10-06T05:31:47.948041+0000","last_unstale":"2023-10-06T05:31:48.891847+0000","last_undegraded":"2023-10-06T05:31:48.891847+0000","last_fullsized":"2023-10-06T05:31:48.891847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:29.801473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.906012+0000","last_change":"2023-10-06T05:31:48.900821+0000","last_active":"2023-10-06T05:31:48.906012+0000","last_peered":"2023-10-06T05:31:48.906012+0000","last_clean":"2023-10-06T05:31:48.906012+0000","last_became_active":"2023-10-06T05:31:48.900516+0000","last_became_peered":"2023-10-06T05:31:48.900516+0000","last_unstale":"2023-10-06T05:31:48.906012+0000","last_undegraded":"2023-10-06T05:31:48.906012+0000","last_fullsized":"2023-10-06T05:31:48.906012+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:05.050675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.960034+0000","last_change":"2023-10-06T05:31:47.951106+0000","last_active":"2023-10-06T05:31:48.960034+0000","last_peered":"2023-10-06T05:31:48.960034+0000","last_clean":"2023-10-06T05:31:48.960034+0000","last_became_active":"2023-10-06T05:31:47.950776+0000","last_became_peered":"2023-10-06T05:31:47.950776+0000","last_unstale":"2023-10-06T05:31:48.960034+0000","last_undegraded":"2023-10-06T05:31:48.960034+0000","last_fullsized":"2023-10-06T05:31:48.960034+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:01.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911072+0000","last_change":"2023-10-06T05:31:48.906545+0000","last_active":"2023-10-06T05:31:48.911072+0000","last_peered":"2023-10-06T05:31:48.911072+0000","last_clean":"2023-10-06T05:31:48.911072+0000","last_became_active":"2023-10-06T05:31:48.906259+0000","last_became_peered":"2023-10-06T05:31:48.906259+0000","last_unstale":"2023-10-06T05:31:48.911072+0000","last_undegraded":"2023-10-06T05:31:48.911072+0000","last_fullsized":"2023-10-06T05:31:48.911072+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894136+0000","last_change":"2023-10-06T05:31:47.940085+0000","last_active":"2023-10-06T05:31:48.894136+0000","last_peered":"2023-10-06T05:31:48.894136+0000","last_clean":"2023-10-06T05:31:48.894136+0000","last_became_active":"2023-10-06T05:31:47.939599+0000","last_became_peered":"2023-10-06T05:31:47.939599+0000","last_unstale":"2023-10-06T05:31:48.894136+0000","last_undegraded":"2023-10-06T05:31:48.894136+0000","last_fullsized":"2023-10-06T05:31:48.894136+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:00.001868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914616+0000","last_change":"2023-10-06T05:31:48.902026+0000","last_active":"2023-10-06T05:31:48.914616+0000","last_peered":"2023-10-06T05:31:48.914616+0000","last_clean":"2023-10-06T05:31:48.914616+0000","last_became_active":"2023-10-06T05:31:48.901752+0000","last_became_peered":"2023-10-06T05:31:48.901752+0000","last_unstale":"2023-10-06T05:31:48.914616+0000","last_undegraded":"2023-10-06T05:31:48.914616+0000","last_fullsized":"2023-10-06T05:31:48.914616+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:16.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909538+0000","last_change":"2023-10-06T05:31:47.943067+0000","last_active":"2023-10-06T05:31:48.909538+0000","last_peered":"2023-10-06T05:31:48.909538+0000","last_clean":"2023-10-06T05:31:48.909538+0000","last_became_active":"2023-10-06T05:31:47.942804+0000","last_became_peered":"2023-10-06T05:31:47.942804+0000","last_unstale":"2023-10-06T05:31:48.909538+0000","last_undegraded":"2023-10-06T05:31:48.909538+0000","last_fullsized":"2023-10-06T05:31:48.909538+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:54.641658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912446+0000","last_change":"2023-10-06T05:31:48.910323+0000","last_active":"2023-10-06T05:31:48.912446+0000","last_peered":"2023-10-06T05:31:48.912446+0000","last_clean":"2023-10-06T05:31:48.912446+0000","last_became_active":"2023-10-06T05:31:48.909960+0000","last_became_peered":"2023-10-06T05:31:48.909960+0000","last_unstale":"2023-10-06T05:31:48.912446+0000","last_undegraded":"2023-10-06T05:31:48.912446+0000","last_fullsized":"2023-10-06T05:31:48.912446+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:14.031440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.895210+0000","last_change":"2023-10-06T05:31:47.953179+0000","last_active":"2023-10-06T05:31:48.895210+0000","last_peered":"2023-10-06T05:31:48.895210+0000","last_clean":"2023-10-06T05:31:48.895210+0000","last_became_active":"2023-10-06T05:31:47.952898+0000","last_became_peered":"2023-10-06T05:31:47.952898+0000","last_unstale":"2023-10-06T05:31:48.895210+0000","last_undegraded":"2023-10-06T05:31:48.895210+0000","last_fullsized":"2023-10-06T05:31:48.895210+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:47.822702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912928+0000","last_change":"2023-10-06T05:31:48.910502+0000","last_active":"2023-10-06T05:31:48.912928+0000","last_peered":"2023-10-06T05:31:48.912928+0000","last_clean":"2023-10-06T05:31:48.912928+0000","last_became_active":"2023-10-06T05:31:48.909897+0000","last_became_peered":"2023-10-06T05:31:48.909897+0000","last_unstale":"2023-10-06T05:31:48.912928+0000","last_undegraded":"2023-10-06T05:31:48.912928+0000","last_fullsized":"2023-10-06T05:31:48.912928+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:23.209259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892653+0000","last_change":"2023-10-06T05:31:47.956512+0000","last_active":"2023-10-06T05:31:48.892653+0000","last_peered":"2023-10-06T05:31:48.892653+0000","last_clean":"2023-10-06T05:31:48.892653+0000","last_became_active":"2023-10-06T05:31:47.955998+0000","last_became_peered":"2023-10-06T05:31:47.955998+0000","last_unstale":"2023-10-06T05:31:48.892653+0000","last_undegraded":"2023-10-06T05:31:48.892653+0000","last_fullsized":"2023-10-06T05:31:48.892653+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:02.398914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914670+0000","last_change":"2023-10-06T05:31:48.905079+0000","last_active":"2023-10-06T05:31:48.914670+0000","last_peered":"2023-10-06T05:31:48.914670+0000","last_clean":"2023-10-06T05:31:48.914670+0000","last_became_active":"2023-10-06T05:31:48.904828+0000","last_became_peered":"2023-10-06T05:31:48.904828+0000","last_unstale":"2023-10-06T05:31:48.914670+0000","last_undegraded":"2023-10-06T05:31:48.914670+0000","last_fullsized":"2023-10-06T05:31:48.914670+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:48.829173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.890154+0000","last_change":"2023-10-06T05:31:47.954583+0000","last_active":"2023-10-06T05:31:48.890154+0000","last_peered":"2023-10-06T05:31:48.890154+0000","last_clean":"2023-10-06T05:31:48.890154+0000","last_became_active":"2023-10-06T05:31:47.954076+0000","last_became_peered":"2023-10-06T05:31:47.954076+0000","last_unstale":"2023-10-06T05:31:48.890154+0000","last_undegraded":"2023-10-06T05:31:48.890154+0000","last_fullsized":"2023-10-06T05:31:48.890154+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:50.673081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907464+0000","last_change":"2023-10-06T05:31:48.903640+0000","last_active":"2023-10-06T05:31:48.907464+0000","last_peered":"2023-10-06T05:31:48.907464+0000","last_clean":"2023-10-06T05:31:48.907464+0000","last_became_active":"2023-10-06T05:31:48.903355+0000","last_became_peered":"2023-10-06T05:31:48.903355+0000","last_unstale":"2023-10-06T05:31:48.907464+0000","last_undegraded":"2023-10-06T05:31:48.907464+0000","last_fullsized":"2023-10-06T05:31:48.907464+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.296854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897773+0000","last_change":"2023-10-06T05:31:47.947624+0000","last_active":"2023-10-06T05:31:48.897773+0000","last_peered":"2023-10-06T05:31:48.897773+0000","last_clean":"2023-10-06T05:31:48.897773+0000","last_became_active":"2023-10-06T05:31:47.947358+0000","last_became_peered":"2023-10-06T05:31:47.947358+0000","last_unstale":"2023-10-06T05:31:48.897773+0000","last_undegraded":"2023-10-06T05:31:48.897773+0000","last_fullsized":"2023-10-06T05:31:48.897773+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:34.635886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916102+0000","last_change":"2023-10-06T05:31:48.908449+0000","last_active":"2023-10-06T05:31:48.916102+0000","last_peered":"2023-10-06T05:31:48.916102+0000","last_clean":"2023-10-06T05:31:48.916102+0000","last_became_active":"2023-10-06T05:31:48.908161+0000","last_became_peered":"2023-10-06T05:31:48.908161+0000","last_unstale":"2023-10-06T05:31:48.916102+0000","last_undegraded":"2023-10-06T05:31:48.916102+0000","last_fullsized":"2023-10-06T05:31:48.916102+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.089537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892001+0000","last_change":"2023-10-06T05:31:47.946988+0000","last_active":"2023-10-06T05:31:48.892001+0000","last_peered":"2023-10-06T05:31:48.892001+0000","last_clean":"2023-10-06T05:31:48.892001+0000","last_became_active":"2023-10-06T05:31:47.946693+0000","last_became_peered":"2023-10-06T05:31:47.946693+0000","last_unstale":"2023-10-06T05:31:48.892001+0000","last_undegraded":"2023-10-06T05:31:48.892001+0000","last_fullsized":"2023-10-06T05:31:48.892001+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:47.788784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914509+0000","last_change":"2023-10-06T05:31:48.909483+0000","last_active":"2023-10-06T05:31:48.914509+0000","last_peered":"2023-10-06T05:31:48.914509+0000","last_clean":"2023-10-06T05:31:48.914509+0000","last_became_active":"2023-10-06T05:31:48.909226+0000","last_became_peered":"2023-10-06T05:31:48.909226+0000","last_unstale":"2023-10-06T05:31:48.914509+0000","last_undegraded":"2023-10-06T05:31:48.914509+0000","last_fullsized":"2023-10-06T05:31:48.914509+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:33.616519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897201+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:31:48.897201+0000","last_peered":"2023-10-06T05:31:48.897201+0000","last_clean":"2023-10-06T05:31:48.897201+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:31:48.897201+0000","last_undegraded":"2023-10-06T05:31:48.897201+0000","last_fullsized":"2023-10-06T05:31:48.897201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916825+0000","last_change":"2023-10-06T05:31:48.909298+0000","last_active":"2023-10-06T05:31:48.916825+0000","last_peered":"2023-10-06T05:31:48.916825+0000","last_clean":"2023-10-06T05:31:48.916825+0000","last_became_active":"2023-10-06T05:31:48.909047+0000","last_became_peered":"2023-10-06T05:31:48.909047+0000","last_unstale":"2023-10-06T05:31:48.916825+0000","last_undegraded":"2023-10-06T05:31:48.916825+0000","last_fullsized":"2023-10-06T05:31:48.916825+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:08.512823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":191,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892157+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:31:48.892157+0000","last_peered":"2023-10-06T05:31:48.892157+0000","last_clean":"2023-10-06T05:31:48.892157+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:31:48.892157+0000","last_undegraded":"2023-10-06T05:31:48.892157+0000","last_fullsized":"2023-10-06T05:31:48.892157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909668+0000","last_change":"2023-10-06T05:31:48.903122+0000","last_active":"2023-10-06T05:31:48.909668+0000","last_peered":"2023-10-06T05:31:48.909668+0000","last_clean":"2023-10-06T05:31:48.909668+0000","last_became_active":"2023-10-06T05:31:48.902268+0000","last_became_peered":"2023-10-06T05:31:48.902268+0000","last_unstale":"2023-10-06T05:31:48.909668+0000","last_undegraded":"2023-10-06T05:31:48.909668+0000","last_fullsized":"2023-10-06T05:31:48.909668+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:56.958369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":195,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.890264+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:31:48.890264+0000","last_peered":"2023-10-06T05:31:48.890264+0000","last_clean":"2023-10-06T05:31:48.890264+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:31:48.890264+0000","last_undegraded":"2023-10-06T05:31:48.890264+0000","last_fullsized":"2023-10-06T05:31:48.890264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.903827+0000","last_change":"2023-10-06T05:31:48.900601+0000","last_active":"2023-10-06T05:31:48.903827+0000","last_peered":"2023-10-06T05:31:48.903827+0000","last_clean":"2023-10-06T05:31:48.903827+0000","last_became_active":"2023-10-06T05:31:48.900332+0000","last_became_peered":"2023-10-06T05:31:48.900332+0000","last_unstale":"2023-10-06T05:31:48.903827+0000","last_undegraded":"2023-10-06T05:31:48.903827+0000","last_fullsized":"2023-10-06T05:31:48.903827+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:39.239971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":191,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.893521+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:31:48.893521+0000","last_peered":"2023-10-06T05:31:48.893521+0000","last_clean":"2023-10-06T05:31:48.893521+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:31:48.893521+0000","last_undegraded":"2023-10-06T05:31:48.893521+0000","last_fullsized":"2023-10-06T05:31:48.893521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":340,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.899096+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:31:48.899096+0000","last_peered":"2023-10-06T05:31:48.899096+0000","last_clean":"2023-10-06T05:31:48.899096+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:31:48.899096+0000","last_undegraded":"2023-10-06T05:31:48.899096+0000","last_fullsized":"2023-10-06T05:31:48.899096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907489+0000","last_change":"2023-10-06T05:31:48.904216+0000","last_active":"2023-10-06T05:31:48.907489+0000","last_peered":"2023-10-06T05:31:48.907489+0000","last_clean":"2023-10-06T05:31:48.907489+0000","last_became_active":"2023-10-06T05:31:48.903941+0000","last_became_peered":"2023-10-06T05:31:48.903941+0000","last_unstale":"2023-10-06T05:31:48.907489+0000","last_undegraded":"2023-10-06T05:31:48.907489+0000","last_fullsized":"2023-10-06T05:31:48.907489+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:47.386588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.899015+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:31:48.899015+0000","last_peered":"2023-10-06T05:31:48.899015+0000","last_clean":"2023-10-06T05:31:48.899015+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:31:48.899015+0000","last_undegraded":"2023-10-06T05:31:48.899015+0000","last_fullsized":"2023-10-06T05:31:48.899015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909647+0000","last_change":"2023-10-06T05:31:48.902910+0000","last_active":"2023-10-06T05:31:48.909647+0000","last_peered":"2023-10-06T05:31:48.909647+0000","last_clean":"2023-10-06T05:31:48.909647+0000","last_became_active":"2023-10-06T05:31:48.902281+0000","last_became_peered":"2023-10-06T05:31:48.902281+0000","last_unstale":"2023-10-06T05:31:48.909647+0000","last_undegraded":"2023-10-06T05:31:48.909647+0000","last_fullsized":"2023-10-06T05:31:48.909647+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:09.147475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.898643+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:31:48.898643+0000","last_peered":"2023-10-06T05:31:48.898643+0000","last_clean":"2023-10-06T05:31:48.898643+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:31:48.898643+0000","last_undegraded":"2023-10-06T05:31:48.898643+0000","last_fullsized":"2023-10-06T05:31:48.898643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916057+0000","last_change":"2023-10-06T05:31:48.909018+0000","last_active":"2023-10-06T05:31:48.916057+0000","last_peered":"2023-10-06T05:31:48.916057+0000","last_clean":"2023-10-06T05:31:48.916057+0000","last_became_active":"2023-10-06T05:31:48.908747+0000","last_became_peered":"2023-10-06T05:31:48.908747+0000","last_unstale":"2023-10-06T05:31:48.916057+0000","last_undegraded":"2023-10-06T05:31:48.916057+0000","last_fullsized":"2023-10-06T05:31:48.916057+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:44.300785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":193,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897439+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:31:48.897439+0000","last_peered":"2023-10-06T05:31:48.897439+0000","last_clean":"2023-10-06T05:31:48.897439+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:31:48.897439+0000","last_undegraded":"2023-10-06T05:31:48.897439+0000","last_fullsized":"2023-10-06T05:31:48.897439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909628+0000","last_change":"2023-10-06T05:31:48.901743+0000","last_active":"2023-10-06T05:31:48.909628+0000","last_peered":"2023-10-06T05:31:48.909628+0000","last_clean":"2023-10-06T05:31:48.909628+0000","last_became_active":"2023-10-06T05:31:48.901339+0000","last_became_peered":"2023-10-06T05:31:48.901339+0000","last_unstale":"2023-10-06T05:31:48.909628+0000","last_undegraded":"2023-10-06T05:31:48.909628+0000","last_fullsized":"2023-10-06T05:31:48.909628+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:39.715978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":226,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894730+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:31:48.894730+0000","last_peered":"2023-10-06T05:31:48.894730+0000","last_clean":"2023-10-06T05:31:48.894730+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:31:48.894730+0000","last_undegraded":"2023-10-06T05:31:48.894730+0000","last_fullsized":"2023-10-06T05:31:48.894730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575065,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304940,"kb_used_data":3556,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066900,"statfs":{"total":96636764160,"available":96324505600,"internally_reserved":0,"allocated":3641344,"data_stored":2011701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.435,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.44400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":1.03,"15min":1.169},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38900000000000001},"max":{"1min":0.84099999999999997,"5min":1.0860000000000001,"15min":1.268},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.47999999999999998,"5min":0.44500000000000001,"15min":0.38300000000000001},"max":{"1min":0.91100000000000003,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83099999999999996,"5min":1.163,"15min":1.163},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0409999999999999,"15min":1.22},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":0.90500000000000003,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.55600000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.117},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.623,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":1.109,"15min":1.258},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.73999999999999999},"min":{"1min":0.53200000000000003,"5min":0.501,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.234,"15min":1.2849999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.51100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.109},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.57299999999999995,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.71799999999999997},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.169,"15min":1.169},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.73499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.242,"15min":1.242},"last":0.94999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304340,"kb_used_data":2956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067500,"statfs":{"total":96636764160,"available":96325120000,"internally_reserved":0,"allocated":3026944,"data_stored":1419173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90800000000000003,"5min":1.044,"15min":1.0720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.42399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.79900000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":1.137,"15min":1.137},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.135},"last":0.91300000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.442},"max":{"1min":1.075,"5min":1.115,"15min":1.115},"last":0.98599999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.70899999999999996},"min":{"1min":0.55000000000000004,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":1.1160000000000001},"last":0.875},{"interface":"front","average":{"1min":0.755,"5min":0.752,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.504,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.0049999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.53900000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91600000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.70999999999999996},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.4610000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.53900000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.995,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304344,"kb_used_data":2960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067496,"statfs":{"total":96636764160,"available":96325115904,"internally_reserved":0,"allocated":3031040,"data_stored":1419263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.84699999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64400000000000002,"5min":0.86099999999999999,"15min":0.88100000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72199999999999998,"5min":1.165,"15min":1.165},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.70699999999999996,"5min":1.077,"15min":1.077},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81100000000000005,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.75600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.024,"5min":1.111,"15min":1.5389999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.52300000000000002,"15min":0.45200000000000001},"max":{"1min":1.0089999999999999,"5min":1.119,"15min":1.4690000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.47899999999999998},"max":{"1min":1.093,"5min":1.248,"15min":1.25},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":1.26,"15min":1.26},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.48199999999999998},"max":{"1min":1.077,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.82099999999999995},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.1459999999999999,"5min":1.2849999999999999,"15min":1.3280000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.80400000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.46999999999999997},"max":{"1min":1.1599999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.621,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.1299999999999999,"5min":1.1879999999999999,"15min":2.6240000000000001},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575064,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309024,"kb_used_data":3544,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062816,"statfs":{"total":96636764160,"available":96320323584,"internally_reserved":0,"allocated":3629056,"data_stored":2010129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.245},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.85199999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.498},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82099999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.504,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.70799999999999996,"5min":0.81899999999999995,"15min":0.90800000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.84999999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.58599999999999997,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.432},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.88400000000000001,"15min":0.90200000000000002},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.47299999999999998},"max":{"1min":0.997,"5min":1.05,"15min":2.8220000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.622,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.113,"5min":1.244,"15min":3.6629999999999998},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.65700000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"max":{"1min":1.1060000000000001,"5min":1.27,"15min":1.27},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.59499999999999997,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":1.0389999999999999,"5min":1.097,"15min":1.1970000000000001},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0580000000000001,"5min":1.093,"15min":1.1839999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.45500000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.2629999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80000000000000004,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.25},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.66300000000000003,"5min":0.55500000000000005,"15min":0.50600000000000001},"max":{"1min":1.1459999999999999,"5min":1.167,"15min":1.2789999999999999},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304340,"kb_used_data":2956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067500,"statfs":{"total":96636764160,"available":96325120000,"internally_reserved":0,"allocated":3026944,"data_stored":1419173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.45200000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.056},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70999999999999996,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.0649999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":1.073},"last":0.75}]},{"osd":3,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.96499999999999997},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.486,"15min":1.486},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.76900000000000002},"min":{"1min":0.62,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":1.5,"15min":1.5},"last":0.997}]},{"osd":5,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74399999999999999,"15min":0.76000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1639999999999999},"last":1.093},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.79200000000000004},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1859999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.626,"5min":0.504,"15min":0.504},"max":{"1min":1.014,"5min":1.014,"15min":1.1830000000000001},"last":0.878},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.137},"last":1.145}]},{"osd":7,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78200000000000003,"15min":0.79700000000000004},"min":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.57499999999999996},"max":{"1min":1.0229999999999999,"5min":1.538,"15min":1.538},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.80500000000000005},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.51600000000000001},"max":{"1min":0.96199999999999997,"5min":1.014,"15min":1.2},"last":1.1870000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575065,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304348,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067492,"statfs":{"total":96636764160,"available":96325111808,"internally_reserved":0,"allocated":3035136,"data_stored":1419265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44400000000000001,"15min":0.45100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.254},"max":{"1min":0.60599999999999998,"5min":0.76700000000000002,"15min":1.0720000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29699999999999999},"max":{"1min":0.61699999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.66900000000000004,"5min":1.369,"15min":1.369},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.81599999999999995,"15min":0.90400000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.754,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.54700000000000004,"15min":0.46000000000000002},"max":{"1min":0.95699999999999996,"5min":1.1100000000000001,"15min":1.2030000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.754,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":1.101,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.002,"5min":1.111,"15min":1.1930000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.58399999999999996,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":1.0820000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.79400000000000004},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":1.022,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78800000000000003,"15min":0.79800000000000004},"min":{"1min":0.56299999999999994,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":0.98499999999999999,"5min":1.1339999999999999,"15min":1.3109999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57099999999999995},"max":{"1min":0.98499999999999999,"5min":1.1419999999999999,"15min":1.206},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.57899999999999996,"5min":0.53700000000000003,"15min":0.53000000000000003},"max":{"1min":0.93899999999999995,"5min":1.0980000000000001,"15min":1.2090000000000001},"last":0.84699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575063,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304356,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067484,"statfs":{"total":96636764160,"available":96325103616,"internally_reserved":0,"allocated":3043328,"data_stored":1420799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0249999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97299999999999998,"5min":1.0369999999999999,"15min":1.05},"last":0.747}]},{"osd":1,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.68700000000000006},"min":{"1min":0.52700000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.99099999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":0.98899999999999999,"15min":1.016},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70099999999999996,"15min":0.71599999999999997},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.89400000000000002,"5min":1.115,"15min":1.115},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.018},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":1.163,"15min":1.163},"last":0.97699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81299999999999994,"5min":1.0189999999999999,"15min":1.0920000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45300000000000001},"max":{"1min":0.86499999999999999,"5min":1.0489999999999999,"15min":1.0609999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0169999999999999,"5min":1.282,"15min":1.282},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95799999999999996,"5min":1.254,"15min":1.254},"last":0.83099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575065,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304348,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067492,"statfs":{"total":96636764160,"available":96325111808,"internally_reserved":0,"allocated":3035136,"data_stored":1419229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.69199999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":1.127,"15min":2.4609999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39800000000000002},"max":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":1.3460000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.054,"5min":1.093,"15min":1.2130000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.0289999999999999,"15min":1.204},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":0.95599999999999996,"15min":1.002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.433},"max":{"1min":1.075,"5min":1.145,"15min":1.2769999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.63700000000000001},"min":{"1min":0.52100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.96599999999999997,"15min":1.0229999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.56000000000000005,"5min":0.442,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.02,"15min":1.1539999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.872,"5min":0.91500000000000004,"15min":1.141},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.60799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.746},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.59499999999999997,"5min":0.51200000000000001,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.248},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:31:53.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:31:53.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.643+0000 7fe83a99d700 1 Processor -- start 2023-10-06T05:31:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.643+0000 7fe83a99d700 1 -- start start 2023-10-06T05:31:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.644+0000 7fe83a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 0x7fe8341641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.644+0000 7fe83a99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83406c820 con 0x7fe8340c6340 2023-10-06T05:31:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.644+0000 7fe83a99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83406b670 con 0x7fe834159520 2023-10-06T05:31:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.644+0000 7fe83a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83406a160 con 0x7fe8340c82a0 2023-10-06T05:31:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.644+0000 7fe83919a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe8340c6340 0x7fe8340c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35060/0 (socket says 172.21.15.33:35060) 2023-10-06T05:31:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.644+0000 7fe83919a700 1 -- 172.21.15.33:0/2072918456 learned_addr learned my addr 172.21.15.33:0/2072918456 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe83a19c700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 0x7fe8341641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659042862 0 0) 0x7fe83406c820 con 0x7fe8340c6340 2023-10-06T05:31:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe820003570 con 0x7fe8340c6340 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904978935 0 0) 0x7fe83406b670 con 0x7fe834159520 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe83406c820 con 0x7fe834159520 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2274848204 0 0) 0x7fe820003570 con 0x7fe8340c6340 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 msgr2=0x7fe8341641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 0x7fe8341641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 >> v1:172.21.15.167:6789/0 conn(0x7fe834159520 legacy=0x7fe8340c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340c8800 con 0x7fe8340c6340 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fe824001470 con 0x7fe8340c6340 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 >> v1:172.21.15.33:6789/0 conn(0x7fe8340c6340 legacy=0x7fe8340c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820003c20 0x7fe820004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.645+0000 7fe838999700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe838999700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 0x7fe8200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83406a160 con 0x7fe820004810 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82000bb70 con 0x7fe820003c20 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82000bd30 con 0x7fe8340c82a0 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83919a700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83a19c700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 0x7fe8200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83999b700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820003c20 0x7fe820004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83919a700 1 -- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 msgr2=0x7fe8200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83919a700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8340c82a0 0x7fe8200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83919a700 1 -- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820003c20 msgr2=0x7fe820004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83919a700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820003c20 0x7fe820004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.646+0000 7fe83919a700 1 -- 172.21.15.33:0/2072918456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820003570 con 0x7fe820004810 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe83919a700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe820004c80 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fe82400c2a0 tx=0x7fe824001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f172d48f9044f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400cb70 con 0x7fe820004810 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82400ccf0 con 0x7fe820004810 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe838999700 1 -- 172.21.15.33:0/2072918456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe824005b70 con 0x7fe820004810 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe83a99d700 1 -- 172.21.15.33:0/2072918456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 msgr2=0x7fe820004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe83a99d700 1 --2- 172.21.15.33:0/2072918456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe820004c80 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fe82400c2a0 tx=0x7fe824001e10 comp rx=0 tx=0).stop 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe83a19c700 1 -- 172.21.15.33:0/2072918456 reap_dead start 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe83a99d700 1 -- 172.21.15.33:0/2072918456 shutdown_connections 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.647+0000 7fe83a99d700 1 -- 172.21.15.33:0/2072918456 >> 172.21.15.33:0/2072918456 conn(0x7fe8340cc930 msgr2=0x7fe8340ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.648+0000 7fe83a99d700 1 -- 172.21.15.33:0/2072918456 shutdown_connections 2023-10-06T05:31:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.648+0000 7fe83a99d700 1 -- 172.21.15.33:0/2072918456 wait complete. 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.649+0000 7fe83a99d700 1 Processor -- start 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.649+0000 7fe83a99d700 1 -- start start 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83a99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe8340cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83415b220 0x7fe83415b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83a99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe834154a10 0x7fe834154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83a99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8340ac060 con 0x7fe820004810 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83a99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8341553c0 con 0x7fe834154a10 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8340b09e0 con 0x7fe83415b220 2023-10-06T05:31:53.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe8340cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe8340cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60860/0 (socket says 172.21.15.33:60860) 2023-10-06T05:31:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.650+0000 7fe83999b700 1 -- 172.21.15.33:0/412984078 learned_addr learned my addr 172.21.15.33:0/412984078 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83919a700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83415b220 0x7fe83415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83a19c700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe834154a10 0x7fe834154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83999b700 1 -- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83415b220 msgr2=0x7fe83415b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83999b700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83415b220 0x7fe83415b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83999b700 1 -- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe834154a10 msgr2=0x7fe834154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83999b700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe834154a10 0x7fe834154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83999b700 1 -- 172.21.15.33:0/412984078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340b0e70 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83999b700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe8340cb330 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fe830002c30 tx=0x7fe830007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000f920 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83000fa80 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000fe40 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.651+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83415bce0 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.652+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8340b0390 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.652+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8340c5c70 con 0x7fe820004810 2023-10-06T05:31:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.653+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe830013600 con 0x7fe820004810 2023-10-06T05:31:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.654+0000 7fe829ffb700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe818066f30 0x7fe8180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.654+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe830084e80 con 0x7fe820004810 2023-10-06T05:31:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.655+0000 7fe83919a700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe818066f30 0x7fe8180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.655+0000 7fe83919a700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe818066f30 0x7fe8180693e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe824007c90 tx=0x7fe824015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.658+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe830051ef0 con 0x7fe820004810 2023-10-06T05:31:53.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.856+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe83404c380 con 0x7fe818066f30 2023-10-06T05:31:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.866+0000 7fe829ffb700 1 -- 172.21.15.33:0/412984078 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311730 (secure 0 0 0) 0x7fe83404c380 con 0x7fe818066f30 2023-10-06T05:31:53.871 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe818066f30 msgr2=0x7fe8180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe818066f30 0x7fe8180693e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe824007c90 tx=0x7fe824015000 comp rx=0 tx=0).stop 2023-10-06T05:31:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 msgr2=0x7fe8340cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe8340cb330 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fe830002c30 tx=0x7fe830007d30 comp rx=0 tx=0).stop 2023-10-06T05:31:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 shutdown_connections 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe818066f30 0x7fe8180693e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe834154a10 0x7fe834154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe820004810 0x7fe8340cb330 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 --2- 172.21.15.33:0/412984078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe83415b220 0x7fe83415b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 >> 172.21.15.33:0/412984078 conn(0x7fe8340cc930 msgr2=0x7fe83406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 shutdown_connections 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:53.870+0000 7fe83a99d700 1 -- 172.21.15.33:0/412984078 wait complete. 2023-10-06T05:31:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:31:53.897 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":762,"stamp":"2023-10-06T05:31:52.086007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2440040,"kb_used_data":24872,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752534680,"statfs":{"total":773094113280,"available":770595512320,"internally_reserved":0,"allocated":25468928,"data_stored":12538732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.159143"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917637+0000","last_change":"2023-10-06T05:31:48.911533+0000","last_active":"2023-10-06T05:31:48.917637+0000","last_peered":"2023-10-06T05:31:48.917637+0000","last_clean":"2023-10-06T05:31:48.917637+0000","last_became_active":"2023-10-06T05:31:48.909401+0000","last_became_peered":"2023-10-06T05:31:48.909401+0000","last_unstale":"2023-10-06T05:31:48.917637+0000","last_undegraded":"2023-10-06T05:31:48.917637+0000","last_fullsized":"2023-10-06T05:31:48.917637+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:30.379004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909120+0000","last_change":"2023-10-06T05:31:48.903476+0000","last_active":"2023-10-06T05:31:48.909120+0000","last_peered":"2023-10-06T05:31:48.909120+0000","last_clean":"2023-10-06T05:31:48.909120+0000","last_became_active":"2023-10-06T05:31:48.903185+0000","last_became_peered":"2023-10-06T05:31:48.903185+0000","last_unstale":"2023-10-06T05:31:48.909120+0000","last_undegraded":"2023-10-06T05:31:48.909120+0000","last_fullsized":"2023-10-06T05:31:48.909120+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:28.209326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915923+0000","last_change":"2023-10-06T05:31:48.906545+0000","last_active":"2023-10-06T05:31:48.915923+0000","last_peered":"2023-10-06T05:31:48.915923+0000","last_clean":"2023-10-06T05:31:48.915923+0000","last_became_active":"2023-10-06T05:31:48.906238+0000","last_became_peered":"2023-10-06T05:31:48.906238+0000","last_unstale":"2023-10-06T05:31:48.915923+0000","last_undegraded":"2023-10-06T05:31:48.915923+0000","last_fullsized":"2023-10-06T05:31:48.915923+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:13.121708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915907+0000","last_change":"2023-10-06T05:31:48.905955+0000","last_active":"2023-10-06T05:31:48.915907+0000","last_peered":"2023-10-06T05:31:48.915907+0000","last_clean":"2023-10-06T05:31:48.915907+0000","last_became_active":"2023-10-06T05:31:48.905661+0000","last_became_peered":"2023-10-06T05:31:48.905661+0000","last_unstale":"2023-10-06T05:31:48.915907+0000","last_undegraded":"2023-10-06T05:31:48.915907+0000","last_fullsized":"2023-10-06T05:31:48.915907+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:19.730450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909219+0000","last_change":"2023-10-06T05:31:48.907884+0000","last_active":"2023-10-06T05:31:48.909219+0000","last_peered":"2023-10-06T05:31:48.909219+0000","last_clean":"2023-10-06T05:31:48.909219+0000","last_became_active":"2023-10-06T05:31:48.907610+0000","last_became_peered":"2023-10-06T05:31:48.907610+0000","last_unstale":"2023-10-06T05:31:48.909219+0000","last_undegraded":"2023-10-06T05:31:48.909219+0000","last_fullsized":"2023-10-06T05:31:48.909219+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:42.859104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908003+0000","last_change":"2023-10-06T05:31:48.903453+0000","last_active":"2023-10-06T05:31:48.908003+0000","last_peered":"2023-10-06T05:31:48.908003+0000","last_clean":"2023-10-06T05:31:48.908003+0000","last_became_active":"2023-10-06T05:31:48.903140+0000","last_became_peered":"2023-10-06T05:31:48.903140+0000","last_unstale":"2023-10-06T05:31:48.908003+0000","last_undegraded":"2023-10-06T05:31:48.908003+0000","last_fullsized":"2023-10-06T05:31:48.908003+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:14.525280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.910728+0000","last_change":"2023-10-06T05:31:48.904072+0000","last_active":"2023-10-06T05:31:48.910728+0000","last_peered":"2023-10-06T05:31:48.910728+0000","last_clean":"2023-10-06T05:31:48.910728+0000","last_became_active":"2023-10-06T05:31:48.903747+0000","last_became_peered":"2023-10-06T05:31:48.903747+0000","last_unstale":"2023-10-06T05:31:48.910728+0000","last_undegraded":"2023-10-06T05:31:48.910728+0000","last_fullsized":"2023-10-06T05:31:48.910728+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:21.613566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914526+0000","last_change":"2023-10-06T05:31:48.904932+0000","last_active":"2023-10-06T05:31:48.914526+0000","last_peered":"2023-10-06T05:31:48.914526+0000","last_clean":"2023-10-06T05:31:48.914526+0000","last_became_active":"2023-10-06T05:31:48.904645+0000","last_became_peered":"2023-10-06T05:31:48.904645+0000","last_unstale":"2023-10-06T05:31:48.914526+0000","last_undegraded":"2023-10-06T05:31:48.914526+0000","last_fullsized":"2023-10-06T05:31:48.914526+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:24.120897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914501+0000","last_change":"2023-10-06T05:31:48.911546+0000","last_active":"2023-10-06T05:31:48.914501+0000","last_peered":"2023-10-06T05:31:48.914501+0000","last_clean":"2023-10-06T05:31:48.914501+0000","last_became_active":"2023-10-06T05:31:48.910461+0000","last_became_peered":"2023-10-06T05:31:48.910461+0000","last_unstale":"2023-10-06T05:31:48.914501+0000","last_undegraded":"2023-10-06T05:31:48.914501+0000","last_fullsized":"2023-10-06T05:31:48.914501+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.296854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.919266+0000","last_change":"2023-10-06T05:31:48.918357+0000","last_active":"2023-10-06T05:31:48.919266+0000","last_peered":"2023-10-06T05:31:48.919266+0000","last_clean":"2023-10-06T05:31:48.919266+0000","last_became_active":"2023-10-06T05:31:48.918043+0000","last_became_peered":"2023-10-06T05:31:48.918043+0000","last_unstale":"2023-10-06T05:31:48.919266+0000","last_undegraded":"2023-10-06T05:31:48.919266+0000","last_fullsized":"2023-10-06T05:31:48.919266+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:06.772610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.918648+0000","last_change":"2023-10-06T05:31:48.918190+0000","last_active":"2023-10-06T05:31:48.918648+0000","last_peered":"2023-10-06T05:31:48.918648+0000","last_clean":"2023-10-06T05:31:48.918648+0000","last_became_active":"2023-10-06T05:31:48.917933+0000","last_became_peered":"2023-10-06T05:31:48.917933+0000","last_unstale":"2023-10-06T05:31:48.918648+0000","last_undegraded":"2023-10-06T05:31:48.918648+0000","last_fullsized":"2023-10-06T05:31:48.918648+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:42.546145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917401+0000","last_change":"2023-10-06T05:31:48.908226+0000","last_active":"2023-10-06T05:31:48.917401+0000","last_peered":"2023-10-06T05:31:48.917401+0000","last_clean":"2023-10-06T05:31:48.917401+0000","last_became_active":"2023-10-06T05:31:48.907843+0000","last_became_peered":"2023-10-06T05:31:48.907843+0000","last_unstale":"2023-10-06T05:31:48.917401+0000","last_undegraded":"2023-10-06T05:31:48.917401+0000","last_fullsized":"2023-10-06T05:31:48.917401+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:31.888428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.905955+0000","last_change":"2023-10-06T05:31:48.899190+0000","last_active":"2023-10-06T05:31:48.905955+0000","last_peered":"2023-10-06T05:31:48.905955+0000","last_clean":"2023-10-06T05:31:48.905955+0000","last_became_active":"2023-10-06T05:31:48.898903+0000","last_became_peered":"2023-10-06T05:31:48.898903+0000","last_unstale":"2023-10-06T05:31:48.905955+0000","last_undegraded":"2023-10-06T05:31:48.905955+0000","last_fullsized":"2023-10-06T05:31:48.905955+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:09.835703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914843+0000","last_change":"2023-10-06T05:31:48.912805+0000","last_active":"2023-10-06T05:31:48.914843+0000","last_peered":"2023-10-06T05:31:48.914843+0000","last_clean":"2023-10-06T05:31:48.914843+0000","last_became_active":"2023-10-06T05:31:48.912534+0000","last_became_peered":"2023-10-06T05:31:48.912534+0000","last_unstale":"2023-10-06T05:31:48.914843+0000","last_undegraded":"2023-10-06T05:31:48.914843+0000","last_fullsized":"2023-10-06T05:31:48.914843+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:47.386588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916253+0000","last_change":"2023-10-06T05:31:48.912841+0000","last_active":"2023-10-06T05:31:48.916253+0000","last_peered":"2023-10-06T05:31:48.916253+0000","last_clean":"2023-10-06T05:31:48.916253+0000","last_became_active":"2023-10-06T05:31:48.912545+0000","last_became_peered":"2023-10-06T05:31:48.912545+0000","last_unstale":"2023-10-06T05:31:48.916253+0000","last_undegraded":"2023-10-06T05:31:48.916253+0000","last_fullsized":"2023-10-06T05:31:48.916253+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:13.302591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907129+0000","last_change":"2023-10-06T05:31:48.903262+0000","last_active":"2023-10-06T05:31:48.907129+0000","last_peered":"2023-10-06T05:31:48.907129+0000","last_clean":"2023-10-06T05:31:48.907129+0000","last_became_active":"2023-10-06T05:31:48.902983+0000","last_became_peered":"2023-10-06T05:31:48.902983+0000","last_unstale":"2023-10-06T05:31:48.907129+0000","last_undegraded":"2023-10-06T05:31:48.907129+0000","last_fullsized":"2023-10-06T05:31:48.907129+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:45.153641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917614+0000","last_change":"2023-10-06T05:31:48.912688+0000","last_active":"2023-10-06T05:31:48.917614+0000","last_peered":"2023-10-06T05:31:48.917614+0000","last_clean":"2023-10-06T05:31:48.917614+0000","last_became_active":"2023-10-06T05:31:48.912420+0000","last_became_peered":"2023-10-06T05:31:48.912420+0000","last_unstale":"2023-10-06T05:31:48.917614+0000","last_undegraded":"2023-10-06T05:31:48.917614+0000","last_fullsized":"2023-10-06T05:31:48.917614+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:51.624561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915899+0000","last_change":"2023-10-06T05:31:48.906293+0000","last_active":"2023-10-06T05:31:48.915899+0000","last_peered":"2023-10-06T05:31:48.915899+0000","last_clean":"2023-10-06T05:31:48.915899+0000","last_became_active":"2023-10-06T05:31:48.906015+0000","last_became_peered":"2023-10-06T05:31:48.906015+0000","last_unstale":"2023-10-06T05:31:48.915899+0000","last_undegraded":"2023-10-06T05:31:48.915899+0000","last_fullsized":"2023-10-06T05:31:48.915899+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:01.008339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911100+0000","last_change":"2023-10-06T05:31:48.906609+0000","last_active":"2023-10-06T05:31:48.911100+0000","last_peered":"2023-10-06T05:31:48.911100+0000","last_clean":"2023-10-06T05:31:48.911100+0000","last_became_active":"2023-10-06T05:31:48.906323+0000","last_became_peered":"2023-10-06T05:31:48.906323+0000","last_unstale":"2023-10-06T05:31:48.911100+0000","last_undegraded":"2023-10-06T05:31:48.911100+0000","last_fullsized":"2023-10-06T05:31:48.911100+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:29.118160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908107+0000","last_change":"2023-10-06T05:31:48.903980+0000","last_active":"2023-10-06T05:31:48.908107+0000","last_peered":"2023-10-06T05:31:48.908107+0000","last_clean":"2023-10-06T05:31:48.908107+0000","last_became_active":"2023-10-06T05:31:48.903624+0000","last_became_peered":"2023-10-06T05:31:48.903624+0000","last_unstale":"2023-10-06T05:31:48.908107+0000","last_undegraded":"2023-10-06T05:31:48.908107+0000","last_fullsized":"2023-10-06T05:31:48.908107+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:25.209531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917471+0000","last_change":"2023-10-06T05:31:48.915364+0000","last_active":"2023-10-06T05:31:48.917471+0000","last_peered":"2023-10-06T05:31:48.917471+0000","last_clean":"2023-10-06T05:31:48.917471+0000","last_became_active":"2023-10-06T05:31:48.915077+0000","last_became_peered":"2023-10-06T05:31:48.915077+0000","last_unstale":"2023-10-06T05:31:48.917471+0000","last_undegraded":"2023-10-06T05:31:48.917471+0000","last_fullsized":"2023-10-06T05:31:48.917471+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:21.147549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909170+0000","last_change":"2023-10-06T05:31:48.907670+0000","last_active":"2023-10-06T05:31:48.909170+0000","last_peered":"2023-10-06T05:31:48.909170+0000","last_clean":"2023-10-06T05:31:48.909170+0000","last_became_active":"2023-10-06T05:31:48.907295+0000","last_became_peered":"2023-10-06T05:31:48.907295+0000","last_unstale":"2023-10-06T05:31:48.909170+0000","last_undegraded":"2023-10-06T05:31:48.909170+0000","last_fullsized":"2023-10-06T05:31:48.909170+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:20.520473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912904+0000","last_change":"2023-10-06T05:31:48.910558+0000","last_active":"2023-10-06T05:31:48.912904+0000","last_peered":"2023-10-06T05:31:48.912904+0000","last_clean":"2023-10-06T05:31:48.912904+0000","last_became_active":"2023-10-06T05:31:48.910184+0000","last_became_peered":"2023-10-06T05:31:48.910184+0000","last_unstale":"2023-10-06T05:31:48.912904+0000","last_undegraded":"2023-10-06T05:31:48.912904+0000","last_fullsized":"2023-10-06T05:31:48.912904+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:16.078845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915757+0000","last_change":"2023-10-06T05:31:48.911578+0000","last_active":"2023-10-06T05:31:48.915757+0000","last_peered":"2023-10-06T05:31:48.915757+0000","last_clean":"2023-10-06T05:31:48.915757+0000","last_became_active":"2023-10-06T05:31:48.911185+0000","last_became_peered":"2023-10-06T05:31:48.911185+0000","last_unstale":"2023-10-06T05:31:48.915757+0000","last_undegraded":"2023-10-06T05:31:48.915757+0000","last_fullsized":"2023-10-06T05:31:48.915757+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:05.302502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914042+0000","last_change":"2023-10-06T05:31:48.910903+0000","last_active":"2023-10-06T05:31:48.914042+0000","last_peered":"2023-10-06T05:31:48.914042+0000","last_clean":"2023-10-06T05:31:48.914042+0000","last_became_active":"2023-10-06T05:31:48.910634+0000","last_became_peered":"2023-10-06T05:31:48.910634+0000","last_unstale":"2023-10-06T05:31:48.914042+0000","last_undegraded":"2023-10-06T05:31:48.914042+0000","last_fullsized":"2023-10-06T05:31:48.914042+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:08.340788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"238'4","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.960353+0000","last_change":"2023-10-06T05:31:47.952906+0000","last_active":"2023-10-06T05:31:48.960353+0000","last_peered":"2023-10-06T05:31:48.960353+0000","last_clean":"2023-10-06T05:31:48.960353+0000","last_became_active":"2023-10-06T05:31:47.952564+0000","last_became_peered":"2023-10-06T05:31:47.952564+0000","last_unstale":"2023-10-06T05:31:48.960353+0000","last_undegraded":"2023-10-06T05:31:48.960353+0000","last_fullsized":"2023-10-06T05:31:48.960353+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:05.046935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912886+0000","last_change":"2023-10-06T05:31:48.910160+0000","last_active":"2023-10-06T05:31:48.912886+0000","last_peered":"2023-10-06T05:31:48.912886+0000","last_clean":"2023-10-06T05:31:48.912886+0000","last_became_active":"2023-10-06T05:31:48.909807+0000","last_became_peered":"2023-10-06T05:31:48.909807+0000","last_unstale":"2023-10-06T05:31:48.912886+0000","last_undegraded":"2023-10-06T05:31:48.912886+0000","last_fullsized":"2023-10-06T05:31:48.912886+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:59.724009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959887+0000","last_change":"2023-10-06T05:31:47.945386+0000","last_active":"2023-10-06T05:31:48.959887+0000","last_peered":"2023-10-06T05:31:48.959887+0000","last_clean":"2023-10-06T05:31:48.959887+0000","last_became_active":"2023-10-06T05:31:47.945103+0000","last_became_peered":"2023-10-06T05:31:47.945103+0000","last_unstale":"2023-10-06T05:31:48.959887+0000","last_undegraded":"2023-10-06T05:31:48.959887+0000","last_fullsized":"2023-10-06T05:31:48.959887+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:31.300931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914337+0000","last_change":"2023-10-06T05:31:48.904809+0000","last_active":"2023-10-06T05:31:48.914337+0000","last_peered":"2023-10-06T05:31:48.914337+0000","last_clean":"2023-10-06T05:31:48.914337+0000","last_became_active":"2023-10-06T05:31:48.904516+0000","last_became_peered":"2023-10-06T05:31:48.904516+0000","last_unstale":"2023-10-06T05:31:48.914337+0000","last_undegraded":"2023-10-06T05:31:48.914337+0000","last_fullsized":"2023-10-06T05:31:48.914337+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:41.441891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959427+0000","last_change":"2023-10-06T05:31:47.944906+0000","last_active":"2023-10-06T05:31:48.959427+0000","last_peered":"2023-10-06T05:31:48.959427+0000","last_clean":"2023-10-06T05:31:48.959427+0000","last_became_active":"2023-10-06T05:31:47.944638+0000","last_became_peered":"2023-10-06T05:31:47.944638+0000","last_unstale":"2023-10-06T05:31:48.959427+0000","last_undegraded":"2023-10-06T05:31:48.959427+0000","last_fullsized":"2023-10-06T05:31:48.959427+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:33.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912467+0000","last_change":"2023-10-06T05:31:48.910974+0000","last_active":"2023-10-06T05:31:48.912467+0000","last_peered":"2023-10-06T05:31:48.912467+0000","last_clean":"2023-10-06T05:31:48.912467+0000","last_became_active":"2023-10-06T05:31:48.910675+0000","last_became_peered":"2023-10-06T05:31:48.910675+0000","last_unstale":"2023-10-06T05:31:48.912467+0000","last_undegraded":"2023-10-06T05:31:48.912467+0000","last_fullsized":"2023-10-06T05:31:48.912467+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:14.525280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959739+0000","last_change":"2023-10-06T05:31:47.944302+0000","last_active":"2023-10-06T05:31:48.959739+0000","last_peered":"2023-10-06T05:31:48.959739+0000","last_clean":"2023-10-06T05:31:48.959739+0000","last_became_active":"2023-10-06T05:31:47.944022+0000","last_became_peered":"2023-10-06T05:31:47.944022+0000","last_unstale":"2023-10-06T05:31:48.959739+0000","last_undegraded":"2023-10-06T05:31:48.959739+0000","last_fullsized":"2023-10-06T05:31:48.959739+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:56.906395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915929+0000","last_change":"2023-10-06T05:31:48.906579+0000","last_active":"2023-10-06T05:31:48.915929+0000","last_peered":"2023-10-06T05:31:48.915929+0000","last_clean":"2023-10-06T05:31:48.915929+0000","last_became_active":"2023-10-06T05:31:48.906306+0000","last_became_peered":"2023-10-06T05:31:48.906306+0000","last_unstale":"2023-10-06T05:31:48.915929+0000","last_undegraded":"2023-10-06T05:31:48.915929+0000","last_fullsized":"2023-10-06T05:31:48.915929+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:21.060836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.959027+0000","last_change":"2023-10-06T05:31:47.954390+0000","last_active":"2023-10-06T05:31:48.959027+0000","last_peered":"2023-10-06T05:31:48.959027+0000","last_clean":"2023-10-06T05:31:48.959027+0000","last_became_active":"2023-10-06T05:31:47.953253+0000","last_became_peered":"2023-10-06T05:31:47.953253+0000","last_unstale":"2023-10-06T05:31:48.959027+0000","last_undegraded":"2023-10-06T05:31:48.959027+0000","last_fullsized":"2023-10-06T05:31:48.959027+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:42.083066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.913003+0000","last_change":"2023-10-06T05:31:48.908592+0000","last_active":"2023-10-06T05:31:48.913003+0000","last_peered":"2023-10-06T05:31:48.913003+0000","last_clean":"2023-10-06T05:31:48.913003+0000","last_became_active":"2023-10-06T05:31:48.908333+0000","last_became_peered":"2023-10-06T05:31:48.908333+0000","last_unstale":"2023-10-06T05:31:48.913003+0000","last_undegraded":"2023-10-06T05:31:48.913003+0000","last_fullsized":"2023-10-06T05:31:48.913003+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:08.610133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.958935+0000","last_change":"2023-10-06T05:31:47.957431+0000","last_active":"2023-10-06T05:31:48.958935+0000","last_peered":"2023-10-06T05:31:48.958935+0000","last_clean":"2023-10-06T05:31:48.958935+0000","last_became_active":"2023-10-06T05:31:47.957167+0000","last_became_peered":"2023-10-06T05:31:47.957167+0000","last_unstale":"2023-10-06T05:31:48.958935+0000","last_undegraded":"2023-10-06T05:31:48.958935+0000","last_fullsized":"2023-10-06T05:31:48.958935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:26.909628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916977+0000","last_change":"2023-10-06T05:31:48.915563+0000","last_active":"2023-10-06T05:31:48.916977+0000","last_peered":"2023-10-06T05:31:48.916977+0000","last_clean":"2023-10-06T05:31:48.916977+0000","last_became_active":"2023-10-06T05:31:48.915310+0000","last_became_peered":"2023-10-06T05:31:48.915310+0000","last_unstale":"2023-10-06T05:31:48.916977+0000","last_undegraded":"2023-10-06T05:31:48.916977+0000","last_fullsized":"2023-10-06T05:31:48.916977+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:38.983840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.891661+0000","last_change":"2023-10-06T05:31:47.944456+0000","last_active":"2023-10-06T05:31:48.891661+0000","last_peered":"2023-10-06T05:31:48.891661+0000","last_clean":"2023-10-06T05:31:48.891661+0000","last_became_active":"2023-10-06T05:31:47.944186+0000","last_became_peered":"2023-10-06T05:31:47.944186+0000","last_unstale":"2023-10-06T05:31:48.891661+0000","last_undegraded":"2023-10-06T05:31:48.891661+0000","last_fullsized":"2023-10-06T05:31:48.891661+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:52.951970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908134+0000","last_change":"2023-10-06T05:31:48.905363+0000","last_active":"2023-10-06T05:31:48.908134+0000","last_peered":"2023-10-06T05:31:48.908134+0000","last_clean":"2023-10-06T05:31:48.908134+0000","last_became_active":"2023-10-06T05:31:48.905055+0000","last_became_peered":"2023-10-06T05:31:48.905055+0000","last_unstale":"2023-10-06T05:31:48.908134+0000","last_undegraded":"2023-10-06T05:31:48.908134+0000","last_fullsized":"2023-10-06T05:31:48.908134+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:09.169488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.891264+0000","last_change":"2023-10-06T05:31:47.954190+0000","last_active":"2023-10-06T05:31:48.891264+0000","last_peered":"2023-10-06T05:31:48.891264+0000","last_clean":"2023-10-06T05:31:48.891264+0000","last_became_active":"2023-10-06T05:31:47.953860+0000","last_became_peered":"2023-10-06T05:31:47.953860+0000","last_unstale":"2023-10-06T05:31:48.891264+0000","last_undegraded":"2023-10-06T05:31:48.891264+0000","last_fullsized":"2023-10-06T05:31:48.891264+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:50.959815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.905989+0000","last_change":"2023-10-06T05:31:48.899990+0000","last_active":"2023-10-06T05:31:48.905989+0000","last_peered":"2023-10-06T05:31:48.905989+0000","last_clean":"2023-10-06T05:31:48.905989+0000","last_became_active":"2023-10-06T05:31:48.899738+0000","last_became_peered":"2023-10-06T05:31:48.899738+0000","last_unstale":"2023-10-06T05:31:48.905989+0000","last_undegraded":"2023-10-06T05:31:48.905989+0000","last_fullsized":"2023-10-06T05:31:48.905989+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:30.977276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897705+0000","last_change":"2023-10-06T05:31:47.945250+0000","last_active":"2023-10-06T05:31:48.897705+0000","last_peered":"2023-10-06T05:31:48.897705+0000","last_clean":"2023-10-06T05:31:48.897705+0000","last_became_active":"2023-10-06T05:31:47.944998+0000","last_became_peered":"2023-10-06T05:31:47.944998+0000","last_unstale":"2023-10-06T05:31:48.897705+0000","last_undegraded":"2023-10-06T05:31:48.897705+0000","last_fullsized":"2023-10-06T05:31:48.897705+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:01.493691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.917660+0000","last_change":"2023-10-06T05:31:48.911796+0000","last_active":"2023-10-06T05:31:48.917660+0000","last_peered":"2023-10-06T05:31:48.917660+0000","last_clean":"2023-10-06T05:31:48.917660+0000","last_became_active":"2023-10-06T05:31:48.909460+0000","last_became_peered":"2023-10-06T05:31:48.909460+0000","last_unstale":"2023-10-06T05:31:48.917660+0000","last_undegraded":"2023-10-06T05:31:48.917660+0000","last_fullsized":"2023-10-06T05:31:48.917660+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:09.029152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892617+0000","last_change":"2023-10-06T05:31:47.945842+0000","last_active":"2023-10-06T05:31:48.892617+0000","last_peered":"2023-10-06T05:31:48.892617+0000","last_clean":"2023-10-06T05:31:48.892617+0000","last_became_active":"2023-10-06T05:31:47.945515+0000","last_became_peered":"2023-10-06T05:31:47.945515+0000","last_unstale":"2023-10-06T05:31:48.892617+0000","last_undegraded":"2023-10-06T05:31:48.892617+0000","last_fullsized":"2023-10-06T05:31:48.892617+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.266037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916282+0000","last_change":"2023-10-06T05:31:48.911006+0000","last_active":"2023-10-06T05:31:48.916282+0000","last_peered":"2023-10-06T05:31:48.916282+0000","last_clean":"2023-10-06T05:31:48.916282+0000","last_became_active":"2023-10-06T05:31:48.910045+0000","last_became_peered":"2023-10-06T05:31:48.910045+0000","last_unstale":"2023-10-06T05:31:48.916282+0000","last_undegraded":"2023-10-06T05:31:48.916282+0000","last_fullsized":"2023-10-06T05:31:48.916282+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:01.628857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"238'3","reported_seq":13,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.958498+0000","last_change":"2023-10-06T05:31:47.954873+0000","last_active":"2023-10-06T05:31:48.958498+0000","last_peered":"2023-10-06T05:31:48.958498+0000","last_clean":"2023-10-06T05:31:48.958498+0000","last_became_active":"2023-10-06T05:31:47.954543+0000","last_became_peered":"2023-10-06T05:31:47.954543+0000","last_unstale":"2023-10-06T05:31:48.958498+0000","last_undegraded":"2023-10-06T05:31:48.958498+0000","last_fullsized":"2023-10-06T05:31:48.958498+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:30.266037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.918070+0000","last_change":"2023-10-06T05:31:48.917169+0000","last_active":"2023-10-06T05:31:48.918070+0000","last_peered":"2023-10-06T05:31:48.918070+0000","last_clean":"2023-10-06T05:31:48.918070+0000","last_became_active":"2023-10-06T05:31:48.916852+0000","last_became_peered":"2023-10-06T05:31:48.916852+0000","last_unstale":"2023-10-06T05:31:48.918070+0000","last_undegraded":"2023-10-06T05:31:48.918070+0000","last_fullsized":"2023-10-06T05:31:48.918070+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:21.613566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"238'6","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.963637+0000","last_change":"2023-10-06T05:31:47.955074+0000","last_active":"2023-10-06T05:31:48.963637+0000","last_peered":"2023-10-06T05:31:48.963637+0000","last_clean":"2023-10-06T05:31:48.963637+0000","last_became_active":"2023-10-06T05:31:47.954190+0000","last_became_peered":"2023-10-06T05:31:47.954190+0000","last_unstale":"2023-10-06T05:31:48.963637+0000","last_undegraded":"2023-10-06T05:31:48.963637+0000","last_fullsized":"2023-10-06T05:31:48.963637+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:39.937926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909059+0000","last_change":"2023-10-06T05:31:48.901775+0000","last_active":"2023-10-06T05:31:48.909059+0000","last_peered":"2023-10-06T05:31:48.909059+0000","last_clean":"2023-10-06T05:31:48.909059+0000","last_became_active":"2023-10-06T05:31:48.901402+0000","last_became_peered":"2023-10-06T05:31:48.901402+0000","last_unstale":"2023-10-06T05:31:48.909059+0000","last_undegraded":"2023-10-06T05:31:48.909059+0000","last_fullsized":"2023-10-06T05:31:48.909059+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:35.249864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.961746+0000","last_change":"2023-10-06T05:31:47.955833+0000","last_active":"2023-10-06T05:31:48.961746+0000","last_peered":"2023-10-06T05:31:48.961746+0000","last_clean":"2023-10-06T05:31:48.961746+0000","last_became_active":"2023-10-06T05:31:47.955574+0000","last_became_peered":"2023-10-06T05:31:47.955574+0000","last_unstale":"2023-10-06T05:31:48.961746+0000","last_undegraded":"2023-10-06T05:31:48.961746+0000","last_fullsized":"2023-10-06T05:31:48.961746+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:27.270314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.903820+0000","last_change":"2023-10-06T05:31:48.897129+0000","last_active":"2023-10-06T05:31:48.903820+0000","last_peered":"2023-10-06T05:31:48.903820+0000","last_clean":"2023-10-06T05:31:48.903820+0000","last_became_active":"2023-10-06T05:31:48.896845+0000","last_became_peered":"2023-10-06T05:31:48.896845+0000","last_unstale":"2023-10-06T05:31:48.903820+0000","last_undegraded":"2023-10-06T05:31:48.903820+0000","last_fullsized":"2023-10-06T05:31:48.903820+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:56.470940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.895220+0000","last_change":"2023-10-06T05:31:47.939790+0000","last_active":"2023-10-06T05:31:48.895220+0000","last_peered":"2023-10-06T05:31:48.895220+0000","last_clean":"2023-10-06T05:31:48.895220+0000","last_became_active":"2023-10-06T05:31:47.939462+0000","last_became_peered":"2023-10-06T05:31:47.939462+0000","last_unstale":"2023-10-06T05:31:48.895220+0000","last_undegraded":"2023-10-06T05:31:48.895220+0000","last_fullsized":"2023-10-06T05:31:48.895220+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:12.115237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911037+0000","last_change":"2023-10-06T05:31:48.905732+0000","last_active":"2023-10-06T05:31:48.911037+0000","last_peered":"2023-10-06T05:31:48.911037+0000","last_clean":"2023-10-06T05:31:48.911037+0000","last_became_active":"2023-10-06T05:31:48.905259+0000","last_became_peered":"2023-10-06T05:31:48.905259+0000","last_unstale":"2023-10-06T05:31:48.911037+0000","last_undegraded":"2023-10-06T05:31:48.911037+0000","last_fullsized":"2023-10-06T05:31:48.911037+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:20.785238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894673+0000","last_change":"2023-10-06T05:31:47.947135+0000","last_active":"2023-10-06T05:31:48.894673+0000","last_peered":"2023-10-06T05:31:48.894673+0000","last_clean":"2023-10-06T05:31:48.894673+0000","last_became_active":"2023-10-06T05:31:47.946865+0000","last_became_peered":"2023-10-06T05:31:47.946865+0000","last_unstale":"2023-10-06T05:31:48.894673+0000","last_undegraded":"2023-10-06T05:31:48.894673+0000","last_fullsized":"2023-10-06T05:31:48.894673+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:29.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908114+0000","last_change":"2023-10-06T05:31:48.903789+0000","last_active":"2023-10-06T05:31:48.908114+0000","last_peered":"2023-10-06T05:31:48.908114+0000","last_clean":"2023-10-06T05:31:48.908114+0000","last_became_active":"2023-10-06T05:31:48.903500+0000","last_became_peered":"2023-10-06T05:31:48.903500+0000","last_unstale":"2023-10-06T05:31:48.908114+0000","last_undegraded":"2023-10-06T05:31:48.908114+0000","last_fullsized":"2023-10-06T05:31:48.908114+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:12.535465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"238'8","reported_seq":18,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.962892+0000","last_change":"2023-10-06T05:31:47.953208+0000","last_active":"2023-10-06T05:31:48.962892+0000","last_peered":"2023-10-06T05:31:48.962892+0000","last_clean":"2023-10-06T05:31:48.962892+0000","last_became_active":"2023-10-06T05:31:47.952770+0000","last_became_peered":"2023-10-06T05:31:47.952770+0000","last_unstale":"2023-10-06T05:31:48.962892+0000","last_undegraded":"2023-10-06T05:31:48.962892+0000","last_fullsized":"2023-10-06T05:31:48.962892+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:13.530112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907394+0000","last_change":"2023-10-06T05:31:48.902861+0000","last_active":"2023-10-06T05:31:48.907394+0000","last_peered":"2023-10-06T05:31:48.907394+0000","last_clean":"2023-10-06T05:31:48.907394+0000","last_became_active":"2023-10-06T05:31:48.902544+0000","last_became_peered":"2023-10-06T05:31:48.902544+0000","last_unstale":"2023-10-06T05:31:48.907394+0000","last_undegraded":"2023-10-06T05:31:48.907394+0000","last_fullsized":"2023-10-06T05:31:48.907394+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:25:26.419754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.960077+0000","last_change":"2023-10-06T05:31:47.945586+0000","last_active":"2023-10-06T05:31:48.960077+0000","last_peered":"2023-10-06T05:31:48.960077+0000","last_clean":"2023-10-06T05:31:48.960077+0000","last_became_active":"2023-10-06T05:31:47.945326+0000","last_became_peered":"2023-10-06T05:31:47.945326+0000","last_unstale":"2023-10-06T05:31:48.960077+0000","last_undegraded":"2023-10-06T05:31:48.960077+0000","last_fullsized":"2023-10-06T05:31:48.960077+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:00.801937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909646+0000","last_change":"2023-10-06T05:31:48.907128+0000","last_active":"2023-10-06T05:31:48.909646+0000","last_peered":"2023-10-06T05:31:48.909646+0000","last_clean":"2023-10-06T05:31:48.909646+0000","last_became_active":"2023-10-06T05:31:48.903132+0000","last_became_peered":"2023-10-06T05:31:48.903132+0000","last_unstale":"2023-10-06T05:31:48.909646+0000","last_undegraded":"2023-10-06T05:31:48.909646+0000","last_fullsized":"2023-10-06T05:31:48.909646+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:41.109366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.957184+0000","last_change":"2023-10-06T05:31:47.955383+0000","last_active":"2023-10-06T05:31:48.957184+0000","last_peered":"2023-10-06T05:31:48.957184+0000","last_clean":"2023-10-06T05:31:48.957184+0000","last_became_active":"2023-10-06T05:31:47.955043+0000","last_became_peered":"2023-10-06T05:31:47.955043+0000","last_unstale":"2023-10-06T05:31:48.957184+0000","last_undegraded":"2023-10-06T05:31:48.957184+0000","last_fullsized":"2023-10-06T05:31:48.957184+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:11.081466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.908711+0000","last_change":"2023-10-06T05:31:48.906847+0000","last_active":"2023-10-06T05:31:48.908711+0000","last_peered":"2023-10-06T05:31:48.908711+0000","last_clean":"2023-10-06T05:31:48.908711+0000","last_became_active":"2023-10-06T05:31:48.906579+0000","last_became_peered":"2023-10-06T05:31:48.906579+0000","last_unstale":"2023-10-06T05:31:48.908711+0000","last_undegraded":"2023-10-06T05:31:48.908711+0000","last_fullsized":"2023-10-06T05:31:48.908711+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:14.387865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.958832+0000","last_change":"2023-10-06T05:31:47.953015+0000","last_active":"2023-10-06T05:31:48.958832+0000","last_peered":"2023-10-06T05:31:48.958832+0000","last_clean":"2023-10-06T05:31:48.958832+0000","last_became_active":"2023-10-06T05:31:47.952651+0000","last_became_peered":"2023-10-06T05:31:47.952651+0000","last_unstale":"2023-10-06T05:31:48.958832+0000","last_undegraded":"2023-10-06T05:31:48.958832+0000","last_fullsized":"2023-10-06T05:31:48.958832+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:03.260271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.903787+0000","last_change":"2023-10-06T05:31:48.898071+0000","last_active":"2023-10-06T05:31:48.903787+0000","last_peered":"2023-10-06T05:31:48.903787+0000","last_clean":"2023-10-06T05:31:48.903787+0000","last_became_active":"2023-10-06T05:31:48.897334+0000","last_became_peered":"2023-10-06T05:31:48.897334+0000","last_unstale":"2023-10-06T05:31:48.903787+0000","last_undegraded":"2023-10-06T05:31:48.903787+0000","last_fullsized":"2023-10-06T05:31:48.903787+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:24.482035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892325+0000","last_change":"2023-10-06T05:31:47.943786+0000","last_active":"2023-10-06T05:31:48.892325+0000","last_peered":"2023-10-06T05:31:48.892325+0000","last_clean":"2023-10-06T05:31:48.892325+0000","last_became_active":"2023-10-06T05:31:47.943458+0000","last_became_peered":"2023-10-06T05:31:47.943458+0000","last_unstale":"2023-10-06T05:31:48.892325+0000","last_undegraded":"2023-10-06T05:31:48.892325+0000","last_fullsized":"2023-10-06T05:31:48.892325+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:34.635886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.915645+0000","last_change":"2023-10-06T05:31:48.911092+0000","last_active":"2023-10-06T05:31:48.915645+0000","last_peered":"2023-10-06T05:31:48.915645+0000","last_clean":"2023-10-06T05:31:48.915645+0000","last_became_active":"2023-10-06T05:31:48.910797+0000","last_became_peered":"2023-10-06T05:31:48.910797+0000","last_unstale":"2023-10-06T05:31:48.915645+0000","last_undegraded":"2023-10-06T05:31:48.915645+0000","last_fullsized":"2023-10-06T05:31:48.915645+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:13.405614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.961651+0000","last_change":"2023-10-06T05:31:47.945188+0000","last_active":"2023-10-06T05:31:48.961651+0000","last_peered":"2023-10-06T05:31:48.961651+0000","last_clean":"2023-10-06T05:31:48.961651+0000","last_became_active":"2023-10-06T05:31:47.944879+0000","last_became_peered":"2023-10-06T05:31:47.944879+0000","last_unstale":"2023-10-06T05:31:48.961651+0000","last_undegraded":"2023-10-06T05:31:48.961651+0000","last_fullsized":"2023-10-06T05:31:48.961651+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:02.072557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.910295+0000","last_change":"2023-10-06T05:31:48.907998+0000","last_active":"2023-10-06T05:31:48.910295+0000","last_peered":"2023-10-06T05:31:48.910295+0000","last_clean":"2023-10-06T05:31:48.910295+0000","last_became_active":"2023-10-06T05:31:48.907678+0000","last_became_peered":"2023-10-06T05:31:48.907678+0000","last_unstale":"2023-10-06T05:31:48.910295+0000","last_undegraded":"2023-10-06T05:31:48.910295+0000","last_fullsized":"2023-10-06T05:31:48.910295+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:21.147549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.889718+0000","last_change":"2023-10-06T05:31:47.956209+0000","last_active":"2023-10-06T05:31:48.889718+0000","last_peered":"2023-10-06T05:31:48.889718+0000","last_clean":"2023-10-06T05:31:48.889718+0000","last_became_active":"2023-10-06T05:31:47.955904+0000","last_became_peered":"2023-10-06T05:31:47.955904+0000","last_unstale":"2023-10-06T05:31:48.889718+0000","last_undegraded":"2023-10-06T05:31:48.889718+0000","last_fullsized":"2023-10-06T05:31:48.889718+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:29.242754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911014+0000","last_change":"2023-10-06T05:31:48.905545+0000","last_active":"2023-10-06T05:31:48.911014+0000","last_peered":"2023-10-06T05:31:48.911014+0000","last_clean":"2023-10-06T05:31:48.911014+0000","last_became_active":"2023-10-06T05:31:48.904927+0000","last_became_peered":"2023-10-06T05:31:48.904927+0000","last_unstale":"2023-10-06T05:31:48.911014+0000","last_undegraded":"2023-10-06T05:31:48.911014+0000","last_fullsized":"2023-10-06T05:31:48.911014+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:19.141959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894696+0000","last_change":"2023-10-06T05:31:47.941963+0000","last_active":"2023-10-06T05:31:48.894696+0000","last_peered":"2023-10-06T05:31:48.894696+0000","last_clean":"2023-10-06T05:31:48.894696+0000","last_became_active":"2023-10-06T05:31:47.941641+0000","last_became_peered":"2023-10-06T05:31:47.941641+0000","last_unstale":"2023-10-06T05:31:48.894696+0000","last_undegraded":"2023-10-06T05:31:48.894696+0000","last_fullsized":"2023-10-06T05:31:48.894696+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:51.696951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912193+0000","last_change":"2023-10-06T05:31:48.908603+0000","last_active":"2023-10-06T05:31:48.912193+0000","last_peered":"2023-10-06T05:31:48.912193+0000","last_clean":"2023-10-06T05:31:48.912193+0000","last_became_active":"2023-10-06T05:31:48.908349+0000","last_became_peered":"2023-10-06T05:31:48.908349+0000","last_unstale":"2023-10-06T05:31:48.912193+0000","last_undegraded":"2023-10-06T05:31:48.912193+0000","last_fullsized":"2023-10-06T05:31:48.912193+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:14.031440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.891847+0000","last_change":"2023-10-06T05:31:47.948871+0000","last_active":"2023-10-06T05:31:48.891847+0000","last_peered":"2023-10-06T05:31:48.891847+0000","last_clean":"2023-10-06T05:31:48.891847+0000","last_became_active":"2023-10-06T05:31:47.948041+0000","last_became_peered":"2023-10-06T05:31:47.948041+0000","last_unstale":"2023-10-06T05:31:48.891847+0000","last_undegraded":"2023-10-06T05:31:48.891847+0000","last_fullsized":"2023-10-06T05:31:48.891847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:29.801473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.906012+0000","last_change":"2023-10-06T05:31:48.900821+0000","last_active":"2023-10-06T05:31:48.906012+0000","last_peered":"2023-10-06T05:31:48.906012+0000","last_clean":"2023-10-06T05:31:48.906012+0000","last_became_active":"2023-10-06T05:31:48.900516+0000","last_became_peered":"2023-10-06T05:31:48.900516+0000","last_unstale":"2023-10-06T05:31:48.906012+0000","last_undegraded":"2023-10-06T05:31:48.906012+0000","last_fullsized":"2023-10-06T05:31:48.906012+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:05.050675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.960034+0000","last_change":"2023-10-06T05:31:47.951106+0000","last_active":"2023-10-06T05:31:48.960034+0000","last_peered":"2023-10-06T05:31:48.960034+0000","last_clean":"2023-10-06T05:31:48.960034+0000","last_became_active":"2023-10-06T05:31:47.950776+0000","last_became_peered":"2023-10-06T05:31:47.950776+0000","last_unstale":"2023-10-06T05:31:48.960034+0000","last_undegraded":"2023-10-06T05:31:48.960034+0000","last_fullsized":"2023-10-06T05:31:48.960034+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:01.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.911072+0000","last_change":"2023-10-06T05:31:48.906545+0000","last_active":"2023-10-06T05:31:48.911072+0000","last_peered":"2023-10-06T05:31:48.911072+0000","last_clean":"2023-10-06T05:31:48.911072+0000","last_became_active":"2023-10-06T05:31:48.906259+0000","last_became_peered":"2023-10-06T05:31:48.906259+0000","last_unstale":"2023-10-06T05:31:48.911072+0000","last_undegraded":"2023-10-06T05:31:48.911072+0000","last_fullsized":"2023-10-06T05:31:48.911072+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894136+0000","last_change":"2023-10-06T05:31:47.940085+0000","last_active":"2023-10-06T05:31:48.894136+0000","last_peered":"2023-10-06T05:31:48.894136+0000","last_clean":"2023-10-06T05:31:48.894136+0000","last_became_active":"2023-10-06T05:31:47.939599+0000","last_became_peered":"2023-10-06T05:31:47.939599+0000","last_unstale":"2023-10-06T05:31:48.894136+0000","last_undegraded":"2023-10-06T05:31:48.894136+0000","last_fullsized":"2023-10-06T05:31:48.894136+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:00.001868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914616+0000","last_change":"2023-10-06T05:31:48.902026+0000","last_active":"2023-10-06T05:31:48.914616+0000","last_peered":"2023-10-06T05:31:48.914616+0000","last_clean":"2023-10-06T05:31:48.914616+0000","last_became_active":"2023-10-06T05:31:48.901752+0000","last_became_peered":"2023-10-06T05:31:48.901752+0000","last_unstale":"2023-10-06T05:31:48.914616+0000","last_undegraded":"2023-10-06T05:31:48.914616+0000","last_fullsized":"2023-10-06T05:31:48.914616+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:16.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909538+0000","last_change":"2023-10-06T05:31:47.943067+0000","last_active":"2023-10-06T05:31:48.909538+0000","last_peered":"2023-10-06T05:31:48.909538+0000","last_clean":"2023-10-06T05:31:48.909538+0000","last_became_active":"2023-10-06T05:31:47.942804+0000","last_became_peered":"2023-10-06T05:31:47.942804+0000","last_unstale":"2023-10-06T05:31:48.909538+0000","last_undegraded":"2023-10-06T05:31:48.909538+0000","last_fullsized":"2023-10-06T05:31:48.909538+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:54.641658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912446+0000","last_change":"2023-10-06T05:31:48.910323+0000","last_active":"2023-10-06T05:31:48.912446+0000","last_peered":"2023-10-06T05:31:48.912446+0000","last_clean":"2023-10-06T05:31:48.912446+0000","last_became_active":"2023-10-06T05:31:48.909960+0000","last_became_peered":"2023-10-06T05:31:48.909960+0000","last_unstale":"2023-10-06T05:31:48.912446+0000","last_undegraded":"2023-10-06T05:31:48.912446+0000","last_fullsized":"2023-10-06T05:31:48.912446+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:14.031440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.895210+0000","last_change":"2023-10-06T05:31:47.953179+0000","last_active":"2023-10-06T05:31:48.895210+0000","last_peered":"2023-10-06T05:31:48.895210+0000","last_clean":"2023-10-06T05:31:48.895210+0000","last_became_active":"2023-10-06T05:31:47.952898+0000","last_became_peered":"2023-10-06T05:31:47.952898+0000","last_unstale":"2023-10-06T05:31:48.895210+0000","last_undegraded":"2023-10-06T05:31:48.895210+0000","last_fullsized":"2023-10-06T05:31:48.895210+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:47.822702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.912928+0000","last_change":"2023-10-06T05:31:48.910502+0000","last_active":"2023-10-06T05:31:48.912928+0000","last_peered":"2023-10-06T05:31:48.912928+0000","last_clean":"2023-10-06T05:31:48.912928+0000","last_became_active":"2023-10-06T05:31:48.909897+0000","last_became_peered":"2023-10-06T05:31:48.909897+0000","last_unstale":"2023-10-06T05:31:48.912928+0000","last_undegraded":"2023-10-06T05:31:48.912928+0000","last_fullsized":"2023-10-06T05:31:48.912928+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:23.209259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892653+0000","last_change":"2023-10-06T05:31:47.956512+0000","last_active":"2023-10-06T05:31:48.892653+0000","last_peered":"2023-10-06T05:31:48.892653+0000","last_clean":"2023-10-06T05:31:48.892653+0000","last_became_active":"2023-10-06T05:31:47.955998+0000","last_became_peered":"2023-10-06T05:31:47.955998+0000","last_unstale":"2023-10-06T05:31:48.892653+0000","last_undegraded":"2023-10-06T05:31:48.892653+0000","last_fullsized":"2023-10-06T05:31:48.892653+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:02.398914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914670+0000","last_change":"2023-10-06T05:31:48.905079+0000","last_active":"2023-10-06T05:31:48.914670+0000","last_peered":"2023-10-06T05:31:48.914670+0000","last_clean":"2023-10-06T05:31:48.914670+0000","last_became_active":"2023-10-06T05:31:48.904828+0000","last_became_peered":"2023-10-06T05:31:48.904828+0000","last_unstale":"2023-10-06T05:31:48.914670+0000","last_undegraded":"2023-10-06T05:31:48.914670+0000","last_fullsized":"2023-10-06T05:31:48.914670+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:48.829173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.890154+0000","last_change":"2023-10-06T05:31:47.954583+0000","last_active":"2023-10-06T05:31:48.890154+0000","last_peered":"2023-10-06T05:31:48.890154+0000","last_clean":"2023-10-06T05:31:48.890154+0000","last_became_active":"2023-10-06T05:31:47.954076+0000","last_became_peered":"2023-10-06T05:31:47.954076+0000","last_unstale":"2023-10-06T05:31:48.890154+0000","last_undegraded":"2023-10-06T05:31:48.890154+0000","last_fullsized":"2023-10-06T05:31:48.890154+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:50.673081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907464+0000","last_change":"2023-10-06T05:31:48.903640+0000","last_active":"2023-10-06T05:31:48.907464+0000","last_peered":"2023-10-06T05:31:48.907464+0000","last_clean":"2023-10-06T05:31:48.907464+0000","last_became_active":"2023-10-06T05:31:48.903355+0000","last_became_peered":"2023-10-06T05:31:48.903355+0000","last_unstale":"2023-10-06T05:31:48.907464+0000","last_undegraded":"2023-10-06T05:31:48.907464+0000","last_fullsized":"2023-10-06T05:31:48.907464+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:47.296854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897773+0000","last_change":"2023-10-06T05:31:47.947624+0000","last_active":"2023-10-06T05:31:48.897773+0000","last_peered":"2023-10-06T05:31:48.897773+0000","last_clean":"2023-10-06T05:31:48.897773+0000","last_became_active":"2023-10-06T05:31:47.947358+0000","last_became_peered":"2023-10-06T05:31:47.947358+0000","last_unstale":"2023-10-06T05:31:48.897773+0000","last_undegraded":"2023-10-06T05:31:48.897773+0000","last_fullsized":"2023-10-06T05:31:48.897773+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:34.635886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916102+0000","last_change":"2023-10-06T05:31:48.908449+0000","last_active":"2023-10-06T05:31:48.916102+0000","last_peered":"2023-10-06T05:31:48.916102+0000","last_clean":"2023-10-06T05:31:48.916102+0000","last_became_active":"2023-10-06T05:31:48.908161+0000","last_became_peered":"2023-10-06T05:31:48.908161+0000","last_unstale":"2023-10-06T05:31:48.916102+0000","last_undegraded":"2023-10-06T05:31:48.916102+0000","last_fullsized":"2023-10-06T05:31:48.916102+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.089537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892001+0000","last_change":"2023-10-06T05:31:47.946988+0000","last_active":"2023-10-06T05:31:48.892001+0000","last_peered":"2023-10-06T05:31:48.892001+0000","last_clean":"2023-10-06T05:31:48.892001+0000","last_became_active":"2023-10-06T05:31:47.946693+0000","last_became_peered":"2023-10-06T05:31:47.946693+0000","last_unstale":"2023-10-06T05:31:48.892001+0000","last_undegraded":"2023-10-06T05:31:48.892001+0000","last_fullsized":"2023-10-06T05:31:48.892001+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:46.917459+0000","last_clean_scrub_stamp":"2023-10-06T05:31:46.917459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:47.788784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.914509+0000","last_change":"2023-10-06T05:31:48.909483+0000","last_active":"2023-10-06T05:31:48.914509+0000","last_peered":"2023-10-06T05:31:48.914509+0000","last_clean":"2023-10-06T05:31:48.914509+0000","last_became_active":"2023-10-06T05:31:48.909226+0000","last_became_peered":"2023-10-06T05:31:48.909226+0000","last_unstale":"2023-10-06T05:31:48.914509+0000","last_undegraded":"2023-10-06T05:31:48.914509+0000","last_fullsized":"2023-10-06T05:31:48.914509+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:33.616519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897201+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:31:48.897201+0000","last_peered":"2023-10-06T05:31:48.897201+0000","last_clean":"2023-10-06T05:31:48.897201+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:31:48.897201+0000","last_undegraded":"2023-10-06T05:31:48.897201+0000","last_fullsized":"2023-10-06T05:31:48.897201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916825+0000","last_change":"2023-10-06T05:31:48.909298+0000","last_active":"2023-10-06T05:31:48.916825+0000","last_peered":"2023-10-06T05:31:48.916825+0000","last_clean":"2023-10-06T05:31:48.916825+0000","last_became_active":"2023-10-06T05:31:48.909047+0000","last_became_peered":"2023-10-06T05:31:48.909047+0000","last_unstale":"2023-10-06T05:31:48.916825+0000","last_undegraded":"2023-10-06T05:31:48.916825+0000","last_fullsized":"2023-10-06T05:31:48.916825+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:08.512823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":191,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.892157+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:31:48.892157+0000","last_peered":"2023-10-06T05:31:48.892157+0000","last_clean":"2023-10-06T05:31:48.892157+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:31:48.892157+0000","last_undegraded":"2023-10-06T05:31:48.892157+0000","last_fullsized":"2023-10-06T05:31:48.892157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909668+0000","last_change":"2023-10-06T05:31:48.903122+0000","last_active":"2023-10-06T05:31:48.909668+0000","last_peered":"2023-10-06T05:31:48.909668+0000","last_clean":"2023-10-06T05:31:48.909668+0000","last_became_active":"2023-10-06T05:31:48.902268+0000","last_became_peered":"2023-10-06T05:31:48.902268+0000","last_unstale":"2023-10-06T05:31:48.909668+0000","last_undegraded":"2023-10-06T05:31:48.909668+0000","last_fullsized":"2023-10-06T05:31:48.909668+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:56.958369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":195,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.890264+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:31:48.890264+0000","last_peered":"2023-10-06T05:31:48.890264+0000","last_clean":"2023-10-06T05:31:48.890264+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:31:48.890264+0000","last_undegraded":"2023-10-06T05:31:48.890264+0000","last_fullsized":"2023-10-06T05:31:48.890264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.903827+0000","last_change":"2023-10-06T05:31:48.900601+0000","last_active":"2023-10-06T05:31:48.903827+0000","last_peered":"2023-10-06T05:31:48.903827+0000","last_clean":"2023-10-06T05:31:48.903827+0000","last_became_active":"2023-10-06T05:31:48.900332+0000","last_became_peered":"2023-10-06T05:31:48.900332+0000","last_unstale":"2023-10-06T05:31:48.903827+0000","last_undegraded":"2023-10-06T05:31:48.903827+0000","last_fullsized":"2023-10-06T05:31:48.903827+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:39.239971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":191,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.893521+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:31:48.893521+0000","last_peered":"2023-10-06T05:31:48.893521+0000","last_clean":"2023-10-06T05:31:48.893521+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:31:48.893521+0000","last_undegraded":"2023-10-06T05:31:48.893521+0000","last_fullsized":"2023-10-06T05:31:48.893521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":340,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.899096+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:31:48.899096+0000","last_peered":"2023-10-06T05:31:48.899096+0000","last_clean":"2023-10-06T05:31:48.899096+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:31:48.899096+0000","last_undegraded":"2023-10-06T05:31:48.899096+0000","last_fullsized":"2023-10-06T05:31:48.899096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.907489+0000","last_change":"2023-10-06T05:31:48.904216+0000","last_active":"2023-10-06T05:31:48.907489+0000","last_peered":"2023-10-06T05:31:48.907489+0000","last_clean":"2023-10-06T05:31:48.907489+0000","last_became_active":"2023-10-06T05:31:48.903941+0000","last_became_peered":"2023-10-06T05:31:48.903941+0000","last_unstale":"2023-10-06T05:31:48.907489+0000","last_undegraded":"2023-10-06T05:31:48.907489+0000","last_fullsized":"2023-10-06T05:31:48.907489+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:47.386588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.899015+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:31:48.899015+0000","last_peered":"2023-10-06T05:31:48.899015+0000","last_clean":"2023-10-06T05:31:48.899015+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:31:48.899015+0000","last_undegraded":"2023-10-06T05:31:48.899015+0000","last_fullsized":"2023-10-06T05:31:48.899015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909647+0000","last_change":"2023-10-06T05:31:48.902910+0000","last_active":"2023-10-06T05:31:48.909647+0000","last_peered":"2023-10-06T05:31:48.909647+0000","last_clean":"2023-10-06T05:31:48.909647+0000","last_became_active":"2023-10-06T05:31:48.902281+0000","last_became_peered":"2023-10-06T05:31:48.902281+0000","last_unstale":"2023-10-06T05:31:48.909647+0000","last_undegraded":"2023-10-06T05:31:48.909647+0000","last_fullsized":"2023-10-06T05:31:48.909647+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:09.147475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.898643+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:31:48.898643+0000","last_peered":"2023-10-06T05:31:48.898643+0000","last_clean":"2023-10-06T05:31:48.898643+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:31:48.898643+0000","last_undegraded":"2023-10-06T05:31:48.898643+0000","last_fullsized":"2023-10-06T05:31:48.898643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.916057+0000","last_change":"2023-10-06T05:31:48.909018+0000","last_active":"2023-10-06T05:31:48.916057+0000","last_peered":"2023-10-06T05:31:48.916057+0000","last_clean":"2023-10-06T05:31:48.916057+0000","last_became_active":"2023-10-06T05:31:48.908747+0000","last_became_peered":"2023-10-06T05:31:48.908747+0000","last_unstale":"2023-10-06T05:31:48.916057+0000","last_undegraded":"2023-10-06T05:31:48.916057+0000","last_fullsized":"2023-10-06T05:31:48.916057+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:44.300785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":193,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.897439+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:31:48.897439+0000","last_peered":"2023-10-06T05:31:48.897439+0000","last_clean":"2023-10-06T05:31:48.897439+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:31:48.897439+0000","last_undegraded":"2023-10-06T05:31:48.897439+0000","last_fullsized":"2023-10-06T05:31:48.897439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.909628+0000","last_change":"2023-10-06T05:31:48.901743+0000","last_active":"2023-10-06T05:31:48.909628+0000","last_peered":"2023-10-06T05:31:48.909628+0000","last_clean":"2023-10-06T05:31:48.909628+0000","last_became_active":"2023-10-06T05:31:48.901339+0000","last_became_peered":"2023-10-06T05:31:48.901339+0000","last_unstale":"2023-10-06T05:31:48.909628+0000","last_undegraded":"2023-10-06T05:31:48.909628+0000","last_fullsized":"2023-10-06T05:31:48.909628+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:31:47.923930+0000","last_clean_scrub_stamp":"2023-10-06T05:31:47.923930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:39.715978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":226,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-06T05:31:48.894730+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:31:48.894730+0000","last_peered":"2023-10-06T05:31:48.894730+0000","last_clean":"2023-10-06T05:31:48.894730+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:31:48.894730+0000","last_undegraded":"2023-10-06T05:31:48.894730+0000","last_fullsized":"2023-10-06T05:31:48.894730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575065,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304940,"kb_used_data":3556,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066900,"statfs":{"total":96636764160,"available":96324505600,"internally_reserved":0,"allocated":3641344,"data_stored":2011701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.435,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":1.0069999999999999,"15min":1.1850000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.44400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":1.03,"15min":1.169},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38900000000000001},"max":{"1min":0.84099999999999997,"5min":1.0860000000000001,"15min":1.268},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.47999999999999998,"5min":0.44500000000000001,"15min":0.38300000000000001},"max":{"1min":0.91100000000000003,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83099999999999996,"5min":1.163,"15min":1.163},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0409999999999999,"15min":1.22},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.53400000000000003,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":0.90500000000000003,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.55600000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0529999999999999,"15min":1.117},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.623,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":1.109,"15min":1.258},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.73999999999999999},"min":{"1min":0.53200000000000003,"5min":0.501,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.234,"15min":1.2849999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.51100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":1.109},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.57299999999999995,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.71799999999999997},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.169,"15min":1.169},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.73499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.242,"15min":1.242},"last":0.94999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304340,"kb_used_data":2956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067500,"statfs":{"total":96636764160,"available":96325120000,"internally_reserved":0,"allocated":3026944,"data_stored":1419173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90800000000000003,"5min":1.044,"15min":1.0720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.42399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.79900000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":1.137,"15min":1.137},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.135},"last":0.91300000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.442},"max":{"1min":1.075,"5min":1.115,"15min":1.115},"last":0.98599999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.70899999999999996},"min":{"1min":0.55000000000000004,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":1.1160000000000001},"last":0.875},{"interface":"front","average":{"1min":0.755,"5min":0.752,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.504,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.0049999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.53900000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91600000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.70999999999999996},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.4610000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.53900000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.995,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304344,"kb_used_data":2960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067496,"statfs":{"total":96636764160,"available":96325115904,"internally_reserved":0,"allocated":3031040,"data_stored":1419263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.84699999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.64400000000000002,"5min":0.86099999999999999,"15min":0.88100000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72199999999999998,"5min":1.165,"15min":1.165},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.313},"max":{"1min":0.70699999999999996,"5min":1.077,"15min":1.077},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.371},"max":{"1min":0.78500000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81100000000000005,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.75600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.024,"5min":1.111,"15min":1.5389999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.56399999999999995,"5min":0.52300000000000002,"15min":0.45200000000000001},"max":{"1min":1.0089999999999999,"5min":1.119,"15min":1.4690000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.47899999999999998},"max":{"1min":1.093,"5min":1.248,"15min":1.25},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":1.26,"15min":1.26},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.48199999999999998},"max":{"1min":1.077,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.82099999999999995},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.1459999999999999,"5min":1.2849999999999999,"15min":1.3280000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.80400000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.46999999999999997},"max":{"1min":1.1599999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81399999999999995,"15min":0.80700000000000005},"min":{"1min":0.621,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.1299999999999999,"5min":1.1879999999999999,"15min":2.6240000000000001},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575064,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309024,"kb_used_data":3544,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062816,"statfs":{"total":96636764160,"available":96320323584,"internally_reserved":0,"allocated":3629056,"data_stored":2010129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.245},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.85199999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.498},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82099999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.504,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.70799999999999996,"5min":0.81899999999999995,"15min":0.90800000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.84999999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.58599999999999997,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.432},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.88400000000000001,"15min":0.90200000000000002},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.47299999999999998},"max":{"1min":0.997,"5min":1.05,"15min":2.8220000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.622,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.113,"5min":1.244,"15min":3.6629999999999998},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.65700000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"max":{"1min":1.1060000000000001,"5min":1.27,"15min":1.27},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.59499999999999997,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":1.0389999999999999,"5min":1.097,"15min":1.1970000000000001},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0580000000000001,"5min":1.093,"15min":1.1839999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.45500000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.2629999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80000000000000004,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.25},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.66300000000000003,"5min":0.55500000000000005,"15min":0.50600000000000001},"max":{"1min":1.1459999999999999,"5min":1.167,"15min":1.2789999999999999},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304340,"kb_used_data":2956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067500,"statfs":{"total":96636764160,"available":96325120000,"internally_reserved":0,"allocated":3026944,"data_stored":1419173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.45200000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.056},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70999999999999996,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.0649999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":1.073},"last":0.75}]},{"osd":3,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.96499999999999997},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.486,"15min":1.486},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.76900000000000002},"min":{"1min":0.62,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":1.5,"15min":1.5},"last":0.997}]},{"osd":5,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74399999999999999,"15min":0.76000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1639999999999999},"last":1.093},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.79200000000000004},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1859999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.626,"5min":0.504,"15min":0.504},"max":{"1min":1.014,"5min":1.014,"15min":1.1830000000000001},"last":0.878},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.137},"last":1.145}]},{"osd":7,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78200000000000003,"15min":0.79700000000000004},"min":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.57499999999999996},"max":{"1min":1.0229999999999999,"5min":1.538,"15min":1.538},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.80500000000000005},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.51600000000000001},"max":{"1min":0.96199999999999997,"5min":1.014,"15min":1.2},"last":1.1870000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575065,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304348,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067492,"statfs":{"total":96636764160,"available":96325111808,"internally_reserved":0,"allocated":3035136,"data_stored":1419265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44400000000000001,"15min":0.45100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.254},"max":{"1min":0.60599999999999998,"5min":0.76700000000000002,"15min":1.0720000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.29699999999999999},"max":{"1min":0.61699999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.66900000000000004,"5min":1.369,"15min":1.369},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.81599999999999995,"15min":0.90400000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.754,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.54700000000000004,"15min":0.46000000000000002},"max":{"1min":0.95699999999999996,"5min":1.1100000000000001,"15min":1.2030000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.754,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":1.101,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":1.002,"5min":1.111,"15min":1.1930000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.58399999999999996,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":1.0820000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.79400000000000004},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":1.022,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78800000000000003,"15min":0.79800000000000004},"min":{"1min":0.56299999999999994,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":0.98499999999999999,"5min":1.1339999999999999,"15min":1.3109999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57099999999999995},"max":{"1min":0.98499999999999999,"5min":1.1419999999999999,"15min":1.206},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.81200000000000006},"min":{"1min":0.57899999999999996,"5min":0.53700000000000003,"15min":0.53000000000000003},"max":{"1min":0.93899999999999995,"5min":1.0980000000000001,"15min":1.2090000000000001},"last":0.84699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575063,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304356,"kb_used_data":2972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067484,"statfs":{"total":96636764160,"available":96325103616,"internally_reserved":0,"allocated":3043328,"data_stored":1420799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0249999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97299999999999998,"5min":1.0369999999999999,"15min":1.05},"last":0.747}]},{"osd":1,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.68700000000000006},"min":{"1min":0.52700000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.99099999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":0.98899999999999999,"15min":1.016},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70099999999999996,"15min":0.71599999999999997},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.89400000000000002,"5min":1.115,"15min":1.115},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.018},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":1.163,"15min":1.163},"last":0.97699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81299999999999994,"5min":1.0189999999999999,"15min":1.0920000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45300000000000001},"max":{"1min":0.86499999999999999,"5min":1.0489999999999999,"15min":1.0609999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0169999999999999,"5min":1.282,"15min":1.282},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95799999999999996,"5min":1.254,"15min":1.254},"last":0.83099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575065,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304348,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067492,"statfs":{"total":96636764160,"available":96325111808,"internally_reserved":0,"allocated":3035136,"data_stored":1419229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.69199999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":1.127,"15min":2.4609999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39800000000000002},"max":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":1.3460000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.054,"5min":1.093,"15min":1.2130000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.0289999999999999,"15min":1.204},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":0.95599999999999996,"15min":1.002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.433},"max":{"1min":1.075,"5min":1.145,"15min":1.2769999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.63700000000000001},"min":{"1min":0.52100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.96599999999999997,"15min":1.0229999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.56000000000000005,"5min":0.442,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.02,"15min":1.1539999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.872,"5min":0.91500000000000004,"15min":1.141},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.60799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.746},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.59499999999999997,"5min":0.51200000000000001,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.248},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:31:54.047 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:31:54.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:31:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.244+0000 7f58e959d700 1 Processor -- start 2023-10-06T05:31:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.245+0000 7f58e959d700 1 -- start start 2023-10-06T05:31:54.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e959d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58e4156370 0x7f58e40aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e959d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f58e4156740 0x7f58e40cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e40cc750 0x7f58e40c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e959d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e406beb0 con 0x7f58e4156370 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e959d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e406b330 con 0x7f58e4156740 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e406b610 con 0x7f58e40cc750 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58e4156370 0x7f58e40aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58e4156370 0x7f58e40aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60866/0 (socket says 172.21.15.33:60866) 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 -- 172.21.15.33:0/320617057 learned_addr learned my addr 172.21.15.33:0/320617057 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:54.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 -- 172.21.15.33:0/320617057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e40cc750 msgr2=0x7f58e40c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 --2- 172.21.15.33:0/320617057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e40cc750 0x7f58e40c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 -- 172.21.15.33:0/320617057 >> v2:172.21.15.167:3300/0 conn(0x7f58e4156740 msgr2=0x7f58e40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 --2- 172.21.15.33:0/320617057 >> v2:172.21.15.167:3300/0 conn(0x7f58e4156740 0x7f58e40cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.246+0000 7f58e3fff700 1 -- 172.21.15.33:0/320617057 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e406b8f0 con 0x7f58e4156370 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.247+0000 7f58e3fff700 1 --2- 172.21.15.33:0/320617057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e4156370 0x7f58e40aef80 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f58e40694d0 tx=0x7f58d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63ae89f5213a4b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.247+0000 7f58e2ffd700 1 -- 172.21.15.33:0/320617057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400b860 con 0x7f58e4156370 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.247+0000 7f58e2ffd700 1 -- 172.21.15.33:0/320617057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d400b9c0 con 0x7f58e4156370 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.247+0000 7f58e2ffd700 1 -- 172.21.15.33:0/320617057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400bd60 con 0x7f58e4156370 2023-10-06T05:31:54.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.247+0000 7f58e959d700 1 -- 172.21.15.33:0/320617057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e4156370 msgr2=0x7f58e40aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.247+0000 7f58e959d700 1 --2- 172.21.15.33:0/320617057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e4156370 0x7f58e40aef80 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f58e40694d0 tx=0x7f58d4009250 comp rx=0 tx=0).stop 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 -- 172.21.15.33:0/320617057 shutdown_connections 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 --2- 172.21.15.33:0/320617057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e40cc750 0x7f58e40c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 --2- 172.21.15.33:0/320617057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e4156370 0x7f58e40aef80 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 --2- 172.21.15.33:0/320617057 >> v2:172.21.15.167:3300/0 conn(0x7f58e4156740 0x7f58e40cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 -- 172.21.15.33:0/320617057 >> 172.21.15.33:0/320617057 conn(0x7f58e406c560 msgr2=0x7f58e406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 -- 172.21.15.33:0/320617057 shutdown_connections 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.248+0000 7f58e959d700 1 -- 172.21.15.33:0/320617057 wait complete. 2023-10-06T05:31:54.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.249+0000 7f58e959d700 1 Processor -- start 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.249+0000 7f58e959d700 1 -- start start 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e959d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 0x7f58e414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e959d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e4156370 0x7f58e414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e959d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e4156740 0x7f58e414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e959d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e414cc50 con 0x7f58e40cc750 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e959d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e414ac30 con 0x7f58e4156740 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e959d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e414c600 con 0x7f58e4156370 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 0x7f58e414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 0x7f58e414d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60882/0 (socket says 172.21.15.33:60882) 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.250+0000 7f58e3fff700 1 -- 172.21.15.33:0/2838175603 learned_addr learned my addr 172.21.15.33:0/2838175603 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e3fff700 1 -- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e4156370 msgr2=0x7f58e414a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e3fff700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e4156370 0x7f58e414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e3fff700 1 -- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e4156740 msgr2=0x7f58e414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e3fff700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e4156740 0x7f58e414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e3fff700 1 -- 172.21.15.33:0/2838175603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40094f0 con 0x7f58e40cc750 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e3fff700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 0x7f58e414d0e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009290 tx=0x7f58d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4011e20 con 0x7f58e40cc750 2023-10-06T05:31:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d401f070 con 0x7f58e40cc750 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d401a890 con 0x7f58e40cc750 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.251+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e414c8e0 con 0x7f58e40cc750 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.252+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e40abdb0 con 0x7f58e40cc750 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.252+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e404c380 con 0x7f58e40cc750 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.253+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58d401ac40 con 0x7f58e40cc750 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.254+0000 7f58cbfff700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58d0066f30 0x7f58d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.254+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f58d4021070 con 0x7f58e40cc750 2023-10-06T05:31:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.255+0000 7f58e37fe700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58d0066f30 0x7f58d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.255+0000 7f58e37fe700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58d0066f30 0x7f58d00693e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f58d8006fd0 tx=0x7f58d8006d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:54.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.259+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58d40537e0 con 0x7f58e40cc750 2023-10-06T05:31:54.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.546+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f58e406dec0 con 0x7f58e40cc750 2023-10-06T05:31:54.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.550+0000 7f58cbfff700 1 -- 172.21.15.33:0/2838175603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) v1 ==== 190+0+0 (secure 0 0 0) 0x7f58d4056e00 con 0x7f58e40cc750 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58d0066f30 msgr2=0x7f58d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58d0066f30 0x7f58d00693e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f58d8006fd0 tx=0x7f58d8006d30 comp rx=0 tx=0).stop 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 msgr2=0x7f58e414d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 0x7f58e414d0e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009290 tx=0x7f58d4017000 comp rx=0 tx=0).stop 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 shutdown_connections 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58d0066f30 0x7f58d00693e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58e4156740 0x7f58e414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40cc750 0x7f58e414d0e0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 --2- 172.21.15.33:0/2838175603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58e4156370 0x7f58e414a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 >> 172.21.15.33:0/2838175603 conn(0x7f58e406c560 msgr2=0x7f58e41537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 shutdown_connections 2023-10-06T05:31:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.553+0000 7f58e959d700 1 -- 172.21.15.33:0/2838175603 wait complete. 2023-10-06T05:31:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:31:54.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.757+0000 7fd0ca79d700 1 Processor -- start 2023-10-06T05:31:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.758+0000 7fd0ca79d700 1 -- start start 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0ca79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0c413faa0 0x7fd0c413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0ca79d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0c40c7940 0x7fd0c40c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0ca79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c406aad0 0x7fd0c406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0ca79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c40c9cb0 con 0x7fd0c413faa0 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0ca79d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c40ca270 con 0x7fd0c40c7940 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0ca79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c406b840 con 0x7fd0c406aad0 2023-10-06T05:31:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0c979b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0c413faa0 0x7fd0c413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0c979b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0c413faa0 0x7fd0c413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60890/0 (socket says 172.21.15.33:60890) 2023-10-06T05:31:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0c979b700 1 -- 172.21.15.33:0/1967100934 learned_addr learned my addr 172.21.15.33:0/1967100934 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0c9f9c700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c406aad0 0x7fd0c406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.759+0000 7fd0c8f9a700 1 --2- 172.21.15.33:0/1967100934 >> v2:172.21.15.167:3300/0 conn(0x7fd0c40c7940 0x7fd0c40c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c979b700 1 -- 172.21.15.33:0/1967100934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c406aad0 msgr2=0x7fd0c406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c979b700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c406aad0 0x7fd0c406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c979b700 1 -- 172.21.15.33:0/1967100934 >> v2:172.21.15.167:3300/0 conn(0x7fd0c40c7940 msgr2=0x7fd0c40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c979b700 1 --2- 172.21.15.33:0/1967100934 >> v2:172.21.15.167:3300/0 conn(0x7fd0c40c7940 0x7fd0c40c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c979b700 1 -- 172.21.15.33:0/1967100934 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c406bb20 con 0x7fd0c413faa0 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c9f9c700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c406aad0 0x7fd0c406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c8f9a700 1 --2- 172.21.15.33:0/1967100934 >> v2:172.21.15.167:3300/0 conn(0x7fd0c40c7940 0x7fd0c40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0c979b700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c413faa0 0x7fd0c413fe70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40cbef0 tx=0x7fd0c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4c9521f276f657 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0bbfff700 1 -- 172.21.15.33:0/1967100934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000b820 con 0x7fd0c413faa0 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0bbfff700 1 -- 172.21.15.33:0/1967100934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c000b980 con 0x7fd0c413faa0 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.760+0000 7fd0bbfff700 1 -- 172.21.15.33:0/1967100934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000bd20 con 0x7fd0c413faa0 2023-10-06T05:31:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1967100934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c413faa0 msgr2=0x7fd0c413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c413faa0 0x7fd0c413fe70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40cbef0 tx=0x7fd0c0009250 comp rx=0 tx=0).stop 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1967100934 shutdown_connections 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c406aad0 0x7fd0c406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1967100934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c413faa0 0x7fd0c413fe70 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1967100934 >> v2:172.21.15.167:3300/0 conn(0x7fd0c40c7940 0x7fd0c40c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1967100934 >> 172.21.15.33:0/1967100934 conn(0x7fd0c406d030 msgr2=0x7fd0c406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1967100934 shutdown_connections 2023-10-06T05:31:54.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.761+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1967100934 wait complete. 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.762+0000 7fd0ca79d700 1 Processor -- start 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.763+0000 7fd0ca79d700 1 -- start start 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.763+0000 7fd0ca79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 0x7fd0c40c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.763+0000 7fd0ca79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c40c7940 0x7fd0c413cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0ca79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c413faa0 0x7fd0c4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0ca79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c413d0f0 con 0x7fd0c406aad0 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0ca79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4169910 con 0x7fd0c413faa0 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0ca79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c413d300 con 0x7fd0c40c7940 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 0x7fd0c40c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 0x7fd0c40c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60906/0 (socket says 172.21.15.33:60906) 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 -- 172.21.15.33:0/1532363267 learned_addr learned my addr 172.21.15.33:0/1532363267 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c8f9a700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c40c7940 0x7fd0c413cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c9f9c700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c413faa0 0x7fd0c4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 -- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c40c7940 msgr2=0x7fd0c413cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c40c7940 0x7fd0c413cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 -- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c413faa0 msgr2=0x7fd0c4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c413faa0 0x7fd0c4158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 -- 172.21.15.33:0/1532363267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00094f0 con 0x7fd0c406aad0 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c8f9a700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c40c7940 0x7fd0c413cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c9f9c700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c413faa0 0x7fd0c4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.764+0000 7fd0c979b700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 0x7fd0c40c55c0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00094c0 tx=0x7fd0c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.765+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0026480 con 0x7fd0c406aad0 2023-10-06T05:31:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.765+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c001f070 con 0x7fd0c406aad0 2023-10-06T05:31:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.765+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0026830 con 0x7fd0c406aad0 2023-10-06T05:31:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.765+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c413d5e0 con 0x7fd0c406aad0 2023-10-06T05:31:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.765+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c413dba0 con 0x7fd0c406aad0 2023-10-06T05:31:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.766+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c404c380 con 0x7fd0c406aad0 2023-10-06T05:31:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.767+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0c0026be0 con 0x7fd0c406aad0 2023-10-06T05:31:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.768+0000 7fd0b97fa700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0a8066f30 0x7fd0a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.768+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0c0021070 con 0x7fd0c406aad0 2023-10-06T05:31:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.769+0000 7fd0c8f9a700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0a8066f30 0x7fd0a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.769+0000 7fd0c8f9a700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0a8066f30 0x7fd0a80693e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd0c413f440 tx=0x7fd0b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:54.773+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0c0053510 con 0x7fd0c406aad0 2023-10-06T05:31:54.922 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:31:54.921+0000 7fecc8df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:31:55.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.021+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0c40c9f00 con 0x7fd0c406aad0 2023-10-06T05:31:55.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.021+0000 7fd0b97fa700 1 -- 172.21.15.33:0/1532363267 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fd0c0056b30 con 0x7fd0c406aad0 2023-10-06T05:31:55.023 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:55.023 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:31:48.429792+0000","modified":"2023-10-06T05:31:51.962941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11169},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11169":{"gid":11169,"name":"a","rank":0,"incarnation":181,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1245472058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1245472058},{"type":"v1","addr":"172.21.15.33:6835","nonce":1245472058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0a8066f30 msgr2=0x7fd0a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0a8066f30 0x7fd0a80693e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd0c413f440 tx=0x7fd0b4008040 comp rx=0 tx=0).stop 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 msgr2=0x7fd0c40c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 0x7fd0c40c55c0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00094c0 tx=0x7fd0c0017000 comp rx=0 tx=0).stop 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 shutdown_connections 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0a8066f30 0x7fd0a80693e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c413faa0 0x7fd0c4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c406aad0 0x7fd0c40c55c0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 --2- 172.21.15.33:0/1532363267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0c40c7940 0x7fd0c413cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 >> 172.21.15.33:0/1532363267 conn(0x7fd0c406d030 msgr2=0x7fd0c4159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 shutdown_connections 2023-10-06T05:31:55.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.024+0000 7fd0ca79d700 1 -- 172.21.15.33:0/1532363267 wait complete. 2023-10-06T05:31:55.027 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 185 2023-10-06T05:31:55.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 185, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:31:48.429792+0000', 'modified': '2023-10-06T05:31:51.962941+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11169}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11169': {'gid': 11169, 'name': 'a', 'rank': 0, 'incarnation': 181, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/1245472058', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6833', 'nonce': 1245472058}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 1245472058}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [36], 'metadata_pool': 35, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:31:55.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:31:55.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:31:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.635+0000 7f756359d700 1 Processor -- start 2023-10-06T05:31:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.635+0000 7f756359d700 1 -- start start 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.636+0000 7f756359d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f755c1596c0 0x7f755c159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.636+0000 7f756359d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f755c06f0d0 0x7f755c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.636+0000 7f756359d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c06ec40 con 0x7f755c06dbb0 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.636+0000 7f756359d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c06e030 con 0x7f755c1596c0 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.636+0000 7f756359d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c06e310 con 0x7f755c06f0d0 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f755c1596c0 0x7f755c159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f755c1596c0 0x7f755c159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60914/0 (socket says 172.21.15.33:60914) 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 -- 172.21.15.33:0/3530009165 learned_addr learned my addr 172.21.15.33:0/3530009165 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f7561d9a700 1 --2- 172.21.15.33:0/3530009165 >> v2:172.21.15.167:3300/0 conn(0x7f755c06f0d0 0x7f755c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 -- 172.21.15.33:0/3530009165 >> v2:172.21.15.167:3300/0 conn(0x7f755c06f0d0 msgr2=0x7f755c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 --2- 172.21.15.33:0/3530009165 >> v2:172.21.15.167:3300/0 conn(0x7f755c06f0d0 0x7f755c06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 -- 172.21.15.33:0/3530009165 >> v1:172.21.15.167:6789/0 conn(0x7f755c06dbb0 legacy=0x7f755c0af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.637+0000 7f756259b700 1 -- 172.21.15.33:0/3530009165 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c06e5f0 con 0x7f755c1596c0 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.638+0000 7f756259b700 1 --2- 172.21.15.33:0/3530009165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c159a90 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f75580099b0 tx=0x7f7558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04179a6143071ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.638+0000 7f7561599700 1 -- 172.21.15.33:0/3530009165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558009ea0 con 0x7f755c1596c0 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.638+0000 7f7561599700 1 -- 172.21.15.33:0/3530009165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800b820 con 0x7f755c1596c0 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.638+0000 7f7561599700 1 -- 172.21.15.33:0/3530009165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800bbc0 con 0x7f755c1596c0 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.638+0000 7f756359d700 1 -- 172.21.15.33:0/3530009165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 msgr2=0x7f755c159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.638+0000 7f756359d700 1 --2- 172.21.15.33:0/3530009165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c159a90 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f75580099b0 tx=0x7f7558009250 comp rx=0 tx=0).stop 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.639+0000 7f756359d700 1 -- 172.21.15.33:0/3530009165 shutdown_connections 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.639+0000 7f756359d700 1 --2- 172.21.15.33:0/3530009165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c159a90 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.639+0000 7f756359d700 1 --2- 172.21.15.33:0/3530009165 >> v2:172.21.15.167:3300/0 conn(0x7f755c06f0d0 0x7f755c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.639+0000 7f756359d700 1 -- 172.21.15.33:0/3530009165 >> 172.21.15.33:0/3530009165 conn(0x7f755c087fb0 msgr2=0x7f755c0883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.639+0000 7f756359d700 1 -- 172.21.15.33:0/3530009165 shutdown_connections 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.639+0000 7f756359d700 1 -- 172.21.15.33:0/3530009165 wait complete. 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.640+0000 7f756359d700 1 Processor -- start 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.640+0000 7f756359d700 1 -- start start 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f756359d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c06f0d0 0x7f755c0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f756359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f756359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f755c1580c0 0x7f755c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f756359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c16ac40 con 0x7f755c1596c0 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f756359d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c16a030 con 0x7f755c06f0d0 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f756359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c16a310 con 0x7f755c1580c0 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.641+0000 7f7561d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c169940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60918/0 (socket says 172.21.15.33:60918) 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 -- 172.21.15.33:0/3111325692 learned_addr learned my addr 172.21.15.33:0/3111325692 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f756259b700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c06f0d0 0x7f755c0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7562d9c700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f755c1580c0 0x7f755c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 -- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f755c1580c0 msgr2=0x7f755c15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f755c1580c0 0x7f755c15bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 -- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c06f0d0 msgr2=0x7f755c0c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c06f0d0 0x7f755c0c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 -- 172.21.15.33:0/3111325692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580094f0 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f7561d9a700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c169940 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ec80 tx=0x7f754c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f75527fc700 1 -- 172.21.15.33:0/3111325692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c00c850 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f75527fc700 1 -- 172.21.15.33:0/3111325692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c00c9b0 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f75527fc700 1 -- 172.21.15.33:0/3111325692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c00cd30 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.642+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c16a5f0 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.643+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c15c120 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.645+0000 7f75527fc700 1 -- 172.21.15.33:0/3111325692 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f754c013680 con 0x7f755c1596c0 2023-10-06T05:31:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.646+0000 7f75527fc700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7540066f30 0x7f75400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.646+0000 7f75527fc700 1 -- 172.21.15.33:0/3111325692 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f754c084d80 con 0x7f755c1596c0 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.646+0000 7f756259b700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7540066f30 0x7f75400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.647+0000 7f756259b700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7540066f30 0x7f75400693e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f75580099b0 tx=0x7f7558017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.647+0000 7f7572ff6b80 1 Processor -- start 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 -- start start 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093b6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093b97920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093aeec80 con 0x562093b6e950 2023-10-06T05:31:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093ac5a70 con 0x562093bb8bc0 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.648+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562093ac3270 con 0x562093b97e60 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093b6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093b6ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60934/0 (socket says 172.21.15.33:60934) 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 -- 172.21.15.33:0/2795514998 learned_addr learned my addr 172.21.15.33:0/2795514998 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f7561d9a700 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093b97920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f7562d9c700 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 -- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 msgr2=0x562093b6a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 -- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 msgr2=0x562093b97920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093b97920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 -- 172.21.15.33:0/2795514998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562093959270 con 0x562093b6e950 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.649+0000 7f756259b700 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093b6ed20 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f755c06a970 tx=0x7f7558021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48637dc8562104a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f753f7fe700 1 -- 172.21.15.33:0/2795514998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755802d070 con 0x562093b6e950 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f753f7fe700 1 -- 172.21.15.33:0/2795514998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755801a930 con 0x562093b6e950 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f753f7fe700 1 -- 172.21.15.33:0/2795514998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558021070 con 0x562093b6e950 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2795514998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 msgr2=0x562093b6ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093b6ed20 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f755c06a970 tx=0x7f7558021000 comp rx=0 tx=0).stop 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2795514998 shutdown_connections 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093b6ed20 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2795514998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093b97920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2795514998 >> 172.21.15.33:0/2795514998 conn(0x562093b70270 msgr2=0x562093b70640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.650+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2795514998 shutdown_connections 2023-10-06T05:31:55.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.651+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2795514998 wait complete. 2023-10-06T05:31:55.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.651+0000 7f7572ff6b80 1 -- start start 2023-10-06T05:31:55.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.651+0000 7f7572ff6b80 1 Processor -- start 2023-10-06T05:31:55.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b6e950 0x562093b93ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093b97e60 0x562093bc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 0x562093b65790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093abf060 con 0x562093bb8bc0 2023-10-06T05:31:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093a6a890 con 0x562093b97e60 2023-10-06T05:31:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562093a85ea0 con 0x562093b6e950 2023-10-06T05:31:55.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.652+0000 7f7572ff6b80 10 client.0 _ll_register_callbacks cb 0x56209395bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 0x562093b65790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 0x562093b65790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60938/0 (socket says 172.21.15.33:60938) 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 -- 172.21.15.33:0/365081764 learned_addr learned my addr 172.21.15.33:0/365081764 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f756259b700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b6e950 0x562093b93ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7561d9a700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093b97e60 0x562093bc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 -- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b6e950 msgr2=0x562093b93ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b6e950 0x562093b93ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 -- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093b97e60 msgr2=0x562093bc54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093b97e60 0x562093bc54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 -- 172.21.15.33:0/365081764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c06e5f0 con 0x562093bb8bc0 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f7562d9c700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 0x562093b65790 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7554001840 tx=0x7f7554009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1848be40420748 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.653+0000 7f753cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f753cff9700 1 -- 172.21.15.33:0/365081764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540153d0 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f753cff9700 1 -- 172.21.15.33:0/365081764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7554017c90 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562093b63690 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f753cff9700 1 -- 172.21.15.33:0/365081764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=185) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7554003a20 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 10 client.11307 fetch_fsmap learned FSMap version 185 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=185,monmap=2+}) v3 -- 0x5620938fbfe0 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f753cff9700 1 -- 172.21.15.33:0/365081764 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562093b63690 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f753cff9700 1 -- 172.21.15.33:0/365081764 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 185) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7554015740 con 0x562093bb8bc0 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 10 client.11307 fetch_fsmap finished waiting for FSMap version 185 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 10 client.11307 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:31:55.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 20 client.11307 populate_metadata read hostname 'smithi033' 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093bb2780 0x562093bb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 4 client.11307 mds_command: new command op to 11169 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.654+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 --> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562093b405c0 con 0x562093bb2780 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.655+0000 7f756259b700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093bb2780 0x562093bb2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.655+0000 7f753cff9700 10 client.11307 ms_handle_connect on v2:172.21.15.33:6833/1245472058 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.655+0000 7f756259b700 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093bb2780 0x562093bb2b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216dedb7ea6c4321 server_cookie=b432fa16acb89408 in_seq=0 out_seq=0 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f753cff9700 1 -- 172.21.15.33:0/365081764 <== mds.0 v2:172.21.15.33:6833/1245472058 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562093b405c0 con 0x562093bb2780 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f753cff9700 10 client.11307 handle_command_reply: tid=0 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 client.11307 shutdown 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 msgr2=0x562093b65790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 0x562093b65790 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7554001840 tx=0x7f7554009040 comp rx=0 tx=0).stop 2023-10-06T05:31:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 shutdown_connections 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093bb8bc0 0x562093b65790 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b6e950 0x562093b93ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093bb2780 0x562093bb2b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/365081764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093b97e60 0x562093bc54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 >> 172.21.15.33:0/365081764 conn(0x562093b70270 msgr2=0x562093bb44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f753cff9700 0 client.11307 ms_handle_reset on v2:172.21.15.33:6833/1245472058 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 shutdown_connections 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.656+0000 7f7572ff6b80 1 -- 172.21.15.33:0/365081764 wait complete. 2023-10-06T05:31:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.657+0000 7f7572ff6b80 20 client.11307 trim_cache size 0 max 16384 2023-10-06T05:31:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.674+0000 7f7572ff6b80 1 Processor -- start 2023-10-06T05:31:55.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.674+0000 7f7572ff6b80 1 -- start start 2023-10-06T05:31:55.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093bb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bb8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093941e30 con 0x562093b6e950 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093b6c540 con 0x562093bb8bc0 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562093bb2bf0 con 0x562093b97e60 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093bb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093bb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60946/0 (socket says 172.21.15.33:60946) 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 -- 172.21.15.33:0/2392947869 learned_addr learned my addr 172.21.15.33:0/2392947869 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7562d9c700 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bb8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f7561d9a700 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 -- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 msgr2=0x562093b6aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 -- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 msgr2=0x562093bb8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bb8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.675+0000 7f756259b700 1 -- 172.21.15.33:0/2392947869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562093a8d780 con 0x562093b6e950 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f756259b700 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093bb1680 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f7558031000 tx=0x7f755801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f5ab6ac7bd5d22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f753ffff700 1 -- 172.21.15.33:0/2392947869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558045070 con 0x562093b6e950 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f753ffff700 1 -- 172.21.15.33:0/2392947869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755801acf0 con 0x562093b6e950 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f753ffff700 1 -- 172.21.15.33:0/2392947869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755802d040 con 0x562093b6e950 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2392947869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 msgr2=0x562093bb1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093bb1680 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f7558031000 tx=0x7f755801c040 comp rx=0 tx=0).stop 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2392947869 shutdown_connections 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.676+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bb8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093bb1680 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2392947869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093b6aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2392947869 >> 172.21.15.33:0/2392947869 conn(0x562093b70270 msgr2=0x562093b657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2392947869 shutdown_connections 2023-10-06T05:31:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2392947869 wait complete. 2023-10-06T05:31:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 -- start start 2023-10-06T05:31:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.677+0000 7f7572ff6b80 1 Processor -- start 2023-10-06T05:31:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093c149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093bc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093c1a890 con 0x562093b6e950 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562093c14ee0 con 0x562093bb8bc0 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x562093c1c010 con 0x562093b97e60 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.678+0000 7f7572ff6b80 10 client.0 _ll_register_callbacks cb 0x56209395bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093c149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093c149a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60952/0 (socket says 172.21.15.33:60952) 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 -- 172.21.15.33:0/2186205421 learned_addr learned my addr 172.21.15.33:0/2186205421 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f7561d9a700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093bc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f7562d9c700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 -- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 msgr2=0x562093bc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093bc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 -- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 msgr2=0x562093bc9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bc9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 -- 172.21.15.33:0/2186205421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562093959270 con 0x562093b6e950 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f7562d9c700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f756259b700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093c149a0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f7558043040 tx=0x7f755803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f24d0eab7c6eab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.679+0000 7f753cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f753cff9700 1 -- 172.21.15.33:0/2186205421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558045050 con 0x562093b6e950 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f753cff9700 1 -- 172.21.15.33:0/2186205421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7558041aa0 con 0x562093b6e950 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f755801a930 con 0x562093b6e950 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f753cff9700 1 -- 172.21.15.33:0/2186205421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=185) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755802d040 con 0x562093b6e950 2023-10-06T05:31:55.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 10 client.11313 fetch_fsmap learned FSMap version 185 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=185,monmap=2+}) v3 -- 0x562093a74470 con 0x562093b6e950 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f753cff9700 1 -- 172.21.15.33:0/2186205421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755801a930 con 0x562093b6e950 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f753cff9700 1 -- 172.21.15.33:0/2186205421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 185) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f755803a040 con 0x562093b6e950 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 10 client.11313 fetch_fsmap finished waiting for FSMap version 185 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 10 client.11313 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 20 client.11313 populate_metadata read hostname 'smithi033' 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093b96a10 0x562093b96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 4 client.11313 mds_command: new command op to 11169 tid=0[{"prefix": "status"}] 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.680+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 --> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562093a61e00 con 0x562093b96a10 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7561d9a700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093b96a10 0x562093b96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7561d9a700 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093b96a10 0x562093b96de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed7d864de22aaac server_cookie=5a1e42b78908bc in_seq=0 out_seq=0 2023-10-06T05:31:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f753cff9700 10 client.11313 ms_handle_connect on v2:172.21.15.33:6833/1245472058 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f753cff9700 1 -- 172.21.15.33:0/2186205421 <== mds.0 v2:172.21.15.33:6833/1245472058 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x562093a61e00 con 0x562093b96a10 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f753cff9700 10 client.11313 handle_command_reply: tid=0 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 client.11313 shutdown 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 msgr2=0x562093c149a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093c149a0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f7558043040 tx=0x7f755803b000 comp rx=0 tx=0).stop 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 shutdown_connections 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:6833/1245472058,v1:172.21.15.33:6835/1245472058] conn(0x562093b96a10 0x562093b96de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562093bb8bc0 0x562093bc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562093b6e950 0x562093c149a0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 --2- 172.21.15.33:0/2186205421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x562093b97e60 0x562093bc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.681+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 >> 172.21.15.33:0/2186205421 conn(0x562093b70270 msgr2=0x562093bade30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f753cff9700 0 client.11313 ms_handle_reset on v2:172.21.15.33:6833/1245472058 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 shutdown_connections 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f7572ff6b80 1 -- 172.21.15.33:0/2186205421 wait complete. 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f7572ff6b80 20 client.11313 trim_cache size 0 max 16384 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:31:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 11169, 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.7364238839999997, 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 185, 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 239, 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 239, 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.483620365 2023-10-06T05:31:55.746 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:31:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7540066f30 msgr2=0x7f75400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7540066f30 0x7f75400693e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f75580099b0 tx=0x7f7558017000 comp rx=0 tx=0).stop 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 msgr2=0x7f755c169940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c169940 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ec80 tx=0x7f754c009120 comp rx=0 tx=0).stop 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 shutdown_connections 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7540066f30 0x7f75400693e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f755c1580c0 0x7f755c15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c06f0d0 0x7f755c0c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 --2- 172.21.15.33:0/3111325692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f755c1596c0 0x7f755c169940 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.682+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 >> 172.21.15.33:0/3111325692 conn(0x7f755c087fb0 msgr2=0x7f755c0b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.683+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 shutdown_connections 2023-10-06T05:31:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.683+0000 7f756359d700 1 -- 172.21.15.33:0/3111325692 wait complete. 2023-10-06T05:31:55.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:31:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.954+0000 7fad3b59e700 1 Processor -- start 2023-10-06T05:31:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.954+0000 7fad3b59e700 1 -- start start 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c0c8050 0x7fad3c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fad3c06aad0 0x7fad3c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c0c9cb0 con 0x7fad3c13faa0 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c0ca270 con 0x7fad3c06aad0 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c06b840 con 0x7fad3c0c8050 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fad3c13faa0 0x7fad3c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35076/0 (socket says 172.21.15.33:35076) 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3a59c700 1 -- 172.21.15.33:0/2489692259 learned_addr learned my addr 172.21.15.33:0/2489692259 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad3ad9d700 1 --2- 172.21.15.33:0/2489692259 >> v2:172.21.15.167:3300/0 conn(0x7fad3c06aad0 0x7fad3c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.956+0000 7fad39d9b700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c0c8050 0x7fad3c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3959a700 1 -- 172.21.15.33:0/2489692259 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676486860 0 0) 0x7fad3c0c9cb0 con 0x7fad3c13faa0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3959a700 1 -- 172.21.15.33:0/2489692259 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad24003570 con 0x7fad3c13faa0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3ad9d700 1 -- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c0c8050 msgr2=0x7fad3c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3ad9d700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c0c8050 0x7fad3c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3ad9d700 1 -- 172.21.15.33:0/2489692259 >> v1:172.21.15.33:6789/0 conn(0x7fad3c13faa0 legacy=0x7fad3c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3ad9d700 1 -- 172.21.15.33:0/2489692259 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c06bb20 con 0x7fad3c06aad0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad39d9b700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c0c8050 0x7fad3c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3ad9d700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c06aad0 0x7fad3c06af50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fad3400d0d0 tx=0x7fad3400b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c80f99a9d2c930 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3959a700 1 -- 172.21.15.33:0/2489692259 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34010070 con 0x7fad3c06aad0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3959a700 1 -- 172.21.15.33:0/2489692259 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3400b2e0 con 0x7fad3c06aad0 2023-10-06T05:31:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.957+0000 7fad3959a700 1 -- 172.21.15.33:0/2489692259 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400dc90 con 0x7fad3c06aad0 2023-10-06T05:31:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 -- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c06aad0 msgr2=0x7fad3c06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c06aad0 0x7fad3c06af50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fad3400d0d0 tx=0x7fad3400b990 comp rx=0 tx=0).stop 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 -- 172.21.15.33:0/2489692259 shutdown_connections 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c06aad0 0x7fad3c06af50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 --2- 172.21.15.33:0/2489692259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c0c8050 0x7fad3c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 -- 172.21.15.33:0/2489692259 >> 172.21.15.33:0/2489692259 conn(0x7fad3c06d030 msgr2=0x7fad3c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 -- 172.21.15.33:0/2489692259 shutdown_connections 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.958+0000 7fad3b59e700 1 -- 172.21.15.33:0/2489692259 wait complete. 2023-10-06T05:31:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.960+0000 7fad3b59e700 1 Processor -- start 2023-10-06T05:31:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.960+0000 7fad3b59e700 1 -- start start 2023-10-06T05:31:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 0x7fad3c16d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0c8050 0x7fad3c13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c13faa0 0x7fad3c0c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c16db10 con 0x7fad3c06aad0 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c159bf0 con 0x7fad3c0c8050 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c159070 con 0x7fad3c13faa0 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 0x7fad3c16d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 0x7fad3c16d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60964/0 (socket says 172.21.15.33:60964) 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 -- 172.21.15.33:0/513115153 learned_addr learned my addr 172.21.15.33:0/513115153 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad39d9b700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0c8050 0x7fad3c13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3ad9d700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c13faa0 0x7fad3c0c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 -- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c13faa0 msgr2=0x7fad3c0c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c13faa0 0x7fad3c0c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 -- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0c8050 msgr2=0x7fad3c13ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0c8050 0x7fad3c13ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad3a59c700 1 -- 172.21.15.33:0/513115153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad30009420 con 0x7fad3c06aad0 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.961+0000 7fad39d9b700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0c8050 0x7fad3c13ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.962+0000 7fad3a59c700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 0x7fad3c16d5d0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c580 tx=0x7fad2c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.962+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c005b30 con 0x7fad3c06aad0 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.962+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c005c90 con 0x7fad3c06aad0 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.962+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c007320 con 0x7fad3c06aad0 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.962+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3400bc30 con 0x7fad3c06aad0 2023-10-06T05:31:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.962+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c159630 con 0x7fad3c06aad0 2023-10-06T05:31:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.963+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c04c380 con 0x7fad3c06aad0 2023-10-06T05:31:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.964+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad2c007690 con 0x7fad3c06aad0 2023-10-06T05:31:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.965+0000 7fad2a7fc700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad1c066f30 0x7fad1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:55.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.965+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad2c083ad0 con 0x7fad3c06aad0 2023-10-06T05:31:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.966+0000 7fad39d9b700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad1c066f30 0x7fad1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.966+0000 7fad39d9b700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad1c066f30 0x7fad1c0693e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fad3c157c60 tx=0x7fad300091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:55.969+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad2c050b40 con 0x7fad3c06aad0 2023-10-06T05:31:56.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.218+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fad3c0c9f00 con 0x7fad3c06aad0 2023-10-06T05:31:56.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.219+0000 7fad2a7fc700 1 -- 172.21.15.33:0/513115153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v185) v1 ==== 53+0+73 (secure 0 0 0) 0x7fad2c017020 con 0x7fad3c06aad0 2023-10-06T05:31:56.221 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad1c066f30 msgr2=0x7fad1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad1c066f30 0x7fad1c0693e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fad3c157c60 tx=0x7fad300091a0 comp rx=0 tx=0).stop 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 msgr2=0x7fad3c16d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 0x7fad3c16d5d0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c580 tx=0x7fad2c007df0 comp rx=0 tx=0).stop 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 shutdown_connections 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fad1c066f30 0x7fad1c0693e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fad3c13faa0 0x7fad3c0c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad3c06aad0 0x7fad3c16d5d0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:56.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 --2- 172.21.15.33:0/513115153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad3c0c8050 0x7fad3c13ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 >> 172.21.15.33:0/513115153 conn(0x7fad3c06d030 msgr2=0x7fad3c0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 shutdown_connections 2023-10-06T05:31:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:56.221+0000 7fad3b59e700 1 -- 172.21.15.33:0/513115153 wait complete. 2023-10-06T05:31:56.240 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:31:56.240 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:31:56.240 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:31:56.240 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:31:56.241 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:31:56.241 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:31:56.241 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:31:56.241 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:31:56.241 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:31:56.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:56.241 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:31:56.278 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:31:56.279 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41241sec preferred_lft 41241sec 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:56.280 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:31:56.281 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:56.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:56.282 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:31:56.378 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:31:56.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:56.382 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:31:56.421 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:31:56.421 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:31:56.421 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:31:56.421 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:31:56.422 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:31:56.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:56.422 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:31:56.422 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:31:56.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:31:56.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:31:56.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:31:56.422 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:31:56.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:56.620 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:31:56.657 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:31:56.657 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:31:56.657 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:31:56.657 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:31:56.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:56.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:31:56.759 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:31:56.759 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:31:56.796 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:31:56.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:31:56.796 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:31:56.873 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:31:56.885 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:31:56.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:31:57.007 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:31:57.007 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:31:57.007 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:31:57.023 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:31:57.024 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:31:57.024 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:31:57.024 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:31:57.024 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:31:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:31:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:31:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:31:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:31:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:31:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:31:57.047 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:31:57.047 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:31:57.051 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:31:57.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:57.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:31:57.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:31:57.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.301+0000 7f6af099d700 1 Processor -- start 2023-10-06T05:31:57.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.301+0000 7f6af099d700 1 -- start start 2023-10-06T05:31:57.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.302+0000 7f6af099d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6aec0cb710 0x7f6aec06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.302+0000 7f6af099d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0c9cb0 con 0x7f6aec13faa0 2023-10-06T05:31:57.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.302+0000 7f6af099d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0ca270 con 0x7f6aec0c8050 2023-10-06T05:31:57.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.302+0000 7f6af099d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec06b840 con 0x7f6aec0cb710 2023-10-06T05:31:57.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.302+0000 7f6aeb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6aec13faa0 0x7f6aec0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:35080/0 (socket says 172.21.15.33:35080) 2023-10-06T05:31:57.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.302+0000 7f6aeb7fe700 1 -- 172.21.15.33:0/1631073983 learned_addr learned my addr 172.21.15.33:0/1631073983 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:31:57.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aebfff700 1 --2- 172.21.15.33:0/1631073983 >> v2:172.21.15.167:3300/0 conn(0x7f6aec0cb710 0x7f6aec06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049306526 0 0) 0x7f6aec0c9cb0 con 0x7f6aec13faa0 2023-10-06T05:31:57.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad4003570 con 0x7f6aec13faa0 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 307958027 0 0) 0x7f6ad4003570 con 0x7f6aec13faa0 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 >> v2:172.21.15.167:3300/0 conn(0x7f6aec0cb710 msgr2=0x7f6aec06b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 --2- 172.21.15.33:0/1631073983 >> v2:172.21.15.167:3300/0 conn(0x7f6aec0cb710 0x7f6aec06b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 >> v1:172.21.15.167:6789/0 conn(0x7f6aec0c8050 legacy=0x7f6aec06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec06bb20 con 0x7f6aec13faa0 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f6adc001470 con 0x7f6aec13faa0 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 >> v1:172.21.15.33:6789/0 conn(0x7f6aec13faa0 legacy=0x7f6aec0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4003a60 0x7f6ad4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad4004810 0x7f6ad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ad40051c0 0x7f6ad400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec06b840 con 0x7f6ad4004810 2023-10-06T05:31:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad400c0d0 con 0x7f6ad4003a60 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.303+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad400c290 con 0x7f6ad40051c0 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad4004810 0x7f6ad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeb7fe700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4003a60 0x7f6ad4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aebfff700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ad40051c0 0x7f6ad400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 -- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ad40051c0 msgr2=0x7f6ad400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ad40051c0 0x7f6ad400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 -- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4003a60 msgr2=0x7f6ad4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4003a60 0x7f6ad4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 -- 172.21.15.33:0/1631073983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4003570 con 0x7f6ad4004810 2023-10-06T05:31:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aebfff700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ad40051c0 0x7f6ad400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aebfff700 1 -- 172.21.15.33:0/1631073983 reap_dead start 2023-10-06T05:31:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.304+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad4004810 0x7f6ad4004c80 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001360 tx=0x7f6ae0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bd0006f5a6506b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00104a0 con 0x7f6ad4004810 2023-10-06T05:31:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0010620 con 0x7f6ad4004810 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6aea7fc700 1 -- 172.21.15.33:0/1631073983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00107d0 con 0x7f6ad4004810 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 -- 172.21.15.33:0/1631073983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad4004810 msgr2=0x7f6ad4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad4004810 0x7f6ad4004c80 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001360 tx=0x7f6ae0007d30 comp rx=0 tx=0).stop 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 -- 172.21.15.33:0/1631073983 shutdown_connections 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 --2- 172.21.15.33:0/1631073983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad4004810 0x7f6ad4004c80 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 -- 172.21.15.33:0/1631073983 >> 172.21.15.33:0/1631073983 conn(0x7f6aec06d030 msgr2=0x7f6aec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 -- 172.21.15.33:0/1631073983 shutdown_connections 2023-10-06T05:31:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.305+0000 7f6af099d700 1 -- 172.21.15.33:0/1631073983 wait complete. 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.307+0000 7f6af099d700 1 Processor -- start 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.307+0000 7f6af099d700 1 -- start start 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6af099d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4004810 0x7f6aec156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6af099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 0x7f6aec157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6af099d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aec0c52e0 0x7f6aec0c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6af099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c5c90 con 0x7f6aec156df0 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6af099d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec157960 con 0x7f6ad4004810 2023-10-06T05:31:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6af099d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec157c40 con 0x7f6aec0c52e0 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 0x7f6aec157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 0x7f6aec157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60980/0 (socket says 172.21.15.33:60980) 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 -- 172.21.15.33:0/2717427046 learned_addr learned my addr 172.21.15.33:0/2717427046 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeb7fe700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4004810 0x7f6aec156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aebfff700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aec0c52e0 0x7f6aec0c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 -- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aec0c52e0 msgr2=0x7f6aec0c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aec0c52e0 0x7f6aec0c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 -- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4004810 msgr2=0x7f6aec156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4004810 0x7f6aec156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeaffd700 1 -- 172.21.15.33:0/2717427046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec157f20 con 0x7f6aec156df0 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.308+0000 7f6aeb7fe700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4004810 0x7f6aec156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.309+0000 7f6aeaffd700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 0x7f6aec157260 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001e70 tx=0x7f6ae0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.309+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0001200 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.309+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00104a0 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.309+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0011e70 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.309+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec158200 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.309+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec0aefb0 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.310+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec0b04d0 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.311+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ae001a680 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.312+0000 7f6ad37fe700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ad8066f30 0x7f6ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.312+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ae0015020 con 0x7f6aec156df0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.313+0000 7f6aeb7fe700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ad8066f30 0x7f6ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.314+0000 7f6aeb7fe700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ad8066f30 0x7f6ad80693e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000e10 tx=0x7f6adc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.317+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6aec0b04d0 con 0x7f6aec156df0 2023-10-06T05:31:57.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.563+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6aec0b04d0 con 0x7f6aec156df0 2023-10-06T05:31:57.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.563+0000 7f6ad37fe700 1 -- 172.21.15.33:0/2717427046 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v185) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6aec0b04d0 con 0x7f6aec156df0 2023-10-06T05:31:57.565 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:31:57.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ad8066f30 msgr2=0x7f6ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ad8066f30 0x7f6ad80693e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000e10 tx=0x7f6adc009040 comp rx=0 tx=0).stop 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 msgr2=0x7f6aec157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 0x7f6aec157260 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001e70 tx=0x7f6ae0017000 comp rx=0 tx=0).stop 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 shutdown_connections 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ad8066f30 0x7f6ad80693e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aec0c52e0 0x7f6aec0c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ad4004810 0x7f6aec156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 --2- 172.21.15.33:0/2717427046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec156df0 0x7f6aec157260 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 >> 172.21.15.33:0/2717427046 conn(0x7f6aec06d030 msgr2=0x7f6aec0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:57.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 shutdown_connections 2023-10-06T05:31:57.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:57.566+0000 7f6af099d700 1 -- 172.21.15.33:0/2717427046 wait complete. 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:31:57.585 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:31:57.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:57.586 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:31:57.626 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:31:57.626 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:31:57.626 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:31:57.627 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41243sec preferred_lft 41243sec 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:31:57.628 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:31:57.629 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:31:57.630 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:31:57.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:57.646 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:31:57.733 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:31:57.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:57.736 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:31:57.775 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:31:57.775 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:31:57.775 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:31:57.775 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:31:57.775 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:31:57.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:57.776 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:31:57.776 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:31:57.776 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:31:57.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:31:57.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:31:57.777 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:31:57.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:57.975 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:31:58.011 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:31:58.011 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:31:58.012 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:31:58.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:58.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:31:58.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:31:58.115 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:31:58.151 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:31:58.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:31:58.151 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:31:58.228 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:31:58.228 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:31:58.228 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:31:58.358 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:31:58.358 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:31:58.358 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:31:58.374 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:31:58.374 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:31:58.374 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:31:58.374 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:31:58.375 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:31:58.395 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:31:58.395 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:31:58.395 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:31:58.396 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:31:58.396 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:31:58.396 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:31:58.396 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:31:58.396 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:31:58.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:31:58.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:58.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:31:58.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:31:58.676 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:31:58.685 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:31:58.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:31:58.889 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:31:58.892 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:31:58.893 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:31:58.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:31:58.982 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:31:59.066 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:31:59.067 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:31:59.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:31:59.067 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:31:59.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:31:59.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:59.151 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:31:59.188 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:31:59.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:31:59.188 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:31:59.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:31:59.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:31:59.302 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:31:59.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.526+0000 7f3b741ff700 1 Processor -- start 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.526+0000 7f3b741ff700 1 -- start start 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.527+0000 7f3b741ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.527+0000 7f3b741ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3b6c15ce50 0x7f3b6c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.527+0000 7f3b741ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3b6c15d910 0x7f3b6c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.527+0000 7f3b741ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c0ab250 con 0x7f3b6c15ce50 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.527+0000 7f3b741ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c06e660 con 0x7f3b6c15d910 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.527+0000 7f3b741ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c06e940 con 0x7f3b6c15ca80 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3b6c15ce50 0x7f3b6c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3b6c15ce50 0x7f3b6c15d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60992/0 (socket says 172.21.15.33:60992) 2023-10-06T05:31:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 -- 172.21.15.33:0/1438594565 learned_addr learned my addr 172.21.15.33:0/1438594565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b71f9b700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7279c700 1 --2- 172.21.15.33:0/1438594565 >> v2:172.21.15.167:3300/0 conn(0x7f3b6c15d910 0x7f3b6c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 -- 172.21.15.33:0/1438594565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 msgr2=0x7f3b6c06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c06e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 -- 172.21.15.33:0/1438594565 >> v2:172.21.15.167:3300/0 conn(0x7f3b6c15d910 msgr2=0x7f3b6c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 --2- 172.21.15.33:0/1438594565 >> v2:172.21.15.167:3300/0 conn(0x7f3b6c15d910 0x7f3b6c15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 -- 172.21.15.33:0/1438594565 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c06ec20 con 0x7f3b6c15ce50 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b71f9b700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7279c700 1 --2- 172.21.15.33:0/1438594565 >> v2:172.21.15.167:3300/0 conn(0x7f3b6c15d910 0x7f3b6c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b7179a700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c15d3d0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c06b900 tx=0x7f3b5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a334cb243138d901 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.528+0000 7f3b70f99700 1 -- 172.21.15.33:0/1438594565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00b860 con 0x7f3b6c15ce50 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b70f99700 1 -- 172.21.15.33:0/1438594565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c00b9c0 con 0x7f3b6c15ce50 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b70f99700 1 -- 172.21.15.33:0/1438594565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00bd60 con 0x7f3b6c15ce50 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 -- 172.21.15.33:0/1438594565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15ce50 msgr2=0x7f3b6c15d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c15d3d0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c06b900 tx=0x7f3b5c009250 comp rx=0 tx=0).stop 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 -- 172.21.15.33:0/1438594565 shutdown_connections 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 --2- 172.21.15.33:0/1438594565 >> v2:172.21.15.167:3300/0 conn(0x7f3b6c15d910 0x7f3b6c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 --2- 172.21.15.33:0/1438594565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c15d3d0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 -- 172.21.15.33:0/1438594565 >> 172.21.15.33:0/1438594565 conn(0x7f3b6c06cd40 msgr2=0x7f3b6c087fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 -- 172.21.15.33:0/1438594565 shutdown_connections 2023-10-06T05:31:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.529+0000 7f3b741ff700 1 -- 172.21.15.33:0/1438594565 wait complete. 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.531+0000 7f3b741ff700 1 Processor -- start 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.531+0000 7f3b741ff700 1 -- start start 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b741ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c09b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b741ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c16aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b741ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 0x7f3b6c16d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b741ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c16e5f0 con 0x7f3b6c15d910 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b741ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c16a5a0 con 0x7f3b6c15ce50 2023-10-06T05:31:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b741ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c09bd90 con 0x7f3b6c15ca80 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 0x7f3b6c16d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b71f9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c09b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 0x7f3b6c16d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32770/0 (socket says 172.21.15.33:32770) 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 -- 172.21.15.33:0/36125107 learned_addr learned my addr 172.21.15.33:0/36125107 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7179a700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c16aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 -- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 msgr2=0x7f3b6c09b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c09b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 -- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c15ce50 msgr2=0x7f3b6c16aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c16aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.532+0000 7f3b7279c700 1 -- 172.21.15.33:0/36125107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0094f0 con 0x7f3b6c15d910 2023-10-06T05:31:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.533+0000 7f3b7279c700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 0x7f3b6c16d690 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400d100 tx=0x7f3b640099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.533+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400dc40 con 0x7f3b6c15d910 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.533+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6400f040 con 0x7f3b6c15d910 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.533+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6c16e030 con 0x7f3b6c15d910 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.533+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015440 con 0x7f3b6c15d910 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.533+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6c16a230 con 0x7f3b6c15d910 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.534+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c16a090 con 0x7f3b6c15d910 2023-10-06T05:31:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.535+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b6401e3b0 con 0x7f3b6c15d910 2023-10-06T05:31:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.536+0000 7f3b61ffb700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b50066f30 0x7f3b500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:31:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.536+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b64084a70 con 0x7f3b6c15d910 2023-10-06T05:31:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.537+0000 7f3b71f9b700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b50066f30 0x7f3b500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:31:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.537+0000 7f3b71f9b700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b50066f30 0x7f3b500693e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3b68006f00 tx=0x7f3b68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:31:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.541+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b6c16a090 con 0x7f3b6c15d910 2023-10-06T05:31:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.788+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b6c16a090 con 0x7f3b6c15d910 2023-10-06T05:31:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.789+0000 7f3b61ffb700 1 -- 172.21.15.33:0/36125107 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f3b6c16a090 con 0x7f3b6c15d910 2023-10-06T05:31:59.790 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:31:59.791 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:31:48.429792+0000","modified":"2023-10-06T05:31:51.962941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11169},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11169":{"gid":11169,"name":"a","rank":0,"incarnation":181,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1245472058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1245472058},{"type":"v1","addr":"172.21.15.33:6835","nonce":1245472058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2023-10-06T05:31:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b50066f30 msgr2=0x7f3b500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b50066f30 0x7f3b500693e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3b68006f00 tx=0x7f3b68008040 comp rx=0 tx=0).stop 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 msgr2=0x7f3b6c16d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 0x7f3b6c16d690 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400d100 tx=0x7f3b640099d0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 shutdown_connections 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3b50066f30 0x7f3b500693e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b6c15d910 0x7f3b6c16d690 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15ca80 0x7f3b6c09b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 --2- 172.21.15.33:0/36125107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c15ce50 0x7f3b6c16aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 >> 172.21.15.33:0/36125107 conn(0x7f3b6c06cd40 msgr2=0x7f3b6c0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 shutdown_connections 2023-10-06T05:31:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.791+0000 7f3b741ff700 1 -- 172.21.15.33:0/36125107 wait complete. 2023-10-06T05:31:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 185 2023-10-06T05:31:59.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:31:59.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.996+0000 7f6a72d9d700 1 Processor -- start 2023-10-06T05:32:00.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.996+0000 7f6a72d9d700 1 -- start start 2023-10-06T05:32:00.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a72d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6a6c155140 0x7f6a6c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a72d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0699c0 0x7f6a6c069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a72d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0c8e40 con 0x7f6a6c0adf50 2023-10-06T05:32:00.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a72d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c06a380 con 0x7f6a6c155140 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a72d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c99e0 con 0x7f6a6c0699c0 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a7159a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6a6c0adf50 0x7f6a6c0c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37242/0 (socket says 172.21.15.33:37242) 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a7159a700 1 -- 172.21.15.33:0/437015046 learned_addr learned my addr 172.21.15.33:0/437015046 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a7259c700 1 --2- 172.21.15.33:0/437015046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0699c0 0x7f6a6c069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a71d9b700 1 --2- 172.21.15.33:0/437015046 >> v2:172.21.15.167:3300/0 conn(0x7f6a6c155140 0x7f6a6c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a70d99700 1 -- 172.21.15.33:0/437015046 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571425766 0 0) 0x7f6a6c0c8e40 con 0x7f6a6c0adf50 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.998+0000 7f6a70d99700 1 -- 172.21.15.33:0/437015046 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a58003570 con 0x7f6a6c0adf50 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.999+0000 7f6a7259c700 1 -- 172.21.15.33:0/437015046 >> v2:172.21.15.167:3300/0 conn(0x7f6a6c155140 msgr2=0x7f6a6c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.999+0000 7f6a7259c700 1 --2- 172.21.15.33:0/437015046 >> v2:172.21.15.167:3300/0 conn(0x7f6a6c155140 0x7f6a6c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.999+0000 7f6a7259c700 1 -- 172.21.15.33:0/437015046 >> v1:172.21.15.33:6789/0 conn(0x7f6a6c0adf50 legacy=0x7f6a6c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.999+0000 7f6a7259c700 1 -- 172.21.15.33:0/437015046 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0ae3f0 con 0x7f6a6c0699c0 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.999+0000 7f6a71d9b700 1 --2- 172.21.15.33:0/437015046 >> v2:172.21.15.167:3300/0 conn(0x7f6a6c155140 0x7f6a6c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:31:59.999+0000 7f6a7259c700 1 --2- 172.21.15.33:0/437015046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0699c0 0x7f6a6c069e40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009e90 tx=0x7f6a64009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa14249378a294f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a70d99700 1 -- 172.21.15.33:0/437015046 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400e070 con 0x7f6a6c0699c0 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a70d99700 1 -- 172.21.15.33:0/437015046 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6400bb40 con 0x7f6a6c0699c0 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a70d99700 1 -- 172.21.15.33:0/437015046 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64012430 con 0x7f6a6c0699c0 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a72d9d700 1 -- 172.21.15.33:0/437015046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0699c0 msgr2=0x7f6a6c069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/437015046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0699c0 0x7f6a6c069e40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009e90 tx=0x7f6a64009780 comp rx=0 tx=0).stop 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a72d9d700 1 -- 172.21.15.33:0/437015046 shutdown_connections 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/437015046 >> v2:172.21.15.167:3300/0 conn(0x7f6a6c155140 0x7f6a6c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/437015046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0699c0 0x7f6a6c069e40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.000+0000 7f6a72d9d700 1 -- 172.21.15.33:0/437015046 >> 172.21.15.33:0/437015046 conn(0x7f6a6c06d030 msgr2=0x7f6a6c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.001+0000 7f6a72d9d700 1 -- 172.21.15.33:0/437015046 shutdown_connections 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.001+0000 7f6a72d9d700 1 -- 172.21.15.33:0/437015046 wait complete. 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.002+0000 7f6a72d9d700 1 Processor -- start 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.002+0000 7f6a72d9d700 1 -- start start 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a72d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 0x7f6a6c152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a72d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0adf50 0x7f6a6c167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a72d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a6c155140 0x7f6a6c0c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a72d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c152750 con 0x7f6a6c155140 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a72d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c5e40 con 0x7f6a6c0699c0 2023-10-06T05:32:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a72d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c155b60 con 0x7f6a6c0adf50 2023-10-06T05:32:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a71d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 0x7f6a6c152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a71d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 0x7f6a6c152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:60062/0 (socket says 172.21.15.33:60062) 2023-10-06T05:32:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a7259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a6c155140 0x7f6a6c0c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a71d9b700 1 -- 172.21.15.33:0/269931443 learned_addr learned my addr 172.21.15.33:0/269931443 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.003+0000 7f6a7159a700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0adf50 0x7f6a6c167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a71d9b700 1 -- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0adf50 msgr2=0x7f6a6c167ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a71d9b700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0adf50 0x7f6a6c167ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a71d9b700 1 -- 172.21.15.33:0/269931443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a6c155140 msgr2=0x7f6a6c0c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a71d9b700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a6c155140 0x7f6a6c0c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a71d9b700 1 -- 172.21.15.33:0/269931443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68009420 con 0x7f6a6c0699c0 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a7259c700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a6c155140 0x7f6a6c0c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a71d9b700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 0x7f6a6c152210 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800ec80 tx=0x7f6a680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6800c920 con 0x7f6a6c0699c0 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6800ca80 con 0x7f6a6c0699c0 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a64009a20 con 0x7f6a6c0699c0 2023-10-06T05:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.004+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6800ce00 con 0x7f6a6c0699c0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.005+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c0c6110 con 0x7f6a6c0699c0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.006+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c0c6490 con 0x7f6a6c0699c0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.006+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a68013680 con 0x7f6a6c0699c0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.007+0000 7f6a61ffb700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a50066f30 0x7f6a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.007+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6a68084e20 con 0x7f6a6c0699c0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.008+0000 7f6a7159a700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a50066f30 0x7f6a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.008+0000 7f6a7159a700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a50066f30 0x7f6a500693e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007fd0 tx=0x7f6a5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.012+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a68051e90 con 0x7f6a6c0699c0 2023-10-06T05:32:00.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.218+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a6c04c380 con 0x7f6a6c0699c0 2023-10-06T05:32:00.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.220+0000 7f6a61ffb700 1 -- 172.21.15.33:0/269931443 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f6a68071200 con 0x7f6a6c0699c0 2023-10-06T05:32:00.221 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:00.222 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":239,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:49.887625+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:31:46.602215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-06T05:31:47.403110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a50066f30 msgr2=0x7f6a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a50066f30 0x7f6a500693e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007fd0 tx=0x7f6a5c009040 comp rx=0 tx=0).stop 2023-10-06T05:32:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 msgr2=0x7f6a6c152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 0x7f6a6c152210 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800ec80 tx=0x7f6a680091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 shutdown_connections 2023-10-06T05:32:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a50066f30 0x7f6a500693e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a6c155140 0x7f6a6c0c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a6c0699c0 0x7f6a6c152210 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 --2- 172.21.15.33:0/269931443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a6c0adf50 0x7f6a6c167ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 >> 172.21.15.33:0/269931443 conn(0x7f6a6c06d030 msgr2=0x7f6a6c0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 shutdown_connections 2023-10-06T05:32:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.222+0000 7f6a72d9d700 1 -- 172.21.15.33:0/269931443 wait complete. 2023-10-06T05:32:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:31:46.602215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:31:47.403110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:32:00.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:32:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.430+0000 7f389a39d700 1 Processor -- start 2023-10-06T05:32:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.430+0000 7f389a39d700 1 -- start start 2023-10-06T05:32:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389a39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f389406c7b0 0x7f3894067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389a39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389415d3a0 con 0x7f389406cb80 2023-10-06T05:32:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389a39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940b0470 con 0x7f38940aff90 2023-10-06T05:32:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389a39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940a9fa0 con 0x7f389406c7b0 2023-10-06T05:32:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f389406c7b0 0x7f3894067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389939b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f389406c7b0 0x7f3894067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46590/0 (socket says 172.21.15.33:46590) 2023-10-06T05:32:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389939b700 1 -- 172.21.15.33:0/3076313084 learned_addr learned my addr 172.21.15.33:0/3076313084 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f3899b9c700 1 --1- 172.21.15.33:0/3076313084 >> v1:172.21.15.33:6789/0 conn(0x7f389406cb80 0x7f389415d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37256/0 (socket says 172.21.15.33:37256) 2023-10-06T05:32:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389939b700 1 -- 172.21.15.33:0/3076313084 >> v1:172.21.15.167:6789/0 conn(0x7f38940aff90 legacy=0x7f38940a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.432+0000 7f389939b700 1 -- 172.21.15.33:0/3076313084 >> v1:172.21.15.33:6789/0 conn(0x7f389406cb80 legacy=0x7f389415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f389939b700 1 -- 172.21.15.33:0/3076313084 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940b0750 con 0x7f389406c7b0 2023-10-06T05:32:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f389939b700 1 --2- 172.21.15.33:0/3076313084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f3894067fb0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f3890009990 tx=0x7f3890009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da217b022fba193 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f388bfff700 1 -- 172.21.15.33:0/3076313084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890005c10 con 0x7f389406c7b0 2023-10-06T05:32:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f388bfff700 1 -- 172.21.15.33:0/3076313084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890005d70 con 0x7f389406c7b0 2023-10-06T05:32:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f388bfff700 1 -- 172.21.15.33:0/3076313084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389000b860 con 0x7f389406c7b0 2023-10-06T05:32:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f389a39d700 1 -- 172.21.15.33:0/3076313084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 msgr2=0x7f3894067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.433+0000 7f389a39d700 1 --2- 172.21.15.33:0/3076313084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f3894067fb0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f3890009990 tx=0x7f3890009220 comp rx=0 tx=0).stop 2023-10-06T05:32:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.434+0000 7f389a39d700 1 -- 172.21.15.33:0/3076313084 shutdown_connections 2023-10-06T05:32:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.434+0000 7f389a39d700 1 --2- 172.21.15.33:0/3076313084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f3894067fb0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.434+0000 7f389a39d700 1 -- 172.21.15.33:0/3076313084 >> 172.21.15.33:0/3076313084 conn(0x7f389406d670 msgr2=0x7f389406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.434+0000 7f389a39d700 1 -- 172.21.15.33:0/3076313084 shutdown_connections 2023-10-06T05:32:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.434+0000 7f389a39d700 1 -- 172.21.15.33:0/3076313084 wait complete. 2023-10-06T05:32:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.435+0000 7f389a39d700 1 Processor -- start 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.435+0000 7f389a39d700 1 -- start start 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389a39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f389416a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389a39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389406cb80 0x7f3894149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389a39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38940aff90 0x7f3894151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389a39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389416b170 con 0x7f389406c7b0 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389a39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389416a640 con 0x7f38940aff90 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389a39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389414a4f0 con 0x7f389406cb80 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f389416a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389939b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f389416a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46594/0 (socket says 172.21.15.33:46594) 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.436+0000 7f389939b700 1 -- 172.21.15.33:0/2117837697 learned_addr learned my addr 172.21.15.33:0/2117837697 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f3899b9c700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38940aff90 0x7f3894151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f3898b9a700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389406cb80 0x7f3894149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389939b700 1 -- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389406cb80 msgr2=0x7f3894149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389939b700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389406cb80 0x7f3894149fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389939b700 1 -- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38940aff90 msgr2=0x7f3894151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389939b700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38940aff90 0x7f3894151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389939b700 1 -- 172.21.15.33:0/2117837697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890009480 con 0x7f389406c7b0 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f3899b9c700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38940aff90 0x7f3894151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f3898b9a700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389406cb80 0x7f3894149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389939b700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f389416a100 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f389406f2c0 tx=0x7f3890016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890011ae0 con 0x7f389406c7b0 2023-10-06T05:32:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890011c60 con 0x7f389406c7b0 2023-10-06T05:32:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389001b430 con 0x7f389406c7b0 2023-10-06T05:32:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.437+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389414e730 con 0x7f389406c7b0 2023-10-06T05:32:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.438+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389416abb0 con 0x7f389406c7b0 2023-10-06T05:32:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.439+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389404c380 con 0x7f389406c7b0 2023-10-06T05:32:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.439+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f389001b840 con 0x7f389406c7b0 2023-10-06T05:32:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.440+0000 7f38897fa700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3878066f30 0x7f38780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.440+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f38900861e0 con 0x7f389406c7b0 2023-10-06T05:32:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.441+0000 7f3898b9a700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3878066f30 0x7f38780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.441+0000 7f3898b9a700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3878066f30 0x7f38780693e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f389416d6c0 tx=0x7f3884009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.444+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3890053250 con 0x7f389406c7b0 2023-10-06T05:32:00.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.646+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f389406efb0 con 0x7f389406c7b0 2023-10-06T05:32:00.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.647+0000 7f38897fa700 1 -- 172.21.15.33:0/2117837697 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v239) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3890056870 con 0x7f389406c7b0 2023-10-06T05:32:00.649 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3878066f30 msgr2=0x7f38780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3878066f30 0x7f38780693e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f389416d6c0 tx=0x7f3884009040 comp rx=0 tx=0).stop 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 msgr2=0x7f389416a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f389416a100 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f389406f2c0 tx=0x7f3890016000 comp rx=0 tx=0).stop 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 shutdown_connections 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3878066f30 0x7f38780693e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38940aff90 0x7f3894151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389406c7b0 0x7f389416a100 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 --2- 172.21.15.33:0/2117837697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f389406cb80 0x7f3894149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 >> 172.21.15.33:0/2117837697 conn(0x7f389406d670 msgr2=0x7f3894154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:00.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 shutdown_connections 2023-10-06T05:32:00.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.649+0000 7f389a39d700 1 -- 172.21.15.33:0/2117837697 wait complete. 2023-10-06T05:32:00.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:32:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.860+0000 7fa25579d700 1 Processor -- start 2023-10-06T05:32:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.860+0000 7fa25579d700 1 -- start start 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa25579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa250156370 0x7fa2500af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa25579d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa2500cc7e0 0x7fa2500c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa25579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2500ad430 con 0x7fa2500b01f0 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa25579d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2500afe20 con 0x7fa2500cc7e0 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa25579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25006bc70 con 0x7fa250156370 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa254f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa2500b01f0 0x7fa2500ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37264/0 (socket says 172.21.15.33:37264) 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa254f9c700 1 -- 172.21.15.33:0/4161802356 learned_addr learned my addr 172.21.15.33:0/4161802356 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa24f7fe700 1 --2- 172.21.15.33:0/4161802356 >> v2:172.21.15.167:3300/0 conn(0x7fa2500cc7e0 0x7fa2500c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa24ffff700 1 --2- 172.21.15.33:0/4161802356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa250156370 0x7fa2500af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.862+0000 7fa24effd700 1 -- 172.21.15.33:0/4161802356 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246175090 0 0) 0x7fa2500ad430 con 0x7fa2500b01f0 2023-10-06T05:32:00.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24effd700 1 -- 172.21.15.33:0/4161802356 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa238003570 con 0x7fa2500b01f0 2023-10-06T05:32:00.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24ffff700 1 -- 172.21.15.33:0/4161802356 >> v2:172.21.15.167:3300/0 conn(0x7fa2500cc7e0 msgr2=0x7fa2500c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:00.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24ffff700 1 --2- 172.21.15.33:0/4161802356 >> v2:172.21.15.167:3300/0 conn(0x7fa2500cc7e0 0x7fa2500c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24ffff700 1 -- 172.21.15.33:0/4161802356 >> v1:172.21.15.33:6789/0 conn(0x7fa2500b01f0 legacy=0x7fa2500ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24ffff700 1 -- 172.21.15.33:0/4161802356 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500c92f0 con 0x7fa250156370 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24f7fe700 1 --2- 172.21.15.33:0/4161802356 >> v2:172.21.15.167:3300/0 conn(0x7fa2500cc7e0 0x7fa2500c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24ffff700 1 --2- 172.21.15.33:0/4161802356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa250156370 0x7fa2500af8e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa2400098e0 tx=0x7fa2400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3a571703716409 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24effd700 1 -- 172.21.15.33:0/4161802356 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240011630 con 0x7fa250156370 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24effd700 1 -- 172.21.15.33:0/4161802356 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240011790 con 0x7fa250156370 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.863+0000 7fa24effd700 1 -- 172.21.15.33:0/4161802356 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240011b30 con 0x7fa250156370 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 -- 172.21.15.33:0/4161802356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa250156370 msgr2=0x7fa2500af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 --2- 172.21.15.33:0/4161802356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa250156370 0x7fa2500af8e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa2400098e0 tx=0x7fa2400091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 -- 172.21.15.33:0/4161802356 shutdown_connections 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 --2- 172.21.15.33:0/4161802356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa250156370 0x7fa2500af8e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 --2- 172.21.15.33:0/4161802356 >> v2:172.21.15.167:3300/0 conn(0x7fa2500cc7e0 0x7fa2500c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 -- 172.21.15.33:0/4161802356 >> 172.21.15.33:0/4161802356 conn(0x7fa25006d030 msgr2=0x7fa2500cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:00.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 -- 172.21.15.33:0/4161802356 shutdown_connections 2023-10-06T05:32:00.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.864+0000 7fa25579d700 1 -- 172.21.15.33:0/4161802356 wait complete. 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.866+0000 7fa25579d700 1 Processor -- start 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.866+0000 7fa25579d700 1 -- start start 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa25579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2500b01f0 0x7fa2500a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa25579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 0x7fa250159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa25579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa250156370 0x7fa250146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa25579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa250159690 con 0x7fa2500cc7e0 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa25579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500a6450 con 0x7fa250156370 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa25579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500a6730 con 0x7fa2500b01f0 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 0x7fa250159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 0x7fa250159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46610/0 (socket says 172.21.15.33:46610) 2023-10-06T05:32:00.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 -- 172.21.15.33:0/59578119 learned_addr learned my addr 172.21.15.33:0/59578119 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24ffff700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2500b01f0 0x7fa2500a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa254f9c700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa250156370 0x7fa250146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 -- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2500b01f0 msgr2=0x7fa2500a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2500b01f0 0x7fa2500a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 -- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa250156370 msgr2=0x7fa250146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa250156370 0x7fa250146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.867+0000 7fa24f7fe700 1 -- 172.21.15.33:0/59578119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009420 con 0x7fa2500cc7e0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.868+0000 7fa24f7fe700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 0x7fa250159150 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa24400ec80 tx=0x7fa2440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.868+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400c850 con 0x7fa2500cc7e0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.868+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24400c9b0 con 0x7fa2500cc7e0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.868+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa240009420 con 0x7fa2500cc7e0 2023-10-06T05:32:00.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.868+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400cd50 con 0x7fa2500cc7e0 2023-10-06T05:32:00.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.868+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2500a6cf0 con 0x7fa2500cc7e0 2023-10-06T05:32:00.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.869+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2500a7340 con 0x7fa2500cc7e0 2023-10-06T05:32:00.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.870+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa244013680 con 0x7fa2500cc7e0 2023-10-06T05:32:00.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.871+0000 7fa237fff700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.871+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa2440847f0 con 0x7fa2500cc7e0 2023-10-06T05:32:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.871+0000 7fa24ffff700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:00.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.872+0000 7fa24ffff700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa23c066f30 0x7fa23c0693e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa2400091e0 tx=0x7fa240017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:00.875+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa244051860 con 0x7fa2500cc7e0 2023-10-06T05:32:01.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.076+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa25004c380 con 0x7fa2500cc7e0 2023-10-06T05:32:01.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.077+0000 7fa237fff700 1 -- 172.21.15.33:0/59578119 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v239) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa244054e80 con 0x7fa2500cc7e0 2023-10-06T05:32:01.078 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:32:01.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa23c066f30 msgr2=0x7fa23c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa23c066f30 0x7fa23c0693e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa2400091e0 tx=0x7fa240017000 comp rx=0 tx=0).stop 2023-10-06T05:32:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 msgr2=0x7fa250159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 0x7fa250159150 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa24400ec80 tx=0x7fa2440091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 shutdown_connections 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa250156370 0x7fa250146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2500b01f0 0x7fa2500a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 --2- 172.21.15.33:0/59578119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2500cc7e0 0x7fa250159150 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 >> 172.21.15.33:0/59578119 conn(0x7fa25006d030 msgr2=0x7fa2501553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 shutdown_connections 2023-10-06T05:32:01.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.079+0000 7fa25579d700 1 -- 172.21.15.33:0/59578119 wait complete. 2023-10-06T05:32:01.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:32:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.287+0000 7f4531b9d700 1 Processor -- start 2023-10-06T05:32:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.287+0000 7f4531b9d700 1 -- start start 2023-10-06T05:32:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.288+0000 7f4531b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 0x7f452c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.288+0000 7f4531b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c0ce890 con 0x7f452c0af540 2023-10-06T05:32:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.288+0000 7f4531b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c15aad0 con 0x7f452c15a650 2023-10-06T05:32:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.288+0000 7f4531b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15d890 con 0x7f452c0cd5e0 2023-10-06T05:32:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f4530b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f452c0af540 0x7f452c0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37272/0 (socket says 172.21.15.33:37272) 2023-10-06T05:32:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f4530b9b700 1 -- 172.21.15.33:0/3088426728 learned_addr learned my addr 172.21.15.33:0/3088426728 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452bfff700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 0x7f452c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940760181 0 0) 0x7f452c0ce890 con 0x7f452c0af540 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f451c003570 con 0x7f452c0af540 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685042423 0 0) 0x7f452c15aad0 con 0x7f452c15a650 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f452c0ce890 con 0x7f452c15a650 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2788656493 0 0) 0x7f451c003570 con 0x7f452c0af540 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 msgr2=0x7f452c0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 0x7f452c0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 >> v1:172.21.15.167:6789/0 conn(0x7f452c15a650 legacy=0x7f452c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.289+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c15db70 con 0x7f452c0af540 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f4520001470 con 0x7f452c0af540 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 >> v1:172.21.15.33:6789/0 conn(0x7f452c0af540 legacy=0x7f452c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f451c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 0x7f451c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451c004d50 0x7f451c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15d890 con 0x7f451c004d50 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c00bb40 con 0x7f451c003c20 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c00bd00 con 0x7f452c0cd5e0 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f4530b9b700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f451c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f453139c700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451c004d50 0x7f451c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f452bfff700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 0x7f451c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f4530b9b700 1 -- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 msgr2=0x7f451c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f4530b9b700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0cd5e0 0x7f451c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f4530b9b700 1 -- 172.21.15.33:0/3088426728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451c004d50 msgr2=0x7f451c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f4530b9b700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451c004d50 0x7f451c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f4530b9b700 1 -- 172.21.15.33:0/3088426728 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c003570 con 0x7f451c003c20 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.290+0000 7f453139c700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451c004d50 0x7f451c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f4530b9b700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f451c004010 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f452000c2d0 tx=0x7f452c0ce800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb67b7a63ef8170d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000cc00 con 0x7f451c003c20 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000cd60 con 0x7f451c003c20 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f452b7fe700 1 -- 172.21.15.33:0/3088426728 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520005b70 con 0x7f451c003c20 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f4531b9d700 1 -- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 msgr2=0x7f451c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f4531b9d700 1 --2- 172.21.15.33:0/3088426728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f451c004010 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f452000c2d0 tx=0x7f452c0ce800 comp rx=0 tx=0).stop 2023-10-06T05:32:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.291+0000 7f453139c700 1 -- 172.21.15.33:0/3088426728 reap_dead start 2023-10-06T05:32:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.292+0000 7f4531b9d700 1 -- 172.21.15.33:0/3088426728 shutdown_connections 2023-10-06T05:32:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.292+0000 7f4531b9d700 1 -- 172.21.15.33:0/3088426728 >> 172.21.15.33:0/3088426728 conn(0x7f452c06d9b0 msgr2=0x7f452c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.292+0000 7f4531b9d700 1 -- 172.21.15.33:0/3088426728 shutdown_connections 2023-10-06T05:32:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.292+0000 7f4531b9d700 1 -- 172.21.15.33:0/3088426728 wait complete. 2023-10-06T05:32:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.293+0000 7f4531b9d700 1 Processor -- start 2023-10-06T05:32:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.293+0000 7f4531b9d700 1 -- start start 2023-10-06T05:32:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f4531b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f452c0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f4531b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0ae530 0x7f452c0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f4531b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 0x7f452c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f4531b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0c8570 con 0x7f452c0ae9a0 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f4531b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c157b60 con 0x7f451c003c20 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f4531b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0c8290 con 0x7f452c0ae530 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f453139c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 0x7f452c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f453139c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 0x7f452c0b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46634/0 (socket says 172.21.15.33:46634) 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.294+0000 7f453139c700 1 -- 172.21.15.33:0/805266863 learned_addr learned my addr 172.21.15.33:0/805266863 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f4530b9b700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f452c0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f452bfff700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0ae530 0x7f452c0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f453139c700 1 -- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0ae530 msgr2=0x7f452c0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f453139c700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0ae530 0x7f452c0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f453139c700 1 -- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 msgr2=0x7f452c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f453139c700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f452c0b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f453139c700 1 -- 172.21.15.33:0/805266863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0b3c20 con 0x7f452c0ae9a0 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f4530b9b700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f452c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f452bfff700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0ae530 0x7f452c0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f453139c700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 0x7f452c0b1a80 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f452c0caa90 tx=0x7f45240017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524019070 con 0x7f452c0ae9a0 2023-10-06T05:32:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524006ba0 con 0x7f452c0ae9a0 2023-10-06T05:32:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c0b3f00 con 0x7f452c0ae9a0 2023-10-06T05:32:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.295+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400cb90 con 0x7f452c0ae9a0 2023-10-06T05:32:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.296+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0b4c40 con 0x7f452c0ae9a0 2023-10-06T05:32:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.296+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0b3a80 con 0x7f452c0ae9a0 2023-10-06T05:32:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.297+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4524015430 con 0x7f452c0ae9a0 2023-10-06T05:32:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.299+0000 7f4528ff9700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4510066f30 0x7f45100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.299+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45240846c0 con 0x7f452c0ae9a0 2023-10-06T05:32:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.299+0000 7f4530b9b700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4510066f30 0x7f45100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.300+0000 7f4530b9b700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4510066f30 0x7f45100693e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4520001ed0 tx=0x7f4520015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.303+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f452404e5a0 con 0x7f452c0ae9a0 2023-10-06T05:32:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.503+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f452c0b4420 con 0x7f452c0ae9a0 2023-10-06T05:32:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.503+0000 7f4528ff9700 1 -- 172.21.15.33:0/805266863 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 104+0+11 (secure 0 0 0) 0x7f452c0b4420 con 0x7f452c0ae9a0 2023-10-06T05:32:01.505 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:32:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.505+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4510066f30 msgr2=0x7f45100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.505+0000 7f4531b9d700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4510066f30 0x7f45100693e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4520001ed0 tx=0x7f4520015000 comp rx=0 tx=0).stop 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 msgr2=0x7f452c0b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 0x7f452c0b1a80 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f452c0caa90 tx=0x7f45240017a0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 shutdown_connections 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4510066f30 0x7f45100693e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f452c0ae9a0 0x7f452c0b1a80 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f451c003c20 0x7f452c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 --2- 172.21.15.33:0/805266863 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f452c0ae530 0x7f452c0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 >> 172.21.15.33:0/805266863 conn(0x7f452c06d9b0 msgr2=0x7f452c15bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 shutdown_connections 2023-10-06T05:32:01.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.506+0000 7f4531b9d700 1 -- 172.21.15.33:0/805266863 wait complete. 2023-10-06T05:32:01.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:32:01.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.719+0000 7fea47fff700 1 Processor -- start 2023-10-06T05:32:01.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.720+0000 7fea47fff700 1 -- start start 2023-10-06T05:32:01.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea47fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea48155a50 0x7fea48155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea47fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea47fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480b06b0 con 0x7fea480c9490 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea47fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea480c99e0 con 0x7fea48155a50 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea47fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480c9cc0 con 0x7fea480afb20 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea46ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea48155a50 0x7fea48155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea46ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fea48155a50 0x7fea48155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46646/0 (socket says 172.21.15.33:46646) 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea46ffd700 1 -- 172.21.15.33:0/1491195163 learned_addr learned my addr 172.21.15.33:0/1491195163 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea467fc700 1 --1- 172.21.15.33:0/1491195163 >> v1:172.21.15.33:6789/0 conn(0x7fea480c9490 0x7fea480af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37282/0 (socket says 172.21.15.33:37282) 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.721+0000 7fea477fe700 1 --2- 172.21.15.33:0/1491195163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea46ffd700 1 -- 172.21.15.33:0/1491195163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 msgr2=0x7fea480caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea46ffd700 1 --2- 172.21.15.33:0/1491195163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea480caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea46ffd700 1 -- 172.21.15.33:0/1491195163 >> v1:172.21.15.33:6789/0 conn(0x7fea480c9490 legacy=0x7fea480af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea46ffd700 1 -- 172.21.15.33:0/1491195163 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480c9fa0 con 0x7fea48155a50 2023-10-06T05:32:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea45ffb700 1 -- 172.21.15.33:0/1491195163 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365448295 0 0) 0x7fea480b06b0 con 0x7fea480c9490 2023-10-06T05:32:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea46ffd700 1 --2- 172.21.15.33:0/1491195163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48155a50 0x7fea48155e20 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fea380099b0 tx=0x7fea38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8a07368b984f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea45ffb700 1 -- 172.21.15.33:0/1491195163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3800b860 con 0x7fea48155a50 2023-10-06T05:32:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea45ffb700 1 -- 172.21.15.33:0/1491195163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3800b9c0 con 0x7fea48155a50 2023-10-06T05:32:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.722+0000 7fea45ffb700 1 -- 172.21.15.33:0/1491195163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3800bd60 con 0x7fea48155a50 2023-10-06T05:32:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 -- 172.21.15.33:0/1491195163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48155a50 msgr2=0x7fea48155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 --2- 172.21.15.33:0/1491195163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48155a50 0x7fea48155e20 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fea380099b0 tx=0x7fea38009250 comp rx=0 tx=0).stop 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 -- 172.21.15.33:0/1491195163 shutdown_connections 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 --2- 172.21.15.33:0/1491195163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48155a50 0x7fea48155e20 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 --2- 172.21.15.33:0/1491195163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 -- 172.21.15.33:0/1491195163 >> 172.21.15.33:0/1491195163 conn(0x7fea4806d030 msgr2=0x7fea480cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 -- 172.21.15.33:0/1491195163 shutdown_connections 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.723+0000 7fea47fff700 1 -- 172.21.15.33:0/1491195163 wait complete. 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.724+0000 7fea47fff700 1 Processor -- start 2023-10-06T05:32:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 -- start start 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea4814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 0x7fea480cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48155a50 0x7fea480ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4816a240 con 0x7fea480c9490 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48169650 con 0x7fea48155a50 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.725+0000 7fea47fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea48169910 con 0x7fea480afb20 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 0x7fea480cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 0x7fea480cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46658/0 (socket says 172.21.15.33:46658) 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 -- 172.21.15.33:0/3897512678 learned_addr learned my addr 172.21.15.33:0/3897512678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea46ffd700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea4814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea477fe700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48155a50 0x7fea480ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 -- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 msgr2=0x7fea4814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea4814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 -- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48155a50 msgr2=0x7fea480ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48155a50 0x7fea480ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 -- 172.21.15.33:0/3897512678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380094f0 con 0x7fea480c9490 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.726+0000 7fea467fc700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 0x7fea480cc5e0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0011e0 tx=0x7fea3c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.727+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005a70 con 0x7fea480c9490 2023-10-06T05:32:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.727+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c005bd0 con 0x7fea480c9490 2023-10-06T05:32:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.727+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0070d0 con 0x7fea480c9490 2023-10-06T05:32:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.727+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea48169bf0 con 0x7fea480c9490 2023-10-06T05:32:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.727+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea480c6b40 con 0x7fea480c9490 2023-10-06T05:32:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.728+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4816a090 con 0x7fea480c9490 2023-10-06T05:32:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.729+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea3c007480 con 0x7fea480c9490 2023-10-06T05:32:01.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.730+0000 7fea2effd700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea34066f30 0x7fea340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:01.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.731+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fea3c083ae0 con 0x7fea480c9490 2023-10-06T05:32:01.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.731+0000 7fea46ffd700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea34066f30 0x7fea340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:01.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.732+0000 7fea46ffd700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea34066f30 0x7fea340693e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fea380099b0 tx=0x7fea38017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:01.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.735+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea3c04b5a0 con 0x7fea480c9490 2023-10-06T05:32:01.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.936+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea4804c380 con 0x7fea480c9490 2023-10-06T05:32:02.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.937+0000 7fea2effd700 1 -- 172.21.15.33:0/3897512678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v239) v1 ==== 100+0+11 (secure 0 0 0) 0x7fea3c017070 con 0x7fea480c9490 2023-10-06T05:32:02.013 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea34066f30 msgr2=0x7fea340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea34066f30 0x7fea340693e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fea380099b0 tx=0x7fea38017000 comp rx=0 tx=0).stop 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 msgr2=0x7fea480cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 0x7fea480cc5e0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0011e0 tx=0x7fea3c007e10 comp rx=0 tx=0).stop 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 shutdown_connections 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fea34066f30 0x7fea340693e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea48155a50 0x7fea480ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea480afb20 0x7fea4814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 --2- 172.21.15.33:0/3897512678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480c9490 0x7fea480cc5e0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 >> 172.21.15.33:0/3897512678 conn(0x7fea4806d030 msgr2=0x7fea48069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 shutdown_connections 2023-10-06T05:32:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:01.939+0000 7fea47fff700 1 -- 172.21.15.33:0/3897512678 wait complete. 2023-10-06T05:32:02.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.202+0000 7f31f5d9d700 1 Processor -- start 2023-10-06T05:32:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.202+0000 7f31f5d9d700 1 -- start start 2023-10-06T05:32:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.203+0000 7f31f5d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31f00ccfc0 0x7f31f00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f5d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31f00cc750 0x7f31f00b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f5d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f00cbd80 con 0x7f31f00cf3b0 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f5d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f00af5f0 con 0x7f31f00cc750 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f5d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f00af8d0 con 0x7f31f00ccfc0 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31f00cc750 0x7f31f00b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31f00cc750 0x7f31f00b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46662/0 (socket says 172.21.15.33:46662) 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 -- 172.21.15.33:0/1296529380 learned_addr learned my addr 172.21.15.33:0/1296529380 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f4d9b700 1 --2- 172.21.15.33:0/1296529380 >> v2:172.21.15.167:3300/0 conn(0x7f31f00ccfc0 0x7f31f00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 -- 172.21.15.33:0/1296529380 >> v2:172.21.15.167:3300/0 conn(0x7f31f00ccfc0 msgr2=0x7f31f00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 --2- 172.21.15.33:0/1296529380 >> v2:172.21.15.167:3300/0 conn(0x7f31f00ccfc0 0x7f31f00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 -- 172.21.15.33:0/1296529380 >> v1:172.21.15.167:6789/0 conn(0x7f31f00cf3b0 legacy=0x7f31f01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31f559c700 1 -- 172.21.15.33:0/1296529380 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00afbb0 con 0x7f31f00cc750 2023-10-06T05:32:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.204+0000 7f31ef7fe700 1 -- 172.21.15.33:0/1296529380 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437521061 0 0) 0x7f31f00cbd80 con 0x7f31f00cf3b0 2023-10-06T05:32:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.205+0000 7f31f559c700 1 --2- 172.21.15.33:0/1296529380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b0460 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009f20 tx=0x7f31e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a9ccee54f3ab99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.205+0000 7f31ef7fe700 1 -- 172.21.15.33:0/1296529380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800e070 con 0x7f31f00cc750 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.205+0000 7f31ef7fe700 1 -- 172.21.15.33:0/1296529380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e800b990 con 0x7f31f00cc750 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.205+0000 7f31ef7fe700 1 -- 172.21.15.33:0/1296529380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80092e0 con 0x7f31f00cc750 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.205+0000 7f31f5d9d700 1 -- 172.21.15.33:0/1296529380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 msgr2=0x7f31f00b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.205+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/1296529380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b0460 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009f20 tx=0x7f31e8009840 comp rx=0 tx=0).stop 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.206+0000 7f31f5d9d700 1 -- 172.21.15.33:0/1296529380 shutdown_connections 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.206+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/1296529380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b0460 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.206+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/1296529380 >> v2:172.21.15.167:3300/0 conn(0x7f31f00ccfc0 0x7f31f00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.206+0000 7f31f5d9d700 1 -- 172.21.15.33:0/1296529380 >> 172.21.15.33:0/1296529380 conn(0x7f31f006cff0 msgr2=0x7f31f006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.206+0000 7f31f5d9d700 1 -- 172.21.15.33:0/1296529380 shutdown_connections 2023-10-06T05:32:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.206+0000 7f31f5d9d700 1 -- 172.21.15.33:0/1296529380 wait complete. 2023-10-06T05:32:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.207+0000 7f31f5d9d700 1 Processor -- start 2023-10-06T05:32:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.207+0000 7f31f5d9d700 1 -- start start 2023-10-06T05:32:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f5d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f5d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00ccfc0 0x7f31f00c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f5d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f0154270 0x7f31f016a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f5d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00c97d0 con 0x7f31f00cc750 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f5d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00b32a0 con 0x7f31f00ccfc0 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f5d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0154710 con 0x7f31f0154270 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f4d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f4d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46672/0 (socket says 172.21.15.33:46672) 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.208+0000 7f31f4d9b700 1 -- 172.21.15.33:0/808731039 learned_addr learned my addr 172.21.15.33:0/808731039 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31effff700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00ccfc0 0x7f31f00c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f559c700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f0154270 0x7f31f016a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f4d9b700 1 -- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f0154270 msgr2=0x7f31f016a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f4d9b700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f0154270 0x7f31f016a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f4d9b700 1 -- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00ccfc0 msgr2=0x7f31f00c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f4d9b700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00ccfc0 0x7f31f00c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f4d9b700 1 -- 172.21.15.33:0/808731039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8009ae0 con 0x7f31f00cc750 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f4d9b700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b4ae0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f31e400ec80 tx=0x7f31e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400c9d0 con 0x7f31f00cc750 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400cb30 con 0x7f31f00cc750 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4013470 con 0x7f31f00cc750 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.209+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00b51a0 con 0x7f31f00cc750 2023-10-06T05:32:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.210+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f00b34f0 con 0x7f31f00cc750 2023-10-06T05:32:02.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.211+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f00b56c0 con 0x7f31f00cc750 2023-10-06T05:32:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.211+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31e4013820 con 0x7f31f00cc750 2023-10-06T05:32:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.212+0000 7f31ecff9700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.212+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31e4084a80 con 0x7f31f00cc750 2023-10-06T05:32:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.213+0000 7f31effff700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.213+0000 7f31effff700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f31f00b64c0 tx=0x7f31dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.216+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31e4051b90 con 0x7f31f00cc750 2023-10-06T05:32:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.463+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31f004c380 con 0x7f31f00cc750 2023-10-06T05:32:02.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.464+0000 7f31ecff9700 1 -- 172.21.15.33:0/808731039 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f31e40551b0 con 0x7f31f00cc750 2023-10-06T05:32:02.466 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:02.466 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:31:48.429792+0000","modified":"2023-10-06T05:31:51.962941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11169},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11169":{"gid":11169,"name":"a","rank":0,"incarnation":181,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1245472058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1245472058},{"type":"v1","addr":"172.21.15.33:6835","nonce":1245472058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2023-10-06T05:32:02.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.466+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31d4066f30 msgr2=0x7f31d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.466+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f31f00b64c0 tx=0x7f31dc009040 comp rx=0 tx=0).stop 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.466+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 msgr2=0x7f31f00b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.466+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b4ae0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f31e400ec80 tx=0x7f31e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 shutdown_connections 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31f0154270 0x7f31f016a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31f00cc750 0x7f31f00b4ae0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 --2- 172.21.15.33:0/808731039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f00ccfc0 0x7f31f00c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 >> 172.21.15.33:0/808731039 conn(0x7f31f006cff0 msgr2=0x7f31f00cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 shutdown_connections 2023-10-06T05:32:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.467+0000 7f31f5d9d700 1 -- 172.21.15.33:0/808731039 wait complete. 2023-10-06T05:32:02.470 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 185 2023-10-06T05:32:02.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:32:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.671+0000 7f4b4659d700 1 Processor -- start 2023-10-06T05:32:02.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.671+0000 7f4b4659d700 1 -- start start 2023-10-06T05:32:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b4659d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b4006ab00 0x7f4b400b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b4659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4015e2d0 con 0x7f4b400b2e30 2023-10-06T05:32:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b4659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4006afa0 con 0x7f4b400b0a40 2023-10-06T05:32:02.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b4659d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4015d6c0 con 0x7f4b4006ab00 2023-10-06T05:32:02.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b44d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b4006ab00 0x7f4b400b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b44d9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b4006ab00 0x7f4b400b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46674/0 (socket says 172.21.15.33:46674) 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3095143548 learned_addr learned my addr 172.21.15.33:0/3095143548 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3095143548 >> v1:172.21.15.167:6789/0 conn(0x7f4b400b0a40 legacy=0x7f4b400b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3095143548 >> v1:172.21.15.33:6789/0 conn(0x7f4b400b2e30 legacy=0x7f4b4015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3095143548 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4015d9a0 con 0x7f4b4006ab00 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.673+0000 7f4b37fff700 1 -- 172.21.15.33:0/3095143548 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062902307 0 0) 0x7f4b4015e2d0 con 0x7f4b400b2e30 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.674+0000 7f4b44d9a700 1 --2- 172.21.15.33:0/3095143548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b4006ab00 0x7f4b400b28f0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f4b300099b0 tx=0x7f4b30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8a6d38f3ea2cab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.674+0000 7f4b37fff700 1 -- 172.21.15.33:0/3095143548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000b860 con 0x7f4b4006ab00 2023-10-06T05:32:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.674+0000 7f4b37fff700 1 -- 172.21.15.33:0/3095143548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3000b9c0 con 0x7f4b4006ab00 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.674+0000 7f4b37fff700 1 -- 172.21.15.33:0/3095143548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000bd80 con 0x7f4b4006ab00 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.674+0000 7f4b4659d700 1 -- 172.21.15.33:0/3095143548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b4006ab00 msgr2=0x7f4b400b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.674+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3095143548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b4006ab00 0x7f4b400b28f0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f4b300099b0 tx=0x7f4b30009250 comp rx=0 tx=0).stop 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.675+0000 7f4b4659d700 1 -- 172.21.15.33:0/3095143548 shutdown_connections 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.675+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3095143548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b4006ab00 0x7f4b400b28f0 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.675+0000 7f4b4659d700 1 -- 172.21.15.33:0/3095143548 >> 172.21.15.33:0/3095143548 conn(0x7f4b4006e0f0 msgr2=0x7f4b400675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.675+0000 7f4b4659d700 1 -- 172.21.15.33:0/3095143548 shutdown_connections 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.675+0000 7f4b4659d700 1 -- 172.21.15.33:0/3095143548 wait complete. 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.676+0000 7f4b4659d700 1 Processor -- start 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.676+0000 7f4b4659d700 1 -- start start 2023-10-06T05:32:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b4006ab00 0x7f4b4015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 0x7f4b401488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b400b2e30 0x7f4b40148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4016ae00 con 0x7f4b400b0a40 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4016ba90 con 0x7f4b400b2e30 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4016b160 con 0x7f4b4006ab00 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b44d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 0x7f4b401488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b44d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 0x7f4b401488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46686/0 (socket says 172.21.15.33:46686) 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3301071144 learned_addr learned my addr 172.21.15.33:0/3301071144 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:02.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.677+0000 7f4b4559b700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b4006ab00 0x7f4b4015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b45d9c700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b400b2e30 0x7f4b40148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b4006ab00 msgr2=0x7f4b4015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b44d9a700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b4006ab00 0x7f4b4015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b400b2e30 msgr2=0x7f4b40148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b44d9a700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b400b2e30 0x7f4b40148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b44d9a700 1 -- 172.21.15.33:0/3301071144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300094f0 con 0x7f4b400b0a40 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b44d9a700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 0x7f4b401488e0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f4b4006b130 tx=0x7f4b30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001f070 con 0x7f4b400b0a40 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b30011c40 con 0x7f4b400b0a40 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4016b440 con 0x7f4b400b0a40 2023-10-06T05:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30002f70 con 0x7f4b400b0a40 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.678+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4015c880 con 0x7f4b400b0a40 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.679+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4006fcc0 con 0x7f4b400b0a40 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.680+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b30023070 con 0x7f4b400b0a40 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.682+0000 7f4b357fa700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b24066f30 0x7f4b240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.682+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4b300858a0 con 0x7f4b400b0a40 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.682+0000 7f4b4559b700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b24066f30 0x7f4b240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.683+0000 7f4b4559b700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b24066f30 0x7f4b240693e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f4b4015ce30 tx=0x7f4b3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:02.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.686+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b30052910 con 0x7f4b400b0a40 2023-10-06T05:32:02.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.889+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b4016bc20 con 0x7f4b400b0a40 2023-10-06T05:32:02.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.890+0000 7f4b357fa700 1 -- 172.21.15.33:0/3301071144 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f4b30015090 con 0x7f4b400b0a40 2023-10-06T05:32:02.892 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:02.892 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":239,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:49.887625+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:31:46.602215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-06T05:31:47.403110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b24066f30 msgr2=0x7f4b240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b24066f30 0x7f4b240693e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f4b4015ce30 tx=0x7f4b3c009040 comp rx=0 tx=0).stop 2023-10-06T05:32:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 msgr2=0x7f4b401488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 0x7f4b401488e0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f4b4006b130 tx=0x7f4b30017000 comp rx=0 tx=0).stop 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 shutdown_connections 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b24066f30 0x7f4b240693e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b400b2e30 0x7f4b40148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b4006ab00 0x7f4b4015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 --2- 172.21.15.33:0/3301071144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b400b0a40 0x7f4b401488e0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 >> 172.21.15.33:0/3301071144 conn(0x7f4b4006e0f0 msgr2=0x7f4b40157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 shutdown_connections 2023-10-06T05:32:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:02.892+0000 7f4b4659d700 1 -- 172.21.15.33:0/3301071144 wait complete. 2023-10-06T05:32:02.911 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:32:02.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:03.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.100+0000 7fd7d579d700 1 Processor -- start 2023-10-06T05:32:03.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.101+0000 7fd7d579d700 1 -- start start 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7d579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d0159530 0x7fd7d006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7d579d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7d006df70 0x7fd7d00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7d579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d006e3e0 con 0x7fd7d00b0310 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7d579d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00b1e00 con 0x7fd7d006df70 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7d579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d006f6b0 con 0x7fd7d0159530 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7cf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7d006df70 0x7fd7d00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7cf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7d006df70 0x7fd7d00afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46692/0 (socket says 172.21.15.33:46692) 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.102+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/857537640 learned_addr learned my addr 172.21.15.33:0/857537640 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/857537640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d0159530 msgr2=0x7fd7d006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cffff700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d0159530 0x7fd7d006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cf7fe700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d0159530 0x7fd7d006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/857537640 >> v1:172.21.15.33:6789/0 conn(0x7fd7d00b0310 legacy=0x7fd7d0159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/857537640 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d006f220 con 0x7fd7d006df70 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cffff700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d0159530 0x7fd7d006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7ceffd700 1 -- 172.21.15.33:0/857537640 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874792795 0 0) 0x7fd7d006e3e0 con 0x7fd7d00b0310 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7cf7fe700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d006df70 0x7fd7d00afdd0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40099b0 tx=0x7fd7c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45f4442237b6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7ceffd700 1 -- 172.21.15.33:0/857537640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400b860 con 0x7fd7d006df70 2023-10-06T05:32:03.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7ceffd700 1 -- 172.21.15.33:0/857537640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400b9c0 con 0x7fd7d006df70 2023-10-06T05:32:03.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.103+0000 7fd7ceffd700 1 -- 172.21.15.33:0/857537640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400bd60 con 0x7fd7d006df70 2023-10-06T05:32:03.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 -- 172.21.15.33:0/857537640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d006df70 msgr2=0x7fd7d00afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d006df70 0x7fd7d00afdd0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40099b0 tx=0x7fd7c4009250 comp rx=0 tx=0).stop 2023-10-06T05:32:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 -- 172.21.15.33:0/857537640 shutdown_connections 2023-10-06T05:32:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d0159530 0x7fd7d006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 --2- 172.21.15.33:0/857537640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d006df70 0x7fd7d00afdd0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 -- 172.21.15.33:0/857537640 >> 172.21.15.33:0/857537640 conn(0x7fd7d006c560 msgr2=0x7fd7d006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:03.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 -- 172.21.15.33:0/857537640 shutdown_connections 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.104+0000 7fd7d579d700 1 -- 172.21.15.33:0/857537640 wait complete. 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.105+0000 7fd7d579d700 1 Processor -- start 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.105+0000 7fd7d579d700 1 -- start start 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.106+0000 7fd7d579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d006df70 0x7fd7d00cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.106+0000 7fd7d579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 0x7fd7d00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.106+0000 7fd7d579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7d0159530 0x7fd7d00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.106+0000 7fd7d579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00cd790 con 0x7fd7d00b0310 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.106+0000 7fd7d579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00ccb80 con 0x7fd7d0159530 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.106+0000 7fd7d579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00cce60 con 0x7fd7d006df70 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 0x7fd7d00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 0x7fd7d00cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46704/0 (socket says 172.21.15.33:46704) 2023-10-06T05:32:03.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/2695860288 learned_addr learned my addr 172.21.15.33:0/2695860288 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cffff700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d006df70 0x7fd7d00cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7d4f9c700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7d0159530 0x7fd7d00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d006df70 msgr2=0x7fd7d00cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d006df70 0x7fd7d00cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7d0159530 msgr2=0x7fd7d00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7d0159530 0x7fd7d00ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 -- 172.21.15.33:0/2695860288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40094f0 con 0x7fd7d00b0310 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7cf7fe700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 0x7fd7d00cdc20 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fd7d006b870 tx=0x7fd7c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.107+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c401a420 con 0x7fd7d00b0310 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.108+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c401a5a0 con 0x7fd7d00b0310 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.108+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c401a960 con 0x7fd7d00b0310 2023-10-06T05:32:03.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.108+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d00cd140 con 0x7fd7d00b0310 2023-10-06T05:32:03.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.108+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d00cab70 con 0x7fd7d00b0310 2023-10-06T05:32:03.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.109+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d00cd940 con 0x7fd7d00b0310 2023-10-06T05:32:03.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.110+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7c401ad10 con 0x7fd7d00b0310 2023-10-06T05:32:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.110+0000 7fd7b7fff700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd7bc066f30 0x7fd7bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.111+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd7c40860c0 con 0x7fd7d00b0310 2023-10-06T05:32:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.111+0000 7fd7cffff700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd7bc066f30 0x7fd7bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.112+0000 7fd7cffff700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd7bc066f30 0x7fd7bc0693e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000ac00 tx=0x7fd7c0006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:03.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.115+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7d00cd940 con 0x7fd7d00b0310 2023-10-06T05:32:03.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.360+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7d00cd940 con 0x7fd7d00b0310 2023-10-06T05:32:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.361+0000 7fd7b7fff700 1 -- 172.21.15.33:0/2695860288 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fd7d00cd940 con 0x7fd7d00b0310 2023-10-06T05:32:03.392 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:03.392 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:31:48.429792+0000","modified":"2023-10-06T05:31:51.962941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11169},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11169":{"gid":11169,"name":"a","rank":0,"incarnation":181,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1245472058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1245472058},{"type":"v1","addr":"172.21.15.33:6835","nonce":1245472058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2023-10-06T05:32:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd7bc066f30 msgr2=0x7fd7bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd7bc066f30 0x7fd7bc0693e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000ac00 tx=0x7fd7c0006f90 comp rx=0 tx=0).stop 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 msgr2=0x7fd7d00cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 0x7fd7d00cdc20 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fd7d006b870 tx=0x7fd7c4017000 comp rx=0 tx=0).stop 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 shutdown_connections 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd7bc066f30 0x7fd7bc0693e0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7d0159530 0x7fd7d00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7d006df70 0x7fd7d00cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 --2- 172.21.15.33:0/2695860288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7d00b0310 0x7fd7d00cdc20 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 >> 172.21.15.33:0/2695860288 conn(0x7fd7d006c560 msgr2=0x7fd7d01586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 shutdown_connections 2023-10-06T05:32:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.363+0000 7fd7d579d700 1 -- 172.21.15.33:0/2695860288 wait complete. 2023-10-06T05:32:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 185 2023-10-06T05:32:03.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.594+0000 7f1dd3fff700 1 Processor -- start 2023-10-06T05:32:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.594+0000 7f1dd3fff700 1 -- start start 2023-10-06T05:32:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.595+0000 7f1dd3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1dd40cb710 0x7f1dd406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.595+0000 7f1dd3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd40c9cb0 con 0x7f1dd40c8050 2023-10-06T05:32:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.595+0000 7f1dd3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd40ca270 con 0x7f1dd413faa0 2023-10-06T05:32:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.595+0000 7f1dd3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd406b840 con 0x7f1dd40cb710 2023-10-06T05:32:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd27fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1dd40c8050 0x7f1dd406aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37302/0 (socket says 172.21.15.33:37302) 2023-10-06T05:32:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd27fc700 1 -- 172.21.15.33:0/1063959262 learned_addr learned my addr 172.21.15.33:0/1063959262 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd37fe700 1 --2- 172.21.15.33:0/1063959262 >> v2:172.21.15.167:3300/0 conn(0x7f1dd40cb710 0x7f1dd406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67339129 0 0) 0x7f1dd40c9cb0 con 0x7f1dd40c8050 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dbc003570 con 0x7f1dd40c8050 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221678274 0 0) 0x7f1dd40ca270 con 0x7f1dd413faa0 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd40c9cb0 con 0x7f1dd413faa0 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd37fe700 1 -- 172.21.15.33:0/1063959262 >> v1:172.21.15.167:6789/0 conn(0x7f1dd413faa0 legacy=0x7f1dd40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd37fe700 1 -- 172.21.15.33:0/1063959262 >> v1:172.21.15.33:6789/0 conn(0x7f1dd40c8050 legacy=0x7f1dd406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.596+0000 7f1dd37fe700 1 -- 172.21.15.33:0/1063959262 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd406bb20 con 0x7f1dd40cb710 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.597+0000 7f1dd37fe700 1 --2- 172.21.15.33:0/1063959262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cb710 0x7f1dd406b1e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009e60 tx=0x7f1dcc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0158ecf4e8ce16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.597+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00bab0 con 0x7f1dd40cb710 2023-10-06T05:32:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.597+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc00bc10 con 0x7f1dd40cb710 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.597+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/1063959262 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc011460 con 0x7f1dd40cb710 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.597+0000 7f1dd3fff700 1 -- 172.21.15.33:0/1063959262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cb710 msgr2=0x7f1dd406b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.597+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/1063959262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cb710 0x7f1dd406b1e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009e60 tx=0x7f1dcc009780 comp rx=0 tx=0).stop 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.598+0000 7f1dd3fff700 1 -- 172.21.15.33:0/1063959262 shutdown_connections 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.598+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/1063959262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cb710 0x7f1dd406b1e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.598+0000 7f1dd3fff700 1 -- 172.21.15.33:0/1063959262 >> 172.21.15.33:0/1063959262 conn(0x7f1dd406d030 msgr2=0x7f1dd406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.598+0000 7f1dd3fff700 1 -- 172.21.15.33:0/1063959262 shutdown_connections 2023-10-06T05:32:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.598+0000 7f1dd3fff700 1 -- 172.21.15.33:0/1063959262 wait complete. 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.599+0000 7f1dd3fff700 1 Processor -- start 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.599+0000 7f1dd3fff700 1 -- start start 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd40c8050 0x7f1dd413cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 0x7f1dd4165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd413faa0 0x7f1dd40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4165a40 con 0x7f1dd40cb710 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c3c40 con 0x7f1dd413faa0 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c6d00 con 0x7f1dd40c8050 2023-10-06T05:32:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 0x7f1dd4165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 0x7f1dd4165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46718/0 (socket says 172.21.15.33:46718) 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.600+0000 7f1dd27fc700 1 -- 172.21.15.33:0/880138770 learned_addr learned my addr 172.21.15.33:0/880138770 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd2ffd700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd40c8050 0x7f1dd413cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd37fe700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd413faa0 0x7f1dd40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd27fc700 1 -- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd40c8050 msgr2=0x7f1dd413cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd27fc700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd40c8050 0x7f1dd413cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd27fc700 1 -- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd413faa0 msgr2=0x7f1dd40b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd27fc700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd413faa0 0x7f1dd40b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd27fc700 1 -- 172.21.15.33:0/880138770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009a20 con 0x7f1dd40cb710 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd37fe700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd413faa0 0x7f1dd40b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd27fc700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 0x7f1dd4165500 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800c790 tx=0x7f1dc8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8005b30 con 0x7f1dd40cb710 2023-10-06T05:32:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc8005c90 con 0x7f1dd40cb710 2023-10-06T05:32:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8007300 con 0x7f1dd40cb710 2023-10-06T05:32:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.601+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd40c3ee0 con 0x7f1dd40cb710 2023-10-06T05:32:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.602+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd40b1990 con 0x7f1dd40cb710 2023-10-06T05:32:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.603+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd404c380 con 0x7f1dd40cb710 2023-10-06T05:32:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.603+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dc801b400 con 0x7f1dd40cb710 2023-10-06T05:32:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.605+0000 7f1dbaffd700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1dc0066f30 0x7f1dc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.605+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1dc8084260 con 0x7f1dd40cb710 2023-10-06T05:32:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.605+0000 7f1dd2ffd700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1dc0066f30 0x7f1dc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.606+0000 7f1dd2ffd700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1dc0066f30 0x7f1dc00693e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40037e0 tx=0x7f1dc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.608+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dc8054e90 con 0x7f1dd40cb710 2023-10-06T05:32:03.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.855+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dd40c9f00 con 0x7f1dd40cb710 2023-10-06T05:32:03.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.855+0000 7f1dbaffd700 1 -- 172.21.15.33:0/880138770 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f1dc80076f0 con 0x7f1dd40cb710 2023-10-06T05:32:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:03.857 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":185,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:31:48.429792+0000","modified":"2023-10-06T05:31:51.962941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11169},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11169":{"gid":11169,"name":"a","rank":0,"incarnation":181,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/1245472058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":1245472058},{"type":"v1","addr":"172.21.15.33:6835","nonce":1245472058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":17}]} 2023-10-06T05:32:03.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.857+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1dc0066f30 msgr2=0x7f1dc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.857+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1dc0066f30 0x7f1dc00693e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40037e0 tx=0x7f1dc4009040 comp rx=0 tx=0).stop 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.857+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 msgr2=0x7f1dd4165500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.857+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 0x7f1dd4165500 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800c790 tx=0x7f1dc8007ea0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 shutdown_connections 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1dc0066f30 0x7f1dc00693e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd413faa0 0x7f1dd40b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd40c8050 0x7f1dd413cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/880138770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40cb710 0x7f1dd4165500 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 >> 172.21.15.33:0/880138770 conn(0x7f1dd406d030 msgr2=0x7f1dd40c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 shutdown_connections 2023-10-06T05:32:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:03.858+0000 7f1dd3fff700 1 -- 172.21.15.33:0/880138770 wait complete. 2023-10-06T05:32:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 185 2023-10-06T05:32:03.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:32:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.067+0000 7fdf34d9d700 1 Processor -- start 2023-10-06T05:32:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.067+0000 7fdf34d9d700 1 -- start start 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.068+0000 7fdf34d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300cf3b0 0x7fdf301594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.068+0000 7fdf34d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf300af0a0 0x7fdf301599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.068+0000 7fdf34d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf300cbd80 con 0x7fdf300ccfc0 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.068+0000 7fdf34d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300af5f0 con 0x7fdf300af0a0 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.068+0000 7fdf34d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300af8d0 con 0x7fdf300cf3b0 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf300af0a0 0x7fdf301599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdf300af0a0 0x7fdf301599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46722/0 (socket says 172.21.15.33:46722) 2023-10-06T05:32:04.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 -- 172.21.15.33:0/3205741376 learned_addr learned my addr 172.21.15.33:0/3205741376 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2effd700 1 --2- 172.21.15.33:0/3205741376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300cf3b0 0x7fdf301594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 -- 172.21.15.33:0/3205741376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300cf3b0 msgr2=0x7fdf301594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 --2- 172.21.15.33:0/3205741376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300cf3b0 0x7fdf301594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 -- 172.21.15.33:0/3205741376 >> v1:172.21.15.167:6789/0 conn(0x7fdf300ccfc0 legacy=0x7fdf300cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 -- 172.21.15.33:0/3205741376 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300afbb0 con 0x7fdf300af0a0 2023-10-06T05:32:04.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.069+0000 7fdf2ffff700 1 --2- 172.21.15.33:0/3205741376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300af0a0 0x7fdf301599e0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009f20 tx=0x7fdf28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7baa28cdd59e87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf2e7fc700 1 -- 172.21.15.33:0/3205741376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800e070 con 0x7fdf300af0a0 2023-10-06T05:32:04.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf2e7fc700 1 -- 172.21.15.33:0/3205741376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2800b9d0 con 0x7fdf300af0a0 2023-10-06T05:32:04.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf2e7fc700 1 -- 172.21.15.33:0/3205741376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280092e0 con 0x7fdf300af0a0 2023-10-06T05:32:04.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 -- 172.21.15.33:0/3205741376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300af0a0 msgr2=0x7fdf301599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/3205741376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300af0a0 0x7fdf301599e0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009f20 tx=0x7fdf28009840 comp rx=0 tx=0).stop 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 -- 172.21.15.33:0/3205741376 shutdown_connections 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/3205741376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300af0a0 0x7fdf301599e0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/3205741376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300cf3b0 0x7fdf301594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 -- 172.21.15.33:0/3205741376 >> 172.21.15.33:0/3205741376 conn(0x7fdf3006cff0 msgr2=0x7fdf3006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 -- 172.21.15.33:0/3205741376 shutdown_connections 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.070+0000 7fdf34d9d700 1 -- 172.21.15.33:0/3205741376 wait complete. 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.072+0000 7fdf34d9d700 1 Processor -- start 2023-10-06T05:32:04.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.072+0000 7fdf34d9d700 1 -- start start 2023-10-06T05:32:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf34d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf300af0a0 0x7fdf30154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf34d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 0x7fdf300c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf34d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300b3c50 0x7fdf300ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf34d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300b50d0 con 0x7fdf300ccfc0 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf34d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300b44c0 con 0x7fdf300af0a0 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf34d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300b47a0 con 0x7fdf300b3c50 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 0x7fdf300c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 0x7fdf300c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46734/0 (socket says 172.21.15.33:46734) 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf2effd700 1 -- 172.21.15.33:0/4288936959 learned_addr learned my addr 172.21.15.33:0/4288936959 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf2f7fe700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf300af0a0 0x7fdf30154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.073+0000 7fdf2ffff700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300b3c50 0x7fdf300ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf2effd700 1 -- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300b3c50 msgr2=0x7fdf300ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf2effd700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300b3c50 0x7fdf300ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf2effd700 1 -- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf300af0a0 msgr2=0x7fdf30154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf2effd700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf300af0a0 0x7fdf30154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf2effd700 1 -- 172.21.15.33:0/4288936959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009ae0 con 0x7fdf300ccfc0 2023-10-06T05:32:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf2effd700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 0x7fdf300c93d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400ec50 tx=0x7fdf240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400c850 con 0x7fdf300ccfc0 2023-10-06T05:32:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400c9b0 con 0x7fdf300ccfc0 2023-10-06T05:32:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400cd70 con 0x7fdf300ccfc0 2023-10-06T05:32:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf300b4a80 con 0x7fdf300ccfc0 2023-10-06T05:32:04.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.074+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf300b40d0 con 0x7fdf300ccfc0 2023-10-06T05:32:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.075+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf300b4f20 con 0x7fdf300ccfc0 2023-10-06T05:32:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.076+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf24013680 con 0x7fdf300ccfc0 2023-10-06T05:32:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.077+0000 7fdf177fe700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf1c066f30 0x7fdf1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.077+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdf24084970 con 0x7fdf300ccfc0 2023-10-06T05:32:04.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.077+0000 7fdf2f7fe700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf1c066f30 0x7fdf1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.078+0000 7fdf2f7fe700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf1c066f30 0x7fdf1c0693e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdf200039e0 tx=0x7fdf20007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.082+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf240519e0 con 0x7fdf300ccfc0 2023-10-06T05:32:04.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.286+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf30169860 con 0x7fdf300ccfc0 2023-10-06T05:32:04.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.287+0000 7fdf177fe700 1 -- 172.21.15.33:0/4288936959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fdf24055000 con 0x7fdf300ccfc0 2023-10-06T05:32:04.289 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:04.290 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":239,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:31:49.887625+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:31:46.602215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-06T05:31:47.403110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.289+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf1c066f30 msgr2=0x7fdf1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.289+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf1c066f30 0x7fdf1c0693e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdf200039e0 tx=0x7fdf20007ec0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.289+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 msgr2=0x7fdf300c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.289+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 0x7fdf300c93d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400ec50 tx=0x7fdf240091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 shutdown_connections 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdf1c066f30 0x7fdf1c0693e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdf300b3c50 0x7fdf300ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf300af0a0 0x7fdf30154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 --2- 172.21.15.33:0/4288936959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf300ccfc0 0x7fdf300c93d0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 >> 172.21.15.33:0/4288936959 conn(0x7fdf3006cff0 msgr2=0x7fdf30155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:04.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 shutdown_connections 2023-10-06T05:32:04.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.290+0000 7fdf34d9d700 1 -- 172.21.15.33:0/4288936959 wait complete. 2023-10-06T05:32:04.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:32:04.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.496+0000 7f82eb59e700 1 Processor -- start 2023-10-06T05:32:04.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.496+0000 7f82eb59e700 1 -- start start 2023-10-06T05:32:04.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82eb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82ec140d20 0x7f82ec0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82eb59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82ec0c9210 0x7f82ec0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82eb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ec15bcb0 con 0x7f82ec069f00 2023-10-06T05:32:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82eb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ec0c9690 con 0x7f82ec140d20 2023-10-06T05:32:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82eb59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ec1410f0 con 0x7f82ec0c9210 2023-10-06T05:32:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82ec140d20 0x7f82ec0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f82ec140d20 0x7f82ec0c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46750/0 (socket says 172.21.15.33:46750) 2023-10-06T05:32:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 -- 172.21.15.33:0/3235619849 learned_addr learned my addr 172.21.15.33:0/3235619849 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82e9d9b700 1 --2- 172.21.15.33:0/3235619849 >> v2:172.21.15.167:3300/0 conn(0x7f82ec0c9210 0x7f82ec0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 -- 172.21.15.33:0/3235619849 >> v2:172.21.15.167:3300/0 conn(0x7f82ec0c9210 msgr2=0x7f82ec0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 --2- 172.21.15.33:0/3235619849 >> v2:172.21.15.167:3300/0 conn(0x7f82ec0c9210 0x7f82ec0699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 -- 172.21.15.33:0/3235619849 >> v1:172.21.15.167:6789/0 conn(0x7f82ec069f00 legacy=0x7f82ec0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82ea59c700 1 -- 172.21.15.33:0/3235619849 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec06a2f0 con 0x7f82ec140d20 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.498+0000 7f82e9d9b700 1 --2- 172.21.15.33:0/3235619849 >> v2:172.21.15.167:3300/0 conn(0x7f82ec0c9210 0x7f82ec0699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.499+0000 7f82ea59c700 1 --2- 172.21.15.33:0/3235619849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec140d20 0x7f82ec0c8cd0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0c66e0 tx=0x7f82dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c50103527727bd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.499+0000 7f82e959a700 1 -- 172.21.15.33:0/3235619849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00b820 con 0x7f82ec140d20 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.499+0000 7f82e959a700 1 -- 172.21.15.33:0/3235619849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00b980 con 0x7f82ec140d20 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.499+0000 7f82e959a700 1 -- 172.21.15.33:0/3235619849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00bd20 con 0x7f82ec140d20 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.499+0000 7f82eb59e700 1 -- 172.21.15.33:0/3235619849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec140d20 msgr2=0x7f82ec0c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.499+0000 7f82eb59e700 1 --2- 172.21.15.33:0/3235619849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec140d20 0x7f82ec0c8cd0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0c66e0 tx=0x7f82dc009250 comp rx=0 tx=0).stop 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.500+0000 7f82eb59e700 1 -- 172.21.15.33:0/3235619849 shutdown_connections 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.500+0000 7f82eb59e700 1 --2- 172.21.15.33:0/3235619849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec140d20 0x7f82ec0c8cd0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.500+0000 7f82eb59e700 1 --2- 172.21.15.33:0/3235619849 >> v2:172.21.15.167:3300/0 conn(0x7f82ec0c9210 0x7f82ec0699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.500+0000 7f82eb59e700 1 -- 172.21.15.33:0/3235619849 >> 172.21.15.33:0/3235619849 conn(0x7f82ec06c560 msgr2=0x7f82ec06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.500+0000 7f82eb59e700 1 -- 172.21.15.33:0/3235619849 shutdown_connections 2023-10-06T05:32:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.500+0000 7f82eb59e700 1 -- 172.21.15.33:0/3235619849 wait complete. 2023-10-06T05:32:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.501+0000 7f82eb59e700 1 Processor -- start 2023-10-06T05:32:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.501+0000 7f82eb59e700 1 -- start start 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82eb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 0x7f82ec13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82eb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ec0c9210 0x7f82ec1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82eb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82ec140d20 0x7f82ec152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82eb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec158ad0 con 0x7f82ec069f00 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82eb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec157a30 con 0x7f82ec0c9210 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82eb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec157d10 con 0x7f82ec140d20 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 0x7f82ec13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 0x7f82ec13ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46760/0 (socket says 172.21.15.33:46760) 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.502+0000 7f82ea59c700 1 -- 172.21.15.33:0/4280104809 learned_addr learned my addr 172.21.15.33:0/4280104809 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ead9d700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82ec140d20 0x7f82ec152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82e9d9b700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ec0c9210 0x7f82ec1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ea59c700 1 -- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82ec140d20 msgr2=0x7f82ec152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ea59c700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82ec140d20 0x7f82ec152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ea59c700 1 -- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ec0c9210 msgr2=0x7f82ec1521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ea59c700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ec0c9210 0x7f82ec1521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ea59c700 1 -- 172.21.15.33:0/4280104809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0094f0 con 0x7f82ec069f00 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ead9d700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82ec140d20 0x7f82ec152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82ea59c700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 0x7f82ec13ed00 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0030c0 tx=0x7f82dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc011e60 con 0x7f82ec069f00 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc020070 con 0x7f82ec069f00 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec157ff0 con 0x7f82ec069f00 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.503+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc01b6a0 con 0x7f82ec069f00 2023-10-06T05:32:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.504+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec1585b0 con 0x7f82ec069f00 2023-10-06T05:32:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.504+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ec1587a0 con 0x7f82ec069f00 2023-10-06T05:32:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.505+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82dc022030 con 0x7f82ec069f00 2023-10-06T05:32:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.506+0000 7f82da7fc700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82cc066f30 0x7f82cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.506+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f82dc0866d0 con 0x7f82ec069f00 2023-10-06T05:32:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.507+0000 7f82e9d9b700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82cc066f30 0x7f82cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.507+0000 7f82e9d9b700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82cc066f30 0x7f82cc0693e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0aedc0 tx=0x7f82e0006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.510+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82dc053740 con 0x7f82ec069f00 2023-10-06T05:32:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.761+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f82ec1587a0 con 0x7f82ec069f00 2023-10-06T05:32:04.763 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:32:04.761+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:32:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.768+0000 7f82da7fc700 1 -- 172.21.15.33:0/4280104809 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v186) v1 ==== 161+0+0 (secure 0 0 0) 0x7f82dc056d60 con 0x7f82ec069f00 2023-10-06T05:32:04.772 INFO:tasks.ceph.mds.a.smithi033.stderr: -634> 2023-10-06T05:31:54.921+0000 7fecc8df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82cc066f30 msgr2=0x7f82cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82cc066f30 0x7f82cc0693e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0aedc0 tx=0x7f82e0006f90 comp rx=0 tx=0).stop 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 msgr2=0x7f82ec13ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 0x7f82ec13ed00 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0030c0 tx=0x7f82dc016000 comp rx=0 tx=0).stop 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 shutdown_connections 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f82cc066f30 0x7f82cc0693e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82ec140d20 0x7f82ec152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ec069f00 0x7f82ec13ed00 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 --2- 172.21.15.33:0/4280104809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ec0c9210 0x7f82ec1521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 >> 172.21.15.33:0/4280104809 conn(0x7f82ec06c560 msgr2=0x7f82ec0c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 shutdown_connections 2023-10-06T05:32:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.771+0000 7f82eb59e700 1 -- 172.21.15.33:0/4280104809 wait complete. 2023-10-06T05:32:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:32:04.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:32:04.807 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:32:04.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.982+0000 7feafd39d700 1 Processor -- start 2023-10-06T05:32:04.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.982+0000 7feafd39d700 1 -- start start 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.983+0000 7feafd39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feaf80abbd0 0x7feaf8067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.983+0000 7feafd39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf8068140 0x7feaf8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.983+0000 7feafd39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf806ae10 con 0x7feaf80b1f90 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.983+0000 7feafd39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80ac050 con 0x7feaf80abbd0 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.983+0000 7feafd39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf806a2d0 con 0x7feaf8068140 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feaf80abbd0 0x7feaf8067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7feaf80b1f90 0x7feaf8070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37316/0 (socket says 172.21.15.33:37316) 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf7fff700 1 -- 172.21.15.33:0/1550262327 learned_addr learned my addr 172.21.15.33:0/1550262327 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feafcb9c700 1 --2- 172.21.15.33:0/1550262327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf8068140 0x7feaf8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf77fe700 1 -- 172.21.15.33:0/1550262327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf8068140 msgr2=0x7feaf8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:04.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf77fe700 1 --2- 172.21.15.33:0/1550262327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf8068140 0x7feaf8159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf77fe700 1 -- 172.21.15.33:0/1550262327 >> v1:172.21.15.33:6789/0 conn(0x7feaf80b1f90 legacy=0x7feaf8070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf6ffd700 1 -- 172.21.15.33:0/1550262327 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019209924 0 0) 0x7feaf806ae10 con 0x7feaf80b1f90 2023-10-06T05:32:04.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.984+0000 7feaf77fe700 1 -- 172.21.15.33:0/1550262327 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8068580 con 0x7feaf80abbd0 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feaf77fe700 1 --2- 172.21.15.33:0/1550262327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf80abbd0 0x7feaf8067c00 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7feaec0098e0 tx=0x7feaec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b51b7065647eb1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feaf6ffd700 1 -- 172.21.15.33:0/1550262327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec009e80 con 0x7feaf80abbd0 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feaf6ffd700 1 -- 172.21.15.33:0/1550262327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec00b860 con 0x7feaf80abbd0 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feaf6ffd700 1 -- 172.21.15.33:0/1550262327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00bbe0 con 0x7feaf80abbd0 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 -- 172.21.15.33:0/1550262327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf80abbd0 msgr2=0x7feaf8067c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 --2- 172.21.15.33:0/1550262327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf80abbd0 0x7feaf8067c00 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7feaec0098e0 tx=0x7feaec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 -- 172.21.15.33:0/1550262327 shutdown_connections 2023-10-06T05:32:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 --2- 172.21.15.33:0/1550262327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf8068140 0x7feaf8159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 --2- 172.21.15.33:0/1550262327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf80abbd0 0x7feaf8067c00 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 -- 172.21.15.33:0/1550262327 >> 172.21.15.33:0/1550262327 conn(0x7feaf806ea70 msgr2=0x7feaf806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 -- 172.21.15.33:0/1550262327 shutdown_connections 2023-10-06T05:32:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.985+0000 7feafd39d700 1 -- 172.21.15.33:0/1550262327 wait complete. 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.987+0000 7feafd39d700 1 Processor -- start 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.987+0000 7feafd39d700 1 -- start start 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafd39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 0x7feaf8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafd39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf80abbd0 0x7feaf81598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafd39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf80b1f90 0x7feaf814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafd39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8159e30 con 0x7feaf8068140 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafd39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf816ac80 con 0x7feaf80abbd0 2023-10-06T05:32:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafd39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf816a070 con 0x7feaf80b1f90 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 0x7feaf8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 0x7feaf8158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46780/0 (socket says 172.21.15.33:46780) 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feaf7fff700 1 -- 172.21.15.33:0/3796747691 learned_addr learned my addr 172.21.15.33:0/3796747691 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feafcb9c700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf80b1f90 0x7feaf814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.988+0000 7feaf77fe700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf80abbd0 0x7feaf81598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feaf7fff700 1 -- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf80b1f90 msgr2=0x7feaf814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feaf7fff700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf80b1f90 0x7feaf814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feaf7fff700 1 -- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf80abbd0 msgr2=0x7feaf81598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feaf7fff700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf80abbd0 0x7feaf81598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feaf7fff700 1 -- 172.21.15.33:0/3796747691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec009420 con 0x7feaf8068140 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feaf7fff700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 0x7feaf8158a30 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7feae800c7e0 tx=0x7feae8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005b30 con 0x7feaf8068140 2023-10-06T05:32:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8005c90 con 0x7feaf8068140 2023-10-06T05:32:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8007300 con 0x7feaf8068140 2023-10-06T05:32:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf816a350 con 0x7feaf8068140 2023-10-06T05:32:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.989+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf816a910 con 0x7feaf8068140 2023-10-06T05:32:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.990+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf816d450 con 0x7feaf8068140 2023-10-06T05:32:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.991+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feae801b400 con 0x7feaf8068140 2023-10-06T05:32:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.992+0000 7feadffff700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feae4067000 0x7feae40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.992+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 6144+0+0 (secure 0 0 0) 0x7feae8084200 con 0x7feaf8068140 2023-10-06T05:32:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.992+0000 7feaf77fe700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feae4067000 0x7feae40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.993+0000 7feaf77fe700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feae4067000 0x7feae40694b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7feaec0091e0 tx=0x7feaec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:04.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:04.996+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feae8051340 con 0x7feaf8068140 2023-10-06T05:32:05.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.246+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feaf804c380 con 0x7feaf8068140 2023-10-06T05:32:05.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.768+0000 7feadffff700 1 -- 172.21.15.33:0/3796747691 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v187) v1 ==== 104+0+0 (secure 0 0 0) 0x7feae8054960 con 0x7feaf8068140 2023-10-06T05:32:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feae4067000 msgr2=0x7feae40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feae4067000 0x7feae40694b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7feaec0091e0 tx=0x7feaec017000 comp rx=0 tx=0).stop 2023-10-06T05:32:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 msgr2=0x7feaf8158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 0x7feaf8158a30 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7feae800c7e0 tx=0x7feae8007df0 comp rx=0 tx=0).stop 2023-10-06T05:32:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 shutdown_connections 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7feae4067000 0x7feae40694b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feaf80b1f90 0x7feaf814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaf8068140 0x7feaf8158a30 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 --2- 172.21.15.33:0/3796747691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf80abbd0 0x7feaf81598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 >> 172.21.15.33:0/3796747691 conn(0x7feaf806ea70 msgr2=0x7feaf80af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 shutdown_connections 2023-10-06T05:32:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:05.771+0000 7feafd39d700 1 -- 172.21.15.33:0/3796747691 wait complete. 2023-10-06T05:32:05.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:32:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.031+0000 7f6f525ff700 1 Processor -- start 2023-10-06T05:32:06.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.031+0000 7f6f525ff700 1 -- start start 2023-10-06T05:32:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f525ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 0x7f6f4c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f525ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f4c0c9a20 0x7f6f4c06cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f525ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c0c8af0 con 0x7f6f4c06c810 2023-10-06T05:32:06.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f525ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c7ee0 con 0x7f6f4c0c9a20 2023-10-06T05:32:06.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f525ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c81c0 con 0x7f6f4c159690 2023-10-06T05:32:06.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 0x7f6f4c159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:06.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f50b9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6f4c0c9a20 0x7f6f4c06cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:06.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 0x7f6f4c159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:60384/0 (socket says 172.21.15.33:60384) 2023-10-06T05:32:06.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f4bfff700 1 -- 172.21.15.33:0/588834922 learned_addr learned my addr 172.21.15.33:0/588834922 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:06.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.033+0000 7f6f4b7fe700 1 --1- 172.21.15.33:0/588834922 >> v1:172.21.15.167:6789/0 conn(0x7f6f4c06c810 0x7f6f4c0c9310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:38788/0 (socket says 172.21.15.33:38788) 2023-10-06T05:32:06.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.034+0000 7f6f4bfff700 1 -- 172.21.15.33:0/588834922 >> v2:172.21.15.167:3300/0 conn(0x7f6f4c0c9a20 msgr2=0x7f6f4c06cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:06.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.034+0000 7f6f4bfff700 1 --2- 172.21.15.33:0/588834922 >> v2:172.21.15.167:3300/0 conn(0x7f6f4c0c9a20 0x7f6f4c06cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:06.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.034+0000 7f6f4bfff700 1 -- 172.21.15.33:0/588834922 >> v1:172.21.15.167:6789/0 conn(0x7f6f4c06c810 legacy=0x7f6f4c0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:06.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.034+0000 7f6f4bfff700 1 -- 172.21.15.33:0/588834922 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0c84a0 con 0x7f6f4c159690 2023-10-06T05:32:06.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.034+0000 7f6f4bfff700 1 --2- 172.21.15.33:0/588834922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 0x7f6f4c159a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0099b0 tx=0x7f6f3c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9383d20d18d6050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.034+0000 7f6f4affd700 1 -- 172.21.15.33:0/588834922 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c009ea0 con 0x7f6f4c159690 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f4affd700 1 -- 172.21.15.33:0/588834922 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00b860 con 0x7f6f4c159690 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f4affd700 1 -- 172.21.15.33:0/588834922 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00bc00 con 0x7f6f4c159690 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 -- 172.21.15.33:0/588834922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 msgr2=0x7f6f4c159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 --2- 172.21.15.33:0/588834922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 0x7f6f4c159a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0099b0 tx=0x7f6f3c009250 comp rx=0 tx=0).stop 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 -- 172.21.15.33:0/588834922 shutdown_connections 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 --2- 172.21.15.33:0/588834922 >> v2:172.21.15.167:3300/0 conn(0x7f6f4c0c9a20 0x7f6f4c06cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:06.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 --2- 172.21.15.33:0/588834922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c159690 0x7f6f4c159a60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:06.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 -- 172.21.15.33:0/588834922 >> 172.21.15.33:0/588834922 conn(0x7f6f4c0cd4c0 msgr2=0x7f6f4c0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:06.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 -- 172.21.15.33:0/588834922 shutdown_connections 2023-10-06T05:32:06.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.035+0000 7f6f525ff700 1 -- 172.21.15.33:0/588834922 wait complete. 2023-10-06T05:32:06.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.037+0000 7f6f525ff700 1 Processor -- start 2023-10-06T05:32:06.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.037+0000 7f6f525ff700 1 -- start start 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f525ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c06c810 0x7f6f4c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f525ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f4c0c9a20 0x7f6f4c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f525ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 0x7f6f4c0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f525ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0ac290 con 0x7f6f4c159690 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f525ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c52b0 con 0x7f6f4c0c9a20 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f525ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c46a0 con 0x7f6f4c06c810 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 0x7f6f4c0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 0x7f6f4c0b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46796/0 (socket says 172.21.15.33:46796) 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 -- 172.21.15.33:0/813945164 learned_addr learned my addr 172.21.15.33:0/813945164 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f4bfff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c06c810 0x7f6f4c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f4b7fe700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f4c0c9a20 0x7f6f4c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 -- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c06c810 msgr2=0x7f6f4c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c06c810 0x7f6f4c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 -- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f4c0c9a20 msgr2=0x7f6f4c0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f4c0c9a20 0x7f6f4c0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.038+0000 7f6f50b9c700 1 -- 172.21.15.33:0/813945164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0094f0 con 0x7f6f4c159690 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.039+0000 7f6f50b9c700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 0x7f6f4c0b0260 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ec0 tx=0x7f6f4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.039+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400cb70 con 0x7f6f4c159690 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.039+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4400f040 con 0x7f6f4c159690 2023-10-06T05:32:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.039+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c0c4980 con 0x7f6f4c159690 2023-10-06T05:32:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.039+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44014480 con 0x7f6f4c159690 2023-10-06T05:32:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.039+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c0c4f40 con 0x7f6f4c159690 2023-10-06T05:32:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.040+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c0c5460 con 0x7f6f4c159690 2023-10-06T05:32:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.041+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f440147f0 con 0x7f6f4c159690 2023-10-06T05:32:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.042+0000 7f6f33fff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f38066f30 0x7f6f380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:06.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.042+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6f440852e0 con 0x7f6f4c159690 2023-10-06T05:32:06.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.042+0000 7f6f4bfff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f38066f30 0x7f6f380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:06.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.043+0000 7f6f4bfff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f38066f30 0x7f6f380693e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0099b0 tx=0x7f6f3c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:06.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.046+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f4c0c5460 con 0x7f6f4c159690 2023-10-06T05:32:06.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:06.253+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f4c0c5420 con 0x7f6f4c159690 2023-10-06T05:32:07.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.028+0000 7f6f33fff700 1 -- 172.21.15.33:0/813945164 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v241) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6f44055e30 con 0x7f6f4c159690 2023-10-06T05:32:07.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f38066f30 msgr2=0x7f6f380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f38066f30 0x7f6f380693e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0099b0 tx=0x7f6f3c017000 comp rx=0 tx=0).stop 2023-10-06T05:32:07.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 msgr2=0x7f6f4c0b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 0x7f6f4c0b0260 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ec0 tx=0x7f6f4400b040 comp rx=0 tx=0).stop 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 shutdown_connections 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6f38066f30 0x7f6f380693e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c159690 0x7f6f4c0b0260 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f4c06c810 0x7f6f4c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 --2- 172.21.15.33:0/813945164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f4c0c9a20 0x7f6f4c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.032+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 >> 172.21.15.33:0/813945164 conn(0x7f6f4c0cd4c0 msgr2=0x7f6f4c0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.033+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 shutdown_connections 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.033+0000 7f6f525ff700 1 -- 172.21.15.33:0/813945164 wait complete. 2023-10-06T05:32:07.035 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:32:07.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:32:07.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.254+0000 7f013599d700 1 Processor -- start 2023-10-06T05:32:07.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.254+0000 7f013599d700 1 -- start start 2023-10-06T05:32:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.255+0000 7f013599d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f01300aef40 0x7f0130155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.255+0000 7f013599d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300af310 con 0x7f013006ea00 2023-10-06T05:32:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.255+0000 7f013599d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0130069ad0 con 0x7f013006eea0 2023-10-06T05:32:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.255+0000 7f013599d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0130069db0 con 0x7f01300aef40 2023-10-06T05:32:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f013006ea00 0x7f01300ac5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37332/0 (socket says 172.21.15.33:37332) 2023-10-06T05:32:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012ffff700 1 -- 172.21.15.33:0/1796472375 learned_addr learned my addr 172.21.15.33:0/1796472375 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f013499b700 1 --2- 172.21.15.33:0/1796472375 >> v2:172.21.15.167:3300/0 conn(0x7f01300aef40 0x7f0130155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918718187 0 0) 0x7f01300af310 con 0x7f013006ea00 2023-10-06T05:32:07.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0120003570 con 0x7f013006ea00 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824936381 0 0) 0x7f0130069ad0 con 0x7f013006eea0 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01300af310 con 0x7f013006eea0 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 288011847 0 0) 0x7f0120003570 con 0x7f013006ea00 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 >> v2:172.21.15.167:3300/0 conn(0x7f01300aef40 msgr2=0x7f0130155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 --2- 172.21.15.33:0/1796472375 >> v2:172.21.15.167:3300/0 conn(0x7f01300aef40 0x7f0130155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.256+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 >> v1:172.21.15.167:6789/0 conn(0x7f013006eea0 legacy=0x7f01300c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013006a090 con 0x7f013006ea00 2023-10-06T05:32:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f011c001470 con 0x7f013006ea00 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 >> v1:172.21.15.33:6789/0 conn(0x7f013006ea00 legacy=0x7f01300ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f0120003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0120004810 0x7f0120004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01200051c0 0x7f012000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0130069db0 con 0x7f0120003bf0 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012000c0b0 con 0x7f01200051c0 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012000c270 con 0x7f0120004810 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013499b700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f0120003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f012ffff700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0120004810 0x7f0120004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013519c700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01200051c0 0x7f012000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013499b700 1 -- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0120004810 msgr2=0x7f0120004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013499b700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0120004810 0x7f0120004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013499b700 1 -- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01200051c0 msgr2=0x7f012000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013499b700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01200051c0 0x7f012000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013499b700 1 -- 172.21.15.33:0/1796472375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120003570 con 0x7f0120003bf0 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.257+0000 7f013519c700 1 -- 172.21.15.33:0/1796472375 reap_dead start 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.258+0000 7f013499b700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f0120003fe0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f0124009c00 tx=0x7f01240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4070b0b7cb02443d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.258+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012400e710 con 0x7f0120003bf0 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.258+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012400e870 con 0x7f0120003bf0 2023-10-06T05:32:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.258+0000 7f012f7fe700 1 -- 172.21.15.33:0/1796472375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012400ec30 con 0x7f0120003bf0 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.258+0000 7f013599d700 1 -- 172.21.15.33:0/1796472375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 msgr2=0x7f0120003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.258+0000 7f013599d700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f0120003fe0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f0124009c00 tx=0x7f01240091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.259+0000 7f013599d700 1 -- 172.21.15.33:0/1796472375 shutdown_connections 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.259+0000 7f013599d700 1 --2- 172.21.15.33:0/1796472375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f0120003fe0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.259+0000 7f013599d700 1 -- 172.21.15.33:0/1796472375 >> 172.21.15.33:0/1796472375 conn(0x7f0130068090 msgr2=0x7f013006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.259+0000 7f013599d700 1 -- 172.21.15.33:0/1796472375 shutdown_connections 2023-10-06T05:32:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.259+0000 7f013599d700 1 -- 172.21.15.33:0/1796472375 wait complete. 2023-10-06T05:32:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.260+0000 7f013599d700 1 Processor -- start 2023-10-06T05:32:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.260+0000 7f013599d700 1 -- start start 2023-10-06T05:32:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013599d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f01301467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013599d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01300b0540 0x7f01300b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f013016ced0 0x7f013016d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013599d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300b0ef0 con 0x7f0120003bf0 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013599d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013014a030 con 0x7f01300b0540 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01300c12a0 con 0x7f013016ced0 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.261+0000 7f013499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f01301467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f01301467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46820/0 (socket says 172.21.15.33:46820) 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 -- 172.21.15.33:0/2851240178 learned_addr learned my addr 172.21.15.33:0/2851240178 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f012ffff700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01300b0540 0x7f01300b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013519c700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f013016ced0 0x7f013016d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 -- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f013016ced0 msgr2=0x7f013016d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f013016ced0 0x7f013016d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 -- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01300b0540 msgr2=0x7f01300b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01300b0540 0x7f01300b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 -- 172.21.15.33:0/2851240178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300c2100 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f012ffff700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01300b0540 0x7f01300b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013519c700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f013016ced0 0x7f013016d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013499b700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f01301467f0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f012400e4b0 tx=0x7f012401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124022070 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124013d00 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012401c6a0 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.262+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0130147070 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.263+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01300c17d0 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.264+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0130063ff0 con 0x7f0120003bf0 2023-10-06T05:32:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.264+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f012401ca90 con 0x7f0120003bf0 2023-10-06T05:32:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.265+0000 7f012cff9700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0114066f30 0x7f01140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.265+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0124085d70 con 0x7f0120003bf0 2023-10-06T05:32:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.266+0000 7f012ffff700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0114066f30 0x7f01140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.266+0000 7f012ffff700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0114066f30 0x7f01140693e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f011c001ca0 tx=0x7f011c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.269+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0124052f40 con 0x7f0120003bf0 2023-10-06T05:32:07.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:07.471+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0130146d30 con 0x7f0120003bf0 2023-10-06T05:32:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.031+0000 7f012cff9700 1 -- 172.21.15.33:0/2851240178 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v242) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0124017020 con 0x7f0120003bf0 2023-10-06T05:32:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0114066f30 msgr2=0x7f01140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0114066f30 0x7f01140693e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f011c001ca0 tx=0x7f011c009040 comp rx=0 tx=0).stop 2023-10-06T05:32:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 msgr2=0x7f01301467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f01301467f0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f012400e4b0 tx=0x7f012401a000 comp rx=0 tx=0).stop 2023-10-06T05:32:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 shutdown_connections 2023-10-06T05:32:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0114066f30 0x7f01140693e0 secure :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f011c001ca0 tx=0x7f011c009040 comp rx=0 tx=0).stop 2023-10-06T05:32:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f013016ced0 0x7f013016d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0120003bf0 0x7f01301467f0 secure :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f012400e4b0 tx=0x7f012401a000 comp rx=0 tx=0).stop 2023-10-06T05:32:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 --2- 172.21.15.33:0/2851240178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01300b0540 0x7f01300b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.038+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 >> 172.21.15.33:0/2851240178 conn(0x7f0130068090 msgr2=0x7f01300b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.040+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 shutdown_connections 2023-10-06T05:32:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.040+0000 7f013599d700 1 -- 172.21.15.33:0/2851240178 wait complete. 2023-10-06T05:32:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:32:08.065 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:32:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.256+0000 7f7846b9d700 1 Processor -- start 2023-10-06T05:32:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.256+0000 7f7846b9d700 1 -- start start 2023-10-06T05:32:08.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.257+0000 7f7846b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78400cd5e0 0x7f78400af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.257+0000 7f7846b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f784015a650 0x7f784015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.257+0000 7f7846b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784015d660 con 0x7f78400af540 2023-10-06T05:32:08.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.257+0000 7f7846b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400ce890 con 0x7f784015a650 2023-10-06T05:32:08.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.257+0000 7f7846b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784015aad0 con 0x7f78400cd5e0 2023-10-06T05:32:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.257+0000 7f784639c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f784015a650 0x7f784015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f784015a650 0x7f784015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46828/0 (socket says 172.21.15.33:46828) 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 -- 172.21.15.33:0/166193960 learned_addr learned my addr 172.21.15.33:0/166193960 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784539a700 1 --2- 172.21.15.33:0/166193960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78400cd5e0 0x7f78400af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 -- 172.21.15.33:0/166193960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78400cd5e0 msgr2=0x7f78400af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 --2- 172.21.15.33:0/166193960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78400cd5e0 0x7f78400af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 -- 172.21.15.33:0/166193960 >> v1:172.21.15.167:6789/0 conn(0x7f78400af540 legacy=0x7f78400cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 -- 172.21.15.33:0/166193960 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784015d890 con 0x7f784015a650 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f7844b99700 1 -- 172.21.15.33:0/166193960 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530495201 0 0) 0x7f784015d660 con 0x7f78400af540 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.258+0000 7f784639c700 1 --2- 172.21.15.33:0/166193960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784015a650 0x7f784015d120 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f7838009f20 tx=0x7f7838009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debac84a767f79fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7844b99700 1 -- 172.21.15.33:0/166193960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783800e070 con 0x7f784015a650 2023-10-06T05:32:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7844b99700 1 -- 172.21.15.33:0/166193960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783800b9d0 con 0x7f784015a650 2023-10-06T05:32:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7844b99700 1 -- 172.21.15.33:0/166193960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78380092e0 con 0x7f784015a650 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 -- 172.21.15.33:0/166193960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784015a650 msgr2=0x7f784015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 --2- 172.21.15.33:0/166193960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784015a650 0x7f784015d120 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f7838009f20 tx=0x7f7838009840 comp rx=0 tx=0).stop 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 -- 172.21.15.33:0/166193960 shutdown_connections 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 --2- 172.21.15.33:0/166193960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f784015a650 0x7f784015d120 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 --2- 172.21.15.33:0/166193960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78400cd5e0 0x7f78400af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 -- 172.21.15.33:0/166193960 >> 172.21.15.33:0/166193960 conn(0x7f784006d9b0 msgr2=0x7f784006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.259+0000 7f7846b9d700 1 -- 172.21.15.33:0/166193960 shutdown_connections 2023-10-06T05:32:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.260+0000 7f7846b9d700 1 -- 172.21.15.33:0/166193960 wait complete. 2023-10-06T05:32:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.261+0000 7f7846b9d700 1 Processor -- start 2023-10-06T05:32:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.261+0000 7f7846b9d700 1 -- start start 2023-10-06T05:32:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7846b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78400af540 0x7f7840169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7846b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 0x7f78400c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7846b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784015a650 0x7f78400c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7846b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840152b10 con 0x7f78400cd5e0 2023-10-06T05:32:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7846b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784015e1c0 con 0x7f78400af540 2023-10-06T05:32:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7846b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784015ea20 con 0x7f784015a650 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f784539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 0x7f78400c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f784539a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 0x7f78400c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46836/0 (socket says 172.21.15.33:46836) 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f784539a700 1 -- 172.21.15.33:0/1323099424 learned_addr learned my addr 172.21.15.33:0/1323099424 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f784639c700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784015a650 0x7f78400c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.262+0000 7f7845b9b700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78400af540 0x7f7840169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f784539a700 1 -- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784015a650 msgr2=0x7f78400c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f784539a700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784015a650 0x7f78400c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f784539a700 1 -- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78400af540 msgr2=0x7f7840169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f784539a700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78400af540 0x7f7840169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f784539a700 1 -- 172.21.15.33:0/1323099424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7838009ae0 con 0x7f78400cd5e0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f7845b9b700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78400af540 0x7f7840169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f784539a700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 0x7f78400c7f90 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f783000ecb0 tx=0x7f78300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000c920 con 0x7f78400cd5e0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783000caa0 con 0x7f78400cd5e0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000ce60 con 0x7f78400cd5e0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784016a0e0 con 0x7f78400cd5e0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.263+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784015e3d0 con 0x7f78400cd5e0 2023-10-06T05:32:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.264+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78400c7110 con 0x7f78400cd5e0 2023-10-06T05:32:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.265+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7830013680 con 0x7f78400cd5e0 2023-10-06T05:32:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.266+0000 7f7835ffb700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7824066f30 0x7f78240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.266+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7830084750 con 0x7f78400cd5e0 2023-10-06T05:32:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.267+0000 7f7845b9b700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7824066f30 0x7f78240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.267+0000 7f7845b9b700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7824066f30 0x7f78240693e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f783c007fd0 tx=0x7f783c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.270+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7830051ac0 con 0x7f78400cd5e0 2023-10-06T05:32:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.474+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f78400afcd0 con 0x7f78400cd5e0 2023-10-06T05:32:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.478+0000 7f7835ffb700 1 -- 172.21.15.33:0/1323099424 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f78300550e0 con 0x7f78400cd5e0 2023-10-06T05:32:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.480+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7824066f30 msgr2=0x7f78240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.480+0000 7f7846b9d700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7824066f30 0x7f78240693e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f783c007fd0 tx=0x7f783c009040 comp rx=0 tx=0).stop 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.480+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 msgr2=0x7f78400c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.480+0000 7f7846b9d700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 0x7f78400c7f90 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f783000ecb0 tx=0x7f78300091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 shutdown_connections 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7824066f30 0x7f78240693e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784015a650 0x7f78400c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78400af540 0x7f7840169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 --2- 172.21.15.33:0/1323099424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78400cd5e0 0x7f78400c7f90 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 >> 172.21.15.33:0/1323099424 conn(0x7f784006d9b0 msgr2=0x7f784006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 shutdown_connections 2023-10-06T05:32:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.481+0000 7f7846b9d700 1 -- 172.21.15.33:0/1323099424 wait complete. 2023-10-06T05:32:08.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_renew' 2023-10-06T05:32:08.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.685+0000 7f7a1bfff700 1 Processor -- start 2023-10-06T05:32:08.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.685+0000 7f7a1bfff700 1 -- start start 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a1c155140 0x7f7a1c158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 0x7f7a1c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1c0c8e40 con 0x7f7a1c0c9810 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1c0ae6c0 con 0x7f7a1c155140 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0ae9a0 con 0x7f7a1c0adf50 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7a1c0c9810 0x7f7a1c0699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37348/0 (socket says 172.21.15.33:37348) 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1b7fe700 1 -- 172.21.15.33:0/3858767129 learned_addr learned my addr 172.21.15.33:0/3858767129 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1affd700 1 --2- 172.21.15.33:0/3858767129 >> v2:172.21.15.167:3300/0 conn(0x7f7a1c155140 0x7f7a1c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a1a7fc700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 0x7f7a1c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.687+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722821748 0 0) 0x7f7a1c0c8e40 con 0x7f7a1c0c9810 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a04003570 con 0x7f7a1c0c9810 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1968567168 0 0) 0x7f7a04003570 con 0x7f7a1c0c9810 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 msgr2=0x7f7a1c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 0x7f7a1c0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 >> v2:172.21.15.167:3300/0 conn(0x7f7a1c155140 msgr2=0x7f7a1c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 --2- 172.21.15.33:0/3858767129 >> v2:172.21.15.167:3300/0 conn(0x7f7a1c155140 0x7f7a1c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0c8ad0 con 0x7f7a1c0c9810 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f7a140017f0 con 0x7f7a1c0c9810 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 >> v1:172.21.15.33:6789/0 conn(0x7f7a1c0c9810 legacy=0x7f7a1c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a04003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 0x7f7a04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a04004d50 0x7f7a040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0ae6c0 con 0x7f7a04003ae0 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0ae9a0 con 0x7f7a04004d50 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0400bb40 con 0x7f7a1c0adf50 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.688+0000 7f7a1affd700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a04003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1a7fc700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 0x7f7a04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1b7fe700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a04004d50 0x7f7a040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1affd700 1 -- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 msgr2=0x7f7a04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1affd700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c0adf50 0x7f7a04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1affd700 1 -- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a04004d50 msgr2=0x7f7a040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1affd700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a04004d50 0x7f7a040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1affd700 1 -- 172.21.15.33:0/3858767129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04003570 con 0x7f7a04003ae0 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a1affd700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a04003ef0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009c00 tx=0x7f7a0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f5f9676bbcc7e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00e6b0 con 0x7f7a04003ae0 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00e810 con 0x7f7a04003ae0 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.689+0000 7f7a19ffb700 1 -- 172.21.15.33:0/3858767129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00ebd0 con 0x7f7a04003ae0 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1bfff700 1 -- 172.21.15.33:0/3858767129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 msgr2=0x7f7a04003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/3858767129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a04003ef0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009c00 tx=0x7f7a0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1b7fe700 1 -- 172.21.15.33:0/3858767129 reap_dead start 2023-10-06T05:32:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1bfff700 1 -- 172.21.15.33:0/3858767129 shutdown_connections 2023-10-06T05:32:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1bfff700 1 -- 172.21.15.33:0/3858767129 >> 172.21.15.33:0/3858767129 conn(0x7f7a1c06d030 msgr2=0x7f7a1c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1bfff700 1 -- 172.21.15.33:0/3858767129 shutdown_connections 2023-10-06T05:32:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.690+0000 7f7a1bfff700 1 -- 172.21.15.33:0/3858767129 wait complete. 2023-10-06T05:32:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.691+0000 7f7a1bfff700 1 Processor -- start 2023-10-06T05:32:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.691+0000 7f7a1bfff700 1 -- start start 2023-10-06T05:32:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a1c158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c14eeb0 0x7f7a1c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c14f320 0x7f7a1c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c159430 con 0x7f7a04003ae0 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c157e50 con 0x7f7a1c14f320 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c158130 con 0x7f7a1c14eeb0 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.692+0000 7f7a1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a1c158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a1c158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46858/0 (socket says 172.21.15.33:46858) 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 -- 172.21.15.33:0/1586573313 learned_addr learned my addr 172.21.15.33:0/1586573313 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1a7fc700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c14eeb0 0x7f7a1c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1b7fe700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c14f320 0x7f7a1c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 -- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c14eeb0 msgr2=0x7f7a1c156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c14eeb0 0x7f7a1c156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 -- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c14f320 msgr2=0x7f7a1c1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c14f320 0x7f7a1c1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 -- 172.21.15.33:0/1586573313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c158410 con 0x7f7a04003ae0 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1a7fc700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c14eeb0 0x7f7a1c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1b7fe700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c14f320 0x7f7a1c1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.693+0000 7f7a1affd700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a1c158ef0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f7a04004730 tx=0x7f7a0c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.694+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c022070 con 0x7f7a04003ae0 2023-10-06T05:32:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.694+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00e980 con 0x7f7a04003ae0 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.694+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c027470 con 0x7f7a04003ae0 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.694+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1c1586f0 con 0x7f7a04003ae0 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.694+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c1560c0 con 0x7f7a04003ae0 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.695+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1c04c380 con 0x7f7a04003ae0 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.696+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a0c01c450 con 0x7f7a04003ae0 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.697+0000 7f7a02ffd700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a08066f30 0x7f7a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.697+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7a0c085fa0 con 0x7f7a04003ae0 2023-10-06T05:32:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.697+0000 7f7a1a7fc700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a08066f30 0x7f7a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.698+0000 7f7a1a7fc700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a08066f30 0x7f7a080693e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0adbc0 tx=0x7f7a10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.701+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a0c0533d0 con 0x7f7a04003ae0 2023-10-06T05:32:08.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:08.985+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_renew"]} v 0) v1 -- 0x7f7a1c06fa40 con 0x7f7a04003ae0 2023-10-06T05:32:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.042+0000 7f7a02ffd700 1 -- 172.21.15.33:0/1586573313 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_renew"]}]=0 v884) v1 ==== 148+0+0 (secure 0 0 0) 0x7f7a0c0170d0 con 0x7f7a04003ae0 2023-10-06T05:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a08066f30 msgr2=0x7f7a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a08066f30 0x7f7a080693e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0adbc0 tx=0x7f7a10008040 comp rx=0 tx=0).stop 2023-10-06T05:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 msgr2=0x7f7a1c158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a1c158ef0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f7a04004730 tx=0x7f7a0c01a000 comp rx=0 tx=0).stop 2023-10-06T05:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 shutdown_connections 2023-10-06T05:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a08066f30 0x7f7a080693e0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c14f320 0x7f7a1c1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a04003ae0 0x7f7a1c158ef0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 --2- 172.21.15.33:0/1586573313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a1c14eeb0 0x7f7a1c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 >> 172.21.15.33:0/1586573313 conn(0x7f7a1c06d030 msgr2=0x7f7a1c1599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 shutdown_connections 2023-10-06T05:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.044+0000 7f7a1bfff700 1 -- 172.21.15.33:0/1586573313 wait complete. 2023-10-06T05:32:09.063 INFO:tasks.cephfs_test_runner:test_stale_renew (tasks.cephfs.test_client_recovery.TestClientRecovery) ... skipped 'Require FUSE client to handle signal STOP/CONT' 2023-10-06T05:32:09.063 INFO:tasks.cephfs_test_runner:Starting test: test_stale_write_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:32:09.064 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_write_caps' 2023-10-06T05:32:09.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.248+0000 7fc92bfff700 1 Processor -- start 2023-10-06T05:32:09.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.249+0000 7fc92bfff700 1 -- start start 2023-10-06T05:32:09.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 0x7fc92c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc92c155ec0 0x7fc92c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c06ab60 con 0x7fc92c155af0 2023-10-06T05:32:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c06b7e0 con 0x7fc92c155ec0 2023-10-06T05:32:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c06ae40 con 0x7fc92c0c8b50 2023-10-06T05:32:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92a7fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 0x7fc92c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92a7fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 0x7fc92c0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:60476/0 (socket says 172.21.15.33:60476) 2023-10-06T05:32:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92a7fc700 1 -- 172.21.15.33:0/1370536120 learned_addr learned my addr 172.21.15.33:0/1370536120 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.250+0000 7fc92b7fe700 1 --2- 172.21.15.33:0/1370536120 >> v2:172.21.15.167:3300/0 conn(0x7fc92c155ec0 0x7fc92c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc929ffb700 1 -- 172.21.15.33:0/1370536120 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092290802 0 0) 0x7fc92c06ab60 con 0x7fc92c155af0 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc929ffb700 1 -- 172.21.15.33:0/1370536120 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc914003570 con 0x7fc92c155af0 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc92a7fc700 1 -- 172.21.15.33:0/1370536120 >> v2:172.21.15.167:3300/0 conn(0x7fc92c155ec0 msgr2=0x7fc92c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc92a7fc700 1 --2- 172.21.15.33:0/1370536120 >> v2:172.21.15.167:3300/0 conn(0x7fc92c155ec0 0x7fc92c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc92a7fc700 1 -- 172.21.15.33:0/1370536120 >> v1:172.21.15.167:6789/0 conn(0x7fc92c155af0 legacy=0x7fc92c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc92a7fc700 1 -- 172.21.15.33:0/1370536120 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c06b120 con 0x7fc92c0c8b50 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc92b7fe700 1 --2- 172.21.15.33:0/1370536120 >> v2:172.21.15.167:3300/0 conn(0x7fc92c155ec0 0x7fc92c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.251+0000 7fc92a7fc700 1 --2- 172.21.15.33:0/1370536120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 0x7fc92c0c7210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9200099b0 tx=0x7fc920009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248964e4de58c053 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc929ffb700 1 -- 172.21.15.33:0/1370536120 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000b680 con 0x7fc92c0c8b50 2023-10-06T05:32:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc929ffb700 1 -- 172.21.15.33:0/1370536120 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92000b7e0 con 0x7fc92c0c8b50 2023-10-06T05:32:09.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc929ffb700 1 -- 172.21.15.33:0/1370536120 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000bb80 con 0x7fc92c0c8b50 2023-10-06T05:32:09.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 -- 172.21.15.33:0/1370536120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 msgr2=0x7fc92c0c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 --2- 172.21.15.33:0/1370536120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 0x7fc92c0c7210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9200099b0 tx=0x7fc920009250 comp rx=0 tx=0).stop 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 -- 172.21.15.33:0/1370536120 shutdown_connections 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 --2- 172.21.15.33:0/1370536120 >> v2:172.21.15.167:3300/0 conn(0x7fc92c155ec0 0x7fc92c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 --2- 172.21.15.33:0/1370536120 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c0c8b50 0x7fc92c0c7210 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 -- 172.21.15.33:0/1370536120 >> 172.21.15.33:0/1370536120 conn(0x7fc92c06d030 msgr2=0x7fc92c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 -- 172.21.15.33:0/1370536120 shutdown_connections 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.252+0000 7fc92bfff700 1 -- 172.21.15.33:0/1370536120 wait complete. 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.254+0000 7fc92bfff700 1 Processor -- start 2023-10-06T05:32:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.254+0000 7fc92bfff700 1 -- start start 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c0c8b50 0x7fc92c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c155af0 0x7fc92c16b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 0x7fc92c06e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0c3500 con 0x7fc92c155ec0 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c154120 con 0x7fc92c0c8b50 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c16b990 con 0x7fc92c155af0 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.255+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 0x7fc92c06e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 0x7fc92c06e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46864/0 (socket says 172.21.15.33:46864) 2023-10-06T05:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 -- 172.21.15.33:0/2573653698 learned_addr learned my addr 172.21.15.33:0/2573653698 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92affd700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c0c8b50 0x7fc92c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92a7fc700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c155af0 0x7fc92c16b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 -- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c155af0 msgr2=0x7fc92c16b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c155af0 0x7fc92c16b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 -- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c0c8b50 msgr2=0x7fc92c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c0c8b50 0x7fc92c153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 -- 172.21.15.33:0/2573653698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924009a20 con 0x7fc92c155ec0 2023-10-06T05:32:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc92b7fe700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 0x7fc92c06e150 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fc0 tx=0x7fc92400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.256+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400cbd0 con 0x7fc92c155ec0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.257+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92400f040 con 0x7fc92c155ec0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.257+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9200094f0 con 0x7fc92c155ec0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.257+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924014480 con 0x7fc92c155ec0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.257+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c16bf50 con 0x7fc92c155ec0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.258+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c0c7f10 con 0x7fc92c155ec0 2023-10-06T05:32:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.258+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc924003020 con 0x7fc92c155ec0 2023-10-06T05:32:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.259+0000 7fc912ffd700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc918066f30 0x7fc9180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.259+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc924084cb0 con 0x7fc92c155ec0 2023-10-06T05:32:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.260+0000 7fc92affd700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc918066f30 0x7fc9180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.261+0000 7fc92affd700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc918066f30 0x7fc9180693e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc91c007fd0 tx=0x7fc91c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.264+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9240520c0 con 0x7fc92c155ec0 2023-10-06T05:32:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:09.554+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_write_caps"]} v 0) v1 -- 0x7fc92c04c380 con 0x7fc92c155ec0 2023-10-06T05:32:10.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.044+0000 7fc912ffd700 1 -- 172.21.15.33:0/2573653698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_write_caps"]}]=0 v885) v1 ==== 156+0+0 (secure 0 0 0) 0x7fc9240556e0 con 0x7fc92c155ec0 2023-10-06T05:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc918066f30 msgr2=0x7fc9180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc918066f30 0x7fc9180693e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc91c007fd0 tx=0x7fc91c007d50 comp rx=0 tx=0).stop 2023-10-06T05:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 msgr2=0x7fc92c06e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 0x7fc92c06e150 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fc0 tx=0x7fc92400b040 comp rx=0 tx=0).stop 2023-10-06T05:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 shutdown_connections 2023-10-06T05:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc918066f30 0x7fc9180693e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc92c155ec0 0x7fc92c06e150 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c0c8b50 0x7fc92c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 --2- 172.21.15.33:0/2573653698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc92c155af0 0x7fc92c16b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 >> 172.21.15.33:0/2573653698 conn(0x7fc92c06d030 msgr2=0x7fc92c0c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:10.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 shutdown_connections 2023-10-06T05:32:10.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.047+0000 7fc92bfff700 1 -- 172.21.15.33:0/2573653698 wait complete. 2023-10-06T05:32:10.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:32:10.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.257+0000 7ffa6359e700 1 Processor -- start 2023-10-06T05:32:10.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.257+0000 7ffa6359e700 1 -- start start 2023-10-06T05:32:10.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa64156370 0x7ffa640af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa640b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa640ad430 con 0x7ffa640cc7e0 2023-10-06T05:32:10.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa640afe20 con 0x7ffa64156370 2023-10-06T05:32:10.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640b0b70 con 0x7ffa640b01f0 2023-10-06T05:32:10.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa64156370 0x7ffa640af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffa64156370 0x7ffa640af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52924/0 (socket says 172.21.15.33:52924) 2023-10-06T05:32:10.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 -- 172.21.15.33:0/853531301 learned_addr learned my addr 172.21.15.33:0/853531301 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:10.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa62d9d700 1 --2- 172.21.15.33:0/853531301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa640b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 -- 172.21.15.33:0/853531301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 msgr2=0x7ffa640b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 --2- 172.21.15.33:0/853531301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa640b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 -- 172.21.15.33:0/853531301 >> v1:172.21.15.167:6789/0 conn(0x7ffa640cc7e0 legacy=0x7ffa640c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.259+0000 7ffa6259c700 1 -- 172.21.15.33:0/853531301 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6406bc70 con 0x7ffa64156370 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.260+0000 7ffa6259c700 1 --2- 172.21.15.33:0/853531301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa640af8e0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400bc50 tx=0x7ffa5400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbdeadb9e15edbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.260+0000 7ffa6159a700 1 -- 172.21.15.33:0/853531301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400be60 con 0x7ffa64156370 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.260+0000 7ffa6159a700 1 -- 172.21.15.33:0/853531301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5400de00 con 0x7ffa64156370 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.260+0000 7ffa6159a700 1 -- 172.21.15.33:0/853531301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540196b0 con 0x7ffa64156370 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.260+0000 7ffa6359e700 1 -- 172.21.15.33:0/853531301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 msgr2=0x7ffa640af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.260+0000 7ffa6359e700 1 --2- 172.21.15.33:0/853531301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa640af8e0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400bc50 tx=0x7ffa5400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.261+0000 7ffa6359e700 1 -- 172.21.15.33:0/853531301 shutdown_connections 2023-10-06T05:32:10.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.261+0000 7ffa6359e700 1 --2- 172.21.15.33:0/853531301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa640af8e0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.261+0000 7ffa6359e700 1 --2- 172.21.15.33:0/853531301 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa640b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.261+0000 7ffa6359e700 1 -- 172.21.15.33:0/853531301 >> 172.21.15.33:0/853531301 conn(0x7ffa6406d030 msgr2=0x7ffa640cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:10.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.261+0000 7ffa6359e700 1 -- 172.21.15.33:0/853531301 shutdown_connections 2023-10-06T05:32:10.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.261+0000 7ffa6359e700 1 -- 172.21.15.33:0/853531301 wait complete. 2023-10-06T05:32:10.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.262+0000 7ffa6359e700 1 Processor -- start 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.262+0000 7ffa6359e700 1 -- start start 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa6359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa641587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa6359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa640cc7e0 0x7ffa64150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa6359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa64153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa6359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64156870 con 0x7ffa64156370 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa6359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64152e60 con 0x7ffa640cc7e0 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa6359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64153140 con 0x7ffa640b01f0 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa64153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa62d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa64153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52932/0 (socket says 172.21.15.33:52932) 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.263+0000 7ffa62d9d700 1 -- 172.21.15.33:0/2495416705 learned_addr learned my addr 172.21.15.33:0/2495416705 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa6259c700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa641587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa61d9b700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa640cc7e0 0x7ffa64150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa62d9d700 1 -- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 msgr2=0x7ffa641587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa62d9d700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa641587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa62d9d700 1 -- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa640cc7e0 msgr2=0x7ffa64150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa62d9d700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa640cc7e0 0x7ffa64150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa62d9d700 1 -- 172.21.15.33:0/2495416705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5400b790 con 0x7ffa64156370 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa6259c700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa641587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa61d9b700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa640cc7e0 0x7ffa64150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa62d9d700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa64153a30 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007f20 tx=0x7ffa5c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00eca0 con 0x7ffa64156370 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c00ee20 con 0x7ffa64156370 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c014670 con 0x7ffa64156370 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.264+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa64153420 con 0x7ffa64156370 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.265+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa64152b30 con 0x7ffa64156370 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.265+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa64156a20 con 0x7ffa64156370 2023-10-06T05:32:10.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.266+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa5c0149e0 con 0x7ffa64156370 2023-10-06T05:32:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.268+0000 7ffa527fc700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa44066f30 0x7ffa440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.268+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ffa5c084aa0 con 0x7ffa64156370 2023-10-06T05:32:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.268+0000 7ffa6259c700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa44066f30 0x7ffa440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.269+0000 7ffa6259c700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa44066f30 0x7ffa440693e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400bc50 tx=0x7ffa5401b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:10.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.271+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa64156a20 con 0x7ffa64156370 2023-10-06T05:32:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.473+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ffa6404c380 con 0x7ffa64156370 2023-10-06T05:32:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.478+0000 7ffa527fc700 1 -- 172.21.15.33:0/2495416705 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffa5c051eb0 con 0x7ffa64156370 2023-10-06T05:32:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.480+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa44066f30 msgr2=0x7ffa440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.480+0000 7ffa6359e700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa44066f30 0x7ffa440693e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400bc50 tx=0x7ffa5401b000 comp rx=0 tx=0).stop 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.480+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 msgr2=0x7ffa64153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.480+0000 7ffa6359e700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa64153a30 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007f20 tx=0x7ffa5c009860 comp rx=0 tx=0).stop 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 shutdown_connections 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffa44066f30 0x7ffa440693e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa64156370 0x7ffa64153a30 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa640b01f0 0x7ffa641587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 --2- 172.21.15.33:0/2495416705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa640cc7e0 0x7ffa64150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 >> 172.21.15.33:0/2495416705 conn(0x7ffa6406d030 msgr2=0x7ffa640abd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 shutdown_connections 2023-10-06T05:32:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.481+0000 7ffa6359e700 1 -- 172.21.15.33:0/2495416705 wait complete. 2023-10-06T05:32:10.500 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:32:10.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:32:10.582 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:32:10.670 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:32:10.671 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:32:10.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.856+0000 7f147139d700 1 Processor -- start 2023-10-06T05:32:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.857+0000 7f147139d700 1 -- start start 2023-10-06T05:32:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f147139d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f146c0cd4b0 0x7f146c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f147139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0b1ae0 con 0x7f146c0b04e0 2023-10-06T05:32:10.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f147139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0b17b0 con 0x7f146c1571f0 2023-10-06T05:32:10.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f147139d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0ad8c0 con 0x7f146c0cd4b0 2023-10-06T05:32:10.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f146b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f146c0b04e0 0x7f146c0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41026/0 (socket says 172.21.15.33:41026) 2023-10-06T05:32:10.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f146b7fe700 1 -- 172.21.15.33:0/1769819654 learned_addr learned my addr 172.21.15.33:0/1769819654 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.858+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135872912 0 0) 0x7f146c0b1ae0 con 0x7f146c0b04e0 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1454003570 con 0x7f146c0b04e0 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f1470b9c700 1 --2- 172.21.15.33:0/1769819654 >> v2:172.21.15.167:3300/0 conn(0x7f146c0cd4b0 0x7f146c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999534896 0 0) 0x7f1454003570 con 0x7f146c0b04e0 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 >> v2:172.21.15.167:3300/0 conn(0x7f146c0cd4b0 msgr2=0x7f146c1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 --2- 172.21.15.33:0/1769819654 >> v2:172.21.15.167:3300/0 conn(0x7f146c0cd4b0 0x7f146c1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 >> v1:172.21.15.167:6789/0 conn(0x7f146c1571f0 legacy=0x7f146c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0cd9f0 con 0x7f146c0b04e0 2023-10-06T05:32:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1460001470 con 0x7f146c0b04e0 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 >> v1:172.21.15.33:6789/0 conn(0x7f146c0b04e0 legacy=0x7f146c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1454003ab0 0x7f1454003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454004810 0x7f1454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1454005210 0x7f145400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0ad8c0 con 0x7f1454003ab0 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145400c130 con 0x7f1454005210 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145400c2f0 con 0x7f1454004810 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146bfff700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1454003ab0 0x7f1454003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f146b7fe700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454004810 0x7f1454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.859+0000 7f1470b9c700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1454005210 0x7f145400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146bfff700 1 -- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454004810 msgr2=0x7f1454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146bfff700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454004810 0x7f1454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146bfff700 1 -- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1454005210 msgr2=0x7f145400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146bfff700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1454005210 0x7f145400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146bfff700 1 -- 172.21.15.33:0/1769819654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454003570 con 0x7f1454003ab0 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f1470b9c700 1 -- 172.21.15.33:0/1769819654 reap_dead start 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146bfff700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1454003ab0 0x7f1454003ec0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f145c0018b0 tx=0x7f145c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bcfd4d32425881 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c012420 con 0x7f1454003ab0 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c0125a0 con 0x7f1454003ab0 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.860+0000 7f146affd700 1 -- 172.21.15.33:0/1769819654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c012960 con 0x7f1454003ab0 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 -- 172.21.15.33:0/1769819654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1454003ab0 msgr2=0x7f1454003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1454003ab0 0x7f1454003ec0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f145c0018b0 tx=0x7f145c007d30 comp rx=0 tx=0).stop 2023-10-06T05:32:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 -- 172.21.15.33:0/1769819654 shutdown_connections 2023-10-06T05:32:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 --2- 172.21.15.33:0/1769819654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1454003ab0 0x7f1454003ec0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 -- 172.21.15.33:0/1769819654 >> 172.21.15.33:0/1769819654 conn(0x7f146c06c560 msgr2=0x7f146c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 -- 172.21.15.33:0/1769819654 shutdown_connections 2023-10-06T05:32:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.861+0000 7f147139d700 1 -- 172.21.15.33:0/1769819654 wait complete. 2023-10-06T05:32:10.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.862+0000 7f147139d700 1 Processor -- start 2023-10-06T05:32:10.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 -- start start 2023-10-06T05:32:10.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454003ab0 0x7f146c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146c15b2b0 0x7f146c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 0x7f146c0c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15bc60 con 0x7f146c0c31a0 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0c4aa0 con 0x7f146c15b2b0 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.863+0000 7f147139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15eef0 con 0x7f1454003ab0 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 0x7f146c0c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 0x7f146c0c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52950/0 (socket says 172.21.15.33:52950) 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 -- 172.21.15.33:0/2811695448 learned_addr learned my addr 172.21.15.33:0/2811695448 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f146bfff700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454003ab0 0x7f146c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f146b7fe700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146c15b2b0 0x7f146c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 -- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454003ab0 msgr2=0x7f146c152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454003ab0 0x7f146c152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 -- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146c15b2b0 msgr2=0x7f146c15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146c15b2b0 0x7f146c15b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 -- 172.21.15.33:0/2811695448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0c3e90 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f146bfff700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454003ab0 0x7f146c152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.864+0000 7f1470b9c700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 0x7f146c0c3610 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f146c0b0f40 tx=0x7f146400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.865+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146400e040 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.865+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146400f040 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.865+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146400fe20 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.865+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0c4170 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.865+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c0c4730 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.866+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0cc320 con 0x7f146c0c31a0 2023-10-06T05:32:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.867+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1464015640 con 0x7f146c0c31a0 2023-10-06T05:32:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.868+0000 7f1453fff700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1458066fb0 0x7f1458069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.868+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1464084d50 con 0x7f146c0c31a0 2023-10-06T05:32:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.868+0000 7f146bfff700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1458066fb0 0x7f1458069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.869+0000 7f146bfff700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1458066fb0 0x7f1458069460 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f145c001690 tx=0x7f145c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:10.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:10.872+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14640520c0 con 0x7f146c0c31a0 2023-10-06T05:32:11.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.121+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f146c0cdcd0 con 0x7f146c0c31a0 2023-10-06T05:32:11.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.122+0000 7f1453fff700 1 -- 172.21.15.33:0/2811695448 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+3537 (secure 0 0 0) 0x7f14640556e0 con 0x7f146c0c31a0 2023-10-06T05:32:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:11.124 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":187,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3642585211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3642585211},{"type":"v1","addr":"172.21.15.33:6834","nonce":3642585211}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10129,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/630939549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":630939549},{"type":"v1","addr":"172.21.15.167:6836","nonce":630939549}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":10132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2160559859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2160559859},{"type":"v1","addr":"172.21.15.167:6837","nonce":2160559859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":179},{"gid":11415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/190642563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":190642563},{"type":"v1","addr":"172.21.15.33:6835","nonce":190642563}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[]} 2023-10-06T05:32:11.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1458066fb0 msgr2=0x7f1458069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:11.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1458066fb0 0x7f1458069460 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f145c001690 tx=0x7f145c015000 comp rx=0 tx=0).stop 2023-10-06T05:32:11.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 msgr2=0x7f146c0c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:11.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 0x7f146c0c3610 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f146c0b0f40 tx=0x7f146400b040 comp rx=0 tx=0).stop 2023-10-06T05:32:11.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 shutdown_connections 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1458066fb0 0x7f1458069460 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f146c0c31a0 0x7f146c0c3610 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1454003ab0 0x7f146c152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 --2- 172.21.15.33:0/2811695448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146c15b2b0 0x7f146c15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 >> 172.21.15.33:0/2811695448 conn(0x7f146c06c560 msgr2=0x7f146c0afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 shutdown_connections 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:11.124+0000 7f147139d700 1 -- 172.21.15.33:0/2811695448 wait complete. 2023-10-06T05:32:11.127 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 187 2023-10-06T05:32:11.143 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:32:11.144 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:32:11.144 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:32:11.144 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:32:11.144 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:32:11.144 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:32:11.145 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:32:11.145 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:32:11.145 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:32:11.145 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:32:11.145 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:32:11.146 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:32:11.146 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:32:11.146 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:32:11.146 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:32:11.146 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:32:11.160 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:32:11.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:32:11.163 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:32:11.163 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:32:11.164 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:32:11.164 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:32:11.205 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:32:11.205 DEBUG:teuthology.parallel:result is None 2023-10-06T05:32:11.208 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:32:11.208 DEBUG:teuthology.parallel:result is None 2023-10-06T05:32:11.211 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:32:11.211 DEBUG:teuthology.parallel:result is None 2023-10-06T05:32:11.336 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:32:11.348 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:32:11.348 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:32:15.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T05:32:17.160 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:32:17.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:32:17.166 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:32:17.166 DEBUG:teuthology.parallel:result is None 2023-10-06T05:32:17.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:32:17.309 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:32:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.366+0000 7f124d59d700 1 Processor -- start 2023-10-06T05:32:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.366+0000 7f124d59d700 1 -- start start 2023-10-06T05:32:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.367+0000 7f124d59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f12480c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.367+0000 7f124d59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 0x7f12480c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.367+0000 7f124d59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124806a820 con 0x7f1248156c50 2023-10-06T05:32:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.367+0000 7f124d59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12480cc290 con 0x7f1248159e10 2023-10-06T05:32:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.367+0000 7f124d59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124806bd10 con 0x7f1248156880 2023-10-06T05:32:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f12477fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 0x7f12480c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f12480c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f12480c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:38900/0 (socket says 172.21.15.33:38900) 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 -- 172.21.15.33:0/2596101067 learned_addr learned my addr 172.21.15.33:0/2596101067 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f12477fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 0x7f12480c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:51688/0 (socket says 172.21.15.33:51688) 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 -- 172.21.15.33:0/2596101067 >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 msgr2=0x7f12480c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 --2- 172.21.15.33:0/2596101067 >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 0x7f12480c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 -- 172.21.15.33:0/2596101067 >> v1:172.21.15.167:6789/0 conn(0x7f1248156c50 legacy=0x7f12481570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f1247fff700 1 -- 172.21.15.33:0/2596101067 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12481577c0 con 0x7f1248156880 2023-10-06T05:32:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.368+0000 7f12477fe700 1 --2- 172.21.15.33:0/2596101067 >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 0x7f12480c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.369+0000 7f1247fff700 1 --2- 172.21.15.33:0/2596101067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f12480c7980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12380099b0 tx=0x7f1238009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=829a21119f7bd511 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.369+0000 7f1246ffd700 1 -- 172.21.15.33:0/2596101067 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238009ea0 con 0x7f1248156880 2023-10-06T05:32:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.369+0000 7f1246ffd700 1 -- 172.21.15.33:0/2596101067 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123800b860 con 0x7f1248156880 2023-10-06T05:32:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.369+0000 7f1246ffd700 1 -- 172.21.15.33:0/2596101067 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123800bc00 con 0x7f1248156880 2023-10-06T05:32:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 -- 172.21.15.33:0/2596101067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 msgr2=0x7f12480c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 --2- 172.21.15.33:0/2596101067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f12480c7980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12380099b0 tx=0x7f1238009250 comp rx=0 tx=0).stop 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 -- 172.21.15.33:0/2596101067 shutdown_connections 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 --2- 172.21.15.33:0/2596101067 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f12480c7980 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 --2- 172.21.15.33:0/2596101067 >> v2:172.21.15.167:3300/0 conn(0x7f1248159e10 0x7f12480c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 -- 172.21.15.33:0/2596101067 >> 172.21.15.33:0/2596101067 conn(0x7f124806c560 msgr2=0x7f124806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 -- 172.21.15.33:0/2596101067 shutdown_connections 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.370+0000 7f124d59d700 1 -- 172.21.15.33:0/2596101067 wait complete. 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.371+0000 7f124d59d700 1 Processor -- start 2023-10-06T05:32:17.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.371+0000 7f124d59d700 1 -- start start 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.372+0000 7f124d59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f124814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.372+0000 7f124d59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248156c50 0x7f12480b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.372+0000 7f124d59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 0x7f12480c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.372+0000 7f124d59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12480afcd0 con 0x7f1248159e10 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.372+0000 7f124d59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12480af0c0 con 0x7f1248156c50 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.372+0000 7f124d59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12480af3a0 con 0x7f1248156880 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 0x7f12480c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f1247fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f124814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f12477fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248156c50 0x7f12480b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 0x7f12480c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52980/0 (socket says 172.21.15.33:52980) 2023-10-06T05:32:17.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 -- 172.21.15.33:0/1294782395 learned_addr learned my addr 172.21.15.33:0/1294782395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 -- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 msgr2=0x7f124814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f124814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 -- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248156c50 msgr2=0x7f12480b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248156c50 0x7f12480b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 -- 172.21.15.33:0/1294782395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c009420 con 0x7f1248159e10 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.373+0000 7f124cd9c700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 0x7f12480c9ac0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f124000c8e0 tx=0x7f1240001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.374+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240012630 con 0x7f1248159e10 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.374+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240012790 con 0x7f1248159e10 2023-10-06T05:32:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.374+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240012b50 con 0x7f1248159e10 2023-10-06T05:32:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.374+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12380094f0 con 0x7f1248159e10 2023-10-06T05:32:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.374+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12480af960 con 0x7f1248159e10 2023-10-06T05:32:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.375+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12480afe80 con 0x7f1248159e10 2023-10-06T05:32:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.376+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f124001a3a0 con 0x7f1248159e10 2023-10-06T05:32:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.377+0000 7f122ffff700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1234066f30 0x7f12340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.377+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f12400844d0 con 0x7f1248159e10 2023-10-06T05:32:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.377+0000 7f1247fff700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1234066f30 0x7f12340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.378+0000 7f1247fff700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1234066f30 0x7f12340693e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f12380099b0 tx=0x7f1238017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:17.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.381+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12480afe80 con 0x7f1248159e10 2023-10-06T05:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:17.597+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f124804c380 con 0x7f1248159e10 2023-10-06T05:32:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.121+0000 7f122ffff700 1 -- 172.21.15.33:0/1294782395 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v247) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1240055370 con 0x7f1248159e10 2023-10-06T05:32:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1234066f30 msgr2=0x7f12340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1234066f30 0x7f12340693e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f12380099b0 tx=0x7f1238017000 comp rx=0 tx=0).stop 2023-10-06T05:32:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 msgr2=0x7f12480c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 0x7f12480c9ac0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f124000c8e0 tx=0x7f1240001710 comp rx=0 tx=0).stop 2023-10-06T05:32:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 shutdown_connections 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1234066f30 0x7f12340693e0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248159e10 0x7f12480c9ac0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1248156880 0x7f124814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 --2- 172.21.15.33:0/1294782395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248156c50 0x7f12480b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 >> 172.21.15.33:0/1294782395 conn(0x7f124806c560 msgr2=0x7f12480c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 shutdown_connections 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.125+0000 7f124d59d700 1 -- 172.21.15.33:0/1294782395 wait complete. 2023-10-06T05:32:19.128 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:32:19.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:32:19.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.337+0000 7fd0df59e700 1 Processor -- start 2023-10-06T05:32:19.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.337+0000 7fd0df59e700 1 -- start start 2023-10-06T05:32:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0df59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0e0159530 0x7fd0e006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0df59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e006e3e0 con 0x7fd0e00b04e0 2023-10-06T05:32:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0df59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e006f220 con 0x7fd0e006df70 2023-10-06T05:32:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0df59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e00b1e00 con 0x7fd0e0159530 2023-10-06T05:32:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0de59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0e0159530 0x7fd0e006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0ded9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd0e00b04e0 0x7fd0e015a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41056/0 (socket says 172.21.15.33:41056) 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0ded9d700 1 -- 172.21.15.33:0/4128673545 learned_addr learned my addr 172.21.15.33:0/4128673545 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0dd59a700 1 -- 172.21.15.33:0/4128673545 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633744620 0 0) 0x7fd0e006e3e0 con 0x7fd0e00b04e0 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0dd59a700 1 -- 172.21.15.33:0/4128673545 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c8003570 con 0x7fd0e00b04e0 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0de59c700 1 -- 172.21.15.33:0/4128673545 >> v1:172.21.15.167:6789/0 conn(0x7fd0e006df70 legacy=0x7fd0e00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0de59c700 1 -- 172.21.15.33:0/4128673545 >> v1:172.21.15.33:6789/0 conn(0x7fd0e00b04e0 legacy=0x7fd0e015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.339+0000 7fd0de59c700 1 -- 172.21.15.33:0/4128673545 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e006e610 con 0x7fd0e0159530 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.340+0000 7fd0de59c700 1 --2- 172.21.15.33:0/4128673545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e0159530 0x7fd0e006da30 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00098e0 tx=0x7fd0d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec95ee261c380dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.340+0000 7fd0dd59a700 1 -- 172.21.15.33:0/4128673545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000b640 con 0x7fd0e0159530 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.340+0000 7fd0dd59a700 1 -- 172.21.15.33:0/4128673545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d000b7a0 con 0x7fd0e0159530 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.340+0000 7fd0dd59a700 1 -- 172.21.15.33:0/4128673545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000bb40 con 0x7fd0e0159530 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 -- 172.21.15.33:0/4128673545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e0159530 msgr2=0x7fd0e006da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4128673545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e0159530 0x7fd0e006da30 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00098e0 tx=0x7fd0d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 -- 172.21.15.33:0/4128673545 shutdown_connections 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4128673545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e0159530 0x7fd0e006da30 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 -- 172.21.15.33:0/4128673545 >> 172.21.15.33:0/4128673545 conn(0x7fd0e006c560 msgr2=0x7fd0e006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 -- 172.21.15.33:0/4128673545 shutdown_connections 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.341+0000 7fd0df59e700 1 -- 172.21.15.33:0/4128673545 wait complete. 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.342+0000 7fd0df59e700 1 Processor -- start 2023-10-06T05:32:19.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.342+0000 7fd0df59e700 1 -- start start 2023-10-06T05:32:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0df59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e006df70 0x7fd0e016a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0df59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 0x7fd0e015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0df59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0159530 0x7fd0e014cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0df59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0169fa0 con 0x7fd0e00b04e0 2023-10-06T05:32:19.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0df59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e016a970 con 0x7fd0e006df70 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0df59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e01699c0 con 0x7fd0e0159530 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.343+0000 7fd0ddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 0x7fd0e015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 0x7fd0e015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53000/0 (socket says 172.21.15.33:53000) 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 -- 172.21.15.33:0/4072007968 learned_addr learned my addr 172.21.15.33:0/4072007968 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0de59c700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e006df70 0x7fd0e016a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ded9d700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0159530 0x7fd0e014cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 -- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0159530 msgr2=0x7fd0e014cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0159530 0x7fd0e014cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 -- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e006df70 msgr2=0x7fd0e016a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e006df70 0x7fd0e016a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 -- 172.21.15.33:0/4072007968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0009420 con 0x7fd0e00b04e0 2023-10-06T05:32:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0de59c700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e006df70 0x7fd0e016a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ddd9b700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 0x7fd0e015e490 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c510 tx=0x7fd0d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011590 con 0x7fd0e00b04e0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d4011710 con 0x7fd0e00b04e0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.344+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011ad0 con 0x7fd0e00b04e0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.345+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e0155200 con 0x7fd0e00b04e0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.345+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e0169c30 con 0x7fd0e00b04e0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.346+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e016a110 con 0x7fd0e00b04e0 2023-10-06T05:32:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.346+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0d4001050 con 0x7fd0e00b04e0 2023-10-06T05:32:19.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.347+0000 7fd0ce7fc700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0c0066f30 0x7fd0c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.347+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd0d4083850 con 0x7fd0e00b04e0 2023-10-06T05:32:19.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.348+0000 7fd0de59c700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0c0066f30 0x7fd0c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.349+0000 7fd0de59c700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0c0066f30 0x7fd0c00693e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00093f0 tx=0x7fd0d0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.352+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0d4050c10 con 0x7fd0e00b04e0 2023-10-06T05:32:19.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.635+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd0e004c380 con 0x7fd0e00b04e0 2023-10-06T05:32:19.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.635+0000 7fd0ce7fc700 1 -- 172.21.15.33:0/4072007968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd0d4017020 con 0x7fd0e00b04e0 2023-10-06T05:32:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:19.637 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:32:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:32:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:32:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:32:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:32:19.653 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:32:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0c0066f30 msgr2=0x7fd0c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0c0066f30 0x7fd0c00693e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00093f0 tx=0x7fd0d0017000 comp rx=0 tx=0).stop 2023-10-06T05:32:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 msgr2=0x7fd0e015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 0x7fd0e015e490 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c510 tx=0x7fd0d400c8d0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 shutdown_connections 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0c0066f30 0x7fd0c00693e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0159530 0x7fd0e014cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e006df70 0x7fd0e016a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 --2- 172.21.15.33:0/4072007968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0e00b04e0 0x7fd0e015e490 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 >> 172.21.15.33:0/4072007968 conn(0x7fd0e006c560 msgr2=0x7fd0e00c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 shutdown_connections 2023-10-06T05:32:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.638+0000 7fd0df59e700 1 -- 172.21.15.33:0/4072007968 wait complete. 2023-10-06T05:32:19.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:32:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.843+0000 7f669359e700 1 Processor -- start 2023-10-06T05:32:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.843+0000 7f669359e700 1 -- start start 2023-10-06T05:32:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.844+0000 7f669359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66941431b0 0x7f66940afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.844+0000 7f669359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 0x7f66940b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.844+0000 7f669359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669415dea0 con 0x7f6694142de0 2023-10-06T05:32:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.844+0000 7f669359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66940b07c0 con 0x7f66941431b0 2023-10-06T05:32:19.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.844+0000 7f669359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669415d290 con 0x7f66940b19f0 2023-10-06T05:32:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6691d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66941431b0 0x7f66940afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6692d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 0x7f66940b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6692d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 0x7f66940b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:38918/0 (socket says 172.21.15.33:38918) 2023-10-06T05:32:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6692d9d700 1 -- 172.21.15.33:0/2467967678 learned_addr learned my addr 172.21.15.33:0/2467967678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:19.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6691d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f66941431b0 0x7f66940afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:51718/0 (socket says 172.21.15.33:51718) 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6691d9b700 1 -- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 msgr2=0x7f66940b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6691d9b700 1 --2- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 0x7f66940b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6691d9b700 1 -- 172.21.15.33:0/2467967678 >> v1:172.21.15.167:6789/0 conn(0x7f6694142de0 legacy=0x7f66940b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6691d9b700 1 -- 172.21.15.33:0/2467967678 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669415d570 con 0x7f66941431b0 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.845+0000 7f6692d9d700 1 --2- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 0x7f66940b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f6691d9b700 1 --2- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66941431b0 0x7f66940afd40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f66880098e0 tx=0x7f66880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b9b17b6d8130ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669159a700 1 -- 172.21.15.33:0/2467967678 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800b640 con 0x7f66941431b0 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669159a700 1 -- 172.21.15.33:0/2467967678 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800b7a0 con 0x7f66941431b0 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669159a700 1 -- 172.21.15.33:0/2467967678 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800bb40 con 0x7f66941431b0 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669359e700 1 -- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66941431b0 msgr2=0x7f66940afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669359e700 1 --2- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66941431b0 0x7f66940afd40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f66880098e0 tx=0x7f66880091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669359e700 1 -- 172.21.15.33:0/2467967678 shutdown_connections 2023-10-06T05:32:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669359e700 1 --2- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66940b19f0 0x7f66940b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669359e700 1 --2- 172.21.15.33:0/2467967678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66941431b0 0x7f66940afd40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.846+0000 7f669359e700 1 -- 172.21.15.33:0/2467967678 >> 172.21.15.33:0/2467967678 conn(0x7f669406c560 msgr2=0x7f669406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.847+0000 7f669359e700 1 -- 172.21.15.33:0/2467967678 shutdown_connections 2023-10-06T05:32:19.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.847+0000 7f669359e700 1 -- 172.21.15.33:0/2467967678 wait complete. 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.848+0000 7f669359e700 1 Processor -- start 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.848+0000 7f669359e700 1 -- start start 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 0x7f6694158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694142de0 0x7f66940cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66941431b0 0x7f6694140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66940c7520 con 0x7f66940b19f0 2023-10-06T05:32:19.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66940c7800 con 0x7f6694142de0 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940c7ae0 con 0x7f66941431b0 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 0x7f6694158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 0x7f6694158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53010/0 (socket says 172.21.15.33:53010) 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f669259c700 1 -- 172.21.15.33:0/2079808034 learned_addr learned my addr 172.21.15.33:0/2079808034 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f6691d9b700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694142de0 0x7f66940cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.849+0000 7f6692d9d700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66941431b0 0x7f6694140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669259c700 1 -- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66941431b0 msgr2=0x7f6694140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669259c700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66941431b0 0x7f6694140470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669259c700 1 -- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694142de0 msgr2=0x7f66940cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669259c700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694142de0 0x7f66940cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:19.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669259c700 1 -- 172.21.15.33:0/2079808034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c009ae0 con 0x7f66940b19f0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669259c700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 0x7f6694158e10 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f668400c590 tx=0x7f668400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684011590 con 0x7f66940b19f0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6684011710 con 0x7f66940b19f0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6688009420 con 0x7f66940b19f0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.850+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684011ad0 con 0x7f66940b19f0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.851+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66940c80a0 con 0x7f66940b19f0 2023-10-06T05:32:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.851+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669404c380 con 0x7f66940b19f0 2023-10-06T05:32:19.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.852+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f668401b3b0 con 0x7f66940b19f0 2023-10-06T05:32:19.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.853+0000 7f66827fc700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6674066f30 0x7f66740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:19.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.854+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6684083850 con 0x7f66940b19f0 2023-10-06T05:32:19.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.854+0000 7f6691d9b700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6674066f30 0x7f66740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:19.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.855+0000 7f6691d9b700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6674066f30 0x7f66740693e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f66940719f0 tx=0x7f6688017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:19.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:19.857+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6684050c10 con 0x7f66940b19f0 2023-10-06T05:32:20.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.061+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66940ce140 con 0x7f66940b19f0 2023-10-06T05:32:20.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.062+0000 7f66827fc700 1 -- 172.21.15.33:0/2079808034 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v247) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f6684017020 con 0x7f66940b19f0 2023-10-06T05:32:20.064 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:20.064 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":247,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:32:19.118339+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.064+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6674066f30 msgr2=0x7f66740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.064+0000 7f669359e700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6674066f30 0x7f66740693e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f66940719f0 tx=0x7f6688017000 comp rx=0 tx=0).stop 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 msgr2=0x7f6694158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 0x7f6694158e10 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f668400c590 tx=0x7f668400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 shutdown_connections 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6674066f30 0x7f66740693e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66941431b0 0x7f6694140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66940b19f0 0x7f6694158e10 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 --2- 172.21.15.33:0/2079808034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6694142de0 0x7f66940cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 >> 172.21.15.33:0/2079808034 conn(0x7f669406c560 msgr2=0x7f669413fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 shutdown_connections 2023-10-06T05:32:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.065+0000 7f669359e700 1 -- 172.21.15.33:0/2079808034 wait complete. 2023-10-06T05:32:20.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:32:20.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:32:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.275+0000 7f3322d9d700 1 Processor -- start 2023-10-06T05:32:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.276+0000 7f3322d9d700 1 -- start start 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3322d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f331c156370 0x7f331c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3322d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f331c0cc7e0 0x7f331c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3322d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c0ad430 con 0x7f331c0b01f0 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3322d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c0afe20 con 0x7f331c156370 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3322d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c06bc70 con 0x7f331c0cc7e0 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3321d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f331c156370 0x7f331c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3321d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f331c156370 0x7f331c0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36756/0 (socket says 172.21.15.33:36756) 2023-10-06T05:32:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f3321d9b700 1 -- 172.21.15.33:0/2185320916 learned_addr learned my addr 172.21.15.33:0/2185320916 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:20.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.277+0000 7f332159a700 1 --2- 172.21.15.33:0/2185320916 >> v2:172.21.15.167:3300/0 conn(0x7f331c0cc7e0 0x7f331c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3321d9b700 1 -- 172.21.15.33:0/2185320916 >> v2:172.21.15.167:3300/0 conn(0x7f331c0cc7e0 msgr2=0x7f331c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3321d9b700 1 --2- 172.21.15.33:0/2185320916 >> v2:172.21.15.167:3300/0 conn(0x7f331c0cc7e0 0x7f331c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3321d9b700 1 -- 172.21.15.33:0/2185320916 >> v1:172.21.15.167:6789/0 conn(0x7f331c0b01f0 legacy=0x7f331c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3321d9b700 1 -- 172.21.15.33:0/2185320916 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0c92f0 con 0x7f331c156370 2023-10-06T05:32:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3321d9b700 1 --2- 172.21.15.33:0/2185320916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c156370 0x7f331c0af8e0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ad3a0 tx=0x7f3318009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505d9bc9a1c87dc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3320d99700 1 -- 172.21.15.33:0/2185320916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318009e70 con 0x7f331c156370 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3320d99700 1 -- 172.21.15.33:0/2185320916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331800b860 con 0x7f331c156370 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.278+0000 7f3320d99700 1 -- 172.21.15.33:0/2185320916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800bc00 con 0x7f331c156370 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 -- 172.21.15.33:0/2185320916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c156370 msgr2=0x7f331c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2185320916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c156370 0x7f331c0af8e0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ad3a0 tx=0x7f3318009250 comp rx=0 tx=0).stop 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 -- 172.21.15.33:0/2185320916 shutdown_connections 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2185320916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c156370 0x7f331c0af8e0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2185320916 >> v2:172.21.15.167:3300/0 conn(0x7f331c0cc7e0 0x7f331c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 -- 172.21.15.33:0/2185320916 >> 172.21.15.33:0/2185320916 conn(0x7f331c06d030 msgr2=0x7f331c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 -- 172.21.15.33:0/2185320916 shutdown_connections 2023-10-06T05:32:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.279+0000 7f3322d9d700 1 -- 172.21.15.33:0/2185320916 wait complete. 2023-10-06T05:32:20.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.280+0000 7f3322d9d700 1 Processor -- start 2023-10-06T05:32:20.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 -- start start 2023-10-06T05:32:20.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 0x7f331c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f331c0cc7e0 0x7f331c139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f331c156370 0x7f331c0a8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c14e840 con 0x7f331c0b01f0 2023-10-06T05:32:20.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c14eae0 con 0x7f331c156370 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.281+0000 7f3322d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c14edc0 con 0x7f331c0cc7e0 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 0x7f331c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 0x7f331c14e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36760/0 (socket says 172.21.15.33:36760) 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 -- 172.21.15.33:0/2791928917 learned_addr learned my addr 172.21.15.33:0/2791928917 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f332159a700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f331c0cc7e0 0x7f331c139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f332259c700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f331c156370 0x7f331c0a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 -- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f331c0cc7e0 msgr2=0x7f331c139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f331c0cc7e0 0x7f331c139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 -- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f331c156370 msgr2=0x7f331c0a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f331c156370 0x7f331c0a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 -- 172.21.15.33:0/2791928917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180094f0 con 0x7f331c0b01f0 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.282+0000 7f3321d9b700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 0x7f331c14e300 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f331c06d4d0 tx=0x7f3318017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.283+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318011420 con 0x7f331c0b01f0 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.283+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3318011580 con 0x7f331c0b01f0 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.283+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318011940 con 0x7f331c0b01f0 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.283+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c14f0a0 con 0x7f331c0b01f0 2023-10-06T05:32:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.283+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c14f660 con 0x7f331c0b01f0 2023-10-06T05:32:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.284+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c04c380 con 0x7f331c0b01f0 2023-10-06T05:32:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.285+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f331802c3b0 con 0x7f331c0b01f0 2023-10-06T05:32:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.285+0000 7f3311ffb700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3300066f30 0x7f33000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.285+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f33180859b0 con 0x7f331c0b01f0 2023-10-06T05:32:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.286+0000 7f332159a700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3300066f30 0x7f33000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.287+0000 7f332159a700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3300066f30 0x7f33000693e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f331c0c85a0 tx=0x7f330c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:20.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.290+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3318052d70 con 0x7f331c0b01f0 2023-10-06T05:32:20.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.490+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f331c06f310 con 0x7f331c0b01f0 2023-10-06T05:32:20.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.491+0000 7f3311ffb700 1 -- 172.21.15.33:0/2791928917 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v247) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3318028020 con 0x7f331c0b01f0 2023-10-06T05:32:20.493 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:32:20.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3300066f30 msgr2=0x7f33000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3300066f30 0x7f33000693e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f331c0c85a0 tx=0x7f330c008040 comp rx=0 tx=0).stop 2023-10-06T05:32:20.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 msgr2=0x7f331c14e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 0x7f331c14e300 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f331c06d4d0 tx=0x7f3318017000 comp rx=0 tx=0).stop 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 shutdown_connections 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3300066f30 0x7f33000693e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f331c156370 0x7f331c0a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f331c0b01f0 0x7f331c14e300 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 --2- 172.21.15.33:0/2791928917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f331c0cc7e0 0x7f331c139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.493+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 >> 172.21.15.33:0/2791928917 conn(0x7f331c06d030 msgr2=0x7f331c0af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.494+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 shutdown_connections 2023-10-06T05:32:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.494+0000 7f3322d9d700 1 -- 172.21.15.33:0/2791928917 wait complete. 2023-10-06T05:32:20.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:32:20.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.701+0000 7f007d39d700 1 Processor -- start 2023-10-06T05:32:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.701+0000 7f007d39d700 1 -- start start 2023-10-06T05:32:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.702+0000 7f007d39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0078154f20 0x7f00780ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.702+0000 7f007d39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00781586a0 con 0x7f00781552f0 2023-10-06T05:32:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.702+0000 7f007d39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0078157ab0 con 0x7f00780b18f0 2023-10-06T05:32:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.702+0000 7f007d39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007806bdf0 con 0x7f0078154f20 2023-10-06T05:32:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.702+0000 7f00777fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f00781552f0 0x7f00780b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42992/0 (socket says 172.21.15.33:42992) 2023-10-06T05:32:20.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.702+0000 7f00777fe700 1 -- 172.21.15.33:0/143578083 learned_addr learned my addr 172.21.15.33:0/143578083 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0077fff700 1 --2- 172.21.15.33:0/143578083 >> v2:172.21.15.167:3300/0 conn(0x7f0078154f20 0x7f00780ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064952637 0 0) 0x7f00781586a0 con 0x7f00781552f0 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0060003570 con 0x7f00781552f0 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 258994961 0 0) 0x7f0060003570 con 0x7f00781552f0 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 >> v2:172.21.15.167:3300/0 conn(0x7f0078154f20 msgr2=0x7f00780ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 --2- 172.21.15.33:0/143578083 >> v2:172.21.15.167:3300/0 conn(0x7f0078154f20 0x7f00780ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 >> v1:172.21.15.167:6789/0 conn(0x7f00780b18f0 legacy=0x7f00780c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0078157d70 con 0x7f00781552f0 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f006c001470 con 0x7f00781552f0 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 >> v1:172.21.15.33:6789/0 conn(0x7f00781552f0 legacy=0x7f00780b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0060003a60 0x7f0060003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0060004810 0x7f0060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00600051c0 0x7f006000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007806bdf0 con 0x7f0060004810 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006000c160 con 0x7f00600051c0 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.703+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006000c320 con 0x7f0060003a60 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0060004810 0x7f0060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f007cb9c700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00600051c0 0x7f006000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f0077fff700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0060003a60 0x7f0060003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 -- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0060003a60 msgr2=0x7f0060003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0060003a60 0x7f0060003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 -- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00600051c0 msgr2=0x7f006000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00600051c0 0x7f006000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 -- 172.21.15.33:0/143578083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060003570 con 0x7f0060004810 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f007cb9c700 1 -- 172.21.15.33:0/143578083 reap_dead start 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f0077fff700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0060003a60 0x7f0060003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.704+0000 7f00777fe700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0060004810 0x7f0060004c80 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c3c0 tx=0x7f006c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbffda5cc4da2fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00cc90 con 0x7f0060004810 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c00cdf0 con 0x7f0060004810 2023-10-06T05:32:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f0076ffd700 1 -- 172.21.15.33:0/143578083 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c005b50 con 0x7f0060004810 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 -- 172.21.15.33:0/143578083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0060004810 msgr2=0x7f0060004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0060004810 0x7f0060004c80 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c3c0 tx=0x7f006c001e50 comp rx=0 tx=0).stop 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 -- 172.21.15.33:0/143578083 shutdown_connections 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 --2- 172.21.15.33:0/143578083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0060004810 0x7f0060004c80 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 -- 172.21.15.33:0/143578083 >> 172.21.15.33:0/143578083 conn(0x7f007806d030 msgr2=0x7f00780cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 -- 172.21.15.33:0/143578083 shutdown_connections 2023-10-06T05:32:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.705+0000 7f007d39d700 1 -- 172.21.15.33:0/143578083 wait complete. 2023-10-06T05:32:20.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.707+0000 7f007d39d700 1 Processor -- start 2023-10-06T05:32:20.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.707+0000 7f007d39d700 1 -- start start 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007d39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060004810 0x7f00780b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007d39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 0x7f0078156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007d39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815c6b0 0x7f0078157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007d39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780aadd0 con 0x7f007815c240 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007d39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780afb10 con 0x7f0060004810 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007d39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780afdf0 con 0x7f007815c6b0 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 0x7f0078156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 0x7f0078156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36786/0 (socket says 172.21.15.33:36786) 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 -- 172.21.15.33:0/214915660 learned_addr learned my addr 172.21.15.33:0/214915660 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f0077fff700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060004810 0x7f00780b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007cb9c700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815c6b0 0x7f0078157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 -- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815c6b0 msgr2=0x7f0078157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815c6b0 0x7f0078157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 -- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060004810 msgr2=0x7f00780b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060004810 0x7f00780b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f00777fe700 1 -- 172.21.15.33:0/214915660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780b00d0 con 0x7f007815c240 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.708+0000 7f007cb9c700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815c6b0 0x7f0078157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.709+0000 7f00777fe700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 0x7f0078156fc0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f006000c130 tx=0x7f006c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.709+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00c9e0 con 0x7f007815c240 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.709+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c001430 con 0x7f007815c240 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.709+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780b03b0 con 0x7f007815c240 2023-10-06T05:32:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.709+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c007300 con 0x7f007815c240 2023-10-06T05:32:20.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.709+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0078153260 con 0x7f007815c240 2023-10-06T05:32:20.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.710+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00780b0850 con 0x7f007815c240 2023-10-06T05:32:20.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.711+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f006c0076f0 con 0x7f007815c240 2023-10-06T05:32:20.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.712+0000 7f005ffff700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:20.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.712+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f006c085340 con 0x7f007815c240 2023-10-06T05:32:20.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.712+0000 7f0077fff700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:20.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.713+0000 7f0077fff700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f007804cc00 tx=0x7f0068008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:20.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.716+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f006c0527b0 con 0x7f007815c240 2023-10-06T05:32:20.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.918+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f007804c380 con 0x7f007815c240 2023-10-06T05:32:20.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.918+0000 7f005ffff700 1 -- 172.21.15.33:0/214915660 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v247) v1 ==== 92+0+10 (secure 0 0 0) 0x7f006c017020 con 0x7f007815c240 2023-10-06T05:32:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:32:20.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 msgr2=0x7f00640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f007804cc00 tx=0x7f0068008040 comp rx=0 tx=0).stop 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 msgr2=0x7f0078156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 0x7f0078156fc0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f006000c130 tx=0x7f006c015000 comp rx=0 tx=0).stop 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 shutdown_connections 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0064066f30 0x7f00640693e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815c6b0 0x7f0078157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060004810 0x7f00780b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 --2- 172.21.15.33:0/214915660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007815c240 0x7f0078156fc0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 >> 172.21.15.33:0/214915660 conn(0x7f007806d030 msgr2=0x7f0078153f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:20.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 shutdown_connections 2023-10-06T05:32:20.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:20.921+0000 7f007d39d700 1 -- 172.21.15.33:0/214915660 wait complete. 2023-10-06T05:32:20.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:32:21.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.124+0000 7fbda6b9d700 1 Processor -- start 2023-10-06T05:32:21.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.124+0000 7fbda6b9d700 1 -- start start 2023-10-06T05:32:21.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda6b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbda0155af0 0x7fbda006a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda6b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda006ab60 con 0x7fbda006a6a0 2023-10-06T05:32:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda6b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda006ae40 con 0x7fbda00c7920 2023-10-06T05:32:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda6b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda006b7e0 con 0x7fbda0155af0 2023-10-06T05:32:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda5b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbda0155af0 0x7fbda006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda539a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbda006a6a0 0x7fbda00c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42996/0 (socket says 172.21.15.33:42996) 2023-10-06T05:32:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda539a700 1 -- 172.21.15.33:0/1760185998 learned_addr learned my addr 172.21.15.33:0/1760185998 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda5b9b700 1 -- 172.21.15.33:0/1760185998 >> v1:172.21.15.167:6789/0 conn(0x7fbda00c7920 legacy=0x7fbda00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda5b9b700 1 -- 172.21.15.33:0/1760185998 >> v1:172.21.15.33:6789/0 conn(0x7fbda006a6a0 legacy=0x7fbda00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda5b9b700 1 -- 172.21.15.33:0/1760185998 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda006b120 con 0x7fbda0155af0 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.126+0000 7fbda4b99700 1 -- 172.21.15.33:0/1760185998 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681953727 0 0) 0x7fbda006ab60 con 0x7fbda006a6a0 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.127+0000 7fbda5b9b700 1 --2- 172.21.15.33:0/1760185998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda0155af0 0x7fbda006a160 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0098e0 tx=0x7fbd9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653bbdc4d5621862 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.127+0000 7fbda4b99700 1 -- 172.21.15.33:0/1760185998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c009e80 con 0x7fbda0155af0 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.127+0000 7fbda4b99700 1 -- 172.21.15.33:0/1760185998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00b820 con 0x7fbda0155af0 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.127+0000 7fbda4b99700 1 -- 172.21.15.33:0/1760185998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00bba0 con 0x7fbda0155af0 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.127+0000 7fbda6b9d700 1 -- 172.21.15.33:0/1760185998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda0155af0 msgr2=0x7fbda006a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.127+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/1760185998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda0155af0 0x7fbda006a160 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0098e0 tx=0x7fbd9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.128+0000 7fbda6b9d700 1 -- 172.21.15.33:0/1760185998 shutdown_connections 2023-10-06T05:32:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.128+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/1760185998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda0155af0 0x7fbda006a160 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.128+0000 7fbda6b9d700 1 -- 172.21.15.33:0/1760185998 >> 172.21.15.33:0/1760185998 conn(0x7fbda006d030 msgr2=0x7fbda006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.128+0000 7fbda6b9d700 1 -- 172.21.15.33:0/1760185998 shutdown_connections 2023-10-06T05:32:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.128+0000 7fbda6b9d700 1 -- 172.21.15.33:0/1760185998 wait complete. 2023-10-06T05:32:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.129+0000 7fbda6b9d700 1 Processor -- start 2023-10-06T05:32:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.129+0000 7fbda6b9d700 1 -- start start 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda6b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 0x7fbda00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda6b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda00c7920 0x7fbda00ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda6b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda0155af0 0x7fbda01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda6b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda016dab0 con 0x7fbda006a6a0 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda6b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00b0e40 con 0x7fbda00c7920 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda6b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda016d460 con 0x7fbda0155af0 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 0x7fbda00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda5b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 0x7fbda00b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36806/0 (socket says 172.21.15.33:36806) 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.130+0000 7fbda5b9b700 1 -- 172.21.15.33:0/3930577049 learned_addr learned my addr 172.21.15.33:0/3930577049 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda539a700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda00c7920 0x7fbda00ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda639c700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda0155af0 0x7fbda01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda5b9b700 1 -- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda0155af0 msgr2=0x7fbda01539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda5b9b700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda0155af0 0x7fbda01539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda5b9b700 1 -- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda00c7920 msgr2=0x7fbda00ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda5b9b700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda00c7920 0x7fbda00ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda5b9b700 1 -- 172.21.15.33:0/3930577049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009420 con 0x7fbda006a6a0 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda539a700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda00c7920 0x7fbda00ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda639c700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda0155af0 0x7fbda01539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbda5b9b700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 0x7fbda00b04d0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0030c0 tx=0x7fbd9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c01b470 con 0x7fbda006a6a0 2023-10-06T05:32:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.131+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c01b5d0 con 0x7fbda006a6a0 2023-10-06T05:32:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.132+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c01b990 con 0x7fbda006a6a0 2023-10-06T05:32:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.132+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda016d740 con 0x7fbda006a6a0 2023-10-06T05:32:21.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.132+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda00ae6f0 con 0x7fbda006a6a0 2023-10-06T05:32:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.133+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda016d8d0 con 0x7fbda006a6a0 2023-10-06T05:32:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.133+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd9c01c450 con 0x7fbda006a6a0 2023-10-06T05:32:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.135+0000 7fbd95ffb700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd84066f30 0x7fbd840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.135+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbd9c0862c0 con 0x7fbda006a6a0 2023-10-06T05:32:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.135+0000 7fbda539a700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd84066f30 0x7fbd840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.136+0000 7fbda539a700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd84066f30 0x7fbd840693e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbda00c56b0 tx=0x7fbd90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:21.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.139+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd9c053680 con 0x7fbda006a6a0 2023-10-06T05:32:21.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:32:21.153 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:32:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.346+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbda00ae8e0 con 0x7fbda006a6a0 2023-10-06T05:32:21.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.346+0000 7fbd95ffb700 1 -- 172.21.15.33:0/3930577049 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v247) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbd9c02a090 con 0x7fbda006a6a0 2023-10-06T05:32:21.348 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:21.348 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:32:21.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:21.348 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:32:21.348 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:32:21.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:32:21.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:32:21.349 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:32:21.349 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:32:21.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:32:21.349 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:32:21.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.348+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd84066f30 msgr2=0x7fbd840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.348+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd84066f30 0x7fbd840693e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbda00c56b0 tx=0x7fbd90009040 comp rx=0 tx=0).stop 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.348+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 msgr2=0x7fbda00b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.348+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 0x7fbda00b04d0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0030c0 tx=0x7fbd9c016000 comp rx=0 tx=0).stop 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 shutdown_connections 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd84066f30 0x7fbd840693e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbda0155af0 0x7fbda01539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbda006a6a0 0x7fbda00b04d0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 --2- 172.21.15.33:0/3930577049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda00c7920 0x7fbda00ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 >> 172.21.15.33:0/3930577049 conn(0x7fbda006d030 msgr2=0x7fbda00c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 shutdown_connections 2023-10-06T05:32:21.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.349+0000 7fbda6b9d700 1 -- 172.21.15.33:0/3930577049 wait complete. 2023-10-06T05:32:21.367 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:32:21.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:32:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.557+0000 7f0d75b9d700 1 Processor -- start 2023-10-06T05:32:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.557+0000 7f0d75b9d700 1 -- start start 2023-10-06T05:32:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.558+0000 7f0d75b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d700c9260 0x7f0d700abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.558+0000 7f0d75b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700aa040 con 0x7f0d7015cb80 2023-10-06T05:32:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d75b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700a9a30 con 0x7f0d7006eae0 2023-10-06T05:32:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d75b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7006aa90 con 0x7f0d700c9260 2023-10-06T05:32:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d7539c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d700c9260 0x7f0d700abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d74b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0d7015cb80 0x7f0d700c8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43000/0 (socket says 172.21.15.33:43000) 2023-10-06T05:32:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d74b9b700 1 -- 172.21.15.33:0/1107852042 learned_addr learned my addr 172.21.15.33:0/1107852042 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d6f7fe700 1 -- 172.21.15.33:0/1107852042 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613609036 0 0) 0x7f0d700aa040 con 0x7f0d7015cb80 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d6f7fe700 1 -- 172.21.15.33:0/1107852042 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d60003570 con 0x7f0d7015cb80 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d7539c700 1 -- 172.21.15.33:0/1107852042 >> v1:172.21.15.167:6789/0 conn(0x7f0d7006eae0 legacy=0x7f0d700aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d7539c700 1 -- 172.21.15.33:0/1107852042 >> v1:172.21.15.33:6789/0 conn(0x7f0d7015cb80 legacy=0x7f0d700c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.559+0000 7f0d7539c700 1 -- 172.21.15.33:0/1107852042 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7006a7b0 con 0x7f0d700c9260 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.560+0000 7f0d7539c700 1 --2- 172.21.15.33:0/1107852042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700c9260 0x7f0d700abc10 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009e60 tx=0x7f0d68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f2c950ed3505aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.560+0000 7f0d6f7fe700 1 -- 172.21.15.33:0/1107852042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800e070 con 0x7f0d700c9260 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.560+0000 7f0d6f7fe700 1 -- 172.21.15.33:0/1107852042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6800bb60 con 0x7f0d700c9260 2023-10-06T05:32:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.560+0000 7f0d6f7fe700 1 -- 172.21.15.33:0/1107852042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68012430 con 0x7f0d700c9260 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.560+0000 7f0d75b9d700 1 -- 172.21.15.33:0/1107852042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700c9260 msgr2=0x7f0d700abc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.560+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/1107852042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700c9260 0x7f0d700abc10 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009e60 tx=0x7f0d68009780 comp rx=0 tx=0).stop 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.561+0000 7f0d75b9d700 1 -- 172.21.15.33:0/1107852042 shutdown_connections 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.561+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/1107852042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700c9260 0x7f0d700abc10 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.561+0000 7f0d75b9d700 1 -- 172.21.15.33:0/1107852042 >> 172.21.15.33:0/1107852042 conn(0x7f0d70068090 msgr2=0x7f0d7006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.561+0000 7f0d75b9d700 1 -- 172.21.15.33:0/1107852042 shutdown_connections 2023-10-06T05:32:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.561+0000 7f0d75b9d700 1 -- 172.21.15.33:0/1107852042 wait complete. 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.562+0000 7f0d75b9d700 1 Processor -- start 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.562+0000 7f0d75b9d700 1 -- start start 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.563+0000 7f0d75b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d7006eae0 0x7f0d7016a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.563+0000 7f0d75b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d700c9260 0x7f0d70168f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.563+0000 7f0d75b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 0x7f0d7016a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.563+0000 7f0d75b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70157870 con 0x7f0d7015cb80 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.563+0000 7f0d75b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d701583f0 con 0x7f0d700c9260 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.563+0000 7f0d75b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70154f40 con 0x7f0d7006eae0 2023-10-06T05:32:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 0x7f0d7016a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d700c9260 0x7f0d70168f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 0x7f0d7016a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36820/0 (socket says 172.21.15.33:36820) 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 -- 172.21.15.33:0/3497445215 learned_addr learned my addr 172.21.15.33:0/3497445215 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d74b9b700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d7006eae0 0x7f0d7016a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 -- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d7006eae0 msgr2=0x7f0d7016a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d7006eae0 0x7f0d7016a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 -- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d700c9260 msgr2=0x7f0d70168f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d700c9260 0x7f0d70168f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 -- 172.21.15.33:0/3497445215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d68009a20 con 0x7f0d7015cb80 2023-10-06T05:32:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.564+0000 7f0d7539c700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 0x7f0d7016a890 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f0d7006e190 tx=0x7f0d68019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.565+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68027470 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.565+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d680275d0 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.565+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68027990 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.565+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d70157ac0 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.565+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d70158080 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.566+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d70063ff0 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.567+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d68024070 con 0x7f0d7015cb80 2023-10-06T05:32:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.567+0000 7f0d6cff9700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d54066f30 0x7f0d540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.567+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0d68086d70 con 0x7f0d7015cb80 2023-10-06T05:32:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.568+0000 7f0d74b9b700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d54066f30 0x7f0d540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.569+0000 7f0d74b9b700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d54066f30 0x7f0d540693e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0d64007fd0 tx=0x7f0d64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.572+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d68054130 con 0x7f0d7015cb80 2023-10-06T05:32:21.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:21.774+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0d70158530 con 0x7f0d7015cb80 2023-10-06T05:32:22.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.152+0000 7f0d6cff9700 1 -- 172.21.15.33:0/3497445215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v248) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0d6802c0b0 con 0x7f0d7015cb80 2023-10-06T05:32:22.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.157+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d54066f30 msgr2=0x7f0d540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:22.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.157+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d54066f30 0x7f0d540693e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0d64007fd0 tx=0x7f0d64009040 comp rx=0 tx=0).stop 2023-10-06T05:32:22.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.157+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 msgr2=0x7f0d7016a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:22.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.157+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 0x7f0d7016a890 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f0d7006e190 tx=0x7f0d68019000 comp rx=0 tx=0).stop 2023-10-06T05:32:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.158+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 shutdown_connections 2023-10-06T05:32:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.158+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0d54066f30 0x7f0d540693e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.158+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d7015cb80 0x7f0d7016a890 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.158+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d7006eae0 0x7f0d7016a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.158+0000 7f0d75b9d700 1 --2- 172.21.15.33:0/3497445215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d700c9260 0x7f0d70168f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.158+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 >> 172.21.15.33:0/3497445215 conn(0x7f0d70068090 msgr2=0x7f0d70159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.159+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 shutdown_connections 2023-10-06T05:32:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.160+0000 7f0d75b9d700 1 -- 172.21.15.33:0/3497445215 wait complete. 2023-10-06T05:32:22.163 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:32:22.180 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:32:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.373+0000 7f72df59e700 1 Processor -- start 2023-10-06T05:32:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.373+0000 7f72df59e700 1 -- start start 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72df59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72e00af540 0x7f72e00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72df59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e015ddb0 con 0x7f72e015a7d0 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72df59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e015d1a0 con 0x7f72e00cced0 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72df59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e015d480 con 0x7f72e00af540 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72de59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72e00af540 0x7f72e00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72ddd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f72e015a7d0 0x7f72e00cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43006/0 (socket says 172.21.15.33:43006) 2023-10-06T05:32:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.374+0000 7f72ddd9b700 1 -- 172.21.15.33:0/2382286398 learned_addr learned my addr 172.21.15.33:0/2382286398 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.375+0000 7f72de59c700 1 -- 172.21.15.33:0/2382286398 >> v1:172.21.15.167:6789/0 conn(0x7f72e00cced0 legacy=0x7f72e00ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.375+0000 7f72de59c700 1 -- 172.21.15.33:0/2382286398 >> v1:172.21.15.33:6789/0 conn(0x7f72e015a7d0 legacy=0x7f72e00cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.375+0000 7f72de59c700 1 -- 172.21.15.33:0/2382286398 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e015d760 con 0x7f72e00af540 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.375+0000 7f72dd59a700 1 -- 172.21.15.33:0/2382286398 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884381189 0 0) 0x7f72e015ddb0 con 0x7f72e015a7d0 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.375+0000 7f72de59c700 1 --2- 172.21.15.33:0/2382286398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e00af910 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f72d00098e0 tx=0x7f72d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d71b35571b4c9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.375+0000 7f72dd59a700 1 -- 172.21.15.33:0/2382286398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0009e80 con 0x7f72e00af540 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72dd59a700 1 -- 172.21.15.33:0/2382286398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000b820 con 0x7f72e00af540 2023-10-06T05:32:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72dd59a700 1 -- 172.21.15.33:0/2382286398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000bba0 con 0x7f72e00af540 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 -- 172.21.15.33:0/2382286398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 msgr2=0x7f72e00af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 --2- 172.21.15.33:0/2382286398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e00af910 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f72d00098e0 tx=0x7f72d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 -- 172.21.15.33:0/2382286398 shutdown_connections 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 --2- 172.21.15.33:0/2382286398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e00af910 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 -- 172.21.15.33:0/2382286398 >> 172.21.15.33:0/2382286398 conn(0x7f72e006d9b0 msgr2=0x7f72e006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 -- 172.21.15.33:0/2382286398 shutdown_connections 2023-10-06T05:32:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.376+0000 7f72df59e700 1 -- 172.21.15.33:0/2382286398 wait complete. 2023-10-06T05:32:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.378+0000 7f72df59e700 1 Processor -- start 2023-10-06T05:32:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.378+0000 7f72df59e700 1 -- start start 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72df59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e0157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72df59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e00cced0 0x7f72e00b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72df59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e015a7d0 0x7f72e00c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72df59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00ca440 con 0x7f72e00af540 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72df59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00c9b10 con 0x7f72e00cced0 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72df59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00c9df0 con 0x7f72e015a7d0 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e0157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e0157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36846/0 (socket says 172.21.15.33:36846) 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 -- 172.21.15.33:0/1134528777 learned_addr learned my addr 172.21.15.33:0/1134528777 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72ddd9b700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e00cced0 0x7f72e00b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72ded9d700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e015a7d0 0x7f72e00c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 -- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e015a7d0 msgr2=0x7f72e00c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e015a7d0 0x7f72e00c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 -- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e00cced0 msgr2=0x7f72e00b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e00cced0 0x7f72e00b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.379+0000 7f72de59c700 1 -- 172.21.15.33:0/1134528777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0009420 con 0x7f72e00af540 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.380+0000 7f72de59c700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e0157310 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f72d00030c0 tx=0x7f72d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.380+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000ba10 con 0x7f72e00af540 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.380+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0020070 con 0x7f72e00af540 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.380+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e00ca0d0 con 0x7f72e00af540 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.380+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d001b6a0 con 0x7f72e00af540 2023-10-06T05:32:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.380+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e00b5e50 con 0x7f72e00af540 2023-10-06T05:32:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.381+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e00b6040 con 0x7f72e00af540 2023-10-06T05:32:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.382+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72d0022030 con 0x7f72e00af540 2023-10-06T05:32:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.383+0000 7f72ce7fc700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c0066f30 0x7f72c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:22.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.383+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f72d0086480 con 0x7f72e00af540 2023-10-06T05:32:22.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.383+0000 7f72ddd9b700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c0066f30 0x7f72c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:22.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.384+0000 7f72ddd9b700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c0066f30 0x7f72c00693e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f72e00b2c40 tx=0x7f72d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:22.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.388+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72e00b6040 con 0x7f72e00af540 2023-10-06T05:32:22.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T05:32:22.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:22.590+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72e00c9c50 con 0x7f72e00af540 2023-10-06T05:32:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.157+0000 7f72ce7fc700 1 -- 172.21.15.33:0/1134528777 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v249) v1 ==== 170+0+0 (secure 0 0 0) 0x7f72e00c9c50 con 0x7f72e00af540 2023-10-06T05:32:23.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.163+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c0066f30 msgr2=0x7f72c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.163+0000 7f72df59e700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c0066f30 0x7f72c00693e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f72e00b2c40 tx=0x7f72d4009040 comp rx=0 tx=0).stop 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.163+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 msgr2=0x7f72e0157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.163+0000 7f72df59e700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e0157310 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f72d00030c0 tx=0x7f72d0016000 comp rx=0 tx=0).stop 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 shutdown_connections 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72c0066f30 0x7f72c00693e0 secure :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f72e00b2c40 tx=0x7f72d4009040 comp rx=0 tx=0).stop 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72e015a7d0 0x7f72e00c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72e00af540 0x7f72e0157310 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 --2- 172.21.15.33:0/1134528777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72e00cced0 0x7f72e00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 >> 172.21.15.33:0/1134528777 conn(0x7f72e006d9b0 msgr2=0x7f72e00c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 shutdown_connections 2023-10-06T05:32:23.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.165+0000 7f72df59e700 1 -- 172.21.15.33:0/1134528777 wait complete. 2023-10-06T05:32:23.717 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:32:23.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:32:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.917+0000 7fdbcd19d700 1 Processor -- start 2023-10-06T05:32:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.918+0000 7fdbcd19d700 1 -- start start 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbcd19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbcd19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80ad430 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbcd19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80afe20 con 0x7fdbc8156370 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbcd19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc806bc70 con 0x7fdbc80b0900 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbc77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdbc80cc7e0 0x7fdbc80b01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43010/0 (socket says 172.21.15.33:43010) 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbc77fe700 1 -- 172.21.15.33:0/1890665333 learned_addr learned my addr 172.21.15.33:0/1890665333 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.919+0000 7fdbcc99c700 1 --2- 172.21.15.33:0/1890665333 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015414932 0 0) 0x7fdbc80ad430 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb0003570 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059163327 0 0) 0x7fdbc80afe20 con 0x7fdbc8156370 2023-10-06T05:32:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbc80ad430 con 0x7fdbc8156370 2023-10-06T05:32:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbcc99c700 1 -- 172.21.15.33:0/1890665333 >> v1:172.21.15.167:6789/0 conn(0x7fdbc8156370 legacy=0x7fdbc80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbcc99c700 1 -- 172.21.15.33:0/1890665333 >> v1:172.21.15.33:6789/0 conn(0x7fdbc80cc7e0 legacy=0x7fdbc80b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557465258 0 0) 0x7fdbb0003570 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbcc99c700 1 -- 172.21.15.33:0/1890665333 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80cccc0 con 0x7fdbc80b0900 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.920+0000 7fdbcc99c700 1 --2- 172.21.15.33:0/1890665333 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc80af8e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009e90 tx=0x7fdbc0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83eb9daa33cbe400 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000e070 con 0x7fdbc80b0900 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc000b870 con 0x7fdbc80b0900 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbc6ffd700 1 -- 172.21.15.33:0/1890665333 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc001470 con 0x7fdbc80b0900 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1890665333 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 msgr2=0x7fdbc80af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1890665333 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc80af8e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009e90 tx=0x7fdbc0009780 comp rx=0 tx=0).stop 2023-10-06T05:32:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1890665333 shutdown_connections 2023-10-06T05:32:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1890665333 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc80af8e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1890665333 >> 172.21.15.33:0/1890665333 conn(0x7fdbc806d030 msgr2=0x7fdbc80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1890665333 shutdown_connections 2023-10-06T05:32:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.921+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1890665333 wait complete. 2023-10-06T05:32:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.923+0000 7fdbcd19d700 1 Processor -- start 2023-10-06T05:32:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.923+0000 7fdbcd19d700 1 -- start start 2023-10-06T05:32:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcd19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc8142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcd19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 0x7fdbc806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcd19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc8156370 0x7fdbc806e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcd19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81432a0 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcd19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc816b000 con 0x7fdbc8156370 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcd19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc816b2e0 con 0x7fdbc80b0900 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 0x7fdbc806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 0x7fdbc806e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36850/0 (socket says 172.21.15.33:36850) 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 -- 172.21.15.33:0/1225811285 learned_addr learned my addr 172.21.15.33:0/1225811285 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbcc99c700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc8156370 0x7fdbc806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc7fff700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc8142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 -- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 msgr2=0x7fdbc8142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc8142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 -- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc8156370 msgr2=0x7fdbc806e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc8156370 0x7fdbc806e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.924+0000 7fdbc77fe700 1 -- 172.21.15.33:0/1225811285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc0009a20 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.925+0000 7fdbc77fe700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 0x7fdbc806e0c0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cb70 tx=0x7fdbbc001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.925+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc001d00 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.925+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc007360 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.925+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc816b5c0 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.925+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc004830 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.925+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc8153f00 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.926+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc816bdc0 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.927+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbbc004ba0 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.928+0000 7fdbaffff700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdbb4066f30 0x7fdbb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.928+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fdbbc084390 con 0x7fdbc80cc7e0 2023-10-06T05:32:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.929+0000 7fdbc7fff700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdbb4066f30 0x7fdbb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.930+0000 7fdbc7fff700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdbb4066f30 0x7fdbb40693e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8003870 tx=0x7fdbb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:23.932+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbbc051560 con 0x7fdbc80cc7e0 2023-10-06T05:32:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:24.199+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdbc804c380 con 0x7fdbc80cc7e0 2023-10-06T05:32:25.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.182+0000 7fdbaffff700 1 -- 172.21.15.33:0/1225811285 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 37 and data pool 38 v191) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdbbc018020 con 0x7fdbc80cc7e0 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.186+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdbb4066f30 msgr2=0x7fdbb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdbb4066f30 0x7fdbb40693e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8003870 tx=0x7fdbb8009040 comp rx=0 tx=0).stop 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 msgr2=0x7fdbc806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 0x7fdbc806e0c0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cb70 tx=0x7fdbbc001ab0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 shutdown_connections 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdbb4066f30 0x7fdbb40693e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc8156370 0x7fdbc806e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc80b0900 0x7fdbc8142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 --2- 172.21.15.33:0/1225811285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc80cc7e0 0x7fdbc806e0c0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 >> 172.21.15.33:0/1225811285 conn(0x7fdbc806d030 msgr2=0x7fdbc80ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:25.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 shutdown_connections 2023-10-06T05:32:25.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.187+0000 7fdbcd19d700 1 -- 172.21.15.33:0/1225811285 wait complete. 2023-10-06T05:32:25.192 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 37 and data pool 38 2023-10-06T05:32:25.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:32:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.406+0000 7f6fe739d700 1 Processor -- start 2023-10-06T05:32:25.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.406+0000 7f6fe739d700 1 -- start start 2023-10-06T05:32:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe739d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6fe006ab00 0x7f6fe00b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6fe00b2e30 0x7f6fe015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe739d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe015e2d0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe006afa0 con 0x7f6fe00b2e30 2023-10-06T05:32:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe739d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe015d6c0 con 0x7f6fe006ab00 2023-10-06T05:32:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe639b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6fe00b0a40 0x7f6fe00b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43022/0 (socket says 172.21.15.33:43022) 2023-10-06T05:32:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe639b700 1 -- 172.21.15.33:0/1317988580 learned_addr learned my addr 172.21.15.33:0/1317988580 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe6b9c700 1 --2- 172.21.15.33:0/1317988580 >> v2:172.21.15.33:3300/0 conn(0x7f6fe00b2e30 0x7f6fe015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe5b9a700 1 --2- 172.21.15.33:0/1317988580 >> v2:172.21.15.167:3300/0 conn(0x7f6fe006ab00 0x7f6fe00b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe5399700 1 -- 172.21.15.33:0/1317988580 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712796878 0 0) 0x7f6fe015e2d0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe5399700 1 -- 172.21.15.33:0/1317988580 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fcc003570 con 0x7f6fe00b0a40 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe6b9c700 1 -- 172.21.15.33:0/1317988580 >> v2:172.21.15.167:3300/0 conn(0x7f6fe006ab00 msgr2=0x7f6fe00b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe6b9c700 1 --2- 172.21.15.33:0/1317988580 >> v2:172.21.15.167:3300/0 conn(0x7f6fe006ab00 0x7f6fe00b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe6b9c700 1 -- 172.21.15.33:0/1317988580 >> v1:172.21.15.33:6789/0 conn(0x7f6fe00b0a40 legacy=0x7f6fe00b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe6b9c700 1 -- 172.21.15.33:0/1317988580 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe015d9a0 con 0x7f6fe00b2e30 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.408+0000 7f6fe5b9a700 1 --2- 172.21.15.33:0/1317988580 >> v2:172.21.15.167:3300/0 conn(0x7f6fe006ab00 0x7f6fe00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.409+0000 7f6fe6b9c700 1 --2- 172.21.15.33:0/1317988580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b2e30 0x7f6fe015a0a0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0065020 tx=0x7f6fd8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9464a3046484d08f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.409+0000 7f6fe5399700 1 -- 172.21.15.33:0/1317988580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd800e070 con 0x7f6fe00b2e30 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.409+0000 7f6fe5399700 1 -- 172.21.15.33:0/1317988580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd800bb60 con 0x7f6fe00b2e30 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.409+0000 7f6fe5399700 1 -- 172.21.15.33:0/1317988580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8012430 con 0x7f6fe00b2e30 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.409+0000 7f6fe739d700 1 -- 172.21.15.33:0/1317988580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b2e30 msgr2=0x7f6fe015a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.409+0000 7f6fe739d700 1 --2- 172.21.15.33:0/1317988580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b2e30 0x7f6fe015a0a0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0065020 tx=0x7f6fd8009780 comp rx=0 tx=0).stop 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.410+0000 7f6fe739d700 1 -- 172.21.15.33:0/1317988580 shutdown_connections 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.410+0000 7f6fe739d700 1 --2- 172.21.15.33:0/1317988580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b2e30 0x7f6fe015a0a0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.410+0000 7f6fe739d700 1 --2- 172.21.15.33:0/1317988580 >> v2:172.21.15.167:3300/0 conn(0x7f6fe006ab00 0x7f6fe00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.410+0000 7f6fe739d700 1 -- 172.21.15.33:0/1317988580 >> 172.21.15.33:0/1317988580 conn(0x7f6fe006e0f0 msgr2=0x7f6fe00675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:25.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.410+0000 7f6fe739d700 1 -- 172.21.15.33:0/1317988580 shutdown_connections 2023-10-06T05:32:25.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.410+0000 7f6fe739d700 1 -- 172.21.15.33:0/1317988580 wait complete. 2023-10-06T05:32:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.411+0000 7f6fe739d700 1 Processor -- start 2023-10-06T05:32:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.411+0000 7f6fe739d700 1 -- start start 2023-10-06T05:32:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fe006ab00 0x7f6fe0149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 0x7f6fe00c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fe00b2e30 0x7f6fe00c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe0146820 con 0x7f6fe00b0a40 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe016b990 con 0x7f6fe006ab00 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe016adc0 con 0x7f6fe00b2e30 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe5b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 0x7f6fe00c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe5b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 0x7f6fe00c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36858/0 (socket says 172.21.15.33:36858) 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.412+0000 7f6fe5b9a700 1 -- 172.21.15.33:0/2280197754 learned_addr learned my addr 172.21.15.33:0/2280197754 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe639b700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fe006ab00 0x7f6fe0149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe5b9a700 1 -- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fe00b2e30 msgr2=0x7f6fe00c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe5b9a700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fe00b2e30 0x7f6fe00c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe5b9a700 1 -- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fe006ab00 msgr2=0x7f6fe0149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe5b9a700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fe006ab00 0x7f6fe0149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe5b9a700 1 -- 172.21.15.33:0/2280197754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8009a20 con 0x7f6fe00b0a40 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe639b700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fe006ab00 0x7f6fe0149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe5b9a700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 0x7f6fe00c5c20 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d860 tx=0x7f6fd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00125f0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0012750 con 0x7f6fe00b0a40 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0012ad0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.413+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe016b060 con 0x7f6fe00b0a40 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.414+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe016b6a0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.414+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe006fcc0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.415+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6fd0014430 con 0x7f6fe00b0a40 2023-10-06T05:32:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.416+0000 7f6fd67fc700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fc4066f30 0x7f6fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.416+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6fd0084b80 con 0x7f6fe00b0a40 2023-10-06T05:32:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.417+0000 7f6fe639b700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fc4066f30 0x7f6fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.418+0000 7f6fe639b700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fc4066f30 0x7f6fc40693e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc007fd0 tx=0x7f6fdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:25.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.420+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fd0051bf0 con 0x7f6fe00b0a40 2023-10-06T05:32:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.624+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fe004c380 con 0x7f6fe00b0a40 2023-10-06T05:32:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.625+0000 7f6fd67fc700 1 -- 172.21.15.33:0/2280197754 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f6fd0055210 con 0x7f6fe00b0a40 2023-10-06T05:32:25.627 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:25.627 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":251,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:32:24.200162+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:32:21.814695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":38,"pool_name":"cephfs_data","create_time":"2023-10-06T05:32:22.631382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.627+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fc4066f30 msgr2=0x7f6fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.627+0000 7f6fe739d700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fc4066f30 0x7f6fc40693e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc007fd0 tx=0x7f6fdc009040 comp rx=0 tx=0).stop 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.627+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 msgr2=0x7f6fe00c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.627+0000 7f6fe739d700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 0x7f6fe00c5c20 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d860 tx=0x7f6fd0008040 comp rx=0 tx=0).stop 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 shutdown_connections 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6fc4066f30 0x7f6fc40693e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fe00b2e30 0x7f6fe00c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fe006ab00 0x7f6fe0149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 --2- 172.21.15.33:0/2280197754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fe00b0a40 0x7f6fe00c5c20 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 >> 172.21.15.33:0/2280197754 conn(0x7f6fe006e0f0 msgr2=0x7f6fe006c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 shutdown_connections 2023-10-06T05:32:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.628+0000 7f6fe739d700 1 -- 172.21.15.33:0/2280197754 wait complete. 2023-10-06T05:32:25.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:32:25.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.833+0000 7f1788b9d700 1 Processor -- start 2023-10-06T05:32:25.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.833+0000 7f1788b9d700 1 -- start start 2023-10-06T05:32:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f1788b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17840b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f1788b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17840b2e30 0x7f178415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f1788b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178415e2d0 con 0x7f17840b0a40 2023-10-06T05:32:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f1788b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178406af70 con 0x7f17840b2e30 2023-10-06T05:32:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f1788b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178415d6c0 con 0x7f178406ab00 2023-10-06T05:32:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f17837fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f17840b0a40 0x7f17840b0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43026/0 (socket says 172.21.15.33:43026) 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f17837fe700 1 -- 172.21.15.33:0/1329887771 learned_addr learned my addr 172.21.15.33:0/1329887771 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.834+0000 7f1783fff700 1 --2- 172.21.15.33:0/1329887771 >> v2:172.21.15.33:3300/0 conn(0x7f17840b2e30 0x7f178415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f1782ffd700 1 --2- 172.21.15.33:0/1329887771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17840b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f17827fc700 1 -- 172.21.15.33:0/1329887771 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985608468 0 0) 0x7f178415e2d0 con 0x7f17840b0a40 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f17827fc700 1 -- 172.21.15.33:0/1329887771 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f176c003570 con 0x7f17840b0a40 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f1783fff700 1 -- 172.21.15.33:0/1329887771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 msgr2=0x7f17840b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f1783fff700 1 --2- 172.21.15.33:0/1329887771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17840b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f1783fff700 1 -- 172.21.15.33:0/1329887771 >> v1:172.21.15.33:6789/0 conn(0x7f17840b0a40 legacy=0x7f17840b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f1783fff700 1 -- 172.21.15.33:0/1329887771 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178415d9a0 con 0x7f17840b2e30 2023-10-06T05:32:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.835+0000 7f1783fff700 1 --2- 172.21.15.33:0/1329887771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b2e30 0x7f178415a0a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f177c009e60 tx=0x7f177c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb92c970559efcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f17827fc700 1 -- 172.21.15.33:0/1329887771 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c00e070 con 0x7f17840b2e30 2023-10-06T05:32:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f17827fc700 1 -- 172.21.15.33:0/1329887771 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c00bab0 con 0x7f17840b2e30 2023-10-06T05:32:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f17827fc700 1 -- 172.21.15.33:0/1329887771 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c00be70 con 0x7f17840b2e30 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 -- 172.21.15.33:0/1329887771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b2e30 msgr2=0x7f178415a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1329887771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b2e30 0x7f178415a0a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f177c009e60 tx=0x7f177c009780 comp rx=0 tx=0).stop 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 -- 172.21.15.33:0/1329887771 shutdown_connections 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1329887771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b2e30 0x7f178415a0a0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1329887771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17840b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 -- 172.21.15.33:0/1329887771 >> 172.21.15.33:0/1329887771 conn(0x7f178406e0f0 msgr2=0x7f17840675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 -- 172.21.15.33:0/1329887771 shutdown_connections 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.836+0000 7f1788b9d700 1 -- 172.21.15.33:0/1329887771 wait complete. 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.838+0000 7f1788b9d700 1 Processor -- start 2023-10-06T05:32:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.838+0000 7f1788b9d700 1 -- start start 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1788b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17841564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1788b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 0x7f17840c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1788b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17840b2e30 0x7f1784169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1788b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178415bc80 con 0x7f17840b0a40 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1788b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178415b070 con 0x7f17840b2e30 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1788b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178415b350 con 0x7f178406ab00 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1782ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 0x7f17840c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1782ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 0x7f17840c3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36878/0 (socket says 172.21.15.33:36878) 2023-10-06T05:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f1782ffd700 1 -- 172.21.15.33:0/1065635415 learned_addr learned my addr 172.21.15.33:0/1065635415 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.839+0000 7f17837fe700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17841564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1783fff700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17840b2e30 0x7f1784169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1782ffd700 1 -- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 msgr2=0x7f17841564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1782ffd700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17841564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1782ffd700 1 -- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17840b2e30 msgr2=0x7f1784169c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1782ffd700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17840b2e30 0x7f1784169c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1782ffd700 1 -- 172.21.15.33:0/1065635415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c009a20 con 0x7f17840b0a40 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f17837fe700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17841564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1783fff700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17840b2e30 0x7f1784169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1782ffd700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 0x7f17840c3690 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f177800e930 tx=0x7f17780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800eed0 con 0x7f17840b0a40 2023-10-06T05:32:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177800c850 con 0x7f17840b0a40 2023-10-06T05:32:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800cbf0 con 0x7f17840b0a40 2023-10-06T05:32:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178415b630 con 0x7f17840b0a40 2023-10-06T05:32:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.840+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1784156a00 con 0x7f17840b0a40 2023-10-06T05:32:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.841+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178406fcc0 con 0x7f17840b0a40 2023-10-06T05:32:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.842+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1778013470 con 0x7f17840b0a40 2023-10-06T05:32:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.843+0000 7f176b7fe700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1770066f30 0x7f17700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.843+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1778084920 con 0x7f17840b0a40 2023-10-06T05:32:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.844+0000 7f17837fe700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1770066f30 0x7f17700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.845+0000 7f17837fe700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1770066f30 0x7f17700693e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f17740058d0 tx=0x7f1774009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:25.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:25.847+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1778051990 con 0x7f17840b0a40 2023-10-06T05:32:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.049+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f178404c380 con 0x7f17840b0a40 2023-10-06T05:32:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.050+0000 7f176b7fe700 1 -- 172.21.15.33:0/1065635415 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f1778054fb0 con 0x7f17840b0a40 2023-10-06T05:32:26.052 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:26.052 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":251,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:32:24.200162+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:32:21.814695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":38,"pool_name":"cephfs_data","create_time":"2023-10-06T05:32:22.631382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1770066f30 msgr2=0x7f17700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1770066f30 0x7f17700693e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f17740058d0 tx=0x7f1774009040 comp rx=0 tx=0).stop 2023-10-06T05:32:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 msgr2=0x7f17840c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 0x7f17840c3690 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f177800e930 tx=0x7f17780091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 shutdown_connections 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1770066f30 0x7f17700693e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17840b2e30 0x7f1784169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f178406ab00 0x7f17841564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 --2- 172.21.15.33:0/1065635415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840b0a40 0x7f17840c3690 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 >> 172.21.15.33:0/1065635415 conn(0x7f178406e0f0 msgr2=0x7f17840af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 shutdown_connections 2023-10-06T05:32:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.052+0000 7f1788b9d700 1 -- 172.21.15.33:0/1065635415 wait complete. 2023-10-06T05:32:26.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:32:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.261+0000 7f5f4a99d700 1 Processor -- start 2023-10-06T05:32:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.261+0000 7f5f4a99d700 1 -- start start 2023-10-06T05:32:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.262+0000 7f5f4a99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f4406c7b0 0x7f5f44067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.262+0000 7f5f4a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4406cb80 0x7f5f4415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4a99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4415d3a0 con 0x7f5f440aff90 2023-10-06T05:32:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4a99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440a9fa0 con 0x7f5f4406c7b0 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b0470 con 0x7f5f4406cb80 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4999b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5f4406c7b0 0x7f5f44067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4919a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5f440aff90 0x7f5f440a9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43040/0 (socket says 172.21.15.33:43040) 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4919a700 1 -- 172.21.15.33:0/1323188215 learned_addr learned my addr 172.21.15.33:0/1323188215 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4a19c700 1 --2- 172.21.15.33:0/1323188215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4406cb80 0x7f5f4415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f48999700 1 -- 172.21.15.33:0/1323188215 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128341695 0 0) 0x7f5f4415d3a0 con 0x7f5f440aff90 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4999b700 1 -- 172.21.15.33:0/1323188215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4406cb80 msgr2=0x7f5f4415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4999b700 1 --2- 172.21.15.33:0/1323188215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4406cb80 0x7f5f4415d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4999b700 1 -- 172.21.15.33:0/1323188215 >> v1:172.21.15.33:6789/0 conn(0x7f5f440aff90 legacy=0x7f5f440a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.263+0000 7f5f4999b700 1 -- 172.21.15.33:0/1323188215 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440b0750 con 0x7f5f4406c7b0 2023-10-06T05:32:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f4999b700 1 --2- 172.21.15.33:0/1323188215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406c7b0 0x7f5f44067fb0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f5f400098d0 tx=0x7f5f40009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de9a1f3b36a1376 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f48999700 1 -- 172.21.15.33:0/1323188215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40009e70 con 0x7f5f4406c7b0 2023-10-06T05:32:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f48999700 1 -- 172.21.15.33:0/1323188215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f40005c10 con 0x7f5f4406c7b0 2023-10-06T05:32:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f48999700 1 -- 172.21.15.33:0/1323188215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000b650 con 0x7f5f4406c7b0 2023-10-06T05:32:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f4a99d700 1 -- 172.21.15.33:0/1323188215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406c7b0 msgr2=0x7f5f44067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/1323188215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406c7b0 0x7f5f44067fb0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f5f400098d0 tx=0x7f5f40009120 comp rx=0 tx=0).stop 2023-10-06T05:32:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.264+0000 7f5f4a99d700 1 -- 172.21.15.33:0/1323188215 shutdown_connections 2023-10-06T05:32:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.265+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/1323188215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406c7b0 0x7f5f44067fb0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.265+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/1323188215 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f4406cb80 0x7f5f4415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.265+0000 7f5f4a99d700 1 -- 172.21.15.33:0/1323188215 >> 172.21.15.33:0/1323188215 conn(0x7f5f4406d670 msgr2=0x7f5f4406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.265+0000 7f5f4a99d700 1 -- 172.21.15.33:0/1323188215 shutdown_connections 2023-10-06T05:32:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.265+0000 7f5f4a99d700 1 -- 172.21.15.33:0/1323188215 wait complete. 2023-10-06T05:32:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.266+0000 7f5f4a99d700 1 Processor -- start 2023-10-06T05:32:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.266+0000 7f5f4a99d700 1 -- start start 2023-10-06T05:32:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4a99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f4406c7b0 0x7f5f4414a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4a99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 0x7f5f4414dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4a99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440aff90 0x7f5f4415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4a99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4414b2b0 con 0x7f5f4406cb80 2023-10-06T05:32:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4a99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4414cee0 con 0x7f5f4406c7b0 2023-10-06T05:32:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4a99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4414d1c0 con 0x7f5f440aff90 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 0x7f5f4414dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4919a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 0x7f5f4414dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36892/0 (socket says 172.21.15.33:36892) 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.267+0000 7f5f4919a700 1 -- 172.21.15.33:0/4216997923 learned_addr learned my addr 172.21.15.33:0/4216997923 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4999b700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f4406c7b0 0x7f5f4414a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4a19c700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440aff90 0x7f5f4415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4919a700 1 -- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440aff90 msgr2=0x7f5f4415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4919a700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440aff90 0x7f5f4415ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4919a700 1 -- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f4406c7b0 msgr2=0x7f5f4414a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4919a700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f4406c7b0 0x7f5f4414a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4919a700 1 -- 172.21.15.33:0/4216997923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f400093c0 con 0x7f5f4406cb80 2023-10-06T05:32:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4a19c700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440aff90 0x7f5f4415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4919a700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 0x7f5f4414dca0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400c5e0 tx=0x7f5f34000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34005b30 con 0x7f5f4406cb80 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34005c90 con 0x7f5f4406cb80 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34007300 con 0x7f5f4406cb80 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4414d4a0 con 0x7f5f4406cb80 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.268+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4414cb00 con 0x7f5f4406cb80 2023-10-06T05:32:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.269+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4404c380 con 0x7f5f4406cb80 2023-10-06T05:32:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.270+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f340076b0 con 0x7f5f4406cb80 2023-10-06T05:32:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.271+0000 7f5f39ffb700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f28066f30 0x7f5f280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.271+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5f34083c60 con 0x7f5f4406cb80 2023-10-06T05:32:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.272+0000 7f5f4999b700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f28066f30 0x7f5f280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.272+0000 7f5f4999b700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f28066f30 0x7f5f280693e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009160 tx=0x7f5f40016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.275+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f34050cd0 con 0x7f5f4406cb80 2023-10-06T05:32:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:26.524+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5f440b0a30 con 0x7f5f4406cb80 2023-10-06T05:32:27.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.195+0000 7f5f39ffb700 1 -- 172.21.15.33:0/4216997923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v194) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5f340170b0 con 0x7f5f4406cb80 2023-10-06T05:32:27.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f28066f30 msgr2=0x7f5f280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:27.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f28066f30 0x7f5f280693e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009160 tx=0x7f5f40016000 comp rx=0 tx=0).stop 2023-10-06T05:32:27.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 msgr2=0x7f5f4414dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:27.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 0x7f5f4414dca0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400c5e0 tx=0x7f5f34000fe0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 shutdown_connections 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f28066f30 0x7f5f280693e0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440aff90 0x7f5f4415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f4406c7b0 0x7f5f4414a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 --2- 172.21.15.33:0/4216997923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4406cb80 0x7f5f4414dca0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 >> 172.21.15.33:0/4216997923 conn(0x7f5f4406d670 msgr2=0x7f5f44158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 shutdown_connections 2023-10-06T05:32:27.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.198+0000 7f5f4a99d700 1 -- 172.21.15.33:0/4216997923 wait complete. 2023-10-06T05:32:27.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:32:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.413+0000 7fa271b9d700 1 Processor -- start 2023-10-06T05:32:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.413+0000 7fa271b9d700 1 -- start start 2023-10-06T05:32:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.414+0000 7fa271b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa26c159850 0x7fa26c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.414+0000 7fa271b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c06d140 0x7fa26c06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.414+0000 7fa271b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c0b0cd0 con 0x7fa26c15acd0 2023-10-06T05:32:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.414+0000 7fa271b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c15a840 con 0x7fa26c159850 2023-10-06T05:32:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa271b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c159f10 con 0x7fa26c06d140 2023-10-06T05:32:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa270b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa26c159850 0x7fa26c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa26bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa26c15acd0 0x7fa26c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43044/0 (socket says 172.21.15.33:43044) 2023-10-06T05:32:27.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa26bfff700 1 -- 172.21.15.33:0/1001664694 learned_addr learned my addr 172.21.15.33:0/1001664694 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa270b9b700 1 -- 172.21.15.33:0/1001664694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c06d140 msgr2=0x7fa26c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa27139c700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c06d140 0x7fa26c06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa270b9b700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c06d140 0x7fa26c06edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa270b9b700 1 -- 172.21.15.33:0/1001664694 >> v1:172.21.15.33:6789/0 conn(0x7fa26c15acd0 legacy=0x7fa26c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa26b7fe700 1 -- 172.21.15.33:0/1001664694 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205784134 0 0) 0x7fa26c0b0cd0 con 0x7fa26c15acd0 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa270b9b700 1 -- 172.21.15.33:0/1001664694 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c15a1f0 con 0x7fa26c159850 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.415+0000 7fa27139c700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c06d140 0x7fa26c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa270b9b700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c06e890 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fa2600098e0 tx=0x7fa2600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8ef32033be00eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa26b7fe700 1 -- 172.21.15.33:0/1001664694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260009e80 con 0x7fa26c159850 2023-10-06T05:32:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa26b7fe700 1 -- 172.21.15.33:0/1001664694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26000b860 con 0x7fa26c159850 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa26b7fe700 1 -- 172.21.15.33:0/1001664694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000bbe0 con 0x7fa26c159850 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa271b9d700 1 -- 172.21.15.33:0/1001664694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 msgr2=0x7fa26c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c06e890 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fa2600098e0 tx=0x7fa2600091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa271b9d700 1 -- 172.21.15.33:0/1001664694 shutdown_connections 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c06e890 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1001664694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c06d140 0x7fa26c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.416+0000 7fa271b9d700 1 -- 172.21.15.33:0/1001664694 >> 172.21.15.33:0/1001664694 conn(0x7fa26c06cd40 msgr2=0x7fa26c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.417+0000 7fa271b9d700 1 -- 172.21.15.33:0/1001664694 shutdown_connections 2023-10-06T05:32:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.417+0000 7fa271b9d700 1 -- 172.21.15.33:0/1001664694 wait complete. 2023-10-06T05:32:27.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.418+0000 7fa271b9d700 1 Processor -- start 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.418+0000 7fa271b9d700 1 -- start start 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa271b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa26c06d140 0x7fa26c146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa271b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa271b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c15acd0 0x7fa26c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa271b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0b1580 con 0x7fa26c159850 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa271b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0b1820 con 0x7fa26c06d140 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa271b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0ca270 con 0x7fa26c15acd0 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.419+0000 7fa26bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36904/0 (socket says 172.21.15.33:36904) 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 -- 172.21.15.33:0/1089159763 learned_addr learned my addr 172.21.15.33:0/1089159763 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:27.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa270b9b700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa26c06d140 0x7fa26c146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa27139c700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c15acd0 0x7fa26c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 -- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c15acd0 msgr2=0x7fa26c0cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c15acd0 0x7fa26c0cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 -- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa26c06d140 msgr2=0x7fa26c146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa26c06d140 0x7fa26c146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 -- 172.21.15.33:0/1089159763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260009420 con 0x7fa26c159850 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa270b9b700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa26c06d140 0x7fa26c146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa27139c700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c15acd0 0x7fa26c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa26bfff700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c0c75a0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c5b0 tx=0x7fa258001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.420+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258005b30 con 0x7fa26c159850 2023-10-06T05:32:27.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.421+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa258005c90 con 0x7fa26c159850 2023-10-06T05:32:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.421+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c0ca550 con 0x7fa26c159850 2023-10-06T05:32:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.421+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258007300 con 0x7fa26c159850 2023-10-06T05:32:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.421+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c146960 con 0x7fa26c159850 2023-10-06T05:32:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.422+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c0b0ce0 con 0x7fa26c159850 2023-10-06T05:32:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.423+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa25801b3b0 con 0x7fa26c159850 2023-10-06T05:32:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.423+0000 7fa268ff9700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa250066f30 0x7fa2500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.423+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa258083bf0 con 0x7fa26c159850 2023-10-06T05:32:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.424+0000 7fa270b9b700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa250066f30 0x7fa2500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.425+0000 7fa270b9b700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa250066f30 0x7fa2500693e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa260003080 tx=0x7fa260017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:27.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.428+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa258050ce0 con 0x7fa26c159850 2023-10-06T05:32:27.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:27.680+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa26c0b1990 con 0x7fa26c159850 2023-10-06T05:32:28.135 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T05:32:28.134+0000 7fd5961f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:32:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.202+0000 7fa268ff9700 1 -- 172.21.15.33:0/1089159763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v195) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2580170c0 con 0x7fa26c159850 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.204+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa250066f30 msgr2=0x7fa2500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.204+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa250066f30 0x7fa2500693e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa260003080 tx=0x7fa260017000 comp rx=0 tx=0).stop 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 msgr2=0x7fa26c0c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c0c75a0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c5b0 tx=0x7fa258001070 comp rx=0 tx=0).stop 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 shutdown_connections 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa250066f30 0x7fa2500693e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa26c15acd0 0x7fa26c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa26c06d140 0x7fa26c146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 --2- 172.21.15.33:0/1089159763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa26c159850 0x7fa26c0c75a0 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 >> 172.21.15.33:0/1089159763 conn(0x7fa26c06cd40 msgr2=0x7fa26c0aae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 shutdown_connections 2023-10-06T05:32:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.205+0000 7fa271b9d700 1 -- 172.21.15.33:0/1089159763 wait complete. 2023-10-06T05:32:28.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:28.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.414+0000 7f6b7a19d700 1 Processor -- start 2023-10-06T05:32:28.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.414+0000 7f6b7a19d700 1 -- start start 2023-10-06T05:32:28.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.415+0000 7f6b7a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740c8050 0x7f6b7413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.415+0000 7f6b7a19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b7406aad0 0x7f6b7406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.415+0000 7f6b7a19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b740c9cb0 con 0x7f6b7413faa0 2023-10-06T05:32:28.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.415+0000 7f6b7a19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b740ca270 con 0x7f6b7406aad0 2023-10-06T05:32:28.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.415+0000 7f6b7a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7406b840 con 0x7f6b740c8050 2023-10-06T05:32:28.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b7406aad0 0x7f6b7406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b7406aad0 0x7f6b7406af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36916/0 (socket says 172.21.15.33:36916) 2023-10-06T05:32:28.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 -- 172.21.15.33:0/897222383 learned_addr learned my addr 172.21.15.33:0/897222383 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:28.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7899a700 1 --2- 172.21.15.33:0/897222383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740c8050 0x7f6b7413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 -- 172.21.15.33:0/897222383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740c8050 msgr2=0x7f6b7413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:28.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 --2- 172.21.15.33:0/897222383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740c8050 0x7f6b7413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 -- 172.21.15.33:0/897222383 >> v1:172.21.15.167:6789/0 conn(0x7f6b7413faa0 legacy=0x7f6b740c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 -- 172.21.15.33:0/897222383 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7406bb20 con 0x7f6b7406aad0 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.416+0000 7f6b7999c700 1 --2- 172.21.15.33:0/897222383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b7406af50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d1a0 tx=0x7f6b6c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb29377e3152933d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b6bfff700 1 -- 172.21.15.33:0/897222383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c010070 con 0x7f6b7406aad0 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b6bfff700 1 -- 172.21.15.33:0/897222383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c00de10 con 0x7f6b7406aad0 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b6bfff700 1 -- 172.21.15.33:0/897222383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00b320 con 0x7f6b7406aad0 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 -- 172.21.15.33:0/897222383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 msgr2=0x7f6b7406af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/897222383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b7406af50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d1a0 tx=0x7f6b6c00ba50 comp rx=0 tx=0).stop 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 -- 172.21.15.33:0/897222383 shutdown_connections 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/897222383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b7406af50 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/897222383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740c8050 0x7f6b7413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 -- 172.21.15.33:0/897222383 >> 172.21.15.33:0/897222383 conn(0x7f6b7406d030 msgr2=0x7f6b7406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:28.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 -- 172.21.15.33:0/897222383 shutdown_connections 2023-10-06T05:32:28.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.417+0000 7f6b7a19d700 1 -- 172.21.15.33:0/897222383 wait complete. 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.419+0000 7f6b7a19d700 1 Processor -- start 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.419+0000 7f6b7a19d700 1 -- start start 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7a19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b740a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7a19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740c8050 0x7f6b74158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7a19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b7413faa0 0x7f6b74159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7a19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74130ef0 con 0x7f6b7406aad0 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7a19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741336f0 con 0x7f6b740c8050 2023-10-06T05:32:28.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7a19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741326e0 con 0x7f6b7413faa0 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b740a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7919b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b740a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36924/0 (socket says 172.21.15.33:36924) 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7919b700 1 -- 172.21.15.33:0/2740403589 learned_addr learned my addr 172.21.15.33:0/2740403589 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7899a700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740c8050 0x7f6b74158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.420+0000 7f6b7999c700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b7413faa0 0x7f6b74159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7919b700 1 -- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b7413faa0 msgr2=0x7f6b74159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7919b700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b7413faa0 0x7f6b74159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7919b700 1 -- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740c8050 msgr2=0x7f6b74158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7919b700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740c8050 0x7f6b74158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7919b700 1 -- 172.21.15.33:0/2740403589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c00bcf0 con 0x7f6b7406aad0 2023-10-06T05:32:28.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7999c700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b7413faa0 0x7f6b74159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7919b700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b740a8a10 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c4e0 tx=0x7f6b7000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70011590 con 0x7f6b7406aad0 2023-10-06T05:32:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70011710 con 0x7f6b7406aad0 2023-10-06T05:32:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70011ad0 con 0x7f6b7406aad0 2023-10-06T05:32:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b741329c0 con 0x7f6b7406aad0 2023-10-06T05:32:28.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.421+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b74133380 con 0x7f6b7406aad0 2023-10-06T05:32:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.422+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7404c380 con 0x7f6b7406aad0 2023-10-06T05:32:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.423+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b7001b3b0 con 0x7f6b7406aad0 2023-10-06T05:32:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.424+0000 7f6b697fa700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b58066f30 0x7f6b580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.424+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b70083ba0 con 0x7f6b7406aad0 2023-10-06T05:32:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.424+0000 7f6b7899a700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b58066f30 0x7f6b580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.425+0000 7f6b7899a700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b58066f30 0x7f6b580693e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6b74132ca0 tx=0x7f6b640091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:28.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.428+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b70050c10 con 0x7f6b7406aad0 2023-10-06T05:32:28.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.675+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b740c9f00 con 0x7f6b7406aad0 2023-10-06T05:32:28.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.676+0000 7f6b697fa700 1 -- 172.21.15.33:0/2740403589 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f6b70017070 con 0x7f6b7406aad0 2023-10-06T05:32:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:28.677 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b58066f30 msgr2=0x7f6b580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b58066f30 0x7f6b580693e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6b74132ca0 tx=0x7f6b640091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 msgr2=0x7f6b740a8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b740a8a10 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c4e0 tx=0x7f6b7000c7f0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 shutdown_connections 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6b58066f30 0x7f6b580693e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b7413faa0 0x7f6b74159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7406aad0 0x7f6b740a8a10 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 --2- 172.21.15.33:0/2740403589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740c8050 0x7f6b74158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 >> 172.21.15.33:0/2740403589 conn(0x7f6b7406d030 msgr2=0x7f6b7406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 shutdown_connections 2023-10-06T05:32:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.678+0000 7f6b7a19d700 1 -- 172.21.15.33:0/2740403589 wait complete. 2023-10-06T05:32:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:32:28.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:32:28.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.883+0000 7f798f19d700 1 Processor -- start 2023-10-06T05:32:28.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.883+0000 7f798f19d700 1 -- start start 2023-10-06T05:32:28.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.884+0000 7f798f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79880cf3b0 0x7f79881594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.884+0000 7f798f19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f79880af0a0 0x7f79881599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.884+0000 7f798f19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880cbd80 con 0x7f79880ccfc0 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.884+0000 7f798f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880af5f0 con 0x7f79880cf3b0 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.884+0000 7f798f19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880af8d0 con 0x7f79880af0a0 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798e19b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f79880ccfc0 0x7f79880cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43056/0 (socket says 172.21.15.33:43056) 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798e19b700 1 -- 172.21.15.33:0/3850009427 learned_addr learned my addr 172.21.15.33:0/3850009427 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d99a700 1 --2- 172.21.15.33:0/3850009427 >> v2:172.21.15.33:3300/0 conn(0x7f79880cf3b0 0x7f79881594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798e99c700 1 --2- 172.21.15.33:0/3850009427 >> v2:172.21.15.167:3300/0 conn(0x7f79880af0a0 0x7f79881599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d199700 1 -- 172.21.15.33:0/3850009427 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091428348 0 0) 0x7f79880cbd80 con 0x7f79880ccfc0 2023-10-06T05:32:28.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d199700 1 -- 172.21.15.33:0/3850009427 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7974003570 con 0x7f79880ccfc0 2023-10-06T05:32:28.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d99a700 1 -- 172.21.15.33:0/3850009427 >> v2:172.21.15.167:3300/0 conn(0x7f79880af0a0 msgr2=0x7f79881599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:28.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d99a700 1 --2- 172.21.15.33:0/3850009427 >> v2:172.21.15.167:3300/0 conn(0x7f79880af0a0 0x7f79881599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d99a700 1 -- 172.21.15.33:0/3850009427 >> v1:172.21.15.33:6789/0 conn(0x7f79880ccfc0 legacy=0x7f79880cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d99a700 1 -- 172.21.15.33:0/3850009427 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880afbb0 con 0x7f79880cf3b0 2023-10-06T05:32:28.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.885+0000 7f798d99a700 1 --2- 172.21.15.33:0/3850009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880cf3b0 0x7f79881594a0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f79780098e0 tx=0x7f79780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8956e35d2978a84d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:28.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798d199700 1 -- 172.21.15.33:0/3850009427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978011630 con 0x7f79880cf3b0 2023-10-06T05:32:28.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798d199700 1 -- 172.21.15.33:0/3850009427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978011790 con 0x7f79880cf3b0 2023-10-06T05:32:28.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798d199700 1 -- 172.21.15.33:0/3850009427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978011b30 con 0x7f79880cf3b0 2023-10-06T05:32:28.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 -- 172.21.15.33:0/3850009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880cf3b0 msgr2=0x7f79881594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 --2- 172.21.15.33:0/3850009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880cf3b0 0x7f79881594a0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f79780098e0 tx=0x7f79780091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 -- 172.21.15.33:0/3850009427 shutdown_connections 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 --2- 172.21.15.33:0/3850009427 >> v2:172.21.15.167:3300/0 conn(0x7f79880af0a0 0x7f79881599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 --2- 172.21.15.33:0/3850009427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880cf3b0 0x7f79881594a0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 -- 172.21.15.33:0/3850009427 >> 172.21.15.33:0/3850009427 conn(0x7f798806cff0 msgr2=0x7f798806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 -- 172.21.15.33:0/3850009427 shutdown_connections 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.886+0000 7f798f19d700 1 -- 172.21.15.33:0/3850009427 wait complete. 2023-10-06T05:32:28.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.888+0000 7f798f19d700 1 Processor -- start 2023-10-06T05:32:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.888+0000 7f798f19d700 1 -- start start 2023-10-06T05:32:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 0x7f79880c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79880ccfc0 0x7f79880c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f798815e9a0 0x7f798816a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c9a80 con 0x7f79880af0a0 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7988169b40 con 0x7f79880ccfc0 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c9cb0 con 0x7f798815e9a0 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 0x7f79880c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 0x7f79880c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36954/0 (socket says 172.21.15.33:36954) 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798e19b700 1 -- 172.21.15.33:0/4265325661 learned_addr learned my addr 172.21.15.33:0/4265325661 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.889+0000 7f798d99a700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79880ccfc0 0x7f79880c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e99c700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f798815e9a0 0x7f798816a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e19b700 1 -- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f798815e9a0 msgr2=0x7f798816a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e19b700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f798815e9a0 0x7f798816a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e19b700 1 -- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79880ccfc0 msgr2=0x7f79880c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e19b700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79880ccfc0 0x7f79880c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e19b700 1 -- 172.21.15.33:0/4265325661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978009420 con 0x7f79880af0a0 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798e19b700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 0x7f79880c8e20 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f798400c590 tx=0x7f7984007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984005b30 con 0x7f79880af0a0 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984005cb0 con 0x7f79880af0a0 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984007320 con 0x7f79880af0a0 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79880c9f90 con 0x7f79880af0a0 2023-10-06T05:32:28.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.890+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7988169e00 con 0x7f79880af0a0 2023-10-06T05:32:28.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.891+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7988169f90 con 0x7f79880af0a0 2023-10-06T05:32:28.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.892+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f798401b3b0 con 0x7f79880af0a0 2023-10-06T05:32:28.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.893+0000 7f797e7fc700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f796c066f30 0x7f796c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:28.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.893+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7984083cf0 con 0x7f79880af0a0 2023-10-06T05:32:28.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.894+0000 7f798d99a700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f796c066f30 0x7f796c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:28.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.894+0000 7f798d99a700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f796c066f30 0x7f796c0693e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f79880b0250 tx=0x7f7978017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:28.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:28.897+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7984050d60 con 0x7f79880af0a0 2023-10-06T05:32:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.101+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7988159f20 con 0x7f79880af0a0 2023-10-06T05:32:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.102+0000 7f797e7fc700 1 -- 172.21.15.33:0/4265325661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f7984017020 con 0x7f79880af0a0 2023-10-06T05:32:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:29.104 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":251,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:32:24.200162+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:32:21.814695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":38,"pool_name":"cephfs_data","create_time":"2023-10-06T05:32:22.631382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:32:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f796c066f30 msgr2=0x7f796c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f796c066f30 0x7f796c0693e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f79880b0250 tx=0x7f7978017000 comp rx=0 tx=0).stop 2023-10-06T05:32:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 msgr2=0x7f79880c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 0x7f79880c8e20 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f798400c590 tx=0x7f7984007e00 comp rx=0 tx=0).stop 2023-10-06T05:32:29.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 shutdown_connections 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f796c066f30 0x7f796c0693e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f798815e9a0 0x7f798816a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79880af0a0 0x7f79880c8e20 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 --2- 172.21.15.33:0/4265325661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79880ccfc0 0x7f79880c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 >> 172.21.15.33:0/4265325661 conn(0x7f798806cff0 msgr2=0x7f79880c67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 shutdown_connections 2023-10-06T05:32:29.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.104+0000 7f798f19d700 1 -- 172.21.15.33:0/4265325661 wait complete. 2023-10-06T05:32:29.123 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:32:29.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:32:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.309+0000 7f67c699d700 1 Processor -- start 2023-10-06T05:32:29.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.310+0000 7f67c699d700 1 -- start start 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f67c00ccfc0 0x7f67c00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c00cf3b0 0x7f67c01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c699d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00cbd80 con 0x7f67c00cc750 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00af5f0 con 0x7f67c00ccfc0 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00af8d0 con 0x7f67c00cf3b0 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c619c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f67c00cc750 0x7f67c00b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43066/0 (socket says 172.21.15.33:43066) 2023-10-06T05:32:29.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.311+0000 7f67c619c700 1 -- 172.21.15.33:0/1595083107 learned_addr learned my addr 172.21.15.33:0/1595083107 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233660959 0 0) 0x7f67c00cbd80 con 0x7f67c00cc750 2023-10-06T05:32:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c519a700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c00cf3b0 0x7f67c01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c599b700 1 --2- 172.21.15.33:0/1595083107 >> v2:172.21.15.167:3300/0 conn(0x7f67c00ccfc0 0x7f67c00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac003570 con 0x7f67c00cc750 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554275658 0 0) 0x7f67ac003570 con 0x7f67c00cc750 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c00cf3b0 msgr2=0x7f67c01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c00cf3b0 0x7f67c01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 >> v2:172.21.15.167:3300/0 conn(0x7f67c00ccfc0 msgr2=0x7f67c00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 --2- 172.21.15.33:0/1595083107 >> v2:172.21.15.167:3300/0 conn(0x7f67c00ccfc0 0x7f67c00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00afbb0 con 0x7f67c00cc750 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f67b80017f0 con 0x7f67c00cc750 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 >> v1:172.21.15.33:6789/0 conn(0x7f67c00cc750 legacy=0x7f67c00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ac003ae0 0x7f67ac003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00af5f0 con 0x7f67ac004810 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00af8d0 con 0x7f67ac003ae0 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac00bfa0 con 0x7f67ac0051c0 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c519a700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.312+0000 7f67c599b700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ac003ae0 0x7f67ac003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c619c700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c519a700 1 -- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac0051c0 msgr2=0x7f67ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c519a700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c519a700 1 -- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ac003ae0 msgr2=0x7f67ac003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c519a700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ac003ae0 0x7f67ac003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c519a700 1 -- 172.21.15.33:0/1595083107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac003570 con 0x7f67ac004810 2023-10-06T05:32:29.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c519a700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f67c00b0250 tx=0x7f67b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a63addde2e0afc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000e710 con 0x7f67ac004810 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b000e870 con 0x7f67ac004810 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.313+0000 7f67c4999700 1 -- 172.21.15.33:0/1595083107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000ec30 con 0x7f67ac004810 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c699d700 1 -- 172.21.15.33:0/1595083107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67ac004810 msgr2=0x7f67ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c699d700 1 --2- 172.21.15.33:0/1595083107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f67c00b0250 tx=0x7f67b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c619c700 1 -- 172.21.15.33:0/1595083107 reap_dead start 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c699d700 1 -- 172.21.15.33:0/1595083107 shutdown_connections 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c699d700 1 -- 172.21.15.33:0/1595083107 >> 172.21.15.33:0/1595083107 conn(0x7f67c006cff0 msgr2=0x7f67c006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c699d700 1 -- 172.21.15.33:0/1595083107 shutdown_connections 2023-10-06T05:32:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.314+0000 7f67c699d700 1 -- 172.21.15.33:0/1595083107 wait complete. 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.315+0000 7f67c699d700 1 Processor -- start 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 -- start start 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac004810 0x7f67c0152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 0x7f67c0169220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0155110 0x7f67c0155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0154c80 con 0x7f67c0150360 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c016a150 con 0x7f67c0155110 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.316+0000 7f67c699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0169b90 con 0x7f67ac004810 2023-10-06T05:32:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 0x7f67c0169220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 0x7f67c0169220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36972/0 (socket says 172.21.15.33:36972) 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 -- 172.21.15.33:0/3084938515 learned_addr learned my addr 172.21.15.33:0/3084938515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c599b700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac004810 0x7f67c0152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c619c700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0155110 0x7f67c0155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 -- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac004810 msgr2=0x7f67c0152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac004810 0x7f67c0152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 -- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0155110 msgr2=0x7f67c0155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0155110 0x7f67c0155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 -- 172.21.15.33:0/3084938515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0169e70 con 0x7f67c0150360 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c619c700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0155110 0x7f67c0155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.317+0000 7f67c519a700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 0x7f67c0169220 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f67ac004790 tx=0x7f67b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.318+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0023070 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.318+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b000e9e0 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.318+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000e710 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.318+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c0154910 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.318+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c0169760 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.319+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c004c380 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.320+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67b0013470 con 0x7f67c0150360 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.320+0000 7f67b5ffb700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67a4066f30 0x7f67a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.320+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f67b0085d00 con 0x7f67c0150360 2023-10-06T05:32:29.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.321+0000 7f67c599b700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67a4066f30 0x7f67a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.321+0000 7f67c599b700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67a4066f30 0x7f67a40693e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f67bc006f00 tx=0x7f67bc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:29.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.325+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67b0052d70 con 0x7f67c0150360 2023-10-06T05:32:29.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.525+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67c00af0a0 con 0x7f67a4066f30 2023-10-06T05:32:29.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.535+0000 7f67b5ffb700 1 -- 172.21.15.33:0/3084938515 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311760 (secure 0 0 0) 0x7f67c00af0a0 con 0x7f67a4066f30 2023-10-06T05:32:29.541 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67a4066f30 msgr2=0x7f67a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67a4066f30 0x7f67a40693e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f67bc006f00 tx=0x7f67bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 msgr2=0x7f67c0169220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 0x7f67c0169220 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f67ac004790 tx=0x7f67b001a000 comp rx=0 tx=0).stop 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 shutdown_connections 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f67a4066f30 0x7f67a40693e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c0155110 0x7f67c0155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ac004810 0x7f67c0152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 --2- 172.21.15.33:0/3084938515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0150360 0x7f67c0169220 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 >> 172.21.15.33:0/3084938515 conn(0x7f67c006cff0 msgr2=0x7f67c00c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:29.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 shutdown_connections 2023-10-06T05:32:29.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.540+0000 7f67c699d700 1 -- 172.21.15.33:0/3084938515 wait complete. 2023-10-06T05:32:29.545 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:32:29.559 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":792,"stamp":"2023-10-06T05:32:28.094049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445352,"kb_used_data":26088,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752529368,"statfs":{"total":773094113280,"available":770590072832,"internally_reserved":0,"allocated":26714112,"data_stored":13118404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.938804"},"pg_stats":[{"pgid":"38.3f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179380+0000","last_change":"2023-10-06T05:32:24.178326+0000","last_active":"2023-10-06T05:32:24.179380+0000","last_peered":"2023-10-06T05:32:24.179380+0000","last_clean":"2023-10-06T05:32:24.179380+0000","last_became_active":"2023-10-06T05:32:24.178049+0000","last_became_peered":"2023-10-06T05:32:24.178049+0000","last_unstale":"2023-10-06T05:32:24.179380+0000","last_undegraded":"2023-10-06T05:32:24.179380+0000","last_fullsized":"2023-10-06T05:32:24.179380+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:49.257644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.3e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180491+0000","last_change":"2023-10-06T05:32:24.173543+0000","last_active":"2023-10-06T05:32:24.180491+0000","last_peered":"2023-10-06T05:32:24.180491+0000","last_clean":"2023-10-06T05:32:24.180491+0000","last_became_active":"2023-10-06T05:32:24.173283+0000","last_became_peered":"2023-10-06T05:32:24.173283+0000","last_unstale":"2023-10-06T05:32:24.180491+0000","last_undegraded":"2023-10-06T05:32:24.180491+0000","last_fullsized":"2023-10-06T05:32:24.180491+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:24.119971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.3d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176954+0000","last_change":"2023-10-06T05:32:24.173585+0000","last_active":"2023-10-06T05:32:24.176954+0000","last_peered":"2023-10-06T05:32:24.176954+0000","last_clean":"2023-10-06T05:32:24.176954+0000","last_became_active":"2023-10-06T05:32:24.173336+0000","last_became_peered":"2023-10-06T05:32:24.173336+0000","last_unstale":"2023-10-06T05:32:24.176954+0000","last_undegraded":"2023-10-06T05:32:24.176954+0000","last_fullsized":"2023-10-06T05:32:24.176954+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:27.592412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.3c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190851+0000","last_change":"2023-10-06T05:32:24.184186+0000","last_active":"2023-10-06T05:32:24.190851+0000","last_peered":"2023-10-06T05:32:24.190851+0000","last_clean":"2023-10-06T05:32:24.190851+0000","last_became_active":"2023-10-06T05:32:24.183928+0000","last_became_peered":"2023-10-06T05:32:24.183928+0000","last_unstale":"2023-10-06T05:32:24.190851+0000","last_undegraded":"2023-10-06T05:32:24.190851+0000","last_fullsized":"2023-10-06T05:32:24.190851+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:51.146580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.3b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176077+0000","last_change":"2023-10-06T05:32:24.173601+0000","last_active":"2023-10-06T05:32:24.176077+0000","last_peered":"2023-10-06T05:32:24.176077+0000","last_clean":"2023-10-06T05:32:24.176077+0000","last_became_active":"2023-10-06T05:32:24.173061+0000","last_became_peered":"2023-10-06T05:32:24.173061+0000","last_unstale":"2023-10-06T05:32:24.176077+0000","last_undegraded":"2023-10-06T05:32:24.176077+0000","last_fullsized":"2023-10-06T05:32:24.176077+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.3a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186495+0000","last_change":"2023-10-06T05:32:24.177953+0000","last_active":"2023-10-06T05:32:24.186495+0000","last_peered":"2023-10-06T05:32:24.186495+0000","last_clean":"2023-10-06T05:32:24.186495+0000","last_became_active":"2023-10-06T05:32:24.177398+0000","last_became_peered":"2023-10-06T05:32:24.177398+0000","last_unstale":"2023-10-06T05:32:24.186495+0000","last_undegraded":"2023-10-06T05:32:24.186495+0000","last_fullsized":"2023-10-06T05:32:24.186495+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:14.210044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.39","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179497+0000","last_change":"2023-10-06T05:32:24.174895+0000","last_active":"2023-10-06T05:32:24.179497+0000","last_peered":"2023-10-06T05:32:24.179497+0000","last_clean":"2023-10-06T05:32:24.179497+0000","last_became_active":"2023-10-06T05:32:24.174641+0000","last_became_peered":"2023-10-06T05:32:24.174641+0000","last_unstale":"2023-10-06T05:32:24.179497+0000","last_undegraded":"2023-10-06T05:32:24.179497+0000","last_fullsized":"2023-10-06T05:32:24.179497+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:21.024627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.38","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.170065+0000","last_change":"2023-10-06T05:32:24.166809+0000","last_active":"2023-10-06T05:32:24.170065+0000","last_peered":"2023-10-06T05:32:24.170065+0000","last_clean":"2023-10-06T05:32:24.170065+0000","last_became_active":"2023-10-06T05:32:24.166477+0000","last_became_peered":"2023-10-06T05:32:24.166477+0000","last_unstale":"2023-10-06T05:32:24.170065+0000","last_undegraded":"2023-10-06T05:32:24.170065+0000","last_fullsized":"2023-10-06T05:32:24.170065+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:32.498083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.37","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.181804+0000","last_change":"2023-10-06T05:32:24.177469+0000","last_active":"2023-10-06T05:32:24.181804+0000","last_peered":"2023-10-06T05:32:24.181804+0000","last_clean":"2023-10-06T05:32:24.181804+0000","last_became_active":"2023-10-06T05:32:24.176411+0000","last_became_peered":"2023-10-06T05:32:24.176411+0000","last_unstale":"2023-10-06T05:32:24.181804+0000","last_undegraded":"2023-10-06T05:32:24.181804+0000","last_fullsized":"2023-10-06T05:32:24.181804+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:42.040536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.36","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184246+0000","last_change":"2023-10-06T05:32:24.176913+0000","last_active":"2023-10-06T05:32:24.184246+0000","last_peered":"2023-10-06T05:32:24.184246+0000","last_clean":"2023-10-06T05:32:24.184246+0000","last_became_active":"2023-10-06T05:32:24.176605+0000","last_became_peered":"2023-10-06T05:32:24.176605+0000","last_unstale":"2023-10-06T05:32:24.184246+0000","last_undegraded":"2023-10-06T05:32:24.184246+0000","last_fullsized":"2023-10-06T05:32:24.184246+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:43.836336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.35","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.163564+0000","last_change":"2023-10-06T05:32:24.162776+0000","last_active":"2023-10-06T05:32:24.163564+0000","last_peered":"2023-10-06T05:32:24.163564+0000","last_clean":"2023-10-06T05:32:24.163564+0000","last_became_active":"2023-10-06T05:32:24.162492+0000","last_became_peered":"2023-10-06T05:32:24.162492+0000","last_unstale":"2023-10-06T05:32:24.163564+0000","last_undegraded":"2023-10-06T05:32:24.163564+0000","last_fullsized":"2023-10-06T05:32:24.163564+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:51.748940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.34","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179890+0000","last_change":"2023-10-06T05:32:24.174493+0000","last_active":"2023-10-06T05:32:24.179890+0000","last_peered":"2023-10-06T05:32:24.179890+0000","last_clean":"2023-10-06T05:32:24.179890+0000","last_became_active":"2023-10-06T05:32:24.174213+0000","last_became_peered":"2023-10-06T05:32:24.174213+0000","last_unstale":"2023-10-06T05:32:24.179890+0000","last_undegraded":"2023-10-06T05:32:24.179890+0000","last_fullsized":"2023-10-06T05:32:24.179890+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:42.202777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.33","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176055+0000","last_change":"2023-10-06T05:32:24.173599+0000","last_active":"2023-10-06T05:32:24.176055+0000","last_peered":"2023-10-06T05:32:24.176055+0000","last_clean":"2023-10-06T05:32:24.176055+0000","last_became_active":"2023-10-06T05:32:24.173113+0000","last_became_peered":"2023-10-06T05:32:24.173113+0000","last_unstale":"2023-10-06T05:32:24.176055+0000","last_undegraded":"2023-10-06T05:32:24.176055+0000","last_fullsized":"2023-10-06T05:32:24.176055+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:17.311578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.32","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186764+0000","last_change":"2023-10-06T05:32:24.176658+0000","last_active":"2023-10-06T05:32:24.186764+0000","last_peered":"2023-10-06T05:32:24.186764+0000","last_clean":"2023-10-06T05:32:24.186764+0000","last_became_active":"2023-10-06T05:32:24.176385+0000","last_became_peered":"2023-10-06T05:32:24.176385+0000","last_unstale":"2023-10-06T05:32:24.186764+0000","last_undegraded":"2023-10-06T05:32:24.186764+0000","last_fullsized":"2023-10-06T05:32:24.186764+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:40.439510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.31","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186606+0000","last_change":"2023-10-06T05:32:24.175674+0000","last_active":"2023-10-06T05:32:24.186606+0000","last_peered":"2023-10-06T05:32:24.186606+0000","last_clean":"2023-10-06T05:32:24.186606+0000","last_became_active":"2023-10-06T05:32:24.175447+0000","last_became_peered":"2023-10-06T05:32:24.175447+0000","last_unstale":"2023-10-06T05:32:24.186606+0000","last_undegraded":"2023-10-06T05:32:24.186606+0000","last_fullsized":"2023-10-06T05:32:24.186606+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:40.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.30","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.189182+0000","last_change":"2023-10-06T05:32:24.185196+0000","last_active":"2023-10-06T05:32:24.189182+0000","last_peered":"2023-10-06T05:32:24.189182+0000","last_clean":"2023-10-06T05:32:24.189182+0000","last_became_active":"2023-10-06T05:32:24.184800+0000","last_became_peered":"2023-10-06T05:32:24.184800+0000","last_unstale":"2023-10-06T05:32:24.189182+0000","last_undegraded":"2023-10-06T05:32:24.189182+0000","last_fullsized":"2023-10-06T05:32:24.189182+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:40.528706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.2f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179588+0000","last_change":"2023-10-06T05:32:24.175329+0000","last_active":"2023-10-06T05:32:24.179588+0000","last_peered":"2023-10-06T05:32:24.179588+0000","last_clean":"2023-10-06T05:32:24.179588+0000","last_became_active":"2023-10-06T05:32:24.175073+0000","last_became_peered":"2023-10-06T05:32:24.175073+0000","last_unstale":"2023-10-06T05:32:24.179588+0000","last_undegraded":"2023-10-06T05:32:24.179588+0000","last_fullsized":"2023-10-06T05:32:24.179588+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:47.162704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.2e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191121+0000","last_change":"2023-10-06T05:32:24.188476+0000","last_active":"2023-10-06T05:32:24.191121+0000","last_peered":"2023-10-06T05:32:24.191121+0000","last_clean":"2023-10-06T05:32:24.191121+0000","last_became_active":"2023-10-06T05:32:24.188210+0000","last_became_peered":"2023-10-06T05:32:24.188210+0000","last_unstale":"2023-10-06T05:32:24.191121+0000","last_undegraded":"2023-10-06T05:32:24.191121+0000","last_fullsized":"2023-10-06T05:32:24.191121+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:39.134144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.175851+0000","last_change":"2023-10-06T05:32:24.173160+0000","last_active":"2023-10-06T05:32:24.175851+0000","last_peered":"2023-10-06T05:32:24.175851+0000","last_clean":"2023-10-06T05:32:24.175851+0000","last_became_active":"2023-10-06T05:32:24.172873+0000","last_became_peered":"2023-10-06T05:32:24.172873+0000","last_unstale":"2023-10-06T05:32:24.175851+0000","last_undegraded":"2023-10-06T05:32:24.175851+0000","last_fullsized":"2023-10-06T05:32:24.175851+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:40.366871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.2c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.193416+0000","last_change":"2023-10-06T05:32:24.189108+0000","last_active":"2023-10-06T05:32:24.193416+0000","last_peered":"2023-10-06T05:32:24.193416+0000","last_clean":"2023-10-06T05:32:24.193416+0000","last_became_active":"2023-10-06T05:32:24.188656+0000","last_became_peered":"2023-10-06T05:32:24.188656+0000","last_unstale":"2023-10-06T05:32:24.193416+0000","last_undegraded":"2023-10-06T05:32:24.193416+0000","last_fullsized":"2023-10-06T05:32:24.193416+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:20.540977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191947+0000","last_change":"2023-10-06T05:32:24.187327+0000","last_active":"2023-10-06T05:32:24.191947+0000","last_peered":"2023-10-06T05:32:24.191947+0000","last_clean":"2023-10-06T05:32:24.191947+0000","last_became_active":"2023-10-06T05:32:24.187090+0000","last_became_peered":"2023-10-06T05:32:24.187090+0000","last_unstale":"2023-10-06T05:32:24.191947+0000","last_undegraded":"2023-10-06T05:32:24.191947+0000","last_fullsized":"2023-10-06T05:32:24.191947+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:31.922663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188699+0000","last_change":"2023-10-06T05:32:24.181600+0000","last_active":"2023-10-06T05:32:24.188699+0000","last_peered":"2023-10-06T05:32:24.188699+0000","last_clean":"2023-10-06T05:32:24.188699+0000","last_became_active":"2023-10-06T05:32:24.181352+0000","last_became_peered":"2023-10-06T05:32:24.181352+0000","last_unstale":"2023-10-06T05:32:24.188699+0000","last_undegraded":"2023-10-06T05:32:24.188699+0000","last_fullsized":"2023-10-06T05:32:24.188699+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:11.081631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.29","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191047+0000","last_change":"2023-10-06T05:32:24.187834+0000","last_active":"2023-10-06T05:32:24.191047+0000","last_peered":"2023-10-06T05:32:24.191047+0000","last_clean":"2023-10-06T05:32:24.191047+0000","last_became_active":"2023-10-06T05:32:24.186942+0000","last_became_peered":"2023-10-06T05:32:24.186942+0000","last_unstale":"2023-10-06T05:32:24.191047+0000","last_undegraded":"2023-10-06T05:32:24.191047+0000","last_fullsized":"2023-10-06T05:32:24.191047+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:41.818468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.28","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.177748+0000","last_change":"2023-10-06T05:32:24.174548+0000","last_active":"2023-10-06T05:32:24.177748+0000","last_peered":"2023-10-06T05:32:24.177748+0000","last_clean":"2023-10-06T05:32:24.177748+0000","last_became_active":"2023-10-06T05:32:24.174294+0000","last_became_peered":"2023-10-06T05:32:24.174294+0000","last_unstale":"2023-10-06T05:32:24.177748+0000","last_undegraded":"2023-10-06T05:32:24.177748+0000","last_fullsized":"2023-10-06T05:32:24.177748+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.426781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.193434+0000","last_change":"2023-10-06T05:32:24.189635+0000","last_active":"2023-10-06T05:32:24.193434+0000","last_peered":"2023-10-06T05:32:24.193434+0000","last_clean":"2023-10-06T05:32:24.193434+0000","last_became_active":"2023-10-06T05:32:24.189018+0000","last_became_peered":"2023-10-06T05:32:24.189018+0000","last_unstale":"2023-10-06T05:32:24.193434+0000","last_undegraded":"2023-10-06T05:32:24.193434+0000","last_fullsized":"2023-10-06T05:32:24.193434+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:58.350917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1f","version":"251'4","reported_seq":14,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.198038+0000","last_change":"2023-10-06T05:32:23.178167+0000","last_active":"2023-10-06T05:32:25.198038+0000","last_peered":"2023-10-06T05:32:25.198038+0000","last_clean":"2023-10-06T05:32:25.198038+0000","last_became_active":"2023-10-06T05:32:23.177689+0000","last_became_peered":"2023-10-06T05:32:23.177689+0000","last_unstale":"2023-10-06T05:32:25.198038+0000","last_undegraded":"2023-10-06T05:32:25.198038+0000","last_fullsized":"2023-10-06T05:32:25.198038+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:15.328039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.1d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166685+0000","last_change":"2023-10-06T05:32:24.163468+0000","last_active":"2023-10-06T05:32:24.166685+0000","last_peered":"2023-10-06T05:32:24.166685+0000","last_clean":"2023-10-06T05:32:24.166685+0000","last_became_active":"2023-10-06T05:32:24.163139+0000","last_became_peered":"2023-10-06T05:32:24.163139+0000","last_unstale":"2023-10-06T05:32:24.166685+0000","last_undegraded":"2023-10-06T05:32:24.166685+0000","last_fullsized":"2023-10-06T05:32:24.166685+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:12:47.306682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1e","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195092+0000","last_change":"2023-10-06T05:32:23.180853+0000","last_active":"2023-10-06T05:32:25.195092+0000","last_peered":"2023-10-06T05:32:25.195092+0000","last_clean":"2023-10-06T05:32:25.195092+0000","last_became_active":"2023-10-06T05:32:23.180531+0000","last_became_peered":"2023-10-06T05:32:23.180531+0000","last_unstale":"2023-10-06T05:32:25.195092+0000","last_undegraded":"2023-10-06T05:32:25.195092+0000","last_fullsized":"2023-10-06T05:32:25.195092+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:14.597456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.1e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186830+0000","last_change":"2023-10-06T05:32:24.176321+0000","last_active":"2023-10-06T05:32:24.186830+0000","last_peered":"2023-10-06T05:32:24.186830+0000","last_clean":"2023-10-06T05:32:24.186830+0000","last_became_active":"2023-10-06T05:32:24.175989+0000","last_became_peered":"2023-10-06T05:32:24.175989+0000","last_unstale":"2023-10-06T05:32:24.186830+0000","last_undegraded":"2023-10-06T05:32:24.186830+0000","last_fullsized":"2023-10-06T05:32:24.186830+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:30.285227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.200383+0000","last_change":"2023-10-06T05:32:23.176926+0000","last_active":"2023-10-06T05:32:25.200383+0000","last_peered":"2023-10-06T05:32:25.200383+0000","last_clean":"2023-10-06T05:32:25.200383+0000","last_became_active":"2023-10-06T05:32:23.176644+0000","last_became_peered":"2023-10-06T05:32:23.176644+0000","last_unstale":"2023-10-06T05:32:25.200383+0000","last_undegraded":"2023-10-06T05:32:25.200383+0000","last_fullsized":"2023-10-06T05:32:25.200383+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:13.577381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.189554+0000","last_change":"2023-10-06T05:32:24.177492+0000","last_active":"2023-10-06T05:32:24.189554+0000","last_peered":"2023-10-06T05:32:24.189554+0000","last_clean":"2023-10-06T05:32:24.189554+0000","last_became_active":"2023-10-06T05:32:24.177174+0000","last_became_peered":"2023-10-06T05:32:24.177174+0000","last_unstale":"2023-10-06T05:32:24.189554+0000","last_undegraded":"2023-10-06T05:32:24.189554+0000","last_fullsized":"2023-10-06T05:32:24.189554+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:52.864938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.198212+0000","last_change":"2023-10-06T05:32:23.174158+0000","last_active":"2023-10-06T05:32:25.198212+0000","last_peered":"2023-10-06T05:32:25.198212+0000","last_clean":"2023-10-06T05:32:25.198212+0000","last_became_active":"2023-10-06T05:32:23.173837+0000","last_became_peered":"2023-10-06T05:32:23.173837+0000","last_unstale":"2023-10-06T05:32:25.198212+0000","last_undegraded":"2023-10-06T05:32:25.198212+0000","last_fullsized":"2023-10-06T05:32:25.198212+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:43.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.18","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186689+0000","last_change":"2023-10-06T05:32:24.181069+0000","last_active":"2023-10-06T05:32:24.186689+0000","last_peered":"2023-10-06T05:32:24.186689+0000","last_clean":"2023-10-06T05:32:24.186689+0000","last_became_active":"2023-10-06T05:32:24.180823+0000","last_became_peered":"2023-10-06T05:32:24.180823+0000","last_unstale":"2023-10-06T05:32:24.186689+0000","last_undegraded":"2023-10-06T05:32:24.186689+0000","last_fullsized":"2023-10-06T05:32:24.186689+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:58.435463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1b","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.193541+0000","last_change":"2023-10-06T05:32:23.181503+0000","last_active":"2023-10-06T05:32:25.193541+0000","last_peered":"2023-10-06T05:32:25.193541+0000","last_clean":"2023-10-06T05:32:25.193541+0000","last_became_active":"2023-10-06T05:32:23.181150+0000","last_became_peered":"2023-10-06T05:32:23.181150+0000","last_unstale":"2023-10-06T05:32:25.193541+0000","last_undegraded":"2023-10-06T05:32:25.193541+0000","last_fullsized":"2023-10-06T05:32:25.193541+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:10.074101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.19","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171078+0000","last_change":"2023-10-06T05:32:24.168394+0000","last_active":"2023-10-06T05:32:24.171078+0000","last_peered":"2023-10-06T05:32:24.171078+0000","last_clean":"2023-10-06T05:32:24.171078+0000","last_became_active":"2023-10-06T05:32:24.168128+0000","last_became_peered":"2023-10-06T05:32:24.168128+0000","last_unstale":"2023-10-06T05:32:24.171078+0000","last_undegraded":"2023-10-06T05:32:24.171078+0000","last_fullsized":"2023-10-06T05:32:24.171078+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:53.832324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1a","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195860+0000","last_change":"2023-10-06T05:32:23.166513+0000","last_active":"2023-10-06T05:32:25.195860+0000","last_peered":"2023-10-06T05:32:25.195860+0000","last_clean":"2023-10-06T05:32:25.195860+0000","last_became_active":"2023-10-06T05:32:23.166258+0000","last_became_peered":"2023-10-06T05:32:23.166258+0000","last_unstale":"2023-10-06T05:32:25.195860+0000","last_undegraded":"2023-10-06T05:32:25.195860+0000","last_fullsized":"2023-10-06T05:32:25.195860+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:48.894291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188682+0000","last_change":"2023-10-06T05:32:24.180189+0000","last_active":"2023-10-06T05:32:24.188682+0000","last_peered":"2023-10-06T05:32:24.188682+0000","last_clean":"2023-10-06T05:32:24.188682+0000","last_became_active":"2023-10-06T05:32:24.179939+0000","last_became_peered":"2023-10-06T05:32:24.179939+0000","last_unstale":"2023-10-06T05:32:24.188682+0000","last_undegraded":"2023-10-06T05:32:24.188682+0000","last_fullsized":"2023-10-06T05:32:24.188682+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:17.068575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171239+0000","last_change":"2023-10-06T05:32:23.174242+0000","last_active":"2023-10-06T05:32:24.171239+0000","last_peered":"2023-10-06T05:32:24.171239+0000","last_clean":"2023-10-06T05:32:24.171239+0000","last_became_active":"2023-10-06T05:32:23.173878+0000","last_became_peered":"2023-10-06T05:32:23.173878+0000","last_unstale":"2023-10-06T05:32:24.171239+0000","last_undegraded":"2023-10-06T05:32:24.171239+0000","last_fullsized":"2023-10-06T05:32:24.171239+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:57.831472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.1b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180411+0000","last_change":"2023-10-06T05:32:24.173615+0000","last_active":"2023-10-06T05:32:24.180411+0000","last_peered":"2023-10-06T05:32:24.180411+0000","last_clean":"2023-10-06T05:32:24.180411+0000","last_became_active":"2023-10-06T05:32:24.173356+0000","last_became_peered":"2023-10-06T05:32:24.173356+0000","last_unstale":"2023-10-06T05:32:24.180411+0000","last_undegraded":"2023-10-06T05:32:24.180411+0000","last_fullsized":"2023-10-06T05:32:24.180411+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:43.048666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.158804+0000","last_change":"2023-10-06T05:32:23.179905+0000","last_active":"2023-10-06T05:32:24.158804+0000","last_peered":"2023-10-06T05:32:24.158804+0000","last_clean":"2023-10-06T05:32:24.158804+0000","last_became_active":"2023-10-06T05:32:23.179020+0000","last_became_peered":"2023-10-06T05:32:23.179020+0000","last_unstale":"2023-10-06T05:32:24.158804+0000","last_undegraded":"2023-10-06T05:32:24.158804+0000","last_fullsized":"2023-10-06T05:32:24.158804+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:50.297518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.14","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190969+0000","last_change":"2023-10-06T05:32:24.186062+0000","last_active":"2023-10-06T05:32:24.190969+0000","last_peered":"2023-10-06T05:32:24.190969+0000","last_clean":"2023-10-06T05:32:24.190969+0000","last_became_active":"2023-10-06T05:32:24.185747+0000","last_became_peered":"2023-10-06T05:32:24.185747+0000","last_unstale":"2023-10-06T05:32:24.190969+0000","last_undegraded":"2023-10-06T05:32:24.190969+0000","last_fullsized":"2023-10-06T05:32:24.190969+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:31.226159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.164731+0000","last_change":"2023-10-06T05:32:23.184729+0000","last_active":"2023-10-06T05:32:24.164731+0000","last_peered":"2023-10-06T05:32:24.164731+0000","last_clean":"2023-10-06T05:32:24.164731+0000","last_became_active":"2023-10-06T05:32:23.184445+0000","last_became_peered":"2023-10-06T05:32:23.184445+0000","last_unstale":"2023-10-06T05:32:24.164731+0000","last_undegraded":"2023-10-06T05:32:24.164731+0000","last_fullsized":"2023-10-06T05:32:24.164731+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:43.366148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.15","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180837+0000","last_change":"2023-10-06T05:32:24.175439+0000","last_active":"2023-10-06T05:32:24.180837+0000","last_peered":"2023-10-06T05:32:24.180837+0000","last_clean":"2023-10-06T05:32:24.180837+0000","last_became_active":"2023-10-06T05:32:24.175124+0000","last_became_peered":"2023-10-06T05:32:24.175124+0000","last_unstale":"2023-10-06T05:32:24.180837+0000","last_undegraded":"2023-10-06T05:32:24.180837+0000","last_fullsized":"2023-10-06T05:32:24.180837+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:48.631818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172865+0000","last_change":"2023-10-06T05:32:23.180183+0000","last_active":"2023-10-06T05:32:24.172865+0000","last_peered":"2023-10-06T05:32:24.172865+0000","last_clean":"2023-10-06T05:32:24.172865+0000","last_became_active":"2023-10-06T05:32:23.179918+0000","last_became_peered":"2023-10-06T05:32:23.179918+0000","last_unstale":"2023-10-06T05:32:24.172865+0000","last_undegraded":"2023-10-06T05:32:24.172865+0000","last_fullsized":"2023-10-06T05:32:24.172865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:59.784967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.16","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186851+0000","last_change":"2023-10-06T05:32:24.176923+0000","last_active":"2023-10-06T05:32:24.186851+0000","last_peered":"2023-10-06T05:32:24.186851+0000","last_clean":"2023-10-06T05:32:24.186851+0000","last_became_active":"2023-10-06T05:32:24.176652+0000","last_became_peered":"2023-10-06T05:32:24.176652+0000","last_unstale":"2023-10-06T05:32:24.186851+0000","last_undegraded":"2023-10-06T05:32:24.186851+0000","last_fullsized":"2023-10-06T05:32:24.186851+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:32.195976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.15","version":"251'3","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.200865+0000","last_change":"2023-10-06T05:32:23.179308+0000","last_active":"2023-10-06T05:32:25.200865+0000","last_peered":"2023-10-06T05:32:25.200865+0000","last_clean":"2023-10-06T05:32:25.200865+0000","last_became_active":"2023-10-06T05:32:23.179038+0000","last_became_peered":"2023-10-06T05:32:23.179038+0000","last_unstale":"2023-10-06T05:32:25.200865+0000","last_undegraded":"2023-10-06T05:32:25.200865+0000","last_fullsized":"2023-10-06T05:32:25.200865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:02.273731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.17","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190880+0000","last_change":"2023-10-06T05:32:24.185229+0000","last_active":"2023-10-06T05:32:24.190880+0000","last_peered":"2023-10-06T05:32:24.190880+0000","last_clean":"2023-10-06T05:32:24.190880+0000","last_became_active":"2023-10-06T05:32:24.184951+0000","last_became_peered":"2023-10-06T05:32:24.184951+0000","last_unstale":"2023-10-06T05:32:24.190880+0000","last_undegraded":"2023-10-06T05:32:24.190880+0000","last_fullsized":"2023-10-06T05:32:24.190880+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:59.568662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.14","version":"251'6","reported_seq":16,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.197454+0000","last_change":"2023-10-06T05:32:23.180146+0000","last_active":"2023-10-06T05:32:25.197454+0000","last_peered":"2023-10-06T05:32:25.197454+0000","last_clean":"2023-10-06T05:32:25.197454+0000","last_became_active":"2023-10-06T05:32:23.179886+0000","last_became_peered":"2023-10-06T05:32:23.179886+0000","last_unstale":"2023-10-06T05:32:25.197454+0000","last_undegraded":"2023-10-06T05:32:25.197454+0000","last_fullsized":"2023-10-06T05:32:25.197454+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:00.888111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.10","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172687+0000","last_change":"2023-10-06T05:32:24.168677+0000","last_active":"2023-10-06T05:32:24.172687+0000","last_peered":"2023-10-06T05:32:24.172687+0000","last_clean":"2023-10-06T05:32:24.172687+0000","last_became_active":"2023-10-06T05:32:24.168348+0000","last_became_peered":"2023-10-06T05:32:24.168348+0000","last_unstale":"2023-10-06T05:32:24.172687+0000","last_undegraded":"2023-10-06T05:32:24.172687+0000","last_fullsized":"2023-10-06T05:32:24.172687+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:15.243862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.13","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195635+0000","last_change":"2023-10-06T05:32:23.180363+0000","last_active":"2023-10-06T05:32:25.195635+0000","last_peered":"2023-10-06T05:32:25.195635+0000","last_clean":"2023-10-06T05:32:25.195635+0000","last_became_active":"2023-10-06T05:32:23.180063+0000","last_became_peered":"2023-10-06T05:32:23.180063+0000","last_unstale":"2023-10-06T05:32:25.195635+0000","last_undegraded":"2023-10-06T05:32:25.195635+0000","last_fullsized":"2023-10-06T05:32:25.195635+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:29.866801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.11","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.181947+0000","last_change":"2023-10-06T05:32:24.181416+0000","last_active":"2023-10-06T05:32:24.181947+0000","last_peered":"2023-10-06T05:32:24.181947+0000","last_clean":"2023-10-06T05:32:24.181947+0000","last_became_active":"2023-10-06T05:32:24.181150+0000","last_became_peered":"2023-10-06T05:32:24.181150+0000","last_unstale":"2023-10-06T05:32:24.181947+0000","last_undegraded":"2023-10-06T05:32:24.181947+0000","last_fullsized":"2023-10-06T05:32:24.181947+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:22.091050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.164816+0000","last_change":"2023-10-06T05:32:23.177156+0000","last_active":"2023-10-06T05:32:24.164816+0000","last_peered":"2023-10-06T05:32:24.164816+0000","last_clean":"2023-10-06T05:32:24.164816+0000","last_became_active":"2023-10-06T05:32:23.176876+0000","last_became_peered":"2023-10-06T05:32:23.176876+0000","last_unstale":"2023-10-06T05:32:24.164816+0000","last_undegraded":"2023-10-06T05:32:24.164816+0000","last_fullsized":"2023-10-06T05:32:24.164816+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:42.828806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.12","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188663+0000","last_change":"2023-10-06T05:32:24.179626+0000","last_active":"2023-10-06T05:32:24.188663+0000","last_peered":"2023-10-06T05:32:24.188663+0000","last_clean":"2023-10-06T05:32:24.188663+0000","last_became_active":"2023-10-06T05:32:24.179326+0000","last_became_peered":"2023-10-06T05:32:24.179326+0000","last_unstale":"2023-10-06T05:32:24.188663+0000","last_undegraded":"2023-10-06T05:32:24.188663+0000","last_fullsized":"2023-10-06T05:32:24.188663+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:51.883357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166185+0000","last_change":"2023-10-06T05:32:23.182163+0000","last_active":"2023-10-06T05:32:24.166185+0000","last_peered":"2023-10-06T05:32:24.166185+0000","last_clean":"2023-10-06T05:32:24.166185+0000","last_became_active":"2023-10-06T05:32:23.181876+0000","last_became_peered":"2023-10-06T05:32:23.181876+0000","last_unstale":"2023-10-06T05:32:24.166185+0000","last_undegraded":"2023-10-06T05:32:24.166185+0000","last_fullsized":"2023-10-06T05:32:24.166185+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:05.667623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.13","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180019+0000","last_change":"2023-10-06T05:32:24.173718+0000","last_active":"2023-10-06T05:32:24.180019+0000","last_peered":"2023-10-06T05:32:24.180019+0000","last_clean":"2023-10-06T05:32:24.180019+0000","last_became_active":"2023-10-06T05:32:24.173451+0000","last_became_peered":"2023-10-06T05:32:24.173451+0000","last_unstale":"2023-10-06T05:32:24.180019+0000","last_undegraded":"2023-10-06T05:32:24.180019+0000","last_fullsized":"2023-10-06T05:32:24.180019+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:43.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.10","version":"251'8","reported_seq":18,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.202231+0000","last_change":"2023-10-06T05:32:23.178713+0000","last_active":"2023-10-06T05:32:25.202231+0000","last_peered":"2023-10-06T05:32:25.202231+0000","last_clean":"2023-10-06T05:32:25.202231+0000","last_became_active":"2023-10-06T05:32:23.178324+0000","last_became_peered":"2023-10-06T05:32:23.178324+0000","last_unstale":"2023-10-06T05:32:25.202231+0000","last_undegraded":"2023-10-06T05:32:25.202231+0000","last_fullsized":"2023-10-06T05:32:25.202231+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:35.285079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.174678+0000","last_change":"2023-10-06T05:32:24.172408+0000","last_active":"2023-10-06T05:32:24.174678+0000","last_peered":"2023-10-06T05:32:24.174678+0000","last_clean":"2023-10-06T05:32:24.174678+0000","last_became_active":"2023-10-06T05:32:24.172160+0000","last_became_peered":"2023-10-06T05:32:24.172160+0000","last_unstale":"2023-10-06T05:32:24.174678+0000","last_undegraded":"2023-10-06T05:32:24.174678+0000","last_fullsized":"2023-10-06T05:32:24.174678+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:41.998814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.f","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195429+0000","last_change":"2023-10-06T05:32:23.177078+0000","last_active":"2023-10-06T05:32:25.195429+0000","last_peered":"2023-10-06T05:32:25.195429+0000","last_clean":"2023-10-06T05:32:25.195429+0000","last_became_active":"2023-10-06T05:32:23.176782+0000","last_became_peered":"2023-10-06T05:32:23.176782+0000","last_unstale":"2023-10-06T05:32:25.195429+0000","last_undegraded":"2023-10-06T05:32:25.195429+0000","last_fullsized":"2023-10-06T05:32:25.195429+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:33.934362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176156+0000","last_change":"2023-10-06T05:32:24.174998+0000","last_active":"2023-10-06T05:32:24.176156+0000","last_peered":"2023-10-06T05:32:24.176156+0000","last_clean":"2023-10-06T05:32:24.176156+0000","last_became_active":"2023-10-06T05:32:24.174637+0000","last_became_peered":"2023-10-06T05:32:24.174637+0000","last_unstale":"2023-10-06T05:32:24.176156+0000","last_undegraded":"2023-10-06T05:32:24.176156+0000","last_fullsized":"2023-10-06T05:32:24.176156+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:57.964936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.e","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.191975+0000","last_change":"2023-10-06T05:32:23.171097+0000","last_active":"2023-10-06T05:32:25.191975+0000","last_peered":"2023-10-06T05:32:25.191975+0000","last_clean":"2023-10-06T05:32:25.191975+0000","last_became_active":"2023-10-06T05:32:23.170822+0000","last_became_peered":"2023-10-06T05:32:23.170822+0000","last_unstale":"2023-10-06T05:32:25.191975+0000","last_undegraded":"2023-10-06T05:32:25.191975+0000","last_fullsized":"2023-10-06T05:32:25.191975+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:49.626561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176787+0000","last_change":"2023-10-06T05:32:24.171850+0000","last_active":"2023-10-06T05:32:24.176787+0000","last_peered":"2023-10-06T05:32:24.176787+0000","last_clean":"2023-10-06T05:32:24.176787+0000","last_became_active":"2023-10-06T05:32:24.171570+0000","last_became_peered":"2023-10-06T05:32:24.171570+0000","last_unstale":"2023-10-06T05:32:24.176787+0000","last_undegraded":"2023-10-06T05:32:24.176787+0000","last_fullsized":"2023-10-06T05:32:24.176787+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:16.335570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.d","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.192351+0000","last_change":"2023-10-06T05:32:23.172247+0000","last_active":"2023-10-06T05:32:25.192351+0000","last_peered":"2023-10-06T05:32:25.192351+0000","last_clean":"2023-10-06T05:32:25.192351+0000","last_became_active":"2023-10-06T05:32:23.171388+0000","last_became_peered":"2023-10-06T05:32:23.171388+0000","last_unstale":"2023-10-06T05:32:25.192351+0000","last_undegraded":"2023-10-06T05:32:25.192351+0000","last_fullsized":"2023-10-06T05:32:25.192351+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:47.624287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186668+0000","last_change":"2023-10-06T05:32:24.182168+0000","last_active":"2023-10-06T05:32:24.186668+0000","last_peered":"2023-10-06T05:32:24.186668+0000","last_clean":"2023-10-06T05:32:24.186668+0000","last_became_active":"2023-10-06T05:32:24.181903+0000","last_became_peered":"2023-10-06T05:32:24.181903+0000","last_unstale":"2023-10-06T05:32:24.186668+0000","last_undegraded":"2023-10-06T05:32:24.186668+0000","last_fullsized":"2023-10-06T05:32:24.186668+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:28.852368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172571+0000","last_change":"2023-10-06T05:32:23.178647+0000","last_active":"2023-10-06T05:32:24.172571+0000","last_peered":"2023-10-06T05:32:24.172571+0000","last_clean":"2023-10-06T05:32:24.172571+0000","last_became_active":"2023-10-06T05:32:23.178372+0000","last_became_peered":"2023-10-06T05:32:23.178372+0000","last_unstale":"2023-10-06T05:32:24.172571+0000","last_undegraded":"2023-10-06T05:32:24.172571+0000","last_fullsized":"2023-10-06T05:32:24.172571+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:30.194013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.8","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190952+0000","last_change":"2023-10-06T05:32:24.186028+0000","last_active":"2023-10-06T05:32:24.190952+0000","last_peered":"2023-10-06T05:32:24.190952+0000","last_clean":"2023-10-06T05:32:24.190952+0000","last_became_active":"2023-10-06T05:32:24.185726+0000","last_became_peered":"2023-10-06T05:32:24.185726+0000","last_unstale":"2023-10-06T05:32:24.190952+0000","last_undegraded":"2023-10-06T05:32:24.190952+0000","last_fullsized":"2023-10-06T05:32:24.190952+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:12.995850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.b","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.197887+0000","last_change":"2023-10-06T05:32:23.177663+0000","last_active":"2023-10-06T05:32:25.197887+0000","last_peered":"2023-10-06T05:32:25.197887+0000","last_clean":"2023-10-06T05:32:25.197887+0000","last_became_active":"2023-10-06T05:32:23.176488+0000","last_became_peered":"2023-10-06T05:32:23.176488+0000","last_unstale":"2023-10-06T05:32:25.197887+0000","last_undegraded":"2023-10-06T05:32:25.197887+0000","last_fullsized":"2023-10-06T05:32:25.197887+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:19.533446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.9","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171060+0000","last_change":"2023-10-06T05:32:24.167891+0000","last_active":"2023-10-06T05:32:24.171060+0000","last_peered":"2023-10-06T05:32:24.171060+0000","last_clean":"2023-10-06T05:32:24.171060+0000","last_became_active":"2023-10-06T05:32:24.167602+0000","last_became_peered":"2023-10-06T05:32:24.167602+0000","last_unstale":"2023-10-06T05:32:24.171060+0000","last_undegraded":"2023-10-06T05:32:24.171060+0000","last_fullsized":"2023-10-06T05:32:24.171060+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:50.317183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.178969+0000","last_change":"2023-10-06T05:32:23.176326+0000","last_active":"2023-10-06T05:32:24.178969+0000","last_peered":"2023-10-06T05:32:24.178969+0000","last_clean":"2023-10-06T05:32:24.178969+0000","last_became_active":"2023-10-06T05:32:23.176027+0000","last_became_peered":"2023-10-06T05:32:23.176027+0000","last_unstale":"2023-10-06T05:32:24.178969+0000","last_undegraded":"2023-10-06T05:32:24.178969+0000","last_fullsized":"2023-10-06T05:32:24.178969+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:08.902322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176122+0000","last_change":"2023-10-06T05:32:24.174385+0000","last_active":"2023-10-06T05:32:24.176122+0000","last_peered":"2023-10-06T05:32:24.176122+0000","last_clean":"2023-10-06T05:32:24.176122+0000","last_became_active":"2023-10-06T05:32:24.174122+0000","last_became_peered":"2023-10-06T05:32:24.174122+0000","last_unstale":"2023-10-06T05:32:24.176122+0000","last_undegraded":"2023-10-06T05:32:24.176122+0000","last_fullsized":"2023-10-06T05:32:24.176122+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:58.601296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.169086+0000","last_change":"2023-10-06T05:32:23.167801+0000","last_active":"2023-10-06T05:32:24.169086+0000","last_peered":"2023-10-06T05:32:24.169086+0000","last_clean":"2023-10-06T05:32:24.169086+0000","last_became_active":"2023-10-06T05:32:23.167534+0000","last_became_peered":"2023-10-06T05:32:23.167534+0000","last_unstale":"2023-10-06T05:32:24.169086+0000","last_undegraded":"2023-10-06T05:32:24.169086+0000","last_fullsized":"2023-10-06T05:32:24.169086+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:28.556557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.175928+0000","last_change":"2023-10-06T05:32:24.171385+0000","last_active":"2023-10-06T05:32:24.175928+0000","last_peered":"2023-10-06T05:32:24.175928+0000","last_clean":"2023-10-06T05:32:24.175928+0000","last_became_active":"2023-10-06T05:32:24.170957+0000","last_became_peered":"2023-10-06T05:32:24.170957+0000","last_unstale":"2023-10-06T05:32:24.175928+0000","last_undegraded":"2023-10-06T05:32:24.175928+0000","last_fullsized":"2023-10-06T05:32:24.175928+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:55.746676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165518+0000","last_change":"2023-10-06T05:32:23.182597+0000","last_active":"2023-10-06T05:32:24.165518+0000","last_peered":"2023-10-06T05:32:24.165518+0000","last_clean":"2023-10-06T05:32:24.165518+0000","last_became_active":"2023-10-06T05:32:23.182079+0000","last_became_peered":"2023-10-06T05:32:23.182079+0000","last_unstale":"2023-10-06T05:32:24.165518+0000","last_undegraded":"2023-10-06T05:32:24.165518+0000","last_fullsized":"2023-10-06T05:32:24.165518+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:40.353782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.4","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191127+0000","last_change":"2023-10-06T05:32:24.188487+0000","last_active":"2023-10-06T05:32:24.191127+0000","last_peered":"2023-10-06T05:32:24.191127+0000","last_clean":"2023-10-06T05:32:24.191127+0000","last_became_active":"2023-10-06T05:32:24.187720+0000","last_became_peered":"2023-10-06T05:32:24.187720+0000","last_unstale":"2023-10-06T05:32:24.191127+0000","last_undegraded":"2023-10-06T05:32:24.191127+0000","last_fullsized":"2023-10-06T05:32:24.191127+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:38.004084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.7","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.191514+0000","last_change":"2023-10-06T05:32:23.180590+0000","last_active":"2023-10-06T05:32:25.191514+0000","last_peered":"2023-10-06T05:32:25.191514+0000","last_clean":"2023-10-06T05:32:25.191514+0000","last_became_active":"2023-10-06T05:32:23.180319+0000","last_became_peered":"2023-10-06T05:32:23.180319+0000","last_unstale":"2023-10-06T05:32:25.191514+0000","last_undegraded":"2023-10-06T05:32:25.191514+0000","last_fullsized":"2023-10-06T05:32:25.191514+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:33.942682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.5","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.185949+0000","last_change":"2023-10-06T05:32:24.181178+0000","last_active":"2023-10-06T05:32:24.185949+0000","last_peered":"2023-10-06T05:32:24.185949+0000","last_clean":"2023-10-06T05:32:24.185949+0000","last_became_active":"2023-10-06T05:32:24.180933+0000","last_became_peered":"2023-10-06T05:32:24.180933+0000","last_unstale":"2023-10-06T05:32:24.185949+0000","last_undegraded":"2023-10-06T05:32:24.185949+0000","last_fullsized":"2023-10-06T05:32:24.185949+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:45.636162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.6","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.159254+0000","last_change":"2023-10-06T05:32:23.178339+0000","last_active":"2023-10-06T05:32:24.159254+0000","last_peered":"2023-10-06T05:32:24.159254+0000","last_clean":"2023-10-06T05:32:24.159254+0000","last_became_active":"2023-10-06T05:32:23.178062+0000","last_became_peered":"2023-10-06T05:32:23.178062+0000","last_unstale":"2023-10-06T05:32:24.159254+0000","last_undegraded":"2023-10-06T05:32:24.159254+0000","last_fullsized":"2023-10-06T05:32:24.159254+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.343386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.6","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180511+0000","last_change":"2023-10-06T05:32:24.174589+0000","last_active":"2023-10-06T05:32:24.180511+0000","last_peered":"2023-10-06T05:32:24.180511+0000","last_clean":"2023-10-06T05:32:24.180511+0000","last_became_active":"2023-10-06T05:32:24.173846+0000","last_became_peered":"2023-10-06T05:32:24.173846+0000","last_unstale":"2023-10-06T05:32:24.180511+0000","last_undegraded":"2023-10-06T05:32:24.180511+0000","last_fullsized":"2023-10-06T05:32:24.180511+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.827006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.168717+0000","last_change":"2023-10-06T05:32:23.182554+0000","last_active":"2023-10-06T05:32:24.168717+0000","last_peered":"2023-10-06T05:32:24.168717+0000","last_clean":"2023-10-06T05:32:24.168717+0000","last_became_active":"2023-10-06T05:32:23.182296+0000","last_became_peered":"2023-10-06T05:32:23.182296+0000","last_unstale":"2023-10-06T05:32:24.168717+0000","last_undegraded":"2023-10-06T05:32:24.168717+0000","last_fullsized":"2023-10-06T05:32:24.168717+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:38.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.7","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171034+0000","last_change":"2023-10-06T05:32:24.169289+0000","last_active":"2023-10-06T05:32:24.171034+0000","last_peered":"2023-10-06T05:32:24.171034+0000","last_clean":"2023-10-06T05:32:24.171034+0000","last_became_active":"2023-10-06T05:32:24.169051+0000","last_became_peered":"2023-10-06T05:32:24.169051+0000","last_unstale":"2023-10-06T05:32:24.171034+0000","last_undegraded":"2023-10-06T05:32:24.171034+0000","last_fullsized":"2023-10-06T05:32:24.171034+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:30.250852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.163782+0000","last_change":"2023-10-06T05:32:23.168886+0000","last_active":"2023-10-06T05:32:24.163782+0000","last_peered":"2023-10-06T05:32:24.163782+0000","last_clean":"2023-10-06T05:32:24.163782+0000","last_became_active":"2023-10-06T05:32:23.168587+0000","last_became_peered":"2023-10-06T05:32:23.168587+0000","last_unstale":"2023-10-06T05:32:24.163782+0000","last_undegraded":"2023-10-06T05:32:24.163782+0000","last_fullsized":"2023-10-06T05:32:24.163782+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:54.739146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.0","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184097+0000","last_change":"2023-10-06T05:32:24.183162+0000","last_active":"2023-10-06T05:32:24.184097+0000","last_peered":"2023-10-06T05:32:24.184097+0000","last_clean":"2023-10-06T05:32:24.184097+0000","last_became_active":"2023-10-06T05:32:24.182826+0000","last_became_peered":"2023-10-06T05:32:24.182826+0000","last_unstale":"2023-10-06T05:32:24.184097+0000","last_undegraded":"2023-10-06T05:32:24.184097+0000","last_fullsized":"2023-10-06T05:32:24.184097+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:42.348407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.160865+0000","last_change":"2023-10-06T05:32:23.172000+0000","last_active":"2023-10-06T05:32:24.160865+0000","last_peered":"2023-10-06T05:32:24.160865+0000","last_clean":"2023-10-06T05:32:24.160865+0000","last_became_active":"2023-10-06T05:32:23.169976+0000","last_became_peered":"2023-10-06T05:32:23.169976+0000","last_unstale":"2023-10-06T05:32:24.160865+0000","last_undegraded":"2023-10-06T05:32:24.160865+0000","last_fullsized":"2023-10-06T05:32:24.160865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:02.134772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.1","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.189500+0000","last_change":"2023-10-06T05:32:24.183880+0000","last_active":"2023-10-06T05:32:24.189500+0000","last_peered":"2023-10-06T05:32:24.189500+0000","last_clean":"2023-10-06T05:32:24.189500+0000","last_became_active":"2023-10-06T05:32:24.183583+0000","last_became_peered":"2023-10-06T05:32:24.183583+0000","last_unstale":"2023-10-06T05:32:24.189500+0000","last_undegraded":"2023-10-06T05:32:24.189500+0000","last_fullsized":"2023-10-06T05:32:24.189500+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:27.031700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171334+0000","last_change":"2023-10-06T05:32:23.173250+0000","last_active":"2023-10-06T05:32:24.171334+0000","last_peered":"2023-10-06T05:32:24.171334+0000","last_clean":"2023-10-06T05:32:24.171334+0000","last_became_active":"2023-10-06T05:32:23.172701+0000","last_became_peered":"2023-10-06T05:32:23.172701+0000","last_unstale":"2023-10-06T05:32:24.171334+0000","last_undegraded":"2023-10-06T05:32:24.171334+0000","last_fullsized":"2023-10-06T05:32:24.171334+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:23.816268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184252+0000","last_change":"2023-10-06T05:32:24.180112+0000","last_active":"2023-10-06T05:32:24.184252+0000","last_peered":"2023-10-06T05:32:24.184252+0000","last_clean":"2023-10-06T05:32:24.184252+0000","last_became_active":"2023-10-06T05:32:24.179874+0000","last_became_peered":"2023-10-06T05:32:24.179874+0000","last_unstale":"2023-10-06T05:32:24.184252+0000","last_undegraded":"2023-10-06T05:32:24.184252+0000","last_fullsized":"2023-10-06T05:32:24.184252+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:09.162284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166257+0000","last_change":"2023-10-06T05:32:23.182061+0000","last_active":"2023-10-06T05:32:24.166257+0000","last_peered":"2023-10-06T05:32:24.166257+0000","last_clean":"2023-10-06T05:32:24.166257+0000","last_became_active":"2023-10-06T05:32:23.181754+0000","last_became_peered":"2023-10-06T05:32:23.181754+0000","last_unstale":"2023-10-06T05:32:24.166257+0000","last_undegraded":"2023-10-06T05:32:24.166257+0000","last_fullsized":"2023-10-06T05:32:24.166257+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:56.425225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.3","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172030+0000","last_change":"2023-10-06T05:32:24.169794+0000","last_active":"2023-10-06T05:32:24.172030+0000","last_peered":"2023-10-06T05:32:24.172030+0000","last_clean":"2023-10-06T05:32:24.172030+0000","last_became_active":"2023-10-06T05:32:24.168940+0000","last_became_peered":"2023-10-06T05:32:24.168940+0000","last_unstale":"2023-10-06T05:32:24.172030+0000","last_undegraded":"2023-10-06T05:32:24.172030+0000","last_fullsized":"2023-10-06T05:32:24.172030+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:21.785939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165603+0000","last_change":"2023-10-06T05:32:23.171370+0000","last_active":"2023-10-06T05:32:24.165603+0000","last_peered":"2023-10-06T05:32:24.165603+0000","last_clean":"2023-10-06T05:32:24.165603+0000","last_became_active":"2023-10-06T05:32:23.171112+0000","last_became_peered":"2023-10-06T05:32:23.171112+0000","last_unstale":"2023-10-06T05:32:24.165603+0000","last_undegraded":"2023-10-06T05:32:24.165603+0000","last_fullsized":"2023-10-06T05:32:24.165603+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:33.934362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.20","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188631+0000","last_change":"2023-10-06T05:32:24.181985+0000","last_active":"2023-10-06T05:32:24.188631+0000","last_peered":"2023-10-06T05:32:24.188631+0000","last_clean":"2023-10-06T05:32:24.188631+0000","last_became_active":"2023-10-06T05:32:24.181507+0000","last_became_peered":"2023-10-06T05:32:24.181507+0000","last_unstale":"2023-10-06T05:32:24.188631+0000","last_undegraded":"2023-10-06T05:32:24.188631+0000","last_fullsized":"2023-10-06T05:32:24.188631+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:17.985878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":204,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.168338+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:32:24.168338+0000","last_peered":"2023-10-06T05:32:24.168338+0000","last_clean":"2023-10-06T05:32:24.168338+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:32:24.168338+0000","last_undegraded":"2023-10-06T05:32:24.168338+0000","last_fullsized":"2023-10-06T05:32:24.168338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.26","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.181947+0000","last_change":"2023-10-06T05:32:24.175849+0000","last_active":"2023-10-06T05:32:24.181947+0000","last_peered":"2023-10-06T05:32:24.181947+0000","last_clean":"2023-10-06T05:32:24.181947+0000","last_became_active":"2023-10-06T05:32:24.175274+0000","last_became_peered":"2023-10-06T05:32:24.175274+0000","last_unstale":"2023-10-06T05:32:24.181947+0000","last_undegraded":"2023-10-06T05:32:24.181947+0000","last_fullsized":"2023-10-06T05:32:24.181947+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:52.864938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":202,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.167330+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:32:24.167330+0000","last_peered":"2023-10-06T05:32:24.167330+0000","last_clean":"2023-10-06T05:32:24.167330+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:32:24.167330+0000","last_undegraded":"2023-10-06T05:32:24.167330+0000","last_fullsized":"2023-10-06T05:32:24.167330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.25","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191027+0000","last_change":"2023-10-06T05:32:24.186903+0000","last_active":"2023-10-06T05:32:24.191027+0000","last_peered":"2023-10-06T05:32:24.191027+0000","last_clean":"2023-10-06T05:32:24.191027+0000","last_became_active":"2023-10-06T05:32:24.186626+0000","last_became_peered":"2023-10-06T05:32:24.186626+0000","last_unstale":"2023-10-06T05:32:24.191027+0000","last_undegraded":"2023-10-06T05:32:24.191027+0000","last_fullsized":"2023-10-06T05:32:24.191027+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:38.075060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":206,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.160803+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:32:24.160803+0000","last_peered":"2023-10-06T05:32:24.160803+0000","last_clean":"2023-10-06T05:32:24.160803+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:32:24.160803+0000","last_undegraded":"2023-10-06T05:32:24.160803+0000","last_fullsized":"2023-10-06T05:32:24.160803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.27","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184298+0000","last_change":"2023-10-06T05:32:24.177118+0000","last_active":"2023-10-06T05:32:24.184298+0000","last_peered":"2023-10-06T05:32:24.184298+0000","last_clean":"2023-10-06T05:32:24.184298+0000","last_became_active":"2023-10-06T05:32:24.176867+0000","last_became_peered":"2023-10-06T05:32:24.176867+0000","last_unstale":"2023-10-06T05:32:24.184298+0000","last_undegraded":"2023-10-06T05:32:24.184298+0000","last_fullsized":"2023-10-06T05:32:24.184298+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:18.415285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":202,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166340+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:32:24.166340+0000","last_peered":"2023-10-06T05:32:24.166340+0000","last_clean":"2023-10-06T05:32:24.166340+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:32:24.166340+0000","last_undegraded":"2023-10-06T05:32:24.166340+0000","last_fullsized":"2023-10-06T05:32:24.166340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":351,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171835+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:32:24.171835+0000","last_peered":"2023-10-06T05:32:24.171835+0000","last_clean":"2023-10-06T05:32:24.171835+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:32:24.171835+0000","last_undegraded":"2023-10-06T05:32:24.171835+0000","last_fullsized":"2023-10-06T05:32:24.171835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.24","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.169724+0000","last_change":"2023-10-06T05:32:24.163962+0000","last_active":"2023-10-06T05:32:24.169724+0000","last_peered":"2023-10-06T05:32:24.169724+0000","last_clean":"2023-10-06T05:32:24.169724+0000","last_became_active":"2023-10-06T05:32:24.163662+0000","last_became_peered":"2023-10-06T05:32:24.163662+0000","last_unstale":"2023-10-06T05:32:24.169724+0000","last_undegraded":"2023-10-06T05:32:24.169724+0000","last_fullsized":"2023-10-06T05:32:24.169724+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:12.139749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171720+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:32:24.171720+0000","last_peered":"2023-10-06T05:32:24.171720+0000","last_clean":"2023-10-06T05:32:24.171720+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:32:24.171720+0000","last_undegraded":"2023-10-06T05:32:24.171720+0000","last_fullsized":"2023-10-06T05:32:24.171720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.21","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165311+0000","last_change":"2023-10-06T05:32:24.163938+0000","last_active":"2023-10-06T05:32:24.165311+0000","last_peered":"2023-10-06T05:32:24.165311+0000","last_clean":"2023-10-06T05:32:24.165311+0000","last_became_active":"2023-10-06T05:32:24.163546+0000","last_became_peered":"2023-10-06T05:32:24.163546+0000","last_unstale":"2023-10-06T05:32:24.165311+0000","last_undegraded":"2023-10-06T05:32:24.165311+0000","last_fullsized":"2023-10-06T05:32:24.165311+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:01.036713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172191+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:32:24.172191+0000","last_peered":"2023-10-06T05:32:24.172191+0000","last_clean":"2023-10-06T05:32:24.172191+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:32:24.172191+0000","last_undegraded":"2023-10-06T05:32:24.172191+0000","last_fullsized":"2023-10-06T05:32:24.172191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.22","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180424+0000","last_change":"2023-10-06T05:32:24.172713+0000","last_active":"2023-10-06T05:32:24.180424+0000","last_peered":"2023-10-06T05:32:24.180424+0000","last_clean":"2023-10-06T05:32:24.180424+0000","last_became_active":"2023-10-06T05:32:24.172391+0000","last_became_peered":"2023-10-06T05:32:24.172391+0000","last_unstale":"2023-10-06T05:32:24.180424+0000","last_undegraded":"2023-10-06T05:32:24.180424+0000","last_fullsized":"2023-10-06T05:32:24.180424+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:08.677384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165233+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:32:24.165233+0000","last_peered":"2023-10-06T05:32:24.165233+0000","last_clean":"2023-10-06T05:32:24.165233+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:32:24.165233+0000","last_undegraded":"2023-10-06T05:32:24.165233+0000","last_fullsized":"2023-10-06T05:32:24.165233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.23","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172891+0000","last_change":"2023-10-06T05:32:24.171903+0000","last_active":"2023-10-06T05:32:24.172891+0000","last_peered":"2023-10-06T05:32:24.172891+0000","last_clean":"2023-10-06T05:32:24.172891+0000","last_became_active":"2023-10-06T05:32:24.171304+0000","last_became_peered":"2023-10-06T05:32:24.171304+0000","last_unstale":"2023-10-06T05:32:24.172891+0000","last_undegraded":"2023-10-06T05:32:24.172891+0000","last_fullsized":"2023-10-06T05:32:24.172891+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:19.842503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.159300+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:32:24.159300+0000","last_peered":"2023-10-06T05:32:24.159300+0000","last_clean":"2023-10-06T05:32:24.159300+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:32:24.159300+0000","last_undegraded":"2023-10-06T05:32:24.159300+0000","last_fullsized":"2023-10-06T05:32:24.159300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":38,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":37,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575072,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305076,"kb_used_data":3692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066764,"statfs":{"total":96636764160,"available":96324366336,"internally_reserved":0,"allocated":3780608,"data_stored":2082022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.97399999999999998,"15min":1.1850000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68500000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.169},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.59999999999999998,"5min":0.42999999999999999,"15min":0.38900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0860000000000001,"15min":1.268},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66500000000000004,"15min":0.68200000000000005},"min":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96899999999999997,"5min":1.163,"15min":1.163},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.22},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.55000000000000004,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91800000000000004,"5min":1.0529999999999999,"15min":1.117},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.96399999999999997,"5min":1.0209999999999999,"15min":1.258},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.75800000000000001,"15min":0.73899999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.39400000000000002},"max":{"1min":0.997,"5min":1.234,"15min":1.2849999999999999},"last":0.40000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.999,"15min":1.109},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.45600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":1.1279999999999999,"15min":1.2290000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.169},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.99099999999999999,"5min":1.242,"15min":1.242},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304500,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067340,"statfs":{"total":96636764160,"available":96324956160,"internally_reserved":0,"allocated":3190784,"data_stored":1492208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90800000000000003,"5min":1.044,"15min":1.0720000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.42399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":1.137,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.135},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.442},"max":{"1min":1.075,"5min":1.115,"15min":1.115},"last":0.747}]},{"osd":3,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.70899999999999996},"min":{"1min":0.55000000000000004,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":1.1160000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.752,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.504,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.0049999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.53900000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91600000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.70999999999999996},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.4610000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.53900000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.995,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.93899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575071,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1491632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70999999999999996,"5min":0.91400000000000003,"15min":1.0149999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.83999999999999997,"5min":0.86099999999999999,"15min":0.88100000000000001},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61299999999999999,"5min":0.88800000000000001,"15min":1.165},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":0.76600000000000001,"15min":1.077},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.96099999999999997,"15min":1.2230000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.53400000000000003},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.2430000000000001},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77500000000000002,"15min":0.76100000000000001},"min":{"1min":0.60799999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96899999999999997,"5min":1.026,"15min":1.5389999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.45200000000000001},"max":{"1min":1.018,"5min":1.119,"15min":1.4690000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78500000000000003,"15min":0.78200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.47899999999999998},"max":{"1min":1.0760000000000001,"5min":1.248,"15min":1.25},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79100000000000004,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"max":{"1min":1.014,"5min":1.26,"15min":1.26},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.81699999999999995},"min":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.48199999999999998},"max":{"1min":1.069,"5min":1.2290000000000001,"15min":1.2909999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84199999999999997,"15min":0.82699999999999996},"min":{"1min":0.60099999999999998,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.111,"5min":1.2849999999999999,"15min":1.3280000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.68700000000000006,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.018,"5min":1.1299999999999999,"15min":2.6240000000000001},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575071,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309172,"kb_used_data":3692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062668,"statfs":{"total":96636764160,"available":96320172032,"internally_reserved":0,"allocated":3780608,"data_stored":2082090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.245},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.85199999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.498},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82099999999999995},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.504,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.70799999999999996,"5min":0.81899999999999995,"15min":0.90800000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.84999999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.58599999999999997,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.499},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.88400000000000001,"15min":0.90200000000000002},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.47299999999999998},"max":{"1min":0.997,"5min":1.05,"15min":2.8220000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.622,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.113,"5min":1.244,"15min":3.6629999999999998},"last":0.96499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.65700000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"max":{"1min":1.1060000000000001,"5min":1.27,"15min":1.27},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.59499999999999997,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":1.0389999999999999,"5min":1.097,"15min":1.1970000000000001},"last":0.97899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0580000000000001,"5min":1.093,"15min":1.1839999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.45500000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.2629999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80000000000000004,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.25},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.66300000000000003,"5min":0.55500000000000005,"15min":0.50600000000000001},"max":{"1min":1.1459999999999999,"5min":1.167,"15min":1.2789999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575071,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304508,"kb_used_data":3124,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067332,"statfs":{"total":96636764160,"available":96324947968,"internally_reserved":0,"allocated":3198976,"data_stored":1493238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.45200000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.056},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70999999999999996,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.0649999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":1.073},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.96499999999999997},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.486,"15min":1.486},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.76900000000000002},"min":{"1min":0.62,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":1.5,"15min":1.5},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74399999999999999,"15min":0.76000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1639999999999999},"last":1.016},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.79200000000000004},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1859999999999999},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.626,"5min":0.504,"15min":0.504},"max":{"1min":1.014,"5min":1.014,"15min":1.1830000000000001},"last":1.22},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.137},"last":1.161}]},{"osd":7,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78200000000000003,"15min":0.79700000000000004},"min":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.57499999999999996},"max":{"1min":1.0229999999999999,"5min":1.538,"15min":1.538},"last":1.282},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.80500000000000005},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.51600000000000001},"max":{"1min":0.96199999999999997,"5min":1.014,"15min":1.2},"last":1.264}]}]},{"osd":3,"up_from":13,"seq":55834575072,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308592,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063248,"statfs":{"total":96636764160,"available":96320765952,"internally_reserved":0,"allocated":3186688,"data_stored":1491666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.748,"5min":0.76700000000000002,"15min":0.89200000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.29699999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69799999999999995,"5min":1.016,"15min":1.369},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70199999999999996,"5min":0.97899999999999998,"15min":1.5369999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.90200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.90400000000000003},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.46000000000000002},"max":{"1min":0.93100000000000005,"5min":1.1100000000000001,"15min":1.2030000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.68600000000000005,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":1.1080000000000001,"15min":1.3009999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76200000000000001,"15min":0.78100000000000003},"min":{"1min":0.67300000000000004,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":0.94499999999999995,"5min":1.111,"15min":1.1539999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.79200000000000004},"min":{"1min":0.67600000000000005,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":1.0049999999999999,"5min":1.0820000000000001,"15min":1.4590000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78000000000000003,"15min":0.79500000000000004},"min":{"1min":0.62,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.3360000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.79800000000000004},"min":{"1min":0.59999999999999998,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":0.90900000000000003,"5min":1.1339999999999999,"15min":1.2390000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.72299999999999998,"5min":0.57299999999999995,"15min":0.57099999999999995},"max":{"1min":0.999,"5min":1.0669999999999999,"15min":1.194},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.81200000000000006},"min":{"1min":0.65800000000000003,"5min":0.53700000000000003,"15min":0.53000000000000003},"max":{"1min":0.95799999999999996,"5min":1.0980000000000001,"15min":1.2090000000000001},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304516,"kb_used_data":3132,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067324,"statfs":{"total":96636764160,"available":96324939776,"internally_reserved":0,"allocated":3207168,"data_stored":1493882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0249999999999999},"last":5.3070000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97299999999999998,"5min":1.0369999999999999,"15min":1.05},"last":5.6299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.68700000000000006},"min":{"1min":0.52700000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.99099999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":3.9609999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":0.98899999999999999,"15min":1.016},"last":4.2169999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70099999999999996,"15min":0.71599999999999997},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.99099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.89400000000000002,"5min":1.115,"15min":1.115},"last":3.839},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.018},"last":4.4050000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":2.5049999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":1.163,"15min":1.163},"last":6.5259999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81299999999999994,"5min":1.0189999999999999,"15min":1.0920000000000001},"last":7.9459999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45300000000000001},"max":{"1min":0.86499999999999999,"5min":1.0489999999999999,"15min":1.0609999999999999},"last":0.621}]},{"osd":7,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0169999999999999,"5min":1.282,"15min":1.282},"last":1.109},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95799999999999996,"5min":1.254,"15min":1.254},"last":3.6619999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575072,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304496,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067344,"statfs":{"total":96636764160,"available":96324960256,"internally_reserved":0,"allocated":3186688,"data_stored":1491666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.69199999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":1.127,"15min":2.4609999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39800000000000002},"max":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":1.3460000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.054,"5min":1.093,"15min":1.2130000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.0289999999999999,"15min":1.204},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":0.95599999999999996,"15min":1.002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.433},"max":{"1min":1.075,"5min":1.145,"15min":1.2769999999999999},"last":0.878}]},{"osd":4,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.63700000000000001},"min":{"1min":0.52100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.96599999999999997,"15min":1.0229999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.56000000000000005,"5min":0.442,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.02,"15min":1.1539999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.872,"5min":0.91500000000000004,"15min":1.141},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.60799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.59499999999999997,"5min":0.51200000000000001,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.248},"last":0.90500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:32:29.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:32:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.785+0000 7fb409b9d700 1 Processor -- start 2023-10-06T05:32:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.785+0000 7fb409b9d700 1 -- start start 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb409b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 0x7fb40406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb409b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4040c9cb0 con 0x7fb40413faa0 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb409b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4040ca270 con 0x7fb4040c8050 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb409b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406b840 con 0x7fb4040cb710 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb408b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb40413faa0 0x7fb4040c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43082/0 (socket says 172.21.15.33:43082) 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb408b9b700 1 -- 172.21.15.33:0/319621806 learned_addr learned my addr 172.21.15.33:0/319621806 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.787+0000 7fb40939c700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 0x7fb40406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633424177 0 0) 0x7fb4040c9cb0 con 0x7fb40413faa0 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f4003570 con 0x7fb40413faa0 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2999769073 0 0) 0x7fb3f4003570 con 0x7fb40413faa0 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 msgr2=0x7fb40406b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 0x7fb40406b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 >> v1:172.21.15.167:6789/0 conn(0x7fb4040c8050 legacy=0x7fb40406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40406bb20 con 0x7fb40413faa0 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb3f8001470 con 0x7fb40413faa0 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 >> v1:172.21.15.33:6789/0 conn(0x7fb40413faa0 legacy=0x7fb4040c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f4003a90 0x7fb3f4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 0x7fb3f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3f4004d50 0x7fb3f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406b840 con 0x7fb3f4003a90 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f400bb40 con 0x7fb3f4004d50 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f400df10 con 0x7fb4040cb710 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb408b9b700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f4003a90 0x7fb3f4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb40939c700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3f4004d50 0x7fb3f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.788+0000 7fb403fff700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 0x7fb3f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb408b9b700 1 -- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 msgr2=0x7fb3f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb408b9b700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4040cb710 0x7fb3f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb408b9b700 1 -- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3f4004d50 msgr2=0x7fb3f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb408b9b700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3f4004d50 0x7fb3f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb408b9b700 1 -- 172.21.15.33:0/319621806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4003570 con 0x7fb3f4003a90 2023-10-06T05:32:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb40939c700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3f4004d50 0x7fb3f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb408b9b700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f4003a90 0x7fb3f4003e60 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c2a0 tx=0x7fb40406be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942ef6dc5de894c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800cb70 con 0x7fb3f4003a90 2023-10-06T05:32:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f800ccf0 con 0x7fb3f4003a90 2023-10-06T05:32:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.789+0000 7fb4037fe700 1 -- 172.21.15.33:0/319621806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8005b70 con 0x7fb3f4003a90 2023-10-06T05:32:29.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb409b9d700 1 -- 172.21.15.33:0/319621806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f4003a90 msgr2=0x7fb3f4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb409b9d700 1 --2- 172.21.15.33:0/319621806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3f4003a90 0x7fb3f4003e60 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c2a0 tx=0x7fb40406be00 comp rx=0 tx=0).stop 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb40939c700 1 -- 172.21.15.33:0/319621806 reap_dead start 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb409b9d700 1 -- 172.21.15.33:0/319621806 shutdown_connections 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb409b9d700 1 -- 172.21.15.33:0/319621806 >> 172.21.15.33:0/319621806 conn(0x7fb40406d030 msgr2=0x7fb40406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb409b9d700 1 -- 172.21.15.33:0/319621806 shutdown_connections 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.790+0000 7fb409b9d700 1 -- 172.21.15.33:0/319621806 wait complete. 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.791+0000 7fb409b9d700 1 Processor -- start 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.791+0000 7fb409b9d700 1 -- start start 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.792+0000 7fb409b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3f4003a90 0x7fb4040b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.792+0000 7fb409b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 0x7fb40416ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.792+0000 7fb409b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4041597c0 0x7fb404155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.792+0000 7fb409b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40413beb0 con 0x7fb404169a70 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.792+0000 7fb409b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb404169ee0 con 0x7fb4041597c0 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.792+0000 7fb409b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40416aa10 con 0x7fb3f4003a90 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 0x7fb40416ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 0x7fb40416ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36982/0 (socket says 172.21.15.33:36982) 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 -- 172.21.15.33:0/583333694 learned_addr learned my addr 172.21.15.33:0/583333694 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb408b9b700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3f4003a90 0x7fb4040b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb40939c700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4041597c0 0x7fb404155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 -- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3f4003a90 msgr2=0x7fb4040b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3f4003a90 0x7fb4040b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 -- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4041597c0 msgr2=0x7fb404155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4041597c0 0x7fb404155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 -- 172.21.15.33:0/583333694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40416a170 con 0x7fb404169a70 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb40939c700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4041597c0 0x7fb404155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.793+0000 7fb403fff700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 0x7fb40416ac40 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000e860 tx=0x7fb3f0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.794+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0013630 con 0x7fb404169a70 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.794+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f00137b0 con 0x7fb404169a70 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.794+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40416a450 con 0x7fb404169a70 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.794+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0013b70 con 0x7fb404169a70 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.794+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40416b180 con 0x7fb404169a70 2023-10-06T05:32:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.795+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4040698c0 con 0x7fb404169a70 2023-10-06T05:32:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.796+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3f001c400 con 0x7fb404169a70 2023-10-06T05:32:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.797+0000 7fb400ff9700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3e8066f30 0x7fb3e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.797+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb3f0084c20 con 0x7fb404169a70 2023-10-06T05:32:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.797+0000 7fb408b9b700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3e8066f30 0x7fb3e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.798+0000 7fb408b9b700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3e8066f30 0x7fb3e80693e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007c20 tx=0x7fb3f8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.801+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3f0051d30 con 0x7fb404169a70 2023-10-06T05:32:29.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:29.997+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb40404c380 con 0x7fb3e8066f30 2023-10-06T05:32:30.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.006+0000 7fb400ff9700 1 -- 172.21.15.33:0/583333694 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311760 (secure 0 0 0) 0x7fb40404c380 con 0x7fb3e8066f30 2023-10-06T05:32:30.012 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3e8066f30 msgr2=0x7fb3e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3e8066f30 0x7fb3e80693e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007c20 tx=0x7fb3f8015000 comp rx=0 tx=0).stop 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 msgr2=0x7fb40416ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 0x7fb40416ac40 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000e860 tx=0x7fb3f0009040 comp rx=0 tx=0).stop 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 shutdown_connections 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb3e8066f30 0x7fb3e80693e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4041597c0 0x7fb404155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3f4003a90 0x7fb4040b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 --2- 172.21.15.33:0/583333694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb404169a70 0x7fb40416ac40 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 >> 172.21.15.33:0/583333694 conn(0x7fb40406d030 msgr2=0x7fb40406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 shutdown_connections 2023-10-06T05:32:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.011+0000 7fb409b9d700 1 -- 172.21.15.33:0/583333694 wait complete. 2023-10-06T05:32:30.016 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:32:30.031 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":792,"stamp":"2023-10-06T05:32:28.094049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445352,"kb_used_data":26088,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752529368,"statfs":{"total":773094113280,"available":770590072832,"internally_reserved":0,"allocated":26714112,"data_stored":13118404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.938804"},"pg_stats":[{"pgid":"38.3f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179380+0000","last_change":"2023-10-06T05:32:24.178326+0000","last_active":"2023-10-06T05:32:24.179380+0000","last_peered":"2023-10-06T05:32:24.179380+0000","last_clean":"2023-10-06T05:32:24.179380+0000","last_became_active":"2023-10-06T05:32:24.178049+0000","last_became_peered":"2023-10-06T05:32:24.178049+0000","last_unstale":"2023-10-06T05:32:24.179380+0000","last_undegraded":"2023-10-06T05:32:24.179380+0000","last_fullsized":"2023-10-06T05:32:24.179380+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:49.257644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.3e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180491+0000","last_change":"2023-10-06T05:32:24.173543+0000","last_active":"2023-10-06T05:32:24.180491+0000","last_peered":"2023-10-06T05:32:24.180491+0000","last_clean":"2023-10-06T05:32:24.180491+0000","last_became_active":"2023-10-06T05:32:24.173283+0000","last_became_peered":"2023-10-06T05:32:24.173283+0000","last_unstale":"2023-10-06T05:32:24.180491+0000","last_undegraded":"2023-10-06T05:32:24.180491+0000","last_fullsized":"2023-10-06T05:32:24.180491+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:24.119971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.3d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176954+0000","last_change":"2023-10-06T05:32:24.173585+0000","last_active":"2023-10-06T05:32:24.176954+0000","last_peered":"2023-10-06T05:32:24.176954+0000","last_clean":"2023-10-06T05:32:24.176954+0000","last_became_active":"2023-10-06T05:32:24.173336+0000","last_became_peered":"2023-10-06T05:32:24.173336+0000","last_unstale":"2023-10-06T05:32:24.176954+0000","last_undegraded":"2023-10-06T05:32:24.176954+0000","last_fullsized":"2023-10-06T05:32:24.176954+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:27.592412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.3c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190851+0000","last_change":"2023-10-06T05:32:24.184186+0000","last_active":"2023-10-06T05:32:24.190851+0000","last_peered":"2023-10-06T05:32:24.190851+0000","last_clean":"2023-10-06T05:32:24.190851+0000","last_became_active":"2023-10-06T05:32:24.183928+0000","last_became_peered":"2023-10-06T05:32:24.183928+0000","last_unstale":"2023-10-06T05:32:24.190851+0000","last_undegraded":"2023-10-06T05:32:24.190851+0000","last_fullsized":"2023-10-06T05:32:24.190851+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:51.146580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.3b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176077+0000","last_change":"2023-10-06T05:32:24.173601+0000","last_active":"2023-10-06T05:32:24.176077+0000","last_peered":"2023-10-06T05:32:24.176077+0000","last_clean":"2023-10-06T05:32:24.176077+0000","last_became_active":"2023-10-06T05:32:24.173061+0000","last_became_peered":"2023-10-06T05:32:24.173061+0000","last_unstale":"2023-10-06T05:32:24.176077+0000","last_undegraded":"2023-10-06T05:32:24.176077+0000","last_fullsized":"2023-10-06T05:32:24.176077+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:05.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.3a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186495+0000","last_change":"2023-10-06T05:32:24.177953+0000","last_active":"2023-10-06T05:32:24.186495+0000","last_peered":"2023-10-06T05:32:24.186495+0000","last_clean":"2023-10-06T05:32:24.186495+0000","last_became_active":"2023-10-06T05:32:24.177398+0000","last_became_peered":"2023-10-06T05:32:24.177398+0000","last_unstale":"2023-10-06T05:32:24.186495+0000","last_undegraded":"2023-10-06T05:32:24.186495+0000","last_fullsized":"2023-10-06T05:32:24.186495+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:14.210044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.39","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179497+0000","last_change":"2023-10-06T05:32:24.174895+0000","last_active":"2023-10-06T05:32:24.179497+0000","last_peered":"2023-10-06T05:32:24.179497+0000","last_clean":"2023-10-06T05:32:24.179497+0000","last_became_active":"2023-10-06T05:32:24.174641+0000","last_became_peered":"2023-10-06T05:32:24.174641+0000","last_unstale":"2023-10-06T05:32:24.179497+0000","last_undegraded":"2023-10-06T05:32:24.179497+0000","last_fullsized":"2023-10-06T05:32:24.179497+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:55:21.024627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.38","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.170065+0000","last_change":"2023-10-06T05:32:24.166809+0000","last_active":"2023-10-06T05:32:24.170065+0000","last_peered":"2023-10-06T05:32:24.170065+0000","last_clean":"2023-10-06T05:32:24.170065+0000","last_became_active":"2023-10-06T05:32:24.166477+0000","last_became_peered":"2023-10-06T05:32:24.166477+0000","last_unstale":"2023-10-06T05:32:24.170065+0000","last_undegraded":"2023-10-06T05:32:24.170065+0000","last_fullsized":"2023-10-06T05:32:24.170065+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:32.498083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.37","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.181804+0000","last_change":"2023-10-06T05:32:24.177469+0000","last_active":"2023-10-06T05:32:24.181804+0000","last_peered":"2023-10-06T05:32:24.181804+0000","last_clean":"2023-10-06T05:32:24.181804+0000","last_became_active":"2023-10-06T05:32:24.176411+0000","last_became_peered":"2023-10-06T05:32:24.176411+0000","last_unstale":"2023-10-06T05:32:24.181804+0000","last_undegraded":"2023-10-06T05:32:24.181804+0000","last_fullsized":"2023-10-06T05:32:24.181804+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:42.040536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.36","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184246+0000","last_change":"2023-10-06T05:32:24.176913+0000","last_active":"2023-10-06T05:32:24.184246+0000","last_peered":"2023-10-06T05:32:24.184246+0000","last_clean":"2023-10-06T05:32:24.184246+0000","last_became_active":"2023-10-06T05:32:24.176605+0000","last_became_peered":"2023-10-06T05:32:24.176605+0000","last_unstale":"2023-10-06T05:32:24.184246+0000","last_undegraded":"2023-10-06T05:32:24.184246+0000","last_fullsized":"2023-10-06T05:32:24.184246+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:43.836336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.35","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.163564+0000","last_change":"2023-10-06T05:32:24.162776+0000","last_active":"2023-10-06T05:32:24.163564+0000","last_peered":"2023-10-06T05:32:24.163564+0000","last_clean":"2023-10-06T05:32:24.163564+0000","last_became_active":"2023-10-06T05:32:24.162492+0000","last_became_peered":"2023-10-06T05:32:24.162492+0000","last_unstale":"2023-10-06T05:32:24.163564+0000","last_undegraded":"2023-10-06T05:32:24.163564+0000","last_fullsized":"2023-10-06T05:32:24.163564+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:51.748940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.34","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179890+0000","last_change":"2023-10-06T05:32:24.174493+0000","last_active":"2023-10-06T05:32:24.179890+0000","last_peered":"2023-10-06T05:32:24.179890+0000","last_clean":"2023-10-06T05:32:24.179890+0000","last_became_active":"2023-10-06T05:32:24.174213+0000","last_became_peered":"2023-10-06T05:32:24.174213+0000","last_unstale":"2023-10-06T05:32:24.179890+0000","last_undegraded":"2023-10-06T05:32:24.179890+0000","last_fullsized":"2023-10-06T05:32:24.179890+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:42.202777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.33","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176055+0000","last_change":"2023-10-06T05:32:24.173599+0000","last_active":"2023-10-06T05:32:24.176055+0000","last_peered":"2023-10-06T05:32:24.176055+0000","last_clean":"2023-10-06T05:32:24.176055+0000","last_became_active":"2023-10-06T05:32:24.173113+0000","last_became_peered":"2023-10-06T05:32:24.173113+0000","last_unstale":"2023-10-06T05:32:24.176055+0000","last_undegraded":"2023-10-06T05:32:24.176055+0000","last_fullsized":"2023-10-06T05:32:24.176055+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:17.311578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.32","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186764+0000","last_change":"2023-10-06T05:32:24.176658+0000","last_active":"2023-10-06T05:32:24.186764+0000","last_peered":"2023-10-06T05:32:24.186764+0000","last_clean":"2023-10-06T05:32:24.186764+0000","last_became_active":"2023-10-06T05:32:24.176385+0000","last_became_peered":"2023-10-06T05:32:24.176385+0000","last_unstale":"2023-10-06T05:32:24.186764+0000","last_undegraded":"2023-10-06T05:32:24.186764+0000","last_fullsized":"2023-10-06T05:32:24.186764+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:40.439510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.31","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186606+0000","last_change":"2023-10-06T05:32:24.175674+0000","last_active":"2023-10-06T05:32:24.186606+0000","last_peered":"2023-10-06T05:32:24.186606+0000","last_clean":"2023-10-06T05:32:24.186606+0000","last_became_active":"2023-10-06T05:32:24.175447+0000","last_became_peered":"2023-10-06T05:32:24.175447+0000","last_unstale":"2023-10-06T05:32:24.186606+0000","last_undegraded":"2023-10-06T05:32:24.186606+0000","last_fullsized":"2023-10-06T05:32:24.186606+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:40.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.30","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.189182+0000","last_change":"2023-10-06T05:32:24.185196+0000","last_active":"2023-10-06T05:32:24.189182+0000","last_peered":"2023-10-06T05:32:24.189182+0000","last_clean":"2023-10-06T05:32:24.189182+0000","last_became_active":"2023-10-06T05:32:24.184800+0000","last_became_peered":"2023-10-06T05:32:24.184800+0000","last_unstale":"2023-10-06T05:32:24.189182+0000","last_undegraded":"2023-10-06T05:32:24.189182+0000","last_fullsized":"2023-10-06T05:32:24.189182+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:40.528706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.2f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.179588+0000","last_change":"2023-10-06T05:32:24.175329+0000","last_active":"2023-10-06T05:32:24.179588+0000","last_peered":"2023-10-06T05:32:24.179588+0000","last_clean":"2023-10-06T05:32:24.179588+0000","last_became_active":"2023-10-06T05:32:24.175073+0000","last_became_peered":"2023-10-06T05:32:24.175073+0000","last_unstale":"2023-10-06T05:32:24.179588+0000","last_undegraded":"2023-10-06T05:32:24.179588+0000","last_fullsized":"2023-10-06T05:32:24.179588+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:47.162704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.2e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191121+0000","last_change":"2023-10-06T05:32:24.188476+0000","last_active":"2023-10-06T05:32:24.191121+0000","last_peered":"2023-10-06T05:32:24.191121+0000","last_clean":"2023-10-06T05:32:24.191121+0000","last_became_active":"2023-10-06T05:32:24.188210+0000","last_became_peered":"2023-10-06T05:32:24.188210+0000","last_unstale":"2023-10-06T05:32:24.191121+0000","last_undegraded":"2023-10-06T05:32:24.191121+0000","last_fullsized":"2023-10-06T05:32:24.191121+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:39.134144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.175851+0000","last_change":"2023-10-06T05:32:24.173160+0000","last_active":"2023-10-06T05:32:24.175851+0000","last_peered":"2023-10-06T05:32:24.175851+0000","last_clean":"2023-10-06T05:32:24.175851+0000","last_became_active":"2023-10-06T05:32:24.172873+0000","last_became_peered":"2023-10-06T05:32:24.172873+0000","last_unstale":"2023-10-06T05:32:24.175851+0000","last_undegraded":"2023-10-06T05:32:24.175851+0000","last_fullsized":"2023-10-06T05:32:24.175851+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:40.366871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.2c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.193416+0000","last_change":"2023-10-06T05:32:24.189108+0000","last_active":"2023-10-06T05:32:24.193416+0000","last_peered":"2023-10-06T05:32:24.193416+0000","last_clean":"2023-10-06T05:32:24.193416+0000","last_became_active":"2023-10-06T05:32:24.188656+0000","last_became_peered":"2023-10-06T05:32:24.188656+0000","last_unstale":"2023-10-06T05:32:24.193416+0000","last_undegraded":"2023-10-06T05:32:24.193416+0000","last_fullsized":"2023-10-06T05:32:24.193416+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:20.540977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191947+0000","last_change":"2023-10-06T05:32:24.187327+0000","last_active":"2023-10-06T05:32:24.191947+0000","last_peered":"2023-10-06T05:32:24.191947+0000","last_clean":"2023-10-06T05:32:24.191947+0000","last_became_active":"2023-10-06T05:32:24.187090+0000","last_became_peered":"2023-10-06T05:32:24.187090+0000","last_unstale":"2023-10-06T05:32:24.191947+0000","last_undegraded":"2023-10-06T05:32:24.191947+0000","last_fullsized":"2023-10-06T05:32:24.191947+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:31.922663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188699+0000","last_change":"2023-10-06T05:32:24.181600+0000","last_active":"2023-10-06T05:32:24.188699+0000","last_peered":"2023-10-06T05:32:24.188699+0000","last_clean":"2023-10-06T05:32:24.188699+0000","last_became_active":"2023-10-06T05:32:24.181352+0000","last_became_peered":"2023-10-06T05:32:24.181352+0000","last_unstale":"2023-10-06T05:32:24.188699+0000","last_undegraded":"2023-10-06T05:32:24.188699+0000","last_fullsized":"2023-10-06T05:32:24.188699+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:11.081631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.29","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191047+0000","last_change":"2023-10-06T05:32:24.187834+0000","last_active":"2023-10-06T05:32:24.191047+0000","last_peered":"2023-10-06T05:32:24.191047+0000","last_clean":"2023-10-06T05:32:24.191047+0000","last_became_active":"2023-10-06T05:32:24.186942+0000","last_became_peered":"2023-10-06T05:32:24.186942+0000","last_unstale":"2023-10-06T05:32:24.191047+0000","last_undegraded":"2023-10-06T05:32:24.191047+0000","last_fullsized":"2023-10-06T05:32:24.191047+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:41.818468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.28","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.177748+0000","last_change":"2023-10-06T05:32:24.174548+0000","last_active":"2023-10-06T05:32:24.177748+0000","last_peered":"2023-10-06T05:32:24.177748+0000","last_clean":"2023-10-06T05:32:24.177748+0000","last_became_active":"2023-10-06T05:32:24.174294+0000","last_became_peered":"2023-10-06T05:32:24.174294+0000","last_unstale":"2023-10-06T05:32:24.177748+0000","last_undegraded":"2023-10-06T05:32:24.177748+0000","last_fullsized":"2023-10-06T05:32:24.177748+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.426781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.193434+0000","last_change":"2023-10-06T05:32:24.189635+0000","last_active":"2023-10-06T05:32:24.193434+0000","last_peered":"2023-10-06T05:32:24.193434+0000","last_clean":"2023-10-06T05:32:24.193434+0000","last_became_active":"2023-10-06T05:32:24.189018+0000","last_became_peered":"2023-10-06T05:32:24.189018+0000","last_unstale":"2023-10-06T05:32:24.193434+0000","last_undegraded":"2023-10-06T05:32:24.193434+0000","last_fullsized":"2023-10-06T05:32:24.193434+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:58.350917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1f","version":"251'4","reported_seq":14,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.198038+0000","last_change":"2023-10-06T05:32:23.178167+0000","last_active":"2023-10-06T05:32:25.198038+0000","last_peered":"2023-10-06T05:32:25.198038+0000","last_clean":"2023-10-06T05:32:25.198038+0000","last_became_active":"2023-10-06T05:32:23.177689+0000","last_became_peered":"2023-10-06T05:32:23.177689+0000","last_unstale":"2023-10-06T05:32:25.198038+0000","last_undegraded":"2023-10-06T05:32:25.198038+0000","last_fullsized":"2023-10-06T05:32:25.198038+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:15.328039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.1d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166685+0000","last_change":"2023-10-06T05:32:24.163468+0000","last_active":"2023-10-06T05:32:24.166685+0000","last_peered":"2023-10-06T05:32:24.166685+0000","last_clean":"2023-10-06T05:32:24.166685+0000","last_became_active":"2023-10-06T05:32:24.163139+0000","last_became_peered":"2023-10-06T05:32:24.163139+0000","last_unstale":"2023-10-06T05:32:24.166685+0000","last_undegraded":"2023-10-06T05:32:24.166685+0000","last_fullsized":"2023-10-06T05:32:24.166685+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:12:47.306682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1e","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195092+0000","last_change":"2023-10-06T05:32:23.180853+0000","last_active":"2023-10-06T05:32:25.195092+0000","last_peered":"2023-10-06T05:32:25.195092+0000","last_clean":"2023-10-06T05:32:25.195092+0000","last_became_active":"2023-10-06T05:32:23.180531+0000","last_became_peered":"2023-10-06T05:32:23.180531+0000","last_unstale":"2023-10-06T05:32:25.195092+0000","last_undegraded":"2023-10-06T05:32:25.195092+0000","last_fullsized":"2023-10-06T05:32:25.195092+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:14.597456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.1e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186830+0000","last_change":"2023-10-06T05:32:24.176321+0000","last_active":"2023-10-06T05:32:24.186830+0000","last_peered":"2023-10-06T05:32:24.186830+0000","last_clean":"2023-10-06T05:32:24.186830+0000","last_became_active":"2023-10-06T05:32:24.175989+0000","last_became_peered":"2023-10-06T05:32:24.175989+0000","last_unstale":"2023-10-06T05:32:24.186830+0000","last_undegraded":"2023-10-06T05:32:24.186830+0000","last_fullsized":"2023-10-06T05:32:24.186830+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:30.285227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.200383+0000","last_change":"2023-10-06T05:32:23.176926+0000","last_active":"2023-10-06T05:32:25.200383+0000","last_peered":"2023-10-06T05:32:25.200383+0000","last_clean":"2023-10-06T05:32:25.200383+0000","last_became_active":"2023-10-06T05:32:23.176644+0000","last_became_peered":"2023-10-06T05:32:23.176644+0000","last_unstale":"2023-10-06T05:32:25.200383+0000","last_undegraded":"2023-10-06T05:32:25.200383+0000","last_fullsized":"2023-10-06T05:32:25.200383+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:13.577381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.189554+0000","last_change":"2023-10-06T05:32:24.177492+0000","last_active":"2023-10-06T05:32:24.189554+0000","last_peered":"2023-10-06T05:32:24.189554+0000","last_clean":"2023-10-06T05:32:24.189554+0000","last_became_active":"2023-10-06T05:32:24.177174+0000","last_became_peered":"2023-10-06T05:32:24.177174+0000","last_unstale":"2023-10-06T05:32:24.189554+0000","last_undegraded":"2023-10-06T05:32:24.189554+0000","last_fullsized":"2023-10-06T05:32:24.189554+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:52.864938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.198212+0000","last_change":"2023-10-06T05:32:23.174158+0000","last_active":"2023-10-06T05:32:25.198212+0000","last_peered":"2023-10-06T05:32:25.198212+0000","last_clean":"2023-10-06T05:32:25.198212+0000","last_became_active":"2023-10-06T05:32:23.173837+0000","last_became_peered":"2023-10-06T05:32:23.173837+0000","last_unstale":"2023-10-06T05:32:25.198212+0000","last_undegraded":"2023-10-06T05:32:25.198212+0000","last_fullsized":"2023-10-06T05:32:25.198212+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:43.730015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.18","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186689+0000","last_change":"2023-10-06T05:32:24.181069+0000","last_active":"2023-10-06T05:32:24.186689+0000","last_peered":"2023-10-06T05:32:24.186689+0000","last_clean":"2023-10-06T05:32:24.186689+0000","last_became_active":"2023-10-06T05:32:24.180823+0000","last_became_peered":"2023-10-06T05:32:24.180823+0000","last_unstale":"2023-10-06T05:32:24.186689+0000","last_undegraded":"2023-10-06T05:32:24.186689+0000","last_fullsized":"2023-10-06T05:32:24.186689+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:58.435463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1b","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.193541+0000","last_change":"2023-10-06T05:32:23.181503+0000","last_active":"2023-10-06T05:32:25.193541+0000","last_peered":"2023-10-06T05:32:25.193541+0000","last_clean":"2023-10-06T05:32:25.193541+0000","last_became_active":"2023-10-06T05:32:23.181150+0000","last_became_peered":"2023-10-06T05:32:23.181150+0000","last_unstale":"2023-10-06T05:32:25.193541+0000","last_undegraded":"2023-10-06T05:32:25.193541+0000","last_fullsized":"2023-10-06T05:32:25.193541+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:10.074101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.19","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171078+0000","last_change":"2023-10-06T05:32:24.168394+0000","last_active":"2023-10-06T05:32:24.171078+0000","last_peered":"2023-10-06T05:32:24.171078+0000","last_clean":"2023-10-06T05:32:24.171078+0000","last_became_active":"2023-10-06T05:32:24.168128+0000","last_became_peered":"2023-10-06T05:32:24.168128+0000","last_unstale":"2023-10-06T05:32:24.171078+0000","last_undegraded":"2023-10-06T05:32:24.171078+0000","last_fullsized":"2023-10-06T05:32:24.171078+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:53.832324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1a","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195860+0000","last_change":"2023-10-06T05:32:23.166513+0000","last_active":"2023-10-06T05:32:25.195860+0000","last_peered":"2023-10-06T05:32:25.195860+0000","last_clean":"2023-10-06T05:32:25.195860+0000","last_became_active":"2023-10-06T05:32:23.166258+0000","last_became_peered":"2023-10-06T05:32:23.166258+0000","last_unstale":"2023-10-06T05:32:25.195860+0000","last_undegraded":"2023-10-06T05:32:25.195860+0000","last_fullsized":"2023-10-06T05:32:25.195860+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:48.894291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188682+0000","last_change":"2023-10-06T05:32:24.180189+0000","last_active":"2023-10-06T05:32:24.188682+0000","last_peered":"2023-10-06T05:32:24.188682+0000","last_clean":"2023-10-06T05:32:24.188682+0000","last_became_active":"2023-10-06T05:32:24.179939+0000","last_became_peered":"2023-10-06T05:32:24.179939+0000","last_unstale":"2023-10-06T05:32:24.188682+0000","last_undegraded":"2023-10-06T05:32:24.188682+0000","last_fullsized":"2023-10-06T05:32:24.188682+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:17.068575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171239+0000","last_change":"2023-10-06T05:32:23.174242+0000","last_active":"2023-10-06T05:32:24.171239+0000","last_peered":"2023-10-06T05:32:24.171239+0000","last_clean":"2023-10-06T05:32:24.171239+0000","last_became_active":"2023-10-06T05:32:23.173878+0000","last_became_peered":"2023-10-06T05:32:23.173878+0000","last_unstale":"2023-10-06T05:32:24.171239+0000","last_undegraded":"2023-10-06T05:32:24.171239+0000","last_fullsized":"2023-10-06T05:32:24.171239+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:57.831472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.1b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180411+0000","last_change":"2023-10-06T05:32:24.173615+0000","last_active":"2023-10-06T05:32:24.180411+0000","last_peered":"2023-10-06T05:32:24.180411+0000","last_clean":"2023-10-06T05:32:24.180411+0000","last_became_active":"2023-10-06T05:32:24.173356+0000","last_became_peered":"2023-10-06T05:32:24.173356+0000","last_unstale":"2023-10-06T05:32:24.180411+0000","last_undegraded":"2023-10-06T05:32:24.180411+0000","last_fullsized":"2023-10-06T05:32:24.180411+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:43.048666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.158804+0000","last_change":"2023-10-06T05:32:23.179905+0000","last_active":"2023-10-06T05:32:24.158804+0000","last_peered":"2023-10-06T05:32:24.158804+0000","last_clean":"2023-10-06T05:32:24.158804+0000","last_became_active":"2023-10-06T05:32:23.179020+0000","last_became_peered":"2023-10-06T05:32:23.179020+0000","last_unstale":"2023-10-06T05:32:24.158804+0000","last_undegraded":"2023-10-06T05:32:24.158804+0000","last_fullsized":"2023-10-06T05:32:24.158804+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:50.297518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.14","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190969+0000","last_change":"2023-10-06T05:32:24.186062+0000","last_active":"2023-10-06T05:32:24.190969+0000","last_peered":"2023-10-06T05:32:24.190969+0000","last_clean":"2023-10-06T05:32:24.190969+0000","last_became_active":"2023-10-06T05:32:24.185747+0000","last_became_peered":"2023-10-06T05:32:24.185747+0000","last_unstale":"2023-10-06T05:32:24.190969+0000","last_undegraded":"2023-10-06T05:32:24.190969+0000","last_fullsized":"2023-10-06T05:32:24.190969+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:31.226159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.164731+0000","last_change":"2023-10-06T05:32:23.184729+0000","last_active":"2023-10-06T05:32:24.164731+0000","last_peered":"2023-10-06T05:32:24.164731+0000","last_clean":"2023-10-06T05:32:24.164731+0000","last_became_active":"2023-10-06T05:32:23.184445+0000","last_became_peered":"2023-10-06T05:32:23.184445+0000","last_unstale":"2023-10-06T05:32:24.164731+0000","last_undegraded":"2023-10-06T05:32:24.164731+0000","last_fullsized":"2023-10-06T05:32:24.164731+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:43.366148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.15","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180837+0000","last_change":"2023-10-06T05:32:24.175439+0000","last_active":"2023-10-06T05:32:24.180837+0000","last_peered":"2023-10-06T05:32:24.180837+0000","last_clean":"2023-10-06T05:32:24.180837+0000","last_became_active":"2023-10-06T05:32:24.175124+0000","last_became_peered":"2023-10-06T05:32:24.175124+0000","last_unstale":"2023-10-06T05:32:24.180837+0000","last_undegraded":"2023-10-06T05:32:24.180837+0000","last_fullsized":"2023-10-06T05:32:24.180837+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:48.631818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172865+0000","last_change":"2023-10-06T05:32:23.180183+0000","last_active":"2023-10-06T05:32:24.172865+0000","last_peered":"2023-10-06T05:32:24.172865+0000","last_clean":"2023-10-06T05:32:24.172865+0000","last_became_active":"2023-10-06T05:32:23.179918+0000","last_became_peered":"2023-10-06T05:32:23.179918+0000","last_unstale":"2023-10-06T05:32:24.172865+0000","last_undegraded":"2023-10-06T05:32:24.172865+0000","last_fullsized":"2023-10-06T05:32:24.172865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:59.784967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.16","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186851+0000","last_change":"2023-10-06T05:32:24.176923+0000","last_active":"2023-10-06T05:32:24.186851+0000","last_peered":"2023-10-06T05:32:24.186851+0000","last_clean":"2023-10-06T05:32:24.186851+0000","last_became_active":"2023-10-06T05:32:24.176652+0000","last_became_peered":"2023-10-06T05:32:24.176652+0000","last_unstale":"2023-10-06T05:32:24.186851+0000","last_undegraded":"2023-10-06T05:32:24.186851+0000","last_fullsized":"2023-10-06T05:32:24.186851+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:32.195976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.15","version":"251'3","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.200865+0000","last_change":"2023-10-06T05:32:23.179308+0000","last_active":"2023-10-06T05:32:25.200865+0000","last_peered":"2023-10-06T05:32:25.200865+0000","last_clean":"2023-10-06T05:32:25.200865+0000","last_became_active":"2023-10-06T05:32:23.179038+0000","last_became_peered":"2023-10-06T05:32:23.179038+0000","last_unstale":"2023-10-06T05:32:25.200865+0000","last_undegraded":"2023-10-06T05:32:25.200865+0000","last_fullsized":"2023-10-06T05:32:25.200865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:02.273731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.17","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190880+0000","last_change":"2023-10-06T05:32:24.185229+0000","last_active":"2023-10-06T05:32:24.190880+0000","last_peered":"2023-10-06T05:32:24.190880+0000","last_clean":"2023-10-06T05:32:24.190880+0000","last_became_active":"2023-10-06T05:32:24.184951+0000","last_became_peered":"2023-10-06T05:32:24.184951+0000","last_unstale":"2023-10-06T05:32:24.190880+0000","last_undegraded":"2023-10-06T05:32:24.190880+0000","last_fullsized":"2023-10-06T05:32:24.190880+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:59.568662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.14","version":"251'6","reported_seq":16,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.197454+0000","last_change":"2023-10-06T05:32:23.180146+0000","last_active":"2023-10-06T05:32:25.197454+0000","last_peered":"2023-10-06T05:32:25.197454+0000","last_clean":"2023-10-06T05:32:25.197454+0000","last_became_active":"2023-10-06T05:32:23.179886+0000","last_became_peered":"2023-10-06T05:32:23.179886+0000","last_unstale":"2023-10-06T05:32:25.197454+0000","last_undegraded":"2023-10-06T05:32:25.197454+0000","last_fullsized":"2023-10-06T05:32:25.197454+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:00.888111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.10","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172687+0000","last_change":"2023-10-06T05:32:24.168677+0000","last_active":"2023-10-06T05:32:24.172687+0000","last_peered":"2023-10-06T05:32:24.172687+0000","last_clean":"2023-10-06T05:32:24.172687+0000","last_became_active":"2023-10-06T05:32:24.168348+0000","last_became_peered":"2023-10-06T05:32:24.168348+0000","last_unstale":"2023-10-06T05:32:24.172687+0000","last_undegraded":"2023-10-06T05:32:24.172687+0000","last_fullsized":"2023-10-06T05:32:24.172687+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:15.243862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.13","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195635+0000","last_change":"2023-10-06T05:32:23.180363+0000","last_active":"2023-10-06T05:32:25.195635+0000","last_peered":"2023-10-06T05:32:25.195635+0000","last_clean":"2023-10-06T05:32:25.195635+0000","last_became_active":"2023-10-06T05:32:23.180063+0000","last_became_peered":"2023-10-06T05:32:23.180063+0000","last_unstale":"2023-10-06T05:32:25.195635+0000","last_undegraded":"2023-10-06T05:32:25.195635+0000","last_fullsized":"2023-10-06T05:32:25.195635+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:29.866801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.11","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.181947+0000","last_change":"2023-10-06T05:32:24.181416+0000","last_active":"2023-10-06T05:32:24.181947+0000","last_peered":"2023-10-06T05:32:24.181947+0000","last_clean":"2023-10-06T05:32:24.181947+0000","last_became_active":"2023-10-06T05:32:24.181150+0000","last_became_peered":"2023-10-06T05:32:24.181150+0000","last_unstale":"2023-10-06T05:32:24.181947+0000","last_undegraded":"2023-10-06T05:32:24.181947+0000","last_fullsized":"2023-10-06T05:32:24.181947+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:22.091050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.164816+0000","last_change":"2023-10-06T05:32:23.177156+0000","last_active":"2023-10-06T05:32:24.164816+0000","last_peered":"2023-10-06T05:32:24.164816+0000","last_clean":"2023-10-06T05:32:24.164816+0000","last_became_active":"2023-10-06T05:32:23.176876+0000","last_became_peered":"2023-10-06T05:32:23.176876+0000","last_unstale":"2023-10-06T05:32:24.164816+0000","last_undegraded":"2023-10-06T05:32:24.164816+0000","last_fullsized":"2023-10-06T05:32:24.164816+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:42.828806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.12","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188663+0000","last_change":"2023-10-06T05:32:24.179626+0000","last_active":"2023-10-06T05:32:24.188663+0000","last_peered":"2023-10-06T05:32:24.188663+0000","last_clean":"2023-10-06T05:32:24.188663+0000","last_became_active":"2023-10-06T05:32:24.179326+0000","last_became_peered":"2023-10-06T05:32:24.179326+0000","last_unstale":"2023-10-06T05:32:24.188663+0000","last_undegraded":"2023-10-06T05:32:24.188663+0000","last_fullsized":"2023-10-06T05:32:24.188663+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:51.883357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166185+0000","last_change":"2023-10-06T05:32:23.182163+0000","last_active":"2023-10-06T05:32:24.166185+0000","last_peered":"2023-10-06T05:32:24.166185+0000","last_clean":"2023-10-06T05:32:24.166185+0000","last_became_active":"2023-10-06T05:32:23.181876+0000","last_became_peered":"2023-10-06T05:32:23.181876+0000","last_unstale":"2023-10-06T05:32:24.166185+0000","last_undegraded":"2023-10-06T05:32:24.166185+0000","last_fullsized":"2023-10-06T05:32:24.166185+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:05.667623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.13","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180019+0000","last_change":"2023-10-06T05:32:24.173718+0000","last_active":"2023-10-06T05:32:24.180019+0000","last_peered":"2023-10-06T05:32:24.180019+0000","last_clean":"2023-10-06T05:32:24.180019+0000","last_became_active":"2023-10-06T05:32:24.173451+0000","last_became_peered":"2023-10-06T05:32:24.173451+0000","last_unstale":"2023-10-06T05:32:24.180019+0000","last_undegraded":"2023-10-06T05:32:24.180019+0000","last_fullsized":"2023-10-06T05:32:24.180019+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:43.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.10","version":"251'8","reported_seq":18,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.202231+0000","last_change":"2023-10-06T05:32:23.178713+0000","last_active":"2023-10-06T05:32:25.202231+0000","last_peered":"2023-10-06T05:32:25.202231+0000","last_clean":"2023-10-06T05:32:25.202231+0000","last_became_active":"2023-10-06T05:32:23.178324+0000","last_became_peered":"2023-10-06T05:32:23.178324+0000","last_unstale":"2023-10-06T05:32:25.202231+0000","last_undegraded":"2023-10-06T05:32:25.202231+0000","last_fullsized":"2023-10-06T05:32:25.202231+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:35.285079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.c","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.174678+0000","last_change":"2023-10-06T05:32:24.172408+0000","last_active":"2023-10-06T05:32:24.174678+0000","last_peered":"2023-10-06T05:32:24.174678+0000","last_clean":"2023-10-06T05:32:24.174678+0000","last_became_active":"2023-10-06T05:32:24.172160+0000","last_became_peered":"2023-10-06T05:32:24.172160+0000","last_unstale":"2023-10-06T05:32:24.174678+0000","last_undegraded":"2023-10-06T05:32:24.174678+0000","last_fullsized":"2023-10-06T05:32:24.174678+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:41.998814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.f","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.195429+0000","last_change":"2023-10-06T05:32:23.177078+0000","last_active":"2023-10-06T05:32:25.195429+0000","last_peered":"2023-10-06T05:32:25.195429+0000","last_clean":"2023-10-06T05:32:25.195429+0000","last_became_active":"2023-10-06T05:32:23.176782+0000","last_became_peered":"2023-10-06T05:32:23.176782+0000","last_unstale":"2023-10-06T05:32:25.195429+0000","last_undegraded":"2023-10-06T05:32:25.195429+0000","last_fullsized":"2023-10-06T05:32:25.195429+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:33.934362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.d","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176156+0000","last_change":"2023-10-06T05:32:24.174998+0000","last_active":"2023-10-06T05:32:24.176156+0000","last_peered":"2023-10-06T05:32:24.176156+0000","last_clean":"2023-10-06T05:32:24.176156+0000","last_became_active":"2023-10-06T05:32:24.174637+0000","last_became_peered":"2023-10-06T05:32:24.174637+0000","last_unstale":"2023-10-06T05:32:24.176156+0000","last_undegraded":"2023-10-06T05:32:24.176156+0000","last_fullsized":"2023-10-06T05:32:24.176156+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:57.964936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.e","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.191975+0000","last_change":"2023-10-06T05:32:23.171097+0000","last_active":"2023-10-06T05:32:25.191975+0000","last_peered":"2023-10-06T05:32:25.191975+0000","last_clean":"2023-10-06T05:32:25.191975+0000","last_became_active":"2023-10-06T05:32:23.170822+0000","last_became_peered":"2023-10-06T05:32:23.170822+0000","last_unstale":"2023-10-06T05:32:25.191975+0000","last_undegraded":"2023-10-06T05:32:25.191975+0000","last_fullsized":"2023-10-06T05:32:25.191975+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:49.626561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.e","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176787+0000","last_change":"2023-10-06T05:32:24.171850+0000","last_active":"2023-10-06T05:32:24.176787+0000","last_peered":"2023-10-06T05:32:24.176787+0000","last_clean":"2023-10-06T05:32:24.176787+0000","last_became_active":"2023-10-06T05:32:24.171570+0000","last_became_peered":"2023-10-06T05:32:24.171570+0000","last_unstale":"2023-10-06T05:32:24.176787+0000","last_undegraded":"2023-10-06T05:32:24.176787+0000","last_fullsized":"2023-10-06T05:32:24.176787+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:16.335570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.d","version":"251'1","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.192351+0000","last_change":"2023-10-06T05:32:23.172247+0000","last_active":"2023-10-06T05:32:25.192351+0000","last_peered":"2023-10-06T05:32:25.192351+0000","last_clean":"2023-10-06T05:32:25.192351+0000","last_became_active":"2023-10-06T05:32:23.171388+0000","last_became_peered":"2023-10-06T05:32:23.171388+0000","last_unstale":"2023-10-06T05:32:25.192351+0000","last_undegraded":"2023-10-06T05:32:25.192351+0000","last_fullsized":"2023-10-06T05:32:25.192351+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:47.624287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.f","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.186668+0000","last_change":"2023-10-06T05:32:24.182168+0000","last_active":"2023-10-06T05:32:24.186668+0000","last_peered":"2023-10-06T05:32:24.186668+0000","last_clean":"2023-10-06T05:32:24.186668+0000","last_became_active":"2023-10-06T05:32:24.181903+0000","last_became_peered":"2023-10-06T05:32:24.181903+0000","last_unstale":"2023-10-06T05:32:24.186668+0000","last_undegraded":"2023-10-06T05:32:24.186668+0000","last_fullsized":"2023-10-06T05:32:24.186668+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:28.852368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172571+0000","last_change":"2023-10-06T05:32:23.178647+0000","last_active":"2023-10-06T05:32:24.172571+0000","last_peered":"2023-10-06T05:32:24.172571+0000","last_clean":"2023-10-06T05:32:24.172571+0000","last_became_active":"2023-10-06T05:32:23.178372+0000","last_became_peered":"2023-10-06T05:32:23.178372+0000","last_unstale":"2023-10-06T05:32:24.172571+0000","last_undegraded":"2023-10-06T05:32:24.172571+0000","last_fullsized":"2023-10-06T05:32:24.172571+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:30.194013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.8","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.190952+0000","last_change":"2023-10-06T05:32:24.186028+0000","last_active":"2023-10-06T05:32:24.190952+0000","last_peered":"2023-10-06T05:32:24.190952+0000","last_clean":"2023-10-06T05:32:24.190952+0000","last_became_active":"2023-10-06T05:32:24.185726+0000","last_became_peered":"2023-10-06T05:32:24.185726+0000","last_unstale":"2023-10-06T05:32:24.190952+0000","last_undegraded":"2023-10-06T05:32:24.190952+0000","last_fullsized":"2023-10-06T05:32:24.190952+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:12.995850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.b","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.197887+0000","last_change":"2023-10-06T05:32:23.177663+0000","last_active":"2023-10-06T05:32:25.197887+0000","last_peered":"2023-10-06T05:32:25.197887+0000","last_clean":"2023-10-06T05:32:25.197887+0000","last_became_active":"2023-10-06T05:32:23.176488+0000","last_became_peered":"2023-10-06T05:32:23.176488+0000","last_unstale":"2023-10-06T05:32:25.197887+0000","last_undegraded":"2023-10-06T05:32:25.197887+0000","last_fullsized":"2023-10-06T05:32:25.197887+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:19.533446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.9","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171060+0000","last_change":"2023-10-06T05:32:24.167891+0000","last_active":"2023-10-06T05:32:24.171060+0000","last_peered":"2023-10-06T05:32:24.171060+0000","last_clean":"2023-10-06T05:32:24.171060+0000","last_became_active":"2023-10-06T05:32:24.167602+0000","last_became_peered":"2023-10-06T05:32:24.167602+0000","last_unstale":"2023-10-06T05:32:24.171060+0000","last_undegraded":"2023-10-06T05:32:24.171060+0000","last_fullsized":"2023-10-06T05:32:24.171060+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:50.317183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.178969+0000","last_change":"2023-10-06T05:32:23.176326+0000","last_active":"2023-10-06T05:32:24.178969+0000","last_peered":"2023-10-06T05:32:24.178969+0000","last_clean":"2023-10-06T05:32:24.178969+0000","last_became_active":"2023-10-06T05:32:23.176027+0000","last_became_peered":"2023-10-06T05:32:23.176027+0000","last_unstale":"2023-10-06T05:32:24.178969+0000","last_undegraded":"2023-10-06T05:32:24.178969+0000","last_fullsized":"2023-10-06T05:32:24.178969+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:08.902322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.a","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.176122+0000","last_change":"2023-10-06T05:32:24.174385+0000","last_active":"2023-10-06T05:32:24.176122+0000","last_peered":"2023-10-06T05:32:24.176122+0000","last_clean":"2023-10-06T05:32:24.176122+0000","last_became_active":"2023-10-06T05:32:24.174122+0000","last_became_peered":"2023-10-06T05:32:24.174122+0000","last_unstale":"2023-10-06T05:32:24.176122+0000","last_undegraded":"2023-10-06T05:32:24.176122+0000","last_fullsized":"2023-10-06T05:32:24.176122+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:58.601296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.169086+0000","last_change":"2023-10-06T05:32:23.167801+0000","last_active":"2023-10-06T05:32:24.169086+0000","last_peered":"2023-10-06T05:32:24.169086+0000","last_clean":"2023-10-06T05:32:24.169086+0000","last_became_active":"2023-10-06T05:32:23.167534+0000","last_became_peered":"2023-10-06T05:32:23.167534+0000","last_unstale":"2023-10-06T05:32:24.169086+0000","last_undegraded":"2023-10-06T05:32:24.169086+0000","last_fullsized":"2023-10-06T05:32:24.169086+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:28.556557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.b","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.175928+0000","last_change":"2023-10-06T05:32:24.171385+0000","last_active":"2023-10-06T05:32:24.175928+0000","last_peered":"2023-10-06T05:32:24.175928+0000","last_clean":"2023-10-06T05:32:24.175928+0000","last_became_active":"2023-10-06T05:32:24.170957+0000","last_became_peered":"2023-10-06T05:32:24.170957+0000","last_unstale":"2023-10-06T05:32:24.175928+0000","last_undegraded":"2023-10-06T05:32:24.175928+0000","last_fullsized":"2023-10-06T05:32:24.175928+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:55.746676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165518+0000","last_change":"2023-10-06T05:32:23.182597+0000","last_active":"2023-10-06T05:32:24.165518+0000","last_peered":"2023-10-06T05:32:24.165518+0000","last_clean":"2023-10-06T05:32:24.165518+0000","last_became_active":"2023-10-06T05:32:23.182079+0000","last_became_peered":"2023-10-06T05:32:23.182079+0000","last_unstale":"2023-10-06T05:32:24.165518+0000","last_undegraded":"2023-10-06T05:32:24.165518+0000","last_fullsized":"2023-10-06T05:32:24.165518+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:26:40.353782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.4","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191127+0000","last_change":"2023-10-06T05:32:24.188487+0000","last_active":"2023-10-06T05:32:24.191127+0000","last_peered":"2023-10-06T05:32:24.191127+0000","last_clean":"2023-10-06T05:32:24.191127+0000","last_became_active":"2023-10-06T05:32:24.187720+0000","last_became_peered":"2023-10-06T05:32:24.187720+0000","last_unstale":"2023-10-06T05:32:24.191127+0000","last_undegraded":"2023-10-06T05:32:24.191127+0000","last_fullsized":"2023-10-06T05:32:24.191127+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:38.004084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.7","version":"251'2","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-10-06T05:32:25.191514+0000","last_change":"2023-10-06T05:32:23.180590+0000","last_active":"2023-10-06T05:32:25.191514+0000","last_peered":"2023-10-06T05:32:25.191514+0000","last_clean":"2023-10-06T05:32:25.191514+0000","last_became_active":"2023-10-06T05:32:23.180319+0000","last_became_peered":"2023-10-06T05:32:23.180319+0000","last_unstale":"2023-10-06T05:32:25.191514+0000","last_undegraded":"2023-10-06T05:32:25.191514+0000","last_fullsized":"2023-10-06T05:32:25.191514+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:33.942682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.5","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.185949+0000","last_change":"2023-10-06T05:32:24.181178+0000","last_active":"2023-10-06T05:32:24.185949+0000","last_peered":"2023-10-06T05:32:24.185949+0000","last_clean":"2023-10-06T05:32:24.185949+0000","last_became_active":"2023-10-06T05:32:24.180933+0000","last_became_peered":"2023-10-06T05:32:24.180933+0000","last_unstale":"2023-10-06T05:32:24.185949+0000","last_undegraded":"2023-10-06T05:32:24.185949+0000","last_fullsized":"2023-10-06T05:32:24.185949+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:45.636162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.6","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.159254+0000","last_change":"2023-10-06T05:32:23.178339+0000","last_active":"2023-10-06T05:32:24.159254+0000","last_peered":"2023-10-06T05:32:24.159254+0000","last_clean":"2023-10-06T05:32:24.159254+0000","last_became_active":"2023-10-06T05:32:23.178062+0000","last_became_peered":"2023-10-06T05:32:23.178062+0000","last_unstale":"2023-10-06T05:32:24.159254+0000","last_undegraded":"2023-10-06T05:32:24.159254+0000","last_fullsized":"2023-10-06T05:32:24.159254+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.343386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.6","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180511+0000","last_change":"2023-10-06T05:32:24.174589+0000","last_active":"2023-10-06T05:32:24.180511+0000","last_peered":"2023-10-06T05:32:24.180511+0000","last_clean":"2023-10-06T05:32:24.180511+0000","last_became_active":"2023-10-06T05:32:24.173846+0000","last_became_peered":"2023-10-06T05:32:24.173846+0000","last_unstale":"2023-10-06T05:32:24.180511+0000","last_undegraded":"2023-10-06T05:32:24.180511+0000","last_fullsized":"2023-10-06T05:32:24.180511+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:25.827006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.168717+0000","last_change":"2023-10-06T05:32:23.182554+0000","last_active":"2023-10-06T05:32:24.168717+0000","last_peered":"2023-10-06T05:32:24.168717+0000","last_clean":"2023-10-06T05:32:24.168717+0000","last_became_active":"2023-10-06T05:32:23.182296+0000","last_became_peered":"2023-10-06T05:32:23.182296+0000","last_unstale":"2023-10-06T05:32:24.168717+0000","last_undegraded":"2023-10-06T05:32:24.168717+0000","last_fullsized":"2023-10-06T05:32:24.168717+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:38.485415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.7","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171034+0000","last_change":"2023-10-06T05:32:24.169289+0000","last_active":"2023-10-06T05:32:24.171034+0000","last_peered":"2023-10-06T05:32:24.171034+0000","last_clean":"2023-10-06T05:32:24.171034+0000","last_became_active":"2023-10-06T05:32:24.169051+0000","last_became_peered":"2023-10-06T05:32:24.169051+0000","last_unstale":"2023-10-06T05:32:24.171034+0000","last_undegraded":"2023-10-06T05:32:24.171034+0000","last_fullsized":"2023-10-06T05:32:24.171034+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:30.250852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.163782+0000","last_change":"2023-10-06T05:32:23.168886+0000","last_active":"2023-10-06T05:32:24.163782+0000","last_peered":"2023-10-06T05:32:24.163782+0000","last_clean":"2023-10-06T05:32:24.163782+0000","last_became_active":"2023-10-06T05:32:23.168587+0000","last_became_peered":"2023-10-06T05:32:23.168587+0000","last_unstale":"2023-10-06T05:32:24.163782+0000","last_undegraded":"2023-10-06T05:32:24.163782+0000","last_fullsized":"2023-10-06T05:32:24.163782+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:54.739146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.0","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184097+0000","last_change":"2023-10-06T05:32:24.183162+0000","last_active":"2023-10-06T05:32:24.184097+0000","last_peered":"2023-10-06T05:32:24.184097+0000","last_clean":"2023-10-06T05:32:24.184097+0000","last_became_active":"2023-10-06T05:32:24.182826+0000","last_became_peered":"2023-10-06T05:32:24.182826+0000","last_unstale":"2023-10-06T05:32:24.184097+0000","last_undegraded":"2023-10-06T05:32:24.184097+0000","last_fullsized":"2023-10-06T05:32:24.184097+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:42.348407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.160865+0000","last_change":"2023-10-06T05:32:23.172000+0000","last_active":"2023-10-06T05:32:24.160865+0000","last_peered":"2023-10-06T05:32:24.160865+0000","last_clean":"2023-10-06T05:32:24.160865+0000","last_became_active":"2023-10-06T05:32:23.169976+0000","last_became_peered":"2023-10-06T05:32:23.169976+0000","last_unstale":"2023-10-06T05:32:24.160865+0000","last_undegraded":"2023-10-06T05:32:24.160865+0000","last_fullsized":"2023-10-06T05:32:24.160865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:02.134772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.1","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.189500+0000","last_change":"2023-10-06T05:32:24.183880+0000","last_active":"2023-10-06T05:32:24.189500+0000","last_peered":"2023-10-06T05:32:24.189500+0000","last_clean":"2023-10-06T05:32:24.189500+0000","last_became_active":"2023-10-06T05:32:24.183583+0000","last_became_peered":"2023-10-06T05:32:24.183583+0000","last_unstale":"2023-10-06T05:32:24.189500+0000","last_undegraded":"2023-10-06T05:32:24.189500+0000","last_fullsized":"2023-10-06T05:32:24.189500+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:27.031700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171334+0000","last_change":"2023-10-06T05:32:23.173250+0000","last_active":"2023-10-06T05:32:24.171334+0000","last_peered":"2023-10-06T05:32:24.171334+0000","last_clean":"2023-10-06T05:32:24.171334+0000","last_became_active":"2023-10-06T05:32:23.172701+0000","last_became_peered":"2023-10-06T05:32:23.172701+0000","last_unstale":"2023-10-06T05:32:24.171334+0000","last_undegraded":"2023-10-06T05:32:24.171334+0000","last_fullsized":"2023-10-06T05:32:24.171334+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:23.816268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184252+0000","last_change":"2023-10-06T05:32:24.180112+0000","last_active":"2023-10-06T05:32:24.184252+0000","last_peered":"2023-10-06T05:32:24.184252+0000","last_clean":"2023-10-06T05:32:24.184252+0000","last_became_active":"2023-10-06T05:32:24.179874+0000","last_became_peered":"2023-10-06T05:32:24.179874+0000","last_unstale":"2023-10-06T05:32:24.184252+0000","last_undegraded":"2023-10-06T05:32:24.184252+0000","last_fullsized":"2023-10-06T05:32:24.184252+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:09.162284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166257+0000","last_change":"2023-10-06T05:32:23.182061+0000","last_active":"2023-10-06T05:32:24.166257+0000","last_peered":"2023-10-06T05:32:24.166257+0000","last_clean":"2023-10-06T05:32:24.166257+0000","last_became_active":"2023-10-06T05:32:23.181754+0000","last_became_peered":"2023-10-06T05:32:23.181754+0000","last_unstale":"2023-10-06T05:32:24.166257+0000","last_undegraded":"2023-10-06T05:32:24.166257+0000","last_fullsized":"2023-10-06T05:32:24.166257+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:56.425225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.3","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172030+0000","last_change":"2023-10-06T05:32:24.169794+0000","last_active":"2023-10-06T05:32:24.172030+0000","last_peered":"2023-10-06T05:32:24.172030+0000","last_clean":"2023-10-06T05:32:24.172030+0000","last_became_active":"2023-10-06T05:32:24.168940+0000","last_became_peered":"2023-10-06T05:32:24.168940+0000","last_unstale":"2023-10-06T05:32:24.172030+0000","last_undegraded":"2023-10-06T05:32:24.172030+0000","last_fullsized":"2023-10-06T05:32:24.172030+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:21.785939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":10,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165603+0000","last_change":"2023-10-06T05:32:23.171370+0000","last_active":"2023-10-06T05:32:24.165603+0000","last_peered":"2023-10-06T05:32:24.165603+0000","last_clean":"2023-10-06T05:32:24.165603+0000","last_became_active":"2023-10-06T05:32:23.171112+0000","last_became_peered":"2023-10-06T05:32:23.171112+0000","last_unstale":"2023-10-06T05:32:24.165603+0000","last_undegraded":"2023-10-06T05:32:24.165603+0000","last_fullsized":"2023-10-06T05:32:24.165603+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:22.142603+0000","last_clean_scrub_stamp":"2023-10-06T05:32:22.142603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:33.934362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.20","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.188631+0000","last_change":"2023-10-06T05:32:24.181985+0000","last_active":"2023-10-06T05:32:24.188631+0000","last_peered":"2023-10-06T05:32:24.188631+0000","last_clean":"2023-10-06T05:32:24.188631+0000","last_became_active":"2023-10-06T05:32:24.181507+0000","last_became_peered":"2023-10-06T05:32:24.181507+0000","last_unstale":"2023-10-06T05:32:24.188631+0000","last_undegraded":"2023-10-06T05:32:24.188631+0000","last_fullsized":"2023-10-06T05:32:24.188631+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:17.985878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":204,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.168338+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:32:24.168338+0000","last_peered":"2023-10-06T05:32:24.168338+0000","last_clean":"2023-10-06T05:32:24.168338+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:32:24.168338+0000","last_undegraded":"2023-10-06T05:32:24.168338+0000","last_fullsized":"2023-10-06T05:32:24.168338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.26","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.181947+0000","last_change":"2023-10-06T05:32:24.175849+0000","last_active":"2023-10-06T05:32:24.181947+0000","last_peered":"2023-10-06T05:32:24.181947+0000","last_clean":"2023-10-06T05:32:24.181947+0000","last_became_active":"2023-10-06T05:32:24.175274+0000","last_became_peered":"2023-10-06T05:32:24.175274+0000","last_unstale":"2023-10-06T05:32:24.181947+0000","last_undegraded":"2023-10-06T05:32:24.181947+0000","last_fullsized":"2023-10-06T05:32:24.181947+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:52.864938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":202,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.167330+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:32:24.167330+0000","last_peered":"2023-10-06T05:32:24.167330+0000","last_clean":"2023-10-06T05:32:24.167330+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:32:24.167330+0000","last_undegraded":"2023-10-06T05:32:24.167330+0000","last_fullsized":"2023-10-06T05:32:24.167330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.25","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.191027+0000","last_change":"2023-10-06T05:32:24.186903+0000","last_active":"2023-10-06T05:32:24.191027+0000","last_peered":"2023-10-06T05:32:24.191027+0000","last_clean":"2023-10-06T05:32:24.191027+0000","last_became_active":"2023-10-06T05:32:24.186626+0000","last_became_peered":"2023-10-06T05:32:24.186626+0000","last_unstale":"2023-10-06T05:32:24.191027+0000","last_undegraded":"2023-10-06T05:32:24.191027+0000","last_fullsized":"2023-10-06T05:32:24.191027+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:38.075060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":206,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.160803+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:32:24.160803+0000","last_peered":"2023-10-06T05:32:24.160803+0000","last_clean":"2023-10-06T05:32:24.160803+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:32:24.160803+0000","last_undegraded":"2023-10-06T05:32:24.160803+0000","last_fullsized":"2023-10-06T05:32:24.160803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.27","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.184298+0000","last_change":"2023-10-06T05:32:24.177118+0000","last_active":"2023-10-06T05:32:24.184298+0000","last_peered":"2023-10-06T05:32:24.184298+0000","last_clean":"2023-10-06T05:32:24.184298+0000","last_became_active":"2023-10-06T05:32:24.176867+0000","last_became_peered":"2023-10-06T05:32:24.176867+0000","last_unstale":"2023-10-06T05:32:24.184298+0000","last_undegraded":"2023-10-06T05:32:24.184298+0000","last_fullsized":"2023-10-06T05:32:24.184298+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:18.415285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":202,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.166340+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:32:24.166340+0000","last_peered":"2023-10-06T05:32:24.166340+0000","last_clean":"2023-10-06T05:32:24.166340+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:32:24.166340+0000","last_undegraded":"2023-10-06T05:32:24.166340+0000","last_fullsized":"2023-10-06T05:32:24.166340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":351,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171835+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:32:24.171835+0000","last_peered":"2023-10-06T05:32:24.171835+0000","last_clean":"2023-10-06T05:32:24.171835+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:32:24.171835+0000","last_undegraded":"2023-10-06T05:32:24.171835+0000","last_fullsized":"2023-10-06T05:32:24.171835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.24","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.169724+0000","last_change":"2023-10-06T05:32:24.163962+0000","last_active":"2023-10-06T05:32:24.169724+0000","last_peered":"2023-10-06T05:32:24.169724+0000","last_clean":"2023-10-06T05:32:24.169724+0000","last_became_active":"2023-10-06T05:32:24.163662+0000","last_became_peered":"2023-10-06T05:32:24.163662+0000","last_unstale":"2023-10-06T05:32:24.169724+0000","last_undegraded":"2023-10-06T05:32:24.169724+0000","last_fullsized":"2023-10-06T05:32:24.169724+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:12.139749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.171720+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:32:24.171720+0000","last_peered":"2023-10-06T05:32:24.171720+0000","last_clean":"2023-10-06T05:32:24.171720+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:32:24.171720+0000","last_undegraded":"2023-10-06T05:32:24.171720+0000","last_fullsized":"2023-10-06T05:32:24.171720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.21","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165311+0000","last_change":"2023-10-06T05:32:24.163938+0000","last_active":"2023-10-06T05:32:24.165311+0000","last_peered":"2023-10-06T05:32:24.165311+0000","last_clean":"2023-10-06T05:32:24.165311+0000","last_became_active":"2023-10-06T05:32:24.163546+0000","last_became_peered":"2023-10-06T05:32:24.163546+0000","last_unstale":"2023-10-06T05:32:24.165311+0000","last_undegraded":"2023-10-06T05:32:24.165311+0000","last_fullsized":"2023-10-06T05:32:24.165311+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:01.036713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172191+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:32:24.172191+0000","last_peered":"2023-10-06T05:32:24.172191+0000","last_clean":"2023-10-06T05:32:24.172191+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:32:24.172191+0000","last_undegraded":"2023-10-06T05:32:24.172191+0000","last_fullsized":"2023-10-06T05:32:24.172191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.22","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.180424+0000","last_change":"2023-10-06T05:32:24.172713+0000","last_active":"2023-10-06T05:32:24.180424+0000","last_peered":"2023-10-06T05:32:24.180424+0000","last_clean":"2023-10-06T05:32:24.180424+0000","last_became_active":"2023-10-06T05:32:24.172391+0000","last_became_peered":"2023-10-06T05:32:24.172391+0000","last_unstale":"2023-10-06T05:32:24.180424+0000","last_undegraded":"2023-10-06T05:32:24.180424+0000","last_fullsized":"2023-10-06T05:32:24.180424+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:08.677384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.165233+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:32:24.165233+0000","last_peered":"2023-10-06T05:32:24.165233+0000","last_clean":"2023-10-06T05:32:24.165233+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:32:24.165233+0000","last_undegraded":"2023-10-06T05:32:24.165233+0000","last_fullsized":"2023-10-06T05:32:24.165233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.23","version":"0'0","reported_seq":9,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.172891+0000","last_change":"2023-10-06T05:32:24.171903+0000","last_active":"2023-10-06T05:32:24.172891+0000","last_peered":"2023-10-06T05:32:24.172891+0000","last_clean":"2023-10-06T05:32:24.172891+0000","last_became_active":"2023-10-06T05:32:24.171304+0000","last_became_peered":"2023-10-06T05:32:24.171304+0000","last_unstale":"2023-10-06T05:32:24.172891+0000","last_undegraded":"2023-10-06T05:32:24.172891+0000","last_fullsized":"2023-10-06T05:32:24.172891+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:32:23.150133+0000","last_clean_scrub_stamp":"2023-10-06T05:32:23.150133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:19.842503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-06T05:32:24.159300+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:32:24.159300+0000","last_peered":"2023-10-06T05:32:24.159300+0000","last_clean":"2023-10-06T05:32:24.159300+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:32:24.159300+0000","last_undegraded":"2023-10-06T05:32:24.159300+0000","last_fullsized":"2023-10-06T05:32:24.159300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":38,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":37,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575072,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305076,"kb_used_data":3692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066764,"statfs":{"total":96636764160,"available":96324366336,"internally_reserved":0,"allocated":3780608,"data_stored":2082022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.97399999999999998,"15min":1.1850000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68500000000000005,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.169},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.59999999999999998,"5min":0.42999999999999999,"15min":0.38900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0860000000000001,"15min":1.268},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66500000000000004,"15min":0.68200000000000005},"min":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96899999999999997,"5min":1.163,"15min":1.163},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.22},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.55000000000000004,"5min":0.5,"15min":0.47599999999999998},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91800000000000004,"5min":1.0529999999999999,"15min":1.117},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.32200000000000001},"max":{"1min":0.96399999999999997,"5min":1.0209999999999999,"15min":1.258},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.75800000000000001,"15min":0.73899999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.39400000000000002},"max":{"1min":0.997,"5min":1.234,"15min":1.2849999999999999},"last":0.40000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.999,"15min":1.109},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.45600000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":1.1279999999999999,"15min":1.2290000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.169},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.41199999999999998,"15min":0.38600000000000001},"max":{"1min":0.99099999999999999,"5min":1.242,"15min":1.242},"last":0.45900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304500,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067340,"statfs":{"total":96636764160,"available":96324956160,"internally_reserved":0,"allocated":3190784,"data_stored":1492208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90800000000000003,"5min":1.044,"15min":1.0720000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.42399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":1.137,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.135},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.442},"max":{"1min":1.075,"5min":1.115,"15min":1.115},"last":0.747}]},{"osd":3,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.70899999999999996},"min":{"1min":0.55000000000000004,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":1.1160000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.752,"15min":0.746},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.5},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.504,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.0049999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.53900000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91600000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86299999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.70999999999999996},"min":{"1min":0.45000000000000001,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0920000000000001,"15min":1.4610000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":1.0920000000000001,"15min":1.1000000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.53900000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.995,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.93899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575071,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1491632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70999999999999996,"5min":0.91400000000000003,"15min":1.0149999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.83999999999999997,"5min":0.86099999999999999,"15min":0.88100000000000001},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61299999999999999,"5min":0.88800000000000001,"15min":1.165},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":0.76600000000000001,"15min":1.077},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.96099999999999997,"15min":1.2230000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.53400000000000003},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.2430000000000001},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77500000000000002,"15min":0.76100000000000001},"min":{"1min":0.60799999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96899999999999997,"5min":1.026,"15min":1.5389999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.45200000000000001},"max":{"1min":1.018,"5min":1.119,"15min":1.4690000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78500000000000003,"15min":0.78200000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.47899999999999998},"max":{"1min":1.0760000000000001,"5min":1.248,"15min":1.25},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79100000000000004,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.50600000000000001},"max":{"1min":1.014,"5min":1.26,"15min":1.26},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.81699999999999995},"min":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.48199999999999998},"max":{"1min":1.069,"5min":1.2290000000000001,"15min":1.2909999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84199999999999997,"15min":0.82699999999999996},"min":{"1min":0.60099999999999998,"5min":0.52800000000000002,"15min":0.503},"max":{"1min":1.111,"5min":1.2849999999999999,"15min":1.3280000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.46999999999999997},"max":{"1min":1.095,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.68700000000000006,"5min":0.53700000000000003,"15min":0.48699999999999999},"max":{"1min":1.018,"5min":1.1299999999999999,"15min":2.6240000000000001},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575071,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309172,"kb_used_data":3692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062668,"statfs":{"total":96636764160,"available":96320172032,"internally_reserved":0,"allocated":3780608,"data_stored":2082090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.245},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.85199999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.498},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82099999999999995},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.504,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.70799999999999996,"5min":0.81899999999999995,"15min":0.90800000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.80500000000000005,"15min":0.84999999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.58599999999999997,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.499},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.88400000000000001,"15min":0.90200000000000002},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.47299999999999998},"max":{"1min":0.997,"5min":1.05,"15min":2.8220000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.622,"5min":0.52000000000000002,"15min":0.47799999999999998},"max":{"1min":1.113,"5min":1.244,"15min":3.6629999999999998},"last":0.96499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.65700000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"max":{"1min":1.1060000000000001,"5min":1.27,"15min":1.27},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.59499999999999997,"5min":0.53000000000000003,"15min":0.49199999999999999},"max":{"1min":1.0389999999999999,"5min":1.097,"15min":1.1970000000000001},"last":0.97899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.58399999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0580000000000001,"5min":1.093,"15min":1.1839999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.45500000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.2629999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80000000000000004,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.25},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.66300000000000003,"5min":0.55500000000000005,"15min":0.50600000000000001},"max":{"1min":1.1459999999999999,"5min":1.167,"15min":1.2789999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575071,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304508,"kb_used_data":3124,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067332,"statfs":{"total":96636764160,"available":96324947968,"internally_reserved":0,"allocated":3198976,"data_stored":1493238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.45200000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.056},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70999999999999996,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.0649999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.45400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":1.073},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":0.70099999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.96499999999999997},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.53700000000000003},"max":{"1min":0.91800000000000004,"5min":1.486,"15min":1.486},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.76900000000000002},"min":{"1min":0.62,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":1.5,"15min":1.5},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74399999999999999,"15min":0.76000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1639999999999999},"last":1.016},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.79200000000000004},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1859999999999999},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.626,"5min":0.504,"15min":0.504},"max":{"1min":1.014,"5min":1.014,"15min":1.1830000000000001},"last":1.22},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.137},"last":1.161}]},{"osd":7,"last update":"Fri Oct 6 05:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78200000000000003,"15min":0.79700000000000004},"min":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.57499999999999996},"max":{"1min":1.0229999999999999,"5min":1.538,"15min":1.538},"last":1.282},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.80500000000000005},"min":{"1min":0.55600000000000005,"5min":0.54100000000000004,"15min":0.51600000000000001},"max":{"1min":0.96199999999999997,"5min":1.014,"15min":1.2},"last":1.264}]}]},{"osd":3,"up_from":13,"seq":55834575072,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308592,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063248,"statfs":{"total":96636764160,"available":96320765952,"internally_reserved":0,"allocated":3186688,"data_stored":1491666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.748,"5min":0.76700000000000002,"15min":0.89200000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.29699999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69799999999999995,"5min":1.016,"15min":1.369},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70199999999999996,"5min":0.97899999999999998,"15min":1.5369999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.90200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.90400000000000003},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.46000000000000002},"max":{"1min":0.93100000000000005,"5min":1.1100000000000001,"15min":1.2030000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.68600000000000005,"5min":0.5,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":1.1080000000000001,"15min":1.3009999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76200000000000001,"15min":0.78100000000000003},"min":{"1min":0.67300000000000004,"5min":0.53800000000000003,"15min":0.52500000000000002},"max":{"1min":0.94499999999999995,"5min":1.111,"15min":1.1539999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.79200000000000004},"min":{"1min":0.67600000000000005,"5min":0.51800000000000002,"15min":0.51100000000000001},"max":{"1min":1.0049999999999999,"5min":1.0820000000000001,"15min":1.4590000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78000000000000003,"15min":0.79500000000000004},"min":{"1min":0.62,"5min":0.55900000000000005,"15min":0.53900000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.3360000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.79800000000000004},"min":{"1min":0.59999999999999998,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":0.90900000000000003,"5min":1.1339999999999999,"15min":1.2390000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.72299999999999998,"5min":0.57299999999999995,"15min":0.57099999999999995},"max":{"1min":0.999,"5min":1.0669999999999999,"15min":1.194},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.81200000000000006},"min":{"1min":0.65800000000000003,"5min":0.53700000000000003,"15min":0.53000000000000003},"max":{"1min":0.95799999999999996,"5min":1.0980000000000001,"15min":1.2090000000000001},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304516,"kb_used_data":3132,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067324,"statfs":{"total":96636764160,"available":96324939776,"internally_reserved":0,"allocated":3207168,"data_stored":1493882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0249999999999999},"last":5.3070000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97299999999999998,"5min":1.0369999999999999,"15min":1.05},"last":5.6299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.68700000000000006},"min":{"1min":0.52700000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.99099999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":3.9609999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.436},"max":{"1min":0.85499999999999998,"5min":0.98899999999999999,"15min":1.016},"last":4.2169999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70099999999999996,"15min":0.71599999999999997},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.99099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.89400000000000002,"5min":1.115,"15min":1.115},"last":3.839},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.018},"last":4.4050000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":2.5049999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":1.163,"15min":1.163},"last":6.5259999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81299999999999994,"5min":1.0189999999999999,"15min":1.0920000000000001},"last":7.9459999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45300000000000001},"max":{"1min":0.86499999999999999,"5min":1.0489999999999999,"15min":1.0609999999999999},"last":0.621}]},{"osd":7,"last update":"Fri Oct 6 05:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0169999999999999,"5min":1.282,"15min":1.282},"last":1.109},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95799999999999996,"5min":1.254,"15min":1.254},"last":3.6619999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575072,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304496,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067344,"statfs":{"total":96636764160,"available":96324960256,"internally_reserved":0,"allocated":3186688,"data_stored":1491666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.69199999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":1.127,"15min":2.4609999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.39800000000000002},"max":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":1.3460000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.054,"5min":1.093,"15min":1.2130000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.441,"15min":0.40899999999999997},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.0289999999999999,"15min":1.204},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":0.95599999999999996,"15min":1.002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.433},"max":{"1min":1.075,"5min":1.145,"15min":1.2769999999999999},"last":0.878}]},{"osd":4,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.63700000000000001},"min":{"1min":0.52100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":0.96599999999999997,"15min":1.0229999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.56000000000000005,"5min":0.442,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.02,"15min":1.1539999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.872,"5min":0.91500000000000004,"15min":1.141},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.60799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":0.88800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.59499999999999997,"5min":0.51200000000000001,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.248},"last":0.90500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:32:30.038 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:32:30.038 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:32:30.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.264+0000 7f26aa79d700 1 Processor -- start 2023-10-06T05:32:30.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.265+0000 7f26aa79d700 1 -- start start 2023-10-06T05:32:30.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26aa79d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26a4156370 0x7f26a40aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26aa79d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a406b330 con 0x7f26a40c7f60 2023-10-06T05:32:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26aa79d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a406b610 con 0x7f26a4156740 2023-10-06T05:32:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26aa79d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a406beb0 con 0x7f26a4156370 2023-10-06T05:32:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26a9f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f26a40c7f60 0x7f26a40abdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37176/0 (socket says 172.21.15.33:37176) 2023-10-06T05:32:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26a979b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26a4156370 0x7f26a40aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.266+0000 7f26a9f9c700 1 -- 172.21.15.33:0/2726636530 learned_addr learned my addr 172.21.15.33:0/2726636530 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f269bfff700 1 -- 172.21.15.33:0/2726636530 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731411138 0 0) 0x7f26a406b330 con 0x7f26a40c7f60 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f269bfff700 1 -- 172.21.15.33:0/2726636530 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2690003570 con 0x7f26a40c7f60 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f26a979b700 1 -- 172.21.15.33:0/2726636530 >> v1:172.21.15.167:6789/0 conn(0x7f26a4156740 legacy=0x7f26a40cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f26a979b700 1 -- 172.21.15.33:0/2726636530 >> v1:172.21.15.33:6789/0 conn(0x7f26a40c7f60 legacy=0x7f26a40abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f26a979b700 1 -- 172.21.15.33:0/2726636530 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a406b8f0 con 0x7f26a4156370 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f26a979b700 1 --2- 172.21.15.33:0/2726636530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a4156370 0x7f26a40aef80 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098e0 tx=0x7f26a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a042b4ba795f3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f269bfff700 1 -- 172.21.15.33:0/2726636530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000b640 con 0x7f26a4156370 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f269bfff700 1 -- 172.21.15.33:0/2726636530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a000b7a0 con 0x7f26a4156370 2023-10-06T05:32:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.267+0000 7f269bfff700 1 -- 172.21.15.33:0/2726636530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000bb40 con 0x7f26a4156370 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 -- 172.21.15.33:0/2726636530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a4156370 msgr2=0x7f26a40aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 --2- 172.21.15.33:0/2726636530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a4156370 0x7f26a40aef80 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098e0 tx=0x7f26a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 -- 172.21.15.33:0/2726636530 shutdown_connections 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 --2- 172.21.15.33:0/2726636530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a4156370 0x7f26a40aef80 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 -- 172.21.15.33:0/2726636530 >> 172.21.15.33:0/2726636530 conn(0x7f26a406c560 msgr2=0x7f26a406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 -- 172.21.15.33:0/2726636530 shutdown_connections 2023-10-06T05:32:30.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.268+0000 7f26aa79d700 1 -- 172.21.15.33:0/2726636530 wait complete. 2023-10-06T05:32:30.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.269+0000 7f26aa79d700 1 Processor -- start 2023-10-06T05:32:30.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 -- start start 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 0x7f26a4152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26a4156370 0x7f26a4153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26a4156740 0x7f26a4141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41550c0 con 0x7f26a40c7f60 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a4155550 con 0x7f26a4156370 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.270+0000 7f26aa79d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41544b0 con 0x7f26a4156740 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 0x7f26a4152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 0x7f26a4152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40536/0 (socket says 172.21.15.33:40536) 2023-10-06T05:32:30.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 -- 172.21.15.33:0/1057810535 learned_addr learned my addr 172.21.15.33:0/1057810535 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a9f9c700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26a4156740 0x7f26a4141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a8f9a700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26a4156370 0x7f26a4153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 -- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26a4156740 msgr2=0x7f26a4141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26a4156740 0x7f26a4141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 -- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26a4156370 msgr2=0x7f26a4153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26a4156370 0x7f26a4153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 -- 172.21.15.33:0/1057810535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0009420 con 0x7f26a40c7f60 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.271+0000 7f26a979b700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 0x7f26a4152540 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f26a4069740 tx=0x7f26a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.272+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000bb20 con 0x7f26a40c7f60 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.272+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0011420 con 0x7f26a40c7f60 2023-10-06T05:32:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.272+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00117c0 con 0x7f26a40c7f60 2023-10-06T05:32:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.272+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a4154790 con 0x7f26a40c7f60 2023-10-06T05:32:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.272+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a4154d50 con 0x7f26a40c7f60 2023-10-06T05:32:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.273+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a404c380 con 0x7f26a40c7f60 2023-10-06T05:32:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.274+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26a0011b70 con 0x7f26a40c7f60 2023-10-06T05:32:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.275+0000 7f26997fa700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2688066f30 0x7f26880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.275+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26a0085de0 con 0x7f26a40c7f60 2023-10-06T05:32:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.275+0000 7f26a8f9a700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2688066f30 0x7f26880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.276+0000 7f26a8f9a700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2688066f30 0x7f26880693e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f26a406fce0 tx=0x7f2694009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:30.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.279+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26a0052e50 con 0x7f26a40c7f60 2023-10-06T05:32:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.568+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f26a406dec0 con 0x7f26a40c7f60 2023-10-06T05:32:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.573+0000 7f26997fa700 1 -- 172.21.15.33:0/1057810535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) v1 ==== 190+0+0 (secure 0 0 0) 0x7f26a0056470 con 0x7f26a40c7f60 2023-10-06T05:32:30.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2688066f30 msgr2=0x7f26880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2688066f30 0x7f26880693e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f26a406fce0 tx=0x7f2694009040 comp rx=0 tx=0).stop 2023-10-06T05:32:30.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 msgr2=0x7f26a4152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 0x7f26a4152540 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f26a4069740 tx=0x7f26a0017000 comp rx=0 tx=0).stop 2023-10-06T05:32:30.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 shutdown_connections 2023-10-06T05:32:30.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2688066f30 0x7f26880693e0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26a4156740 0x7f26a4141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26a40c7f60 0x7f26a4152540 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 --2- 172.21.15.33:0/1057810535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26a4156370 0x7f26a4153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 >> 172.21.15.33:0/1057810535 conn(0x7f26a406c560 msgr2=0x7f26a4147350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:30.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 shutdown_connections 2023-10-06T05:32:30.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.575+0000 7f26aa79d700 1 -- 172.21.15.33:0/1057810535 wait complete. 2023-10-06T05:32:30.579 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:32:30.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.779+0000 7f125139d700 1 Processor -- start 2023-10-06T05:32:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.779+0000 7f125139d700 1 -- start start 2023-10-06T05:32:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f125139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c6340 0x7f124c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f125139d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c06c820 con 0x7f124c0c80d0 2023-10-06T05:32:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f125139d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c06a160 con 0x7f124c159520 2023-10-06T05:32:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f125139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0c8800 con 0x7f124c0c6340 2023-10-06T05:32:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f1250b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f124c0c80d0 0x7f124c06a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37188/0 (socket says 172.21.15.33:37188) 2023-10-06T05:32:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f1250b9c700 1 -- 172.21.15.33:0/2637289524 learned_addr learned my addr 172.21.15.33:0/2637289524 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f124b7fe700 1 --2- 172.21.15.33:0/2637289524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c6340 0x7f124c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332602087 0 0) 0x7f124c06c820 con 0x7f124c0c80d0 2023-10-06T05:32:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.781+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1234003570 con 0x7f124c0c80d0 2023-10-06T05:32:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403103219 0 0) 0x7f124c06a160 con 0x7f124c159520 2023-10-06T05:32:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f124c06c820 con 0x7f124c159520 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124b7fe700 1 -- 172.21.15.33:0/2637289524 >> v1:172.21.15.167:6789/0 conn(0x7f124c159520 legacy=0x7f124c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124b7fe700 1 -- 172.21.15.33:0/2637289524 >> v1:172.21.15.33:6789/0 conn(0x7f124c0c80d0 legacy=0x7f124c06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124b7fe700 1 -- 172.21.15.33:0/2637289524 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0c8ae0 con 0x7f124c0c6340 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124b7fe700 1 --2- 172.21.15.33:0/2637289524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c6340 0x7f124c0c7b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1240009910 tx=0x7f12400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de2ff6a5c517b262 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240011630 con 0x7f124c0c6340 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240011790 con 0x7f124c0c6340 2023-10-06T05:32:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.782+0000 7f124affd700 1 -- 172.21.15.33:0/2637289524 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240011b30 con 0x7f124c0c6340 2023-10-06T05:32:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 -- 172.21.15.33:0/2637289524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c6340 msgr2=0x7f124c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 --2- 172.21.15.33:0/2637289524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c6340 0x7f124c0c7b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1240009910 tx=0x7f12400091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 -- 172.21.15.33:0/2637289524 shutdown_connections 2023-10-06T05:32:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 --2- 172.21.15.33:0/2637289524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c6340 0x7f124c0c7b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 -- 172.21.15.33:0/2637289524 >> 172.21.15.33:0/2637289524 conn(0x7f124c0cc930 msgr2=0x7f124c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 -- 172.21.15.33:0/2637289524 shutdown_connections 2023-10-06T05:32:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.783+0000 7f125139d700 1 -- 172.21.15.33:0/2637289524 wait complete. 2023-10-06T05:32:30.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.784+0000 7f125139d700 1 Processor -- start 2023-10-06T05:32:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 -- start start 2023-10-06T05:32:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f124c0c6340 0x7f124c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c80d0 0x7f124c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 0x7f124c0ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c15a5a0 con 0x7f124c159520 2023-10-06T05:32:30.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c159990 con 0x7f124c0c6340 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.785+0000 7f125139d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c159c70 con 0x7f124c0c80d0 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 0x7f124c0ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 0x7f124c0ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40540/0 (socket says 172.21.15.33:40540) 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 -- 172.21.15.33:0/2607836637 learned_addr learned my addr 172.21.15.33:0/2607836637 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f124b7fe700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c80d0 0x7f124c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f124bfff700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f124c0c6340 0x7f124c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 -- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c80d0 msgr2=0x7f124c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c80d0 0x7f124c15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 -- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f124c0c6340 msgr2=0x7f124c15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:30.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f124c0c6340 0x7f124c15d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 -- 172.21.15.33:0/2607836637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009420 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f124b7fe700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c80d0 0x7f124c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f124bfff700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f124c0c6340 0x7f124c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.786+0000 7f1250b9c700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 0x7f124c0ac940 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f124400cc60 tx=0x7f1244001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.787+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400ce70 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.787+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400ab70 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.787+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c159f50 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.787+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440070f0 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.787+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c1568a0 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.788+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c15a750 con 0x7f124c159520 2023-10-06T05:32:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.789+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12440074a0 con 0x7f124c159520 2023-10-06T05:32:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.789+0000 7f1233fff700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1238066f30 0x7f12380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.790+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1244084b60 con 0x7f124c159520 2023-10-06T05:32:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.790+0000 7f124bfff700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1238066f30 0x7f12380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.791+0000 7f124bfff700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1238066f30 0x7f12380693e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f123c003990 tx=0x7f123c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:30.793+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f124c15a750 con 0x7f124c159520 2023-10-06T05:32:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.037+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f124c15a750 con 0x7f124c159520 2023-10-06T05:32:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.037+0000 7f1233fff700 1 -- 172.21.15.33:0/2607836637 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f124c15a750 con 0x7f124c159520 2023-10-06T05:32:31.039 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:31.039 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1238066f30 msgr2=0x7f12380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1238066f30 0x7f12380693e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f123c003990 tx=0x7f123c009040 comp rx=0 tx=0).stop 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 msgr2=0x7f124c0ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 0x7f124c0ac940 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f124400cc60 tx=0x7f1244001ae0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 shutdown_connections 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1238066f30 0x7f12380693e0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124c159520 0x7f124c0ac940 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f124c0c6340 0x7f124c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 --2- 172.21.15.33:0/2607836637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f124c0c80d0 0x7f124c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 >> 172.21.15.33:0/2607836637 conn(0x7f124c0cc930 msgr2=0x7f124c156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 shutdown_connections 2023-10-06T05:32:31.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.040+0000 7f125139d700 1 -- 172.21.15.33:0/2607836637 wait complete. 2023-10-06T05:32:31.043 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:32:31.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 195, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:32:24.200367+0000', 'modified': '2023-10-06T05:32:28.195313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11496': {'gid': 11496, 'name': 'a', 'rank': 0, 'incarnation': 191, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/3696795298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 3696795298}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 3696795298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [38], 'metadata_pool': 37, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:32:31.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:32:31.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:32:31.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.652+0000 7f98fffff700 1 Processor -- start 2023-10-06T05:32:31.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.652+0000 7f98fffff700 1 -- start start 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98fffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9900140d20 0x7f99000c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98fffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990015bcb0 con 0x7f99000c9210 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98fffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99001410f0 con 0x7f990006a0d0 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98fffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99000c9690 con 0x7f9900140d20 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98fe7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f99000c9210 0x7f99000699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37196/0 (socket says 172.21.15.33:37196) 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98fe7fc700 1 -- 172.21.15.33:0/1426510349 learned_addr learned my addr 172.21.15.33:0/1426510349 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:31.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.653+0000 7f98feffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9900140d20 0x7f99000c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98fdffb700 1 -- 172.21.15.33:0/1426510349 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690546673 0 0) 0x7f990015bcb0 con 0x7f99000c9210 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98feffd700 1 -- 172.21.15.33:0/1426510349 >> v1:172.21.15.167:6789/0 conn(0x7f990006a0d0 legacy=0x7f99000c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98feffd700 1 -- 172.21.15.33:0/1426510349 >> v1:172.21.15.33:6789/0 conn(0x7f99000c9210 legacy=0x7f99000699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98feffd700 1 -- 172.21.15.33:0/1426510349 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990015b0a0 con 0x7f9900140d20 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98feffd700 1 --2- 172.21.15.33:0/1426510349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c8cd0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f98f00098e0 tx=0x7f98f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d945e237911ce0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98fdffb700 1 -- 172.21.15.33:0/1426510349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0009e80 con 0x7f9900140d20 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98fdffb700 1 -- 172.21.15.33:0/1426510349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f000b860 con 0x7f9900140d20 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.654+0000 7f98fdffb700 1 -- 172.21.15.33:0/1426510349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000bbe0 con 0x7f9900140d20 2023-10-06T05:32:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 -- 172.21.15.33:0/1426510349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 msgr2=0x7f99000c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 --2- 172.21.15.33:0/1426510349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c8cd0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f98f00098e0 tx=0x7f98f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 -- 172.21.15.33:0/1426510349 shutdown_connections 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 --2- 172.21.15.33:0/1426510349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c8cd0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 -- 172.21.15.33:0/1426510349 >> 172.21.15.33:0/1426510349 conn(0x7f990006c560 msgr2=0x7f990006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 -- 172.21.15.33:0/1426510349 shutdown_connections 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.655+0000 7f98fffff700 1 -- 172.21.15.33:0/1426510349 wait complete. 2023-10-06T05:32:31.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.656+0000 7f98fffff700 1 Processor -- start 2023-10-06T05:32:31.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.657+0000 7f98fffff700 1 -- start start 2023-10-06T05:32:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.657+0000 7f98fffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990006a0d0 0x7f990013ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98fffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99000c9210 0x7f99000c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98fffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000c5660 con 0x7f9900140d20 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98fffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900157b30 con 0x7f99000c9210 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98fffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9900157dd0 con 0x7f990006a0d0 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40552/0 (socket says 172.21.15.33:40552) 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 -- 172.21.15.33:0/771985262 learned_addr learned my addr 172.21.15.33:0/771985262 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98feffd700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990006a0d0 0x7f990013ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99000c9210 0x7f99000c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 -- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990006a0d0 msgr2=0x7f990013ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990006a0d0 0x7f990013ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 -- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99000c9210 msgr2=0x7f99000c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99000c9210 0x7f99000c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.658+0000 7f98ff7fe700 1 -- 172.21.15.33:0/771985262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0009420 con 0x7f9900140d20 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.659+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c5080 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f98f800c840 tx=0x7f98f8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.659+0000 7f98e6ffd700 1 -- 172.21.15.33:0/771985262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8012630 con 0x7f9900140d20 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.659+0000 7f98e6ffd700 1 -- 172.21.15.33:0/771985262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f80127b0 con 0x7f9900140d20 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.659+0000 7f98e6ffd700 1 -- 172.21.15.33:0/771985262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8012b70 con 0x7f9900140d20 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.659+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99001580b0 con 0x7f9900140d20 2023-10-06T05:32:31.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.659+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99000ae730 con 0x7f9900140d20 2023-10-06T05:32:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.661+0000 7f98e6ffd700 1 -- 172.21.15.33:0/771985262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98f801a3a0 con 0x7f9900140d20 2023-10-06T05:32:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.662+0000 7f98e6ffd700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98ec066f30 0x7f98ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.662+0000 7f98e6ffd700 1 -- 172.21.15.33:0/771985262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98f80847f0 con 0x7f9900140d20 2023-10-06T05:32:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.662+0000 7f98feffd700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98ec066f30 0x7f98ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.663+0000 7f98feffd700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98ec066f30 0x7f98ec0693e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f98f00093f0 tx=0x7f98f0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.664+0000 7f9913cddb80 1 Processor -- start 2023-10-06T05:32:31.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.664+0000 7f9913cddb80 1 -- start start 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f361dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3667bc0 0x55a7f3646920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f3619990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f9913cddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f359dc80 con 0x55a7f361d950 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f3574a70 con 0x55a7f3646e60 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7f3572270 con 0x55a7f3667bc0 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f361dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f361dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40554/0 (socket says 172.21.15.33:40554) 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 -- 172.21.15.33:0/844733424 learned_addr learned my addr 172.21.15.33:0/844733424 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3667bc0 0x55a7f3646920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f3619990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 -- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3667bc0 msgr2=0x55a7f3646920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3667bc0 0x55a7f3646920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 -- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 msgr2=0x55a7f3619990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f3619990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98feffd700 1 -- 172.21.15.33:0/844733424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f3408270 con 0x55a7f361d950 2023-10-06T05:32:31.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.665+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3667bc0 0x55a7f3646920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.666+0000 7f98feffd700 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f361dd20 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f98f000b3c0 tx=0x7f98f0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df2dde24fc06187 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.666+0000 7f98e4ff9700 1 -- 172.21.15.33:0/844733424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f002c070 con 0x55a7f361d950 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.666+0000 7f98e4ff9700 1 -- 172.21.15.33:0/844733424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0020040 con 0x55a7f361d950 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.666+0000 7f98e4ff9700 1 -- 172.21.15.33:0/844733424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f001ae70 con 0x55a7f361d950 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 -- 172.21.15.33:0/844733424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 msgr2=0x55a7f361dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f361dd20 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f98f000b3c0 tx=0x7f98f0022000 comp rx=0 tx=0).stop 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 -- 172.21.15.33:0/844733424 shutdown_connections 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f3619990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f361dd20 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 --2- 172.21.15.33:0/844733424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3667bc0 0x55a7f3646920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 -- 172.21.15.33:0/844733424 >> 172.21.15.33:0/844733424 conn(0x55a7f361f270 msgr2=0x55a7f361f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 -- 172.21.15.33:0/844733424 shutdown_connections 2023-10-06T05:32:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.667+0000 7f9913cddb80 1 -- 172.21.15.33:0/844733424 wait complete. 2023-10-06T05:32:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.668+0000 7f9913cddb80 1 -- start start 2023-10-06T05:32:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.668+0000 7f9913cddb80 1 Processor -- start 2023-10-06T05:32:31.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f361d950 0x55a7f3661990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f36141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 0x55a7f36146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f356e060 con 0x55a7f3667bc0 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f3519890 con 0x55a7f3646e60 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7f3534ea0 con 0x55a7f361d950 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f9913cddb80 10 client.0 _ll_register_callbacks cb 0x55a7f340af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f98ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 0x55a7f36146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f98ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 0x55a7f36146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40570/0 (socket says 172.21.15.33:40570) 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f98ff7fe700 1 -- 172.21.15.33:0/1072073169 learned_addr learned my addr 172.21.15.33:0/1072073169 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f98feffd700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f361d950 0x55a7f3661990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.669+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f36141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98ff7fe700 1 -- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f361d950 msgr2=0x55a7f3661990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f361d950 0x55a7f3661990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98ff7fe700 1 -- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 msgr2=0x55a7f36141a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f36141a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98ff7fe700 1 -- 172.21.15.33:0/1072073169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990015b0a0 con 0x55a7f3667bc0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 0x55a7f36146e0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f98f800ce30 tx=0x7f98f8082a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882617add922f7d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98dd7fa700 1 -- 172.21.15.33:0/1072073169 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f806f070 con 0x55a7f3667bc0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98dd7fa700 1 -- 172.21.15.33:0/1072073169 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98f805c4c0 con 0x55a7f3667bc0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7f3612690 con 0x55a7f3667bc0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98dd7fa700 1 -- 172.21.15.33:0/1072073169 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f8055990 con 0x55a7f3667bc0 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f9913cddb80 10 client.11649 fetch_fsmap learned FSMap version 195 2023-10-06T05:32:31.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=195,monmap=2+}) v3 -- 0x55a7f33aafe0 con 0x55a7f3667bc0 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98dd7fa700 1 -- 172.21.15.33:0/1072073169 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7f3612690 con 0x55a7f3667bc0 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.670+0000 7f98dd7fa700 1 -- 172.21.15.33:0/1072073169 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 195) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f98f8054a10 con 0x55a7f3667bc0 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f9913cddb80 10 client.11649 fetch_fsmap finished waiting for FSMap version 195 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f9913cddb80 10 client.11649 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f9913cddb80 20 client.11649 populate_metadata read hostname 'smithi033' 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f9913cddb80 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f36458e0 0x55a7f3645cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f9913cddb80 4 client.11649 mds_command: new command op to 11496 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 --> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7f35ef5c0 con 0x55a7f36458e0 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f98feffd700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f36458e0 0x55a7f3645cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f98feffd700 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f36458e0 0x55a7f3645cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5525cb39f3eccab4 server_cookie=13534c5e3628c44b in_seq=0 out_seq=0 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.671+0000 7f98dd7fa700 10 client.11649 ms_handle_connect on v2:172.21.15.33:6834/3696795298 2023-10-06T05:32:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f98dd7fa700 1 -- 172.21.15.33:0/1072073169 <== mds.0 v2:172.21.15.33:6834/3696795298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a7f35ef5c0 con 0x55a7f36458e0 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f98dd7fa700 10 client.11649 handle_command_reply: tid=0 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f9913cddb80 1 client.11649 shutdown 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 msgr2=0x55a7f36146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f9913cddb80 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 0x55a7f36146e0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f98f800ce30 tx=0x7f98f8082a70 comp rx=0 tx=0).stop 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 shutdown_connections 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.672+0000 7f9913cddb80 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f3667bc0 0x55a7f36146e0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f361d950 0x55a7f3661990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f36458e0 0x55a7f3645cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 1 --2- 172.21.15.33:0/1072073169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f3646e60 0x55a7f36141a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 >> 172.21.15.33:0/1072073169 conn(0x55a7f361f270 msgr2=0x55a7f361e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f98dd7fa700 0 client.11649 ms_handle_reset on v2:172.21.15.33:6834/3696795298 2023-10-06T05:32:31.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 shutdown_connections 2023-10-06T05:32:31.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 1 -- 172.21.15.33:0/1072073169 wait complete. 2023-10-06T05:32:31.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.673+0000 7f9913cddb80 20 client.11649 trim_cache size 0 max 16384 2023-10-06T05:32:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.690+0000 7f9913cddb80 1 Processor -- start 2023-10-06T05:32:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.690+0000 7f9913cddb80 1 -- start start 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f3642dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f3643300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f361dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f9913cddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f33f0e30 con 0x55a7f361d950 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f365dc40 con 0x55a7f361f270 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7f3646d00 con 0x55a7f3646e60 2023-10-06T05:32:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.691+0000 7f98fe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f3643300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f3643300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40584/0 (socket says 172.21.15.33:40584) 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 -- 172.21.15.33:0/3747924569 learned_addr learned my addr 172.21.15.33:0/3747924569 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98feffd700 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f3642dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f361dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 -- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 msgr2=0x55a7f361dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f361dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 -- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 msgr2=0x55a7f3642dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f3642dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 -- 172.21.15.33:0/3747924569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f353c780 con 0x55a7f361d950 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f3643300 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f98f40011e0 tx=0x7f98f4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8a51b973ceef97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98e57fa700 1 -- 172.21.15.33:0/3747924569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40016f0 con 0x55a7f361d950 2023-10-06T05:32:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98e57fa700 1 -- 172.21.15.33:0/3747924569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4001870 con 0x55a7f361d950 2023-10-06T05:32:31.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.692+0000 7f98e57fa700 1 -- 172.21.15.33:0/3747924569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4001c30 con 0x55a7f361d950 2023-10-06T05:32:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 -- 172.21.15.33:0/3747924569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 msgr2=0x55a7f3643300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f3643300 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f98f40011e0 tx=0x7f98f4009040 comp rx=0 tx=0).stop 2023-10-06T05:32:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 -- 172.21.15.33:0/3747924569 shutdown_connections 2023-10-06T05:32:31.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f361dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f3642dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3747924569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f3643300 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 -- 172.21.15.33:0/3747924569 >> 172.21.15.33:0/3747924569 conn(0x55a7f361c9b0 msgr2=0x55a7f3660d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 -- 172.21.15.33:0/3747924569 shutdown_connections 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.693+0000 7f9913cddb80 1 -- 172.21.15.33:0/3747924569 wait complete. 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.694+0000 7f9913cddb80 1 -- start start 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.694+0000 7f9913cddb80 1 Processor -- start 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f36b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f365bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f365c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f3681560 con 0x55a7f361d950 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f36cb230 con 0x55a7f361f270 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7f36ca900 con 0x55a7f3646e60 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f9913cddb80 10 client.0 _ll_register_callbacks cb 0x55a7f340af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f98feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f36b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f98feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f36b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40598/0 (socket says 172.21.15.33:40598) 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f98feffd700 1 -- 172.21.15.33:0/3123863237 learned_addr learned my addr 172.21.15.33:0/3123863237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f365bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.695+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f365c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98feffd700 1 -- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 msgr2=0x55a7f365c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98feffd700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f365c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98feffd700 1 -- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 msgr2=0x55a7f365bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98feffd700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f365bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98feffd700 1 -- 172.21.15.33:0/3123863237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f4007040 con 0x55a7f361d950 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98ff7fe700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f365c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98feffd700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f36b3820 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f98f0030000 tx=0x7f98f001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa4b2a37b05aabe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98dd7fa700 1 -- 172.21.15.33:0/3123863237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f002c040 con 0x55a7f361d950 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98f0020040 con 0x55a7f361d950 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98dd7fa700 1 -- 172.21.15.33:0/3123863237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98f001abc0 con 0x55a7f361d950 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.696+0000 7f98dd7fa700 1 -- 172.21.15.33:0/3123863237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f0031680 con 0x55a7f361d950 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 10 client.11655 fetch_fsmap learned FSMap version 195 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=195,monmap=2+}) v3 -- 0x55a7f3523470 con 0x55a7f361d950 2023-10-06T05:32:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f98dd7fa700 1 -- 172.21.15.33:0/3123863237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0020040 con 0x55a7f361d950 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f98dd7fa700 1 -- 172.21.15.33:0/3123863237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 195) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f98f00317e0 con 0x55a7f361d950 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 10 client.11655 fetch_fsmap finished waiting for FSMap version 195 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 10 client.11655 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 20 client.11655 populate_metadata read hostname 'smithi033' 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f3667bc0 0x55a7f3680810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 4 client.11655 mds_command: new command op to 11496 tid=0[{"prefix": "status"}] 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 --> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a7f3510e00 con 0x55a7f3667bc0 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.697+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f3667bc0 0x55a7f3680810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f98fe7fc700 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f3667bc0 0x55a7f3680810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9b6b394acf3309 server_cookie=9f6ec4f944d30bb4 in_seq=0 out_seq=0 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f98dd7fa700 10 client.11655 ms_handle_connect on v2:172.21.15.33:6834/3696795298 2023-10-06T05:32:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f98dd7fa700 1 -- 172.21.15.33:0/3123863237 <== mds.0 v2:172.21.15.33:6834/3696795298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55a7f3510e00 con 0x55a7f3667bc0 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f98dd7fa700 10 client.11655 handle_command_reply: tid=0 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 client.11655 shutdown 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 msgr2=0x55a7f36b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f36b3820 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f98f0030000 tx=0x7f98f001c040 comp rx=0 tx=0).stop 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 shutdown_connections 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55a7f3646e60 0x55a7f365c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7f361d950 0x55a7f36b3820 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.33:6834/3696795298,v1:172.21.15.33:6835/3696795298] conn(0x55a7f3667bc0 0x55a7f3680810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 --2- 172.21.15.33:0/3123863237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7f361f270 0x55a7f365bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 >> 172.21.15.33:0/3123863237 conn(0x55a7f361c9b0 msgr2=0x55a7f365d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f98dd7fa700 0 client.11655 ms_handle_reset on v2:172.21.15.33:6834/3696795298 2023-10-06T05:32:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 shutdown_connections 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.698+0000 7f9913cddb80 1 -- 172.21.15.33:0/3123863237 wait complete. 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f9913cddb80 20 client.11655 trim_cache size 0 max 16384 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 11496, 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:32:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:32:31.708 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5181665469999999, 2023-10-06T05:32:31.708 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 195, 2023-10-06T05:32:31.708 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 251, 2023-10-06T05:32:31.708 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 251, 2023-10-06T05:32:31.708 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 14.388432592999999 2023-10-06T05:32:31.708 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98ec066f30 msgr2=0x7f98ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98ec066f30 0x7f98ec0693e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f98f00093f0 tx=0x7f98f0017000 comp rx=0 tx=0).stop 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 msgr2=0x7f99000c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c5080 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f98f800c840 tx=0x7f98f8001670 comp rx=0 tx=0).stop 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 shutdown_connections 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f98ec066f30 0x7f98ec0693e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9900140d20 0x7f99000c5080 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f990006a0d0 0x7f990013ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 --2- 172.21.15.33:0/771985262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99000c9210 0x7f99000c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 >> 172.21.15.33:0/771985262 conn(0x7f990006c560 msgr2=0x7f99000c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 shutdown_connections 2023-10-06T05:32:31.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.699+0000 7f98fffff700 1 -- 172.21.15.33:0/771985262 wait complete. 2023-10-06T05:32:31.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:32:31.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.904+0000 7f125539d700 1 Processor -- start 2023-10-06T05:32:31.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.904+0000 7f125539d700 1 -- start start 2023-10-06T05:32:31.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.905+0000 7f125539d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f12500b0a40 0x7f125006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.905+0000 7f125539d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125015e2d0 con 0x7f12500b0e10 2023-10-06T05:32:31.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.905+0000 7f125539d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125015d6c0 con 0x7f125015a0a0 2023-10-06T05:32:31.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.905+0000 7f125539d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125015d9a0 con 0x7f12500b0a40 2023-10-06T05:32:31.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f12500b0e10 0x7f12500b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37198/0 (socket says 172.21.15.33:37198) 2023-10-06T05:32:31.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124f7fe700 1 -- 172.21.15.33:0/998789176 learned_addr learned my addr 172.21.15.33:0/998789176 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:31.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124ffff700 1 --2- 172.21.15.33:0/998789176 >> v2:172.21.15.167:3300/0 conn(0x7f12500b0a40 0x7f125006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098390151 0 0) 0x7f125015e2d0 con 0x7f12500b0e10 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1238003570 con 0x7f12500b0e10 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618866400 0 0) 0x7f125015d6c0 con 0x7f125015a0a0 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f125015e2d0 con 0x7f125015a0a0 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124ffff700 1 -- 172.21.15.33:0/998789176 >> v1:172.21.15.167:6789/0 conn(0x7f125015a0a0 legacy=0x7f125015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124ffff700 1 -- 172.21.15.33:0/998789176 >> v1:172.21.15.33:6789/0 conn(0x7f12500b0e10 legacy=0x7f12500b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.906+0000 7f124ffff700 1 -- 172.21.15.33:0/998789176 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125015dc80 con 0x7f12500b0a40 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.907+0000 7f124ffff700 1 --2- 172.21.15.33:0/998789176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12500b0a40 0x7f125006ab00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f12400098e0 tx=0x7f12400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3982831c6910075 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.907+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240011630 con 0x7f12500b0a40 2023-10-06T05:32:31.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.907+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240011790 con 0x7f12500b0a40 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.907+0000 7f124effd700 1 -- 172.21.15.33:0/998789176 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240011b10 con 0x7f12500b0a40 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 -- 172.21.15.33:0/998789176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12500b0a40 msgr2=0x7f125006ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 --2- 172.21.15.33:0/998789176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12500b0a40 0x7f125006ab00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f12400098e0 tx=0x7f12400091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 -- 172.21.15.33:0/998789176 shutdown_connections 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 --2- 172.21.15.33:0/998789176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12500b0a40 0x7f125006ab00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 -- 172.21.15.33:0/998789176 >> 172.21.15.33:0/998789176 conn(0x7f125006e0f0 msgr2=0x7f12500675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 -- 172.21.15.33:0/998789176 shutdown_connections 2023-10-06T05:32:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.908+0000 7f125539d700 1 -- 172.21.15.33:0/998789176 wait complete. 2023-10-06T05:32:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.909+0000 7f125539d700 1 Processor -- start 2023-10-06T05:32:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.909+0000 7f125539d700 1 -- start start 2023-10-06T05:32:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.910+0000 7f125539d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12500b0a40 0x7f12500c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.910+0000 7f125539d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 0x7f12500ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.910+0000 7f125539d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125015a0a0 0x7f12500addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.910+0000 7f125539d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125016a170 con 0x7f12500b0e10 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.910+0000 7f125539d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500ab8a0 con 0x7f125015a0a0 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.910+0000 7f125539d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12500c2660 con 0x7f12500b0a40 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 0x7f12500ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 0x7f12500ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40612/0 (socket says 172.21.15.33:40612) 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 -- 172.21.15.33:0/1771101084 learned_addr learned my addr 172.21.15.33:0/1771101084 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124ffff700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12500b0a40 0x7f12500c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f1254b9c700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125015a0a0 0x7f12500addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 -- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12500b0a40 msgr2=0x7f12500c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12500b0a40 0x7f12500c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 -- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125015a0a0 msgr2=0x7f12500addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125015a0a0 0x7f12500addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 -- 172.21.15.33:0/1771101084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009420 con 0x7f12500b0e10 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f1254b9c700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125015a0a0 0x7f12500addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f124f7fe700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 0x7f12500ab360 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f124400c790 tx=0x7f1244007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.911+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244005b30 con 0x7f12500b0e10 2023-10-06T05:32:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.912+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244005c90 con 0x7f12500b0e10 2023-10-06T05:32:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.912+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12500c2940 con 0x7f12500b0e10 2023-10-06T05:32:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.912+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244007300 con 0x7f12500b0e10 2023-10-06T05:32:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.912+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12500c2f00 con 0x7f12500b0e10 2023-10-06T05:32:31.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.913+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125016a2b0 con 0x7f12500b0e10 2023-10-06T05:32:31.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.913+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f124401b3b0 con 0x7f12500b0e10 2023-10-06T05:32:31.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.914+0000 7f1237fff700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f123c066f30 0x7f123c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:31.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.914+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1244083cc0 con 0x7f12500b0e10 2023-10-06T05:32:31.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.915+0000 7f124ffff700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f123c066f30 0x7f123c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:31.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.915+0000 7f124ffff700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f123c066f30 0x7f123c0693e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f12400093f0 tx=0x7f1240017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:31.918+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f125016a2b0 con 0x7f12500b0e10 2023-10-06T05:32:32.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.166+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f125016a2b0 con 0x7f12500b0e10 2023-10-06T05:32:32.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.167+0000 7f1237fff700 1 -- 172.21.15.33:0/1771101084 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v195) v1 ==== 53+0+73 (secure 0 0 0) 0x7f125016a2b0 con 0x7f12500b0e10 2023-10-06T05:32:32.169 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:32:32.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f123c066f30 msgr2=0x7f123c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:32.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f123c066f30 0x7f123c0693e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f12400093f0 tx=0x7f1240017000 comp rx=0 tx=0).stop 2023-10-06T05:32:32.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 msgr2=0x7f12500ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 0x7f12500ab360 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f124400c790 tx=0x7f1244007ea0 comp rx=0 tx=0).stop 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 shutdown_connections 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f123c066f30 0x7f123c0693e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125015a0a0 0x7f12500addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12500b0a40 0x7f12500c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 --2- 172.21.15.33:0/1771101084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12500b0e10 0x7f12500ab360 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 >> 172.21.15.33:0/1771101084 conn(0x7f125006e0f0 msgr2=0x7f125015c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 shutdown_connections 2023-10-06T05:32:32.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:32.169+0000 7f125539d700 1 -- 172.21.15.33:0/1771101084 wait complete. 2023-10-06T05:32:32.191 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:32:32.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:32:32.192 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:32:32.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.192 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:32:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:32:32.230 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:32:32.230 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:32:32.230 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41205sec preferred_lft 41205sec 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:32:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:32:32.234 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:32:32.234 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:32.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.234 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:32:32.330 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:32:32.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.334 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:32:32.372 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:32:32.372 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:32:32.372 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:32:32.372 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:32:32.372 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:32:32.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.374 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:32:32.374 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:32:32.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:32:32.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:32:32.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:32:32.374 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:32:32.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.572 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:32:32.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:32:32.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:32:32.609 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:32:32.609 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:32:32.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:32:32.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:32:32.712 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:32:32.748 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:32:32.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:32:32.748 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:32:32.824 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:32:32.824 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:32:32.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:32:32.956 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:32:32.957 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:32:32.957 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:32:32.972 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:32:32.973 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:32:32.973 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:32:32.973 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:32:32.973 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:32:32.994 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:32:32.998 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:32:32.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:32.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:32:33.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:32:33.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.245+0000 7fdd0299d700 1 Processor -- start 2023-10-06T05:32:33.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.245+0000 7fdd0299d700 1 -- start start 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0299d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdcfc0adf50 0x7fdcfc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0299d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdcfc0c9810 0x7fdcfc0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0299d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc0c8e40 con 0x7fdcfc0c9810 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0299d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc0ae6c0 con 0x7fdcfc0adf50 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0ae9a0 con 0x7fdcfc155140 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdcfc0c9810 0x7fdcfc0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0219c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdcfc0c9810 0x7fdcfc0699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40618/0 (socket says 172.21.15.33:40618) 2023-10-06T05:32:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0199b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0219c700 1 -- 172.21.15.33:0/3476583958 learned_addr learned my addr 172.21.15.33:0/3476583958 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.247+0000 7fdd0119a700 1 --2- 172.21.15.33:0/3476583958 >> v2:172.21.15.167:3300/0 conn(0x7fdcfc0adf50 0x7fdcfc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd0219c700 1 -- 172.21.15.33:0/3476583958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 msgr2=0x7fdcfc158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd0219c700 1 --2- 172.21.15.33:0/3476583958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd0219c700 1 -- 172.21.15.33:0/3476583958 >> v2:172.21.15.167:3300/0 conn(0x7fdcfc0adf50 msgr2=0x7fdcfc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd0219c700 1 --2- 172.21.15.33:0/3476583958 >> v2:172.21.15.167:3300/0 conn(0x7fdcfc0adf50 0x7fdcfc0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd0219c700 1 -- 172.21.15.33:0/3476583958 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0c8ad0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd0219c700 1 --2- 172.21.15.33:0/3476583958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0699c0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0719a0 tx=0x7fdcf4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b5a7e4d16fcfd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd00999700 1 -- 172.21.15.33:0/3476583958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400e070 con 0x7fdcfc0c9810 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd00999700 1 -- 172.21.15.33:0/3476583958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400b970 con 0x7fdcfc0c9810 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.248+0000 7fdd00999700 1 -- 172.21.15.33:0/3476583958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40092e0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 -- 172.21.15.33:0/3476583958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 msgr2=0x7fdcfc0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 --2- 172.21.15.33:0/3476583958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0699c0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0719a0 tx=0x7fdcf4009840 comp rx=0 tx=0).stop 2023-10-06T05:32:33.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 -- 172.21.15.33:0/3476583958 shutdown_connections 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 --2- 172.21.15.33:0/3476583958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0699c0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 --2- 172.21.15.33:0/3476583958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 --2- 172.21.15.33:0/3476583958 >> v2:172.21.15.167:3300/0 conn(0x7fdcfc0adf50 0x7fdcfc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 -- 172.21.15.33:0/3476583958 >> 172.21.15.33:0/3476583958 conn(0x7fdcfc06d030 msgr2=0x7fdcfc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 -- 172.21.15.33:0/3476583958 shutdown_connections 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.249+0000 7fdd0299d700 1 -- 172.21.15.33:0/3476583958 wait complete. 2023-10-06T05:32:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.250+0000 7fdd0299d700 1 Processor -- start 2023-10-06T05:32:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.250+0000 7fdd0299d700 1 -- start start 2023-10-06T05:32:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.251+0000 7fdd0299d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcfc0adf50 0x7fdcfc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.251+0000 7fdd0299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.251+0000 7fdd0299d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.251+0000 7fdd0299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc151ea0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.251+0000 7fdd0299d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc167700 con 0x7fdcfc0adf50 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.251+0000 7fdd0299d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc154560 con 0x7fdcfc155140 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40624/0 (socket says 172.21.15.33:40624) 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 -- 172.21.15.33:0/1455631616 learned_addr learned my addr 172.21.15.33:0/1455631616 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0199b700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcfc0adf50 0x7fdcfc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0219c700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 -- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 msgr2=0x7fdcfc150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 -- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcfc0adf50 msgr2=0x7fdcfc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcfc0adf50 0x7fdcfc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 -- 172.21.15.33:0/1455631616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4009ae0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.252+0000 7fdd0119a700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0b12c0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00ec80 tx=0x7fdcec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.253+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00c850 con 0x7fdcfc0c9810 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.253+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00c9b0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.253+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00cd50 con 0x7fdcfc0c9810 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.253+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc151850 con 0x7fdcfc0c9810 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.253+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc0b1800 con 0x7fdcfc0c9810 2023-10-06T05:32:33.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.254+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc1516b0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.255+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcec013680 con 0x7fdcfc0c9810 2023-10-06T05:32:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.255+0000 7fdcf1ffb700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce0066f30 0x7fdce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.256+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdcec084cf0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.256+0000 7fdd0199b700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce0066f30 0x7fdce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:33.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.257+0000 7fdd0199b700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce0066f30 0x7fdce00693e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d10 tx=0x7fdcf80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:33.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.260+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcfc1516b0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.507+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdcfc1516b0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.508+0000 7fdcf1ffb700 1 -- 172.21.15.33:0/1455631616 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v195) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdcfc1516b0 con 0x7fdcfc0c9810 2023-10-06T05:32:33.511 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:32:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce0066f30 msgr2=0x7fdce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce0066f30 0x7fdce00693e0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d10 tx=0x7fdcf80091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 msgr2=0x7fdcfc0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0b12c0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00ec80 tx=0x7fdcec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 shutdown_connections 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdce0066f30 0x7fdce00693e0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcfc155140 0x7fdcfc150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcfc0adf50 0x7fdcfc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 --2- 172.21.15.33:0/1455631616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc0c9810 0x7fdcfc0b12c0 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 >> 172.21.15.33:0/1455631616 conn(0x7fdcfc06d030 msgr2=0x7fdcfc0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 shutdown_connections 2023-10-06T05:32:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:33.510+0000 7fdd0299d700 1 -- 172.21.15.33:0/1455631616 wait complete. 2023-10-06T05:32:33.532 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:32:33.532 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:32:33.533 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:32:33.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:33.533 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:32:33.575 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:32:33.575 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:32:33.575 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:32:33.575 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.575 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:32:33.575 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:33.576 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41207sec preferred_lft 41207sec 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:32:33.577 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:32:33.578 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.578 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:32:33.578 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:32:33.578 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:32:33.578 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:32:33.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:33.578 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:32:33.677 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:32:33.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:33.681 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:32:33.720 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:32:33.721 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:32:33.721 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:32:33.721 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:32:33.721 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:32:33.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:33.721 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:32:33.722 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:32:33.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:32:33.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:32:33.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:32:33.722 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:32:33.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:33.918 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:32:33.954 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:32:33.955 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:32:33.955 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:32:33.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:33.955 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:32:34.055 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:32:34.055 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:32:34.091 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:32:34.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:32:34.092 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:32:34.167 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:32:34.167 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:32:34.167 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:32:34.299 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:32:34.300 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:32:34.300 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:32:34.315 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:32:34.316 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:32:34.316 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:32:34.316 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:32:34.316 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:32:34.336 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:32:34.337 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:32:34.342 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:32:34.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:34.343 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:32:34.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:32:34.618 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:32:34.619 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:32:34.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:32:34.823 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:32:34.823 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:32:34.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.017+0000 7faa3359e700 1 Processor -- start 2023-10-06T05:32:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.017+0000 7faa3359e700 1 -- start start 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa3359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faa3413faa0 0x7faa3413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa3359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa340c9cb0 con 0x7faa340c7940 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa3359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa340ca270 con 0x7faa3406aad0 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa3359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3406b640 con 0x7faa3413faa0 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa31d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faa340c7940 0x7faa340c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37202/0 (socket says 172.21.15.33:37202) 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa31d9b700 1 -- 172.21.15.33:0/3710603629 learned_addr learned my addr 172.21.15.33:0/3710603629 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:35.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.019+0000 7faa3259c700 1 --2- 172.21.15.33:0/3710603629 >> v2:172.21.15.167:3300/0 conn(0x7faa3413faa0 0x7faa3413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3159a700 1 -- 172.21.15.33:0/3710603629 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766889599 0 0) 0x7faa340c9cb0 con 0x7faa340c7940 2023-10-06T05:32:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3159a700 1 -- 172.21.15.33:0/3710603629 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa1c003570 con 0x7faa340c7940 2023-10-06T05:32:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3259c700 1 -- 172.21.15.33:0/3710603629 >> v1:172.21.15.167:6789/0 conn(0x7faa3406aad0 legacy=0x7faa3406af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3159a700 1 -- 172.21.15.33:0/3710603629 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095877673 0 0) 0x7faa1c003570 con 0x7faa340c7940 2023-10-06T05:32:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3259c700 1 -- 172.21.15.33:0/3710603629 >> v1:172.21.15.33:6789/0 conn(0x7faa340c7940 legacy=0x7faa340c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3259c700 1 -- 172.21.15.33:0/3710603629 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3406b840 con 0x7faa3413faa0 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3259c700 1 --2- 172.21.15.33:0/3710603629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3413faa0 0x7faa3413fe70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faa24009860 tx=0x7faa240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5bfbd51a7513c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.020+0000 7faa3159a700 1 -- 172.21.15.33:0/3710603629 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28001470 con 0x7faa3413faa0 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3159a700 1 -- 172.21.15.33:0/3710603629 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa24011420 con 0x7faa3413faa0 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3159a700 1 -- 172.21.15.33:0/3710603629 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240117c0 con 0x7faa3413faa0 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 -- 172.21.15.33:0/3710603629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3413faa0 msgr2=0x7faa3413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 --2- 172.21.15.33:0/3710603629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3413faa0 0x7faa3413fe70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faa24009860 tx=0x7faa240091a0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 -- 172.21.15.33:0/3710603629 shutdown_connections 2023-10-06T05:32:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 --2- 172.21.15.33:0/3710603629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3413faa0 0x7faa3413fe70 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 -- 172.21.15.33:0/3710603629 >> 172.21.15.33:0/3710603629 conn(0x7faa3406d030 msgr2=0x7faa3406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 -- 172.21.15.33:0/3710603629 shutdown_connections 2023-10-06T05:32:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.021+0000 7faa3359e700 1 -- 172.21.15.33:0/3710603629 wait complete. 2023-10-06T05:32:35.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.023+0000 7faa3359e700 1 Processor -- start 2023-10-06T05:32:35.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.023+0000 7faa3359e700 1 -- start start 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 0x7faa3413c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa340c7940 0x7faa3416a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3413faa0 0x7faa340b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34168310 con 0x7faa3413faa0 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34167770 con 0x7faa3406aad0 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa341679e0 con 0x7faa340c7940 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 0x7faa3413c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3413faa0 0x7faa340b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 0x7faa3413c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:38704/0 (socket says 172.21.15.33:38704) 2023-10-06T05:32:35.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 -- 172.21.15.33:0/155802776 learned_addr learned my addr 172.21.15.33:0/155802776 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa31d9b700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa340c7940 0x7faa3416a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 -- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa340c7940 msgr2=0x7faa3416a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa340c7940 0x7faa3416a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 -- 172.21.15.33:0/155802776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3413faa0 msgr2=0x7faa340b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.024+0000 7faa3259c700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3413faa0 0x7faa340b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa3259c700 1 -- 172.21.15.33:0/155802776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24009420 con 0x7faa3406aad0 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa32d9d700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3413faa0 0x7faa340b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa3259c700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 0x7faa3413c440 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faa240030c0 tx=0x7faa24017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24010ac0 con 0x7faa3406aad0 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa24010c20 con 0x7faa3406aad0 2023-10-06T05:32:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2401d430 con 0x7faa3406aad0 2023-10-06T05:32:35.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa34167cc0 con 0x7faa3406aad0 2023-10-06T05:32:35.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.025+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3413c980 con 0x7faa3406aad0 2023-10-06T05:32:35.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.026+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3404c380 con 0x7faa3406aad0 2023-10-06T05:32:35.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.027+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa2401d7a0 con 0x7faa3406aad0 2023-10-06T05:32:35.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.028+0000 7faa227fc700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa14066f30 0x7faa140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:35.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.028+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa24086280 con 0x7faa3406aad0 2023-10-06T05:32:35.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.028+0000 7faa31d9b700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa14066f30 0x7faa140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:35.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.029+0000 7faa31d9b700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa14066f30 0x7faa140693e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faa34156170 tx=0x7faa28000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:35.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.033+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa240532f0 con 0x7faa3406aad0 2023-10-06T05:32:35.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.282+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa3406bb20 con 0x7faa3406aad0 2023-10-06T05:32:35.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.283+0000 7faa227fc700 1 -- 172.21.15.33:0/155802776 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7faa24056910 con 0x7faa3406aad0 2023-10-06T05:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa14066f30 msgr2=0x7faa140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa14066f30 0x7faa140693e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faa34156170 tx=0x7faa28000bc0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 msgr2=0x7faa3413c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 0x7faa3413c440 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faa240030c0 tx=0x7faa24017000 comp rx=0 tx=0).stop 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 shutdown_connections 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa14066f30 0x7faa140693e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3413faa0 0x7faa340b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa3406aad0 0x7faa3413c440 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 --2- 172.21.15.33:0/155802776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa340c7940 0x7faa3416a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:35.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 >> 172.21.15.33:0/155802776 conn(0x7faa3406d030 msgr2=0x7faa3406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:35.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 shutdown_connections 2023-10-06T05:32:35.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:35.286+0000 7faa3359e700 1 -- 172.21.15.33:0/155802776 wait complete. 2023-10-06T05:32:35.290 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:32:35.306 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:32:35.306 DEBUG:teuthology.orchestra.run.smithi033:> import time 2023-10-06T05:32:35.306 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T05:32:35.306 DEBUG:teuthology.orchestra.run.smithi033:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2023-10-06T05:32:35.306 DEBUG:teuthology.orchestra.run.smithi033:> f.write("NHKGT19IQ7FK2J37") 2023-10-06T05:32:35.306 DEBUG:teuthology.orchestra.run.smithi033:> f.flush() 2023-10-06T05:32:35.307 DEBUG:teuthology.orchestra.run.smithi033:> while True: 2023-10-06T05:32:35.307 DEBUG:teuthology.orchestra.run.smithi033:> time.sleep(1) 2023-10-06T05:32:35.307 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T05:32:35.311 DEBUG:teuthology.orchestra.run.smithi033:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/background_file 2023-10-06T05:32:35.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:32:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2023-10-06T05:32:36.351 DEBUG:teuthology.orchestra.run.smithi033:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/background_file 2023-10-06T05:32:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:16 2023-10-06T05:32:36.393 INFO:tasks.cephfs.mount:File background_file became visible with size 16 from 0 after 1s 2023-10-06T05:32:36.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:32:36.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.624+0000 7f372cd9d700 1 Processor -- start 2023-10-06T05:32:36.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.624+0000 7f372cd9d700 1 -- start start 2023-10-06T05:32:36.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f372cd9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f37280cb710 0x7f372806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f372cd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37280c9cb0 con 0x7f37280c8050 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f372cd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37280ca270 con 0x7f372813faa0 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f372cd9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372806b840 con 0x7f37280cb710 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f3727fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f37280cb710 0x7f372806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f3726ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f37280c8050 0x7f372806aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37216/0 (socket says 172.21.15.33:37216) 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f3726ffd700 1 -- 172.21.15.33:0/3686596167 learned_addr learned my addr 172.21.15.33:0/3686596167 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f3727fff700 1 -- 172.21.15.33:0/3686596167 >> v1:172.21.15.167:6789/0 conn(0x7f372813faa0 legacy=0x7f37280c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f3727fff700 1 -- 172.21.15.33:0/3686596167 >> v1:172.21.15.33:6789/0 conn(0x7f37280c8050 legacy=0x7f372806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f3727fff700 1 -- 172.21.15.33:0/3686596167 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372806bb20 con 0x7f37280cb710 2023-10-06T05:32:36.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.626+0000 7f37267fc700 1 -- 172.21.15.33:0/3686596167 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947837551 0 0) 0x7f37280c9cb0 con 0x7f37280c8050 2023-10-06T05:32:36.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f3727fff700 1 --2- 172.21.15.33:0/3686596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f372806b1e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f3720009e60 tx=0x7f3720009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7fb18c94c4d8a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:36.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f37267fc700 1 -- 172.21.15.33:0/3686596167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372000e070 con 0x7f37280cb710 2023-10-06T05:32:36.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f37267fc700 1 -- 172.21.15.33:0/3686596167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372000bc30 con 0x7f37280cb710 2023-10-06T05:32:36.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f37267fc700 1 -- 172.21.15.33:0/3686596167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720012430 con 0x7f37280cb710 2023-10-06T05:32:36.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f372cd9d700 1 -- 172.21.15.33:0/3686596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 msgr2=0x7f372806b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.627+0000 7f372cd9d700 1 --2- 172.21.15.33:0/3686596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f372806b1e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f3720009e60 tx=0x7f3720009780 comp rx=0 tx=0).stop 2023-10-06T05:32:36.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.628+0000 7f372cd9d700 1 -- 172.21.15.33:0/3686596167 shutdown_connections 2023-10-06T05:32:36.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.628+0000 7f372cd9d700 1 --2- 172.21.15.33:0/3686596167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f372806b1e0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.628+0000 7f372cd9d700 1 -- 172.21.15.33:0/3686596167 >> 172.21.15.33:0/3686596167 conn(0x7f372806d030 msgr2=0x7f372806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.628+0000 7f372cd9d700 1 -- 172.21.15.33:0/3686596167 shutdown_connections 2023-10-06T05:32:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.628+0000 7f372cd9d700 1 -- 172.21.15.33:0/3686596167 wait complete. 2023-10-06T05:32:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.629+0000 7f372cd9d700 1 Processor -- start 2023-10-06T05:32:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.629+0000 7f372cd9d700 1 -- start start 2023-10-06T05:32:36.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f372cd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37280c8050 0x7f372815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:36.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f372cd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f37280c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:36.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f372cd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f372813faa0 0x7f372813eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f372cd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372815d1d0 con 0x7f37280cb710 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f372cd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372815d4b0 con 0x7f37280c8050 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f372cd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37280c5790 con 0x7f372813faa0 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f3726ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f37280c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f3726ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f37280c3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40662/0 (socket says 172.21.15.33:40662) 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.630+0000 7f3726ffd700 1 -- 172.21.15.33:0/504507678 learned_addr learned my addr 172.21.15.33:0/504507678 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3727fff700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f372813faa0 0x7f372813eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f37277fe700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37280c8050 0x7f372815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3726ffd700 1 -- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f372813faa0 msgr2=0x7f372813eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3726ffd700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f372813faa0 0x7f372813eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3726ffd700 1 -- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37280c8050 msgr2=0x7f372815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3726ffd700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37280c8050 0x7f372815cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3726ffd700 1 -- 172.21.15.33:0/504507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720009a20 con 0x7f37280cb710 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f37277fe700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37280c8050 0x7f372815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f3726ffd700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f37280c3880 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f371c00c7e0 tx=0x7f371c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c005b30 con 0x7f37280cb710 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c005c90 con 0x7f37280cb710 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c007300 con 0x7f37280cb710 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.631+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37280c5a70 con 0x7f37280cb710 2023-10-06T05:32:36.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.632+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372813f060 con 0x7f37280cb710 2023-10-06T05:32:36.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.633+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372804c380 con 0x7f37280cb710 2023-10-06T05:32:36.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.633+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f371c0076b0 con 0x7f37280cb710 2023-10-06T05:32:36.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.634+0000 7f370f7fe700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3714066f30 0x7f37140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:32:36.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.634+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f371c083ca0 con 0x7f37280cb710 2023-10-06T05:32:36.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.635+0000 7f37277fe700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3714066f30 0x7f37140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:32:36.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.636+0000 7f37277fe700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3714066f30 0x7f37140693e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3718007fd0 tx=0x7f3718009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:32:36.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.639+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f371c050d10 con 0x7f37280cb710 2023-10-06T05:32:36.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.884+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37280c9f00 con 0x7f37280cb710 2023-10-06T05:32:36.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.884+0000 7f370f7fe700 1 -- 172.21.15.33:0/504507678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f371c017090 con 0x7f37280cb710 2023-10-06T05:32:36.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:32:36.887 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:32:36.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3714066f30 msgr2=0x7f37140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3714066f30 0x7f37140693e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3718007fd0 tx=0x7f3718009040 comp rx=0 tx=0).stop 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 msgr2=0x7f37280c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f37280c3880 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f371c00c7e0 tx=0x7f371c007df0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 shutdown_connections 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3714066f30 0x7f37140693e0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f372813faa0 0x7f372813eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37280c8050 0x7f372815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 --2- 172.21.15.33:0/504507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280cb710 0x7f37280c3880 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 >> 172.21.15.33:0/504507678 conn(0x7f372806d030 msgr2=0x7f372806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:32:36.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 shutdown_connections 2023-10-06T05:32:36.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:32:36.887+0000 7f372cd9d700 1 -- 172.21.15.33:0/504507678 wait complete. 2023-10-06T05:32:36.891 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:32:36.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T05:32:37.118 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":10543,"entity":{"name":{"type":"client","num":10543},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":3106862436}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":2.770956773,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9858355614917471,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.10543 v1:192.168.0.1:0/3106862436","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":10534,"entity":{"name":{"type":"client","num":10534},"addr":{"type":"v1","addr":"172.21.15.33:0","nonce":1348218675}},"state":"open","num_leases":1,"num_caps":2,"request_load_avg":3,"uptime":4.1129358280000003,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":4.9749722765650279,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.10534 v1:172.21.15.33:0/1348218675","completed_requests":[{"tid":4,"created_ino":"0x10000000000"}],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:32:37.119 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3106862436, "type": "v1" }, "name": { "num": 10543, "type": "client" } }, "id": 10543, "inst": "client.10543 v1:192.168.0.1:0/3106862436", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.985835561491747 }, "state": "open", "uptime": 2.770956773 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000000", "tid": 4 } ], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.33:0", "nonce": 1348218675, "type": "v1" }, "name": { "num": 10534, "type": "client" } }, "id": 10534, "inst": "client.10534 v1:172.21.15.33:0/1348218675", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4.974972276565028 }, "state": "open", "uptime": 4.112935828 } ] 2023-10-06T05:32:37.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:32:37.120 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:32:37.191 INFO:teuthology.orchestra.run.smithi033.stdout:client10534 2023-10-06T05:32:37.192 DEBUG:teuthology.orchestra.run.smithi167:> stat --printf=%s /home/ubuntu/cephtest/mnt.1/background_file 2023-10-06T05:32:37.239 INFO:teuthology.orchestra.run.smithi167.stdout:16 2023-10-06T05:32:37.239 INFO:tasks.cephfs.mount:File background_file became visible with size 16 from 1 after 0s 2023-10-06T05:32:37.239 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:32:37.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:32:37.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 down 2023-10-06T05:32:42.310 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:32:42.310 DEBUG:teuthology.orchestra.run.smithi167:> import os 2023-10-06T05:32:42.310 DEBUG:teuthology.orchestra.run.smithi167:> import time 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> try: 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> while True: 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> os.write(fd, b'"'"'content'"'"') 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> time.sleep(1) 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> if not False: 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> break 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> except IOError as e: 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> pass 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> os.close(fd) 2023-10-06T05:32:42.311 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T05:32:51.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:32:51.261 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:33:21.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:33:21.385 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:33:39.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:33:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.290+0000 7f9f5bfff700 1 Processor -- start 2023-10-06T05:33:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.291+0000 7f9f5bfff700 1 -- start start 2023-10-06T05:33:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c80d0 0x7f9f5c06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c06c820 con 0x7f9f5c159520 2023-10-06T05:33:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5bfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c06a160 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c8800 con 0x7f9f5c0c6340 2023-10-06T05:33:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9f5c159520 0x7f9f5c0c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47180/0 (socket says 172.21.15.33:47180) 2023-10-06T05:33:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5affd700 1 -- 172.21.15.33:0/3495225947 learned_addr learned my addr 172.21.15.33:0/3495225947 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5a7fc700 1 --2- 172.21.15.33:0/3495225947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.292+0000 7f9f5b7fe700 1 --2- 172.21.15.33:0/3495225947 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c80d0 0x7f9f5c06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f59ffb700 1 -- 172.21.15.33:0/3495225947 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919475872 0 0) 0x7f9f5c06c820 con 0x7f9f5c159520 2023-10-06T05:33:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f59ffb700 1 -- 172.21.15.33:0/3495225947 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f44003570 con 0x7f9f5c159520 2023-10-06T05:33:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3495225947 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c80d0 msgr2=0x7f9f5c06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f5a7fc700 1 --2- 172.21.15.33:0/3495225947 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c80d0 0x7f9f5c06a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3495225947 >> v1:172.21.15.33:6789/0 conn(0x7f9f5c159520 legacy=0x7f9f5c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3495225947 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0c8ae0 con 0x7f9f5c0c6340 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f5b7fe700 1 --2- 172.21.15.33:0/3495225947 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c80d0 0x7f9f5c06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.293+0000 7f9f5a7fc700 1 --2- 172.21.15.33:0/3495225947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c0c7b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f500098e0 tx=0x7f9f500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d1813401aea8cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f59ffb700 1 -- 172.21.15.33:0/3495225947 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50011630 con 0x7f9f5c0c6340 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f59ffb700 1 -- 172.21.15.33:0/3495225947 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50011790 con 0x7f9f5c0c6340 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f59ffb700 1 -- 172.21.15.33:0/3495225947 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50011b10 con 0x7f9f5c0c6340 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3495225947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 msgr2=0x7f9f5c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3495225947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c0c7b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f500098e0 tx=0x7f9f500091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3495225947 shutdown_connections 2023-10-06T05:33:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3495225947 >> v2:172.21.15.167:3300/0 conn(0x7f9f5c0c80d0 0x7f9f5c06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3495225947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c0c7b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3495225947 >> 172.21.15.33:0/3495225947 conn(0x7f9f5c0cc930 msgr2=0x7f9f5c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3495225947 shutdown_connections 2023-10-06T05:33:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.294+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3495225947 wait complete. 2023-10-06T05:33:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.296+0000 7f9f5bfff700 1 Processor -- start 2023-10-06T05:33:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.296+0000 7f9f5bfff700 1 -- start start 2023-10-06T05:33:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 0x7f9f5c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c159520 0x7f9f5c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c152780 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15e400 con 0x7f9f5c159520 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15ca70 con 0x7f9f5c0c6340 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 0x7f9f5c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 0x7f9f5c15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51192/0 (socket says 172.21.15.33:51192) 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3206827243 learned_addr learned my addr 172.21.15.33:0/3206827243 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5affd700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.297+0000 7f9f5b7fe700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c159520 0x7f9f5c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 msgr2=0x7f9f5c15c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5a7fc700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c15c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c159520 msgr2=0x7f9f5c1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5a7fc700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c159520 0x7f9f5c1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5a7fc700 1 -- 172.21.15.33:0/3206827243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54009a20 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5affd700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5a7fc700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 0x7f9f5c15dec0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0c8940 tx=0x7f9f50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50011900 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50011a80 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50009420 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5001a680 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.298+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c15d030 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.299+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c04c380 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.300+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f5001aa70 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.301+0000 7f9f42ffd700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f48066f30 0x7f9f480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.301+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9f50086370 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.301+0000 7f9f5affd700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f48066f30 0x7f9f480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.302+0000 7f9f5affd700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f48066f30 0x7f9f480693e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c16ce50 tx=0x7f9f4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.305+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f50053420 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.554+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f5c0c8dc0 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.555+0000 7f9f42ffd700 1 -- 172.21.15.33:0/3206827243 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f9f50056a40 con 0x7f9f5c0c80d0 2023-10-06T05:33:39.558 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:39.558 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f48066f30 msgr2=0x7f9f480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f48066f30 0x7f9f480693e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c16ce50 tx=0x7f9f4c009040 comp rx=0 tx=0).stop 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 msgr2=0x7f9f5c15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 0x7f9f5c15dec0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0c8940 tx=0x7f9f50017000 comp rx=0 tx=0).stop 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 shutdown_connections 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f48066f30 0x7f9f480693e0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f5c159520 0x7f9f5c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f5c0c6340 0x7f9f5c15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 --2- 172.21.15.33:0/3206827243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f5c0c80d0 0x7f9f5c15dec0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 >> 172.21.15.33:0/3206827243 conn(0x7f9f5c0cc930 msgr2=0x7f9f5c0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:39.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 shutdown_connections 2023-10-06T05:33:39.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:39.558+0000 7f9f5bfff700 1 -- 172.21.15.33:0/3206827243 wait complete. 2023-10-06T05:33:39.562 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:33:39.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-06T05:33:39.789 INFO:teuthology.orchestra.run.smithi033.stdout:[{"id":10543,"entity":{"name":{"type":"client","num":10543},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":3106862436}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":65.441979504000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":7.9066579667651524,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.10543 v1:192.168.0.1:0/3106862436","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:33:39.790 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3106862436, "type": "v1" }, "name": { "num": 10543, "type": "client" } }, "id": 10543, "inst": "client.10543 v1:192.168.0.1:0/3106862436", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 7.906657966765152 }, "state": "open", "uptime": 65.441979504 } ] 2023-10-06T05:33:39.790 INFO:tasks.cephfs.test_client_recovery:cap_waiter waited 56.777108669281006s 2023-10-06T05:33:39.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:33:39.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:33:39.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:33:39.867 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:33:39.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:33:39.991 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:33:40.097 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:33:40.097 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:33:40.097 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:33:40.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:33:40.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:33:40.494 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:33:40.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:33:40.495 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:33:40.532 INFO:tasks.cephfs.mount:Terminating background process 2023-10-06T05:33:40.532 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:33:40.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:33:40.532 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:33:40.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:33:40.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:33:40.631 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:33:40.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:33:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.855+0000 7f3669b9d700 1 Processor -- start 2023-10-06T05:33:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.855+0000 7f3669b9d700 1 -- start start 2023-10-06T05:33:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.856+0000 7f3669b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36640ccfc0 0x7f36640cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.856+0000 7f3669b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640cf3b0 0x7f36641594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.856+0000 7f3669b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36640cbd80 con 0x7f36640cc750 2023-10-06T05:33:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.856+0000 7f3669b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640af5f0 con 0x7f36640ccfc0 2023-10-06T05:33:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.856+0000 7f3669b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640af8d0 con 0x7f36640cf3b0 2023-10-06T05:33:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36640ccfc0 0x7f36640cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f36640ccfc0 0x7f36640cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56298/0 (socket says 172.21.15.33:56298) 2023-10-06T05:33:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 -- 172.21.15.33:0/1107789166 learned_addr learned my addr 172.21.15.33:0/1107789166 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:40.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3663fff700 1 --2- 172.21.15.33:0/1107789166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640cf3b0 0x7f36641594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 -- 172.21.15.33:0/1107789166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640cf3b0 msgr2=0x7f36641594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 --2- 172.21.15.33:0/1107789166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640cf3b0 0x7f36641594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 -- 172.21.15.33:0/1107789166 >> v1:172.21.15.167:6789/0 conn(0x7f36640cc750 legacy=0x7f36640b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 -- 172.21.15.33:0/1107789166 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640afbb0 con 0x7f36640ccfc0 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.857+0000 7f3668b9b700 1 --2- 172.21.15.33:0/1107789166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640ccfc0 0x7f36640cc210 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f36580099b0 tx=0x7f3658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434dd13cc691bc9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f36637fe700 1 -- 172.21.15.33:0/1107789166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800b860 con 0x7f36640ccfc0 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f36637fe700 1 -- 172.21.15.33:0/1107789166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800b9c0 con 0x7f36640ccfc0 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f36637fe700 1 -- 172.21.15.33:0/1107789166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800bd60 con 0x7f36640ccfc0 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 -- 172.21.15.33:0/1107789166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640ccfc0 msgr2=0x7f36640cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:40.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 --2- 172.21.15.33:0/1107789166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640ccfc0 0x7f36640cc210 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f36580099b0 tx=0x7f3658009250 comp rx=0 tx=0).stop 2023-10-06T05:33:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 -- 172.21.15.33:0/1107789166 shutdown_connections 2023-10-06T05:33:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 --2- 172.21.15.33:0/1107789166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640ccfc0 0x7f36640cc210 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 --2- 172.21.15.33:0/1107789166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640cf3b0 0x7f36641594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 -- 172.21.15.33:0/1107789166 >> 172.21.15.33:0/1107789166 conn(0x7f366406cff0 msgr2=0x7f366406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 -- 172.21.15.33:0/1107789166 shutdown_connections 2023-10-06T05:33:40.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.858+0000 7f3669b9d700 1 -- 172.21.15.33:0/1107789166 wait complete. 2023-10-06T05:33:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.860+0000 7f3669b9d700 1 Processor -- start 2023-10-06T05:33:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.860+0000 7f3669b9d700 1 -- start start 2023-10-06T05:33:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3669b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 0x7f366416dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3669b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640ccfc0 0x7f36640b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3669b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36640b5ec0 0x7f3664142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3669b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b3a60 con 0x7f36640cc750 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3669b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b4ea0 con 0x7f36640b5ec0 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3669b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b37e0 con 0x7f36640ccfc0 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3668b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 0x7f366416dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3668b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 0x7f366416dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56308/0 (socket says 172.21.15.33:56308) 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3668b9b700 1 -- 172.21.15.33:0/4030242615 learned_addr learned my addr 172.21.15.33:0/4030242615 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f3663fff700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640ccfc0 0x7f36640b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.861+0000 7f366939c700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36640b5ec0 0x7f3664142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:40.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3668b9b700 1 -- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640ccfc0 msgr2=0x7f36640b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3668b9b700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640ccfc0 0x7f36640b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3668b9b700 1 -- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36640b5ec0 msgr2=0x7f3664142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3668b9b700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36640b5ec0 0x7f3664142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3668b9b700 1 -- 172.21.15.33:0/4030242615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580094f0 con 0x7f36640cc750 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f366939c700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36640b5ec0 0x7f3664142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3668b9b700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 0x7f366416dba0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f366406be20 tx=0x7f3658017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658024440 con 0x7f36640cc750 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36580245c0 con 0x7f36640cc750 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36640b5120 con 0x7f36640cc750 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658024980 con 0x7f36640cc750 2023-10-06T05:33:40.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.862+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640b56c0 con 0x7f36640cc750 2023-10-06T05:33:40.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.863+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366404c380 con 0x7f36640cc750 2023-10-06T05:33:40.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.864+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f365801a420 con 0x7f36640cc750 2023-10-06T05:33:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.865+0000 7f3660ff9700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3648066f30 0x7f36480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.865+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3658086490 con 0x7f36640cc750 2023-10-06T05:33:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.866+0000 7f3663fff700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3648066f30 0x7f36480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.866+0000 7f3663fff700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3648066f30 0x7f36480693e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f36640b27b0 tx=0x7f3650008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:40.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:40.870+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3658053480 con 0x7f36640cc750 2023-10-06T05:33:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.116+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36640d1330 con 0x7f36640cc750 2023-10-06T05:33:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.116+0000 7f3660ff9700 1 -- 172.21.15.33:0/4030242615 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f3658056aa0 con 0x7f36640cc750 2023-10-06T05:33:41.119 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:41.120 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:33:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3648066f30 msgr2=0x7f36480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3648066f30 0x7f36480693e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f36640b27b0 tx=0x7f3650008040 comp rx=0 tx=0).stop 2023-10-06T05:33:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 msgr2=0x7f366416dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 0x7f366416dba0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f366406be20 tx=0x7f3658017000 comp rx=0 tx=0).stop 2023-10-06T05:33:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 shutdown_connections 2023-10-06T05:33:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3648066f30 0x7f36480693e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36640b5ec0 0x7f3664142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36640cc750 0x7f366416dba0 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 --2- 172.21.15.33:0/4030242615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36640ccfc0 0x7f36640b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 >> 172.21.15.33:0/4030242615 conn(0x7f366406cff0 msgr2=0x7f366414ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 shutdown_connections 2023-10-06T05:33:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.119+0000 7f3669b9d700 1 -- 172.21.15.33:0/4030242615 wait complete. 2023-10-06T05:33:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:33:41.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:33:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.323+0000 7f0313fff700 1 Processor -- start 2023-10-06T05:33:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.323+0000 7f0313fff700 1 -- start start 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.324+0000 7f0313fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f031406ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.324+0000 7f0313fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f031415a0a0 0x7f031415a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.324+0000 7f0313fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031415e2d0 con 0x7f03140b0e10 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f0313fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031415d6c0 con 0x7f031415a0a0 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f0313fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031415d9a0 con 0x7f03140b0a40 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f031415a0a0 0x7f031415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f031415a0a0 0x7f031415a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56324/0 (socket says 172.21.15.33:56324) 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 -- 172.21.15.33:0/224487732 learned_addr learned my addr 172.21.15.33:0/224487732 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f0312ffd700 1 --2- 172.21.15.33:0/224487732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f031406ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 -- 172.21.15.33:0/224487732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 msgr2=0x7f031406ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 --2- 172.21.15.33:0/224487732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f031406ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 -- 172.21.15.33:0/224487732 >> v1:172.21.15.167:6789/0 conn(0x7f03140b0e10 legacy=0x7f03140b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.325+0000 7f03137fe700 1 -- 172.21.15.33:0/224487732 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031415dc80 con 0x7f031415a0a0 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f03137fe700 1 --2- 172.21.15.33:0/224487732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031415a0a0 0x7f031415a520 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f030c009f20 tx=0x7f030c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337bc12db5a11968 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0311ffb700 1 -- 172.21.15.33:0/224487732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00e070 con 0x7f031415a0a0 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0311ffb700 1 -- 172.21.15.33:0/224487732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00b9d0 con 0x7f031415a0a0 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0311ffb700 1 -- 172.21.15.33:0/224487732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0092e0 con 0x7f031415a0a0 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 -- 172.21.15.33:0/224487732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031415a0a0 msgr2=0x7f031415a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 --2- 172.21.15.33:0/224487732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031415a0a0 0x7f031415a520 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f030c009f20 tx=0x7f030c009840 comp rx=0 tx=0).stop 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 -- 172.21.15.33:0/224487732 shutdown_connections 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 --2- 172.21.15.33:0/224487732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031415a0a0 0x7f031415a520 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 --2- 172.21.15.33:0/224487732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f031406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 -- 172.21.15.33:0/224487732 >> 172.21.15.33:0/224487732 conn(0x7f031406e0f0 msgr2=0x7f03140675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:41.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.326+0000 7f0313fff700 1 -- 172.21.15.33:0/224487732 shutdown_connections 2023-10-06T05:33:41.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.327+0000 7f0313fff700 1 -- 172.21.15.33:0/224487732 wait complete. 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.328+0000 7f0313fff700 1 Processor -- start 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.328+0000 7f0313fff700 1 -- start start 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f0313fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f0314147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f0313fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 0x7f031416dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f0313fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f031415a0a0 0x7f03140ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f0313fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031416e2c0 con 0x7f03140b0e10 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f0313fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140c3780 con 0x7f031415a0a0 2023-10-06T05:33:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f0313fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140c2b70 con 0x7f03140b0a40 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f03127fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 0x7f031416dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f03127fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 0x7f031416dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56340/0 (socket says 172.21.15.33:56340) 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.329+0000 7f03127fc700 1 -- 172.21.15.33:0/3747421125 learned_addr learned my addr 172.21.15.33:0/3747421125 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f0312ffd700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f0314147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03137fe700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f031415a0a0 0x7f03140ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03127fc700 1 -- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 msgr2=0x7f0314147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03127fc700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f0314147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03127fc700 1 -- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f031415a0a0 msgr2=0x7f03140ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03127fc700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f031415a0a0 0x7f03140ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03127fc700 1 -- 172.21.15.33:0/3747421125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009ae0 con 0x7f03140b0e10 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f03127fc700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 0x7f031416dd80 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f030800c590 tx=0x7f030800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308011590 con 0x7f03140b0e10 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03080116f0 con 0x7f03140b0e10 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308011ab0 con 0x7f03140b0e10 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03140c2e50 con 0x7f03140b0e10 2023-10-06T05:33:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.330+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03140c3410 con 0x7f03140b0e10 2023-10-06T05:33:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.331+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031406fcc0 con 0x7f03140b0e10 2023-10-06T05:33:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.332+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f030801b3b0 con 0x7f03140b0e10 2023-10-06T05:33:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.333+0000 7f02faffd700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0300066f30 0x7f03000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.334+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f0308083d60 con 0x7f03140b0e10 2023-10-06T05:33:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.334+0000 7f0312ffd700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0300066f30 0x7f03000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.335+0000 7f0312ffd700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0300066f30 0x7f03000693e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0304009d40 tx=0x7f0304008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.337+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0308050d50 con 0x7f03140b0e10 2023-10-06T05:33:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.540+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03140c38f0 con 0x7f03140b0e10 2023-10-06T05:33:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.541+0000 7f02faffd700 1 -- 172.21.15.33:0/3747421125 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f0308017020 con 0x7f03140b0e10 2023-10-06T05:33:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:41.545 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":254,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:38.046427+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:32:21.814695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":38,"pool_name":"cephfs_data","create_time":"2023-10-06T05:32:22.631382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"38.1","mappings":[{"from":7,"to":4}]},{"pgid":"38.17","mappings":[{"from":7,"to":2}]},{"pgid":"38.1c","mappings":[{"from":7,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1348218675":"2023-10-06T06:33:37.316586+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:33:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0300066f30 msgr2=0x7f03000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0300066f30 0x7f03000693e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0304009d40 tx=0x7f0304008040 comp rx=0 tx=0).stop 2023-10-06T05:33:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 msgr2=0x7f031416dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 0x7f031416dd80 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f030800c590 tx=0x7f030800c950 comp rx=0 tx=0).stop 2023-10-06T05:33:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 shutdown_connections 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0300066f30 0x7f03000693e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f031415a0a0 0x7f03140ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03140b0a40 0x7f0314147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 --2- 172.21.15.33:0/3747421125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03140b0e10 0x7f031416dd80 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 >> 172.21.15.33:0/3747421125 conn(0x7f031406e0f0 msgr2=0x7f031415cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 shutdown_connections 2023-10-06T05:33:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.544+0000 7f0313fff700 1 -- 172.21.15.33:0/3747421125 wait complete. 2023-10-06T05:33:41.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 37, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:32:21.814695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '251', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 38, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:32:22.631382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '251', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:33:41.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:33:41.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.752+0000 7fb1ea99d700 1 Processor -- start 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.752+0000 7fb1ea99d700 1 -- start start 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.753+0000 7fb1ea99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e40cc7e0 0x7fb1e40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.753+0000 7fb1ea99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1e40b01f0 0x7fb1e40b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.753+0000 7fb1ea99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40ad430 con 0x7fb1e4156370 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.753+0000 7fb1ea99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40afe20 con 0x7fb1e40b01f0 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.753+0000 7fb1ea99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b0b70 con 0x7fb1e40cc7e0 2023-10-06T05:33:41.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1e999b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb1e4156370 0x7fb1e40c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43902/0 (socket says 172.21.15.33:43902) 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1e999b700 1 -- 172.21.15.33:0/2341031051 learned_addr learned my addr 172.21.15.33:0/2341031051 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1e919a700 1 --2- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e40cc7e0 0x7fb1e40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1ea19c700 1 --2- 172.21.15.33:0/2341031051 >> v2:172.21.15.167:3300/0 conn(0x7fb1e40b01f0 0x7fb1e40b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1e8999700 1 -- 172.21.15.33:0/2341031051 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11938590 0 0) 0x7fb1e40ad430 con 0x7fb1e4156370 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1e8999700 1 -- 172.21.15.33:0/2341031051 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003570 con 0x7fb1e4156370 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1ea19c700 1 -- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e40cc7e0 msgr2=0x7fb1e40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1ea19c700 1 --2- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e40cc7e0 0x7fb1e40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1e8999700 1 -- 172.21.15.33:0/2341031051 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1429496428 0 0) 0x7fb1d0003570 con 0x7fb1e4156370 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1ea19c700 1 -- 172.21.15.33:0/2341031051 >> v1:172.21.15.33:6789/0 conn(0x7fb1e4156370 legacy=0x7fb1e40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.754+0000 7fb1ea19c700 1 -- 172.21.15.33:0/2341031051 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e406bc70 con 0x7fb1e40b01f0 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.755+0000 7fb1ea19c700 1 --2- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40b01f0 0x7fb1e40b0630 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009e60 tx=0x7fb1dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f883bb573ef5bed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.755+0000 7fb1e8999700 1 -- 172.21.15.33:0/2341031051 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00e070 con 0x7fb1e40b01f0 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.755+0000 7fb1e8999700 1 -- 172.21.15.33:0/2341031051 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00b7a0 con 0x7fb1e40b01f0 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.755+0000 7fb1e8999700 1 -- 172.21.15.33:0/2341031051 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0001470 con 0x7fb1e40b01f0 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 -- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40b01f0 msgr2=0x7fb1e40b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40b01f0 0x7fb1e40b0630 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009e60 tx=0x7fb1dc009780 comp rx=0 tx=0).stop 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 -- 172.21.15.33:0/2341031051 shutdown_connections 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40b01f0 0x7fb1e40b0630 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/2341031051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e40cc7e0 0x7fb1e40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 -- 172.21.15.33:0/2341031051 >> 172.21.15.33:0/2341031051 conn(0x7fb1e406d030 msgr2=0x7fb1e40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:41.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 -- 172.21.15.33:0/2341031051 shutdown_connections 2023-10-06T05:33:41.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.756+0000 7fb1ea99d700 1 -- 172.21.15.33:0/2341031051 wait complete. 2023-10-06T05:33:41.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.757+0000 7fb1ea99d700 1 Processor -- start 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.757+0000 7fb1ea99d700 1 -- start start 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1ea99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 0x7fb1e414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1ea99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40cc7e0 0x7fb1e4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1ea99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e4156370 0x7fb1e4158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1ea99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41595f0 con 0x7fb1e40b01f0 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1ea99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4157330 con 0x7fb1e40cc7e0 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1ea99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41504f0 con 0x7fb1e4156370 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.758+0000 7fb1e999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 0x7fb1e414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 0x7fb1e414c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56352/0 (socket says 172.21.15.33:56352) 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 -- 172.21.15.33:0/532090010 learned_addr learned my addr 172.21.15.33:0/532090010 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:41.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1ea19c700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e4156370 0x7fb1e4158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e919a700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40cc7e0 0x7fb1e4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 -- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e4156370 msgr2=0x7fb1e4158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e4156370 0x7fb1e4158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 -- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40cc7e0 msgr2=0x7fb1e4156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40cc7e0 0x7fb1e4156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 -- 172.21.15.33:0/532090010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc009a20 con 0x7fb1e40b01f0 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e919a700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40cc7e0 0x7fb1e4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1e999b700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 0x7fb1e414c7f0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000cb90 tx=0x7fb1e0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:41.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0001d10 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0007360 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e4159030 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.759+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0007700 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.760+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e4150790 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.761+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e41509b0 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.762+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1e0004a40 con 0x7fb1e40b01f0 2023-10-06T05:33:41.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.762+0000 7fb1d9ffb700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1c8066f30 0x7fb1c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.762+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb1e0084100 con 0x7fb1e40b01f0 2023-10-06T05:33:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.763+0000 7fb1e919a700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1c8066f30 0x7fb1c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.763+0000 7fb1e919a700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1c8066f30 0x7fb1c80693e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ae470 tx=0x7fb1d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:41.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.766+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1e00510f0 con 0x7fb1e40b01f0 2023-10-06T05:33:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.967+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1e404c380 con 0x7fb1e40b01f0 2023-10-06T05:33:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.968+0000 7fb1d9ffb700 1 -- 172.21.15.33:0/532090010 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v254) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1e00180a0 con 0x7fb1e40b01f0 2023-10-06T05:33:41.970 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:33:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1c8066f30 msgr2=0x7fb1c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1c8066f30 0x7fb1c80693e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ae470 tx=0x7fb1d4008040 comp rx=0 tx=0).stop 2023-10-06T05:33:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 msgr2=0x7fb1e414c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 0x7fb1e414c7f0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000cb90 tx=0x7fb1e0001a80 comp rx=0 tx=0).stop 2023-10-06T05:33:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 shutdown_connections 2023-10-06T05:33:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb1c8066f30 0x7fb1c80693e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1e4156370 0x7fb1e4158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1e40b01f0 0x7fb1e414c7f0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 --2- 172.21.15.33:0/532090010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1e40cc7e0 0x7fb1e4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 >> 172.21.15.33:0/532090010 conn(0x7fb1e406d030 msgr2=0x7fb1e414fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 shutdown_connections 2023-10-06T05:33:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:41.970+0000 7fb1ea99d700 1 -- 172.21.15.33:0/532090010 wait complete. 2023-10-06T05:33:41.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:33:42.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.177+0000 7f552bfff700 1 Processor -- start 2023-10-06T05:33:42.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.177+0000 7f552bfff700 1 -- start start 2023-10-06T05:33:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.178+0000 7f552bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.178+0000 7f552bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f552c0cd5e0 0x7f552c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552c15d660 con 0x7f552c15a650 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552bfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552c0ce890 con 0x7f552c0cd5e0 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c15aac0 con 0x7f552c0af540 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f552c0cd5e0 0x7f552c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552a7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f552c15a650 0x7f552c0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43918/0 (socket says 172.21.15.33:43918) 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552a7fc700 1 -- 172.21.15.33:0/3527453311 learned_addr learned my addr 172.21.15.33:0/3527453311 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552affd700 1 --2- 172.21.15.33:0/3527453311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f5529ffb700 1 -- 172.21.15.33:0/3527453311 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842908428 0 0) 0x7f552c15d660 con 0x7f552c15a650 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f5529ffb700 1 -- 172.21.15.33:0/3527453311 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5514003570 con 0x7f552c15a650 2023-10-06T05:33:42.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552b7fe700 1 -- 172.21.15.33:0/3527453311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 msgr2=0x7f552c0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552b7fe700 1 --2- 172.21.15.33:0/3527453311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552b7fe700 1 -- 172.21.15.33:0/3527453311 >> v1:172.21.15.33:6789/0 conn(0x7f552c15a650 legacy=0x7f552c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.179+0000 7f552b7fe700 1 -- 172.21.15.33:0/3527453311 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c15d890 con 0x7f552c0cd5e0 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.180+0000 7f552b7fe700 1 --2- 172.21.15.33:0/3527453311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15d120 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f5524009e60 tx=0x7f5524009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bcba66c7f1f344 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.180+0000 7f5529ffb700 1 -- 172.21.15.33:0/3527453311 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552400e070 con 0x7f552c0cd5e0 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.180+0000 7f5529ffb700 1 -- 172.21.15.33:0/3527453311 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552400bab0 con 0x7f552c0cd5e0 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.180+0000 7f5529ffb700 1 -- 172.21.15.33:0/3527453311 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552400be30 con 0x7f552c0cd5e0 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.180+0000 7f552bfff700 1 -- 172.21.15.33:0/3527453311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 msgr2=0x7f552c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 --2- 172.21.15.33:0/3527453311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15d120 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f5524009e60 tx=0x7f5524009780 comp rx=0 tx=0).stop 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 -- 172.21.15.33:0/3527453311 shutdown_connections 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 --2- 172.21.15.33:0/3527453311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15d120 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 --2- 172.21.15.33:0/3527453311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 -- 172.21.15.33:0/3527453311 >> 172.21.15.33:0/3527453311 conn(0x7f552c06d9b0 msgr2=0x7f552c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 -- 172.21.15.33:0/3527453311 shutdown_connections 2023-10-06T05:33:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.181+0000 7f552bfff700 1 -- 172.21.15.33:0/3527453311 wait complete. 2023-10-06T05:33:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.182+0000 7f552bfff700 1 Processor -- start 2023-10-06T05:33:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.182+0000 7f552bfff700 1 -- start start 2023-10-06T05:33:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.183+0000 7f552bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.183+0000 7f552bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.183+0000 7f552bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552c15a650 0x7f552c15e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.183+0000 7f552bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0b4690 con 0x7f552c0cd5e0 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.183+0000 7f552bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0b4930 con 0x7f552c15a650 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.183+0000 7f552bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0c8cc0 con 0x7f552c0af540 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56372/0 (socket says 172.21.15.33:56372) 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 -- 172.21.15.33:0/2833123172 learned_addr learned my addr 172.21.15.33:0/2833123172 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552affd700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552b7fe700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552c15a650 0x7f552c15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 -- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 msgr2=0x7f552c156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 -- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552c15a650 msgr2=0x7f552c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552c15a650 0x7f552c15e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.184+0000 7f552a7fc700 1 -- 172.21.15.33:0/2833123172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524009a20 con 0x7f552c0cd5e0 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.185+0000 7f552a7fc700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15e1c0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f552000c790 tx=0x7f5520007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.185+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520005b30 con 0x7f552c0cd5e0 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.185+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520005c90 con 0x7f552c0cd5e0 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.185+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c0c8420 con 0x7f552c0cd5e0 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.185+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520007320 con 0x7f552c0cd5e0 2023-10-06T05:33:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.185+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c0c89e0 con 0x7f552c0cd5e0 2023-10-06T05:33:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.186+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c0b4ac0 con 0x7f552c0cd5e0 2023-10-06T05:33:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.187+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5520007690 con 0x7f552c0cd5e0 2023-10-06T05:33:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.188+0000 7f5512ffd700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5518066f30 0x7f55180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.188+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f5520083bd0 con 0x7f552c0cd5e0 2023-10-06T05:33:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.188+0000 7f552affd700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5518066f30 0x7f55180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.189+0000 7f552affd700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5518066f30 0x7f55180693e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f551c009d10 tx=0x7f551c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.192+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5520050bc0 con 0x7f552c0cd5e0 2023-10-06T05:33:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.395+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f552c04c380 con 0x7f552c0cd5e0 2023-10-06T05:33:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.396+0000 7f5512ffd700 1 -- 172.21.15.33:0/2833123172 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v254) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5520017020 con 0x7f552c0cd5e0 2023-10-06T05:33:42.399 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:33:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5518066f30 msgr2=0x7f55180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5518066f30 0x7f55180693e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f551c009d10 tx=0x7f551c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 msgr2=0x7f552c15e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15e1c0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f552000c790 tx=0x7f5520007df0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 shutdown_connections 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5518066f30 0x7f55180693e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552c15a650 0x7f552c15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552c0af540 0x7f552c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 --2- 172.21.15.33:0/2833123172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552c0cd5e0 0x7f552c15e1c0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.398+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 >> 172.21.15.33:0/2833123172 conn(0x7f552c06d9b0 msgr2=0x7f552c15ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.399+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 shutdown_connections 2023-10-06T05:33:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.399+0000 7f552bfff700 1 -- 172.21.15.33:0/2833123172 wait complete. 2023-10-06T05:33:42.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:33:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.606+0000 7f223979d700 1 Processor -- start 2023-10-06T05:33:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.607+0000 7f223979d700 1 -- start start 2023-10-06T05:33:42.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.608+0000 7f223979d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f22341552f0 0x7f22340ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.608+0000 7f223979d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22341586a0 con 0x7f22340b18f0 2023-10-06T05:33:42.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.608+0000 7f223979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2234157ab0 con 0x7f2234154f20 2023-10-06T05:33:42.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.608+0000 7f223979d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223406bdf0 con 0x7f22341552f0 2023-10-06T05:33:42.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.608+0000 7f2238f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f22340b18f0 0x7f22340c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43926/0 (socket says 172.21.15.33:43926) 2023-10-06T05:33:42.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.608+0000 7f2238f9c700 1 -- 172.21.15.33:0/3247481806 learned_addr learned my addr 172.21.15.33:0/3247481806 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f22337fe700 1 --2- 172.21.15.33:0/3247481806 >> v2:172.21.15.167:3300/0 conn(0x7f22341552f0 0x7f22340ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224897512 0 0) 0x7f22341586a0 con 0x7f22340b18f0 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f221c003570 con 0x7f22340b18f0 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 535887692 0 0) 0x7f221c003570 con 0x7f22340b18f0 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 >> v2:172.21.15.167:3300/0 conn(0x7f22341552f0 msgr2=0x7f22340ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 --2- 172.21.15.33:0/3247481806 >> v2:172.21.15.167:3300/0 conn(0x7f22341552f0 0x7f22340ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 >> v1:172.21.15.167:6789/0 conn(0x7f2234154f20 legacy=0x7f22340b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234157d70 con 0x7f22340b18f0 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f222c0017f0 con 0x7f22340b18f0 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 >> v1:172.21.15.33:6789/0 conn(0x7f22340b18f0 legacy=0x7f22340c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f221c003a60 0x7f221c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f221c004810 0x7f221c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f221c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223406bdf0 con 0x7f221c0051c0 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c00c130 con 0x7f221c004810 2023-10-06T05:33:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.609+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c00c2f0 con 0x7f221c003a60 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2233fff700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f221c003a60 0x7f221c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f221c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f22337fe700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f221c004810 0x7f221c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 -- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f221c003a60 msgr2=0x7f221c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f221c003a60 0x7f221c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 -- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f221c004810 msgr2=0x7f221c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f221c004810 0x7f221c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 -- 172.21.15.33:0/3247481806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c003570 con 0x7f221c0051c0 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 -- 172.21.15.33:0/3247481806 reap_dead start 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2238f9c700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f221c00ba60 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f222c00a850 tx=0x7f222c002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ddee04faf726ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.610+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00a210 con 0x7f221c0051c0 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0070f0 con 0x7f221c0051c0 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f2232ffd700 1 -- 172.21.15.33:0/3247481806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c007490 con 0x7f221c0051c0 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 -- 172.21.15.33:0/3247481806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 msgr2=0x7f221c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f221c00ba60 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f222c00a850 tx=0x7f222c002120 comp rx=0 tx=0).stop 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 -- 172.21.15.33:0/3247481806 shutdown_connections 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 --2- 172.21.15.33:0/3247481806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f221c00ba60 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 -- 172.21.15.33:0/3247481806 >> 172.21.15.33:0/3247481806 conn(0x7f223406d030 msgr2=0x7f22340cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 -- 172.21.15.33:0/3247481806 shutdown_connections 2023-10-06T05:33:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.611+0000 7f223979d700 1 -- 172.21.15.33:0/3247481806 wait complete. 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.613+0000 7f223979d700 1 Processor -- start 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.613+0000 7f223979d700 1 -- start start 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f223979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f2234152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f223979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22341570f0 0x7f2234157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f223979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234167350 0x7f22341677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f223979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234167d30 con 0x7f221c0051c0 2023-10-06T05:33:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f223979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341560e0 con 0x7f2234167350 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f223979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2234156330 con 0x7f22341570f0 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f2234152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f2234152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56396/0 (socket says 172.21.15.33:56396) 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 -- 172.21.15.33:0/3793881800 learned_addr learned my addr 172.21.15.33:0/3793881800 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2238f9c700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234167350 0x7f22341677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f22337fe700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22341570f0 0x7f2234157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 -- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22341570f0 msgr2=0x7f2234157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22341570f0 0x7f2234157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 -- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234167350 msgr2=0x7f22341677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234167350 0x7f22341677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.614+0000 7f2233fff700 1 -- 172.21.15.33:0/3793881800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234156610 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.615+0000 7f2233fff700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f2234152070 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f2224002ab0 tx=0x7f2224007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.615+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224003470 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.615+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222400f9a0 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.615+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22341568f0 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.615+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400fd60 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.615+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341525b0 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.616+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2234156e10 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.617+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2224013600 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.618+0000 7f221bfff700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.618+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2224084dd0 con 0x7f221c0051c0 2023-10-06T05:33:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.618+0000 7f22337fe700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.619+0000 7f22337fe700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2220066f30 0x7f22200693e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f22340cb990 tx=0x7f2228008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.622+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2234156e10 con 0x7f221c0051c0 2023-10-06T05:33:42.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.825+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2234156e10 con 0x7f221c0051c0 2023-10-06T05:33:42.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.825+0000 7f221bfff700 1 -- 172.21.15.33:0/3793881800 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v254) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2224017020 con 0x7f221c0051c0 2023-10-06T05:33:42.828 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:33:42.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.827+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2220066f30 msgr2=0x7f22200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.827+0000 7f223979d700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2220066f30 0x7f22200693e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f22340cb990 tx=0x7f2228008040 comp rx=0 tx=0).stop 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.827+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 msgr2=0x7f2234152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.827+0000 7f223979d700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f2234152070 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f2224002ab0 tx=0x7f2224007d10 comp rx=0 tx=0).stop 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 shutdown_connections 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2234167350 0x7f22341677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f221c0051c0 0x7f2234152070 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 --2- 172.21.15.33:0/3793881800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22341570f0 0x7f2234157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 >> 172.21.15.33:0/3793881800 conn(0x7f223406d030 msgr2=0x7f2234153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:42.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 shutdown_connections 2023-10-06T05:33:42.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:42.828+0000 7f223979d700 1 -- 172.21.15.33:0/3793881800 wait complete. 2023-10-06T05:33:42.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:33:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.035+0000 7fb926d9d700 1 Processor -- start 2023-10-06T05:33:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.036+0000 7fb926d9d700 1 -- start start 2023-10-06T05:33:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb926d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb920155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb926d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb920159740 0x7fb920159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb926d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9200b06b0 con 0x7fb9200c9490 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb926d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9200ccef0 con 0x7fb920159740 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb926d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200c99e0 con 0x7fb920155a50 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb92659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb920159740 0x7fb920159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb925d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb9200c9490 0x7fb9200cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43942/0 (socket says 172.21.15.33:43942) 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb925d9b700 1 -- 172.21.15.33:0/44360134 learned_addr learned my addr 172.21.15.33:0/44360134 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.037+0000 7fb92559a700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb920155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb92659c700 1 -- 172.21.15.33:0/44360134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 msgr2=0x7fb920155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb92659c700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb920155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb92659c700 1 -- 172.21.15.33:0/44360134 >> v1:172.21.15.33:6789/0 conn(0x7fb9200c9490 legacy=0x7fb9200cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb92659c700 1 -- 172.21.15.33:0/44360134 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200c9cc0 con 0x7fb920159740 2023-10-06T05:33:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb92559a700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb920155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb924d99700 1 -- 172.21.15.33:0/44360134 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727546409 0 0) 0x7fb9200b06b0 con 0x7fb9200c9490 2023-10-06T05:33:43.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb92659c700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb920159740 0x7fb920159ba0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fb918009e60 tx=0x7fb918009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faa05d7a824a12a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb924d99700 1 -- 172.21.15.33:0/44360134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91800e070 con 0x7fb920159740 2023-10-06T05:33:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb924d99700 1 -- 172.21.15.33:0/44360134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91800bc30 con 0x7fb920159740 2023-10-06T05:33:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.038+0000 7fb924d99700 1 -- 172.21.15.33:0/44360134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918012430 con 0x7fb920159740 2023-10-06T05:33:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 -- 172.21.15.33:0/44360134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb920159740 msgr2=0x7fb920159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb920159740 0x7fb920159ba0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fb918009e60 tx=0x7fb918009780 comp rx=0 tx=0).stop 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 -- 172.21.15.33:0/44360134 shutdown_connections 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb920159740 0x7fb920159ba0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 --2- 172.21.15.33:0/44360134 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb920155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 -- 172.21.15.33:0/44360134 >> 172.21.15.33:0/44360134 conn(0x7fb92006d030 msgr2=0x7fb9200cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 -- 172.21.15.33:0/44360134 shutdown_connections 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.039+0000 7fb926d9d700 1 -- 172.21.15.33:0/44360134 wait complete. 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.040+0000 7fb926d9d700 1 Processor -- start 2023-10-06T05:33:43.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 -- start start 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 0x7fb92015c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb9200c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb920159740 0x7fb92014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92015bf60 con 0x7fb9200c9490 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92015c8d0 con 0x7fb920159740 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.041+0000 7fb926d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200c82d0 con 0x7fb920155a50 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 0x7fb92015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 0x7fb92015c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56412/0 (socket says 172.21.15.33:56412) 2023-10-06T05:33:43.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 -- 172.21.15.33:0/1739575395 learned_addr learned my addr 172.21.15.33:0/1739575395 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 -- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 msgr2=0x7fb9200c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb9200c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 -- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb920159740 msgr2=0x7fb92014ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb920159740 0x7fb92014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 -- 172.21.15.33:0/1739575395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009a20 con 0x7fb9200c9490 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb925d9b700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 0x7fb92015c280 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c810 tx=0x7fb91c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.042+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c005b30 con 0x7fb9200c9490 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.043+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c005c90 con 0x7fb9200c9490 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.043+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c007300 con 0x7fb9200c9490 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.043+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92015b630 con 0x7fb9200c9490 2023-10-06T05:33:43.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.043+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92015bbf0 con 0x7fb9200c9490 2023-10-06T05:33:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.044+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92004c380 con 0x7fb9200c9490 2023-10-06T05:33:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.044+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb91c01b400 con 0x7fb9200c9490 2023-10-06T05:33:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.045+0000 7fb915ffb700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb904066f30 0x7fb9040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.045+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb91c0842c0 con 0x7fb9200c9490 2023-10-06T05:33:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.046+0000 7fb92559a700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb904066f30 0x7fb9040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.046+0000 7fb92559a700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb904066f30 0x7fb9040693e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb910006fd0 tx=0x7fb910008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.049+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb91c051330 con 0x7fb9200c9490 2023-10-06T05:33:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.252+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9200c9fa0 con 0x7fb9200c9490 2023-10-06T05:33:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.252+0000 7fb915ffb700 1 -- 172.21.15.33:0/1739575395 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v254) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb91c054950 con 0x7fb9200c9490 2023-10-06T05:33:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:33:43.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.254+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb904066f30 msgr2=0x7fb9040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.254+0000 7fb926d9d700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb904066f30 0x7fb9040693e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb910006fd0 tx=0x7fb910008040 comp rx=0 tx=0).stop 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.254+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 msgr2=0x7fb92015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.254+0000 7fb926d9d700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 0x7fb92015c280 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c810 tx=0x7fb91c007df0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 shutdown_connections 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb904066f30 0x7fb9040693e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb920159740 0x7fb92014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9200c9490 0x7fb92015c280 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 --2- 172.21.15.33:0/1739575395 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb920155a50 0x7fb9200c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 >> 172.21.15.33:0/1739575395 conn(0x7fb92006d030 msgr2=0x7fb9201512d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 shutdown_connections 2023-10-06T05:33:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.255+0000 7fb926d9d700 1 -- 172.21.15.33:0/1739575395 wait complete. 2023-10-06T05:33:43.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:33:43.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.458+0000 7fa3d259d700 1 Processor -- start 2023-10-06T05:33:43.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.459+0000 7fa3d259d700 1 -- start start 2023-10-06T05:33:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d259d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa3cc0b0a40 0x7fa3cc06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d259d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa3cc0b0e10 0x7fa3cc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d259d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc15e2d0 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d259d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc15d6c0 con 0x7fa3cc0b0e10 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15d9a0 con 0x7fa3cc0b2e30 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d159b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa3cc0b0a40 0x7fa3cc06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d159b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa3cc0b0a40 0x7fa3cc06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56424/0 (socket says 172.21.15.33:56424) 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d159b700 1 -- 172.21.15.33:0/428433512 learned_addr learned my addr 172.21.15.33:0/428433512 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d0d9a700 1 --2- 172.21.15.33:0/428433512 >> v2:172.21.15.167:3300/0 conn(0x7fa3cc0b0e10 0x7fa3cc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.460+0000 7fa3d1d9c700 1 --2- 172.21.15.33:0/428433512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d159b700 1 -- 172.21.15.33:0/428433512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 msgr2=0x7fa3cc15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d159b700 1 --2- 172.21.15.33:0/428433512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d159b700 1 -- 172.21.15.33:0/428433512 >> v2:172.21.15.167:3300/0 conn(0x7fa3cc0b0e10 msgr2=0x7fa3cc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d159b700 1 --2- 172.21.15.33:0/428433512 >> v2:172.21.15.167:3300/0 conn(0x7fa3cc0b0e10 0x7fa3cc0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d159b700 1 -- 172.21.15.33:0/428433512 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc15dc80 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d0d9a700 1 --2- 172.21.15.33:0/428433512 >> v2:172.21.15.167:3300/0 conn(0x7fa3cc0b0e10 0x7fa3cc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d159b700 1 --2- 172.21.15.33:0/428433512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc06ab00 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc065020 tx=0x7fa3c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa26c87f87d1ef14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3c3fff700 1 -- 172.21.15.33:0/428433512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800b860 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3c3fff700 1 -- 172.21.15.33:0/428433512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800b9c0 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3c3fff700 1 -- 172.21.15.33:0/428433512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800bd60 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.461+0000 7fa3d259d700 1 -- 172.21.15.33:0/428433512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 msgr2=0x7fa3cc06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 --2- 172.21.15.33:0/428433512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc06ab00 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc065020 tx=0x7fa3c8009250 comp rx=0 tx=0).stop 2023-10-06T05:33:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 -- 172.21.15.33:0/428433512 shutdown_connections 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 --2- 172.21.15.33:0/428433512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 --2- 172.21.15.33:0/428433512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc06ab00 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 --2- 172.21.15.33:0/428433512 >> v2:172.21.15.167:3300/0 conn(0x7fa3cc0b0e10 0x7fa3cc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 -- 172.21.15.33:0/428433512 >> 172.21.15.33:0/428433512 conn(0x7fa3cc06e0f0 msgr2=0x7fa3cc0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 -- 172.21.15.33:0/428433512 shutdown_connections 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.462+0000 7fa3d259d700 1 -- 172.21.15.33:0/428433512 wait complete. 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.463+0000 7fa3d259d700 1 Processor -- start 2023-10-06T05:33:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.463+0000 7fa3d259d700 1 -- start start 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.464+0000 7fa3d259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc16ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.464+0000 7fa3d259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3cc0b0e10 0x7fa3cc16a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.464+0000 7fa3d259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc0c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.464+0000 7fa3d259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0c4510 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.464+0000 7fa3d259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15b620 con 0x7fa3cc0b0e10 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.464+0000 7fa3d259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15b900 con 0x7fa3cc0b2e30 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc16ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc16ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56440/0 (socket says 172.21.15.33:56440) 2023-10-06T05:33:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 -- 172.21.15.33:0/1207425028 learned_addr learned my addr 172.21.15.33:0/1207425028 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d1d9c700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc0c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d0d9a700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3cc0b0e10 0x7fa3cc16a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 -- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 msgr2=0x7fa3cc0c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc0c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 -- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3cc0b0e10 msgr2=0x7fa3cc16a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3cc0b0e10 0x7fa3cc16a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 -- 172.21.15.33:0/1207425028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80094f0 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.465+0000 7fa3d159b700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc16ce30 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009290 tx=0x7fa3c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.466+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011e40 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.466+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c801a450 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.466+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c801a810 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.466+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc15bbe0 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.466+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc0c2d20 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.467+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc06fcc0 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.468+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3c801abc0 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.468+0000 7fa3c17fa700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3b0066f30 0x7fa3b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.468+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa3c8085d90 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.469+0000 7fa3d0d9a700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3b0066f30 0x7fa3b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.470+0000 7fa3d0d9a700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3b0066f30 0x7fa3b00693e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc006f00 tx=0x7fa3bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.473+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3c8052d80 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.718+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3cc15c100 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.719+0000 7fa3c17fa700 1 -- 172.21.15.33:0/1207425028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fa3c8015090 con 0x7fa3cc0b0a40 2023-10-06T05:33:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:43.722 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3b0066f30 msgr2=0x7fa3b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3b0066f30 0x7fa3b00693e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc006f00 tx=0x7fa3bc008040 comp rx=0 tx=0).stop 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 msgr2=0x7fa3cc16ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc16ce30 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009290 tx=0x7fa3c8017000 comp rx=0 tx=0).stop 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 shutdown_connections 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa3b0066f30 0x7fa3b00693e0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3cc0b2e30 0x7fa3cc0c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3cc0b0a40 0x7fa3cc16ce30 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 --2- 172.21.15.33:0/1207425028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3cc0b0e10 0x7fa3cc16a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 >> 172.21.15.33:0/1207425028 conn(0x7fa3cc06e0f0 msgr2=0x7fa3cc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 shutdown_connections 2023-10-06T05:33:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.721+0000 7fa3d259d700 1 -- 172.21.15.33:0/1207425028 wait complete. 2023-10-06T05:33:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:33:43.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:33:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.925+0000 7f41475ff700 1 Processor -- start 2023-10-06T05:33:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.925+0000 7f41475ff700 1 -- start start 2023-10-06T05:33:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.926+0000 7f41475ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f414015ca80 0x7f414006e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.926+0000 7f41475ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ce50 0x7f414015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.926+0000 7f41475ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41400ab250 con 0x7f414015d910 2023-10-06T05:33:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.926+0000 7f41475ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414006e660 con 0x7f414015ca80 2023-10-06T05:33:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.926+0000 7f41475ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414006e940 con 0x7f414015ce50 2023-10-06T05:33:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f414539b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f414015ca80 0x7f414006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f4145b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f414015d910 0x7f41400aefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43958/0 (socket says 172.21.15.33:43958) 2023-10-06T05:33:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f4145b9c700 1 -- 172.21.15.33:0/879991252 learned_addr learned my addr 172.21.15.33:0/879991252 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f4144b9a700 1 --2- 172.21.15.33:0/879991252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ce50 0x7f414015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f4137fff700 1 -- 172.21.15.33:0/879991252 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766556008 0 0) 0x7f41400ab250 con 0x7f414015d910 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f414539b700 1 -- 172.21.15.33:0/879991252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ce50 msgr2=0x7f414015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f414539b700 1 --2- 172.21.15.33:0/879991252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ce50 0x7f414015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f414539b700 1 -- 172.21.15.33:0/879991252 >> v1:172.21.15.33:6789/0 conn(0x7f414015d910 legacy=0x7f41400aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f414539b700 1 -- 172.21.15.33:0/879991252 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414006ec20 con 0x7f414015ca80 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.927+0000 7f414539b700 1 --2- 172.21.15.33:0/879991252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ca80 0x7f414006e050 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f413c0098e0 tx=0x7f413c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb694affedd4719c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f4137fff700 1 -- 172.21.15.33:0/879991252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c009e80 con 0x7f414015ca80 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f4137fff700 1 -- 172.21.15.33:0/879991252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c00b860 con 0x7f414015ca80 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f4137fff700 1 -- 172.21.15.33:0/879991252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00bbe0 con 0x7f414015ca80 2023-10-06T05:33:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 -- 172.21.15.33:0/879991252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ca80 msgr2=0x7f414006e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 --2- 172.21.15.33:0/879991252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ca80 0x7f414006e050 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f413c0098e0 tx=0x7f413c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 -- 172.21.15.33:0/879991252 shutdown_connections 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 --2- 172.21.15.33:0/879991252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ca80 0x7f414006e050 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 --2- 172.21.15.33:0/879991252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ce50 0x7f414015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 -- 172.21.15.33:0/879991252 >> 172.21.15.33:0/879991252 conn(0x7f414006cd40 msgr2=0x7f4140087fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 -- 172.21.15.33:0/879991252 shutdown_connections 2023-10-06T05:33:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.928+0000 7f41475ff700 1 -- 172.21.15.33:0/879991252 wait complete. 2023-10-06T05:33:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.930+0000 7f41475ff700 1 Processor -- start 2023-10-06T05:33:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.930+0000 7f41475ff700 1 -- start start 2023-10-06T05:33:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f41475ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ca80 0x7f414009bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f41475ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 0x7f414016a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f41475ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f414015d910 0x7f41400cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f41475ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c9200 con 0x7f414015ce50 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f41475ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400cc3c0 con 0x7f414015d910 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f41475ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414009c260 con 0x7f414015ca80 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 0x7f414016a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 0x7f414016a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56456/0 (socket says 172.21.15.33:56456) 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 -- 172.21.15.33:0/4262883964 learned_addr learned my addr 172.21.15.33:0/4262883964 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f414539b700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ca80 0x7f414009bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4145b9c700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f414015d910 0x7f41400cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 -- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ca80 msgr2=0x7f414009bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ca80 0x7f414009bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 -- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f414015d910 msgr2=0x7f41400cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f414015d910 0x7f41400cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4144b9a700 1 -- 172.21.15.33:0/4262883964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c009420 con 0x7f414015ce50 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.931+0000 7f4145b9c700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f414015d910 0x7f41400cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.932+0000 7f4144b9a700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 0x7f414016a410 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f413000d860 tx=0x7f4130008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.932+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130018070 con 0x7f414015ce50 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.932+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41300125f0 con 0x7f414015ce50 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.932+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414016a950 con 0x7f414015ce50 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.932+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300129b0 con 0x7f414015ce50 2023-10-06T05:33:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.932+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41400cbd70 con 0x7f414015ce50 2023-10-06T05:33:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.933+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400cc550 con 0x7f414015ce50 2023-10-06T05:33:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.934+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4130012d20 con 0x7f414015ce50 2023-10-06T05:33:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.935+0000 7f41357fa700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4124066f30 0x7f41240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.935+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f4130084a80 con 0x7f414015ce50 2023-10-06T05:33:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.936+0000 7f414539b700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4124066f30 0x7f41240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.936+0000 7f414539b700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4124066f30 0x7f41240693e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f413c0091e0 tx=0x7f413c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:43.939+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4130051a70 con 0x7f414015ce50 2023-10-06T05:33:44.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.145+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4140063ff0 con 0x7f414015ce50 2023-10-06T05:33:44.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.146+0000 7f41357fa700 1 -- 172.21.15.33:0/4262883964 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f4130055090 con 0x7f414015ce50 2023-10-06T05:33:44.150 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:44.150 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":254,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:38.046427+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:32:21.814695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":38,"pool_name":"cephfs_data","create_time":"2023-10-06T05:32:22.631382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"38.1","mappings":[{"from":7,"to":4}]},{"pgid":"38.17","mappings":[{"from":7,"to":2}]},{"pgid":"38.1c","mappings":[{"from":7,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1348218675":"2023-10-06T06:33:37.316586+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:33:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4124066f30 msgr2=0x7f41240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4124066f30 0x7f41240693e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f413c0091e0 tx=0x7f413c017000 comp rx=0 tx=0).stop 2023-10-06T05:33:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 msgr2=0x7f414016a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 0x7f414016a410 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f413000d860 tx=0x7f4130008040 comp rx=0 tx=0).stop 2023-10-06T05:33:44.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 shutdown_connections 2023-10-06T05:33:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4124066f30 0x7f41240693e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f414015d910 0x7f41400cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f414015ca80 0x7f414009bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 --2- 172.21.15.33:0/4262883964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414015ce50 0x7f414016a410 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 >> 172.21.15.33:0/4262883964 conn(0x7f414006cd40 msgr2=0x7f41400b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 shutdown_connections 2023-10-06T05:33:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.149+0000 7f41475ff700 1 -- 172.21.15.33:0/4262883964 wait complete. 2023-10-06T05:33:44.170 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:33:44.170 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:33:44.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.357+0000 7ff3db59e700 1 Processor -- start 2023-10-06T05:33:44.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.357+0000 7ff3db59e700 1 -- start start 2023-10-06T05:33:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.358+0000 7ff3db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc155af0 0x7ff3dc06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.358+0000 7ff3db59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff3dc06a6a0 0x7ff3dc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3db59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3dc0c8b50 0x7ff3dc0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3db59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc06b7e0 con 0x7ff3dc0c8b50 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3db59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc06ab60 con 0x7ff3dc06a6a0 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc06ae40 con 0x7ff3dc155af0 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3dc0c8b50 0x7ff3dc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3dc0c8b50 0x7ff3dc0c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56462/0 (socket says 172.21.15.33:56462) 2023-10-06T05:33:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 -- 172.21.15.33:0/862610675 learned_addr learned my addr 172.21.15.33:0/862610675 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3da59c700 1 --2- 172.21.15.33:0/862610675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc155af0 0x7ff3dc06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3d9d9b700 1 --2- 172.21.15.33:0/862610675 >> v2:172.21.15.167:3300/0 conn(0x7ff3dc06a6a0 0x7ff3dc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 -- 172.21.15.33:0/862610675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc155af0 msgr2=0x7ff3dc06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 --2- 172.21.15.33:0/862610675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc155af0 0x7ff3dc06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 -- 172.21.15.33:0/862610675 >> v2:172.21.15.167:3300/0 conn(0x7ff3dc06a6a0 msgr2=0x7ff3dc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 --2- 172.21.15.33:0/862610675 >> v2:172.21.15.167:3300/0 conn(0x7ff3dc06a6a0 0x7ff3dc0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3dad9d700 1 -- 172.21.15.33:0/862610675 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc06b120 con 0x7ff3dc0c8b50 2023-10-06T05:33:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.359+0000 7ff3d9d9b700 1 --2- 172.21.15.33:0/862610675 >> v2:172.21.15.167:3300/0 conn(0x7ff3dc06a6a0 0x7ff3dc0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:33:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.360+0000 7ff3dad9d700 1 --2- 172.21.15.33:0/862610675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc0c8b50 0x7ff3dc0c7750 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0690f0 tx=0x7ff3d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1207143cd5a5321 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.360+0000 7ff3d959a700 1 -- 172.21.15.33:0/862610675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400e070 con 0x7ff3dc0c8b50 2023-10-06T05:33:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.360+0000 7ff3d959a700 1 -- 172.21.15.33:0/862610675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d400ba50 con 0x7ff3dc0c8b50 2023-10-06T05:33:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.360+0000 7ff3d959a700 1 -- 172.21.15.33:0/862610675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40092e0 con 0x7ff3dc0c8b50 2023-10-06T05:33:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 -- 172.21.15.33:0/862610675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc0c8b50 msgr2=0x7ff3dc0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 --2- 172.21.15.33:0/862610675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc0c8b50 0x7ff3dc0c7750 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0690f0 tx=0x7ff3d4009840 comp rx=0 tx=0).stop 2023-10-06T05:33:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 -- 172.21.15.33:0/862610675 shutdown_connections 2023-10-06T05:33:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 --2- 172.21.15.33:0/862610675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc0c8b50 0x7ff3dc0c7750 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 --2- 172.21.15.33:0/862610675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc155af0 0x7ff3dc06a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 --2- 172.21.15.33:0/862610675 >> v2:172.21.15.167:3300/0 conn(0x7ff3dc06a6a0 0x7ff3dc0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 -- 172.21.15.33:0/862610675 >> 172.21.15.33:0/862610675 conn(0x7ff3dc06d030 msgr2=0x7ff3dc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 -- 172.21.15.33:0/862610675 shutdown_connections 2023-10-06T05:33:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.361+0000 7ff3db59e700 1 -- 172.21.15.33:0/862610675 wait complete. 2023-10-06T05:33:44.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.362+0000 7ff3db59e700 1 Processor -- start 2023-10-06T05:33:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.363+0000 7ff3db59e700 1 -- start start 2023-10-06T05:33:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.363+0000 7ff3db59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3dc06a6a0 0x7ff3dc151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.363+0000 7ff3db59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc0c8b50 0x7ff3dc157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3db59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 0x7ff3dc0ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3db59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0abda0 con 0x7ff3dc155af0 2023-10-06T05:33:44.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3db59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0ac640 con 0x7ff3dc06a6a0 2023-10-06T05:33:44.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3db59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc168670 con 0x7ff3dc0c8b50 2023-10-06T05:33:44.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 0x7ff3dc0ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 0x7ff3dc0ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56472/0 (socket says 172.21.15.33:56472) 2023-10-06T05:33:44.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 -- 172.21.15.33:0/551140456 learned_addr learned my addr 172.21.15.33:0/551140456 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:44.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3d9d9b700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc0c8b50 0x7ff3dc157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3da59c700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3dc06a6a0 0x7ff3dc151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 -- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc0c8b50 msgr2=0x7ff3dc157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc0c8b50 0x7ff3dc157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 -- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3dc06a6a0 msgr2=0x7ff3dc151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3dc06a6a0 0x7ff3dc151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.364+0000 7ff3dad9d700 1 -- 172.21.15.33:0/551140456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009ae0 con 0x7ff3dc155af0 2023-10-06T05:33:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.365+0000 7ff3dad9d700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 0x7ff3dc0ae120 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4012000 tx=0x7ff3d4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.365+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40032f0 con 0x7ff3dc155af0 2023-10-06T05:33:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.365+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4008d20 con 0x7ff3dc155af0 2023-10-06T05:33:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.365+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc0abff0 con 0x7ff3dc155af0 2023-10-06T05:33:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.365+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40266b0 con 0x7ff3dc155af0 2023-10-06T05:33:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.365+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc152280 con 0x7ff3dc155af0 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.366+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc04c380 con 0x7ff3dc155af0 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.367+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3d4003040 con 0x7ff3dc155af0 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.368+0000 7ff3ca7fc700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3bc066f30 0x7ff3bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.368+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff3d4086620 con 0x7ff3dc155af0 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.369+0000 7ff3da59c700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3bc066f30 0x7ff3bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.369+0000 7ff3da59c700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3bc066f30 0x7ff3bc0693e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0c2000 tx=0x7ff3cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.372+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3d4053610 con 0x7ff3dc155af0 2023-10-06T05:33:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.618+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3dc06b400 con 0x7ff3dc155af0 2023-10-06T05:33:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.619+0000 7ff3ca7fc700 1 -- 172.21.15.33:0/551140456 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7ff3d4009940 con 0x7ff3dc155af0 2023-10-06T05:33:44.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:44.622 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:33:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3bc066f30 msgr2=0x7ff3bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3bc066f30 0x7ff3bc0693e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0c2000 tx=0x7ff3cc008040 comp rx=0 tx=0).stop 2023-10-06T05:33:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 msgr2=0x7ff3dc0ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 0x7ff3dc0ae120 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4012000 tx=0x7ff3d4018000 comp rx=0 tx=0).stop 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 shutdown_connections 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff3bc066f30 0x7ff3bc0693e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3dc155af0 0x7ff3dc0ae120 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3dc06a6a0 0x7ff3dc151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.621+0000 7ff3db59e700 1 --2- 172.21.15.33:0/551140456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3dc0c8b50 0x7ff3dc157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.622+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 >> 172.21.15.33:0/551140456 conn(0x7ff3dc06d030 msgr2=0x7ff3dc06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.622+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 shutdown_connections 2023-10-06T05:33:44.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.622+0000 7ff3db59e700 1 -- 172.21.15.33:0/551140456 wait complete. 2023-10-06T05:33:44.626 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:33:44.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:33:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.826+0000 7f7a4ab9d700 1 Processor -- start 2023-10-06T05:33:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.826+0000 7f7a4ab9d700 1 -- start start 2023-10-06T05:33:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.827+0000 7f7a4ab9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a440ccfc0 0x7f7a440cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.827+0000 7f7a4ab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440cf3b0 0x7f7a441594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.827+0000 7f7a4ab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a440cc750 0x7f7a441599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.827+0000 7f7a4ab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440cbd80 con 0x7f7a440cc750 2023-10-06T05:33:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.827+0000 7f7a4ab9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440af5f0 con 0x7f7a440ccfc0 2023-10-06T05:33:44.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.827+0000 7f7a4ab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440af8d0 con 0x7f7a440cf3b0 2023-10-06T05:33:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a440cc750 0x7f7a441599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a440cc750 0x7f7a441599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56478/0 (socket says 172.21.15.33:56478) 2023-10-06T05:33:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 -- 172.21.15.33:0/2741609680 learned_addr learned my addr 172.21.15.33:0/2741609680 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a49b9b700 1 --2- 172.21.15.33:0/2741609680 >> v2:172.21.15.167:3300/0 conn(0x7f7a440ccfc0 0x7f7a440cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4939a700 1 --2- 172.21.15.33:0/2741609680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440cf3b0 0x7f7a441594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 -- 172.21.15.33:0/2741609680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440cf3b0 msgr2=0x7f7a441594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 --2- 172.21.15.33:0/2741609680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440cf3b0 0x7f7a441594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 -- 172.21.15.33:0/2741609680 >> v2:172.21.15.167:3300/0 conn(0x7f7a440ccfc0 msgr2=0x7f7a440cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 --2- 172.21.15.33:0/2741609680 >> v2:172.21.15.167:3300/0 conn(0x7f7a440ccfc0 0x7f7a440cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 -- 172.21.15.33:0/2741609680 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a440afbb0 con 0x7f7a440cc750 2023-10-06T05:33:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.828+0000 7f7a4a39c700 1 --2- 172.21.15.33:0/2741609680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a441599e0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f7a4406d790 tx=0x7f7a3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae24a1e8157344e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a48b99700 1 -- 172.21.15.33:0/2741609680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00e070 con 0x7f7a440cc750 2023-10-06T05:33:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a48b99700 1 -- 172.21.15.33:0/2741609680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00b970 con 0x7f7a440cc750 2023-10-06T05:33:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a48b99700 1 -- 172.21.15.33:0/2741609680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0092e0 con 0x7f7a440cc750 2023-10-06T05:33:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/2741609680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 msgr2=0x7f7a441599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/2741609680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a441599e0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f7a4406d790 tx=0x7f7a3c009840 comp rx=0 tx=0).stop 2023-10-06T05:33:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/2741609680 shutdown_connections 2023-10-06T05:33:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/2741609680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a441599e0 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/2741609680 >> v2:172.21.15.167:3300/0 conn(0x7f7a440ccfc0 0x7f7a440cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/2741609680 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440cf3b0 0x7f7a441594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/2741609680 >> 172.21.15.33:0/2741609680 conn(0x7f7a4406cff0 msgr2=0x7f7a4406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.829+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/2741609680 shutdown_connections 2023-10-06T05:33:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.830+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/2741609680 wait complete. 2023-10-06T05:33:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.831+0000 7f7a4ab9d700 1 Processor -- start 2023-10-06T05:33:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.831+0000 7f7a4ab9d700 1 -- start start 2023-10-06T05:33:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a4ab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a440b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a4ab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440ccfc0 0x7f7a44153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a4ab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a440c8f60 0x7f7a441558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a4ab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4415f040 con 0x7f7a440cc750 2023-10-06T05:33:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a4ab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440b41d0 con 0x7f7a440c8f60 2023-10-06T05:33:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a4ab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a44169a10 con 0x7f7a440ccfc0 2023-10-06T05:33:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a49b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a440b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a49b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a440b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56484/0 (socket says 172.21.15.33:56484) 2023-10-06T05:33:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.832+0000 7f7a49b9b700 1 -- 172.21.15.33:0/452102884 learned_addr learned my addr 172.21.15.33:0/452102884 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a49b9b700 1 -- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440ccfc0 msgr2=0x7f7a44153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a49b9b700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440ccfc0 0x7f7a44153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a49b9b700 1 -- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a440c8f60 msgr2=0x7f7a441558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a4a39c700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a440c8f60 0x7f7a441558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a49b9b700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a440c8f60 0x7f7a441558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a49b9b700 1 -- 172.21.15.33:0/452102884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009ae0 con 0x7f7a440cc750 2023-10-06T05:33:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a4a39c700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a440c8f60 0x7f7a441558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:33:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a49b9b700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a440b32a0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ec80 tx=0x7f7a400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000c9d0 con 0x7f7a440cc750 2023-10-06T05:33:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000cb30 con 0x7f7a440cc750 2023-10-06T05:33:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40013470 con 0x7f7a440cc750 2023-10-06T05:33:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a44169c80 con 0x7f7a440cc750 2023-10-06T05:33:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.833+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a440b3930 con 0x7f7a440cc750 2023-10-06T05:33:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.834+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4404c380 con 0x7f7a440cc750 2023-10-06T05:33:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.835+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a40013820 con 0x7f7a440cc750 2023-10-06T05:33:44.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.836+0000 7f7a39ffb700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a28066f30 0x7f7a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:44.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.836+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f7a40084b40 con 0x7f7a440cc750 2023-10-06T05:33:44.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.837+0000 7f7a4939a700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a28066f30 0x7f7a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:44.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.837+0000 7f7a4939a700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a28066f30 0x7f7a280693e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7a440cf1a0 tx=0x7f7a340091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:44.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:44.840+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a40051b30 con 0x7f7a440cc750 2023-10-06T05:33:45.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.088+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a4406bfe0 con 0x7f7a440cc750 2023-10-06T05:33:45.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.088+0000 7f7a39ffb700 1 -- 172.21.15.33:0/452102884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f7a40055150 con 0x7f7a440cc750 2023-10-06T05:33:45.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:45.091 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":195,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:32:24.200367+0000","modified":"2023-10-06T05:32:28.195313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"a","rank":0,"incarnation":191,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3696795298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3696795298},{"type":"v1","addr":"172.21.15.33:6835","nonce":3696795298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":18}]} 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a28066f30 msgr2=0x7f7a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a28066f30 0x7f7a280693e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7a440cf1a0 tx=0x7f7a340091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 msgr2=0x7f7a440b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a440b32a0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ec80 tx=0x7f7a400091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 shutdown_connections 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7a28066f30 0x7f7a280693e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a440c8f60 0x7f7a441558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a440cc750 0x7f7a440b32a0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 --2- 172.21.15.33:0/452102884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a440ccfc0 0x7f7a44153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 >> 172.21.15.33:0/452102884 conn(0x7f7a4406cff0 msgr2=0x7f7a440c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 shutdown_connections 2023-10-06T05:33:45.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.091+0000 7f7a4ab9d700 1 -- 172.21.15.33:0/452102884 wait complete. 2023-10-06T05:33:45.096 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 195 2023-10-06T05:33:45.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:33:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.295+0000 7f9fd679d700 1 Processor -- start 2023-10-06T05:33:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.295+0000 7f9fd679d700 1 -- start start 2023-10-06T05:33:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.296+0000 7f9fd679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fd0158700 0x7f9fd00adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.296+0000 7f9fd679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd0158ba0 con 0x7f9fd00699c0 2023-10-06T05:33:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.296+0000 7f9fd679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd00ae490 con 0x7f9fd0155140 2023-10-06T05:33:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.296+0000 7f9fd679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd00c8e40 con 0x7f9fd0158700 2023-10-06T05:33:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd4f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9fd0158700 0x7f9fd00adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd5f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9fd00699c0 0x7f9fd00c6720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43974/0 (socket says 172.21.15.33:43974) 2023-10-06T05:33:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd5f9c700 1 -- 172.21.15.33:0/2898714494 learned_addr learned my addr 172.21.15.33:0/2898714494 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:45.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fc7fff700 1 -- 172.21.15.33:0/2898714494 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156967137 0 0) 0x7f9fd0158ba0 con 0x7f9fd00699c0 2023-10-06T05:33:45.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fc7fff700 1 -- 172.21.15.33:0/2898714494 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fbc003570 con 0x7f9fd00699c0 2023-10-06T05:33:45.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd4f9a700 1 -- 172.21.15.33:0/2898714494 >> v1:172.21.15.167:6789/0 conn(0x7f9fd0155140 legacy=0x7f9fd00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd4f9a700 1 -- 172.21.15.33:0/2898714494 >> v1:172.21.15.33:6789/0 conn(0x7f9fd00699c0 legacy=0x7f9fd00c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd4f9a700 1 -- 172.21.15.33:0/2898714494 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00c99e0 con 0x7f9fd0158700 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.297+0000 7f9fd4f9a700 1 --2- 172.21.15.33:0/2898714494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd0158700 0x7f9fd00adf50 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009910 tx=0x7f9fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5e01dca422159d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fc7fff700 1 -- 172.21.15.33:0/2898714494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000b640 con 0x7f9fd0158700 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fc7fff700 1 -- 172.21.15.33:0/2898714494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000b7a0 con 0x7f9fd0158700 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fc7fff700 1 -- 172.21.15.33:0/2898714494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000bb60 con 0x7f9fd0158700 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 -- 172.21.15.33:0/2898714494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd0158700 msgr2=0x7f9fd00adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 --2- 172.21.15.33:0/2898714494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd0158700 0x7f9fd00adf50 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009910 tx=0x7f9fc00091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 -- 172.21.15.33:0/2898714494 shutdown_connections 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 --2- 172.21.15.33:0/2898714494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd0158700 0x7f9fd00adf50 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 -- 172.21.15.33:0/2898714494 >> 172.21.15.33:0/2898714494 conn(0x7f9fd006d030 msgr2=0x7f9fd006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 -- 172.21.15.33:0/2898714494 shutdown_connections 2023-10-06T05:33:45.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.298+0000 7f9fd679d700 1 -- 172.21.15.33:0/2898714494 wait complete. 2023-10-06T05:33:45.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.300+0000 7f9fd679d700 1 Processor -- start 2023-10-06T05:33:45.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.300+0000 7f9fd679d700 1 -- start start 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 0x7f9fd0153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fd0155140 0x7f9fd00c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd0158700 0x7f9fd00c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd01558c0 con 0x7f9fd00699c0 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0155b10 con 0x7f9fd0158700 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0155df0 con 0x7f9fd0155140 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 0x7f9fd0153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd579b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 0x7f9fd0153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56508/0 (socket says 172.21.15.33:56508) 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.301+0000 7f9fd579b700 1 -- 172.21.15.33:0/216031606 learned_addr learned my addr 172.21.15.33:0/216031606 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd579b700 1 -- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fd0155140 msgr2=0x7f9fd00c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd579b700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fd0155140 0x7f9fd00c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd579b700 1 -- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd0158700 msgr2=0x7f9fd00c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd579b700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd0158700 0x7f9fd00c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd579b700 1 -- 172.21.15.33:0/216031606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0009420 con 0x7f9fd00699c0 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd579b700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 0x7f9fd0153000 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c590 tx=0x7f9fcc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc011590 con 0x7f9fd00699c0 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0116f0 con 0x7f9fd00699c0 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc011ab0 con 0x7f9fd00699c0 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd01560d0 con 0x7f9fd00699c0 2023-10-06T05:33:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.302+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd0156690 con 0x7f9fd00699c0 2023-10-06T05:33:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.303+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd004c380 con 0x7f9fd00699c0 2023-10-06T05:33:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.304+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fcc01b3b0 con 0x7f9fd00699c0 2023-10-06T05:33:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.305+0000 7f9fc57fa700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.305+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9fcc083d10 con 0x7f9fd00699c0 2023-10-06T05:33:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.306+0000 7f9fd4f9a700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.306+0000 7f9fd4f9a700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0159320 tx=0x7f9fc0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:45.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.309+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fcc089020 con 0x7f9fd00699c0 2023-10-06T05:33:45.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.513+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd01537f0 con 0x7f9fd00699c0 2023-10-06T05:33:45.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.514+0000 7f9fc57fa700 1 -- 172.21.15.33:0/216031606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f9fcc017020 con 0x7f9fd00699c0 2023-10-06T05:33:45.517 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:45.517 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":254,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:38.046427+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:32:21.814695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":38,"pool_name":"cephfs_data","create_time":"2023-10-06T05:32:22.631382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"38.1","mappings":[{"from":7,"to":4}]},{"pgid":"38.17","mappings":[{"from":7,"to":2}]},{"pgid":"38.1c","mappings":[{"from":7,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1348218675":"2023-10-06T06:33:37.316586+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:33:45.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb4066f30 msgr2=0x7f9fb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0159320 tx=0x7f9fc0017000 comp rx=0 tx=0).stop 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 msgr2=0x7f9fd0153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 0x7f9fd0153000 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c590 tx=0x7f9fcc00c950 comp rx=0 tx=0).stop 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 shutdown_connections 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd0158700 0x7f9fd00c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd00699c0 0x7f9fd0153000 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 --2- 172.21.15.33:0/216031606 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fd0155140 0x7f9fd00c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 >> 172.21.15.33:0/216031606 conn(0x7f9fd006d030 msgr2=0x7f9fd00cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.516+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 shutdown_connections 2023-10-06T05:33:45.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.517+0000 7f9fd679d700 1 -- 172.21.15.33:0/216031606 wait complete. 2023-10-06T05:33:45.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:33:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.721+0000 7faa5679d700 1 Processor -- start 2023-10-06T05:33:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.722+0000 7faa5679d700 1 -- start start 2023-10-06T05:33:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa5679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50156880 0x7faa500c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa5679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faa500c7ec0 0x7faa50157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa5679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa500c8320 con 0x7faa50159e10 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa5679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa500cc290 con 0x7faa500c7ec0 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa5679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5006bd10 con 0x7faa50156880 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa55f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faa500c7ec0 0x7faa50157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa55f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faa500c7ec0 0x7faa50157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56514/0 (socket says 172.21.15.33:56514) 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa55f9c700 1 -- 172.21.15.33:0/1375685466 learned_addr learned my addr 172.21.15.33:0/1375685466 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.723+0000 7faa5579b700 1 --2- 172.21.15.33:0/1375685466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50156880 0x7faa500c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa55f9c700 1 -- 172.21.15.33:0/1375685466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50156880 msgr2=0x7faa500c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa55f9c700 1 --2- 172.21.15.33:0/1375685466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50156880 0x7faa500c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa55f9c700 1 -- 172.21.15.33:0/1375685466 >> v1:172.21.15.167:6789/0 conn(0x7faa50159e10 legacy=0x7faa50156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa55f9c700 1 -- 172.21.15.33:0/1375685466 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5006a820 con 0x7faa500c7ec0 2023-10-06T05:33:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa55f9c700 1 --2- 172.21.15.33:0/1375685466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa500c7ec0 0x7faa50157360 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7faa48009f20 tx=0x7faa48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee550eab6b1bc6ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa47fff700 1 -- 172.21.15.33:0/1375685466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800e070 con 0x7faa500c7ec0 2023-10-06T05:33:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa47fff700 1 -- 172.21.15.33:0/1375685466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4800ba80 con 0x7faa500c7ec0 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.724+0000 7faa47fff700 1 -- 172.21.15.33:0/1375685466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480092e0 con 0x7faa500c7ec0 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 -- 172.21.15.33:0/1375685466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa500c7ec0 msgr2=0x7faa50157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 --2- 172.21.15.33:0/1375685466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa500c7ec0 0x7faa50157360 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7faa48009f20 tx=0x7faa48009840 comp rx=0 tx=0).stop 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 -- 172.21.15.33:0/1375685466 shutdown_connections 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 --2- 172.21.15.33:0/1375685466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa500c7ec0 0x7faa50157360 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 --2- 172.21.15.33:0/1375685466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa50156880 0x7faa500c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 -- 172.21.15.33:0/1375685466 >> 172.21.15.33:0/1375685466 conn(0x7faa5006c560 msgr2=0x7faa5006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 -- 172.21.15.33:0/1375685466 shutdown_connections 2023-10-06T05:33:45.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.725+0000 7faa5679d700 1 -- 172.21.15.33:0/1375685466 wait complete. 2023-10-06T05:33:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.726+0000 7faa5679d700 1 Processor -- start 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.727+0000 7faa5679d700 1 -- start start 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.727+0000 7faa5679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500c7ec0 0x7faa5014cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.727+0000 7faa5679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 0x7faa500b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.727+0000 7faa5679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50159e10 0x7faa50146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa5679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5014bb00 con 0x7faa50156880 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa5679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500b0d40 con 0x7faa50159e10 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa5679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa500b0f90 con 0x7faa500c7ec0 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 0x7faa500b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 0x7faa500b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56530/0 (socket says 172.21.15.33:56530) 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 -- 172.21.15.33:0/948918482 learned_addr learned my addr 172.21.15.33:0/948918482 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa5579b700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500c7ec0 0x7faa5014cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa55f9c700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50159e10 0x7faa50146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 -- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500c7ec0 msgr2=0x7faa5014cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500c7ec0 0x7faa5014cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 -- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50159e10 msgr2=0x7faa50146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50159e10 0x7faa50146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 -- 172.21.15.33:0/948918482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48009ae0 con 0x7faa50156880 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.728+0000 7faa54f9a700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 0x7faa500b0370 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7faa4000c590 tx=0x7faa40007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.729+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40011590 con 0x7faa50156880 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.729+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa40011710 con 0x7faa50156880 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.729+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40011ad0 con 0x7faa50156880 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.729+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa500b1270 con 0x7faa50156880 2023-10-06T05:33:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.729+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa500b1830 con 0x7faa50156880 2023-10-06T05:33:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.730+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5004c380 con 0x7faa50156880 2023-10-06T05:33:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.731+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa4001a360 con 0x7faa50156880 2023-10-06T05:33:45.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.731+0000 7faa457fa700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa34066f30 0x7faa340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.732+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6188+0+0 (secure 0 0 0) 0x7faa400835e0 con 0x7faa50156880 2023-10-06T05:33:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.732+0000 7faa5579b700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa34066f30 0x7faa340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.733+0000 7faa5579b700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa34066f30 0x7faa340693e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.736+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa400505d0 con 0x7faa50156880 2023-10-06T05:33:45.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.982+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faa500c8320 con 0x7faa50156880 2023-10-06T05:33:45.985 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:33:45.983+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:33:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.987+0000 7faa457fa700 1 -- 172.21.15.33:0/948918482 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v196) v1 ==== 161+0+0 (secure 0 0 0) 0x7faa500c8320 con 0x7faa50156880 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa34066f30 msgr2=0x7faa340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa34066f30 0x7faa340693e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c008040 comp rx=0 tx=0).stop 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 msgr2=0x7faa500b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 0x7faa500b0370 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7faa4000c590 tx=0x7faa40007f50 comp rx=0 tx=0).stop 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 shutdown_connections 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7faa34066f30 0x7faa340693e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50159e10 0x7faa50146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500c7ec0 0x7faa5014cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 --2- 172.21.15.33:0/948918482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50156880 0x7faa500b0370 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 >> 172.21.15.33:0/948918482 conn(0x7faa5006c560 msgr2=0x7faa501481a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:45.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 shutdown_connections 2023-10-06T05:33:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:45.991+0000 7faa5679d700 1 -- 172.21.15.33:0/948918482 wait complete. 2023-10-06T05:33:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:33:46.003 INFO:tasks.ceph.mds.a.smithi033.stderr: -4826> 2023-10-06T05:32:28.134+0000 7fd5961f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:33:46.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:33:46.056 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:33:46.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.209+0000 7f570ffff700 1 Processor -- start 2023-10-06T05:33:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.209+0000 7f570ffff700 1 -- start start 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.210+0000 7f570ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57100cd5e0 0x7f57100af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.210+0000 7f570ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.210+0000 7f570ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571015d660 con 0x7f57100af540 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.210+0000 7f570ffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100ce890 con 0x7f57100cd5e0 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.210+0000 7f570ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571015aab0 con 0x7f571015a650 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.210+0000 7f570effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f57100af540 0x7f57100cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43980/0 (socket says 172.21.15.33:43980) 2023-10-06T05:33:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f570effd700 1 -- 172.21.15.33:0/791360599 learned_addr learned my addr 172.21.15.33:0/791360599 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f5707fff700 1 --2- 172.21.15.33:0/791360599 >> v2:172.21.15.33:3300/0 conn(0x7f57100cd5e0 0x7f57100af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f570f7fe700 1 --2- 172.21.15.33:0/791360599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f570e7fc700 1 -- 172.21.15.33:0/791360599 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804049338 0 0) 0x7f571015d660 con 0x7f57100af540 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f570e7fc700 1 -- 172.21.15.33:0/791360599 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56fc003570 con 0x7f57100af540 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f5707fff700 1 -- 172.21.15.33:0/791360599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 msgr2=0x7f571015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f5707fff700 1 --2- 172.21.15.33:0/791360599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f570e7fc700 1 -- 172.21.15.33:0/791360599 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3864966029 0 0) 0x7f56fc003570 con 0x7f57100af540 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f5707fff700 1 -- 172.21.15.33:0/791360599 >> v1:172.21.15.33:6789/0 conn(0x7f57100af540 legacy=0x7f57100cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.211+0000 7f5707fff700 1 -- 172.21.15.33:0/791360599 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571015d890 con 0x7f57100cd5e0 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.212+0000 7f5707fff700 1 --2- 172.21.15.33:0/791360599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100af910 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098e0 tx=0x7f56f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a303f8872755a06d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.212+0000 7f570e7fc700 1 -- 172.21.15.33:0/791360599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700001470 con 0x7f57100cd5e0 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.212+0000 7f570e7fc700 1 -- 172.21.15.33:0/791360599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8011420 con 0x7f57100cd5e0 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.212+0000 7f570e7fc700 1 -- 172.21.15.33:0/791360599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80117c0 con 0x7f57100cd5e0 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.212+0000 7f570ffff700 1 -- 172.21.15.33:0/791360599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 msgr2=0x7f57100af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.212+0000 7f570ffff700 1 --2- 172.21.15.33:0/791360599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100af910 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098e0 tx=0x7f56f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.213+0000 7f570ffff700 1 -- 172.21.15.33:0/791360599 shutdown_connections 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.213+0000 7f570ffff700 1 --2- 172.21.15.33:0/791360599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.213+0000 7f570ffff700 1 --2- 172.21.15.33:0/791360599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100af910 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.213+0000 7f570ffff700 1 -- 172.21.15.33:0/791360599 >> 172.21.15.33:0/791360599 conn(0x7f571006d9b0 msgr2=0x7f571006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.213+0000 7f570ffff700 1 -- 172.21.15.33:0/791360599 shutdown_connections 2023-10-06T05:33:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.213+0000 7f570ffff700 1 -- 172.21.15.33:0/791360599 wait complete. 2023-10-06T05:33:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.214+0000 7f570ffff700 1 Processor -- start 2023-10-06T05:33:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.214+0000 7f570ffff700 1 -- start start 2023-10-06T05:33:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f570ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57100af540 0x7f57100c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f570ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f570ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f570ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b3f20 con 0x7f57100cd5e0 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f570ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b3310 con 0x7f57100af540 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f570ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b35f0 con 0x7f571015a650 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f5707fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f5707fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56566/0 (socket says 172.21.15.33:56566) 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.215+0000 7f5707fff700 1 -- 172.21.15.33:0/3916572668 learned_addr learned my addr 172.21.15.33:0/3916572668 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5707fff700 1 -- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 msgr2=0x7f571014bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5707fff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5707fff700 1 -- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57100af540 msgr2=0x7f57100c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5707fff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57100af540 0x7f57100c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5707fff700 1 -- 172.21.15.33:0/3916572668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009420 con 0x7f57100cd5e0 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5707fff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100ca2d0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f56f80030c0 tx=0x7f56f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f801cc90 con 0x7f57100cd5e0 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f801cdf0 con 0x7f57100cd5e0 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57100b38d0 con 0x7f57100cd5e0 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8025680 con 0x7f57100cd5e0 2023-10-06T05:33:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.216+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57100ca810 con 0x7f57100cd5e0 2023-10-06T05:33:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.217+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571004c380 con 0x7f57100cd5e0 2023-10-06T05:33:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.219+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56f80193f0 con 0x7f57100cd5e0 2023-10-06T05:33:46.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.220+0000 7f5706ffd700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56f0066f60 0x7f56f0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:46.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.220+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f56f8085b60 con 0x7f57100cd5e0 2023-10-06T05:33:46.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.221+0000 7f570effd700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56f0066f60 0x7f56f0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:46.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.221+0000 7f570effd700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56f0066f60 0x7f56f0069410 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f57000011e0 tx=0x7f5700001320 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:46.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.224+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56f8052b70 con 0x7f57100cd5e0 2023-10-06T05:33:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.471+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57100b4060 con 0x7f57100cd5e0 2023-10-06T05:33:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.989+0000 7f5706ffd700 1 -- 172.21.15.33:0/3916572668 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v197) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56f8056190 con 0x7f57100cd5e0 2023-10-06T05:33:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56f0066f60 msgr2=0x7f56f0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56f0066f60 0x7f56f0069410 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f57000011e0 tx=0x7f5700001320 comp rx=0 tx=0).stop 2023-10-06T05:33:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 msgr2=0x7f57100ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100ca2d0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f56f80030c0 tx=0x7f56f8016000 comp rx=0 tx=0).stop 2023-10-06T05:33:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 shutdown_connections 2023-10-06T05:33:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f56f0066f60 0x7f56f0069410 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571015a650 0x7f571014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57100af540 0x7f57100c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 --2- 172.21.15.33:0/3916572668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57100cd5e0 0x7f57100ca2d0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:46.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 >> 172.21.15.33:0/3916572668 conn(0x7f571006d9b0 msgr2=0x7f57100c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:46.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 shutdown_connections 2023-10-06T05:33:46.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:46.992+0000 7f570ffff700 1 -- 172.21.15.33:0/3916572668 wait complete. 2023-10-06T05:33:47.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:33:47.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.246+0000 7f01cd5ff700 1 Processor -- start 2023-10-06T05:33:47.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.246+0000 7f01cd5ff700 1 -- start start 2023-10-06T05:33:47.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01cd5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8159370 0x7f01c80cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:47.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01cd5ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01c80ca990 0x7f01c80c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:47.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01cd5ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c80c6e30 con 0x7f01c8158fa0 2023-10-06T05:33:47.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01cd5ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c806a170 con 0x7f01c80ca990 2023-10-06T05:33:47.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01cd5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80c82e0 con 0x7f01c8159370 2023-10-06T05:33:47.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01c6ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01c8158fa0 0x7f01c80c7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43990/0 (socket says 172.21.15.33:43990) 2023-10-06T05:33:47.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01c6ffd700 1 -- 172.21.15.33:0/2146199554 learned_addr learned my addr 172.21.15.33:0/2146199554 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:47.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.247+0000 7f01c77fe700 1 --2- 172.21.15.33:0/2146199554 >> v2:172.21.15.33:3300/0 conn(0x7f01c80ca990 0x7f01c80c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:47.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c67fc700 1 --2- 172.21.15.33:0/2146199554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8159370 0x7f01c80cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:47.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c5ffb700 1 -- 172.21.15.33:0/2146199554 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881253658 0 0) 0x7f01c80c6e30 con 0x7f01c8158fa0 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c5ffb700 1 -- 172.21.15.33:0/2146199554 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b0003570 con 0x7f01c8158fa0 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c77fe700 1 -- 172.21.15.33:0/2146199554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8159370 msgr2=0x7f01c80cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c77fe700 1 --2- 172.21.15.33:0/2146199554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8159370 0x7f01c80cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c77fe700 1 -- 172.21.15.33:0/2146199554 >> v1:172.21.15.33:6789/0 conn(0x7f01c8158fa0 legacy=0x7f01c80c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c77fe700 1 -- 172.21.15.33:0/2146199554 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80cb100 con 0x7f01c80ca990 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c77fe700 1 --2- 172.21.15.33:0/2146199554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80ca990 0x7f01c80c8cb0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009e60 tx=0x7f01c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cc3bac0d95a22d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c5ffb700 1 -- 172.21.15.33:0/2146199554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c000e070 con 0x7f01c80ca990 2023-10-06T05:33:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c5ffb700 1 -- 172.21.15.33:0/2146199554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c000bb60 con 0x7f01c80ca990 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.248+0000 7f01c5ffb700 1 -- 172.21.15.33:0/2146199554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0012430 con 0x7f01c80ca990 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 -- 172.21.15.33:0/2146199554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80ca990 msgr2=0x7f01c80c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/2146199554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80ca990 0x7f01c80c8cb0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009e60 tx=0x7f01c0009780 comp rx=0 tx=0).stop 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 -- 172.21.15.33:0/2146199554 shutdown_connections 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/2146199554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c80ca990 0x7f01c80c8cb0 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/2146199554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8159370 0x7f01c80cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 -- 172.21.15.33:0/2146199554 >> 172.21.15.33:0/2146199554 conn(0x7f01c806e630 msgr2=0x7f01c806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 -- 172.21.15.33:0/2146199554 shutdown_connections 2023-10-06T05:33:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.249+0000 7f01cd5ff700 1 -- 172.21.15.33:0/2146199554 wait complete. 2023-10-06T05:33:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.251+0000 7f01cd5ff700 1 Processor -- start 2023-10-06T05:33:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.251+0000 7f01cd5ff700 1 -- start start 2023-10-06T05:33:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01cd5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01c80ca990 0x7f01c8157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01cd5ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8158fa0 0x7f01c80c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01cd5ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 0x7f01c80c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01cd5ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80c39a0 con 0x7f01c8159370 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01cd5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80c47e0 con 0x7f01c80ca990 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01cd5ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80c3bd0 con 0x7f01c8158fa0 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 0x7f01c80c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 0x7f01c80c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56588/0 (socket says 172.21.15.33:56588) 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 -- 172.21.15.33:0/393736990 learned_addr learned my addr 172.21.15.33:0/393736990 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c6ffd700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01c80ca990 0x7f01c8157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c67fc700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8158fa0 0x7f01c80c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:47.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 -- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8158fa0 msgr2=0x7f01c80c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8158fa0 0x7f01c80c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 -- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01c80ca990 msgr2=0x7f01c8157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01c80ca990 0x7f01c8157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c77fe700 1 -- 172.21.15.33:0/393736990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c0009a20 con 0x7f01c8159370 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.252+0000 7f01c6ffd700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01c80ca990 0x7f01c8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.253+0000 7f01c77fe700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 0x7f01c80c2520 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f01c80c32a0 tx=0x7f01c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.253+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0027470 con 0x7f01c8159370 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.253+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c00275f0 con 0x7f01c8159370 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.253+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c80c3eb0 con 0x7f01c8159370 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.253+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c00279b0 con 0x7f01c8159370 2023-10-06T05:33:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.253+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c80c4470 con 0x7f01c8159370 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.254+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c80718b0 con 0x7f01c8159370 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.255+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01c0024070 con 0x7f01c8159370 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.256+0000 7f01aeffd700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01b4066f30 0x7f01b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.256+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f01c0087240 con 0x7f01c8159370 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.256+0000 7f01c6ffd700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01b4066f30 0x7f01b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.257+0000 7f01c6ffd700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01b4066f30 0x7f01b40693e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f01b80058d0 tx=0x7f01b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:47.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.260+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01c00541d0 con 0x7f01c8159370 2023-10-06T05:33:47.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:47.461+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01c80c70f0 con 0x7f01c8159370 2023-10-06T05:33:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.093+0000 7f01aeffd700 1 -- 172.21.15.33:0/393736990 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v256) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01c00577f0 con 0x7f01c8159370 2023-10-06T05:33:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.096+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01b4066f30 msgr2=0x7f01b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.096+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01b4066f30 0x7f01b40693e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f01b80058d0 tx=0x7f01b8009040 comp rx=0 tx=0).stop 2023-10-06T05:33:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.096+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 msgr2=0x7f01c80c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.096+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 0x7f01c80c2520 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f01c80c32a0 tx=0x7f01c0019000 comp rx=0 tx=0).stop 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 shutdown_connections 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f01b4066f30 0x7f01b40693e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01c8159370 0x7f01c80c2520 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01c80ca990 0x7f01c8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 --2- 172.21.15.33:0/393736990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01c8158fa0 0x7f01c80c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 >> 172.21.15.33:0/393736990 conn(0x7f01c806e630 msgr2=0x7f01c80cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 shutdown_connections 2023-10-06T05:33:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.097+0000 7f01cd5ff700 1 -- 172.21.15.33:0/393736990 wait complete. 2023-10-06T05:33:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:33:48.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:33:48.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.319+0000 7f7bcf59e700 1 Processor -- start 2023-10-06T05:33:48.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.319+0000 7f7bcf59e700 1 -- start start 2023-10-06T05:33:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.320+0000 7f7bcf59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7bd00ad830 0x7f7bd0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.320+0000 7f7bcf59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7bd006ee20 0x7f7bd00ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:48.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.320+0000 7f7bcf59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd00ad3a0 con 0x7f7bd0157320 2023-10-06T05:33:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.320+0000 7f7bcf59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd00ac580 con 0x7f7bd006ee20 2023-10-06T05:33:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.320+0000 7f7bcf59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd00afcc0 con 0x7f7bd00ad830 2023-10-06T05:33:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7bd006ee20 0x7f7bd00ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7bd006ee20 0x7f7bd00ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56590/0 (socket says 172.21.15.33:56590) 2023-10-06T05:33:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 -- 172.21.15.33:0/4056840119 learned_addr learned my addr 172.21.15.33:0/4056840119 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:48.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bcdd9b700 1 --2- 172.21.15.33:0/4056840119 >> v2:172.21.15.167:3300/0 conn(0x7f7bd00ad830 0x7f7bd0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 -- 172.21.15.33:0/4056840119 >> v2:172.21.15.167:3300/0 conn(0x7f7bd00ad830 msgr2=0x7f7bd0159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:48.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 --2- 172.21.15.33:0/4056840119 >> v2:172.21.15.167:3300/0 conn(0x7f7bd00ad830 0x7f7bd0159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 -- 172.21.15.33:0/4056840119 >> v1:172.21.15.167:6789/0 conn(0x7f7bd0157320 legacy=0x7f7bd006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.321+0000 7f7bced9d700 1 -- 172.21.15.33:0/4056840119 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd01576f0 con 0x7f7bd006ee20 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bced9d700 1 --2- 172.21.15.33:0/4056840119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd00ac040 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00712d0 tx=0x7f7bc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbbd47576065a46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcd59a700 1 -- 172.21.15.33:0/4056840119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800e070 con 0x7f7bd006ee20 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcd59a700 1 -- 172.21.15.33:0/4056840119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800ba20 con 0x7f7bd006ee20 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcd59a700 1 -- 172.21.15.33:0/4056840119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80092e0 con 0x7f7bd006ee20 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcf59e700 1 -- 172.21.15.33:0/4056840119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 msgr2=0x7f7bd00ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/4056840119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd00ac040 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00712d0 tx=0x7f7bc8009840 comp rx=0 tx=0).stop 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcf59e700 1 -- 172.21.15.33:0/4056840119 shutdown_connections 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/4056840119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd00ac040 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/4056840119 >> v2:172.21.15.167:3300/0 conn(0x7f7bd00ad830 0x7f7bd0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.322+0000 7f7bcf59e700 1 -- 172.21.15.33:0/4056840119 >> 172.21.15.33:0/4056840119 conn(0x7f7bd006cd40 msgr2=0x7f7bd00ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.323+0000 7f7bcf59e700 1 -- 172.21.15.33:0/4056840119 shutdown_connections 2023-10-06T05:33:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.323+0000 7f7bcf59e700 1 -- 172.21.15.33:0/4056840119 wait complete. 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.324+0000 7f7bcf59e700 1 Processor -- start 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.324+0000 7f7bcf59e700 1 -- start start 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bcf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd0068840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bcf59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd00ad830 0x7f7bd015df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bcf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bd0157320 0x7f7bd015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bcf59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd00cbc00 con 0x7f7bd006ee20 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bcf59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015efe0 con 0x7f7bd00ad830 2023-10-06T05:33:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bcf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015ed00 con 0x7f7bd0157320 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd0068840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd0068840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56602/0 (socket says 172.21.15.33:56602) 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.325+0000 7f7bce59c700 1 -- 172.21.15.33:0/2194151775 learned_addr learned my addr 172.21.15.33:0/2194151775 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bcdd9b700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd00ad830 0x7f7bd015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bced9d700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bd0157320 0x7f7bd015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bce59c700 1 -- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bd0157320 msgr2=0x7f7bd015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bce59c700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bd0157320 0x7f7bd015e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bce59c700 1 -- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd00ad830 msgr2=0x7f7bd015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bce59c700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd00ad830 0x7f7bd015df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bce59c700 1 -- 172.21.15.33:0/2194151775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009ae0 con 0x7f7bd006ee20 2023-10-06T05:33:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bced9d700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bd0157320 0x7f7bd015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bce59c700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd0068840 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000c510 tx=0x7f7bc000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0011590 con 0x7f7bd006ee20 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc0011710 con 0x7f7bd006ee20 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0011ad0 con 0x7f7bd006ee20 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.326+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd00cb5b0 con 0x7f7bd006ee20 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.327+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd015ea10 con 0x7f7bd006ee20 2023-10-06T05:33:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.328+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd0063ff0 con 0x7f7bd006ee20 2023-10-06T05:33:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.328+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bc0001020 con 0x7f7bd006ee20 2023-10-06T05:33:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.329+0000 7f7bbe7fc700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.330+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5836+0+0 (secure 0 0 0) 0x7f7bc0083a00 con 0x7f7bd006ee20 2023-10-06T05:33:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.330+0000 7f7bcdd9b700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:48.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.331+0000 7f7bcdd9b700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bb0066f30 0x7f7bb00693e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009db0 tx=0x7f7bc40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.333+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc0050b50 con 0x7f7bd006ee20 2023-10-06T05:33:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:48.534+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7bd00ac790 con 0x7f7bd006ee20 2023-10-06T05:33:49.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.101+0000 7f7bbe7fc700 1 -- 172.21.15.33:0/2194151775 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v257) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7bc0017070 con 0x7f7bd006ee20 2023-10-06T05:33:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.106+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bb0066f30 msgr2=0x7f7bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.106+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bb0066f30 0x7f7bb00693e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009db0 tx=0x7f7bc40091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.106+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 msgr2=0x7f7bd0068840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.106+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd0068840 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000c510 tx=0x7f7bc000c820 comp rx=0 tx=0).stop 2023-10-06T05:33:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 shutdown_connections 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bb0066f30 0x7f7bb00693e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bd0157320 0x7f7bd015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bd006ee20 0x7f7bd0068840 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 --2- 172.21.15.33:0/2194151775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd00ad830 0x7f7bd015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 >> 172.21.15.33:0/2194151775 conn(0x7f7bd006cd40 msgr2=0x7f7bd0067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 shutdown_connections 2023-10-06T05:33:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.107+0000 7f7bcf59e700 1 -- 172.21.15.33:0/2194151775 wait complete. 2023-10-06T05:33:49.112 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:33:49.131 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:33:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.318+0000 7f695b19d700 1 Processor -- start 2023-10-06T05:33:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.318+0000 7f695b19d700 1 -- start start 2023-10-06T05:33:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6954154f20 0x7f69540ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69541552f0 0x7f69540cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695b19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69541586a0 con 0x7f69540b11e0 2023-10-06T05:33:49.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695406bdf0 con 0x7f6954154f20 2023-10-06T05:33:49.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6954157ab0 con 0x7f69541552f0 2023-10-06T05:33:49.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695a99c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f69540b11e0 0x7f695415b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:43994/0 (socket says 172.21.15.33:43994) 2023-10-06T05:33:49.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695a99c700 1 -- 172.21.15.33:0/2506937248 learned_addr learned my addr 172.21.15.33:0/2506937248 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:49.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695a19b700 1 --2- 172.21.15.33:0/2506937248 >> v2:172.21.15.33:3300/0 conn(0x7f6954154f20 0x7f69540ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f695999a700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69541552f0 0x7f69540cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.320+0000 7f6959199700 1 -- 172.21.15.33:0/2506937248 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031608136 0 0) 0x7f69541586a0 con 0x7f69540b11e0 2023-10-06T05:33:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f6959199700 1 -- 172.21.15.33:0/2506937248 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6940003570 con 0x7f69540b11e0 2023-10-06T05:33:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f695a19b700 1 -- 172.21.15.33:0/2506937248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69541552f0 msgr2=0x7f69540cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f695a19b700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69541552f0 0x7f69540cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f695a19b700 1 -- 172.21.15.33:0/2506937248 >> v1:172.21.15.33:6789/0 conn(0x7f69540b11e0 legacy=0x7f695415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f695a19b700 1 -- 172.21.15.33:0/2506937248 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954157d70 con 0x7f6954154f20 2023-10-06T05:33:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f695999a700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69541552f0 0x7f69540cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f695a19b700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540ae720 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f69500098e0 tx=0x7f69500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d73929a876fcccc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f6959199700 1 -- 172.21.15.33:0/2506937248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000b640 con 0x7f6954154f20 2023-10-06T05:33:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f6959199700 1 -- 172.21.15.33:0/2506937248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695000b7a0 con 0x7f6954154f20 2023-10-06T05:33:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.321+0000 7f6959199700 1 -- 172.21.15.33:0/2506937248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000bb40 con 0x7f6954154f20 2023-10-06T05:33:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 -- 172.21.15.33:0/2506937248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 msgr2=0x7f69540ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540ae720 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f69500098e0 tx=0x7f69500091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 -- 172.21.15.33:0/2506937248 shutdown_connections 2023-10-06T05:33:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540ae720 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 --2- 172.21.15.33:0/2506937248 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69541552f0 0x7f69540cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 -- 172.21.15.33:0/2506937248 >> 172.21.15.33:0/2506937248 conn(0x7f695406d030 msgr2=0x7f69540cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 -- 172.21.15.33:0/2506937248 shutdown_connections 2023-10-06T05:33:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.322+0000 7f695b19d700 1 -- 172.21.15.33:0/2506937248 wait complete. 2023-10-06T05:33:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.323+0000 7f695b19d700 1 Processor -- start 2023-10-06T05:33:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.323+0000 7f695b19d700 1 -- start start 2023-10-06T05:33:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.324+0000 7f695b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69540b11e0 0x7f69540c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.324+0000 7f695b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.324+0000 7f695b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69541694c0 0x7f6954169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.324+0000 7f695b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69540b0590 con 0x7f6954154f20 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.324+0000 7f695b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69540b0870 con 0x7f69541694c0 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.324+0000 7f695b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69540b0b50 con 0x7f69540b11e0 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56626/0 (socket says 172.21.15.33:56626) 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 -- 172.21.15.33:0/3669045964 learned_addr learned my addr 172.21.15.33:0/3669045964 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695a19b700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69540b11e0 0x7f69540c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695a99c700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69541694c0 0x7f6954169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 -- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69540b11e0 msgr2=0x7f69540c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69540b11e0 0x7f69540c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 -- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69541694c0 msgr2=0x7f6954169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69541694c0 0x7f6954169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 -- 172.21.15.33:0/3669045964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950009420 con 0x7f6954154f20 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.325+0000 7f695999a700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540afd60 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f694400e8d0 tx=0x7f6944008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.326+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694400ee70 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.326+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694400c5e0 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.326+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69540b0e30 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.326+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694400c980 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.326+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69540aa060 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.327+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695404c380 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.328+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6944013410 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.329+0000 7f694a7fc700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6938066f30 0x7f69380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.329+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6944084270 con 0x7f6954154f20 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.329+0000 7f695a19b700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6938066f30 0x7f69380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.330+0000 7f695a19b700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6938066f30 0x7f69380693e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f69500093f0 tx=0x7f6950017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:49.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.333+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69440515b0 con 0x7f6954154f20 2023-10-06T05:33:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.535+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f695406f370 con 0x7f6954154f20 2023-10-06T05:33:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.540+0000 7f694a7fc700 1 -- 172.21.15.33:0/3669045964 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6944054bd0 con 0x7f6954154f20 2023-10-06T05:33:49.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.542+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6938066f30 msgr2=0x7f69380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.542+0000 7f695b19d700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6938066f30 0x7f69380693e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f69500093f0 tx=0x7f6950017000 comp rx=0 tx=0).stop 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.542+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 msgr2=0x7f69540afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.542+0000 7f695b19d700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540afd60 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f694400e8d0 tx=0x7f6944008040 comp rx=0 tx=0).stop 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 shutdown_connections 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6938066f30 0x7f69380693e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69541694c0 0x7f6954169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69540b11e0 0x7f69540c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 --2- 172.21.15.33:0/3669045964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6954154f20 0x7f69540afd60 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 >> 172.21.15.33:0/3669045964 conn(0x7f695406d030 msgr2=0x7f69540c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 shutdown_connections 2023-10-06T05:33:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.543+0000 7f695b19d700 1 -- 172.21.15.33:0/3669045964 wait complete. 2023-10-06T05:33:49.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_write_caps' 2023-10-06T05:33:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.753+0000 7f72a279d700 1 Processor -- start 2023-10-06T05:33:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.753+0000 7f72a279d700 1 -- start start 2023-10-06T05:33:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.754+0000 7f72a279d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f729c13faa0 0x7f729c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.754+0000 7f72a279d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f729c0c7940 0x7f729c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.754+0000 7f72a279d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c0c9cb0 con 0x7f729c06aad0 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.754+0000 7f72a279d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c06b640 con 0x7f729c0c7940 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.754+0000 7f72a279d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c0ca270 con 0x7f729c13faa0 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a0f9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f729c0c7940 0x7f729c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a1f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f729c06aad0 0x7f729c06af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44004/0 (socket says 172.21.15.33:44004) 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a1f9c700 1 -- 172.21.15.33:0/1897838464 learned_addr learned my addr 172.21.15.33:0/1897838464 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a179b700 1 --2- 172.21.15.33:0/1897838464 >> v2:172.21.15.167:3300/0 conn(0x7f729c13faa0 0x7f729c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a0f9a700 1 -- 172.21.15.33:0/1897838464 >> v2:172.21.15.167:3300/0 conn(0x7f729c13faa0 msgr2=0x7f729c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a0f9a700 1 --2- 172.21.15.33:0/1897838464 >> v2:172.21.15.167:3300/0 conn(0x7f729c13faa0 0x7f729c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a0f9a700 1 -- 172.21.15.33:0/1897838464 >> v1:172.21.15.33:6789/0 conn(0x7f729c06aad0 legacy=0x7f729c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a0f9a700 1 -- 172.21.15.33:0/1897838464 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c06b840 con 0x7f729c0c7940 2023-10-06T05:33:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f7293fff700 1 -- 172.21.15.33:0/1897838464 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080479605 0 0) 0x7f729c0c9cb0 con 0x7f729c06aad0 2023-10-06T05:33:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.755+0000 7f72a0f9a700 1 --2- 172.21.15.33:0/1897838464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c0c7940 0x7f729c0c7db0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f729c06fe50 tx=0x7f728c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5ff6844ce4ca4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f7293fff700 1 -- 172.21.15.33:0/1897838464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00b6b0 con 0x7f729c0c7940 2023-10-06T05:33:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f7293fff700 1 -- 172.21.15.33:0/1897838464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c00b810 con 0x7f729c0c7940 2023-10-06T05:33:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f7293fff700 1 -- 172.21.15.33:0/1897838464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00bbb0 con 0x7f729c0c7940 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 -- 172.21.15.33:0/1897838464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c0c7940 msgr2=0x7f729c0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 --2- 172.21.15.33:0/1897838464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c0c7940 0x7f729c0c7db0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f729c06fe50 tx=0x7f728c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 -- 172.21.15.33:0/1897838464 shutdown_connections 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 --2- 172.21.15.33:0/1897838464 >> v2:172.21.15.167:3300/0 conn(0x7f729c13faa0 0x7f729c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 --2- 172.21.15.33:0/1897838464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c0c7940 0x7f729c0c7db0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 -- 172.21.15.33:0/1897838464 >> 172.21.15.33:0/1897838464 conn(0x7f729c06d030 msgr2=0x7f729c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 -- 172.21.15.33:0/1897838464 shutdown_connections 2023-10-06T05:33:49.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.756+0000 7f72a279d700 1 -- 172.21.15.33:0/1897838464 wait complete. 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.758+0000 7f72a279d700 1 Processor -- start 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.758+0000 7f72a279d700 1 -- start start 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a279d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 0x7f729c155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a279d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f729c0c7940 0x7f729c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a279d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f729c13faa0 0x7f729c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a279d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15ed10 con 0x7f729c06aad0 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a279d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15e100 con 0x7f729c13faa0 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a279d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15e3e0 con 0x7f729c0c7940 2023-10-06T05:33:49.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 0x7f729c155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 0x7f729c155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56638/0 (socket says 172.21.15.33:56638) 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 -- 172.21.15.33:0/3764943487 learned_addr learned my addr 172.21.15.33:0/3764943487 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 -- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f729c0c7940 msgr2=0x7f729c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f729c0c7940 0x7f729c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 -- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f729c13faa0 msgr2=0x7f729c158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f729c13faa0 0x7f729c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:49.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.759+0000 7f72a179b700 1 -- 172.21.15.33:0/3764943487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c009420 con 0x7f729c06aad0 2023-10-06T05:33:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.760+0000 7f72a179b700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 0x7f729c155d40 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f729800d860 tx=0x7f7298008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.760+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009610 con 0x7f729c06aad0 2023-10-06T05:33:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.760+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7298009770 con 0x7f729c06aad0 2023-10-06T05:33:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.760+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009b30 con 0x7f729c06aad0 2023-10-06T05:33:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.760+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c15e6c0 con 0x7f729c06aad0 2023-10-06T05:33:49.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.760+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c156280 con 0x7f729c06aad0 2023-10-06T05:33:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.761+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c04c380 con 0x7f729c06aad0 2023-10-06T05:33:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.762+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f729801e460 con 0x7f729c06aad0 2023-10-06T05:33:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.763+0000 7f72917fa700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7280066f30 0x7f72800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:49.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.763+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7298084a80 con 0x7f729c06aad0 2023-10-06T05:33:49.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.763+0000 7f72a0f9a700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7280066f30 0x7f72800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:49.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.764+0000 7f72a0f9a700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7280066f30 0x7f72800693e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f728c0091e0 tx=0x7f728c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:49.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:49.767+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7298051dc0 con 0x7f729c06aad0 2023-10-06T05:33:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.050+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_write_caps"]} v 0) v1 -- 0x7f729c06bb20 con 0x7f729c06aad0 2023-10-06T05:33:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.101+0000 7f72917fa700 1 -- 172.21.15.33:0/3764943487 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_stale_write_caps"]}]=0 v954) v1 ==== 153+0+0 (secure 0 0 0) 0x7f72980553e0 con 0x7f729c06aad0 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7280066f30 msgr2=0x7f72800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7280066f30 0x7f72800693e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f728c0091e0 tx=0x7f728c017000 comp rx=0 tx=0).stop 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 msgr2=0x7f729c155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 0x7f729c155d40 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f729800d860 tx=0x7f7298008040 comp rx=0 tx=0).stop 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 shutdown_connections 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7280066f30 0x7f72800693e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f729c13faa0 0x7f729c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f729c06aad0 0x7f729c155d40 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 --2- 172.21.15.33:0/3764943487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f729c0c7940 0x7f729c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 >> 172.21.15.33:0/3764943487 conn(0x7f729c06d030 msgr2=0x7f729c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 shutdown_connections 2023-10-06T05:33:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.104+0000 7f72a279d700 1 -- 172.21.15.33:0/3764943487 wait complete. 2023-10-06T05:33:50.124 INFO:tasks.cephfs_test_runner:test_stale_write_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:33:50.124 INFO:tasks.cephfs_test_runner:Starting test: test_trim_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:33:50.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_trim_caps' 2023-10-06T05:33:50.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.308+0000 7f9596d9d700 1 Processor -- start 2023-10-06T05:33:50.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.308+0000 7f9596d9d700 1 -- start start 2023-10-06T05:33:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9596d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f95900c8b50 0x7f95900c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9596d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9590155ec0 0x7f95900c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9596d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959006ab60 con 0x7f9590155af0 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9596d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959006b7e0 con 0x7f9590155ec0 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9596d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959006ae40 con 0x7f95900c8b50 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9595d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9590155af0 0x7f959006a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48714/0 (socket says 172.21.15.33:48714) 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f9595d9b700 1 -- 172.21.15.33:0/3814505494 learned_addr learned my addr 172.21.15.33:0/3814505494 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.309+0000 7f959659c700 1 --2- 172.21.15.33:0/3814505494 >> v2:172.21.15.33:3300/0 conn(0x7f9590155ec0 0x7f95900c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f959559a700 1 --2- 172.21.15.33:0/3814505494 >> v2:172.21.15.167:3300/0 conn(0x7f95900c8b50 0x7f95900c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:50.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f9594d99700 1 -- 172.21.15.33:0/3814505494 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727316453 0 0) 0x7f959006ab60 con 0x7f9590155af0 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f9594d99700 1 -- 172.21.15.33:0/3814505494 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f957c003570 con 0x7f9590155af0 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f959659c700 1 -- 172.21.15.33:0/3814505494 >> v2:172.21.15.167:3300/0 conn(0x7f95900c8b50 msgr2=0x7f95900c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f959659c700 1 --2- 172.21.15.33:0/3814505494 >> v2:172.21.15.167:3300/0 conn(0x7f95900c8b50 0x7f95900c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f959659c700 1 -- 172.21.15.33:0/3814505494 >> v1:172.21.15.33:6789/0 conn(0x7f9590155af0 legacy=0x7f959006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f959659c700 1 -- 172.21.15.33:0/3814505494 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959006b120 con 0x7f9590155ec0 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f959659c700 1 --2- 172.21.15.33:0/3814505494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95900c7750 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f959006f250 tx=0x7f9588009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cc62518f67ec45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:50.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f9594d99700 1 -- 172.21.15.33:0/3814505494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800e070 con 0x7f9590155ec0 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f9594d99700 1 -- 172.21.15.33:0/3814505494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958800bab0 con 0x7f9590155ec0 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.310+0000 7f9594d99700 1 -- 172.21.15.33:0/3814505494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800be70 con 0x7f9590155ec0 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 -- 172.21.15.33:0/3814505494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 msgr2=0x7f95900c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3814505494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95900c7750 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f959006f250 tx=0x7f9588009780 comp rx=0 tx=0).stop 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 -- 172.21.15.33:0/3814505494 shutdown_connections 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3814505494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95900c7750 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3814505494 >> v2:172.21.15.167:3300/0 conn(0x7f95900c8b50 0x7f95900c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 -- 172.21.15.33:0/3814505494 >> 172.21.15.33:0/3814505494 conn(0x7f959006d030 msgr2=0x7f959006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 -- 172.21.15.33:0/3814505494 shutdown_connections 2023-10-06T05:33:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.311+0000 7f9596d9d700 1 -- 172.21.15.33:0/3814505494 wait complete. 2023-10-06T05:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.313+0000 7f9596d9d700 1 Processor -- start 2023-10-06T05:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.313+0000 7f9596d9d700 1 -- start start 2023-10-06T05:33:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9596d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95900c8b50 0x7f95900ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9596d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590155af0 0x7f95900b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9596d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95901563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9596d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590159110 con 0x7f9590155ec0 2023-10-06T05:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9596d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b1170 con 0x7f9590155af0 2023-10-06T05:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9596d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b0560 con 0x7f95900c8b50 2023-10-06T05:33:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95901563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95901563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47496/0 (socket says 172.21.15.33:47496) 2023-10-06T05:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 -- 172.21.15.33:0/3214645241 learned_addr learned my addr 172.21.15.33:0/3214645241 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f9595d9b700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95900c8b50 0x7f95900ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959559a700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590155af0 0x7f95900b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 -- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95900c8b50 msgr2=0x7f95900ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95900c8b50 0x7f95900ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 -- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590155af0 msgr2=0x7f95900b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590155af0 0x7f95900b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.314+0000 7f959659c700 1 -- 172.21.15.33:0/3214645241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009a20 con 0x7f9590155ec0 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.315+0000 7f959659c700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95901563b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f95900635d0 tx=0x7f9588019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.315+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588012dd0 con 0x7f9590155ec0 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.315+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588027470 con 0x7f9590155ec0 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.315+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588027830 con 0x7f9590155ec0 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.315+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95900b0840 con 0x7f9590155ec0 2023-10-06T05:33:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.315+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95900b0e00 con 0x7f9590155ec0 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.316+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959004c380 con 0x7f9590155ec0 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.317+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9588024070 con 0x7f9590155ec0 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.318+0000 7f9585ffb700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9574066f30 0x7f95740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.318+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9588086e00 con 0x7f9590155ec0 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.318+0000 7f9595d9b700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9574066f30 0x7f95740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.319+0000 7f9595d9b700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9574066f30 0x7f95740693e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f958c007fd0 tx=0x7f958c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.322+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9588054140 con 0x7f9590155ec0 2023-10-06T05:33:50.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:50.604+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_trim_caps"]} v 0) v1 -- 0x7f959006b400 con 0x7f9590155ec0 2023-10-06T05:33:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.107+0000 7f9585ffb700 1 -- 172.21.15.33:0/3214645241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_trim_caps"]}]=0 v955) v1 ==== 149+0+0 (secure 0 0 0) 0x7f9588057760 con 0x7f9590155ec0 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9574066f30 msgr2=0x7f95740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9574066f30 0x7f95740693e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f958c007fd0 tx=0x7f958c009040 comp rx=0 tx=0).stop 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 msgr2=0x7f95901563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95901563b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f95900635d0 tx=0x7f9588019000 comp rx=0 tx=0).stop 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 shutdown_connections 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9574066f30 0x7f95740693e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9590155ec0 0x7f95901563b0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95900c8b50 0x7f95900ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 --2- 172.21.15.33:0/3214645241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590155af0 0x7f95900b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 >> 172.21.15.33:0/3214645241 conn(0x7f959006d030 msgr2=0x7f95900c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 shutdown_connections 2023-10-06T05:33:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.110+0000 7f9596d9d700 1 -- 172.21.15.33:0/3214645241 wait complete. 2023-10-06T05:33:51.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:33:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.323+0000 7fbd88d9d700 1 Processor -- start 2023-10-06T05:33:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.323+0000 7fbd88d9d700 1 -- start start 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd88d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd84156370 0x7fbd840af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd88d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 0x7fbd840b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd88d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd840ad430 con 0x7fbd840cc7e0 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd88d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd840afe20 con 0x7fbd84156370 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd88d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b0b70 con 0x7fbd840b01f0 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd7bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbd840cc7e0 0x7fbd840c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48730/0 (socket says 172.21.15.33:48730) 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.324+0000 7fbd7bfff700 1 -- 172.21.15.33:0/1025521251 learned_addr learned my addr 172.21.15.33:0/1025521251 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd83fff700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 0x7fbd840b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd837fe700 1 --2- 172.21.15.33:0/1025521251 >> v2:172.21.15.167:3300/0 conn(0x7fbd84156370 0x7fbd840af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125131152 0 0) 0x7fbd840ad430 con 0x7fbd840cc7e0 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd70003570 con 0x7fbd840cc7e0 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640878630 0 0) 0x7fbd70003570 con 0x7fbd840cc7e0 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 msgr2=0x7fbd840b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 0x7fbd840b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 >> v2:172.21.15.167:3300/0 conn(0x7fbd84156370 msgr2=0x7fbd840af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 --2- 172.21.15.33:0/1025521251 >> v2:172.21.15.167:3300/0 conn(0x7fbd84156370 0x7fbd840af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8406bc70 con 0x7fbd840cc7e0 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fbd6c001470 con 0x7fbd840cc7e0 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 >> v1:172.21.15.33:6789/0 conn(0x7fbd840cc7e0 legacy=0x7fbd840c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd70003a60 0x7fbd70003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 0x7fbd70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70004d50 0x7fbd700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840afe20 con 0x7fbd70003a60 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b0b70 con 0x7fbd70004d50 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.325+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7000ba60 con 0x7fbd840b01f0 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd70003a60 0x7fbd70003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd83fff700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70004d50 0x7fbd700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd7bfff700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 0x7fbd70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 -- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 msgr2=0x7fbd70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd840b01f0 0x7fbd70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 -- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70004d50 msgr2=0x7fbd700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70004d50 0x7fbd700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 -- 172.21.15.33:0/1025521251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70003570 con 0x7fbd70003a60 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd83fff700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70004d50 0x7fbd700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.326+0000 7fbd837fe700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd70003a60 0x7fbd70003e70 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009c00 tx=0x7fbd740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420ae5c6aabc07a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7400e6b0 con 0x7fbd70003a60 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7400e810 con 0x7fbd70003a60 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd82ffd700 1 -- 172.21.15.33:0/1025521251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7400ebd0 con 0x7fbd70003a60 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd88d9d700 1 -- 172.21.15.33:0/1025521251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd70003a60 msgr2=0x7fbd70003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/1025521251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd70003a60 0x7fbd70003e70 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009c00 tx=0x7fbd740091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd83fff700 1 -- 172.21.15.33:0/1025521251 reap_dead start 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd88d9d700 1 -- 172.21.15.33:0/1025521251 shutdown_connections 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd88d9d700 1 -- 172.21.15.33:0/1025521251 >> 172.21.15.33:0/1025521251 conn(0x7fbd8406d030 msgr2=0x7fbd840cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd88d9d700 1 -- 172.21.15.33:0/1025521251 shutdown_connections 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.327+0000 7fbd88d9d700 1 -- 172.21.15.33:0/1025521251 wait complete. 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.329+0000 7fbd88d9d700 1 Processor -- start 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.329+0000 7fbd88d9d700 1 -- start start 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd88d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70003a60 0x7fbd840caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd88d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd841578c0 0x7fbd840cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd88d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 0x7fbd840aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd88d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd841590b0 con 0x7fbd840aa5b0 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd88d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84158990 con 0x7fbd70003a60 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd88d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84157e10 con 0x7fbd841578c0 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 0x7fbd840aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 0x7fbd840aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47516/0 (socket says 172.21.15.33:47516) 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 -- 172.21.15.33:0/2986778252 learned_addr learned my addr 172.21.15.33:0/2986778252 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd837fe700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70003a60 0x7fbd840caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd7bfff700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd841578c0 0x7fbd840cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 -- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd841578c0 msgr2=0x7fbd840cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd841578c0 0x7fbd840cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 -- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70003a60 msgr2=0x7fbd840caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70003a60 0x7fbd840caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd83fff700 1 -- 172.21.15.33:0/2986778252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd841580f0 con 0x7fbd840aa5b0 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.330+0000 7fbd837fe700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70003a60 0x7fbd840caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.331+0000 7fbd83fff700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 0x7fbd840aaa30 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fbd84071330 tx=0x7fbd7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.331+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c01b070 con 0x7fbd840aa5b0 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.331+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c00f040 con 0x7fbd840aa5b0 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.331+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00e050 con 0x7fbd840aa5b0 2023-10-06T05:33:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.331+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd841583d0 con 0x7fbd840aa5b0 2023-10-06T05:33:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.331+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd840aaf70 con 0x7fbd840aa5b0 2023-10-06T05:33:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.332+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8404c380 con 0x7fbd840aa5b0 2023-10-06T05:33:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.333+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd7c003020 con 0x7fbd840aa5b0 2023-10-06T05:33:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.334+0000 7fbd7b7fe700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd64066f30 0x7fbd640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.335+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbd7c084a20 con 0x7fbd840aa5b0 2023-10-06T05:33:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.335+0000 7fbd837fe700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd64066f30 0x7fbd640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.336+0000 7fbd837fe700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd64066f30 0x7fbd640693e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbd7401c040 tx=0x7fbd74019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.339+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd7c051d60 con 0x7fbd840aa5b0 2023-10-06T05:33:51.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:33:51.521 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:33:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.543+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbd8406f310 con 0x7fbd840aa5b0 2023-10-06T05:33:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.548+0000 7fbd7b7fe700 1 -- 172.21.15.33:0/2986778252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd7c055380 con 0x7fbd840aa5b0 2023-10-06T05:33:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd64066f30 msgr2=0x7fbd640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd64066f30 0x7fbd640693e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbd7401c040 tx=0x7fbd74019000 comp rx=0 tx=0).stop 2023-10-06T05:33:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 msgr2=0x7fbd840aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 0x7fbd840aaa30 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fbd84071330 tx=0x7fbd7c00b040 comp rx=0 tx=0).stop 2023-10-06T05:33:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 shutdown_connections 2023-10-06T05:33:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbd64066f30 0x7fbd640693e0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd840aa5b0 0x7fbd840aaa30 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd70003a60 0x7fbd840caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 --2- 172.21.15.33:0/2986778252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbd841578c0 0x7fbd840cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 >> 172.21.15.33:0/2986778252 conn(0x7fbd8406d030 msgr2=0x7fbd8406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 shutdown_connections 2023-10-06T05:33:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:51.550+0000 7fbd88d9d700 1 -- 172.21.15.33:0/2986778252 wait complete. 2023-10-06T05:33:51.572 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:33:51.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:33:51.731 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:33:51.818 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:33:51.819 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:33:51.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:33:52.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.004+0000 7f1fb999d700 1 Processor -- start 2023-10-06T05:33:52.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.004+0000 7f1fb999d700 1 -- start start 2023-10-06T05:33:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb999d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1fb40cb710 0x7f1fb406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb999d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40c9cb0 con 0x7f1fb413faa0 2023-10-06T05:33:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb999d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40ca270 con 0x7f1fb40c8050 2023-10-06T05:33:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb999d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb406b840 con 0x7f1fb40cb710 2023-10-06T05:33:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb899b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1fb413faa0 0x7f1fb40c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48742/0 (socket says 172.21.15.33:48742) 2023-10-06T05:33:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb899b700 1 -- 172.21.15.33:0/346728276 learned_addr learned my addr 172.21.15.33:0/346728276 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb919c700 1 --2- 172.21.15.33:0/346728276 >> v2:172.21.15.33:3300/0 conn(0x7f1fb40cb710 0x7f1fb406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb37fe700 1 -- 172.21.15.33:0/346728276 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184840058 0 0) 0x7f1fb40c9cb0 con 0x7f1fb413faa0 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb37fe700 1 -- 172.21.15.33:0/346728276 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa4003570 con 0x7f1fb413faa0 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb919c700 1 -- 172.21.15.33:0/346728276 >> v1:172.21.15.167:6789/0 conn(0x7f1fb40c8050 legacy=0x7f1fb406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb919c700 1 -- 172.21.15.33:0/346728276 >> v1:172.21.15.33:6789/0 conn(0x7f1fb413faa0 legacy=0x7f1fb40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.005+0000 7f1fb919c700 1 -- 172.21.15.33:0/346728276 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb406bb20 con 0x7f1fb40cb710 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb919c700 1 --2- 172.21.15.33:0/346728276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb40cb710 0x7f1fb406b1e0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009e60 tx=0x7f1fac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a728795607e6726 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb37fe700 1 -- 172.21.15.33:0/346728276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00e070 con 0x7f1fb40cb710 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb37fe700 1 -- 172.21.15.33:0/346728276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00bb60 con 0x7f1fb40cb710 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb37fe700 1 -- 172.21.15.33:0/346728276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac012430 con 0x7f1fb40cb710 2023-10-06T05:33:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb999d700 1 -- 172.21.15.33:0/346728276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb40cb710 msgr2=0x7f1fb406b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb999d700 1 --2- 172.21.15.33:0/346728276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb40cb710 0x7f1fb406b1e0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009e60 tx=0x7f1fac009780 comp rx=0 tx=0).stop 2023-10-06T05:33:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb999d700 1 -- 172.21.15.33:0/346728276 shutdown_connections 2023-10-06T05:33:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb999d700 1 --2- 172.21.15.33:0/346728276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb40cb710 0x7f1fb406b1e0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.006+0000 7f1fb999d700 1 -- 172.21.15.33:0/346728276 >> 172.21.15.33:0/346728276 conn(0x7f1fb406d030 msgr2=0x7f1fb406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.007+0000 7f1fb999d700 1 -- 172.21.15.33:0/346728276 shutdown_connections 2023-10-06T05:33:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.007+0000 7f1fb999d700 1 -- 172.21.15.33:0/346728276 wait complete. 2023-10-06T05:33:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.008+0000 7f1fb999d700 1 Processor -- start 2023-10-06T05:33:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.008+0000 7f1fb999d700 1 -- start start 2023-10-06T05:33:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fb40c8050 0x7f1fb40ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fb40cb710 0x7f1fb413d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 0x7f1fb413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb413e500 con 0x7f1fb413faa0 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb413cd30 con 0x7f1fb40c8050 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40c32c0 con 0x7f1fb40cb710 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 0x7f1fb413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 0x7f1fb413ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47524/0 (socket says 172.21.15.33:47524) 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 -- 172.21.15.33:0/2458987291 learned_addr learned my addr 172.21.15.33:0/2458987291 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb3fff700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fb40cb710 0x7f1fb413d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb899b700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fb40c8050 0x7f1fb40ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 -- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fb40cb710 msgr2=0x7f1fb413d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fb40cb710 0x7f1fb413d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 -- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fb40c8050 msgr2=0x7f1fb40ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fb40c8050 0x7f1fb40ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 -- 172.21.15.33:0/2458987291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009a20 con 0x7f1fb413faa0 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb899b700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fb40c8050 0x7f1fb40ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb919c700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 0x7f1fb413ded0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b6c0 tx=0x7f1fac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac012e80 con 0x7f1fb413faa0 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac01d630 con 0x7f1fb413faa0 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb40c35a0 con 0x7f1fb413faa0 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac01d9d0 con 0x7f1fb413faa0 2023-10-06T05:33:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.010+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb40c3b60 con 0x7f1fb413faa0 2023-10-06T05:33:52.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.011+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb40c3d80 con 0x7f1fb413faa0 2023-10-06T05:33:52.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.012+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fac024070 con 0x7f1fb413faa0 2023-10-06T05:33:52.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.013+0000 7f1fb0ff9700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f98066f30 0x7f1f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.013+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1fac086cb0 con 0x7f1fb413faa0 2023-10-06T05:33:52.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.014+0000 7f1fb899b700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f98066f30 0x7f1f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.014+0000 7f1fb899b700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f98066f30 0x7f1f980693e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8007fd0 tx=0x7f1fa8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:52.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.018+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fac053ff0 con 0x7f1fb413faa0 2023-10-06T05:33:52.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.266+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fb404c380 con 0x7f1fb413faa0 2023-10-06T05:33:52.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.267+0000 7f1fb0ff9700 1 -- 172.21.15.33:0/2458987291 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+3544 (secure 0 0 0) 0x7f1fac02cdf0 con 0x7f1fb413faa0 2023-10-06T05:33:52.271 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:52.271 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":197,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3907649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3907649464},{"type":"v1","addr":"172.21.15.167:6836","nonce":3907649464}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/2638833874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2638833874},{"type":"v1","addr":"172.21.15.33:6833","nonce":2638833874}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":188},{"gid":11487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1584791676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1584791676},{"type":"v1","addr":"172.21.15.167:6837","nonce":1584791676}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":189},{"gid":11778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2913855532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2913855532},{"type":"v1","addr":"172.21.15.33:6835","nonce":2913855532}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[]} 2023-10-06T05:33:52.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f98066f30 msgr2=0x7f1f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f98066f30 0x7f1f980693e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8007fd0 tx=0x7f1fa8009040 comp rx=0 tx=0).stop 2023-10-06T05:33:52.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 msgr2=0x7f1fb413ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 0x7f1fb413ded0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b6c0 tx=0x7f1fac019000 comp rx=0 tx=0).stop 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 shutdown_connections 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1f98066f30 0x7f1f980693e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fb413faa0 0x7f1fb413ded0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fb40c8050 0x7f1fb40ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 --2- 172.21.15.33:0/2458987291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fb40cb710 0x7f1fb413d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 >> 172.21.15.33:0/2458987291 conn(0x7f1fb406d030 msgr2=0x7f1fb40c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 shutdown_connections 2023-10-06T05:33:52.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.270+0000 7f1fb999d700 1 -- 172.21.15.33:0/2458987291 wait complete. 2023-10-06T05:33:52.274 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 197 2023-10-06T05:33:52.290 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:33:52.290 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:33:52.290 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:33:52.290 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:33:52.290 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:33:52.291 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:33:52.291 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:33:52.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:33:52.291 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:33:52.291 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:33:52.291 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:33:52.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:33:52.292 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:33:52.292 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:33:52.292 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:33:52.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:33:52.305 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:33:52.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:33:52.306 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:33:52.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:33:52.308 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:33:52.308 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:33:52.309 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:33:52.309 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:33:52.350 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:33:52.350 DEBUG:teuthology.parallel:result is None 2023-10-06T05:33:52.352 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:33:52.352 DEBUG:teuthology.parallel:result is None 2023-10-06T05:33:52.353 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:33:52.353 DEBUG:teuthology.parallel:result is None 2023-10-06T05:33:52.355 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:33:52.355 DEBUG:teuthology.parallel:result is None 2023-10-06T05:33:52.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:33:52.487 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:33:52.487 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:33:52.491 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:33:52.491 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:33:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.550+0000 7f8472b9d700 1 Processor -- start 2023-10-06T05:33:52.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.550+0000 7f8472b9d700 1 -- start start 2023-10-06T05:33:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8472b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f846c0cc0d0 0x7f846c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8472b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f846c0cc540 0x7f846c0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8472b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c06a410 con 0x7f846c157150 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8472b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c0cccd0 con 0x7f846c0cc0d0 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8472b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c0c7500 con 0x7f846c0cc540 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8471b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f846c157150 0x7f846c1557f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48750/0 (socket says 172.21.15.33:48750) 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8471b9b700 1 -- 172.21.15.33:0/514701968 learned_addr learned my addr 172.21.15.33:0/514701968 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847139a700 1 --2- 172.21.15.33:0/514701968 >> v2:172.21.15.33:3300/0 conn(0x7f846c0cc0d0 0x7f846c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847239c700 1 --2- 172.21.15.33:0/514701968 >> v2:172.21.15.167:3300/0 conn(0x7f846c0cc540 0x7f846c0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8470b99700 1 -- 172.21.15.33:0/514701968 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790354527 0 0) 0x7f846c06a410 con 0x7f846c157150 2023-10-06T05:33:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8470b99700 1 -- 172.21.15.33:0/514701968 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8458003570 con 0x7f846c157150 2023-10-06T05:33:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847139a700 1 -- 172.21.15.33:0/514701968 >> v2:172.21.15.167:3300/0 conn(0x7f846c0cc540 msgr2=0x7f846c0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847139a700 1 --2- 172.21.15.33:0/514701968 >> v2:172.21.15.167:3300/0 conn(0x7f846c0cc540 0x7f846c0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847139a700 1 -- 172.21.15.33:0/514701968 >> v1:172.21.15.33:6789/0 conn(0x7f846c157150 legacy=0x7f846c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847139a700 1 -- 172.21.15.33:0/514701968 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c0c77e0 con 0x7f846c0cc0d0 2023-10-06T05:33:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f847139a700 1 --2- 172.21.15.33:0/514701968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c0c6960 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f846c06d280 tx=0x7f845c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391ccda43f8f5cef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.552+0000 7f8470b99700 1 -- 172.21.15.33:0/514701968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c011630 con 0x7f846c0cc0d0 2023-10-06T05:33:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8470b99700 1 -- 172.21.15.33:0/514701968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c0117b0 con 0x7f846c0cc0d0 2023-10-06T05:33:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8470b99700 1 -- 172.21.15.33:0/514701968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c011b70 con 0x7f846c0cc0d0 2023-10-06T05:33:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 -- 172.21.15.33:0/514701968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 msgr2=0x7f846c0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 --2- 172.21.15.33:0/514701968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c0c6960 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f846c06d280 tx=0x7f845c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 -- 172.21.15.33:0/514701968 shutdown_connections 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 --2- 172.21.15.33:0/514701968 >> v2:172.21.15.167:3300/0 conn(0x7f846c0cc540 0x7f846c0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 --2- 172.21.15.33:0/514701968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c0c6960 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 -- 172.21.15.33:0/514701968 >> 172.21.15.33:0/514701968 conn(0x7f846c06c560 msgr2=0x7f846c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 -- 172.21.15.33:0/514701968 shutdown_connections 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.553+0000 7f8472b9d700 1 -- 172.21.15.33:0/514701968 wait complete. 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.555+0000 7f8472b9d700 1 Processor -- start 2023-10-06T05:33:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.555+0000 7f8472b9d700 1 -- start start 2023-10-06T05:33:52.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8472b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8472b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0cc540 0x7f846c1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8472b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c157150 0x7f846c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8472b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c16aa50 con 0x7f846c0cc0d0 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8472b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c154ad0 con 0x7f846c0cc540 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8472b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c154db0 con 0x7f846c157150 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47574/0 (socket says 172.21.15.33:47574) 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 -- 172.21.15.33:0/169553626 learned_addr learned my addr 172.21.15.33:0/169553626 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 -- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c157150 msgr2=0x7f846c0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c157150 0x7f846c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 -- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0cc540 msgr2=0x7f846c1540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0cc540 0x7f846c1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.556+0000 7f8471b9b700 1 -- 172.21.15.33:0/169553626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c009420 con 0x7f846c0cc0d0 2023-10-06T05:33:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.557+0000 7f8471b9b700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c143b20 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f846800c590 tx=0x7f8468007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:52.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.557+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468005b30 con 0x7f846c0cc0d0 2023-10-06T05:33:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.557+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8468005cb0 con 0x7f846c0cc0d0 2023-10-06T05:33:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.557+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468007320 con 0x7f846c0cc0d0 2023-10-06T05:33:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.557+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c16a400 con 0x7f846c0cc0d0 2023-10-06T05:33:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.558+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c144060 con 0x7f846c0cc0d0 2023-10-06T05:33:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.558+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c04c380 con 0x7f846c0cc0d0 2023-10-06T05:33:52.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.559+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f846801b3b0 con 0x7f846c0cc0d0 2023-10-06T05:33:52.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.560+0000 7f8461ffb700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8450066f30 0x7f84500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:52.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.560+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8468083a30 con 0x7f846c0cc0d0 2023-10-06T05:33:52.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.560+0000 7f847139a700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8450066f30 0x7f84500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:52.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.561+0000 7f847139a700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8450066f30 0x7f84500693e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f846c06d280 tx=0x7f845c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:52.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.564+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8468050d70 con 0x7f846c0cc0d0 2023-10-06T05:33:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:52.781+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f846c064750 con 0x7f846c0cc0d0 2023-10-06T05:33:53.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.124+0000 7f8461ffb700 1 -- 172.21.15.33:0/169553626 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v259) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8468017020 con 0x7f846c0cc0d0 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8450066f30 msgr2=0x7f84500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8450066f30 0x7f84500693e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f846c06d280 tx=0x7f845c017000 comp rx=0 tx=0).stop 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 msgr2=0x7f846c143b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c143b20 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f846800c590 tx=0x7f8468007e00 comp rx=0 tx=0).stop 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 shutdown_connections 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8450066f30 0x7f84500693e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c157150 0x7f846c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846c0cc0d0 0x7f846c143b20 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 --2- 172.21.15.33:0/169553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0cc540 0x7f846c1540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 >> 172.21.15.33:0/169553626 conn(0x7f846c06c560 msgr2=0x7f846c06a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:53.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 shutdown_connections 2023-10-06T05:33:53.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.127+0000 7f8472b9d700 1 -- 172.21.15.33:0/169553626 wait complete. 2023-10-06T05:33:53.130 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:33:53.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:33:53.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.336+0000 7f7fd89fe700 1 Processor -- start 2023-10-06T05:33:53.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.336+0000 7f7fd89fe700 1 -- start start 2023-10-06T05:33:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.337+0000 7f7fd89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.337+0000 7f7fd89fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7fd406aad0 0x7f7fd406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.337+0000 7f7fd89fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd40c9cb0 con 0x7f7fd40c7940 2023-10-06T05:33:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.337+0000 7f7fd89fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd40ca270 con 0x7f7fd406aad0 2023-10-06T05:33:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.337+0000 7f7fd89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd406b840 con 0x7f7fd413faa0 2023-10-06T05:33:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.337+0000 7f7fd2d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7fd406aad0 0x7f7fd406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41676/0 (socket says 172.21.15.33:41676) 2023-10-06T05:33:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 -- 172.21.15.33:0/3010619471 learned_addr learned my addr 172.21.15.33:0/3010619471 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd259c700 1 --1- 172.21.15.33:0/3010619471 >> v1:172.21.15.167:6789/0 conn(0x7f7fd40c7940 0x7f7fd40c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49934/0 (socket says 172.21.15.33:49934) 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 -- 172.21.15.33:0/3010619471 >> v2:172.21.15.167:3300/0 conn(0x7f7fd406aad0 msgr2=0x7f7fd406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 --2- 172.21.15.33:0/3010619471 >> v2:172.21.15.167:3300/0 conn(0x7f7fd406aad0 0x7f7fd406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 -- 172.21.15.33:0/3010619471 >> v1:172.21.15.167:6789/0 conn(0x7f7fd40c7940 legacy=0x7f7fd40c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 -- 172.21.15.33:0/3010619471 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd406bb20 con 0x7f7fd413faa0 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.338+0000 7f7fd2d9d700 1 --2- 172.21.15.33:0/3010619471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd413fe70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40099b0 tx=0x7f7fc4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d3ef19049e95d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd1d9b700 1 -- 172.21.15.33:0/3010619471 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4009ea0 con 0x7f7fd413faa0 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd1d9b700 1 -- 172.21.15.33:0/3010619471 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc400b860 con 0x7f7fd413faa0 2023-10-06T05:33:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd1d9b700 1 -- 172.21.15.33:0/3010619471 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400bc00 con 0x7f7fd413faa0 2023-10-06T05:33:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3010619471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 msgr2=0x7f7fd413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3010619471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd413fe70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40099b0 tx=0x7f7fc4009250 comp rx=0 tx=0).stop 2023-10-06T05:33:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3010619471 shutdown_connections 2023-10-06T05:33:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3010619471 >> v2:172.21.15.167:3300/0 conn(0x7f7fd406aad0 0x7f7fd406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3010619471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd413fe70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3010619471 >> 172.21.15.33:0/3010619471 conn(0x7f7fd406d030 msgr2=0x7f7fd406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3010619471 shutdown_connections 2023-10-06T05:33:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.339+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3010619471 wait complete. 2023-10-06T05:33:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.341+0000 7f7fd89fe700 1 Processor -- start 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.341+0000 7f7fd89fe700 1 -- start start 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd89fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd406aad0 0x7f7fd40adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd89fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 0x7f7fd4159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd40ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd89fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd40c3b60 con 0x7f7fd40c7940 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd89fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415a980 con 0x7f7fd406aad0 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd40c3230 con 0x7f7fd413faa0 2023-10-06T05:33:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 0x7f7fd4159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 0x7f7fd4159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47590/0 (socket says 172.21.15.33:47590) 2023-10-06T05:33:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd259c700 1 -- 172.21.15.33:0/3003088025 learned_addr learned my addr 172.21.15.33:0/3003088025 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd2d9d700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd406aad0 0x7f7fd40adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.342+0000 7f7fd359e700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd40ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd259c700 1 -- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 msgr2=0x7f7fd40ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd259c700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd40ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd259c700 1 -- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd406aad0 msgr2=0x7f7fd40adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd259c700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd406aad0 0x7f7fd40adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd259c700 1 -- 172.21.15.33:0/3003088025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40094f0 con 0x7f7fd40c7940 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd259c700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 0x7f7fd4159ee0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c7c0 tx=0x7f7fc8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8005a70 con 0x7f7fd40c7940 2023-10-06T05:33:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc8005bd0 con 0x7f7fd40c7940 2023-10-06T05:33:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80070d0 con 0x7f7fd40c7940 2023-10-06T05:33:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.343+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd40c3510 con 0x7f7fd40c7940 2023-10-06T05:33:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.344+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd40c6730 con 0x7f7fd40c7940 2023-10-06T05:33:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.345+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd404c380 con 0x7f7fd40c7940 2023-10-06T05:33:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.345+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fc8007440 con 0x7f7fd40c7940 2023-10-06T05:33:53.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.347+0000 7f7fbaffd700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7fc0066f30 0x7f7fc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.347+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7fc80837e0 con 0x7f7fd40c7940 2023-10-06T05:33:53.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.348+0000 7f7fd2d9d700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7fc0066f30 0x7f7fc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.348+0000 7f7fd2d9d700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7fc0066f30 0x7f7fc00693e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40099b0 tx=0x7f7fc4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:53.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.351+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fc8050ba0 con 0x7f7fd40c7940 2023-10-06T05:33:53.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.632+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7fd40c9f00 con 0x7f7fd40c7940 2023-10-06T05:33:53.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.633+0000 7f7fbaffd700 1 -- 172.21.15.33:0/3003088025 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7fc8017090 con 0x7f7fd40c7940 2023-10-06T05:33:53.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:53.634 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:33:53.634 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:33:53.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.635 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:33:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.639 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.641 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:33:53.645 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.647 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:33:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:33:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:33:53.651 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:33:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7fc0066f30 msgr2=0x7f7fc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7fc0066f30 0x7f7fc00693e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40099b0 tx=0x7f7fc4017000 comp rx=0 tx=0).stop 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 msgr2=0x7f7fd4159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 0x7f7fd4159ee0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c7c0 tx=0x7f7fc8007f70 comp rx=0 tx=0).stop 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 shutdown_connections 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7fc0066f30 0x7f7fc00693e0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd413faa0 0x7f7fd40ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd406aad0 0x7f7fd40adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 --2- 172.21.15.33:0/3003088025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd40c7940 0x7f7fd4159ee0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 >> 172.21.15.33:0/3003088025 conn(0x7f7fd406d030 msgr2=0x7f7fd413ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 shutdown_connections 2023-10-06T05:33:53.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.635+0000 7f7fd89fe700 1 -- 172.21.15.33:0/3003088025 wait complete. 2023-10-06T05:33:53.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:33:53.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.839+0000 7fb6c759e700 1 Processor -- start 2023-10-06T05:33:53.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.839+0000 7fb6c759e700 1 -- start start 2023-10-06T05:33:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.840+0000 7fb6c759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c813faa0 0x7fb6c813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.840+0000 7fb6c759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6c806aad0 0x7fb6c806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.840+0000 7fb6c759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c80c9cb0 con 0x7fb6c80c7940 2023-10-06T05:33:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.840+0000 7fb6c759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c80ca270 con 0x7fb6c806aad0 2023-10-06T05:33:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.840+0000 7fb6c759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c806b840 con 0x7fb6c813faa0 2023-10-06T05:33:53.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c6d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb6c806aad0 0x7fb6c806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c5d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb6c80c7940 0x7fb6c80c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48764/0 (socket says 172.21.15.33:48764) 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c5d9b700 1 -- 172.21.15.33:0/4067648031 learned_addr learned my addr 172.21.15.33:0/4067648031 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c659c700 1 --2- 172.21.15.33:0/4067648031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c813faa0 0x7fb6c813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c559a700 1 -- 172.21.15.33:0/4067648031 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949685054 0 0) 0x7fb6c80c9cb0 con 0x7fb6c80c7940 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c559a700 1 -- 172.21.15.33:0/4067648031 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b0003570 con 0x7fb6c80c7940 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c6d9d700 1 -- 172.21.15.33:0/4067648031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c813faa0 msgr2=0x7fb6c813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c6d9d700 1 --2- 172.21.15.33:0/4067648031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c813faa0 0x7fb6c813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c6d9d700 1 -- 172.21.15.33:0/4067648031 >> v1:172.21.15.33:6789/0 conn(0x7fb6c80c7940 legacy=0x7fb6c80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.841+0000 7fb6c6d9d700 1 -- 172.21.15.33:0/4067648031 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c806bb20 con 0x7fb6c806aad0 2023-10-06T05:33:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.842+0000 7fb6c6d9d700 1 --2- 172.21.15.33:0/4067648031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c806af50 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009e60 tx=0x7fb6c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1c4609b60420ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.842+0000 7fb6c559a700 1 -- 172.21.15.33:0/4067648031 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000e070 con 0x7fb6c806aad0 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.842+0000 7fb6c559a700 1 -- 172.21.15.33:0/4067648031 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000bab0 con 0x7fb6c806aad0 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.842+0000 7fb6c559a700 1 -- 172.21.15.33:0/4067648031 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000be70 con 0x7fb6c806aad0 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.842+0000 7fb6c759e700 1 -- 172.21.15.33:0/4067648031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 msgr2=0x7fb6c806af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.842+0000 7fb6c759e700 1 --2- 172.21.15.33:0/4067648031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c806af50 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009e60 tx=0x7fb6c0009780 comp rx=0 tx=0).stop 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.843+0000 7fb6c759e700 1 -- 172.21.15.33:0/4067648031 shutdown_connections 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.843+0000 7fb6c759e700 1 --2- 172.21.15.33:0/4067648031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c806af50 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.843+0000 7fb6c759e700 1 --2- 172.21.15.33:0/4067648031 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c813faa0 0x7fb6c813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.843+0000 7fb6c759e700 1 -- 172.21.15.33:0/4067648031 >> 172.21.15.33:0/4067648031 conn(0x7fb6c806d030 msgr2=0x7fb6c806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.843+0000 7fb6c759e700 1 -- 172.21.15.33:0/4067648031 shutdown_connections 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.843+0000 7fb6c759e700 1 -- 172.21.15.33:0/4067648031 wait complete. 2023-10-06T05:33:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.844+0000 7fb6c759e700 1 Processor -- start 2023-10-06T05:33:53.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.844+0000 7fb6c759e700 1 -- start start 2023-10-06T05:33:53.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c813ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c80c7940 0x7fb6c8137060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c813faa0 0x7fb6c8159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c815a130 con 0x7fb6c806aad0 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c813bf60 con 0x7fb6c813faa0 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c813b350 con 0x7fb6c80c7940 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.845+0000 7fb6c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c813ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c813ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47604/0 (socket says 172.21.15.33:47604) 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 -- 172.21.15.33:0/2136542075 learned_addr learned my addr 172.21.15.33:0/2136542075 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c5d9b700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c80c7940 0x7fb6c8137060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c6d9d700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c813faa0 0x7fb6c8159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 -- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c80c7940 msgr2=0x7fb6c8137060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c80c7940 0x7fb6c8137060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 -- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c813faa0 msgr2=0x7fb6c8159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c813faa0 0x7fb6c8159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 -- 172.21.15.33:0/2136542075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0009a20 con 0x7fb6c806aad0 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6c659c700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c813ace0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800ebb0 tx=0x7fb6b800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.846+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800db70 con 0x7fb6c806aad0 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.847+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b800dcd0 con 0x7fb6c806aad0 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.847+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c813b630 con 0x7fb6c806aad0 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.847+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8010650 con 0x7fb6c806aad0 2023-10-06T05:33:53.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.847+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c813bbf0 con 0x7fb6c806aad0 2023-10-06T05:33:53.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.848+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c813c240 con 0x7fb6c806aad0 2023-10-06T05:33:53.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.848+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6b8010a40 con 0x7fb6c806aad0 2023-10-06T05:33:53.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.849+0000 7fb6b67fc700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6a8066f30 0x7fb6a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:53.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.849+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb6b8084690 con 0x7fb6c806aad0 2023-10-06T05:33:53.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.850+0000 7fb6c5d9b700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6a8066f30 0x7fb6a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:53.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.851+0000 7fb6c5d9b700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6a8066f30 0x7fb6a80693e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8137e00 tx=0x7fb6bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:53.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:53.854+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6b8051a50 con 0x7fb6c806aad0 2023-10-06T05:33:54.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.056+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6c804c380 con 0x7fb6c806aad0 2023-10-06T05:33:54.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.057+0000 7fb6b67fc700 1 -- 172.21.15.33:0/2136542075 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v261) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fb6b8055070 con 0x7fb6c806aad0 2023-10-06T05:33:54.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:54.059 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":261,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:53.132616+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:33:54.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6a8066f30 msgr2=0x7fb6a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6a8066f30 0x7fb6a80693e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8137e00 tx=0x7fb6bc009040 comp rx=0 tx=0).stop 2023-10-06T05:33:54.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 msgr2=0x7fb6c813ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c813ace0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800ebb0 tx=0x7fb6b800b430 comp rx=0 tx=0).stop 2023-10-06T05:33:54.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 shutdown_connections 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb6a8066f30 0x7fb6a80693e0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6c813faa0 0x7fb6c8159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6c806aad0 0x7fb6c813ace0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 --2- 172.21.15.33:0/2136542075 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6c80c7940 0x7fb6c8137060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 >> 172.21.15.33:0/2136542075 conn(0x7fb6c806d030 msgr2=0x7fb6c80b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 shutdown_connections 2023-10-06T05:33:54.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.059+0000 7fb6c759e700 1 -- 172.21.15.33:0/2136542075 wait complete. 2023-10-06T05:33:54.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:33:54.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:33:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.268+0000 7ff74b59e700 1 Processor -- start 2023-10-06T05:33:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.268+0000 7ff74b59e700 1 -- start start 2023-10-06T05:33:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c159850 0x7ff74c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff74c15acd0 0x7ff74c06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74c0b0cd0 con 0x7ff74c06d140 2023-10-06T05:33:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74c15a840 con 0x7ff74c15acd0 2023-10-06T05:33:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c159f10 con 0x7ff74c159850 2023-10-06T05:33:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff749d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff74c15acd0 0x7ff74c06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74a59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c159850 0x7ff74c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74ad9d700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff74c06d140 0x7ff74c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49944/0 (socket says 172.21.15.33:49944) 2023-10-06T05:33:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.270+0000 7ff74ad9d700 1 -- 172.21.15.33:0/1559583851 learned_addr learned my addr 172.21.15.33:0/1559583851 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff74959a700 1 -- 172.21.15.33:0/1559583851 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852513847 0 0) 0x7ff74c0b0cd0 con 0x7ff74c06d140 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff74a59c700 1 -- 172.21.15.33:0/1559583851 >> v2:172.21.15.167:3300/0 conn(0x7ff74c15acd0 msgr2=0x7ff74c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff74a59c700 1 --2- 172.21.15.33:0/1559583851 >> v2:172.21.15.167:3300/0 conn(0x7ff74c15acd0 0x7ff74c06edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff74a59c700 1 -- 172.21.15.33:0/1559583851 >> v1:172.21.15.167:6789/0 conn(0x7ff74c06d140 legacy=0x7ff74c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff74a59c700 1 -- 172.21.15.33:0/1559583851 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c15a1f0 con 0x7ff74c159850 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff749d9b700 1 --2- 172.21.15.33:0/1559583851 >> v2:172.21.15.167:3300/0 conn(0x7ff74c15acd0 0x7ff74c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.271+0000 7ff74a59c700 1 --2- 172.21.15.33:0/1559583851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c159850 0x7ff74c06e890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098e0 tx=0x7ff73c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df7f57f444d4ffeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74959a700 1 -- 172.21.15.33:0/1559583851 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00b640 con 0x7ff74c159850 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74959a700 1 -- 172.21.15.33:0/1559583851 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00b7a0 con 0x7ff74c159850 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74959a700 1 -- 172.21.15.33:0/1559583851 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00bb40 con 0x7ff74c159850 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 -- 172.21.15.33:0/1559583851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c159850 msgr2=0x7ff74c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 --2- 172.21.15.33:0/1559583851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c159850 0x7ff74c06e890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098e0 tx=0x7ff73c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 -- 172.21.15.33:0/1559583851 shutdown_connections 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 --2- 172.21.15.33:0/1559583851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c159850 0x7ff74c06e890 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 --2- 172.21.15.33:0/1559583851 >> v2:172.21.15.167:3300/0 conn(0x7ff74c15acd0 0x7ff74c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 -- 172.21.15.33:0/1559583851 >> 172.21.15.33:0/1559583851 conn(0x7ff74c06cd40 msgr2=0x7ff74c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 -- 172.21.15.33:0/1559583851 shutdown_connections 2023-10-06T05:33:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.272+0000 7ff74b59e700 1 -- 172.21.15.33:0/1559583851 wait complete. 2023-10-06T05:33:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.274+0000 7ff74b59e700 1 Processor -- start 2023-10-06T05:33:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.274+0000 7ff74b59e700 1 -- start start 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 0x7ff74c0abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff74c159850 0x7ff74c0ac530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c15acd0 0x7ff74c171210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c16df80 con 0x7ff74c06d140 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c16e260 con 0x7ff74c159850 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c16e540 con 0x7ff74c15acd0 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 0x7ff74c0abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 0x7ff74c0abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47618/0 (socket says 172.21.15.33:47618) 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74a59c700 1 -- 172.21.15.33:0/4205223381 learned_addr learned my addr 172.21.15.33:0/4205223381 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff749d9b700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff74c159850 0x7ff74c0ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.275+0000 7ff74ad9d700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c15acd0 0x7ff74c171210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74a59c700 1 -- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c15acd0 msgr2=0x7ff74c171210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74a59c700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c15acd0 0x7ff74c171210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74a59c700 1 -- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff74c159850 msgr2=0x7ff74c0ac530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74a59c700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff74c159850 0x7ff74c0ac530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74a59c700 1 -- 172.21.15.33:0/4205223381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff740009420 con 0x7ff74c06d140 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74ad9d700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c15acd0 0x7ff74c171210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74a59c700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 0x7ff74c0abff0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7ff74c06c350 tx=0x7ff73c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c000d10 con 0x7ff74c06d140 2023-10-06T05:33:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00bb20 con 0x7ff74c06d140 2023-10-06T05:33:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c009420 con 0x7ff74c06d140 2023-10-06T05:33:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011650 con 0x7ff74c06d140 2023-10-06T05:33:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.276+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c0b2850 con 0x7ff74c06d140 2023-10-06T05:33:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.277+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c04c380 con 0x7ff74c06d140 2023-10-06T05:33:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.278+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff73c0119c0 con 0x7ff74c06d140 2023-10-06T05:33:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.280+0000 7ff73a7fc700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff72c066f30 0x7ff72c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.280+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff73c085760 con 0x7ff74c06d140 2023-10-06T05:33:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.280+0000 7ff749d9b700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff72c066f30 0x7ff72c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.281+0000 7ff749d9b700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff72c066f30 0x7ff72c0693e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff74c164a90 tx=0x7ff7400091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:54.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.285+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff73c052ba0 con 0x7ff74c06d140 2023-10-06T05:33:54.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.486+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff74c0abd10 con 0x7ff74c06d140 2023-10-06T05:33:54.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.486+0000 7ff73a7fc700 1 -- 172.21.15.33:0/4205223381 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v261) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff73c0561c0 con 0x7ff74c06d140 2023-10-06T05:33:54.488 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff72c066f30 msgr2=0x7ff72c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff72c066f30 0x7ff72c0693e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff74c164a90 tx=0x7ff7400091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 msgr2=0x7ff74c0abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 0x7ff74c0abff0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7ff74c06c350 tx=0x7ff73c017000 comp rx=0 tx=0).stop 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 shutdown_connections 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff72c066f30 0x7ff72c0693e0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff74c15acd0 0x7ff74c171210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff74c06d140 0x7ff74c0abff0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 --2- 172.21.15.33:0/4205223381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff74c159850 0x7ff74c0ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 >> 172.21.15.33:0/4205223381 conn(0x7ff74c06cd40 msgr2=0x7ff74c0aae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:54.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 shutdown_connections 2023-10-06T05:33:54.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.489+0000 7ff74b59e700 1 -- 172.21.15.33:0/4205223381 wait complete. 2023-10-06T05:33:54.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:33:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.696+0000 7fc667fff700 1 Processor -- start 2023-10-06T05:33:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.696+0000 7fc667fff700 1 -- start start 2023-10-06T05:33:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.697+0000 7fc667fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680aff90 0x7fc6680a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.697+0000 7fc667fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc66806cb80 0x7fc66815d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.697+0000 7fc667fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66815d3a0 con 0x7fc66806c7b0 2023-10-06T05:33:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.697+0000 7fc667fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6680686c0 con 0x7fc66806cb80 2023-10-06T05:33:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.697+0000 7fc667fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b0470 con 0x7fc6680aff90 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc6677fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc66806cb80 0x7fc66815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc6667fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680aff90 0x7fc6680a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc666ffd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc66806c7b0 0x7fc668067fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:49948/0 (socket says 172.21.15.33:49948) 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc666ffd700 1 -- 172.21.15.33:0/67355142 learned_addr learned my addr 172.21.15.33:0/67355142 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc665ffb700 1 -- 172.21.15.33:0/67355142 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085516888 0 0) 0x7fc66815d3a0 con 0x7fc66806c7b0 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc665ffb700 1 -- 172.21.15.33:0/67355142 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc650003570 con 0x7fc66806c7b0 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.698+0000 7fc6667fc700 1 -- 172.21.15.33:0/67355142 >> v2:172.21.15.167:3300/0 conn(0x7fc66806cb80 msgr2=0x7fc66815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc6667fc700 1 --2- 172.21.15.33:0/67355142 >> v2:172.21.15.167:3300/0 conn(0x7fc66806cb80 0x7fc66815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc6667fc700 1 -- 172.21.15.33:0/67355142 >> v1:172.21.15.167:6789/0 conn(0x7fc66806c7b0 legacy=0x7fc668067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc6667fc700 1 -- 172.21.15.33:0/67355142 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680b0750 con 0x7fc6680aff90 2023-10-06T05:33:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc6667fc700 1 --2- 172.21.15.33:0/67355142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680aff90 0x7fc6680a9890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0098e0 tx=0x7fc65c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235b57c87653a106 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc665ffb700 1 -- 172.21.15.33:0/67355142 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011630 con 0x7fc6680aff90 2023-10-06T05:33:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc665ffb700 1 -- 172.21.15.33:0/67355142 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c011790 con 0x7fc6680aff90 2023-10-06T05:33:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.699+0000 7fc665ffb700 1 -- 172.21.15.33:0/67355142 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011b30 con 0x7fc6680aff90 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 -- 172.21.15.33:0/67355142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680aff90 msgr2=0x7fc6680a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 --2- 172.21.15.33:0/67355142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680aff90 0x7fc6680a9890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0098e0 tx=0x7fc65c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 -- 172.21.15.33:0/67355142 shutdown_connections 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 --2- 172.21.15.33:0/67355142 >> v2:172.21.15.167:3300/0 conn(0x7fc66806cb80 0x7fc66815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 --2- 172.21.15.33:0/67355142 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6680aff90 0x7fc6680a9890 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 -- 172.21.15.33:0/67355142 >> 172.21.15.33:0/67355142 conn(0x7fc66806d670 msgr2=0x7fc66806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 -- 172.21.15.33:0/67355142 shutdown_connections 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.700+0000 7fc667fff700 1 -- 172.21.15.33:0/67355142 wait complete. 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.701+0000 7fc667fff700 1 Processor -- start 2023-10-06T05:33:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.701+0000 7fc667fff700 1 -- start start 2023-10-06T05:33:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc667fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc66806c7b0 0x7fc6681699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc667fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66806cb80 0x7fc668158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc667fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 0x7fc66816a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc667fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668159520 con 0x7fc6680aff90 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc667fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6681597e0 con 0x7fc66806c7b0 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc667fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc668159ac0 con 0x7fc66806cb80 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.702+0000 7fc6677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 0x7fc66816a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 0x7fc66816a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47632/0 (socket says 172.21.15.33:47632) 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 -- 172.21.15.33:0/1853943885 learned_addr learned my addr 172.21.15.33:0/1853943885 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6667fc700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66806cb80 0x7fc668158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc666ffd700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc66806c7b0 0x7fc6681699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 -- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66806cb80 msgr2=0x7fc668158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66806cb80 0x7fc668158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 -- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc66806c7b0 msgr2=0x7fc6681699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc66806c7b0 0x7fc6681699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 -- 172.21.15.33:0/1853943885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009420 con 0x7fc6680aff90 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc6677fe700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 0x7fc66816a360 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fc660009e90 tx=0x7fc66000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66000cb70 con 0x7fc6680aff90 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66000f040 con 0x7fc6680aff90 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660014480 con 0x7fc6680aff90 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.703+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc668159da0 con 0x7fc6680aff90 2023-10-06T05:33:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.704+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66815a360 con 0x7fc6680aff90 2023-10-06T05:33:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.704+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66815a550 con 0x7fc6680aff90 2023-10-06T05:33:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.705+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc660014830 con 0x7fc6680aff90 2023-10-06T05:33:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.706+0000 7fc64effd700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc654066f30 0x7fc6540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.707+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc660084e10 con 0x7fc6680aff90 2023-10-06T05:33:54.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.707+0000 7fc666ffd700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc654066f30 0x7fc6540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:54.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.708+0000 7fc666ffd700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc654066f30 0x7fc6540693e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc658005920 tx=0x7fc658009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.710+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc660088030 con 0x7fc6680aff90 2023-10-06T05:33:54.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.913+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc66804c380 con 0x7fc6680aff90 2023-10-06T05:33:54.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.914+0000 7fc64effd700 1 -- 172.21.15.33:0/1853943885 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v261) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc660018020 con 0x7fc6680aff90 2023-10-06T05:33:54.915 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:33:54.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc654066f30 msgr2=0x7fc6540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc654066f30 0x7fc6540693e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc658005920 tx=0x7fc658009040 comp rx=0 tx=0).stop 2023-10-06T05:33:54.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 msgr2=0x7fc66816a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:54.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 0x7fc66816a360 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fc660009e90 tx=0x7fc66000b040 comp rx=0 tx=0).stop 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 shutdown_connections 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc654066f30 0x7fc6540693e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6680aff90 0x7fc66816a360 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc66806c7b0 0x7fc6681699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 --2- 172.21.15.33:0/1853943885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc66806cb80 0x7fc668158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 >> 172.21.15.33:0/1853943885 conn(0x7fc66806d670 msgr2=0x7fc6680ab180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 shutdown_connections 2023-10-06T05:33:54.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:54.916+0000 7fc667fff700 1 -- 172.21.15.33:0/1853943885 wait complete. 2023-10-06T05:33:54.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:33:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.121+0000 7fd0fab9d700 1 Processor -- start 2023-10-06T05:33:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.121+0000 7fd0fab9d700 1 -- start start 2023-10-06T05:33:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fab9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0f406df70 0x7fd0f40afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fab9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f406e3e0 con 0x7fd0f40b0310 2023-10-06T05:33:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fab9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f40b1e00 con 0x7fd0f406df70 2023-10-06T05:33:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f406f6b0 con 0x7fd0f4159530 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fa39c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd0f40b0310 0x7fd0f4159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48776/0 (socket says 172.21.15.33:48776) 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0f939a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0f406df70 0x7fd0f40afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.122+0000 7fd0fa39c700 1 -- 172.21.15.33:0/1194708376 learned_addr learned my addr 172.21.15.33:0/1194708376 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f9b9b700 1 --2- 172.21.15.33:0/1194708376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f939a700 1 -- 172.21.15.33:0/1194708376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 msgr2=0x7fd0f406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f8b99700 1 -- 172.21.15.33:0/1194708376 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76714650 0 0) 0x7fd0f406e3e0 con 0x7fd0f40b0310 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f939a700 1 --2- 172.21.15.33:0/1194708376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f939a700 1 -- 172.21.15.33:0/1194708376 >> v1:172.21.15.33:6789/0 conn(0x7fd0f40b0310 legacy=0x7fd0f4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f939a700 1 -- 172.21.15.33:0/1194708376 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f406f220 con 0x7fd0f406df70 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f939a700 1 --2- 172.21.15.33:0/1194708376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40afdd0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40098e0 tx=0x7fd0e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b10c6dd68d59ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.123+0000 7fd0f8b99700 1 -- 172.21.15.33:0/1194708376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4009e80 con 0x7fd0f406df70 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0f8b99700 1 -- 172.21.15.33:0/1194708376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e400b860 con 0x7fd0f406df70 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0f8b99700 1 -- 172.21.15.33:0/1194708376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400bbe0 con 0x7fd0f406df70 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1194708376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 msgr2=0x7fd0f40afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1194708376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40afdd0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40098e0 tx=0x7fd0e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1194708376 shutdown_connections 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1194708376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1194708376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40afdd0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1194708376 >> 172.21.15.33:0/1194708376 conn(0x7fd0f406c560 msgr2=0x7fd0f406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1194708376 shutdown_connections 2023-10-06T05:33:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.124+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1194708376 wait complete. 2023-10-06T05:33:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.126+0000 7fd0fab9d700 1 Processor -- start 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.126+0000 7fd0fab9d700 1 -- start start 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fab9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f40b0310 0x7fd0f40cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fab9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f41692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fab9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40cb820 con 0x7fd0f406df70 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40cb0e0 con 0x7fd0f40b0310 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fab9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4169da0 con 0x7fd0f4159530 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0f9b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0f9b9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47654/0 (socket says 172.21.15.33:47654) 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0f9b9b700 1 -- 172.21.15.33:0/1713897491 learned_addr learned my addr 172.21.15.33:0/1713897491 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0fa39c700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f41692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.127+0000 7fd0f939a700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f40b0310 0x7fd0f40cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0f9b9b700 1 -- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 msgr2=0x7fd0f41692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0f9b9b700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f41692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0f9b9b700 1 -- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f40b0310 msgr2=0x7fd0f40cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0f9b9b700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f40b0310 0x7fd0f40cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0f9b9b700 1 -- 172.21.15.33:0/1713897491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e4009420 con 0x7fd0f406df70 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0fa39c700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f41692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0f9b9b700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40cdfd0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000d890 tx=0x7fd0f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00125f0 con 0x7fd0f406df70 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0012750 con 0x7fd0f406df70 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0012ad0 con 0x7fd0f406df70 2023-10-06T05:33:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f416a080 con 0x7fd0f406df70 2023-10-06T05:33:55.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.128+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f40cbd50 con 0x7fd0f406df70 2023-10-06T05:33:55.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.129+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f40cc0c0 con 0x7fd0f406df70 2023-10-06T05:33:55.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.130+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0f001e460 con 0x7fd0f406df70 2023-10-06T05:33:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.131+0000 7fd0e9ffb700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0d8066f30 0x7fd0d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.131+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd0f0084a00 con 0x7fd0f406df70 2023-10-06T05:33:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.131+0000 7fd0f939a700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0d8066f30 0x7fd0d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.132+0000 7fd0f939a700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0d8066f30 0x7fd0d80693e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40091e0 tx=0x7fd0e4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:55.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.136+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0f0051dc0 con 0x7fd0f406df70 2023-10-06T05:33:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.341+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd0f404c380 con 0x7fd0f406df70 2023-10-06T05:33:55.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.342+0000 7fd0e9ffb700 1 -- 172.21.15.33:0/1713897491 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v261) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd0f00553e0 con 0x7fd0f406df70 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:33:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:33:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:33:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:33:55.680 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:33:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0d8066f30 msgr2=0x7fd0d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0d8066f30 0x7fd0d80693e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40091e0 tx=0x7fd0e4017000 comp rx=0 tx=0).stop 2023-10-06T05:33:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 msgr2=0x7fd0f40cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40cdfd0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000d890 tx=0x7fd0f0008040 comp rx=0 tx=0).stop 2023-10-06T05:33:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 shutdown_connections 2023-10-06T05:33:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd0d8066f30 0x7fd0d80693e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0f4159530 0x7fd0f41692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0f406df70 0x7fd0f40cdfd0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 --2- 172.21.15.33:0/1713897491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0f40b0310 0x7fd0f40cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 >> 172.21.15.33:0/1713897491 conn(0x7fd0f406c560 msgr2=0x7fd0f40af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 shutdown_connections 2023-10-06T05:33:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.344+0000 7fd0fab9d700 1 -- 172.21.15.33:0/1713897491 wait complete. 2023-10-06T05:33:55.683 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:33:55.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:33:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.876+0000 7f04a9f9d700 1 Processor -- start 2023-10-06T05:33:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.876+0000 7f04a9f9d700 1 -- start start 2023-10-06T05:33:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.877+0000 7f04a9f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.877+0000 7f04a9f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04a406dfd0 0x7f04a415a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.877+0000 7f04a9f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a415d280 con 0x7f04a406dc00 2023-10-06T05:33:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.877+0000 7f04a9f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a415e290 con 0x7f04a406dfd0 2023-10-06T05:33:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.877+0000 7f04a9f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415de00 con 0x7f04a41597f0 2023-10-06T05:33:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04a406dfd0 0x7f04a415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04a406dfd0 0x7f04a415a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47662/0 (socket says 172.21.15.33:47662) 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 -- 172.21.15.33:0/3128944440 learned_addr learned my addr 172.21.15.33:0/3128944440 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a3fff700 1 --2- 172.21.15.33:0/3128944440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 -- 172.21.15.33:0/3128944440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 msgr2=0x7f04a415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 --2- 172.21.15.33:0/3128944440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 -- 172.21.15.33:0/3128944440 >> v1:172.21.15.167:6789/0 conn(0x7f04a406dc00 legacy=0x7f04a41590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 -- 172.21.15.33:0/3128944440 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415d4d0 con 0x7f04a406dfd0 2023-10-06T05:33:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a37fe700 1 -- 172.21.15.33:0/3128944440 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924749265 0 0) 0x7f04a415d280 con 0x7f04a406dc00 2023-10-06T05:33:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.878+0000 7f04a979c700 1 --2- 172.21.15.33:0/3128944440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dfd0 0x7f04a415a570 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f049c009f20 tx=0x7f049c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8130886f2e2e69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a37fe700 1 -- 172.21.15.33:0/3128944440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c00e070 con 0x7f04a406dfd0 2023-10-06T05:33:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a37fe700 1 -- 172.21.15.33:0/3128944440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c00b9d0 con 0x7f04a406dfd0 2023-10-06T05:33:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a37fe700 1 -- 172.21.15.33:0/3128944440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0092e0 con 0x7f04a406dfd0 2023-10-06T05:33:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3128944440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dfd0 msgr2=0x7f04a415a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3128944440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dfd0 0x7f04a415a570 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f049c009f20 tx=0x7f049c009840 comp rx=0 tx=0).stop 2023-10-06T05:33:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3128944440 shutdown_connections 2023-10-06T05:33:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3128944440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dfd0 0x7f04a415a570 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3128944440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3128944440 >> 172.21.15.33:0/3128944440 conn(0x7f04a40d0550 msgr2=0x7f04a40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3128944440 shutdown_connections 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.879+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3128944440 wait complete. 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.881+0000 7f04a9f9d700 1 Processor -- start 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.881+0000 7f04a9f9d700 1 -- start start 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a9f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 0x7f04a40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a9f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a406dfd0 0x7f04a416a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a9f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a40c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a9f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4169850 con 0x7f04a406dc00 2023-10-06T05:33:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a9f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c7cf0 con 0x7f04a406dfd0 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a9f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a40c7730 con 0x7f04a41597f0 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 0x7f04a40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 0x7f04a40ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47674/0 (socket says 172.21.15.33:47674) 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 -- 172.21.15.33:0/3686510360 learned_addr learned my addr 172.21.15.33:0/3686510360 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a979c700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a40c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a3fff700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a406dfd0 0x7f04a416a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 -- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 msgr2=0x7f04a40c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a40c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 -- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a406dfd0 msgr2=0x7f04a416a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a406dfd0 0x7f04a416a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.882+0000 7f04a8f9b700 1 -- 172.21.15.33:0/3686510360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c009ae0 con 0x7f04a406dc00 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.883+0000 7f04a8f9b700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 0x7f04a40ca040 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f049800c7e0 tx=0x7f0498007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.883+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498012700 con 0x7f04a406dc00 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.883+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498012880 con 0x7f04a406dc00 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.883+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a40c7a10 con 0x7f04a406dc00 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.883+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498012c40 con 0x7f04a406dc00 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.883+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a4169d10 con 0x7f04a406dc00 2023-10-06T05:33:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.884+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a4063ff0 con 0x7f04a406dc00 2023-10-06T05:33:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.885+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f049801b400 con 0x7f04a406dc00 2023-10-06T05:33:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.886+0000 7f04a0ff9700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0488066f30 0x7f04880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.886+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0498083fe0 con 0x7f04a406dc00 2023-10-06T05:33:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.886+0000 7f04a3fff700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0488066f30 0x7f04880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.887+0000 7f04a3fff700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0488066f30 0x7f04880693e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f04a415d390 tx=0x7f0490008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:55.890+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04980513a0 con 0x7f04a406dc00 2023-10-06T05:33:56.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.095+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04a415d7b0 con 0x7f04a406dc00 2023-10-06T05:33:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.194+0000 7f04a0ff9700 1 -- 172.21.15.33:0/3686510360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v262) v1 ==== 138+0+0 (secure 0 0 0) 0x7f04980549c0 con 0x7f04a406dc00 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.200+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0488066f30 msgr2=0x7f04880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.200+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0488066f30 0x7f04880693e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f04a415d390 tx=0x7f0490008040 comp rx=0 tx=0).stop 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.200+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 msgr2=0x7f04a40ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.200+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 0x7f04a40ca040 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f049800c7e0 tx=0x7f0498007df0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 shutdown_connections 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0488066f30 0x7f04880693e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04a41597f0 0x7f04a40c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04a406dc00 0x7f04a40ca040 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 --2- 172.21.15.33:0/3686510360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a406dfd0 0x7f04a416a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 >> 172.21.15.33:0/3686510360 conn(0x7f04a40d0550 msgr2=0x7f04a415a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 shutdown_connections 2023-10-06T05:33:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.201+0000 7f04a9f9d700 1 -- 172.21.15.33:0/3686510360 wait complete. 2023-10-06T05:33:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:33:56.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:33:56.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.414+0000 7f170db9d700 1 Processor -- start 2023-10-06T05:33:56.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.415+0000 7f170db9d700 1 -- start start 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170db9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17080b0740 0x7f170815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f170806efa0 0x7f17080abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170db9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170815a5f0 con 0x7f17080cea20 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170db9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17080cf170 con 0x7f17080b0740 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1708159fa0 con 0x7f170806efa0 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170cb9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f17080b0740 0x7f170815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f1707fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f17080cea20 0x7f170806e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48780/0 (socket says 172.21.15.33:48780) 2023-10-06T05:33:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f1707fff700 1 -- 172.21.15.33:0/2443466904 learned_addr learned my addr 172.21.15.33:0/2443466904 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:56.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.416+0000 7f170d39c700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f170806efa0 0x7f17080abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:56.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f170cb9b700 1 -- 172.21.15.33:0/2443466904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f170806efa0 msgr2=0x7f17080abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:56.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f170cb9b700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f170806efa0 0x7f17080abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f170cb9b700 1 -- 172.21.15.33:0/2443466904 >> v1:172.21.15.33:6789/0 conn(0x7f17080cea20 legacy=0x7f170806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:56.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f170cb9b700 1 -- 172.21.15.33:0/2443466904 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170815a280 con 0x7f17080b0740 2023-10-06T05:33:56.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f170d39c700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f170806efa0 0x7f17080abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:56.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f17077fe700 1 -- 172.21.15.33:0/2443466904 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038210418 0 0) 0x7f170815a5f0 con 0x7f17080cea20 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f170cb9b700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b0740 0x7f170815dd10 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0098e0 tx=0x7f16fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd017b6e64a0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f17077fe700 1 -- 172.21.15.33:0/2443466904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc009e80 con 0x7f17080b0740 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f17077fe700 1 -- 172.21.15.33:0/2443466904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00b860 con 0x7f17080b0740 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.417+0000 7f17077fe700 1 -- 172.21.15.33:0/2443466904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00bbe0 con 0x7f17080b0740 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 -- 172.21.15.33:0/2443466904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b0740 msgr2=0x7f170815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b0740 0x7f170815dd10 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0098e0 tx=0x7f16fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 -- 172.21.15.33:0/2443466904 shutdown_connections 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17080b0740 0x7f170815dd10 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 --2- 172.21.15.33:0/2443466904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f170806efa0 0x7f17080abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 -- 172.21.15.33:0/2443466904 >> 172.21.15.33:0/2443466904 conn(0x7f170806cd40 msgr2=0x7f17080ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 -- 172.21.15.33:0/2443466904 shutdown_connections 2023-10-06T05:33:56.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.418+0000 7f170db9d700 1 -- 172.21.15.33:0/2443466904 wait complete. 2023-10-06T05:33:56.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.419+0000 7f170db9d700 1 Processor -- start 2023-10-06T05:33:56.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.420+0000 7f170db9d700 1 -- start start 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.420+0000 7f170db9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 0x7f17080cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.420+0000 7f170db9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17080b0740 0x7f170814add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170db9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17080cea20 0x7f170816a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170db9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080c9020 con 0x7f170806efa0 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170db9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080cca00 con 0x7f17080cea20 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170db9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080ccce0 con 0x7f17080b0740 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 0x7f17080cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 0x7f17080cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47694/0 (socket says 172.21.15.33:47694) 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 -- 172.21.15.33:0/4199495914 learned_addr learned my addr 172.21.15.33:0/4199495914 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f1707fff700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17080b0740 0x7f170814add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170d39c700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17080cea20 0x7f170816a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 -- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17080b0740 msgr2=0x7f170814add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17080b0740 0x7f170814add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 -- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17080cea20 msgr2=0x7f170816a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17080cea20 0x7f170816a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.421+0000 7f170cb9b700 1 -- 172.21.15.33:0/4199495914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009420 con 0x7f170806efa0 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.422+0000 7f170cb9b700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 0x7f17080cc420 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f1708065080 tx=0x7f16fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.422+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00b9d0 con 0x7f170806efa0 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.422+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00bb30 con 0x7f170806efa0 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.422+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc011670 con 0x7f170806efa0 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.422+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17080ccfc0 con 0x7f170806efa0 2023-10-06T05:33:56.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.422+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170816aab0 con 0x7f170806efa0 2023-10-06T05:33:56.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.423+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17080c9160 con 0x7f170806efa0 2023-10-06T05:33:56.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.424+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16fc011a60 con 0x7f170806efa0 2023-10-06T05:33:56.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.425+0000 7f1704ff9700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16ec066f30 0x7f16ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:56.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.425+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f16fc085a00 con 0x7f170806efa0 2023-10-06T05:33:56.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.425+0000 7f1707fff700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16ec066f30 0x7f16ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:56.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.426+0000 7f1707fff700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16ec066f30 0x7f16ec0693e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f170816d0a0 tx=0x7f16f4007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:56.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.430+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16fc052cd0 con 0x7f170806efa0 2023-10-06T05:33:56.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:56.635+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f170804c380 con 0x7f170806efa0 2023-10-06T05:33:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.199+0000 7f1704ff9700 1 -- 172.21.15.33:0/4199495914 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v263) v1 ==== 170+0+0 (secure 0 0 0) 0x7f16fc0562f0 con 0x7f170806efa0 2023-10-06T05:33:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.207+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16ec066f30 msgr2=0x7f16ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.207+0000 7f170db9d700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16ec066f30 0x7f16ec0693e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f170816d0a0 tx=0x7f16f4007f50 comp rx=0 tx=0).stop 2023-10-06T05:33:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.207+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 msgr2=0x7f17080cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.207+0000 7f170db9d700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 0x7f17080cc420 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f1708065080 tx=0x7f16fc017000 comp rx=0 tx=0).stop 2023-10-06T05:33:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.209+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 shutdown_connections 2023-10-06T05:33:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.209+0000 7f170db9d700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f16ec066f30 0x7f16ec0693e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.209+0000 7f170db9d700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17080cea20 0x7f170816a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.209+0000 7f170db9d700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170806efa0 0x7f17080cc420 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.209+0000 7f170db9d700 1 --2- 172.21.15.33:0/4199495914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17080b0740 0x7f170814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.209+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 >> 172.21.15.33:0/4199495914 conn(0x7f170806cd40 msgr2=0x7f170806d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.215+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 shutdown_connections 2023-10-06T05:33:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.217+0000 7f170db9d700 1 -- 172.21.15.33:0/4199495914 wait complete. 2023-10-06T05:33:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:33:57.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:33:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.431+0000 7f4e9119d700 1 Processor -- start 2023-10-06T05:33:57.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.431+0000 7f4e9119d700 1 -- start start 2023-10-06T05:33:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.432+0000 7f4e9119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c0af540 0x7f4e8c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.432+0000 7f4e9119d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e8c15a650 0x7f4e8c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.432+0000 7f4e9119d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c15d660 con 0x7f4e8c0cced0 2023-10-06T05:33:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.432+0000 7f4e9119d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c0ce890 con 0x7f4e8c15a650 2023-10-06T05:33:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.432+0000 7f4e9119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c15aac0 con 0x7f4e8c0af540 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e9099c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4e8c0cced0 0x7f4e8c06c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48796/0 (socket says 172.21.15.33:48796) 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e9099c700 1 -- 172.21.15.33:0/2592423469 learned_addr learned my addr 172.21.15.33:0/2592423469 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8bfff700 1 --2- 172.21.15.33:0/2592423469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c0af540 0x7f4e8c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8b7fe700 1 --2- 172.21.15.33:0/2592423469 >> v2:172.21.15.167:3300/0 conn(0x7f4e8c15a650 0x7f4e8c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8affd700 1 -- 172.21.15.33:0/2592423469 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921090989 0 0) 0x7f4e8c15d660 con 0x7f4e8c0cced0 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8affd700 1 -- 172.21.15.33:0/2592423469 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e74003570 con 0x7f4e8c0cced0 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2592423469 >> v2:172.21.15.167:3300/0 conn(0x7f4e8c15a650 msgr2=0x7f4e8c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8bfff700 1 --2- 172.21.15.33:0/2592423469 >> v2:172.21.15.167:3300/0 conn(0x7f4e8c15a650 0x7f4e8c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2592423469 >> v1:172.21.15.33:6789/0 conn(0x7f4e8c0cced0 legacy=0x7f4e8c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2592423469 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c15d890 con 0x7f4e8c0af540 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.433+0000 7f4e8b7fe700 1 --2- 172.21.15.33:0/2592423469 >> v2:172.21.15.167:3300/0 conn(0x7f4e8c15a650 0x7f4e8c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:33:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.434+0000 7f4e8bfff700 1 --2- 172.21.15.33:0/2592423469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c0af540 0x7f4e8c0af910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0098e0 tx=0x7f4e7c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8011cc6220bd893d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.434+0000 7f4e8affd700 1 -- 172.21.15.33:0/2592423469 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c011630 con 0x7f4e8c0af540 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.434+0000 7f4e8affd700 1 -- 172.21.15.33:0/2592423469 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c011790 con 0x7f4e8c0af540 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.434+0000 7f4e8affd700 1 -- 172.21.15.33:0/2592423469 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c011b30 con 0x7f4e8c0af540 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 -- 172.21.15.33:0/2592423469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c0af540 msgr2=0x7f4e8c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2592423469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c0af540 0x7f4e8c0af910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0098e0 tx=0x7f4e7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 -- 172.21.15.33:0/2592423469 shutdown_connections 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2592423469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c0af540 0x7f4e8c0af910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2592423469 >> v2:172.21.15.167:3300/0 conn(0x7f4e8c15a650 0x7f4e8c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 -- 172.21.15.33:0/2592423469 >> 172.21.15.33:0/2592423469 conn(0x7f4e8c06d9b0 msgr2=0x7f4e8c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 -- 172.21.15.33:0/2592423469 shutdown_connections 2023-10-06T05:33:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.435+0000 7f4e9119d700 1 -- 172.21.15.33:0/2592423469 wait complete. 2023-10-06T05:33:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.436+0000 7f4e9119d700 1 Processor -- start 2023-10-06T05:33:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.436+0000 7f4e9119d700 1 -- start start 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e9119d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 0x7f4e8c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e9119d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c0cced0 0x7f4e8c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e9119d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c15a650 0x7f4e8c147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e9119d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0ca200 con 0x7f4e8c0af540 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e9119d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0b6320 con 0x7f4e8c0cced0 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e9119d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0b5e90 con 0x7f4e8c15a650 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.437+0000 7f4e8bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 0x7f4e8c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 0x7f4e8c0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47704/0 (socket says 172.21.15.33:47704) 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2680171670 learned_addr learned my addr 172.21.15.33:0/2680171670 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8b7fe700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c0cced0 0x7f4e8c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e9099c700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c15a650 0x7f4e8c147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c15a650 msgr2=0x7f4e8c147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c15a650 0x7f4e8c147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c0cced0 msgr2=0x7f4e8c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c0cced0 0x7f4e8c0b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 -- 172.21.15.33:0/2680171670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80009420 con 0x7f4e8c0af540 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8b7fe700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c0cced0 0x7f4e8c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e8bfff700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 0x7f4e8c0b4c80 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009ed0 tx=0x7f4e7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.438+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c011b10 con 0x7f4e8c0af540 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.439+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0253f0 con 0x7f4e8c0af540 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.439+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c025790 con 0x7f4e8c0af540 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.439+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c009420 con 0x7f4e8c0af540 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.439+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c0b5560 con 0x7f4e8c0af540 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.440+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c157180 con 0x7f4e8c0af540 2023-10-06T05:33:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.442+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e7c025b40 con 0x7f4e8c0af540 2023-10-06T05:33:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.442+0000 7f4e73fff700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.443+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f4e7c0866d0 con 0x7f4e8c0af540 2023-10-06T05:33:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.443+0000 7f4e8b7fe700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.444+0000 7f4e8b7fe700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4e800093f0 tx=0x7f4e80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.446+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e7c053820 con 0x7f4e8c0af540 2023-10-06T05:33:57.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:57.696+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4e8c04c380 con 0x7f4e8c0af540 2023-10-06T05:33:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.229+0000 7f4e73fff700 1 -- 172.21.15.33:0/2680171670 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 39 and data pool 40 v200) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4e7c0150c0 con 0x7f4e8c0af540 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e78066f30 msgr2=0x7f4e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4e800093f0 tx=0x7f4e80008040 comp rx=0 tx=0).stop 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 msgr2=0x7f4e8c0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 0x7f4e8c0b4c80 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009ed0 tx=0x7f4e7c017000 comp rx=0 tx=0).stop 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 shutdown_connections 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4e800093f0 tx=0x7f4e80008040 comp rx=0 tx=0).stop 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c15a650 0x7f4e8c147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e8c0af540 0x7f4e8c0b4c80 secure :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009ed0 tx=0x7f4e7c017000 comp rx=0 tx=0).stop 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 --2- 172.21.15.33:0/2680171670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c0cced0 0x7f4e8c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.236+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 >> 172.21.15.33:0/2680171670 conn(0x7f4e8c06d9b0 msgr2=0x7f4e8c0caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.237+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 shutdown_connections 2023-10-06T05:33:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.238+0000 7f4e9119d700 1 -- 172.21.15.33:0/2680171670 wait complete. 2023-10-06T05:33:58.243 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 39 and data pool 40 2023-10-06T05:33:58.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:33:58.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.457+0000 7fc3fb59e700 1 Processor -- start 2023-10-06T05:33:58.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.457+0000 7fc3fb59e700 1 -- start start 2023-10-06T05:33:58.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.458+0000 7fc3fb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3fc0cf3b0 0x7fc3fc1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.458+0000 7fc3fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0af0a0 0x7fc3fc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.458+0000 7fc3fb59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc0cbd80 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.458+0000 7fc3fb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0af5f0 con 0x7fc3fc0cf3b0 2023-10-06T05:33:58.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.458+0000 7fc3fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0af8d0 con 0x7fc3fc0af0a0 2023-10-06T05:33:58.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc3fc0cf3b0 0x7fc3fc1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3fa59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc3fc0ccfc0 0x7fc3fc0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48800/0 (socket says 172.21.15.33:48800) 2023-10-06T05:33:58.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3fa59c700 1 -- 172.21.15.33:0/902477038 learned_addr learned my addr 172.21.15.33:0/902477038 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3fad9d700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0af0a0 0x7fc3fc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/902477038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0af0a0 msgr2=0x7fc3fc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f9d9b700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0af0a0 0x7fc3fc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/902477038 >> v1:172.21.15.33:6789/0 conn(0x7fc3fc0ccfc0 legacy=0x7fc3fc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/902477038 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0afbb0 con 0x7fc3fc0cf3b0 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3fad9d700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0af0a0 0x7fc3fc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f959a700 1 -- 172.21.15.33:0/902477038 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679391097 0 0) 0x7fc3fc0cbd80 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.459+0000 7fc3f9d9b700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0cf3b0 0x7fc3fc1594a0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009910 tx=0x7fc3f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca6e69f3043ae3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3f959a700 1 -- 172.21.15.33:0/902477038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f000b820 con 0x7fc3fc0cf3b0 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3f959a700 1 -- 172.21.15.33:0/902477038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f000b980 con 0x7fc3fc0cf3b0 2023-10-06T05:33:58.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3f959a700 1 -- 172.21.15.33:0/902477038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f000bd20 con 0x7fc3fc0cf3b0 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 -- 172.21.15.33:0/902477038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0cf3b0 msgr2=0x7fc3fc1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0cf3b0 0x7fc3fc1594a0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009910 tx=0x7fc3f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 -- 172.21.15.33:0/902477038 shutdown_connections 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0af0a0 0x7fc3fc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/902477038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0cf3b0 0x7fc3fc1594a0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 -- 172.21.15.33:0/902477038 >> 172.21.15.33:0/902477038 conn(0x7fc3fc06cff0 msgr2=0x7fc3fc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 -- 172.21.15.33:0/902477038 shutdown_connections 2023-10-06T05:33:58.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.460+0000 7fc3fb59e700 1 -- 172.21.15.33:0/902477038 wait complete. 2023-10-06T05:33:58.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.462+0000 7fc3fb59e700 1 Processor -- start 2023-10-06T05:33:58.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.462+0000 7fc3fb59e700 1 -- start start 2023-10-06T05:33:58.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3fc0af0a0 0x7fc3fc0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 0x7fc3fc0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0b5450 0x7fc3fc0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0b4ab0 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0b37e0 con 0x7fc3fc0af0a0 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0b4620 con 0x7fc3fc0b5450 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 0x7fc3fc0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 0x7fc3fc0b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47720/0 (socket says 172.21.15.33:47720) 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/3779465360 learned_addr learned my addr 172.21.15.33:0/3779465360 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fad9d700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0b5450 0x7fc3fc0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.463+0000 7fc3fa59c700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3fc0af0a0 0x7fc3fc0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0b5450 msgr2=0x7fc3fc0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3f9d9b700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0b5450 0x7fc3fc0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3fc0af0a0 msgr2=0x7fc3fc0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3f9d9b700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3fc0af0a0 0x7fc3fc0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3f9d9b700 1 -- 172.21.15.33:0/3779465360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f0009420 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3f9d9b700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 0x7fc3fc0b32a0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00091e0 tx=0x7fc3f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0011e90 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f0024440 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc0b3a10 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00247e0 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.464+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc0b3fd0 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.465+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc0b47b0 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.466+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3f0024b90 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.467+0000 7fc3ea7fc700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3dc066f30 0x7fc3dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.467+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc3f0086430 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.468+0000 7fc3fa59c700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3dc066f30 0x7fc3dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.468+0000 7fc3fa59c700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3dc066f30 0x7fc3dc0693e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0011e0 tx=0x7fc3ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.472+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3f00534a0 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.674+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3fc04c380 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.675+0000 7fc3ea7fc700 1 -- 172.21.15.33:0/3779465360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v264) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fc3f0056ac0 con 0x7fc3fc0ccfc0 2023-10-06T05:33:58.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:58.677 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":264,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:57.697604+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:33:56.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":40,"pool_name":"cephfs_data","create_time":"2023-10-06T05:33:56.676003+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:33:58.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3dc066f30 msgr2=0x7fc3dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3dc066f30 0x7fc3dc0693e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0011e0 tx=0x7fc3ec009040 comp rx=0 tx=0).stop 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 msgr2=0x7fc3fc0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 0x7fc3fc0b32a0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00091e0 tx=0x7fc3f0017000 comp rx=0 tx=0).stop 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 shutdown_connections 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc3dc066f30 0x7fc3dc0693e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3fc0b5450 0x7fc3fc0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3fc0af0a0 0x7fc3fc0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 --2- 172.21.15.33:0/3779465360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3fc0ccfc0 0x7fc3fc0b32a0 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 >> 172.21.15.33:0/3779465360 conn(0x7fc3fc06cff0 msgr2=0x7fc3fc1571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 shutdown_connections 2023-10-06T05:33:58.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.678+0000 7fc3fb59e700 1 -- 172.21.15.33:0/3779465360 wait complete. 2023-10-06T05:33:58.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:33:58.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.882+0000 7fd720d9d700 1 Processor -- start 2023-10-06T05:33:58.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.882+0000 7fd720d9d700 1 -- start start 2023-10-06T05:33:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.883+0000 7fd720d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b0a40 0x7fd71c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.883+0000 7fd720d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd71c0b0e10 0x7fd71c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.883+0000 7fd720d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c15e2d0 con 0x7fd71c0b2e30 2023-10-06T05:33:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.883+0000 7fd720d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c15d6c0 con 0x7fd71c0b0e10 2023-10-06T05:33:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.883+0000 7fd720d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c15d9a0 con 0x7fd71c0b0a40 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.883+0000 7fd71affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd71c0b0e10 0x7fd71c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd71c0b0e10 0x7fd71c0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47732/0 (socket says 172.21.15.33:47732) 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 -- 172.21.15.33:0/3946502571 learned_addr learned my addr 172.21.15.33:0/3946502571 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71b7fe700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b0a40 0x7fd71c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 -- 172.21.15.33:0/3946502571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b0a40 msgr2=0x7fd71c06ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b0a40 0x7fd71c06ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 -- 172.21.15.33:0/3946502571 >> v1:172.21.15.167:6789/0 conn(0x7fd71c0b2e30 legacy=0x7fd71c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 -- 172.21.15.33:0/3946502571 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c15dc80 con 0x7fd71c0b0e10 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71b7fe700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b0a40 0x7fd71c06ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.884+0000 7fd71affd700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c0b28f0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fd7100099b0 tx=0x7fd710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd3876f5d749b92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd71a7fc700 1 -- 172.21.15.33:0/3946502571 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71000b860 con 0x7fd71c0b0e10 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd71a7fc700 1 -- 172.21.15.33:0/3946502571 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71000b9c0 con 0x7fd71c0b0e10 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd71a7fc700 1 -- 172.21.15.33:0/3946502571 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71000bd60 con 0x7fd71c0b0e10 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 -- 172.21.15.33:0/3946502571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 msgr2=0x7fd71c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c0b28f0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fd7100099b0 tx=0x7fd710009250 comp rx=0 tx=0).stop 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 -- 172.21.15.33:0/3946502571 shutdown_connections 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b0a40 0x7fd71c06ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 --2- 172.21.15.33:0/3946502571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c0b28f0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 -- 172.21.15.33:0/3946502571 >> 172.21.15.33:0/3946502571 conn(0x7fd71c06e0f0 msgr2=0x7fd71c0675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 -- 172.21.15.33:0/3946502571 shutdown_connections 2023-10-06T05:33:58.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.885+0000 7fd720d9d700 1 -- 172.21.15.33:0/3946502571 wait complete. 2023-10-06T05:33:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.887+0000 7fd720d9d700 1 Processor -- start 2023-10-06T05:33:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.887+0000 7fd720d9d700 1 -- start start 2023-10-06T05:33:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd720d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0b0a40 0x7fd71c142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd720d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd720d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b2e30 0x7fd71c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd720d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c15c780 con 0x7fd71c0b0e10 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd720d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c142f70 con 0x7fd71c0b0a40 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd720d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c15bb70 con 0x7fd71c0b2e30 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd71affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd71affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47740/0 (socket says 172.21.15.33:47740) 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd71affd700 1 -- 172.21.15.33:0/711361708 learned_addr learned my addr 172.21.15.33:0/711361708 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd71bfff700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b2e30 0x7fd71c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.888+0000 7fd71b7fe700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0b0a40 0x7fd71c142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd71affd700 1 -- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b2e30 msgr2=0x7fd71c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd71affd700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b2e30 0x7fd71c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd71affd700 1 -- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0b0a40 msgr2=0x7fd71c142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd71affd700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0b0a40 0x7fd71c142a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:58.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd71affd700 1 -- 172.21.15.33:0/711361708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100094f0 con 0x7fd71c0b0e10 2023-10-06T05:33:58.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd71affd700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c155bd0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fd7100090b0 tx=0x7fd710017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710011dc0 con 0x7fd71c0b0e10 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71001f070 con 0x7fd71c0b0e10 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7100266d0 con 0x7fd71c0b0e10 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c15be50 con 0x7fd71c0b0e10 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.889+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c15c410 con 0x7fd71c0b0e10 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.890+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c06fcc0 con 0x7fd71c0b0e10 2023-10-06T05:33:58.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.891+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd710026a40 con 0x7fd71c0b0e10 2023-10-06T05:33:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.892+0000 7fd7037fe700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd708066f30 0x7fd7080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.892+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd710021070 con 0x7fd71c0b0e10 2023-10-06T05:33:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.892+0000 7fd71b7fe700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd708066f30 0x7fd7080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:58.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.893+0000 7fd71b7fe700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd708066f30 0x7fd7080693e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd70c006f00 tx=0x7fd70c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:58.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:58.896+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd710053510 con 0x7fd71c0b0e10 2023-10-06T05:33:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.101+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd71c04c380 con 0x7fd71c0b0e10 2023-10-06T05:33:59.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.102+0000 7fd7037fe700 1 -- 172.21.15.33:0/711361708 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v264) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fd710056b30 con 0x7fd71c0b0e10 2023-10-06T05:33:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:33:59.104 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":264,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:57.697604+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:33:56.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":40,"pool_name":"cephfs_data","create_time":"2023-10-06T05:33:56.676003+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:33:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd708066f30 msgr2=0x7fd7080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:59.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd708066f30 0x7fd7080693e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd70c006f00 tx=0x7fd70c008040 comp rx=0 tx=0).stop 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 msgr2=0x7fd71c155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c155bd0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fd7100090b0 tx=0x7fd710017000 comp rx=0 tx=0).stop 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 shutdown_connections 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd708066f30 0x7fd7080693e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c0b2e30 0x7fd71c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.104+0000 7fd720d9d700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0b0a40 0x7fd71c142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.105+0000 7fd720d9d700 1 --2- 172.21.15.33:0/711361708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd71c0b0e10 0x7fd71c155bd0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.105+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 >> 172.21.15.33:0/711361708 conn(0x7fd71c06e0f0 msgr2=0x7fd71c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.105+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 shutdown_connections 2023-10-06T05:33:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.105+0000 7fd720d9d700 1 -- 172.21.15.33:0/711361708 wait complete. 2023-10-06T05:33:59.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:33:59.130 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:33:59.128+0000 7fe783bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:33:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.318+0000 7f8cf8f9d700 1 Processor -- start 2023-10-06T05:33:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.318+0000 7f8cf8f9d700 1 -- start start 2023-10-06T05:33:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.319+0000 7f8cf8f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8cf4156370 0x7f8cf40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.319+0000 7f8cf8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf40cc7e0 0x7f8cf40c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.319+0000 7f8cf8f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf40ad430 con 0x7f8cf40b01f0 2023-10-06T05:33:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.319+0000 7f8cf8f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf40afe20 con 0x7f8cf4156370 2023-10-06T05:33:59.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.319+0000 7f8cf8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf406bc70 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8cf4156370 0x7f8cf40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:59.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8cf4156370 0x7f8cf40af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47744/0 (socket says 172.21.15.33:47744) 2023-10-06T05:33:59.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 -- 172.21.15.33:0/2711173624 learned_addr learned my addr 172.21.15.33:0/2711173624 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:59.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf2ffd700 1 --2- 172.21.15.33:0/2711173624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf40cc7e0 0x7f8cf40c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:59.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 -- 172.21.15.33:0/2711173624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf40cc7e0 msgr2=0x7f8cf40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:33:59.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 --2- 172.21.15.33:0/2711173624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf40cc7e0 0x7f8cf40c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 -- 172.21.15.33:0/2711173624 >> v1:172.21.15.167:6789/0 conn(0x7f8cf40b01f0 legacy=0x7f8cf40ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 -- 172.21.15.33:0/2711173624 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40c92f0 con 0x7f8cf4156370 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.320+0000 7f8cf37fe700 1 --2- 172.21.15.33:0/2711173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf4156370 0x7f8cf40af8e0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bc50 tx=0x7f8ce400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d8b950f97dbf3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf27fc700 1 -- 172.21.15.33:0/2711173624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400be60 con 0x7f8cf4156370 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf27fc700 1 -- 172.21.15.33:0/2711173624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400de00 con 0x7f8cf4156370 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf27fc700 1 -- 172.21.15.33:0/2711173624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40196b0 con 0x7f8cf4156370 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/2711173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf4156370 msgr2=0x7f8cf40af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/2711173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf4156370 0x7f8cf40af8e0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bc50 tx=0x7f8ce400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/2711173624 shutdown_connections 2023-10-06T05:33:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/2711173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf4156370 0x7f8cf40af8e0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/2711173624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf40cc7e0 0x7f8cf40c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/2711173624 >> 172.21.15.33:0/2711173624 conn(0x7f8cf406d030 msgr2=0x7f8cf40cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:33:59.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/2711173624 shutdown_connections 2023-10-06T05:33:59.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.321+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/2711173624 wait complete. 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.323+0000 7f8cf8f9d700 1 Processor -- start 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.323+0000 7f8cf8f9d700 1 -- start start 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf8f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cf40b01f0 0x7f8cf4156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf8f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 0x7f8cf414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf8f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf4156370 0x7f8cf4150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf8f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4154ab0 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf8f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41517c0 con 0x7f8cf40b01f0 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf8f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4151aa0 con 0x7f8cf4156370 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 0x7f8cf414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 0x7f8cf414f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47756/0 (socket says 172.21.15.33:47756) 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf2ffd700 1 -- 172.21.15.33:0/4172087418 learned_addr learned my addr 172.21.15.33:0/4172087418 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf37fe700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cf40b01f0 0x7f8cf4156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.324+0000 7f8cf3fff700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf4156370 0x7f8cf4150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf2ffd700 1 -- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf4156370 msgr2=0x7f8cf4150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf2ffd700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf4156370 0x7f8cf4150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf2ffd700 1 -- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cf40b01f0 msgr2=0x7f8cf4156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:33:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf2ffd700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cf40b01f0 0x7f8cf4156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf2ffd700 1 -- 172.21.15.33:0/4172087418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce400b790 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf2ffd700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 0x7f8cf414f530 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800d860 tx=0x7f8ce8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80125f0 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce8012750 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf4154740 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8012af0 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.325+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41588c0 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.326+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf41545a0 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.327+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ce8013480 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.329+0000 7f8cdb7fe700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8ce0066f30 0x7f8ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:33:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.329+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ce8084a90 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.329+0000 7f8cf37fe700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8ce0066f30 0x7f8ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:33:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.330+0000 7f8cf37fe700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8ce0066f30 0x7f8ce00693e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bc50 tx=0x7f8ce401b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:33:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.333+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cf41545a0 con 0x7f8cf40cc7e0 2023-10-06T05:33:59.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:33:59.582+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8cf41545a0 con 0x7f8cf40cc7e0 2023-10-06T05:34:00.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.235+0000 7f8cdb7fe700 1 -- 172.21.15.33:0/4172087418 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v202) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8cf41545a0 con 0x7f8cf40cc7e0 2023-10-06T05:34:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8ce0066f30 msgr2=0x7f8ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8ce0066f30 0x7f8ce00693e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bc50 tx=0x7f8ce401b000 comp rx=0 tx=0).stop 2023-10-06T05:34:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 msgr2=0x7f8cf414f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 0x7f8cf414f530 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800d860 tx=0x7f8ce8008040 comp rx=0 tx=0).stop 2023-10-06T05:34:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 shutdown_connections 2023-10-06T05:34:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8ce0066f30 0x7f8ce00693e0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8cf4156370 0x7f8cf4150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cf40b01f0 0x7f8cf4156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 --2- 172.21.15.33:0/4172087418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cf40cc7e0 0x7f8cf414f530 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 >> 172.21.15.33:0/4172087418 conn(0x7f8cf406d030 msgr2=0x7f8cf40ac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 shutdown_connections 2023-10-06T05:34:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.238+0000 7f8cf8f9d700 1 -- 172.21.15.33:0/4172087418 wait complete. 2023-10-06T05:34:00.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:34:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.454+0000 7f51c9d9d700 1 Processor -- start 2023-10-06T05:34:00.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.454+0000 7f51c9d9d700 1 -- start start 2023-10-06T05:34:00.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.455+0000 7f51c9d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51c40cd5e0 0x7f51c40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:00.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.455+0000 7f51c9d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40ce890 con 0x7f51c415a650 2023-10-06T05:34:00.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.455+0000 7f51c9d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c415aa60 con 0x7f51c40af540 2023-10-06T05:34:00.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.455+0000 7f51c9d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c415d890 con 0x7f51c40cd5e0 2023-10-06T05:34:00.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.455+0000 7f51c959c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f51c415a650 0x7f51c40ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59594/0 (socket says 172.21.15.33:59594) 2023-10-06T05:34:00.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c959c700 1 -- 172.21.15.33:0/4021154328 learned_addr learned my addr 172.21.15.33:0/4021154328 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:00.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c3fff700 1 --2- 172.21.15.33:0/4021154328 >> v2:172.21.15.167:3300/0 conn(0x7f51c40cd5e0 0x7f51c40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958003942 0 0) 0x7f51c40ce890 con 0x7f51c415a650 2023-10-06T05:34:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b4003570 con 0x7f51c415a650 2023-10-06T05:34:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35448789 0 0) 0x7f51c415aa60 con 0x7f51c40af540 2023-10-06T05:34:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c40ce890 con 0x7f51c40af540 2023-10-06T05:34:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c3fff700 1 -- 172.21.15.33:0/4021154328 >> v1:172.21.15.167:6789/0 conn(0x7f51c40af540 legacy=0x7f51c40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c3fff700 1 -- 172.21.15.33:0/4021154328 >> v1:172.21.15.33:6789/0 conn(0x7f51c415a650 legacy=0x7f51c40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.456+0000 7f51c3fff700 1 -- 172.21.15.33:0/4021154328 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c415db70 con 0x7f51c40cd5e0 2023-10-06T05:34:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.457+0000 7f51c3fff700 1 --2- 172.21.15.33:0/4021154328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40cd5e0 0x7f51c40af910 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f51b00098e0 tx=0x7f51b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9aa6aac7fdc7ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.457+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0011630 con 0x7f51c40cd5e0 2023-10-06T05:34:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.457+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0011790 con 0x7f51c40cd5e0 2023-10-06T05:34:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.457+0000 7f51c37fe700 1 -- 172.21.15.33:0/4021154328 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0011b30 con 0x7f51c40cd5e0 2023-10-06T05:34:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.457+0000 7f51c9d9d700 1 -- 172.21.15.33:0/4021154328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40cd5e0 msgr2=0x7f51c40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.457+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/4021154328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40cd5e0 0x7f51c40af910 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f51b00098e0 tx=0x7f51b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.458+0000 7f51c9d9d700 1 -- 172.21.15.33:0/4021154328 shutdown_connections 2023-10-06T05:34:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.458+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/4021154328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40cd5e0 0x7f51c40af910 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.458+0000 7f51c9d9d700 1 -- 172.21.15.33:0/4021154328 >> 172.21.15.33:0/4021154328 conn(0x7f51c406d9b0 msgr2=0x7f51c406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.458+0000 7f51c9d9d700 1 -- 172.21.15.33:0/4021154328 shutdown_connections 2023-10-06T05:34:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.458+0000 7f51c9d9d700 1 -- 172.21.15.33:0/4021154328 wait complete. 2023-10-06T05:34:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.459+0000 7f51c9d9d700 1 Processor -- start 2023-10-06T05:34:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.459+0000 7f51c9d9d700 1 -- start start 2023-10-06T05:34:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c9d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40af540 0x7f51c40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c9d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51c40cd5e0 0x7f51c4148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:00.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c9d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 0x7f51c415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:00.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c9d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40c6520 con 0x7f51c415a650 2023-10-06T05:34:00.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c9d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40c5cc0 con 0x7f51c40af540 2023-10-06T05:34:00.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c9d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40ca8f0 con 0x7f51c40cd5e0 2023-10-06T05:34:00.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 0x7f51c415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:00.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c959c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 0x7f51c415bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54314/0 (socket says 172.21.15.33:54314) 2023-10-06T05:34:00.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.460+0000 7f51c959c700 1 -- 172.21.15.33:0/103807428 learned_addr learned my addr 172.21.15.33:0/103807428 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:00.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c3fff700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51c40cd5e0 0x7f51c4148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:00.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c8d9b700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40af540 0x7f51c40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:00.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c959c700 1 -- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51c40cd5e0 msgr2=0x7f51c4148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c959c700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51c40cd5e0 0x7f51c4148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c959c700 1 -- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40af540 msgr2=0x7f51c40ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:00.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c959c700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40af540 0x7f51c40ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:00.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c959c700 1 -- 172.21.15.33:0/103807428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0009420 con 0x7f51c415a650 2023-10-06T05:34:00.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c8d9b700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40af540 0x7f51c40ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:00.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c959c700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 0x7f51c415bde0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00cc60 tx=0x7f51bc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:00.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00ce70 con 0x7f51c415a650 2023-10-06T05:34:00.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc00ab10 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0070f0 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.461+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c40c7ab0 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.462+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c40c61b0 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.463+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c40b42d0 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.463+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51bc0074a0 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.464+0000 7f51c0ff9700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51a8066f30 0x7f51a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.464+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51bc084b70 con 0x7f51c415a650 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.465+0000 7f51c8d9b700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51a8066f30 0x7f51a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:00.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.465+0000 7f51c8d9b700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51a8066f30 0x7f51a80693e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f51c40c5320 tx=0x7f51b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:00.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.468+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51c40b42d0 con 0x7f51c415a650 2023-10-06T05:34:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:00.724+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51c40b42d0 con 0x7f51c415a650 2023-10-06T05:34:01.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.236+0000 7f51c0ff9700 1 -- 172.21.15.33:0/103807428 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v203) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51c40b42d0 con 0x7f51c415a650 2023-10-06T05:34:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51a8066f30 msgr2=0x7f51a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51a8066f30 0x7f51a80693e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f51c40c5320 tx=0x7f51b8009040 comp rx=0 tx=0).stop 2023-10-06T05:34:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 msgr2=0x7f51c415bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 0x7f51c415bde0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00cc60 tx=0x7f51bc001ae0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 shutdown_connections 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f51a8066f30 0x7f51a80693e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c415a650 0x7f51c415bde0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51c40af540 0x7f51c40ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 --2- 172.21.15.33:0/103807428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51c40cd5e0 0x7f51c4148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 >> 172.21.15.33:0/103807428 conn(0x7f51c406d9b0 msgr2=0x7f51c4158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 shutdown_connections 2023-10-06T05:34:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.239+0000 7f51c9d9d700 1 -- 172.21.15.33:0/103807428 wait complete. 2023-10-06T05:34:01.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.448+0000 7f7753fff700 1 Processor -- start 2023-10-06T05:34:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.448+0000 7f7753fff700 1 -- start start 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.449+0000 7f7753fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7754159530 0x7f775406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.449+0000 7f7753fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f775406df70 0x7f77540afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.449+0000 7f7753fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f7754159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.449+0000 7f7753fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77540b1e00 con 0x7f775406df70 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.449+0000 7f7753fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775406f6b0 con 0x7f7754159530 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.449+0000 7f7753fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775406e3e0 con 0x7f77540b0310 2023-10-06T05:34:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f775406df70 0x7f77540afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f775406df70 0x7f77540afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54324/0 (socket says 172.21.15.33:54324) 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 -- 172.21.15.33:0/4099833077 learned_addr learned my addr 172.21.15.33:0/4099833077 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f7752ffd700 1 --2- 172.21.15.33:0/4099833077 >> v2:172.21.15.167:3300/0 conn(0x7f7754159530 0x7f775406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77537fe700 1 --2- 172.21.15.33:0/4099833077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f7754159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 -- 172.21.15.33:0/4099833077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 msgr2=0x7f7754159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 --2- 172.21.15.33:0/4099833077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f7754159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 -- 172.21.15.33:0/4099833077 >> v2:172.21.15.167:3300/0 conn(0x7f7754159530 msgr2=0x7f775406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 --2- 172.21.15.33:0/4099833077 >> v2:172.21.15.167:3300/0 conn(0x7f7754159530 0x7f775406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 -- 172.21.15.33:0/4099833077 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775406f220 con 0x7f775406df70 2023-10-06T05:34:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.450+0000 7f77527fc700 1 --2- 172.21.15.33:0/4099833077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f775406df70 0x7f77540afdd0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f77540cf4d0 tx=0x7f7748009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2794f168a8c13da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7751ffb700 1 -- 172.21.15.33:0/4099833077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774800b640 con 0x7f775406df70 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7751ffb700 1 -- 172.21.15.33:0/4099833077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774800b7a0 con 0x7f775406df70 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7751ffb700 1 -- 172.21.15.33:0/4099833077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774800bb40 con 0x7f775406df70 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 -- 172.21.15.33:0/4099833077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f775406df70 msgr2=0x7f77540afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 --2- 172.21.15.33:0/4099833077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f775406df70 0x7f77540afdd0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f77540cf4d0 tx=0x7f7748009250 comp rx=0 tx=0).stop 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 -- 172.21.15.33:0/4099833077 shutdown_connections 2023-10-06T05:34:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 --2- 172.21.15.33:0/4099833077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f7754159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 --2- 172.21.15.33:0/4099833077 >> v2:172.21.15.167:3300/0 conn(0x7f7754159530 0x7f775406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 --2- 172.21.15.33:0/4099833077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f775406df70 0x7f77540afdd0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 -- 172.21.15.33:0/4099833077 >> 172.21.15.33:0/4099833077 conn(0x7f775406c560 msgr2=0x7f775406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.451+0000 7f7753fff700 1 -- 172.21.15.33:0/4099833077 shutdown_connections 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.452+0000 7f7753fff700 1 -- 172.21.15.33:0/4099833077 wait complete. 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.453+0000 7f7753fff700 1 Processor -- start 2023-10-06T05:34:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.453+0000 7f7753fff700 1 -- start start 2023-10-06T05:34:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f7753fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775406df70 0x7f7754138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f7753fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f77540af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f7753fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 0x7f77540af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f7753fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775414d3b0 con 0x7f7754159530 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f7753fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775414c4c0 con 0x7f775406df70 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f7753fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775414c7a0 con 0x7f77540b0310 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f77537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 0x7f77540af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f77537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 0x7f77540af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54336/0 (socket says 172.21.15.33:54336) 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.454+0000 7f77537fe700 1 -- 172.21.15.33:0/1724863627 learned_addr learned my addr 172.21.15.33:0/1724863627 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f7752ffd700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775406df70 0x7f7754138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77527fc700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f77540af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77537fe700 1 -- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 msgr2=0x7f77540af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77537fe700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f77540af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77537fe700 1 -- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775406df70 msgr2=0x7f7754138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77537fe700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775406df70 0x7f7754138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77537fe700 1 -- 172.21.15.33:0/1724863627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480094f0 con 0x7f7754159530 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f77537fe700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 0x7f77540af8d0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f774c009e90 tx=0x7f774c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00cb70 con 0x7f7754159530 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c00f040 con 0x7f7754159530 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c014480 con 0x7f7754159530 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.455+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775414ca80 con 0x7f7754159530 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.456+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775414d040 con 0x7f7754159530 2023-10-06T05:34:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.457+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775414d520 con 0x7f7754159530 2023-10-06T05:34:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.458+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f774c014830 con 0x7f7754159530 2023-10-06T05:34:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.459+0000 7f773affd700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7740066f30 0x7f77400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.460+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f774c085010 con 0x7f7754159530 2023-10-06T05:34:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.460+0000 7f7752ffd700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7740066f30 0x7f77400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.461+0000 7f7752ffd700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7740066f30 0x7f77400693e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7744009db0 tx=0x7f77440091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:01.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.463+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f774c052120 con 0x7f7754159530 2023-10-06T05:34:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.709+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f775415d0e0 con 0x7f7754159530 2023-10-06T05:34:01.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.710+0000 7f773affd700 1 -- 172.21.15.33:0/1724863627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f774c055740 con 0x7f7754159530 2023-10-06T05:34:01.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:01.712 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":203,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8921,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3798274970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3798274970},{"type":"v1","addr":"172.21.15.33:6834","nonce":3798274970}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":198},{"gid":8924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2969193715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2969193715},{"type":"v1","addr":"172.21.15.33:6835","nonce":2969193715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199},{"gid":10678,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3478690144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3478690144},{"type":"v1","addr":"172.21.15.167:6836","nonce":3478690144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:01.234255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10681":{"gid":10681,"name":"b","rank":0,"incarnation":200,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3376723254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3376723254},{"type":"v1","addr":"172.21.15.167:6837","nonce":3376723254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7740066f30 msgr2=0x7f77400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7740066f30 0x7f77400693e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7744009db0 tx=0x7f77440091a0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 msgr2=0x7f77540af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 0x7f77540af8d0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f774c009e90 tx=0x7f774c00b040 comp rx=0 tx=0).stop 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 shutdown_connections 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7740066f30 0x7f77400693e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7754159530 0x7f77540af8d0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775406df70 0x7f7754138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 --2- 172.21.15.33:0/1724863627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77540b0310 0x7f77540af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 >> 172.21.15.33:0/1724863627 conn(0x7f775406c560 msgr2=0x7f77540b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 shutdown_connections 2023-10-06T05:34:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.712+0000 7f7753fff700 1 -- 172.21.15.33:0/1724863627 wait complete. 2023-10-06T05:34:01.716 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 203 2023-10-06T05:34:01.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:34:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.916+0000 7f9342b9d700 1 Processor -- start 2023-10-06T05:34:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.916+0000 7f9342b9d700 1 -- start start 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9342b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f933c155af0 0x7f933c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9342b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9342b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c06ab60 con 0x7f933c06a6a0 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9342b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c06b7e0 con 0x7f933c155af0 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9342b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c06ae40 con 0x7f933c0c7920 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9341b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f933c155af0 0x7f933c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9341b9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f933c155af0 0x7f933c06a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54346/0 (socket says 172.21.15.33:54346) 2023-10-06T05:34:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9341b9b700 1 -- 172.21.15.33:0/2490083685 learned_addr learned my addr 172.21.15.33:0/2490083685 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f934239c700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9341b9b700 1 -- 172.21.15.33:0/2490083685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 msgr2=0x7f933c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9341b9b700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.918+0000 7f9341b9b700 1 -- 172.21.15.33:0/2490083685 >> v1:172.21.15.167:6789/0 conn(0x7f933c06a6a0 legacy=0x7f933c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.919+0000 7f9341b9b700 1 -- 172.21.15.33:0/2490083685 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c06b120 con 0x7f933c155af0 2023-10-06T05:34:01.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.919+0000 7f934239c700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.919+0000 7f9341b9b700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c06a160 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f93380099b0 tx=0x7f9338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326729275be1d2eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.919+0000 7f9340b99700 1 -- 172.21.15.33:0/2490083685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800b860 con 0x7f933c155af0 2023-10-06T05:34:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.919+0000 7f9340b99700 1 -- 172.21.15.33:0/2490083685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933800b9c0 con 0x7f933c155af0 2023-10-06T05:34:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.919+0000 7f9340b99700 1 -- 172.21.15.33:0/2490083685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800bd60 con 0x7f933c155af0 2023-10-06T05:34:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 -- 172.21.15.33:0/2490083685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 msgr2=0x7f933c06a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c06a160 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f93380099b0 tx=0x7f9338009250 comp rx=0 tx=0).stop 2023-10-06T05:34:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 -- 172.21.15.33:0/2490083685 shutdown_connections 2023-10-06T05:34:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c06a160 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 --2- 172.21.15.33:0/2490083685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 -- 172.21.15.33:0/2490083685 >> 172.21.15.33:0/2490083685 conn(0x7f933c06d030 msgr2=0x7f933c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 -- 172.21.15.33:0/2490083685 shutdown_connections 2023-10-06T05:34:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.920+0000 7f9342b9d700 1 -- 172.21.15.33:0/2490083685 wait complete. 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.921+0000 7f9342b9d700 1 Processor -- start 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.921+0000 7f9342b9d700 1 -- start start 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f9342b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c06a6a0 0x7f933c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f9342b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f9342b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f9342b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ca9f0 con 0x7f933c155af0 2023-10-06T05:34:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f9342b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c152430 con 0x7f933c06a6a0 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f9342b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0c9de0 con 0x7f933c0c7920 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.922+0000 7f934239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c0ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54360/0 (socket says 172.21.15.33:54360) 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 -- 172.21.15.33:0/1782850934 learned_addr learned my addr 172.21.15.33:0/1782850934 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f9341b9b700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c06a6a0 0x7f933c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934139a700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 -- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 msgr2=0x7f933c0ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 -- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c06a6a0 msgr2=0x7f933c151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c06a6a0 0x7f933c151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 -- 172.21.15.33:0/1782850934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380094f0 con 0x7f933c155af0 2023-10-06T05:34:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f934239c700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c0ae5e0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f9334009fd0 tx=0x7f93340099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933400dc90 con 0x7f933c155af0 2023-10-06T05:34:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933400f040 con 0x7f933c155af0 2023-10-06T05:34:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334015440 con 0x7f933c155af0 2023-10-06T05:34:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.923+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0ca0c0 con 0x7f933c155af0 2023-10-06T05:34:01.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.924+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c0ca680 con 0x7f933c155af0 2023-10-06T05:34:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.924+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c0caba0 con 0x7f933c155af0 2023-10-06T05:34:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.926+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f933401e3b0 con 0x7f933c155af0 2023-10-06T05:34:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.926+0000 7f9331ffb700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9320066f30 0x7f93200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.926+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9334084bc0 con 0x7f933c155af0 2023-10-06T05:34:01.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.927+0000 7f9341b9b700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9320066f30 0x7f93200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:01.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.928+0000 7f9341b9b700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9320066f30 0x7f93200693e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f93380099b0 tx=0x7f9338017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:01.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:01.931+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f933c0caba0 con 0x7f933c155af0 2023-10-06T05:34:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.135+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f933c04c380 con 0x7f933c155af0 2023-10-06T05:34:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.136+0000 7f9331ffb700 1 -- 172.21.15.33:0/1782850934 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f933401a090 con 0x7f933c155af0 2023-10-06T05:34:02.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:02.139 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":265,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:33:59.211528+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:33:56.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":40,"pool_name":"cephfs_data","create_time":"2023-10-06T05:33:56.676003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:34:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9320066f30 msgr2=0x7f93200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9320066f30 0x7f93200693e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f93380099b0 tx=0x7f9338017000 comp rx=0 tx=0).stop 2023-10-06T05:34:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 msgr2=0x7f933c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c0ae5e0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f9334009fd0 tx=0x7f93340099f0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 shutdown_connections 2023-10-06T05:34:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9320066f30 0x7f93200693e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c155af0 0x7f933c0ae5e0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f933c06a6a0 0x7f933c151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 --2- 172.21.15.33:0/1782850934 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f933c0c7920 0x7f933c0ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 >> 172.21.15.33:0/1782850934 conn(0x7f933c06d030 msgr2=0x7f933c153dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 shutdown_connections 2023-10-06T05:34:02.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.139+0000 7f9342b9d700 1 -- 172.21.15.33:0/1782850934 wait complete. 2023-10-06T05:34:02.159 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:34:02.159 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:34:02.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.346+0000 7f4e0599d700 1 Processor -- start 2023-10-06T05:34:02.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.346+0000 7f4e0599d700 1 -- start start 2023-10-06T05:34:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.347+0000 7f4e0599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 0x7f4e0006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.347+0000 7f4e0599d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000c9cb0 con 0x7f4e000c8050 2023-10-06T05:34:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.347+0000 7f4e0599d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000ca270 con 0x7f4e0013faa0 2023-10-06T05:34:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.347+0000 7f4e0599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0006b840 con 0x7f4e000cb710 2023-10-06T05:34:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.347+0000 7f4dfffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4e000c8050 0x7f4e0006aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59608/0 (socket says 172.21.15.33:59608) 2023-10-06T05:34:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.347+0000 7f4dfffff700 1 -- 172.21.15.33:0/1365823492 learned_addr learned my addr 172.21.15.33:0/1365823492 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:02.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4e0519c700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 0x7f4e0006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371481303 0 0) 0x7f4e000c9cb0 con 0x7f4e000c8050 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df0003570 con 0x7f4e000c8050 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546044977 0 0) 0x7f4e000ca270 con 0x7f4e0013faa0 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e000c9cb0 con 0x7f4e0013faa0 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3065755443 0 0) 0x7f4df0003570 con 0x7f4e000c8050 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 msgr2=0x7f4e0006b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 0x7f4e0006b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 >> v1:172.21.15.167:6789/0 conn(0x7f4e0013faa0 legacy=0x7f4e000c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0006bb20 con 0x7f4e000c8050 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f4dec001470 con 0x7f4e000c8050 2023-10-06T05:34:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 >> v1:172.21.15.33:6789/0 conn(0x7f4e000c8050 legacy=0x7f4e0006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4df0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 0x7f4df0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4df0004d50 0x7f4df00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0006b840 con 0x7f4df0003c20 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df000bbf0 con 0x7f4df0004d50 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.348+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df000bd60 con 0x7f4e000cb710 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4df0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4dfffff700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 0x7f4df0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0519c700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4df0004d50 0x7f4df00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 -- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 msgr2=0x7f4df0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e000cb710 0x7f4df0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 -- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4df0004d50 msgr2=0x7f4df00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4df0004d50 0x7f4df00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 -- 172.21.15.33:0/1365823492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0003570 con 0x7f4df0003c20 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0519c700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4df0004d50 0x7f4df00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4e0499b700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4df0004010 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f4df40014c0 tx=0x7f4df4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f0aa03769153fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400c5d0 con 0x7f4df0003c20 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df400c730 con 0x7f4df0003c20 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.349+0000 7f4dff7fe700 1 -- 172.21.15.33:0/1365823492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400c8c0 con 0x7f4df0003c20 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0599d700 1 -- 172.21.15.33:0/1365823492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 msgr2=0x7f4df0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0599d700 1 --2- 172.21.15.33:0/1365823492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4df0004010 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f4df40014c0 tx=0x7f4df4007d30 comp rx=0 tx=0).stop 2023-10-06T05:34:02.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0519c700 1 -- 172.21.15.33:0/1365823492 reap_dead start 2023-10-06T05:34:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0599d700 1 -- 172.21.15.33:0/1365823492 shutdown_connections 2023-10-06T05:34:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0599d700 1 -- 172.21.15.33:0/1365823492 >> 172.21.15.33:0/1365823492 conn(0x7f4e0006d030 msgr2=0x7f4e0006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0599d700 1 -- 172.21.15.33:0/1365823492 shutdown_connections 2023-10-06T05:34:02.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.350+0000 7f4e0599d700 1 -- 172.21.15.33:0/1365823492 wait complete. 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.352+0000 7f4e0599d700 1 Processor -- start 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.352+0000 7f4e0599d700 1 -- start start 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0599d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4e001598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0599d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e00153750 0x7f4e00156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0599d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e00156740 0x7f4e00156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0599d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00155d70 con 0x7f4df0003c20 2023-10-06T05:34:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0599d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00159e10 con 0x7f4e00156740 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0599d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00154ec0 con 0x7f4e00153750 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4e001598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4e001598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54366/0 (socket says 172.21.15.33:54366) 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 -- 172.21.15.33:0/478877565 learned_addr learned my addr 172.21.15.33:0/478877565 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4dfffff700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e00153750 0x7f4e00156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0519c700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e00156740 0x7f4e00156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 -- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e00153750 msgr2=0x7f4e00156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e00153750 0x7f4e00156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 -- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e00156740 msgr2=0x7f4e00156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e00156740 0x7f4e00156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0499b700 1 -- 172.21.15.33:0/478877565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e00155160 con 0x7f4df0003c20 2023-10-06T05:34:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.353+0000 7f4e0519c700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e00156740 0x7f4e00156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.354+0000 7f4e0499b700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4e001598d0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f4df4008fd0 tx=0x7f4df4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.354+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400c5d0 con 0x7f4df0003c20 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.354+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df400c8a0 con 0x7f4df0003c20 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.354+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4001360 con 0x7f4df0003c20 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.354+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00155440 con 0x7f4df0003c20 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.354+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e00155a00 con 0x7f4df0003c20 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.355+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e00156050 con 0x7f4df0003c20 2023-10-06T05:34:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.356+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4df4006a30 con 0x7f4df0003c20 2023-10-06T05:34:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.357+0000 7f4dfcff9700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4de4066f30 0x7f4de40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.357+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4df4015020 con 0x7f4df0003c20 2023-10-06T05:34:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.358+0000 7f4dfffff700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4de4066f30 0x7f4de40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.358+0000 7f4dfffff700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4de4066f30 0x7f4de40693e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4e00153b20 tx=0x7f4e0006be00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.361+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4df4052860 con 0x7f4df0003c20 2023-10-06T05:34:02.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.560+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e000698c0 con 0x7f4de4066f30 2023-10-06T05:34:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.570+0000 7f4dfcff9700 1 -- 172.21.15.33:0/478877565 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311566 (secure 0 0 0) 0x7f4e000698c0 con 0x7f4de4066f30 2023-10-06T05:34:02.576 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.574+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4de4066f30 msgr2=0x7f4de40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.574+0000 7f4e0599d700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4de4066f30 0x7f4de40693e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4e00153b20 tx=0x7f4e0006be00 comp rx=0 tx=0).stop 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 msgr2=0x7f4e001598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4e001598d0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f4df4008fd0 tx=0x7f4df4017000 comp rx=0 tx=0).stop 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 shutdown_connections 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4de4066f30 0x7f4de40693e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e00156740 0x7f4e00156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df0003c20 0x7f4e001598d0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 --2- 172.21.15.33:0/478877565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e00153750 0x7f4e00156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 >> 172.21.15.33:0/478877565 conn(0x7f4e0006d030 msgr2=0x7f4e000c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 shutdown_connections 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.575+0000 7f4e0599d700 1 -- 172.21.15.33:0/478877565 wait complete. 2023-10-06T05:34:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:34:02.594 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":853,"stamp":"2023-10-06T05:34:02.121142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450856,"kb_used_data":27496,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752523864,"statfs":{"total":773094113280,"available":770584436736,"internally_reserved":0,"allocated":28155904,"data_stored":13811132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928484"},"pg_stats":[{"pgid":"40.3f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.232305+0000","last_change":"2023-10-06T05:33:58.228484+0000","last_active":"2023-10-06T05:33:58.232305+0000","last_peered":"2023-10-06T05:33:58.232305+0000","last_clean":"2023-10-06T05:33:58.232305+0000","last_became_active":"2023-10-06T05:33:58.228229+0000","last_became_peered":"2023-10-06T05:33:58.228229+0000","last_unstale":"2023-10-06T05:33:58.232305+0000","last_undegraded":"2023-10-06T05:33:58.232305+0000","last_fullsized":"2023-10-06T05:33:58.232305+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:52.372410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.3e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.237676+0000","last_change":"2023-10-06T05:33:58.236047+0000","last_active":"2023-10-06T05:33:58.237676+0000","last_peered":"2023-10-06T05:33:58.237676+0000","last_clean":"2023-10-06T05:33:58.237676+0000","last_became_active":"2023-10-06T05:33:58.235709+0000","last_became_peered":"2023-10-06T05:33:58.235709+0000","last_unstale":"2023-10-06T05:33:58.237676+0000","last_undegraded":"2023-10-06T05:33:58.237676+0000","last_fullsized":"2023-10-06T05:33:58.237676+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.3d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256482+0000","last_change":"2023-10-06T05:33:58.248285+0000","last_active":"2023-10-06T05:33:58.256482+0000","last_peered":"2023-10-06T05:33:58.256482+0000","last_clean":"2023-10-06T05:33:58.256482+0000","last_became_active":"2023-10-06T05:33:58.243811+0000","last_became_peered":"2023-10-06T05:33:58.243811+0000","last_unstale":"2023-10-06T05:33:58.256482+0000","last_undegraded":"2023-10-06T05:33:58.256482+0000","last_fullsized":"2023-10-06T05:33:58.256482+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:28.718086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.3c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.236427+0000","last_change":"2023-10-06T05:33:58.231915+0000","last_active":"2023-10-06T05:33:58.236427+0000","last_peered":"2023-10-06T05:33:58.236427+0000","last_clean":"2023-10-06T05:33:58.236427+0000","last_became_active":"2023-10-06T05:33:58.231622+0000","last_became_peered":"2023-10-06T05:33:58.231622+0000","last_unstale":"2023-10-06T05:33:58.236427+0000","last_undegraded":"2023-10-06T05:33:58.236427+0000","last_fullsized":"2023-10-06T05:33:58.236427+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:33.652138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.3b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239827+0000","last_change":"2023-10-06T05:33:58.235929+0000","last_active":"2023-10-06T05:33:58.239827+0000","last_peered":"2023-10-06T05:33:58.239827+0000","last_clean":"2023-10-06T05:33:58.239827+0000","last_became_active":"2023-10-06T05:33:58.235502+0000","last_became_peered":"2023-10-06T05:33:58.235502+0000","last_unstale":"2023-10-06T05:33:58.239827+0000","last_undegraded":"2023-10-06T05:33:58.239827+0000","last_fullsized":"2023-10-06T05:33:58.239827+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:05.165142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.255037+0000","last_change":"2023-10-06T05:33:58.243182+0000","last_active":"2023-10-06T05:33:58.255037+0000","last_peered":"2023-10-06T05:33:58.255037+0000","last_clean":"2023-10-06T05:33:58.255037+0000","last_became_active":"2023-10-06T05:33:58.242929+0000","last_became_peered":"2023-10-06T05:33:58.242929+0000","last_unstale":"2023-10-06T05:33:58.255037+0000","last_undegraded":"2023-10-06T05:33:58.255037+0000","last_fullsized":"2023-10-06T05:33:58.255037+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:49.305365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.39","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242732+0000","last_change":"2023-10-06T05:33:58.236358+0000","last_active":"2023-10-06T05:33:58.242732+0000","last_peered":"2023-10-06T05:33:58.242732+0000","last_clean":"2023-10-06T05:33:58.242732+0000","last_became_active":"2023-10-06T05:33:58.236096+0000","last_became_peered":"2023-10-06T05:33:58.236096+0000","last_unstale":"2023-10-06T05:33:58.242732+0000","last_undegraded":"2023-10-06T05:33:58.242732+0000","last_fullsized":"2023-10-06T05:33:58.242732+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:50.296293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.38","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240277+0000","last_change":"2023-10-06T05:33:58.237761+0000","last_active":"2023-10-06T05:33:58.240277+0000","last_peered":"2023-10-06T05:33:58.240277+0000","last_clean":"2023-10-06T05:33:58.240277+0000","last_became_active":"2023-10-06T05:33:58.237235+0000","last_became_peered":"2023-10-06T05:33:58.237235+0000","last_unstale":"2023-10-06T05:33:58.240277+0000","last_undegraded":"2023-10-06T05:33:58.240277+0000","last_fullsized":"2023-10-06T05:33:58.240277+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:09.140740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.37","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239873+0000","last_change":"2023-10-06T05:33:58.234111+0000","last_active":"2023-10-06T05:33:58.239873+0000","last_peered":"2023-10-06T05:33:58.239873+0000","last_clean":"2023-10-06T05:33:58.239873+0000","last_became_active":"2023-10-06T05:33:58.233851+0000","last_became_peered":"2023-10-06T05:33:58.233851+0000","last_unstale":"2023-10-06T05:33:58.239873+0000","last_undegraded":"2023-10-06T05:33:58.239873+0000","last_fullsized":"2023-10-06T05:33:58.239873+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:50.639231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.36","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.235875+0000","last_change":"2023-10-06T05:33:58.224923+0000","last_active":"2023-10-06T05:33:58.235875+0000","last_peered":"2023-10-06T05:33:58.235875+0000","last_clean":"2023-10-06T05:33:58.235875+0000","last_became_active":"2023-10-06T05:33:58.224600+0000","last_became_peered":"2023-10-06T05:33:58.224600+0000","last_unstale":"2023-10-06T05:33:58.235875+0000","last_undegraded":"2023-10-06T05:33:58.235875+0000","last_fullsized":"2023-10-06T05:33:58.235875+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:58.181784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.35","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242471+0000","last_change":"2023-10-06T05:33:58.237061+0000","last_active":"2023-10-06T05:33:58.242471+0000","last_peered":"2023-10-06T05:33:58.242471+0000","last_clean":"2023-10-06T05:33:58.242471+0000","last_became_active":"2023-10-06T05:33:58.236782+0000","last_became_peered":"2023-10-06T05:33:58.236782+0000","last_unstale":"2023-10-06T05:33:58.242471+0000","last_undegraded":"2023-10-06T05:33:58.242471+0000","last_fullsized":"2023-10-06T05:33:58.242471+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:29.743815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.34","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.232454+0000","last_change":"2023-10-06T05:33:58.228085+0000","last_active":"2023-10-06T05:33:58.232454+0000","last_peered":"2023-10-06T05:33:58.232454+0000","last_clean":"2023-10-06T05:33:58.232454+0000","last_became_active":"2023-10-06T05:33:58.227272+0000","last_became_peered":"2023-10-06T05:33:58.227272+0000","last_unstale":"2023-10-06T05:33:58.232454+0000","last_undegraded":"2023-10-06T05:33:58.232454+0000","last_fullsized":"2023-10-06T05:33:58.232454+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:57.170240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.33","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257027+0000","last_change":"2023-10-06T05:33:58.255574+0000","last_active":"2023-10-06T05:33:58.257027+0000","last_peered":"2023-10-06T05:33:58.257027+0000","last_clean":"2023-10-06T05:33:58.257027+0000","last_became_active":"2023-10-06T05:33:58.255202+0000","last_became_peered":"2023-10-06T05:33:58.255202+0000","last_unstale":"2023-10-06T05:33:58.257027+0000","last_undegraded":"2023-10-06T05:33:58.257027+0000","last_fullsized":"2023-10-06T05:33:58.257027+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:21.213909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.32","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258731+0000","last_change":"2023-10-06T05:33:58.256295+0000","last_active":"2023-10-06T05:33:58.258731+0000","last_peered":"2023-10-06T05:33:58.258731+0000","last_clean":"2023-10-06T05:33:58.258731+0000","last_became_active":"2023-10-06T05:33:58.255584+0000","last_became_peered":"2023-10-06T05:33:58.255584+0000","last_unstale":"2023-10-06T05:33:58.258731+0000","last_undegraded":"2023-10-06T05:33:58.258731+0000","last_fullsized":"2023-10-06T05:33:58.258731+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:49.426905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.31","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.255606+0000","last_change":"2023-10-06T05:33:58.245395+0000","last_active":"2023-10-06T05:33:58.255606+0000","last_peered":"2023-10-06T05:33:58.255606+0000","last_clean":"2023-10-06T05:33:58.255606+0000","last_became_active":"2023-10-06T05:33:58.242865+0000","last_became_peered":"2023-10-06T05:33:58.242865+0000","last_unstale":"2023-10-06T05:33:58.255606+0000","last_undegraded":"2023-10-06T05:33:58.255606+0000","last_fullsized":"2023-10-06T05:33:58.255606+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:18.456691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.30","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257556+0000","last_change":"2023-10-06T05:33:58.244913+0000","last_active":"2023-10-06T05:33:58.257556+0000","last_peered":"2023-10-06T05:33:58.257556+0000","last_clean":"2023-10-06T05:33:58.257556+0000","last_became_active":"2023-10-06T05:33:58.244593+0000","last_became_peered":"2023-10-06T05:33:58.244593+0000","last_unstale":"2023-10-06T05:33:58.257556+0000","last_undegraded":"2023-10-06T05:33:58.257556+0000","last_fullsized":"2023-10-06T05:33:58.257556+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:03.290098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.27","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239893+0000","last_change":"2023-10-06T05:33:58.236551+0000","last_active":"2023-10-06T05:33:58.239893+0000","last_peered":"2023-10-06T05:33:58.239893+0000","last_clean":"2023-10-06T05:33:58.239893+0000","last_became_active":"2023-10-06T05:33:58.236257+0000","last_became_peered":"2023-10-06T05:33:58.236257+0000","last_unstale":"2023-10-06T05:33:58.239893+0000","last_undegraded":"2023-10-06T05:33:58.239893+0000","last_fullsized":"2023-10-06T05:33:58.239893+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:54.326226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.26","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240054+0000","last_change":"2023-10-06T05:33:58.237071+0000","last_active":"2023-10-06T05:33:58.240054+0000","last_peered":"2023-10-06T05:33:58.240054+0000","last_clean":"2023-10-06T05:33:58.240054+0000","last_became_active":"2023-10-06T05:33:58.236794+0000","last_became_peered":"2023-10-06T05:33:58.236794+0000","last_unstale":"2023-10-06T05:33:58.240054+0000","last_undegraded":"2023-10-06T05:33:58.240054+0000","last_fullsized":"2023-10-06T05:33:58.240054+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:49.645094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.25","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256503+0000","last_change":"2023-10-06T05:33:58.248772+0000","last_active":"2023-10-06T05:33:58.256503+0000","last_peered":"2023-10-06T05:33:58.256503+0000","last_clean":"2023-10-06T05:33:58.256503+0000","last_became_active":"2023-10-06T05:33:58.248509+0000","last_became_peered":"2023-10-06T05:33:58.248509+0000","last_unstale":"2023-10-06T05:33:58.256503+0000","last_undegraded":"2023-10-06T05:33:58.256503+0000","last_fullsized":"2023-10-06T05:33:58.256503+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:23.874793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.24","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.249413+0000","last_change":"2023-10-06T05:33:58.246489+0000","last_active":"2023-10-06T05:33:58.249413+0000","last_peered":"2023-10-06T05:33:58.249413+0000","last_clean":"2023-10-06T05:33:58.249413+0000","last_became_active":"2023-10-06T05:33:58.246201+0000","last_became_peered":"2023-10-06T05:33:58.246201+0000","last_unstale":"2023-10-06T05:33:58.249413+0000","last_undegraded":"2023-10-06T05:33:58.249413+0000","last_fullsized":"2023-10-06T05:33:58.249413+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:48.750654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.23","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256207+0000","last_change":"2023-10-06T05:33:58.255050+0000","last_active":"2023-10-06T05:33:58.256207+0000","last_peered":"2023-10-06T05:33:58.256207+0000","last_clean":"2023-10-06T05:33:58.256207+0000","last_became_active":"2023-10-06T05:33:58.254472+0000","last_became_peered":"2023-10-06T05:33:58.254472+0000","last_unstale":"2023-10-06T05:33:58.256207+0000","last_undegraded":"2023-10-06T05:33:58.256207+0000","last_fullsized":"2023-10-06T05:33:58.256207+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.22","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.249312+0000","last_change":"2023-10-06T05:33:58.245502+0000","last_active":"2023-10-06T05:33:58.249312+0000","last_peered":"2023-10-06T05:33:58.249312+0000","last_clean":"2023-10-06T05:33:58.249312+0000","last_became_active":"2023-10-06T05:33:58.245192+0000","last_became_peered":"2023-10-06T05:33:58.245192+0000","last_unstale":"2023-10-06T05:33:58.249312+0000","last_undegraded":"2023-10-06T05:33:58.249312+0000","last_fullsized":"2023-10-06T05:33:58.249312+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:03.651492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.21","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256259+0000","last_change":"2023-10-06T05:33:58.253322+0000","last_active":"2023-10-06T05:33:58.256259+0000","last_peered":"2023-10-06T05:33:58.256259+0000","last_clean":"2023-10-06T05:33:58.256259+0000","last_became_active":"2023-10-06T05:33:58.252965+0000","last_became_peered":"2023-10-06T05:33:58.252965+0000","last_unstale":"2023-10-06T05:33:58.256259+0000","last_undegraded":"2023-10-06T05:33:58.256259+0000","last_fullsized":"2023-10-06T05:33:58.256259+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:31.822293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.20","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.251717+0000","last_change":"2023-10-06T05:33:58.248484+0000","last_active":"2023-10-06T05:33:58.251717+0000","last_peered":"2023-10-06T05:33:58.251717+0000","last_clean":"2023-10-06T05:33:58.251717+0000","last_became_active":"2023-10-06T05:33:58.248222+0000","last_became_peered":"2023-10-06T05:33:58.248222+0000","last_unstale":"2023-10-06T05:33:58.251717+0000","last_undegraded":"2023-10-06T05:33:58.251717+0000","last_fullsized":"2023-10-06T05:33:58.251717+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:31.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.10","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.238371+0000","last_change":"2023-10-06T05:33:58.230228+0000","last_active":"2023-10-06T05:33:58.238371+0000","last_peered":"2023-10-06T05:33:58.238371+0000","last_clean":"2023-10-06T05:33:58.238371+0000","last_became_active":"2023-10-06T05:33:58.229966+0000","last_became_peered":"2023-10-06T05:33:58.229966+0000","last_unstale":"2023-10-06T05:33:58.238371+0000","last_undegraded":"2023-10-06T05:33:58.238371+0000","last_fullsized":"2023-10-06T05:33:58.238371+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.767359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.1f","version":"264'4","reported_seq":14,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258134+0000","last_change":"2023-10-06T05:33:57.211470+0000","last_active":"2023-10-06T05:33:58.258134+0000","last_peered":"2023-10-06T05:33:58.258134+0000","last_clean":"2023-10-06T05:33:58.258134+0000","last_became_active":"2023-10-06T05:33:57.211195+0000","last_became_peered":"2023-10-06T05:33:57.211195+0000","last_unstale":"2023-10-06T05:33:58.258134+0000","last_undegraded":"2023-10-06T05:33:58.258134+0000","last_fullsized":"2023-10-06T05:33:58.258134+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:25.943869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.11","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240688+0000","last_change":"2023-10-06T05:33:58.239563+0000","last_active":"2023-10-06T05:33:58.240688+0000","last_peered":"2023-10-06T05:33:58.240688+0000","last_clean":"2023-10-06T05:33:58.240688+0000","last_became_active":"2023-10-06T05:33:58.239250+0000","last_became_peered":"2023-10-06T05:33:58.239250+0000","last_unstale":"2023-10-06T05:33:58.240688+0000","last_undegraded":"2023-10-06T05:33:58.240688+0000","last_fullsized":"2023-10-06T05:33:58.240688+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:43.288728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.1e","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.254685+0000","last_change":"2023-10-06T05:33:57.208860+0000","last_active":"2023-10-06T05:33:58.254685+0000","last_peered":"2023-10-06T05:33:58.254685+0000","last_clean":"2023-10-06T05:33:58.254685+0000","last_became_active":"2023-10-06T05:33:57.208589+0000","last_became_peered":"2023-10-06T05:33:57.208589+0000","last_unstale":"2023-10-06T05:33:58.254685+0000","last_undegraded":"2023-10-06T05:33:58.254685+0000","last_fullsized":"2023-10-06T05:33:58.254685+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:47.258732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.12","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230170+0000","last_change":"2023-10-06T05:33:58.223688+0000","last_active":"2023-10-06T05:33:58.230170+0000","last_peered":"2023-10-06T05:33:58.230170+0000","last_clean":"2023-10-06T05:33:58.230170+0000","last_became_active":"2023-10-06T05:33:58.223356+0000","last_became_peered":"2023-10-06T05:33:58.223356+0000","last_unstale":"2023-10-06T05:33:58.230170+0000","last_undegraded":"2023-10-06T05:33:58.230170+0000","last_fullsized":"2023-10-06T05:33:58.230170+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:18.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.1d","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.254514+0000","last_change":"2023-10-06T05:33:57.208902+0000","last_active":"2023-10-06T05:33:58.254514+0000","last_peered":"2023-10-06T05:33:58.254514+0000","last_clean":"2023-10-06T05:33:58.254514+0000","last_became_active":"2023-10-06T05:33:57.208579+0000","last_became_peered":"2023-10-06T05:33:57.208579+0000","last_unstale":"2023-10-06T05:33:58.254514+0000","last_undegraded":"2023-10-06T05:33:58.254514+0000","last_fullsized":"2023-10-06T05:33:58.254514+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:47.393911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.13","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.246672+0000","last_change":"2023-10-06T05:33:58.244529+0000","last_active":"2023-10-06T05:33:58.246672+0000","last_peered":"2023-10-06T05:33:58.246672+0000","last_clean":"2023-10-06T05:33:58.246672+0000","last_became_active":"2023-10-06T05:33:58.244070+0000","last_became_peered":"2023-10-06T05:33:58.244070+0000","last_unstale":"2023-10-06T05:33:58.246672+0000","last_undegraded":"2023-10-06T05:33:58.246672+0000","last_fullsized":"2023-10-06T05:33:58.246672+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:07.590244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.1c","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.252019+0000","last_change":"2023-10-06T05:33:57.201640+0000","last_active":"2023-10-06T05:33:58.252019+0000","last_peered":"2023-10-06T05:33:58.252019+0000","last_clean":"2023-10-06T05:33:58.252019+0000","last_became_active":"2023-10-06T05:33:57.201375+0000","last_became_peered":"2023-10-06T05:33:57.201375+0000","last_unstale":"2023-10-06T05:33:58.252019+0000","last_undegraded":"2023-10-06T05:33:58.252019+0000","last_fullsized":"2023-10-06T05:33:58.252019+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:51.855744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.14","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239552+0000","last_change":"2023-10-06T05:33:58.236460+0000","last_active":"2023-10-06T05:33:58.239552+0000","last_peered":"2023-10-06T05:33:58.239552+0000","last_clean":"2023-10-06T05:33:58.239552+0000","last_became_active":"2023-10-06T05:33:58.236012+0000","last_became_peered":"2023-10-06T05:33:58.236012+0000","last_unstale":"2023-10-06T05:33:58.239552+0000","last_undegraded":"2023-10-06T05:33:58.239552+0000","last_fullsized":"2023-10-06T05:33:58.239552+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:46.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.1b","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.253654+0000","last_change":"2023-10-06T05:33:57.217712+0000","last_active":"2023-10-06T05:33:58.253654+0000","last_peered":"2023-10-06T05:33:58.253654+0000","last_clean":"2023-10-06T05:33:58.253654+0000","last_became_active":"2023-10-06T05:33:57.217464+0000","last_became_peered":"2023-10-06T05:33:57.217464+0000","last_unstale":"2023-10-06T05:33:58.253654+0000","last_undegraded":"2023-10-06T05:33:58.253654+0000","last_fullsized":"2023-10-06T05:33:58.253654+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:37.685725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.15","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.236824+0000","last_change":"2023-10-06T05:33:58.235707+0000","last_active":"2023-10-06T05:33:58.236824+0000","last_peered":"2023-10-06T05:33:58.236824+0000","last_clean":"2023-10-06T05:33:58.236824+0000","last_became_active":"2023-10-06T05:33:58.235419+0000","last_became_peered":"2023-10-06T05:33:58.235419+0000","last_unstale":"2023-10-06T05:33:58.236824+0000","last_undegraded":"2023-10-06T05:33:58.236824+0000","last_fullsized":"2023-10-06T05:33:58.236824+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:52.288996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.1a","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257457+0000","last_change":"2023-10-06T05:33:57.220422+0000","last_active":"2023-10-06T05:33:58.257457+0000","last_peered":"2023-10-06T05:33:58.257457+0000","last_clean":"2023-10-06T05:33:58.257457+0000","last_became_active":"2023-10-06T05:33:57.220134+0000","last_became_peered":"2023-10-06T05:33:57.220134+0000","last_unstale":"2023-10-06T05:33:58.257457+0000","last_undegraded":"2023-10-06T05:33:58.257457+0000","last_fullsized":"2023-10-06T05:33:58.257457+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:58.439257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.16","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231574+0000","last_change":"2023-10-06T05:33:58.227512+0000","last_active":"2023-10-06T05:33:58.231574+0000","last_peered":"2023-10-06T05:33:58.231574+0000","last_clean":"2023-10-06T05:33:58.231574+0000","last_became_active":"2023-10-06T05:33:58.227214+0000","last_became_peered":"2023-10-06T05:33:58.227214+0000","last_unstale":"2023-10-06T05:33:58.231574+0000","last_undegraded":"2023-10-06T05:33:58.231574+0000","last_fullsized":"2023-10-06T05:33:58.231574+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:38.504928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.19","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219187+0000","last_change":"2023-10-06T05:33:57.209024+0000","last_active":"2023-10-06T05:33:58.219187+0000","last_peered":"2023-10-06T05:33:58.219187+0000","last_clean":"2023-10-06T05:33:58.219187+0000","last_became_active":"2023-10-06T05:33:57.208696+0000","last_became_peered":"2023-10-06T05:33:57.208696+0000","last_unstale":"2023-10-06T05:33:58.219187+0000","last_undegraded":"2023-10-06T05:33:58.219187+0000","last_fullsized":"2023-10-06T05:33:58.219187+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:45.979360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.17","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256346+0000","last_change":"2023-10-06T05:33:58.252477+0000","last_active":"2023-10-06T05:33:58.256346+0000","last_peered":"2023-10-06T05:33:58.256346+0000","last_clean":"2023-10-06T05:33:58.256346+0000","last_became_active":"2023-10-06T05:33:58.252204+0000","last_became_peered":"2023-10-06T05:33:58.252204+0000","last_unstale":"2023-10-06T05:33:58.256346+0000","last_undegraded":"2023-10-06T05:33:58.256346+0000","last_fullsized":"2023-10-06T05:33:58.256346+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:13.846951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.18","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.220059+0000","last_change":"2023-10-06T05:33:57.212430+0000","last_active":"2023-10-06T05:33:58.220059+0000","last_peered":"2023-10-06T05:33:58.220059+0000","last_clean":"2023-10-06T05:33:58.220059+0000","last_became_active":"2023-10-06T05:33:57.212187+0000","last_became_peered":"2023-10-06T05:33:57.212187+0000","last_unstale":"2023-10-06T05:33:58.220059+0000","last_undegraded":"2023-10-06T05:33:58.220059+0000","last_fullsized":"2023-10-06T05:33:58.220059+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:28.543454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.18","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240981+0000","last_change":"2023-10-06T05:33:58.239029+0000","last_active":"2023-10-06T05:33:58.240981+0000","last_peered":"2023-10-06T05:33:58.240981+0000","last_clean":"2023-10-06T05:33:58.240981+0000","last_became_active":"2023-10-06T05:33:58.238394+0000","last_became_peered":"2023-10-06T05:33:58.238394+0000","last_unstale":"2023-10-06T05:33:58.240981+0000","last_undegraded":"2023-10-06T05:33:58.240981+0000","last_fullsized":"2023-10-06T05:33:58.240981+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:06.769180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.17","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.223057+0000","last_change":"2023-10-06T05:33:57.209644+0000","last_active":"2023-10-06T05:33:58.223057+0000","last_peered":"2023-10-06T05:33:58.223057+0000","last_clean":"2023-10-06T05:33:58.223057+0000","last_became_active":"2023-10-06T05:33:57.209371+0000","last_became_peered":"2023-10-06T05:33:57.209371+0000","last_unstale":"2023-10-06T05:33:58.223057+0000","last_undegraded":"2023-10-06T05:33:58.223057+0000","last_fullsized":"2023-10-06T05:33:58.223057+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:09.971282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.19","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242751+0000","last_change":"2023-10-06T05:33:58.237113+0000","last_active":"2023-10-06T05:33:58.242751+0000","last_peered":"2023-10-06T05:33:58.242751+0000","last_clean":"2023-10-06T05:33:58.242751+0000","last_became_active":"2023-10-06T05:33:58.236752+0000","last_became_peered":"2023-10-06T05:33:58.236752+0000","last_unstale":"2023-10-06T05:33:58.242751+0000","last_undegraded":"2023-10-06T05:33:58.242751+0000","last_fullsized":"2023-10-06T05:33:58.242751+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:05.138335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.16","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219788+0000","last_change":"2023-10-06T05:33:57.214441+0000","last_active":"2023-10-06T05:33:58.219788+0000","last_peered":"2023-10-06T05:33:58.219788+0000","last_clean":"2023-10-06T05:33:58.219788+0000","last_became_active":"2023-10-06T05:33:57.214152+0000","last_became_peered":"2023-10-06T05:33:57.214152+0000","last_unstale":"2023-10-06T05:33:58.219788+0000","last_undegraded":"2023-10-06T05:33:58.219788+0000","last_fullsized":"2023-10-06T05:33:58.219788+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:07.238120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.1a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.236895+0000","last_change":"2023-10-06T05:33:58.235295+0000","last_active":"2023-10-06T05:33:58.236895+0000","last_peered":"2023-10-06T05:33:58.236895+0000","last_clean":"2023-10-06T05:33:58.236895+0000","last_became_active":"2023-10-06T05:33:58.234985+0000","last_became_peered":"2023-10-06T05:33:58.234985+0000","last_unstale":"2023-10-06T05:33:58.236895+0000","last_undegraded":"2023-10-06T05:33:58.236895+0000","last_fullsized":"2023-10-06T05:33:58.236895+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.497369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.15","version":"264'3","reported_seq":13,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.259353+0000","last_change":"2023-10-06T05:33:57.214405+0000","last_active":"2023-10-06T05:33:58.259353+0000","last_peered":"2023-10-06T05:33:58.259353+0000","last_clean":"2023-10-06T05:33:58.259353+0000","last_became_active":"2023-10-06T05:33:57.214141+0000","last_became_peered":"2023-10-06T05:33:57.214141+0000","last_unstale":"2023-10-06T05:33:58.259353+0000","last_undegraded":"2023-10-06T05:33:58.259353+0000","last_fullsized":"2023-10-06T05:33:58.259353+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:46.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.1b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256405+0000","last_change":"2023-10-06T05:33:58.246407+0000","last_active":"2023-10-06T05:33:58.256405+0000","last_peered":"2023-10-06T05:33:58.256405+0000","last_clean":"2023-10-06T05:33:58.256405+0000","last_became_active":"2023-10-06T05:33:58.246137+0000","last_became_peered":"2023-10-06T05:33:58.246137+0000","last_unstale":"2023-10-06T05:33:58.256405+0000","last_undegraded":"2023-10-06T05:33:58.256405+0000","last_fullsized":"2023-10-06T05:33:58.256405+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:51.321834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.14","version":"264'6","reported_seq":16,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257722+0000","last_change":"2023-10-06T05:33:57.219848+0000","last_active":"2023-10-06T05:33:58.257722+0000","last_peered":"2023-10-06T05:33:58.257722+0000","last_clean":"2023-10-06T05:33:58.257722+0000","last_became_active":"2023-10-06T05:33:57.219559+0000","last_became_peered":"2023-10-06T05:33:57.219559+0000","last_unstale":"2023-10-06T05:33:58.257722+0000","last_undegraded":"2023-10-06T05:33:58.257722+0000","last_fullsized":"2023-10-06T05:33:58.257722+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:01.792918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.251402+0000","last_change":"2023-10-06T05:33:58.247110+0000","last_active":"2023-10-06T05:33:58.251402+0000","last_peered":"2023-10-06T05:33:58.251402+0000","last_clean":"2023-10-06T05:33:58.251402+0000","last_became_active":"2023-10-06T05:33:58.246796+0000","last_became_peered":"2023-10-06T05:33:58.246796+0000","last_unstale":"2023-10-06T05:33:58.251402+0000","last_undegraded":"2023-10-06T05:33:58.251402+0000","last_fullsized":"2023-10-06T05:33:58.251402+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:59.153527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"39.13","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.254953+0000","last_change":"2023-10-06T05:33:57.213165+0000","last_active":"2023-10-06T05:33:58.254953+0000","last_peered":"2023-10-06T05:33:58.254953+0000","last_clean":"2023-10-06T05:33:58.254953+0000","last_became_active":"2023-10-06T05:33:57.212893+0000","last_became_peered":"2023-10-06T05:33:57.212893+0000","last_unstale":"2023-10-06T05:33:58.254953+0000","last_undegraded":"2023-10-06T05:33:58.254953+0000","last_fullsized":"2023-10-06T05:33:58.254953+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:22.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.1d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256448+0000","last_change":"2023-10-06T05:33:58.245790+0000","last_active":"2023-10-06T05:33:58.256448+0000","last_peered":"2023-10-06T05:33:58.256448+0000","last_clean":"2023-10-06T05:33:58.256448+0000","last_became_active":"2023-10-06T05:33:58.245211+0000","last_became_peered":"2023-10-06T05:33:58.245211+0000","last_unstale":"2023-10-06T05:33:58.256448+0000","last_undegraded":"2023-10-06T05:33:58.256448+0000","last_fullsized":"2023-10-06T05:33:58.256448+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:40.504784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.12","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.221624+0000","last_change":"2023-10-06T05:33:57.216297+0000","last_active":"2023-10-06T05:33:58.221624+0000","last_peered":"2023-10-06T05:33:58.221624+0000","last_clean":"2023-10-06T05:33:58.221624+0000","last_became_active":"2023-10-06T05:33:57.216034+0000","last_became_peered":"2023-10-06T05:33:57.216034+0000","last_unstale":"2023-10-06T05:33:58.221624+0000","last_undegraded":"2023-10-06T05:33:58.221624+0000","last_fullsized":"2023-10-06T05:33:58.221624+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:40.241630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.247230+0000","last_change":"2023-10-06T05:33:58.242204+0000","last_active":"2023-10-06T05:33:58.247230+0000","last_peered":"2023-10-06T05:33:58.247230+0000","last_clean":"2023-10-06T05:33:58.247230+0000","last_became_active":"2023-10-06T05:33:58.241908+0000","last_became_peered":"2023-10-06T05:33:58.241908+0000","last_unstale":"2023-10-06T05:33:58.247230+0000","last_undegraded":"2023-10-06T05:33:58.247230+0000","last_fullsized":"2023-10-06T05:33:58.247230+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:26.349127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.11","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.223475+0000","last_change":"2023-10-06T05:33:57.213618+0000","last_active":"2023-10-06T05:33:58.223475+0000","last_peered":"2023-10-06T05:33:58.223475+0000","last_clean":"2023-10-06T05:33:58.223475+0000","last_became_active":"2023-10-06T05:33:57.213044+0000","last_became_peered":"2023-10-06T05:33:57.213044+0000","last_unstale":"2023-10-06T05:33:58.223475+0000","last_undegraded":"2023-10-06T05:33:58.223475+0000","last_fullsized":"2023-10-06T05:33:58.223475+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:49.583324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.1f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.234264+0000","last_change":"2023-10-06T05:33:58.232186+0000","last_active":"2023-10-06T05:33:58.234264+0000","last_peered":"2023-10-06T05:33:58.234264+0000","last_clean":"2023-10-06T05:33:58.234264+0000","last_became_active":"2023-10-06T05:33:58.231906+0000","last_became_peered":"2023-10-06T05:33:58.231906+0000","last_unstale":"2023-10-06T05:33:58.234264+0000","last_undegraded":"2023-10-06T05:33:58.234264+0000","last_fullsized":"2023-10-06T05:33:58.234264+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:01.560383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.10","version":"264'8","reported_seq":18,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.259502+0000","last_change":"2023-10-06T05:33:57.220778+0000","last_active":"2023-10-06T05:33:58.259502+0000","last_peered":"2023-10-06T05:33:58.259502+0000","last_clean":"2023-10-06T05:33:58.259502+0000","last_became_active":"2023-10-06T05:33:57.220494+0000","last_became_peered":"2023-10-06T05:33:57.220494+0000","last_unstale":"2023-10-06T05:33:58.259502+0000","last_undegraded":"2023-10-06T05:33:58.259502+0000","last_fullsized":"2023-10-06T05:33:58.259502+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:48.090866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.0","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.238775+0000","last_change":"2023-10-06T05:33:58.237078+0000","last_active":"2023-10-06T05:33:58.238775+0000","last_peered":"2023-10-06T05:33:58.238775+0000","last_clean":"2023-10-06T05:33:58.238775+0000","last_became_active":"2023-10-06T05:33:58.236774+0000","last_became_peered":"2023-10-06T05:33:58.236774+0000","last_unstale":"2023-10-06T05:33:58.238775+0000","last_undegraded":"2023-10-06T05:33:58.238775+0000","last_fullsized":"2023-10-06T05:33:58.238775+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:04.122713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.f","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257011+0000","last_change":"2023-10-06T05:33:57.220688+0000","last_active":"2023-10-06T05:33:58.257011+0000","last_peered":"2023-10-06T05:33:58.257011+0000","last_clean":"2023-10-06T05:33:58.257011+0000","last_became_active":"2023-10-06T05:33:57.220357+0000","last_became_peered":"2023-10-06T05:33:57.220357+0000","last_unstale":"2023-10-06T05:33:58.257011+0000","last_undegraded":"2023-10-06T05:33:58.257011+0000","last_fullsized":"2023-10-06T05:33:58.257011+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:04.480786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.246084+0000","last_change":"2023-10-06T05:33:58.244265+0000","last_active":"2023-10-06T05:33:58.246084+0000","last_peered":"2023-10-06T05:33:58.246084+0000","last_clean":"2023-10-06T05:33:58.246084+0000","last_became_active":"2023-10-06T05:33:58.243982+0000","last_became_peered":"2023-10-06T05:33:58.243982+0000","last_unstale":"2023-10-06T05:33:58.246084+0000","last_undegraded":"2023-10-06T05:33:58.246084+0000","last_fullsized":"2023-10-06T05:33:58.246084+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:25.280355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.e","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257408+0000","last_change":"2023-10-06T05:33:57.210535+0000","last_active":"2023-10-06T05:33:58.257408+0000","last_peered":"2023-10-06T05:33:58.257408+0000","last_clean":"2023-10-06T05:33:58.257408+0000","last_became_active":"2023-10-06T05:33:57.210009+0000","last_became_peered":"2023-10-06T05:33:57.210009+0000","last_unstale":"2023-10-06T05:33:58.257408+0000","last_undegraded":"2023-10-06T05:33:58.257408+0000","last_fullsized":"2023-10-06T05:33:58.257408+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:39.063128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.2","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230901+0000","last_change":"2023-10-06T05:33:58.224925+0000","last_active":"2023-10-06T05:33:58.230901+0000","last_peered":"2023-10-06T05:33:58.230901+0000","last_clean":"2023-10-06T05:33:58.230901+0000","last_became_active":"2023-10-06T05:33:58.224647+0000","last_became_peered":"2023-10-06T05:33:58.224647+0000","last_unstale":"2023-10-06T05:33:58.230901+0000","last_undegraded":"2023-10-06T05:33:58.230901+0000","last_fullsized":"2023-10-06T05:33:58.230901+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:33.642670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.d","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258037+0000","last_change":"2023-10-06T05:33:57.213496+0000","last_active":"2023-10-06T05:33:58.258037+0000","last_peered":"2023-10-06T05:33:58.258037+0000","last_clean":"2023-10-06T05:33:58.258037+0000","last_became_active":"2023-10-06T05:33:57.213151+0000","last_became_peered":"2023-10-06T05:33:57.213151+0000","last_unstale":"2023-10-06T05:33:58.258037+0000","last_undegraded":"2023-10-06T05:33:58.258037+0000","last_fullsized":"2023-10-06T05:33:58.258037+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:02.526806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.234364+0000","last_change":"2023-10-06T05:33:58.232155+0000","last_active":"2023-10-06T05:33:58.234364+0000","last_peered":"2023-10-06T05:33:58.234364+0000","last_clean":"2023-10-06T05:33:58.234364+0000","last_became_active":"2023-10-06T05:33:58.231820+0000","last_became_peered":"2023-10-06T05:33:58.231820+0000","last_unstale":"2023-10-06T05:33:58.234364+0000","last_undegraded":"2023-10-06T05:33:58.234364+0000","last_fullsized":"2023-10-06T05:33:58.234364+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:28.611573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.c","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.213989+0000","last_change":"2023-10-06T05:33:57.212116+0000","last_active":"2023-10-06T05:33:58.213989+0000","last_peered":"2023-10-06T05:33:58.213989+0000","last_clean":"2023-10-06T05:33:58.213989+0000","last_became_active":"2023-10-06T05:33:57.211832+0000","last_became_peered":"2023-10-06T05:33:57.211832+0000","last_unstale":"2023-10-06T05:33:58.213989+0000","last_undegraded":"2023-10-06T05:33:58.213989+0000","last_fullsized":"2023-10-06T05:33:58.213989+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:52.701119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.4","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239535+0000","last_change":"2023-10-06T05:33:58.236271+0000","last_active":"2023-10-06T05:33:58.239535+0000","last_peered":"2023-10-06T05:33:58.239535+0000","last_clean":"2023-10-06T05:33:58.239535+0000","last_became_active":"2023-10-06T05:33:58.235630+0000","last_became_peered":"2023-10-06T05:33:58.235630+0000","last_unstale":"2023-10-06T05:33:58.239535+0000","last_undegraded":"2023-10-06T05:33:58.239535+0000","last_fullsized":"2023-10-06T05:33:58.239535+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:50.694592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.b","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258891+0000","last_change":"2023-10-06T05:33:57.215743+0000","last_active":"2023-10-06T05:33:58.258891+0000","last_peered":"2023-10-06T05:33:58.258891+0000","last_clean":"2023-10-06T05:33:58.258891+0000","last_became_active":"2023-10-06T05:33:57.215471+0000","last_became_peered":"2023-10-06T05:33:57.215471+0000","last_unstale":"2023-10-06T05:33:58.258891+0000","last_undegraded":"2023-10-06T05:33:58.258891+0000","last_fullsized":"2023-10-06T05:33:58.258891+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:48.301763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.5","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239499+0000","last_change":"2023-10-06T05:33:58.237144+0000","last_active":"2023-10-06T05:33:58.239499+0000","last_peered":"2023-10-06T05:33:58.239499+0000","last_clean":"2023-10-06T05:33:58.239499+0000","last_became_active":"2023-10-06T05:33:58.236871+0000","last_became_peered":"2023-10-06T05:33:58.236871+0000","last_unstale":"2023-10-06T05:33:58.239499+0000","last_undegraded":"2023-10-06T05:33:58.239499+0000","last_fullsized":"2023-10-06T05:33:58.239499+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.867316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.a","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.220218+0000","last_change":"2023-10-06T05:33:57.219837+0000","last_active":"2023-10-06T05:33:58.220218+0000","last_peered":"2023-10-06T05:33:58.220218+0000","last_clean":"2023-10-06T05:33:58.220218+0000","last_became_active":"2023-10-06T05:33:57.219357+0000","last_became_peered":"2023-10-06T05:33:57.219357+0000","last_unstale":"2023-10-06T05:33:58.220218+0000","last_undegraded":"2023-10-06T05:33:58.220218+0000","last_fullsized":"2023-10-06T05:33:58.220218+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.615257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.6","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258804+0000","last_change":"2023-10-06T05:33:58.255923+0000","last_active":"2023-10-06T05:33:58.258804+0000","last_peered":"2023-10-06T05:33:58.258804+0000","last_clean":"2023-10-06T05:33:58.258804+0000","last_became_active":"2023-10-06T05:33:58.253621+0000","last_became_peered":"2023-10-06T05:33:58.253621+0000","last_unstale":"2023-10-06T05:33:58.258804+0000","last_undegraded":"2023-10-06T05:33:58.258804+0000","last_fullsized":"2023-10-06T05:33:58.258804+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:07.511618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.9","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.227958+0000","last_change":"2023-10-06T05:33:57.211314+0000","last_active":"2023-10-06T05:33:58.227958+0000","last_peered":"2023-10-06T05:33:58.227958+0000","last_clean":"2023-10-06T05:33:58.227958+0000","last_became_active":"2023-10-06T05:33:57.211044+0000","last_became_peered":"2023-10-06T05:33:57.211044+0000","last_unstale":"2023-10-06T05:33:58.227958+0000","last_undegraded":"2023-10-06T05:33:58.227958+0000","last_fullsized":"2023-10-06T05:33:58.227958+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:09.130716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.7","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230873+0000","last_change":"2023-10-06T05:33:58.225999+0000","last_active":"2023-10-06T05:33:58.230873+0000","last_peered":"2023-10-06T05:33:58.230873+0000","last_clean":"2023-10-06T05:33:58.230873+0000","last_became_active":"2023-10-06T05:33:58.225398+0000","last_became_peered":"2023-10-06T05:33:58.225398+0000","last_unstale":"2023-10-06T05:33:58.230873+0000","last_undegraded":"2023-10-06T05:33:58.230873+0000","last_fullsized":"2023-10-06T05:33:58.230873+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:30.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.8","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219032+0000","last_change":"2023-10-06T05:33:57.220886+0000","last_active":"2023-10-06T05:33:58.219032+0000","last_peered":"2023-10-06T05:33:58.219032+0000","last_clean":"2023-10-06T05:33:58.219032+0000","last_became_active":"2023-10-06T05:33:57.219421+0000","last_became_peered":"2023-10-06T05:33:57.219421+0000","last_unstale":"2023-10-06T05:33:58.219032+0000","last_undegraded":"2023-10-06T05:33:58.219032+0000","last_fullsized":"2023-10-06T05:33:58.219032+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:20.210307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.8","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.232466+0000","last_change":"2023-10-06T05:33:58.227972+0000","last_active":"2023-10-06T05:33:58.232466+0000","last_peered":"2023-10-06T05:33:58.232466+0000","last_clean":"2023-10-06T05:33:58.232466+0000","last_became_active":"2023-10-06T05:33:58.227695+0000","last_became_peered":"2023-10-06T05:33:58.227695+0000","last_unstale":"2023-10-06T05:33:58.232466+0000","last_undegraded":"2023-10-06T05:33:58.232466+0000","last_fullsized":"2023-10-06T05:33:58.232466+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:22.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.7","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.253455+0000","last_change":"2023-10-06T05:33:57.211470+0000","last_active":"2023-10-06T05:33:58.253455+0000","last_peered":"2023-10-06T05:33:58.253455+0000","last_clean":"2023-10-06T05:33:58.253455+0000","last_became_active":"2023-10-06T05:33:57.211200+0000","last_became_peered":"2023-10-06T05:33:57.211200+0000","last_unstale":"2023-10-06T05:33:58.253455+0000","last_undegraded":"2023-10-06T05:33:58.253455+0000","last_fullsized":"2023-10-06T05:33:58.253455+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:22.074836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.9","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.253149+0000","last_change":"2023-10-06T05:33:58.242211+0000","last_active":"2023-10-06T05:33:58.253149+0000","last_peered":"2023-10-06T05:33:58.253149+0000","last_clean":"2023-10-06T05:33:58.253149+0000","last_became_active":"2023-10-06T05:33:58.241935+0000","last_became_peered":"2023-10-06T05:33:58.241935+0000","last_unstale":"2023-10-06T05:33:58.253149+0000","last_undegraded":"2023-10-06T05:33:58.253149+0000","last_fullsized":"2023-10-06T05:33:58.253149+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:53.704721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.6","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219894+0000","last_change":"2023-10-06T05:33:57.212082+0000","last_active":"2023-10-06T05:33:58.219894+0000","last_peered":"2023-10-06T05:33:58.219894+0000","last_clean":"2023-10-06T05:33:58.219894+0000","last_became_active":"2023-10-06T05:33:57.211832+0000","last_became_peered":"2023-10-06T05:33:57.211832+0000","last_unstale":"2023-10-06T05:33:58.219894+0000","last_undegraded":"2023-10-06T05:33:58.219894+0000","last_fullsized":"2023-10-06T05:33:58.219894+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:12.712613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231240+0000","last_change":"2023-10-06T05:33:58.227756+0000","last_active":"2023-10-06T05:33:58.231240+0000","last_peered":"2023-10-06T05:33:58.231240+0000","last_clean":"2023-10-06T05:33:58.231240+0000","last_became_active":"2023-10-06T05:33:58.227483+0000","last_became_peered":"2023-10-06T05:33:58.227483+0000","last_unstale":"2023-10-06T05:33:58.231240+0000","last_undegraded":"2023-10-06T05:33:58.231240+0000","last_fullsized":"2023-10-06T05:33:58.231240+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:27.605388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.5","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.229887+0000","last_change":"2023-10-06T05:33:57.208277+0000","last_active":"2023-10-06T05:33:58.229887+0000","last_peered":"2023-10-06T05:33:58.229887+0000","last_clean":"2023-10-06T05:33:58.229887+0000","last_became_active":"2023-10-06T05:33:57.207986+0000","last_became_peered":"2023-10-06T05:33:57.207986+0000","last_unstale":"2023-10-06T05:33:58.229887+0000","last_undegraded":"2023-10-06T05:33:58.229887+0000","last_fullsized":"2023-10-06T05:33:58.229887+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:17.227104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.226172+0000","last_change":"2023-10-06T05:33:58.224556+0000","last_active":"2023-10-06T05:33:58.226172+0000","last_peered":"2023-10-06T05:33:58.226172+0000","last_clean":"2023-10-06T05:33:58.226172+0000","last_became_active":"2023-10-06T05:33:58.224236+0000","last_became_peered":"2023-10-06T05:33:58.224236+0000","last_unstale":"2023-10-06T05:33:58.226172+0000","last_undegraded":"2023-10-06T05:33:58.226172+0000","last_fullsized":"2023-10-06T05:33:58.226172+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:14.113190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.4","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.221713+0000","last_change":"2023-10-06T05:33:57.222307+0000","last_active":"2023-10-06T05:33:58.221713+0000","last_peered":"2023-10-06T05:33:58.221713+0000","last_clean":"2023-10-06T05:33:58.221713+0000","last_became_active":"2023-10-06T05:33:57.222029+0000","last_became_peered":"2023-10-06T05:33:57.222029+0000","last_unstale":"2023-10-06T05:33:58.221713+0000","last_undegraded":"2023-10-06T05:33:58.221713+0000","last_fullsized":"2023-10-06T05:33:58.221713+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:08.474193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256305+0000","last_change":"2023-10-06T05:33:58.244206+0000","last_active":"2023-10-06T05:33:58.256305+0000","last_peered":"2023-10-06T05:33:58.256305+0000","last_clean":"2023-10-06T05:33:58.256305+0000","last_became_active":"2023-10-06T05:33:58.243858+0000","last_became_peered":"2023-10-06T05:33:58.243858+0000","last_unstale":"2023-10-06T05:33:58.256305+0000","last_undegraded":"2023-10-06T05:33:58.256305+0000","last_fullsized":"2023-10-06T05:33:58.256305+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:49.094468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.3","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.233972+0000","last_change":"2023-10-06T05:33:57.217885+0000","last_active":"2023-10-06T05:33:58.233972+0000","last_peered":"2023-10-06T05:33:58.233972+0000","last_clean":"2023-10-06T05:33:58.233972+0000","last_became_active":"2023-10-06T05:33:57.217572+0000","last_became_peered":"2023-10-06T05:33:57.217572+0000","last_unstale":"2023-10-06T05:33:58.233972+0000","last_undegraded":"2023-10-06T05:33:58.233972+0000","last_fullsized":"2023-10-06T05:33:58.233972+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:17.453089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242157+0000","last_change":"2023-10-06T05:33:58.241088+0000","last_active":"2023-10-06T05:33:58.242157+0000","last_peered":"2023-10-06T05:33:58.242157+0000","last_clean":"2023-10-06T05:33:58.242157+0000","last_became_active":"2023-10-06T05:33:58.240345+0000","last_became_peered":"2023-10-06T05:33:58.240345+0000","last_unstale":"2023-10-06T05:33:58.242157+0000","last_undegraded":"2023-10-06T05:33:58.242157+0000","last_fullsized":"2023-10-06T05:33:58.242157+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:17.700406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.2","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.220377+0000","last_change":"2023-10-06T05:33:57.209493+0000","last_active":"2023-10-06T05:33:58.220377+0000","last_peered":"2023-10-06T05:33:58.220377+0000","last_clean":"2023-10-06T05:33:58.220377+0000","last_became_active":"2023-10-06T05:33:57.209210+0000","last_became_peered":"2023-10-06T05:33:57.209210+0000","last_unstale":"2023-10-06T05:33:58.220377+0000","last_undegraded":"2023-10-06T05:33:58.220377+0000","last_fullsized":"2023-10-06T05:33:58.220377+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:12.317182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.259361+0000","last_change":"2023-10-06T05:33:58.256880+0000","last_active":"2023-10-06T05:33:58.259361+0000","last_peered":"2023-10-06T05:33:58.259361+0000","last_clean":"2023-10-06T05:33:58.259361+0000","last_became_active":"2023-10-06T05:33:58.253967+0000","last_became_peered":"2023-10-06T05:33:58.253967+0000","last_unstale":"2023-10-06T05:33:58.259361+0000","last_undegraded":"2023-10-06T05:33:58.259361+0000","last_fullsized":"2023-10-06T05:33:58.259361+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:52.859346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.1","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.228053+0000","last_change":"2023-10-06T05:33:57.220447+0000","last_active":"2023-10-06T05:33:58.228053+0000","last_peered":"2023-10-06T05:33:58.228053+0000","last_clean":"2023-10-06T05:33:58.228053+0000","last_became_active":"2023-10-06T05:33:57.220080+0000","last_became_peered":"2023-10-06T05:33:57.220080+0000","last_unstale":"2023-10-06T05:33:58.228053+0000","last_undegraded":"2023-10-06T05:33:58.228053+0000","last_fullsized":"2023-10-06T05:33:58.228053+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:03.119111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239462+0000","last_change":"2023-10-06T05:33:58.227561+0000","last_active":"2023-10-06T05:33:58.239462+0000","last_peered":"2023-10-06T05:33:58.239462+0000","last_clean":"2023-10-06T05:33:58.239462+0000","last_became_active":"2023-10-06T05:33:58.226993+0000","last_became_peered":"2023-10-06T05:33:58.226993+0000","last_unstale":"2023-10-06T05:33:58.239462+0000","last_undegraded":"2023-10-06T05:33:58.239462+0000","last_fullsized":"2023-10-06T05:33:58.239462+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:32.585336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.0","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.226043+0000","last_change":"2023-10-06T05:33:57.216458+0000","last_active":"2023-10-06T05:33:58.226043+0000","last_peered":"2023-10-06T05:33:58.226043+0000","last_clean":"2023-10-06T05:33:58.226043+0000","last_became_active":"2023-10-06T05:33:57.216184+0000","last_became_peered":"2023-10-06T05:33:57.216184+0000","last_unstale":"2023-10-06T05:33:58.226043+0000","last_undegraded":"2023-10-06T05:33:58.226043+0000","last_fullsized":"2023-10-06T05:33:58.226043+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:25.968162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239606+0000","last_change":"2023-10-06T05:33:58.232572+0000","last_active":"2023-10-06T05:33:58.239606+0000","last_peered":"2023-10-06T05:33:58.239606+0000","last_clean":"2023-10-06T05:33:58.239606+0000","last_became_active":"2023-10-06T05:33:58.232314+0000","last_became_peered":"2023-10-06T05:33:58.232314+0000","last_unstale":"2023-10-06T05:33:58.239606+0000","last_undegraded":"2023-10-06T05:33:58.239606+0000","last_fullsized":"2023-10-06T05:33:58.239606+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:35.244556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":217,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.214064+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:33:58.214064+0000","last_peered":"2023-10-06T05:33:58.214064+0000","last_clean":"2023-10-06T05:33:58.214064+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:33:58.214064+0000","last_undegraded":"2023-10-06T05:33:58.214064+0000","last_fullsized":"2023-10-06T05:33:58.214064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.28","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.237409+0000","last_change":"2023-10-06T05:33:58.235984+0000","last_active":"2023-10-06T05:33:58.237409+0000","last_peered":"2023-10-06T05:33:58.237409+0000","last_clean":"2023-10-06T05:33:58.237409+0000","last_became_active":"2023-10-06T05:33:58.235473+0000","last_became_peered":"2023-10-06T05:33:58.235473+0000","last_unstale":"2023-10-06T05:33:58.237409+0000","last_undegraded":"2023-10-06T05:33:58.237409+0000","last_fullsized":"2023-10-06T05:33:58.237409+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:37.257024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":214,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.227400+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:33:58.227400+0000","last_peered":"2023-10-06T05:33:58.227400+0000","last_clean":"2023-10-06T05:33:58.227400+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:33:58.227400+0000","last_undegraded":"2023-10-06T05:33:58.227400+0000","last_fullsized":"2023-10-06T05:33:58.227400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.237685+0000","last_change":"2023-10-06T05:33:58.234853+0000","last_active":"2023-10-06T05:33:58.237685+0000","last_peered":"2023-10-06T05:33:58.237685+0000","last_clean":"2023-10-06T05:33:58.237685+0000","last_became_active":"2023-10-06T05:33:58.234451+0000","last_became_peered":"2023-10-06T05:33:58.234451+0000","last_unstale":"2023-10-06T05:33:58.237685+0000","last_undegraded":"2023-10-06T05:33:58.237685+0000","last_fullsized":"2023-10-06T05:33:58.237685+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:10.134318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.218744+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:33:58.218744+0000","last_peered":"2023-10-06T05:33:58.218744+0000","last_clean":"2023-10-06T05:33:58.218744+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:33:58.218744+0000","last_undegraded":"2023-10-06T05:33:58.218744+0000","last_fullsized":"2023-10-06T05:33:58.218744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.29","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256377+0000","last_change":"2023-10-06T05:33:58.247760+0000","last_active":"2023-10-06T05:33:58.256377+0000","last_peered":"2023-10-06T05:33:58.256377+0000","last_clean":"2023-10-06T05:33:58.256377+0000","last_became_active":"2023-10-06T05:33:58.247399+0000","last_became_peered":"2023-10-06T05:33:58.247399+0000","last_unstale":"2023-10-06T05:33:58.256377+0000","last_undegraded":"2023-10-06T05:33:58.256377+0000","last_fullsized":"2023-10-06T05:33:58.256377+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:25.280355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":214,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.224610+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:33:58.224610+0000","last_peered":"2023-10-06T05:33:58.224610+0000","last_clean":"2023-10-06T05:33:58.224610+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:33:58.224610+0000","last_undegraded":"2023-10-06T05:33:58.224610+0000","last_fullsized":"2023-10-06T05:33:58.224610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":363,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230374+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:33:58.230374+0000","last_peered":"2023-10-06T05:33:58.230374+0000","last_clean":"2023-10-06T05:33:58.230374+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:33:58.230374+0000","last_undegraded":"2023-10-06T05:33:58.230374+0000","last_fullsized":"2023-10-06T05:33:58.230374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.2a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231259+0000","last_change":"2023-10-06T05:33:58.229901+0000","last_active":"2023-10-06T05:33:58.231259+0000","last_peered":"2023-10-06T05:33:58.231259+0000","last_clean":"2023-10-06T05:33:58.231259+0000","last_became_active":"2023-10-06T05:33:58.229598+0000","last_became_peered":"2023-10-06T05:33:58.229598+0000","last_unstale":"2023-10-06T05:33:58.231259+0000","last_undegraded":"2023-10-06T05:33:58.231259+0000","last_fullsized":"2023-10-06T05:33:58.231259+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.010048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":210,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230211+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:33:58.230211+0000","last_peered":"2023-10-06T05:33:58.230211+0000","last_clean":"2023-10-06T05:33:58.230211+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:33:58.230211+0000","last_undegraded":"2023-10-06T05:33:58.230211+0000","last_fullsized":"2023-10-06T05:33:58.230211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.2f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231593+0000","last_change":"2023-10-06T05:33:58.229224+0000","last_active":"2023-10-06T05:33:58.231593+0000","last_peered":"2023-10-06T05:33:58.231593+0000","last_clean":"2023-10-06T05:33:58.231593+0000","last_became_active":"2023-10-06T05:33:58.228884+0000","last_became_peered":"2023-10-06T05:33:58.228884+0000","last_unstale":"2023-10-06T05:33:58.231593+0000","last_undegraded":"2023-10-06T05:33:58.231593+0000","last_fullsized":"2023-10-06T05:33:58.231593+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:42.856878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":210,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.229981+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:33:58.229981+0000","last_peered":"2023-10-06T05:33:58.229981+0000","last_clean":"2023-10-06T05:33:58.229981+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:33:58.229981+0000","last_undegraded":"2023-10-06T05:33:58.229981+0000","last_fullsized":"2023-10-06T05:33:58.229981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.2c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256329+0000","last_change":"2023-10-06T05:33:58.244815+0000","last_active":"2023-10-06T05:33:58.256329+0000","last_peered":"2023-10-06T05:33:58.256329+0000","last_clean":"2023-10-06T05:33:58.256329+0000","last_became_active":"2023-10-06T05:33:58.244561+0000","last_became_peered":"2023-10-06T05:33:58.244561+0000","last_unstale":"2023-10-06T05:33:58.256329+0000","last_undegraded":"2023-10-06T05:33:58.256329+0000","last_fullsized":"2023-10-06T05:33:58.256329+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:47.614963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.214345+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:33:58.214345+0000","last_peered":"2023-10-06T05:33:58.214345+0000","last_clean":"2023-10-06T05:33:58.214345+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:33:58.214345+0000","last_undegraded":"2023-10-06T05:33:58.214345+0000","last_fullsized":"2023-10-06T05:33:58.214345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.2d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256466+0000","last_change":"2023-10-06T05:33:58.245982+0000","last_active":"2023-10-06T05:33:58.256466+0000","last_peered":"2023-10-06T05:33:58.256466+0000","last_clean":"2023-10-06T05:33:58.256466+0000","last_became_active":"2023-10-06T05:33:58.245534+0000","last_became_peered":"2023-10-06T05:33:58.245534+0000","last_unstale":"2023-10-06T05:33:58.256466+0000","last_undegraded":"2023-10-06T05:33:58.256466+0000","last_fullsized":"2023-10-06T05:33:58.256466+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:47.028242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219169+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:33:58.219169+0000","last_peered":"2023-10-06T05:33:58.219169+0000","last_clean":"2023-10-06T05:33:58.219169+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:33:58.219169+0000","last_undegraded":"2023-10-06T05:33:58.219169+0000","last_fullsized":"2023-10-06T05:33:58.219169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":40,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":39,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575091,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305252,"kb_used_data":3868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066588,"statfs":{"total":96636764160,"available":96324186112,"internally_reserved":0,"allocated":3960832,"data_stored":2168625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.501,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.89200000000000002,"5min":0.97399999999999998,"15min":1.1850000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":1.0189999999999999,"15min":1.169},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.0860000000000001,"15min":1.268},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.54500000000000004,"5min":0.46700000000000003,"15min":0.38300000000000001},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.55300000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98399999999999999,"5min":1.0509999999999999,"15min":1.163},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.56599999999999995,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.22},"last":0.629}]},{"osd":3,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.47599999999999998},"max":{"1min":0.96699999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73399999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46300000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.117},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.628,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.81599999999999995,"5min":0.96399999999999997,"15min":1.258},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.234,"15min":1.2849999999999999},"last":0.754}]},{"osd":5,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":0.999,"15min":1.109},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.746},"min":{"1min":0.46700000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1279999999999999,"15min":1.2290000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.51300000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.88100000000000001,"5min":0.999,"15min":1.169},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":1.242,"15min":1.242},"last":0.79900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308764,"kb_used_data":3284,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063076,"statfs":{"total":96636764160,"available":96320589824,"internally_reserved":0,"allocated":3362816,"data_stored":1578223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.754},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54800000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.1000000000000001,"15min":1.135},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.442},"max":{"1min":1.038,"5min":1.075,"15min":1.115},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51800000000000002,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.749,"15min":0.747},"min":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":1.006,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.77400000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90700000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72799999999999998,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.143,"15min":1.143},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304668,"kb_used_data":3284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067172,"statfs":{"total":96636764160,"available":96324784128,"internally_reserved":0,"allocated":3362816,"data_stored":1578223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.26600000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.272},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":2.0030000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.28199999999999997},"max":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":1.165},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":1.22}]},{"osd":3,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.2430000000000001},"last":1.238}]},{"osd":4,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79900000000000004,"15min":0.76600000000000001},"min":{"1min":0.56999999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78600000000000003,"15min":0.77300000000000002},"min":{"1min":0.56499999999999995,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":2.0249999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.80800000000000005,"15min":0.79400000000000004},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":2.2919999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.50600000000000001},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.492}]},{"osd":6,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83199999999999996,"15min":0.82199999999999995},"min":{"1min":0.622,"5min":0.57199999999999995,"15min":0.48199999999999998},"max":{"1min":1.085,"5min":1.262,"15min":1.2909999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.52800000000000002},"max":{"1min":1.02,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":0.88600000000000001,"15min":0.83399999999999996},"min":{"1min":0.54600000000000004,"5min":0.53900000000000003,"15min":0.46999999999999997},"max":{"1min":7.0019999999999998,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":7.0019999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.85099999999999998,"15min":0.82499999999999996},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.53400000000000003},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":3.1080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575090,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309360,"kb_used_data":3880,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062480,"statfs":{"total":96636764160,"available":96319979520,"internally_reserved":0,"allocated":3973120,"data_stored":2170149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44500000000000001,"15min":0.45200000000000001},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.245},"max":{"1min":0.71199999999999997,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.438,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":0.82099999999999995},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.83099999999999996,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.318},"max":{"1min":0.70699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.47899999999999998,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51000000000000001},"max":{"1min":1.022,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":1.105,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79100000000000004,"15min":0.78800000000000003},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.008,"5min":1.1379999999999999,"15min":1.27},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.49199999999999999},"max":{"1min":1.026,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.147,"15min":1.2629999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":1.2,"15min":1.25},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80800000000000005,"15min":0.80700000000000005},"min":{"1min":0.623,"5min":0.55500000000000005,"15min":0.50600000000000001},"max":{"1min":1.073,"5min":1.1459999999999999,"15min":1.2789999999999999},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575090,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304684,"kb_used_data":3300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067156,"statfs":{"total":96636764160,"available":96324767744,"internally_reserved":0,"allocated":3379200,"data_stored":1580313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.056},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.47299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80200000000000005,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77200000000000002,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.5},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.503,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.376,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.54400000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79100000000000004,"15min":0.78800000000000003},"min":{"1min":0.67900000000000005,"5min":0.56299999999999994,"15min":0.53700000000000003},"max":{"1min":0.99099999999999999,"5min":1.2709999999999999,"15min":1.486},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.59699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.026,"5min":1.089,"15min":1.5},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.73599999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.1639999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.67500000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.048,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.70499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.97999999999999998,"5min":1.22,"15min":1.22},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.79500000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.034,"5min":1.341,"15min":1.341},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.80400000000000005,"15min":0.80000000000000004},"min":{"1min":0.72299999999999998,"5min":0.60199999999999998,"15min":0.57499999999999996},"max":{"1min":1.026,"5min":1.282,"15min":1.538},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.66400000000000003,"5min":0.54100000000000004,"15min":0.51600000000000001},"max":{"1min":1.085,"5min":1.264,"15min":1.264},"last":0.71399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575091,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308772,"kb_used_data":3292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063068,"statfs":{"total":96636764160,"available":96320581632,"internally_reserved":0,"allocated":3371008,"data_stored":1578327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44,"15min":0.45300000000000001},"min":{"1min":0.32100000000000001,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.63600000000000001,"5min":0.748,"15min":0.89200000000000002},"last":0.374},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.67900000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.504}]},{"osd":1,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":1.369},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.52500000000000002},"min":{"1min":0.40600000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.97899999999999998,"15min":1.5369999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.437,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.90200000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.46800000000000003},"max":{"1min":1.085,"5min":1.129,"15min":1.129},"last":0.878},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.5},"max":{"1min":1.087,"5min":1.101,"15min":1.3009999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.52500000000000002},"max":{"1min":1.0740000000000001,"5min":1.113,"15min":1.145},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79100000000000004,"15min":0.79000000000000004},"min":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.51100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.753}]},{"osd":6,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.0489999999999999,"5min":1.157,"15min":1.3360000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":1.069,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.57099999999999995},"max":{"1min":1.0169999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.70399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.131,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575089,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304676,"kb_used_data":3292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067164,"statfs":{"total":96636764160,"available":96324775936,"internally_reserved":0,"allocated":3371008,"data_stored":1578335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.442,"15min":0.39100000000000001},"max":{"1min":1.012,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78100000000000003,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.028,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93200000000000005,"5min":1.026,"15min":1.026},"last":0.505},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96499999999999997,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.69899999999999995},"min":{"1min":0.57499999999999996,"5min":0.47899999999999998,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.54300000000000004,"5min":0.44600000000000001,"15min":0.41799999999999998},"max":{"1min":0.89800000000000002,"5min":1.099,"15min":1.099},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.625,"5min":0.497,"15min":0.443},"max":{"1min":0.92700000000000005,"5min":3.839,"15min":3.839},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.47799999999999998},"max":{"1min":0.94399999999999995,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.496,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.71999999999999997,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.47299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.73999999999999999,"15min":0.71599999999999997},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88200000000000001,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70599999999999996,"15min":0.72599999999999998},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81499999999999995,"5min":1.0489999999999999,"15min":1.0609999999999999},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.51900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.93799999999999994,"5min":1.282,"15min":1.282},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":0.93799999999999994,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.87}]}]},{"osd":5,"up_from":13,"seq":55834575091,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304680,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067160,"statfs":{"total":96636764160,"available":96324771840,"internally_reserved":0,"allocated":3375104,"data_stored":1578937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":1.018,"15min":1.3460000000000001},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.55400000000000005,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.115,"5min":1.115,"15min":1.2130000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.47299999999999998,"15min":0.40899999999999997},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":1.024,"15min":1.204},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.57099999999999995,"5min":0.48899999999999999,"15min":0.46899999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.075,"15min":1.2769999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93899999999999995,"5min":1.02,"15min":1.1539999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.88,"5min":0.92500000000000004,"15min":1.141},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.73999999999999999},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.40899999999999997},"max":{"1min":0.99199999999999999,"5min":1.0069999999999999,"15min":1.248},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:34:02.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:34:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.918+0000 7fdb0b59e700 1 Processor -- start 2023-10-06T05:34:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.919+0000 7fdb0b59e700 1 -- start start 2023-10-06T05:34:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb0c159520 0x7fdb0c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c82a0 0x7fdb0c06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c06c820 con 0x7fdb0c0c6170 2023-10-06T05:34:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0b59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0c6610 con 0x7fdb0c159520 2023-10-06T05:34:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c06a160 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb09d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb0c0c6170 0x7fdb0c0c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59616/0 (socket says 172.21.15.33:59616) 2023-10-06T05:34:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb09d9b700 1 -- 172.21.15.33:0/3566031755 learned_addr learned my addr 172.21.15.33:0/3566031755 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0a59c700 1 --2- 172.21.15.33:0/3566031755 >> v2:172.21.15.167:3300/0 conn(0x7fdb0c159520 0x7fdb0c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.920+0000 7fdb0ad9d700 1 --2- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c82a0 0x7fdb0c06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0959a700 1 -- 172.21.15.33:0/3566031755 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642781104 0 0) 0x7fdb0c06c820 con 0x7fdb0c0c6170 2023-10-06T05:34:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0959a700 1 -- 172.21.15.33:0/3566031755 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf4003570 con 0x7fdb0c0c6170 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0a59c700 1 -- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c82a0 msgr2=0x7fdb0c06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0a59c700 1 --2- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c82a0 0x7fdb0c06a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0a59c700 1 -- 172.21.15.33:0/3566031755 >> v1:172.21.15.33:6789/0 conn(0x7fdb0c0c6170 legacy=0x7fdb0c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0959a700 1 -- 172.21.15.33:0/3566031755 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2196393985 0 0) 0x7fdaf4003570 con 0x7fdb0c0c6170 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0a59c700 1 -- 172.21.15.33:0/3566031755 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0c8800 con 0x7fdb0c159520 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.921+0000 7fdb0a59c700 1 --2- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c0c5c30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0098e0 tx=0x7fdafc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68450263742d4e76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0959a700 1 -- 172.21.15.33:0/3566031755 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00001470 con 0x7fdb0c159520 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0959a700 1 -- 172.21.15.33:0/3566031755 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc011420 con 0x7fdb0c159520 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0959a700 1 -- 172.21.15.33:0/3566031755 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0117c0 con 0x7fdb0c159520 2023-10-06T05:34:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 -- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 msgr2=0x7fdb0c0c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c0c5c30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0098e0 tx=0x7fdafc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 -- 172.21.15.33:0/3566031755 shutdown_connections 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c0c5c30 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/3566031755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c82a0 0x7fdb0c06a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 -- 172.21.15.33:0/3566031755 >> 172.21.15.33:0/3566031755 conn(0x7fdb0c0cc930 msgr2=0x7fdb0c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 -- 172.21.15.33:0/3566031755 shutdown_connections 2023-10-06T05:34:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.922+0000 7fdb0b59e700 1 -- 172.21.15.33:0/3566031755 wait complete. 2023-10-06T05:34:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.924+0000 7fdb0b59e700 1 Processor -- start 2023-10-06T05:34:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.924+0000 7fdb0b59e700 1 -- start start 2023-10-06T05:34:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c6170 0x7fdb0c0abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 0x7fdb0c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c15f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c152660 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1519e0 con 0x7fdb0c159520 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1527f0 con 0x7fdb0c0c6170 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb09d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 0x7fdb0c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb09d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 0x7fdb0c1514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54378/0 (socket says 172.21.15.33:54378) 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb09d9b700 1 -- 172.21.15.33:0/4091509409 learned_addr learned my addr 172.21.15.33:0/4091509409 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0ad9d700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.925+0000 7fdb0a59c700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c6170 0x7fdb0c0abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb09d9b700 1 -- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c6170 msgr2=0x7fdb0c0abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb09d9b700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c6170 0x7fdb0c0abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb09d9b700 1 -- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 msgr2=0x7fdb0c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb09d9b700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c15f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb09d9b700 1 -- 172.21.15.33:0/4091509409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009420 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb09d9b700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 0x7fdb0c1514a0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000cb90 tx=0x7fdb00001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00007e90 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c0add70 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb00007360 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00007700 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.926+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c0ac5b0 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.927+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c04c380 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.928+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb00004a40 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.929+0000 7fdafa7fc700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaec066f30 0x7fdaec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.929+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb00083f30 con 0x7fdb0c0c82a0 2023-10-06T05:34:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.930+0000 7fdb0a59c700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaec066f30 0x7fdaec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.930+0000 7fdb0a59c700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaec066f30 0x7fdaec0693e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0030c0 tx=0x7fdafc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:02.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:02.933+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb00050fa0 con 0x7fdb0c0c82a0 2023-10-06T05:34:03.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.131+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb0c152930 con 0x7fdaec066f30 2023-10-06T05:34:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.141+0000 7fdafa7fc700 1 -- 172.21.15.33:0/4091509409 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311566 (secure 0 0 0) 0x7fdb0c152930 con 0x7fdaec066f30 2023-10-06T05:34:03.209 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:03.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaec066f30 msgr2=0x7fdaec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaec066f30 0x7fdaec0693e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0030c0 tx=0x7fdafc016000 comp rx=0 tx=0).stop 2023-10-06T05:34:03.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 msgr2=0x7fdb0c1514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 0x7fdb0c1514a0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000cb90 tx=0x7fdb00001b50 comp rx=0 tx=0).stop 2023-10-06T05:34:03.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 shutdown_connections 2023-10-06T05:34:03.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fdaec066f30 0x7fdaec0693e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb0c159520 0x7fdb0c15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb0c0c6170 0x7fdb0c0abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 --2- 172.21.15.33:0/4091509409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb0c0c82a0 0x7fdb0c1514a0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 >> 172.21.15.33:0/4091509409 conn(0x7fdb0c0cc930 msgr2=0x7fdb0c06ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 shutdown_connections 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.145+0000 7fdb0b59e700 1 -- 172.21.15.33:0/4091509409 wait complete. 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:34:03.212 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":853,"stamp":"2023-10-06T05:34:02.121142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2450856,"kb_used_data":27496,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752523864,"statfs":{"total":773094113280,"available":770584436736,"internally_reserved":0,"allocated":28155904,"data_stored":13811132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928484"},"pg_stats":[{"pgid":"40.3f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.232305+0000","last_change":"2023-10-06T05:33:58.228484+0000","last_active":"2023-10-06T05:33:58.232305+0000","last_peered":"2023-10-06T05:33:58.232305+0000","last_clean":"2023-10-06T05:33:58.232305+0000","last_became_active":"2023-10-06T05:33:58.228229+0000","last_became_peered":"2023-10-06T05:33:58.228229+0000","last_unstale":"2023-10-06T05:33:58.232305+0000","last_undegraded":"2023-10-06T05:33:58.232305+0000","last_fullsized":"2023-10-06T05:33:58.232305+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:52.372410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.3e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.237676+0000","last_change":"2023-10-06T05:33:58.236047+0000","last_active":"2023-10-06T05:33:58.237676+0000","last_peered":"2023-10-06T05:33:58.237676+0000","last_clean":"2023-10-06T05:33:58.237676+0000","last_became_active":"2023-10-06T05:33:58.235709+0000","last_became_peered":"2023-10-06T05:33:58.235709+0000","last_unstale":"2023-10-06T05:33:58.237676+0000","last_undegraded":"2023-10-06T05:33:58.237676+0000","last_fullsized":"2023-10-06T05:33:58.237676+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:23.936802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.3d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256482+0000","last_change":"2023-10-06T05:33:58.248285+0000","last_active":"2023-10-06T05:33:58.256482+0000","last_peered":"2023-10-06T05:33:58.256482+0000","last_clean":"2023-10-06T05:33:58.256482+0000","last_became_active":"2023-10-06T05:33:58.243811+0000","last_became_peered":"2023-10-06T05:33:58.243811+0000","last_unstale":"2023-10-06T05:33:58.256482+0000","last_undegraded":"2023-10-06T05:33:58.256482+0000","last_fullsized":"2023-10-06T05:33:58.256482+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:28.718086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.3c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.236427+0000","last_change":"2023-10-06T05:33:58.231915+0000","last_active":"2023-10-06T05:33:58.236427+0000","last_peered":"2023-10-06T05:33:58.236427+0000","last_clean":"2023-10-06T05:33:58.236427+0000","last_became_active":"2023-10-06T05:33:58.231622+0000","last_became_peered":"2023-10-06T05:33:58.231622+0000","last_unstale":"2023-10-06T05:33:58.236427+0000","last_undegraded":"2023-10-06T05:33:58.236427+0000","last_fullsized":"2023-10-06T05:33:58.236427+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:33.652138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.3b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239827+0000","last_change":"2023-10-06T05:33:58.235929+0000","last_active":"2023-10-06T05:33:58.239827+0000","last_peered":"2023-10-06T05:33:58.239827+0000","last_clean":"2023-10-06T05:33:58.239827+0000","last_became_active":"2023-10-06T05:33:58.235502+0000","last_became_peered":"2023-10-06T05:33:58.235502+0000","last_unstale":"2023-10-06T05:33:58.239827+0000","last_undegraded":"2023-10-06T05:33:58.239827+0000","last_fullsized":"2023-10-06T05:33:58.239827+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:05.165142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.255037+0000","last_change":"2023-10-06T05:33:58.243182+0000","last_active":"2023-10-06T05:33:58.255037+0000","last_peered":"2023-10-06T05:33:58.255037+0000","last_clean":"2023-10-06T05:33:58.255037+0000","last_became_active":"2023-10-06T05:33:58.242929+0000","last_became_peered":"2023-10-06T05:33:58.242929+0000","last_unstale":"2023-10-06T05:33:58.255037+0000","last_undegraded":"2023-10-06T05:33:58.255037+0000","last_fullsized":"2023-10-06T05:33:58.255037+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:49.305365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.39","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242732+0000","last_change":"2023-10-06T05:33:58.236358+0000","last_active":"2023-10-06T05:33:58.242732+0000","last_peered":"2023-10-06T05:33:58.242732+0000","last_clean":"2023-10-06T05:33:58.242732+0000","last_became_active":"2023-10-06T05:33:58.236096+0000","last_became_peered":"2023-10-06T05:33:58.236096+0000","last_unstale":"2023-10-06T05:33:58.242732+0000","last_undegraded":"2023-10-06T05:33:58.242732+0000","last_fullsized":"2023-10-06T05:33:58.242732+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:50.296293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.38","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240277+0000","last_change":"2023-10-06T05:33:58.237761+0000","last_active":"2023-10-06T05:33:58.240277+0000","last_peered":"2023-10-06T05:33:58.240277+0000","last_clean":"2023-10-06T05:33:58.240277+0000","last_became_active":"2023-10-06T05:33:58.237235+0000","last_became_peered":"2023-10-06T05:33:58.237235+0000","last_unstale":"2023-10-06T05:33:58.240277+0000","last_undegraded":"2023-10-06T05:33:58.240277+0000","last_fullsized":"2023-10-06T05:33:58.240277+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:09.140740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.37","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239873+0000","last_change":"2023-10-06T05:33:58.234111+0000","last_active":"2023-10-06T05:33:58.239873+0000","last_peered":"2023-10-06T05:33:58.239873+0000","last_clean":"2023-10-06T05:33:58.239873+0000","last_became_active":"2023-10-06T05:33:58.233851+0000","last_became_peered":"2023-10-06T05:33:58.233851+0000","last_unstale":"2023-10-06T05:33:58.239873+0000","last_undegraded":"2023-10-06T05:33:58.239873+0000","last_fullsized":"2023-10-06T05:33:58.239873+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:50.639231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.36","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.235875+0000","last_change":"2023-10-06T05:33:58.224923+0000","last_active":"2023-10-06T05:33:58.235875+0000","last_peered":"2023-10-06T05:33:58.235875+0000","last_clean":"2023-10-06T05:33:58.235875+0000","last_became_active":"2023-10-06T05:33:58.224600+0000","last_became_peered":"2023-10-06T05:33:58.224600+0000","last_unstale":"2023-10-06T05:33:58.235875+0000","last_undegraded":"2023-10-06T05:33:58.235875+0000","last_fullsized":"2023-10-06T05:33:58.235875+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:58.181784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.35","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242471+0000","last_change":"2023-10-06T05:33:58.237061+0000","last_active":"2023-10-06T05:33:58.242471+0000","last_peered":"2023-10-06T05:33:58.242471+0000","last_clean":"2023-10-06T05:33:58.242471+0000","last_became_active":"2023-10-06T05:33:58.236782+0000","last_became_peered":"2023-10-06T05:33:58.236782+0000","last_unstale":"2023-10-06T05:33:58.242471+0000","last_undegraded":"2023-10-06T05:33:58.242471+0000","last_fullsized":"2023-10-06T05:33:58.242471+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:29.743815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.34","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.232454+0000","last_change":"2023-10-06T05:33:58.228085+0000","last_active":"2023-10-06T05:33:58.232454+0000","last_peered":"2023-10-06T05:33:58.232454+0000","last_clean":"2023-10-06T05:33:58.232454+0000","last_became_active":"2023-10-06T05:33:58.227272+0000","last_became_peered":"2023-10-06T05:33:58.227272+0000","last_unstale":"2023-10-06T05:33:58.232454+0000","last_undegraded":"2023-10-06T05:33:58.232454+0000","last_fullsized":"2023-10-06T05:33:58.232454+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:57.170240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.33","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257027+0000","last_change":"2023-10-06T05:33:58.255574+0000","last_active":"2023-10-06T05:33:58.257027+0000","last_peered":"2023-10-06T05:33:58.257027+0000","last_clean":"2023-10-06T05:33:58.257027+0000","last_became_active":"2023-10-06T05:33:58.255202+0000","last_became_peered":"2023-10-06T05:33:58.255202+0000","last_unstale":"2023-10-06T05:33:58.257027+0000","last_undegraded":"2023-10-06T05:33:58.257027+0000","last_fullsized":"2023-10-06T05:33:58.257027+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:21.213909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.32","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258731+0000","last_change":"2023-10-06T05:33:58.256295+0000","last_active":"2023-10-06T05:33:58.258731+0000","last_peered":"2023-10-06T05:33:58.258731+0000","last_clean":"2023-10-06T05:33:58.258731+0000","last_became_active":"2023-10-06T05:33:58.255584+0000","last_became_peered":"2023-10-06T05:33:58.255584+0000","last_unstale":"2023-10-06T05:33:58.258731+0000","last_undegraded":"2023-10-06T05:33:58.258731+0000","last_fullsized":"2023-10-06T05:33:58.258731+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:49.426905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.31","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.255606+0000","last_change":"2023-10-06T05:33:58.245395+0000","last_active":"2023-10-06T05:33:58.255606+0000","last_peered":"2023-10-06T05:33:58.255606+0000","last_clean":"2023-10-06T05:33:58.255606+0000","last_became_active":"2023-10-06T05:33:58.242865+0000","last_became_peered":"2023-10-06T05:33:58.242865+0000","last_unstale":"2023-10-06T05:33:58.255606+0000","last_undegraded":"2023-10-06T05:33:58.255606+0000","last_fullsized":"2023-10-06T05:33:58.255606+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:18.456691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.30","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257556+0000","last_change":"2023-10-06T05:33:58.244913+0000","last_active":"2023-10-06T05:33:58.257556+0000","last_peered":"2023-10-06T05:33:58.257556+0000","last_clean":"2023-10-06T05:33:58.257556+0000","last_became_active":"2023-10-06T05:33:58.244593+0000","last_became_peered":"2023-10-06T05:33:58.244593+0000","last_unstale":"2023-10-06T05:33:58.257556+0000","last_undegraded":"2023-10-06T05:33:58.257556+0000","last_fullsized":"2023-10-06T05:33:58.257556+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:03.290098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.27","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239893+0000","last_change":"2023-10-06T05:33:58.236551+0000","last_active":"2023-10-06T05:33:58.239893+0000","last_peered":"2023-10-06T05:33:58.239893+0000","last_clean":"2023-10-06T05:33:58.239893+0000","last_became_active":"2023-10-06T05:33:58.236257+0000","last_became_peered":"2023-10-06T05:33:58.236257+0000","last_unstale":"2023-10-06T05:33:58.239893+0000","last_undegraded":"2023-10-06T05:33:58.239893+0000","last_fullsized":"2023-10-06T05:33:58.239893+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:54.326226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.26","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240054+0000","last_change":"2023-10-06T05:33:58.237071+0000","last_active":"2023-10-06T05:33:58.240054+0000","last_peered":"2023-10-06T05:33:58.240054+0000","last_clean":"2023-10-06T05:33:58.240054+0000","last_became_active":"2023-10-06T05:33:58.236794+0000","last_became_peered":"2023-10-06T05:33:58.236794+0000","last_unstale":"2023-10-06T05:33:58.240054+0000","last_undegraded":"2023-10-06T05:33:58.240054+0000","last_fullsized":"2023-10-06T05:33:58.240054+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:49.645094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.25","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256503+0000","last_change":"2023-10-06T05:33:58.248772+0000","last_active":"2023-10-06T05:33:58.256503+0000","last_peered":"2023-10-06T05:33:58.256503+0000","last_clean":"2023-10-06T05:33:58.256503+0000","last_became_active":"2023-10-06T05:33:58.248509+0000","last_became_peered":"2023-10-06T05:33:58.248509+0000","last_unstale":"2023-10-06T05:33:58.256503+0000","last_undegraded":"2023-10-06T05:33:58.256503+0000","last_fullsized":"2023-10-06T05:33:58.256503+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:23.874793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.24","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.249413+0000","last_change":"2023-10-06T05:33:58.246489+0000","last_active":"2023-10-06T05:33:58.249413+0000","last_peered":"2023-10-06T05:33:58.249413+0000","last_clean":"2023-10-06T05:33:58.249413+0000","last_became_active":"2023-10-06T05:33:58.246201+0000","last_became_peered":"2023-10-06T05:33:58.246201+0000","last_unstale":"2023-10-06T05:33:58.249413+0000","last_undegraded":"2023-10-06T05:33:58.249413+0000","last_fullsized":"2023-10-06T05:33:58.249413+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:48.750654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.23","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256207+0000","last_change":"2023-10-06T05:33:58.255050+0000","last_active":"2023-10-06T05:33:58.256207+0000","last_peered":"2023-10-06T05:33:58.256207+0000","last_clean":"2023-10-06T05:33:58.256207+0000","last_became_active":"2023-10-06T05:33:58.254472+0000","last_became_peered":"2023-10-06T05:33:58.254472+0000","last_unstale":"2023-10-06T05:33:58.256207+0000","last_undegraded":"2023-10-06T05:33:58.256207+0000","last_fullsized":"2023-10-06T05:33:58.256207+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.22","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.249312+0000","last_change":"2023-10-06T05:33:58.245502+0000","last_active":"2023-10-06T05:33:58.249312+0000","last_peered":"2023-10-06T05:33:58.249312+0000","last_clean":"2023-10-06T05:33:58.249312+0000","last_became_active":"2023-10-06T05:33:58.245192+0000","last_became_peered":"2023-10-06T05:33:58.245192+0000","last_unstale":"2023-10-06T05:33:58.249312+0000","last_undegraded":"2023-10-06T05:33:58.249312+0000","last_fullsized":"2023-10-06T05:33:58.249312+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:03.651492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.21","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256259+0000","last_change":"2023-10-06T05:33:58.253322+0000","last_active":"2023-10-06T05:33:58.256259+0000","last_peered":"2023-10-06T05:33:58.256259+0000","last_clean":"2023-10-06T05:33:58.256259+0000","last_became_active":"2023-10-06T05:33:58.252965+0000","last_became_peered":"2023-10-06T05:33:58.252965+0000","last_unstale":"2023-10-06T05:33:58.256259+0000","last_undegraded":"2023-10-06T05:33:58.256259+0000","last_fullsized":"2023-10-06T05:33:58.256259+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:31.822293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.20","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.251717+0000","last_change":"2023-10-06T05:33:58.248484+0000","last_active":"2023-10-06T05:33:58.251717+0000","last_peered":"2023-10-06T05:33:58.251717+0000","last_clean":"2023-10-06T05:33:58.251717+0000","last_became_active":"2023-10-06T05:33:58.248222+0000","last_became_peered":"2023-10-06T05:33:58.248222+0000","last_unstale":"2023-10-06T05:33:58.251717+0000","last_undegraded":"2023-10-06T05:33:58.251717+0000","last_fullsized":"2023-10-06T05:33:58.251717+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:31.949214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.10","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.238371+0000","last_change":"2023-10-06T05:33:58.230228+0000","last_active":"2023-10-06T05:33:58.238371+0000","last_peered":"2023-10-06T05:33:58.238371+0000","last_clean":"2023-10-06T05:33:58.238371+0000","last_became_active":"2023-10-06T05:33:58.229966+0000","last_became_peered":"2023-10-06T05:33:58.229966+0000","last_unstale":"2023-10-06T05:33:58.238371+0000","last_undegraded":"2023-10-06T05:33:58.238371+0000","last_fullsized":"2023-10-06T05:33:58.238371+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:14.767359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.1f","version":"264'4","reported_seq":14,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258134+0000","last_change":"2023-10-06T05:33:57.211470+0000","last_active":"2023-10-06T05:33:58.258134+0000","last_peered":"2023-10-06T05:33:58.258134+0000","last_clean":"2023-10-06T05:33:58.258134+0000","last_became_active":"2023-10-06T05:33:57.211195+0000","last_became_peered":"2023-10-06T05:33:57.211195+0000","last_unstale":"2023-10-06T05:33:58.258134+0000","last_undegraded":"2023-10-06T05:33:58.258134+0000","last_fullsized":"2023-10-06T05:33:58.258134+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:25.943869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.11","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240688+0000","last_change":"2023-10-06T05:33:58.239563+0000","last_active":"2023-10-06T05:33:58.240688+0000","last_peered":"2023-10-06T05:33:58.240688+0000","last_clean":"2023-10-06T05:33:58.240688+0000","last_became_active":"2023-10-06T05:33:58.239250+0000","last_became_peered":"2023-10-06T05:33:58.239250+0000","last_unstale":"2023-10-06T05:33:58.240688+0000","last_undegraded":"2023-10-06T05:33:58.240688+0000","last_fullsized":"2023-10-06T05:33:58.240688+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:43.288728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.1e","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.254685+0000","last_change":"2023-10-06T05:33:57.208860+0000","last_active":"2023-10-06T05:33:58.254685+0000","last_peered":"2023-10-06T05:33:58.254685+0000","last_clean":"2023-10-06T05:33:58.254685+0000","last_became_active":"2023-10-06T05:33:57.208589+0000","last_became_peered":"2023-10-06T05:33:57.208589+0000","last_unstale":"2023-10-06T05:33:58.254685+0000","last_undegraded":"2023-10-06T05:33:58.254685+0000","last_fullsized":"2023-10-06T05:33:58.254685+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:48:47.258732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.12","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230170+0000","last_change":"2023-10-06T05:33:58.223688+0000","last_active":"2023-10-06T05:33:58.230170+0000","last_peered":"2023-10-06T05:33:58.230170+0000","last_clean":"2023-10-06T05:33:58.230170+0000","last_became_active":"2023-10-06T05:33:58.223356+0000","last_became_peered":"2023-10-06T05:33:58.223356+0000","last_unstale":"2023-10-06T05:33:58.230170+0000","last_undegraded":"2023-10-06T05:33:58.230170+0000","last_fullsized":"2023-10-06T05:33:58.230170+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:18.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.1d","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.254514+0000","last_change":"2023-10-06T05:33:57.208902+0000","last_active":"2023-10-06T05:33:58.254514+0000","last_peered":"2023-10-06T05:33:58.254514+0000","last_clean":"2023-10-06T05:33:58.254514+0000","last_became_active":"2023-10-06T05:33:57.208579+0000","last_became_peered":"2023-10-06T05:33:57.208579+0000","last_unstale":"2023-10-06T05:33:58.254514+0000","last_undegraded":"2023-10-06T05:33:58.254514+0000","last_fullsized":"2023-10-06T05:33:58.254514+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:47.393911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.13","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.246672+0000","last_change":"2023-10-06T05:33:58.244529+0000","last_active":"2023-10-06T05:33:58.246672+0000","last_peered":"2023-10-06T05:33:58.246672+0000","last_clean":"2023-10-06T05:33:58.246672+0000","last_became_active":"2023-10-06T05:33:58.244070+0000","last_became_peered":"2023-10-06T05:33:58.244070+0000","last_unstale":"2023-10-06T05:33:58.246672+0000","last_undegraded":"2023-10-06T05:33:58.246672+0000","last_fullsized":"2023-10-06T05:33:58.246672+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:07.590244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.1c","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.252019+0000","last_change":"2023-10-06T05:33:57.201640+0000","last_active":"2023-10-06T05:33:58.252019+0000","last_peered":"2023-10-06T05:33:58.252019+0000","last_clean":"2023-10-06T05:33:58.252019+0000","last_became_active":"2023-10-06T05:33:57.201375+0000","last_became_peered":"2023-10-06T05:33:57.201375+0000","last_unstale":"2023-10-06T05:33:58.252019+0000","last_undegraded":"2023-10-06T05:33:58.252019+0000","last_fullsized":"2023-10-06T05:33:58.252019+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:51.855744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.14","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239552+0000","last_change":"2023-10-06T05:33:58.236460+0000","last_active":"2023-10-06T05:33:58.239552+0000","last_peered":"2023-10-06T05:33:58.239552+0000","last_clean":"2023-10-06T05:33:58.239552+0000","last_became_active":"2023-10-06T05:33:58.236012+0000","last_became_peered":"2023-10-06T05:33:58.236012+0000","last_unstale":"2023-10-06T05:33:58.239552+0000","last_undegraded":"2023-10-06T05:33:58.239552+0000","last_fullsized":"2023-10-06T05:33:58.239552+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:46.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.1b","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.253654+0000","last_change":"2023-10-06T05:33:57.217712+0000","last_active":"2023-10-06T05:33:58.253654+0000","last_peered":"2023-10-06T05:33:58.253654+0000","last_clean":"2023-10-06T05:33:58.253654+0000","last_became_active":"2023-10-06T05:33:57.217464+0000","last_became_peered":"2023-10-06T05:33:57.217464+0000","last_unstale":"2023-10-06T05:33:58.253654+0000","last_undegraded":"2023-10-06T05:33:58.253654+0000","last_fullsized":"2023-10-06T05:33:58.253654+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:37.685725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.15","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.236824+0000","last_change":"2023-10-06T05:33:58.235707+0000","last_active":"2023-10-06T05:33:58.236824+0000","last_peered":"2023-10-06T05:33:58.236824+0000","last_clean":"2023-10-06T05:33:58.236824+0000","last_became_active":"2023-10-06T05:33:58.235419+0000","last_became_peered":"2023-10-06T05:33:58.235419+0000","last_unstale":"2023-10-06T05:33:58.236824+0000","last_undegraded":"2023-10-06T05:33:58.236824+0000","last_fullsized":"2023-10-06T05:33:58.236824+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:52.288996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.1a","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257457+0000","last_change":"2023-10-06T05:33:57.220422+0000","last_active":"2023-10-06T05:33:58.257457+0000","last_peered":"2023-10-06T05:33:58.257457+0000","last_clean":"2023-10-06T05:33:58.257457+0000","last_became_active":"2023-10-06T05:33:57.220134+0000","last_became_peered":"2023-10-06T05:33:57.220134+0000","last_unstale":"2023-10-06T05:33:58.257457+0000","last_undegraded":"2023-10-06T05:33:58.257457+0000","last_fullsized":"2023-10-06T05:33:58.257457+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:58.439257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.16","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231574+0000","last_change":"2023-10-06T05:33:58.227512+0000","last_active":"2023-10-06T05:33:58.231574+0000","last_peered":"2023-10-06T05:33:58.231574+0000","last_clean":"2023-10-06T05:33:58.231574+0000","last_became_active":"2023-10-06T05:33:58.227214+0000","last_became_peered":"2023-10-06T05:33:58.227214+0000","last_unstale":"2023-10-06T05:33:58.231574+0000","last_undegraded":"2023-10-06T05:33:58.231574+0000","last_fullsized":"2023-10-06T05:33:58.231574+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:38.504928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.19","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219187+0000","last_change":"2023-10-06T05:33:57.209024+0000","last_active":"2023-10-06T05:33:58.219187+0000","last_peered":"2023-10-06T05:33:58.219187+0000","last_clean":"2023-10-06T05:33:58.219187+0000","last_became_active":"2023-10-06T05:33:57.208696+0000","last_became_peered":"2023-10-06T05:33:57.208696+0000","last_unstale":"2023-10-06T05:33:58.219187+0000","last_undegraded":"2023-10-06T05:33:58.219187+0000","last_fullsized":"2023-10-06T05:33:58.219187+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:45.979360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.17","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256346+0000","last_change":"2023-10-06T05:33:58.252477+0000","last_active":"2023-10-06T05:33:58.256346+0000","last_peered":"2023-10-06T05:33:58.256346+0000","last_clean":"2023-10-06T05:33:58.256346+0000","last_became_active":"2023-10-06T05:33:58.252204+0000","last_became_peered":"2023-10-06T05:33:58.252204+0000","last_unstale":"2023-10-06T05:33:58.256346+0000","last_undegraded":"2023-10-06T05:33:58.256346+0000","last_fullsized":"2023-10-06T05:33:58.256346+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:13.846951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.18","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.220059+0000","last_change":"2023-10-06T05:33:57.212430+0000","last_active":"2023-10-06T05:33:58.220059+0000","last_peered":"2023-10-06T05:33:58.220059+0000","last_clean":"2023-10-06T05:33:58.220059+0000","last_became_active":"2023-10-06T05:33:57.212187+0000","last_became_peered":"2023-10-06T05:33:57.212187+0000","last_unstale":"2023-10-06T05:33:58.220059+0000","last_undegraded":"2023-10-06T05:33:58.220059+0000","last_fullsized":"2023-10-06T05:33:58.220059+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:28.543454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.18","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.240981+0000","last_change":"2023-10-06T05:33:58.239029+0000","last_active":"2023-10-06T05:33:58.240981+0000","last_peered":"2023-10-06T05:33:58.240981+0000","last_clean":"2023-10-06T05:33:58.240981+0000","last_became_active":"2023-10-06T05:33:58.238394+0000","last_became_peered":"2023-10-06T05:33:58.238394+0000","last_unstale":"2023-10-06T05:33:58.240981+0000","last_undegraded":"2023-10-06T05:33:58.240981+0000","last_fullsized":"2023-10-06T05:33:58.240981+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:06.769180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.17","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.223057+0000","last_change":"2023-10-06T05:33:57.209644+0000","last_active":"2023-10-06T05:33:58.223057+0000","last_peered":"2023-10-06T05:33:58.223057+0000","last_clean":"2023-10-06T05:33:58.223057+0000","last_became_active":"2023-10-06T05:33:57.209371+0000","last_became_peered":"2023-10-06T05:33:57.209371+0000","last_unstale":"2023-10-06T05:33:58.223057+0000","last_undegraded":"2023-10-06T05:33:58.223057+0000","last_fullsized":"2023-10-06T05:33:58.223057+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:09.971282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.19","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242751+0000","last_change":"2023-10-06T05:33:58.237113+0000","last_active":"2023-10-06T05:33:58.242751+0000","last_peered":"2023-10-06T05:33:58.242751+0000","last_clean":"2023-10-06T05:33:58.242751+0000","last_became_active":"2023-10-06T05:33:58.236752+0000","last_became_peered":"2023-10-06T05:33:58.236752+0000","last_unstale":"2023-10-06T05:33:58.242751+0000","last_undegraded":"2023-10-06T05:33:58.242751+0000","last_fullsized":"2023-10-06T05:33:58.242751+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:05.138335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.16","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219788+0000","last_change":"2023-10-06T05:33:57.214441+0000","last_active":"2023-10-06T05:33:58.219788+0000","last_peered":"2023-10-06T05:33:58.219788+0000","last_clean":"2023-10-06T05:33:58.219788+0000","last_became_active":"2023-10-06T05:33:57.214152+0000","last_became_peered":"2023-10-06T05:33:57.214152+0000","last_unstale":"2023-10-06T05:33:58.219788+0000","last_undegraded":"2023-10-06T05:33:58.219788+0000","last_fullsized":"2023-10-06T05:33:58.219788+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:07.238120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.1a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.236895+0000","last_change":"2023-10-06T05:33:58.235295+0000","last_active":"2023-10-06T05:33:58.236895+0000","last_peered":"2023-10-06T05:33:58.236895+0000","last_clean":"2023-10-06T05:33:58.236895+0000","last_became_active":"2023-10-06T05:33:58.234985+0000","last_became_peered":"2023-10-06T05:33:58.234985+0000","last_unstale":"2023-10-06T05:33:58.236895+0000","last_undegraded":"2023-10-06T05:33:58.236895+0000","last_fullsized":"2023-10-06T05:33:58.236895+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:26.497369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.15","version":"264'3","reported_seq":13,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.259353+0000","last_change":"2023-10-06T05:33:57.214405+0000","last_active":"2023-10-06T05:33:58.259353+0000","last_peered":"2023-10-06T05:33:58.259353+0000","last_clean":"2023-10-06T05:33:58.259353+0000","last_became_active":"2023-10-06T05:33:57.214141+0000","last_became_peered":"2023-10-06T05:33:57.214141+0000","last_unstale":"2023-10-06T05:33:58.259353+0000","last_undegraded":"2023-10-06T05:33:58.259353+0000","last_fullsized":"2023-10-06T05:33:58.259353+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:46.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.1b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256405+0000","last_change":"2023-10-06T05:33:58.246407+0000","last_active":"2023-10-06T05:33:58.256405+0000","last_peered":"2023-10-06T05:33:58.256405+0000","last_clean":"2023-10-06T05:33:58.256405+0000","last_became_active":"2023-10-06T05:33:58.246137+0000","last_became_peered":"2023-10-06T05:33:58.246137+0000","last_unstale":"2023-10-06T05:33:58.256405+0000","last_undegraded":"2023-10-06T05:33:58.256405+0000","last_fullsized":"2023-10-06T05:33:58.256405+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:51.321834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.14","version":"264'6","reported_seq":16,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257722+0000","last_change":"2023-10-06T05:33:57.219848+0000","last_active":"2023-10-06T05:33:58.257722+0000","last_peered":"2023-10-06T05:33:58.257722+0000","last_clean":"2023-10-06T05:33:58.257722+0000","last_became_active":"2023-10-06T05:33:57.219559+0000","last_became_peered":"2023-10-06T05:33:57.219559+0000","last_unstale":"2023-10-06T05:33:58.257722+0000","last_undegraded":"2023-10-06T05:33:58.257722+0000","last_fullsized":"2023-10-06T05:33:58.257722+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:01.792918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.251402+0000","last_change":"2023-10-06T05:33:58.247110+0000","last_active":"2023-10-06T05:33:58.251402+0000","last_peered":"2023-10-06T05:33:58.251402+0000","last_clean":"2023-10-06T05:33:58.251402+0000","last_became_active":"2023-10-06T05:33:58.246796+0000","last_became_peered":"2023-10-06T05:33:58.246796+0000","last_unstale":"2023-10-06T05:33:58.251402+0000","last_undegraded":"2023-10-06T05:33:58.251402+0000","last_fullsized":"2023-10-06T05:33:58.251402+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:59.153527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"39.13","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.254953+0000","last_change":"2023-10-06T05:33:57.213165+0000","last_active":"2023-10-06T05:33:58.254953+0000","last_peered":"2023-10-06T05:33:58.254953+0000","last_clean":"2023-10-06T05:33:58.254953+0000","last_became_active":"2023-10-06T05:33:57.212893+0000","last_became_peered":"2023-10-06T05:33:57.212893+0000","last_unstale":"2023-10-06T05:33:58.254953+0000","last_undegraded":"2023-10-06T05:33:58.254953+0000","last_fullsized":"2023-10-06T05:33:58.254953+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:22.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.1d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256448+0000","last_change":"2023-10-06T05:33:58.245790+0000","last_active":"2023-10-06T05:33:58.256448+0000","last_peered":"2023-10-06T05:33:58.256448+0000","last_clean":"2023-10-06T05:33:58.256448+0000","last_became_active":"2023-10-06T05:33:58.245211+0000","last_became_peered":"2023-10-06T05:33:58.245211+0000","last_unstale":"2023-10-06T05:33:58.256448+0000","last_undegraded":"2023-10-06T05:33:58.256448+0000","last_fullsized":"2023-10-06T05:33:58.256448+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:40.504784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.12","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.221624+0000","last_change":"2023-10-06T05:33:57.216297+0000","last_active":"2023-10-06T05:33:58.221624+0000","last_peered":"2023-10-06T05:33:58.221624+0000","last_clean":"2023-10-06T05:33:58.221624+0000","last_became_active":"2023-10-06T05:33:57.216034+0000","last_became_peered":"2023-10-06T05:33:57.216034+0000","last_unstale":"2023-10-06T05:33:58.221624+0000","last_undegraded":"2023-10-06T05:33:58.221624+0000","last_fullsized":"2023-10-06T05:33:58.221624+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:40.241630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.247230+0000","last_change":"2023-10-06T05:33:58.242204+0000","last_active":"2023-10-06T05:33:58.247230+0000","last_peered":"2023-10-06T05:33:58.247230+0000","last_clean":"2023-10-06T05:33:58.247230+0000","last_became_active":"2023-10-06T05:33:58.241908+0000","last_became_peered":"2023-10-06T05:33:58.241908+0000","last_unstale":"2023-10-06T05:33:58.247230+0000","last_undegraded":"2023-10-06T05:33:58.247230+0000","last_fullsized":"2023-10-06T05:33:58.247230+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:26.349127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.11","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.223475+0000","last_change":"2023-10-06T05:33:57.213618+0000","last_active":"2023-10-06T05:33:58.223475+0000","last_peered":"2023-10-06T05:33:58.223475+0000","last_clean":"2023-10-06T05:33:58.223475+0000","last_became_active":"2023-10-06T05:33:57.213044+0000","last_became_peered":"2023-10-06T05:33:57.213044+0000","last_unstale":"2023-10-06T05:33:58.223475+0000","last_undegraded":"2023-10-06T05:33:58.223475+0000","last_fullsized":"2023-10-06T05:33:58.223475+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:49.583324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.1f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.234264+0000","last_change":"2023-10-06T05:33:58.232186+0000","last_active":"2023-10-06T05:33:58.234264+0000","last_peered":"2023-10-06T05:33:58.234264+0000","last_clean":"2023-10-06T05:33:58.234264+0000","last_became_active":"2023-10-06T05:33:58.231906+0000","last_became_peered":"2023-10-06T05:33:58.231906+0000","last_unstale":"2023-10-06T05:33:58.234264+0000","last_undegraded":"2023-10-06T05:33:58.234264+0000","last_fullsized":"2023-10-06T05:33:58.234264+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:01.560383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.10","version":"264'8","reported_seq":18,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.259502+0000","last_change":"2023-10-06T05:33:57.220778+0000","last_active":"2023-10-06T05:33:58.259502+0000","last_peered":"2023-10-06T05:33:58.259502+0000","last_clean":"2023-10-06T05:33:58.259502+0000","last_became_active":"2023-10-06T05:33:57.220494+0000","last_became_peered":"2023-10-06T05:33:57.220494+0000","last_unstale":"2023-10-06T05:33:58.259502+0000","last_undegraded":"2023-10-06T05:33:58.259502+0000","last_fullsized":"2023-10-06T05:33:58.259502+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:48.090866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.0","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.238775+0000","last_change":"2023-10-06T05:33:58.237078+0000","last_active":"2023-10-06T05:33:58.238775+0000","last_peered":"2023-10-06T05:33:58.238775+0000","last_clean":"2023-10-06T05:33:58.238775+0000","last_became_active":"2023-10-06T05:33:58.236774+0000","last_became_peered":"2023-10-06T05:33:58.236774+0000","last_unstale":"2023-10-06T05:33:58.238775+0000","last_undegraded":"2023-10-06T05:33:58.238775+0000","last_fullsized":"2023-10-06T05:33:58.238775+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:04.122713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.f","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257011+0000","last_change":"2023-10-06T05:33:57.220688+0000","last_active":"2023-10-06T05:33:58.257011+0000","last_peered":"2023-10-06T05:33:58.257011+0000","last_clean":"2023-10-06T05:33:58.257011+0000","last_became_active":"2023-10-06T05:33:57.220357+0000","last_became_peered":"2023-10-06T05:33:57.220357+0000","last_unstale":"2023-10-06T05:33:58.257011+0000","last_undegraded":"2023-10-06T05:33:58.257011+0000","last_fullsized":"2023-10-06T05:33:58.257011+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:04.480786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.246084+0000","last_change":"2023-10-06T05:33:58.244265+0000","last_active":"2023-10-06T05:33:58.246084+0000","last_peered":"2023-10-06T05:33:58.246084+0000","last_clean":"2023-10-06T05:33:58.246084+0000","last_became_active":"2023-10-06T05:33:58.243982+0000","last_became_peered":"2023-10-06T05:33:58.243982+0000","last_unstale":"2023-10-06T05:33:58.246084+0000","last_undegraded":"2023-10-06T05:33:58.246084+0000","last_fullsized":"2023-10-06T05:33:58.246084+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:25.280355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.e","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.257408+0000","last_change":"2023-10-06T05:33:57.210535+0000","last_active":"2023-10-06T05:33:58.257408+0000","last_peered":"2023-10-06T05:33:58.257408+0000","last_clean":"2023-10-06T05:33:58.257408+0000","last_became_active":"2023-10-06T05:33:57.210009+0000","last_became_peered":"2023-10-06T05:33:57.210009+0000","last_unstale":"2023-10-06T05:33:58.257408+0000","last_undegraded":"2023-10-06T05:33:58.257408+0000","last_fullsized":"2023-10-06T05:33:58.257408+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:39.063128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.2","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230901+0000","last_change":"2023-10-06T05:33:58.224925+0000","last_active":"2023-10-06T05:33:58.230901+0000","last_peered":"2023-10-06T05:33:58.230901+0000","last_clean":"2023-10-06T05:33:58.230901+0000","last_became_active":"2023-10-06T05:33:58.224647+0000","last_became_peered":"2023-10-06T05:33:58.224647+0000","last_unstale":"2023-10-06T05:33:58.230901+0000","last_undegraded":"2023-10-06T05:33:58.230901+0000","last_fullsized":"2023-10-06T05:33:58.230901+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:33.642670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.d","version":"264'1","reported_seq":11,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258037+0000","last_change":"2023-10-06T05:33:57.213496+0000","last_active":"2023-10-06T05:33:58.258037+0000","last_peered":"2023-10-06T05:33:58.258037+0000","last_clean":"2023-10-06T05:33:58.258037+0000","last_became_active":"2023-10-06T05:33:57.213151+0000","last_became_peered":"2023-10-06T05:33:57.213151+0000","last_unstale":"2023-10-06T05:33:58.258037+0000","last_undegraded":"2023-10-06T05:33:58.258037+0000","last_fullsized":"2023-10-06T05:33:58.258037+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:02.526806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.234364+0000","last_change":"2023-10-06T05:33:58.232155+0000","last_active":"2023-10-06T05:33:58.234364+0000","last_peered":"2023-10-06T05:33:58.234364+0000","last_clean":"2023-10-06T05:33:58.234364+0000","last_became_active":"2023-10-06T05:33:58.231820+0000","last_became_peered":"2023-10-06T05:33:58.231820+0000","last_unstale":"2023-10-06T05:33:58.234364+0000","last_undegraded":"2023-10-06T05:33:58.234364+0000","last_fullsized":"2023-10-06T05:33:58.234364+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:28.611573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.c","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.213989+0000","last_change":"2023-10-06T05:33:57.212116+0000","last_active":"2023-10-06T05:33:58.213989+0000","last_peered":"2023-10-06T05:33:58.213989+0000","last_clean":"2023-10-06T05:33:58.213989+0000","last_became_active":"2023-10-06T05:33:57.211832+0000","last_became_peered":"2023-10-06T05:33:57.211832+0000","last_unstale":"2023-10-06T05:33:58.213989+0000","last_undegraded":"2023-10-06T05:33:58.213989+0000","last_fullsized":"2023-10-06T05:33:58.213989+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:52.701119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.4","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239535+0000","last_change":"2023-10-06T05:33:58.236271+0000","last_active":"2023-10-06T05:33:58.239535+0000","last_peered":"2023-10-06T05:33:58.239535+0000","last_clean":"2023-10-06T05:33:58.239535+0000","last_became_active":"2023-10-06T05:33:58.235630+0000","last_became_peered":"2023-10-06T05:33:58.235630+0000","last_unstale":"2023-10-06T05:33:58.239535+0000","last_undegraded":"2023-10-06T05:33:58.239535+0000","last_fullsized":"2023-10-06T05:33:58.239535+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:50.694592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.b","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258891+0000","last_change":"2023-10-06T05:33:57.215743+0000","last_active":"2023-10-06T05:33:58.258891+0000","last_peered":"2023-10-06T05:33:58.258891+0000","last_clean":"2023-10-06T05:33:58.258891+0000","last_became_active":"2023-10-06T05:33:57.215471+0000","last_became_peered":"2023-10-06T05:33:57.215471+0000","last_unstale":"2023-10-06T05:33:58.258891+0000","last_undegraded":"2023-10-06T05:33:58.258891+0000","last_fullsized":"2023-10-06T05:33:58.258891+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:48.301763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.5","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239499+0000","last_change":"2023-10-06T05:33:58.237144+0000","last_active":"2023-10-06T05:33:58.239499+0000","last_peered":"2023-10-06T05:33:58.239499+0000","last_clean":"2023-10-06T05:33:58.239499+0000","last_became_active":"2023-10-06T05:33:58.236871+0000","last_became_peered":"2023-10-06T05:33:58.236871+0000","last_unstale":"2023-10-06T05:33:58.239499+0000","last_undegraded":"2023-10-06T05:33:58.239499+0000","last_fullsized":"2023-10-06T05:33:58.239499+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.867316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.a","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.220218+0000","last_change":"2023-10-06T05:33:57.219837+0000","last_active":"2023-10-06T05:33:58.220218+0000","last_peered":"2023-10-06T05:33:58.220218+0000","last_clean":"2023-10-06T05:33:58.220218+0000","last_became_active":"2023-10-06T05:33:57.219357+0000","last_became_peered":"2023-10-06T05:33:57.219357+0000","last_unstale":"2023-10-06T05:33:58.220218+0000","last_undegraded":"2023-10-06T05:33:58.220218+0000","last_fullsized":"2023-10-06T05:33:58.220218+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:12.615257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.6","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.258804+0000","last_change":"2023-10-06T05:33:58.255923+0000","last_active":"2023-10-06T05:33:58.258804+0000","last_peered":"2023-10-06T05:33:58.258804+0000","last_clean":"2023-10-06T05:33:58.258804+0000","last_became_active":"2023-10-06T05:33:58.253621+0000","last_became_peered":"2023-10-06T05:33:58.253621+0000","last_unstale":"2023-10-06T05:33:58.258804+0000","last_undegraded":"2023-10-06T05:33:58.258804+0000","last_fullsized":"2023-10-06T05:33:58.258804+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:07.511618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.9","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.227958+0000","last_change":"2023-10-06T05:33:57.211314+0000","last_active":"2023-10-06T05:33:58.227958+0000","last_peered":"2023-10-06T05:33:58.227958+0000","last_clean":"2023-10-06T05:33:58.227958+0000","last_became_active":"2023-10-06T05:33:57.211044+0000","last_became_peered":"2023-10-06T05:33:57.211044+0000","last_unstale":"2023-10-06T05:33:58.227958+0000","last_undegraded":"2023-10-06T05:33:58.227958+0000","last_fullsized":"2023-10-06T05:33:58.227958+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:09.130716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.7","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230873+0000","last_change":"2023-10-06T05:33:58.225999+0000","last_active":"2023-10-06T05:33:58.230873+0000","last_peered":"2023-10-06T05:33:58.230873+0000","last_clean":"2023-10-06T05:33:58.230873+0000","last_became_active":"2023-10-06T05:33:58.225398+0000","last_became_peered":"2023-10-06T05:33:58.225398+0000","last_unstale":"2023-10-06T05:33:58.230873+0000","last_undegraded":"2023-10-06T05:33:58.230873+0000","last_fullsized":"2023-10-06T05:33:58.230873+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:30.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.8","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219032+0000","last_change":"2023-10-06T05:33:57.220886+0000","last_active":"2023-10-06T05:33:58.219032+0000","last_peered":"2023-10-06T05:33:58.219032+0000","last_clean":"2023-10-06T05:33:58.219032+0000","last_became_active":"2023-10-06T05:33:57.219421+0000","last_became_peered":"2023-10-06T05:33:57.219421+0000","last_unstale":"2023-10-06T05:33:58.219032+0000","last_undegraded":"2023-10-06T05:33:58.219032+0000","last_fullsized":"2023-10-06T05:33:58.219032+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:20.210307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.8","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.232466+0000","last_change":"2023-10-06T05:33:58.227972+0000","last_active":"2023-10-06T05:33:58.232466+0000","last_peered":"2023-10-06T05:33:58.232466+0000","last_clean":"2023-10-06T05:33:58.232466+0000","last_became_active":"2023-10-06T05:33:58.227695+0000","last_became_peered":"2023-10-06T05:33:58.227695+0000","last_unstale":"2023-10-06T05:33:58.232466+0000","last_undegraded":"2023-10-06T05:33:58.232466+0000","last_fullsized":"2023-10-06T05:33:58.232466+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:22.076905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.7","version":"264'2","reported_seq":12,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.253455+0000","last_change":"2023-10-06T05:33:57.211470+0000","last_active":"2023-10-06T05:33:58.253455+0000","last_peered":"2023-10-06T05:33:58.253455+0000","last_clean":"2023-10-06T05:33:58.253455+0000","last_became_active":"2023-10-06T05:33:57.211200+0000","last_became_peered":"2023-10-06T05:33:57.211200+0000","last_unstale":"2023-10-06T05:33:58.253455+0000","last_undegraded":"2023-10-06T05:33:58.253455+0000","last_fullsized":"2023-10-06T05:33:58.253455+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:22.074836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.9","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.253149+0000","last_change":"2023-10-06T05:33:58.242211+0000","last_active":"2023-10-06T05:33:58.253149+0000","last_peered":"2023-10-06T05:33:58.253149+0000","last_clean":"2023-10-06T05:33:58.253149+0000","last_became_active":"2023-10-06T05:33:58.241935+0000","last_became_peered":"2023-10-06T05:33:58.241935+0000","last_unstale":"2023-10-06T05:33:58.253149+0000","last_undegraded":"2023-10-06T05:33:58.253149+0000","last_fullsized":"2023-10-06T05:33:58.253149+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:53.704721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.6","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219894+0000","last_change":"2023-10-06T05:33:57.212082+0000","last_active":"2023-10-06T05:33:58.219894+0000","last_peered":"2023-10-06T05:33:58.219894+0000","last_clean":"2023-10-06T05:33:58.219894+0000","last_became_active":"2023-10-06T05:33:57.211832+0000","last_became_peered":"2023-10-06T05:33:57.211832+0000","last_unstale":"2023-10-06T05:33:58.219894+0000","last_undegraded":"2023-10-06T05:33:58.219894+0000","last_fullsized":"2023-10-06T05:33:58.219894+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:12.712613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231240+0000","last_change":"2023-10-06T05:33:58.227756+0000","last_active":"2023-10-06T05:33:58.231240+0000","last_peered":"2023-10-06T05:33:58.231240+0000","last_clean":"2023-10-06T05:33:58.231240+0000","last_became_active":"2023-10-06T05:33:58.227483+0000","last_became_peered":"2023-10-06T05:33:58.227483+0000","last_unstale":"2023-10-06T05:33:58.231240+0000","last_undegraded":"2023-10-06T05:33:58.231240+0000","last_fullsized":"2023-10-06T05:33:58.231240+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:27.605388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.5","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.229887+0000","last_change":"2023-10-06T05:33:57.208277+0000","last_active":"2023-10-06T05:33:58.229887+0000","last_peered":"2023-10-06T05:33:58.229887+0000","last_clean":"2023-10-06T05:33:58.229887+0000","last_became_active":"2023-10-06T05:33:57.207986+0000","last_became_peered":"2023-10-06T05:33:57.207986+0000","last_unstale":"2023-10-06T05:33:58.229887+0000","last_undegraded":"2023-10-06T05:33:58.229887+0000","last_fullsized":"2023-10-06T05:33:58.229887+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:17.227104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.226172+0000","last_change":"2023-10-06T05:33:58.224556+0000","last_active":"2023-10-06T05:33:58.226172+0000","last_peered":"2023-10-06T05:33:58.226172+0000","last_clean":"2023-10-06T05:33:58.226172+0000","last_became_active":"2023-10-06T05:33:58.224236+0000","last_became_peered":"2023-10-06T05:33:58.224236+0000","last_unstale":"2023-10-06T05:33:58.226172+0000","last_undegraded":"2023-10-06T05:33:58.226172+0000","last_fullsized":"2023-10-06T05:33:58.226172+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:14.113190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.4","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.221713+0000","last_change":"2023-10-06T05:33:57.222307+0000","last_active":"2023-10-06T05:33:58.221713+0000","last_peered":"2023-10-06T05:33:58.221713+0000","last_clean":"2023-10-06T05:33:58.221713+0000","last_became_active":"2023-10-06T05:33:57.222029+0000","last_became_peered":"2023-10-06T05:33:57.222029+0000","last_unstale":"2023-10-06T05:33:58.221713+0000","last_undegraded":"2023-10-06T05:33:58.221713+0000","last_fullsized":"2023-10-06T05:33:58.221713+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:08.474193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256305+0000","last_change":"2023-10-06T05:33:58.244206+0000","last_active":"2023-10-06T05:33:58.256305+0000","last_peered":"2023-10-06T05:33:58.256305+0000","last_clean":"2023-10-06T05:33:58.256305+0000","last_became_active":"2023-10-06T05:33:58.243858+0000","last_became_peered":"2023-10-06T05:33:58.243858+0000","last_unstale":"2023-10-06T05:33:58.256305+0000","last_undegraded":"2023-10-06T05:33:58.256305+0000","last_fullsized":"2023-10-06T05:33:58.256305+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:49.094468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.3","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.233972+0000","last_change":"2023-10-06T05:33:57.217885+0000","last_active":"2023-10-06T05:33:58.233972+0000","last_peered":"2023-10-06T05:33:58.233972+0000","last_clean":"2023-10-06T05:33:58.233972+0000","last_became_active":"2023-10-06T05:33:57.217572+0000","last_became_peered":"2023-10-06T05:33:57.217572+0000","last_unstale":"2023-10-06T05:33:58.233972+0000","last_undegraded":"2023-10-06T05:33:58.233972+0000","last_fullsized":"2023-10-06T05:33:58.233972+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:17.453089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.242157+0000","last_change":"2023-10-06T05:33:58.241088+0000","last_active":"2023-10-06T05:33:58.242157+0000","last_peered":"2023-10-06T05:33:58.242157+0000","last_clean":"2023-10-06T05:33:58.242157+0000","last_became_active":"2023-10-06T05:33:58.240345+0000","last_became_peered":"2023-10-06T05:33:58.240345+0000","last_unstale":"2023-10-06T05:33:58.242157+0000","last_undegraded":"2023-10-06T05:33:58.242157+0000","last_fullsized":"2023-10-06T05:33:58.242157+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:17.700406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.2","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.220377+0000","last_change":"2023-10-06T05:33:57.209493+0000","last_active":"2023-10-06T05:33:58.220377+0000","last_peered":"2023-10-06T05:33:58.220377+0000","last_clean":"2023-10-06T05:33:58.220377+0000","last_became_active":"2023-10-06T05:33:57.209210+0000","last_became_peered":"2023-10-06T05:33:57.209210+0000","last_unstale":"2023-10-06T05:33:58.220377+0000","last_undegraded":"2023-10-06T05:33:58.220377+0000","last_fullsized":"2023-10-06T05:33:58.220377+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:12.317182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.259361+0000","last_change":"2023-10-06T05:33:58.256880+0000","last_active":"2023-10-06T05:33:58.259361+0000","last_peered":"2023-10-06T05:33:58.259361+0000","last_clean":"2023-10-06T05:33:58.259361+0000","last_became_active":"2023-10-06T05:33:58.253967+0000","last_became_peered":"2023-10-06T05:33:58.253967+0000","last_unstale":"2023-10-06T05:33:58.259361+0000","last_undegraded":"2023-10-06T05:33:58.259361+0000","last_fullsized":"2023-10-06T05:33:58.259361+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:52.859346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.1","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.228053+0000","last_change":"2023-10-06T05:33:57.220447+0000","last_active":"2023-10-06T05:33:58.228053+0000","last_peered":"2023-10-06T05:33:58.228053+0000","last_clean":"2023-10-06T05:33:58.228053+0000","last_became_active":"2023-10-06T05:33:57.220080+0000","last_became_peered":"2023-10-06T05:33:57.220080+0000","last_unstale":"2023-10-06T05:33:58.228053+0000","last_undegraded":"2023-10-06T05:33:58.228053+0000","last_fullsized":"2023-10-06T05:33:58.228053+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:03.119111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239462+0000","last_change":"2023-10-06T05:33:58.227561+0000","last_active":"2023-10-06T05:33:58.239462+0000","last_peered":"2023-10-06T05:33:58.239462+0000","last_clean":"2023-10-06T05:33:58.239462+0000","last_became_active":"2023-10-06T05:33:58.226993+0000","last_became_peered":"2023-10-06T05:33:58.226993+0000","last_unstale":"2023-10-06T05:33:58.239462+0000","last_undegraded":"2023-10-06T05:33:58.239462+0000","last_fullsized":"2023-10-06T05:33:58.239462+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:32.585336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.0","version":"0'0","reported_seq":10,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.226043+0000","last_change":"2023-10-06T05:33:57.216458+0000","last_active":"2023-10-06T05:33:58.226043+0000","last_peered":"2023-10-06T05:33:58.226043+0000","last_clean":"2023-10-06T05:33:58.226043+0000","last_became_active":"2023-10-06T05:33:57.216184+0000","last_became_peered":"2023-10-06T05:33:57.216184+0000","last_unstale":"2023-10-06T05:33:58.226043+0000","last_undegraded":"2023-10-06T05:33:58.226043+0000","last_fullsized":"2023-10-06T05:33:58.226043+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:56.187234+0000","last_clean_scrub_stamp":"2023-10-06T05:33:56.187234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:25.968162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2e","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.239606+0000","last_change":"2023-10-06T05:33:58.232572+0000","last_active":"2023-10-06T05:33:58.239606+0000","last_peered":"2023-10-06T05:33:58.239606+0000","last_clean":"2023-10-06T05:33:58.239606+0000","last_became_active":"2023-10-06T05:33:58.232314+0000","last_became_peered":"2023-10-06T05:33:58.232314+0000","last_unstale":"2023-10-06T05:33:58.239606+0000","last_undegraded":"2023-10-06T05:33:58.239606+0000","last_fullsized":"2023-10-06T05:33:58.239606+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:35.244556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":217,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.214064+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:33:58.214064+0000","last_peered":"2023-10-06T05:33:58.214064+0000","last_clean":"2023-10-06T05:33:58.214064+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:33:58.214064+0000","last_undegraded":"2023-10-06T05:33:58.214064+0000","last_fullsized":"2023-10-06T05:33:58.214064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.28","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.237409+0000","last_change":"2023-10-06T05:33:58.235984+0000","last_active":"2023-10-06T05:33:58.237409+0000","last_peered":"2023-10-06T05:33:58.237409+0000","last_clean":"2023-10-06T05:33:58.237409+0000","last_became_active":"2023-10-06T05:33:58.235473+0000","last_became_peered":"2023-10-06T05:33:58.235473+0000","last_unstale":"2023-10-06T05:33:58.237409+0000","last_undegraded":"2023-10-06T05:33:58.237409+0000","last_fullsized":"2023-10-06T05:33:58.237409+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:37.257024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":214,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.227400+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:33:58.227400+0000","last_peered":"2023-10-06T05:33:58.227400+0000","last_clean":"2023-10-06T05:33:58.227400+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:33:58.227400+0000","last_undegraded":"2023-10-06T05:33:58.227400+0000","last_fullsized":"2023-10-06T05:33:58.227400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2b","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.237685+0000","last_change":"2023-10-06T05:33:58.234853+0000","last_active":"2023-10-06T05:33:58.237685+0000","last_peered":"2023-10-06T05:33:58.237685+0000","last_clean":"2023-10-06T05:33:58.237685+0000","last_became_active":"2023-10-06T05:33:58.234451+0000","last_became_peered":"2023-10-06T05:33:58.234451+0000","last_unstale":"2023-10-06T05:33:58.237685+0000","last_undegraded":"2023-10-06T05:33:58.237685+0000","last_fullsized":"2023-10-06T05:33:58.237685+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:10.134318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.218744+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:33:58.218744+0000","last_peered":"2023-10-06T05:33:58.218744+0000","last_clean":"2023-10-06T05:33:58.218744+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:33:58.218744+0000","last_undegraded":"2023-10-06T05:33:58.218744+0000","last_fullsized":"2023-10-06T05:33:58.218744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.29","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256377+0000","last_change":"2023-10-06T05:33:58.247760+0000","last_active":"2023-10-06T05:33:58.256377+0000","last_peered":"2023-10-06T05:33:58.256377+0000","last_clean":"2023-10-06T05:33:58.256377+0000","last_became_active":"2023-10-06T05:33:58.247399+0000","last_became_peered":"2023-10-06T05:33:58.247399+0000","last_unstale":"2023-10-06T05:33:58.256377+0000","last_undegraded":"2023-10-06T05:33:58.256377+0000","last_fullsized":"2023-10-06T05:33:58.256377+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:25.280355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":214,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.224610+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:33:58.224610+0000","last_peered":"2023-10-06T05:33:58.224610+0000","last_clean":"2023-10-06T05:33:58.224610+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:33:58.224610+0000","last_undegraded":"2023-10-06T05:33:58.224610+0000","last_fullsized":"2023-10-06T05:33:58.224610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":363,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230374+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:33:58.230374+0000","last_peered":"2023-10-06T05:33:58.230374+0000","last_clean":"2023-10-06T05:33:58.230374+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:33:58.230374+0000","last_undegraded":"2023-10-06T05:33:58.230374+0000","last_fullsized":"2023-10-06T05:33:58.230374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.2a","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231259+0000","last_change":"2023-10-06T05:33:58.229901+0000","last_active":"2023-10-06T05:33:58.231259+0000","last_peered":"2023-10-06T05:33:58.231259+0000","last_clean":"2023-10-06T05:33:58.231259+0000","last_became_active":"2023-10-06T05:33:58.229598+0000","last_became_peered":"2023-10-06T05:33:58.229598+0000","last_unstale":"2023-10-06T05:33:58.231259+0000","last_undegraded":"2023-10-06T05:33:58.231259+0000","last_fullsized":"2023-10-06T05:33:58.231259+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.010048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":210,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.230211+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:33:58.230211+0000","last_peered":"2023-10-06T05:33:58.230211+0000","last_clean":"2023-10-06T05:33:58.230211+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:33:58.230211+0000","last_undegraded":"2023-10-06T05:33:58.230211+0000","last_fullsized":"2023-10-06T05:33:58.230211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.2f","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.231593+0000","last_change":"2023-10-06T05:33:58.229224+0000","last_active":"2023-10-06T05:33:58.231593+0000","last_peered":"2023-10-06T05:33:58.231593+0000","last_clean":"2023-10-06T05:33:58.231593+0000","last_became_active":"2023-10-06T05:33:58.228884+0000","last_became_peered":"2023-10-06T05:33:58.228884+0000","last_unstale":"2023-10-06T05:33:58.231593+0000","last_undegraded":"2023-10-06T05:33:58.231593+0000","last_fullsized":"2023-10-06T05:33:58.231593+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:42.856878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":210,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.229981+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:33:58.229981+0000","last_peered":"2023-10-06T05:33:58.229981+0000","last_clean":"2023-10-06T05:33:58.229981+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:33:58.229981+0000","last_undegraded":"2023-10-06T05:33:58.229981+0000","last_fullsized":"2023-10-06T05:33:58.229981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.2c","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256329+0000","last_change":"2023-10-06T05:33:58.244815+0000","last_active":"2023-10-06T05:33:58.256329+0000","last_peered":"2023-10-06T05:33:58.256329+0000","last_clean":"2023-10-06T05:33:58.256329+0000","last_became_active":"2023-10-06T05:33:58.244561+0000","last_became_peered":"2023-10-06T05:33:58.244561+0000","last_unstale":"2023-10-06T05:33:58.256329+0000","last_undegraded":"2023-10-06T05:33:58.256329+0000","last_fullsized":"2023-10-06T05:33:58.256329+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:47.614963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.214345+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:33:58.214345+0000","last_peered":"2023-10-06T05:33:58.214345+0000","last_clean":"2023-10-06T05:33:58.214345+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:33:58.214345+0000","last_undegraded":"2023-10-06T05:33:58.214345+0000","last_fullsized":"2023-10-06T05:33:58.214345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.2d","version":"0'0","reported_seq":9,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.256466+0000","last_change":"2023-10-06T05:33:58.245982+0000","last_active":"2023-10-06T05:33:58.256466+0000","last_peered":"2023-10-06T05:33:58.256466+0000","last_clean":"2023-10-06T05:33:58.256466+0000","last_became_active":"2023-10-06T05:33:58.245534+0000","last_became_peered":"2023-10-06T05:33:58.245534+0000","last_unstale":"2023-10-06T05:33:58.256466+0000","last_undegraded":"2023-10-06T05:33:58.256466+0000","last_fullsized":"2023-10-06T05:33:58.256466+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:33:57.190836+0000","last_clean_scrub_stamp":"2023-10-06T05:33:57.190836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:47.028242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":264,"state":"active+clean","last_fresh":"2023-10-06T05:33:58.219169+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:33:58.219169+0000","last_peered":"2023-10-06T05:33:58.219169+0000","last_clean":"2023-10-06T05:33:58.219169+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:33:58.219169+0000","last_undegraded":"2023-10-06T05:33:58.219169+0000","last_fullsized":"2023-10-06T05:33:58.219169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":40,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":39,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575091,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305252,"kb_used_data":3868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066588,"statfs":{"total":96636764160,"available":96324186112,"internally_reserved":0,"allocated":3960832,"data_stored":2168625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.501,"5min":0.38800000000000001,"15min":0.32300000000000001},"max":{"1min":0.89200000000000002,"5min":0.97399999999999998,"15min":1.1850000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":1.0189999999999999,"15min":1.169},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.38900000000000001},"max":{"1min":0.875,"5min":1.0860000000000001,"15min":1.268},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.54500000000000004,"5min":0.46700000000000003,"15min":0.38300000000000001},"max":{"1min":0.93600000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.55300000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98399999999999999,"5min":1.0509999999999999,"15min":1.163},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.56599999999999995,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.22},"last":0.629}]},{"osd":3,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.47599999999999998},"max":{"1min":0.96699999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73399999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.46300000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.117},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.628,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.81599999999999995,"5min":0.96399999999999997,"15min":1.258},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39400000000000002},"max":{"1min":0.89600000000000002,"5min":1.234,"15min":1.2849999999999999},"last":0.754}]},{"osd":5,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":0.999,"15min":1.109},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.747,"15min":0.746},"min":{"1min":0.46700000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1279999999999999,"15min":1.2290000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Oct 6 05:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.51300000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"max":{"1min":0.88100000000000001,"5min":0.999,"15min":1.169},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":1.242,"15min":1.242},"last":0.79900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308764,"kb_used_data":3284,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063076,"statfs":{"total":96636764160,"available":96320589824,"internally_reserved":0,"allocated":3362816,"data_stored":1578223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.754},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54800000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.1000000000000001,"15min":1.135},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.442},"max":{"1min":1.038,"5min":1.075,"15min":1.115},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51800000000000002,"5min":0.48899999999999999,"15min":0.40899999999999997},"max":{"1min":0.95299999999999996,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.749,"15min":0.747},"min":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":1.006,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.77400000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90700000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72799999999999998,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.143,"15min":1.143},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304668,"kb_used_data":3284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067172,"statfs":{"total":96636764160,"available":96324784128,"internally_reserved":0,"allocated":3362816,"data_stored":1578223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.26600000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.272},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":2.0030000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.28199999999999997},"max":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":1.165},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":1.22}]},{"osd":3,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.2430000000000001},"last":1.238}]},{"osd":4,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79900000000000004,"15min":0.76600000000000001},"min":{"1min":0.56999999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78600000000000003,"15min":0.77300000000000002},"min":{"1min":0.56499999999999995,"5min":0.52300000000000002,"15min":0.49299999999999999},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":2.0249999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.80800000000000005,"15min":0.79400000000000004},"min":{"1min":0.54200000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":2.2919999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.50600000000000001},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.492}]},{"osd":6,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83199999999999996,"15min":0.82199999999999995},"min":{"1min":0.622,"5min":0.57199999999999995,"15min":0.48199999999999998},"max":{"1min":1.085,"5min":1.262,"15min":1.2909999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.52800000000000002},"max":{"1min":1.02,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":0.88600000000000001,"15min":0.83399999999999996},"min":{"1min":0.54600000000000004,"5min":0.53900000000000003,"15min":0.46999999999999997},"max":{"1min":7.0019999999999998,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":7.0019999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.85099999999999998,"15min":0.82499999999999996},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.53400000000000003},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":3.1080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575090,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309360,"kb_used_data":3880,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062480,"statfs":{"total":96636764160,"available":96319979520,"internally_reserved":0,"allocated":3973120,"data_stored":2170149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44500000000000001,"15min":0.45200000000000001},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.245},"max":{"1min":0.71199999999999997,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.498},"min":{"1min":0.438,"5min":0.29499999999999998,"15min":0.26900000000000002},"max":{"1min":0.77100000000000002,"5min":0.81200000000000006,"15min":0.82099999999999995},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.83099999999999996,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.318},"max":{"1min":0.70699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.47899999999999998,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51000000000000001},"max":{"1min":1.022,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":1.105,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79100000000000004,"15min":0.78800000000000003},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.008,"5min":1.1379999999999999,"15min":1.27},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.49199999999999999},"max":{"1min":1.026,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.147,"15min":1.2629999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":1.2,"15min":1.25},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80800000000000005,"15min":0.80700000000000005},"min":{"1min":0.623,"5min":0.55500000000000005,"15min":0.50600000000000001},"max":{"1min":1.073,"5min":1.1459999999999999,"15min":1.2789999999999999},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575090,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304684,"kb_used_data":3300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067156,"statfs":{"total":96636764160,"available":96324767744,"internally_reserved":0,"allocated":3379200,"data_stored":1580313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.056},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.47299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80200000000000005,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77200000000000002,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.5},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.503,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.376,"5min":0.31,"15min":0.30299999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.54400000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79100000000000004,"15min":0.78800000000000003},"min":{"1min":0.67900000000000005,"5min":0.56299999999999994,"15min":0.53700000000000003},"max":{"1min":0.99099999999999999,"5min":1.2709999999999999,"15min":1.486},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.59699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.026,"5min":1.089,"15min":1.5},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.73599999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.1639999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.67500000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.048,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.70499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.97999999999999998,"5min":1.22,"15min":1.22},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.79500000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.034,"5min":1.341,"15min":1.341},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.80400000000000005,"15min":0.80000000000000004},"min":{"1min":0.72299999999999998,"5min":0.60199999999999998,"15min":0.57499999999999996},"max":{"1min":1.026,"5min":1.282,"15min":1.538},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.66400000000000003,"5min":0.54100000000000004,"15min":0.51600000000000001},"max":{"1min":1.085,"5min":1.264,"15min":1.264},"last":0.71399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575091,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308772,"kb_used_data":3292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063068,"statfs":{"total":96636764160,"available":96320581632,"internally_reserved":0,"allocated":3371008,"data_stored":1578327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44,"15min":0.45300000000000001},"min":{"1min":0.32100000000000001,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.63600000000000001,"5min":0.748,"15min":0.89200000000000002},"last":0.374},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.67900000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.504}]},{"osd":1,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":1.369},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.52500000000000002},"min":{"1min":0.40600000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84199999999999997,"5min":0.97899999999999998,"15min":1.5369999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.437,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.90200000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.46800000000000003},"max":{"1min":1.085,"5min":1.129,"15min":1.129},"last":0.878},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.5},"max":{"1min":1.087,"5min":1.101,"15min":1.3009999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.52500000000000002},"max":{"1min":1.0740000000000001,"5min":1.113,"15min":1.145},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79100000000000004,"15min":0.79000000000000004},"min":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.51100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.753}]},{"osd":6,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53900000000000003},"max":{"1min":1.0489999999999999,"5min":1.157,"15min":1.3360000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":1.069,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Oct 6 05:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.57099999999999995},"max":{"1min":1.0169999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.70399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.131,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575089,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304676,"kb_used_data":3292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067164,"statfs":{"total":96636764160,"available":96324775936,"internally_reserved":0,"allocated":3371008,"data_stored":1578335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.72899999999999998},"min":{"1min":0.58099999999999996,"5min":0.442,"15min":0.39100000000000001},"max":{"1min":1.012,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78100000000000003,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.028,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93200000000000005,"5min":1.026,"15min":1.026},"last":0.505},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.49199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96499999999999997,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.69899999999999995},"min":{"1min":0.57499999999999996,"5min":0.47899999999999998,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.54300000000000004,"5min":0.44600000000000001,"15min":0.41799999999999998},"max":{"1min":0.89800000000000002,"5min":1.099,"15min":1.099},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.73699999999999999},"min":{"1min":0.625,"5min":0.497,"15min":0.443},"max":{"1min":0.92700000000000005,"5min":3.839,"15min":3.839},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.47799999999999998},"max":{"1min":0.94399999999999995,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.496,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.71999999999999997,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.47299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.73999999999999999,"15min":0.71599999999999997},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88200000000000001,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70599999999999996,"15min":0.72599999999999998},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81499999999999995,"5min":1.0489999999999999,"15min":1.0609999999999999},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:33:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.51900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.93799999999999994,"5min":1.282,"15min":1.282},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":0.93799999999999994,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.87}]}]},{"osd":5,"up_from":13,"seq":55834575091,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304680,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067160,"statfs":{"total":96636764160,"available":96324771840,"internally_reserved":0,"allocated":3375104,"data_stored":1578937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66500000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.39000000000000001,"15min":0.374},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":1.018,"15min":1.3460000000000001},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.55400000000000005,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.115,"5min":1.115,"15min":1.2130000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.47299999999999998,"15min":0.40899999999999997},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":1.024,"15min":1.204},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.57099999999999995,"5min":0.48899999999999999,"15min":0.46899999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.075,"15min":1.2769999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93899999999999995,"5min":1.02,"15min":1.1539999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.88,"5min":0.92500000000000004,"15min":1.141},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72999999999999998,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.73999999999999999},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.40899999999999997},"max":{"1min":0.99199999999999999,"5min":1.0069999999999999,"15min":1.248},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:34:03.219 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:34:03.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:34:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.412+0000 7f7c04f9d700 1 Processor -- start 2023-10-06T05:34:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.413+0000 7f7c04f9d700 1 -- start start 2023-10-06T05:34:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7c04f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c000c7f60 0x7f7c000c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7c04f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c000c8900 con 0x7f7c00156740 2023-10-06T05:34:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7c04f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0006b330 con 0x7f7c00156370 2023-10-06T05:34:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7c04f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0006beb0 con 0x7f7c000c7f60 2023-10-06T05:34:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7bfffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c000c7f60 0x7f7c000c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7bfeffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7c00156740 0x7f7c000cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59630/0 (socket says 172.21.15.33:59630) 2023-10-06T05:34:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.414+0000 7f7bfeffd700 1 -- 172.21.15.33:0/3084257341 learned_addr learned my addr 172.21.15.33:0/3084257341 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfffff700 1 -- 172.21.15.33:0/3084257341 >> v1:172.21.15.167:6789/0 conn(0x7f7c00156370 legacy=0x7f7c000aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfffff700 1 -- 172.21.15.33:0/3084257341 >> v1:172.21.15.33:6789/0 conn(0x7f7c00156740 legacy=0x7f7c000cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfffff700 1 -- 172.21.15.33:0/3084257341 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0006b610 con 0x7f7c000c7f60 2023-10-06T05:34:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfffff700 1 --2- 172.21.15.33:0/3084257341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c000c83c0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009e60 tx=0x7f7bf8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf1d6fa1eeab341 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/3084257341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800e070 con 0x7f7c000c7f60 2023-10-06T05:34:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/3084257341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf800bc30 con 0x7f7c000c7f60 2023-10-06T05:34:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.415+0000 7f7bfe7fc700 1 -- 172.21.15.33:0/3084257341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8012430 con 0x7f7c000c7f60 2023-10-06T05:34:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 -- 172.21.15.33:0/3084257341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 msgr2=0x7f7c000c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/3084257341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c000c83c0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009e60 tx=0x7f7bf8009780 comp rx=0 tx=0).stop 2023-10-06T05:34:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 -- 172.21.15.33:0/3084257341 shutdown_connections 2023-10-06T05:34:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/3084257341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c000c83c0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 -- 172.21.15.33:0/3084257341 >> 172.21.15.33:0/3084257341 conn(0x7f7c0006c560 msgr2=0x7f7c0006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:03.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 -- 172.21.15.33:0/3084257341 shutdown_connections 2023-10-06T05:34:03.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.416+0000 7f7c04f9d700 1 -- 172.21.15.33:0/3084257341 wait complete. 2023-10-06T05:34:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.417+0000 7f7c04f9d700 1 Processor -- start 2023-10-06T05:34:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.418+0000 7f7c04f9d700 1 -- start start 2023-10-06T05:34:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.418+0000 7f7c04f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c00153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.418+0000 7f7c04f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c00156370 0x7f7c0014a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7c04f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c00156740 0x7f7c0006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7c04f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c001520b0 con 0x7f7c000c7f60 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7c04f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00152390 con 0x7f7c00156740 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7c04f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00152670 con 0x7f7c00156370 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c00153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c00153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54386/0 (socket says 172.21.15.33:54386) 2023-10-06T05:34:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 -- 172.21.15.33:0/1314014103 learned_addr learned my addr 172.21.15.33:0/1314014103 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bfffff700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c00156740 0x7f7c0006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bfeffd700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c00156370 0x7f7c0014a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 -- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c00156370 msgr2=0x7f7c0014a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c00156370 0x7f7c0014a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 -- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c00156740 msgr2=0x7f7c0006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c00156740 0x7f7c0006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 -- 172.21.15.33:0/1314014103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8009a20 con 0x7f7c000c7f60 2023-10-06T05:34:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.419+0000 7f7bff7fe700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c00153150 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c590 tx=0x7f7bf000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.420+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0011590 con 0x7f7c000c7f60 2023-10-06T05:34:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.420+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf00116f0 con 0x7f7c000c7f60 2023-10-06T05:34:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.420+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0011ab0 con 0x7f7c000c7f60 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.420+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c00152950 con 0x7f7c000c7f60 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.420+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0014a640 con 0x7f7c000c7f60 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.421+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0004c380 con 0x7f7c000c7f60 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.422+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bf001b3b0 con 0x7f7c000c7f60 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.423+0000 7f7be77fe700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bec066f30 0x7f7bec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.423+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7bf0083c60 con 0x7f7c000c7f60 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.423+0000 7f7bfeffd700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bec066f30 0x7f7bec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.424+0000 7f7bfeffd700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bec066f30 0x7f7bec0693e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7c000b1fa0 tx=0x7f7bf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.427+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bf0050cd0 con 0x7f7c000c7f60 2023-10-06T05:34:03.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.708+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7c0006dec0 con 0x7f7c000c7f60 2023-10-06T05:34:03.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.713+0000 7f7be77fe700 1 -- 172.21.15.33:0/1314014103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7bf0017070 con 0x7f7c000c7f60 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bec066f30 msgr2=0x7f7bec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bec066f30 0x7f7bec0693e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7c000b1fa0 tx=0x7f7bf4009040 comp rx=0 tx=0).stop 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 msgr2=0x7f7c00153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c00153150 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c590 tx=0x7f7bf000c950 comp rx=0 tx=0).stop 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 shutdown_connections 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7bec066f30 0x7f7bec0693e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c00156740 0x7f7c0006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c000c7f60 0x7f7c00153150 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 --2- 172.21.15.33:0/1314014103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c00156370 0x7f7c0014a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 >> 172.21.15.33:0/1314014103 conn(0x7f7c0006c560 msgr2=0x7f7c000abfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:03.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 shutdown_connections 2023-10-06T05:34:03.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.715+0000 7f7c04f9d700 1 -- 172.21.15.33:0/1314014103 wait complete. 2023-10-06T05:34:03.718 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:34:03.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:03.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.920+0000 7f121999d700 1 Processor -- start 2023-10-06T05:34:03.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.921+0000 7f121999d700 1 -- start start 2023-10-06T05:34:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.922+0000 7f121999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214159bb0 0x7f12140b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.922+0000 7f121999d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12140cbd80 con 0x7f12140cf3b0 2023-10-06T05:34:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.922+0000 7f121999d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12140af5f0 con 0x7f12140ccfc0 2023-10-06T05:34:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.922+0000 7f121999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140af8d0 con 0x7f1214159bb0 2023-10-06T05:34:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.922+0000 7f1213fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f12140cf3b0 0x7f12141594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59634/0 (socket says 172.21.15.33:59634) 2023-10-06T05:34:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.922+0000 7f1213fff700 1 -- 172.21.15.33:0/2800769677 learned_addr learned my addr 172.21.15.33:0/2800769677 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.923+0000 7f121919c700 1 --2- 172.21.15.33:0/2800769677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214159bb0 0x7f12140b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.923+0000 7f121919c700 1 -- 172.21.15.33:0/2800769677 >> v1:172.21.15.167:6789/0 conn(0x7f12140ccfc0 legacy=0x7f12140cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.923+0000 7f121919c700 1 -- 172.21.15.33:0/2800769677 >> v1:172.21.15.33:6789/0 conn(0x7f12140cf3b0 legacy=0x7f12141594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.923+0000 7f121919c700 1 -- 172.21.15.33:0/2800769677 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140afbb0 con 0x7f1214159bb0 2023-10-06T05:34:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.924+0000 7f121919c700 1 --2- 172.21.15.33:0/2800769677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214159bb0 0x7f12140b0460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f120c009ec0 tx=0x7f120c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2a6e846ed42d95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.926+0000 7f12137fe700 1 -- 172.21.15.33:0/2800769677 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00e070 con 0x7f1214159bb0 2023-10-06T05:34:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f12137fe700 1 -- 172.21.15.33:0/2800769677 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c00bbf0 con 0x7f1214159bb0 2023-10-06T05:34:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f12137fe700 1 -- 172.21.15.33:0/2800769677 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c012430 con 0x7f1214159bb0 2023-10-06T05:34:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 -- 172.21.15.33:0/2800769677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214159bb0 msgr2=0x7f12140b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 --2- 172.21.15.33:0/2800769677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214159bb0 0x7f12140b0460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f120c009ec0 tx=0x7f120c009780 comp rx=0 tx=0).stop 2023-10-06T05:34:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 -- 172.21.15.33:0/2800769677 shutdown_connections 2023-10-06T05:34:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 --2- 172.21.15.33:0/2800769677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214159bb0 0x7f12140b0460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 -- 172.21.15.33:0/2800769677 >> 172.21.15.33:0/2800769677 conn(0x7f121406cff0 msgr2=0x7f121406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 -- 172.21.15.33:0/2800769677 shutdown_connections 2023-10-06T05:34:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.927+0000 7f121999d700 1 -- 172.21.15.33:0/2800769677 wait complete. 2023-10-06T05:34:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.929+0000 7f121999d700 1 Processor -- start 2023-10-06T05:34:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.929+0000 7f121999d700 1 -- start start 2023-10-06T05:34:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121999d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12140ccfc0 0x7f1214156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121999d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 0x7f12140b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121999d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214156540 0x7f12140cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121999d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140b3d60 con 0x7f1214159bb0 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121999d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140b4040 con 0x7f12140ccfc0 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121999d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140b4320 con 0x7f1214156540 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f1213fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 0x7f12140b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f1213fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 0x7f12140b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54398/0 (socket says 172.21.15.33:54398) 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f1213fff700 1 -- 172.21.15.33:0/1347784191 learned_addr learned my addr 172.21.15.33:0/1347784191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f121919c700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214156540 0x7f12140cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.930+0000 7f121899b700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12140ccfc0 0x7f1214156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1213fff700 1 -- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214156540 msgr2=0x7f12140cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1213fff700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214156540 0x7f12140cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1213fff700 1 -- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12140ccfc0 msgr2=0x7f1214156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1213fff700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12140ccfc0 0x7f1214156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1213fff700 1 -- 172.21.15.33:0/1347784191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c009a20 con 0x7f1214159bb0 2023-10-06T05:34:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f121919c700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214156540 0x7f12140cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f121899b700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12140ccfc0 0x7f1214156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1213fff700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 0x7f12140b32a0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f120000c5b0 tx=0x7f1200007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200005b30 con 0x7f1214159bb0 2023-10-06T05:34:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1200005cb0 con 0x7f1214159bb0 2023-10-06T05:34:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200004a80 con 0x7f1214159bb0 2023-10-06T05:34:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12140b4600 con 0x7f1214159bb0 2023-10-06T05:34:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.931+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12140b37e0 con 0x7f1214159bb0 2023-10-06T05:34:03.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.933+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1214169c60 con 0x7f1214159bb0 2023-10-06T05:34:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.933+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1200005030 con 0x7f1214159bb0 2023-10-06T05:34:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.934+0000 7f1210ff9700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11f8066f30 0x7f11f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.934+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f12000840e0 con 0x7f1214159bb0 2023-10-06T05:34:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.935+0000 7f121899b700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11f8066f30 0x7f11f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:03.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.935+0000 7f121899b700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11f8066f30 0x7f11f80693e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1208007fd0 tx=0x7f1208009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:03.939+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1214169c60 con 0x7f1214159bb0 2023-10-06T05:34:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.185+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f121404c380 con 0x7f1214159bb0 2023-10-06T05:34:04.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.185+0000 7f1210ff9700 1 -- 172.21.15.33:0/1347784191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f1200051150 con 0x7f1214159bb0 2023-10-06T05:34:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:04.187 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":203,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8921,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3798274970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3798274970},{"type":"v1","addr":"172.21.15.33:6834","nonce":3798274970}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":198},{"gid":8924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2969193715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2969193715},{"type":"v1","addr":"172.21.15.33:6835","nonce":2969193715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199},{"gid":10678,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3478690144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3478690144},{"type":"v1","addr":"172.21.15.167:6836","nonce":3478690144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:01.234255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10681":{"gid":10681,"name":"b","rank":0,"incarnation":200,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3376723254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3376723254},{"type":"v1","addr":"172.21.15.167:6837","nonce":3376723254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:04.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11f8066f30 msgr2=0x7f11f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11f8066f30 0x7f11f80693e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1208007fd0 tx=0x7f1208009040 comp rx=0 tx=0).stop 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 msgr2=0x7f12140b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 0x7f12140b32a0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f120000c5b0 tx=0x7f1200007f10 comp rx=0 tx=0).stop 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 shutdown_connections 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f11f8066f30 0x7f11f80693e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1214156540 0x7f12140cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12140ccfc0 0x7f1214156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 --2- 172.21.15.33:0/1347784191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1214159bb0 0x7f12140b32a0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 >> 172.21.15.33:0/1347784191 conn(0x7f121406cff0 msgr2=0x7f121414f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 shutdown_connections 2023-10-06T05:34:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.188+0000 7f121999d700 1 -- 172.21.15.33:0/1347784191 wait complete. 2023-10-06T05:34:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 203 2023-10-06T05:34:04.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 203, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:33:57.697950+0000', 'modified': '2023-10-06T05:34:01.234255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10681}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10681': {'gid': 10681, 'name': 'b', 'rank': 0, 'incarnation': 200, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/3376723254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 3376723254}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3376723254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [40], 'metadata_pool': 39, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:34:04.610 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:34:04.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:34:04.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.800+0000 7f4ba3fff700 1 Processor -- start 2023-10-06T05:34:04.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.800+0000 7f4ba3fff700 1 -- start start 2023-10-06T05:34:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.801+0000 7f4ba3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4ba40af600 0x7f4ba40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.801+0000 7f4ba3fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba406ec40 con 0x7f4ba41596c0 2023-10-06T05:34:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.801+0000 7f4ba3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba406dd80 con 0x7f4ba4159a90 2023-10-06T05:34:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.801+0000 7f4ba3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba40affc0 con 0x7f4ba40af600 2023-10-06T05:34:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.801+0000 7f4ba2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4ba41596c0 0x7f4ba406f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59646/0 (socket says 172.21.15.33:59646) 2023-10-06T05:34:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.801+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/3295615547 learned_addr learned my addr 172.21.15.33:0/3295615547 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:04.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/3295615547 >> v2:172.21.15.167:3300/0 conn(0x7f4ba40af600 0x7f4ba40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758867744 0 0) 0x7f4ba406ec40 con 0x7f4ba41596c0 2023-10-06T05:34:04.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b8c003570 con 0x7f4ba41596c0 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494596932 0 0) 0x7f4ba406dd80 con 0x7f4ba4159a90 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba406ec40 con 0x7f4ba4159a90 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 137180076 0 0) 0x7f4b8c003570 con 0x7f4ba41596c0 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 >> v2:172.21.15.167:3300/0 conn(0x7f4ba40af600 msgr2=0x7f4ba40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 --2- 172.21.15.33:0/3295615547 >> v2:172.21.15.167:3300/0 conn(0x7f4ba40af600 0x7f4ba40afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 >> v1:172.21.15.167:6789/0 conn(0x7f4ba4159a90 legacy=0x7f4ba406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba406e030 con 0x7f4ba41596c0 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f4b94001470 con 0x7f4ba41596c0 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 >> v1:172.21.15.33:6789/0 conn(0x7f4ba41596c0 legacy=0x7f4ba406f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b8c003c40 0x7f4b8c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4b8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8c005210 0x7f4b8c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba40affc0 con 0x7f4b8c004810 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c00c1b0 con 0x7f4b8c003c40 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c00c370 con 0x7f4b8c005210 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.802+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/3295615547 >> v2:172.21.15.167:3300/0 conn(0x7f4ba40af600 0x7f4ba40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4b8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b8c003c40 0x7f4b8c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8c005210 0x7f4b8c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 -- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8c005210 msgr2=0x7f4b8c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8c005210 0x7f4b8c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 -- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b8c003c40 msgr2=0x7f4b8c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b8c003c40 0x7f4b8c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 -- 172.21.15.33:0/3295615547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c003570 con 0x7f4b8c004810 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba37fe700 1 -- 172.21.15.33:0/3295615547 reap_dead start 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.803+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4b8c004c80 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f4b98007fd0 tx=0x7f4b98007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10a10f8b39ac3b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980013f0 con 0x7f4b8c004810 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800c5d0 con 0x7f4b8c004810 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba1ffb700 1 -- 172.21.15.33:0/3295615547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800c970 con 0x7f4b8c004810 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 -- 172.21.15.33:0/3295615547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 msgr2=0x7f4b8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4b8c004c80 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f4b98007fd0 tx=0x7f4b98007d80 comp rx=0 tx=0).stop 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 -- 172.21.15.33:0/3295615547 shutdown_connections 2023-10-06T05:34:04.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/3295615547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4b8c004c80 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 -- 172.21.15.33:0/3295615547 >> 172.21.15.33:0/3295615547 conn(0x7f4ba4087fb0 msgr2=0x7f4ba40883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 -- 172.21.15.33:0/3295615547 shutdown_connections 2023-10-06T05:34:04.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.804+0000 7f4ba3fff700 1 -- 172.21.15.33:0/3295615547 wait complete. 2023-10-06T05:34:04.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.806+0000 7f4ba3fff700 1 Processor -- start 2023-10-06T05:34:04.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.806+0000 7f4ba3fff700 1 -- start start 2023-10-06T05:34:04.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4ba4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ba40982e0 0x7f4ba409ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ba40990e0 0x7f4ba40b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba409a6a0 con 0x7f4b8c004810 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4151b80 con 0x7f4ba40990e0 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4099a90 con 0x7f4ba40982e0 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4ba4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4ba4151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54422/0 (socket says 172.21.15.33:54422) 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/283919920 learned_addr learned my addr 172.21.15.33:0/283919920 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ba40982e0 0x7f4ba409ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ba40990e0 0x7f4ba40b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ba40982e0 msgr2=0x7f4ba409ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ba40982e0 0x7f4ba409ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ba40990e0 msgr2=0x7f4ba40b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ba40990e0 0x7f4ba40b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/283919920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba4099d70 con 0x7f4b8c004810 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.807+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ba40990e0 0x7f4ba40b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.808+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4ba4151640 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f4b94007390 tx=0x7f4b94001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.808+0000 7f4b8affd700 1 -- 172.21.15.33:0/283919920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94007750 con 0x7f4b8c004810 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.808+0000 7f4b8affd700 1 -- 172.21.15.33:0/283919920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94000f50 con 0x7f4b8c004810 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.808+0000 7f4b8affd700 1 -- 172.21.15.33:0/283919920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400fb20 con 0x7f4b8c004810 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.808+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba409a050 con 0x7f4b8c004810 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.808+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba409b070 con 0x7f4b8c004810 2023-10-06T05:34:04.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.810+0000 7f4b8affd700 1 -- 172.21.15.33:0/283919920 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b9401d460 con 0x7f4b8c004810 2023-10-06T05:34:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.811+0000 7f4b8affd700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b90066f30 0x7f4b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.811+0000 7f4b8affd700 1 -- 172.21.15.33:0/283919920 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4b94084720 con 0x7f4b8c004810 2023-10-06T05:34:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.811+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b90066f30 0x7f4b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.812+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b90066f30 0x7f4b900693e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4b98000c00 tx=0x7f4b98015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.813+0000 7f4bb7cd3b80 1 Processor -- start 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.813+0000 7f4bb7cd3b80 1 -- start start 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.813+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601757a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x56017579f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560175723c80 con 0x5601757a3950 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601756faa70 con 0x5601757cce60 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601756f8270 con 0x5601757edbc0 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601757a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601757a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54426/0 (socket says 172.21.15.33:54426) 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2773399731 learned_addr learned my addr 172.21.15.33:0/2773399731 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x56017579f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 msgr2=0x5601757cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 msgr2=0x56017579f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x56017579f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2773399731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56017558e270 con 0x5601757a3950 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x56017579f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.814+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601757a3d20 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f4b94051950 tx=0x7f4b94006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb32802af432e13b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4b83fff700 1 -- 172.21.15.33:0/2773399731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94082ea0 con 0x5601757a3950 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4b83fff700 1 -- 172.21.15.33:0/2773399731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94055220 con 0x5601757a3950 2023-10-06T05:34:04.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4b83fff700 1 -- 172.21.15.33:0/2773399731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94055450 con 0x5601757a3950 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2773399731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 msgr2=0x5601757a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601757a3d20 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f4b94051950 tx=0x7f4b94006040 comp rx=0 tx=0).stop 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2773399731 shutdown_connections 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x56017579f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601757a3d20 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2773399731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2773399731 >> 172.21.15.33:0/2773399731 conn(0x5601757a5270 msgr2=0x5601757a5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.815+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2773399731 shutdown_connections 2023-10-06T05:34:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.816+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2773399731 wait complete. 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.816+0000 7f4bb7cd3b80 1 -- start start 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.816+0000 7f4bb7cd3b80 1 Processor -- start 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x5601757e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 0x5601757e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601756f4060 con 0x5601757edbc0 2023-10-06T05:34:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017569f890 con 0x5601757cce60 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601756baea0 con 0x5601757a3950 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.817+0000 7f4bb7cd3b80 10 client.0 _ll_register_callbacks cb 0x560175590f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 0x5601757e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 0x5601757e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54436/0 (socket says 172.21.15.33:54436) 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 -- 172.21.15.33:0/3831629555 learned_addr learned my addr 172.21.15.33:0/3831629555 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x5601757e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 -- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 msgr2=0x5601757fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 -- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 msgr2=0x5601757e3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x5601757e3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 -- 172.21.15.33:0/3831629555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba409b070 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 0x5601757e43c0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4159e60 tx=0x7f4b9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4eedbad079c799 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4b817fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.818+0000 7f4b817fa700 1 -- 172.21.15.33:0/3831629555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c00e040 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4b817fa700 1 -- 172.21.15.33:0/3831629555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4b9c012ca0 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560175798690 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4b817fa700 1 -- 172.21.15.33:0/3831629555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b9c012e60 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 10 client.12000 fetch_fsmap learned FSMap version 203 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) v3 -- 0x560175530fe0 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4b817fa700 1 -- 172.21.15.33:0/3831629555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560175798690 con 0x5601757edbc0 2023-10-06T05:34:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4b817fa700 1 -- 172.21.15.33:0/3831629555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 203) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4b9c014050 con 0x5601757edbc0 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 10 client.12000 fetch_fsmap finished waiting for FSMap version 203 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 10 client.12000 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 20 client.12000 populate_metadata read hostname 'smithi033' 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x56017579a8a0 0x5601757cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 4 client.12000 mds_command: new command op to 10681 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.819+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 --> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601757755c0 con 0x56017579a8a0 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.820+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x56017579a8a0 0x5601757cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.820+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x56017579a8a0 0x5601757cb010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da64a9ab23ef3e95 server_cookie=27a51924f198e65 in_seq=0 out_seq=0 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.820+0000 7f4b817fa700 10 client.12000 ms_handle_connect on v2:172.21.15.167:6835/3376723254 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4b817fa700 1 -- 172.21.15.33:0/3831629555 <== mds.0 v2:172.21.15.167:6835/3376723254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5601757755c0 con 0x56017579a8a0 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4b817fa700 10 client.12000 handle_command_reply: tid=0 2023-10-06T05:34:04.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 client.12000 shutdown 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 msgr2=0x5601757e43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 0x5601757e43c0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4159e60 tx=0x7f4b9c00b040 comp rx=0 tx=0).stop 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 shutdown_connections 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757edbc0 0x5601757e43c0 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x56017579a8a0 0x5601757cb010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/3831629555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x5601757e3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4b817fa700 0 client.12000 ms_handle_reset on v2:172.21.15.167:6835/3376723254 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.821+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 >> 172.21.15.33:0/3831629555 conn(0x5601757a5270 msgr2=0x5601757e94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.822+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 shutdown_connections 2023-10-06T05:34:04.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.822+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/3831629555 wait complete. 2023-10-06T05:34:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.822+0000 7f4bb7cd3b80 20 client.12000 trim_cache size 0 max 16384 2023-10-06T05:34:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.839+0000 7f4bb7cd3b80 1 Processor -- start 2023-10-06T05:34:04.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 -- start start 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 0x5601757e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757edbc0 0x5601757cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601757a2c40 con 0x5601757cce60 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560175576e30 con 0x5601757edbc0 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.840+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560175783030 con 0x5601757a3950 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 0x5601757e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 0x5601757e3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54438/0 (socket says 172.21.15.33:54438) 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 -- 172.21.15.33:0/319007694 learned_addr learned my addr 172.21.15.33:0/319007694 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 -- 172.21.15.33:0/319007694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 msgr2=0x5601757d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 -- 172.21.15.33:0/319007694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757edbc0 msgr2=0x5601757cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757edbc0 0x5601757cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 -- 172.21.15.33:0/319007694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601757a5910 con 0x5601757cce60 2023-10-06T05:34:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.841+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 0x5601757e3d40 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f4b98001430 tx=0x7f4b98021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb46503b9d64b2a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4b88ff9700 1 -- 172.21.15.33:0/319007694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98030070 con 0x5601757cce60 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4b88ff9700 1 -- 172.21.15.33:0/319007694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b98023070 con 0x5601757cce60 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4b88ff9700 1 -- 172.21.15.33:0/319007694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980069c0 con 0x5601757cce60 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/319007694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 msgr2=0x5601757e3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 0x5601757e3d40 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f4b98001430 tx=0x7f4b98021070 comp rx=0 tx=0).stop 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/319007694 shutdown_connections 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757edbc0 0x5601757cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757a3950 0x5601757d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/319007694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757cce60 0x5601757e3d40 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/319007694 >> 172.21.15.33:0/319007694 conn(0x5601757a5270 msgr2=0x5601757cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/319007694 shutdown_connections 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.842+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/319007694 wait complete. 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.843+0000 7f4bb7cd3b80 1 -- start start 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.843+0000 7f4bb7cd3b80 1 Processor -- start 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601758432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x560175849150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601757ccbb0 con 0x5601757a3950 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560175847bf0 con 0x5601757cce60 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x560175849b50 con 0x5601757edbc0 2023-10-06T05:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4bb7cd3b80 10 client.0 _ll_register_callbacks cb 0x560175590f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601758432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601758432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54440/0 (socket says 172.21.15.33:54440) 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2678566918 learned_addr learned my addr 172.21.15.33:0/2678566918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.844+0000 7f4ba37fe700 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 msgr2=0x5601757e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757e3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 msgr2=0x560175849150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x560175849150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4ba2ffd700 1 -- 172.21.15.33:0/2678566918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c003570 con 0x5601757a3950 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4ba2ffd700 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601758432b0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f4b94055390 tx=0x7f4b94020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1995fcf529370e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4b817fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4b817fa700 1 -- 172.21.15.33:0/2678566918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94055450 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4b817fa700 1 -- 172.21.15.33:0/2678566918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4b940586a0 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b94055220 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4b817fa700 1 -- 172.21.15.33:0/2678566918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b940556b0 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4bb7cd3b80 10 client.12006 fetch_fsmap learned FSMap version 203 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) v3 -- 0x5601756c2780 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.845+0000 7f4b817fa700 1 -- 172.21.15.33:0/2678566918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94061080 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4b817fa700 1 -- 172.21.15.33:0/2678566918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 203) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4b94081060 con 0x5601757a3950 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4bb7cd3b80 10 client.12006 fetch_fsmap finished waiting for FSMap version 203 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4bb7cd3b80 10 client.12006 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:34:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4bb7cd3b80 20 client.12006 populate_metadata read hostname 'smithi033' 2023-10-06T05:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x560175846c30 0x560175847000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4bb7cd3b80 4 client.12006 mds_command: new command op to 10681 tid=0[{"prefix": "status"}] 2023-10-06T05:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 --> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560175696e00 con 0x560175846c30 2023-10-06T05:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.846+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x560175846c30 0x560175847000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4ba27fc700 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x560175846c30 0x560175847000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c67dd491c7dcce server_cookie=8a165e90dcdde21a in_seq=0 out_seq=0 2023-10-06T05:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4b817fa700 10 client.12006 ms_handle_connect on v2:172.21.15.167:6835/3376723254 2023-10-06T05:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4b817fa700 1 -- 172.21.15.33:0/2678566918 <== mds.0 v2:172.21.15.167:6835/3376723254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x560175696e00 con 0x560175846c30 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4b817fa700 10 client.12006 handle_command_reply: tid=0 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 client.12006 shutdown 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 msgr2=0x5601758432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601758432b0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f4b94055390 tx=0x7f4b94020040 comp rx=0 tx=0).stop 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 shutdown_connections 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:6835/3376723254,v1:172.21.15.167:6837/3376723254] conn(0x560175846c30 0x560175847000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5601757edbc0 0x5601757e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601757a3950 0x5601758432b0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4b817fa700 0 client.12006 ms_handle_reset on v2:172.21.15.167:6835/3376723254 2023-10-06T05:34:04.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 --2- 172.21.15.33:0/2678566918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5601757cce60 0x560175849150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 >> 172.21.15.33:0/2678566918 conn(0x5601757a5270 msgr2=0x5601757e1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 shutdown_connections 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.847+0000 7f4bb7cd3b80 1 -- 172.21.15.33:0/2678566918 wait complete. 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4bb7cd3b80 20 client.12006 trim_cache size 0 max 16384 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 10681, 2023-10-06T05:34:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.6200312309999996, 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 203, 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 265, 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 265, 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.351648269 2023-10-06T05:34:04.856 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b90066f30 msgr2=0x7f4b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b90066f30 0x7f4b900693e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4b98000c00 tx=0x7f4b98015000 comp rx=0 tx=0).stop 2023-10-06T05:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 msgr2=0x7f4ba4151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4ba4151640 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f4b94007390 tx=0x7f4b94001d20 comp rx=0 tx=0).stop 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 shutdown_connections 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b90066f30 0x7f4b900693e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ba40990e0 0x7f4ba40b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8c004810 0x7f4ba4151640 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 --2- 172.21.15.33:0/283919920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ba40982e0 0x7f4ba409ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 >> 172.21.15.33:0/283919920 conn(0x7f4ba4087fb0 msgr2=0x7f4ba40b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 shutdown_connections 2023-10-06T05:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:04.848+0000 7f4ba3fff700 1 -- 172.21.15.33:0/283919920 wait complete. 2023-10-06T05:34:04.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:34:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.052+0000 7f9a9659d700 1 Processor -- start 2023-10-06T05:34:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.052+0000 7f9a9659d700 1 -- start start 2023-10-06T05:34:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.053+0000 7f9a9659d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a900ccfc0 0x7f9a900cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.053+0000 7f9a9659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900cbd80 con 0x7f9a900cf3b0 2023-10-06T05:34:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.053+0000 7f9a9659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900af5f0 con 0x7f9a900cc750 2023-10-06T05:34:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.053+0000 7f9a9659d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900af8d0 con 0x7f9a900ccfc0 2023-10-06T05:34:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a9559b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9a900ccfc0 0x7f9a900cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a94d9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9a900cf3b0 0x7f9a901594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59660/0 (socket says 172.21.15.33:59660) 2023-10-06T05:34:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a94d9a700 1 -- 172.21.15.33:0/1497672307 learned_addr learned my addr 172.21.15.33:0/1497672307 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a9559b700 1 -- 172.21.15.33:0/1497672307 >> v1:172.21.15.167:6789/0 conn(0x7f9a900cc750 legacy=0x7f9a900b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a9559b700 1 -- 172.21.15.33:0/1497672307 >> v1:172.21.15.33:6789/0 conn(0x7f9a900cf3b0 legacy=0x7f9a901594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a87fff700 1 -- 172.21.15.33:0/1497672307 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135712179 0 0) 0x7f9a900cbd80 con 0x7f9a900cf3b0 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a9559b700 1 -- 172.21.15.33:0/1497672307 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900afbb0 con 0x7f9a900ccfc0 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.054+0000 7f9a9559b700 1 --2- 172.21.15.33:0/1497672307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900ccfc0 0x7f9a900cc210 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009910 tx=0x7f9a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4149f07bb5a2849 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a87fff700 1 -- 172.21.15.33:0/1497672307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00b820 con 0x7f9a900ccfc0 2023-10-06T05:34:05.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a87fff700 1 -- 172.21.15.33:0/1497672307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00b980 con 0x7f9a900ccfc0 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a87fff700 1 -- 172.21.15.33:0/1497672307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00bd20 con 0x7f9a900ccfc0 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 -- 172.21.15.33:0/1497672307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900ccfc0 msgr2=0x7f9a900cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1497672307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900ccfc0 0x7f9a900cc210 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009910 tx=0x7f9a8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 -- 172.21.15.33:0/1497672307 shutdown_connections 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1497672307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900ccfc0 0x7f9a900cc210 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 -- 172.21.15.33:0/1497672307 >> 172.21.15.33:0/1497672307 conn(0x7f9a9006cff0 msgr2=0x7f9a9006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 -- 172.21.15.33:0/1497672307 shutdown_connections 2023-10-06T05:34:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.055+0000 7f9a9659d700 1 -- 172.21.15.33:0/1497672307 wait complete. 2023-10-06T05:34:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.057+0000 7f9a9659d700 1 Processor -- start 2023-10-06T05:34:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.057+0000 7f9a9659d700 1 -- start start 2023-10-06T05:34:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 0x7f9a9016df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a900ccfc0 0x7f9a90169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a900c5d40 0x7f9a900c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900c66f0 con 0x7f9a900cc750 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900b3350 con 0x7f9a900c5d40 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9016e490 con 0x7f9a900ccfc0 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 0x7f9a9016df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9559b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 0x7f9a9016df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54470/0 (socket says 172.21.15.33:54470) 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9559b700 1 -- 172.21.15.33:0/1062659110 learned_addr learned my addr 172.21.15.33:0/1062659110 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a94d9a700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a900ccfc0 0x7f9a90169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a95d9c700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a900c5d40 0x7f9a900c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9559b700 1 -- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a900ccfc0 msgr2=0x7f9a90169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.058+0000 7f9a9559b700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a900ccfc0 0x7f9a90169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a9559b700 1 -- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a900c5d40 msgr2=0x7f9a900c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a9559b700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a900c5d40 0x7f9a900c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a9559b700 1 -- 172.21.15.33:0/1062659110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c009420 con 0x7f9a900cc750 2023-10-06T05:34:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a9559b700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 0x7f9a9016df50 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0093f0 tx=0x7f9a8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c01f070 con 0x7f9a900cc750 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00baf0 con 0x7f9a900cc750 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a900b35a0 con 0x7f9a900cc750 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c025420 con 0x7f9a900cc750 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.059+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a900b3b60 con 0x7f9a900cc750 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.060+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9004c380 con 0x7f9a900cc750 2023-10-06T05:34:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.061+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a8c023070 con 0x7f9a900cc750 2023-10-06T05:34:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.062+0000 7f9a857fa700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a74066f30 0x7f9a740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.063+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9a8c085d20 con 0x7f9a900cc750 2023-10-06T05:34:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.063+0000 7f9a94d9a700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a74066f30 0x7f9a740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.064+0000 7f9a94d9a700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a74066f30 0x7f9a740693e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9a9006d950 tx=0x7f9a80007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.067+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a8c052d90 con 0x7f9a900cc750 2023-10-06T05:34:05.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.315+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a900b62d0 con 0x7f9a900cc750 2023-10-06T05:34:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.315+0000 7f9a857fa700 1 -- 172.21.15.33:0/1062659110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v203) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9a8c02a070 con 0x7f9a900cc750 2023-10-06T05:34:05.317 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:34:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a74066f30 msgr2=0x7f9a740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a74066f30 0x7f9a740693e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9a9006d950 tx=0x7f9a80007f30 comp rx=0 tx=0).stop 2023-10-06T05:34:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 msgr2=0x7f9a9016df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 0x7f9a9016df50 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0093f0 tx=0x7f9a8c017000 comp rx=0 tx=0).stop 2023-10-06T05:34:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 shutdown_connections 2023-10-06T05:34:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9a74066f30 0x7f9a740693e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a900c5d40 0x7f9a900c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a900cc750 0x7f9a9016df50 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 --2- 172.21.15.33:0/1062659110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9a900ccfc0 0x7f9a90169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:05.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.317+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 >> 172.21.15.33:0/1062659110 conn(0x7f9a9006cff0 msgr2=0x7f9a9014efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:05.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.318+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 shutdown_connections 2023-10-06T05:34:05.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:05.318+0000 7f9a9659d700 1 -- 172.21.15.33:0/1062659110 wait complete. 2023-10-06T05:34:05.337 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:34:05.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:34:05.338 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:34:05.338 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:34:05.338 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:34:05.338 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:34:05.339 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:34:05.339 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:34:05.339 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:34:05.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:05.340 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:34:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41112sec preferred_lft 41112sec 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:34:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:34:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:05.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:05.381 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:34:05.477 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:34:05.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:05.481 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:34:05.519 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:34:05.519 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:34:05.519 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:34:05.519 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:34:05.519 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:34:05.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:05.520 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:34:05.520 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:34:05.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:34:05.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:34:05.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:34:05.521 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:34:05.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:05.719 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:34:05.756 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:34:05.756 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:34:05.756 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:34:05.757 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:34:05.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:05.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:34:05.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:34:05.860 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:34:05.896 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:34:05.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:34:05.896 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:34:05.972 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:34:05.972 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:34:05.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:34:06.105 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:34:06.105 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:34:06.105 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:34:06.121 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:34:06.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:34:06.122 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:34:06.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:34:06.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:34:06.146 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:34:06.147 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:34:06.148 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:34:06.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:06.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:34:06.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:34:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.400+0000 7ff45759e700 1 Processor -- start 2023-10-06T05:34:06.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.400+0000 7ff45759e700 1 -- start start 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff458155140 0x7ff458158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4580adf50 0x7ff4580c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4580c8e40 con 0x7ff4580c9810 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4580ae3f0 con 0x7ff458155140 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4580ae6c0 con 0x7ff4580adf50 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff458155140 0x7ff458158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff458155140 0x7ff458158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54476/0 (socket says 172.21.15.33:54476) 2023-10-06T05:34:06.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 -- 172.21.15.33:0/3870556838 learned_addr learned my addr 172.21.15.33:0/3870556838 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff456d9d700 1 --1- 172.21.15.33:0/3870556838 >> v1:172.21.15.33:6789/0 conn(0x7ff4580c9810 0x7ff4580699c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59672/0 (socket says 172.21.15.33:59672) 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff455d9b700 1 --2- 172.21.15.33:0/3870556838 >> v2:172.21.15.167:3300/0 conn(0x7ff4580adf50 0x7ff4580c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 -- 172.21.15.33:0/3870556838 >> v2:172.21.15.167:3300/0 conn(0x7ff4580adf50 msgr2=0x7ff4580c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 --2- 172.21.15.33:0/3870556838 >> v2:172.21.15.167:3300/0 conn(0x7ff4580adf50 0x7ff4580c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 -- 172.21.15.33:0/3870556838 >> v1:172.21.15.33:6789/0 conn(0x7ff4580c9810 legacy=0x7ff4580699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45659c700 1 -- 172.21.15.33:0/3870556838 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580ae9a0 con 0x7ff458155140 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.402+0000 7ff45559a700 1 -- 172.21.15.33:0/3870556838 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117648869 0 0) 0x7ff4580c8e40 con 0x7ff4580c9810 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.403+0000 7ff45659c700 1 --2- 172.21.15.33:0/3870556838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff458158700 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7ff4480099b0 tx=0x7ff448009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4273fb948f450130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:06.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.403+0000 7ff45559a700 1 -- 172.21.15.33:0/3870556838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800b820 con 0x7ff458155140 2023-10-06T05:34:06.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.403+0000 7ff45559a700 1 -- 172.21.15.33:0/3870556838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44800b980 con 0x7ff458155140 2023-10-06T05:34:06.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.403+0000 7ff45559a700 1 -- 172.21.15.33:0/3870556838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800bd20 con 0x7ff458155140 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.403+0000 7ff45759e700 1 -- 172.21.15.33:0/3870556838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 msgr2=0x7ff458158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.403+0000 7ff45759e700 1 --2- 172.21.15.33:0/3870556838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff458158700 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7ff4480099b0 tx=0x7ff448009250 comp rx=0 tx=0).stop 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.404+0000 7ff45759e700 1 -- 172.21.15.33:0/3870556838 shutdown_connections 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.404+0000 7ff45759e700 1 --2- 172.21.15.33:0/3870556838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff458158700 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.404+0000 7ff45759e700 1 --2- 172.21.15.33:0/3870556838 >> v2:172.21.15.167:3300/0 conn(0x7ff4580adf50 0x7ff4580c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.404+0000 7ff45759e700 1 -- 172.21.15.33:0/3870556838 >> 172.21.15.33:0/3870556838 conn(0x7ff45806d030 msgr2=0x7ff45806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.404+0000 7ff45759e700 1 -- 172.21.15.33:0/3870556838 shutdown_connections 2023-10-06T05:34:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.404+0000 7ff45759e700 1 -- 172.21.15.33:0/3870556838 wait complete. 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.405+0000 7ff45759e700 1 Processor -- start 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.405+0000 7ff45759e700 1 -- start start 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff45759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4580adf50 0x7ff45814e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff45759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4580c9810 0x7ff45813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff45759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff4580af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff45759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45814e990 con 0x7ff458155140 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff45759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45814d9b0 con 0x7ff4580c9810 2023-10-06T05:34:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff45759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff458151600 con 0x7ff4580adf50 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff456d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff4580af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.406+0000 7ff456d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff4580af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54490/0 (socket says 172.21.15.33:54490) 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 -- 172.21.15.33:0/3443936758 learned_addr learned my addr 172.21.15.33:0/3443936758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff455d9b700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4580c9810 0x7ff45813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff45659c700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4580adf50 0x7ff45814e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 -- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4580adf50 msgr2=0x7ff45814e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4580adf50 0x7ff45814e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 -- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4580c9810 msgr2=0x7ff45813ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4580c9810 0x7ff45813ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 -- 172.21.15.33:0/3443936758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480094f0 con 0x7ff458155140 2023-10-06T05:34:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff456d9d700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff4580af180 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7ff45000cd00 tx=0x7ff450001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450005df0 con 0x7ff458155140 2023-10-06T05:34:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45000a960 con 0x7ff458155140 2023-10-06T05:34:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000ad20 con 0x7ff458155140 2023-10-06T05:34:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.407+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4581518e0 con 0x7ff458155140 2023-10-06T05:34:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.408+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45814df30 con 0x7ff458155140 2023-10-06T05:34:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.409+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45814e150 con 0x7ff458155140 2023-10-06T05:34:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.410+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff450007300 con 0x7ff458155140 2023-10-06T05:34:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.410+0000 7ff4467fc700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff438066f30 0x7ff4380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.410+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff450084840 con 0x7ff458155140 2023-10-06T05:34:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.411+0000 7ff45659c700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff438066f30 0x7ff4380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.412+0000 7ff45659c700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff438066f30 0x7ff4380693e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff4480099b0 tx=0x7ff448017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.415+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4500519d0 con 0x7ff458155140 2023-10-06T05:34:06.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.663+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff45804c380 con 0x7ff458155140 2023-10-06T05:34:06.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.663+0000 7ff4467fc700 1 -- 172.21.15.33:0/3443936758 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v203) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff450054ff0 con 0x7ff458155140 2023-10-06T05:34:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:34:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.665+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff438066f30 msgr2=0x7ff4380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.665+0000 7ff45759e700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff438066f30 0x7ff4380693e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff4480099b0 tx=0x7ff448017000 comp rx=0 tx=0).stop 2023-10-06T05:34:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.665+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 msgr2=0x7ff4580af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.665+0000 7ff45759e700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff4580af180 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7ff45000cd00 tx=0x7ff450001ac0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 shutdown_connections 2023-10-06T05:34:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff438066f30 0x7ff4380693e0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff458155140 0x7ff4580af180 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4580adf50 0x7ff45814e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 --2- 172.21.15.33:0/3443936758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4580c9810 0x7ff45813ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 >> 172.21.15.33:0/3443936758 conn(0x7ff45806d030 msgr2=0x7ff4580cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 shutdown_connections 2023-10-06T05:34:06.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:06.666+0000 7ff45759e700 1 -- 172.21.15.33:0/3443936758 wait complete. 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:34:06.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:34:06.686 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:34:06.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:06.686 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:34:06.729 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:34:06.729 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:34:06.730 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41114sec preferred_lft 41114sec 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:34:06.731 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:34:06.732 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.733 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:34:06.733 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:34:06.733 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:34:06.733 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:34:06.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:06.733 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:34:06.832 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:34:06.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:06.836 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:34:06.876 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:34:06.876 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:34:06.876 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:34:06.876 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:34:06.876 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:34:06.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:06.877 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:34:06.877 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:34:06.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:34:06.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:34:06.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:34:06.878 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:34:07.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:07.074 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:34:07.111 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:34:07.111 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:34:07.111 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:34:07.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:07.111 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:34:07.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:34:07.212 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:34:07.248 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:34:07.249 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:34:07.249 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:34:07.323 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:34:07.323 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:34:07.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:34:07.454 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:34:07.454 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:34:07.454 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:34:07.470 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:34:07.470 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:34:07.470 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:34:07.470 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:34:07.470 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:34:07.491 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:34:07.496 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:34:07.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:07.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:34:07.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:34:07.771 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:34:07.772 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:34:07.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:34:07.977 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:34:07.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:34:07.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:07.978 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f0) 2023-10-06T05:34:08.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.020 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f1) 2023-10-06T05:34:08.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.100 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f2) 2023-10-06T05:34:08.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.182 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f3) 2023-10-06T05:34:08.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.264 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f4) 2023-10-06T05:34:08.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.346 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f5) 2023-10-06T05:34:08.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.428 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f6) 2023-10-06T05:34:08.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.510 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f7) 2023-10-06T05:34:08.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.592 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f8) 2023-10-06T05:34:08.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.675 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f9) 2023-10-06T05:34:08.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.756 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f10) 2023-10-06T05:34:08.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.838 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f11) 2023-10-06T05:34:08.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:08.921 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f12) 2023-10-06T05:34:09.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.005 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f13) 2023-10-06T05:34:09.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.086 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f14) 2023-10-06T05:34:09.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.167 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f15) 2023-10-06T05:34:09.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.249 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f16) 2023-10-06T05:34:09.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.332 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f17) 2023-10-06T05:34:09.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.413 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f18) 2023-10-06T05:34:09.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.496 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f19) 2023-10-06T05:34:09.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.577 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f20) 2023-10-06T05:34:09.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.660 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f21) 2023-10-06T05:34:09.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.741 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f22) 2023-10-06T05:34:09.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.823 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f23) 2023-10-06T05:34:09.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.906 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f24) 2023-10-06T05:34:09.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:09.991 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f25) 2023-10-06T05:34:10.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.074 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f26) 2023-10-06T05:34:10.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.156 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f27) 2023-10-06T05:34:10.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.238 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f28) 2023-10-06T05:34:10.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.320 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f29) 2023-10-06T05:34:10.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.403 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f30) 2023-10-06T05:34:10.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.484 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f31) 2023-10-06T05:34:10.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.567 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f32) 2023-10-06T05:34:10.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.650 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f33) 2023-10-06T05:34:10.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.733 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f34) 2023-10-06T05:34:10.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.814 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f35) 2023-10-06T05:34:10.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.896 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f36) 2023-10-06T05:34:10.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:10.982 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f37) 2023-10-06T05:34:11.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.063 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f38) 2023-10-06T05:34:11.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.145 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f39) 2023-10-06T05:34:11.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.227 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f40) 2023-10-06T05:34:11.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.310 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f41) 2023-10-06T05:34:11.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.392 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f42) 2023-10-06T05:34:11.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.474 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f43) 2023-10-06T05:34:11.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.556 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f44) 2023-10-06T05:34:11.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.638 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f45) 2023-10-06T05:34:11.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.720 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f46) 2023-10-06T05:34:11.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.803 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f47) 2023-10-06T05:34:11.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.884 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f48) 2023-10-06T05:34:11.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:11.966 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f49) 2023-10-06T05:34:12.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.050 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f50) 2023-10-06T05:34:12.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.132 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f51) 2023-10-06T05:34:12.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.213 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f52) 2023-10-06T05:34:12.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.297 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f53) 2023-10-06T05:34:12.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.379 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f54) 2023-10-06T05:34:12.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.460 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f55) 2023-10-06T05:34:12.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.540 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f56) 2023-10-06T05:34:12.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.622 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f57) 2023-10-06T05:34:12.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.703 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f58) 2023-10-06T05:34:12.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.785 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f59) 2023-10-06T05:34:12.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.869 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f60) 2023-10-06T05:34:12.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:12.951 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f61) 2023-10-06T05:34:13.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.034 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f62) 2023-10-06T05:34:13.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.115 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f63) 2023-10-06T05:34:13.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.198 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f64) 2023-10-06T05:34:13.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.279 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f65) 2023-10-06T05:34:13.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.361 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f66) 2023-10-06T05:34:13.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.443 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f67) 2023-10-06T05:34:13.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.525 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f68) 2023-10-06T05:34:13.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.607 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f69) 2023-10-06T05:34:13.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.690 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f70) 2023-10-06T05:34:13.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.772 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f71) 2023-10-06T05:34:13.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.855 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f72) 2023-10-06T05:34:13.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:13.937 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f73) 2023-10-06T05:34:14.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.020 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f74) 2023-10-06T05:34:14.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.103 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f75) 2023-10-06T05:34:14.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.186 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f76) 2023-10-06T05:34:14.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.269 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f77) 2023-10-06T05:34:14.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.352 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f78) 2023-10-06T05:34:14.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.434 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f79) 2023-10-06T05:34:14.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.517 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f80) 2023-10-06T05:34:14.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.599 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f81) 2023-10-06T05:34:14.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.681 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f82) 2023-10-06T05:34:14.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.763 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f83) 2023-10-06T05:34:14.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.845 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f84) 2023-10-06T05:34:14.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:14.927 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f85) 2023-10-06T05:34:15.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.010 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f86) 2023-10-06T05:34:15.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.092 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f87) 2023-10-06T05:34:15.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.174 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f88) 2023-10-06T05:34:15.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.256 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f89) 2023-10-06T05:34:15.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.338 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f90) 2023-10-06T05:34:15.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.419 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f91) 2023-10-06T05:34:15.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.501 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f92) 2023-10-06T05:34:15.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.586 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f93) 2023-10-06T05:34:15.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.668 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f94) 2023-10-06T05:34:15.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.750 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f95) 2023-10-06T05:34:15.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.832 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f96) 2023-10-06T05:34:15.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.914 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f97) 2023-10-06T05:34:15.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:15.997 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f98) 2023-10-06T05:34:16.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.079 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f99) 2023-10-06T05:34:16.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.161 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f100) 2023-10-06T05:34:16.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.245 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f101) 2023-10-06T05:34:16.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.326 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f102) 2023-10-06T05:34:16.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.408 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f103) 2023-10-06T05:34:16.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.490 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f104) 2023-10-06T05:34:16.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.573 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f105) 2023-10-06T05:34:16.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.670 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f106) 2023-10-06T05:34:16.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.735 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f107) 2023-10-06T05:34:16.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.817 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f108) 2023-10-06T05:34:16.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.899 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f109) 2023-10-06T05:34:16.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:16.982 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f110) 2023-10-06T05:34:17.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.064 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f111) 2023-10-06T05:34:17.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.146 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f112) 2023-10-06T05:34:17.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.229 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f113) 2023-10-06T05:34:17.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.310 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f114) 2023-10-06T05:34:17.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.392 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f115) 2023-10-06T05:34:17.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.475 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f116) 2023-10-06T05:34:17.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.554 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f117) 2023-10-06T05:34:17.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.636 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f118) 2023-10-06T05:34:17.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.718 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f119) 2023-10-06T05:34:17.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.800 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f120) 2023-10-06T05:34:17.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.883 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f121) 2023-10-06T05:34:17.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:17.966 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f122) 2023-10-06T05:34:18.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.049 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f123) 2023-10-06T05:34:18.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.131 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f124) 2023-10-06T05:34:18.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.214 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f125) 2023-10-06T05:34:18.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.295 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f126) 2023-10-06T05:34:18.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.377 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f127) 2023-10-06T05:34:18.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.460 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f128) 2023-10-06T05:34:18.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.542 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f129) 2023-10-06T05:34:18.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.625 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f130) 2023-10-06T05:34:18.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.706 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f131) 2023-10-06T05:34:18.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.788 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f132) 2023-10-06T05:34:18.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.871 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f133) 2023-10-06T05:34:18.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:18.954 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f134) 2023-10-06T05:34:19.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.037 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f135) 2023-10-06T05:34:19.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.118 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f136) 2023-10-06T05:34:19.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.201 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f137) 2023-10-06T05:34:19.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.283 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f138) 2023-10-06T05:34:19.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.365 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f139) 2023-10-06T05:34:19.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.447 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f140) 2023-10-06T05:34:19.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.530 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f141) 2023-10-06T05:34:19.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.613 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f142) 2023-10-06T05:34:19.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.696 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f143) 2023-10-06T05:34:19.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.779 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f144) 2023-10-06T05:34:19.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.861 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f145) 2023-10-06T05:34:19.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:19.944 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f146) 2023-10-06T05:34:20.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.026 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f147) 2023-10-06T05:34:20.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.108 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f148) 2023-10-06T05:34:20.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.191 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f149) 2023-10-06T05:34:20.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.273 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f150) 2023-10-06T05:34:20.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.356 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f151) 2023-10-06T05:34:20.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.438 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f152) 2023-10-06T05:34:20.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.520 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f153) 2023-10-06T05:34:20.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.603 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f154) 2023-10-06T05:34:20.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.686 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f155) 2023-10-06T05:34:20.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.769 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f156) 2023-10-06T05:34:20.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.852 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f157) 2023-10-06T05:34:20.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:20.934 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f158) 2023-10-06T05:34:21.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.017 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f159) 2023-10-06T05:34:21.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.099 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f160) 2023-10-06T05:34:21.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.181 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f161) 2023-10-06T05:34:21.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.263 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f162) 2023-10-06T05:34:21.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.346 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f163) 2023-10-06T05:34:21.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.428 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f164) 2023-10-06T05:34:21.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.511 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f165) 2023-10-06T05:34:21.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.592 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f166) 2023-10-06T05:34:21.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:34:21.639 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:34:21.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.674 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f167) 2023-10-06T05:34:21.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.755 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f168) 2023-10-06T05:34:21.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.838 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f169) 2023-10-06T05:34:21.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:21.921 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f170) 2023-10-06T05:34:22.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.003 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f171) 2023-10-06T05:34:22.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.085 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f172) 2023-10-06T05:34:22.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.168 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f173) 2023-10-06T05:34:22.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.253 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f174) 2023-10-06T05:34:22.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.334 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f175) 2023-10-06T05:34:22.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.416 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f176) 2023-10-06T05:34:22.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.498 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f177) 2023-10-06T05:34:22.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.578 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f178) 2023-10-06T05:34:22.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.660 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f179) 2023-10-06T05:34:22.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.743 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f180) 2023-10-06T05:34:22.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.826 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f181) 2023-10-06T05:34:22.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.908 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f182) 2023-10-06T05:34:22.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:22.992 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f183) 2023-10-06T05:34:23.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.074 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f184) 2023-10-06T05:34:23.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.157 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f185) 2023-10-06T05:34:23.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.240 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f186) 2023-10-06T05:34:23.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.322 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f187) 2023-10-06T05:34:23.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.405 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f188) 2023-10-06T05:34:23.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.488 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f189) 2023-10-06T05:34:23.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.571 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f190) 2023-10-06T05:34:23.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.654 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f191) 2023-10-06T05:34:23.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.738 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f192) 2023-10-06T05:34:23.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.820 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f193) 2023-10-06T05:34:23.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.902 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f194) 2023-10-06T05:34:23.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:23.985 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f195) 2023-10-06T05:34:24.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.066 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f196) 2023-10-06T05:34:24.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.149 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f197) 2023-10-06T05:34:24.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.231 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f198) 2023-10-06T05:34:24.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.314 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f199) 2023-10-06T05:34:24.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.395 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f200) 2023-10-06T05:34:24.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.477 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f201) 2023-10-06T05:34:24.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.559 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f202) 2023-10-06T05:34:24.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.643 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f203) 2023-10-06T05:34:24.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.724 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f204) 2023-10-06T05:34:24.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.806 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f205) 2023-10-06T05:34:24.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.888 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f206) 2023-10-06T05:34:24.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:24.970 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f207) 2023-10-06T05:34:25.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.053 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f208) 2023-10-06T05:34:25.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.134 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f209) 2023-10-06T05:34:25.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.216 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f210) 2023-10-06T05:34:25.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.299 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f211) 2023-10-06T05:34:25.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.381 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f212) 2023-10-06T05:34:25.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.463 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f213) 2023-10-06T05:34:25.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.545 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f214) 2023-10-06T05:34:25.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.627 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f215) 2023-10-06T05:34:25.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.709 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f216) 2023-10-06T05:34:25.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.792 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f217) 2023-10-06T05:34:25.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.875 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f218) 2023-10-06T05:34:25.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:25.957 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f219) 2023-10-06T05:34:26.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.040 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f220) 2023-10-06T05:34:26.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.121 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f221) 2023-10-06T05:34:26.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.203 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f222) 2023-10-06T05:34:26.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.286 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f223) 2023-10-06T05:34:26.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.368 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f224) 2023-10-06T05:34:26.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.451 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f225) 2023-10-06T05:34:26.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.533 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f226) 2023-10-06T05:34:26.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.615 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f227) 2023-10-06T05:34:26.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.698 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f228) 2023-10-06T05:34:26.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.779 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f229) 2023-10-06T05:34:26.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.861 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f230) 2023-10-06T05:34:26.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:26.943 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f231) 2023-10-06T05:34:27.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.025 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f232) 2023-10-06T05:34:27.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.107 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f233) 2023-10-06T05:34:27.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.189 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f234) 2023-10-06T05:34:27.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.273 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f235) 2023-10-06T05:34:27.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.355 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f236) 2023-10-06T05:34:27.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.438 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f237) 2023-10-06T05:34:27.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.518 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f238) 2023-10-06T05:34:27.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.599 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f239) 2023-10-06T05:34:27.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.682 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f240) 2023-10-06T05:34:27.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.765 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f241) 2023-10-06T05:34:27.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.848 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f242) 2023-10-06T05:34:27.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:27.930 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f243) 2023-10-06T05:34:28.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.013 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f244) 2023-10-06T05:34:28.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.096 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f245) 2023-10-06T05:34:28.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.179 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f246) 2023-10-06T05:34:28.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.262 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f247) 2023-10-06T05:34:28.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.345 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f248) 2023-10-06T05:34:28.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.428 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f249) 2023-10-06T05:34:28.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.510 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f250) 2023-10-06T05:34:28.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.593 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f251) 2023-10-06T05:34:28.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.676 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f252) 2023-10-06T05:34:28.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.757 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f253) 2023-10-06T05:34:28.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.840 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f254) 2023-10-06T05:34:28.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:28.923 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f255) 2023-10-06T05:34:29.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.005 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f256) 2023-10-06T05:34:29.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.247 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f257) 2023-10-06T05:34:29.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.290 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f258) 2023-10-06T05:34:29.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.372 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f259) 2023-10-06T05:34:29.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.454 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f260) 2023-10-06T05:34:29.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.536 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f261) 2023-10-06T05:34:29.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.619 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f262) 2023-10-06T05:34:29.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.701 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f263) 2023-10-06T05:34:29.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.782 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f264) 2023-10-06T05:34:29.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.865 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f265) 2023-10-06T05:34:29.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:29.948 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f266) 2023-10-06T05:34:30.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.032 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f267) 2023-10-06T05:34:30.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.113 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f268) 2023-10-06T05:34:30.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.196 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f269) 2023-10-06T05:34:30.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.279 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f270) 2023-10-06T05:34:30.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.361 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f271) 2023-10-06T05:34:30.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.443 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f272) 2023-10-06T05:34:30.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.526 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f273) 2023-10-06T05:34:30.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.607 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f274) 2023-10-06T05:34:30.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.690 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f275) 2023-10-06T05:34:30.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.773 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f276) 2023-10-06T05:34:30.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.854 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f277) 2023-10-06T05:34:30.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:30.937 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f278) 2023-10-06T05:34:31.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.020 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f279) 2023-10-06T05:34:31.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.103 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f280) 2023-10-06T05:34:31.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.185 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f281) 2023-10-06T05:34:31.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.268 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f282) 2023-10-06T05:34:31.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.350 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f283) 2023-10-06T05:34:31.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.432 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f284) 2023-10-06T05:34:31.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.514 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f285) 2023-10-06T05:34:31.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.597 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f286) 2023-10-06T05:34:31.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.679 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f287) 2023-10-06T05:34:31.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.760 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f288) 2023-10-06T05:34:31.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.842 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f289) 2023-10-06T05:34:31.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:31.925 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f290) 2023-10-06T05:34:32.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.007 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f291) 2023-10-06T05:34:32.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.090 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f292) 2023-10-06T05:34:32.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.172 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f293) 2023-10-06T05:34:32.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.255 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f294) 2023-10-06T05:34:32.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f295) 2023-10-06T05:34:32.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.418 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f296) 2023-10-06T05:34:32.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.500 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f297) 2023-10-06T05:34:32.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.580 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f298) 2023-10-06T05:34:32.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.663 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f299) 2023-10-06T05:34:32.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.746 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f300) 2023-10-06T05:34:32.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.828 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f301) 2023-10-06T05:34:32.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.911 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f302) 2023-10-06T05:34:32.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:32.994 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f303) 2023-10-06T05:34:33.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.076 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f304) 2023-10-06T05:34:33.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.158 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f305) 2023-10-06T05:34:33.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.241 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f306) 2023-10-06T05:34:33.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.325 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f307) 2023-10-06T05:34:33.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.406 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f308) 2023-10-06T05:34:33.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.489 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f309) 2023-10-06T05:34:33.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.572 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f310) 2023-10-06T05:34:33.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.655 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f311) 2023-10-06T05:34:33.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.736 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f312) 2023-10-06T05:34:33.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.819 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f313) 2023-10-06T05:34:33.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.908 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f314) 2023-10-06T05:34:33.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:33.984 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f315) 2023-10-06T05:34:34.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.067 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f316) 2023-10-06T05:34:34.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.150 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f317) 2023-10-06T05:34:34.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.233 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f318) 2023-10-06T05:34:34.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.314 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f319) 2023-10-06T05:34:34.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.396 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f320) 2023-10-06T05:34:34.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.479 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f321) 2023-10-06T05:34:34.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.561 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f322) 2023-10-06T05:34:34.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.644 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f323) 2023-10-06T05:34:34.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.727 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f324) 2023-10-06T05:34:34.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.810 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f325) 2023-10-06T05:34:34.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.892 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f326) 2023-10-06T05:34:34.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:34.975 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f327) 2023-10-06T05:34:35.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.058 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f328) 2023-10-06T05:34:35.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.140 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f329) 2023-10-06T05:34:35.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.223 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f330) 2023-10-06T05:34:35.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.306 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f331) 2023-10-06T05:34:35.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.389 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f332) 2023-10-06T05:34:35.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.475 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f333) 2023-10-06T05:34:35.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.556 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f334) 2023-10-06T05:34:35.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.639 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f335) 2023-10-06T05:34:35.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.720 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f336) 2023-10-06T05:34:35.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.803 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f337) 2023-10-06T05:34:35.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.886 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f338) 2023-10-06T05:34:35.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:35.968 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f339) 2023-10-06T05:34:36.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.051 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f340) 2023-10-06T05:34:36.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.137 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f341) 2023-10-06T05:34:36.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.216 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f342) 2023-10-06T05:34:36.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.299 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f343) 2023-10-06T05:34:36.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.382 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f344) 2023-10-06T05:34:36.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.463 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f345) 2023-10-06T05:34:36.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.546 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f346) 2023-10-06T05:34:36.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.629 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f347) 2023-10-06T05:34:36.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.712 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f348) 2023-10-06T05:34:36.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.794 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f349) 2023-10-06T05:34:36.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.876 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f350) 2023-10-06T05:34:36.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:36.959 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f351) 2023-10-06T05:34:37.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.043 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f352) 2023-10-06T05:34:37.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.125 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f353) 2023-10-06T05:34:37.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.208 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f354) 2023-10-06T05:34:37.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.290 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f355) 2023-10-06T05:34:37.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.373 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f356) 2023-10-06T05:34:37.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.454 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f357) 2023-10-06T05:34:37.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.535 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f358) 2023-10-06T05:34:37.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.618 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f359) 2023-10-06T05:34:37.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.700 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f360) 2023-10-06T05:34:37.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.782 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f361) 2023-10-06T05:34:37.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.866 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f362) 2023-10-06T05:34:37.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:37.949 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f363) 2023-10-06T05:34:38.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.032 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f364) 2023-10-06T05:34:38.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.115 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f365) 2023-10-06T05:34:38.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.198 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f366) 2023-10-06T05:34:38.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.281 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f367) 2023-10-06T05:34:38.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.364 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f368) 2023-10-06T05:34:38.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.446 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f369) 2023-10-06T05:34:38.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.529 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f370) 2023-10-06T05:34:38.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.611 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f371) 2023-10-06T05:34:38.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.693 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f372) 2023-10-06T05:34:38.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.776 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f373) 2023-10-06T05:34:38.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.857 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f374) 2023-10-06T05:34:38.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:38.939 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f375) 2023-10-06T05:34:39.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.023 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f376) 2023-10-06T05:34:39.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.105 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f377) 2023-10-06T05:34:39.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.187 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f378) 2023-10-06T05:34:39.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.269 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f379) 2023-10-06T05:34:39.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.352 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f380) 2023-10-06T05:34:39.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.434 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f381) 2023-10-06T05:34:39.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.515 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f382) 2023-10-06T05:34:39.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.597 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f383) 2023-10-06T05:34:39.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.679 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f384) 2023-10-06T05:34:39.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.761 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f385) 2023-10-06T05:34:39.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.844 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f386) 2023-10-06T05:34:39.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:39.927 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f387) 2023-10-06T05:34:40.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.010 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f388) 2023-10-06T05:34:40.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.092 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f389) 2023-10-06T05:34:40.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.174 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f390) 2023-10-06T05:34:40.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.257 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f391) 2023-10-06T05:34:40.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.339 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f392) 2023-10-06T05:34:40.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.424 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f393) 2023-10-06T05:34:40.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.505 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f394) 2023-10-06T05:34:40.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.587 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f395) 2023-10-06T05:34:40.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.670 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f396) 2023-10-06T05:34:40.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.753 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f397) 2023-10-06T05:34:40.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.835 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f398) 2023-10-06T05:34:40.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.917 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f399) 2023-10-06T05:34:40.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:40.999 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f400) 2023-10-06T05:34:41.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.082 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f401) 2023-10-06T05:34:41.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.166 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f402) 2023-10-06T05:34:41.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.248 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f403) 2023-10-06T05:34:41.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.330 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f404) 2023-10-06T05:34:41.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.413 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f405) 2023-10-06T05:34:41.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.495 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f406) 2023-10-06T05:34:41.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.578 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f407) 2023-10-06T05:34:41.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.660 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f408) 2023-10-06T05:34:41.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.742 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f409) 2023-10-06T05:34:41.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.825 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f410) 2023-10-06T05:34:41.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.908 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f411) 2023-10-06T05:34:41.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:41.990 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f412) 2023-10-06T05:34:42.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.072 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f413) 2023-10-06T05:34:42.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.154 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f414) 2023-10-06T05:34:42.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.236 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f415) 2023-10-06T05:34:42.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.318 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f416) 2023-10-06T05:34:42.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.400 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f417) 2023-10-06T05:34:42.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.483 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f418) 2023-10-06T05:34:42.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.562 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f419) 2023-10-06T05:34:42.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.645 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f420) 2023-10-06T05:34:42.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.727 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f421) 2023-10-06T05:34:42.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.810 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f422) 2023-10-06T05:34:42.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.891 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f423) 2023-10-06T05:34:42.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:42.974 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f424) 2023-10-06T05:34:43.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.057 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f425) 2023-10-06T05:34:43.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.138 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f426) 2023-10-06T05:34:43.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.220 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f427) 2023-10-06T05:34:43.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.303 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f428) 2023-10-06T05:34:43.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.384 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f429) 2023-10-06T05:34:43.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.467 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f430) 2023-10-06T05:34:43.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.549 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f431) 2023-10-06T05:34:43.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.632 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f432) 2023-10-06T05:34:43.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.715 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f433) 2023-10-06T05:34:43.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.797 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f434) 2023-10-06T05:34:43.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.880 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f435) 2023-10-06T05:34:43.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:43.963 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f436) 2023-10-06T05:34:44.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.047 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f437) 2023-10-06T05:34:44.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.130 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f438) 2023-10-06T05:34:44.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.211 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f439) 2023-10-06T05:34:44.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.293 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f440) 2023-10-06T05:34:44.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.375 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f441) 2023-10-06T05:34:44.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.457 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f442) 2023-10-06T05:34:44.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.540 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f443) 2023-10-06T05:34:44.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.621 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f444) 2023-10-06T05:34:44.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.703 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f445) 2023-10-06T05:34:44.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.785 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f446) 2023-10-06T05:34:44.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.867 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f447) 2023-10-06T05:34:44.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:44.949 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f448) 2023-10-06T05:34:45.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.032 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f449) 2023-10-06T05:34:45.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.115 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f450) 2023-10-06T05:34:45.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.196 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f451) 2023-10-06T05:34:45.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.279 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f452) 2023-10-06T05:34:45.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.362 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f453) 2023-10-06T05:34:45.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.444 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f454) 2023-10-06T05:34:45.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.525 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f455) 2023-10-06T05:34:45.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.607 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f456) 2023-10-06T05:34:45.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.689 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f457) 2023-10-06T05:34:45.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.771 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f458) 2023-10-06T05:34:45.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.854 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f459) 2023-10-06T05:34:45.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:45.935 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f460) 2023-10-06T05:34:46.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.035 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f461) 2023-10-06T05:34:46.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.076 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f462) 2023-10-06T05:34:46.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.158 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f463) 2023-10-06T05:34:46.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.241 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f464) 2023-10-06T05:34:46.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.323 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f465) 2023-10-06T05:34:46.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.405 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f466) 2023-10-06T05:34:46.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.488 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f467) 2023-10-06T05:34:46.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.572 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f468) 2023-10-06T05:34:46.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.655 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f469) 2023-10-06T05:34:46.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.738 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f470) 2023-10-06T05:34:46.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.820 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f471) 2023-10-06T05:34:46.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.903 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f472) 2023-10-06T05:34:46.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:46.985 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f473) 2023-10-06T05:34:47.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.069 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f474) 2023-10-06T05:34:47.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.151 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f475) 2023-10-06T05:34:47.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.232 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f476) 2023-10-06T05:34:47.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.315 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f477) 2023-10-06T05:34:47.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.399 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f478) 2023-10-06T05:34:47.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.482 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f479) 2023-10-06T05:34:47.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.563 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f480) 2023-10-06T05:34:47.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.646 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f481) 2023-10-06T05:34:47.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.728 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f482) 2023-10-06T05:34:47.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.811 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f483) 2023-10-06T05:34:47.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.894 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f484) 2023-10-06T05:34:47.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:47.978 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f485) 2023-10-06T05:34:48.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.062 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f486) 2023-10-06T05:34:48.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.145 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f487) 2023-10-06T05:34:48.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.229 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f488) 2023-10-06T05:34:48.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.311 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f489) 2023-10-06T05:34:48.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.394 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f490) 2023-10-06T05:34:48.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.478 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f491) 2023-10-06T05:34:48.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.560 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f492) 2023-10-06T05:34:48.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.643 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f493) 2023-10-06T05:34:48.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.726 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f494) 2023-10-06T05:34:48.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.809 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f495) 2023-10-06T05:34:48.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.892 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f496) 2023-10-06T05:34:48.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:48.975 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f497) 2023-10-06T05:34:49.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:49.059 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f498) 2023-10-06T05:34:49.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:49.142 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch f499) 2023-10-06T05:34:49.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:49.226 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec ls -l) 2023-10-06T05:34:50.739 INFO:teuthology.orchestra.run.smithi167.stdout:total 0 2023-10-06T05:34:50.739 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f0 2023-10-06T05:34:50.739 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f1 2023-10-06T05:34:50.739 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f10 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f100 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f101 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f102 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f103 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f104 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f105 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f106 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f107 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f108 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f109 2023-10-06T05:34:50.740 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f11 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f110 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f111 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f112 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f113 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f114 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f115 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f116 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f117 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f118 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f119 2023-10-06T05:34:50.741 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f12 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f120 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f121 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f122 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f123 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f124 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f125 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f126 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f127 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f128 2023-10-06T05:34:50.742 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f129 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f13 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f130 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f131 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f132 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f133 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f134 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f135 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f136 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f137 2023-10-06T05:34:50.743 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f138 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f139 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f14 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f140 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f141 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f142 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f143 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f144 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f145 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f146 2023-10-06T05:34:50.744 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f147 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f148 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f149 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f15 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f150 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f151 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f152 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f153 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f154 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f155 2023-10-06T05:34:50.745 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f156 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f157 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f158 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f159 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f16 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f160 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f161 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f162 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f163 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f164 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f165 2023-10-06T05:34:50.746 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f166 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f167 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f168 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f169 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f17 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f170 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f171 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f172 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f173 2023-10-06T05:34:50.747 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f174 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f175 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f176 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f177 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f178 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f179 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f18 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f180 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f181 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f182 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f183 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f184 2023-10-06T05:34:50.748 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f185 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f186 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f187 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f188 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f189 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f19 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f190 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f191 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f192 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f193 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f194 2023-10-06T05:34:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f195 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f196 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f197 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f198 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f199 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f2 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f20 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f200 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f201 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f202 2023-10-06T05:34:50.750 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f203 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f204 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f205 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f206 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f207 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f208 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f209 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f21 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f210 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f211 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f212 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f213 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f214 2023-10-06T05:34:50.751 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f215 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f216 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f217 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f218 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f219 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f22 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f220 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f221 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f222 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f223 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f224 2023-10-06T05:34:50.752 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f225 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f226 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f227 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f228 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f229 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f23 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f230 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f231 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f232 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f233 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f234 2023-10-06T05:34:50.753 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f235 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f236 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f237 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f238 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f239 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f24 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f240 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f241 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f242 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f243 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f244 2023-10-06T05:34:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f245 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f246 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f247 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f248 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f249 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f25 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f250 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f251 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f252 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f253 2023-10-06T05:34:50.755 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f254 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f255 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f256 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f257 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f258 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f259 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f26 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f260 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f261 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f262 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f263 2023-10-06T05:34:50.756 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f264 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f265 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f266 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f267 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f268 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f269 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f27 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f270 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f271 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f272 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f273 2023-10-06T05:34:50.757 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f274 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f275 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f276 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f277 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f278 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f279 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f28 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f280 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f281 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f282 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f283 2023-10-06T05:34:50.758 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f284 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f285 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f286 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f287 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f288 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f289 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f29 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f290 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f291 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f292 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f293 2023-10-06T05:34:50.759 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f294 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f295 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f296 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f297 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f298 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f299 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f3 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f30 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f300 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f301 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f302 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f303 2023-10-06T05:34:50.760 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f304 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f305 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f306 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f307 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f308 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f309 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f31 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f310 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f311 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f312 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f313 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f314 2023-10-06T05:34:50.761 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f315 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f316 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f317 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f318 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f319 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f32 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f320 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f321 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f322 2023-10-06T05:34:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f323 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f324 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f325 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f326 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f327 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f328 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f329 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f33 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f330 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f331 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f332 2023-10-06T05:34:50.763 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f333 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f334 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f335 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f336 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f337 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f338 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f339 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f34 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f340 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f341 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f342 2023-10-06T05:34:50.764 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f343 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f344 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f345 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f346 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f347 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f348 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f349 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f35 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f350 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f351 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f352 2023-10-06T05:34:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f353 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f354 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f355 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f356 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f357 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f358 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f359 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f36 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f360 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f361 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f362 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f363 2023-10-06T05:34:50.766 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f364 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f365 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f366 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f367 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f368 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f369 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f37 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f370 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f371 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f372 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f373 2023-10-06T05:34:50.767 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f374 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f375 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f376 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f377 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f378 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f379 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f38 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f380 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f381 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f382 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f383 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f384 2023-10-06T05:34:50.768 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f385 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f386 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f387 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f388 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f389 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f39 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f390 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f391 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f392 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f393 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f394 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f395 2023-10-06T05:34:50.769 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f396 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f397 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f398 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f399 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f4 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f40 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f400 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f401 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f402 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f403 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f404 2023-10-06T05:34:50.770 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f405 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f406 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f407 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f408 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f409 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f41 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f410 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f411 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f412 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f413 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f414 2023-10-06T05:34:50.771 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f415 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f416 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f417 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f418 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f419 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f42 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f420 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f421 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f422 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f423 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f424 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f425 2023-10-06T05:34:50.772 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f426 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f427 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f428 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f429 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f43 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f430 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f431 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f432 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f433 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f434 2023-10-06T05:34:50.773 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f435 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f436 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f437 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f438 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f439 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f44 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f440 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f441 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f442 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f443 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f444 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f445 2023-10-06T05:34:50.774 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f446 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f447 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f448 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f449 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f45 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f450 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f451 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f452 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f453 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f454 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f455 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f456 2023-10-06T05:34:50.775 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f457 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f458 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f459 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f46 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f460 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f461 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f462 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f463 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f464 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f465 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f466 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f467 2023-10-06T05:34:50.776 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f468 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f469 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f47 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f470 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f471 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f472 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f473 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f474 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f475 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f476 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f477 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f478 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f479 2023-10-06T05:34:50.777 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f48 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f480 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f481 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f482 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f483 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f484 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f485 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f486 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f487 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f488 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f489 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f49 2023-10-06T05:34:50.778 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f490 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f491 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f492 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f493 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f494 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f495 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f496 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f497 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f498 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f499 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f5 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f50 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f51 2023-10-06T05:34:50.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f52 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f53 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f54 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f55 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f56 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f57 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f58 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f59 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f6 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f60 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f61 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f62 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f63 2023-10-06T05:34:50.780 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f64 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f65 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f66 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f67 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f68 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f69 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f7 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f70 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f71 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f72 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f73 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f74 2023-10-06T05:34:50.781 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f75 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f76 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f77 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f78 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f79 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f8 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f80 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f81 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f82 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f83 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f84 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f85 2023-10-06T05:34:50.782 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f86 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f87 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f88 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f89 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f9 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f90 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f91 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f92 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f93 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f94 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f95 2023-10-06T05:34:50.783 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f96 2023-10-06T05:34:50.784 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f97 2023-10-06T05:34:50.784 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f98 2023-10-06T05:34:50.784 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 0 Oct 6 05:34 f99 2023-10-06T05:34:50.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:50.785 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:34:50.855 INFO:teuthology.orchestra.run.smithi167.stdout:client10819 2023-10-06T05:34:50.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.040+0000 7fcf6f59d700 1 Processor -- start 2023-10-06T05:34:51.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.040+0000 7fcf6f59d700 1 -- start start 2023-10-06T05:34:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.041+0000 7fcf6f59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcf680ccfc0 0x7fcf680cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.041+0000 7fcf6f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 0x7fcf680b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.041+0000 7fcf6f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf680cbd80 con 0x7fcf680cf3b0 2023-10-06T05:34:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.041+0000 7fcf6f59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680af5f0 con 0x7fcf680ccfc0 2023-10-06T05:34:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.041+0000 7fcf6f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680af8d0 con 0x7fcf680cc750 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6dd9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcf680cf3b0 0x7fcf681594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51922/0 (socket says 172.21.15.33:51922) 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6dd9a700 1 -- 172.21.15.33:0/3328964799 learned_addr learned my addr 172.21.15.33:0/3328964799 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 0x7fcf680b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6e59b700 1 --2- 172.21.15.33:0/3328964799 >> v2:172.21.15.167:3300/0 conn(0x7fcf680ccfc0 0x7fcf680cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803826502 0 0) 0x7fcf680cbd80 con 0x7fcf680cf3b0 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf54003570 con 0x7fcf680cf3b0 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 595394586 0 0) 0x7fcf54003570 con 0x7fcf680cf3b0 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 msgr2=0x7fcf680b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 0x7fcf680b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 >> v2:172.21.15.167:3300/0 conn(0x7fcf680ccfc0 msgr2=0x7fcf680cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 --2- 172.21.15.33:0/3328964799 >> v2:172.21.15.167:3300/0 conn(0x7fcf680ccfc0 0x7fcf680cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680afbb0 con 0x7fcf680cf3b0 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fcf58001470 con 0x7fcf680cf3b0 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.042+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 >> v1:172.21.15.33:6789/0 conn(0x7fcf680cf3b0 legacy=0x7fcf681594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6d599700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54003ae0 0x7fcf54003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6d599700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 0x7fcf54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6d599700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf54004d50 0x7fcf540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680af5f0 con 0x7fcf54004d50 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680af8d0 con 0x7fcf54003ae0 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5400bb60 con 0x7fcf680cc750 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf54004d50 0x7fcf540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6e59b700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54003ae0 0x7fcf54003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6dd9a700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 0x7fcf54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6ed9c700 1 -- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 msgr2=0x7fcf54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680cc750 0x7fcf54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6ed9c700 1 -- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54003ae0 msgr2=0x7fcf54003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54003ae0 0x7fcf54003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.043+0000 7fcf6ed9c700 1 -- 172.21.15.33:0/3328964799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54003570 con 0x7fcf54004d50 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf54004d50 0x7fcf540051c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fcf600094d0 tx=0x7fcf6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c3ed5c77199d4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000ea90 con 0x7fcf54004d50 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6000f040 con 0x7fcf54004d50 2023-10-06T05:34:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6d599700 1 -- 172.21.15.33:0/3328964799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000ee20 con 0x7fcf54004d50 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3328964799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf54004d50 msgr2=0x7fcf540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3328964799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf54004d50 0x7fcf540051c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fcf600094d0 tx=0x7fcf6000b040 comp rx=0 tx=0).stop 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6ed9c700 1 -- 172.21.15.33:0/3328964799 reap_dead start 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3328964799 shutdown_connections 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.044+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3328964799 >> 172.21.15.33:0/3328964799 conn(0x7fcf6806cff0 msgr2=0x7fcf6806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.045+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3328964799 shutdown_connections 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.045+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3328964799 wait complete. 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.046+0000 7fcf6f59d700 1 Processor -- start 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.046+0000 7fcf6f59d700 1 -- start start 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54004d50 0x7fcf681550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 0x7fcf680c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680c9ed0 0x7fcf68169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6816df50 con 0x7fcf680b32a0 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c86a0 con 0x7fcf54004d50 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c9500 con 0x7fcf680c9ed0 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 0x7fcf680c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6dd9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 0x7fcf680c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50468/0 (socket says 172.21.15.33:50468) 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.047+0000 7fcf6dd9a700 1 -- 172.21.15.33:0/3129002130 learned_addr learned my addr 172.21.15.33:0/3129002130 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6e59b700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54004d50 0x7fcf681550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680c9ed0 0x7fcf68169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6dd9a700 1 -- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680c9ed0 msgr2=0x7fcf68169ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6dd9a700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680c9ed0 0x7fcf68169ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6dd9a700 1 -- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54004d50 msgr2=0x7fcf681550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6dd9a700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54004d50 0x7fcf681550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6dd9a700 1 -- 172.21.15.33:0/3129002130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680c88f0 con 0x7fcf680b32a0 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6ed9c700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680c9ed0 0x7fcf68169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6dd9a700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 0x7fcf680c9990 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e7b0 tx=0x7fcf680cfce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58012500 con 0x7fcf680b32a0 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58010040 con 0x7fcf680b32a0 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf680c8bd0 con 0x7fcf680b32a0 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580128d0 con 0x7fcf680b32a0 2023-10-06T05:34:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.048+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf680c9190 con 0x7fcf680b32a0 2023-10-06T05:34:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.049+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6804c380 con 0x7fcf680b32a0 2023-10-06T05:34:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.050+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf58012c40 con 0x7fcf680b32a0 2023-10-06T05:34:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.052+0000 7fcf5e7fc700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.052+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fcf580839f0 con 0x7fcf680b32a0 2023-10-06T05:34:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.052+0000 7fcf6e59b700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.053+0000 7fcf6e59b700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006ed0 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.056+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf58050ae0 con 0x7fcf680b32a0 2023-10-06T05:34:51.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.302+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf680c9640 con 0x7fcf680b32a0 2023-10-06T05:34:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.302+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/3129002130 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+4617 (secure 0 0 0) 0x7fcf58019070 con 0x7fcf680b32a0 2023-10-06T05:34:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:51.305 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":203,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8921,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3798274970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3798274970},{"type":"v1","addr":"172.21.15.33:6834","nonce":3798274970}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":198},{"gid":8924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2969193715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2969193715},{"type":"v1","addr":"172.21.15.33:6835","nonce":2969193715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199},{"gid":10678,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3478690144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3478690144},{"type":"v1","addr":"172.21.15.167:6836","nonce":3478690144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:01.234255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10681":{"gid":10681,"name":"b","rank":0,"incarnation":200,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3376723254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3376723254},{"type":"v1","addr":"172.21.15.167:6837","nonce":3376723254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf4c066f30 msgr2=0x7fcf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006ed0 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 msgr2=0x7fcf680c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 0x7fcf680c9990 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e7b0 tx=0x7fcf680cfce0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 shutdown_connections 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf680c9ed0 0x7fcf68169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf54004d50 0x7fcf681550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 --2- 172.21.15.33:0/3129002130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680b32a0 0x7fcf680c9990 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 >> 172.21.15.33:0/3129002130 conn(0x7fcf6806cff0 msgr2=0x7fcf680c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 shutdown_connections 2023-10-06T05:34:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.305+0000 7fcf6f59d700 1 -- 172.21.15.33:0/3129002130 wait complete. 2023-10-06T05:34:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 203 2023-10-06T05:34:51.325 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T05:34:51.531 INFO:teuthology.orchestra.run.smithi167.stdout:[{"id":10819,"entity":{"name":{"type":"client","num":10819},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1471562529}},"state":"open","num_leases":0,"num_caps":501,"request_load_avg":843,"uptime":44.029211001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":416.1653342171706,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1124.982290277532,"halflife":300},"cap_acquisition":{"value":474.86895021833561,"halflife":30},"last_trim_completed_requests_tid":859,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.10819 v1:192.168.0.1:0/1471562529","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":12018,"entity":{"name":{"type":"client","num":12018},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1011352528}},"state":"open","num_leases":338,"num_caps":403,"request_load_avg":776,"uptime":45.378186822000004,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":96.926072246468081,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1872.6607261056838,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1002,"last_trim_completed_flushes_tid":501,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.12018 v1:192.168.0.1:0/1011352528","completed_requests":[{"tid":1002,"created_ino":"0x100000003e7"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000003e8","length":503}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:34:51.531 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 474.8689502183356 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1471562529, "type": "v1" }, "name": { "num": 10819, "type": "client" } }, "id": 10819, "inst": "client.10819 v1:192.168.0.1:0/1471562529", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 859, "num_caps": 501, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 416.1653342171706 }, "request_load_avg": 843, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1124.982290277532 }, "state": "open", "uptime": 44.029211001 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e7", "tid": 1002 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1011352528, "type": "v1" }, "name": { "num": 12018, "type": "client" } }, "id": 12018, "inst": "client.12018 v1:192.168.0.1:0/1011352528", "last_trim_completed_flushes_tid": 501, "last_trim_completed_requests_tid": 1002, "num_caps": 403, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 338, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 503, "start": "0x100000003e8" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 96.92607224646808 }, "request_load_avg": 776, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1872.6607261056838 }, "state": "open", "uptime": 45.378186822 } ] 2023-10-06T05:34:51.532 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:34:51.532 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:34:51.533 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:34:51.533 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:34:51.533 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:34:51.533 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:34:51.533 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:34:51.534 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:34:51.546 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:34:51.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2023-10-06T05:34:51.547 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:34:51.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2023-10-06T05:34:51.549 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:34:51.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2023-10-06T05:34:51.551 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:34:51.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2023-10-06T05:34:51.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:34:51.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:34:51.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.797+0000 7f934f59e700 1 Processor -- start 2023-10-06T05:34:51.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.797+0000 7f934f59e700 1 -- start start 2023-10-06T05:34:51.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.798+0000 7f934f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9350140d20 0x7f93500c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f935006a0d0 0x7f93500c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935015bcb0 con 0x7f93500c9210 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500c9680 con 0x7f935006a0d0 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93501410f0 con 0x7f9350140d20 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f935006a0d0 0x7f93500c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934e59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9350140d20 0x7f93500c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934e59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9350140d20 0x7f93500c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40324/0 (socket says 172.21.15.33:40324) 2023-10-06T05:34:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.799+0000 7f934e59c700 1 -- 172.21.15.33:0/3510822373 learned_addr learned my addr 172.21.15.33:0/3510822373 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.800+0000 7f934ed9d700 1 -- 172.21.15.33:0/3510822373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9350140d20 msgr2=0x7f93500c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.800+0000 7f934ed9d700 1 --2- 172.21.15.33:0/3510822373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9350140d20 0x7f93500c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.800+0000 7f934ed9d700 1 -- 172.21.15.33:0/3510822373 >> v1:172.21.15.33:6789/0 conn(0x7f93500c9210 legacy=0x7f93500699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.800+0000 7f934ed9d700 1 -- 172.21.15.33:0/3510822373 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935015b0a0 con 0x7f935006a0d0 2023-10-06T05:34:51.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.800+0000 7f934ed9d700 1 --2- 172.21.15.33:0/3510822373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f935006a0d0 0x7f93500c6850 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f9348009e60 tx=0x7f9348009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b765aab809e6315 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.801+0000 7f934d59a700 1 -- 172.21.15.33:0/3510822373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800e070 con 0x7f935006a0d0 2023-10-06T05:34:51.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.801+0000 7f934d59a700 1 -- 172.21.15.33:0/3510822373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800bb20 con 0x7f935006a0d0 2023-10-06T05:34:51.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.801+0000 7f934d59a700 1 -- 172.21.15.33:0/3510822373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348012430 con 0x7f935006a0d0 2023-10-06T05:34:51.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.801+0000 7f934f59e700 1 -- 172.21.15.33:0/3510822373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f935006a0d0 msgr2=0x7f93500c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.801+0000 7f934f59e700 1 --2- 172.21.15.33:0/3510822373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f935006a0d0 0x7f93500c6850 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f9348009e60 tx=0x7f9348009780 comp rx=0 tx=0).stop 2023-10-06T05:34:51.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f7d14bff700 1 Processor -- start 2023-10-06T05:34:51.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f934f59e700 1 -- 172.21.15.33:0/3510822373 shutdown_connections 2023-10-06T05:34:51.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f934f59e700 1 --2- 172.21.15.33:0/3510822373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f935006a0d0 0x7f93500c6850 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f934f59e700 1 --2- 172.21.15.33:0/3510822373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9350140d20 0x7f93500c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f934f59e700 1 -- 172.21.15.33:0/3510822373 >> 172.21.15.33:0/3510822373 conn(0x7f935006c560 msgr2=0x7f935006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:51.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f7d14bff700 1 -- start start 2023-10-06T05:34:51.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f934f59e700 1 -- 172.21.15.33:0/3510822373 shutdown_connections 2023-10-06T05:34:51.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.802+0000 7f934f59e700 1 -- 172.21.15.33:0/3510822373 wait complete. 2023-10-06T05:34:51.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.804+0000 7f7d14bff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d1015d610 0x7f7d100cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.804+0000 7f934f59e700 1 Processor -- start 2023-10-06T05:34:51.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.804+0000 7f934f59e700 1 -- start start 2023-10-06T05:34:51.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.804+0000 7f7d14bff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1015aed0 con 0x7f7d100cb1b0 2023-10-06T05:34:51.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.804+0000 7f7d0dd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7d100cb1b0 0x7f7d10150290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51938/0 (socket says 172.21.15.33:51938) 2023-10-06T05:34:51.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1310089512 learned_addr learned my addr 172.21.15.33:0/1310089512 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:51.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f700e9ff700 1 Processor -- start 2023-10-06T05:34:51.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d100cb680 con 0x7f7d101509a0 2023-10-06T05:34:51.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f7d0ed9d700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7d101509a0 0x7f7d1015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57498/0 (socket says 172.21.15.33:57498) 2023-10-06T05:34:51.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f700e9ff700 1 -- start start 2023-10-06T05:34:51.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f7d0e59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d1015d610 0x7f7d100cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.805+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970367459 0 0) 0x7f7d1015aed0 con 0x7f7d100cb1b0 2023-10-06T05:34:51.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f935006a0d0 0x7f93500c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93500c9210 0x7f93500c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 0x7f9350157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100cb960 con 0x7f7d1015d610 2023-10-06T05:34:51.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9350157a10 con 0x7f9350140d20 2023-10-06T05:34:51.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b1320 con 0x7f93500c9210 2023-10-06T05:34:51.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cfc003570 con 0x7f7d100cb1b0 2023-10-06T05:34:51.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225909622 0 0) 0x7f7d100cb680 con 0x7f7d101509a0 2023-10-06T05:34:51.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1015aed0 con 0x7f7d101509a0 2023-10-06T05:34:51.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b1600 con 0x7f935006a0d0 2023-10-06T05:34:51.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 0x7f9350157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 0x7f9350157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50482/0 (socket says 172.21.15.33:50482) 2023-10-06T05:34:51.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153549493 0 0) 0x7f7cfc003570 con 0x7f7d100cb1b0 2023-10-06T05:34:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d1015d610 msgr2=0x7f7d100cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d1015d610 0x7f7d100cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 >> v1:172.21.15.167:6789/0 conn(0x7f7d101509a0 legacy=0x7f7d1015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100cbc40 con 0x7f7d100cb1b0 2023-10-06T05:34:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f7d00001470 con 0x7f7d100cb1b0 2023-10-06T05:34:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 >> v1:172.21.15.33:6789/0 conn(0x7f7d100cb1b0 legacy=0x7f7d10150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.806+0000 7f934ed9d700 1 -- 172.21.15.33:0/543167891 learned_addr learned my addr 172.21.15.33:0/543167891 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f934ed9d700 1 -- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f935006a0d0 msgr2=0x7f93500c3a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:34:51.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f934ed9d700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f935006a0d0 0x7f93500c3a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f934ed9d700 1 -- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93500c9210 msgr2=0x7f93500c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f934ed9d700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93500c9210 0x7f93500c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f934ed9d700 1 -- 172.21.15.33:0/543167891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348009a20 con 0x7f9350140d20 2023-10-06T05:34:51.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cfc004c80 0x7f7cfc005070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cfc0037b0 0x7f7cfc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc004160 0x7f7cfc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100cb960 con 0x7f7cfc004160 2023-10-06T05:34:51.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc00c020 con 0x7f7cfc0037b0 2023-10-06T05:34:51.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc00c1e0 con 0x7f7cfc004c80 2023-10-06T05:34:51.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0e59c700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cfc004c80 0x7f7cfc005070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f934ed9d700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 0x7f9350157ea0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f9348003000 tx=0x7f9348018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.807+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cfc0037b0 0x7f7cfc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f700e9ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7008151b50 0x7f70080b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0ed9d700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc004160 0x7f7cfc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348027470 con 0x7f9350140d20 2023-10-06T05:34:51.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348027630 con 0x7f9350140d20 2023-10-06T05:34:51.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500b18e0 con 0x7f9350140d20 2023-10-06T05:34:51.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cfc004c80 msgr2=0x7f7cfc005070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480279f0 con 0x7f9350140d20 2023-10-06T05:34:51.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cfc004c80 0x7f7cfc005070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f700e9ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70080b3d60 0x7f7008157de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7007fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7008151b50 0x7f70080b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1310089512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc004160 msgr2=0x7f7cfc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f7007fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7008151b50 0x7f70080b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50494/0 (socket says 172.21.15.33:50494) 2023-10-06T05:34:51.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc004160 0x7f7cfc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f7007fff700 1 -- 172.21.15.33:0/2439276635 learned_addr learned my addr 172.21.15.33:0/2439276635 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1310089512 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100cbc40 con 0x7f7cfc0037b0 2023-10-06T05:34:51.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080cae50 con 0x7f7008158320 2023-10-06T05:34:51.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70080cbd10 con 0x7f7008151b50 2023-10-06T05:34:51.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700806a490 con 0x7f70080b3d60 2023-10-06T05:34:51.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cfc0037b0 0x7f7cfc003c20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c360 tx=0x7f7d00001e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73dcfc6193430fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000cc10 con 0x7f7cfc0037b0 2023-10-06T05:34:51.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f70077fe700 1 --2- 172.21.15.33:0/2439276635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70080b3d60 0x7f7008157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.808+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500b0f40 con 0x7f9350140d20 2023-10-06T05:34:51.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0000cd90 con 0x7f7cfc0037b0 2023-10-06T05:34:51.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.809+0000 7f7d0d59a700 1 -- 172.21.15.33:0/1310089512 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00015960 con 0x7f7cfc0037b0 2023-10-06T05:34:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.810+0000 7f7007fff700 1 -- 172.21.15.33:0/2439276635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70080b3d60 msgr2=0x7f7008157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.810+0000 7f7007fff700 1 --2- 172.21.15.33:0/2439276635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70080b3d60 0x7f7008157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.810+0000 7f7007fff700 1 -- 172.21.15.33:0/2439276635 >> v1:172.21.15.33:6789/0 conn(0x7f7008158320 legacy=0x7f700815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.810+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93340052b0 con 0x7f9350140d20 2023-10-06T05:34:51.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.810+0000 7f7007fff700 1 -- 172.21.15.33:0/2439276635 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7008069480 con 0x7f7008151b50 2023-10-06T05:34:51.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7007fff700 1 --2- 172.21.15.33:0/2439276635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7008151b50 0x7f70080b3820 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80099b0 tx=0x7f6ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfd825930fc8d45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cfc0037b0 msgr2=0x7f7cfc003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1310089512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cfc0037b0 0x7f7cfc003c20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c360 tx=0x7f7d00001e20 comp rx=0 tx=0).stop 2023-10-06T05:34:51.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7006ffd700 1 -- 172.21.15.33:0/2439276635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8009ea0 con 0x7f7008151b50 2023-10-06T05:34:51.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7d0ed9d700 1 -- 172.21.15.33:0/1310089512 reap_dead start 2023-10-06T05:34:51.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7006ffd700 1 -- 172.21.15.33:0/2439276635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff800b860 con 0x7f7008151b50 2023-10-06T05:34:51.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.811+0000 7f7006ffd700 1 -- 172.21.15.33:0/2439276635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800bc00 con 0x7f7008151b50 2023-10-06T05:34:51.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 shutdown_connections 2023-10-06T05:34:51.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 >> 172.21.15.33:0/1310089512 conn(0x7f7d10099960 msgr2=0x7f7d1006cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:51.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 shutdown_connections 2023-10-06T05:34:51.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9348024070 con 0x7f9350140d20 2023-10-06T05:34:51.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7008151b50 msgr2=0x7f70080b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f700e9ff700 1 --2- 172.21.15.33:0/2439276635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7008151b50 0x7f70080b3820 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80099b0 tx=0x7f6ff8009250 comp rx=0 tx=0).stop 2023-10-06T05:34:51.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.812+0000 7f7d14bff700 1 -- 172.21.15.33:0/1310089512 wait complete. 2023-10-06T05:34:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.813+0000 7f933e7fc700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9330067000 0x7f93300694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.813+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 shutdown_connections 2023-10-06T05:34:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.813+0000 7f700e9ff700 1 --2- 172.21.15.33:0/2439276635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7008151b50 0x7f70080b3820 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.813+0000 7f700e9ff700 1 --2- 172.21.15.33:0/2439276635 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70080b3d60 0x7f7008157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.813+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 >> 172.21.15.33:0/2439276635 conn(0x7f7008067df0 msgr2=0x7f70080681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:51.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.814+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9348086f90 con 0x7f9350140d20 2023-10-06T05:34:51.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.814+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 shutdown_connections 2023-10-06T05:34:51.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.814+0000 7f700e9ff700 1 -- 172.21.15.33:0/2439276635 wait complete. 2023-10-06T05:34:51.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.814+0000 7f934e59c700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9330067000 0x7f93300694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.815+0000 7f934e59c700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9330067000 0x7f93300694b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9340006fd0 tx=0x7f9340009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.815+0000 7f700e9ff700 1 Processor -- start 2023-10-06T05:34:51.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.816+0000 7f700e9ff700 1 -- start start 2023-10-06T05:34:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.816+0000 7f7d14bff700 1 Processor -- start 2023-10-06T05:34:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.816+0000 7f7d14bff700 1 -- start start 2023-10-06T05:34:51.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700e9ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 0x7f70081665d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700e9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7008151b50 0x7f700814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700e9ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7008158320 0x7f7008154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700e9ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008155070 con 0x7f70080b3d60 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700e9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008155350 con 0x7f7008151b50 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700e9ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7008165dd0 con 0x7f7008158320 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f7007fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 0x7f70081665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f7007fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 0x7f70081665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50510/0 (socket says 172.21.15.33:50510) 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f7007fff700 1 -- 172.21.15.33:0/3103490762 learned_addr learned my addr 172.21.15.33:0/3103490762 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f70077fe700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7008151b50 0x7f700814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.817+0000 7f700cf9c700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7008158320 0x7f7008154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7007fff700 1 -- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7008158320 msgr2=0x7f7008154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7007fff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7008158320 0x7f7008154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7007fff700 1 -- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7008151b50 msgr2=0x7f700814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7007fff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7008151b50 0x7f700814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7007fff700 1 -- 172.21.15.33:0/3103490762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80094f0 con 0x7f70080b3d60 2023-10-06T05:34:51.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f70077fe700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7008151b50 0x7f700814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:51.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7007fff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 0x7f70081665d0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f7008064e40 tx=0x7f6ff8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.818+0000 7f7d14bff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc0037b0 0x7f7d100c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d14bff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 0x7f7d100b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d14bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 0x7f7d101533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d14bff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10153920 con 0x7f7cfc0037b0 2023-10-06T05:34:51.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d14bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100c4a70 con 0x7f7d10152f70 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d14bff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100c41b0 con 0x7f7d100b4850 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d0e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc0037b0 0x7f7d100c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 0x7f7d101533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d0dd9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 0x7f7d100b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 0x7f7d101533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:50896/0 (socket says 172.21.15.33:50896) 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d0dd9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 0x7f7d100b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40380/0 (socket says 172.21.15.33:40380) 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.819+0000 7f7d0ed9d700 1 -- 172.21.15.33:0/1186007732 learned_addr learned my addr 172.21.15.33:0/1186007732 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800bbe0 con 0x7f70080b3d60 2023-10-06T05:34:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011420 con 0x7f70080b3d60 2023-10-06T05:34:51.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80117c0 con 0x7f70080b3d60 2023-10-06T05:34:51.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 msgr2=0x7f7d101533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 0x7f7d101533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1186007732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc0037b0 msgr2=0x7f7d100c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9348054110 con 0x7f9350140d20 2023-10-06T05:34:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70081660b0 con 0x7f70080b3d60 2023-10-06T05:34:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc0037b0 0x7f7d100c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0dd9b700 1 -- 172.21.15.33:0/1186007732 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100c4420 con 0x7f7d100b4850 2023-10-06T05:34:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0ed9d700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 0x7f7d101533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0e59c700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc0037b0 0x7f7d100c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:51.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f7d0dd9b700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 0x7f7d100b4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7d00001eb0 tx=0x7f7d0001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.820+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7008154cd0 con 0x7f70080b3d60 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.821+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00015960 con 0x7f7d100b4850 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.821+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00004a40 con 0x7f7d100b4850 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.821+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00004e00 con 0x7f7d100b4850 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.821+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d100c4700 con 0x7f7d100b4850 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.821+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d100b5310 con 0x7f7d100b4850 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.823+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d100c4c20 con 0x7f7d100b4850 2023-10-06T05:34:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.823+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008063ff0 con 0x7f70080b3d60 2023-10-06T05:34:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.823+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d0001b020 con 0x7f7d100b4850 2023-10-06T05:34:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.824+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ff8011bf0 con 0x7f70080b3d60 2023-10-06T05:34:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.824+0000 7f7d067fc700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7cf0067000 0x7f7cf00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.824+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7d0008d3c0 con 0x7f7d100b4850 2023-10-06T05:34:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.825+0000 7f7d0e59c700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7cf0067000 0x7f7cf00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.825+0000 7f7d0e59c700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7cf0067000 0x7f7cf00694b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8006d60 tx=0x7f7cf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.826+0000 7f6feffff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ff4067000 0x7f6ff40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.826+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6ff8085f20 con 0x7f70080b3d60 2023-10-06T05:34:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.826+0000 7f70077fe700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ff4067000 0x7f6ff40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.827+0000 7f70077fe700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ff4067000 0x7f6ff40694b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7008163670 tx=0x7f6ffc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.829+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d100c4c20 con 0x7f7d100b4850 2023-10-06T05:34:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.833+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ff8052ff0 con 0x7f70080b3d60 2023-10-06T05:34:51.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.869+0000 7f38d739d700 1 Processor -- start 2023-10-06T05:34:51.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.869+0000 7f38d739d700 1 -- start start 2023-10-06T05:34:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.870+0000 7f38d739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d0159400 0x7f38d00cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.870+0000 7f38d739d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38d006cc70 0x7f38d00cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.870+0000 7f38d739d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00d0ed0 con 0x7f38d01597d0 2023-10-06T05:34:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.870+0000 7f38d739d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00b1160 con 0x7f38d006cc70 2023-10-06T05:34:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.870+0000 7f38d739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00cb6d0 con 0x7f38d0159400 2023-10-06T05:34:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38d006cc70 0x7f38d00cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38d006cc70 0x7f38d00cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50526/0 (socket says 172.21.15.33:50526) 2023-10-06T05:34:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 -- 172.21.15.33:0/3778079891 learned_addr learned my addr 172.21.15.33:0/3778079891 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d639b700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d0159400 0x7f38d00cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 -- 172.21.15.33:0/3778079891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d0159400 msgr2=0x7f38d00cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d0159400 0x7f38d00cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 -- 172.21.15.33:0/3778079891 >> v1:172.21.15.167:6789/0 conn(0x7f38d01597d0 legacy=0x7f38d006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d6b9c700 1 -- 172.21.15.33:0/3778079891 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00cb9b0 con 0x7f38d006cc70 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.871+0000 7f38d639b700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d0159400 0x7f38d00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.872+0000 7f38d6b9c700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d006cc70 0x7f38d00cb0a0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009f20 tx=0x7f38c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9240e4ff7c2e4d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.872+0000 7f38d5399700 1 -- 172.21.15.33:0/3778079891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800e070 con 0x7f38d006cc70 2023-10-06T05:34:51.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.872+0000 7f38d5399700 1 -- 172.21.15.33:0/3778079891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c800b8d0 con 0x7f38d006cc70 2023-10-06T05:34:51.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.872+0000 7f38d5399700 1 -- 172.21.15.33:0/3778079891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80092e0 con 0x7f38d006cc70 2023-10-06T05:34:51.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 -- 172.21.15.33:0/3778079891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d006cc70 msgr2=0x7f38d00cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d006cc70 0x7f38d00cb0a0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009f20 tx=0x7f38c8009840 comp rx=0 tx=0).stop 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 -- 172.21.15.33:0/3778079891 shutdown_connections 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d006cc70 0x7f38d00cb0a0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 --2- 172.21.15.33:0/3778079891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d0159400 0x7f38d00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 -- 172.21.15.33:0/3778079891 >> 172.21.15.33:0/3778079891 conn(0x7f38d006e4b0 msgr2=0x7f38d00d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 -- 172.21.15.33:0/3778079891 shutdown_connections 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.873+0000 7f38d739d700 1 -- 172.21.15.33:0/3778079891 wait complete. 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.875+0000 7f38d739d700 1 Processor -- start 2023-10-06T05:34:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.875+0000 7f38d739d700 1 -- start start 2023-10-06T05:34:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38d006cc70 0x7f38d0153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d739d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 0x7f38d0169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d739d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d01597d0 0x7f38d015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d739d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d016ddc0 con 0x7f38d0159400 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b3d20 con 0x7f38d006cc70 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d739d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b33f0 con 0x7f38d01597d0 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d5b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 0x7f38d0169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d5b9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 0x7f38d0169890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50532/0 (socket says 172.21.15.33:50532) 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d5b9a700 1 -- 172.21.15.33:0/2160309439 learned_addr learned my addr 172.21.15.33:0/2160309439 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.876+0000 7f38d6b9c700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d01597d0 0x7f38d015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d639b700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38d006cc70 0x7f38d0153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d5b9a700 1 -- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d01597d0 msgr2=0x7f38d015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d5b9a700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d01597d0 0x7f38d015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d5b9a700 1 -- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38d006cc70 msgr2=0x7f38d0153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d5b9a700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38d006cc70 0x7f38d0153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d5b9a700 1 -- 172.21.15.33:0/2160309439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009420 con 0x7f38d0159400 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d639b700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38d006cc70 0x7f38d0153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d5b9a700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 0x7f38d0169890 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c810 tx=0x7f38c0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0011590 con 0x7f38d0159400 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c00116f0 con 0x7f38d0159400 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0011ab0 con 0x7f38d0159400 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.877+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c8009ae0 con 0x7f38d0159400 2023-10-06T05:34:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.878+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00b39b0 con 0x7f38d0159400 2023-10-06T05:34:51.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.879+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d004c380 con 0x7f38d0159400 2023-10-06T05:34:51.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.880+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38c001b3b0 con 0x7f38d0159400 2023-10-06T05:34:51.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.880+0000 7f38c67fc700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f38b4066fe0 0x7f38b4069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:51.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.881+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f38c0083ce0 con 0x7f38d0159400 2023-10-06T05:34:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.881+0000 7f38d639b700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f38b4066fe0 0x7f38b4069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.882+0000 7f38d639b700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f38b4066fe0 0x7f38b4069490 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0093f0 tx=0x7f38cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:51.885+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38c0050d30 con 0x7f38d0159400 2023-10-06T05:34:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.128+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f7d100c4c20 con 0x7f7d100b4850 2023-10-06T05:34:52.132 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:34:52.129+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:34:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.139+0000 7f7d067fc700 1 -- 172.21.15.33:0/1186007732 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 10681 v204) v1 ==== 96+0+0 (secure 0 0 0) 0x7f7d100c4c20 con 0x7f7d100b4850 2023-10-06T05:34:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7cf0067000 msgr2=0x7f7cf00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7cf0067000 0x7f7cf00694b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8006d60 tx=0x7f7cf8008040 comp rx=0 tx=0).stop 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 msgr2=0x7f7d100b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 0x7f7d100b4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7d00001eb0 tx=0x7f7d0001a000 comp rx=0 tx=0).stop 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 shutdown_connections 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7cf0067000 0x7f7cf00694b0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d10152f70 0x7f7d101533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc0037b0 0x7f7d100c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 --2- 172.21.15.33:0/1186007732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d100b4850 0x7f7d100b4cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 >> 172.21.15.33:0/1186007732 conn(0x7f7d10099960 msgr2=0x7f7d100aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 shutdown_connections 2023-10-06T05:34:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.143+0000 7f7d14bff700 1 -- 172.21.15.33:0/1186007732 wait complete. 2023-10-06T05:34:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 10681 2023-10-06T05:34:52.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.149+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f9334005560 con 0x7f9350140d20 2023-10-06T05:34:52.165 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:34:52.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:34:52.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.162+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f70080cf280 con 0x7f70080b3d60 2023-10-06T05:34:52.169 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:34:52.170 DEBUG:teuthology.parallel:result is None 2023-10-06T05:34:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:52.185+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f38d00cbc90 con 0x7f38d0159400 2023-10-06T05:34:52.299 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:34:53.032 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:34:53.029+0000 7f50273ec700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:34:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.037+0000 7f933e7fc700 1 -- 172.21.15.33:0/543167891 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 8924 v206) v1 ==== 95+0+0 (secure 0 0 0) 0x7f9348057730 con 0x7f9350140d20 2023-10-06T05:34:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.037+0000 7f6feffff700 1 -- 172.21.15.33:0/3103490762 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 10678 v206) v1 ==== 96+0+0 (secure 0 0 0) 0x7f6ff8056610 con 0x7f70080b3d60 2023-10-06T05:34:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ff4067000 msgr2=0x7f6ff40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ff4067000 0x7f6ff40694b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7008163670 tx=0x7f6ffc008040 comp rx=0 tx=0).stop 2023-10-06T05:34:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 msgr2=0x7f70081665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 0x7f70081665d0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f7008064e40 tx=0x7f6ff8017000 comp rx=0 tx=0).stop 2023-10-06T05:34:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9330067000 msgr2=0x7f93300694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9330067000 0x7f93300694b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9340006fd0 tx=0x7f9340009250 comp rx=0 tx=0).stop 2023-10-06T05:34:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 msgr2=0x7f9350157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 0x7f9350157ea0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f9348003000 tx=0x7f9348018000 comp rx=0 tx=0).stop 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 shutdown_connections 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6ff4067000 0x7f6ff40694b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7008158320 0x7f7008154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70080b3d60 0x7f70081665d0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 --2- 172.21.15.33:0/3103490762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7008151b50 0x7f700814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 >> 172.21.15.33:0/3103490762 conn(0x7f7008067df0 msgr2=0x7f700806e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 shutdown_connections 2023-10-06T05:34:53.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 shutdown_connections 2023-10-06T05:34:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9330067000 0x7f93300694b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f700e9ff700 1 -- 172.21.15.33:0/3103490762 wait complete. 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9350140d20 0x7f9350157ea0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f935006a0d0 0x7f93500c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 --2- 172.21.15.33:0/543167891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93500c9210 0x7f93500c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 >> 172.21.15.33:0/543167891 conn(0x7f935006c560 msgr2=0x7f93500c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.041+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 shutdown_connections 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.042+0000 7f934f59e700 1 -- 172.21.15.33:0/543167891 wait complete. 2023-10-06T05:34:53.047 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 10678 2023-10-06T05:34:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 8924 2023-10-06T05:34:53.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.045+0000 7f38c67fc700 1 -- 172.21.15.33:0/2160309439 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 8921 v207) v1 ==== 95+0+0 (secure 0 0 0) 0x7f38c0017070 con 0x7f38d0159400 2023-10-06T05:34:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.050+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f38b4066fe0 msgr2=0x7f38b4069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.050+0000 7f38d739d700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f38b4066fe0 0x7f38b4069490 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0093f0 tx=0x7f38cc008040 comp rx=0 tx=0).stop 2023-10-06T05:34:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.050+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 msgr2=0x7f38d0169890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.050+0000 7f38d739d700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 0x7f38d0169890 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c810 tx=0x7f38c0007f30 comp rx=0 tx=0).stop 2023-10-06T05:34:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.050+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 shutdown_connections 2023-10-06T05:34:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f38b4066fe0 0x7f38b4069490 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38d01597d0 0x7f38d015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38d006cc70 0x7f38d0153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 --2- 172.21.15.33:0/2160309439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38d0159400 0x7f38d0169890 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 >> 172.21.15.33:0/2160309439 conn(0x7f38d006e4b0 msgr2=0x7f38d00c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:53.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 shutdown_connections 2023-10-06T05:34:53.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.051+0000 7f38d739d700 1 -- 172.21.15.33:0/2160309439 wait complete. 2023-10-06T05:34:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:failed mds gid 8921 2023-10-06T05:34:53.074 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:34:53.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:34:53.076 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:34:53.076 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:34:53.079 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:34:53.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:34:53.082 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:34:53.082 DEBUG:teuthology.parallel:result is None 2023-10-06T05:34:53.083 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:34:53.083 DEBUG:teuthology.parallel:result is None 2023-10-06T05:34:53.086 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:34:53.086 DEBUG:teuthology.parallel:result is None 2023-10-06T05:34:53.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:53.219 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:34:53.232 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:34:53.232 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:34:53.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.294+0000 7ff821d9d700 1 Processor -- start 2023-10-06T05:34:53.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.294+0000 7ff821d9d700 1 -- start start 2023-10-06T05:34:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.295+0000 7ff821d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff81c13faa0 0x7ff81c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.295+0000 7ff821d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 0x7ff81c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.295+0000 7ff821d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0c9cb0 con 0x7ff81c06aad0 2023-10-06T05:34:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.295+0000 7ff821d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06b660 con 0x7ff81c13faa0 2023-10-06T05:34:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.295+0000 7ff821d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0ca270 con 0x7ff81c0c7940 2023-10-06T05:34:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 0x7ff81c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff820d9b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff81c13faa0 0x7ff81c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 0x7ff81c0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40466/0 (socket says 172.21.15.33:40466) 2023-10-06T05:34:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff81bfff700 1 -- 172.21.15.33:0/1175247192 learned_addr learned my addr 172.21.15.33:0/1175247192 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff81b7fe700 1 -- 172.21.15.33:0/1175247192 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868499038 0 0) 0x7ff81c0c9cb0 con 0x7ff81c06aad0 2023-10-06T05:34:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff81b7fe700 1 -- 172.21.15.33:0/1175247192 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80c003570 con 0x7ff81c06aad0 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff820d9b700 1 -- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 msgr2=0x7ff81c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff820d9b700 1 --2- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 0x7ff81c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff820d9b700 1 -- 172.21.15.33:0/1175247192 >> v1:172.21.15.167:6789/0 conn(0x7ff81c06aad0 legacy=0x7ff81c06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff820d9b700 1 -- 172.21.15.33:0/1175247192 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c06b840 con 0x7ff81c13faa0 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff81bfff700 1 --2- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 0x7ff81c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.296+0000 7ff820d9b700 1 --2- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c13faa0 0x7ff81c13fe70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff810009e60 tx=0x7ff810009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f642f6e773bf55a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff81b7fe700 1 -- 172.21.15.33:0/1175247192 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000e070 con 0x7ff81c13faa0 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff81b7fe700 1 -- 172.21.15.33:0/1175247192 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000bab0 con 0x7ff81c13faa0 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff81b7fe700 1 -- 172.21.15.33:0/1175247192 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000be70 con 0x7ff81c13faa0 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 -- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c13faa0 msgr2=0x7ff81c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 --2- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c13faa0 0x7ff81c13fe70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff810009e60 tx=0x7ff810009780 comp rx=0 tx=0).stop 2023-10-06T05:34:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 -- 172.21.15.33:0/1175247192 shutdown_connections 2023-10-06T05:34:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 --2- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c13faa0 0x7ff81c13fe70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 --2- 172.21.15.33:0/1175247192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c0c7940 0x7ff81c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 -- 172.21.15.33:0/1175247192 >> 172.21.15.33:0/1175247192 conn(0x7ff81c06d030 msgr2=0x7ff81c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 -- 172.21.15.33:0/1175247192 shutdown_connections 2023-10-06T05:34:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.297+0000 7ff821d9d700 1 -- 172.21.15.33:0/1175247192 wait complete. 2023-10-06T05:34:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.299+0000 7ff821d9d700 1 Processor -- start 2023-10-06T05:34:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.299+0000 7ff821d9d700 1 -- start start 2023-10-06T05:34:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff821d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c06aad0 0x7ff81c16c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff821d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 0x7ff81c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff821d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c13faa0 0x7ff81c16c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff821d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c16a430 con 0x7ff81c0c7940 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff821d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15e8c0 con 0x7ff81c06aad0 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff821d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15eba0 con 0x7ff81c13faa0 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 0x7ff81c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff820d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c06aad0 0x7ff81c16c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 0x7ff81c15de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50572/0 (socket says 172.21.15.33:50572) 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff81bfff700 1 -- 172.21.15.33:0/3942539984 learned_addr learned my addr 172.21.15.33:0/3942539984 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.300+0000 7ff82159c700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c13faa0 0x7ff81c16c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff82159c700 1 -- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c06aad0 msgr2=0x7ff81c16c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff82159c700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c06aad0 0x7ff81c16c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff82159c700 1 -- 172.21.15.33:0/3942539984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 msgr2=0x7ff81c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff82159c700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 0x7ff81c15de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff82159c700 1 -- 172.21.15.33:0/3942539984 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080094f0 con 0x7ff81c13faa0 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff81bfff700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 0x7ff81c15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff82159c700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c13faa0 0x7ff81c16c920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c560 tx=0x7ff814007cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814011590 con 0x7ff81c13faa0 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff814011710 con 0x7ff81c13faa0 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814011ad0 con 0x7ff81c13faa0 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.301+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810009a20 con 0x7ff81c13faa0 2023-10-06T05:34:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.302+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c15f160 con 0x7ff81c13faa0 2023-10-06T05:34:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.302+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c16a600 con 0x7ff81c13faa0 2023-10-06T05:34:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.304+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff814012420 con 0x7ff81c13faa0 2023-10-06T05:34:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.304+0000 7ff818ff9700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff800066f30 0x7ff8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.305+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff814084160 con 0x7ff81c13faa0 2023-10-06T05:34:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.305+0000 7ff820d9b700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff800066f30 0x7ff8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:53.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.305+0000 7ff820d9b700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff800066f30 0x7ff8000693e0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff8100097c0 tx=0x7ff810019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.309+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff81c16a600 con 0x7ff81c13faa0 2023-10-06T05:34:53.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.560+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff81c15ece0 con 0x7ff81c13faa0 2023-10-06T05:34:53.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.562+0000 7ff818ff9700 1 -- 172.21.15.33:0/3942539984 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+2339 (secure 0 0 0) 0x7ff81c15ece0 con 0x7ff81c13faa0 2023-10-06T05:34:53.564 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:53.564 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":207,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":207,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:53.039206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:replay","state_seq":1,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:53.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.563+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff800066f30 msgr2=0x7ff8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.563+0000 7ff821d9d700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff800066f30 0x7ff8000693e0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff8100097c0 tx=0x7ff810019000 comp rx=0 tx=0).stop 2023-10-06T05:34:53.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.563+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c13faa0 msgr2=0x7ff81c16c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.563+0000 7ff821d9d700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c13faa0 0x7ff81c16c920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c560 tx=0x7ff814007cf0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.563+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 shutdown_connections 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff800066f30 0x7ff8000693e0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff81c13faa0 0x7ff81c16c920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c06aad0 0x7ff81c16c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 --2- 172.21.15.33:0/3942539984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81c0c7940 0x7ff81c15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 >> 172.21.15.33:0/3942539984 conn(0x7ff81c06d030 msgr2=0x7ff81c06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 shutdown_connections 2023-10-06T05:34:53.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:53.564+0000 7ff821d9d700 1 -- 172.21.15.33:0/3942539984 wait complete. 2023-10-06T05:34:53.568 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 207 2023-10-06T05:34:53.584 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2023-10-06T05:34:54.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:54.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.782+0000 7fa6e5f9d700 1 Processor -- start 2023-10-06T05:34:54.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.782+0000 7fa6e5f9d700 1 -- start start 2023-10-06T05:34:54.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6e5f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa6e00cb710 0x7fa6e006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6e5f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e00c9cb0 con 0x7fa6e00c8050 2023-10-06T05:34:54.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6e5f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e00ca270 con 0x7fa6e013faa0 2023-10-06T05:34:54.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6e5f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e006b840 con 0x7fa6e00cb710 2023-10-06T05:34:54.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6dffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa6e00c8050 0x7fa6e006aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51966/0 (socket says 172.21.15.33:51966) 2023-10-06T05:34:54.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6dffff700 1 -- 172.21.15.33:0/2899134559 learned_addr learned my addr 172.21.15.33:0/2899134559 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6e579c700 1 --2- 172.21.15.33:0/2899134559 >> v2:172.21.15.167:3300/0 conn(0x7fa6e00cb710 0x7fa6e006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287141916 0 0) 0x7fa6e00c9cb0 con 0x7fa6e00c8050 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.784+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d0003570 con 0x7fa6e00c8050 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318964045 0 0) 0x7fa6d0003570 con 0x7fa6e00c8050 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 >> v2:172.21.15.167:3300/0 conn(0x7fa6e00cb710 msgr2=0x7fa6e006b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 --2- 172.21.15.33:0/2899134559 >> v2:172.21.15.167:3300/0 conn(0x7fa6e00cb710 0x7fa6e006b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 >> v1:172.21.15.167:6789/0 conn(0x7fa6e013faa0 legacy=0x7fa6e00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e006bb20 con 0x7fa6e00c8050 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa6cc001470 con 0x7fa6e00c8050 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 >> v1:172.21.15.33:6789/0 conn(0x7fa6e00c8050 legacy=0x7fa6e006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0003ab0 0x7fa6d0003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0004810 0x7fa6d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6d0005210 0x7fa6d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e006b840 con 0x7fa6d0003ab0 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000c130 con 0x7fa6d0004810 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000c2f0 con 0x7fa6d0005210 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6e579c700 1 --2- 172.21.15.33:0/2899134559 >> v2:172.21.15.167:3300/0 conn(0x7fa6e00cb710 0x7fa6e006b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0003ab0 0x7fa6d0003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6dffff700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0004810 0x7fa6d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.785+0000 7fa6e579c700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6d0005210 0x7fa6d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e4f9b700 1 -- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6d0005210 msgr2=0x7fa6d000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6d0005210 0x7fa6d000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e4f9b700 1 -- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0004810 msgr2=0x7fa6d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0004810 0x7fa6d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e4f9b700 1 -- 172.21.15.33:0/2899134559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0003570 con 0x7fa6d0003ab0 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6dffff700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0004810 0x7fa6d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e579c700 1 -- 172.21.15.33:0/2899134559 reap_dead start 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0003ab0 0x7fa6d0003ec0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40018b0 tx=0x7fa6d4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d833d00902b0bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4012420 con 0x7fa6d0003ab0 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d4012580 con 0x7fa6d0003ab0 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.786+0000 7fa6df7fe700 1 -- 172.21.15.33:0/2899134559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4012920 con 0x7fa6d0003ab0 2023-10-06T05:34:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2899134559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0003ab0 msgr2=0x7fa6d0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0003ab0 0x7fa6d0003ec0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40018b0 tx=0x7fa6d4007d30 comp rx=0 tx=0).stop 2023-10-06T05:34:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2899134559 shutdown_connections 2023-10-06T05:34:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2899134559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6d0003ab0 0x7fa6d0003ec0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2899134559 >> 172.21.15.33:0/2899134559 conn(0x7fa6e006d030 msgr2=0x7fa6e006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2899134559 shutdown_connections 2023-10-06T05:34:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.787+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2899134559 wait complete. 2023-10-06T05:34:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.788+0000 7fa6e5f9d700 1 Processor -- start 2023-10-06T05:34:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.788+0000 7fa6e5f9d700 1 -- start start 2023-10-06T05:34:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.789+0000 7fa6e5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0003ab0 0x7fa6e0158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.789+0000 7fa6e5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6e0165b80 0x7fa6e013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.789+0000 7fa6e5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 0x7fa6e0158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.789+0000 7fa6e5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01587c0 con 0x7fa6e00a56b0 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.789+0000 7fa6e5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01571e0 con 0x7fa6d0003ab0 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.789+0000 7fa6e5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01574c0 con 0x7fa6e0165b80 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 0x7fa6e0158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 0x7fa6e0158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50582/0 (socket says 172.21.15.33:50582) 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 -- 172.21.15.33:0/2209202163 learned_addr learned my addr 172.21.15.33:0/2209202163 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0003ab0 0x7fa6e0158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6dffff700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6e0165b80 0x7fa6e013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 -- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6e0165b80 msgr2=0x7fa6e013dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:54.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6e0165b80 0x7fa6e013dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 -- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0003ab0 msgr2=0x7fa6e0158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0003ab0 0x7fa6e0158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 -- 172.21.15.33:0/2209202163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00c42e0 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6e579c700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 0x7fa6e0158f60 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00cd0f0 tx=0x7fa6d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.790+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011e60 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.791+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8014450 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.791+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8014810 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.791+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e00c45c0 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.791+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e013e3c0 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.792+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e004c380 con 0x7fa6e00a56b0 2023-10-06T05:34:54.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.792+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6d8014b80 con 0x7fa6e00a56b0 2023-10-06T05:34:54.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.793+0000 7fa6dcff9700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa6c4066f30 0x7fa6c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:54.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.793+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fa6d8087690 con 0x7fa6e00a56b0 2023-10-06T05:34:54.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.794+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa6c4066f30 0x7fa6c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:54.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.795+0000 7fa6e4f9b700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa6c4066f30 0x7fa6c40693e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001010 tx=0x7fa6d4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:54.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:54.797+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6d8054630 con 0x7fa6e00a56b0 2023-10-06T05:34:55.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.045+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6e00c9f00 con 0x7fa6e00a56b0 2023-10-06T05:34:55.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.046+0000 7fa6dcff9700 1 -- 172.21.15.33:0/2209202163 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4620 (secure 0 0 0) 0x7fa6d8057c50 con 0x7fa6e00a56b0 2023-10-06T05:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":208,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:54.041235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:55.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa6c4066f30 msgr2=0x7fa6c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:55.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa6c4066f30 0x7fa6c40693e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001010 tx=0x7fa6d4015000 comp rx=0 tx=0).stop 2023-10-06T05:34:55.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 msgr2=0x7fa6e0158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:55.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 0x7fa6e0158f60 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00cd0f0 tx=0x7fa6d800b040 comp rx=0 tx=0).stop 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 shutdown_connections 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa6c4066f30 0x7fa6c40693e0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6e00a56b0 0x7fa6e0158f60 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6d0003ab0 0x7fa6e0158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 --2- 172.21.15.33:0/2209202163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6e0165b80 0x7fa6e013dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 >> 172.21.15.33:0/2209202163 conn(0x7fa6e006d030 msgr2=0x7fa6e00c93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 shutdown_connections 2023-10-06T05:34:55.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:55.048+0000 7fa6e5f9d700 1 -- 172.21.15.33:0/2209202163 wait complete. 2023-10-06T05:34:55.053 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 208 2023-10-06T05:34:55.069 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2023-10-06T05:34:56.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.265+0000 7fd7b5f9d700 1 Processor -- start 2023-10-06T05:34:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.265+0000 7fd7b5f9d700 1 -- start start 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7b5f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7b013faa0 0x7fd7b013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7b5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b006aad0 0x7fd7b006af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7b5f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b00c9cb0 con 0x7fd7b00c7940 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7b5f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b00ca270 con 0x7fd7b013faa0 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7b5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b006b840 con 0x7fd7b006aad0 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7b4f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd7b013faa0 0x7fd7b013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7affff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd7b00c7940 0x7fd7b00c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51978/0 (socket says 172.21.15.33:51978) 2023-10-06T05:34:56.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.266+0000 7fd7affff700 1 -- 172.21.15.33:0/1399450862 learned_addr learned my addr 172.21.15.33:0/1399450862 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b579c700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b006aad0 0x7fd7b006af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7af7fe700 1 -- 172.21.15.33:0/1399450862 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307310146 0 0) 0x7fd7b00c9cb0 con 0x7fd7b00c7940 2023-10-06T05:34:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7af7fe700 1 -- 172.21.15.33:0/1399450862 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a0003570 con 0x7fd7b00c7940 2023-10-06T05:34:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1399450862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b006aad0 msgr2=0x7fd7b006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b4f9b700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b006aad0 0x7fd7b006af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1399450862 >> v1:172.21.15.33:6789/0 conn(0x7fd7b00c7940 legacy=0x7fd7b00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1399450862 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b006bb20 con 0x7fd7b013faa0 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b579c700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b006aad0 0x7fd7b006af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7b4f9b700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b013faa0 0x7fd7b013fe70 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40098e0 tx=0x7fd7a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dc635cfa911b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7af7fe700 1 -- 172.21.15.33:0/1399450862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400b640 con 0x7fd7b013faa0 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7af7fe700 1 -- 172.21.15.33:0/1399450862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400b7a0 con 0x7fd7b013faa0 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.267+0000 7fd7af7fe700 1 -- 172.21.15.33:0/1399450862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400bb20 con 0x7fd7b013faa0 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1399450862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b013faa0 msgr2=0x7fd7b013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b013faa0 0x7fd7b013fe70 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40098e0 tx=0x7fd7a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1399450862 shutdown_connections 2023-10-06T05:34:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b013faa0 0x7fd7b013fe70 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1399450862 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b006aad0 0x7fd7b006af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1399450862 >> 172.21.15.33:0/1399450862 conn(0x7fd7b006d030 msgr2=0x7fd7b006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1399450862 shutdown_connections 2023-10-06T05:34:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.268+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1399450862 wait complete. 2023-10-06T05:34:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.269+0000 7fd7b5f9d700 1 Processor -- start 2023-10-06T05:34:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.270+0000 7fd7b5f9d700 1 -- start start 2023-10-06T05:34:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b5f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 0x7fd7b0158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00c7940 0x7fd7b013bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b5f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b013faa0 0x7fd7b016d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b5f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01585d0 con 0x7fd7b006aad0 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b0157d10 con 0x7fd7b00c7940 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b5f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b016def0 con 0x7fd7b013faa0 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 0x7fd7b0158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 0x7fd7b0158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50596/0 (socket says 172.21.15.33:50596) 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1959467598 learned_addr learned my addr 172.21.15.33:0/1959467598 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b579c700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b013faa0 0x7fd7b016d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7affff700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00c7940 0x7fd7b013bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b013faa0 msgr2=0x7fd7b016d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b013faa0 0x7fd7b016d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00c7940 msgr2=0x7fd7b013bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00c7940 0x7fd7b013bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7b4f9b700 1 -- 172.21.15.33:0/1959467598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4009420 con 0x7fd7b006aad0 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.271+0000 7fd7affff700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00c7940 0x7fd7b013bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.272+0000 7fd7b4f9b700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 0x7fd7b0158a60 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00719c0 tx=0x7fd7a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.272+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400b910 con 0x7fd7b006aad0 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.272+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400bab0 con 0x7fd7b006aad0 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.272+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4025670 con 0x7fd7b006aad0 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.272+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b0157f80 con 0x7fd7b006aad0 2023-10-06T05:34:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.272+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b0155cb0 con 0x7fd7b006aad0 2023-10-06T05:34:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.273+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b006c180 con 0x7fd7b006aad0 2023-10-06T05:34:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.274+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7a4023070 con 0x7fd7b006aad0 2023-10-06T05:34:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.275+0000 7fd7acff9700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd794066f30 0x7fd7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.275+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fd7a4085c00 con 0x7fd7b006aad0 2023-10-06T05:34:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.275+0000 7fd7affff700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd794066f30 0x7fd7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.276+0000 7fd7affff700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd794066f30 0x7fd7940693e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0058d0 tx=0x7fd79c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.279+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7a4052c50 con 0x7fd7b006aad0 2023-10-06T05:34:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.527+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b004c380 con 0x7fd7b006aad0 2023-10-06T05:34:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.528+0000 7fd7acff9700 1 -- 172.21.15.33:0/1959467598 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4617 (secure 0 0 0) 0x7fd7a4015090 con 0x7fd7b006aad0 2023-10-06T05:34:56.530 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:56.531 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:56.050401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd794066f30 msgr2=0x7fd7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd794066f30 0x7fd7940693e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0058d0 tx=0x7fd79c009040 comp rx=0 tx=0).stop 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 msgr2=0x7fd7b0158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 0x7fd7b0158a60 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00719c0 tx=0x7fd7a4017000 comp rx=0 tx=0).stop 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 shutdown_connections 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd794066f30 0x7fd7940693e0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd7b013faa0 0x7fd7b016d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7b006aad0 0x7fd7b0158a60 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 --2- 172.21.15.33:0/1959467598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b00c7940 0x7fd7b013bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 >> 172.21.15.33:0/1959467598 conn(0x7fd7b006d030 msgr2=0x7fd7b00cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 shutdown_connections 2023-10-06T05:34:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.530+0000 7fd7b5f9d700 1 -- 172.21.15.33:0/1959467598 wait complete. 2023-10-06T05:34:56.534 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 210 2023-10-06T05:34:56.550 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2023-10-06T05:34:56.550 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 3.463827610015869s 2023-10-06T05:34:56.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:56.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.733+0000 7ff16f39d700 1 Processor -- start 2023-10-06T05:34:56.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.734+0000 7ff16f39d700 1 -- start start 2023-10-06T05:34:56.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.735+0000 7ff16f39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff16813faa0 0x7ff16813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.735+0000 7ff16f39d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680c9cb0 con 0x7ff16806aad0 2023-10-06T05:34:56.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.735+0000 7ff16f39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680ca270 con 0x7ff1680c7940 2023-10-06T05:34:56.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.735+0000 7ff16f39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16806b640 con 0x7ff16813faa0 2023-10-06T05:34:56.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.735+0000 7ff16eb9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff16806aad0 0x7ff16806af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51980/0 (socket says 172.21.15.33:51980) 2023-10-06T05:34:56.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.735+0000 7ff16eb9c700 1 -- 172.21.15.33:0/2411546977 learned_addr learned my addr 172.21.15.33:0/2411546977 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:34:56.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16e39b700 1 --2- 172.21.15.33:0/2411546977 >> v2:172.21.15.167:3300/0 conn(0x7ff16813faa0 0x7ff16813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711909098 0 0) 0x7ff1680c9cb0 con 0x7ff16806aad0 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff154003570 con 0x7ff16806aad0 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668334629 0 0) 0x7ff1680ca270 con 0x7ff1680c7940 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1680c9cb0 con 0x7ff1680c7940 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402246404 0 0) 0x7ff154003570 con 0x7ff16806aad0 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 >> v2:172.21.15.167:3300/0 conn(0x7ff16813faa0 msgr2=0x7ff16813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 --2- 172.21.15.33:0/2411546977 >> v2:172.21.15.167:3300/0 conn(0x7ff16813faa0 0x7ff16813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 >> v1:172.21.15.167:6789/0 conn(0x7ff1680c7940 legacy=0x7ff1680c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16806b840 con 0x7ff16806aad0 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7ff1600017f0 con 0x7ff16806aad0 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16e39b700 1 --2- 172.21.15.33:0/2411546977 >> v2:172.21.15.167:3300/0 conn(0x7ff16813faa0 0x7ff16813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:34:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 >> v1:172.21.15.33:6789/0 conn(0x7ff16806aad0 legacy=0x7ff16806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff154003bf0 0x7ff154003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff154004810 0x7ff154004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff15400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16806b640 con 0x7ff1540051c0 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15400c0b0 con 0x7ff154003bf0 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.736+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15400c270 con 0x7ff154004810 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff15400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16e39b700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff154003bf0 0x7ff154003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16db9a700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff154004810 0x7ff154004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 -- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff154004810 msgr2=0x7ff154004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff154004810 0x7ff154004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 -- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff154003bf0 msgr2=0x7ff154003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff154003bf0 0x7ff154003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 -- 172.21.15.33:0/2411546977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff154003570 con 0x7ff1540051c0 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 -- 172.21.15.33:0/2411546977 reap_dead start 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.737+0000 7ff16eb9c700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff15400ba60 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7ff16000a3a0 tx=0x7ff1680c84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3239b24e0b2412c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160010570 con 0x7ff1540051c0 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff160010710 con 0x7ff1540051c0 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16d399700 1 -- 172.21.15.33:0/2411546977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160010ad0 con 0x7ff1540051c0 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 -- 172.21.15.33:0/2411546977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 msgr2=0x7ff15400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff15400ba60 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7ff16000a3a0 tx=0x7ff1680c84c0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 -- 172.21.15.33:0/2411546977 shutdown_connections 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 --2- 172.21.15.33:0/2411546977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff15400ba60 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 -- 172.21.15.33:0/2411546977 >> 172.21.15.33:0/2411546977 conn(0x7ff16806d030 msgr2=0x7ff16806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:56.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 -- 172.21.15.33:0/2411546977 shutdown_connections 2023-10-06T05:34:56.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.738+0000 7ff16f39d700 1 -- 172.21.15.33:0/2411546977 wait complete. 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.741+0000 7ff16f39d700 1 Processor -- start 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.741+0000 7ff16f39d700 1 -- start start 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff16813ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff168159360 0x7ff1681558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff16815a0a0 0x7ff16815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16815aa50 con 0x7ff1540051c0 2023-10-06T05:34:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681697c0 con 0x7ff168159360 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681597d0 con 0x7ff16815a0a0 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff16813ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff16813ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50608/0 (socket says 172.21.15.33:50608) 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 -- 172.21.15.33:0/23432171 learned_addr learned my addr 172.21.15.33:0/23432171 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16db9a700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff168159360 0x7ff1681558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16eb9c700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff16815a0a0 0x7ff16815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 -- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff16815a0a0 msgr2=0x7ff16815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff16815a0a0 0x7ff16815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 -- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff168159360 msgr2=0x7ff1681558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff168159360 0x7ff1681558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 -- 172.21.15.33:0/23432171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16813db00 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16e39b700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff16813ed80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7ff164011d70 tx=0x7ff1640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164011e70 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16400fcd0 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16813dde0 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16401d6d0 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.742+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff168155de0 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.743+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff168169aa0 con 0x7ff1540051c0 2023-10-06T05:34:56.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.744+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff16401dac0 con 0x7ff1540051c0 2023-10-06T05:34:56.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.745+0000 7ff15e7fc700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff14c066f30 0x7ff14c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:56.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.745+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff164018070 con 0x7ff1540051c0 2023-10-06T05:34:56.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.745+0000 7ff16db9a700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff14c066f30 0x7ff14c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:56.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.746+0000 7ff16db9a700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff14c066f30 0x7ff14c0693e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff158007fd0 tx=0x7ff15800c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.749+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff164052620 con 0x7ff1540051c0 2023-10-06T05:34:56.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.993+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff16804c380 con 0x7ff1540051c0 2023-10-06T05:34:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.993+0000 7ff15e7fc700 1 -- 172.21.15.33:0/23432171 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4617 (secure 0 0 0) 0x7ff164055c40 con 0x7ff1540051c0 2023-10-06T05:34:56.996 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:56.996 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:56.050401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff14c066f30 msgr2=0x7ff14c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff14c066f30 0x7ff14c0693e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff158007fd0 tx=0x7ff15800c040 comp rx=0 tx=0).stop 2023-10-06T05:34:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 msgr2=0x7ff16813ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff16813ed80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7ff164011d70 tx=0x7ff1640091a0 comp rx=0 tx=0).stop 2023-10-06T05:34:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 shutdown_connections 2023-10-06T05:34:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff14c066f30 0x7ff14c0693e0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff16815a0a0 0x7ff16815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1540051c0 0x7ff16813ed80 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 --2- 172.21.15.33:0/23432171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff168159360 0x7ff1681558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 >> 172.21.15.33:0/23432171 conn(0x7ff16806d030 msgr2=0x7ff1680cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 shutdown_connections 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:56.996+0000 7ff16f39d700 1 -- 172.21.15.33:0/23432171 wait complete. 2023-10-06T05:34:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 210 2023-10-06T05:34:57.016 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2023-10-06T05:34:57.231 INFO:teuthology.orchestra.run.smithi167.stdout:[{"id":10819,"entity":{"name":{"type":"client","num":10819},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1471562529}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.151925651,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.985455723564352,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.10819 v1:192.168.0.1:0/1471562529","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi167","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}},{"id":12018,"entity":{"name":{"type":"client","num":12018},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1011352528}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.151925651,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.985441961729395,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1002,"last_trim_completed_flushes_tid":501,"delegated_inos":[],"inst":"client.12018 v1:192.168.0.1:0/1011352528","completed_requests":[{"tid":1002,"created_ino":"0x100000003e7"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000003e8","length":503}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi033","kernel_version":"6.5.0-rc7-gfef64f93da2b","root":"/"}}] 2023-10-06T05:34:57.233 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi167", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1471562529, "type": "v1" }, "name": { "num": 10819, "type": "client" } }, "id": 10819, "inst": "client.10819 v1:192.168.0.1:0/1471562529", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.985455723564352 }, "state": "open", "uptime": 4.151925651 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi033", "kernel_version": "6.5.0-rc7-gfef64f93da2b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e7", "tid": 1002 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1011352528, "type": "v1" }, "name": { "num": 12018, "type": "client" } }, "id": 12018, "inst": "client.12018 v1:192.168.0.1:0/1011352528", "last_trim_completed_flushes_tid": 501, "last_trim_completed_requests_tid": 1002, "num_caps": 1, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 503, "start": "0x100000003e8" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.985441961729395 }, "state": "open", "uptime": 4.151925651 } ] 2023-10-06T05:34:57.233 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:34:57.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:34:57.313 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:34:57.397 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:34:57.397 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:34:57.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:57.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:34:57.482 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:34:57.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:57.482 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:34:57.519 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:34:57.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:34:57.519 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:34:57.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:34:57.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:34:57.611 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:34:57.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:34:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.833+0000 7f7ec679d700 1 Processor -- start 2023-10-06T05:34:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.833+0000 7f7ec679d700 1 -- start start 2023-10-06T05:34:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.834+0000 7f7ec679d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7ec013faa0 0x7f7ec013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.834+0000 7f7ec679d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7ec006aad0 0x7f7ec006af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.834+0000 7f7ec679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00c9cb0 con 0x7f7ec00c7940 2023-10-06T05:34:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.834+0000 7f7ec679d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00ca270 con 0x7f7ec013faa0 2023-10-06T05:34:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.834+0000 7f7ec679d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec006b840 con 0x7f7ec006aad0 2023-10-06T05:34:57.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7ec013faa0 0x7f7ec013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7ec013faa0 0x7f7ec013fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50614/0 (socket says 172.21.15.33:50614) 2023-10-06T05:34:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 -- 172.21.15.33:0/4098760267 learned_addr learned my addr 172.21.15.33:0/4098760267 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec5f9c700 1 --2- 172.21.15.33:0/4098760267 >> v2:172.21.15.167:3300/0 conn(0x7f7ec006aad0 0x7f7ec006af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 -- 172.21.15.33:0/4098760267 >> v2:172.21.15.167:3300/0 conn(0x7f7ec006aad0 msgr2=0x7f7ec006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:57.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 --2- 172.21.15.33:0/4098760267 >> v2:172.21.15.167:3300/0 conn(0x7f7ec006aad0 0x7f7ec006af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 -- 172.21.15.33:0/4098760267 >> v1:172.21.15.167:6789/0 conn(0x7f7ec00c7940 legacy=0x7f7ec00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 -- 172.21.15.33:0/4098760267 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec006bb20 con 0x7f7ec013faa0 2023-10-06T05:34:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec5f9c700 1 --2- 172.21.15.33:0/4098760267 >> v2:172.21.15.167:3300/0 conn(0x7f7ec006aad0 0x7f7ec006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7ec579b700 1 --2- 172.21.15.33:0/4098760267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec013fe70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0156b90 tx=0x7f7ebc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2318054f6fa99d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.835+0000 7f7eb7fff700 1 -- 172.21.15.33:0/4098760267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00b820 con 0x7f7ec013faa0 2023-10-06T05:34:57.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7eb7fff700 1 -- 172.21.15.33:0/4098760267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc00b980 con 0x7f7ec013faa0 2023-10-06T05:34:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7eb7fff700 1 -- 172.21.15.33:0/4098760267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00bd20 con 0x7f7ec013faa0 2023-10-06T05:34:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 -- 172.21.15.33:0/4098760267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 msgr2=0x7f7ec013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 --2- 172.21.15.33:0/4098760267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec013fe70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0156b90 tx=0x7f7ebc009250 comp rx=0 tx=0).stop 2023-10-06T05:34:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 -- 172.21.15.33:0/4098760267 shutdown_connections 2023-10-06T05:34:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 --2- 172.21.15.33:0/4098760267 >> v2:172.21.15.167:3300/0 conn(0x7f7ec006aad0 0x7f7ec006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 --2- 172.21.15.33:0/4098760267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec013fe70 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 -- 172.21.15.33:0/4098760267 >> 172.21.15.33:0/4098760267 conn(0x7f7ec006d030 msgr2=0x7f7ec006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:57.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 -- 172.21.15.33:0/4098760267 shutdown_connections 2023-10-06T05:34:57.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.836+0000 7f7ec679d700 1 -- 172.21.15.33:0/4098760267 wait complete. 2023-10-06T05:34:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.838+0000 7f7ec679d700 1 Processor -- start 2023-10-06T05:34:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.838+0000 7f7ec679d700 1 -- start start 2023-10-06T05:34:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ec006aad0 0x7f7ec00c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec00c7940 0x7f7ec01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec0169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:57.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec0157f70 con 0x7f7ec013faa0 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00c5260 con 0x7f7ec00c7940 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01567e0 con 0x7f7ec006aad0 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec5f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec0169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec5f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec0169ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50622/0 (socket says 172.21.15.33:50622) 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec5f9c700 1 -- 172.21.15.33:0/2269998161 learned_addr learned my addr 172.21.15.33:0/2269998161 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec579b700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ec006aad0 0x7f7ec00c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.839+0000 7f7ec4f9a700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec00c7940 0x7f7ec01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec5f9c700 1 -- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ec006aad0 msgr2=0x7f7ec00c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec5f9c700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ec006aad0 0x7f7ec00c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec5f9c700 1 -- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec00c7940 msgr2=0x7f7ec01562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec5f9c700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec00c7940 0x7f7ec01562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec5f9c700 1 -- 172.21.15.33:0/2269998161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0094f0 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec4f9a700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec00c7940 0x7f7ec01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec5f9c700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec0169ad0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800a380 tx=0x7f7eb8007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8015440 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb80155a0 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8015960 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec0159b00 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.840+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec00c4c10 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.841+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec004c380 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.842+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7eb8015d50 con 0x7f7ec013faa0 2023-10-06T05:34:57.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.843+0000 7f7eb57fa700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.843+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f7eb80850b0 con 0x7f7ec013faa0 2023-10-06T05:34:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.844+0000 7f7ec579b700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.844+0000 7f7ec579b700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ea4066f30 0x7f7ea40693e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0156b90 tx=0x7f7ebc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:57.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:57.847+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7eb8052050 con 0x7f7ec013faa0 2023-10-06T05:34:58.046 INFO:tasks.ceph.mds.b.smithi167.stderr:2023-10-06T05:34:58.044+0000 7f9f329f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:34:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.096+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec00c9f00 con 0x7f7ec013faa0 2023-10-06T05:34:58.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.097+0000 7f7eb57fa700 1 -- 172.21.15.33:0/2269998161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f7eb8055670 con 0x7f7ec013faa0 2023-10-06T05:34:58.098 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:58.098 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:56.050401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:34:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ea4066f30 msgr2=0x7f7ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ea4066f30 0x7f7ea40693e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0156b90 tx=0x7f7ebc016000 comp rx=0 tx=0).stop 2023-10-06T05:34:58.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 msgr2=0x7f7ec0169ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec0169ad0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800a380 tx=0x7f7eb8007900 comp rx=0 tx=0).stop 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 shutdown_connections 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec013faa0 0x7f7ec0169ad0 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ec006aad0 0x7f7ec00c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 --2- 172.21.15.33:0/2269998161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec00c7940 0x7f7ec01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 >> 172.21.15.33:0/2269998161 conn(0x7f7ec006d030 msgr2=0x7f7ec015a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.099+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 shutdown_connections 2023-10-06T05:34:58.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.100+0000 7f7ec679d700 1 -- 172.21.15.33:0/2269998161 wait complete. 2023-10-06T05:34:58.103 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 210 2023-10-06T05:34:58.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:34:58.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.304+0000 7fb27919d700 1 Processor -- start 2023-10-06T05:34:58.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.304+0000 7fb27919d700 1 -- start start 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb27919d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2740cf3b0 0x7fb2741594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb27919d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740cbd80 con 0x7fb2740ccfc0 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb27919d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740af5f0 con 0x7fb2740af0a0 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb27919d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2740af8d0 con 0x7fb2740cf3b0 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb2737fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2740cf3b0 0x7fb2741594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb2737fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb2740cf3b0 0x7fb2741594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50632/0 (socket says 172.21.15.33:50632) 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.305+0000 7fb2737fe700 1 -- 172.21.15.33:0/1475847744 learned_addr learned my addr 172.21.15.33:0/1475847744 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:58.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb2737fe700 1 -- 172.21.15.33:0/1475847744 >> v1:172.21.15.167:6789/0 conn(0x7fb2740af0a0 legacy=0x7fb2740b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb2737fe700 1 -- 172.21.15.33:0/1475847744 >> v1:172.21.15.33:6789/0 conn(0x7fb2740ccfc0 legacy=0x7fb2740cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb2737fe700 1 -- 172.21.15.33:0/1475847744 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740afbb0 con 0x7fb2740cf3b0 2023-10-06T05:34:58.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb2737fe700 1 --2- 172.21.15.33:0/1475847744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2740cf3b0 0x7fb2741594a0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fb2680099e0 tx=0x7fb268009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ab55f1b8842789 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:58.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb272ffd700 1 -- 172.21.15.33:0/1475847744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800b820 con 0x7fb2740cf3b0 2023-10-06T05:34:58.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb272ffd700 1 -- 172.21.15.33:0/1475847744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26800b980 con 0x7fb2740cf3b0 2023-10-06T05:34:58.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.306+0000 7fb272ffd700 1 -- 172.21.15.33:0/1475847744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800bd20 con 0x7fb2740cf3b0 2023-10-06T05:34:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 -- 172.21.15.33:0/1475847744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2740cf3b0 msgr2=0x7fb2741594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 --2- 172.21.15.33:0/1475847744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2740cf3b0 0x7fb2741594a0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fb2680099e0 tx=0x7fb268009250 comp rx=0 tx=0).stop 2023-10-06T05:34:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 -- 172.21.15.33:0/1475847744 shutdown_connections 2023-10-06T05:34:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 --2- 172.21.15.33:0/1475847744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2740cf3b0 0x7fb2741594a0 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 -- 172.21.15.33:0/1475847744 >> 172.21.15.33:0/1475847744 conn(0x7fb27406cff0 msgr2=0x7fb27406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 -- 172.21.15.33:0/1475847744 shutdown_connections 2023-10-06T05:34:58.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.307+0000 7fb27919d700 1 -- 172.21.15.33:0/1475847744 wait complete. 2023-10-06T05:34:58.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.308+0000 7fb27919d700 1 Processor -- start 2023-10-06T05:34:58.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.309+0000 7fb27919d700 1 -- start start 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.309+0000 7fb27919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2740af0a0 0x7fb2740c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.309+0000 7fb27919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2740ccfc0 0x7fb274153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.309+0000 7fb27919d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 0x7fb2741571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.309+0000 7fb27919d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416ae60 con 0x7fb274156d20 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.309+0000 7fb27919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2741543f0 con 0x7fb2740af0a0 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2741576e0 con 0x7fb2740ccfc0 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 0x7fb2741571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 0x7fb2741571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50640/0 (socket says 172.21.15.33:50640) 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 -- 172.21.15.33:0/677498917 learned_addr learned my addr 172.21.15.33:0/677498917 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb2737fe700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2740ccfc0 0x7fb274153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb273fff700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2740af0a0 0x7fb2740c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 -- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2740ccfc0 msgr2=0x7fb274153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2740ccfc0 0x7fb274153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 -- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2740af0a0 msgr2=0x7fb2740c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2740af0a0 0x7fb2740c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.310+0000 7fb27899c700 1 -- 172.21.15.33:0/677498917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2680094f0 con 0x7fb274156d20 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.311+0000 7fb27899c700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 0x7fb2741571a0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c840 tx=0x7fb26c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.311+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c012630 con 0x7fb274156d20 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.311+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0127b0 con 0x7fb274156d20 2023-10-06T05:34:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.311+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2740c8d70 con 0x7fb274156d20 2023-10-06T05:34:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.311+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c012b70 con 0x7fb274156d20 2023-10-06T05:34:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.311+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2740c8720 con 0x7fb274156d20 2023-10-06T05:34:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.312+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2740c8580 con 0x7fb274156d20 2023-10-06T05:34:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.313+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb26c003030 con 0x7fb274156d20 2023-10-06T05:34:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.314+0000 7fb25bfff700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb260066f30 0x7fb2600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.314+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fb26c084650 con 0x7fb274156d20 2023-10-06T05:34:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.314+0000 7fb273fff700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb260066f30 0x7fb2600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.315+0000 7fb273fff700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb260066f30 0x7fb2600693e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb27415f2a0 tx=0x7fb264009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.318+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2740c8580 con 0x7fb274156d20 2023-10-06T05:34:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.521+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2740c8580 con 0x7fb274156d20 2023-10-06T05:34:58.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.522+0000 7fb25bfff700 1 -- 172.21.15.33:0/677498917 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+17193 (secure 0 0 0) 0x7fb2740c8580 con 0x7fb274156d20 2023-10-06T05:34:58.524 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:34:58.525 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":271,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:34:53.038933+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:33:56.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":40,"pool_name":"cephfs_data","create_time":"2023-10-06T05:33:56.676003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"40.1c","mappings":[{"from":5,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/3478690144":"2023-10-07T05:34:53.030921+0000","172.21.15.167:6834/3478690144":"2023-10-07T05:34:53.030921+0000","172.21.15.167:6837/3376723254":"2023-10-07T05:34:52.131020+0000","172.21.15.167:6835/3376723254":"2023-10-07T05:34:52.131020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb260066f30 msgr2=0x7fb2600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb260066f30 0x7fb2600693e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb27415f2a0 tx=0x7fb264009040 comp rx=0 tx=0).stop 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 msgr2=0x7fb2741571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 0x7fb2741571a0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c840 tx=0x7fb26c001650 comp rx=0 tx=0).stop 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 shutdown_connections 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb260066f30 0x7fb2600693e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb274156d20 0x7fb2741571a0 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2740af0a0 0x7fb2740c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 --2- 172.21.15.33:0/677498917 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2740ccfc0 0x7fb274153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 >> 172.21.15.33:0/677498917 conn(0x7fb27406cff0 msgr2=0x7fb2740b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 shutdown_connections 2023-10-06T05:34:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.525+0000 7fb27919d700 1 -- 172.21.15.33:0/677498917 wait complete. 2023-10-06T05:34:58.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 39, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:33:56.136996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '264', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 40, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:33:56.676003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '265', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:34:58.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:34:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.733+0000 7f14a7fff700 1 Processor -- start 2023-10-06T05:34:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.733+0000 7f14a7fff700 1 -- start start 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14a80cc7e0 0x7f14a80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f14a80b01f0 0x7f14a80b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a80ad430 con 0x7f14a8156370 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a80afe20 con 0x7f14a80cc7e0 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a80b0b70 con 0x7f14a80b01f0 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14a80cc7e0 0x7f14a80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a67fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f14a80cc7e0 0x7f14a80af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50656/0 (socket says 172.21.15.33:50656) 2023-10-06T05:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.734+0000 7f14a67fc700 1 -- 172.21.15.33:0/4275840518 learned_addr learned my addr 172.21.15.33:0/4275840518 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a77fe700 1 --2- 172.21.15.33:0/4275840518 >> v2:172.21.15.167:3300/0 conn(0x7f14a80b01f0 0x7f14a80b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a67fc700 1 -- 172.21.15.33:0/4275840518 >> v2:172.21.15.167:3300/0 conn(0x7f14a80b01f0 msgr2=0x7f14a80b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a67fc700 1 --2- 172.21.15.33:0/4275840518 >> v2:172.21.15.167:3300/0 conn(0x7f14a80b01f0 0x7f14a80b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a67fc700 1 -- 172.21.15.33:0/4275840518 >> v1:172.21.15.33:6789/0 conn(0x7f14a8156370 legacy=0x7f14a80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a67fc700 1 -- 172.21.15.33:0/4275840518 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a806bc70 con 0x7f14a80cc7e0 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a77fe700 1 --2- 172.21.15.33:0/4275840518 >> v2:172.21.15.167:3300/0 conn(0x7f14a80b01f0 0x7f14a80b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a67fc700 1 --2- 172.21.15.33:0/4275840518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a80cc7e0 0x7f14a80af8e0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f14a80b13b0 tx=0x7f149c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b3486471df61de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a5ffb700 1 -- 172.21.15.33:0/4275840518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00b860 con 0x7f14a80cc7e0 2023-10-06T05:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a5ffb700 1 -- 172.21.15.33:0/4275840518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00b9c0 con 0x7f14a80cc7e0 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.735+0000 7f14a5ffb700 1 -- 172.21.15.33:0/4275840518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00bd60 con 0x7f14a80cc7e0 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 -- 172.21.15.33:0/4275840518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a80cc7e0 msgr2=0x7f14a80af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 --2- 172.21.15.33:0/4275840518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a80cc7e0 0x7f14a80af8e0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f14a80b13b0 tx=0x7f149c009250 comp rx=0 tx=0).stop 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 -- 172.21.15.33:0/4275840518 shutdown_connections 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 --2- 172.21.15.33:0/4275840518 >> v2:172.21.15.167:3300/0 conn(0x7f14a80b01f0 0x7f14a80b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 --2- 172.21.15.33:0/4275840518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a80cc7e0 0x7f14a80af8e0 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 -- 172.21.15.33:0/4275840518 >> 172.21.15.33:0/4275840518 conn(0x7f14a806d030 msgr2=0x7f14a80cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 -- 172.21.15.33:0/4275840518 shutdown_connections 2023-10-06T05:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.736+0000 7f14a7fff700 1 -- 172.21.15.33:0/4275840518 wait complete. 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.738+0000 7f14a7fff700 1 Processor -- start 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.738+0000 7f14a7fff700 1 -- start start 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a80b01f0 0x7f14a8156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14a80cc7e0 0x7f14a8157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 0x7f14a8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80ab1c0 con 0x7f14a8156370 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80aa640 con 0x7f14a80b01f0 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80aa920 con 0x7f14a80cc7e0 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 0x7f14a8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 0x7f14a8157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50660/0 (socket says 172.21.15.33:50660) 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 -- 172.21.15.33:0/409533522 learned_addr learned my addr 172.21.15.33:0/409533522 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a6ffd700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a80b01f0 0x7f14a8156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a67fc700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14a80cc7e0 0x7f14a8157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 -- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14a80cc7e0 msgr2=0x7f14a8157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14a80cc7e0 0x7f14a8157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 -- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a80b01f0 msgr2=0x7f14a8156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a80b01f0 0x7f14a8156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.739+0000 7f14a77fe700 1 -- 172.21.15.33:0/409533522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0094f0 con 0x7f14a8156370 2023-10-06T05:34:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.740+0000 7f14a77fe700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 0x7f14a8157650 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f14a000cef0 tx=0x7f14a0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.740+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000ad20 con 0x7f14a8156370 2023-10-06T05:34:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.740+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0016420 con 0x7f14a8156370 2023-10-06T05:34:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.740+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000aea0 con 0x7f14a8156370 2023-10-06T05:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.740+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a80aac00 con 0x7f14a8156370 2023-10-06T05:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.740+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a80aa2a0 con 0x7f14a8156370 2023-10-06T05:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.741+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a804c380 con 0x7f14a8156370 2023-10-06T05:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.742+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14a0016a70 con 0x7f14a8156370 2023-10-06T05:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.743+0000 7f148effd700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.743+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f14a0012070 con 0x7f14a8156370 2023-10-06T05:34:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.743+0000 7f14a6ffd700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.744+0000 7f14a6ffd700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1494066f30 0x7f14940693e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1498003a80 tx=0x7f1498007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:58.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.747+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14a004e5a0 con 0x7f14a8156370 2023-10-06T05:34:58.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.948+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14a80b0950 con 0x7f14a8156370 2023-10-06T05:34:58.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.949+0000 7f148effd700 1 -- 172.21.15.33:0/409533522 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v271) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14a0051fc0 con 0x7f14a8156370 2023-10-06T05:34:58.950 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:34:58.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1494066f30 msgr2=0x7f14940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1494066f30 0x7f14940693e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1498003a80 tx=0x7f1498007ec0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 msgr2=0x7f14a8157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:58.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 0x7f14a8157650 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f14a000cef0 tx=0x7f14a0007860 comp rx=0 tx=0).stop 2023-10-06T05:34:58.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 shutdown_connections 2023-10-06T05:34:58.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14a8156370 0x7f14a8157650 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a80b01f0 0x7f14a8156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 --2- 172.21.15.33:0/409533522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14a80cc7e0 0x7f14a8157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:58.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 >> 172.21.15.33:0/409533522 conn(0x7f14a806d030 msgr2=0x7f14a80af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:58.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 shutdown_connections 2023-10-06T05:34:58.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:58.951+0000 7f14a7fff700 1 -- 172.21.15.33:0/409533522 wait complete. 2023-10-06T05:34:58.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:34:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.161+0000 7f789699d700 1 Processor -- start 2023-10-06T05:34:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.162+0000 7f789699d700 1 -- start start 2023-10-06T05:34:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789699d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7890156370 0x7f78900af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78900b01f0 0x7f78900b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789699d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900ad430 con 0x7f78900cc7e0 2023-10-06T05:34:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789699d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900afe20 con 0x7f7890156370 2023-10-06T05:34:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900b0b70 con 0x7f78900b01f0 2023-10-06T05:34:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789599b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7890156370 0x7f78900af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789619c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78900b01f0 0x7f78900b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789599b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7890156370 0x7f78900af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:51072/0 (socket says 172.21.15.33:51072) 2023-10-06T05:34:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.163+0000 7f789599b700 1 -- 172.21.15.33:0/3490450724 learned_addr learned my addr 172.21.15.33:0/3490450724 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.164+0000 7f789599b700 1 -- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78900b01f0 msgr2=0x7f78900b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.164+0000 7f789599b700 1 --2- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78900b01f0 0x7f78900b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.164+0000 7f789599b700 1 -- 172.21.15.33:0/3490450724 >> v1:172.21.15.167:6789/0 conn(0x7f78900cc7e0 legacy=0x7f78900c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.164+0000 7f789599b700 1 -- 172.21.15.33:0/3490450724 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789006bc70 con 0x7f7890156370 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.164+0000 7f789599b700 1 --2- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7890156370 0x7f78900af8e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f788c00bc50 tx=0x7f788c00b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13471ec5f74e8ca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.164+0000 7f7894999700 1 -- 172.21.15.33:0/3490450724 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c00dbf0 con 0x7f7890156370 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f7894999700 1 -- 172.21.15.33:0/3490450724 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c00dd50 con 0x7f7890156370 2023-10-06T05:34:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f7894999700 1 -- 172.21.15.33:0/3490450724 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c0196b0 con 0x7f7890156370 2023-10-06T05:34:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 -- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7890156370 msgr2=0x7f78900af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 --2- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7890156370 0x7f78900af8e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f788c00bc50 tx=0x7f788c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 -- 172.21.15.33:0/3490450724 shutdown_connections 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 --2- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7890156370 0x7f78900af8e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 --2- 172.21.15.33:0/3490450724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78900b01f0 0x7f78900b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 -- 172.21.15.33:0/3490450724 >> 172.21.15.33:0/3490450724 conn(0x7f789006d030 msgr2=0x7f78900cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 -- 172.21.15.33:0/3490450724 shutdown_connections 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.165+0000 7f789699d700 1 -- 172.21.15.33:0/3490450724 wait complete. 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.167+0000 7f789699d700 1 Processor -- start 2023-10-06T05:34:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.167+0000 7f789699d700 1 -- start start 2023-10-06T05:34:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789699d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 0x7f7890155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789699d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78900cc7e0 0x7f789006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789699d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7890156370 0x7f789006a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789699d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890155c60 con 0x7f78900b01f0 2023-10-06T05:34:59.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789699d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890151bc0 con 0x7f78900cc7e0 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789699d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7890151ea0 con 0x7f7890156370 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 0x7f7890155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789599b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 0x7f7890155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50664/0 (socket says 172.21.15.33:50664) 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789599b700 1 -- 172.21.15.33:0/3392653007 learned_addr learned my addr 172.21.15.33:0/3392653007 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789519a700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78900cc7e0 0x7f789006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.168+0000 7f789619c700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7890156370 0x7f789006a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789599b700 1 -- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7890156370 msgr2=0x7f789006a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789599b700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7890156370 0x7f789006a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789599b700 1 -- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78900cc7e0 msgr2=0x7f789006a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789599b700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78900cc7e0 0x7f789006a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789599b700 1 -- 172.21.15.33:0/3392653007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c00b790 con 0x7f78900b01f0 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789519a700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78900cc7e0 0x7f789006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789599b700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 0x7f7890155720 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f789006d4d0 tx=0x7f788c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c010540 con 0x7f78900b01f0 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c0106c0 con 0x7f78900b01f0 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c010a80 con 0x7f78900b01f0 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7890158f80 con 0x7f78900b01f0 2023-10-06T05:34:59.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.169+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7890159540 con 0x7f78900b01f0 2023-10-06T05:34:59.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.170+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789004c380 con 0x7f78900b01f0 2023-10-06T05:34:59.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.171+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f788c022030 con 0x7f78900b01f0 2023-10-06T05:34:59.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.172+0000 7f7885ffb700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7874066f30 0x7f78740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.173+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f788c014070 con 0x7f78900b01f0 2023-10-06T05:34:59.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.173+0000 7f789519a700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7874066f30 0x7f78740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.174+0000 7f789519a700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7874066f30 0x7f78740693e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7880007fd0 tx=0x7f7880009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.177+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f788c055940 con 0x7f78900b01f0 2023-10-06T05:34:59.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.378+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f789006f310 con 0x7f78900b01f0 2023-10-06T05:34:59.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.379+0000 7f7885ffb700 1 -- 172.21.15.33:0/3392653007 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v271) v1 ==== 92+0+10 (secure 0 0 0) 0x7f788c058f60 con 0x7f78900b01f0 2023-10-06T05:34:59.380 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7874066f30 msgr2=0x7f78740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7874066f30 0x7f78740693e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7880007fd0 tx=0x7f7880009040 comp rx=0 tx=0).stop 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 msgr2=0x7f7890155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 0x7f7890155720 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f789006d4d0 tx=0x7f788c01b000 comp rx=0 tx=0).stop 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 shutdown_connections 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7874066f30 0x7f78740693e0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7890156370 0x7f789006a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78900b01f0 0x7f7890155720 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 --2- 172.21.15.33:0/3392653007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78900cc7e0 0x7f789006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 >> 172.21.15.33:0/3392653007 conn(0x7f789006d030 msgr2=0x7f789015a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 shutdown_connections 2023-10-06T05:34:59.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.381+0000 7f789699d700 1 -- 172.21.15.33:0/3392653007 wait complete. 2023-10-06T05:34:59.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:34:59.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.590+0000 7fc7cffff700 1 Processor -- start 2023-10-06T05:34:59.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.590+0000 7fc7cffff700 1 -- start start 2023-10-06T05:34:59.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.591+0000 7fc7cffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7d00c9490 0x7fc7d0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.591+0000 7fc7cffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00b06b0 con 0x7fc7d00afb20 2023-10-06T05:34:59.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.591+0000 7fc7cffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00c99e0 con 0x7fc7d0155a50 2023-10-06T05:34:59.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.591+0000 7fc7cffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d00c9cc0 con 0x7fc7d00c9490 2023-10-06T05:34:59.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7ce7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7d00c9490 0x7fc7d0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7ce7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7d00c9490 0x7fc7d0155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50674/0 (socket says 172.21.15.33:50674) 2023-10-06T05:34:59.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/4272886852 learned_addr learned my addr 172.21.15.33:0/4272886852 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7cdffb700 1 -- 172.21.15.33:0/4272886852 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260657075 0 0) 0x7fc7d00c99e0 con 0x7fc7d0155a50 2023-10-06T05:34:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/4272886852 >> v1:172.21.15.167:6789/0 conn(0x7fc7d0155a50 legacy=0x7fc7d00af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/4272886852 >> v1:172.21.15.33:6789/0 conn(0x7fc7d00afb20 legacy=0x7fc7d00caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.592+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/4272886852 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00c9fa0 con 0x7fc7d00c9490 2023-10-06T05:34:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7ce7fc700 1 --2- 172.21.15.33:0/4272886852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0155e20 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400bc50 tx=0x7fc7c400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8dad206a9016e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:59.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cdffb700 1 -- 172.21.15.33:0/4272886852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400be60 con 0x7fc7d00c9490 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cdffb700 1 -- 172.21.15.33:0/4272886852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c400de00 con 0x7fc7d00c9490 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cdffb700 1 -- 172.21.15.33:0/4272886852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40196b0 con 0x7fc7d00c9490 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cffff700 1 -- 172.21.15.33:0/4272886852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 msgr2=0x7fc7d0155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cffff700 1 --2- 172.21.15.33:0/4272886852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0155e20 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400bc50 tx=0x7fc7c400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cffff700 1 -- 172.21.15.33:0/4272886852 shutdown_connections 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cffff700 1 --2- 172.21.15.33:0/4272886852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0155e20 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.593+0000 7fc7cffff700 1 -- 172.21.15.33:0/4272886852 >> 172.21.15.33:0/4272886852 conn(0x7fc7d006d030 msgr2=0x7fc7d00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.594+0000 7fc7cffff700 1 -- 172.21.15.33:0/4272886852 shutdown_connections 2023-10-06T05:34:59.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.594+0000 7fc7cffff700 1 -- 172.21.15.33:0/4272886852 wait complete. 2023-10-06T05:34:59.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.595+0000 7fc7cffff700 1 Processor -- start 2023-10-06T05:34:59.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.595+0000 7fc7cffff700 1 -- start start 2023-10-06T05:34:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7cffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d00afb20 0x7fc7d0169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7cffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d0155a50 0x7fc7d00cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7cffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d013ba90 con 0x7fc7d00c9490 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7cffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0169b70 con 0x7fc7d00afb20 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7cffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0158240 con 0x7fc7d0155a50 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7ce7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7ce7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50676/0 (socket says 172.21.15.33:50676) 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.596+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/518757515 learned_addr learned my addr 172.21.15.33:0/518757515 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7cf7fe700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d0155a50 0x7fc7d00cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ceffd700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d00afb20 0x7fc7d0169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d0155a50 msgr2=0x7fc7d00cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ce7fc700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d0155a50 0x7fc7d00cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d00afb20 msgr2=0x7fc7d0169630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ce7fc700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d00afb20 0x7fc7d0169630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ce7fc700 1 -- 172.21.15.33:0/518757515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c400b790 con 0x7fc7d00c9490 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7cf7fe700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d0155a50 0x7fc7d00cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ceffd700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d00afb20 0x7fc7d0169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7ce7fc700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0164e80 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fc7d006d4d0 tx=0x7fc7c401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4010750 con 0x7fc7d00c9490 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c40108d0 con 0x7fc7d00c9490 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4010c90 con 0x7fc7d00c9490 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.597+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d0157630 con 0x7fc7d00c9490 2023-10-06T05:34:59.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.598+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d0157bf0 con 0x7fc7d00c9490 2023-10-06T05:34:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.598+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d004c380 con 0x7fc7d00c9490 2023-10-06T05:34:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.599+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7c402c440 con 0x7fc7d00c9490 2023-10-06T05:34:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.600+0000 7fc7b6ffd700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7bc066f30 0x7fc7bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:34:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.601+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc7c4014070 con 0x7fc7d00c9490 2023-10-06T05:34:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.601+0000 7fc7ceffd700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7bc066f30 0x7fc7bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:34:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.602+0000 7fc7ceffd700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7bc066f30 0x7fc7bc0693e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0007fd0 tx=0x7fc7c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:34:59.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.605+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7c4053450 con 0x7fc7d00c9490 2023-10-06T05:34:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.805+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7d0158380 con 0x7fc7d00c9490 2023-10-06T05:34:59.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.805+0000 7fc7b6ffd700 1 -- 172.21.15.33:0/518757515 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v271) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7c4056a70 con 0x7fc7d00c9490 2023-10-06T05:34:59.807 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:34:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7bc066f30 msgr2=0x7fc7bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7bc066f30 0x7fc7bc0693e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0007fd0 tx=0x7fc7c0009040 comp rx=0 tx=0).stop 2023-10-06T05:34:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 msgr2=0x7fc7d0164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:34:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0164e80 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fc7d006d4d0 tx=0x7fc7c401b000 comp rx=0 tx=0).stop 2023-10-06T05:34:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 shutdown_connections 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc7bc066f30 0x7fc7bc0693e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7d0155a50 0x7fc7d00cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7d00afb20 0x7fc7d0169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 --2- 172.21.15.33:0/518757515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d00c9490 0x7fc7d0164e80 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 >> 172.21.15.33:0/518757515 conn(0x7fc7d006d030 msgr2=0x7fc7d00aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 shutdown_connections 2023-10-06T05:34:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:34:59.808+0000 7fc7cffff700 1 -- 172.21.15.33:0/518757515 wait complete. 2023-10-06T05:34:59.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:35:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.017+0000 7fb10179d700 1 Processor -- start 2023-10-06T05:35:00.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.017+0000 7fb10179d700 1 -- start start 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb10179d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0fc0abbd0 0x7fb0fc067c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb10179d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc06ae10 con 0x7fb0fc068140 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb10179d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc068580 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb10179d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0fc0ac050 con 0x7fb0fc0abbd0 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb0fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0fc0abbd0 0x7fb0fc067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb0fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0fc0abbd0 0x7fb0fc067c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49178/0 (socket says 172.21.15.33:49178) 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/2077539835 learned_addr learned my addr 172.21.15.33:0/2077539835 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:00.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.019+0000 7fb100f9c700 1 --1- 172.21.15.33:0/2077539835 >> v1:172.21.15.33:6789/0 conn(0x7fb0fc068140 0x7fb0fc159dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59662/0 (socket says 172.21.15.33:59662) 2023-10-06T05:35:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0faffd700 1 -- 172.21.15.33:0/2077539835 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597889140 0 0) 0x7fb0fc068580 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0faffd700 1 -- 172.21.15.33:0/2077539835 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e4003570 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/2077539835 >> v1:172.21.15.167:6789/0 conn(0x7fb0fc0b1f90 legacy=0x7fb0fc070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/2077539835 >> v1:172.21.15.33:6789/0 conn(0x7fb0fc068140 legacy=0x7fb0fc159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0faffd700 1 -- 172.21.15.33:0/2077539835 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722142682 0 0) 0x7fb0fc06ae10 con 0x7fb0fc068140 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0fb7fe700 1 -- 172.21.15.33:0/2077539835 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc06a4e0 con 0x7fb0fc0abbd0 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0fb7fe700 1 --2- 172.21.15.33:0/2077539835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc067c00 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00099b0 tx=0x7fb0f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc8824f96a7bd0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0faffd700 1 -- 172.21.15.33:0/2077539835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0009ea0 con 0x7fb0fc0abbd0 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0faffd700 1 -- 172.21.15.33:0/2077539835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000b860 con 0x7fb0fc0abbd0 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.020+0000 7fb0faffd700 1 -- 172.21.15.33:0/2077539835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000bbe0 con 0x7fb0fc0abbd0 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 -- 172.21.15.33:0/2077539835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0abbd0 msgr2=0x7fb0fc067c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 --2- 172.21.15.33:0/2077539835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc067c00 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00099b0 tx=0x7fb0f0009250 comp rx=0 tx=0).stop 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 -- 172.21.15.33:0/2077539835 shutdown_connections 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 --2- 172.21.15.33:0/2077539835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc067c00 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 -- 172.21.15.33:0/2077539835 >> 172.21.15.33:0/2077539835 conn(0x7fb0fc06ea70 msgr2=0x7fb0fc06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:00.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 -- 172.21.15.33:0/2077539835 shutdown_connections 2023-10-06T05:35:00.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.021+0000 7fb10179d700 1 -- 172.21.15.33:0/2077539835 wait complete. 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.022+0000 7fb10179d700 1 Processor -- start 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 -- start start 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0fc068140 0x7fb0fc15e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 0x7fb0fc13d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc15aab0 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc159c00 con 0x7fb0fc0abbd0 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.023+0000 7fb10179d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc159ea0 con 0x7fb0fc068140 2023-10-06T05:35:00.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 0x7fb0fc13d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 0x7fb0fc13d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49180/0 (socket says 172.21.15.33:49180) 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 -- 172.21.15.33:0/4004110470 learned_addr learned my addr 172.21.15.33:0/4004110470 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb0fb7fe700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb0fbfff700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0fc068140 0x7fb0fc15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 -- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0fc068140 msgr2=0x7fb0fc15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0fc068140 0x7fb0fc15e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 -- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc0abbd0 msgr2=0x7fb0fc1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc1596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.024+0000 7fb100f9c700 1 -- 172.21.15.33:0/4004110470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00094f0 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.025+0000 7fb100f9c700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 0x7fb0fc13d1b0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400ccc0 tx=0x7fb0f4001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.025+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4005df0 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.025+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f400a600 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.025+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400a9c0 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.025+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc15a180 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.025+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc15a740 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.026+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc15ac20 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.027+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0f40070f0 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.028+0000 7fb0e3fff700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0e8066f30 0x7fb0e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.028+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fb0f4084860 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.028+0000 7fb0fbfff700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0e8066f30 0x7fb0e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.029+0000 7fb0fbfff700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0e8066f30 0x7fb0e80693e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec007fd0 tx=0x7fb0ec007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.032+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f4051930 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.234+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0fc04c380 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.235+0000 7fb0e3fff700 1 -- 172.21.15.33:0/4004110470 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v271) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0f4054f50 con 0x7fb0fc0b1f90 2023-10-06T05:35:00.237 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:35:00.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0e8066f30 msgr2=0x7fb0e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0e8066f30 0x7fb0e80693e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec007fd0 tx=0x7fb0ec007d50 comp rx=0 tx=0).stop 2023-10-06T05:35:00.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 msgr2=0x7fb0fc13d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 0x7fb0fc13d1b0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400ccc0 tx=0x7fb0f4001a70 comp rx=0 tx=0).stop 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 shutdown_connections 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb0e8066f30 0x7fb0e80693e0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc0b1f90 0x7fb0fc13d1b0 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb0fc068140 0x7fb0fc15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 --2- 172.21.15.33:0/4004110470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc0abbd0 0x7fb0fc1596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 >> 172.21.15.33:0/4004110470 conn(0x7fb0fc06ea70 msgr2=0x7fb0fc153530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 shutdown_connections 2023-10-06T05:35:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.237+0000 7fb10179d700 1 -- 172.21.15.33:0/4004110470 wait complete. 2023-10-06T05:35:00.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:00.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.442+0000 7f4d3919d700 1 Processor -- start 2023-10-06T05:35:00.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.442+0000 7f4d3919d700 1 -- start start 2023-10-06T05:35:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d34153b10 0x7f4d340c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3919d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d340c2610 0x7f4d340c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3919d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3415abd0 con 0x7f4d340c6b30 2023-10-06T05:35:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3919d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d340c7510 con 0x7f4d340c2610 2023-10-06T05:35:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c9b00 con 0x7f4d34153b10 2023-10-06T05:35:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3899c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4d340c2610 0x7f4d340c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d337fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4d340c6b30 0x7f4d340675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59674/0 (socket says 172.21.15.33:59674) 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d337fe700 1 -- 172.21.15.33:0/1393052648 learned_addr learned my addr 172.21.15.33:0/1393052648 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d33fff700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d34153b10 0x7f4d340c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3899c700 1 -- 172.21.15.33:0/1393052648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d34153b10 msgr2=0x7f4d340c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3899c700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d34153b10 0x7f4d340c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3899c700 1 -- 172.21.15.33:0/1393052648 >> v1:172.21.15.33:6789/0 conn(0x7f4d340c6b30 legacy=0x7f4d340675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d3899c700 1 -- 172.21.15.33:0/1393052648 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3415a050 con 0x7f4d340c2610 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.444+0000 7f4d33fff700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d34153b10 0x7f4d340c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3899c700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d340c6fd0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009e60 tx=0x7f4d2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e004a78e2319d0f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d32ffd700 1 -- 172.21.15.33:0/1393052648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00e070 con 0x7f4d340c2610 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d32ffd700 1 -- 172.21.15.33:0/1393052648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c00bc30 con 0x7f4d340c2610 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d32ffd700 1 -- 172.21.15.33:0/1393052648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c012430 con 0x7f4d340c2610 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3919d700 1 -- 172.21.15.33:0/1393052648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 msgr2=0x7f4d340c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3919d700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d340c6fd0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009e60 tx=0x7f4d2c009780 comp rx=0 tx=0).stop 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3919d700 1 -- 172.21.15.33:0/1393052648 shutdown_connections 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3919d700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d340c6fd0 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3919d700 1 --2- 172.21.15.33:0/1393052648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d34153b10 0x7f4d340c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.445+0000 7f4d3919d700 1 -- 172.21.15.33:0/1393052648 >> 172.21.15.33:0/1393052648 conn(0x7f4d3406c560 msgr2=0x7f4d3406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.446+0000 7f4d3919d700 1 -- 172.21.15.33:0/1393052648 shutdown_connections 2023-10-06T05:35:00.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.446+0000 7f4d3919d700 1 -- 172.21.15.33:0/1393052648 wait complete. 2023-10-06T05:35:00.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.447+0000 7f4d3919d700 1 Processor -- start 2023-10-06T05:35:00.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.447+0000 7f4d3919d700 1 -- start start 2023-10-06T05:35:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d3919d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d3413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d3919d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340c6b30 0x7f4d340b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d3919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34153b10 0x7f4d340c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d3919d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c3fb0 con 0x7f4d340c2610 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d3919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c4e10 con 0x7f4d34153b10 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d3919d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340c61e0 con 0x7f4d340c6b30 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d3413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d3413bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49186/0 (socket says 172.21.15.33:49186) 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d33fff700 1 -- 172.21.15.33:0/844073343 learned_addr learned my addr 172.21.15.33:0/844073343 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.448+0000 7f4d337fe700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340c6b30 0x7f4d340b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d3899c700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34153b10 0x7f4d340c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d33fff700 1 -- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340c6b30 msgr2=0x7f4d340b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d33fff700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340c6b30 0x7f4d340b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d33fff700 1 -- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34153b10 msgr2=0x7f4d340c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d33fff700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34153b10 0x7f4d340c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d33fff700 1 -- 172.21.15.33:0/844073343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c009a20 con 0x7f4d340c2610 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d33fff700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d3413bea0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c890 tx=0x7f4d24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400a620 con 0x7f4d340c2610 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2400a780 con 0x7f4d340c2610 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400ab40 con 0x7f4d340c2610 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d340c4200 con 0x7f4d340c2610 2023-10-06T05:35:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.449+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d340c47c0 con 0x7f4d340c2610 2023-10-06T05:35:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.451+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3404c380 con 0x7f4d340c2610 2023-10-06T05:35:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.451+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d240113f0 con 0x7f4d340c2610 2023-10-06T05:35:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.453+0000 7f4d1bfff700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.453+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f4d240847f0 con 0x7f4d340c2610 2023-10-06T05:35:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.453+0000 7f4d337fe700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.454+0000 7f4d337fe700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4d34159f00 tx=0x7f4d28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.457+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d24051830 con 0x7f4d340c2610 2023-10-06T05:35:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.705+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d340c9d00 con 0x7f4d340c2610 2023-10-06T05:35:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.706+0000 7f4d1bfff700 1 -- 172.21.15.33:0/844073343 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f4d24054e50 con 0x7f4d340c2610 2023-10-06T05:35:00.707 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:00.707 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:56.050401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 msgr2=0x7f4d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4d34159f00 tx=0x7f4d28009040 comp rx=0 tx=0).stop 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 msgr2=0x7f4d3413bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d3413bea0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c890 tx=0x7f4d24008040 comp rx=0 tx=0).stop 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 shutdown_connections 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4d20066f30 0x7f4d200693e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d34153b10 0x7f4d340c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d340c2610 0x7f4d3413bea0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 --2- 172.21.15.33:0/844073343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d340c6b30 0x7f4d340b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 >> 172.21.15.33:0/844073343 conn(0x7f4d3406c560 msgr2=0x7f4d340675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 shutdown_connections 2023-10-06T05:35:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.708+0000 7f4d3919d700 1 -- 172.21.15.33:0/844073343 wait complete. 2023-10-06T05:35:00.711 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 210 2023-10-06T05:35:00.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.914+0000 7fd06359d700 1 Processor -- start 2023-10-06T05:35:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.914+0000 7fd06359d700 1 -- start start 2023-10-06T05:35:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd06359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c159530 0x7fd05c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd06359d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd05c06df70 0x7fd05c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd06359d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c06e3e0 con 0x7fd05c0b0310 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd06359d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05c0b1e00 con 0x7fd05c06df70 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd06359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c06f6b0 con 0x7fd05c159530 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd062d9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd05c0b0310 0x7fd05c159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59680/0 (socket says 172.21.15.33:59680) 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd062d9c700 1 -- 172.21.15.33:0/3792447412 learned_addr learned my addr 172.21.15.33:0/3792447412 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd06259b700 1 --2- 172.21.15.33:0/3792447412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c159530 0x7fd05c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.915+0000 7fd061d9a700 1 --2- 172.21.15.33:0/3792447412 >> v2:172.21.15.167:3300/0 conn(0x7fd05c06df70 0x7fd05c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd061599700 1 -- 172.21.15.33:0/3792447412 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124709835 0 0) 0x7fd05c06e3e0 con 0x7fd05c0b0310 2023-10-06T05:35:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd061599700 1 -- 172.21.15.33:0/3792447412 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd048003570 con 0x7fd05c0b0310 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd06259b700 1 -- 172.21.15.33:0/3792447412 >> v2:172.21.15.167:3300/0 conn(0x7fd05c06df70 msgr2=0x7fd05c0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd06259b700 1 --2- 172.21.15.33:0/3792447412 >> v2:172.21.15.167:3300/0 conn(0x7fd05c06df70 0x7fd05c0afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd06259b700 1 -- 172.21.15.33:0/3792447412 >> v1:172.21.15.33:6789/0 conn(0x7fd05c0b0310 legacy=0x7fd05c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd06259b700 1 -- 172.21.15.33:0/3792447412 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c06f220 con 0x7fd05c159530 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd061d9a700 1 --2- 172.21.15.33:0/3792447412 >> v2:172.21.15.167:3300/0 conn(0x7fd05c06df70 0x7fd05c0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.916+0000 7fd06259b700 1 --2- 172.21.15.33:0/3792447412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c159530 0x7fd05c06da30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0580098e0 tx=0x7fd0580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6439caaa8ba471d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd061599700 1 -- 172.21.15.33:0/3792447412 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011630 con 0x7fd05c159530 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd061599700 1 -- 172.21.15.33:0/3792447412 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011790 con 0x7fd05c159530 2023-10-06T05:35:00.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd061599700 1 -- 172.21.15.33:0/3792447412 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011b10 con 0x7fd05c159530 2023-10-06T05:35:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 -- 172.21.15.33:0/3792447412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c159530 msgr2=0x7fd05c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 --2- 172.21.15.33:0/3792447412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c159530 0x7fd05c06da30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0580098e0 tx=0x7fd0580091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 -- 172.21.15.33:0/3792447412 shutdown_connections 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 --2- 172.21.15.33:0/3792447412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c159530 0x7fd05c06da30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 --2- 172.21.15.33:0/3792447412 >> v2:172.21.15.167:3300/0 conn(0x7fd05c06df70 0x7fd05c0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 -- 172.21.15.33:0/3792447412 >> 172.21.15.33:0/3792447412 conn(0x7fd05c06c560 msgr2=0x7fd05c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 -- 172.21.15.33:0/3792447412 shutdown_connections 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.917+0000 7fd06359d700 1 -- 172.21.15.33:0/3792447412 wait complete. 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.919+0000 7fd06359d700 1 Processor -- start 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.919+0000 7fd06359d700 1 -- start start 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd06359d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c06df70 0x7fd05c153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd06359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c0b0310 0x7fd05c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd06359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 0x7fd05c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd06359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0c9750 con 0x7fd05c159530 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd06359d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0c8b40 con 0x7fd05c06df70 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd06359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0c8e20 con 0x7fd05c0b0310 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd062d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 0x7fd05c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd062d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 0x7fd05c15dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49188/0 (socket says 172.21.15.33:49188) 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.920+0000 7fd062d9c700 1 -- 172.21.15.33:0/3403151828 learned_addr learned my addr 172.21.15.33:0/3403151828 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd061d9a700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c0b0310 0x7fd05c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd06259b700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c06df70 0x7fd05c153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd062d9c700 1 -- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c0b0310 msgr2=0x7fd05c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd062d9c700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c0b0310 0x7fd05c0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd062d9c700 1 -- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c06df70 msgr2=0x7fd05c153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd062d9c700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c06df70 0x7fd05c153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd062d9c700 1 -- 172.21.15.33:0/3403151828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009420 con 0x7fd05c159530 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd062d9c700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 0x7fd05c15dc50 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fd054001460 tx=0x7fd054001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05400ce80 con 0x7fd05c159530 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05400ab70 con 0x7fd05c159530 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd058009420 con 0x7fd05c159530 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0540070f0 con 0x7fd05c159530 2023-10-06T05:35:00.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.921+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c0c93e0 con 0x7fd05c159530 2023-10-06T05:35:00.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.922+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c0c9c30 con 0x7fd05c159530 2023-10-06T05:35:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.923+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd054007520 con 0x7fd05c159530 2023-10-06T05:35:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.924+0000 7fd0527fc700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.924+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fd054084d00 con 0x7fd05c159530 2023-10-06T05:35:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.925+0000 7fd06259b700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:00.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.925+0000 7fd06259b700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd040066f30 0x7fd0400693e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd0580091e0 tx=0x7fd058017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:00.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:00.928+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd054051c90 con 0x7fd05c159530 2023-10-06T05:35:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.135+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd05c04c380 con 0x7fd05c159530 2023-10-06T05:35:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.136+0000 7fd0527fc700 1 -- 172.21.15.33:0/3403151828 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+17193 (secure 0 0 0) 0x7fd0540552b0 con 0x7fd05c159530 2023-10-06T05:35:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:01.139 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":271,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:34:53.038933+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:33:56.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":40,"pool_name":"cephfs_data","create_time":"2023-10-06T05:33:56.676003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"40.1c","mappings":[{"from":5,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/3478690144":"2023-10-07T05:34:53.030921+0000","172.21.15.167:6834/3478690144":"2023-10-07T05:34:53.030921+0000","172.21.15.167:6837/3376723254":"2023-10-07T05:34:52.131020+0000","172.21.15.167:6835/3376723254":"2023-10-07T05:34:52.131020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd040066f30 msgr2=0x7fd0400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd040066f30 0x7fd0400693e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd0580091e0 tx=0x7fd058017000 comp rx=0 tx=0).stop 2023-10-06T05:35:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 msgr2=0x7fd05c15dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 0x7fd05c15dc50 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fd054001460 tx=0x7fd054001ae0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 shutdown_connections 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd05c159530 0x7fd05c15dc50 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd05c06df70 0x7fd05c153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 --2- 172.21.15.33:0/3403151828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05c0b0310 0x7fd05c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 >> 172.21.15.33:0/3403151828 conn(0x7fd05c06c560 msgr2=0x7fd05c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 shutdown_connections 2023-10-06T05:35:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.139+0000 7fd06359d700 1 -- 172.21.15.33:0/3403151828 wait complete. 2023-10-06T05:35:01.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:35:01.159 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.346+0000 7f42effff700 1 Processor -- start 2023-10-06T05:35:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.346+0000 7f42effff700 1 -- start start 2023-10-06T05:35:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.347+0000 7f42effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f015b5f0 0x7f42f0161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.347+0000 7f42effff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42f0161a50 0x7f42f006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.347+0000 7f42effff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f0161eb0 con 0x7f42f015b220 2023-10-06T05:35:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.347+0000 7f42effff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f0069da0 con 0x7f42f0161a50 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f006a360 con 0x7f42f015b5f0 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42f0161a50 0x7f42f006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42f0161a50 0x7f42f006a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49204/0 (socket says 172.21.15.33:49204) 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 -- 172.21.15.33:0/41735649 learned_addr learned my addr 172.21.15.33:0/41735649 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ee7fc700 1 --2- 172.21.15.33:0/41735649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f015b5f0 0x7f42f0161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 -- 172.21.15.33:0/41735649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f015b5f0 msgr2=0x7f42f0161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 --2- 172.21.15.33:0/41735649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f015b5f0 0x7f42f0161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 -- 172.21.15.33:0/41735649 >> v1:172.21.15.167:6789/0 conn(0x7f42f015b220 legacy=0x7f42f00adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42ef7fe700 1 -- 172.21.15.33:0/41735649 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00ae760 con 0x7f42f0161a50 2023-10-06T05:35:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.348+0000 7f42edffb700 1 -- 172.21.15.33:0/41735649 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708440948 0 0) 0x7f42e0001110 con 0x7f42f015b220 2023-10-06T05:35:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42ef7fe700 1 --2- 172.21.15.33:0/41735649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0161a50 0x7f42f006a7f0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009f20 tx=0x7f42e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b642dc33e6050b2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42edffb700 1 -- 172.21.15.33:0/41735649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e800e070 con 0x7f42f0161a50 2023-10-06T05:35:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42edffb700 1 -- 172.21.15.33:0/41735649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e800b9d0 con 0x7f42f0161a50 2023-10-06T05:35:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42edffb700 1 -- 172.21.15.33:0/41735649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e80092e0 con 0x7f42f0161a50 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42effff700 1 -- 172.21.15.33:0/41735649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0161a50 msgr2=0x7f42f006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42effff700 1 --2- 172.21.15.33:0/41735649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0161a50 0x7f42f006a7f0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009f20 tx=0x7f42e8009840 comp rx=0 tx=0).stop 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42effff700 1 -- 172.21.15.33:0/41735649 shutdown_connections 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42effff700 1 --2- 172.21.15.33:0/41735649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0161a50 0x7f42f006a7f0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42effff700 1 --2- 172.21.15.33:0/41735649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f015b5f0 0x7f42f0161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.349+0000 7f42effff700 1 -- 172.21.15.33:0/41735649 >> 172.21.15.33:0/41735649 conn(0x7f42f006c560 msgr2=0x7f42f006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:01.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.350+0000 7f42effff700 1 -- 172.21.15.33:0/41735649 shutdown_connections 2023-10-06T05:35:01.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.350+0000 7f42effff700 1 -- 172.21.15.33:0/41735649 wait complete. 2023-10-06T05:35:01.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.351+0000 7f42effff700 1 Processor -- start 2023-10-06T05:35:01.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.351+0000 7f42effff700 1 -- start start 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 0x7f42f00aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42effff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f015b5f0 0x7f42f00a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42effff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f0161a50 0x7f42f014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42effff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00a8be0 con 0x7f42f015b220 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42effff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00abe30 con 0x7f42f015b5f0 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42effff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00ab220 con 0x7f42f0161a50 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 0x7f42f00aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 0x7f42f00aab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49210/0 (socket says 172.21.15.33:49210) 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42eeffd700 1 -- 172.21.15.33:0/4147197464 learned_addr learned my addr 172.21.15.33:0/4147197464 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42ef7fe700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f0161a50 0x7f42f014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.352+0000 7f42ee7fc700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f015b5f0 0x7f42f00a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42eeffd700 1 -- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f0161a50 msgr2=0x7f42f014b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42eeffd700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f0161a50 0x7f42f014b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42eeffd700 1 -- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f015b5f0 msgr2=0x7f42f00a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42eeffd700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f015b5f0 0x7f42f00a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42eeffd700 1 -- 172.21.15.33:0/4147197464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8009ae0 con 0x7f42f015b220 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42eeffd700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 0x7f42f00aab10 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f42e000c7e0 tx=0x7f42e0007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0011590 con 0x7f42f015b220 2023-10-06T05:35:01.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0011710 con 0x7f42f015b220 2023-10-06T05:35:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0011ad0 con 0x7f42f015b220 2023-10-06T05:35:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.353+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f00ab500 con 0x7f42f015b220 2023-10-06T05:35:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.354+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f00abac0 con 0x7f42f015b220 2023-10-06T05:35:01.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.355+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f004c380 con 0x7f42f015b220 2023-10-06T05:35:01.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.355+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42e0012420 con 0x7f42f015b220 2023-10-06T05:35:01.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.356+0000 7f42d6ffd700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42dc066f30 0x7f42dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.357+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f42e0083950 con 0x7f42f015b220 2023-10-06T05:35:01.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.357+0000 7f42ee7fc700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42dc066f30 0x7f42dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.358+0000 7f42ee7fc700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42dc066f30 0x7f42dc0693e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f42f0161740 tx=0x7f42e40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.360+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42e004b5a0 con 0x7f42f015b220 2023-10-06T05:35:01.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.607+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42f0161eb0 con 0x7f42f015b220 2023-10-06T05:35:01.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.608+0000 7f42d6ffd700 1 -- 172.21.15.33:0/4147197464 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4617 (secure 0 0 0) 0x7f42f0161eb0 con 0x7f42f015b220 2023-10-06T05:35:01.610 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:01.610 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:56.050401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:35:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42dc066f30 msgr2=0x7f42dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42dc066f30 0x7f42dc0693e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f42f0161740 tx=0x7f42e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 msgr2=0x7f42f00aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 0x7f42f00aab10 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f42e000c7e0 tx=0x7f42e0007c60 comp rx=0 tx=0).stop 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 shutdown_connections 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42dc066f30 0x7f42dc0693e0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f0161a50 0x7f42f014b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f015b220 0x7f42f00aab10 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 --2- 172.21.15.33:0/4147197464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f015b5f0 0x7f42f00a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 >> 172.21.15.33:0/4147197464 conn(0x7f42f006c560 msgr2=0x7f42f015d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 shutdown_connections 2023-10-06T05:35:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.610+0000 7f42effff700 1 -- 172.21.15.33:0/4147197464 wait complete. 2023-10-06T05:35:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 210 2023-10-06T05:35:01.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:01.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.814+0000 7fe16759e700 1 Processor -- start 2023-10-06T05:35:01.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.815+0000 7fe16759e700 1 -- start start 2023-10-06T05:35:01.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe168155140 0x7fe168158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1680adf50 0x7fe1680c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1680c9810 0x7fe1680699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1680c8e40 con 0x7fe168155140 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1680ae3f0 con 0x7fe1680adf50 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680ae6c0 con 0x7fe1680c9810 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe168155140 0x7fe168158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe168155140 0x7fe168158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49224/0 (socket says 172.21.15.33:49224) 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe16659c700 1 -- 172.21.15.33:0/1438649994 learned_addr learned my addr 172.21.15.33:0/1438649994 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe166d9d700 1 --2- 172.21.15.33:0/1438649994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1680c9810 0x7fe1680699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.816+0000 7fe165d9b700 1 --2- 172.21.15.33:0/1438649994 >> v2:172.21.15.167:3300/0 conn(0x7fe1680adf50 0x7fe1680c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16659c700 1 -- 172.21.15.33:0/1438649994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1680c9810 msgr2=0x7fe1680699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16659c700 1 --2- 172.21.15.33:0/1438649994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1680c9810 0x7fe1680699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16659c700 1 -- 172.21.15.33:0/1438649994 >> v2:172.21.15.167:3300/0 conn(0x7fe1680adf50 msgr2=0x7fe1680c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:01.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16659c700 1 --2- 172.21.15.33:0/1438649994 >> v2:172.21.15.167:3300/0 conn(0x7fe1680adf50 0x7fe1680c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16659c700 1 -- 172.21.15.33:0/1438649994 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680ae9a0 con 0x7fe168155140 2023-10-06T05:35:01.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16659c700 1 --2- 172.21.15.33:0/1438649994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168155140 0x7fe168158700 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fe1580099b0 tx=0x7fe158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a015138d11d68d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:01.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16559a700 1 -- 172.21.15.33:0/1438649994 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800b820 con 0x7fe168155140 2023-10-06T05:35:01.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16559a700 1 -- 172.21.15.33:0/1438649994 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15800b980 con 0x7fe168155140 2023-10-06T05:35:01.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.817+0000 7fe16559a700 1 -- 172.21.15.33:0/1438649994 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800bd20 con 0x7fe168155140 2023-10-06T05:35:01.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 -- 172.21.15.33:0/1438649994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168155140 msgr2=0x7fe168158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 --2- 172.21.15.33:0/1438649994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168155140 0x7fe168158700 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fe1580099b0 tx=0x7fe158009250 comp rx=0 tx=0).stop 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 -- 172.21.15.33:0/1438649994 shutdown_connections 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 --2- 172.21.15.33:0/1438649994 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1680c9810 0x7fe1680699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 --2- 172.21.15.33:0/1438649994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe168155140 0x7fe168158700 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 --2- 172.21.15.33:0/1438649994 >> v2:172.21.15.167:3300/0 conn(0x7fe1680adf50 0x7fe1680c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 -- 172.21.15.33:0/1438649994 >> 172.21.15.33:0/1438649994 conn(0x7fe16806d030 msgr2=0x7fe16806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 -- 172.21.15.33:0/1438649994 shutdown_connections 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.818+0000 7fe16759e700 1 -- 172.21.15.33:0/1438649994 wait complete. 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.819+0000 7fe16759e700 1 Processor -- start 2023-10-06T05:35:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.819+0000 7fe16759e700 1 -- start start 2023-10-06T05:35:01.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.820+0000 7fe16759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 0x7fe168152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.820+0000 7fe16759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1680c9810 0x7fe16814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.820+0000 7fe16759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe168155140 0x7fe168151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.820+0000 7fe16759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe168151cf0 con 0x7fe1680adf50 2023-10-06T05:35:01.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.820+0000 7fe16759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe168151fd0 con 0x7fe1680c9810 2023-10-06T05:35:01.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.820+0000 7fe16759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1681522b0 con 0x7fe168155140 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 0x7fe168152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 0x7fe168152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49234/0 (socket says 172.21.15.33:49234) 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 -- 172.21.15.33:0/1569900692 learned_addr learned my addr 172.21.15.33:0/1569900692 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe165d9b700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1680c9810 0x7fe16814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe166d9d700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe168155140 0x7fe168151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 -- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe168155140 msgr2=0x7fe168151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe168155140 0x7fe168151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 -- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1680c9810 msgr2=0x7fe16814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1680c9810 0x7fe16814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 -- 172.21.15.33:0/1569900692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580094f0 con 0x7fe1680adf50 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe16659c700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 0x7fe168152d90 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fe1580094c0 tx=0x7fe158017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.821+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158024440 con 0x7fe1680adf50 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.822+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1580245e0 con 0x7fe1680adf50 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.822+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580249a0 con 0x7fe1680adf50 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.822+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe168152590 con 0x7fe1680adf50 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.822+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1680b1700 con 0x7fe1680adf50 2023-10-06T05:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.823+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16804c380 con 0x7fe1680adf50 2023-10-06T05:35:01.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.823+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe158024d50 con 0x7fe1680adf50 2023-10-06T05:35:01.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.825+0000 7fe1567fc700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe148066f30 0x7fe1480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:01.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.825+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fe158086360 con 0x7fe1680adf50 2023-10-06T05:35:01.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.825+0000 7fe165d9b700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe148066f30 0x7fe1480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:01.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.826+0000 7fe165d9b700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe148066f30 0x7fe1480693e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe1680b1f80 tx=0x7fe15c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:01.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:01.829+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe158053300 con 0x7fe1680adf50 2023-10-06T05:35:02.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.075+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1680c2970 con 0x7fe1680adf50 2023-10-06T05:35:02.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.075+0000 7fe1567fc700 1 -- 172.21.15.33:0/1569900692 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) v1 ==== 95+0+4617 (secure 0 0 0) 0x7fe158056920 con 0x7fe1680adf50 2023-10-06T05:35:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:02.077 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":210,"default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:33:57.697950+0000","modified":"2023-10-06T05:34:56.050401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":270,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9077},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9077":{"gid":9077,"name":"b","rank":0,"incarnation":207,"state":"up:active","state_seq":4,"addr":"172.21.15.167:6835/3640840489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3640840489},{"type":"v1","addr":"172.21.15.167:6835","nonce":3640840489}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":19}]} 2023-10-06T05:35:02.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe148066f30 msgr2=0x7fe1480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe148066f30 0x7fe1480693e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe1680b1f80 tx=0x7fe15c008040 comp rx=0 tx=0).stop 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 msgr2=0x7fe168152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 0x7fe168152d90 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fe1580094c0 tx=0x7fe158017000 comp rx=0 tx=0).stop 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 shutdown_connections 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fe148066f30 0x7fe1480693e0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe168155140 0x7fe168151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1680adf50 0x7fe168152d90 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 --2- 172.21.15.33:0/1569900692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1680c9810 0x7fe16814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 >> 172.21.15.33:0/1569900692 conn(0x7fe16806d030 msgr2=0x7fe1680cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:02.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 shutdown_connections 2023-10-06T05:35:02.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.078+0000 7fe16759e700 1 -- 172.21.15.33:0/1569900692 wait complete. 2023-10-06T05:35:02.081 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 210 2023-10-06T05:35:02.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.285+0000 7fb870b9d700 1 Processor -- start 2023-10-06T05:35:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.285+0000 7fb870b9d700 1 -- start start 2023-10-06T05:35:02.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb870b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb86c0c7920 0x7fb86c160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb870b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c06ab60 con 0x7fb86c155af0 2023-10-06T05:35:02.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb870b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c06ae40 con 0x7fb86c0c8b50 2023-10-06T05:35:02.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb870b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c06b7e0 con 0x7fb86c0c7920 2023-10-06T05:35:02.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb86b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb86c155af0 0x7fb86c06a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59688/0 (socket says 172.21.15.33:59688) 2023-10-06T05:35:02.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb86b7fe700 1 -- 172.21.15.33:0/2574674340 learned_addr learned my addr 172.21.15.33:0/2574674340 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb86bfff700 1 --2- 172.21.15.33:0/2574674340 >> v2:172.21.15.167:3300/0 conn(0x7fb86c0c7920 0x7fb86c160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90121483 0 0) 0x7fb86c06ab60 con 0x7fb86c155af0 2023-10-06T05:35:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.287+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb854003570 con 0x7fb86c155af0 2023-10-06T05:35:02.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519737803 0 0) 0x7fb86c06ae40 con 0x7fb86c0c8b50 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86c06ab60 con 0x7fb86c0c8b50 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3963654067 0 0) 0x7fb854003570 con 0x7fb86c155af0 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 >> v2:172.21.15.167:3300/0 conn(0x7fb86c0c7920 msgr2=0x7fb86c160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 --2- 172.21.15.33:0/2574674340 >> v2:172.21.15.167:3300/0 conn(0x7fb86c0c7920 0x7fb86c160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 >> v1:172.21.15.167:6789/0 conn(0x7fb86c0c8b50 legacy=0x7fb86c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c06b120 con 0x7fb86c155af0 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb85c001470 con 0x7fb86c155af0 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86bfff700 1 --2- 172.21.15.33:0/2574674340 >> v2:172.21.15.167:3300/0 conn(0x7fb86c0c7920 0x7fb86c160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 >> v1:172.21.15.33:6789/0 conn(0x7fb86c155af0 legacy=0x7fb86c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb854004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb854004810 0x7fb854004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb854005210 0x7fb85400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c06b7e0 con 0x7fb854003c40 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85400c1b0 con 0x7fb854004810 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85400e580 con 0x7fb854005210 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb854004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86affd700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb854004810 0x7fb854004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86bfff700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb854005210 0x7fb85400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86b7fe700 1 -- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb854005210 msgr2=0x7fb85400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb854005210 0x7fb85400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86b7fe700 1 -- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb854004810 msgr2=0x7fb854004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.288+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb854004810 0x7fb854004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb86b7fe700 1 -- 172.21.15.33:0/2574674340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854003570 con 0x7fb854003c40 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb86bfff700 1 -- 172.21.15.33:0/2574674340 reap_dead start 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb854004030 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fb85c007fd0 tx=0x7fb85c001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34a8fc1fb219fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00cae0 con 0x7fb854003c40 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c00cc60 con 0x7fb854003c40 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb86a7fc700 1 -- 172.21.15.33:0/2574674340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c005920 con 0x7fb854003c40 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb870b9d700 1 -- 172.21.15.33:0/2574674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 msgr2=0x7fb854004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.289+0000 7fb870b9d700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb854004030 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fb85c007fd0 tx=0x7fb85c001d40 comp rx=0 tx=0).stop 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.290+0000 7fb870b9d700 1 -- 172.21.15.33:0/2574674340 shutdown_connections 2023-10-06T05:35:02.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.290+0000 7fb870b9d700 1 --2- 172.21.15.33:0/2574674340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb854004030 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.290+0000 7fb870b9d700 1 -- 172.21.15.33:0/2574674340 >> 172.21.15.33:0/2574674340 conn(0x7fb86c06d030 msgr2=0x7fb86c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.290+0000 7fb870b9d700 1 -- 172.21.15.33:0/2574674340 shutdown_connections 2023-10-06T05:35:02.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.290+0000 7fb870b9d700 1 -- 172.21.15.33:0/2574674340 wait complete. 2023-10-06T05:35:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.291+0000 7fb870b9d700 1 Processor -- start 2023-10-06T05:35:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.291+0000 7fb870b9d700 1 -- start start 2023-10-06T05:35:02.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb870b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb86c0ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb870b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb86c153a20 0x7fb86c169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb870b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb86c153e90 0x7fb86c16a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb870b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0abad0 con 0x7fb854003c40 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb870b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0ac090 con 0x7fb86c153a20 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb870b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0aca60 con 0x7fb86c153e90 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.292+0000 7fb86b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb86c0ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb86c0ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49248/0 (socket says 172.21.15.33:49248) 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 -- 172.21.15.33:0/4152690655 learned_addr learned my addr 172.21.15.33:0/4152690655 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86bfff700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb86c153e90 0x7fb86c16a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86affd700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb86c153a20 0x7fb86c169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 -- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb86c153e90 msgr2=0x7fb86c16a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb86c153e90 0x7fb86c16a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 -- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb86c153a20 msgr2=0x7fb86c169f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb86c153a20 0x7fb86c169f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 -- 172.21.15.33:0/4152690655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c16aa80 con 0x7fb854003c40 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86bfff700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb86c153e90 0x7fb86c16a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86affd700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb86c153a20 0x7fb86c169f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb86b7fe700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb86c0ac520 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fb85400c180 tx=0x7fb85c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.293+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c023070 con 0x7fb854003c40 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.294+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c004a40 con 0x7fb854003c40 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.294+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c001d40 con 0x7fb854003c40 2023-10-06T05:35:02.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.294+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c16ad60 con 0x7fb854003c40 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.294+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c0c6050 con 0x7fb854003c40 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.295+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c0c2d00 con 0x7fb854003c40 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.296+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb85c019070 con 0x7fb854003c40 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.296+0000 7fb8537fe700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb858066f30 0x7fb8580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.297+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fb85c0851e0 con 0x7fb854003c40 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.297+0000 7fb86affd700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb858066f30 0x7fb8580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.298+0000 7fb86affd700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb858066f30 0x7fb8580693e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0accd0 tx=0x7fb860009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:02.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.301+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb85c052230 con 0x7fb854003c40 2023-10-06T05:35:02.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.503+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb86c04c380 con 0x7fb854003c40 2023-10-06T05:35:02.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.504+0000 7fb8537fe700 1 -- 172.21.15.33:0/4152690655 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+17193 (secure 0 0 0) 0x7fb85c055850 con 0x7fb854003c40 2023-10-06T05:35:02.505 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:02.506 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":271,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:34:53.038933+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:33:56.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":40,"pool_name":"cephfs_data","create_time":"2023-10-06T05:33:56.676003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"40.1c","mappings":[{"from":5,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/3478690144":"2023-10-07T05:34:53.030921+0000","172.21.15.167:6834/3478690144":"2023-10-07T05:34:53.030921+0000","172.21.15.167:6837/3376723254":"2023-10-07T05:34:52.131020+0000","172.21.15.167:6835/3376723254":"2023-10-07T05:34:52.131020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:02.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb858066f30 msgr2=0x7fb8580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb858066f30 0x7fb8580693e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0accd0 tx=0x7fb860009040 comp rx=0 tx=0).stop 2023-10-06T05:35:02.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 msgr2=0x7fb86c0ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb86c0ac520 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fb85400c180 tx=0x7fb85c015000 comp rx=0 tx=0).stop 2023-10-06T05:35:02.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 shutdown_connections 2023-10-06T05:35:02.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb858066f30 0x7fb8580693e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb86c153e90 0x7fb86c16a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb854003c40 0x7fb86c0ac520 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 --2- 172.21.15.33:0/4152690655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb86c153a20 0x7fb86c169f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 >> 172.21.15.33:0/4152690655 conn(0x7fb86c06d030 msgr2=0x7fb86c0c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:02.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 shutdown_connections 2023-10-06T05:35:02.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.506+0000 7fb870b9d700 1 -- 172.21.15.33:0/4152690655 wait complete. 2023-10-06T05:35:02.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:35:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.711+0000 7f0cca19d700 1 Processor -- start 2023-10-06T05:35:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.711+0000 7f0cca19d700 1 -- start start 2023-10-06T05:35:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.712+0000 7f0cca19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0cc41597d0 0x7f0cc406c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.712+0000 7f0cca19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.712+0000 7f0cca19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40d0ed0 con 0x7f0cc4159400 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.712+0000 7f0cca19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40b1160 con 0x7f0cc41597d0 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.712+0000 7f0cca19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40cb6d0 con 0x7f0cc40cc770 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0cc41597d0 0x7f0cc406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc999c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0cc41597d0 0x7f0cc406c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:56876/0 (socket says 172.21.15.33:56876) 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 -- 172.21.15.33:0/215199760 learned_addr learned my addr 172.21.15.33:0/215199760 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc919b700 1 --1- 172.21.15.33:0/215199760 >> v1:172.21.15.167:6789/0 conn(0x7f0cc4159400 0x7f0cc40cab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:34208/0 (socket says 172.21.15.33:34208) 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 -- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 msgr2=0x7f0cc406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 --2- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc406caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 -- 172.21.15.33:0/215199760 >> v1:172.21.15.167:6789/0 conn(0x7f0cc4159400 legacy=0x7f0cc40cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.713+0000 7f0cc899a700 1 -- 172.21.15.33:0/215199760 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40cb9b0 con 0x7f0cc41597d0 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.714+0000 7f0cc899a700 1 --2- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41597d0 0x7f0cc406c560 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40099b0 tx=0x7f0cb4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f1d581d02b4cc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.714+0000 7f0cbbfff700 1 -- 172.21.15.33:0/215199760 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4009ea0 con 0x7f0cc41597d0 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.714+0000 7f0cbbfff700 1 -- 172.21.15.33:0/215199760 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb400b860 con 0x7f0cc41597d0 2023-10-06T05:35:02.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.714+0000 7f0cbbfff700 1 -- 172.21.15.33:0/215199760 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400bc00 con 0x7f0cc41597d0 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 -- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41597d0 msgr2=0x7f0cc406c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 --2- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41597d0 0x7f0cc406c560 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40099b0 tx=0x7f0cb4009250 comp rx=0 tx=0).stop 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 -- 172.21.15.33:0/215199760 shutdown_connections 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 --2- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 --2- 172.21.15.33:0/215199760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41597d0 0x7f0cc406c560 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 -- 172.21.15.33:0/215199760 >> 172.21.15.33:0/215199760 conn(0x7f0cc406e4b0 msgr2=0x7f0cc40d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 -- 172.21.15.33:0/215199760 shutdown_connections 2023-10-06T05:35:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.715+0000 7f0cca19d700 1 -- 172.21.15.33:0/215199760 wait complete. 2023-10-06T05:35:02.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.716+0000 7f0cca19d700 1 Processor -- start 2023-10-06T05:35:02.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.716+0000 7f0cca19d700 1 -- start start 2023-10-06T05:35:02.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cca19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc4148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cca19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc4159400 0x7f0cc40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cca19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 0x7f0cc4155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cca19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc414b330 con 0x7f0cc41597d0 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cca19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc414b4c0 con 0x7f0cc4159400 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cca19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40b3d60 con 0x7f0cc40cc770 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.717+0000 7f0cc999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 0x7f0cc4155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 0x7f0cc4155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49250/0 (socket says 172.21.15.33:49250) 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 -- 172.21.15.33:0/3654709157 learned_addr learned my addr 172.21.15.33:0/3654709157 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc899a700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc4159400 0x7f0cc40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc919b700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc4148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 -- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 msgr2=0x7f0cc4148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc4148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 -- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc4159400 msgr2=0x7f0cc40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc4159400 0x7f0cc40b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 -- 172.21.15.33:0/3654709157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40094f0 con 0x7f0cc41597d0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc899a700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc4159400 0x7f0cc40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cc999c700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 0x7f0cc4155ab0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009ef0 tx=0x7f0cbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.718+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00cc50 con 0x7f0cc41597d0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.719+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc00f040 con 0x7f0cc41597d0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.719+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc014480 con 0x7f0cc41597d0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.719+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc40b4040 con 0x7f0cc41597d0 2023-10-06T05:35:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.719+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc40b4600 con 0x7f0cc41597d0 2023-10-06T05:35:02.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.720+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc414b630 con 0x7f0cc41597d0 2023-10-06T05:35:02.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.721+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cbc014830 con 0x7f0cc41597d0 2023-10-06T05:35:02.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.721+0000 7f0cb97fa700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca8066f30 0x7f0ca80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:02.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.722+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f0cbc085540 con 0x7f0cc41597d0 2023-10-06T05:35:02.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.722+0000 7f0cc919b700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca8066f30 0x7f0ca80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:02.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.723+0000 7f0cc919b700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca8066f30 0x7f0ca80693e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0005f80 tx=0x7f0cc0007ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:02.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.726+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cbc052580 con 0x7f0cc41597d0 2023-10-06T05:35:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.974+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0cc404c380 con 0x7f0cc41597d0 2023-10-06T05:35:02.977 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:35:02.975+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:35:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.979+0000 7f0cb97fa700 1 -- 172.21.15.33:0/3654709157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v211) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0cbc055ba0 con 0x7f0cc41597d0 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca8066f30 msgr2=0x7f0ca80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca8066f30 0x7f0ca80693e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0005f80 tx=0x7f0cc0007ce0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 msgr2=0x7f0cc4155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 0x7f0cc4155ab0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009ef0 tx=0x7f0cbc00b040 comp rx=0 tx=0).stop 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 shutdown_connections 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0ca8066f30 0x7f0ca80693e0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc41597d0 0x7f0cc4155ab0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc40cc770 0x7f0cc4148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 --2- 172.21.15.33:0/3654709157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc4159400 0x7f0cc40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 >> 172.21.15.33:0/3654709157 conn(0x7f0cc406e4b0 msgr2=0x7f0cc40c52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 shutdown_connections 2023-10-06T05:35:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:02.983+0000 7f0cca19d700 1 -- 172.21.15.33:0/3654709157 wait complete. 2023-10-06T05:35:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:35:03.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:35:03.006 INFO:tasks.ceph.mds.b.smithi167.stderr: -1433> 2023-10-06T05:34:58.044+0000 7f9f329f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:35:03.045 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:35:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.190+0000 7f4b87fff700 1 Processor -- start 2023-10-06T05:35:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.190+0000 7f4b87fff700 1 -- start start 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b87fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b880aff90 0x7f4b880a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8815d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b87fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8815d3a0 con 0x7f4b8806c7b0 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b87fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880686c0 con 0x7f4b880aff90 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b87fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880b0470 con 0x7f4b8806cb80 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b880aff90 0x7f4b880a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b880aff90 0x7f4b880a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49256/0 (socket says 172.21.15.33:49256) 2023-10-06T05:35:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 -- 172.21.15.33:0/578465892 learned_addr learned my addr 172.21.15.33:0/578465892 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b877fe700 1 --2- 172.21.15.33:0/578465892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 -- 172.21.15.33:0/578465892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 msgr2=0x7f4b8815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 --2- 172.21.15.33:0/578465892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 -- 172.21.15.33:0/578465892 >> v1:172.21.15.33:6789/0 conn(0x7f4b8806c7b0 legacy=0x7f4b88067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:03.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.192+0000 7f4b867fc700 1 -- 172.21.15.33:0/578465892 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880b0750 con 0x7f4b880aff90 2023-10-06T05:35:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.193+0000 7f4b867fc700 1 --2- 172.21.15.33:0/578465892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880aff90 0x7f4b880a9890 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0099b0 tx=0x7f4b7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ba0396ad9db2c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.193+0000 7f4b85ffb700 1 -- 172.21.15.33:0/578465892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00b860 con 0x7f4b880aff90 2023-10-06T05:35:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.193+0000 7f4b85ffb700 1 -- 172.21.15.33:0/578465892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00b9c0 con 0x7f4b880aff90 2023-10-06T05:35:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.193+0000 7f4b85ffb700 1 -- 172.21.15.33:0/578465892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00bd60 con 0x7f4b880aff90 2023-10-06T05:35:03.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.193+0000 7f4b87fff700 1 -- 172.21.15.33:0/578465892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880aff90 msgr2=0x7f4b880a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.193+0000 7f4b87fff700 1 --2- 172.21.15.33:0/578465892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880aff90 0x7f4b880a9890 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0099b0 tx=0x7f4b7c009250 comp rx=0 tx=0).stop 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.194+0000 7f4b87fff700 1 -- 172.21.15.33:0/578465892 shutdown_connections 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.194+0000 7f4b87fff700 1 --2- 172.21.15.33:0/578465892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.194+0000 7f4b87fff700 1 --2- 172.21.15.33:0/578465892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b880aff90 0x7f4b880a9890 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.194+0000 7f4b87fff700 1 -- 172.21.15.33:0/578465892 >> 172.21.15.33:0/578465892 conn(0x7f4b8806d670 msgr2=0x7f4b8806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.194+0000 7f4b87fff700 1 -- 172.21.15.33:0/578465892 shutdown_connections 2023-10-06T05:35:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.194+0000 7f4b87fff700 1 -- 172.21.15.33:0/578465892 wait complete. 2023-10-06T05:35:03.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.195+0000 7f4b87fff700 1 Processor -- start 2023-10-06T05:35:03.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.195+0000 7f4b87fff700 1 -- start start 2023-10-06T05:35:03.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 0x7f4b8816d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:03.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8816d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:03.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b880aff90 0x7f4b88169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b87fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8815c300 con 0x7f4b8806c7b0 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b87fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8816acb0 con 0x7f4b880aff90 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b87fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8816b830 con 0x7f4b8806cb80 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 0x7f4b8816d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 0x7f4b8816d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49266/0 (socket says 172.21.15.33:49266) 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.196+0000 7f4b86ffd700 1 -- 172.21.15.33:0/412932438 learned_addr learned my addr 172.21.15.33:0/412932438 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b877fe700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b880aff90 0x7f4b88169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b86ffd700 1 -- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 msgr2=0x7f4b8816d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8816d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b86ffd700 1 -- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b880aff90 msgr2=0x7f4b88169ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b880aff90 0x7f4b88169ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b86ffd700 1 -- 172.21.15.33:0/412932438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0094f0 con 0x7f4b8806c7b0 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b877fe700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b880aff90 0x7f4b88169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b86ffd700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 0x7f4b8816d0e0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c7f0 tx=0x7f4b78007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78010510 con 0x7f4b8806c7b0 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78010690 con 0x7f4b8806c7b0 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78010a50 con 0x7f4b8806c7b0 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8816af00 con 0x7f4b8806c7b0 2023-10-06T05:35:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.197+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8816b4c0 con 0x7f4b8806c7b0 2023-10-06T05:35:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.198+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8804c380 con 0x7f4b8806c7b0 2023-10-06T05:35:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.199+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b7801b400 con 0x7f4b8806c7b0 2023-10-06T05:35:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.200+0000 7f4b6effd700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.200+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f4b78084360 con 0x7f4b8806c7b0 2023-10-06T05:35:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.201+0000 7f4b867fc700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:03.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.202+0000 7f4b867fc700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0099b0 tx=0x7f4b7c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.204+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b78051330 con 0x7f4b8806c7b0 2023-10-06T05:35:03.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.451+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b880b0a30 con 0x7f4b8806c7b0 2023-10-06T05:35:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.981+0000 7f4b6effd700 1 -- 172.21.15.33:0/412932438 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v212) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b78054950 con 0x7f4b8806c7b0 2023-10-06T05:35:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.983+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 msgr2=0x7f4b740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.983+0000 7f4b87fff700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0099b0 tx=0x7f4b7c017000 comp rx=0 tx=0).stop 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.983+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 msgr2=0x7f4b8816d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.983+0000 7f4b87fff700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 0x7f4b8816d0e0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c7f0 tx=0x7f4b78007f00 comp rx=0 tx=0).stop 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 shutdown_connections 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4b74066f30 0x7f4b740693e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b880aff90 0x7f4b88169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8806c7b0 0x7f4b8816d0e0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 --2- 172.21.15.33:0/412932438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4b8806cb80 0x7f4b8816d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 >> 172.21.15.33:0/412932438 conn(0x7f4b8806d670 msgr2=0x7f4b880af340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 shutdown_connections 2023-10-06T05:35:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:03.984+0000 7f4b87fff700 1 -- 172.21.15.33:0/412932438 wait complete. 2023-10-06T05:35:04.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:35:04.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.197+0000 7f663f59d700 1 Processor -- start 2023-10-06T05:35:04.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.198+0000 7f663f59d700 1 -- start start 2023-10-06T05:35:04.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f66380aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:04.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663f59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6638155e70 0x7f66380c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:04.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663f59d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6638069ad0 con 0x7f6638155aa0 2023-10-06T05:35:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663f59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6638069db0 con 0x7f6638155e70 2023-10-06T05:35:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663806a090 con 0x7f66380acd00 2023-10-06T05:35:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663e59b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6638155aa0 0x7f66380ac5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59696/0 (socket says 172.21.15.33:59696) 2023-10-06T05:35:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663e59b700 1 -- 172.21.15.33:0/1896923371 learned_addr learned my addr 172.21.15.33:0/1896923371 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663ed9c700 1 --2- 172.21.15.33:0/1896923371 >> v2:172.21.15.33:3300/0 conn(0x7f6638155e70 0x7f66380c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:04.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.199+0000 7f663dd9a700 1 --2- 172.21.15.33:0/1896923371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f66380aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663ed9c700 1 -- 172.21.15.33:0/1896923371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 msgr2=0x7f66380aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663ed9c700 1 --2- 172.21.15.33:0/1896923371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f66380aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663ed9c700 1 -- 172.21.15.33:0/1896923371 >> v1:172.21.15.33:6789/0 conn(0x7f6638155aa0 legacy=0x7f66380ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663ed9c700 1 -- 172.21.15.33:0/1896923371 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663806a370 con 0x7f6638155e70 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663ed9c700 1 --2- 172.21.15.33:0/1896923371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f66380c90c0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f6630009e60 tx=0x7f6630009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4de95516edfbfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663d599700 1 -- 172.21.15.33:0/1896923371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000e070 con 0x7f6638155e70 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663d599700 1 -- 172.21.15.33:0/1896923371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000bb60 con 0x7f6638155e70 2023-10-06T05:35:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.200+0000 7f663d599700 1 -- 172.21.15.33:0/1896923371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630012430 con 0x7f6638155e70 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 -- 172.21.15.33:0/1896923371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 msgr2=0x7f66380c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 --2- 172.21.15.33:0/1896923371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f66380c90c0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f6630009e60 tx=0x7f6630009780 comp rx=0 tx=0).stop 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 -- 172.21.15.33:0/1896923371 shutdown_connections 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 --2- 172.21.15.33:0/1896923371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f66380c90c0 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 --2- 172.21.15.33:0/1896923371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f66380aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 -- 172.21.15.33:0/1896923371 >> 172.21.15.33:0/1896923371 conn(0x7f6638068090 msgr2=0x7f663806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:04.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 -- 172.21.15.33:0/1896923371 shutdown_connections 2023-10-06T05:35:04.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.201+0000 7f663f59d700 1 -- 172.21.15.33:0/1896923371 wait complete. 2023-10-06T05:35:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.203+0000 7f663f59d700 1 Processor -- start 2023-10-06T05:35:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.203+0000 7f663f59d700 1 -- start start 2023-10-06T05:35:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663f59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f663814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663f59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6638155aa0 0x7f6638152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663f59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f6638150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663f59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663814e3a0 con 0x7f6638155e70 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663f59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380cbb20 con 0x7f6638155aa0 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663f59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66380cb1f0 con 0x7f66380acd00 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f6638150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f6638150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49282/0 (socket says 172.21.15.33:49282) 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 -- 172.21.15.33:0/3765925963 learned_addr learned my addr 172.21.15.33:0/3765925963 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 -- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 msgr2=0x7f663814de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:04.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663e59b700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f663814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663dd9a700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6638155aa0 0x7f6638152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f663814de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 -- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6638155aa0 msgr2=0x7f6638152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6638155aa0 0x7f6638152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663ed9c700 1 -- 172.21.15.33:0/3765925963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009a20 con 0x7f6638155e70 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663dd9a700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6638155aa0 0x7f6638152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.204+0000 7f663e59b700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f663814de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.205+0000 7f663ed9c700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f6638150a70 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f6630013000 tx=0x7f6630019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:04.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.205+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630027470 con 0x7f6638155e70 2023-10-06T05:35:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.205+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66300275f0 con 0x7f6638155e70 2023-10-06T05:35:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.205+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66380cb4d0 con 0x7f6638155e70 2023-10-06T05:35:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.205+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300279b0 con 0x7f6638155e70 2023-10-06T05:35:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.205+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66380ccfd0 con 0x7f6638155e70 2023-10-06T05:35:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.206+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66380cbcb0 con 0x7f6638155e70 2023-10-06T05:35:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.207+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6630024070 con 0x7f6638155e70 2023-10-06T05:35:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.208+0000 7f662e7fc700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f661c066f30 0x7f661c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.208+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f6630087300 con 0x7f6638155e70 2023-10-06T05:35:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.208+0000 7f663e59b700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f661c066f30 0x7f661c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.209+0000 7f663e59b700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f661c066f30 0x7f661c0693e0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6634001820 tx=0x7f6634009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:04.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.212+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6630054250 con 0x7f6638155e70 2023-10-06T05:35:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:04.416+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6638063ff0 con 0x7f6638155e70 2023-10-06T05:35:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.096+0000 7f662e7fc700 1 -- 172.21.15.33:0/3765925963 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v273) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6630057870 con 0x7f6638155e70 2023-10-06T05:35:05.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f661c066f30 msgr2=0x7f661c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:05.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f661c066f30 0x7f661c0693e0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6634001820 tx=0x7f6634009040 comp rx=0 tx=0).stop 2023-10-06T05:35:05.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 msgr2=0x7f6638150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:05.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f6638150a70 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f6630013000 tx=0x7f6630019000 comp rx=0 tx=0).stop 2023-10-06T05:35:05.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 shutdown_connections 2023-10-06T05:35:05.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f661c066f30 0x7f661c0693e0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6638155e70 0x7f6638150a70 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66380acd00 0x7f663814de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 --2- 172.21.15.33:0/3765925963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6638155aa0 0x7f6638152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.101+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 >> 172.21.15.33:0/3765925963 conn(0x7f6638068090 msgr2=0x7f66380aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:05.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.102+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 shutdown_connections 2023-10-06T05:35:05.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.102+0000 7f663f59d700 1 -- 172.21.15.33:0/3765925963 wait complete. 2023-10-06T05:35:05.105 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:35:05.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:35:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.319+0000 7f737f39d700 1 Processor -- start 2023-10-06T05:35:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.319+0000 7f737f39d700 1 -- start start 2023-10-06T05:35:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737f39d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7378155aa0 0x7f73780aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737f39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7378155e70 0x7f73780ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737f39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780acb30 0x7f737806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737f39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73780acfb0 con 0x7f7378155e70 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737f39d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7378069ad0 con 0x7f7378155aa0 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737f39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7378069db0 con 0x7f73780acb30 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737db9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7378155e70 0x7f73780ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737db9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7378155e70 0x7f73780ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49292/0 (socket says 172.21.15.33:49292) 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737db9a700 1 -- 172.21.15.33:0/3260232349 learned_addr learned my addr 172.21.15.33:0/3260232349 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737e39b700 1 --2- 172.21.15.33:0/3260232349 >> v2:172.21.15.167:3300/0 conn(0x7f7378155aa0 0x7f73780aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.321+0000 7f737eb9c700 1 --2- 172.21.15.33:0/3260232349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780acb30 0x7f737806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737db9a700 1 -- 172.21.15.33:0/3260232349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780acb30 msgr2=0x7f737806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737db9a700 1 --2- 172.21.15.33:0/3260232349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780acb30 0x7f737806ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737db9a700 1 -- 172.21.15.33:0/3260232349 >> v2:172.21.15.167:3300/0 conn(0x7f7378155aa0 msgr2=0x7f73780aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737db9a700 1 --2- 172.21.15.33:0/3260232349 >> v2:172.21.15.167:3300/0 conn(0x7f7378155aa0 0x7f73780aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737db9a700 1 -- 172.21.15.33:0/3260232349 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737806a090 con 0x7f7378155e70 2023-10-06T05:35:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737db9a700 1 --2- 172.21.15.33:0/3260232349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7378155e70 0x7f73780ac5f0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f7378064670 tx=0x7f7368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26eef55988d578e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737d399700 1 -- 172.21.15.33:0/3260232349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736800b640 con 0x7f7378155e70 2023-10-06T05:35:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737d399700 1 -- 172.21.15.33:0/3260232349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736800b7a0 con 0x7f7378155e70 2023-10-06T05:35:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.322+0000 7f737d399700 1 -- 172.21.15.33:0/3260232349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736800bb40 con 0x7f7378155e70 2023-10-06T05:35:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 -- 172.21.15.33:0/3260232349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7378155e70 msgr2=0x7f73780ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 --2- 172.21.15.33:0/3260232349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7378155e70 0x7f73780ac5f0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f7378064670 tx=0x7f7368009250 comp rx=0 tx=0).stop 2023-10-06T05:35:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 -- 172.21.15.33:0/3260232349 shutdown_connections 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 --2- 172.21.15.33:0/3260232349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780acb30 0x7f737806ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 --2- 172.21.15.33:0/3260232349 >> v2:172.21.15.167:3300/0 conn(0x7f7378155aa0 0x7f73780aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 --2- 172.21.15.33:0/3260232349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7378155e70 0x7f73780ac5f0 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 -- 172.21.15.33:0/3260232349 >> 172.21.15.33:0/3260232349 conn(0x7f7378068090 msgr2=0x7f737806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 -- 172.21.15.33:0/3260232349 shutdown_connections 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.323+0000 7f737f39d700 1 -- 172.21.15.33:0/3260232349 wait complete. 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.324+0000 7f737f39d700 1 Processor -- start 2023-10-06T05:35:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.324+0000 7f737f39d700 1 -- start start 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737f39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 0x7f73780c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737f39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378155aa0 0x7f737815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737f39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378155e70 0x7f737815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737f39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737816ca50 con 0x7f73780acb30 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737f39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7378158720 con 0x7f7378155aa0 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737f39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7378158a00 con 0x7f7378155e70 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 0x7f73780c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.325+0000 7f737e39b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 0x7f73780c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49302/0 (socket says 172.21.15.33:49302) 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 -- 172.21.15.33:0/4215903574 learned_addr learned my addr 172.21.15.33:0/4215903574 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 -- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378155e70 msgr2=0x7f737815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378155e70 0x7f737815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 -- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378155aa0 msgr2=0x7f737815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378155aa0 0x7f737815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 -- 172.21.15.33:0/4215903574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680094f0 con 0x7f73780acb30 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737e39b700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 0x7f73780c9420 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f737400ec80 tx=0x7f73740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400c9d0 con 0x7f73780acb30 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737400cb30 con 0x7f73780acb30 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374013470 con 0x7f73780acb30 2023-10-06T05:35:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7378158ce0 con 0x7f73780acb30 2023-10-06T05:35:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.326+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737816a7d0 con 0x7f73780acb30 2023-10-06T05:35:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.327+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7378063ff0 con 0x7f73780acb30 2023-10-06T05:35:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.328+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7374013820 con 0x7f73780acb30 2023-10-06T05:35:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.329+0000 7f736e7fc700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.329+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(273..273 src has 1..273) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f7374084a30 con 0x7f73780acb30 2023-10-06T05:35:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.330+0000 7f737db9a700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.330+0000 7f737db9a700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f735c066f30 0x7f735c0693e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7378064670 tx=0x7f7368017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.333+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7374051b30 con 0x7f73780acb30 2023-10-06T05:35:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:05.537+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7378069640 con 0x7f73780acb30 2023-10-06T05:35:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.097+0000 7f736e7fc700 1 -- 172.21.15.33:0/4215903574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v274) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7374055150 con 0x7f73780acb30 2023-10-06T05:35:06.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.103+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f735c066f30 msgr2=0x7f735c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.103+0000 7f737f39d700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f735c066f30 0x7f735c0693e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7378064670 tx=0x7f7368017000 comp rx=0 tx=0).stop 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.103+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 msgr2=0x7f73780c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.103+0000 7f737f39d700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 0x7f73780c9420 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f737400ec80 tx=0x7f73740091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 shutdown_connections 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7378155e70 0x7f737815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780acb30 0x7f73780c9420 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 --2- 172.21.15.33:0/4215903574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7378155aa0 0x7f737815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 >> 172.21.15.33:0/4215903574 conn(0x7f7378068090 msgr2=0x7f73780b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:06.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 shutdown_connections 2023-10-06T05:35:06.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.104+0000 7f737f39d700 1 -- 172.21.15.33:0/4215903574 wait complete. 2023-10-06T05:35:06.108 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:35:06.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:35:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.320+0000 7f82b659d700 1 Processor -- start 2023-10-06T05:35:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.321+0000 7f82b659d700 1 -- start start 2023-10-06T05:35:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 0x7f82b00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b659d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00ad430 con 0x7f82b0156370 2023-10-06T05:35:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b659d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00afe20 con 0x7f82b00cc7e0 2023-10-06T05:35:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006bc70 con 0x7f82b00b0900 2023-10-06T05:35:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b559b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f82b0156370 0x7f82b00c8db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59708/0 (socket says 172.21.15.33:59708) 2023-10-06T05:35:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b559b700 1 -- 172.21.15.33:0/2146749182 learned_addr learned my addr 172.21.15.33:0/2146749182 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.322+0000 7f82b5d9c700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 0x7f82b00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834127762 0 0) 0x7f82b00ad430 con 0x7f82b0156370 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f829c003570 con 0x7f82b0156370 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313256255 0 0) 0x7f82b00afe20 con 0x7f82b00cc7e0 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b00ad430 con 0x7f82b00cc7e0 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301500534 0 0) 0x7f829c003570 con 0x7f82b0156370 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 msgr2=0x7f82b00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 0x7f82b00af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 >> v1:172.21.15.167:6789/0 conn(0x7f82b00cc7e0 legacy=0x7f82b00b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00cccc0 con 0x7f82b0156370 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f82ac001470 con 0x7f82b0156370 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 >> v1:172.21.15.33:6789/0 conn(0x7f82b0156370 legacy=0x7f82b00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 0x7f829c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f829c0041d0 0x7f829c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f829c004d50 0x7f829c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006bc70 con 0x7f829c0041d0 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c00bbf0 con 0x7f829c004d50 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c00dfc0 con 0x7f82b00b0900 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.323+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f829c0041d0 0x7f829c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b5d9c700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f829c004d50 0x7f829c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b559b700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 0x7f829c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b4d9a700 1 -- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 msgr2=0x7f829c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82b00b0900 0x7f829c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b4d9a700 1 -- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f829c004d50 msgr2=0x7f829c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f829c004d50 0x7f829c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b4d9a700 1 -- 172.21.15.33:0/2146749182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c003570 con 0x7f829c0041d0 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f829c0041d0 0x7f829c004810 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f82a0001a10 tx=0x7f82a0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd61a9a9f8107cb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000c5d0 con 0x7f829c0041d0 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a000c730 con 0x7f829c0041d0 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.324+0000 7f82a7fff700 1 -- 172.21.15.33:0/2146749182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000cad0 con 0x7f829c0041d0 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b659d700 1 -- 172.21.15.33:0/2146749182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f829c0041d0 msgr2=0x7f829c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b659d700 1 --2- 172.21.15.33:0/2146749182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f829c0041d0 0x7f829c004810 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f82a0001a10 tx=0x7f82a0007d30 comp rx=0 tx=0).stop 2023-10-06T05:35:06.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b5d9c700 1 -- 172.21.15.33:0/2146749182 reap_dead start 2023-10-06T05:35:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b659d700 1 -- 172.21.15.33:0/2146749182 shutdown_connections 2023-10-06T05:35:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b659d700 1 -- 172.21.15.33:0/2146749182 >> 172.21.15.33:0/2146749182 conn(0x7f82b006d030 msgr2=0x7f82b00cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b659d700 1 -- 172.21.15.33:0/2146749182 shutdown_connections 2023-10-06T05:35:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.325+0000 7f82b659d700 1 -- 172.21.15.33:0/2146749182 wait complete. 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.326+0000 7f82b659d700 1 Processor -- start 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.327+0000 7f82b659d700 1 -- start start 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.327+0000 7f82b659d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f829c0041d0 0x7f82b01505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.327+0000 7f82b659d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 0x7f82b01550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.327+0000 7f82b659d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0155600 0x7f82b00aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b659d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0155aa0 con 0x7f82b0150af0 2023-10-06T05:35:06.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b659d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00a9890 con 0x7f82b0155600 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b659d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00aa410 con 0x7f829c0041d0 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 0x7f82b01550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 0x7f82b01550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49312/0 (socket says 172.21.15.33:49312) 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 -- 172.21.15.33:0/3920925856 learned_addr learned my addr 172.21.15.33:0/3920925856 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 -- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f829c0041d0 msgr2=0x7f82b01505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b559b700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f829c0041d0 0x7f82b01505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f829c0041d0 0x7f82b01505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 -- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0155600 msgr2=0x7f82b00aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0155600 0x7f82b00aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 -- 172.21.15.33:0/3920925856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00aa8a0 con 0x7f82b0150af0 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b559b700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f829c0041d0 0x7f82b01505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:06.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.328+0000 7f82b4d9a700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 0x7f82b01550c0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f829c0057f0 tx=0x7f82a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.329+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000c5d0 con 0x7f82b0150af0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.329+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0001360 con 0x7f82b0150af0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.329+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b00a9ae0 con 0x7f82b0150af0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.329+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0006a70 con 0x7f82b0150af0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.329+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b00aa0a0 con 0x7f82b0150af0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.330+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b00aa5a0 con 0x7f82b0150af0 2023-10-06T05:35:06.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.331+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82a0019070 con 0x7f82b0150af0 2023-10-06T05:35:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.331+0000 7f82a57fa700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8294066f30 0x7f82940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.331+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f82a00859d0 con 0x7f82b0150af0 2023-10-06T05:35:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.332+0000 7f82b559b700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8294066f30 0x7f82940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.333+0000 7f82b559b700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8294066f30 0x7f82940693e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f82ac001bb0 tx=0x7f82ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:06.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.336+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82a0052c90 con 0x7f82b0150af0 2023-10-06T05:35:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.542+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f82b004c380 con 0x7f82b0150af0 2023-10-06T05:35:06.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.546+0000 7f82a57fa700 1 -- 172.21.15.33:0/3920925856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f82a00170a0 con 0x7f82b0150af0 2023-10-06T05:35:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8294066f30 msgr2=0x7f82940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8294066f30 0x7f82940693e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f82ac001bb0 tx=0x7f82ac009040 comp rx=0 tx=0).stop 2023-10-06T05:35:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 msgr2=0x7f82b01550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 0x7f82b01550c0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f829c0057f0 tx=0x7f82a0015000 comp rx=0 tx=0).stop 2023-10-06T05:35:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 shutdown_connections 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8294066f30 0x7f82940693e0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b0155600 0x7f82b00aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f829c0041d0 0x7f82b01505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 --2- 172.21.15.33:0/3920925856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82b0150af0 0x7f82b01550c0 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 >> 172.21.15.33:0/3920925856 conn(0x7f82b006d030 msgr2=0x7f82b0069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 shutdown_connections 2023-10-06T05:35:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.549+0000 7f82b659d700 1 -- 172.21.15.33:0/3920925856 wait complete. 2023-10-06T05:35:06.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_trim_caps' 2023-10-06T05:35:06.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.754+0000 7f6d72b9d700 1 Processor -- start 2023-10-06T05:35:06.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.755+0000 7f6d72b9d700 1 -- start start 2023-10-06T05:35:06.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d72b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c13faa0 0x7f6d6c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d72b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 0x7f6d6c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d72b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c0c9cb0 con 0x7f6d6c0c7940 2023-10-06T05:35:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d72b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c0ca270 con 0x7f6d6c06aad0 2023-10-06T05:35:06.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d72b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c06b840 con 0x7f6d6c13faa0 2023-10-06T05:35:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d7239c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 0x7f6d6c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d71b9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c13faa0 0x7f6d6c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.756+0000 7f6d7239c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 0x7f6d6c06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:56938/0 (socket says 172.21.15.33:56938) 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d7239c700 1 -- 172.21.15.33:0/3179114697 learned_addr learned my addr 172.21.15.33:0/3179114697 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d70b99700 1 -- 172.21.15.33:0/3179114697 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680834988 0 0) 0x7f6d6c0c9cb0 con 0x7f6d6c0c7940 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d70b99700 1 -- 172.21.15.33:0/3179114697 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d58003570 con 0x7f6d6c0c7940 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d71b9b700 1 -- 172.21.15.33:0/3179114697 >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 msgr2=0x7f6d6c06af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d71b9b700 1 --2- 172.21.15.33:0/3179114697 >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 0x7f6d6c06af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d71b9b700 1 -- 172.21.15.33:0/3179114697 >> v1:172.21.15.167:6789/0 conn(0x7f6d6c0c7940 legacy=0x7f6d6c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d71b9b700 1 -- 172.21.15.33:0/3179114697 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c06bb20 con 0x7f6d6c13faa0 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d7239c700 1 --2- 172.21.15.33:0/3179114697 >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 0x7f6d6c06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.757+0000 7f6d71b9b700 1 --2- 172.21.15.33:0/3179114697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c13faa0 0x7f6d6c13fe70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d680098e0 tx=0x7f6d680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734ab25c0cde6119 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d70b99700 1 -- 172.21.15.33:0/3179114697 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68011630 con 0x7f6d6c13faa0 2023-10-06T05:35:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d70b99700 1 -- 172.21.15.33:0/3179114697 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d68011790 con 0x7f6d6c13faa0 2023-10-06T05:35:06.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d70b99700 1 -- 172.21.15.33:0/3179114697 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68011b10 con 0x7f6d6c13faa0 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 -- 172.21.15.33:0/3179114697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c13faa0 msgr2=0x7f6d6c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/3179114697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c13faa0 0x7f6d6c13fe70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d680098e0 tx=0x7f6d680091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 -- 172.21.15.33:0/3179114697 shutdown_connections 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/3179114697 >> v2:172.21.15.167:3300/0 conn(0x7f6d6c06aad0 0x7f6d6c06af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/3179114697 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c13faa0 0x7f6d6c13fe70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 -- 172.21.15.33:0/3179114697 >> 172.21.15.33:0/3179114697 conn(0x7f6d6c06d030 msgr2=0x7f6d6c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 -- 172.21.15.33:0/3179114697 shutdown_connections 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.758+0000 7f6d72b9d700 1 -- 172.21.15.33:0/3179114697 wait complete. 2023-10-06T05:35:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.760+0000 7f6d72b9d700 1 Processor -- start 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.760+0000 7f6d72b9d700 1 -- start start 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d72b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c06aad0 0x7f6d6c0a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d72b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 0x7f6d6c16ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d72b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c13faa0 0x7f6d6c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d72b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c167e20 con 0x7f6d6c0c7940 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d72b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c16b160 con 0x7f6d6c13faa0 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d72b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c15a150 con 0x7f6d6c06aad0 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d7139a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 0x7f6d6c16ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d7139a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 0x7f6d6c16ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49324/0 (socket says 172.21.15.33:49324) 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d7139a700 1 -- 172.21.15.33:0/4078563537 learned_addr learned my addr 172.21.15.33:0/4078563537 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d71b9b700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c06aad0 0x7f6d6c0a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.761+0000 7f6d7239c700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c13faa0 0x7f6d6c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d7139a700 1 -- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c06aad0 msgr2=0x7f6d6c0a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d7139a700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c06aad0 0x7f6d6c0a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d7139a700 1 -- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c13faa0 msgr2=0x7f6d6c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d7139a700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c13faa0 0x7f6d6c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d7139a700 1 -- 172.21.15.33:0/4078563537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d64009ae0 con 0x7f6d6c0c7940 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d7139a700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 0x7f6d6c16ac20 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c7c0 tx=0x7f6d5c007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c002ea0 con 0x7f6d6c0c7940 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c010510 con 0x7f6d6c0c7940 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d68009420 con 0x7f6d6c0c7940 2023-10-06T05:35:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0108b0 con 0x7f6d6c0c7940 2023-10-06T05:35:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.762+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c15a710 con 0x7f6d6c0c7940 2023-10-06T05:35:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.763+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c0ae720 con 0x7f6d6c0c7940 2023-10-06T05:35:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.764+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d5c010c60 con 0x7f6d6c0c7940 2023-10-06T05:35:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.765+0000 7f6d61ffb700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.765+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6d5c083bb0 con 0x7f6d6c0c7940 2023-10-06T05:35:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.765+0000 7f6d71b9b700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.766+0000 7f6d71b9b700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d50066f30 0x7f6d500693e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6d680091e0 tx=0x7f6d68017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:06.769+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d5c050e70 con 0x7f6d6c0c7940 2023-10-06T05:35:07.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.051+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_trim_caps"]} v 0) v1 -- 0x7f6d6c04c380 con 0x7f6d6c0c7940 2023-10-06T05:35:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.110+0000 7f6d61ffb700 1 -- 172.21.15.33:0/4078563537 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_trim_caps"]}]=0 v1009) v1 ==== 146+0+0 (secure 0 0 0) 0x7f6d5c017070 con 0x7f6d6c0c7940 2023-10-06T05:35:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d50066f30 msgr2=0x7f6d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d50066f30 0x7f6d500693e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6d680091e0 tx=0x7f6d68017000 comp rx=0 tx=0).stop 2023-10-06T05:35:07.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 msgr2=0x7f6d6c16ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 0x7f6d6c16ac20 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c7c0 tx=0x7f6d5c007dc0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 shutdown_connections 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c13faa0 0x7f6d6c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6d6c06aad0 0x7f6d6c0a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 --2- 172.21.15.33:0/4078563537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d6c0c7940 0x7f6d6c16ac20 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 >> 172.21.15.33:0/4078563537 conn(0x7f6d6c06d030 msgr2=0x7f6d6c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 shutdown_connections 2023-10-06T05:35:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.112+0000 7f6d72b9d700 1 -- 172.21.15.33:0/4078563537 wait complete. 2023-10-06T05:35:07.131 INFO:tasks.cephfs_test_runner:test_trim_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) ... ok 2023-10-06T05:35:07.132 INFO:tasks.cephfs_test_runner:Starting test: test_unmount_for_evicted_client (tasks.cephfs.test_client_recovery.TestClientRecovery) Test if client hangs on unmount after evicting the client. 2023-10-06T05:35:07.132 INFO:tasks.cephfs_test_runner:test_unmount_for_evicted_client (tasks.cephfs.test_client_recovery.TestClientRecovery) 2023-10-06T05:35:07.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_unmount_for_evicted_client' 2023-10-06T05:35:07.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.322+0000 7f3cdf59e700 1 Processor -- start 2023-10-06T05:35:07.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.322+0000 7f3cdf59e700 1 -- start start 2023-10-06T05:35:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.323+0000 7f3cdf59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ce0155140 0x7f3ce0158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.323+0000 7f3cdf59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00c8e40 con 0x7f3ce00699c0 2023-10-06T05:35:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.323+0000 7f3cdf59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00c99e0 con 0x7f3ce00adf50 2023-10-06T05:35:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.323+0000 7f3cdf59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00ae3f0 con 0x7f3ce0155140 2023-10-06T05:35:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ce0155140 0x7f3ce0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:07.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3ce0155140 0x7f3ce0158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49330/0 (socket says 172.21.15.33:49330) 2023-10-06T05:35:07.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 -- 172.21.15.33:0/2948152232 learned_addr learned my addr 172.21.15.33:0/2948152232 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 -- 172.21.15.33:0/2948152232 >> v1:172.21.15.167:6789/0 conn(0x7f3ce00adf50 legacy=0x7f3ce00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 -- 172.21.15.33:0/2948152232 >> v1:172.21.15.33:6789/0 conn(0x7f3ce00699c0 legacy=0x7f3ce00c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 -- 172.21.15.33:0/2948152232 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00ae6c0 con 0x7f3ce0155140 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.324+0000 7f3cde59c700 1 --2- 172.21.15.33:0/2948152232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce0155140 0x7f3ce0158700 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00099e0 tx=0x7f3cd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b821c9c6841aeca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdd59a700 1 -- 172.21.15.33:0/2948152232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000b820 con 0x7f3ce0155140 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdd59a700 1 -- 172.21.15.33:0/2948152232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd000b980 con 0x7f3ce0155140 2023-10-06T05:35:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdd59a700 1 -- 172.21.15.33:0/2948152232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000bd20 con 0x7f3ce0155140 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 -- 172.21.15.33:0/2948152232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce0155140 msgr2=0x7f3ce0158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/2948152232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce0155140 0x7f3ce0158700 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00099e0 tx=0x7f3cd0009250 comp rx=0 tx=0).stop 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 -- 172.21.15.33:0/2948152232 shutdown_connections 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/2948152232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce0155140 0x7f3ce0158700 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 -- 172.21.15.33:0/2948152232 >> 172.21.15.33:0/2948152232 conn(0x7f3ce006d030 msgr2=0x7f3ce006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 -- 172.21.15.33:0/2948152232 shutdown_connections 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.325+0000 7f3cdf59e700 1 -- 172.21.15.33:0/2948152232 wait complete. 2023-10-06T05:35:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.327+0000 7f3cdf59e700 1 Processor -- start 2023-10-06T05:35:07.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.327+0000 7f3cdf59e700 1 -- start start 2023-10-06T05:35:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cdf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 0x7f3ce00c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cdf59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ce00adf50 0x7f3ce00c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cdf59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce0155140 0x7f3ce00c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cdf59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0143550 con 0x7f3ce00699c0 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cdf59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce014f9b0 con 0x7f3ce0155140 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cdf59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00c5fb0 con 0x7f3ce00adf50 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 0x7f3ce00c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 0x7f3ce00c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49346/0 (socket says 172.21.15.33:49346) 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 -- 172.21.15.33:0/3253134682 learned_addr learned my addr 172.21.15.33:0/3253134682 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cddd9b700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ce00adf50 0x7f3ce00c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:07.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cded9d700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce0155140 0x7f3ce00c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 -- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ce00adf50 msgr2=0x7f3ce00c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ce00adf50 0x7f3ce00c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 -- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce0155140 msgr2=0x7f3ce00c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.328+0000 7f3cde59c700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce0155140 0x7f3ce00c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cde59c700 1 -- 172.21.15.33:0/3253134682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd00094f0 con 0x7f3ce00699c0 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cddd9b700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ce00adf50 0x7f3ce00c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cded9d700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce0155140 0x7f3ce00c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cde59c700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 0x7f3ce00c4890 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00094c0 tx=0x7f3cd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000bcc0 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd001a630 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd001a9f0 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce0158300 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.329+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce01588c0 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.330+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce004c380 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.331+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cd002e450 con 0x7f3ce00699c0 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.332+0000 7f3cce7fc700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cc0066f30 0x7f3cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.332+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3cd0085f90 con 0x7f3ce00699c0 2023-10-06T05:35:07.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.333+0000 7f3cddd9b700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cc0066f30 0x7f3cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:07.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.333+0000 7f3cddd9b700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cc0066f30 0x7f3cc00693e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4007fd0 tx=0x7f3cd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:07.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.336+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cd0053250 con 0x7f3ce00699c0 2023-10-06T05:35:07.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:07.618+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_unmount_for_evicted_client"]} v 0) v1 -- 0x7f3ce006fa40 con 0x7f3ce00699c0 2023-10-06T05:35:08.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.112+0000 7f3cce7fc700 1 -- 172.21.15.33:0/3253134682 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientRecovery.test_unmount_for_evicted_client"]}]=0 v1010) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3cd00150e0 con 0x7f3ce00699c0 2023-10-06T05:35:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cc0066f30 msgr2=0x7f3cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cc0066f30 0x7f3cc00693e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4007fd0 tx=0x7f3cd4009040 comp rx=0 tx=0).stop 2023-10-06T05:35:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 msgr2=0x7f3ce00c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 0x7f3ce00c4890 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00094c0 tx=0x7f3cd0017000 comp rx=0 tx=0).stop 2023-10-06T05:35:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 shutdown_connections 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3cc0066f30 0x7f3cc00693e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ce0155140 0x7f3ce00c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce00699c0 0x7f3ce00c4890 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 --2- 172.21.15.33:0/3253134682 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3ce00adf50 0x7f3ce00c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 >> 172.21.15.33:0/3253134682 conn(0x7f3ce006d030 msgr2=0x7f3ce006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 shutdown_connections 2023-10-06T05:35:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.115+0000 7f3cdf59e700 1 -- 172.21.15.33:0/3253134682 wait complete. 2023-10-06T05:35:08.137 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:35:08.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.329+0000 7f66f359e700 1 Processor -- start 2023-10-06T05:35:08.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.329+0000 7f66f359e700 1 -- start start 2023-10-06T05:35:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.330+0000 7f66f359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66f40cd5e0 0x7f66f40af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.330+0000 7f66f359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40ce890 con 0x7f66f415a650 2023-10-06T05:35:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.330+0000 7f66f359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f415aa60 con 0x7f66f40af540 2023-10-06T05:35:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.330+0000 7f66f359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f415d890 con 0x7f66f40cd5e0 2023-10-06T05:35:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.330+0000 7f66f1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66f40cd5e0 0x7f66f40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66f40cd5e0 0x7f66f40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49350/0 (socket says 172.21.15.33:49350) 2023-10-06T05:35:08.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f1d9b700 1 -- 172.21.15.33:0/92161597 learned_addr learned my addr 172.21.15.33:0/92161597 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:08.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f1d9b700 1 -- 172.21.15.33:0/92161597 >> v1:172.21.15.167:6789/0 conn(0x7f66f40af540 legacy=0x7f66f40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f1d9b700 1 -- 172.21.15.33:0/92161597 >> v1:172.21.15.33:6789/0 conn(0x7f66f415a650 legacy=0x7f66f40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f1d9b700 1 -- 172.21.15.33:0/92161597 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f415db70 con 0x7f66f40cd5e0 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f1d9b700 1 --2- 172.21.15.33:0/92161597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f40af910 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f66e80099b0 tx=0x7f66e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794468158f0adb59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.331+0000 7f66f159a700 1 -- 172.21.15.33:0/92161597 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800b860 con 0x7f66f40cd5e0 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f159a700 1 -- 172.21.15.33:0/92161597 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e800b9c0 con 0x7f66f40cd5e0 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f159a700 1 -- 172.21.15.33:0/92161597 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800bd60 con 0x7f66f40cd5e0 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 -- 172.21.15.33:0/92161597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 msgr2=0x7f66f40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 --2- 172.21.15.33:0/92161597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f40af910 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f66e80099b0 tx=0x7f66e8009250 comp rx=0 tx=0).stop 2023-10-06T05:35:08.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 -- 172.21.15.33:0/92161597 shutdown_connections 2023-10-06T05:35:08.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 --2- 172.21.15.33:0/92161597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f40af910 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 -- 172.21.15.33:0/92161597 >> 172.21.15.33:0/92161597 conn(0x7f66f406d9b0 msgr2=0x7f66f406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:08.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 -- 172.21.15.33:0/92161597 shutdown_connections 2023-10-06T05:35:08.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.332+0000 7f66f359e700 1 -- 172.21.15.33:0/92161597 wait complete. 2023-10-06T05:35:08.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.334+0000 7f66f359e700 1 Processor -- start 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.334+0000 7f66f359e700 1 -- start start 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66f40af540 0x7f66f40b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f416cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66f415a650 0x7f66f40b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40c8550 con 0x7f66f40cd5e0 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40c87a0 con 0x7f66f415a650 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40c8a80 con 0x7f66f40af540 2023-10-06T05:35:08.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f416cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f416cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49356/0 (socket says 172.21.15.33:49356) 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f1d9b700 1 -- 172.21.15.33:0/3604146376 learned_addr learned my addr 172.21.15.33:0/3604146376 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f259c700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66f40af540 0x7f66f40b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.335+0000 7f66f2d9d700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66f415a650 0x7f66f40b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f1d9b700 1 -- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66f40af540 msgr2=0x7f66f40b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f1d9b700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66f40af540 0x7f66f40b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f1d9b700 1 -- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66f415a650 msgr2=0x7f66f40b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f1d9b700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66f415a650 0x7f66f40b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f1d9b700 1 -- 172.21.15.33:0/3604146376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80094f0 con 0x7f66f40cd5e0 2023-10-06T05:35:08.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f259c700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66f40af540 0x7f66f40b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f1d9b700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f416cc30 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f66f40c7320 tx=0x7f66e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8011ea0 con 0x7f66f40cd5e0 2023-10-06T05:35:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e801a630 con 0x7f66f40cd5e0 2023-10-06T05:35:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e801a9b0 con 0x7f66f40cd5e0 2023-10-06T05:35:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f40c8d60 con 0x7f66f40cd5e0 2023-10-06T05:35:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.336+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f416d170 con 0x7f66f40cd5e0 2023-10-06T05:35:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.337+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f40c9280 con 0x7f66f40cd5e0 2023-10-06T05:35:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.338+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66e802d440 con 0x7f66f40cd5e0 2023-10-06T05:35:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.340+0000 7f66e27fc700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66d4066f30 0x7f66d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.340+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f66e8086130 con 0x7f66f40cd5e0 2023-10-06T05:35:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.341+0000 7f66f259c700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66d4066f30 0x7f66d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.341+0000 7f66f259c700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66d4066f30 0x7f66d40693e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f66e4007fd0 tx=0x7f66e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:08.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.344+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66f40c9280 con 0x7f66f40cd5e0 2023-10-06T05:35:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.546+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f66f40c9280 con 0x7f66f40cd5e0 2023-10-06T05:35:08.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.552+0000 7f66e27fc700 1 -- 172.21.15.33:0/3604146376 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f66f40c9280 con 0x7f66f40cd5e0 2023-10-06T05:35:08.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66d4066f30 msgr2=0x7f66d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66d4066f30 0x7f66d40693e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f66e4007fd0 tx=0x7f66e4009040 comp rx=0 tx=0).stop 2023-10-06T05:35:08.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 msgr2=0x7f66f416cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f416cc30 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f66f40c7320 tx=0x7f66e8017000 comp rx=0 tx=0).stop 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 shutdown_connections 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f66d4066f30 0x7f66d40693e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66f415a650 0x7f66f40b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66f40af540 0x7f66f40b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 --2- 172.21.15.33:0/3604146376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f40cd5e0 0x7f66f416cc30 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 >> 172.21.15.33:0/3604146376 conn(0x7f66f406d9b0 msgr2=0x7f66f40c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 shutdown_connections 2023-10-06T05:35:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.555+0000 7f66f359e700 1 -- 172.21.15.33:0/3604146376 wait complete. 2023-10-06T05:35:08.575 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:35:08.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:35:08.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:35:08.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:35:08.744 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:35:08.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.929+0000 7f9faffff700 1 Processor -- start 2023-10-06T05:35:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.929+0000 7f9faffff700 1 -- start start 2023-10-06T05:35:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.930+0000 7f9faffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9fb00afdd0 0x7f9fb00b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.930+0000 7f9faffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb006f220 con 0x7f9fb006e140 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.930+0000 7f9faffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb006e610 con 0x7f9fb0159530 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.930+0000 7f9faffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00b1e00 con 0x7f9fb00afdd0 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fae7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9fb006e140 0x7f9fb015a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59726/0 (socket says 172.21.15.33:59726) 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fae7fc700 1 -- 172.21.15.33:0/917659228 learned_addr learned my addr 172.21.15.33:0/917659228 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9faf7fe700 1 --2- 172.21.15.33:0/917659228 >> v2:172.21.15.167:3300/0 conn(0x7f9fb00afdd0 0x7f9fb00b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125720550 0 0) 0x7f9fb006f220 con 0x7f9fb006e140 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f98003570 con 0x7f9fb006e140 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929294115 0 0) 0x7f9fb006e610 con 0x7f9fb0159530 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb006f220 con 0x7f9fb0159530 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2488741698 0 0) 0x7f9f98003570 con 0x7f9fb006e140 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 >> v2:172.21.15.167:3300/0 conn(0x7f9fb00afdd0 msgr2=0x7f9fb00b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 --2- 172.21.15.33:0/917659228 >> v2:172.21.15.167:3300/0 conn(0x7f9fb00afdd0 0x7f9fb00b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 >> v1:172.21.15.167:6789/0 conn(0x7f9fb0159530 legacy=0x7f9fb006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb006e8f0 con 0x7f9fb006e140 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9fa4001470 con 0x7f9fb006e140 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.931+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 >> v1:172.21.15.33:6789/0 conn(0x7f9fb006e140 legacy=0x7f9fb015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fadffb700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9f98004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fadffb700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98004810 0x7f9f98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fadffb700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f98005210 0x7f9f9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00b1e00 con 0x7f9f98003c40 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9800c130 con 0x7f9f98004810 2023-10-06T05:35:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9800c2c0 con 0x7f9f98005210 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faeffd700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9f98004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9fae7fc700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98004810 0x7f9f98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faf7fe700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f98005210 0x7f9f9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faeffd700 1 -- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f98005210 msgr2=0x7f9f9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faeffd700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f98005210 0x7f9f9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faeffd700 1 -- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98004810 msgr2=0x7f9f98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faeffd700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98004810 0x7f9f98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faeffd700 1 -- 172.21.15.33:0/917659228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98003570 con 0x7f9f98003c40 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.932+0000 7f9faf7fe700 1 -- 172.21.15.33:0/917659228 reap_dead start 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faeffd700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9f98004030 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0001a40 tx=0x7f9fa0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3079c28451bcbcd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000c590 con 0x7f9f98003c40 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa000c6f0 con 0x7f9f98003c40 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9fadffb700 1 -- 172.21.15.33:0/917659228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000cab0 con 0x7f9f98003c40 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faffff700 1 -- 172.21.15.33:0/917659228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 msgr2=0x7f9f98004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faffff700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9f98004030 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0001a40 tx=0x7f9fa0007d30 comp rx=0 tx=0).stop 2023-10-06T05:35:08.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faffff700 1 -- 172.21.15.33:0/917659228 shutdown_connections 2023-10-06T05:35:08.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faffff700 1 --2- 172.21.15.33:0/917659228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9f98004030 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faffff700 1 -- 172.21.15.33:0/917659228 >> 172.21.15.33:0/917659228 conn(0x7f9fb006c560 msgr2=0x7f9fb006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:08.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.933+0000 7f9faffff700 1 -- 172.21.15.33:0/917659228 shutdown_connections 2023-10-06T05:35:08.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.934+0000 7f9faffff700 1 -- 172.21.15.33:0/917659228 wait complete. 2023-10-06T05:35:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.935+0000 7f9faffff700 1 Processor -- start 2023-10-06T05:35:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.935+0000 7f9faffff700 1 -- start start 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9fb0152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb0139c90 0x7f9fb013a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb0154090 0x7f9fb01544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0155570 con 0x7f9f98003c40 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0150120 con 0x7f9fb0154090 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb013a640 con 0x7f9fb0139c90 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9fb0152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9fb0152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49374/0 (socket says 172.21.15.33:49374) 2023-10-06T05:35:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faeffd700 1 -- 172.21.15.33:0/4278729727 learned_addr learned my addr 172.21.15.33:0/4278729727 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9fae7fc700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb0139c90 0x7f9fb013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.936+0000 7f9faf7fe700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb0154090 0x7f9fb01544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faeffd700 1 -- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb0139c90 msgr2=0x7f9fb013a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faeffd700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb0139c90 0x7f9fb013a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faeffd700 1 -- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb0154090 msgr2=0x7f9fb01544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faeffd700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb0154090 0x7f9fb01544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faeffd700 1 -- 172.21.15.33:0/4278729727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0154fb0 con 0x7f9f98003c40 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faf7fe700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb0154090 0x7f9fb01544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9fae7fc700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb0139c90 0x7f9fb013a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.937+0000 7f9faeffd700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9fb0152300 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0003000 tx=0x7f9fa0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.939+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000c860 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.939+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa000c9c0 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.939+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0006a30 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.939+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb0155290 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.939+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb0152840 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.940+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb014fc10 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.941+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fa0019070 con 0x7f9f98003c40 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.942+0000 7f9f96ffd700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f9c066f60 0x7f9f9c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.942+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9fa0085dc0 con 0x7f9f98003c40 2023-10-06T05:35:08.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.942+0000 7f9fae7fc700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f9c066f60 0x7f9f9c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:08.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.943+0000 7f9fae7fc700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f9c066f60 0x7f9f9c069410 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4001010 tx=0x7f9fa4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:08.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:08.946+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fb014fc10 con 0x7f9f98003c40 2023-10-06T05:35:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.193+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fb014fc10 con 0x7f9f98003c40 2023-10-06T05:35:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.194+0000 7f9f96ffd700 1 -- 172.21.15.33:0/4278729727 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 212 v212) v1 ==== 95+0+3542 (secure 0 0 0) 0x7f9fb014fc10 con 0x7f9f98003c40 2023-10-06T05:35:09.195 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:09.196 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":212,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1363449112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1363449112},{"type":"v1","addr":"172.21.15.167:6837","nonce":1363449112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":9158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2002308640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2002308640},{"type":"v1","addr":"172.21.15.167:6835","nonce":2002308640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":212},{"gid":12072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/3413714968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3413714968},{"type":"v1","addr":"172.21.15.33:6834","nonce":3413714968}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208},{"gid":12075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3615476646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3615476646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3615476646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":208}],"filesystems":[]} 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f9c066f60 msgr2=0x7f9f9c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f9c066f60 0x7f9f9c069410 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4001010 tx=0x7f9fa4009040 comp rx=0 tx=0).stop 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 msgr2=0x7f9fb0152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9fb0152300 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0003000 tx=0x7f9fa0015000 comp rx=0 tx=0).stop 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 shutdown_connections 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f9c066f60 0x7f9f9c069410 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fb0154090 0x7f9fb01544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f98003c40 0x7f9fb0152300 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 --2- 172.21.15.33:0/4278729727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9fb0139c90 0x7f9fb013a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 >> 172.21.15.33:0/4278729727 conn(0x7f9fb006c560 msgr2=0x7f9fb0159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:09.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 shutdown_connections 2023-10-06T05:35:09.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.196+0000 7f9faffff700 1 -- 172.21.15.33:0/4278729727 wait complete. 2023-10-06T05:35:09.199 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 212 2023-10-06T05:35:09.215 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:35:09.215 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:35:09.216 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:35:09.216 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:35:09.216 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:35:09.216 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:35:09.216 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:35:09.216 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:35:09.216 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:35:09.217 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:35:09.217 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:35:09.217 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:35:09.217 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:35:09.217 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:35:09.217 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:35:09.217 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:35:09.229 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:35:09.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:35:09.230 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:35:09.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:35:09.232 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:35:09.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:35:09.233 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:35:09.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:35:09.274 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:35:09.274 DEBUG:teuthology.parallel:result is None 2023-10-06T05:35:09.277 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:35:09.277 DEBUG:teuthology.parallel:result is None 2023-10-06T05:35:09.278 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:35:09.278 DEBUG:teuthology.parallel:result is None 2023-10-06T05:35:09.280 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:35:09.280 DEBUG:teuthology.parallel:result is None 2023-10-06T05:35:09.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:35:09.411 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T05:35:09.412 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T05:35:09.420 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:35:09.421 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T05:35:09.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.476+0000 7f8321d9d700 1 Processor -- start 2023-10-06T05:35:09.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.476+0000 7f8321d9d700 1 -- start start 2023-10-06T05:35:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f8321d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f831c0c93e0 0x7f831c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f8321d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c06a360 con 0x7f831c140d20 2023-10-06T05:35:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f8321d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c15bcb0 con 0x7f831c069f00 2023-10-06T05:35:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f8321d9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c1410f0 con 0x7f831c0c93e0 2023-10-06T05:35:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f8320d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f831c140d20 0x7f831c0c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59746/0 (socket says 172.21.15.33:59746) 2023-10-06T05:35:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f8320d9b700 1 -- 172.21.15.33:0/3259370379 learned_addr learned my addr 172.21.15.33:0/3259370379 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f831bfff700 1 --2- 172.21.15.33:0/3259370379 >> v2:172.21.15.167:3300/0 conn(0x7f831c0c93e0 0x7f831c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171187952 0 0) 0x7f8310001110 con 0x7f831c140d20 2023-10-06T05:35:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.478+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c003570 con 0x7f831c140d20 2023-10-06T05:35:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218394898 0 0) 0x7f831c15bcb0 con 0x7f831c069f00 2023-10-06T05:35:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8310001110 con 0x7f831c069f00 2023-10-06T05:35:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831bfff700 1 -- 172.21.15.33:0/3259370379 >> v1:172.21.15.167:6789/0 conn(0x7f831c069f00 legacy=0x7f831c0c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831bfff700 1 -- 172.21.15.33:0/3259370379 >> v1:172.21.15.33:6789/0 conn(0x7f831c140d20 legacy=0x7f831c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831bfff700 1 -- 172.21.15.33:0/3259370379 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c15b0a0 con 0x7f831c0c93e0 2023-10-06T05:35:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831bfff700 1 --2- 172.21.15.33:0/3259370379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c0c93e0 0x7f831c0699c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f83080098e0 tx=0x7f83080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc62e1618e14570 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:09.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308011630 con 0x7f831c0c93e0 2023-10-06T05:35:09.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308011790 con 0x7f831c0c93e0 2023-10-06T05:35:09.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.479+0000 7f831b7fe700 1 -- 172.21.15.33:0/3259370379 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308011b30 con 0x7f831c0c93e0 2023-10-06T05:35:09.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 -- 172.21.15.33:0/3259370379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c0c93e0 msgr2=0x7f831c0699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 --2- 172.21.15.33:0/3259370379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c0c93e0 0x7f831c0699c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f83080098e0 tx=0x7f83080091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 -- 172.21.15.33:0/3259370379 shutdown_connections 2023-10-06T05:35:09.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 --2- 172.21.15.33:0/3259370379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c0c93e0 0x7f831c0699c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 -- 172.21.15.33:0/3259370379 >> 172.21.15.33:0/3259370379 conn(0x7f831c06c560 msgr2=0x7f831c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:09.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 -- 172.21.15.33:0/3259370379 shutdown_connections 2023-10-06T05:35:09.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.480+0000 7f8321d9d700 1 -- 172.21.15.33:0/3259370379 wait complete. 2023-10-06T05:35:09.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.481+0000 7f8321d9d700 1 Processor -- start 2023-10-06T05:35:09.488 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.481+0000 7f8321d9d700 1 -- start start 2023-10-06T05:35:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f8321d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c069f00 0x7f831c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f8321d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 0x7f831c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f8321d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f831c140d20 0x7f831c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f8321d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c4300 con 0x7f831c0c93e0 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f8321d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c36f0 con 0x7f831c069f00 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f8321d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c0c39d0 con 0x7f831c140d20 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f831bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 0x7f831c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f831bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 0x7f831c156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49436/0 (socket says 172.21.15.33:49436) 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.482+0000 7f831bfff700 1 -- 172.21.15.33:0/4273367044 learned_addr learned my addr 172.21.15.33:0/4273367044 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f8320d9b700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c069f00 0x7f831c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f832159c700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f831c140d20 0x7f831c159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f831bfff700 1 -- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f831c140d20 msgr2=0x7f831c159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f831bfff700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f831c140d20 0x7f831c159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f831bfff700 1 -- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c069f00 msgr2=0x7f831c0a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f831bfff700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c069f00 0x7f831c0a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f831bfff700 1 -- 172.21.15.33:0/4273367044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308009420 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f8320d9b700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c069f00 0x7f831c0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f831bfff700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 0x7f831c156c50 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f8308013000 tx=0x7f8308017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.483+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830801f070 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.484+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308000c90 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.484+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c0c3cb0 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.484+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308025420 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.484+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c0a5a30 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.485+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c0c4e40 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.485+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8308023070 con 0x7f831c0c93e0 2023-10-06T05:35:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.486+0000 7f8318ff9700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8300066f30 0x7f83000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.486+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f83080858d0 con 0x7f831c0c93e0 2023-10-06T05:35:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.487+0000 7f8320d9b700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8300066f30 0x7f83000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.488+0000 7f8320d9b700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8300066f30 0x7f83000693e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f831c0cca80 tx=0x7f8310009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.490+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f831c0c4e40 con 0x7f831c0c93e0 2023-10-06T05:35:09.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:09.712+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f831c0c4e40 con 0x7f831c0c93e0 2023-10-06T05:35:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.138+0000 7f8318ff9700 1 -- 172.21.15.33:0/4273367044 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v276) v1 ==== 97+0+0 (secure 0 0 0) 0x7f831c0c4e40 con 0x7f831c0c93e0 2023-10-06T05:35:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8300066f30 msgr2=0x7f83000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8300066f30 0x7f83000693e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f831c0cca80 tx=0x7f8310009040 comp rx=0 tx=0).stop 2023-10-06T05:35:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 msgr2=0x7f831c156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 0x7f831c156c50 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f8308013000 tx=0x7f8308017000 comp rx=0 tx=0).stop 2023-10-06T05:35:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 shutdown_connections 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8300066f30 0x7f83000693e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f831c140d20 0x7f831c159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f831c069f00 0x7f831c0a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 --2- 172.21.15.33:0/4273367044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c0c93e0 0x7f831c156c50 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 >> 172.21.15.33:0/4273367044 conn(0x7f831c06c560 msgr2=0x7f831c13fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 shutdown_connections 2023-10-06T05:35:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.141+0000 7f8321d9d700 1 -- 172.21.15.33:0/4273367044 wait complete. 2023-10-06T05:35:10.145 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T05:35:10.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:35:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.352+0000 7f4beffff700 1 Processor -- start 2023-10-06T05:35:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.352+0000 7f4beffff700 1 -- start start 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.353+0000 7f4beffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf01573c0 0x7f4bf015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4beffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4bf0157790 0x7f4bf00af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4beffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4bf00af600 0x7f4bf006c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4beffff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf015a8b0 con 0x7f4bf0157790 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4beffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00afb40 con 0x7f4bf00af600 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4beffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf015a260 con 0x7f4bf01573c0 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4bf0157790 0x7f4bf00af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4bf0157790 0x7f4bf00af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59454/0 (socket says 172.21.15.33:59454) 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 -- 172.21.15.33:0/1817678496 learned_addr learned my addr 172.21.15.33:0/1817678496 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bef7fe700 1 --2- 172.21.15.33:0/1817678496 >> v2:172.21.15.167:3300/0 conn(0x7f4bf00af600 0x7f4bf006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4beeffd700 1 --2- 172.21.15.33:0/1817678496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf01573c0 0x7f4bf015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 -- 172.21.15.33:0/1817678496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf01573c0 msgr2=0x7f4bf015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 --2- 172.21.15.33:0/1817678496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf01573c0 0x7f4bf015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 -- 172.21.15.33:0/1817678496 >> v2:172.21.15.167:3300/0 conn(0x7f4bf00af600 msgr2=0x7f4bf006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 --2- 172.21.15.33:0/1817678496 >> v2:172.21.15.167:3300/0 conn(0x7f4bf00af600 0x7f4bf006c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.354+0000 7f4bee7fc700 1 -- 172.21.15.33:0/1817678496 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf015a540 con 0x7f4bf0157790 2023-10-06T05:35:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.355+0000 7f4bee7fc700 1 --2- 172.21.15.33:0/1817678496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0157790 0x7f4bf00af0c0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f4bf006d470 tx=0x7f4be4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99a1a85a2215a5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:10.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.355+0000 7f4bedffb700 1 -- 172.21.15.33:0/1817678496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be400b860 con 0x7f4bf0157790 2023-10-06T05:35:10.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.355+0000 7f4bedffb700 1 -- 172.21.15.33:0/1817678496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be400b9c0 con 0x7f4bf0157790 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.355+0000 7f4bedffb700 1 -- 172.21.15.33:0/1817678496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be400bd60 con 0x7f4bf0157790 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.355+0000 7f4beffff700 1 -- 172.21.15.33:0/1817678496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0157790 msgr2=0x7f4bf00af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.355+0000 7f4beffff700 1 --2- 172.21.15.33:0/1817678496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0157790 0x7f4bf00af0c0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f4bf006d470 tx=0x7f4be4009250 comp rx=0 tx=0).stop 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 -- 172.21.15.33:0/1817678496 shutdown_connections 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 --2- 172.21.15.33:0/1817678496 >> v2:172.21.15.167:3300/0 conn(0x7f4bf00af600 0x7f4bf006c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 --2- 172.21.15.33:0/1817678496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf01573c0 0x7f4bf015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 --2- 172.21.15.33:0/1817678496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0157790 0x7f4bf00af0c0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 -- 172.21.15.33:0/1817678496 >> 172.21.15.33:0/1817678496 conn(0x7f4bf006e430 msgr2=0x7f4bf00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 -- 172.21.15.33:0/1817678496 shutdown_connections 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.356+0000 7f4beffff700 1 -- 172.21.15.33:0/1817678496 wait complete. 2023-10-06T05:35:10.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.357+0000 7f4beffff700 1 Processor -- start 2023-10-06T05:35:10.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.357+0000 7f4beffff700 1 -- start start 2023-10-06T05:35:10.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 0x7f4bf00675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01573c0 0x7f4bf00691f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf0157790 0x7f4bf0069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00b5080 con 0x7f4bf00af600 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00c8390 con 0x7f4bf01573c0 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf006b810 con 0x7f4bf0157790 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 0x7f4bf00675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 0x7f4bf00675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59456/0 (socket says 172.21.15.33:59456) 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.358+0000 7f4beeffd700 1 -- 172.21.15.33:0/704001328 learned_addr learned my addr 172.21.15.33:0/704001328 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4bee7fc700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01573c0 0x7f4bf00691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4bef7fe700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf0157790 0x7f4bf0069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beeffd700 1 -- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf0157790 msgr2=0x7f4bf0069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beeffd700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf0157790 0x7f4bf0069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beeffd700 1 -- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01573c0 msgr2=0x7f4bf00691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beeffd700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01573c0 0x7f4bf00691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beeffd700 1 -- 172.21.15.33:0/704001328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40094f0 con 0x7f4bf00af600 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4bef7fe700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf0157790 0x7f4bf0069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beeffd700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 0x7f4bf00675f0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f4be000d8c0 tx=0x7f4be0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0018070 con 0x7f4bf00af600 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0005a00 con 0x7f4bf00af600 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0005dc0 con 0x7f4bf00af600 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.359+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf00c7dd0 con 0x7f4bf00af600 2023-10-06T05:35:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.360+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00b4d10 con 0x7f4bf00af600 2023-10-06T05:35:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.361+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf00644e0 con 0x7f4bf00af600 2023-10-06T05:35:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.361+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4be0012800 con 0x7f4bf00af600 2023-10-06T05:35:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.363+0000 7f4bd6ffd700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4bdc066f30 0x7f4bdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.363+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4be0084500 con 0x7f4bf00af600 2023-10-06T05:35:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.363+0000 7f4bee7fc700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4bdc066f30 0x7f4bdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.364+0000 7f4bee7fc700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4bdc066f30 0x7f4bdc0693e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4bf006d470 tx=0x7f4be4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:10.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.367+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4be00518c0 con 0x7f4bf00af600 2023-10-06T05:35:10.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.646+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4bf004c380 con 0x7f4bf00af600 2023-10-06T05:35:10.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.647+0000 7f4bd6ffd700 1 -- 172.21.15.33:0/704001328 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4be0054ee0 con 0x7f4bf00af600 2023-10-06T05:35:10.648 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lF/KYFhAARvt3nOYdGT9/oWKkBmNGTg==", 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lCVXPKxAAOGhHn/7Vca5KTaAxnr1TGA==", 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lEHQ+IRAA+W7OTcjmag3deOb09EQYDg==", 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lGehANhAALvNoGNSW6smA3MNbPd4njQ==", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB2lx9l8ylOIhAAtSyujKRv+N2zLLMsMj6rWQ==", 2023-10-06T05:35:10.652 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB5lx9lCdeNJRAAG/LRqjSbHCVYkGrTK9scfw==", 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.653 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB8lx9lNaFQKRAAlNRArL13XqqUnpqQAsuvtQ==", 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQB/lx9l45hRLBAAh32q5TNcXCsCZUgYf8C5UQ==", 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCGlx9lU2KUAhAA0txcXHlvwo+ozwBCwnDiyg==", 2023-10-06T05:35:10.655 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCJlx9l3gzhBBAAqOck4lmWXIQ+KjuKVr5oiw==", 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCMlx9lmj6hBhAA2UoxEAMct/bE+Jv4K8F6sg==", 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCPlx9ly4nFCBAAJ4KsufSdcl/GLIX3UMAZAw==", 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T05:35:10.658 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lPb6sAxAAF85RYNEW97FBFF3nzMHhSA==", 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow r", 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lzstgERAAkvvySHwreA3CGow32E7zrA==", 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.659 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lC+KWChAAOlp4zRvO26jPcrSuxlMY5A==", 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:35:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBxlx9lDUEkGBAASEuSzqs/g1r12Xub6Rs1dA==", 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T05:35:10.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBvlx9l6MFmIBAARzEtoOEaMUwFdYZOzBESJQ==", 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9lWHaHDxAAg3fOHQ3zTOAvnVeojmeNkQ==", 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T05:35:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQBwlx9l+tJrCBAAVKsnZg2o6tPsYwr65+20Sg==", 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T05:35:10.664 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:10.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4bdc066f30 msgr2=0x7f4bdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4bdc066f30 0x7f4bdc0693e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4bf006d470 tx=0x7f4be4017000 comp rx=0 tx=0).stop 2023-10-06T05:35:10.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 msgr2=0x7f4bf00675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 0x7f4bf00675f0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f4be000d8c0 tx=0x7f4be0008040 comp rx=0 tx=0).stop 2023-10-06T05:35:10.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 shutdown_connections 2023-10-06T05:35:10.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4bdc066f30 0x7f4bdc0693e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bf0157790 0x7f4bf0069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00af600 0x7f4bf00675f0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 --2- 172.21.15.33:0/704001328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01573c0 0x7f4bf00691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 >> 172.21.15.33:0/704001328 conn(0x7f4bf006e430 msgr2=0x7f4bf00c6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 shutdown_connections 2023-10-06T05:35:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.649+0000 7f4beffff700 1 -- 172.21.15.33:0/704001328 wait complete. 2023-10-06T05:35:10.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.854+0000 7f572579d700 1 Processor -- start 2023-10-06T05:35:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.855+0000 7f572579d700 1 -- start start 2023-10-06T05:35:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f572579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f57200adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f572579d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57200699c0 0x7f5720069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f572579d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57200ae490 con 0x7f5720155140 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f572579d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572006a380 con 0x7f57200699c0 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f572579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c8e40 con 0x7f5720158700 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f5724f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57200699c0 0x7f5720069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f5724f9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57200699c0 0x7f5720069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59470/0 (socket says 172.21.15.33:59470) 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f5724f9c700 1 -- 172.21.15.33:0/1280998583 learned_addr learned my addr 172.21.15.33:0/1280998583 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.856+0000 7f571f7fe700 1 --2- 172.21.15.33:0/1280998583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f57200adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f5724f9c700 1 -- 172.21.15.33:0/1280998583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 msgr2=0x7f57200adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f5724f9c700 1 --2- 172.21.15.33:0/1280998583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f57200adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f5724f9c700 1 -- 172.21.15.33:0/1280998583 >> v1:172.21.15.167:6789/0 conn(0x7f5720155140 legacy=0x7f57200c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f5724f9c700 1 -- 172.21.15.33:0/1280998583 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720158b70 con 0x7f57200699c0 2023-10-06T05:35:10.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f5724f9c700 1 --2- 172.21.15.33:0/1280998583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57200699c0 0x7f5720069e40 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f5718009f20 tx=0x7f5718009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d56a829331bf35f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f571effd700 1 -- 172.21.15.33:0/1280998583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800e070 con 0x7f57200699c0 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f571effd700 1 -- 172.21.15.33:0/1280998583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571800ba80 con 0x7f57200699c0 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.857+0000 7f571effd700 1 -- 172.21.15.33:0/1280998583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57180092e0 con 0x7f57200699c0 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 -- 172.21.15.33:0/1280998583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57200699c0 msgr2=0x7f5720069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 --2- 172.21.15.33:0/1280998583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57200699c0 0x7f5720069e40 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f5718009f20 tx=0x7f5718009840 comp rx=0 tx=0).stop 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 -- 172.21.15.33:0/1280998583 shutdown_connections 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 --2- 172.21.15.33:0/1280998583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57200699c0 0x7f5720069e40 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 --2- 172.21.15.33:0/1280998583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f57200adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 -- 172.21.15.33:0/1280998583 >> 172.21.15.33:0/1280998583 conn(0x7f572006d030 msgr2=0x7f572006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:10.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 -- 172.21.15.33:0/1280998583 shutdown_connections 2023-10-06T05:35:10.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.858+0000 7f572579d700 1 -- 172.21.15.33:0/1280998583 wait complete. 2023-10-06T05:35:10.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.859+0000 7f572579d700 1 Processor -- start 2023-10-06T05:35:10.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.859+0000 7f572579d700 1 -- start start 2023-10-06T05:35:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.860+0000 7f572579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57200699c0 0x7f57201542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.860+0000 7f572579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 0x7f57200c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.860+0000 7f572579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f572015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.860+0000 7f572579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200af350 con 0x7f5720155140 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.860+0000 7f572579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572015a280 con 0x7f57200699c0 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.860+0000 7f572579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572015a560 con 0x7f5720158700 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 0x7f57200c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 0x7f57200c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59476/0 (socket says 172.21.15.33:59476) 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 -- 172.21.15.33:0/3281108859 learned_addr learned my addr 172.21.15.33:0/3281108859 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f5724f9c700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f572015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571ffff700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57200699c0 0x7f57201542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 -- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 msgr2=0x7f572015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f572015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 -- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57200699c0 msgr2=0x7f57201542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57200699c0 0x7f57201542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 -- 172.21.15.33:0/3281108859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718009ae0 con 0x7f5720155140 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.861+0000 7f571f7fe700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 0x7f57200c3190 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f571400d890 tx=0x7f5714008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.862+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140125f0 con 0x7f5720155140 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.862+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714012770 con 0x7f5720155140 2023-10-06T05:35:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.862+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714012b30 con 0x7f5720155140 2023-10-06T05:35:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.862+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572015a840 con 0x7f5720155140 2023-10-06T05:35:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.862+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572015ae00 con 0x7f5720155140 2023-10-06T05:35:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.863+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57200adc60 con 0x7f5720155140 2023-10-06T05:35:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.864+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f571401e460 con 0x7f5720155140 2023-10-06T05:35:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.865+0000 7f5707fff700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f570c066f30 0x7f570c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.865+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5714084a40 con 0x7f5720155140 2023-10-06T05:35:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.865+0000 7f571ffff700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f570c066f30 0x7f570c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.866+0000 7f571ffff700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f570c066f30 0x7f570c0693e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5710003860 tx=0x7f5710009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:10.869+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5714086e60 con 0x7f5720155140 2023-10-06T05:35:11.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.072+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f572004c380 con 0x7f5720155140 2023-10-06T05:35:11.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.073+0000 7f5707fff700 1 -- 172.21.15.33:0/3281108859 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v278) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f5714055930 con 0x7f5720155140 2023-10-06T05:35:11.075 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:11.075 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":278,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:10.142896+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.075+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f570c066f30 msgr2=0x7f570c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f570c066f30 0x7f570c0693e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5710003860 tx=0x7f5710009040 comp rx=0 tx=0).stop 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 msgr2=0x7f57200c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 0x7f57200c3190 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f571400d890 tx=0x7f5714008040 comp rx=0 tx=0).stop 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 shutdown_connections 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f570c066f30 0x7f570c0693e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5720158700 0x7f572015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57200699c0 0x7f57201542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 --2- 172.21.15.33:0/3281108859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5720155140 0x7f57200c3190 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 >> 172.21.15.33:0/3281108859 conn(0x7f572006d030 msgr2=0x7f57200c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 shutdown_connections 2023-10-06T05:35:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.076+0000 7f572579d700 1 -- 172.21.15.33:0/3281108859 wait complete. 2023-10-06T05:35:11.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:35:11.096 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:35:11.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.286+0000 7fb4d239d700 1 Processor -- start 2023-10-06T05:35:11.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.286+0000 7fb4d239d700 1 -- start start 2023-10-06T05:35:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d239d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4cc0af540 0x7fb4cc0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc0cd5e0 0x7fb4cc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc15d660 con 0x7fb4cc15a650 2023-10-06T05:35:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d239d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0ce890 con 0x7fb4cc0af540 2023-10-06T05:35:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15aac0 con 0x7fb4cc0cd5e0 2023-10-06T05:35:11.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d139b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4cc0af540 0x7fb4cc0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d0b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb4cc15a650 0x7fb4cc0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38342/0 (socket says 172.21.15.33:38342) 2023-10-06T05:35:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.287+0000 7fb4d0b9a700 1 -- 172.21.15.33:0/2492231527 learned_addr learned my addr 172.21.15.33:0/2492231527 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:11.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4d1b9c700 1 --2- 172.21.15.33:0/2492231527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc0cd5e0 0x7fb4cc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4c3fff700 1 -- 172.21.15.33:0/2492231527 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201544642 0 0) 0x7fb4cc15d660 con 0x7fb4cc15a650 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4c3fff700 1 -- 172.21.15.33:0/2492231527 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b8003570 con 0x7fb4cc15a650 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4d139b700 1 -- 172.21.15.33:0/2492231527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc0cd5e0 msgr2=0x7fb4cc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4d139b700 1 --2- 172.21.15.33:0/2492231527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc0cd5e0 0x7fb4cc15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4d139b700 1 -- 172.21.15.33:0/2492231527 >> v1:172.21.15.33:6789/0 conn(0x7fb4cc15a650 legacy=0x7fb4cc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4d139b700 1 -- 172.21.15.33:0/2492231527 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc15d890 con 0x7fb4cc0af540 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.288+0000 7fb4d139b700 1 --2- 172.21.15.33:0/2492231527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc0af910 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098e0 tx=0x7fb4c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa25774fdae528d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4c3fff700 1 -- 172.21.15.33:0/2492231527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8011630 con 0x7fb4cc0af540 2023-10-06T05:35:11.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4c3fff700 1 -- 172.21.15.33:0/2492231527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c8011790 con 0x7fb4cc0af540 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4c3fff700 1 -- 172.21.15.33:0/2492231527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8011b30 con 0x7fb4cc0af540 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 -- 172.21.15.33:0/2492231527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 msgr2=0x7fb4cc0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 --2- 172.21.15.33:0/2492231527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc0af910 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098e0 tx=0x7fb4c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 -- 172.21.15.33:0/2492231527 shutdown_connections 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 --2- 172.21.15.33:0/2492231527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc0af910 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 --2- 172.21.15.33:0/2492231527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc0cd5e0 0x7fb4cc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 -- 172.21.15.33:0/2492231527 >> 172.21.15.33:0/2492231527 conn(0x7fb4cc06d9b0 msgr2=0x7fb4cc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 -- 172.21.15.33:0/2492231527 shutdown_connections 2023-10-06T05:35:11.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.289+0000 7fb4d239d700 1 -- 172.21.15.33:0/2492231527 wait complete. 2023-10-06T05:35:11.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.291+0000 7fb4d239d700 1 Processor -- start 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.291+0000 7fb4d239d700 1 -- start start 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc16a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4cc0cd5e0 0x7fb4cc0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc15a650 0x7fb4cc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b6130 con 0x7fb4cc0af540 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc16a950 con 0x7fb4cc0cd5e0 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b5210 con 0x7fb4cc15a650 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc16a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc16a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59490/0 (socket says 172.21.15.33:59490) 2023-10-06T05:35:11.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 -- 172.21.15.33:0/1453070513 learned_addr learned my addr 172.21.15.33:0/1453070513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d1b9c700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc15a650 0x7fb4cc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d0b9a700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4cc0cd5e0 0x7fb4cc0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 -- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc15a650 msgr2=0x7fb4cc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc15a650 0x7fb4cc1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 -- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4cc0cd5e0 msgr2=0x7fb4cc0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4cc0cd5e0 0x7fb4cc0b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.292+0000 7fb4d139b700 1 -- 172.21.15.33:0/1453070513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009420 con 0x7fb4cc0af540 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.293+0000 7fb4d139b700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc16a410 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c7320 tx=0x7fb4c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.293+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8011b10 con 0x7fb4cc0af540 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.293+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c80253f0 con 0x7fb4cc0af540 2023-10-06T05:35:11.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.293+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80257b0 con 0x7fb4cc0af540 2023-10-06T05:35:11.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.293+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc0b54b0 con 0x7fb4cc0af540 2023-10-06T05:35:11.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.293+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc0b5a70 con 0x7fb4cc0af540 2023-10-06T05:35:11.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.294+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc04c380 con 0x7fb4cc0af540 2023-10-06T05:35:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.295+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4c8025ba0 con 0x7fb4cc0af540 2023-10-06T05:35:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.296+0000 7fb4c17fa700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.296+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb4c8085c80 con 0x7fb4cc0af540 2023-10-06T05:35:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.296+0000 7fb4d0b9a700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.297+0000 7fb4d0b9a700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0058d0 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.300+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c8053040 con 0x7fb4cc0af540 2023-10-06T05:35:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.502+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4cc15db70 con 0x7fb4cc0af540 2023-10-06T05:35:11.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.503+0000 7fb4c17fa700 1 -- 172.21.15.33:0/1453070513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v278) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4c8056660 con 0x7fb4cc0af540 2023-10-06T05:35:11.505 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 msgr2=0x7fb4b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0058d0 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 msgr2=0x7fb4cc16a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc16a410 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c7320 tx=0x7fb4c8017000 comp rx=0 tx=0).stop 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 shutdown_connections 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb4b0066f30 0x7fb4b00693e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4cc15a650 0x7fb4cc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4cc0af540 0x7fb4cc16a410 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 --2- 172.21.15.33:0/1453070513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4cc0cd5e0 0x7fb4cc0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 >> 172.21.15.33:0/1453070513 conn(0x7fb4cc06d9b0 msgr2=0x7fb4cc159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:11.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 shutdown_connections 2023-10-06T05:35:11.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.505+0000 7fb4d239d700 1 -- 172.21.15.33:0/1453070513 wait complete. 2023-10-06T05:35:11.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:35:11.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.713+0000 7f552e19d700 1 Processor -- start 2023-10-06T05:35:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.713+0000 7f552e19d700 1 -- start start 2023-10-06T05:35:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.714+0000 7f552e19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f55280af540 0x7f55280af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.714+0000 7f552e19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55280ce890 con 0x7f552815a650 2023-10-06T05:35:11.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.714+0000 7f552e19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552815aac0 con 0x7f55280cd5e0 2023-10-06T05:35:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.714+0000 7f552e19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552815d890 con 0x7f55280af540 2023-10-06T05:35:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f552c99a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f552815a650 0x7f55280cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38352/0 (socket says 172.21.15.33:38352) 2023-10-06T05:35:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f552c99a700 1 -- 172.21.15.33:0/3065150237 learned_addr learned my addr 172.21.15.33:0/3065150237 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:11.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f552d19b700 1 --2- 172.21.15.33:0/3065150237 >> v2:172.21.15.167:3300/0 conn(0x7f55280af540 0x7f55280af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962236977 0 0) 0x7f55280ce890 con 0x7f552815a650 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5514003570 con 0x7f552815a650 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317035944 0 0) 0x7f552815aac0 con 0x7f55280cd5e0 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55280ce890 con 0x7f55280cd5e0 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f552d19b700 1 -- 172.21.15.33:0/3065150237 >> v1:172.21.15.167:6789/0 conn(0x7f55280cd5e0 legacy=0x7f55280ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f552d19b700 1 -- 172.21.15.33:0/3065150237 >> v1:172.21.15.33:6789/0 conn(0x7f552815a650 legacy=0x7f55280cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.715+0000 7f552d19b700 1 -- 172.21.15.33:0/3065150237 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552815db70 con 0x7f55280af540 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.716+0000 7f552d19b700 1 --2- 172.21.15.33:0/3065150237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280af540 0x7f55280af910 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f55240098e0 tx=0x7f55240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eec0ac0d9bc9238 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.716+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524011630 con 0x7f55280af540 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.716+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524011790 con 0x7f55280af540 2023-10-06T05:35:11.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.716+0000 7f551ffff700 1 -- 172.21.15.33:0/3065150237 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524011b30 con 0x7f55280af540 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.716+0000 7f552e19d700 1 -- 172.21.15.33:0/3065150237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280af540 msgr2=0x7f55280af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.716+0000 7f552e19d700 1 --2- 172.21.15.33:0/3065150237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280af540 0x7f55280af910 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f55240098e0 tx=0x7f55240091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.717+0000 7f552e19d700 1 -- 172.21.15.33:0/3065150237 shutdown_connections 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.717+0000 7f552e19d700 1 --2- 172.21.15.33:0/3065150237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280af540 0x7f55280af910 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.717+0000 7f552e19d700 1 -- 172.21.15.33:0/3065150237 >> 172.21.15.33:0/3065150237 conn(0x7f552806d9b0 msgr2=0x7f552806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.717+0000 7f552e19d700 1 -- 172.21.15.33:0/3065150237 shutdown_connections 2023-10-06T05:35:11.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.717+0000 7f552e19d700 1 -- 172.21.15.33:0/3065150237 wait complete. 2023-10-06T05:35:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.718+0000 7f552e19d700 1 Processor -- start 2023-10-06T05:35:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.718+0000 7f552e19d700 1 -- start start 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552e19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 0x7f5528169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552e19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280cd5e0 0x7f55280b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552e19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552815a650 0x7f55280c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552e19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280b4370 con 0x7f55280af540 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552e19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280b6300 con 0x7f55280cd5e0 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552e19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280c7ac0 con 0x7f552815a650 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 0x7f5528169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552d19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 0x7f5528169bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59504/0 (socket says 172.21.15.33:59504) 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.719+0000 7f552d19b700 1 -- 172.21.15.33:0/1774021095 learned_addr learned my addr 172.21.15.33:0/1774021095 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d19b700 1 -- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552815a650 msgr2=0x7f55280c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552c99a700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280cd5e0 0x7f55280b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d99c700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552815a650 0x7f55280c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d19b700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552815a650 0x7f55280c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d19b700 1 -- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280cd5e0 msgr2=0x7f55280b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d19b700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280cd5e0 0x7f55280b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d19b700 1 -- 172.21.15.33:0/1774021095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524009420 con 0x7f55280af540 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552c99a700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280cd5e0 0x7f55280b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d99c700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552815a650 0x7f55280c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f552d19b700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 0x7f5528169bf0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f5524009ed0 tx=0x7f5524017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552401f070 con 0x7f55280af540 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524000c90 con 0x7f55280af540 2023-10-06T05:35:11.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.720+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524025420 con 0x7f55280af540 2023-10-06T05:35:11.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.721+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552816a270 con 0x7f55280af540 2023-10-06T05:35:11.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.721+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552816a830 con 0x7f55280af540 2023-10-06T05:35:11.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.722+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552804c380 con 0x7f55280af540 2023-10-06T05:35:11.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.722+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5524023070 con 0x7f55280af540 2023-10-06T05:35:11.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.723+0000 7f551d7fa700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f550c066f30 0x7f550c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:11.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.723+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5524085720 con 0x7f55280af540 2023-10-06T05:35:11.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.724+0000 7f552c99a700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f550c066f30 0x7f550c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:11.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.724+0000 7f552c99a700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f550c066f30 0x7f550c0693e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5518007fd0 tx=0x7f5518009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.728+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5524052ae0 con 0x7f55280af540 2023-10-06T05:35:11.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.928+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f552815de50 con 0x7f55280af540 2023-10-06T05:35:11.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.929+0000 7f551d7fa700 1 -- 172.21.15.33:0/1774021095 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v278) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55240150a0 con 0x7f55280af540 2023-10-06T05:35:11.931 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:35:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f550c066f30 msgr2=0x7f550c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f550c066f30 0x7f550c0693e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5518007fd0 tx=0x7f5518009040 comp rx=0 tx=0).stop 2023-10-06T05:35:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 msgr2=0x7f5528169bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 0x7f5528169bf0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f5524009ed0 tx=0x7f5524017000 comp rx=0 tx=0).stop 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 shutdown_connections 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f550c066f30 0x7f550c0693e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552815a650 0x7f55280c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55280af540 0x7f5528169bf0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 --2- 172.21.15.33:0/1774021095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55280cd5e0 0x7f55280b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 >> 172.21.15.33:0/1774021095 conn(0x7f552806d9b0 msgr2=0x7f55280cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 shutdown_connections 2023-10-06T05:35:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:11.931+0000 7f552e19d700 1 -- 172.21.15.33:0/1774021095 wait complete. 2023-10-06T05:35:11.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:35:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.137+0000 7f6a59b9d700 1 Processor -- start 2023-10-06T05:35:12.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.138+0000 7f6a59b9d700 1 -- start start 2023-10-06T05:35:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a59b9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6a54159bb0 0x7f6a540b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a59b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a540cbd80 con 0x7f6a540ccfc0 2023-10-06T05:35:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a59b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a540af5f0 con 0x7f6a540cf3b0 2023-10-06T05:35:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a59b9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a540af8d0 con 0x7f6a54159bb0 2023-10-06T05:35:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a58b9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6a540ccfc0 0x7f6a540cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38356/0 (socket says 172.21.15.33:38356) 2023-10-06T05:35:12.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a58b9b700 1 -- 172.21.15.33:0/583618790 learned_addr learned my addr 172.21.15.33:0/583618790 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.139+0000 7f6a5939c700 1 --2- 172.21.15.33:0/583618790 >> v2:172.21.15.167:3300/0 conn(0x7f6a54159bb0 0x7f6a540b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a537fe700 1 -- 172.21.15.33:0/583618790 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319977229 0 0) 0x7f6a540cbd80 con 0x7f6a540ccfc0 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a537fe700 1 -- 172.21.15.33:0/583618790 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a44003570 con 0x7f6a540ccfc0 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a5939c700 1 -- 172.21.15.33:0/583618790 >> v1:172.21.15.167:6789/0 conn(0x7f6a540cf3b0 legacy=0x7f6a541594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a5939c700 1 -- 172.21.15.33:0/583618790 >> v1:172.21.15.33:6789/0 conn(0x7f6a540ccfc0 legacy=0x7f6a540cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a5939c700 1 -- 172.21.15.33:0/583618790 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a540afbb0 con 0x7f6a54159bb0 2023-10-06T05:35:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a5939c700 1 --2- 172.21.15.33:0/583618790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54159bb0 0x7f6a540b0460 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009e90 tx=0x7f6a4c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec393ed393f229d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.140+0000 7f6a537fe700 1 -- 172.21.15.33:0/583618790 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00e070 con 0x7f6a54159bb0 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a537fe700 1 -- 172.21.15.33:0/583618790 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00bc40 con 0x7f6a54159bb0 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a537fe700 1 -- 172.21.15.33:0/583618790 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c012430 con 0x7f6a54159bb0 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 -- 172.21.15.33:0/583618790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54159bb0 msgr2=0x7f6a540b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/583618790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54159bb0 0x7f6a540b0460 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009e90 tx=0x7f6a4c009780 comp rx=0 tx=0).stop 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 -- 172.21.15.33:0/583618790 shutdown_connections 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/583618790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54159bb0 0x7f6a540b0460 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 -- 172.21.15.33:0/583618790 >> 172.21.15.33:0/583618790 conn(0x7f6a5406cff0 msgr2=0x7f6a5406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:12.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 -- 172.21.15.33:0/583618790 shutdown_connections 2023-10-06T05:35:12.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.141+0000 7f6a59b9d700 1 -- 172.21.15.33:0/583618790 wait complete. 2023-10-06T05:35:12.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.143+0000 7f6a59b9d700 1 Processor -- start 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.143+0000 7f6a59b9d700 1 -- start start 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a59b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a540ccfc0 0x7f6a540c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a59b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 0x7f6a540c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a59b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54169c90 0x7f6a540c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a59b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540ca2a0 con 0x7f6a54159bb0 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a59b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540b3d50 con 0x7f6a54169c90 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a59b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540b3420 con 0x7f6a540ccfc0 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a53fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 0x7f6a540c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a53fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 0x7f6a540c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59512/0 (socket says 172.21.15.33:59512) 2023-10-06T05:35:12.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a53fff700 1 -- 172.21.15.33:0/646332918 learned_addr learned my addr 172.21.15.33:0/646332918 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.144+0000 7f6a58b9b700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a540ccfc0 0x7f6a540c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a5939c700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54169c90 0x7f6a540c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a53fff700 1 -- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a540ccfc0 msgr2=0x7f6a540c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a53fff700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a540ccfc0 0x7f6a540c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a53fff700 1 -- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54169c90 msgr2=0x7f6a540c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a53fff700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54169c90 0x7f6a540c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a53fff700 1 -- 172.21.15.33:0/646332918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c009a20 con 0x7f6a54159bb0 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a5939c700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54169c90 0x7f6a540c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a58b9b700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a540ccfc0 0x7f6a540c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a53fff700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 0x7f6a540c9d60 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000c790 tx=0x7f6a40007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:12.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40011590 con 0x7f6a54159bb0 2023-10-06T05:35:12.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a400116f0 con 0x7f6a54159bb0 2023-10-06T05:35:12.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40011ab0 con 0x7f6a54159bb0 2023-10-06T05:35:12.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a540b3700 con 0x7f6a54159bb0 2023-10-06T05:35:12.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.145+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a540c6d50 con 0x7f6a54159bb0 2023-10-06T05:35:12.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.147+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a540b3f00 con 0x7f6a54159bb0 2023-10-06T05:35:12.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.147+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a4001a360 con 0x7f6a54159bb0 2023-10-06T05:35:12.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.148+0000 7f6a50ff9700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a38066f30 0x7f6a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.148+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6a40083430 con 0x7f6a54159bb0 2023-10-06T05:35:12.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.149+0000 7f6a58b9b700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a38066f30 0x7f6a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.149+0000 7f6a58b9b700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a38066f30 0x7f6a380693e0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6a480058d0 tx=0x7f6a48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:12.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.152+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a540b3f00 con 0x7f6a54159bb0 2023-10-06T05:35:12.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.355+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6a540c6fd0 con 0x7f6a54159bb0 2023-10-06T05:35:12.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.355+0000 7f6a50ff9700 1 -- 172.21.15.33:0/646332918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v278) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6a400507f0 con 0x7f6a54159bb0 2023-10-06T05:35:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:12.357 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T05:35:12.358 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.357+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a38066f30 msgr2=0x7f6a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.357+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a38066f30 0x7f6a380693e0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6a480058d0 tx=0x7f6a48009040 comp rx=0 tx=0).stop 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.357+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 msgr2=0x7f6a540c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.357+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 0x7f6a540c9d60 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000c790 tx=0x7f6a40007f20 comp rx=0 tx=0).stop 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 shutdown_connections 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a38066f30 0x7f6a380693e0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a54169c90 0x7f6a540c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a540ccfc0 0x7f6a540c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 --2- 172.21.15.33:0/646332918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a54159bb0 0x7f6a540c9d60 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 >> 172.21.15.33:0/646332918 conn(0x7f6a5406cff0 msgr2=0x7f6a54150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:12.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 shutdown_connections 2023-10-06T05:35:12.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.358+0000 7f6a59b9d700 1 -- 172.21.15.33:0/646332918 wait complete. 2023-10-06T05:35:12.377 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:35:12.378 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:35:12.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.567+0000 7f0c8b19d700 1 Processor -- start 2023-10-06T05:35:12.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.567+0000 7f0c8b19d700 1 -- start start 2023-10-06T05:35:12.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c840ad010 0x7f0c8406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8b19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0c8415acd0 0x7f0c84158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8b19d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8406b1f0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8406a3f0 con 0x7f0c840ad010 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8b19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8406a5e0 con 0x7f0c8415acd0 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c840ad010 0x7f0c8406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c840ad010 0x7f0c8406ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59516/0 (socket says 172.21.15.33:59516) 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8a19b700 1 -- 172.21.15.33:0/926719859 learned_addr learned my addr 172.21.15.33:0/926719859 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.569+0000 7f0c8a99c700 1 --2- 172.21.15.33:0/926719859 >> v2:172.21.15.167:3300/0 conn(0x7f0c8415acd0 0x7f0c84158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c8a19b700 1 -- 172.21.15.33:0/926719859 >> v2:172.21.15.167:3300/0 conn(0x7f0c8415acd0 msgr2=0x7f0c84158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c8a19b700 1 --2- 172.21.15.33:0/926719859 >> v2:172.21.15.167:3300/0 conn(0x7f0c8415acd0 0x7f0c84158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c8a19b700 1 -- 172.21.15.33:0/926719859 >> v1:172.21.15.167:6789/0 conn(0x7f0c840ad3e0 legacy=0x7f0c8415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c8a19b700 1 -- 172.21.15.33:0/926719859 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8406a8c0 con 0x7f0c840ad010 2023-10-06T05:35:12.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c8a19b700 1 --2- 172.21.15.33:0/926719859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad010 0x7f0c8406ea00 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f0c8406beb0 tx=0x7f0c80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9de69346a2d441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:12.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c89199700 1 -- 172.21.15.33:0/926719859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80009e70 con 0x7f0c840ad010 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c89199700 1 -- 172.21.15.33:0/926719859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8000b8f0 con 0x7f0c840ad010 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.570+0000 7f0c89199700 1 -- 172.21.15.33:0/926719859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000bc90 con 0x7f0c840ad010 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 -- 172.21.15.33:0/926719859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad010 msgr2=0x7f0c8406ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/926719859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad010 0x7f0c8406ea00 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f0c8406beb0 tx=0x7f0c80009250 comp rx=0 tx=0).stop 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 -- 172.21.15.33:0/926719859 shutdown_connections 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/926719859 >> v2:172.21.15.167:3300/0 conn(0x7f0c8415acd0 0x7f0c84158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/926719859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad010 0x7f0c8406ea00 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 -- 172.21.15.33:0/926719859 >> 172.21.15.33:0/926719859 conn(0x7f0c84068090 msgr2=0x7f0c8406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 -- 172.21.15.33:0/926719859 shutdown_connections 2023-10-06T05:35:12.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.571+0000 7f0c8b19d700 1 -- 172.21.15.33:0/926719859 wait complete. 2023-10-06T05:35:12.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.572+0000 7f0c8b19d700 1 Processor -- start 2023-10-06T05:35:12.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 -- start start 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c840ad010 0x7f0c840c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 0x7f0c841561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c8415acd0 0x7f0c841545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84155d40 con 0x7f0c840ad3e0 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84152d00 con 0x7f0c840ad010 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.573+0000 7f0c8b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84155130 con 0x7f0c8415acd0 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 0x7f0c841561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 0x7f0c841561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59532/0 (socket says 172.21.15.33:59532) 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 -- 172.21.15.33:0/2384992839 learned_addr learned my addr 172.21.15.33:0/2384992839 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:12.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8a19b700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c840ad010 0x7f0c840c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8a99c700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c8415acd0 0x7f0c841545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 -- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c8415acd0 msgr2=0x7f0c841545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c8415acd0 0x7f0c841545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 -- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c840ad010 msgr2=0x7f0c840c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c840ad010 0x7f0c840c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 -- 172.21.15.33:0/2384992839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800094f0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8a99c700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c8415acd0 0x7f0c841545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.574+0000 7f0c8999a700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 0x7f0c841561d0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c5c0 tx=0x7f0c7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.575+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74011590 con 0x7f0c840ad3e0 2023-10-06T05:35:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.575+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74011710 con 0x7f0c840ad3e0 2023-10-06T05:35:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.575+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74011ad0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.575+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c84155410 con 0x7f0c840ad3e0 2023-10-06T05:35:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.575+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c841559d0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.576+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c84155ef0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.577+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c7401b3b0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.578+0000 7f0c7a7fc700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c68066f30 0x7f0c680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.578+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0c74083910 con 0x7f0c840ad3e0 2023-10-06T05:35:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.578+0000 7f0c8a19b700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c68066f30 0x7f0c680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:12.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.579+0000 7f0c8a19b700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c68066f30 0x7f0c680693e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0c8406beb0 tx=0x7f0c80017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.582+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c84155ef0 con 0x7f0c840ad3e0 2023-10-06T05:35:12.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:12.785+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0c84155ef0 con 0x7f0c840ad3e0 2023-10-06T05:35:13.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.145+0000 7f0c7a7fc700 1 -- 172.21.15.33:0/2384992839 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v279) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0c84155ef0 con 0x7f0c840ad3e0 2023-10-06T05:35:13.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.150+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c68066f30 msgr2=0x7f0c680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.150+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c68066f30 0x7f0c680693e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0c8406beb0 tx=0x7f0c80017000 comp rx=0 tx=0).stop 2023-10-06T05:35:13.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.150+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 msgr2=0x7f0c841561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.150+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 0x7f0c841561d0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c5c0 tx=0x7f0c7400c8d0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.151+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 shutdown_connections 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.151+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0c68066f30 0x7f0c680693e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.151+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c8415acd0 0x7f0c841545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.151+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c840ad010 0x7f0c840c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.151+0000 7f0c8b19d700 1 --2- 172.21.15.33:0/2384992839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c840ad3e0 0x7f0c841561d0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.151+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 >> 172.21.15.33:0/2384992839 conn(0x7f0c84068090 msgr2=0x7f0c840b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:13.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.152+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 shutdown_connections 2023-10-06T05:35:13.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.152+0000 7f0c8b19d700 1 -- 172.21.15.33:0/2384992839 wait complete. 2023-10-06T05:35:13.155 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T05:35:13.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:35:13.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.366+0000 7efd9579d700 1 Processor -- start 2023-10-06T05:35:13.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.366+0000 7efd9579d700 1 -- start start 2023-10-06T05:35:13.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.367+0000 7efd9579d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd90156370 0x7efd900aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:13.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd9579d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9006b330 con 0x7efd900c7f60 2023-10-06T05:35:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd9579d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9006b610 con 0x7efd90156740 2023-10-06T05:35:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd9579d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9006beb0 con 0x7efd90156370 2023-10-06T05:35:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd8ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd90156370 0x7efd900aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd94f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efd900c7f60 0x7efd900abdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38358/0 (socket says 172.21.15.33:38358) 2023-10-06T05:35:13.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd94f9c700 1 -- 172.21.15.33:0/3355065291 learned_addr learned my addr 172.21.15.33:0/3355065291 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:13.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd8ffff700 1 -- 172.21.15.33:0/3355065291 >> v1:172.21.15.167:6789/0 conn(0x7efd90156740 legacy=0x7efd900cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd8ffff700 1 -- 172.21.15.33:0/3355065291 >> v1:172.21.15.33:6789/0 conn(0x7efd900c7f60 legacy=0x7efd900abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd8ffff700 1 -- 172.21.15.33:0/3355065291 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9006b8f0 con 0x7efd90156370 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.368+0000 7efd8effd700 1 -- 172.21.15.33:0/3355065291 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807975033 0 0) 0x7efd9006b330 con 0x7efd900c7f60 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.369+0000 7efd8ffff700 1 --2- 172.21.15.33:0/3355065291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd900aef80 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7efd800098e0 tx=0x7efd800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a77cd44f4b26b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.369+0000 7efd8effd700 1 -- 172.21.15.33:0/3355065291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80009e80 con 0x7efd90156370 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.369+0000 7efd8effd700 1 -- 172.21.15.33:0/3355065291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8000b860 con 0x7efd90156370 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.369+0000 7efd8effd700 1 -- 172.21.15.33:0/3355065291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000bbe0 con 0x7efd90156370 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.369+0000 7efd9579d700 1 -- 172.21.15.33:0/3355065291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 msgr2=0x7efd900aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.369+0000 7efd9579d700 1 --2- 172.21.15.33:0/3355065291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd900aef80 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7efd800098e0 tx=0x7efd800091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.370+0000 7efd9579d700 1 -- 172.21.15.33:0/3355065291 shutdown_connections 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.370+0000 7efd9579d700 1 --2- 172.21.15.33:0/3355065291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd900aef80 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.370+0000 7efd9579d700 1 -- 172.21.15.33:0/3355065291 >> 172.21.15.33:0/3355065291 conn(0x7efd9006c560 msgr2=0x7efd9006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:13.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.370+0000 7efd9579d700 1 -- 172.21.15.33:0/3355065291 shutdown_connections 2023-10-06T05:35:13.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.370+0000 7efd9579d700 1 -- 172.21.15.33:0/3355065291 wait complete. 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.371+0000 7efd9579d700 1 Processor -- start 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.371+0000 7efd9579d700 1 -- start start 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd9579d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd900c7f60 0x7efd9006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd9579d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd9015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd9579d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd90156740 0x7efd900b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd9579d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900afc50 con 0x7efd90156370 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd9579d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900aff30 con 0x7efd90156740 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd9579d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd90159950 con 0x7efd900c7f60 2023-10-06T05:35:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd9015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd9015a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59554/0 (socket says 172.21.15.33:59554) 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.372+0000 7efd8f7fe700 1 -- 172.21.15.33:0/1841140985 learned_addr learned my addr 172.21.15.33:0/1841140985 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd94f9c700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd90156740 0x7efd900b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8ffff700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd900c7f60 0x7efd9006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8f7fe700 1 -- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd900c7f60 msgr2=0x7efd9006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8f7fe700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd900c7f60 0x7efd9006e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8f7fe700 1 -- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd90156740 msgr2=0x7efd900b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8f7fe700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd90156740 0x7efd900b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8f7fe700 1 -- 172.21.15.33:0/1841140985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80009420 con 0x7efd90156370 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd8f7fe700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd9015a430 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c7c0 tx=0x7efd84007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84011590 con 0x7efd90156370 2023-10-06T05:35:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd840116f0 con 0x7efd90156370 2023-10-06T05:35:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84011ab0 con 0x7efd90156370 2023-10-06T05:35:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd90159c30 con 0x7efd90156370 2023-10-06T05:35:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.373+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9006ed80 con 0x7efd90156370 2023-10-06T05:35:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.374+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9006efa0 con 0x7efd90156370 2023-10-06T05:35:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.376+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd8401b3b0 con 0x7efd90156370 2023-10-06T05:35:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.377+0000 7efd77fff700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd7c066f30 0x7efd7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.377+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(279..279 src has 1..279) v4 ==== 5576+0+0 (secure 0 0 0) 0x7efd84083ab0 con 0x7efd90156370 2023-10-06T05:35:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.377+0000 7efd8ffff700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd7c066f30 0x7efd7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.378+0000 7efd8ffff700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd7c066f30 0x7efd7c0693e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7efd800093f0 tx=0x7efd80017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:13.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.382+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd84050d00 con 0x7efd90156370 2023-10-06T05:35:13.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:13.586+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efd9015a150 con 0x7efd90156370 2023-10-06T05:35:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.149+0000 7efd77fff700 1 -- 172.21.15.33:0/1841140985 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v280) v1 ==== 170+0+0 (secure 0 0 0) 0x7efd9015a150 con 0x7efd90156370 2023-10-06T05:35:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.159+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd7c066f30 msgr2=0x7efd7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.159+0000 7efd9579d700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd7c066f30 0x7efd7c0693e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7efd800093f0 tx=0x7efd80017000 comp rx=0 tx=0).stop 2023-10-06T05:35:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.159+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 msgr2=0x7efd9015a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.159+0000 7efd9579d700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd9015a430 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c7c0 tx=0x7efd84007f50 comp rx=0 tx=0).stop 2023-10-06T05:35:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.160+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 shutdown_connections 2023-10-06T05:35:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.160+0000 7efd9579d700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7efd7c066f30 0x7efd7c0693e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.160+0000 7efd9579d700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd90156740 0x7efd900b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.160+0000 7efd9579d700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd900c7f60 0x7efd9006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.160+0000 7efd9579d700 1 --2- 172.21.15.33:0/1841140985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd90156370 0x7efd9015a430 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.160+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 >> 172.21.15.33:0/1841140985 conn(0x7efd9006c560 msgr2=0x7efd9014cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.168+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 shutdown_connections 2023-10-06T05:35:14.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.168+0000 7efd9579d700 1 -- 172.21.15.33:0/1841140985 wait complete. 2023-10-06T05:35:14.174 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T05:35:14.194 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:35:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.386+0000 7ff00b59e700 1 Processor -- start 2023-10-06T05:35:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.386+0000 7ff00b59e700 1 -- start start 2023-10-06T05:35:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff00b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff00c0b0900 0x7ff00c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff00b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00c0ad430 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff00b59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00c0afe20 con 0x7ff00c156370 2023-10-06T05:35:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff00b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00c06bc70 con 0x7ff00c0b0900 2023-10-06T05:35:14.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff00ad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff00c0b0900 0x7ff00c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff009d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff00c0cc7e0 0x7ff00c0b01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38372/0 (socket says 172.21.15.33:38372) 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.387+0000 7ff009d9b700 1 -- 172.21.15.33:0/46666497 learned_addr learned my addr 172.21.15.33:0/46666497 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00ad9d700 1 -- 172.21.15.33:0/46666497 >> v1:172.21.15.167:6789/0 conn(0x7ff00c156370 legacy=0x7ff00c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00ad9d700 1 -- 172.21.15.33:0/46666497 >> v1:172.21.15.33:6789/0 conn(0x7ff00c0cc7e0 legacy=0x7ff00c0b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00ad9d700 1 -- 172.21.15.33:0/46666497 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c0cccc0 con 0x7ff00c0b0900 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00959a700 1 -- 172.21.15.33:0/46666497 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288979799 0 0) 0x7ff00c0ad430 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00ad9d700 1 --2- 172.21.15.33:0/46666497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0b0900 0x7ff00c0af8e0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7ff004009e60 tx=0x7ff004009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119409e4d5db5ee1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00959a700 1 -- 172.21.15.33:0/46666497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00400e070 con 0x7ff00c0b0900 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00959a700 1 -- 172.21.15.33:0/46666497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00400bc30 con 0x7ff00c0b0900 2023-10-06T05:35:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.388+0000 7ff00959a700 1 -- 172.21.15.33:0/46666497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004012430 con 0x7ff00c0b0900 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 -- 172.21.15.33:0/46666497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0b0900 msgr2=0x7ff00c0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 --2- 172.21.15.33:0/46666497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0b0900 0x7ff00c0af8e0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7ff004009e60 tx=0x7ff004009780 comp rx=0 tx=0).stop 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 -- 172.21.15.33:0/46666497 shutdown_connections 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 --2- 172.21.15.33:0/46666497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0b0900 0x7ff00c0af8e0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 -- 172.21.15.33:0/46666497 >> 172.21.15.33:0/46666497 conn(0x7ff00c06d030 msgr2=0x7ff00c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 -- 172.21.15.33:0/46666497 shutdown_connections 2023-10-06T05:35:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.389+0000 7ff00b59e700 1 -- 172.21.15.33:0/46666497 wait complete. 2023-10-06T05:35:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.391+0000 7ff00b59e700 1 Processor -- start 2023-10-06T05:35:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.391+0000 7ff00b59e700 1 -- start start 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff00c0b0900 0x7ff00c167bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 0x7ff00c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff00c156370 0x7ff00c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c14a8e0 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c16b360 con 0x7ff00c0b0900 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00c1684e0 con 0x7ff00c156370 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 0x7ff00c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 0x7ff00c151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59582/0 (socket says 172.21.15.33:59582) 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 -- 172.21.15.33:0/1457130713 learned_addr learned my addr 172.21.15.33:0/1457130713 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00a59c700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff00c0b0900 0x7ff00c167bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00ad9d700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff00c156370 0x7ff00c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 -- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff00c156370 msgr2=0x7ff00c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff00c156370 0x7ff00c151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 -- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff00c0b0900 msgr2=0x7ff00c167bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff00c0b0900 0x7ff00c167bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff009d9b700 1 -- 172.21.15.33:0/1457130713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004009a20 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.392+0000 7ff00ad9d700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff00c156370 0x7ff00c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.393+0000 7ff009d9b700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 0x7ff00c151700 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7ff00000c7e0 tx=0x7ff000007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.393+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000005b30 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.393+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000005c90 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.393+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00c15cde0 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.393+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000007300 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.393+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00c16ad10 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.394+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00c06f950 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.395+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0000076b0 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.396+0000 7feffa7fc700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fefec066f30 0x7fefec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.397+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 5958+0+0 (secure 0 0 0) 0x7ff000083c40 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.397+0000 7ff00a59c700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fefec066f30 0x7fefec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.398+0000 7ff00a59c700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fefec066f30 0x7fefec0693e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff00c14e540 tx=0x7feffc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.402+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff000050d10 con 0x7ff00c0cc7e0 2023-10-06T05:35:14.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:14.650+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff00c04c380 con 0x7ff00c0cc7e0 2023-10-06T05:35:15.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.164+0000 7feffa7fc700 1 -- 172.21.15.33:0/1457130713 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 41 and data pool 42 v215) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff0000170c0 con 0x7ff00c0cc7e0 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fefec066f30 msgr2=0x7fefec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fefec066f30 0x7fefec0693e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff00c14e540 tx=0x7feffc009040 comp rx=0 tx=0).stop 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 msgr2=0x7ff00c151700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 0x7ff00c151700 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7ff00000c7e0 tx=0x7ff000007df0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 shutdown_connections 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fefec066f30 0x7fefec0693e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff00c156370 0x7ff00c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff00c0b0900 0x7ff00c167bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 --2- 172.21.15.33:0/1457130713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff00c0cc7e0 0x7ff00c151700 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 >> 172.21.15.33:0/1457130713 conn(0x7ff00c06d030 msgr2=0x7ff00c0ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 shutdown_connections 2023-10-06T05:35:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.167+0000 7ff00b59e700 1 -- 172.21.15.33:0/1457130713 wait complete. 2023-10-06T05:35:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 41 and data pool 42 2023-10-06T05:35:15.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.379+0000 7f97b719d700 1 Processor -- start 2023-10-06T05:35:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.379+0000 7f97b719d700 1 -- start start 2023-10-06T05:35:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.380+0000 7f97b719d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f97b0153b10 0x7f97b00c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.380+0000 7f97b719d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97b00c2610 0x7f97b00c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.380+0000 7f97b719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b015abd0 con 0x7f97b00c6b30 2023-10-06T05:35:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.380+0000 7f97b719d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00c74f0 con 0x7f97b00c2610 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.380+0000 7f97b719d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00c9b00 con 0x7f97b0153b10 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b699c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97b00c2610 0x7f97b00c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97b00c6b30 0x7f97b00675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38386/0 (socket says 172.21.15.33:38386) 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b599a700 1 -- 172.21.15.33:0/2618172476 learned_addr learned my addr 172.21.15.33:0/2618172476 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b619b700 1 --2- 172.21.15.33:0/2618172476 >> v2:172.21.15.167:3300/0 conn(0x7f97b0153b10 0x7f97b00c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b699c700 1 -- 172.21.15.33:0/2618172476 >> v2:172.21.15.167:3300/0 conn(0x7f97b0153b10 msgr2=0x7f97b00c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b699c700 1 --2- 172.21.15.33:0/2618172476 >> v2:172.21.15.167:3300/0 conn(0x7f97b0153b10 0x7f97b00c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b699c700 1 -- 172.21.15.33:0/2618172476 >> v1:172.21.15.33:6789/0 conn(0x7f97b00c6b30 legacy=0x7f97b00675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b699c700 1 -- 172.21.15.33:0/2618172476 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b015a050 con 0x7f97b00c2610 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.381+0000 7f97b699c700 1 --2- 172.21.15.33:0/2618172476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c2610 0x7f97b00c6fb0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f97b00c9ca0 tx=0x7f97a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c7be3de0181996 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b5199700 1 -- 172.21.15.33:0/2618172476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8016070 con 0x7f97b00c2610 2023-10-06T05:35:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b5199700 1 -- 172.21.15.33:0/2618172476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a800bc00 con 0x7f97b00c2610 2023-10-06T05:35:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b5199700 1 -- 172.21.15.33:0/2618172476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800e070 con 0x7f97b00c2610 2023-10-06T05:35:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b719d700 1 -- 172.21.15.33:0/2618172476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c2610 msgr2=0x7f97b00c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b719d700 1 --2- 172.21.15.33:0/2618172476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c2610 0x7f97b00c6fb0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f97b00c9ca0 tx=0x7f97a8009780 comp rx=0 tx=0).stop 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b719d700 1 -- 172.21.15.33:0/2618172476 shutdown_connections 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b719d700 1 --2- 172.21.15.33:0/2618172476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c2610 0x7f97b00c6fb0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b719d700 1 --2- 172.21.15.33:0/2618172476 >> v2:172.21.15.167:3300/0 conn(0x7f97b0153b10 0x7f97b00c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.382+0000 7f97b719d700 1 -- 172.21.15.33:0/2618172476 >> 172.21.15.33:0/2618172476 conn(0x7f97b006c560 msgr2=0x7f97b006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.383+0000 7f97b719d700 1 -- 172.21.15.33:0/2618172476 shutdown_connections 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.383+0000 7f97b719d700 1 -- 172.21.15.33:0/2618172476 wait complete. 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.384+0000 7f97b719d700 1 Processor -- start 2023-10-06T05:35:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.384+0000 7f97b719d700 1 -- start start 2023-10-06T05:35:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b00c2610 0x7f97b013cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 0x7f97b013add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b0153b10 0x7f97b013e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b013d1a0 con 0x7f97b00c6b30 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b013d440 con 0x7f97b00c2610 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b013d720 con 0x7f97b0153b10 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 0x7f97b013add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b599a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 0x7f97b013add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59604/0 (socket says 172.21.15.33:59604) 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b599a700 1 -- 172.21.15.33:0/1576456329 learned_addr learned my addr 172.21.15.33:0/1576456329 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b619b700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b00c2610 0x7f97b013cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.385+0000 7f97b699c700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b0153b10 0x7f97b013e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b599a700 1 -- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b0153b10 msgr2=0x7f97b013e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b599a700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b0153b10 0x7f97b013e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b599a700 1 -- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b00c2610 msgr2=0x7f97b013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b599a700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b00c2610 0x7f97b013cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b599a700 1 -- 172.21.15.33:0/1576456329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009a20 con 0x7f97b00c6b30 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b599a700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 0x7f97b013add0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c7c0 tx=0x7f97a0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0005b30 con 0x7f97b00c6b30 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a0005cb0 con 0x7f97b00c6b30 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0004a80 con 0x7f97b00c6b30 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b013da00 con 0x7f97b00c6b30 2023-10-06T05:35:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.386+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b013dfc0 con 0x7f97b00c6b30 2023-10-06T05:35:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.387+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b013e1e0 con 0x7f97b00c6b30 2023-10-06T05:35:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.388+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97a00070f0 con 0x7f97b00c6b30 2023-10-06T05:35:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.389+0000 7f97a67fc700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9794066f30 0x7f97940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.389+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f97a0083bb0 con 0x7f97b00c6b30 2023-10-06T05:35:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.390+0000 7f97b619b700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9794066f30 0x7f97940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.390+0000 7f97b619b700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9794066f30 0x7f97940693e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f97ac006fd0 tx=0x7f97ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.393+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97a0050ca0 con 0x7f97b00c6b30 2023-10-06T05:35:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.595+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97b004c380 con 0x7f97b00c6b30 2023-10-06T05:35:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.596+0000 7f97a67fc700 1 -- 172.21.15.33:0/1576456329 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v281) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f97a0018070 con 0x7f97b00c6b30 2023-10-06T05:35:15.598 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:15.599 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":281,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:14.651901+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:35:12.826592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":42,"pool_name":"cephfs_data","create_time":"2023-10-06T05:35:13.627133+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9794066f30 msgr2=0x7f97940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9794066f30 0x7f97940693e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f97ac006fd0 tx=0x7f97ac008040 comp rx=0 tx=0).stop 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 msgr2=0x7f97b013add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 0x7f97b013add0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c7c0 tx=0x7f97a0007f70 comp rx=0 tx=0).stop 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 shutdown_connections 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9794066f30 0x7f97940693e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97b0153b10 0x7f97b013e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97b00c2610 0x7f97b013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 --2- 172.21.15.33:0/1576456329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b00c6b30 0x7f97b013add0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 >> 172.21.15.33:0/1576456329 conn(0x7f97b006c560 msgr2=0x7f97b013bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 shutdown_connections 2023-10-06T05:35:15.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.599+0000 7f97b719d700 1 -- 172.21.15.33:0/1576456329 wait complete. 2023-10-06T05:35:15.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.806+0000 7ff2109fe700 1 Processor -- start 2023-10-06T05:35:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.806+0000 7ff2109fe700 1 -- start start 2023-10-06T05:35:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff2109fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff20c0c8050 0x7ff20c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff2109fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff20c06aad0 0x7ff20c06af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff2109fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0c9cb0 con 0x7ff20c13faa0 2023-10-06T05:35:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff2109fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c0ca270 con 0x7ff20c0c8050 2023-10-06T05:35:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff2109fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c06b840 con 0x7ff20c06aad0 2023-10-06T05:35:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff20c0c8050 0x7ff20c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20ad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff20c13faa0 0x7ff20c0c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38400/0 (socket says 172.21.15.33:38400) 2023-10-06T05:35:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20ad9d700 1 -- 172.21.15.33:0/4167620412 learned_addr learned my addr 172.21.15.33:0/4167620412 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20b59e700 1 --2- 172.21.15.33:0/4167620412 >> v2:172.21.15.167:3300/0 conn(0x7ff20c06aad0 0x7ff20c06af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20a59c700 1 -- 172.21.15.33:0/4167620412 >> v2:172.21.15.167:3300/0 conn(0x7ff20c06aad0 msgr2=0x7ff20c06af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20a59c700 1 --2- 172.21.15.33:0/4167620412 >> v2:172.21.15.167:3300/0 conn(0x7ff20c06aad0 0x7ff20c06af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20a59c700 1 -- 172.21.15.33:0/4167620412 >> v1:172.21.15.33:6789/0 conn(0x7ff20c13faa0 legacy=0x7ff20c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff20a59c700 1 -- 172.21.15.33:0/4167620412 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c06bb20 con 0x7ff20c0c8050 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.808+0000 7ff209d9b700 1 -- 172.21.15.33:0/4167620412 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041956239 0 0) 0x7ff20c0c9cb0 con 0x7ff20c13faa0 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.809+0000 7ff20a59c700 1 --2- 172.21.15.33:0/4167620412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c0c8050 0x7ff20c13fe70 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7ff20c156b90 tx=0x7ff2000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa2f112d47a3bb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.809+0000 7ff209d9b700 1 -- 172.21.15.33:0/4167620412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20000b6b0 con 0x7ff20c0c8050 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.809+0000 7ff209d9b700 1 -- 172.21.15.33:0/4167620412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20000b810 con 0x7ff20c0c8050 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.809+0000 7ff209d9b700 1 -- 172.21.15.33:0/4167620412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20000bbb0 con 0x7ff20c0c8050 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 -- 172.21.15.33:0/4167620412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c0c8050 msgr2=0x7ff20c13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 --2- 172.21.15.33:0/4167620412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c0c8050 0x7ff20c13fe70 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7ff20c156b90 tx=0x7ff2000091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 -- 172.21.15.33:0/4167620412 shutdown_connections 2023-10-06T05:35:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 --2- 172.21.15.33:0/4167620412 >> v2:172.21.15.167:3300/0 conn(0x7ff20c06aad0 0x7ff20c06af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 --2- 172.21.15.33:0/4167620412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c0c8050 0x7ff20c13fe70 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 -- 172.21.15.33:0/4167620412 >> 172.21.15.33:0/4167620412 conn(0x7ff20c06d030 msgr2=0x7ff20c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 -- 172.21.15.33:0/4167620412 shutdown_connections 2023-10-06T05:35:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.810+0000 7ff2109fe700 1 -- 172.21.15.33:0/4167620412 wait complete. 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.811+0000 7ff2109fe700 1 Processor -- start 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.811+0000 7ff2109fe700 1 -- start start 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff2109fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 0x7ff20c0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff2109fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff20c0c8050 0x7ff20c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff2109fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c13faa0 0x7ff20c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff2109fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c13d130 con 0x7ff20c06aad0 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff2109fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0c3b00 con 0x7ff20c13faa0 2023-10-06T05:35:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff2109fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0c3de0 con 0x7ff20c0c8050 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff20ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 0x7ff20c0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff20ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 0x7ff20c0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59628/0 (socket says 172.21.15.33:59628) 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.812+0000 7ff20ad9d700 1 -- 172.21.15.33:0/1057201776 learned_addr learned my addr 172.21.15.33:0/1057201776 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20b59e700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c13faa0 0x7ff20c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20a59c700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff20c0c8050 0x7ff20c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20ad9d700 1 -- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff20c0c8050 msgr2=0x7ff20c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20ad9d700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff20c0c8050 0x7ff20c0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20ad9d700 1 -- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c13faa0 msgr2=0x7ff20c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20ad9d700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c13faa0 0x7ff20c0b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20ad9d700 1 -- 172.21.15.33:0/1057201776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200009420 con 0x7ff20c06aad0 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20a59c700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff20c0c8050 0x7ff20c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20b59e700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c13faa0 0x7ff20c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff20ad9d700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 0x7ff20c0c3190 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00c5b0 tx=0x7ff1fc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc005b30 con 0x7ff20c06aad0 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc005c90 con 0x7ff20c06aad0 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc007300 con 0x7ff20c06aad0 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.813+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c0c40c0 con 0x7ff20c06aad0 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.814+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c0c36d0 con 0x7ff20c06aad0 2023-10-06T05:35:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.814+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c13cf00 con 0x7ff20c06aad0 2023-10-06T05:35:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.815+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1fc01b3b0 con 0x7ff20c06aad0 2023-10-06T05:35:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.817+0000 7ff1f2ffd700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1f8066f30 0x7ff1f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.817+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1fc083c60 con 0x7ff20c06aad0 2023-10-06T05:35:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.817+0000 7ff20a59c700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1f8066f30 0x7ff1f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:15.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.818+0000 7ff20a59c700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1f8066f30 0x7ff1f80693e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff20c13f510 tx=0x7ff200017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:15.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:15.821+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff20c13cf00 con 0x7ff20c06aad0 2023-10-06T05:35:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.026+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff20c13cf00 con 0x7ff20c06aad0 2023-10-06T05:35:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.027+0000 7ff1f2ffd700 1 -- 172.21.15.33:0/1057201776 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v282) v1 ==== 74+0+16880 (secure 0 0 0) 0x7ff20c13cf00 con 0x7ff20c06aad0 2023-10-06T05:35:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:16.030 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":282,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:15.940527+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:35:12.826592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":42,"pool_name":"cephfs_data","create_time":"2023-10-06T05:35:13.627133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"282","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1f8066f30 msgr2=0x7ff1f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1f8066f30 0x7ff1f80693e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff20c13f510 tx=0x7ff200017000 comp rx=0 tx=0).stop 2023-10-06T05:35:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 msgr2=0x7ff20c0c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 0x7ff20c0c3190 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00c5b0 tx=0x7ff1fc001070 comp rx=0 tx=0).stop 2023-10-06T05:35:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 shutdown_connections 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1f8066f30 0x7ff1f80693e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff20c13faa0 0x7ff20c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff20c06aad0 0x7ff20c0c3190 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 --2- 172.21.15.33:0/1057201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff20c0c8050 0x7ff20c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 >> 172.21.15.33:0/1057201776 conn(0x7ff20c06d030 msgr2=0x7ff20c0c2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 shutdown_connections 2023-10-06T05:35:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.030+0000 7ff2109fe700 1 -- 172.21.15.33:0/1057201776 wait complete. 2023-10-06T05:35:16.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:35:16.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.240+0000 7f9e85b9d700 1 Processor -- start 2023-10-06T05:35:16.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.241+0000 7f9e85b9d700 1 -- start start 2023-10-06T05:35:16.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e85b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 0x7f9e800abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e85b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8006ae10 con 0x7f9e80068310 2023-10-06T05:35:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e85b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8006a4e0 con 0x7f9e80070440 2023-10-06T05:35:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e85b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8006a2d0 con 0x7f9e800b1f90 2023-10-06T05:35:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e8539c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e80068310 0x7f9e80159dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38412/0 (socket says 172.21.15.33:38412) 2023-10-06T05:35:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e8539c700 1 -- 172.21.15.33:0/52402069 learned_addr learned my addr 172.21.15.33:0/52402069 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.242+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078592203 0 0) 0x7f9e8006ae10 con 0x7f9e80068310 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e70003570 con 0x7f9e80068310 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e84b9b700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 0x7f9e800abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232889938 0 0) 0x7f9e70003570 con 0x7f9e80068310 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 msgr2=0x7f9e800abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 0x7f9e800abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 >> v1:172.21.15.167:6789/0 conn(0x7f9e80070440 legacy=0x7f9e80067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8006a7c0 con 0x7f9e80068310 2023-10-06T05:35:16.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f9e780017f0 con 0x7f9e80068310 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 >> v1:172.21.15.33:6789/0 conn(0x7f9e80068310 legacy=0x7f9e80159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 0x7f9e70003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e700040c0 0x7f9e70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e70004da0 0x7f9e700051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8006a2d0 con 0x7f9e70004da0 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7000bc70 con 0x7f9e700040c0 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7000be30 con 0x7f9e800b1f90 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e8539c700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e70004da0 0x7f9e700051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e84b9b700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 0x7f9e70003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.243+0000 7f9e7ffff700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e700040c0 0x7f9e70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e8539c700 1 -- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 msgr2=0x7f9e70003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e8539c700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e800b1f90 0x7f9e70003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e8539c700 1 -- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e700040c0 msgr2=0x7f9e70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e8539c700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e700040c0 0x7f9e70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e8539c700 1 -- 172.21.15.33:0/52402069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e70003570 con 0x7f9e70004da0 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e8539c700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e70004da0 0x7f9e700051f0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800a900 tx=0x7f9e780021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ba74623d747246 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e780070d0 con 0x7f9e70004da0 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e78007230 con 0x7f9e70004da0 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.244+0000 7f9e7f7fe700 1 -- 172.21.15.33:0/52402069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e780075d0 con 0x7f9e70004da0 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e85b9d700 1 -- 172.21.15.33:0/52402069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e70004da0 msgr2=0x7f9e700051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/52402069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e70004da0 0x7f9e700051f0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800a900 tx=0x7f9e780021d0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e8539c700 1 -- 172.21.15.33:0/52402069 reap_dead start 2023-10-06T05:35:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e85b9d700 1 -- 172.21.15.33:0/52402069 shutdown_connections 2023-10-06T05:35:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e85b9d700 1 -- 172.21.15.33:0/52402069 >> 172.21.15.33:0/52402069 conn(0x7f9e8006ea70 msgr2=0x7f9e8006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e85b9d700 1 -- 172.21.15.33:0/52402069 shutdown_connections 2023-10-06T05:35:16.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.245+0000 7f9e85b9d700 1 -- 172.21.15.33:0/52402069 wait complete. 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.246+0000 7f9e85b9d700 1 Processor -- start 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 -- start start 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e70004da0 0x7f9e8015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 0x7f9e80151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e801527f0 0x7f9e80152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e801531d0 con 0x7f9e801543a0 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80169880 con 0x7f9e801527f0 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.247+0000 7f9e85b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80154810 con 0x7f9e70004da0 2023-10-06T05:35:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 0x7f9e80151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 0x7f9e80151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59646/0 (socket says 172.21.15.33:59646) 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e7ffff700 1 -- 172.21.15.33:0/3643849034 learned_addr learned my addr 172.21.15.33:0/3643849034 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e84b9b700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e70004da0 0x7f9e8015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e8539c700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e801527f0 0x7f9e80152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e84b9b700 1 -- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e801527f0 msgr2=0x7f9e80152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e84b9b700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e801527f0 0x7f9e80152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.248+0000 7f9e84b9b700 1 -- 172.21.15.33:0/3643849034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 msgr2=0x7f9e80151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.249+0000 7f9e84b9b700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 0x7f9e80151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.249+0000 7f9e84b9b700 1 -- 172.21.15.33:0/3643849034 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80154ad0 con 0x7f9e70004da0 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.249+0000 7f9e7ffff700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 0x7f9e80151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.249+0000 7f9e84b9b700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e70004da0 0x7f9e8015dea0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e74010a50 tx=0x7f9e74008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.249+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74010e90 con 0x7f9e70004da0 2023-10-06T05:35:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.249+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7400e870 con 0x7f9e70004da0 2023-10-06T05:35:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.250+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400ec30 con 0x7f9e70004da0 2023-10-06T05:35:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.250+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e80154db0 con 0x7f9e70004da0 2023-10-06T05:35:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.250+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e80155370 con 0x7f9e70004da0 2023-10-06T05:35:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.252+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8004c380 con 0x7f9e70004da0 2023-10-06T05:35:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.253+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e74014430 con 0x7f9e70004da0 2023-10-06T05:35:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.254+0000 7f9e7cff9700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e64066f30 0x7f9e640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.254+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9e74084d70 con 0x7f9e70004da0 2023-10-06T05:35:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.254+0000 7f9e7ffff700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e64066f30 0x7f9e640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.255+0000 7f9e7ffff700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e64066f30 0x7f9e640693e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c007e40 tx=0x7f9e6c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:16.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.259+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e74051de0 con 0x7f9e70004da0 2023-10-06T05:35:16.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:16.506+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9e8015c2d0 con 0x7f9e70004da0 2023-10-06T05:35:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.189+0000 7f9e7cff9700 1 -- 172.21.15.33:0/3643849034 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v218) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9e740180b0 con 0x7f9e70004da0 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e64066f30 msgr2=0x7f9e640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e64066f30 0x7f9e640693e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c007e40 tx=0x7f9e6c007d30 comp rx=0 tx=0).stop 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e70004da0 msgr2=0x7f9e8015dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e70004da0 0x7f9e8015dea0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e74010a50 tx=0x7f9e74008040 comp rx=0 tx=0).stop 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 shutdown_connections 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9e64066f30 0x7f9e640693e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e801527f0 0x7f9e80152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e70004da0 0x7f9e8015dea0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 --2- 172.21.15.33:0/3643849034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e801543a0 0x7f9e80151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 >> 172.21.15.33:0/3643849034 conn(0x7f9e8006ea70 msgr2=0x7f9e80153630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 shutdown_connections 2023-10-06T05:35:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.192+0000 7f9e85b9d700 1 -- 172.21.15.33:0/3643849034 wait complete. 2023-10-06T05:35:17.214 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:35:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.407+0000 7f251d99d700 1 Processor -- start 2023-10-06T05:35:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.407+0000 7f251d99d700 1 -- start start 2023-10-06T05:35:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.408+0000 7f251d99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25180cd5e0 0x7f25180af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.408+0000 7f251d99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25180ce890 con 0x7f251815a650 2023-10-06T05:35:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.408+0000 7f251d99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251815aa60 con 0x7f25180af540 2023-10-06T05:35:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.408+0000 7f251d99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251815d890 con 0x7f25180cd5e0 2023-10-06T05:35:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f251d19c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f251815a650 0x7f25180ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38414/0 (socket says 172.21.15.33:38414) 2023-10-06T05:35:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f251d19c700 1 -- 172.21.15.33:0/1631735590 learned_addr learned my addr 172.21.15.33:0/1631735590 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f2517fff700 1 --2- 172.21.15.33:0/1631735590 >> v2:172.21.15.33:3300/0 conn(0x7f25180cd5e0 0x7f25180af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f2517fff700 1 -- 172.21.15.33:0/1631735590 >> v1:172.21.15.167:6789/0 conn(0x7f25180af540 legacy=0x7f25180cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f2517fff700 1 -- 172.21.15.33:0/1631735590 >> v1:172.21.15.33:6789/0 conn(0x7f251815a650 legacy=0x7f25180ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f2517fff700 1 -- 172.21.15.33:0/1631735590 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251815db70 con 0x7f25180cd5e0 2023-10-06T05:35:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.409+0000 7f2517fff700 1 --2- 172.21.15.33:0/1631735590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f25180af910 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f25040098e0 tx=0x7f25040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2183b32214f1a72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f25177fe700 1 -- 172.21.15.33:0/1631735590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504009e80 con 0x7f25180cd5e0 2023-10-06T05:35:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f25177fe700 1 -- 172.21.15.33:0/1631735590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250400b860 con 0x7f25180cd5e0 2023-10-06T05:35:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f25177fe700 1 -- 172.21.15.33:0/1631735590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400bbe0 con 0x7f25180cd5e0 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 -- 172.21.15.33:0/1631735590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 msgr2=0x7f25180af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 --2- 172.21.15.33:0/1631735590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f25180af910 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f25040098e0 tx=0x7f25040091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 -- 172.21.15.33:0/1631735590 shutdown_connections 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 --2- 172.21.15.33:0/1631735590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f25180af910 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 -- 172.21.15.33:0/1631735590 >> 172.21.15.33:0/1631735590 conn(0x7f251806d9b0 msgr2=0x7f251806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 -- 172.21.15.33:0/1631735590 shutdown_connections 2023-10-06T05:35:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.410+0000 7f251d99d700 1 -- 172.21.15.33:0/1631735590 wait complete. 2023-10-06T05:35:17.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.412+0000 7f251d99d700 1 Processor -- start 2023-10-06T05:35:17.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.412+0000 7f251d99d700 1 -- start start 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25180af540 0x7f25180b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f251816cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251815a650 0x7f25180b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180c8550 con 0x7f25180cd5e0 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180c87a0 con 0x7f251815a650 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25180c8a80 con 0x7f25180af540 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f2517fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f251816cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f2517fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f251816cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59662/0 (socket says 172.21.15.33:59662) 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f2517fff700 1 -- 172.21.15.33:0/3019107627 learned_addr learned my addr 172.21.15.33:0/3019107627 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251d19c700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251815a650 0x7f25180b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f251c99b700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25180af540 0x7f25180b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.413+0000 7f2517fff700 1 -- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25180af540 msgr2=0x7f25180b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2517fff700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25180af540 0x7f25180b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2517fff700 1 -- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251815a650 msgr2=0x7f25180b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2517fff700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251815a650 0x7f25180b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2517fff700 1 -- 172.21.15.33:0/3019107627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504009420 con 0x7f25180cd5e0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f251c99b700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25180af540 0x7f25180b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2517fff700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f251816cc30 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f25180c7320 tx=0x7f2504017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400bb20 con 0x7f25180cd5e0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504025420 con 0x7f25180cd5e0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25180c8d60 con 0x7f25180cd5e0 2023-10-06T05:35:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25040257c0 con 0x7f25180cd5e0 2023-10-06T05:35:17.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.414+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251816d170 con 0x7f25180cd5e0 2023-10-06T05:35:17.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.415+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25180c9260 con 0x7f25180cd5e0 2023-10-06T05:35:17.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.416+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2504023070 con 0x7f25180cd5e0 2023-10-06T05:35:17.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.417+0000 7f2514ff9700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f24fc066f30 0x7f24fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:17.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.417+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2504085ed0 con 0x7f25180cd5e0 2023-10-06T05:35:17.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.417+0000 7f251c99b700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f24fc066f30 0x7f24fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.418+0000 7f251c99b700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f24fc066f30 0x7f24fc0693e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f250c0017a0 tx=0x7f250c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.421+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2504052f40 con 0x7f25180cd5e0 2023-10-06T05:35:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:17.667+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f251804c380 con 0x7f25180cd5e0 2023-10-06T05:35:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.196+0000 7f2514ff9700 1 -- 172.21.15.33:0/3019107627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v219) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2504056560 con 0x7f25180cd5e0 2023-10-06T05:35:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f24fc066f30 msgr2=0x7f24fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f24fc066f30 0x7f24fc0693e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f250c0017a0 tx=0x7f250c009040 comp rx=0 tx=0).stop 2023-10-06T05:35:18.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 msgr2=0x7f251816cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f251816cc30 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f25180c7320 tx=0x7f2504017000 comp rx=0 tx=0).stop 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 shutdown_connections 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f24fc066f30 0x7f24fc0693e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f251815a650 0x7f25180b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.198+0000 7f251d99d700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25180af540 0x7f25180b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.199+0000 7f251d99d700 1 --2- 172.21.15.33:0/3019107627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25180cd5e0 0x7f251816cc30 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.199+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 >> 172.21.15.33:0/3019107627 conn(0x7f251806d9b0 msgr2=0x7f25180c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.199+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 shutdown_connections 2023-10-06T05:35:18.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.199+0000 7f251d99d700 1 -- 172.21.15.33:0/3019107627 wait complete. 2023-10-06T05:35:18.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.410+0000 7f0d1b19d700 1 Processor -- start 2023-10-06T05:35:18.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.410+0000 7f0d1b19d700 1 -- start start 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1b19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d1413faa0 0x7f0d1413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d140c7940 0x7f0d140c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1b19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0d1406aad0 0x7f0d1406af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1b19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140c9cb0 con 0x7f0d1413faa0 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1b19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140ca270 con 0x7f0d1406aad0 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1406b840 con 0x7f0d140c7940 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d1413faa0 0x7f0d1413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1a19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d1413faa0 0x7f0d1413fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59666/0 (socket says 172.21.15.33:59666) 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1a19b700 1 -- 172.21.15.33:0/1238977285 learned_addr learned my addr 172.21.15.33:0/1238977285 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1999a700 1 --2- 172.21.15.33:0/1238977285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d140c7940 0x7f0d140c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.412+0000 7f0d1a99c700 1 --2- 172.21.15.33:0/1238977285 >> v2:172.21.15.167:3300/0 conn(0x7f0d1406aad0 0x7f0d1406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d1a19b700 1 -- 172.21.15.33:0/1238977285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d140c7940 msgr2=0x7f0d140c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/1238977285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d140c7940 0x7f0d140c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d1a19b700 1 -- 172.21.15.33:0/1238977285 >> v2:172.21.15.167:3300/0 conn(0x7f0d1406aad0 msgr2=0x7f0d1406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/1238977285 >> v2:172.21.15.167:3300/0 conn(0x7f0d1406aad0 0x7f0d1406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d1a19b700 1 -- 172.21.15.33:0/1238977285 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1406bb20 con 0x7f0d1413faa0 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/1238977285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d1413faa0 0x7f0d1413fe70 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f0d140cc0b0 tx=0x7f0d1000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44937c194b4e88d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d19199700 1 -- 172.21.15.33:0/1238977285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1000db40 con 0x7f0d1413faa0 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d19199700 1 -- 172.21.15.33:0/1238977285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1000dca0 con 0x7f0d1413faa0 2023-10-06T05:35:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.413+0000 7f0d19199700 1 -- 172.21.15.33:0/1238977285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100196b0 con 0x7f0d1413faa0 2023-10-06T05:35:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 -- 172.21.15.33:0/1238977285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d1413faa0 msgr2=0x7f0d1413fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/1238977285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d1413faa0 0x7f0d1413fe70 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f0d140cc0b0 tx=0x7f0d1000b4f0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 -- 172.21.15.33:0/1238977285 shutdown_connections 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/1238977285 >> v2:172.21.15.167:3300/0 conn(0x7f0d1406aad0 0x7f0d1406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/1238977285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d1413faa0 0x7f0d1413fe70 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/1238977285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d140c7940 0x7f0d140c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 -- 172.21.15.33:0/1238977285 >> 172.21.15.33:0/1238977285 conn(0x7f0d1406d030 msgr2=0x7f0d1406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 -- 172.21.15.33:0/1238977285 shutdown_connections 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.414+0000 7f0d1b19d700 1 -- 172.21.15.33:0/1238977285 wait complete. 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.415+0000 7f0d1b19d700 1 Processor -- start 2023-10-06T05:35:18.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.415+0000 7f0d1b19d700 1 -- start start 2023-10-06T05:35:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.416+0000 7f0d1b19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d1406aad0 0x7f0d1415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.416+0000 7f0d1b19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 0x7f0d141598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.416+0000 7f0d1b19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d1413faa0 0x7f0d1415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.416+0000 7f0d1b19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415de20 con 0x7f0d140c7940 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.416+0000 7f0d1b19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14160630 con 0x7f0d1406aad0 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.416+0000 7f0d1b19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1413a910 con 0x7f0d1413faa0 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 0x7f0d141598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 0x7f0d141598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59668/0 (socket says 172.21.15.33:59668) 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 -- 172.21.15.33:0/3917664325 learned_addr learned my addr 172.21.15.33:0/3917664325 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d1406aad0 0x7f0d1415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1a99c700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d1413faa0 0x7f0d1415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 -- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d1413faa0 msgr2=0x7f0d1415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d1413faa0 0x7f0d1415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 -- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d1406aad0 msgr2=0x7f0d1415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d1406aad0 0x7f0d1415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 -- 172.21.15.33:0/3917664325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1000b790 con 0x7f0d140c7940 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d1406aad0 0x7f0d1415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.417+0000 7f0d1999a700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 0x7f0d141598e0 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ec80 tx=0x7f0d040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.418+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0400c9d0 con 0x7f0d140c7940 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.418+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0400cb50 con 0x7f0d140c7940 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.418+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04013470 con 0x7f0d140c7940 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.418+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1415db40 con 0x7f0d140c7940 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.418+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d141602c0 con 0x7f0d140c7940 2023-10-06T05:35:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.419+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d140698c0 con 0x7f0d140c7940 2023-10-06T05:35:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.420+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d04013860 con 0x7f0d140c7940 2023-10-06T05:35:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.421+0000 7f0d0a7fc700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0cf8066f30 0x7f0cf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.421+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d04084ad0 con 0x7f0d140c7940 2023-10-06T05:35:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.421+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0cf8066f30 0x7f0cf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.422+0000 7f0d1a19b700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0cf8066f30 0x7f0cf80693e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0d140cc0b0 tx=0x7f0d1001b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.425+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d04051b40 con 0x7f0d140c7940 2023-10-06T05:35:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.671+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d14159e80 con 0x7f0d140c7940 2023-10-06T05:35:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.671+0000 7f0d0a7fc700 1 -- 172.21.15.33:0/3917664325 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f0d04055160 con 0x7f0d140c7940 2023-10-06T05:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0cf8066f30 msgr2=0x7f0cf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0cf8066f30 0x7f0cf80693e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0d140cc0b0 tx=0x7f0d1001b000 comp rx=0 tx=0).stop 2023-10-06T05:35:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 msgr2=0x7f0d141598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 0x7f0d141598e0 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ec80 tx=0x7f0d040091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 shutdown_connections 2023-10-06T05:35:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0cf8066f30 0x7f0cf80693e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d1413faa0 0x7f0d1415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d1406aad0 0x7f0d1415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 --2- 172.21.15.33:0/3917664325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d140c7940 0x7f0d141598e0 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 >> 172.21.15.33:0/3917664325 conn(0x7f0d1406d030 msgr2=0x7f0d1406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 shutdown_connections 2023-10-06T05:35:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.674+0000 7f0d1b19d700 1 -- 172.21.15.33:0/3917664325 wait complete. 2023-10-06T05:35:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:18.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.879+0000 7f44b759e700 1 Processor -- start 2023-10-06T05:35:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.879+0000 7f44b759e700 1 -- start start 2023-10-06T05:35:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b80c6fb0 0x7f44b80c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b815abd0 con 0x7f44b80c6b30 2023-10-06T05:35:18.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b815a050 con 0x7f44b8153b10 2023-10-06T05:35:18.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b815a330 con 0x7f44b80c6fb0 2023-10-06T05:35:18.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b5d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f44b80c6b30 0x7f44b80c1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38418/0 (socket says 172.21.15.33:38418) 2023-10-06T05:35:18.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b5d9b700 1 -- 172.21.15.33:0/4067567530 learned_addr learned my addr 172.21.15.33:0/4067567530 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:18.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b6d9d700 1 --2- 172.21.15.33:0/4067567530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b80c6fb0 0x7f44b80c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963351721 0 0) 0x7f44b815abd0 con 0x7f44b80c6b30 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.881+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003570 con 0x7f44b80c6b30 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811974927 0 0) 0x7f44b815a050 con 0x7f44b8153b10 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b815abd0 con 0x7f44b8153b10 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b6d9d700 1 -- 172.21.15.33:0/4067567530 >> v1:172.21.15.167:6789/0 conn(0x7f44b8153b10 legacy=0x7f44b80c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b6d9d700 1 -- 172.21.15.33:0/4067567530 >> v1:172.21.15.33:6789/0 conn(0x7f44b80c6b30 legacy=0x7f44b80c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b6d9d700 1 -- 172.21.15.33:0/4067567530 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b815a610 con 0x7f44b80c6fb0 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b6d9d700 1 --2- 172.21.15.33:0/4067567530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b80c6fb0 0x7f44b80c1d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009e90 tx=0x7f44b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44757195e313a288 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b000e070 con 0x7f44b80c6fb0 2023-10-06T05:35:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b000bb80 con 0x7f44b80c6fb0 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.882+0000 7f44b559a700 1 -- 172.21.15.33:0/4067567530 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0012430 con 0x7f44b80c6fb0 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 -- 172.21.15.33:0/4067567530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b80c6fb0 msgr2=0x7f44b80c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 --2- 172.21.15.33:0/4067567530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b80c6fb0 0x7f44b80c1d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009e90 tx=0x7f44b0009780 comp rx=0 tx=0).stop 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 -- 172.21.15.33:0/4067567530 shutdown_connections 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 --2- 172.21.15.33:0/4067567530 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b80c6fb0 0x7f44b80c1d10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 -- 172.21.15.33:0/4067567530 >> 172.21.15.33:0/4067567530 conn(0x7f44b806c560 msgr2=0x7f44b806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 -- 172.21.15.33:0/4067567530 shutdown_connections 2023-10-06T05:35:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.883+0000 7f44b759e700 1 -- 172.21.15.33:0/4067567530 wait complete. 2023-10-06T05:35:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.884+0000 7f44b759e700 1 Processor -- start 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 -- start start 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 0x7f44b813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44b80c6fb0 0x7f44b8158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b8153b10 0x7f44b80b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b813e3d0 con 0x7f44b80c6b30 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b813f230 con 0x7f44b80c6fb0 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.885+0000 7f44b759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b813e620 con 0x7f44b8153b10 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 0x7f44b813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 0x7f44b813f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59674/0 (socket says 172.21.15.33:59674) 2023-10-06T05:35:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 -- 172.21.15.33:0/1694127345 learned_addr learned my addr 172.21.15.33:0/1694127345 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b5d9b700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44b80c6fb0 0x7f44b8158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b6d9d700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b8153b10 0x7f44b80b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 -- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b8153b10 msgr2=0x7f44b80b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b8153b10 0x7f44b80b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 -- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44b80c6fb0 msgr2=0x7f44b8158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44b80c6fb0 0x7f44b8158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 -- 172.21.15.33:0/1694127345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0009a20 con 0x7f44b80c6b30 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b6d9d700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b8153b10 0x7f44b80b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b5d9b700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44b80c6fb0 0x7f44b8158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44b659c700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 0x7f44b813f6c0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f44a800c7c0 tx=0x7f44a8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.886+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8010510 con 0x7f44b80c6b30 2023-10-06T05:35:18.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.887+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a8010690 con 0x7f44b80c6b30 2023-10-06T05:35:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.887+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b813e900 con 0x7f44b80c6b30 2023-10-06T05:35:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.887+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8010a50 con 0x7f44b80c6b30 2023-10-06T05:35:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.887+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b813eec0 con 0x7f44b80c6b30 2023-10-06T05:35:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.888+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b813fc00 con 0x7f44b80c6b30 2023-10-06T05:35:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.889+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44a801b400 con 0x7f44b80c6b30 2023-10-06T05:35:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.890+0000 7f44a67fc700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4498066f30 0x7f44980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.890+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f44a8084360 con 0x7f44b80c6b30 2023-10-06T05:35:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.890+0000 7f44b5d9b700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4498066f30 0x7f44980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.891+0000 7f44b5d9b700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4498066f30 0x7f44980693e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f44ac007fd0 tx=0x7f44ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:18.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:18.894+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44a8051470 con 0x7f44b80c6b30 2023-10-06T05:35:19.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.099+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44b813f510 con 0x7f44b80c6b30 2023-10-06T05:35:19.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.100+0000 7f44a67fc700 1 -- 172.21.15.33:0/1694127345 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v282) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f44a8054a90 con 0x7f44b80c6b30 2023-10-06T05:35:19.102 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:19.103 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":282,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:15.940527+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:35:12.826592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":42,"pool_name":"cephfs_data","create_time":"2023-10-06T05:35:13.627133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"282","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:19.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.102+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4498066f30 msgr2=0x7f44980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.102+0000 7f44b759e700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4498066f30 0x7f44980693e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f44ac007fd0 tx=0x7f44ac009040 comp rx=0 tx=0).stop 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 msgr2=0x7f44b813f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 0x7f44b813f6c0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f44a800c7c0 tx=0x7f44a8007e10 comp rx=0 tx=0).stop 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 shutdown_connections 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4498066f30 0x7f44980693e0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f44b8153b10 0x7f44b80b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b80c6b30 0x7f44b813f6c0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 --2- 172.21.15.33:0/1694127345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44b80c6fb0 0x7f44b8158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 >> 172.21.15.33:0/1694127345 conn(0x7f44b806c560 msgr2=0x7f44b806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 shutdown_connections 2023-10-06T05:35:19.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.103+0000 7f44b759e700 1 -- 172.21.15.33:0/1694127345 wait complete. 2023-10-06T05:35:19.122 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:35:19.122 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:35:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.307+0000 7ff4be59d700 1 Processor -- start 2023-10-06T05:35:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.308+0000 7ff4be59d700 1 -- start start 2023-10-06T05:35:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4be59d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4b813faa0 0x7ff4b813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4be59d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff4b80c7940 0x7ff4b80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4be59d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80c9cb0 con 0x7ff4b806aad0 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4be59d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b806b640 con 0x7ff4b813faa0 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4be59d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80ca270 con 0x7ff4b80c7940 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4bd59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4b813faa0 0x7ff4b813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4bd59b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4b813faa0 0x7ff4b813fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59688/0 (socket says 172.21.15.33:59688) 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4bd59b700 1 -- 172.21.15.33:0/2523946710 learned_addr learned my addr 172.21.15.33:0/2523946710 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.309+0000 7ff4bcd9a700 1 --2- 172.21.15.33:0/2523946710 >> v2:172.21.15.167:3300/0 conn(0x7ff4b80c7940 0x7ff4b80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4bd59b700 1 -- 172.21.15.33:0/2523946710 >> v2:172.21.15.167:3300/0 conn(0x7ff4b80c7940 msgr2=0x7ff4b80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4bd59b700 1 --2- 172.21.15.33:0/2523946710 >> v2:172.21.15.167:3300/0 conn(0x7ff4b80c7940 0x7ff4b80c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4bd59b700 1 -- 172.21.15.33:0/2523946710 >> v1:172.21.15.167:6789/0 conn(0x7ff4b806aad0 legacy=0x7ff4b806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4bd59b700 1 -- 172.21.15.33:0/2523946710 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b806b840 con 0x7ff4b813faa0 2023-10-06T05:35:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4bd59b700 1 --2- 172.21.15.33:0/2523946710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b813faa0 0x7ff4b813fe70 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80cbef0 tx=0x7ff4b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1969fe6b95d4e94b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4affff700 1 -- 172.21.15.33:0/2523946710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400b860 con 0x7ff4b813faa0 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4affff700 1 -- 172.21.15.33:0/2523946710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b400b9c0 con 0x7ff4b813faa0 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.310+0000 7ff4affff700 1 -- 172.21.15.33:0/2523946710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400bd60 con 0x7ff4b813faa0 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 -- 172.21.15.33:0/2523946710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b813faa0 msgr2=0x7ff4b813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 --2- 172.21.15.33:0/2523946710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b813faa0 0x7ff4b813fe70 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80cbef0 tx=0x7ff4b4009250 comp rx=0 tx=0).stop 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 -- 172.21.15.33:0/2523946710 shutdown_connections 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 --2- 172.21.15.33:0/2523946710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b813faa0 0x7ff4b813fe70 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 --2- 172.21.15.33:0/2523946710 >> v2:172.21.15.167:3300/0 conn(0x7ff4b80c7940 0x7ff4b80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 -- 172.21.15.33:0/2523946710 >> 172.21.15.33:0/2523946710 conn(0x7ff4b806d030 msgr2=0x7ff4b806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 -- 172.21.15.33:0/2523946710 shutdown_connections 2023-10-06T05:35:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.311+0000 7ff4be59d700 1 -- 172.21.15.33:0/2523946710 wait complete. 2023-10-06T05:35:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.312+0000 7ff4be59d700 1 Processor -- start 2023-10-06T05:35:19.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.313+0000 7ff4be59d700 1 -- start start 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.313+0000 7ff4be59d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 0x7ff4b813c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4be59d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b80c7940 0x7ff4b813df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4be59d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b813faa0 0x7ff4b80aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4be59d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80afa90 con 0x7ff4b806aad0 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4be59d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80af480 con 0x7ff4b813faa0 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4be59d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b813e920 con 0x7ff4b80c7940 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 0x7ff4b813c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 0x7ff4b813c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59698/0 (socket says 172.21.15.33:59698) 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 -- 172.21.15.33:0/3143769414 learned_addr learned my addr 172.21.15.33:0/3143769414 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bdd9c700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b813faa0 0x7ff4b80aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bcd9a700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b80c7940 0x7ff4b813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 -- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b80c7940 msgr2=0x7ff4b813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b80c7940 0x7ff4b813df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 -- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b813faa0 msgr2=0x7ff4b80aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b813faa0 0x7ff4b80aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 -- 172.21.15.33:0/3143769414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40094f0 con 0x7ff4b806aad0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.314+0000 7ff4bd59b700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 0x7ff4b813c0d0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80719c0 tx=0x7ff4b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.315+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4011e60 con 0x7ff4b806aad0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.315+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b401a630 con 0x7ff4b806aad0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.315+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401a9f0 con 0x7ff4b806aad0 2023-10-06T05:35:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.315+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b813ec00 con 0x7ff4b806aad0 2023-10-06T05:35:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.315+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b813c610 con 0x7ff4b806aad0 2023-10-06T05:35:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.316+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b813c830 con 0x7ff4b806aad0 2023-10-06T05:35:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.317+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4b402d440 con 0x7ff4b806aad0 2023-10-06T05:35:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.318+0000 7ff4ad7fa700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff49c066f30 0x7ff49c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.318+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff4b4086310 con 0x7ff4b806aad0 2023-10-06T05:35:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.318+0000 7ff4bcd9a700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff49c066f30 0x7ff49c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.319+0000 7ff4bcd9a700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff49c066f30 0x7ff49c0693e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8138880 tx=0x7ff4a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:19.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.322+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4b4053380 con 0x7ff4b806aad0 2023-10-06T05:35:19.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.518+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4b80afbd0 con 0x7ff49c066f30 2023-10-06T05:35:19.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.529+0000 7ff4ad7fa700 1 -- 172.21.15.33:0/3143769414 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311626 (secure 0 0 0) 0x7ff4a800f6e0 con 0x7ff49c066f30 2023-10-06T05:35:19.535 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff49c066f30 msgr2=0x7ff49c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff49c066f30 0x7ff49c0693e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8138880 tx=0x7ff4a8008040 comp rx=0 tx=0).stop 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 msgr2=0x7ff4b813c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 0x7ff4b813c0d0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80719c0 tx=0x7ff4b4017000 comp rx=0 tx=0).stop 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 shutdown_connections 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff49c066f30 0x7ff49c0693e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b813faa0 0x7ff4b80aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b806aad0 0x7ff4b813c0d0 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 --2- 172.21.15.33:0/3143769414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4b80c7940 0x7ff4b813df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 >> 172.21.15.33:0/3143769414 conn(0x7ff4b806d030 msgr2=0x7ff4b80cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:19.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 shutdown_connections 2023-10-06T05:35:19.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.533+0000 7ff4be59d700 1 -- 172.21.15.33:0/3143769414 wait complete. 2023-10-06T05:35:19.539 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:35:19.553 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":908,"stamp":"2023-10-06T05:35:18.141481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2468924,"kb_used_data":29180,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752505796,"statfs":{"total":773094113280,"available":770565935104,"internally_reserved":0,"allocated":29880320,"data_stored":14654515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.996291"},"pg_stats":[{"pgid":"42.3f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.954845+0000","last_change":"2023-10-06T05:35:14.952704+0000","last_active":"2023-10-06T05:35:14.954845+0000","last_peered":"2023-10-06T05:35:14.954845+0000","last_clean":"2023-10-06T05:35:14.954845+0000","last_became_active":"2023-10-06T05:35:14.952420+0000","last_became_peered":"2023-10-06T05:35:14.952420+0000","last_unstale":"2023-10-06T05:35:14.954845+0000","last_undegraded":"2023-10-06T05:35:14.954845+0000","last_fullsized":"2023-10-06T05:35:14.954845+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:44.558068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970965+0000","last_change":"2023-10-06T05:35:14.965791+0000","last_active":"2023-10-06T05:35:14.970965+0000","last_peered":"2023-10-06T05:35:14.970965+0000","last_clean":"2023-10-06T05:35:14.970965+0000","last_became_active":"2023-10-06T05:35:14.965458+0000","last_became_peered":"2023-10-06T05:35:14.965458+0000","last_unstale":"2023-10-06T05:35:14.970965+0000","last_undegraded":"2023-10-06T05:35:14.970965+0000","last_fullsized":"2023-10-06T05:35:14.970965+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:10.311289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.3d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959875+0000","last_change":"2023-10-06T05:35:14.956773+0000","last_active":"2023-10-06T05:35:14.959875+0000","last_peered":"2023-10-06T05:35:14.959875+0000","last_clean":"2023-10-06T05:35:14.959875+0000","last_became_active":"2023-10-06T05:35:14.956495+0000","last_became_peered":"2023-10-06T05:35:14.956495+0000","last_unstale":"2023-10-06T05:35:14.959875+0000","last_undegraded":"2023-10-06T05:35:14.959875+0000","last_fullsized":"2023-10-06T05:35:14.959875+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:12.470072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959801+0000","last_change":"2023-10-06T05:35:14.956046+0000","last_active":"2023-10-06T05:35:14.959801+0000","last_peered":"2023-10-06T05:35:14.959801+0000","last_clean":"2023-10-06T05:35:14.959801+0000","last_became_active":"2023-10-06T05:35:14.955777+0000","last_became_peered":"2023-10-06T05:35:14.955777+0000","last_unstale":"2023-10-06T05:35:14.959801+0000","last_undegraded":"2023-10-06T05:35:14.959801+0000","last_fullsized":"2023-10-06T05:35:14.959801+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:03.786775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964607+0000","last_change":"2023-10-06T05:35:14.958122+0000","last_active":"2023-10-06T05:35:14.964607+0000","last_peered":"2023-10-06T05:35:14.964607+0000","last_clean":"2023-10-06T05:35:14.964607+0000","last_became_active":"2023-10-06T05:35:14.957817+0000","last_became_peered":"2023-10-06T05:35:14.957817+0000","last_unstale":"2023-10-06T05:35:14.964607+0000","last_undegraded":"2023-10-06T05:35:14.964607+0000","last_fullsized":"2023-10-06T05:35:14.964607+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:36.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.3a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.969066+0000","last_change":"2023-10-06T05:35:14.961007+0000","last_active":"2023-10-06T05:35:14.969066+0000","last_peered":"2023-10-06T05:35:14.969066+0000","last_clean":"2023-10-06T05:35:14.969066+0000","last_became_active":"2023-10-06T05:35:14.960740+0000","last_became_peered":"2023-10-06T05:35:14.960740+0000","last_unstale":"2023-10-06T05:35:14.969066+0000","last_undegraded":"2023-10-06T05:35:14.969066+0000","last_fullsized":"2023-10-06T05:35:14.969066+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:46.539643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.39","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966988+0000","last_change":"2023-10-06T05:35:14.963285+0000","last_active":"2023-10-06T05:35:14.966988+0000","last_peered":"2023-10-06T05:35:14.966988+0000","last_clean":"2023-10-06T05:35:14.966988+0000","last_became_active":"2023-10-06T05:35:14.963035+0000","last_became_peered":"2023-10-06T05:35:14.963035+0000","last_unstale":"2023-10-06T05:35:14.966988+0000","last_undegraded":"2023-10-06T05:35:14.966988+0000","last_fullsized":"2023-10-06T05:35:14.966988+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:36.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.38","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959652+0000","last_change":"2023-10-06T05:35:14.953368+0000","last_active":"2023-10-06T05:35:14.959652+0000","last_peered":"2023-10-06T05:35:14.959652+0000","last_clean":"2023-10-06T05:35:14.959652+0000","last_became_active":"2023-10-06T05:35:14.953104+0000","last_became_peered":"2023-10-06T05:35:14.953104+0000","last_unstale":"2023-10-06T05:35:14.959652+0000","last_undegraded":"2023-10-06T05:35:14.959652+0000","last_fullsized":"2023-10-06T05:35:14.959652+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:24.776553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.37","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962374+0000","last_change":"2023-10-06T05:35:14.958551+0000","last_active":"2023-10-06T05:35:14.962374+0000","last_peered":"2023-10-06T05:35:14.962374+0000","last_clean":"2023-10-06T05:35:14.962374+0000","last_became_active":"2023-10-06T05:35:14.958287+0000","last_became_peered":"2023-10-06T05:35:14.958287+0000","last_unstale":"2023-10-06T05:35:14.962374+0000","last_undegraded":"2023-10-06T05:35:14.962374+0000","last_fullsized":"2023-10-06T05:35:14.962374+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:18.513063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.36","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962463+0000","last_change":"2023-10-06T05:35:14.960474+0000","last_active":"2023-10-06T05:35:14.962463+0000","last_peered":"2023-10-06T05:35:14.962463+0000","last_clean":"2023-10-06T05:35:14.962463+0000","last_became_active":"2023-10-06T05:35:14.960221+0000","last_became_peered":"2023-10-06T05:35:14.960221+0000","last_unstale":"2023-10-06T05:35:14.962463+0000","last_undegraded":"2023-10-06T05:35:14.962463+0000","last_fullsized":"2023-10-06T05:35:14.962463+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:18.649823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.35","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971177+0000","last_change":"2023-10-06T05:35:14.966264+0000","last_active":"2023-10-06T05:35:14.971177+0000","last_peered":"2023-10-06T05:35:14.971177+0000","last_clean":"2023-10-06T05:35:14.971177+0000","last_became_active":"2023-10-06T05:35:14.965330+0000","last_became_peered":"2023-10-06T05:35:14.965330+0000","last_unstale":"2023-10-06T05:35:14.971177+0000","last_undegraded":"2023-10-06T05:35:14.971177+0000","last_fullsized":"2023-10-06T05:35:14.971177+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:44.751891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.34","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966812+0000","last_change":"2023-10-06T05:35:14.960305+0000","last_active":"2023-10-06T05:35:14.966812+0000","last_peered":"2023-10-06T05:35:14.966812+0000","last_clean":"2023-10-06T05:35:14.966812+0000","last_became_active":"2023-10-06T05:35:14.959652+0000","last_became_peered":"2023-10-06T05:35:14.959652+0000","last_unstale":"2023-10-06T05:35:14.966812+0000","last_undegraded":"2023-10-06T05:35:14.966812+0000","last_fullsized":"2023-10-06T05:35:14.966812+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:45.564253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.33","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964508+0000","last_change":"2023-10-06T05:35:14.959792+0000","last_active":"2023-10-06T05:35:14.964508+0000","last_peered":"2023-10-06T05:35:14.964508+0000","last_clean":"2023-10-06T05:35:14.964508+0000","last_became_active":"2023-10-06T05:35:14.959533+0000","last_became_peered":"2023-10-06T05:35:14.959533+0000","last_unstale":"2023-10-06T05:35:14.964508+0000","last_undegraded":"2023-10-06T05:35:14.964508+0000","last_fullsized":"2023-10-06T05:35:14.964508+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:41.072798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.32","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971328+0000","last_change":"2023-10-06T05:35:14.963155+0000","last_active":"2023-10-06T05:35:14.971328+0000","last_peered":"2023-10-06T05:35:14.971328+0000","last_clean":"2023-10-06T05:35:14.971328+0000","last_became_active":"2023-10-06T05:35:14.962865+0000","last_became_peered":"2023-10-06T05:35:14.962865+0000","last_unstale":"2023-10-06T05:35:14.971328+0000","last_undegraded":"2023-10-06T05:35:14.971328+0000","last_fullsized":"2023-10-06T05:35:14.971328+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:00.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.31","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971064+0000","last_change":"2023-10-06T05:35:14.965653+0000","last_active":"2023-10-06T05:35:14.971064+0000","last_peered":"2023-10-06T05:35:14.971064+0000","last_clean":"2023-10-06T05:35:14.971064+0000","last_became_active":"2023-10-06T05:35:14.965354+0000","last_became_peered":"2023-10-06T05:35:14.965354+0000","last_unstale":"2023-10-06T05:35:14.971064+0000","last_undegraded":"2023-10-06T05:35:14.971064+0000","last_fullsized":"2023-10-06T05:35:14.971064+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:02.954439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.30","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971379+0000","last_change":"2023-10-06T05:35:14.964352+0000","last_active":"2023-10-06T05:35:14.971379+0000","last_peered":"2023-10-06T05:35:14.971379+0000","last_clean":"2023-10-06T05:35:14.971379+0000","last_became_active":"2023-10-06T05:35:14.963454+0000","last_became_peered":"2023-10-06T05:35:14.963454+0000","last_unstale":"2023-10-06T05:35:14.971379+0000","last_undegraded":"2023-10-06T05:35:14.971379+0000","last_fullsized":"2023-10-06T05:35:14.971379+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:41.072798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.27","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961271+0000","last_change":"2023-10-06T05:35:14.957895+0000","last_active":"2023-10-06T05:35:14.961271+0000","last_peered":"2023-10-06T05:35:14.961271+0000","last_clean":"2023-10-06T05:35:14.961271+0000","last_became_active":"2023-10-06T05:35:14.955242+0000","last_became_peered":"2023-10-06T05:35:14.955242+0000","last_unstale":"2023-10-06T05:35:14.961271+0000","last_undegraded":"2023-10-06T05:35:14.961271+0000","last_fullsized":"2023-10-06T05:35:14.961271+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:58.197912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.26","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975932+0000","last_change":"2023-10-06T05:35:14.973178+0000","last_active":"2023-10-06T05:35:14.975932+0000","last_peered":"2023-10-06T05:35:14.975932+0000","last_clean":"2023-10-06T05:35:14.975932+0000","last_became_active":"2023-10-06T05:35:14.972889+0000","last_became_peered":"2023-10-06T05:35:14.972889+0000","last_unstale":"2023-10-06T05:35:14.975932+0000","last_undegraded":"2023-10-06T05:35:14.975932+0000","last_fullsized":"2023-10-06T05:35:14.975932+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:50.127598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.25","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971231+0000","last_change":"2023-10-06T05:35:14.968543+0000","last_active":"2023-10-06T05:35:14.971231+0000","last_peered":"2023-10-06T05:35:14.971231+0000","last_clean":"2023-10-06T05:35:14.971231+0000","last_became_active":"2023-10-06T05:35:14.968275+0000","last_became_peered":"2023-10-06T05:35:14.968275+0000","last_unstale":"2023-10-06T05:35:14.971231+0000","last_undegraded":"2023-10-06T05:35:14.971231+0000","last_fullsized":"2023-10-06T05:35:14.971231+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:00.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.24","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961160+0000","last_change":"2023-10-06T05:35:14.958402+0000","last_active":"2023-10-06T05:35:14.961160+0000","last_peered":"2023-10-06T05:35:14.961160+0000","last_clean":"2023-10-06T05:35:14.961160+0000","last_became_active":"2023-10-06T05:35:14.958146+0000","last_became_peered":"2023-10-06T05:35:14.958146+0000","last_unstale":"2023-10-06T05:35:14.961160+0000","last_undegraded":"2023-10-06T05:35:14.961160+0000","last_fullsized":"2023-10-06T05:35:14.961160+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:29.466625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.23","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.957041+0000","last_change":"2023-10-06T05:35:14.955594+0000","last_active":"2023-10-06T05:35:14.957041+0000","last_peered":"2023-10-06T05:35:14.957041+0000","last_clean":"2023-10-06T05:35:14.957041+0000","last_became_active":"2023-10-06T05:35:14.954688+0000","last_became_peered":"2023-10-06T05:35:14.954688+0000","last_unstale":"2023-10-06T05:35:14.957041+0000","last_undegraded":"2023-10-06T05:35:14.957041+0000","last_fullsized":"2023-10-06T05:35:14.957041+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:24.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.22","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964306+0000","last_change":"2023-10-06T05:35:14.960038+0000","last_active":"2023-10-06T05:35:14.964306+0000","last_peered":"2023-10-06T05:35:14.964306+0000","last_clean":"2023-10-06T05:35:14.964306+0000","last_became_active":"2023-10-06T05:35:14.958611+0000","last_became_peered":"2023-10-06T05:35:14.958611+0000","last_unstale":"2023-10-06T05:35:14.964306+0000","last_undegraded":"2023-10-06T05:35:14.964306+0000","last_fullsized":"2023-10-06T05:35:14.964306+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:09.325090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.21","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.958791+0000","last_change":"2023-10-06T05:35:14.954526+0000","last_active":"2023-10-06T05:35:14.958791+0000","last_peered":"2023-10-06T05:35:14.958791+0000","last_clean":"2023-10-06T05:35:14.958791+0000","last_became_active":"2023-10-06T05:35:14.954274+0000","last_became_peered":"2023-10-06T05:35:14.954274+0000","last_unstale":"2023-10-06T05:35:14.958791+0000","last_undegraded":"2023-10-06T05:35:14.958791+0000","last_fullsized":"2023-10-06T05:35:14.958791+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:24.391202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.20","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975783+0000","last_change":"2023-10-06T05:35:14.971634+0000","last_active":"2023-10-06T05:35:14.975783+0000","last_peered":"2023-10-06T05:35:14.975783+0000","last_clean":"2023-10-06T05:35:14.975783+0000","last_became_active":"2023-10-06T05:35:14.971381+0000","last_became_peered":"2023-10-06T05:35:14.971381+0000","last_unstale":"2023-10-06T05:35:14.975783+0000","last_undegraded":"2023-10-06T05:35:14.975783+0000","last_fullsized":"2023-10-06T05:35:14.975783+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:35.929197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.1c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961144+0000","last_change":"2023-10-06T05:35:14.957821+0000","last_active":"2023-10-06T05:35:14.961144+0000","last_peered":"2023-10-06T05:35:14.961144+0000","last_clean":"2023-10-06T05:35:14.961144+0000","last_became_active":"2023-10-06T05:35:14.957569+0000","last_became_peered":"2023-10-06T05:35:14.957569+0000","last_unstale":"2023-10-06T05:35:14.961144+0000","last_undegraded":"2023-10-06T05:35:14.961144+0000","last_fullsized":"2023-10-06T05:35:14.961144+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:35.929197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1f","version":"281'4","reported_seq":14,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178664+0000","last_change":"2023-10-06T05:35:14.171157+0000","last_active":"2023-10-06T05:35:15.178664+0000","last_peered":"2023-10-06T05:35:15.178664+0000","last_clean":"2023-10-06T05:35:15.178664+0000","last_became_active":"2023-10-06T05:35:14.170854+0000","last_became_peered":"2023-10-06T05:35:14.170854+0000","last_unstale":"2023-10-06T05:35:15.178664+0000","last_undegraded":"2023-10-06T05:35:15.178664+0000","last_fullsized":"2023-10-06T05:35:15.178664+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:09.305299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.1d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959559+0000","last_change":"2023-10-06T05:35:14.956972+0000","last_active":"2023-10-06T05:35:14.959559+0000","last_peered":"2023-10-06T05:35:14.959559+0000","last_clean":"2023-10-06T05:35:14.959559+0000","last_became_active":"2023-10-06T05:35:14.956707+0000","last_became_peered":"2023-10-06T05:35:14.956707+0000","last_unstale":"2023-10-06T05:35:14.959559+0000","last_undegraded":"2023-10-06T05:35:14.959559+0000","last_fullsized":"2023-10-06T05:35:14.959559+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:36.015570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.1e","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178480+0000","last_change":"2023-10-06T05:35:14.160242+0000","last_active":"2023-10-06T05:35:15.178480+0000","last_peered":"2023-10-06T05:35:15.178480+0000","last_clean":"2023-10-06T05:35:15.178480+0000","last_became_active":"2023-10-06T05:35:14.159956+0000","last_became_peered":"2023-10-06T05:35:14.159956+0000","last_unstale":"2023-10-06T05:35:15.178480+0000","last_undegraded":"2023-10-06T05:35:15.178480+0000","last_fullsized":"2023-10-06T05:35:15.178480+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:04.697343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952708+0000","last_change":"2023-10-06T05:35:14.949526+0000","last_active":"2023-10-06T05:35:14.952708+0000","last_peered":"2023-10-06T05:35:14.952708+0000","last_clean":"2023-10-06T05:35:14.952708+0000","last_became_active":"2023-10-06T05:35:14.949245+0000","last_became_peered":"2023-10-06T05:35:14.949245+0000","last_unstale":"2023-10-06T05:35:14.952708+0000","last_undegraded":"2023-10-06T05:35:14.952708+0000","last_fullsized":"2023-10-06T05:35:14.952708+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:47.574339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1d","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.177531+0000","last_change":"2023-10-06T05:35:14.170331+0000","last_active":"2023-10-06T05:35:15.177531+0000","last_peered":"2023-10-06T05:35:15.177531+0000","last_clean":"2023-10-06T05:35:15.177531+0000","last_became_active":"2023-10-06T05:35:14.170013+0000","last_became_peered":"2023-10-06T05:35:14.170013+0000","last_unstale":"2023-10-06T05:35:15.177531+0000","last_undegraded":"2023-10-06T05:35:15.177531+0000","last_fullsized":"2023-10-06T05:35:15.177531+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:02.353698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.1f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970962+0000","last_change":"2023-10-06T05:35:14.965479+0000","last_active":"2023-10-06T05:35:14.970962+0000","last_peered":"2023-10-06T05:35:14.970962+0000","last_clean":"2023-10-06T05:35:14.970962+0000","last_became_active":"2023-10-06T05:35:14.965206+0000","last_became_peered":"2023-10-06T05:35:14.965206+0000","last_unstale":"2023-10-06T05:35:14.970962+0000","last_undegraded":"2023-10-06T05:35:14.970962+0000","last_fullsized":"2023-10-06T05:35:14.970962+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:03.510004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1c","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.175835+0000","last_change":"2023-10-06T05:35:14.169581+0000","last_active":"2023-10-06T05:35:15.175835+0000","last_peered":"2023-10-06T05:35:15.175835+0000","last_clean":"2023-10-06T05:35:15.175835+0000","last_became_active":"2023-10-06T05:35:14.169271+0000","last_became_peered":"2023-10-06T05:35:14.169271+0000","last_unstale":"2023-10-06T05:35:15.175835+0000","last_undegraded":"2023-10-06T05:35:15.175835+0000","last_fullsized":"2023-10-06T05:35:15.175835+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:35.464792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.18","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.958024+0000","last_change":"2023-10-06T05:35:14.955453+0000","last_active":"2023-10-06T05:35:14.958024+0000","last_peered":"2023-10-06T05:35:14.958024+0000","last_clean":"2023-10-06T05:35:14.958024+0000","last_became_active":"2023-10-06T05:35:14.955193+0000","last_became_peered":"2023-10-06T05:35:14.955193+0000","last_unstale":"2023-10-06T05:35:14.958024+0000","last_undegraded":"2023-10-06T05:35:14.958024+0000","last_fullsized":"2023-10-06T05:35:14.958024+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:28.395765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1b","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178027+0000","last_change":"2023-10-06T05:35:14.169263+0000","last_active":"2023-10-06T05:35:15.178027+0000","last_peered":"2023-10-06T05:35:15.178027+0000","last_clean":"2023-10-06T05:35:15.178027+0000","last_became_active":"2023-10-06T05:35:14.168966+0000","last_became_peered":"2023-10-06T05:35:14.168966+0000","last_unstale":"2023-10-06T05:35:15.178027+0000","last_undegraded":"2023-10-06T05:35:15.178027+0000","last_fullsized":"2023-10-06T05:35:15.178027+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:37.206845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.19","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.974319+0000","last_change":"2023-10-06T05:35:14.969094+0000","last_active":"2023-10-06T05:35:14.974319+0000","last_peered":"2023-10-06T05:35:14.974319+0000","last_clean":"2023-10-06T05:35:14.974319+0000","last_became_active":"2023-10-06T05:35:14.968737+0000","last_became_peered":"2023-10-06T05:35:14.968737+0000","last_unstale":"2023-10-06T05:35:14.974319+0000","last_undegraded":"2023-10-06T05:35:14.974319+0000","last_fullsized":"2023-10-06T05:35:14.974319+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:09.057325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1a","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.176478+0000","last_change":"2023-10-06T05:35:14.172187+0000","last_active":"2023-10-06T05:35:15.176478+0000","last_peered":"2023-10-06T05:35:15.176478+0000","last_clean":"2023-10-06T05:35:15.176478+0000","last_became_active":"2023-10-06T05:35:14.171886+0000","last_became_peered":"2023-10-06T05:35:14.171886+0000","last_unstale":"2023-10-06T05:35:15.176478+0000","last_undegraded":"2023-10-06T05:35:15.176478+0000","last_fullsized":"2023-10-06T05:35:15.176478+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:17.296461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.1a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952778+0000","last_change":"2023-10-06T05:35:14.949549+0000","last_active":"2023-10-06T05:35:14.952778+0000","last_peered":"2023-10-06T05:35:14.952778+0000","last_clean":"2023-10-06T05:35:14.952778+0000","last_became_active":"2023-10-06T05:35:14.949278+0000","last_became_peered":"2023-10-06T05:35:14.949278+0000","last_unstale":"2023-10-06T05:35:14.952778+0000","last_undegraded":"2023-10-06T05:35:14.952778+0000","last_fullsized":"2023-10-06T05:35:14.952778+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:11.278905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.19","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.948555+0000","last_change":"2023-10-06T05:35:14.157269+0000","last_active":"2023-10-06T05:35:14.948555+0000","last_peered":"2023-10-06T05:35:14.948555+0000","last_clean":"2023-10-06T05:35:14.948555+0000","last_became_active":"2023-10-06T05:35:14.157003+0000","last_became_peered":"2023-10-06T05:35:14.157003+0000","last_unstale":"2023-10-06T05:35:14.948555+0000","last_undegraded":"2023-10-06T05:35:14.948555+0000","last_fullsized":"2023-10-06T05:35:14.948555+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:54.008885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.1b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.953534+0000","last_change":"2023-10-06T05:35:14.951536+0000","last_active":"2023-10-06T05:35:14.953534+0000","last_peered":"2023-10-06T05:35:14.953534+0000","last_clean":"2023-10-06T05:35:14.953534+0000","last_became_active":"2023-10-06T05:35:14.951195+0000","last_became_peered":"2023-10-06T05:35:14.951195+0000","last_unstale":"2023-10-06T05:35:14.953534+0000","last_undegraded":"2023-10-06T05:35:14.953534+0000","last_fullsized":"2023-10-06T05:35:14.953534+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:24.542923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.18","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952910+0000","last_change":"2023-10-06T05:35:14.171591+0000","last_active":"2023-10-06T05:35:14.952910+0000","last_peered":"2023-10-06T05:35:14.952910+0000","last_clean":"2023-10-06T05:35:14.952910+0000","last_became_active":"2023-10-06T05:35:14.170154+0000","last_became_peered":"2023-10-06T05:35:14.170154+0000","last_unstale":"2023-10-06T05:35:14.952910+0000","last_undegraded":"2023-10-06T05:35:14.952910+0000","last_fullsized":"2023-10-06T05:35:14.952910+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:16.141953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.14","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966832+0000","last_change":"2023-10-06T05:35:14.960359+0000","last_active":"2023-10-06T05:35:14.966832+0000","last_peered":"2023-10-06T05:35:14.966832+0000","last_clean":"2023-10-06T05:35:14.966832+0000","last_became_active":"2023-10-06T05:35:14.959342+0000","last_became_peered":"2023-10-06T05:35:14.959342+0000","last_unstale":"2023-10-06T05:35:14.966832+0000","last_undegraded":"2023-10-06T05:35:14.966832+0000","last_fullsized":"2023-10-06T05:35:14.966832+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:43.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.17","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952030+0000","last_change":"2023-10-06T05:35:14.171324+0000","last_active":"2023-10-06T05:35:14.952030+0000","last_peered":"2023-10-06T05:35:14.952030+0000","last_clean":"2023-10-06T05:35:14.952030+0000","last_became_active":"2023-10-06T05:35:14.171058+0000","last_became_peered":"2023-10-06T05:35:14.171058+0000","last_unstale":"2023-10-06T05:35:14.952030+0000","last_undegraded":"2023-10-06T05:35:14.952030+0000","last_fullsized":"2023-10-06T05:35:14.952030+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:38.076119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.15","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975711+0000","last_change":"2023-10-06T05:35:14.971000+0000","last_active":"2023-10-06T05:35:14.975711+0000","last_peered":"2023-10-06T05:35:14.975711+0000","last_clean":"2023-10-06T05:35:14.975711+0000","last_became_active":"2023-10-06T05:35:14.970732+0000","last_became_peered":"2023-10-06T05:35:14.970732+0000","last_unstale":"2023-10-06T05:35:14.975711+0000","last_undegraded":"2023-10-06T05:35:14.975711+0000","last_fullsized":"2023-10-06T05:35:14.975711+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:58.534732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.16","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.957185+0000","last_change":"2023-10-06T05:35:14.169746+0000","last_active":"2023-10-06T05:35:14.957185+0000","last_peered":"2023-10-06T05:35:14.957185+0000","last_clean":"2023-10-06T05:35:14.957185+0000","last_became_active":"2023-10-06T05:35:14.169452+0000","last_became_peered":"2023-10-06T05:35:14.169452+0000","last_unstale":"2023-10-06T05:35:14.957185+0000","last_undegraded":"2023-10-06T05:35:14.957185+0000","last_fullsized":"2023-10-06T05:35:14.957185+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:54.692971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.16","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961273+0000","last_change":"2023-10-06T05:35:14.958582+0000","last_active":"2023-10-06T05:35:14.961273+0000","last_peered":"2023-10-06T05:35:14.961273+0000","last_clean":"2023-10-06T05:35:14.961273+0000","last_became_active":"2023-10-06T05:35:14.958325+0000","last_became_peered":"2023-10-06T05:35:14.958325+0000","last_unstale":"2023-10-06T05:35:14.961273+0000","last_undegraded":"2023-10-06T05:35:14.961273+0000","last_fullsized":"2023-10-06T05:35:14.961273+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:02.773613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.15","version":"281'3","reported_seq":13,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180569+0000","last_change":"2023-10-06T05:35:14.162068+0000","last_active":"2023-10-06T05:35:15.180569+0000","last_peered":"2023-10-06T05:35:15.180569+0000","last_clean":"2023-10-06T05:35:15.180569+0000","last_became_active":"2023-10-06T05:35:14.161765+0000","last_became_peered":"2023-10-06T05:35:14.161765+0000","last_unstale":"2023-10-06T05:35:15.180569+0000","last_undegraded":"2023-10-06T05:35:15.180569+0000","last_fullsized":"2023-10-06T05:35:15.180569+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:07.056603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.17","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970982+0000","last_change":"2023-10-06T05:35:14.966153+0000","last_active":"2023-10-06T05:35:14.970982+0000","last_peered":"2023-10-06T05:35:14.970982+0000","last_clean":"2023-10-06T05:35:14.970982+0000","last_became_active":"2023-10-06T05:35:14.965885+0000","last_became_peered":"2023-10-06T05:35:14.965885+0000","last_unstale":"2023-10-06T05:35:14.970982+0000","last_undegraded":"2023-10-06T05:35:14.970982+0000","last_fullsized":"2023-10-06T05:35:14.970982+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:41.287166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.14","version":"281'6","reported_seq":16,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180769+0000","last_change":"2023-10-06T05:35:14.170638+0000","last_active":"2023-10-06T05:35:15.180769+0000","last_peered":"2023-10-06T05:35:15.180769+0000","last_clean":"2023-10-06T05:35:15.180769+0000","last_became_active":"2023-10-06T05:35:14.170236+0000","last_became_peered":"2023-10-06T05:35:14.170236+0000","last_unstale":"2023-10-06T05:35:15.180769+0000","last_undegraded":"2023-10-06T05:35:15.180769+0000","last_fullsized":"2023-10-06T05:35:15.180769+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:35.865981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.10","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.954547+0000","last_change":"2023-10-06T05:35:14.952703+0000","last_active":"2023-10-06T05:35:14.954547+0000","last_peered":"2023-10-06T05:35:14.954547+0000","last_clean":"2023-10-06T05:35:14.954547+0000","last_became_active":"2023-10-06T05:35:14.952428+0000","last_became_peered":"2023-10-06T05:35:14.952428+0000","last_unstale":"2023-10-06T05:35:14.954547+0000","last_undegraded":"2023-10-06T05:35:14.954547+0000","last_fullsized":"2023-10-06T05:35:14.954547+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:09.962727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.13","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178141+0000","last_change":"2023-10-06T05:35:14.170563+0000","last_active":"2023-10-06T05:35:15.178141+0000","last_peered":"2023-10-06T05:35:15.178141+0000","last_clean":"2023-10-06T05:35:15.178141+0000","last_became_active":"2023-10-06T05:35:14.170261+0000","last_became_peered":"2023-10-06T05:35:14.170261+0000","last_unstale":"2023-10-06T05:35:15.178141+0000","last_undegraded":"2023-10-06T05:35:15.178141+0000","last_fullsized":"2023-10-06T05:35:15.178141+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:49.511722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.11","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959766+0000","last_change":"2023-10-06T05:35:14.956226+0000","last_active":"2023-10-06T05:35:14.959766+0000","last_peered":"2023-10-06T05:35:14.959766+0000","last_clean":"2023-10-06T05:35:14.959766+0000","last_became_active":"2023-10-06T05:35:14.955621+0000","last_became_peered":"2023-10-06T05:35:14.955621+0000","last_unstale":"2023-10-06T05:35:14.959766+0000","last_undegraded":"2023-10-06T05:35:14.959766+0000","last_fullsized":"2023-10-06T05:35:14.959766+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:33.356172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"41.12","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945576+0000","last_change":"2023-10-06T05:35:14.170815+0000","last_active":"2023-10-06T05:35:14.945576+0000","last_peered":"2023-10-06T05:35:14.945576+0000","last_clean":"2023-10-06T05:35:14.945576+0000","last_became_active":"2023-10-06T05:35:14.170451+0000","last_became_peered":"2023-10-06T05:35:14.170451+0000","last_unstale":"2023-10-06T05:35:14.945576+0000","last_undegraded":"2023-10-06T05:35:14.945576+0000","last_fullsized":"2023-10-06T05:35:14.945576+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:25.087429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.12","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975731+0000","last_change":"2023-10-06T05:35:14.971045+0000","last_active":"2023-10-06T05:35:14.975731+0000","last_peered":"2023-10-06T05:35:14.975731+0000","last_clean":"2023-10-06T05:35:14.975731+0000","last_became_active":"2023-10-06T05:35:14.970755+0000","last_became_peered":"2023-10-06T05:35:14.970755+0000","last_unstale":"2023-10-06T05:35:14.975731+0000","last_undegraded":"2023-10-06T05:35:14.975731+0000","last_fullsized":"2023-10-06T05:35:14.975731+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:29.466625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.11","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.954298+0000","last_change":"2023-10-06T05:35:14.172791+0000","last_active":"2023-10-06T05:35:14.954298+0000","last_peered":"2023-10-06T05:35:14.954298+0000","last_clean":"2023-10-06T05:35:14.954298+0000","last_became_active":"2023-10-06T05:35:14.172516+0000","last_became_peered":"2023-10-06T05:35:14.172516+0000","last_unstale":"2023-10-06T05:35:14.954298+0000","last_undegraded":"2023-10-06T05:35:14.954298+0000","last_fullsized":"2023-10-06T05:35:14.954298+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:56.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.13","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971643+0000","last_change":"2023-10-06T05:35:14.966424+0000","last_active":"2023-10-06T05:35:14.971643+0000","last_peered":"2023-10-06T05:35:14.971643+0000","last_clean":"2023-10-06T05:35:14.971643+0000","last_became_active":"2023-10-06T05:35:14.966095+0000","last_became_peered":"2023-10-06T05:35:14.966095+0000","last_unstale":"2023-10-06T05:35:14.971643+0000","last_undegraded":"2023-10-06T05:35:14.971643+0000","last_fullsized":"2023-10-06T05:35:14.971643+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:40.225983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.10","version":"281'8","reported_seq":18,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.179452+0000","last_change":"2023-10-06T05:35:14.174919+0000","last_active":"2023-10-06T05:35:15.179452+0000","last_peered":"2023-10-06T05:35:15.179452+0000","last_clean":"2023-10-06T05:35:15.179452+0000","last_became_active":"2023-10-06T05:35:14.174608+0000","last_became_peered":"2023-10-06T05:35:14.174608+0000","last_unstale":"2023-10-06T05:35:15.179452+0000","last_undegraded":"2023-10-06T05:35:15.179452+0000","last_fullsized":"2023-10-06T05:35:15.179452+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:02.504014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962339+0000","last_change":"2023-10-06T05:35:14.959390+0000","last_active":"2023-10-06T05:35:14.962339+0000","last_peered":"2023-10-06T05:35:14.962339+0000","last_clean":"2023-10-06T05:35:14.962339+0000","last_became_active":"2023-10-06T05:35:14.959141+0000","last_became_peered":"2023-10-06T05:35:14.959141+0000","last_unstale":"2023-10-06T05:35:14.962339+0000","last_undegraded":"2023-10-06T05:35:14.962339+0000","last_fullsized":"2023-10-06T05:35:14.962339+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:07.812426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.f","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.176457+0000","last_change":"2023-10-06T05:35:14.164959+0000","last_active":"2023-10-06T05:35:15.176457+0000","last_peered":"2023-10-06T05:35:15.176457+0000","last_clean":"2023-10-06T05:35:15.176457+0000","last_became_active":"2023-10-06T05:35:14.164661+0000","last_became_peered":"2023-10-06T05:35:14.164661+0000","last_unstale":"2023-10-06T05:35:15.176457+0000","last_undegraded":"2023-10-06T05:35:15.176457+0000","last_fullsized":"2023-10-06T05:35:15.176457+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:26.921573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.968336+0000","last_change":"2023-10-06T05:35:14.967111+0000","last_active":"2023-10-06T05:35:14.968336+0000","last_peered":"2023-10-06T05:35:14.968336+0000","last_clean":"2023-10-06T05:35:14.968336+0000","last_became_active":"2023-10-06T05:35:14.966845+0000","last_became_peered":"2023-10-06T05:35:14.966845+0000","last_unstale":"2023-10-06T05:35:14.968336+0000","last_undegraded":"2023-10-06T05:35:14.968336+0000","last_fullsized":"2023-10-06T05:35:14.968336+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:34.653085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.e","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178563+0000","last_change":"2023-10-06T05:35:14.172723+0000","last_active":"2023-10-06T05:35:15.178563+0000","last_peered":"2023-10-06T05:35:15.178563+0000","last_clean":"2023-10-06T05:35:15.178563+0000","last_became_active":"2023-10-06T05:35:14.172474+0000","last_became_peered":"2023-10-06T05:35:14.172474+0000","last_unstale":"2023-10-06T05:35:15.178563+0000","last_undegraded":"2023-10-06T05:35:15.178563+0000","last_fullsized":"2023-10-06T05:35:15.178563+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:51.255904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961928+0000","last_change":"2023-10-06T05:35:14.961264+0000","last_active":"2023-10-06T05:35:14.961928+0000","last_peered":"2023-10-06T05:35:14.961928+0000","last_clean":"2023-10-06T05:35:14.961928+0000","last_became_active":"2023-10-06T05:35:14.960997+0000","last_became_peered":"2023-10-06T05:35:14.960997+0000","last_unstale":"2023-10-06T05:35:14.961928+0000","last_undegraded":"2023-10-06T05:35:14.961928+0000","last_fullsized":"2023-10-06T05:35:14.961928+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:45.611610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.d","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180484+0000","last_change":"2023-10-06T05:35:14.169611+0000","last_active":"2023-10-06T05:35:15.180484+0000","last_peered":"2023-10-06T05:35:15.180484+0000","last_clean":"2023-10-06T05:35:15.180484+0000","last_became_active":"2023-10-06T05:35:14.169146+0000","last_became_peered":"2023-10-06T05:35:14.169146+0000","last_unstale":"2023-10-06T05:35:15.180484+0000","last_undegraded":"2023-10-06T05:35:15.180484+0000","last_fullsized":"2023-10-06T05:35:15.180484+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:07.066950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975851+0000","last_change":"2023-10-06T05:35:14.972529+0000","last_active":"2023-10-06T05:35:14.975851+0000","last_peered":"2023-10-06T05:35:14.975851+0000","last_clean":"2023-10-06T05:35:14.975851+0000","last_became_active":"2023-10-06T05:35:14.972243+0000","last_became_peered":"2023-10-06T05:35:14.972243+0000","last_unstale":"2023-10-06T05:35:14.975851+0000","last_undegraded":"2023-10-06T05:35:14.975851+0000","last_fullsized":"2023-10-06T05:35:14.975851+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:02.773613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.c","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950403+0000","last_change":"2023-10-06T05:35:14.159696+0000","last_active":"2023-10-06T05:35:14.950403+0000","last_peered":"2023-10-06T05:35:14.950403+0000","last_clean":"2023-10-06T05:35:14.950403+0000","last_became_active":"2023-10-06T05:35:14.159420+0000","last_became_peered":"2023-10-06T05:35:14.159420+0000","last_unstale":"2023-10-06T05:35:14.950403+0000","last_undegraded":"2023-10-06T05:35:14.950403+0000","last_fullsized":"2023-10-06T05:35:14.950403+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:08.051335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.8","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975995+0000","last_change":"2023-10-06T05:35:14.973360+0000","last_active":"2023-10-06T05:35:14.975995+0000","last_peered":"2023-10-06T05:35:14.975995+0000","last_clean":"2023-10-06T05:35:14.975995+0000","last_became_active":"2023-10-06T05:35:14.973067+0000","last_became_peered":"2023-10-06T05:35:14.973067+0000","last_unstale":"2023-10-06T05:35:14.975995+0000","last_undegraded":"2023-10-06T05:35:14.975995+0000","last_fullsized":"2023-10-06T05:35:14.975995+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:24.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.b","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180381+0000","last_change":"2023-10-06T05:35:14.165388+0000","last_active":"2023-10-06T05:35:15.180381+0000","last_peered":"2023-10-06T05:35:15.180381+0000","last_clean":"2023-10-06T05:35:15.180381+0000","last_became_active":"2023-10-06T05:35:14.165093+0000","last_became_peered":"2023-10-06T05:35:14.165093+0000","last_unstale":"2023-10-06T05:35:15.180381+0000","last_undegraded":"2023-10-06T05:35:15.180381+0000","last_fullsized":"2023-10-06T05:35:15.180381+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:31.952859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.9","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962437+0000","last_change":"2023-10-06T05:35:14.957510+0000","last_active":"2023-10-06T05:35:14.962437+0000","last_peered":"2023-10-06T05:35:14.962437+0000","last_clean":"2023-10-06T05:35:14.962437+0000","last_became_active":"2023-10-06T05:35:14.957201+0000","last_became_peered":"2023-10-06T05:35:14.957201+0000","last_unstale":"2023-10-06T05:35:14.962437+0000","last_undegraded":"2023-10-06T05:35:14.962437+0000","last_fullsized":"2023-10-06T05:35:14.962437+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:54.209703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.a","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952293+0000","last_change":"2023-10-06T05:35:14.169771+0000","last_active":"2023-10-06T05:35:14.952293+0000","last_peered":"2023-10-06T05:35:14.952293+0000","last_clean":"2023-10-06T05:35:14.952293+0000","last_became_active":"2023-10-06T05:35:14.169468+0000","last_became_peered":"2023-10-06T05:35:14.169468+0000","last_unstale":"2023-10-06T05:35:14.952293+0000","last_undegraded":"2023-10-06T05:35:14.952293+0000","last_fullsized":"2023-10-06T05:35:14.952293+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:08.051335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.969044+0000","last_change":"2023-10-06T05:35:14.960368+0000","last_active":"2023-10-06T05:35:14.969044+0000","last_peered":"2023-10-06T05:35:14.969044+0000","last_clean":"2023-10-06T05:35:14.969044+0000","last_became_active":"2023-10-06T05:35:14.960113+0000","last_became_peered":"2023-10-06T05:35:14.960113+0000","last_unstale":"2023-10-06T05:35:14.969044+0000","last_undegraded":"2023-10-06T05:35:14.969044+0000","last_fullsized":"2023-10-06T05:35:14.969044+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.514762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.9","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.948655+0000","last_change":"2023-10-06T05:35:14.174404+0000","last_active":"2023-10-06T05:35:14.948655+0000","last_peered":"2023-10-06T05:35:14.948655+0000","last_clean":"2023-10-06T05:35:14.948655+0000","last_became_active":"2023-10-06T05:35:14.173994+0000","last_became_peered":"2023-10-06T05:35:14.173994+0000","last_unstale":"2023-10-06T05:35:14.948655+0000","last_undegraded":"2023-10-06T05:35:14.948655+0000","last_fullsized":"2023-10-06T05:35:14.948655+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:08.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964528+0000","last_change":"2023-10-06T05:35:14.959930+0000","last_active":"2023-10-06T05:35:14.964528+0000","last_peered":"2023-10-06T05:35:14.964528+0000","last_clean":"2023-10-06T05:35:14.964528+0000","last_became_active":"2023-10-06T05:35:14.959662+0000","last_became_peered":"2023-10-06T05:35:14.959662+0000","last_unstale":"2023-10-06T05:35:14.964528+0000","last_undegraded":"2023-10-06T05:35:14.964528+0000","last_fullsized":"2023-10-06T05:35:14.964528+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.514762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.8","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952784+0000","last_change":"2023-10-06T05:35:14.170606+0000","last_active":"2023-10-06T05:35:14.952784+0000","last_peered":"2023-10-06T05:35:14.952784+0000","last_clean":"2023-10-06T05:35:14.952784+0000","last_became_active":"2023-10-06T05:35:14.169831+0000","last_became_peered":"2023-10-06T05:35:14.169831+0000","last_unstale":"2023-10-06T05:35:14.952784+0000","last_undegraded":"2023-10-06T05:35:14.952784+0000","last_fullsized":"2023-10-06T05:35:14.952784+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:42.684244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.4","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975991+0000","last_change":"2023-10-06T05:35:14.970970+0000","last_active":"2023-10-06T05:35:14.975991+0000","last_peered":"2023-10-06T05:35:14.975991+0000","last_clean":"2023-10-06T05:35:14.975991+0000","last_became_active":"2023-10-06T05:35:14.970689+0000","last_became_peered":"2023-10-06T05:35:14.970689+0000","last_unstale":"2023-10-06T05:35:14.975991+0000","last_undegraded":"2023-10-06T05:35:14.975991+0000","last_fullsized":"2023-10-06T05:35:14.975991+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:44.077245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.7","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178665+0000","last_change":"2023-10-06T05:35:14.173660+0000","last_active":"2023-10-06T05:35:15.178665+0000","last_peered":"2023-10-06T05:35:15.178665+0000","last_clean":"2023-10-06T05:35:15.178665+0000","last_became_active":"2023-10-06T05:35:14.173367+0000","last_became_peered":"2023-10-06T05:35:14.173367+0000","last_unstale":"2023-10-06T05:35:15.178665+0000","last_undegraded":"2023-10-06T05:35:15.178665+0000","last_fullsized":"2023-10-06T05:35:15.178665+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:21.111831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.5","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971154+0000","last_change":"2023-10-06T05:35:14.962959+0000","last_active":"2023-10-06T05:35:14.971154+0000","last_peered":"2023-10-06T05:35:14.971154+0000","last_clean":"2023-10-06T05:35:14.971154+0000","last_became_active":"2023-10-06T05:35:14.961859+0000","last_became_peered":"2023-10-06T05:35:14.961859+0000","last_unstale":"2023-10-06T05:35:14.971154+0000","last_undegraded":"2023-10-06T05:35:14.971154+0000","last_fullsized":"2023-10-06T05:35:14.971154+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:33.968063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.6","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.957295+0000","last_change":"2023-10-06T05:35:14.171239+0000","last_active":"2023-10-06T05:35:14.957295+0000","last_peered":"2023-10-06T05:35:14.957295+0000","last_clean":"2023-10-06T05:35:14.957295+0000","last_became_active":"2023-10-06T05:35:14.170908+0000","last_became_peered":"2023-10-06T05:35:14.170908+0000","last_unstale":"2023-10-06T05:35:14.957295+0000","last_undegraded":"2023-10-06T05:35:14.957295+0000","last_fullsized":"2023-10-06T05:35:14.957295+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:32.701909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.6","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970934+0000","last_change":"2023-10-06T05:35:14.962610+0000","last_active":"2023-10-06T05:35:14.970934+0000","last_peered":"2023-10-06T05:35:14.970934+0000","last_clean":"2023-10-06T05:35:14.970934+0000","last_became_active":"2023-10-06T05:35:14.961691+0000","last_became_peered":"2023-10-06T05:35:14.961691+0000","last_unstale":"2023-10-06T05:35:14.970934+0000","last_undegraded":"2023-10-06T05:35:14.970934+0000","last_fullsized":"2023-10-06T05:35:14.970934+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:44.077245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.5","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.943859+0000","last_change":"2023-10-06T05:35:14.168409+0000","last_active":"2023-10-06T05:35:14.943859+0000","last_peered":"2023-10-06T05:35:14.943859+0000","last_clean":"2023-10-06T05:35:14.943859+0000","last_became_active":"2023-10-06T05:35:14.168188+0000","last_became_peered":"2023-10-06T05:35:14.168188+0000","last_unstale":"2023-10-06T05:35:14.943859+0000","last_undegraded":"2023-10-06T05:35:14.943859+0000","last_fullsized":"2023-10-06T05:35:14.943859+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:15.864842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.7","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.968521+0000","last_change":"2023-10-06T05:35:14.964193+0000","last_active":"2023-10-06T05:35:14.968521+0000","last_peered":"2023-10-06T05:35:14.968521+0000","last_clean":"2023-10-06T05:35:14.968521+0000","last_became_active":"2023-10-06T05:35:14.963905+0000","last_became_peered":"2023-10-06T05:35:14.963905+0000","last_unstale":"2023-10-06T05:35:14.968521+0000","last_undegraded":"2023-10-06T05:35:14.968521+0000","last_fullsized":"2023-10-06T05:35:14.968521+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:07.812426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.4","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945718+0000","last_change":"2023-10-06T05:35:14.163454+0000","last_active":"2023-10-06T05:35:14.945718+0000","last_peered":"2023-10-06T05:35:14.945718+0000","last_clean":"2023-10-06T05:35:14.945718+0000","last_became_active":"2023-10-06T05:35:14.163043+0000","last_became_peered":"2023-10-06T05:35:14.163043+0000","last_unstale":"2023-10-06T05:35:14.945718+0000","last_undegraded":"2023-10-06T05:35:14.945718+0000","last_fullsized":"2023-10-06T05:35:14.945718+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:46.568349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.0","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971397+0000","last_change":"2023-10-06T05:35:14.966070+0000","last_active":"2023-10-06T05:35:14.971397+0000","last_peered":"2023-10-06T05:35:14.971397+0000","last_clean":"2023-10-06T05:35:14.971397+0000","last_became_active":"2023-10-06T05:35:14.964657+0000","last_became_peered":"2023-10-06T05:35:14.964657+0000","last_unstale":"2023-10-06T05:35:14.971397+0000","last_undegraded":"2023-10-06T05:35:14.971397+0000","last_fullsized":"2023-10-06T05:35:14.971397+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:55.698961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.3","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.944809+0000","last_change":"2023-10-06T05:35:14.163900+0000","last_active":"2023-10-06T05:35:14.944809+0000","last_peered":"2023-10-06T05:35:14.944809+0000","last_clean":"2023-10-06T05:35:14.944809+0000","last_became_active":"2023-10-06T05:35:14.163613+0000","last_became_peered":"2023-10-06T05:35:14.163613+0000","last_unstale":"2023-10-06T05:35:14.944809+0000","last_undegraded":"2023-10-06T05:35:14.944809+0000","last_fullsized":"2023-10-06T05:35:14.944809+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:22.390099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.1","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.967128+0000","last_change":"2023-10-06T05:35:14.963840+0000","last_active":"2023-10-06T05:35:14.967128+0000","last_peered":"2023-10-06T05:35:14.967128+0000","last_clean":"2023-10-06T05:35:14.967128+0000","last_became_active":"2023-10-06T05:35:14.963518+0000","last_became_peered":"2023-10-06T05:35:14.963518+0000","last_unstale":"2023-10-06T05:35:14.967128+0000","last_undegraded":"2023-10-06T05:35:14.967128+0000","last_fullsized":"2023-10-06T05:35:14.967128+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:50.127598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.2","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950357+0000","last_change":"2023-10-06T05:35:14.167638+0000","last_active":"2023-10-06T05:35:14.950357+0000","last_peered":"2023-10-06T05:35:14.950357+0000","last_clean":"2023-10-06T05:35:14.950357+0000","last_became_active":"2023-10-06T05:35:14.167368+0000","last_became_peered":"2023-10-06T05:35:14.167368+0000","last_unstale":"2023-10-06T05:35:14.950357+0000","last_undegraded":"2023-10-06T05:35:14.950357+0000","last_fullsized":"2023-10-06T05:35:14.950357+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:43.745901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959156+0000","last_change":"2023-10-06T05:35:14.954878+0000","last_active":"2023-10-06T05:35:14.959156+0000","last_peered":"2023-10-06T05:35:14.959156+0000","last_clean":"2023-10-06T05:35:14.959156+0000","last_became_active":"2023-10-06T05:35:14.954588+0000","last_became_peered":"2023-10-06T05:35:14.954588+0000","last_unstale":"2023-10-06T05:35:14.959156+0000","last_undegraded":"2023-10-06T05:35:14.959156+0000","last_fullsized":"2023-10-06T05:35:14.959156+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:50.595349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945290+0000","last_change":"2023-10-06T05:35:14.166524+0000","last_active":"2023-10-06T05:35:14.945290+0000","last_peered":"2023-10-06T05:35:14.945290+0000","last_clean":"2023-10-06T05:35:14.945290+0000","last_became_active":"2023-10-06T05:35:14.166246+0000","last_became_peered":"2023-10-06T05:35:14.166246+0000","last_unstale":"2023-10-06T05:35:14.945290+0000","last_undegraded":"2023-10-06T05:35:14.945290+0000","last_fullsized":"2023-10-06T05:35:14.945290+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:08.762974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.3","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961809+0000","last_change":"2023-10-06T05:35:14.955749+0000","last_active":"2023-10-06T05:35:14.961809+0000","last_peered":"2023-10-06T05:35:14.961809+0000","last_clean":"2023-10-06T05:35:14.961809+0000","last_became_active":"2023-10-06T05:35:14.955431+0000","last_became_peered":"2023-10-06T05:35:14.955431+0000","last_unstale":"2023-10-06T05:35:14.961809+0000","last_undegraded":"2023-10-06T05:35:14.961809+0000","last_fullsized":"2023-10-06T05:35:14.961809+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:09.241675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.0","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952051+0000","last_change":"2023-10-06T05:35:14.174580+0000","last_active":"2023-10-06T05:35:14.952051+0000","last_peered":"2023-10-06T05:35:14.952051+0000","last_clean":"2023-10-06T05:35:14.952051+0000","last_became_active":"2023-10-06T05:35:14.174296+0000","last_became_peered":"2023-10-06T05:35:14.174296+0000","last_unstale":"2023-10-06T05:35:14.952051+0000","last_undegraded":"2023-10-06T05:35:14.952051+0000","last_fullsized":"2023-10-06T05:35:14.952051+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:09.305299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.2c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961125+0000","last_change":"2023-10-06T05:35:14.957248+0000","last_active":"2023-10-06T05:35:14.961125+0000","last_peered":"2023-10-06T05:35:14.961125+0000","last_clean":"2023-10-06T05:35:14.961125+0000","last_became_active":"2023-10-06T05:35:14.956989+0000","last_became_peered":"2023-10-06T05:35:14.956989+0000","last_unstale":"2023-10-06T05:35:14.961125+0000","last_undegraded":"2023-10-06T05:35:14.961125+0000","last_fullsized":"2023-10-06T05:35:14.961125+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:24.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":231,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950503+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:35:14.950503+0000","last_peered":"2023-10-06T05:35:14.950503+0000","last_clean":"2023-10-06T05:35:14.950503+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:14.950503+0000","last_undegraded":"2023-10-06T05:35:14.950503+0000","last_fullsized":"2023-10-06T05:35:14.950503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959836+0000","last_change":"2023-10-06T05:35:14.957001+0000","last_active":"2023-10-06T05:35:14.959836+0000","last_peered":"2023-10-06T05:35:14.959836+0000","last_clean":"2023-10-06T05:35:14.959836+0000","last_became_active":"2023-10-06T05:35:14.956693+0000","last_became_peered":"2023-10-06T05:35:14.956693+0000","last_unstale":"2023-10-06T05:35:14.959836+0000","last_undegraded":"2023-10-06T05:35:14.959836+0000","last_fullsized":"2023-10-06T05:35:14.959836+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:26.481851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":226,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950020+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:35:14.950020+0000","last_peered":"2023-10-06T05:35:14.950020+0000","last_clean":"2023-10-06T05:35:14.950020+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:14.950020+0000","last_undegraded":"2023-10-06T05:35:14.950020+0000","last_fullsized":"2023-10-06T05:35:14.950020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.29","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964423+0000","last_change":"2023-10-06T05:35:14.961967+0000","last_active":"2023-10-06T05:35:14.964423+0000","last_peered":"2023-10-06T05:35:14.964423+0000","last_clean":"2023-10-06T05:35:14.964423+0000","last_became_active":"2023-10-06T05:35:14.961687+0000","last_became_peered":"2023-10-06T05:35:14.961687+0000","last_unstale":"2023-10-06T05:35:14.964423+0000","last_undegraded":"2023-10-06T05:35:14.964423+0000","last_fullsized":"2023-10-06T05:35:14.964423+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:19.749199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.943018+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:35:14.943018+0000","last_peered":"2023-10-06T05:35:14.943018+0000","last_clean":"2023-10-06T05:35:14.943018+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:14.943018+0000","last_undegraded":"2023-10-06T05:35:14.943018+0000","last_fullsized":"2023-10-06T05:35:14.943018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.2b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966755+0000","last_change":"2023-10-06T05:35:14.962866+0000","last_active":"2023-10-06T05:35:14.966755+0000","last_peered":"2023-10-06T05:35:14.966755+0000","last_clean":"2023-10-06T05:35:14.966755+0000","last_became_active":"2023-10-06T05:35:14.962317+0000","last_became_peered":"2023-10-06T05:35:14.962317+0000","last_unstale":"2023-10-06T05:35:14.966755+0000","last_undegraded":"2023-10-06T05:35:14.966755+0000","last_fullsized":"2023-10-06T05:35:14.966755+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.487051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":226,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949588+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:35:14.949588+0000","last_peered":"2023-10-06T05:35:14.949588+0000","last_clean":"2023-10-06T05:35:14.949588+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:14.949588+0000","last_undegraded":"2023-10-06T05:35:14.949588+0000","last_fullsized":"2023-10-06T05:35:14.949588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":376,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949844+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:35:14.949844+0000","last_peered":"2023-10-06T05:35:14.949844+0000","last_clean":"2023-10-06T05:35:14.949844+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:14.949844+0000","last_undegraded":"2023-10-06T05:35:14.949844+0000","last_fullsized":"2023-10-06T05:35:14.949844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.28","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975803+0000","last_change":"2023-10-06T05:35:14.972273+0000","last_active":"2023-10-06T05:35:14.975803+0000","last_peered":"2023-10-06T05:35:14.975803+0000","last_clean":"2023-10-06T05:35:14.975803+0000","last_became_active":"2023-10-06T05:35:14.972017+0000","last_became_peered":"2023-10-06T05:35:14.972017+0000","last_unstale":"2023-10-06T05:35:14.975803+0000","last_undegraded":"2023-10-06T05:35:14.975803+0000","last_fullsized":"2023-10-06T05:35:14.975803+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:18.649823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":223,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949693+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:35:14.949693+0000","last_peered":"2023-10-06T05:35:14.949693+0000","last_clean":"2023-10-06T05:35:14.949693+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:14.949693+0000","last_undegraded":"2023-10-06T05:35:14.949693+0000","last_fullsized":"2023-10-06T05:35:14.949693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966797+0000","last_change":"2023-10-06T05:35:14.963971+0000","last_active":"2023-10-06T05:35:14.966797+0000","last_peered":"2023-10-06T05:35:14.966797+0000","last_clean":"2023-10-06T05:35:14.966797+0000","last_became_active":"2023-10-06T05:35:14.963681+0000","last_became_peered":"2023-10-06T05:35:14.963681+0000","last_unstale":"2023-10-06T05:35:14.966797+0000","last_undegraded":"2023-10-06T05:35:14.966797+0000","last_fullsized":"2023-10-06T05:35:14.966797+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:22.458660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":223,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.951299+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:35:14.951299+0000","last_peered":"2023-10-06T05:35:14.951299+0000","last_clean":"2023-10-06T05:35:14.951299+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:14.951299+0000","last_undegraded":"2023-10-06T05:35:14.951299+0000","last_fullsized":"2023-10-06T05:35:14.951299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.967137+0000","last_change":"2023-10-06T05:35:14.964083+0000","last_active":"2023-10-06T05:35:14.967137+0000","last_peered":"2023-10-06T05:35:14.967137+0000","last_clean":"2023-10-06T05:35:14.967137+0000","last_became_active":"2023-10-06T05:35:14.963502+0000","last_became_peered":"2023-10-06T05:35:14.963502+0000","last_unstale":"2023-10-06T05:35:14.967137+0000","last_undegraded":"2023-10-06T05:35:14.967137+0000","last_fullsized":"2023-10-06T05:35:14.967137+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:58.534732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":231,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949230+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:35:14.949230+0000","last_peered":"2023-10-06T05:35:14.949230+0000","last_clean":"2023-10-06T05:35:14.949230+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:14.949230+0000","last_undegraded":"2023-10-06T05:35:14.949230+0000","last_fullsized":"2023-10-06T05:35:14.949230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964265+0000","last_change":"2023-10-06T05:35:14.961862+0000","last_active":"2023-10-06T05:35:14.964265+0000","last_peered":"2023-10-06T05:35:14.964265+0000","last_clean":"2023-10-06T05:35:14.964265+0000","last_became_active":"2023-10-06T05:35:14.961601+0000","last_became_peered":"2023-10-06T05:35:14.961601+0000","last_unstale":"2023-10-06T05:35:14.964265+0000","last_undegraded":"2023-10-06T05:35:14.964265+0000","last_fullsized":"2023-10-06T05:35:14.964265+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:57:57.891612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945697+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:35:14.945697+0000","last_peered":"2023-10-06T05:35:14.945697+0000","last_clean":"2023-10-06T05:35:14.945697+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:14.945697+0000","last_undegraded":"2023-10-06T05:35:14.945697+0000","last_fullsized":"2023-10-06T05:35:14.945697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":42,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":41,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575106,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309560,"kb_used_data":4080,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062280,"statfs":{"total":96636764160,"available":96319774720,"internally_reserved":0,"allocated":4177920,"data_stored":2274934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":1.026},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":1.01}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.629},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308976,"kb_used_data":3496,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062864,"statfs":{"total":96636764160,"available":96320372736,"internally_reserved":0,"allocated":3579904,"data_stored":1684476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.47899999999999998,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.70599999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.751},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.115},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.56000000000000005,"5min":0.504,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.022,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.54400000000000004,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":1.0129999999999999,"5min":1.143,"15min":1.143},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.038,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575105,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304892,"kb_used_data":3508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066948,"statfs":{"total":96636764160,"available":96324554752,"internally_reserved":0,"allocated":3592192,"data_stored":1685122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":1.272,"15min":1.272},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.68000000000000005,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.312}]},{"osd":2,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.72199999999999998,"15min":1.165},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":1.22,"15min":1.22},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.432},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":1.238,"15min":1.2430000000000001},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84999999999999998,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96299999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1080000000000001,"5min":1.492,"15min":1.492},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.262,"15min":1.2909999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.82999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.0609999999999999,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.85299999999999998,"15min":0.82199999999999995},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":0.92300000000000004,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575105,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313664,"kb_used_data":4088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058176,"statfs":{"total":96636764160,"available":96315572224,"internally_reserved":0,"allocated":4186112,"data_stored":2275558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.95299999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.375}]},{"osd":3,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.307},"max":{"1min":0.628,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0980000000000001,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.1379999999999999,"15min":1.27},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.49199999999999999},"max":{"1min":1.151,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.81499999999999995,"15min":0.80100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.147,"15min":1.153},"last":0.91400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.496},"max":{"1min":1.0629999999999999,"5min":1.2,"15min":1.2010000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.90800000000000003,"5min":1.1459999999999999,"15min":1.274},"last":0.82299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575105,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308992,"kb_used_data":3512,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062848,"statfs":{"total":96636764160,"available":96320356352,"internally_reserved":0,"allocated":3596288,"data_stored":1686034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.79600000000000004,"5min":0.82999999999999996,"15min":1.056},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57099999999999995,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.64000000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.58599999999999997,"5min":1.117,"15min":1.117},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95599999999999996,"5min":1.2709999999999999,"15min":1.486},"last":0.627},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.089,"15min":1.5},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93899999999999995,"5min":1.157,"15min":1.1639999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92400000000000004,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.22,"15min":1.22},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.59699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.341,"15min":1.341},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":1.282,"15min":1.538},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.264,"15min":1.264},"last":0.78300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575106,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308976,"kb_used_data":3496,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062864,"statfs":{"total":96636764160,"available":96320372736,"internally_reserved":0,"allocated":3579904,"data_stored":1684476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.872},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.82999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575104,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308980,"kb_used_data":3500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062860,"statfs":{"total":96636764160,"available":96320368640,"internally_reserved":0,"allocated":3584000,"data_stored":1684510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.751},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.877}]},{"osd":1,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.82199999999999995,"5min":1.026,"15min":1.026},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.69499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41799999999999998},"max":{"1min":0.88600000000000001,"5min":1.099,"15min":1.099},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76200000000000001,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.443},"max":{"1min":0.85699999999999998,"5min":3.839,"15min":3.839},"last":0.745},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.873,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":0.96499999999999997,"15min":1.0609999999999999},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":0.93999999999999995,"5min":1.109,"15min":1.282},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.72499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575106,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304884,"kb_used_data":3500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066956,"statfs":{"total":96636764160,"available":96324562944,"internally_reserved":0,"allocated":3584000,"data_stored":1679405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.372},"max":{"1min":1.071,"5min":1.115,"15min":1.2130000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.55700000000000005,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":1.169,"15min":1.169},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.145},"last":0.77900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.92800000000000005,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.02,"15min":1.1539999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.96899999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.52000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.87},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.60199999999999998,"5min":0.53300000000000003,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.0609999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:35:19.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:35:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.778+0000 7fa78c99d700 1 Processor -- start 2023-10-06T05:35:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.778+0000 7fa78c99d700 1 -- start start 2023-10-06T05:35:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa78c99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 0x7fa78806a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa78c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 0x7fa7880c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa78c99d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78806ab60 con 0x7fa7880c8b50 2023-10-06T05:35:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa78c99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78806b7e0 con 0x7fa788155af0 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa78c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78806ae40 con 0x7fa78806a6a0 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa786ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 0x7fa7880c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa7877fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 0x7fa78806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.780+0000 7fa787fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa7880c8b50 0x7fa7881598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:43966/0 (socket says 172.21.15.33:43966) 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa7877fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 0x7fa78806a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:53412/0 (socket says 172.21.15.33:53412) 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa787fff700 1 -- 172.21.15.33:0/707710258 learned_addr learned my addr 172.21.15.33:0/707710258 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa786ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 0x7fa7880c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55228/0 (socket says 172.21.15.33:55228) 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa786ffd700 1 -- 172.21.15.33:0/707710258 >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 msgr2=0x7fa78806a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa786ffd700 1 --2- 172.21.15.33:0/707710258 >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 0x7fa78806a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa786ffd700 1 -- 172.21.15.33:0/707710258 >> v1:172.21.15.167:6789/0 conn(0x7fa7880c8b50 legacy=0x7fa7881598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa786ffd700 1 -- 172.21.15.33:0/707710258 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78806b120 con 0x7fa78806a6a0 2023-10-06T05:35:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa7877fe700 1 --2- 172.21.15.33:0/707710258 >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 0x7fa78806a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.781+0000 7fa786ffd700 1 --2- 172.21.15.33:0/707710258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 0x7fa7880c7210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0098e0 tx=0x7fa77c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bcff68fd54b5095 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa7867fc700 1 -- 172.21.15.33:0/707710258 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00f750 con 0x7fa78806a6a0 2023-10-06T05:35:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa7867fc700 1 -- 172.21.15.33:0/707710258 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00f8b0 con 0x7fa78806a6a0 2023-10-06T05:35:19.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa7867fc700 1 -- 172.21.15.33:0/707710258 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00fc50 con 0x7fa78806a6a0 2023-10-06T05:35:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 -- 172.21.15.33:0/707710258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 msgr2=0x7fa7880c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 --2- 172.21.15.33:0/707710258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 0x7fa7880c7210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0098e0 tx=0x7fa77c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 -- 172.21.15.33:0/707710258 shutdown_connections 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 --2- 172.21.15.33:0/707710258 >> v2:172.21.15.167:3300/0 conn(0x7fa788155af0 0x7fa78806a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 --2- 172.21.15.33:0/707710258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa78806a6a0 0x7fa7880c7210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 -- 172.21.15.33:0/707710258 >> 172.21.15.33:0/707710258 conn(0x7fa78806d030 msgr2=0x7fa78806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 -- 172.21.15.33:0/707710258 shutdown_connections 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.782+0000 7fa78c99d700 1 -- 172.21.15.33:0/707710258 wait complete. 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.784+0000 7fa78c99d700 1 Processor -- start 2023-10-06T05:35:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.784+0000 7fa78c99d700 1 -- start start 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa78c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 0x7fa7880aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa78c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7880c8b50 0x7fa7880abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa78c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa788155af0 0x7fa788155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa78c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880ae1e0 con 0x7fa78806a6a0 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa78c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880aaed0 con 0x7fa788155af0 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa78c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa788156f60 con 0x7fa7880c8b50 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa7877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 0x7fa7880aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa7877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 0x7fa7880aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59700/0 (socket says 172.21.15.33:59700) 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa7877fe700 1 -- 172.21.15.33:0/4210714795 learned_addr learned my addr 172.21.15.33:0/4210714795 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa786ffd700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7880c8b50 0x7fa7880abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa787fff700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa788155af0 0x7fa788155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.785+0000 7fa7877fe700 1 -- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7880c8b50 msgr2=0x7fa7880abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa7877fe700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7880c8b50 0x7fa7880abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa7877fe700 1 -- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa788155af0 msgr2=0x7fa788155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa7877fe700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa788155af0 0x7fa788155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa7877fe700 1 -- 172.21.15.33:0/4210714795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009420 con 0x7fa78806a6a0 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa787fff700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa788155af0 0x7fa788155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa7877fe700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 0x7fa7880aa8f0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7fa77800ecb0 tx=0x7fa7780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77800c9d0 con 0x7fa78806a6a0 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77800cb30 con 0x7fa78806a6a0 2023-10-06T05:35:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77800ceb0 con 0x7fa78806a6a0 2023-10-06T05:35:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c009420 con 0x7fa78806a6a0 2023-10-06T05:35:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.786+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7880ae6f0 con 0x7fa78806a6a0 2023-10-06T05:35:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.787+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa788157240 con 0x7fa78806a6a0 2023-10-06T05:35:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.788+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa778013680 con 0x7fa78806a6a0 2023-10-06T05:35:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.789+0000 7fa76f7fe700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa774066f30 0x7fa7740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.790+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa778084ad0 con 0x7fa78806a6a0 2023-10-06T05:35:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.790+0000 7fa786ffd700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa774066f30 0x7fa7740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.791+0000 7fa786ffd700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa774066f30 0x7fa7740693e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa78806aa90 tx=0x7fa77c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.794+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa778051b40 con 0x7fa78806a6a0 2023-10-06T05:35:19.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:19.991+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa78804c380 con 0x7fa774066f30 2023-10-06T05:35:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.000+0000 7fa76f7fe700 1 -- 172.21.15.33:0/4210714795 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311626 (secure 0 0 0) 0x7fa78804c380 con 0x7fa774066f30 2023-10-06T05:35:20.008 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa774066f30 msgr2=0x7fa7740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa774066f30 0x7fa7740693e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa78806aa90 tx=0x7fa77c019000 comp rx=0 tx=0).stop 2023-10-06T05:35:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 msgr2=0x7fa7880aa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 0x7fa7880aa8f0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7fa77800ecb0 tx=0x7fa7780091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 shutdown_connections 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa774066f30 0x7fa7740693e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa788155af0 0x7fa788155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa78806a6a0 0x7fa7880aa8f0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 --2- 172.21.15.33:0/4210714795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7880c8b50 0x7fa7880abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 >> 172.21.15.33:0/4210714795 conn(0x7fa78806d030 msgr2=0x7fa7880c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 shutdown_connections 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.005+0000 7fa78c99d700 1 -- 172.21.15.33:0/4210714795 wait complete. 2023-10-06T05:35:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:35:20.025 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":908,"stamp":"2023-10-06T05:35:18.141481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2468924,"kb_used_data":29180,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752505796,"statfs":{"total":773094113280,"available":770565935104,"internally_reserved":0,"allocated":29880320,"data_stored":14654515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.996291"},"pg_stats":[{"pgid":"42.3f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.954845+0000","last_change":"2023-10-06T05:35:14.952704+0000","last_active":"2023-10-06T05:35:14.954845+0000","last_peered":"2023-10-06T05:35:14.954845+0000","last_clean":"2023-10-06T05:35:14.954845+0000","last_became_active":"2023-10-06T05:35:14.952420+0000","last_became_peered":"2023-10-06T05:35:14.952420+0000","last_unstale":"2023-10-06T05:35:14.954845+0000","last_undegraded":"2023-10-06T05:35:14.954845+0000","last_fullsized":"2023-10-06T05:35:14.954845+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:44.558068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970965+0000","last_change":"2023-10-06T05:35:14.965791+0000","last_active":"2023-10-06T05:35:14.970965+0000","last_peered":"2023-10-06T05:35:14.970965+0000","last_clean":"2023-10-06T05:35:14.970965+0000","last_became_active":"2023-10-06T05:35:14.965458+0000","last_became_peered":"2023-10-06T05:35:14.965458+0000","last_unstale":"2023-10-06T05:35:14.970965+0000","last_undegraded":"2023-10-06T05:35:14.970965+0000","last_fullsized":"2023-10-06T05:35:14.970965+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:10.311289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.3d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959875+0000","last_change":"2023-10-06T05:35:14.956773+0000","last_active":"2023-10-06T05:35:14.959875+0000","last_peered":"2023-10-06T05:35:14.959875+0000","last_clean":"2023-10-06T05:35:14.959875+0000","last_became_active":"2023-10-06T05:35:14.956495+0000","last_became_peered":"2023-10-06T05:35:14.956495+0000","last_unstale":"2023-10-06T05:35:14.959875+0000","last_undegraded":"2023-10-06T05:35:14.959875+0000","last_fullsized":"2023-10-06T05:35:14.959875+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:12.470072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959801+0000","last_change":"2023-10-06T05:35:14.956046+0000","last_active":"2023-10-06T05:35:14.959801+0000","last_peered":"2023-10-06T05:35:14.959801+0000","last_clean":"2023-10-06T05:35:14.959801+0000","last_became_active":"2023-10-06T05:35:14.955777+0000","last_became_peered":"2023-10-06T05:35:14.955777+0000","last_unstale":"2023-10-06T05:35:14.959801+0000","last_undegraded":"2023-10-06T05:35:14.959801+0000","last_fullsized":"2023-10-06T05:35:14.959801+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:03.786775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964607+0000","last_change":"2023-10-06T05:35:14.958122+0000","last_active":"2023-10-06T05:35:14.964607+0000","last_peered":"2023-10-06T05:35:14.964607+0000","last_clean":"2023-10-06T05:35:14.964607+0000","last_became_active":"2023-10-06T05:35:14.957817+0000","last_became_peered":"2023-10-06T05:35:14.957817+0000","last_unstale":"2023-10-06T05:35:14.964607+0000","last_undegraded":"2023-10-06T05:35:14.964607+0000","last_fullsized":"2023-10-06T05:35:14.964607+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:36.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.3a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.969066+0000","last_change":"2023-10-06T05:35:14.961007+0000","last_active":"2023-10-06T05:35:14.969066+0000","last_peered":"2023-10-06T05:35:14.969066+0000","last_clean":"2023-10-06T05:35:14.969066+0000","last_became_active":"2023-10-06T05:35:14.960740+0000","last_became_peered":"2023-10-06T05:35:14.960740+0000","last_unstale":"2023-10-06T05:35:14.969066+0000","last_undegraded":"2023-10-06T05:35:14.969066+0000","last_fullsized":"2023-10-06T05:35:14.969066+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:46.539643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.39","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966988+0000","last_change":"2023-10-06T05:35:14.963285+0000","last_active":"2023-10-06T05:35:14.966988+0000","last_peered":"2023-10-06T05:35:14.966988+0000","last_clean":"2023-10-06T05:35:14.966988+0000","last_became_active":"2023-10-06T05:35:14.963035+0000","last_became_peered":"2023-10-06T05:35:14.963035+0000","last_unstale":"2023-10-06T05:35:14.966988+0000","last_undegraded":"2023-10-06T05:35:14.966988+0000","last_fullsized":"2023-10-06T05:35:14.966988+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:36.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.38","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959652+0000","last_change":"2023-10-06T05:35:14.953368+0000","last_active":"2023-10-06T05:35:14.959652+0000","last_peered":"2023-10-06T05:35:14.959652+0000","last_clean":"2023-10-06T05:35:14.959652+0000","last_became_active":"2023-10-06T05:35:14.953104+0000","last_became_peered":"2023-10-06T05:35:14.953104+0000","last_unstale":"2023-10-06T05:35:14.959652+0000","last_undegraded":"2023-10-06T05:35:14.959652+0000","last_fullsized":"2023-10-06T05:35:14.959652+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:24.776553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.37","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962374+0000","last_change":"2023-10-06T05:35:14.958551+0000","last_active":"2023-10-06T05:35:14.962374+0000","last_peered":"2023-10-06T05:35:14.962374+0000","last_clean":"2023-10-06T05:35:14.962374+0000","last_became_active":"2023-10-06T05:35:14.958287+0000","last_became_peered":"2023-10-06T05:35:14.958287+0000","last_unstale":"2023-10-06T05:35:14.962374+0000","last_undegraded":"2023-10-06T05:35:14.962374+0000","last_fullsized":"2023-10-06T05:35:14.962374+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:18.513063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.36","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962463+0000","last_change":"2023-10-06T05:35:14.960474+0000","last_active":"2023-10-06T05:35:14.962463+0000","last_peered":"2023-10-06T05:35:14.962463+0000","last_clean":"2023-10-06T05:35:14.962463+0000","last_became_active":"2023-10-06T05:35:14.960221+0000","last_became_peered":"2023-10-06T05:35:14.960221+0000","last_unstale":"2023-10-06T05:35:14.962463+0000","last_undegraded":"2023-10-06T05:35:14.962463+0000","last_fullsized":"2023-10-06T05:35:14.962463+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:18.649823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.35","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971177+0000","last_change":"2023-10-06T05:35:14.966264+0000","last_active":"2023-10-06T05:35:14.971177+0000","last_peered":"2023-10-06T05:35:14.971177+0000","last_clean":"2023-10-06T05:35:14.971177+0000","last_became_active":"2023-10-06T05:35:14.965330+0000","last_became_peered":"2023-10-06T05:35:14.965330+0000","last_unstale":"2023-10-06T05:35:14.971177+0000","last_undegraded":"2023-10-06T05:35:14.971177+0000","last_fullsized":"2023-10-06T05:35:14.971177+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:44.751891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.34","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966812+0000","last_change":"2023-10-06T05:35:14.960305+0000","last_active":"2023-10-06T05:35:14.966812+0000","last_peered":"2023-10-06T05:35:14.966812+0000","last_clean":"2023-10-06T05:35:14.966812+0000","last_became_active":"2023-10-06T05:35:14.959652+0000","last_became_peered":"2023-10-06T05:35:14.959652+0000","last_unstale":"2023-10-06T05:35:14.966812+0000","last_undegraded":"2023-10-06T05:35:14.966812+0000","last_fullsized":"2023-10-06T05:35:14.966812+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:45.564253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.33","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964508+0000","last_change":"2023-10-06T05:35:14.959792+0000","last_active":"2023-10-06T05:35:14.964508+0000","last_peered":"2023-10-06T05:35:14.964508+0000","last_clean":"2023-10-06T05:35:14.964508+0000","last_became_active":"2023-10-06T05:35:14.959533+0000","last_became_peered":"2023-10-06T05:35:14.959533+0000","last_unstale":"2023-10-06T05:35:14.964508+0000","last_undegraded":"2023-10-06T05:35:14.964508+0000","last_fullsized":"2023-10-06T05:35:14.964508+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:41.072798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.32","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971328+0000","last_change":"2023-10-06T05:35:14.963155+0000","last_active":"2023-10-06T05:35:14.971328+0000","last_peered":"2023-10-06T05:35:14.971328+0000","last_clean":"2023-10-06T05:35:14.971328+0000","last_became_active":"2023-10-06T05:35:14.962865+0000","last_became_peered":"2023-10-06T05:35:14.962865+0000","last_unstale":"2023-10-06T05:35:14.971328+0000","last_undegraded":"2023-10-06T05:35:14.971328+0000","last_fullsized":"2023-10-06T05:35:14.971328+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:00.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.31","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971064+0000","last_change":"2023-10-06T05:35:14.965653+0000","last_active":"2023-10-06T05:35:14.971064+0000","last_peered":"2023-10-06T05:35:14.971064+0000","last_clean":"2023-10-06T05:35:14.971064+0000","last_became_active":"2023-10-06T05:35:14.965354+0000","last_became_peered":"2023-10-06T05:35:14.965354+0000","last_unstale":"2023-10-06T05:35:14.971064+0000","last_undegraded":"2023-10-06T05:35:14.971064+0000","last_fullsized":"2023-10-06T05:35:14.971064+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:02.954439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.30","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971379+0000","last_change":"2023-10-06T05:35:14.964352+0000","last_active":"2023-10-06T05:35:14.971379+0000","last_peered":"2023-10-06T05:35:14.971379+0000","last_clean":"2023-10-06T05:35:14.971379+0000","last_became_active":"2023-10-06T05:35:14.963454+0000","last_became_peered":"2023-10-06T05:35:14.963454+0000","last_unstale":"2023-10-06T05:35:14.971379+0000","last_undegraded":"2023-10-06T05:35:14.971379+0000","last_fullsized":"2023-10-06T05:35:14.971379+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:41.072798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.27","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961271+0000","last_change":"2023-10-06T05:35:14.957895+0000","last_active":"2023-10-06T05:35:14.961271+0000","last_peered":"2023-10-06T05:35:14.961271+0000","last_clean":"2023-10-06T05:35:14.961271+0000","last_became_active":"2023-10-06T05:35:14.955242+0000","last_became_peered":"2023-10-06T05:35:14.955242+0000","last_unstale":"2023-10-06T05:35:14.961271+0000","last_undegraded":"2023-10-06T05:35:14.961271+0000","last_fullsized":"2023-10-06T05:35:14.961271+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:58.197912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.26","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975932+0000","last_change":"2023-10-06T05:35:14.973178+0000","last_active":"2023-10-06T05:35:14.975932+0000","last_peered":"2023-10-06T05:35:14.975932+0000","last_clean":"2023-10-06T05:35:14.975932+0000","last_became_active":"2023-10-06T05:35:14.972889+0000","last_became_peered":"2023-10-06T05:35:14.972889+0000","last_unstale":"2023-10-06T05:35:14.975932+0000","last_undegraded":"2023-10-06T05:35:14.975932+0000","last_fullsized":"2023-10-06T05:35:14.975932+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:50.127598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.25","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971231+0000","last_change":"2023-10-06T05:35:14.968543+0000","last_active":"2023-10-06T05:35:14.971231+0000","last_peered":"2023-10-06T05:35:14.971231+0000","last_clean":"2023-10-06T05:35:14.971231+0000","last_became_active":"2023-10-06T05:35:14.968275+0000","last_became_peered":"2023-10-06T05:35:14.968275+0000","last_unstale":"2023-10-06T05:35:14.971231+0000","last_undegraded":"2023-10-06T05:35:14.971231+0000","last_fullsized":"2023-10-06T05:35:14.971231+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:00.587143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.24","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961160+0000","last_change":"2023-10-06T05:35:14.958402+0000","last_active":"2023-10-06T05:35:14.961160+0000","last_peered":"2023-10-06T05:35:14.961160+0000","last_clean":"2023-10-06T05:35:14.961160+0000","last_became_active":"2023-10-06T05:35:14.958146+0000","last_became_peered":"2023-10-06T05:35:14.958146+0000","last_unstale":"2023-10-06T05:35:14.961160+0000","last_undegraded":"2023-10-06T05:35:14.961160+0000","last_fullsized":"2023-10-06T05:35:14.961160+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:29.466625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.23","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.957041+0000","last_change":"2023-10-06T05:35:14.955594+0000","last_active":"2023-10-06T05:35:14.957041+0000","last_peered":"2023-10-06T05:35:14.957041+0000","last_clean":"2023-10-06T05:35:14.957041+0000","last_became_active":"2023-10-06T05:35:14.954688+0000","last_became_peered":"2023-10-06T05:35:14.954688+0000","last_unstale":"2023-10-06T05:35:14.957041+0000","last_undegraded":"2023-10-06T05:35:14.957041+0000","last_fullsized":"2023-10-06T05:35:14.957041+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:24.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.22","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964306+0000","last_change":"2023-10-06T05:35:14.960038+0000","last_active":"2023-10-06T05:35:14.964306+0000","last_peered":"2023-10-06T05:35:14.964306+0000","last_clean":"2023-10-06T05:35:14.964306+0000","last_became_active":"2023-10-06T05:35:14.958611+0000","last_became_peered":"2023-10-06T05:35:14.958611+0000","last_unstale":"2023-10-06T05:35:14.964306+0000","last_undegraded":"2023-10-06T05:35:14.964306+0000","last_fullsized":"2023-10-06T05:35:14.964306+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:09.325090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.21","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.958791+0000","last_change":"2023-10-06T05:35:14.954526+0000","last_active":"2023-10-06T05:35:14.958791+0000","last_peered":"2023-10-06T05:35:14.958791+0000","last_clean":"2023-10-06T05:35:14.958791+0000","last_became_active":"2023-10-06T05:35:14.954274+0000","last_became_peered":"2023-10-06T05:35:14.954274+0000","last_unstale":"2023-10-06T05:35:14.958791+0000","last_undegraded":"2023-10-06T05:35:14.958791+0000","last_fullsized":"2023-10-06T05:35:14.958791+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:24.391202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.20","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975783+0000","last_change":"2023-10-06T05:35:14.971634+0000","last_active":"2023-10-06T05:35:14.975783+0000","last_peered":"2023-10-06T05:35:14.975783+0000","last_clean":"2023-10-06T05:35:14.975783+0000","last_became_active":"2023-10-06T05:35:14.971381+0000","last_became_peered":"2023-10-06T05:35:14.971381+0000","last_unstale":"2023-10-06T05:35:14.975783+0000","last_undegraded":"2023-10-06T05:35:14.975783+0000","last_fullsized":"2023-10-06T05:35:14.975783+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:35.929197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.1c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961144+0000","last_change":"2023-10-06T05:35:14.957821+0000","last_active":"2023-10-06T05:35:14.961144+0000","last_peered":"2023-10-06T05:35:14.961144+0000","last_clean":"2023-10-06T05:35:14.961144+0000","last_became_active":"2023-10-06T05:35:14.957569+0000","last_became_peered":"2023-10-06T05:35:14.957569+0000","last_unstale":"2023-10-06T05:35:14.961144+0000","last_undegraded":"2023-10-06T05:35:14.961144+0000","last_fullsized":"2023-10-06T05:35:14.961144+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:35.929197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1f","version":"281'4","reported_seq":14,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178664+0000","last_change":"2023-10-06T05:35:14.171157+0000","last_active":"2023-10-06T05:35:15.178664+0000","last_peered":"2023-10-06T05:35:15.178664+0000","last_clean":"2023-10-06T05:35:15.178664+0000","last_became_active":"2023-10-06T05:35:14.170854+0000","last_became_peered":"2023-10-06T05:35:14.170854+0000","last_unstale":"2023-10-06T05:35:15.178664+0000","last_undegraded":"2023-10-06T05:35:15.178664+0000","last_fullsized":"2023-10-06T05:35:15.178664+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:09.305299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.1d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959559+0000","last_change":"2023-10-06T05:35:14.956972+0000","last_active":"2023-10-06T05:35:14.959559+0000","last_peered":"2023-10-06T05:35:14.959559+0000","last_clean":"2023-10-06T05:35:14.959559+0000","last_became_active":"2023-10-06T05:35:14.956707+0000","last_became_peered":"2023-10-06T05:35:14.956707+0000","last_unstale":"2023-10-06T05:35:14.959559+0000","last_undegraded":"2023-10-06T05:35:14.959559+0000","last_fullsized":"2023-10-06T05:35:14.959559+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:36.015570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.1e","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178480+0000","last_change":"2023-10-06T05:35:14.160242+0000","last_active":"2023-10-06T05:35:15.178480+0000","last_peered":"2023-10-06T05:35:15.178480+0000","last_clean":"2023-10-06T05:35:15.178480+0000","last_became_active":"2023-10-06T05:35:14.159956+0000","last_became_peered":"2023-10-06T05:35:14.159956+0000","last_unstale":"2023-10-06T05:35:15.178480+0000","last_undegraded":"2023-10-06T05:35:15.178480+0000","last_fullsized":"2023-10-06T05:35:15.178480+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:04.697343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952708+0000","last_change":"2023-10-06T05:35:14.949526+0000","last_active":"2023-10-06T05:35:14.952708+0000","last_peered":"2023-10-06T05:35:14.952708+0000","last_clean":"2023-10-06T05:35:14.952708+0000","last_became_active":"2023-10-06T05:35:14.949245+0000","last_became_peered":"2023-10-06T05:35:14.949245+0000","last_unstale":"2023-10-06T05:35:14.952708+0000","last_undegraded":"2023-10-06T05:35:14.952708+0000","last_fullsized":"2023-10-06T05:35:14.952708+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:47.574339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1d","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.177531+0000","last_change":"2023-10-06T05:35:14.170331+0000","last_active":"2023-10-06T05:35:15.177531+0000","last_peered":"2023-10-06T05:35:15.177531+0000","last_clean":"2023-10-06T05:35:15.177531+0000","last_became_active":"2023-10-06T05:35:14.170013+0000","last_became_peered":"2023-10-06T05:35:14.170013+0000","last_unstale":"2023-10-06T05:35:15.177531+0000","last_undegraded":"2023-10-06T05:35:15.177531+0000","last_fullsized":"2023-10-06T05:35:15.177531+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:02.353698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.1f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970962+0000","last_change":"2023-10-06T05:35:14.965479+0000","last_active":"2023-10-06T05:35:14.970962+0000","last_peered":"2023-10-06T05:35:14.970962+0000","last_clean":"2023-10-06T05:35:14.970962+0000","last_became_active":"2023-10-06T05:35:14.965206+0000","last_became_peered":"2023-10-06T05:35:14.965206+0000","last_unstale":"2023-10-06T05:35:14.970962+0000","last_undegraded":"2023-10-06T05:35:14.970962+0000","last_fullsized":"2023-10-06T05:35:14.970962+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:03.510004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1c","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.175835+0000","last_change":"2023-10-06T05:35:14.169581+0000","last_active":"2023-10-06T05:35:15.175835+0000","last_peered":"2023-10-06T05:35:15.175835+0000","last_clean":"2023-10-06T05:35:15.175835+0000","last_became_active":"2023-10-06T05:35:14.169271+0000","last_became_peered":"2023-10-06T05:35:14.169271+0000","last_unstale":"2023-10-06T05:35:15.175835+0000","last_undegraded":"2023-10-06T05:35:15.175835+0000","last_fullsized":"2023-10-06T05:35:15.175835+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:35.464792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.18","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.958024+0000","last_change":"2023-10-06T05:35:14.955453+0000","last_active":"2023-10-06T05:35:14.958024+0000","last_peered":"2023-10-06T05:35:14.958024+0000","last_clean":"2023-10-06T05:35:14.958024+0000","last_became_active":"2023-10-06T05:35:14.955193+0000","last_became_peered":"2023-10-06T05:35:14.955193+0000","last_unstale":"2023-10-06T05:35:14.958024+0000","last_undegraded":"2023-10-06T05:35:14.958024+0000","last_fullsized":"2023-10-06T05:35:14.958024+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:28.395765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1b","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178027+0000","last_change":"2023-10-06T05:35:14.169263+0000","last_active":"2023-10-06T05:35:15.178027+0000","last_peered":"2023-10-06T05:35:15.178027+0000","last_clean":"2023-10-06T05:35:15.178027+0000","last_became_active":"2023-10-06T05:35:14.168966+0000","last_became_peered":"2023-10-06T05:35:14.168966+0000","last_unstale":"2023-10-06T05:35:15.178027+0000","last_undegraded":"2023-10-06T05:35:15.178027+0000","last_fullsized":"2023-10-06T05:35:15.178027+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:37.206845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.19","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.974319+0000","last_change":"2023-10-06T05:35:14.969094+0000","last_active":"2023-10-06T05:35:14.974319+0000","last_peered":"2023-10-06T05:35:14.974319+0000","last_clean":"2023-10-06T05:35:14.974319+0000","last_became_active":"2023-10-06T05:35:14.968737+0000","last_became_peered":"2023-10-06T05:35:14.968737+0000","last_unstale":"2023-10-06T05:35:14.974319+0000","last_undegraded":"2023-10-06T05:35:14.974319+0000","last_fullsized":"2023-10-06T05:35:14.974319+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:09.057325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1a","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.176478+0000","last_change":"2023-10-06T05:35:14.172187+0000","last_active":"2023-10-06T05:35:15.176478+0000","last_peered":"2023-10-06T05:35:15.176478+0000","last_clean":"2023-10-06T05:35:15.176478+0000","last_became_active":"2023-10-06T05:35:14.171886+0000","last_became_peered":"2023-10-06T05:35:14.171886+0000","last_unstale":"2023-10-06T05:35:15.176478+0000","last_undegraded":"2023-10-06T05:35:15.176478+0000","last_fullsized":"2023-10-06T05:35:15.176478+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:17.296461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.1a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952778+0000","last_change":"2023-10-06T05:35:14.949549+0000","last_active":"2023-10-06T05:35:14.952778+0000","last_peered":"2023-10-06T05:35:14.952778+0000","last_clean":"2023-10-06T05:35:14.952778+0000","last_became_active":"2023-10-06T05:35:14.949278+0000","last_became_peered":"2023-10-06T05:35:14.949278+0000","last_unstale":"2023-10-06T05:35:14.952778+0000","last_undegraded":"2023-10-06T05:35:14.952778+0000","last_fullsized":"2023-10-06T05:35:14.952778+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:11.278905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.19","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.948555+0000","last_change":"2023-10-06T05:35:14.157269+0000","last_active":"2023-10-06T05:35:14.948555+0000","last_peered":"2023-10-06T05:35:14.948555+0000","last_clean":"2023-10-06T05:35:14.948555+0000","last_became_active":"2023-10-06T05:35:14.157003+0000","last_became_peered":"2023-10-06T05:35:14.157003+0000","last_unstale":"2023-10-06T05:35:14.948555+0000","last_undegraded":"2023-10-06T05:35:14.948555+0000","last_fullsized":"2023-10-06T05:35:14.948555+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:54.008885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.1b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.953534+0000","last_change":"2023-10-06T05:35:14.951536+0000","last_active":"2023-10-06T05:35:14.953534+0000","last_peered":"2023-10-06T05:35:14.953534+0000","last_clean":"2023-10-06T05:35:14.953534+0000","last_became_active":"2023-10-06T05:35:14.951195+0000","last_became_peered":"2023-10-06T05:35:14.951195+0000","last_unstale":"2023-10-06T05:35:14.953534+0000","last_undegraded":"2023-10-06T05:35:14.953534+0000","last_fullsized":"2023-10-06T05:35:14.953534+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:24.542923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.18","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952910+0000","last_change":"2023-10-06T05:35:14.171591+0000","last_active":"2023-10-06T05:35:14.952910+0000","last_peered":"2023-10-06T05:35:14.952910+0000","last_clean":"2023-10-06T05:35:14.952910+0000","last_became_active":"2023-10-06T05:35:14.170154+0000","last_became_peered":"2023-10-06T05:35:14.170154+0000","last_unstale":"2023-10-06T05:35:14.952910+0000","last_undegraded":"2023-10-06T05:35:14.952910+0000","last_fullsized":"2023-10-06T05:35:14.952910+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:16.141953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.14","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966832+0000","last_change":"2023-10-06T05:35:14.960359+0000","last_active":"2023-10-06T05:35:14.966832+0000","last_peered":"2023-10-06T05:35:14.966832+0000","last_clean":"2023-10-06T05:35:14.966832+0000","last_became_active":"2023-10-06T05:35:14.959342+0000","last_became_peered":"2023-10-06T05:35:14.959342+0000","last_unstale":"2023-10-06T05:35:14.966832+0000","last_undegraded":"2023-10-06T05:35:14.966832+0000","last_fullsized":"2023-10-06T05:35:14.966832+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:43.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.17","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952030+0000","last_change":"2023-10-06T05:35:14.171324+0000","last_active":"2023-10-06T05:35:14.952030+0000","last_peered":"2023-10-06T05:35:14.952030+0000","last_clean":"2023-10-06T05:35:14.952030+0000","last_became_active":"2023-10-06T05:35:14.171058+0000","last_became_peered":"2023-10-06T05:35:14.171058+0000","last_unstale":"2023-10-06T05:35:14.952030+0000","last_undegraded":"2023-10-06T05:35:14.952030+0000","last_fullsized":"2023-10-06T05:35:14.952030+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:38.076119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.15","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975711+0000","last_change":"2023-10-06T05:35:14.971000+0000","last_active":"2023-10-06T05:35:14.975711+0000","last_peered":"2023-10-06T05:35:14.975711+0000","last_clean":"2023-10-06T05:35:14.975711+0000","last_became_active":"2023-10-06T05:35:14.970732+0000","last_became_peered":"2023-10-06T05:35:14.970732+0000","last_unstale":"2023-10-06T05:35:14.975711+0000","last_undegraded":"2023-10-06T05:35:14.975711+0000","last_fullsized":"2023-10-06T05:35:14.975711+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:58.534732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.16","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.957185+0000","last_change":"2023-10-06T05:35:14.169746+0000","last_active":"2023-10-06T05:35:14.957185+0000","last_peered":"2023-10-06T05:35:14.957185+0000","last_clean":"2023-10-06T05:35:14.957185+0000","last_became_active":"2023-10-06T05:35:14.169452+0000","last_became_peered":"2023-10-06T05:35:14.169452+0000","last_unstale":"2023-10-06T05:35:14.957185+0000","last_undegraded":"2023-10-06T05:35:14.957185+0000","last_fullsized":"2023-10-06T05:35:14.957185+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:54.692971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.16","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961273+0000","last_change":"2023-10-06T05:35:14.958582+0000","last_active":"2023-10-06T05:35:14.961273+0000","last_peered":"2023-10-06T05:35:14.961273+0000","last_clean":"2023-10-06T05:35:14.961273+0000","last_became_active":"2023-10-06T05:35:14.958325+0000","last_became_peered":"2023-10-06T05:35:14.958325+0000","last_unstale":"2023-10-06T05:35:14.961273+0000","last_undegraded":"2023-10-06T05:35:14.961273+0000","last_fullsized":"2023-10-06T05:35:14.961273+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:02.773613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.15","version":"281'3","reported_seq":13,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180569+0000","last_change":"2023-10-06T05:35:14.162068+0000","last_active":"2023-10-06T05:35:15.180569+0000","last_peered":"2023-10-06T05:35:15.180569+0000","last_clean":"2023-10-06T05:35:15.180569+0000","last_became_active":"2023-10-06T05:35:14.161765+0000","last_became_peered":"2023-10-06T05:35:14.161765+0000","last_unstale":"2023-10-06T05:35:15.180569+0000","last_undegraded":"2023-10-06T05:35:15.180569+0000","last_fullsized":"2023-10-06T05:35:15.180569+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:07.056603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.17","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970982+0000","last_change":"2023-10-06T05:35:14.966153+0000","last_active":"2023-10-06T05:35:14.970982+0000","last_peered":"2023-10-06T05:35:14.970982+0000","last_clean":"2023-10-06T05:35:14.970982+0000","last_became_active":"2023-10-06T05:35:14.965885+0000","last_became_peered":"2023-10-06T05:35:14.965885+0000","last_unstale":"2023-10-06T05:35:14.970982+0000","last_undegraded":"2023-10-06T05:35:14.970982+0000","last_fullsized":"2023-10-06T05:35:14.970982+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:41.287166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.14","version":"281'6","reported_seq":16,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180769+0000","last_change":"2023-10-06T05:35:14.170638+0000","last_active":"2023-10-06T05:35:15.180769+0000","last_peered":"2023-10-06T05:35:15.180769+0000","last_clean":"2023-10-06T05:35:15.180769+0000","last_became_active":"2023-10-06T05:35:14.170236+0000","last_became_peered":"2023-10-06T05:35:14.170236+0000","last_unstale":"2023-10-06T05:35:15.180769+0000","last_undegraded":"2023-10-06T05:35:15.180769+0000","last_fullsized":"2023-10-06T05:35:15.180769+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:35.865981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.10","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.954547+0000","last_change":"2023-10-06T05:35:14.952703+0000","last_active":"2023-10-06T05:35:14.954547+0000","last_peered":"2023-10-06T05:35:14.954547+0000","last_clean":"2023-10-06T05:35:14.954547+0000","last_became_active":"2023-10-06T05:35:14.952428+0000","last_became_peered":"2023-10-06T05:35:14.952428+0000","last_unstale":"2023-10-06T05:35:14.954547+0000","last_undegraded":"2023-10-06T05:35:14.954547+0000","last_fullsized":"2023-10-06T05:35:14.954547+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:09.962727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.13","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178141+0000","last_change":"2023-10-06T05:35:14.170563+0000","last_active":"2023-10-06T05:35:15.178141+0000","last_peered":"2023-10-06T05:35:15.178141+0000","last_clean":"2023-10-06T05:35:15.178141+0000","last_became_active":"2023-10-06T05:35:14.170261+0000","last_became_peered":"2023-10-06T05:35:14.170261+0000","last_unstale":"2023-10-06T05:35:15.178141+0000","last_undegraded":"2023-10-06T05:35:15.178141+0000","last_fullsized":"2023-10-06T05:35:15.178141+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:49.511722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.11","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959766+0000","last_change":"2023-10-06T05:35:14.956226+0000","last_active":"2023-10-06T05:35:14.959766+0000","last_peered":"2023-10-06T05:35:14.959766+0000","last_clean":"2023-10-06T05:35:14.959766+0000","last_became_active":"2023-10-06T05:35:14.955621+0000","last_became_peered":"2023-10-06T05:35:14.955621+0000","last_unstale":"2023-10-06T05:35:14.959766+0000","last_undegraded":"2023-10-06T05:35:14.959766+0000","last_fullsized":"2023-10-06T05:35:14.959766+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:33.356172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"41.12","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945576+0000","last_change":"2023-10-06T05:35:14.170815+0000","last_active":"2023-10-06T05:35:14.945576+0000","last_peered":"2023-10-06T05:35:14.945576+0000","last_clean":"2023-10-06T05:35:14.945576+0000","last_became_active":"2023-10-06T05:35:14.170451+0000","last_became_peered":"2023-10-06T05:35:14.170451+0000","last_unstale":"2023-10-06T05:35:14.945576+0000","last_undegraded":"2023-10-06T05:35:14.945576+0000","last_fullsized":"2023-10-06T05:35:14.945576+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:25.087429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.12","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975731+0000","last_change":"2023-10-06T05:35:14.971045+0000","last_active":"2023-10-06T05:35:14.975731+0000","last_peered":"2023-10-06T05:35:14.975731+0000","last_clean":"2023-10-06T05:35:14.975731+0000","last_became_active":"2023-10-06T05:35:14.970755+0000","last_became_peered":"2023-10-06T05:35:14.970755+0000","last_unstale":"2023-10-06T05:35:14.975731+0000","last_undegraded":"2023-10-06T05:35:14.975731+0000","last_fullsized":"2023-10-06T05:35:14.975731+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:29.466625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.11","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.954298+0000","last_change":"2023-10-06T05:35:14.172791+0000","last_active":"2023-10-06T05:35:14.954298+0000","last_peered":"2023-10-06T05:35:14.954298+0000","last_clean":"2023-10-06T05:35:14.954298+0000","last_became_active":"2023-10-06T05:35:14.172516+0000","last_became_peered":"2023-10-06T05:35:14.172516+0000","last_unstale":"2023-10-06T05:35:14.954298+0000","last_undegraded":"2023-10-06T05:35:14.954298+0000","last_fullsized":"2023-10-06T05:35:14.954298+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:56.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.13","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971643+0000","last_change":"2023-10-06T05:35:14.966424+0000","last_active":"2023-10-06T05:35:14.971643+0000","last_peered":"2023-10-06T05:35:14.971643+0000","last_clean":"2023-10-06T05:35:14.971643+0000","last_became_active":"2023-10-06T05:35:14.966095+0000","last_became_peered":"2023-10-06T05:35:14.966095+0000","last_unstale":"2023-10-06T05:35:14.971643+0000","last_undegraded":"2023-10-06T05:35:14.971643+0000","last_fullsized":"2023-10-06T05:35:14.971643+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:40.225983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.10","version":"281'8","reported_seq":18,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.179452+0000","last_change":"2023-10-06T05:35:14.174919+0000","last_active":"2023-10-06T05:35:15.179452+0000","last_peered":"2023-10-06T05:35:15.179452+0000","last_clean":"2023-10-06T05:35:15.179452+0000","last_became_active":"2023-10-06T05:35:14.174608+0000","last_became_peered":"2023-10-06T05:35:14.174608+0000","last_unstale":"2023-10-06T05:35:15.179452+0000","last_undegraded":"2023-10-06T05:35:15.179452+0000","last_fullsized":"2023-10-06T05:35:15.179452+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:02.504014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962339+0000","last_change":"2023-10-06T05:35:14.959390+0000","last_active":"2023-10-06T05:35:14.962339+0000","last_peered":"2023-10-06T05:35:14.962339+0000","last_clean":"2023-10-06T05:35:14.962339+0000","last_became_active":"2023-10-06T05:35:14.959141+0000","last_became_peered":"2023-10-06T05:35:14.959141+0000","last_unstale":"2023-10-06T05:35:14.962339+0000","last_undegraded":"2023-10-06T05:35:14.962339+0000","last_fullsized":"2023-10-06T05:35:14.962339+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:07.812426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.f","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.176457+0000","last_change":"2023-10-06T05:35:14.164959+0000","last_active":"2023-10-06T05:35:15.176457+0000","last_peered":"2023-10-06T05:35:15.176457+0000","last_clean":"2023-10-06T05:35:15.176457+0000","last_became_active":"2023-10-06T05:35:14.164661+0000","last_became_peered":"2023-10-06T05:35:14.164661+0000","last_unstale":"2023-10-06T05:35:15.176457+0000","last_undegraded":"2023-10-06T05:35:15.176457+0000","last_fullsized":"2023-10-06T05:35:15.176457+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:26.921573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.968336+0000","last_change":"2023-10-06T05:35:14.967111+0000","last_active":"2023-10-06T05:35:14.968336+0000","last_peered":"2023-10-06T05:35:14.968336+0000","last_clean":"2023-10-06T05:35:14.968336+0000","last_became_active":"2023-10-06T05:35:14.966845+0000","last_became_peered":"2023-10-06T05:35:14.966845+0000","last_unstale":"2023-10-06T05:35:14.968336+0000","last_undegraded":"2023-10-06T05:35:14.968336+0000","last_fullsized":"2023-10-06T05:35:14.968336+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:28:34.653085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.e","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178563+0000","last_change":"2023-10-06T05:35:14.172723+0000","last_active":"2023-10-06T05:35:15.178563+0000","last_peered":"2023-10-06T05:35:15.178563+0000","last_clean":"2023-10-06T05:35:15.178563+0000","last_became_active":"2023-10-06T05:35:14.172474+0000","last_became_peered":"2023-10-06T05:35:14.172474+0000","last_unstale":"2023-10-06T05:35:15.178563+0000","last_undegraded":"2023-10-06T05:35:15.178563+0000","last_fullsized":"2023-10-06T05:35:15.178563+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:51.255904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961928+0000","last_change":"2023-10-06T05:35:14.961264+0000","last_active":"2023-10-06T05:35:14.961928+0000","last_peered":"2023-10-06T05:35:14.961928+0000","last_clean":"2023-10-06T05:35:14.961928+0000","last_became_active":"2023-10-06T05:35:14.960997+0000","last_became_peered":"2023-10-06T05:35:14.960997+0000","last_unstale":"2023-10-06T05:35:14.961928+0000","last_undegraded":"2023-10-06T05:35:14.961928+0000","last_fullsized":"2023-10-06T05:35:14.961928+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:45.611610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.d","version":"281'1","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180484+0000","last_change":"2023-10-06T05:35:14.169611+0000","last_active":"2023-10-06T05:35:15.180484+0000","last_peered":"2023-10-06T05:35:15.180484+0000","last_clean":"2023-10-06T05:35:15.180484+0000","last_became_active":"2023-10-06T05:35:14.169146+0000","last_became_peered":"2023-10-06T05:35:14.169146+0000","last_unstale":"2023-10-06T05:35:15.180484+0000","last_undegraded":"2023-10-06T05:35:15.180484+0000","last_fullsized":"2023-10-06T05:35:15.180484+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:07.066950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975851+0000","last_change":"2023-10-06T05:35:14.972529+0000","last_active":"2023-10-06T05:35:14.975851+0000","last_peered":"2023-10-06T05:35:14.975851+0000","last_clean":"2023-10-06T05:35:14.975851+0000","last_became_active":"2023-10-06T05:35:14.972243+0000","last_became_peered":"2023-10-06T05:35:14.972243+0000","last_unstale":"2023-10-06T05:35:14.975851+0000","last_undegraded":"2023-10-06T05:35:14.975851+0000","last_fullsized":"2023-10-06T05:35:14.975851+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:02.773613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.c","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950403+0000","last_change":"2023-10-06T05:35:14.159696+0000","last_active":"2023-10-06T05:35:14.950403+0000","last_peered":"2023-10-06T05:35:14.950403+0000","last_clean":"2023-10-06T05:35:14.950403+0000","last_became_active":"2023-10-06T05:35:14.159420+0000","last_became_peered":"2023-10-06T05:35:14.159420+0000","last_unstale":"2023-10-06T05:35:14.950403+0000","last_undegraded":"2023-10-06T05:35:14.950403+0000","last_fullsized":"2023-10-06T05:35:14.950403+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:08.051335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.8","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975995+0000","last_change":"2023-10-06T05:35:14.973360+0000","last_active":"2023-10-06T05:35:14.975995+0000","last_peered":"2023-10-06T05:35:14.975995+0000","last_clean":"2023-10-06T05:35:14.975995+0000","last_became_active":"2023-10-06T05:35:14.973067+0000","last_became_peered":"2023-10-06T05:35:14.973067+0000","last_unstale":"2023-10-06T05:35:14.975995+0000","last_undegraded":"2023-10-06T05:35:14.975995+0000","last_fullsized":"2023-10-06T05:35:14.975995+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:24.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.b","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.180381+0000","last_change":"2023-10-06T05:35:14.165388+0000","last_active":"2023-10-06T05:35:15.180381+0000","last_peered":"2023-10-06T05:35:15.180381+0000","last_clean":"2023-10-06T05:35:15.180381+0000","last_became_active":"2023-10-06T05:35:14.165093+0000","last_became_peered":"2023-10-06T05:35:14.165093+0000","last_unstale":"2023-10-06T05:35:15.180381+0000","last_undegraded":"2023-10-06T05:35:15.180381+0000","last_fullsized":"2023-10-06T05:35:15.180381+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:31.952859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.9","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.962437+0000","last_change":"2023-10-06T05:35:14.957510+0000","last_active":"2023-10-06T05:35:14.962437+0000","last_peered":"2023-10-06T05:35:14.962437+0000","last_clean":"2023-10-06T05:35:14.962437+0000","last_became_active":"2023-10-06T05:35:14.957201+0000","last_became_peered":"2023-10-06T05:35:14.957201+0000","last_unstale":"2023-10-06T05:35:14.962437+0000","last_undegraded":"2023-10-06T05:35:14.962437+0000","last_fullsized":"2023-10-06T05:35:14.962437+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:54.209703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.a","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952293+0000","last_change":"2023-10-06T05:35:14.169771+0000","last_active":"2023-10-06T05:35:14.952293+0000","last_peered":"2023-10-06T05:35:14.952293+0000","last_clean":"2023-10-06T05:35:14.952293+0000","last_became_active":"2023-10-06T05:35:14.169468+0000","last_became_peered":"2023-10-06T05:35:14.169468+0000","last_unstale":"2023-10-06T05:35:14.952293+0000","last_undegraded":"2023-10-06T05:35:14.952293+0000","last_fullsized":"2023-10-06T05:35:14.952293+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:08.051335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.969044+0000","last_change":"2023-10-06T05:35:14.960368+0000","last_active":"2023-10-06T05:35:14.969044+0000","last_peered":"2023-10-06T05:35:14.969044+0000","last_clean":"2023-10-06T05:35:14.969044+0000","last_became_active":"2023-10-06T05:35:14.960113+0000","last_became_peered":"2023-10-06T05:35:14.960113+0000","last_unstale":"2023-10-06T05:35:14.969044+0000","last_undegraded":"2023-10-06T05:35:14.969044+0000","last_fullsized":"2023-10-06T05:35:14.969044+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.514762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.9","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.948655+0000","last_change":"2023-10-06T05:35:14.174404+0000","last_active":"2023-10-06T05:35:14.948655+0000","last_peered":"2023-10-06T05:35:14.948655+0000","last_clean":"2023-10-06T05:35:14.948655+0000","last_became_active":"2023-10-06T05:35:14.173994+0000","last_became_peered":"2023-10-06T05:35:14.173994+0000","last_unstale":"2023-10-06T05:35:14.948655+0000","last_undegraded":"2023-10-06T05:35:14.948655+0000","last_fullsized":"2023-10-06T05:35:14.948655+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:08.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964528+0000","last_change":"2023-10-06T05:35:14.959930+0000","last_active":"2023-10-06T05:35:14.964528+0000","last_peered":"2023-10-06T05:35:14.964528+0000","last_clean":"2023-10-06T05:35:14.964528+0000","last_became_active":"2023-10-06T05:35:14.959662+0000","last_became_peered":"2023-10-06T05:35:14.959662+0000","last_unstale":"2023-10-06T05:35:14.964528+0000","last_undegraded":"2023-10-06T05:35:14.964528+0000","last_fullsized":"2023-10-06T05:35:14.964528+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.514762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.8","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952784+0000","last_change":"2023-10-06T05:35:14.170606+0000","last_active":"2023-10-06T05:35:14.952784+0000","last_peered":"2023-10-06T05:35:14.952784+0000","last_clean":"2023-10-06T05:35:14.952784+0000","last_became_active":"2023-10-06T05:35:14.169831+0000","last_became_peered":"2023-10-06T05:35:14.169831+0000","last_unstale":"2023-10-06T05:35:14.952784+0000","last_undegraded":"2023-10-06T05:35:14.952784+0000","last_fullsized":"2023-10-06T05:35:14.952784+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:42.684244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.4","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975991+0000","last_change":"2023-10-06T05:35:14.970970+0000","last_active":"2023-10-06T05:35:14.975991+0000","last_peered":"2023-10-06T05:35:14.975991+0000","last_clean":"2023-10-06T05:35:14.975991+0000","last_became_active":"2023-10-06T05:35:14.970689+0000","last_became_peered":"2023-10-06T05:35:14.970689+0000","last_unstale":"2023-10-06T05:35:14.975991+0000","last_undegraded":"2023-10-06T05:35:14.975991+0000","last_fullsized":"2023-10-06T05:35:14.975991+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:44.077245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.7","version":"281'2","reported_seq":12,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:15.178665+0000","last_change":"2023-10-06T05:35:14.173660+0000","last_active":"2023-10-06T05:35:15.178665+0000","last_peered":"2023-10-06T05:35:15.178665+0000","last_clean":"2023-10-06T05:35:15.178665+0000","last_became_active":"2023-10-06T05:35:14.173367+0000","last_became_peered":"2023-10-06T05:35:14.173367+0000","last_unstale":"2023-10-06T05:35:15.178665+0000","last_undegraded":"2023-10-06T05:35:15.178665+0000","last_fullsized":"2023-10-06T05:35:15.178665+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:21.111831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.5","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971154+0000","last_change":"2023-10-06T05:35:14.962959+0000","last_active":"2023-10-06T05:35:14.971154+0000","last_peered":"2023-10-06T05:35:14.971154+0000","last_clean":"2023-10-06T05:35:14.971154+0000","last_became_active":"2023-10-06T05:35:14.961859+0000","last_became_peered":"2023-10-06T05:35:14.961859+0000","last_unstale":"2023-10-06T05:35:14.971154+0000","last_undegraded":"2023-10-06T05:35:14.971154+0000","last_fullsized":"2023-10-06T05:35:14.971154+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:33.968063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.6","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.957295+0000","last_change":"2023-10-06T05:35:14.171239+0000","last_active":"2023-10-06T05:35:14.957295+0000","last_peered":"2023-10-06T05:35:14.957295+0000","last_clean":"2023-10-06T05:35:14.957295+0000","last_became_active":"2023-10-06T05:35:14.170908+0000","last_became_peered":"2023-10-06T05:35:14.170908+0000","last_unstale":"2023-10-06T05:35:14.957295+0000","last_undegraded":"2023-10-06T05:35:14.957295+0000","last_fullsized":"2023-10-06T05:35:14.957295+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:32.701909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.6","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.970934+0000","last_change":"2023-10-06T05:35:14.962610+0000","last_active":"2023-10-06T05:35:14.970934+0000","last_peered":"2023-10-06T05:35:14.970934+0000","last_clean":"2023-10-06T05:35:14.970934+0000","last_became_active":"2023-10-06T05:35:14.961691+0000","last_became_peered":"2023-10-06T05:35:14.961691+0000","last_unstale":"2023-10-06T05:35:14.970934+0000","last_undegraded":"2023-10-06T05:35:14.970934+0000","last_fullsized":"2023-10-06T05:35:14.970934+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:44.077245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.5","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.943859+0000","last_change":"2023-10-06T05:35:14.168409+0000","last_active":"2023-10-06T05:35:14.943859+0000","last_peered":"2023-10-06T05:35:14.943859+0000","last_clean":"2023-10-06T05:35:14.943859+0000","last_became_active":"2023-10-06T05:35:14.168188+0000","last_became_peered":"2023-10-06T05:35:14.168188+0000","last_unstale":"2023-10-06T05:35:14.943859+0000","last_undegraded":"2023-10-06T05:35:14.943859+0000","last_fullsized":"2023-10-06T05:35:14.943859+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:15.864842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.7","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.968521+0000","last_change":"2023-10-06T05:35:14.964193+0000","last_active":"2023-10-06T05:35:14.968521+0000","last_peered":"2023-10-06T05:35:14.968521+0000","last_clean":"2023-10-06T05:35:14.968521+0000","last_became_active":"2023-10-06T05:35:14.963905+0000","last_became_peered":"2023-10-06T05:35:14.963905+0000","last_unstale":"2023-10-06T05:35:14.968521+0000","last_undegraded":"2023-10-06T05:35:14.968521+0000","last_fullsized":"2023-10-06T05:35:14.968521+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:07.812426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.4","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945718+0000","last_change":"2023-10-06T05:35:14.163454+0000","last_active":"2023-10-06T05:35:14.945718+0000","last_peered":"2023-10-06T05:35:14.945718+0000","last_clean":"2023-10-06T05:35:14.945718+0000","last_became_active":"2023-10-06T05:35:14.163043+0000","last_became_peered":"2023-10-06T05:35:14.163043+0000","last_unstale":"2023-10-06T05:35:14.945718+0000","last_undegraded":"2023-10-06T05:35:14.945718+0000","last_fullsized":"2023-10-06T05:35:14.945718+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:46.568349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.0","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.971397+0000","last_change":"2023-10-06T05:35:14.966070+0000","last_active":"2023-10-06T05:35:14.971397+0000","last_peered":"2023-10-06T05:35:14.971397+0000","last_clean":"2023-10-06T05:35:14.971397+0000","last_became_active":"2023-10-06T05:35:14.964657+0000","last_became_peered":"2023-10-06T05:35:14.964657+0000","last_unstale":"2023-10-06T05:35:14.971397+0000","last_undegraded":"2023-10-06T05:35:14.971397+0000","last_fullsized":"2023-10-06T05:35:14.971397+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:55.698961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.3","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.944809+0000","last_change":"2023-10-06T05:35:14.163900+0000","last_active":"2023-10-06T05:35:14.944809+0000","last_peered":"2023-10-06T05:35:14.944809+0000","last_clean":"2023-10-06T05:35:14.944809+0000","last_became_active":"2023-10-06T05:35:14.163613+0000","last_became_peered":"2023-10-06T05:35:14.163613+0000","last_unstale":"2023-10-06T05:35:14.944809+0000","last_undegraded":"2023-10-06T05:35:14.944809+0000","last_fullsized":"2023-10-06T05:35:14.944809+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:22.390099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.1","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.967128+0000","last_change":"2023-10-06T05:35:14.963840+0000","last_active":"2023-10-06T05:35:14.967128+0000","last_peered":"2023-10-06T05:35:14.967128+0000","last_clean":"2023-10-06T05:35:14.967128+0000","last_became_active":"2023-10-06T05:35:14.963518+0000","last_became_peered":"2023-10-06T05:35:14.963518+0000","last_unstale":"2023-10-06T05:35:14.967128+0000","last_undegraded":"2023-10-06T05:35:14.967128+0000","last_fullsized":"2023-10-06T05:35:14.967128+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:50.127598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.2","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950357+0000","last_change":"2023-10-06T05:35:14.167638+0000","last_active":"2023-10-06T05:35:14.950357+0000","last_peered":"2023-10-06T05:35:14.950357+0000","last_clean":"2023-10-06T05:35:14.950357+0000","last_became_active":"2023-10-06T05:35:14.167368+0000","last_became_peered":"2023-10-06T05:35:14.167368+0000","last_unstale":"2023-10-06T05:35:14.950357+0000","last_undegraded":"2023-10-06T05:35:14.950357+0000","last_fullsized":"2023-10-06T05:35:14.950357+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:43.745901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959156+0000","last_change":"2023-10-06T05:35:14.954878+0000","last_active":"2023-10-06T05:35:14.959156+0000","last_peered":"2023-10-06T05:35:14.959156+0000","last_clean":"2023-10-06T05:35:14.959156+0000","last_became_active":"2023-10-06T05:35:14.954588+0000","last_became_peered":"2023-10-06T05:35:14.954588+0000","last_unstale":"2023-10-06T05:35:14.959156+0000","last_undegraded":"2023-10-06T05:35:14.959156+0000","last_fullsized":"2023-10-06T05:35:14.959156+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:50.595349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945290+0000","last_change":"2023-10-06T05:35:14.166524+0000","last_active":"2023-10-06T05:35:14.945290+0000","last_peered":"2023-10-06T05:35:14.945290+0000","last_clean":"2023-10-06T05:35:14.945290+0000","last_became_active":"2023-10-06T05:35:14.166246+0000","last_became_peered":"2023-10-06T05:35:14.166246+0000","last_unstale":"2023-10-06T05:35:14.945290+0000","last_undegraded":"2023-10-06T05:35:14.945290+0000","last_fullsized":"2023-10-06T05:35:14.945290+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:08.762974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.3","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961809+0000","last_change":"2023-10-06T05:35:14.955749+0000","last_active":"2023-10-06T05:35:14.961809+0000","last_peered":"2023-10-06T05:35:14.961809+0000","last_clean":"2023-10-06T05:35:14.961809+0000","last_became_active":"2023-10-06T05:35:14.955431+0000","last_became_peered":"2023-10-06T05:35:14.955431+0000","last_unstale":"2023-10-06T05:35:14.961809+0000","last_undegraded":"2023-10-06T05:35:14.961809+0000","last_fullsized":"2023-10-06T05:35:14.961809+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:09.241675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.0","version":"0'0","reported_seq":10,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.952051+0000","last_change":"2023-10-06T05:35:14.174580+0000","last_active":"2023-10-06T05:35:14.952051+0000","last_peered":"2023-10-06T05:35:14.952051+0000","last_clean":"2023-10-06T05:35:14.952051+0000","last_became_active":"2023-10-06T05:35:14.174296+0000","last_became_peered":"2023-10-06T05:35:14.174296+0000","last_unstale":"2023-10-06T05:35:14.952051+0000","last_undegraded":"2023-10-06T05:35:14.952051+0000","last_fullsized":"2023-10-06T05:35:14.952051+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:13.137525+0000","last_clean_scrub_stamp":"2023-10-06T05:35:13.137525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:09.305299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.2c","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.961125+0000","last_change":"2023-10-06T05:35:14.957248+0000","last_active":"2023-10-06T05:35:14.961125+0000","last_peered":"2023-10-06T05:35:14.961125+0000","last_clean":"2023-10-06T05:35:14.961125+0000","last_became_active":"2023-10-06T05:35:14.956989+0000","last_became_peered":"2023-10-06T05:35:14.956989+0000","last_unstale":"2023-10-06T05:35:14.961125+0000","last_undegraded":"2023-10-06T05:35:14.961125+0000","last_fullsized":"2023-10-06T05:35:14.961125+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:24.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":231,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950503+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:35:14.950503+0000","last_peered":"2023-10-06T05:35:14.950503+0000","last_clean":"2023-10-06T05:35:14.950503+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:14.950503+0000","last_undegraded":"2023-10-06T05:35:14.950503+0000","last_fullsized":"2023-10-06T05:35:14.950503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2a","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.959836+0000","last_change":"2023-10-06T05:35:14.957001+0000","last_active":"2023-10-06T05:35:14.959836+0000","last_peered":"2023-10-06T05:35:14.959836+0000","last_clean":"2023-10-06T05:35:14.959836+0000","last_became_active":"2023-10-06T05:35:14.956693+0000","last_became_peered":"2023-10-06T05:35:14.956693+0000","last_unstale":"2023-10-06T05:35:14.959836+0000","last_undegraded":"2023-10-06T05:35:14.959836+0000","last_fullsized":"2023-10-06T05:35:14.959836+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:26.481851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":226,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.950020+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:35:14.950020+0000","last_peered":"2023-10-06T05:35:14.950020+0000","last_clean":"2023-10-06T05:35:14.950020+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:14.950020+0000","last_undegraded":"2023-10-06T05:35:14.950020+0000","last_fullsized":"2023-10-06T05:35:14.950020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.29","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964423+0000","last_change":"2023-10-06T05:35:14.961967+0000","last_active":"2023-10-06T05:35:14.964423+0000","last_peered":"2023-10-06T05:35:14.964423+0000","last_clean":"2023-10-06T05:35:14.964423+0000","last_became_active":"2023-10-06T05:35:14.961687+0000","last_became_peered":"2023-10-06T05:35:14.961687+0000","last_unstale":"2023-10-06T05:35:14.964423+0000","last_undegraded":"2023-10-06T05:35:14.964423+0000","last_fullsized":"2023-10-06T05:35:14.964423+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:19.749199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.943018+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:35:14.943018+0000","last_peered":"2023-10-06T05:35:14.943018+0000","last_clean":"2023-10-06T05:35:14.943018+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:14.943018+0000","last_undegraded":"2023-10-06T05:35:14.943018+0000","last_fullsized":"2023-10-06T05:35:14.943018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.2b","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966755+0000","last_change":"2023-10-06T05:35:14.962866+0000","last_active":"2023-10-06T05:35:14.966755+0000","last_peered":"2023-10-06T05:35:14.966755+0000","last_clean":"2023-10-06T05:35:14.966755+0000","last_became_active":"2023-10-06T05:35:14.962317+0000","last_became_peered":"2023-10-06T05:35:14.962317+0000","last_unstale":"2023-10-06T05:35:14.966755+0000","last_undegraded":"2023-10-06T05:35:14.966755+0000","last_fullsized":"2023-10-06T05:35:14.966755+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:27.487051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":226,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949588+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:35:14.949588+0000","last_peered":"2023-10-06T05:35:14.949588+0000","last_clean":"2023-10-06T05:35:14.949588+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:14.949588+0000","last_undegraded":"2023-10-06T05:35:14.949588+0000","last_fullsized":"2023-10-06T05:35:14.949588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":376,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949844+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:35:14.949844+0000","last_peered":"2023-10-06T05:35:14.949844+0000","last_clean":"2023-10-06T05:35:14.949844+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:14.949844+0000","last_undegraded":"2023-10-06T05:35:14.949844+0000","last_fullsized":"2023-10-06T05:35:14.949844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.28","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.975803+0000","last_change":"2023-10-06T05:35:14.972273+0000","last_active":"2023-10-06T05:35:14.975803+0000","last_peered":"2023-10-06T05:35:14.975803+0000","last_clean":"2023-10-06T05:35:14.975803+0000","last_became_active":"2023-10-06T05:35:14.972017+0000","last_became_peered":"2023-10-06T05:35:14.972017+0000","last_unstale":"2023-10-06T05:35:14.975803+0000","last_undegraded":"2023-10-06T05:35:14.975803+0000","last_fullsized":"2023-10-06T05:35:14.975803+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:18.649823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":223,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949693+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:35:14.949693+0000","last_peered":"2023-10-06T05:35:14.949693+0000","last_clean":"2023-10-06T05:35:14.949693+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:14.949693+0000","last_undegraded":"2023-10-06T05:35:14.949693+0000","last_fullsized":"2023-10-06T05:35:14.949693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2d","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.966797+0000","last_change":"2023-10-06T05:35:14.963971+0000","last_active":"2023-10-06T05:35:14.966797+0000","last_peered":"2023-10-06T05:35:14.966797+0000","last_clean":"2023-10-06T05:35:14.966797+0000","last_became_active":"2023-10-06T05:35:14.963681+0000","last_became_peered":"2023-10-06T05:35:14.963681+0000","last_unstale":"2023-10-06T05:35:14.966797+0000","last_undegraded":"2023-10-06T05:35:14.966797+0000","last_fullsized":"2023-10-06T05:35:14.966797+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:22.458660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":223,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.951299+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:35:14.951299+0000","last_peered":"2023-10-06T05:35:14.951299+0000","last_clean":"2023-10-06T05:35:14.951299+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:14.951299+0000","last_undegraded":"2023-10-06T05:35:14.951299+0000","last_fullsized":"2023-10-06T05:35:14.951299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2e","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.967137+0000","last_change":"2023-10-06T05:35:14.964083+0000","last_active":"2023-10-06T05:35:14.967137+0000","last_peered":"2023-10-06T05:35:14.967137+0000","last_clean":"2023-10-06T05:35:14.967137+0000","last_became_active":"2023-10-06T05:35:14.963502+0000","last_became_peered":"2023-10-06T05:35:14.963502+0000","last_unstale":"2023-10-06T05:35:14.967137+0000","last_undegraded":"2023-10-06T05:35:14.967137+0000","last_fullsized":"2023-10-06T05:35:14.967137+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:58.534732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":231,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.949230+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:35:14.949230+0000","last_peered":"2023-10-06T05:35:14.949230+0000","last_clean":"2023-10-06T05:35:14.949230+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:14.949230+0000","last_undegraded":"2023-10-06T05:35:14.949230+0000","last_fullsized":"2023-10-06T05:35:14.949230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2f","version":"0'0","reported_seq":9,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.964265+0000","last_change":"2023-10-06T05:35:14.961862+0000","last_active":"2023-10-06T05:35:14.964265+0000","last_peered":"2023-10-06T05:35:14.964265+0000","last_clean":"2023-10-06T05:35:14.964265+0000","last_became_active":"2023-10-06T05:35:14.961601+0000","last_became_peered":"2023-10-06T05:35:14.961601+0000","last_unstale":"2023-10-06T05:35:14.964265+0000","last_undegraded":"2023-10-06T05:35:14.964265+0000","last_fullsized":"2023-10-06T05:35:14.964265+0000","mapping_epoch":280,"log_start":"0'0","ondisk_log_start":"0'0","created":280,"last_epoch_clean":281,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:14.143515+0000","last_clean_scrub_stamp":"2023-10-06T05:35:14.143515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:57:57.891612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-06T05:35:14.945697+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:35:14.945697+0000","last_peered":"2023-10-06T05:35:14.945697+0000","last_clean":"2023-10-06T05:35:14.945697+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:14.945697+0000","last_undegraded":"2023-10-06T05:35:14.945697+0000","last_fullsized":"2023-10-06T05:35:14.945697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":42,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":41,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575106,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309560,"kb_used_data":4080,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062280,"statfs":{"total":96636764160,"available":96319774720,"internally_reserved":0,"allocated":4177920,"data_stored":2274934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":1.026},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":1.01}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.629},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308976,"kb_used_data":3496,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062864,"statfs":{"total":96636764160,"available":96320372736,"internally_reserved":0,"allocated":3579904,"data_stored":1684476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.47899999999999998,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.70599999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.751},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.115},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.56000000000000005,"5min":0.504,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.022,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.54400000000000004,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":1.0129999999999999,"5min":1.143,"15min":1.143},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.038,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575105,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304892,"kb_used_data":3508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066948,"statfs":{"total":96636764160,"available":96324554752,"internally_reserved":0,"allocated":3592192,"data_stored":1685122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":1.272,"15min":1.272},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.68000000000000005,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.312}]},{"osd":2,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.72199999999999998,"15min":1.165},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":1.22,"15min":1.22},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.432},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":1.238,"15min":1.2430000000000001},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84999999999999998,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96299999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1080000000000001,"5min":1.492,"15min":1.492},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.262,"15min":1.2909999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.82999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.0609999999999999,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.85299999999999998,"15min":0.82199999999999995},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":0.92300000000000004,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575105,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313664,"kb_used_data":4088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058176,"statfs":{"total":96636764160,"available":96315572224,"internally_reserved":0,"allocated":4186112,"data_stored":2275558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.95299999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.375}]},{"osd":3,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.307},"max":{"1min":0.628,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0980000000000001,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.1379999999999999,"15min":1.27},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.49199999999999999},"max":{"1min":1.151,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.81499999999999995,"15min":0.80100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.147,"15min":1.153},"last":0.91400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.496},"max":{"1min":1.0629999999999999,"5min":1.2,"15min":1.2010000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.90800000000000003,"5min":1.1459999999999999,"15min":1.274},"last":0.82299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575105,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308992,"kb_used_data":3512,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062848,"statfs":{"total":96636764160,"available":96320356352,"internally_reserved":0,"allocated":3596288,"data_stored":1686034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.79600000000000004,"5min":0.82999999999999996,"15min":1.056},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57099999999999995,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.64000000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.58599999999999997,"5min":1.117,"15min":1.117},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95599999999999996,"5min":1.2709999999999999,"15min":1.486},"last":0.627},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.089,"15min":1.5},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93899999999999995,"5min":1.157,"15min":1.1639999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92400000000000004,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.22,"15min":1.22},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.59699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.341,"15min":1.341},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":1.282,"15min":1.538},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.264,"15min":1.264},"last":0.78300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575106,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308976,"kb_used_data":3496,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062864,"statfs":{"total":96636764160,"available":96320372736,"internally_reserved":0,"allocated":3579904,"data_stored":1684476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.872},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.82999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575104,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308980,"kb_used_data":3500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062860,"statfs":{"total":96636764160,"available":96320368640,"internally_reserved":0,"allocated":3584000,"data_stored":1684510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.751},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.877}]},{"osd":1,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.82199999999999995,"5min":1.026,"15min":1.026},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.69499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41799999999999998},"max":{"1min":0.88600000000000001,"5min":1.099,"15min":1.099},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76200000000000001,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.443},"max":{"1min":0.85699999999999998,"5min":3.839,"15min":3.839},"last":0.745},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.873,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":0.96499999999999997,"15min":1.0609999999999999},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":0.93999999999999995,"5min":1.109,"15min":1.282},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.72499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575106,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304884,"kb_used_data":3500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066956,"statfs":{"total":96636764160,"available":96324562944,"internally_reserved":0,"allocated":3584000,"data_stored":1679405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.73999999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.372},"max":{"1min":1.071,"5min":1.115,"15min":1.2130000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.55700000000000005,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":1.169,"15min":1.169},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.145},"last":0.77900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.92800000000000005,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.02,"15min":1.1539999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.96899999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.52000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.87},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.60199999999999998,"5min":0.53300000000000003,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.0609999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:35:20.035 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:35:20.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:35:20.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.261+0000 7f5f7719d700 1 Processor -- start 2023-10-06T05:35:20.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.262+0000 7f5f7719d700 1 -- start start 2023-10-06T05:35:20.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.263+0000 7f5f7719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f70067e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.263+0000 7f5f7719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700b0350 con 0x7f5f70068340 2023-10-06T05:35:20.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.263+0000 7f5f7719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700b0630 con 0x7f5f700cc600 2023-10-06T05:35:20.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.263+0000 7f5f7719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70159d10 con 0x7f5f700cc9d0 2023-10-06T05:35:20.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.263+0000 7f5f7699c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5f70068340 0x7f5f7006c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40964/0 (socket says 172.21.15.33:40964) 2023-10-06T05:35:20.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.263+0000 7f5f7699c700 1 -- 172.21.15.33:0/2505088484 learned_addr learned my addr 172.21.15.33:0/2505088484 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f7599a700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f70067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854067275 0 0) 0x7f5f700b0350 con 0x7f5f70068340 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f5c003570 con 0x7f5f70068340 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532737514 0 0) 0x7f5f700b0630 con 0x7f5f700cc600 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f700b0350 con 0x7f5f700cc600 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888808085 0 0) 0x7f5f5c003570 con 0x7f5f70068340 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 msgr2=0x7f5f70067e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:20.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f70067e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 >> v1:172.21.15.167:6789/0 conn(0x7f5f700cc600 legacy=0x7f5f700ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700b0910 con 0x7f5f70068340 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f5f68001770 con 0x7f5f70068340 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 >> v1:172.21.15.33:6789/0 conn(0x7f5f70068340 legacy=0x7f5f7006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f5c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f5c0041d0 0x7f5f5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c004d50 0x7f5f5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70159d10 con 0x7f5f5c004d50 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c00bbf0 con 0x7f5f5c0041d0 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.264+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c00bdb0 con 0x7f5f700cc9d0 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f5c0041d0 0x7f5f5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7619b700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f5c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7699c700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c004d50 0x7f5f5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 -- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 msgr2=0x7f5f5c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f5c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 -- 172.21.15.33:0/2505088484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c004d50 msgr2=0x7f5f5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c004d50 0x7f5f5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 -- 172.21.15.33:0/2505088484 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c003570 con 0x7f5f5c0041d0 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7619b700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f700cc9d0 0x7f5f5c003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.265+0000 7f5f7599a700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f5c0041d0 0x7f5f5c004810 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009c00 tx=0x7f5f600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915b0091959aff33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000e710 con 0x7f5f5c0041d0 2023-10-06T05:35:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000e870 con 0x7f5f5c0041d0 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f75199700 1 -- 172.21.15.33:0/2505088484 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000ec30 con 0x7f5f5c0041d0 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7719d700 1 -- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f5c0041d0 msgr2=0x7f5f5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7719d700 1 --2- 172.21.15.33:0/2505088484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f5c0041d0 0x7f5f5c004810 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009c00 tx=0x7f5f600091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7699c700 1 -- 172.21.15.33:0/2505088484 reap_dead start 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7719d700 1 -- 172.21.15.33:0/2505088484 shutdown_connections 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7719d700 1 -- 172.21.15.33:0/2505088484 >> 172.21.15.33:0/2505088484 conn(0x7f5f7006d670 msgr2=0x7f5f7006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7719d700 1 -- 172.21.15.33:0/2505088484 shutdown_connections 2023-10-06T05:35:20.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.266+0000 7f5f7719d700 1 -- 172.21.15.33:0/2505088484 wait complete. 2023-10-06T05:35:20.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.268+0000 7f5f7719d700 1 Processor -- start 2023-10-06T05:35:20.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.268+0000 7f5f7719d700 1 -- start start 2023-10-06T05:35:20.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f5c0041d0 0x7f5f7016ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7014d000 0x7f5f701574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f701579e0 0x7f5f700acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700aee00 con 0x7f5f701579e0 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70157e50 con 0x7f5f7014d000 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7016b260 con 0x7f5f5c0041d0 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7599a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7014d000 0x7f5f701574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f701579e0 0x7f5f700acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7619b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f5c0041d0 0x7f5f7016ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7599a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7014d000 0x7f5f701574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:59764/0 (socket says 172.21.15.33:59764) 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.269+0000 7f5f7599a700 1 -- 172.21.15.33:0/782432276 learned_addr learned my addr 172.21.15.33:0/782432276 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:20.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7699c700 1 -- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f5c0041d0 msgr2=0x7f5f7016ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7699c700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f5c0041d0 0x7f5f7016ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7699c700 1 -- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7014d000 msgr2=0x7f5f701574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7699c700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7014d000 0x7f5f701574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7699c700 1 -- 172.21.15.33:0/782432276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700ae180 con 0x7f5f701579e0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7699c700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f701579e0 0x7f5f700acb80 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f5f70064640 tx=0x7f5f68002020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800fb80 con 0x7f5f701579e0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800fce0 con 0x7f5f701579e0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f700ae460 con 0x7f5f701579e0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68013640 con 0x7f5f701579e0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.270+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f700aea20 con 0x7f5f701579e0 2023-10-06T05:35:20.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.271+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f700aef40 con 0x7f5f701579e0 2023-10-06T05:35:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.272+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f680139f0 con 0x7f5f701579e0 2023-10-06T05:35:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.273+0000 7f5f667fc700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f54066f30 0x7f5f540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.273+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5f68084bb0 con 0x7f5f701579e0 2023-10-06T05:35:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.274+0000 7f5f7619b700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f54066f30 0x7f5f540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.274+0000 7f5f7619b700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f54066f30 0x7f5f540693e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5f70154d60 tx=0x7f5f6c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.277+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f700aef40 con 0x7f5f701579e0 2023-10-06T05:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.560+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5f700aef40 con 0x7f5f701579e0 2023-10-06T05:35:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.564+0000 7f5f667fc700 1 -- 172.21.15.33:0/782432276 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5f700aef40 con 0x7f5f701579e0 2023-10-06T05:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f54066f30 msgr2=0x7f5f540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f54066f30 0x7f5f540693e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5f70154d60 tx=0x7f5f6c007d30 comp rx=0 tx=0).stop 2023-10-06T05:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f701579e0 msgr2=0x7f5f700acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f701579e0 0x7f5f700acb80 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f5f70064640 tx=0x7f5f68002020 comp rx=0 tx=0).stop 2023-10-06T05:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 shutdown_connections 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5f54066f30 0x7f5f540693e0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f701579e0 0x7f5f700acb80 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f5c0041d0 0x7f5f7016ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 --2- 172.21.15.33:0/782432276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7014d000 0x7f5f701574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 >> 172.21.15.33:0/782432276 conn(0x7f5f7006d670 msgr2=0x7f5f700aab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 shutdown_connections 2023-10-06T05:35:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.566+0000 7f5f7719d700 1 -- 172.21.15.33:0/782432276 wait complete. 2023-10-06T05:35:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T05:35:20.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:20.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.770+0000 7f5d7759e700 1 Processor -- start 2023-10-06T05:35:20.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.770+0000 7f5d7759e700 1 -- start start 2023-10-06T05:35:20.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.771+0000 7f5d7759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5d780ccfc0 0x7f5d780cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.771+0000 7f5d7759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780cbd80 con 0x7f5d780cf3b0 2023-10-06T05:35:20.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.771+0000 7f5d7759e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780af5f0 con 0x7f5d780cc750 2023-10-06T05:35:20.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.771+0000 7f5d7759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780af8d0 con 0x7f5d780ccfc0 2023-10-06T05:35:20.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d75d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5d780cf3b0 0x7f5d781594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40972/0 (socket says 172.21.15.33:40972) 2023-10-06T05:35:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d75d9b700 1 -- 172.21.15.33:0/500812099 learned_addr learned my addr 172.21.15.33:0/500812099 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d7659c700 1 --2- 172.21.15.33:0/500812099 >> v2:172.21.15.167:3300/0 conn(0x7f5d780ccfc0 0x7f5d780cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d7559a700 1 -- 172.21.15.33:0/500812099 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957939871 0 0) 0x7f5d780cbd80 con 0x7f5d780cf3b0 2023-10-06T05:35:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d7559a700 1 -- 172.21.15.33:0/500812099 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d60003570 con 0x7f5d780cf3b0 2023-10-06T05:35:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d7659c700 1 -- 172.21.15.33:0/500812099 >> v1:172.21.15.167:6789/0 conn(0x7f5d780cc750 legacy=0x7f5d780b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d7659c700 1 -- 172.21.15.33:0/500812099 >> v1:172.21.15.33:6789/0 conn(0x7f5d780cf3b0 legacy=0x7f5d781594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.772+0000 7f5d7659c700 1 -- 172.21.15.33:0/500812099 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780afbb0 con 0x7f5d780ccfc0 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7659c700 1 --2- 172.21.15.33:0/500812099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780ccfc0 0x7f5d780cc210 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009910 tx=0x7f5d680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e5849d89369bb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7559a700 1 -- 172.21.15.33:0/500812099 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800b640 con 0x7f5d780ccfc0 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7559a700 1 -- 172.21.15.33:0/500812099 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6800b7a0 con 0x7f5d780ccfc0 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7559a700 1 -- 172.21.15.33:0/500812099 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800bb60 con 0x7f5d780ccfc0 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7759e700 1 -- 172.21.15.33:0/500812099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780ccfc0 msgr2=0x7f5d780cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7759e700 1 --2- 172.21.15.33:0/500812099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780ccfc0 0x7f5d780cc210 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009910 tx=0x7f5d680091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7759e700 1 -- 172.21.15.33:0/500812099 shutdown_connections 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7759e700 1 --2- 172.21.15.33:0/500812099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780ccfc0 0x7f5d780cc210 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.773+0000 7f5d7759e700 1 -- 172.21.15.33:0/500812099 >> 172.21.15.33:0/500812099 conn(0x7f5d7806cff0 msgr2=0x7f5d7806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.774+0000 7f5d7759e700 1 -- 172.21.15.33:0/500812099 shutdown_connections 2023-10-06T05:35:20.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.774+0000 7f5d7759e700 1 -- 172.21.15.33:0/500812099 wait complete. 2023-10-06T05:35:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.775+0000 7f5d7759e700 1 Processor -- start 2023-10-06T05:35:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.775+0000 7f5d7759e700 1 -- start start 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d7759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780cc750 0x7f5d7816dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d7759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780ccfc0 0x7f5d780c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d7759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 0x7f5d780c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d7759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c7ee0 con 0x7f5d780c9be0 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d7759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7816d4b0 con 0x7f5d780cc750 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d7759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780ca0d0 con 0x7f5d780ccfc0 2023-10-06T05:35:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d76d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 0x7f5d780c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d76d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 0x7f5d780c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39356/0 (socket says 172.21.15.33:39356) 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.776+0000 7f5d76d9d700 1 -- 172.21.15.33:0/4107825340 learned_addr learned my addr 172.21.15.33:0/4107825340 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d7659c700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780cc750 0x7f5d7816dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d75d9b700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780ccfc0 0x7f5d780c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d76d9d700 1 -- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780ccfc0 msgr2=0x7f5d780c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d76d9d700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780ccfc0 0x7f5d780c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d76d9d700 1 -- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780cc750 msgr2=0x7f5d7816dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d76d9d700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780cc750 0x7f5d7816dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d76d9d700 1 -- 172.21.15.33:0/4107825340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68009420 con 0x7f5d780c9be0 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d75d9b700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780ccfc0 0x7f5d780c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d7659c700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780cc750 0x7f5d7816dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d76d9d700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 0x7f5d780c5ff0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c8e0 tx=0x7f5d70001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000ce80 con 0x7f5d780c9be0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000a7e0 con 0x7f5d780c9be0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000ab80 con 0x7f5d780c9be0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.777+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780c75b0 con 0x7f5d780c9be0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.778+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d780c7b70 con 0x7f5d780c9be0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.778+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7804c380 con 0x7f5d780c9be0 2023-10-06T05:35:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.780+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d70012420 con 0x7f5d780c9be0 2023-10-06T05:35:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.780+0000 7f5d667fc700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d58066f60 0x7f5d58069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.780+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d70084a20 con 0x7f5d780c9be0 2023-10-06T05:35:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.781+0000 7f5d7659c700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d58066f60 0x7f5d58069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.782+0000 7f5d7659c700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d58066f60 0x7f5d58069410 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5d680093f0 tx=0x7f5d68017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:20.784+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d70051a90 con 0x7f5d780c9be0 2023-10-06T05:35:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.028+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d78156730 con 0x7f5d780c9be0 2023-10-06T05:35:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.029+0000 7f5d667fc700 1 -- 172.21.15.33:0/4107825340 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f5d700550b0 con 0x7f5d780c9be0 2023-10-06T05:35:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:21.031 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:21.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.031+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d58066f60 msgr2=0x7f5d58069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.031+0000 7f5d7759e700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d58066f60 0x7f5d58069410 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5d680093f0 tx=0x7f5d68017000 comp rx=0 tx=0).stop 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.031+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 msgr2=0x7f5d780c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.031+0000 7f5d7759e700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 0x7f5d780c5ff0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c8e0 tx=0x7f5d70001710 comp rx=0 tx=0).stop 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.031+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 shutdown_connections 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5d58066f60 0x7f5d58069410 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d780c9be0 0x7f5d780c5ff0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d780cc750 0x7f5d7816dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 --2- 172.21.15.33:0/4107825340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d780ccfc0 0x7f5d780c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 >> 172.21.15.33:0/4107825340 conn(0x7f5d7806cff0 msgr2=0x7f5d78150640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 shutdown_connections 2023-10-06T05:35:21.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.032+0000 7f5d7759e700 1 -- 172.21.15.33:0/4107825340 wait complete. 2023-10-06T05:35:21.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:21.144 INFO:tasks.ceph.mds.d.smithi167.stderr:2023-10-06T05:35:21.142+0000 7efdd57f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:35:21.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 219, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:35:14.652229+0000', 'modified': '2023-10-06T05:35:18.187888+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12264': {'gid': 12264, 'name': 'd', 'rank': 0, 'incarnation': 215, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6836/3559384937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3559384937}, {'type': 'v1', 'addr': '172.21.15.167:6836', 'nonce': 3559384937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [42], 'metadata_pool': 41, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:35:21.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:35:21.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:35:21.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.642+0000 7fd32979d700 1 Processor -- start 2023-10-06T05:35:21.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.642+0000 7fd32979d700 1 -- start start 2023-10-06T05:35:21.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd32979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd32979d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd324070810 con 0x7fd32406c560 2023-10-06T05:35:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd32979d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32406da40 con 0x7fd3240ca7a0 2023-10-06T05:35:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd32979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32406dd20 con 0x7fd3241598a0 2023-10-06T05:35:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd3237fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd32406c560 0x7fd3240c7b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40974/0 (socket says 172.21.15.33:40974) 2023-10-06T05:35:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd3237fe700 1 -- 172.21.15.33:0/2435518559 learned_addr learned my addr 172.21.15.33:0/2435518559 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd328f9c700 1 --2- 172.21.15.33:0/2435518559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.644+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674396893 0 0) 0x7fd324070810 con 0x7fd32406c560 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30c003570 con 0x7fd32406c560 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565923968 0 0) 0x7fd32406da40 con 0x7fd3240ca7a0 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd324070810 con 0x7fd3240ca7a0 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd328f9c700 1 -- 172.21.15.33:0/2435518559 >> v1:172.21.15.167:6789/0 conn(0x7fd3240ca7a0 legacy=0x7fd324159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd328f9c700 1 -- 172.21.15.33:0/2435518559 >> v1:172.21.15.33:6789/0 conn(0x7fd32406c560 legacy=0x7fd3240c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd328f9c700 1 -- 172.21.15.33:0/2435518559 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240c7060 con 0x7fd3241598a0 2023-10-06T05:35:21.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd328f9c700 1 --2- 172.21.15.33:0/2435518559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c8250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009e90 tx=0x7fd31c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70667558ebe5e53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00e070 con 0x7fd3241598a0 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c00bb80 con 0x7fd3241598a0 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.645+0000 7fd322ffd700 1 -- 172.21.15.33:0/2435518559 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c012430 con 0x7fd3241598a0 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 -- 172.21.15.33:0/2435518559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 msgr2=0x7fd3240c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 --2- 172.21.15.33:0/2435518559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c8250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009e90 tx=0x7fd31c009780 comp rx=0 tx=0).stop 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 -- 172.21.15.33:0/2435518559 shutdown_connections 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 --2- 172.21.15.33:0/2435518559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c8250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 -- 172.21.15.33:0/2435518559 >> 172.21.15.33:0/2435518559 conn(0x7fd32406e650 msgr2=0x7fd32406ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 -- 172.21.15.33:0/2435518559 shutdown_connections 2023-10-06T05:35:21.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.646+0000 7fd32979d700 1 -- 172.21.15.33:0/2435518559 wait complete. 2023-10-06T05:35:21.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.647+0000 7fd32979d700 1 Processor -- start 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 -- start start 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 0x7fd32414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3240ca7a0 0x7fd3240c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3240c6bc0 con 0x7fd32406c560 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3241570a0 con 0x7fd3240ca7a0 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.648+0000 7fd32979d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd324157380 con 0x7fd3241598a0 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 0x7fd32414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 0x7fd32414e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39360/0 (socket says 172.21.15.33:39360) 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 -- 172.21.15.33:0/2471964583 learned_addr learned my addr 172.21.15.33:0/2471964583 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd3237fe700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3240ca7a0 0x7fd3240c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd328f9c700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 -- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 msgr2=0x7fd3240c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 -- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3240ca7a0 msgr2=0x7fd3240c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3240ca7a0 0x7fd3240c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 -- 172.21.15.33:0/2471964583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c009a20 con 0x7fd32406c560 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.649+0000 7fd323fff700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 0x7fd32414e5c0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c790 tx=0x7fd314007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.650+0000 7fd30bfff700 1 -- 172.21.15.33:0/2471964583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314010510 con 0x7fd32406c560 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.650+0000 7fd30bfff700 1 -- 172.21.15.33:0/2471964583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314010690 con 0x7fd32406c560 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.650+0000 7fd30bfff700 1 -- 172.21.15.33:0/2471964583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314010a50 con 0x7fd32406c560 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.650+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd324157660 con 0x7fd32406c560 2023-10-06T05:35:21.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.650+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32414eb00 con 0x7fd32406c560 2023-10-06T05:35:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.652+0000 7fd30bfff700 1 -- 172.21.15.33:0/2471964583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd31401b400 con 0x7fd32406c560 2023-10-06T05:35:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.653+0000 7fd30bfff700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd310066f30 0x7fd3100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.653+0000 7fd30bfff700 1 -- 172.21.15.33:0/2471964583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd314084460 con 0x7fd32406c560 2023-10-06T05:35:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.653+0000 7fd3237fe700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd310066f30 0x7fd3100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.654+0000 7fd3237fe700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd310066f30 0x7fd3100693e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd3180058d0 tx=0x7fd318009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.655+0000 7fd33905eb80 1 Processor -- start 2023-10-06T05:35:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.655+0000 7fd33905eb80 1 -- start start 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511a86950 0x563511a86d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511ad0bc0 0x563511aaf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 0x563511a82990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd33905eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563511a06c80 con 0x563511aafe60 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635119dda70 con 0x563511ad0bc0 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635119db270 con 0x563511a86950 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 0x563511a82990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 0x563511a82990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39372/0 (socket says 172.21.15.33:39372) 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 -- 172.21.15.33:0/3020672224 learned_addr learned my addr 172.21.15.33:0/3020672224 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd323fff700 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511a86950 0x563511a86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd3237fe700 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511ad0bc0 0x563511aaf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 -- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511a86950 msgr2=0x563511a86d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511a86950 0x563511a86d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 -- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511ad0bc0 msgr2=0x563511aaf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511ad0bc0 0x563511aaf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd328f9c700 1 -- 172.21.15.33:0/3020672224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563511871270 con 0x563511aafe60 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.656+0000 7fd3237fe700 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511ad0bc0 0x563511aaf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.657+0000 7fd328f9c700 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 0x563511a82990 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fd3240cc4f0 tx=0x7fd31c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bc02dec9d1fbe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.657+0000 7fd3097fa700 1 -- 172.21.15.33:0/3020672224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c021070 con 0x563511aafe60 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.657+0000 7fd3097fa700 1 -- 172.21.15.33:0/3020672224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c0032f0 con 0x563511aafe60 2023-10-06T05:35:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.657+0000 7fd3097fa700 1 -- 172.21.15.33:0/3020672224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0127f0 con 0x563511aafe60 2023-10-06T05:35:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.657+0000 7fd33905eb80 1 -- 172.21.15.33:0/3020672224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 msgr2=0x563511a82990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.657+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 0x563511a82990 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fd3240cc4f0 tx=0x7fd31c019000 comp rx=0 tx=0).stop 2023-10-06T05:35:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 -- 172.21.15.33:0/3020672224 shutdown_connections 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511aafe60 0x563511a82990 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511a86950 0x563511a86d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3020672224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511ad0bc0 0x563511aaf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 -- 172.21.15.33:0/3020672224 >> 172.21.15.33:0/3020672224 conn(0x563511a88270 msgr2=0x563511a88640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 -- 172.21.15.33:0/3020672224 shutdown_connections 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 -- 172.21.15.33:0/3020672224 wait complete. 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.658+0000 7fd33905eb80 1 -- start start 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.659+0000 7fd33905eb80 1 Processor -- start 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511ac6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511adcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 0x563511aaea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635119d7060 con 0x563511ad0bc0 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563511982890 con 0x563511a86950 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56351199dea0 con 0x563511aafe60 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd33905eb80 10 client.0 _ll_register_callbacks cb 0x563511873f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 0x563511aaea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 0x563511aaea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39380/0 (socket says 172.21.15.33:39380) 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 -- 172.21.15.33:0/1561280207 learned_addr learned my addr 172.21.15.33:0/1561280207 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd323fff700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511ac6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd3237fe700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511adcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 -- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 msgr2=0x563511adcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511adcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 -- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 msgr2=0x563511ac6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511ac6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.660+0000 7fd328f9c700 1 -- 172.21.15.33:0/1561280207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3240c7060 con 0x563511ad0bc0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd328f9c700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 0x563511aaea60 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097c0 tx=0x7fd31c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36d2d95bbe5b499 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd3027fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd3027fc700 1 -- 172.21.15.33:0/1561280207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c030440 con 0x563511ad0bc0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd3027fc700 1 -- 172.21.15.33:0/1561280207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd31c0305a0 con 0x563511ad0bc0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563511a585c0 con 0x563511ad0bc0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd3027fc700 1 -- 172.21.15.33:0/1561280207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd31c030970 con 0x563511ad0bc0 2023-10-06T05:35:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd33905eb80 10 client.12408 fetch_fsmap learned FSMap version 219 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=219,monmap=2+}) v3 -- 0x563511813fe0 con 0x563511ad0bc0 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.661+0000 7fd3027fc700 1 -- 172.21.15.33:0/1561280207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563511a585c0 con 0x563511ad0bc0 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd3027fc700 1 -- 172.21.15.33:0/1561280207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 219) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd31c030ad0 con 0x563511ad0bc0 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd33905eb80 10 client.12408 fetch_fsmap finished waiting for FSMap version 219 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd33905eb80 10 client.12408 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd33905eb80 20 client.12408 populate_metadata read hostname 'smithi033' 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd33905eb80 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ac5290 0x563511a7d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd33905eb80 4 client.12408 mds_command: new command op to 12264 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 --> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563511979e00 con 0x563511ac5290 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.662+0000 7fd323fff700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ac5290 0x563511a7d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.663+0000 7fd323fff700 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ac5290 0x563511a7d690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3ba6d7ada24852 server_cookie=83db893ef4b33c02 in_seq=0 out_seq=0 2023-10-06T05:35:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.663+0000 7fd3027fc700 10 client.12408 ms_handle_connect on v2:172.21.15.167:6834/3559384937 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.663+0000 7fd3027fc700 1 -- 172.21.15.33:0/1561280207 <== mds.0 v2:172.21.15.167:6834/3559384937 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563511979e00 con 0x563511ac5290 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.663+0000 7fd3027fc700 10 client.12408 handle_command_reply: tid=0 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 client.12408 shutdown 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 msgr2=0x563511aaea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 0x563511aaea60 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097c0 tx=0x7fd31c021000 comp rx=0 tx=0).stop 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 shutdown_connections 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ac5290 0x563511a7d690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511ad0bc0 0x563511aaea60 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511ac6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 --2- 172.21.15.33:0/1561280207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511adcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd3027fc700 0 client.12408 ms_handle_reset on v2:172.21.15.167:6834/3559384937 2023-10-06T05:35:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 >> 172.21.15.33:0/1561280207 conn(0x563511a88270 msgr2=0x563511a87190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 shutdown_connections 2023-10-06T05:35:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 1 -- 172.21.15.33:0/1561280207 wait complete. 2023-10-06T05:35:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.664+0000 7fd33905eb80 20 client.12408 trim_cache size 0 max 16384 2023-10-06T05:35:21.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 Processor -- start 2023-10-06T05:35:21.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 -- start start 2023-10-06T05:35:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511a87190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511a876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511aaca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563511859e30 con 0x563511a88270 2023-10-06T05:35:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563511a660d0 con 0x563511a86950 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.682+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x563511a86f70 con 0x563511aafe60 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511a87190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511a87190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39388/0 (socket says 172.21.15.33:39388) 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 -- 172.21.15.33:0/2247374434 learned_addr learned my addr 172.21.15.33:0/2247374434 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd328f9c700 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511aaca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd3237fe700 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511a876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 -- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 msgr2=0x563511aaca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511aaca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 -- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 msgr2=0x563511a876d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511a876d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 -- 172.21.15.33:0/2247374434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635119a5780 con 0x563511a88270 2023-10-06T05:35:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd3237fe700 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511a876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd323fff700 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511a87190 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fd3140827b0 tx=0x7fd31401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c426ada67e173c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.683+0000 7fd309ffb700 1 -- 172.21.15.33:0/2247374434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314054cd0 con 0x563511a88270 2023-10-06T05:35:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd309ffb700 1 -- 172.21.15.33:0/2247374434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314055190 con 0x563511a88270 2023-10-06T05:35:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd309ffb700 1 -- 172.21.15.33:0/2247374434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3140532d0 con 0x563511a88270 2023-10-06T05:35:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 -- 172.21.15.33:0/2247374434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 msgr2=0x563511a87190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511a87190 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fd3140827b0 tx=0x7fd31401d040 comp rx=0 tx=0).stop 2023-10-06T05:35:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 -- 172.21.15.33:0/2247374434 shutdown_connections 2023-10-06T05:35:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511aaca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511a87190 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 --2- 172.21.15.33:0/2247374434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511a876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 -- 172.21.15.33:0/2247374434 >> 172.21.15.33:0/2247374434 conn(0x563511a83140 msgr2=0x563511aaf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 -- 172.21.15.33:0/2247374434 shutdown_connections 2023-10-06T05:35:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.684+0000 7fd33905eb80 1 -- 172.21.15.33:0/2247374434 wait complete. 2023-10-06T05:35:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.685+0000 7fd33905eb80 1 -- start start 2023-10-06T05:35:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.685+0000 7fd33905eb80 1 Processor -- start 2023-10-06T05:35:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511b198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511ae6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511ac5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563511b34170 con 0x563511a88270 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563511b33410 con 0x563511a86950 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x563511b25e00 con 0x563511aafe60 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd33905eb80 10 client.0 _ll_register_callbacks cb 0x563511873f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd3237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511ae6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd3237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511ae6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39392/0 (socket says 172.21.15.33:39392) 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd3237fe700 1 -- 172.21.15.33:0/3831625791 learned_addr learned my addr 172.21.15.33:0/3831625791 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd323fff700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511b198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.686+0000 7fd328f9c700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511ac5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3237fe700 1 -- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 msgr2=0x563511ac5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3237fe700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511ac5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3237fe700 1 -- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 msgr2=0x563511b198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3237fe700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511b198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3237fe700 1 -- 172.21.15.33:0/3831625791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32414eb00 con 0x563511a88270 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3237fe700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511ae6420 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x563511a85980 tx=0x7fd31801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5202d35327cac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3027fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T05:35:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3027fc700 1 -- 172.21.15.33:0/3831625791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800ecd0 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563511a7b690 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3027fc700 1 -- 172.21.15.33:0/3831625791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd31800ee30 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.687+0000 7fd3027fc700 1 -- 172.21.15.33:0/3831625791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd318015a60 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 10 client.12414 fetch_fsmap learned FSMap version 219 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=219,monmap=2+}) v3 -- 0x56351198c470 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd3027fc700 1 -- 172.21.15.33:0/3831625791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563511a7b690 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd3027fc700 1 -- 172.21.15.33:0/3831625791 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 219) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd318015bc0 con 0x563511a88270 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 10 client.12414 fetch_fsmap finished waiting for FSMap version 219 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 10 client.12414 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 20 client.12414 populate_metadata read hostname 'smithi033' 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ad0bc0 0x563511ac6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 4 client.12414 mds_command: new command op to 12264 tid=0[{"prefix": "status"}] 2023-10-06T05:35:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 --> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563511979b20 con 0x563511ad0bc0 2023-10-06T05:35:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.688+0000 7fd323fff700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ad0bc0 0x563511ac6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd323fff700 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ad0bc0 0x563511ac6e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9488d683f555f20f server_cookie=777b643a2df6ab37 in_seq=0 out_seq=0 2023-10-06T05:35:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd3027fc700 10 client.12414 ms_handle_connect on v2:172.21.15.167:6834/3559384937 2023-10-06T05:35:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd3027fc700 1 -- 172.21.15.33:0/3831625791 <== mds.0 v2:172.21.15.167:6834/3559384937 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x563511979b20 con 0x563511ad0bc0 2023-10-06T05:35:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd3027fc700 10 client.12414 handle_command_reply: tid=0 2023-10-06T05:35:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 client.12414 shutdown 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 msgr2=0x563511ae6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511ae6420 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x563511a85980 tx=0x7fd31801a040 comp rx=0 tx=0).stop 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 shutdown_connections 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:6834/3559384937,v1:172.21.15.167:6836/3559384937] conn(0x563511ad0bc0 0x563511ac6e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd3027fc700 0 client.12414 ms_handle_reset on v2:172.21.15.167:6834/3559384937 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x563511aafe60 0x563511ac5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563511a86950 0x563511b198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 --2- 172.21.15.33:0/3831625791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563511a88270 0x563511ae6420 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.689+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 >> 172.21.15.33:0/3831625791 conn(0x563511a83140 msgr2=0x563511ab6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 shutdown_connections 2023-10-06T05:35:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd33905eb80 1 -- 172.21.15.33:0/3831625791 wait complete. 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd33905eb80 20 client.12414 trim_cache size 0 max 16384 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "abad2d63-0b71-4f3e-871a-ad3209a2fd6e", 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 12264, 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T05:35:21.701 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.5258774700000002, 2023-10-06T05:35:21.702 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 219, 2023-10-06T05:35:21.702 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 282, 2023-10-06T05:35:21.702 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 282, 2023-10-06T05:35:21.702 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.266788467 2023-10-06T05:35:21.702 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd310066f30 msgr2=0x7fd3100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd310066f30 0x7fd3100693e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd3180058d0 tx=0x7fd318009040 comp rx=0 tx=0).stop 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 msgr2=0x7fd32414e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 0x7fd32414e5c0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c790 tx=0x7fd314007e10 comp rx=0 tx=0).stop 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 shutdown_connections 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd310066f30 0x7fd3100693e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3241598a0 0x7fd3240c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32406c560 0x7fd32414e5c0 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 --2- 172.21.15.33:0/2471964583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3240ca7a0 0x7fd3240c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.690+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 >> 172.21.15.33:0/2471964583 conn(0x7fd32406e650 msgr2=0x7fd324150490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.691+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 shutdown_connections 2023-10-06T05:35:21.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.691+0000 7fd32979d700 1 -- 172.21.15.33:0/2471964583 wait complete. 2023-10-06T05:35:21.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:35:21.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:35:21.882 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:35:21.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.894+0000 7f37599ff700 1 Processor -- start 2023-10-06T05:35:21.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.894+0000 7f37599ff700 1 -- start start 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.895+0000 7f37599ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3754159630 0x7f37540cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.895+0000 7f37599ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37540cea10 0x7f37540b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.895+0000 7f37599ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37540cda00 con 0x7f37540cd590 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.895+0000 7f37599ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37540cee90 con 0x7f3754159630 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.895+0000 7f37599ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37540cfc80 con 0x7f37540cea10 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3754159630 0x7f37540cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3754159630 0x7f37540cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39406/0 (socket says 172.21.15.33:39406) 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 -- 172.21.15.33:0/3423347505 learned_addr learned my addr 172.21.15.33:0/3423347505 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f37537fe700 1 --2- 172.21.15.33:0/3423347505 >> v2:172.21.15.167:3300/0 conn(0x7f37540cea10 0x7f37540b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 -- 172.21.15.33:0/3423347505 >> v2:172.21.15.167:3300/0 conn(0x7f37540cea10 msgr2=0x7f37540b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 --2- 172.21.15.33:0/3423347505 >> v2:172.21.15.167:3300/0 conn(0x7f37540cea10 0x7f37540b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 -- 172.21.15.33:0/3423347505 >> v1:172.21.15.167:6789/0 conn(0x7f37540cd590 legacy=0x7f37540ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.896+0000 7f3752ffd700 1 -- 172.21.15.33:0/3423347505 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37540cf070 con 0x7f3754159630 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.897+0000 7f3752ffd700 1 --2- 172.21.15.33:0/3423347505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3754159630 0x7f37540cd050 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f37540b00a0 tx=0x7f3744009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c4f3d8ebb8e8cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.897+0000 7f3751ffb700 1 -- 172.21.15.33:0/3423347505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744009e70 con 0x7f3754159630 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.897+0000 7f3751ffb700 1 -- 172.21.15.33:0/3423347505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400b860 con 0x7f3754159630 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.897+0000 7f3751ffb700 1 -- 172.21.15.33:0/3423347505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400bc00 con 0x7f3754159630 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 -- 172.21.15.33:0/3423347505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3754159630 msgr2=0x7f37540cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 --2- 172.21.15.33:0/3423347505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3754159630 0x7f37540cd050 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f37540b00a0 tx=0x7f3744009250 comp rx=0 tx=0).stop 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 -- 172.21.15.33:0/3423347505 shutdown_connections 2023-10-06T05:35:21.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 --2- 172.21.15.33:0/3423347505 >> v2:172.21.15.167:3300/0 conn(0x7f37540cea10 0x7f37540b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 --2- 172.21.15.33:0/3423347505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3754159630 0x7f37540cd050 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 -- 172.21.15.33:0/3423347505 >> 172.21.15.33:0/3423347505 conn(0x7f375406cff0 msgr2=0x7f375406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 -- 172.21.15.33:0/3423347505 shutdown_connections 2023-10-06T05:35:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.898+0000 7f37599ff700 1 -- 172.21.15.33:0/3423347505 wait complete. 2023-10-06T05:35:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.899+0000 7f37599ff700 1 Processor -- start 2023-10-06T05:35:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 -- start start 2023-10-06T05:35:21.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37540cd590 0x7f375416d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 0x7f3754169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3754159630 0x7f375416ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375416b070 con 0x7f37540cea10 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375416b310 con 0x7f3754159630 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.900+0000 7f37599ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375416dbf0 con 0x7f37540cd590 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 0x7f3754169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 0x7f3754169fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39410/0 (socket says 172.21.15.33:39410) 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 -- 172.21.15.33:0/3030964802 learned_addr learned my addr 172.21.15.33:0/3030964802 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37537fe700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3754159630 0x7f375416ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f3752ffd700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37540cd590 0x7f375416d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 -- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37540cd590 msgr2=0x7f375416d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37540cd590 0x7f375416d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 -- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3754159630 msgr2=0x7f375416ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3754159630 0x7f375416ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 -- 172.21.15.33:0/3030964802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440094f0 con 0x7f37540cea10 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37537fe700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3754159630 0x7f375416ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f3752ffd700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37540cd590 0x7f375416d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.901+0000 7f37527fc700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 0x7f3754169fe0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f374800c590 tx=0x7f374800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.902+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748011590 con 0x7f37540cea10 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.902+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37480116f0 con 0x7f37540cea10 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.902+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748011ab0 con 0x7f37540cea10 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.902+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375416ded0 con 0x7f37540cea10 2023-10-06T05:35:21.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.902+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375416e490 con 0x7f37540cea10 2023-10-06T05:35:21.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.903+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754063ff0 con 0x7f37540cea10 2023-10-06T05:35:21.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.904+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f374801b3b0 con 0x7f37540cea10 2023-10-06T05:35:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.905+0000 7f373affd700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3740066f30 0x7f37400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.905+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3748083c90 con 0x7f37540cea10 2023-10-06T05:35:21.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.906+0000 7f3752ffd700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3740066f30 0x7f37400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.906+0000 7f3752ffd700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3740066f30 0x7f37400693e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f37540b00a0 tx=0x7f3744017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:21.909+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3748050d00 con 0x7f37540cea10 2023-10-06T05:35:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.160+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37541522c0 con 0x7f37540cea10 2023-10-06T05:35:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.161+0000 7f373affd700 1 -- 172.21.15.33:0/3030964802 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v219) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3748017020 con 0x7f37540cea10 2023-10-06T05:35:22.163 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3740066f30 msgr2=0x7f37400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3740066f30 0x7f37400693e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f37540b00a0 tx=0x7f3744017000 comp rx=0 tx=0).stop 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 msgr2=0x7f3754169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 0x7f3754169fe0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f374800c590 tx=0x7f374800c950 comp rx=0 tx=0).stop 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 shutdown_connections 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f3740066f30 0x7f37400693e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3754159630 0x7f375416ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37540cd590 0x7f375416d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 --2- 172.21.15.33:0/3030964802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37540cea10 0x7f3754169fe0 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 >> 172.21.15.33:0/3030964802 conn(0x7f375406cff0 msgr2=0x7f37540c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 shutdown_connections 2023-10-06T05:35:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:22.163+0000 7f37599ff700 1 -- 172.21.15.33:0/3030964802 wait complete. 2023-10-06T05:35:22.183 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:35:22.183 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:35:22.183 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T05:35:22.183 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:35:22.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:35:22.183 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:35:22.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:35:22.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:35:22.184 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:35:22.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.184 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T05:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41035sec preferred_lft 41035sec 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::f45c:8aff:fe75:a5da/64 scope link 2023-10-06T05:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ca:49:24:83:5d:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c849:24ff:fe83:5dff/64 scope link 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 12:5a:a6:5b:af:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::105a:a6ff:fe5b:af52/64 scope link 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout:9: brx.2@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:29:91:a0:fe:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c29:91ff:fea0:fede/64 scope link 2023-10-06T05:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:22.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.226 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:35:22.323 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T05:35:22.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.326 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T05:35:22.364 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T05:35:22.364 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:35:22.364 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:35:22.364 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:35:22.365 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:35:22.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.366 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T05:35:22.366 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T05:35:22.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:35:22.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:35:22.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:35:22.366 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T05:35:22.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.564 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:35:22.601 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:35:22.602 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:35:22.602 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:35:22.602 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:35:22.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T05:35:22.703 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:35:22.704 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:35:22.740 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:35:22.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:35:22.740 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:35:22.816 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:35:22.816 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:35:22.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:35:22.948 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:35:22.948 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:35:22.948 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:35:22.967 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:35:22.967 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:35:22.968 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:35:22.968 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:35:22.968 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:35:22.989 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:35:22.989 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:35:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2023-10-06T05:35:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:35:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:35:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:35:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:35:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:35:22.994 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:35:22.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:22.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:35:23.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:35:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.245+0000 7f123499d700 1 Processor -- start 2023-10-06T05:35:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.245+0000 7f123499d700 1 -- start start 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f123499d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12300c8b50 0x7f12300c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f123499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f123499d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123006ab60 con 0x7f1230155af0 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f123499d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123006b7e0 con 0x7f12300c8b50 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f123499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123006ae40 con 0x7f1230155ec0 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f122effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12300c8b50 0x7f12300c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f122effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12300c8b50 0x7f12300c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39422/0 (socket says 172.21.15.33:39422) 2023-10-06T05:35:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f122effd700 1 -- 172.21.15.33:0/3657486641 learned_addr learned my addr 172.21.15.33:0/3657486641 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.247+0000 7f122ffff700 1 --2- 172.21.15.33:0/3657486641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122effd700 1 -- 172.21.15.33:0/3657486641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 msgr2=0x7f12300c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122effd700 1 --2- 172.21.15.33:0/3657486641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122effd700 1 -- 172.21.15.33:0/3657486641 >> v1:172.21.15.167:6789/0 conn(0x7f1230155af0 legacy=0x7f123006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122effd700 1 -- 172.21.15.33:0/3657486641 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123006b120 con 0x7f12300c8b50 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122effd700 1 --2- 172.21.15.33:0/3657486641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12300c8b50 0x7f12300c7210 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f122400bc50 tx=0x7f122400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2765d97681efcaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122e7fc700 1 -- 172.21.15.33:0/3657486641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400be60 con 0x7f12300c8b50 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122e7fc700 1 -- 172.21.15.33:0/3657486641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400de00 con 0x7f12300c8b50 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.248+0000 7f122e7fc700 1 -- 172.21.15.33:0/3657486641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240196b0 con 0x7f12300c8b50 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 -- 172.21.15.33:0/3657486641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12300c8b50 msgr2=0x7f12300c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 --2- 172.21.15.33:0/3657486641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12300c8b50 0x7f12300c7210 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f122400bc50 tx=0x7f122400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 -- 172.21.15.33:0/3657486641 shutdown_connections 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 --2- 172.21.15.33:0/3657486641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 --2- 172.21.15.33:0/3657486641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12300c8b50 0x7f12300c7210 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 -- 172.21.15.33:0/3657486641 >> 172.21.15.33:0/3657486641 conn(0x7f123006d030 msgr2=0x7f123006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 -- 172.21.15.33:0/3657486641 shutdown_connections 2023-10-06T05:35:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.249+0000 7f123499d700 1 -- 172.21.15.33:0/3657486641 wait complete. 2023-10-06T05:35:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.250+0000 7f123499d700 1 Processor -- start 2023-10-06T05:35:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.250+0000 7f123499d700 1 -- start start 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.251+0000 7f123499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12300c8b50 0x7f12300c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.251+0000 7f123499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 0x7f12300c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.251+0000 7f123499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.251+0000 7f123499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300c39c0 con 0x7f1230155af0 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.251+0000 7f123499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300c5760 con 0x7f12300c8b50 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.251+0000 7f123499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12300c4e70 con 0x7f1230155ec0 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 0x7f12300c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 0x7f12300c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39424/0 (socket says 172.21.15.33:39424) 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 -- 172.21.15.33:0/993758789 learned_addr learned my addr 172.21.15.33:0/993758789 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122f7fe700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12300c8b50 0x7f12300c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122ffff700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 -- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 msgr2=0x7f12300c1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 -- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12300c8b50 msgr2=0x7f12300c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12300c8b50 0x7f12300c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 -- 172.21.15.33:0/993758789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122400b790 con 0x7f1230155af0 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122f7fe700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12300c8b50 0x7f12300c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.252+0000 7f122effd700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 0x7f12300c7290 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f12300635d0 tx=0x7f122401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.253+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224010750 con 0x7f1230155af0 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.253+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12240108b0 con 0x7f1230155af0 2023-10-06T05:35:23.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.253+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12300c77d0 con 0x7f1230155af0 2023-10-06T05:35:23.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.253+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224010c70 con 0x7f1230155af0 2023-10-06T05:35:23.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.253+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12300c5110 con 0x7f1230155af0 2023-10-06T05:35:23.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.254+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12300ca730 con 0x7f1230155af0 2023-10-06T05:35:23.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.255+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f122402c430 con 0x7f1230155af0 2023-10-06T05:35:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.256+0000 7f12177fe700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f121c066f30 0x7f121c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.256+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1224014070 con 0x7f1230155af0 2023-10-06T05:35:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.256+0000 7f122f7fe700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f121c066f30 0x7f121c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.257+0000 7f122f7fe700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f121c066f30 0x7f121c0693e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f1220007fd0 tx=0x7f1220009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:23.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.260+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12240539c0 con 0x7f1230155af0 2023-10-06T05:35:23.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.517+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f123004c380 con 0x7f1230155af0 2023-10-06T05:35:23.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.517+0000 7f12177fe700 1 -- 172.21.15.33:0/993758789 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v219) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1224056fe0 con 0x7f1230155af0 2023-10-06T05:35:23.519 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f121c066f30 msgr2=0x7f121c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f121c066f30 0x7f121c0693e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f1220007fd0 tx=0x7f1220009040 comp rx=0 tx=0).stop 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 msgr2=0x7f12300c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 0x7f12300c7290 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f12300635d0 tx=0x7f122401b000 comp rx=0 tx=0).stop 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 shutdown_connections 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f121c066f30 0x7f121c0693e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1230155ec0 0x7f12300c1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12300c8b50 0x7f12300c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 --2- 172.21.15.33:0/993758789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1230155af0 0x7f12300c7290 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 >> 172.21.15.33:0/993758789 conn(0x7f123006d030 msgr2=0x7f12300ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 shutdown_connections 2023-10-06T05:35:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:23.520+0000 7f123499d700 1 -- 172.21.15.33:0/993758789 wait complete. 2023-10-06T05:35:23.539 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:35:23.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:35:23.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T05:35:23.540 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:35:23.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:35:23.540 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:35:23.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:35:23.540 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:35:23.540 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:35:23.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:23.540 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T05:35:23.581 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:35:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:35:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:35:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T05:35:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.582 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:35:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 41037sec preferred_lft 41037sec 2023-10-06T05:35:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T05:35:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.584 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:35:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:35:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:35:23.585 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.585 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::302b:cfff:fea3:d2c/64 scope link 2023-10-06T05:35:23.586 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.586 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:35:23.586 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether f6:d2:93:53:18:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:35:23.586 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::f4d2:93ff:fe53:1830/64 scope link 2023-10-06T05:35:23.586 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.586 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:35:23.587 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 66:18:3b:26:ed:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:35:23.587 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::6418:3bff:fe26:ed39/64 scope link 2023-10-06T05:35:23.587 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:35:23.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:23.588 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:35:23.683 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T05:35:23.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:23.687 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T05:35:23.726 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T05:35:23.726 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:35:23.726 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:35:23.726 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:35:23.727 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:35:23.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:23.727 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T05:35:23.727 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T05:35:23.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:35:23.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:35:23.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:35:23.727 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T05:35:23.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:23.922 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:35:23.958 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:35:23.958 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:35:23.959 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:35:23.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:23.959 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T05:35:24.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:35:24.059 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:35:24.095 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:35:24.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:35:24.095 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:35:24.171 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:35:24.171 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:35:24.171 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:35:24.303 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:35:24.303 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:35:24.303 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:35:24.319 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:35:24.320 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:35:24.320 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:35:24.320 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:35:24.320 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.167:3300,172.21.15.167:3301" 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790" 2023-10-06T05:35:24.341 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2023-10-06T05:35:24.342 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=abad2d63-0b71-4f3e-871a-ad3209a2fd6e" will pass to kernel 2023-10-06T05:35:24.346 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:35:24.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:24.347 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:35:24.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_reconnect_timeout 2023-10-06T05:35:24.623 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_reconnect_timeout":"45.000000"} 2023-10-06T05:35:24.623 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_reconnect_timeout": "45.000000" } 2023-10-06T05:35:24.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get ms_max_backoff 2023-10-06T05:35:24.827 INFO:teuthology.orchestra.run.smithi033.stdout:{"ms_max_backoff":"15.000000"} 2023-10-06T05:35:24.827 DEBUG:tasks.cephfs.filesystem:_json_asok output { "ms_max_backoff": "15.000000" } 2023-10-06T05:35:24.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:35:24.827 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T05:35:24.899 INFO:teuthology.orchestra.run.smithi033.stdout:client9293 2023-10-06T05:35:24.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:25.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.086+0000 7fd05eb9d700 1 Processor -- start 2023-10-06T05:35:25.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.086+0000 7fd05eb9d700 1 -- start start 2023-10-06T05:35:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.087+0000 7fd05eb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0580cf3b0 0x7fd0581594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.087+0000 7fd05eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 0x7fd0581599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05eb9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0580cbd80 con 0x7fd0580ccfc0 2023-10-06T05:35:25.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05eb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0580af5f0 con 0x7fd0580cf3b0 2023-10-06T05:35:25.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580af8d0 con 0x7fd0580af0a0 2023-10-06T05:35:25.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05e39c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 0x7fd0581599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05d39a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0580cf3b0 0x7fd0581594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05db9b700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd0580ccfc0 0x7fd0580cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47356/0 (socket says 172.21.15.33:47356) 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05d39a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd0580cf3b0 0x7fd0581594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:59858/0 (socket says 172.21.15.33:59858) 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05e39c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 0x7fd0581599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:46682/0 (socket says 172.21.15.33:46682) 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.088+0000 7fd05db9b700 1 -- 172.21.15.33:0/1981547579 learned_addr learned my addr 172.21.15.33:0/1981547579 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05cb99700 1 -- 172.21.15.33:0/1981547579 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313644898 0 0) 0x7fd0580cbd80 con 0x7fd0580ccfc0 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05cb99700 1 -- 172.21.15.33:0/1981547579 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd044003570 con 0x7fd0580ccfc0 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05d39a700 1 -- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 msgr2=0x7fd0581599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05d39a700 1 --2- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 0x7fd0581599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05d39a700 1 -- 172.21.15.33:0/1981547579 >> v1:172.21.15.167:6789/0 conn(0x7fd0580ccfc0 legacy=0x7fd0580cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:25.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05d39a700 1 -- 172.21.15.33:0/1981547579 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580afbb0 con 0x7fd0580cf3b0 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05e39c700 1 --2- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 0x7fd0581599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05d39a700 1 --2- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580cf3b0 0x7fd0581594a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd048009910 tx=0x7fd0480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49803020d7afdce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05cb99700 1 -- 172.21.15.33:0/1981547579 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048011630 con 0x7fd0580cf3b0 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05cb99700 1 -- 172.21.15.33:0/1981547579 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd048011790 con 0x7fd0580cf3b0 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.089+0000 7fd05cb99700 1 -- 172.21.15.33:0/1981547579 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048011b30 con 0x7fd0580cf3b0 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 -- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580cf3b0 msgr2=0x7fd0581594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580cf3b0 0x7fd0581594a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd048009910 tx=0x7fd0480091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 -- 172.21.15.33:0/1981547579 shutdown_connections 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0580af0a0 0x7fd0581599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/1981547579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580cf3b0 0x7fd0581594a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 -- 172.21.15.33:0/1981547579 >> 172.21.15.33:0/1981547579 conn(0x7fd05806cff0 msgr2=0x7fd05806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:25.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 -- 172.21.15.33:0/1981547579 shutdown_connections 2023-10-06T05:35:25.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.090+0000 7fd05eb9d700 1 -- 172.21.15.33:0/1981547579 wait complete. 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.091+0000 7fd05eb9d700 1 Processor -- start 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.092+0000 7fd05eb9d700 1 -- start start 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.092+0000 7fd05eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 0x7fd0580b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.092+0000 7fd05eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580ccfc0 0x7fd0580b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05814b2e0 0x7fd0580c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580b2e30 con 0x7fd0580af0a0 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580c76e0 con 0x7fd0580ccfc0 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580b1320 con 0x7fd05814b2e0 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 0x7fd0580b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:25.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 0x7fd0580b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39428/0 (socket says 172.21.15.33:39428) 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05db9b700 1 -- 172.21.15.33:0/685111368 learned_addr learned my addr 172.21.15.33:0/685111368 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05d39a700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580ccfc0 0x7fd0580b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05e39c700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05814b2e0 0x7fd0580c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05d39a700 1 -- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05814b2e0 msgr2=0x7fd0580c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05d39a700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05814b2e0 0x7fd0580c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05d39a700 1 -- 172.21.15.33:0/685111368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 msgr2=0x7fd0580b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05d39a700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 0x7fd0580b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05d39a700 1 -- 172.21.15.33:0/685111368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd050009a20 con 0x7fd0580ccfc0 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.093+0000 7fd05db9b700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 0x7fd0580b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.094+0000 7fd05d39a700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580ccfc0 0x7fd0580b0de0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd05806d790 tx=0x7fd048017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.094+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048011b10 con 0x7fd0580ccfc0 2023-10-06T05:35:25.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.094+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04801be00 con 0x7fd0580ccfc0 2023-10-06T05:35:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.094+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd048009420 con 0x7fd0580ccfc0 2023-10-06T05:35:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.094+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048025640 con 0x7fd0580ccfc0 2023-10-06T05:35:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.094+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0580b1810 con 0x7fd0580ccfc0 2023-10-06T05:35:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.095+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0580c7850 con 0x7fd0580ccfc0 2023-10-06T05:35:25.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.096+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0480193b0 con 0x7fd0580ccfc0 2023-10-06T05:35:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.097+0000 7fd04dffb700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd03c066f30 0x7fd03c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.097+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(282..282 src has 1..282) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd048085f70 con 0x7fd0580ccfc0 2023-10-06T05:35:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.098+0000 7fd05db9b700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd03c066f30 0x7fd03c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.098+0000 7fd05db9b700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd03c066f30 0x7fd03c0693e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd054001ed0 tx=0x7fd054009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.102+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0580c7850 con 0x7fd0580ccfc0 2023-10-06T05:35:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.350+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd058157c40 con 0x7fd0580ccfc0 2023-10-06T05:35:25.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.351+0000 7fd04dffb700 1 -- 172.21.15.33:0/685111368 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fd058157c40 con 0x7fd0580ccfc0 2023-10-06T05:35:25.353 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:25.353 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:25.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd03c066f30 msgr2=0x7fd03c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd03c066f30 0x7fd03c0693e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd054001ed0 tx=0x7fd054009040 comp rx=0 tx=0).stop 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580ccfc0 msgr2=0x7fd0580b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580ccfc0 0x7fd0580b0de0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd05806d790 tx=0x7fd048017000 comp rx=0 tx=0).stop 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 shutdown_connections 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd03c066f30 0x7fd03c0693e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd05814b2e0 0x7fd0580c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0580af0a0 0x7fd0580b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 --2- 172.21.15.33:0/685111368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0580ccfc0 0x7fd0580b0de0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 >> 172.21.15.33:0/685111368 conn(0x7fd05806cff0 msgr2=0x7fd058159270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 shutdown_connections 2023-10-06T05:35:25.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:25.354+0000 7fd05eb9d700 1 -- 172.21.15.33:0/685111368 wait complete. 2023-10-06T05:35:25.357 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:25.373 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session evict 9293 2023-10-06T05:35:25.967 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-06T05:35:25.968 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:35:25.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:35:25.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:35:26.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:35:26.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:26.087 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:35:26.124 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:35:26.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:35:26.245 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:35:26.324 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:35:26.324 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:35:26.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:35:26.325 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:35:26.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:35:26.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:26.406 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:35:26.442 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.628+0000 7f24a619d700 1 Processor -- start 2023-10-06T05:35:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.628+0000 7f24a619d700 1 -- start start 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.629+0000 7f24a619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a00c8b50 0x7f24a00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.629+0000 7f24a619d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a006ab60 con 0x7f24a0155ec0 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.629+0000 7f24a619d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a006ae40 con 0x7f24a0155af0 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.629+0000 7f24a619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a006b7e0 con 0x7f24a00c8b50 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.629+0000 7f24a599c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f24a0155ec0 0x7f24a00c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40980/0 (socket says 172.21.15.33:40980) 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.629+0000 7f24a599c700 1 -- 172.21.15.33:0/1612195260 learned_addr learned my addr 172.21.15.33:0/1612195260 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:26.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f24a499a700 1 --2- 172.21.15.33:0/1612195260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a00c8b50 0x7f24a00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616569498 0 0) 0x7f24a006ab60 con 0x7f24a0155ec0 2023-10-06T05:35:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f248c003570 con 0x7f24a0155ec0 2023-10-06T05:35:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736018095 0 0) 0x7f24a006ae40 con 0x7f24a0155af0 2023-10-06T05:35:26.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a006ab60 con 0x7f24a0155af0 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f24a499a700 1 -- 172.21.15.33:0/1612195260 >> v1:172.21.15.167:6789/0 conn(0x7f24a0155af0 legacy=0x7f24a006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f24a499a700 1 -- 172.21.15.33:0/1612195260 >> v1:172.21.15.33:6789/0 conn(0x7f24a0155ec0 legacy=0x7f24a00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f24a499a700 1 -- 172.21.15.33:0/1612195260 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a006b120 con 0x7f24a00c8b50 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.630+0000 7f24a499a700 1 --2- 172.21.15.33:0/1612195260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a00c8b50 0x7f24a00c7210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2490009910 tx=0x7f24900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba64f28f8952a74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490011630 con 0x7f24a00c8b50 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2490011790 con 0x7f24a00c8b50 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f2497fff700 1 -- 172.21.15.33:0/1612195260 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490011b30 con 0x7f24a00c8b50 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 -- 172.21.15.33:0/1612195260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a00c8b50 msgr2=0x7f24a00c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 --2- 172.21.15.33:0/1612195260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a00c8b50 0x7f24a00c7210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2490009910 tx=0x7f24900091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 -- 172.21.15.33:0/1612195260 shutdown_connections 2023-10-06T05:35:26.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 --2- 172.21.15.33:0/1612195260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a00c8b50 0x7f24a00c7210 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 -- 172.21.15.33:0/1612195260 >> 172.21.15.33:0/1612195260 conn(0x7f24a006d030 msgr2=0x7f24a006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 -- 172.21.15.33:0/1612195260 shutdown_connections 2023-10-06T05:35:26.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.631+0000 7f24a619d700 1 -- 172.21.15.33:0/1612195260 wait complete. 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.633+0000 7f24a619d700 1 Processor -- start 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.633+0000 7f24a619d700 1 -- start start 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a619d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a00c8b50 0x7f24a0158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a619d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a0155af0 0x7f24a01589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a619d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 0x7f24a00aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a619d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00b12d0 con 0x7f24a0155ec0 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a619d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0159670 con 0x7f24a00c8b50 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a619d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0142f20 con 0x7f24a0155af0 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 0x7f24a00aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 0x7f24a00aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39434/0 (socket says 172.21.15.33:39434) 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 -- 172.21.15.33:0/3402404609 learned_addr learned my addr 172.21.15.33:0/3402404609 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a519b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a00c8b50 0x7f24a0158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a499a700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a0155af0 0x7f24a01589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 -- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a0155af0 msgr2=0x7f24a01589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a0155af0 0x7f24a01589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 -- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a00c8b50 msgr2=0x7f24a0158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a00c8b50 0x7f24a0158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.634+0000 7f24a599c700 1 -- 172.21.15.33:0/3402404609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490009420 con 0x7f24a0155ec0 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.635+0000 7f24a599c700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 0x7f24a00aff80 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f249800cd00 tx=0x7f2498001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:26.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.635+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800ce00 con 0x7f24a0155ec0 2023-10-06T05:35:26.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.635+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800aac0 con 0x7f24a0155ec0 2023-10-06T05:35:26.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.635+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a01590b0 con 0x7f24a0155ec0 2023-10-06T05:35:26.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.635+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800ae80 con 0x7f24a0155ec0 2023-10-06T05:35:26.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.635+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a0142bb0 con 0x7f24a0155ec0 2023-10-06T05:35:26.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.636+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a0143090 con 0x7f24a0155ec0 2023-10-06T05:35:26.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.637+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2498007300 con 0x7f24a0155ec0 2023-10-06T05:35:26.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.638+0000 7f24957fa700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2484066f30 0x7f24840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:26.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.638+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24980849f0 con 0x7f24a0155ec0 2023-10-06T05:35:26.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.638+0000 7f24a519b700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2484066f30 0x7f24840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:26.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.639+0000 7f24a519b700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2484066f30 0x7f24840693e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f249c007fd0 tx=0x7f249c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:26.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.642+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2498051ab0 con 0x7f24a0155ec0 2023-10-06T05:35:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.889+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24a004c380 con 0x7f24a0155ec0 2023-10-06T05:35:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.889+0000 7f24957fa700 1 -- 172.21.15.33:0/3402404609 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f24980550d0 con 0x7f24a0155ec0 2023-10-06T05:35:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:26.893 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.891+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2484066f30 msgr2=0x7f24840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.891+0000 7f24a619d700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2484066f30 0x7f24840693e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f249c007fd0 tx=0x7f249c007d50 comp rx=0 tx=0).stop 2023-10-06T05:35:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.891+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 msgr2=0x7f24a00aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.891+0000 7f24a619d700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 0x7f24a00aff80 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f249800cd00 tx=0x7f2498001ac0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 shutdown_connections 2023-10-06T05:35:26.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2484066f30 0x7f24840693e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24a0155ec0 0x7f24a00aff80 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a00c8b50 0x7f24a0158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 --2- 172.21.15.33:0/3402404609 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a0155af0 0x7f24a01589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 >> 172.21.15.33:0/3402404609 conn(0x7f24a006d030 msgr2=0x7f24a00cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 shutdown_connections 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:26.892+0000 7f24a619d700 1 -- 172.21.15.33:0/3402404609 wait complete. 2023-10-06T05:35:26.895 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:26.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.097+0000 7f1596f9d700 1 Processor -- start 2023-10-06T05:35:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.097+0000 7f1596f9d700 1 -- start start 2023-10-06T05:35:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f1596f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1590147390 0x7f15900cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f1596f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15900c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f1596f9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900c72d0 con 0x7f15900c6e70 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f1596f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15900cb5f0 con 0x7f1590147390 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f1596f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900ca9e0 con 0x7f1590147760 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f159679c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f15900c6e70 0x7f15900c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40996/0 (socket says 172.21.15.33:40996) 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f159679c700 1 -- 172.21.15.33:0/2660984789 learned_addr learned my addr 172.21.15.33:0/2660984789 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f159579a700 1 --2- 172.21.15.33:0/2660984789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15900c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.099+0000 7f1595f9b700 1 --2- 172.21.15.33:0/2660984789 >> v2:172.21.15.167:3300/0 conn(0x7f1590147390 0x7f15900cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f1594f99700 1 -- 172.21.15.33:0/2660984789 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885130406 0 0) 0x7f1588001490 con 0x7f15900c6e70 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f1594f99700 1 -- 172.21.15.33:0/2660984789 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f157c003570 con 0x7f15900c6e70 2023-10-06T05:35:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f159579a700 1 -- 172.21.15.33:0/2660984789 >> v2:172.21.15.167:3300/0 conn(0x7f1590147390 msgr2=0x7f15900cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f159579a700 1 --2- 172.21.15.33:0/2660984789 >> v2:172.21.15.167:3300/0 conn(0x7f1590147390 0x7f15900cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f159579a700 1 -- 172.21.15.33:0/2660984789 >> v1:172.21.15.33:6789/0 conn(0x7f15900c6e70 legacy=0x7f15900c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f159579a700 1 -- 172.21.15.33:0/2660984789 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900cacc0 con 0x7f1590147760 2023-10-06T05:35:27.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f159579a700 1 --2- 172.21.15.33:0/2660984789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15900c6930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f15800098e0 tx=0x7f15800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad4478ca1fca297 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f1594f99700 1 -- 172.21.15.33:0/2660984789 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580011630 con 0x7f1590147760 2023-10-06T05:35:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f1594f99700 1 -- 172.21.15.33:0/2660984789 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580011790 con 0x7f1590147760 2023-10-06T05:35:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.100+0000 7f1594f99700 1 -- 172.21.15.33:0/2660984789 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580011b30 con 0x7f1590147760 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 -- 172.21.15.33:0/2660984789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 msgr2=0x7f15900c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 --2- 172.21.15.33:0/2660984789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15900c6930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f15800098e0 tx=0x7f15800091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 -- 172.21.15.33:0/2660984789 shutdown_connections 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 --2- 172.21.15.33:0/2660984789 >> v2:172.21.15.167:3300/0 conn(0x7f1590147390 0x7f15900cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 --2- 172.21.15.33:0/2660984789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15900c6930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 -- 172.21.15.33:0/2660984789 >> 172.21.15.33:0/2660984789 conn(0x7f159006d030 msgr2=0x7f159006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 -- 172.21.15.33:0/2660984789 shutdown_connections 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.101+0000 7f1596f9d700 1 -- 172.21.15.33:0/2660984789 wait complete. 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.103+0000 7f1596f9d700 1 Processor -- start 2023-10-06T05:35:27.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.103+0000 7f1596f9d700 1 -- start start 2023-10-06T05:35:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1596f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15900c6e70 0x7f159016de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1596f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 0x7f159015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1596f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15901450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1596f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900c6060 con 0x7f1590147390 2023-10-06T05:35:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1596f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159015ec30 con 0x7f15900c6e70 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1596f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159015ef10 con 0x7f1590147760 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f159579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 0x7f159015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f159579a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 0x7f159015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39436/0 (socket says 172.21.15.33:39436) 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f159579a700 1 -- 172.21.15.33:0/3252036886 learned_addr learned my addr 172.21.15.33:0/3252036886 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f1595f9b700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15900c6e70 0x7f159016de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f159679c700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15901450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.104+0000 7f159579a700 1 -- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 msgr2=0x7f15901450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f159579a700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15901450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f159579a700 1 -- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15900c6e70 msgr2=0x7f159016de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f159579a700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15900c6e70 0x7f159016de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f159579a700 1 -- 172.21.15.33:0/3252036886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1580009420 con 0x7f1590147390 2023-10-06T05:35:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f159579a700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 0x7f159015e400 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f1590068570 tx=0x7f1580017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580011900 con 0x7f1590147390 2023-10-06T05:35:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580011a60 con 0x7f1590147390 2023-10-06T05:35:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580025640 con 0x7f1590147390 2023-10-06T05:35:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159015f1f0 con 0x7f1590147390 2023-10-06T05:35:27.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.105+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159015f7b0 con 0x7f1590147390 2023-10-06T05:35:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.106+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15900cd010 con 0x7f1590147390 2023-10-06T05:35:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.107+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1580025a30 con 0x7f1590147390 2023-10-06T05:35:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.108+0000 7f1585ffb700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1574066f30 0x7f15740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.108+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1580085db0 con 0x7f1590147390 2023-10-06T05:35:27.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.109+0000 7f1595f9b700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1574066f30 0x7f15740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.109+0000 7f1595f9b700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1574066f30 0x7f15740693e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1590146170 tx=0x7f158c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.112+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f158008a070 con 0x7f1590147390 2023-10-06T05:35:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.314+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f159015f9d0 con 0x7f1590147390 2023-10-06T05:35:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.315+0000 7f1585ffb700 1 -- 172.21.15.33:0/3252036886 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v283) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f1580052df0 con 0x7f1590147390 2023-10-06T05:35:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":283,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:25.942126+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:35:12.826592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":42,"pool_name":"cephfs_data","create_time":"2023-10-06T05:35:13.627133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"282","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2784413487":"2023-10-06T06:35:25.570337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1574066f30 msgr2=0x7f15740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1574066f30 0x7f15740693e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1590146170 tx=0x7f158c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 msgr2=0x7f159015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 0x7f159015e400 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f1590068570 tx=0x7f1580017000 comp rx=0 tx=0).stop 2023-10-06T05:35:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 shutdown_connections 2023-10-06T05:35:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1574066f30 0x7f15740693e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1590147760 0x7f15901450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15900c6e70 0x7f159016de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 --2- 172.21.15.33:0/3252036886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1590147390 0x7f159015e400 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 >> 172.21.15.33:0/3252036886 conn(0x7f159006d030 msgr2=0x7f1590146490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 shutdown_connections 2023-10-06T05:35:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.317+0000 7f1596f9d700 1 -- 172.21.15.33:0/3252036886 wait complete. 2023-10-06T05:35:27.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:11.879286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:18.463299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 41, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:35:12.826592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '281', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 42, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:35:13.627133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '282', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:35:27.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:35:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.526+0000 7f0343fff700 1 Processor -- start 2023-10-06T05:35:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.526+0000 7f0343fff700 1 -- start start 2023-10-06T05:35:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.527+0000 7f0343fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0344154f20 0x7f03440ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.527+0000 7f0343fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f03441552f0 0x7f03440cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.527+0000 7f0343fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03441586a0 con 0x7f03440b11e0 2023-10-06T05:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.527+0000 7f0343fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034406bdf0 con 0x7f03441552f0 2023-10-06T05:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.527+0000 7f0343fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0344157ab0 con 0x7f0344154f20 2023-10-06T05:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f03437fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f03440b11e0 0x7f034415b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41012/0 (socket says 172.21.15.33:41012) 2023-10-06T05:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f03437fe700 1 -- 172.21.15.33:0/4150060143 learned_addr learned my addr 172.21.15.33:0/4150060143 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0342ffd700 1 --2- 172.21.15.33:0/4150060143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0344154f20 0x7f03440ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f03427fc700 1 --2- 172.21.15.33:0/4150060143 >> v2:172.21.15.167:3300/0 conn(0x7f03441552f0 0x7f03440cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0341ffb700 1 -- 172.21.15.33:0/4150060143 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237648951 0 0) 0x7f03441586a0 con 0x7f03440b11e0 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0341ffb700 1 -- 172.21.15.33:0/4150060143 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f032c003570 con 0x7f03440b11e0 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0342ffd700 1 -- 172.21.15.33:0/4150060143 >> v2:172.21.15.167:3300/0 conn(0x7f03441552f0 msgr2=0x7f03440cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0342ffd700 1 --2- 172.21.15.33:0/4150060143 >> v2:172.21.15.167:3300/0 conn(0x7f03441552f0 0x7f03440cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0342ffd700 1 -- 172.21.15.33:0/4150060143 >> v1:172.21.15.33:6789/0 conn(0x7f03440b11e0 legacy=0x7f034415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.528+0000 7f0342ffd700 1 -- 172.21.15.33:0/4150060143 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0344157d70 con 0x7f0344154f20 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0342ffd700 1 --2- 172.21.15.33:0/4150060143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0344154f20 0x7f03440ae720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f03340098e0 tx=0x7f03340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f97cd1faff4aef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0341ffb700 1 -- 172.21.15.33:0/4150060143 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334011630 con 0x7f0344154f20 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0341ffb700 1 -- 172.21.15.33:0/4150060143 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334011790 con 0x7f0344154f20 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0341ffb700 1 -- 172.21.15.33:0/4150060143 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334011b30 con 0x7f0344154f20 2023-10-06T05:35:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0343fff700 1 -- 172.21.15.33:0/4150060143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0344154f20 msgr2=0x7f03440ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0343fff700 1 --2- 172.21.15.33:0/4150060143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0344154f20 0x7f03440ae720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f03340098e0 tx=0x7f03340091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0343fff700 1 -- 172.21.15.33:0/4150060143 shutdown_connections 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0343fff700 1 --2- 172.21.15.33:0/4150060143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0344154f20 0x7f03440ae720 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0343fff700 1 --2- 172.21.15.33:0/4150060143 >> v2:172.21.15.167:3300/0 conn(0x7f03441552f0 0x7f03440cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.529+0000 7f0343fff700 1 -- 172.21.15.33:0/4150060143 >> 172.21.15.33:0/4150060143 conn(0x7f034406d030 msgr2=0x7f03440cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.530+0000 7f0343fff700 1 -- 172.21.15.33:0/4150060143 shutdown_connections 2023-10-06T05:35:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.530+0000 7f0343fff700 1 -- 172.21.15.33:0/4150060143 wait complete. 2023-10-06T05:35:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.531+0000 7f0343fff700 1 Processor -- start 2023-10-06T05:35:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.531+0000 7f0343fff700 1 -- start start 2023-10-06T05:35:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0343fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03440b11e0 0x7f034413f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0343fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 0x7f03440aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0343fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03440aaf50 0x7f03440af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0343fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0344169f10 con 0x7f0344154f20 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0343fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440ab3d0 con 0x7f03440b11e0 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0343fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034415af50 con 0x7f03440aaf50 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f03427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 0x7f03440aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f03427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 0x7f03440aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39446/0 (socket says 172.21.15.33:39446) 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f03427fc700 1 -- 172.21.15.33:0/3130729186 learned_addr learned my addr 172.21.15.33:0/3130729186 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f0342ffd700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03440b11e0 0x7f034413f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.532+0000 7f03437fe700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03440aaf50 0x7f03440af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03427fc700 1 -- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03440aaf50 msgr2=0x7f03440af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03427fc700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03440aaf50 0x7f03440af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03427fc700 1 -- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03440b11e0 msgr2=0x7f034413f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03427fc700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03440b11e0 0x7f034413f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03427fc700 1 -- 172.21.15.33:0/3130729186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009420 con 0x7f0344154f20 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03437fe700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03440aaf50 0x7f03440af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f03427fc700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 0x7f03440aaa10 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f033800ecb0 tx=0x7f03380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800c920 con 0x7f0344154f20 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033800ca80 con 0x7f0344154f20 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034415b7f0 con 0x7f0344154f20 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800ce20 con 0x7f0344154f20 2023-10-06T05:35:27.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.533+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034415b1a0 con 0x7f0344154f20 2023-10-06T05:35:27.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.534+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034415b980 con 0x7f0344154f20 2023-10-06T05:35:27.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.535+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0338013680 con 0x7f0344154f20 2023-10-06T05:35:27.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.536+0000 7f032affd700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.536+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0338084a60 con 0x7f0344154f20 2023-10-06T05:35:27.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.537+0000 7f0342ffd700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.537+0000 7f0342ffd700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f03340091e0 tx=0x7f0334017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.540+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0338051aa0 con 0x7f0344154f20 2023-10-06T05:35:27.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.741+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f034404c380 con 0x7f0344154f20 2023-10-06T05:35:27.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.741+0000 7f032affd700 1 -- 172.21.15.33:0/3130729186 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v283) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0338084020 con 0x7f0344154f20 2023-10-06T05:35:27.743 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T05:35:27.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 msgr2=0x7f03300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f03340091e0 tx=0x7f0334017000 comp rx=0 tx=0).stop 2023-10-06T05:35:27.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 msgr2=0x7f03440aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 0x7f03440aaa10 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f033800ecb0 tx=0x7f03380091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 shutdown_connections 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f0330066f30 0x7f03300693e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03440aaf50 0x7f03440af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03440b11e0 0x7f034413f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 --2- 172.21.15.33:0/3130729186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0344154f20 0x7f03440aaa10 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 >> 172.21.15.33:0/3130729186 conn(0x7f034406d030 msgr2=0x7f03440ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 shutdown_connections 2023-10-06T05:35:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.743+0000 7f0343fff700 1 -- 172.21.15.33:0/3130729186 wait complete. 2023-10-06T05:35:27.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:35:27.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.950+0000 7f486679d700 1 Processor -- start 2023-10-06T05:35:27.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.950+0000 7f486679d700 1 -- start start 2023-10-06T05:35:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.951+0000 7f486679d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48600cd5e0 0x7f48600af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.951+0000 7f486679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486015a650 0x7f486015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.951+0000 7f486679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486015d660 con 0x7f48600af540 2023-10-06T05:35:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.951+0000 7f486679d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600ce890 con 0x7f48600cd5e0 2023-10-06T05:35:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.951+0000 7f486679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486015aab0 con 0x7f486015a650 2023-10-06T05:35:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f486579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f48600af540 0x7f48600cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41016/0 (socket says 172.21.15.33:41016) 2023-10-06T05:35:27.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f486579b700 1 -- 172.21.15.33:0/136597911 learned_addr learned my addr 172.21.15.33:0/136597911 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4865f9c700 1 --2- 172.21.15.33:0/136597911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486015a650 0x7f486015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4864f9a700 1 --2- 172.21.15.33:0/136597911 >> v2:172.21.15.167:3300/0 conn(0x7f48600cd5e0 0x7f48600af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4857fff700 1 -- 172.21.15.33:0/136597911 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139750581 0 0) 0x7f486015d660 con 0x7f48600af540 2023-10-06T05:35:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4857fff700 1 -- 172.21.15.33:0/136597911 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f484c003570 con 0x7f48600af540 2023-10-06T05:35:27.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4865f9c700 1 -- 172.21.15.33:0/136597911 >> v2:172.21.15.167:3300/0 conn(0x7f48600cd5e0 msgr2=0x7f48600af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4865f9c700 1 --2- 172.21.15.33:0/136597911 >> v2:172.21.15.167:3300/0 conn(0x7f48600cd5e0 0x7f48600af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4865f9c700 1 -- 172.21.15.33:0/136597911 >> v1:172.21.15.33:6789/0 conn(0x7f48600af540 legacy=0x7f48600cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.952+0000 7f4865f9c700 1 -- 172.21.15.33:0/136597911 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486015d890 con 0x7f486015a650 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.953+0000 7f4865f9c700 1 --2- 172.21.15.33:0/136597911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486015a650 0x7f486015d120 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4858009e60 tx=0x7f4858009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5b8e39b841971d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.953+0000 7f4857fff700 1 -- 172.21.15.33:0/136597911 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800e070 con 0x7f486015a650 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.953+0000 7f4857fff700 1 -- 172.21.15.33:0/136597911 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485800bab0 con 0x7f486015a650 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.953+0000 7f4857fff700 1 -- 172.21.15.33:0/136597911 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800be70 con 0x7f486015a650 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.953+0000 7f486679d700 1 -- 172.21.15.33:0/136597911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486015a650 msgr2=0x7f486015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.953+0000 7f486679d700 1 --2- 172.21.15.33:0/136597911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486015a650 0x7f486015d120 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4858009e60 tx=0x7f4858009780 comp rx=0 tx=0).stop 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.954+0000 7f486679d700 1 -- 172.21.15.33:0/136597911 shutdown_connections 2023-10-06T05:35:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.954+0000 7f486679d700 1 --2- 172.21.15.33:0/136597911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486015a650 0x7f486015d120 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.954+0000 7f486679d700 1 --2- 172.21.15.33:0/136597911 >> v2:172.21.15.167:3300/0 conn(0x7f48600cd5e0 0x7f48600af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.954+0000 7f486679d700 1 -- 172.21.15.33:0/136597911 >> 172.21.15.33:0/136597911 conn(0x7f486006d9b0 msgr2=0x7f486006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.954+0000 7f486679d700 1 -- 172.21.15.33:0/136597911 shutdown_connections 2023-10-06T05:35:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.954+0000 7f486679d700 1 -- 172.21.15.33:0/136597911 wait complete. 2023-10-06T05:35:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.955+0000 7f486679d700 1 Processor -- start 2023-10-06T05:35:27.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.955+0000 7f486679d700 1 -- start start 2023-10-06T05:35:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.956+0000 7f486679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48600af540 0x7f486014c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.956+0000 7f486679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48600cd5e0 0x7f48600b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.956+0000 7f486679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 0x7f48600c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.956+0000 7f486679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600b41c0 con 0x7f486015a650 2023-10-06T05:35:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.956+0000 7f486679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600b43f0 con 0x7f48600af540 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.956+0000 7f486679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48600b1ae0 con 0x7f48600cd5e0 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 0x7f48600c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 0x7f48600c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39454/0 (socket says 172.21.15.33:39454) 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 -- 172.21.15.33:0/3762124158 learned_addr learned my addr 172.21.15.33:0/3762124158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4864f9a700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48600cd5e0 0x7f48600b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f486579b700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48600af540 0x7f486014c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 -- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48600cd5e0 msgr2=0x7f48600b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48600cd5e0 0x7f48600b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 -- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48600af540 msgr2=0x7f486014c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48600af540 0x7f486014c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:27.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 -- 172.21.15.33:0/3762124158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858009a20 con 0x7f486015a650 2023-10-06T05:35:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.957+0000 7f4865f9c700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 0x7f48600c8250 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f485800b610 tx=0x7f4858019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.958+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858012d70 con 0x7f486015a650 2023-10-06T05:35:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.958+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4858027470 con 0x7f486015a650 2023-10-06T05:35:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.958+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48600b11b0 con 0x7f486015a650 2023-10-06T05:35:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.958+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858027830 con 0x7f486015a650 2023-10-06T05:35:27.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.958+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600b1770 con 0x7f486015a650 2023-10-06T05:35:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.959+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600d1350 con 0x7f486015a650 2023-10-06T05:35:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.960+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4858024070 con 0x7f486015a650 2023-10-06T05:35:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.961+0000 7f48557fa700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4844066f30 0x7f48440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:27.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.961+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4858086d10 con 0x7f486015a650 2023-10-06T05:35:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.961+0000 7f486579b700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4844066f30 0x7f48440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:27.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.962+0000 7f486579b700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4844066f30 0x7f48440693e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f485c007fd0 tx=0x7f485c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:27.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:27.965+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f485808c020 con 0x7f486015a650 2023-10-06T05:35:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.166+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48600b1c70 con 0x7f486015a650 2023-10-06T05:35:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.166+0000 7f48557fa700 1 -- 172.21.15.33:0/3762124158 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v283) v1 ==== 92+0+10 (secure 0 0 0) 0x7f485802c0a0 con 0x7f486015a650 2023-10-06T05:35:28.168 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4844066f30 msgr2=0x7f48440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4844066f30 0x7f48440693e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f485c007fd0 tx=0x7f485c009040 comp rx=0 tx=0).stop 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 msgr2=0x7f48600c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 0x7f48600c8250 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f485800b610 tx=0x7f4858019000 comp rx=0 tx=0).stop 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 shutdown_connections 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4844066f30 0x7f48440693e0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486015a650 0x7f48600c8250 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48600af540 0x7f486014c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 --2- 172.21.15.33:0/3762124158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48600cd5e0 0x7f48600b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 >> 172.21.15.33:0/3762124158 conn(0x7f486006d9b0 msgr2=0x7f48600c56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 shutdown_connections 2023-10-06T05:35:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.169+0000 7f486679d700 1 -- 172.21.15.33:0/3762124158 wait complete. 2023-10-06T05:35:28.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:35:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.376+0000 7f731639d700 1 Processor -- start 2023-10-06T05:35:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.376+0000 7f731639d700 1 -- start start 2023-10-06T05:35:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f731639d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7310154f20 0x7f73100ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f731639d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73100b18f0 0x7f73100cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f731639d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73101586a0 con 0x7f73101552f0 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f731639d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731006bdf0 con 0x7f73100b18f0 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f731639d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7310157ab0 con 0x7f7310154f20 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7315b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73100b18f0 0x7f73100cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7314b9a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73101552f0 0x7f73100b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41018/0 (socket says 172.21.15.33:41018) 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7314b9a700 1 -- 172.21.15.33:0/3850085439 learned_addr learned my addr 172.21.15.33:0/3850085439 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:28.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f731539b700 1 --2- 172.21.15.33:0/3850085439 >> v2:172.21.15.167:3300/0 conn(0x7f7310154f20 0x7f73100ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7315b9c700 1 -- 172.21.15.33:0/3850085439 >> v2:172.21.15.167:3300/0 conn(0x7f7310154f20 msgr2=0x7f73100ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7315b9c700 1 --2- 172.21.15.33:0/3850085439 >> v2:172.21.15.167:3300/0 conn(0x7f7310154f20 0x7f73100ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7315b9c700 1 -- 172.21.15.33:0/3850085439 >> v1:172.21.15.33:6789/0 conn(0x7f73101552f0 legacy=0x7f73100b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7315b9c700 1 -- 172.21.15.33:0/3850085439 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310157d70 con 0x7f73100b18f0 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.378+0000 7f7307fff700 1 -- 172.21.15.33:0/3850085439 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444189381 0 0) 0x7f73101586a0 con 0x7f73101552f0 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.379+0000 7f7315b9c700 1 --2- 172.21.15.33:0/3850085439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73100cc7e0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f73100713f0 tx=0x7f7308009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e107f6881fc3b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.379+0000 7f7307fff700 1 -- 172.21.15.33:0/3850085439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800e070 con 0x7f73100b18f0 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.379+0000 7f7307fff700 1 -- 172.21.15.33:0/3850085439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730800bb60 con 0x7f73100b18f0 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.379+0000 7f7307fff700 1 -- 172.21.15.33:0/3850085439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308012430 con 0x7f73100b18f0 2023-10-06T05:35:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.379+0000 7f731639d700 1 -- 172.21.15.33:0/3850085439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 msgr2=0x7f73100cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.379+0000 7f731639d700 1 --2- 172.21.15.33:0/3850085439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73100cc7e0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f73100713f0 tx=0x7f7308009780 comp rx=0 tx=0).stop 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.380+0000 7f731639d700 1 -- 172.21.15.33:0/3850085439 shutdown_connections 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.380+0000 7f731639d700 1 --2- 172.21.15.33:0/3850085439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73100cc7e0 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.380+0000 7f731639d700 1 --2- 172.21.15.33:0/3850085439 >> v2:172.21.15.167:3300/0 conn(0x7f7310154f20 0x7f73100ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.380+0000 7f731639d700 1 -- 172.21.15.33:0/3850085439 >> 172.21.15.33:0/3850085439 conn(0x7f731006d030 msgr2=0x7f73100cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.380+0000 7f731639d700 1 -- 172.21.15.33:0/3850085439 shutdown_connections 2023-10-06T05:35:28.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.380+0000 7f731639d700 1 -- 172.21.15.33:0/3850085439 wait complete. 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.381+0000 7f731639d700 1 Processor -- start 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.381+0000 7f731639d700 1 -- start start 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73101530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7310154f20 0x7f73100a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731015cae0 0x7f73100a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100a6790 con 0x7f73100b18f0 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100a6ce0 con 0x7f7310154f20 2023-10-06T05:35:28.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100a6fc0 con 0x7f731015cae0 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73101530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731539b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73101530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39464/0 (socket says 172.21.15.33:39464) 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.382+0000 7f731539b700 1 -- 172.21.15.33:0/2748609674 learned_addr learned my addr 172.21.15.33:0/2748609674 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f7315b9c700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731015cae0 0x7f73100a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f7314b9a700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7310154f20 0x7f73100a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731539b700 1 -- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731015cae0 msgr2=0x7f73100a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731539b700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731015cae0 0x7f73100a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731539b700 1 -- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7310154f20 msgr2=0x7f73100a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731539b700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7310154f20 0x7f73100a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731539b700 1 -- 172.21.15.33:0/2748609674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308009a20 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f7314b9a700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7310154f20 0x7f73100a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731539b700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73101530a0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f730c00e950 tx=0x7f730c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00c850 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c00c9b0 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00cdb0 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.383+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73100a72a0 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.384+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73100a6380 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.384+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7310067db0 con 0x7f73100b18f0 2023-10-06T05:35:28.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.385+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f730c013680 con 0x7f73100b18f0 2023-10-06T05:35:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.386+0000 7f73057fa700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72f4066f30 0x7f72f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.386+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f730c0849f0 con 0x7f73100b18f0 2023-10-06T05:35:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.387+0000 7f7314b9a700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72f4066f30 0x7f72f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.387+0000 7f7314b9a700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72f4066f30 0x7f72f40693e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f73000011e0 tx=0x7f7300009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:28.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.390+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f730c051a30 con 0x7f73100b18f0 2023-10-06T05:35:28.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.593+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f731004c380 con 0x7f73100b18f0 2023-10-06T05:35:28.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.593+0000 7f73057fa700 1 -- 172.21.15.33:0/2748609674 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v283) v1 ==== 104+0+11 (secure 0 0 0) 0x7f730c055050 con 0x7f73100b18f0 2023-10-06T05:35:28.595 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72f4066f30 msgr2=0x7f72f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72f4066f30 0x7f72f40693e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f73000011e0 tx=0x7f7300009040 comp rx=0 tx=0).stop 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 msgr2=0x7f73101530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73101530a0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f730c00e950 tx=0x7f730c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 shutdown_connections 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f72f4066f30 0x7f72f40693e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731015cae0 0x7f73100a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73100b18f0 0x7f73101530a0 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 --2- 172.21.15.33:0/2748609674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7310154f20 0x7f73100a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 >> 172.21.15.33:0/2748609674 conn(0x7f731006d030 msgr2=0x7f7310153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:28.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 shutdown_connections 2023-10-06T05:35:28.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.596+0000 7f731639d700 1 -- 172.21.15.33:0/2748609674 wait complete. 2023-10-06T05:35:28.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:35:28.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.802+0000 7f6e2bfff700 1 Processor -- start 2023-10-06T05:35:28.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.803+0000 7f6e2bfff700 1 -- start start 2023-10-06T05:35:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2bfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0b06b0 con 0x7f6e2c0afb20 2023-10-06T05:35:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2bfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0c99e0 con 0x7f6e2c0c9490 2023-10-06T05:35:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c9cc0 con 0x7f6e2c155a50 2023-10-06T05:35:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6e2c0afb20 0x7f6e2c0caa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41030/0 (socket says 172.21.15.33:41030) 2023-10-06T05:35:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2b7fe700 1 -- 172.21.15.33:0/323462280 learned_addr learned my addr 172.21.15.33:0/323462280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:28.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.804+0000 7f6e2affd700 1 --2- 172.21.15.33:0/323462280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248207141 0 0) 0x7f6e2c0b06b0 con 0x7f6e2c0afb20 2023-10-06T05:35:28.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e14004680 con 0x7f6e2c0afb20 2023-10-06T05:35:28.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219942195 0 0) 0x7f6e2c0c99e0 con 0x7f6e2c0c9490 2023-10-06T05:35:28.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c0b06b0 con 0x7f6e2c0c9490 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e2affd700 1 -- 172.21.15.33:0/323462280 >> v1:172.21.15.167:6789/0 conn(0x7f6e2c0c9490 legacy=0x7f6e2c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e2affd700 1 -- 172.21.15.33:0/323462280 >> v1:172.21.15.33:6789/0 conn(0x7f6e2c0afb20 legacy=0x7f6e2c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076553465 0 0) 0x7f6e14004680 con 0x7f6e2c0afb20 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e2affd700 1 -- 172.21.15.33:0/323462280 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0c9fa0 con 0x7f6e2c155a50 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.805+0000 7f6e2affd700 1 --2- 172.21.15.33:0/323462280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c155e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009940 tx=0x7f6e1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42da4c2d881698d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240017f0 con 0x7f6e2c155a50 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c011420 con 0x7f6e2c155a50 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e29ffb700 1 -- 172.21.15.33:0/323462280 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0117c0 con 0x7f6e2c155a50 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 -- 172.21.15.33:0/323462280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 msgr2=0x7f6e2c155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/323462280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c155e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009940 tx=0x7f6e1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 -- 172.21.15.33:0/323462280 shutdown_connections 2023-10-06T05:35:28.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/323462280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c155e20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 -- 172.21.15.33:0/323462280 >> 172.21.15.33:0/323462280 conn(0x7f6e2c06d030 msgr2=0x7f6e2c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:28.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 -- 172.21.15.33:0/323462280 shutdown_connections 2023-10-06T05:35:28.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.806+0000 7f6e2bfff700 1 -- 172.21.15.33:0/323462280 wait complete. 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.808+0000 7f6e2bfff700 1 Processor -- start 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.808+0000 7f6e2bfff700 1 -- start start 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e2c0afb20 0x7f6e2c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 0x7f6e2c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2bfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c14f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2bfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c5760 con 0x7f6e2c0c9490 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c6000 con 0x7f6e2c0afb20 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2bfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c1569b0 con 0x7f6e2c155a50 2023-10-06T05:35:28.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 0x7f6e2c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e2c0afb20 0x7f6e2c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 0x7f6e2c14de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39472/0 (socket says 172.21.15.33:39472) 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2a7fc700 1 -- 172.21.15.33:0/3562257410 learned_addr learned my addr 172.21.15.33:0/3562257410 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.809+0000 7f6e2b7fe700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c14f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2affd700 1 -- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 msgr2=0x7f6e2c14f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2affd700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c14f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2affd700 1 -- 172.21.15.33:0/3562257410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 msgr2=0x7f6e2c14de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2affd700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 0x7f6e2c14de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2affd700 1 -- 172.21.15.33:0/3562257410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c009420 con 0x7f6e2c0afb20 2023-10-06T05:35:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2a7fc700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 0x7f6e2c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:28.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2affd700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e2c0afb20 0x7f6e2c156470 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c015000 tx=0x7f6e1c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:28.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c01b8d0 con 0x7f6e2c0afb20 2023-10-06T05:35:28.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c020070 con 0x7f6e2c0afb20 2023-10-06T05:35:28.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c01bca0 con 0x7f6e2c0afb20 2023-10-06T05:35:28.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c1502f0 con 0x7f6e2c0afb20 2023-10-06T05:35:28.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.810+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c0c5c90 con 0x7f6e2c0afb20 2023-10-06T05:35:28.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.811+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c04c380 con 0x7f6e2c0afb20 2023-10-06T05:35:28.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.812+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e1c022030 con 0x7f6e2c0afb20 2023-10-06T05:35:28.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.814+0000 7f6e12ffd700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.814+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e1c085df0 con 0x7f6e2c0afb20 2023-10-06T05:35:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.814+0000 7f6e2a7fc700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.815+0000 7f6e2a7fc700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e18066f30 0x7f6e180693e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c15e5e0 tx=0x7f6e20007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:28.817+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e1c052eb0 con 0x7f6e2c0afb20 2023-10-06T05:35:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.020+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e2c06bb10 con 0x7f6e2c0afb20 2023-10-06T05:35:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.021+0000 7f6e12ffd700 1 -- 172.21.15.33:0/3562257410 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v283) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e1c02d3a0 con 0x7f6e2c0afb20 2023-10-06T05:35:29.023 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e18066f30 msgr2=0x7f6e180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e18066f30 0x7f6e180693e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c15e5e0 tx=0x7f6e20007e10 comp rx=0 tx=0).stop 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e2c0afb20 msgr2=0x7f6e2c156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e2c0afb20 0x7f6e2c156470 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c015000 tx=0x7f6e1c016000 comp rx=0 tx=0).stop 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 shutdown_connections 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e2c155a50 0x7f6e2c14f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e2c0afb20 0x7f6e2c156470 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 --2- 172.21.15.33:0/3562257410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e2c0c9490 0x7f6e2c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 >> 172.21.15.33:0/3562257410 conn(0x7f6e2c06d030 msgr2=0x7f6e2c15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.023+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 shutdown_connections 2023-10-06T05:35:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.024+0000 7f6e2bfff700 1 -- 172.21.15.33:0/3562257410 wait complete. 2023-10-06T05:35:29.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.227+0000 7f07ce39d700 1 Processor -- start 2023-10-06T05:35:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.228+0000 7f07ce39d700 1 -- start start 2023-10-06T05:35:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ce39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c8155140 0x7f07c8158700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ce39d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07c80adf50 0x7f07c80c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ce39d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c80c8e40 con 0x7f07c80c9810 2023-10-06T05:35:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ce39d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c80ae6c0 con 0x7f07c80adf50 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ce39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80ae9a0 con 0x7f07c8155140 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ccb9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07c80adf50 0x7f07c80c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ccb9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f07c80adf50 0x7f07c80c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39478/0 (socket says 172.21.15.33:39478) 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07ccb9a700 1 -- 172.21.15.33:0/364635674 learned_addr learned my addr 172.21.15.33:0/364635674 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.229+0000 7f07cd39b700 1 --2- 172.21.15.33:0/364635674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c8155140 0x7f07c8158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07ccb9a700 1 -- 172.21.15.33:0/364635674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c8155140 msgr2=0x7f07c8158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07ccb9a700 1 --2- 172.21.15.33:0/364635674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c8155140 0x7f07c8158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07ccb9a700 1 -- 172.21.15.33:0/364635674 >> v1:172.21.15.167:6789/0 conn(0x7f07c80c9810 legacy=0x7f07c80699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07ccb9a700 1 -- 172.21.15.33:0/364635674 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c80c8ad0 con 0x7f07c80adf50 2023-10-06T05:35:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07ccb9a700 1 --2- 172.21.15.33:0/364635674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c80adf50 0x7f07c80c92d0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f07b80099b0 tx=0x7f07b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ada95f7bdeeb5e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07bffff700 1 -- 172.21.15.33:0/364635674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800b860 con 0x7f07c80adf50 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07bffff700 1 -- 172.21.15.33:0/364635674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b800b9c0 con 0x7f07c80adf50 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.230+0000 7f07bffff700 1 -- 172.21.15.33:0/364635674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800bd60 con 0x7f07c80adf50 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 -- 172.21.15.33:0/364635674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c80adf50 msgr2=0x7f07c80c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 --2- 172.21.15.33:0/364635674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c80adf50 0x7f07c80c92d0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f07b80099b0 tx=0x7f07b8009250 comp rx=0 tx=0).stop 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 -- 172.21.15.33:0/364635674 shutdown_connections 2023-10-06T05:35:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 --2- 172.21.15.33:0/364635674 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c8155140 0x7f07c8158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 --2- 172.21.15.33:0/364635674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c80adf50 0x7f07c80c92d0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 -- 172.21.15.33:0/364635674 >> 172.21.15.33:0/364635674 conn(0x7f07c806d030 msgr2=0x7f07c806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 -- 172.21.15.33:0/364635674 shutdown_connections 2023-10-06T05:35:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.231+0000 7f07ce39d700 1 -- 172.21.15.33:0/364635674 wait complete. 2023-10-06T05:35:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.232+0000 7f07ce39d700 1 Processor -- start 2023-10-06T05:35:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.232+0000 7f07ce39d700 1 -- start start 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.233+0000 7f07ce39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07c80adf50 0x7f07c80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.233+0000 7f07ce39d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c80c9810 0x7f07c814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.233+0000 7f07ce39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 0x7f07c816afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.233+0000 7f07ce39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8169df0 con 0x7f07c8155140 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.233+0000 7f07ce39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80cb5c0 con 0x7f07c80adf50 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.233+0000 7f07ce39d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c814e450 con 0x7f07c80c9810 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 0x7f07c816afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 0x7f07c816afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39482/0 (socket says 172.21.15.33:39482) 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 -- 172.21.15.33:0/388625345 learned_addr learned my addr 172.21.15.33:0/388625345 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07ccb9a700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c80c9810 0x7f07c814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cd39b700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07c80adf50 0x7f07c80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 -- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c80c9810 msgr2=0x7f07c814df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c80c9810 0x7f07c814df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 -- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07c80adf50 msgr2=0x7f07c80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07c80adf50 0x7f07c80c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.234+0000 7f07cdb9c700 1 -- 172.21.15.33:0/388625345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80094f0 con 0x7f07c8155140 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.235+0000 7f07cdb9c700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 0x7f07c816afc0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f07c000c910 tx=0x7f07c00016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.235+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0012630 con 0x7f07c8155140 2023-10-06T05:35:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.235+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00127b0 con 0x7f07c8155140 2023-10-06T05:35:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.235+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c80cad20 con 0x7f07c8155140 2023-10-06T05:35:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.235+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0012b70 con 0x7f07c8155140 2023-10-06T05:35:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.235+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c816a000 con 0x7f07c8155140 2023-10-06T05:35:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.236+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c80cb120 con 0x7f07c8155140 2023-10-06T05:35:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.237+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07c001a3a0 con 0x7f07c8155140 2023-10-06T05:35:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.238+0000 7f07bd7fa700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07ac066f30 0x7f07ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.238+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07c0084850 con 0x7f07c8155140 2023-10-06T05:35:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.239+0000 7f07cd39b700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07ac066f30 0x7f07ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.239+0000 7f07cd39b700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07ac066f30 0x7f07ac0693e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f07c4006fd0 tx=0x7f07c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.243+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07c0051890 con 0x7f07c8155140 2023-10-06T05:35:29.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.489+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07c804c380 con 0x7f07c8155140 2023-10-06T05:35:29.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.490+0000 7f07bd7fa700 1 -- 172.21.15.33:0/388625345 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f07c0054eb0 con 0x7f07c8155140 2023-10-06T05:35:29.492 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:29.492 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:29.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.492+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07ac066f30 msgr2=0x7f07ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.492+0000 7f07ce39d700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07ac066f30 0x7f07ac0693e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f07c4006fd0 tx=0x7f07c4008040 comp rx=0 tx=0).stop 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.492+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 msgr2=0x7f07c816afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.492+0000 7f07ce39d700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 0x7f07c816afc0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f07c000c910 tx=0x7f07c00016b0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 shutdown_connections 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f07ac066f30 0x7f07ac0693e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07c8155140 0x7f07c816afc0 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07c80adf50 0x7f07c80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 --2- 172.21.15.33:0/388625345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07c80c9810 0x7f07c814df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 >> 172.21.15.33:0/388625345 conn(0x7f07c806d030 msgr2=0x7f07c80ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 shutdown_connections 2023-10-06T05:35:29.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.493+0000 7f07ce39d700 1 -- 172.21.15.33:0/388625345 wait complete. 2023-10-06T05:35:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:29.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.697+0000 7fd1c539d700 1 Processor -- start 2023-10-06T05:35:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.698+0000 7fd1c539d700 1 -- start start 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1c539d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1c006e140 0x7fd1c015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1c539d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c006f220 con 0x7fd1c0159530 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1c539d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c015ae50 con 0x7fd1c00afdd0 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1c539d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c00b1e00 con 0x7fd1c006e140 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1bffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd1c0159530 0x7fd1c006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41042/0 (socket says 172.21.15.33:41042) 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1bf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1c006e140 0x7fd1c015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.699+0000 7fd1bffff700 1 -- 172.21.15.33:0/4222174368 learned_addr learned my addr 172.21.15.33:0/4222174368 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/4222174368 >> v1:172.21.15.167:6789/0 conn(0x7fd1c00afdd0 legacy=0x7fd1c00b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/4222174368 >> v1:172.21.15.33:6789/0 conn(0x7fd1c0159530 legacy=0x7fd1c006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/4222174368 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c006e610 con 0x7fd1c006e140 2023-10-06T05:35:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1beffd700 1 -- 172.21.15.33:0/4222174368 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336730249 0 0) 0x7fd1c006f220 con 0x7fd1c0159530 2023-10-06T05:35:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1bf7fe700 1 --2- 172.21.15.33:0/4222174368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c006e140 0x7fd1c015a910 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098e0 tx=0x7fd1b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6225cf45b8002782 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1beffd700 1 -- 172.21.15.33:0/4222174368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4009e80 con 0x7fd1c006e140 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1beffd700 1 -- 172.21.15.33:0/4222174368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400b860 con 0x7fd1c006e140 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.700+0000 7fd1beffd700 1 -- 172.21.15.33:0/4222174368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400bbe0 con 0x7fd1c006e140 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 -- 172.21.15.33:0/4222174368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c006e140 msgr2=0x7fd1c015a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 --2- 172.21.15.33:0/4222174368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c006e140 0x7fd1c015a910 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098e0 tx=0x7fd1b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 -- 172.21.15.33:0/4222174368 shutdown_connections 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 --2- 172.21.15.33:0/4222174368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c006e140 0x7fd1c015a910 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 -- 172.21.15.33:0/4222174368 >> 172.21.15.33:0/4222174368 conn(0x7fd1c006c560 msgr2=0x7fd1c006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 -- 172.21.15.33:0/4222174368 shutdown_connections 2023-10-06T05:35:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.701+0000 7fd1c539d700 1 -- 172.21.15.33:0/4222174368 wait complete. 2023-10-06T05:35:29.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.702+0000 7fd1c539d700 1 Processor -- start 2023-10-06T05:35:29.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 -- start start 2023-10-06T05:35:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1c006e140 0x7fd1c013f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 0x7fd1c015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1c0159530 0x7fd1c015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00cd240 con 0x7fd1c00afdd0 2023-10-06T05:35:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00cb320 con 0x7fd1c0159530 2023-10-06T05:35:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.703+0000 7fd1c539d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c016dd80 con 0x7fd1c006e140 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 0x7fd1c015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 0x7fd1c015bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39500/0 (socket says 172.21.15.33:39500) 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/922204260 learned_addr learned my addr 172.21.15.33:0/922204260 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bffff700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1c006e140 0x7fd1c013f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1c4b9c700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1c0159530 0x7fd1c015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1c006e140 msgr2=0x7fd1c013f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1c006e140 0x7fd1c013f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1c0159530 msgr2=0x7fd1c015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1c0159530 0x7fd1c015c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1bf7fe700 1 -- 172.21.15.33:0/922204260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b4009420 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.704+0000 7fd1c4b9c700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1c0159530 0x7fd1c015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.705+0000 7fd1bf7fe700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 0x7fd1c015bed0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b3c0 tx=0x7fd1b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.705+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b401f070 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.705+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400b9d0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.705+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4025650 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.705+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c00cc910 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.705+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c00cced0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.706+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c00cd3f0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.707+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1b4023070 con 0x7fd1c00afdd0 2023-10-06T05:35:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.708+0000 7fd1a7fff700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.708+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1b4085cb0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.708+0000 7fd1bffff700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.709+0000 7fd1bffff700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1ac066f30 0x7fd1ac0693e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0153760 tx=0x7fd1b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.712+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1c00cd3f0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.915+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c00cd3f0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.916+0000 7fd1a7fff700 1 -- 172.21.15.33:0/922204260 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v283) v1 ==== 74+0+16941 (secure 0 0 0) 0x7fd1c00cd3f0 con 0x7fd1c00afdd0 2023-10-06T05:35:29.918 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:29.918 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":283,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:25.942126+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:35:12.826592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":42,"pool_name":"cephfs_data","create_time":"2023-10-06T05:35:13.627133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"282","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2784413487":"2023-10-06T06:35:25.570337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1ac066f30 msgr2=0x7fd1ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1ac066f30 0x7fd1ac0693e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0153760 tx=0x7fd1b0009040 comp rx=0 tx=0).stop 2023-10-06T05:35:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 msgr2=0x7fd1c015bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 0x7fd1c015bed0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b3c0 tx=0x7fd1b4017000 comp rx=0 tx=0).stop 2023-10-06T05:35:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 shutdown_connections 2023-10-06T05:35:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1c0159530 0x7fd1c015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1c006e140 0x7fd1c013f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 --2- 172.21.15.33:0/922204260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c00afdd0 0x7fd1c015bed0 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.918+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 >> 172.21.15.33:0/922204260 conn(0x7fd1c006c560 msgr2=0x7fd1c0155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.919+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 shutdown_connections 2023-10-06T05:35:29.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:29.919+0000 7fd1c539d700 1 -- 172.21.15.33:0/922204260 wait complete. 2023-10-06T05:35:29.938 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:35:29.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:30.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.124+0000 7fbad339d700 1 Processor -- start 2023-10-06T05:35:30.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.124+0000 7fbad339d700 1 -- start start 2023-10-06T05:35:30.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.125+0000 7fbad339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.125+0000 7fbad339d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbacc0699c0 0x7fbacc069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.125+0000 7fbad339d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbacc0ae490 con 0x7fbacc155140 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.125+0000 7fbad339d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc06a380 con 0x7fbacc0699c0 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.125+0000 7fbad339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0c8e40 con 0x7fbacc158700 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.125+0000 7fbad2b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbacc0699c0 0x7fbacc069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbacc0699c0 0x7fbacc069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44618/0 (socket says 172.21.15.33:44618) 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 -- 172.21.15.33:0/3802860790 learned_addr learned my addr 172.21.15.33:0/3802860790 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad1b9a700 1 --2- 172.21.15.33:0/3802860790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 -- 172.21.15.33:0/3802860790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 msgr2=0x7fbacc0adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:30.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 --2- 172.21.15.33:0/3802860790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc0adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 -- 172.21.15.33:0/3802860790 >> v1:172.21.15.167:6789/0 conn(0x7fbacc155140 legacy=0x7fbacc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 -- 172.21.15.33:0/3802860790 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc158b70 con 0x7fbacc0699c0 2023-10-06T05:35:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad2b9c700 1 --2- 172.21.15.33:0/3802860790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc069e40 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f20 tx=0x7fbac4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930da5a97dba01e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.126+0000 7fbad1399700 1 -- 172.21.15.33:0/3802860790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400e070 con 0x7fbacc0699c0 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad1399700 1 -- 172.21.15.33:0/3802860790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400b9d0 con 0x7fbacc0699c0 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad1399700 1 -- 172.21.15.33:0/3802860790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40092e0 con 0x7fbacc0699c0 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 -- 172.21.15.33:0/3802860790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 msgr2=0x7fbacc069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 --2- 172.21.15.33:0/3802860790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc069e40 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f20 tx=0x7fbac4009840 comp rx=0 tx=0).stop 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 -- 172.21.15.33:0/3802860790 shutdown_connections 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 --2- 172.21.15.33:0/3802860790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc069e40 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 --2- 172.21.15.33:0/3802860790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc0adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 -- 172.21.15.33:0/3802860790 >> 172.21.15.33:0/3802860790 conn(0x7fbacc06d030 msgr2=0x7fbacc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 -- 172.21.15.33:0/3802860790 shutdown_connections 2023-10-06T05:35:30.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.127+0000 7fbad339d700 1 -- 172.21.15.33:0/3802860790 wait complete. 2023-10-06T05:35:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.129+0000 7fbad339d700 1 Processor -- start 2023-10-06T05:35:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.129+0000 7fbad339d700 1 -- start start 2023-10-06T05:35:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad339d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad339d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbacc155140 0x7fbacc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad339d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc167910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad339d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc155b60 con 0x7fbacc0699c0 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad339d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc168430 con 0x7fbacc155140 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad339d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc168710 con 0x7fbacc158700 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44624/0 (socket says 172.21.15.33:44624) 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 -- 172.21.15.33:0/384721801 learned_addr learned my addr 172.21.15.33:0/384721801 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad1b9a700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbacc155140 0x7fbacc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad2b9c700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 -- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 msgr2=0x7fbacc167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 -- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbacc155140 msgr2=0x7fbacc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbacc155140 0x7fbacc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad239b700 1 -- 172.21.15.33:0/384721801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4009ae0 con 0x7fbacc0699c0 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad1b9a700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbacc155140 0x7fbacc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.130+0000 7fbad2b9c700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc167910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.131+0000 7fbad239b700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc155ff0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c590 tx=0x7fbac800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.131+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011590 con 0x7fbacc0699c0 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.131+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac80116f0 con 0x7fbacc0699c0 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.131+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc155510 con 0x7fbacc0699c0 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.131+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011ab0 con 0x7fbacc0699c0 2023-10-06T05:35:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.131+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc1532d0 con 0x7fbacc0699c0 2023-10-06T05:35:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.132+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc1559b0 con 0x7fbacc0699c0 2023-10-06T05:35:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.133+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbac801b3b0 con 0x7fbacc0699c0 2023-10-06T05:35:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.134+0000 7fbac27fc700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbab0066f30 0x7fbab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.134+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbac8083c60 con 0x7fbacc0699c0 2023-10-06T05:35:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.135+0000 7fbad1b9a700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbab0066f30 0x7fbab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.135+0000 7fbad1b9a700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbab0066f30 0x7fbab00693e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbabc006f00 tx=0x7fbabc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.138+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbac8050ca0 con 0x7fbacc0699c0 2023-10-06T05:35:30.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.390+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbacc155e40 con 0x7fbacc0699c0 2023-10-06T05:35:30.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.391+0000 7fbac27fc700 1 -- 172.21.15.33:0/384721801 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7fbac8017070 con 0x7fbacc0699c0 2023-10-06T05:35:30.392 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:30.393 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:30.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbab0066f30 msgr2=0x7fbab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbab0066f30 0x7fbab00693e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbabc006f00 tx=0x7fbabc008040 comp rx=0 tx=0).stop 2023-10-06T05:35:30.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 msgr2=0x7fbacc155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc155ff0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c590 tx=0x7fbac800c950 comp rx=0 tx=0).stop 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 shutdown_connections 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fbab0066f30 0x7fbab00693e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbacc158700 0x7fbacc167910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbacc0699c0 0x7fbacc155ff0 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 --2- 172.21.15.33:0/384721801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbacc155140 0x7fbacc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 >> 172.21.15.33:0/384721801 conn(0x7fbacc06d030 msgr2=0x7fbacc0c2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 shutdown_connections 2023-10-06T05:35:30.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.393+0000 7fbad339d700 1 -- 172.21.15.33:0/384721801 wait complete. 2023-10-06T05:35:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:30.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:35:30.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.598+0000 7f94109fe700 1 Processor -- start 2023-10-06T05:35:30.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.599+0000 7f94109fe700 1 -- start start 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f94109fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f940c0c8b50 0x7f940c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f94109fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f940c155ec0 0x7f940c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f94109fe700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940c06ab60 con 0x7f940c155af0 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f94109fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940c06b7e0 con 0x7f940c155ec0 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f94109fe700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940c06ae40 con 0x7f940c0c8b50 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f940b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f940c155ec0 0x7f940c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f940b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f940c155ec0 0x7f940c0c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44626/0 (socket says 172.21.15.33:44626) 2023-10-06T05:35:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.600+0000 7f940b59e700 1 -- 172.21.15.33:0/3835121640 learned_addr learned my addr 172.21.15.33:0/3835121640 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:30.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f940a59c700 1 --2- 172.21.15.33:0/3835121640 >> v2:172.21.15.167:3300/0 conn(0x7f940c0c8b50 0x7f940c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f940b59e700 1 -- 172.21.15.33:0/3835121640 >> v2:172.21.15.167:3300/0 conn(0x7f940c0c8b50 msgr2=0x7f940c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:30.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f940b59e700 1 --2- 172.21.15.33:0/3835121640 >> v2:172.21.15.167:3300/0 conn(0x7f940c0c8b50 0x7f940c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f940b59e700 1 -- 172.21.15.33:0/3835121640 >> v1:172.21.15.167:6789/0 conn(0x7f940c155af0 legacy=0x7f940c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f940b59e700 1 -- 172.21.15.33:0/3835121640 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c06b120 con 0x7f940c155ec0 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f940b59e700 1 --2- 172.21.15.33:0/3835121640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c0c7750 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f940c06f250 tx=0x7f9404009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b4b4b32cb51db6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f9409d9b700 1 -- 172.21.15.33:0/3835121640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400e070 con 0x7f940c155ec0 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f9409d9b700 1 -- 172.21.15.33:0/3835121640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940400ba10 con 0x7f940c155ec0 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.601+0000 7f9409d9b700 1 -- 172.21.15.33:0/3835121640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94040092e0 con 0x7f940c155ec0 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 -- 172.21.15.33:0/3835121640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 msgr2=0x7f940c0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 --2- 172.21.15.33:0/3835121640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c0c7750 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f940c06f250 tx=0x7f9404009840 comp rx=0 tx=0).stop 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 -- 172.21.15.33:0/3835121640 shutdown_connections 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 --2- 172.21.15.33:0/3835121640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c0c7750 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 --2- 172.21.15.33:0/3835121640 >> v2:172.21.15.167:3300/0 conn(0x7f940c0c8b50 0x7f940c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 -- 172.21.15.33:0/3835121640 >> 172.21.15.33:0/3835121640 conn(0x7f940c06d030 msgr2=0x7f940c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:30.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 -- 172.21.15.33:0/3835121640 shutdown_connections 2023-10-06T05:35:30.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.602+0000 7f94109fe700 1 -- 172.21.15.33:0/3835121640 wait complete. 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.603+0000 7f94109fe700 1 Processor -- start 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 -- start start 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f940c0c8b50 0x7f940c156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f940c155af0 0x7f940c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c16cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0c3900 con 0x7f940c155ec0 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0c3be0 con 0x7f940c155af0 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.604+0000 7f94109fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0b1170 con 0x7f940c0c8b50 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c16cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c16cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44638/0 (socket says 172.21.15.33:44638) 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 -- 172.21.15.33:0/3064293685 learned_addr learned my addr 172.21.15.33:0/3064293685 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940a59c700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f940c155af0 0x7f940c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940ad9d700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f940c0c8b50 0x7f940c156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 -- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f940c0c8b50 msgr2=0x7f940c156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f940c0c8b50 0x7f940c156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 -- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f940c155af0 msgr2=0x7f940c0b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f940c155af0 0x7f940c0b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 -- 172.21.15.33:0/3064293685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404009ae0 con 0x7f940c155ec0 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.605+0000 7f940b59e700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c16cec0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f9404009fd0 tx=0x7f9404017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.606+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400bee0 con 0x7f940c155ec0 2023-10-06T05:35:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.606+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94040094e0 con 0x7f940c155ec0 2023-10-06T05:35:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.606+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c0b1450 con 0x7f940c155ec0 2023-10-06T05:35:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.606+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94040032f0 con 0x7f940c155ec0 2023-10-06T05:35:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.606+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c0b0e80 con 0x7f940c155ec0 2023-10-06T05:35:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.607+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c0c3d70 con 0x7f940c155ec0 2023-10-06T05:35:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.608+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f940401d630 con 0x7f940c155ec0 2023-10-06T05:35:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.609+0000 7f93f2ffd700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f93f8066f30 0x7f93f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.609+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9404085e10 con 0x7f940c155ec0 2023-10-06T05:35:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.609+0000 7f940ad9d700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f93f8066f30 0x7f93f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.610+0000 7f940ad9d700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f93f8066f30 0x7f93f80693e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f93fc003870 tx=0x7f93fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.613+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9404089030 con 0x7f940c155ec0 2023-10-06T05:35:30.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.859+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f940c04c380 con 0x7f940c155ec0 2023-10-06T05:35:30.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.860+0000 7f93f2ffd700 1 -- 172.21.15.33:0/3064293685 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+4618 (secure 0 0 0) 0x7f9404052e50 con 0x7f940c155ec0 2023-10-06T05:35:30.861 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:30.862 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":219,"default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:35:14.652229+0000","modified":"2023-10-06T05:35:18.187888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12264":{"gid":12264,"name":"d","rank":0,"incarnation":215,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/3559384937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3559384937},{"type":"v1","addr":"172.21.15.167:6836","nonce":3559384937}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":20}]} 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f93f8066f30 msgr2=0x7f93f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f93f8066f30 0x7f93f80693e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f93fc003870 tx=0x7f93fc009040 comp rx=0 tx=0).stop 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 msgr2=0x7f940c16cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c16cec0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f9404009fd0 tx=0x7f9404017000 comp rx=0 tx=0).stop 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 shutdown_connections 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f93f8066f30 0x7f93f80693e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f940c155ec0 0x7f940c16cec0 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f940c0c8b50 0x7f940c156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 --2- 172.21.15.33:0/3064293685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f940c155af0 0x7f940c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:30.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 >> 172.21.15.33:0/3064293685 conn(0x7f940c06d030 msgr2=0x7f940c0c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:30.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 shutdown_connections 2023-10-06T05:35:30.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:30.862+0000 7f94109fe700 1 -- 172.21.15.33:0/3064293685 wait complete. 2023-10-06T05:35:30.865 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 219 2023-10-06T05:35:30.881 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.070+0000 7fba2679d700 1 Processor -- start 2023-10-06T05:35:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.070+0000 7fba2679d700 1 -- start start 2023-10-06T05:35:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.071+0000 7fba2679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba2006b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.071+0000 7fba2679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200c9cb0 con 0x7fba2013faa0 2023-10-06T05:35:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.071+0000 7fba2679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200ca270 con 0x7fba200c8050 2023-10-06T05:35:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.071+0000 7fba2679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2006b840 con 0x7fba200cb710 2023-10-06T05:35:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.071+0000 7fba2579b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fba2013faa0 0x7fba200c7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52272/0 (socket says 172.21.15.33:52272) 2023-10-06T05:35:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.071+0000 7fba2579b700 1 -- 172.21.15.33:0/2927817182 learned_addr learned my addr 172.21.15.33:0/2927817182 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:31.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba25f9c700 1 --2- 172.21.15.33:0/2927817182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba2006b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791329465 0 0) 0x7fba200c9cb0 con 0x7fba2013faa0 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba0c003570 con 0x7fba2013faa0 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256275009 0 0) 0x7fba200ca270 con 0x7fba200c8050 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba200c9cb0 con 0x7fba200c8050 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba25f9c700 1 -- 172.21.15.33:0/2927817182 >> v1:172.21.15.167:6789/0 conn(0x7fba200c8050 legacy=0x7fba2006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba25f9c700 1 -- 172.21.15.33:0/2927817182 >> v1:172.21.15.33:6789/0 conn(0x7fba2013faa0 legacy=0x7fba200c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518806587 0 0) 0x7fba0c003570 con 0x7fba2013faa0 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.072+0000 7fba25f9c700 1 -- 172.21.15.33:0/2927817182 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2006bb20 con 0x7fba200cb710 2023-10-06T05:35:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba25f9c700 1 --2- 172.21.15.33:0/2927817182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba2006b1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e90 tx=0x7fba18009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d1d3093ba80249 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800e070 con 0x7fba200cb710 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800bb80 con 0x7fba200cb710 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba17fff700 1 -- 172.21.15.33:0/2927817182 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18012430 con 0x7fba200cb710 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba2679d700 1 -- 172.21.15.33:0/2927817182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 msgr2=0x7fba2006b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba2679d700 1 --2- 172.21.15.33:0/2927817182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba2006b1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e90 tx=0x7fba18009780 comp rx=0 tx=0).stop 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba2679d700 1 -- 172.21.15.33:0/2927817182 shutdown_connections 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba2679d700 1 --2- 172.21.15.33:0/2927817182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba2006b1e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.073+0000 7fba2679d700 1 -- 172.21.15.33:0/2927817182 >> 172.21.15.33:0/2927817182 conn(0x7fba2006d030 msgr2=0x7fba2006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.074+0000 7fba2679d700 1 -- 172.21.15.33:0/2927817182 shutdown_connections 2023-10-06T05:35:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.074+0000 7fba2679d700 1 -- 172.21.15.33:0/2927817182 wait complete. 2023-10-06T05:35:31.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.075+0000 7fba2679d700 1 Processor -- start 2023-10-06T05:35:31.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.075+0000 7fba2679d700 1 -- start start 2023-10-06T05:35:31.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba200c8050 0x7fba20158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba200a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 0x7fba200cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c43b0 con 0x7fba2013faa0 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c4650 con 0x7fba200c8050 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c4930 con 0x7fba200cb710 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba25f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 0x7fba200cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba25f9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 0x7fba200cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44650/0 (socket says 172.21.15.33:44650) 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba25f9c700 1 -- 172.21.15.33:0/3863719090 learned_addr learned my addr 172.21.15.33:0/3863719090 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba2579b700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba200c8050 0x7fba20158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.076+0000 7fba24f9a700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba200a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba25f9c700 1 -- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 msgr2=0x7fba200a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba25f9c700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba200a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba25f9c700 1 -- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba200c8050 msgr2=0x7fba20158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba25f9c700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba200c8050 0x7fba20158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba25f9c700 1 -- 172.21.15.33:0/3863719090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18009a20 con 0x7fba2013faa0 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba25f9c700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 0x7fba200cbee0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fba200719c0 tx=0x7fba18019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18027470 con 0x7fba2013faa0 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba180275f0 con 0x7fba2013faa0 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba180279b0 con 0x7fba2013faa0 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.077+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200c4c10 con 0x7fba2013faa0 2023-10-06T05:35:31.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.078+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba20158f90 con 0x7fba2013faa0 2023-10-06T05:35:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.078+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200c5110 con 0x7fba2013faa0 2023-10-06T05:35:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.079+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba18024070 con 0x7fba2013faa0 2023-10-06T05:35:31.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.080+0000 7fba157fa700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba04066fb0 0x7fba04069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.080+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba18087150 con 0x7fba2013faa0 2023-10-06T05:35:31.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.081+0000 7fba2579b700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba04066fb0 0x7fba04069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.081+0000 7fba2579b700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba04066fb0 0x7fba04069460 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fba1c000fb0 tx=0x7fba1c007ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.084+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba18054190 con 0x7fba2013faa0 2023-10-06T05:35:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.289+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba2004c380 con 0x7fba2013faa0 2023-10-06T05:35:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.290+0000 7fba157fa700 1 -- 172.21.15.33:0/3863719090 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v283) v1 ==== 74+0+16941 (secure 0 0 0) 0x7fba180577b0 con 0x7fba2013faa0 2023-10-06T05:35:31.292 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:31.292 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":283,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:25.942126+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:35:12.826592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":42,"pool_name":"cephfs_data","create_time":"2023-10-06T05:35:13.627133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"282","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2784413487":"2023-10-06T06:35:25.570337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba04066fb0 msgr2=0x7fba04069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba04066fb0 0x7fba04069460 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fba1c000fb0 tx=0x7fba1c007ce0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 msgr2=0x7fba200cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 0x7fba200cbee0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fba200719c0 tx=0x7fba18019000 comp rx=0 tx=0).stop 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 shutdown_connections 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fba04066fb0 0x7fba04069460 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba2013faa0 0x7fba200cbee0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba200c8050 0x7fba20158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 --2- 172.21.15.33:0/3863719090 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba200cb710 0x7fba200a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.292+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 >> 172.21.15.33:0/3863719090 conn(0x7fba2006d030 msgr2=0x7fba200c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:31.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.293+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 shutdown_connections 2023-10-06T05:35:31.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.293+0000 7fba2679d700 1 -- 172.21.15.33:0/3863719090 wait complete. 2023-10-06T05:35:31.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:35:31.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.499+0000 7ff1e9f9d700 1 Processor -- start 2023-10-06T05:35:31.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.499+0000 7ff1e9f9d700 1 -- start start 2023-10-06T05:35:31.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e9f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156880 0x7ff1e40c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e9f9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1e4159e10 0x7ff1e40c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e9f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e406a820 con 0x7ff1e4156c50 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e9f9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40cc290 con 0x7ff1e4159e10 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e9f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e406bd10 con 0x7ff1e4156880 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1e4159e10 0x7ff1e40c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e3fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1e4159e10 0x7ff1e40c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44662/0 (socket says 172.21.15.33:44662) 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.500+0000 7ff1e3fff700 1 -- 172.21.15.33:0/559039313 learned_addr learned my addr 172.21.15.33:0/559039313 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e8f9b700 1 --2- 172.21.15.33:0/559039313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156880 0x7ff1e40c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e3fff700 1 -- 172.21.15.33:0/559039313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156880 msgr2=0x7ff1e40c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e3fff700 1 --2- 172.21.15.33:0/559039313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156880 0x7ff1e40c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e3fff700 1 -- 172.21.15.33:0/559039313 >> v1:172.21.15.167:6789/0 conn(0x7ff1e4156c50 legacy=0x7ff1e41570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e3fff700 1 -- 172.21.15.33:0/559039313 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e41577c0 con 0x7ff1e4159e10 2023-10-06T05:35:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e3fff700 1 --2- 172.21.15.33:0/559039313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e40c7ec0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00099b0 tx=0x7ff1d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fa930413aae9f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e37fe700 1 -- 172.21.15.33:0/559039313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000b860 con 0x7ff1e4159e10 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e37fe700 1 -- 172.21.15.33:0/559039313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000b9c0 con 0x7ff1e4159e10 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.501+0000 7ff1e37fe700 1 -- 172.21.15.33:0/559039313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000bd60 con 0x7ff1e4159e10 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/559039313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 msgr2=0x7ff1e40c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/559039313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e40c7ec0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00099b0 tx=0x7ff1d0009250 comp rx=0 tx=0).stop 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/559039313 shutdown_connections 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/559039313 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156880 0x7ff1e40c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/559039313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e40c7ec0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/559039313 >> 172.21.15.33:0/559039313 conn(0x7ff1e406c560 msgr2=0x7ff1e406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/559039313 shutdown_connections 2023-10-06T05:35:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.502+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/559039313 wait complete. 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.503+0000 7ff1e9f9d700 1 Processor -- start 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.504+0000 7ff1e9f9d700 1 -- start start 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e9f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e4156880 0x7ff1e40ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e9f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 0x7ff1e413f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e9f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e413f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e9f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e413eff0 con 0x7ff1e4159e10 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e9f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e413eaf0 con 0x7ff1e4156880 2023-10-06T05:35:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e9f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40c9a10 con 0x7ff1e4156c50 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e979c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e413f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e8f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e4156880 0x7ff1e40ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 0x7ff1e413f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 0x7ff1e413f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:33534/0 (socket says 172.21.15.33:33534) 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e8f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e4156880 0x7ff1e40ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:47800/0 (socket says 172.21.15.33:47800) 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 -- 172.21.15.33:0/850802624 learned_addr learned my addr 172.21.15.33:0/850802624 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 -- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e4156880 msgr2=0x7ff1e40ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e4156880 0x7ff1e40ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 -- 172.21.15.33:0/850802624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 msgr2=0x7ff1e413f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e413f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e3fff700 1 -- 172.21.15.33:0/850802624 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d00094f0 con 0x7ff1e4156c50 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.505+0000 7ff1e979c700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e413f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.506+0000 7ff1e3fff700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 0x7ff1e413f280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40636f0 tx=0x7ff1d0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.506+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0011e40 con 0x7ff1e4156c50 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.506+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0024440 con 0x7ff1e4156c50 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.506+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0024800 con 0x7ff1e4156c50 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.506+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e40c8c70 con 0x7ff1e4156c50 2023-10-06T05:35:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.506+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e40c9210 con 0x7ff1e4156c50 2023-10-06T05:35:31.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.507+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e404c380 con 0x7ff1e4156c50 2023-10-06T05:35:31.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.508+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1d001a420 con 0x7ff1e4156c50 2023-10-06T05:35:31.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.509+0000 7ff1e0ff9700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1c8066f30 0x7ff1c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.509+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(283..283 src has 1..283) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1d0086560 con 0x7ff1e4156c50 2023-10-06T05:35:31.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.509+0000 7ff1e8f9b700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1c8066f30 0x7ff1c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.510+0000 7ff1e8f9b700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1c8066f30 0x7ff1c80693e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8006fd0 tx=0x7ff1d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.514+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1d00535a0 con 0x7ff1e4156c50 2023-10-06T05:35:31.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.766+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff1e415a350 con 0x7ff1e4156c50 2023-10-06T05:35:31.768 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T05:35:31.767+0000 7f5029bf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:35:31.779 INFO:tasks.ceph.mds.d.smithi167.stderr: -812> 2023-10-06T05:35:21.142+0000 7efdd57f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:35:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.777+0000 7ff1e0ff9700 1 -- 172.21.15.33:0/850802624 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v220) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff1d0056bc0 con 0x7ff1e4156c50 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1c8066f30 msgr2=0x7ff1c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1c8066f30 0x7ff1c80693e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8006fd0 tx=0x7ff1d8008040 comp rx=0 tx=0).stop 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 msgr2=0x7ff1e413f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 0x7ff1e413f280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40636f0 tx=0x7ff1d0017000 comp rx=0 tx=0).stop 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 shutdown_connections 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ff1c8066f30 0x7ff1c80693e0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4159e10 0x7ff1e413f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1e4156880 0x7ff1e40ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 --2- 172.21.15.33:0/850802624 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff1e4156c50 0x7ff1e413f280 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 >> 172.21.15.33:0/850802624 conn(0x7ff1e406c560 msgr2=0x7ff1e4157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.779+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 shutdown_connections 2023-10-06T05:35:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.780+0000 7ff1e9f9d700 1 -- 172.21.15.33:0/850802624 wait complete. 2023-10-06T05:35:31.783 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:35:31.799 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:35:31.816 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T05:35:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.989+0000 7f50ccd9d700 1 Processor -- start 2023-10-06T05:35:31.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.989+0000 7f50ccd9d700 1 -- start start 2023-10-06T05:35:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.990+0000 7f50ccd9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50c80aff90 0x7f50c80a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.990+0000 7f50ccd9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c815d3a0 con 0x7f50c806c7b0 2023-10-06T05:35:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.990+0000 7f50ccd9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c80b0470 con 0x7f50c806cb80 2023-10-06T05:35:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.990+0000 7f50ccd9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80686c0 con 0x7f50c80aff90 2023-10-06T05:35:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.990+0000 7f50c77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50c806c7b0 0x7f50c8067fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52284/0 (socket says 172.21.15.33:52284) 2023-10-06T05:35:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.990+0000 7f50c77fe700 1 -- 172.21.15.33:0/2492385460 learned_addr learned my addr 172.21.15.33:0/2492385460 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c6ffd700 1 --2- 172.21.15.33:0/2492385460 >> v2:172.21.15.167:3300/0 conn(0x7f50c80aff90 0x7f50c80a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170756938 0 0) 0x7f50c815d3a0 con 0x7f50c806c7b0 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b0003570 con 0x7f50c806c7b0 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559794769 0 0) 0x7f50c80b0470 con 0x7f50c806cb80 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c815d3a0 con 0x7f50c806cb80 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c6ffd700 1 -- 172.21.15.33:0/2492385460 >> v1:172.21.15.167:6789/0 conn(0x7f50c806cb80 legacy=0x7f50c815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c6ffd700 1 -- 172.21.15.33:0/2492385460 >> v1:172.21.15.33:6789/0 conn(0x7f50c806c7b0 legacy=0x7f50c8067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c6ffd700 1 -- 172.21.15.33:0/2492385460 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80b0750 con 0x7f50c80aff90 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.991+0000 7f50c6ffd700 1 --2- 172.21.15.33:0/2492385460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80aff90 0x7f50c80a9890 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0098e0 tx=0x7f50bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755e6e5bbfc6bb3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011630 con 0x7f50c80aff90 2023-10-06T05:35:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc011790 con 0x7f50c80aff90 2023-10-06T05:35:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50c67fc700 1 -- 172.21.15.33:0/2492385460 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc011b30 con 0x7f50c80aff90 2023-10-06T05:35:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 -- 172.21.15.33:0/2492385460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80aff90 msgr2=0x7f50c80a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/2492385460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80aff90 0x7f50c80a9890 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0098e0 tx=0x7f50bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 -- 172.21.15.33:0/2492385460 shutdown_connections 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/2492385460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80aff90 0x7f50c80a9890 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 -- 172.21.15.33:0/2492385460 >> 172.21.15.33:0/2492385460 conn(0x7f50c806d670 msgr2=0x7f50c806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 -- 172.21.15.33:0/2492385460 shutdown_connections 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.992+0000 7f50ccd9d700 1 -- 172.21.15.33:0/2492385460 wait complete. 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.994+0000 7f50ccd9d700 1 Processor -- start 2023-10-06T05:35:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.994+0000 7f50ccd9d700 1 -- start start 2023-10-06T05:35:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50ccd9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 0x7f50c814ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50ccd9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c806cb80 0x7f50c8158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50ccd9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80aff90 0x7f50c8152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50ccd9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8151070 con 0x7f50c806c7b0 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50ccd9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8150460 con 0x7f50c806cb80 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50ccd9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8150740 con 0x7f50c80aff90 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 0x7f50c814ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 0x7f50c814ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44672/0 (socket says 172.21.15.33:44672) 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50c77fe700 1 -- 172.21.15.33:0/3690657278 learned_addr learned my addr 172.21.15.33:0/3690657278 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50c7fff700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80aff90 0x7f50c8152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.995+0000 7f50c6ffd700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c806cb80 0x7f50c8158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c77fe700 1 -- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80aff90 msgr2=0x7f50c8152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c77fe700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80aff90 0x7f50c8152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c77fe700 1 -- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c806cb80 msgr2=0x7f50c8158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c77fe700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c806cb80 0x7f50c8158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c77fe700 1 -- 172.21.15.33:0/3690657278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc009420 con 0x7f50c806c7b0 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c7fff700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80aff90 0x7f50c8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50c77fe700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 0x7f50c814ab30 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c790 tx=0x7f50b8007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80072e0 con 0x7f50c806c7b0 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8007440 con 0x7f50c806c7b0 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c8150a20 con 0x7f50c806c7b0 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80057f0 con 0x7f50c806c7b0 2023-10-06T05:35:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.996+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c8158ad0 con 0x7f50c806c7b0 2023-10-06T05:35:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.997+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c804c380 con 0x7f50c806c7b0 2023-10-06T05:35:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:31.998+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50b8005ba0 con 0x7f50c806c7b0 2023-10-06T05:35:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.000+0000 7f50af7fe700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50b4066f30 0x7f50b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.000+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(284..284 src has 1..284) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f50b8083d10 con 0x7f50c806c7b0 2023-10-06T05:35:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.000+0000 7f50c6ffd700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50b4066f30 0x7f50b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.001+0000 7f50c6ffd700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50b4066f30 0x7f50b40693e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f50c80b03a0 tx=0x7f50bc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.004+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50b8089020 con 0x7f50c806c7b0 2023-10-06T05:35:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.251+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f50c80b0a30 con 0x7f50c806c7b0 2023-10-06T05:35:32.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.773+0000 7f50af7fe700 1 -- 172.21.15.33:0/3690657278 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v221) v1 ==== 104+0+0 (secure 0 0 0) 0x7f50b80170a0 con 0x7f50c806c7b0 2023-10-06T05:35:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50b4066f30 msgr2=0x7f50b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50b4066f30 0x7f50b40693e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f50c80b03a0 tx=0x7f50bc017000 comp rx=0 tx=0).stop 2023-10-06T05:35:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 msgr2=0x7f50c814ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 0x7f50c814ab30 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c790 tx=0x7f50b8007ce0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 shutdown_connections 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f50b4066f30 0x7f50b40693e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80aff90 0x7f50c8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c806c7b0 0x7f50c814ab30 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 --2- 172.21.15.33:0/3690657278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c806cb80 0x7f50c8158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.775+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 >> 172.21.15.33:0/3690657278 conn(0x7f50c806d670 msgr2=0x7f50c80af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.776+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 shutdown_connections 2023-10-06T05:35:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.776+0000 7f50ccd9d700 1 -- 172.21.15.33:0/3690657278 wait complete. 2023-10-06T05:35:32.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:35:32.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.987+0000 7f61c4b9d700 1 Processor -- start 2023-10-06T05:35:32.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.988+0000 7f61c4b9d700 1 -- start start 2023-10-06T05:35:32.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61c4b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c0067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:32.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61c4b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61c006cdd0 0x7f61c00c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:32.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61c4b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c0068500 con 0x7f61c0068090 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61c4b9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c00c93c0 con 0x7f61c006cdd0 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61c4b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c006d380 con 0x7f61c01559a0 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61bffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61c006cdd0 0x7f61c00c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61bffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f61c006cdd0 0x7f61c00c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44676/0 (socket says 172.21.15.33:44676) 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61bffff700 1 -- 172.21.15.33:0/2386739021 learned_addr learned my addr 172.21.15.33:0/2386739021 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.989+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/2386739021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c0067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61bffff700 1 -- 172.21.15.33:0/2386739021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 msgr2=0x7f61c0067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61bffff700 1 --2- 172.21.15.33:0/2386739021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c0067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61bffff700 1 -- 172.21.15.33:0/2386739021 >> v1:172.21.15.33:6789/0 conn(0x7f61c0068090 legacy=0x7f61c006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61bffff700 1 -- 172.21.15.33:0/2386739021 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c006d660 con 0x7f61c006cdd0 2023-10-06T05:35:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61be7fc700 1 -- 172.21.15.33:0/2386739021 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209234852 0 0) 0x7f61c0068500 con 0x7f61c0068090 2023-10-06T05:35:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61bffff700 1 --2- 172.21.15.33:0/2386739021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c006cdd0 0x7f61c00c9850 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009f20 tx=0x7f61b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e885202557e023 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61be7fc700 1 -- 172.21.15.33:0/2386739021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800e070 con 0x7f61c006cdd0 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61be7fc700 1 -- 172.21.15.33:0/2386739021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b800b9d0 con 0x7f61c006cdd0 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.990+0000 7f61be7fc700 1 -- 172.21.15.33:0/2386739021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80092e0 con 0x7f61c006cdd0 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 -- 172.21.15.33:0/2386739021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c006cdd0 msgr2=0x7f61c00c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/2386739021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c006cdd0 0x7f61c00c9850 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009f20 tx=0x7f61b8009840 comp rx=0 tx=0).stop 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 -- 172.21.15.33:0/2386739021 shutdown_connections 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/2386739021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c006cdd0 0x7f61c00c9850 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/2386739021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c0067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 -- 172.21.15.33:0/2386739021 >> 172.21.15.33:0/2386739021 conn(0x7f61c006ebf0 msgr2=0x7f61c006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 -- 172.21.15.33:0/2386739021 shutdown_connections 2023-10-06T05:35:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.991+0000 7f61c4b9d700 1 -- 172.21.15.33:0/2386739021 wait complete. 2023-10-06T05:35:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.992+0000 7f61c4b9d700 1 Processor -- start 2023-10-06T05:35:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.992+0000 7f61c4b9d700 1 -- start start 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.993+0000 7f61c4b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 0x7f61c0137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.993+0000 7f61c4b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c006cdd0 0x7f61c00c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.993+0000 7f61c4b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.993+0000 7f61c4b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0137b70 con 0x7f61c0068090 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.993+0000 7f61c4b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00c37c0 con 0x7f61c006cdd0 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.993+0000 7f61c4b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00c4620 con 0x7f61c01559a0 2023-10-06T05:35:32.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 0x7f61c0137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 0x7f61c0137630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44680/0 (socket says 172.21.15.33:44680) 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 -- 172.21.15.33:0/1484656833 learned_addr learned my addr 172.21.15.33:0/1484656833 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bffff700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 -- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 msgr2=0x7f61c00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61beffd700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c006cdd0 0x7f61c00c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 -- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c006cdd0 msgr2=0x7f61c00c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c006cdd0 0x7f61c00c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 -- 172.21.15.33:0/1484656833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8009ae0 con 0x7f61c0068090 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bffff700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:32.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61beffd700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c006cdd0 0x7f61c00c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.994+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 0x7f61c0137630 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e960 tx=0x7f61b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.995+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b000c850 con 0x7f61c0068090 2023-10-06T05:35:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.995+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b000c9b0 con 0x7f61c0068090 2023-10-06T05:35:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.995+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b000cd50 con 0x7f61c0068090 2023-10-06T05:35:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.995+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c00c3a10 con 0x7f61c0068090 2023-10-06T05:35:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.995+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c00c3fd0 con 0x7f61c0068090 2023-10-06T05:35:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.996+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c00c4900 con 0x7f61c0068090 2023-10-06T05:35:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.997+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61b0013680 con 0x7f61c0068090 2023-10-06T05:35:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.998+0000 7f61a77fe700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61ac06b340 0x7f61ac06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.998+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(284..284 src has 1..284) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f61b00849e0 con 0x7f61c0068090 2023-10-06T05:35:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.998+0000 7f61beffd700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61ac06b340 0x7f61ac06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:33.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:32.999+0000 7f61beffd700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61ac06b340 0x7f61ac06d7f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f61b40011e0 tx=0x7f61b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.002+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61b00519d0 con 0x7f61c0068090 2023-10-06T05:35:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.204+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61c0063ff0 con 0x7f61c0068090 2023-10-06T05:35:33.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.980+0000 7f61a77fe700 1 -- 172.21.15.33:0/1484656833 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v285) v1 ==== 181+0+0 (secure 0 0 0) 0x7f61b0054ff0 con 0x7f61c0068090 2023-10-06T05:35:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61ac06b340 msgr2=0x7f61ac06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61ac06b340 0x7f61ac06d7f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f61b40011e0 tx=0x7f61b4009040 comp rx=0 tx=0).stop 2023-10-06T05:35:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 msgr2=0x7f61c0137630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 0x7f61c0137630 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e960 tx=0x7f61b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 shutdown_connections 2023-10-06T05:35:33.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f61ac06b340 0x7f61ac06d7f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61c01559a0 0x7f61c00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0068090 0x7f61c0137630 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 --2- 172.21.15.33:0/1484656833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61c006cdd0 0x7f61c00c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 >> 172.21.15.33:0/1484656833 conn(0x7f61c006ebf0 msgr2=0x7f61c00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.985+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 shutdown_connections 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:33.986+0000 7f61c4b9d700 1 -- 172.21.15.33:0/1484656833 wait complete. 2023-10-06T05:35:33.988 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:35:34.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:35:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.212+0000 7fb34ad9d700 1 Processor -- start 2023-10-06T05:35:34.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.212+0000 7fb34ad9d700 1 -- start start 2023-10-06T05:35:34.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.213+0000 7fb34ad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 0x7fb3440c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.213+0000 7fb34ad9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440c7fa0 con 0x7fb344155890 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb34ad9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440c7390 con 0x7fb3440c6660 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb34ad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c7670 con 0x7fb34406e3e0 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb349d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb344155890 0x7fb34406dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52312/0 (socket says 172.21.15.33:52312) 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb349d9b700 1 -- 172.21.15.33:0/573483700 learned_addr learned my addr 172.21.15.33:0/573483700 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb34959a700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 0x7fb3440c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990946155 0 0) 0x7fb3440c7fa0 con 0x7fb344155890 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb330004680 con 0x7fb344155890 2023-10-06T05:35:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855788004 0 0) 0x7fb330004680 con 0x7fb344155890 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 msgr2=0x7fb3440c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 0x7fb3440c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 >> v1:172.21.15.167:6789/0 conn(0x7fb3440c6660 legacy=0x7fb3440c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440c7950 con 0x7fb344155890 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.214+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fb340001470 con 0x7fb344155890 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 >> v1:172.21.15.33:6789/0 conn(0x7fb344155890 legacy=0x7fb34406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 0x7fb330004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3300050e0 0x7fb330005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb3300062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c7670 con 0x7fb330005e60 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33000cc50 con 0x7fb3300050e0 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb330003510 con 0x7fb34406e3e0 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34a59c700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb3300062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34959a700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3300050e0 0x7fb330005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb349d9b700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 0x7fb330004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34a59c700 1 -- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 msgr2=0x7fb330004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34a59c700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb34406e3e0 0x7fb330004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34a59c700 1 -- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3300050e0 msgr2=0x7fb330005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34a59c700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3300050e0 0x7fb330005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:34.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.215+0000 7fb34a59c700 1 -- 172.21.15.33:0/573483700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330004680 con 0x7fb330005e60 2023-10-06T05:35:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.216+0000 7fb34a59c700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb3300062d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fb33c002170 tx=0x7fb33c0017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cce42b0bfc1f47e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.216+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c001ae0 con 0x7fb330005e60 2023-10-06T05:35:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.216+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c00c940 con 0x7fb330005e60 2023-10-06T05:35:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.216+0000 7fb348d99700 1 -- 172.21.15.33:0/573483700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00cce0 con 0x7fb330005e60 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34ad9d700 1 -- 172.21.15.33:0/573483700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 msgr2=0x7fb3300062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/573483700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb3300062d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fb33c002170 tx=0x7fb33c0017d0 comp rx=0 tx=0).stop 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34a59c700 1 -- 172.21.15.33:0/573483700 reap_dead start 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34ad9d700 1 -- 172.21.15.33:0/573483700 shutdown_connections 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34ad9d700 1 -- 172.21.15.33:0/573483700 >> 172.21.15.33:0/573483700 conn(0x7fb34406cfe0 msgr2=0x7fb3440cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34ad9d700 1 -- 172.21.15.33:0/573483700 shutdown_connections 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.217+0000 7fb34ad9d700 1 -- 172.21.15.33:0/573483700 wait complete. 2023-10-06T05:35:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.218+0000 7fb34ad9d700 1 Processor -- start 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.218+0000 7fb34ad9d700 1 -- start start 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb34ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb344158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb34ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb344155d80 0x7fb344156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb34ad9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344156990 0x7fb344156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb34ad9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344165a80 con 0x7fb330005e60 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb34ad9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344165150 con 0x7fb344155d80 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb34ad9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb344165430 con 0x7fb344156990 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb349d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb344158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb349d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb344158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44694/0 (socket says 172.21.15.33:44694) 2023-10-06T05:35:34.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.219+0000 7fb349d9b700 1 -- 172.21.15.33:0/860902870 learned_addr learned my addr 172.21.15.33:0/860902870 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb34959a700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb344155d80 0x7fb344156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb34a59c700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344156990 0x7fb344156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb349d9b700 1 -- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344156990 msgr2=0x7fb344156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb349d9b700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344156990 0x7fb344156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb349d9b700 1 -- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb344155d80 msgr2=0x7fb344156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb349d9b700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb344155d80 0x7fb344156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb349d9b700 1 -- 172.21.15.33:0/860902870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344165710 con 0x7fb330005e60 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb34959a700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb344155d80 0x7fb344156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb349d9b700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb344158700 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fb340007390 tx=0x7fb340001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340018030 con 0x7fb330005e60 2023-10-06T05:35:34.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340007800 con 0x7fb330005e60 2023-10-06T05:35:34.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb344165f10 con 0x7fb330005e60 2023-10-06T05:35:34.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34000fb00 con 0x7fb330005e60 2023-10-06T05:35:34.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.220+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb344158c40 con 0x7fb330005e60 2023-10-06T05:35:34.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.221+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34404c380 con 0x7fb330005e60 2023-10-06T05:35:34.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.222+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb34001b070 con 0x7fb330005e60 2023-10-06T05:35:34.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.223+0000 7fb339ffb700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb328066f30 0x7fb3280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:34.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.223+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(285..285 src has 1..285) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fb340019070 con 0x7fb330005e60 2023-10-06T05:35:34.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.224+0000 7fb34959a700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb328066f30 0x7fb3280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:34.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.224+0000 7fb34959a700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb328066f30 0x7fb3280693e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb34406ec80 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.227+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb340051590 con 0x7fb330005e60 2023-10-06T05:35:34.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.436+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb344165d60 con 0x7fb330005e60 2023-10-06T05:35:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:34.995+0000 7fb339ffb700 1 -- 172.21.15.33:0/860902870 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v286) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb340054bb0 con 0x7fb330005e60 2023-10-06T05:35:35.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.000+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb328066f30 msgr2=0x7fb3280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.000+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb328066f30 0x7fb3280693e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb34406ec80 tx=0x7fb334008040 comp rx=0 tx=0).stop 2023-10-06T05:35:35.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.000+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 msgr2=0x7fb344158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.000+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb344158700 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fb340007390 tx=0x7fb340001df0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.000+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 shutdown_connections 2023-10-06T05:35:35.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fb328066f30 0x7fb3280693e0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344156990 0x7fb344156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb330005e60 0x7fb344158700 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 --2- 172.21.15.33:0/860902870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb344155d80 0x7fb344156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 >> 172.21.15.33:0/860902870 conn(0x7fb34406cfe0 msgr2=0x7fb344153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:35.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 shutdown_connections 2023-10-06T05:35:35.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.001+0000 7fb34ad9d700 1 -- 172.21.15.33:0/860902870 wait complete. 2023-10-06T05:35:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T05:35:35.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:35:35.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.217+0000 7f8c96d9d700 1 Processor -- start 2023-10-06T05:35:35.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.217+0000 7f8c96d9d700 1 -- start start 2023-10-06T05:35:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c96d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c90155e20 0x7f8c90160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c96d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c900b0230 con 0x7f8c90155a50 2023-10-06T05:35:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c96d9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c900b06b0 con 0x7f8c900c9490 2023-10-06T05:35:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c96d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c900cba90 con 0x7f8c90155e20 2023-10-06T05:35:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c95d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8c90155a50 0x7f8c900af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52314/0 (socket says 172.21.15.33:52314) 2023-10-06T05:35:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c95d9b700 1 -- 172.21.15.33:0/1528931161 learned_addr learned my addr 172.21.15.33:0/1528931161 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c9659c700 1 --2- 172.21.15.33:0/1528931161 >> v2:172.21.15.33:3300/0 conn(0x7f8c90155e20 0x7f8c90160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c9659c700 1 -- 172.21.15.33:0/1528931161 >> v1:172.21.15.167:6789/0 conn(0x7f8c900c9490 legacy=0x7f8c900afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.219+0000 7f8c94d99700 1 -- 172.21.15.33:0/1528931161 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306587040 0 0) 0x7f8c900b0230 con 0x7f8c90155a50 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.220+0000 7f8c9659c700 1 -- 172.21.15.33:0/1528931161 >> v1:172.21.15.33:6789/0 conn(0x7f8c90155a50 legacy=0x7f8c900af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.220+0000 7f8c9659c700 1 -- 172.21.15.33:0/1528931161 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90156260 con 0x7f8c90155e20 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.220+0000 7f8c9659c700 1 --2- 172.21.15.33:0/1528931161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c90155e20 0x7f8c90160ae0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009e60 tx=0x7f8c88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1965a9e45d536541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.220+0000 7f8c94d99700 1 -- 172.21.15.33:0/1528931161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800bc30 con 0x7f8c90155e20 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.220+0000 7f8c94d99700 1 -- 172.21.15.33:0/1528931161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8800bd90 con 0x7f8c90155e20 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.220+0000 7f8c94d99700 1 -- 172.21.15.33:0/1528931161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88011650 con 0x7f8c90155e20 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 -- 172.21.15.33:0/1528931161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c90155e20 msgr2=0x7f8c90160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/1528931161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c90155e20 0x7f8c90160ae0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009e60 tx=0x7f8c88009780 comp rx=0 tx=0).stop 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 -- 172.21.15.33:0/1528931161 shutdown_connections 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/1528931161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c90155e20 0x7f8c90160ae0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 -- 172.21.15.33:0/1528931161 >> 172.21.15.33:0/1528931161 conn(0x7f8c9006d030 msgr2=0x7f8c900cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 -- 172.21.15.33:0/1528931161 shutdown_connections 2023-10-06T05:35:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.221+0000 7f8c96d9d700 1 -- 172.21.15.33:0/1528931161 wait complete. 2023-10-06T05:35:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.222+0000 7f8c96d9d700 1 Processor -- start 2023-10-06T05:35:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.222+0000 7f8c96d9d700 1 -- start start 2023-10-06T05:35:35.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c96d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 0x7f8c90157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c96d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c90155a50 0x7f8c90146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c96d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c90155e20 0x7f8c900c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c96d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9014e780 con 0x7f8c900c9490 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c96d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900c58d0 con 0x7f8c90155e20 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c96d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c901472f0 con 0x7f8c90155a50 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.223+0000 7f8c95d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 0x7f8c90157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 0x7f8c90157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44702/0 (socket says 172.21.15.33:44702) 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 -- 172.21.15.33:0/2597967055 learned_addr learned my addr 172.21.15.33:0/2597967055 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c9659c700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c90155e20 0x7f8c900c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c9559a700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c90155a50 0x7f8c90146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 -- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c90155a50 msgr2=0x7f8c90146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c90155a50 0x7f8c90146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 -- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c90155e20 msgr2=0x7f8c900c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c90155e20 0x7f8c900c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 -- 172.21.15.33:0/2597967055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88009a20 con 0x7f8c900c9490 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c9659c700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c90155e20 0x7f8c900c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.224+0000 7f8c95d9b700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 0x7f8c90157ae0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00c7e0 tx=0x7f8c8c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.225+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c005b30 con 0x7f8c900c9490 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.225+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c005c90 con 0x7f8c900c9490 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.225+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c901581e0 con 0x7f8c900c9490 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.225+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c007300 con 0x7f8c900c9490 2023-10-06T05:35:35.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.225+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900c5280 con 0x7f8c900c9490 2023-10-06T05:35:35.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.226+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9004c380 con 0x7f8c900c9490 2023-10-06T05:35:35.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.227+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c8c0076b0 con 0x7f8c900c9490 2023-10-06T05:35:35.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.228+0000 7f8c85ffb700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c74066f30 0x7f8c740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.228+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8c8c083f70 con 0x7f8c900c9490 2023-10-06T05:35:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.228+0000 7f8c9559a700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c74066f30 0x7f8c740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.229+0000 7f8c9559a700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c74066f30 0x7f8c740693e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8c900c8580 tx=0x7f8c80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:35.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.232+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c8c051330 con 0x7f8c900c9490 2023-10-06T05:35:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.434+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8c900c99e0 con 0x7f8c900c9490 2023-10-06T05:35:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.438+0000 7f8c85ffb700 1 -- 172.21.15.33:0/2597967055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c8c054950 con 0x7f8c900c9490 2023-10-06T05:35:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c74066f30 msgr2=0x7f8c740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c74066f30 0x7f8c740693e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8c900c8580 tx=0x7f8c80009040 comp rx=0 tx=0).stop 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 msgr2=0x7f8c90157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 0x7f8c90157ae0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00c7e0 tx=0x7f8c8c007df0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 shutdown_connections 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8c74066f30 0x7f8c740693e0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c90155e20 0x7f8c900c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c900c9490 0x7f8c90157ae0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 --2- 172.21.15.33:0/2597967055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8c90155a50 0x7f8c90146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 >> 172.21.15.33:0/2597967055 conn(0x7f8c9006d030 msgr2=0x7f8c90151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 shutdown_connections 2023-10-06T05:35:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.441+0000 7f8c96d9d700 1 -- 172.21.15.33:0/2597967055 wait complete. 2023-10-06T05:35:35.460 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_unmount_for_evicted_client' 2023-10-06T05:35:35.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.645+0000 7f6c7c99d700 1 Processor -- start 2023-10-06T05:35:35.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.646+0000 7f6c7c99d700 1 -- start start 2023-10-06T05:35:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c7c99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c7813faa0 0x7f6c7813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c7c99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c7806aad0 0x7f6c7806af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c7c99d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780c9cb0 con 0x7f6c780c7940 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c7c99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780ca270 con 0x7f6c7813faa0 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c7c99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7806b840 con 0x7f6c7806aad0 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c777fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6c7813faa0 0x7f6c7813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c76ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c780c7940 0x7f6c780c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52328/0 (socket says 172.21.15.33:52328) 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c76ffd700 1 -- 172.21.15.33:0/1488651026 learned_addr learned my addr 172.21.15.33:0/1488651026 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.647+0000 7f6c77fff700 1 --2- 172.21.15.33:0/1488651026 >> v2:172.21.15.167:3300/0 conn(0x7f6c7806aad0 0x7f6c7806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c777fe700 1 -- 172.21.15.33:0/1488651026 >> v2:172.21.15.167:3300/0 conn(0x7f6c7806aad0 msgr2=0x7f6c7806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c777fe700 1 --2- 172.21.15.33:0/1488651026 >> v2:172.21.15.167:3300/0 conn(0x7f6c7806aad0 0x7f6c7806af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c777fe700 1 -- 172.21.15.33:0/1488651026 >> v1:172.21.15.33:6789/0 conn(0x7f6c780c7940 legacy=0x7f6c780c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c777fe700 1 -- 172.21.15.33:0/1488651026 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7806bb20 con 0x7f6c7813faa0 2023-10-06T05:35:35.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c767fc700 1 -- 172.21.15.33:0/1488651026 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258793119 0 0) 0x7f6c780c9cb0 con 0x7f6c780c7940 2023-10-06T05:35:35.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c777fe700 1 --2- 172.21.15.33:0/1488651026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7813faa0 0x7f6c7813fe70 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f6c78156b90 tx=0x7f6c680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0fdf1007be03c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c767fc700 1 -- 172.21.15.33:0/1488651026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800b6b0 con 0x7f6c7813faa0 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c767fc700 1 -- 172.21.15.33:0/1488651026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6800b810 con 0x7f6c7813faa0 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.648+0000 7f6c767fc700 1 -- 172.21.15.33:0/1488651026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800bbb0 con 0x7f6c7813faa0 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1488651026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7813faa0 msgr2=0x7f6c7813fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1488651026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7813faa0 0x7f6c7813fe70 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f6c78156b90 tx=0x7f6c680091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1488651026 shutdown_connections 2023-10-06T05:35:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1488651026 >> v2:172.21.15.167:3300/0 conn(0x7f6c7806aad0 0x7f6c7806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1488651026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7813faa0 0x7f6c7813fe70 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1488651026 >> 172.21.15.33:0/1488651026 conn(0x7f6c7806d030 msgr2=0x7f6c7806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1488651026 shutdown_connections 2023-10-06T05:35:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.649+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1488651026 wait complete. 2023-10-06T05:35:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.650+0000 7f6c7c99d700 1 Processor -- start 2023-10-06T05:35:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 -- start start 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7806aad0 0x7f6c781562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 0x7f6c780b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7813faa0 0x7f6c780c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78156830 con 0x7f6c780c7940 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b1510 con 0x7f6c7813faa0 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.651+0000 7f6c7c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780af4c0 con 0x7f6c7806aad0 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 0x7f6c780b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 0x7f6c780b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44712/0 (socket says 172.21.15.33:44712) 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 -- 172.21.15.33:0/1198773214 learned_addr learned my addr 172.21.15.33:0/1198773214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c77fff700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7813faa0 0x7f6c780c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c777fe700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7806aad0 0x7f6c781562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 -- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7806aad0 msgr2=0x7f6c781562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7806aad0 0x7f6c781562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 -- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7813faa0 msgr2=0x7f6c780c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7813faa0 0x7f6c780c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 -- 172.21.15.33:0/1198773214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68009420 con 0x7f6c780c7940 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.652+0000 7f6c76ffd700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 0x7f6c780b0fd0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c810 tx=0x7f6c6c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.653+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c005b30 con 0x7f6c780c7940 2023-10-06T05:35:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.653+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c005c90 con 0x7f6c780c7940 2023-10-06T05:35:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.653+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c007300 con 0x7f6c780c7940 2023-10-06T05:35:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.653+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780b1700 con 0x7f6c780c7940 2023-10-06T05:35:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.653+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c780af150 con 0x7f6c780c7940 2023-10-06T05:35:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.654+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c780aefb0 con 0x7f6c780c7940 2023-10-06T05:35:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.655+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c6c01b400 con 0x7f6c780c7940 2023-10-06T05:35:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.655+0000 7f6c5f7fe700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c64066f30 0x7f6c640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.656+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c6c083f30 con 0x7f6c780c7940 2023-10-06T05:35:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.656+0000 7f6c777fe700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c64066f30 0x7f6c640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.657+0000 7f6c777fe700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c64066f30 0x7f6c640693e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6c680093f0 tx=0x7f6c68017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.660+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c780aefb0 con 0x7f6c780c7940 2023-10-06T05:35:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.940+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_unmount_for_evicted_client"]} v 0) v1 -- 0x7f6c780aefb0 con 0x7f6c780c7940 2023-10-06T05:35:36.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:35.998+0000 7f6c5f7fe700 1 -- 172.21.15.33:0/1198773214 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientRecovery.test_unmount_for_evicted_client"]}]=0 v1037) v1 ==== 163+0+0 (secure 0 0 0) 0x7f6c780aefb0 con 0x7f6c780c7940 2023-10-06T05:35:36.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c64066f30 msgr2=0x7f6c640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c64066f30 0x7f6c640693e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6c680093f0 tx=0x7f6c68017000 comp rx=0 tx=0).stop 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 msgr2=0x7f6c780b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 0x7f6c780b0fd0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c810 tx=0x7f6c6c007df0 comp rx=0 tx=0).stop 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 shutdown_connections 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6c64066f30 0x7f6c640693e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c7813faa0 0x7f6c780c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c7806aad0 0x7f6c781562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:36.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 --2- 172.21.15.33:0/1198773214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c780c7940 0x7f6c780b0fd0 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 >> 172.21.15.33:0/1198773214 conn(0x7f6c7806d030 msgr2=0x7f6c780cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 shutdown_connections 2023-10-06T05:35:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:36.001+0000 7f6c7c99d700 1 -- 172.21.15.33:0/1198773214 wait complete. 2023-10-06T05:35:36.020 INFO:tasks.cephfs_test_runner:Test if client hangs on unmount after evicting the client. ... ok 2023-10-06T05:35:36.020 INFO:tasks.cephfs_test_runner: 2023-10-06T05:35:36.020 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T05:35:36.021 INFO:tasks.cephfs_test_runner:Ran 20 tests in 1231.912s 2023-10-06T05:35:36.021 INFO:tasks.cephfs_test_runner: 2023-10-06T05:35:36.021 INFO:tasks.cephfs_test_runner:OK (skipped=4) 2023-10-06T05:35:36.021 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T05:35:36.030 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T05:35:36.095 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T05:35:36.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.096 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:35:36.132 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 2) 2023-10-06T05:35:36.132 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:35:36.132 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:35:36.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.133 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:35:36.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:35:36.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:35:36.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-10-06T05:35:36.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.529 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T05:35:36.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-10-06T05:35:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2023-10-06T05:35:36.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:35:36.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.679 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:35:36.720 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:35:36.720 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:35:36.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.720 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:35:36.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:35:36.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:36.902 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2023-10-06T05:35:37.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:37.036 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T05:35:37.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:35:37.075 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2023-10-06T05:35:37.180 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2023-10-06T05:35:37.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:35:37.183 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T05:35:37.193 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T05:35:37.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.382+0000 7f751afff700 1 Processor -- start 2023-10-06T05:35:37.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.383+0000 7f751afff700 1 -- start start 2023-10-06T05:35:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f751afff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140c9e10 0x7f75140ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f751afff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f75140af000 0x7f75140af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f751afff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75140ca8d0 con 0x7f751413fb70 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f751afff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75140af9a0 con 0x7f75140af000 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f751afff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75140cabb0 con 0x7f75140c9e10 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f7518d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f751413fb70 0x7f751406b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52342/0 (socket says 172.21.15.33:52342) 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f7518d9b700 1 -- 172.21.15.33:0/3222258375 learned_addr learned my addr 172.21.15.33:0/3222258375 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.384+0000 7f751959c700 1 --2- 172.21.15.33:0/3222258375 >> v2:172.21.15.33:3300/0 conn(0x7f75140af000 0x7f75140af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f75137fe700 1 -- 172.21.15.33:0/3222258375 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501737395 0 0) 0x7f75140ca8d0 con 0x7f751413fb70 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f75137fe700 1 -- 172.21.15.33:0/3222258375 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7504003570 con 0x7f751413fb70 2023-10-06T05:35:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f751959c700 1 -- 172.21.15.33:0/3222258375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140c9e10 msgr2=0x7f75140ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f751959c700 1 --2- 172.21.15.33:0/3222258375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140c9e10 0x7f75140ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f751959c700 1 -- 172.21.15.33:0/3222258375 >> v1:172.21.15.33:6789/0 conn(0x7f751413fb70 legacy=0x7f751406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f751959c700 1 -- 172.21.15.33:0/3222258375 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751406ad00 con 0x7f75140af000 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f751959c700 1 --2- 172.21.15.33:0/3222258375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140af000 0x7f75140af460 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f750c009e60 tx=0x7f750c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dca955547069a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f75137fe700 1 -- 172.21.15.33:0/3222258375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00e070 con 0x7f75140af000 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f75137fe700 1 -- 172.21.15.33:0/3222258375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750c00bb60 con 0x7f75140af000 2023-10-06T05:35:37.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.385+0000 7f75137fe700 1 -- 172.21.15.33:0/3222258375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c012430 con 0x7f75140af000 2023-10-06T05:35:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 -- 172.21.15.33:0/3222258375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140af000 msgr2=0x7f75140af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 --2- 172.21.15.33:0/3222258375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140af000 0x7f75140af460 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f750c009e60 tx=0x7f750c009780 comp rx=0 tx=0).stop 2023-10-06T05:35:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 -- 172.21.15.33:0/3222258375 shutdown_connections 2023-10-06T05:35:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 --2- 172.21.15.33:0/3222258375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140af000 0x7f75140af460 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 --2- 172.21.15.33:0/3222258375 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140c9e10 0x7f75140ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 -- 172.21.15.33:0/3222258375 >> 172.21.15.33:0/3222258375 conn(0x7f75140675f0 msgr2=0x7f75140679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 -- 172.21.15.33:0/3222258375 shutdown_connections 2023-10-06T05:35:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.386+0000 7f751afff700 1 -- 172.21.15.33:0/3222258375 wait complete. 2023-10-06T05:35:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.388+0000 7f751afff700 1 Processor -- start 2023-10-06T05:35:37.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.388+0000 7f751afff700 1 -- start start 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751afff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140af000 0x7f75140c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751afff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 0x7f751413df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751afff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f751413fb70 0x7f75140adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751afff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75140c5c10 con 0x7f75140c9e10 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751afff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751416ccf0 con 0x7f751413fb70 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751afff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751416cfd0 con 0x7f75140af000 2023-10-06T05:35:37.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 0x7f751413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7518d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140af000 0x7f75140c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 0x7f751413df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44728/0 (socket says 172.21.15.33:44728) 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 -- 172.21.15.33:0/1320912714 learned_addr learned my addr 172.21.15.33:0/1320912714 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f751959c700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f751413fb70 0x7f75140adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 -- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140af000 msgr2=0x7f75140c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140af000 0x7f75140c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 -- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f751413fb70 msgr2=0x7f75140adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f751413fb70 0x7f75140adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.389+0000 7f7513fff700 1 -- 172.21.15.33:0/1320912714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c009a20 con 0x7f75140c9e10 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.390+0000 7f7513fff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 0x7f751413df20 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f750000b810 tx=0x7f7500009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.390+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000f510 con 0x7f75140c9e10 2023-10-06T05:35:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.390+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000f690 con 0x7f75140c9e10 2023-10-06T05:35:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.390+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000fa50 con 0x7f75140c9e10 2023-10-06T05:35:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.390+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751416d2b0 con 0x7f75140c9e10 2023-10-06T05:35:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.390+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751416c9e0 con 0x7f75140c9e10 2023-10-06T05:35:37.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.391+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514063ff0 con 0x7f75140c9e10 2023-10-06T05:35:37.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.392+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f750001a400 con 0x7f75140c9e10 2023-10-06T05:35:37.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.393+0000 7f7510ff9700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74f8066f30 0x7f74f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.393+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7500083fb0 con 0x7f75140c9e10 2023-10-06T05:35:37.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.393+0000 7f7518d9b700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74f8066f30 0x7f74f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.394+0000 7f7518d9b700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74f8066f30 0x7f74f80693e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7508007fd0 tx=0x7f7508009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:37.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.397+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75000512f0 con 0x7f75140c9e10 2023-10-06T05:35:37.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.603+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75140c3000 con 0x7f75140c9e10 2023-10-06T05:35:37.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.604+0000 7f7510ff9700 1 -- 172.21.15.33:0/1320912714 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v286) v1 ==== 74+0+13853 (secure 0 0 0) 0x7f7500054910 con 0x7f75140c9e10 2023-10-06T05:35:37.606 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:37.606 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":286,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:34.986539+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/3559384937":"2023-10-07T05:35:31.767811+0000","172.21.15.167:6834/3559384937":"2023-10-07T05:35:31.767811+0000","172.21.15.33:0/2784413487":"2023-10-06T06:35:25.570337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74f8066f30 msgr2=0x7f74f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74f8066f30 0x7f74f80693e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7508007fd0 tx=0x7f7508009040 comp rx=0 tx=0).stop 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 msgr2=0x7f751413df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 0x7f751413df20 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f750000b810 tx=0x7f7500009250 comp rx=0 tx=0).stop 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 shutdown_connections 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f74f8066f30 0x7f74f80693e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f751413fb70 0x7f75140adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75140af000 0x7f75140c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 --2- 172.21.15.33:0/1320912714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75140c9e10 0x7f751413df20 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 >> 172.21.15.33:0/1320912714 conn(0x7f75140675f0 msgr2=0x7f75140c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 shutdown_connections 2023-10-06T05:35:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.607+0000 7f751afff700 1 -- 172.21.15.33:0/1320912714 wait complete. 2023-10-06T05:35:37.626 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:35:37.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:35:37.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.811+0000 7f4e0c99d700 1 Processor -- start 2023-10-06T05:35:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.811+0000 7f4e0c99d700 1 -- start start 2023-10-06T05:35:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e0c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e0813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e0c99d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e080c7940 0x7f4e080c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e0c99d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4e0806aad0 0x7f4e0806af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e0c99d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e080c9cb0 con 0x7f4e080c7940 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e0c99d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e080ca270 con 0x7f4e0806aad0 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e0c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0806b840 con 0x7f4e0813faa0 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e06ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e080c7940 0x7f4e080c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.812+0000 7f4e06ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e080c7940 0x7f4e080c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44732/0 (socket says 172.21.15.33:44732) 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 -- 172.21.15.33:0/1247349826 learned_addr learned my addr 172.21.15.33:0/1247349826 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e077fe700 1 --2- 172.21.15.33:0/1247349826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e0813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e07fff700 1 --2- 172.21.15.33:0/1247349826 >> v2:172.21.15.167:3300/0 conn(0x7f4e0806aad0 0x7f4e0806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 -- 172.21.15.33:0/1247349826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 msgr2=0x7f4e0813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/1247349826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e0813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 -- 172.21.15.33:0/1247349826 >> v2:172.21.15.167:3300/0 conn(0x7f4e0806aad0 msgr2=0x7f4e0806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/1247349826 >> v2:172.21.15.167:3300/0 conn(0x7f4e0806aad0 0x7f4e0806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 -- 172.21.15.33:0/1247349826 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0806bb20 con 0x7f4e080c7940 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e07fff700 1 --2- 172.21.15.33:0/1247349826 >> v2:172.21.15.167:3300/0 conn(0x7f4e0806aad0 0x7f4e0806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/1247349826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080c7940 0x7f4e080c7db0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f4e080cc0b0 tx=0x7f4dfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eebf777cfd9aab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:37.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e067fc700 1 -- 172.21.15.33:0/1247349826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc009e70 con 0x7f4e080c7940 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e067fc700 1 -- 172.21.15.33:0/1247349826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc00b860 con 0x7f4e080c7940 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.813+0000 7f4e067fc700 1 -- 172.21.15.33:0/1247349826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00bc00 con 0x7f4e080c7940 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 -- 172.21.15.33:0/1247349826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080c7940 msgr2=0x7f4e080c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/1247349826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080c7940 0x7f4e080c7db0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f4e080cc0b0 tx=0x7f4dfc009250 comp rx=0 tx=0).stop 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 -- 172.21.15.33:0/1247349826 shutdown_connections 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/1247349826 >> v2:172.21.15.167:3300/0 conn(0x7f4e0806aad0 0x7f4e0806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/1247349826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e0813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/1247349826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080c7940 0x7f4e080c7db0 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 -- 172.21.15.33:0/1247349826 >> 172.21.15.33:0/1247349826 conn(0x7f4e0806d030 msgr2=0x7f4e0806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 -- 172.21.15.33:0/1247349826 shutdown_connections 2023-10-06T05:35:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.814+0000 7f4e0c99d700 1 -- 172.21.15.33:0/1247349826 wait complete. 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.816+0000 7f4e0c99d700 1 Processor -- start 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.816+0000 7f4e0c99d700 1 -- start start 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e0c99d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 0x7f4e0815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e0c99d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e080c7940 0x7f4e0815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e0c99d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e08169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e0c99d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0816a870 con 0x7f4e0806aad0 2023-10-06T05:35:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e0c99d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0815a0c0 con 0x7f4e080c7940 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e0c99d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0815a3a0 con 0x7f4e0813faa0 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 0x7f4e0815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 0x7f4e0815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44744/0 (socket says 172.21.15.33:44744) 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 -- 172.21.15.33:0/4065318416 learned_addr learned my addr 172.21.15.33:0/4065318416 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e07fff700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e08169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e080c7940 0x7f4e0815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 -- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 msgr2=0x7f4e08169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e08169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 -- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e080c7940 msgr2=0x7f4e0815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e080c7940 0x7f4e0815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 -- 172.21.15.33:0/4065318416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0094f0 con 0x7f4e0806aad0 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e07fff700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e08169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e080c7940 0x7f4e0815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.817+0000 7f4e077fe700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 0x7f4e0815ebb0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f4df800e950 tx=0x7f4df80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.818+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800c850 con 0x7f4e0806aad0 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.818+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df800c9b0 con 0x7f4e0806aad0 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.818+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0815a680 con 0x7f4e0806aad0 2023-10-06T05:35:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.818+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800cdb0 con 0x7f4e0806aad0 2023-10-06T05:35:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.818+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0816a130 con 0x7f4e0806aad0 2023-10-06T05:35:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.819+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0804c380 con 0x7f4e0806aad0 2023-10-06T05:35:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.820+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4df8013680 con 0x7f4e0806aad0 2023-10-06T05:35:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.821+0000 7f4def7fe700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4df4066f30 0x7f4df40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.821+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4df8084690 con 0x7f4e0806aad0 2023-10-06T05:35:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.821+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4df4066f30 0x7f4df40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.822+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4df4066f30 0x7f4df40693e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4e080cc0b0 tx=0x7f4dfc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:37.825+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4df80519d0 con 0x7f4e0806aad0 2023-10-06T05:35:38.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.020+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e0815a230 con 0x7f4df4066f30 2023-10-06T05:35:38.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.023+0000 7f4def7fe700 1 -- 172.21.15.33:0/4065318416 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72675 (secure 0 0 0) 0x7f4e0815a230 con 0x7f4df4066f30 2023-10-06T05:35:38.026 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.025+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4df4066f30 msgr2=0x7f4df40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.025+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4df4066f30 0x7f4df40693e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4e080cc0b0 tx=0x7f4dfc017000 comp rx=0 tx=0).stop 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.025+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 msgr2=0x7f4e0815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.025+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 0x7f4e0815ebb0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f4df800e950 tx=0x7f4df80091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 shutdown_connections 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f4df4066f30 0x7f4df40693e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e0813faa0 0x7f4e08169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e0806aad0 0x7f4e0815ebb0 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 --2- 172.21.15.33:0/4065318416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e080c7940 0x7f4e0815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 >> 172.21.15.33:0/4065318416 conn(0x7f4e0806d030 msgr2=0x7f4e0806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 shutdown_connections 2023-10-06T05:35:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.026+0000 7f4e0c99d700 1 -- 172.21.15.33:0/4065318416 wait complete. 2023-10-06T05:35:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:35:38.045 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":921,"stamp":"2023-10-06T05:35:36.146183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2469044,"kb_used_data":29300,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752505676,"statfs":{"total":773094113280,"available":770565812224,"internally_reserved":0,"allocated":30003200,"data_stored":14721823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4063,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001615"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.252023+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:35:16.252023+0000","last_peered":"2023-10-06T05:35:16.252023+0000","last_clean":"2023-10-06T05:35:16.252023+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:16.252023+0000","last_undegraded":"2023-10-06T05:35:16.252023+0000","last_fullsized":"2023-10-06T05:35:16.252023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":227,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.253365+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:35:16.253365+0000","last_peered":"2023-10-06T05:35:16.253365+0000","last_clean":"2023-10-06T05:35:16.253365+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:16.253365+0000","last_undegraded":"2023-10-06T05:35:16.253365+0000","last_fullsized":"2023-10-06T05:35:16.253365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":283,"state":"active+clean","last_fresh":"2023-10-06T05:35:25.952151+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:35:25.952151+0000","last_peered":"2023-10-06T05:35:25.952151+0000","last_clean":"2023-10-06T05:35:25.952151+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:25.952151+0000","last_undegraded":"2023-10-06T05:35:25.952151+0000","last_fullsized":"2023-10-06T05:35:25.952151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":227,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.252902+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:35:16.252902+0000","last_peered":"2023-10-06T05:35:16.252902+0000","last_clean":"2023-10-06T05:35:16.252902+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:16.252902+0000","last_undegraded":"2023-10-06T05:35:16.252902+0000","last_fullsized":"2023-10-06T05:35:16.252902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":377,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.252687+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:35:16.252687+0000","last_peered":"2023-10-06T05:35:16.252687+0000","last_clean":"2023-10-06T05:35:16.252687+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:16.252687+0000","last_undegraded":"2023-10-06T05:35:16.252687+0000","last_fullsized":"2023-10-06T05:35:16.252687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":224,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.252590+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:35:16.252590+0000","last_peered":"2023-10-06T05:35:16.252590+0000","last_clean":"2023-10-06T05:35:16.252590+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:16.252590+0000","last_undegraded":"2023-10-06T05:35:16.252590+0000","last_fullsized":"2023-10-06T05:35:16.252590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":224,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.254032+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:35:16.254032+0000","last_peered":"2023-10-06T05:35:16.254032+0000","last_clean":"2023-10-06T05:35:16.254032+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:16.254032+0000","last_undegraded":"2023-10-06T05:35:16.254032+0000","last_fullsized":"2023-10-06T05:35:16.254032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":282,"state":"active+clean","last_fresh":"2023-10-06T05:35:16.252270+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:35:16.252270+0000","last_peered":"2023-10-06T05:35:16.252270+0000","last_clean":"2023-10-06T05:35:16.252270+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:16.252270+0000","last_undegraded":"2023-10-06T05:35:16.252270+0000","last_fullsized":"2023-10-06T05:35:16.252270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":270,"reported_epoch":283,"state":"active+clean","last_fresh":"2023-10-06T05:35:25.947692+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:35:25.947692+0000","last_peered":"2023-10-06T05:35:25.947692+0000","last_clean":"2023-10-06T05:35:25.947692+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:25.947692+0000","last_undegraded":"2023-10-06T05:35:25.947692+0000","last_fullsized":"2023-10-06T05:35:25.947692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575109,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309572,"kb_used_data":4092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062268,"statfs":{"total":96636764160,"available":96319762432,"internally_reserved":0,"allocated":4190208,"data_stored":2281270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.751},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.79000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575107,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309000,"kb_used_data":3520,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062840,"statfs":{"total":96636764160,"available":96320348160,"internally_reserved":0,"allocated":3604480,"data_stored":1697277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.47899999999999998,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.754}]},{"osd":1,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.70599999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.115},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.56000000000000005,"5min":0.504,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.022,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.54400000000000004,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":1.0129999999999999,"5min":1.143,"15min":1.143},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.038,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575108,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304904,"kb_used_data":3520,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066936,"statfs":{"total":96636764160,"available":96324542464,"internally_reserved":0,"allocated":3604480,"data_stored":1691458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":1.272,"15min":1.272},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.68000000000000005,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.72199999999999998,"15min":1.165},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":1.22,"15min":1.22},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":1.238,"15min":1.2430000000000001},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84999999999999998,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96299999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1080000000000001,"5min":1.492,"15min":1.492},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.262,"15min":1.2909999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.82999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.0609999999999999,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.85299999999999998,"15min":0.82199999999999995},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":0.92300000000000004,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.753}]}]},{"osd":0,"up_from":13,"seq":55834575108,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313676,"kb_used_data":4100,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058164,"statfs":{"total":96636764160,"available":96315559936,"internally_reserved":0,"allocated":4198400,"data_stored":2281894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.95299999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.307},"max":{"1min":0.628,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0980000000000001,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.94299999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.1379999999999999,"15min":1.27},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.49199999999999999},"max":{"1min":1.151,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.81499999999999995,"15min":0.80100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.147,"15min":1.153},"last":1.0589999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.496},"max":{"1min":1.0629999999999999,"5min":1.2,"15min":1.2010000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.90800000000000003,"5min":1.1459999999999999,"15min":1.274},"last":0.91500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575108,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309004,"kb_used_data":3524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062836,"statfs":{"total":96636764160,"available":96320344064,"internally_reserved":0,"allocated":3608576,"data_stored":1694127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.79600000000000004,"5min":0.82999999999999996,"15min":1.056},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57099999999999995,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.64000000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.58599999999999997,"5min":1.117,"15min":1.117},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95599999999999996,"5min":1.2709999999999999,"15min":1.486},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.089,"15min":1.5},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93899999999999995,"5min":1.157,"15min":1.1639999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92400000000000004,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.22,"15min":1.22},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.59699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.341,"15min":1.341},"last":0.628}]},{"osd":7,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":1.282,"15min":1.538},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.264,"15min":1.264},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575109,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308988,"kb_used_data":3508,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062852,"statfs":{"total":96636764160,"available":96320360448,"internally_reserved":0,"allocated":3592192,"data_stored":1690812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.85999999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.87}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.84699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575107,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308992,"kb_used_data":3512,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062848,"statfs":{"total":96636764160,"available":96320356352,"internally_reserved":0,"allocated":3596288,"data_stored":1690846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.751},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.77300000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.82199999999999995,"5min":1.026,"15min":1.026},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.755}]},{"osd":2,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.69499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41799999999999998},"max":{"1min":0.88600000000000001,"5min":1.099,"15min":1.099},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76200000000000001,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.443},"max":{"1min":0.85699999999999998,"5min":3.839,"15min":3.839},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.873,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":0.96499999999999997,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":0.93999999999999995,"5min":1.109,"15min":1.282},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.71699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575109,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304908,"kb_used_data":3524,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066932,"statfs":{"total":96636764160,"available":96324538368,"internally_reserved":0,"allocated":3608576,"data_stored":1694139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.372},"max":{"1min":1.071,"5min":1.115,"15min":1.2130000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.79700000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.55700000000000005,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":1.169,"15min":1.169},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.624},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.145},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.92800000000000005,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.02,"15min":1.1539999999999999},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.96899999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.52000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.60199999999999998,"5min":0.53300000000000003,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.0609999999999999},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:35:38.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:35:38.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.233+0000 7fd637fff700 1 Processor -- start 2023-10-06T05:35:38.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.233+0000 7fd637fff700 1 -- start start 2023-10-06T05:35:38.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.234+0000 7fd637fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380ccfc0 0x7fd6380cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.234+0000 7fd637fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380cbd80 con 0x7fd6380cc750 2023-10-06T05:35:38.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.234+0000 7fd637fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380af5f0 con 0x7fd6380cf3b0 2023-10-06T05:35:38.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.234+0000 7fd637fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380af8d0 con 0x7fd6380ccfc0 2023-10-06T05:35:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd6377fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd6380cc750 0x7fd6380b0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52358/0 (socket says 172.21.15.33:52358) 2023-10-06T05:35:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd6377fe700 1 -- 172.21.15.33:0/3967234186 learned_addr learned my addr 172.21.15.33:0/3967234186 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd636ffd700 1 --2- 172.21.15.33:0/3967234186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380ccfc0 0x7fd6380cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339460979 0 0) 0x7fd6380cbd80 con 0x7fd6380cc750 2023-10-06T05:35:38.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd620003570 con 0x7fd6380cc750 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756633220 0 0) 0x7fd6380af5f0 con 0x7fd6380cf3b0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6380cbd80 con 0x7fd6380cf3b0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd636ffd700 1 -- 172.21.15.33:0/3967234186 >> v1:172.21.15.167:6789/0 conn(0x7fd6380cf3b0 legacy=0x7fd6381594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd636ffd700 1 -- 172.21.15.33:0/3967234186 >> v1:172.21.15.33:6789/0 conn(0x7fd6380cc750 legacy=0x7fd6380b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.235+0000 7fd636ffd700 1 -- 172.21.15.33:0/3967234186 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380afbb0 con 0x7fd6380ccfc0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.236+0000 7fd636ffd700 1 --2- 172.21.15.33:0/3967234186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380ccfc0 0x7fd6380cc210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd628009910 tx=0x7fd6280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b052266c016b2145 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.236+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011630 con 0x7fd6380ccfc0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.236+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628011790 con 0x7fd6380ccfc0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.236+0000 7fd635ffb700 1 -- 172.21.15.33:0/3967234186 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011b30 con 0x7fd6380ccfc0 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.236+0000 7fd637fff700 1 -- 172.21.15.33:0/3967234186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380ccfc0 msgr2=0x7fd6380cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.236+0000 7fd637fff700 1 --2- 172.21.15.33:0/3967234186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380ccfc0 0x7fd6380cc210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd628009910 tx=0x7fd6280091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.237+0000 7fd637fff700 1 -- 172.21.15.33:0/3967234186 shutdown_connections 2023-10-06T05:35:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.237+0000 7fd637fff700 1 --2- 172.21.15.33:0/3967234186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380ccfc0 0x7fd6380cc210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.237+0000 7fd637fff700 1 -- 172.21.15.33:0/3967234186 >> 172.21.15.33:0/3967234186 conn(0x7fd63806cff0 msgr2=0x7fd63806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.237+0000 7fd637fff700 1 -- 172.21.15.33:0/3967234186 shutdown_connections 2023-10-06T05:35:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.237+0000 7fd637fff700 1 -- 172.21.15.33:0/3967234186 wait complete. 2023-10-06T05:35:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.238+0000 7fd637fff700 1 Processor -- start 2023-10-06T05:35:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.238+0000 7fd637fff700 1 -- start start 2023-10-06T05:35:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd637fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6380cc750 0x7fd638150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd637fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 0x7fd6380b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd637fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380c8f30 0x7fd638151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd637fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638150aa0 con 0x7fd6380ccfc0 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd637fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638150d80 con 0x7fd6380cc750 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd637fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd638151060 con 0x7fd6380c8f30 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd6367fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 0x7fd6380b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd6367fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 0x7fd6380b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44748/0 (socket says 172.21.15.33:44748) 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.239+0000 7fd6367fc700 1 -- 172.21.15.33:0/672718782 learned_addr learned my addr 172.21.15.33:0/672718782 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd636ffd700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6380cc750 0x7fd638150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6377fe700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380c8f30 0x7fd638151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6367fc700 1 -- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380c8f30 msgr2=0x7fd638151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6367fc700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380c8f30 0x7fd638151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6367fc700 1 -- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6380cc750 msgr2=0x7fd638150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6367fc700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6380cc750 0x7fd638150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6367fc700 1 -- 172.21.15.33:0/672718782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd628009420 con 0x7fd6380ccfc0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6377fe700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380c8f30 0x7fd638151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd6367fc700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 0x7fd6380b32a0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c590 tx=0x7fd62c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c010510 con 0x7fd6380ccfc0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c010690 con 0x7fd6380ccfc0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c010a50 con 0x7fd6380ccfc0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.240+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638151340 con 0x7fd6380ccfc0 2023-10-06T05:35:38.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.241+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd638150690 con 0x7fd6380ccfc0 2023-10-06T05:35:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.242+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd638151860 con 0x7fd6380ccfc0 2023-10-06T05:35:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.243+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd62c01b3b0 con 0x7fd6380ccfc0 2023-10-06T05:35:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.244+0000 7fd61effd700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.244+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd62c083b90 con 0x7fd6380ccfc0 2023-10-06T05:35:38.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.244+0000 7fd636ffd700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.245+0000 7fd636ffd700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd624066f30 0x7fd6240693e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd628009860 tx=0x7fd628017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:38.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.248+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd638151860 con 0x7fd6380ccfc0 2023-10-06T05:35:38.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.446+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd638151860 con 0x7fd624066f30 2023-10-06T05:35:38.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.450+0000 7fd61effd700 1 -- 172.21.15.33:0/672718782 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72671 (secure 0 0 0) 0x7fd638151860 con 0x7fd624066f30 2023-10-06T05:35:38.452 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:38.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd624066f30 msgr2=0x7fd6240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd624066f30 0x7fd6240693e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd628009860 tx=0x7fd628017000 comp rx=0 tx=0).stop 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 msgr2=0x7fd6380b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 0x7fd6380b32a0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c590 tx=0x7fd62c007de0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 shutdown_connections 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6380c8f30 0x7fd638151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6380cc750 0x7fd638150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 --2- 172.21.15.33:0/672718782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6380ccfc0 0x7fd6380b32a0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.452+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 >> 172.21.15.33:0/672718782 conn(0x7fd63806cff0 msgr2=0x7fd6380c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.453+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 shutdown_connections 2023-10-06T05:35:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.453+0000 7fd637fff700 1 -- 172.21.15.33:0/672718782 wait complete. 2023-10-06T05:35:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:35:38.471 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":922,"stamp":"2023-10-06T05:35:38.146689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473416,"kb_used_data":29576,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752501304,"statfs":{"total":773094113280,"available":770561335296,"internally_reserved":0,"allocated":30285824,"data_stored":14857640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4063,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.373270"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":235,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.279038+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:35:34.279038+0000","last_peered":"2023-10-06T05:35:34.279038+0000","last_clean":"2023-10-06T05:35:34.279038+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:34.279038+0000","last_undegraded":"2023-10-06T05:35:34.279038+0000","last_fullsized":"2023-10-06T05:35:34.279038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":230,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.986211+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:35:33.986211+0000","last_peered":"2023-10-06T05:35:33.986211+0000","last_clean":"2023-10-06T05:35:33.986211+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:33.986211+0000","last_undegraded":"2023-10-06T05:35:33.986211+0000","last_fullsized":"2023-10-06T05:35:33.986211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":236,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.982242+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:35:33.982242+0000","last_peered":"2023-10-06T05:35:33.982242+0000","last_clean":"2023-10-06T05:35:33.982242+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:33.982242+0000","last_undegraded":"2023-10-06T05:35:33.982242+0000","last_fullsized":"2023-10-06T05:35:33.982242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":230,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.987332+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:35:33.987332+0000","last_peered":"2023-10-06T05:35:33.987332+0000","last_clean":"2023-10-06T05:35:33.987332+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:33.987332+0000","last_undegraded":"2023-10-06T05:35:33.987332+0000","last_fullsized":"2023-10-06T05:35:33.987332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":380,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275502+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:35:34.275502+0000","last_peered":"2023-10-06T05:35:34.275502+0000","last_clean":"2023-10-06T05:35:34.275502+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:34.275502+0000","last_undegraded":"2023-10-06T05:35:34.275502+0000","last_fullsized":"2023-10-06T05:35:34.275502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275538+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:35:34.275538+0000","last_peered":"2023-10-06T05:35:34.275538+0000","last_clean":"2023-10-06T05:35:34.275538+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:34.275538+0000","last_undegraded":"2023-10-06T05:35:34.275538+0000","last_fullsized":"2023-10-06T05:35:34.275538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":227,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275933+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:35:34.275933+0000","last_peered":"2023-10-06T05:35:34.275933+0000","last_clean":"2023-10-06T05:35:34.275933+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:34.275933+0000","last_undegraded":"2023-10-06T05:35:34.275933+0000","last_fullsized":"2023-10-06T05:35:34.275933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.278733+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:35:34.278733+0000","last_peered":"2023-10-06T05:35:34.278733+0000","last_clean":"2023-10-06T05:35:34.278733+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:34.278733+0000","last_undegraded":"2023-10-06T05:35:34.278733+0000","last_fullsized":"2023-10-06T05:35:34.278733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":272,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.985861+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:35:33.985861+0000","last_peered":"2023-10-06T05:35:33.985861+0000","last_clean":"2023-10-06T05:35:33.985861+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:33.985861+0000","last_undegraded":"2023-10-06T05:35:33.985861+0000","last_fullsized":"2023-10-06T05:35:33.985861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575110,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309612,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062228,"statfs":{"total":96636764160,"available":96319721472,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.875},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.502},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.628},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.47299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575108,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.47899999999999998,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.70599999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.115},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.56000000000000005,"5min":0.504,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.022,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.54400000000000004,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":1.0129999999999999,"5min":1.143,"15min":1.143},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.038,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575109,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":1.272,"15min":1.272},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.68000000000000005,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.72199999999999998,"15min":1.165},"last":0.442},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":1.22,"15min":1.22},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.503},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":1.238,"15min":1.2430000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84999999999999998,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96299999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1080000000000001,"5min":1.492,"15min":1.492},"last":1.163}]},{"osd":6,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.262,"15min":1.2909999999999999},"last":1.101},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.82999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.0609999999999999,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.85299999999999998,"15min":0.82199999999999995},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":0.92300000000000004,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.1970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575109,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313708,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058132,"statfs":{"total":96636764160,"available":96315527168,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.95299999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.307},"max":{"1min":0.628,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0980000000000001,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.1379999999999999,"15min":1.27},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.49199999999999999},"max":{"1min":1.151,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.81499999999999995,"15min":0.80100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.147,"15min":1.153},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.496},"max":{"1min":1.0629999999999999,"5min":1.2,"15min":1.2010000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.90800000000000003,"5min":1.1459999999999999,"15min":1.274},"last":0.627}]}]},{"osd":2,"up_from":13,"seq":55834575109,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313128,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058712,"statfs":{"total":96636764160,"available":96316121088,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.79600000000000004,"5min":0.82999999999999996,"15min":1.056},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57099999999999995,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.64000000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.58599999999999997,"5min":1.117,"15min":1.117},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.752}]},{"osd":4,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95599999999999996,"5min":1.2709999999999999,"15min":1.486},"last":1.149},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.089,"15min":1.5},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93899999999999995,"5min":1.157,"15min":1.1639999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92400000000000004,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":1.0600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.22,"15min":1.22},"last":1.042},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.59699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.341,"15min":1.341},"last":1.0229999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":1.282,"15min":1.538},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.264,"15min":1.264},"last":1.1319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575110,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.316},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.443}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575108,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.629},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.751},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.82199999999999995,"5min":1.026,"15min":1.026},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.69499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41799999999999998},"max":{"1min":0.88600000000000001,"5min":1.099,"15min":1.099},"last":0.91700000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76200000000000001,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.443},"max":{"1min":0.85699999999999998,"5min":3.839,"15min":3.839},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.873,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":0.96499999999999997,"15min":1.0609999999999999},"last":1.012}]},{"osd":7,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":0.93999999999999995,"5min":1.109,"15min":1.282},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.89400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575110,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.90700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.372},"max":{"1min":1.071,"5min":1.115,"15min":1.2130000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.55700000000000005,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":1.169,"15min":1.169},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.145},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.92800000000000005,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.02,"15min":1.1539999999999999},"last":0.51000000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.96899999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.52000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.016},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.60199999999999998,"5min":0.53300000000000003,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.0609999999999999},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:35:38.474 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:35:38.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:35:38.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.659+0000 7fd5d89fe700 1 Processor -- start 2023-10-06T05:35:38.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.660+0000 7fd5d89fe700 1 -- start start 2023-10-06T05:35:38.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d4158700 0x7fd5d40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d89fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5d40699c0 0x7fd5d4069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d89fe700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d40ae490 con 0x7fd5d4155140 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d89fe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d406a380 con 0x7fd5d40699c0 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c8e40 con 0x7fd5d4158700 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5d40699c0 0x7fd5d4069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d2d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd5d4155140 0x7fd5d40c92d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52360/0 (socket says 172.21.15.33:52360) 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d2d9d700 1 -- 172.21.15.33:0/2724664133 learned_addr learned my addr 172.21.15.33:0/2724664133 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.661+0000 7fd5d259c700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d4158700 0x7fd5d40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d359e700 1 -- 172.21.15.33:0/2724664133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d4158700 msgr2=0x7fd5d40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d359e700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d4158700 0x7fd5d40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d359e700 1 -- 172.21.15.33:0/2724664133 >> v1:172.21.15.33:6789/0 conn(0x7fd5d4155140 legacy=0x7fd5d40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d359e700 1 -- 172.21.15.33:0/2724664133 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4158b70 con 0x7fd5d40699c0 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d259c700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d4158700 0x7fd5d40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d1d9b700 1 -- 172.21.15.33:0/2724664133 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502576598 0 0) 0x7fd5d40ae490 con 0x7fd5d4155140 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d359e700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d40699c0 0x7fd5d4069e40 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009e60 tx=0x7fd5cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dd252983f24cdc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d1d9b700 1 -- 172.21.15.33:0/2724664133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00e070 con 0x7fd5d40699c0 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d1d9b700 1 -- 172.21.15.33:0/2724664133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc00bc30 con 0x7fd5d40699c0 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.662+0000 7fd5d1d9b700 1 -- 172.21.15.33:0/2724664133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc012430 con 0x7fd5d40699c0 2023-10-06T05:35:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 -- 172.21.15.33:0/2724664133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d40699c0 msgr2=0x7fd5d4069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d40699c0 0x7fd5d4069e40 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009e60 tx=0x7fd5cc009780 comp rx=0 tx=0).stop 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 -- 172.21.15.33:0/2724664133 shutdown_connections 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d40699c0 0x7fd5d4069e40 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/2724664133 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d4158700 0x7fd5d40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 -- 172.21.15.33:0/2724664133 >> 172.21.15.33:0/2724664133 conn(0x7fd5d406d030 msgr2=0x7fd5d406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 -- 172.21.15.33:0/2724664133 shutdown_connections 2023-10-06T05:35:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.663+0000 7fd5d89fe700 1 -- 172.21.15.33:0/2724664133 wait complete. 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.664+0000 7fd5d89fe700 1 Processor -- start 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 -- start start 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d40699c0 0x7fd5d40c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d4155140 0x7fd5d40cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 0x7fd5d4152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c3590 con 0x7fd5d4158700 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c43f0 con 0x7fd5d4155140 2023-10-06T05:35:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.665+0000 7fd5d89fe700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c4dc0 con 0x7fd5d40699c0 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 0x7fd5d4152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 0x7fd5d4152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44764/0 (socket says 172.21.15.33:44764) 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 -- 172.21.15.33:0/1240183019 learned_addr learned my addr 172.21.15.33:0/1240183019 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d259c700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d4155140 0x7fd5d40cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d2d9d700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d40699c0 0x7fd5d40c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 -- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d40699c0 msgr2=0x7fd5d40c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d40699c0 0x7fd5d40c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 -- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d4155140 msgr2=0x7fd5d40cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d4155140 0x7fd5d40cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d359e700 1 -- 172.21.15.33:0/1240183019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc009a20 con 0x7fd5d4158700 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.666+0000 7fd5d2d9d700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d40699c0 0x7fd5d40c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.667+0000 7fd5d359e700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 0x7fd5d4152cd0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc013000 tx=0x7fd5cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.667+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc01d630 con 0x7fd5d4158700 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.667+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc01d790 con 0x7fd5d4158700 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.667+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc01db50 con 0x7fd5d4158700 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.667+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d40c37e0 con 0x7fd5d4158700 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.667+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d40c3e20 con 0x7fd5d4158700 2023-10-06T05:35:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.668+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d40ccf80 con 0x7fd5d4158700 2023-10-06T05:35:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.669+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5cc024070 con 0x7fd5d4158700 2023-10-06T05:35:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.670+0000 7fd5baffd700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5c0066f30 0x7fd5c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.670+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd5cc086d10 con 0x7fd5d4158700 2023-10-06T05:35:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.670+0000 7fd5d2d9d700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5c0066f30 0x7fd5c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.671+0000 7fd5d2d9d700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5c0066f30 0x7fd5c00693e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40011e0 tx=0x7fd5c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:38.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.674+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5cc08c020 con 0x7fd5d4158700 2023-10-06T05:35:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.871+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5d404c380 con 0x7fd5c0066f30 2023-10-06T05:35:38.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.875+0000 7fd5baffd700 1 -- 172.21.15.33:0/1240183019 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72671 (secure 0 0 0) 0x7fd5d404c380 con 0x7fd5c0066f30 2023-10-06T05:35:38.877 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:38.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.876+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5c0066f30 msgr2=0x7fd5c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.876+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5c0066f30 0x7fd5c00693e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40011e0 tx=0x7fd5c4009040 comp rx=0 tx=0).stop 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.876+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 msgr2=0x7fd5d4152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.876+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 0x7fd5d4152cd0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc013000 tx=0x7fd5cc019000 comp rx=0 tx=0).stop 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 shutdown_connections 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd5c0066f30 0x7fd5c00693e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4158700 0x7fd5d4152cd0 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5d40699c0 0x7fd5d40c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 --2- 172.21.15.33:0/1240183019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d4155140 0x7fd5d40cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 >> 172.21.15.33:0/1240183019 conn(0x7fd5d406d030 msgr2=0x7fd5d40c1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 shutdown_connections 2023-10-06T05:35:38.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:38.877+0000 7fd5d89fe700 1 -- 172.21.15.33:0/1240183019 wait complete. 2023-10-06T05:35:38.880 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:35:38.895 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":922,"stamp":"2023-10-06T05:35:38.146689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473416,"kb_used_data":29576,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752501304,"statfs":{"total":773094113280,"available":770561335296,"internally_reserved":0,"allocated":30285824,"data_stored":14857640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4063,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.373270"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":235,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.279038+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:35:34.279038+0000","last_peered":"2023-10-06T05:35:34.279038+0000","last_clean":"2023-10-06T05:35:34.279038+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:34.279038+0000","last_undegraded":"2023-10-06T05:35:34.279038+0000","last_fullsized":"2023-10-06T05:35:34.279038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":230,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.986211+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:35:33.986211+0000","last_peered":"2023-10-06T05:35:33.986211+0000","last_clean":"2023-10-06T05:35:33.986211+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:33.986211+0000","last_undegraded":"2023-10-06T05:35:33.986211+0000","last_fullsized":"2023-10-06T05:35:33.986211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":236,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.982242+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:35:33.982242+0000","last_peered":"2023-10-06T05:35:33.982242+0000","last_clean":"2023-10-06T05:35:33.982242+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:33.982242+0000","last_undegraded":"2023-10-06T05:35:33.982242+0000","last_fullsized":"2023-10-06T05:35:33.982242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":230,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.987332+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:35:33.987332+0000","last_peered":"2023-10-06T05:35:33.987332+0000","last_clean":"2023-10-06T05:35:33.987332+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:33.987332+0000","last_undegraded":"2023-10-06T05:35:33.987332+0000","last_fullsized":"2023-10-06T05:35:33.987332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":380,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275502+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:35:34.275502+0000","last_peered":"2023-10-06T05:35:34.275502+0000","last_clean":"2023-10-06T05:35:34.275502+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:34.275502+0000","last_undegraded":"2023-10-06T05:35:34.275502+0000","last_fullsized":"2023-10-06T05:35:34.275502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275538+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:35:34.275538+0000","last_peered":"2023-10-06T05:35:34.275538+0000","last_clean":"2023-10-06T05:35:34.275538+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:34.275538+0000","last_undegraded":"2023-10-06T05:35:34.275538+0000","last_fullsized":"2023-10-06T05:35:34.275538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":227,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275933+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:35:34.275933+0000","last_peered":"2023-10-06T05:35:34.275933+0000","last_clean":"2023-10-06T05:35:34.275933+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:34.275933+0000","last_undegraded":"2023-10-06T05:35:34.275933+0000","last_fullsized":"2023-10-06T05:35:34.275933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":235,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.278733+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:35:34.278733+0000","last_peered":"2023-10-06T05:35:34.278733+0000","last_clean":"2023-10-06T05:35:34.278733+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:34.278733+0000","last_undegraded":"2023-10-06T05:35:34.278733+0000","last_fullsized":"2023-10-06T05:35:34.278733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:25.082305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":272,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.985861+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:35:33.985861+0000","last_peered":"2023-10-06T05:35:33.985861+0000","last_clean":"2023-10-06T05:35:33.985861+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:33.985861+0000","last_undegraded":"2023-10-06T05:35:33.985861+0000","last_fullsized":"2023-10-06T05:35:33.985861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575110,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309612,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062228,"statfs":{"total":96636764160,"available":96319721472,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.875},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.502},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.628},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.47299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575108,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.47899999999999998,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.70599999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.115},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.56000000000000005,"5min":0.504,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.022,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.54400000000000004,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":1.0129999999999999,"5min":1.143,"15min":1.143},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.038,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.81599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575109,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":1.272,"15min":1.272},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.68000000000000005,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.72199999999999998,"15min":1.165},"last":0.442},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":1.22,"15min":1.22},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.503},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":1.238,"15min":1.2430000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84999999999999998,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96299999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1080000000000001,"5min":1.492,"15min":1.492},"last":1.163}]},{"osd":6,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.262,"15min":1.2909999999999999},"last":1.101},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.82999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.0609999999999999,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.85299999999999998,"15min":0.82199999999999995},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":0.92300000000000004,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.1970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575109,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313708,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058132,"statfs":{"total":96636764160,"available":96315527168,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.95299999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.307},"max":{"1min":0.628,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0980000000000001,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.1379999999999999,"15min":1.27},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.49199999999999999},"max":{"1min":1.151,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.81499999999999995,"15min":0.80100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.147,"15min":1.153},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.496},"max":{"1min":1.0629999999999999,"5min":1.2,"15min":1.2010000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.90800000000000003,"5min":1.1459999999999999,"15min":1.274},"last":0.627}]}]},{"osd":2,"up_from":13,"seq":55834575109,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313128,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058712,"statfs":{"total":96636764160,"available":96316121088,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.79600000000000004,"5min":0.82999999999999996,"15min":1.056},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57099999999999995,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.64000000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.58599999999999997,"5min":1.117,"15min":1.117},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.752}]},{"osd":4,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95599999999999996,"5min":1.2709999999999999,"15min":1.486},"last":1.149},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.089,"15min":1.5},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93899999999999995,"5min":1.157,"15min":1.1639999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92400000000000004,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":1.0600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.22,"15min":1.22},"last":1.042},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.59699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.341,"15min":1.341},"last":1.0229999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":1.282,"15min":1.538},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.264,"15min":1.264},"last":1.1319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575110,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.316},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.443}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575108,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.629},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.751},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.82199999999999995,"5min":1.026,"15min":1.026},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.69499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41799999999999998},"max":{"1min":0.88600000000000001,"5min":1.099,"15min":1.099},"last":0.91700000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76200000000000001,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.443},"max":{"1min":0.85699999999999998,"5min":3.839,"15min":3.839},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.873,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":0.96499999999999997,"15min":1.0609999999999999},"last":1.012}]},{"osd":7,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":0.93999999999999995,"5min":1.109,"15min":1.282},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.89400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575110,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.90700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.372},"max":{"1min":1.071,"5min":1.115,"15min":1.2130000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.55700000000000005,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":1.169,"15min":1.169},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.145},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.92800000000000005,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.02,"15min":1.1539999999999999},"last":0.51000000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.96899999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.52000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.016},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.60199999999999998,"5min":0.53300000000000003,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.0609999999999999},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:35:38.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:35:39.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.087+0000 7fc87719d700 1 Processor -- start 2023-10-06T05:35:39.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.087+0000 7fc87719d700 1 -- start start 2023-10-06T05:35:39.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87013faa0 0x7fc87013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:39.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87719d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8700c9cb0 con 0x7fc8700c7940 2023-10-06T05:35:39.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87719d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8700ca270 con 0x7fc87006aad0 2023-10-06T05:35:39.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87006b660 con 0x7fc87013faa0 2023-10-06T05:35:39.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87599a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc8700c7940 0x7fc8700c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52376/0 (socket says 172.21.15.33:52376) 2023-10-06T05:35:39.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87599a700 1 -- 172.21.15.33:0/1003251123 learned_addr learned my addr 172.21.15.33:0/1003251123 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:39.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc87619b700 1 --2- 172.21.15.33:0/1003251123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87013faa0 0x7fc87013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400030761 0 0) 0x7fc8700c9cb0 con 0x7fc8700c7940 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc85c003570 con 0x7fc8700c7940 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.089+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707374973 0 0) 0x7fc8700ca270 con 0x7fc87006aad0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8700c9cb0 con 0x7fc87006aad0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc87619b700 1 -- 172.21.15.33:0/1003251123 >> v1:172.21.15.167:6789/0 conn(0x7fc87006aad0 legacy=0x7fc87006af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc87619b700 1 -- 172.21.15.33:0/1003251123 >> v1:172.21.15.33:6789/0 conn(0x7fc8700c7940 legacy=0x7fc8700c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc87619b700 1 -- 172.21.15.33:0/1003251123 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87006b840 con 0x7fc87013faa0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc87619b700 1 --2- 172.21.15.33:0/1003251123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87013faa0 0x7fc87013fe70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009910 tx=0x7fc86c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6961a7356f02c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c011630 con 0x7fc87013faa0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c011790 con 0x7fc87013faa0 2023-10-06T05:35:39.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.090+0000 7fc875199700 1 -- 172.21.15.33:0/1003251123 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c011b30 con 0x7fc87013faa0 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 -- 172.21.15.33:0/1003251123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87013faa0 msgr2=0x7fc87013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 --2- 172.21.15.33:0/1003251123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87013faa0 0x7fc87013fe70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009910 tx=0x7fc86c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 -- 172.21.15.33:0/1003251123 shutdown_connections 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 --2- 172.21.15.33:0/1003251123 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87013faa0 0x7fc87013fe70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 -- 172.21.15.33:0/1003251123 >> 172.21.15.33:0/1003251123 conn(0x7fc87006d030 msgr2=0x7fc87006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 -- 172.21.15.33:0/1003251123 shutdown_connections 2023-10-06T05:35:39.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.091+0000 7fc87719d700 1 -- 172.21.15.33:0/1003251123 wait complete. 2023-10-06T05:35:39.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.092+0000 7fc87719d700 1 Processor -- start 2023-10-06T05:35:39.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.093+0000 7fc87719d700 1 -- start start 2023-10-06T05:35:39.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.093+0000 7fc87719d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87006aad0 0x7fc8701561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:39.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.093+0000 7fc87719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8700c7940 0x7fc8700b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87719d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 0x7fc87013aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87719d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc870169c70 con 0x7fc87013faa0 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc870169f50 con 0x7fc8700c7940 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87719d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87016a230 con 0x7fc87006aad0 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 0x7fc87013aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 0x7fc87013aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44766/0 (socket says 172.21.15.33:44766) 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 -- 172.21.15.33:0/1080854246 learned_addr learned my addr 172.21.15.33:0/1080854246 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87619b700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87006aad0 0x7fc8701561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87599a700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8700c7940 0x7fc8700b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 -- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87006aad0 msgr2=0x7fc8701561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.097 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87006aad0 0x7fc8701561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 -- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8700c7940 msgr2=0x7fc8700b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8700c7940 0x7fc8700b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.094+0000 7fc87699c700 1 -- 172.21.15.33:0/1080854246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c009420 con 0x7fc87013faa0 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.095+0000 7fc87699c700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 0x7fc87013aea0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c910 tx=0x7fc8680027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.095+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868012630 con 0x7fc87013faa0 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.095+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8680127b0 con 0x7fc87013faa0 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.095+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87016a510 con 0x7fc87013faa0 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.095+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868012b70 con 0x7fc87013faa0 2023-10-06T05:35:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.095+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc870137aa0 con 0x7fc87013faa0 2023-10-06T05:35:39.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.096+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87016aa30 con 0x7fc87013faa0 2023-10-06T05:35:39.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.097+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc86801b400 con 0x7fc87013faa0 2023-10-06T05:35:39.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.098+0000 7fc8667fc700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc854066f30 0x7fc8540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:39.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.098+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc8680847f0 con 0x7fc87013faa0 2023-10-06T05:35:39.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.098+0000 7fc87619b700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc854066f30 0x7fc8540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:39.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.099+0000 7fc87619b700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc854066f30 0x7fc8540693e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0091e0 tx=0x7fc86c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.102+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc87016aa30 con 0x7fc87013faa0 2023-10-06T05:35:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.306+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87016aa30 con 0x7fc87013faa0 2023-10-06T05:35:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.307+0000 7fc8667fc700 1 -- 172.21.15.33:0/1080854246 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v286) v1 ==== 74+0+13853 (secure 0 0 0) 0x7fc87016aa30 con 0x7fc87013faa0 2023-10-06T05:35:39.309 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:39.309 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":286,"fsid":"abad2d63-0b71-4f3e-871a-ad3209a2fd6e","created":"2023-10-06T05:13:58.820108+0000","modified":"2023-10-06T05:35:34.986539+0000","last_up_change":"2023-10-06T05:14:11.078922+0000","last_in_change":"2023-10-06T05:14:02.113264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:11.879286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:18.463299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c4fc2102-e639-40b8-914d-00d54ee9b747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":85577},{"type":"v1","addr":"172.21.15.33:6801","nonce":85577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":85577},{"type":"v1","addr":"172.21.15.33:6803","nonce":85577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":85577},{"type":"v1","addr":"172.21.15.33:6807","nonce":85577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":85577},{"type":"v1","addr":"172.21.15.33:6805","nonce":85577}]},"public_addr":"172.21.15.33:6801/85577","cluster_addr":"172.21.15.33:6803/85577","heartbeat_back_addr":"172.21.15.33:6807/85577","heartbeat_front_addr":"172.21.15.33:6805/85577","state":["exists","up"]},{"osd":1,"uuid":"a4410d06-3707-4e1d-9194-02f96c4861c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":85579},{"type":"v1","addr":"172.21.15.33:6817","nonce":85579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":85579},{"type":"v1","addr":"172.21.15.33:6819","nonce":85579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":85579},{"type":"v1","addr":"172.21.15.33:6823","nonce":85579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":85579},{"type":"v1","addr":"172.21.15.33:6821","nonce":85579}]},"public_addr":"172.21.15.33:6817/85579","cluster_addr":"172.21.15.33:6819/85579","heartbeat_back_addr":"172.21.15.33:6823/85579","heartbeat_front_addr":"172.21.15.33:6821/85579","state":["exists","up"]},{"osd":2,"uuid":"c3b0c4a3-8a0b-4774-8a01-d5abfe37776c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":85578},{"type":"v1","addr":"172.21.15.33:6809","nonce":85578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":85578},{"type":"v1","addr":"172.21.15.33:6811","nonce":85578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":85578},{"type":"v1","addr":"172.21.15.33:6815","nonce":85578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":85578},{"type":"v1","addr":"172.21.15.33:6813","nonce":85578}]},"public_addr":"172.21.15.33:6809/85578","cluster_addr":"172.21.15.33:6811/85578","heartbeat_back_addr":"172.21.15.33:6815/85578","heartbeat_front_addr":"172.21.15.33:6813/85578","state":["exists","up"]},{"osd":3,"uuid":"3d9cc09d-af9f-4ae7-bfb6-b06490f028a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":85580},{"type":"v1","addr":"172.21.15.33:6825","nonce":85580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":85580},{"type":"v1","addr":"172.21.15.33:6827","nonce":85580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":85580},{"type":"v1","addr":"172.21.15.33:6831","nonce":85580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":85580},{"type":"v1","addr":"172.21.15.33:6829","nonce":85580}]},"public_addr":"172.21.15.33:6825/85580","cluster_addr":"172.21.15.33:6827/85580","heartbeat_back_addr":"172.21.15.33:6831/85580","heartbeat_front_addr":"172.21.15.33:6829/85580","state":["exists","up"]},{"osd":4,"uuid":"db1d7cf0-6c8d-47b7-a0bb-b45726447197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":85769},{"type":"v1","addr":"172.21.15.167:6801","nonce":85769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":85769},{"type":"v1","addr":"172.21.15.167:6803","nonce":85769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":85769},{"type":"v1","addr":"172.21.15.167:6807","nonce":85769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":85769},{"type":"v1","addr":"172.21.15.167:6805","nonce":85769}]},"public_addr":"172.21.15.167:6801/85769","cluster_addr":"172.21.15.167:6803/85769","heartbeat_back_addr":"172.21.15.167:6807/85769","heartbeat_front_addr":"172.21.15.167:6805/85769","state":["exists","up"]},{"osd":5,"uuid":"ca568a38-27d1-437e-8a93-19c9a04d9cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":85770},{"type":"v1","addr":"172.21.15.167:6817","nonce":85770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":85770},{"type":"v1","addr":"172.21.15.167:6819","nonce":85770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":85770},{"type":"v1","addr":"172.21.15.167:6823","nonce":85770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":85770},{"type":"v1","addr":"172.21.15.167:6821","nonce":85770}]},"public_addr":"172.21.15.167:6817/85770","cluster_addr":"172.21.15.167:6819/85770","heartbeat_back_addr":"172.21.15.167:6823/85770","heartbeat_front_addr":"172.21.15.167:6821/85770","state":["exists","up"]},{"osd":6,"uuid":"895c30ab-410a-411b-84ad-e7f759abf82a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":85771},{"type":"v1","addr":"172.21.15.167:6809","nonce":85771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":85771},{"type":"v1","addr":"172.21.15.167:6811","nonce":85771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":85771},{"type":"v1","addr":"172.21.15.167:6815","nonce":85771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":85771},{"type":"v1","addr":"172.21.15.167:6813","nonce":85771}]},"public_addr":"172.21.15.167:6809/85771","cluster_addr":"172.21.15.167:6811/85771","heartbeat_back_addr":"172.21.15.167:6815/85771","heartbeat_front_addr":"172.21.15.167:6813/85771","state":["exists","up"]},{"osd":7,"uuid":"8e18fadc-363e-4401-a5ee-07229d17b12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":280,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":85794},{"type":"v1","addr":"172.21.15.167:6825","nonce":85794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":85794},{"type":"v1","addr":"172.21.15.167:6827","nonce":85794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":85794},{"type":"v1","addr":"172.21.15.167:6831","nonce":85794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":85794},{"type":"v1","addr":"172.21.15.167:6829","nonce":85794}]},"public_addr":"172.21.15.167:6825/85794","cluster_addr":"172.21.15.167:6827/85794","heartbeat_back_addr":"172.21.15.167:6831/85794","heartbeat_front_addr":"172.21.15.167:6829/85794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.665291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.448877+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.721162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.426594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.409381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.446037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.213569+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:07.458037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/3559384937":"2023-10-07T05:35:31.767811+0000","172.21.15.167:6834/3559384937":"2023-10-07T05:35:31.767811+0000","172.21.15.33:0/2784413487":"2023-10-06T06:35:25.570337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:35:39.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc854066f30 msgr2=0x7fc8540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc854066f30 0x7fc8540693e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0091e0 tx=0x7fc86c017000 comp rx=0 tx=0).stop 2023-10-06T05:35:39.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 msgr2=0x7fc87013aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 0x7fc87013aea0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c910 tx=0x7fc8680027a0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 shutdown_connections 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc854066f30 0x7fc8540693e0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc87013faa0 0x7fc87013aea0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc87006aad0 0x7fc8701561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 --2- 172.21.15.33:0/1080854246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8700c7940 0x7fc8700b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.309+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 >> 172.21.15.33:0/1080854246 conn(0x7fc87006d030 msgr2=0x7fc8700c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.310+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 shutdown_connections 2023-10-06T05:35:39.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:39.310+0000 7fc87719d700 1 -- 172.21.15.33:0/1080854246 wait complete. 2023-10-06T05:35:40.330 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T05:35:40.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:40.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.532+0000 7f9f41f9d700 1 Processor -- start 2023-10-06T05:35:40.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.532+0000 7f9f41f9d700 1 -- start start 2023-10-06T05:35:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.533+0000 7f9f41f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 0x7f9f3c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.533+0000 7f9f41f9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9f3c0acd00 0x7f9f3c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.533+0000 7f9f41f9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3c069ad0 con 0x7f9f3c155e70 2023-10-06T05:35:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.533+0000 7f9f41f9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c069db0 con 0x7f9f3c0acd00 2023-10-06T05:35:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.533+0000 7f9f41f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c06a090 con 0x7f9f3c155aa0 2023-10-06T05:35:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f4179c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9f3c0acd00 0x7f9f3c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f40f9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 0x7f9f3c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f40f9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 0x7f9f3c0aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:59928/0 (socket says 172.21.15.33:59928) 2023-10-06T05:35:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f40f9b700 1 -- 172.21.15.33:0/645504501 learned_addr learned my addr 172.21.15.33:0/645504501 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f4179c700 1 -- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 msgr2=0x7f9f3c0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f4179c700 1 --2- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 0x7f9f3c0aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f4179c700 1 -- 172.21.15.33:0/645504501 >> v1:172.21.15.167:6789/0 conn(0x7f9f3c155e70 legacy=0x7f9f3c0ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f4179c700 1 -- 172.21.15.33:0/645504501 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c06a370 con 0x7f9f3c0acd00 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f40f9b700 1 --2- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 0x7f9f3c0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.534+0000 7f9f3b7fe700 1 -- 172.21.15.33:0/645504501 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666967326 0 0) 0x7f9f3c069ad0 con 0x7f9f3c155e70 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f4179c700 1 --2- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c0acd00 0x7f9f3c0c90c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009e60 tx=0x7f9f34009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ec13e4106dcf2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f3b7fe700 1 -- 172.21.15.33:0/645504501 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400bb60 con 0x7f9f3c0acd00 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f3b7fe700 1 -- 172.21.15.33:0/645504501 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3400bcc0 con 0x7f9f3c0acd00 2023-10-06T05:35:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f3b7fe700 1 -- 172.21.15.33:0/645504501 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34011650 con 0x7f9f3c0acd00 2023-10-06T05:35:40.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 -- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c0acd00 msgr2=0x7f9f3c0c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 --2- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c0acd00 0x7f9f3c0c90c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009e60 tx=0x7f9f34009780 comp rx=0 tx=0).stop 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 -- 172.21.15.33:0/645504501 shutdown_connections 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 --2- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c0acd00 0x7f9f3c0c90c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 --2- 172.21.15.33:0/645504501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c155aa0 0x7f9f3c0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 -- 172.21.15.33:0/645504501 >> 172.21.15.33:0/645504501 conn(0x7f9f3c068090 msgr2=0x7f9f3c06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 -- 172.21.15.33:0/645504501 shutdown_connections 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.535+0000 7f9f41f9d700 1 -- 172.21.15.33:0/645504501 wait complete. 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.537+0000 7f9f41f9d700 1 Processor -- start 2023-10-06T05:35:40.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.537+0000 7f9f41f9d700 1 -- start start 2023-10-06T05:35:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f41f9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c0acd00 0x7f9f3c0c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f41f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f3c155aa0 0x7f9f3c152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f41f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c155e70 0x7f9f3c16aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f41f9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0ab750 con 0x7f9f3c155aa0 2023-10-06T05:35:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f41f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0aa6b0 con 0x7f9f3c155e70 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f41f9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0aa990 con 0x7f9f3c0acd00 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f3bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f3c155aa0 0x7f9f3c152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f3bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f3c155aa0 0x7f9f3c152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52208/0 (socket says 172.21.15.33:52208) 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f3bfff700 1 -- 172.21.15.33:0/1981734893 learned_addr learned my addr 172.21.15.33:0/1981734893 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f40f9b700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c0acd00 0x7f9f3c0c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.538+0000 7f9f4179c700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c155e70 0x7f9f3c16aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f3bfff700 1 -- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c0acd00 msgr2=0x7f9f3c0c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f3bfff700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f3c0acd00 0x7f9f3c0c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f3bfff700 1 -- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c155e70 msgr2=0x7f9f3c16aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f3bfff700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f3c155e70 0x7f9f3c16aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f3bfff700 1 -- 172.21.15.33:0/1981734893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300094f0 con 0x7f9f3c155aa0 2023-10-06T05:35:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f3bfff700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f3c155aa0 0x7f9f3c152e10 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800c530 tx=0x7f9f2800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28011590 con 0x7f9f3c155aa0 2023-10-06T05:35:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f28011710 con 0x7f9f3c155aa0 2023-10-06T05:35:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f34009a20 con 0x7f9f3c155aa0 2023-10-06T05:35:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28011ad0 con 0x7f9f3c155aa0 2023-10-06T05:35:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.539+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c0aad20 con 0x7f9f3c155aa0 2023-10-06T05:35:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.540+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f3c063ff0 con 0x7f9f3c155aa0 2023-10-06T05:35:40.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.541+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f2801b3b0 con 0x7f9f3c155aa0 2023-10-06T05:35:40.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.542+0000 7f9f38ff9700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f20066f30 0x7f9f200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.542+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9f280838c0 con 0x7f9f3c155aa0 2023-10-06T05:35:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.543+0000 7f9f38ff9700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f9f2006d3f0 0x7f9f2006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.543+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 --> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f2006feb0 con 0x7f9f2006d3f0 2023-10-06T05:35:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.543+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f28085dc0 con 0x7f9f3c155aa0 2023-10-06T05:35:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.543+0000 7f9f4179c700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f9f2006d3f0 0x7f9f2006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.543+0000 7f9f40f9b700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f20066f30 0x7f9f200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.543+0000 7f9f4179c700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f9f2006d3f0 0x7f9f2006f800 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.544+0000 7f9f40f9b700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f20066f30 0x7f9f200693e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006d20 tx=0x7f9f30009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.544+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== osd.0 v2:172.21.15.33:6800/85577 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9f2006feb0 con 0x7f9f2006d3f0 2023-10-06T05:35:40.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.559+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 --> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9f3c0aab50 con 0x7f9f2006d3f0 2023-10-06T05:35:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.559+0000 7f9f38ff9700 1 -- 172.21.15.33:0/1981734893 <== osd.0 v2:172.21.15.33:6800/85577 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9f34021040 con 0x7f9f2006d3f0 2023-10-06T05:35:40.561 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:40.561 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:40.561 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f9f2006d3f0 msgr2=0x7f9f2006f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:6800/85577,v1:172.21.15.33:6801/85577] conn(0x7f9f2006d3f0 0x7f9f2006f800 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f20066f30 msgr2=0x7f9f200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9f20066f30 0x7f9f200693e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006d20 tx=0x7f9f30009250 comp rx=0 tx=0).stop 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f3c155aa0 msgr2=0x7f9f3c152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 --2- 172.21.15.33:0/1981734893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f3c155aa0 0x7f9f3c152e10 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800c530 tx=0x7f9f2800c840 comp rx=0 tx=0).stop 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f4179c700 1 -- 172.21.15.33:0/1981734893 reap_dead start 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 shutdown_connections 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 >> 172.21.15.33:0/1981734893 conn(0x7f9f3c068090 msgr2=0x7f9f3c06e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 shutdown_connections 2023-10-06T05:35:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.560+0000 7f9f41f9d700 1 -- 172.21.15.33:0/1981734893 wait complete. 2023-10-06T05:35:40.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T05:35:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.764+0000 7f99fb59e700 1 Processor -- start 2023-10-06T05:35:40.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.764+0000 7f99fb59e700 1 -- start start 2023-10-06T05:35:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fb59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99fc159400 0x7f99fc0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc1597d0 0x7f99fc0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fb59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0d0ed0 con 0x7f99fc0cc770 2023-10-06T05:35:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fb59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0b1160 con 0x7f99fc159400 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0cb6d0 con 0x7f99fc1597d0 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fa59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99fc159400 0x7f99fc0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fa59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f99fc159400 0x7f99fc0cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52224/0 (socket says 172.21.15.33:52224) 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99fa59c700 1 -- 172.21.15.33:0/1051483616 learned_addr learned my addr 172.21.15.33:0/1051483616 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.766+0000 7f99f9d9b700 1 --2- 172.21.15.33:0/1051483616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc1597d0 0x7f99fc0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99fa59c700 1 -- 172.21.15.33:0/1051483616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc1597d0 msgr2=0x7f99fc0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99fa59c700 1 --2- 172.21.15.33:0/1051483616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc1597d0 0x7f99fc0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99fa59c700 1 -- 172.21.15.33:0/1051483616 >> v1:172.21.15.167:6789/0 conn(0x7f99fc0cc770 legacy=0x7f99fc06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99fa59c700 1 -- 172.21.15.33:0/1051483616 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0cb9b0 con 0x7f99fc159400 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99fa59c700 1 --2- 172.21.15.33:0/1051483616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc0cab60 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0099b0 tx=0x7f99ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b14852bd452f26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99f959a700 1 -- 172.21.15.33:0/1051483616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00b860 con 0x7f99fc159400 2023-10-06T05:35:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99f959a700 1 -- 172.21.15.33:0/1051483616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00b9c0 con 0x7f99fc159400 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.767+0000 7f99f959a700 1 -- 172.21.15.33:0/1051483616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00bd60 con 0x7f99fc159400 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 -- 172.21.15.33:0/1051483616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 msgr2=0x7f99fc0cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 --2- 172.21.15.33:0/1051483616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc0cab60 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0099b0 tx=0x7f99ec009250 comp rx=0 tx=0).stop 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 -- 172.21.15.33:0/1051483616 shutdown_connections 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 --2- 172.21.15.33:0/1051483616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc0cab60 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 --2- 172.21.15.33:0/1051483616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc1597d0 0x7f99fc0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 -- 172.21.15.33:0/1051483616 >> 172.21.15.33:0/1051483616 conn(0x7f99fc06e4b0 msgr2=0x7f99fc0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 -- 172.21.15.33:0/1051483616 shutdown_connections 2023-10-06T05:35:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.768+0000 7f99fb59e700 1 -- 172.21.15.33:0/1051483616 wait complete. 2023-10-06T05:35:40.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.769+0000 7f99fb59e700 1 Processor -- start 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.769+0000 7f99fb59e700 1 -- start start 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.770+0000 7f99fb59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc0cc770 0x7f99fc0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.770+0000 7f99fb59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.770+0000 7f99fb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99fc1597d0 0x7f99fc0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.770+0000 7f99fb59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc15bf40 con 0x7f99fc159400 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.770+0000 7f99fb59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc15cd30 con 0x7f99fc1597d0 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.770+0000 7f99fb59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0cb390 con 0x7f99fc0cc770 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52232/0 (socket says 172.21.15.33:52232) 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 -- 172.21.15.33:0/3591563302 learned_addr learned my addr 172.21.15.33:0/3591563302 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99fa59c700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc0cc770 0x7f99fc0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99fad9d700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99fc1597d0 0x7f99fc0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 -- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc0cc770 msgr2=0x7f99fc0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc0cc770 0x7f99fc0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 -- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99fc1597d0 msgr2=0x7f99fc0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99fc1597d0 0x7f99fc0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 -- 172.21.15.33:0/3591563302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0094f0 con 0x7f99fc159400 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99fad9d700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99fc1597d0 0x7f99fc0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99f9d9b700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc158410 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f99f000d860 tx=0x7f99f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.771+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00125f0 con 0x7f99fc159400 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.772+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0012750 con 0x7f99fc159400 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.772+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0012b10 con 0x7f99fc159400 2023-10-06T05:35:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.772+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc15c120 con 0x7f99fc159400 2023-10-06T05:35:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.772+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc15c6e0 con 0x7f99fc159400 2023-10-06T05:35:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.773+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc15cec0 con 0x7f99fc159400 2023-10-06T05:35:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.774+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99f001e460 con 0x7f99fc159400 2023-10-06T05:35:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.775+0000 7f99ea7fc700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f99dc066f30 0x7f99dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.775+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f99f0084af0 con 0x7f99fc159400 2023-10-06T05:35:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.775+0000 7f99fa59c700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f99dc066f30 0x7f99dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.776+0000 7f99fa59c700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f99dc066f30 0x7f99dc0693e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0099b0 tx=0x7f99ec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.778+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99f0051e30 con 0x7f99fc159400 2023-10-06T05:35:40.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.983+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f99fc04c3e0 con 0x7f99dc066f30 2023-10-06T05:35:40.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.984+0000 7f99ea7fc700 1 -- 172.21.15.33:0/3591563302 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f99fc04c3e0 con 0x7f99dc066f30 2023-10-06T05:35:40.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f99dc066f30 msgr2=0x7f99dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f99dc066f30 0x7f99dc0693e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0099b0 tx=0x7f99ec017000 comp rx=0 tx=0).stop 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 msgr2=0x7f99fc158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc158410 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f99f000d860 tx=0x7f99f0008040 comp rx=0 tx=0).stop 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 shutdown_connections 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f99dc066f30 0x7f99dc0693e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99fc1597d0 0x7f99fc0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99fc0cc770 0x7f99fc0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 --2- 172.21.15.33:0/3591563302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc159400 0x7f99fc158410 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 >> 172.21.15.33:0/3591563302 conn(0x7f99fc06e4b0 msgr2=0x7f99fc154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:40.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 shutdown_connections 2023-10-06T05:35:40.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:40.986+0000 7f99fb59e700 1 -- 172.21.15.33:0/3591563302 wait complete. 2023-10-06T05:35:40.989 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T05:35:41.005 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T05:35:41.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.195+0000 7f7da499d700 1 Processor -- start 2023-10-06T05:35:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.195+0000 7f7da499d700 1 -- start start 2023-10-06T05:35:41.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7da499d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7da0157320 0x7f7da00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7da499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da00ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7da499d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da00b0840 con 0x7f7da00b13d0 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7da499d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da01585c0 con 0x7f7da0157320 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7da499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00b1ff0 con 0x7f7da01576f0 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7da0157320 0x7f7da00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7da0157320 0x7f7da00adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52244/0 (socket says 172.21.15.33:52244) 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1986864433 learned_addr learned my addr 172.21.15.33:0/1986864433 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9ffff700 1 --1- 172.21.15.33:0/1986864433 >> v1:172.21.15.33:6789/0 conn(0x7f7da00b13d0 0x7f7da00b1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58116/0 (socket says 172.21.15.33:58116) 2023-10-06T05:35:41.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9effd700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da00ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1986864433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 msgr2=0x7f7da00ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da00ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1986864433 >> v1:172.21.15.33:6789/0 conn(0x7f7da00b13d0 legacy=0x7f7da00b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1986864433 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0157c90 con 0x7f7da0157320 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.197+0000 7f7d9effd700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da00ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.198+0000 7f7d9f7fe700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da0157320 0x7f7da00adf10 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f7d900099b0 tx=0x7f7d90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66195e207e8187de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.198+0000 7f7d9e7fc700 1 -- 172.21.15.33:0/1986864433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000b8f0 con 0x7f7da0157320 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.198+0000 7f7d9e7fc700 1 -- 172.21.15.33:0/1986864433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9000ba50 con 0x7f7da0157320 2023-10-06T05:35:41.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.198+0000 7f7d9e7fc700 1 -- 172.21.15.33:0/1986864433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000bdf0 con 0x7f7da0157320 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.198+0000 7f7da499d700 1 -- 172.21.15.33:0/1986864433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da0157320 msgr2=0x7f7da00adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.198+0000 7f7da499d700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da0157320 0x7f7da00adf10 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f7d900099b0 tx=0x7f7d90009250 comp rx=0 tx=0).stop 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.199+0000 7f7da499d700 1 -- 172.21.15.33:0/1986864433 shutdown_connections 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.199+0000 7f7da499d700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da0157320 0x7f7da00adf10 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.199+0000 7f7da499d700 1 --2- 172.21.15.33:0/1986864433 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da00ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.199+0000 7f7da499d700 1 -- 172.21.15.33:0/1986864433 >> 172.21.15.33:0/1986864433 conn(0x7f7da0068090 msgr2=0x7f7da006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.199+0000 7f7da499d700 1 -- 172.21.15.33:0/1986864433 shutdown_connections 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.199+0000 7f7da499d700 1 -- 172.21.15.33:0/1986864433 wait complete. 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.200+0000 7f7da499d700 1 Processor -- start 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.200+0000 7f7da499d700 1 -- start start 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7da499d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da00b13d0 0x7f7da014a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7da499d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7da0157320 0x7f7da0153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7da499d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da0153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7da499d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da014f840 con 0x7f7da00b13d0 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7da499d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da014ec30 con 0x7f7da0157320 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7da499d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da014ef10 con 0x7f7da01576f0 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7d9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da00b13d0 0x7f7da014a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7d9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da00b13d0 0x7f7da014a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52254/0 (socket says 172.21.15.33:52254) 2023-10-06T05:35:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.201+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1119236427 learned_addr learned my addr 172.21.15.33:0/1119236427 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 msgr2=0x7f7da0153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d9f7fe700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7da01576f0 0x7f7da0153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7da0157320 msgr2=0x7f7da0153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d9f7fe700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7da0157320 0x7f7da0153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d9f7fe700 1 -- 172.21.15.33:0/1119236427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900094f0 con 0x7f7da00b13d0 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d9f7fe700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da00b13d0 0x7f7da014a020 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f7da006e190 tx=0x7f7d90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9001a630 con 0x7f7da00b13d0 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9001a7b0 con 0x7f7da00b13d0 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da014f1f0 con 0x7f7da00b13d0 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9001ab70 con 0x7f7da00b13d0 2023-10-06T05:35:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.202+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da014a560 con 0x7f7da00b13d0 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.203+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7da0063ff0 con 0x7f7da00b13d0 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.204+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d900233e0 con 0x7f7da00b13d0 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.205+0000 7f7d877fe700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7d8c066f30 0x7f7d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.205+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7d90085fe0 con 0x7f7da00b13d0 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.205+0000 7f7d877fe700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f7d8c06d3f0 0x7f7d8c06f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.205+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 --> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d8c06feb0 con 0x7f7d8c06d3f0 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.205+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d90086310 con 0x7f7da00b13d0 2023-10-06T05:35:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.205+0000 7f7d9ffff700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f7d8c06d3f0 0x7f7d8c06f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.206+0000 7f7d9effd700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7d8c066f30 0x7f7d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.206+0000 7f7d9ffff700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f7d8c06d3f0 0x7f7d8c06f800 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.206+0000 7f7d9effd700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7d8c066f30 0x7f7d8c0693e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.207+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== osd.1 v2:172.21.15.33:6816/85579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7d8c06feb0 con 0x7f7d8c06d3f0 2023-10-06T05:35:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.221+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 --> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7da006bcf0 con 0x7f7d8c06d3f0 2023-10-06T05:35:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7d877fe700 1 -- 172.21.15.33:0/1119236427 <== osd.1 v2:172.21.15.33:6816/85579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7da006bcf0 con 0x7f7d8c06d3f0 2023-10-06T05:35:41.223 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:41.224 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:41.224 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f7d8c06d3f0 msgr2=0x7f7d8c06f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:6816/85579,v1:172.21.15.33:6817/85579] conn(0x7f7d8c06d3f0 0x7f7d8c06f800 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7d8c066f30 msgr2=0x7f7d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f7d8c066f30 0x7f7d8c0693e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d94008040 comp rx=0 tx=0).stop 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da00b13d0 msgr2=0x7f7da014a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 --2- 172.21.15.33:0/1119236427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da00b13d0 0x7f7da014a020 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f7da006e190 tx=0x7f7d90017000 comp rx=0 tx=0).stop 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7d9ffff700 1 -- 172.21.15.33:0/1119236427 reap_dead start 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 shutdown_connections 2023-10-06T05:35:41.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 >> 172.21.15.33:0/1119236427 conn(0x7f7da0068090 msgr2=0x7f7da014bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.222+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 shutdown_connections 2023-10-06T05:35:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.223+0000 7f7da499d700 1 -- 172.21.15.33:0/1119236427 wait complete. 2023-10-06T05:35:41.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T05:35:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.427+0000 7f632f59e700 1 Processor -- start 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.427+0000 7f632f59e700 1 -- start start 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.428+0000 7f632f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f63300c8090 0x7f6330156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.428+0000 7f632f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330159e10 0x7f6330157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633006a820 con 0x7f6330156880 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63300cc290 con 0x7f63300c8090 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633006bd10 con 0x7f6330159e10 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f63300c8090 0x7f6330156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f63300c8090 0x7f6330156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52268/0 (socket says 172.21.15.33:52268) 2023-10-06T05:35:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 -- 172.21.15.33:0/4094793827 learned_addr learned my addr 172.21.15.33:0/4094793827 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632ed9d700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330159e10 0x7f6330157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 -- 172.21.15.33:0/4094793827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330159e10 msgr2=0x7f6330157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330159e10 0x7f6330157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 -- 172.21.15.33:0/4094793827 >> v1:172.21.15.167:6789/0 conn(0x7f6330156880 legacy=0x7f63300c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632dd9b700 1 -- 172.21.15.33:0/4094793827 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633015a2f0 con 0x7f63300c8090 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.429+0000 7f632ed9d700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330159e10 0x7f6330157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632dd9b700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f6330156c50 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f63240099e0 tx=0x7f6324009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9aeeb190ee07b78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632d59a700 1 -- 172.21.15.33:0/4094793827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400b820 con 0x7f63300c8090 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632d59a700 1 -- 172.21.15.33:0/4094793827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632400b980 con 0x7f63300c8090 2023-10-06T05:35:41.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632d59a700 1 -- 172.21.15.33:0/4094793827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400bd20 con 0x7f63300c8090 2023-10-06T05:35:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632f59e700 1 -- 172.21.15.33:0/4094793827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 msgr2=0x7f6330156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632f59e700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f6330156c50 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f63240099e0 tx=0x7f6324009250 comp rx=0 tx=0).stop 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632f59e700 1 -- 172.21.15.33:0/4094793827 shutdown_connections 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632f59e700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330159e10 0x7f6330157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632f59e700 1 --2- 172.21.15.33:0/4094793827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f6330156c50 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.430+0000 7f632f59e700 1 -- 172.21.15.33:0/4094793827 >> 172.21.15.33:0/4094793827 conn(0x7f633006c560 msgr2=0x7f633006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.431+0000 7f632f59e700 1 -- 172.21.15.33:0/4094793827 shutdown_connections 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.431+0000 7f632f59e700 1 -- 172.21.15.33:0/4094793827 wait complete. 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.432+0000 7f632f59e700 1 Processor -- start 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.432+0000 7f632f59e700 1 -- start start 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f633013dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330156880 0x7f633006e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6330159e10 0x7f633006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63300afd60 con 0x7f63300c8090 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63300af150 con 0x7f6330159e10 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63300af430 con 0x7f6330156880 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f633013dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f633013dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52276/0 (socket says 172.21.15.33:52276) 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632e59c700 1 -- 172.21.15.33:0/1371143228 learned_addr learned my addr 172.21.15.33:0/1371143228 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632dd9b700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330156880 0x7f633006e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.433+0000 7f632ed9d700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6330159e10 0x7f633006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632e59c700 1 -- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330156880 msgr2=0x7f633006e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632e59c700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330156880 0x7f633006e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632e59c700 1 -- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6330159e10 msgr2=0x7f633006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632e59c700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6330159e10 0x7f633006e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632e59c700 1 -- 172.21.15.33:0/1371143228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240094f0 con 0x7f63300c8090 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632e59c700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f633013dac0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f632000c590 tx=0x7f632000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320011590 con 0x7f63300c8090 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320011710 con 0x7f63300c8090 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320011ad0 con 0x7f63300c8090 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.434+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63300af710 con 0x7f63300c8090 2023-10-06T05:35:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.435+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633013e000 con 0x7f63300c8090 2023-10-06T05:35:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.436+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633004c380 con 0x7f63300c8090 2023-10-06T05:35:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.437+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f632001b3b0 con 0x7f63300c8090 2023-10-06T05:35:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.438+0000 7f631e7fc700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6310066f30 0x7f63100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.438+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6320083830 con 0x7f63300c8090 2023-10-06T05:35:41.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.438+0000 7f632dd9b700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6310066f30 0x7f63100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.439+0000 7f632dd9b700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6310066f30 0x7f63100693e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f63240099e0 tx=0x7f6324017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.441+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6320050c10 con 0x7f63300c8090 2023-10-06T05:35:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.645+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f63300afea0 con 0x7f6310066f30 2023-10-06T05:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.646+0000 7f631e7fc700 1 -- 172.21.15.33:0/1371143228 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f632400baf0 con 0x7f6310066f30 2023-10-06T05:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6310066f30 msgr2=0x7f63100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6310066f30 0x7f63100693e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f63240099e0 tx=0x7f6324017000 comp rx=0 tx=0).stop 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 msgr2=0x7f633013dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f633013dac0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f632000c590 tx=0x7f632000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 shutdown_connections 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6310066f30 0x7f63100693e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6330159e10 0x7f633006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63300c8090 0x7f633013dac0 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 --2- 172.21.15.33:0/1371143228 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6330156880 0x7f633006e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 >> 172.21.15.33:0/1371143228 conn(0x7f633006c560 msgr2=0x7f63300caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 shutdown_connections 2023-10-06T05:35:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.648+0000 7f632f59e700 1 -- 172.21.15.33:0/1371143228 wait complete. 2023-10-06T05:35:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T05:35:41.668 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T05:35:41.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:41.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.858+0000 7fd353fff700 1 Processor -- start 2023-10-06T05:35:41.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.859+0000 7fd353fff700 1 -- start start 2023-10-06T05:35:41.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd353fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd354155e70 0x7fd3540aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd353fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd354069ad0 con 0x7fd3540acd00 2023-10-06T05:35:41.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd353fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd354069db0 con 0x7fd354155aa0 2023-10-06T05:35:41.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd353fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35406a090 con 0x7fd354155e70 2023-10-06T05:35:41.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd3527fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd3540acd00 0x7fd3540c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58120/0 (socket says 172.21.15.33:58120) 2023-10-06T05:35:41.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd3527fc700 1 -- 172.21.15.33:0/1531122072 learned_addr learned my addr 172.21.15.33:0/1531122072 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:41.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd3537fe700 1 --2- 172.21.15.33:0/1531122072 >> v2:172.21.15.33:3300/0 conn(0x7fd354155e70 0x7fd3540aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.860+0000 7fd351ffb700 1 -- 172.21.15.33:0/1531122072 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997960894 0 0) 0x7fd354069ad0 con 0x7fd3540acd00 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd351ffb700 1 -- 172.21.15.33:0/1531122072 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd33c003570 con 0x7fd3540acd00 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd3537fe700 1 -- 172.21.15.33:0/1531122072 >> v1:172.21.15.167:6789/0 conn(0x7fd354155aa0 legacy=0x7fd3540ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd3537fe700 1 -- 172.21.15.33:0/1531122072 >> v1:172.21.15.33:6789/0 conn(0x7fd3540acd00 legacy=0x7fd3540c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd3537fe700 1 -- 172.21.15.33:0/1531122072 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35406a370 con 0x7fd354155e70 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd3537fe700 1 --2- 172.21.15.33:0/1531122072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540aef40 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00af70 tx=0x7fd34c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eb5996bae3f803 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd351ffb700 1 -- 172.21.15.33:0/1531122072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00f070 con 0x7fd354155e70 2023-10-06T05:35:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd351ffb700 1 -- 172.21.15.33:0/1531122072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c00a2e0 con 0x7fd354155e70 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.861+0000 7fd351ffb700 1 -- 172.21.15.33:0/1531122072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00cc70 con 0x7fd354155e70 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 -- 172.21.15.33:0/1531122072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 msgr2=0x7fd3540aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 --2- 172.21.15.33:0/1531122072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540aef40 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00af70 tx=0x7fd34c00a890 comp rx=0 tx=0).stop 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 -- 172.21.15.33:0/1531122072 shutdown_connections 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 --2- 172.21.15.33:0/1531122072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540aef40 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 -- 172.21.15.33:0/1531122072 >> 172.21.15.33:0/1531122072 conn(0x7fd354068090 msgr2=0x7fd35406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 -- 172.21.15.33:0/1531122072 shutdown_connections 2023-10-06T05:35:41.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.862+0000 7fd353fff700 1 -- 172.21.15.33:0/1531122072 wait complete. 2023-10-06T05:35:41.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.863+0000 7fd353fff700 1 Processor -- start 2023-10-06T05:35:41.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.863+0000 7fd353fff700 1 -- start start 2023-10-06T05:35:41.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.864+0000 7fd353fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540acd00 0x7fd354142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.864+0000 7fd353fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd354155aa0 0x7fd3540a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.864+0000 7fd353fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.864+0000 7fd353fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541526d0 con 0x7fd354155e70 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.864+0000 7fd353fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354151ac0 con 0x7fd3540acd00 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.864+0000 7fd353fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd354151da0 con 0x7fd354155aa0 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52292/0 (socket says 172.21.15.33:52292) 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 -- 172.21.15.33:0/3721672837 learned_addr learned my addr 172.21.15.33:0/3721672837 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3527fc700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd354155aa0 0x7fd3540a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd352ffd700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540acd00 0x7fd354142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 -- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd354155aa0 msgr2=0x7fd3540a7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd354155aa0 0x7fd3540a7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 -- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540acd00 msgr2=0x7fd354142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3540acd00 0x7fd354142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 -- 172.21.15.33:0/3721672837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c00ab30 con 0x7fd354155e70 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.865+0000 7fd3537fe700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540a8500 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00ab00 tx=0x7fd34c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.866+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c009ad0 con 0x7fd354155e70 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.866+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c009c70 con 0x7fd354155e70 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.866+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c027410 con 0x7fd354155e70 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.866+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd354152080 con 0x7fd354155e70 2023-10-06T05:35:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.866+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd354143380 con 0x7fd354155e70 2023-10-06T05:35:41.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.867+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd354063ff0 con 0x7fd354155e70 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.867+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd34c01d410 con 0x7fd354155e70 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd33affd700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd34c085c50 con 0x7fd354155e70 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd33affd700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7fd34006d3f0 0x7fd34006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 --> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd34006feb0 con 0x7fd34006d3f0 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd34c08a220 con 0x7fd354155e70 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd352ffd700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.869+0000 7fd3527fc700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7fd34006d3f0 0x7fd34006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.870+0000 7fd3527fc700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7fd34006d3f0 0x7fd34006f800 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.870+0000 7fd352ffd700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd340066f30 0x7fd3400693e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd344003a80 tx=0x7fd344007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:41.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.870+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== osd.2 v2:172.21.15.33:6808/85578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd34006feb0 con 0x7fd34006d3f0 2023-10-06T05:35:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.885+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 --> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd3540cc470 con 0x7fd34006d3f0 2023-10-06T05:35:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.885+0000 7fd33affd700 1 -- 172.21.15.33:0/3721672837 <== osd.2 v2:172.21.15.33:6808/85578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd3540cc470 con 0x7fd34006d3f0 2023-10-06T05:35:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:41.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7fd34006d3f0 msgr2=0x7fd34006f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:6808/85578,v1:172.21.15.33:6809/85578] conn(0x7fd34006d3f0 0x7fd34006f800 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd340066f30 msgr2=0x7fd3400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fd340066f30 0x7fd3400693e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd344003a80 tx=0x7fd344007ec0 comp rx=0 tx=0).stop 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 msgr2=0x7fd3540a8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 --2- 172.21.15.33:0/3721672837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd354155e70 0x7fd3540a8500 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00ab00 tx=0x7fd34c019000 comp rx=0 tx=0).stop 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd3537fe700 1 -- 172.21.15.33:0/3721672837 reap_dead start 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 shutdown_connections 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 >> 172.21.15.33:0/3721672837 conn(0x7fd354068090 msgr2=0x7fd354158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 shutdown_connections 2023-10-06T05:35:41.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:41.886+0000 7fd353fff700 1 -- 172.21.15.33:0/3721672837 wait complete. 2023-10-06T05:35:41.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T05:35:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.092+0000 7f9330b9d700 1 Processor -- start 2023-10-06T05:35:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.092+0000 7f9330b9d700 1 -- start start 2023-10-06T05:35:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f9330b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f932c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f9330b9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c06a410 con 0x7f932c1557f0 2023-10-06T05:35:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f9330b9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0cccd0 con 0x7f932c155bc0 2023-10-06T05:35:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f9330b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0c7500 con 0x7f932c0c7070 2023-10-06T05:35:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f932b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f932c1557f0 0x7f932c0c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58130/0 (socket says 172.21.15.33:58130) 2023-10-06T05:35:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f932b7fe700 1 -- 172.21.15.33:0/552275655 learned_addr learned my addr 172.21.15.33:0/552275655 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:42.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f932affd700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f932c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451421288 0 0) 0x7f932c06a410 con 0x7f932c1557f0 2023-10-06T05:35:42.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.094+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9314003570 con 0x7f932c1557f0 2023-10-06T05:35:42.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002958474 0 0) 0x7f9314003570 con 0x7f932c1557f0 2023-10-06T05:35:42.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 msgr2=0x7f932c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:42.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f932c0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 >> v1:172.21.15.167:6789/0 conn(0x7f932c155bc0 legacy=0x7f932c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0c77e0 con 0x7f932c1557f0 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f931c001470 con 0x7f932c1557f0 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 >> v1:172.21.15.33:6789/0 conn(0x7f932c1557f0 legacy=0x7f932c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f9314003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9314003fd0 0x7f9314004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9314004d50 0x7f93140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0c7500 con 0x7f9314004d50 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931400bba0 con 0x7f9314003fd0 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931400df70 con 0x7f932c0c7070 2023-10-06T05:35:42.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932b7fe700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f9314003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932affd700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9314003fd0 0x7f9314004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.095+0000 7f932bfff700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9314004d50 0x7f93140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932b7fe700 1 -- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9314003fd0 msgr2=0x7f9314004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932b7fe700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9314003fd0 0x7f9314004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932b7fe700 1 -- 172.21.15.33:0/552275655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9314004d50 msgr2=0x7f93140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932b7fe700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9314004d50 0x7f93140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932b7fe700 1 -- 172.21.15.33:0/552275655 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314003570 con 0x7f932c0c7070 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932affd700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9314003fd0 0x7f9314004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932bfff700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9314004d50 0x7f93140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932b7fe700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f9314003a90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c2d0 tx=0x7f931c001da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eefea88765f6f68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00cba0 con 0x7f932c0c7070 2023-10-06T05:35:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00cd20 con 0x7f932c0c7070 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.096+0000 7f932a7fc700 1 -- 172.21.15.33:0/552275655 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c005b70 con 0x7f932c0c7070 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f9330b9d700 1 -- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 msgr2=0x7f9314003a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f9330b9d700 1 --2- 172.21.15.33:0/552275655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c0c7070 0x7f9314003a90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c2d0 tx=0x7f931c001da0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f932bfff700 1 -- 172.21.15.33:0/552275655 reap_dead start 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f9330b9d700 1 -- 172.21.15.33:0/552275655 shutdown_connections 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f9330b9d700 1 -- 172.21.15.33:0/552275655 >> 172.21.15.33:0/552275655 conn(0x7f932c06c560 msgr2=0x7f932c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f9330b9d700 1 -- 172.21.15.33:0/552275655 shutdown_connections 2023-10-06T05:35:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.097+0000 7f9330b9d700 1 -- 172.21.15.33:0/552275655 wait complete. 2023-10-06T05:35:42.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.099+0000 7f9330b9d700 1 Processor -- start 2023-10-06T05:35:42.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.099+0000 7f9330b9d700 1 -- start start 2023-10-06T05:35:42.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.099+0000 7f9330b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f932c0c7070 0x7f932c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f9330b9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c15e460 0x7f932c1697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f9330b9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 0x7f932c16a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f9330b9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c14db00 con 0x7f932c169d10 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f9330b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c16a6c0 con 0x7f932c0c7070 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f9330b9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c15ebe0 con 0x7f932c15e460 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 0x7f932c16a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 0x7f932c16a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52314/0 (socket says 172.21.15.33:52314) 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 -- 172.21.15.33:0/3180729710 learned_addr learned my addr 172.21.15.33:0/3180729710 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932b7fe700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f932c0c7070 0x7f932c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932affd700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c15e460 0x7f932c1697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 -- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c15e460 msgr2=0x7f932c1697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c15e460 0x7f932c1697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 -- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f932c0c7070 msgr2=0x7f932c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f932c0c7070 0x7f932c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.100+0000 7f932bfff700 1 -- 172.21.15.33:0/3180729710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0af020 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.101+0000 7f932bfff700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 0x7f932c16a180 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f932c1594e0 tx=0x7f9324009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.101+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400da70 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.101+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400dbf0 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.101+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c15d1b0 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.101+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240133f0 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.101+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c15d770 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.102+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c0af240 con 0x7f932c169d10 2023-10-06T05:35:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.103+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9324003030 con 0x7f932c169d10 2023-10-06T05:35:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.104+0000 7f93137fe700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9318066f30 0x7f93180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.104+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9324084670 con 0x7f932c169d10 2023-10-06T05:35:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.104+0000 7f932b7fe700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9318066f30 0x7f93180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.105+0000 7f932b7fe700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9318066f30 0x7f93180693e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f932c15b650 tx=0x7f931c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.108+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f932c0af240 con 0x7f932c169d10 2023-10-06T05:35:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.310+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f932c04c380 con 0x7f9318066f30 2023-10-06T05:35:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.311+0000 7f93137fe700 1 -- 172.21.15.33:0/3180729710 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f932c04c380 con 0x7f9318066f30 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9318066f30 msgr2=0x7f93180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9318066f30 0x7f93180693e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f932c15b650 tx=0x7f931c015000 comp rx=0 tx=0).stop 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 msgr2=0x7f932c16a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 0x7f932c16a180 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f932c1594e0 tx=0x7f9324009040 comp rx=0 tx=0).stop 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 shutdown_connections 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f9318066f30 0x7f93180693e0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c169d10 0x7f932c16a180 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f932c0c7070 0x7f932c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 --2- 172.21.15.33:0/3180729710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f932c15e460 0x7f932c1697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 >> 172.21.15.33:0/3180729710 conn(0x7f932c06c560 msgr2=0x7f932c0c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 shutdown_connections 2023-10-06T05:35:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.313+0000 7f9330b9d700 1 -- 172.21.15.33:0/3180729710 wait complete. 2023-10-06T05:35:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T05:35:42.332 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T05:35:42.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.522+0000 7f8438d9d700 1 Processor -- start 2023-10-06T05:35:42.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.522+0000 7f8438d9d700 1 -- start start 2023-10-06T05:35:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.523+0000 7f8438d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f84341590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.523+0000 7f8438d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f843415a740 0x7f8434159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.523+0000 7f8438d9d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843415d280 con 0x7f843406dfd0 2023-10-06T05:35:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.523+0000 7f8438d9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843415e290 con 0x7f843415a740 2023-10-06T05:35:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.523+0000 7f8438d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843415ab90 con 0x7f843406dc00 2023-10-06T05:35:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8432ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f843406dfd0 0x7f843415a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58134/0 (socket says 172.21.15.33:58134) 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8433fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f843415a740 0x7f8434159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8432ffd700 1 -- 172.21.15.33:0/4281466887 learned_addr learned my addr 172.21.15.33:0/4281466887 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f84337fe700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f84341590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8433fff700 1 -- 172.21.15.33:0/4281466887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 msgr2=0x7f84341590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8433fff700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f84341590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8433fff700 1 -- 172.21.15.33:0/4281466887 >> v1:172.21.15.33:6789/0 conn(0x7f843406dfd0 legacy=0x7f843415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8433fff700 1 -- 172.21.15.33:0/4281466887 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843415de00 con 0x7f843415a740 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f84337fe700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f84341590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f8433fff700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843415a740 0x7f8434159620 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f842c009e90 tx=0x7f842c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb9d0dfb79cd82c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.524+0000 7f84327fc700 1 -- 172.21.15.33:0/4281466887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00e070 con 0x7f843415a740 2023-10-06T05:35:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f84327fc700 1 -- 172.21.15.33:0/4281466887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c00bbf0 con 0x7f843415a740 2023-10-06T05:35:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f84327fc700 1 -- 172.21.15.33:0/4281466887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c012430 con 0x7f843415a740 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 -- 172.21.15.33:0/4281466887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843415a740 msgr2=0x7f8434159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843415a740 0x7f8434159620 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f842c009e90 tx=0x7f842c009780 comp rx=0 tx=0).stop 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 -- 172.21.15.33:0/4281466887 shutdown_connections 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843415a740 0x7f8434159620 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 --2- 172.21.15.33:0/4281466887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f84341590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 -- 172.21.15.33:0/4281466887 >> 172.21.15.33:0/4281466887 conn(0x7f84340d0550 msgr2=0x7f84340d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 -- 172.21.15.33:0/4281466887 shutdown_connections 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.525+0000 7f8438d9d700 1 -- 172.21.15.33:0/4281466887 wait complete. 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.527+0000 7f8438d9d700 1 Processor -- start 2023-10-06T05:35:42.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.527+0000 7f8438d9d700 1 -- start start 2023-10-06T05:35:42.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8438d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f843414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8438d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843406dfd0 0x7f8434155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8438d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f843415a740 0x7f8434156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8438d9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84340c7600 con 0x7f843406dfd0 2023-10-06T05:35:42.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8438d9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84340b1510 con 0x7f843415a740 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8438d9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8434152cf0 con 0x7f843406dc00 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8432ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843406dfd0 0x7f8434155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8432ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843406dfd0 0x7f8434155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52342/0 (socket says 172.21.15.33:52342) 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8432ffd700 1 -- 172.21.15.33:0/3332329949 learned_addr learned my addr 172.21.15.33:0/3332329949 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f84337fe700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f843414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.528+0000 7f8433fff700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f843415a740 0x7f8434156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8432ffd700 1 -- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 msgr2=0x7f843414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8432ffd700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f843406dc00 0x7f843414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8432ffd700 1 -- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f843415a740 msgr2=0x7f8434156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8432ffd700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f843415a740 0x7f8434156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8432ffd700 1 -- 172.21.15.33:0/3332329949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c009a20 con 0x7f843406dfd0 2023-10-06T05:35:42.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8432ffd700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843406dfd0 0x7f8434155b40 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f842800c560 tx=0x7f8428001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428005b30 con 0x7f843406dfd0 2023-10-06T05:35:42.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428005c90 con 0x7f843406dfd0 2023-10-06T05:35:42.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428007320 con 0x7f843406dfd0 2023-10-06T05:35:42.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84340b1230 con 0x7f843406dfd0 2023-10-06T05:35:42.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.529+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434152980 con 0x7f843406dfd0 2023-10-06T05:35:42.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.530+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8434063ff0 con 0x7f843406dfd0 2023-10-06T05:35:42.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.531+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f842801b3b0 con 0x7f843406dfd0 2023-10-06T05:35:42.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.532+0000 7f841b7fe700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8420066f30 0x7f84200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.532+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f84280837d0 con 0x7f843406dfd0 2023-10-06T05:35:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.532+0000 7f841b7fe700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f842006d3f0 0x7f842006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.532+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 --> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f842006feb0 con 0x7f842006d3f0 2023-10-06T05:35:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.532+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8428085cd0 con 0x7f843406dfd0 2023-10-06T05:35:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.533+0000 7f84337fe700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8420066f30 0x7f84200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.533+0000 7f8433fff700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f842006d3f0 0x7f842006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.533+0000 7f84337fe700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8420066f30 0x7f84200693e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8424006f00 tx=0x7f8424008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.533+0000 7f8433fff700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f842006d3f0 0x7f842006f800 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.534+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== osd.3 v2:172.21.15.33:6824/85580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f842006feb0 con 0x7f842006d3f0 2023-10-06T05:35:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 --> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f84340cccd0 con 0x7f842006d3f0 2023-10-06T05:35:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f841b7fe700 1 -- 172.21.15.33:0/3332329949 <== osd.3 v2:172.21.15.33:6824/85580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f84340cccd0 con 0x7f842006d3f0 2023-10-06T05:35:42.551 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:42.551 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:42.551 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f842006d3f0 msgr2=0x7f842006f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:6824/85580,v1:172.21.15.33:6825/85580] conn(0x7f842006d3f0 0x7f842006f800 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8420066f30 msgr2=0x7f84200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8420066f30 0x7f84200693e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8424006f00 tx=0x7f8424008040 comp rx=0 tx=0).stop 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843406dfd0 msgr2=0x7f8434155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 --2- 172.21.15.33:0/3332329949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f843406dfd0 0x7f8434155b40 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f842800c560 tx=0x7f8428001020 comp rx=0 tx=0).stop 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8433fff700 1 -- 172.21.15.33:0/3332329949 reap_dead start 2023-10-06T05:35:42.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.549+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 shutdown_connections 2023-10-06T05:35:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.550+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 >> 172.21.15.33:0/3332329949 conn(0x7f84340d0550 msgr2=0x7f84340c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.550+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 shutdown_connections 2023-10-06T05:35:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.550+0000 7f8438d9d700 1 -- 172.21.15.33:0/3332329949 wait complete. 2023-10-06T05:35:42.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T05:35:42.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.754+0000 7ffb6eb9d700 1 Processor -- start 2023-10-06T05:35:42.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.754+0000 7ffb6eb9d700 1 -- start start 2023-10-06T05:35:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.755+0000 7ffb6eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb68159400 0x7ffb680cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.755+0000 7ffb6eb9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ffb681597d0 0x7ffb680cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.755+0000 7ffb6eb9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb680cc770 0x7ffb6806c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.755+0000 7ffb6eb9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb680d0ed0 con 0x7ffb680cc770 2023-10-06T05:35:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6eb9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb680cb6d0 con 0x7ffb681597d0 2023-10-06T05:35:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680cb9b0 con 0x7ffb68159400 2023-10-06T05:35:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb680cc770 0x7ffb6806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb680cc770 0x7ffb6806c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52358/0 (socket says 172.21.15.33:52358) 2023-10-06T05:35:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 -- 172.21.15.33:0/1438514980 learned_addr learned my addr 172.21.15.33:0/1438514980 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6db9b700 1 --2- 172.21.15.33:0/1438514980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb68159400 0x7ffb680cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6d39a700 1 --2- 172.21.15.33:0/1438514980 >> v2:172.21.15.167:3300/0 conn(0x7ffb681597d0 0x7ffb680cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 -- 172.21.15.33:0/1438514980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb68159400 msgr2=0x7ffb680cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:42.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 --2- 172.21.15.33:0/1438514980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb68159400 0x7ffb680cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 -- 172.21.15.33:0/1438514980 >> v2:172.21.15.167:3300/0 conn(0x7ffb681597d0 msgr2=0x7ffb680cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:42.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 --2- 172.21.15.33:0/1438514980 >> v2:172.21.15.167:3300/0 conn(0x7ffb681597d0 0x7ffb680cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.756+0000 7ffb6e39c700 1 -- 172.21.15.33:0/1438514980 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680cbc90 con 0x7ffb680cc770 2023-10-06T05:35:42.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.757+0000 7ffb6e39c700 1 --2- 172.21.15.33:0/1438514980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb6806c560 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7ffb68071250 tx=0x7ffb60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94f0d41a4a5b86a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.757+0000 7ffb6cb99700 1 -- 172.21.15.33:0/1438514980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000e070 con 0x7ffb680cc770 2023-10-06T05:35:42.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.757+0000 7ffb6cb99700 1 -- 172.21.15.33:0/1438514980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6000b970 con 0x7ffb680cc770 2023-10-06T05:35:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.757+0000 7ffb6cb99700 1 -- 172.21.15.33:0/1438514980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600092e0 con 0x7ffb680cc770 2023-10-06T05:35:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/1438514980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 msgr2=0x7ffb6806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/1438514980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb6806c560 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7ffb68071250 tx=0x7ffb60009840 comp rx=0 tx=0).stop 2023-10-06T05:35:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/1438514980 shutdown_connections 2023-10-06T05:35:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/1438514980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb6806c560 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/1438514980 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb68159400 0x7ffb680cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/1438514980 >> v2:172.21.15.167:3300/0 conn(0x7ffb681597d0 0x7ffb680cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/1438514980 >> 172.21.15.33:0/1438514980 conn(0x7ffb6806e4b0 msgr2=0x7ffb680d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:42.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/1438514980 shutdown_connections 2023-10-06T05:35:42.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.758+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/1438514980 wait complete. 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.759+0000 7ffb6eb9d700 1 Processor -- start 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.759+0000 7ffb6eb9d700 1 -- start start 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.760+0000 7ffb6eb9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb68151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.760+0000 7ffb6eb9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb68159400 0x7ffb68158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.760+0000 7ffb6eb9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb681597d0 0x7ffb680c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.760+0000 7ffb6eb9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680c75d0 con 0x7ffb680cc770 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.760+0000 7ffb6eb9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680c7b90 con 0x7ffb68159400 2023-10-06T05:35:42.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.760+0000 7ffb6eb9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68158730 con 0x7ffb681597d0 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb68151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb68151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52374/0 (socket says 172.21.15.33:52374) 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 -- 172.21.15.33:0/3604477887 learned_addr learned my addr 172.21.15.33:0/3604477887 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6d39a700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb68159400 0x7ffb68158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6e39c700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb681597d0 0x7ffb680c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 -- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb681597d0 msgr2=0x7ffb680c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb681597d0 0x7ffb680c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 -- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb68159400 msgr2=0x7ffb68158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb68159400 0x7ffb68158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 -- 172.21.15.33:0/3604477887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009ae0 con 0x7ffb680cc770 2023-10-06T05:35:42.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.761+0000 7ffb6db9b700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb68151b30 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400ecb0 tx=0x7ffb640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.762+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400c850 con 0x7ffb680cc770 2023-10-06T05:35:42.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.762+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6400c9b0 con 0x7ffb680cc770 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.762+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb68158a10 con 0x7ffb680cc770 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.762+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400cd50 con 0x7ffb680cc770 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.762+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb681522c0 con 0x7ffb680cc770 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.763+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb680c7e70 con 0x7ffb680cc770 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.764+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb64013680 con 0x7ffb680cc770 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.765+0000 7ffb5dffb700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb4c066fb0 0x7ffb4c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:42.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.765+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ffb640846c0 con 0x7ffb680cc770 2023-10-06T05:35:42.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.766+0000 7ffb6d39a700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb4c066fb0 0x7ffb4c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:42.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.766+0000 7ffb6d39a700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb4c066fb0 0x7ffb4c069460 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ffb680b12e0 tx=0x7ffb580091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:42.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.770+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb64051a00 con 0x7ffb680cc770 2023-10-06T05:35:42.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.974+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb6804c380 con 0x7ffb4c066fb0 2023-10-06T05:35:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.974+0000 7ffb5dffb700 1 -- 172.21.15.33:0/3604477887 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffb6804c380 con 0x7ffb4c066fb0 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb4c066fb0 msgr2=0x7ffb4c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb4c066fb0 0x7ffb4c069460 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ffb680b12e0 tx=0x7ffb580091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 msgr2=0x7ffb68151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb68151b30 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400ecb0 tx=0x7ffb640091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 shutdown_connections 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7ffb4c066fb0 0x7ffb4c069460 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb681597d0 0x7ffb680c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb680cc770 0x7ffb68151b30 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 --2- 172.21.15.33:0/3604477887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb68159400 0x7ffb68158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 >> 172.21.15.33:0/3604477887 conn(0x7ffb6806e4b0 msgr2=0x7ffb680c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 shutdown_connections 2023-10-06T05:35:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:42.977+0000 7ffb6eb9d700 1 -- 172.21.15.33:0/3604477887 wait complete. 2023-10-06T05:35:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T05:35:42.996 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T05:35:42.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:43.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.186+0000 7f6a5f19d700 1 Processor -- start 2023-10-06T05:35:43.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.186+0000 7f6a5f19d700 1 -- start start 2023-10-06T05:35:43.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.187+0000 7f6a5f19d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a580ad010 0x7f6a5806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.187+0000 7f6a5f19d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6a5815acd0 0x7f6a58158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.187+0000 7f6a5f19d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5806b1f0 con 0x7f6a580ad3e0 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.187+0000 7f6a5f19d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5806a3f0 con 0x7f6a580ad010 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.187+0000 7f6a5f19d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5806a5e0 con 0x7f6a5815acd0 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.187+0000 7f6a5e19b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a580ad010 0x7f6a5806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5d99a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6a580ad3e0 0x7f6a5815a5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58140/0 (socket says 172.21.15.33:58140) 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5d99a700 1 -- 172.21.15.33:0/2829572041 learned_addr learned my addr 172.21.15.33:0/2829572041 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5e99c700 1 --2- 172.21.15.33:0/2829572041 >> v2:172.21.15.167:3300/0 conn(0x7f6a5815acd0 0x7f6a58158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5e19b700 1 -- 172.21.15.33:0/2829572041 >> v2:172.21.15.167:3300/0 conn(0x7f6a5815acd0 msgr2=0x7f6a58158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5e19b700 1 --2- 172.21.15.33:0/2829572041 >> v2:172.21.15.167:3300/0 conn(0x7f6a5815acd0 0x7f6a58158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5e19b700 1 -- 172.21.15.33:0/2829572041 >> v1:172.21.15.33:6789/0 conn(0x7f6a580ad3e0 legacy=0x7f6a5815a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5e19b700 1 -- 172.21.15.33:0/2829572041 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5806a8c0 con 0x7f6a580ad010 2023-10-06T05:35:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5d199700 1 -- 172.21.15.33:0/2829572041 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170035538 0 0) 0x7f6a5806b1f0 con 0x7f6a580ad3e0 2023-10-06T05:35:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.188+0000 7f6a5e19b700 1 --2- 172.21.15.33:0/2829572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a5806ea00 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f6a5806beb0 tx=0x7f6a540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6302156aafca4c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5d199700 1 -- 172.21.15.33:0/2829572041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400b740 con 0x7f6a580ad010 2023-10-06T05:35:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5d199700 1 -- 172.21.15.33:0/2829572041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5400b8a0 con 0x7f6a580ad010 2023-10-06T05:35:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5d199700 1 -- 172.21.15.33:0/2829572041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400bc40 con 0x7f6a580ad010 2023-10-06T05:35:43.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 -- 172.21.15.33:0/2829572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 msgr2=0x7f6a5806ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 --2- 172.21.15.33:0/2829572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a5806ea00 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f6a5806beb0 tx=0x7f6a540091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 -- 172.21.15.33:0/2829572041 shutdown_connections 2023-10-06T05:35:43.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 --2- 172.21.15.33:0/2829572041 >> v2:172.21.15.167:3300/0 conn(0x7f6a5815acd0 0x7f6a58158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 --2- 172.21.15.33:0/2829572041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a5806ea00 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 -- 172.21.15.33:0/2829572041 >> 172.21.15.33:0/2829572041 conn(0x7f6a58068090 msgr2=0x7f6a5806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 -- 172.21.15.33:0/2829572041 shutdown_connections 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.189+0000 7f6a5f19d700 1 -- 172.21.15.33:0/2829572041 wait complete. 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.191+0000 7f6a5f19d700 1 Processor -- start 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.191+0000 7f6a5f19d700 1 -- start start 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5f19d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a580a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5f19d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ad3e0 0x7f6a58150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5f19d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a5815acd0 0x7f6a580c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5f19d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580ca080 con 0x7f6a580ad010 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5f19d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58150f70 con 0x7f6a5815acd0 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5f19d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58151a20 con 0x7f6a580ad3e0 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a580a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a580a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52402/0 (socket says 172.21.15.33:52402) 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 -- 172.21.15.33:0/960025704 learned_addr learned my addr 172.21.15.33:0/960025704 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e99c700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a5815acd0 0x7f6a580c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5d99a700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ad3e0 0x7f6a58150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 -- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ad3e0 msgr2=0x7f6a58150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ad3e0 0x7f6a58150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 -- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a5815acd0 msgr2=0x7f6a580c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a5815acd0 0x7f6a580c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e19b700 1 -- 172.21.15.33:0/960025704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009420 con 0x7f6a580ad010 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.192+0000 7f6a5e99c700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a5815acd0 0x7f6a580c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:43.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.193+0000 7f6a5e19b700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a580a9890 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f6a5806e190 tx=0x7f6a54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.193+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54011420 con 0x7f6a580ad010 2023-10-06T05:35:43.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.193+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54011580 con 0x7f6a580ad010 2023-10-06T05:35:43.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.193+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54011940 con 0x7f6a580ad010 2023-10-06T05:35:43.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.193+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580ca290 con 0x7f6a580ad010 2023-10-06T05:35:43.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.193+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a580ca850 con 0x7f6a580ad010 2023-10-06T05:35:43.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.194+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a58063ff0 con 0x7f6a580ad010 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.195+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a54011cb0 con 0x7f6a580ad010 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.196+0000 7f6a4e7fc700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a3c066f30 0x7f6a3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.196+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a54085dc0 con 0x7f6a580ad010 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.196+0000 7f6a4e7fc700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f6a3c06d3f0 0x7f6a3c06f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.196+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 --> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a3c06feb0 con 0x7f6a3c06d3f0 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.196+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a540882c0 con 0x7f6a580ad010 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.196+0000 7f6a5d99a700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a3c066f30 0x7f6a3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.197+0000 7f6a5e99c700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f6a3c06d3f0 0x7f6a3c06f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.197+0000 7f6a5d99a700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a3c066f30 0x7f6a3c0693e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6a580cbd30 tx=0x7f6a48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.197+0000 7f6a5e99c700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f6a3c06d3f0 0x7f6a3c06f800 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.198+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== osd.4 v2:172.21.15.167:6800/85769 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6a3c06feb0 con 0x7f6a3c06d3f0 2023-10-06T05:35:43.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.213+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 --> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6a58064790 con 0x7f6a3c06d3f0 2023-10-06T05:35:43.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.214+0000 7f6a4e7fc700 1 -- 172.21.15.33:0/960025704 <== osd.4 v2:172.21.15.167:6800/85769 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6a58064790 con 0x7f6a3c06d3f0 2023-10-06T05:35:43.216 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:43.216 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:43.216 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.214+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f6a3c06d3f0 msgr2=0x7f6a3c06f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.214+0000 7f6a5f19d700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6800/85769,v1:172.21.15.167:6801/85769] conn(0x7f6a3c06d3f0 0x7f6a3c06f800 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a3c066f30 msgr2=0x7f6a3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f6a3c066f30 0x7f6a3c0693e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6a580cbd30 tx=0x7f6a48009040 comp rx=0 tx=0).stop 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 msgr2=0x7f6a580a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 --2- 172.21.15.33:0/960025704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580ad010 0x7f6a580a9890 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f6a5806e190 tx=0x7f6a54017000 comp rx=0 tx=0).stop 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5e99c700 1 -- 172.21.15.33:0/960025704 reap_dead start 2023-10-06T05:35:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 shutdown_connections 2023-10-06T05:35:43.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 >> 172.21.15.33:0/960025704 conn(0x7f6a58068090 msgr2=0x7f6a58158760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:43.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 shutdown_connections 2023-10-06T05:35:43.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.215+0000 7f6a5f19d700 1 -- 172.21.15.33:0/960025704 wait complete. 2023-10-06T05:35:43.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T05:35:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.417+0000 7f285639d700 1 Processor -- start 2023-10-06T05:35:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.417+0000 7f285639d700 1 -- start start 2023-10-06T05:35:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f285639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501596b0 0x7f28500cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f285639d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500b03a0 con 0x7f285006c560 2023-10-06T05:35:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f285639d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500b0f40 con 0x7f28501592e0 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f285639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285006ccd0 con 0x7f28501596b0 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2855b9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f285006c560 0x7f28500aeca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58154/0 (socket says 172.21.15.33:58154) 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2855b9c700 1 -- 172.21.15.33:0/3201747708 learned_addr learned my addr 172.21.15.33:0/3201747708 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2854b9a700 1 --2- 172.21.15.33:0/3201747708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501596b0 0x7f28500cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445034090 0 0) 0x7f28500b03a0 con 0x7f285006c560 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f283c003570 con 0x7f285006c560 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535138524 0 0) 0x7f28500b0f40 con 0x7f28501592e0 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.419+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28500b03a0 con 0x7f28501592e0 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2854b9a700 1 -- 172.21.15.33:0/3201747708 >> v1:172.21.15.167:6789/0 conn(0x7f28501592e0 legacy=0x7f28500b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2854b9a700 1 -- 172.21.15.33:0/3201747708 >> v1:172.21.15.33:6789/0 conn(0x7f285006c560 legacy=0x7f28500aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2854b9a700 1 -- 172.21.15.33:0/3201747708 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285006cfb0 con 0x7f28501596b0 2023-10-06T05:35:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2854b9a700 1 --2- 172.21.15.33:0/3201747708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501596b0 0x7f28500cdd30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2840009910 tx=0x7f28400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace93af9549b772d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011630 con 0x7f28501596b0 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840011790 con 0x7f28501596b0 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.420+0000 7f2847fff700 1 -- 172.21.15.33:0/3201747708 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011b30 con 0x7f28501596b0 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 -- 172.21.15.33:0/3201747708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501596b0 msgr2=0x7f28500cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 --2- 172.21.15.33:0/3201747708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501596b0 0x7f28500cdd30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2840009910 tx=0x7f28400091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 -- 172.21.15.33:0/3201747708 shutdown_connections 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 --2- 172.21.15.33:0/3201747708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501596b0 0x7f28500cdd30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 -- 172.21.15.33:0/3201747708 >> 172.21.15.33:0/3201747708 conn(0x7f28500675f0 msgr2=0x7f28500679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 -- 172.21.15.33:0/3201747708 shutdown_connections 2023-10-06T05:35:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.421+0000 7f285639d700 1 -- 172.21.15.33:0/3201747708 wait complete. 2023-10-06T05:35:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.422+0000 7f285639d700 1 Processor -- start 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 -- start start 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f285006c560 0x7f2850145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501592e0 0x7f2850158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 0x7f285015a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500af600 con 0x7f28501596b0 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500ae9f0 con 0x7f285006c560 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.423+0000 7f285639d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500aecd0 con 0x7f28501592e0 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 0x7f285015a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 0x7f285015a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52406/0 (socket says 172.21.15.33:52406) 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 -- 172.21.15.33:0/2531648331 learned_addr learned my addr 172.21.15.33:0/2531648331 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2854b9a700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501592e0 0x7f2850158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f285539b700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f285006c560 0x7f2850145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 -- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501592e0 msgr2=0x7f2850158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501592e0 0x7f2850158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 -- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f285006c560 msgr2=0x7f2850145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f285006c560 0x7f2850145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 -- 172.21.15.33:0/2531648331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840009420 con 0x7f28501596b0 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.424+0000 7f2855b9c700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 0x7f285015a270 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f284800cc60 tx=0x7f2848001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.425+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800a8b0 con 0x7f28501596b0 2023-10-06T05:35:43.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.425+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284800aa30 con 0x7f28501596b0 2023-10-06T05:35:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.425+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28500aefb0 con 0x7f28501596b0 2023-10-06T05:35:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.425+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800adf0 con 0x7f28501596b0 2023-10-06T05:35:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.425+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28501460f0 con 0x7f28501596b0 2023-10-06T05:35:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.426+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28500b1a90 con 0x7f28501596b0 2023-10-06T05:35:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.427+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2848007300 con 0x7f28501596b0 2023-10-06T05:35:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.427+0000 7f28457fa700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2834066f30 0x7f28340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.428+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2848084790 con 0x7f28501596b0 2023-10-06T05:35:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.428+0000 7f285539b700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2834066f30 0x7f28340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.429+0000 7f285539b700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2834066f30 0x7f28340693e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f284c007fd0 tx=0x7f284c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.431+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28480555e0 con 0x7f28501596b0 2023-10-06T05:35:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.634+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f285004c380 con 0x7f2834066f30 2023-10-06T05:35:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.635+0000 7f28457fa700 1 -- 172.21.15.33:0/2531648331 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f285004c380 con 0x7f2834066f30 2023-10-06T05:35:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2834066f30 msgr2=0x7f28340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2834066f30 0x7f28340693e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f284c007fd0 tx=0x7f284c007d50 comp rx=0 tx=0).stop 2023-10-06T05:35:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 msgr2=0x7f285015a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 0x7f285015a270 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f284800cc60 tx=0x7f2848001ae0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 shutdown_connections 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f2834066f30 0x7f28340693e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28501596b0 0x7f285015a270 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f285006c560 0x7f2850145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 --2- 172.21.15.33:0/2531648331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28501592e0 0x7f2850158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 >> 172.21.15.33:0/2531648331 conn(0x7f28500675f0 msgr2=0x7f2850153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.637+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 shutdown_connections 2023-10-06T05:35:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.638+0000 7f285639d700 1 -- 172.21.15.33:0/2531648331 wait complete. 2023-10-06T05:35:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T05:35:43.657 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T05:35:43.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.845+0000 7f571e7ff700 1 Processor -- start 2023-10-06T05:35:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.846+0000 7f571e7ff700 1 -- start start 2023-10-06T05:35:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f571e7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f571806a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f571e7ff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57180c77f0 0x7f57180c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f571e7ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57180af170 con 0x7f57180c92c0 2023-10-06T05:35:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f571e7ff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571806b840 con 0x7f57180c77f0 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f571e7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57180afa10 con 0x7f571815ab90 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f57177fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57180c77f0 0x7f57180c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f57177fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57180c77f0 0x7f57180c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52414/0 (socket says 172.21.15.33:52414) 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f57177fe700 1 -- 172.21.15.33:0/4015454154 learned_addr learned my addr 172.21.15.33:0/4015454154 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.847+0000 7f5717fff700 1 --2- 172.21.15.33:0/4015454154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f571806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f57177fe700 1 -- 172.21.15.33:0/4015454154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 msgr2=0x7f571806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f57177fe700 1 --2- 172.21.15.33:0/4015454154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f571806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f57177fe700 1 -- 172.21.15.33:0/4015454154 >> v1:172.21.15.167:6789/0 conn(0x7f57180c92c0 legacy=0x7f57180cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f57177fe700 1 -- 172.21.15.33:0/4015454154 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57180c96c0 con 0x7f57180c77f0 2023-10-06T05:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f57177fe700 1 --2- 172.21.15.33:0/4015454154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c77f0 0x7f57180c8d80 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f570c0099b0 tx=0x7f570c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2732171b764e6bf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f5716ffd700 1 -- 172.21.15.33:0/4015454154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00b820 con 0x7f57180c77f0 2023-10-06T05:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f5716ffd700 1 -- 172.21.15.33:0/4015454154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c00b980 con 0x7f57180c77f0 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.848+0000 7f5716ffd700 1 -- 172.21.15.33:0/4015454154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00bd20 con 0x7f57180c77f0 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 -- 172.21.15.33:0/4015454154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c77f0 msgr2=0x7f57180c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 --2- 172.21.15.33:0/4015454154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c77f0 0x7f57180c8d80 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f570c0099b0 tx=0x7f570c009250 comp rx=0 tx=0).stop 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 -- 172.21.15.33:0/4015454154 shutdown_connections 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 --2- 172.21.15.33:0/4015454154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f571806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 --2- 172.21.15.33:0/4015454154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c77f0 0x7f57180c8d80 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 -- 172.21.15.33:0/4015454154 >> 172.21.15.33:0/4015454154 conn(0x7f571806e630 msgr2=0x7f571806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 -- 172.21.15.33:0/4015454154 shutdown_connections 2023-10-06T05:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.849+0000 7f571e7ff700 1 -- 172.21.15.33:0/4015454154 wait complete. 2023-10-06T05:35:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.850+0000 7f571e7ff700 1 Processor -- start 2023-10-06T05:35:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.850+0000 7f571e7ff700 1 -- start start 2023-10-06T05:35:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.851+0000 7f571e7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57180c77f0 0x7f57181698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.851+0000 7f571e7ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c92c0 0x7f57180c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.851+0000 7f571e7ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f5718146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.851+0000 7f571e7ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5718169de0 con 0x7f57180c92c0 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.851+0000 7f571e7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571816a8a0 con 0x7f57180c77f0 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.851+0000 7f571e7ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57180ca120 con 0x7f571815ab90 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c92c0 0x7f57180c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c92c0 0x7f57180c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52428/0 (socket says 172.21.15.33:52428) 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 -- 172.21.15.33:0/3753182931 learned_addr learned my addr 172.21.15.33:0/3753182931 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f5717fff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57180c77f0 0x7f57181698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f571cd9c700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f5718146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 -- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 msgr2=0x7f5718146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f571815ab90 0x7f5718146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 -- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57180c77f0 msgr2=0x7f57181698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57180c77f0 0x7f57181698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 -- 172.21.15.33:0/3753182931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0094f0 con 0x7f57180c92c0 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.852+0000 7f57177fe700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c92c0 0x7f57180c6220 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f570c0094c0 tx=0x7f570c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.853+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c011e00 con 0x7f57180c92c0 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.853+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c01b470 con 0x7f57180c92c0 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.853+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c01b830 con 0x7f57180c92c0 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.853+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57180c6760 con 0x7f57180c92c0 2023-10-06T05:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.853+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571816a2e0 con 0x7f57180c92c0 2023-10-06T05:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.853+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57180718b0 con 0x7f57180c92c0 2023-10-06T05:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.855+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f570c01bc20 con 0x7f57180c92c0 2023-10-06T05:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f56fffff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5704066f30 0x7f57040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f570c086560 con 0x7f57180c92c0 2023-10-06T05:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f56fffff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7f570406d3f0 0x7f570406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 --> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f570406feb0 con 0x7f570406d3f0 2023-10-06T05:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f570c086850 con 0x7f57180c92c0 2023-10-06T05:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f5717fff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5704066f30 0x7f57040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.856+0000 7f571cd9c700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7f570406d3f0 0x7f570406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.857+0000 7f5717fff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5704066f30 0x7f57040693e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5708006fd0 tx=0x7f5708008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.857+0000 7f571cd9c700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7f570406d3f0 0x7f570406f800 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.858+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== osd.5 v2:172.21.15.167:6816/85770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f570406feb0 con 0x7f570406d3f0 2023-10-06T05:35:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.872+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 --> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f57180c8020 con 0x7f570406d3f0 2023-10-06T05:35:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.873+0000 7f56fffff700 1 -- 172.21.15.33:0/3753182931 <== osd.5 v2:172.21.15.167:6816/85770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f57180c8020 con 0x7f570406d3f0 2023-10-06T05:35:43.876 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:43.877 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:43.877 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7f570406d3f0 msgr2=0x7f570406f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6816/85770,v1:172.21.15.167:6817/85770] conn(0x7f570406d3f0 0x7f570406f800 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5704066f30 msgr2=0x7f57040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f5704066f30 0x7f57040693e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5708006fd0 tx=0x7f5708008040 comp rx=0 tx=0).stop 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c92c0 msgr2=0x7f57180c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 --2- 172.21.15.33:0/3753182931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57180c92c0 0x7f57180c6220 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f570c0094c0 tx=0x7f570c016000 comp rx=0 tx=0).stop 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571cd9c700 1 -- 172.21.15.33:0/3753182931 reap_dead start 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 shutdown_connections 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 >> 172.21.15.33:0/3753182931 conn(0x7f571806e630 msgr2=0x7f5718157420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 shutdown_connections 2023-10-06T05:35:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:43.874+0000 7f571e7ff700 1 -- 172.21.15.33:0/3753182931 wait complete. 2023-10-06T05:35:43.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T05:35:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.081+0000 7fac7199d700 1 Processor -- start 2023-10-06T05:35:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.081+0000 7fac7199d700 1 -- start start 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.082+0000 7fac7199d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac6c1596b0 0x7fac6c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c06c560 0x7fac6c0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7199d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c0b03a0 con 0x7fac6c1592e0 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7199d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c0b0f40 con 0x7fac6c1596b0 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c06ccd0 con 0x7fac6c06c560 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7099b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fac6c1592e0 0x7fac6c0b0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58170/0 (socket says 172.21.15.33:58170) 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7099b700 1 -- 172.21.15.33:0/267156755 learned_addr learned my addr 172.21.15.33:0/267156755 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac6bfff700 1 --2- 172.21.15.33:0/267156755 >> v2:172.21.15.33:3300/0 conn(0x7fac6c1596b0 0x7fac6c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac7119c700 1 --2- 172.21.15.33:0/267156755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c06c560 0x7fac6c0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac6b7fe700 1 -- 172.21.15.33:0/267156755 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881270327 0 0) 0x7fac6c0b03a0 con 0x7fac6c1592e0 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac6bfff700 1 -- 172.21.15.33:0/267156755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c06c560 msgr2=0x7fac6c0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac6bfff700 1 --2- 172.21.15.33:0/267156755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c06c560 0x7fac6c0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac6bfff700 1 -- 172.21.15.33:0/267156755 >> v1:172.21.15.33:6789/0 conn(0x7fac6c1592e0 legacy=0x7fac6c0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.083+0000 7fac6bfff700 1 -- 172.21.15.33:0/267156755 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c06cfb0 con 0x7fac6c1596b0 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.084+0000 7fac6bfff700 1 --2- 172.21.15.33:0/267156755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c1596b0 0x7fac6c0cdd30 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fac580098e0 tx=0x7fac580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5fecd9ca13513 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.084+0000 7fac6b7fe700 1 -- 172.21.15.33:0/267156755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800b640 con 0x7fac6c1596b0 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.084+0000 7fac6b7fe700 1 -- 172.21.15.33:0/267156755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5800b7a0 con 0x7fac6c1596b0 2023-10-06T05:35:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.084+0000 7fac6b7fe700 1 -- 172.21.15.33:0/267156755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800bb40 con 0x7fac6c1596b0 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.084+0000 7fac7199d700 1 -- 172.21.15.33:0/267156755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c1596b0 msgr2=0x7fac6c0cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.084+0000 7fac7199d700 1 --2- 172.21.15.33:0/267156755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c1596b0 0x7fac6c0cdd30 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fac580098e0 tx=0x7fac580091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.085+0000 7fac7199d700 1 -- 172.21.15.33:0/267156755 shutdown_connections 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.085+0000 7fac7199d700 1 --2- 172.21.15.33:0/267156755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c06c560 0x7fac6c0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.085+0000 7fac7199d700 1 --2- 172.21.15.33:0/267156755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c1596b0 0x7fac6c0cdd30 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.085+0000 7fac7199d700 1 -- 172.21.15.33:0/267156755 >> 172.21.15.33:0/267156755 conn(0x7fac6c0675f0 msgr2=0x7fac6c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.085+0000 7fac7199d700 1 -- 172.21.15.33:0/267156755 shutdown_connections 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.085+0000 7fac7199d700 1 -- 172.21.15.33:0/267156755 wait complete. 2023-10-06T05:35:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.086+0000 7fac7199d700 1 Processor -- start 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.086+0000 7fac7199d700 1 -- start start 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7199d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 0x7fac6c14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c1592e0 0x7fac6c15d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7199d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c1596b0 0x7fac6c0b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7199d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15e080 con 0x7fac6c06c560 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15e360 con 0x7fac6c1592e0 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7199d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15e640 con 0x7fac6c1596b0 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 0x7fac6c14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7099b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 0x7fac6c14dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52438/0 (socket says 172.21.15.33:52438) 2023-10-06T05:35:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.087+0000 7fac7099b700 1 -- 172.21.15.33:0/3064768211 learned_addr learned my addr 172.21.15.33:0/3064768211 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7119c700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c1596b0 0x7fac6c0b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac6bfff700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c1592e0 0x7fac6c15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7099b700 1 -- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c1596b0 msgr2=0x7fac6c0b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7099b700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c1596b0 0x7fac6c0b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7099b700 1 -- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c1592e0 msgr2=0x7fac6c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7099b700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c1592e0 0x7fac6c15d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7099b700 1 -- 172.21.15.33:0/3064768211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009420 con 0x7fac6c06c560 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7099b700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 0x7fac6c14dd60 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fac6000c7e0 tx=0x7fac60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60005b30 con 0x7fac6c06c560 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac60005c90 con 0x7fac6c06c560 2023-10-06T05:35:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60007300 con 0x7fac6c06c560 2023-10-06T05:35:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c15e920 con 0x7fac6c06c560 2023-10-06T05:35:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.088+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c1575a0 con 0x7fac6c06c560 2023-10-06T05:35:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.089+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c1577f0 con 0x7fac6c06c560 2023-10-06T05:35:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.090+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac6001b400 con 0x7fac6c06c560 2023-10-06T05:35:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.091+0000 7fac68ff9700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac50066f30 0x7fac500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.091+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fac60083f60 con 0x7fac6c06c560 2023-10-06T05:35:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.092+0000 7fac6bfff700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac50066f30 0x7fac500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.093+0000 7fac6bfff700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac50066f30 0x7fac500693e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fac580091e0 tx=0x7fac58017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.096+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac60051320 con 0x7fac6c06c560 2023-10-06T05:35:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.300+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fac6c04c380 con 0x7fac50066f30 2023-10-06T05:35:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.301+0000 7fac68ff9700 1 -- 172.21.15.33:0/3064768211 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac6c04c380 con 0x7fac50066f30 2023-10-06T05:35:44.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac50066f30 msgr2=0x7fac500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac50066f30 0x7fac500693e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fac580091e0 tx=0x7fac58017000 comp rx=0 tx=0).stop 2023-10-06T05:35:44.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 msgr2=0x7fac6c14dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 0x7fac6c14dd60 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fac6000c7e0 tx=0x7fac60007df0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 shutdown_connections 2023-10-06T05:35:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fac50066f30 0x7fac500693e0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac6c1596b0 0x7fac6c0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac6c06c560 0x7fac6c14dd60 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 --2- 172.21.15.33:0/3064768211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac6c1592e0 0x7fac6c15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 >> 172.21.15.33:0/3064768211 conn(0x7fac6c0675f0 msgr2=0x7fac6c06b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:44.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 shutdown_connections 2023-10-06T05:35:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.303+0000 7fac7199d700 1 -- 172.21.15.33:0/3064768211 wait complete. 2023-10-06T05:35:44.310 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T05:35:44.324 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T05:35:44.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:44.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.512+0000 7f58b899d700 1 Processor -- start 2023-10-06T05:35:44.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.512+0000 7f58b899d700 1 -- start start 2023-10-06T05:35:44.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.513+0000 7f58b899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cb80 0x7f58b40abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.513+0000 7f58b899d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58b406eae0 0x7f58b40ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b899d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b40aa040 con 0x7f58b415cf50 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b899d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40a9a30 con 0x7f58b406eae0 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b406aa90 con 0x7f58b415cb80 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58b406eae0 0x7f58b40ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58b406eae0 0x7f58b40ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52446/0 (socket says 172.21.15.33:52446) 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 -- 172.21.15.33:0/505822685 learned_addr learned my addr 172.21.15.33:0/505822685 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:44.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b3fff700 1 --1- 172.21.15.33:0/505822685 >> v1:172.21.15.33:6789/0 conn(0x7f58b415cf50 0x7f58b40c8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58180/0 (socket says 172.21.15.33:58180) 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b37fe700 1 --2- 172.21.15.33:0/505822685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cb80 0x7f58b40abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 -- 172.21.15.33:0/505822685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cb80 msgr2=0x7f58b40abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/505822685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cb80 0x7f58b40abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 -- 172.21.15.33:0/505822685 >> v1:172.21.15.33:6789/0 conn(0x7f58b415cf50 legacy=0x7f58b40c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b2ffd700 1 -- 172.21.15.33:0/505822685 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b406a7b0 con 0x7f58b406eae0 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.514+0000 7f58b27fc700 1 -- 172.21.15.33:0/505822685 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016070824 0 0) 0x7f58b40aa040 con 0x7f58b415cf50 2023-10-06T05:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/505822685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b406eae0 0x7f58b40ab5a0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f58a80099b0 tx=0x7f58a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd68d11403631cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b27fc700 1 -- 172.21.15.33:0/505822685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a800b8f0 con 0x7f58b406eae0 2023-10-06T05:35:44.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b27fc700 1 -- 172.21.15.33:0/505822685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a800ba50 con 0x7f58b406eae0 2023-10-06T05:35:44.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b27fc700 1 -- 172.21.15.33:0/505822685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a800bdf0 con 0x7f58b406eae0 2023-10-06T05:35:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b899d700 1 -- 172.21.15.33:0/505822685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b406eae0 msgr2=0x7f58b40ab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b899d700 1 --2- 172.21.15.33:0/505822685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b406eae0 0x7f58b40ab5a0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f58a80099b0 tx=0x7f58a8009250 comp rx=0 tx=0).stop 2023-10-06T05:35:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b899d700 1 -- 172.21.15.33:0/505822685 shutdown_connections 2023-10-06T05:35:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b899d700 1 --2- 172.21.15.33:0/505822685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cb80 0x7f58b40abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b899d700 1 --2- 172.21.15.33:0/505822685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b406eae0 0x7f58b40ab5a0 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.515+0000 7f58b899d700 1 -- 172.21.15.33:0/505822685 >> 172.21.15.33:0/505822685 conn(0x7f58b4068090 msgr2=0x7f58b406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.516+0000 7f58b899d700 1 -- 172.21.15.33:0/505822685 shutdown_connections 2023-10-06T05:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.516+0000 7f58b899d700 1 -- 172.21.15.33:0/505822685 wait complete. 2023-10-06T05:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.517+0000 7f58b899d700 1 Processor -- start 2023-10-06T05:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.517+0000 7f58b899d700 1 -- start start 2023-10-06T05:35:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58b406eae0 0x7f58b4153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b415cb80 0x7f58b41544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cf50 0x7f58b4154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4154f70 con 0x7f58b415cb80 2023-10-06T05:35:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40caf50 con 0x7f58b406eae0 2023-10-06T05:35:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40ca660 con 0x7f58b415cf50 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b415cb80 0x7f58b41544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b415cb80 0x7f58b41544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52460/0 (socket says 172.21.15.33:52460) 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b2ffd700 1 -- 172.21.15.33:0/203914722 learned_addr learned my addr 172.21.15.33:0/203914722 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b37fe700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58b406eae0 0x7f58b4153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.518+0000 7f58b3fff700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cf50 0x7f58b4154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b2ffd700 1 -- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cf50 msgr2=0x7f58b4154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58b415cf50 0x7f58b4154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b2ffd700 1 -- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58b406eae0 msgr2=0x7f58b4153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58b406eae0 0x7f58b4153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b2ffd700 1 -- 172.21.15.33:0/203914722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80094f0 con 0x7f58b415cb80 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b37fe700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58b406eae0 0x7f58b4153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b415cb80 0x7f58b41544f0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f58b406e190 tx=0x7f58a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8024440 con 0x7f58b415cb80 2023-10-06T05:35:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a80245c0 con 0x7f58b415cb80 2023-10-06T05:35:44.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b40ca900 con 0x7f58b415cb80 2023-10-06T05:35:44.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8024980 con 0x7f58b415cb80 2023-10-06T05:35:44.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.519+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b40cac90 con 0x7f58b415cb80 2023-10-06T05:35:44.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.520+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58b4063ff0 con 0x7f58b415cb80 2023-10-06T05:35:44.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.521+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58a801a420 con 0x7f58b415cb80 2023-10-06T05:35:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.522+0000 7f589b7fe700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a0066f30 0x7f58a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.522+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f58a80861e0 con 0x7f58b415cb80 2023-10-06T05:35:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.522+0000 7f589b7fe700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f58a006d3f0 0x7f58a006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.522+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 --> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58a006feb0 con 0x7f58a006d3f0 2023-10-06T05:35:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.522+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a8088740 con 0x7f58b415cb80 2023-10-06T05:35:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.523+0000 7f58b37fe700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a0066f30 0x7f58a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.523+0000 7f58b3fff700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f58a006d3f0 0x7f58a006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.523+0000 7f58b37fe700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a0066f30 0x7f58a00693e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f58a4006fd0 tx=0x7f58a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.523+0000 7f58b3fff700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f58a006d3f0 0x7f58a006f800 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.524+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== osd.6 v2:172.21.15.167:6808/85771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f58a006feb0 con 0x7f58a006d3f0 2023-10-06T05:35:44.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.539+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 --> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f58b40caa90 con 0x7f58a006d3f0 2023-10-06T05:35:44.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f589b7fe700 1 -- 172.21.15.33:0/203914722 <== osd.6 v2:172.21.15.167:6808/85771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f58b40caa90 con 0x7f58a006d3f0 2023-10-06T05:35:44.542 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:44.543 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:44.543 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f58a006d3f0 msgr2=0x7f58a006f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6808/85771,v1:172.21.15.167:6809/85771] conn(0x7f58a006d3f0 0x7f58a006f800 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a0066f30 msgr2=0x7f58a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f58a0066f30 0x7f58a00693e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f58a4006fd0 tx=0x7f58a4008040 comp rx=0 tx=0).stop 2023-10-06T05:35:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b415cb80 msgr2=0x7f58b41544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 --2- 172.21.15.33:0/203914722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b415cb80 0x7f58b41544f0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f58b406e190 tx=0x7f58a8017000 comp rx=0 tx=0).stop 2023-10-06T05:35:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b3fff700 1 -- 172.21.15.33:0/203914722 reap_dead start 2023-10-06T05:35:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 shutdown_connections 2023-10-06T05:35:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 >> 172.21.15.33:0/203914722 conn(0x7f58b4068090 msgr2=0x7f58b40af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 shutdown_connections 2023-10-06T05:35:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.540+0000 7f58b899d700 1 -- 172.21.15.33:0/203914722 wait complete. 2023-10-06T05:35:44.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T05:35:44.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.746+0000 7f42f359d700 1 Processor -- start 2023-10-06T05:35:44.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.746+0000 7f42f359d700 1 -- start start 2023-10-06T05:35:44.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f359d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f42ec159e10 0x7f42ec157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f359d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec06bd10 con 0x7f42ec156880 2023-10-06T05:35:44.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f359d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec06a820 con 0x7f42ec0c8090 2023-10-06T05:35:44.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f359d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0cc290 con 0x7f42ec159e10 2023-10-06T05:35:44.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f259b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f42ec156880 0x7f42ec0c7980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58186/0 (socket says 172.21.15.33:58186) 2023-10-06T05:35:44.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f259b700 1 -- 172.21.15.33:0/2036154569 learned_addr learned my addr 172.21.15.33:0/2036154569 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:44.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.747+0000 7f42f2d9c700 1 --2- 172.21.15.33:0/2036154569 >> v2:172.21.15.33:3300/0 conn(0x7f42ec159e10 0x7f42ec157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f1599700 1 -- 172.21.15.33:0/2036154569 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366370520 0 0) 0x7f42ec06bd10 con 0x7f42ec156880 2023-10-06T05:35:44.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f1599700 1 -- 172.21.15.33:0/2036154569 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d8003570 con 0x7f42ec156880 2023-10-06T05:35:44.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f2d9c700 1 -- 172.21.15.33:0/2036154569 >> v1:172.21.15.167:6789/0 conn(0x7f42ec0c8090 legacy=0x7f42ec156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f1599700 1 -- 172.21.15.33:0/2036154569 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770913838 0 0) 0x7f42ec06a820 con 0x7f42ec0c8090 2023-10-06T05:35:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f2d9c700 1 -- 172.21.15.33:0/2036154569 >> v1:172.21.15.33:6789/0 conn(0x7f42ec156880 legacy=0x7f42ec0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f2d9c700 1 -- 172.21.15.33:0/2036154569 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec15a2f0 con 0x7f42ec159e10 2023-10-06T05:35:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.748+0000 7f42f2d9c700 1 --2- 172.21.15.33:0/2036154569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec159e10 0x7f42ec157360 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009e90 tx=0x7f42e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfdd55ebb931f14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f1599700 1 -- 172.21.15.33:0/2036154569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e400e070 con 0x7f42ec159e10 2023-10-06T05:35:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f1599700 1 -- 172.21.15.33:0/2036154569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e400bc40 con 0x7f42ec159e10 2023-10-06T05:35:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f1599700 1 -- 172.21.15.33:0/2036154569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4012430 con 0x7f42ec159e10 2023-10-06T05:35:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 -- 172.21.15.33:0/2036154569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec159e10 msgr2=0x7f42ec157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 --2- 172.21.15.33:0/2036154569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec159e10 0x7f42ec157360 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009e90 tx=0x7f42e4009780 comp rx=0 tx=0).stop 2023-10-06T05:35:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 -- 172.21.15.33:0/2036154569 shutdown_connections 2023-10-06T05:35:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 --2- 172.21.15.33:0/2036154569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec159e10 0x7f42ec157360 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 -- 172.21.15.33:0/2036154569 >> 172.21.15.33:0/2036154569 conn(0x7f42ec06c560 msgr2=0x7f42ec06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:44.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 -- 172.21.15.33:0/2036154569 shutdown_connections 2023-10-06T05:35:44.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.749+0000 7f42f359d700 1 -- 172.21.15.33:0/2036154569 wait complete. 2023-10-06T05:35:44.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.751+0000 7f42f359d700 1 Processor -- start 2023-10-06T05:35:44.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.751+0000 7f42f359d700 1 -- start start 2023-10-06T05:35:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f359d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42ec0c8090 0x7f42ec14dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 0x7f42ec0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f359d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42ec159e10 0x7f42ec152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec155170 con 0x7f42ec156880 2023-10-06T05:35:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f359d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec16a120 con 0x7f42ec159e10 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f359d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec154b20 con 0x7f42ec0c8090 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f1d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 0x7f42ec0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f1d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 0x7f42ec0af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52476/0 (socket says 172.21.15.33:52476) 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f1d9a700 1 -- 172.21.15.33:0/4040099758 learned_addr learned my addr 172.21.15.33:0/4040099758 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f259b700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42ec0c8090 0x7f42ec14dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.752+0000 7f42f2d9c700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42ec159e10 0x7f42ec152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f1d9a700 1 -- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42ec0c8090 msgr2=0x7f42ec14dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f1d9a700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42ec0c8090 0x7f42ec14dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f1d9a700 1 -- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42ec159e10 msgr2=0x7f42ec152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f1d9a700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42ec159e10 0x7f42ec152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f1d9a700 1 -- 172.21.15.33:0/4040099758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4009a20 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f2d9c700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42ec159e10 0x7f42ec152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f1d9a700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 0x7f42ec0af020 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c530 tx=0x7f42dc007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc011590 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc011710 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc011ad0 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec154e00 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.753+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec14e110 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.755+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec154990 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.755+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42dc01b3b0 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.756+0000 7f42e27fc700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42d0066f30 0x7f42d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.756+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f42dc083890 con 0x7f42ec156880 2023-10-06T05:35:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.757+0000 7f42f259b700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42d0066f30 0x7f42d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.758+0000 7f42f259b700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42d0066f30 0x7f42d00693e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0b2060 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.760+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42dc050c70 con 0x7f42ec156880 2023-10-06T05:35:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.964+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f42ec04c380 con 0x7f42d0066f30 2023-10-06T05:35:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.965+0000 7f42e27fc700 1 -- 172.21.15.33:0/4040099758 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f42ec04c380 con 0x7f42d0066f30 2023-10-06T05:35:44.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42d0066f30 msgr2=0x7f42d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42d0066f30 0x7f42d00693e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0b2060 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2023-10-06T05:35:44.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 msgr2=0x7f42ec0af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 0x7f42ec0af020 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c530 tx=0x7f42dc007f20 comp rx=0 tx=0).stop 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 shutdown_connections 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f42d0066f30 0x7f42d00693e0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42ec159e10 0x7f42ec152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42ec0c8090 0x7f42ec14dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 --2- 172.21.15.33:0/4040099758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ec156880 0x7f42ec0af020 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:44.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 >> 172.21.15.33:0/4040099758 conn(0x7f42ec06c560 msgr2=0x7f42ec0cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 shutdown_connections 2023-10-06T05:35:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:44.967+0000 7f42f359d700 1 -- 172.21.15.33:0/4040099758 wait complete. 2023-10-06T05:35:44.973 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T05:35:44.987 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T05:35:44.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:35:45.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.178+0000 7f49c899d700 1 Processor -- start 2023-10-06T05:35:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.178+0000 7f49c899d700 1 -- start start 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c899d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49c4159400 0x7f49c40a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c899d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f49c406dbf0 0x7f49c4067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c899d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40a9dd0 con 0x7f49c41597d0 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c899d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c406d760 con 0x7f49c4159400 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c899d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40aa8c0 con 0x7f49c406dbf0 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49c4159400 0x7f49c40a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.179+0000 7f49c3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f49c41597d0 0x7f49c406a260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58200/0 (socket says 172.21.15.33:58200) 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c3fff700 1 -- 172.21.15.33:0/4189802743 learned_addr learned my addr 172.21.15.33:0/4189802743 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c2ffd700 1 --2- 172.21.15.33:0/4189802743 >> v2:172.21.15.167:3300/0 conn(0x7f49c406dbf0 0x7f49c4067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c37fe700 1 -- 172.21.15.33:0/4189802743 >> v2:172.21.15.167:3300/0 conn(0x7f49c406dbf0 msgr2=0x7f49c4067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c37fe700 1 --2- 172.21.15.33:0/4189802743 >> v2:172.21.15.167:3300/0 conn(0x7f49c406dbf0 0x7f49c4067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c37fe700 1 -- 172.21.15.33:0/4189802743 >> v1:172.21.15.33:6789/0 conn(0x7f49c41597d0 legacy=0x7f49c406a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c37fe700 1 -- 172.21.15.33:0/4189802743 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40aa5e0 con 0x7f49c4159400 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c27fc700 1 -- 172.21.15.33:0/4189802743 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286115144 0 0) 0x7f49bc001490 con 0x7f49c41597d0 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.180+0000 7f49c37fe700 1 --2- 172.21.15.33:0/4189802743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c4159400 0x7f49c40a9890 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f49b40098e0 tx=0x7f49b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73567c7098af4892 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c27fc700 1 -- 172.21.15.33:0/4189802743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4009e80 con 0x7f49c4159400 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c27fc700 1 -- 172.21.15.33:0/4189802743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400b820 con 0x7f49c4159400 2023-10-06T05:35:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c27fc700 1 -- 172.21.15.33:0/4189802743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400bba0 con 0x7f49c4159400 2023-10-06T05:35:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 -- 172.21.15.33:0/4189802743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c4159400 msgr2=0x7f49c40a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 --2- 172.21.15.33:0/4189802743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c4159400 0x7f49c40a9890 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f49b40098e0 tx=0x7f49b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 -- 172.21.15.33:0/4189802743 shutdown_connections 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 --2- 172.21.15.33:0/4189802743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c4159400 0x7f49c40a9890 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 --2- 172.21.15.33:0/4189802743 >> v2:172.21.15.167:3300/0 conn(0x7f49c406dbf0 0x7f49c4067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 -- 172.21.15.33:0/4189802743 >> 172.21.15.33:0/4189802743 conn(0x7f49c406ebf0 msgr2=0x7f49c406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 -- 172.21.15.33:0/4189802743 shutdown_connections 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.181+0000 7f49c899d700 1 -- 172.21.15.33:0/4189802743 wait complete. 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.183+0000 7f49c899d700 1 Processor -- start 2023-10-06T05:35:45.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.183+0000 7f49c899d700 1 -- start start 2023-10-06T05:35:45.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c899d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c406dbf0 0x7f49c414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c899d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c4159400 0x7f49c4152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c899d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c41597d0 0x7f49c4154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c899d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41542b0 con 0x7f49c41597d0 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c899d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c414c440 con 0x7f49c406dbf0 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c899d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c414f9c0 con 0x7f49c4159400 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c41597d0 0x7f49c4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c41597d0 0x7f49c4154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52486/0 (socket says 172.21.15.33:52486) 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c3fff700 1 -- 172.21.15.33:0/2344487593 learned_addr learned my addr 172.21.15.33:0/2344487593 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c2ffd700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c4159400 0x7f49c4152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.184+0000 7f49c37fe700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c406dbf0 0x7f49c414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c3fff700 1 -- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c4159400 msgr2=0x7f49c4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c3fff700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49c4159400 0x7f49c4152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c3fff700 1 -- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c406dbf0 msgr2=0x7f49c414bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c3fff700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c406dbf0 0x7f49c414bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c3fff700 1 -- 172.21.15.33:0/2344487593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b4009420 con 0x7f49c41597d0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c3fff700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c41597d0 0x7f49c4154780 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00cc90 tx=0x7f49bc001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc004bb0 con 0x7f49c41597d0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc004d10 con 0x7f49c41597d0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c40af210 con 0x7f49c41597d0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00ab00 con 0x7f49c41597d0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.185+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c4153cf0 con 0x7f49c41597d0 2023-10-06T05:35:45.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.186+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49c4063ff0 con 0x7f49c41597d0 2023-10-06T05:35:45.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.187+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49bc0070f0 con 0x7f49c41597d0 2023-10-06T05:35:45.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.188+0000 7f49ab7fe700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49b0066f30 0x7f49b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.188+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f49bc084070 con 0x7f49c41597d0 2023-10-06T05:35:45.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.188+0000 7f49ab7fe700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7f49b006d3f0 0x7f49b006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.188+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 --> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49b006feb0 con 0x7f49b006d3f0 2023-10-06T05:35:45.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.188+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=286) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49bc086570 con 0x7f49c41597d0 2023-10-06T05:35:45.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.189+0000 7f49c37fe700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49b0066f30 0x7f49b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.189+0000 7f49c2ffd700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7f49b006d3f0 0x7f49b006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.190+0000 7f49c2ffd700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7f49b006d3f0 0x7f49b006f800 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.190+0000 7f49c37fe700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49b0066f30 0x7f49b00693e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f49b40093f0 tx=0x7f49b4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.191+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== osd.7 v2:172.21.15.167:6824/85794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f49b006feb0 con 0x7f49b006d3f0 2023-10-06T05:35:45.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.206+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 --> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f49c40a9dd0 con 0x7f49b006d3f0 2023-10-06T05:35:45.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.206+0000 7f49ab7fe700 1 -- 172.21.15.33:0/2344487593 <== osd.7 v2:172.21.15.167:6824/85794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f49c40a9dd0 con 0x7f49b006d3f0 2023-10-06T05:35:45.209 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T05:35:45.210 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:35:45.210 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7f49b006d3f0 msgr2=0x7f49b006f800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6824/85794,v1:172.21.15.167:6825/85794] conn(0x7f49b006d3f0 0x7f49b006f800 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49b0066f30 msgr2=0x7f49b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f49b0066f30 0x7f49b00693e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f49b40093f0 tx=0x7f49b4017000 comp rx=0 tx=0).stop 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c41597d0 msgr2=0x7f49c4154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 --2- 172.21.15.33:0/2344487593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49c41597d0 0x7f49c4154780 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00cc90 tx=0x7f49bc001a50 comp rx=0 tx=0).stop 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c3fff700 1 -- 172.21.15.33:0/2344487593 reap_dead start 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 shutdown_connections 2023-10-06T05:35:45.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 >> 172.21.15.33:0/2344487593 conn(0x7f49c406ebf0 msgr2=0x7f49c414ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:45.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 shutdown_connections 2023-10-06T05:35:45.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.207+0000 7f49c899d700 1 -- 172.21.15.33:0/2344487593 wait complete. 2023-10-06T05:35:45.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T05:35:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.411+0000 7fc7aaf9d700 1 Processor -- start 2023-10-06T05:35:45.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.411+0000 7fc7aaf9d700 1 -- start start 2023-10-06T05:35:45.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.412+0000 7fc7aaf9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7a4067840 0x7fc7a4159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.412+0000 7fc7aaf9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7a40683e0 0x7fc7a4142d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.412+0000 7fc7aaf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a41432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.412+0000 7fc7aaf9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a415a810 con 0x7fc7a4067840 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.412+0000 7fc7aaf9d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a415aaf0 con 0x7fc7a40683e0 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.412+0000 7fc7aaf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415add0 con 0x7fc7a4067c10 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7a4067840 0x7fc7a4159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7a4067840 0x7fc7a4159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52490/0 (socket says 172.21.15.33:52490) 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 -- 172.21.15.33:0/1111512567 learned_addr learned my addr 172.21.15.33:0/1111512567 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a979a700 1 --2- 172.21.15.33:0/1111512567 >> v2:172.21.15.167:3300/0 conn(0x7fc7a40683e0 0x7fc7a4142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7aa79c700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a41432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 -- 172.21.15.33:0/1111512567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 msgr2=0x7fc7a41432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a41432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 -- 172.21.15.33:0/1111512567 >> v2:172.21.15.167:3300/0 conn(0x7fc7a40683e0 msgr2=0x7fc7a4142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 --2- 172.21.15.33:0/1111512567 >> v2:172.21.15.167:3300/0 conn(0x7fc7a40683e0 0x7fc7a4142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 -- 172.21.15.33:0/1111512567 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a40ca1d0 con 0x7fc7a4067840 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7aa79c700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a41432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.413+0000 7fc7a9f9b700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a4067840 0x7fc7a4159f50 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fc7a406df70 tx=0x7fc7a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5527085d9691a178 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7a8f99700 1 -- 172.21.15.33:0/1111512567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000b860 con 0x7fc7a4067840 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7a8f99700 1 -- 172.21.15.33:0/1111512567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000b9c0 con 0x7fc7a4067840 2023-10-06T05:35:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7a8f99700 1 -- 172.21.15.33:0/1111512567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000bd60 con 0x7fc7a4067840 2023-10-06T05:35:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/1111512567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a4067840 msgr2=0x7fc7a4159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a4067840 0x7fc7a4159f50 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fc7a406df70 tx=0x7fc7a0009250 comp rx=0 tx=0).stop 2023-10-06T05:35:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/1111512567 shutdown_connections 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a41432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/1111512567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a4067840 0x7fc7a4159f50 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/1111512567 >> v2:172.21.15.167:3300/0 conn(0x7fc7a40683e0 0x7fc7a4142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/1111512567 >> 172.21.15.33:0/1111512567 conn(0x7fc7a40cd060 msgr2=0x7fc7a40c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/1111512567 shutdown_connections 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.414+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/1111512567 wait complete. 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.416+0000 7fc7aaf9d700 1 Processor -- start 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.416+0000 7fc7aaf9d700 1 -- start start 2023-10-06T05:35:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aaf9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a4067840 0x7fc7a40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aaf9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aaf9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 0x7fc7a4141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aaf9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40af300 con 0x7fc7a40683e0 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aaf9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40af5e0 con 0x7fc7a4067840 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aaf9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415b940 con 0x7fc7a4067c10 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aa79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 0x7fc7a4141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aa79c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 0x7fc7a4141270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52506/0 (socket says 172.21.15.33:52506) 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7aa79c700 1 -- 172.21.15.33:0/452597907 learned_addr learned my addr 172.21.15.33:0/452597907 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7a9f9b700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a4067840 0x7fc7a40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.417+0000 7fc7a979a700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aa79c700 1 -- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 msgr2=0x7fc7a415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aa79c700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aa79c700 1 -- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a4067840 msgr2=0x7fc7a40b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aa79c700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a4067840 0x7fc7a40b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aa79c700 1 -- 172.21.15.33:0/452597907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a00094f0 con 0x7fc7a40683e0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aa79c700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 0x7fc7a4141270 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00a380 tx=0x7fc79c007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c015440 con 0x7fc7a40683e0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c0155a0 con 0x7fc7a40683e0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c015960 con 0x7fc7a40683e0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a415bc20 con 0x7fc7a40683e0 2023-10-06T05:35:45.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.418+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a415b510 con 0x7fc7a40683e0 2023-10-06T05:35:45.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.419+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a40c67d0 con 0x7fc7a40683e0 2023-10-06T05:35:45.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.420+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc79c00d440 con 0x7fc7a40683e0 2023-10-06T05:35:45.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.422+0000 7fc799ffb700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc788066f30 0x7fc7880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.422+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc79c084f20 con 0x7fc7a40683e0 2023-10-06T05:35:45.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.422+0000 7fc7a9f9b700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc788066f30 0x7fc7880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.423+0000 7fc7a9f9b700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc788066f30 0x7fc7880693e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc7a406df70 tx=0x7fc7a0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.426+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc79c052260 con 0x7fc7a40683e0 2023-10-06T05:35:45.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.626+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7a404c380 con 0x7fc788066f30 2023-10-06T05:35:45.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.626+0000 7fc799ffb700 1 -- 172.21.15.33:0/452597907 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc7a404c380 con 0x7fc788066f30 2023-10-06T05:35:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc788066f30 msgr2=0x7fc7880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc788066f30 0x7fc7880693e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc7a406df70 tx=0x7fc7a0017000 comp rx=0 tx=0).stop 2023-10-06T05:35:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 msgr2=0x7fc7a4141270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 0x7fc7a4141270 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00a380 tx=0x7fc79c007900 comp rx=0 tx=0).stop 2023-10-06T05:35:45.632 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 shutdown_connections 2023-10-06T05:35:45.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc788066f30 0x7fc7880693e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a40683e0 0x7fc7a4141270 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a4067840 0x7fc7a40b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.628+0000 7fc7aaf9d700 1 --2- 172.21.15.33:0/452597907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a4067c10 0x7fc7a415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.629+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 >> 172.21.15.33:0/452597907 conn(0x7fc7a40cd060 msgr2=0x7fc7a40c1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:45.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.629+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 shutdown_connections 2023-10-06T05:35:45.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.629+0000 7fc7aaf9d700 1 -- 172.21.15.33:0/452597907 wait complete. 2023-10-06T05:35:45.634 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T05:35:45.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:35:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.832+0000 7fa60679d700 1 Processor -- start 2023-10-06T05:35:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.833+0000 7fa60679d700 1 -- start start 2023-10-06T05:35:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.834+0000 7fa60679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 0x7fa60015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.834+0000 7fa60679d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60006f220 con 0x7fa6000afdd0 2023-10-06T05:35:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.834+0000 7fa60679d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60015ae50 con 0x7fa600159530 2023-10-06T05:35:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.834+0000 7fa60679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000b1e00 con 0x7fa60006e140 2023-10-06T05:35:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.834+0000 7fa605f9c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa6000afdd0 0x7fa6000b0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58216/0 (socket says 172.21.15.33:58216) 2023-10-06T05:35:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.834+0000 7fa605f9c700 1 -- 172.21.15.33:0/35806797 learned_addr learned my addr 172.21.15.33:0/35806797 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa604f9a700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 0x7fa60015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055571796 0 0) 0x7fa60006f220 con 0x7fa6000afdd0 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ec003570 con 0x7fa6000afdd0 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790762851 0 0) 0x7fa5ec003570 con 0x7fa6000afdd0 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 msgr2=0x7fa60015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 0x7fa60015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 >> v1:172.21.15.167:6789/0 conn(0x7fa600159530 legacy=0x7fa60006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60006e610 con 0x7fa6000afdd0 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fa5f80017f0 con 0x7fa6000afdd0 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 >> v1:172.21.15.33:6789/0 conn(0x7fa6000afdd0 legacy=0x7fa6000b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 0x7fa5ec003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5ec004060 0x7fa5ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5ec004d50 0x7fa5ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000b1e00 con 0x7fa5ec004060 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec00bc20 con 0x7fa5ec004d50 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec00bde0 con 0x7fa60006e140 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.835+0000 7fa604f9a700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5ec004060 0x7fa5ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa605f9c700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5ec004d50 0x7fa5ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa60579b700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 0x7fa5ec003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa604f9a700 1 -- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 msgr2=0x7fa5ec003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa604f9a700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa60006e140 0x7fa5ec003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa604f9a700 1 -- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5ec004d50 msgr2=0x7fa5ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:35:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa604f9a700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5ec004d50 0x7fa5ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa604f9a700 1 -- 172.21.15.33:0/35806797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec003570 con 0x7fa5ec004060 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa605f9c700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5ec004d50 0x7fa5ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa604f9a700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5ec004060 0x7fa5ec004810 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009c00 tx=0x7fa5f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73cc37baf9c110e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000e710 con 0x7fa5ec004060 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000e870 con 0x7fa5ec004060 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.836+0000 7fa5f7fff700 1 -- 172.21.15.33:0/35806797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000ec30 con 0x7fa5ec004060 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa60679d700 1 -- 172.21.15.33:0/35806797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5ec004060 msgr2=0x7fa5ec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa60679d700 1 --2- 172.21.15.33:0/35806797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5ec004060 0x7fa5ec004810 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009c00 tx=0x7fa5f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa605f9c700 1 -- 172.21.15.33:0/35806797 reap_dead start 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa60679d700 1 -- 172.21.15.33:0/35806797 shutdown_connections 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa60679d700 1 -- 172.21.15.33:0/35806797 >> 172.21.15.33:0/35806797 conn(0x7fa60006c560 msgr2=0x7fa60006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa60679d700 1 -- 172.21.15.33:0/35806797 shutdown_connections 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.837+0000 7fa60679d700 1 -- 172.21.15.33:0/35806797 wait complete. 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.838+0000 7fa60679d700 1 Processor -- start 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.839+0000 7fa60679d700 1 -- start start 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.839+0000 7fa60679d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ec004060 0x7fa6000cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa60679d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 0x7fa6001417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa60679d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6001557c0 0x7fa600155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa60679d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000cb5e0 con 0x7fa60016a550 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa60679d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001561a0 con 0x7fa6001557c0 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa60679d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60016ce50 con 0x7fa5ec004060 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 0x7fa6001417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 0x7fa6001417d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52520/0 (socket says 172.21.15.33:52520) 2023-10-06T05:35:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 -- 172.21.15.33:0/1301225924 learned_addr learned my addr 172.21.15.33:0/1301225924 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa60579b700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ec004060 0x7fa6000cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa605f9c700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6001557c0 0x7fa600155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 -- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ec004060 msgr2=0x7fa6000cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ec004060 0x7fa6000cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 -- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6001557c0 msgr2=0x7fa600155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6001557c0 0x7fa600155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.840+0000 7fa604f9a700 1 -- 172.21.15.33:0/1301225924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60016d990 con 0x7fa60016a550 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.841+0000 7fa604f9a700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 0x7fa6001417d0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00bbf0 tx=0x7fa5f001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.841+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0022070 con 0x7fa60016a550 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.841+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000e9e0 con 0x7fa60016a550 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.841+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001c450 con 0x7fa60016a550 2023-10-06T05:35:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.841+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60016d060 con 0x7fa60016a550 2023-10-06T05:35:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.841+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60016d620 con 0x7fa60016a550 2023-10-06T05:35:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.842+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60016db20 con 0x7fa60016a550 2023-10-06T05:35:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.843+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5f001c880 con 0x7fa60016a550 2023-10-06T05:35:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.844+0000 7fa5f57fa700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5e4066f30 0x7fa5e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:35:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.844+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa5f0085b90 con 0x7fa60016a550 2023-10-06T05:35:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.844+0000 7fa60579b700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5e4066f30 0x7fa5e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:35:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.845+0000 7fa60579b700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5e4066f30 0x7fa5e40693e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa6000c7ed0 tx=0x7fa5fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:35:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:45.848+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5f0052f80 con 0x7fa60016a550 2023-10-06T05:35:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.046+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6000af740 con 0x7fa5e4066f30 2023-10-06T05:35:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.050+0000 7fa5f57fa700 1 -- 172.21.15.33:0/1301225924 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72527 (secure 0 0 0) 0x7fa6000af740 con 0x7fa5e4066f30 2023-10-06T05:35:46.053 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:35:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5e4066f30 msgr2=0x7fa5e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5e4066f30 0x7fa5e40693e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa6000c7ed0 tx=0x7fa5fc009040 comp rx=0 tx=0).stop 2023-10-06T05:35:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 msgr2=0x7fa6001417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:35:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 0x7fa6001417d0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00bbf0 tx=0x7fa5f001a000 comp rx=0 tx=0).stop 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 shutdown_connections 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fa5e4066f30 0x7fa5e40693e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6001557c0 0x7fa600155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ec004060 0x7fa6000cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 --2- 172.21.15.33:0/1301225924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60016a550 0x7fa6001417d0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 >> 172.21.15.33:0/1301225924 conn(0x7fa60006c560 msgr2=0x7fa6000b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 shutdown_connections 2023-10-06T05:35:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:35:46.053+0000 7fa60679d700 1 -- 172.21.15.33:0/1301225924 wait complete. 2023-10-06T05:35:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:35:46.073 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":925,"stamp":"2023-10-06T05:35:44.148144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473416,"kb_used_data":29576,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752501304,"statfs":{"total":773094113280,"available":770561335296,"internally_reserved":0,"allocated":30285824,"data_stored":14857640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002450"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:35.137043+0000","last_change":"2023-10-06T05:14:19.711025+0000","last_active":"2023-10-06T05:35:35.137043+0000","last_peered":"2023-10-06T05:35:35.137043+0000","last_clean":"2023-10-06T05:35:35.137043+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:35.137043+0000","last_undegraded":"2023-10-06T05:35:35.137043+0000","last_fullsized":"2023-10-06T05:35:35.137043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":230,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.986211+0000","last_change":"2023-10-06T05:14:19.680302+0000","last_active":"2023-10-06T05:35:33.986211+0000","last_peered":"2023-10-06T05:35:33.986211+0000","last_clean":"2023-10-06T05:35:33.986211+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:33.986211+0000","last_undegraded":"2023-10-06T05:35:33.986211+0000","last_fullsized":"2023-10-06T05:35:33.986211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:02.530668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":236,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.982242+0000","last_change":"2023-10-06T05:14:19.778631+0000","last_active":"2023-10-06T05:35:33.982242+0000","last_peered":"2023-10-06T05:35:33.982242+0000","last_clean":"2023-10-06T05:35:33.982242+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:33.982242+0000","last_undegraded":"2023-10-06T05:35:33.982242+0000","last_fullsized":"2023-10-06T05:35:33.982242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:25.298405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":230,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.987332+0000","last_change":"2023-10-06T05:14:19.679987+0000","last_active":"2023-10-06T05:35:33.987332+0000","last_peered":"2023-10-06T05:35:33.987332+0000","last_clean":"2023-10-06T05:35:33.987332+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:33.987332+0000","last_undegraded":"2023-10-06T05:35:33.987332+0000","last_fullsized":"2023-10-06T05:35:33.987332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":380,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275502+0000","last_change":"2023-10-06T05:14:13.687645+0000","last_active":"2023-10-06T05:35:34.275502+0000","last_peered":"2023-10-06T05:35:34.275502+0000","last_clean":"2023-10-06T05:35:34.275502+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:34.275502+0000","last_undegraded":"2023-10-06T05:35:34.275502+0000","last_fullsized":"2023-10-06T05:35:34.275502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:12.087568+0000","last_clean_scrub_stamp":"2023-10-06T05:14:12.087568+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:32.536851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275538+0000","last_change":"2023-10-06T05:14:19.678875+0000","last_active":"2023-10-06T05:35:34.275538+0000","last_peered":"2023-10-06T05:35:34.275538+0000","last_clean":"2023-10-06T05:35:34.275538+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:34.275538+0000","last_undegraded":"2023-10-06T05:35:34.275538+0000","last_fullsized":"2023-10-06T05:35:34.275538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:01.500506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":227,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:34.275933+0000","last_change":"2023-10-06T05:14:19.710882+0000","last_active":"2023-10-06T05:35:34.275933+0000","last_peered":"2023-10-06T05:35:34.275933+0000","last_clean":"2023-10-06T05:35:34.275933+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:34.275933+0000","last_undegraded":"2023-10-06T05:35:34.275933+0000","last_fullsized":"2023-10-06T05:35:34.275933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:38:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":236,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:35.138861+0000","last_change":"2023-10-06T05:14:19.676169+0000","last_active":"2023-10-06T05:35:35.138861+0000","last_peered":"2023-10-06T05:35:35.138861+0000","last_clean":"2023-10-06T05:35:35.138861+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:35.138861+0000","last_undegraded":"2023-10-06T05:35:35.138861+0000","last_fullsized":"2023-10-06T05:35:35.138861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":272,"reported_epoch":285,"state":"active+clean","last_fresh":"2023-10-06T05:35:33.985861+0000","last_change":"2023-10-06T05:14:19.679316+0000","last_active":"2023-10-06T05:35:33.985861+0000","last_peered":"2023-10-06T05:35:33.985861+0000","last_clean":"2023-10-06T05:35:33.985861+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:33.985861+0000","last_undegraded":"2023-10-06T05:35:33.985861+0000","last_fullsized":"2023-10-06T05:35:33.985861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:18.653137+0000","last_clean_scrub_stamp":"2023-10-06T05:14:18.653137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:35.027294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575111,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309612,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062228,"statfs":{"total":96636764160,"available":96319721472,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":0.879}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.499},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.79900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575109,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.47899999999999998,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88700000000000001,"5min":1.0860000000000001,"15min":1.137},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.40799999999999997},"max":{"1min":0.95899999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.625}]},{"osd":2,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.70599999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.751},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.88600000000000001,"5min":1.075,"15min":1.115},"last":0.69999999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.56000000000000005,"5min":0.504,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.5},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0860000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67900000000000005,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.022,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.54400000000000004,"5min":0.439,"15min":0.40000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:34:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":1.0129999999999999,"5min":1.143,"15min":1.143},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.038,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.0149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575110,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":1.272,"15min":1.272},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.68000000000000005,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.72199999999999998,"15min":1.165},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.313},"max":{"1min":0.65800000000000003,"5min":1.22,"15min":1.22},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.70699999999999996,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":1.238,"15min":1.2430000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84999999999999998,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.79000000000000004},"min":{"1min":0.51900000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96299999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1080000000000001,"5min":1.492,"15min":1.492},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.262,"15min":1.2909999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.3280000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:35:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.82999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.0609999999999999,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.85299999999999998,"15min":0.82199999999999995},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":0.92300000000000004,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575110,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313708,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058132,"statfs":{"total":96636764160,"available":96315527168,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.95299999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.307},"max":{"1min":0.628,"5min":0.79900000000000004,"15min":0.93600000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0980000000000001,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":1.069,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.495}]},{"osd":5,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.1379999999999999,"15min":1.27},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.49199999999999999},"max":{"1min":1.151,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.81499999999999995,"15min":0.80100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.147,"15min":1.153},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:34:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.496},"max":{"1min":1.0629999999999999,"5min":1.2,"15min":1.2010000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.90800000000000003,"5min":1.1459999999999999,"15min":1.274},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575110,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313128,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058712,"statfs":{"total":96636764160,"available":96316121088,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.79600000000000004,"5min":0.82999999999999996,"15min":1.056},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.82399999999999995,"15min":0.92000000000000004},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57099999999999995,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.64000000000000001,"5min":0.82299999999999995,"15min":1.073},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.58599999999999997,"5min":1.117,"15min":1.117},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57199999999999995,"15min":0.55200000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95599999999999996,"5min":1.2709999999999999,"15min":1.486},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":0.98699999999999999,"5min":1.089,"15min":1.5},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93899999999999995,"5min":1.157,"15min":1.1639999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92400000000000004,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.22,"15min":1.22},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.59699999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0189999999999999,"5min":1.341,"15min":1.341},"last":0.751}]},{"osd":7,"last update":"Fri Oct 6 05:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.79000000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":1.282,"15min":1.538},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.264,"15min":1.264},"last":0.70799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.307},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575109,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.745,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.443},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.751},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.82199999999999995,"5min":1.026,"15min":1.026},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.69499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41799999999999998},"max":{"1min":0.88600000000000001,"5min":1.099,"15min":1.099},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76200000000000001,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.497,"15min":0.443},"max":{"1min":0.85699999999999998,"5min":3.839,"15min":3.839},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.873,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.746}]},{"osd":5,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.626},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.71599999999999997},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.72399999999999998},"min":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":0.96499999999999997,"15min":1.0609999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":0.93999999999999995,"5min":1.109,"15min":1.282},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.995}]}]},{"osd":5,"up_from":13,"seq":55834575111,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.372},"max":{"1min":1.071,"5min":1.115,"15min":1.2130000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.40899999999999997},"max":{"1min":1.0589999999999999,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.42699999999999999,"15min":0.42499999999999999},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.752,"15min":0.73199999999999998},"min":{"1min":0.55700000000000005,"5min":0.495,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":1.169,"15min":1.169},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.145},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"max":{"1min":0.92800000000000005,"5min":0.96599999999999997,"15min":1.0089999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.02,"15min":1.1539999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.96899999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.52000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":1.0429999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.60199999999999998,"5min":0.53300000000000003,"15min":0.40899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.0609999999999999},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:35:46.079 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.079 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.079 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.079 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.079 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T05:14:12.087568+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=12, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.080 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.080 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.080 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.080 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T05:14:18.653137+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=18, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=35, tm_sec=39, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:35:46.080 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T05:35:52.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:35:52.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:36:06.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:36:06.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.278+0000 7f8bfc3ff700 1 Processor -- start 2023-10-06T05:36:06.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.278+0000 7f8bfc3ff700 1 -- start start 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bfc3ff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8bf40c9e10 0x7f8bf40ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bfc3ff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf40ca8d0 con 0x7f8bf413fb70 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bfc3ff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf40cabb0 con 0x7f8bf40af000 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bfc3ff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf406ad00 con 0x7f8bf40c9e10 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bfa19b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8bf413fb70 0x7f8bf406b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42540/0 (socket says 172.21.15.33:42540) 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bfa19b700 1 -- 172.21.15.33:0/3283430960 learned_addr learned my addr 172.21.15.33:0/3283430960 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:36:06.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.280+0000 7f8bf999a700 1 --2- 172.21.15.33:0/3283430960 >> v2:172.21.15.167:3300/0 conn(0x7f8bf40c9e10 0x7f8bf40ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf9199700 1 -- 172.21.15.33:0/3283430960 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944658272 0 0) 0x7f8bf40ca8d0 con 0x7f8bf413fb70 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf9199700 1 -- 172.21.15.33:0/3283430960 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be0005780 con 0x7f8bf413fb70 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf999a700 1 -- 172.21.15.33:0/3283430960 >> v1:172.21.15.167:6789/0 conn(0x7f8bf40af000 legacy=0x7f8bf40c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf9199700 1 -- 172.21.15.33:0/3283430960 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 606989652 0 0) 0x7f8be0005780 con 0x7f8bf413fb70 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf999a700 1 -- 172.21.15.33:0/3283430960 >> v1:172.21.15.33:6789/0 conn(0x7f8bf413fb70 legacy=0x7f8bf406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf999a700 1 -- 172.21.15.33:0/3283430960 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf406afe0 con 0x7f8bf40c9e10 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf999a700 1 --2- 172.21.15.33:0/3283430960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf40ca290 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098e0 tx=0x7f8be40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4bc18289ebf0f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf9199700 1 -- 172.21.15.33:0/3283430960 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0001470 con 0x7f8bf40c9e10 2023-10-06T05:36:06.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf9199700 1 -- 172.21.15.33:0/3283430960 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4011420 con 0x7f8bf40c9e10 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.281+0000 7f8bf9199700 1 -- 172.21.15.33:0/3283430960 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be40117c0 con 0x7f8bf40c9e10 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/3283430960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 msgr2=0x7f8bf40ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/3283430960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf40ca290 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098e0 tx=0x7f8be40091a0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/3283430960 shutdown_connections 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/3283430960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf40ca290 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/3283430960 >> 172.21.15.33:0/3283430960 conn(0x7f8bf40675f0 msgr2=0x7f8bf40679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/3283430960 shutdown_connections 2023-10-06T05:36:06.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.282+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/3283430960 wait complete. 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.283+0000 7f8bfc3ff700 1 Processor -- start 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.284+0000 7f8bfc3ff700 1 -- start start 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.284+0000 7f8bfc3ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bf40af000 0x7f8bf41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.284+0000 7f8bfc3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf4156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfc3ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 0x7f8bf4169310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfc3ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4157f00 con 0x7f8bf413fb70 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfc3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4157090 con 0x7f8bf40c9e10 2023-10-06T05:36:06.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfc3ff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41572f0 con 0x7f8bf40af000 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 0x7f8bf4169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 0x7f8bf4169310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36184/0 (socket says 172.21.15.33:36184) 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 -- 172.21.15.33:0/4133340411 learned_addr learned my addr 172.21.15.33:0/4133340411 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bf999a700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf4156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa19b700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bf40af000 0x7f8bf41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 -- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bf40af000 msgr2=0x7f8bf41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bf40af000 0x7f8bf41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 -- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 msgr2=0x7f8bf4156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf4156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.285+0000 7f8bfa99c700 1 -- 172.21.15.33:0/4133340411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4009420 con 0x7f8bf413fb70 2023-10-06T05:36:06.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.286+0000 7f8bfa99c700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 0x7f8bf4169310 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c8e0 tx=0x7f8bec001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:06.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.286+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec012630 con 0x7f8bf413fb70 2023-10-06T05:36:06.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.286+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec012790 con 0x7f8bf413fb70 2023-10-06T05:36:06.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.286+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf41575d0 con 0x7f8bf413fb70 2023-10-06T05:36:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.286+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec012b50 con 0x7f8bf413fb70 2023-10-06T05:36:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.286+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf4157b90 con 0x7f8bf413fb70 2023-10-06T05:36:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.287+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4063ff0 con 0x7f8bf413fb70 2023-10-06T05:36:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.288+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bec00a3b0 con 0x7f8bf413fb70 2023-10-06T05:36:06.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.289+0000 7f8bea7fc700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8bd8066f30 0x7f8bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.289+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8bec0847d0 con 0x7f8bf413fb70 2023-10-06T05:36:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.289+0000 7f8bfa19b700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8bd8066f30 0x7f8bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.290+0000 7f8bfa19b700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8bd8066f30 0x7f8bd80693e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40aeb00 tx=0x7f8bf0001120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:06.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.293+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bec051b10 con 0x7f8bf413fb70 2023-10-06T05:36:06.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.491+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 --> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bf406d570 con 0x7f8bd8066f30 2023-10-06T05:36:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.494+0000 7f8bea7fc700 1 -- 172.21.15.33:0/4133340411 <== mgr.4099 v2:172.21.15.167:6832/85179 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72914 (secure 0 0 0) 0x7f8bf406d570 con 0x7f8bd8066f30 2023-10-06T05:36:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:36:06.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8bd8066f30 msgr2=0x7f8bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8bd8066f30 0x7f8bd80693e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40aeb00 tx=0x7f8bf0001120 comp rx=0 tx=0).stop 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 msgr2=0x7f8bf4169310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 0x7f8bf4169310 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c8e0 tx=0x7f8bec001710 comp rx=0 tx=0).stop 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 shutdown_connections 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f8bd8066f30 0x7f8bd80693e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf413fb70 0x7f8bf4169310 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8bf40af000 0x7f8bf41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 --2- 172.21.15.33:0/4133340411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bf40c9e10 0x7f8bf4156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 >> 172.21.15.33:0/4133340411 conn(0x7f8bf40675f0 msgr2=0x7f8bf40679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 shutdown_connections 2023-10-06T05:36:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.497+0000 7f8bfc3ff700 1 -- 172.21.15.33:0/4133340411 wait complete. 2023-10-06T05:36:06.500 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T05:36:06.516 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":936,"stamp":"2023-10-06T05:36:06.153541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2473416,"kb_used_data":29576,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752501304,"statfs":{"total":773094113280,"available":770561335296,"internally_reserved":0,"allocated":30285824,"data_stored":14857640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002947"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":246,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:47.466903+0000","last_change":"2023-10-06T05:35:47.466868+0000","last_active":"2023-10-06T05:35:47.466903+0000","last_peered":"2023-10-06T05:35:47.466903+0000","last_clean":"2023-10-06T05:35:47.466903+0000","last_became_active":"2023-10-06T05:14:19.710617+0000","last_became_peered":"2023-10-06T05:14:19.710617+0000","last_unstale":"2023-10-06T05:35:47.466903+0000","last_undegraded":"2023-10-06T05:35:47.466903+0000","last_fullsized":"2023-10-06T05:35:47.466903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:47.466750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:47.466750+0000","last_clean_scrub_stamp":"2023-10-06T05:35:47.466750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:28.338109+0000","scrub_duration":0.002063881,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":241,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:45.056839+0000","last_change":"2023-10-06T05:35:45.056787+0000","last_active":"2023-10-06T05:35:45.056839+0000","last_peered":"2023-10-06T05:35:45.056839+0000","last_clean":"2023-10-06T05:35:45.056839+0000","last_became_active":"2023-10-06T05:14:19.680003+0000","last_became_peered":"2023-10-06T05:14:19.680003+0000","last_unstale":"2023-10-06T05:35:45.056839+0000","last_undegraded":"2023-10-06T05:35:45.056839+0000","last_fullsized":"2023-10-06T05:35:45.056839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:45.056676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:45.056676+0000","last_clean_scrub_stamp":"2023-10-06T05:35:45.056676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:56.005234+0000","scrub_duration":0.0020007240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:44.612922+0000","last_change":"2023-10-06T05:35:44.612898+0000","last_active":"2023-10-06T05:35:44.612922+0000","last_peered":"2023-10-06T05:35:44.612922+0000","last_clean":"2023-10-06T05:35:44.612922+0000","last_became_active":"2023-10-06T05:14:19.778263+0000","last_became_peered":"2023-10-06T05:14:19.778263+0000","last_unstale":"2023-10-06T05:35:44.612922+0000","last_undegraded":"2023-10-06T05:35:44.612922+0000","last_fullsized":"2023-10-06T05:35:44.612922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:44.612820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:44.612820+0000","last_clean_scrub_stamp":"2023-10-06T05:35:44.612820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:23.019577+0000","scrub_duration":0.001867145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":241,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:46.047833+0000","last_change":"2023-10-06T05:35:46.047801+0000","last_active":"2023-10-06T05:35:46.047833+0000","last_peered":"2023-10-06T05:35:46.047833+0000","last_clean":"2023-10-06T05:35:46.047833+0000","last_became_active":"2023-10-06T05:14:19.679594+0000","last_became_peered":"2023-10-06T05:14:19.679594+0000","last_unstale":"2023-10-06T05:35:46.047833+0000","last_undegraded":"2023-10-06T05:35:46.047833+0000","last_fullsized":"2023-10-06T05:35:46.047833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:46.047667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:46.047667+0000","last_clean_scrub_stamp":"2023-10-06T05:35:46.047667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:42.024496+0000","scrub_duration":0.0023406289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"286'119","reported_seq":393,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:47.513166+0000","last_change":"2023-10-06T05:35:47.513132+0000","last_active":"2023-10-06T05:35:47.513166+0000","last_peered":"2023-10-06T05:35:47.513166+0000","last_clean":"2023-10-06T05:35:47.513166+0000","last_became_active":"2023-10-06T05:14:13.686571+0000","last_became_peered":"2023-10-06T05:14:13.686571+0000","last_unstale":"2023-10-06T05:35:47.513166+0000","last_undegraded":"2023-10-06T05:35:47.513166+0000","last_fullsized":"2023-10-06T05:35:47.513166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"286'119","last_scrub_stamp":"2023-10-06T05:35:47.513029+0000","last_deep_scrub":"286'119","last_deep_scrub_stamp":"2023-10-06T05:35:47.513029+0000","last_clean_scrub_stamp":"2023-10-06T05:35:47.513029+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:28.384388+0000","scrub_duration":0.006196188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:50.557974+0000","last_change":"2023-10-06T05:35:50.557939+0000","last_active":"2023-10-06T05:35:50.557974+0000","last_peered":"2023-10-06T05:35:50.557974+0000","last_clean":"2023-10-06T05:35:50.557974+0000","last_became_active":"2023-10-06T05:14:19.678573+0000","last_became_peered":"2023-10-06T05:14:19.678573+0000","last_unstale":"2023-10-06T05:35:50.557974+0000","last_undegraded":"2023-10-06T05:35:50.557974+0000","last_fullsized":"2023-10-06T05:35:50.557974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:50.557812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:50.557812+0000","last_clean_scrub_stamp":"2023-10-06T05:35:50.557812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:12.077361+0000","scrub_duration":0.0024537690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:52.551127+0000","last_change":"2023-10-06T05:35:52.551094+0000","last_active":"2023-10-06T05:35:52.551127+0000","last_peered":"2023-10-06T05:35:52.551127+0000","last_clean":"2023-10-06T05:35:52.551127+0000","last_became_active":"2023-10-06T05:14:19.710533+0000","last_became_peered":"2023-10-06T05:14:19.710533+0000","last_unstale":"2023-10-06T05:35:52.551127+0000","last_undegraded":"2023-10-06T05:35:52.551127+0000","last_fullsized":"2023-10-06T05:35:52.551127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:52.550987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:52.550987+0000","last_clean_scrub_stamp":"2023-10-06T05:35:52.550987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:56.283516+0000","scrub_duration":0.0026365960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":246,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:50.378891+0000","last_change":"2023-10-06T05:35:50.378857+0000","last_active":"2023-10-06T05:35:50.378891+0000","last_peered":"2023-10-06T05:35:50.378891+0000","last_clean":"2023-10-06T05:35:50.378891+0000","last_became_active":"2023-10-06T05:14:19.675815+0000","last_became_peered":"2023-10-06T05:14:19.675815+0000","last_unstale":"2023-10-06T05:35:50.378891+0000","last_undegraded":"2023-10-06T05:35:50.378891+0000","last_fullsized":"2023-10-06T05:35:50.378891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:50.378754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:50.378754+0000","last_clean_scrub_stamp":"2023-10-06T05:35:50.378754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:11.898304+0000","scrub_duration":0.0027268380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":283,"reported_epoch":286,"state":"active+clean","last_fresh":"2023-10-06T05:35:48.921245+0000","last_change":"2023-10-06T05:35:48.921199+0000","last_active":"2023-10-06T05:35:48.921245+0000","last_peered":"2023-10-06T05:35:48.921245+0000","last_clean":"2023-10-06T05:35:48.921245+0000","last_became_active":"2023-10-06T05:14:19.678956+0000","last_became_peered":"2023-10-06T05:14:19.678956+0000","last_unstale":"2023-10-06T05:35:48.921245+0000","last_undegraded":"2023-10-06T05:35:48.921245+0000","last_fullsized":"2023-10-06T05:35:48.921245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:35:48.921092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:35:48.921092+0000","last_clean_scrub_stamp":"2023-10-06T05:35:48.921092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:08.745641+0000","scrub_duration":0.0019576540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575115,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309612,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062228,"statfs":{"total":96636764160,"available":96319721472,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":1.0069999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0189999999999999,"15min":1.1339999999999999},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69499999999999995,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0860000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.38300000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0840000000000001},"last":0.623}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.163},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.0409999999999999},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1599999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.46999999999999997,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0960000000000001},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.91400000000000003,"5min":0.96399999999999997,"15min":1.109},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95199999999999996,"5min":1.046,"15min":1.234},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.41199999999999998},"max":{"1min":0.91800000000000004,"5min":0.999,"15min":1.109},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.38500000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.2290000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.169},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.073,"15min":1.242},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834575113,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.67400000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76800000000000002,"5min":1.0009999999999999,"15min":1.0720000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66000000000000003,"15min":0.67000000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.754,"5min":1.0169999999999999,"15min":1.0369999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.42499999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.0860000000000001,"15min":1.137},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.40799999999999997},"max":{"1min":0.82499999999999996,"5min":1.1000000000000001,"15min":1.135},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95599999999999996,"5min":1.0569999999999999,"15min":1.0980000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.45900000000000002,"15min":0.442},"max":{"1min":0.77400000000000002,"5min":1.075,"15min":1.115},"last":0.754}]},{"osd":3,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68799999999999994,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.40899999999999997},"max":{"1min":0.78900000000000003,"5min":1.0009999999999999,"15min":1.1160000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.86099999999999999,"5min":1.006,"15min":1.0229999999999999},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68000000000000005,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.624},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.34999999999999998},"max":{"1min":0.86499999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69999999999999996,"15min":0.68799999999999994},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92900000000000005,"5min":1.0469999999999999,"15min":1.0669999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72999999999999998,"15min":0.72399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90100000000000002,"5min":1.1259999999999999,"15min":1.4610000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:35:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.71599999999999997,"15min":0.71299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86099999999999999,"5min":1.143,"15min":1.143},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0149999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575114,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.67300000000000004,"5min":1.272,"15min":1.272},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.248},"max":{"1min":0.97199999999999998,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75,"5min":0.75,"15min":1.165},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80200000000000005,"5min":1.22,"15min":1.22},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":0.91400000000000003,"15min":1.2230000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.33600000000000002},"max":{"1min":0.999,"5min":1.238,"15min":1.2430000000000001},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78500000000000003,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.51500000000000001,"15min":0.48799999999999999},"max":{"1min":1.0880000000000001,"5min":1.2929999999999999,"15min":1.5389999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79300000000000004,"15min":0.77600000000000002},"min":{"1min":0.61099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0820000000000001,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.58099999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1319999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.81000000000000005,"15min":0.79800000000000004},"min":{"1min":0.51100000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.2190000000000001,"5min":1.492,"15min":1.492},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.84299999999999997,"15min":0.82599999999999996},"min":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.48199999999999998},"max":{"1min":1.1479999999999999,"5min":1.262,"15min":1.2909999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.83599999999999997,"15min":0.83299999999999996},"min":{"1min":0.64300000000000002,"5min":0.53500000000000003,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.115,"15min":1.3280000000000001},"last":0.82099999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:36:01 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88900000000000001,"15min":0.83599999999999997},"min":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.46999999999999997},"max":{"1min":1.159,"5min":7.0019999999999998,"15min":7.0019999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85699999999999998,"15min":0.82799999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1970000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.99299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575114,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313708,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058132,"statfs":{"total":96636764160,"available":96315527168,"internally_reserved":0,"allocated":4231168,"data_stored":2299981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.28699999999999998,"5min":0.27000000000000002,"15min":0.245},"max":{"1min":0.64100000000000001,"5min":0.73799999999999999,"15min":0.85199999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.36899999999999999,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.66100000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.40400000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.79900000000000004,"5min":0.90600000000000003,"15min":0.90800000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.71699999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47899999999999998,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.307},"max":{"1min":0.54200000000000004,"5min":0.77500000000000002,"15min":0.93600000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.71199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94099999999999995,"5min":1.1100000000000001,"15min":2.8220000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.47799999999999998},"max":{"1min":0.94299999999999995,"5min":1.1719999999999999,"15min":3.6629999999999998},"last":0.873}]},{"osd":5,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77900000000000003,"15min":0.77800000000000002},"min":{"1min":0.57299999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.0980000000000001,"5min":1.1379999999999999,"15min":1.27},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.78300000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0620000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.53900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":1.1319999999999999,"15min":1.1830000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45500000000000002},"max":{"1min":1.0589999999999999,"5min":1.147,"15min":1.153},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.496},"max":{"1min":0.995,"5min":1.2,"15min":1.2010000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79200000000000004,"15min":0.79600000000000004},"min":{"1min":0.55500000000000005,"5min":0.51800000000000002,"15min":0.50600000000000001},"max":{"1min":0.93000000000000005,"5min":1.1459999999999999,"15min":1.274},"last":1.0269999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575114,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313128,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94058712,"statfs":{"total":96636764160,"available":96316121088,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46700000000000003,"15min":0.45100000000000001},"min":{"1min":0.314,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71199999999999997,"5min":0.82999999999999996,"15min":1.056},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48599999999999999,"15min":0.47099999999999997},"min":{"1min":0.33100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.88200000000000001,"15min":1.0649999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":1.073},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51000000000000001,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":1.117,"15min":1.117},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57299999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.36699999999999999,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.77500000000000002},"min":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.149,"5min":1.2709999999999999,"15min":1.486},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79200000000000004,"15min":0.77000000000000002},"min":{"1min":0.59699999999999998,"5min":0.48899999999999999,"15min":0.45100000000000001},"max":{"1min":1.016,"5min":1.089,"15min":1.5},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80000000000000004,"15min":0.76000000000000001},"min":{"1min":0.628,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.879},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.61499999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0600000000000001,"5min":1.1299999999999999,"15min":1.1859999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80800000000000005,"15min":0.78200000000000003},"min":{"1min":0.56599999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.042,"5min":1.22,"15min":1.22},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81299999999999994,"15min":0.78500000000000003},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0229999999999999,"5min":1.341,"15min":1.341},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81000000000000005,"15min":0.78800000000000003},"min":{"1min":0.53000000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0569999999999999,"5min":1.282,"15min":1.538},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.79700000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51600000000000001},"max":{"1min":1.1319999999999999,"5min":1.264,"15min":1.264},"last":0.83699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575115,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44,"15min":0.44900000000000001},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.748,"15min":0.89200000000000002},"last":0.438},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.31900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60099999999999998,"5min":0.80700000000000005,"15min":1.369},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32500000000000001},"max":{"1min":0.626,"5min":0.84199999999999997,"15min":1.5369999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.878,"15min":0.90200000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.46800000000000003},"max":{"1min":0.91200000000000003,"5min":1.129,"15min":1.129},"last":0.747},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.101,"15min":1.3009999999999999},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.113,"15min":1.145},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.57799999999999996,"5min":0.54900000000000004,"15min":0.51100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0900000000000001,"15min":1.4590000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.3360000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.1779999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:35:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.80300000000000005},"min":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.57099999999999995},"max":{"1min":1.0409999999999999,"5min":1.0660000000000001,"15min":1.194},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.82199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575113,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":309032,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94062808,"statfs":{"total":96636764160,"available":96320315392,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.71599999999999997},"min":{"1min":0.443,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":1.214,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77500000000000002,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.96399999999999997,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.39800000000000002},"max":{"1min":0.871,"5min":1.026,"15min":1.026},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72699999999999998,"15min":0.70299999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1180000000000001,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.438,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":4.2169999999999996,"15min":4.2169999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.50800000000000001,"15min":0.41799999999999998},"max":{"1min":0.91700000000000004,"5min":1.099,"15min":1.099},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":3.839,"15min":3.839},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.76700000000000002,"15min":0.746},"min":{"1min":0.53300000000000003,"5min":0.51200000000000001,"15min":0.47799999999999998},"max":{"1min":0.90600000000000003,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.96999999999999997,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94299999999999995,"5min":6.5259999999999998,"15min":6.5259999999999998},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77000000000000002,"15min":0.71999999999999997},"min":{"1min":0.44900000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0740000000000001,"5min":7.9459999999999997,"15min":7.9459999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.48699999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.54300000000000004,"5min":0.48699999999999999,"15min":0.436},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.282},"last":0.87},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80400000000000005,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.46100000000000002,"15min":0.45200000000000001},"max":{"1min":1.048,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.83699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575115,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304936,"kb_used_data":3552,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94066904,"statfs":{"total":96636764160,"available":96324509696,"internally_reserved":0,"allocated":3637248,"data_stored":1709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.98499999999999999,"15min":2.4609999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90700000000000003,"5min":1.0429999999999999,"15min":1.3460000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.372},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.2130000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.40899999999999997},"max":{"1min":0.82699999999999996,"5min":1.0940000000000001,"15min":1.1060000000000001},"last":0.874}]},{"osd":2,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.68999999999999995},"min":{"1min":0.58299999999999996,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.86699999999999999,"5min":1.024,"15min":1.0289999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.74099999999999999,"15min":0.72699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41399999999999998},"max":{"1min":0.89700000000000002,"5min":1.169,"15min":1.169},"last":0.80900000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.72699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.57399999999999995,"5min":0.51300000000000001,"15min":0.47999999999999998},"max":{"1min":0.97599999999999998,"5min":1.089,"15min":1.145},"last":0.85699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66400000000000003,"15min":0.64800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":1.0089999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.68700000000000006},"min":{"1min":0.495,"5min":0.45100000000000001,"15min":0.41999999999999998},"max":{"1min":0.96099999999999997,"5min":0.96299999999999997,"15min":1.1539999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.41899999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.748,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.47499999999999998,"5min":0.46300000000000002,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.746},"min":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.40899999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:36:06.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T05:36:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.699+0000 7f118239d700 1 Processor -- start 2023-10-06T05:36:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.699+0000 7f118239d700 1 -- start start 2023-10-06T05:36:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.700+0000 7f118239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 0x7f117c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.700+0000 7f118239d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0b06b0 con 0x7f117c155a50 2023-10-06T05:36:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.700+0000 7f118239d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0c99e0 con 0x7f117c0afb20 2023-10-06T05:36:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.700+0000 7f118239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c9cc0 con 0x7f117c0c9490 2023-10-06T05:36:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f118139b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f117c155a50 0x7f117c0af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42542/0 (socket says 172.21.15.33:42542) 2023-10-06T05:36:06.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f118139b700 1 -- 172.21.15.33:0/2910665596 learned_addr learned my addr 172.21.15.33:0/2910665596 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:36:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745750957 0 0) 0x7f117c0b06b0 con 0x7f117c155a50 2023-10-06T05:36:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1180b9a700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 0x7f117c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003570 con 0x7f117c155a50 2023-10-06T05:36:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470854660 0 0) 0x7f1168003570 con 0x7f117c155a50 2023-10-06T05:36:06.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 msgr2=0x7f117c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 0x7f117c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 >> v1:172.21.15.167:6789/0 conn(0x7f117c0afb20 legacy=0x7f117c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0c9fa0 con 0x7f117c155a50 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.701+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7f1178001470 con 0x7f117c155a50 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 >> v1:172.21.15.33:6789/0 conn(0x7f117c155a50 legacy=0x7f117c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 0x7f1168003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1168003fd0 0x7f1168004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1168004d50 0x7f11680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c9cc0 con 0x7f1168003fd0 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116800bb40 con 0x7f1168004d50 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116800bd00 con 0x7f117c0c9490 2023-10-06T05:36:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1180b9a700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1168003fd0 0x7f1168004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f118139b700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 0x7f1168003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1181b9c700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1168004d50 0x7f11680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1180b9a700 1 -- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 msgr2=0x7f1168003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1180b9a700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117c0c9490 0x7f1168003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1180b9a700 1 -- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1168004d50 msgr2=0x7f11680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1180b9a700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1168004d50 0x7f11680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1180b9a700 1 -- 172.21.15.33:0/2910665596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168003570 con 0x7f1168003fd0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.702+0000 7f1181b9c700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1168004d50 0x7f11680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f1180b9a700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1168003fd0 0x7f1168004810 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f116c009c00 tx=0x7f116c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b93d90ca0d00b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00e710 con 0x7f1168003fd0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c00e870 con 0x7f1168003fd0 2023-10-06T05:36:06.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f1173fff700 1 -- 172.21.15.33:0/2910665596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00ec30 con 0x7f1168003fd0 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f118239d700 1 -- 172.21.15.33:0/2910665596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1168003fd0 msgr2=0x7f1168004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f118239d700 1 --2- 172.21.15.33:0/2910665596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1168003fd0 0x7f1168004810 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f116c009c00 tx=0x7f116c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f1181b9c700 1 -- 172.21.15.33:0/2910665596 reap_dead start 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f118239d700 1 -- 172.21.15.33:0/2910665596 shutdown_connections 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.703+0000 7f118239d700 1 -- 172.21.15.33:0/2910665596 >> 172.21.15.33:0/2910665596 conn(0x7f117c06d030 msgr2=0x7f117c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.704+0000 7f118239d700 1 -- 172.21.15.33:0/2910665596 shutdown_connections 2023-10-06T05:36:06.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.704+0000 7f118239d700 1 -- 172.21.15.33:0/2910665596 wait complete. 2023-10-06T05:36:06.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.705+0000 7f118239d700 1 Processor -- start 2023-10-06T05:36:06.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.705+0000 7f118239d700 1 -- start start 2023-10-06T05:36:06.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f118239d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1168003fd0 0x7f117c143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f118239d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c15d380 0x7f117c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f118239d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 0x7f117c0c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f118239d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c2ab0 con 0x7f117c0c2100 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f118239d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c15ea50 con 0x7f117c15d380 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f118239d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c15de40 con 0x7f1168003fd0 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f1181b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 0x7f117c0c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f1181b9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 0x7f117c0c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36214/0 (socket says 172.21.15.33:36214) 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.706+0000 7f1181b9c700 1 -- 172.21.15.33:0/609654820 learned_addr learned my addr 172.21.15.33:0/609654820 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f118139b700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1168003fd0 0x7f117c143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1180b9a700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c15d380 0x7f117c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1181b9c700 1 -- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1168003fd0 msgr2=0x7f117c143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1181b9c700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1168003fd0 0x7f117c143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1181b9c700 1 -- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c15d380 msgr2=0x7f117c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1181b9c700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c15d380 0x7f117c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1181b9c700 1 -- 172.21.15.33:0/609654820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c15e120 con 0x7f117c0c2100 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f118139b700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1168003fd0 0x7f117c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1180b9a700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c15d380 0x7f117c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f1181b9c700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 0x7f117c0c2570 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f117c0cc0a0 tx=0x7f1174008bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174006db0 con 0x7f117c0c2100 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11740080d0 con 0x7f117c0c2100 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174008490 con 0x7f117c0c2100 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.707+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c15e400 con 0x7f117c0c2100 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.708+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c0c10e0 con 0x7f117c0c2100 2023-10-06T05:36:06.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.709+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c04c380 con 0x7f117c0c2100 2023-10-06T05:36:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.709+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1174003070 con 0x7f117c0c2100 2023-10-06T05:36:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.710+0000 7f11717fa700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1160066f30 0x7f11600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.710+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1174083b40 con 0x7f117c0c2100 2023-10-06T05:36:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.711+0000 7f118139b700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1160066f30 0x7f11600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.711+0000 7f118139b700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1160066f30 0x7f11600693e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1178000e10 tx=0x7f1178009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:06.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.714+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1174050e80 con 0x7f117c0c2100 2023-10-06T05:36:06.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.916+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f117c15eb90 con 0x7f117c0c2100 2023-10-06T05:36:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.921+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1174008d20 con 0x7f117c0c2100 2023-10-06T05:36:06.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.921+0000 7f11717fa700 1 -- 172.21.15.33:0/609654820 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v48)=0 v48) v1 ==== 123+0+0 (secure 0 0 0) 0x7f11740190c0 con 0x7f117c0c2100 2023-10-06T05:36:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.923+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1160066f30 msgr2=0x7f11600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.923+0000 7f118239d700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1160066f30 0x7f11600693e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1178000e10 tx=0x7f1178009040 comp rx=0 tx=0).stop 2023-10-06T05:36:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 msgr2=0x7f117c0c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 0x7f117c0c2570 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f117c0cc0a0 tx=0x7f1174008bc0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 shutdown_connections 2023-10-06T05:36:06.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7f1160066f30 0x7f11600693e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f117c0c2100 0x7f117c0c2570 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1168003fd0 0x7f117c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 --2- 172.21.15.33:0/609654820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c15d380 0x7f117c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 >> 172.21.15.33:0/609654820 conn(0x7f117c06d030 msgr2=0x7f117c069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:36:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 shutdown_connections 2023-10-06T05:36:06.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:06.924+0000 7f118239d700 1 -- 172.21.15.33:0/609654820 wait complete. 2023-10-06T05:36:06.943 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:36:06.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:36:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.131+0000 7fc73259d700 1 Processor -- start 2023-10-06T05:36:07.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.131+0000 7fc73259d700 1 -- start start 2023-10-06T05:36:07.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.132+0000 7fc73259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0cf3b0 0x7fc72c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.132+0000 7fc73259d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc72c0af0a0 0x7fc72c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.132+0000 7fc73259d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72c0cbd80 con 0x7fc72c0ccfc0 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.132+0000 7fc73259d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72c0af5f0 con 0x7fc72c0af0a0 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.132+0000 7fc73259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0af8d0 con 0x7fc72c0cf3b0 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc73159b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc72c0ccfc0 0x7fc72c0cc210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42552/0 (socket says 172.21.15.33:42552) 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc73159b700 1 -- 172.21.15.33:0/480838394 learned_addr learned my addr 172.21.15.33:0/480838394 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068642358 0 0) 0x7fc72c0cbd80 con 0x7fc72c0ccfc0 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc718003570 con 0x7fc72c0ccfc0 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc730d9a700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0cf3b0 0x7fc72c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc731d9c700 1 --2- 172.21.15.33:0/480838394 >> v2:172.21.15.167:3300/0 conn(0x7fc72c0af0a0 0x7fc72c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 616342335 0 0) 0x7fc718003570 con 0x7fc72c0ccfc0 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0cf3b0 msgr2=0x7fc72c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0cf3b0 0x7fc72c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 >> v2:172.21.15.167:3300/0 conn(0x7fc72c0af0a0 msgr2=0x7fc72c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 --2- 172.21.15.33:0/480838394 >> v2:172.21.15.167:3300/0 conn(0x7fc72c0af0a0 0x7fc72c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0afbb0 con 0x7fc72c0ccfc0 2023-10-06T05:36:07.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.133+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3659070531 0 0) 0x7fc728001470 con 0x7fc72c0ccfc0 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 >> v1:172.21.15.33:6789/0 conn(0x7fc72c0ccfc0 legacy=0x7fc72c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc718003a60 0x7fc718003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7180043b0 0x7fc718004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc718004ee0 0x7fc71800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0af5f0 con 0x7fc7180043b0 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0af8d0 con 0x7fc718004ee0 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180056d0 con 0x7fc718003a60 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc730d9a700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7180043b0 0x7fc718004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc73159b700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc718003a60 0x7fc718003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc731d9c700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc718004ee0 0x7fc71800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc730d9a700 1 -- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc718003a60 msgr2=0x7fc718003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc730d9a700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc718003a60 0x7fc718003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc730d9a700 1 -- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc718004ee0 msgr2=0x7fc71800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc730d9a700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc718004ee0 0x7fc71800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc730d9a700 1 -- 172.21.15.33:0/480838394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718003570 con 0x7fc7180043b0 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc731d9c700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc718004ee0 0x7fc71800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:36:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.134+0000 7fc73159b700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc718003a60 0x7fc718003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:36:07.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc730d9a700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7180043b0 0x7fc718004870 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0b0250 tx=0x7fc71c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e66c4134045b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:07.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00e710 con 0x7fc7180043b0 2023-10-06T05:36:07.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc71c00e870 con 0x7fc7180043b0 2023-10-06T05:36:07.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc723fff700 1 -- 172.21.15.33:0/480838394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00ec50 con 0x7fc7180043b0 2023-10-06T05:36:07.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc73259d700 1 -- 172.21.15.33:0/480838394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7180043b0 msgr2=0x7fc718004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc73259d700 1 --2- 172.21.15.33:0/480838394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7180043b0 0x7fc718004870 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0b0250 tx=0x7fc71c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc731d9c700 1 -- 172.21.15.33:0/480838394 reap_dead start 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc73259d700 1 -- 172.21.15.33:0/480838394 shutdown_connections 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc73259d700 1 -- 172.21.15.33:0/480838394 >> 172.21.15.33:0/480838394 conn(0x7fc72c06cff0 msgr2=0x7fc72c06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.135+0000 7fc73259d700 1 -- 172.21.15.33:0/480838394 shutdown_connections 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.136+0000 7fc73259d700 1 -- 172.21.15.33:0/480838394 wait complete. 2023-10-06T05:36:07.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.137+0000 7fc73259d700 1 Processor -- start 2023-10-06T05:36:07.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.137+0000 7fc73259d700 1 -- start start 2023-10-06T05:36:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73259d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7180043b0 0x7fc72c154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73259d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c0b42a0 0x7fc72c0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73259d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 0x7fc72c0c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73259d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0b4dd0 con 0x7fc72c0c62e0 2023-10-06T05:36:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73259d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0c6cc0 con 0x7fc72c0b42a0 2023-10-06T05:36:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73259d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1696c0 con 0x7fc7180043b0 2023-10-06T05:36:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc731d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 0x7fc72c0c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc731d9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 0x7fc72c0c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36238/0 (socket says 172.21.15.33:36238) 2023-10-06T05:36:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc731d9c700 1 -- 172.21.15.33:0/2241195161 learned_addr learned my addr 172.21.15.33:0/2241195161 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T05:36:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc73159b700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7180043b0 0x7fc72c154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.138+0000 7fc730d9a700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c0b42a0 0x7fc72c0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc731d9c700 1 -- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7180043b0 msgr2=0x7fc72c154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc731d9c700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7180043b0 0x7fc72c154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc731d9c700 1 -- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c0b42a0 msgr2=0x7fc72c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc731d9c700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c0b42a0 0x7fc72c0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc731d9c700 1 -- 172.21.15.33:0/2241195161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c16a200 con 0x7fc72c0c62e0 2023-10-06T05:36:07.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc73159b700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7180043b0 0x7fc72c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:36:07.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc730d9a700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c0b42a0 0x7fc72c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:36:07.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc731d9c700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 0x7fc72c0c6780 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0d1550 tx=0x7fc72400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:07.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72400e030 con 0x7fc72c0c62e0 2023-10-06T05:36:07.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc724011d00 con 0x7fc72c0c62e0 2023-10-06T05:36:07.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c0c8540 con 0x7fc72c0c62e0 2023-10-06T05:36:07.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7240166e0 con 0x7fc72c0c62e0 2023-10-06T05:36:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.139+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72c169bb0 con 0x7fc72c0c62e0 2023-10-06T05:36:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.140+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72c16a390 con 0x7fc72c0c62e0 2023-10-06T05:36:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.141+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc724016a90 con 0x7fc72c0c62e0 2023-10-06T05:36:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.142+0000 7fc7217fa700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc710066f30 0x7fc7100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:36:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.142+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(286..286 src has 1..286) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc72400f070 con 0x7fc72c0c62e0 2023-10-06T05:36:07.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.142+0000 7fc73159b700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc710066f30 0x7fc7100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:36:07.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.143+0000 7fc73159b700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc710066f30 0x7fc7100693e0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc728000e10 tx=0x7fc728009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:36:07.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.146+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc724052200 con 0x7fc72c0c62e0 2023-10-06T05:36:07.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.393+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc72c159f20 con 0x7fc72c0c62e0 2023-10-06T05:36:07.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.394+0000 7fc7217fa700 1 -- 172.21.15.33:0/2241195161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) v1 ==== 95+0+3542 (secure 0 0 0) 0x7fc724055820 con 0x7fc72c0c62e0 2023-10-06T05:36:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T05:36:07.395 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":221,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1757117622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1757117622},{"type":"v1","addr":"172.21.15.167:6837","nonce":1757117622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214},{"gid":9371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1227588647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1227588647},{"type":"v1","addr":"172.21.15.167:6836","nonce":1227588647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":221},{"gid":12252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/1759201535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1759201535},{"type":"v1","addr":"172.21.15.33:6834","nonce":1759201535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":213},{"gid":12255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3189300076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":3189300076},{"type":"v1","addr":"172.21.15.33:6835","nonce":3189300076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":214}],"filesystems":[]} 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc710066f30 msgr2=0x7fc7100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc710066f30 0x7fc7100693e0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc728000e10 tx=0x7fc728009040 comp rx=0 tx=0).stop 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 msgr2=0x7fc72c0c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 0x7fc72c0c6780 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0d1550 tx=0x7fc72400b040 comp rx=0 tx=0).stop 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 shutdown_connections 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:6832/85179,v1:172.21.15.167:6833/85179] conn(0x7fc710066f30 0x7fc7100693e0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc72c0c62e0 0x7fc72c0c6780 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7180043b0 0x7fc72c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 --2- 172.21.15.33:0/2241195161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c0b42a0 0x7fc72c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:36:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 >> 172.21.15.33:0/2241195161 conn(0x7fc72c06cff0 msgr2=0x7fc72c0cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:36:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 shutdown_connections 2023-10-06T05:36:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T05:36:07.396+0000 7fc73259d700 1 -- 172.21.15.33:0/2241195161 wait complete. 2023-10-06T05:36:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 221 2023-10-06T05:36:07.415 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T05:36:07.415 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T05:36:07.415 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:36:07.416 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.428 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:36:07.428 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:36:07.428 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.482 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:36:07.482 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:36:07.482 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.497 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:36:07.497 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:36:07.497 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.553 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:36:07.553 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T05:36:07.553 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T05:36:07.553 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.595 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T05:36:07.595 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T05:36:07.596 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.639 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T05:36:07.639 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T05:36:07.639 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.688 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T05:36:07.688 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T05:36:07.688 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.747 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T05:36:07.748 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T05:36:07.748 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.792 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T05:36:07.792 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T05:36:07.792 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.847 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T05:36:07.847 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T05:36:07.847 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.895 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T05:36:07.895 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T05:36:07.895 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:07.946 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T05:36:07.946 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T05:36:07.946 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T05:36:07.946 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:08.005 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T05:36:08.005 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T05:36:08.005 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:08.069 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T05:36:08.069 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T05:36:08.069 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T05:36:08.070 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:08.093 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T05:36:08.093 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T05:36:08.093 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:08.112 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T05:36:08.112 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T05:36:08.112 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:36:08.171 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T05:36:08.172 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T05:36:08.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 't responding to mclientcaps' | head -n 1 2023-10-06T05:36:08.247 INFO:teuthology.orchestra.run.smithi033.stdout:1696569264.6939275 mon.a (mon.0) 140 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:36:08.248 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T05:36:08.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 't responding to mclientcaps' | head -n 1 2023-10-06T05:36:08.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 't responding to mclientcaps' | head -n 1 2023-10-06T05:36:08.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 't responding to mclientcaps' | head -n 1 2023-10-06T05:36:08.478 INFO:teuthology.orchestra.run.smithi033.stdout:1696569264.6939275 mon.a (mon.0) 140 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:36:08.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:36:08.479 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T05:36:08.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:36:08.800 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T05:36:09.005 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:36:09.005 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T05:36:09.214 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:36:09.214 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T05:36:09.409 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:36:09.410 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T05:36:09.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:36:09.778 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T05:36:09.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:36:09.981 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T05:36:10.201 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:36:10.201 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T05:36:10.383 INFO:tasks.ceph:Archiving mon data... 2023-10-06T05:36:10.384 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/data/mon.a.tgz 2023-10-06T05:36:10.384 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2023-10-06T05:36:10.425 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.VHGvwSyqba 2023-10-06T05:36:10.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.VHGvwSyqba 2023-10-06T05:36:10.650 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.VHGvwSyqba 2023-10-06T05:36:10.753 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.VHGvwSyqba is 509KB 2023-10-06T05:36:10.845 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.VHGvwSyqba 2023-10-06T05:36:10.882 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/data/mon.b.tgz 2023-10-06T05:36:10.882 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T05:36:10.920 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.LSznWoTA3w 2023-10-06T05:36:10.920 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LSznWoTA3w 2023-10-06T05:36:11.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.LSznWoTA3w 2023-10-06T05:36:11.240 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.LSznWoTA3w is 519KB 2023-10-06T05:36:11.332 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.LSznWoTA3w 2023-10-06T05:36:11.368 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/data/mon.c.tgz 2023-10-06T05:36:11.368 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T05:36:11.443 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.VEKWLQ7O7i 2023-10-06T05:36:11.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VEKWLQ7O7i 2023-10-06T05:36:11.648 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.VEKWLQ7O7i 2023-10-06T05:36:11.748 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.VEKWLQ7O7i is 519KB 2023-10-06T05:36:11.839 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.VEKWLQ7O7i 2023-10-06T05:36:11.875 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T05:36:11.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:36:11.880 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:36:11.976 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T05:36:11.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:36:11.981 DEBUG:teuthology.orchestra.run.smithi167:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:36:12.047 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:36:12.060 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:36:12.076 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T05:36:12.078 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/remote/smithi033/crash 2023-10-06T05:36:12.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:36:12.140 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/remote/smithi167/crash 2023-10-06T05:36:12.141 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:36:12.200 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T05:36:12.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:36:12.205 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:36:12.321 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:36:12.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:36:12.443 INFO:tasks.ceph:Compressing logs... 2023-10-06T05:36:12.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:36:12.447 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:36:20.374 INFO:tasks.ceph:Archiving logs... 2023-10-06T05:36:20.374 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/remote/smithi033/log 2023-10-06T05:36:20.375 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:36:22.295 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/remote/smithi167/log 2023-10-06T05:36:22.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:36:23.181 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T05:36:23.191 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T05:36:23.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:36:23.196 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:36:23.311 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T05:36:23.321 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T05:36:23.321 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:36:23.326 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:36:23.358 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2023-10-06T05:36:23.360 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-10-06T05:36:23.363 INFO:teuthology.orchestra.run.smithi033.stdout:210 Number of sources = 4 2023-10-06T05:36:23.363 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:36:23.363 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2023-10-06T05:36:23.363 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +54us[ +83us] +/- 79ms 2023-10-06T05:36:23.363 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 7 377 34 -325us[ -295us] +/- 57ms 2023-10-06T05:36:23.364 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +324us[ +353us] +/- 58ms 2023-10-06T05:36:23.364 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:36:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:210 Number of sources = 4 2023-10-06T05:36:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:36:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-10-06T05:36:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +96us[ +96us] +/- 79ms 2023-10-06T05:36:23.366 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -285us[ -280us] +/- 57ms 2023-10-06T05:36:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +328us[ +328us] +/- 58ms 2023-10-06T05:36:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:36:23.367 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T05:36:23.377 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T05:36:23.378 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T05:36:23.411 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:36:23.535 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:36:23.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:36:23.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:36:23.770 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:36:23.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:36:23.838 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T05:36:23.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T05:36:23.878 INFO:teuthology.task.internal:Duration was 2209.023060 seconds 2023-10-06T05:36:23.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T05:36:23.928 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T05:36:23.928 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:36:23.933 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:36:24.041 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:36:24.044 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:36:24.358 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T05:36:24.358 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T05:36:24.358 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:36:24.410 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T05:36:24.410 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:36:24.462 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T05:36:24.463 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:36:24.467 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:36:24.556 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T05:36:24.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:36:24.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:36:24.718 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T05:36:24.727 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T05:36:24.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:36:24.790 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:36:24.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T05:36:24.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:36:24.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:36:24.924 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2023-10-06T05:36:24.927 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2023-10-06T05:36:24.962 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:36:24.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:36:24.997 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:36:25.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:36:25.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T05:36:25.042 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T05:36:25.043 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/remote/smithi033 2023-10-06T05:36:25.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:36:25.116 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318/remote/smithi167 2023-10-06T05:36:25.116 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:36:25.188 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T05:36:25.189 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:36:25.193 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:36:25.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T05:36:25.241 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T05:36:25.241 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T05:36:25.270 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T05:36:25.270 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:36:25.275 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:36:25.307 INFO:teuthology.orchestra.run.smithi033.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:36 /home/ubuntu/cephtest 2023-10-06T05:36:25.311 INFO:teuthology.orchestra.run.smithi167.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:36 /home/ubuntu/cephtest 2023-10-06T05:36:25.314 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T05:36:25.325 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T05:36:25.470 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T05:36:25.493 DEBUG:teuthology.nuke:shortname: smithi033 2023-10-06T05:36:25.493 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:36:25.515 DEBUG:teuthology.nuke:shortname: smithi167 2023-10-06T05:36:25.515 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:36:25.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:50:30.495883', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsN2KlXCfhDPmYLfoVfF8ye4qHC8UXV+jzCMG27elNHskyYRXcyMSrNycVha+feEaRJ54JvvraDvWOgcUF7W2I='} 2023-10-06T05:36:25.543 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412318', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:50:30.494548', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGh2SIXiMe51UgQvKWo7LPq52jMpsA7kU3YeiWFADY9lDVWX3mJJIxTnL9dtfZo12gEyWHIUJoC+Wr3mferqT6U='} 2023-10-06T05:36:25.573 INFO:teuthology.orchestra.console.smithi033:Power off 2023-10-06T05:36:25.574 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:36:25.592 INFO:teuthology.orchestra.console.smithi167:Power off 2023-10-06T05:36:25.592 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:36:25.602 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2023-10-06T05:36:25.602 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:36:25.614 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-10-06T05:36:25.614 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:36:25.626 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-10-06T05:36:25.635 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T05:36:29.627 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:36:29.636 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:36:29.738 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-10-06T05:36:29.745 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T05:36:33.742 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:36:33.752 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:36:33.855 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2023-10-06T05:36:33.863 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-10-06T05:36:33.956 INFO:teuthology.orchestra.console.smithi033:Power off completed 2023-10-06T05:36:33.964 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-10-06T05:36:34.069 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} duration: 2209.023059606552 failure_reason: '"1696569264.6939275 mon.a (mon.0) 140 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T05:36:34.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T05:36:34.150 INFO:teuthology.run:FAIL